2024-04-25T09:47:52.340 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T09:47:52.340 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035 --owner scheduled_neesingh@teuthology --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --verbose -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035/orig.config.yaml 2024-04-25T09:47:52.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:47:52.473 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673035' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBvOoOfT9zvd5ASkAErg5geX1soqO2/oh9Yql8zG4TL9vqi9JKIDPoR33AhWUxThmNpcv3hkkE9MRHBjciSIrpU= smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUtMffjk06lbdABLHgo03pOnX557+8j4SiyfoWJprl3cf2jFaCcZFh3fzVLMJyzxNUW0S25plwfekcykzmSF0k= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T09:47:52.473 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T09:47:52.474 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T09:47:52.474 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T09:47:52.475 INFO:teuthology.task.internal:Checking packages... 2024-04-25T09:47:52.499 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T09:47:52.499 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T09:47:52.499 INFO:teuthology.packaging:ref: None 2024-04-25T09:47:52.499 INFO:teuthology.packaging:tag: None 2024-04-25T09:47:52.499 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T09:47:52.499 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:47:52.499 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T09:47:52.650 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:47:52.651 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T09:47:52.659 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T09:47:52.659 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T09:47:52.677 INFO:teuthology.task.internal:Saving configuration 2024-04-25T09:47:52.694 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T09:47:52.700 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:47:52.724 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:41:49.783681', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFXvuiVTZPhsQpXnc1FJ+PICarYvzm2imQ1yDBAwBTqA'} 2024-04-25T09:47:52.744 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:41:49.785207', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM8Cawz/b/hJkarRrphXn17ndNzhzGA6pDmENMMnjOY8'} 2024-04-25T09:47:52.744 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T09:47:52.751 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T09:47:52.751 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T09:47:52.751 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T09:47:52.825 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f76720210d0>, signals=[15]) 2024-04-25T09:47:52.826 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T09:47:52.832 INFO:teuthology.task.internal:Opening connections... 2024-04-25T09:47:52.833 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T09:47:52.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:47:52.915 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T09:47:52.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:47:52.994 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T09:47:53.001 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-04-25T09:47:53.006 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-04-25T09:47:53.006 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-04-25T09:47:53.051 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T09:47:53.051 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Ubuntu" 2024-04-25T09:47:53.051 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="22.04" 2024-04-25T09:47:53.051 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T09:47:53.051 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_CODENAME=jammy 2024-04-25T09:47:53.051 INFO:teuthology.orchestra.run.smithi133.stdout:ID=ubuntu 2024-04-25T09:47:53.051 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE=debian 2024-04-25T09:47:53.051 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T09:47:53.051 INFO:teuthology.orchestra.run.smithi133.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T09:47:53.051 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T09:47:53.052 INFO:teuthology.orchestra.run.smithi133.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T09:47:53.052 INFO:teuthology.orchestra.run.smithi133.stdout:UBUNTU_CODENAME=jammy 2024-04-25T09:47:53.052 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-04-25T09:47:53.080 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2024-04-25T09:47:53.085 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2024-04-25T09:47:53.086 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-04-25T09:47:53.133 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T09:47:53.133 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Ubuntu" 2024-04-25T09:47:53.133 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="22.04" 2024-04-25T09:47:53.133 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T09:47:53.134 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_CODENAME=jammy 2024-04-25T09:47:53.134 INFO:teuthology.orchestra.run.smithi162.stdout:ID=ubuntu 2024-04-25T09:47:53.134 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE=debian 2024-04-25T09:47:53.134 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T09:47:53.134 INFO:teuthology.orchestra.run.smithi162.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T09:47:53.134 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T09:47:53.134 INFO:teuthology.orchestra.run.smithi162.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T09:47:53.134 INFO:teuthology.orchestra.run.smithi162.stdout:UBUNTU_CODENAME=jammy 2024-04-25T09:47:53.134 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-04-25T09:47:53.163 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T09:47:53.179 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T09:47:53.216 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T09:47:53.216 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:47:53.219 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:47:53.226 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T09:47:53.232 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T09:47:53.232 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:47:53.265 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:47:53.279 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T09:47:53.337 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T09:47:53.352 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T09:47:53.352 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T09:47:53.352 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:47:53.353 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:47:53.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:47:53.357 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-04-25T09:47:53.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:47:53.359 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-04-25T09:47:53.403 INFO:teuthology.orchestra.run.smithi133.stdout:5.15.0-56-generic 2024-04-25T09:47:53.404 INFO:teuthology.task.kernel:Running kernel on smithi133: 5.15.0-56-generic 2024-04-25T09:47:53.404 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get clean 2024-04-25T09:47:53.405 INFO:teuthology.orchestra.run.smithi162.stdout:5.15.0-56-generic 2024-04-25T09:47:53.405 INFO:teuthology.task.kernel:Running kernel on smithi162: 5.15.0-56-generic 2024-04-25T09:47:53.405 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-get clean 2024-04-25T09:47:53.531 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-get update 2024-04-25T09:47:53.538 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2024-04-25T09:47:53.673 INFO:teuthology.orchestra.run.smithi162.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T09:47:53.707 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:47:53.707 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T09:47:53.707 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T09:47:53.780 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:47:53.781 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T09:47:53.863 INFO:teuthology.orchestra.run.smithi162.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T09:47:53.900 INFO:teuthology.orchestra.run.smithi162.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T09:47:53.943 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T09:47:53.982 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T09:47:54.018 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T09:47:54.025 INFO:teuthology.orchestra.run.smithi162.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T09:47:54.027 INFO:teuthology.orchestra.run.smithi162.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T09:47:54.027 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T09:47:54.028 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T09:47:54.028 INFO:teuthology.orchestra.run.smithi162.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T09:47:54.059 INFO:teuthology.orchestra.run.smithi162.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T09:47:54.063 INFO:teuthology.orchestra.run.smithi162.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:47:54.064 INFO:teuthology.orchestra.run.smithi162.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T09:47:54.075 INFO:teuthology.orchestra.run.smithi162.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T09:47:54.083 INFO:teuthology.orchestra.run.smithi162.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T09:47:54.086 INFO:teuthology.orchestra.run.smithi162.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T09:47:54.096 INFO:teuthology.orchestra.run.smithi162.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T09:47:54.097 INFO:teuthology.orchestra.run.smithi162.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T09:47:54.101 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T09:47:54.104 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T09:47:54.105 INFO:teuthology.orchestra.run.smithi133.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T09:47:54.105 INFO:teuthology.orchestra.run.smithi133.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T09:47:54.123 INFO:teuthology.orchestra.run.smithi162.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T09:47:54.131 INFO:teuthology.orchestra.run.smithi133.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T09:47:54.131 INFO:teuthology.orchestra.run.smithi162.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T09:47:54.154 INFO:teuthology.orchestra.run.smithi133.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T09:47:54.159 INFO:teuthology.orchestra.run.smithi133.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:47:54.160 INFO:teuthology.orchestra.run.smithi133.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T09:47:54.175 INFO:teuthology.orchestra.run.smithi133.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T09:47:54.214 INFO:teuthology.orchestra.run.smithi133.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T09:47:54.214 INFO:teuthology.orchestra.run.smithi133.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T09:47:54.215 INFO:teuthology.orchestra.run.smithi133.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T09:47:54.216 INFO:teuthology.orchestra.run.smithi133.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T09:47:54.216 INFO:teuthology.orchestra.run.smithi133.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T09:47:54.217 INFO:teuthology.orchestra.run.smithi133.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T09:47:54.217 INFO:teuthology.orchestra.run.smithi133.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T09:47:54.224 INFO:teuthology.orchestra.run.smithi133.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T09:47:54.225 INFO:teuthology.orchestra.run.smithi133.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T09:47:54.229 INFO:teuthology.orchestra.run.smithi162.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T09:47:54.276 INFO:teuthology.orchestra.run.smithi133.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T09:47:54.277 INFO:teuthology.orchestra.run.smithi133.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T09:47:54.283 INFO:teuthology.orchestra.run.smithi133.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:47:54.283 INFO:teuthology.orchestra.run.smithi133.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T09:47:54.312 INFO:teuthology.orchestra.run.smithi133.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T09:47:54.328 INFO:teuthology.orchestra.run.smithi133.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T09:47:54.332 INFO:teuthology.orchestra.run.smithi133.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T09:47:54.332 INFO:teuthology.orchestra.run.smithi133.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T09:47:54.333 INFO:teuthology.orchestra.run.smithi133.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T09:47:54.335 INFO:teuthology.orchestra.run.smithi133.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T09:47:54.336 INFO:teuthology.orchestra.run.smithi133.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T09:47:54.336 INFO:teuthology.orchestra.run.smithi133.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T09:47:54.337 INFO:teuthology.orchestra.run.smithi133.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T09:47:54.354 INFO:teuthology.orchestra.run.smithi133.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T09:47:54.355 INFO:teuthology.orchestra.run.smithi133.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T09:47:54.355 INFO:teuthology.orchestra.run.smithi133.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T09:47:54.355 INFO:teuthology.orchestra.run.smithi133.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T09:47:54.356 INFO:teuthology.orchestra.run.smithi133.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T09:47:54.357 INFO:teuthology.orchestra.run.smithi133.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T09:47:54.387 INFO:teuthology.orchestra.run.smithi162.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T09:47:54.622 INFO:teuthology.orchestra.run.smithi162.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T09:47:54.859 INFO:teuthology.orchestra.run.smithi162.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T09:47:54.862 INFO:teuthology.orchestra.run.smithi162.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T09:47:54.862 INFO:teuthology.orchestra.run.smithi162.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T09:47:54.863 INFO:teuthology.orchestra.run.smithi162.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T09:47:54.881 INFO:teuthology.orchestra.run.smithi162.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T09:47:54.891 INFO:teuthology.orchestra.run.smithi162.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:47:55.055 INFO:teuthology.orchestra.run.smithi162.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T09:47:55.753 INFO:teuthology.orchestra.run.smithi162.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T09:47:55.834 INFO:teuthology.orchestra.run.smithi162.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T09:47:55.855 INFO:teuthology.orchestra.run.smithi162.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T09:47:55.858 INFO:teuthology.orchestra.run.smithi162.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T09:47:55.858 INFO:teuthology.orchestra.run.smithi162.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T09:47:55.859 INFO:teuthology.orchestra.run.smithi162.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T09:47:55.860 INFO:teuthology.orchestra.run.smithi162.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T09:47:55.860 INFO:teuthology.orchestra.run.smithi162.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T09:47:55.865 INFO:teuthology.orchestra.run.smithi162.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T09:47:55.871 INFO:teuthology.orchestra.run.smithi162.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T09:47:56.040 INFO:teuthology.orchestra.run.smithi162.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T09:47:56.125 INFO:teuthology.orchestra.run.smithi162.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T09:47:56.257 INFO:teuthology.orchestra.run.smithi162.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T09:47:56.297 INFO:teuthology.orchestra.run.smithi162.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T09:47:56.330 INFO:teuthology.orchestra.run.smithi162.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T09:47:59.881 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 13.3 MB in 2s (7,052 kB/s) 2024-04-25T09:48:00.501 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 13.3 MB in 3s (4,861 kB/s) 2024-04-25T09:48:01.170 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-04-25T09:48:01.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:48:01.249 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-04-25T09:48:01.448 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2024-04-25T09:48:01.448 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2024-04-25T09:48:01.624 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:48:01.624 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:48:01.624 INFO:teuthology.orchestra.run.smithi133.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:48:01.625 INFO:teuthology.orchestra.run.smithi133.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:48:01.625 INFO:teuthology.orchestra.run.smithi133.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T09:48:01.625 INFO:teuthology.orchestra.run.smithi133.stdout: python2.7-minimal 2024-04-25T09:48:01.625 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:48:01.626 INFO:teuthology.orchestra.run.smithi133.stdout:The following additional packages will be installed: 2024-04-25T09:48:01.626 INFO:teuthology.orchestra.run.smithi133.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:48:01.626 INFO:teuthology.orchestra.run.smithi133.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T09:48:01.626 INFO:teuthology.orchestra.run.smithi133.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T09:48:01.628 INFO:teuthology.orchestra.run.smithi133.stdout:Suggested packages: 2024-04-25T09:48:01.628 INFO:teuthology.orchestra.run.smithi133.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T09:48:01.628 INFO:teuthology.orchestra.run.smithi133.stdout:Recommended packages: 2024-04-25T09:48:01.628 INFO:teuthology.orchestra.run.smithi133.stdout: thermald 2024-04-25T09:48:01.681 INFO:teuthology.orchestra.run.smithi133.stdout:The following NEW packages will be installed: 2024-04-25T09:48:01.681 INFO:teuthology.orchestra.run.smithi133.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:48:01.681 INFO:teuthology.orchestra.run.smithi133.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T09:48:01.681 INFO:teuthology.orchestra.run.smithi133.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T09:48:01.682 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages will be upgraded: 2024-04-25T09:48:01.683 INFO:teuthology.orchestra.run.smithi133.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T09:48:01.864 INFO:teuthology.orchestra.run.smithi133.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:48:01.864 INFO:teuthology.orchestra.run.smithi133.stdout:Need to get 113 MB of archives. 2024-04-25T09:48:01.864 INFO:teuthology.orchestra.run.smithi133.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T09:48:01.864 INFO:teuthology.orchestra.run.smithi133.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T09:48:01.887 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-04-25T09:48:01.904 DEBUG:teuthology.orchestra.run.smithi162:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:48:01.963 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-04-25T09:48:02.165 INFO:teuthology.orchestra.run.smithi162.stdout:Building dependency tree... 2024-04-25T09:48:02.165 INFO:teuthology.orchestra.run.smithi162.stdout:Reading state information... 2024-04-25T09:48:02.343 INFO:teuthology.orchestra.run.smithi162.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:48:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:48:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:48:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:48:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T09:48:02.343 INFO:teuthology.orchestra.run.smithi162.stdout: python2.7-minimal 2024-04-25T09:48:02.343 INFO:teuthology.orchestra.run.smithi162.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:48:02.344 INFO:teuthology.orchestra.run.smithi162.stdout:The following additional packages will be installed: 2024-04-25T09:48:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:48:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T09:48:02.344 INFO:teuthology.orchestra.run.smithi162.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T09:48:02.345 INFO:teuthology.orchestra.run.smithi162.stdout:Suggested packages: 2024-04-25T09:48:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T09:48:02.345 INFO:teuthology.orchestra.run.smithi162.stdout:Recommended packages: 2024-04-25T09:48:02.345 INFO:teuthology.orchestra.run.smithi162.stdout: thermald 2024-04-25T09:48:02.394 INFO:teuthology.orchestra.run.smithi162.stdout:The following NEW packages will be installed: 2024-04-25T09:48:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:48:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T09:48:02.394 INFO:teuthology.orchestra.run.smithi162.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T09:48:02.396 INFO:teuthology.orchestra.run.smithi162.stdout:The following packages will be upgraded: 2024-04-25T09:48:02.396 INFO:teuthology.orchestra.run.smithi162.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T09:48:02.485 INFO:teuthology.orchestra.run.smithi162.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:48:02.485 INFO:teuthology.orchestra.run.smithi162.stdout:Need to get 113 MB of archives. 2024-04-25T09:48:02.485 INFO:teuthology.orchestra.run.smithi162.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T09:48:02.485 INFO:teuthology.orchestra.run.smithi162.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T09:48:03.252 INFO:teuthology.orchestra.run.smithi162.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T09:48:03.272 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T09:48:03.500 INFO:teuthology.orchestra.run.smithi162.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T09:48:03.625 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T09:48:04.878 INFO:teuthology.orchestra.run.smithi162.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T09:48:04.879 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T09:48:04.879 INFO:teuthology.orchestra.run.smithi162.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T09:48:05.036 INFO:teuthology.orchestra.run.smithi162.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T09:48:05.070 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T09:48:05.461 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 113 MB in 3s (42.5 MB/s) 2024-04-25T09:48:05.655 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T09:48:05.849 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T09:48:05.849 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T09:48:05.850 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T09:48:06.721 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T09:48:06.781 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T09:48:07.210 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 113 MB in 5s (22.3 MB/s) 2024-04-25T09:48:07.404 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T09:48:07.610 INFO:teuthology.orchestra.run.smithi162.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T09:48:07.614 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:07.710 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:09.410 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T09:48:09.414 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:09.500 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:09.894 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T09:48:09.915 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:10.005 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:10.480 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T09:48:10.501 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:10.538 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:11.884 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T09:48:11.905 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:11.984 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:12.479 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T09:48:12.500 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:12.546 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:17.812 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:48:17.940 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:48:18.281 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:48:18.410 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:48:18.721 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T09:48:18.743 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T09:48:18.787 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:48:19.009 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:48:19.111 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:48:19.438 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:48:19.557 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:48:19.859 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T09:48:19.882 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T09:48:19.943 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:48:24.971 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T09:48:25.010 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:25.054 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:25.889 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T09:48:25.926 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:25.964 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:27.700 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:48:27.816 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:48:28.166 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:48:28.313 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:28.439 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T09:48:28.567 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:28.569 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:48:28.695 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:48:29.066 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:48:29.256 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:29.882 INFO:teuthology.orchestra.run.smithi162.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:48:29.882 INFO:teuthology.orchestra.run.smithi162.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T09:48:29.882 INFO:teuthology.orchestra.run.smithi162.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:48:29.882 INFO:teuthology.orchestra.run.smithi162.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T09:48:30.048 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T09:48:30.095 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:30.151 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:31.466 INFO:teuthology.orchestra.run.smithi133.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:48:31.466 INFO:teuthology.orchestra.run.smithi133.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T09:48:31.466 INFO:teuthology.orchestra.run.smithi133.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:48:31.466 INFO:teuthology.orchestra.run.smithi133.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T09:48:31.538 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:31.645 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:32.955 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T09:48:33.056 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:33.081 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T09:48:33.207 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:33.283 INFO:teuthology.orchestra.run.smithi162.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T09:48:33.283 INFO:teuthology.orchestra.run.smithi162.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:48:34.414 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T09:48:34.540 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T09:48:34.666 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:34.731 INFO:teuthology.orchestra.run.smithi133.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T09:48:34.731 INFO:teuthology.orchestra.run.smithi133.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:48:47.323 INFO:teuthology.orchestra.run.smithi162.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T09:48:47.323 INFO:teuthology.orchestra.run.smithi162.stdout:Sourcing file `/etc/default/grub' 2024-04-25T09:48:47.347 INFO:teuthology.orchestra.run.smithi162.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:48:47.367 INFO:teuthology.orchestra.run.smithi162.stdout:Generating grub configuration file ... 2024-04-25T09:48:47.605 INFO:teuthology.orchestra.run.smithi162.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:48:47.635 INFO:teuthology.orchestra.run.smithi162.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:48:47.930 INFO:teuthology.orchestra.run.smithi162.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:48:47.935 INFO:teuthology.orchestra.run.smithi162.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:48:48.008 INFO:teuthology.orchestra.run.smithi162.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:48:48.014 INFO:teuthology.orchestra.run.smithi162.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:48:48.152 INFO:teuthology.orchestra.run.smithi162.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:48:48.152 INFO:teuthology.orchestra.run.smithi162.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:48:48.152 INFO:teuthology.orchestra.run.smithi162.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:48:48.204 INFO:teuthology.orchestra.run.smithi162.stdout:done 2024-04-25T09:48:48.464 INFO:teuthology.orchestra.run.smithi133.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T09:48:48.465 INFO:teuthology.orchestra.run.smithi133.stdout:Sourcing file `/etc/default/grub' 2024-04-25T09:48:48.470 INFO:teuthology.orchestra.run.smithi133.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:48:48.535 INFO:teuthology.orchestra.run.smithi133.stdout:Generating grub configuration file ... 2024-04-25T09:48:48.697 DEBUG:teuthology.orchestra.run.smithi162:> dpkg -s linux-image-generic 2024-04-25T09:48:48.722 INFO:teuthology.orchestra.run.smithi162.stdout:Package: linux-image-generic 2024-04-25T09:48:48.722 INFO:teuthology.orchestra.run.smithi162.stdout:Status: install ok installed 2024-04-25T09:48:48.722 INFO:teuthology.orchestra.run.smithi162.stdout:Priority: optional 2024-04-25T09:48:48.722 INFO:teuthology.orchestra.run.smithi162.stdout:Section: kernel 2024-04-25T09:48:48.723 INFO:teuthology.orchestra.run.smithi162.stdout:Installed-Size: 21 2024-04-25T09:48:48.723 INFO:teuthology.orchestra.run.smithi162.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:48:48.723 INFO:teuthology.orchestra.run.smithi162.stdout:Architecture: amd64 2024-04-25T09:48:48.723 INFO:teuthology.orchestra.run.smithi162.stdout:Source: linux-meta 2024-04-25T09:48:48.723 INFO:teuthology.orchestra.run.smithi162.stdout:Version: 5.15.0.105.102 2024-04-25T09:48:48.723 INFO:teuthology.orchestra.run.smithi162.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:48:48.723 INFO:teuthology.orchestra.run.smithi162.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:48:48.723 INFO:teuthology.orchestra.run.smithi162.stdout:Recommends: thermald 2024-04-25T09:48:48.723 INFO:teuthology.orchestra.run.smithi162.stdout:Description: Generic Linux kernel image 2024-04-25T09:48:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:48:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: available. 2024-04-25T09:48:48.724 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T09:48:48.724 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T09:48:48.724 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T09:48:48.724 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T09:48:48.724 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi162.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T09:48:48.724 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-get clean 2024-04-25T09:48:48.850 INFO:teuthology.orchestra.run.smithi133.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:48:48.871 INFO:teuthology.orchestra.run.smithi133.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:48:48.906 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-get update 2024-04-25T09:48:49.033 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:48:49.033 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T09:48:49.053 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T09:48:49.143 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T09:48:49.265 INFO:teuthology.orchestra.run.smithi133.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:48:49.271 INFO:teuthology.orchestra.run.smithi133.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:48:49.338 INFO:teuthology.orchestra.run.smithi133.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:48:49.344 INFO:teuthology.orchestra.run.smithi133.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:48:49.494 INFO:teuthology.orchestra.run.smithi133.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:48:49.495 INFO:teuthology.orchestra.run.smithi133.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:48:49.495 INFO:teuthology.orchestra.run.smithi133.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:48:49.537 INFO:teuthology.orchestra.run.smithi133.stdout:done 2024-04-25T09:48:50.040 DEBUG:teuthology.orchestra.run.smithi133:> dpkg -s linux-image-generic 2024-04-25T09:48:50.063 INFO:teuthology.orchestra.run.smithi133.stdout:Package: linux-image-generic 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Status: install ok installed 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Priority: optional 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Section: kernel 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Installed-Size: 21 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: amd64 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Source: linux-meta 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Version: 5.15.0.105.102 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Recommends: thermald 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout:Description: Generic Linux kernel image 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:48:50.064 INFO:teuthology.orchestra.run.smithi133.stdout: available. 2024-04-25T09:48:50.065 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T09:48:50.065 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T09:48:50.065 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T09:48:50.065 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T09:48:50.065 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T09:48:50.065 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get clean 2024-04-25T09:48:50.234 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2024-04-25T09:48:50.357 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T09:48:50.425 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-04-25T09:48:50.443 DEBUG:teuthology.orchestra.run.smithi162:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:48:50.476 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:48:50.503 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-04-25T09:48:50.561 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T09:48:50.663 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T09:48:50.713 INFO:teuthology.orchestra.run.smithi162.stdout:Building dependency tree... 2024-04-25T09:48:50.713 INFO:teuthology.orchestra.run.smithi162.stdout:Reading state information... 2024-04-25T09:48:50.892 INFO:teuthology.orchestra.run.smithi162.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T09:48:50.892 INFO:teuthology.orchestra.run.smithi162.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:48:50.893 INFO:teuthology.orchestra.run.smithi162.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:48:50.893 INFO:teuthology.orchestra.run.smithi162.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:48:50.893 INFO:teuthology.orchestra.run.smithi162.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:48:50.893 INFO:teuthology.orchestra.run.smithi162.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T09:48:50.894 INFO:teuthology.orchestra.run.smithi162.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T09:48:50.894 INFO:teuthology.orchestra.run.smithi162.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:48:50.935 INFO:teuthology.orchestra.run.smithi162.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:48:50.937 DEBUG:teuthology.orchestra.run.smithi162:> dpkg -s linux-image-generic 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Package: linux-image-generic 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Status: install ok installed 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Priority: optional 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Section: kernel 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Installed-Size: 21 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Architecture: amd64 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Source: linux-meta 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Version: 5.15.0.105.102 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:48:50.953 INFO:teuthology.orchestra.run.smithi162.stdout:Recommends: thermald 2024-04-25T09:48:50.954 INFO:teuthology.orchestra.run.smithi162.stdout:Description: Generic Linux kernel image 2024-04-25T09:48:50.954 INFO:teuthology.orchestra.run.smithi162.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:48:50.954 INFO:teuthology.orchestra.run.smithi162.stdout: available. 2024-04-25T09:48:50.954 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-04-25T09:48:51.001 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.CPnwTfcyZn 2024-04-25T09:48:51.001 DEBUG:teuthology.orchestra.run.smithi162:> sudo cp /boot/grub/grub.cfg /tmp/tmp.CPnwTfcyZn 2024-04-25T09:48:51.057 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.CPnwTfcyZn 2024-04-25T09:48:51.172 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.CPnwTfcyZn is 10KB 2024-04-25T09:48:51.187 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.CPnwTfcyZn 2024-04-25T09:48:51.193 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T09:48:51.249 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T09:48:51.249 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T09:48:51.249 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T09:48:51.318 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T09:48:51.318 DEBUG:teuthology.orchestra.run.smithi162:> sudo update-grub 2024-04-25T09:48:51.825 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-04-25T09:48:51.842 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:48:51.896 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-04-25T09:48:52.086 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2024-04-25T09:48:52.086 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2024-04-25T09:48:52.246 INFO:teuthology.orchestra.run.smithi133.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T09:48:52.247 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:48:52.247 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:48:52.247 INFO:teuthology.orchestra.run.smithi133.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:48:52.247 INFO:teuthology.orchestra.run.smithi133.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:48:52.247 INFO:teuthology.orchestra.run.smithi133.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T09:48:52.247 INFO:teuthology.orchestra.run.smithi133.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T09:48:52.247 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:48:52.288 INFO:teuthology.orchestra.run.smithi133.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:48:52.290 DEBUG:teuthology.orchestra.run.smithi133:> dpkg -s linux-image-generic 2024-04-25T09:48:52.345 INFO:teuthology.orchestra.run.smithi133.stdout:Package: linux-image-generic 2024-04-25T09:48:52.345 INFO:teuthology.orchestra.run.smithi133.stdout:Status: install ok installed 2024-04-25T09:48:52.345 INFO:teuthology.orchestra.run.smithi133.stdout:Priority: optional 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout:Section: kernel 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout:Installed-Size: 21 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: amd64 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout:Source: linux-meta 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout:Version: 5.15.0.105.102 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout:Recommends: thermald 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout:Description: Generic Linux kernel image 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:48:52.346 INFO:teuthology.orchestra.run.smithi133.stdout: available. 2024-04-25T09:48:52.347 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-04-25T09:48:52.391 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.0N33ILK1hk 2024-04-25T09:48:52.391 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub/grub.cfg /tmp/tmp.0N33ILK1hk 2024-04-25T09:48:52.447 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.0N33ILK1hk 2024-04-25T09:48:52.568 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.0N33ILK1hk is 10KB 2024-04-25T09:48:52.580 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.0N33ILK1hk 2024-04-25T09:48:52.585 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T09:48:52.621 INFO:teuthology.orchestra.run.smithi162.stderr:Sourcing file `/etc/default/grub' 2024-04-25T09:48:52.622 INFO:teuthology.orchestra.run.smithi162.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:48:52.625 INFO:teuthology.orchestra.run.smithi162.stderr:Generating grub configuration file ... 2024-04-25T09:48:52.639 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T09:48:52.639 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T09:48:52.639 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T09:48:52.707 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T09:48:52.707 DEBUG:teuthology.orchestra.run.smithi133:> sudo update-grub 2024-04-25T09:48:52.818 INFO:teuthology.orchestra.run.smithi162.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:48:52.824 INFO:teuthology.orchestra.run.smithi162.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:48:53.093 INFO:teuthology.orchestra.run.smithi162.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:48:53.098 INFO:teuthology.orchestra.run.smithi162.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:48:53.139 INFO:teuthology.orchestra.run.smithi162.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:48:53.144 INFO:teuthology.orchestra.run.smithi162.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:48:53.257 INFO:teuthology.orchestra.run.smithi162.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:48:53.257 INFO:teuthology.orchestra.run.smithi162.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:48:53.258 INFO:teuthology.orchestra.run.smithi162.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:48:53.280 INFO:teuthology.orchestra.run.smithi162.stderr:done 2024-04-25T09:48:53.289 DEBUG:teuthology.orchestra.run.smithi162:> sudo shutdown -r now 2024-04-25T09:48:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub' 2024-04-25T09:48:53.965 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:48:53.969 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2024-04-25T09:48:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:48:54.219 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:48:54.574 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:48:54.580 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:48:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:48:54.624 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:48:54.745 INFO:teuthology.orchestra.run.smithi133.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:48:54.745 INFO:teuthology.orchestra.run.smithi133.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:48:54.745 INFO:teuthology.orchestra.run.smithi133.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:48:54.757 INFO:teuthology.orchestra.run.smithi133.stderr:done 2024-04-25T09:48:54.759 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2024-04-25T09:49:23.318 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T09:49:23.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-04-25T09:49:23.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:49:24.762 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T09:49:24.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-25T09:49:24.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:49:57.919 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-04-25T09:50:06.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-04-25T09:50:06.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:50:24.783 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T09:50:25.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-04-25T09:50:33.791 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-25T09:50:33.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:50:33.794 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-04-25T09:50:37.314 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-04-25T09:50:37.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:50:40.386 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-04-25T09:50:45.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-25T09:50:45.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:50:46.042 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-04-25T09:50:46.642 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-25T09:50:46.642 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T09:50:46.642 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-04-25T09:50:46.690 INFO:teuthology.orchestra.run.smithi133.stdout:5.15.0-105-generic 2024-04-25T09:50:46.690 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T09:50:46.690 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:50:46.690 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T09:50:47.691 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T09:50:47.691 DEBUG:teuthology.orchestra.run.smithi133:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T09:50:48.051 INFO:teuthology.orchestra.run.smithi133.stdout:ttyS1 2024-04-25T09:50:48.075 DEBUG:teuthology.parallel:result is None 2024-04-25T09:50:55.387 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-04-25T09:50:55.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:50:55.696 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-04-25T09:50:56.278 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-04-25T09:50:56.278 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T09:50:56.278 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-04-25T09:50:56.325 INFO:teuthology.orchestra.run.smithi162.stdout:5.15.0-105-generic 2024-04-25T09:50:56.325 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T09:50:56.325 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:50:56.325 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T09:50:57.326 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T09:50:57.326 DEBUG:teuthology.orchestra.run.smithi162:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T09:50:57.676 INFO:teuthology.orchestra.run.smithi162.stdout:ttyS1 2024-04-25T09:50:57.707 DEBUG:teuthology.parallel:result is None 2024-04-25T09:50:57.707 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T09:50:57.715 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T09:50:57.715 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:50:57.718 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:50:57.723 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T09:50:57.759 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T09:50:57.865 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T09:50:57.865 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:50:57.867 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:50:57.888 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T09:50:57.895 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T09:50:57.895 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:50:57.927 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:50:57.945 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:50:57.950 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:50:57.953 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:50:57.958 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:50:57.960 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T09:50:57.967 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T09:50:57.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:50:57.999 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:50:58.021 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T09:50:58.032 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T09:50:58.032 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:50:58.055 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:50:58.069 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:50:58.106 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:50:58.154 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T09:50:58.154 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:50:58.224 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:50:58.229 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:50:58.277 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T09:50:58.277 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:50:58.355 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-04-25T09:50:58.357 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2024-04-25T09:50:58.428 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T09:50:58.436 INFO:teuthology.task.internal:Starting timer... 2024-04-25T09:50:58.436 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T09:50:58.446 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T09:50:58.454 DEBUG:teuthology.task.selinux:Excluding smithi133: OS 'ubuntu' does not support SELinux 2024-04-25T09:50:58.454 DEBUG:teuthology.task.selinux:Excluding smithi162: OS 'ubuntu' does not support SELinux 2024-04-25T09:50:58.454 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T09:50:58.454 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T09:50:58.454 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T09:50:58.454 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T09:50:58.463 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T09:50:58.482 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T09:50:58.483 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi133.front.sepia.ceph.com,smithi162.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T09:58:00.276 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi133.front.sepia.ceph.com'), Remote(name='ubuntu@smithi162.front.sepia.ceph.com')] 2024-04-25T09:58:00.278 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-25T09:58:00.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:58:00.364 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-04-25T09:58:00.443 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-25T09:58:00.443 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-04-25T09:58:00.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:58:00.525 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-04-25T09:58:00.602 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-04-25T09:58:00.602 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T09:58:00.617 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T09:58:00.618 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:58:00.618 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:58:00.621 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:58:00.621 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:58:00.650 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T09:58:00.650 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: Command line: ntpd -gq 2024-04-25T09:58:00.650 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: ---------------------------------------------------- 2024-04-25T09:58:00.651 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T09:58:00.651 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T09:58:00.651 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: corporation. Support and training for ntp-4 are 2024-04-25T09:58:00.651 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: available at https://www.nwtime.org/support 2024-04-25T09:58:00.651 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: ---------------------------------------------------- 2024-04-25T09:58:00.651 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: proto: precision = 0.075 usec (-24) 2024-04-25T09:58:00.652 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: basedate set to 2022-02-04 2024-04-25T09:58:00.652 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: gps base set to 2022-02-06 (week 2196) 2024-04-25T09:58:00.652 INFO:teuthology.orchestra.run.smithi133.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T09:58:00.652 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T09:58:00.652 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T09:58:00.653 INFO:teuthology.orchestra.run.smithi133.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T09:58:00.654 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T09:58:00.654 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T09:58:00.654 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T09:58:00.654 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: Listen normally on 3 enp3s0f1 172.21.15.133:123 2024-04-25T09:58:00.654 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: Listen normally on 4 lo [::1]:123 2024-04-25T09:58:00.654 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8087%5]:123 2024-04-25T09:58:00.654 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:00 ntpd[17230]: Listening on routing socket on fd #22 for interface updates 2024-04-25T09:58:00.671 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T09:58:00.671 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: Command line: ntpd -gq 2024-04-25T09:58:00.671 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: ---------------------------------------------------- 2024-04-25T09:58:00.671 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T09:58:00.671 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T09:58:00.671 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: corporation. Support and training for ntp-4 are 2024-04-25T09:58:00.671 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: available at https://www.nwtime.org/support 2024-04-25T09:58:00.672 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: ---------------------------------------------------- 2024-04-25T09:58:00.672 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: proto: precision = 0.043 usec (-24) 2024-04-25T09:58:00.672 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: basedate set to 2022-02-04 2024-04-25T09:58:00.672 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: gps base set to 2022-02-06 (week 2196) 2024-04-25T09:58:00.672 INFO:teuthology.orchestra.run.smithi162.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T09:58:00.673 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T09:58:00.673 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T09:58:00.673 INFO:teuthology.orchestra.run.smithi162.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T09:58:00.674 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T09:58:00.674 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T09:58:00.674 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T09:58:00.674 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: Listen normally on 3 enp3s0f1 172.21.15.162:123 2024-04-25T09:58:00.674 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: Listen normally on 4 lo [::1]:123 2024-04-25T09:58:00.674 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f41%5]:123 2024-04-25T09:58:00.674 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:00 ntpd[17234]: Listening on routing socket on fd #22 for interface updates 2024-04-25T09:58:01.651 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:01 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:01.651 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:01 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:01.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:01 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:01.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:01 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:01.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:01 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:01.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:01 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:01.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:01 ntpd[17230]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:01.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:01 ntpd[17230]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:01.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:01 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:01.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:01 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:01.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:01 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:01.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:01 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:01.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:01 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:01.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:01 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:01.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:01 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:01.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:01 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:03.653 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:03 ntpd[17230]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:03.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:03 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:03.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:03 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:03.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:03 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:03.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:03 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:03.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:03 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:03.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:03 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:04 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:04 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:04 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:04.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:04 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:04.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:04 ntpd[17230]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:04.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:04 ntpd[17230]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:04.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:04 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:04.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:04 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:04.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:04 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:04.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:04 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:04.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:04 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:04.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:04 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:05.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:05.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:05.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:05.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:05.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:05.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:05.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:05.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:05.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:05.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:05.653 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:05.653 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:05 ntpd[17230]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:05.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:05.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:05.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:05.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:05.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:05.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:05.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:05.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:05.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:05.673 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:05.673 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:05.673 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:05 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:06.651 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:06 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:06.651 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:06 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:06.651 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:06 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:06.651 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:06 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:06.651 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:06 ntpd[17230]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:06.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:06 ntpd[17230]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:06.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:06 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:06.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:06 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:06.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:06 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:06.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:06 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:06.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:06 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:06.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:06 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:07.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:07.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:07.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:07.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:07.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:07.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:07.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:07.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:07.652 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:07.653 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:07.653 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:07.653 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:07.653 INFO:teuthology.orchestra.run.smithi133.stdout:25 Apr 09:58:07 ntpd[17230]: ntpd: time slew -0.002445 s 2024-04-25T09:58:07.653 INFO:teuthology.orchestra.run.smithi133.stdout:ntpd: time slew -0.002445s 2024-04-25T09:58:07.653 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T09:58:07.654 INFO:teuthology.orchestra.run.smithi133.stderr:25 Apr 09:58:07 ntpd[17230]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T09:58:07.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:07 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:07.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:07 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:07.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:07 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:07.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:07 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:07.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:07 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:07.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:07 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:07.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:07 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:07.672 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:07 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:07.672 INFO:teuthology.orchestra.run.smithi162.stdout:25 Apr 09:58:07 ntpd[17234]: ntpd: time slew -0.001543 s 2024-04-25T09:58:07.672 INFO:teuthology.orchestra.run.smithi162.stdout:ntpd: time slew -0.001543s 2024-04-25T09:58:07.673 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:07 ntpd[17234]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T09:58:07.673 INFO:teuthology.orchestra.run.smithi162.stderr:25 Apr 09:58:07 ntpd[17234]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T09:58:07.707 INFO:teuthology.orchestra.run.smithi133.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T09:58:07.707 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================== 2024-04-25T09:58:07.707 INFO:teuthology.orchestra.run.smithi133.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:07.707 INFO:teuthology.orchestra.run.smithi133.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:07.707 INFO:teuthology.orchestra.run.smithi133.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:07.707 INFO:teuthology.orchestra.run.smithi133.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T09:58:07.720 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================== 2024-04-25T09:58:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:07.721 INFO:teuthology.run_tasks:Running task install... 2024-04-25T09:58:07.730 DEBUG:teuthology.task.install:project ceph 2024-04-25T09:58:07.730 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T09:58:07.730 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T09:58:07.730 INFO:teuthology.task.install:Using flavor: default 2024-04-25T09:58:07.736 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T09:58:07.736 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T09:58:07.737 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-key list | grep Ceph 2024-04-25T09:58:07.737 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-key list | grep Ceph 2024-04-25T09:58:07.802 INFO:teuthology.orchestra.run.smithi133.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T09:58:07.820 INFO:teuthology.orchestra.run.smithi162.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T09:58:07.827 INFO:teuthology.orchestra.run.smithi133.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T09:58:07.828 INFO:teuthology.orchestra.run.smithi133.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T09:58:07.829 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T09:58:07.829 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T09:58:07.829 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:58:07.847 INFO:teuthology.orchestra.run.smithi162.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T09:58:07.847 INFO:teuthology.orchestra.run.smithi162.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T09:58:07.847 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T09:58:07.847 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T09:58:07.847 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:58:07.985 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T09:58:07.985 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:58:08.001 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T09:58:08.001 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:58:08.119 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T09:58:08.119 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T09:58:08.123 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T09:58:08.123 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T09:58:08.137 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2024-04-25T09:58:08.168 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-get update 2024-04-25T09:58:08.308 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:58:08.321 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T09:58:08.322 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T09:58:08.340 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:58:08.341 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T09:58:08.355 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T09:58:08.366 INFO:teuthology.orchestra.run.smithi133.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T09:58:08.376 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T09:58:08.390 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T09:58:08.408 INFO:teuthology.orchestra.run.smithi162.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T09:58:08.415 INFO:teuthology.orchestra.run.smithi133.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T09:58:08.434 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T09:58:08.438 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T09:58:08.455 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T09:58:08.460 INFO:teuthology.orchestra.run.smithi162.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T09:58:08.469 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T09:58:08.486 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T09:58:08.518 INFO:teuthology.orchestra.run.smithi162.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T09:58:08.679 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 28.9 kB in 0s (85.8 kB/s) 2024-04-25T09:58:08.689 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 28.9 kB in 0s (79.3 kB/s) 2024-04-25T09:58:09.620 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-04-25T09:58:09.636 DEBUG:teuthology.orchestra.run.smithi162:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:58:09.684 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-04-25T09:58:09.687 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-04-25T09:58:09.701 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:58:09.763 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-04-25T09:58:09.864 INFO:teuthology.orchestra.run.smithi162.stdout:Building dependency tree... 2024-04-25T09:58:09.864 INFO:teuthology.orchestra.run.smithi162.stdout:Reading state information... 2024-04-25T09:58:09.963 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2024-04-25T09:58:09.963 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2024-04-25T09:58:10.037 INFO:teuthology.orchestra.run.smithi162.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:58:10.037 INFO:teuthology.orchestra.run.smithi162.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:58:10.037 INFO:teuthology.orchestra.run.smithi162.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:58:10.037 INFO:teuthology.orchestra.run.smithi162.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:58:10.037 INFO:teuthology.orchestra.run.smithi162.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T09:58:10.038 INFO:teuthology.orchestra.run.smithi162.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T09:58:10.038 INFO:teuthology.orchestra.run.smithi162.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:58:10.039 INFO:teuthology.orchestra.run.smithi162.stdout:The following additional packages will be installed: 2024-04-25T09:58:10.039 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T09:58:10.039 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T09:58:10.039 INFO:teuthology.orchestra.run.smithi162.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T09:58:10.039 INFO:teuthology.orchestra.run.smithi162.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T09:58:10.040 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T09:58:10.040 INFO:teuthology.orchestra.run.smithi162.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T09:58:10.040 INFO:teuthology.orchestra.run.smithi162.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T09:58:10.040 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T09:58:10.040 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T09:58:10.040 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T09:58:10.040 INFO:teuthology.orchestra.run.smithi162.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T09:58:10.040 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T09:58:10.040 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T09:58:10.040 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T09:58:10.041 INFO:teuthology.orchestra.run.smithi162.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T09:58:10.041 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T09:58:10.041 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T09:58:10.041 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T09:58:10.041 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T09:58:10.041 INFO:teuthology.orchestra.run.smithi162.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T09:58:10.041 INFO:teuthology.orchestra.run.smithi162.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T09:58:10.041 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T09:58:10.041 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T09:58:10.041 INFO:teuthology.orchestra.run.smithi162.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T09:58:10.041 INFO:teuthology.orchestra.run.smithi162.stdout:Suggested packages: 2024-04-25T09:58:10.042 INFO:teuthology.orchestra.run.smithi162.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T09:58:10.042 INFO:teuthology.orchestra.run.smithi162.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T09:58:10.042 INFO:teuthology.orchestra.run.smithi162.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T09:58:10.042 INFO:teuthology.orchestra.run.smithi162.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T09:58:10.042 INFO:teuthology.orchestra.run.smithi162.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T09:58:10.042 INFO:teuthology.orchestra.run.smithi162.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T09:58:10.042 INFO:teuthology.orchestra.run.smithi162.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T09:58:10.042 INFO:teuthology.orchestra.run.smithi162.stdout:Recommended packages: 2024-04-25T09:58:10.042 INFO:teuthology.orchestra.run.smithi162.stdout: btrfs-tools 2024-04-25T09:58:10.146 INFO:teuthology.orchestra.run.smithi162.stdout:The following NEW packages will be installed: 2024-04-25T09:58:10.146 INFO:teuthology.orchestra.run.smithi162.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T09:58:10.146 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T09:58:10.146 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T09:58:10.146 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T09:58:10.146 INFO:teuthology.orchestra.run.smithi162.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T09:58:10.146 INFO:teuthology.orchestra.run.smithi162.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T09:58:10.147 INFO:teuthology.orchestra.run.smithi162.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T09:58:10.148 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T09:58:10.148 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T09:58:10.148 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T09:58:10.148 INFO:teuthology.orchestra.run.smithi162.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T09:58:10.148 INFO:teuthology.orchestra.run.smithi162.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T09:58:10.148 INFO:teuthology.orchestra.run.smithi162.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T09:58:10.148 INFO:teuthology.orchestra.run.smithi162.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T09:58:10.148 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T09:58:10.148 INFO:teuthology.orchestra.run.smithi162.stdout: socat unzip xmlstarlet zip 2024-04-25T09:58:10.159 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:58:10.159 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:58:10.159 INFO:teuthology.orchestra.run.smithi133.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:58:10.159 INFO:teuthology.orchestra.run.smithi133.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:58:10.160 INFO:teuthology.orchestra.run.smithi133.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T09:58:10.160 INFO:teuthology.orchestra.run.smithi133.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T09:58:10.160 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:58:10.161 INFO:teuthology.orchestra.run.smithi133.stdout:The following additional packages will be installed: 2024-04-25T09:58:10.161 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T09:58:10.161 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T09:58:10.161 INFO:teuthology.orchestra.run.smithi133.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T09:58:10.161 INFO:teuthology.orchestra.run.smithi133.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T09:58:10.164 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout:Suggested packages: 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T09:58:10.165 INFO:teuthology.orchestra.run.smithi133.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T09:58:10.166 INFO:teuthology.orchestra.run.smithi133.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T09:58:10.166 INFO:teuthology.orchestra.run.smithi133.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T09:58:10.166 INFO:teuthology.orchestra.run.smithi133.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T09:58:10.166 INFO:teuthology.orchestra.run.smithi133.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T09:58:10.166 INFO:teuthology.orchestra.run.smithi133.stdout:Recommended packages: 2024-04-25T09:58:10.166 INFO:teuthology.orchestra.run.smithi133.stdout: btrfs-tools 2024-04-25T09:58:10.269 INFO:teuthology.orchestra.run.smithi133.stdout:The following NEW packages will be installed: 2024-04-25T09:58:10.270 INFO:teuthology.orchestra.run.smithi133.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T09:58:10.270 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T09:58:10.270 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T09:58:10.270 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T09:58:10.271 INFO:teuthology.orchestra.run.smithi133.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T09:58:10.271 INFO:teuthology.orchestra.run.smithi133.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T09:58:10.271 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T09:58:10.271 INFO:teuthology.orchestra.run.smithi133.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T09:58:10.271 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T09:58:10.273 INFO:teuthology.orchestra.run.smithi133.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T09:58:10.273 INFO:teuthology.orchestra.run.smithi133.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T09:58:10.273 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T09:58:10.273 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T09:58:10.273 INFO:teuthology.orchestra.run.smithi133.stdout: socat unzip xmlstarlet zip 2024-04-25T09:58:10.287 INFO:teuthology.orchestra.run.smithi162.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T09:58:10.287 INFO:teuthology.orchestra.run.smithi162.stdout:Need to get 176 MB of archives. 2024-04-25T09:58:10.287 INFO:teuthology.orchestra.run.smithi162.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T09:58:10.287 INFO:teuthology.orchestra.run.smithi162.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T09:58:10.334 INFO:teuthology.orchestra.run.smithi162.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T09:58:10.336 INFO:teuthology.orchestra.run.smithi133.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T09:58:10.336 INFO:teuthology.orchestra.run.smithi133.stdout:Need to get 176 MB of archives. 2024-04-25T09:58:10.336 INFO:teuthology.orchestra.run.smithi133.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T09:58:10.336 INFO:teuthology.orchestra.run.smithi133.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T09:58:10.358 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T09:58:10.369 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T09:58:10.417 INFO:teuthology.orchestra.run.smithi162.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T09:58:10.429 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T09:58:10.441 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T09:58:10.455 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T09:58:10.458 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T09:58:10.472 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T09:58:10.472 INFO:teuthology.orchestra.run.smithi162.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T09:58:10.498 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T09:58:10.531 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T09:58:10.536 INFO:teuthology.orchestra.run.smithi133.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T09:58:10.548 INFO:teuthology.orchestra.run.smithi133.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T09:58:10.551 INFO:teuthology.orchestra.run.smithi133.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T09:58:10.552 INFO:teuthology.orchestra.run.smithi133.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T09:58:10.560 INFO:teuthology.orchestra.run.smithi133.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T09:58:10.562 INFO:teuthology.orchestra.run.smithi133.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T09:58:10.564 INFO:teuthology.orchestra.run.smithi133.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T09:58:10.565 INFO:teuthology.orchestra.run.smithi133.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T09:58:10.565 INFO:teuthology.orchestra.run.smithi133.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T09:58:10.577 INFO:teuthology.orchestra.run.smithi133.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T09:58:10.580 INFO:teuthology.orchestra.run.smithi133.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T09:58:10.581 INFO:teuthology.orchestra.run.smithi133.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T09:58:10.582 INFO:teuthology.orchestra.run.smithi133.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T09:58:10.601 INFO:teuthology.orchestra.run.smithi133.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T09:58:10.602 INFO:teuthology.orchestra.run.smithi133.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T09:58:10.602 INFO:teuthology.orchestra.run.smithi133.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T09:58:10.602 INFO:teuthology.orchestra.run.smithi133.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T09:58:10.603 INFO:teuthology.orchestra.run.smithi133.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T09:58:10.603 INFO:teuthology.orchestra.run.smithi133.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T09:58:10.604 INFO:teuthology.orchestra.run.smithi162.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T09:58:10.622 INFO:teuthology.orchestra.run.smithi133.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T09:58:10.622 INFO:teuthology.orchestra.run.smithi133.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T09:58:10.625 INFO:teuthology.orchestra.run.smithi133.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T09:58:10.625 INFO:teuthology.orchestra.run.smithi162.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T09:58:10.625 INFO:teuthology.orchestra.run.smithi133.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T09:58:10.641 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T09:58:10.641 INFO:teuthology.orchestra.run.smithi162.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T09:58:10.642 INFO:teuthology.orchestra.run.smithi133.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T09:58:10.642 INFO:teuthology.orchestra.run.smithi133.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T09:58:10.644 INFO:teuthology.orchestra.run.smithi133.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T09:58:10.644 INFO:teuthology.orchestra.run.smithi133.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T09:58:10.644 INFO:teuthology.orchestra.run.smithi133.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T09:58:10.645 INFO:teuthology.orchestra.run.smithi133.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T09:58:10.647 INFO:teuthology.orchestra.run.smithi162.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T09:58:10.649 INFO:teuthology.orchestra.run.smithi162.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T09:58:10.660 INFO:teuthology.orchestra.run.smithi162.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T09:58:10.664 INFO:teuthology.orchestra.run.smithi133.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T09:58:10.664 INFO:teuthology.orchestra.run.smithi133.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T09:58:10.670 INFO:teuthology.orchestra.run.smithi133.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T09:58:10.670 INFO:teuthology.orchestra.run.smithi133.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T09:58:10.676 INFO:teuthology.orchestra.run.smithi162.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T09:58:10.685 INFO:teuthology.orchestra.run.smithi133.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T09:58:10.686 INFO:teuthology.orchestra.run.smithi133.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T09:58:10.687 INFO:teuthology.orchestra.run.smithi133.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T09:58:10.689 INFO:teuthology.orchestra.run.smithi133.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T09:58:10.690 INFO:teuthology.orchestra.run.smithi133.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T09:58:10.691 INFO:teuthology.orchestra.run.smithi133.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T09:58:10.705 INFO:teuthology.orchestra.run.smithi133.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T09:58:10.708 INFO:teuthology.orchestra.run.smithi133.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T09:58:10.708 INFO:teuthology.orchestra.run.smithi133.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T09:58:10.709 INFO:teuthology.orchestra.run.smithi133.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T09:58:10.726 INFO:teuthology.orchestra.run.smithi133.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T09:58:10.784 INFO:teuthology.orchestra.run.smithi133.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T09:58:10.785 INFO:teuthology.orchestra.run.smithi133.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T09:58:10.787 INFO:teuthology.orchestra.run.smithi133.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T09:58:10.787 INFO:teuthology.orchestra.run.smithi133.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T09:58:10.788 INFO:teuthology.orchestra.run.smithi133.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T09:58:10.809 INFO:teuthology.orchestra.run.smithi133.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T09:58:10.811 INFO:teuthology.orchestra.run.smithi133.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T09:58:10.811 INFO:teuthology.orchestra.run.smithi133.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T09:58:10.834 INFO:teuthology.orchestra.run.smithi133.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T09:58:10.834 INFO:teuthology.orchestra.run.smithi133.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T09:58:10.843 INFO:teuthology.orchestra.run.smithi133.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T09:58:10.843 INFO:teuthology.orchestra.run.smithi133.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T09:58:10.844 INFO:teuthology.orchestra.run.smithi133.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T09:58:10.845 INFO:teuthology.orchestra.run.smithi133.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T09:58:10.845 INFO:teuthology.orchestra.run.smithi133.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T09:58:10.845 INFO:teuthology.orchestra.run.smithi133.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T09:58:10.849 INFO:teuthology.orchestra.run.smithi133.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T09:58:10.854 INFO:teuthology.orchestra.run.smithi162.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T09:58:10.864 INFO:teuthology.orchestra.run.smithi133.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T09:58:10.865 INFO:teuthology.orchestra.run.smithi133.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T09:58:10.866 INFO:teuthology.orchestra.run.smithi133.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T09:58:10.866 INFO:teuthology.orchestra.run.smithi162.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T09:58:10.884 INFO:teuthology.orchestra.run.smithi133.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T09:58:10.887 INFO:teuthology.orchestra.run.smithi133.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T09:58:10.887 INFO:teuthology.orchestra.run.smithi133.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T09:58:10.888 INFO:teuthology.orchestra.run.smithi133.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T09:58:10.889 INFO:teuthology.orchestra.run.smithi133.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T09:58:10.893 INFO:teuthology.orchestra.run.smithi133.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T09:58:10.894 INFO:teuthology.orchestra.run.smithi133.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T09:58:10.906 INFO:teuthology.orchestra.run.smithi162.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T09:58:10.907 INFO:teuthology.orchestra.run.smithi133.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T09:58:10.907 INFO:teuthology.orchestra.run.smithi133.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T09:58:10.907 INFO:teuthology.orchestra.run.smithi133.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T09:58:10.908 INFO:teuthology.orchestra.run.smithi162.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T09:58:10.908 INFO:teuthology.orchestra.run.smithi162.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T09:58:10.917 INFO:teuthology.orchestra.run.smithi162.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T09:58:10.926 INFO:teuthology.orchestra.run.smithi133.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T09:58:10.927 INFO:teuthology.orchestra.run.smithi133.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T09:58:10.928 INFO:teuthology.orchestra.run.smithi133.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T09:58:10.935 INFO:teuthology.orchestra.run.smithi133.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T09:58:10.935 INFO:teuthology.orchestra.run.smithi133.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T09:58:10.935 INFO:teuthology.orchestra.run.smithi133.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T09:58:10.937 INFO:teuthology.orchestra.run.smithi133.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T09:58:10.946 INFO:teuthology.orchestra.run.smithi133.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T09:58:11.150 INFO:teuthology.orchestra.run.smithi162.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T09:58:11.159 INFO:teuthology.orchestra.run.smithi162.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T09:58:11.191 INFO:teuthology.orchestra.run.smithi162.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T09:58:11.201 INFO:teuthology.orchestra.run.smithi162.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T09:58:11.367 INFO:teuthology.orchestra.run.smithi162.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T09:58:11.863 INFO:teuthology.orchestra.run.smithi133.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T09:58:11.872 INFO:teuthology.orchestra.run.smithi162.stdout:Get:25 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T09:58:11.954 INFO:teuthology.orchestra.run.smithi162.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T09:58:11.969 INFO:teuthology.orchestra.run.smithi162.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T09:58:11.971 INFO:teuthology.orchestra.run.smithi162.stdout:Get:28 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T09:58:11.978 INFO:teuthology.orchestra.run.smithi162.stdout:Get:29 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T09:58:11.982 INFO:teuthology.orchestra.run.smithi162.stdout:Get:30 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T09:58:11.985 INFO:teuthology.orchestra.run.smithi162.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T09:58:11.996 INFO:teuthology.orchestra.run.smithi162.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T09:58:11.996 INFO:teuthology.orchestra.run.smithi162.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T09:58:12.014 INFO:teuthology.orchestra.run.smithi162.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T09:58:12.037 INFO:teuthology.orchestra.run.smithi162.stdout:Get:35 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T09:58:12.050 INFO:teuthology.orchestra.run.smithi162.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T09:58:12.062 INFO:teuthology.orchestra.run.smithi162.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T09:58:12.066 INFO:teuthology.orchestra.run.smithi162.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T09:58:12.066 INFO:teuthology.orchestra.run.smithi162.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T09:58:12.237 INFO:teuthology.orchestra.run.smithi162.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T09:58:12.262 INFO:teuthology.orchestra.run.smithi162.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T09:58:12.324 INFO:teuthology.orchestra.run.smithi162.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T09:58:12.348 INFO:teuthology.orchestra.run.smithi162.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T09:58:12.406 INFO:teuthology.orchestra.run.smithi162.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T09:58:12.450 INFO:teuthology.orchestra.run.smithi162.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T09:58:12.471 INFO:teuthology.orchestra.run.smithi162.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T09:58:12.499 INFO:teuthology.orchestra.run.smithi162.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T09:58:12.682 INFO:teuthology.orchestra.run.smithi162.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T09:58:12.697 INFO:teuthology.orchestra.run.smithi162.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T09:58:12.709 INFO:teuthology.orchestra.run.smithi162.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T09:58:12.719 INFO:teuthology.orchestra.run.smithi162.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T09:58:12.903 INFO:teuthology.orchestra.run.smithi162.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T09:58:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T09:58:12.987 INFO:teuthology.orchestra.run.smithi162.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T09:58:13.001 INFO:teuthology.orchestra.run.smithi162.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T09:58:13.165 INFO:teuthology.orchestra.run.smithi162.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T09:58:13.166 INFO:teuthology.orchestra.run.smithi162.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T09:58:13.178 INFO:teuthology.orchestra.run.smithi162.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T09:58:13.183 INFO:teuthology.orchestra.run.smithi162.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T09:58:13.186 INFO:teuthology.orchestra.run.smithi133.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T09:58:13.220 INFO:teuthology.orchestra.run.smithi162.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T09:58:13.225 INFO:teuthology.orchestra.run.smithi162.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T09:58:13.230 INFO:teuthology.orchestra.run.smithi162.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T09:58:13.232 INFO:teuthology.orchestra.run.smithi162.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T09:58:13.238 INFO:teuthology.orchestra.run.smithi162.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T09:58:13.298 INFO:teuthology.orchestra.run.smithi162.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T09:58:13.309 INFO:teuthology.orchestra.run.smithi133.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T09:58:13.310 INFO:teuthology.orchestra.run.smithi133.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T09:58:13.373 INFO:teuthology.orchestra.run.smithi133.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi162.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T09:58:13.422 INFO:teuthology.orchestra.run.smithi162.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T09:58:13.426 INFO:teuthology.orchestra.run.smithi162.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T09:58:13.431 INFO:teuthology.orchestra.run.smithi162.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T09:58:13.432 INFO:teuthology.orchestra.run.smithi133.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T09:58:13.449 INFO:teuthology.orchestra.run.smithi162.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T09:58:13.558 INFO:teuthology.orchestra.run.smithi133.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T09:58:13.612 INFO:teuthology.orchestra.run.smithi162.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T09:58:13.612 INFO:teuthology.orchestra.run.smithi162.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T09:58:13.636 INFO:teuthology.orchestra.run.smithi162.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T09:58:13.900 INFO:teuthology.orchestra.run.smithi162.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T09:58:14.015 INFO:teuthology.orchestra.run.smithi162.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T09:58:14.059 INFO:teuthology.orchestra.run.smithi162.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T09:58:14.149 INFO:teuthology.orchestra.run.smithi162.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T09:58:14.163 INFO:teuthology.orchestra.run.smithi162.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T09:58:14.178 INFO:teuthology.orchestra.run.smithi162.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T09:58:14.281 INFO:teuthology.orchestra.run.smithi162.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T09:58:14.694 INFO:teuthology.orchestra.run.smithi162.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T09:58:14.696 INFO:teuthology.orchestra.run.smithi162.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T09:58:14.698 INFO:teuthology.orchestra.run.smithi162.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T09:58:14.863 INFO:teuthology.orchestra.run.smithi162.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T09:58:15.190 INFO:teuthology.orchestra.run.smithi162.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T09:58:15.629 INFO:teuthology.orchestra.run.smithi162.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T09:58:15.632 INFO:teuthology.orchestra.run.smithi162.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T09:58:15.636 INFO:teuthology.orchestra.run.smithi162.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T09:58:15.637 INFO:teuthology.orchestra.run.smithi162.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T09:58:15.638 INFO:teuthology.orchestra.run.smithi162.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T09:58:15.639 INFO:teuthology.orchestra.run.smithi162.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T09:58:15.684 INFO:teuthology.orchestra.run.smithi133.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T09:58:15.696 INFO:teuthology.orchestra.run.smithi133.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T09:58:15.907 INFO:teuthology.orchestra.run.smithi133.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T09:58:15.971 INFO:teuthology.orchestra.run.smithi162.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T09:58:16.038 INFO:teuthology.orchestra.run.smithi162.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T09:58:16.038 INFO:teuthology.orchestra.run.smithi162.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T09:58:16.039 INFO:teuthology.orchestra.run.smithi162.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T09:58:16.224 INFO:teuthology.orchestra.run.smithi162.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T09:58:16.224 INFO:teuthology.orchestra.run.smithi162.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T09:58:16.230 INFO:teuthology.orchestra.run.smithi162.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T09:58:16.231 INFO:teuthology.orchestra.run.smithi162.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T09:58:16.233 INFO:teuthology.orchestra.run.smithi162.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T09:58:16.235 INFO:teuthology.orchestra.run.smithi162.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T09:58:16.400 INFO:teuthology.orchestra.run.smithi162.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T09:58:16.607 INFO:teuthology.orchestra.run.smithi162.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T09:58:16.917 INFO:teuthology.orchestra.run.smithi162.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T09:58:16.972 INFO:teuthology.orchestra.run.smithi162.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T09:58:16.997 INFO:teuthology.orchestra.run.smithi162.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T09:58:17.005 INFO:teuthology.orchestra.run.smithi162.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T09:58:17.060 INFO:teuthology.orchestra.run.smithi162.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T09:58:17.089 INFO:teuthology.orchestra.run.smithi162.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T09:58:17.091 INFO:teuthology.orchestra.run.smithi162.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T09:58:17.097 INFO:teuthology.orchestra.run.smithi162.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T09:58:17.101 INFO:teuthology.orchestra.run.smithi162.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T09:58:17.296 INFO:teuthology.orchestra.run.smithi162.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T09:58:17.630 INFO:teuthology.orchestra.run.smithi162.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T09:58:18.058 INFO:teuthology.orchestra.run.smithi162.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T09:58:18.060 INFO:teuthology.orchestra.run.smithi162.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T09:58:18.091 INFO:teuthology.orchestra.run.smithi162.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T09:58:18.106 INFO:teuthology.orchestra.run.smithi162.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T09:58:18.225 INFO:teuthology.orchestra.run.smithi162.stdout:Get:118 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T09:58:18.237 INFO:teuthology.orchestra.run.smithi133.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T09:58:18.250 INFO:teuthology.orchestra.run.smithi162.stdout:Get:119 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T09:58:18.449 INFO:teuthology.orchestra.run.smithi162.stdout:Get:120 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T09:58:18.452 INFO:teuthology.orchestra.run.smithi162.stdout:Get:121 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T09:58:18.456 INFO:teuthology.orchestra.run.smithi162.stdout:Get:122 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T09:58:18.595 INFO:teuthology.orchestra.run.smithi133.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T09:58:18.613 INFO:teuthology.orchestra.run.smithi133.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T09:58:18.616 INFO:teuthology.orchestra.run.smithi133.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T09:58:18.652 INFO:teuthology.orchestra.run.smithi162.stdout:Get:123 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T09:58:18.736 INFO:teuthology.orchestra.run.smithi133.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T09:58:18.930 INFO:teuthology.orchestra.run.smithi162.stdout:Get:124 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T09:58:19.170 INFO:teuthology.orchestra.run.smithi133.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T09:58:19.993 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 176 MB in 9s (19.1 MB/s) 2024-04-25T09:58:20.135 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T09:58:20.235 INFO:teuthology.orchestra.run.smithi162.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T09:58:20.241 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T09:58:20.291 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T09:58:20.560 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T09:58:20.587 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T09:58:20.619 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T09:58:20.716 INFO:teuthology.orchestra.run.smithi133.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T09:58:20.716 INFO:teuthology.orchestra.run.smithi133.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T09:58:20.739 INFO:teuthology.orchestra.run.smithi133.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T09:58:20.863 INFO:teuthology.orchestra.run.smithi133.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T09:58:20.904 INFO:teuthology.orchestra.run.smithi133.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T09:58:20.920 INFO:teuthology.orchestra.run.smithi133.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T09:58:20.931 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T09:58:20.957 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T09:58:20.997 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T09:58:21.025 INFO:teuthology.orchestra.run.smithi133.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T09:58:21.350 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T09:58:21.377 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T09:58:21.458 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T09:58:21.501 INFO:teuthology.orchestra.run.smithi133.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T09:58:21.501 INFO:teuthology.orchestra.run.smithi133.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T09:58:21.787 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T09:58:21.814 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T09:58:21.854 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T09:58:22.207 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T09:58:22.233 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T09:58:22.265 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T09:58:22.693 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T09:58:22.720 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:22.768 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T09:58:23.173 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:23.213 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:23.525 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T09:58:23.552 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:23.592 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:23.937 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T09:58:23.963 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T09:58:24.003 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T09:58:24.217 INFO:teuthology.orchestra.run.smithi133.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T09:58:24.219 INFO:teuthology.orchestra.run.smithi133.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T09:58:24.224 INFO:teuthology.orchestra.run.smithi133.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T09:58:24.225 INFO:teuthology.orchestra.run.smithi133.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T09:58:24.225 INFO:teuthology.orchestra.run.smithi133.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T09:58:24.226 INFO:teuthology.orchestra.run.smithi133.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T09:58:24.281 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package librados2. 2024-04-25T09:58:24.305 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:24.339 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:24.704 INFO:teuthology.orchestra.run.smithi133.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T09:58:24.801 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package librbd1. 2024-04-25T09:58:24.827 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:24.867 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:25.312 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T09:58:25.320 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 176 MB in 14s (12.2 MB/s) 2024-04-25T09:58:25.339 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:25.378 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:25.457 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T09:58:25.517 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T09:58:25.522 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T09:58:25.563 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T09:58:25.640 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rados. 2024-04-25T09:58:25.655 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:25.689 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:25.858 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T09:58:25.874 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T09:58:25.907 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T09:58:25.984 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T09:58:26.010 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:26.051 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:26.210 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T09:58:26.237 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T09:58:26.277 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T09:58:26.337 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T09:58:26.363 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:26.403 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:26.580 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T09:58:26.606 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T09:58:26.672 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T09:58:26.706 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T09:58:26.732 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:26.772 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:27.000 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T09:58:27.026 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T09:58:27.059 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T09:58:27.066 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T09:58:27.085 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T09:58:27.117 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T09:58:27.345 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T09:58:27.361 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:27.395 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:27.402 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T09:58:27.429 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T09:58:27.469 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T09:58:27.715 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T09:58:27.741 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T09:58:27.781 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T09:58:27.889 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T09:58:27.916 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:27.955 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:28.051 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T09:58:28.077 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T09:58:28.117 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T09:58:28.342 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T09:58:28.369 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:28.403 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T09:58:28.419 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T09:58:28.454 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T09:58:28.575 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:28.749 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package lua5.1. 2024-04-25T09:58:28.775 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T09:58:28.815 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T09:58:29.295 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T09:58:29.318 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package lua-any. 2024-04-25T09:58:29.322 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:29.334 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T09:58:29.361 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:29.377 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T09:58:29.621 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package zip. 2024-04-25T09:58:29.637 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T09:58:29.665 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T09:58:29.671 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T09:58:29.691 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T09:58:29.723 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T09:58:29.941 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package unzip. 2024-04-25T09:58:29.967 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T09:58:30.001 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librados2. 2024-04-25T09:58:30.007 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T09:58:30.028 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:30.067 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:30.429 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package luarocks. 2024-04-25T09:58:30.455 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T09:58:30.495 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T09:58:30.511 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librbd1. 2024-04-25T09:58:30.538 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:30.578 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:30.878 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package librgw2. 2024-04-25T09:58:30.905 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:30.945 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:31.023 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T09:58:31.038 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:31.081 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:31.333 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rados. 2024-04-25T09:58:31.348 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:31.383 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:31.464 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T09:58:31.479 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:31.514 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:31.686 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T09:58:31.712 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:31.752 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:31.859 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T09:58:31.886 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T09:58:31.925 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T09:58:32.005 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T09:58:32.020 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:32.063 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:32.239 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T09:58:32.265 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:32.297 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:32.332 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T09:58:32.359 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:32.399 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:32.692 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-common. 2024-04-25T09:58:32.702 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T09:58:32.719 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:32.728 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T09:58:32.758 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:32.760 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T09:58:32.987 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T09:58:33.003 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:33.046 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:33.465 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T09:58:33.492 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T09:58:33.540 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T09:58:33.834 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T09:58:33.850 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T09:58:33.893 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T09:58:34.031 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-base. 2024-04-25T09:58:34.058 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:34.106 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T09:58:34.230 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T09:58:34.270 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T09:58:34.632 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package lua5.1. 2024-04-25T09:58:34.658 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T09:58:34.707 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T09:58:34.869 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T09:58:34.895 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T09:58:34.935 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T09:58:35.085 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package lua-any. 2024-04-25T09:58:35.111 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T09:58:35.151 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T09:58:35.180 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T09:58:35.206 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T09:58:35.238 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T09:58:35.412 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package zip. 2024-04-25T09:58:35.427 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T09:58:35.470 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T09:58:35.558 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T09:58:35.584 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T09:58:35.624 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T09:58:35.782 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package unzip. 2024-04-25T09:58:35.809 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T09:58:35.848 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T09:58:35.927 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T09:58:35.954 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T09:58:35.994 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T09:58:36.281 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T09:58:36.307 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T09:58:36.310 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package luarocks. 2024-04-25T09:58:36.337 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T09:58:36.338 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T09:58:36.385 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T09:58:36.608 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T09:58:36.634 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T09:58:36.675 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T09:58:36.801 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librgw2. 2024-04-25T09:58:36.828 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:36.868 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:36.978 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-portend. 2024-04-25T09:58:37.004 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T09:58:37.044 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T09:58:37.331 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T09:58:37.397 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T09:58:37.403 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T09:58:37.419 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:37.461 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:37.692 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T09:58:37.719 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T09:58:37.758 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T09:58:37.815 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T09:58:37.842 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T09:58:37.881 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T09:58:38.114 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T09:58:38.139 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T09:58:38.150 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T09:58:38.177 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:38.178 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T09:58:38.209 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:38.532 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T09:58:38.559 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T09:58:38.599 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T09:58:38.670 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-common. 2024-04-25T09:58:38.697 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:38.737 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:38.877 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T09:58:38.903 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T09:58:38.935 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T09:58:39.205 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-mako. 2024-04-25T09:58:39.231 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T09:58:39.271 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T09:58:39.608 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T09:58:39.634 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T09:58:39.674 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T09:58:39.977 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T09:58:40.004 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T09:58:40.044 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T09:58:40.092 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-base. 2024-04-25T09:58:40.118 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:40.168 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:40.305 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-webob. 2024-04-25T09:58:40.332 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T09:58:40.364 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T09:58:40.633 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T09:58:40.660 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T09:58:40.696 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T09:58:40.703 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T09:58:40.711 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T09:58:40.754 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T09:58:41.003 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T09:58:41.030 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T09:58:41.057 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T09:58:41.069 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T09:58:41.083 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T09:58:41.124 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T09:58:41.356 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-paste. 2024-04-25T09:58:41.383 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T09:58:41.422 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T09:58:41.452 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T09:58:41.478 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T09:58:41.518 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T09:58:41.725 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T09:58:41.741 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T09:58:41.784 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T09:58:41.829 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T09:58:41.855 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T09:58:41.896 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T09:58:42.095 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T09:58:42.122 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T09:58:42.162 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T09:58:42.165 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T09:58:42.192 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T09:58:42.457 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T09:58:42.483 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T09:58:42.523 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T09:58:42.534 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T09:58:42.561 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T09:58:42.601 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T09:58:42.818 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T09:58:42.845 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T09:58:42.876 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T09:58:42.912 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-portend. 2024-04-25T09:58:42.938 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T09:58:42.979 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T09:58:43.240 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T09:58:43.257 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T09:58:43.267 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T09:58:43.283 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T09:58:43.306 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T09:58:43.315 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T09:58:43.976 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-idna. 2024-04-25T09:58:44.003 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T09:58:44.252 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T09:58:44.278 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T09:58:44.318 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T09:58:44.409 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T09:58:44.646 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T09:58:44.673 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T09:58:44.696 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T09:58:44.713 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T09:58:44.722 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T09:58:44.762 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T09:58:44.983 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T09:58:45.009 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T09:58:45.049 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T09:58:45.049 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-requests. 2024-04-25T09:58:45.076 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T09:58:45.107 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T09:58:45.352 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T09:58:45.378 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T09:58:45.385 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T09:58:45.412 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T09:58:45.418 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T09:58:45.452 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T09:58:45.730 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T09:58:45.746 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:45.746 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-mako. 2024-04-25T09:58:45.772 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T09:58:45.788 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:45.821 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T09:58:46.110 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T09:58:46.126 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:46.132 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T09:58:46.159 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T09:58:46.160 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T09:58:46.435 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T09:58:46.462 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T09:58:46.488 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T09:58:46.501 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T09:58:46.515 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T09:58:46.554 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T09:58:46.804 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-webob. 2024-04-25T09:58:46.831 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T09:58:46.871 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T09:58:46.883 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T09:58:46.910 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T09:58:46.949 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T09:58:47.234 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T09:58:47.244 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T09:58:47.271 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:47.277 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T09:58:47.302 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:47.568 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T09:58:47.595 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T09:58:47.635 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T09:58:47.689 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T09:58:47.716 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:47.755 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-paste. 2024-04-25T09:58:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T09:58:47.962 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T09:58:48.274 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T09:58:48.289 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T09:58:48.323 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T09:58:48.368 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T09:58:48.395 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:48.434 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:48.585 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T09:58:48.611 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T09:58:48.651 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T09:58:48.954 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T09:58:48.980 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T09:58:49.020 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T09:58:49.340 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T09:58:49.367 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T09:58:49.407 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T09:58:49.454 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph. 2024-04-25T09:58:49.469 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:49.504 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:49.753 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T09:58:49.780 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T09:58:49.819 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T09:58:49.832 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T09:58:49.859 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:49.898 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:50.356 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-idna. 2024-04-25T09:58:50.382 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T09:58:50.422 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T09:58:50.451 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T09:58:50.478 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:50.518 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:50.709 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T09:58:50.735 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T09:58:50.775 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T09:58:50.896 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package cephadm. 2024-04-25T09:58:50.922 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:50.962 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:51.103 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-requests. 2024-04-25T09:58:51.130 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T09:58:51.169 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T09:58:51.232 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T09:58:51.259 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T09:58:51.299 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T09:58:51.489 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T09:58:51.516 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T09:58:51.555 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T09:58:51.593 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T09:58:51.608 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T09:58:51.651 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T09:58:51.850 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T09:58:51.877 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:51.917 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:52.263 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T09:58:52.278 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:52.312 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:52.582 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T09:58:52.609 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T09:58:52.648 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-babel. 2024-04-25T09:58:52.649 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T09:58:52.675 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T09:58:52.714 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T09:58:52.918 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T09:58:52.945 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T09:58:52.977 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T09:58:53.018 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T09:58:53.044 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T09:58:53.084 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T09:58:53.321 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T09:58:53.348 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:53.387 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:53.437 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T09:58:53.464 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:53.504 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:53.791 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T09:58:53.818 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:53.849 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T09:58:53.857 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:53.876 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T09:58:53.916 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T09:58:54.202 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-routes. 2024-04-25T09:58:54.228 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T09:58:54.268 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T09:58:54.394 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:54.630 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T09:58:54.657 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:54.705 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:55.108 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T09:58:55.124 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T09:58:55.158 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T09:58:55.522 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph. 2024-04-25T09:58:55.537 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:55.588 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:55.670 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T09:58:55.697 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T09:58:55.736 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T09:58:55.908 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T09:58:55.934 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:55.974 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:56.099 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T09:58:56.126 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T09:58:56.242 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T09:58:56.269 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T09:58:56.296 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:56.344 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:56.512 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T09:58:56.539 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T09:58:56.570 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T09:58:56.746 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package cephadm. 2024-04-25T09:58:56.773 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:56.813 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:57.124 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T09:58:57.151 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T09:58:57.190 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T09:58:57.238 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T09:58:57.254 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:57.288 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:57.518 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T09:58:57.544 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T09:58:57.584 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T09:58:58.200 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T09:58:58.227 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T09:58:58.266 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T09:58:58.569 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T09:58:58.573 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-babel. 2024-04-25T09:58:58.596 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T09:58:58.599 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T09:58:58.636 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T09:58:58.639 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T09:58:58.931 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T09:58:58.933 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T09:58:58.958 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T09:58:58.960 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T09:58:58.991 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T09:58:58.997 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T09:58:59.259 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T09:58:59.286 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T09:58:59.311 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T09:58:59.317 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T09:58:59.338 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:59.638 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T09:58:59.665 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T09:58:59.704 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T09:58:59.723 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T09:58:59.750 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T09:58:59.781 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T09:59:00.025 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T09:59:00.052 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T09:59:00.091 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T09:59:00.093 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-routes. 2024-04-25T09:59:00.120 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T09:59:00.158 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T09:59:00.403 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T09:59:00.430 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T09:59:00.469 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T09:59:00.645 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T09:59:00.672 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:00.711 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:00.973 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T09:59:01.000 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T09:59:01.040 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T09:59:01.125 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T09:59:01.141 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T09:59:01.175 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T09:59:01.620 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T09:59:01.647 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T09:59:01.653 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T09:59:01.670 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:01.678 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T09:59:01.703 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:02.023 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T09:59:02.032 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T09:59:02.050 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T09:59:02.059 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T09:59:02.089 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T09:59:02.184 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T09:59:02.418 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T09:59:02.445 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T09:59:02.484 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T09:59:02.495 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T09:59:02.522 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T09:59:02.561 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T09:59:02.711 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package jq. 2024-04-25T09:59:02.727 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T09:59:02.762 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T09:59:03.006 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package socat. 2024-04-25T09:59:03.022 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T09:59:03.090 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T09:59:03.254 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T09:59:03.281 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:03.320 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:03.393 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T09:59:03.408 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T09:59:03.451 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T09:59:03.822 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-test. 2024-04-25T09:59:03.849 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:03.897 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:04.315 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T09:59:04.342 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T09:59:04.381 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T09:59:04.660 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T09:59:04.675 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T09:59:04.717 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T09:59:05.038 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T09:59:05.065 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T09:59:05.104 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T09:59:05.415 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T09:59:05.443 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T09:59:05.482 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T09:59:05.794 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T09:59:05.822 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T09:59:05.860 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T09:59:06.172 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T09:59:06.199 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T09:59:06.235 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T09:59:06.238 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T09:59:06.262 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:06.301 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:06.500 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T09:59:06.527 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T09:59:06.570 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T09:59:06.681 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T09:59:06.708 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:06.747 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:06.878 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T09:59:06.905 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T09:59:06.944 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T09:59:07.067 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T09:59:07.094 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T09:59:07.133 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T09:59:07.361 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T09:59:07.377 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:07.419 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:07.658 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T09:59:07.672 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T09:59:07.686 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:07.699 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:07.725 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:07.738 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:07.991 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T09:59:07.994 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T09:59:08.018 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:08.022 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T09:59:08.052 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T09:59:08.057 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:08.352 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T09:59:08.379 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T09:59:08.380 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T09:59:08.408 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T09:59:08.418 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T09:59:08.447 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T09:59:08.733 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package jq. 2024-04-25T09:59:08.740 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T09:59:08.760 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T09:59:08.799 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T09:59:08.806 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T09:59:09.052 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package socat. 2024-04-25T09:59:09.068 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T09:59:09.102 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T09:59:09.186 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T09:59:09.213 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T09:59:09.252 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T09:59:09.388 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T09:59:09.404 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T09:59:09.438 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T09:59:09.633 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package pkg-config. 2024-04-25T09:59:09.661 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T09:59:09.699 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T09:59:09.776 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-test. 2024-04-25T09:59:09.804 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:09.842 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:09.919 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T09:59:09.935 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T09:59:09.977 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T09:59:10.349 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T09:59:10.377 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T09:59:10.416 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T09:59:10.702 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T09:59:10.730 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T09:59:10.768 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T09:59:11.097 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T09:59:11.124 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T09:59:11.163 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T09:59:11.508 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T09:59:11.536 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T09:59:11.574 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T09:59:11.894 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-py. 2024-04-25T09:59:11.922 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T09:59:11.968 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T09:59:12.163 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T09:59:12.178 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:12.213 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:12.305 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T09:59:12.333 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T09:59:12.371 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T09:59:12.525 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T09:59:12.553 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:12.584 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:12.851 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T09:59:12.879 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T09:59:12.886 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T09:59:12.914 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T09:59:12.917 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T09:59:12.953 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T09:59:13.187 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-toml. 2024-04-25T09:59:13.214 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T09:59:13.214 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T09:59:13.229 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:13.253 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T09:59:13.272 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:13.549 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T09:59:13.556 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T09:59:13.566 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:13.584 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T09:59:13.608 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:13.622 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T09:59:13.843 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T09:59:13.860 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:13.893 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:13.918 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T09:59:13.934 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T09:59:13.967 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T09:59:14.179 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T09:59:14.207 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T09:59:14.220 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T09:59:14.248 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T09:59:14.262 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T09:59:14.286 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T09:59:14.592 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T09:59:14.620 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T09:59:14.658 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T09:59:14.990 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package radosgw. 2024-04-25T09:59:15.018 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:15.021 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T09:59:15.048 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T09:59:15.056 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:15.087 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T09:59:15.510 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package pkg-config. 2024-04-25T09:59:15.537 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T09:59:15.575 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T09:59:15.829 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T09:59:15.856 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T09:59:15.876 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T09:59:15.886 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T09:59:15.893 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:15.935 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:16.290 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T09:59:16.318 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T09:59:16.356 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T09:59:16.481 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T09:59:16.626 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T09:59:16.654 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T09:59:16.692 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T09:59:16.723 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T09:59:16.849 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T09:59:17.029 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T09:59:17.056 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T09:59:17.095 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T09:59:17.113 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T09:59:17.406 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T09:59:17.428 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T09:59:17.434 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T09:59:17.473 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T09:59:17.826 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-py. 2024-04-25T09:59:17.829 INFO:teuthology.orchestra.run.smithi162.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T09:59:17.854 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T09:59:17.892 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T09:59:17.901 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:18.204 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T09:59:18.231 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T09:59:18.270 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T09:59:18.392 INFO:teuthology.orchestra.run.smithi162.stdout:Adding system user cephadm....done 2024-04-25T09:59:18.524 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T09:59:18.691 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T09:59:18.719 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T09:59:18.757 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T09:59:18.800 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T09:59:19.060 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T09:59:19.077 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-toml. 2024-04-25T09:59:19.105 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T09:59:19.143 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T09:59:19.186 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T09:59:19.446 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T09:59:19.463 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T09:59:19.463 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T09:59:19.504 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T09:59:19.705 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T09:59:19.806 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T09:59:19.858 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T09:59:19.885 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T09:59:19.924 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T09:59:20.108 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T09:59:20.194 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T09:59:20.210 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T09:59:20.252 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T09:59:20.468 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T09:59:20.727 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T09:59:20.956 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package radosgw. 2024-04-25T09:59:20.978 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T09:59:20.983 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:21.022 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:21.230 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T09:59:21.539 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T09:59:21.799 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T09:59:21.875 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T09:59:21.892 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:21.925 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:22.058 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T09:59:22.229 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T09:59:22.368 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:22.488 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T09:59:22.589 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T09:59:22.636 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T09:59:22.761 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T09:59:22.827 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T09:59:22.871 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T09:59:23.122 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T09:59:23.159 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T09:59:23.256 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T09:59:23.516 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T09:59:23.610 INFO:teuthology.orchestra.run.smithi133.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T09:59:23.674 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:23.964 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T09:59:23.997 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T09:59:24.069 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T09:59:24.132 INFO:teuthology.orchestra.run.smithi133.stdout:Adding system user cephadm....done 2024-04-25T09:59:24.194 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T09:59:24.288 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T09:59:24.546 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T09:59:24.655 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T09:59:24.791 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T09:59:24.892 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T09:59:24.923 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T09:59:25.176 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T09:59:25.182 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T09:59:25.436 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T09:59:25.492 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T09:59:25.562 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T09:59:25.618 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T09:59:25.863 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T09:59:26.223 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T09:59:26.303 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T09:59:26.466 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T09:59:26.529 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T09:59:26.638 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T09:59:26.717 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T09:59:26.748 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T09:59:26.985 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T09:59:27.057 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T09:59:27.245 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T09:59:27.433 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T09:59:27.505 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T09:59:27.684 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T09:59:27.772 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T09:59:28.006 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:28.069 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T09:59:28.241 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T09:59:28.345 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T09:59:28.359 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T09:59:28.485 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T09:59:28.621 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T09:59:28.769 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T09:59:28.895 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T09:59:28.931 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T09:59:29.155 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T09:59:29.174 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T09:59:29.300 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:29.538 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T09:59:29.560 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T09:59:29.593 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T09:59:29.608 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T09:59:29.717 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T09:59:29.910 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T09:59:30.068 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:30.194 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T09:59:30.454 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T09:59:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T09:59:31.090 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T09:59:31.100 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T09:59:31.190 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T09:59:31.360 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:31.494 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T09:59:31.818 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T09:59:31.950 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T09:59:32.163 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T09:59:32.252 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T09:59:32.288 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T09:59:32.378 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T09:59:32.504 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T09:59:32.590 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T09:59:32.788 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T09:59:32.849 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T09:59:33.100 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:33.156 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T09:59:33.227 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T09:59:33.391 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T09:59:33.528 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T09:59:33.751 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T09:59:33.754 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T09:59:33.997 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T09:59:34.036 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T09:59:34.256 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T09:59:34.345 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T09:59:34.507 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T09:59:34.705 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T09:59:34.775 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T09:59:34.901 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T09:59:34.956 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T09:59:35.057 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:35.185 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T09:59:35.294 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T09:59:35.341 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T09:59:35.579 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T09:59:35.651 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T09:59:35.897 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T09:59:36.156 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T09:59:36.282 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T09:59:36.408 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:36.497 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T09:59:36.735 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T09:59:36.848 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T09:59:36.869 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T09:59:37.083 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:37.218 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T09:59:37.237 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T09:59:37.485 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T09:59:37.488 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T09:59:37.611 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T09:59:37.623 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T09:59:37.737 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T09:59:37.874 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:38.016 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T09:59:38.022 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T09:59:38.256 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T09:59:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T09:59:38.500 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:38.625 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T09:59:38.635 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T09:59:38.886 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T09:59:38.928 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T09:59:39.073 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T09:59:39.349 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T09:59:39.446 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T09:59:39.625 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T09:59:39.688 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T09:59:39.893 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T09:59:39.914 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T09:59:40.157 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T09:59:40.284 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T09:59:40.295 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T09:59:40.570 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T09:59:40.571 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T09:59:40.694 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T09:59:40.915 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T09:59:40.987 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T09:59:41.041 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:41.167 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:41.277 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T09:59:41.288 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T09:59:41.522 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T09:59:41.554 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:41.640 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T09:59:41.681 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:41.758 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:41.807 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T09:59:41.848 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T09:59:42.083 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:42.084 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T09:59:42.201 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:42.210 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T09:59:42.319 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:42.513 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T09:59:42.513 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T09:59:42.623 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T09:59:42.910 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T09:59:42.961 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:43.022 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T09:59:43.062 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:43.197 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:43.265 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:43.323 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:43.383 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T09:59:43.449 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T09:59:43.718 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T09:59:44.018 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T09:59:44.295 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T09:59:44.506 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T09:59:44.611 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:44.737 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:44.757 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T09:59:44.862 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:44.997 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:45.019 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T09:59:45.123 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:45.249 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:45.270 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T09:59:45.646 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T09:59:45.749 INFO:teuthology.orchestra.run.smithi162.stdout:Adding group ceph....done 2024-04-25T09:59:45.906 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T09:59:46.118 INFO:teuthology.orchestra.run.smithi162.stdout:Adding system user ceph....done 2024-04-25T09:59:46.217 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T09:59:46.262 INFO:teuthology.orchestra.run.smithi162.stdout:Setting system user ceph properties....done 2024-04-25T09:59:46.280 INFO:teuthology.orchestra.run.smithi162.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T09:59:46.343 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:46.416 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T09:59:46.487 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:46.628 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T09:59:46.782 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T09:59:46.922 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:47.023 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:47.134 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T09:59:47.242 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:47.409 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:47.442 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:59:47.442 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:59:47.543 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:47.669 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T09:59:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T09:59:47.901 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:48.002 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:48.379 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:48.454 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:59:48.455 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:59:48.497 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:48.598 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:48.699 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:48.808 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T09:59:48.912 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:49.259 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T09:59:49.739 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:49.962 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T09:59:49.962 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T09:59:49.969 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:50.112 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:50.222 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:50.323 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:50.424 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:50.541 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:50.583 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:50.796 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T09:59:50.796 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T09:59:51.076 INFO:teuthology.orchestra.run.smithi133.stdout:Adding group ceph....done 2024-04-25T09:59:51.268 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:51.452 INFO:teuthology.orchestra.run.smithi133.stdout:Adding system user ceph....done 2024-04-25T09:59:51.546 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T09:59:51.547 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T09:59:51.562 INFO:teuthology.orchestra.run.smithi133.stdout:Setting system user ceph properties....done 2024-04-25T09:59:51.577 INFO:teuthology.orchestra.run.smithi133.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T09:59:51.720 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T09:59:52.037 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:52.063 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T09:59:52.163 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:52.339 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:52.525 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:52.545 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T09:59:52.545 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T09:59:52.734 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:59:52.734 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:59:53.194 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:53.278 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:53.345 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:53.445 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:53.471 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:53.614 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:53.907 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:59:53.907 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:59:53.942 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T09:59:54.043 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T09:59:54.187 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T09:59:54.389 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T09:59:55.224 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:55.304 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T09:59:55.501 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T09:59:55.501 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T09:59:55.954 INFO:teuthology.orchestra.run.smithi162.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T09:59:55.958 DEBUG:teuthology.orchestra.run.smithi162:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T09:59:55.992 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:56.065 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-04-25T09:59:56.208 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T09:59:56.208 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T09:59:56.265 INFO:teuthology.orchestra.run.smithi162.stdout:Building dependency tree... 2024-04-25T09:59:56.266 INFO:teuthology.orchestra.run.smithi162.stdout:Reading state information... 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi162.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi162.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi162.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi162.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi162.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi162.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi162.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi162.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T09:59:56.459 INFO:teuthology.orchestra.run.smithi162.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T09:59:56.459 INFO:teuthology.orchestra.run.smithi162.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:59:56.460 INFO:teuthology.orchestra.run.smithi162.stdout:The following additional packages will be installed: 2024-04-25T09:59:56.460 INFO:teuthology.orchestra.run.smithi162.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T09:59:56.460 INFO:teuthology.orchestra.run.smithi162.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T09:59:56.460 INFO:teuthology.orchestra.run.smithi162.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T09:59:56.460 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T09:59:56.460 INFO:teuthology.orchestra.run.smithi162.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T09:59:56.461 INFO:teuthology.orchestra.run.smithi162.stdout:Suggested packages: 2024-04-25T09:59:56.461 INFO:teuthology.orchestra.run.smithi162.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T09:59:56.462 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T09:59:56.462 INFO:teuthology.orchestra.run.smithi162.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T09:59:56.504 INFO:teuthology.orchestra.run.smithi162.stdout:The following NEW packages will be installed: 2024-04-25T09:59:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T09:59:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T09:59:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T09:59:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T09:59:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T09:59:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T09:59:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T09:59:56.507 INFO:teuthology.orchestra.run.smithi162.stdout:The following packages will be upgraded: 2024-04-25T09:59:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T09:59:56.569 INFO:teuthology.orchestra.run.smithi162.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T09:59:56.570 INFO:teuthology.orchestra.run.smithi162.stdout:Need to get 55.7 MB of archives. 2024-04-25T09:59:56.570 INFO:teuthology.orchestra.run.smithi162.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T09:59:56.570 INFO:teuthology.orchestra.run.smithi162.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T09:59:56.702 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:56.735 INFO:teuthology.orchestra.run.smithi162.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T09:59:56.742 INFO:teuthology.orchestra.run.smithi162.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T09:59:56.781 INFO:teuthology.orchestra.run.smithi162.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T09:59:56.781 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T09:59:56.782 INFO:teuthology.orchestra.run.smithi162.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T09:59:56.793 INFO:teuthology.orchestra.run.smithi162.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T09:59:56.794 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T09:59:56.796 INFO:teuthology.orchestra.run.smithi162.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T09:59:56.798 INFO:teuthology.orchestra.run.smithi162.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T09:59:56.798 INFO:teuthology.orchestra.run.smithi162.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T09:59:56.800 INFO:teuthology.orchestra.run.smithi162.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T09:59:56.803 INFO:teuthology.orchestra.run.smithi162.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T09:59:56.822 INFO:teuthology.orchestra.run.smithi162.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T09:59:56.822 INFO:teuthology.orchestra.run.smithi162.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T09:59:56.824 INFO:teuthology.orchestra.run.smithi162.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T09:59:56.985 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T09:59:56.985 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T09:59:57.140 INFO:teuthology.orchestra.run.smithi162.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T09:59:57.141 INFO:teuthology.orchestra.run.smithi162.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T09:59:57.141 INFO:teuthology.orchestra.run.smithi162.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T09:59:57.144 INFO:teuthology.orchestra.run.smithi162.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T09:59:57.144 INFO:teuthology.orchestra.run.smithi162.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T09:59:57.148 INFO:teuthology.orchestra.run.smithi162.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T09:59:57.149 INFO:teuthology.orchestra.run.smithi162.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T09:59:57.156 INFO:teuthology.orchestra.run.smithi162.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T09:59:57.156 INFO:teuthology.orchestra.run.smithi162.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T09:59:57.174 INFO:teuthology.orchestra.run.smithi162.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T09:59:57.178 INFO:teuthology.orchestra.run.smithi162.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T09:59:57.188 INFO:teuthology.orchestra.run.smithi162.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T09:59:57.210 INFO:teuthology.orchestra.run.smithi162.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T09:59:57.213 INFO:teuthology.orchestra.run.smithi162.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T09:59:57.214 INFO:teuthology.orchestra.run.smithi162.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T09:59:57.214 INFO:teuthology.orchestra.run.smithi162.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T09:59:57.214 INFO:teuthology.orchestra.run.smithi162.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T09:59:57.225 INFO:teuthology.orchestra.run.smithi162.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T09:59:57.227 INFO:teuthology.orchestra.run.smithi162.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T09:59:57.450 INFO:teuthology.orchestra.run.smithi162.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T09:59:57.450 INFO:teuthology.orchestra.run.smithi162.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T09:59:57.450 INFO:teuthology.orchestra.run.smithi162.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T09:59:57.450 INFO:teuthology.orchestra.run.smithi162.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T09:59:57.450 INFO:teuthology.orchestra.run.smithi162.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T09:59:57.450 INFO:teuthology.orchestra.run.smithi162.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T09:59:57.462 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:57.462 INFO:teuthology.orchestra.run.smithi162.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T09:59:57.462 INFO:teuthology.orchestra.run.smithi162.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T09:59:57.588 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:57.739 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:57.865 INFO:teuthology.orchestra.run.smithi162.stdout:Preconfiguring packages ... 2024-04-25T09:59:57.939 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T09:59:57.940 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T09:59:58.004 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 55.7 MB in 1s (58.9 MB/s) 2024-04-25T09:59:58.432 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:58.459 INFO:teuthology.orchestra.run.smithi162.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T09:59:58.464 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T09:59:58.575 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:58.601 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T09:59:58.701 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:58.852 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:59.196 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T09:59:59.240 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T09:59:59.297 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T09:59:59.433 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:00:00.166 INFO:teuthology.orchestra.run.smithi162.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T10:00:00.441 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package flex. 2024-04-25T10:00:00.543 INFO:teuthology.orchestra.run.smithi162.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T10:00:00.548 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:00.588 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T10:00:00.591 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:00:01.038 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:00:01.211 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T10:00:01.236 INFO:teuthology.orchestra.run.smithi133.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:00:01.239 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T10:00:01.294 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-04-25T10:00:01.480 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2024-04-25T10:00:01.480 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2024-04-25T10:00:01.615 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:01.649 INFO:teuthology.orchestra.run.smithi133.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T10:00:01.649 INFO:teuthology.orchestra.run.smithi133.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T10:00:01.649 INFO:teuthology.orchestra.run.smithi133.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T10:00:01.649 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:00:01.650 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:00:01.650 INFO:teuthology.orchestra.run.smithi133.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:00:01.650 INFO:teuthology.orchestra.run.smithi133.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:00:01.650 INFO:teuthology.orchestra.run.smithi133.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:00:01.651 INFO:teuthology.orchestra.run.smithi133.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:00:01.651 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:00:01.651 INFO:teuthology.orchestra.run.smithi133.stdout:The following additional packages will be installed: 2024-04-25T10:00:01.651 INFO:teuthology.orchestra.run.smithi133.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T10:00:01.651 INFO:teuthology.orchestra.run.smithi133.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T10:00:01.652 INFO:teuthology.orchestra.run.smithi133.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T10:00:01.652 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T10:00:01.652 INFO:teuthology.orchestra.run.smithi133.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:00:01.652 INFO:teuthology.orchestra.run.smithi133.stdout:Suggested packages: 2024-04-25T10:00:01.653 INFO:teuthology.orchestra.run.smithi133.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T10:00:01.653 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T10:00:01.653 INFO:teuthology.orchestra.run.smithi133.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T10:00:01.693 INFO:teuthology.orchestra.run.smithi133.stdout:The following NEW packages will be installed: 2024-04-25T10:00:01.694 INFO:teuthology.orchestra.run.smithi133.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T10:00:01.694 INFO:teuthology.orchestra.run.smithi133.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T10:00:01.694 INFO:teuthology.orchestra.run.smithi133.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T10:00:01.694 INFO:teuthology.orchestra.run.smithi133.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T10:00:01.694 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T10:00:01.694 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T10:00:01.694 INFO:teuthology.orchestra.run.smithi133.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:00:01.695 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages will be upgraded: 2024-04-25T10:00:01.695 INFO:teuthology.orchestra.run.smithi133.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T10:00:02.131 INFO:teuthology.orchestra.run.smithi162.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:00:02.136 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:00:02.183 INFO:teuthology.orchestra.run.smithi133.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T10:00:02.183 INFO:teuthology.orchestra.run.smithi133.stdout:Need to get 55.7 MB of archives. 2024-04-25T10:00:02.183 INFO:teuthology.orchestra.run.smithi133.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T10:00:02.183 INFO:teuthology.orchestra.run.smithi133.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T10:00:02.261 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:00:02.412 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T10:00:02.425 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T10:00:02.620 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:00:02.638 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T10:00:02.639 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T10:00:02.640 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T10:00:02.657 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T10:00:02.659 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T10:00:02.662 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T10:00:02.665 INFO:teuthology.orchestra.run.smithi133.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T10:00:02.666 INFO:teuthology.orchestra.run.smithi133.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T10:00:02.668 INFO:teuthology.orchestra.run.smithi133.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T10:00:02.689 INFO:teuthology.orchestra.run.smithi133.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T10:00:02.719 INFO:teuthology.orchestra.run.smithi133.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T10:00:02.720 INFO:teuthology.orchestra.run.smithi133.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T10:00:02.721 INFO:teuthology.orchestra.run.smithi133.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T10:00:02.751 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:00:03.078 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:00:03.427 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package bison. 2024-04-25T10:00:03.524 INFO:teuthology.orchestra.run.smithi133.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T10:00:03.530 INFO:teuthology.orchestra.run.smithi133.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T10:00:03.530 INFO:teuthology.orchestra.run.smithi133.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T10:00:03.530 INFO:teuthology.orchestra.run.smithi162.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:00:03.533 INFO:teuthology.orchestra.run.smithi133.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T10:00:03.534 INFO:teuthology.orchestra.run.smithi133.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T10:00:03.536 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T10:00:03.542 INFO:teuthology.orchestra.run.smithi133.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T10:00:03.546 INFO:teuthology.orchestra.run.smithi133.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T10:00:03.558 INFO:teuthology.orchestra.run.smithi133.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T10:00:03.558 INFO:teuthology.orchestra.run.smithi133.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T10:00:03.579 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:00:03.591 INFO:teuthology.orchestra.run.smithi133.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T10:00:03.598 INFO:teuthology.orchestra.run.smithi133.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T10:00:03.615 INFO:teuthology.orchestra.run.smithi133.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T10:00:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T10:00:03.671 INFO:teuthology.orchestra.run.smithi133.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T10:00:03.672 INFO:teuthology.orchestra.run.smithi133.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T10:00:03.673 INFO:teuthology.orchestra.run.smithi133.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T10:00:03.673 INFO:teuthology.orchestra.run.smithi133.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T10:00:03.688 INFO:teuthology.orchestra.run.smithi133.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T10:00:03.703 INFO:teuthology.orchestra.run.smithi133.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T10:00:03.954 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T10:00:03.970 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T10:00:04.004 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:00:04.139 INFO:teuthology.orchestra.run.smithi133.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T10:00:04.140 INFO:teuthology.orchestra.run.smithi133.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T10:00:04.140 INFO:teuthology.orchestra.run.smithi133.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T10:00:04.140 INFO:teuthology.orchestra.run.smithi133.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T10:00:04.140 INFO:teuthology.orchestra.run.smithi133.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T10:00:04.140 INFO:teuthology.orchestra.run.smithi133.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T10:00:04.159 INFO:teuthology.orchestra.run.smithi133.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T10:00:04.160 INFO:teuthology.orchestra.run.smithi133.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T10:00:04.299 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package dump. 2024-04-25T10:00:04.326 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T10:00:04.365 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T10:00:04.565 INFO:teuthology.orchestra.run.smithi133.stdout:Preconfiguring packages ... 2024-04-25T10:00:04.661 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package indent. 2024-04-25T10:00:04.688 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:04.727 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:00:04.744 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 55.7 MB in 2s (22.7 MB/s) 2024-04-25T10:00:05.011 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T10:00:05.016 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:00:05.038 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T10:00:05.054 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T10:00:05.096 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:00:05.137 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:00:05.434 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T10:00:05.461 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T10:00:05.500 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:00:05.702 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T10:00:05.718 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T10:00:05.751 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T10:00:05.761 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:00:06.089 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T10:00:06.117 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:06.156 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:06.408 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T10:00:06.424 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:06.467 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:06.686 INFO:teuthology.orchestra.run.smithi133.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T10:00:06.711 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T10:00:06.727 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T10:00:06.769 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T10:00:06.952 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package flex. 2024-04-25T10:00:07.054 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T10:00:07.060 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:07.063 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T10:00:07.088 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T10:00:07.099 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T10:00:07.130 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:00:07.608 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:00:07.780 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T10:00:08.224 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:08.744 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:00:08.750 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:00:08.775 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T10:00:08.802 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:00:08.841 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:08.956 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:00:09.102 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T10:00:09.130 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:00:09.169 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:09.331 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:00:09.445 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:00:09.538 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:09.781 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:00:09.883 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T10:00:09.910 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T10:00:09.949 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:00:10.120 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package bison. 2024-04-25T10:00:10.190 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:00:10.196 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T10:00:10.239 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:00:10.269 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T10:00:10.296 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:00:10.335 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:00:10.665 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T10:00:10.671 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T10:00:10.693 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T10:00:10.699 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:10.732 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:00:10.738 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:11.058 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T10:00:11.076 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package dump. 2024-04-25T10:00:11.085 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:11.104 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T10:00:11.124 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:11.143 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T10:00:11.397 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package indent. 2024-04-25T10:00:11.413 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:11.427 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T10:00:11.446 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:00:11.454 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:11.493 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:11.724 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T10:00:11.737 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T10:00:11.740 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T10:00:11.753 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:00:11.782 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:00:11.795 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:12.112 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T10:00:12.139 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T10:00:12.178 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:00:12.332 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T10:00:12.360 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T10:00:12.399 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:00:12.422 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T10:00:12.438 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T10:00:12.472 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:00:12.777 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package modemmanager. 2024-04-25T10:00:12.800 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T10:00:12.804 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:00:12.828 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:12.843 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:12.867 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:13.128 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T10:00:13.156 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:13.195 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:13.447 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T10:00:13.458 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package network-manager. 2024-04-25T10:00:13.475 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T10:00:13.486 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:00:13.514 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T10:00:13.533 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:00:13.786 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T10:00:13.811 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T10:00:13.842 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:00:14.124 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ppp. 2024-04-25T10:00:14.152 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T10:00:14.439 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:00:14.745 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T10:00:14.773 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T10:00:14.812 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T10:00:15.064 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T10:00:15.080 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T10:00:15.122 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T10:00:15.409 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T10:00:15.436 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T10:00:15.475 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T10:00:15.537 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T10:00:15.564 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:00:15.603 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:15.779 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T10:00:15.806 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:15.845 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:15.872 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T10:00:15.888 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:00:15.931 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:16.242 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T10:00:16.270 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:00:16.300 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:16.367 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T10:00:16.395 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T10:00:16.443 INFO:teuthology.orchestra.run.smithi162.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T10:00:16.473 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking postgresql-common (238) ... 2024-04-25T10:00:16.645 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T10:00:16.672 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T10:00:16.711 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:00:16.965 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T10:00:16.993 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:17.035 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:17.047 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T10:00:17.075 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:00:17.114 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:00:17.442 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T10:00:17.469 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:17.508 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:17.837 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T10:00:17.864 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:17.903 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:18.205 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T10:00:18.233 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:18.272 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:18.326 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package postgresql. 2024-04-25T10:00:18.355 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T10:00:18.392 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking postgresql (14+238) ... 2024-04-25T10:00:18.517 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T10:00:18.543 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:00:18.583 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:18.636 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T10:00:18.652 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T10:00:18.694 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T10:00:18.938 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T10:00:18.954 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T10:00:18.996 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T10:00:19.274 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T10:00:19.302 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T10:00:19.332 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:00:19.354 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T10:00:19.381 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T10:00:19.412 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:00:19.618 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package tcl. 2024-04-25T10:00:19.646 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T10:00:19.689 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T10:00:19.732 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package modemmanager. 2024-04-25T10:00:19.759 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:00:19.815 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:20.037 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T10:00:20.065 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T10:00:20.104 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:00:20.449 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T10:00:20.466 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T10:00:20.472 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package network-manager. 2024-04-25T10:00:20.500 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:00:20.508 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T10:00:20.546 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:00:20.911 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T10:00:20.939 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T10:00:20.986 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:00:21.195 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ppp. 2024-04-25T10:00:21.224 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T10:00:21.273 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T10:00:21.491 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T10:00:21.494 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:00:21.618 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:00:21.743 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:21.817 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T10:00:21.845 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:00:21.845 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T10:00:21.883 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T10:00:21.945 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:00:22.046 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:22.136 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T10:00:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T10:00:22.489 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T10:00:22.517 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T10:00:22.524 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T10:00:22.547 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T10:00:22.602 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:00:22.834 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T10:00:22.862 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:22.901 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:22.990 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:00:22.990 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:00:23.454 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:00:23.489 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T10:00:23.517 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T10:00:23.572 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:00:23.574 INFO:teuthology.orchestra.run.smithi133.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T10:00:23.603 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-common (238) ... 2024-04-25T10:00:23.790 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:24.181 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T10:00:24.211 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:24.291 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T10:00:24.418 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:00:24.458 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:24.542 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:24.668 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:00:24.769 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:00:25.500 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:00:25.634 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T10:00:25.728 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T10:00:25.785 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:25.911 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:00:26.012 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T10:00:26.113 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T10:00:26.133 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql. 2024-04-25T10:00:26.162 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T10:00:26.208 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql (14+238) ... 2024-04-25T10:00:26.222 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:00:26.313 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T10:00:26.374 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:00:26.477 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T10:00:26.500 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:00:26.505 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T10:00:26.543 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T10:00:26.676 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:00:26.796 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T10:00:26.802 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T10:00:26.862 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T10:00:26.928 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T10:00:27.071 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T10:00:27.172 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:00:27.206 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T10:00:27.234 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T10:00:27.281 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:00:27.567 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package tcl. 2024-04-25T10:00:27.584 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T10:00:27.630 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T10:00:27.636 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:27.636 INFO:teuthology.orchestra.run.smithi162.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T10:00:27.636 INFO:teuthology.orchestra.run.smithi162.stdout:which means they are currently configured by ifupdown: 2024-04-25T10:00:27.636 INFO:teuthology.orchestra.run.smithi162.stdout:- enp3s0f1 2024-04-25T10:00:27.636 INFO:teuthology.orchestra.run.smithi162.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T10:00:27.636 INFO:teuthology.orchestra.run.smithi162.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T10:00:27.636 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:27.765 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T10:00:27.955 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T10:00:27.983 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T10:00:28.022 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:00:28.075 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T10:00:28.393 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T10:00:28.409 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T10:00:28.426 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T10:00:28.476 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T10:00:28.874 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:28.879 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T10:00:28.907 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T10:00:28.954 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:00:29.025 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:29.168 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T10:00:29.241 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T10:00:29.269 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up postgresql-common (238) ... 2024-04-25T10:00:29.443 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T10:00:29.553 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:00:29.662 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:29.830 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:00:29.957 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:00:30.082 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:30.564 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T10:00:30.572 INFO:teuthology.orchestra.run.smithi162.stdout:Adding user postgres to group ssl-cert 2024-04-25T10:00:30.654 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:00:30.795 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:30.795 INFO:teuthology.orchestra.run.smithi162.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T10:00:31.013 INFO:teuthology.orchestra.run.smithi162.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T10:00:31.013 INFO:teuthology.orchestra.run.smithi162.stdout:Removing obsolete dictionary files: 2024-04-25T10:00:31.033 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:00:31.033 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:00:31.545 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T10:00:31.556 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:00:31.682 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:00:31.809 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:31.928 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:31.934 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T10:00:32.044 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:00:32.161 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:32.241 INFO:teuthology.orchestra.run.smithi162.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T10:00:32.258 INFO:teuthology.orchestra.run.smithi162.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T10:00:32.271 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:00:32.306 INFO:teuthology.orchestra.run.smithi162.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T10:00:32.306 INFO:teuthology.orchestra.run.smithi162.stdout:This user must also own the server process. 2024-04-25T10:00:32.306 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:32.306 INFO:teuthology.orchestra.run.smithi162.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T10:00:32.306 INFO:teuthology.orchestra.run.smithi162.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T10:00:32.306 INFO:teuthology.orchestra.run.smithi162.stdout:The default text search configuration will be set to "english". 2024-04-25T10:00:32.306 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:32.306 INFO:teuthology.orchestra.run.smithi162.stdout:Data page checksums are disabled. 2024-04-25T10:00:32.307 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:32.307 INFO:teuthology.orchestra.run.smithi162.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T10:00:32.307 INFO:teuthology.orchestra.run.smithi162.stdout:creating subdirectories ... ok 2024-04-25T10:00:32.307 INFO:teuthology.orchestra.run.smithi162.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T10:00:32.318 INFO:teuthology.orchestra.run.smithi162.stdout:selecting default max_connections ... 100 2024-04-25T10:00:32.340 INFO:teuthology.orchestra.run.smithi162.stdout:selecting default shared_buffers ... 128MB 2024-04-25T10:00:32.349 INFO:teuthology.orchestra.run.smithi162.stdout:selecting default time zone ... Etc/UTC 2024-04-25T10:00:32.350 INFO:teuthology.orchestra.run.smithi162.stdout:creating configuration files ... ok 2024-04-25T10:00:32.398 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:00:32.465 INFO:teuthology.orchestra.run.smithi162.stdout:running bootstrap script ... ok 2024-04-25T10:00:33.043 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:00:33.152 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T10:00:33.229 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T10:00:33.287 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:33.290 INFO:teuthology.orchestra.run.smithi162.stdout:performing post-bootstrap initialization ... ok 2024-04-25T10:00:33.438 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:00:33.564 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T10:00:33.690 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T10:00:33.816 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:00:33.908 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T10:00:33.967 INFO:teuthology.orchestra.run.smithi162.stdout:syncing data to disk ... ok 2024-04-25T10:00:33.976 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:00:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:00:34.354 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:34.389 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T10:00:34.472 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T10:00:34.642 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T10:00:34.766 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:00:35.611 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:35.611 INFO:teuthology.orchestra.run.smithi133.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T10:00:35.611 INFO:teuthology.orchestra.run.smithi133.stdout:which means they are currently configured by ifupdown: 2024-04-25T10:00:35.611 INFO:teuthology.orchestra.run.smithi133.stdout:- enp3s0f1 2024-04-25T10:00:35.611 INFO:teuthology.orchestra.run.smithi133.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T10:00:35.611 INFO:teuthology.orchestra.run.smithi133.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T10:00:35.611 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:35.734 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T10:00:36.065 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T10:00:36.396 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T10:00:36.924 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:37.070 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:37.159 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:37.204 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T10:00:37.276 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T10:00:37.330 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-common (238) ... 2024-04-25T10:00:37.377 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up postgresql (14+238) ... 2024-04-25T10:00:37.487 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:37.612 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:37.889 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:00:37.890 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:00:38.262 INFO:teuthology.orchestra.run.smithi133.stdout:Adding user postgres to group ssl-cert 2024-04-25T10:00:38.482 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:00:38.494 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:38.494 INFO:teuthology.orchestra.run.smithi133.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T10:00:38.574 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T10:00:38.699 INFO:teuthology.orchestra.run.smithi133.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T10:00:38.699 INFO:teuthology.orchestra.run.smithi133.stdout:Removing obsolete dictionary files: 2024-04-25T10:00:38.909 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:00:39.062 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:00:39.253 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T10:00:39.649 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:39.953 INFO:teuthology.orchestra.run.smithi133.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T10:00:39.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T10:00:40.025 INFO:teuthology.orchestra.run.smithi133.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi133.stdout:This user must also own the server process. 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi133.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi133.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi133.stdout:The default text search configuration will be set to "english". 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi133.stdout:Data page checksums are disabled. 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi133.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi133.stdout:creating subdirectories ... ok 2024-04-25T10:00:40.026 INFO:teuthology.orchestra.run.smithi133.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T10:00:40.037 INFO:teuthology.orchestra.run.smithi133.stdout:selecting default max_connections ... 100 2024-04-25T10:00:40.058 INFO:teuthology.orchestra.run.smithi133.stdout:selecting default shared_buffers ... 128MB 2024-04-25T10:00:40.065 INFO:teuthology.orchestra.run.smithi133.stdout:selecting default time zone ... Etc/UTC 2024-04-25T10:00:40.065 INFO:teuthology.orchestra.run.smithi133.stdout:creating configuration files ... ok 2024-04-25T10:00:40.175 INFO:teuthology.orchestra.run.smithi133.stdout:running bootstrap script ... ok 2024-04-25T10:00:40.347 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T10:00:40.439 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:00:40.908 INFO:teuthology.orchestra.run.smithi162.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:00:40.914 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:41.189 INFO:teuthology.orchestra.run.smithi133.stdout:performing post-bootstrap initialization ... ok 2024-04-25T10:00:41.891 INFO:teuthology.orchestra.run.smithi133.stdout:syncing data to disk ... ok 2024-04-25T10:00:42.301 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T10:00:45.085 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:45.211 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T10:00:45.337 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up postgresql (14+238) ... 2024-04-25T10:00:45.438 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:45.539 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:45.785 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:00:45.785 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:00:46.350 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:00:46.451 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T10:00:46.802 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:00:46.946 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:00:48.172 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T10:00:48.272 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:00:48.773 INFO:teuthology.orchestra.run.smithi133.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:00:48.779 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:48.779 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:00:48.926 DEBUG:teuthology.orchestra.run.smithi133:> dpkg-query -W -f '${Version}' ceph 2024-04-25T10:00:48.945 INFO:teuthology.orchestra.run.smithi133.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:00:48.945 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:00:48.945 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T10:00:48.947 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:00:49.088 DEBUG:teuthology.orchestra.run.smithi162:> dpkg-query -W -f '${Version}' ceph 2024-04-25T10:00:49.107 INFO:teuthology.orchestra.run.smithi162.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:00:49.107 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:00:49.108 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T10:00:49.109 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:00:49.110 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:00:49.110 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:00:49.124 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:00:49.125 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:00:49.163 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:00:49.164 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:00:49.164 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:00:49.177 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:00:49.234 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:00:49.234 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:00:49.247 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:00:49.304 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:00:49.304 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:00:49.305 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:00:49.318 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:00:49.373 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:00:49.373 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:00:49.524 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:00:49.578 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:00:49.578 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:00:49.578 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:00:49.595 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:00:49.649 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:00:49.649 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:00:49.663 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:00:49.719 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:00:49.808 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:00:49.808 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /var/log/ceph 2024-04-25T10:00:49.811 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /var/log/ceph 2024-04-25T10:00:49.823 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:00:49.824 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:00:49.867 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:00:49.879 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:00:49.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:00:49.919 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:00:49.941 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:00:49.941 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:00:49.941 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:00:49.941 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:00:49.942 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:00:49.974 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:50.050 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:00:50.050 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:00:50.056 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:50.135 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:00:50.136 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:00:50.136 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673035', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi133.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBvOoOfT9zvd5ASkAErg5geX1soqO2/oh9Yql8zG4TL9vqi9JKIDPoR33AhWUxThmNpcv3hkkE9MRHBjciSIrpU=', 'smithi162.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGUtMffjk06lbdABLHgo03pOnX557+8j4SiyfoWJprl3cf2jFaCcZFh3fzVLMJyzxNUW0S25plwfekcykzmSF0k='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:00:50.137 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:00:50.139 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:00:50.179 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:00:50.187 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:00:50.235 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:00:50.235 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:00:50.246 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:00:50.246 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-04-25T10:00:50.294 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:00:50.294 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:50.294 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 879 Links: 1 2024-04-25T10:00:50.294 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:50.294 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-25 09:54:42.643253148 +0000 2024-04-25T10:00:50.294 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-25 09:54:42.391255395 +0000 2024-04-25T10:00:50.295 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-25 09:54:42.391255395 +0000 2024-04-25T10:00:50.295 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-04-25T10:00:50.295 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:00:50.348 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:00:50.349 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:00:50.349 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000448837 s, 1.1 MB/s 2024-04-25T10:00:50.350 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:00:50.399 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-04-25T10:00:50.446 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:00:50.447 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:50.447 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 902 Links: 1 2024-04-25T10:00:50.447 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:50.447 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-25 09:54:43.139248723 +0000 2024-04-25T10:00:50.447 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-25 09:54:42.883251007 +0000 2024-04-25T10:00:50.447 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-25 09:54:42.883251007 +0000 2024-04-25T10:00:50.447 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-04-25T10:00:50.447 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:00:50.500 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:00:50.500 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:00:50.500 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000335079 s, 1.5 MB/s 2024-04-25T10:00:50.501 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:00:50.551 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-04-25T10:00:50.599 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:00:50.599 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:50.599 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-04-25T10:00:50.599 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:50.599 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-25 09:54:43.611244512 +0000 2024-04-25T10:00:50.599 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-25 09:54:43.375246618 +0000 2024-04-25T10:00:50.599 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-25 09:54:43.375246618 +0000 2024-04-25T10:00:50.599 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-04-25T10:00:50.599 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:00:50.652 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:00:50.652 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:00:50.652 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000412554 s, 1.2 MB/s 2024-04-25T10:00:50.653 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:00:50.703 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-04-25T10:00:50.750 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:00:50.750 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:50.750 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 936 Links: 1 2024-04-25T10:00:50.750 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:50.751 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-25 09:54:44.123239945 +0000 2024-04-25T10:00:50.751 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-25 09:54:43.859242300 +0000 2024-04-25T10:00:50.751 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-25 09:54:43.859242300 +0000 2024-04-25T10:00:50.751 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-04-25T10:00:50.751 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:00:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:00:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:00:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000380327 s, 1.3 MB/s 2024-04-25T10:00:50.806 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:00:50.855 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:50.856 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:00:50.856 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:00:50.861 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:00:50.862 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2024-04-25T10:00:50.911 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:00:50.912 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:50.912 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 883 Links: 1 2024-04-25T10:00:50.912 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:50.912 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-25 09:54:43.892571622 +0000 2024-04-25T10:00:50.912 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-25 09:54:43.640573873 +0000 2024-04-25T10:00:50.912 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-25 09:54:43.640573873 +0000 2024-04-25T10:00:50.912 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2024-04-25T10:00:50.912 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:00:50.966 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-25T10:00:50.966 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-25T10:00:50.966 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.00045341 s, 1.1 MB/s 2024-04-25T10:00:50.967 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:00:51.017 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2024-04-25T10:00:51.063 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:00:51.063 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:51.063 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 897 Links: 1 2024-04-25T10:00:51.064 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:51.064 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-25 09:54:44.388567192 +0000 2024-04-25T10:00:51.064 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-25 09:54:44.128569514 +0000 2024-04-25T10:00:51.064 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-25 09:54:44.128569514 +0000 2024-04-25T10:00:51.064 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2024-04-25T10:00:51.064 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:00:51.118 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-25T10:00:51.118 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-25T10:00:51.118 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000455857 s, 1.1 MB/s 2024-04-25T10:00:51.119 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:00:51.168 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2024-04-25T10:00:51.215 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:00:51.215 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:51.215 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 914 Links: 1 2024-04-25T10:00:51.216 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:51.216 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-25 09:54:44.880562797 +0000 2024-04-25T10:00:51.216 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-25 09:54:44.620565120 +0000 2024-04-25T10:00:51.216 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-25 09:54:44.620565120 +0000 2024-04-25T10:00:51.216 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2024-04-25T10:00:51.216 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:00:51.269 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-25T10:00:51.282 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-25T10:00:51.282 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000307606 s, 1.7 MB/s 2024-04-25T10:00:51.283 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:00:51.333 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2024-04-25T10:00:51.383 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:00:51.384 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:51.384 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 929 Links: 1 2024-04-25T10:00:51.384 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:51.384 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-04-25 09:54:45.348558617 +0000 2024-04-25T10:00:51.384 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-04-25 09:54:45.112560724 +0000 2024-04-25T10:00:51.384 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-04-25 09:54:45.112560724 +0000 2024-04-25T10:00:51.384 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2024-04-25T10:00:51.384 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:00:51.438 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-04-25T10:00:51.438 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-04-25T10:00:51.438 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000410644 s, 1.2 MB/s 2024-04-25T10:00:51.440 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:00:51.488 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:51.489 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:00:51.489 INFO:tasks.ceph:Generating config... 2024-04-25T10:00:51.490 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:00:51.490 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:00:51.490 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:00:51.490 INFO:tasks.ceph:[client] pid file = 2024-04-25T10:00:51.490 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:00:51.491 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:00:51.491 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:00:51.492 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:00:51.492 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:00:51.492 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:00:51.492 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:00:51.492 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:00:51.492 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:00:51.492 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:00:51.492 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T10:00:51.492 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T10:00:51.492 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:00:51.492 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:00:51.492 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T10:00:51.492 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:00:51.492 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:00:51.493 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:00:51.493 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:00:51.493 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:00:51.493 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:00:51.493 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:00:51.493 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:00:51.526 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:00:51.547 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:00:51.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:00:51.682 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.133'), ('mon.b', '172.21.15.162'), ('mon.c', '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]')] 2024-04-25T10:00:51.682 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.133,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:00:51.683 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:00:51.683 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:00:51.730 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.133 --add b 172.21.15.162 --addv c '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: generated fsid 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:00:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:setting min_mon_release = quincy 2024-04-25T10:00:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 0 2024-04-25T10:00:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:fsid 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:00:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:last_changed 2024-04-25T10:00:51.813289+0000 2024-04-25T10:00:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:created 2024-04-25T10:00:51.813289+0000 2024-04-25T10:00:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:min_mon_release 17 (quincy) 2024-04-25T10:00:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:election_strategy: 1 2024-04-25T10:00:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-04-25T10:00:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.b 2024-04-25T10:00:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:2: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.c 2024-04-25T10:00:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:00:51.818 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:00:51.867 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 599cdb88-206d-4299-bbcd-9f7da90fb181... 2024-04-25T10:00:51.868 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:00:51.911 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:00:51.940 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-04-25T10:00:51.940 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-04-25T10:00:51.940 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:00:51.940 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-04-25T10:00:51.940 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.940 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-04-25T10:00:51.940 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.941 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:00:51.942 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.133,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-04-25T10:00:51.943 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.944 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 20 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 20 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:00:51.945 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 10 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-25T10:00:51.946 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:51.947 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.948 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:00:51.948 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:00:51.948 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:00:51.948 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.948 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:00:51.948 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:00:51.948 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:00:51.948 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.948 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-04-25T10:00:51.948 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-04-25T10:00:51.949 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:00:51.950 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-04-25T10:00:51.950 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-04-25T10:00:51.950 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-04-25T10:00:51.950 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-04-25T10:00:51.950 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-04-25T10:00:51.950 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-04-25T10:00:51.950 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-04-25T10:00:51.950 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-04-25T10:00:51.950 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-04-25T10:00:51.950 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-04-25T10:00:51.950 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: mds cache max size = 1000 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = True 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: mds log max segments = 1 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = True 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-04-25T10:00:51.951 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-04-25T10:00:51.953 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-04-25T10:00:51.953 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2024-04-25T10:00:51.954 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2024-04-25T10:00:51.955 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.955 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:00:51.955 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2024-04-25T10:00:51.955 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.955 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:00:51.955 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:00:51.955 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.955 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2024-04-25T10:00:51.955 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.955 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2024-04-25T10:00:51.955 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.956 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.133,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.957 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2024-04-25T10:00:51.958 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2024-04-25T10:00:51.958 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.958 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2024-04-25T10:00:51.958 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2024-04-25T10:00:51.958 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2024-04-25T10:00:51.958 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug trim objects = true 2024-04-25T10:00:51.958 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.958 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2024-04-25T10:00:51.958 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2024-04-25T10:00:51.958 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:00:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2024-04-25T10:00:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2024-04-25T10:00:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:00:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2024-04-25T10:00:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.959 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2024-04-25T10:00:51.960 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2024-04-25T10:00:51.960 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2024-04-25T10:00:51.960 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2024-04-25T10:00:51.960 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2024-04-25T10:00:51.960 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2024-04-25T10:00:51.960 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:00:51.960 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2024-04-25T10:00:51.960 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2024-04-25T10:00:51.960 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:00:51.960 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2024-04-25T10:00:51.960 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:00:51.961 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.962 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:00:51.963 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = "" 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2024-04-25T10:00:51.964 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds balancer = 20 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout: mds cache max size = 1000 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = True 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout: mds log max segments = 1 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = True 2024-04-25T10:00:51.965 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-04-25T10:00:51.966 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-04-25T10:00:51.966 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-04-25T10:00:51.966 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:00:51.966 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:00:52.046 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:00:52.046 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:00:52.047 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:00:52.052 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:00:52.052 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:00:52.098 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:00:52.099 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:00:52.099 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:00:52.099 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:00:52.162 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:00:52.162 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:52.210 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:00:52.210 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:00:52.210 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:00:52.210 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:00:52.234 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:00:52.234 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:52.279 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:00:52.280 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:00:52.280 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:00:52.323 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:00:52.326 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:00:52.369 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:00:52.382 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:00:52.382 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:00:52.421 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:00:52.424 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:00:52.477 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:00:52.560 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:00:52.562 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:00:52.617 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:00:52.664 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:00:52.666 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:00:52.679 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:00:52.760 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:00:52.762 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:00:52.818 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:00:52.818 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:00:52.855 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:00:52.867 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T10:00:52.941 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T10:00:52.951 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T10:00:52.987 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T10:00:52.997 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T10:00:53.073 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T10:00:53.084 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:00:53.084 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:00:53.084 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:00:53.098 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:53.098 INFO:tasks.ceph:role: osd.0 2024-04-25T10:00:53.098 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:00:53.098 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:00:53.164 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:00:53.164 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:00:53.164 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:00:53.164 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:00:53.164 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:00:53.164 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:00:53.164 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:00:53.165 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:00:53.165 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:00:53.165 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:00:53.626 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-04-25T10:00:53.628 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-04-25T10:00:53.629 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:00:53.709 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:00:53.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:00:53.763 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:00:53.763 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:00:53.817 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:53.817 INFO:tasks.ceph:role: osd.1 2024-04-25T10:00:53.817 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:00:53.817 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:00:53.882 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:00:53.882 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:00:53.882 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:00:53.882 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:00:53.882 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:00:53.882 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:00:53.882 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:00:53.882 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:00:53.882 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:00:53.883 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:00:54.349 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-04-25T10:00:54.350 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-04-25T10:00:54.350 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:00:54.512 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:00:54.523 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:00:54.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:00:54.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:00:54.582 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:54.582 INFO:tasks.ceph:role: osd.2 2024-04-25T10:00:54.582 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:00:54.583 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:00:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:00:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:00:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:00:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:00:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:00:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:00:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:00:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:00:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:00:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:00:55.107 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-04-25T10:00:55.108 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-04-25T10:00:55.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:00:55.249 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:00:55.261 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:00:55.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:00:55.262 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:00:55.317 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:55.317 INFO:tasks.ceph:role: osd.3 2024-04-25T10:00:55.317 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:00:55.317 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:00:55.382 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:00:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:00:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:00:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:00:55.382 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:00:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:00:55.382 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:00:55.382 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:00:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:00:55.383 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:00:55.801 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-04-25T10:00:55.803 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-04-25T10:00:55.803 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:00:55.895 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:00:55.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:00:55.906 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:00:55.907 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:56.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:56.013+0000 7faf7e8587c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:00:56.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:56.013+0000 7faf7e8587c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:00:56.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:56.013+0000 7faf7e8587c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:00:56.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:56.013+0000 7faf7e8587c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:00:56.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:56.013+0000 7faf7e8587c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:00:56.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:56.017+0000 7faf7e8587c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:00:59.014 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:00:59.028 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:59.101+0000 7f23509347c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:00:59.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:59.101+0000 7f23509347c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:00:59.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:59.101+0000 7f23509347c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:00:59.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:59.101+0000 7f23509347c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:00:59.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:59.101+0000 7f23509347c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:00:59.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:00:59.101+0000 7f23509347c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:01:02.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:01:02.146 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:02.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:02.225+0000 7f37703587c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:01:02.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:02.225+0000 7f37703587c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:01:02.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:02.225+0000 7f37703587c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:02.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:02.225+0000 7f37703587c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:02.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:02.225+0000 7f37703587c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:02.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:02.229+0000 7f37703587c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:01:05.212 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:01:05.267 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:05.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:05.345+0000 7fe3eba957c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:01:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:05.345+0000 7fe3eba957c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:01:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:05.345+0000 7fe3eba957c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:05.345+0000 7fe3eba957c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:05.345+0000 7fe3eba957c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:05.345+0000 7fe3eba957c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:01:08.333 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:01:08.391 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:08.405 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:08.405 INFO:tasks.ceph:role: osd.4 2024-04-25T10:01:08.405 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:01:08.405 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:01:08.469 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:08.470 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:08.470 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:08.470 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:08.470 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:08.470 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:08.470 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:08.470 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:08.470 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:08.470 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:08.956 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-04-25T10:01:08.958 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-04-25T10:01:08.958 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:09.104 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:09.114 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:09.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:09.115 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:09.175 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:09.175 INFO:tasks.ceph:role: osd.5 2024-04-25T10:01:09.175 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:01:09.175 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:01:09.239 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:09.239 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:09.239 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:09.239 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:09.239 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:09.239 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:09.239 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:09.240 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:09.240 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:09.240 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:09.681 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-04-25T10:01:09.683 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-04-25T10:01:09.683 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:09.776 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:09.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:09.788 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:09.789 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:09.842 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:09.843 INFO:tasks.ceph:role: osd.6 2024-04-25T10:01:09.843 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:01:09.843 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:09.907 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:10.362 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-04-25T10:01:10.364 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-04-25T10:01:10.364 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:10.476 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:10.486 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:10.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:10.487 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:10.542 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:10.543 INFO:tasks.ceph:role: osd.7 2024-04-25T10:01:10.543 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:01:10.543 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:01:10.607 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:10.607 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:10.607 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:10.607 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:10.607 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:10.607 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:10.607 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:10.608 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:10.608 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:10.608 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:11.104 INFO:teuthology.orchestra.run.smithi162.stdout:Discarding blocks...Done. 2024-04-25T10:01:11.107 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-04-25T10:01:11.107 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:11.245 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:11.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:11.256 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:11.256 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:11.663 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:11.659+0000 7fbbe93197c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:01:11.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:11.659+0000 7fbbe93197c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:01:11.666 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:11.659+0000 7fbbe93197c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:11.666 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:11.659+0000 7fbbe93197c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:11.666 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:11.659+0000 7fbbe93197c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:11.666 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:11.659+0000 7fbbe93197c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:01:14.638 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:14.652 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:14.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:14.727+0000 7f98f0fbb7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:01:14.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:14.727+0000 7f98f0fbb7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:01:14.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:14.727+0000 7f98f0fbb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:14.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:14.727+0000 7f98f0fbb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:14.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:14.727+0000 7f98f0fbb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:14.733 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:14.727+0000 7f98f0fbb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:01:17.705 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:17.759 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:17.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:17.835+0000 7ff165e077c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:01:17.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:17.835+0000 7ff165e077c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:01:17.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:17.835+0000 7ff165e077c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:17.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:17.835+0000 7ff165e077c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:17.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:17.835+0000 7ff165e077c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:17.842 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:17.835+0000 7ff165e077c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:01:20.139 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:20.142 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:20.846 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:20.860 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:20.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:20.935+0000 7f887de0d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:01:20.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:20.935+0000 7f887de0d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:01:20.941 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:20.935+0000 7f887de0d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:20.941 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:20.935+0000 7f887de0d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:20.941 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:20.935+0000 7f887de0d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:20.941 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:20.935+0000 7f887de0d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:01:23.914 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:23.970 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:01:23.970 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:23.970 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:01:23.984 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:23.984 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:01:24.037 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:24.037 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:01:24.092 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:24.092 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:01:24.145 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:24.145 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:01:24.201 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:24.201 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:01:24.256 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:24.256 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:01:24.309 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:24.309 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:01:24.322 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:24.322 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:01:24.378 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:24.378 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:01:24.433 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:24.433 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:01:24.486 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:24.486 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:01:24.541 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:24.541 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:01:24.594 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:24.594 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:01:24.645 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:24.645 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:01:24.651 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:24.651 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T10:01:24.698 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:24.698 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T10:01:24.703 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:24.703 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T10:01:24.751 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:01:24.751 DEBUG:teuthology.orchestra.run.smithi133:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:01:24.753 DEBUG:teuthology.orchestra.run.smithi162:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:01:24.797 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.y] 2024-04-25T10:01:24.797 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg== 2024-04-25T10:01:24.797 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-04-25T10:01:24.797 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg== 2024-04-25T10:01:24.797 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-04-25T10:01:24.797 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw== 2024-04-25T10:01:24.797 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.0] 2024-04-25T10:01:24.797 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA== 2024-04-25T10:01:24.797 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.1] 2024-04-25T10:01:24.797 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ== 2024-04-25T10:01:24.798 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.2] 2024-04-25T10:01:24.798 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA== 2024-04-25T10:01:24.798 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.3] 2024-04-25T10:01:24.798 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA== 2024-04-25T10:01:24.798 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.x] 2024-04-25T10:01:24.798 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA== 2024-04-25T10:01:24.798 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-04-25T10:01:24.798 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug== 2024-04-25T10:01:24.798 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-04-25T10:01:24.798 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g== 2024-04-25T10:01:24.798 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.4] 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg== 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.5] 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg== 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.6] 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw== 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.7] 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A== 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout:[client.2] 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw== 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout:[client.1] 2024-04-25T10:01:24.799 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg== 2024-04-25T10:01:24.800 INFO:teuthology.orchestra.run.smithi133.stdout:[client.3] 2024-04-25T10:01:24.800 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew== 2024-04-25T10:01:24.804 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.y] 2024-04-25T10:01:24.804 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg== 2024-04-25T10:01:24.804 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg== 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw== 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.0] 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA== 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.1] 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ== 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.2] 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA== 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.3] 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA== 2024-04-25T10:01:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.x] 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA== 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug== 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g== 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.4] 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg== 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.5] 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg== 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.6] 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw== 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.7] 2024-04-25T10:01:24.806 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A== 2024-04-25T10:01:24.807 INFO:teuthology.orchestra.run.smithi162.stdout:[client.0] 2024-04-25T10:01:24.807 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:01:24.807 INFO:teuthology.orchestra.run.smithi162.stdout:[client.2] 2024-04-25T10:01:24.807 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw== 2024-04-25T10:01:24.807 INFO:teuthology.orchestra.run.smithi162.stdout:[client.1] 2024-04-25T10:01:24.807 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg== 2024-04-25T10:01:24.807 INFO:teuthology.orchestra.run.smithi162.stdout:[client.3] 2024-04-25T10:01:24.807 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew== 2024-04-25T10:01:24.808 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:24.843 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:24.894 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:24.897 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:24.982 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:24.984 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:25.065 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.075 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.145 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.148 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.227 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.232 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.312 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.314 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.399 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:25.402 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:25.440 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:25.483 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:25.525 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:25.567 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:25.613 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.616 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.688 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.699 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.768 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.783 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.852 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.870 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:25.933 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:25.958 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:26.017 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:26.043 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:26.100 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:26.103 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:26.185 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:26.191 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:26.273 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:01:26.273 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:01:26.284 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:26.374 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:01:26.384 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:01:26.400 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:26.491 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:01:26.501 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:01:26.552 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:26.643 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:01:26.653 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:26.656 DEBUG:teuthology.orchestra.run.smithi162:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:26.698 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:01:26.698 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:01:26.699 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:01:26.703 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:01:26.703 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:01:26.703 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:01:26.743 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:01:26.744 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:01:26.744 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:01:26.746 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:01:26.746 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:01:26.746 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:01:26.746 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:01:26.749 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:01:26.749 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:01:26.749 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:01:26.751 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:01:26.751 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:01:26.751 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:01:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcdcac1640 1 Processor -- start 2024-04-25T10:01:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcdcac1640 1 -- start start 2024-04-25T10:01:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcdcac1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 0x7fbcd80cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcdcac1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcdcac1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 0x7fbcd80cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcdcac1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd80cd690 con 0x7fbcd80d21c0 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcdcac1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd80ccdf0 con 0x7fbcd815f020 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcdcac1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd80cd0d0 con 0x7fbcd80cfcf0 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50984/0 (socket says 172.21.15.133:50984) 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd5d9b640 1 -- 172.21.15.133:0/3270618499 learned_addr learned my addr 172.21.15.133:0/3270618499 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd5d9b640 1 -- 172.21.15.133:0/3270618499 >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 msgr2=0x7fbcd80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd5d9b640 1 -- 172.21.15.133:0/3270618499 >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 msgr2=0x7fbcd80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd5d9b640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd5d9b640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd6d9d640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 0x7fbcd80cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd659c640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 0x7fbcd80cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd6d9d640 1 -- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 msgr2=0x7fbcd80cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:01:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd6d9d640 1 -- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 msgr2=0x7fbcd80cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:01:26.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd6d9d640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 0x7fbcd80cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:01:26.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd6d9d640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 0x7fbcd80cdf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:01:26.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd659c640 1 -- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 msgr2=0x7fbcd80cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:01:26.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd659c640 1 -- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 msgr2=0x7fbcd80cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:01:26.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd659c640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 0x7fbcd80cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:01:26.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:26.861+0000 7fbcd659c640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 0x7fbcd80cf270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:01:27.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd5d9b640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd659c640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 0x7fbcd80cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd6d9d640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 0x7fbcd80cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd659c640 1 -- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 msgr2=0x7fbcd80cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd659c640 1 -- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 msgr2=0x7fbcd80cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd659c640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 0x7fbcd80cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd5d9b640 1 -- 172.21.15.133:0/3270618499 >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 msgr2=0x7fbcd80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd5d9b640 1 -- 172.21.15.133:0/3270618499 >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 msgr2=0x7fbcd80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd5d9b640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd659c640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 0x7fbcd80cf270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd6d9d640 1 -- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 msgr2=0x7fbcd80cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd6d9d640 1 -- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 msgr2=0x7fbcd80cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd6d9d640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 0x7fbcd80cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:01:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd5d9b640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:01:27.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.065+0000 7fbcd6d9d640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 0x7fbcd80cdf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:01:27.351 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:27.347+0000 7f60cbf65180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:01:27.370 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:27.365+0000 7f16e84e3180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:01:27.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.469+0000 7fbcd5d9b640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.133:3300/0 conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.469+0000 7fbcd659c640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 0x7fbcd80cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.469+0000 7fbcd6d9d640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 0x7fbcd80cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.469+0000 7fbcd5d9b640 1 -- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 msgr2=0x7fbcd80cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:27.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.469+0000 7fbcd5d9b640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 0x7fbcd80cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.469+0000 7fbcd5d9b640 1 -- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 msgr2=0x7fbcd80cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:27.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.469+0000 7fbcd5d9b640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 0x7fbcd80cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.469+0000 7fbcd5d9b640 1 -- 172.21.15.133:0/3270618499 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd80cd3b0 con 0x7fbcd80d21c0 2024-04-25T10:01:27.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.469+0000 7fbcd659c640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 0x7fbcd80cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:27.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcd5d9b640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbcd808bdf0 tx=0x7fbccc01cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9594169d0463455b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcd559a640 1 -- 172.21.15.133:0/3270618499 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc0ab070 con 0x7fbcd80d21c0 2024-04-25T10:01:27.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcd559a640 1 -- 172.21.15.133:0/3270618499 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc0a59b0 con 0x7fbcd80d21c0 2024-04-25T10:01:27.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcd559a640 1 -- 172.21.15.133:0/3270618499 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00f040 con 0x7fbcd80d21c0 2024-04-25T10:01:27.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3270618499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd80d21c0 msgr2=0x7fbcd80cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcdcac1640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbcd808bdf0 tx=0x7fbccc01cb10 comp rx=0 tx=0).stop 2024-04-25T10:01:27.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3270618499 shutdown_connections 2024-04-25T10:01:27.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcdcac1640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd80cfcf0 0x7fbcd80cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcdcac1640 1 --2- 172.21.15.133:0/3270618499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd80d21c0 0x7fbcd80cf7b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcdcac1640 1 --2- 172.21.15.133:0/3270618499 >> v2:172.21.15.162:3300/0 conn(0x7fbcd815f020 0x7fbcd80cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3270618499 >> 172.21.15.133:0/3270618499 conn(0x7fbcd80d16a0 msgr2=0x7fbcd80d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3270618499 shutdown_connections 2024-04-25T10:01:27.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.473+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3270618499 wait complete. 2024-04-25T10:01:27.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcdcac1640 1 Processor -- start 2024-04-25T10:01:27.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcdcac1640 1 -- start start 2024-04-25T10:01:27.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcdcac1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd80cfcf0 0x7fbcd80c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcdcac1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd80d21c0 0x7fbcd814fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcdcac1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd815f020 0x7fbcd81f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcdcac1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd80ceef0 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcdcac1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd80d0100 con 0x7fbcd80d21c0 2024-04-25T10:01:27.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcdcac1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd806f8f0 con 0x7fbcd815f020 2024-04-25T10:01:27.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd80cfcf0 0x7fbcd80c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd80cfcf0 0x7fbcd80c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51032/0 (socket says 172.21.15.133:51032) 2024-04-25T10:01:27.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd659c640 1 -- 172.21.15.133:0/3011357091 learned_addr learned my addr 172.21.15.133:0/3011357091 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:27.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd5d9b640 1 --2- 172.21.15.133:0/3011357091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd80d21c0 0x7fbcd814fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd6d9d640 1 --2- 172.21.15.133:0/3011357091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd815f020 0x7fbcd81f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd659c640 1 -- 172.21.15.133:0/3011357091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd815f020 msgr2=0x7fbcd81f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd659c640 1 --2- 172.21.15.133:0/3011357091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd815f020 0x7fbcd81f78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd659c640 1 -- 172.21.15.133:0/3011357091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd80d21c0 msgr2=0x7fbcd814fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd659c640 1 --2- 172.21.15.133:0/3011357091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd80d21c0 0x7fbcd814fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd659c640 1 -- 172.21.15.133:0/3011357091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd80c77f0 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd5d9b640 1 --2- 172.21.15.133:0/3011357091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd80d21c0 0x7fbcd814fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:27.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcd659c640 1 --2- 172.21.15.133:0/3011357091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd80cfcf0 0x7fbcd80c6c30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8012910 tx=0x7fbcc8012dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcc6ffd640 1 -- 172.21.15.133:0/3011357091 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8019030 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcc6ffd640 1 -- 172.21.15.133:0/3011357091 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc800c9a0 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3011357091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd80c7ad0 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcc6ffd640 1 -- 172.21.15.133:0/3011357091 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc800f040 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3011357091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd80c7170 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcc6ffd640 1 -- 172.21.15.133:0/3011357091 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbcc800cd50 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcc6ffd640 1 -- 172.21.15.133:0/3011357091 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbcc8015d50 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.477+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3011357091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd8067c70 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.481+0000 7fbcc6ffd640 1 -- 172.21.15.133:0/3011357091 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbcc80153f0 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.545+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3011357091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbcd8155ed0 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.545+0000 7fbcc6ffd640 1 -- 172.21.15.133:0/3011357091 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbcc801e980 con 0x7fbcd80cfcf0 2024-04-25T10:01:27.553 INFO:teuthology.orchestra.run.smithi133.stderr:adjusted tunables profile to default 2024-04-25T10:01:27.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.549+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3011357091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd80cfcf0 msgr2=0x7fbcd80c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.549+0000 7fbcdcac1640 1 --2- 172.21.15.133:0/3011357091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd80cfcf0 0x7fbcd80c6c30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8012910 tx=0x7fbcc8012dd0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.549+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3011357091 shutdown_connections 2024-04-25T10:01:27.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.549+0000 7fbcdcac1640 1 --2- 172.21.15.133:0/3011357091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd815f020 0x7fbcd81f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.549+0000 7fbcdcac1640 1 --2- 172.21.15.133:0/3011357091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd80d21c0 0x7fbcd814fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.549+0000 7fbcdcac1640 1 --2- 172.21.15.133:0/3011357091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd80cfcf0 0x7fbcd80c6c30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.549+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3011357091 >> 172.21.15.133:0/3011357091 conn(0x7fbcd80d16a0 msgr2=0x7fbcd80af890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.549+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3011357091 shutdown_connections 2024-04-25T10:01:27.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:27.549+0000 7fbcdcac1640 1 -- 172.21.15.133:0/3011357091 wait complete. 2024-04-25T10:01:27.571 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:01:27.571 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:01:27.572 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:27.572 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:01:27.581 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:27.581 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:01:27.637 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:27.637 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:01:27.688 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:01:27.688 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:01:27.741 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:27.741 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:01:27.755 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:27.755 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:01:27.795 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:27.791+0000 7f60cbf65180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:01:27.807 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:27.807 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:01:27.858 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:27.853+0000 7f16e84e3180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:01:27.861 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:01:27.861 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:01:27.913 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new b6489a91-3b21-46bb-9548-89484d0e1f11 0 2024-04-25T10:01:27.982 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:27.975+0000 7f60cbf65180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:01:28.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84ac961640 1 Processor -- start 2024-04-25T10:01:28.067 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84ac961640 1 -- start start 2024-04-25T10:01:28.067 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:28.061+0000 7f16e84e3180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:01:28.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84ac961640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f84a80cc330 0x7f84a80ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84ac961640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a80afe00 con 0x7f84a80cc790 2024-04-25T10:01:28.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84ac961640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a80b00e0 con 0x7f84a815bbb0 2024-04-25T10:01:28.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84ac961640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a80b0730 con 0x7f84a80cc330 2024-04-25T10:01:28.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a659c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f84a815bbb0 0x7f84a80b0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:39312/0 (socket says 172.21.15.162:39312) 2024-04-25T10:01:28.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a659c640 1 -- 172.21.15.162:0/3617339287 learned_addr learned my addr 172.21.15.162:0/3617339287 (peer_addr_for_me v1:172.21.15.162:0/0) 2024-04-25T10:01:28.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a5d9b640 1 --2- 172.21.15.162:0/3617339287 >> v2:172.21.15.133:3300/0 conn(0x7f84a80cc330 0x7f84a80ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.070 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328890137 0 0) 0x7f84a80b00e0 con 0x7f84a815bbb0 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8494003530 con 0x7f84a815bbb0 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1205295351 0 0) 0x7f8494003530 con 0x7f84a815bbb0 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 >> v2:172.21.15.133:3300/0 conn(0x7f84a80cc330 msgr2=0x7f84a80ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 --2- 172.21.15.162:0/3617339287 >> v2:172.21.15.133:3300/0 conn(0x7f84a80cc330 0x7f84a80ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 >> v1:172.21.15.133:6789/0 conn(0x7f84a80cc790 legacy=0x7f84a80ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a80b03c0 con 0x7f84a815bbb0 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f8498001680 con 0x7f84a815bbb0 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 >> v1:172.21.15.162:6789/0 conn(0x7f84a815bbb0 legacy=0x7f84a80b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 --2- 172.21.15.162:0/3617339287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84940039c0 0x7f8494003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 --2- 172.21.15.162:0/3617339287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8494004810 0x7f8494004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 --2- 172.21.15.162:0/3617339287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 0x7f849400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a80b0730 con 0x7f84940039c0 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849400c1c0 con 0x7f84940052f0 2024-04-25T10:01:28.071 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849400c3c0 con 0x7f8494004810 2024-04-25T10:01:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a5d9b640 1 --2- 172.21.15.162:0/3617339287 >> v2:172.21.15.133:3300/0 conn(0x7f84a80cc330 0x7f84a80ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a6d9d640 1 --2- 172.21.15.162:0/3617339287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 0x7f849400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a659c640 1 --2- 172.21.15.162:0/3617339287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84940039c0 0x7f8494003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a5d9b640 1 --2- 172.21.15.162:0/3617339287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8494004810 0x7f8494004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a6d9d640 1 -- 172.21.15.162:0/3617339287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8494004810 msgr2=0x7f8494004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a6d9d640 1 --2- 172.21.15.162:0/3617339287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8494004810 0x7f8494004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a6d9d640 1 -- 172.21.15.162:0/3617339287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84940039c0 msgr2=0x7f8494003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a6d9d640 1 --2- 172.21.15.162:0/3617339287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84940039c0 0x7f8494003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.072 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.063+0000 7f84a6d9d640 1 -- 172.21.15.162:0/3617339287 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8494003530 con 0x7f84940052f0 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84a6d9d640 1 -- 172.21.15.162:0/3617339287 reap_dead start 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84a6d9d640 1 --2- 172.21.15.162:0/3617339287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 0x7f849400b9d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f84a0001e40 tx=0x7f84a001c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e95225007ab1b51c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0001a90 con 0x7f84940052f0 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a00a7070 con 0x7f84940052f0 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84a559a640 1 -- 172.21.15.162:0/3617339287 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0003dc0 con 0x7f84940052f0 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84ac961640 1 -- 172.21.15.162:0/3617339287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 msgr2=0x7f849400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84ac961640 1 --2- 172.21.15.162:0/3617339287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 0x7f849400b9d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f84a0001e40 tx=0x7f84a001c760 comp rx=0 tx=0).stop 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84ac961640 1 -- 172.21.15.162:0/3617339287 shutdown_connections 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84ac961640 1 --2- 172.21.15.162:0/3617339287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 0x7f849400b9d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84ac961640 1 -- 172.21.15.162:0/3617339287 >> 172.21.15.162:0/3617339287 conn(0x7f84a80d0570 msgr2=0x7f84a80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84ac961640 1 -- 172.21.15.162:0/3617339287 shutdown_connections 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84ac961640 1 -- 172.21.15.162:0/3617339287 wait complete. 2024-04-25T10:01:28.074 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84ac961640 1 Processor -- start 2024-04-25T10:01:28.075 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.067+0000 7f84ac961640 1 -- start start 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84ac961640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 0x7f84a80cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84ac961640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a8151350 0x7f84a81504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84ac961640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84a81fa0e0 0x7f84a81fa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84ac961640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a80ad550 con 0x7f84a8151350 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84ac961640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a80ad3d0 con 0x7f84940052f0 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84ac961640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a80ccf80 con 0x7f84a81fa0e0 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84a659c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 0x7f84a80cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84a659c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 0x7f84a80cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46276/0 (socket says 172.21.15.162:46276) 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84a659c640 1 -- 172.21.15.162:0/853719652 learned_addr learned my addr 172.21.15.162:0/853719652 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84a6d9d640 1 --2- 172.21.15.162:0/853719652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84a81fa0e0 0x7f84a81fa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84a5d9b640 1 --2- 172.21.15.162:0/853719652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a8151350 0x7f84a81504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84a659c640 1 -- 172.21.15.162:0/853719652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84a81fa0e0 msgr2=0x7f84a81fa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.077 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84a659c640 1 --2- 172.21.15.162:0/853719652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84a81fa0e0 0x7f84a81fa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84a659c640 1 -- 172.21.15.162:0/853719652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a8151350 msgr2=0x7f84a81504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84a659c640 1 --2- 172.21.15.162:0/853719652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a8151350 0x7f84a81504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84a659c640 1 -- 172.21.15.162:0/853719652 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a80ceb50 con 0x7f84940052f0 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84a659c640 1 --2- 172.21.15.162:0/853719652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 0x7f84a80cf5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f84980072c0 tx=0x7f8498007780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f849effd640 1 -- 172.21.15.162:0/853719652 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849800f9c0 con 0x7f84940052f0 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f849effd640 1 -- 172.21.15.162:0/853719652 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8498018070 con 0x7f84940052f0 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84ac961640 1 -- 172.21.15.162:0/853719652 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a81541d0 con 0x7f84940052f0 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f849effd640 1 -- 172.21.15.162:0/853719652 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849800fdc0 con 0x7f84940052f0 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84ac961640 1 -- 172.21.15.162:0/853719652 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a80cedf0 con 0x7f84940052f0 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f849effd640 1 -- 172.21.15.162:0/853719652 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f849801c7a0 con 0x7f84940052f0 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f849effd640 1 -- 172.21.15.162:0/853719652 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8498013a80 con 0x7f84940052f0 2024-04-25T10:01:28.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.071+0000 7f84ac961640 1 -- 172.21.15.162:0/853719652 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a8067c70 con 0x7f84940052f0 2024-04-25T10:01:28.081 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.075+0000 7f849effd640 1 -- 172.21.15.162:0/853719652 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8498013cd0 con 0x7f84940052f0 2024-04-25T10:01:28.149 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.143+0000 7f84ac961640 1 -- 172.21.15.162:0/853719652 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b6489a91-3b21-46bb-9548-89484d0e1f11", "id": 0} v 0) v1 -- 0x7f84a80b29a0 con 0x7f84940052f0 2024-04-25T10:01:28.158 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.151+0000 7f849effd640 1 -- 172.21.15.162:0/853719652 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b6489a91-3b21-46bb-9548-89484d0e1f11", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f84980133f0 con 0x7f84940052f0 2024-04-25T10:01:28.158 INFO:teuthology.orchestra.run.smithi162.stdout:0 2024-04-25T10:01:28.159 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.155+0000 7f84ac961640 1 -- 172.21.15.162:0/853719652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 msgr2=0x7f84a80cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.155+0000 7f84ac961640 1 --2- 172.21.15.162:0/853719652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 0x7f84a80cf5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f84980072c0 tx=0x7f8498007780 comp rx=0 tx=0).stop 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.155+0000 7f84ac961640 1 -- 172.21.15.162:0/853719652 shutdown_connections 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.155+0000 7f84ac961640 1 --2- 172.21.15.162:0/853719652 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84a81fa0e0 0x7f84a81fa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.155+0000 7f84ac961640 1 --2- 172.21.15.162:0/853719652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84a8151350 0x7f84a81504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.155+0000 7f84ac961640 1 --2- 172.21.15.162:0/853719652 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84940052f0 0x7f84a80cf5f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.155+0000 7f84ac961640 1 -- 172.21.15.162:0/853719652 >> 172.21.15.162:0/853719652 conn(0x7f84a80d0570 msgr2=0x7f84a80d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.155+0000 7f84ac961640 1 -- 172.21.15.162:0/853719652 shutdown_connections 2024-04-25T10:01:28.160 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.155+0000 7f84ac961640 1 -- 172.21.15.162:0/853719652 wait complete. 2024-04-25T10:01:28.175 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new afabe68c-82ae-4784-9c51-6998ffa2016e 1 2024-04-25T10:01:28.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.327+0000 7fe0e267f640 1 Processor -- start 2024-04-25T10:01:28.334 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.327+0000 7fe0e267f640 1 -- start start 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0e267f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe0dc155da0 0x7fe0dc15bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0e267f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc06c2f0 0x7fe0dc0ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0e267f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0dc06b250 con 0x7fe0dc06ac90 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0e267f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0dc06b530 con 0x7fe0dc155da0 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0e267f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc06b810 con 0x7fe0dc06c2f0 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0dbfff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe0dc155da0 0x7fe0dc15bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0db7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc06c2f0 0x7fe0dc0ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0dbfff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe0dc155da0 0x7fe0dc15bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46290/0 (socket says 172.21.15.162:46290) 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0dbfff640 1 -- 172.21.15.162:0/3422735277 learned_addr learned my addr 172.21.15.162:0/3422735277 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0db7fe640 1 -- 172.21.15.162:0/3422735277 >> v2:172.21.15.162:3300/0 conn(0x7fe0dc155da0 msgr2=0x7fe0dc15bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0db7fe640 1 --2- 172.21.15.162:0/3422735277 >> v2:172.21.15.162:3300/0 conn(0x7fe0dc155da0 0x7fe0dc15bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.336 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0db7fe640 1 -- 172.21.15.162:0/3422735277 >> v1:172.21.15.133:6789/0 conn(0x7fe0dc06ac90 legacy=0x7fe0dc15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0db7fe640 1 -- 172.21.15.162:0/3422735277 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc06baf0 con 0x7fe0dc06c2f0 2024-04-25T10:01:28.337 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0dbfff640 1 --2- 172.21.15.162:0/3422735277 >> v2:172.21.15.162:3300/0 conn(0x7fe0dc155da0 0x7fe0dc15bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:28.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0db7fe640 1 --2- 172.21.15.162:0/3422735277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc06c2f0 0x7fe0dc0ca3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40097c0 tx=0x7fe0c401c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=573ae75b5b6c1306 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0daffd640 1 -- 172.21.15.162:0/3422735277 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40b0070 con 0x7fe0dc06c2f0 2024-04-25T10:01:28.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0daffd640 1 -- 172.21.15.162:0/3422735277 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c40ab600 con 0x7fe0dc06c2f0 2024-04-25T10:01:28.338 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.331+0000 7fe0daffd640 1 -- 172.21.15.162:0/3422735277 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40ab9d0 con 0x7fe0dc06c2f0 2024-04-25T10:01:28.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 -- 172.21.15.162:0/3422735277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc06c2f0 msgr2=0x7fe0dc0ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 --2- 172.21.15.162:0/3422735277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc06c2f0 0x7fe0dc0ca3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40097c0 tx=0x7fe0c401c3a0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.339 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 -- 172.21.15.162:0/3422735277 shutdown_connections 2024-04-25T10:01:28.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 --2- 172.21.15.162:0/3422735277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc06c2f0 0x7fe0dc0ca3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 --2- 172.21.15.162:0/3422735277 >> v2:172.21.15.162:3300/0 conn(0x7fe0dc155da0 0x7fe0dc15bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 -- 172.21.15.162:0/3422735277 >> 172.21.15.162:0/3422735277 conn(0x7fe0dc0cc810 msgr2=0x7fe0dc0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 -- 172.21.15.162:0/3422735277 shutdown_connections 2024-04-25T10:01:28.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 -- 172.21.15.162:0/3422735277 wait complete. 2024-04-25T10:01:28.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 Processor -- start 2024-04-25T10:01:28.340 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 -- start start 2024-04-25T10:01:28.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0dc06ac90 0x7fe0dc0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0dc06c2f0 0x7fe0dc0b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.342 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc155da0 0x7fe0dc150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc168860 con 0x7fe0dc06ac90 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc1686e0 con 0x7fe0dc06c2f0 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc1689e0 con 0x7fe0dc155da0 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0db7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0dc06c2f0 0x7fe0dc0b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e0c1c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc155da0 0x7fe0dc150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0db7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0dc06c2f0 0x7fe0dc0b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46294/0 (socket says 172.21.15.162:46294) 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0db7fe640 1 -- 172.21.15.162:0/3122941075 learned_addr learned my addr 172.21.15.162:0/3122941075 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0dbfff640 1 --2- 172.21.15.162:0/3122941075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0dc06ac90 0x7fe0dc0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e0c1c640 1 -- 172.21.15.162:0/3122941075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0dc06c2f0 msgr2=0x7fe0dc0b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e0c1c640 1 --2- 172.21.15.162:0/3122941075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0dc06c2f0 0x7fe0dc0b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e0c1c640 1 -- 172.21.15.162:0/3122941075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0dc06ac90 msgr2=0x7fe0dc0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e0c1c640 1 --2- 172.21.15.162:0/3122941075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0dc06ac90 0x7fe0dc0b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.343 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e0c1c640 1 -- 172.21.15.162:0/3122941075 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc073d80 con 0x7fe0dc155da0 2024-04-25T10:01:28.344 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0db7fe640 1 --2- 172.21.15.162:0/3122941075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0dc06c2f0 0x7fe0dc0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:28.344 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e0c1c640 1 --2- 172.21.15.162:0/3122941075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc155da0 0x7fe0dc150da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c780 tx=0x7fe0cc00cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.344 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0d8ff9640 1 -- 172.21.15.162:0/3122941075 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc016020 con 0x7fe0dc155da0 2024-04-25T10:01:28.344 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0d8ff9640 1 -- 172.21.15.162:0/3122941075 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0cc00aa50 con 0x7fe0dc155da0 2024-04-25T10:01:28.344 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 -- 172.21.15.162:0/3122941075 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc153120 con 0x7fe0dc155da0 2024-04-25T10:01:28.344 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0d8ff9640 1 -- 172.21.15.162:0/3122941075 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc00ae40 con 0x7fe0dc155da0 2024-04-25T10:01:28.344 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0e267f640 1 -- 172.21.15.162:0/3122941075 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc0b2d10 con 0x7fe0dc155da0 2024-04-25T10:01:28.344 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0d8ff9640 1 -- 172.21.15.162:0/3122941075 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe0cc019020 con 0x7fe0dc155da0 2024-04-25T10:01:28.344 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.335+0000 7fe0d8ff9640 1 -- 172.21.15.162:0/3122941075 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fe0cc012460 con 0x7fe0dc155da0 2024-04-25T10:01:28.344 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.339+0000 7fe0e267f640 1 -- 172.21.15.162:0/3122941075 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc067c70 con 0x7fe0dc155da0 2024-04-25T10:01:28.345 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.339+0000 7fe0d8ff9640 1 -- 172.21.15.162:0/3122941075 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe0cc020030 con 0x7fe0dc155da0 2024-04-25T10:01:28.413 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.407+0000 7fe0e267f640 1 -- 172.21.15.162:0/3122941075 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "afabe68c-82ae-4784-9c51-6998ffa2016e", "id": 1} v 0) v1 -- 0x7fe0dc0b5d10 con 0x7fe0dc155da0 2024-04-25T10:01:28.420 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.415+0000 7fe0d8ff9640 1 -- 172.21.15.162:0/3122941075 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "afabe68c-82ae-4784-9c51-6998ffa2016e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe0cc005020 con 0x7fe0dc155da0 2024-04-25T10:01:28.420 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-04-25T10:01:28.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.415+0000 7fe0e267f640 1 -- 172.21.15.162:0/3122941075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc155da0 msgr2=0x7fe0dc150da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.415+0000 7fe0e267f640 1 --2- 172.21.15.162:0/3122941075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc155da0 0x7fe0dc150da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c780 tx=0x7fe0cc00cc40 comp rx=0 tx=0).stop 2024-04-25T10:01:28.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.415+0000 7fe0e267f640 1 -- 172.21.15.162:0/3122941075 shutdown_connections 2024-04-25T10:01:28.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.415+0000 7fe0e267f640 1 --2- 172.21.15.162:0/3122941075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0dc155da0 0x7fe0dc150da0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.415+0000 7fe0e267f640 1 --2- 172.21.15.162:0/3122941075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0dc06c2f0 0x7fe0dc0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.415+0000 7fe0e267f640 1 --2- 172.21.15.162:0/3122941075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0dc06ac90 0x7fe0dc0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.415+0000 7fe0e267f640 1 -- 172.21.15.162:0/3122941075 >> 172.21.15.162:0/3122941075 conn(0x7fe0dc0cc810 msgr2=0x7fe0dc159970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.415+0000 7fe0e267f640 1 -- 172.21.15.162:0/3122941075 shutdown_connections 2024-04-25T10:01:28.422 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.415+0000 7fe0e267f640 1 -- 172.21.15.162:0/3122941075 wait complete. 2024-04-25T10:01:28.437 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 3ebbc6d1-c28b-4e7c-a9b6-de419e68550b 2 2024-04-25T10:01:28.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.547+0000 7f4b8c307640 1 Processor -- start 2024-04-25T10:01:28.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.547+0000 7f4b8c307640 1 -- start start 2024-04-25T10:01:28.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8c307640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4b8406e860 0x7f4b840cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8c307640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8415b170 con 0x7f4b84159c10 2024-04-25T10:01:28.555 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8c307640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8415db80 con 0x7f4b8406ed60 2024-04-25T10:01:28.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8c307640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b8415e1d0 con 0x7f4b8406e860 2024-04-25T10:01:28.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b898a2640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4b8406ed60 0x7f4b840cbc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.162:39320/0 (socket says 172.21.15.162:39320) 2024-04-25T10:01:28.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b898a2640 1 -- 172.21.15.162:0/881457603 learned_addr learned my addr 172.21.15.162:0/881457603 (peer_addr_for_me v1:172.21.15.162:0/0) 2024-04-25T10:01:28.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8a8a4640 1 --2- 172.21.15.162:0/881457603 >> v2:172.21.15.133:3300/0 conn(0x7f4b8406e860 0x7f4b840cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.556 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605681342 0 0) 0x7f4b8415db80 con 0x7f4b8406ed60 2024-04-25T10:01:28.556 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f60cbf65180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b70003450 con 0x7f4b8406ed60 2024-04-25T10:01:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139608868 0 0) 0x7f4b8415b170 con 0x7f4b84159c10 2024-04-25T10:01:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b8415db80 con 0x7f4b84159c10 2024-04-25T10:01:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1837982839 0 0) 0x7f4b70003450 con 0x7f4b8406ed60 2024-04-25T10:01:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 >> v2:172.21.15.133:3300/0 conn(0x7f4b8406e860 msgr2=0x7f4b840cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 --2- 172.21.15.162:0/881457603 >> v2:172.21.15.133:3300/0 conn(0x7f4b8406e860 0x7f4b840cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 >> v1:172.21.15.133:6789/0 conn(0x7f4b84159c10 legacy=0x7f4b8406e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8415de60 con 0x7f4b8406ed60 2024-04-25T10:01:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4b74001680 con 0x7f4b8406ed60 2024-04-25T10:01:28.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 >> v1:172.21.15.162:6789/0 conn(0x7f4b8406ed60 legacy=0x7f4b840cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 --2- 172.21.15.162:0/881457603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b70003c40 0x7f4b70008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 --2- 172.21.15.162:0/881457603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b70004170 0x7f4b7000ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 --2- 172.21.15.162:0/881457603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b7000b4c0 0x7f4b7000d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8415e1d0 con 0x7f4b70004170 2024-04-25T10:01:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7000e260 con 0x7f4b70003c40 2024-04-25T10:01:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7000e460 con 0x7f4b7000b4c0 2024-04-25T10:01:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8a0a3640 1 --2- 172.21.15.162:0/881457603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b70003c40 0x7f4b70008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8a8a4640 1 --2- 172.21.15.162:0/881457603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b7000b4c0 0x7f4b7000d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b898a2640 1 --2- 172.21.15.162:0/881457603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b70004170 0x7f4b7000ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8a0a3640 1 -- 172.21.15.162:0/881457603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b7000b4c0 msgr2=0x7f4b7000d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.561 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8a0a3640 1 --2- 172.21.15.162:0/881457603 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b7000b4c0 0x7f4b7000d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8a0a3640 1 -- 172.21.15.162:0/881457603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b70004170 msgr2=0x7f4b7000ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8a0a3640 1 --2- 172.21.15.162:0/881457603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b70004170 0x7f4b7000ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8a0a3640 1 -- 172.21.15.162:0/881457603 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b70003450 con 0x7f4b70003c40 2024-04-25T10:01:28.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.551+0000 7f4b8a8a4640 1 -- 172.21.15.162:0/881457603 reap_dead start 2024-04-25T10:01:28.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8a0a3640 1 --2- 172.21.15.162:0/881457603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b70003c40 0x7f4b70008860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c001fd0 tx=0x7f4b7c01f030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa0f120f5e9be29c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c001c20 con 0x7f4b70003c40 2024-04-25T10:01:28.562 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c0a8c70 con 0x7f4b70003c40 2024-04-25T10:01:28.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b890a1640 1 -- 172.21.15.162:0/881457603 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c003bc0 con 0x7f4b70003c40 2024-04-25T10:01:28.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 -- 172.21.15.162:0/881457603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b70003c40 msgr2=0x7f4b70008860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 --2- 172.21.15.162:0/881457603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b70003c40 0x7f4b70008860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c001fd0 tx=0x7f4b7c01f030 comp rx=0 tx=0).stop 2024-04-25T10:01:28.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 -- 172.21.15.162:0/881457603 shutdown_connections 2024-04-25T10:01:28.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 --2- 172.21.15.162:0/881457603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b70003c40 0x7f4b70008860 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 -- 172.21.15.162:0/881457603 >> 172.21.15.162:0/881457603 conn(0x7f4b840cc810 msgr2=0x7f4b840ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 -- 172.21.15.162:0/881457603 shutdown_connections 2024-04-25T10:01:28.563 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 -- 172.21.15.162:0/881457603 wait complete. 2024-04-25T10:01:28.565 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 Processor -- start 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 -- start start 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b70003c40 0x7f4b840af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b840acd10 0x7f4b84150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b84158150 0x7f4b84158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8415a170 con 0x7f4b84158150 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84159ff0 con 0x7f4b840acd10 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8c307640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84150760 con 0x7f4b70003c40 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b898a2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b840acd10 0x7f4b84150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8a0a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b70003c40 0x7f4b840af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b898a2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b840acd10 0x7f4b84150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46314/0 (socket says 172.21.15.162:46314) 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b898a2640 1 -- 172.21.15.162:0/3923107819 learned_addr learned my addr 172.21.15.162:0/3923107819 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8a8a4640 1 --2- 172.21.15.162:0/3923107819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b84158150 0x7f4b84158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b898a2640 1 -- 172.21.15.162:0/3923107819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b70003c40 msgr2=0x7f4b840af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b898a2640 1 --2- 172.21.15.162:0/3923107819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b70003c40 0x7f4b840af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b898a2640 1 -- 172.21.15.162:0/3923107819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b84158150 msgr2=0x7f4b84158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b898a2640 1 --2- 172.21.15.162:0/3923107819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b84158150 0x7f4b84158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b898a2640 1 -- 172.21.15.162:0/3923107819 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840c7600 con 0x7f4b840acd10 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.555+0000 7f4b8a8a4640 1 --2- 172.21.15.162:0/3923107819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b84158150 0x7f4b84158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.559+0000 7f4b898a2640 1 --2- 172.21.15.162:0/3923107819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b840acd10 0x7f4b84150220 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4b74007290 tx=0x7f4b74007750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.559+0000 7f4b7affd640 1 -- 172.21.15.162:0/3923107819 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400f940 con 0x7f4b840acd10 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.559+0000 7f4b7affd640 1 -- 172.21.15.162:0/3923107819 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b74018070 con 0x7f4b840acd10 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.559+0000 7f4b8c307640 1 -- 172.21.15.162:0/3923107819 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b840ad1d0 con 0x7f4b840acd10 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.559+0000 7f4b7affd640 1 -- 172.21.15.162:0/3923107819 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400fd40 con 0x7f4b840acd10 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.559+0000 7f4b8c307640 1 -- 172.21.15.162:0/3923107819 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b840ad790 con 0x7f4b840acd10 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.559+0000 7f4b7affd640 1 -- 172.21.15.162:0/3923107819 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4b7401d460 con 0x7f4b840acd10 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.559+0000 7f4b7affd640 1 -- 172.21.15.162:0/3923107819 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f4b74013a90 con 0x7f4b840acd10 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.559+0000 7f4b8c307640 1 -- 172.21.15.162:0/3923107819 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b84067c70 con 0x7f4b840acd10 2024-04-25T10:01:28.567 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.559+0000 7f4b7affd640 1 -- 172.21.15.162:0/3923107819 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4b74013d70 con 0x7f4b840acd10 2024-04-25T10:01:28.612 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:28.607+0000 7f60cbf65180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:01:28.617 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:28.613+0000 7f16e84e3180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:28.633 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.627+0000 7f4b8c307640 1 -- 172.21.15.162:0/3923107819 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3ebbc6d1-c28b-4e7c-a9b6-de419e68550b", "id": 2} v 0) v1 -- 0x7f4b840c77d0 con 0x7f4b840acd10 2024-04-25T10:01:28.641 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.635+0000 7f4b7affd640 1 -- 172.21.15.162:0/3923107819 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3ebbc6d1-c28b-4e7c-a9b6-de419e68550b", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4b74013670 con 0x7f4b840acd10 2024-04-25T10:01:28.641 INFO:teuthology.orchestra.run.smithi162.stdout:2 2024-04-25T10:01:28.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.639+0000 7f4b8c307640 1 -- 172.21.15.162:0/3923107819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b840acd10 msgr2=0x7f4b84150220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.639+0000 7f4b8c307640 1 --2- 172.21.15.162:0/3923107819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b840acd10 0x7f4b84150220 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4b74007290 tx=0x7f4b74007750 comp rx=0 tx=0).stop 2024-04-25T10:01:28.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.639+0000 7f4b8c307640 1 -- 172.21.15.162:0/3923107819 shutdown_connections 2024-04-25T10:01:28.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.639+0000 7f4b8c307640 1 --2- 172.21.15.162:0/3923107819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b84158150 0x7f4b84158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.639+0000 7f4b8c307640 1 --2- 172.21.15.162:0/3923107819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b840acd10 0x7f4b84150220 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.639+0000 7f4b8c307640 1 --2- 172.21.15.162:0/3923107819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b70003c40 0x7f4b840af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.639+0000 7f4b8c307640 1 -- 172.21.15.162:0/3923107819 >> 172.21.15.162:0/3923107819 conn(0x7f4b840cc810 msgr2=0x7f4b8406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.639+0000 7f4b8c307640 1 -- 172.21.15.162:0/3923107819 shutdown_connections 2024-04-25T10:01:28.643 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.639+0000 7f4b8c307640 1 -- 172.21.15.162:0/3923107819 wait complete. 2024-04-25T10:01:28.662 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 2c2000e2-21f4-40f6-b8f6-a7a181a2dd49 3 2024-04-25T10:01:28.672 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:28.665+0000 7f16e84e3180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:01:28.676 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:28.671+0000 7f60cbf65180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:01:28.724 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:28.717+0000 7f16e84e3180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:01:28.756 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:28.751+0000 7f60cbf65180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.771+0000 7feba9bbc640 1 Processor -- start 2024-04-25T10:01:28.777 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.771+0000 7feba9bbc640 1 -- start start 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.771+0000 7feba9bbc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba415f020 0x7feba40cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.771+0000 7feba9bbc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feba40d21c0 0x7feba40cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.771+0000 7feba9bbc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba40ccdf0 con 0x7feba40cfcf0 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.771+0000 7feba9bbc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba40cd690 con 0x7feba40d21c0 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.771+0000 7feba9bbc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba40cd0d0 con 0x7feba415f020 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.771+0000 7feba37fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba415f020 0x7feba40cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.771+0000 7feba37fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba415f020 0x7feba40cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:57234/0 (socket says 172.21.15.162:57234) 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.771+0000 7feba37fe640 1 -- 172.21.15.162:0/1898679092 learned_addr learned my addr 172.21.15.162:0/1898679092 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba2ffd640 1 --2- 172.21.15.162:0/1898679092 >> v2:172.21.15.133:3300/0 conn(0x7feba40d21c0 0x7feba40cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba37fe640 1 -- 172.21.15.162:0/1898679092 >> v2:172.21.15.133:3300/0 conn(0x7feba40d21c0 msgr2=0x7feba40cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba37fe640 1 --2- 172.21.15.162:0/1898679092 >> v2:172.21.15.133:3300/0 conn(0x7feba40d21c0 0x7feba40cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba37fe640 1 -- 172.21.15.162:0/1898679092 >> v1:172.21.15.133:6789/0 conn(0x7feba40cfcf0 legacy=0x7feba40cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.779 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba37fe640 1 -- 172.21.15.162:0/1898679092 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba40cd3b0 con 0x7feba415f020 2024-04-25T10:01:28.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba37fe640 1 --2- 172.21.15.162:0/1898679092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba415f020 0x7feba40cf270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feb94009870 tx=0x7feb9401c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9a92cbf5cd04d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba27fc640 1 -- 172.21.15.162:0/1898679092 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb940ab5f0 con 0x7feba415f020 2024-04-25T10:01:28.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba27fc640 1 -- 172.21.15.162:0/1898679092 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb940b0070 con 0x7feba415f020 2024-04-25T10:01:28.781 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba27fc640 1 -- 172.21.15.162:0/1898679092 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb940ab9b0 con 0x7feba415f020 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba9bbc640 1 -- 172.21.15.162:0/1898679092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba415f020 msgr2=0x7feba40cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba9bbc640 1 --2- 172.21.15.162:0/1898679092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba415f020 0x7feba40cf270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feb94009870 tx=0x7feb9401c520 comp rx=0 tx=0).stop 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba9bbc640 1 -- 172.21.15.162:0/1898679092 shutdown_connections 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba9bbc640 1 --2- 172.21.15.162:0/1898679092 >> v2:172.21.15.133:3300/0 conn(0x7feba40d21c0 0x7feba40cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba9bbc640 1 --2- 172.21.15.162:0/1898679092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba415f020 0x7feba40cf270 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba9bbc640 1 -- 172.21.15.162:0/1898679092 >> 172.21.15.162:0/1898679092 conn(0x7feba40d16a0 msgr2=0x7feba40d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.782 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba9bbc640 1 -- 172.21.15.162:0/1898679092 shutdown_connections 2024-04-25T10:01:28.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.775+0000 7feba9bbc640 1 -- 172.21.15.162:0/1898679092 wait complete. 2024-04-25T10:01:28.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba9bbc640 1 Processor -- start 2024-04-25T10:01:28.783 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba9bbc640 1 -- start start 2024-04-25T10:01:28.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba9bbc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba40cfcf0 0x7feba40b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba9bbc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba40d21c0 0x7feba414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba9bbc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feba415f020 0x7feba40c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba9bbc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba41688f0 con 0x7feba415f020 2024-04-25T10:01:28.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba9bbc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba4168770 con 0x7feba40d21c0 2024-04-25T10:01:28.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba9bbc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba4168a70 con 0x7feba40cfcf0 2024-04-25T10:01:28.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba37fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba40cfcf0 0x7feba40b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba37fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba40cfcf0 0x7feba40b6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:57246/0 (socket says 172.21.15.162:57246) 2024-04-25T10:01:28.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba37fe640 1 -- 172.21.15.162:0/1072839707 learned_addr learned my addr 172.21.15.162:0/1072839707 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:28.785 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba2ffd640 1 --2- 172.21.15.162:0/1072839707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba40d21c0 0x7feba414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba3fff640 1 --2- 172.21.15.162:0/1072839707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feba415f020 0x7feba40c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba2ffd640 1 -- 172.21.15.162:0/1072839707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba40cfcf0 msgr2=0x7feba40b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba2ffd640 1 --2- 172.21.15.162:0/1072839707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba40cfcf0 0x7feba40b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba2ffd640 1 -- 172.21.15.162:0/1072839707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feba415f020 msgr2=0x7feba40c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba2ffd640 1 --2- 172.21.15.162:0/1072839707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feba415f020 0x7feba40c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba2ffd640 1 -- 172.21.15.162:0/1072839707 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba40b72a0 con 0x7feba40d21c0 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba3fff640 1 --2- 172.21.15.162:0/1072839707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feba415f020 0x7feba40c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba2ffd640 1 --2- 172.21.15.162:0/1072839707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba40d21c0 0x7feba414ed40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feb9800d5f0 tx=0x7feb9800dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feb8bfff640 1 -- 172.21.15.162:0/1072839707 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980125f0 con 0x7feba40d21c0 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feb8bfff640 1 -- 172.21.15.162:0/1072839707 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb98019070 con 0x7feba40d21c0 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba9bbc640 1 -- 172.21.15.162:0/1072839707 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba4150f70 con 0x7feba40d21c0 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feb8bfff640 1 -- 172.21.15.162:0/1072839707 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980129d0 con 0x7feba40d21c0 2024-04-25T10:01:28.786 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba9bbc640 1 -- 172.21.15.162:0/1072839707 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba4151530 con 0x7feba40d21c0 2024-04-25T10:01:28.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feb8bfff640 1 -- 172.21.15.162:0/1072839707 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7feb98012b70 con 0x7feba40d21c0 2024-04-25T10:01:28.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feb8bfff640 1 -- 172.21.15.162:0/1072839707 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7feb98014960 con 0x7feba40d21c0 2024-04-25T10:01:28.787 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.779+0000 7feba9bbc640 1 -- 172.21.15.162:0/1072839707 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba4067c70 con 0x7feba40d21c0 2024-04-25T10:01:28.789 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.783+0000 7feb8bfff640 1 -- 172.21.15.162:0/1072839707 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7feb98014c00 con 0x7feba40d21c0 2024-04-25T10:01:28.801 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:28.793+0000 7f16e84e3180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:01:28.812 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:28.807+0000 7f60cbf65180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:01:28.852 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:28.845+0000 7f16e84e3180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:01:28.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.851+0000 7feba9bbc640 1 -- 172.21.15.162:0/1072839707 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2c2000e2-21f4-40f6-b8f6-a7a181a2dd49", "id": 3} v 0) v1 -- 0x7feba41559b0 con 0x7feba40d21c0 2024-04-25T10:01:28.863 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.859+0000 7feb8bfff640 1 -- 172.21.15.162:0/1072839707 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2c2000e2-21f4-40f6-b8f6-a7a181a2dd49", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7feb980144d0 con 0x7feba40d21c0 2024-04-25T10:01:28.864 INFO:teuthology.orchestra.run.smithi162.stdout:3 2024-04-25T10:01:28.864 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.859+0000 7feba9bbc640 1 -- 172.21.15.162:0/1072839707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba40d21c0 msgr2=0x7feba414ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.859+0000 7feba9bbc640 1 --2- 172.21.15.162:0/1072839707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba40d21c0 0x7feba414ed40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feb9800d5f0 tx=0x7feb9800dab0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.859+0000 7feba9bbc640 1 -- 172.21.15.162:0/1072839707 shutdown_connections 2024-04-25T10:01:28.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.859+0000 7feba9bbc640 1 --2- 172.21.15.162:0/1072839707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feba415f020 0x7feba40c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.859+0000 7feba9bbc640 1 --2- 172.21.15.162:0/1072839707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feba40d21c0 0x7feba414ed40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.859+0000 7feba9bbc640 1 --2- 172.21.15.162:0/1072839707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feba40cfcf0 0x7feba40b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.859+0000 7feba9bbc640 1 -- 172.21.15.162:0/1072839707 >> 172.21.15.162:0/1072839707 conn(0x7feba40d16a0 msgr2=0x7feba40cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.859+0000 7feba9bbc640 1 -- 172.21.15.162:0/1072839707 shutdown_connections 2024-04-25T10:01:28.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:28.859+0000 7feba9bbc640 1 -- 172.21.15.162:0/1072839707 wait complete. 2024-04-25T10:01:28.881 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 322330bb-a145-4561-9bf4-24b807d2e51a 4 2024-04-25T10:01:28.893 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:28.887+0000 7f60cbf65180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:01:28.929 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:28.925+0000 7f16e84e3180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:01:28.949 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:28.943+0000 7f60cbf65180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:01:28.982 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:28.977+0000 7f16e84e3180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:01:29.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.003+0000 7ff0f5012640 1 Processor -- start 2024-04-25T10:01:29.009 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.003+0000 7ff0f5012640 1 -- start start 2024-04-25T10:01:29.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015a650 0x7ff0f006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0f00cf290 0x7ff0f00cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f015e720 con 0x7ff0f015e1f0 2024-04-25T10:01:29.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f006ddf0 con 0x7ff0f00cf290 2024-04-25T10:01:29.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f015ea00 con 0x7ff0f015a650 2024-04-25T10:01:29.011 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0eed9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015a650 0x7ff0f006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0eed9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015a650 0x7ff0f006d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:57250/0 (socket says 172.21.15.162:57250) 2024-04-25T10:01:29.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0eed9d640 1 -- 172.21.15.162:0/373103306 learned_addr learned my addr 172.21.15.162:0/373103306 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:29.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0ef59e640 1 --2- 172.21.15.162:0/373103306 >> v2:172.21.15.133:3300/0 conn(0x7ff0f00cf290 0x7ff0f00cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0eed9d640 1 -- 172.21.15.162:0/373103306 >> v2:172.21.15.133:3300/0 conn(0x7ff0f00cf290 msgr2=0x7ff0f00cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:29.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0eed9d640 1 --2- 172.21.15.162:0/373103306 >> v2:172.21.15.133:3300/0 conn(0x7ff0f00cf290 0x7ff0f00cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0eed9d640 1 -- 172.21.15.162:0/373103306 >> v1:172.21.15.133:6789/0 conn(0x7ff0f015e1f0 legacy=0x7ff0f00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:29.012 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0eed9d640 1 -- 172.21.15.162:0/373103306 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f015ece0 con 0x7ff0f015a650 2024-04-25T10:01:29.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0eed9d640 1 --2- 172.21.15.162:0/373103306 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015a650 0x7ff0f006d8b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800bb10 tx=0x7ff0e801f650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56bd76560653966e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0edd9b640 1 -- 172.21.15.162:0/373103306 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e80b4070 con 0x7ff0f015a650 2024-04-25T10:01:29.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0edd9b640 1 -- 172.21.15.162:0/373103306 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e80a7ca0 con 0x7ff0f015a650 2024-04-25T10:01:29.013 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0edd9b640 1 -- 172.21.15.162:0/373103306 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e80af660 con 0x7ff0f015a650 2024-04-25T10:01:29.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 -- 172.21.15.162:0/373103306 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015a650 msgr2=0x7ff0f006d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 --2- 172.21.15.162:0/373103306 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015a650 0x7ff0f006d8b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800bb10 tx=0x7ff0e801f650 comp rx=0 tx=0).stop 2024-04-25T10:01:29.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 -- 172.21.15.162:0/373103306 shutdown_connections 2024-04-25T10:01:29.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 --2- 172.21.15.162:0/373103306 >> v2:172.21.15.133:3300/0 conn(0x7ff0f00cf290 0x7ff0f00cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 --2- 172.21.15.162:0/373103306 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015a650 0x7ff0f006d8b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.015 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 -- 172.21.15.162:0/373103306 >> 172.21.15.162:0/373103306 conn(0x7ff0f00cc810 msgr2=0x7ff0f00ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.015 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 -- 172.21.15.162:0/373103306 shutdown_connections 2024-04-25T10:01:29.015 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.007+0000 7ff0f5012640 1 -- 172.21.15.162:0/373103306 wait complete. 2024-04-25T10:01:29.015 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0f5012640 1 Processor -- start 2024-04-25T10:01:29.015 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0f5012640 1 -- start start 2024-04-25T10:01:29.017 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0f5012640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0f00cf290 0x7ff0f014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.017 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0f5012640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f015a650 0x7ff0f006b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.017 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0f5012640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015e1f0 0x7ff0f006b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.017 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0f5012640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0066c20 con 0x7ff0f015a650 2024-04-25T10:01:29.017 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0f5012640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0066aa0 con 0x7ff0f00cf290 2024-04-25T10:01:29.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0f5012640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f006ac90 con 0x7ff0f015e1f0 2024-04-25T10:01:29.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0eed9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0f00cf290 0x7ff0f014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0ef59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015e1f0 0x7ff0f006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0ef59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015e1f0 0x7ff0f006b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:57256/0 (socket says 172.21.15.162:57256) 2024-04-25T10:01:29.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0eed9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0f00cf290 0x7ff0f014e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46336/0 (socket says 172.21.15.162:46336) 2024-04-25T10:01:29.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0ef59e640 1 -- 172.21.15.162:0/599781129 learned_addr learned my addr 172.21.15.162:0/599781129 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:29.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0ee59c640 1 --2- 172.21.15.162:0/599781129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f015a650 0x7ff0f006b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0ef59e640 1 -- 172.21.15.162:0/599781129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0f00cf290 msgr2=0x7ff0f014e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0ef59e640 1 --2- 172.21.15.162:0/599781129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0f00cf290 0x7ff0f014e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0ef59e640 1 -- 172.21.15.162:0/599781129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f015a650 msgr2=0x7ff0f006b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0ef59e640 1 --2- 172.21.15.162:0/599781129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f015a650 0x7ff0f006b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0ef59e640 1 -- 172.21.15.162:0/599781129 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f014e8d0 con 0x7ff0f015e1f0 2024-04-25T10:01:29.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0eed9d640 1 --2- 172.21.15.162:0/599781129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0f00cf290 0x7ff0f014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:29.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0ef59e640 1 --2- 172.21.15.162:0/599781129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015e1f0 0x7ff0f006b620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0007e70 tx=0x7ff0e000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0e77fe640 1 -- 172.21.15.162:0/599781129 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0014620 con 0x7ff0f015e1f0 2024-04-25T10:01:29.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0e77fe640 1 -- 172.21.15.162:0/599781129 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e0019070 con 0x7ff0f015e1f0 2024-04-25T10:01:29.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0f5012640 1 -- 172.21.15.162:0/599781129 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f01637b0 con 0x7ff0f015e1f0 2024-04-25T10:01:29.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0e77fe640 1 -- 172.21.15.162:0/599781129 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0014a20 con 0x7ff0f015e1f0 2024-04-25T10:01:29.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0f5012640 1 -- 172.21.15.162:0/599781129 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f0163d70 con 0x7ff0f015e1f0 2024-04-25T10:01:29.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0e77fe640 1 -- 172.21.15.162:0/599781129 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff0e0014bc0 con 0x7ff0f015e1f0 2024-04-25T10:01:29.019 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0e77fe640 1 -- 172.21.15.162:0/599781129 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff0e001e970 con 0x7ff0f015e1f0 2024-04-25T10:01:29.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.011+0000 7ff0f5012640 1 -- 172.21.15.162:0/599781129 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f0067c70 con 0x7ff0f015e1f0 2024-04-25T10:01:29.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.015+0000 7ff0e77fe640 1 -- 172.21.15.162:0/599781129 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff0e001e460 con 0x7ff0f015e1f0 2024-04-25T10:01:29.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.083+0000 7ff0f5012640 1 -- 172.21.15.162:0/599781129 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "322330bb-a145-4561-9bf4-24b807d2e51a", "id": 4} v 0) v1 -- 0x7ff0f006e5c0 con 0x7ff0f015e1f0 2024-04-25T10:01:29.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.087+0000 7ff0e77fe640 1 -- 172.21.15.162:0/599781129 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "322330bb-a145-4561-9bf4-24b807d2e51a", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff0e001e640 con 0x7ff0f015e1f0 2024-04-25T10:01:29.094 INFO:teuthology.orchestra.run.smithi162.stdout:4 2024-04-25T10:01:29.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.091+0000 7ff0f5012640 1 -- 172.21.15.162:0/599781129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015e1f0 msgr2=0x7ff0f006b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.091+0000 7ff0f5012640 1 --2- 172.21.15.162:0/599781129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015e1f0 0x7ff0f006b620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0007e70 tx=0x7ff0e000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.091+0000 7ff0f5012640 1 -- 172.21.15.162:0/599781129 shutdown_connections 2024-04-25T10:01:29.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.091+0000 7ff0f5012640 1 --2- 172.21.15.162:0/599781129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f015e1f0 0x7ff0f006b620 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.091+0000 7ff0f5012640 1 --2- 172.21.15.162:0/599781129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f015a650 0x7ff0f006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.091+0000 7ff0f5012640 1 --2- 172.21.15.162:0/599781129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0f00cf290 0x7ff0f014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.091+0000 7ff0f5012640 1 -- 172.21.15.162:0/599781129 >> 172.21.15.162:0/599781129 conn(0x7ff0f00cc810 msgr2=0x7ff0f00acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.091+0000 7ff0f5012640 1 -- 172.21.15.162:0/599781129 shutdown_connections 2024-04-25T10:01:29.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.091+0000 7ff0f5012640 1 -- 172.21.15.162:0/599781129 wait complete. 2024-04-25T10:01:29.105 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:29.099+0000 7f60cbf65180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:01:29.111 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 58d88fea-42ba-4586-853d-fcfcb2883e81 5 2024-04-25T10:01:29.131 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:29.125+0000 7f16e84e3180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:01:29.171 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:29.167+0000 7f60cbf65180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:01:29.192 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:29.185+0000 7f16e84e3180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:01:29.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d3d72640 1 Processor -- start 2024-04-25T10:01:29.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d3d72640 1 -- start start 2024-04-25T10:01:29.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d3d72640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6cc0cc440 0x7ff6cc15cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d3d72640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff6cc15ba70 0x7ff6cc0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d3d72640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc06ad20 0x7ff6cc06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d3d72640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6cc15beb0 con 0x7ff6cc15ba70 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d3d72640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6cc0ccd60 con 0x7ff6cc0cc440 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d3d72640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc06baa0 con 0x7ff6cc06ad20 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d230f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc06ad20 0x7ff6cc06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d1b0e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6cc0cc440 0x7ff6cc15cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d230f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc06ad20 0x7ff6cc06b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:57260/0 (socket says 172.21.15.162:57260) 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d230f640 1 -- 172.21.15.162:0/2600997962 learned_addr learned my addr 172.21.15.162:0/2600997962 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d130d640 1 --2- 172.21.15.162:0/2600997962 >> v2:172.21.15.133:3300/0 conn(0x7ff6cc15ba70 0x7ff6cc0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d230f640 1 -- 172.21.15.162:0/2600997962 >> v2:172.21.15.162:3300/0 conn(0x7ff6cc0cc440 msgr2=0x7ff6cc15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d230f640 1 --2- 172.21.15.162:0/2600997962 >> v2:172.21.15.162:3300/0 conn(0x7ff6cc0cc440 0x7ff6cc15cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d230f640 1 -- 172.21.15.162:0/2600997962 >> v2:172.21.15.133:3300/0 conn(0x7ff6cc15ba70 msgr2=0x7ff6cc0cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:29.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d230f640 1 --2- 172.21.15.162:0/2600997962 >> v2:172.21.15.133:3300/0 conn(0x7ff6cc15ba70 0x7ff6cc0cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d230f640 1 -- 172.21.15.162:0/2600997962 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc15c100 con 0x7ff6cc06ad20 2024-04-25T10:01:29.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d1b0e640 1 --2- 172.21.15.162:0/2600997962 >> v2:172.21.15.162:3300/0 conn(0x7ff6cc0cc440 0x7ff6cc15cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:29.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.219+0000 7ff6d130d640 1 --2- 172.21.15.162:0/2600997962 >> v2:172.21.15.133:3300/0 conn(0x7ff6cc15ba70 0x7ff6cc0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:29.229 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.223+0000 7ff6d230f640 1 --2- 172.21.15.162:0/2600997962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc06ad20 0x7ff6cc06b1b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0ccf80 tx=0x7ff6c401c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d8a21d0d93c6b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.230 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.223+0000 7ff6d0b0c640 1 -- 172.21.15.162:0/2600997962 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c40a5840 con 0x7ff6cc06ad20 2024-04-25T10:01:29.230 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.223+0000 7ff6d0b0c640 1 -- 172.21.15.162:0/2600997962 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c40a59e0 con 0x7ff6cc06ad20 2024-04-25T10:01:29.230 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.223+0000 7ff6d0b0c640 1 -- 172.21.15.162:0/2600997962 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c40a5db0 con 0x7ff6cc06ad20 2024-04-25T10:01:29.231 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 -- 172.21.15.162:0/2600997962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc06ad20 msgr2=0x7ff6cc06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.231 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 --2- 172.21.15.162:0/2600997962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc06ad20 0x7ff6cc06b1b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0ccf80 tx=0x7ff6c401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.231 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 -- 172.21.15.162:0/2600997962 shutdown_connections 2024-04-25T10:01:29.231 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 --2- 172.21.15.162:0/2600997962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc06ad20 0x7ff6cc06b1b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.231 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 --2- 172.21.15.162:0/2600997962 >> v2:172.21.15.133:3300/0 conn(0x7ff6cc15ba70 0x7ff6cc0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.231 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 --2- 172.21.15.162:0/2600997962 >> v2:172.21.15.162:3300/0 conn(0x7ff6cc0cc440 0x7ff6cc15cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.231 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 -- 172.21.15.162:0/2600997962 >> 172.21.15.162:0/2600997962 conn(0x7ff6cc0999d0 msgr2=0x7ff6cc0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.231 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 -- 172.21.15.162:0/2600997962 shutdown_connections 2024-04-25T10:01:29.231 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 -- 172.21.15.162:0/2600997962 wait complete. 2024-04-25T10:01:29.232 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 Processor -- start 2024-04-25T10:01:29.232 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 -- start start 2024-04-25T10:01:29.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6cc06ad20 0x7ff6cc1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc0cc440 0x7ff6cc0c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6cc15ba70 0x7ff6cc14f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0b0860 con 0x7ff6cc15ba70 2024-04-25T10:01:29.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0b06e0 con 0x7ff6cc06ad20 2024-04-25T10:01:29.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0b09e0 con 0x7ff6cc0cc440 2024-04-25T10:01:29.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d1b0e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6cc06ad20 0x7ff6cc1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d130d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc0cc440 0x7ff6cc0c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d130d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc0cc440 0x7ff6cc0c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:57268/0 (socket says 172.21.15.162:57268) 2024-04-25T10:01:29.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d1b0e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6cc06ad20 0x7ff6cc1512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46356/0 (socket says 172.21.15.162:46356) 2024-04-25T10:01:29.234 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d130d640 1 -- 172.21.15.162:0/1753200370 learned_addr learned my addr 172.21.15.162:0/1753200370 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d230f640 1 --2- 172.21.15.162:0/1753200370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6cc15ba70 0x7ff6cc14f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d130d640 1 -- 172.21.15.162:0/1753200370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6cc06ad20 msgr2=0x7ff6cc1512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d130d640 1 --2- 172.21.15.162:0/1753200370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6cc06ad20 0x7ff6cc1512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d130d640 1 -- 172.21.15.162:0/1753200370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6cc15ba70 msgr2=0x7ff6cc14f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d130d640 1 --2- 172.21.15.162:0/1753200370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6cc15ba70 0x7ff6cc14f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d130d640 1 -- 172.21.15.162:0/1753200370 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc0c6760 con 0x7ff6cc0cc440 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d1b0e640 1 --2- 172.21.15.162:0/1753200370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6cc06ad20 0x7ff6cc1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d130d640 1 --2- 172.21.15.162:0/1753200370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc0cc440 0x7ff6cc0c6110 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00c820 tx=0x7ff6bc00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6c27fc640 1 -- 172.21.15.162:0/1753200370 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc015070 con 0x7ff6cc0cc440 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6c27fc640 1 -- 172.21.15.162:0/1753200370 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6bc007a30 con 0x7ff6cc0cc440 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 -- 172.21.15.162:0/1753200370 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc0c6a40 con 0x7ff6cc0cc440 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6c27fc640 1 -- 172.21.15.162:0/1753200370 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc007e40 con 0x7ff6cc0cc440 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6d3d72640 1 -- 172.21.15.162:0/1753200370 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc158f00 con 0x7ff6cc0cc440 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6c27fc640 1 -- 172.21.15.162:0/1753200370 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff6bc018070 con 0x7ff6cc0cc440 2024-04-25T10:01:29.235 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.227+0000 7ff6c27fc640 1 -- 172.21.15.162:0/1753200370 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7ff6bc011970 con 0x7ff6cc0cc440 2024-04-25T10:01:29.236 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.231+0000 7ff6d3d72640 1 -- 172.21.15.162:0/1753200370 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6cc067c70 con 0x7ff6cc0cc440 2024-04-25T10:01:29.236 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.231+0000 7ff6c27fc640 1 -- 172.21.15.162:0/1753200370 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff6bc011cb0 con 0x7ff6cc0cc440 2024-04-25T10:01:29.304 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.299+0000 7ff6d3d72640 1 -- 172.21.15.162:0/1753200370 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "58d88fea-42ba-4586-853d-fcfcb2883e81", "id": 5} v 0) v1 -- 0x7ff6cc06e3c0 con 0x7ff6cc0cc440 2024-04-25T10:01:29.310 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.303+0000 7ff6c27fc640 1 -- 172.21.15.162:0/1753200370 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "58d88fea-42ba-4586-853d-fcfcb2883e81", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff6bc0113f0 con 0x7ff6cc0cc440 2024-04-25T10:01:29.310 INFO:teuthology.orchestra.run.smithi162.stdout:5 2024-04-25T10:01:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.307+0000 7ff6d3d72640 1 -- 172.21.15.162:0/1753200370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc0cc440 msgr2=0x7ff6cc0c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.307+0000 7ff6d3d72640 1 --2- 172.21.15.162:0/1753200370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc0cc440 0x7ff6cc0c6110 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00c820 tx=0x7ff6bc00cce0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.307+0000 7ff6d3d72640 1 -- 172.21.15.162:0/1753200370 shutdown_connections 2024-04-25T10:01:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.307+0000 7ff6d3d72640 1 --2- 172.21.15.162:0/1753200370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6cc15ba70 0x7ff6cc14f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.307+0000 7ff6d3d72640 1 --2- 172.21.15.162:0/1753200370 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6cc0cc440 0x7ff6cc0c6110 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.307+0000 7ff6d3d72640 1 --2- 172.21.15.162:0/1753200370 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6cc06ad20 0x7ff6cc1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.307+0000 7ff6d3d72640 1 -- 172.21.15.162:0/1753200370 >> 172.21.15.162:0/1753200370 conn(0x7ff6cc0999d0 msgr2=0x7ff6cc06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.307+0000 7ff6d3d72640 1 -- 172.21.15.162:0/1753200370 shutdown_connections 2024-04-25T10:01:29.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.307+0000 7ff6d3d72640 1 -- 172.21.15.162:0/1753200370 wait complete. 2024-04-25T10:01:29.328 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new ada17685-6a97-424f-822e-89180562d474 6 2024-04-25T10:01:29.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.427+0000 7f60da12c640 1 Processor -- start 2024-04-25T10:01:29.433 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.427+0000 7f60da12c640 1 -- start start 2024-04-25T10:01:29.435 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f60d40ced70 0x7f60d40ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.435 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f60d40ce1e0 0x7f60d415da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d40cb2e0 con 0x7f60d40ce990 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d40cb5e0 con 0x7f60d40ced70 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d40cd920 con 0x7f60d40ce1e0 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d3fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f60d40ce1e0 0x7f60d415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d3fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f60d40ce1e0 0x7f60d415da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:46370/0 (socket says 172.21.15.162:46370) 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d3fff640 1 -- 172.21.15.162:0/1669952499 learned_addr learned my addr 172.21.15.162:0/1669952499 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d2ffd640 1 --2- 172.21.15.162:0/1669952499 >> v2:172.21.15.133:3300/0 conn(0x7f60d40ced70 0x7f60d40ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d3fff640 1 -- 172.21.15.162:0/1669952499 >> v2:172.21.15.133:3300/0 conn(0x7f60d40ced70 msgr2=0x7f60d40ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d3fff640 1 --2- 172.21.15.162:0/1669952499 >> v2:172.21.15.133:3300/0 conn(0x7f60d40ced70 0x7f60d40ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d3fff640 1 -- 172.21.15.162:0/1669952499 >> v1:172.21.15.162:6789/0 conn(0x7f60d40ce990 legacy=0x7f60d40d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d3fff640 1 -- 172.21.15.162:0/1669952499 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d40cdbc0 con 0x7f60d40ce1e0 2024-04-25T10:01:29.436 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d2ffd640 1 --2- 172.21.15.162:0/1669952499 >> v2:172.21.15.133:3300/0 conn(0x7f60d40ced70 0x7f60d40ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:29.438 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d3fff640 1 --2- 172.21.15.162:0/1669952499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d40ce1e0 0x7f60d415da30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f60d40c7600 tx=0x7f60cc01c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb5ab6eb73d72620 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.438 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d27fc640 1 -- 172.21.15.162:0/1669952499 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0a58b0 con 0x7f60d40ce1e0 2024-04-25T10:01:29.438 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d27fc640 1 -- 172.21.15.162:0/1669952499 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60cc0a5a50 con 0x7f60d40ce1e0 2024-04-25T10:01:29.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60d27fc640 1 -- 172.21.15.162:0/1669952499 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0a5e20 con 0x7f60d40ce1e0 2024-04-25T10:01:29.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 -- 172.21.15.162:0/1669952499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d40ce1e0 msgr2=0x7f60d415da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 --2- 172.21.15.162:0/1669952499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d40ce1e0 0x7f60d415da30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f60d40c7600 tx=0x7f60cc01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 -- 172.21.15.162:0/1669952499 shutdown_connections 2024-04-25T10:01:29.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 --2- 172.21.15.162:0/1669952499 >> v2:172.21.15.133:3300/0 conn(0x7f60d40ced70 0x7f60d40ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 --2- 172.21.15.162:0/1669952499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d40ce1e0 0x7f60d415da30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 -- 172.21.15.162:0/1669952499 >> 172.21.15.162:0/1669952499 conn(0x7f60d40999d0 msgr2=0x7f60d40d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 -- 172.21.15.162:0/1669952499 shutdown_connections 2024-04-25T10:01:29.439 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.431+0000 7f60da12c640 1 -- 172.21.15.162:0/1669952499 wait complete. 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60da12c640 1 Processor -- start 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60da12c640 1 -- start start 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60da12c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d40ce1e0 0x7f60d41653d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60da12c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40ce990 0x7f60d40c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60da12c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d40ced70 0x7f60d414db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60da12c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40cc3e0 con 0x7f60d40ce1e0 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60da12c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d40cc260 con 0x7f60d40ced70 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60da12c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d4167960 con 0x7f60d40ce990 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d2ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40ce990 0x7f60d40c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d2ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40ce990 0x7f60d40c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:57270/0 (socket says 172.21.15.162:57270) 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d2ffd640 1 -- 172.21.15.162:0/3320192817 learned_addr learned my addr 172.21.15.162:0/3320192817 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:29.442 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d3fff640 1 --2- 172.21.15.162:0/3320192817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d40ced70 0x7f60d414db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d37fe640 1 --2- 172.21.15.162:0/3320192817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d40ce1e0 0x7f60d41653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d3fff640 1 -- 172.21.15.162:0/3320192817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40ce990 msgr2=0x7f60d40c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d3fff640 1 --2- 172.21.15.162:0/3320192817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40ce990 0x7f60d40c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d3fff640 1 -- 172.21.15.162:0/3320192817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d40ce1e0 msgr2=0x7f60d41653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d3fff640 1 --2- 172.21.15.162:0/3320192817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d40ce1e0 0x7f60d41653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d3fff640 1 -- 172.21.15.162:0/3320192817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d40aeb20 con 0x7f60d40ced70 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d2ffd640 1 --2- 172.21.15.162:0/3320192817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40ce990 0x7f60d40c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60d3fff640 1 --2- 172.21.15.162:0/3320192817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d40ced70 0x7f60d414db00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0ab000 tx=0x7f60cc019790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60cbfff640 1 -- 172.21.15.162:0/3320192817 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0a58b0 con 0x7f60d40ced70 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60cbfff640 1 -- 172.21.15.162:0/3320192817 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60cc0090d0 con 0x7f60d40ced70 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60da12c640 1 -- 172.21.15.162:0/3320192817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d4164e60 con 0x7f60d40ced70 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60cbfff640 1 -- 172.21.15.162:0/3320192817 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc009510 con 0x7f60d40ced70 2024-04-25T10:01:29.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60da12c640 1 -- 172.21.15.162:0/3320192817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d4155340 con 0x7f60d40ced70 2024-04-25T10:01:29.444 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60cbfff640 1 -- 172.21.15.162:0/3320192817 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f60cc0c0020 con 0x7f60d40ced70 2024-04-25T10:01:29.444 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60cbfff640 1 -- 172.21.15.162:0/3320192817 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f60cc0c2aa0 con 0x7f60d40ced70 2024-04-25T10:01:29.444 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.435+0000 7f60da12c640 1 -- 172.21.15.162:0/3320192817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d4067c70 con 0x7f60d40ced70 2024-04-25T10:01:29.444 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.439+0000 7f60cbfff640 1 -- 172.21.15.162:0/3320192817 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f60cc0c7020 con 0x7f60d40ced70 2024-04-25T10:01:29.541 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.535+0000 7f60da12c640 1 -- 172.21.15.162:0/3320192817 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ada17685-6a97-424f-822e-89180562d474", "id": 6} v 0) v1 -- 0x7f60d40b6520 con 0x7f60d40ced70 2024-04-25T10:01:29.548 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.543+0000 7f60cbfff640 1 -- 172.21.15.162:0/3320192817 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ada17685-6a97-424f-822e-89180562d474", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f60cc0165e0 con 0x7f60d40ced70 2024-04-25T10:01:29.548 INFO:teuthology.orchestra.run.smithi162.stdout:6 2024-04-25T10:01:29.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.551+0000 7f60da12c640 1 -- 172.21.15.162:0/3320192817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d40ced70 msgr2=0x7f60d414db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.551+0000 7f60da12c640 1 --2- 172.21.15.162:0/3320192817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d40ced70 0x7f60d414db00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0ab000 tx=0x7f60cc019790 comp rx=0 tx=0).stop 2024-04-25T10:01:29.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.551+0000 7f60da12c640 1 -- 172.21.15.162:0/3320192817 shutdown_connections 2024-04-25T10:01:29.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.551+0000 7f60da12c640 1 --2- 172.21.15.162:0/3320192817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60d40ced70 0x7f60d414db00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.551+0000 7f60da12c640 1 --2- 172.21.15.162:0/3320192817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d40ce990 0x7f60d40c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.555+0000 7f60da12c640 1 --2- 172.21.15.162:0/3320192817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d40ce1e0 0x7f60d41653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.555+0000 7f60da12c640 1 -- 172.21.15.162:0/3320192817 >> 172.21.15.162:0/3320192817 conn(0x7f60d40999d0 msgr2=0x7f60d40ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.555+0000 7f60da12c640 1 -- 172.21.15.162:0/3320192817 shutdown_connections 2024-04-25T10:01:29.559 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.555+0000 7f60da12c640 1 -- 172.21.15.162:0/3320192817 wait complete. 2024-04-25T10:01:29.577 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd new 11a90ffe-04b2-4c01-aa42-faaf9fd347d8 7 2024-04-25T10:01:29.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f77971c4640 1 Processor -- start 2024-04-25T10:01:29.680 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f77971c4640 1 -- start start 2024-04-25T10:01:29.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f77971c4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77900cc440 0x7f779015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f77971c4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779006baa0 con 0x7f779015ba70 2024-04-25T10:01:29.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f77971c4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779015c100 con 0x7f779006ad20 2024-04-25T10:01:29.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f77971c4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779015beb0 con 0x7f77900cc440 2024-04-25T10:01:29.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f7794f60640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77900cc440 0x7f779015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f7794f60640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77900cc440 0x7f779015cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:57272/0 (socket says 172.21.15.162:57272) 2024-04-25T10:01:29.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f7794f60640 1 -- 172.21.15.162:0/4086506039 learned_addr learned my addr 172.21.15.162:0/4086506039 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:29.681 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f7794f60640 1 -- 172.21.15.162:0/4086506039 >> v1:172.21.15.162:6789/0 conn(0x7f779006ad20 legacy=0x7f779006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.682 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f7794f60640 1 -- 172.21.15.162:0/4086506039 >> v1:172.21.15.133:6789/0 conn(0x7f779015ba70 legacy=0x7f77900cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:29.682 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.675+0000 7f7794f60640 1 -- 172.21.15.162:0/4086506039 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779015c3e0 con 0x7f77900cc440 2024-04-25T10:01:29.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f7794f60640 1 --2- 172.21.15.162:0/4086506039 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77900cc440 0x7f779015cbe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f77880098d0 tx=0x7f778801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d618150db5bfac25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f778f7fe640 1 -- 172.21.15.162:0/4086506039 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77880ab650 con 0x7f77900cc440 2024-04-25T10:01:29.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f778f7fe640 1 -- 172.21.15.162:0/4086506039 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77880b0070 con 0x7f77900cc440 2024-04-25T10:01:29.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f778f7fe640 1 -- 172.21.15.162:0/4086506039 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77880aba10 con 0x7f77900cc440 2024-04-25T10:01:29.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 -- 172.21.15.162:0/4086506039 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77900cc440 msgr2=0x7f779015cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 --2- 172.21.15.162:0/4086506039 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77900cc440 0x7f779015cbe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f77880098d0 tx=0x7f778801c540 comp rx=0 tx=0).stop 2024-04-25T10:01:29.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 -- 172.21.15.162:0/4086506039 shutdown_connections 2024-04-25T10:01:29.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 --2- 172.21.15.162:0/4086506039 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77900cc440 0x7f779015cbe0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 -- 172.21.15.162:0/4086506039 >> 172.21.15.162:0/4086506039 conn(0x7f77900999d0 msgr2=0x7f77900d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 -- 172.21.15.162:0/4086506039 shutdown_connections 2024-04-25T10:01:29.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 -- 172.21.15.162:0/4086506039 wait complete. 2024-04-25T10:01:29.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 Processor -- start 2024-04-25T10:01:29.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 -- start start 2024-04-25T10:01:29.687 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779006ad20 0x7f77900b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.687 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77900cc440 0x7f77900add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.687 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015ba70 0x7f77900ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.687 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77900b0af0 con 0x7f779015ba70 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77900b0970 con 0x7f77900cc440 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f77971c4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77900b0c70 con 0x7f779006ad20 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f7794f60640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779006ad20 0x7f77900b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f7794f60640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779006ad20 0x7f77900b33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.162:57282/0 (socket says 172.21.15.162:57282) 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f778ffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77900cc440 0x7f77900add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f7794f60640 1 -- 172.21.15.162:0/4260023568 learned_addr learned my addr 172.21.15.162:0/4260023568 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f7795761640 1 --2- 172.21.15.162:0/4260023568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015ba70 0x7f77900ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f7794f60640 1 -- 172.21.15.162:0/4260023568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77900cc440 msgr2=0x7f77900add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f7794f60640 1 --2- 172.21.15.162:0/4260023568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77900cc440 0x7f77900add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f7794f60640 1 -- 172.21.15.162:0/4260023568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015ba70 msgr2=0x7f77900ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f7794f60640 1 --2- 172.21.15.162:0/4260023568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015ba70 0x7f77900ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f7794f60640 1 -- 172.21.15.162:0/4260023568 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77900b3930 con 0x7f779006ad20 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.679+0000 7f7795761640 1 --2- 172.21.15.162:0/4260023568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015ba70 0x7f77900ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.683+0000 7f7794f60640 1 --2- 172.21.15.162:0/4260023568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779006ad20 0x7f77900b33f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f77880ad040 tx=0x7f778801ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.688 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.683+0000 7f778d7fa640 1 -- 172.21.15.162:0/4260023568 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77880b48c0 con 0x7f779006ad20 2024-04-25T10:01:29.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.683+0000 7f778d7fa640 1 -- 172.21.15.162:0/4260023568 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77880b4a60 con 0x7f779006ad20 2024-04-25T10:01:29.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.683+0000 7f77971c4640 1 -- 172.21.15.162:0/4260023568 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77900ce210 con 0x7f779006ad20 2024-04-25T10:01:29.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.683+0000 7f778d7fa640 1 -- 172.21.15.162:0/4260023568 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77880b4e30 con 0x7f779006ad20 2024-04-25T10:01:29.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.683+0000 7f77971c4640 1 -- 172.21.15.162:0/4260023568 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77900ce7d0 con 0x7f779006ad20 2024-04-25T10:01:29.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.683+0000 7f778d7fa640 1 -- 172.21.15.162:0/4260023568 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f77880be780 con 0x7f779006ad20 2024-04-25T10:01:29.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.683+0000 7f778d7fa640 1 -- 172.21.15.162:0/4260023568 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f77880c7390 con 0x7f779006ad20 2024-04-25T10:01:29.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.683+0000 7f77971c4640 1 -- 172.21.15.162:0/4260023568 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7790067c70 con 0x7f779006ad20 2024-04-25T10:01:29.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.683+0000 7f778d7fa640 1 -- 172.21.15.162:0/4260023568 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f77880c4060 con 0x7f779006ad20 2024-04-25T10:01:29.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.751+0000 7f77971c4640 1 -- 172.21.15.162:0/4260023568 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd new", "uuid": "11a90ffe-04b2-4c01-aa42-faaf9fd347d8", "id": 7} v 0) v1 -- 0x7f779006fef0 con 0x7f779006ad20 2024-04-25T10:01:29.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.755+0000 7f778d7fa640 1 -- 172.21.15.162:0/4260023568 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "11a90ffe-04b2-4c01-aa42-faaf9fd347d8", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7788016130 con 0x7f779006ad20 2024-04-25T10:01:29.760 INFO:teuthology.orchestra.run.smithi162.stdout:7 2024-04-25T10:01:29.761 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.755+0000 7f77971c4640 1 -- 172.21.15.162:0/4260023568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779006ad20 msgr2=0x7f77900b33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.755+0000 7f77971c4640 1 --2- 172.21.15.162:0/4260023568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779006ad20 0x7f77900b33f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f77880ad040 tx=0x7f778801ad80 comp rx=0 tx=0).stop 2024-04-25T10:01:29.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.755+0000 7f77971c4640 1 -- 172.21.15.162:0/4260023568 shutdown_connections 2024-04-25T10:01:29.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.755+0000 7f77971c4640 1 --2- 172.21.15.162:0/4260023568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015ba70 0x7f77900ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.755+0000 7f77971c4640 1 --2- 172.21.15.162:0/4260023568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77900cc440 0x7f77900add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.755+0000 7f77971c4640 1 --2- 172.21.15.162:0/4260023568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779006ad20 0x7f77900b33f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.755+0000 7f77971c4640 1 -- 172.21.15.162:0/4260023568 >> 172.21.15.162:0/4260023568 conn(0x7f77900999d0 msgr2=0x7f77900b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.755+0000 7f77971c4640 1 -- 172.21.15.162:0/4260023568 shutdown_connections 2024-04-25T10:01:29.762 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:01:29.755+0000 7f77971c4640 1 -- 172.21.15.162:0/4260023568 wait complete. 2024-04-25T10:01:29.776 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:01:29.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:01:29.778 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:01:29.779 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:01:29.779 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:01:29.781 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:01:29.781 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:01:29.781 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:01:29.784 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:01:29.784 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:01:29.784 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:01:29.787 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:01:29.787 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:01:29.787 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:01:29.789 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:01:29.789 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:01:29.789 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:01:29.791 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:01:29.791 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:01:29.791 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:01:29.793 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:01:29.793 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:01:29.793 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:01:29.795 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:01:29.796 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:29.814 INFO:tasks.ceph.mgr.y.smithi133.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T10:01:29.814 INFO:tasks.ceph.mgr.y.smithi133.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T10:01:29.814 INFO:tasks.ceph.mgr.y.smithi133.stderr: from numpy import show_config as show_numpy_config 2024-04-25T10:01:29.820 INFO:tasks.ceph.mgr.x.smithi162.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T10:01:29.821 INFO:tasks.ceph.mgr.x.smithi162.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T10:01:29.821 INFO:tasks.ceph.mgr.x.smithi162.stderr: from numpy import show_config as show_numpy_config 2024-04-25T10:01:29.863 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:29.857+0000 7f16e84e3180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:01:29.881 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:29.875+0000 7f60cbf65180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:01:29.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f3345f79640 1 Processor -- start 2024-04-25T10:01:29.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f3345f79640 1 -- start start 2024-04-25T10:01:29.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f3345f79640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f33400b4b70 0x7f334015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f3345f79640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f33400b3f40 0x7f334015e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f3345f79640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334015b490 con 0x7f334015eeb0 2024-04-25T10:01:29.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f3345f79640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3340155150 con 0x7f33400b3f40 2024-04-25T10:01:29.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f3345f79640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334015e200 con 0x7f33400b4b70 2024-04-25T10:01:29.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f333effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f33400b3f40 0x7f334015e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f333ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f334015eeb0 0x7f33400ae410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:39734/0 (socket says 172.21.15.133:39734) 2024-04-25T10:01:29.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f333effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f33400b3f40 0x7f334015e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51048/0 (socket says 172.21.15.133:51048) 2024-04-25T10:01:29.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f333ffff640 1 -- 172.21.15.133:0/2444851699 learned_addr learned my addr 172.21.15.133:0/2444851699 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:01:29.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f333f7fe640 1 --2- 172.21.15.133:0/2444851699 >> v2:172.21.15.162:3300/0 conn(0x7f33400b4b70 0x7f334015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f333effd640 1 -- 172.21.15.133:0/2444851699 >> v2:172.21.15.162:3300/0 conn(0x7f33400b4b70 msgr2=0x7f334015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:29.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f333effd640 1 --2- 172.21.15.133:0/2444851699 >> v2:172.21.15.162:3300/0 conn(0x7f33400b4b70 0x7f334015ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f333effd640 1 -- 172.21.15.133:0/2444851699 >> v1:172.21.15.133:6789/0 conn(0x7f334015eeb0 legacy=0x7f33400ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f333effd640 1 -- 172.21.15.133:0/2444851699 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334015b770 con 0x7f33400b3f40 2024-04-25T10:01:29.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f333e7fc640 1 -- 172.21.15.133:0/2444851699 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892176129 0 0) 0x7f334015b490 con 0x7f334015eeb0 2024-04-25T10:01:29.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.909+0000 7f333f7fe640 1 --2- 172.21.15.133:0/2444851699 >> v2:172.21.15.162:3300/0 conn(0x7f33400b4b70 0x7f334015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:29.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f333effd640 1 --2- 172.21.15.133:0/2444851699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33400b3f40 0x7f334015e970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f334015e380 tx=0x7f333001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6405c427dd0c98c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f333e7fc640 1 -- 172.21.15.133:0/2444851699 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300b0070 con 0x7f33400b3f40 2024-04-25T10:01:29.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f333e7fc640 1 -- 172.21.15.133:0/2444851699 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33300a4e50 con 0x7f33400b3f40 2024-04-25T10:01:29.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f333e7fc640 1 -- 172.21.15.133:0/2444851699 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300ab820 con 0x7f33400b3f40 2024-04-25T10:01:29.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f3345f79640 1 -- 172.21.15.133:0/2444851699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33400b3f40 msgr2=0x7f334015e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f3345f79640 1 --2- 172.21.15.133:0/2444851699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33400b3f40 0x7f334015e970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f334015e380 tx=0x7f333001c340 comp rx=0 tx=0).stop 2024-04-25T10:01:29.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f3345f79640 1 -- 172.21.15.133:0/2444851699 shutdown_connections 2024-04-25T10:01:29.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f3345f79640 1 --2- 172.21.15.133:0/2444851699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33400b3f40 0x7f334015e970 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f3345f79640 1 --2- 172.21.15.133:0/2444851699 >> v2:172.21.15.162:3300/0 conn(0x7f33400b4b70 0x7f334015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f3345f79640 1 -- 172.21.15.133:0/2444851699 >> 172.21.15.133:0/2444851699 conn(0x7f3340071aa0 msgr2=0x7f3340071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f3345f79640 1 -- 172.21.15.133:0/2444851699 shutdown_connections 2024-04-25T10:01:29.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.913+0000 7f3345f79640 1 -- 172.21.15.133:0/2444851699 wait complete. 2024-04-25T10:01:29.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3345f79640 1 Processor -- start 2024-04-25T10:01:29.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3345f79640 1 -- start start 2024-04-25T10:01:29.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3345f79640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33400b3f40 0x7f3340148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3345f79640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33400b4b70 0x7f3340157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3345f79640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3340157750 0x7f33401552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3345f79640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3340165590 con 0x7f33400b4b70 2024-04-25T10:01:29.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3345f79640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400aed30 con 0x7f33400b3f40 2024-04-25T10:01:29.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3345f79640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3340165710 con 0x7f3340157750 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f333effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33400b4b70 0x7f3340157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f333effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33400b4b70 0x7f3340157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51052/0 (socket says 172.21.15.133:51052) 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f333effd640 1 -- 172.21.15.133:0/2594884946 learned_addr learned my addr 172.21.15.133:0/2594884946 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f333f7fe640 1 --2- 172.21.15.133:0/2594884946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33400b3f40 0x7f3340148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f333ffff640 1 --2- 172.21.15.133:0/2594884946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3340157750 0x7f33401552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f333effd640 1 -- 172.21.15.133:0/2594884946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3340157750 msgr2=0x7f33401552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f333effd640 1 --2- 172.21.15.133:0/2594884946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3340157750 0x7f33401552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f333effd640 1 -- 172.21.15.133:0/2594884946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33400b3f40 msgr2=0x7f3340148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f333effd640 1 --2- 172.21.15.133:0/2594884946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33400b3f40 0x7f3340148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f333effd640 1 -- 172.21.15.133:0/2594884946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3340157d60 con 0x7f33400b4b70 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f333effd640 1 --2- 172.21.15.133:0/2594884946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33400b4b70 0x7f3340157210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f33300ae040 tx=0x7f33300190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3327fff640 1 -- 172.21.15.133:0/2594884946 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300b0040 con 0x7f33400b4b70 2024-04-25T10:01:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3327fff640 1 -- 172.21.15.133:0/2594884946 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33300b9070 con 0x7f33400b4b70 2024-04-25T10:01:29.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3345f79640 1 -- 172.21.15.133:0/2594884946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3340158040 con 0x7f33400b4b70 2024-04-25T10:01:29.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3327fff640 1 -- 172.21.15.133:0/2594884946 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300b4870 con 0x7f33400b4b70 2024-04-25T10:01:29.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3345f79640 1 -- 172.21.15.133:0/2594884946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334014d9d0 con 0x7f33400b4b70 2024-04-25T10:01:29.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3327fff640 1 -- 172.21.15.133:0/2594884946 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f33300b4a10 con 0x7f33400b4b70 2024-04-25T10:01:29.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3327fff640 1 -- 172.21.15.133:0/2594884946 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f33300beaf0 con 0x7f33400b4b70 2024-04-25T10:01:29.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.917+0000 7f3345f79640 1 -- 172.21.15.133:0/2594884946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3340067c70 con 0x7f33400b4b70 2024-04-25T10:01:29.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.925+0000 7f3327fff640 1 -- 172.21.15.133:0/2594884946 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3330016050 con 0x7f33400b4b70 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.981+0000 7f3345f79640 1 -- 172.21.15.133:0/2594884946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3340066b50 con 0x7f33400b4b70 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.985+0000 7f3327fff640 1 -- 172.21.15.133:0/2594884946 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f33300a4df0 con 0x7f33400b4b70 2024-04-25T10:01:29.991 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:01:29.991 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:29.753511+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.985+0000 7f3345f79640 1 -- 172.21.15.133:0/2594884946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33400b4b70 msgr2=0x7f3340157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.985+0000 7f3345f79640 1 --2- 172.21.15.133:0/2594884946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33400b4b70 0x7f3340157210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f33300ae040 tx=0x7f33300190e0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.985+0000 7f3345f79640 1 -- 172.21.15.133:0/2594884946 shutdown_connections 2024-04-25T10:01:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.985+0000 7f3345f79640 1 --2- 172.21.15.133:0/2594884946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3340157750 0x7f33401552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.985+0000 7f3345f79640 1 --2- 172.21.15.133:0/2594884946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33400b4b70 0x7f3340157210 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.985+0000 7f3345f79640 1 --2- 172.21.15.133:0/2594884946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33400b3f40 0x7f3340148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.985+0000 7f3345f79640 1 -- 172.21.15.133:0/2594884946 >> 172.21.15.133:0/2594884946 conn(0x7f3340071aa0 msgr2=0x7f334006fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.985+0000 7f3345f79640 1 -- 172.21.15.133:0/2594884946 shutdown_connections 2024-04-25T10:01:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:29.985+0000 7f3345f79640 1 -- 172.21.15.133:0/2594884946 wait complete. 2024-04-25T10:01:30.001 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:29.993+0000 7f16e84e3180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:30.007 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:01:30.007 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:01:30.029 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:30.023+0000 7f60cbf65180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:30.168 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:30.161+0000 7f16e84e3180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:30.190 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:30.183+0000 7f60cbf65180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:30.231 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:30.225+0000 7f16e84e3180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:01:30.266 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:30.259+0000 7f60cbf65180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:01:30.364 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:30.357+0000 7f16e84e3180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:30.399 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:30.395+0000 7f60cbf65180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:30.404 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-04-25T10:01:30.399+0000 7ffb9eed47c0 -1 Falling back to public interface 2024-04-25T10:01:30.404 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-04-25T10:01:30.399+0000 7ff785c3a7c0 -1 Falling back to public interface 2024-04-25T10:01:30.404 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-04-25T10:01:30.399+0000 7ffb9de94640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-25T10:01:30.404 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-04-25T10:01:30.399+0000 7f4bbae217c0 -1 Falling back to public interface 2024-04-25T10:01:30.405 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-04-25T10:01:30.397+0000 7fdcd254d7c0 -1 Falling back to public interface 2024-04-25T10:01:30.405 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-04-25T10:01:30.397+0000 7f1cc5e997c0 -1 Falling back to public interface 2024-04-25T10:01:30.405 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-04-25T10:01:30.397+0000 7f1cc4e59640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-25T10:01:30.414 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-04-25T10:01:30.409+0000 7f008d9717c0 -1 Falling back to public interface 2024-04-25T10:01:30.414 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-04-25T10:01:30.409+0000 7f059977d7c0 -1 Falling back to public interface 2024-04-25T10:01:30.415 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-04-25T10:01:30.411+0000 7f53427e97c0 -1 Falling back to public interface 2024-04-25T10:01:30.812 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:30.921 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.913+0000 7fd6549b4640 1 Processor -- start 2024-04-25T10:01:30.921 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.913+0000 7fd6549b4640 1 -- start start 2024-04-25T10:01:30.923 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd6549b4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd65015b750 0x7fd65015eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.923 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd6549b4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd6500b1630 0x7fd6500cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.923 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd6549b4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6500cfcd0 0x7fd65006ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.923 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd6549b4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd650161eb0 con 0x7fd6500b1630 2024-04-25T10:01:30.923 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd6549b4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6500b1e20 con 0x7fd65015b750 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd6549b4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd650161580 con 0x7fd6500cfcd0 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd64dd9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd6500b1630 0x7fd6500cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd64dd9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd6500b1630 0x7fd6500cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51068/0 (socket says 172.21.15.133:51068) 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd64dd9b640 1 -- 172.21.15.133:0/2643346598 learned_addr learned my addr 172.21.15.133:0/2643346598 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd64ed9d640 1 --2- 172.21.15.133:0/2643346598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6500cfcd0 0x7fd65006ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd64e59c640 1 --2- 172.21.15.133:0/2643346598 >> v2:172.21.15.162:3300/0 conn(0x7fd65015b750 0x7fd65015eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd64dd9b640 1 -- 172.21.15.133:0/2643346598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6500cfcd0 msgr2=0x7fd65006ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd64dd9b640 1 --2- 172.21.15.133:0/2643346598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6500cfcd0 0x7fd65006ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd64dd9b640 1 -- 172.21.15.133:0/2643346598 >> v2:172.21.15.162:3300/0 conn(0x7fd65015b750 msgr2=0x7fd65015eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd64dd9b640 1 --2- 172.21.15.133:0/2643346598 >> v2:172.21.15.162:3300/0 conn(0x7fd65015b750 0x7fd65015eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd64dd9b640 1 -- 172.21.15.133:0/2643346598 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd650161860 con 0x7fd6500b1630 2024-04-25T10:01:30.924 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.917+0000 7fd64e59c640 1 --2- 172.21.15.133:0/2643346598 >> v2:172.21.15.162:3300/0 conn(0x7fd65015b750 0x7fd65015eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:30.925 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64dd9b640 1 --2- 172.21.15.133:0/2643346598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6500b1630 0x7fd6500cf790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd650072ef0 tx=0x7fd64401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b091f7bfbbfeb3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.925 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64d59a640 1 -- 172.21.15.133:0/2643346598 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6440b0070 con 0x7fd6500b1630 2024-04-25T10:01:30.925 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64d59a640 1 -- 172.21.15.133:0/2643346598 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6440ab600 con 0x7fd6500b1630 2024-04-25T10:01:30.926 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64d59a640 1 -- 172.21.15.133:0/2643346598 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6440ab9d0 con 0x7fd6500b1630 2024-04-25T10:01:30.926 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 -- 172.21.15.133:0/2643346598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6500b1630 msgr2=0x7fd6500cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.926 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 --2- 172.21.15.133:0/2643346598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6500b1630 0x7fd6500cf790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd650072ef0 tx=0x7fd64401c230 comp rx=0 tx=0).stop 2024-04-25T10:01:30.927 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 -- 172.21.15.133:0/2643346598 shutdown_connections 2024-04-25T10:01:30.927 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 --2- 172.21.15.133:0/2643346598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6500cfcd0 0x7fd65006ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.927 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 --2- 172.21.15.133:0/2643346598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6500b1630 0x7fd6500cf790 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.927 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 --2- 172.21.15.133:0/2643346598 >> v2:172.21.15.162:3300/0 conn(0x7fd65015b750 0x7fd65015eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.927 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 -- 172.21.15.133:0/2643346598 >> 172.21.15.133:0/2643346598 conn(0x7fd650099a40 msgr2=0x7fd6500b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.927 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 -- 172.21.15.133:0/2643346598 shutdown_connections 2024-04-25T10:01:30.927 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 -- 172.21.15.133:0/2643346598 wait complete. 2024-04-25T10:01:30.928 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 Processor -- start 2024-04-25T10:01:30.928 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 -- start start 2024-04-25T10:01:30.930 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6500cfcd0 0x7fd650150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.930 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd65015b750 0x7fd6500b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.930 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6500b3dc0 0x7fd6500b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.930 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6501624a0 con 0x7fd65015b750 2024-04-25T10:01:30.930 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd650162320 con 0x7fd6500cfcd0 2024-04-25T10:01:30.930 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd6549b4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6500cab90 con 0x7fd6500b3dc0 2024-04-25T10:01:30.930 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd65015b750 0x7fd6500b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.930 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd65015b750 0x7fd6500b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51076/0 (socket says 172.21.15.133:51076) 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64dd9b640 1 -- 172.21.15.133:0/2741177977 learned_addr learned my addr 172.21.15.133:0/2741177977 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64ed9d640 1 --2- 172.21.15.133:0/2741177977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6500b3dc0 0x7fd6500b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64e59c640 1 --2- 172.21.15.133:0/2741177977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6500cfcd0 0x7fd650150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64dd9b640 1 -- 172.21.15.133:0/2741177977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6500b3dc0 msgr2=0x7fd6500b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64dd9b640 1 --2- 172.21.15.133:0/2741177977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6500b3dc0 0x7fd6500b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64dd9b640 1 -- 172.21.15.133:0/2741177977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6500cfcd0 msgr2=0x7fd650150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64dd9b640 1 --2- 172.21.15.133:0/2741177977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6500cfcd0 0x7fd650150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64dd9b640 1 -- 172.21.15.133:0/2741177977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd650152440 con 0x7fd65015b750 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.921+0000 7fd64dd9b640 1 --2- 172.21.15.133:0/2741177977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd65015b750 0x7fd6500b3880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd64401cad0 tx=0x7fd644018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.925+0000 7fd63effd640 1 -- 172.21.15.133:0/2741177977 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6440bb070 con 0x7fd65015b750 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.925+0000 7fd63effd640 1 -- 172.21.15.133:0/2741177977 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd644004030 con 0x7fd65015b750 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.925+0000 7fd6549b4640 1 -- 172.21.15.133:0/2741177977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd650150d70 con 0x7fd65015b750 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.925+0000 7fd63effd640 1 -- 172.21.15.133:0/2741177977 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6440b0040 con 0x7fd65015b750 2024-04-25T10:01:30.931 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.925+0000 7fd6549b4640 1 -- 172.21.15.133:0/2741177977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65015ad10 con 0x7fd65015b750 2024-04-25T10:01:30.932 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.925+0000 7fd63effd640 1 -- 172.21.15.133:0/2741177977 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd6440be070 con 0x7fd65015b750 2024-04-25T10:01:30.932 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.925+0000 7fd63effd640 1 -- 172.21.15.133:0/2741177977 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd6440b6b40 con 0x7fd65015b750 2024-04-25T10:01:30.932 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.925+0000 7fd6549b4640 1 -- 172.21.15.133:0/2741177977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd650074de0 con 0x7fd65015b750 2024-04-25T10:01:30.932 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.925+0000 7fd63effd640 1 -- 172.21.15.133:0/2741177977 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd650150d70 con 0x7fd65015b750 2024-04-25T10:01:30.979 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:30.973+0000 7f16e84e3180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:01:30.989 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.985+0000 7fd6549b4640 1 -- 172.21.15.133:0/2741177977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd650099f20 con 0x7fd65015b750 2024-04-25T10:01:30.990 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.985+0000 7fd63effd640 1 -- 172.21.15.133:0/2741177977 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fd6440c6090 con 0x7fd65015b750 2024-04-25T10:01:30.990 INFO:teuthology.misc.health.smithi133.stdout: 2024-04-25T10:01:30.990 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":10,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:29.753511+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:30.991 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.985+0000 7fd6549b4640 1 -- 172.21.15.133:0/2741177977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd65015b750 msgr2=0x7fd6500b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.991 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.985+0000 7fd6549b4640 1 --2- 172.21.15.133:0/2741177977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd65015b750 0x7fd6500b3880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd64401cad0 tx=0x7fd644018fd0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.991 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.985+0000 7fd6549b4640 1 -- 172.21.15.133:0/2741177977 shutdown_connections 2024-04-25T10:01:30.991 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.985+0000 7fd6549b4640 1 --2- 172.21.15.133:0/2741177977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6500b3dc0 0x7fd6500b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.991 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.985+0000 7fd6549b4640 1 --2- 172.21.15.133:0/2741177977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd65015b750 0x7fd6500b3880 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.991 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.985+0000 7fd6549b4640 1 --2- 172.21.15.133:0/2741177977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6500cfcd0 0x7fd650150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.991 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.985+0000 7fd6549b4640 1 -- 172.21.15.133:0/2741177977 >> 172.21.15.133:0/2741177977 conn(0x7fd650099a40 msgr2=0x7fd6500b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.991 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.985+0000 7fd6549b4640 1 -- 172.21.15.133:0/2741177977 shutdown_connections 2024-04-25T10:01:30.992 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:30.985+0000 7fd6549b4640 1 -- 172.21.15.133:0/2741177977 wait complete. 2024-04-25T10:01:31.006 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:01:31.022 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:31.015+0000 7f60cbf65180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:01:31.064 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:31.057+0000 7f16e84e3180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:01:31.099 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:31.095+0000 7f60cbf65180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:01:31.250 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:31.245+0000 7f16e84e3180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:01:31.280 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:31.275+0000 7f60cbf65180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:01:31.323 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:31.317+0000 7f16e84e3180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:01:31.354 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:31.347+0000 7f60cbf65180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:01:31.613 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:31.605+0000 7f16e84e3180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:01:31.644 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:31.639+0000 7f60cbf65180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:01:31.675 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:31.669+0000 7f16e84e3180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:01:31.708 INFO:tasks.ceph.mgr.x.smithi162.stderr:2024-04-25T10:01:31.703+0000 7f60cbf65180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:01:32.719 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:32.713+0000 7f168fa31640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:01:32.720 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-04-25T10:01:32.713+0000 7f168fa31640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:01:34.186 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-04-25T10:01:34.181+0000 7fdcd254d7c0 -1 osd.1 0 log_to_monitors true 2024-04-25T10:01:34.269 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-04-25T10:01:34.265+0000 7f008d9717c0 -1 osd.0 0 log_to_monitors true 2024-04-25T10:01:34.309 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-04-25T10:01:34.301+0000 7f1cc5e997c0 -1 osd.3 0 log_to_monitors true 2024-04-25T10:01:34.319 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-04-25T10:01:34.315+0000 7ffb9eed47c0 -1 osd.4 0 log_to_monitors true 2024-04-25T10:01:34.320 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-04-25T10:01:34.313+0000 7f059977d7c0 -1 osd.2 0 log_to_monitors true 2024-04-25T10:01:34.330 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-04-25T10:01:34.323+0000 7f53427e97c0 -1 osd.6 0 log_to_monitors true 2024-04-25T10:01:34.337 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-04-25T10:01:34.331+0000 7ff785c3a7c0 -1 osd.7 0 log_to_monitors true 2024-04-25T10:01:34.391 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-04-25T10:01:34.387+0000 7f4bbae217c0 -1 osd.5 0 log_to_monitors true 2024-04-25T10:01:37.773 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-04-25T10:01:37.765+0000 7f008991a640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:01:37.798 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-04-25T10:01:37.793+0000 7f008174f640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:37.813 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:37.844 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-04-25T10:01:37.837+0000 7fdcce4f6640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:01:37.883 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-04-25T10:01:37.877+0000 7fdcc632b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:37.966 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-04-25T10:01:37.961+0000 7f1cc1e42640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:01:37.972 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.965+0000 7f83ed35a640 1 Processor -- start 2024-04-25T10:01:37.972 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.965+0000 7f83ed35a640 1 -- start start 2024-04-25T10:01:37.974 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83ed35a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f83e8160f80 0x7f83e80b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.974 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83ed35a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f83e80d0800 0x7f83e80d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.975 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83ed35a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e8161360 con 0x7f83e80d1940 2024-04-25T10:01:37.975 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83ed35a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e80d11a0 con 0x7f83e8160f80 2024-04-25T10:01:37.975 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83ed35a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e80d2550 con 0x7f83e80d0800 2024-04-25T10:01:37.975 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e6ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f83e8160f80 0x7f83e80b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.975 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e77fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f83e80d1940 0x7f83e8161600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51240/0 (socket says 172.21.15.133:51240) 2024-04-25T10:01:37.975 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e77fe640 1 -- 172.21.15.133:0/385828018 learned_addr learned my addr 172.21.15.133:0/385828018 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:01:37.975 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e6ffd640 1 -- 172.21.15.133:0/385828018 >> v2:172.21.15.162:3300/0 conn(0x7f83e80d0800 msgr2=0x7f83e80d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:37.975 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e6ffd640 1 --2- 172.21.15.133:0/385828018 >> v2:172.21.15.162:3300/0 conn(0x7f83e80d0800 0x7f83e80d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.975 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e5ffb640 1 -- 172.21.15.133:0/385828018 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103024288 0 0) 0x7f83e8161360 con 0x7f83e80d1940 2024-04-25T10:01:37.975 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e6ffd640 1 -- 172.21.15.133:0/385828018 >> v1:172.21.15.133:6789/0 conn(0x7f83e80d1940 legacy=0x7f83e8161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.976 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e6ffd640 1 -- 172.21.15.133:0/385828018 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80d2830 con 0x7f83e8160f80 2024-04-25T10:01:37.976 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e6ffd640 1 --2- 172.21.15.133:0/385828018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e8160f80 0x7f83e80b6920 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b6140 tx=0x7f83e001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd9b77dc9d5472a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.976 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e5ffb640 1 -- 172.21.15.133:0/385828018 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00b2070 con 0x7f83e8160f80 2024-04-25T10:01:37.977 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e5ffb640 1 -- 172.21.15.133:0/385828018 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e00a57f0 con 0x7f83e8160f80 2024-04-25T10:01:37.977 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.969+0000 7f83e5ffb640 1 -- 172.21.15.133:0/385828018 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00a5bc0 con 0x7f83e8160f80 2024-04-25T10:01:37.978 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 -- 172.21.15.133:0/385828018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e8160f80 msgr2=0x7f83e80b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.978 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 --2- 172.21.15.133:0/385828018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e8160f80 0x7f83e80b6920 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b6140 tx=0x7f83e001c930 comp rx=0 tx=0).stop 2024-04-25T10:01:37.978 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 -- 172.21.15.133:0/385828018 shutdown_connections 2024-04-25T10:01:37.978 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 --2- 172.21.15.133:0/385828018 >> v2:172.21.15.162:3300/0 conn(0x7f83e80d0800 0x7f83e80d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.978 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 --2- 172.21.15.133:0/385828018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e8160f80 0x7f83e80b6920 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.978 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 -- 172.21.15.133:0/385828018 >> 172.21.15.133:0/385828018 conn(0x7f83e80727a0 msgr2=0x7f83e8072bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.978 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 -- 172.21.15.133:0/385828018 shutdown_connections 2024-04-25T10:01:37.978 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 -- 172.21.15.133:0/385828018 wait complete. 2024-04-25T10:01:37.981 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 Processor -- start 2024-04-25T10:01:37.981 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 -- start start 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e80d0800 0x7f83e80b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83e80d1940 0x7f83e815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83e8160f80 0x7f83e80b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e816a8b0 con 0x7f83e80d0800 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e816a730 con 0x7f83e80d1940 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e816aa30 con 0x7f83e8160f80 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e80d0800 0x7f83e80b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e80d0800 0x7f83e80b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58426/0 (socket says 172.21.15.133:58426) 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e6ffd640 1 -- 172.21.15.133:0/208640401 learned_addr learned my addr 172.21.15.133:0/208640401 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e77fe640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83e8160f80 0x7f83e80b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e67fc640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83e80d1940 0x7f83e815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.982 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e6ffd640 1 -- 172.21.15.133:0/208640401 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83e8160f80 msgr2=0x7f83e80b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e6ffd640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83e8160f80 0x7f83e80b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e6ffd640 1 -- 172.21.15.133:0/208640401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83e80d1940 msgr2=0x7f83e815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e6ffd640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83e80d1940 0x7f83e815db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e6ffd640 1 -- 172.21.15.133:0/208640401 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80b4620 con 0x7f83e80d0800 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e67fc640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83e80d1940 0x7f83e815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83e6ffd640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e80d0800 0x7f83e80b40e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f83e001a560 tx=0x7f83e00196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83cf7fe640 1 -- 172.21.15.133:0/208640401 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00b2040 con 0x7f83e80d0800 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83cf7fe640 1 -- 172.21.15.133:0/208640401 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e00bf070 con 0x7f83e80d0800 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 -- 172.21.15.133:0/208640401 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e815cd70 con 0x7f83e80d0800 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83cf7fe640 1 -- 172.21.15.133:0/208640401 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00b8990 con 0x7f83e80d0800 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.973+0000 7f83ed35a640 1 -- 172.21.15.133:0/208640401 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e815d330 con 0x7f83e80d0800 2024-04-25T10:01:37.983 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.977+0000 7f83ed35a640 1 -- 172.21.15.133:0/208640401 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e8074de0 con 0x7f83e80d0800 2024-04-25T10:01:37.984 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.977+0000 7f83cf7fe640 1 -- 172.21.15.133:0/208640401 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) v1 ==== 86671+0+0 (secure 0 0 0) 0x7f83e00b8b30 con 0x7f83e80d0800 2024-04-25T10:01:37.984 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.977+0000 7f83cf7fe640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d4066ed0 0x7f83d4069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.984 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.977+0000 7f83cf7fe640 1 -- 172.21.15.133:0/208640401 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f83e011fb90 con 0x7f83e80d0800 2024-04-25T10:01:37.984 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.977+0000 7f83e67fc640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d4066ed0 0x7f83d4069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.984 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.977+0000 7f83e67fc640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d4066ed0 0x7f83d4069390 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b4dd0 tx=0x7f83dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.986 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:37.981+0000 7f83cf7fe640 1 -- 172.21.15.133:0/208640401 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83e0016520 con 0x7f83e80d0800 2024-04-25T10:01:38.002 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-04-25T10:01:37.997+0000 7f1cb9c77640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:38.014 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-04-25T10:01:38.009+0000 7f0595726640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:01:38.034 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-04-25T10:01:38.026+0000 7ff781be3640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:01:38.038 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-04-25T10:01:38.033+0000 7f058d55b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:38.043 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-04-25T10:01:38.038+0000 7f533e792640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:01:38.050 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-04-25T10:01:38.042+0000 7f4bb6dca640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:01:38.051 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-04-25T10:01:38.042+0000 7ffb9ae7d640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:01:38.059 INFO:tasks.ceph.osd.7.smithi162.stderr:2024-04-25T10:01:38.054+0000 7ff779a18640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:38.071 INFO:tasks.ceph.osd.5.smithi162.stderr:2024-04-25T10:01:38.066+0000 7f4baebff640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:38.072 INFO:tasks.ceph.osd.6.smithi162.stderr:2024-04-25T10:01:38.066+0000 7f53365c7640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:38.083 INFO:tasks.ceph.osd.4.smithi162.stderr:2024-04-25T10:01:38.078+0000 7ffb92cb2640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:38.142 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 -- 172.21.15.133:0/208640401 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83e806b000 con 0x7f83e80d0800 2024-04-25T10:01:38.143 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83cf7fe640 1 -- 172.21.15.133:0/208640401 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f83e00f1af0 con 0x7f83e80d0800 2024-04-25T10:01:38.143 INFO:teuthology.misc.health.smithi133.stdout: 2024-04-25T10:01:38.143 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":12,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:35.713951+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:38.145 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 -- 172.21.15.133:0/208640401 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d4066ed0 msgr2=0x7f83d4069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.145 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d4066ed0 0x7f83d4069390 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b4dd0 tx=0x7f83dc008040 comp rx=0 tx=0).stop 2024-04-25T10:01:38.145 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 -- 172.21.15.133:0/208640401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e80d0800 msgr2=0x7f83e80b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.145 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e80d0800 0x7f83e80b40e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f83e001a560 tx=0x7f83e00196d0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.146 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 -- 172.21.15.133:0/208640401 shutdown_connections 2024-04-25T10:01:38.146 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83e8160f80 0x7f83e80b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.146 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83e80d1940 0x7f83e815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.146 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d4066ed0 0x7f83d4069390 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.146 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 --2- 172.21.15.133:0/208640401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e80d0800 0x7f83e80b40e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.146 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 -- 172.21.15.133:0/208640401 >> 172.21.15.133:0/208640401 conn(0x7f83e80727a0 msgr2=0x7f83e80b6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.146 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 -- 172.21.15.133:0/208640401 shutdown_connections 2024-04-25T10:01:38.146 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:38.137+0000 7f83ed35a640 1 -- 172.21.15.133:0/208640401 wait complete. 2024-04-25T10:01:38.158 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:01:44.966 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:45.089 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.081+0000 7f25407e2640 1 Processor -- start 2024-04-25T10:01:45.089 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.081+0000 7f25407e2640 1 -- start start 2024-04-25T10:01:45.091 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f25407e2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2538157b90 0x7f253815af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.091 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f25407e2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f25380b1cb0 0x7f253815b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.091 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f25407e2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25380aedf0 con 0x7f2538071950 2024-04-25T10:01:45.091 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f25407e2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253815d880 con 0x7f25380b1cb0 2024-04-25T10:01:45.091 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f25407e2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25380b44d0 con 0x7f2538157b90 2024-04-25T10:01:45.091 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f253ed7f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2538071950 0x7f25380b49b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51256/0 (socket says 172.21.15.133:51256) 2024-04-25T10:01:45.091 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f253ed7f640 1 -- 172.21.15.133:0/1938282046 learned_addr learned my addr 172.21.15.133:0/1938282046 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:01:45.091 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f253dd7d640 1 --2- 172.21.15.133:0/1938282046 >> v2:172.21.15.133:3300/0 conn(0x7f25380b1cb0 0x7f253815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.092 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f253e57e640 1 --2- 172.21.15.133:0/1938282046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2538157b90 0x7f253815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.092 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f253d57c640 1 -- 172.21.15.133:0/1938282046 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544641885 0 0) 0x7f25380aedf0 con 0x7f2538071950 2024-04-25T10:01:45.092 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f253d57c640 1 -- 172.21.15.133:0/1938282046 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2524003530 con 0x7f2538071950 2024-04-25T10:01:45.092 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f253dd7d640 1 -- 172.21.15.133:0/1938282046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2538157b90 msgr2=0x7f253815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.092 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f253dd7d640 1 --2- 172.21.15.133:0/1938282046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2538157b90 0x7f253815af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.092 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f253dd7d640 1 -- 172.21.15.133:0/1938282046 >> v1:172.21.15.133:6789/0 conn(0x7f2538071950 legacy=0x7f25380b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.092 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f253dd7d640 1 -- 172.21.15.133:0/1938282046 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25380b27d0 con 0x7f25380b1cb0 2024-04-25T10:01:45.092 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.085+0000 7f253e57e640 1 --2- 172.21.15.133:0/1938282046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2538157b90 0x7f253815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:45.093 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253dd7d640 1 --2- 172.21.15.133:0/1938282046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25380b1cb0 0x7f253815b650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f25280097c0 tx=0x7f252801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9ac8052d1b20bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.094 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253d57c640 1 -- 172.21.15.133:0/1938282046 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25280b0070 con 0x7f25380b1cb0 2024-04-25T10:01:45.094 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253d57c640 1 -- 172.21.15.133:0/1938282046 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25280ab600 con 0x7f25380b1cb0 2024-04-25T10:01:45.094 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253d57c640 1 -- 172.21.15.133:0/1938282046 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25280ab9b0 con 0x7f25380b1cb0 2024-04-25T10:01:45.095 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 -- 172.21.15.133:0/1938282046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25380b1cb0 msgr2=0x7f253815b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.095 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 --2- 172.21.15.133:0/1938282046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25380b1cb0 0x7f253815b650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f25280097c0 tx=0x7f252801c410 comp rx=0 tx=0).stop 2024-04-25T10:01:45.095 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 -- 172.21.15.133:0/1938282046 shutdown_connections 2024-04-25T10:01:45.095 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 --2- 172.21.15.133:0/1938282046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25380b1cb0 0x7f253815b650 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.095 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 --2- 172.21.15.133:0/1938282046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2538157b90 0x7f253815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.095 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 -- 172.21.15.133:0/1938282046 >> 172.21.15.133:0/1938282046 conn(0x7f253806d190 msgr2=0x7f253806d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.095 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 -- 172.21.15.133:0/1938282046 shutdown_connections 2024-04-25T10:01:45.095 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 -- 172.21.15.133:0/1938282046 wait complete. 2024-04-25T10:01:45.096 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 Processor -- start 2024-04-25T10:01:45.096 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 -- start start 2024-04-25T10:01:45.098 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2538071950 0x7f25380ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.098 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25380b1cb0 0x7f25381f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.098 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2538157b90 0x7f25381f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.098 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2538071240 con 0x7f2538157b90 2024-04-25T10:01:45.098 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25380710c0 con 0x7f2538071950 2024-04-25T10:01:45.098 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f25407e2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25380713c0 con 0x7f25380b1cb0 2024-04-25T10:01:45.098 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253ed7f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2538157b90 0x7f25381f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.098 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253ed7f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2538157b90 0x7f25381f7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58484/0 (socket says 172.21.15.133:58484) 2024-04-25T10:01:45.098 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253ed7f640 1 -- 172.21.15.133:0/2923729354 learned_addr learned my addr 172.21.15.133:0/2923729354 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:45.099 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253dd7d640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25380b1cb0 0x7f25381f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.099 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253e57e640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2538071950 0x7f25380ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.099 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253ed7f640 1 -- 172.21.15.133:0/2923729354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25380b1cb0 msgr2=0x7f25381f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.099 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253ed7f640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25380b1cb0 0x7f25381f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.099 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253ed7f640 1 -- 172.21.15.133:0/2923729354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2538071950 msgr2=0x7f25380ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.099 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253ed7f640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2538071950 0x7f25380ae010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.099 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253ed7f640 1 -- 172.21.15.133:0/2923729354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25381648c0 con 0x7f2538157b90 2024-04-25T10:01:45.099 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.089+0000 7f253dd7d640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25380b1cb0 0x7f25381f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:45.099 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f253ed7f640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2538157b90 0x7f25381f7030 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f253000cb70 tx=0x7f253000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.099 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f252effd640 1 -- 172.21.15.133:0/2923729354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530017070 con 0x7f2538157b90 2024-04-25T10:01:45.100 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f252effd640 1 -- 172.21.15.133:0/2923729354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253000ace0 con 0x7f2538157b90 2024-04-25T10:01:45.100 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f25407e2640 1 -- 172.21.15.133:0/2923729354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25380ae550 con 0x7f2538157b90 2024-04-25T10:01:45.100 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f252effd640 1 -- 172.21.15.133:0/2923729354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530012620 con 0x7f2538157b90 2024-04-25T10:01:45.100 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f25407e2640 1 -- 172.21.15.133:0/2923729354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2538164020 con 0x7f2538157b90 2024-04-25T10:01:45.101 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f25407e2640 1 -- 172.21.15.133:0/2923729354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2538074de0 con 0x7f2538157b90 2024-04-25T10:01:45.101 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f252effd640 1 -- 172.21.15.133:0/2923729354 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f253001a070 con 0x7f2538157b90 2024-04-25T10:01:45.101 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f252effd640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.101 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f252effd640 1 -- 172.21.15.133:0/2923729354 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2530084a50 con 0x7f2538157b90 2024-04-25T10:01:45.101 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f253e57e640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.101 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.093+0000 7f253e57e640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f251c066f60 0x7f251c069420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25340096d0 tx=0x7f2534008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.104 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.097+0000 7f252effd640 1 -- 172.21.15.133:0/2923729354 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2530055e40 con 0x7f2538157b90 2024-04-25T10:01:45.263 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.257+0000 7f25407e2640 1 -- 172.21.15.133:0/2923729354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25380cbd90 con 0x7f2538157b90 2024-04-25T10:01:45.265 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.257+0000 7f252effd640 1 -- 172.21.15.133:0/2923729354 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12908 (secure 0 0 0) 0x7f2530055740 con 0x7f2538157b90 2024-04-25T10:01:45.265 INFO:teuthology.misc.health.smithi133.stdout: 2024-04-25T10:01:45.265 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":16,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:41.865750+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:45.267 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 -- 172.21.15.133:0/2923729354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f251c066f60 msgr2=0x7f251c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.267 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f251c066f60 0x7f251c069420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25340096d0 tx=0x7f2534008040 comp rx=0 tx=0).stop 2024-04-25T10:01:45.268 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 -- 172.21.15.133:0/2923729354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2538157b90 msgr2=0x7f25381f7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.268 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2538157b90 0x7f25381f7030 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f253000cb70 tx=0x7f253000a430 comp rx=0 tx=0).stop 2024-04-25T10:01:45.268 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 -- 172.21.15.133:0/2923729354 shutdown_connections 2024-04-25T10:01:45.268 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.268 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2538157b90 0x7f25381f7030 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.268 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25380b1cb0 0x7f25381f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.268 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 --2- 172.21.15.133:0/2923729354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2538071950 0x7f25380ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.268 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 -- 172.21.15.133:0/2923729354 >> 172.21.15.133:0/2923729354 conn(0x7f253806d190 msgr2=0x7f25380b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.268 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 -- 172.21.15.133:0/2923729354 shutdown_connections 2024-04-25T10:01:45.268 INFO:teuthology.misc.health.smithi133.stderr:2024-04-25T10:01:45.261+0000 7f25407e2640 1 -- 172.21.15.133:0/2923729354 wait complete. 2024-04-25T10:01:45.282 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:01:45.282 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:01:45.283 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:01:45.283 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:01:45.285 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:01:45.285 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:01:45.285 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:01:45.288 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:01:45.288 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:01:45.288 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:01:45.291 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:01:45.291 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:01:45.291 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:01:45.293 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:01:45.294 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:01:45.296 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:45.379 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:01:45.382 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:01:45.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.397+0000 7fc5d6a39640 1 Processor -- start 2024-04-25T10:01:45.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.397+0000 7fc5d6a39640 1 -- start start 2024-04-25T10:01:45.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc5d00b4b70 0x7fc5d015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5d00b3f40 0x7fc5d015e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5d015eeb0 0x7fc5d00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d0155150 con 0x7fc5d00b4b70 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d015e200 con 0x7fc5d00b3f40 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d015e4e0 con 0x7fc5d015eeb0 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5cffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc5d00b4b70 0x7fc5d015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5cffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc5d00b4b70 0x7fc5d015ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58496/0 (socket says 172.21.15.133:58496) 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5cffff640 1 -- 172.21.15.133:0/3556829507 learned_addr learned my addr 172.21.15.133:0/3556829507 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5cf7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5d00b3f40 0x7fc5d015e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d4fd6640 1 --2- 172.21.15.133:0/3556829507 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5d015eeb0 0x7fc5d00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5cffff640 1 -- 172.21.15.133:0/3556829507 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5d015eeb0 msgr2=0x7fc5d00ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5cffff640 1 --2- 172.21.15.133:0/3556829507 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5d015eeb0 0x7fc5d00ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5cffff640 1 -- 172.21.15.133:0/3556829507 >> v2:172.21.15.162:3300/0 conn(0x7fc5d00b3f40 msgr2=0x7fc5d015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5cffff640 1 --2- 172.21.15.133:0/3556829507 >> v2:172.21.15.162:3300/0 conn(0x7fc5d00b3f40 0x7fc5d015e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5cffff640 1 -- 172.21.15.133:0/3556829507 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d015b490 con 0x7fc5d00b4b70 2024-04-25T10:01:45.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5cffff640 1 --2- 172.21.15.133:0/3556829507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00b4b70 0x7fc5d015ac10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009870 tx=0x7fc5c401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f58b407d3c680e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5ceffd640 1 -- 172.21.15.133:0/3556829507 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40b0070 con 0x7fc5d00b4b70 2024-04-25T10:01:45.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5ceffd640 1 -- 172.21.15.133:0/3556829507 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c40ab600 con 0x7fc5d00b4b70 2024-04-25T10:01:45.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5ceffd640 1 -- 172.21.15.133:0/3556829507 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40ab9d0 con 0x7fc5d00b4b70 2024-04-25T10:01:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3556829507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00b4b70 msgr2=0x7fc5d015ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 --2- 172.21.15.133:0/3556829507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00b4b70 0x7fc5d015ac10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009870 tx=0x7fc5c401c3d0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3556829507 shutdown_connections 2024-04-25T10:01:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 --2- 172.21.15.133:0/3556829507 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5d015eeb0 0x7fc5d00ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 --2- 172.21.15.133:0/3556829507 >> v2:172.21.15.162:3300/0 conn(0x7fc5d00b3f40 0x7fc5d015e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 --2- 172.21.15.133:0/3556829507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00b4b70 0x7fc5d015ac10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3556829507 >> 172.21.15.133:0/3556829507 conn(0x7fc5d0071aa0 msgr2=0x7fc5d0071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3556829507 shutdown_connections 2024-04-25T10:01:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.401+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3556829507 wait complete. 2024-04-25T10:01:45.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d6a39640 1 Processor -- start 2024-04-25T10:01:45.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d6a39640 1 -- start start 2024-04-25T10:01:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d6a39640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5d00b3f40 0x7fc5d014f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d6a39640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00b4b70 0x7fc5d0156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d6a39640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5d01497e0 0x7fc5d006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d6a39640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00ace90 con 0x7fc5d00b4b70 2024-04-25T10:01:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d6a39640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00acd10 con 0x7fc5d01497e0 2024-04-25T10:01:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d6a39640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00ad010 con 0x7fc5d00b3f40 2024-04-25T10:01:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5cf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00b4b70 0x7fc5d0156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5cffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5d00b3f40 0x7fc5d014f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5cf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00b4b70 0x7fc5d0156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58506/0 (socket says 172.21.15.133:58506) 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5cf7fe640 1 -- 172.21.15.133:0/3403103636 learned_addr learned my addr 172.21.15.133:0/3403103636 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d4fd6640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5d01497e0 0x7fc5d006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5cf7fe640 1 -- 172.21.15.133:0/3403103636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5d00b3f40 msgr2=0x7fc5d014f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5cf7fe640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5d00b3f40 0x7fc5d014f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5cf7fe640 1 -- 172.21.15.133:0/3403103636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5d01497e0 msgr2=0x7fc5d006ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5cf7fe640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5d01497e0 0x7fc5d006ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5cf7fe640 1 -- 172.21.15.133:0/3403103636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d014f1d0 con 0x7fc5d00b4b70 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5cf7fe640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00b4b70 0x7fc5d0156450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e8b0 tx=0x7fc5c000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5ccff9640 1 -- 172.21.15.133:0/3403103636 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0017070 con 0x7fc5d00b4b70 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5ccff9640 1 -- 172.21.15.133:0/3403103636 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c000c940 con 0x7fc5d00b4b70 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5ccff9640 1 -- 172.21.15.133:0/3403103636 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c000cd10 con 0x7fc5d00b4b70 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3403103636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d0156ae0 con 0x7fc5d00b4b70 2024-04-25T10:01:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3403103636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d01570a0 con 0x7fc5d00b4b70 2024-04-25T10:01:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3403103636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d0067c70 con 0x7fc5d00b4b70 2024-04-25T10:01:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.405+0000 7fc5ccff9640 1 -- 172.21.15.133:0/3403103636 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5c001a070 con 0x7fc5d00b4b70 2024-04-25T10:01:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.409+0000 7fc5ccff9640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5bc066f60 0x7fc5bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.409+0000 7fc5cffff640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5bc066f60 0x7fc5bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.409+0000 7fc5ccff9640 1 -- 172.21.15.133:0/3403103636 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc5c0084840 con 0x7fc5d00b4b70 2024-04-25T10:01:45.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.409+0000 7fc5cffff640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5bc066f60 0x7fc5bc069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40ad000 tx=0x7fc5c40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.413+0000 7fc5ccff9640 1 -- 172.21.15.133:0/3403103636 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5c0055550 con 0x7fc5d00b4b70 2024-04-25T10:01:45.466 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:01:45.466 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:01:45.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.569+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3403103636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5d0066b50 con 0x7fc5d00b4b70 2024-04-25T10:01:45.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.569+0000 7fc5ccff9640 1 -- 172.21.15.133:0/3403103636 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12908 (secure 0 0 0) 0x7fc5c00595f0 con 0x7fc5d00b4b70 2024-04-25T10:01:45.576 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:01:45.576 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":16,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:41.865750+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3403103636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5bc066f60 msgr2=0x7fc5bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5bc066f60 0x7fc5bc069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40ad000 tx=0x7fc5c40a6000 comp rx=0 tx=0).stop 2024-04-25T10:01:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3403103636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00b4b70 msgr2=0x7fc5d0156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00b4b70 0x7fc5d0156450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e8b0 tx=0x7fc5c000ed70 comp rx=0 tx=0).stop 2024-04-25T10:01:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3403103636 shutdown_connections 2024-04-25T10:01:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5d01497e0 0x7fc5d006ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5bc066f60 0x7fc5bc069420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00b4b70 0x7fc5d0156450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 --2- 172.21.15.133:0/3403103636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5d00b3f40 0x7fc5d014f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3403103636 >> 172.21.15.133:0/3403103636 conn(0x7fc5d0071aa0 msgr2=0x7fc5d006dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3403103636 shutdown_connections 2024-04-25T10:01:45.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.573+0000 7fc5d6a39640 1 -- 172.21.15.133:0/3403103636 wait complete. 2024-04-25T10:01:45.593 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:45.594 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:45.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f595ab33640 1 Processor -- start 2024-04-25T10:01:45.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f595ab33640 1 -- start start 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f595ab33640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f595406cea0 0x7f59540d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f595ab33640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59540d0a40 0x7f595415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f595ab33640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59540710f0 con 0x7f5954070d10 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f595ab33640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59540d0ef0 con 0x7f595406cea0 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f595ab33640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59540cbc60 con 0x7f59540d0a40 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f5953fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f595406cea0 0x7f59540d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f5953fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f595406cea0 0x7f59540d0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58554/0 (socket says 172.21.15.133:58554) 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f5953fff640 1 -- 172.21.15.133:0/1160510880 learned_addr learned my addr 172.21.15.133:0/1160510880 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f59590d0640 1 --2- 172.21.15.133:0/1160510880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59540d0a40 0x7f595415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f5953fff640 1 -- 172.21.15.133:0/1160510880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59540d0a40 msgr2=0x7f595415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f5953fff640 1 --2- 172.21.15.133:0/1160510880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59540d0a40 0x7f595415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f5953fff640 1 -- 172.21.15.133:0/1160510880 >> v1:172.21.15.133:6789/0 conn(0x7f5954070d10 legacy=0x7f595406c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f5953fff640 1 -- 172.21.15.133:0/1160510880 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595406f240 con 0x7f595406cea0 2024-04-25T10:01:45.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.705+0000 7f59590d0640 1 --2- 172.21.15.133:0/1160510880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59540d0a40 0x7f595415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:45.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f5953fff640 1 --2- 172.21.15.133:0/1160510880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f595406cea0 0x7f59540d0500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5948009870 tx=0x7f594801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6885728e00337a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f59537fe640 1 -- 172.21.15.133:0/1160510880 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480ab650 con 0x7f595406cea0 2024-04-25T10:01:45.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f59537fe640 1 -- 172.21.15.133:0/1160510880 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59480b0070 con 0x7f595406cea0 2024-04-25T10:01:45.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f59537fe640 1 -- 172.21.15.133:0/1160510880 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480aba10 con 0x7f595406cea0 2024-04-25T10:01:45.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f595ab33640 1 -- 172.21.15.133:0/1160510880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f595406cea0 msgr2=0x7f59540d0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f595ab33640 1 --2- 172.21.15.133:0/1160510880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f595406cea0 0x7f59540d0500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5948009870 tx=0x7f594801c560 comp rx=0 tx=0).stop 2024-04-25T10:01:45.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f595ab33640 1 -- 172.21.15.133:0/1160510880 shutdown_connections 2024-04-25T10:01:45.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f595ab33640 1 --2- 172.21.15.133:0/1160510880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59540d0a40 0x7f595415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f595ab33640 1 --2- 172.21.15.133:0/1160510880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f595406cea0 0x7f59540d0500 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f595ab33640 1 -- 172.21.15.133:0/1160510880 >> 172.21.15.133:0/1160510880 conn(0x7f59540d1610 msgr2=0x7f59540d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f595ab33640 1 -- 172.21.15.133:0/1160510880 shutdown_connections 2024-04-25T10:01:45.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f595ab33640 1 -- 172.21.15.133:0/1160510880 wait complete. 2024-04-25T10:01:45.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f595ab33640 1 Processor -- start 2024-04-25T10:01:45.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.709+0000 7f595ab33640 1 -- start start 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f595ab33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f595406cea0 0x7f59540ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f595ab33640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5954070d10 0x7f5954146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f595ab33640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59540d0a40 0x7f5954146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f595ab33640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540cc990 con 0x7f595406cea0 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f595ab33640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540cc470 con 0x7f5954070d10 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f595ab33640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59540ccb10 con 0x7f59540d0a40 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59588cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f595406cea0 0x7f59540ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59588cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f595406cea0 0x7f59540ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58566/0 (socket says 172.21.15.133:58566) 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59588cf640 1 -- 172.21.15.133:0/2370831163 learned_addr learned my addr 172.21.15.133:0/2370831163 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59588cf640 1 -- 172.21.15.133:0/2370831163 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59540d0a40 msgr2=0x7f5954146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59588cf640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59540d0a40 0x7f5954146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59588cf640 1 -- 172.21.15.133:0/2370831163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5954070d10 msgr2=0x7f5954146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59588cf640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5954070d10 0x7f5954146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59588cf640 1 -- 172.21.15.133:0/2370831163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5954157be0 con 0x7f595406cea0 2024-04-25T10:01:45.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59588cf640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f595406cea0 0x7f59540ce0b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f594400c420 tx=0x7f594400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59517fa640 1 -- 172.21.15.133:0/2370831163 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440052d0 con 0x7f595406cea0 2024-04-25T10:01:45.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59517fa640 1 -- 172.21.15.133:0/2370831163 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5944005470 con 0x7f595406cea0 2024-04-25T10:01:45.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f595ab33640 1 -- 172.21.15.133:0/2370831163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5954157ec0 con 0x7f595406cea0 2024-04-25T10:01:45.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59517fa640 1 -- 172.21.15.133:0/2370831163 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944007260 con 0x7f595406cea0 2024-04-25T10:01:45.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f595ab33640 1 -- 172.21.15.133:0/2370831163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5954158480 con 0x7f595406cea0 2024-04-25T10:01:45.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f595ab33640 1 -- 172.21.15.133:0/2370831163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5954067c70 con 0x7f595406cea0 2024-04-25T10:01:45.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59517fa640 1 -- 172.21.15.133:0/2370831163 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5944007400 con 0x7f595406cea0 2024-04-25T10:01:45.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59517fa640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5940066f90 0x7f5940069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f5953fff640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5940066f90 0x7f5940069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f59517fa640 1 -- 172.21.15.133:0/2370831163 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5944084190 con 0x7f595406cea0 2024-04-25T10:01:45.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.713+0000 7f5953fff640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5940066f90 0x7f5940069450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59480af000 tx=0x7f5948004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.717+0000 7f59517fa640 1 -- 172.21.15.133:0/2370831163 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59540710f0 con 0x7f595406cea0 2024-04-25T10:01:45.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.865+0000 7f595ab33640 1 -- 172.21.15.133:0/2370831163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5954156750 con 0x7f595406cea0 2024-04-25T10:01:45.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.865+0000 7f59517fa640 1 -- 172.21.15.133:0/2370831163 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f59440554f0 con 0x7f595406cea0 2024-04-25T10:01:45.873 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:01:45.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 -- 172.21.15.133:0/2370831163 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5940066f90 msgr2=0x7f5940069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5940066f90 0x7f5940069450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59480af000 tx=0x7f5948004740 comp rx=0 tx=0).stop 2024-04-25T10:01:45.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 -- 172.21.15.133:0/2370831163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f595406cea0 msgr2=0x7f59540ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f595406cea0 0x7f59540ce0b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f594400c420 tx=0x7f594400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 -- 172.21.15.133:0/2370831163 shutdown_connections 2024-04-25T10:01:45.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59540d0a40 0x7f5954146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5954070d10 0x7f5954146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5940066f90 0x7f5940069450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 --2- 172.21.15.133:0/2370831163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f595406cea0 0x7f59540ce0b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 -- 172.21.15.133:0/2370831163 >> 172.21.15.133:0/2370831163 conn(0x7f59540d1610 msgr2=0x7f5954160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 -- 172.21.15.133:0/2370831163 shutdown_connections 2024-04-25T10:01:45.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.869+0000 7f595ab33640 1 -- 172.21.15.133:0/2370831163 wait complete. 2024-04-25T10:01:45.890 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:45.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f4562a65640 1 Processor -- start 2024-04-25T10:01:45.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f4562a65640 1 -- start start 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f4562a65640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f455c15c9a0 0x7f455c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f4562a65640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f455c0b1f70 0x7f455c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f4562a65640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f455c0ce330 0x7f455c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f4562a65640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455c0cece0 con 0x7f455c0b1f70 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f4562a65640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455c0c61d0 con 0x7f455c0ce330 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f4562a65640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0c64b0 con 0x7f455c15c9a0 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f455bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f455c0b1f70 0x7f455c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f455bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f455c0b1f70 0x7f455c06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58580/0 (socket says 172.21.15.133:58580) 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f455bfff640 1 -- 172.21.15.133:0/1057934999 learned_addr learned my addr 172.21.15.133:0/1057934999 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f455bfff640 1 -- 172.21.15.133:0/1057934999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f455c15c9a0 msgr2=0x7f455c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f455bfff640 1 --2- 172.21.15.133:0/1057934999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f455c15c9a0 0x7f455c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f455bfff640 1 -- 172.21.15.133:0/1057934999 >> v2:172.21.15.162:3300/0 conn(0x7f455c0ce330 msgr2=0x7f455c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:46.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f4561002640 1 --2- 172.21.15.133:0/1057934999 >> v2:172.21.15.162:3300/0 conn(0x7f455c0ce330 0x7f455c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f455bfff640 1 --2- 172.21.15.133:0/1057934999 >> v2:172.21.15.162:3300/0 conn(0x7f455c0ce330 0x7f455c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.993+0000 7f455bfff640 1 -- 172.21.15.133:0/1057934999 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c06d4f0 con 0x7f455c0b1f70 2024-04-25T10:01:46.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455bfff640 1 --2- 172.21.15.133:0/1057934999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f455c0b1f70 0x7f455c06dcf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f455c0ceef0 tx=0x7f454401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53bfdbc1631536f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455b7fe640 1 -- 172.21.15.133:0/1057934999 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440a7070 con 0x7f455c0b1f70 2024-04-25T10:01:46.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455b7fe640 1 -- 172.21.15.133:0/1057934999 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45440b0070 con 0x7f455c0b1f70 2024-04-25T10:01:46.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455b7fe640 1 -- 172.21.15.133:0/1057934999 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440ab890 con 0x7f455c0b1f70 2024-04-25T10:01:46.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 -- 172.21.15.133:0/1057934999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f455c0b1f70 msgr2=0x7f455c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 --2- 172.21.15.133:0/1057934999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f455c0b1f70 0x7f455c06dcf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f455c0ceef0 tx=0x7f454401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 -- 172.21.15.133:0/1057934999 shutdown_connections 2024-04-25T10:01:46.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 --2- 172.21.15.133:0/1057934999 >> v2:172.21.15.162:3300/0 conn(0x7f455c0ce330 0x7f455c0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 --2- 172.21.15.133:0/1057934999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f455c0b1f70 0x7f455c06dcf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 --2- 172.21.15.133:0/1057934999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f455c15c9a0 0x7f455c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 -- 172.21.15.133:0/1057934999 >> 172.21.15.133:0/1057934999 conn(0x7f455c067000 msgr2=0x7f455c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:46.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 -- 172.21.15.133:0/1057934999 shutdown_connections 2024-04-25T10:01:46.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 -- 172.21.15.133:0/1057934999 wait complete. 2024-04-25T10:01:46.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 Processor -- start 2024-04-25T10:01:46.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 -- start start 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f455c0b1f70 0x7f455c157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f455c0ce330 0x7f455c14ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f455c15c9a0 0x7f455c155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c15cf00 con 0x7f455c0ce330 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c15cd80 con 0x7f455c0b1f70 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c15d950 con 0x7f455c15c9a0 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f455c0ce330 0x7f455c14ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f455c0ce330 0x7f455c14ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58592/0 (socket says 172.21.15.133:58592) 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455bfff640 1 -- 172.21.15.133:0/3251249847 learned_addr learned my addr 172.21.15.133:0/3251249847 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455bfff640 1 -- 172.21.15.133:0/3251249847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f455c15c9a0 msgr2=0x7f455c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455bfff640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f455c15c9a0 0x7f455c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455bfff640 1 -- 172.21.15.133:0/3251249847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f455c0b1f70 msgr2=0x7f455c157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455bfff640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f455c0b1f70 0x7f455c157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455bfff640 1 -- 172.21.15.133:0/3251249847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c158130 con 0x7f455c0ce330 2024-04-25T10:01:46.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f455bfff640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f455c0ce330 0x7f455c14ff50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f45440ad040 tx=0x7f454401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f45597fa640 1 -- 172.21.15.133:0/3251249847 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440abe60 con 0x7f455c0ce330 2024-04-25T10:01:46.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f45597fa640 1 -- 172.21.15.133:0/3251249847 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45440b4820 con 0x7f455c0ce330 2024-04-25T10:01:46.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f45597fa640 1 -- 172.21.15.133:0/3251249847 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440b4c10 con 0x7f455c0ce330 2024-04-25T10:01:46.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 -- 172.21.15.133:0/3251249847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455c158410 con 0x7f455c0ce330 2024-04-25T10:01:46.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:45.997+0000 7f4562a65640 1 -- 172.21.15.133:0/3251249847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c1589d0 con 0x7f455c0ce330 2024-04-25T10:01:46.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.001+0000 7f4562a65640 1 -- 172.21.15.133:0/3251249847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455c068ef0 con 0x7f455c0ce330 2024-04-25T10:01:46.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.001+0000 7f45597fa640 1 -- 172.21.15.133:0/3251249847 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45440c7390 con 0x7f455c0ce330 2024-04-25T10:01:46.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.001+0000 7f45597fa640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f453c066f90 0x7f453c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.001+0000 7f45597fa640 1 -- 172.21.15.133:0/3251249847 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f454411f510 con 0x7f455c0ce330 2024-04-25T10:01:46.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.001+0000 7f4560801640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f453c066f90 0x7f453c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.001+0000 7f4560801640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f453c066f90 0x7f453c069450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f455c149bc0 tx=0x7f454c009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.005+0000 7f45597fa640 1 -- 172.21.15.133:0/3251249847 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45440c4060 con 0x7f455c0ce330 2024-04-25T10:01:46.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 -- 172.21.15.133:0/3251249847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f455c06e790 con 0x7f455c0ce330 2024-04-25T10:01:46.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f45597fa640 1 -- 172.21.15.133:0/3251249847 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f454411f070 con 0x7f455c0ce330 2024-04-25T10:01:46.202 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:01:46.202 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:01:46.203 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 2 2024-04-25T10:01:46.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 -- 172.21.15.133:0/3251249847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f453c066f90 msgr2=0x7f453c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f453c066f90 0x7f453c069450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f455c149bc0 tx=0x7f454c009250 comp rx=0 tx=0).stop 2024-04-25T10:01:46.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 -- 172.21.15.133:0/3251249847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f455c0ce330 msgr2=0x7f455c14ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f455c0ce330 0x7f455c14ff50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f45440ad040 tx=0x7f454401ad30 comp rx=0 tx=0).stop 2024-04-25T10:01:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 -- 172.21.15.133:0/3251249847 shutdown_connections 2024-04-25T10:01:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f455c15c9a0 0x7f455c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f453c066f90 0x7f453c069450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f455c0ce330 0x7f455c14ff50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 --2- 172.21.15.133:0/3251249847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f455c0b1f70 0x7f455c157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 -- 172.21.15.133:0/3251249847 >> 172.21.15.133:0/3251249847 conn(0x7f455c067000 msgr2=0x7f455c153400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 -- 172.21.15.133:0/3251249847 shutdown_connections 2024-04-25T10:01:46.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:46.197+0000 7f4562a65640 1 -- 172.21.15.133:0/3251249847 wait complete. 2024-04-25T10:01:48.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.341+0000 7f0923b35640 1 Processor -- start 2024-04-25T10:01:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.341+0000 7f0923b35640 1 -- start start 2024-04-25T10:01:48.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f0923b35640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f091c15db00 0x7f091c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f0923b35640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091c0d0770 0x7f091c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f0923b35640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f091c1614c0 0x7f091c161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f0923b35640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091c0d1110 con 0x7f091c15db00 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f0923b35640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091c0ceb20 con 0x7f091c1614c0 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f0923b35640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0b7740 con 0x7f091c0d0770 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09218d1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f091c15db00 0x7f091c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09218d1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f091c15db00 0x7f091c0b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57026/0 (socket says 172.21.15.133:57026) 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09218d1640 1 -- 172.21.15.133:0/3349341840 learned_addr learned my addr 172.21.15.133:0/3349341840 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09210d0640 1 --2- 172.21.15.133:0/3349341840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091c0d0770 0x7f091c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09218d1640 1 -- 172.21.15.133:0/3349341840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091c0d0770 msgr2=0x7f091c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09220d2640 1 --2- 172.21.15.133:0/3349341840 >> v2:172.21.15.162:3300/0 conn(0x7f091c1614c0 0x7f091c161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09218d1640 1 --2- 172.21.15.133:0/3349341840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091c0d0770 0x7f091c0d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09218d1640 1 -- 172.21.15.133:0/3349341840 >> v2:172.21.15.162:3300/0 conn(0x7f091c1614c0 msgr2=0x7f091c161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09218d1640 1 --2- 172.21.15.133:0/3349341840 >> v2:172.21.15.162:3300/0 conn(0x7f091c1614c0 0x7f091c161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09218d1640 1 -- 172.21.15.133:0/3349341840 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c0b7a20 con 0x7f091c15db00 2024-04-25T10:01:48.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09218d1640 1 --2- 172.21.15.133:0/3349341840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f091c15db00 0x7f091c0b2990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f091c06cf70 tx=0x7f091801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f897066723f2a18c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:48.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09208cf640 1 -- 172.21.15.133:0/3349341840 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180aa610 con 0x7f091c15db00 2024-04-25T10:01:48.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09208cf640 1 -- 172.21.15.133:0/3349341840 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09180af070 con 0x7f091c15db00 2024-04-25T10:01:48.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.345+0000 7f09208cf640 1 -- 172.21.15.133:0/3349341840 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180aa9d0 con 0x7f091c15db00 2024-04-25T10:01:48.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 -- 172.21.15.133:0/3349341840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f091c15db00 msgr2=0x7f091c0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 --2- 172.21.15.133:0/3349341840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f091c15db00 0x7f091c0b2990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f091c06cf70 tx=0x7f091801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 -- 172.21.15.133:0/3349341840 shutdown_connections 2024-04-25T10:01:48.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 --2- 172.21.15.133:0/3349341840 >> v2:172.21.15.162:3300/0 conn(0x7f091c1614c0 0x7f091c161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 --2- 172.21.15.133:0/3349341840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091c0d0770 0x7f091c0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 --2- 172.21.15.133:0/3349341840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f091c15db00 0x7f091c0b2990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 -- 172.21.15.133:0/3349341840 >> 172.21.15.133:0/3349341840 conn(0x7f091c0727c0 msgr2=0x7f091c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:48.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 -- 172.21.15.133:0/3349341840 shutdown_connections 2024-04-25T10:01:48.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 -- 172.21.15.133:0/3349341840 wait complete. 2024-04-25T10:01:48.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 Processor -- start 2024-04-25T10:01:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 -- start start 2024-04-25T10:01:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f091c0d0770 0x7f091c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091c15db00 0x7f091c1604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f091c1614c0 0x7f091c1fa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0b7430 con 0x7f091c1614c0 2024-04-25T10:01:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0b82c0 con 0x7f091c0d0770 2024-04-25T10:01:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0d18a0 con 0x7f091c15db00 2024-04-25T10:01:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09220d2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f091c1614c0 0x7f091c1fa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09220d2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f091c1614c0 0x7f091c1fa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57032/0 (socket says 172.21.15.133:57032) 2024-04-25T10:01:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09220d2640 1 -- 172.21.15.133:0/3384500638 learned_addr learned my addr 172.21.15.133:0/3384500638 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09210d0640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091c15db00 0x7f091c1604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09218d1640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f091c0d0770 0x7f091c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09220d2640 1 -- 172.21.15.133:0/3384500638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091c15db00 msgr2=0x7f091c1604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09220d2640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091c15db00 0x7f091c1604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09220d2640 1 -- 172.21.15.133:0/3384500638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f091c0d0770 msgr2=0x7f091c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09220d2640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f091c0d0770 0x7f091c15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09220d2640 1 -- 172.21.15.133:0/3384500638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c0d34e0 con 0x7f091c1614c0 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09220d2640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f091c1614c0 0x7f091c1fa6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f091400cda0 tx=0x7f091400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09127fc640 1 -- 172.21.15.133:0/3384500638 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914003ba0 con 0x7f091c1614c0 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09127fc640 1 -- 172.21.15.133:0/3384500638 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0914003d40 con 0x7f091c1614c0 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 -- 172.21.15.133:0/3384500638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c0d37c0 con 0x7f091c1614c0 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f09127fc640 1 -- 172.21.15.133:0/3384500638 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914004240 con 0x7f091c1614c0 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.349+0000 7f0923b35640 1 -- 172.21.15.133:0/3384500638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c160de0 con 0x7f091c1614c0 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.353+0000 7f0923b35640 1 -- 172.21.15.133:0/3384500638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c068ef0 con 0x7f091c1614c0 2024-04-25T10:01:48.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.353+0000 7f09127fc640 1 -- 172.21.15.133:0/3384500638 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09140043e0 con 0x7f091c1614c0 2024-04-25T10:01:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.353+0000 7f09127fc640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0900066f60 0x7f0900069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.353+0000 7f09127fc640 1 -- 172.21.15.133:0/3384500638 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f09140858e0 con 0x7f091c1614c0 2024-04-25T10:01:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.353+0000 7f09218d1640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0900066f60 0x7f0900069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.353+0000 7f09218d1640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0900066f60 0x7f0900069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f09180ae000 tx=0x7f09180a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:48.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.357+0000 7f09127fc640 1 -- 172.21.15.133:0/3384500638 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0914056ba0 con 0x7f091c1614c0 2024-04-25T10:01:48.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.545+0000 7f0923b35640 1 -- 172.21.15.133:0/3384500638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f091c0b2750 con 0x7f091c1614c0 2024-04-25T10:01:48.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.545+0000 7f09127fc640 1 -- 172.21.15.133:0/3384500638 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3703 (secure 0 0 0) 0x7f09140564a0 con 0x7f091c1614c0 2024-04-25T10:01:48.551 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:01:48.552 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2230831968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2230831968},{"type":"v1","addr":"172.21.15.162:6835","nonce":2230831968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1456396097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1456396097},{"type":"v1","addr":"172.21.15.133:6836","nonce":1456396097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4296,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2462154167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2462154167},{"type":"v1","addr":"172.21.15.133:6837","nonce":2462154167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:01:48.552 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 3 2024-04-25T10:01:48.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 -- 172.21.15.133:0/3384500638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0900066f60 msgr2=0x7f0900069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0900066f60 0x7f0900069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f09180ae000 tx=0x7f09180a7000 comp rx=0 tx=0).stop 2024-04-25T10:01:48.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 -- 172.21.15.133:0/3384500638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f091c1614c0 msgr2=0x7f091c1fa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f091c1614c0 0x7f091c1fa6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f091400cda0 tx=0x7f091400a660 comp rx=0 tx=0).stop 2024-04-25T10:01:48.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 -- 172.21.15.133:0/3384500638 shutdown_connections 2024-04-25T10:01:48.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0900066f60 0x7f0900069420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f091c1614c0 0x7f091c1fa6e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f091c15db00 0x7f091c1604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 --2- 172.21.15.133:0/3384500638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f091c0d0770 0x7f091c15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 -- 172.21.15.133:0/3384500638 >> 172.21.15.133:0/3384500638 conn(0x7f091c0727c0 msgr2=0x7f091c15cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:48.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 -- 172.21.15.133:0/3384500638 shutdown_connections 2024-04-25T10:01:48.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.549+0000 7f0923b35640 1 -- 172.21.15.133:0/3384500638 wait complete. 2024-04-25T10:01:48.569 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:48.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa12aba640 1 Processor -- start 2024-04-25T10:01:48.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa12aba640 1 -- start start 2024-04-25T10:01:48.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa12aba640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa0c06fdc0 0x7faa0c15c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa12aba640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0c06e150 con 0x7faa0c06c3b0 2024-04-25T10:01:48.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa12aba640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0c06e410 con 0x7faa0c1591d0 2024-04-25T10:01:48.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa12aba640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0c06d9e0 con 0x7faa0c06fdc0 2024-04-25T10:01:48.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa11057640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa0c06fdc0 0x7faa0c15c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa11057640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa0c06fdc0 0x7faa0c15c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57036/0 (socket says 172.21.15.133:57036) 2024-04-25T10:01:48.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa11057640 1 -- 172.21.15.133:0/2530035498 learned_addr learned my addr 172.21.15.133:0/2530035498 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:48.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa11057640 1 -- 172.21.15.133:0/2530035498 >> v1:172.21.15.162:6789/0 conn(0x7faa0c1591d0 legacy=0x7faa0c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:48.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa11057640 1 -- 172.21.15.133:0/2530035498 >> v1:172.21.15.133:6789/0 conn(0x7faa0c06c3b0 legacy=0x7faa0c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa11057640 1 -- 172.21.15.133:0/2530035498 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c06e6f0 con 0x7faa0c06fdc0 2024-04-25T10:01:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa11057640 1 --2- 172.21.15.133:0/2530035498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa0c06fdc0 0x7faa0c15c6f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa04009e50 tx=0x7faa0401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c461f818e66cbc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.681+0000 7faa0b7fe640 1 -- 172.21.15.133:0/2530035498 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa040a57d0 con 0x7faa0c06fdc0 2024-04-25T10:01:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa0b7fe640 1 -- 172.21.15.133:0/2530035498 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa040a5970 con 0x7faa0c06fdc0 2024-04-25T10:01:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa0b7fe640 1 -- 172.21.15.133:0/2530035498 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa040a5d40 con 0x7faa0c06fdc0 2024-04-25T10:01:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 -- 172.21.15.133:0/2530035498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa0c06fdc0 msgr2=0x7faa0c15c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 --2- 172.21.15.133:0/2530035498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa0c06fdc0 0x7faa0c15c6f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa04009e50 tx=0x7faa0401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 -- 172.21.15.133:0/2530035498 shutdown_connections 2024-04-25T10:01:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 --2- 172.21.15.133:0/2530035498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa0c06fdc0 0x7faa0c15c6f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 -- 172.21.15.133:0/2530035498 >> 172.21.15.133:0/2530035498 conn(0x7faa0c071aa0 msgr2=0x7faa0c071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 -- 172.21.15.133:0/2530035498 shutdown_connections 2024-04-25T10:01:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 -- 172.21.15.133:0/2530035498 wait complete. 2024-04-25T10:01:48.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 Processor -- start 2024-04-25T10:01:48.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 -- start start 2024-04-25T10:01:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa0c06c3b0 0x7faa0c1f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa0c06fdc0 0x7faa0c159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa0c1591d0 0x7faa0c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c15e810 con 0x7faa0c06c3b0 2024-04-25T10:01:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c15e690 con 0x7faa0c06fdc0 2024-04-25T10:01:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c169610 con 0x7faa0c1591d0 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa10856640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa0c06c3b0 0x7faa0c1f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa10856640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa0c06c3b0 0x7faa0c1f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57050/0 (socket says 172.21.15.133:57050) 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa10856640 1 -- 172.21.15.133:0/2260627766 learned_addr learned my addr 172.21.15.133:0/2260627766 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa0bfff640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa0c06fdc0 0x7faa0c159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa10856640 1 -- 172.21.15.133:0/2260627766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa0c1591d0 msgr2=0x7faa0c159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa10856640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa0c1591d0 0x7faa0c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa10856640 1 -- 172.21.15.133:0/2260627766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa0c06fdc0 msgr2=0x7faa0c159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa10856640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa0c06fdc0 0x7faa0c159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa10856640 1 -- 172.21.15.133:0/2260627766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c15b2c0 con 0x7faa0c06c3b0 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa10856640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa0c06c3b0 0x7faa0c1f98f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa0000b390 tx=0x7faa0000b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa097fa640 1 -- 172.21.15.133:0/2260627766 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00015020 con 0x7faa0c06c3b0 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa097fa640 1 -- 172.21.15.133:0/2260627766 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa00011830 con 0x7faa0c06c3b0 2024-04-25T10:01:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 -- 172.21.15.133:0/2260627766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0c1f9e30 con 0x7faa0c06c3b0 2024-04-25T10:01:48.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa097fa640 1 -- 172.21.15.133:0/2260627766 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00011c20 con 0x7faa0c06c3b0 2024-04-25T10:01:48.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.685+0000 7faa12aba640 1 -- 172.21.15.133:0/2260627766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0c0ae1c0 con 0x7faa0c06c3b0 2024-04-25T10:01:48.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.689+0000 7faa12aba640 1 -- 172.21.15.133:0/2260627766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0c067c70 con 0x7faa0c06c3b0 2024-04-25T10:01:48.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.689+0000 7faa097fa640 1 -- 172.21.15.133:0/2260627766 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa00018020 con 0x7faa0c06c3b0 2024-04-25T10:01:48.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.689+0000 7faa097fa640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9f0066f60 0x7fa9f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.689+0000 7faa097fa640 1 -- 172.21.15.133:0/2260627766 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faa00084af0 con 0x7faa0c06c3b0 2024-04-25T10:01:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.689+0000 7faa0bfff640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9f0066f60 0x7fa9f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.689+0000 7faa0bfff640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9f0066f60 0x7fa9f0069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8007fd0 tx=0x7fa9f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.693+0000 7faa097fa640 1 -- 172.21.15.133:0/2260627766 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa0001f020 con 0x7faa0c06c3b0 2024-04-25T10:01:48.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.849+0000 7faa12aba640 1 -- 172.21.15.133:0/2260627766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa0c06f430 con 0x7faa0c06c3b0 2024-04-25T10:01:48.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.853+0000 7faa097fa640 1 -- 172.21.15.133:0/2260627766 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12908 (secure 0 0 0) 0x7faa000172c0 con 0x7faa0c06c3b0 2024-04-25T10:01:48.859 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:01:48.859 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":16,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:41.865750+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:48.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.853+0000 7faa12aba640 1 -- 172.21.15.133:0/2260627766 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9f0066f60 msgr2=0x7fa9f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.853+0000 7faa12aba640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9f0066f60 0x7fa9f0069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8007fd0 tx=0x7fa9f8009040 comp rx=0 tx=0).stop 2024-04-25T10:01:48.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.853+0000 7faa12aba640 1 -- 172.21.15.133:0/2260627766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa0c06c3b0 msgr2=0x7faa0c1f98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.853+0000 7faa12aba640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa0c06c3b0 0x7faa0c1f98f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa0000b390 tx=0x7faa0000b850 comp rx=0 tx=0).stop 2024-04-25T10:01:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.857+0000 7faa12aba640 1 -- 172.21.15.133:0/2260627766 shutdown_connections 2024-04-25T10:01:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.857+0000 7faa12aba640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa0c1591d0 0x7faa0c159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.857+0000 7faa12aba640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa0c06fdc0 0x7faa0c159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.857+0000 7faa12aba640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9f0066f60 0x7fa9f0069420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.857+0000 7faa12aba640 1 --2- 172.21.15.133:0/2260627766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa0c06c3b0 0x7faa0c1f98f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.857+0000 7faa12aba640 1 -- 172.21.15.133:0/2260627766 >> 172.21.15.133:0/2260627766 conn(0x7faa0c071aa0 msgr2=0x7faa0c15d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.857+0000 7faa12aba640 1 -- 172.21.15.133:0/2260627766 shutdown_connections 2024-04-25T10:01:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.857+0000 7faa12aba640 1 -- 172.21.15.133:0/2260627766 wait complete. 2024-04-25T10:01:48.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:48.878 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:49.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.993+0000 7fd0055a6640 1 Processor -- start 2024-04-25T10:01:49.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.993+0000 7fd0055a6640 1 -- start start 2024-04-25T10:01:49.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fd0055a6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0000cd820 0x7fd0000cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fd0055a6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd00015ced0 0x7fd00015d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fd0055a6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0000ce2e0 con 0x7fd000145170 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fd0055a6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00006db00 con 0x7fd00015ced0 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fd0055a6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd000145550 con 0x7fd0000cd820 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffeffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd000145170 0x7fd00006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57240/0 (socket says 172.21.15.133:57240) 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffeffd640 1 -- 172.21.15.133:0/2985346256 learned_addr learned my addr 172.21.15.133:0/2985346256 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcfff7fe640 1 --2- 172.21.15.133:0/2985346256 >> v2:172.21.15.162:3300/0 conn(0x7fd00015ced0 0x7fd00015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffe7fc640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0000cd820 0x7fd0000cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827463757 0 0) 0x7fd0000ce2e0 con 0x7fd000145170 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe8003530 con 0x7fd000145170 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1064401083 0 0) 0x7fcfe8003530 con 0x7fd000145170 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0000cd820 msgr2=0x7fd0000cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0000cd820 0x7fd0000cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 >> v2:172.21.15.162:3300/0 conn(0x7fd00015ced0 msgr2=0x7fd00015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 --2- 172.21.15.133:0/2985346256 >> v2:172.21.15.162:3300/0 conn(0x7fd00015ced0 0x7fd00015d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00006d4b0 con 0x7fd000145170 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fcff0001680 con 0x7fd000145170 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 >> v1:172.21.15.133:6789/0 conn(0x7fd000145170 legacy=0x7fd00006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfe80039c0 0x7fcfe8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfe8004810 0x7fcfe8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0000cd820 0x7fcfe80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00006db00 con 0x7fcfe8004810 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd000145550 con 0x7fcfe80039c0 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe800b9d0 con 0x7fd0000cd820 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffe7fc640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfe8004810 0x7fcfe8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffeffd640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfe80039c0 0x7fcfe8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcfff7fe640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0000cd820 0x7fcfe80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffe7fc640 1 -- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0000cd820 msgr2=0x7fcfe80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffe7fc640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0000cd820 0x7fcfe80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffe7fc640 1 -- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfe80039c0 msgr2=0x7fcfe8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffe7fc640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfe80039c0 0x7fcfe8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffe7fc640 1 -- 172.21.15.133:0/2985346256 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8003530 con 0x7fcfe8004810 2024-04-25T10:01:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcfff7fe640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0000cd820 0x7fcfe80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:48.997+0000 7fcffe7fc640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfe8004810 0x7fcfe8004c90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009ae0 tx=0x7fcff401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6494a7b524cbc8bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff40b2070 con 0x7fcfe8004810 2024-04-25T10:01:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff40a5580 con 0x7fcfe8004810 2024-04-25T10:01:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffdffb640 1 -- 172.21.15.133:0/2985346256 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff40096b0 con 0x7fcfe8004810 2024-04-25T10:01:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 -- 172.21.15.133:0/2985346256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfe8004810 msgr2=0x7fcfe8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 --2- 172.21.15.133:0/2985346256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfe8004810 0x7fcfe8004c90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009ae0 tx=0x7fcff401ca70 comp rx=0 tx=0).stop 2024-04-25T10:01:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcfff7fe640 1 -- 172.21.15.133:0/2985346256 reap_dead start 2024-04-25T10:01:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 -- 172.21.15.133:0/2985346256 shutdown_connections 2024-04-25T10:01:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 -- 172.21.15.133:0/2985346256 >> 172.21.15.133:0/2985346256 conn(0x7fd000066f70 msgr2=0x7fd00006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 -- 172.21.15.133:0/2985346256 shutdown_connections 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 -- 172.21.15.133:0/2985346256 wait complete. 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 Processor -- start 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 -- start start 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfe8004810 0x7fd0000adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd00013acd0 0x7fd0001f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd00013b130 0x7fd00013b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00015cb70 con 0x7fd00013acd0 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00006e6a0 con 0x7fd00013b130 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0000cdca0 con 0x7fcfe8004810 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffe7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd00013acd0 0x7fd0001f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffe7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd00013acd0 0x7fd0001f30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57060/0 (socket says 172.21.15.133:57060) 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffe7fc640 1 -- 172.21.15.133:0/203977047 learned_addr learned my addr 172.21.15.133:0/203977047 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffeffd640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfe8004810 0x7fd0000adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcfff7fe640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd00013b130 0x7fd00013b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffe7fc640 1 -- 172.21.15.133:0/203977047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfe8004810 msgr2=0x7fd0000adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffe7fc640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfe8004810 0x7fd0000adfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffe7fc640 1 -- 172.21.15.133:0/203977047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd00013b130 msgr2=0x7fd00013b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffe7fc640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd00013b130 0x7fd00013b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffe7fc640 1 -- 172.21.15.133:0/203977047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0000ae510 con 0x7fd00013acd0 2024-04-25T10:01:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcffe7fc640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd00013acd0 0x7fd0001f30d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcff40b0040 tx=0x7fcfe8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcfe77fe640 1 -- 172.21.15.133:0/203977047 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff40bd020 con 0x7fd00013acd0 2024-04-25T10:01:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcfe77fe640 1 -- 172.21.15.133:0/203977047 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff4004020 con 0x7fd00013acd0 2024-04-25T10:01:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 -- 172.21.15.133:0/203977047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0000acf30 con 0x7fd00013acd0 2024-04-25T10:01:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fcfe77fe640 1 -- 172.21.15.133:0/203977047 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff40b2040 con 0x7fd00013acd0 2024-04-25T10:01:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.001+0000 7fd0055a6640 1 -- 172.21.15.133:0/203977047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0000ad4f0 con 0x7fd00013acd0 2024-04-25T10:01:49.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.005+0000 7fd0055a6640 1 -- 172.21.15.133:0/203977047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd000068e80 con 0x7fd00013acd0 2024-04-25T10:01:49.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.005+0000 7fcfe77fe640 1 -- 172.21.15.133:0/203977047 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcff40c0070 con 0x7fd00013acd0 2024-04-25T10:01:49.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.005+0000 7fcfe77fe640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfec066f90 0x7fcfec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.005+0000 7fcffeffd640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfec066f90 0x7fcfec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.005+0000 7fcfe77fe640 1 -- 172.21.15.133:0/203977047 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcff411e900 con 0x7fd00013acd0 2024-04-25T10:01:49.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.005+0000 7fcffeffd640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfec066f90 0x7fcfec069450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcff0001060 tx=0x7fcff0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.009+0000 7fcfe77fe640 1 -- 172.21.15.133:0/203977047 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcff4016780 con 0x7fd00013acd0 2024-04-25T10:01:49.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.153+0000 7fd0055a6640 1 -- 172.21.15.133:0/203977047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd000140b40 con 0x7fd00013acd0 2024-04-25T10:01:49.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fcfe77fe640 1 -- 172.21.15.133:0/203977047 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcff40efcf0 con 0x7fd00013acd0 2024-04-25T10:01:49.161 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 -- 172.21.15.133:0/203977047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfec066f90 msgr2=0x7fcfec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfec066f90 0x7fcfec069450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcff0001060 tx=0x7fcff0009040 comp rx=0 tx=0).stop 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 -- 172.21.15.133:0/203977047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd00013acd0 msgr2=0x7fd0001f30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd00013acd0 0x7fd0001f30d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcff40b0040 tx=0x7fcfe8005840 comp rx=0 tx=0).stop 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 -- 172.21.15.133:0/203977047 shutdown_connections 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd00013b130 0x7fd00013b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfec066f90 0x7fcfec069450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd00013acd0 0x7fd0001f30d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 --2- 172.21.15.133:0/203977047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfe8004810 0x7fd0000adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 -- 172.21.15.133:0/203977047 >> 172.21.15.133:0/203977047 conn(0x7fd000066f70 msgr2=0x7fd00015da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 -- 172.21.15.133:0/203977047 shutdown_connections 2024-04-25T10:01:49.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.157+0000 7fd0055a6640 1 -- 172.21.15.133:0/203977047 wait complete. 2024-04-25T10:01:49.179 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:01:49.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53ce361640 1 Processor -- start 2024-04-25T10:01:49.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53ce361640 1 -- start start 2024-04-25T10:01:49.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53ce361640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f53c815c9a0 0x7f53c80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53ce361640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53c80ce330 0x7f53c80ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53ce361640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c80c61d0 con 0x7f53c80b1f70 2024-04-25T10:01:49.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53ce361640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c80cecc0 con 0x7f53c80ce330 2024-04-25T10:01:49.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53ce361640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c80c64b0 con 0x7f53c815c9a0 2024-04-25T10:01:49.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53cc8fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53c80ce330 0x7f53c80ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53cc8fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53c80ce330 0x7f53c80ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57070/0 (socket says 172.21.15.133:57070) 2024-04-25T10:01:49.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53cc8fe640 1 -- 172.21.15.133:0/1858792711 learned_addr learned my addr 172.21.15.133:0/1858792711 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:49.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53c7fff640 1 --2- 172.21.15.133:0/1858792711 >> v2:172.21.15.162:3300/0 conn(0x7f53c815c9a0 0x7f53c80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53cc8fe640 1 -- 172.21.15.133:0/1858792711 >> v2:172.21.15.162:3300/0 conn(0x7f53c815c9a0 msgr2=0x7f53c80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53cc8fe640 1 --2- 172.21.15.133:0/1858792711 >> v2:172.21.15.162:3300/0 conn(0x7f53c815c9a0 0x7f53c80c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53cc8fe640 1 -- 172.21.15.133:0/1858792711 >> v1:172.21.15.162:6789/0 conn(0x7f53c80b1f70 legacy=0x7f53c806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53cc8fe640 1 -- 172.21.15.133:0/1858792711 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c806d4f0 con 0x7f53c80ce330 2024-04-25T10:01:49.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.289+0000 7f53cc8fe640 1 --2- 172.21.15.133:0/1858792711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c80ce330 0x7f53c80ce780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53c806f5f0 tx=0x7f53c001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf74189a4028fdbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c6ffd640 1 -- 172.21.15.133:0/1858792711 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00a5720 con 0x7f53c80ce330 2024-04-25T10:01:49.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c6ffd640 1 -- 172.21.15.133:0/1858792711 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c00a58c0 con 0x7f53c80ce330 2024-04-25T10:01:49.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c6ffd640 1 -- 172.21.15.133:0/1858792711 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00a5c90 con 0x7f53c80ce330 2024-04-25T10:01:49.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 -- 172.21.15.133:0/1858792711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c80ce330 msgr2=0x7f53c80ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 --2- 172.21.15.133:0/1858792711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c80ce330 0x7f53c80ce780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53c806f5f0 tx=0x7f53c001c9b0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 -- 172.21.15.133:0/1858792711 shutdown_connections 2024-04-25T10:01:49.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 --2- 172.21.15.133:0/1858792711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c80ce330 0x7f53c80ce780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 --2- 172.21.15.133:0/1858792711 >> v2:172.21.15.162:3300/0 conn(0x7f53c815c9a0 0x7f53c80c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 -- 172.21.15.133:0/1858792711 >> 172.21.15.133:0/1858792711 conn(0x7f53c8067000 msgr2=0x7f53c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:49.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 -- 172.21.15.133:0/1858792711 shutdown_connections 2024-04-25T10:01:49.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 -- 172.21.15.133:0/1858792711 wait complete. 2024-04-25T10:01:49.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 Processor -- start 2024-04-25T10:01:49.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 -- start start 2024-04-25T10:01:49.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c80b1f70 0x7f53c81550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53c80ce330 0x7f53c80b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53c815c9a0 0x7f53c80b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c815dad0 con 0x7f53c80b1f70 2024-04-25T10:01:49.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c815d950 con 0x7f53c815c9a0 2024-04-25T10:01:49.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c815dc50 con 0x7f53c80ce330 2024-04-25T10:01:49.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c80b1f70 0x7f53c81550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c80b1f70 0x7f53c81550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57078/0 (socket says 172.21.15.133:57078) 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c7fff640 1 -- 172.21.15.133:0/1525700754 learned_addr learned my addr 172.21.15.133:0/1525700754 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c7fff640 1 -- 172.21.15.133:0/1525700754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53c80ce330 msgr2=0x7f53c80b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c7fff640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53c80ce330 0x7f53c80b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c7fff640 1 -- 172.21.15.133:0/1525700754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53c815c9a0 msgr2=0x7f53c80b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c7fff640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53c815c9a0 0x7f53c80b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c7fff640 1 -- 172.21.15.133:0/1525700754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c8156fa0 con 0x7f53c80b1f70 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c7fff640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c80b1f70 0x7f53c81550f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f53b800c7f0 tx=0x7f53b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c4ff9640 1 -- 172.21.15.133:0/1525700754 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8015070 con 0x7f53c80b1f70 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c4ff9640 1 -- 172.21.15.133:0/1525700754 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b8007820 con 0x7f53c80b1f70 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 -- 172.21.15.133:0/1525700754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c8157280 con 0x7f53c80b1f70 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53c4ff9640 1 -- 172.21.15.133:0/1525700754 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8007c10 con 0x7f53c80b1f70 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.293+0000 7f53ce361640 1 -- 172.21.15.133:0/1525700754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c8150500 con 0x7f53c80b1f70 2024-04-25T10:01:49.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.297+0000 7f53ce361640 1 -- 172.21.15.133:0/1525700754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c8068ef0 con 0x7f53c80b1f70 2024-04-25T10:01:49.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.297+0000 7f53c4ff9640 1 -- 172.21.15.133:0/1525700754 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53b8018070 con 0x7f53c80b1f70 2024-04-25T10:01:49.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.297+0000 7f53c4ff9640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53b4066f60 0x7f53b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.297+0000 7f53c4ff9640 1 -- 172.21.15.133:0/1525700754 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f53b8084d90 con 0x7f53c80b1f70 2024-04-25T10:01:49.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.297+0000 7f53c77fe640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53b4066f60 0x7f53b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.297+0000 7f53c77fe640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53b4066f60 0x7f53b4069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f53c8067fd0 tx=0x7f53bc006bb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.301+0000 7f53c4ff9640 1 -- 172.21.15.133:0/1525700754 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f53b80560f0 con 0x7f53c80b1f70 2024-04-25T10:01:49.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.449+0000 7f53ce361640 1 -- 172.21.15.133:0/1525700754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f53c8150be0 con 0x7f53c80b1f70 2024-04-25T10:01:49.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53c4ff9640 1 -- 172.21.15.133:0/1525700754 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f53b80559f0 con 0x7f53c80b1f70 2024-04-25T10:01:49.458 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:01:49.458 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:01:49.458 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:01:49.458 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:01:49.459 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:01:49.459 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:01:49.459 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 -- 172.21.15.133:0/1525700754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53b4066f60 msgr2=0x7f53b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53b4066f60 0x7f53b4069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f53c8067fd0 tx=0x7f53bc006bb0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 -- 172.21.15.133:0/1525700754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c80b1f70 msgr2=0x7f53c81550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c80b1f70 0x7f53c81550f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f53b800c7f0 tx=0x7f53b800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 -- 172.21.15.133:0/1525700754 shutdown_connections 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53c815c9a0 0x7f53c80b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53c80ce330 0x7f53c80b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53b4066f60 0x7f53b4069420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 --2- 172.21.15.133:0/1525700754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c80b1f70 0x7f53c81550f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 -- 172.21.15.133:0/1525700754 >> 172.21.15.133:0/1525700754 conn(0x7f53c8067000 msgr2=0x7f53c81513e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 -- 172.21.15.133:0/1525700754 shutdown_connections 2024-04-25T10:01:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.453+0000 7f53ce361640 1 -- 172.21.15.133:0/1525700754 wait complete. 2024-04-25T10:01:49.475 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:01:49.476 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:01:49.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.597+0000 7f70b38a5640 1 Processor -- start 2024-04-25T10:01:49.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.597+0000 7f70b38a5640 1 -- start start 2024-04-25T10:01:49.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70ac15b530 0x7f70ac15b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70ac0ceda0 0x7f70ac0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ac15bf40 con 0x7f70ac0cc890 2024-04-25T10:01:49.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ac1589c0 con 0x7f70ac15b530 2024-04-25T10:01:49.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac15c200 con 0x7f70ac0ceda0 2024-04-25T10:01:49.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b1641640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70ac15b530 0x7f70ac15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b1641640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70ac15b530 0x7f70ac15b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57090/0 (socket says 172.21.15.133:57090) 2024-04-25T10:01:49.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b1641640 1 -- 172.21.15.133:0/3405394855 learned_addr learned my addr 172.21.15.133:0/3405394855 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:49.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b1e42640 1 --2- 172.21.15.133:0/3405394855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70ac0ceda0 0x7f70ac0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b1641640 1 -- 172.21.15.133:0/3405394855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70ac0ceda0 msgr2=0x7f70ac0ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b1641640 1 --2- 172.21.15.133:0/3405394855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70ac0ceda0 0x7f70ac0ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b1641640 1 -- 172.21.15.133:0/3405394855 >> v1:172.21.15.162:6789/0 conn(0x7f70ac0cc890 legacy=0x7f70ac0ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b1641640 1 -- 172.21.15.133:0/3405394855 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac15c4e0 con 0x7f70ac15b530 2024-04-25T10:01:49.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b1641640 1 --2- 172.21.15.133:0/3405394855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70ac15b530 0x7f70ac15b910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009870 tx=0x7f70a801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11ae481f4b7566f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70a3fff640 1 -- 172.21.15.133:0/3405394855 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a80b0070 con 0x7f70ac15b530 2024-04-25T10:01:49.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70a3fff640 1 -- 172.21.15.133:0/3405394855 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a80ab600 con 0x7f70ac15b530 2024-04-25T10:01:49.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70a3fff640 1 -- 172.21.15.133:0/3405394855 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a80ab9d0 con 0x7f70ac15b530 2024-04-25T10:01:49.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 -- 172.21.15.133:0/3405394855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70ac15b530 msgr2=0x7f70ac15b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 --2- 172.21.15.133:0/3405394855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70ac15b530 0x7f70ac15b910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009870 tx=0x7f70a801c490 comp rx=0 tx=0).stop 2024-04-25T10:01:49.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 -- 172.21.15.133:0/3405394855 shutdown_connections 2024-04-25T10:01:49.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 --2- 172.21.15.133:0/3405394855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70ac0ceda0 0x7f70ac0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 --2- 172.21.15.133:0/3405394855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70ac15b530 0x7f70ac15b910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 -- 172.21.15.133:0/3405394855 >> 172.21.15.133:0/3405394855 conn(0x7f70ac06d370 msgr2=0x7f70ac06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:49.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 -- 172.21.15.133:0/3405394855 shutdown_connections 2024-04-25T10:01:49.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.601+0000 7f70b38a5640 1 -- 172.21.15.133:0/3405394855 wait complete. 2024-04-25T10:01:49.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b38a5640 1 Processor -- start 2024-04-25T10:01:49.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b38a5640 1 -- start start 2024-04-25T10:01:49.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b38a5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70ac0cc890 0x7f70ac1fa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b38a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70ac0ceda0 0x7f70ac157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b38a5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70ac15b530 0x7f70ac157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b38a5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac15dba0 con 0x7f70ac15b530 2024-04-25T10:01:49.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b38a5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac15da20 con 0x7f70ac0cc890 2024-04-25T10:01:49.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b38a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac15dd20 con 0x7f70ac0ceda0 2024-04-25T10:01:49.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b1e42640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70ac15b530 0x7f70ac157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b1e42640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70ac15b530 0x7f70ac157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57100/0 (socket says 172.21.15.133:57100) 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b1e42640 1 -- 172.21.15.133:0/1002002807 learned_addr learned my addr 172.21.15.133:0/1002002807 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b1641640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70ac0cc890 0x7f70ac1fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b0e40640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70ac0ceda0 0x7f70ac157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b1e42640 1 -- 172.21.15.133:0/1002002807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70ac0ceda0 msgr2=0x7f70ac157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b1e42640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70ac0ceda0 0x7f70ac157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b1e42640 1 -- 172.21.15.133:0/1002002807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70ac0cc890 msgr2=0x7f70ac1fa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b1e42640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70ac0cc890 0x7f70ac1fa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b1e42640 1 -- 172.21.15.133:0/1002002807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac14e770 con 0x7f70ac15b530 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b0e40640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70ac0ceda0 0x7f70ac157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b1e42640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70ac15b530 0x7f70ac157ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70a400cc80 tx=0x7f70a400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70a1ffb640 1 -- 172.21.15.133:0/1002002807 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40039a0 con 0x7f70ac15b530 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70a1ffb640 1 -- 172.21.15.133:0/1002002807 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a4003b60 con 0x7f70ac15b530 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b38a5640 1 -- 172.21.15.133:0/1002002807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac14ea50 con 0x7f70ac15b530 2024-04-25T10:01:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70a1ffb640 1 -- 172.21.15.133:0/1002002807 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4004020 con 0x7f70ac15b530 2024-04-25T10:01:49.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b38a5640 1 -- 172.21.15.133:0/1002002807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ac14f010 con 0x7f70ac15b530 2024-04-25T10:01:49.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70b38a5640 1 -- 172.21.15.133:0/1002002807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ac069000 con 0x7f70ac15b530 2024-04-25T10:01:49.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.605+0000 7f70a1ffb640 1 -- 172.21.15.133:0/1002002807 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70a40041c0 con 0x7f70ac15b530 2024-04-25T10:01:49.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.609+0000 7f70a1ffb640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7090066f60 0x7f7090069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.609+0000 7f70a1ffb640 1 -- 172.21.15.133:0/1002002807 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f70a4085190 con 0x7f70ac15b530 2024-04-25T10:01:49.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.609+0000 7f70b1641640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7090066f60 0x7f7090069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.609+0000 7f70b1641640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7090066f60 0x7f7090069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70a80ad000 tx=0x7f70a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.613+0000 7f70a1ffb640 1 -- 172.21.15.133:0/1002002807 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70a4014030 con 0x7f70ac15b530 2024-04-25T10:01:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.769+0000 7f70b38a5640 1 -- 172.21.15.133:0/1002002807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f70ac0b04c0 con 0x7f70ac15b530 2024-04-25T10:01:49.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.897+0000 7f70a1ffb640 1 -- 172.21.15.133:0/1002002807 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f70a40564f0 con 0x7f70ac15b530 2024-04-25T10:01:49.905 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:01:49.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 -- 172.21.15.133:0/1002002807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7090066f60 msgr2=0x7f7090069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7090066f60 0x7f7090069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70a80ad000 tx=0x7f70a8008040 comp rx=0 tx=0).stop 2024-04-25T10:01:49.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 -- 172.21.15.133:0/1002002807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70ac15b530 msgr2=0x7f70ac157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70ac15b530 0x7f70ac157ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70a400cc80 tx=0x7f70a400a5c0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 -- 172.21.15.133:0/1002002807 shutdown_connections 2024-04-25T10:01:49.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7090066f60 0x7f7090069420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70ac15b530 0x7f70ac157ad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70ac0ceda0 0x7f70ac157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 --2- 172.21.15.133:0/1002002807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70ac0cc890 0x7f70ac1fa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 -- 172.21.15.133:0/1002002807 >> 172.21.15.133:0/1002002807 conn(0x7f70ac06d370 msgr2=0x7f70ac0ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:49.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 -- 172.21.15.133:0/1002002807 shutdown_connections 2024-04-25T10:01:49.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:49.901+0000 7f70b38a5640 1 -- 172.21.15.133:0/1002002807 wait complete. 2024-04-25T10:01:49.935 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T10:01:50.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa3d78640 1 Processor -- start 2024-04-25T10:01:50.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa3d78640 1 -- start start 2024-04-25T10:01:50.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa3d78640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0f9c0ce990 0x7f0f9c0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa3d78640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0f9c0ce1e0 0x7f0f9c15da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa3d78640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f9c0cb2e0 con 0x7f0f9c0ced70 2024-04-25T10:01:50.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa3d78640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f9c0ada40 con 0x7f0f9c0ce1e0 2024-04-25T10:01:50.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa3d78640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f9c0cb5e0 con 0x7f0f9c0ce990 2024-04-25T10:01:50.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa2315640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0f9c0ce1e0 0x7f0f9c15da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa2315640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0f9c0ce1e0 0x7f0f9c15da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57106/0 (socket says 172.21.15.133:57106) 2024-04-25T10:01:50.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa2315640 1 -- 172.21.15.133:0/2074164449 learned_addr learned my addr 172.21.15.133:0/2074164449 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:50.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa1b14640 1 --2- 172.21.15.133:0/2074164449 >> v2:172.21.15.162:3300/0 conn(0x7f0f9c0ce990 0x7f0f9c0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa2315640 1 -- 172.21.15.133:0/2074164449 >> v2:172.21.15.162:3300/0 conn(0x7f0f9c0ce990 msgr2=0x7f0f9c0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa2315640 1 --2- 172.21.15.133:0/2074164449 >> v2:172.21.15.162:3300/0 conn(0x7f0f9c0ce990 0x7f0f9c0d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa2315640 1 -- 172.21.15.133:0/2074164449 >> v1:172.21.15.162:6789/0 conn(0x7f0f9c0ced70 legacy=0x7f0f9c0ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.057+0000 7f0fa2315640 1 -- 172.21.15.133:0/2074164449 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c0cd920 con 0x7f0f9c0ce1e0 2024-04-25T10:01:50.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa2315640 1 --2- 172.21.15.133:0/2074164449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f9c0ce1e0 0x7f0f9c15da30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c099f00 tx=0x7f0f8c01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb90c7cbce2a9dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa0b12640 1 -- 172.21.15.133:0/2074164449 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0a5840 con 0x7f0f9c0ce1e0 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa0b12640 1 -- 172.21.15.133:0/2074164449 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c0a59e0 con 0x7f0f9c0ce1e0 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa0b12640 1 -- 172.21.15.133:0/2074164449 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0a5db0 con 0x7f0f9c0ce1e0 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa3d78640 1 -- 172.21.15.133:0/2074164449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f9c0ce1e0 msgr2=0x7f0f9c15da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa3d78640 1 --2- 172.21.15.133:0/2074164449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f9c0ce1e0 0x7f0f9c15da30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c099f00 tx=0x7f0f8c01c980 comp rx=0 tx=0).stop 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa3d78640 1 -- 172.21.15.133:0/2074164449 shutdown_connections 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa3d78640 1 --2- 172.21.15.133:0/2074164449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f9c0ce1e0 0x7f0f9c15da30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa3d78640 1 --2- 172.21.15.133:0/2074164449 >> v2:172.21.15.162:3300/0 conn(0x7f0f9c0ce990 0x7f0f9c0d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa3d78640 1 -- 172.21.15.133:0/2074164449 >> 172.21.15.133:0/2074164449 conn(0x7f0f9c0999d0 msgr2=0x7f0f9c0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa3d78640 1 -- 172.21.15.133:0/2074164449 shutdown_connections 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa3d78640 1 -- 172.21.15.133:0/2074164449 wait complete. 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa3d78640 1 Processor -- start 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.061+0000 7f0fa3d78640 1 -- start start 2024-04-25T10:01:50.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa3d78640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f9c0ce1e0 0x7f0f9c0b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa3d78640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f9c0ce990 0x7f0f9c154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa3d78640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f9c0ced70 0x7f0f9c0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa3d78640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c0cc260 con 0x7f0f9c0ced70 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa3d78640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c0d1420 con 0x7f0f9c0ce1e0 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa3d78640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c0cc3e0 con 0x7f0f9c0ce990 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa2315640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f9c0ced70 0x7f0f9c0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa2315640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f9c0ced70 0x7f0f9c0ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57108/0 (socket says 172.21.15.133:57108) 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa2315640 1 -- 172.21.15.133:0/3232634193 learned_addr learned my addr 172.21.15.133:0/3232634193 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa1313640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f9c0ce990 0x7f0f9c154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa1b14640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f9c0ce1e0 0x7f0f9c0b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa2315640 1 -- 172.21.15.133:0/3232634193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f9c0ce990 msgr2=0x7f0f9c154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa2315640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f9c0ce990 0x7f0f9c154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa2315640 1 -- 172.21.15.133:0/3232634193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f9c0ce1e0 msgr2=0x7f0f9c0b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa2315640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f9c0ce1e0 0x7f0f9c0b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa2315640 1 -- 172.21.15.133:0/3232634193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c1640e0 con 0x7f0f9c0ced70 2024-04-25T10:01:50.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa1b14640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f9c0ce1e0 0x7f0f9c0b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:50.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa2315640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f9c0ced70 0x7f0f9c0ae3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c01a5b0 tx=0x7f0f8c019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0f9a7fc640 1 -- 172.21.15.133:0/3232634193 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0bc020 con 0x7f0f9c0ced70 2024-04-25T10:01:50.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0f9a7fc640 1 -- 172.21.15.133:0/3232634193 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c0b86b0 con 0x7f0f9c0ced70 2024-04-25T10:01:50.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa3d78640 1 -- 172.21.15.133:0/3232634193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9c0af9a0 con 0x7f0f9c0ced70 2024-04-25T10:01:50.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0f9a7fc640 1 -- 172.21.15.133:0/3232634193 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0b8aa0 con 0x7f0f9c0ced70 2024-04-25T10:01:50.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa3d78640 1 -- 172.21.15.133:0/3232634193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9c0aff60 con 0x7f0f9c0ced70 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa3d78640 1 -- 172.21.15.133:0/3232634193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9c067c70 con 0x7f0f9c0ced70 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0f9a7fc640 1 -- 172.21.15.133:0/3232634193 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f8c0bf020 con 0x7f0f9c0ced70 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0f9a7fc640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f88066f60 0x7f0f88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0f9a7fc640 1 -- 172.21.15.133:0/3232634193 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f0f8c1200b0 con 0x7f0f9c0ced70 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa1b14640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f88066f60 0x7f0f88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.065+0000 7f0fa1b14640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f88066f60 0x7f0f88069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009840 tx=0x7f0f94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.069+0000 7f0f9a7fc640 1 -- 172.21.15.133:0/3232634193 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f8c0c6030 con 0x7f0f9c0ced70 2024-04-25T10:01:50.208 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:50.210 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:50.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.229+0000 7f0fa3d78640 1 -- 172.21.15.133:0/3232634193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f0f9c0b6520 con 0x7f0f9c0ced70 2024-04-25T10:01:50.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.909+0000 7f0f9a7fc640 1 -- 172.21.15.133:0/3232634193 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f0f8c0165a0 con 0x7f0f9c0ced70 2024-04-25T10:01:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:01:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.913+0000 7f0fa3d78640 1 -- 172.21.15.133:0/3232634193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f88066f60 msgr2=0x7f0f88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.913+0000 7f0fa3d78640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f88066f60 0x7f0f88069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009840 tx=0x7f0f94008040 comp rx=0 tx=0).stop 2024-04-25T10:01:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.913+0000 7f0fa3d78640 1 -- 172.21.15.133:0/3232634193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f9c0ced70 msgr2=0x7f0f9c0ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.913+0000 7f0fa3d78640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f9c0ced70 0x7f0f9c0ae3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c01a5b0 tx=0x7f0f8c019720 comp rx=0 tx=0).stop 2024-04-25T10:01:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.913+0000 7f0fa3d78640 1 -- 172.21.15.133:0/3232634193 shutdown_connections 2024-04-25T10:01:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.913+0000 7f0fa3d78640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f88066f60 0x7f0f88069420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.913+0000 7f0fa3d78640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f9c0ced70 0x7f0f9c0ae3f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.913+0000 7f0fa3d78640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f9c0ce990 0x7f0f9c154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.913+0000 7f0fa3d78640 1 --2- 172.21.15.133:0/3232634193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f9c0ce1e0 0x7f0f9c0b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.913+0000 7f0fa3d78640 1 -- 172.21.15.133:0/3232634193 >> 172.21.15.133:0/3232634193 conn(0x7f0f9c0999d0 msgr2=0x7f0f9c0b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.917+0000 7f0fa3d78640 1 -- 172.21.15.133:0/3232634193 shutdown_connections 2024-04-25T10:01:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:50.917+0000 7f0fa3d78640 1 -- 172.21.15.133:0/3232634193 wait complete. 2024-04-25T10:01:50.956 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:01:51.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.065+0000 7f47bd4e1640 1 Processor -- start 2024-04-25T10:01:51.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.065+0000 7f47bd4e1640 1 -- start start 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.065+0000 7f47bd4e1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f47b80b4700 0x7f47b80ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.065+0000 7f47bd4e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47b80ae890 0x7f47b80aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47bd4e1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b815e580 con 0x7f47b80b4320 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47bd4e1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b8155160 con 0x7f47b80b4700 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47bd4e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b815e860 con 0x7f47b80ae890 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b6ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f47b80b4320 0x7f47b815b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57250/0 (socket says 172.21.15.133:57250) 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b6ffd640 1 -- 172.21.15.133:0/1304121248 learned_addr learned my addr 172.21.15.133:0/1304121248 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b77fe640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47b80ae890 0x7f47b80aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744002061 0 0) 0x7f47b815e580 con 0x7f47b80b4320 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b67fc640 1 --2- 172.21.15.133:0/1304121248 >> v2:172.21.15.162:3300/0 conn(0x7f47b80b4700 0x7f47b80ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a0003530 con 0x7f47b80b4320 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3214471194 0 0) 0x7f47a0003530 con 0x7f47b80b4320 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47b80ae890 msgr2=0x7f47b80aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47b80ae890 0x7f47b80aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 >> v2:172.21.15.162:3300/0 conn(0x7f47b80b4700 msgr2=0x7f47b80ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 --2- 172.21.15.133:0/1304121248 >> v2:172.21.15.162:3300/0 conn(0x7f47b80b4700 0x7f47b80ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b815eb40 con 0x7f47b80b4320 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f47a8001680 con 0x7f47b80b4320 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 >> v1:172.21.15.133:6789/0 conn(0x7f47b80b4320 legacy=0x7f47b815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47a00039c0 0x7f47a0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47b80ae890 0x7f47a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a0004d50 0x7f47a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8155160 con 0x7f47a00039c0 2024-04-25T10:01:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b815e860 con 0x7f47a0004d50 2024-04-25T10:01:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a000b9d0 con 0x7f47b80ae890 2024-04-25T10:01:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b6ffd640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47a00039c0 0x7f47a0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b77fe640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a0004d50 0x7f47a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b6ffd640 1 -- 172.21.15.133:0/1304121248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47b80ae890 msgr2=0x7f47a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b6ffd640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47b80ae890 0x7f47a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b6ffd640 1 -- 172.21.15.133:0/1304121248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a0004d50 msgr2=0x7f47a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b6ffd640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a0004d50 0x7f47a00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b6ffd640 1 -- 172.21.15.133:0/1304121248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a0003530 con 0x7f47a00039c0 2024-04-25T10:01:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b77fe640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47a0004d50 0x7f47a00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.069+0000 7f47b6ffd640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47a00039c0 0x7f47a0003de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47a8001060 tx=0x7f47a801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e19abcfb68dc7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8007d80 con 0x7f47a00039c0 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a80a9020 con 0x7f47a00039c0 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b5ffb640 1 -- 172.21.15.133:0/1304121248 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8005b00 con 0x7f47a00039c0 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1304121248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47a00039c0 msgr2=0x7f47a0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 --2- 172.21.15.133:0/1304121248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47a00039c0 0x7f47a0003de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47a8001060 tx=0x7f47a801cc70 comp rx=0 tx=0).stop 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b77fe640 1 -- 172.21.15.133:0/1304121248 reap_dead start 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1304121248 shutdown_connections 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1304121248 >> 172.21.15.133:0/1304121248 conn(0x7f47b80cfd50 msgr2=0x7f47b80d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1304121248 shutdown_connections 2024-04-25T10:01:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1304121248 wait complete. 2024-04-25T10:01:51.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 Processor -- start 2024-04-25T10:01:51.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 -- start start 2024-04-25T10:01:51.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a00039c0 0x7f47b8157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b815bd00 0x7f47b815c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47b81f81a0 0x7f47b81f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8066d90 con 0x7f47b815bd00 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8066c10 con 0x7f47b81f81a0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b806d7b0 con 0x7f47a00039c0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b815bd00 0x7f47b815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b815bd00 0x7f47b815c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57136/0 (socket says 172.21.15.133:57136) 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b67fc640 1 -- 172.21.15.133:0/1193918991 learned_addr learned my addr 172.21.15.133:0/1193918991 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b77fe640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47b81f81a0 0x7f47b81f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b6ffd640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a00039c0 0x7f47b8157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b67fc640 1 -- 172.21.15.133:0/1193918991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a00039c0 msgr2=0x7f47b8157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b67fc640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a00039c0 0x7f47b8157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b67fc640 1 -- 172.21.15.133:0/1193918991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47b81f81a0 msgr2=0x7f47b81f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b67fc640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47b81f81a0 0x7f47b81f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b67fc640 1 -- 172.21.15.133:0/1193918991 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b814ff50 con 0x7f47b815bd00 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b6ffd640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a00039c0 0x7f47b8157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47b67fc640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b815bd00 0x7f47b815c180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00e870 tx=0x7f47ac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f479f7fe640 1 -- 172.21.15.133:0/1193918991 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac017070 con 0x7f47b815bd00 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f479f7fe640 1 -- 172.21.15.133:0/1193918991 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac00c860 con 0x7f47b815bd00 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1193918991 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b815c870 con 0x7f47b815bd00 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f479f7fe640 1 -- 172.21.15.133:0/1193918991 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac00cc50 con 0x7f47b815bd00 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.073+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1193918991 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b815ce30 con 0x7f47b815bd00 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.077+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1193918991 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b8067c70 con 0x7f47b815bd00 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.077+0000 7f479f7fe640 1 -- 172.21.15.133:0/1193918991 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47ac01a070 con 0x7f47b815bd00 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.077+0000 7f479f7fe640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47a4066f60 0x7f47a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.077+0000 7f479f7fe640 1 -- 172.21.15.133:0/1193918991 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f47ac085460 con 0x7f47b815bd00 2024-04-25T10:01:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.077+0000 7f47b6ffd640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47a4066f60 0x7f47a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.077+0000 7f47b6ffd640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47a4066f60 0x7f47a4069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f47a80aa000 tx=0x7f47a8001a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.081+0000 7f479f7fe640 1 -- 172.21.15.133:0/1193918991 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47ac0565b0 con 0x7f47b815bd00 2024-04-25T10:01:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.273+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1193918991 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f47b80cfac0 con 0x7f47b815bd00 2024-04-25T10:01:51.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.937+0000 7f479f7fe640 1 -- 172.21.15.133:0/1193918991 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f47ac055eb0 con 0x7f47b815bd00 2024-04-25T10:01:51.947 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:01:51.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.953+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1193918991 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47a4066f60 msgr2=0x7f47a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.953+0000 7f47bd4e1640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47a4066f60 0x7f47a4069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f47a80aa000 tx=0x7f47a8001a50 comp rx=0 tx=0).stop 2024-04-25T10:01:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.953+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1193918991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b815bd00 msgr2=0x7f47b815c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.953+0000 7f47bd4e1640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b815bd00 0x7f47b815c180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00e870 tx=0x7f47ac00ed30 comp rx=0 tx=0).stop 2024-04-25T10:01:51.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.957+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1193918991 shutdown_connections 2024-04-25T10:01:51.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.957+0000 7f47bd4e1640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47b81f81a0 0x7f47b81f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.957+0000 7f47bd4e1640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47a4066f60 0x7f47a4069420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.957+0000 7f47bd4e1640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b815bd00 0x7f47b815c180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.957+0000 7f47bd4e1640 1 --2- 172.21.15.133:0/1193918991 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47a00039c0 0x7f47b8157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.957+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1193918991 >> 172.21.15.133:0/1193918991 conn(0x7f47b80cfd50 msgr2=0x7f47b80aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.957+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1193918991 shutdown_connections 2024-04-25T10:01:51.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:51.957+0000 7f47bd4e1640 1 -- 172.21.15.133:0/1193918991 wait complete. 2024-04-25T10:01:51.988 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T10:01:51.988 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T10:01:52.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c166c3640 1 Processor -- start 2024-04-25T10:01:52.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c166c3640 1 -- start start 2024-04-25T10:01:52.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c166c3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c10071cb0 0x7f9c1015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c166c3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c1015c250 0x7f9c100cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c166c3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1015c9b0 con 0x7f9c100b4940 2024-04-25T10:01:52.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c166c3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c10072a90 con 0x7f9c10071cb0 2024-04-25T10:01:52.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c166c3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c100a85d0 con 0x7f9c1015c250 2024-04-25T10:01:52.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c0f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c10071cb0 0x7f9c1015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c0f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c10071cb0 0x7f9c1015bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57138/0 (socket says 172.21.15.133:57138) 2024-04-25T10:01:52.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c0f7fe640 1 -- 172.21.15.133:0/947554693 learned_addr learned my addr 172.21.15.133:0/947554693 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:52.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c0f7fe640 1 -- 172.21.15.133:0/947554693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c1015c250 msgr2=0x7f9c100cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:52.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.105+0000 7f9c14c60640 1 --2- 172.21.15.133:0/947554693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c1015c250 0x7f9c100cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c0f7fe640 1 --2- 172.21.15.133:0/947554693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c1015c250 0x7f9c100cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c0f7fe640 1 -- 172.21.15.133:0/947554693 >> v1:172.21.15.133:6789/0 conn(0x7f9c100b4940 legacy=0x7f9c100b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c0f7fe640 1 -- 172.21.15.133:0/947554693 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1015cc90 con 0x7f9c10071cb0 2024-04-25T10:01:52.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c0f7fe640 1 --2- 172.21.15.133:0/947554693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c10071cb0 0x7f9c1015bd10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009870 tx=0x7f9c0401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431322362e0ad102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c0effd640 1 -- 172.21.15.133:0/947554693 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c040a95f0 con 0x7f9c10071cb0 2024-04-25T10:01:52.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c0effd640 1 -- 172.21.15.133:0/947554693 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c040a9790 con 0x7f9c10071cb0 2024-04-25T10:01:52.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c0effd640 1 -- 172.21.15.133:0/947554693 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c040a9b60 con 0x7f9c10071cb0 2024-04-25T10:01:52.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c166c3640 1 -- 172.21.15.133:0/947554693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c10071cb0 msgr2=0x7f9c1015bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c166c3640 1 --2- 172.21.15.133:0/947554693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c10071cb0 0x7f9c1015bd10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009870 tx=0x7f9c0401c530 comp rx=0 tx=0).stop 2024-04-25T10:01:52.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c166c3640 1 -- 172.21.15.133:0/947554693 shutdown_connections 2024-04-25T10:01:52.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c166c3640 1 --2- 172.21.15.133:0/947554693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c1015c250 0x7f9c100cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c166c3640 1 --2- 172.21.15.133:0/947554693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c10071cb0 0x7f9c1015bd10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c166c3640 1 -- 172.21.15.133:0/947554693 >> 172.21.15.133:0/947554693 conn(0x7f9c1006d370 msgr2=0x7f9c1006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:52.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c166c3640 1 -- 172.21.15.133:0/947554693 shutdown_connections 2024-04-25T10:01:52.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.109+0000 7f9c166c3640 1 -- 172.21.15.133:0/947554693 wait complete. 2024-04-25T10:01:52.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c166c3640 1 Processor -- start 2024-04-25T10:01:52.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c166c3640 1 -- start start 2024-04-25T10:01:52.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c166c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c10071cb0 0x7f9c100ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c166c3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c100b4940 0x7f9c1013ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c166c3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c1015c250 0x7f9c100b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c166c3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c100cc960 con 0x7f9c10071cb0 2024-04-25T10:01:52.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c166c3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1015c660 con 0x7f9c1015c250 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c166c3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c100ccae0 con 0x7f9c100b4940 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c10071cb0 0x7f9c100ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c10071cb0 0x7f9c100ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57140/0 (socket says 172.21.15.133:57140) 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0ffff640 1 -- 172.21.15.133:0/3415508644 learned_addr learned my addr 172.21.15.133:0/3415508644 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c14c60640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c1015c250 0x7f9c100b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0f7fe640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c100b4940 0x7f9c1013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0ffff640 1 -- 172.21.15.133:0/3415508644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c100b4940 msgr2=0x7f9c1013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0ffff640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c100b4940 0x7f9c1013ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0ffff640 1 -- 172.21.15.133:0/3415508644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c1015c250 msgr2=0x7f9c100b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0ffff640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c1015c250 0x7f9c100b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0ffff640 1 -- 172.21.15.133:0/3415508644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c100ad0e0 con 0x7f9c10071cb0 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0ffff640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c10071cb0 0x7f9c100ae180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000c400 tx=0x7f9c0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0cff9640 1 -- 172.21.15.133:0/3415508644 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00017070 con 0x7f9c10071cb0 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0cff9640 1 -- 172.21.15.133:0/3415508644 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c00005310 con 0x7f9c10071cb0 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c166c3640 1 -- 172.21.15.133:0/3415508644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c100ad3c0 con 0x7f9c10071cb0 2024-04-25T10:01:52.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0cff9640 1 -- 172.21.15.133:0/3415508644 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c000054c0 con 0x7f9c10071cb0 2024-04-25T10:01:52.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c166c3640 1 -- 172.21.15.133:0/3415508644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c100ad980 con 0x7f9c10071cb0 2024-04-25T10:01:52.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c166c3640 1 -- 172.21.15.133:0/3415508644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c10069000 con 0x7f9c10071cb0 2024-04-25T10:01:52.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0cff9640 1 -- 172.21.15.133:0/3415508644 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c0001a070 con 0x7f9c10071cb0 2024-04-25T10:01:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0cff9640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9bfc066f60 0x7f9bfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0cff9640 1 -- 172.21.15.133:0/3415508644 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c000842d0 con 0x7f9c10071cb0 2024-04-25T10:01:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.113+0000 7f9c0f7fe640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9bfc066f60 0x7f9bfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.117+0000 7f9c0f7fe640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9bfc066f60 0x7f9bfc069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c040ae000 tx=0x7f9c040b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.121+0000 7f9c0cff9640 1 -- 172.21.15.133:0/3415508644 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c00054c90 con 0x7f9c10071cb0 2024-04-25T10:01:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.285+0000 7f9c166c3640 1 -- 172.21.15.133:0/3415508644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f9c10073770 con 0x7f9c10071cb0 2024-04-25T10:01:52.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.945+0000 7f9c0cff9640 1 -- 172.21.15.133:0/3415508644 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f9c00058d30 con 0x7f9c10071cb0 2024-04-25T10:01:52.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.949+0000 7f9c166c3640 1 -- 172.21.15.133:0/3415508644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9bfc066f60 msgr2=0x7f9bfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.949+0000 7f9c166c3640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9bfc066f60 0x7f9bfc069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c040ae000 tx=0x7f9c040b3040 comp rx=0 tx=0).stop 2024-04-25T10:01:52.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.949+0000 7f9c166c3640 1 -- 172.21.15.133:0/3415508644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c10071cb0 msgr2=0x7f9c100ae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.949+0000 7f9c166c3640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c10071cb0 0x7f9c100ae180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000c400 tx=0x7f9c0000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.949+0000 7f9c166c3640 1 -- 172.21.15.133:0/3415508644 shutdown_connections 2024-04-25T10:01:52.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.953+0000 7f9c166c3640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c1015c250 0x7f9c100b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.953+0000 7f9c166c3640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c100b4940 0x7f9c1013ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.953+0000 7f9c166c3640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9bfc066f60 0x7f9bfc069420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.953+0000 7f9c166c3640 1 --2- 172.21.15.133:0/3415508644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c10071cb0 0x7f9c100ae180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.953+0000 7f9c166c3640 1 -- 172.21.15.133:0/3415508644 >> 172.21.15.133:0/3415508644 conn(0x7f9c1006d370 msgr2=0x7f9c1015d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:52.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.953+0000 7f9c166c3640 1 -- 172.21.15.133:0/3415508644 shutdown_connections 2024-04-25T10:01:52.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:52.953+0000 7f9c166c3640 1 -- 172.21.15.133:0/3415508644 wait complete. 2024-04-25T10:01:52.978 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:01:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.101+0000 7fe373310640 1 Processor -- start 2024-04-25T10:01:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.101+0000 7fe373310640 1 -- start start 2024-04-25T10:01:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe373310640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe36c06c3c0 0x7fe36c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe373310640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe36c0ce070 0x7fe36c167860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe373310640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c0af620 con 0x7fe36c15eae0 2024-04-25T10:01:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe373310640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c06b320 con 0x7fe36c0ce070 2024-04-25T10:01:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe373310640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c06b600 con 0x7fe36c06c3c0 2024-04-25T10:01:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe3710ac640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe36c15eae0 0x7fe36c0cd960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57264/0 (socket says 172.21.15.133:57264) 2024-04-25T10:01:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe3710ac640 1 -- 172.21.15.133:0/2896894583 learned_addr learned my addr 172.21.15.133:0/2896894583 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:01:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe3718ad640 1 --2- 172.21.15.133:0/2896894583 >> v2:172.21.15.133:3300/0 conn(0x7fe36c0ce070 0x7fe36c167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe3708ab640 1 --2- 172.21.15.133:0/2896894583 >> v2:172.21.15.162:3300/0 conn(0x7fe36c06c3c0 0x7fe36c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe3718ad640 1 -- 172.21.15.133:0/2896894583 >> v2:172.21.15.162:3300/0 conn(0x7fe36c06c3c0 msgr2=0x7fe36c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe3718ad640 1 --2- 172.21.15.133:0/2896894583 >> v2:172.21.15.162:3300/0 conn(0x7fe36c06c3c0 0x7fe36c06ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe3718ad640 1 -- 172.21.15.133:0/2896894583 >> v1:172.21.15.133:6789/0 conn(0x7fe36c15eae0 legacy=0x7fe36c0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe3718ad640 1 -- 172.21.15.133:0/2896894583 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c06b8e0 con 0x7fe36c0ce070 2024-04-25T10:01:53.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe363fff640 1 -- 172.21.15.133:0/2896894583 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162941194 0 0) 0x7fe36c0af620 con 0x7fe36c15eae0 2024-04-25T10:01:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe3718ad640 1 --2- 172.21.15.133:0/2896894583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe36c0ce070 0x7fe36c167860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe36c154f40 tx=0x7fe36401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd533046cbece5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.105+0000 7fe363fff640 1 -- 172.21.15.133:0/2896894583 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3640b2070 con 0x7fe36c0ce070 2024-04-25T10:01:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe363fff640 1 -- 172.21.15.133:0/2896894583 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3640a5900 con 0x7fe36c0ce070 2024-04-25T10:01:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe363fff640 1 -- 172.21.15.133:0/2896894583 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3640a5cd0 con 0x7fe36c0ce070 2024-04-25T10:01:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 -- 172.21.15.133:0/2896894583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe36c0ce070 msgr2=0x7fe36c167860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 --2- 172.21.15.133:0/2896894583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe36c0ce070 0x7fe36c167860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe36c154f40 tx=0x7fe36401c930 comp rx=0 tx=0).stop 2024-04-25T10:01:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 -- 172.21.15.133:0/2896894583 shutdown_connections 2024-04-25T10:01:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 --2- 172.21.15.133:0/2896894583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe36c0ce070 0x7fe36c167860 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 --2- 172.21.15.133:0/2896894583 >> v2:172.21.15.162:3300/0 conn(0x7fe36c06c3c0 0x7fe36c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 -- 172.21.15.133:0/2896894583 >> 172.21.15.133:0/2896894583 conn(0x7fe36c0cc810 msgr2=0x7fe36c0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 -- 172.21.15.133:0/2896894583 shutdown_connections 2024-04-25T10:01:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 -- 172.21.15.133:0/2896894583 wait complete. 2024-04-25T10:01:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 Processor -- start 2024-04-25T10:01:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 -- start start 2024-04-25T10:01:53.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe36c06c3c0 0x7fe36c154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe36c0ce070 0x7fe36c0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe36c15eae0 0x7fe36c0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c16a950 con 0x7fe36c06c3c0 2024-04-25T10:01:53.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c16a7d0 con 0x7fe36c15eae0 2024-04-25T10:01:53.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c16aad0 con 0x7fe36c0ce070 2024-04-25T10:01:53.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe3710ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe36c06c3c0 0x7fe36c154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe3710ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe36c06c3c0 0x7fe36c154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57152/0 (socket says 172.21.15.133:57152) 2024-04-25T10:01:53.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe3710ac640 1 -- 172.21.15.133:0/518726354 learned_addr learned my addr 172.21.15.133:0/518726354 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:53.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe3710ac640 1 -- 172.21.15.133:0/518726354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe36c0ce070 msgr2=0x7fe36c0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe3710ac640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe36c0ce070 0x7fe36c0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe3710ac640 1 -- 172.21.15.133:0/518726354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe36c15eae0 msgr2=0x7fe36c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe3710ac640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe36c15eae0 0x7fe36c0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe3710ac640 1 -- 172.21.15.133:0/518726354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0b5b40 con 0x7fe36c06c3c0 2024-04-25T10:01:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe3710ac640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe36c06c3c0 0x7fe36c154f20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe3680011f0 tx=0x7fe36800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe361ffb640 1 -- 172.21.15.133:0/518726354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368018070 con 0x7fe36c06c3c0 2024-04-25T10:01:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe361ffb640 1 -- 172.21.15.133:0/518726354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3680074e0 con 0x7fe36c06c3c0 2024-04-25T10:01:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 -- 172.21.15.133:0/518726354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c0b4290 con 0x7fe36c06c3c0 2024-04-25T10:01:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe361ffb640 1 -- 172.21.15.133:0/518726354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368004a10 con 0x7fe36c06c3c0 2024-04-25T10:01:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.109+0000 7fe373310640 1 -- 172.21.15.133:0/518726354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c0b4b30 con 0x7fe36c06c3c0 2024-04-25T10:01:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.113+0000 7fe373310640 1 -- 172.21.15.133:0/518726354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c067c70 con 0x7fe36c06c3c0 2024-04-25T10:01:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.113+0000 7fe361ffb640 1 -- 172.21.15.133:0/518726354 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe36801b070 con 0x7fe36c06c3c0 2024-04-25T10:01:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.113+0000 7fe361ffb640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe350066f60 0x7fe350069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.113+0000 7fe3708ab640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe350066f60 0x7fe350069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.113+0000 7fe361ffb640 1 -- 172.21.15.133:0/518726354 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6007+0+0 (secure 0 0 0) 0x7fe368084540 con 0x7fe36c06c3c0 2024-04-25T10:01:53.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.113+0000 7fe3708ab640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe350066f60 0x7fe350069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe36c157990 tx=0x7fe35c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.117+0000 7fe361ffb640 1 -- 172.21.15.133:0/518726354 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe368055380 con 0x7fe36c06c3c0 2024-04-25T10:01:53.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:53.273+0000 7fe373310640 1 -- 172.21.15.133:0/518726354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe36c15dce0 con 0x7fe36c06c3c0 2024-04-25T10:01:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.965+0000 7fe361ffb640 1 -- 172.21.15.133:0/518726354 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fe36805b080 con 0x7fe36c06c3c0 2024-04-25T10:01:54.972 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data_ec' created 2024-04-25T10:01:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 -- 172.21.15.133:0/518726354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe350066f60 msgr2=0x7fe350069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe350066f60 0x7fe350069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe36c157990 tx=0x7fe35c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 -- 172.21.15.133:0/518726354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe36c06c3c0 msgr2=0x7fe36c154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe36c06c3c0 0x7fe36c154f20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe3680011f0 tx=0x7fe36800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 -- 172.21.15.133:0/518726354 shutdown_connections 2024-04-25T10:01:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe36c15eae0 0x7fe36c0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe36c0ce070 0x7fe36c0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe350066f60 0x7fe350069420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 --2- 172.21.15.133:0/518726354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe36c06c3c0 0x7fe36c154f20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 -- 172.21.15.133:0/518726354 >> 172.21.15.133:0/518726354 conn(0x7fe36c0cc810 msgr2=0x7fe36c06e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 -- 172.21.15.133:0/518726354 shutdown_connections 2024-04-25T10:01:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:54.969+0000 7fe373310640 1 -- 172.21.15.133:0/518726354 wait complete. 2024-04-25T10:01:55.004 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T10:01:55.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c9f45640 1 Processor -- start 2024-04-25T10:01:55.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c9f45640 1 -- start start 2024-04-25T10:01:55.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c9f45640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c40b4940 0x7f92c40b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c9f45640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f92c4071cb0 0x7f92c415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c9f45640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92c415c250 0x7f92c40cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c9f45640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c4072a90 con 0x7f92c415c250 2024-04-25T10:01:55.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c9f45640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c40a85d0 con 0x7f92c4071cb0 2024-04-25T10:01:55.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c9f45640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c40b5260 con 0x7f92c40b4940 2024-04-25T10:01:55.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92c415c250 0x7f92c40cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92c415c250 0x7f92c40cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57154/0 (socket says 172.21.15.133:57154) 2024-04-25T10:01:55.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c3fff640 1 -- 172.21.15.133:0/3911483404 learned_addr learned my addr 172.21.15.133:0/3911483404 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:55.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92baffd640 1 --2- 172.21.15.133:0/3911483404 >> v2:172.21.15.162:3300/0 conn(0x7f92c4071cb0 0x7f92c415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c37fe640 1 --2- 172.21.15.133:0/3911483404 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c40b4940 0x7f92c40b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c3fff640 1 -- 172.21.15.133:0/3911483404 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c40b4940 msgr2=0x7f92c40b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:55.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c3fff640 1 --2- 172.21.15.133:0/3911483404 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c40b4940 0x7f92c40b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c3fff640 1 -- 172.21.15.133:0/3911483404 >> v2:172.21.15.162:3300/0 conn(0x7f92c4071cb0 msgr2=0x7f92c415bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:55.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c3fff640 1 --2- 172.21.15.133:0/3911483404 >> v2:172.21.15.162:3300/0 conn(0x7f92c4071cb0 0x7f92c415bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.121+0000 7f92c3fff640 1 -- 172.21.15.133:0/3911483404 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c415c9b0 con 0x7f92c415c250 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c3fff640 1 --2- 172.21.15.133:0/3911483404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92c415c250 0x7f92c40cc010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f92c4070aa0 tx=0x7f92bc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24841cac2914dde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c2ffd640 1 -- 172.21.15.133:0/3911483404 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0a5840 con 0x7f92c415c250 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c2ffd640 1 -- 172.21.15.133:0/3911483404 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc0a59e0 con 0x7f92c415c250 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c2ffd640 1 -- 172.21.15.133:0/3911483404 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0a5db0 con 0x7f92c415c250 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c9f45640 1 -- 172.21.15.133:0/3911483404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92c415c250 msgr2=0x7f92c40cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c9f45640 1 --2- 172.21.15.133:0/3911483404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92c415c250 0x7f92c40cc010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f92c4070aa0 tx=0x7f92bc01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c9f45640 1 -- 172.21.15.133:0/3911483404 shutdown_connections 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c9f45640 1 --2- 172.21.15.133:0/3911483404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92c415c250 0x7f92c40cc010 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c9f45640 1 --2- 172.21.15.133:0/3911483404 >> v2:172.21.15.162:3300/0 conn(0x7f92c4071cb0 0x7f92c415bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c9f45640 1 --2- 172.21.15.133:0/3911483404 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c40b4940 0x7f92c40b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c9f45640 1 -- 172.21.15.133:0/3911483404 >> 172.21.15.133:0/3911483404 conn(0x7f92c406d370 msgr2=0x7f92c406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c9f45640 1 -- 172.21.15.133:0/3911483404 shutdown_connections 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.125+0000 7f92c9f45640 1 -- 172.21.15.133:0/3911483404 wait complete. 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c9f45640 1 Processor -- start 2024-04-25T10:01:55.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c9f45640 1 -- start start 2024-04-25T10:01:55.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c9f45640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c4071cb0 0x7f92c415a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c9f45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92c40b4940 0x7f92c413dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c9f45640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c415c250 0x7f92c4145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c9f45640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c415c680 con 0x7f92c40b4940 2024-04-25T10:01:55.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c9f45640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c4071790 con 0x7f92c415c250 2024-04-25T10:01:55.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c9f45640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c40cc960 con 0x7f92c4071cb0 2024-04-25T10:01:55.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92baffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92c40b4940 0x7f92c413dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92baffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92c40b4940 0x7f92c413dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57160/0 (socket says 172.21.15.133:57160) 2024-04-25T10:01:55.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92baffd640 1 -- 172.21.15.133:0/1735922740 learned_addr learned my addr 172.21.15.133:0/1735922740 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:55.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c37fe640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c4071cb0 0x7f92c415a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c3fff640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c415c250 0x7f92c4145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92baffd640 1 -- 172.21.15.133:0/1735922740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c4071cb0 msgr2=0x7f92c415a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92baffd640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c4071cb0 0x7f92c415a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92baffd640 1 -- 172.21.15.133:0/1735922740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c415c250 msgr2=0x7f92c4145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92baffd640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c415c250 0x7f92c4145ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92baffd640 1 -- 172.21.15.133:0/1735922740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c413c830 con 0x7f92c40b4940 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92baffd640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92c40b4940 0x7f92c413dbb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00d800 tx=0x7f92ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c0ff9640 1 -- 172.21.15.133:0/1735922740 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac018020 con 0x7f92c40b4940 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c0ff9640 1 -- 172.21.15.133:0/1735922740 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac0096a0 con 0x7f92c40b4940 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c9f45640 1 -- 172.21.15.133:0/1735922740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c413cb10 con 0x7f92c40b4940 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c0ff9640 1 -- 172.21.15.133:0/1735922740 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac009a50 con 0x7f92c40b4940 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c9f45640 1 -- 172.21.15.133:0/1735922740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c413d0d0 con 0x7f92c40b4940 2024-04-25T10:01:55.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c9f45640 1 -- 172.21.15.133:0/1735922740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c4069000 con 0x7f92c40b4940 2024-04-25T10:01:55.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.129+0000 7f92c0ff9640 1 -- 172.21.15.133:0/1735922740 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92ac01b020 con 0x7f92c40b4940 2024-04-25T10:01:55.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.133+0000 7f92c0ff9640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f92a4066f90 0x7f92a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.133+0000 7f92c0ff9640 1 -- 172.21.15.133:0/1735922740 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f92ac0858c0 con 0x7f92c40b4940 2024-04-25T10:01:55.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.133+0000 7f92c37fe640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f92a4066f90 0x7f92a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.133+0000 7f92c37fe640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f92a4066f90 0x7f92a4069450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f92c413e450 tx=0x7f92b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.137+0000 7f92c0ff9640 1 -- 172.21.15.133:0/1735922740 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92ac022020 con 0x7f92c40b4940 2024-04-25T10:01:55.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.297+0000 7f92c9f45640 1 -- 172.21.15.133:0/1735922740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f92c415afe0 con 0x7f92c40b4940 2024-04-25T10:01:55.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.969+0000 7f92c0ff9640 1 -- 172.21.15.133:0/1735922740 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f92ac0565b0 con 0x7f92c40b4940 2024-04-25T10:01:55.974 INFO:teuthology.orchestra.run.smithi133.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T10:01:55.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.969+0000 7f92c9f45640 1 -- 172.21.15.133:0/1735922740 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f92a4066f90 msgr2=0x7f92a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.969+0000 7f92c9f45640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f92a4066f90 0x7f92a4069450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f92c413e450 tx=0x7f92b4008040 comp rx=0 tx=0).stop 2024-04-25T10:01:55.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.969+0000 7f92c9f45640 1 -- 172.21.15.133:0/1735922740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92c40b4940 msgr2=0x7f92c413dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.969+0000 7f92c9f45640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92c40b4940 0x7f92c413dbb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00d800 tx=0x7f92ac00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.973+0000 7f92c9f45640 1 -- 172.21.15.133:0/1735922740 shutdown_connections 2024-04-25T10:01:55.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.973+0000 7f92c9f45640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92c415c250 0x7f92c4145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.973+0000 7f92c9f45640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f92a4066f90 0x7f92a4069450 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.973+0000 7f92c9f45640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92c40b4940 0x7f92c413dbb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.973+0000 7f92c9f45640 1 --2- 172.21.15.133:0/1735922740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92c4071cb0 0x7f92c415a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.973+0000 7f92c9f45640 1 -- 172.21.15.133:0/1735922740 >> 172.21.15.133:0/1735922740 conn(0x7f92c406d370 msgr2=0x7f92c4141e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:55.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.973+0000 7f92c9f45640 1 -- 172.21.15.133:0/1735922740 shutdown_connections 2024-04-25T10:01:55.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:55.973+0000 7f92c9f45640 1 -- 172.21.15.133:0/1735922740 wait complete. 2024-04-25T10:01:56.019 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T10:01:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf7ec6640 1 Processor -- start 2024-04-25T10:01:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf7ec6640 1 -- start start 2024-04-25T10:01:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf7ec6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 0x7fbcf0140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf7ec6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbcf015d9f0 0x7fbcf015de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf7ec6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf00cf280 con 0x7fbcf0142920 2024-04-25T10:01:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf7ec6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf006d440 con 0x7fbcf015d9f0 2024-04-25T10:01:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf7ec6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf006d720 con 0x7fbcf0143560 2024-04-25T10:01:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf6463640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbcf015d9f0 0x7fbcf015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf5461640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 0x7fbcf0140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf5461640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 0x7fbcf0140a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:42046/0 (socket says 172.21.15.133:42046) 2024-04-25T10:01:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf5461640 1 -- 172.21.15.133:0/255145100 learned_addr learned my addr 172.21.15.133:0/255145100 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf6463640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbcf015d9f0 0x7fbcf015de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:51644/0 (socket says 172.21.15.133:51644) 2024-04-25T10:01:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.137+0000 7fbcf5c62640 1 --1- 172.21.15.133:0/255145100 >> v1:172.21.15.162:6789/0 conn(0x7fbcf0142920 0x7fbcf00ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:46150/0 (socket says 172.21.15.133:46150) 2024-04-25T10:01:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf5461640 1 -- 172.21.15.133:0/255145100 >> v2:172.21.15.162:3300/0 conn(0x7fbcf015d9f0 msgr2=0x7fbcf015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf5461640 1 --2- 172.21.15.133:0/255145100 >> v2:172.21.15.162:3300/0 conn(0x7fbcf015d9f0 0x7fbcf015de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf5461640 1 -- 172.21.15.133:0/255145100 >> v1:172.21.15.162:6789/0 conn(0x7fbcf0142920 legacy=0x7fbcf00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf5461640 1 -- 172.21.15.133:0/255145100 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf006da00 con 0x7fbcf0143560 2024-04-25T10:01:56.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf5461640 1 --2- 172.21.15.133:0/255145100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 0x7fbcf0140a40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbce000aa10 tx=0x7fbce001d8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78107a55235ee09e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf4c60640 1 -- 172.21.15.133:0/255145100 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00b3070 con 0x7fbcf0143560 2024-04-25T10:01:56.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf4c60640 1 -- 172.21.15.133:0/255145100 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce00a6800 con 0x7fbcf0143560 2024-04-25T10:01:56.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf4c60640 1 -- 172.21.15.133:0/255145100 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce000b040 con 0x7fbcf0143560 2024-04-25T10:01:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/255145100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 msgr2=0x7fbcf0140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf7ec6640 1 --2- 172.21.15.133:0/255145100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 0x7fbcf0140a40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbce000aa10 tx=0x7fbce001d8a0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/255145100 shutdown_connections 2024-04-25T10:01:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf7ec6640 1 --2- 172.21.15.133:0/255145100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 0x7fbcf0140a40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf7ec6640 1 --2- 172.21.15.133:0/255145100 >> v2:172.21.15.162:3300/0 conn(0x7fbcf015d9f0 0x7fbcf015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/255145100 >> 172.21.15.133:0/255145100 conn(0x7fbcf0066f70 msgr2=0x7fbcf006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/255145100 shutdown_connections 2024-04-25T10:01:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.141+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/255145100 wait complete. 2024-04-25T10:01:56.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf7ec6640 1 Processor -- start 2024-04-25T10:01:56.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf7ec6640 1 -- start start 2024-04-25T10:01:56.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf7ec6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf0142920 0x7fbcf013dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf7ec6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 0x7fbcf0131d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf7ec6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf015d9f0 0x7fbcf0156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf7ec6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf006eac0 con 0x7fbcf015d9f0 2024-04-25T10:01:56.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf7ec6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf006e940 con 0x7fbcf0142920 2024-04-25T10:01:56.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf7ec6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf0166d90 con 0x7fbcf0143560 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf6463640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf015d9f0 0x7fbcf0156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf6463640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf015d9f0 0x7fbcf0156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57164/0 (socket says 172.21.15.133:57164) 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf6463640 1 -- 172.21.15.133:0/1540142 learned_addr learned my addr 172.21.15.133:0/1540142 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf5c62640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf0142920 0x7fbcf013dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf5461640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 0x7fbcf0131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf6463640 1 -- 172.21.15.133:0/1540142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 msgr2=0x7fbcf0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf6463640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 0x7fbcf0131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf6463640 1 -- 172.21.15.133:0/1540142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf0142920 msgr2=0x7fbcf013dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf6463640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf0142920 0x7fbcf013dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf6463640 1 -- 172.21.15.133:0/1540142 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf00b1740 con 0x7fbcf015d9f0 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf5461640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 0x7fbcf0131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf5c62640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf0142920 0x7fbcf013dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf6463640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf015d9f0 0x7fbcf0156f70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbce800efc0 tx=0x7fbce800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbce67fc640 1 -- 172.21.15.133:0/1540142 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8018020 con 0x7fbcf015d9f0 2024-04-25T10:01:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbce67fc640 1 -- 172.21.15.133:0/1540142 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce8014480 con 0x7fbcf015d9f0 2024-04-25T10:01:56.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/1540142 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf00b06a0 con 0x7fbcf015d9f0 2024-04-25T10:01:56.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbce67fc640 1 -- 172.21.15.133:0/1540142 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8014870 con 0x7fbcf015d9f0 2024-04-25T10:01:56.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/1540142 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf00b0f40 con 0x7fbcf015d9f0 2024-04-25T10:01:56.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.145+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/1540142 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf00b0a90 con 0x7fbcf015d9f0 2024-04-25T10:01:56.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.149+0000 7fbce67fc640 1 -- 172.21.15.133:0/1540142 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbce801b020 con 0x7fbcf015d9f0 2024-04-25T10:01:56.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.149+0000 7fbce67fc640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcd4066f60 0x7fbcd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.149+0000 7fbce67fc640 1 -- 172.21.15.133:0/1540142 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbce8085f00 con 0x7fbcf015d9f0 2024-04-25T10:01:56.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.149+0000 7fbcf5c62640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcd4066f60 0x7fbcd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.149+0000 7fbcf5c62640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcd4066f60 0x7fbcd4069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbcec000f70 tx=0x7fbcec007d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.153+0000 7fbce67fc640 1 -- 172.21.15.133:0/1540142 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbce8022070 con 0x7fbcf015d9f0 2024-04-25T10:01:56.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.345+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/1540142 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fbcf00afa70 con 0x7fbcf015d9f0 2024-04-25T10:01:57.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:56.997+0000 7fbce67fc640 1 -- 172.21.15.133:0/1540142 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fbce8010040 con 0x7fbcf015d9f0 2024-04-25T10:01:57.003 INFO:teuthology.orchestra.run.smithi133.stderr:added data pool 4 to fsmap 2024-04-25T10:01:57.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/1540142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcd4066f60 msgr2=0x7fbcd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcd4066f60 0x7fbcd4069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbcec000f70 tx=0x7fbcec007d70 comp rx=0 tx=0).stop 2024-04-25T10:01:57.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/1540142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf015d9f0 msgr2=0x7fbcf0156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf015d9f0 0x7fbcf0156f70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbce800efc0 tx=0x7fbce800c490 comp rx=0 tx=0).stop 2024-04-25T10:01:57.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/1540142 shutdown_connections 2024-04-25T10:01:57.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcd4066f60 0x7fbcd4069420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf015d9f0 0x7fbcf0156f70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcf0143560 0x7fbcf0131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 --2- 172.21.15.133:0/1540142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcf0142920 0x7fbcf013dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/1540142 >> 172.21.15.133:0/1540142 conn(0x7fbcf0066f70 msgr2=0x7fbcf00b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/1540142 shutdown_connections 2024-04-25T10:01:57.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.001+0000 7fbcf7ec6640 1 -- 172.21.15.133:0/1540142 wait complete. 2024-04-25T10:01:57.027 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f477af13640 1 Processor -- start 2024-04-25T10:01:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f477af13640 1 -- start start 2024-04-25T10:01:57.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f477af13640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4774161f00 0x7f4774072f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f477af13640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47740d1210 0x7f47740cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f477af13640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47740b3b00 con 0x7f47740b3670 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f477af13640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47741622e0 con 0x7f4774161f00 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f477af13640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47740d0730 con 0x7f47740d1210 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f4778caf640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4774161f00 0x7f4774072f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f4778caf640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4774161f00 0x7f4774072f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46006/0 (socket says 172.21.15.133:46006) 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f4778caf640 1 -- 172.21.15.133:0/1732218845 learned_addr learned my addr 172.21.15.133:0/1732218845 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f47794b0640 1 --2- 172.21.15.133:0/1732218845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47740d1210 0x7f47740cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f4778caf640 1 -- 172.21.15.133:0/1732218845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47740d1210 msgr2=0x7f47740cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f4778caf640 1 --2- 172.21.15.133:0/1732218845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47740d1210 0x7f47740cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f4778caf640 1 -- 172.21.15.133:0/1732218845 >> v1:172.21.15.162:6789/0 conn(0x7f47740b3670 legacy=0x7f477406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.145+0000 7f4778caf640 1 -- 172.21.15.133:0/1732218845 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47740d0a10 con 0x7f4774161f00 2024-04-25T10:01:57.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f4778caf640 1 --2- 172.21.15.133:0/1732218845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4774161f00 0x7f4774072f10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4768009870 tx=0x7f476801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626fdeb1b34ce106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f47737fe640 1 -- 172.21.15.133:0/1732218845 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47680ab5f0 con 0x7f4774161f00 2024-04-25T10:01:57.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f47737fe640 1 -- 172.21.15.133:0/1732218845 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47680b0070 con 0x7f4774161f00 2024-04-25T10:01:57.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f47737fe640 1 -- 172.21.15.133:0/1732218845 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47680ab9b0 con 0x7f4774161f00 2024-04-25T10:01:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 -- 172.21.15.133:0/1732218845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4774161f00 msgr2=0x7f4774072f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 --2- 172.21.15.133:0/1732218845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4774161f00 0x7f4774072f10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4768009870 tx=0x7f476801c520 comp rx=0 tx=0).stop 2024-04-25T10:01:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 -- 172.21.15.133:0/1732218845 shutdown_connections 2024-04-25T10:01:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 --2- 172.21.15.133:0/1732218845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47740d1210 0x7f47740cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 --2- 172.21.15.133:0/1732218845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4774161f00 0x7f4774072f10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 -- 172.21.15.133:0/1732218845 >> 172.21.15.133:0/1732218845 conn(0x7f47740b2370 msgr2=0x7f47740b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 -- 172.21.15.133:0/1732218845 shutdown_connections 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 -- 172.21.15.133:0/1732218845 wait complete. 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 Processor -- start 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 -- start start 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47740d1210 0x7f4774159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4774161f00 0x7f4774158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.149+0000 7f477af13640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f477406fde0 0x7f4774162b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f477af13640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47740ce230 con 0x7f4774161f00 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f477af13640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477406b5f0 con 0x7f47740d1210 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f477af13640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47740ce3b0 con 0x7f477406fde0 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4773fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4774161f00 0x7f4774158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4773fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4774161f00 0x7f4774158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46018/0 (socket says 172.21.15.133:46018) 2024-04-25T10:01:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4773fff640 1 -- 172.21.15.133:0/2479169994 learned_addr learned my addr 172.21.15.133:0/2479169994 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4778caf640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47740d1210 0x7f4774159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f47794b0640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f477406fde0 0x7f4774162b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4773fff640 1 -- 172.21.15.133:0/2479169994 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f477406fde0 msgr2=0x7f4774162b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4773fff640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f477406fde0 0x7f4774162b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4773fff640 1 -- 172.21.15.133:0/2479169994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47740d1210 msgr2=0x7f4774159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4773fff640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47740d1210 0x7f4774159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4773fff640 1 -- 172.21.15.133:0/2479169994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4774159020 con 0x7f4774161f00 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4778caf640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47740d1210 0x7f4774159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f47794b0640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f477406fde0 0x7f4774162b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4773fff640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4774161f00 0x7f4774158ae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f476400c480 tx=0x7f476400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f47717fa640 1 -- 172.21.15.133:0/2479169994 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764012830 con 0x7f4774161f00 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f47717fa640 1 -- 172.21.15.133:0/2479169994 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764017070 con 0x7f4774161f00 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f477af13640 1 -- 172.21.15.133:0/2479169994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477415a3f0 con 0x7f4774161f00 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f47717fa640 1 -- 172.21.15.133:0/2479169994 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764012c30 con 0x7f4774161f00 2024-04-25T10:01:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f477af13640 1 -- 172.21.15.133:0/2479169994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477415a9b0 con 0x7f4774161f00 2024-04-25T10:01:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f477af13640 1 -- 172.21.15.133:0/2479169994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477415abb0 con 0x7f4774161f00 2024-04-25T10:01:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f47717fa640 1 -- 172.21.15.133:0/2479169994 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f476401c460 con 0x7f4774161f00 2024-04-25T10:01:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f47717fa640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4760066f60 0x7f4760069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f47717fa640 1 -- 172.21.15.133:0/2479169994 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4764085090 con 0x7f4774161f00 2024-04-25T10:01:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4778caf640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4760066f60 0x7f4760069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.153+0000 7f4778caf640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4760066f60 0x7f4760069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47680af000 tx=0x7f4768008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.161+0000 7f47717fa640 1 -- 172.21.15.133:0/2479169994 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f477415abb0 con 0x7f4774161f00 2024-04-25T10:01:57.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.345+0000 7f477af13640 1 -- 172.21.15.133:0/2479169994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4774067c70 con 0x7f4774161f00 2024-04-25T10:01:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f47717fa640 1 -- 172.21.15.133:0/2479169994 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f4764055df0 con 0x7f4774161f00 2024-04-25T10:01:57.353 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:01:57.354 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2230831968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2230831968},{"type":"v1","addr":"172.21.15.162:6835","nonce":2230831968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1456396097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1456396097},{"type":"v1","addr":"172.21.15.133:6836","nonce":1456396097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:51.276631+0000","modified":"2024-04-25T10:01:56.349405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2462154167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2462154167},{"type":"v1","addr":"172.21.15.133:6837","nonce":2462154167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T10:01:57.354 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-04-25T10:01:57.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 -- 172.21.15.133:0/2479169994 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4760066f60 msgr2=0x7f4760069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4760066f60 0x7f4760069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47680af000 tx=0x7f4768008040 comp rx=0 tx=0).stop 2024-04-25T10:01:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 -- 172.21.15.133:0/2479169994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4774161f00 msgr2=0x7f4774158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4774161f00 0x7f4774158ae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f476400c480 tx=0x7f476400c940 comp rx=0 tx=0).stop 2024-04-25T10:01:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 -- 172.21.15.133:0/2479169994 shutdown_connections 2024-04-25T10:01:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f477406fde0 0x7f4774162b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4760066f60 0x7f4760069420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4774161f00 0x7f4774158ae0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 --2- 172.21.15.133:0/2479169994 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47740d1210 0x7f4774159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 -- 172.21.15.133:0/2479169994 >> 172.21.15.133:0/2479169994 conn(0x7f47740b2370 msgr2=0x7f47740cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 -- 172.21.15.133:0/2479169994 shutdown_connections 2024-04-25T10:01:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.349+0000 7f477af13640 1 -- 172.21.15.133:0/2479169994 wait complete. 2024-04-25T10:01:57.371 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:57.415 INFO:tasks.ceph.mds.c.smithi133.stderr:2024-04-25T10:01:57.409+0000 7f6220dec640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:01:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f86942cb640 1 Processor -- start 2024-04-25T10:01:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f86942cb640 1 -- start start 2024-04-25T10:01:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f86942cb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868c06dca0 0x7f868c0b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f86942cb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f868c06bda0 0x7f868c0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f86942cb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868c0cfdc0 con 0x7f868c0b20a0 2024-04-25T10:01:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f86942cb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868c0d0660 con 0x7f868c06bda0 2024-04-25T10:01:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f86942cb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868c0d00a0 con 0x7f868c06dca0 2024-04-25T10:01:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f8691866640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f868c06bda0 0x7f868c0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f8691866640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f868c06bda0 0x7f868c0b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46026/0 (socket says 172.21.15.133:46026) 2024-04-25T10:01:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f8691866640 1 -- 172.21.15.133:0/1943528473 learned_addr learned my addr 172.21.15.133:0/1943528473 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f8692067640 1 --2- 172.21.15.133:0/1943528473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868c06dca0 0x7f868c0b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f8691866640 1 -- 172.21.15.133:0/1943528473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868c06dca0 msgr2=0x7f868c0b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f8691866640 1 --2- 172.21.15.133:0/1943528473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868c06dca0 0x7f868c0b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f8691866640 1 -- 172.21.15.133:0/1943528473 >> v1:172.21.15.133:6789/0 conn(0x7f868c0b20a0 legacy=0x7f868c0999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.485+0000 7f8691866640 1 -- 172.21.15.133:0/1943528473 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c0d0380 con 0x7f868c06bda0 2024-04-25T10:01:57.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691866640 1 --2- 172.21.15.133:0/1943528473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c06bda0 0x7f868c0b1b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8680009e50 tx=0x7f868001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260ec4bd0ef23ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691065640 1 -- 172.21.15.133:0/1943528473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86800a55d0 con 0x7f868c06bda0 2024-04-25T10:01:57.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691065640 1 -- 172.21.15.133:0/1943528473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86800a5770 con 0x7f868c06bda0 2024-04-25T10:01:57.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691065640 1 -- 172.21.15.133:0/1943528473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86800a5b40 con 0x7f868c06bda0 2024-04-25T10:01:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 -- 172.21.15.133:0/1943528473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c06bda0 msgr2=0x7f868c0b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 --2- 172.21.15.133:0/1943528473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c06bda0 0x7f868c0b1b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8680009e50 tx=0x7f868001c910 comp rx=0 tx=0).stop 2024-04-25T10:01:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 -- 172.21.15.133:0/1943528473 shutdown_connections 2024-04-25T10:01:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 --2- 172.21.15.133:0/1943528473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c06bda0 0x7f868c0b1b60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 --2- 172.21.15.133:0/1943528473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868c06dca0 0x7f868c0b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 -- 172.21.15.133:0/1943528473 >> 172.21.15.133:0/1943528473 conn(0x7f868c0b31a0 msgr2=0x7f868c0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 -- 172.21.15.133:0/1943528473 shutdown_connections 2024-04-25T10:01:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 -- 172.21.15.133:0/1943528473 wait complete. 2024-04-25T10:01:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 Processor -- start 2024-04-25T10:01:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 -- start start 2024-04-25T10:01:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f868c06bda0 0x7f868c159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c06dca0 0x7f868c159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868c0b20a0 0x7f868c0c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c16a9b0 con 0x7f868c06dca0 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c0cd670 con 0x7f868c06bda0 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f86942cb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868c16ab30 con 0x7f868c0b20a0 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691866640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c06dca0 0x7f868c159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691866640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c06dca0 0x7f868c159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46032/0 (socket says 172.21.15.133:46032) 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691866640 1 -- 172.21.15.133:0/3130840951 learned_addr learned my addr 172.21.15.133:0/3130840951 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691866640 1 -- 172.21.15.133:0/3130840951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868c0b20a0 msgr2=0x7f868c0c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691866640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868c0b20a0 0x7f868c0c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691866640 1 -- 172.21.15.133:0/3130840951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f868c06bda0 msgr2=0x7f868c159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691866640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f868c06bda0 0x7f868c159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691866640 1 -- 172.21.15.133:0/3130840951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c0b4e20 con 0x7f868c06dca0 2024-04-25T10:01:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.489+0000 7f8691866640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c06dca0 0x7f868c159170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86800af040 tx=0x7f86800196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.493+0000 7f868affd640 1 -- 172.21.15.133:0/3130840951 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86800bc020 con 0x7f868c06dca0 2024-04-25T10:01:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.493+0000 7f868affd640 1 -- 172.21.15.133:0/3130840951 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86800b8690 con 0x7f868c06dca0 2024-04-25T10:01:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.493+0000 7f86942cb640 1 -- 172.21.15.133:0/3130840951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c0b50c0 con 0x7f868c06dca0 2024-04-25T10:01:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.493+0000 7f868affd640 1 -- 172.21.15.133:0/3130840951 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680009510 con 0x7f868c06dca0 2024-04-25T10:01:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.493+0000 7f86942cb640 1 -- 172.21.15.133:0/3130840951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c0b5680 con 0x7f868c06dca0 2024-04-25T10:01:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.493+0000 7f86942cb640 1 -- 172.21.15.133:0/3130840951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868c067c70 con 0x7f868c06dca0 2024-04-25T10:01:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.493+0000 7f868affd640 1 -- 172.21.15.133:0/3130840951 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86800bf020 con 0x7f868c06dca0 2024-04-25T10:01:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.493+0000 7f868affd640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8670066f90 0x7f8670069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.493+0000 7f868affd640 1 -- 172.21.15.133:0/3130840951 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8680120190 con 0x7f868c06dca0 2024-04-25T10:01:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.493+0000 7f8692067640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8670066f90 0x7f8670069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.493+0000 7f8692067640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8670066f90 0x7f8670069450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f867800b850 tx=0x7f8678008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.497+0000 7f868affd640 1 -- 172.21.15.133:0/3130840951 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8680016620 con 0x7f868c06dca0 2024-04-25T10:01:57.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.661+0000 7f86942cb640 1 -- 172.21.15.133:0/3130840951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f868c15a900 con 0x7f868c06dca0 2024-04-25T10:01:57.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.661+0000 7f868affd640 1 -- 172.21.15.133:0/3130840951 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7f86800f0e60 con 0x7f868c06dca0 2024-04-25T10:01:57.669 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:01:57.669 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:57.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 -- 172.21.15.133:0/3130840951 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8670066f90 msgr2=0x7f8670069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8670066f90 0x7f8670069450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f867800b850 tx=0x7f8678008040 comp rx=0 tx=0).stop 2024-04-25T10:01:57.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 -- 172.21.15.133:0/3130840951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c06dca0 msgr2=0x7f868c159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c06dca0 0x7f868c159170 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f86800af040 tx=0x7f86800196b0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 -- 172.21.15.133:0/3130840951 shutdown_connections 2024-04-25T10:01:57.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868c0b20a0 0x7f868c0c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8670066f90 0x7f8670069450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c06dca0 0x7f868c159170 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 --2- 172.21.15.133:0/3130840951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f868c06bda0 0x7f868c159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 -- 172.21.15.133:0/3130840951 >> 172.21.15.133:0/3130840951 conn(0x7f868c0b31a0 msgr2=0x7f868c0cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 -- 172.21.15.133:0/3130840951 shutdown_connections 2024-04-25T10:01:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.665+0000 7f86942cb640 1 -- 172.21.15.133:0/3130840951 wait complete. 2024-04-25T10:01:57.688 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:57.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f111447d640 1 Processor -- start 2024-04-25T10:01:57.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f111447d640 1 -- start start 2024-04-25T10:01:57.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f111447d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c0b1f70 0x7f110c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f111447d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f110c0ce330 0x7f110c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f111447d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110c0c61d0 con 0x7f110c15c9a0 2024-04-25T10:01:57.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f111447d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110c0cece0 con 0x7f110c0ce330 2024-04-25T10:01:57.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f111447d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0c64b0 con 0x7f110c0b1f70 2024-04-25T10:01:57.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1112219640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f110c15c9a0 0x7f110c0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38296/0 (socket says 172.21.15.133:38296) 2024-04-25T10:01:57.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1112219640 1 -- 172.21.15.133:0/9916067 learned_addr learned my addr 172.21.15.133:0/9916067 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:01:57.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111a18640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c0b1f70 0x7f110c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1112a1a640 1 --2- 172.21.15.133:0/9916067 >> v2:172.21.15.162:3300/0 conn(0x7f110c0ce330 0x7f110c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177029451 0 0) 0x7f110c0c61d0 con 0x7f110c15c9a0 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10f8003530 con 0x7f110c15c9a0 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3589309798 0 0) 0x7f10f8003530 con 0x7f110c15c9a0 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c0b1f70 msgr2=0x7f110c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c0b1f70 0x7f110c06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 >> v2:172.21.15.162:3300/0 conn(0x7f110c0ce330 msgr2=0x7f110c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 --2- 172.21.15.133:0/9916067 >> v2:172.21.15.162:3300/0 conn(0x7f110c0ce330 0x7f110c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c06d4f0 con 0x7f110c15c9a0 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f10fc001680 con 0x7f110c15c9a0 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 >> v1:172.21.15.133:6789/0 conn(0x7f110c15c9a0 legacy=0x7f110c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f80039c0 0x7f10f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c0b1f70 0x7f10f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10f8004d50 0x7f10f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0cece0 con 0x7f10f8004d50 2024-04-25T10:01:57.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0c64b0 con 0x7f10f80039c0 2024-04-25T10:01:57.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f800b9d0 con 0x7f110c0b1f70 2024-04-25T10:01:57.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1112a1a640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10f8004d50 0x7f10f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1111a18640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c0b1f70 0x7f10f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1112219640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f80039c0 0x7f10f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1112a1a640 1 -- 172.21.15.133:0/9916067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c0b1f70 msgr2=0x7f10f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1112a1a640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c0b1f70 0x7f10f8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1112a1a640 1 -- 172.21.15.133:0/9916067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f80039c0 msgr2=0x7f10f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1112a1a640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10f80039c0 0x7f10f8003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.845+0000 7f1112a1a640 1 -- 172.21.15.133:0/9916067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f8003530 con 0x7f10f8004d50 2024-04-25T10:01:57.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f1112a1a640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10f8004d50 0x7f10f80051d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1104009fc0 tx=0x7f110401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c1953ad954b936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11040b2070 con 0x7f10f8004d50 2024-04-25T10:01:57.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11040a5cd0 con 0x7f10f8004d50 2024-04-25T10:01:57.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f1111217640 1 -- 172.21.15.133:0/9916067 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104009c10 con 0x7f10f8004d50 2024-04-25T10:01:57.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 -- 172.21.15.133:0/9916067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10f8004d50 msgr2=0x7f10f80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 --2- 172.21.15.133:0/9916067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10f8004d50 0x7f10f80051d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1104009fc0 tx=0x7f110401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f1112a1a640 1 -- 172.21.15.133:0/9916067 reap_dead start 2024-04-25T10:01:57.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 -- 172.21.15.133:0/9916067 shutdown_connections 2024-04-25T10:01:57.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 -- 172.21.15.133:0/9916067 >> 172.21.15.133:0/9916067 conn(0x7f110c067000 msgr2=0x7f110c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 -- 172.21.15.133:0/9916067 shutdown_connections 2024-04-25T10:01:57.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 -- 172.21.15.133:0/9916067 wait complete. 2024-04-25T10:01:57.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 Processor -- start 2024-04-25T10:01:57.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 -- start start 2024-04-25T10:01:57.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f110c0cc380 0x7f110c14dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c14e120 0x7f110c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110c0ccca0 0x7f110c0cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c15cf00 con 0x7f110c0cc380 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c15cd80 con 0x7f110c0ccca0 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f111447d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c06dcf0 con 0x7f110c14e120 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.849+0000 7f1112219640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f110c0cc380 0x7f110c14dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1112219640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f110c0cc380 0x7f110c14dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46040/0 (socket says 172.21.15.133:46040) 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1112219640 1 -- 172.21.15.133:0/906169446 learned_addr learned my addr 172.21.15.133:0/906169446 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1112a1a640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110c0ccca0 0x7f110c0cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1111a18640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c14e120 0x7f110c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1112219640 1 -- 172.21.15.133:0/906169446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c14e120 msgr2=0x7f110c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1112219640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c14e120 0x7f110c0cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1112219640 1 -- 172.21.15.133:0/906169446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110c0ccca0 msgr2=0x7f110c0cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1112219640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110c0ccca0 0x7f110c0cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1112219640 1 -- 172.21.15.133:0/906169446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c0cd820 con 0x7f110c0cc380 2024-04-25T10:01:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1112a1a640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110c0ccca0 0x7f110c0cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1112219640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f110c0cc380 0x7f110c14dbe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f10fc000f70 tx=0x7f10fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1102ffd640 1 -- 172.21.15.133:0/906169446 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc019070 con 0x7f110c0cc380 2024-04-25T10:01:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1102ffd640 1 -- 172.21.15.133:0/906169446 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc0074e0 con 0x7f110c0cc380 2024-04-25T10:01:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f111447d640 1 -- 172.21.15.133:0/906169446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c0c8980 con 0x7f110c0cc380 2024-04-25T10:01:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1102ffd640 1 -- 172.21.15.133:0/906169446 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc004a10 con 0x7f110c0cc380 2024-04-25T10:01:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f111447d640 1 -- 172.21.15.133:0/906169446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c14fc10 con 0x7f110c0cc380 2024-04-25T10:01:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f111447d640 1 -- 172.21.15.133:0/906169446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110c068ef0 con 0x7f110c0cc380 2024-04-25T10:01:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1102ffd640 1 -- 172.21.15.133:0/906169446 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10fc01c070 con 0x7f110c0cc380 2024-04-25T10:01:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1102ffd640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f0066f60 0x7f10f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1111a18640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f0066f60 0x7f10f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1102ffd640 1 -- 172.21.15.133:0/906169446 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f10fc084dd0 con 0x7f110c0cc380 2024-04-25T10:01:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.853+0000 7f1111a18640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f0066f60 0x7f10f0069420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f110c0b12b0 tx=0x7f1108008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:57.857+0000 7f1102ffd640 1 -- 172.21.15.133:0/906169446 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10fc055b30 con 0x7f110c0cc380 2024-04-25T10:01:58.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.017+0000 7f111447d640 1 -- 172.21.15.133:0/906169446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f110c14e500 con 0x7f110c0cc380 2024-04-25T10:01:58.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.021+0000 7f1102ffd640 1 -- 172.21.15.133:0/906169446 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7f10fc086090 con 0x7f110c0cc380 2024-04-25T10:01:58.027 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:01:58.027 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 -- 172.21.15.133:0/906169446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f0066f60 msgr2=0x7f10f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f0066f60 0x7f10f0069420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f110c0b12b0 tx=0x7f1108008040 comp rx=0 tx=0).stop 2024-04-25T10:01:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 -- 172.21.15.133:0/906169446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f110c0cc380 msgr2=0x7f110c14dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f110c0cc380 0x7f110c14dbe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f10fc000f70 tx=0x7f10fc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 -- 172.21.15.133:0/906169446 shutdown_connections 2024-04-25T10:01:58.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110c0ccca0 0x7f110c0cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f110c14e120 0x7f110c0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f0066f60 0x7f10f0069420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 --2- 172.21.15.133:0/906169446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f110c0cc380 0x7f110c14dbe0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 -- 172.21.15.133:0/906169446 >> 172.21.15.133:0/906169446 conn(0x7f110c067000 msgr2=0x7f110c15ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 -- 172.21.15.133:0/906169446 shutdown_connections 2024-04-25T10:01:58.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.025+0000 7f111447d640 1 -- 172.21.15.133:0/906169446 wait complete. 2024-04-25T10:01:58.059 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:58.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe994067640 1 Processor -- start 2024-04-25T10:01:58.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe994067640 1 -- start start 2024-04-25T10:01:58.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe994067640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe98c15dd90 0x7fe98c0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe994067640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe98c0b19f0 0x7fe98c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe994067640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c0ca9b0 con 0x7fe98c15d9b0 2024-04-25T10:01:58.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe994067640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98c0cbe30 con 0x7fe98c0b19f0 2024-04-25T10:01:58.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe994067640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98c0cac90 con 0x7fe98c15dd90 2024-04-25T10:01:58.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe992604640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe98c0b19f0 0x7fe98c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe992604640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe98c0b19f0 0x7fe98c0cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46052/0 (socket says 172.21.15.133:46052) 2024-04-25T10:01:58.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe992604640 1 -- 172.21.15.133:0/1343832687 learned_addr learned my addr 172.21.15.133:0/1343832687 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:58.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe992604640 1 -- 172.21.15.133:0/1343832687 >> v2:172.21.15.162:3300/0 conn(0x7fe98c15dd90 msgr2=0x7fe98c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:58.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe992604640 1 --2- 172.21.15.133:0/1343832687 >> v2:172.21.15.162:3300/0 conn(0x7fe98c15dd90 0x7fe98c0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe992604640 1 -- 172.21.15.133:0/1343832687 >> v1:172.21.15.133:6789/0 conn(0x7fe98c15d9b0 legacy=0x7fe98c0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.161+0000 7fe992604640 1 -- 172.21.15.133:0/1343832687 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c0caf70 con 0x7fe98c0b19f0 2024-04-25T10:01:58.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe992604640 1 --2- 172.21.15.133:0/1343832687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe98c0b19f0 0x7fe98c0cecb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0cc000 tx=0x7fe98401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70306a5034676672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe990e01640 1 -- 172.21.15.133:0/1343832687 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9840a5840 con 0x7fe98c0b19f0 2024-04-25T10:01:58.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe990e01640 1 -- 172.21.15.133:0/1343832687 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9840a59e0 con 0x7fe98c0b19f0 2024-04-25T10:01:58.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe990e01640 1 -- 172.21.15.133:0/1343832687 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9840a5db0 con 0x7fe98c0b19f0 2024-04-25T10:01:58.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 -- 172.21.15.133:0/1343832687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe98c0b19f0 msgr2=0x7fe98c0cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 --2- 172.21.15.133:0/1343832687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe98c0b19f0 0x7fe98c0cecb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0cc000 tx=0x7fe98401c960 comp rx=0 tx=0).stop 2024-04-25T10:01:58.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 -- 172.21.15.133:0/1343832687 shutdown_connections 2024-04-25T10:01:58.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 --2- 172.21.15.133:0/1343832687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe98c0b19f0 0x7fe98c0cecb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 --2- 172.21.15.133:0/1343832687 >> v2:172.21.15.162:3300/0 conn(0x7fe98c15dd90 0x7fe98c0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 -- 172.21.15.133:0/1343832687 >> 172.21.15.133:0/1343832687 conn(0x7fe98c067000 msgr2=0x7fe98c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 -- 172.21.15.133:0/1343832687 shutdown_connections 2024-04-25T10:01:58.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 -- 172.21.15.133:0/1343832687 wait complete. 2024-04-25T10:01:58.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 Processor -- start 2024-04-25T10:01:58.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 -- start start 2024-04-25T10:01:58.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe98c0b19f0 0x7fe98c144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe98c15d9b0 0x7fe98c1546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe98c15dd90 0x7fe98c14b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c06e270 con 0x7fe98c15dd90 2024-04-25T10:01:58.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c06e0f0 con 0x7fe98c15d9b0 2024-04-25T10:01:58.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe994067640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c06e3f0 con 0x7fe98c0b19f0 2024-04-25T10:01:58.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe992604640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe98c15dd90 0x7fe98c14b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe992604640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe98c15dd90 0x7fe98c14b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46060/0 (socket says 172.21.15.133:46060) 2024-04-25T10:01:58.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe992604640 1 -- 172.21.15.133:0/1939650848 learned_addr learned my addr 172.21.15.133:0/1939650848 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:58.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe992604640 1 -- 172.21.15.133:0/1939650848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe98c0b19f0 msgr2=0x7fe98c144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe992604640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe98c0b19f0 0x7fe98c144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe992604640 1 -- 172.21.15.133:0/1939650848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe98c15d9b0 msgr2=0x7fe98c1546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe992604640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe98c15d9b0 0x7fe98c1546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe992604640 1 -- 172.21.15.133:0/1939650848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c1550f0 con 0x7fe98c15dd90 2024-04-25T10:01:58.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.165+0000 7fe992604640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe98c15dd90 0x7fe98c14b730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9840af040 tx=0x7fe984019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.169+0000 7fe9827fc640 1 -- 172.21.15.133:0/1939650848 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9840bc020 con 0x7fe98c15dd90 2024-04-25T10:01:58.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.169+0000 7fe9827fc640 1 -- 172.21.15.133:0/1939650848 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe984007bb0 con 0x7fe98c15dd90 2024-04-25T10:01:58.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.169+0000 7fe994067640 1 -- 172.21.15.133:0/1939650848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c1553d0 con 0x7fe98c15dd90 2024-04-25T10:01:58.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.169+0000 7fe9827fc640 1 -- 172.21.15.133:0/1939650848 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9840b8470 con 0x7fe98c15dd90 2024-04-25T10:01:58.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.169+0000 7fe994067640 1 -- 172.21.15.133:0/1939650848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c155990 con 0x7fe98c15dd90 2024-04-25T10:01:58.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.169+0000 7fe994067640 1 -- 172.21.15.133:0/1939650848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c068ef0 con 0x7fe98c15dd90 2024-04-25T10:01:58.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.169+0000 7fe9827fc640 1 -- 172.21.15.133:0/1939650848 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9840bf020 con 0x7fe98c15dd90 2024-04-25T10:01:58.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.169+0000 7fe9827fc640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe970066f60 0x7fe970069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.169+0000 7fe9827fc640 1 -- 172.21.15.133:0/1939650848 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe984120a50 con 0x7fe98c15dd90 2024-04-25T10:01:58.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.169+0000 7fe991e03640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe970066f60 0x7fe970069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.169+0000 7fe991e03640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe970066f60 0x7fe970069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe988000f70 tx=0x7fe9880079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.173+0000 7fe9827fc640 1 -- 172.21.15.133:0/1939650848 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9840c6030 con 0x7fe98c15dd90 2024-04-25T10:01:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.353+0000 7fe994067640 1 -- 172.21.15.133:0/1939650848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe98c0cc2f0 con 0x7fe98c15dd90 2024-04-25T10:01:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.353+0000 7fe9827fc640 1 -- 172.21.15.133:0/1939650848 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4943 (secure 0 0 0) 0x7fe9840165e0 con 0x7fe98c15dd90 2024-04-25T10:01:58.361 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:01:58.361 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2230831968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2230831968},{"type":"v1","addr":"172.21.15.162:6835","nonce":2230831968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1456396097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1456396097},{"type":"v1","addr":"172.21.15.133:6836","nonce":1456396097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:51.276631+0000","modified":"2024-04-25T10:01:56.349405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2462154167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2462154167},{"type":"v1","addr":"172.21.15.133:6837","nonce":2462154167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T10:01:58.361 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-04-25T10:01:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 -- 172.21.15.133:0/1939650848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe970066f60 msgr2=0x7fe970069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe970066f60 0x7fe970069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe988000f70 tx=0x7fe9880079e0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 -- 172.21.15.133:0/1939650848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe98c15dd90 msgr2=0x7fe98c14b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe98c15dd90 0x7fe98c14b730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9840af040 tx=0x7fe984019700 comp rx=0 tx=0).stop 2024-04-25T10:01:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 -- 172.21.15.133:0/1939650848 shutdown_connections 2024-04-25T10:01:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe970066f60 0x7fe970069420 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe98c15dd90 0x7fe98c14b730 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe98c15d9b0 0x7fe98c1546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 --2- 172.21.15.133:0/1939650848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe98c0b19f0 0x7fe98c144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 -- 172.21.15.133:0/1939650848 >> 172.21.15.133:0/1939650848 conn(0x7fe98c067000 msgr2=0x7fe98c0b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 -- 172.21.15.133:0/1939650848 shutdown_connections 2024-04-25T10:01:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.357+0000 7fe994067640 1 -- 172.21.15.133:0/1939650848 wait complete. 2024-04-25T10:01:58.383 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:58.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f03a0865640 1 Processor -- start 2024-04-25T10:01:58.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f03a0865640 1 -- start start 2024-04-25T10:01:58.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f03a0865640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f039c15dbf0 0x7f039c0afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f03a0865640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039c06ac90 0x7f039c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f03a0865640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c0b3e40 con 0x7f039c0d0f80 2024-04-25T10:01:58.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f03a0865640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039c0b3880 con 0x7f039c15dbf0 2024-04-25T10:01:58.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f03a0865640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c0b3b60 con 0x7f039c06ac90 2024-04-25T10:01:58.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f039ad9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039c06ac90 0x7f039c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f039a59c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f039c15dbf0 0x7f039c0afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f039a59c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f039c15dbf0 0x7f039c0afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:54408/0 (socket says 172.21.15.133:54408) 2024-04-25T10:01:58.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f039a59c640 1 -- 172.21.15.133:0/218077771 learned_addr learned my addr 172.21.15.133:0/218077771 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:58.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f039a59c640 1 -- 172.21.15.133:0/218077771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039c06ac90 msgr2=0x7f039c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:58.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f039a59c640 1 --2- 172.21.15.133:0/218077771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039c06ac90 0x7f039c06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f039a59c640 1 -- 172.21.15.133:0/218077771 >> v1:172.21.15.162:6789/0 conn(0x7f039c0d0f80 legacy=0x7f039c06ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.485+0000 7f039a59c640 1 -- 172.21.15.133:0/218077771 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c06e540 con 0x7f039c15dbf0 2024-04-25T10:01:58.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f039a59c640 1 --2- 172.21.15.133:0/218077771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f039c15dbf0 0x7f039c0afc00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f038c009870 tx=0x7f038c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fae10b282dc6c6af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f039959a640 1 -- 172.21.15.133:0/218077771 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0b0070 con 0x7f039c15dbf0 2024-04-25T10:01:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f039959a640 1 -- 172.21.15.133:0/218077771 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c0ab600 con 0x7f039c15dbf0 2024-04-25T10:01:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f039959a640 1 -- 172.21.15.133:0/218077771 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0ab9d0 con 0x7f039c15dbf0 2024-04-25T10:01:58.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f03a0865640 1 -- 172.21.15.133:0/218077771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f039c15dbf0 msgr2=0x7f039c0afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f03a0865640 1 --2- 172.21.15.133:0/218077771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f039c15dbf0 0x7f039c0afc00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f038c009870 tx=0x7f038c01c4d0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f03a0865640 1 -- 172.21.15.133:0/218077771 shutdown_connections 2024-04-25T10:01:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f03a0865640 1 --2- 172.21.15.133:0/218077771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039c06ac90 0x7f039c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f03a0865640 1 --2- 172.21.15.133:0/218077771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f039c15dbf0 0x7f039c0afc00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f03a0865640 1 -- 172.21.15.133:0/218077771 >> 172.21.15.133:0/218077771 conn(0x7f039c072430 msgr2=0x7f039c072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f03a0865640 1 -- 172.21.15.133:0/218077771 shutdown_connections 2024-04-25T10:01:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f03a0865640 1 -- 172.21.15.133:0/218077771 wait complete. 2024-04-25T10:01:58.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f03a0865640 1 Processor -- start 2024-04-25T10:01:58.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.489+0000 7f03a0865640 1 -- start start 2024-04-25T10:01:58.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f03a0865640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f039c06ac90 0x7f039c15e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f03a0865640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039c0d0f80 0x7f039c158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f03a0865640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f039c15dbf0 0x7f039c14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f03a0865640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c16aa40 con 0x7f039c06ac90 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f03a0865640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c162140 con 0x7f039c15dbf0 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f03a0865640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c16abc0 con 0x7f039c0d0f80 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f039a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f039c06ac90 0x7f039c15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f039a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f039c06ac90 0x7f039c15e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46066/0 (socket says 172.21.15.133:46066) 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f039a59c640 1 -- 172.21.15.133:0/3429876038 learned_addr learned my addr 172.21.15.133:0/3429876038 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f039ad9d640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f039c15dbf0 0x7f039c14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f0399d9b640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039c0d0f80 0x7f039c158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f039a59c640 1 -- 172.21.15.133:0/3429876038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039c0d0f80 msgr2=0x7f039c158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f039a59c640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039c0d0f80 0x7f039c158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f039a59c640 1 -- 172.21.15.133:0/3429876038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f039c15dbf0 msgr2=0x7f039c14c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f039a59c640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f039c15dbf0 0x7f039c14c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f039a59c640 1 -- 172.21.15.133:0/3429876038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c15b1e0 con 0x7f039c06ac90 2024-04-25T10:01:58.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f039a59c640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f039c06ac90 0x7f039c15e350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f038c0ae040 tx=0x7f038c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f038affd640 1 -- 172.21.15.133:0/3429876038 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0b0040 con 0x7f039c06ac90 2024-04-25T10:01:58.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f038affd640 1 -- 172.21.15.133:0/3429876038 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c0ab400 con 0x7f039c06ac90 2024-04-25T10:01:58.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f03a0865640 1 -- 172.21.15.133:0/3429876038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c15b4c0 con 0x7f039c06ac90 2024-04-25T10:01:58.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f038affd640 1 -- 172.21.15.133:0/3429876038 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0aa970 con 0x7f039c06ac90 2024-04-25T10:01:58.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f03a0865640 1 -- 172.21.15.133:0/3429876038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c15ba80 con 0x7f039c06ac90 2024-04-25T10:01:58.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f03a0865640 1 -- 172.21.15.133:0/3429876038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c067c70 con 0x7f039c06ac90 2024-04-25T10:01:58.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f038affd640 1 -- 172.21.15.133:0/3429876038 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f038c0bc020 con 0x7f039c06ac90 2024-04-25T10:01:58.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f038affd640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f037c066f60 0x7f037c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f0399d9b640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f037c066f60 0x7f037c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f038affd640 1 -- 172.21.15.133:0/3429876038 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f038c11eb00 con 0x7f039c06ac90 2024-04-25T10:01:58.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.493+0000 7f0399d9b640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f037c066f60 0x7f037c069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0390007fd0 tx=0x7f0390009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.497+0000 7f038affd640 1 -- 172.21.15.133:0/3429876038 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f038c0c3020 con 0x7f039c06ac90 2024-04-25T10:01:58.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.645+0000 7f03a0865640 1 -- 172.21.15.133:0/3429876038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f039c0d0d60 con 0x7f039c06ac90 2024-04-25T10:01:58.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.645+0000 7f038affd640 1 -- 172.21.15.133:0/3429876038 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7f038c016150 con 0x7f039c06ac90 2024-04-25T10:01:58.653 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:01:58.653 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:58.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 -- 172.21.15.133:0/3429876038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f037c066f60 msgr2=0x7f037c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f037c066f60 0x7f037c069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0390007fd0 tx=0x7f0390009040 comp rx=0 tx=0).stop 2024-04-25T10:01:58.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 -- 172.21.15.133:0/3429876038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f039c06ac90 msgr2=0x7f039c15e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f039c06ac90 0x7f039c15e350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f038c0ae040 tx=0x7f038c019270 comp rx=0 tx=0).stop 2024-04-25T10:01:58.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 -- 172.21.15.133:0/3429876038 shutdown_connections 2024-04-25T10:01:58.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f039c15dbf0 0x7f039c14c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f039c0d0f80 0x7f039c158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f037c066f60 0x7f037c069420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 --2- 172.21.15.133:0/3429876038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f039c06ac90 0x7f039c15e350 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 -- 172.21.15.133:0/3429876038 >> 172.21.15.133:0/3429876038 conn(0x7f039c072430 msgr2=0x7f039c0cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 -- 172.21.15.133:0/3429876038 shutdown_connections 2024-04-25T10:01:58.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:58.649+0000 7f03a0865640 1 -- 172.21.15.133:0/3429876038 wait complete. 2024-04-25T10:01:59.073 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:01:51.276631+0000', 'modified': '2024-04-25T10:01:56.349405+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/2462154167', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 2462154167}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 2462154167}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T10:01:59.073 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:01:59.073 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:01:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb859b640 1 Processor -- start 2024-04-25T10:01:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb859b640 1 -- start start 2024-04-25T10:01:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb859b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efcb015db30 0x7efcb00b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb859b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb01618b0 con 0x7efcb00ce090 2024-04-25T10:01:59.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb859b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb0161b90 con 0x7efcb00d0860 2024-04-25T10:01:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb859b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb00b2a00 con 0x7efcb015db30 2024-04-25T10:01:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5b36640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7efcb00ce090 0x7efcb00d0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38312/0 (socket says 172.21.15.133:38312) 2024-04-25T10:01:59.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5b36640 1 -- 172.21.15.133:0/771711338 learned_addr learned my addr 172.21.15.133:0/771711338 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140054471 0 0) 0x7efcb01618b0 con 0x7efcb00ce090 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc9c003530 con 0x7efcb00ce090 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb6337640 1 --2- 172.21.15.133:0/771711338 >> v2:172.21.15.162:3300/0 conn(0x7efcb015db30 0x7efcb00b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1820431702 0 0) 0x7efc9c003530 con 0x7efcb00ce090 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 >> v2:172.21.15.162:3300/0 conn(0x7efcb015db30 msgr2=0x7efcb00b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 --2- 172.21.15.133:0/771711338 >> v2:172.21.15.162:3300/0 conn(0x7efcb015db30 0x7efcb00b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 >> v1:172.21.15.162:6789/0 conn(0x7efcb00d0860 legacy=0x7efcb00d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb0161e70 con 0x7efcb00ce090 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7efcac0016d0 con 0x7efcb00ce090 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 >> v1:172.21.15.133:6789/0 conn(0x7efcb00ce090 legacy=0x7efcb00d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 --2- 172.21.15.133:0/771711338 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc9c0039c0 0x7efc9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 --2- 172.21.15.133:0/771711338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc9c004810 0x7efc9c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 --2- 172.21.15.133:0/771711338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc9c0051d0 0x7efc9c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c00c040 con 0x7efc9c0051d0 2024-04-25T10:01:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c00c1f0 con 0x7efc9c004810 2024-04-25T10:01:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c00c3f0 con 0x7efc9c0039c0 2024-04-25T10:01:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb6b38640 1 --2- 172.21.15.133:0/771711338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc9c0051d0 0x7efc9c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb5b36640 1 --2- 172.21.15.133:0/771711338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc9c004810 0x7efc9c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb6337640 1 --2- 172.21.15.133:0/771711338 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc9c0039c0 0x7efc9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb6b38640 1 -- 172.21.15.133:0/771711338 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc9c0039c0 msgr2=0x7efc9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb6b38640 1 --2- 172.21.15.133:0/771711338 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc9c0039c0 0x7efc9c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb6b38640 1 -- 172.21.15.133:0/771711338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc9c004810 msgr2=0x7efc9c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb6b38640 1 --2- 172.21.15.133:0/771711338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc9c004810 0x7efc9c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.200+0000 7efcb6b38640 1 -- 172.21.15.133:0/771711338 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9c003530 con 0x7efc9c0051d0 2024-04-25T10:01:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb6b38640 1 -- 172.21.15.133:0/771711338 reap_dead start 2024-04-25T10:01:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb6b38640 1 --2- 172.21.15.133:0/771711338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc9c0051d0 0x7efc9c00b9d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efca8001e40 tx=0x7efca801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ce419fa1375891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8001a90 con 0x7efc9c0051d0 2024-04-25T10:01:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca8003b80 con 0x7efc9c0051d0 2024-04-25T10:01:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb5335640 1 -- 172.21.15.133:0/771711338 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8005020 con 0x7efc9c0051d0 2024-04-25T10:01:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb859b640 1 -- 172.21.15.133:0/771711338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc9c0051d0 msgr2=0x7efc9c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb859b640 1 --2- 172.21.15.133:0/771711338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc9c0051d0 0x7efc9c00b9d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efca8001e40 tx=0x7efca801c760 comp rx=0 tx=0).stop 2024-04-25T10:01:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb859b640 1 -- 172.21.15.133:0/771711338 shutdown_connections 2024-04-25T10:01:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb859b640 1 --2- 172.21.15.133:0/771711338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc9c0051d0 0x7efc9c00b9d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb859b640 1 -- 172.21.15.133:0/771711338 >> 172.21.15.133:0/771711338 conn(0x7efcb006bfc0 msgr2=0x7efcb006c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb859b640 1 -- 172.21.15.133:0/771711338 shutdown_connections 2024-04-25T10:01:59.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb859b640 1 -- 172.21.15.133:0/771711338 wait complete. 2024-04-25T10:01:59.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.204+0000 7efcb859b640 1 Processor -- start 2024-04-25T10:01:59.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb859b640 1 -- start start 2024-04-25T10:01:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb859b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc9c0051d0 0x7efcb00b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb859b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb0159f60 0x7efcb00b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb859b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb00cc740 0x7efcb00ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb859b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb00ce560 con 0x7efcb0159f60 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb859b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb015df10 con 0x7efc9c0051d0 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb859b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb015e090 con 0x7efcb00cc740 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb5b36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb0159f60 0x7efcb00b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb5b36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb0159f60 0x7efcb00b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46080/0 (socket says 172.21.15.133:46080) 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb5b36640 1 -- 172.21.15.133:0/1411794959 learned_addr learned my addr 172.21.15.133:0/1411794959 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb6b38640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb00cc740 0x7efcb00ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb6337640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc9c0051d0 0x7efcb00b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb5b36640 1 -- 172.21.15.133:0/1411794959 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb00cc740 msgr2=0x7efcb00ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb5b36640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb00cc740 0x7efcb00ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb5b36640 1 -- 172.21.15.133:0/1411794959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc9c0051d0 msgr2=0x7efcb00b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb5b36640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc9c0051d0 0x7efcb00b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb5b36640 1 -- 172.21.15.133:0/1411794959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb00aa9d0 con 0x7efcb0159f60 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb5b36640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb0159f60 0x7efcb00b3f80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efcac00e7e0 tx=0x7efcac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efca6ffd640 1 -- 172.21.15.133:0/1411794959 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac00c6c0 con 0x7efcb0159f60 2024-04-25T10:01:59.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efca6ffd640 1 -- 172.21.15.133:0/1411794959 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac018070 con 0x7efcb0159f60 2024-04-25T10:01:59.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efca6ffd640 1 -- 172.21.15.133:0/1411794959 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac00caa0 con 0x7efcb0159f60 2024-04-25T10:01:59.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb859b640 1 -- 172.21.15.133:0/1411794959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb00b39d0 con 0x7efcb0159f60 2024-04-25T10:01:59.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efcb859b640 1 -- 172.21.15.133:0/1411794959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb00aa6f0 con 0x7efcb0159f60 2024-04-25T10:01:59.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.208+0000 7efca6ffd640 1 -- 172.21.15.133:0/1411794959 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efcac00cc40 con 0x7efcb0159f60 2024-04-25T10:01:59.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efca6ffd640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc94066f60 0x7efc94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efca6ffd640 1 -- 172.21.15.133:0/1411794959 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efcac0851e0 con 0x7efcb0159f60 2024-04-25T10:01:59.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcb6337640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc94066f60 0x7efc94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcb6337640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc94066f60 0x7efc94069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efca00099b0 tx=0x7efca0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 Processor -- start 2024-04-25T10:01:59.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 -- start start 2024-04-25T10:01:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 0x557a58bacd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bad240 0x557a58b629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58b63310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a58a4a5f0 con 0x557a58b62f30 2024-04-25T10:01:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a589b1da0 con 0x557a58bad240 2024-04-25T10:01:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a5893a150 con 0x557a58bac920 2024-04-25T10:01:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcb6b38640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58b63310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcb6b38640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58b63310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46088/0 (socket says 172.21.15.133:46088) 2024-04-25T10:01:59.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcb6b38640 1 -- 172.21.15.133:0/3225877197 learned_addr learned my addr 172.21.15.133:0/3225877197 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcb6b38640 1 -- 172.21.15.133:0/3225877197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 msgr2=0x557a58bacd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcb6b38640 1 --2- 172.21.15.133:0/3225877197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 0x557a58bacd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcb6b38640 1 -- 172.21.15.133:0/3225877197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bad240 msgr2=0x557a58b629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcb6b38640 1 --2- 172.21.15.133:0/3225877197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bad240 0x557a58b629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcb6b38640 1 -- 172.21.15.133:0/3225877197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a58953ca0 con 0x557a58b62f30 2024-04-25T10:01:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcb6b38640 1 --2- 172.21.15.133:0/3225877197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58b63310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efca801d000 tx=0x7efca80014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56196e8d66edbc51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efc93fff640 1 -- 172.21.15.133:0/3225877197 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8004a90 con 0x557a58b62f30 2024-04-25T10:01:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efc93fff640 1 -- 172.21.15.133:0/3225877197 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca80ad040 con 0x557a58b62f30 2024-04-25T10:01:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efc93fff640 1 -- 172.21.15.133:0/3225877197 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8004e30 con 0x557a58b62f30 2024-04-25T10:01:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 -- 172.21.15.133:0/3225877197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 msgr2=0x557a58b63310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 --2- 172.21.15.133:0/3225877197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58b63310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efca801d000 tx=0x7efca80014b0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 -- 172.21.15.133:0/3225877197 shutdown_connections 2024-04-25T10:01:59.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 --2- 172.21.15.133:0/3225877197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58b63310 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 --2- 172.21.15.133:0/3225877197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bad240 0x557a58b629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 --2- 172.21.15.133:0/3225877197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 0x557a58bacd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 -- 172.21.15.133:0/3225877197 >> 172.21.15.133:0/3225877197 conn(0x557a58b692a0 msgr2=0x557a58b612d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 -- 172.21.15.133:0/3225877197 shutdown_connections 2024-04-25T10:01:59.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 -- 172.21.15.133:0/3225877197 wait complete. 2024-04-25T10:01:59.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 -- start start 2024-04-25T10:01:59.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.212+0000 7efcbb708000 1 Processor -- start 2024-04-25T10:01:59.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58b62f30 0x557a58bb3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 0x557a58bb4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58bad240 0x557a58b66aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5896c990 con 0x557a58bad240 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a589ade40 con 0x557a58b62f30 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a5893d900 con 0x557a58bac920 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 10 client.0 _ll_register_callbacks cb 0x557a58a651c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6b38640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58bad240 0x557a58b66aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6b38640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58bad240 0x557a58b66aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46092/0 (socket says 172.21.15.133:46092) 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6b38640 1 -- 172.21.15.133:0/38355478 learned_addr learned my addr 172.21.15.133:0/38355478 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6b38640 1 -- 172.21.15.133:0/38355478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 msgr2=0x557a58bb4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6b38640 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 0x557a58bb4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6b38640 1 -- 172.21.15.133:0/38355478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58b62f30 msgr2=0x557a58bb3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6337640 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58b62f30 0x557a58bb3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6b38640 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58b62f30 0x557a58bb3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6b38640 1 -- 172.21.15.133:0/38355478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a5896a630 con 0x557a58bad240 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6b38640 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58bad240 0x557a58b66aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efca801d000 tx=0x7efca801afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883bbfa1f457a21d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efc917fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efc917fa640 1 -- 172.21.15.133:0/38355478 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca80be060 con 0x557a58bad240 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efc917fa640 1 -- 172.21.15.133:0/38355478 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efca8006b60 con 0x557a58bad240 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 -- 172.21.15.133:0/38355478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efca80ad040 con 0x557a58bad240 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efc917fa640 1 -- 172.21.15.133:0/38355478 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7efca80c4050 con 0x557a58bad240 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 10 client.4422 fetch_fsmap learned FSMap version 7 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 -- 172.21.15.133:0/38355478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x557a5893a3a0 con 0x557a58bad240 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efc917fa640 1 -- 172.21.15.133:0/38355478 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca80ad040 con 0x557a58bad240 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efc917fa640 1 -- 172.21.15.133:0/38355478 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7efca80c7020 con 0x557a58bad240 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 10 client.4422 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 10 client.4422 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 20 client.4422 populate_metadata read hostname 'smithi133' 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x557a58b68160 0x557a58bbde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 4 client.4422 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 -- 172.21.15.133:0/38355478 --> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a5894da50 con 0x557a58b68160 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6337640 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x557a58b68160 0x557a58bbde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efc917fa640 10 client.4422 ms_handle_connect on v2:172.21.15.133:6835/2462154167 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcb6337640 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x557a58b68160 0x557a58bbde50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edda979f9cca38b8 server_cookie=bc0065250be1f9ba in_seq=0 out_seq=0 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efc917fa640 1 -- 172.21.15.133:0/38355478 <== mds.0 v2:172.21.15.133:6835/2462154167 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a5894da50 con 0x557a58b68160 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efc917fa640 10 client.4422 handle_command_reply: tid=0 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 client.4422 shutdown 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 -- 172.21.15.133:0/38355478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58bad240 msgr2=0x557a58b66aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58bad240 0x557a58b66aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efca801d000 tx=0x7efca801afa0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 -- 172.21.15.133:0/38355478 shutdown_connections 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x557a58b68160 0x557a58bbde50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58bad240 0x557a58b66aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efc917fa640 0 client.4422 ms_handle_reset on v2:172.21.15.133:6835/2462154167 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 0x557a58bb4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 --2- 172.21.15.133:0/38355478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58b62f30 0x557a58bb3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 -- 172.21.15.133:0/38355478 >> 172.21.15.133:0/38355478 conn(0x557a58b692a0 msgr2=0x557a58b619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 -- 172.21.15.133:0/38355478 shutdown_connections 2024-04-25T10:01:59.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 1 -- 172.21.15.133:0/38355478 wait complete. 2024-04-25T10:01:59.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.216+0000 7efcbb708000 20 client.4422 trim_cache size 0 max 16384 2024-04-25T10:01:59.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcbb708000 1 Processor -- start 2024-04-25T10:01:59.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcbb708000 1 -- start start 2024-04-25T10:01:59.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcbb708000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c106f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcbb708000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 0x557a58c10c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcbb708000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bad240 0x557a58c11170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcbb708000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb0200670 con 0x557a58b62f30 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcbb708000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb01f49a0 con 0x557a58bad240 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcbb708000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb0200070 con 0x557a58bac920 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcb6337640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcb6337640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c106f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46108/0 (socket says 172.21.15.133:46108) 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcb6337640 1 -- 172.21.15.133:0/1050689280 learned_addr learned my addr 172.21.15.133:0/1050689280 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcb6337640 1 -- 172.21.15.133:0/1050689280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 msgr2=0x557a58c10c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcb6337640 1 --2- 172.21.15.133:0/1050689280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 0x557a58c10c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcb6337640 1 -- 172.21.15.133:0/1050689280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bad240 msgr2=0x557a58c11170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcb6337640 1 --2- 172.21.15.133:0/1050689280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bad240 0x557a58c11170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcb6337640 1 -- 172.21.15.133:0/1050689280 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a58bad070 con 0x557a58b62f30 2024-04-25T10:01:59.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efcb6337640 1 --2- 172.21.15.133:0/1050689280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c106f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x557a5895dfa0 tx=0x7efca00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85db88db087ff527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efc937fe640 1 -- 172.21.15.133:0/1050689280 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca001e040 con 0x557a58b62f30 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efc937fe640 1 -- 172.21.15.133:0/1050689280 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca002e070 con 0x557a58b62f30 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.232+0000 7efc937fe640 1 -- 172.21.15.133:0/1050689280 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0016db0 con 0x557a58b62f30 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- 172.21.15.133:0/1050689280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 msgr2=0x557a58c106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- 172.21.15.133:0/1050689280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c106f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x557a5895dfa0 tx=0x7efca00046d0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- 172.21.15.133:0/1050689280 shutdown_connections 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- 172.21.15.133:0/1050689280 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bad240 0x557a58c11170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- 172.21.15.133:0/1050689280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bac920 0x557a58c10c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- 172.21.15.133:0/1050689280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c106f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- 172.21.15.133:0/1050689280 >> 172.21.15.133:0/1050689280 conn(0x557a58b692a0 msgr2=0x557a58b66220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- 172.21.15.133:0/1050689280 shutdown_connections 2024-04-25T10:01:59.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- 172.21.15.133:0/1050689280 wait complete. 2024-04-25T10:01:59.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- start start 2024-04-25T10:01:59.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 Processor -- start 2024-04-25T10:01:59.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c15eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bac920 0x557a58c163f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bad240 0x557a58c12420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a589b63d0 con 0x557a58b62f30 2024-04-25T10:01:59.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a58a6dd50 con 0x557a58bac920 2024-04-25T10:01:59.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a5894c980 con 0x557a58bad240 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 10 client.0 _ll_register_callbacks cb 0x7efcb006ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb6337640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c15eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb6337640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c15eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46114/0 (socket says 172.21.15.133:46114) 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb6337640 1 -- 172.21.15.133:0/875390687 learned_addr learned my addr 172.21.15.133:0/875390687 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb6b38640 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bad240 0x557a58c12420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb5b36640 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bac920 0x557a58c163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb6337640 1 -- 172.21.15.133:0/875390687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bad240 msgr2=0x557a58c12420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb6337640 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bad240 0x557a58c12420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb6337640 1 -- 172.21.15.133:0/875390687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bac920 msgr2=0x557a58c163f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb6337640 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bac920 0x557a58c163f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb6337640 1 -- 172.21.15.133:0/875390687 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a58a33600 con 0x557a58b62f30 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb6b38640 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bad240 0x557a58c12420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb5b36640 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bac920 0x557a58c163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:59.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb6337640 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c15eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efca00136b0 tx=0x7efca0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9089246a75496616 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efc917fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efc917fa640 1 -- 172.21.15.133:0/875390687 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0016bb0 con 0x557a58b62f30 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efc917fa640 1 -- 172.21.15.133:0/875390687 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efca0016d50 con 0x557a58b62f30 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- 172.21.15.133:0/875390687 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efca002e070 con 0x557a58b62f30 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efc917fa640 1 -- 172.21.15.133:0/875390687 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7efca0044050 con 0x557a58b62f30 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 10 client.4428 fetch_fsmap learned FSMap version 7 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- 172.21.15.133:0/875390687 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x557a589469b0 con 0x557a58b62f30 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efc917fa640 1 -- 172.21.15.133:0/875390687 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca002e070 con 0x557a58b62f30 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efc917fa640 1 -- 172.21.15.133:0/875390687 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7efca0047020 con 0x557a58b62f30 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 10 client.4428 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 20 client.4428 populate_metadata read hostname 'smithi133' 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x557a58c12960 0x557a58c12d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 4 client.4428 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- 172.21.15.133:0/875390687 --> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557a589bf2e0 con 0x557a58c12960 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb5b36640 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x557a58c12960 0x557a58c12d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcb5b36640 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x557a58c12960 0x557a58c12d40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=455a87dadfb16265 server_cookie=1eaa5e0f8b61c1bc in_seq=0 out_seq=0 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efc917fa640 10 client.4428 ms_handle_connect on v2:172.21.15.133:6835/2462154167 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efc917fa640 1 -- 172.21.15.133:0/875390687 <== mds.0 v2:172.21.15.133:6835/2462154167 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x557a589bf2e0 con 0x557a58c12960 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efc917fa640 10 client.4428 handle_command_reply: tid=0 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 client.4428 shutdown 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- 172.21.15.133:0/875390687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 msgr2=0x557a58c15eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c15eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efca00136b0 tx=0x7efca0004740 comp rx=0 tx=0).stop 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- 172.21.15.133:0/875390687 shutdown_connections 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x557a58c12960 0x557a58c12d40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557a58bad240 0x557a58c12420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557a58bac920 0x557a58c163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 --2- 172.21.15.133:0/875390687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a58b62f30 0x557a58c15eb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efc917fa640 0 client.4428 ms_handle_reset on v2:172.21.15.133:6835/2462154167 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.236+0000 7efcbb708000 1 -- 172.21.15.133:0/875390687 >> 172.21.15.133:0/875390687 conn(0x557a58b692a0 msgr2=0x557a58bb49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcbb708000 1 -- 172.21.15.133:0/875390687 shutdown_connections 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcbb708000 1 -- 172.21.15.133:0/875390687 wait complete. 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcbb708000 20 client.4428 trim_cache size 0 max 16384 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4296, 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:01:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:01:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 7.2988654249999998, 2024-04-25T10:01:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 7, 2024-04-25T10:01:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 24, 2024-04-25T10:01:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 23, 2024-04-25T10:01:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 13.7767628 2024-04-25T10:01:59.252 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:01:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 -- 172.21.15.133:0/1411794959 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc94066f60 msgr2=0x7efc94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc94066f60 0x7efc94069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efca00099b0 tx=0x7efca0008040 comp rx=0 tx=0).stop 2024-04-25T10:01:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 -- 172.21.15.133:0/1411794959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb0159f60 msgr2=0x7efcb00b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb0159f60 0x7efcb00b3f80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efcac00e7e0 tx=0x7efcac00eca0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 -- 172.21.15.133:0/1411794959 shutdown_connections 2024-04-25T10:01:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcb00cc740 0x7efcb00ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc94066f60 0x7efc94069420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcb0159f60 0x7efcb00b3f80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 --2- 172.21.15.133:0/1411794959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc9c0051d0 0x7efcb00b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 -- 172.21.15.133:0/1411794959 >> 172.21.15.133:0/1411794959 conn(0x7efcb006bfc0 msgr2=0x7efcb015e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 -- 172.21.15.133:0/1411794959 shutdown_connections 2024-04-25T10:01:59.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.240+0000 7efcb859b640 1 -- 172.21.15.133:0/1411794959 wait complete. 2024-04-25T10:01:59.261 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:01:59.262 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:01:59.262 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:01:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.364+0000 7fa5c42d5640 1 Processor -- start 2024-04-25T10:01:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.364+0000 7fa5c42d5640 1 -- start start 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c42d5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5bc15db00 0x7fa5bc0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c42d5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5bc0d0bd0 0x7fa5bc161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c42d5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5bc0ceb20 con 0x7fa5bc0d0770 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c42d5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5bc0b7740 con 0x7fa5bc15db00 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c42d5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc0b7a20 con 0x7fa5bc0d0bd0 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c2071640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5bc15db00 0x7fa5bc0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c2071640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5bc15db00 0x7fa5bc0b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46128/0 (socket says 172.21.15.133:46128) 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c2071640 1 -- 172.21.15.133:0/1807594872 learned_addr learned my addr 172.21.15.133:0/1807594872 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c2872640 1 --2- 172.21.15.133:0/1807594872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5bc0d0bd0 0x7fa5bc161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c2071640 1 -- 172.21.15.133:0/1807594872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5bc0d0bd0 msgr2=0x7fa5bc161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c2071640 1 --2- 172.21.15.133:0/1807594872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5bc0d0bd0 0x7fa5bc161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c2071640 1 -- 172.21.15.133:0/1807594872 >> v1:172.21.15.162:6789/0 conn(0x7fa5bc0d0770 legacy=0x7fa5bc1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c2071640 1 -- 172.21.15.133:0/1807594872 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc0b97f0 con 0x7fa5bc15db00 2024-04-25T10:01:59.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.368+0000 7fa5c2071640 1 --2- 172.21.15.133:0/1807594872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5bc15db00 0x7fa5bc0b2990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009870 tx=0x7fa5ac01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4eb20451d92b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c106f640 1 -- 172.21.15.133:0/1807594872 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac0aa610 con 0x7fa5bc15db00 2024-04-25T10:01:59.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c106f640 1 -- 172.21.15.133:0/1807594872 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac0af070 con 0x7fa5bc15db00 2024-04-25T10:01:59.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c106f640 1 -- 172.21.15.133:0/1807594872 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac0aa9d0 con 0x7fa5bc15db00 2024-04-25T10:01:59.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 -- 172.21.15.133:0/1807594872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5bc15db00 msgr2=0x7fa5bc0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 --2- 172.21.15.133:0/1807594872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5bc15db00 0x7fa5bc0b2990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009870 tx=0x7fa5ac01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 -- 172.21.15.133:0/1807594872 shutdown_connections 2024-04-25T10:01:59.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 --2- 172.21.15.133:0/1807594872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5bc0d0bd0 0x7fa5bc161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 --2- 172.21.15.133:0/1807594872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5bc15db00 0x7fa5bc0b2990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 -- 172.21.15.133:0/1807594872 >> 172.21.15.133:0/1807594872 conn(0x7fa5bc0727c0 msgr2=0x7fa5bc072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 -- 172.21.15.133:0/1807594872 shutdown_connections 2024-04-25T10:01:59.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 -- 172.21.15.133:0/1807594872 wait complete. 2024-04-25T10:01:59.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 Processor -- start 2024-04-25T10:01:59.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 -- start start 2024-04-25T10:01:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5bc0d0770 0x7fa5bc167c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5bc0d0bd0 0x7fa5bc0c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5bc15db00 0x7fa5bc157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc0d0fe0 con 0x7fa5bc15db00 2024-04-25T10:01:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc0b7430 con 0x7fa5bc0d0770 2024-04-25T10:01:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c42d5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc0d18a0 con 0x7fa5bc0d0bd0 2024-04-25T10:01:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c2872640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5bc15db00 0x7fa5bc157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c2872640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5bc15db00 0x7fa5bc157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46132/0 (socket says 172.21.15.133:46132) 2024-04-25T10:01:59.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c2872640 1 -- 172.21.15.133:0/3829127417 learned_addr learned my addr 172.21.15.133:0/3829127417 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c1870640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5bc0d0bd0 0x7fa5bc0c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c2872640 1 -- 172.21.15.133:0/3829127417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5bc0d0bd0 msgr2=0x7fa5bc0c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c2071640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5bc0d0770 0x7fa5bc167c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c2872640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5bc0d0bd0 0x7fa5bc0c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c2872640 1 -- 172.21.15.133:0/3829127417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5bc0d0770 msgr2=0x7fa5bc167c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c2872640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5bc0d0770 0x7fa5bc167c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c2872640 1 -- 172.21.15.133:0/3829127417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc0cacd0 con 0x7fa5bc15db00 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5c2872640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5bc15db00 0x7fa5bc157970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4007e40 tx=0x7fa5b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/3829127417 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4014620 con 0x7fa5bc15db00 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/3829127417 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b4019070 con 0x7fa5bc15db00 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.372+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/3829127417 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4014a20 con 0x7fa5bc15db00 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.376+0000 7fa5c42d5640 1 -- 172.21.15.133:0/3829127417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5bc15e130 con 0x7fa5bc15db00 2024-04-25T10:01:59.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.376+0000 7fa5c42d5640 1 -- 172.21.15.133:0/3829127417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc15e6f0 con 0x7fa5bc15db00 2024-04-25T10:01:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.376+0000 7fa5c42d5640 1 -- 172.21.15.133:0/3829127417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5bc15e8f0 con 0x7fa5bc15db00 2024-04-25T10:01:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.376+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/3829127417 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5b4014bc0 con 0x7fa5bc15db00 2024-04-25T10:01:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.376+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5a0066f60 0x7fa5a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.376+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/3829127417 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa5b4085c60 con 0x7fa5bc15db00 2024-04-25T10:01:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.376+0000 7fa5c2071640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5a0066f60 0x7fa5a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.376+0000 7fa5c2071640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5a0066f60 0x7fa5a0069420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0ae000 tx=0x7fa5ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.380+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/3829127417 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5b4056920 con 0x7fa5bc15db00 2024-04-25T10:01:59.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.564+0000 7fa5c42d5640 1 -- 172.21.15.133:0/3829127417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa5bc168320 con 0x7fa5bc15db00 2024-04-25T10:01:59.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.564+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/3829127417 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa5b4056220 con 0x7fa5bc15db00 2024-04-25T10:01:59.573 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:01:59.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 -- 172.21.15.133:0/3829127417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5a0066f60 msgr2=0x7fa5a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5a0066f60 0x7fa5a0069420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0ae000 tx=0x7fa5ac008040 comp rx=0 tx=0).stop 2024-04-25T10:01:59.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 -- 172.21.15.133:0/3829127417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5bc15db00 msgr2=0x7fa5bc157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5bc15db00 0x7fa5bc157970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4007e40 tx=0x7fa5b400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 -- 172.21.15.133:0/3829127417 shutdown_connections 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5a0066f60 0x7fa5a0069420 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5bc15db00 0x7fa5bc157970 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5bc0d0bd0 0x7fa5bc0c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 --2- 172.21.15.133:0/3829127417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5bc0d0770 0x7fa5bc167c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 -- 172.21.15.133:0/3829127417 >> 172.21.15.133:0/3829127417 conn(0x7fa5bc0727c0 msgr2=0x7fa5bc0d23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 -- 172.21.15.133:0/3829127417 shutdown_connections 2024-04-25T10:01:59.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:01:59.568+0000 7fa5c42d5640 1 -- 172.21.15.133:0/3829127417 wait complete. 2024-04-25T10:01:59.590 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:01:59.590 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:01:59.590 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:01:59.590 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T10:01:59.591 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:01:59.591 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:01:59.591 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:01:59.591 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:01:59.591 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T10:01:59.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:59.591 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:01:59.597 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:01:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:59.598 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:01:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:01:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42512sec preferred_lft 42512sec 2024-04-25T10:01:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:01:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:59.598 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:01:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:01:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:59.599 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:01:59.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:59.599 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:01:59.599 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:01:59.599 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:01:59.599 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2024-04-25T10:01:59.599 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2024-04-25T10:01:59.599 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:01:59.599 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:01:59.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:59.674 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:01:59.725 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:01:59.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:59.726 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:01:59.786 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:01:59.787 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:01:59.787 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:01:59.787 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:01:59.787 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:01:59.787 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:01:59.787 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:01:59.787 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:01:59.787 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:01:59.787 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:01:59.787 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:01:59.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:59.788 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:01:59.788 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:01:59.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:01:59.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:01:59.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:01:59.788 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:01:59.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:59.859 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:01:59.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:59.906 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-04-25T10:01:59.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:59.954 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:01:59.955 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:01:59.955 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:01:59.955 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T10:01:59.955 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:00.019 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T10:02:00.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:00.020 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:02:00.020 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:02:00.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T10:02:00.021 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:00.021 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T10:02:00.021 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T10:02:00.021 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T10:02:00.021 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:00.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:00.134 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:02:00.134 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:02:00.134 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:02:00.134 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:00.134 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:00.153 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T10:02:00.153 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:00.153 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:00.201 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T10:02:00.202 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:00.202 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:00.249 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:02:00.306 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:02:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:02:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:02:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:02:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:02:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:02:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:02:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:02:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:02:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:02:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:02:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:00.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:00.358 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:02:00.406 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:02:00.406 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T10:02:00.451 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:02:00.465 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:02:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:02:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:02:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:02:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:02:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:02:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:02:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:02:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:02:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:02:00.516 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:00.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:00.517 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:02:00.533 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:2024-04-25T10:02:00.528+0000 7f7874116580 -1 init, newargv = 0x55bb526cfb50 newargc=15 2024-04-25T10:02:00.534 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[34022]: starting ceph client 2024-04-25T10:02:00.540 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[34022]: starting fuse 2024-04-25T10:02:01.640 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:02:01.656 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:01.703 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:02:01.703 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:02:01.703 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:02:01.703 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:02:01.703 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:02:01.703 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:02:01.703 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:02:01.703 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:02:01.703 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:01.704 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:02:01.705 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:02:01.705 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:02:01.705 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:01.705 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:01.705 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:01.705 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:01.705 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:02:01.705 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:01.705 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:01.705 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:01.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:01.706 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:02:01.754 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:02:01.754 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:02:01.755 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.admin")) 2024-04-25T10:02:01.756 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:02:01.879 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.admin.34022.asok 2024-04-25T10:02:01.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:01.891 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.34022.asok 2024-04-25T10:02:01.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:01.891 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.34022.asok status 2024-04-25T10:02:02.046 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:02:02.046 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "admin", 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "34022", 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4440, 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:02:02.047 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 4440 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1904437087 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1904437087 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.4440 192.168.0.1:0/1904437087", 2024-04-25T10:02:02.048 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/1904437087", 2024-04-25T10:02:02.049 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:02:02.049 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 7, 2024-04-25T10:02:02.049 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 24, 2024-04-25T10:02:02.049 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:02.049 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:02:02.049 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:02:02.049 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:02:02.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:02.061 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:02:02.061 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:02.067 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:02:02.067 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:02.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:02.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:02.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:02.121 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T10:02:02.192 INFO:teuthology.orchestra.run.smithi133.stdout:# file: . 2024-04-25T10:02:02.193 INFO:teuthology.orchestra.run.smithi133.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T10:02:02.193 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:02.193 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:02:02.193 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:02.239 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.404+0000 7f263b057640 1 Processor -- start 2024-04-25T10:02:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.404+0000 7f263b057640 1 -- start start 2024-04-25T10:02:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f263b057640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f26340b1f70 0x7f263406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f263b057640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26340ce330 0x7f26340ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f263b057640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26340c61d0 con 0x7f263415c9a0 2024-04-25T10:02:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f263b057640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26340cecc0 con 0x7f26340b1f70 2024-04-25T10:02:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f263b057640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26340c64b0 con 0x7f26340ce330 2024-04-25T10:02:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f2633fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f26340b1f70 0x7f263406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f2633fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f26340b1f70 0x7f263406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46138/0 (socket says 172.21.15.133:46138) 2024-04-25T10:02:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f2633fff640 1 -- 172.21.15.133:0/3637350311 learned_addr learned my addr 172.21.15.133:0/3637350311 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f26395f4640 1 --2- 172.21.15.133:0/3637350311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26340ce330 0x7f26340ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f2633fff640 1 -- 172.21.15.133:0/3637350311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26340ce330 msgr2=0x7f26340ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f2633fff640 1 --2- 172.21.15.133:0/3637350311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26340ce330 0x7f26340ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f2633fff640 1 -- 172.21.15.133:0/3637350311 >> v1:172.21.15.162:6789/0 conn(0x7f263415c9a0 legacy=0x7f26340c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f2633fff640 1 -- 172.21.15.133:0/3637350311 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263406d4f0 con 0x7f26340b1f70 2024-04-25T10:02:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f2633fff640 1 --2- 172.21.15.133:0/3637350311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26340b1f70 0x7f263406dcf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2628009870 tx=0x7f262801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bb2b4465420a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f26337fe640 1 -- 172.21.15.133:0/3637350311 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26280aa430 con 0x7f26340b1f70 2024-04-25T10:02:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f26337fe640 1 -- 172.21.15.133:0/3637350311 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26280af070 con 0x7f26340b1f70 2024-04-25T10:02:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f26337fe640 1 -- 172.21.15.133:0/3637350311 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26280aa7f0 con 0x7f26340b1f70 2024-04-25T10:02:02.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f263b057640 1 -- 172.21.15.133:0/3637350311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26340b1f70 msgr2=0x7f263406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f263b057640 1 --2- 172.21.15.133:0/3637350311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26340b1f70 0x7f263406dcf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2628009870 tx=0x7f262801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f263b057640 1 -- 172.21.15.133:0/3637350311 shutdown_connections 2024-04-25T10:02:02.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f263b057640 1 --2- 172.21.15.133:0/3637350311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26340ce330 0x7f26340ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f263b057640 1 --2- 172.21.15.133:0/3637350311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26340b1f70 0x7f263406dcf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.408+0000 7f263b057640 1 -- 172.21.15.133:0/3637350311 >> 172.21.15.133:0/3637350311 conn(0x7f2634067000 msgr2=0x7f263406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 -- 172.21.15.133:0/3637350311 shutdown_connections 2024-04-25T10:02:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 -- 172.21.15.133:0/3637350311 wait complete. 2024-04-25T10:02:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 Processor -- start 2024-04-25T10:02:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 -- start start 2024-04-25T10:02:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26340b1f70 0x7f2634155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26340ce330 0x7f2634147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263415c9a0 0x7f263414c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263415dad0 con 0x7f263415c9a0 2024-04-25T10:02:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263415d950 con 0x7f26340b1f70 2024-04-25T10:02:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263415dc50 con 0x7f26340ce330 2024-04-25T10:02:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26395f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263415c9a0 0x7f263414c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26395f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263415c9a0 0x7f263414c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46142/0 (socket says 172.21.15.133:46142) 2024-04-25T10:02:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26395f4640 1 -- 172.21.15.133:0/641147782 learned_addr learned my addr 172.21.15.133:0/641147782 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f2633fff640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26340ce330 0x7f2634147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f2638df3640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26340b1f70 0x7f2634155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26395f4640 1 -- 172.21.15.133:0/641147782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26340ce330 msgr2=0x7f2634147ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26395f4640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26340ce330 0x7f2634147ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26395f4640 1 -- 172.21.15.133:0/641147782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26340b1f70 msgr2=0x7f2634155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26395f4640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26340b1f70 0x7f2634155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26395f4640 1 -- 172.21.15.133:0/641147782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2634156360 con 0x7f263415c9a0 2024-04-25T10:02:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f2633fff640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26340ce330 0x7f2634147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f2638df3640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26340b1f70 0x7f2634155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26395f4640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263415c9a0 0x7f263414c790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f262c007e60 tx=0x7f262c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26317fa640 1 -- 172.21.15.133:0/641147782 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c014420 con 0x7f263415c9a0 2024-04-25T10:02:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26317fa640 1 -- 172.21.15.133:0/641147782 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c019070 con 0x7f263415c9a0 2024-04-25T10:02:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 -- 172.21.15.133:0/641147782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2634156640 con 0x7f263415c9a0 2024-04-25T10:02:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f26317fa640 1 -- 172.21.15.133:0/641147782 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c014820 con 0x7f263415c9a0 2024-04-25T10:02:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 -- 172.21.15.133:0/641147782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2634156c00 con 0x7f263415c9a0 2024-04-25T10:02:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.412+0000 7f263b057640 1 -- 172.21.15.133:0/641147782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263406bc20 con 0x7f263415c9a0 2024-04-25T10:02:02.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.416+0000 7f26317fa640 1 -- 172.21.15.133:0/641147782 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f262c0149c0 con 0x7f263415c9a0 2024-04-25T10:02:02.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.416+0000 7f26317fa640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2620066f60 0x7f2620069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.416+0000 7f26317fa640 1 -- 172.21.15.133:0/641147782 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f262c086a20 con 0x7f263415c9a0 2024-04-25T10:02:02.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.416+0000 7f2638df3640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2620066f60 0x7f2620069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.416+0000 7f2638df3640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2620066f60 0x7f2620069420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2624003e40 tx=0x7f2624009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.420+0000 7f26317fa640 1 -- 172.21.15.133:0/641147782 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f262c0577d0 con 0x7f263415c9a0 2024-04-25T10:02:02.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.576+0000 7f263b057640 1 -- 172.21.15.133:0/641147782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2634068ef0 con 0x7f263415c9a0 2024-04-25T10:02:02.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.580+0000 7f26317fa640 1 -- 172.21.15.133:0/641147782 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7f262c0570d0 con 0x7f263415c9a0 2024-04-25T10:02:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:02.587 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 -- 172.21.15.133:0/641147782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2620066f60 msgr2=0x7f2620069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2620066f60 0x7f2620069420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2624003e40 tx=0x7f2624009040 comp rx=0 tx=0).stop 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 -- 172.21.15.133:0/641147782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263415c9a0 msgr2=0x7f263414c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263415c9a0 0x7f263414c790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f262c007e60 tx=0x7f262c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 -- 172.21.15.133:0/641147782 shutdown_connections 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2620066f60 0x7f2620069420 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263415c9a0 0x7f263414c790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26340ce330 0x7f2634147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 --2- 172.21.15.133:0/641147782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26340b1f70 0x7f2634155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 -- 172.21.15.133:0/641147782 >> 172.21.15.133:0/641147782 conn(0x7f2634067000 msgr2=0x7f26340c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 -- 172.21.15.133:0/641147782 shutdown_connections 2024-04-25T10:02:02.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.584+0000 7f263b057640 1 -- 172.21.15.133:0/641147782 wait complete. 2024-04-25T10:02:02.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:02.607 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.724+0000 7f7c1e867640 1 Processor -- start 2024-04-25T10:02:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.724+0000 7f7c1e867640 1 -- start start 2024-04-25T10:02:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.724+0000 7f7c1e867640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c1815deb0 0x7f7c1806f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c1e867640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c180cb580 0x7f7c18066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c1e867640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1806e560 con 0x7f7c180669a0 2024-04-25T10:02:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c1e867640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1806e840 con 0x7f7c1815deb0 2024-04-25T10:02:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c1e867640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1806eb20 con 0x7f7c180cb580 2024-04-25T10:02:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c17fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c1815deb0 0x7f7c1806f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c1ce04640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c180cb580 0x7f7c18066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c17fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c1815deb0 0x7f7c1806f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:54502/0 (socket says 172.21.15.133:54502) 2024-04-25T10:02:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c0f7fe640 1 --1- 172.21.15.133:0/2735529122 >> v1:172.21.15.162:6789/0 conn(0x7f7c180669a0 0x7f7c180cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:53864/0 (socket says 172.21.15.133:53864) 2024-04-25T10:02:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c17fff640 1 -- 172.21.15.133:0/2735529122 learned_addr learned my addr 172.21.15.133:0/2735529122 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c1ce04640 1 -- 172.21.15.133:0/2735529122 >> v2:172.21.15.162:3300/0 conn(0x7f7c1815deb0 msgr2=0x7f7c1806f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c1ce04640 1 --2- 172.21.15.133:0/2735529122 >> v2:172.21.15.162:3300/0 conn(0x7f7c1815deb0 0x7f7c1806f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c1ce04640 1 -- 172.21.15.133:0/2735529122 >> v1:172.21.15.162:6789/0 conn(0x7f7c180669a0 legacy=0x7f7c180cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c1ce04640 1 -- 172.21.15.133:0/2735529122 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1806ee00 con 0x7f7c180cb580 2024-04-25T10:02:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c17fff640 1 --2- 172.21.15.133:0/2735529122 >> v2:172.21.15.162:3300/0 conn(0x7f7c1815deb0 0x7f7c1806f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c1ce04640 1 --2- 172.21.15.133:0/2735529122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c180cb580 0x7f7c18066e50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000bfa0 tx=0x7f7c1001eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4abbb487a7df927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c177fe640 1 -- 172.21.15.133:0/2735529122 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1000b2d0 con 0x7f7c180cb580 2024-04-25T10:02:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c177fe640 1 -- 172.21.15.133:0/2735529122 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c100a7a60 con 0x7f7c180cb580 2024-04-25T10:02:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.728+0000 7f7c177fe640 1 -- 172.21.15.133:0/2735529122 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c100a7e30 con 0x7f7c180cb580 2024-04-25T10:02:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 -- 172.21.15.133:0/2735529122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c180cb580 msgr2=0x7f7c18066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 --2- 172.21.15.133:0/2735529122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c180cb580 0x7f7c18066e50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000bfa0 tx=0x7f7c1001eb70 comp rx=0 tx=0).stop 2024-04-25T10:02:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 -- 172.21.15.133:0/2735529122 shutdown_connections 2024-04-25T10:02:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 --2- 172.21.15.133:0/2735529122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c180cb580 0x7f7c18066e50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 --2- 172.21.15.133:0/2735529122 >> v2:172.21.15.162:3300/0 conn(0x7f7c1815deb0 0x7f7c1806f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 -- 172.21.15.133:0/2735529122 >> 172.21.15.133:0/2735529122 conn(0x7f7c180b1800 msgr2=0x7f7c180b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 -- 172.21.15.133:0/2735529122 shutdown_connections 2024-04-25T10:02:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 -- 172.21.15.133:0/2735529122 wait complete. 2024-04-25T10:02:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 Processor -- start 2024-04-25T10:02:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 -- start start 2024-04-25T10:02:02.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c180669a0 0x7f7c180b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c180cb580 0x7f7c181556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c1815deb0 0x7f7c18155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c180ce270 con 0x7f7c180669a0 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c180ce0f0 con 0x7f7c1815deb0 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c1e867640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c180ce3f0 con 0x7f7c180cb580 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c17fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c180669a0 0x7f7c180b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c17fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c180669a0 0x7f7c180b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46148/0 (socket says 172.21.15.133:46148) 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c17fff640 1 -- 172.21.15.133:0/2411500329 learned_addr learned my addr 172.21.15.133:0/2411500329 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c17fff640 1 -- 172.21.15.133:0/2411500329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c180cb580 msgr2=0x7f7c181556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c17fff640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c180cb580 0x7f7c181556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c17fff640 1 -- 172.21.15.133:0/2411500329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c1815deb0 msgr2=0x7f7c18155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c17fff640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c1815deb0 0x7f7c18155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c17fff640 1 -- 172.21.15.133:0/2411500329 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c181568a0 con 0x7f7c180669a0 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.732+0000 7f7c17fff640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c180669a0 0x7f7c180b4980 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800e930 tx=0x7f7c0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.736+0000 7f7c157fa640 1 -- 172.21.15.133:0/2411500329 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08017070 con 0x7f7c180669a0 2024-04-25T10:02:02.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.736+0000 7f7c157fa640 1 -- 172.21.15.133:0/2411500329 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0800ca00 con 0x7f7c180669a0 2024-04-25T10:02:02.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.736+0000 7f7c1e867640 1 -- 172.21.15.133:0/2411500329 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c18156150 con 0x7f7c180669a0 2024-04-25T10:02:02.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.736+0000 7f7c157fa640 1 -- 172.21.15.133:0/2411500329 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0800cdd0 con 0x7f7c180669a0 2024-04-25T10:02:02.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.736+0000 7f7c1e867640 1 -- 172.21.15.133:0/2411500329 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c18156b40 con 0x7f7c180669a0 2024-04-25T10:02:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.736+0000 7f7c1e867640 1 -- 172.21.15.133:0/2411500329 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c18067c70 con 0x7f7c180669a0 2024-04-25T10:02:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.736+0000 7f7c157fa640 1 -- 172.21.15.133:0/2411500329 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c0801a070 con 0x7f7c180669a0 2024-04-25T10:02:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.736+0000 7f7c157fa640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7bf8066f60 0x7f7bf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.736+0000 7f7c157fa640 1 -- 172.21.15.133:0/2411500329 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7c080851a0 con 0x7f7c180669a0 2024-04-25T10:02:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.736+0000 7f7c0f7fe640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7bf8066f60 0x7f7bf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.736+0000 7f7c0f7fe640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7bf8066f60 0x7f7bf8069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7c00000f70 tx=0x7f7c00007a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.740+0000 7f7c157fa640 1 -- 172.21.15.133:0/2411500329 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c08055f00 con 0x7f7c180669a0 2024-04-25T10:02:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.900+0000 7f7c1e867640 1 -- 172.21.15.133:0/2411500329 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c180b5c90 con 0x7f7c180669a0 2024-04-25T10:02:02.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.900+0000 7f7c157fa640 1 -- 172.21.15.133:0/2411500329 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c08055800 con 0x7f7c180669a0 2024-04-25T10:02:02.908 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 -- 172.21.15.133:0/2411500329 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7bf8066f60 msgr2=0x7f7bf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7bf8066f60 0x7f7bf8069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7c00000f70 tx=0x7f7c00007a10 comp rx=0 tx=0).stop 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 -- 172.21.15.133:0/2411500329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c180669a0 msgr2=0x7f7c180b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c180669a0 0x7f7c180b4980 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800e930 tx=0x7f7c0800edf0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 -- 172.21.15.133:0/2411500329 shutdown_connections 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c1815deb0 0x7f7c18155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c180cb580 0x7f7c181556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7bf8066f60 0x7f7bf8069420 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 --2- 172.21.15.133:0/2411500329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c180669a0 0x7f7c180b4980 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 -- 172.21.15.133:0/2411500329 >> 172.21.15.133:0/2411500329 conn(0x7f7c180b1800 msgr2=0x7f7c18071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 -- 172.21.15.133:0/2411500329 shutdown_connections 2024-04-25T10:02:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:02.904+0000 7f7c1e867640 1 -- 172.21.15.133:0/2411500329 wait complete. 2024-04-25T10:02:02.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d6c089640 1 Processor -- start 2024-04-25T10:02:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d6c089640 1 -- start start 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d6c089640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6d6415daf0 0x7f6d640cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d6c089640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6415ded0 con 0x7f6d640b91c0 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d6c089640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d640d10d0 con 0x7f6d640cec10 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d6c089640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d640d1da0 con 0x7f6d6415daf0 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d69624640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6d640b91c0 0x7f6d640ce500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38314/0 (socket says 172.21.15.133:38314) 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d69624640 1 -- 172.21.15.133:0/552772607 learned_addr learned my addr 172.21.15.133:0/552772607 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d69e25640 1 --2- 172.21.15.133:0/552772607 >> v2:172.21.15.162:3300/0 conn(0x7f6d6415daf0 0x7f6d640cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3270882927 0 0) 0x7f6d6415ded0 con 0x7f6d640b91c0 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d50003530 con 0x7f6d640b91c0 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1251557420 0 0) 0x7f6d50003530 con 0x7f6d640b91c0 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 >> v2:172.21.15.162:3300/0 conn(0x7f6d6415daf0 msgr2=0x7f6d640cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 --2- 172.21.15.133:0/552772607 >> v2:172.21.15.162:3300/0 conn(0x7f6d6415daf0 0x7f6d640cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 >> v1:172.21.15.162:6789/0 conn(0x7f6d640cec10 legacy=0x7f6d640d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640d0830 con 0x7f6d640b91c0 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6d54001680 con 0x7f6d640b91c0 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 >> v1:172.21.15.133:6789/0 conn(0x7f6d640b91c0 legacy=0x7f6d640ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 --2- 172.21.15.133:0/552772607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d500039c0 0x7f6d50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 --2- 172.21.15.133:0/552772607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d50004810 0x7f6d50004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 --2- 172.21.15.133:0/552772607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d500051d0 0x7f6d5000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640d1da0 con 0x7f6d50004810 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5000c170 con 0x7f6d500039c0 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5000c370 con 0x7f6d500051d0 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d69624640 1 --2- 172.21.15.133:0/552772607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d50004810 0x7f6d50004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d69e25640 1 --2- 172.21.15.133:0/552772607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d500039c0 0x7f6d50003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d69624640 1 -- 172.21.15.133:0/552772607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d500051d0 msgr2=0x7f6d5000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d6a626640 1 --2- 172.21.15.133:0/552772607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d500051d0 0x7f6d5000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d69624640 1 --2- 172.21.15.133:0/552772607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d500051d0 0x7f6d5000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d69624640 1 -- 172.21.15.133:0/552772607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d500039c0 msgr2=0x7f6d50003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d69624640 1 --2- 172.21.15.133:0/552772607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d500039c0 0x7f6d50003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d69624640 1 -- 172.21.15.133:0/552772607 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d50003530 con 0x7f6d50004810 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.040+0000 7f6d6a626640 1 -- 172.21.15.133:0/552772607 reap_dead start 2024-04-25T10:02:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69624640 1 --2- 172.21.15.133:0/552772607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d50004810 0x7f6d50004c90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400c210 tx=0x7f6d5401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b0ec00a5eadaa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54007d40 con 0x7f6d50004810 2024-04-25T10:02:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d540a9070 con 0x7f6d50004810 2024-04-25T10:02:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d68e23640 1 -- 172.21.15.133:0/552772607 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54005d80 con 0x7f6d50004810 2024-04-25T10:02:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 -- 172.21.15.133:0/552772607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d50004810 msgr2=0x7f6d50004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 --2- 172.21.15.133:0/552772607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d50004810 0x7f6d50004c90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400c210 tx=0x7f6d5401cca0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 -- 172.21.15.133:0/552772607 shutdown_connections 2024-04-25T10:02:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 --2- 172.21.15.133:0/552772607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d50004810 0x7f6d50004c90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 -- 172.21.15.133:0/552772607 >> 172.21.15.133:0/552772607 conn(0x7f6d6406bf80 msgr2=0x7f6d6406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 -- 172.21.15.133:0/552772607 shutdown_connections 2024-04-25T10:02:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 -- 172.21.15.133:0/552772607 wait complete. 2024-04-25T10:02:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 Processor -- start 2024-04-25T10:02:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 -- start start 2024-04-25T10:02:03.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d50004810 0x7f6d6415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d64158600 0x7f6d641e9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d641f7180 0x7f6d6415bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640d0560 con 0x7f6d64158600 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640d03e0 con 0x7f6d50004810 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d6c089640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d64161db0 con 0x7f6d641f7180 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69624640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d64158600 0x7f6d641e9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69624640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d64158600 0x7f6d641e9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46172/0 (socket says 172.21.15.133:46172) 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69624640 1 -- 172.21.15.133:0/1076243508 learned_addr learned my addr 172.21.15.133:0/1076243508 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69e25640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d50004810 0x7f6d6415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69624640 1 -- 172.21.15.133:0/1076243508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d641f7180 msgr2=0x7f6d6415bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69624640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d641f7180 0x7f6d6415bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69624640 1 -- 172.21.15.133:0/1076243508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d50004810 msgr2=0x7f6d6415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69624640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d50004810 0x7f6d6415cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69624640 1 -- 172.21.15.133:0/1076243508 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640737c0 con 0x7f6d64158600 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69e25640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d50004810 0x7f6d6415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.044+0000 7f6d69624640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d64158600 0x7f6d641e9f50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6d5401cad0 tx=0x7f6d50005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.048+0000 7f6d5a7fc640 1 -- 172.21.15.133:0/1076243508 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54007d40 con 0x7f6d64158600 2024-04-25T10:02:03.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.048+0000 7f6d5a7fc640 1 -- 172.21.15.133:0/1076243508 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d54000ee0 con 0x7f6d64158600 2024-04-25T10:02:03.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.048+0000 7f6d6c089640 1 -- 172.21.15.133:0/1076243508 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d641e9660 con 0x7f6d64158600 2024-04-25T10:02:03.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.048+0000 7f6d5a7fc640 1 -- 172.21.15.133:0/1076243508 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d540047f0 con 0x7f6d64158600 2024-04-25T10:02:03.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.048+0000 7f6d6c089640 1 -- 172.21.15.133:0/1076243508 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6415b4b0 con 0x7f6d64158600 2024-04-25T10:02:03.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.048+0000 7f6d6c089640 1 -- 172.21.15.133:0/1076243508 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d64068e80 con 0x7f6d64158600 2024-04-25T10:02:03.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.048+0000 7f6d5a7fc640 1 -- 172.21.15.133:0/1076243508 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d54004990 con 0x7f6d64158600 2024-04-25T10:02:03.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.048+0000 7f6d5a7fc640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d48066f60 0x7f6d48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.048+0000 7f6d5a7fc640 1 -- 172.21.15.133:0/1076243508 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6d5411ec90 con 0x7f6d64158600 2024-04-25T10:02:03.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.048+0000 7f6d69e25640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d48066f60 0x7f6d48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.048+0000 7f6d69e25640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d48066f60 0x7f6d48069420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d640b76d0 tx=0x7f6d60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.052+0000 7f6d5a7fc640 1 -- 172.21.15.133:0/1076243508 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d540bf060 con 0x7f6d64158600 2024-04-25T10:02:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 -- 172.21.15.133:0/1076243508 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d640c7920 con 0x7f6d64158600 2024-04-25T10:02:03.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d5a7fc640 1 -- 172.21.15.133:0/1076243508 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d540169b0 con 0x7f6d64158600 2024-04-25T10:02:03.214 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 -- 172.21.15.133:0/1076243508 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d48066f60 msgr2=0x7f6d48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d48066f60 0x7f6d48069420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d640b76d0 tx=0x7f6d60008040 comp rx=0 tx=0).stop 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 -- 172.21.15.133:0/1076243508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d64158600 msgr2=0x7f6d641e9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d64158600 0x7f6d641e9f50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6d5401cad0 tx=0x7f6d50005840 comp rx=0 tx=0).stop 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 -- 172.21.15.133:0/1076243508 shutdown_connections 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d641f7180 0x7f6d6415bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d48066f60 0x7f6d48069420 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d64158600 0x7f6d641e9f50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 --2- 172.21.15.133:0/1076243508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d50004810 0x7f6d6415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 -- 172.21.15.133:0/1076243508 >> 172.21.15.133:0/1076243508 conn(0x7f6d6406bf80 msgr2=0x7f6d640cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 -- 172.21.15.133:0/1076243508 shutdown_connections 2024-04-25T10:02:03.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.208+0000 7f6d6c089640 1 -- 172.21.15.133:0/1076243508 wait complete. 2024-04-25T10:02:03.232 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:03.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f5080026640 1 Processor -- start 2024-04-25T10:02:03.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f5080026640 1 -- start start 2024-04-25T10:02:03.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f5080026640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f507815f020 0x7f50780cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f5080026640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50780cefa0 con 0x7f507806f8f0 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f5080026640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50780ccdf0 con 0x7f50780d21c0 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f5080026640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50780cd690 con 0x7f507815f020 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507e5c3640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f507806f8f0 0x7f507806fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38328/0 (socket says 172.21.15.133:38328) 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507e5c3640 1 -- 172.21.15.133:0/3133427143 learned_addr learned my addr 172.21.15.133:0/3133427143 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507ddc2640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f507815f020 0x7f50780cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244854942 0 0) 0x7f50700014a0 con 0x7f507806f8f0 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5064003530 con 0x7f507806f8f0 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 939612300 0 0) 0x7f5064003530 con 0x7f507806f8f0 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f507815f020 msgr2=0x7f50780cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f507815f020 0x7f50780cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 >> v1:172.21.15.162:6789/0 conn(0x7f50780d21c0 legacy=0x7f50780cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780cd0d0 con 0x7f507806f8f0 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5070001bd0 con 0x7f507806f8f0 2024-04-25T10:02:03.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 >> v1:172.21.15.133:6789/0 conn(0x7f507806f8f0 legacy=0x7f507806fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f507815f020 0x7f50640039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5064003f70 0x7f5064004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5064004d50 0x7f50640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780cd690 con 0x7f5064003f70 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506400bab0 con 0x7f5064004d50 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506400bcb0 con 0x7f507815f020 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507d5c1640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5064003f70 0x7f5064004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507ddc2640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f507815f020 0x7f50640039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507e5c3640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5064004d50 0x7f50640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507d5c1640 1 -- 172.21.15.133:0/3133427143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f507815f020 msgr2=0x7f50640039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507d5c1640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f507815f020 0x7f50640039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507d5c1640 1 -- 172.21.15.133:0/3133427143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5064004d50 msgr2=0x7f50640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507d5c1640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5064004d50 0x7f50640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.344+0000 7f507d5c1640 1 -- 172.21.15.133:0/3133427143 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5064003530 con 0x7f5064003f70 2024-04-25T10:02:03.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f507d5c1640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5064003f70 0x7f5064004810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5068001780 tx=0x7f506801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983decd0b217cf1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50680013a0 con 0x7f5064003f70 2024-04-25T10:02:03.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50680a9070 con 0x7f5064003f70 2024-04-25T10:02:03.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f507cdc0640 1 -- 172.21.15.133:0/3133427143 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068003600 con 0x7f5064003f70 2024-04-25T10:02:03.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 -- 172.21.15.133:0/3133427143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5064003f70 msgr2=0x7f5064004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 --2- 172.21.15.133:0/3133427143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5064003f70 0x7f5064004810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5068001780 tx=0x7f506801c420 comp rx=0 tx=0).stop 2024-04-25T10:02:03.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f507e5c3640 1 -- 172.21.15.133:0/3133427143 reap_dead start 2024-04-25T10:02:03.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 -- 172.21.15.133:0/3133427143 shutdown_connections 2024-04-25T10:02:03.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 -- 172.21.15.133:0/3133427143 >> 172.21.15.133:0/3133427143 conn(0x7f50780d16a0 msgr2=0x7f50780d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 -- 172.21.15.133:0/3133427143 shutdown_connections 2024-04-25T10:02:03.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 -- 172.21.15.133:0/3133427143 wait complete. 2024-04-25T10:02:03.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 Processor -- start 2024-04-25T10:02:03.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 -- start start 2024-04-25T10:02:03.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5064003f70 0x7f50780c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50780cb4c0 0x7f50781f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f507809d7d0 0x7f507809dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780cfec0 con 0x7f507809d7d0 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507806f3d0 con 0x7f50780cb4c0 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.348+0000 7f5080026640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50780d0040 con 0x7f5064003f70 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507e5c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f507809d7d0 0x7f507809dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507e5c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f507809d7d0 0x7f507809dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46194/0 (socket says 172.21.15.133:46194) 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507e5c3640 1 -- 172.21.15.133:0/1346681889 learned_addr learned my addr 172.21.15.133:0/1346681889 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507ddc2640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5064003f70 0x7f50780c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507e5c3640 1 -- 172.21.15.133:0/1346681889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5064003f70 msgr2=0x7f50780c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507e5c3640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5064003f70 0x7f50780c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507e5c3640 1 -- 172.21.15.133:0/1346681889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50780cb4c0 msgr2=0x7f50781f0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507e5c3640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50780cb4c0 0x7f50781f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507e5c3640 1 -- 172.21.15.133:0/1346681889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507809b4c0 con 0x7f507809d7d0 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507ddc2640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5064003f70 0x7f50780c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507e5c3640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f507809d7d0 0x7f507809dc50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5078072f90 tx=0x7f50700108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f506e7fc640 1 -- 172.21.15.133:0/1346681889 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507001a070 con 0x7f507809d7d0 2024-04-25T10:02:03.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f506e7fc640 1 -- 172.21.15.133:0/1346681889 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5070007070 con 0x7f507809d7d0 2024-04-25T10:02:03.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f5080026640 1 -- 172.21.15.133:0/1346681889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50780c99d0 con 0x7f507809d7d0 2024-04-25T10:02:03.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f506e7fc640 1 -- 172.21.15.133:0/1346681889 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070007460 con 0x7f507809d7d0 2024-04-25T10:02:03.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f5080026640 1 -- 172.21.15.133:0/1346681889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50780c9f90 con 0x7f507809d7d0 2024-04-25T10:02:03.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f5080026640 1 -- 172.21.15.133:0/1346681889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5078067610 con 0x7f507809d7d0 2024-04-25T10:02:03.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f506e7fc640 1 -- 172.21.15.133:0/1346681889 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f507001d070 con 0x7f507809d7d0 2024-04-25T10:02:03.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f506e7fc640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f505c066f60 0x7f505c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f506e7fc640 1 -- 172.21.15.133:0/1346681889 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5070087500 con 0x7f507809d7d0 2024-04-25T10:02:03.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507ddc2640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f505c066f60 0x7f505c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.352+0000 7f507ddc2640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f505c066f60 0x7f505c069420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5074009a20 tx=0x7f5074008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.356+0000 7f506e7fc640 1 -- 172.21.15.133:0/1346681889 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50780cefa0 con 0x7f507809d7d0 2024-04-25T10:02:03.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.508+0000 7f5080026640 1 -- 172.21.15.133:0/1346681889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5078067c70 con 0x7f507809d7d0 2024-04-25T10:02:03.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.508+0000 7f506e7fc640 1 -- 172.21.15.133:0/1346681889 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5070058140 con 0x7f507809d7d0 2024-04-25T10:02:03.517 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 -- 172.21.15.133:0/1346681889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f505c066f60 msgr2=0x7f505c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f505c066f60 0x7f505c069420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5074009a20 tx=0x7f5074008040 comp rx=0 tx=0).stop 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 -- 172.21.15.133:0/1346681889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f507809d7d0 msgr2=0x7f507809dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f507809d7d0 0x7f507809dc50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5078072f90 tx=0x7f50700108f0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 -- 172.21.15.133:0/1346681889 shutdown_connections 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f505c066f60 0x7f505c069420 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f507809d7d0 0x7f507809dc50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50780cb4c0 0x7f50781f0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 --2- 172.21.15.133:0/1346681889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5064003f70 0x7f50780c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 -- 172.21.15.133:0/1346681889 >> 172.21.15.133:0/1346681889 conn(0x7f50780d16a0 msgr2=0x7f507815bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 -- 172.21.15.133:0/1346681889 shutdown_connections 2024-04-25T10:02:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.512+0000 7f5080026640 1 -- 172.21.15.133:0/1346681889 wait complete. 2024-04-25T10:02:03.534 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:03.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9023f60640 1 Processor -- start 2024-04-25T10:02:03.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9023f60640 1 -- start start 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9023f60640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f901c0d2030 0x7f901c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9023f60640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901c0d24c0 con 0x7f901c15d9a0 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9023f60640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901c15ebd0 con 0x7f901c0ce600 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9023f60640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c15eeb0 con 0x7f901c0d2030 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9021cfc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f901c15d9a0 0x7f901c0d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38332/0 (socket says 172.21.15.133:38332) 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9021cfc640 1 -- 172.21.15.133:0/2598221473 learned_addr learned my addr 172.21.15.133:0/2598221473 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f90224fd640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f901c0d2030 0x7f901c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38326499 0 0) 0x7f901c0d24c0 con 0x7f901c15d9a0 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9008003530 con 0x7f901c15d9a0 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3265417097 0 0) 0x7f9008003530 con 0x7f901c15d9a0 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f901c0d2030 msgr2=0x7f901c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f901c0d2030 0x7f901c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 >> v1:172.21.15.162:6789/0 conn(0x7f901c0ce600 legacy=0x7f901c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c0d0330 con 0x7f901c15d9a0 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f900c001680 con 0x7f901c15d9a0 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 >> v1:172.21.15.133:6789/0 conn(0x7f901c15d9a0 legacy=0x7f901c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90080039f0 0x7f9008003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90080043a0 0x7f9008004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f901c0d2030 0x7f9008005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c15eeb0 con 0x7f90080039f0 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900800ba80 con 0x7f90080043a0 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.640+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900800bc80 con 0x7f901c0d2030 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9021cfc640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90080039f0 0x7f9008003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f90224fd640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f901c0d2030 0x7f9008005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f90214fb640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90080043a0 0x7f9008004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9021cfc640 1 -- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f901c0d2030 msgr2=0x7f9008005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9021cfc640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f901c0d2030 0x7f9008005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9021cfc640 1 -- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90080043a0 msgr2=0x7f9008004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9021cfc640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90080043a0 0x7f9008004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9021cfc640 1 -- 172.21.15.133:0/2598221473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9008003530 con 0x7f90080039f0 2024-04-25T10:02:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f90224fd640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f901c0d2030 0x7f9008005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:03.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f90214fb640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90080043a0 0x7f9008004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:03.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9021cfc640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90080039f0 0x7f9008003dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f900c00c240 tx=0x7f900c01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca968c1d993af952 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c007d40 con 0x7f90080039f0 2024-04-25T10:02:03.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c0a9070 con 0x7f90080039f0 2024-04-25T10:02:03.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9020cfa640 1 -- 172.21.15.133:0/2598221473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c0a5dd0 con 0x7f90080039f0 2024-04-25T10:02:03.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9023f60640 1 -- 172.21.15.133:0/2598221473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90080039f0 msgr2=0x7f9008003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9023f60640 1 --2- 172.21.15.133:0/2598221473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90080039f0 0x7f9008003dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f900c00c240 tx=0x7f900c01cc70 comp rx=0 tx=0).stop 2024-04-25T10:02:03.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f90224fd640 1 -- 172.21.15.133:0/2598221473 reap_dead start 2024-04-25T10:02:03.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9023f60640 1 -- 172.21.15.133:0/2598221473 shutdown_connections 2024-04-25T10:02:03.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9023f60640 1 -- 172.21.15.133:0/2598221473 >> 172.21.15.133:0/2598221473 conn(0x7f901c06bf80 msgr2=0x7f901c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9023f60640 1 -- 172.21.15.133:0/2598221473 shutdown_connections 2024-04-25T10:02:03.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.644+0000 7f9023f60640 1 -- 172.21.15.133:0/2598221473 wait complete. 2024-04-25T10:02:03.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9023f60640 1 Processor -- start 2024-04-25T10:02:03.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9023f60640 1 -- start start 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9023f60640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90080039f0 0x7f901c0b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9023f60640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c15cab0 0x7f901c0b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9023f60640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f901c14acf0 0x7f901c14b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9023f60640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0b9820 con 0x7f901c15cab0 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9023f60640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c0b96a0 con 0x7f901c14acf0 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9023f60640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901c15e590 con 0x7f90080039f0 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90214fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c15cab0 0x7f901c0b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90214fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c15cab0 0x7f901c0b8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46224/0 (socket says 172.21.15.133:46224) 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90214fb640 1 -- 172.21.15.133:0/1565533161 learned_addr learned my addr 172.21.15.133:0/1565533161 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90224fd640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f901c14acf0 0x7f901c14b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9021cfc640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90080039f0 0x7f901c0b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90214fb640 1 -- 172.21.15.133:0/1565533161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90080039f0 msgr2=0x7f901c0b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90214fb640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90080039f0 0x7f901c0b7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90214fb640 1 -- 172.21.15.133:0/1565533161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f901c14acf0 msgr2=0x7f901c14b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90214fb640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f901c14acf0 0x7f901c14b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90214fb640 1 -- 172.21.15.133:0/1565533161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c0cd400 con 0x7f901c15cab0 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90214fb640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c15cab0 0x7f901c0b8380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f901800d7f0 tx=0x7f901800dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90127fc640 1 -- 172.21.15.133:0/1565533161 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018005480 con 0x7f901c15cab0 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90127fc640 1 -- 172.21.15.133:0/1565533161 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9018017070 con 0x7f901c15cab0 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9023f60640 1 -- 172.21.15.133:0/1565533161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c0cd670 con 0x7f901c15cab0 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90127fc640 1 -- 172.21.15.133:0/1565533161 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901800b8d0 con 0x7f901c15cab0 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9023f60640 1 -- 172.21.15.133:0/1565533161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c14b6b0 con 0x7f901c15cab0 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9023f60640 1 -- 172.21.15.133:0/1565533161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c14b970 con 0x7f901c15cab0 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90127fc640 1 -- 172.21.15.133:0/1565533161 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f901800ba70 con 0x7f901c15cab0 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90127fc640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 0x7f9000069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f90127fc640 1 -- 172.21.15.133:0/1565533161 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f90180852e0 con 0x7f901c15cab0 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9021cfc640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 0x7f9000069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.648+0000 7f9021cfc640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 0x7f9000069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f900c0ab000 tx=0x7f900c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.652+0000 7f90127fc640 1 -- 172.21.15.133:0/1565533161 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9018019310 con 0x7f901c15cab0 2024-04-25T10:02:03.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.800+0000 7f9023f60640 1 -- 172.21.15.133:0/1565533161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f901c068e80 con 0x7f901c15cab0 2024-04-25T10:02:03.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.800+0000 7f90127fc640 1 -- 172.21.15.133:0/1565533161 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9018055fa0 con 0x7f901c15cab0 2024-04-25T10:02:03.808 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:03.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 -- 172.21.15.133:0/1565533161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 msgr2=0x7f9000069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 0x7f9000069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f900c0ab000 tx=0x7f900c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:02:03.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 -- 172.21.15.133:0/1565533161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c15cab0 msgr2=0x7f901c0b8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c15cab0 0x7f901c0b8380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f901800d7f0 tx=0x7f901800dcb0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 -- 172.21.15.133:0/1565533161 shutdown_connections 2024-04-25T10:02:03.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f901c14acf0 0x7f901c14b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 0x7f9000069420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f901c15cab0 0x7f901c0b8380 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 --2- 172.21.15.133:0/1565533161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90080039f0 0x7f901c0b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 -- 172.21.15.133:0/1565533161 >> 172.21.15.133:0/1565533161 conn(0x7f901c06bf80 msgr2=0x7f901c0cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 -- 172.21.15.133:0/1565533161 shutdown_connections 2024-04-25T10:02:03.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.804+0000 7f9023f60640 1 -- 172.21.15.133:0/1565533161 wait complete. 2024-04-25T10:02:03.826 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:03.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f73a1679640 1 Processor -- start 2024-04-25T10:02:03.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f73a1679640 1 -- start start 2024-04-25T10:02:03.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f73a1679640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f739c143560 0x7f739c140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f73a1679640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f739c15d9f0 0x7f739c15de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f73a1679640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c0cf280 con 0x7f739c142920 2024-04-25T10:02:03.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f73a1679640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739c06d440 con 0x7f739c143560 2024-04-25T10:02:03.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f73a1679640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739c06d720 con 0x7f739c15d9f0 2024-04-25T10:02:03.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739affd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f739c142920 0x7f739c0ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38346/0 (socket says 172.21.15.133:38346) 2024-04-25T10:02:03.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739affd640 1 -- 172.21.15.133:0/2772746100 learned_addr learned my addr 172.21.15.133:0/2772746100 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:03.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555839428 0 0) 0x7f739c0cf280 con 0x7f739c142920 2024-04-25T10:02:03.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739a7fc640 1 --2- 172.21.15.133:0/2772746100 >> v2:172.21.15.162:3300/0 conn(0x7f739c143560 0x7f739c140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739b7fe640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f739c15d9f0 0x7f739c15de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7384004640 con 0x7f739c142920 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4194852975 0 0) 0x7f7384004640 con 0x7f739c142920 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f739c15d9f0 msgr2=0x7f739c15de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f739c15d9f0 0x7f739c15de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 >> v2:172.21.15.162:3300/0 conn(0x7f739c143560 msgr2=0x7f739c140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 --2- 172.21.15.133:0/2772746100 >> v2:172.21.15.162:3300/0 conn(0x7f739c143560 0x7f739c140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c06da00 con 0x7f739c142920 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f73900016a0 con 0x7f739c142920 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739b7fe640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f739c15d9f0 0x7f739c15de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 >> v1:172.21.15.133:6789/0 conn(0x7f739c142920 legacy=0x7f739c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7384004ad0 0x7f7384004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7384005920 0x7f7384005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f739c15d9f0 0x7f73840062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c06d440 con 0x7f7384005920 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c06d720 con 0x7f7384004ad0 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738400cb60 con 0x7f739c15d9f0 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739a7fc640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7384005920 0x7f7384005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739b7fe640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f739c15d9f0 0x7f73840062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739affd640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7384004ad0 0x7f7384004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739a7fc640 1 -- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f739c15d9f0 msgr2=0x7f73840062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739a7fc640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f739c15d9f0 0x7f73840062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739a7fc640 1 -- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7384004ad0 msgr2=0x7f7384004ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739a7fc640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7384004ad0 0x7f7384004ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.928+0000 7f739a7fc640 1 -- 172.21.15.133:0/2772746100 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7384004640 con 0x7f7384005920 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f739a7fc640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7384005920 0x7f7384005da0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f738c009ae0 tx=0x7f738c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85e13c14f5032ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c0096b0 con 0x7f7384005920 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738c0a6440 con 0x7f7384005920 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f7399ffb640 1 -- 172.21.15.133:0/2772746100 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c0a6830 con 0x7f7384005920 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f73a1679640 1 -- 172.21.15.133:0/2772746100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7384005920 msgr2=0x7f7384005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f73a1679640 1 --2- 172.21.15.133:0/2772746100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7384005920 0x7f7384005da0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f738c009ae0 tx=0x7f738c01cab0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f739b7fe640 1 -- 172.21.15.133:0/2772746100 reap_dead start 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f73a1679640 1 -- 172.21.15.133:0/2772746100 shutdown_connections 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f73a1679640 1 -- 172.21.15.133:0/2772746100 >> 172.21.15.133:0/2772746100 conn(0x7f739c066f70 msgr2=0x7f739c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f73a1679640 1 -- 172.21.15.133:0/2772746100 shutdown_connections 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f73a1679640 1 -- 172.21.15.133:0/2772746100 wait complete. 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f73a1679640 1 Processor -- start 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.932+0000 7f73a1679640 1 -- start start 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73a1679640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7384005920 0x7f739c1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73a1679640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f739c13ece0 0x7f739c13f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73a1679640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f739c155b00 0x7f739c155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73a1679640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c06e2f0 con 0x7f739c155b00 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73a1679640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c06e170 con 0x7f739c13ece0 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73a1679640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739c06e470 con 0x7f7384005920 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f739c155b00 0x7f739c155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f739c155b00 0x7f739c155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46246/0 (socket says 172.21.15.133:46246) 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739b7fe640 1 -- 172.21.15.133:0/1502097386 learned_addr learned my addr 172.21.15.133:0/1502097386 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739a7fc640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f739c13ece0 0x7f739c13f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739affd640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7384005920 0x7f739c1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739b7fe640 1 -- 172.21.15.133:0/1502097386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7384005920 msgr2=0x7f739c1548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739b7fe640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7384005920 0x7f739c1548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739b7fe640 1 -- 172.21.15.133:0/1502097386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f739c13ece0 msgr2=0x7f739c13f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739b7fe640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f739c13ece0 0x7f739c13f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739b7fe640 1 -- 172.21.15.133:0/1502097386 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c154f80 con 0x7f739c155b00 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739b7fe640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f739c155b00 0x7f739c155fb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f739c0afa40 tx=0x7f7394004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73837fe640 1 -- 172.21.15.133:0/1502097386 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394007d20 con 0x7f739c155b00 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73837fe640 1 -- 172.21.15.133:0/1502097386 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394019070 con 0x7f739c155b00 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73a1679640 1 -- 172.21.15.133:0/1502097386 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739c155260 con 0x7f739c155b00 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73837fe640 1 -- 172.21.15.133:0/1502097386 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394014650 con 0x7f739c155b00 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73a1679640 1 -- 172.21.15.133:0/1502097386 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c155820 con 0x7f739c155b00 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73a1679640 1 -- 172.21.15.133:0/1502097386 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739c068e80 con 0x7f739c155b00 2024-04-25T10:02:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73837fe640 1 -- 172.21.15.133:0/1502097386 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f73940147f0 con 0x7f739c155b00 2024-04-25T10:02:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73837fe640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7388066f60 0x7f7388069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f73837fe640 1 -- 172.21.15.133:0/1502097386 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7394085770 con 0x7f739c155b00 2024-04-25T10:02:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739affd640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7388066f60 0x7f7388069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.936+0000 7f739affd640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7388066f60 0x7f7388069420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f739c06f600 tx=0x7f7390009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:03.940+0000 7f73837fe640 1 -- 172.21.15.133:0/1502097386 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73940564d0 con 0x7f739c155b00 2024-04-25T10:02:04.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 -- 172.21.15.133:0/1502097386 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f739c15c5a0 con 0x7f739c155b00 2024-04-25T10:02:04.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73837fe640 1 -- 172.21.15.133:0/1502097386 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7394055dd0 con 0x7f739c155b00 2024-04-25T10:02:04.115 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 -- 172.21.15.133:0/1502097386 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7388066f60 msgr2=0x7f7388069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7388066f60 0x7f7388069420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f739c06f600 tx=0x7f7390009040 comp rx=0 tx=0).stop 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 -- 172.21.15.133:0/1502097386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f739c155b00 msgr2=0x7f739c155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f739c155b00 0x7f739c155fb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f739c0afa40 tx=0x7f7394004700 comp rx=0 tx=0).stop 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 -- 172.21.15.133:0/1502097386 shutdown_connections 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7388066f60 0x7f7388069420 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f739c155b00 0x7f739c155fb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f739c13ece0 0x7f739c13f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 --2- 172.21.15.133:0/1502097386 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7384005920 0x7f739c1548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 -- 172.21.15.133:0/1502097386 >> 172.21.15.133:0/1502097386 conn(0x7f739c066f70 msgr2=0x7f739c0afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 -- 172.21.15.133:0/1502097386 shutdown_connections 2024-04-25T10:02:04.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.108+0000 7f73a1679640 1 -- 172.21.15.133:0/1502097386 wait complete. 2024-04-25T10:02:04.132 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:02:04.132 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:04.138 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:04.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.292+0000 7f645c710640 1 Processor -- start 2024-04-25T10:02:04.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645c710640 1 -- start start 2024-04-25T10:02:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645c710640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f645415c9a0 0x7f64540c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645c710640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f64540b1f70 0x7f645406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645c710640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64540c61d0 con 0x7f64540ce330 2024-04-25T10:02:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645c710640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64540c64b0 con 0x7f64540b1f70 2024-04-25T10:02:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645c710640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645406d4f0 con 0x7f645415c9a0 2024-04-25T10:02:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645a4ac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f645415c9a0 0x7f64540c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f6459cab640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f64540b1f70 0x7f645406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645acad640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f64540ce330 0x7f645415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:53890/0 (socket says 172.21.15.133:53890) 2024-04-25T10:02:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645acad640 1 -- 172.21.15.133:0/4237115893 learned_addr learned my addr 172.21.15.133:0/4237115893 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f64594aa640 1 -- 172.21.15.133:0/4237115893 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027005172 0 0) 0x7f64540c61d0 con 0x7f64540ce330 2024-04-25T10:02:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f64594aa640 1 -- 172.21.15.133:0/4237115893 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6440003530 con 0x7f64540ce330 2024-04-25T10:02:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645a4ac640 1 -- 172.21.15.133:0/4237115893 >> v2:172.21.15.162:3300/0 conn(0x7f64540b1f70 msgr2=0x7f645406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645a4ac640 1 --2- 172.21.15.133:0/4237115893 >> v2:172.21.15.162:3300/0 conn(0x7f64540b1f70 0x7f645406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645a4ac640 1 -- 172.21.15.133:0/4237115893 >> v1:172.21.15.162:6789/0 conn(0x7f64540ce330 legacy=0x7f645415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.296+0000 7f645a4ac640 1 -- 172.21.15.133:0/4237115893 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645406d7d0 con 0x7f645415c9a0 2024-04-25T10:02:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645a4ac640 1 --2- 172.21.15.133:0/4237115893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f645415c9a0 0x7f64540c5980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f64500097c0 tx=0x7f645001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ba259b3710e874 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f64594aa640 1 -- 172.21.15.133:0/4237115893 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500b0070 con 0x7f645415c9a0 2024-04-25T10:02:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f64594aa640 1 -- 172.21.15.133:0/4237115893 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6450004230 con 0x7f645415c9a0 2024-04-25T10:02:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f64594aa640 1 -- 172.21.15.133:0/4237115893 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450003620 con 0x7f645415c9a0 2024-04-25T10:02:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 -- 172.21.15.133:0/4237115893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f645415c9a0 msgr2=0x7f64540c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 --2- 172.21.15.133:0/4237115893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f645415c9a0 0x7f64540c5980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f64500097c0 tx=0x7f645001c440 comp rx=0 tx=0).stop 2024-04-25T10:02:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 -- 172.21.15.133:0/4237115893 shutdown_connections 2024-04-25T10:02:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 --2- 172.21.15.133:0/4237115893 >> v2:172.21.15.162:3300/0 conn(0x7f64540b1f70 0x7f645406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 --2- 172.21.15.133:0/4237115893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f645415c9a0 0x7f64540c5980 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 -- 172.21.15.133:0/4237115893 >> 172.21.15.133:0/4237115893 conn(0x7f6454067000 msgr2=0x7f645406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 -- 172.21.15.133:0/4237115893 shutdown_connections 2024-04-25T10:02:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 -- 172.21.15.133:0/4237115893 wait complete. 2024-04-25T10:02:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 Processor -- start 2024-04-25T10:02:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 -- start start 2024-04-25T10:02:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64540b1f70 0x7f645414f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64540ce330 0x7f645415de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f645415c9a0 0x7f645415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64540ceb50 con 0x7f64540b1f70 2024-04-25T10:02:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645415cd80 con 0x7f645415c9a0 2024-04-25T10:02:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645c710640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645414fcb0 con 0x7f64540ce330 2024-04-25T10:02:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645a4ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64540b1f70 0x7f645414f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645a4ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64540b1f70 0x7f645414f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46250/0 (socket says 172.21.15.133:46250) 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645a4ac640 1 -- 172.21.15.133:0/3190172099 learned_addr learned my addr 172.21.15.133:0/3190172099 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f6459cab640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64540ce330 0x7f645415de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645acad640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f645415c9a0 0x7f645415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645a4ac640 1 -- 172.21.15.133:0/3190172099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64540ce330 msgr2=0x7f645415de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645a4ac640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64540ce330 0x7f645415de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645a4ac640 1 -- 172.21.15.133:0/3190172099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f645415c9a0 msgr2=0x7f645415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645a4ac640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f645415c9a0 0x7f645415e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.300+0000 7f645a4ac640 1 -- 172.21.15.133:0/3190172099 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645413d780 con 0x7f64540b1f70 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f645a4ac640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64540b1f70 0x7f645414f770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f645001a0d0 tx=0x7f6450019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f644affd640 1 -- 172.21.15.133:0/3190172099 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500b0040 con 0x7f64540b1f70 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f644affd640 1 -- 172.21.15.133:0/3190172099 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6450004030 con 0x7f64540b1f70 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f645c710640 1 -- 172.21.15.133:0/3190172099 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645414e9b0 con 0x7f64540b1f70 2024-04-25T10:02:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f644affd640 1 -- 172.21.15.133:0/3190172099 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450003d00 con 0x7f64540b1f70 2024-04-25T10:02:04.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f645c710640 1 -- 172.21.15.133:0/3190172099 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645414ef70 con 0x7f64540b1f70 2024-04-25T10:02:04.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f645c710640 1 -- 172.21.15.133:0/3190172099 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645406bc20 con 0x7f64540b1f70 2024-04-25T10:02:04.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f644affd640 1 -- 172.21.15.133:0/3190172099 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64500be030 con 0x7f64540b1f70 2024-04-25T10:02:04.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f644affd640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6438066f60 0x7f6438069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f644affd640 1 -- 172.21.15.133:0/3190172099 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f645011ea20 con 0x7f64540b1f70 2024-04-25T10:02:04.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f6459cab640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6438066f60 0x7f6438069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.304+0000 7f6459cab640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6438066f60 0x7f6438069420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6454153140 tx=0x7f6444008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.308+0000 7f644affd640 1 -- 172.21.15.133:0/3190172099 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64500c5060 con 0x7f64540b1f70 2024-04-25T10:02:04.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.456+0000 7f645c710640 1 -- 172.21.15.133:0/3190172099 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6454153b70 con 0x7f64540b1f70 2024-04-25T10:02:04.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.456+0000 7f644affd640 1 -- 172.21.15.133:0/3190172099 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7f6450016090 con 0x7f64540b1f70 2024-04-25T10:02:04.465 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:04.465 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 -- 172.21.15.133:0/3190172099 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6438066f60 msgr2=0x7f6438069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6438066f60 0x7f6438069420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6454153140 tx=0x7f6444008040 comp rx=0 tx=0).stop 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 -- 172.21.15.133:0/3190172099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64540b1f70 msgr2=0x7f645414f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64540b1f70 0x7f645414f770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f645001a0d0 tx=0x7f6450019240 comp rx=0 tx=0).stop 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 -- 172.21.15.133:0/3190172099 shutdown_connections 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f645415c9a0 0x7f645415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64540ce330 0x7f645415de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6438066f60 0x7f6438069420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 --2- 172.21.15.133:0/3190172099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64540b1f70 0x7f645414f770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 -- 172.21.15.133:0/3190172099 >> 172.21.15.133:0/3190172099 conn(0x7f6454067000 msgr2=0x7f6454156190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 -- 172.21.15.133:0/3190172099 shutdown_connections 2024-04-25T10:02:04.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.460+0000 7f645c710640 1 -- 172.21.15.133:0/3190172099 wait complete. 2024-04-25T10:02:04.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:04.482 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb3276d640 1 Processor -- start 2024-04-25T10:02:04.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb3276d640 1 -- start start 2024-04-25T10:02:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb3276d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb2c0cc090 0x7ffb2c0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb3276d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb2c0ce250 0x7ffb2c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb3276d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb2c0b14d0 0x7ffb2c0b1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb3276d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2c0b1e80 con 0x7ffb2c0cc090 2024-04-25T10:02:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb3276d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2c06b1d0 con 0x7ffb2c0ce250 2024-04-25T10:02:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb3276d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c06b4a0 con 0x7ffb2c0b14d0 2024-04-25T10:02:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb2bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb2c0cc090 0x7ffb2c0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb2bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb2c0cc090 0x7ffb2c0b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46264/0 (socket says 172.21.15.133:46264) 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb2bfff640 1 -- 172.21.15.133:0/340839281 learned_addr learned my addr 172.21.15.133:0/340839281 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb30d0a640 1 --2- 172.21.15.133:0/340839281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb2c0b14d0 0x7ffb2c0b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb23fff640 1 --2- 172.21.15.133:0/340839281 >> v2:172.21.15.162:3300/0 conn(0x7ffb2c0ce250 0x7ffb2c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb2bfff640 1 -- 172.21.15.133:0/340839281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb2c0b14d0 msgr2=0x7ffb2c0b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb2bfff640 1 --2- 172.21.15.133:0/340839281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb2c0b14d0 0x7ffb2c0b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb2bfff640 1 -- 172.21.15.133:0/340839281 >> v2:172.21.15.162:3300/0 conn(0x7ffb2c0ce250 msgr2=0x7ffb2c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb2bfff640 1 --2- 172.21.15.133:0/340839281 >> v2:172.21.15.162:3300/0 conn(0x7ffb2c0ce250 0x7ffb2c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb2bfff640 1 -- 172.21.15.133:0/340839281 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c0affa0 con 0x7ffb2c0cc090 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.600+0000 7ffb23fff640 1 --2- 172.21.15.133:0/340839281 >> v2:172.21.15.162:3300/0 conn(0x7ffb2c0ce250 0x7ffb2c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb2bfff640 1 --2- 172.21.15.133:0/340839281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb2c0cc090 0x7ffb2c0b0a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c06bcd0 tx=0x7ffb1c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420102ae98ab5c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb2b7fe640 1 -- 172.21.15.133:0/340839281 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0b1070 con 0x7ffb2c0cc090 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb2b7fe640 1 -- 172.21.15.133:0/340839281 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c0ac620 con 0x7ffb2c0cc090 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb2b7fe640 1 -- 172.21.15.133:0/340839281 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0ac9f0 con 0x7ffb2c0cc090 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 -- 172.21.15.133:0/340839281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb2c0cc090 msgr2=0x7ffb2c0b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 --2- 172.21.15.133:0/340839281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb2c0cc090 0x7ffb2c0b0a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c06bcd0 tx=0x7ffb1c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 -- 172.21.15.133:0/340839281 shutdown_connections 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 --2- 172.21.15.133:0/340839281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb2c0b14d0 0x7ffb2c0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 --2- 172.21.15.133:0/340839281 >> v2:172.21.15.162:3300/0 conn(0x7ffb2c0ce250 0x7ffb2c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 --2- 172.21.15.133:0/340839281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb2c0cc090 0x7ffb2c0b0a80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 -- 172.21.15.133:0/340839281 >> 172.21.15.133:0/340839281 conn(0x7ffb2c0999d0 msgr2=0x7ffb2c0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 -- 172.21.15.133:0/340839281 shutdown_connections 2024-04-25T10:02:04.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 -- 172.21.15.133:0/340839281 wait complete. 2024-04-25T10:02:04.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 Processor -- start 2024-04-25T10:02:04.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 -- start start 2024-04-25T10:02:04.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb2c0b14d0 0x7ffb2c14a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb2c0cc090 0x7ffb2c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb2c0ce250 0x7ffb2c0b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0cd120 con 0x7ffb2c0cc090 2024-04-25T10:02:04.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c06b780 con 0x7ffb2c0ce250 2024-04-25T10:02:04.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb3276d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0cd2a0 con 0x7ffb2c0b14d0 2024-04-25T10:02:04.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb23fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb2c0cc090 0x7ffb2c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb23fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb2c0cc090 0x7ffb2c0b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46268/0 (socket says 172.21.15.133:46268) 2024-04-25T10:02:04.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.604+0000 7ffb23fff640 1 -- 172.21.15.133:0/2058796573 learned_addr learned my addr 172.21.15.133:0/2058796573 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb30d0a640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb2c0ce250 0x7ffb2c0b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb2bfff640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb2c0b14d0 0x7ffb2c14a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb23fff640 1 -- 172.21.15.133:0/2058796573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb2c0b14d0 msgr2=0x7ffb2c14a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb23fff640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb2c0b14d0 0x7ffb2c14a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb23fff640 1 -- 172.21.15.133:0/2058796573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb2c0ce250 msgr2=0x7ffb2c0b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb23fff640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb2c0ce250 0x7ffb2c0b59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb23fff640 1 -- 172.21.15.133:0/2058796573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c0ca2e0 con 0x7ffb2c0cc090 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb2bfff640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb2c0b14d0 0x7ffb2c14a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb23fff640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb2c0cc090 0x7ffb2c0b3e00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400c7f0 tx=0x7ffb1400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb297fa640 1 -- 172.21.15.133:0/2058796573 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14015070 con 0x7ffb2c0cc090 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb297fa640 1 -- 172.21.15.133:0/2058796573 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb14007820 con 0x7ffb2c0cc090 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb3276d640 1 -- 172.21.15.133:0/2058796573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c0ca5c0 con 0x7ffb2c0cc090 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb297fa640 1 -- 172.21.15.133:0/2058796573 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14007c10 con 0x7ffb2c0cc090 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb3276d640 1 -- 172.21.15.133:0/2058796573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2c1524b0 con 0x7ffb2c0cc090 2024-04-25T10:02:04.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb3276d640 1 -- 172.21.15.133:0/2058796573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c067c70 con 0x7ffb2c0cc090 2024-04-25T10:02:04.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb297fa640 1 -- 172.21.15.133:0/2058796573 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb14018070 con 0x7ffb2c0cc090 2024-04-25T10:02:04.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb297fa640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb0c066f60 0x7ffb0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb2bfff640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb0c066f60 0x7ffb0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb297fa640 1 -- 172.21.15.133:0/2058796573 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffb14085450 con 0x7ffb2c0cc090 2024-04-25T10:02:04.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.608+0000 7ffb2bfff640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb0c066f60 0x7ffb0c069420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0ae000 tx=0x7ffb1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.612+0000 7ffb297fa640 1 -- 172.21.15.133:0/2058796573 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb140561b0 con 0x7ffb2c0cc090 2024-04-25T10:02:04.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.760+0000 7ffb3276d640 1 -- 172.21.15.133:0/2058796573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb2c0af7b0 con 0x7ffb2c0cc090 2024-04-25T10:02:04.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.760+0000 7ffb297fa640 1 -- 172.21.15.133:0/2058796573 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb14055ab0 con 0x7ffb2c0cc090 2024-04-25T10:02:04.769 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 -- 172.21.15.133:0/2058796573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb0c066f60 msgr2=0x7ffb0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb0c066f60 0x7ffb0c069420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0ae000 tx=0x7ffb1c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 -- 172.21.15.133:0/2058796573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb2c0cc090 msgr2=0x7ffb2c0b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb2c0cc090 0x7ffb2c0b3e00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400c7f0 tx=0x7ffb1400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 -- 172.21.15.133:0/2058796573 shutdown_connections 2024-04-25T10:02:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb2c0ce250 0x7ffb2c0b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb0c066f60 0x7ffb0c069420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb2c0cc090 0x7ffb2c0b3e00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 --2- 172.21.15.133:0/2058796573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb2c0b14d0 0x7ffb2c14a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 -- 172.21.15.133:0/2058796573 >> 172.21.15.133:0/2058796573 conn(0x7ffb2c0999d0 msgr2=0x7ffb2c0caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 -- 172.21.15.133:0/2058796573 shutdown_connections 2024-04-25T10:02:04.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.764+0000 7ffb3276d640 1 -- 172.21.15.133:0/2058796573 wait complete. 2024-04-25T10:02:04.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:04.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.888+0000 7f5cbdde4640 1 Processor -- start 2024-04-25T10:02:04.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.888+0000 7f5cbdde4640 1 -- start start 2024-04-25T10:02:04.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cbdde4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5cb8142920 0x7f5cb8143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cbdde4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb8140ea0 con 0x7f5cb815d9f0 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cbdde4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb80cf280 con 0x7f5cb8140a40 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cbdde4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb806d440 con 0x7f5cb8142920 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5cb815d9f0 0x7f5cb806df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38358/0 (socket says 172.21.15.133:38358) 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb7fff640 1 -- 172.21.15.133:0/3662819871 learned_addr learned my addr 172.21.15.133:0/3662819871 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb77fe640 1 --2- 172.21.15.133:0/3662819871 >> v2:172.21.15.162:3300/0 conn(0x7f5cb8142920 0x7f5cb8143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430361493 0 0) 0x7f5cb8140ea0 con 0x7f5cb815d9f0 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca4003530 con 0x7f5cb815d9f0 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 91086234 0 0) 0x7f5ca4003530 con 0x7f5cb815d9f0 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 >> v2:172.21.15.162:3300/0 conn(0x7f5cb8142920 msgr2=0x7f5cb8143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 --2- 172.21.15.133:0/3662819871 >> v2:172.21.15.162:3300/0 conn(0x7f5cb8142920 0x7f5cb8143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 >> v1:172.21.15.162:6789/0 conn(0x7f5cb8140a40 legacy=0x7f5cb80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb806d720 con 0x7f5cb815d9f0 2024-04-25T10:02:04.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5cb0001a20 con 0x7f5cb815d9f0 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 >> v1:172.21.15.133:6789/0 conn(0x7f5cb815d9f0 legacy=0x7f5cb806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 --2- 172.21.15.133:0/3662819871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ca40039c0 0x7f5ca4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 --2- 172.21.15.133:0/3662819871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ca4004810 0x7f5ca4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 --2- 172.21.15.133:0/3662819871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ca40051d0 0x7f5ca400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb806d440 con 0x7f5ca40039c0 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca400c170 con 0x7f5ca40051d0 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca400c370 con 0x7f5ca4004810 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb77fe640 1 --2- 172.21.15.133:0/3662819871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ca40039c0 0x7f5ca4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb7fff640 1 --2- 172.21.15.133:0/3662819871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ca40051d0 0x7f5ca400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb77fe640 1 -- 172.21.15.133:0/3662819871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ca4004810 msgr2=0x7f5ca4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb77fe640 1 --2- 172.21.15.133:0/3662819871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ca4004810 0x7f5ca4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb77fe640 1 -- 172.21.15.133:0/3662819871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ca40051d0 msgr2=0x7f5ca400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb77fe640 1 --2- 172.21.15.133:0/3662819871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ca40051d0 0x7f5ca400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb77fe640 1 -- 172.21.15.133:0/3662819871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca4003530 con 0x7f5ca40039c0 2024-04-25T10:02:04.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.892+0000 7f5cb7fff640 1 -- 172.21.15.133:0/3662819871 reap_dead start 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cb77fe640 1 --2- 172.21.15.133:0/3662819871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ca40039c0 0x7f5ca4003de0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009ae0 tx=0x7f5ca801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449e932bfb275bb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80096b0 con 0x7f5ca40039c0 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca80a6450 con 0x7f5ca40039c0 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cb6ffd640 1 -- 172.21.15.133:0/3662819871 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80a6840 con 0x7f5ca40039c0 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3662819871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ca40039c0 msgr2=0x7f5ca4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 --2- 172.21.15.133:0/3662819871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ca40039c0 0x7f5ca4003de0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009ae0 tx=0x7f5ca801ca70 comp rx=0 tx=0).stop 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3662819871 shutdown_connections 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 --2- 172.21.15.133:0/3662819871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ca40039c0 0x7f5ca4003de0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3662819871 >> 172.21.15.133:0/3662819871 conn(0x7f5cb8066f70 msgr2=0x7f5cb806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3662819871 shutdown_connections 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3662819871 wait complete. 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 Processor -- start 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 -- start start 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ca40039c0 0x7f5cb8155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb80b1570 0x7f5cb813b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cb813f810 0x7f5cb813fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb815df50 con 0x7f5cb80b1570 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb815ddd0 con 0x7f5ca40039c0 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb815e0d0 con 0x7f5cb813f810 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5caffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb80b1570 0x7f5cb813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5caffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb80b1570 0x7f5cb813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46290/0 (socket says 172.21.15.133:46290) 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5caffff640 1 -- 172.21.15.133:0/3606999193 learned_addr learned my addr 172.21.15.133:0/3606999193 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cb7fff640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cb813f810 0x7f5cb813fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5caffff640 1 -- 172.21.15.133:0/3606999193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cb813f810 msgr2=0x7f5cb813fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5caffff640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cb813f810 0x7f5cb813fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5caffff640 1 -- 172.21.15.133:0/3606999193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ca40039c0 msgr2=0x7f5cb8155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5caffff640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ca40039c0 0x7f5cb8155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5caffff640 1 -- 172.21.15.133:0/3606999193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb8157a00 con 0x7f5cb80b1570 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5caffff640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb80b1570 0x7f5cb813b160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000c530 tx=0x7f5ca000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cb4ff9640 1 -- 172.21.15.133:0/3606999193 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca00052d0 con 0x7f5cb80b1570 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cb4ff9640 1 -- 172.21.15.133:0/3606999193 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca0005470 con 0x7f5cb80b1570 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3606999193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb813e230 con 0x7f5cb80b1570 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cb4ff9640 1 -- 172.21.15.133:0/3606999193 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0012850 con 0x7f5cb80b1570 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.896+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3606999193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb80b0a90 con 0x7f5cb80b1570 2024-04-25T10:02:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.900+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3606999193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb8068e80 con 0x7f5cb80b1570 2024-04-25T10:02:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.900+0000 7f5cb4ff9640 1 -- 172.21.15.133:0/3606999193 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ca00129f0 con 0x7f5cb80b1570 2024-04-25T10:02:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.900+0000 7f5cb4ff9640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c98066f60 0x7f5c98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.900+0000 7f5cb4ff9640 1 -- 172.21.15.133:0/3606999193 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5ca0084fb0 con 0x7f5cb80b1570 2024-04-25T10:02:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.900+0000 7f5cb77fe640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c98066f60 0x7f5c98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.900+0000 7f5cb77fe640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c98066f60 0x7f5c98069420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80ad000 tx=0x7f5ca80a9040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:04.904+0000 7f5cb4ff9640 1 -- 172.21.15.133:0/3606999193 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5cb813e230 con 0x7f5cb80b1570 2024-04-25T10:02:05.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.048+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3606999193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5cb80afa70 con 0x7f5cb80b1570 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cb4ff9640 1 -- 172.21.15.133:0/3606999193 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ca0055620 con 0x7f5cb80b1570 2024-04-25T10:02:05.057 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:05.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3606999193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c98066f60 msgr2=0x7f5c98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c98066f60 0x7f5c98069420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80ad000 tx=0x7f5ca80a9040 comp rx=0 tx=0).stop 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3606999193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb80b1570 msgr2=0x7f5cb813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb80b1570 0x7f5cb813b160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000c530 tx=0x7f5ca000c9f0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3606999193 shutdown_connections 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5cb813f810 0x7f5cb813fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c98066f60 0x7f5c98069420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb80b1570 0x7f5cb813b160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 --2- 172.21.15.133:0/3606999193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ca40039c0 0x7f5cb8155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3606999193 >> 172.21.15.133:0/3606999193 conn(0x7f5cb8066f70 msgr2=0x7f5cb80b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3606999193 shutdown_connections 2024-04-25T10:02:05.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.052+0000 7f5cbdde4640 1 -- 172.21.15.133:0/3606999193 wait complete. 2024-04-25T10:02:05.075 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:05.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f918a9cd640 1 Processor -- start 2024-04-25T10:02:05.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f918a9cd640 1 -- start start 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f918a9cd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9184143ea0 0x7f91840cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f918a9cd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f918406eac0 0x7f91840b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f918a9cd640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f918415ac50 con 0x7f91840b2150 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f918a9cd640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91840cd180 con 0x7f918406eac0 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f918a9cd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918415aed0 con 0x7f9184143ea0 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f9188f6a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f918406eac0 0x7f91840b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f9188f6a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f918406eac0 0x7f91840b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46298/0 (socket says 172.21.15.133:46298) 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f9188f6a640 1 -- 172.21.15.133:0/1754537735 learned_addr learned my addr 172.21.15.133:0/1754537735 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f9183fff640 1 --2- 172.21.15.133:0/1754537735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9184143ea0 0x7f91840cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f9188f6a640 1 -- 172.21.15.133:0/1754537735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9184143ea0 msgr2=0x7f91840cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f9188f6a640 1 --2- 172.21.15.133:0/1754537735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9184143ea0 0x7f91840cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f9188f6a640 1 -- 172.21.15.133:0/1754537735 >> v1:172.21.15.162:6789/0 conn(0x7f91840b2150 legacy=0x7f918406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.192+0000 7f9188f6a640 1 -- 172.21.15.133:0/1754537735 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918415b1b0 con 0x7f918406eac0 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f9188f6a640 1 --2- 172.21.15.133:0/1754537735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f918406eac0 0x7f91840b06d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f917c009e50 tx=0x7f917c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4d0401d5528d1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f91837fe640 1 -- 172.21.15.133:0/1754537735 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0a5810 con 0x7f918406eac0 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f91837fe640 1 -- 172.21.15.133:0/1754537735 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c0a59b0 con 0x7f918406eac0 2024-04-25T10:02:05.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f91837fe640 1 -- 172.21.15.133:0/1754537735 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0a5d80 con 0x7f918406eac0 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 -- 172.21.15.133:0/1754537735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f918406eac0 msgr2=0x7f91840b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 --2- 172.21.15.133:0/1754537735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f918406eac0 0x7f91840b06d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f917c009e50 tx=0x7f917c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 -- 172.21.15.133:0/1754537735 shutdown_connections 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 --2- 172.21.15.133:0/1754537735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f918406eac0 0x7f91840b06d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 --2- 172.21.15.133:0/1754537735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9184143ea0 0x7f91840cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 -- 172.21.15.133:0/1754537735 >> 172.21.15.133:0/1754537735 conn(0x7f9184066f70 msgr2=0x7f918406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 -- 172.21.15.133:0/1754537735 shutdown_connections 2024-04-25T10:02:05.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 -- 172.21.15.133:0/1754537735 wait complete. 2024-04-25T10:02:05.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 Processor -- start 2024-04-25T10:02:05.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 -- start start 2024-04-25T10:02:05.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f918406eac0 0x7f918415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91840b2150 0x7f918413d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9184143ea0 0x7f918413dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9184167550 con 0x7f9184143ea0 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841673d0 con 0x7f91840b2150 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91841676d0 con 0x7f918406eac0 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f9188f6a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9184143ea0 0x7f918413dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f9188f6a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9184143ea0 0x7f918413dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46300/0 (socket says 172.21.15.133:46300) 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f9188f6a640 1 -- 172.21.15.133:0/2199854109 learned_addr learned my addr 172.21.15.133:0/2199854109 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f917bfff640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91840b2150 0x7f918413d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f9183fff640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f918406eac0 0x7f918415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f9188f6a640 1 -- 172.21.15.133:0/2199854109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f918406eac0 msgr2=0x7f918415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f9188f6a640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f918406eac0 0x7f918415eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f9188f6a640 1 -- 172.21.15.133:0/2199854109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91840b2150 msgr2=0x7f918413d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f9188f6a640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91840b2150 0x7f918413d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f9188f6a640 1 -- 172.21.15.133:0/2199854109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91841425d0 con 0x7f9184143ea0 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f917bfff640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91840b2150 0x7f918413d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f9188f6a640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9184143ea0 0x7f918413dc40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f917c0af040 tx=0x7f917c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f91817fa640 1 -- 172.21.15.133:0/2199854109 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0bc020 con 0x7f9184143ea0 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f91817fa640 1 -- 172.21.15.133:0/2199854109 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c0b8670 con 0x7f9184143ea0 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 -- 172.21.15.133:0/2199854109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9184141530 con 0x7f9184143ea0 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f91817fa640 1 -- 172.21.15.133:0/2199854109 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0b8a40 con 0x7f9184143ea0 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.196+0000 7f918a9cd640 1 -- 172.21.15.133:0/2199854109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9184141dd0 con 0x7f9184143ea0 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.200+0000 7f918a9cd640 1 -- 172.21.15.133:0/2199854109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9184141920 con 0x7f9184143ea0 2024-04-25T10:02:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.200+0000 7f91817fa640 1 -- 172.21.15.133:0/2199854109 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f917c0bf020 con 0x7f9184143ea0 2024-04-25T10:02:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.200+0000 7f91817fa640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9164066f60 0x7f9164069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.200+0000 7f91817fa640 1 -- 172.21.15.133:0/2199854109 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f917c120940 con 0x7f9184143ea0 2024-04-25T10:02:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.200+0000 7f9183fff640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9164066f60 0x7f9164069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.200+0000 7f9183fff640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9164066f60 0x7f9164069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f917400b930 tx=0x7f9174008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.204+0000 7f91817fa640 1 -- 172.21.15.133:0/2199854109 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f917c0c6030 con 0x7f9184143ea0 2024-04-25T10:02:05.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 -- 172.21.15.133:0/2199854109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91840cebb0 con 0x7f9184143ea0 2024-04-25T10:02:05.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f91817fa640 1 -- 172.21.15.133:0/2199854109 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f917c016620 con 0x7f9184143ea0 2024-04-25T10:02:05.367 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 -- 172.21.15.133:0/2199854109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9164066f60 msgr2=0x7f9164069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9164066f60 0x7f9164069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f917400b930 tx=0x7f9174008040 comp rx=0 tx=0).stop 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 -- 172.21.15.133:0/2199854109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9184143ea0 msgr2=0x7f918413dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9184143ea0 0x7f918413dc40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f917c0af040 tx=0x7f917c0197d0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 -- 172.21.15.133:0/2199854109 shutdown_connections 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9164066f60 0x7f9164069420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9184143ea0 0x7f918413dc40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91840b2150 0x7f918413d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 --2- 172.21.15.133:0/2199854109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f918406eac0 0x7f918415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 -- 172.21.15.133:0/2199854109 >> 172.21.15.133:0/2199854109 conn(0x7f9184066f70 msgr2=0x7f91840b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 -- 172.21.15.133:0/2199854109 shutdown_connections 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.360+0000 7f918a9cd640 1 -- 172.21.15.133:0/2199854109 wait complete. 2024-04-25T10:02:05.385 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:05.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a2226e640 1 Processor -- start 2024-04-25T10:02:05.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a2226e640 1 -- start start 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a2226e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c0cd820 0x7f3a1c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a2226e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1c0ce2e0 con 0x7f3a1c15ced0 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a2226e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1c06d4b0 con 0x7f3a1c145170 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a2226e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c06db00 con 0x7f3a1c0cd820 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a2080b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3a1c15ced0 0x7f3a1c0b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38372/0 (socket says 172.21.15.133:38372) 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a2080b640 1 -- 172.21.15.133:0/1037082563 learned_addr learned my addr 172.21.15.133:0/1037082563 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1b7fe640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c0cd820 0x7f3a1c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241030937 0 0) 0x7f3a1c0ce2e0 con 0x7f3a1c15ced0 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a04003530 con 0x7f3a1c15ced0 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 542563857 0 0) 0x7f3a04003530 con 0x7f3a1c15ced0 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c0cd820 msgr2=0x7f3a1c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c0cd820 0x7f3a1c0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 >> v1:172.21.15.162:6789/0 conn(0x7f3a1c145170 legacy=0x7f3a1c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c06d790 con 0x7f3a1c15ced0 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f3a14001a00 con 0x7f3a1c15ced0 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 >> v1:172.21.15.133:6789/0 conn(0x7f3a1c15ced0 legacy=0x7f3a1c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 0x7f3a04003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a04004810 0x7f3a04004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c0cd820 0x7f3a040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c06db00 con 0x7f3a040039f0 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0400ba80 con 0x7f3a04004810 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0400bc80 con 0x7f3a1c0cd820 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1bfff640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 0x7f3a04003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1b7fe640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a04004810 0x7f3a04004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a2080b640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c0cd820 0x7f3a040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1bfff640 1 -- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c0cd820 msgr2=0x7f3a040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1bfff640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c0cd820 0x7f3a040051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1bfff640 1 -- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a04004810 msgr2=0x7f3a04004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1bfff640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a04004810 0x7f3a04004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1bfff640 1 -- 172.21.15.133:0/1037082563 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a04003530 con 0x7f3a040039f0 2024-04-25T10:02:05.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.496+0000 7f3a1b7fe640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a04004810 0x7f3a04004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a1bfff640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 0x7f3a04003e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c001750 tx=0x7f3a0c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19905504bf3a7e52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0013a0 con 0x7f3a040039f0 2024-04-25T10:02:05.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c0052f0 con 0x7f3a040039f0 2024-04-25T10:02:05.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a1affd640 1 -- 172.21.15.133:0/1037082563 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c003840 con 0x7f3a040039f0 2024-04-25T10:02:05.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 -- 172.21.15.133:0/1037082563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 msgr2=0x7f3a04003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 --2- 172.21.15.133:0/1037082563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 0x7f3a04003e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c001750 tx=0x7f3a0c01c3c0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2080b640 1 -- 172.21.15.133:0/1037082563 reap_dead start 2024-04-25T10:02:05.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 -- 172.21.15.133:0/1037082563 shutdown_connections 2024-04-25T10:02:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 -- 172.21.15.133:0/1037082563 >> 172.21.15.133:0/1037082563 conn(0x7f3a1c066f70 msgr2=0x7f3a1c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 -- 172.21.15.133:0/1037082563 shutdown_connections 2024-04-25T10:02:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 -- 172.21.15.133:0/1037082563 wait complete. 2024-04-25T10:02:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 Processor -- start 2024-04-25T10:02:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 -- start start 2024-04-25T10:02:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 0x7f3a1c13bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c1f4290 0x7f3a1c1f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a1c161290 0x7f3a1c161710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c145550 con 0x7f3a040039f0 2024-04-25T10:02:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b1fe0 con 0x7f3a1c161290 2024-04-25T10:02:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a2226e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0cdc00 con 0x7f3a1c1f4290 2024-04-25T10:02:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a1bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 0x7f3a1c13bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a1bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 0x7f3a1c13bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46314/0 (socket says 172.21.15.133:46314) 2024-04-25T10:02:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.500+0000 7f3a1bfff640 1 -- 172.21.15.133:0/1389969973 learned_addr learned my addr 172.21.15.133:0/1389969973 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a1bfff640 1 -- 172.21.15.133:0/1389969973 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c1f4290 msgr2=0x7f3a1c1f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a1bfff640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c1f4290 0x7f3a1c1f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a1bfff640 1 -- 172.21.15.133:0/1389969973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a1c161290 msgr2=0x7f3a1c161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a2080b640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a1c161290 0x7f3a1c161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a1bfff640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a1c161290 0x7f3a1c161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a1bfff640 1 -- 172.21.15.133:0/1389969973 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c164610 con 0x7f3a040039f0 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a1bfff640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 0x7f3a1c13bf50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c005040 tx=0x7f3a04005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a18ff9640 1 -- 172.21.15.133:0/1389969973 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0013a0 con 0x7f3a040039f0 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a18ff9640 1 -- 172.21.15.133:0/1389969973 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c0bb070 con 0x7f3a040039f0 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a2226e640 1 -- 172.21.15.133:0/1389969973 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c1648f0 con 0x7f3a040039f0 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a18ff9640 1 -- 172.21.15.133:0/1389969973 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c007a40 con 0x7f3a040039f0 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a2226e640 1 -- 172.21.15.133:0/1389969973 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c164eb0 con 0x7f3a040039f0 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a2226e640 1 -- 172.21.15.133:0/1389969973 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c068e80 con 0x7f3a040039f0 2024-04-25T10:02:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a18ff9640 1 -- 172.21.15.133:0/1389969973 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a0c0b4040 con 0x7f3a040039f0 2024-04-25T10:02:05.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a18ff9640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a08066f90 0x7f3a08069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a18ff9640 1 -- 172.21.15.133:0/1389969973 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3a0c11f430 con 0x7f3a040039f0 2024-04-25T10:02:05.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a1b7fe640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a08066f90 0x7f3a08069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.504+0000 7f3a1b7fe640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a08066f90 0x7f3a08069450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c161df0 tx=0x7f3a10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.508+0000 7f3a18ff9640 1 -- 172.21.15.133:0/1389969973 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a0c0160d0 con 0x7f3a040039f0 2024-04-25T10:02:05.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.652+0000 7f3a2226e640 1 -- 172.21.15.133:0/1389969973 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3a1c1426f0 con 0x7f3a040039f0 2024-04-25T10:02:05.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.652+0000 7f3a18ff9640 1 -- 172.21.15.133:0/1389969973 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3a0c0f0220 con 0x7f3a040039f0 2024-04-25T10:02:05.660 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:05.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 -- 172.21.15.133:0/1389969973 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a08066f90 msgr2=0x7f3a08069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a08066f90 0x7f3a08069450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c161df0 tx=0x7f3a10008040 comp rx=0 tx=0).stop 2024-04-25T10:02:05.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 -- 172.21.15.133:0/1389969973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 msgr2=0x7f3a1c13bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 0x7f3a1c13bf50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c005040 tx=0x7f3a04005840 comp rx=0 tx=0).stop 2024-04-25T10:02:05.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 -- 172.21.15.133:0/1389969973 shutdown_connections 2024-04-25T10:02:05.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a1c161290 0x7f3a1c161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a1c1f4290 0x7f3a1c1f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a08066f90 0x7f3a08069450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 --2- 172.21.15.133:0/1389969973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a040039f0 0x7f3a1c13bf50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 -- 172.21.15.133:0/1389969973 >> 172.21.15.133:0/1389969973 conn(0x7f3a1c066f70 msgr2=0x7f3a1c140f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 -- 172.21.15.133:0/1389969973 shutdown_connections 2024-04-25T10:02:05.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.656+0000 7f3a2226e640 1 -- 172.21.15.133:0/1389969973 wait complete. 2024-04-25T10:02:05.678 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:05.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25baee5640 1 Processor -- start 2024-04-25T10:02:05.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25baee5640 1 -- start start 2024-04-25T10:02:05.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25baee5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f25b415d9a0 0x7f25b40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25baee5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f25b40ce600 0x7f25b40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25baee5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b415ebd0 con 0x7f25b40d1920 2024-04-25T10:02:05.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25baee5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b415eeb0 con 0x7f25b415d9a0 2024-04-25T10:02:05.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25baee5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b40d0330 con 0x7f25b40ce600 2024-04-25T10:02:05.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25b8c81640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f25b415d9a0 0x7f25b40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25b9482640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f25b40d1920 0x7f25b40d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38386/0 (socket says 172.21.15.133:38386) 2024-04-25T10:02:05.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25b9482640 1 -- 172.21.15.133:0/2427972580 learned_addr learned my addr 172.21.15.133:0/2427972580 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:05.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25b3fff640 1 --2- 172.21.15.133:0/2427972580 >> v2:172.21.15.162:3300/0 conn(0x7f25b40ce600 0x7f25b40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25b8c81640 1 -- 172.21.15.133:0/2427972580 >> v2:172.21.15.162:3300/0 conn(0x7f25b40ce600 msgr2=0x7f25b40cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25b8c81640 1 --2- 172.21.15.133:0/2427972580 >> v2:172.21.15.162:3300/0 conn(0x7f25b40ce600 0x7f25b40cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25b8c81640 1 -- 172.21.15.133:0/2427972580 >> v1:172.21.15.133:6789/0 conn(0x7f25b40d1920 legacy=0x7f25b40d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25b8c81640 1 -- 172.21.15.133:0/2427972580 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b40d0610 con 0x7f25b415d9a0 2024-04-25T10:02:05.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.792+0000 7f25b3fff640 1 --2- 172.21.15.133:0/2427972580 >> v2:172.21.15.162:3300/0 conn(0x7f25b40ce600 0x7f25b40cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25b8c81640 1 --2- 172.21.15.133:0/2427972580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b415d9a0 0x7f25b40d0b30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f25b40ce080 tx=0x7f25a801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c64fabec1814b86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25b37fe640 1 -- 172.21.15.133:0/2427972580 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80b0070 con 0x7f25b415d9a0 2024-04-25T10:02:05.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25b37fe640 1 -- 172.21.15.133:0/2427972580 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a80ab600 con 0x7f25b415d9a0 2024-04-25T10:02:05.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25b37fe640 1 -- 172.21.15.133:0/2427972580 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80ab9d0 con 0x7f25b415d9a0 2024-04-25T10:02:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25baee5640 1 -- 172.21.15.133:0/2427972580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b415d9a0 msgr2=0x7f25b40d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25baee5640 1 --2- 172.21.15.133:0/2427972580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b415d9a0 0x7f25b40d0b30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f25b40ce080 tx=0x7f25a801c3f0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25baee5640 1 -- 172.21.15.133:0/2427972580 shutdown_connections 2024-04-25T10:02:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25baee5640 1 --2- 172.21.15.133:0/2427972580 >> v2:172.21.15.162:3300/0 conn(0x7f25b40ce600 0x7f25b40cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25baee5640 1 --2- 172.21.15.133:0/2427972580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b415d9a0 0x7f25b40d0b30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25baee5640 1 -- 172.21.15.133:0/2427972580 >> 172.21.15.133:0/2427972580 conn(0x7f25b406bf80 msgr2=0x7f25b406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25baee5640 1 -- 172.21.15.133:0/2427972580 shutdown_connections 2024-04-25T10:02:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.796+0000 7f25baee5640 1 -- 172.21.15.133:0/2427972580 wait complete. 2024-04-25T10:02:05.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25baee5640 1 Processor -- start 2024-04-25T10:02:05.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25baee5640 1 -- start start 2024-04-25T10:02:05.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25baee5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b40ce600 0x7f25b415cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25baee5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25b40d1920 0x7f25b41566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25baee5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25b415d9a0 0x7f25b415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25baee5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40b9820 con 0x7f25b40ce600 2024-04-25T10:02:05.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25baee5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40b96a0 con 0x7f25b415d9a0 2024-04-25T10:02:05.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25baee5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b415e590 con 0x7f25b40d1920 2024-04-25T10:02:05.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b8c81640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b40ce600 0x7f25b415cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b8c81640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b40ce600 0x7f25b415cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46320/0 (socket says 172.21.15.133:46320) 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b8c81640 1 -- 172.21.15.133:0/1174165707 learned_addr learned my addr 172.21.15.133:0/1174165707 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b8c81640 1 -- 172.21.15.133:0/1174165707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25b40d1920 msgr2=0x7f25b41566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b8c81640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25b40d1920 0x7f25b41566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b8c81640 1 -- 172.21.15.133:0/1174165707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25b415d9a0 msgr2=0x7f25b415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b8c81640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25b415d9a0 0x7f25b415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b8c81640 1 -- 172.21.15.133:0/1174165707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b415d310 con 0x7f25b40ce600 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b8c81640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b40ce600 0x7f25b415cdd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25a8019000 tx=0x7f25a8019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b17fa640 1 -- 172.21.15.133:0/1174165707 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80b0040 con 0x7f25b40ce600 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b17fa640 1 -- 172.21.15.133:0/1174165707 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a80b9070 con 0x7f25b40ce600 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25baee5640 1 -- 172.21.15.133:0/1174165707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b4156c30 con 0x7f25b40ce600 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b17fa640 1 -- 172.21.15.133:0/1174165707 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80b4830 con 0x7f25b40ce600 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25baee5640 1 -- 172.21.15.133:0/1174165707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b40c9a10 con 0x7f25b40ce600 2024-04-25T10:02:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25baee5640 1 -- 172.21.15.133:0/1174165707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b4068e80 con 0x7f25b40ce600 2024-04-25T10:02:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b17fa640 1 -- 172.21.15.133:0/1174165707 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25a80ab4c0 con 0x7f25b40ce600 2024-04-25T10:02:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b17fa640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2598066f60 0x7f2598069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b17fa640 1 -- 172.21.15.133:0/1174165707 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f25a811f1c0 con 0x7f25b40ce600 2024-04-25T10:02:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.800+0000 7f25b3fff640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2598066f60 0x7f2598069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.804+0000 7f25b3fff640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2598066f60 0x7f2598069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f25b415b7d0 tx=0x7f25a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.808+0000 7f25b17fa640 1 -- 172.21.15.133:0/1174165707 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25b415d310 con 0x7f25b40ce600 2024-04-25T10:02:05.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.976+0000 7f25baee5640 1 -- 172.21.15.133:0/1174165707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f25b40c6ed0 con 0x7f25b40ce600 2024-04-25T10:02:05.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.976+0000 7f25b17fa640 1 -- 172.21.15.133:0/1174165707 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f25a8016070 con 0x7f25b40ce600 2024-04-25T10:02:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:02:05.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 -- 172.21.15.133:0/1174165707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2598066f60 msgr2=0x7f2598069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2598066f60 0x7f2598069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f25b415b7d0 tx=0x7f25a0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 -- 172.21.15.133:0/1174165707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b40ce600 msgr2=0x7f25b415cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b40ce600 0x7f25b415cdd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25a8019000 tx=0x7f25a8019190 comp rx=0 tx=0).stop 2024-04-25T10:02:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 -- 172.21.15.133:0/1174165707 shutdown_connections 2024-04-25T10:02:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25b415d9a0 0x7f25b415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25b40d1920 0x7f25b41566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2598066f60 0x7f2598069420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 --2- 172.21.15.133:0/1174165707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b40ce600 0x7f25b415cdd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 -- 172.21.15.133:0/1174165707 >> 172.21.15.133:0/1174165707 conn(0x7f25b406bf80 msgr2=0x7f25b40b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 -- 172.21.15.133:0/1174165707 shutdown_connections 2024-04-25T10:02:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:05.980+0000 7f25baee5640 1 -- 172.21.15.133:0/1174165707 wait complete. 2024-04-25T10:02:06.003 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:02:06.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:06.003 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:06.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[34022]: fuse finished with error 0 and tester_r 0 2024-04-25T10:02:06.025 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:12.127 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:02:12.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:12.127 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T10:02:12.149 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.316+0000 7fe7b25bd640 1 Processor -- start 2024-04-25T10:02:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.316+0000 7fe7b25bd640 1 -- start start 2024-04-25T10:02:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b25bd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac15b9c0 0x7fe7ac072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b25bd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe7ac0704a0 0x7fe7ac0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b25bd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ac15bda0 con 0x7fe7ac0ccf90 2024-04-25T10:02:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b25bd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ac0cd450 con 0x7fe7ac0704a0 2024-04-25T10:02:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b25bd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0cd730 con 0x7fe7ac15b9c0 2024-04-25T10:02:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7a37fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe7ac0ccf90 0x7fe7ac06fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42904/0 (socket says 172.21.15.133:42904) 2024-04-25T10:02:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7a37fe640 1 -- 172.21.15.133:0/2679287229 learned_addr learned my addr 172.21.15.133:0/2679287229 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b0b5a640 1 --2- 172.21.15.133:0/2679287229 >> v2:172.21.15.162:3300/0 conn(0x7fe7ac0704a0 0x7fe7ac0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7abfff640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac15b9c0 0x7fe7ac072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081017345 0 0) 0x7fe7ac15bda0 con 0x7fe7ac0ccf90 2024-04-25T10:02:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe798003530 con 0x7fe7ac0ccf90 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3848732528 0 0) 0x7fe798003530 con 0x7fe7ac0ccf90 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac15b9c0 msgr2=0x7fe7ac072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac15b9c0 0x7fe7ac072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 >> v2:172.21.15.162:3300/0 conn(0x7fe7ac0704a0 msgr2=0x7fe7ac0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 --2- 172.21.15.133:0/2679287229 >> v2:172.21.15.162:3300/0 conn(0x7fe7ac0704a0 0x7fe7ac0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac0cda10 con 0x7fe7ac0ccf90 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe794001680 con 0x7fe7ac0ccf90 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 >> v1:172.21.15.133:6789/0 conn(0x7fe7ac0ccf90 legacy=0x7fe7ac06fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac15b9c0 0x7fe7980039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe798003f40 0x7fe798004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 0x7fe7980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0cd450 con 0x7fe798004d50 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0cd730 con 0x7fe798003f40 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79800ba20 con 0x7fe7ac15b9c0 2024-04-25T10:02:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b0b5a640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 0x7fe7980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7abfff640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac15b9c0 0x7fe7980039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7a37fe640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe798003f40 0x7fe798004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b0b5a640 1 -- 172.21.15.133:0/2679287229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac15b9c0 msgr2=0x7fe7980039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b0b5a640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac15b9c0 0x7fe7980039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b0b5a640 1 -- 172.21.15.133:0/2679287229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe798003f40 msgr2=0x7fe798004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b0b5a640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe798003f40 0x7fe798004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b0b5a640 1 -- 172.21.15.133:0/2679287229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798003530 con 0x7fe798004d50 2024-04-25T10:02:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7b0b5a640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 0x7fe7980051d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0b5b90 tx=0x7fe7a401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c15c9dc64a5b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40b2070 con 0x7fe798004d50 2024-04-25T10:02:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a40a5c30 con 0x7fe798004d50 2024-04-25T10:02:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.320+0000 7fe7ab7fe640 1 -- 172.21.15.133:0/2679287229 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4009c10 con 0x7fe798004d50 2024-04-25T10:02:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 -- 172.21.15.133:0/2679287229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 msgr2=0x7fe7980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 --2- 172.21.15.133:0/2679287229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 0x7fe7980051d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0b5b90 tx=0x7fe7a401cc20 comp rx=0 tx=0).stop 2024-04-25T10:02:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b0b5a640 1 -- 172.21.15.133:0/2679287229 reap_dead start 2024-04-25T10:02:12.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 -- 172.21.15.133:0/2679287229 shutdown_connections 2024-04-25T10:02:12.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 -- 172.21.15.133:0/2679287229 >> 172.21.15.133:0/2679287229 conn(0x7fe7ac0b2370 msgr2=0x7fe7ac0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 -- 172.21.15.133:0/2679287229 shutdown_connections 2024-04-25T10:02:12.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 -- 172.21.15.133:0/2679287229 wait complete. 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 Processor -- start 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 -- start start 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 0x7fe7ac1451f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7ac14ec40 0x7fe7ac14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac0b4690 0x7fe7ac0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac072e30 con 0x7fe798004d50 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac072cb0 con 0x7fe7ac14ec40 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac072fb0 con 0x7fe7ac0b4690 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 0x7fe7ac1451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 0x7fe7ac1451f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49492/0 (socket says 172.21.15.133:49492) 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7abfff640 1 -- 172.21.15.133:0/1786486541 learned_addr learned my addr 172.21.15.133:0/1786486541 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b0b5a640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac0b4690 0x7fe7ac0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7a37fe640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7ac14ec40 0x7fe7ac14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7abfff640 1 -- 172.21.15.133:0/1786486541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac0b4690 msgr2=0x7fe7ac0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7abfff640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac0b4690 0x7fe7ac0b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7abfff640 1 -- 172.21.15.133:0/1786486541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7ac14ec40 msgr2=0x7fe7ac14f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7abfff640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7ac14ec40 0x7fe7ac14f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7abfff640 1 -- 172.21.15.133:0/1786486541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac0b5050 con 0x7fe798004d50 2024-04-25T10:02:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7abfff640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 0x7fe7ac1451f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe79c011610 tx=0x7fe79c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7a97fa640 1 -- 172.21.15.133:0/1786486541 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c015870 con 0x7fe798004d50 2024-04-25T10:02:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7a97fa640 1 -- 172.21.15.133:0/1786486541 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c015a10 con 0x7fe798004d50 2024-04-25T10:02:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 -- 172.21.15.133:0/1786486541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ac0c9a90 con 0x7fe798004d50 2024-04-25T10:02:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7a97fa640 1 -- 172.21.15.133:0/1786486541 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c015de0 con 0x7fe798004d50 2024-04-25T10:02:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.324+0000 7fe7b25bd640 1 -- 172.21.15.133:0/1786486541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac0ca050 con 0x7fe798004d50 2024-04-25T10:02:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.328+0000 7fe7b25bd640 1 -- 172.21.15.133:0/1786486541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ac14f600 con 0x7fe798004d50 2024-04-25T10:02:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.328+0000 7fe7a97fa640 1 -- 172.21.15.133:0/1786486541 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe79c01d400 con 0x7fe798004d50 2024-04-25T10:02:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.328+0000 7fe7a97fa640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe78c066f90 0x7fe78c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.328+0000 7fe7a97fa640 1 -- 172.21.15.133:0/1786486541 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe79c085c80 con 0x7fe798004d50 2024-04-25T10:02:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.328+0000 7fe7a37fe640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe78c066f90 0x7fe78c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.328+0000 7fe7a37fe640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe78c066f90 0x7fe78c069450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac15efa0 tx=0x7fe7940015c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.332+0000 7fe7a97fa640 1 -- 172.21.15.133:0/1786486541 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe79c019030 con 0x7fe798004d50 2024-04-25T10:02:12.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.488+0000 7fe7b25bd640 1 -- 172.21.15.133:0/1786486541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7ac070ba0 con 0x7fe798004d50 2024-04-25T10:02:12.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.492+0000 7fe7a97fa640 1 -- 172.21.15.133:0/1786486541 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7fe79c0568b0 con 0x7fe798004d50 2024-04-25T10:02:12.500 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:12.500 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:12.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 -- 172.21.15.133:0/1786486541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe78c066f90 msgr2=0x7fe78c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe78c066f90 0x7fe78c069450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac15efa0 tx=0x7fe7940015c0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 -- 172.21.15.133:0/1786486541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 msgr2=0x7fe7ac1451f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 0x7fe7ac1451f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe79c011610 tx=0x7fe79c011ad0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 -- 172.21.15.133:0/1786486541 shutdown_connections 2024-04-25T10:02:12.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7ac0b4690 0x7fe7ac0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7ac14ec40 0x7fe7ac14f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe78c066f90 0x7fe78c069450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 --2- 172.21.15.133:0/1786486541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe798004d50 0x7fe7ac1451f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 -- 172.21.15.133:0/1786486541 >> 172.21.15.133:0/1786486541 conn(0x7fe7ac0b2370 msgr2=0x7fe7ac0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 -- 172.21.15.133:0/1786486541 shutdown_connections 2024-04-25T10:02:12.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.496+0000 7fe7b25bd640 1 -- 172.21.15.133:0/1786486541 wait complete. 2024-04-25T10:02:12.518 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.628+0000 7fae7b834640 1 Processor -- start 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.628+0000 7fae7b834640 1 -- start start 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae7b834640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158110 0x7fae740b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae7b834640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fae741584f0 0x7fae7415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae7b834640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae740cbe60 con 0x7fae7406e460 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae7b834640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae740b19b0 con 0x7fae741584f0 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae7b834640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7415b360 con 0x7fae74158110 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae79dd1640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fae7406e460 0x7fae7415e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42920/0 (socket says 172.21.15.133:42920) 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae79dd1640 1 -- 172.21.15.133:0/20952839 learned_addr learned my addr 172.21.15.133:0/20952839 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae795d0640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158110 0x7fae740b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae78dcf640 1 --2- 172.21.15.133:0/20952839 >> v2:172.21.15.162:3300/0 conn(0x7fae741584f0 0x7fae7415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277660583 0 0) 0x7fae740cbe60 con 0x7fae7406e460 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae60005740 con 0x7fae7406e460 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3653977203 0 0) 0x7fae60005740 con 0x7fae7406e460 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158110 msgr2=0x7fae740b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158110 0x7fae740b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 >> v2:172.21.15.162:3300/0 conn(0x7fae741584f0 msgr2=0x7fae7415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 --2- 172.21.15.133:0/20952839 >> v2:172.21.15.162:3300/0 conn(0x7fae741584f0 0x7fae7415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7415b640 con 0x7fae7406e460 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fae6c001a00 con 0x7fae7406e460 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 >> v1:172.21.15.133:6789/0 conn(0x7fae7406e460 legacy=0x7fae7415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158110 0x7fae60005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae60006150 0x7fae60006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 0x7fae600073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae740b19b0 con 0x7fae60006f60 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7415b360 con 0x7fae60006150 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6000dbe0 con 0x7fae74158110 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae79dd1640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 0x7fae600073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae795d0640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158110 0x7fae60005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae78dcf640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae60006150 0x7fae60006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae79dd1640 1 -- 172.21.15.133:0/20952839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158110 msgr2=0x7fae60005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae79dd1640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158110 0x7fae60005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae79dd1640 1 -- 172.21.15.133:0/20952839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae60006150 msgr2=0x7fae60006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae79dd1640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae60006150 0x7fae60006a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.632+0000 7fae79dd1640 1 -- 172.21.15.133:0/20952839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae60005740 con 0x7fae60006f60 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae79dd1640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 0x7fae600073e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00a920 tx=0x7fae6c01d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa75f21cc1047f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00a4c0 con 0x7fae60006f60 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c0072f0 con 0x7fae60006f60 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae67fff640 1 -- 172.21.15.133:0/20952839 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c007720 con 0x7fae60006f60 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 -- 172.21.15.133:0/20952839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 msgr2=0x7fae600073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 --2- 172.21.15.133:0/20952839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 0x7fae600073e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00a920 tx=0x7fae6c01d3e0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae79dd1640 1 -- 172.21.15.133:0/20952839 reap_dead start 2024-04-25T10:02:12.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 -- 172.21.15.133:0/20952839 shutdown_connections 2024-04-25T10:02:12.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 -- 172.21.15.133:0/20952839 >> 172.21.15.133:0/20952839 conn(0x7fae74067000 msgr2=0x7fae7406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 -- 172.21.15.133:0/20952839 shutdown_connections 2024-04-25T10:02:12.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 -- 172.21.15.133:0/20952839 wait complete. 2024-04-25T10:02:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 Processor -- start 2024-04-25T10:02:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 -- start start 2024-04-25T10:02:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 0x7fae741540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158c10 0x7fae740c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae7415d430 0x7fae7415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae740cd670 con 0x7fae60006f60 2024-04-25T10:02:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7415bc00 con 0x7fae7415d430 2024-04-25T10:02:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae7b834640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae740cd7f0 con 0x7fae74158c10 2024-04-25T10:02:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae795d0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 0x7fae741540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae795d0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 0x7fae741540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49502/0 (socket says 172.21.15.133:49502) 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae795d0640 1 -- 172.21.15.133:0/3913358569 learned_addr learned my addr 172.21.15.133:0/3913358569 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae78dcf640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158c10 0x7fae740c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae79dd1640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae7415d430 0x7fae7415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae795d0640 1 -- 172.21.15.133:0/3913358569 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158c10 msgr2=0x7fae740c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae795d0640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158c10 0x7fae740c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae795d0640 1 -- 172.21.15.133:0/3913358569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae7415d430 msgr2=0x7fae7415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae795d0640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae7415d430 0x7fae7415d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae795d0640 1 -- 172.21.15.133:0/3913358569 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7415a000 con 0x7fae60006f60 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.636+0000 7fae79dd1640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae7415d430 0x7fae7415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae795d0640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 0x7fae741540b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae70011610 tx=0x7fae70011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae65ffb640 1 -- 172.21.15.133:0/3913358569 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7000f8f0 con 0x7fae60006f60 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae65ffb640 1 -- 172.21.15.133:0/3913358569 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7000fa90 con 0x7fae60006f60 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae7b834640 1 -- 172.21.15.133:0/3913358569 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae740c6b80 con 0x7fae60006f60 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae65ffb640 1 -- 172.21.15.133:0/3913358569 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7000fe40 con 0x7fae60006f60 2024-04-25T10:02:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae7b834640 1 -- 172.21.15.133:0/3913358569 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae740ca650 con 0x7fae60006f60 2024-04-25T10:02:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae7b834640 1 -- 172.21.15.133:0/3913358569 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7406bc20 con 0x7fae60006f60 2024-04-25T10:02:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae65ffb640 1 -- 172.21.15.133:0/3913358569 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae70015460 con 0x7fae60006f60 2024-04-25T10:02:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae65ffb640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae58066f90 0x7fae58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae78dcf640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae58066f90 0x7fae58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae65ffb640 1 -- 172.21.15.133:0/3913358569 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fae70085ca0 con 0x7fae60006f60 2024-04-25T10:02:12.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.640+0000 7fae78dcf640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae58066f90 0x7fae58069450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae740b1980 tx=0x7fae68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.644+0000 7fae65ffb640 1 -- 172.21.15.133:0/3913358569 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae70019030 con 0x7fae60006f60 2024-04-25T10:02:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.804+0000 7fae7b834640 1 -- 172.21.15.133:0/3913358569 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae74068ef0 con 0x7fae60006f60 2024-04-25T10:02:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.804+0000 7fae65ffb640 1 -- 172.21.15.133:0/3913358569 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7fae70056970 con 0x7fae60006f60 2024-04-25T10:02:12.813 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:12.813 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 -- 172.21.15.133:0/3913358569 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae58066f90 msgr2=0x7fae58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae58066f90 0x7fae58069450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae740b1980 tx=0x7fae68008040 comp rx=0 tx=0).stop 2024-04-25T10:02:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 -- 172.21.15.133:0/3913358569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 msgr2=0x7fae741540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 0x7fae741540b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae70011610 tx=0x7fae70011ad0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 -- 172.21.15.133:0/3913358569 shutdown_connections 2024-04-25T10:02:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae7415d430 0x7fae7415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae74158c10 0x7fae740c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae58066f90 0x7fae58069450 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 --2- 172.21.15.133:0/3913358569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae60006f60 0x7fae741540b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 -- 172.21.15.133:0/3913358569 >> 172.21.15.133:0/3913358569 conn(0x7fae74067000 msgr2=0x7fae74154f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 -- 172.21.15.133:0/3913358569 shutdown_connections 2024-04-25T10:02:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.808+0000 7fae7b834640 1 -- 172.21.15.133:0/3913358569 wait complete. 2024-04-25T10:02:12.832 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:02:12.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb13471a640 1 Processor -- start 2024-04-25T10:02:12.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb13471a640 1 -- start start 2024-04-25T10:02:12.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb13471a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb12c0cd820 0x7fb12c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb13471a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb12c15ced0 0x7fb12c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb13471a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12c0ce2e0 con 0x7fb12c145170 2024-04-25T10:02:12.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb13471a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12c06db00 con 0x7fb12c0cd820 2024-04-25T10:02:12.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb13471a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c145550 con 0x7fb12c15ced0 2024-04-25T10:02:12.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb1324b6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb12c145170 0x7fb12c06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42936/0 (socket says 172.21.15.133:42936) 2024-04-25T10:02:12.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb1324b6640 1 -- 172.21.15.133:0/422936297 learned_addr learned my addr 172.21.15.133:0/422936297 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:12.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb131cb5640 1 --2- 172.21.15.133:0/422936297 >> v2:172.21.15.133:3300/0 conn(0x7fb12c0cd820 0x7fb12c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb132cb7640 1 --2- 172.21.15.133:0/422936297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb12c15ced0 0x7fb12c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb131cb5640 1 -- 172.21.15.133:0/422936297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb12c15ced0 msgr2=0x7fb12c15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb131cb5640 1 --2- 172.21.15.133:0/422936297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb12c15ced0 0x7fb12c15d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb131cb5640 1 -- 172.21.15.133:0/422936297 >> v1:172.21.15.133:6789/0 conn(0x7fb12c145170 legacy=0x7fb12c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb131cb5640 1 -- 172.21.15.133:0/422936297 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c06d4b0 con 0x7fb12c0cd820 2024-04-25T10:02:12.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.952+0000 7fb132cb7640 1 --2- 172.21.15.133:0/422936297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb12c15ced0 0x7fb12c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:12.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb131cb5640 1 --2- 172.21.15.133:0/422936297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb12c0cd820 0x7fb12c0cdca0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0097c0 tx=0x7fb11c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e8aaa00292f684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb1314b4640 1 -- 172.21.15.133:0/422936297 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c0b0070 con 0x7fb12c0cd820 2024-04-25T10:02:12.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb1314b4640 1 -- 172.21.15.133:0/422936297 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c0ab600 con 0x7fb12c0cd820 2024-04-25T10:02:12.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb1314b4640 1 -- 172.21.15.133:0/422936297 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c0ab9d0 con 0x7fb12c0cd820 2024-04-25T10:02:12.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 -- 172.21.15.133:0/422936297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb12c0cd820 msgr2=0x7fb12c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 --2- 172.21.15.133:0/422936297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb12c0cd820 0x7fb12c0cdca0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0097c0 tx=0x7fb11c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 -- 172.21.15.133:0/422936297 shutdown_connections 2024-04-25T10:02:12.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 --2- 172.21.15.133:0/422936297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb12c15ced0 0x7fb12c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 --2- 172.21.15.133:0/422936297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb12c0cd820 0x7fb12c0cdca0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 -- 172.21.15.133:0/422936297 >> 172.21.15.133:0/422936297 conn(0x7fb12c066f70 msgr2=0x7fb12c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 -- 172.21.15.133:0/422936297 shutdown_connections 2024-04-25T10:02:12.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 -- 172.21.15.133:0/422936297 wait complete. 2024-04-25T10:02:12.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 Processor -- start 2024-04-25T10:02:12.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 -- start start 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb12c0cd820 0x7fb12c143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb12c145170 0x7fb12c142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c15ced0 0x7fb12c1391d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c168610 con 0x7fb12c0cd820 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c168490 con 0x7fb12c15ced0 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb13471a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c168790 con 0x7fb12c145170 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb1324b6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb12c0cd820 0x7fb12c143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb1324b6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb12c0cd820 0x7fb12c143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49522/0 (socket says 172.21.15.133:49522) 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb1324b6640 1 -- 172.21.15.133:0/1630192586 learned_addr learned my addr 172.21.15.133:0/1630192586 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb132cb7640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c15ced0 0x7fb12c1391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb1324b6640 1 -- 172.21.15.133:0/1630192586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb12c145170 msgr2=0x7fb12c142d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb1324b6640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb12c145170 0x7fb12c142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb1324b6640 1 -- 172.21.15.133:0/1630192586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c15ced0 msgr2=0x7fb12c1391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb1324b6640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c15ced0 0x7fb12c1391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.956+0000 7fb1324b6640 1 -- 172.21.15.133:0/1630192586 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c163cb0 con 0x7fb12c0cd820 2024-04-25T10:02:12.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb1324b6640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb12c0cd820 0x7fb12c143690 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb12800c3a0 tx=0x7fb12800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb122ffd640 1 -- 172.21.15.133:0/1630192586 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128017070 con 0x7fb12c0cd820 2024-04-25T10:02:12.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb122ffd640 1 -- 172.21.15.133:0/1630192586 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb128005cf0 con 0x7fb12c0cd820 2024-04-25T10:02:12.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb13471a640 1 -- 172.21.15.133:0/1630192586 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12c143cb0 con 0x7fb12c0cd820 2024-04-25T10:02:12.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb122ffd640 1 -- 172.21.15.133:0/1630192586 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1280072d0 con 0x7fb12c0cd820 2024-04-25T10:02:12.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb13471a640 1 -- 172.21.15.133:0/1630192586 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12c144270 con 0x7fb12c0cd820 2024-04-25T10:02:12.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb13471a640 1 -- 172.21.15.133:0/1630192586 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12c068e80 con 0x7fb12c0cd820 2024-04-25T10:02:12.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb122ffd640 1 -- 172.21.15.133:0/1630192586 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb12801a070 con 0x7fb12c0cd820 2024-04-25T10:02:12.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb122ffd640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb110066f60 0x7fb110069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb122ffd640 1 -- 172.21.15.133:0/1630192586 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb128084530 con 0x7fb12c0cd820 2024-04-25T10:02:12.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb131cb5640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb110066f60 0x7fb110069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.960+0000 7fb131cb5640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb110066f60 0x7fb110069420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0ad000 tx=0x7fb11c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:12.964+0000 7fb122ffd640 1 -- 172.21.15.133:0/1630192586 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb128054bb0 con 0x7fb12c0cd820 2024-04-25T10:02:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:13.160+0000 7fb13471a640 1 -- 172.21.15.133:0/1630192586 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb12c15f2b0 con 0x7fb12c0cd820 2024-04-25T10:02:14.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.072+0000 7fb122ffd640 1 -- 172.21.15.133:0/1630192586 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb128058c50 con 0x7fb12c0cd820 2024-04-25T10:02:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 -- 172.21.15.133:0/1630192586 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb110066f60 msgr2=0x7fb110069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb110066f60 0x7fb110069420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0ad000 tx=0x7fb11c004740 comp rx=0 tx=0).stop 2024-04-25T10:02:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 -- 172.21.15.133:0/1630192586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb12c0cd820 msgr2=0x7fb12c143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb12c0cd820 0x7fb12c143690 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb12800c3a0 tx=0x7fb12800c860 comp rx=0 tx=0).stop 2024-04-25T10:02:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 -- 172.21.15.133:0/1630192586 shutdown_connections 2024-04-25T10:02:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb12c15ced0 0x7fb12c1391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb12c145170 0x7fb12c142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb110066f60 0x7fb110069420 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 --2- 172.21.15.133:0/1630192586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb12c0cd820 0x7fb12c143690 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 -- 172.21.15.133:0/1630192586 >> 172.21.15.133:0/1630192586 conn(0x7fb12c066f70 msgr2=0x7fb12c15e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 -- 172.21.15.133:0/1630192586 shutdown_connections 2024-04-25T10:02:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.076+0000 7fb13471a640 1 -- 172.21.15.133:0/1630192586 wait complete. 2024-04-25T10:02:14.102 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T10:02:14.102 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:02:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.224+0000 7f086dde5640 1 Processor -- start 2024-04-25T10:02:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f086dde5640 1 -- start start 2024-04-25T10:02:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f086dde5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086815b660 0x7f086806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f086dde5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f08680ca3e0 0x7f08680ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f086dde5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086806b270 con 0x7f086806c310 2024-04-25T10:02:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f086dde5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08680cad90 con 0x7f08680ca3e0 2024-04-25T10:02:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f086dde5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086806b550 con 0x7f086815b660 2024-04-25T10:02:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f0867fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f08680ca3e0 0x7f08680ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f0867fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f08680ca3e0 0x7f08680ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49532/0 (socket says 172.21.15.133:49532) 2024-04-25T10:02:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f0867fff640 1 -- 172.21.15.133:0/239064843 learned_addr learned my addr 172.21.15.133:0/239064843 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f08677fe640 1 --2- 172.21.15.133:0/239064843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086815b660 0x7f086806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f0867fff640 1 -- 172.21.15.133:0/239064843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086815b660 msgr2=0x7f086806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f0867fff640 1 --2- 172.21.15.133:0/239064843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086815b660 0x7f086806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f0867fff640 1 -- 172.21.15.133:0/239064843 >> v1:172.21.15.133:6789/0 conn(0x7f086806c310 legacy=0x7f086815fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f0867fff640 1 -- 172.21.15.133:0/239064843 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086806b830 con 0x7f08680ca3e0 2024-04-25T10:02:14.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.228+0000 7f0867fff640 1 --2- 172.21.15.133:0/239064843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08680ca3e0 0x7f08680ca850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0860009e50 tx=0x7f086001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283e6b1092d510d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f0866ffd640 1 -- 172.21.15.133:0/239064843 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600a58f0 con 0x7f08680ca3e0 2024-04-25T10:02:14.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f0866ffd640 1 -- 172.21.15.133:0/239064843 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08600a5a90 con 0x7f08680ca3e0 2024-04-25T10:02:14.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f0866ffd640 1 -- 172.21.15.133:0/239064843 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600a5e60 con 0x7f08680ca3e0 2024-04-25T10:02:14.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 -- 172.21.15.133:0/239064843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08680ca3e0 msgr2=0x7f08680ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 --2- 172.21.15.133:0/239064843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08680ca3e0 0x7f08680ca850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0860009e50 tx=0x7f086001ca00 comp rx=0 tx=0).stop 2024-04-25T10:02:14.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 -- 172.21.15.133:0/239064843 shutdown_connections 2024-04-25T10:02:14.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 --2- 172.21.15.133:0/239064843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08680ca3e0 0x7f08680ca850 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 --2- 172.21.15.133:0/239064843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f086815b660 0x7f086806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 -- 172.21.15.133:0/239064843 >> 172.21.15.133:0/239064843 conn(0x7f08680cc810 msgr2=0x7f08680ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 -- 172.21.15.133:0/239064843 shutdown_connections 2024-04-25T10:02:14.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 -- 172.21.15.133:0/239064843 wait complete. 2024-04-25T10:02:14.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 Processor -- start 2024-04-25T10:02:14.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 -- start start 2024-04-25T10:02:14.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086806c310 0x7f0868149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08680ca3e0 0x7f0868163ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f086815b660 0x7f08680cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680ce500 con 0x7f086806c310 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680ce380 con 0x7f086815b660 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680ce680 con 0x7f08680ca3e0 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f08677fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086806c310 0x7f0868149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f08677fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086806c310 0x7f0868149080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49546/0 (socket says 172.21.15.133:49546) 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f08677fe640 1 -- 172.21.15.133:0/3236351676 learned_addr learned my addr 172.21.15.133:0/3236351676 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f085ffff640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08680ca3e0 0x7f0868163ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f08677fe640 1 -- 172.21.15.133:0/3236351676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08680ca3e0 msgr2=0x7f0868163ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f0867fff640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f086815b660 0x7f08680cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f08677fe640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08680ca3e0 0x7f0868163ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f08677fe640 1 -- 172.21.15.133:0/3236351676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f086815b660 msgr2=0x7f08680cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f08677fe640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f086815b660 0x7f08680cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f08677fe640 1 -- 172.21.15.133:0/3236351676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868164940 con 0x7f086806c310 2024-04-25T10:02:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f08677fe640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086806c310 0x7f0868149080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f085800d5f0 tx=0x7f085800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f0864ff9640 1 -- 172.21.15.133:0/3236351676 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08580125f0 con 0x7f086806c310 2024-04-25T10:02:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f0864ff9640 1 -- 172.21.15.133:0/3236351676 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0858019070 con 0x7f086806c310 2024-04-25T10:02:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f0864ff9640 1 -- 172.21.15.133:0/3236351676 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08580129b0 con 0x7f086806c310 2024-04-25T10:02:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 -- 172.21.15.133:0/3236351676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0868164c20 con 0x7f086806c310 2024-04-25T10:02:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.232+0000 7f086dde5640 1 -- 172.21.15.133:0/3236351676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08681651e0 con 0x7f086806c310 2024-04-25T10:02:14.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.236+0000 7f086dde5640 1 -- 172.21.15.133:0/3236351676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0868067c70 con 0x7f086806c310 2024-04-25T10:02:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.236+0000 7f0864ff9640 1 -- 172.21.15.133:0/3236351676 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0858012b50 con 0x7f086806c310 2024-04-25T10:02:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.236+0000 7f0864ff9640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0848066f60 0x7f0848069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.236+0000 7f0864ff9640 1 -- 172.21.15.133:0/3236351676 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f08580857b0 con 0x7f086806c310 2024-04-25T10:02:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.236+0000 7f085ffff640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0848066f60 0x7f0848069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.236+0000 7f085ffff640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0848066f60 0x7f0848069420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0850007fd0 tx=0x7f0850009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.240+0000 7f0864ff9640 1 -- 172.21.15.133:0/3236351676 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0858056480 con 0x7f086806c310 2024-04-25T10:02:14.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:14.436+0000 7f086dde5640 1 -- 172.21.15.133:0/3236351676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f08680b5d10 con 0x7f086806c310 2024-04-25T10:02:15.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.080+0000 7f0864ff9640 1 -- 172.21.15.133:0/3236351676 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0858055d80 con 0x7f086806c310 2024-04-25T10:02:15.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 -- 172.21.15.133:0/3236351676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0848066f60 msgr2=0x7f0848069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0848066f60 0x7f0848069420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0850007fd0 tx=0x7f0850009040 comp rx=0 tx=0).stop 2024-04-25T10:02:15.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 -- 172.21.15.133:0/3236351676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086806c310 msgr2=0x7f0868149080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086806c310 0x7f0868149080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f085800d5f0 tx=0x7f085800dab0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 -- 172.21.15.133:0/3236351676 shutdown_connections 2024-04-25T10:02:15.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f086815b660 0x7f08680cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08680ca3e0 0x7f0868163ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0848066f60 0x7f0848069420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 --2- 172.21.15.133:0/3236351676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086806c310 0x7f0868149080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 -- 172.21.15.133:0/3236351676 >> 172.21.15.133:0/3236351676 conn(0x7f08680cc810 msgr2=0x7f086815def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 -- 172.21.15.133:0/3236351676 shutdown_connections 2024-04-25T10:02:15.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.084+0000 7f086dde5640 1 -- 172.21.15.133:0/3236351676 wait complete. 2024-04-25T10:02:15.112 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:02:15.113 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:15.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.228+0000 7f68c7344640 1 Processor -- start 2024-04-25T10:02:15.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.228+0000 7f68c7344640 1 -- start start 2024-04-25T10:02:15.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c7344640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f68c015c9a0 0x7f68c00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c7344640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68c00ce330 0x7f68c00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c7344640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c00c61d0 con 0x7f68c00b1f70 2024-04-25T10:02:15.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c7344640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c00cecc0 con 0x7f68c015c9a0 2024-04-25T10:02:15.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c7344640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c00c64b0 con 0x7f68c00ce330 2024-04-25T10:02:15.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c50e0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f68c015c9a0 0x7f68c00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c50e0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f68c015c9a0 0x7f68c00c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49548/0 (socket says 172.21.15.133:49548) 2024-04-25T10:02:15.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c50e0640 1 -- 172.21.15.133:0/64011637 learned_addr learned my addr 172.21.15.133:0/64011637 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c50e0640 1 -- 172.21.15.133:0/64011637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68c00ce330 msgr2=0x7f68c00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c58e1640 1 --2- 172.21.15.133:0/64011637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68c00ce330 0x7f68c00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c50e0640 1 --2- 172.21.15.133:0/64011637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68c00ce330 0x7f68c00ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c50e0640 1 -- 172.21.15.133:0/64011637 >> v1:172.21.15.162:6789/0 conn(0x7f68c00b1f70 legacy=0x7f68c006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c50e0640 1 -- 172.21.15.133:0/64011637 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c006d4f0 con 0x7f68c015c9a0 2024-04-25T10:02:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68c50e0640 1 --2- 172.21.15.133:0/64011637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68c015c9a0 0x7f68c00c5980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009870 tx=0x7f68bc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95613853edefa90d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68b7fff640 1 -- 172.21.15.133:0/64011637 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b2070 con 0x7f68c015c9a0 2024-04-25T10:02:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68b7fff640 1 -- 172.21.15.133:0/64011637 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc0ad640 con 0x7f68c015c9a0 2024-04-25T10:02:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.232+0000 7f68b7fff640 1 -- 172.21.15.133:0/64011637 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0ada10 con 0x7f68c015c9a0 2024-04-25T10:02:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 -- 172.21.15.133:0/64011637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68c015c9a0 msgr2=0x7f68c00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 --2- 172.21.15.133:0/64011637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68c015c9a0 0x7f68c00c5980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009870 tx=0x7f68bc01c520 comp rx=0 tx=0).stop 2024-04-25T10:02:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 -- 172.21.15.133:0/64011637 shutdown_connections 2024-04-25T10:02:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 --2- 172.21.15.133:0/64011637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68c00ce330 0x7f68c00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 --2- 172.21.15.133:0/64011637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68c015c9a0 0x7f68c00c5980 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 -- 172.21.15.133:0/64011637 >> 172.21.15.133:0/64011637 conn(0x7f68c0067000 msgr2=0x7f68c006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 -- 172.21.15.133:0/64011637 shutdown_connections 2024-04-25T10:02:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 -- 172.21.15.133:0/64011637 wait complete. 2024-04-25T10:02:15.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 Processor -- start 2024-04-25T10:02:15.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 -- start start 2024-04-25T10:02:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68c00b1f70 0x7f68c013a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68c00ce330 0x7f68c015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68c015c9a0 0x7f68c015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c015dad0 con 0x7f68c00b1f70 2024-04-25T10:02:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c015d950 con 0x7f68c015c9a0 2024-04-25T10:02:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c015dc50 con 0x7f68c00ce330 2024-04-25T10:02:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c50e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68c00b1f70 0x7f68c013a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c50e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68c00b1f70 0x7f68c013a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49552/0 (socket says 172.21.15.133:49552) 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c50e0640 1 -- 172.21.15.133:0/1764350878 learned_addr learned my addr 172.21.15.133:0/1764350878 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c58e1640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68c015c9a0 0x7f68c015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c48df640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68c00ce330 0x7f68c015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c50e0640 1 -- 172.21.15.133:0/1764350878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68c00ce330 msgr2=0x7f68c015a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c50e0640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68c00ce330 0x7f68c015a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c50e0640 1 -- 172.21.15.133:0/1764350878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68c015c9a0 msgr2=0x7f68c015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c50e0640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68c015c9a0 0x7f68c015ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c50e0640 1 -- 172.21.15.133:0/1764350878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c013dcc0 con 0x7f68c00b1f70 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c50e0640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68c00b1f70 0x7f68c013a610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0b0040 tx=0x7f68bc0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68b5ffb640 1 -- 172.21.15.133:0/1764350878 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b2040 con 0x7f68c00b1f70 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68b5ffb640 1 -- 172.21.15.133:0/1764350878 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc0b6410 con 0x7f68c00b1f70 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 -- 172.21.15.133:0/1764350878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c01f4a10 con 0x7f68c00b1f70 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68b5ffb640 1 -- 172.21.15.133:0/1764350878 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b6820 con 0x7f68c00b1f70 2024-04-25T10:02:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.236+0000 7f68c7344640 1 -- 172.21.15.133:0/1764350878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c01f4fd0 con 0x7f68c00b1f70 2024-04-25T10:02:15.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.240+0000 7f68c7344640 1 -- 172.21.15.133:0/1764350878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c0068ef0 con 0x7f68c00b1f70 2024-04-25T10:02:15.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.240+0000 7f68b5ffb640 1 -- 172.21.15.133:0/1764350878 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68bc0be020 con 0x7f68c00b1f70 2024-04-25T10:02:15.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.240+0000 7f68b5ffb640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68a4066f60 0x7f68a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.240+0000 7f68b5ffb640 1 -- 172.21.15.133:0/1764350878 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f68bc11ec10 con 0x7f68c00b1f70 2024-04-25T10:02:15.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.240+0000 7f68c48df640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68a4066f60 0x7f68a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.240+0000 7f68c48df640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68a4066f60 0x7f68a4069420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f68c006b9d0 tx=0x7f68b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.244+0000 7f68b5ffb640 1 -- 172.21.15.133:0/1764350878 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68bc0c5020 con 0x7f68c00b1f70 2024-04-25T10:02:15.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.436+0000 7f68c7344640 1 -- 172.21.15.133:0/1764350878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68c01530b0 con 0x7f68c00b1f70 2024-04-25T10:02:15.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.436+0000 7f68b5ffb640 1 -- 172.21.15.133:0/1764350878 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f68bc122020 con 0x7f68c00b1f70 2024-04-25T10:02:15.443 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:15.443 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2230831968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2230831968},{"type":"v1","addr":"172.21.15.162:6835","nonce":2230831968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1456396097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1456396097},{"type":"v1","addr":"172.21.15.133:6836","nonce":1456396097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:51.276631+0000","modified":"2024-04-25T10:02:14.440426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2462154167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2462154167},{"type":"v1","addr":"172.21.15.133:6837","nonce":2462154167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:15.443 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 9 2024-04-25T10:02:15.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.436+0000 7f68c7344640 1 -- 172.21.15.133:0/1764350878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68a4066f60 msgr2=0x7f68a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.436+0000 7f68c7344640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68a4066f60 0x7f68a4069420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f68c006b9d0 tx=0x7f68b0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:15.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.436+0000 7f68c7344640 1 -- 172.21.15.133:0/1764350878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68c00b1f70 msgr2=0x7f68c013a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.436+0000 7f68c7344640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68c00b1f70 0x7f68c013a610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0b0040 tx=0x7f68bc0192c0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.440+0000 7f68c7344640 1 -- 172.21.15.133:0/1764350878 shutdown_connections 2024-04-25T10:02:15.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.440+0000 7f68c7344640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68c015c9a0 0x7f68c015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.440+0000 7f68c7344640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68c00ce330 0x7f68c015a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.440+0000 7f68c7344640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68a4066f60 0x7f68a4069420 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.440+0000 7f68c7344640 1 --2- 172.21.15.133:0/1764350878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68c00b1f70 0x7f68c013a610 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.440+0000 7f68c7344640 1 -- 172.21.15.133:0/1764350878 >> 172.21.15.133:0/1764350878 conn(0x7f68c0067000 msgr2=0x7f68c01535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.440+0000 7f68c7344640 1 -- 172.21.15.133:0/1764350878 shutdown_connections 2024-04-25T10:02:15.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.440+0000 7f68c7344640 1 -- 172.21.15.133:0/1764350878 wait complete. 2024-04-25T10:02:15.461 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:15.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.572+0000 7faedcb01640 1 Processor -- start 2024-04-25T10:02:15.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.572+0000 7faedcb01640 1 -- start start 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faedcb01640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faed815d9b0 0x7faed80cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faedcb01640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faed815dd90 0x7faed80ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faedcb01640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faed80b19f0 0x7faed80cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faedcb01640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed80cbe30 con 0x7faed815dd90 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faedcb01640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed80ca9b0 con 0x7faed815d9b0 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faedcb01640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed80cac90 con 0x7faed80b19f0 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faed815dd90 0x7faed80ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faed815dd90 0x7faed80ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49554/0 (socket says 172.21.15.133:49554) 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed5d9b640 1 -- 172.21.15.133:0/607780583 learned_addr learned my addr 172.21.15.133:0/607780583 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed6d9d640 1 --2- 172.21.15.133:0/607780583 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faed80b19f0 0x7faed80cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed659c640 1 --2- 172.21.15.133:0/607780583 >> v2:172.21.15.162:3300/0 conn(0x7faed815d9b0 0x7faed80cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed5d9b640 1 -- 172.21.15.133:0/607780583 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faed80b19f0 msgr2=0x7faed80cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed5d9b640 1 --2- 172.21.15.133:0/607780583 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faed80b19f0 0x7faed80cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed5d9b640 1 -- 172.21.15.133:0/607780583 >> v2:172.21.15.162:3300/0 conn(0x7faed815d9b0 msgr2=0x7faed80cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed5d9b640 1 --2- 172.21.15.133:0/607780583 >> v2:172.21.15.162:3300/0 conn(0x7faed815d9b0 0x7faed80cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed5d9b640 1 -- 172.21.15.133:0/607780583 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed80caf70 con 0x7faed815dd90 2024-04-25T10:02:15.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed659c640 1 --2- 172.21.15.133:0/607780583 >> v2:172.21.15.162:3300/0 conn(0x7faed815d9b0 0x7faed80cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:15.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed5d9b640 1 --2- 172.21.15.133:0/607780583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed815dd90 0x7faed80ce770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faed80b4c70 tx=0x7faecc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25669d9d5251900b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed559a640 1 -- 172.21.15.133:0/607780583 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc0ab610 con 0x7faed815dd90 2024-04-25T10:02:15.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed559a640 1 -- 172.21.15.133:0/607780583 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc0b0070 con 0x7faed815dd90 2024-04-25T10:02:15.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.576+0000 7faed559a640 1 -- 172.21.15.133:0/607780583 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc0ab9d0 con 0x7faed815dd90 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 -- 172.21.15.133:0/607780583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed815dd90 msgr2=0x7faed80ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 --2- 172.21.15.133:0/607780583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed815dd90 0x7faed80ce770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faed80b4c70 tx=0x7faecc01c430 comp rx=0 tx=0).stop 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 -- 172.21.15.133:0/607780583 shutdown_connections 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 --2- 172.21.15.133:0/607780583 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faed80b19f0 0x7faed80cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 --2- 172.21.15.133:0/607780583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed815dd90 0x7faed80ce770 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 --2- 172.21.15.133:0/607780583 >> v2:172.21.15.162:3300/0 conn(0x7faed815d9b0 0x7faed80cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 -- 172.21.15.133:0/607780583 >> 172.21.15.133:0/607780583 conn(0x7faed8067000 msgr2=0x7faed806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 -- 172.21.15.133:0/607780583 shutdown_connections 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 -- 172.21.15.133:0/607780583 wait complete. 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 Processor -- start 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 -- start start 2024-04-25T10:02:15.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed80b19f0 0x7faed815fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed815d9b0 0x7faed80ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faed815dd90 0x7faed80cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed8162290 con 0x7faed815d9b0 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed8162110 con 0x7faed80b19f0 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed8162410 con 0x7faed815dd90 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faed5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed815d9b0 0x7faed80ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faed5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed815d9b0 0x7faed80ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49568/0 (socket says 172.21.15.133:49568) 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faed5d9b640 1 -- 172.21.15.133:0/3648416447 learned_addr learned my addr 172.21.15.133:0/3648416447 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faed659c640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed80b19f0 0x7faed815fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faed6d9d640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faed815dd90 0x7faed80cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faed5d9b640 1 -- 172.21.15.133:0/3648416447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faed815dd90 msgr2=0x7faed80cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faed5d9b640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faed815dd90 0x7faed80cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faed5d9b640 1 -- 172.21.15.133:0/3648416447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed80b19f0 msgr2=0x7faed815fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faed5d9b640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed80b19f0 0x7faed815fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faed5d9b640 1 -- 172.21.15.133:0/3648416447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed8160400 con 0x7faed815d9b0 2024-04-25T10:02:15.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faed5d9b640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed815d9b0 0x7faed80ccce0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faecc01ab40 tx=0x7faecc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faec6ffd640 1 -- 172.21.15.133:0/3648416447 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc0b4800 con 0x7faed815d9b0 2024-04-25T10:02:15.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faec6ffd640 1 -- 172.21.15.133:0/3648416447 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc0b49a0 con 0x7faed815d9b0 2024-04-25T10:02:15.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 -- 172.21.15.133:0/3648416447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed81606e0 con 0x7faed815d9b0 2024-04-25T10:02:15.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faec6ffd640 1 -- 172.21.15.133:0/3648416447 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc0b4d90 con 0x7faed815d9b0 2024-04-25T10:02:15.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.580+0000 7faedcb01640 1 -- 172.21.15.133:0/3648416447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed8160ca0 con 0x7faed815d9b0 2024-04-25T10:02:15.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.584+0000 7faedcb01640 1 -- 172.21.15.133:0/3648416447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed8068ef0 con 0x7faed815d9b0 2024-04-25T10:02:15.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.584+0000 7faec6ffd640 1 -- 172.21.15.133:0/3648416447 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faecc0c7390 con 0x7faed815d9b0 2024-04-25T10:02:15.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.584+0000 7faec6ffd640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faeb8066f60 0x7faeb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.584+0000 7faec6ffd640 1 -- 172.21.15.133:0/3648416447 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faecc11fd40 con 0x7faed815d9b0 2024-04-25T10:02:15.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.584+0000 7faed659c640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faeb8066f60 0x7faeb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.584+0000 7faed659c640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faeb8066f60 0x7faeb8069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faed80ce150 tx=0x7faec8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.588+0000 7faec6ffd640 1 -- 172.21.15.133:0/3648416447 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faecc0c4060 con 0x7faed815d9b0 2024-04-25T10:02:15.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.748+0000 7faedcb01640 1 -- 172.21.15.133:0/3648416447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faed806d900 con 0x7faed815d9b0 2024-04-25T10:02:15.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.748+0000 7faec6ffd640 1 -- 172.21.15.133:0/3648416447 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7faecc0a8050 con 0x7faed815d9b0 2024-04-25T10:02:15.756 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:15.757 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 -- 172.21.15.133:0/3648416447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faeb8066f60 msgr2=0x7faeb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faeb8066f60 0x7faeb8069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faed80ce150 tx=0x7faec8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 -- 172.21.15.133:0/3648416447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed815d9b0 msgr2=0x7faed80ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed815d9b0 0x7faed80ccce0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faecc01ab40 tx=0x7faecc01acd0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 -- 172.21.15.133:0/3648416447 shutdown_connections 2024-04-25T10:02:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faed815dd90 0x7faed80cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faeb8066f60 0x7faeb8069420 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faed815d9b0 0x7faed80ccce0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 --2- 172.21.15.133:0/3648416447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faed80b19f0 0x7faed815fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 -- 172.21.15.133:0/3648416447 >> 172.21.15.133:0/3648416447 conn(0x7faed8067000 msgr2=0x7faed806e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 -- 172.21.15.133:0/3648416447 shutdown_connections 2024-04-25T10:02:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.752+0000 7faedcb01640 1 -- 172.21.15.133:0/3648416447 wait complete. 2024-04-25T10:02:15.776 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:02:15.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:15.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fceea5d6640 1 Processor -- start 2024-04-25T10:02:15.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fceea5d6640 1 -- start start 2024-04-25T10:02:15.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fceea5d6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcee406eb70 0x7fcee40c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fceea5d6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcee415a9d0 0x7fcee40cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fceea5d6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee40cc5a0 con 0x7fcee415a5f0 2024-04-25T10:02:15.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fceea5d6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee40cc880 con 0x7fcee406eb70 2024-04-25T10:02:15.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fceea5d6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40ccb60 con 0x7fcee415a9d0 2024-04-25T10:02:15.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fcedb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcee406eb70 0x7fcee40c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fcedb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcee406eb70 0x7fcee40c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49576/0 (socket says 172.21.15.133:49576) 2024-04-25T10:02:15.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fcedb7fe640 1 -- 172.21.15.133:0/2390668415 learned_addr learned my addr 172.21.15.133:0/2390668415 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:15.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fcee8b73640 1 --2- 172.21.15.133:0/2390668415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcee415a9d0 0x7fcee40cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fcedb7fe640 1 -- 172.21.15.133:0/2390668415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcee415a9d0 msgr2=0x7fcee40cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fcedb7fe640 1 --2- 172.21.15.133:0/2390668415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcee415a9d0 0x7fcee40cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fcedb7fe640 1 -- 172.21.15.133:0/2390668415 >> v1:172.21.15.162:6789/0 conn(0x7fcee415a5f0 legacy=0x7fcee406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.884+0000 7fcedb7fe640 1 -- 172.21.15.133:0/2390668415 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40cce40 con 0x7fcee406eb70 2024-04-25T10:02:15.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcedb7fe640 1 --2- 172.21.15.133:0/2390668415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcee406eb70 0x7fcee40c5260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009870 tx=0x7fcecc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbf98d2e499fc69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee37fe640 1 -- 172.21.15.133:0/2390668415 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0b0070 con 0x7fcee406eb70 2024-04-25T10:02:15.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee37fe640 1 -- 172.21.15.133:0/2390668415 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc0ab600 con 0x7fcee406eb70 2024-04-25T10:02:15.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee37fe640 1 -- 172.21.15.133:0/2390668415 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0ab9d0 con 0x7fcee406eb70 2024-04-25T10:02:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 -- 172.21.15.133:0/2390668415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcee406eb70 msgr2=0x7fcee40c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 --2- 172.21.15.133:0/2390668415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcee406eb70 0x7fcee40c5260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009870 tx=0x7fcecc01c520 comp rx=0 tx=0).stop 2024-04-25T10:02:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 -- 172.21.15.133:0/2390668415 shutdown_connections 2024-04-25T10:02:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 --2- 172.21.15.133:0/2390668415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcee415a9d0 0x7fcee40cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 --2- 172.21.15.133:0/2390668415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcee406eb70 0x7fcee40c5260 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 -- 172.21.15.133:0/2390668415 >> 172.21.15.133:0/2390668415 conn(0x7fcee4067000 msgr2=0x7fcee406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 -- 172.21.15.133:0/2390668415 shutdown_connections 2024-04-25T10:02:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 -- 172.21.15.133:0/2390668415 wait complete. 2024-04-25T10:02:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 Processor -- start 2024-04-25T10:02:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 -- start start 2024-04-25T10:02:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcee406eb70 0x7fcee4154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee415a5f0 0x7fcee414dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcee415a9d0 0x7fcee415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40ce4d0 con 0x7fcee415a9d0 2024-04-25T10:02:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40ce350 con 0x7fcee415a5f0 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40ce650 con 0x7fcee406eb70 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee8b73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcee415a9d0 0x7fcee415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee8b73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcee415a9d0 0x7fcee415bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49582/0 (socket says 172.21.15.133:49582) 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee8b73640 1 -- 172.21.15.133:0/905188389 learned_addr learned my addr 172.21.15.133:0/905188389 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcedb7fe640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee415a5f0 0x7fcee414dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee3fff640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcee406eb70 0x7fcee4154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee8b73640 1 -- 172.21.15.133:0/905188389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcee406eb70 msgr2=0x7fcee4154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee8b73640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcee406eb70 0x7fcee4154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee8b73640 1 -- 172.21.15.133:0/905188389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee415a5f0 msgr2=0x7fcee414dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee8b73640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee415a5f0 0x7fcee414dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee8b73640 1 -- 172.21.15.133:0/905188389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee415c330 con 0x7fcee415a9d0 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee8b73640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcee415a9d0 0x7fcee415bdf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcedc007d80 tx=0x7fcedc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee17fa640 1 -- 172.21.15.133:0/905188389 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc014420 con 0x7fcee415a9d0 2024-04-25T10:02:15.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee17fa640 1 -- 172.21.15.133:0/905188389 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc019070 con 0x7fcee415a9d0 2024-04-25T10:02:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 -- 172.21.15.133:0/905188389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee415c5d0 con 0x7fcee415a9d0 2024-04-25T10:02:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fcee17fa640 1 -- 172.21.15.133:0/905188389 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc014800 con 0x7fcee415a9d0 2024-04-25T10:02:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.888+0000 7fceea5d6640 1 -- 172.21.15.133:0/905188389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee415cb90 con 0x7fcee415a9d0 2024-04-25T10:02:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.892+0000 7fceea5d6640 1 -- 172.21.15.133:0/905188389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee4068ef0 con 0x7fcee415a9d0 2024-04-25T10:02:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.892+0000 7fcee17fa640 1 -- 172.21.15.133:0/905188389 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcedc0149a0 con 0x7fcee415a9d0 2024-04-25T10:02:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.892+0000 7fcee17fa640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcec4066f90 0x7fcec4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.892+0000 7fcee17fa640 1 -- 172.21.15.133:0/905188389 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcedc085b80 con 0x7fcee415a9d0 2024-04-25T10:02:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.892+0000 7fcee3fff640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcec4066f90 0x7fcec4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.892+0000 7fcee3fff640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcec4066f90 0x7fcec4069450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fced4004e30 tx=0x7fced4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:15.896+0000 7fcee17fa640 1 -- 172.21.15.133:0/905188389 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcedc056850 con 0x7fcee415a9d0 2024-04-25T10:02:16.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.048+0000 7fceea5d6640 1 -- 172.21.15.133:0/905188389 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcee4159e00 con 0x7fcec4066f90 2024-04-25T10:02:16.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.072+0000 7fcee17fa640 1 -- 172.21.15.133:0/905188389 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7fcee4159e00 con 0x7fcec4066f90 2024-04-25T10:02:16.085 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:16.087 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:02:16.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.076+0000 7fceea5d6640 1 -- 172.21.15.133:0/905188389 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcec4066f90 msgr2=0x7fcec4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.076+0000 7fceea5d6640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcec4066f90 0x7fcec4069450 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fced4004e30 tx=0x7fced4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:16.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.076+0000 7fceea5d6640 1 -- 172.21.15.133:0/905188389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcee415a9d0 msgr2=0x7fcee415bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.076+0000 7fceea5d6640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcee415a9d0 0x7fcee415bdf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcedc007d80 tx=0x7fcedc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.076+0000 7fceea5d6640 1 -- 172.21.15.133:0/905188389 shutdown_connections 2024-04-25T10:02:16.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.076+0000 7fceea5d6640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcec4066f90 0x7fcec4069450 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.076+0000 7fceea5d6640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcee415a9d0 0x7fcee415bdf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.076+0000 7fceea5d6640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcee415a5f0 0x7fcee414dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.076+0000 7fceea5d6640 1 --2- 172.21.15.133:0/905188389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcee406eb70 0x7fcee4154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.076+0000 7fceea5d6640 1 -- 172.21.15.133:0/905188389 >> 172.21.15.133:0/905188389 conn(0x7fcee4067000 msgr2=0x7fcee40b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:16.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.080+0000 7fceea5d6640 1 -- 172.21.15.133:0/905188389 shutdown_connections 2024-04-25T10:02:16.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.080+0000 7fceea5d6640 1 -- 172.21.15.133:0/905188389 wait complete. 2024-04-25T10:02:16.102 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-25T10:02:15.710509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219388,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755332,"statfs":{"total":773094113280,"available":772869459968,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004059"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631430+0000","last_change":"2024-04-25T10:01:50.953534+0000","last_active":"2024-04-25T10:01:57.631430+0000","last_peered":"2024-04-25T10:01:57.631430+0000","last_clean":"2024-04-25T10:01:57.631430+0000","last_became_active":"2024-04-25T10:01:50.953054+0000","last_became_peered":"2024-04-25T10:01:50.953054+0000","last_unstale":"2024-04-25T10:01:57.631430+0000","last_undegraded":"2024-04-25T10:01:57.631430+0000","last_fullsized":"2024-04-25T10:01:57.631430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:19.695230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633973+0000","last_change":"2024-04-25T10:01:51.958018+0000","last_active":"2024-04-25T10:01:57.633973+0000","last_peered":"2024-04-25T10:01:57.633973+0000","last_clean":"2024-04-25T10:01:57.633973+0000","last_became_active":"2024-04-25T10:01:51.957717+0000","last_became_peered":"2024-04-25T10:01:51.957717+0000","last_unstale":"2024-04-25T10:01:57.633973+0000","last_undegraded":"2024-04-25T10:01:57.633973+0000","last_fullsized":"2024-04-25T10:01:57.633973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:55.810277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631489+0000","last_change":"2024-04-25T10:01:56.111100+0000","last_active":"2024-04-25T10:01:57.631489+0000","last_peered":"2024-04-25T10:01:57.631489+0000","last_clean":"2024-04-25T10:01:57.631489+0000","last_became_active":"2024-04-25T10:01:56.110327+0000","last_became_peered":"2024-04-25T10:01:56.110327+0000","last_unstale":"2024-04-25T10:01:57.631489+0000","last_undegraded":"2024-04-25T10:01:57.631489+0000","last_fullsized":"2024-04-25T10:01:57.631489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634369+0000","last_change":"2024-04-25T10:01:50.940954+0000","last_active":"2024-04-25T10:01:57.634369+0000","last_peered":"2024-04-25T10:01:57.634369+0000","last_clean":"2024-04-25T10:01:57.634369+0000","last_became_active":"2024-04-25T10:01:50.940601+0000","last_became_peered":"2024-04-25T10:01:50.940601+0000","last_unstale":"2024-04-25T10:01:57.634369+0000","last_undegraded":"2024-04-25T10:01:57.634369+0000","last_fullsized":"2024-04-25T10:01:57.634369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631026+0000","last_change":"2024-04-25T10:01:51.994632+0000","last_active":"2024-04-25T10:01:57.631026+0000","last_peered":"2024-04-25T10:01:57.631026+0000","last_clean":"2024-04-25T10:01:57.631026+0000","last_became_active":"2024-04-25T10:01:51.994229+0000","last_became_peered":"2024-04-25T10:01:51.994229+0000","last_unstale":"2024-04-25T10:01:57.631026+0000","last_undegraded":"2024-04-25T10:01:57.631026+0000","last_fullsized":"2024-04-25T10:01:57.631026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005703+0000","last_change":"2024-04-25T10:01:56.024604+0000","last_active":"2024-04-25T10:01:57.005703+0000","last_peered":"2024-04-25T10:01:57.005703+0000","last_clean":"2024-04-25T10:01:57.005703+0000","last_became_active":"2024-04-25T10:01:56.023667+0000","last_became_peered":"2024-04-25T10:01:56.023667+0000","last_unstale":"2024-04-25T10:01:57.005703+0000","last_undegraded":"2024-04-25T10:01:57.005703+0000","last_fullsized":"2024-04-25T10:01:57.005703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:50.018687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348063+0000","last_change":"2024-04-25T10:01:50.957056+0000","last_active":"2024-04-25T10:01:57.348063+0000","last_peered":"2024-04-25T10:01:57.348063+0000","last_clean":"2024-04-25T10:01:57.348063+0000","last_became_active":"2024-04-25T10:01:50.956522+0000","last_became_peered":"2024-04-25T10:01:50.956522+0000","last_unstale":"2024-04-25T10:01:57.348063+0000","last_undegraded":"2024-04-25T10:01:57.348063+0000","last_fullsized":"2024-04-25T10:01:57.348063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:25.085344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101183+0000","last_change":"2024-04-25T10:01:51.987821+0000","last_active":"2024-04-25T10:01:57.101183+0000","last_peered":"2024-04-25T10:01:57.101183+0000","last_clean":"2024-04-25T10:01:57.101183+0000","last_became_active":"2024-04-25T10:01:51.978435+0000","last_became_peered":"2024-04-25T10:01:51.978435+0000","last_unstale":"2024-04-25T10:01:57.101183+0000","last_undegraded":"2024-04-25T10:01:57.101183+0000","last_fullsized":"2024-04-25T10:01:57.101183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.714429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009007+0000","last_change":"2024-04-25T10:01:56.024538+0000","last_active":"2024-04-25T10:01:57.009007+0000","last_peered":"2024-04-25T10:01:57.009007+0000","last_clean":"2024-04-25T10:01:57.009007+0000","last_became_active":"2024-04-25T10:01:56.023560+0000","last_became_peered":"2024-04-25T10:01:56.023560+0000","last_unstale":"2024-04-25T10:01:57.009007+0000","last_undegraded":"2024-04-25T10:01:57.009007+0000","last_fullsized":"2024-04-25T10:01:57.009007+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.432097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347087+0000","last_change":"2024-04-25T10:01:50.954944+0000","last_active":"2024-04-25T10:01:57.347087+0000","last_peered":"2024-04-25T10:01:57.347087+0000","last_clean":"2024-04-25T10:01:57.347087+0000","last_became_active":"2024-04-25T10:01:50.954098+0000","last_became_peered":"2024-04-25T10:01:50.954098+0000","last_unstale":"2024-04-25T10:01:57.347087+0000","last_undegraded":"2024-04-25T10:01:57.347087+0000","last_fullsized":"2024-04-25T10:01:57.347087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:51.011075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101791+0000","last_change":"2024-04-25T10:01:51.976877+0000","last_active":"2024-04-25T10:01:57.101791+0000","last_peered":"2024-04-25T10:01:57.101791+0000","last_clean":"2024-04-25T10:01:57.101791+0000","last_became_active":"2024-04-25T10:01:51.967649+0000","last_became_peered":"2024-04-25T10:01:51.967649+0000","last_unstale":"2024-04-25T10:01:57.101791+0000","last_undegraded":"2024-04-25T10:01:57.101791+0000","last_fullsized":"2024-04-25T10:01:57.101791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006978+0000","last_change":"2024-04-25T10:01:56.018179+0000","last_active":"2024-04-25T10:01:57.006978+0000","last_peered":"2024-04-25T10:01:57.006978+0000","last_clean":"2024-04-25T10:01:57.006978+0000","last_became_active":"2024-04-25T10:01:56.017199+0000","last_became_peered":"2024-04-25T10:01:56.017199+0000","last_unstale":"2024-04-25T10:01:57.006978+0000","last_undegraded":"2024-04-25T10:01:57.006978+0000","last_fullsized":"2024-04-25T10:01:57.006978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:26.814142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007115+0000","last_change":"2024-04-25T10:01:51.935248+0000","last_active":"2024-04-25T10:01:57.007115+0000","last_peered":"2024-04-25T10:01:57.007115+0000","last_clean":"2024-04-25T10:01:57.007115+0000","last_became_active":"2024-04-25T10:01:51.932825+0000","last_became_peered":"2024-04-25T10:01:51.932825+0000","last_unstale":"2024-04-25T10:01:57.007115+0000","last_undegraded":"2024-04-25T10:01:57.007115+0000","last_fullsized":"2024-04-25T10:01:57.007115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347924+0000","last_change":"2024-04-25T10:01:50.956325+0000","last_active":"2024-04-25T10:01:57.347924+0000","last_peered":"2024-04-25T10:01:57.347924+0000","last_clean":"2024-04-25T10:01:57.347924+0000","last_became_active":"2024-04-25T10:01:50.954154+0000","last_became_peered":"2024-04-25T10:01:50.954154+0000","last_unstale":"2024-04-25T10:01:57.347924+0000","last_undegraded":"2024-04-25T10:01:57.347924+0000","last_fullsized":"2024-04-25T10:01:57.347924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:32.182247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008038+0000","last_change":"2024-04-25T10:01:56.024877+0000","last_active":"2024-04-25T10:01:57.008038+0000","last_peered":"2024-04-25T10:01:57.008038+0000","last_clean":"2024-04-25T10:01:57.008038+0000","last_became_active":"2024-04-25T10:01:56.023788+0000","last_became_peered":"2024-04-25T10:01:56.023788+0000","last_unstale":"2024-04-25T10:01:57.008038+0000","last_undegraded":"2024-04-25T10:01:57.008038+0000","last_fullsized":"2024-04-25T10:01:57.008038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:52.776865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633155+0000","last_change":"2024-04-25T10:01:51.952977+0000","last_active":"2024-04-25T10:01:57.633155+0000","last_peered":"2024-04-25T10:01:57.633155+0000","last_clean":"2024-04-25T10:01:57.633155+0000","last_became_active":"2024-04-25T10:01:51.951977+0000","last_became_peered":"2024-04-25T10:01:51.951977+0000","last_unstale":"2024-04-25T10:01:57.633155+0000","last_undegraded":"2024-04-25T10:01:57.633155+0000","last_fullsized":"2024-04-25T10:01:57.633155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.714429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630222+0000","last_change":"2024-04-25T10:01:50.934823+0000","last_active":"2024-04-25T10:01:57.630222+0000","last_peered":"2024-04-25T10:01:57.630222+0000","last_clean":"2024-04-25T10:01:57.630222+0000","last_became_active":"2024-04-25T10:01:50.934535+0000","last_became_peered":"2024-04-25T10:01:50.934535+0000","last_unstale":"2024-04-25T10:01:57.630222+0000","last_undegraded":"2024-04-25T10:01:57.630222+0000","last_fullsized":"2024-04-25T10:01:57.630222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007823+0000","last_change":"2024-04-25T10:01:56.017858+0000","last_active":"2024-04-25T10:01:57.007823+0000","last_peered":"2024-04-25T10:01:57.007823+0000","last_clean":"2024-04-25T10:01:57.007823+0000","last_became_active":"2024-04-25T10:01:56.016904+0000","last_became_peered":"2024-04-25T10:01:56.016904+0000","last_unstale":"2024-04-25T10:01:57.007823+0000","last_undegraded":"2024-04-25T10:01:57.007823+0000","last_fullsized":"2024-04-25T10:01:57.007823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:38.512242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631872+0000","last_change":"2024-04-25T10:01:51.948751+0000","last_active":"2024-04-25T10:01:57.631872+0000","last_peered":"2024-04-25T10:01:57.631872+0000","last_clean":"2024-04-25T10:01:57.631872+0000","last_became_active":"2024-04-25T10:01:51.946530+0000","last_became_peered":"2024-04-25T10:01:51.946530+0000","last_unstale":"2024-04-25T10:01:57.631872+0000","last_undegraded":"2024-04-25T10:01:57.631872+0000","last_fullsized":"2024-04-25T10:01:57.631872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627984+0000","last_change":"2024-04-25T10:01:50.953620+0000","last_active":"2024-04-25T10:01:57.627984+0000","last_peered":"2024-04-25T10:01:57.627984+0000","last_clean":"2024-04-25T10:01:57.627984+0000","last_became_active":"2024-04-25T10:01:50.953363+0000","last_became_peered":"2024-04-25T10:01:50.953363+0000","last_unstale":"2024-04-25T10:01:57.627984+0000","last_undegraded":"2024-04-25T10:01:57.627984+0000","last_fullsized":"2024-04-25T10:01:57.627984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629574+0000","last_change":"2024-04-25T10:01:56.111935+0000","last_active":"2024-04-25T10:01:57.629574+0000","last_peered":"2024-04-25T10:01:57.629574+0000","last_clean":"2024-04-25T10:01:57.629574+0000","last_became_active":"2024-04-25T10:01:56.111568+0000","last_became_peered":"2024-04-25T10:01:56.111568+0000","last_unstale":"2024-04-25T10:01:57.629574+0000","last_undegraded":"2024-04-25T10:01:57.629574+0000","last_fullsized":"2024-04-25T10:01:57.629574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:18.254251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006627+0000","last_change":"2024-04-25T10:01:51.965350+0000","last_active":"2024-04-25T10:01:57.006627+0000","last_peered":"2024-04-25T10:01:57.006627+0000","last_clean":"2024-04-25T10:01:57.006627+0000","last_became_active":"2024-04-25T10:01:51.964456+0000","last_became_peered":"2024-04-25T10:01:51.964456+0000","last_unstale":"2024-04-25T10:01:57.006627+0000","last_undegraded":"2024-04-25T10:01:57.006627+0000","last_fullsized":"2024-04-25T10:01:57.006627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:59.032520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346651+0000","last_change":"2024-04-25T10:01:50.954332+0000","last_active":"2024-04-25T10:01:57.346651+0000","last_peered":"2024-04-25T10:01:57.346651+0000","last_clean":"2024-04-25T10:01:57.346651+0000","last_became_active":"2024-04-25T10:01:50.953854+0000","last_became_peered":"2024-04-25T10:01:50.953854+0000","last_unstale":"2024-04-25T10:01:57.346651+0000","last_undegraded":"2024-04-25T10:01:57.346651+0000","last_fullsized":"2024-04-25T10:01:57.346651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:38.278178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008718+0000","last_change":"2024-04-25T10:01:56.017972+0000","last_active":"2024-04-25T10:01:57.008718+0000","last_peered":"2024-04-25T10:01:57.008718+0000","last_clean":"2024-04-25T10:01:57.008718+0000","last_became_active":"2024-04-25T10:01:56.016953+0000","last_became_peered":"2024-04-25T10:01:56.016953+0000","last_unstale":"2024-04-25T10:01:57.008718+0000","last_undegraded":"2024-04-25T10:01:57.008718+0000","last_fullsized":"2024-04-25T10:01:57.008718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:59.872712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630912+0000","last_change":"2024-04-25T10:01:51.944937+0000","last_active":"2024-04-25T10:01:57.630912+0000","last_peered":"2024-04-25T10:01:57.630912+0000","last_clean":"2024-04-25T10:01:57.630912+0000","last_became_active":"2024-04-25T10:01:51.944599+0000","last_became_peered":"2024-04-25T10:01:51.944599+0000","last_unstale":"2024-04-25T10:01:57.630912+0000","last_undegraded":"2024-04-25T10:01:57.630912+0000","last_fullsized":"2024-04-25T10:01:57.630912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008241+0000","last_change":"2024-04-25T10:01:50.955661+0000","last_active":"2024-04-25T10:01:57.008241+0000","last_peered":"2024-04-25T10:01:57.008241+0000","last_clean":"2024-04-25T10:01:57.008241+0000","last_became_active":"2024-04-25T10:01:50.954955+0000","last_became_peered":"2024-04-25T10:01:50.954955+0000","last_unstale":"2024-04-25T10:01:57.008241+0000","last_undegraded":"2024-04-25T10:01:57.008241+0000","last_fullsized":"2024-04-25T10:01:57.008241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:33.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006345+0000","last_change":"2024-04-25T10:01:56.108996+0000","last_active":"2024-04-25T10:01:57.006345+0000","last_peered":"2024-04-25T10:01:57.006345+0000","last_clean":"2024-04-25T10:01:57.006345+0000","last_became_active":"2024-04-25T10:01:56.107964+0000","last_became_peered":"2024-04-25T10:01:56.107964+0000","last_unstale":"2024-04-25T10:01:57.006345+0000","last_undegraded":"2024-04-25T10:01:57.006345+0000","last_fullsized":"2024-04-25T10:01:57.006345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.917422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634815+0000","last_change":"2024-04-25T10:01:51.959144+0000","last_active":"2024-04-25T10:01:57.634815+0000","last_peered":"2024-04-25T10:01:57.634815+0000","last_clean":"2024-04-25T10:01:57.634815+0000","last_became_active":"2024-04-25T10:01:51.958032+0000","last_became_peered":"2024-04-25T10:01:51.958032+0000","last_unstale":"2024-04-25T10:01:57.634815+0000","last_undegraded":"2024-04-25T10:01:57.634815+0000","last_fullsized":"2024-04-25T10:01:57.634815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:34.449807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629847+0000","last_change":"2024-04-25T10:01:50.951871+0000","last_active":"2024-04-25T10:01:57.629847+0000","last_peered":"2024-04-25T10:01:57.629847+0000","last_clean":"2024-04-25T10:01:57.629847+0000","last_became_active":"2024-04-25T10:01:50.951527+0000","last_became_peered":"2024-04-25T10:01:50.951527+0000","last_unstale":"2024-04-25T10:01:57.629847+0000","last_undegraded":"2024-04-25T10:01:57.629847+0000","last_fullsized":"2024-04-25T10:01:57.629847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:38.278178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630769+0000","last_change":"2024-04-25T10:01:56.101023+0000","last_active":"2024-04-25T10:01:57.630769+0000","last_peered":"2024-04-25T10:01:57.630769+0000","last_clean":"2024-04-25T10:01:57.630769+0000","last_became_active":"2024-04-25T10:01:56.100400+0000","last_became_peered":"2024-04-25T10:01:56.100400+0000","last_unstale":"2024-04-25T10:01:57.630769+0000","last_undegraded":"2024-04-25T10:01:57.630769+0000","last_fullsized":"2024-04-25T10:01:57.630769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:40.633384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628600+0000","last_change":"2024-04-25T10:01:51.951713+0000","last_active":"2024-04-25T10:01:57.628600+0000","last_peered":"2024-04-25T10:01:57.628600+0000","last_clean":"2024-04-25T10:01:57.628600+0000","last_became_active":"2024-04-25T10:01:51.951438+0000","last_became_peered":"2024-04-25T10:01:51.951438+0000","last_unstale":"2024-04-25T10:01:57.628600+0000","last_undegraded":"2024-04-25T10:01:57.628600+0000","last_fullsized":"2024-04-25T10:01:57.628600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348330+0000","last_change":"2024-04-25T10:01:50.956823+0000","last_active":"2024-04-25T10:01:57.348330+0000","last_peered":"2024-04-25T10:01:57.348330+0000","last_clean":"2024-04-25T10:01:57.348330+0000","last_became_active":"2024-04-25T10:01:50.956170+0000","last_became_peered":"2024-04-25T10:01:50.956170+0000","last_unstale":"2024-04-25T10:01:57.348330+0000","last_undegraded":"2024-04-25T10:01:57.348330+0000","last_fullsized":"2024-04-25T10:01:57.348330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:43.784003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629539+0000","last_change":"2024-04-25T10:01:56.101938+0000","last_active":"2024-04-25T10:01:57.629539+0000","last_peered":"2024-04-25T10:01:57.629539+0000","last_clean":"2024-04-25T10:01:57.629539+0000","last_became_active":"2024-04-25T10:01:56.101501+0000","last_became_peered":"2024-04-25T10:01:56.101501+0000","last_unstale":"2024-04-25T10:01:57.629539+0000","last_undegraded":"2024-04-25T10:01:57.629539+0000","last_fullsized":"2024-04-25T10:01:57.629539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:03.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.102065+0000","last_change":"2024-04-25T10:01:51.977453+0000","last_active":"2024-04-25T10:01:57.102065+0000","last_peered":"2024-04-25T10:01:57.102065+0000","last_clean":"2024-04-25T10:01:57.102065+0000","last_became_active":"2024-04-25T10:01:51.968804+0000","last_became_peered":"2024-04-25T10:01:51.968804+0000","last_unstale":"2024-04-25T10:01:57.102065+0000","last_undegraded":"2024-04-25T10:01:57.102065+0000","last_fullsized":"2024-04-25T10:01:57.102065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:34.449807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629477+0000","last_change":"2024-04-25T10:01:50.934637+0000","last_active":"2024-04-25T10:01:57.629477+0000","last_peered":"2024-04-25T10:01:57.629477+0000","last_clean":"2024-04-25T10:01:57.629477+0000","last_became_active":"2024-04-25T10:01:50.933810+0000","last_became_peered":"2024-04-25T10:01:50.933810+0000","last_unstale":"2024-04-25T10:01:57.629477+0000","last_undegraded":"2024-04-25T10:01:57.629477+0000","last_fullsized":"2024-04-25T10:01:57.629477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:21.571798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627408+0000","last_change":"2024-04-25T10:01:56.030222+0000","last_active":"2024-04-25T10:01:57.627408+0000","last_peered":"2024-04-25T10:01:57.627408+0000","last_clean":"2024-04-25T10:01:57.627408+0000","last_became_active":"2024-04-25T10:01:56.023359+0000","last_became_peered":"2024-04-25T10:01:56.023359+0000","last_unstale":"2024-04-25T10:01:57.627408+0000","last_undegraded":"2024-04-25T10:01:57.627408+0000","last_fullsized":"2024-04-25T10:01:57.627408+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009639+0000","last_change":"2024-04-25T10:01:51.941059+0000","last_active":"2024-04-25T10:01:57.009639+0000","last_peered":"2024-04-25T10:01:57.009639+0000","last_clean":"2024-04-25T10:01:57.009639+0000","last_became_active":"2024-04-25T10:01:51.938978+0000","last_became_peered":"2024-04-25T10:01:51.938978+0000","last_unstale":"2024-04-25T10:01:57.009639+0000","last_undegraded":"2024-04-25T10:01:57.009639+0000","last_fullsized":"2024-04-25T10:01:57.009639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:39.550415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101806+0000","last_change":"2024-04-25T10:01:50.952791+0000","last_active":"2024-04-25T10:01:57.101806+0000","last_peered":"2024-04-25T10:01:57.101806+0000","last_clean":"2024-04-25T10:01:57.101806+0000","last_became_active":"2024-04-25T10:01:50.950155+0000","last_became_peered":"2024-04-25T10:01:50.950155+0000","last_unstale":"2024-04-25T10:01:57.101806+0000","last_undegraded":"2024-04-25T10:01:57.101806+0000","last_fullsized":"2024-04-25T10:01:57.101806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629324+0000","last_change":"2024-04-25T10:01:56.016879+0000","last_active":"2024-04-25T10:01:57.629324+0000","last_peered":"2024-04-25T10:01:57.629324+0000","last_clean":"2024-04-25T10:01:57.629324+0000","last_became_active":"2024-04-25T10:01:56.016202+0000","last_became_peered":"2024-04-25T10:01:56.016202+0000","last_unstale":"2024-04-25T10:01:57.629324+0000","last_undegraded":"2024-04-25T10:01:57.629324+0000","last_fullsized":"2024-04-25T10:01:57.629324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:48.574592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627562+0000","last_change":"2024-04-25T10:01:51.952368+0000","last_active":"2024-04-25T10:01:57.627562+0000","last_peered":"2024-04-25T10:01:57.627562+0000","last_clean":"2024-04-25T10:01:57.627562+0000","last_became_active":"2024-04-25T10:01:51.952095+0000","last_became_peered":"2024-04-25T10:01:51.952095+0000","last_unstale":"2024-04-25T10:01:57.627562+0000","last_undegraded":"2024-04-25T10:01:57.627562+0000","last_fullsized":"2024-04-25T10:01:57.627562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004117+0000","last_change":"2024-04-25T10:01:50.940514+0000","last_active":"2024-04-25T10:01:57.004117+0000","last_peered":"2024-04-25T10:01:57.004117+0000","last_clean":"2024-04-25T10:01:57.004117+0000","last_became_active":"2024-04-25T10:01:50.935395+0000","last_became_peered":"2024-04-25T10:01:50.935395+0000","last_unstale":"2024-04-25T10:01:57.004117+0000","last_undegraded":"2024-04-25T10:01:57.004117+0000","last_fullsized":"2024-04-25T10:01:57.004117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.010630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629857+0000","last_change":"2024-04-25T10:01:56.015853+0000","last_active":"2024-04-25T10:01:57.629857+0000","last_peered":"2024-04-25T10:01:57.629857+0000","last_clean":"2024-04-25T10:01:57.629857+0000","last_became_active":"2024-04-25T10:01:56.015382+0000","last_became_peered":"2024-04-25T10:01:56.015382+0000","last_unstale":"2024-04-25T10:01:57.629857+0000","last_undegraded":"2024-04-25T10:01:57.629857+0000","last_fullsized":"2024-04-25T10:01:57.629857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:32.289340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628148+0000","last_change":"2024-04-25T10:01:51.939122+0000","last_active":"2024-04-25T10:01:57.628148+0000","last_peered":"2024-04-25T10:01:57.628148+0000","last_clean":"2024-04-25T10:01:57.628148+0000","last_became_active":"2024-04-25T10:01:51.938342+0000","last_became_peered":"2024-04-25T10:01:51.938342+0000","last_unstale":"2024-04-25T10:01:57.628148+0000","last_undegraded":"2024-04-25T10:01:57.628148+0000","last_fullsized":"2024-04-25T10:01:57.628148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:33.192217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346666+0000","last_change":"2024-04-25T10:01:50.957985+0000","last_active":"2024-04-25T10:01:57.346666+0000","last_peered":"2024-04-25T10:01:57.346666+0000","last_clean":"2024-04-25T10:01:57.346666+0000","last_became_active":"2024-04-25T10:01:50.957155+0000","last_became_peered":"2024-04-25T10:01:50.957155+0000","last_unstale":"2024-04-25T10:01:57.346666+0000","last_undegraded":"2024-04-25T10:01:57.346666+0000","last_fullsized":"2024-04-25T10:01:57.346666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:21.571798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631051+0000","last_change":"2024-04-25T10:01:56.105198+0000","last_active":"2024-04-25T10:01:57.631051+0000","last_peered":"2024-04-25T10:01:57.631051+0000","last_clean":"2024-04-25T10:01:57.631051+0000","last_became_active":"2024-04-25T10:01:56.104858+0000","last_became_peered":"2024-04-25T10:01:56.104858+0000","last_unstale":"2024-04-25T10:01:57.631051+0000","last_undegraded":"2024-04-25T10:01:57.631051+0000","last_fullsized":"2024-04-25T10:01:57.631051+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:48.574592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632717+0000","last_change":"2024-04-25T10:01:51.994401+0000","last_active":"2024-04-25T10:01:57.632717+0000","last_peered":"2024-04-25T10:01:57.632717+0000","last_clean":"2024-04-25T10:01:57.632717+0000","last_became_active":"2024-04-25T10:01:51.994086+0000","last_became_peered":"2024-04-25T10:01:51.994086+0000","last_unstale":"2024-04-25T10:01:57.632717+0000","last_undegraded":"2024-04-25T10:01:57.632717+0000","last_fullsized":"2024-04-25T10:01:57.632717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101375+0000","last_change":"2024-04-25T10:01:50.950324+0000","last_active":"2024-04-25T10:01:57.101375+0000","last_peered":"2024-04-25T10:01:57.101375+0000","last_clean":"2024-04-25T10:01:57.101375+0000","last_became_active":"2024-04-25T10:01:50.949810+0000","last_became_peered":"2024-04-25T10:01:50.949810+0000","last_unstale":"2024-04-25T10:01:57.101375+0000","last_undegraded":"2024-04-25T10:01:57.101375+0000","last_fullsized":"2024-04-25T10:01:57.101375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346935+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:01:57.346935+0000","last_peered":"2024-04-25T10:01:57.346935+0000","last_clean":"2024-04-25T10:01:57.346935+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:01:57.346935+0000","last_undegraded":"2024-04-25T10:01:57.346935+0000","last_fullsized":"2024-04-25T10:01:57.346935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007399+0000","last_change":"2024-04-25T10:01:56.108134+0000","last_active":"2024-04-25T10:01:57.007399+0000","last_peered":"2024-04-25T10:01:57.007399+0000","last_clean":"2024-04-25T10:01:57.007399+0000","last_became_active":"2024-04-25T10:01:56.107123+0000","last_became_peered":"2024-04-25T10:01:56.107123+0000","last_unstale":"2024-04-25T10:01:57.007399+0000","last_undegraded":"2024-04-25T10:01:57.007399+0000","last_fullsized":"2024-04-25T10:01:57.007399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:03.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347779+0000","last_change":"2024-04-25T10:01:51.983906+0000","last_active":"2024-04-25T10:01:57.347779+0000","last_peered":"2024-04-25T10:01:57.347779+0000","last_clean":"2024-04-25T10:01:57.347779+0000","last_became_active":"2024-04-25T10:01:51.982118+0000","last_became_peered":"2024-04-25T10:01:51.982118+0000","last_unstale":"2024-04-25T10:01:57.347779+0000","last_undegraded":"2024-04-25T10:01:57.347779+0000","last_fullsized":"2024-04-25T10:01:57.347779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:14.191816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004595+0000","last_change":"2024-04-25T10:01:50.946054+0000","last_active":"2024-04-25T10:01:57.004595+0000","last_peered":"2024-04-25T10:01:57.004595+0000","last_clean":"2024-04-25T10:01:57.004595+0000","last_became_active":"2024-04-25T10:01:50.945744+0000","last_became_peered":"2024-04-25T10:01:50.945744+0000","last_unstale":"2024-04-25T10:01:57.004595+0000","last_undegraded":"2024-04-25T10:01:57.004595+0000","last_fullsized":"2024-04-25T10:01:57.004595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006974+0000","last_change":"2024-04-25T10:01:56.104590+0000","last_active":"2024-04-25T10:01:57.006974+0000","last_peered":"2024-04-25T10:01:57.006974+0000","last_clean":"2024-04-25T10:01:57.006974+0000","last_became_active":"2024-04-25T10:01:56.103498+0000","last_became_peered":"2024-04-25T10:01:56.103498+0000","last_unstale":"2024-04-25T10:01:57.006974+0000","last_undegraded":"2024-04-25T10:01:57.006974+0000","last_fullsized":"2024-04-25T10:01:57.006974+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101453+0000","last_change":"2024-04-25T10:01:51.968651+0000","last_active":"2024-04-25T10:01:57.101453+0000","last_peered":"2024-04-25T10:01:57.101453+0000","last_clean":"2024-04-25T10:01:57.101453+0000","last_became_active":"2024-04-25T10:01:51.962327+0000","last_became_peered":"2024-04-25T10:01:51.962327+0000","last_unstale":"2024-04-25T10:01:57.101453+0000","last_undegraded":"2024-04-25T10:01:57.101453+0000","last_fullsized":"2024-04-25T10:01:57.101453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:55.810277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629450+0000","last_change":"2024-04-25T10:01:50.942416+0000","last_active":"2024-04-25T10:01:57.629450+0000","last_peered":"2024-04-25T10:01:57.629450+0000","last_clean":"2024-04-25T10:01:57.629450+0000","last_became_active":"2024-04-25T10:01:50.942009+0000","last_became_peered":"2024-04-25T10:01:50.942009+0000","last_unstale":"2024-04-25T10:01:57.629450+0000","last_undegraded":"2024-04-25T10:01:57.629450+0000","last_fullsized":"2024-04-25T10:01:57.629450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:25.085344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630028+0000","last_change":"2024-04-25T10:01:56.023691+0000","last_active":"2024-04-25T10:01:57.630028+0000","last_peered":"2024-04-25T10:01:57.630028+0000","last_clean":"2024-04-25T10:01:57.630028+0000","last_became_active":"2024-04-25T10:01:56.022988+0000","last_became_peered":"2024-04-25T10:01:56.022988+0000","last_unstale":"2024-04-25T10:01:57.630028+0000","last_undegraded":"2024-04-25T10:01:57.630028+0000","last_fullsized":"2024-04-25T10:01:57.630028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347762+0000","last_change":"2024-04-25T10:01:51.987406+0000","last_active":"2024-04-25T10:01:57.347762+0000","last_peered":"2024-04-25T10:01:57.347762+0000","last_clean":"2024-04-25T10:01:57.347762+0000","last_became_active":"2024-04-25T10:01:51.986930+0000","last_became_peered":"2024-04-25T10:01:51.986930+0000","last_unstale":"2024-04-25T10:01:57.347762+0000","last_undegraded":"2024-04-25T10:01:57.347762+0000","last_fullsized":"2024-04-25T10:01:57.347762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:27.239681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004599+0000","last_change":"2024-04-25T10:01:50.932604+0000","last_active":"2024-04-25T10:01:57.004599+0000","last_peered":"2024-04-25T10:01:57.004599+0000","last_clean":"2024-04-25T10:01:57.004599+0000","last_became_active":"2024-04-25T10:01:50.932205+0000","last_became_peered":"2024-04-25T10:01:50.932205+0000","last_unstale":"2024-04-25T10:01:57.004599+0000","last_undegraded":"2024-04-25T10:01:57.004599+0000","last_fullsized":"2024-04-25T10:01:57.004599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:35.306064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100705+0000","last_change":"2024-04-25T10:01:56.105972+0000","last_active":"2024-04-25T10:01:57.100705+0000","last_peered":"2024-04-25T10:01:57.100705+0000","last_clean":"2024-04-25T10:01:57.100705+0000","last_became_active":"2024-04-25T10:01:56.104834+0000","last_became_peered":"2024-04-25T10:01:56.104834+0000","last_unstale":"2024-04-25T10:01:57.100705+0000","last_undegraded":"2024-04-25T10:01:57.100705+0000","last_fullsized":"2024-04-25T10:01:57.100705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347690+0000","last_change":"2024-04-25T10:01:51.991881+0000","last_active":"2024-04-25T10:01:57.347690+0000","last_peered":"2024-04-25T10:01:57.347690+0000","last_clean":"2024-04-25T10:01:57.347690+0000","last_became_active":"2024-04-25T10:01:51.991266+0000","last_became_peered":"2024-04-25T10:01:51.991266+0000","last_unstale":"2024-04-25T10:01:57.347690+0000","last_undegraded":"2024-04-25T10:01:57.347690+0000","last_fullsized":"2024-04-25T10:01:57.347690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:39.074042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009212+0000","last_change":"2024-04-25T10:01:50.945437+0000","last_active":"2024-04-25T10:01:57.009212+0000","last_peered":"2024-04-25T10:01:57.009212+0000","last_clean":"2024-04-25T10:01:57.009212+0000","last_became_active":"2024-04-25T10:01:50.944786+0000","last_became_peered":"2024-04-25T10:01:50.944786+0000","last_unstale":"2024-04-25T10:01:57.009212+0000","last_undegraded":"2024-04-25T10:01:57.009212+0000","last_fullsized":"2024-04-25T10:01:57.009212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:53.181549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005662+0000","last_change":"2024-04-25T10:01:56.000803+0000","last_active":"2024-04-25T10:01:57.005662+0000","last_peered":"2024-04-25T10:01:57.005662+0000","last_clean":"2024-04-25T10:01:57.005662+0000","last_became_active":"2024-04-25T10:01:55.999071+0000","last_became_peered":"2024-04-25T10:01:55.999071+0000","last_unstale":"2024-04-25T10:01:57.005662+0000","last_undegraded":"2024-04-25T10:01:57.005662+0000","last_fullsized":"2024-04-25T10:01:57.005662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:50.609054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347419+0000","last_change":"2024-04-25T10:01:51.982017+0000","last_active":"2024-04-25T10:01:57.347419+0000","last_peered":"2024-04-25T10:01:57.347419+0000","last_clean":"2024-04-25T10:01:57.347419+0000","last_became_active":"2024-04-25T10:01:51.981340+0000","last_became_peered":"2024-04-25T10:01:51.981340+0000","last_unstale":"2024-04-25T10:01:57.347419+0000","last_undegraded":"2024-04-25T10:01:57.347419+0000","last_fullsized":"2024-04-25T10:01:57.347419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:22.999554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004023+0000","last_change":"2024-04-25T10:01:50.934868+0000","last_active":"2024-04-25T10:01:57.004023+0000","last_peered":"2024-04-25T10:01:57.004023+0000","last_clean":"2024-04-25T10:01:57.004023+0000","last_became_active":"2024-04-25T10:01:50.934560+0000","last_became_peered":"2024-04-25T10:01:50.934560+0000","last_unstale":"2024-04-25T10:01:57.004023+0000","last_undegraded":"2024-04-25T10:01:57.004023+0000","last_fullsized":"2024-04-25T10:01:57.004023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630724+0000","last_change":"2024-04-25T10:01:56.012427+0000","last_active":"2024-04-25T10:01:57.630724+0000","last_peered":"2024-04-25T10:01:57.630724+0000","last_clean":"2024-04-25T10:01:57.630724+0000","last_became_active":"2024-04-25T10:01:56.008358+0000","last_became_peered":"2024-04-25T10:01:56.008358+0000","last_unstale":"2024-04-25T10:01:57.630724+0000","last_undegraded":"2024-04-25T10:01:57.630724+0000","last_fullsized":"2024-04-25T10:01:57.630724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:56.864362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006288+0000","last_change":"2024-04-25T10:01:51.947557+0000","last_active":"2024-04-25T10:01:57.006288+0000","last_peered":"2024-04-25T10:01:57.006288+0000","last_clean":"2024-04-25T10:01:57.006288+0000","last_became_active":"2024-04-25T10:01:51.942206+0000","last_became_peered":"2024-04-25T10:01:51.942206+0000","last_unstale":"2024-04-25T10:01:57.006288+0000","last_undegraded":"2024-04-25T10:01:57.006288+0000","last_fullsized":"2024-04-25T10:01:57.006288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006959+0000","last_change":"2024-04-25T10:01:50.930663+0000","last_active":"2024-04-25T10:01:57.006959+0000","last_peered":"2024-04-25T10:01:57.006959+0000","last_clean":"2024-04-25T10:01:57.006959+0000","last_became_active":"2024-04-25T10:01:50.930224+0000","last_became_peered":"2024-04-25T10:01:50.930224+0000","last_unstale":"2024-04-25T10:01:57.006959+0000","last_undegraded":"2024-04-25T10:01:57.006959+0000","last_fullsized":"2024-04-25T10:01:57.006959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634460+0000","last_change":"2024-04-25T10:01:56.110077+0000","last_active":"2024-04-25T10:01:57.634460+0000","last_peered":"2024-04-25T10:01:57.634460+0000","last_clean":"2024-04-25T10:01:57.634460+0000","last_became_active":"2024-04-25T10:01:56.109686+0000","last_became_peered":"2024-04-25T10:01:56.109686+0000","last_unstale":"2024-04-25T10:01:57.634460+0000","last_undegraded":"2024-04-25T10:01:57.634460+0000","last_fullsized":"2024-04-25T10:01:57.634460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:20.441998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347395+0000","last_change":"2024-04-25T10:01:51.981399+0000","last_active":"2024-04-25T10:01:57.347395+0000","last_peered":"2024-04-25T10:01:57.347395+0000","last_clean":"2024-04-25T10:01:57.347395+0000","last_became_active":"2024-04-25T10:01:51.980512+0000","last_became_peered":"2024-04-25T10:01:51.980512+0000","last_unstale":"2024-04-25T10:01:57.347395+0000","last_undegraded":"2024-04-25T10:01:57.347395+0000","last_fullsized":"2024-04-25T10:01:57.347395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:52.801927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101702+0000","last_change":"2024-04-25T10:01:50.949394+0000","last_active":"2024-04-25T10:01:57.101702+0000","last_peered":"2024-04-25T10:01:57.101702+0000","last_clean":"2024-04-25T10:01:57.101702+0000","last_became_active":"2024-04-25T10:01:50.948515+0000","last_became_peered":"2024-04-25T10:01:50.948515+0000","last_unstale":"2024-04-25T10:01:57.101702+0000","last_undegraded":"2024-04-25T10:01:57.101702+0000","last_fullsized":"2024-04-25T10:01:57.101702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633739+0000","last_change":"2024-04-25T10:01:56.103664+0000","last_active":"2024-04-25T10:01:57.633739+0000","last_peered":"2024-04-25T10:01:57.633739+0000","last_clean":"2024-04-25T10:01:57.633739+0000","last_became_active":"2024-04-25T10:01:56.103259+0000","last_became_peered":"2024-04-25T10:01:56.103259+0000","last_unstale":"2024-04-25T10:01:57.633739+0000","last_undegraded":"2024-04-25T10:01:57.633739+0000","last_fullsized":"2024-04-25T10:01:57.633739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:19.687438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629659+0000","last_change":"2024-04-25T10:01:51.946200+0000","last_active":"2024-04-25T10:01:57.629659+0000","last_peered":"2024-04-25T10:01:57.629659+0000","last_clean":"2024-04-25T10:01:57.629659+0000","last_became_active":"2024-04-25T10:01:51.945344+0000","last_became_peered":"2024-04-25T10:01:51.945344+0000","last_unstale":"2024-04-25T10:01:57.629659+0000","last_undegraded":"2024-04-25T10:01:57.629659+0000","last_fullsized":"2024-04-25T10:01:57.629659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:13.244438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629997+0000","last_change":"2024-04-25T10:01:50.943130+0000","last_active":"2024-04-25T10:01:57.629997+0000","last_peered":"2024-04-25T10:01:57.629997+0000","last_clean":"2024-04-25T10:01:57.629997+0000","last_became_active":"2024-04-25T10:01:50.942800+0000","last_became_peered":"2024-04-25T10:01:50.942800+0000","last_unstale":"2024-04-25T10:01:57.629997+0000","last_undegraded":"2024-04-25T10:01:57.629997+0000","last_fullsized":"2024-04-25T10:01:57.629997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:58.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.102211+0000","last_change":"2024-04-25T10:01:56.103005+0000","last_active":"2024-04-25T10:01:57.102211+0000","last_peered":"2024-04-25T10:01:57.102211+0000","last_clean":"2024-04-25T10:01:57.102211+0000","last_became_active":"2024-04-25T10:01:56.101534+0000","last_became_peered":"2024-04-25T10:01:56.101534+0000","last_unstale":"2024-04-25T10:01:57.102211+0000","last_undegraded":"2024-04-25T10:01:57.102211+0000","last_fullsized":"2024-04-25T10:01:57.102211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:31.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347218+0000","last_change":"2024-04-25T10:01:51.982629+0000","last_active":"2024-04-25T10:01:57.347218+0000","last_peered":"2024-04-25T10:01:57.347218+0000","last_clean":"2024-04-25T10:01:57.347218+0000","last_became_active":"2024-04-25T10:01:51.981703+0000","last_became_peered":"2024-04-25T10:01:51.981703+0000","last_unstale":"2024-04-25T10:01:57.347218+0000","last_undegraded":"2024-04-25T10:01:57.347218+0000","last_fullsized":"2024-04-25T10:01:57.347218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:44.512157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:02.186202+0000","last_change":"2024-04-25T10:01:50.936030+0000","last_active":"2024-04-25T10:02:02.186202+0000","last_peered":"2024-04-25T10:02:02.186202+0000","last_clean":"2024-04-25T10:02:02.186202+0000","last_became_active":"2024-04-25T10:01:50.935276+0000","last_became_peered":"2024-04-25T10:01:50.935276+0000","last_unstale":"2024-04-25T10:02:02.186202+0000","last_undegraded":"2024-04-25T10:02:02.186202+0000","last_fullsized":"2024-04-25T10:02:02.186202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627939+0000","last_change":"2024-04-25T10:01:56.029517+0000","last_active":"2024-04-25T10:01:57.627939+0000","last_peered":"2024-04-25T10:01:57.627939+0000","last_clean":"2024-04-25T10:01:57.627939+0000","last_became_active":"2024-04-25T10:01:56.018301+0000","last_became_peered":"2024-04-25T10:01:56.018301+0000","last_unstale":"2024-04-25T10:01:57.627939+0000","last_undegraded":"2024-04-25T10:01:57.627939+0000","last_fullsized":"2024-04-25T10:01:57.627939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:56.864362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008458+0000","last_change":"2024-04-25T10:01:51.952954+0000","last_active":"2024-04-25T10:01:57.008458+0000","last_peered":"2024-04-25T10:01:57.008458+0000","last_clean":"2024-04-25T10:01:57.008458+0000","last_became_active":"2024-04-25T10:01:51.952204+0000","last_became_peered":"2024-04-25T10:01:51.952204+0000","last_unstale":"2024-04-25T10:01:57.008458+0000","last_undegraded":"2024-04-25T10:01:57.008458+0000","last_fullsized":"2024-04-25T10:01:57.008458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:39.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629335+0000","last_change":"2024-04-25T10:01:50.950696+0000","last_active":"2024-04-25T10:01:57.629335+0000","last_peered":"2024-04-25T10:01:57.629335+0000","last_clean":"2024-04-25T10:01:57.629335+0000","last_became_active":"2024-04-25T10:01:50.950369+0000","last_became_peered":"2024-04-25T10:01:50.950369+0000","last_unstale":"2024-04-25T10:01:57.629335+0000","last_undegraded":"2024-04-25T10:01:57.629335+0000","last_fullsized":"2024-04-25T10:01:57.629335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:38.540445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009378+0000","last_change":"2024-04-25T10:01:56.025446+0000","last_active":"2024-04-25T10:01:57.009378+0000","last_peered":"2024-04-25T10:01:57.009378+0000","last_clean":"2024-04-25T10:01:57.009378+0000","last_became_active":"2024-04-25T10:01:56.023974+0000","last_became_peered":"2024-04-25T10:01:56.023974+0000","last_unstale":"2024-04-25T10:01:57.009378+0000","last_undegraded":"2024-04-25T10:01:57.009378+0000","last_fullsized":"2024-04-25T10:01:57.009378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631278+0000","last_change":"2024-04-25T10:01:51.969210+0000","last_active":"2024-04-25T10:01:57.631278+0000","last_peered":"2024-04-25T10:01:57.631278+0000","last_clean":"2024-04-25T10:01:57.631278+0000","last_became_active":"2024-04-25T10:01:51.968830+0000","last_became_peered":"2024-04-25T10:01:51.968830+0000","last_unstale":"2024-04-25T10:01:57.631278+0000","last_undegraded":"2024-04-25T10:01:57.631278+0000","last_fullsized":"2024-04-25T10:01:57.631278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101496+0000","last_change":"2024-04-25T10:01:50.948707+0000","last_active":"2024-04-25T10:01:57.101496+0000","last_peered":"2024-04-25T10:01:57.101496+0000","last_clean":"2024-04-25T10:01:57.101496+0000","last_became_active":"2024-04-25T10:01:50.947988+0000","last_became_peered":"2024-04-25T10:01:50.947988+0000","last_unstale":"2024-04-25T10:01:57.101496+0000","last_undegraded":"2024-04-25T10:01:57.101496+0000","last_fullsized":"2024-04-25T10:01:57.101496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631528+0000","last_change":"2024-04-25T10:01:56.017001+0000","last_active":"2024-04-25T10:01:57.631528+0000","last_peered":"2024-04-25T10:01:57.631528+0000","last_clean":"2024-04-25T10:01:57.631528+0000","last_became_active":"2024-04-25T10:01:56.016220+0000","last_became_peered":"2024-04-25T10:01:56.016220+0000","last_unstale":"2024-04-25T10:01:57.631528+0000","last_undegraded":"2024-04-25T10:01:57.631528+0000","last_fullsized":"2024-04-25T10:01:57.631528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:31.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101724+0000","last_change":"2024-04-25T10:01:51.978944+0000","last_active":"2024-04-25T10:01:57.101724+0000","last_peered":"2024-04-25T10:01:57.101724+0000","last_clean":"2024-04-25T10:01:57.101724+0000","last_became_active":"2024-04-25T10:01:51.978133+0000","last_became_peered":"2024-04-25T10:01:51.978133+0000","last_unstale":"2024-04-25T10:01:57.101724+0000","last_undegraded":"2024-04-25T10:01:57.101724+0000","last_fullsized":"2024-04-25T10:01:57.101724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007998+0000","last_change":"2024-04-25T10:01:50.953423+0000","last_active":"2024-04-25T10:01:57.007998+0000","last_peered":"2024-04-25T10:01:57.007998+0000","last_clean":"2024-04-25T10:01:57.007998+0000","last_became_active":"2024-04-25T10:01:50.952221+0000","last_became_peered":"2024-04-25T10:01:50.952221+0000","last_unstale":"2024-04-25T10:01:57.007998+0000","last_undegraded":"2024-04-25T10:01:57.007998+0000","last_fullsized":"2024-04-25T10:01:57.007998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631879+0000","last_change":"2024-04-25T10:01:56.110756+0000","last_active":"2024-04-25T10:01:57.631879+0000","last_peered":"2024-04-25T10:01:57.631879+0000","last_clean":"2024-04-25T10:01:57.631879+0000","last_became_active":"2024-04-25T10:01:56.109749+0000","last_became_peered":"2024-04-25T10:01:56.109749+0000","last_unstale":"2024-04-25T10:01:57.631879+0000","last_undegraded":"2024-04-25T10:01:57.631879+0000","last_fullsized":"2024-04-25T10:01:57.631879+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:48.763078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628771+0000","last_change":"2024-04-25T10:01:56.106331+0000","last_active":"2024-04-25T10:01:57.628771+0000","last_peered":"2024-04-25T10:01:57.628771+0000","last_clean":"2024-04-25T10:01:57.628771+0000","last_became_active":"2024-04-25T10:01:56.105560+0000","last_became_peered":"2024-04-25T10:01:56.105560+0000","last_unstale":"2024-04-25T10:01:57.628771+0000","last_undegraded":"2024-04-25T10:01:57.628771+0000","last_fullsized":"2024-04-25T10:01:57.628771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627589+0000","last_change":"2024-04-25T10:01:51.959889+0000","last_active":"2024-04-25T10:01:57.627589+0000","last_peered":"2024-04-25T10:01:57.627589+0000","last_clean":"2024-04-25T10:01:57.627589+0000","last_became_active":"2024-04-25T10:01:51.959609+0000","last_became_peered":"2024-04-25T10:01:51.959609+0000","last_unstale":"2024-04-25T10:01:57.627589+0000","last_undegraded":"2024-04-25T10:01:57.627589+0000","last_fullsized":"2024-04-25T10:01:57.627589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101160+0000","last_change":"2024-04-25T10:01:50.947616+0000","last_active":"2024-04-25T10:01:57.101160+0000","last_peered":"2024-04-25T10:01:57.101160+0000","last_clean":"2024-04-25T10:01:57.101160+0000","last_became_active":"2024-04-25T10:01:50.946930+0000","last_became_peered":"2024-04-25T10:01:50.946930+0000","last_unstale":"2024-04-25T10:01:57.101160+0000","last_undegraded":"2024-04-25T10:01:57.101160+0000","last_fullsized":"2024-04-25T10:01:57.101160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:33.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007713+0000","last_change":"2024-04-25T10:01:56.018502+0000","last_active":"2024-04-25T10:01:57.007713+0000","last_peered":"2024-04-25T10:01:57.007713+0000","last_clean":"2024-04-25T10:01:57.007713+0000","last_became_active":"2024-04-25T10:01:56.017379+0000","last_became_peered":"2024-04-25T10:01:56.017379+0000","last_unstale":"2024-04-25T10:01:57.007713+0000","last_undegraded":"2024-04-25T10:01:57.007713+0000","last_fullsized":"2024-04-25T10:01:57.007713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631645+0000","last_change":"2024-04-25T10:01:51.953615+0000","last_active":"2024-04-25T10:01:57.631645+0000","last_peered":"2024-04-25T10:01:57.631645+0000","last_clean":"2024-04-25T10:01:57.631645+0000","last_became_active":"2024-04-25T10:01:51.953362+0000","last_became_peered":"2024-04-25T10:01:51.953362+0000","last_unstale":"2024-04-25T10:01:57.631645+0000","last_undegraded":"2024-04-25T10:01:57.631645+0000","last_fullsized":"2024-04-25T10:01:57.631645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632491+0000","last_change":"2024-04-25T10:01:50.945460+0000","last_active":"2024-04-25T10:01:57.632491+0000","last_peered":"2024-04-25T10:01:57.632491+0000","last_clean":"2024-04-25T10:01:57.632491+0000","last_became_active":"2024-04-25T10:01:50.945086+0000","last_became_peered":"2024-04-25T10:01:50.945086+0000","last_unstale":"2024-04-25T10:01:57.632491+0000","last_undegraded":"2024-04-25T10:01:57.632491+0000","last_fullsized":"2024-04-25T10:01:57.632491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.010630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007381+0000","last_change":"2024-04-25T10:01:50.944204+0000","last_active":"2024-04-25T10:01:57.007381+0000","last_peered":"2024-04-25T10:01:57.007381+0000","last_clean":"2024-04-25T10:01:57.007381+0000","last_became_active":"2024-04-25T10:01:50.943478+0000","last_became_peered":"2024-04-25T10:01:50.943478+0000","last_unstale":"2024-04-25T10:01:57.007381+0000","last_undegraded":"2024-04-25T10:01:57.007381+0000","last_fullsized":"2024-04-25T10:01:57.007381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629902+0000","last_change":"2024-04-25T10:01:51.939383+0000","last_active":"2024-04-25T10:01:57.629902+0000","last_peered":"2024-04-25T10:01:57.629902+0000","last_clean":"2024-04-25T10:01:57.629902+0000","last_became_active":"2024-04-25T10:01:51.938491+0000","last_became_peered":"2024-04-25T10:01:51.938491+0000","last_unstale":"2024-04-25T10:01:57.629902+0000","last_undegraded":"2024-04-25T10:01:57.629902+0000","last_fullsized":"2024-04-25T10:01:57.629902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004475+0000","last_change":"2024-04-25T10:01:56.024292+0000","last_active":"2024-04-25T10:01:57.004475+0000","last_peered":"2024-04-25T10:01:57.004475+0000","last_clean":"2024-04-25T10:01:57.004475+0000","last_became_active":"2024-04-25T10:01:56.023778+0000","last_became_peered":"2024-04-25T10:01:56.023778+0000","last_unstale":"2024-04-25T10:01:57.004475+0000","last_undegraded":"2024-04-25T10:01:57.004475+0000","last_fullsized":"2024-04-25T10:01:57.004475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:30.157749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633372+0000","last_change":"2024-04-25T10:01:50.954588+0000","last_active":"2024-04-25T10:01:57.633372+0000","last_peered":"2024-04-25T10:01:57.633372+0000","last_clean":"2024-04-25T10:01:57.633372+0000","last_became_active":"2024-04-25T10:01:50.954119+0000","last_became_peered":"2024-04-25T10:01:50.954119+0000","last_unstale":"2024-04-25T10:01:57.633372+0000","last_undegraded":"2024-04-25T10:01:57.633372+0000","last_fullsized":"2024-04-25T10:01:57.633372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:35.306064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007283+0000","last_change":"2024-04-25T10:01:51.941227+0000","last_active":"2024-04-25T10:01:57.007283+0000","last_peered":"2024-04-25T10:01:57.007283+0000","last_clean":"2024-04-25T10:01:57.007283+0000","last_became_active":"2024-04-25T10:01:51.940116+0000","last_became_peered":"2024-04-25T10:01:51.940116+0000","last_unstale":"2024-04-25T10:01:57.007283+0000","last_undegraded":"2024-04-25T10:01:57.007283+0000","last_fullsized":"2024-04-25T10:01:57.007283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:49.915603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633711+0000","last_change":"2024-04-25T10:01:56.024226+0000","last_active":"2024-04-25T10:01:57.633711+0000","last_peered":"2024-04-25T10:01:57.633711+0000","last_clean":"2024-04-25T10:01:57.633711+0000","last_became_active":"2024-04-25T10:01:56.023568+0000","last_became_peered":"2024-04-25T10:01:56.023568+0000","last_unstale":"2024-04-25T10:01:57.633711+0000","last_undegraded":"2024-04-25T10:01:57.633711+0000","last_fullsized":"2024-04-25T10:01:57.633711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.434943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005947+0000","last_change":"2024-04-25T10:01:51.948178+0000","last_active":"2024-04-25T10:01:57.005947+0000","last_peered":"2024-04-25T10:01:57.005947+0000","last_clean":"2024-04-25T10:01:57.005947+0000","last_became_active":"2024-04-25T10:01:51.943159+0000","last_became_peered":"2024-04-25T10:01:51.943159+0000","last_unstale":"2024-04-25T10:01:57.005947+0000","last_undegraded":"2024-04-25T10:01:57.005947+0000","last_fullsized":"2024-04-25T10:01:57.005947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:13.244438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004086+0000","last_change":"2024-04-25T10:01:51.977923+0000","last_active":"2024-04-25T10:01:57.004086+0000","last_peered":"2024-04-25T10:01:57.004086+0000","last_clean":"2024-04-25T10:01:57.004086+0000","last_became_active":"2024-04-25T10:01:51.977601+0000","last_became_peered":"2024-04-25T10:01:51.977601+0000","last_unstale":"2024-04-25T10:01:57.004086+0000","last_undegraded":"2024-04-25T10:01:57.004086+0000","last_fullsized":"2024-04-25T10:01:57.004086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:39.550415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101440+0000","last_change":"2024-04-25T10:01:51.977323+0000","last_active":"2024-04-25T10:01:57.101440+0000","last_peered":"2024-04-25T10:01:57.101440+0000","last_clean":"2024-04-25T10:01:57.101440+0000","last_became_active":"2024-04-25T10:01:51.976662+0000","last_became_peered":"2024-04-25T10:01:51.976662+0000","last_unstale":"2024-04-25T10:01:57.101440+0000","last_undegraded":"2024-04-25T10:01:57.101440+0000","last_fullsized":"2024-04-25T10:01:57.101440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:52.801927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629808+0000","last_change":"2024-04-25T10:01:51.962727+0000","last_active":"2024-04-25T10:01:57.629808+0000","last_peered":"2024-04-25T10:01:57.629808+0000","last_clean":"2024-04-25T10:01:57.629808+0000","last_became_active":"2024-04-25T10:01:51.962048+0000","last_became_peered":"2024-04-25T10:01:51.962048+0000","last_unstale":"2024-04-25T10:01:57.629808+0000","last_undegraded":"2024-04-25T10:01:57.629808+0000","last_fullsized":"2024-04-25T10:01:57.629808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633852+0000","last_change":"2024-04-25T10:01:51.956859+0000","last_active":"2024-04-25T10:01:57.633852+0000","last_peered":"2024-04-25T10:01:57.633852+0000","last_clean":"2024-04-25T10:01:57.633852+0000","last_became_active":"2024-04-25T10:01:51.956531+0000","last_became_peered":"2024-04-25T10:01:51.956531+0000","last_unstale":"2024-04-25T10:01:57.633852+0000","last_undegraded":"2024-04-25T10:01:57.633852+0000","last_fullsized":"2024-04-25T10:01:57.633852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628823+0000","last_change":"2024-04-25T10:01:51.960245+0000","last_active":"2024-04-25T10:01:57.628823+0000","last_peered":"2024-04-25T10:01:57.628823+0000","last_clean":"2024-04-25T10:01:57.628823+0000","last_became_active":"2024-04-25T10:01:51.959954+0000","last_became_peered":"2024-04-25T10:01:51.959954+0000","last_unstale":"2024-04-25T10:01:57.628823+0000","last_undegraded":"2024-04-25T10:01:57.628823+0000","last_fullsized":"2024-04-25T10:01:57.628823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008265+0000","last_change":"2024-04-25T10:01:51.951215+0000","last_active":"2024-04-25T10:01:57.008265+0000","last_peered":"2024-04-25T10:01:57.008265+0000","last_clean":"2024-04-25T10:01:57.008265+0000","last_became_active":"2024-04-25T10:01:51.950168+0000","last_became_peered":"2024-04-25T10:01:51.950168+0000","last_unstale":"2024-04-25T10:01:57.008265+0000","last_undegraded":"2024-04-25T10:01:57.008265+0000","last_fullsized":"2024-04-25T10:01:57.008265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634588+0000","last_change":"2024-04-25T10:01:51.959477+0000","last_active":"2024-04-25T10:01:57.634588+0000","last_peered":"2024-04-25T10:01:57.634588+0000","last_clean":"2024-04-25T10:01:57.634588+0000","last_became_active":"2024-04-25T10:01:51.958740+0000","last_became_peered":"2024-04-25T10:01:51.958740+0000","last_unstale":"2024-04-25T10:01:57.634588+0000","last_undegraded":"2024-04-25T10:01:57.634588+0000","last_fullsized":"2024-04-25T10:01:57.634588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100898+0000","last_change":"2024-04-25T10:01:51.987810+0000","last_active":"2024-04-25T10:01:57.100898+0000","last_peered":"2024-04-25T10:01:57.100898+0000","last_clean":"2024-04-25T10:01:57.100898+0000","last_became_active":"2024-04-25T10:01:51.978504+0000","last_became_peered":"2024-04-25T10:01:51.978504+0000","last_unstale":"2024-04-25T10:01:57.100898+0000","last_undegraded":"2024-04-25T10:01:57.100898+0000","last_fullsized":"2024-04-25T10:01:57.100898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:44.512157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632135+0000","last_change":"2024-04-25T10:01:51.949190+0000","last_active":"2024-04-25T10:01:57.632135+0000","last_peered":"2024-04-25T10:01:57.632135+0000","last_clean":"2024-04-25T10:01:57.632135+0000","last_became_active":"2024-04-25T10:01:51.948886+0000","last_became_peered":"2024-04-25T10:01:51.948886+0000","last_unstale":"2024-04-25T10:01:57.632135+0000","last_undegraded":"2024-04-25T10:01:57.632135+0000","last_fullsized":"2024-04-25T10:01:57.632135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101158+0000","last_change":"2024-04-25T10:01:51.987833+0000","last_active":"2024-04-25T10:01:57.101158+0000","last_peered":"2024-04-25T10:01:57.101158+0000","last_clean":"2024-04-25T10:01:57.101158+0000","last_became_active":"2024-04-25T10:01:51.978154+0000","last_became_peered":"2024-04-25T10:01:51.978154+0000","last_unstale":"2024-04-25T10:01:57.101158+0000","last_undegraded":"2024-04-25T10:01:57.101158+0000","last_fullsized":"2024-04-25T10:01:57.101158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:22.999554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629659+0000","last_change":"2024-04-25T10:01:51.947940+0000","last_active":"2024-04-25T10:01:57.629659+0000","last_peered":"2024-04-25T10:01:57.629659+0000","last_clean":"2024-04-25T10:01:57.629659+0000","last_became_active":"2024-04-25T10:01:51.947672+0000","last_became_peered":"2024-04-25T10:01:51.947672+0000","last_unstale":"2024-04-25T10:01:57.629659+0000","last_undegraded":"2024-04-25T10:01:57.629659+0000","last_fullsized":"2024-04-25T10:01:57.629659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:39.087518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007150+0000","last_change":"2024-04-25T10:01:51.936013+0000","last_active":"2024-04-25T10:01:57.007150+0000","last_peered":"2024-04-25T10:01:57.007150+0000","last_clean":"2024-04-25T10:01:57.007150+0000","last_became_active":"2024-04-25T10:01:51.933077+0000","last_became_peered":"2024-04-25T10:01:51.933077+0000","last_unstale":"2024-04-25T10:01:57.007150+0000","last_undegraded":"2024-04-25T10:01:57.007150+0000","last_fullsized":"2024-04-25T10:01:57.007150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348064+0000","last_change":"2024-04-25T10:01:51.987425+0000","last_active":"2024-04-25T10:01:57.348064+0000","last_peered":"2024-04-25T10:01:57.348064+0000","last_clean":"2024-04-25T10:01:57.348064+0000","last_became_active":"2024-04-25T10:01:51.986698+0000","last_became_peered":"2024-04-25T10:01:51.986698+0000","last_unstale":"2024-04-25T10:01:57.348064+0000","last_undegraded":"2024-04-25T10:01:57.348064+0000","last_fullsized":"2024-04-25T10:01:57.348064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:28.226905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004602+0000","last_change":"2024-04-25T10:01:51.962633+0000","last_active":"2024-04-25T10:01:57.004602+0000","last_peered":"2024-04-25T10:01:57.004602+0000","last_clean":"2024-04-25T10:01:57.004602+0000","last_became_active":"2024-04-25T10:01:51.962235+0000","last_became_peered":"2024-04-25T10:01:51.962235+0000","last_unstale":"2024-04-25T10:01:57.004602+0000","last_undegraded":"2024-04-25T10:01:57.004602+0000","last_fullsized":"2024-04-25T10:01:57.004602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:39.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101861+0000","last_change":"2024-04-25T10:01:51.992557+0000","last_active":"2024-04-25T10:01:57.101861+0000","last_peered":"2024-04-25T10:01:57.101861+0000","last_clean":"2024-04-25T10:01:57.101861+0000","last_became_active":"2024-04-25T10:01:51.991982+0000","last_became_peered":"2024-04-25T10:01:51.991982+0000","last_unstale":"2024-04-25T10:01:57.101861+0000","last_undegraded":"2024-04-25T10:01:57.101861+0000","last_fullsized":"2024-04-25T10:01:57.101861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100581+0000","last_change":"2024-04-25T10:01:51.988776+0000","last_active":"2024-04-25T10:01:57.100581+0000","last_peered":"2024-04-25T10:01:57.100581+0000","last_clean":"2024-04-25T10:01:57.100581+0000","last_became_active":"2024-04-25T10:01:51.987983+0000","last_became_peered":"2024-04-25T10:01:51.987983+0000","last_unstale":"2024-04-25T10:01:57.100581+0000","last_undegraded":"2024-04-25T10:01:57.100581+0000","last_fullsized":"2024-04-25T10:01:57.100581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:39.074042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627816+0000","last_change":"2024-04-25T10:01:51.946481+0000","last_active":"2024-04-25T10:01:57.627816+0000","last_peered":"2024-04-25T10:01:57.627816+0000","last_clean":"2024-04-25T10:01:57.627816+0000","last_became_active":"2024-04-25T10:01:51.946161+0000","last_became_peered":"2024-04-25T10:01:51.946161+0000","last_unstale":"2024-04-25T10:01:57.627816+0000","last_undegraded":"2024-04-25T10:01:57.627816+0000","last_fullsized":"2024-04-25T10:01:57.627816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629440+0000","last_change":"2024-04-25T10:01:51.946480+0000","last_active":"2024-04-25T10:01:57.629440+0000","last_peered":"2024-04-25T10:01:57.629440+0000","last_clean":"2024-04-25T10:01:57.629440+0000","last_became_active":"2024-04-25T10:01:51.945872+0000","last_became_peered":"2024-04-25T10:01:51.945872+0000","last_unstale":"2024-04-25T10:01:57.629440+0000","last_undegraded":"2024-04-25T10:01:57.629440+0000","last_fullsized":"2024-04-25T10:01:57.629440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:44.793972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633094+0000","last_change":"2024-04-25T10:01:51.958336+0000","last_active":"2024-04-25T10:01:57.633094+0000","last_peered":"2024-04-25T10:01:57.633094+0000","last_clean":"2024-04-25T10:01:57.633094+0000","last_became_active":"2024-04-25T10:01:51.957993+0000","last_became_peered":"2024-04-25T10:01:51.957993+0000","last_unstale":"2024-04-25T10:01:57.633094+0000","last_undegraded":"2024-04-25T10:01:57.633094+0000","last_fullsized":"2024-04-25T10:01:57.633094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008236+0000","last_change":"2024-04-25T10:01:51.943174+0000","last_active":"2024-04-25T10:01:57.008236+0000","last_peered":"2024-04-25T10:01:57.008236+0000","last_clean":"2024-04-25T10:01:57.008236+0000","last_became_active":"2024-04-25T10:01:51.942327+0000","last_became_peered":"2024-04-25T10:01:51.942327+0000","last_unstale":"2024-04-25T10:01:57.008236+0000","last_undegraded":"2024-04-25T10:01:57.008236+0000","last_fullsized":"2024-04-25T10:01:57.008236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:22.581767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007882+0000","last_change":"2024-04-25T10:01:51.963532+0000","last_active":"2024-04-25T10:01:57.007882+0000","last_peered":"2024-04-25T10:01:57.007882+0000","last_clean":"2024-04-25T10:01:57.007882+0000","last_became_active":"2024-04-25T10:01:51.962793+0000","last_became_peered":"2024-04-25T10:01:51.962793+0000","last_unstale":"2024-04-25T10:01:57.007882+0000","last_undegraded":"2024-04-25T10:01:57.007882+0000","last_fullsized":"2024-04-25T10:01:57.007882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008103+0000","last_change":"2024-04-25T10:01:51.970144+0000","last_active":"2024-04-25T10:01:57.008103+0000","last_peered":"2024-04-25T10:01:57.008103+0000","last_clean":"2024-04-25T10:01:57.008103+0000","last_became_active":"2024-04-25T10:01:51.967189+0000","last_became_peered":"2024-04-25T10:01:51.967189+0000","last_unstale":"2024-04-25T10:01:57.008103+0000","last_undegraded":"2024-04-25T10:01:57.008103+0000","last_fullsized":"2024-04-25T10:01:57.008103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347824+0000","last_change":"2024-04-25T10:01:51.995155+0000","last_active":"2024-04-25T10:01:57.347824+0000","last_peered":"2024-04-25T10:01:57.347824+0000","last_clean":"2024-04-25T10:01:57.347824+0000","last_became_active":"2024-04-25T10:01:51.994514+0000","last_became_peered":"2024-04-25T10:01:51.994514+0000","last_unstale":"2024-04-25T10:01:57.347824+0000","last_undegraded":"2024-04-25T10:01:57.347824+0000","last_fullsized":"2024-04-25T10:01:57.347824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:36.570949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631050+0000","last_change":"2024-04-25T10:01:51.977705+0000","last_active":"2024-04-25T10:01:57.631050+0000","last_peered":"2024-04-25T10:01:57.631050+0000","last_clean":"2024-04-25T10:01:57.631050+0000","last_became_active":"2024-04-25T10:01:51.977373+0000","last_became_peered":"2024-04-25T10:01:51.977373+0000","last_unstale":"2024-04-25T10:01:57.631050+0000","last_undegraded":"2024-04-25T10:01:57.631050+0000","last_fullsized":"2024-04-25T10:01:57.631050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347155+0000","last_change":"2024-04-25T10:01:51.958773+0000","last_active":"2024-04-25T10:01:57.347155+0000","last_peered":"2024-04-25T10:01:57.347155+0000","last_clean":"2024-04-25T10:01:57.347155+0000","last_became_active":"2024-04-25T10:01:51.958015+0000","last_became_peered":"2024-04-25T10:01:51.958015+0000","last_unstale":"2024-04-25T10:01:57.347155+0000","last_undegraded":"2024-04-25T10:01:57.347155+0000","last_fullsized":"2024-04-25T10:01:57.347155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100869+0000","last_change":"2024-04-25T10:01:51.994172+0000","last_active":"2024-04-25T10:01:57.100869+0000","last_peered":"2024-04-25T10:01:57.100869+0000","last_clean":"2024-04-25T10:01:57.100869+0000","last_became_active":"2024-04-25T10:01:51.993498+0000","last_became_peered":"2024-04-25T10:01:51.993498+0000","last_unstale":"2024-04-25T10:01:57.100869+0000","last_undegraded":"2024-04-25T10:01:57.100869+0000","last_fullsized":"2024-04-25T10:01:57.100869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:59.098969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346909+0000","last_change":"2024-04-25T10:01:51.958621+0000","last_active":"2024-04-25T10:01:57.346909+0000","last_peered":"2024-04-25T10:01:57.346909+0000","last_clean":"2024-04-25T10:01:57.346909+0000","last_became_active":"2024-04-25T10:01:51.957937+0000","last_became_peered":"2024-04-25T10:01:51.957937+0000","last_unstale":"2024-04-25T10:01:57.346909+0000","last_undegraded":"2024-04-25T10:01:57.346909+0000","last_fullsized":"2024-04-25T10:01:57.346909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:59.098969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101663+0000","last_change":"2024-04-25T10:01:51.967673+0000","last_active":"2024-04-25T10:01:57.101663+0000","last_peered":"2024-04-25T10:01:57.101663+0000","last_clean":"2024-04-25T10:01:57.101663+0000","last_became_active":"2024-04-25T10:01:51.966329+0000","last_became_peered":"2024-04-25T10:01:51.966329+0000","last_unstale":"2024-04-25T10:01:57.101663+0000","last_undegraded":"2024-04-25T10:01:57.101663+0000","last_fullsized":"2024-04-25T10:01:57.101663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630321+0000","last_change":"2024-04-25T10:01:51.950122+0000","last_active":"2024-04-25T10:01:57.630321+0000","last_peered":"2024-04-25T10:01:57.630321+0000","last_clean":"2024-04-25T10:01:57.630321+0000","last_became_active":"2024-04-25T10:01:51.949559+0000","last_became_peered":"2024-04-25T10:01:51.949559+0000","last_unstale":"2024-04-25T10:01:57.630321+0000","last_undegraded":"2024-04-25T10:01:57.630321+0000","last_fullsized":"2024-04-25T10:01:57.630321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:52.021045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629583+0000","last_change":"2024-04-25T10:01:51.943127+0000","last_active":"2024-04-25T10:01:57.629583+0000","last_peered":"2024-04-25T10:01:57.629583+0000","last_clean":"2024-04-25T10:01:57.629583+0000","last_became_active":"2024-04-25T10:01:51.942304+0000","last_became_peered":"2024-04-25T10:01:51.942304+0000","last_unstale":"2024-04-25T10:01:57.629583+0000","last_undegraded":"2024-04-25T10:01:57.629583+0000","last_fullsized":"2024-04-25T10:01:57.629583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347294+0000","last_change":"2024-04-25T10:01:51.987276+0000","last_active":"2024-04-25T10:01:57.347294+0000","last_peered":"2024-04-25T10:01:57.347294+0000","last_clean":"2024-04-25T10:01:57.347294+0000","last_became_active":"2024-04-25T10:01:51.986766+0000","last_became_peered":"2024-04-25T10:01:51.986766+0000","last_unstale":"2024-04-25T10:01:57.347294+0000","last_undegraded":"2024-04-25T10:01:57.347294+0000","last_fullsized":"2024-04-25T10:01:57.347294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:57.724919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:16.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:16.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.260+0000 7f40268d1640 1 Processor -- start 2024-04-25T10:02:16.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.260+0000 7f40268d1640 1 -- start start 2024-04-25T10:02:16.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f40268d1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4020158110 0x7f40200b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f40268d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40201584f0 0x7f402015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f40268d1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f402006e460 0x7f402006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f40268d1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402006edf0 con 0x7f402006e460 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f40268d1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40200cbe60 con 0x7f4020158110 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f40268d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40200b19b0 con 0x7f40201584f0 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f4024e6e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f402006e460 0x7f402006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f4024e6e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f402006e460 0x7f402006e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48254/0 (socket says 172.21.15.133:48254) 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f4024e6e640 1 -- 172.21.15.133:0/2746880290 learned_addr learned my addr 172.21.15.133:0/2746880290 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f401ffff640 1 --2- 172.21.15.133:0/2746880290 >> v2:172.21.15.162:3300/0 conn(0x7f4020158110 0x7f40200b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f401f7fe640 1 --2- 172.21.15.133:0/2746880290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40201584f0 0x7f402015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f4024e6e640 1 -- 172.21.15.133:0/2746880290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40201584f0 msgr2=0x7f402015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f4024e6e640 1 --2- 172.21.15.133:0/2746880290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40201584f0 0x7f402015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f4024e6e640 1 -- 172.21.15.133:0/2746880290 >> v2:172.21.15.162:3300/0 conn(0x7f4020158110 msgr2=0x7f40200b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f4024e6e640 1 --2- 172.21.15.133:0/2746880290 >> v2:172.21.15.162:3300/0 conn(0x7f4020158110 0x7f40200b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f4024e6e640 1 -- 172.21.15.133:0/2746880290 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402015b360 con 0x7f402006e460 2024-04-25T10:02:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f4024e6e640 1 --2- 172.21.15.133:0/2746880290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402006e460 0x7f402006e8b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40200b4d40 tx=0x7f401801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43330ab68b3d863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f401effd640 1 -- 172.21.15.133:0/2746880290 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40180a5840 con 0x7f402006e460 2024-04-25T10:02:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f401effd640 1 -- 172.21.15.133:0/2746880290 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40180a59e0 con 0x7f402006e460 2024-04-25T10:02:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.264+0000 7f401effd640 1 -- 172.21.15.133:0/2746880290 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40180a5db0 con 0x7f402006e460 2024-04-25T10:02:16.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- 172.21.15.133:0/2746880290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402006e460 msgr2=0x7f402006e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 --2- 172.21.15.133:0/2746880290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402006e460 0x7f402006e8b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40200b4d40 tx=0x7f401801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- 172.21.15.133:0/2746880290 shutdown_connections 2024-04-25T10:02:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 --2- 172.21.15.133:0/2746880290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402006e460 0x7f402006e8b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 --2- 172.21.15.133:0/2746880290 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40201584f0 0x7f402015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 --2- 172.21.15.133:0/2746880290 >> v2:172.21.15.162:3300/0 conn(0x7f4020158110 0x7f40200b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- 172.21.15.133:0/2746880290 >> 172.21.15.133:0/2746880290 conn(0x7f4020067000 msgr2=0x7f402006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- 172.21.15.133:0/2746880290 shutdown_connections 2024-04-25T10:02:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- 172.21.15.133:0/2746880290 wait complete. 2024-04-25T10:02:16.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 Processor -- start 2024-04-25T10:02:16.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- start start 2024-04-25T10:02:16.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f402006e460 0x7f402015c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4020158110 0x7f40201550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40201584f0 0x7f40201513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40200cd7f0 con 0x7f4020158110 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40200cd670 con 0x7f40201584f0 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402015eb10 con 0x7f402006e460 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4020158110 0x7f40201550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4020158110 0x7f40201550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48266/0 (socket says 172.21.15.133:48266) 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401f7fe640 1 -- 172.21.15.133:0/3746926526 learned_addr learned my addr 172.21.15.133:0/3746926526 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401ffff640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f402006e460 0x7f402015c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f4024e6e640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40201584f0 0x7f40201513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401f7fe640 1 -- 172.21.15.133:0/3746926526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f402006e460 msgr2=0x7f402015c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401f7fe640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f402006e460 0x7f402015c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401f7fe640 1 -- 172.21.15.133:0/3746926526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40201584f0 msgr2=0x7f40201513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401f7fe640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40201584f0 0x7f40201513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401f7fe640 1 -- 172.21.15.133:0/3746926526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402015dad0 con 0x7f4020158110 2024-04-25T10:02:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401f7fe640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4020158110 0x7f40201550f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f401400d5f0 tx=0x7f401400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401cff9640 1 -- 172.21.15.133:0/3746926526 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140125f0 con 0x7f4020158110 2024-04-25T10:02:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401cff9640 1 -- 172.21.15.133:0/3746926526 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4014019070 con 0x7f4020158110 2024-04-25T10:02:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- 172.21.15.133:0/3746926526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402015ddb0 con 0x7f4020158110 2024-04-25T10:02:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f401cff9640 1 -- 172.21.15.133:0/3746926526 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140129b0 con 0x7f4020158110 2024-04-25T10:02:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- 172.21.15.133:0/3746926526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402015c190 con 0x7f4020158110 2024-04-25T10:02:16.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.268+0000 7f40268d1640 1 -- 172.21.15.133:0/3746926526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402006bc20 con 0x7f4020158110 2024-04-25T10:02:16.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.272+0000 7f401cff9640 1 -- 172.21.15.133:0/3746926526 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4014012b50 con 0x7f4020158110 2024-04-25T10:02:16.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.272+0000 7f401cff9640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f400c066f90 0x7f400c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.272+0000 7f401cff9640 1 -- 172.21.15.133:0/3746926526 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4014085750 con 0x7f4020158110 2024-04-25T10:02:16.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.272+0000 7f401ffff640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f400c066f90 0x7f400c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.272+0000 7f401ffff640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f400c066f90 0x7f400c069450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f401000ba30 tx=0x7f4010008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.276+0000 7f401cff9640 1 -- 172.21.15.133:0/3746926526 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4014056400 con 0x7f4020158110 2024-04-25T10:02:16.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.428+0000 7f40268d1640 1 -- 172.21.15.133:0/3746926526 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4020068ef0 con 0x7f400c066f90 2024-04-25T10:02:16.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.452+0000 7f401cff9640 1 -- 172.21.15.133:0/3746926526 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f4020068ef0 con 0x7f400c066f90 2024-04-25T10:02:16.461 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:16.464 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:02:16.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 -- 172.21.15.133:0/3746926526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f400c066f90 msgr2=0x7f400c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f400c066f90 0x7f400c069450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f401000ba30 tx=0x7f4010008040 comp rx=0 tx=0).stop 2024-04-25T10:02:16.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 -- 172.21.15.133:0/3746926526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4020158110 msgr2=0x7f40201550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4020158110 0x7f40201550f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f401400d5f0 tx=0x7f401400dab0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 -- 172.21.15.133:0/3746926526 shutdown_connections 2024-04-25T10:02:16.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40201584f0 0x7f40201513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f400c066f90 0x7f400c069450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4020158110 0x7f40201550f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 --2- 172.21.15.133:0/3746926526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f402006e460 0x7f402015c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 -- 172.21.15.133:0/3746926526 >> 172.21.15.133:0/3746926526 conn(0x7f4020067000 msgr2=0x7f40200cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:16.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 -- 172.21.15.133:0/3746926526 shutdown_connections 2024-04-25T10:02:16.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.456+0000 7f40268d1640 1 -- 172.21.15.133:0/3746926526 wait complete. 2024-04-25T10:02:16.479 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-25T10:02:15.710509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219388,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755332,"statfs":{"total":773094113280,"available":772869459968,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004059"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631430+0000","last_change":"2024-04-25T10:01:50.953534+0000","last_active":"2024-04-25T10:01:57.631430+0000","last_peered":"2024-04-25T10:01:57.631430+0000","last_clean":"2024-04-25T10:01:57.631430+0000","last_became_active":"2024-04-25T10:01:50.953054+0000","last_became_peered":"2024-04-25T10:01:50.953054+0000","last_unstale":"2024-04-25T10:01:57.631430+0000","last_undegraded":"2024-04-25T10:01:57.631430+0000","last_fullsized":"2024-04-25T10:01:57.631430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:19.695230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633973+0000","last_change":"2024-04-25T10:01:51.958018+0000","last_active":"2024-04-25T10:01:57.633973+0000","last_peered":"2024-04-25T10:01:57.633973+0000","last_clean":"2024-04-25T10:01:57.633973+0000","last_became_active":"2024-04-25T10:01:51.957717+0000","last_became_peered":"2024-04-25T10:01:51.957717+0000","last_unstale":"2024-04-25T10:01:57.633973+0000","last_undegraded":"2024-04-25T10:01:57.633973+0000","last_fullsized":"2024-04-25T10:01:57.633973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:55.810277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631489+0000","last_change":"2024-04-25T10:01:56.111100+0000","last_active":"2024-04-25T10:01:57.631489+0000","last_peered":"2024-04-25T10:01:57.631489+0000","last_clean":"2024-04-25T10:01:57.631489+0000","last_became_active":"2024-04-25T10:01:56.110327+0000","last_became_peered":"2024-04-25T10:01:56.110327+0000","last_unstale":"2024-04-25T10:01:57.631489+0000","last_undegraded":"2024-04-25T10:01:57.631489+0000","last_fullsized":"2024-04-25T10:01:57.631489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634369+0000","last_change":"2024-04-25T10:01:50.940954+0000","last_active":"2024-04-25T10:01:57.634369+0000","last_peered":"2024-04-25T10:01:57.634369+0000","last_clean":"2024-04-25T10:01:57.634369+0000","last_became_active":"2024-04-25T10:01:50.940601+0000","last_became_peered":"2024-04-25T10:01:50.940601+0000","last_unstale":"2024-04-25T10:01:57.634369+0000","last_undegraded":"2024-04-25T10:01:57.634369+0000","last_fullsized":"2024-04-25T10:01:57.634369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631026+0000","last_change":"2024-04-25T10:01:51.994632+0000","last_active":"2024-04-25T10:01:57.631026+0000","last_peered":"2024-04-25T10:01:57.631026+0000","last_clean":"2024-04-25T10:01:57.631026+0000","last_became_active":"2024-04-25T10:01:51.994229+0000","last_became_peered":"2024-04-25T10:01:51.994229+0000","last_unstale":"2024-04-25T10:01:57.631026+0000","last_undegraded":"2024-04-25T10:01:57.631026+0000","last_fullsized":"2024-04-25T10:01:57.631026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005703+0000","last_change":"2024-04-25T10:01:56.024604+0000","last_active":"2024-04-25T10:01:57.005703+0000","last_peered":"2024-04-25T10:01:57.005703+0000","last_clean":"2024-04-25T10:01:57.005703+0000","last_became_active":"2024-04-25T10:01:56.023667+0000","last_became_peered":"2024-04-25T10:01:56.023667+0000","last_unstale":"2024-04-25T10:01:57.005703+0000","last_undegraded":"2024-04-25T10:01:57.005703+0000","last_fullsized":"2024-04-25T10:01:57.005703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:50.018687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348063+0000","last_change":"2024-04-25T10:01:50.957056+0000","last_active":"2024-04-25T10:01:57.348063+0000","last_peered":"2024-04-25T10:01:57.348063+0000","last_clean":"2024-04-25T10:01:57.348063+0000","last_became_active":"2024-04-25T10:01:50.956522+0000","last_became_peered":"2024-04-25T10:01:50.956522+0000","last_unstale":"2024-04-25T10:01:57.348063+0000","last_undegraded":"2024-04-25T10:01:57.348063+0000","last_fullsized":"2024-04-25T10:01:57.348063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:25.085344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101183+0000","last_change":"2024-04-25T10:01:51.987821+0000","last_active":"2024-04-25T10:01:57.101183+0000","last_peered":"2024-04-25T10:01:57.101183+0000","last_clean":"2024-04-25T10:01:57.101183+0000","last_became_active":"2024-04-25T10:01:51.978435+0000","last_became_peered":"2024-04-25T10:01:51.978435+0000","last_unstale":"2024-04-25T10:01:57.101183+0000","last_undegraded":"2024-04-25T10:01:57.101183+0000","last_fullsized":"2024-04-25T10:01:57.101183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.714429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009007+0000","last_change":"2024-04-25T10:01:56.024538+0000","last_active":"2024-04-25T10:01:57.009007+0000","last_peered":"2024-04-25T10:01:57.009007+0000","last_clean":"2024-04-25T10:01:57.009007+0000","last_became_active":"2024-04-25T10:01:56.023560+0000","last_became_peered":"2024-04-25T10:01:56.023560+0000","last_unstale":"2024-04-25T10:01:57.009007+0000","last_undegraded":"2024-04-25T10:01:57.009007+0000","last_fullsized":"2024-04-25T10:01:57.009007+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.432097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347087+0000","last_change":"2024-04-25T10:01:50.954944+0000","last_active":"2024-04-25T10:01:57.347087+0000","last_peered":"2024-04-25T10:01:57.347087+0000","last_clean":"2024-04-25T10:01:57.347087+0000","last_became_active":"2024-04-25T10:01:50.954098+0000","last_became_peered":"2024-04-25T10:01:50.954098+0000","last_unstale":"2024-04-25T10:01:57.347087+0000","last_undegraded":"2024-04-25T10:01:57.347087+0000","last_fullsized":"2024-04-25T10:01:57.347087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:51.011075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101791+0000","last_change":"2024-04-25T10:01:51.976877+0000","last_active":"2024-04-25T10:01:57.101791+0000","last_peered":"2024-04-25T10:01:57.101791+0000","last_clean":"2024-04-25T10:01:57.101791+0000","last_became_active":"2024-04-25T10:01:51.967649+0000","last_became_peered":"2024-04-25T10:01:51.967649+0000","last_unstale":"2024-04-25T10:01:57.101791+0000","last_undegraded":"2024-04-25T10:01:57.101791+0000","last_fullsized":"2024-04-25T10:01:57.101791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006978+0000","last_change":"2024-04-25T10:01:56.018179+0000","last_active":"2024-04-25T10:01:57.006978+0000","last_peered":"2024-04-25T10:01:57.006978+0000","last_clean":"2024-04-25T10:01:57.006978+0000","last_became_active":"2024-04-25T10:01:56.017199+0000","last_became_peered":"2024-04-25T10:01:56.017199+0000","last_unstale":"2024-04-25T10:01:57.006978+0000","last_undegraded":"2024-04-25T10:01:57.006978+0000","last_fullsized":"2024-04-25T10:01:57.006978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:26.814142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007115+0000","last_change":"2024-04-25T10:01:51.935248+0000","last_active":"2024-04-25T10:01:57.007115+0000","last_peered":"2024-04-25T10:01:57.007115+0000","last_clean":"2024-04-25T10:01:57.007115+0000","last_became_active":"2024-04-25T10:01:51.932825+0000","last_became_peered":"2024-04-25T10:01:51.932825+0000","last_unstale":"2024-04-25T10:01:57.007115+0000","last_undegraded":"2024-04-25T10:01:57.007115+0000","last_fullsized":"2024-04-25T10:01:57.007115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347924+0000","last_change":"2024-04-25T10:01:50.956325+0000","last_active":"2024-04-25T10:01:57.347924+0000","last_peered":"2024-04-25T10:01:57.347924+0000","last_clean":"2024-04-25T10:01:57.347924+0000","last_became_active":"2024-04-25T10:01:50.954154+0000","last_became_peered":"2024-04-25T10:01:50.954154+0000","last_unstale":"2024-04-25T10:01:57.347924+0000","last_undegraded":"2024-04-25T10:01:57.347924+0000","last_fullsized":"2024-04-25T10:01:57.347924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:32.182247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008038+0000","last_change":"2024-04-25T10:01:56.024877+0000","last_active":"2024-04-25T10:01:57.008038+0000","last_peered":"2024-04-25T10:01:57.008038+0000","last_clean":"2024-04-25T10:01:57.008038+0000","last_became_active":"2024-04-25T10:01:56.023788+0000","last_became_peered":"2024-04-25T10:01:56.023788+0000","last_unstale":"2024-04-25T10:01:57.008038+0000","last_undegraded":"2024-04-25T10:01:57.008038+0000","last_fullsized":"2024-04-25T10:01:57.008038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:52.776865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633155+0000","last_change":"2024-04-25T10:01:51.952977+0000","last_active":"2024-04-25T10:01:57.633155+0000","last_peered":"2024-04-25T10:01:57.633155+0000","last_clean":"2024-04-25T10:01:57.633155+0000","last_became_active":"2024-04-25T10:01:51.951977+0000","last_became_peered":"2024-04-25T10:01:51.951977+0000","last_unstale":"2024-04-25T10:01:57.633155+0000","last_undegraded":"2024-04-25T10:01:57.633155+0000","last_fullsized":"2024-04-25T10:01:57.633155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.714429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630222+0000","last_change":"2024-04-25T10:01:50.934823+0000","last_active":"2024-04-25T10:01:57.630222+0000","last_peered":"2024-04-25T10:01:57.630222+0000","last_clean":"2024-04-25T10:01:57.630222+0000","last_became_active":"2024-04-25T10:01:50.934535+0000","last_became_peered":"2024-04-25T10:01:50.934535+0000","last_unstale":"2024-04-25T10:01:57.630222+0000","last_undegraded":"2024-04-25T10:01:57.630222+0000","last_fullsized":"2024-04-25T10:01:57.630222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007823+0000","last_change":"2024-04-25T10:01:56.017858+0000","last_active":"2024-04-25T10:01:57.007823+0000","last_peered":"2024-04-25T10:01:57.007823+0000","last_clean":"2024-04-25T10:01:57.007823+0000","last_became_active":"2024-04-25T10:01:56.016904+0000","last_became_peered":"2024-04-25T10:01:56.016904+0000","last_unstale":"2024-04-25T10:01:57.007823+0000","last_undegraded":"2024-04-25T10:01:57.007823+0000","last_fullsized":"2024-04-25T10:01:57.007823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:38.512242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631872+0000","last_change":"2024-04-25T10:01:51.948751+0000","last_active":"2024-04-25T10:01:57.631872+0000","last_peered":"2024-04-25T10:01:57.631872+0000","last_clean":"2024-04-25T10:01:57.631872+0000","last_became_active":"2024-04-25T10:01:51.946530+0000","last_became_peered":"2024-04-25T10:01:51.946530+0000","last_unstale":"2024-04-25T10:01:57.631872+0000","last_undegraded":"2024-04-25T10:01:57.631872+0000","last_fullsized":"2024-04-25T10:01:57.631872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627984+0000","last_change":"2024-04-25T10:01:50.953620+0000","last_active":"2024-04-25T10:01:57.627984+0000","last_peered":"2024-04-25T10:01:57.627984+0000","last_clean":"2024-04-25T10:01:57.627984+0000","last_became_active":"2024-04-25T10:01:50.953363+0000","last_became_peered":"2024-04-25T10:01:50.953363+0000","last_unstale":"2024-04-25T10:01:57.627984+0000","last_undegraded":"2024-04-25T10:01:57.627984+0000","last_fullsized":"2024-04-25T10:01:57.627984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629574+0000","last_change":"2024-04-25T10:01:56.111935+0000","last_active":"2024-04-25T10:01:57.629574+0000","last_peered":"2024-04-25T10:01:57.629574+0000","last_clean":"2024-04-25T10:01:57.629574+0000","last_became_active":"2024-04-25T10:01:56.111568+0000","last_became_peered":"2024-04-25T10:01:56.111568+0000","last_unstale":"2024-04-25T10:01:57.629574+0000","last_undegraded":"2024-04-25T10:01:57.629574+0000","last_fullsized":"2024-04-25T10:01:57.629574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:18.254251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006627+0000","last_change":"2024-04-25T10:01:51.965350+0000","last_active":"2024-04-25T10:01:57.006627+0000","last_peered":"2024-04-25T10:01:57.006627+0000","last_clean":"2024-04-25T10:01:57.006627+0000","last_became_active":"2024-04-25T10:01:51.964456+0000","last_became_peered":"2024-04-25T10:01:51.964456+0000","last_unstale":"2024-04-25T10:01:57.006627+0000","last_undegraded":"2024-04-25T10:01:57.006627+0000","last_fullsized":"2024-04-25T10:01:57.006627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:59.032520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346651+0000","last_change":"2024-04-25T10:01:50.954332+0000","last_active":"2024-04-25T10:01:57.346651+0000","last_peered":"2024-04-25T10:01:57.346651+0000","last_clean":"2024-04-25T10:01:57.346651+0000","last_became_active":"2024-04-25T10:01:50.953854+0000","last_became_peered":"2024-04-25T10:01:50.953854+0000","last_unstale":"2024-04-25T10:01:57.346651+0000","last_undegraded":"2024-04-25T10:01:57.346651+0000","last_fullsized":"2024-04-25T10:01:57.346651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:38.278178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008718+0000","last_change":"2024-04-25T10:01:56.017972+0000","last_active":"2024-04-25T10:01:57.008718+0000","last_peered":"2024-04-25T10:01:57.008718+0000","last_clean":"2024-04-25T10:01:57.008718+0000","last_became_active":"2024-04-25T10:01:56.016953+0000","last_became_peered":"2024-04-25T10:01:56.016953+0000","last_unstale":"2024-04-25T10:01:57.008718+0000","last_undegraded":"2024-04-25T10:01:57.008718+0000","last_fullsized":"2024-04-25T10:01:57.008718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:59.872712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630912+0000","last_change":"2024-04-25T10:01:51.944937+0000","last_active":"2024-04-25T10:01:57.630912+0000","last_peered":"2024-04-25T10:01:57.630912+0000","last_clean":"2024-04-25T10:01:57.630912+0000","last_became_active":"2024-04-25T10:01:51.944599+0000","last_became_peered":"2024-04-25T10:01:51.944599+0000","last_unstale":"2024-04-25T10:01:57.630912+0000","last_undegraded":"2024-04-25T10:01:57.630912+0000","last_fullsized":"2024-04-25T10:01:57.630912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008241+0000","last_change":"2024-04-25T10:01:50.955661+0000","last_active":"2024-04-25T10:01:57.008241+0000","last_peered":"2024-04-25T10:01:57.008241+0000","last_clean":"2024-04-25T10:01:57.008241+0000","last_became_active":"2024-04-25T10:01:50.954955+0000","last_became_peered":"2024-04-25T10:01:50.954955+0000","last_unstale":"2024-04-25T10:01:57.008241+0000","last_undegraded":"2024-04-25T10:01:57.008241+0000","last_fullsized":"2024-04-25T10:01:57.008241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:33.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006345+0000","last_change":"2024-04-25T10:01:56.108996+0000","last_active":"2024-04-25T10:01:57.006345+0000","last_peered":"2024-04-25T10:01:57.006345+0000","last_clean":"2024-04-25T10:01:57.006345+0000","last_became_active":"2024-04-25T10:01:56.107964+0000","last_became_peered":"2024-04-25T10:01:56.107964+0000","last_unstale":"2024-04-25T10:01:57.006345+0000","last_undegraded":"2024-04-25T10:01:57.006345+0000","last_fullsized":"2024-04-25T10:01:57.006345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.917422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634815+0000","last_change":"2024-04-25T10:01:51.959144+0000","last_active":"2024-04-25T10:01:57.634815+0000","last_peered":"2024-04-25T10:01:57.634815+0000","last_clean":"2024-04-25T10:01:57.634815+0000","last_became_active":"2024-04-25T10:01:51.958032+0000","last_became_peered":"2024-04-25T10:01:51.958032+0000","last_unstale":"2024-04-25T10:01:57.634815+0000","last_undegraded":"2024-04-25T10:01:57.634815+0000","last_fullsized":"2024-04-25T10:01:57.634815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:34.449807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629847+0000","last_change":"2024-04-25T10:01:50.951871+0000","last_active":"2024-04-25T10:01:57.629847+0000","last_peered":"2024-04-25T10:01:57.629847+0000","last_clean":"2024-04-25T10:01:57.629847+0000","last_became_active":"2024-04-25T10:01:50.951527+0000","last_became_peered":"2024-04-25T10:01:50.951527+0000","last_unstale":"2024-04-25T10:01:57.629847+0000","last_undegraded":"2024-04-25T10:01:57.629847+0000","last_fullsized":"2024-04-25T10:01:57.629847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:38.278178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630769+0000","last_change":"2024-04-25T10:01:56.101023+0000","last_active":"2024-04-25T10:01:57.630769+0000","last_peered":"2024-04-25T10:01:57.630769+0000","last_clean":"2024-04-25T10:01:57.630769+0000","last_became_active":"2024-04-25T10:01:56.100400+0000","last_became_peered":"2024-04-25T10:01:56.100400+0000","last_unstale":"2024-04-25T10:01:57.630769+0000","last_undegraded":"2024-04-25T10:01:57.630769+0000","last_fullsized":"2024-04-25T10:01:57.630769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:40.633384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628600+0000","last_change":"2024-04-25T10:01:51.951713+0000","last_active":"2024-04-25T10:01:57.628600+0000","last_peered":"2024-04-25T10:01:57.628600+0000","last_clean":"2024-04-25T10:01:57.628600+0000","last_became_active":"2024-04-25T10:01:51.951438+0000","last_became_peered":"2024-04-25T10:01:51.951438+0000","last_unstale":"2024-04-25T10:01:57.628600+0000","last_undegraded":"2024-04-25T10:01:57.628600+0000","last_fullsized":"2024-04-25T10:01:57.628600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348330+0000","last_change":"2024-04-25T10:01:50.956823+0000","last_active":"2024-04-25T10:01:57.348330+0000","last_peered":"2024-04-25T10:01:57.348330+0000","last_clean":"2024-04-25T10:01:57.348330+0000","last_became_active":"2024-04-25T10:01:50.956170+0000","last_became_peered":"2024-04-25T10:01:50.956170+0000","last_unstale":"2024-04-25T10:01:57.348330+0000","last_undegraded":"2024-04-25T10:01:57.348330+0000","last_fullsized":"2024-04-25T10:01:57.348330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:43.784003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629539+0000","last_change":"2024-04-25T10:01:56.101938+0000","last_active":"2024-04-25T10:01:57.629539+0000","last_peered":"2024-04-25T10:01:57.629539+0000","last_clean":"2024-04-25T10:01:57.629539+0000","last_became_active":"2024-04-25T10:01:56.101501+0000","last_became_peered":"2024-04-25T10:01:56.101501+0000","last_unstale":"2024-04-25T10:01:57.629539+0000","last_undegraded":"2024-04-25T10:01:57.629539+0000","last_fullsized":"2024-04-25T10:01:57.629539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:03.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.102065+0000","last_change":"2024-04-25T10:01:51.977453+0000","last_active":"2024-04-25T10:01:57.102065+0000","last_peered":"2024-04-25T10:01:57.102065+0000","last_clean":"2024-04-25T10:01:57.102065+0000","last_became_active":"2024-04-25T10:01:51.968804+0000","last_became_peered":"2024-04-25T10:01:51.968804+0000","last_unstale":"2024-04-25T10:01:57.102065+0000","last_undegraded":"2024-04-25T10:01:57.102065+0000","last_fullsized":"2024-04-25T10:01:57.102065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:34.449807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629477+0000","last_change":"2024-04-25T10:01:50.934637+0000","last_active":"2024-04-25T10:01:57.629477+0000","last_peered":"2024-04-25T10:01:57.629477+0000","last_clean":"2024-04-25T10:01:57.629477+0000","last_became_active":"2024-04-25T10:01:50.933810+0000","last_became_peered":"2024-04-25T10:01:50.933810+0000","last_unstale":"2024-04-25T10:01:57.629477+0000","last_undegraded":"2024-04-25T10:01:57.629477+0000","last_fullsized":"2024-04-25T10:01:57.629477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:21.571798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627408+0000","last_change":"2024-04-25T10:01:56.030222+0000","last_active":"2024-04-25T10:01:57.627408+0000","last_peered":"2024-04-25T10:01:57.627408+0000","last_clean":"2024-04-25T10:01:57.627408+0000","last_became_active":"2024-04-25T10:01:56.023359+0000","last_became_peered":"2024-04-25T10:01:56.023359+0000","last_unstale":"2024-04-25T10:01:57.627408+0000","last_undegraded":"2024-04-25T10:01:57.627408+0000","last_fullsized":"2024-04-25T10:01:57.627408+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009639+0000","last_change":"2024-04-25T10:01:51.941059+0000","last_active":"2024-04-25T10:01:57.009639+0000","last_peered":"2024-04-25T10:01:57.009639+0000","last_clean":"2024-04-25T10:01:57.009639+0000","last_became_active":"2024-04-25T10:01:51.938978+0000","last_became_peered":"2024-04-25T10:01:51.938978+0000","last_unstale":"2024-04-25T10:01:57.009639+0000","last_undegraded":"2024-04-25T10:01:57.009639+0000","last_fullsized":"2024-04-25T10:01:57.009639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:39.550415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101806+0000","last_change":"2024-04-25T10:01:50.952791+0000","last_active":"2024-04-25T10:01:57.101806+0000","last_peered":"2024-04-25T10:01:57.101806+0000","last_clean":"2024-04-25T10:01:57.101806+0000","last_became_active":"2024-04-25T10:01:50.950155+0000","last_became_peered":"2024-04-25T10:01:50.950155+0000","last_unstale":"2024-04-25T10:01:57.101806+0000","last_undegraded":"2024-04-25T10:01:57.101806+0000","last_fullsized":"2024-04-25T10:01:57.101806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629324+0000","last_change":"2024-04-25T10:01:56.016879+0000","last_active":"2024-04-25T10:01:57.629324+0000","last_peered":"2024-04-25T10:01:57.629324+0000","last_clean":"2024-04-25T10:01:57.629324+0000","last_became_active":"2024-04-25T10:01:56.016202+0000","last_became_peered":"2024-04-25T10:01:56.016202+0000","last_unstale":"2024-04-25T10:01:57.629324+0000","last_undegraded":"2024-04-25T10:01:57.629324+0000","last_fullsized":"2024-04-25T10:01:57.629324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:48.574592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627562+0000","last_change":"2024-04-25T10:01:51.952368+0000","last_active":"2024-04-25T10:01:57.627562+0000","last_peered":"2024-04-25T10:01:57.627562+0000","last_clean":"2024-04-25T10:01:57.627562+0000","last_became_active":"2024-04-25T10:01:51.952095+0000","last_became_peered":"2024-04-25T10:01:51.952095+0000","last_unstale":"2024-04-25T10:01:57.627562+0000","last_undegraded":"2024-04-25T10:01:57.627562+0000","last_fullsized":"2024-04-25T10:01:57.627562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004117+0000","last_change":"2024-04-25T10:01:50.940514+0000","last_active":"2024-04-25T10:01:57.004117+0000","last_peered":"2024-04-25T10:01:57.004117+0000","last_clean":"2024-04-25T10:01:57.004117+0000","last_became_active":"2024-04-25T10:01:50.935395+0000","last_became_peered":"2024-04-25T10:01:50.935395+0000","last_unstale":"2024-04-25T10:01:57.004117+0000","last_undegraded":"2024-04-25T10:01:57.004117+0000","last_fullsized":"2024-04-25T10:01:57.004117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.010630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629857+0000","last_change":"2024-04-25T10:01:56.015853+0000","last_active":"2024-04-25T10:01:57.629857+0000","last_peered":"2024-04-25T10:01:57.629857+0000","last_clean":"2024-04-25T10:01:57.629857+0000","last_became_active":"2024-04-25T10:01:56.015382+0000","last_became_peered":"2024-04-25T10:01:56.015382+0000","last_unstale":"2024-04-25T10:01:57.629857+0000","last_undegraded":"2024-04-25T10:01:57.629857+0000","last_fullsized":"2024-04-25T10:01:57.629857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:32.289340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628148+0000","last_change":"2024-04-25T10:01:51.939122+0000","last_active":"2024-04-25T10:01:57.628148+0000","last_peered":"2024-04-25T10:01:57.628148+0000","last_clean":"2024-04-25T10:01:57.628148+0000","last_became_active":"2024-04-25T10:01:51.938342+0000","last_became_peered":"2024-04-25T10:01:51.938342+0000","last_unstale":"2024-04-25T10:01:57.628148+0000","last_undegraded":"2024-04-25T10:01:57.628148+0000","last_fullsized":"2024-04-25T10:01:57.628148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:33.192217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346666+0000","last_change":"2024-04-25T10:01:50.957985+0000","last_active":"2024-04-25T10:01:57.346666+0000","last_peered":"2024-04-25T10:01:57.346666+0000","last_clean":"2024-04-25T10:01:57.346666+0000","last_became_active":"2024-04-25T10:01:50.957155+0000","last_became_peered":"2024-04-25T10:01:50.957155+0000","last_unstale":"2024-04-25T10:01:57.346666+0000","last_undegraded":"2024-04-25T10:01:57.346666+0000","last_fullsized":"2024-04-25T10:01:57.346666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:21.571798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631051+0000","last_change":"2024-04-25T10:01:56.105198+0000","last_active":"2024-04-25T10:01:57.631051+0000","last_peered":"2024-04-25T10:01:57.631051+0000","last_clean":"2024-04-25T10:01:57.631051+0000","last_became_active":"2024-04-25T10:01:56.104858+0000","last_became_peered":"2024-04-25T10:01:56.104858+0000","last_unstale":"2024-04-25T10:01:57.631051+0000","last_undegraded":"2024-04-25T10:01:57.631051+0000","last_fullsized":"2024-04-25T10:01:57.631051+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:48.574592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632717+0000","last_change":"2024-04-25T10:01:51.994401+0000","last_active":"2024-04-25T10:01:57.632717+0000","last_peered":"2024-04-25T10:01:57.632717+0000","last_clean":"2024-04-25T10:01:57.632717+0000","last_became_active":"2024-04-25T10:01:51.994086+0000","last_became_peered":"2024-04-25T10:01:51.994086+0000","last_unstale":"2024-04-25T10:01:57.632717+0000","last_undegraded":"2024-04-25T10:01:57.632717+0000","last_fullsized":"2024-04-25T10:01:57.632717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101375+0000","last_change":"2024-04-25T10:01:50.950324+0000","last_active":"2024-04-25T10:01:57.101375+0000","last_peered":"2024-04-25T10:01:57.101375+0000","last_clean":"2024-04-25T10:01:57.101375+0000","last_became_active":"2024-04-25T10:01:50.949810+0000","last_became_peered":"2024-04-25T10:01:50.949810+0000","last_unstale":"2024-04-25T10:01:57.101375+0000","last_undegraded":"2024-04-25T10:01:57.101375+0000","last_fullsized":"2024-04-25T10:01:57.101375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346935+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:01:57.346935+0000","last_peered":"2024-04-25T10:01:57.346935+0000","last_clean":"2024-04-25T10:01:57.346935+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:01:57.346935+0000","last_undegraded":"2024-04-25T10:01:57.346935+0000","last_fullsized":"2024-04-25T10:01:57.346935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007399+0000","last_change":"2024-04-25T10:01:56.108134+0000","last_active":"2024-04-25T10:01:57.007399+0000","last_peered":"2024-04-25T10:01:57.007399+0000","last_clean":"2024-04-25T10:01:57.007399+0000","last_became_active":"2024-04-25T10:01:56.107123+0000","last_became_peered":"2024-04-25T10:01:56.107123+0000","last_unstale":"2024-04-25T10:01:57.007399+0000","last_undegraded":"2024-04-25T10:01:57.007399+0000","last_fullsized":"2024-04-25T10:01:57.007399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:03.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347779+0000","last_change":"2024-04-25T10:01:51.983906+0000","last_active":"2024-04-25T10:01:57.347779+0000","last_peered":"2024-04-25T10:01:57.347779+0000","last_clean":"2024-04-25T10:01:57.347779+0000","last_became_active":"2024-04-25T10:01:51.982118+0000","last_became_peered":"2024-04-25T10:01:51.982118+0000","last_unstale":"2024-04-25T10:01:57.347779+0000","last_undegraded":"2024-04-25T10:01:57.347779+0000","last_fullsized":"2024-04-25T10:01:57.347779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:14.191816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004595+0000","last_change":"2024-04-25T10:01:50.946054+0000","last_active":"2024-04-25T10:01:57.004595+0000","last_peered":"2024-04-25T10:01:57.004595+0000","last_clean":"2024-04-25T10:01:57.004595+0000","last_became_active":"2024-04-25T10:01:50.945744+0000","last_became_peered":"2024-04-25T10:01:50.945744+0000","last_unstale":"2024-04-25T10:01:57.004595+0000","last_undegraded":"2024-04-25T10:01:57.004595+0000","last_fullsized":"2024-04-25T10:01:57.004595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006974+0000","last_change":"2024-04-25T10:01:56.104590+0000","last_active":"2024-04-25T10:01:57.006974+0000","last_peered":"2024-04-25T10:01:57.006974+0000","last_clean":"2024-04-25T10:01:57.006974+0000","last_became_active":"2024-04-25T10:01:56.103498+0000","last_became_peered":"2024-04-25T10:01:56.103498+0000","last_unstale":"2024-04-25T10:01:57.006974+0000","last_undegraded":"2024-04-25T10:01:57.006974+0000","last_fullsized":"2024-04-25T10:01:57.006974+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101453+0000","last_change":"2024-04-25T10:01:51.968651+0000","last_active":"2024-04-25T10:01:57.101453+0000","last_peered":"2024-04-25T10:01:57.101453+0000","last_clean":"2024-04-25T10:01:57.101453+0000","last_became_active":"2024-04-25T10:01:51.962327+0000","last_became_peered":"2024-04-25T10:01:51.962327+0000","last_unstale":"2024-04-25T10:01:57.101453+0000","last_undegraded":"2024-04-25T10:01:57.101453+0000","last_fullsized":"2024-04-25T10:01:57.101453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:55.810277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629450+0000","last_change":"2024-04-25T10:01:50.942416+0000","last_active":"2024-04-25T10:01:57.629450+0000","last_peered":"2024-04-25T10:01:57.629450+0000","last_clean":"2024-04-25T10:01:57.629450+0000","last_became_active":"2024-04-25T10:01:50.942009+0000","last_became_peered":"2024-04-25T10:01:50.942009+0000","last_unstale":"2024-04-25T10:01:57.629450+0000","last_undegraded":"2024-04-25T10:01:57.629450+0000","last_fullsized":"2024-04-25T10:01:57.629450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:25.085344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630028+0000","last_change":"2024-04-25T10:01:56.023691+0000","last_active":"2024-04-25T10:01:57.630028+0000","last_peered":"2024-04-25T10:01:57.630028+0000","last_clean":"2024-04-25T10:01:57.630028+0000","last_became_active":"2024-04-25T10:01:56.022988+0000","last_became_peered":"2024-04-25T10:01:56.022988+0000","last_unstale":"2024-04-25T10:01:57.630028+0000","last_undegraded":"2024-04-25T10:01:57.630028+0000","last_fullsized":"2024-04-25T10:01:57.630028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347762+0000","last_change":"2024-04-25T10:01:51.987406+0000","last_active":"2024-04-25T10:01:57.347762+0000","last_peered":"2024-04-25T10:01:57.347762+0000","last_clean":"2024-04-25T10:01:57.347762+0000","last_became_active":"2024-04-25T10:01:51.986930+0000","last_became_peered":"2024-04-25T10:01:51.986930+0000","last_unstale":"2024-04-25T10:01:57.347762+0000","last_undegraded":"2024-04-25T10:01:57.347762+0000","last_fullsized":"2024-04-25T10:01:57.347762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:27.239681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004599+0000","last_change":"2024-04-25T10:01:50.932604+0000","last_active":"2024-04-25T10:01:57.004599+0000","last_peered":"2024-04-25T10:01:57.004599+0000","last_clean":"2024-04-25T10:01:57.004599+0000","last_became_active":"2024-04-25T10:01:50.932205+0000","last_became_peered":"2024-04-25T10:01:50.932205+0000","last_unstale":"2024-04-25T10:01:57.004599+0000","last_undegraded":"2024-04-25T10:01:57.004599+0000","last_fullsized":"2024-04-25T10:01:57.004599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:35.306064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100705+0000","last_change":"2024-04-25T10:01:56.105972+0000","last_active":"2024-04-25T10:01:57.100705+0000","last_peered":"2024-04-25T10:01:57.100705+0000","last_clean":"2024-04-25T10:01:57.100705+0000","last_became_active":"2024-04-25T10:01:56.104834+0000","last_became_peered":"2024-04-25T10:01:56.104834+0000","last_unstale":"2024-04-25T10:01:57.100705+0000","last_undegraded":"2024-04-25T10:01:57.100705+0000","last_fullsized":"2024-04-25T10:01:57.100705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347690+0000","last_change":"2024-04-25T10:01:51.991881+0000","last_active":"2024-04-25T10:01:57.347690+0000","last_peered":"2024-04-25T10:01:57.347690+0000","last_clean":"2024-04-25T10:01:57.347690+0000","last_became_active":"2024-04-25T10:01:51.991266+0000","last_became_peered":"2024-04-25T10:01:51.991266+0000","last_unstale":"2024-04-25T10:01:57.347690+0000","last_undegraded":"2024-04-25T10:01:57.347690+0000","last_fullsized":"2024-04-25T10:01:57.347690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:39.074042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009212+0000","last_change":"2024-04-25T10:01:50.945437+0000","last_active":"2024-04-25T10:01:57.009212+0000","last_peered":"2024-04-25T10:01:57.009212+0000","last_clean":"2024-04-25T10:01:57.009212+0000","last_became_active":"2024-04-25T10:01:50.944786+0000","last_became_peered":"2024-04-25T10:01:50.944786+0000","last_unstale":"2024-04-25T10:01:57.009212+0000","last_undegraded":"2024-04-25T10:01:57.009212+0000","last_fullsized":"2024-04-25T10:01:57.009212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:53.181549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005662+0000","last_change":"2024-04-25T10:01:56.000803+0000","last_active":"2024-04-25T10:01:57.005662+0000","last_peered":"2024-04-25T10:01:57.005662+0000","last_clean":"2024-04-25T10:01:57.005662+0000","last_became_active":"2024-04-25T10:01:55.999071+0000","last_became_peered":"2024-04-25T10:01:55.999071+0000","last_unstale":"2024-04-25T10:01:57.005662+0000","last_undegraded":"2024-04-25T10:01:57.005662+0000","last_fullsized":"2024-04-25T10:01:57.005662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:50.609054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347419+0000","last_change":"2024-04-25T10:01:51.982017+0000","last_active":"2024-04-25T10:01:57.347419+0000","last_peered":"2024-04-25T10:01:57.347419+0000","last_clean":"2024-04-25T10:01:57.347419+0000","last_became_active":"2024-04-25T10:01:51.981340+0000","last_became_peered":"2024-04-25T10:01:51.981340+0000","last_unstale":"2024-04-25T10:01:57.347419+0000","last_undegraded":"2024-04-25T10:01:57.347419+0000","last_fullsized":"2024-04-25T10:01:57.347419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:22.999554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004023+0000","last_change":"2024-04-25T10:01:50.934868+0000","last_active":"2024-04-25T10:01:57.004023+0000","last_peered":"2024-04-25T10:01:57.004023+0000","last_clean":"2024-04-25T10:01:57.004023+0000","last_became_active":"2024-04-25T10:01:50.934560+0000","last_became_peered":"2024-04-25T10:01:50.934560+0000","last_unstale":"2024-04-25T10:01:57.004023+0000","last_undegraded":"2024-04-25T10:01:57.004023+0000","last_fullsized":"2024-04-25T10:01:57.004023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630724+0000","last_change":"2024-04-25T10:01:56.012427+0000","last_active":"2024-04-25T10:01:57.630724+0000","last_peered":"2024-04-25T10:01:57.630724+0000","last_clean":"2024-04-25T10:01:57.630724+0000","last_became_active":"2024-04-25T10:01:56.008358+0000","last_became_peered":"2024-04-25T10:01:56.008358+0000","last_unstale":"2024-04-25T10:01:57.630724+0000","last_undegraded":"2024-04-25T10:01:57.630724+0000","last_fullsized":"2024-04-25T10:01:57.630724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:56.864362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006288+0000","last_change":"2024-04-25T10:01:51.947557+0000","last_active":"2024-04-25T10:01:57.006288+0000","last_peered":"2024-04-25T10:01:57.006288+0000","last_clean":"2024-04-25T10:01:57.006288+0000","last_became_active":"2024-04-25T10:01:51.942206+0000","last_became_peered":"2024-04-25T10:01:51.942206+0000","last_unstale":"2024-04-25T10:01:57.006288+0000","last_undegraded":"2024-04-25T10:01:57.006288+0000","last_fullsized":"2024-04-25T10:01:57.006288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006959+0000","last_change":"2024-04-25T10:01:50.930663+0000","last_active":"2024-04-25T10:01:57.006959+0000","last_peered":"2024-04-25T10:01:57.006959+0000","last_clean":"2024-04-25T10:01:57.006959+0000","last_became_active":"2024-04-25T10:01:50.930224+0000","last_became_peered":"2024-04-25T10:01:50.930224+0000","last_unstale":"2024-04-25T10:01:57.006959+0000","last_undegraded":"2024-04-25T10:01:57.006959+0000","last_fullsized":"2024-04-25T10:01:57.006959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634460+0000","last_change":"2024-04-25T10:01:56.110077+0000","last_active":"2024-04-25T10:01:57.634460+0000","last_peered":"2024-04-25T10:01:57.634460+0000","last_clean":"2024-04-25T10:01:57.634460+0000","last_became_active":"2024-04-25T10:01:56.109686+0000","last_became_peered":"2024-04-25T10:01:56.109686+0000","last_unstale":"2024-04-25T10:01:57.634460+0000","last_undegraded":"2024-04-25T10:01:57.634460+0000","last_fullsized":"2024-04-25T10:01:57.634460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:20.441998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347395+0000","last_change":"2024-04-25T10:01:51.981399+0000","last_active":"2024-04-25T10:01:57.347395+0000","last_peered":"2024-04-25T10:01:57.347395+0000","last_clean":"2024-04-25T10:01:57.347395+0000","last_became_active":"2024-04-25T10:01:51.980512+0000","last_became_peered":"2024-04-25T10:01:51.980512+0000","last_unstale":"2024-04-25T10:01:57.347395+0000","last_undegraded":"2024-04-25T10:01:57.347395+0000","last_fullsized":"2024-04-25T10:01:57.347395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:52.801927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101702+0000","last_change":"2024-04-25T10:01:50.949394+0000","last_active":"2024-04-25T10:01:57.101702+0000","last_peered":"2024-04-25T10:01:57.101702+0000","last_clean":"2024-04-25T10:01:57.101702+0000","last_became_active":"2024-04-25T10:01:50.948515+0000","last_became_peered":"2024-04-25T10:01:50.948515+0000","last_unstale":"2024-04-25T10:01:57.101702+0000","last_undegraded":"2024-04-25T10:01:57.101702+0000","last_fullsized":"2024-04-25T10:01:57.101702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633739+0000","last_change":"2024-04-25T10:01:56.103664+0000","last_active":"2024-04-25T10:01:57.633739+0000","last_peered":"2024-04-25T10:01:57.633739+0000","last_clean":"2024-04-25T10:01:57.633739+0000","last_became_active":"2024-04-25T10:01:56.103259+0000","last_became_peered":"2024-04-25T10:01:56.103259+0000","last_unstale":"2024-04-25T10:01:57.633739+0000","last_undegraded":"2024-04-25T10:01:57.633739+0000","last_fullsized":"2024-04-25T10:01:57.633739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:19.687438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629659+0000","last_change":"2024-04-25T10:01:51.946200+0000","last_active":"2024-04-25T10:01:57.629659+0000","last_peered":"2024-04-25T10:01:57.629659+0000","last_clean":"2024-04-25T10:01:57.629659+0000","last_became_active":"2024-04-25T10:01:51.945344+0000","last_became_peered":"2024-04-25T10:01:51.945344+0000","last_unstale":"2024-04-25T10:01:57.629659+0000","last_undegraded":"2024-04-25T10:01:57.629659+0000","last_fullsized":"2024-04-25T10:01:57.629659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:13.244438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629997+0000","last_change":"2024-04-25T10:01:50.943130+0000","last_active":"2024-04-25T10:01:57.629997+0000","last_peered":"2024-04-25T10:01:57.629997+0000","last_clean":"2024-04-25T10:01:57.629997+0000","last_became_active":"2024-04-25T10:01:50.942800+0000","last_became_peered":"2024-04-25T10:01:50.942800+0000","last_unstale":"2024-04-25T10:01:57.629997+0000","last_undegraded":"2024-04-25T10:01:57.629997+0000","last_fullsized":"2024-04-25T10:01:57.629997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:58.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.102211+0000","last_change":"2024-04-25T10:01:56.103005+0000","last_active":"2024-04-25T10:01:57.102211+0000","last_peered":"2024-04-25T10:01:57.102211+0000","last_clean":"2024-04-25T10:01:57.102211+0000","last_became_active":"2024-04-25T10:01:56.101534+0000","last_became_peered":"2024-04-25T10:01:56.101534+0000","last_unstale":"2024-04-25T10:01:57.102211+0000","last_undegraded":"2024-04-25T10:01:57.102211+0000","last_fullsized":"2024-04-25T10:01:57.102211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:31.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347218+0000","last_change":"2024-04-25T10:01:51.982629+0000","last_active":"2024-04-25T10:01:57.347218+0000","last_peered":"2024-04-25T10:01:57.347218+0000","last_clean":"2024-04-25T10:01:57.347218+0000","last_became_active":"2024-04-25T10:01:51.981703+0000","last_became_peered":"2024-04-25T10:01:51.981703+0000","last_unstale":"2024-04-25T10:01:57.347218+0000","last_undegraded":"2024-04-25T10:01:57.347218+0000","last_fullsized":"2024-04-25T10:01:57.347218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:44.512157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:02.186202+0000","last_change":"2024-04-25T10:01:50.936030+0000","last_active":"2024-04-25T10:02:02.186202+0000","last_peered":"2024-04-25T10:02:02.186202+0000","last_clean":"2024-04-25T10:02:02.186202+0000","last_became_active":"2024-04-25T10:01:50.935276+0000","last_became_peered":"2024-04-25T10:01:50.935276+0000","last_unstale":"2024-04-25T10:02:02.186202+0000","last_undegraded":"2024-04-25T10:02:02.186202+0000","last_fullsized":"2024-04-25T10:02:02.186202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627939+0000","last_change":"2024-04-25T10:01:56.029517+0000","last_active":"2024-04-25T10:01:57.627939+0000","last_peered":"2024-04-25T10:01:57.627939+0000","last_clean":"2024-04-25T10:01:57.627939+0000","last_became_active":"2024-04-25T10:01:56.018301+0000","last_became_peered":"2024-04-25T10:01:56.018301+0000","last_unstale":"2024-04-25T10:01:57.627939+0000","last_undegraded":"2024-04-25T10:01:57.627939+0000","last_fullsized":"2024-04-25T10:01:57.627939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:56.864362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008458+0000","last_change":"2024-04-25T10:01:51.952954+0000","last_active":"2024-04-25T10:01:57.008458+0000","last_peered":"2024-04-25T10:01:57.008458+0000","last_clean":"2024-04-25T10:01:57.008458+0000","last_became_active":"2024-04-25T10:01:51.952204+0000","last_became_peered":"2024-04-25T10:01:51.952204+0000","last_unstale":"2024-04-25T10:01:57.008458+0000","last_undegraded":"2024-04-25T10:01:57.008458+0000","last_fullsized":"2024-04-25T10:01:57.008458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:39.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629335+0000","last_change":"2024-04-25T10:01:50.950696+0000","last_active":"2024-04-25T10:01:57.629335+0000","last_peered":"2024-04-25T10:01:57.629335+0000","last_clean":"2024-04-25T10:01:57.629335+0000","last_became_active":"2024-04-25T10:01:50.950369+0000","last_became_peered":"2024-04-25T10:01:50.950369+0000","last_unstale":"2024-04-25T10:01:57.629335+0000","last_undegraded":"2024-04-25T10:01:57.629335+0000","last_fullsized":"2024-04-25T10:01:57.629335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:38.540445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009378+0000","last_change":"2024-04-25T10:01:56.025446+0000","last_active":"2024-04-25T10:01:57.009378+0000","last_peered":"2024-04-25T10:01:57.009378+0000","last_clean":"2024-04-25T10:01:57.009378+0000","last_became_active":"2024-04-25T10:01:56.023974+0000","last_became_peered":"2024-04-25T10:01:56.023974+0000","last_unstale":"2024-04-25T10:01:57.009378+0000","last_undegraded":"2024-04-25T10:01:57.009378+0000","last_fullsized":"2024-04-25T10:01:57.009378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631278+0000","last_change":"2024-04-25T10:01:51.969210+0000","last_active":"2024-04-25T10:01:57.631278+0000","last_peered":"2024-04-25T10:01:57.631278+0000","last_clean":"2024-04-25T10:01:57.631278+0000","last_became_active":"2024-04-25T10:01:51.968830+0000","last_became_peered":"2024-04-25T10:01:51.968830+0000","last_unstale":"2024-04-25T10:01:57.631278+0000","last_undegraded":"2024-04-25T10:01:57.631278+0000","last_fullsized":"2024-04-25T10:01:57.631278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101496+0000","last_change":"2024-04-25T10:01:50.948707+0000","last_active":"2024-04-25T10:01:57.101496+0000","last_peered":"2024-04-25T10:01:57.101496+0000","last_clean":"2024-04-25T10:01:57.101496+0000","last_became_active":"2024-04-25T10:01:50.947988+0000","last_became_peered":"2024-04-25T10:01:50.947988+0000","last_unstale":"2024-04-25T10:01:57.101496+0000","last_undegraded":"2024-04-25T10:01:57.101496+0000","last_fullsized":"2024-04-25T10:01:57.101496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631528+0000","last_change":"2024-04-25T10:01:56.017001+0000","last_active":"2024-04-25T10:01:57.631528+0000","last_peered":"2024-04-25T10:01:57.631528+0000","last_clean":"2024-04-25T10:01:57.631528+0000","last_became_active":"2024-04-25T10:01:56.016220+0000","last_became_peered":"2024-04-25T10:01:56.016220+0000","last_unstale":"2024-04-25T10:01:57.631528+0000","last_undegraded":"2024-04-25T10:01:57.631528+0000","last_fullsized":"2024-04-25T10:01:57.631528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:31.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101724+0000","last_change":"2024-04-25T10:01:51.978944+0000","last_active":"2024-04-25T10:01:57.101724+0000","last_peered":"2024-04-25T10:01:57.101724+0000","last_clean":"2024-04-25T10:01:57.101724+0000","last_became_active":"2024-04-25T10:01:51.978133+0000","last_became_peered":"2024-04-25T10:01:51.978133+0000","last_unstale":"2024-04-25T10:01:57.101724+0000","last_undegraded":"2024-04-25T10:01:57.101724+0000","last_fullsized":"2024-04-25T10:01:57.101724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007998+0000","last_change":"2024-04-25T10:01:50.953423+0000","last_active":"2024-04-25T10:01:57.007998+0000","last_peered":"2024-04-25T10:01:57.007998+0000","last_clean":"2024-04-25T10:01:57.007998+0000","last_became_active":"2024-04-25T10:01:50.952221+0000","last_became_peered":"2024-04-25T10:01:50.952221+0000","last_unstale":"2024-04-25T10:01:57.007998+0000","last_undegraded":"2024-04-25T10:01:57.007998+0000","last_fullsized":"2024-04-25T10:01:57.007998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631879+0000","last_change":"2024-04-25T10:01:56.110756+0000","last_active":"2024-04-25T10:01:57.631879+0000","last_peered":"2024-04-25T10:01:57.631879+0000","last_clean":"2024-04-25T10:01:57.631879+0000","last_became_active":"2024-04-25T10:01:56.109749+0000","last_became_peered":"2024-04-25T10:01:56.109749+0000","last_unstale":"2024-04-25T10:01:57.631879+0000","last_undegraded":"2024-04-25T10:01:57.631879+0000","last_fullsized":"2024-04-25T10:01:57.631879+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:48.763078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628771+0000","last_change":"2024-04-25T10:01:56.106331+0000","last_active":"2024-04-25T10:01:57.628771+0000","last_peered":"2024-04-25T10:01:57.628771+0000","last_clean":"2024-04-25T10:01:57.628771+0000","last_became_active":"2024-04-25T10:01:56.105560+0000","last_became_peered":"2024-04-25T10:01:56.105560+0000","last_unstale":"2024-04-25T10:01:57.628771+0000","last_undegraded":"2024-04-25T10:01:57.628771+0000","last_fullsized":"2024-04-25T10:01:57.628771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627589+0000","last_change":"2024-04-25T10:01:51.959889+0000","last_active":"2024-04-25T10:01:57.627589+0000","last_peered":"2024-04-25T10:01:57.627589+0000","last_clean":"2024-04-25T10:01:57.627589+0000","last_became_active":"2024-04-25T10:01:51.959609+0000","last_became_peered":"2024-04-25T10:01:51.959609+0000","last_unstale":"2024-04-25T10:01:57.627589+0000","last_undegraded":"2024-04-25T10:01:57.627589+0000","last_fullsized":"2024-04-25T10:01:57.627589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101160+0000","last_change":"2024-04-25T10:01:50.947616+0000","last_active":"2024-04-25T10:01:57.101160+0000","last_peered":"2024-04-25T10:01:57.101160+0000","last_clean":"2024-04-25T10:01:57.101160+0000","last_became_active":"2024-04-25T10:01:50.946930+0000","last_became_peered":"2024-04-25T10:01:50.946930+0000","last_unstale":"2024-04-25T10:01:57.101160+0000","last_undegraded":"2024-04-25T10:01:57.101160+0000","last_fullsized":"2024-04-25T10:01:57.101160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:33.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007713+0000","last_change":"2024-04-25T10:01:56.018502+0000","last_active":"2024-04-25T10:01:57.007713+0000","last_peered":"2024-04-25T10:01:57.007713+0000","last_clean":"2024-04-25T10:01:57.007713+0000","last_became_active":"2024-04-25T10:01:56.017379+0000","last_became_peered":"2024-04-25T10:01:56.017379+0000","last_unstale":"2024-04-25T10:01:57.007713+0000","last_undegraded":"2024-04-25T10:01:57.007713+0000","last_fullsized":"2024-04-25T10:01:57.007713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631645+0000","last_change":"2024-04-25T10:01:51.953615+0000","last_active":"2024-04-25T10:01:57.631645+0000","last_peered":"2024-04-25T10:01:57.631645+0000","last_clean":"2024-04-25T10:01:57.631645+0000","last_became_active":"2024-04-25T10:01:51.953362+0000","last_became_peered":"2024-04-25T10:01:51.953362+0000","last_unstale":"2024-04-25T10:01:57.631645+0000","last_undegraded":"2024-04-25T10:01:57.631645+0000","last_fullsized":"2024-04-25T10:01:57.631645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632491+0000","last_change":"2024-04-25T10:01:50.945460+0000","last_active":"2024-04-25T10:01:57.632491+0000","last_peered":"2024-04-25T10:01:57.632491+0000","last_clean":"2024-04-25T10:01:57.632491+0000","last_became_active":"2024-04-25T10:01:50.945086+0000","last_became_peered":"2024-04-25T10:01:50.945086+0000","last_unstale":"2024-04-25T10:01:57.632491+0000","last_undegraded":"2024-04-25T10:01:57.632491+0000","last_fullsized":"2024-04-25T10:01:57.632491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.010630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007381+0000","last_change":"2024-04-25T10:01:50.944204+0000","last_active":"2024-04-25T10:01:57.007381+0000","last_peered":"2024-04-25T10:01:57.007381+0000","last_clean":"2024-04-25T10:01:57.007381+0000","last_became_active":"2024-04-25T10:01:50.943478+0000","last_became_peered":"2024-04-25T10:01:50.943478+0000","last_unstale":"2024-04-25T10:01:57.007381+0000","last_undegraded":"2024-04-25T10:01:57.007381+0000","last_fullsized":"2024-04-25T10:01:57.007381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629902+0000","last_change":"2024-04-25T10:01:51.939383+0000","last_active":"2024-04-25T10:01:57.629902+0000","last_peered":"2024-04-25T10:01:57.629902+0000","last_clean":"2024-04-25T10:01:57.629902+0000","last_became_active":"2024-04-25T10:01:51.938491+0000","last_became_peered":"2024-04-25T10:01:51.938491+0000","last_unstale":"2024-04-25T10:01:57.629902+0000","last_undegraded":"2024-04-25T10:01:57.629902+0000","last_fullsized":"2024-04-25T10:01:57.629902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004475+0000","last_change":"2024-04-25T10:01:56.024292+0000","last_active":"2024-04-25T10:01:57.004475+0000","last_peered":"2024-04-25T10:01:57.004475+0000","last_clean":"2024-04-25T10:01:57.004475+0000","last_became_active":"2024-04-25T10:01:56.023778+0000","last_became_peered":"2024-04-25T10:01:56.023778+0000","last_unstale":"2024-04-25T10:01:57.004475+0000","last_undegraded":"2024-04-25T10:01:57.004475+0000","last_fullsized":"2024-04-25T10:01:57.004475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:30.157749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633372+0000","last_change":"2024-04-25T10:01:50.954588+0000","last_active":"2024-04-25T10:01:57.633372+0000","last_peered":"2024-04-25T10:01:57.633372+0000","last_clean":"2024-04-25T10:01:57.633372+0000","last_became_active":"2024-04-25T10:01:50.954119+0000","last_became_peered":"2024-04-25T10:01:50.954119+0000","last_unstale":"2024-04-25T10:01:57.633372+0000","last_undegraded":"2024-04-25T10:01:57.633372+0000","last_fullsized":"2024-04-25T10:01:57.633372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:35.306064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007283+0000","last_change":"2024-04-25T10:01:51.941227+0000","last_active":"2024-04-25T10:01:57.007283+0000","last_peered":"2024-04-25T10:01:57.007283+0000","last_clean":"2024-04-25T10:01:57.007283+0000","last_became_active":"2024-04-25T10:01:51.940116+0000","last_became_peered":"2024-04-25T10:01:51.940116+0000","last_unstale":"2024-04-25T10:01:57.007283+0000","last_undegraded":"2024-04-25T10:01:57.007283+0000","last_fullsized":"2024-04-25T10:01:57.007283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:49.915603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633711+0000","last_change":"2024-04-25T10:01:56.024226+0000","last_active":"2024-04-25T10:01:57.633711+0000","last_peered":"2024-04-25T10:01:57.633711+0000","last_clean":"2024-04-25T10:01:57.633711+0000","last_became_active":"2024-04-25T10:01:56.023568+0000","last_became_peered":"2024-04-25T10:01:56.023568+0000","last_unstale":"2024-04-25T10:01:57.633711+0000","last_undegraded":"2024-04-25T10:01:57.633711+0000","last_fullsized":"2024-04-25T10:01:57.633711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.434943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005947+0000","last_change":"2024-04-25T10:01:51.948178+0000","last_active":"2024-04-25T10:01:57.005947+0000","last_peered":"2024-04-25T10:01:57.005947+0000","last_clean":"2024-04-25T10:01:57.005947+0000","last_became_active":"2024-04-25T10:01:51.943159+0000","last_became_peered":"2024-04-25T10:01:51.943159+0000","last_unstale":"2024-04-25T10:01:57.005947+0000","last_undegraded":"2024-04-25T10:01:57.005947+0000","last_fullsized":"2024-04-25T10:01:57.005947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:13.244438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004086+0000","last_change":"2024-04-25T10:01:51.977923+0000","last_active":"2024-04-25T10:01:57.004086+0000","last_peered":"2024-04-25T10:01:57.004086+0000","last_clean":"2024-04-25T10:01:57.004086+0000","last_became_active":"2024-04-25T10:01:51.977601+0000","last_became_peered":"2024-04-25T10:01:51.977601+0000","last_unstale":"2024-04-25T10:01:57.004086+0000","last_undegraded":"2024-04-25T10:01:57.004086+0000","last_fullsized":"2024-04-25T10:01:57.004086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:39.550415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101440+0000","last_change":"2024-04-25T10:01:51.977323+0000","last_active":"2024-04-25T10:01:57.101440+0000","last_peered":"2024-04-25T10:01:57.101440+0000","last_clean":"2024-04-25T10:01:57.101440+0000","last_became_active":"2024-04-25T10:01:51.976662+0000","last_became_peered":"2024-04-25T10:01:51.976662+0000","last_unstale":"2024-04-25T10:01:57.101440+0000","last_undegraded":"2024-04-25T10:01:57.101440+0000","last_fullsized":"2024-04-25T10:01:57.101440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:52.801927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629808+0000","last_change":"2024-04-25T10:01:51.962727+0000","last_active":"2024-04-25T10:01:57.629808+0000","last_peered":"2024-04-25T10:01:57.629808+0000","last_clean":"2024-04-25T10:01:57.629808+0000","last_became_active":"2024-04-25T10:01:51.962048+0000","last_became_peered":"2024-04-25T10:01:51.962048+0000","last_unstale":"2024-04-25T10:01:57.629808+0000","last_undegraded":"2024-04-25T10:01:57.629808+0000","last_fullsized":"2024-04-25T10:01:57.629808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633852+0000","last_change":"2024-04-25T10:01:51.956859+0000","last_active":"2024-04-25T10:01:57.633852+0000","last_peered":"2024-04-25T10:01:57.633852+0000","last_clean":"2024-04-25T10:01:57.633852+0000","last_became_active":"2024-04-25T10:01:51.956531+0000","last_became_peered":"2024-04-25T10:01:51.956531+0000","last_unstale":"2024-04-25T10:01:57.633852+0000","last_undegraded":"2024-04-25T10:01:57.633852+0000","last_fullsized":"2024-04-25T10:01:57.633852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628823+0000","last_change":"2024-04-25T10:01:51.960245+0000","last_active":"2024-04-25T10:01:57.628823+0000","last_peered":"2024-04-25T10:01:57.628823+0000","last_clean":"2024-04-25T10:01:57.628823+0000","last_became_active":"2024-04-25T10:01:51.959954+0000","last_became_peered":"2024-04-25T10:01:51.959954+0000","last_unstale":"2024-04-25T10:01:57.628823+0000","last_undegraded":"2024-04-25T10:01:57.628823+0000","last_fullsized":"2024-04-25T10:01:57.628823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008265+0000","last_change":"2024-04-25T10:01:51.951215+0000","last_active":"2024-04-25T10:01:57.008265+0000","last_peered":"2024-04-25T10:01:57.008265+0000","last_clean":"2024-04-25T10:01:57.008265+0000","last_became_active":"2024-04-25T10:01:51.950168+0000","last_became_peered":"2024-04-25T10:01:51.950168+0000","last_unstale":"2024-04-25T10:01:57.008265+0000","last_undegraded":"2024-04-25T10:01:57.008265+0000","last_fullsized":"2024-04-25T10:01:57.008265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634588+0000","last_change":"2024-04-25T10:01:51.959477+0000","last_active":"2024-04-25T10:01:57.634588+0000","last_peered":"2024-04-25T10:01:57.634588+0000","last_clean":"2024-04-25T10:01:57.634588+0000","last_became_active":"2024-04-25T10:01:51.958740+0000","last_became_peered":"2024-04-25T10:01:51.958740+0000","last_unstale":"2024-04-25T10:01:57.634588+0000","last_undegraded":"2024-04-25T10:01:57.634588+0000","last_fullsized":"2024-04-25T10:01:57.634588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100898+0000","last_change":"2024-04-25T10:01:51.987810+0000","last_active":"2024-04-25T10:01:57.100898+0000","last_peered":"2024-04-25T10:01:57.100898+0000","last_clean":"2024-04-25T10:01:57.100898+0000","last_became_active":"2024-04-25T10:01:51.978504+0000","last_became_peered":"2024-04-25T10:01:51.978504+0000","last_unstale":"2024-04-25T10:01:57.100898+0000","last_undegraded":"2024-04-25T10:01:57.100898+0000","last_fullsized":"2024-04-25T10:01:57.100898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:44.512157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632135+0000","last_change":"2024-04-25T10:01:51.949190+0000","last_active":"2024-04-25T10:01:57.632135+0000","last_peered":"2024-04-25T10:01:57.632135+0000","last_clean":"2024-04-25T10:01:57.632135+0000","last_became_active":"2024-04-25T10:01:51.948886+0000","last_became_peered":"2024-04-25T10:01:51.948886+0000","last_unstale":"2024-04-25T10:01:57.632135+0000","last_undegraded":"2024-04-25T10:01:57.632135+0000","last_fullsized":"2024-04-25T10:01:57.632135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101158+0000","last_change":"2024-04-25T10:01:51.987833+0000","last_active":"2024-04-25T10:01:57.101158+0000","last_peered":"2024-04-25T10:01:57.101158+0000","last_clean":"2024-04-25T10:01:57.101158+0000","last_became_active":"2024-04-25T10:01:51.978154+0000","last_became_peered":"2024-04-25T10:01:51.978154+0000","last_unstale":"2024-04-25T10:01:57.101158+0000","last_undegraded":"2024-04-25T10:01:57.101158+0000","last_fullsized":"2024-04-25T10:01:57.101158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:22.999554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629659+0000","last_change":"2024-04-25T10:01:51.947940+0000","last_active":"2024-04-25T10:01:57.629659+0000","last_peered":"2024-04-25T10:01:57.629659+0000","last_clean":"2024-04-25T10:01:57.629659+0000","last_became_active":"2024-04-25T10:01:51.947672+0000","last_became_peered":"2024-04-25T10:01:51.947672+0000","last_unstale":"2024-04-25T10:01:57.629659+0000","last_undegraded":"2024-04-25T10:01:57.629659+0000","last_fullsized":"2024-04-25T10:01:57.629659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:39.087518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007150+0000","last_change":"2024-04-25T10:01:51.936013+0000","last_active":"2024-04-25T10:01:57.007150+0000","last_peered":"2024-04-25T10:01:57.007150+0000","last_clean":"2024-04-25T10:01:57.007150+0000","last_became_active":"2024-04-25T10:01:51.933077+0000","last_became_peered":"2024-04-25T10:01:51.933077+0000","last_unstale":"2024-04-25T10:01:57.007150+0000","last_undegraded":"2024-04-25T10:01:57.007150+0000","last_fullsized":"2024-04-25T10:01:57.007150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348064+0000","last_change":"2024-04-25T10:01:51.987425+0000","last_active":"2024-04-25T10:01:57.348064+0000","last_peered":"2024-04-25T10:01:57.348064+0000","last_clean":"2024-04-25T10:01:57.348064+0000","last_became_active":"2024-04-25T10:01:51.986698+0000","last_became_peered":"2024-04-25T10:01:51.986698+0000","last_unstale":"2024-04-25T10:01:57.348064+0000","last_undegraded":"2024-04-25T10:01:57.348064+0000","last_fullsized":"2024-04-25T10:01:57.348064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:28.226905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004602+0000","last_change":"2024-04-25T10:01:51.962633+0000","last_active":"2024-04-25T10:01:57.004602+0000","last_peered":"2024-04-25T10:01:57.004602+0000","last_clean":"2024-04-25T10:01:57.004602+0000","last_became_active":"2024-04-25T10:01:51.962235+0000","last_became_peered":"2024-04-25T10:01:51.962235+0000","last_unstale":"2024-04-25T10:01:57.004602+0000","last_undegraded":"2024-04-25T10:01:57.004602+0000","last_fullsized":"2024-04-25T10:01:57.004602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:39.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101861+0000","last_change":"2024-04-25T10:01:51.992557+0000","last_active":"2024-04-25T10:01:57.101861+0000","last_peered":"2024-04-25T10:01:57.101861+0000","last_clean":"2024-04-25T10:01:57.101861+0000","last_became_active":"2024-04-25T10:01:51.991982+0000","last_became_peered":"2024-04-25T10:01:51.991982+0000","last_unstale":"2024-04-25T10:01:57.101861+0000","last_undegraded":"2024-04-25T10:01:57.101861+0000","last_fullsized":"2024-04-25T10:01:57.101861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100581+0000","last_change":"2024-04-25T10:01:51.988776+0000","last_active":"2024-04-25T10:01:57.100581+0000","last_peered":"2024-04-25T10:01:57.100581+0000","last_clean":"2024-04-25T10:01:57.100581+0000","last_became_active":"2024-04-25T10:01:51.987983+0000","last_became_peered":"2024-04-25T10:01:51.987983+0000","last_unstale":"2024-04-25T10:01:57.100581+0000","last_undegraded":"2024-04-25T10:01:57.100581+0000","last_fullsized":"2024-04-25T10:01:57.100581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:39.074042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627816+0000","last_change":"2024-04-25T10:01:51.946481+0000","last_active":"2024-04-25T10:01:57.627816+0000","last_peered":"2024-04-25T10:01:57.627816+0000","last_clean":"2024-04-25T10:01:57.627816+0000","last_became_active":"2024-04-25T10:01:51.946161+0000","last_became_peered":"2024-04-25T10:01:51.946161+0000","last_unstale":"2024-04-25T10:01:57.627816+0000","last_undegraded":"2024-04-25T10:01:57.627816+0000","last_fullsized":"2024-04-25T10:01:57.627816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629440+0000","last_change":"2024-04-25T10:01:51.946480+0000","last_active":"2024-04-25T10:01:57.629440+0000","last_peered":"2024-04-25T10:01:57.629440+0000","last_clean":"2024-04-25T10:01:57.629440+0000","last_became_active":"2024-04-25T10:01:51.945872+0000","last_became_peered":"2024-04-25T10:01:51.945872+0000","last_unstale":"2024-04-25T10:01:57.629440+0000","last_undegraded":"2024-04-25T10:01:57.629440+0000","last_fullsized":"2024-04-25T10:01:57.629440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:44.793972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633094+0000","last_change":"2024-04-25T10:01:51.958336+0000","last_active":"2024-04-25T10:01:57.633094+0000","last_peered":"2024-04-25T10:01:57.633094+0000","last_clean":"2024-04-25T10:01:57.633094+0000","last_became_active":"2024-04-25T10:01:51.957993+0000","last_became_peered":"2024-04-25T10:01:51.957993+0000","last_unstale":"2024-04-25T10:01:57.633094+0000","last_undegraded":"2024-04-25T10:01:57.633094+0000","last_fullsized":"2024-04-25T10:01:57.633094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008236+0000","last_change":"2024-04-25T10:01:51.943174+0000","last_active":"2024-04-25T10:01:57.008236+0000","last_peered":"2024-04-25T10:01:57.008236+0000","last_clean":"2024-04-25T10:01:57.008236+0000","last_became_active":"2024-04-25T10:01:51.942327+0000","last_became_peered":"2024-04-25T10:01:51.942327+0000","last_unstale":"2024-04-25T10:01:57.008236+0000","last_undegraded":"2024-04-25T10:01:57.008236+0000","last_fullsized":"2024-04-25T10:01:57.008236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:22.581767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007882+0000","last_change":"2024-04-25T10:01:51.963532+0000","last_active":"2024-04-25T10:01:57.007882+0000","last_peered":"2024-04-25T10:01:57.007882+0000","last_clean":"2024-04-25T10:01:57.007882+0000","last_became_active":"2024-04-25T10:01:51.962793+0000","last_became_peered":"2024-04-25T10:01:51.962793+0000","last_unstale":"2024-04-25T10:01:57.007882+0000","last_undegraded":"2024-04-25T10:01:57.007882+0000","last_fullsized":"2024-04-25T10:01:57.007882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008103+0000","last_change":"2024-04-25T10:01:51.970144+0000","last_active":"2024-04-25T10:01:57.008103+0000","last_peered":"2024-04-25T10:01:57.008103+0000","last_clean":"2024-04-25T10:01:57.008103+0000","last_became_active":"2024-04-25T10:01:51.967189+0000","last_became_peered":"2024-04-25T10:01:51.967189+0000","last_unstale":"2024-04-25T10:01:57.008103+0000","last_undegraded":"2024-04-25T10:01:57.008103+0000","last_fullsized":"2024-04-25T10:01:57.008103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347824+0000","last_change":"2024-04-25T10:01:51.995155+0000","last_active":"2024-04-25T10:01:57.347824+0000","last_peered":"2024-04-25T10:01:57.347824+0000","last_clean":"2024-04-25T10:01:57.347824+0000","last_became_active":"2024-04-25T10:01:51.994514+0000","last_became_peered":"2024-04-25T10:01:51.994514+0000","last_unstale":"2024-04-25T10:01:57.347824+0000","last_undegraded":"2024-04-25T10:01:57.347824+0000","last_fullsized":"2024-04-25T10:01:57.347824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:36.570949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631050+0000","last_change":"2024-04-25T10:01:51.977705+0000","last_active":"2024-04-25T10:01:57.631050+0000","last_peered":"2024-04-25T10:01:57.631050+0000","last_clean":"2024-04-25T10:01:57.631050+0000","last_became_active":"2024-04-25T10:01:51.977373+0000","last_became_peered":"2024-04-25T10:01:51.977373+0000","last_unstale":"2024-04-25T10:01:57.631050+0000","last_undegraded":"2024-04-25T10:01:57.631050+0000","last_fullsized":"2024-04-25T10:01:57.631050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347155+0000","last_change":"2024-04-25T10:01:51.958773+0000","last_active":"2024-04-25T10:01:57.347155+0000","last_peered":"2024-04-25T10:01:57.347155+0000","last_clean":"2024-04-25T10:01:57.347155+0000","last_became_active":"2024-04-25T10:01:51.958015+0000","last_became_peered":"2024-04-25T10:01:51.958015+0000","last_unstale":"2024-04-25T10:01:57.347155+0000","last_undegraded":"2024-04-25T10:01:57.347155+0000","last_fullsized":"2024-04-25T10:01:57.347155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100869+0000","last_change":"2024-04-25T10:01:51.994172+0000","last_active":"2024-04-25T10:01:57.100869+0000","last_peered":"2024-04-25T10:01:57.100869+0000","last_clean":"2024-04-25T10:01:57.100869+0000","last_became_active":"2024-04-25T10:01:51.993498+0000","last_became_peered":"2024-04-25T10:01:51.993498+0000","last_unstale":"2024-04-25T10:01:57.100869+0000","last_undegraded":"2024-04-25T10:01:57.100869+0000","last_fullsized":"2024-04-25T10:01:57.100869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:59.098969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346909+0000","last_change":"2024-04-25T10:01:51.958621+0000","last_active":"2024-04-25T10:01:57.346909+0000","last_peered":"2024-04-25T10:01:57.346909+0000","last_clean":"2024-04-25T10:01:57.346909+0000","last_became_active":"2024-04-25T10:01:51.957937+0000","last_became_peered":"2024-04-25T10:01:51.957937+0000","last_unstale":"2024-04-25T10:01:57.346909+0000","last_undegraded":"2024-04-25T10:01:57.346909+0000","last_fullsized":"2024-04-25T10:01:57.346909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:59.098969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101663+0000","last_change":"2024-04-25T10:01:51.967673+0000","last_active":"2024-04-25T10:01:57.101663+0000","last_peered":"2024-04-25T10:01:57.101663+0000","last_clean":"2024-04-25T10:01:57.101663+0000","last_became_active":"2024-04-25T10:01:51.966329+0000","last_became_peered":"2024-04-25T10:01:51.966329+0000","last_unstale":"2024-04-25T10:01:57.101663+0000","last_undegraded":"2024-04-25T10:01:57.101663+0000","last_fullsized":"2024-04-25T10:01:57.101663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630321+0000","last_change":"2024-04-25T10:01:51.950122+0000","last_active":"2024-04-25T10:01:57.630321+0000","last_peered":"2024-04-25T10:01:57.630321+0000","last_clean":"2024-04-25T10:01:57.630321+0000","last_became_active":"2024-04-25T10:01:51.949559+0000","last_became_peered":"2024-04-25T10:01:51.949559+0000","last_unstale":"2024-04-25T10:01:57.630321+0000","last_undegraded":"2024-04-25T10:01:57.630321+0000","last_fullsized":"2024-04-25T10:01:57.630321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:52.021045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629583+0000","last_change":"2024-04-25T10:01:51.943127+0000","last_active":"2024-04-25T10:01:57.629583+0000","last_peered":"2024-04-25T10:01:57.629583+0000","last_clean":"2024-04-25T10:01:57.629583+0000","last_became_active":"2024-04-25T10:01:51.942304+0000","last_became_peered":"2024-04-25T10:01:51.942304+0000","last_unstale":"2024-04-25T10:01:57.629583+0000","last_undegraded":"2024-04-25T10:01:57.629583+0000","last_fullsized":"2024-04-25T10:01:57.629583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347294+0000","last_change":"2024-04-25T10:01:51.987276+0000","last_active":"2024-04-25T10:01:57.347294+0000","last_peered":"2024-04-25T10:01:57.347294+0000","last_clean":"2024-04-25T10:01:57.347294+0000","last_became_active":"2024-04-25T10:01:51.986766+0000","last_became_peered":"2024-04-25T10:01:51.986766+0000","last_unstale":"2024-04-25T10:01:57.347294+0000","last_undegraded":"2024-04-25T10:01:57.347294+0000","last_fullsized":"2024-04-25T10:01:57.347294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:57.724919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:16.484 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:02:16.484 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:02:16.484 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:02:16.484 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:02:16.485 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:02:16.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.632+0000 7f6f08ff8640 1 Processor -- start 2024-04-25T10:02:16.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.632+0000 7f6f08ff8640 1 -- start start 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.632+0000 7f6f08ff8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f04158110 0x7f6f040b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6f0406eb70 0x7f6f0415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f040cbe60 con 0x7f6f041584f0 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f040b19b0 con 0x7f6f0406eb70 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0415b360 con 0x7f6f04158110 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f02d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6f0406eb70 0x7f6f0415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f02d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6f0406eb70 0x7f6f0415ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48270/0 (socket says 172.21.15.133:48270) 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f02d9d640 1 -- 172.21.15.133:0/1433416642 learned_addr learned my addr 172.21.15.133:0/1433416642 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f0259c640 1 --2- 172.21.15.133:0/1433416642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f04158110 0x7f6f040b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f02d9d640 1 -- 172.21.15.133:0/1433416642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f04158110 msgr2=0x7f6f040b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f02d9d640 1 --2- 172.21.15.133:0/1433416642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f04158110 0x7f6f040b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f02d9d640 1 -- 172.21.15.133:0/1433416642 >> v1:172.21.15.162:6789/0 conn(0x7f6f041584f0 legacy=0x7f6f0406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:16.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f02d9d640 1 -- 172.21.15.133:0/1433416642 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0415b640 con 0x7f6f0406eb70 2024-04-25T10:02:16.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f02d9d640 1 --2- 172.21.15.133:0/1433416642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f0406eb70 0x7f6f0415ad10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009e50 tx=0x7f6efc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd052973c7b82d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:16.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f0159a640 1 -- 172.21.15.133:0/1433416642 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0a5860 con 0x7f6f0406eb70 2024-04-25T10:02:16.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f0159a640 1 -- 172.21.15.133:0/1433416642 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6efc0a5a00 con 0x7f6f0406eb70 2024-04-25T10:02:16.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f0159a640 1 -- 172.21.15.133:0/1433416642 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0a5dd0 con 0x7f6f0406eb70 2024-04-25T10:02:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1433416642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f0406eb70 msgr2=0x7f6f0415ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 --2- 172.21.15.133:0/1433416642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f0406eb70 0x7f6f0415ad10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009e50 tx=0x7f6efc01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1433416642 shutdown_connections 2024-04-25T10:02:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 --2- 172.21.15.133:0/1433416642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f0406eb70 0x7f6f0415ad10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 --2- 172.21.15.133:0/1433416642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f04158110 0x7f6f040b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1433416642 >> 172.21.15.133:0/1433416642 conn(0x7f6f04067000 msgr2=0x7f6f0406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1433416642 shutdown_connections 2024-04-25T10:02:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.636+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1433416642 wait complete. 2024-04-25T10:02:16.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f08ff8640 1 Processor -- start 2024-04-25T10:02:16.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f08ff8640 1 -- start start 2024-04-25T10:02:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f08ff8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f0406eb70 0x7f6f041649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f08ff8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f04158110 0x7f6f041f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f08ff8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f041584f0 0x7f6f04144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f08ff8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f040ccca0 con 0x7f6f04158110 2024-04-25T10:02:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f08ff8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f040ccb20 con 0x7f6f041584f0 2024-04-25T10:02:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f08ff8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f040cce20 con 0x7f6f0406eb70 2024-04-25T10:02:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f01d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f04158110 0x7f6f041f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f01d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f04158110 0x7f6f041f72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48274/0 (socket says 172.21.15.133:48274) 2024-04-25T10:02:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f01d9b640 1 -- 172.21.15.133:0/1762368241 learned_addr learned my addr 172.21.15.133:0/1762368241 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f0259c640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f0406eb70 0x7f6f041649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f02d9d640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f041584f0 0x7f6f04144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f01d9b640 1 -- 172.21.15.133:0/1762368241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f0406eb70 msgr2=0x7f6f041649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f01d9b640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f0406eb70 0x7f6f041649a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f01d9b640 1 -- 172.21.15.133:0/1762368241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f041584f0 msgr2=0x7f6f04144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f01d9b640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f041584f0 0x7f6f04144ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f01d9b640 1 -- 172.21.15.133:0/1762368241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f04164f80 con 0x7f6f04158110 2024-04-25T10:02:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f01d9b640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f04158110 0x7f6f041f72e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b410 tx=0x7f6ef800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6ef2ffd640 1 -- 172.21.15.133:0/1762368241 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8015030 con 0x7f6f04158110 2024-04-25T10:02:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6ef2ffd640 1 -- 172.21.15.133:0/1762368241 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef8004020 con 0x7f6f04158110 2024-04-25T10:02:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1762368241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f041f7b30 con 0x7f6f04158110 2024-04-25T10:02:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6ef2ffd640 1 -- 172.21.15.133:0/1762368241 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8004450 con 0x7f6f04158110 2024-04-25T10:02:16.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1762368241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f041f80f0 con 0x7f6f04158110 2024-04-25T10:02:16.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.640+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1762368241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0406bc20 con 0x7f6f04158110 2024-04-25T10:02:16.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.644+0000 7f6ef2ffd640 1 -- 172.21.15.133:0/1762368241 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ef8018070 con 0x7f6f04158110 2024-04-25T10:02:16.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.644+0000 7f6ef2ffd640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ee4066f60 0x7f6ee4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.644+0000 7f6f0259c640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ee4066f60 0x7f6ee4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.644+0000 7f6ef2ffd640 1 -- 172.21.15.133:0/1762368241 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6ef8084e90 con 0x7f6f04158110 2024-04-25T10:02:16.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.644+0000 7f6f0259c640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ee4066f60 0x7f6ee4069420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6f0406ea90 tx=0x7f6ef4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:16.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.648+0000 7f6ef2ffd640 1 -- 172.21.15.133:0/1762368241 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ef8055bf0 con 0x7f6f04158110 2024-04-25T10:02:16.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.840+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1762368241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6f04068ef0 con 0x7f6f04158110 2024-04-25T10:02:16.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.848+0000 7f6ef2ffd640 1 -- 172.21.15.133:0/1762368241 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165738 (secure 0 0 0) 0x7f6ef80554f0 con 0x7f6f04158110 2024-04-25T10:02:16.858 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:16.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1762368241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ee4066f60 msgr2=0x7f6ee4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ee4066f60 0x7f6ee4069420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6f0406ea90 tx=0x7f6ef4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:16.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1762368241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f04158110 msgr2=0x7f6f041f72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f04158110 0x7f6f041f72e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b410 tx=0x7f6ef800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1762368241 shutdown_connections 2024-04-25T10:02:16.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f041584f0 0x7f6f04144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ee4066f60 0x7f6ee4069420 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f04158110 0x7f6f041f72e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 --2- 172.21.15.133:0/1762368241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f0406eb70 0x7f6f041649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1762368241 >> 172.21.15.133:0/1762368241 conn(0x7f6f04067000 msgr2=0x7f6f040cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:16.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1762368241 shutdown_connections 2024-04-25T10:02:16.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.852+0000 7f6f08ff8640 1 -- 172.21.15.133:0/1762368241 wait complete. 2024-04-25T10:02:16.876 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1012310613},{"type":"v1","addr":"172.21.15.133:6833","nonce":1012310613}]},"active_addr":"172.21.15.133:6833/1012310613","active_change":"2024-04-25T10:01:31.691825+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":4257538017}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":3522829350}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":637892664}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":818209056}]}]} 2024-04-25T10:02:16.879 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:02:16.879 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:02:16.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:16.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.984+0000 7f331daa4640 1 Processor -- start 2024-04-25T10:02:16.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.984+0000 7f331daa4640 1 -- start start 2024-04-25T10:02:16.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.988+0000 7f331daa4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f33180cee80 0x7f3318168850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.988+0000 7f331daa4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33180ca9b0 con 0x7f331815dd90 2024-04-25T10:02:16.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.988+0000 7f331daa4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33180cac90 con 0x7f331815d9b0 2024-04-25T10:02:16.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.988+0000 7f331daa4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33180cbe30 con 0x7f33180cee80 2024-04-25T10:02:16.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.988+0000 7f3317fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f33180cee80 0x7f3318168850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.988+0000 7f3317fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f33180cee80 0x7f3318168850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48278/0 (socket says 172.21.15.133:48278) 2024-04-25T10:02:16.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.988+0000 7f3317fff640 1 -- 172.21.15.133:0/3222656309 learned_addr learned my addr 172.21.15.133:0/3222656309 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:16.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.988+0000 7f3316ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f331815dd90 0x7f33180ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51882/0 (socket says 172.21.15.133:51882) 2024-04-25T10:02:16.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.988+0000 7f3317fff640 1 -- 172.21.15.133:0/3222656309 >> v1:172.21.15.162:6789/0 conn(0x7f331815d9b0 legacy=0x7f33180cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.988+0000 7f3317fff640 1 -- 172.21.15.133:0/3222656309 >> v1:172.21.15.133:6789/0 conn(0x7f331815dd90 legacy=0x7f33180ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.988+0000 7f3317fff640 1 -- 172.21.15.133:0/3222656309 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33180caf70 con 0x7f33180cee80 2024-04-25T10:02:16.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f3317fff640 1 --2- 172.21.15.133:0/3222656309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33180cee80 0x7f3318168850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3310009870 tx=0x7f331001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1701efeca4887f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:16.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f33167fc640 1 -- 172.21.15.133:0/3222656309 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33100aa650 con 0x7f33180cee80 2024-04-25T10:02:16.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f33167fc640 1 -- 172.21.15.133:0/3222656309 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33100af070 con 0x7f33180cee80 2024-04-25T10:02:16.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f33167fc640 1 -- 172.21.15.133:0/3222656309 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33100aaa10 con 0x7f33180cee80 2024-04-25T10:02:16.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 -- 172.21.15.133:0/3222656309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33180cee80 msgr2=0x7f3318168850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 --2- 172.21.15.133:0/3222656309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33180cee80 0x7f3318168850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3310009870 tx=0x7f331001c4d0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 -- 172.21.15.133:0/3222656309 shutdown_connections 2024-04-25T10:02:16.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 --2- 172.21.15.133:0/3222656309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33180cee80 0x7f3318168850 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 -- 172.21.15.133:0/3222656309 >> 172.21.15.133:0/3222656309 conn(0x7f3318067000 msgr2=0x7f331806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:16.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 -- 172.21.15.133:0/3222656309 shutdown_connections 2024-04-25T10:02:16.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 -- 172.21.15.133:0/3222656309 wait complete. 2024-04-25T10:02:16.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 Processor -- start 2024-04-25T10:02:16.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 -- start start 2024-04-25T10:02:17.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33180cee80 0x7f3318152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f331815d9b0 0x7f3318152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f331815dd90 0x7f33181fa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331816bf60 con 0x7f331815d9b0 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331816bde0 con 0x7f331815dd90 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331816c0e0 con 0x7f33180cee80 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f3316ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f331815d9b0 0x7f3318152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f3316ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f331815d9b0 0x7f3318152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48292/0 (socket says 172.21.15.133:48292) 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f3316ffd640 1 -- 172.21.15.133:0/3041022931 learned_addr learned my addr 172.21.15.133:0/3041022931 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f3317fff640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f331815dd90 0x7f33181fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f33177fe640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33180cee80 0x7f3318152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f3316ffd640 1 -- 172.21.15.133:0/3041022931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33180cee80 msgr2=0x7f3318152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f3316ffd640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33180cee80 0x7f3318152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f3316ffd640 1 -- 172.21.15.133:0/3041022931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f331815dd90 msgr2=0x7f33181fa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f3316ffd640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f331815dd90 0x7f33181fa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f3316ffd640 1 -- 172.21.15.133:0/3041022931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331814f7e0 con 0x7f331815d9b0 2024-04-25T10:02:17.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f33177fe640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33180cee80 0x7f3318152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:17.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f3316ffd640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f331815d9b0 0x7f3318152760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f330000c460 tx=0x7f330000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f330ffff640 1 -- 172.21.15.133:0/3041022931 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300018070 con 0x7f331815d9b0 2024-04-25T10:02:17.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f330ffff640 1 -- 172.21.15.133:0/3041022931 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33000074a0 con 0x7f331815d9b0 2024-04-25T10:02:17.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 -- 172.21.15.133:0/3041022931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331814fac0 con 0x7f331815d9b0 2024-04-25T10:02:17.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f330ffff640 1 -- 172.21.15.133:0/3041022931 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300004a30 con 0x7f331815d9b0 2024-04-25T10:02:17.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.992+0000 7f331daa4640 1 -- 172.21.15.133:0/3041022931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3318150080 con 0x7f331815d9b0 2024-04-25T10:02:17.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.996+0000 7f331daa4640 1 -- 172.21.15.133:0/3041022931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3318068ef0 con 0x7f331815d9b0 2024-04-25T10:02:17.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.996+0000 7f330ffff640 1 -- 172.21.15.133:0/3041022931 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f330001b070 con 0x7f331815d9b0 2024-04-25T10:02:17.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.996+0000 7f330ffff640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32f8066f60 0x7f32f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.996+0000 7f330ffff640 1 -- 172.21.15.133:0/3041022931 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f33000848d0 con 0x7f331815d9b0 2024-04-25T10:02:17.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.996+0000 7f33177fe640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32f8066f60 0x7f32f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:16.996+0000 7f33177fe640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32f8066f60 0x7f32f8069420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3308007fd0 tx=0x7f3308009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.000+0000 7f330ffff640 1 -- 172.21.15.133:0/3041022931 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3300055680 con 0x7f331815d9b0 2024-04-25T10:02:17.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.148+0000 7f331daa4640 1 -- 172.21.15.133:0/3041022931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33180b4200 con 0x7f331815d9b0 2024-04-25T10:02:17.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f330ffff640 1 -- 172.21.15.133:0/3041022931 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7f3300054f80 con 0x7f331815d9b0 2024-04-25T10:02:17.157 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:17.157 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:17.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 -- 172.21.15.133:0/3041022931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32f8066f60 msgr2=0x7f32f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32f8066f60 0x7f32f8069420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3308007fd0 tx=0x7f3308009040 comp rx=0 tx=0).stop 2024-04-25T10:02:17.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 -- 172.21.15.133:0/3041022931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f331815d9b0 msgr2=0x7f3318152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f331815d9b0 0x7f3318152760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f330000c460 tx=0x7f330000c920 comp rx=0 tx=0).stop 2024-04-25T10:02:17.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 -- 172.21.15.133:0/3041022931 shutdown_connections 2024-04-25T10:02:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f331815dd90 0x7f33181fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32f8066f60 0x7f32f8069420 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f331815d9b0 0x7f3318152760 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 --2- 172.21.15.133:0/3041022931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f33180cee80 0x7f3318152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 -- 172.21.15.133:0/3041022931 >> 172.21.15.133:0/3041022931 conn(0x7f3318067000 msgr2=0x7f33180b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 -- 172.21.15.133:0/3041022931 shutdown_connections 2024-04-25T10:02:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.152+0000 7f331daa4640 1 -- 172.21.15.133:0/3041022931 wait complete. 2024-04-25T10:02:17.175 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:02:17.175 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:17.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.280+0000 7fdff6d68640 1 Processor -- start 2024-04-25T10:02:17.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.280+0000 7fdff6d68640 1 -- start start 2024-04-25T10:02:17.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdff6d68640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdff006b3a0 0x7fdff00b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdff6d68640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff00b4300 con 0x7fdff00b3670 2024-04-25T10:02:17.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdff6d68640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff00b45e0 con 0x7fdff015da90 2024-04-25T10:02:17.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdff6d68640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff00b48c0 con 0x7fdff006b3a0 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfeffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdff00b3670 0x7fdff006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51892/0 (socket says 172.21.15.133:51892) 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfeffff640 1 -- 172.21.15.133:0/1965134013 learned_addr learned my addr 172.21.15.133:0/1965134013 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdff5305640 1 --2- 172.21.15.133:0/1965134013 >> v2:172.21.15.162:3300/0 conn(0x7fdff006b3a0 0x7fdff00b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645747692 0 0) 0x7fdff00b4300 con 0x7fdff00b3670 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd8003530 con 0x7fdff00b3670 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3702596274 0 0) 0x7fdfd8003530 con 0x7fdff00b3670 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 >> v2:172.21.15.162:3300/0 conn(0x7fdff006b3a0 msgr2=0x7fdff00b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 --2- 172.21.15.133:0/1965134013 >> v2:172.21.15.162:3300/0 conn(0x7fdff006b3a0 0x7fdff00b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 >> v1:172.21.15.162:6789/0 conn(0x7fdff015da90 legacy=0x7fdff00729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff0072480 con 0x7fdff00b3670 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fdfe4001680 con 0x7fdff00b3670 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 >> v1:172.21.15.133:6789/0 conn(0x7fdff00b3670 legacy=0x7fdff006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfd80039c0 0x7fdfd8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 0x7fdfd8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdfd80051d0 0x7fdfd800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00b48c0 con 0x7fdfd8004810 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd800c070 con 0x7fdfd80051d0 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd800c270 con 0x7fdfd80039c0 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfeffff640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 0x7fdfd8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdff5305640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdfd80051d0 0x7fdfd800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdff4b04640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfd80039c0 0x7fdfd8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfeffff640 1 -- 172.21.15.133:0/1965134013 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfd80039c0 msgr2=0x7fdfd8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfeffff640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdfd80039c0 0x7fdfd8003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfeffff640 1 -- 172.21.15.133:0/1965134013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdfd80051d0 msgr2=0x7fdfd800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfeffff640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdfd80051d0 0x7fdfd800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdfeffff640 1 -- 172.21.15.133:0/1965134013 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd8003530 con 0x7fdfd8004810 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdff5305640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdfd80051d0 0x7fdfd800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.284+0000 7fdff5305640 1 -- 172.21.15.133:0/1965134013 reap_dead start 2024-04-25T10:02:17.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdfeffff640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 0x7fdfd8004c90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400c210 tx=0x7fdfe401f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d7d9910cbbcbba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4007d40 con 0x7fdfd8004810 2024-04-25T10:02:17.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe4005b00 con 0x7fdfd8004810 2024-04-25T10:02:17.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdfef7fe640 1 -- 172.21.15.133:0/1965134013 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4007070 con 0x7fdfd8004810 2024-04-25T10:02:17.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 -- 172.21.15.133:0/1965134013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 msgr2=0x7fdfd8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 0x7fdfd8004c90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400c210 tx=0x7fdfe401f370 comp rx=0 tx=0).stop 2024-04-25T10:02:17.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 -- 172.21.15.133:0/1965134013 shutdown_connections 2024-04-25T10:02:17.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 --2- 172.21.15.133:0/1965134013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 0x7fdfd8004c90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 -- 172.21.15.133:0/1965134013 >> 172.21.15.133:0/1965134013 conn(0x7fdff00b2370 msgr2=0x7fdff00b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 -- 172.21.15.133:0/1965134013 shutdown_connections 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 -- 172.21.15.133:0/1965134013 wait complete. 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 Processor -- start 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 -- start start 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 0x7fdff00ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff01609b0 0x7fdff01f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff00af590 0x7fdff00cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00b3c50 con 0x7fdfd8004810 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00b3ad0 con 0x7fdff01609b0 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff6d68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00b3dd0 con 0x7fdff00af590 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff4b04640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 0x7fdff00ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff4b04640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 0x7fdff00ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48316/0 (socket says 172.21.15.133:48316) 2024-04-25T10:02:17.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff4b04640 1 -- 172.21.15.133:0/1584037785 learned_addr learned my addr 172.21.15.133:0/1584037785 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff5305640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff00af590 0x7fdff00cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdfeffff640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff01609b0 0x7fdff01f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff4b04640 1 -- 172.21.15.133:0/1584037785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff00af590 msgr2=0x7fdff00cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff4b04640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff00af590 0x7fdff00cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff4b04640 1 -- 172.21.15.133:0/1584037785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff01609b0 msgr2=0x7fdff01f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff4b04640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff01609b0 0x7fdff01f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff4b04640 1 -- 172.21.15.133:0/1584037785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff00cb040 con 0x7fdfd8004810 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdfeffff640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff01609b0 0x7fdff01f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdff4b04640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 0x7fdff00ca8d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000d4c0 tx=0x7fdfe000d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.288+0000 7fdfed7fa640 1 -- 172.21.15.133:0/1584037785 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0005480 con 0x7fdfd8004810 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.292+0000 7fdfed7fa640 1 -- 172.21.15.133:0/1584037785 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe0017070 con 0x7fdfd8004810 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.292+0000 7fdfed7fa640 1 -- 172.21.15.133:0/1584037785 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0003840 con 0x7fdfd8004810 2024-04-25T10:02:17.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.292+0000 7fdff6d68640 1 -- 172.21.15.133:0/1584037785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff015e0e0 con 0x7fdfd8004810 2024-04-25T10:02:17.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.292+0000 7fdff6d68640 1 -- 172.21.15.133:0/1584037785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff015e6a0 con 0x7fdfd8004810 2024-04-25T10:02:17.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.292+0000 7fdff6d68640 1 -- 172.21.15.133:0/1584037785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff0068ef0 con 0x7fdfd8004810 2024-04-25T10:02:17.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.292+0000 7fdfed7fa640 1 -- 172.21.15.133:0/1584037785 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdfe00039e0 con 0x7fdfd8004810 2024-04-25T10:02:17.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.292+0000 7fdfed7fa640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdfdc066f90 0x7fdfdc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.292+0000 7fdfed7fa640 1 -- 172.21.15.133:0/1584037785 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdfe0085140 con 0x7fdfd8004810 2024-04-25T10:02:17.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.292+0000 7fdfeffff640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdfdc066f90 0x7fdfdc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.292+0000 7fdfeffff640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdfdc066f90 0x7fdfdc069450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40af000 tx=0x7fdfe40aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.296+0000 7fdfed7fa640 1 -- 172.21.15.133:0/1584037785 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdfe0055e10 con 0x7fdfd8004810 2024-04-25T10:02:17.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.456+0000 7fdff6d68640 1 -- 172.21.15.133:0/1584037785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdff00b2e90 con 0x7fdfd8004810 2024-04-25T10:02:17.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.456+0000 7fdfed7fa640 1 -- 172.21.15.133:0/1584037785 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7fdfe0055710 con 0x7fdfd8004810 2024-04-25T10:02:17.465 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:17.465 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:17.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 -- 172.21.15.133:0/1584037785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdfdc066f90 msgr2=0x7fdfdc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdfdc066f90 0x7fdfdc069450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40af000 tx=0x7fdfe40aa000 comp rx=0 tx=0).stop 2024-04-25T10:02:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 -- 172.21.15.133:0/1584037785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 msgr2=0x7fdff00ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 0x7fdff00ca8d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000d4c0 tx=0x7fdfe000d980 comp rx=0 tx=0).stop 2024-04-25T10:02:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 -- 172.21.15.133:0/1584037785 shutdown_connections 2024-04-25T10:02:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdff00af590 0x7fdff00cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdff01609b0 0x7fdff01f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdfdc066f90 0x7fdfdc069450 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 --2- 172.21.15.133:0/1584037785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfd8004810 0x7fdff00ca8d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 -- 172.21.15.133:0/1584037785 >> 172.21.15.133:0/1584037785 conn(0x7fdff00b2370 msgr2=0x7fdff00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 -- 172.21.15.133:0/1584037785 shutdown_connections 2024-04-25T10:02:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.460+0000 7fdff6d68640 1 -- 172.21.15.133:0/1584037785 wait complete. 2024-04-25T10:02:17.484 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:02:17.484 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:02:17.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:02:17.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:02:17.486 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:02:17.486 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:02:17.486 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:02:17.487 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:02:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.652+0000 7ffbcb06b640 1 Processor -- start 2024-04-25T10:02:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.652+0000 7ffbcb06b640 1 -- start start 2024-04-25T10:02:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.652+0000 7ffbcb06b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffbbc0501b0 0x7ffbbc006540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.652+0000 7ffbcb06b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbbc033d00 0x7ffbbc006a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.652+0000 7ffbcb06b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbbc054120 con 0x7ffbbc0df2b0 2024-04-25T10:02:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.652+0000 7ffbcb06b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbbc0df690 con 0x7ffbbc0501b0 2024-04-25T10:02:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.652+0000 7ffbcb06b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc054400 con 0x7ffbbc033d00 2024-04-25T10:02:17.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7ffbc3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffbbc0501b0 0x7ffbbc006540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7ffbc3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffbbc0501b0 0x7ffbbc006540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48328/0 (socket says 172.21.15.133:48328) 2024-04-25T10:02:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7ffbc3fff640 1 -- 172.21.15.133:0/3867187899 learned_addr learned my addr 172.21.15.133:0/3867187899 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7ffbc9608640 1 --2- 172.21.15.133:0/3867187899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbbc033d00 0x7ffbbc006a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7ffbc3fff640 1 -- 172.21.15.133:0/3867187899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbbc033d00 msgr2=0x7ffbbc006a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7ffbc3fff640 1 --2- 172.21.15.133:0/3867187899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbbc033d00 0x7ffbbc006a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7ffbc3fff640 1 -- 172.21.15.133:0/3867187899 >> v1:172.21.15.133:6789/0 conn(0x7ffbbc0df2b0 legacy=0x7ffbbc04faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7ffbc3fff640 1 -- 172.21.15.133:0/3867187899 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc0546e0 con 0x7ffbbc0501b0 2024-04-25T10:02:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7fc198217640 1 Processor -- start 2024-04-25T10:02:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7fc198217640 1 -- start start 2024-04-25T10:02:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7f214bfff640 1 Processor -- start 2024-04-25T10:02:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7ffbc3fff640 1 --2- 172.21.15.133:0/3867187899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0501b0 0x7ffbbc006540 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8008870 tx=0x7ffbb801e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7c9ef6d7600ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7ffbc37fe640 1 -- 172.21.15.133:0/3867187899 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb80ad070 con 0x7ffbbc0501b0 2024-04-25T10:02:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbc37fe640 1 -- 172.21.15.133:0/3867187899 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb80a7870 con 0x7ffbbc0501b0 2024-04-25T10:02:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbc37fe640 1 -- 172.21.15.133:0/3867187899 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb80a7c40 con 0x7ffbbc0501b0 2024-04-25T10:02:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f1e1e110640 1 Processor -- start 2024-04-25T10:02:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f1e1e110640 1 -- start start 2024-04-25T10:02:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.656+0000 7f214bfff640 1 -- start start 2024-04-25T10:02:17.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbcb06b640 1 -- 172.21.15.133:0/3867187899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0501b0 msgr2=0x7ffbbc006540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbcb06b640 1 --2- 172.21.15.133:0/3867187899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0501b0 0x7ffbbc006540 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8008870 tx=0x7ffbb801e910 comp rx=0 tx=0).stop 2024-04-25T10:02:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f214bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21440e32e0 0x7f214403e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f214bfff640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21440e7a60 con 0x7f2144007810 2024-04-25T10:02:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f214bfff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2144057df0 con 0x7f2144053c20 2024-04-25T10:02:17.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f214bfff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214403bfd0 con 0x7f21440e32e0 2024-04-25T10:02:17.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc198217640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc1900ae710 0x7fc1900b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc198217640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1900b3bb0 0x7fc190071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc198217640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc190070d90 con 0x7fc190159d10 2024-04-25T10:02:17.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc198217640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19015a0f0 con 0x7fc1900ae710 2024-04-25T10:02:17.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc198217640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900b0d10 con 0x7fc1900b3bb0 2024-04-25T10:02:17.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc195fb3640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc190159d10 0x7fc1900ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51906/0 (socket says 172.21.15.133:51906) 2024-04-25T10:02:17.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc195fb3640 1 -- 172.21.15.133:0/2542229282 learned_addr learned my addr 172.21.15.133:0/2542229282 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:17.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc1967b4640 1 --2- 172.21.15.133:0/2542229282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1900b3bb0 0x7fc190071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735209619 0 0) 0x7fc190070d90 con 0x7fc190159d10 2024-04-25T10:02:17.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc17c003530 con 0x7fc190159d10 2024-04-25T10:02:17.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1584288460 0 0) 0x7fc17c003530 con 0x7fc190159d10 2024-04-25T10:02:17.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1900b3bb0 msgr2=0x7fc190071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 --2- 172.21.15.133:0/2542229282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1900b3bb0 0x7fc190071270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 >> v2:172.21.15.162:3300/0 conn(0x7fc1900ae710 msgr2=0x7fc1900b3670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:02:17.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 --2- 172.21.15.133:0/2542229282 >> v2:172.21.15.162:3300/0 conn(0x7fc1900ae710 0x7fc1900b3670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1900b0ff0 con 0x7fc190159d10 2024-04-25T10:02:17.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbcb06b640 1 -- 172.21.15.133:0/3867187899 shutdown_connections 2024-04-25T10:02:17.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbcb06b640 1 --2- 172.21.15.133:0/3867187899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbbc033d00 0x7ffbbc006a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbcb06b640 1 --2- 172.21.15.133:0/3867187899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0501b0 0x7ffbbc006540 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbcb06b640 1 -- 172.21.15.133:0/3867187899 >> 172.21.15.133:0/3867187899 conn(0x7ffbbc009fc0 msgr2=0x7ffbbc00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f214affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21440e32e0 0x7f214403e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fc180001660 con 0x7fc190159d10 2024-04-25T10:02:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 >> v1:172.21.15.133:6789/0 conn(0x7fc190159d10 legacy=0x7fc1900ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 --2- 172.21.15.133:0/2542229282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc17c003a00 0x7fc17c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 --2- 172.21.15.133:0/2542229282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1900b3bb0 0x7fc17c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbcb06b640 1 -- 172.21.15.133:0/3867187899 shutdown_connections 2024-04-25T10:02:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 --2- 172.21.15.133:0/2542229282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc17c004d50 0x7fc17c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19015a0f0 con 0x7fc17c004d50 2024-04-25T10:02:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900b0d10 con 0x7fc17c003a00 2024-04-25T10:02:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c00bb40 con 0x7fc1900b3bb0 2024-04-25T10:02:17.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc1967b4640 1 --2- 172.21.15.133:0/2542229282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc17c004d50 0x7fc17c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f214b7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2144007810 0x7f21440e7350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51910/0 (socket says 172.21.15.133:51910) 2024-04-25T10:02:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f214b7fe640 1 -- 172.21.15.133:0/1503962058 learned_addr learned my addr 172.21.15.133:0/1503962058 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc1967b4640 1 -- 172.21.15.133:0/2542229282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1900b3bb0 msgr2=0x7fc17c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc1967b4640 1 --2- 172.21.15.133:0/2542229282 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1900b3bb0 0x7fc17c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc195fb3640 1 --2- 172.21.15.133:0/2542229282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc17c003a00 0x7fc17c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc1967b4640 1 -- 172.21.15.133:0/2542229282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc17c003a00 msgr2=0x7fc17c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc1967b4640 1 --2- 172.21.15.133:0/2542229282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc17c003a00 0x7fc17c003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7fc1967b4640 1 -- 172.21.15.133:0/2542229282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1900b0ff0 con 0x7fc17c004d50 2024-04-25T10:02:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbcb06b640 1 -- 172.21.15.133:0/3867187899 wait complete. 2024-04-25T10:02:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f8517fff640 1 Processor -- start 2024-04-25T10:02:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f8517fff640 1 -- start start 2024-04-25T10:02:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbcb06b640 1 Processor -- start 2024-04-25T10:02:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7ffbcb06b640 1 -- start start 2024-04-25T10:02:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f9679d9b640 1 Processor -- start 2024-04-25T10:02:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f9679d9b640 1 -- start start 2024-04-25T10:02:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f214affd640 1 -- 172.21.15.133:0/1503962058 >> v1:172.21.15.162:6789/0 conn(0x7f2144053c20 legacy=0x7f21440e6c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:02:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f214affd640 1 -- 172.21.15.133:0/1503962058 >> v1:172.21.15.133:6789/0 conn(0x7f2144007810 legacy=0x7f21440e7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f214affd640 1 -- 172.21.15.133:0/1503962058 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21440580d0 con 0x7f21440e32e0 2024-04-25T10:02:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f1e1e110640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e10004b40 0x7f1e10053d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.660+0000 7f1e1e110640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e1003a8a0 0x7f1e1004fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e177fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e10004b40 0x7f1e10053d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e16ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e1003a8a0 0x7f1e1004fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e16ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e1003a8a0 0x7f1e1004fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48358/0 (socket says 172.21.15.133:48358) 2024-04-25T10:02:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f8517fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8518162dc0 0x7f85180c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f8517fff640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85180b0070 con 0x7f851815f9d0 2024-04-25T10:02:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f8517fff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851806e0b0 con 0x7f85180cc800 2024-04-25T10:02:17.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f8517fff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8518163220 con 0x7f8518162dc0 2024-04-25T10:02:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbcb06b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbbc033d00 0x7ffbbc04d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbcb06b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0501b0 0x7ffbbc04c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbcb06b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbbc0df2b0 0x7ffbbc0dab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbcb06b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc04f7e0 con 0x7ffbbc0501b0 2024-04-25T10:02:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbcb06b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc04f660 con 0x7ffbbc0df2b0 2024-04-25T10:02:17.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbcb06b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0ec050 con 0x7ffbbc033d00 2024-04-25T10:02:17.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e177fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e10004b40 0x7f1e10053d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39364/0 (socket says 172.21.15.133:39364) 2024-04-25T10:02:17.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e16ffd640 1 -- 172.21.15.133:0/565653444 learned_addr learned my addr 172.21.15.133:0/565653444 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e1e110640 1 -- 172.21.15.133:0/565653444 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e10054680 con 0x7f1e10050120 2024-04-25T10:02:17.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f8516ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f851815f9d0 0x7f85180cc0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51926/0 (socket says 172.21.15.133:51926) 2024-04-25T10:02:17.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f8516ffd640 1 -- 172.21.15.133:0/682167898 learned_addr learned my addr 172.21.15.133:0/682167898 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:17.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e1e110640 1 -- 172.21.15.133:0/565653444 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e10037fb0 con 0x7f1e1003a8a0 2024-04-25T10:02:17.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0501b0 0x7ffbbc04c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0501b0 0x7ffbbc04c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48362/0 (socket says 172.21.15.133:48362) 2024-04-25T10:02:17.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbc3fff640 1 -- 172.21.15.133:0/1100568899 learned_addr learned my addr 172.21.15.133:0/1100568899 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e1e110640 1 -- 172.21.15.133:0/565653444 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e10054960 con 0x7f1e10004b40 2024-04-25T10:02:17.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbc8e07640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbbc033d00 0x7ffbbc04d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbc9608640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbbc0df2b0 0x7ffbbc0dab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85177fe640 1 --2- 172.21.15.133:0/682167898 >> v2:172.21.15.162:3300/0 conn(0x7f8518162dc0 0x7f85180c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e167fc640 1 -- 172.21.15.133:0/565653444 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064255227 0 0) 0x7f1e10054680 con 0x7f1e10050120 2024-04-25T10:02:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9679d9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967415d830 0x7f96740d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9679d9b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f96741615a0 0x7f96740d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9679d9b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9674161d10 con 0x7f967415dc10 2024-04-25T10:02:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9679d9b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9674161ff0 con 0x7f96741615a0 2024-04-25T10:02:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9679d9b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96741622d0 con 0x7f967415d830 2024-04-25T10:02:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fc1967b4640 1 --2- 172.21.15.133:0/2542229282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc17c004d50 0x7fc17c0051d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc188010290 tx=0x7fc18801eff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49caf4362fcfa77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e167fc640 1 -- 172.21.15.133:0/565653444 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e08003530 con 0x7f1e10050120 2024-04-25T10:02:17.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9673fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f967415dc10 0x7f967406bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51932/0 (socket says 172.21.15.133:51932) 2024-04-25T10:02:17.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9673fff640 1 -- 172.21.15.133:0/3863646947 learned_addr learned my addr 172.21.15.133:0/3863646947 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:17.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9678d99640 1 --2- 172.21.15.133:0/3863646947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967415d830 0x7f96740d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338643281 0 0) 0x7f8508001120 con 0x7f851815f9d0 2024-04-25T10:02:17.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8504003530 con 0x7f851815f9d0 2024-04-25T10:02:17.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f967959a640 1 --2- 172.21.15.133:0/3863646947 >> v2:172.21.15.162:3300/0 conn(0x7f96741615a0 0x7f96740d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fd97f59e640 1 Processor -- start 2024-04-25T10:02:17.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1880b4070 con 0x7fc17c004d50 2024-04-25T10:02:17.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1880af440 con 0x7fc17c004d50 2024-04-25T10:02:17.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fc194fb1640 1 -- 172.21.15.133:0/2542229282 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188009d60 con 0x7fc17c004d50 2024-04-25T10:02:17.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fd97f59e640 1 -- start start 2024-04-25T10:02:17.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e16ffd640 1 -- 172.21.15.133:0/565653444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e10004b40 msgr2=0x7f1e10053d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e16ffd640 1 --2- 172.21.15.133:0/565653444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e10004b40 0x7f1e10053d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e16ffd640 1 -- 172.21.15.133:0/565653444 >> v1:172.21.15.133:6789/0 conn(0x7f1e10050120 legacy=0x7f1e10006bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f1e16ffd640 1 -- 172.21.15.133:0/565653444 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e10054c40 con 0x7f1e1003a8a0 2024-04-25T10:02:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3002285556 0 0) 0x7f8504003530 con 0x7f851815f9d0 2024-04-25T10:02:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 >> v2:172.21.15.162:3300/0 conn(0x7f8518162dc0 msgr2=0x7f85180c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 --2- 172.21.15.133:0/682167898 >> v2:172.21.15.162:3300/0 conn(0x7f8518162dc0 0x7f85180c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 >> v1:172.21.15.162:6789/0 conn(0x7f85180cc800 legacy=0x7f85180b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851806e390 con 0x7f851815f9d0 2024-04-25T10:02:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f8508001850 con 0x7f851815f9d0 2024-04-25T10:02:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 >> v1:172.21.15.133:6789/0 conn(0x7f851815f9d0 legacy=0x7f85180cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 --2- 172.21.15.133:0/682167898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85040039c0 0x7f8504003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 --2- 172.21.15.133:0/682167898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8504004810 0x7f8504004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 --2- 172.21.15.133:0/682167898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85040051d0 0x7f850400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8518163220 con 0x7f85040051d0 2024-04-25T10:02:17.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850400c070 con 0x7f8504004810 2024-04-25T10:02:17.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850400c270 con 0x7f85040039c0 2024-04-25T10:02:17.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f214affd640 1 --2- 172.21.15.133:0/1503962058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21440e32e0 0x7f214403e850 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f213c0097f0 tx=0x7f213c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72d64f55dc8daee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbc8e07640 1 -- 172.21.15.133:0/1100568899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbbc0df2b0 msgr2=0x7ffbbc0dab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbc8e07640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbbc0df2b0 0x7ffbbc0dab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbc8e07640 1 -- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0501b0 msgr2=0x7ffbbc04c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbc8e07640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0501b0 0x7ffbbc04c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbc8e07640 1 -- 172.21.15.133:0/1100568899 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc04cc10 con 0x7ffbbc033d00 2024-04-25T10:02:17.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f850ffff640 1 --2- 172.21.15.133:0/682167898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8504004810 0x7f8504004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85177fe640 1 --2- 172.21.15.133:0/682167898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85040051d0 0x7f850400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f8516ffd640 1 --2- 172.21.15.133:0/682167898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85040039c0 0x7f8504003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fc198217640 1 -- 172.21.15.133:0/2542229282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc17c004d50 msgr2=0x7fc17c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fc198217640 1 --2- 172.21.15.133:0/2542229282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc17c004d50 0x7fc17c0051d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc188010290 tx=0x7fc18801eff0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7ffbc8e07640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbbc033d00 0x7ffbbc04d700 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffbc404fd80 tx=0x7ffbc4068750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fc1967b4640 1 -- 172.21.15.133:0/2542229282 reap_dead start 2024-04-25T10:02:17.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85177fe640 1 -- 172.21.15.133:0/682167898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85040039c0 msgr2=0x7f8504003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85177fe640 1 --2- 172.21.15.133:0/682167898 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85040039c0 0x7f8504003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85177fe640 1 -- 172.21.15.133:0/682167898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8504004810 msgr2=0x7f8504004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85177fe640 1 --2- 172.21.15.133:0/682167898 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8504004810 0x7f8504004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f85177fe640 1 -- 172.21.15.133:0/682167898 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851806e390 con 0x7f85040051d0 2024-04-25T10:02:17.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fc198217640 1 -- 172.21.15.133:0/2542229282 shutdown_connections 2024-04-25T10:02:17.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fc198217640 1 -- 172.21.15.133:0/2542229282 >> 172.21.15.133:0/2542229282 conn(0x7fc1900cbd10 msgr2=0x7fc1900cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fc198217640 1 -- 172.21.15.133:0/2542229282 shutdown_connections 2024-04-25T10:02:17.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fc198217640 1 -- 172.21.15.133:0/2542229282 wait complete. 2024-04-25T10:02:17.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9678d99640 1 -- 172.21.15.133:0/3863646947 >> v2:172.21.15.162:3300/0 conn(0x7f96741615a0 msgr2=0x7f96740d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9678d99640 1 --2- 172.21.15.133:0/3863646947 >> v2:172.21.15.162:3300/0 conn(0x7f96741615a0 0x7f96740d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9678d99640 1 -- 172.21.15.133:0/3863646947 >> v1:172.21.15.133:6789/0 conn(0x7f967415dc10 legacy=0x7f967406bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f96737fe640 1 -- 172.21.15.133:0/3863646947 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857216829 0 0) 0x7f9674161d10 con 0x7f967415dc10 2024-04-25T10:02:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f9678d99640 1 -- 172.21.15.133:0/3863646947 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674161a10 con 0x7f967415d830 2024-04-25T10:02:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fd97f59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 0x7fd9780eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fd97f59e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd978034220 con 0x7fd978047c20 2024-04-25T10:02:17.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f2149ffb640 1 -- 172.21.15.133:0/1503962058 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c0b2070 con 0x7f21440e32e0 2024-04-25T10:02:17.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f2149ffb640 1 -- 172.21.15.133:0/1503962058 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c0ad640 con 0x7f21440e32e0 2024-04-25T10:02:17.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7f2149ffb640 1 -- 172.21.15.133:0/1503962058 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c0ada10 con 0x7f21440e32e0 2024-04-25T10:02:17.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fd97dd9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd978047c20 0x7fd9780eb6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51944/0 (socket says 172.21.15.133:51944) 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fd97dd9b640 1 -- 172.21.15.133:0/2976906528 learned_addr learned my addr 172.21.15.133:0/2976906528 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.664+0000 7fd97e59c640 1 --2- 172.21.15.133:0/2976906528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 0x7fd9780eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fd97f59e640 1 -- 172.21.15.133:0/2976906528 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97804fa30 con 0x7fd978005ea0 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fd97f59e640 1 -- 172.21.15.133:0/2976906528 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd978050210 con 0x7fd978033e40 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fd97d59a640 1 -- 172.21.15.133:0/2976906528 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780425653 0 0) 0x7fd978034220 con 0x7fd978047c20 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fd97d59a640 1 -- 172.21.15.133:0/2976906528 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd968003530 con 0x7fd978047c20 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fd97d59a640 1 -- 172.21.15.133:0/2976906528 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109197062 0 0) 0x7fd97804fa30 con 0x7fd978005ea0 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fd97d59a640 1 -- 172.21.15.133:0/2976906528 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd978034220 con 0x7fd978005ea0 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fd97e59c640 1 -- 172.21.15.133:0/2976906528 >> v1:172.21.15.162:6789/0 conn(0x7fd978005ea0 legacy=0x7fd9780062f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fd97e59c640 1 -- 172.21.15.133:0/2976906528 >> v1:172.21.15.133:6789/0 conn(0x7fd978047c20 legacy=0x7fd9780eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f214bfff640 1 -- 172.21.15.133:0/1503962058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21440e32e0 msgr2=0x7f214403e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f214bfff640 1 --2- 172.21.15.133:0/1503962058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21440e32e0 0x7f214403e850 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f213c0097f0 tx=0x7f213c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f214bfff640 1 -- 172.21.15.133:0/1503962058 shutdown_connections 2024-04-25T10:02:17.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f214bfff640 1 --2- 172.21.15.133:0/1503962058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21440e32e0 0x7f214403e850 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f214bfff640 1 -- 172.21.15.133:0/1503962058 >> 172.21.15.133:0/1503962058 conn(0x7f21440559f0 msgr2=0x7f2144055e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc198217640 1 Processor -- start 2024-04-25T10:02:17.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fd97e59c640 1 -- 172.21.15.133:0/2976906528 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97804f2e0 con 0x7fd978033e40 2024-04-25T10:02:17.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e16ffd640 1 --2- 172.21.15.133:0/565653444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e1003a8a0 0x7f1e1004fbe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009870 tx=0x7f1e0401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f7601637b0ca7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e167fc640 1 -- 172.21.15.133:0/565653444 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e040b5070 con 0x7f1e1003a8a0 2024-04-25T10:02:17.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e167fc640 1 -- 172.21.15.133:0/565653444 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e040a8a50 con 0x7f1e1003a8a0 2024-04-25T10:02:17.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e167fc640 1 -- 172.21.15.133:0/565653444 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e040a8e20 con 0x7f1e1003a8a0 2024-04-25T10:02:17.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc198217640 1 -- start start 2024-04-25T10:02:17.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc198217640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc17c004d50 0x7fc1900c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc198217640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc190155730 0x7fc1900acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc198217640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc190155bb0 0x7fc1900ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc198217640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900b1890 con 0x7fc190155bb0 2024-04-25T10:02:17.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc198217640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900b15b0 con 0x7fc190155730 2024-04-25T10:02:17.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc198217640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900cc830 con 0x7fc17c004d50 2024-04-25T10:02:17.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f9678d99640 1 --2- 172.21.15.133:0/3863646947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967415d830 0x7f96740d15c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f96600097c0 tx=0x7f966001c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f542a3e1ff399f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f85177fe640 1 -- 172.21.15.133:0/682167898 reap_dead start 2024-04-25T10:02:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc195fb3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc17c004d50 0x7fc1900c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1967b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc190155bb0 0x7fc1900ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f96737fe640 1 -- 172.21.15.133:0/3863646947 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96600b0070 con 0x7f967415d830 2024-04-25T10:02:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc195fb3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc17c004d50 0x7fc1900c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39394/0 (socket says 172.21.15.133:39394) 2024-04-25T10:02:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc195fb3640 1 -- 172.21.15.133:0/875313149 learned_addr learned my addr 172.21.15.133:0/875313149 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1967b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc190155bb0 0x7fc1900ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48376/0 (socket says 172.21.15.133:48376) 2024-04-25T10:02:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1957b2640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc190155730 0x7fc1900acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f96737fe640 1 -- 172.21.15.133:0/3863646947 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96600ab600 con 0x7f967415d830 2024-04-25T10:02:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f96737fe640 1 -- 172.21.15.133:0/3863646947 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96600ab9d0 con 0x7f967415d830 2024-04-25T10:02:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1967b4640 1 -- 172.21.15.133:0/875313149 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc17c004d50 msgr2=0x7fc1900c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1967b4640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc17c004d50 0x7fc1900c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1967b4640 1 -- 172.21.15.133:0/875313149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc190155730 msgr2=0x7fc1900acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1967b4640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc190155730 0x7fc1900acf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1967b4640 1 -- 172.21.15.133:0/875313149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1901ef2c0 con 0x7fc190155bb0 2024-04-25T10:02:17.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e1e110640 1 -- 172.21.15.133:0/565653444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e1003a8a0 msgr2=0x7f1e1004fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e1e110640 1 --2- 172.21.15.133:0/565653444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e1003a8a0 0x7f1e1004fbe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009870 tx=0x7f1e0401ed50 comp rx=0 tx=0).stop 2024-04-25T10:02:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e1e110640 1 -- 172.21.15.133:0/565653444 shutdown_connections 2024-04-25T10:02:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e1e110640 1 --2- 172.21.15.133:0/565653444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e1003a8a0 0x7f1e1004fbe0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e1e110640 1 --2- 172.21.15.133:0/565653444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e10004b40 0x7f1e10053d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e1e110640 1 -- 172.21.15.133:0/565653444 >> 172.21.15.133:0/565653444 conn(0x7f1e1000a440 msgr2=0x7f1e1000a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f9679d9b640 1 -- 172.21.15.133:0/3863646947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967415d830 msgr2=0x7f96740d15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f9679d9b640 1 --2- 172.21.15.133:0/3863646947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967415d830 0x7f96740d15c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f96600097c0 tx=0x7f966001c430 comp rx=0 tx=0).stop 2024-04-25T10:02:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1967b4640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc190155bb0 0x7fc1900ad450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc1880ae000 tx=0x7fc188009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e1e110640 1 -- 172.21.15.133:0/565653444 shutdown_connections 2024-04-25T10:02:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f9679d9b640 1 -- 172.21.15.133:0/3863646947 shutdown_connections 2024-04-25T10:02:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f9679d9b640 1 --2- 172.21.15.133:0/3863646947 >> v2:172.21.15.162:3300/0 conn(0x7f96741615a0 0x7f96740d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f9679d9b640 1 --2- 172.21.15.133:0/3863646947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967415d830 0x7f96740d15c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f9679d9b640 1 -- 172.21.15.133:0/3863646947 >> 172.21.15.133:0/3863646947 conn(0x7f967409c9c0 msgr2=0x7f96740d36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f1e1e110640 1 -- 172.21.15.133:0/565653444 wait complete. 2024-04-25T10:02:17.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7ffbc17fa640 1 -- 172.21.15.133:0/1100568899 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40731c0 con 0x7ffbbc033d00 2024-04-25T10:02:17.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7ffbc17fa640 1 -- 172.21.15.133:0/1100568899 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc4073b20 con 0x7ffbbc033d00 2024-04-25T10:02:17.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7ffbc17fa640 1 -- 172.21.15.133:0/1100568899 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4092400 con 0x7ffbbc033d00 2024-04-25T10:02:17.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f9679d9b640 1 -- 172.21.15.133:0/3863646947 shutdown_connections 2024-04-25T10:02:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1867fc640 1 -- 172.21.15.133:0/875313149 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1880b4040 con 0x7fc190155bb0 2024-04-25T10:02:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1867fc640 1 -- 172.21.15.133:0/875313149 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1880b0c20 con 0x7fc190155bb0 2024-04-25T10:02:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc1867fc640 1 -- 172.21.15.133:0/875313149 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188009d60 con 0x7fc190155bb0 2024-04-25T10:02:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc198217640 1 -- 172.21.15.133:0/875313149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1900ada80 con 0x7fc190155bb0 2024-04-25T10:02:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f85177fe640 1 --2- 172.21.15.133:0/682167898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85040051d0 0x7f850400b9d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8510009fa0 tx=0x7f851001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee642e6e2cf52f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7fc198217640 1 -- 172.21.15.133:0/875313149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1900c4a00 con 0x7fc190155bb0 2024-04-25T10:02:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7ffbcb06b640 1 -- 172.21.15.133:0/1100568899 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc04de70 con 0x7ffbbc033d00 2024-04-25T10:02:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fd97e59c640 1 --2- 172.21.15.133:0/2976906528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 0x7fd9780eb180 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd9700097f0 tx=0x7fd97001ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d2f12b68645bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100b2070 con 0x7f85040051d0 2024-04-25T10:02:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85100a5d50 con 0x7f85040051d0 2024-04-25T10:02:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f85167fc640 1 -- 172.21.15.133:0/682167898 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510009bf0 con 0x7f85040051d0 2024-04-25T10:02:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fc198217640 1 -- 172.21.15.133:0/875313149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1900c50c0 con 0x7fc190155bb0 2024-04-25T10:02:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.668+0000 7f9679d9b640 1 -- 172.21.15.133:0/3863646947 wait complete. 2024-04-25T10:02:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214bfff640 1 -- 172.21.15.133:0/1503962058 shutdown_connections 2024-04-25T10:02:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214bfff640 1 -- 172.21.15.133:0/1503962058 wait complete. 2024-04-25T10:02:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f778ffff640 1 Processor -- start 2024-04-25T10:02:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9679d9b640 1 Processor -- start 2024-04-25T10:02:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9679d9b640 1 -- start start 2024-04-25T10:02:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fd97d59a640 1 -- 172.21.15.133:0/2976906528 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700b5070 con 0x7fd978033e40 2024-04-25T10:02:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fd97d59a640 1 -- 172.21.15.133:0/2976906528 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9700a89f0 con 0x7fd978033e40 2024-04-25T10:02:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fd97d59a640 1 -- 172.21.15.133:0/2976906528 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700a8da0 con 0x7fd978033e40 2024-04-25T10:02:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f8517fff640 1 -- 172.21.15.133:0/682167898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85040051d0 msgr2=0x7f850400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f8517fff640 1 --2- 172.21.15.133:0/682167898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85040051d0 0x7f850400b9d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8510009fa0 tx=0x7f851001cc80 comp rx=0 tx=0).stop 2024-04-25T10:02:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7ffbcb06b640 1 -- 172.21.15.133:0/1100568899 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc04e430 con 0x7ffbbc033d00 2024-04-25T10:02:17.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fc1867fc640 1 -- 172.21.15.133:0/875313149 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1880039a0 con 0x7fc190155bb0 2024-04-25T10:02:17.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f8517fff640 1 -- 172.21.15.133:0/682167898 shutdown_connections 2024-04-25T10:02:17.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f8517fff640 1 --2- 172.21.15.133:0/682167898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85040051d0 0x7f850400b9d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f8517fff640 1 -- 172.21.15.133:0/682167898 >> 172.21.15.133:0/682167898 conn(0x7f851806ac90 msgr2=0x7f851806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f1e1e110640 1 Processor -- start 2024-04-25T10:02:17.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f8517fff640 1 -- 172.21.15.133:0/682167898 shutdown_connections 2024-04-25T10:02:17.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f1e1e110640 1 -- start start 2024-04-25T10:02:17.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fc1867fc640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc174066e40 0x7fc174069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fc1867fc640 1 -- 172.21.15.133:0/875313149 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc1880c0070 con 0x7fc190155bb0 2024-04-25T10:02:17.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9679d9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f967415d830 0x7f967414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9679d9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967415dc10 0x7f967415c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9679d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96741615a0 0x7f967415f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9679d9b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96740cf5e0 con 0x7f96741615a0 2024-04-25T10:02:17.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9679d9b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96740cf460 con 0x7f967415d830 2024-04-25T10:02:17.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9679d9b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967416a8e0 con 0x7f967415dc10 2024-04-25T10:02:17.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214bfff640 1 Processor -- start 2024-04-25T10:02:17.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214bfff640 1 -- start start 2024-04-25T10:02:17.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fc1867fc640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] conn(0x7fc1740704c0 0x7fc174072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fc1867fc640 1 -- 172.21.15.133:0/875313149 --> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1880c0e50 con 0x7fc1740704c0 2024-04-25T10:02:17.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9678d99640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f967415d830 0x7f967414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9678d99640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f967415d830 0x7f967414b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55428/0 (socket says 172.21.15.133:55428) 2024-04-25T10:02:17.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9678d99640 1 -- 172.21.15.133:0/1984560354 learned_addr learned my addr 172.21.15.133:0/1984560354 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f967959a640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96741615a0 0x7f967415f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f778ffff640 1 -- start start 2024-04-25T10:02:17.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fc1957b2640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] conn(0x7fc1740704c0 0x7fc174072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f967959a640 1 -- 172.21.15.133:0/1984560354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967415dc10 msgr2=0x7f967415c420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:02:17.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f967959a640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f967415dc10 0x7f967415c420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f967959a640 1 -- 172.21.15.133:0/1984560354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f967415d830 msgr2=0x7f967414b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f967959a640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f967415d830 0x7f967414b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f967959a640 1 -- 172.21.15.133:0/1984560354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674155510 con 0x7f96741615a0 2024-04-25T10:02:17.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f1e1e110640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e10004b40 0x7f1e100d7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f9678d99640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f967415d830 0x7f967414b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:17.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fd97f59e640 1 -- 172.21.15.133:0/2976906528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 msgr2=0x7fd9780eb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fd97f59e640 1 --2- 172.21.15.133:0/2976906528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 0x7fd9780eb180 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd9700097f0 tx=0x7fd97001ecf0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2144007810 0x7f214404e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f967959a640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96741615a0 0x7f967415f110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f966c00ebd0 tx=0x7f966c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2144053c20 0x7f2144183ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2144052c60 0x7f214404c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214bfff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2144057bd0 con 0x7f2144053c20 2024-04-25T10:02:17.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214bfff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2144057a50 con 0x7f2144007810 2024-04-25T10:02:17.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214bfff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214403ed90 con 0x7f2144052c60 2024-04-25T10:02:17.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f1e177fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e10004b40 0x7f1e100d7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f1e177fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e10004b40 0x7f1e100d7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48398/0 (socket says 172.21.15.133:48398) 2024-04-25T10:02:17.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fc195fb3640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc174066e40 0x7fc174069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fd97f59e640 1 -- 172.21.15.133:0/2976906528 shutdown_connections 2024-04-25T10:02:17.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fd97f59e640 1 --2- 172.21.15.133:0/2976906528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 0x7fd9780eb180 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fd97f59e640 1 -- 172.21.15.133:0/2976906528 >> 172.21.15.133:0/2976906528 conn(0x7fd978009fc0 msgr2=0x7fd97800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f1e177fe640 1 -- 172.21.15.133:0/2440266642 learned_addr learned my addr 172.21.15.133:0/2440266642 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f1e1e110640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1003a8a0 0x7f1e1004ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fc1957b2640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] conn(0x7fc1740704c0 0x7fc174072900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fd97f59e640 1 -- 172.21.15.133:0/2976906528 shutdown_connections 2024-04-25T10:02:17.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7fd97f59e640 1 -- 172.21.15.133:0/2976906528 wait complete. 2024-04-25T10:02:17.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214affd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2144007810 0x7f214404e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f8517fff640 1 -- 172.21.15.133:0/682167898 wait complete. 2024-04-25T10:02:17.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f8517fff640 1 Processor -- start 2024-04-25T10:02:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214affd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2144007810 0x7f214404e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55434/0 (socket says 172.21.15.133:55434) 2024-04-25T10:02:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214affd640 1 -- 172.21.15.133:0/3444021216 learned_addr learned my addr 172.21.15.133:0/3444021216 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f214b7fe640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2144052c60 0x7f214404c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f1e1e110640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e10050120 0x7f1e100d4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f1e1e110640 1 -- 172.21.15.133:0/2440266642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100e04e0 con 0x7f1e10004b40 2024-04-25T10:02:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7f1e1e110640 1 -- 172.21.15.133:0/2440266642 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100066a0 con 0x7f1e10050120 2024-04-25T10:02:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e1e110640 1 -- 172.21.15.133:0/2440266642 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100e0880 con 0x7f1e1003a8a0 2024-04-25T10:02:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e16ffd640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1003a8a0 0x7f1e1004ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fc195fb3640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc174066e40 0x7fc174069300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc1800017d0 tx=0x7fc180009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e17fff640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e10050120 0x7f1e100d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f214affd640 1 -- 172.21.15.133:0/3444021216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2144052c60 msgr2=0x7f214404c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f214affd640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2144052c60 0x7f214404c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f214affd640 1 -- 172.21.15.133:0/3444021216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2144053c20 msgr2=0x7f2144183ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f214affd640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2144053c20 0x7f2144183ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f214affd640 1 -- 172.21.15.133:0/3444021216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214404ef80 con 0x7f2144007810 2024-04-25T10:02:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778ffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779015a960 0x7f779006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f214affd640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2144007810 0x7f214404e980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f213c01a0d0 tx=0x7f213c019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e177fe640 1 -- 172.21.15.133:0/2440266642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1003a8a0 msgr2=0x7f1e1004ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e177fe640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1003a8a0 0x7f1e1004ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e177fe640 1 -- 172.21.15.133:0/2440266642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e10050120 msgr2=0x7f1e100d4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e177fe640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e10050120 0x7f1e100d4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e177fe640 1 -- 172.21.15.133:0/2440266642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e100d78c0 con 0x7f1e10004b40 2024-04-25T10:02:17.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778ffff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f77900cd980 0x7f77900b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f77900b2310 0x7f7790071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97f59e640 1 Processor -- start 2024-04-25T10:02:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778ffff640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779015ad40 con 0x7f77900b2310 2024-04-25T10:02:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f21337fe640 1 -- 172.21.15.133:0/3444021216 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c0bd020 con 0x7f2144007810 2024-04-25T10:02:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e177fe640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e10004b40 0x7f1e100d7070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00d5f0 tx=0x7f1e0c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778effd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779015a960 0x7f779006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778effd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779015a960 0x7f779006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39416/0 (socket says 172.21.15.133:39416) 2024-04-25T10:02:17.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778effd640 1 -- 172.21.15.133:0/1169910659 learned_addr learned my addr 172.21.15.133:0/1169910659 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778ffff640 1 -- 172.21.15.133:0/1169910659 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779006f090 con 0x7f77900cd980 2024-04-25T10:02:17.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778ffff640 1 -- 172.21.15.133:0/1169910659 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77900ce5c0 con 0x7f779015a960 2024-04-25T10:02:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778e7fc640 1 --2- 172.21.15.133:0/1169910659 >> v2:172.21.15.162:3300/0 conn(0x7f77900cd980 0x7f77900b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778f7fe640 1 --2- 172.21.15.133:0/1169910659 >> v2:172.21.15.133:3300/0 conn(0x7f77900b2310 0x7f7790071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7ffbcb06b640 1 -- 172.21.15.133:0/1100568899 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffbbc0053a0 con 0x7ffbbc033d00 2024-04-25T10:02:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.672+0000 7ffbc17fa640 1 -- 172.21.15.133:0/1100568899 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffbc40925c0 con 0x7ffbbc033d00 2024-04-25T10:02:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7ffbc17fa640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffba8067030 0x7ffba80694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7ffbc17fa640 1 -- 172.21.15.133:0/1100568899 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffbc40ff580 con 0x7ffbbc033d00 2024-04-25T10:02:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7ffbc17fa640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] conn(0x7ffba8070730 0x7ffba8072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7ffbc17fa640 1 -- 172.21.15.133:0/1100568899 --> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffbc408ce50 con 0x7ffba8070730 2024-04-25T10:02:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7ffbc9608640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] conn(0x7ffba8070730 0x7ffba8072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7ffbc3fff640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffba8067030 0x7ffba80694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97f59e640 1 -- start start 2024-04-25T10:02:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778f7fe640 1 -- 172.21.15.133:0/1169910659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779015a960 msgr2=0x7f779006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778f7fe640 1 --2- 172.21.15.133:0/1169910659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779015a960 0x7f779006f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778f7fe640 1 -- 172.21.15.133:0/1169910659 >> v2:172.21.15.162:3300/0 conn(0x7f77900cd980 msgr2=0x7f77900b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778f7fe640 1 --2- 172.21.15.133:0/1169910659 >> v2:172.21.15.162:3300/0 conn(0x7f77900cd980 0x7f77900b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f778f7fe640 1 -- 172.21.15.133:0/1169910659 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779006ea40 con 0x7f77900b2310 2024-04-25T10:02:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7ffbc3fff640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffba8067030 0x7ffba80694f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8000c10 tx=0x7ffbb800a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e03fff640 1 -- 172.21.15.133:0/2440266642 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0125f0 con 0x7f1e10004b40 2024-04-25T10:02:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e03fff640 1 -- 172.21.15.133:0/2440266642 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c019070 con 0x7f1e10004b40 2024-04-25T10:02:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e1e110640 1 -- 172.21.15.133:0/2440266642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e100d7ba0 con 0x7f1e10004b40 2024-04-25T10:02:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f1e1e110640 1 -- 172.21.15.133:0/2440266642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e100d8160 con 0x7f1e10004b40 2024-04-25T10:02:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f96717fa640 1 -- 172.21.15.133:0/1984560354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c018020 con 0x7f96741615a0 2024-04-25T10:02:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f9679d9b640 1 -- 172.21.15.133:0/1984560354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96741557f0 con 0x7f96741615a0 2024-04-25T10:02:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f96717fa640 1 -- 172.21.15.133:0/1984560354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966c00cd20 con 0x7f96741615a0 2024-04-25T10:02:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f96717fa640 1 -- 172.21.15.133:0/1984560354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c0146c0 con 0x7f96741615a0 2024-04-25T10:02:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f9679d9b640 1 -- 172.21.15.133:0/1984560354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967414c410 con 0x7f96741615a0 2024-04-25T10:02:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fc1867fc640 1 -- 172.21.15.133:0/875313149 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc188126050 con 0x7fc190155bb0 2024-04-25T10:02:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fc1867fc640 1 -- 172.21.15.133:0/875313149 <== osd.5 v2:172.21.15.162:6803/4006238640 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc1880c0e50 con 0x7fc1740704c0 2024-04-25T10:02:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd978005ea0 0x7fd9780d8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97f59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 0x7fd9780e7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97f59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd978047c20 0x7fd978036890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97f59e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97804f090 con 0x7fd978005ea0 2024-04-25T10:02:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97f59e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97804fd80 con 0x7fd978047c20 2024-04-25T10:02:17.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97f59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780482c0 con 0x7fd978033e40 2024-04-25T10:02:17.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97dd9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 0x7fd9780e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97dd9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 0x7fd9780e7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39426/0 (socket says 172.21.15.133:39426) 2024-04-25T10:02:17.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97dd9b640 1 -- 172.21.15.133:0/3217083110 learned_addr learned my addr 172.21.15.133:0/3217083110 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f214bfff640 1 -- 172.21.15.133:0/3444021216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214404f260 con 0x7f2144007810 2024-04-25T10:02:17.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f214bfff640 1 -- 172.21.15.133:0/3444021216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21440e3fb0 con 0x7f2144007810 2024-04-25T10:02:17.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f21337fe640 1 -- 172.21.15.133:0/3444021216 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c0b8670 con 0x7f2144007810 2024-04-25T10:02:17.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f21337fe640 1 -- 172.21.15.133:0/3444021216 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c0b2040 con 0x7f2144007810 2024-04-25T10:02:17.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f214bfff640 1 -- 172.21.15.133:0/3444021216 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f21440057f0 con 0x7f2144007810 2024-04-25T10:02:17.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97dd9b640 1 -- 172.21.15.133:0/3217083110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd978047c20 msgr2=0x7fd978036890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:02:17.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97dd9b640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd978047c20 0x7fd978036890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97dd9b640 1 -- 172.21.15.133:0/3217083110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd978005ea0 msgr2=0x7fd9780d8090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:02:17.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97dd9b640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd978005ea0 0x7fd9780d8090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97dd9b640 1 -- 172.21.15.133:0/3217083110 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780497b0 con 0x7fd978033e40 2024-04-25T10:02:17.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7ffbc17fa640 1 -- 172.21.15.133:0/1100568899 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbc4106050 con 0x7ffbbc033d00 2024-04-25T10:02:17.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7fd97dd9b640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 0x7fd9780e7940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c460 tx=0x7fd97400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f9679d9b640 1 -- 172.21.15.133:0/1984560354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9674074da0 con 0x7f96741615a0 2024-04-25T10:02:17.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.676+0000 7f8517fff640 1 -- start start 2024-04-25T10:02:17.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f8517fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85040051d0 0x7f8518143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7ffbc9608640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] conn(0x7ffba8070730 0x7ffba8072b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7fd96effd640 1 -- 172.21.15.133:0/3217083110 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974017070 con 0x7fd978033e40 2024-04-25T10:02:17.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7fd96effd640 1 -- 172.21.15.133:0/3217083110 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd974005db0 con 0x7fd978033e40 2024-04-25T10:02:17.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7fd97f59e640 1 -- 172.21.15.133:0/3217083110 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd978049a90 con 0x7fd978033e40 2024-04-25T10:02:17.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7fd97f59e640 1 -- 172.21.15.133:0/3217083110 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97804a050 con 0x7fd978033e40 2024-04-25T10:02:17.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f8517fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f851815fe10 0x7f85180ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7fd96effd640 1 -- 172.21.15.133:0/3217083110 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9740072b0 con 0x7fd978033e40 2024-04-25T10:02:17.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7fd97f59e640 1 -- 172.21.15.133:0/3217083110 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9780053a0 con 0x7fd978033e40 2024-04-25T10:02:17.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f1e03fff640 1 -- 172.21.15.133:0/2440266642 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0129b0 con 0x7f1e10004b40 2024-04-25T10:02:17.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f96717fa640 1 -- 172.21.15.133:0/1984560354 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f966c01b020 con 0x7f96741615a0 2024-04-25T10:02:17.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f96717fa640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9658066f60 0x7f9658069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f96717fa640 1 -- 172.21.15.133:0/1984560354 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f966c086020 con 0x7f96741615a0 2024-04-25T10:02:17.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f96717fa640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x7f9658070650 0x7f9658072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f96717fa640 1 -- 172.21.15.133:0/1984560354 --> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f966c0020c0 con 0x7f9658070650 2024-04-25T10:02:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f9678d99640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9658066f60 0x7f9658069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f96717fa640 1 -- 172.21.15.133:0/1984560354 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f966c08d050 con 0x7f96741615a0 2024-04-25T10:02:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f9678d99640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9658066f60 0x7f9658069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f966001aae0 tx=0x7f9660008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85040051d0 0x7f8518143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f8516ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85040051d0 0x7f8518143850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55452/0 (socket says 172.21.15.133:55452) 2024-04-25T10:02:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f8516ffd640 1 -- 172.21.15.133:0/1821469699 learned_addr learned my addr 172.21.15.133:0/1821469699 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f8517fff640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85181619e0 0x7f85180ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180c9d30 con 0x7f851815fe10 2024-04-25T10:02:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f850ffff640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f851815fe10 0x7f85180ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180b02f0 con 0x7f85040051d0 2024-04-25T10:02:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85180c9eb0 con 0x7f85181619e0 2024-04-25T10:02:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f85177fe640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85181619e0 0x7f85180ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f778f7fe640 1 --2- 172.21.15.133:0/1169910659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77900b2310 0x7f7790071ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f77900712f0 tx=0x7f778401c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5b1ffc69b6c862 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f850ffff640 1 -- 172.21.15.133:0/1821469699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85181619e0 msgr2=0x7f85180ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f850ffff640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85181619e0 0x7f85180ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f850ffff640 1 -- 172.21.15.133:0/1821469699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85040051d0 msgr2=0x7f8518143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f850ffff640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85040051d0 0x7f8518143850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f850ffff640 1 -- 172.21.15.133:0/1821469699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85181602d0 con 0x7f851815fe10 2024-04-25T10:02:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7ffbc17fa640 1 -- 172.21.15.133:0/1100568899 <== osd.3 v2:172.21.15.133:6802/447485461 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffbc408ce50 con 0x7ffba8070730 2024-04-25T10:02:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f778dffb640 1 -- 172.21.15.133:0/1169910659 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778400d040 con 0x7f77900b2310 2024-04-25T10:02:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f778dffb640 1 -- 172.21.15.133:0/1169910659 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77840ab440 con 0x7f77900b2310 2024-04-25T10:02:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f778dffb640 1 -- 172.21.15.133:0/1169910659 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77840ab850 con 0x7f77900b2310 2024-04-25T10:02:17.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f850ffff640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f851815fe10 0x7f85180ce030 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f850000f520 tx=0x7f850000f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f1e03fff640 1 -- 172.21.15.133:0/2440266642 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e0c012b50 con 0x7f1e10004b40 2024-04-25T10:02:17.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f1e03fff640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e0806b800 0x7f1e0806dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7fd96effd640 1 -- 172.21.15.133:0/3217083110 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd97401a070 con 0x7fd978033e40 2024-04-25T10:02:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f778ffff640 1 -- 172.21.15.133:0/1169910659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77900b2310 msgr2=0x7f7790071ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f778ffff640 1 --2- 172.21.15.133:0/1169910659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77900b2310 0x7f7790071ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f77900712f0 tx=0x7f778401c6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f1e1e110640 1 -- 172.21.15.133:0/2440266642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e10005820 con 0x7f1e10004b40 2024-04-25T10:02:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f778ffff640 1 -- 172.21.15.133:0/1169910659 shutdown_connections 2024-04-25T10:02:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f778ffff640 1 --2- 172.21.15.133:0/1169910659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77900b2310 0x7f7790071ea0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f778ffff640 1 --2- 172.21.15.133:0/1169910659 >> v2:172.21.15.162:3300/0 conn(0x7f77900cd980 0x7f77900b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f778ffff640 1 --2- 172.21.15.133:0/1169910659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779015a960 0x7f779006f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f778ffff640 1 -- 172.21.15.133:0/1169910659 >> 172.21.15.133:0/1169910659 conn(0x7f77900ccdd0 msgr2=0x7f77900cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f850f7fe640 1 -- 172.21.15.133:0/1821469699 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85000053a0 con 0x7f851815fe10 2024-04-25T10:02:17.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778ffff640 1 -- 172.21.15.133:0/1169910659 shutdown_connections 2024-04-25T10:02:17.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778ffff640 1 -- 172.21.15.133:0/1169910659 wait complete. 2024-04-25T10:02:17.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778ffff640 1 Processor -- start 2024-04-25T10:02:17.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778ffff640 1 -- start start 2024-04-25T10:02:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f9673fff640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x7f9658070650 0x7f9658072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f850f7fe640 1 -- 172.21.15.133:0/1821469699 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8500018070 con 0x7f851815fe10 2024-04-25T10:02:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.680+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85181605b0 con 0x7f851815fe10 2024-04-25T10:02:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f850f7fe640 1 -- 172.21.15.133:0/1821469699 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500003820 con 0x7f851815fe10 2024-04-25T10:02:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8518160b70 con 0x7f851815fe10 2024-04-25T10:02:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8518074da0 con 0x7f851815fe10 2024-04-25T10:02:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7fd96effd640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd960066f60 0x7fd960069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7fd96effd640 1 -- 172.21.15.133:0/3217083110 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd974084b70 con 0x7fd978033e40 2024-04-25T10:02:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7fd96effd640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x7fd9600705e0 0x7fd960072a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7fd96effd640 1 -- 172.21.15.133:0/3217083110 --> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd974009ec0 con 0x7fd9600705e0 2024-04-25T10:02:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7fd96effd640 1 -- 172.21.15.133:0/3217083110 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd974085000 con 0x7fd978033e40 2024-04-25T10:02:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7fd97e59c640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd960066f60 0x7fd960069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7fd97ed9d640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x7fd9600705e0 0x7fd960072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7fd97e59c640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd960066f60 0x7fd960069420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9700b2000 tx=0x7fd9700aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7fd97ed9d640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x7fd9600705e0 0x7fd960072a20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f1e16ffd640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e0806b800 0x7f1e0806dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f1e03fff640 1 -- 172.21.15.133:0/2440266642 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1e0c085680 con 0x7f1e10004b40 2024-04-25T10:02:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f1e16ffd640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e0806b800 0x7f1e0806dcc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e040b2000 tx=0x7f1e04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778ffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77900b2310 0x7f7790202df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778ffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77900cd980 0x7f7790158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015a960 0x7f7790159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778ffff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779006f680 con 0x7f779015a960 2024-04-25T10:02:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778ffff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779006f500 con 0x7f77900b2310 2024-04-25T10:02:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778ffff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779006ac90 con 0x7f77900cd980 2024-04-25T10:02:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7fd96effd640 1 -- 172.21.15.133:0/3217083110 <== osd.6 v2:172.21.15.162:6824/2819373209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd974009ec0 con 0x7fd9600705e0 2024-04-25T10:02:17.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015a960 0x7f7790159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015a960 0x7f7790159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48436/0 (socket says 172.21.15.133:48436) 2024-04-25T10:02:17.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778f7fe640 1 -- 172.21.15.133:0/1608349028 learned_addr learned my addr 172.21.15.133:0/1608349028 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778effd640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77900b2310 0x7f7790202df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778e7fc640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77900cd980 0x7f7790158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778f7fe640 1 -- 172.21.15.133:0/1608349028 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77900cd980 msgr2=0x7f7790158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778f7fe640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f77900cd980 0x7f7790158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778f7fe640 1 -- 172.21.15.133:0/1608349028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77900b2310 msgr2=0x7f7790202df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778f7fe640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77900b2310 0x7f7790202df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778f7fe640 1 -- 172.21.15.133:0/1608349028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7790155270 con 0x7f779015a960 2024-04-25T10:02:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778effd640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77900b2310 0x7f7790202df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.684+0000 7f778f7fe640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015a960 0x7f7790159250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7784007610 tx=0x7f7784019440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f1e03fff640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x7f1e08074f80 0x7f1e080773c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f77777fe640 1 -- 172.21.15.133:0/1608349028 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778400d000 con 0x7f779015a960 2024-04-25T10:02:17.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f778ffff640 1 -- 172.21.15.133:0/1608349028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7790155550 con 0x7f779015a960 2024-04-25T10:02:17.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f1e17fff640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x7f1e08074f80 0x7f1e080773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f1e17fff640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x7f1e08074f80 0x7f1e080773c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f1e03fff640 1 -- 172.21.15.133:0/2440266642 --> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e0c019070 con 0x7f1e08074f80 2024-04-25T10:02:17.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f778ffff640 1 -- 172.21.15.133:0/1608349028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7790155b10 con 0x7f779015a960 2024-04-25T10:02:17.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f77777fe640 1 -- 172.21.15.133:0/1608349028 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77840b4410 con 0x7f779015a960 2024-04-25T10:02:17.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f77777fe640 1 -- 172.21.15.133:0/1608349028 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77840b47e0 con 0x7f779015a960 2024-04-25T10:02:17.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f1e03fff640 1 -- 172.21.15.133:0/2440266642 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e0c08c050 con 0x7f1e10004b40 2024-04-25T10:02:17.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f778ffff640 1 -- 172.21.15.133:0/1608349028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7770001040 con 0x7f779015a960 2024-04-25T10:02:17.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f850f7fe640 1 -- 172.21.15.133:0/1821469699 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85000039c0 con 0x7f851815fe10 2024-04-25T10:02:17.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f850f7fe640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f84f8066f90 0x7f84f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f850f7fe640 1 -- 172.21.15.133:0/1821469699 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8500085960 con 0x7f851815fe10 2024-04-25T10:02:17.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f8516ffd640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f84f8066f90 0x7f84f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f850f7fe640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x7f84f8070710 0x7f84f8072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f850f7fe640 1 -- 172.21.15.133:0/1821469699 --> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8500003e50 con 0x7f84f8070710 2024-04-25T10:02:17.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f850f7fe640 1 -- 172.21.15.133:0/1821469699 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8500085d50 con 0x7f851815fe10 2024-04-25T10:02:17.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f21337fe640 1 -- 172.21.15.133:0/3444021216 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f213c0c0020 con 0x7f2144007810 2024-04-25T10:02:17.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f9673fff640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x7f9658070650 0x7f9658072a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f8516ffd640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f84f8066f90 0x7f84f8069450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8508001840 tx=0x7f8508001120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f85177fe640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x7f84f8070710 0x7f84f8072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f85177fe640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x7f84f8070710 0x7f84f8072b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f21337fe640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2138066f90 0x7f2138069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f21337fe640 1 -- 172.21.15.133:0/3444021216 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f213c1203d0 con 0x7f2144007810 2024-04-25T10:02:17.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f214a7fc640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2138066f90 0x7f2138069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f21337fe640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] conn(0x7f2138070710 0x7f2138072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f21337fe640 1 -- 172.21.15.133:0/3444021216 --> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f213c0bbe50 con 0x7f2138070710 2024-04-25T10:02:17.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f1e03fff640 1 -- 172.21.15.133:0/2440266642 <== osd.0 v2:172.21.15.133:6816/136834454 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1e0c019070 con 0x7f1e08074f80 2024-04-25T10:02:17.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f214b7fe640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] conn(0x7f2138070710 0x7f2138072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f214a7fc640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2138066f90 0x7f2138069450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f21440da920 tx=0x7f2140006c50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f21337fe640 1 -- 172.21.15.133:0/3444021216 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f213c1207c0 con 0x7f2144007810 2024-04-25T10:02:17.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f214b7fe640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] conn(0x7f2138070710 0x7f2138072b50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.688+0000 7f77777fe640 1 -- 172.21.15.133:0/1608349028 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77840b2040 con 0x7f779015a960 2024-04-25T10:02:17.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.692+0000 7f96717fa640 1 -- 172.21.15.133:0/1984560354 <== osd.2 v2:172.21.15.133:6817/148430844 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f966c0020c0 con 0x7f9658070650 2024-04-25T10:02:17.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.692+0000 7f21337fe640 1 -- 172.21.15.133:0/3444021216 <== osd.7 v2:172.21.15.162:6800/1166832594 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f213c0bbe50 con 0x7f2138070710 2024-04-25T10:02:17.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.696+0000 7f850f7fe640 1 -- 172.21.15.133:0/1821469699 <== osd.4 v2:172.21.15.162:6801/3188774481 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8500003e50 con 0x7f84f8070710 2024-04-25T10:02:17.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.696+0000 7f77777fe640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f777c066f60 0x7f777c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.696+0000 7f77777fe640 1 -- 172.21.15.133:0/1608349028 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f778411ff10 con 0x7f779015a960 2024-04-25T10:02:17.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.696+0000 7f778effd640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f777c066f60 0x7f777c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.696+0000 7f77777fe640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] conn(0x7f777c0706e0 0x7f777c072b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.696+0000 7f77777fe640 1 -- 172.21.15.133:0/1608349028 --> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77840ace90 con 0x7f777c0706e0 2024-04-25T10:02:17.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.696+0000 7f77777fe640 1 -- 172.21.15.133:0/1608349028 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7784122570 con 0x7f779015a960 2024-04-25T10:02:17.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.696+0000 7f778e7fc640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] conn(0x7f777c0706e0 0x7f777c072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.696+0000 7f778e7fc640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] conn(0x7f777c0706e0 0x7f777c072b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.696+0000 7f778effd640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f777c066f60 0x7f777c069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7788009440 tx=0x7f778800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.700+0000 7f77777fe640 1 -- 172.21.15.133:0/1608349028 <== osd.1 v2:172.21.15.133:6800/2082178647 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f77840ace90 con 0x7f777c0706e0 2024-04-25T10:02:17.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.704+0000 7fd97f59e640 1 -- 172.21.15.133:0/3217083110 --> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd978005a70 con 0x7fd9600705e0 2024-04-25T10:02:17.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.704+0000 7fd96effd640 1 -- 172.21.15.133:0/3217083110 <== osd.6 v2:172.21.15.162:6824/2819373209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd978005a70 con 0x7fd9600705e0 2024-04-25T10:02:17.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.704+0000 7fd97f59e640 1 -- 172.21.15.133:0/3217083110 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x7fd9600705e0 msgr2=0x7fd960072a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.704+0000 7fd97f59e640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x7fd9600705e0 0x7fd960072a20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.708+0000 7fd97f59e640 1 -- 172.21.15.133:0/3217083110 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd960066f60 msgr2=0x7fd960069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.708+0000 7fd97f59e640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd960066f60 0x7fd960069420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9700b2000 tx=0x7fd9700aa070 comp rx=0 tx=0).stop 2024-04-25T10:02:17.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.708+0000 7fd97f59e640 1 -- 172.21.15.133:0/3217083110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 msgr2=0x7fd9780e7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.708+0000 7fd97f59e640 1 --2- 172.21.15.133:0/3217083110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd978033e40 0x7fd9780e7940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c460 tx=0x7fd97400c920 comp rx=0 tx=0).stop 2024-04-25T10:02:17.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.708+0000 7fd97ed9d640 1 -- 172.21.15.133:0/3217083110 reap_dead start 2024-04-25T10:02:17.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.708+0000 7fd97f59e640 1 -- 172.21.15.133:0/3217083110 shutdown_connections 2024-04-25T10:02:17.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.708+0000 7fd97f59e640 1 -- 172.21.15.133:0/3217083110 >> 172.21.15.133:0/3217083110 conn(0x7fd978009fc0 msgr2=0x7fd978053dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.708+0000 7fd97f59e640 1 -- 172.21.15.133:0/3217083110 shutdown_connections 2024-04-25T10:02:17.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.708+0000 7fd97f59e640 1 -- 172.21.15.133:0/3217083110 wait complete. 2024-04-25T10:02:17.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.712+0000 7ffbcb06b640 1 -- 172.21.15.133:0/1100568899 --> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffbbc0053a0 con 0x7ffba8070730 2024-04-25T10:02:17.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.712+0000 7ffbc17fa640 1 -- 172.21.15.133:0/1100568899 <== osd.3 v2:172.21.15.133:6802/447485461 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffbbc0053a0 con 0x7ffba8070730 2024-04-25T10:02:17.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.712+0000 7ffbaeffd640 1 -- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] conn(0x7ffba8070730 msgr2=0x7ffba8072b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.712+0000 7ffbaeffd640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] conn(0x7ffba8070730 0x7ffba8072b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.712+0000 7ffbaeffd640 1 -- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffba8067030 msgr2=0x7ffba80694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.712+0000 7ffbaeffd640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffba8067030 0x7ffba80694f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8000c10 tx=0x7ffbb800a040 comp rx=0 tx=0).stop 2024-04-25T10:02:17.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.716+0000 7ffbaeffd640 1 -- 172.21.15.133:0/1100568899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbbc033d00 msgr2=0x7ffbbc04d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.716+0000 7ffbaeffd640 1 --2- 172.21.15.133:0/1100568899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbbc033d00 0x7ffbbc04d700 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffbc404fd80 tx=0x7ffbc4068750 comp rx=0 tx=0).stop 2024-04-25T10:02:17.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.716+0000 7ffbc9608640 1 -- 172.21.15.133:0/1100568899 reap_dead start 2024-04-25T10:02:17.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.716+0000 7ffbaeffd640 1 -- 172.21.15.133:0/1100568899 shutdown_connections 2024-04-25T10:02:17.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.716+0000 7ffbaeffd640 1 -- 172.21.15.133:0/1100568899 >> 172.21.15.133:0/1100568899 conn(0x7ffbbc009fc0 msgr2=0x7ffbbc00a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.716+0000 7ffbaeffd640 1 -- 172.21.15.133:0/1100568899 shutdown_connections 2024-04-25T10:02:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.716+0000 7ffbaeffd640 1 -- 172.21.15.133:0/1100568899 wait complete. 2024-04-25T10:02:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.720+0000 7f214bfff640 1 -- 172.21.15.133:0/3444021216 --> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f21440057f0 con 0x7f2138070710 2024-04-25T10:02:17.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.720+0000 7fc198217640 1 -- 172.21.15.133:0/875313149 --> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc190074da0 con 0x7fc1740704c0 2024-04-25T10:02:17.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.720+0000 7f9679d9b640 1 -- 172.21.15.133:0/1984560354 --> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96740d0ef0 con 0x7f9658070650 2024-04-25T10:02:17.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.720+0000 7f96717fa640 1 -- 172.21.15.133:0/1984560354 <== osd.2 v2:172.21.15.133:6817/148430844 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f96740d0ef0 con 0x7f9658070650 2024-04-25T10:02:17.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.720+0000 7fc1867fc640 1 -- 172.21.15.133:0/875313149 <== osd.5 v2:172.21.15.162:6803/4006238640 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc190074da0 con 0x7fc1740704c0 2024-04-25T10:02:17.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.720+0000 7f21337fe640 1 -- 172.21.15.133:0/3444021216 <== osd.7 v2:172.21.15.162:6800/1166832594 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f21440057f0 con 0x7f2138070710 2024-04-25T10:02:17.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.720+0000 7f214bfff640 1 -- 172.21.15.133:0/3444021216 >> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] conn(0x7f2138070710 msgr2=0x7f2138072b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.720+0000 7f214bfff640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] conn(0x7f2138070710 0x7f2138072b50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.720+0000 7fc198217640 1 -- 172.21.15.133:0/875313149 >> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] conn(0x7fc1740704c0 msgr2=0x7fc174072900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.720+0000 7fc198217640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] conn(0x7fc1740704c0 0x7fc174072900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7fc198217640 1 -- 172.21.15.133:0/875313149 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc174066e40 msgr2=0x7fc174069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7fc198217640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc174066e40 0x7fc174069300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc1800017d0 tx=0x7fc180009040 comp rx=0 tx=0).stop 2024-04-25T10:02:17.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7fc198217640 1 -- 172.21.15.133:0/875313149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc190155bb0 msgr2=0x7fc1900ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7fc198217640 1 --2- 172.21.15.133:0/875313149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc190155bb0 0x7fc1900ad450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc1880ae000 tx=0x7fc188009750 comp rx=0 tx=0).stop 2024-04-25T10:02:17.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7fc1967b4640 1 -- 172.21.15.133:0/875313149 reap_dead start 2024-04-25T10:02:17.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7fc198217640 1 -- 172.21.15.133:0/875313149 shutdown_connections 2024-04-25T10:02:17.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7fc198217640 1 -- 172.21.15.133:0/875313149 >> 172.21.15.133:0/875313149 conn(0x7fc1900cbd10 msgr2=0x7fc19006e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e1e110640 1 -- 172.21.15.133:0/2440266642 --> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e1000af90 con 0x7f1e08074f80 2024-04-25T10:02:17.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7fc198217640 1 -- 172.21.15.133:0/875313149 shutdown_connections 2024-04-25T10:02:17.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7fc198217640 1 -- 172.21.15.133:0/875313149 wait complete. 2024-04-25T10:02:17.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e03fff640 1 -- 172.21.15.133:0/2440266642 <== osd.0 v2:172.21.15.133:6816/136834454 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1e1000af90 con 0x7f1e08074f80 2024-04-25T10:02:17.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e01ffb640 1 -- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x7f1e08074f80 msgr2=0x7f1e080773c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e01ffb640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x7f1e08074f80 0x7f1e080773c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e01ffb640 1 -- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e0806b800 msgr2=0x7f1e0806dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e01ffb640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e0806b800 0x7f1e0806dcc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e040b2000 tx=0x7f1e04008040 comp rx=0 tx=0).stop 2024-04-25T10:02:17.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e01ffb640 1 -- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e10004b40 msgr2=0x7f1e100d7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e01ffb640 1 --2- 172.21.15.133:0/2440266642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e10004b40 0x7f1e100d7070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00d5f0 tx=0x7f1e0c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e17fff640 1 -- 172.21.15.133:0/2440266642 reap_dead start 2024-04-25T10:02:17.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f214bfff640 1 -- 172.21.15.133:0/3444021216 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2138066f90 msgr2=0x7f2138069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f214bfff640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2138066f90 0x7f2138069450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f21440da920 tx=0x7f2140006c50 comp rx=0 tx=0).stop 2024-04-25T10:02:17.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f214bfff640 1 -- 172.21.15.133:0/3444021216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2144007810 msgr2=0x7f214404e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f214bfff640 1 --2- 172.21.15.133:0/3444021216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2144007810 0x7f214404e980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f213c01a0d0 tx=0x7f213c019240 comp rx=0 tx=0).stop 2024-04-25T10:02:17.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e01ffb640 1 -- 172.21.15.133:0/2440266642 shutdown_connections 2024-04-25T10:02:17.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e01ffb640 1 -- 172.21.15.133:0/2440266642 >> 172.21.15.133:0/2440266642 conn(0x7f1e1000a440 msgr2=0x7f1e100d6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f214b7fe640 1 -- 172.21.15.133:0/3444021216 reap_dead start 2024-04-25T10:02:17.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f214bfff640 1 -- 172.21.15.133:0/3444021216 shutdown_connections 2024-04-25T10:02:17.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f214bfff640 1 -- 172.21.15.133:0/3444021216 >> 172.21.15.133:0/3444021216 conn(0x7f21440559f0 msgr2=0x7f2144039d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e01ffb640 1 -- 172.21.15.133:0/2440266642 shutdown_connections 2024-04-25T10:02:17.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f1e01ffb640 1 -- 172.21.15.133:0/2440266642 wait complete. 2024-04-25T10:02:17.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f214bfff640 1 -- 172.21.15.133:0/3444021216 shutdown_connections 2024-04-25T10:02:17.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f214bfff640 1 -- 172.21.15.133:0/3444021216 wait complete. 2024-04-25T10:02:17.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f9679d9b640 1 -- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x7f9658070650 msgr2=0x7f9658072a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f9679d9b640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x7f9658070650 0x7f9658072a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 --> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f851814eed0 con 0x7f84f8070710 2024-04-25T10:02:17.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.724+0000 7f850f7fe640 1 -- 172.21.15.133:0/1821469699 <== osd.4 v2:172.21.15.162:6801/3188774481 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f851814eed0 con 0x7f84f8070710 2024-04-25T10:02:17.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.728+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x7f84f8070710 msgr2=0x7f84f8072b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.728+0000 7f8517fff640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x7f84f8070710 0x7f84f8072b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.728+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f84f8066f90 msgr2=0x7f84f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.728+0000 7f8517fff640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f84f8066f90 0x7f84f8069450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8508001840 tx=0x7f8508001120 comp rx=0 tx=0).stop 2024-04-25T10:02:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.728+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f851815fe10 msgr2=0x7f85180ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.728+0000 7f8517fff640 1 --2- 172.21.15.133:0/1821469699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f851815fe10 0x7f85180ce030 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f850000f520 tx=0x7f850000f9e0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.728+0000 7f85177fe640 1 -- 172.21.15.133:0/1821469699 reap_dead start 2024-04-25T10:02:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.728+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 shutdown_connections 2024-04-25T10:02:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.728+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 >> 172.21.15.133:0/1821469699 conn(0x7f851806ac90 msgr2=0x7f85180ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.728+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 shutdown_connections 2024-04-25T10:02:17.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.728+0000 7f8517fff640 1 -- 172.21.15.133:0/1821469699 wait complete. 2024-04-25T10:02:17.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.732+0000 7f778ffff640 1 -- 172.21.15.133:0/1608349028 --> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7770002de0 con 0x7f777c0706e0 2024-04-25T10:02:17.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.732+0000 7f77777fe640 1 -- 172.21.15.133:0/1608349028 <== osd.1 v2:172.21.15.133:6800/2082178647 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7770002de0 con 0x7f777c0706e0 2024-04-25T10:02:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.732+0000 7f778ffff640 1 -- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] conn(0x7f777c0706e0 msgr2=0x7f777c072b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.732+0000 7f778ffff640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] conn(0x7f777c0706e0 0x7f777c072b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.732+0000 7f778ffff640 1 -- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f777c066f60 msgr2=0x7f777c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.732+0000 7f778ffff640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f777c066f60 0x7f777c069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7788009440 tx=0x7f778800b040 comp rx=0 tx=0).stop 2024-04-25T10:02:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.732+0000 7f778ffff640 1 -- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015a960 msgr2=0x7f7790159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.732+0000 7f778ffff640 1 --2- 172.21.15.133:0/1608349028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779015a960 0x7f7790159250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7784007610 tx=0x7f7784019440 comp rx=0 tx=0).stop 2024-04-25T10:02:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.732+0000 7f778f7fe640 1 -- 172.21.15.133:0/1608349028 reap_dead start 2024-04-25T10:02:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.732+0000 7f778ffff640 1 -- 172.21.15.133:0/1608349028 shutdown_connections 2024-04-25T10:02:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.732+0000 7f778ffff640 1 -- 172.21.15.133:0/1608349028 >> 172.21.15.133:0/1608349028 conn(0x7f77900ccdd0 msgr2=0x7f77900b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.736+0000 7f778ffff640 1 -- 172.21.15.133:0/1608349028 shutdown_connections 2024-04-25T10:02:17.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.736+0000 7f778ffff640 1 -- 172.21.15.133:0/1608349028 wait complete. 2024-04-25T10:02:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.736+0000 7f9679d9b640 1 -- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9658066f60 msgr2=0x7f9658069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.736+0000 7f9679d9b640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9658066f60 0x7f9658069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f966001aae0 tx=0x7f9660008040 comp rx=0 tx=0).stop 2024-04-25T10:02:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.736+0000 7f9679d9b640 1 -- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96741615a0 msgr2=0x7f967415f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.736+0000 7f9679d9b640 1 --2- 172.21.15.133:0/1984560354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96741615a0 0x7f967415f110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f966c00ebd0 tx=0x7f966c00c490 comp rx=0 tx=0).stop 2024-04-25T10:02:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.736+0000 7f967959a640 1 -- 172.21.15.133:0/1984560354 reap_dead start 2024-04-25T10:02:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.736+0000 7f9679d9b640 1 -- 172.21.15.133:0/1984560354 shutdown_connections 2024-04-25T10:02:17.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.736+0000 7f9679d9b640 1 -- 172.21.15.133:0/1984560354 >> 172.21.15.133:0/1984560354 conn(0x7f967409c9c0 msgr2=0x7f96740b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.772 INFO:teuthology.orchestra.run.smithi133.stdout:55834574860 2024-04-25T10:02:17.772 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:02:17.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.740+0000 7f9679d9b640 1 -- 172.21.15.133:0/1984560354 shutdown_connections 2024-04-25T10:02:17.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.740+0000 7f9679d9b640 1 -- 172.21.15.133:0/1984560354 wait complete. 2024-04-25T10:02:17.773 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:17.774 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:02:17.775 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:17.775 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:02:17.776 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:17.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:02:17.777 INFO:teuthology.orchestra.run.smithi133.stdout:55834574860 2024-04-25T10:02:17.777 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:02:17.778 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:17.778 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:02:17.779 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:17.779 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:02:17.780 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:17.781 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:02:17.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.964+0000 7fd5147a4640 1 Processor -- start 2024-04-25T10:02:17.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.964+0000 7fd5147a4640 1 -- start start 2024-04-25T10:02:17.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.964+0000 7fd5147a4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd50c0b36c0 0x7fd50c08a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.964+0000 7fd5147a4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd50c0b1720 0x7fd50c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.964+0000 7fd512540640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd50c0b36c0 0x7fd50c08a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.964+0000 7fd512540640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd50c0b36c0 0x7fd50c08a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48440/0 (socket says 172.21.15.133:48440) 2024-04-25T10:02:17.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.964+0000 7fd512540640 1 -- 172.21.15.133:0/3285391047 learned_addr learned my addr 172.21.15.133:0/3285391047 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.964+0000 7fd5147a4640 1 -- 172.21.15.133:0/3285391047 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c0b2180 con 0x7fd50c0b2bf0 2024-04-25T10:02:17.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.964+0000 7fd5147a4640 1 -- 172.21.15.133:0/3285391047 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c0b3fe0 con 0x7fd50c0b36c0 2024-04-25T10:02:17.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.964+0000 7fd5147a4640 1 -- 172.21.15.133:0/3285391047 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c0cb780 con 0x7fd50c0b1720 2024-04-25T10:02:17.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.968+0000 7fd511d3f640 1 --2- 172.21.15.133:0/3285391047 >> v2:172.21.15.162:3300/0 conn(0x7fd50c0b1720 0x7fd50c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.968+0000 7fd512540640 1 -- 172.21.15.133:0/3285391047 >> v2:172.21.15.162:3300/0 conn(0x7fd50c0b1720 msgr2=0x7fd50c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.968+0000 7fd512540640 1 --2- 172.21.15.133:0/3285391047 >> v2:172.21.15.162:3300/0 conn(0x7fd50c0b1720 0x7fd50c0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.968+0000 7fd512540640 1 -- 172.21.15.133:0/3285391047 >> v1:172.21.15.162:6789/0 conn(0x7fd50c0b2bf0 legacy=0x7fd50c0c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.968+0000 7fd512540640 1 -- 172.21.15.133:0/3285391047 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0b23f0 con 0x7fd50c0b36c0 2024-04-25T10:02:17.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.968+0000 7fd512540640 1 --2- 172.21.15.133:0/3285391047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0b36c0 0x7fd50c08a310 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0d02d0 tx=0x7fd5040252a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2a918322c4b808 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.968+0000 7fd51153e640 1 -- 172.21.15.133:0/3285391047 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5040b9070 con 0x7fd50c0b36c0 2024-04-25T10:02:17.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.968+0000 7fd51153e640 1 -- 172.21.15.133:0/3285391047 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5040b4600 con 0x7fd50c0b36c0 2024-04-25T10:02:17.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.968+0000 7fd51153e640 1 -- 172.21.15.133:0/3285391047 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5040b49d0 con 0x7fd50c0b36c0 2024-04-25T10:02:17.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.972+0000 7fd5147a4640 1 -- 172.21.15.133:0/3285391047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0b36c0 msgr2=0x7fd50c08a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.972+0000 7fd5147a4640 1 --2- 172.21.15.133:0/3285391047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0b36c0 0x7fd50c08a310 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0d02d0 tx=0x7fd5040252a0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.972+0000 7fd5147a4640 1 -- 172.21.15.133:0/3285391047 shutdown_connections 2024-04-25T10:02:17.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.972+0000 7fd5147a4640 1 --2- 172.21.15.133:0/3285391047 >> v2:172.21.15.162:3300/0 conn(0x7fd50c0b1720 0x7fd50c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.972+0000 7fd5147a4640 1 --2- 172.21.15.133:0/3285391047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0b36c0 0x7fd50c08a310 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.972+0000 7fd5147a4640 1 -- 172.21.15.133:0/3285391047 >> 172.21.15.133:0/3285391047 conn(0x7fd50c0cbd10 msgr2=0x7fd50c0cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.972+0000 7fd5147a4640 1 -- 172.21.15.133:0/3285391047 shutdown_connections 2024-04-25T10:02:17.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.972+0000 7fd5147a4640 1 -- 172.21.15.133:0/3285391047 wait complete. 2024-04-25T10:02:17.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.972+0000 7fd5147a4640 1 Processor -- start 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.972+0000 7fd5147a4640 1 -- start start 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd5147a4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0b1720 0x7fd50c0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd5147a4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd50c0b2bf0 0x7fd50c172770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd5147a4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd50c0b36c0 0x7fd50c0c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd5147a4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0cb520 con 0x7fd50c0b1720 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd5147a4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0cb3a0 con 0x7fd50c0b2bf0 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd5147a4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0c9ff0 con 0x7fd50c0b36c0 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd512540640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0b1720 0x7fd50c0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd512540640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0b1720 0x7fd50c0c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48452/0 (socket says 172.21.15.133:48452) 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd512540640 1 -- 172.21.15.133:0/2825820693 learned_addr learned my addr 172.21.15.133:0/2825820693 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd511d3f640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd50c0b2bf0 0x7fd50c172770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd512d41640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd50c0b36c0 0x7fd50c0c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd512540640 1 -- 172.21.15.133:0/2825820693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd50c0b36c0 msgr2=0x7fd50c0c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd512540640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd50c0b36c0 0x7fd50c0c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd512540640 1 -- 172.21.15.133:0/2825820693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd50c0b2bf0 msgr2=0x7fd50c172770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd512540640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd50c0b2bf0 0x7fd50c172770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd512540640 1 -- 172.21.15.133:0/2825820693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0afde0 con 0x7fd50c0b1720 2024-04-25T10:02:17.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd512540640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0b1720 0x7fd50c0c6360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd504021eb0 tx=0x7fd504022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd502ffd640 1 -- 172.21.15.133:0/2825820693 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5040b9030 con 0x7fd50c0b1720 2024-04-25T10:02:17.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd502ffd640 1 -- 172.21.15.133:0/2825820693 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5040193c0 con 0x7fd50c0b1720 2024-04-25T10:02:17.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd502ffd640 1 -- 172.21.15.133:0/2825820693 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504019590 con 0x7fd50c0b1720 2024-04-25T10:02:17.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd5147a4640 1 -- 172.21.15.133:0/2825820693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c0b00c0 con 0x7fd50c0b1720 2024-04-25T10:02:17.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd5147a4640 1 -- 172.21.15.133:0/2825820693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c0c69f0 con 0x7fd50c0b1720 2024-04-25T10:02:17.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.976+0000 7fd5147a4640 1 -- 172.21.15.133:0/2825820693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50c074da0 con 0x7fd50c0b1720 2024-04-25T10:02:17.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.980+0000 7fd502ffd640 1 -- 172.21.15.133:0/2825820693 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5040bc070 con 0x7fd50c0b1720 2024-04-25T10:02:17.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.980+0000 7fd502ffd640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4f0066f90 0x7fd4f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.980+0000 7fd502ffd640 1 -- 172.21.15.133:0/2825820693 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd50411f0d0 con 0x7fd50c0b1720 2024-04-25T10:02:17.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.980+0000 7fd511d3f640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4f0066f90 0x7fd4f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.980+0000 7fd511d3f640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4f0066f90 0x7fd4f0069450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd50c207140 tx=0x7fd508008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.984+0000 7fd502ffd640 1 -- 172.21.15.133:0/2825820693 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd50401ef50 con 0x7fd50c0b1720 2024-04-25T10:02:17.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.984+0000 7f4b5d84b640 1 Processor -- start 2024-04-25T10:02:17.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.984+0000 7f4b5d84b640 1 -- start start 2024-04-25T10:02:17.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.984+0000 7f4b5d84b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4b58159d50 0x7f4b58070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.984+0000 7f4b5d84b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5815a130 con 0x7f4b5815df50 2024-04-25T10:02:17.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.984+0000 7f4b5d84b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b580c9ef0 con 0x7f4b5806cca0 2024-04-25T10:02:17.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.984+0000 7f4b5d84b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b58071c30 con 0x7f4b58159d50 2024-04-25T10:02:17.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.984+0000 7f4b5d04a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4b5815df50 0x7f4b580cacb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51958/0 (socket says 172.21.15.133:51958) 2024-04-25T10:02:17.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.984+0000 7f4b5d04a640 1 -- 172.21.15.133:0/1691704369 learned_addr learned my addr 172.21.15.133:0/1691704369 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:17.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.984+0000 7f4b5c849640 1 --2- 172.21.15.133:0/1691704369 >> v2:172.21.15.162:3300/0 conn(0x7f4b58159d50 0x7f4b58070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732790358 0 0) 0x7f4b5815a130 con 0x7f4b5815df50 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b48003530 con 0x7f4b5815df50 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1706883318 0 0) 0x7f4b48003530 con 0x7f4b5815df50 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 >> v2:172.21.15.162:3300/0 conn(0x7f4b58159d50 msgr2=0x7f4b58070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 --2- 172.21.15.133:0/1691704369 >> v2:172.21.15.162:3300/0 conn(0x7f4b58159d50 0x7f4b58070ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 >> v1:172.21.15.162:6789/0 conn(0x7f4b5806cca0 legacy=0x7f4b5815d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b580ca1d0 con 0x7f4b5815df50 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4b50001a00 con 0x7f4b5815df50 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 >> v1:172.21.15.133:6789/0 conn(0x7f4b5815df50 legacy=0x7f4b580cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 --2- 172.21.15.133:0/1691704369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b48003a10 0x7f4b48003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 --2- 172.21.15.133:0/1691704369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b48004810 0x7f4b48004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 --2- 172.21.15.133:0/1691704369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b48005220 0x7f4b480079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b58071c30 con 0x7f4b48003a10 2024-04-25T10:02:17.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b48008150 con 0x7f4b48005220 2024-04-25T10:02:17.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b48008350 con 0x7f4b48004810 2024-04-25T10:02:17.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b5c849640 1 --2- 172.21.15.133:0/1691704369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b48003a10 0x7f4b48003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b5d04a640 1 --2- 172.21.15.133:0/1691704369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b48005220 0x7f4b480079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b57fff640 1 --2- 172.21.15.133:0/1691704369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b48004810 0x7f4b48004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b5c849640 1 -- 172.21.15.133:0/1691704369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b48004810 msgr2=0x7f4b48004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b5c849640 1 --2- 172.21.15.133:0/1691704369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b48004810 0x7f4b48004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b5c849640 1 -- 172.21.15.133:0/1691704369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b48005220 msgr2=0x7f4b480079b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b5c849640 1 --2- 172.21.15.133:0/1691704369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b48005220 0x7f4b480079b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b5d04a640 1 -- 172.21.15.133:0/1691704369 reap_dead start 2024-04-25T10:02:17.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7f4b5c849640 1 -- 172.21.15.133:0/1691704369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b48003530 con 0x7f4b48003a10 2024-04-25T10:02:17.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7fd9e886a640 1 Processor -- start 2024-04-25T10:02:17.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7fd9e886a640 1 -- start start 2024-04-25T10:02:17.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7fd9e886a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd9e40afa40 0x7fd9e40ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7fd9e886a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e40afe20 0x7fd9e40b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7fd9e886a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e40af660 con 0x7fd9e40b1110 2024-04-25T10:02:17.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7fd9e886a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e40cdad0 con 0x7fd9e40afa40 2024-04-25T10:02:17.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.988+0000 7fd9e886a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e40cdd70 con 0x7fd9e40afe20 2024-04-25T10:02:17.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fd9e259c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd9e40afa40 0x7fd9e40ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15761a9640 1 Processor -- start 2024-04-25T10:02:17.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fd9e259c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd9e40afa40 0x7fd9e40ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55496/0 (socket says 172.21.15.133:55496) 2024-04-25T10:02:17.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fd9e259c640 1 -- 172.21.15.133:0/755104936 learned_addr learned my addr 172.21.15.133:0/755104936 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:17.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fd9e2d9d640 1 --2- 172.21.15.133:0/755104936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e40afe20 0x7fd9e40b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f4b5c849640 1 --2- 172.21.15.133:0/1691704369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b48003a10 0x7f4b48003e30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009ab0 tx=0x7f4b4c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17e43f32b48649e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae30f93640 1 Processor -- start 2024-04-25T10:02:17.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fd9e259c640 1 -- 172.21.15.133:0/755104936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e40afe20 msgr2=0x7fd9e40b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fd9e259c640 1 --2- 172.21.15.133:0/755104936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e40afe20 0x7fd9e40b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fd9e259c640 1 -- 172.21.15.133:0/755104936 >> v1:172.21.15.162:6789/0 conn(0x7fd9e40b1110 legacy=0x7fd9e40cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fd9e259c640 1 -- 172.21.15.133:0/755104936 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e40ce050 con 0x7fd9e40afa40 2024-04-25T10:02:17.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15761a9640 1 -- start start 2024-04-25T10:02:18.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0096b0 con 0x7f4b48003a10 2024-04-25T10:02:18.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c0a6410 con 0x7f4b48003a10 2024-04-25T10:02:18.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f4b577fe640 1 -- 172.21.15.133:0/1691704369 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0096b0 con 0x7f4b48003a10 2024-04-25T10:02:18.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae30f93640 1 -- start start 2024-04-25T10:02:18.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15761a9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f15700725e0 0x7f15700ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15761a9640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15700cadf0 con 0x7f1570162fd0 2024-04-25T10:02:18.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15761a9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1570162530 con 0x7f15700ae6a0 2024-04-25T10:02:18.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15761a9640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15700cb090 con 0x7f15700725e0 2024-04-25T10:02:18.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15759a8640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f15700ae6a0 0x7f15700c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:48074/0 (socket says 172.21.15.133:48074) 2024-04-25T10:02:18.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15759a8640 1 -- 172.21.15.133:0/3001667139 learned_addr learned my addr 172.21.15.133:0/3001667139 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:18.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15751a7640 1 --2- 172.21.15.133:0/3001667139 >> v2:172.21.15.133:3300/0 conn(0x7f15700725e0 0x7f15700ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f4b5d84b640 1 -- 172.21.15.133:0/1691704369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b48003a10 msgr2=0x7f4b48003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f4b5d84b640 1 --2- 172.21.15.133:0/1691704369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b48003a10 0x7f4b48003e30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009ab0 tx=0x7f4b4c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:02:18.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f4b5d84b640 1 -- 172.21.15.133:0/1691704369 shutdown_connections 2024-04-25T10:02:18.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f4b5d84b640 1 --2- 172.21.15.133:0/1691704369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b48003a10 0x7f4b48003e30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f4b5d84b640 1 -- 172.21.15.133:0/1691704369 >> 172.21.15.133:0/1691704369 conn(0x7f4b580cbcd0 msgr2=0x7f4b580cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae30f93640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae2c15dab0 0x7fae2c0b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae30f93640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c0b72d0 con 0x7fae2c0b9590 2024-04-25T10:02:18.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae30f93640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c0b75b0 con 0x7fae2c15d6d0 2024-04-25T10:02:18.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae30f93640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0b7890 con 0x7fae2c15dab0 2024-04-25T10:02:18.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15751a7640 1 -- 172.21.15.133:0/3001667139 >> v1:172.21.15.162:6789/0 conn(0x7f15700ae6a0 legacy=0x7f15700c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15751a7640 1 -- 172.21.15.133:0/3001667139 >> v1:172.21.15.133:6789/0 conn(0x7f1570162fd0 legacy=0x7f15700729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f1567fff640 1 -- 172.21.15.133:0/3001667139 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624007198 0 0) 0x7f1570162530 con 0x7f15700ae6a0 2024-04-25T10:02:18.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f15751a7640 1 -- 172.21.15.133:0/3001667139 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15701627d0 con 0x7f15700725e0 2024-04-25T10:02:18.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fef196d6640 1 Processor -- start 2024-04-25T10:02:18.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae29d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fae2c0b9590 0x7fae2c0d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51980/0 (socket says 172.21.15.133:51980) 2024-04-25T10:02:18.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae29d9b640 1 -- 172.21.15.133:0/1712741095 learned_addr learned my addr 172.21.15.133:0/1712741095 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:18.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f4b5d84b640 1 -- 172.21.15.133:0/1691704369 shutdown_connections 2024-04-25T10:02:18.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae2ad9d640 1 --2- 172.21.15.133:0/1712741095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae2c15dab0 0x7fae2c0b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae2959a640 1 -- 172.21.15.133:0/1712741095 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760893495 0 0) 0x7fae2c0b72d0 con 0x7fae2c0b9590 2024-04-25T10:02:18.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae2959a640 1 -- 172.21.15.133:0/1712741095 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae14003530 con 0x7fae2c0b9590 2024-04-25T10:02:18.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fef196d6640 1 -- start start 2024-04-25T10:02:18.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7f4b5d84b640 1 -- 172.21.15.133:0/1691704369 wait complete. 2024-04-25T10:02:18.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fef196d6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef14161e10 0x7fef1409cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fef196d6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fef140b20e0 0x7fef140b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fef196d6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fef140ce7d0 0x7fef140cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fef196d6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef140ce340 con 0x7fef140b20e0 2024-04-25T10:02:18.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fef196d6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1406e2b0 con 0x7fef140ce7d0 2024-04-25T10:02:18.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fef196d6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef140cdcf0 con 0x7fef14161e10 2024-04-25T10:02:18.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fef0bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fef140b20e0 0x7fef140b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fef0bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fef140b20e0 0x7fef140b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48476/0 (socket says 172.21.15.133:48476) 2024-04-25T10:02:18.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fef0bfff640 1 -- 172.21.15.133:0/1673893041 learned_addr learned my addr 172.21.15.133:0/1673893041 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:18.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae2ad9d640 1 -- 172.21.15.133:0/1712741095 >> v1:172.21.15.162:6789/0 conn(0x7fae2c15d6d0 legacy=0x7fae2c0b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae2ad9d640 1 -- 172.21.15.133:0/1712741095 >> v1:172.21.15.133:6789/0 conn(0x7fae2c0b9590 legacy=0x7fae2c0d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.992+0000 7fae2ad9d640 1 -- 172.21.15.133:0/1712741095 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c0b7b70 con 0x7fae2c15dab0 2024-04-25T10:02:18.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f85d9943640 1 Processor -- start 2024-04-25T10:02:18.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e259c640 1 --2- 172.21.15.133:0/755104936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e40afa40 0x7fd9e40ceb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009870 tx=0x7fd9d801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52bb3b65b985b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f85d9943640 1 -- start start 2024-04-25T10:02:18.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fef0bfff640 1 -- 172.21.15.133:0/1673893041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef14161e10 msgr2=0x7fef1409cc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:02:18.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fef0bfff640 1 --2- 172.21.15.133:0/1673893041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef14161e10 0x7fef1409cc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fef0bfff640 1 -- 172.21.15.133:0/1673893041 >> v2:172.21.15.162:3300/0 conn(0x7fef140ce7d0 msgr2=0x7fef140cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fef137fe640 1 --2- 172.21.15.133:0/1673893041 >> v2:172.21.15.162:3300/0 conn(0x7fef140ce7d0 0x7fef140cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f15751a7640 1 --2- 172.21.15.133:0/3001667139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15700725e0 0x7f15700ced10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f15600097c0 tx=0x7f156001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b081e8eaffc0572c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e159a640 1 -- 172.21.15.133:0/755104936 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80ab650 con 0x7fd9e40afa40 2024-04-25T10:02:18.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e159a640 1 -- 172.21.15.133:0/755104936 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d80b0070 con 0x7fd9e40afa40 2024-04-25T10:02:18.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e159a640 1 -- 172.21.15.133:0/755104936 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80aba10 con 0x7fd9e40afa40 2024-04-25T10:02:18.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e886a640 1 -- 172.21.15.133:0/755104936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e40afa40 msgr2=0x7fd9e40ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e886a640 1 --2- 172.21.15.133:0/755104936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e40afa40 0x7fd9e40ceb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009870 tx=0x7fd9d801c560 comp rx=0 tx=0).stop 2024-04-25T10:02:18.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e886a640 1 -- 172.21.15.133:0/755104936 shutdown_connections 2024-04-25T10:02:18.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e886a640 1 --2- 172.21.15.133:0/755104936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e40afe20 0x7fd9e40b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e886a640 1 --2- 172.21.15.133:0/755104936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e40afa40 0x7fd9e40ceb30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e886a640 1 -- 172.21.15.133:0/755104936 >> 172.21.15.133:0/755104936 conn(0x7fd9e4066b70 msgr2=0x7fd9e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5d84b640 1 Processor -- start 2024-04-25T10:02:18.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5d84b640 1 -- start start 2024-04-25T10:02:18.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e886a640 1 -- 172.21.15.133:0/755104936 shutdown_connections 2024-04-25T10:02:18.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e886a640 1 -- 172.21.15.133:0/755104936 wait complete. 2024-04-25T10:02:18.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fae2ad9d640 1 --2- 172.21.15.133:0/1712741095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae2c15dab0 0x7fae2c0b8370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae24009dc0 tx=0x7fae2401f160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e5d749d59b217e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5d84b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b580c6f20 0x7f4b58156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5d84b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b580c53e0 0x7f4b58157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5d84b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b580c7300 0x7f4b582032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5d84b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5815dbd0 con 0x7f4b580c7300 2024-04-25T10:02:18.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5d84b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5815da50 con 0x7f4b580c6f20 2024-04-25T10:02:18.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5d84b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5815dd50 con 0x7f4b580c53e0 2024-04-25T10:02:18.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5c849640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b580c6f20 0x7f4b58156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5c849640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b580c6f20 0x7f4b58156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55518/0 (socket says 172.21.15.133:55518) 2024-04-25T10:02:18.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5c849640 1 -- 172.21.15.133:0/2236357397 learned_addr learned my addr 172.21.15.133:0/2236357397 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:18.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b57fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b580c53e0 0x7f4b58157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fae2959a640 1 -- 172.21.15.133:0/1712741095 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae240b4070 con 0x7fae2c15dab0 2024-04-25T10:02:18.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fae2959a640 1 -- 172.21.15.133:0/1712741095 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae240af600 con 0x7fae2c15dab0 2024-04-25T10:02:18.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fae2959a640 1 -- 172.21.15.133:0/1712741095 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae240afa10 con 0x7fae2c15dab0 2024-04-25T10:02:18.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e886a640 1 Processor -- start 2024-04-25T10:02:18.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fef0bfff640 1 --2- 172.21.15.133:0/1673893041 >> v2:172.21.15.162:3300/0 conn(0x7fef140ce7d0 0x7fef140cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fef0bfff640 1 -- 172.21.15.133:0/1673893041 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef140cdfd0 con 0x7fef140b20e0 2024-04-25T10:02:18.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5c849640 1 -- 172.21.15.133:0/2236357397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b580c53e0 msgr2=0x7f4b58157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5c849640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b580c53e0 0x7f4b58157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5c849640 1 -- 172.21.15.133:0/2236357397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b580c7300 msgr2=0x7f4b582032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5c849640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b580c7300 0x7f4b582032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5c849640 1 -- 172.21.15.133:0/2236357397 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b58156f40 con 0x7f4b580c6f20 2024-04-25T10:02:18.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b57fff640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b580c53e0 0x7f4b58157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:18.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f4b5c849640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b580c6f20 0x7f4b58156960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c01cf10 tx=0x7f4b4c0197d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f1567fff640 1 -- 172.21.15.133:0/3001667139 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15600b0070 con 0x7f15700725e0 2024-04-25T10:02:18.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f1567fff640 1 -- 172.21.15.133:0/3001667139 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15600ab600 con 0x7f15700725e0 2024-04-25T10:02:18.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f1567fff640 1 -- 172.21.15.133:0/3001667139 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15600ab9d0 con 0x7f15700725e0 2024-04-25T10:02:18.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae30f93640 1 -- 172.21.15.133:0/1712741095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae2c15dab0 msgr2=0x7fae2c0b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae30f93640 1 --2- 172.21.15.133:0/1712741095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae2c15dab0 0x7fae2c0b8370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae24009dc0 tx=0x7fae2401f160 comp rx=0 tx=0).stop 2024-04-25T10:02:18.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae30f93640 1 -- 172.21.15.133:0/1712741095 shutdown_connections 2024-04-25T10:02:18.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae30f93640 1 --2- 172.21.15.133:0/1712741095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae2c15dab0 0x7fae2c0b8370 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae30f93640 1 -- 172.21.15.133:0/1712741095 >> 172.21.15.133:0/1712741095 conn(0x7fae2c0727c0 msgr2=0x7fae2c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae30f93640 1 -- 172.21.15.133:0/1712741095 shutdown_connections 2024-04-25T10:02:18.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fef0bfff640 1 --2- 172.21.15.133:0/1673893041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef140b20e0 0x7fef140b2540 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef141668a0 tx=0x7fef0401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7f241b583cfcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7fd9e886a640 1 -- start start 2024-04-25T10:02:18.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e886a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e40afa40 0x7fd9e414a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e886a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9e40afe20 0x7fd9e41484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e886a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e40b1110 0x7fd9e414c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e886a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e406dab0 con 0x7fd9e40afe20 2024-04-25T10:02:18.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e886a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e4159d70 con 0x7fd9e40afa40 2024-04-25T10:02:18.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e886a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e406dc30 con 0x7fd9e40b1110 2024-04-25T10:02:18.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f85d9943640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f85cc053ec0 0x7f85cc03aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f85d9943640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc007140 0x7f85cc03af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f85d9943640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85cc054430 con 0x7f85cc006d60 2024-04-25T10:02:18.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f85d9943640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85cc055040 con 0x7f85cc053ec0 2024-04-25T10:02:18.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:17.996+0000 7f85d9943640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc054710 con 0x7f85cc007140 2024-04-25T10:02:18.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f85d8941640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f85cc006d60 0x7f85cc0537b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51994/0 (socket says 172.21.15.133:51994) 2024-04-25T10:02:18.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f85d8941640 1 -- 172.21.15.133:0/2193118903 learned_addr learned my addr 172.21.15.133:0/2193118903 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:18.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f85d9142640 1 --2- 172.21.15.133:0/2193118903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc007140 0x7f85cc03af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f85d3fff640 1 --2- 172.21.15.133:0/2193118903 >> v2:172.21.15.162:3300/0 conn(0x7f85cc053ec0 0x7f85cc03aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f85d37fe640 1 -- 172.21.15.133:0/2193118903 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524058150 0 0) 0x7f85cc054430 con 0x7f85cc006d60 2024-04-25T10:02:18.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f85d37fe640 1 -- 172.21.15.133:0/2193118903 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85c4003530 con 0x7f85cc006d60 2024-04-25T10:02:18.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f85d9142640 1 -- 172.21.15.133:0/2193118903 >> v2:172.21.15.162:3300/0 conn(0x7f85cc053ec0 msgr2=0x7f85cc03aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f85d9142640 1 --2- 172.21.15.133:0/2193118903 >> v2:172.21.15.162:3300/0 conn(0x7f85cc053ec0 0x7f85cc03aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f85d9142640 1 -- 172.21.15.133:0/2193118903 >> v1:172.21.15.133:6789/0 conn(0x7f85cc006d60 legacy=0x7f85cc0537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f85d9142640 1 -- 172.21.15.133:0/2193118903 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc0549f0 con 0x7f85cc007140 2024-04-25T10:02:18.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f15761a9640 1 -- 172.21.15.133:0/3001667139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15700725e0 msgr2=0x7f15700ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f15761a9640 1 --2- 172.21.15.133:0/3001667139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15700725e0 0x7f15700ced10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f15600097c0 tx=0x7f156001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f15761a9640 1 -- 172.21.15.133:0/3001667139 shutdown_connections 2024-04-25T10:02:18.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f15761a9640 1 --2- 172.21.15.133:0/3001667139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15700725e0 0x7f15700ced10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f15761a9640 1 -- 172.21.15.133:0/3001667139 >> 172.21.15.133:0/3001667139 conn(0x7f15700cbcd0 msgr2=0x7f15700cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f15761a9640 1 -- 172.21.15.133:0/3001667139 shutdown_connections 2024-04-25T10:02:18.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9e40afe20 0x7fd9e41484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9e40afe20 0x7fd9e41484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48488/0 (socket says 172.21.15.133:48488) 2024-04-25T10:02:18.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e1d9b640 1 -- 172.21.15.133:0/4082982160 learned_addr learned my addr 172.21.15.133:0/4082982160 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:18.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e2d9d640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e40b1110 0x7fd9e414c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e1d9b640 1 -- 172.21.15.133:0/4082982160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e40b1110 msgr2=0x7fd9e414c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e1d9b640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e40b1110 0x7fd9e414c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e1d9b640 1 -- 172.21.15.133:0/4082982160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e40afa40 msgr2=0x7fd9e414a440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:02:18.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e1d9b640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e40afa40 0x7fd9e414a440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e1d9b640 1 -- 172.21.15.133:0/4082982160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e4148a20 con 0x7fd9e40afe20 2024-04-25T10:02:18.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e1d9b640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9e40afe20 0x7fd9e41484e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00c760 tx=0x7fd9dc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fef127fc640 1 -- 172.21.15.133:0/1673893041 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040a56c0 con 0x7fef140b20e0 2024-04-25T10:02:18.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fef127fc640 1 -- 172.21.15.133:0/1673893041 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef040a5860 con 0x7fef140b20e0 2024-04-25T10:02:18.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fef127fc640 1 -- 172.21.15.133:0/1673893041 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040a5c30 con 0x7fef140b20e0 2024-04-25T10:02:18.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae30f93640 1 -- 172.21.15.133:0/1712741095 wait complete. 2024-04-25T10:02:18.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae30f93640 1 Processor -- start 2024-04-25T10:02:18.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9d2ffd640 1 -- 172.21.15.133:0/4082982160 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc016020 con 0x7fd9e40afe20 2024-04-25T10:02:18.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9d2ffd640 1 -- 172.21.15.133:0/4082982160 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc006d60 con 0x7fd9e40afe20 2024-04-25T10:02:18.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9d2ffd640 1 -- 172.21.15.133:0/4082982160 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc003bc0 con 0x7fd9e40afe20 2024-04-25T10:02:18.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e886a640 1 -- 172.21.15.133:0/4082982160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e414aa70 con 0x7fd9e40afe20 2024-04-25T10:02:18.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9e886a640 1 -- 172.21.15.133:0/4082982160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e41576e0 con 0x7fd9e40afe20 2024-04-25T10:02:18.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f15761a9640 1 -- 172.21.15.133:0/3001667139 wait complete. 2024-04-25T10:02:18.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f15761a9640 1 Processor -- start 2024-04-25T10:02:18.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f4b557fa640 1 -- 172.21.15.133:0/2236357397 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0096b0 con 0x7f4b580c6f20 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f4b557fa640 1 -- 172.21.15.133:0/2236357397 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c004030 con 0x7f4b580c6f20 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f4b557fa640 1 -- 172.21.15.133:0/2236357397 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c004420 con 0x7f4b580c6f20 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f4b5d84b640 1 -- 172.21.15.133:0/2236357397 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b58157220 con 0x7f4b580c6f20 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f4b5d84b640 1 -- 172.21.15.133:0/2236357397 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b581577e0 con 0x7f4b580c6f20 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae30f93640 1 -- start start 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae30f93640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0b9590 0x7fae2c153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae2a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0b9590 0x7fae2c153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fae2a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0b9590 0x7fae2c153e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48498/0 (socket says 172.21.15.133:48498) 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae30f93640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae2c15d6d0 0x7fae2c0c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae30f93640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae2c15dab0 0x7fae2c0c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae30f93640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0d1f60 con 0x7fae2c0b9590 2024-04-25T10:02:18.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae30f93640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0b6f90 con 0x7fae2c15d6d0 2024-04-25T10:02:18.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae2a59c640 1 -- 172.21.15.133:0/835028089 learned_addr learned my addr 172.21.15.133:0/835028089 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:18.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae30f93640 1 -- 172.21.15.133:0/835028089 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0b88b0 con 0x7fae2c15dab0 2024-04-25T10:02:18.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae29d9b640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae2c15d6d0 0x7fae2c0c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae2ad9d640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae2c15dab0 0x7fae2c0c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fd9d0ff9640 1 -- 172.21.15.133:0/4082982160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc0052a0 con 0x7fd9e40afe20 2024-04-25T10:02:18.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d9142640 1 --2- 172.21.15.133:0/2193118903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc007140 0x7f85cc03af90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f85c80097c0 tx=0x7f85c801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8683ebbfe3c77960 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7f15761a9640 1 -- start start 2024-04-25T10:02:18.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15761a9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700725e0 0x7f157015af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15761a9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15700ae6a0 0x7f157015fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15761a9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1570162fd0 0x7f15700c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15761a9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157016ff70 con 0x7f1570162fd0 2024-04-25T10:02:18.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15761a9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157016fdf0 con 0x7f15700ae6a0 2024-04-25T10:02:18.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15761a9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15701700f0 con 0x7f15700725e0 2024-04-25T10:02:18.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15749a6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15700ae6a0 0x7f157015fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15749a6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15700ae6a0 0x7f157015fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55548/0 (socket says 172.21.15.133:55548) 2024-04-25T10:02:18.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15749a6640 1 -- 172.21.15.133:0/3525085848 learned_addr learned my addr 172.21.15.133:0/3525085848 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:18.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15751a7640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700725e0 0x7f157015af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fef196d6640 1 -- 172.21.15.133:0/1673893041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef140b20e0 msgr2=0x7fef140b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.000+0000 7fef196d6640 1 --2- 172.21.15.133:0/1673893041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef140b20e0 0x7fef140b2540 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef141668a0 tx=0x7fef0401ca30 comp rx=0 tx=0).stop 2024-04-25T10:02:18.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fef137fe640 1 --2- 172.21.15.133:0/1673893041 >> v2:172.21.15.162:3300/0 conn(0x7fef140ce7d0 0x7fef140cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:18.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae2a59c640 1 -- 172.21.15.133:0/835028089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae2c15dab0 msgr2=0x7fae2c0c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae2a59c640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae2c15dab0 0x7fae2c0c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae2a59c640 1 -- 172.21.15.133:0/835028089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae2c15d6d0 msgr2=0x7fae2c0c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae2a59c640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae2c15d6d0 0x7fae2c0c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae2a59c640 1 -- 172.21.15.133:0/835028089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c154370 con 0x7fae2c0b9590 2024-04-25T10:02:18.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fae2a59c640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0b9590 0x7fae2c153e30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c450 tx=0x7fae2000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d37fe640 1 -- 172.21.15.133:0/2193118903 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80b1070 con 0x7f85cc007140 2024-04-25T10:02:18.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d37fe640 1 -- 172.21.15.133:0/2193118903 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c80ac410 con 0x7f85cc007140 2024-04-25T10:02:18.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d37fe640 1 -- 172.21.15.133:0/2193118903 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80ac7e0 con 0x7f85cc007140 2024-04-25T10:02:18.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fef196d6640 1 -- 172.21.15.133:0/1673893041 shutdown_connections 2024-04-25T10:02:18.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fef196d6640 1 --2- 172.21.15.133:0/1673893041 >> v2:172.21.15.162:3300/0 conn(0x7fef140ce7d0 0x7fef140cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fef196d6640 1 --2- 172.21.15.133:0/1673893041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef140b20e0 0x7fef140b2540 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fef196d6640 1 --2- 172.21.15.133:0/1673893041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef14161e10 0x7fef1409cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fef196d6640 1 -- 172.21.15.133:0/1673893041 >> 172.21.15.133:0/1673893041 conn(0x7fef140cbd10 msgr2=0x7fef140cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fef196d6640 1 -- 172.21.15.133:0/1673893041 shutdown_connections 2024-04-25T10:02:18.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fef196d6640 1 -- 172.21.15.133:0/1673893041 wait complete. 2024-04-25T10:02:18.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fd9d2ffd640 1 -- 172.21.15.133:0/4082982160 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9dc00a7f0 con 0x7fd9e40afe20 2024-04-25T10:02:18.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fd9d2ffd640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd9c4066f90 0x7fd9c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fd9d2ffd640 1 -- 172.21.15.133:0/4082982160 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd9dc085bd0 con 0x7fd9e40afe20 2024-04-25T10:02:18.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f4b5d84b640 1 -- 172.21.15.133:0/2236357397 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b58074da0 con 0x7f4b580c6f20 2024-04-25T10:02:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d9943640 1 -- 172.21.15.133:0/2193118903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc007140 msgr2=0x7f85cc03af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d9943640 1 --2- 172.21.15.133:0/2193118903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc007140 0x7f85cc03af90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f85c80097c0 tx=0x7f85c801c4c0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15759a8640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1570162fd0 0x7f15700c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15751a7640 1 -- 172.21.15.133:0/3525085848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15700ae6a0 msgr2=0x7f157015fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15751a7640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15700ae6a0 0x7f157015fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15751a7640 1 -- 172.21.15.133:0/3525085848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1570162fd0 msgr2=0x7f15700c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15751a7640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1570162fd0 0x7f15700c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15751a7640 1 -- 172.21.15.133:0/3525085848 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157015ecc0 con 0x7f15700725e0 2024-04-25T10:02:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f15751a7640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700725e0 0x7f157015af00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15600ab000 tx=0x7f1560019240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f1565ffb640 1 -- 172.21.15.133:0/3525085848 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15600b0040 con 0x7f15700725e0 2024-04-25T10:02:18.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f1565ffb640 1 -- 172.21.15.133:0/3525085848 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15600bb070 con 0x7f15700725e0 2024-04-25T10:02:18.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f1565ffb640 1 -- 172.21.15.133:0/3525085848 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15600b58e0 con 0x7f15700725e0 2024-04-25T10:02:18.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fef196d6640 1 Processor -- start 2024-04-25T10:02:18.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fd9e259c640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd9c4066f90 0x7fd9c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d9943640 1 -- 172.21.15.133:0/2193118903 shutdown_connections 2024-04-25T10:02:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d9943640 1 --2- 172.21.15.133:0/2193118903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc007140 0x7f85cc03af90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d9943640 1 --2- 172.21.15.133:0/2193118903 >> v2:172.21.15.162:3300/0 conn(0x7f85cc053ec0 0x7f85cc03aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d9943640 1 -- 172.21.15.133:0/2193118903 >> 172.21.15.133:0/2193118903 conn(0x7f85cc055be0 msgr2=0x7f85cc055ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d9943640 1 -- 172.21.15.133:0/2193118903 shutdown_connections 2024-04-25T10:02:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f4b557fa640 1 -- 172.21.15.133:0/2236357397 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b4c003620 con 0x7f4b580c6f20 2024-04-25T10:02:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f4b557fa640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b3c066fb0 0x7f4b3c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7f4b557fa640 1 -- 172.21.15.133:0/2236357397 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4b4c11f210 con 0x7f4b580c6f20 2024-04-25T10:02:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7f4b57fff640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b3c066fb0 0x7f4b3c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7f4b57fff640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b3c066fb0 0x7f4b3c069470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4b580a04c0 tx=0x7f4b44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fd9e259c640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd9c4066f90 0x7fd9c4069450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80af000 tx=0x7fd9d80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7f85d9943640 1 -- 172.21.15.133:0/2193118903 wait complete. 2024-04-25T10:02:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7f85d9943640 1 Processor -- start 2024-04-25T10:02:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.004+0000 7fef196d6640 1 -- start start 2024-04-25T10:02:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef196d6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef140b20e0 0x7fef14155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef196d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef140ce7d0 0x7fef141558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef196d6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef14161e10 0x7fef14155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef196d6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef14165800 con 0x7fef140ce7d0 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef196d6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1406d2d0 con 0x7fef14161e10 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef196d6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef14165980 con 0x7fef140b20e0 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef137fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef14161e10 0x7fef14155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef137fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef14161e10 0x7fef14155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55564/0 (socket says 172.21.15.133:55564) 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef137fe640 1 -- 172.21.15.133:0/679861746 learned_addr learned my addr 172.21.15.133:0/679861746 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef137fe640 1 -- 172.21.15.133:0/679861746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef140b20e0 msgr2=0x7fef14155100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef137fe640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef140b20e0 0x7fef14155100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef137fe640 1 -- 172.21.15.133:0/679861746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef140ce7d0 msgr2=0x7fef141558c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef137fe640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef140ce7d0 0x7fef141558c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef137fe640 1 -- 172.21.15.133:0/679861746 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1409c3b0 con 0x7fef14161e10 2024-04-25T10:02:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7fef137fe640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef14161e10 0x7fef14155e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00c820 tx=0x7fef0c00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7f85d9943640 1 -- start start 2024-04-25T10:02:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7f85d9943640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc006d60 0x7f85cc17ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7f85d9943640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85cc007140 0x7f85cc178320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7f85d9943640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85cc053ec0 0x7f85cc0e5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f15761a9640 1 -- 172.21.15.133:0/3525085848 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157015efa0 con 0x7f15700725e0 2024-04-25T10:02:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f15761a9640 1 -- 172.21.15.133:0/3525085848 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157015f560 con 0x7f15700725e0 2024-04-25T10:02:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.008+0000 7f85d8941640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc006d60 0x7f85cc17ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d8941640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc006d60 0x7f85cc17ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39540/0 (socket says 172.21.15.133:39540) 2024-04-25T10:02:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d3fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85cc007140 0x7f85cc178320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d3fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85cc007140 0x7f85cc178320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55576/0 (socket says 172.21.15.133:55576) 2024-04-25T10:02:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9142640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85cc053ec0 0x7f85cc0e5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9142640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85cc053ec0 0x7f85cc0e5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48510/0 (socket says 172.21.15.133:48510) 2024-04-25T10:02:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d8941640 1 -- 172.21.15.133:0/3595493 learned_addr learned my addr 172.21.15.133:0/3595493 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0510b0 con 0x7f85cc053ec0 2024-04-25T10:02:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc050f30 con 0x7f85cc007140 2024-04-25T10:02:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc051230 con 0x7f85cc006d60 2024-04-25T10:02:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f15537fe640 1 -- 172.21.15.133:0/3525085848 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c0052a0 con 0x7f15700725e0 2024-04-25T10:02:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9142640 1 -- 172.21.15.133:0/3595493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc006d60 msgr2=0x7f85cc17ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9142640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc006d60 0x7f85cc17ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9142640 1 -- 172.21.15.133:0/3595493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85cc007140 msgr2=0x7f85cc178320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9142640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85cc007140 0x7f85cc178320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9142640 1 -- 172.21.15.133:0/3595493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc17b500 con 0x7f85cc053ec0 2024-04-25T10:02:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d8941640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc006d60 0x7f85cc17ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9142640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85cc053ec0 0x7f85cc0e5f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f85c80ac000 tx=0x7f85c8019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f1565ffb640 1 -- 172.21.15.133:0/3525085848 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15600b3070 con 0x7f15700725e0 2024-04-25T10:02:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7fd9d2ffd640 1 -- 172.21.15.133:0/4082982160 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9dc020020 con 0x7fd9e40afe20 2024-04-25T10:02:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7fae1affd640 1 -- 172.21.15.133:0/835028089 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20012830 con 0x7fae2c0b9590 2024-04-25T10:02:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7fae1affd640 1 -- 172.21.15.133:0/835028089 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae20017070 con 0x7fae2c0b9590 2024-04-25T10:02:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7fae30f93640 1 -- 172.21.15.133:0/835028089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2c15fd40 con 0x7fae2c0b9590 2024-04-25T10:02:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7fae30f93640 1 -- 172.21.15.133:0/835028089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2c160300 con 0x7fae2c0b9590 2024-04-25T10:02:18.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d17fa640 1 -- 172.21.15.133:0/3595493 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80b1040 con 0x7f85cc053ec0 2024-04-25T10:02:18.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7fae1affd640 1 -- 172.21.15.133:0/835028089 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20012bf0 con 0x7fae2c0b9590 2024-04-25T10:02:18.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f1565ffb640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1554066f60 0x7f1554069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f1565ffb640 1 -- 172.21.15.133:0/3525085848 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f156011fc20 con 0x7f15700725e0 2024-04-25T10:02:18.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f15749a6640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1554066f60 0x7f1554069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85cc17b7e0 con 0x7f85cc053ec0 2024-04-25T10:02:18.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.016+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85cc04ce50 con 0x7f85cc053ec0 2024-04-25T10:02:18.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.012+0000 7f85d17fa640 1 -- 172.21.15.133:0/3595493 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c80bb070 con 0x7f85cc053ec0 2024-04-25T10:02:18.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.016+0000 7f85d17fa640 1 -- 172.21.15.133:0/3595493 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80b5410 con 0x7f85cc053ec0 2024-04-25T10:02:18.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.016+0000 7f15749a6640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1554066f60 0x7f1554069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15701fe5b0 tx=0x7f1568009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.016+0000 7fae30f93640 1 -- 172.21.15.133:0/835028089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2c068e80 con 0x7fae2c0b9590 2024-04-25T10:02:18.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.016+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b40052a0 con 0x7f85cc053ec0 2024-04-25T10:02:18.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.016+0000 7fae1affd640 1 -- 172.21.15.133:0/835028089 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae2001c460 con 0x7fae2c0b9590 2024-04-25T10:02:18.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.016+0000 7fae1affd640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae0c066f60 0x7fae0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.020+0000 7fae29d9b640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae0c066f60 0x7fae0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.020+0000 7fae1affd640 1 -- 172.21.15.133:0/835028089 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fae20054210 con 0x7fae2c0b9590 2024-04-25T10:02:18.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.020+0000 7fae29d9b640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae0c066f60 0x7fae0c069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0ca360 tx=0x7fae1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.020+0000 7f85d17fa640 1 -- 172.21.15.133:0/3595493 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85c80b55b0 con 0x7f85cc053ec0 2024-04-25T10:02:18.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.020+0000 7f85d17fa640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85b8066f60 0x7f85b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.020+0000 7f85d8941640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85b8066f60 0x7f85b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.020+0000 7f85d8941640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85b8066f60 0x7f85b8069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f85d4064290 tx=0x7f85d4074c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.024+0000 7fae1affd640 1 -- 172.21.15.133:0/835028089 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae20056170 con 0x7fae2c0b9590 2024-04-25T10:02:18.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.024+0000 7f85d17fa640 1 -- 172.21.15.133:0/3595493 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f85c80edf50 con 0x7f85cc053ec0 2024-04-25T10:02:18.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.028+0000 7f4b557fa640 1 -- 172.21.15.133:0/2236357397 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b4c0bf070 con 0x7f4b580c6f20 2024-04-25T10:02:18.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.028+0000 7fef0b7fe640 1 -- 172.21.15.133:0/679861746 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c015070 con 0x7fef14161e10 2024-04-25T10:02:18.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.028+0000 7fef0b7fe640 1 -- 172.21.15.133:0/679861746 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0c007a30 con 0x7fef14161e10 2024-04-25T10:02:18.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.028+0000 7fef0b7fe640 1 -- 172.21.15.133:0/679861746 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c007e20 con 0x7fef14161e10 2024-04-25T10:02:18.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.028+0000 7fef196d6640 1 -- 172.21.15.133:0/679861746 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef14071940 con 0x7fef14161e10 2024-04-25T10:02:18.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.028+0000 7fef196d6640 1 -- 172.21.15.133:0/679861746 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef14155640 con 0x7fef14161e10 2024-04-25T10:02:18.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.028+0000 7f85d17fa640 1 -- 172.21.15.133:0/3595493 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85c80160b0 con 0x7f85cc053ec0 2024-04-25T10:02:18.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.028+0000 7f3573143640 1 Processor -- start 2024-04-25T10:02:18.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.028+0000 7fef196d6640 1 -- 172.21.15.133:0/679861746 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef14074da0 con 0x7fef14161e10 2024-04-25T10:02:18.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.028+0000 7fef0b7fe640 1 -- 172.21.15.133:0/679861746 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef0c018070 con 0x7fef14161e10 2024-04-25T10:02:18.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f3573143640 1 -- start start 2024-04-25T10:02:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7fef0b7fe640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feef4066fb0 0x7feef4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7fef0b7fe640 1 -- 172.21.15.133:0/679861746 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fef0c085660 con 0x7fef14161e10 2024-04-25T10:02:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7fef12ffd640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feef4066fb0 0x7feef4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7fef12ffd640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feef4066fb0 0x7feef4069470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fef140c5ec0 tx=0x7feefc006b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f3573143640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f356c0ce3b0 0x7f356c0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f3573143640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356c0b1900 con 0x7f356c0ce790 2024-04-25T10:02:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f3573143640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356c0b1be0 con 0x7f356c06cec0 2024-04-25T10:02:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f3573143640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356c0b2510 con 0x7f356c0ce3b0 2024-04-25T10:02:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f356bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f356c0ce790 0x7f356c06c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51996/0 (socket says 172.21.15.133:51996) 2024-04-25T10:02:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f356bfff640 1 -- 172.21.15.133:0/468935330 learned_addr learned my addr 172.21.15.133:0/468935330 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f3570edf640 1 --2- 172.21.15.133:0/468935330 >> v2:172.21.15.162:3300/0 conn(0x7f356c0ce3b0 0x7f356c0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f356b7fe640 1 -- 172.21.15.133:0/468935330 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255268481 0 0) 0x7f356c0b1900 con 0x7f356c0ce790 2024-04-25T10:02:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f356b7fe640 1 -- 172.21.15.133:0/468935330 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f355c003530 con 0x7f356c0ce790 2024-04-25T10:02:18.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f3570edf640 1 -- 172.21.15.133:0/468935330 >> v1:172.21.15.162:6789/0 conn(0x7f356c06cec0 legacy=0x7f356c15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f3570edf640 1 -- 172.21.15.133:0/468935330 >> v1:172.21.15.133:6789/0 conn(0x7f356c0ce790 legacy=0x7f356c06c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.032+0000 7f3570edf640 1 -- 172.21.15.133:0/468935330 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c0b1ec0 con 0x7f356c0ce3b0 2024-04-25T10:02:18.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.036+0000 7f3570edf640 1 --2- 172.21.15.133:0/468935330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c0ce3b0 0x7f356c0b1320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f356c0b2690 tx=0x7f355801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14bd22c4c77732df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.036+0000 7f1565ffb640 1 -- 172.21.15.133:0/3525085848 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1560016090 con 0x7f15700725e0 2024-04-25T10:02:18.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.044+0000 7f356b7fe640 1 -- 172.21.15.133:0/468935330 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35580b1070 con 0x7f356c0ce3b0 2024-04-25T10:02:18.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.044+0000 7f356b7fe640 1 -- 172.21.15.133:0/468935330 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35580ac5f0 con 0x7f356c0ce3b0 2024-04-25T10:02:18.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.044+0000 7f356b7fe640 1 -- 172.21.15.133:0/468935330 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35580ac9c0 con 0x7f356c0ce3b0 2024-04-25T10:02:18.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.044+0000 7fef0b7fe640 1 -- 172.21.15.133:0/679861746 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef0c0564a0 con 0x7fef14161e10 2024-04-25T10:02:18.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.048+0000 7f3573143640 1 -- 172.21.15.133:0/468935330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c0ce3b0 msgr2=0x7f356c0b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.048+0000 7f3573143640 1 --2- 172.21.15.133:0/468935330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c0ce3b0 0x7f356c0b1320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f356c0b2690 tx=0x7f355801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.048+0000 7f3573143640 1 -- 172.21.15.133:0/468935330 shutdown_connections 2024-04-25T10:02:18.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.048+0000 7f3573143640 1 --2- 172.21.15.133:0/468935330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c0ce3b0 0x7f356c0b1320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.048+0000 7f3573143640 1 -- 172.21.15.133:0/468935330 >> 172.21.15.133:0/468935330 conn(0x7f356c08af60 msgr2=0x7f356c08b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.048+0000 7f3573143640 1 -- 172.21.15.133:0/468935330 shutdown_connections 2024-04-25T10:02:18.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.048+0000 7f3573143640 1 -- 172.21.15.133:0/468935330 wait complete. 2024-04-25T10:02:18.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.048+0000 7f3573143640 1 Processor -- start 2024-04-25T10:02:18.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.048+0000 7f3573143640 1 -- start start 2024-04-25T10:02:18.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.048+0000 7f3573143640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f356c06cec0 0x7f356c0c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.048+0000 7f3573143640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c0ce3b0 0x7f356c14c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f3573143640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f356c0ce790 0x7f356c0c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f356bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c0ce3b0 0x7f356c14c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f356bfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c0ce3b0 0x7f356c14c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55604/0 (socket says 172.21.15.133:55604) 2024-04-25T10:02:18.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f35716e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f356c0ce790 0x7f356c0c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f35716e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f356c0ce790 0x7f356c0c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48512/0 (socket says 172.21.15.133:48512) 2024-04-25T10:02:18.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f3570edf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f356c06cec0 0x7f356c0c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f3570edf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f356c06cec0 0x7f356c0c7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39542/0 (socket says 172.21.15.133:39542) 2024-04-25T10:02:18.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f356bfff640 1 -- 172.21.15.133:0/3490149992 learned_addr learned my addr 172.21.15.133:0/3490149992 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c16c330 con 0x7f356c0ce790 2024-04-25T10:02:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c16c1b0 con 0x7f356c0ce3b0 2024-04-25T10:02:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c16c4b0 con 0x7f356c06cec0 2024-04-25T10:02:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f35716e0640 1 -- 172.21.15.133:0/3490149992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f356c06cec0 msgr2=0x7f356c0c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f35716e0640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f356c06cec0 0x7f356c0c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f35716e0640 1 -- 172.21.15.133:0/3490149992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c0ce3b0 msgr2=0x7f356c14c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f35716e0640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c0ce3b0 0x7f356c14c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f35716e0640 1 -- 172.21.15.133:0/3490149992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c0c7cf0 con 0x7f356c0ce790 2024-04-25T10:02:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.052+0000 7f35716e0640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f356c0ce790 0x7f356c0c8a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f356400c700 tx=0x7f356400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.056+0000 7f35697fa640 1 -- 172.21.15.133:0/3490149992 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3564016020 con 0x7f356c0ce790 2024-04-25T10:02:18.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.056+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c0c7fd0 con 0x7f356c0ce790 2024-04-25T10:02:18.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.056+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356c070590 con 0x7f356c0ce790 2024-04-25T10:02:18.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.056+0000 7f35697fa640 1 -- 172.21.15.133:0/3490149992 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3564006d60 con 0x7f356c0ce790 2024-04-25T10:02:18.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.056+0000 7f35697fa640 1 -- 172.21.15.133:0/3490149992 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3564003ba0 con 0x7f356c0ce790 2024-04-25T10:02:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.056+0000 7f35697fa640 1 -- 172.21.15.133:0/3490149992 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35640050d0 con 0x7f356c0ce790 2024-04-25T10:02:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.056+0000 7f35697fa640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3550067030 0x7f35500694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.056+0000 7f3570edf640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3550067030 0x7f35500694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.060+0000 7f3570edf640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3550067030 0x7f35500694f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f35580ae000 tx=0x7f3558008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.060+0000 7f35697fa640 1 -- 172.21.15.133:0/3490149992 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3564089060 con 0x7f356c0ce790 2024-04-25T10:02:18.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.068+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c0052a0 con 0x7f356c0ce790 2024-04-25T10:02:18.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.076+0000 7f35697fa640 1 -- 172.21.15.133:0/3490149992 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3564054e00 con 0x7f356c0ce790 2024-04-25T10:02:18.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.304+0000 7fd5147a4640 1 -- 172.21.15.133:0/2825820693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd50c0ca830 con 0x7fd50c0b1720 2024-04-25T10:02:18.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.304+0000 7fd502ffd640 1 -- 172.21.15.133:0/2825820693 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd5040efda0 con 0x7fd50c0b1720 2024-04-25T10:02:18.314 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:18.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 -- 172.21.15.133:0/2825820693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4f0066f90 msgr2=0x7fd4f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4f0066f90 0x7fd4f0069450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd50c207140 tx=0x7fd508008040 comp rx=0 tx=0).stop 2024-04-25T10:02:18.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 -- 172.21.15.133:0/2825820693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0b1720 msgr2=0x7fd50c0c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0b1720 0x7fd50c0c6360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd504021eb0 tx=0x7fd504022040 comp rx=0 tx=0).stop 2024-04-25T10:02:18.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 -- 172.21.15.133:0/2825820693 shutdown_connections 2024-04-25T10:02:18.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd50c0b36c0 0x7fd50c0c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd50c0b2bf0 0x7fd50c172770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4f0066f90 0x7fd4f0069450 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 --2- 172.21.15.133:0/2825820693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0b1720 0x7fd50c0c6360 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 -- 172.21.15.133:0/2825820693 >> 172.21.15.133:0/2825820693 conn(0x7fd50c0cbd10 msgr2=0x7fd50c0c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 -- 172.21.15.133:0/2825820693 shutdown_connections 2024-04-25T10:02:18.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.308+0000 7fd500ff9640 1 -- 172.21.15.133:0/2825820693 wait complete. 2024-04-25T10:02:18.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.320+0000 7fd9d0ff9640 1 -- 172.21.15.133:0/4082982160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd9bc005860 con 0x7fd9e40afe20 2024-04-25T10:02:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.320+0000 7fd9d2ffd640 1 -- 172.21.15.133:0/4082982160 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd9dc056a10 con 0x7fd9e40afe20 2024-04-25T10:02:18.332 INFO:teuthology.orchestra.run.smithi133.stdout:55834574860 2024-04-25T10:02:18.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.324+0000 7fd9e886a640 1 -- 172.21.15.133:0/4082982160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd9c4066f90 msgr2=0x7fd9c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.324+0000 7fd9e886a640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd9c4066f90 0x7fd9c4069450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80af000 tx=0x7fd9d80a7000 comp rx=0 tx=0).stop 2024-04-25T10:02:18.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.328+0000 7fd9e886a640 1 -- 172.21.15.133:0/4082982160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9e40afe20 msgr2=0x7fd9e41484e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.328+0000 7fd9e886a640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9e40afe20 0x7fd9e41484e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00c760 tx=0x7fd9dc00cc20 comp rx=0 tx=0).stop 2024-04-25T10:02:18.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.328+0000 7fd9e886a640 1 -- 172.21.15.133:0/4082982160 shutdown_connections 2024-04-25T10:02:18.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.328+0000 7fd9e886a640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd9e40b1110 0x7fd9e414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.328+0000 7fd9e886a640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd9c4066f90 0x7fd9c4069450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.328+0000 7fd9e886a640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9e40afe20 0x7fd9e41484e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.328+0000 7fd9e886a640 1 --2- 172.21.15.133:0/4082982160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9e40afa40 0x7fd9e414a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.328+0000 7fd9e886a640 1 -- 172.21.15.133:0/4082982160 >> 172.21.15.133:0/4082982160 conn(0x7fd9e4066b70 msgr2=0x7fd9e415a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.328+0000 7fd9e886a640 1 -- 172.21.15.133:0/4082982160 shutdown_connections 2024-04-25T10:02:18.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.328+0000 7fd9e886a640 1 -- 172.21.15.133:0/4082982160 wait complete. 2024-04-25T10:02:18.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-04-25T10:02:18.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.340+0000 7f4b5d84b640 1 -- 172.21.15.133:0/2236357397 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4b580c98e0 con 0x7f4b580c6f20 2024-04-25T10:02:18.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.340+0000 7f4b557fa640 1 -- 172.21.15.133:0/2236357397 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b4c0ac020 con 0x7f4b580c6f20 2024-04-25T10:02:18.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.340+0000 7fae30f93640 1 -- 172.21.15.133:0/835028089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fae2c073770 con 0x7fae2c0b9590 2024-04-25T10:02:18.348 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:18.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.340+0000 7fae1affd640 1 -- 172.21.15.133:0/835028089 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae2005a210 con 0x7fae2c0b9590 2024-04-25T10:02:18.349 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 -- 172.21.15.133:0/2236357397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b3c066fb0 msgr2=0x7f4b3c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b3c066fb0 0x7f4b3c069470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4b580a04c0 tx=0x7f4b44009040 comp rx=0 tx=0).stop 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 -- 172.21.15.133:0/2236357397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b580c6f20 msgr2=0x7f4b58156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b580c6f20 0x7f4b58156960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c01cf10 tx=0x7f4b4c0197d0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 -- 172.21.15.133:0/2236357397 shutdown_connections 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b3c066fb0 0x7f4b3c069470 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b580c7300 0x7f4b582032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b580c53e0 0x7f4b58157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 --2- 172.21.15.133:0/2236357397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b580c6f20 0x7f4b58156960 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 -- 172.21.15.133:0/2236357397 >> 172.21.15.133:0/2236357397 conn(0x7f4b580cbcd0 msgr2=0x7f4b5806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.6 2024-04-25T10:02:18.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 -- 172.21.15.133:0/2236357397 shutdown_connections 2024-04-25T10:02:18.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7f4b5d84b640 1 -- 172.21.15.133:0/2236357397 wait complete. 2024-04-25T10:02:18.354 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:18.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 -- 172.21.15.133:0/835028089 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae0c066f60 msgr2=0x7fae0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae0c066f60 0x7fae0c069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0ca360 tx=0x7fae1c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:18.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 -- 172.21.15.133:0/835028089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0b9590 msgr2=0x7fae2c153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0b9590 0x7fae2c153e30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c450 tx=0x7fae2000c910 comp rx=0 tx=0).stop 2024-04-25T10:02:18.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 -- 172.21.15.133:0/835028089 shutdown_connections 2024-04-25T10:02:18.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae2c15dab0 0x7fae2c0c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae2c15d6d0 0x7fae2c0c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae0c066f60 0x7fae0c069420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 --2- 172.21.15.133:0/835028089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae2c0b9590 0x7fae2c153e30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 -- 172.21.15.133:0/835028089 >> 172.21.15.133:0/835028089 conn(0x7fae2c0727c0 msgr2=0x7fae2c0b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 -- 172.21.15.133:0/835028089 shutdown_connections 2024-04-25T10:02:18.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.344+0000 7fae30f93640 1 -- 172.21.15.133:0/835028089 wait complete. 2024-04-25T10:02:18.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.348+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f85b4005860 con 0x7f85cc053ec0 2024-04-25T10:02:18.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.348+0000 7f15537fe640 1 -- 172.21.15.133:0/3525085848 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f154c005860 con 0x7f15700725e0 2024-04-25T10:02:18.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.348+0000 7f85d17fa640 1 -- 172.21.15.133:0/3595493 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f85c80efeb0 con 0x7f85cc053ec0 2024-04-25T10:02:18.357 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:18.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.348+0000 7f1565ffb640 1 -- 172.21.15.133:0/3525085848 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f15600f08d0 con 0x7f15700725e0 2024-04-25T10:02:18.358 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:18.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 -- 172.21.15.133:0/3525085848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1554066f60 msgr2=0x7f1554069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1554066f60 0x7f1554069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15701fe5b0 tx=0x7f1568009040 comp rx=0 tx=0).stop 2024-04-25T10:02:18.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 -- 172.21.15.133:0/3525085848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700725e0 msgr2=0x7f157015af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700725e0 0x7f157015af00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15600ab000 tx=0x7f1560019240 comp rx=0 tx=0).stop 2024-04-25T10:02:18.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 -- 172.21.15.133:0/3525085848 shutdown_connections 2024-04-25T10:02:18.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1554066f60 0x7f1554069420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1570162fd0 0x7f15700c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f15700ae6a0 0x7f157015fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 --2- 172.21.15.133:0/3525085848 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f15700725e0 0x7f157015af00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 -- 172.21.15.133:0/3525085848 >> 172.21.15.133:0/3525085848 conn(0x7f15700cbcd0 msgr2=0x7f15700b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 -- 172.21.15.133:0/3525085848 shutdown_connections 2024-04-25T10:02:18.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f15537fe640 1 -- 172.21.15.133:0/3525085848 wait complete. 2024-04-25T10:02:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85b8066f60 msgr2=0x7f85b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85b8066f60 0x7f85b8069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f85d4064290 tx=0x7f85d4074c10 comp rx=0 tx=0).stop 2024-04-25T10:02:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85cc053ec0 msgr2=0x7f85cc0e5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85cc053ec0 0x7f85cc0e5f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f85c80ac000 tx=0x7f85c8019260 comp rx=0 tx=0).stop 2024-04-25T10:02:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 shutdown_connections 2024-04-25T10:02:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85b8066f60 0x7f85b8069420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85cc053ec0 0x7f85cc0e5f00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85cc007140 0x7f85cc178320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 --2- 172.21.15.133:0/3595493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85cc006d60 0x7f85cc17ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 >> 172.21.15.133:0/3595493 conn(0x7f85cc055be0 msgr2=0x7f85cc007770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 shutdown_connections 2024-04-25T10:02:18.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.352+0000 7f85d9943640 1 -- 172.21.15.133:0/3595493 wait complete. 2024-04-25T10:02:18.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.356+0000 7fef196d6640 1 -- 172.21.15.133:0/679861746 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fef140b1980 con 0x7fef14161e10 2024-04-25T10:02:18.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.356+0000 7fef0b7fe640 1 -- 172.21.15.133:0/679861746 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fef0c055da0 con 0x7fef14161e10 2024-04-25T10:02:18.364 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:18.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 -- 172.21.15.133:0/679861746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feef4066fb0 msgr2=0x7feef4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feef4066fb0 0x7feef4069470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fef140c5ec0 tx=0x7feefc006b50 comp rx=0 tx=0).stop 2024-04-25T10:02:18.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 -- 172.21.15.133:0/679861746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef14161e10 msgr2=0x7fef14155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef14161e10 0x7fef14155e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00c820 tx=0x7fef0c00cce0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 -- 172.21.15.133:0/679861746 shutdown_connections 2024-04-25T10:02:18.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef14161e10 0x7fef14155e00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feef4066fb0 0x7feef4069470 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef140ce7d0 0x7fef141558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 --2- 172.21.15.133:0/679861746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef140b20e0 0x7fef14155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 -- 172.21.15.133:0/679861746 >> 172.21.15.133:0/679861746 conn(0x7fef140cbd10 msgr2=0x7fef140af720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 -- 172.21.15.133:0/679861746 shutdown_connections 2024-04-25T10:02:18.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.360+0000 7fef196d6640 1 -- 172.21.15.133:0/679861746 wait complete. 2024-04-25T10:02:18.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2024-04-25T10:02:18.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-04-25T10:02:18.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.0 2024-04-25T10:02:18.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2024-04-25T10:02:18.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2024-04-25T10:02:18.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f354c005860 con 0x7f356c0ce790 2024-04-25T10:02:18.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f35697fa640 1 -- 172.21.15.133:0/3490149992 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3564058ea0 con 0x7f356c0ce790 2024-04-25T10:02:18.399 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3550067030 msgr2=0x7f35500694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3550067030 0x7f35500694f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f35580ae000 tx=0x7f3558008040 comp rx=0 tx=0).stop 2024-04-25T10:02:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f356c0ce790 msgr2=0x7f356c0c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f356c0ce790 0x7f356c0c8a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f356400c700 tx=0x7f356400cbc0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 shutdown_connections 2024-04-25T10:02:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3550067030 0x7f35500694f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f356c0ce790 0x7f356c0c8a70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f356c0ce3b0 0x7f356c14c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 --2- 172.21.15.133:0/3490149992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f356c06cec0 0x7f356c0c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 >> 172.21.15.133:0/3490149992 conn(0x7f356c08af60 msgr2=0x7f356c0cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 shutdown_connections 2024-04-25T10:02:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:18.392+0000 7f3573143640 1 -- 172.21.15.133:0/3490149992 wait complete. 2024-04-25T10:02:18.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2024-04-25T10:02:19.342 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:02:19.383 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:02:19.388 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:02:19.388 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:02:19.391 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:02:19.393 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:02:19.417 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:02:19.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f7636e92640 1 Processor -- start 2024-04-25T10:02:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f7636e92640 1 -- start start 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f7636e92640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f76300c4d80 0x7f76300cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f7636e92640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7630166980 con 0x7f7630166410 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f7636e92640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763009a9e0 con 0x7f76300d0c50 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f7636e92640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76300b0eb0 con 0x7f76300c4d80 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f763542f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7630166410 0x7f76300b1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52012/0 (socket says 172.21.15.133:52012) 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f763542f640 1 -- 172.21.15.133:0/2335767170 learned_addr learned my addr 172.21.15.133:0/2335767170 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762ffff640 1 --2- 172.21.15.133:0/2335767170 >> v2:172.21.15.162:3300/0 conn(0x7f76300c4d80 0x7f76300cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867937341 0 0) 0x7f7630166980 con 0x7f7630166410 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7620003530 con 0x7f7630166410 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615243935 0 0) 0x7f761c001120 con 0x7f76300d0c50 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7630166980 con 0x7f76300d0c50 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3248806761 0 0) 0x7f7620003530 con 0x7f7630166410 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 >> v2:172.21.15.162:3300/0 conn(0x7f76300c4d80 msgr2=0x7f76300cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 --2- 172.21.15.133:0/2335767170 >> v2:172.21.15.162:3300/0 conn(0x7f76300c4d80 0x7f76300cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 >> v1:172.21.15.162:6789/0 conn(0x7f76300d0c50 legacy=0x7f76300c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763006beb0 con 0x7f7630166410 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7628001a00 con 0x7f7630166410 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 >> v1:172.21.15.133:6789/0 conn(0x7f7630166410 legacy=0x7f76300b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 --2- 172.21.15.133:0/2335767170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7620003b70 0x7f7620003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 --2- 172.21.15.133:0/2335767170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7620004810 0x7f7620004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 --2- 172.21.15.133:0/2335767170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 0x7f762000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76300b0eb0 con 0x7f76200051d0 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762000c0d0 con 0x7f7620004810 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.488+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762000c2d0 con 0x7f7620003b70 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.492+0000 7f763542f640 1 --2- 172.21.15.133:0/2335767170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 0x7f762000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.492+0000 7f7634c2e640 1 --2- 172.21.15.133:0/2335767170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7620003b70 0x7f7620003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.492+0000 7f762ffff640 1 --2- 172.21.15.133:0/2335767170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7620004810 0x7f7620004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.492+0000 7f763542f640 1 -- 172.21.15.133:0/2335767170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7620003b70 msgr2=0x7f7620003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.492+0000 7f763542f640 1 --2- 172.21.15.133:0/2335767170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7620003b70 0x7f7620003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.492+0000 7f763542f640 1 -- 172.21.15.133:0/2335767170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7620004810 msgr2=0x7f7620004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.492+0000 7f763542f640 1 --2- 172.21.15.133:0/2335767170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7620004810 0x7f7620004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.492+0000 7f763542f640 1 -- 172.21.15.133:0/2335767170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c001120 con 0x7f76200051d0 2024-04-25T10:02:19.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.492+0000 7f763542f640 1 -- 172.21.15.133:0/2335767170 reap_dead start 2024-04-25T10:02:19.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.492+0000 7f763542f640 1 --2- 172.21.15.133:0/2335767170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 0x7f762000b9d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f762800a7a0 tx=0x7f762801fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3823ec6c9c5e99bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762800a3f0 con 0x7f76200051d0 2024-04-25T10:02:19.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76280aa650 con 0x7f76200051d0 2024-04-25T10:02:19.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f762f7fe640 1 -- 172.21.15.133:0/2335767170 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76280aaa20 con 0x7f76200051d0 2024-04-25T10:02:19.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f7636e92640 1 -- 172.21.15.133:0/2335767170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 msgr2=0x7f762000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f7636e92640 1 --2- 172.21.15.133:0/2335767170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 0x7f762000b9d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f762800a7a0 tx=0x7f762801fa20 comp rx=0 tx=0).stop 2024-04-25T10:02:19.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f7636e92640 1 -- 172.21.15.133:0/2335767170 shutdown_connections 2024-04-25T10:02:19.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f7636e92640 1 --2- 172.21.15.133:0/2335767170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 0x7f762000b9d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f7636e92640 1 -- 172.21.15.133:0/2335767170 >> 172.21.15.133:0/2335767170 conn(0x7f76300d16a0 msgr2=0x7f76300d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f7636e92640 1 -- 172.21.15.133:0/2335767170 shutdown_connections 2024-04-25T10:02:19.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f7636e92640 1 -- 172.21.15.133:0/2335767170 wait complete. 2024-04-25T10:02:19.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f7636e92640 1 Processor -- start 2024-04-25T10:02:19.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.496+0000 7f7636e92640 1 -- start start 2024-04-25T10:02:19.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7636e92640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 0x7f76301594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7636e92640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76300b4b50 0x7f7630158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7636e92640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7630162570 0x7f76301629f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7636e92640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76300d1030 con 0x7f76200051d0 2024-04-25T10:02:19.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7636e92640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763006c470 con 0x7f76300b4b50 2024-04-25T10:02:19.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7636e92640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76300d11b0 con 0x7f7630162570 2024-04-25T10:02:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7634c2e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 0x7f76301594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7634c2e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 0x7f76301594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48520/0 (socket says 172.21.15.133:48520) 2024-04-25T10:02:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7634c2e640 1 -- 172.21.15.133:0/3303905768 learned_addr learned my addr 172.21.15.133:0/3303905768 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f763542f640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7630162570 0x7f76301629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7634c2e640 1 -- 172.21.15.133:0/3303905768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7630162570 msgr2=0x7f76301629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7634c2e640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7630162570 0x7f76301629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7634c2e640 1 -- 172.21.15.133:0/3303905768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76300b4b50 msgr2=0x7f7630158c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:02:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7634c2e640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76300b4b50 0x7f7630158c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7634c2e640 1 -- 172.21.15.133:0/3303905768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7630159c60 con 0x7f76200051d0 2024-04-25T10:02:19.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f7634c2e640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 0x7f76301594f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f761c001ec0 tx=0x7f761c00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.500+0000 7f762d7fa640 1 -- 172.21.15.133:0/3303905768 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c01a070 con 0x7f76200051d0 2024-04-25T10:02:19.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.504+0000 7f762d7fa640 1 -- 172.21.15.133:0/3303905768 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761c005320 con 0x7f76200051d0 2024-04-25T10:02:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.504+0000 7f762d7fa640 1 -- 172.21.15.133:0/3303905768 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c006a00 con 0x7f76200051d0 2024-04-25T10:02:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.504+0000 7f7636e92640 1 -- 172.21.15.133:0/3303905768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7630159f40 con 0x7f76200051d0 2024-04-25T10:02:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.504+0000 7f7636e92640 1 -- 172.21.15.133:0/3303905768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7630159150 con 0x7f76200051d0 2024-04-25T10:02:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.504+0000 7f7636e92640 1 -- 172.21.15.133:0/3303905768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7630067c70 con 0x7f76200051d0 2024-04-25T10:02:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.504+0000 7f762d7fa640 1 -- 172.21.15.133:0/3303905768 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f761c01d070 con 0x7f76200051d0 2024-04-25T10:02:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.504+0000 7f762d7fa640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7614066fe0 0x7f76140694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.508+0000 7f762ffff640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7614066fe0 0x7f76140694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.508+0000 7f762d7fa640 1 -- 172.21.15.133:0/3303905768 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f761c088430 con 0x7f76200051d0 2024-04-25T10:02:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.508+0000 7f762ffff640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7614066fe0 0x7f76140694a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f76300c5d70 tx=0x7f7624008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.512+0000 7f762d7fa640 1 -- 172.21.15.133:0/3303905768 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f761c08e080 con 0x7f76200051d0 2024-04-25T10:02:19.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bdf59e640 1 Processor -- start 2024-04-25T10:02:19.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bdf59e640 1 -- start start 2024-04-25T10:02:19.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bdf59e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3bd8055260 0x7f3bd80355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bdf59e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3bd8036fa0 0x7f3bd8052590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bdf59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8052ad0 0x7f3bd80df8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bdf59e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd803b670 con 0x7f3bd8036fa0 2024-04-25T10:02:19.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bdf59e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd8055640 con 0x7f3bd8055260 2024-04-25T10:02:19.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bde59c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3bd8055260 0x7f3bd80355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bdf59e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd803aa60 con 0x7f3bd8052ad0 2024-04-25T10:02:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bde59c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3bd8055260 0x7f3bd80355b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55636/0 (socket says 172.21.15.133:55636) 2024-04-25T10:02:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bde59c640 1 -- 172.21.15.133:0/591917683 learned_addr learned my addr 172.21.15.133:0/591917683 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.520+0000 7f3bded9d640 1 --2- 172.21.15.133:0/591917683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8052ad0 0x7f3bd80df8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.524+0000 7f3bddd9b640 1 --2- 172.21.15.133:0/591917683 >> v2:172.21.15.133:3300/0 conn(0x7f3bd8036fa0 0x7f3bd8052590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.524+0000 7f3bde59c640 1 -- 172.21.15.133:0/591917683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8052ad0 msgr2=0x7f3bd80df8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.524+0000 7f3bde59c640 1 --2- 172.21.15.133:0/591917683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8052ad0 0x7f3bd80df8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.524+0000 7f3bde59c640 1 -- 172.21.15.133:0/591917683 >> v2:172.21.15.133:3300/0 conn(0x7f3bd8036fa0 msgr2=0x7f3bd8052590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.524+0000 7f3bde59c640 1 --2- 172.21.15.133:0/591917683 >> v2:172.21.15.133:3300/0 conn(0x7f3bd8036fa0 0x7f3bd8052590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.524+0000 7f3bde59c640 1 -- 172.21.15.133:0/591917683 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd803ad40 con 0x7f3bd8055260 2024-04-25T10:02:19.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.524+0000 7f3bded9d640 1 --2- 172.21.15.133:0/591917683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8052ad0 0x7f3bd80df8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:19.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.524+0000 7f3bddd9b640 1 --2- 172.21.15.133:0/591917683 >> v2:172.21.15.133:3300/0 conn(0x7f3bd8036fa0 0x7f3bd8052590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:19.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bde59c640 1 --2- 172.21.15.133:0/591917683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd8055260 0x7f3bd80355b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8036c90 tx=0x7f3bd401c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c6f514b0ec835ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdd59a640 1 -- 172.21.15.133:0/591917683 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd40b0070 con 0x7f3bd8055260 2024-04-25T10:02:19.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdd59a640 1 -- 172.21.15.133:0/591917683 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd40ab600 con 0x7f3bd8055260 2024-04-25T10:02:19.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdd59a640 1 -- 172.21.15.133:0/591917683 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd40ab9d0 con 0x7f3bd8055260 2024-04-25T10:02:19.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdf59e640 1 -- 172.21.15.133:0/591917683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd8055260 msgr2=0x7f3bd80355b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdf59e640 1 --2- 172.21.15.133:0/591917683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd8055260 0x7f3bd80355b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8036c90 tx=0x7f3bd401c520 comp rx=0 tx=0).stop 2024-04-25T10:02:19.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdf59e640 1 -- 172.21.15.133:0/591917683 shutdown_connections 2024-04-25T10:02:19.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdf59e640 1 --2- 172.21.15.133:0/591917683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8052ad0 0x7f3bd80df8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdf59e640 1 --2- 172.21.15.133:0/591917683 >> v2:172.21.15.133:3300/0 conn(0x7f3bd8036fa0 0x7f3bd8052590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdf59e640 1 --2- 172.21.15.133:0/591917683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd8055260 0x7f3bd80355b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdf59e640 1 -- 172.21.15.133:0/591917683 >> 172.21.15.133:0/591917683 conn(0x7f3bd801ef50 msgr2=0x7f3bd8055c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdf59e640 1 -- 172.21.15.133:0/591917683 shutdown_connections 2024-04-25T10:02:19.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdf59e640 1 -- 172.21.15.133:0/591917683 wait complete. 2024-04-25T10:02:19.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdf59e640 1 Processor -- start 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.528+0000 7f3bdf59e640 1 -- start start 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bdf59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd8036fa0 0x7f3bd804d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bdf59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8052ad0 0x7f3bd804f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bddd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8052ad0 0x7f3bd804f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bddd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8052ad0 0x7f3bd804f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48528/0 (socket says 172.21.15.133:48528) 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bddd9b640 1 -- 172.21.15.133:0/1617663697 learned_addr learned my addr 172.21.15.133:0/1617663697 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bdf59e640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8055260 0x7f3bd804f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bde59c640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd8036fa0 0x7f3bd804d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd80e03a0 con 0x7f3bd8052ad0 2024-04-25T10:02:19.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd80e0220 con 0x7f3bd8036fa0 2024-04-25T10:02:19.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd80e4300 con 0x7f3bd8055260 2024-04-25T10:02:19.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bded9d640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8055260 0x7f3bd804f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bddd9b640 1 -- 172.21.15.133:0/1617663697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8055260 msgr2=0x7f3bd804f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bddd9b640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8055260 0x7f3bd804f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bddd9b640 1 -- 172.21.15.133:0/1617663697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd8036fa0 msgr2=0x7f3bd804d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bddd9b640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd8036fa0 0x7f3bd804d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bddd9b640 1 -- 172.21.15.133:0/1617663697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd80e21d0 con 0x7f3bd8052ad0 2024-04-25T10:02:19.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bde59c640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd8036fa0 0x7f3bd804d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:19.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bded9d640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8055260 0x7f3bd804f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:19.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bddd9b640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8052ad0 0x7f3bd804f390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000e9e0 tx=0x7f3bd000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bceffd640 1 -- 172.21.15.133:0/1617663697 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0017070 con 0x7f3bd8052ad0 2024-04-25T10:02:19.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd80e24b0 con 0x7f3bd8052ad0 2024-04-25T10:02:19.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.532+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd80e2a70 con 0x7f3bd8052ad0 2024-04-25T10:02:19.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.536+0000 7f3bceffd640 1 -- 172.21.15.133:0/1617663697 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd000ca30 con 0x7f3bd8052ad0 2024-04-25T10:02:19.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.536+0000 7f3bceffd640 1 -- 172.21.15.133:0/1617663697 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd000ce00 con 0x7f3bd8052ad0 2024-04-25T10:02:19.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.536+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc40052a0 con 0x7f3bd8052ad0 2024-04-25T10:02:19.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.536+0000 7f3bceffd640 1 -- 172.21.15.133:0/1617663697 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3bd001a070 con 0x7f3bd8052ad0 2024-04-25T10:02:19.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.536+0000 7f3bceffd640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bc0067060 0x7f3bc0069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.536+0000 7f3bceffd640 1 -- 172.21.15.133:0/1617663697 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3bd0085680 con 0x7f3bd8052ad0 2024-04-25T10:02:19.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.536+0000 7f3bde59c640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bc0067060 0x7f3bc0069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.536+0000 7fbb0eb1f640 1 Processor -- start 2024-04-25T10:02:19.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.536+0000 7f3bde59c640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bc0067060 0x7f3bc0069520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40ad000 tx=0x7f3bd40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.536+0000 7fbb0eb1f640 1 -- start start 2024-04-25T10:02:19.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb0eb1f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb080b1df0 0x7fbb080cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb0eb1f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbb080af4b0 0x7fbb080b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb0eb1f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbb0806df00 0x7fbb0806e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb0eb1f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb080ce270 con 0x7fbb0806df00 2024-04-25T10:02:19.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb0eb1f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb080b2710 con 0x7fbb080af4b0 2024-04-25T10:02:19.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb0eb1f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0806fac0 con 0x7fbb080b1df0 2024-04-25T10:02:19.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7f29fabd4640 1 Processor -- start 2024-04-25T10:02:19.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7f29fabd4640 1 -- start start 2024-04-25T10:02:19.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb07fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbb080af4b0 0x7fbb080b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb07fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbb080af4b0 0x7fbb080b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55654/0 (socket says 172.21.15.133:55654) 2024-04-25T10:02:19.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb07fff640 1 -- 172.21.15.133:0/111512836 learned_addr learned my addr 172.21.15.133:0/111512836 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:19.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb0c8bb640 1 --2- 172.21.15.133:0/111512836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb080b1df0 0x7fbb080cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7f29fabd4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f29f4068e80 0x7f29f4153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7f29fabd4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f415d610 con 0x7f29f40b38d0 2024-04-25T10:02:19.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7f29fabd4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f415d8f0 con 0x7f29f40b26f0 2024-04-25T10:02:19.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7f29fabd4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f415afd0 con 0x7f29f4068e80 2024-04-25T10:02:19.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb07fff640 1 -- 172.21.15.133:0/111512836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb080b1df0 msgr2=0x7fbb080cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb07fff640 1 --2- 172.21.15.133:0/111512836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb080b1df0 0x7fbb080cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb07fff640 1 -- 172.21.15.133:0/111512836 >> v2:172.21.15.133:3300/0 conn(0x7fbb0806df00 msgr2=0x7fbb0806e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb07fff640 1 --2- 172.21.15.133:0/111512836 >> v2:172.21.15.133:3300/0 conn(0x7fbb0806df00 0x7fbb0806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb07fff640 1 -- 172.21.15.133:0/111512836 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb080729e0 con 0x7fbb080af4b0 2024-04-25T10:02:19.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7f29f3fff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f29f40b26f0 0x7f29f40b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:48110/0 (socket says 172.21.15.133:48110) 2024-04-25T10:02:19.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7f29f3fff640 1 -- 172.21.15.133:0/839199319 learned_addr learned my addr 172.21.15.133:0/839199319 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:19.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.540+0000 7fbb0c8bb640 1 --2- 172.21.15.133:0/111512836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb080b1df0 0x7fbb080cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:19.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f8970640 1 --2- 172.21.15.133:0/839199319 >> v2:172.21.15.133:3300/0 conn(0x7f29f4068e80 0x7f29f4153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb07fff640 1 --2- 172.21.15.133:0/111512836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb080af4b0 0x7fbb080b21d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb0806d270 tx=0x7fbafc020db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fc547a542b11aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb077fe640 1 -- 172.21.15.133:0/111512836 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0b6070 con 0x7fbb080af4b0 2024-04-25T10:02:19.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb077fe640 1 -- 172.21.15.133:0/111512836 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbafc0a9d10 con 0x7fbb080af4b0 2024-04-25T10:02:19.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb077fe640 1 -- 172.21.15.133:0/111512836 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0b1660 con 0x7fbb080af4b0 2024-04-25T10:02:19.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136717876 0 0) 0x7f29f415d8f0 con 0x7f29f40b26f0 2024-04-25T10:02:19.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29e4003530 con 0x7f29f40b26f0 2024-04-25T10:02:19.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1337676291 0 0) 0x7f29e4003530 con 0x7f29f40b26f0 2024-04-25T10:02:19.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 >> v2:172.21.15.133:3300/0 conn(0x7f29f4068e80 msgr2=0x7f29f4153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 --2- 172.21.15.133:0/839199319 >> v2:172.21.15.133:3300/0 conn(0x7f29f4068e80 0x7f29f4153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 >> v1:172.21.15.133:6789/0 conn(0x7f29f40b38d0 legacy=0x7f29f406dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f415dbd0 con 0x7f29f40b26f0 2024-04-25T10:02:19.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f29e0001680 con 0x7f29f40b26f0 2024-04-25T10:02:19.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 >> v1:172.21.15.162:6789/0 conn(0x7f29f40b26f0 legacy=0x7f29f40b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 --2- 172.21.15.133:0/839199319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40039c0 0x7f29e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 --2- 172.21.15.133:0/839199319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29e4004810 0x7f29e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 --2- 172.21.15.133:0/839199319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29e40051d0 0x7f29e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e400c080 con 0x7f29e40039c0 2024-04-25T10:02:19.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e400c230 con 0x7f29e4004810 2024-04-25T10:02:19.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e400c430 con 0x7f29e40051d0 2024-04-25T10:02:19.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f9171640 1 --2- 172.21.15.133:0/839199319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29e40051d0 0x7f29e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/111512836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb080af4b0 msgr2=0x7fbb080b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb0eb1f640 1 --2- 172.21.15.133:0/111512836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb080af4b0 0x7fbb080b21d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb0806d270 tx=0x7fbafc020db0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/111512836 shutdown_connections 2024-04-25T10:02:19.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb0eb1f640 1 --2- 172.21.15.133:0/111512836 >> v2:172.21.15.133:3300/0 conn(0x7fbb0806df00 0x7fbb0806e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb0eb1f640 1 --2- 172.21.15.133:0/111512836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb080af4b0 0x7fbb080b21d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb0eb1f640 1 --2- 172.21.15.133:0/111512836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb080b1df0 0x7fbb080cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/111512836 >> 172.21.15.133:0/111512836 conn(0x7fbb080ccdf0 msgr2=0x7fbb080cd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f3fff640 1 --2- 172.21.15.133:0/839199319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29e4004810 0x7f29e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/111512836 shutdown_connections 2024-04-25T10:02:19.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/111512836 wait complete. 2024-04-25T10:02:19.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f3bceffd640 1 -- 172.21.15.133:0/1617663697 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3bd0055ce0 con 0x7f3bd8052ad0 2024-04-25T10:02:19.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f9171640 1 -- 172.21.15.133:0/839199319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29e4004810 msgr2=0x7f29e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f9171640 1 --2- 172.21.15.133:0/839199319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29e4004810 0x7f29e4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f9171640 1 -- 172.21.15.133:0/839199319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40039c0 msgr2=0x7f29e4003de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:02:19.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f9171640 1 --2- 172.21.15.133:0/839199319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40039c0 0x7f29e4003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.544+0000 7f29f9171640 1 -- 172.21.15.133:0/839199319 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f415dbd0 con 0x7f29e40051d0 2024-04-25T10:02:19.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0eb1f640 1 Processor -- start 2024-04-25T10:02:19.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0eb1f640 1 -- start start 2024-04-25T10:02:19.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29f9171640 1 -- 172.21.15.133:0/839199319 reap_dead start 2024-04-25T10:02:19.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29f9171640 1 --2- 172.21.15.133:0/839199319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29e40051d0 0x7f29e400b9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f29f40af330 tx=0x7f29ec01c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d33c06a6ca7bab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0eb1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb0806df00 0x7fbb08158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0eb1f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb080af4b0 0x7fbb081572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec001af0 con 0x7f29e40051d0 2024-04-25T10:02:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0c8bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb0806df00 0x7fbb08158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0c8bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb0806df00 0x7fbb08158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48552/0 (socket says 172.21.15.133:48552) 2024-04-25T10:02:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0eb1f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb080b1df0 0x7fbb081524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0c8bb640 1 -- 172.21.15.133:0/1757948752 learned_addr learned my addr 172.21.15.133:0/1757948752 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ec003bb0 con 0x7f29e40051d0 2024-04-25T10:02:19.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29f37fe640 1 -- 172.21.15.133:0/839199319 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec0050d0 con 0x7f29e40051d0 2024-04-25T10:02:19.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb080ce880 con 0x7fbb0806df00 2024-04-25T10:02:19.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb080ce700 con 0x7fbb080af4b0 2024-04-25T10:02:19.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0806ac90 con 0x7fbb080b1df0 2024-04-25T10:02:19.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0d0bc640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb080b1df0 0x7fbb081524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0c8bb640 1 -- 172.21.15.133:0/1757948752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb080b1df0 msgr2=0x7fbb081524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0c8bb640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb080b1df0 0x7fbb081524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0c8bb640 1 -- 172.21.15.133:0/1757948752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb080af4b0 msgr2=0x7fbb081572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0c8bb640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb080af4b0 0x7fbb081572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0c8bb640 1 -- 172.21.15.133:0/1757948752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb081531d0 con 0x7fbb0806df00 2024-04-25T10:02:19.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29fabd4640 1 -- 172.21.15.133:0/839199319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29e40051d0 msgr2=0x7f29e400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29fabd4640 1 --2- 172.21.15.133:0/839199319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29e40051d0 0x7f29e400b9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f29f40af330 tx=0x7f29ec01c760 comp rx=0 tx=0).stop 2024-04-25T10:02:19.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb0c8bb640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb0806df00 0x7fbb08158140 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400e810 tx=0x7fbaf400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb057fa640 1 -- 172.21.15.133:0/1757948752 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4017070 con 0x7fbb0806df00 2024-04-25T10:02:19.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb057fa640 1 -- 172.21.15.133:0/1757948752 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf400c8d0 con 0x7fbb0806df00 2024-04-25T10:02:19.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7fbb057fa640 1 -- 172.21.15.133:0/1757948752 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf400f040 con 0x7fbb0806df00 2024-04-25T10:02:19.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29fabd4640 1 -- 172.21.15.133:0/839199319 shutdown_connections 2024-04-25T10:02:19.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29fabd4640 1 --2- 172.21.15.133:0/839199319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29e40051d0 0x7f29e400b9d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29fabd4640 1 -- 172.21.15.133:0/839199319 >> 172.21.15.133:0/839199319 conn(0x7f29f406ac90 msgr2=0x7f29f406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29fabd4640 1 -- 172.21.15.133:0/839199319 shutdown_connections 2024-04-25T10:02:19.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb081534b0 con 0x7fbb0806df00 2024-04-25T10:02:19.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.548+0000 7f29fabd4640 1 -- 172.21.15.133:0/839199319 wait complete. 2024-04-25T10:02:19.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29fabd4640 1 Processor -- start 2024-04-25T10:02:19.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29fabd4640 1 -- start start 2024-04-25T10:02:19.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29fabd4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40051d0 0x7f29f4150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29fabd4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29f4203080 0x7f29f4203690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29fabd4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29f4203bd0 0x7f29f41568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29fabd4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f415e190 con 0x7f29e40051d0 2024-04-25T10:02:19.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7fbb057fa640 1 -- 172.21.15.133:0/1757948752 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbaf401a070 con 0x7fbb0806df00 2024-04-25T10:02:19.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f8970640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40051d0 0x7f29f4150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f8970640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40051d0 0x7f29f4150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48558/0 (socket says 172.21.15.133:48558) 2024-04-25T10:02:19.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f8970640 1 -- 172.21.15.133:0/2808128705 learned_addr learned my addr 172.21.15.133:0/2808128705 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:19.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29fabd4640 1 -- 172.21.15.133:0/2808128705 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f415deb0 con 0x7f29f4203080 2024-04-25T10:02:19.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29fabd4640 1 -- 172.21.15.133:0/2808128705 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f415ad50 con 0x7f29f4203bd0 2024-04-25T10:02:19.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f3fff640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29f4203080 0x7f29f4203690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f9171640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29f4203bd0 0x7f29f41568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7fbb057fa640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaec066fb0 0x7fbaec069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb08153a70 con 0x7fbb0806df00 2024-04-25T10:02:19.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7fbb07fff640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaec066fb0 0x7fbaec069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7fbb057fa640 1 -- 172.21.15.133:0/1757948752 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbaf4052bf0 con 0x7fbb0806df00 2024-04-25T10:02:19.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7fbb07fff640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaec066fb0 0x7fbaec069470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0b3000 tx=0x7fbafc004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f8970640 1 -- 172.21.15.133:0/2808128705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29f4203bd0 msgr2=0x7f29f41568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f8970640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29f4203bd0 0x7f29f41568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f8970640 1 -- 172.21.15.133:0/2808128705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29f4203080 msgr2=0x7f29f4203690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f8970640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29f4203080 0x7f29f4203690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f8970640 1 -- 172.21.15.133:0/2808128705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f40cdf50 con 0x7f29e40051d0 2024-04-25T10:02:19.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f3fff640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29f4203080 0x7f29f4203690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:19.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.552+0000 7f29f8970640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40051d0 0x7f29f4150720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f29e800d5d0 tx=0x7f29e800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7f29f17fa640 1 -- 172.21.15.133:0/2808128705 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8018020 con 0x7f29e40051d0 2024-04-25T10:02:19.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7f29f17fa640 1 -- 172.21.15.133:0/2808128705 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e8014620 con 0x7f29e40051d0 2024-04-25T10:02:19.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7f29f17fa640 1 -- 172.21.15.133:0/2808128705 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e80149f0 con 0x7f29e40051d0 2024-04-25T10:02:19.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7f29fabd4640 1 -- 172.21.15.133:0/2808128705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f40ce1c0 con 0x7f29e40051d0 2024-04-25T10:02:19.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7f29fabd4640 1 -- 172.21.15.133:0/2808128705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f40ce780 con 0x7f29e40051d0 2024-04-25T10:02:19.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7f29deffd640 1 -- 172.21.15.133:0/2808128705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29cc0052a0 con 0x7f29e40051d0 2024-04-25T10:02:19.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb08074da0 con 0x7fbb0806df00 2024-04-25T10:02:19.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7f29f17fa640 1 -- 172.21.15.133:0/2808128705 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29e801b020 con 0x7f29e40051d0 2024-04-25T10:02:19.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7f29f17fa640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29d8067060 0x7f29d8069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7f29f17fa640 1 -- 172.21.15.133:0/2808128705 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f29e8085580 con 0x7f29e40051d0 2024-04-25T10:02:19.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7f29f3fff640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29d8067060 0x7f29d8069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.556+0000 7f29f3fff640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29d8067060 0x7f29d8069520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29f4152d40 tx=0x7f29e0007f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.564+0000 7fbb057fa640 1 -- 172.21.15.133:0/1757948752 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbaf4054b50 con 0x7fbb0806df00 2024-04-25T10:02:19.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.576+0000 7f29f17fa640 1 -- 172.21.15.133:0/2808128705 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29e8022020 con 0x7f29e40051d0 2024-04-25T10:02:19.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.584+0000 7fe7d19c8640 1 Processor -- start 2024-04-25T10:02:19.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.584+0000 7fe7d19c8640 1 -- start start 2024-04-25T10:02:19.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7d19c8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe7cc0ccd00 0x7fe7cc0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7d19c8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe7cc15aa50 0x7fe7cc0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7ca7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe7cc15aa50 0x7fe7cc0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7ca7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe7cc15aa50 0x7fe7cc0cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48570/0 (socket says 172.21.15.133:48570) 2024-04-25T10:02:19.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7caffd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe7cc0ccd00 0x7fe7cc0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7ca7fc640 1 -- 172.21.15.133:0/1445055993 learned_addr learned my addr 172.21.15.133:0/1445055993 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:19.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1445055993 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7cc0cd620 con 0x7fe7cc0cc7f0 2024-04-25T10:02:19.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1445055993 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc0cc360 con 0x7fe7cc15aa50 2024-04-25T10:02:19.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1445055993 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc0b2f10 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7ca7fc640 1 -- 172.21.15.133:0/1445055993 >> v2:172.21.15.162:3300/0 conn(0x7fe7cc0ccd00 msgr2=0x7fe7cc0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7ca7fc640 1 --2- 172.21.15.133:0/1445055993 >> v2:172.21.15.162:3300/0 conn(0x7fe7cc0ccd00 0x7fe7cc0b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7ca7fc640 1 -- 172.21.15.133:0/1445055993 >> v1:172.21.15.133:6789/0 conn(0x7fe7cc0cc7f0 legacy=0x7fe7cc0caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.588+0000 7fe7ca7fc640 1 -- 172.21.15.133:0/1445055993 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc0cb750 con 0x7fe7cc15aa50 2024-04-25T10:02:19.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7c9ffb640 1 -- 172.21.15.133:0/1445055993 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511722943 0 0) 0x7fe7cc0cd620 con 0x7fe7cc0cc7f0 2024-04-25T10:02:19.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7ca7fc640 1 --2- 172.21.15.133:0/1445055993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7cc15aa50 0x7fe7cc0cd0e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc070c10 tx=0x7fe7bc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503c14096fa00025 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7c9ffb640 1 -- 172.21.15.133:0/1445055993 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0a95f0 con 0x7fe7cc15aa50 2024-04-25T10:02:19.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7c9ffb640 1 -- 172.21.15.133:0/1445055993 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc0a9790 con 0x7fe7cc15aa50 2024-04-25T10:02:19.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7c9ffb640 1 -- 172.21.15.133:0/1445055993 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0a9ba0 con 0x7fe7cc15aa50 2024-04-25T10:02:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1445055993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7cc15aa50 msgr2=0x7fe7cc0cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7d19c8640 1 --2- 172.21.15.133:0/1445055993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7cc15aa50 0x7fe7cc0cd0e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc070c10 tx=0x7fe7bc01c530 comp rx=0 tx=0).stop 2024-04-25T10:02:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1445055993 shutdown_connections 2024-04-25T10:02:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7d19c8640 1 --2- 172.21.15.133:0/1445055993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7cc15aa50 0x7fe7cc0cd0e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7d19c8640 1 --2- 172.21.15.133:0/1445055993 >> v2:172.21.15.162:3300/0 conn(0x7fe7cc0ccd00 0x7fe7cc0b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1445055993 >> 172.21.15.133:0/1445055993 conn(0x7fe7cc0999d0 msgr2=0x7fe7cc0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7f9900d55640 1 Processor -- start 2024-04-25T10:02:19.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7f9900d55640 1 -- start start 2024-04-25T10:02:19.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1445055993 shutdown_connections 2024-04-25T10:02:19.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1445055993 wait complete. 2024-04-25T10:02:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.592+0000 7f9900d55640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f98fc0cde80 0x7f98fc0cca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f9900d55640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98fc0d1ea0 0x7f98fc0ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f9900d55640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98fc0ce7a0 con 0x7f98fc0cf4f0 2024-04-25T10:02:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f9900d55640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98fc0cd170 con 0x7f98fc0cde80 2024-04-25T10:02:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f9900d55640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0b4440 con 0x7f98fc0d1ea0 2024-04-25T10:02:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f98fb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f98fc0cde80 0x7f98fc0cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f98fb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f98fc0cde80 0x7f98fc0cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48578/0 (socket says 172.21.15.133:48578) 2024-04-25T10:02:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f98fb7fe640 1 -- 172.21.15.133:0/974580053 learned_addr learned my addr 172.21.15.133:0/974580053 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7d19c8640 1 Processor -- start 2024-04-25T10:02:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f98fb7fe640 1 -- 172.21.15.133:0/974580053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98fc0d1ea0 msgr2=0x7f98fc0ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f98fb7fe640 1 --2- 172.21.15.133:0/974580053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98fc0d1ea0 0x7f98fc0ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f98fb7fe640 1 -- 172.21.15.133:0/974580053 >> v1:172.21.15.133:6789/0 conn(0x7f98fc0cf4f0 legacy=0x7f98fc0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f98fb7fe640 1 -- 172.21.15.133:0/974580053 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc0b4720 con 0x7f98fc0cde80 2024-04-25T10:02:19.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7d19c8640 1 -- start start 2024-04-25T10:02:19.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7d19c8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7cc0cc7f0 0x7fe7cc159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7d19c8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7cc0ccd00 0x7fe7cc156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7d19c8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7cc15aa50 0x7fe7cc0c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7d19c8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc06ec50 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7d19c8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc06ead0 con 0x7fe7cc15aa50 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7d19c8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc06edd0 con 0x7fe7cc0cc7f0 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7caffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7cc0cc7f0 0x7fe7cc159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7caffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7cc0cc7f0 0x7fe7cc159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39632/0 (socket says 172.21.15.133:39632) 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7caffd640 1 -- 172.21.15.133:0/1817949815 learned_addr learned my addr 172.21.15.133:0/1817949815 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7ca7fc640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7cc0ccd00 0x7fe7cc156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7cb7fe640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7cc15aa50 0x7fe7cc0c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7ca7fc640 1 -- 172.21.15.133:0/1817949815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7cc0cc7f0 msgr2=0x7fe7cc159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7ca7fc640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7cc0cc7f0 0x7fe7cc159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7ca7fc640 1 -- 172.21.15.133:0/1817949815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7cc15aa50 msgr2=0x7fe7cc0c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7ca7fc640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7cc15aa50 0x7fe7cc0c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7ca7fc640 1 -- 172.21.15.133:0/1817949815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc1fe150 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7ca7fc640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7cc0ccd00 0x7fe7cc156a20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0aa000 tx=0x7fe7bc0192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7b3fff640 1 -- 172.21.15.133:0/1817949815 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0bb5f0 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f98fb7fe640 1 --2- 172.21.15.133:0/974580053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cde80 0x7f98fc0cca90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009870 tx=0x7f98ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f17e5207729a781 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1817949815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc1fe430 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1817949815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc1fe9f0 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f98fa7fc640 1 -- 172.21.15.133:0/974580053 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0b2070 con 0x7f98fc0cde80 2024-04-25T10:02:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f98fa7fc640 1 -- 172.21.15.133:0/974580053 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec0a9070 con 0x7f98fc0cde80 2024-04-25T10:02:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.596+0000 7f98fa7fc640 1 -- 172.21.15.133:0/974580053 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0ad880 con 0x7f98fc0cde80 2024-04-25T10:02:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7fe7b3fff640 1 -- 172.21.15.133:0/1817949815 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc0bb7b0 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7fe7b3fff640 1 -- 172.21.15.133:0/1817949815 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0bbba0 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7fe7b3fff640 1 -- 172.21.15.133:0/1817949815 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7bc0b13f0 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7f9900d55640 1 -- 172.21.15.133:0/974580053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cde80 msgr2=0x7f98fc0cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7f9900d55640 1 --2- 172.21.15.133:0/974580053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cde80 0x7f98fc0cca90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009870 tx=0x7f98ec01c490 comp rx=0 tx=0).stop 2024-04-25T10:02:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7f9900d55640 1 -- 172.21.15.133:0/974580053 shutdown_connections 2024-04-25T10:02:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7f9900d55640 1 --2- 172.21.15.133:0/974580053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98fc0d1ea0 0x7f98fc0ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7f9900d55640 1 --2- 172.21.15.133:0/974580053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cde80 0x7f98fc0cca90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7f9900d55640 1 -- 172.21.15.133:0/974580053 >> 172.21.15.133:0/974580053 conn(0x7f98fc09a800 msgr2=0x7f98fc0d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7f9900d55640 1 -- 172.21.15.133:0/974580053 shutdown_connections 2024-04-25T10:02:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1817949815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ac0052a0 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7fe7b3fff640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe7b8066fb0 0x7fe7b8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7f9900d55640 1 -- 172.21.15.133:0/974580053 wait complete. 2024-04-25T10:02:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7f9900d55640 1 Processor -- start 2024-04-25T10:02:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7fe7caffd640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe7b8066fb0 0x7fe7b8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7fe7b3fff640 1 -- 172.21.15.133:0/1817949815 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe7bc11fa50 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.600+0000 7f9900d55640 1 -- start start 2024-04-25T10:02:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f9900d55640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cde80 0x7f98fc15b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f9900d55640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98fc0cf4f0 0x7f98fc06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f9900d55640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98fc0d1ea0 0x7f98fc06c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f9900d55640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc168dd0 con 0x7f98fc0cde80 2024-04-25T10:02:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f9900d55640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc168c50 con 0x7f98fc0cf4f0 2024-04-25T10:02:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f9900d55640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc168f50 con 0x7f98fc0d1ea0 2024-04-25T10:02:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7fe7caffd640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe7b8066fb0 0x7fe7b8069470 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0007b10 tx=0x7fe7c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f98fb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cde80 0x7f98fc15b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f98fb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cde80 0x7f98fc15b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48602/0 (socket says 172.21.15.133:48602) 2024-04-25T10:02:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f98faffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98fc0cf4f0 0x7f98fc06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f98fb7fe640 1 -- 172.21.15.133:0/946410906 learned_addr learned my addr 172.21.15.133:0/946410906 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f98fbfff640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98fc0d1ea0 0x7f98fc06c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f98fb7fe640 1 -- 172.21.15.133:0/946410906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98fc0d1ea0 msgr2=0x7f98fc06c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f98fb7fe640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98fc0d1ea0 0x7f98fc06c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f98fb7fe640 1 -- 172.21.15.133:0/946410906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98fc0cf4f0 msgr2=0x7f98fc06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f98fb7fe640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98fc0cf4f0 0x7f98fc06bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f98fb7fe640 1 -- 172.21.15.133:0/946410906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc15b8e0 con 0x7f98fc0cde80 2024-04-25T10:02:19.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7f98fb7fe640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cde80 0x7f98fc15b3a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98ec01cd30 tx=0x7f98ec019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7fe49231c640 1 Processor -- start 2024-04-25T10:02:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.604+0000 7fe49231c640 1 -- start start 2024-04-25T10:02:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7f98e3fff640 1 -- 172.21.15.133:0/946410906 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0b2050 con 0x7f98fc0cde80 2024-04-25T10:02:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7fe7b3fff640 1 -- 172.21.15.133:0/1817949815 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe7bc0b7070 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7f9900d55640 1 -- 172.21.15.133:0/946410906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc0c4350 con 0x7f98fc0cde80 2024-04-25T10:02:19.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7f9900d55640 1 -- 172.21.15.133:0/946410906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc158dd0 con 0x7f98fc0cde80 2024-04-25T10:02:19.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7f98e3fff640 1 -- 172.21.15.133:0/946410906 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec0bc070 con 0x7f98fc0cde80 2024-04-25T10:02:19.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7f98e3fff640 1 -- 172.21.15.133:0/946410906 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0b7840 con 0x7f98fc0cde80 2024-04-25T10:02:19.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7fe49231c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe4840e1320 0x7fe484054b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7f98e3fff640 1 -- 172.21.15.133:0/946410906 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98ec0ab070 con 0x7f98fc0cde80 2024-04-25T10:02:19.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7f98e3fff640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98e8066f60 0x7f98e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7fe49231c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe484055a70 0x7fe4840366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7fe490b19640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe484052710 0x7fe484055360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52046/0 (socket says 172.21.15.133:52046) 2024-04-25T10:02:19.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7fe490b19640 1 -- 172.21.15.133:0/1638843705 learned_addr learned my addr 172.21.15.133:0/1638843705 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:19.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7f98faffd640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98e8066f60 0x7f98e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7f98e3fff640 1 -- 172.21.15.133:0/946410906 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f98ec0ed530 con 0x7f98fc0cde80 2024-04-25T10:02:19.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7fe49231c640 1 -- 172.21.15.133:0/1638843705 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4840e5aa0 con 0x7fe484052710 2024-04-25T10:02:19.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7fe49231c640 1 -- 172.21.15.133:0/1638843705 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe484051c30 con 0x7fe4840e1320 2024-04-25T10:02:19.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7fe49231c640 1 -- 172.21.15.133:0/1638843705 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe484051f10 con 0x7fe484055a70 2024-04-25T10:02:19.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032167813 0 0) 0x7fe4840e5aa0 con 0x7fe484052710 2024-04-25T10:02:19.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe478003530 con 0x7fe484052710 2024-04-25T10:02:19.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7f98faffd640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98e8066f60 0x7f98e8069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f98fc158770 tx=0x7f98f0009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.608+0000 7fe491b1b640 1 --2- 172.21.15.133:0/1638843705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe484055a70 0x7fe4840366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2009387126 0 0) 0x7fe478003530 con 0x7fe484052710 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe484055a70 msgr2=0x7fe4840366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 --2- 172.21.15.133:0/1638843705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe484055a70 0x7fe4840366b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 >> v2:172.21.15.162:3300/0 conn(0x7fe4840e1320 msgr2=0x7fe484054b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 --2- 172.21.15.133:0/1638843705 >> v2:172.21.15.162:3300/0 conn(0x7fe4840e1320 0x7fe484054b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4840521f0 con 0x7fe484052710 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe47c001680 con 0x7fe484052710 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 >> v1:172.21.15.133:6789/0 conn(0x7fe484052710 legacy=0x7fe484055360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 --2- 172.21.15.133:0/1638843705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4780039c0 0x7fe478003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 --2- 172.21.15.133:0/1638843705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe478004810 0x7fe478004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 --2- 172.21.15.133:0/1638843705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe478005220 0x7fe47800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484051c30 con 0x7fe478005220 2024-04-25T10:02:19.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484051f10 con 0x7fe4780039c0 2024-04-25T10:02:19.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4780105e0 con 0x7fe478004810 2024-04-25T10:02:19.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe490b19640 1 --2- 172.21.15.133:0/1638843705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe478004810 0x7fe478004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe490b19640 1 -- 172.21.15.133:0/1638843705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4780039c0 msgr2=0x7fe478003de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:02:19.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe490b19640 1 --2- 172.21.15.133:0/1638843705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4780039c0 0x7fe478003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe490b19640 1 -- 172.21.15.133:0/1638843705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe478005220 msgr2=0x7fe47800b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:02:19.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe490b19640 1 --2- 172.21.15.133:0/1638843705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe478005220 0x7fe47800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.612+0000 7fe490b19640 1 -- 172.21.15.133:0/1638843705 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe478003530 con 0x7fe478004810 2024-04-25T10:02:19.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.616+0000 7fe490b19640 1 --2- 172.21.15.133:0/1638843705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe478004810 0x7fe478004c90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c2f0 tx=0x7fe47c01cd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8f0e2d019b4ba9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.616+0000 7f9900d55640 1 -- 172.21.15.133:0/946410906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc0052a0 con 0x7f98fc0cde80 2024-04-25T10:02:19.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.616+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c007da0 con 0x7fe478004810 2024-04-25T10:02:19.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.616+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c0a9020 con 0x7fe478004810 2024-04-25T10:02:19.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.616+0000 7fe483fff640 1 -- 172.21.15.133:0/1638843705 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0a5bc0 con 0x7fe478004810 2024-04-25T10:02:19.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.616+0000 7fe49231c640 1 -- 172.21.15.133:0/1638843705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe478004810 msgr2=0x7fe478004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.616+0000 7fe49231c640 1 --2- 172.21.15.133:0/1638843705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe478004810 0x7fe478004c90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c2f0 tx=0x7fe47c01cd00 comp rx=0 tx=0).stop 2024-04-25T10:02:19.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.616+0000 7fe491b1b640 1 -- 172.21.15.133:0/1638843705 reap_dead start 2024-04-25T10:02:19.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.616+0000 7fe49231c640 1 -- 172.21.15.133:0/1638843705 shutdown_connections 2024-04-25T10:02:19.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.616+0000 7fe49231c640 1 -- 172.21.15.133:0/1638843705 >> 172.21.15.133:0/1638843705 conn(0x7fe484020050 msgr2=0x7fe484056d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.620+0000 7fe49231c640 1 -- 172.21.15.133:0/1638843705 shutdown_connections 2024-04-25T10:02:19.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.620+0000 7f98e3fff640 1 -- 172.21.15.133:0/946410906 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98ec016110 con 0x7f98fc0cde80 2024-04-25T10:02:19.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.620+0000 7fe49231c640 1 -- 172.21.15.133:0/1638843705 wait complete. 2024-04-25T10:02:19.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.620+0000 7fe49231c640 1 Processor -- start 2024-04-25T10:02:19.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.620+0000 7fe49231c640 1 -- start start 2024-04-25T10:02:19.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe49231c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4840d9bc0 0x7fe484181ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe49231c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4840dfc50 0x7fe4840de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe49231c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe48404cbf0 0x7fe48404d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe49231c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484051a10 con 0x7fe4840dfc50 2024-04-25T10:02:19.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe49231c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484051890 con 0x7fe4840d9bc0 2024-04-25T10:02:19.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe49231c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840e5690 con 0x7fe48404cbf0 2024-04-25T10:02:19.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe490b19640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4840dfc50 0x7fe4840de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe490b19640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4840dfc50 0x7fe4840de3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48610/0 (socket says 172.21.15.133:48610) 2024-04-25T10:02:19.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe490b19640 1 -- 172.21.15.133:0/2550724739 learned_addr learned my addr 172.21.15.133:0/2550724739 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:19.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe491b1b640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe48404cbf0 0x7fe48404d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe490b19640 1 -- 172.21.15.133:0/2550724739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe48404cbf0 msgr2=0x7fe48404d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe490b19640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe48404cbf0 0x7fe48404d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe490b19640 1 -- 172.21.15.133:0/2550724739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4840d9bc0 msgr2=0x7fe484181ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:02:19.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe490b19640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4840d9bc0 0x7fe484181ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe490b19640 1 -- 172.21.15.133:0/2550724739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4840de910 con 0x7fe4840dfc50 2024-04-25T10:02:19.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.624+0000 7fe490b19640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4840dfc50 0x7fe4840de3d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c2f0 tx=0x7fe478005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.632+0000 7fe481ffb640 1 -- 172.21.15.133:0/2550724739 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c007da0 con 0x7fe4840dfc50 2024-04-25T10:02:19.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.632+0000 7fe481ffb640 1 -- 172.21.15.133:0/2550724739 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c0b2070 con 0x7fe4840dfc50 2024-04-25T10:02:19.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.632+0000 7fe49231c640 1 -- 172.21.15.133:0/2550724739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4840debb0 con 0x7fe4840dfc50 2024-04-25T10:02:19.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.632+0000 7fe481ffb640 1 -- 172.21.15.133:0/2550724739 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c001ce0 con 0x7fe4840dfc50 2024-04-25T10:02:19.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.632+0000 7fe49231c640 1 -- 172.21.15.133:0/2550724739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4840df170 con 0x7fe4840dfc50 2024-04-25T10:02:19.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.632+0000 7fe481ffb640 1 -- 172.21.15.133:0/2550724739 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe47c0070d0 con 0x7fe4840dfc50 2024-04-25T10:02:19.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.632+0000 7fe49231c640 1 -- 172.21.15.133:0/2550724739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4840057f0 con 0x7fe4840dfc50 2024-04-25T10:02:19.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.632+0000 7fe481ffb640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe470066fe0 0x7fe4700694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.632+0000 7fe49131a640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe470066fe0 0x7fe4700694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.636+0000 7fe481ffb640 1 -- 172.21.15.133:0/2550724739 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe47c11e890 con 0x7fe4840dfc50 2024-04-25T10:02:19.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.636+0000 7fe49131a640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe470066fe0 0x7fe4700694a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe488009690 tx=0x7fe488006be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.640+0000 7fe481ffb640 1 -- 172.21.15.133:0/2550724739 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe47c016a10 con 0x7fe4840dfc50 2024-04-25T10:02:19.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.796+0000 7fe49231c640 1 -- 172.21.15.133:0/2550724739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe484055140 con 0x7fe4840dfc50 2024-04-25T10:02:19.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.796+0000 7fe481ffb640 1 -- 172.21.15.133:0/2550724739 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe47c0eeef0 con 0x7fe4840dfc50 2024-04-25T10:02:19.804 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:19.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 -- 172.21.15.133:0/2550724739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe470066fe0 msgr2=0x7fe4700694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe470066fe0 0x7fe4700694a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe488009690 tx=0x7fe488006be0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 -- 172.21.15.133:0/2550724739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4840dfc50 msgr2=0x7fe4840de3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4840dfc50 0x7fe4840de3d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c2f0 tx=0x7fe478005840 comp rx=0 tx=0).stop 2024-04-25T10:02:19.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 -- 172.21.15.133:0/2550724739 shutdown_connections 2024-04-25T10:02:19.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe48404cbf0 0x7fe48404d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe470066fe0 0x7fe4700694a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4840dfc50 0x7fe4840de3d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 --2- 172.21.15.133:0/2550724739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe4840d9bc0 0x7fe484181ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 -- 172.21.15.133:0/2550724739 >> 172.21.15.133:0/2550724739 conn(0x7fe484020050 msgr2=0x7fe484035150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 -- 172.21.15.133:0/2550724739 shutdown_connections 2024-04-25T10:02:19.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.800+0000 7fe49231c640 1 -- 172.21.15.133:0/2550724739 wait complete. 2024-04-25T10:02:19.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2024-04-25T10:02:19.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.836+0000 7f7636e92640 1 -- 172.21.15.133:0/3303905768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f763015a110 con 0x7f76200051d0 2024-04-25T10:02:19.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.836+0000 7f762d7fa640 1 -- 172.21.15.133:0/3303905768 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f763009a9e0 con 0x7f76200051d0 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 -- 172.21.15.133:0/3303905768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7614066fe0 msgr2=0x7f76140694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7614066fe0 0x7f76140694a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f76300c5d70 tx=0x7f7624008040 comp rx=0 tx=0).stop 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 -- 172.21.15.133:0/3303905768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 msgr2=0x7f76301594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 0x7f76301594f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f761c001ec0 tx=0x7f761c00fd40 comp rx=0 tx=0).stop 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 -- 172.21.15.133:0/3303905768 shutdown_connections 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7630162570 0x7f76301629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76300b4b50 0x7f7630158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7614066fe0 0x7f76140694a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 --2- 172.21.15.133:0/3303905768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76200051d0 0x7f76301594f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 -- 172.21.15.133:0/3303905768 >> 172.21.15.133:0/3303905768 conn(0x7f76300d16a0 msgr2=0x7f76300b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 -- 172.21.15.133:0/3303905768 shutdown_connections 2024-04-25T10:02:19.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.840+0000 7f7636e92640 1 -- 172.21.15.133:0/3303905768 wait complete. 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3bc4005590 con 0x7f3bd8052ad0 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bceffd640 1 -- 172.21.15.133:0/1617663697 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3bd0059d80 con 0x7f3bd8052ad0 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bc0067060 msgr2=0x7f3bc0069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bc0067060 0x7f3bc0069520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40ad000 tx=0x7f3bd40a6000 comp rx=0 tx=0).stop 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8052ad0 msgr2=0x7f3bd804f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8052ad0 0x7f3bd804f390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000e9e0 tx=0x7f3bd000eea0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 shutdown_connections 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd8055260 0x7f3bd804f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bc0067060 0x7f3bc0069520 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8052ad0 0x7f3bd804f390 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 --2- 172.21.15.133:0/1617663697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd8036fa0 0x7f3bd804d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 >> 172.21.15.133:0/1617663697 conn(0x7f3bd801ef50 msgr2=0x7f3bd8051b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.856+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 shutdown_connections 2024-04-25T10:02:19.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.860+0000 7f3bdf59e640 1 -- 172.21.15.133:0/1617663697 wait complete. 2024-04-25T10:02:19.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-04-25T10:02:19.881 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.0 2024-04-25T10:02:19.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 -- 172.21.15.133:0/2808128705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f29cc005590 con 0x7f29e40051d0 2024-04-25T10:02:19.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbb080ad220 con 0x7fbb0806df00 2024-04-25T10:02:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29f17fa640 1 -- 172.21.15.133:0/2808128705 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29e80562c0 con 0x7f29e40051d0 2024-04-25T10:02:19.882 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb057fa640 1 -- 172.21.15.133:0/1757948752 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbaf4058bf0 con 0x7fbb0806df00 2024-04-25T10:02:19.883 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:19.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaec066fb0 msgr2=0x7fbaec069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaec066fb0 0x7fbaec069470 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0b3000 tx=0x7fbafc004740 comp rx=0 tx=0).stop 2024-04-25T10:02:19.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb0806df00 msgr2=0x7fbb08158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb0806df00 0x7fbb08158140 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400e810 tx=0x7fbaf400ecd0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 shutdown_connections 2024-04-25T10:02:19.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb080b1df0 0x7fbb081524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb080af4b0 0x7fbb081572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaec066fb0 0x7fbaec069470 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 --2- 172.21.15.133:0/1757948752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb0806df00 0x7fbb08158140 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 >> 172.21.15.133:0/1757948752 conn(0x7fbb080ccdf0 msgr2=0x7fbb080add40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 shutdown_connections 2024-04-25T10:02:19.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7fbb0eb1f640 1 -- 172.21.15.133:0/1757948752 wait complete. 2024-04-25T10:02:19.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 -- 172.21.15.133:0/2808128705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29d8067060 msgr2=0x7f29d8069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29d8067060 0x7f29d8069520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29f4152d40 tx=0x7f29e0007f10 comp rx=0 tx=0).stop 2024-04-25T10:02:19.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 -- 172.21.15.133:0/2808128705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40051d0 msgr2=0x7f29f4150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40051d0 0x7f29f4150720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f29e800d5d0 tx=0x7f29e800d9d0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 -- 172.21.15.133:0/2808128705 shutdown_connections 2024-04-25T10:02:19.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29f4203bd0 0x7f29f41568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29f4203080 0x7f29f4203690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29d8067060 0x7f29d8069520 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 --2- 172.21.15.133:0/2808128705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29e40051d0 0x7f29f4150720 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 -- 172.21.15.133:0/2808128705 >> 172.21.15.133:0/2808128705 conn(0x7f29f406ac90 msgr2=0x7f29f406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.876+0000 7f29deffd640 1 -- 172.21.15.133:0/2808128705 shutdown_connections 2024-04-25T10:02:19.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.880+0000 7f29deffd640 1 -- 172.21.15.133:0/2808128705 wait complete. 2024-04-25T10:02:19.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.892+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1817949815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe7ac005590 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.892+0000 7fe7b3fff640 1 -- 172.21.15.133:0/1817949815 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe7bc0ad020 con 0x7fe7cc0ccd00 2024-04-25T10:02:19.899 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1817949815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe7b8066fb0 msgr2=0x7fe7b8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe7b8066fb0 0x7fe7b8069470 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0007b10 tx=0x7fe7c0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1817949815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7cc0ccd00 msgr2=0x7fe7cc156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7cc0ccd00 0x7fe7cc156a20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0aa000 tx=0x7fe7bc0192d0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1817949815 shutdown_connections 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7cc15aa50 0x7fe7cc0c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe7b8066fb0 0x7fe7b8069470 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7cc0ccd00 0x7fe7cc156a20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 --2- 172.21.15.133:0/1817949815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe7cc0cc7f0 0x7fe7cc159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1817949815 >> 172.21.15.133:0/1817949815 conn(0x7fe7cc0999d0 msgr2=0x7fe7cc0ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1817949815 shutdown_connections 2024-04-25T10:02:19.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.896+0000 7fe7d19c8640 1 -- 172.21.15.133:0/1817949815 wait complete. 2024-04-25T10:02:19.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2024-04-25T10:02:19.910 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-04-25T10:02:19.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.904+0000 7f9900d55640 1 -- 172.21.15.133:0/946410906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f98dc005860 con 0x7f98fc0cde80 2024-04-25T10:02:19.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f98e3fff640 1 -- 172.21.15.133:0/946410906 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f98ec0ef490 con 0x7f98fc0cde80 2024-04-25T10:02:19.913 INFO:teuthology.orchestra.run.smithi133.stdout:55834574858 2024-04-25T10:02:19.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 -- 172.21.15.133:0/946410906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98e8066f60 msgr2=0x7f98e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98e8066f60 0x7f98e8069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f98fc158770 tx=0x7f98f0009110 comp rx=0 tx=0).stop 2024-04-25T10:02:19.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 -- 172.21.15.133:0/946410906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cde80 msgr2=0x7f98fc15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cde80 0x7f98fc15b3a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98ec01cd30 tx=0x7f98ec019230 comp rx=0 tx=0).stop 2024-04-25T10:02:19.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 -- 172.21.15.133:0/946410906 shutdown_connections 2024-04-25T10:02:19.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98fc0d1ea0 0x7f98fc06c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98fc0cf4f0 0x7f98fc06bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98e8066f60 0x7f98e8069420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 --2- 172.21.15.133:0/946410906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98fc0cde80 0x7f98fc15b3a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 -- 172.21.15.133:0/946410906 >> 172.21.15.133:0/946410906 conn(0x7f98fc09a800 msgr2=0x7f98fc06fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 -- 172.21.15.133:0/946410906 shutdown_connections 2024-04-25T10:02:19.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:19.908+0000 7f9900d55640 1 -- 172.21.15.133:0/946410906 wait complete. 2024-04-25T10:02:19.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2024-04-25T10:02:19.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2024-04-25T10:02:20.257 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:20.260 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:20.823 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:02:20.868 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:02:20.882 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:02:20.910 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:02:20.910 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:02:20.919 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:02:20.931 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:02:20.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b6d6bd640 1 Processor -- start 2024-04-25T10:02:20.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b6d6bd640 1 -- start start 2024-04-25T10:02:20.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b6d6bd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4b6815dc60 0x7f4b680b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b6d6bd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680cc690 0x7f4b680cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b6d6bd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b680d0380 con 0x7f4b680b1420 2024-04-25T10:02:20.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b6d6bd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b680cb300 con 0x7f4b6815dc60 2024-04-25T10:02:20.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b6d6bd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680cacf0 con 0x7f4b680cc690 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b677fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4b680b1420 0x7f4b680cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52050/0 (socket says 172.21.15.133:52050) 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b677fe640 1 -- 172.21.15.133:0/3077068195 learned_addr learned my addr 172.21.15.133:0/3077068195 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b667fc640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680cc690 0x7f4b680cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b66ffd640 1 --2- 172.21.15.133:0/3077068195 >> v2:172.21.15.162:3300/0 conn(0x7f4b6815dc60 0x7f4b680b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466840166 0 0) 0x7f4b680d0380 con 0x7f4b680b1420 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b50003530 con 0x7f4b680b1420 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3217408585 0 0) 0x7f4b50003530 con 0x7f4b680b1420 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680cc690 msgr2=0x7f4b680cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680cc690 0x7f4b680cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 >> v2:172.21.15.162:3300/0 conn(0x7f4b6815dc60 msgr2=0x7f4b680b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 --2- 172.21.15.133:0/3077068195 >> v2:172.21.15.162:3300/0 conn(0x7f4b6815dc60 0x7f4b680b0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b680d00a0 con 0x7f4b680b1420 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4b60001a00 con 0x7f4b680b1420 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 >> v1:172.21.15.133:6789/0 conn(0x7f4b680b1420 legacy=0x7f4b680cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b500039c0 0x7f4b50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b50004810 0x7f4b50004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b500051d0 0x7f4b5000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680cb300 con 0x7f4b50004810 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b680cacf0 con 0x7f4b500039c0 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5000c040 con 0x7f4b500051d0 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b667fc640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680cc690 0x7f4b680cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.944+0000 7f4b677fe640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b500051d0 0x7f4b5000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.948+0000 7f4b667fc640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b50004810 0x7f4b50004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.948+0000 7f4b677fe640 1 -- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b500039c0 msgr2=0x7f4b50003de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.948+0000 7f4b677fe640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b500039c0 0x7f4b50003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.948+0000 7f4b677fe640 1 -- 172.21.15.133:0/3077068195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b50004810 msgr2=0x7f4b50004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.948+0000 7f4b677fe640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b50004810 0x7f4b50004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.948+0000 7f4b677fe640 1 -- 172.21.15.133:0/3077068195 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50003530 con 0x7f4b500051d0 2024-04-25T10:02:20.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.948+0000 7f4b677fe640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b500051d0 0x7f4b5000b9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000a7d0 tx=0x7f4b6001d2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fd8f8909beb23c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.952+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000a3f0 con 0x7f4b500051d0 2024-04-25T10:02:20.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.952+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b600ad650 con 0x7f4b500051d0 2024-04-25T10:02:20.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.952+0000 7f4b65ffb640 1 -- 172.21.15.133:0/3077068195 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600adac0 con 0x7f4b500051d0 2024-04-25T10:02:20.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.952+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b500051d0 msgr2=0x7f4b5000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.952+0000 7f4b6d6bd640 1 --2- 172.21.15.133:0/3077068195 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b500051d0 0x7f4b5000b9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000a7d0 tx=0x7f4b6001d2b0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.952+0000 7f4b677fe640 1 -- 172.21.15.133:0/3077068195 reap_dead start 2024-04-25T10:02:20.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.952+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/3077068195 shutdown_connections 2024-04-25T10:02:20.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.952+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/3077068195 >> 172.21.15.133:0/3077068195 conn(0x7f4b680999d0 msgr2=0x7f4b680d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.952+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/3077068195 shutdown_connections 2024-04-25T10:02:20.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.952+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/3077068195 wait complete. 2024-04-25T10:02:20.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b6d6bd640 1 Processor -- start 2024-04-25T10:02:20.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b6d6bd640 1 -- start start 2024-04-25T10:02:20.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b6d6bd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b500051d0 0x7f4b681fc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b6d6bd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680c8050 0x7f4b681ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b6d6bd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b68152a90 0x7f4b680cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b6d6bd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b681687d0 con 0x7f4b500051d0 2024-04-25T10:02:20.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b6d6bd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b68168650 con 0x7f4b68152a90 2024-04-25T10:02:20.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b6d6bd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b68168950 con 0x7f4b680c8050 2024-04-25T10:02:20.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b667fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680c8050 0x7f4b681ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b667fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680c8050 0x7f4b681ff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39690/0 (socket says 172.21.15.133:39690) 2024-04-25T10:02:20.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b667fc640 1 -- 172.21.15.133:0/832909462 learned_addr learned my addr 172.21.15.133:0/832909462 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:20.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b66ffd640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b500051d0 0x7f4b681fc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b677fe640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b68152a90 0x7f4b680cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b677fe640 1 -- 172.21.15.133:0/832909462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680c8050 msgr2=0x7f4b681ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b677fe640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680c8050 0x7f4b681ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b677fe640 1 -- 172.21.15.133:0/832909462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b500051d0 msgr2=0x7f4b681fc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b677fe640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b500051d0 0x7f4b681fc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b677fe640 1 -- 172.21.15.133:0/832909462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b681ffdd0 con 0x7f4b68152a90 2024-04-25T10:02:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b66ffd640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b500051d0 0x7f4b681fc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b667fc640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680c8050 0x7f4b681ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.956+0000 7f4b677fe640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b68152a90 0x7f4b680cce80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b600aa000 tx=0x7f4b50005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.960+0000 7f4b4f7fe640 1 -- 172.21.15.133:0/832909462 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000a3f0 con 0x7f4b68152a90 2024-04-25T10:02:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.960+0000 7f4b4f7fe640 1 -- 172.21.15.133:0/832909462 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b600ad430 con 0x7f4b68152a90 2024-04-25T10:02:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.960+0000 7f4b4f7fe640 1 -- 172.21.15.133:0/832909462 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600017c0 con 0x7f4b68152a90 2024-04-25T10:02:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.960+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/832909462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b681fb9a0 con 0x7f4b68152a90 2024-04-25T10:02:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.960+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/832909462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b681fbc40 con 0x7f4b68152a90 2024-04-25T10:02:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.960+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/832909462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b68067c70 con 0x7f4b68152a90 2024-04-25T10:02:20.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.960+0000 7f4b4f7fe640 1 -- 172.21.15.133:0/832909462 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b600bd060 con 0x7f4b68152a90 2024-04-25T10:02:20.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.960+0000 7f4b4f7fe640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 0x7f4b54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.960+0000 7f4b4f7fe640 1 -- 172.21.15.133:0/832909462 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4b6011efe0 con 0x7f4b68152a90 2024-04-25T10:02:20.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.964+0000 7f4b66ffd640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 0x7f4b54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.964+0000 7f4b66ffd640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 0x7f4b54069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009f50 tx=0x7f4b58009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:20.972+0000 7f4b4f7fe640 1 -- 172.21.15.133:0/832909462 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b681ffdd0 con 0x7f4b68152a90 2024-04-25T10:02:21.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.004+0000 7f8529ae3640 1 Processor -- start 2024-04-25T10:02:21.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.004+0000 7f8529ae3640 1 -- start start 2024-04-25T10:02:21.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.004+0000 7f8529ae3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f852415ddd0 0x7f85240b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.004+0000 7f8529ae3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f85240cc250 0x7f85240c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.004+0000 7f8529ae3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85240d0ed0 con 0x7f852415e1b0 2024-04-25T10:02:21.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.004+0000 7f8529ae3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85240d0c80 con 0x7f85240cc250 2024-04-25T10:02:21.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.004+0000 7f8529ae3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85240cbdc0 con 0x7f852415ddd0 2024-04-25T10:02:21.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.004+0000 7f8523fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f85240cc250 0x7f85240c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.004+0000 7f8523fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f85240cc250 0x7f85240c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55746/0 (socket says 172.21.15.133:55746) 2024-04-25T10:02:21.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.004+0000 7f8523fff640 1 -- 172.21.15.133:0/398449402 learned_addr learned my addr 172.21.15.133:0/398449402 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:21.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.004+0000 7f85237fe640 1 --2- 172.21.15.133:0/398449402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f852415ddd0 0x7f85240b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.008+0000 7f8523fff640 1 -- 172.21.15.133:0/398449402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f852415ddd0 msgr2=0x7f85240b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.008+0000 7f8523fff640 1 --2- 172.21.15.133:0/398449402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f852415ddd0 0x7f85240b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.008+0000 7f8523fff640 1 -- 172.21.15.133:0/398449402 >> v1:172.21.15.162:6789/0 conn(0x7f852415e1b0 legacy=0x7f85240c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.008+0000 7f8523fff640 1 -- 172.21.15.133:0/398449402 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85240d11b0 con 0x7f85240cc250 2024-04-25T10:02:21.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.008+0000 7f8523fff640 1 --2- 172.21.15.133:0/398449402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85240cc250 0x7f85240c7d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d150 tx=0x7f851c0213e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d44e07714f1f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.008+0000 7f85227fc640 1 -- 172.21.15.133:0/398449402 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c0b6070 con 0x7f85240cc250 2024-04-25T10:02:21.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f85227fc640 1 -- 172.21.15.133:0/398449402 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c00b2d0 con 0x7f85240cc250 2024-04-25T10:02:21.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f85227fc640 1 -- 172.21.15.133:0/398449402 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00b480 con 0x7f85240cc250 2024-04-25T10:02:21.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f8529ae3640 1 -- 172.21.15.133:0/398449402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85240cc250 msgr2=0x7f85240c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f8529ae3640 1 --2- 172.21.15.133:0/398449402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85240cc250 0x7f85240c7d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d150 tx=0x7f851c0213e0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f8529ae3640 1 -- 172.21.15.133:0/398449402 shutdown_connections 2024-04-25T10:02:21.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f8529ae3640 1 --2- 172.21.15.133:0/398449402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85240cc250 0x7f85240c7d50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f8529ae3640 1 --2- 172.21.15.133:0/398449402 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f852415ddd0 0x7f85240b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f8529ae3640 1 -- 172.21.15.133:0/398449402 >> 172.21.15.133:0/398449402 conn(0x7f85240999d0 msgr2=0x7f85240d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f8529ae3640 1 -- 172.21.15.133:0/398449402 shutdown_connections 2024-04-25T10:02:21.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f8529ae3640 1 -- 172.21.15.133:0/398449402 wait complete. 2024-04-25T10:02:21.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f8529ae3640 1 Processor -- start 2024-04-25T10:02:21.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.012+0000 7f8529ae3640 1 -- start start 2024-04-25T10:02:21.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8529ae3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85240cc250 0x7f85241e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8529ae3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f852415ddd0 0x7f85241fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8529ae3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f852415e1b0 0x7f8524201220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8529ae3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85240c8410 con 0x7f852415e1b0 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8529ae3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85240c8290 con 0x7f852415ddd0 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8529ae3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85240ce140 con 0x7f85240cc250 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8523fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f852415e1b0 0x7f8524201220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8523fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f852415e1b0 0x7f8524201220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48638/0 (socket says 172.21.15.133:48638) 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8523fff640 1 -- 172.21.15.133:0/4291964483 learned_addr learned my addr 172.21.15.133:0/4291964483 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f85237fe640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85240cc250 0x7f85241e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8522ffd640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f852415ddd0 0x7f85241fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8523fff640 1 -- 172.21.15.133:0/4291964483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85240cc250 msgr2=0x7f85241e7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8523fff640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85240cc250 0x7f85241e7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8523fff640 1 -- 172.21.15.133:0/4291964483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f852415ddd0 msgr2=0x7f85241fa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8523fff640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f852415ddd0 0x7f85241fa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8523fff640 1 -- 172.21.15.133:0/4291964483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85241e6300 con 0x7f852415e1b0 2024-04-25T10:02:21.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8523fff640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f852415e1b0 0x7f8524201220 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f851c021c80 tx=0x7f851c01e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f850bfff640 1 -- 172.21.15.133:0/4291964483 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c0b6070 con 0x7f852415e1b0 2024-04-25T10:02:21.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f850bfff640 1 -- 172.21.15.133:0/4291964483 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c0bc420 con 0x7f852415e1b0 2024-04-25T10:02:21.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f850bfff640 1 -- 172.21.15.133:0/4291964483 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00b480 con 0x7f852415e1b0 2024-04-25T10:02:21.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8529ae3640 1 -- 172.21.15.133:0/4291964483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85241e5a60 con 0x7f852415e1b0 2024-04-25T10:02:21.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8529ae3640 1 -- 172.21.15.133:0/4291964483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85241e6020 con 0x7f852415e1b0 2024-04-25T10:02:21.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.016+0000 7f8529ae3640 1 -- 172.21.15.133:0/4291964483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8524067c70 con 0x7f852415e1b0 2024-04-25T10:02:21.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.020+0000 7f850bfff640 1 -- 172.21.15.133:0/4291964483 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f851c0ba030 con 0x7f852415e1b0 2024-04-25T10:02:21.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.020+0000 7f850bfff640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8510066fb0 0x7f8510069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.020+0000 7f850bfff640 1 -- 172.21.15.133:0/4291964483 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f851c122970 con 0x7f852415e1b0 2024-04-25T10:02:21.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.020+0000 7f85237fe640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8510066fb0 0x7f8510069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.020+0000 7f85237fe640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8510066fb0 0x7f8510069470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f85241e73b0 tx=0x7f8518009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.028+0000 7f850bfff640 1 -- 172.21.15.133:0/4291964483 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f851c0c2060 con 0x7f852415e1b0 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f1865c89640 1 Processor -- start 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f1865c89640 1 -- start start 2024-04-25T10:02:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f1865c89640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f18600b9590 0x7f18600d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f1865c89640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18600d1d90 0x7f18600b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f1865c89640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18600b72d0 con 0x7f186015d6d0 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f1865c89640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18600b75b0 con 0x7f18600b9590 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f1865c89640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600b7890 con 0x7f18600d1d90 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f185effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f18600b9590 0x7f18600d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f185f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f186015d6d0 0x7f18600b6880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52052/0 (socket says 172.21.15.133:52052) 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f185effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f18600b9590 0x7f18600d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48650/0 (socket says 172.21.15.133:48650) 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f185f7fe640 1 -- 172.21.15.133:0/2118531396 learned_addr learned my addr 172.21.15.133:0/2118531396 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f185ffff640 1 --2- 172.21.15.133:0/2118531396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18600d1d90 0x7f18600b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f185effd640 1 -- 172.21.15.133:0/2118531396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18600d1d90 msgr2=0x7f18600b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f185effd640 1 --2- 172.21.15.133:0/2118531396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18600d1d90 0x7f18600b8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f185effd640 1 -- 172.21.15.133:0/2118531396 >> v1:172.21.15.133:6789/0 conn(0x7f186015d6d0 legacy=0x7f18600b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.040+0000 7f185effd640 1 -- 172.21.15.133:0/2118531396 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600b7b70 con 0x7f18600b9590 2024-04-25T10:02:21.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.044+0000 7f185effd640 1 --2- 172.21.15.133:0/2118531396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18600b9590 0x7f18600d1850 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f18540097c0 tx=0x7f185401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2acfdff0bfd9cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f185e7fc640 1 -- 172.21.15.133:0/2118531396 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18540b5070 con 0x7f18600b9590 2024-04-25T10:02:21.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f185e7fc640 1 -- 172.21.15.133:0/2118531396 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18540a8990 con 0x7f18600b9590 2024-04-25T10:02:21.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f185e7fc640 1 -- 172.21.15.133:0/2118531396 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18540a8d60 con 0x7f18600b9590 2024-04-25T10:02:21.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 -- 172.21.15.133:0/2118531396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18600b9590 msgr2=0x7f18600d1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 --2- 172.21.15.133:0/2118531396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18600b9590 0x7f18600d1850 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f18540097c0 tx=0x7f185401ec90 comp rx=0 tx=0).stop 2024-04-25T10:02:21.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 -- 172.21.15.133:0/2118531396 shutdown_connections 2024-04-25T10:02:21.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 --2- 172.21.15.133:0/2118531396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18600d1d90 0x7f18600b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 --2- 172.21.15.133:0/2118531396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18600b9590 0x7f18600d1850 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 -- 172.21.15.133:0/2118531396 >> 172.21.15.133:0/2118531396 conn(0x7f18600727c0 msgr2=0x7f1860072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 -- 172.21.15.133:0/2118531396 shutdown_connections 2024-04-25T10:02:21.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 -- 172.21.15.133:0/2118531396 wait complete. 2024-04-25T10:02:21.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 Processor -- start 2024-04-25T10:02:21.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 -- start start 2024-04-25T10:02:21.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18600b9590 0x7f18601f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18600d1d90 0x7f18600c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f186015d6d0 0x7f18600b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186015dab0 con 0x7f18600d1d90 2024-04-25T10:02:21.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18600b6f90 con 0x7f186015d6d0 2024-04-25T10:02:21.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f1865c89640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186015dc30 con 0x7f18600b9590 2024-04-25T10:02:21.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f185effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18600d1d90 0x7f18600c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f185f7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18600b9590 0x7f18601f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f185effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18600d1d90 0x7f18600c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48660/0 (socket says 172.21.15.133:48660) 2024-04-25T10:02:21.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f185effd640 1 -- 172.21.15.133:0/1692340739 learned_addr learned my addr 172.21.15.133:0/1692340739 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:21.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.048+0000 7f185ffff640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f186015d6d0 0x7f18600b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.052+0000 7f185effd640 1 -- 172.21.15.133:0/1692340739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18600b9590 msgr2=0x7f18601f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.052+0000 7f185effd640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18600b9590 0x7f18601f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.052+0000 7f185effd640 1 -- 172.21.15.133:0/1692340739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f186015d6d0 msgr2=0x7f18600b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.052+0000 7f185effd640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f186015d6d0 0x7f18600b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.052+0000 7f185effd640 1 -- 172.21.15.133:0/1692340739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600d0a80 con 0x7f18600d1d90 2024-04-25T10:02:21.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.052+0000 7f185effd640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18600d1d90 0x7f18600c88c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f18540b3040 tx=0x7f185401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.056+0000 7f1847fff640 1 -- 172.21.15.133:0/1692340739 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18540bf070 con 0x7f18600d1d90 2024-04-25T10:02:21.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.056+0000 7f1865c89640 1 -- 172.21.15.133:0/1692340739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18600c7820 con 0x7f18600d1d90 2024-04-25T10:02:21.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.056+0000 7f1865c89640 1 -- 172.21.15.133:0/1692340739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18600c7de0 con 0x7f18600d1d90 2024-04-25T10:02:21.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.056+0000 7f1847fff640 1 -- 172.21.15.133:0/1692340739 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18540b0820 con 0x7f18600d1d90 2024-04-25T10:02:21.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.056+0000 7f1847fff640 1 -- 172.21.15.133:0/1692340739 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18540b5040 con 0x7f18600d1d90 2024-04-25T10:02:21.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.056+0000 7f1865c89640 1 -- 172.21.15.133:0/1692340739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18400052a0 con 0x7f18600d1d90 2024-04-25T10:02:21.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.056+0000 7f1847fff640 1 -- 172.21.15.133:0/1692340739 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18540c2070 con 0x7f18600d1d90 2024-04-25T10:02:21.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.056+0000 7f1847fff640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f184c066fb0 0x7f184c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.056+0000 7f185f7fe640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f184c066fb0 0x7f184c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.060+0000 7f185f7fe640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f184c066fb0 0x7f184c069470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1850000f70 tx=0x7f1850007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.060+0000 7f1847fff640 1 -- 172.21.15.133:0/1692340739 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1854121d00 con 0x7f18600d1d90 2024-04-25T10:02:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.064+0000 7f1847fff640 1 -- 172.21.15.133:0/1692340739 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18540187f0 con 0x7f18600d1d90 2024-04-25T10:02:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb69b56b640 1 Processor -- start 2024-04-25T10:02:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb69b56b640 1 -- start start 2024-04-25T10:02:21.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb69b56b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6940d0c50 0x7fb6940cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb69b56b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb694166980 con 0x7fb694166410 2024-04-25T10:02:21.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb69b56b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69409a9e0 con 0x7fb6940c4d80 2024-04-25T10:02:21.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb698b06640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb694166410 0x7fb6940c4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52062/0 (socket says 172.21.15.133:52062) 2024-04-25T10:02:21.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb698b06640 1 -- 172.21.15.133:0/1181612106 learned_addr learned my addr 172.21.15.133:0/1181612106 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:21.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb69b56b640 1 -- 172.21.15.133:0/1181612106 --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6940b0eb0 con 0x7fb6940d0c50 2024-04-25T10:02:21.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb699307640 1 --2- 172.21.15.133:0/1181612106 >> v2:172.21.15.162:3300/0 conn(0x7fb6940d0c50 0x7fb6940cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb68bfff640 1 -- 172.21.15.133:0/1181612106 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813932158 0 0) 0x7fb694166980 con 0x7fb694166410 2024-04-25T10:02:21.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7f4d1efbb640 1 Processor -- start 2024-04-25T10:02:21.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb68bfff640 1 -- 172.21.15.133:0/1181612106 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb680003530 con 0x7fb694166410 2024-04-25T10:02:21.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb68bfff640 1 -- 172.21.15.133:0/1181612106 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375732946 0 0) 0x7fb68c0014a0 con 0x7fb6940c4d80 2024-04-25T10:02:21.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb68bfff640 1 -- 172.21.15.133:0/1181612106 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb694166980 con 0x7fb6940c4d80 2024-04-25T10:02:21.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7f4d1efbb640 1 -- start start 2024-04-25T10:02:21.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb699307640 1 -- 172.21.15.133:0/1181612106 >> v1:172.21.15.162:6789/0 conn(0x7fb6940c4d80 legacy=0x7fb6940b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb68bfff640 1 -- 172.21.15.133:0/1181612106 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1952341225 0 0) 0x7fb680003530 con 0x7fb694166410 2024-04-25T10:02:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb699307640 1 -- 172.21.15.133:0/1181612106 >> v1:172.21.15.133:6789/0 conn(0x7fb694166410 legacy=0x7fb6940c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.084+0000 7fb699307640 1 -- 172.21.15.133:0/1181612106 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69406beb0 con 0x7fb6940d0c50 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d1efbb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d1806e970 0x7f4d18162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d1efbb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d180ce450 0x7f4d180b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d1efbb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d180cd6f0 con 0x7f4d180ce8e0 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d1efbb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d18163340 con 0x7f4d1806e970 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d1efbb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d180cf270 con 0x7f4d180ce450 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d1cd57640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d1806e970 0x7f4d18162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d17fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d180ce450 0x7f4d180b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d17fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d180ce450 0x7f4d180b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55790/0 (socket says 172.21.15.133:55790) 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d17fff640 1 -- 172.21.15.133:0/2381591009 learned_addr learned my addr 172.21.15.133:0/2381591009 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d17fff640 1 -- 172.21.15.133:0/2381591009 >> v2:172.21.15.133:3300/0 conn(0x7f4d1806e970 msgr2=0x7f4d18162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d17fff640 1 --2- 172.21.15.133:0/2381591009 >> v2:172.21.15.133:3300/0 conn(0x7f4d1806e970 0x7f4d18162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d17fff640 1 -- 172.21.15.133:0/2381591009 >> v1:172.21.15.133:6789/0 conn(0x7f4d180ce8e0 legacy=0x7f4d1806d2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:02:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d17fff640 1 -- 172.21.15.133:0/2381591009 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180cd970 con 0x7f4d180ce450 2024-04-25T10:02:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7f4d1cd57640 1 --2- 172.21.15.133:0/2381591009 >> v2:172.21.15.133:3300/0 conn(0x7f4d1806e970 0x7f4d18162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7fb699307640 1 --2- 172.21.15.133:0/1181612106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6940d0c50 0x7fb6940cf270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb6840097c0 tx=0x7fb68401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1acd3d1f459b26c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7fb68bfff640 1 -- 172.21.15.133:0/1181612106 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840b0070 con 0x7fb6940d0c50 2024-04-25T10:02:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7fb68bfff640 1 -- 172.21.15.133:0/1181612106 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6840ab400 con 0x7fb6940d0c50 2024-04-25T10:02:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7fb68bfff640 1 -- 172.21.15.133:0/1181612106 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690001680 con 0x7fb6940d0c50 2024-04-25T10:02:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7fb69b56b640 1 -- 172.21.15.133:0/1181612106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6940d0c50 msgr2=0x7fb6940cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7fb69b56b640 1 --2- 172.21.15.133:0/1181612106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6940d0c50 0x7fb6940cf270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb6840097c0 tx=0x7fb68401c390 comp rx=0 tx=0).stop 2024-04-25T10:02:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7fb69b56b640 1 -- 172.21.15.133:0/1181612106 shutdown_connections 2024-04-25T10:02:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7fb69b56b640 1 --2- 172.21.15.133:0/1181612106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6940d0c50 0x7fb6940cf270 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7fb69b56b640 1 -- 172.21.15.133:0/1181612106 >> 172.21.15.133:0/1181612106 conn(0x7fb6940d16a0 msgr2=0x7fb6940d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7fb69b56b640 1 -- 172.21.15.133:0/1181612106 shutdown_connections 2024-04-25T10:02:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.088+0000 7fb69b56b640 1 -- 172.21.15.133:0/1181612106 wait complete. 2024-04-25T10:02:21.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7f4d17fff640 1 --2- 172.21.15.133:0/2381591009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d180ce450 0x7f4d180b14a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d1806f130 tx=0x7f4d0801c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c97087464fc34b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7f4d177fe640 1 -- 172.21.15.133:0/2381591009 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d080b0070 con 0x7f4d180ce450 2024-04-25T10:02:21.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7f4d177fe640 1 -- 172.21.15.133:0/2381591009 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d080ab600 con 0x7f4d180ce450 2024-04-25T10:02:21.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7f4d177fe640 1 -- 172.21.15.133:0/2381591009 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d080ab9d0 con 0x7f4d180ce450 2024-04-25T10:02:21.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb69b56b640 1 Processor -- start 2024-04-25T10:02:21.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb69b56b640 1 -- start start 2024-04-25T10:02:21.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7f4d1efbb640 1 -- 172.21.15.133:0/2381591009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d180ce450 msgr2=0x7f4d180b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7f4d1efbb640 1 --2- 172.21.15.133:0/2381591009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d180ce450 0x7f4d180b14a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d1806f130 tx=0x7f4d0801c270 comp rx=0 tx=0).stop 2024-04-25T10:02:21.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb69b56b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6940c4d80 0x7fb694164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb69b56b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6940d0c50 0x7fb694160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb69b56b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb694166410 0x7fb694162150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb69b56b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6940d11b0 con 0x7fb6940c4d80 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb69b56b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6940d1030 con 0x7fb6940d0c50 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb69b56b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69416fdb0 con 0x7fb694166410 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb699307640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6940c4d80 0x7fb694164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb699307640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6940c4d80 0x7fb694164ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48680/0 (socket says 172.21.15.133:48680) 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb699307640 1 -- 172.21.15.133:0/1138273332 learned_addr learned my addr 172.21.15.133:0/1138273332 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb699b08640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb694166410 0x7fb694162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb698b06640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6940d0c50 0x7fb694160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb699307640 1 -- 172.21.15.133:0/1138273332 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb694166410 msgr2=0x7fb694162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb699307640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb694166410 0x7fb694162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb699307640 1 -- 172.21.15.133:0/1138273332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6940d0c50 msgr2=0x7fb694160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb699307640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6940d0c50 0x7fb694160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb699307640 1 -- 172.21.15.133:0/1138273332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6940c53b0 con 0x7fb6940c4d80 2024-04-25T10:02:21.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.092+0000 7fb699307640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6940c4d80 0x7fb694164ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6840ae040 tx=0x7fb68401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7fb689ffb640 1 -- 172.21.15.133:0/1138273332 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840bb020 con 0x7fb6940c4d80 2024-04-25T10:02:21.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7fb689ffb640 1 -- 172.21.15.133:0/1138273332 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6840b1040 con 0x7fb6940c4d80 2024-04-25T10:02:21.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 -- 172.21.15.133:0/2381591009 shutdown_connections 2024-04-25T10:02:21.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 --2- 172.21.15.133:0/2381591009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d180ce450 0x7f4d180b14a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 --2- 172.21.15.133:0/2381591009 >> v2:172.21.15.133:3300/0 conn(0x7f4d1806e970 0x7f4d18162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 -- 172.21.15.133:0/2381591009 >> 172.21.15.133:0/2381591009 conn(0x7f4d18066f70 msgr2=0x7f4d1806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 -- 172.21.15.133:0/2381591009 shutdown_connections 2024-04-25T10:02:21.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 -- 172.21.15.133:0/2381591009 wait complete. 2024-04-25T10:02:21.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7fb689ffb640 1 -- 172.21.15.133:0/1138273332 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840b0050 con 0x7fb6940c4d80 2024-04-25T10:02:21.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7fb69b56b640 1 -- 172.21.15.133:0/1138273332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6940c5690 con 0x7fb6940c4d80 2024-04-25T10:02:21.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7fb69b56b640 1 -- 172.21.15.133:0/1138273332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb694164be0 con 0x7fb6940c4d80 2024-04-25T10:02:21.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7fb69b56b640 1 -- 172.21.15.133:0/1138273332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb694067c70 con 0x7fb6940c4d80 2024-04-25T10:02:21.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 Processor -- start 2024-04-25T10:02:21.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7fb689ffb640 1 -- 172.21.15.133:0/1138273332 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6840be070 con 0x7fb6940c4d80 2024-04-25T10:02:21.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 -- start start 2024-04-25T10:02:21.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1806e970 0x7f4d18154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d180ce450 0x7f4d180b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d180ce8e0 0x7f4d1815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180cf050 con 0x7f4d1806e970 2024-04-25T10:02:21.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180ceed0 con 0x7f4d180ce450 2024-04-25T10:02:21.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1efbb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180b21c0 con 0x7f4d180ce8e0 2024-04-25T10:02:21.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7fb689ffb640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6800673d0 0x7fb680069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1d558640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d180ce8e0 0x7f4d1815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1d558640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d180ce8e0 0x7f4d1815c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39744/0 (socket says 172.21.15.133:39744) 2024-04-25T10:02:21.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7f4d1d558640 1 -- 172.21.15.133:0/292457343 learned_addr learned my addr 172.21.15.133:0/292457343 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:21.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7fb689ffb640 1 -- 172.21.15.133:0/1138273332 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb68411e710 con 0x7fb6940c4d80 2024-04-25T10:02:21.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.096+0000 7fb698b06640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6800673d0 0x7fb680069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d17fff640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d180ce450 0x7f4d180b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d1d558640 1 -- 172.21.15.133:0/292457343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d180ce450 msgr2=0x7f4d180b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d1d558640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d180ce450 0x7f4d180b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d1d558640 1 -- 172.21.15.133:0/292457343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1806e970 msgr2=0x7f4d18154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d1d558640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1806e970 0x7f4d18154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d1d558640 1 -- 172.21.15.133:0/292457343 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180b2da0 con 0x7f4d180ce8e0 2024-04-25T10:02:21.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d17fff640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d180ce450 0x7f4d180b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:21.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7fb698b06640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6800673d0 0x7fb680069890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb69415e3b0 tx=0x7fb690001720 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d1d558640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d180ce8e0 0x7f4d1815c2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00b410 tx=0x7f4d0c00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d157fa640 1 -- 172.21.15.133:0/292457343 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c015020 con 0x7f4d180ce8e0 2024-04-25T10:02:21.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d1efbb640 1 -- 172.21.15.133:0/292457343 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d180b3080 con 0x7f4d180ce8e0 2024-04-25T10:02:21.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d157fa640 1 -- 172.21.15.133:0/292457343 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c011830 con 0x7f4d180ce8e0 2024-04-25T10:02:21.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d157fa640 1 -- 172.21.15.133:0/292457343 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c011c20 con 0x7f4d180ce8e0 2024-04-25T10:02:21.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d1efbb640 1 -- 172.21.15.133:0/292457343 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d180b3640 con 0x7f4d180ce8e0 2024-04-25T10:02:21.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4cfeffd640 1 -- 172.21.15.133:0/292457343 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf00052a0 con 0x7f4d180ce8e0 2024-04-25T10:02:21.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d157fa640 1 -- 172.21.15.133:0/292457343 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d0c018020 con 0x7f4d180ce8e0 2024-04-25T10:02:21.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d157fa640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d04066f90 0x7f4d04069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.100+0000 7f4d157fa640 1 -- 172.21.15.133:0/292457343 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4d0c085820 con 0x7f4d180ce8e0 2024-04-25T10:02:21.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.104+0000 7f4d1cd57640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d04066f90 0x7f4d04069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.104+0000 7f4d1cd57640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d04066f90 0x7f4d04069450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000e250 tx=0x7f4d1000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.108+0000 7fb689ffb640 1 -- 172.21.15.133:0/1138273332 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb684015fb0 con 0x7fb6940c4d80 2024-04-25T10:02:21.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.108+0000 7f713a376640 1 Processor -- start 2024-04-25T10:02:21.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.108+0000 7f713a376640 1 -- start start 2024-04-25T10:02:21.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f713a376640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f712c0dfee0 0x7f712c055580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f713a376640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f712c03b570 0x7f712c053c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f713a376640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f712c050920 0x7f712c050db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f713a376640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712c0550f0 con 0x7f712c03b570 2024-04-25T10:02:21.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f713a376640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712c0e02c0 con 0x7f712c0dfee0 2024-04-25T10:02:21.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f713a376640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0544e0 con 0x7f712c050920 2024-04-25T10:02:21.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f7138b73640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f712c03b570 0x7f712c053c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f7138b73640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f712c03b570 0x7f712c053c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48700/0 (socket says 172.21.15.133:48700) 2024-04-25T10:02:21.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f7138b73640 1 -- 172.21.15.133:0/3765148487 learned_addr learned my addr 172.21.15.133:0/3765148487 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:21.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f7139b75640 1 --2- 172.21.15.133:0/3765148487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f712c050920 0x7f712c050db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f7139374640 1 --2- 172.21.15.133:0/3765148487 >> v2:172.21.15.162:3300/0 conn(0x7f712c0dfee0 0x7f712c055580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f7138b73640 1 -- 172.21.15.133:0/3765148487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f712c050920 msgr2=0x7f712c050db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f7138b73640 1 --2- 172.21.15.133:0/3765148487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f712c050920 0x7f712c050db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f7138b73640 1 -- 172.21.15.133:0/3765148487 >> v2:172.21.15.162:3300/0 conn(0x7f712c0dfee0 msgr2=0x7f712c055580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f7138b73640 1 --2- 172.21.15.133:0/3765148487 >> v2:172.21.15.162:3300/0 conn(0x7f712c0dfee0 0x7f712c055580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.112+0000 7f7138b73640 1 -- 172.21.15.133:0/3765148487 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c0547c0 con 0x7f712c03b570 2024-04-25T10:02:21.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f4d157fa640 1 -- 172.21.15.133:0/292457343 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d0c01f020 con 0x7f4d180ce8e0 2024-04-25T10:02:21.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f7138b73640 1 --2- 172.21.15.133:0/3765148487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f712c03b570 0x7f712c053c90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f712c055b40 tx=0x7f712401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f45ed3a729d3ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f712bfff640 1 -- 172.21.15.133:0/3765148487 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71240ab5f0 con 0x7f712c03b570 2024-04-25T10:02:21.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f712bfff640 1 -- 172.21.15.133:0/3765148487 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71240b0070 con 0x7f712c03b570 2024-04-25T10:02:21.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f713a376640 1 -- 172.21.15.133:0/3765148487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f712c03b570 msgr2=0x7f712c053c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f713a376640 1 --2- 172.21.15.133:0/3765148487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f712c03b570 0x7f712c053c90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f712c055b40 tx=0x7f712401c540 comp rx=0 tx=0).stop 2024-04-25T10:02:21.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f713a376640 1 -- 172.21.15.133:0/3765148487 shutdown_connections 2024-04-25T10:02:21.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f713a376640 1 --2- 172.21.15.133:0/3765148487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f712c050920 0x7f712c050db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f713a376640 1 --2- 172.21.15.133:0/3765148487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f712c03b570 0x7f712c053c90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f713a376640 1 --2- 172.21.15.133:0/3765148487 >> v2:172.21.15.162:3300/0 conn(0x7f712c0dfee0 0x7f712c055580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f713a376640 1 -- 172.21.15.133:0/3765148487 >> 172.21.15.133:0/3765148487 conn(0x7f712c055be0 msgr2=0x7f712c055ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f713a376640 1 -- 172.21.15.133:0/3765148487 shutdown_connections 2024-04-25T10:02:21.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.116+0000 7f713a376640 1 -- 172.21.15.133:0/3765148487 wait complete. 2024-04-25T10:02:21.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f713a376640 1 Processor -- start 2024-04-25T10:02:21.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f713a376640 1 -- start start 2024-04-25T10:02:21.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f713a376640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f712c03b570 0x7f712c0df130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f713a376640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f712c050920 0x7f712c0e1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f713a376640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f712c0dfee0 0x7f712c0d7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f713a376640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c008580 con 0x7f712c0dfee0 2024-04-25T10:02:21.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f713a376640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c008400 con 0x7f712c03b570 2024-04-25T10:02:21.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7139374640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f712c03b570 0x7f712c0df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7139b75640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f712c0dfee0 0x7f712c0d7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7139b75640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f712c0dfee0 0x7f712c0d7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48704/0 (socket says 172.21.15.133:48704) 2024-04-25T10:02:21.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7139b75640 1 -- 172.21.15.133:0/735598244 learned_addr learned my addr 172.21.15.133:0/735598244 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:21.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7138b73640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f712c050920 0x7f712c0e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f713a376640 1 -- 172.21.15.133:0/735598244 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c008700 con 0x7f712c050920 2024-04-25T10:02:21.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7139b75640 1 -- 172.21.15.133:0/735598244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f712c050920 msgr2=0x7f712c0e1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7139b75640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f712c050920 0x7f712c0e1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7139b75640 1 -- 172.21.15.133:0/735598244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f712c03b570 msgr2=0x7f712c0df130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7139b75640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f712c03b570 0x7f712c0df130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7139b75640 1 -- 172.21.15.133:0/735598244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c0de090 con 0x7f712c0dfee0 2024-04-25T10:02:21.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7139b75640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f712c0dfee0 0x7f712c0d7ce0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f71340751a0 tx=0x7f7134068040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7129ffb640 1 -- 172.21.15.133:0/735598244 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134097070 con 0x7f712c0dfee0 2024-04-25T10:02:21.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f7129ffb640 1 -- 172.21.15.133:0/735598244 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71340739e0 con 0x7f712c0dfee0 2024-04-25T10:02:21.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f1ab61ee640 1 Processor -- start 2024-04-25T10:02:21.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.120+0000 7f1ab61ee640 1 -- start start 2024-04-25T10:02:21.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f7129ffb640 1 -- 172.21.15.133:0/735598244 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134073dd0 con 0x7f712c0dfee0 2024-04-25T10:02:21.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f713a376640 1 -- 172.21.15.133:0/735598244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c0de370 con 0x7f712c0dfee0 2024-04-25T10:02:21.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f713a376640 1 -- 172.21.15.133:0/735598244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c0de930 con 0x7f712c0dfee0 2024-04-25T10:02:21.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab61ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1aa80501b0 0x7f1aa80df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab61ee640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1aa8033d00 0x7f1aa8006540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab61ee640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa8054120 con 0x7f1aa800aae0 2024-04-25T10:02:21.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab61ee640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa8054400 con 0x7f1aa8033d00 2024-04-25T10:02:21.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab61ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa80546e0 con 0x7f1aa80501b0 2024-04-25T10:02:21.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab51ec640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f1aa800aae0 0x7f1aa804faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:48128/0 (socket says 172.21.15.133:48128) 2024-04-25T10:02:21.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab51ec640 1 -- 172.21.15.133:0/952057663 learned_addr learned my addr 172.21.15.133:0/952057663 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:21.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab49eb640 1 --2- 172.21.15.133:0/952057663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1aa80501b0 0x7f1aa80df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab59ed640 1 --2- 172.21.15.133:0/952057663 >> v2:172.21.15.162:3300/0 conn(0x7f1aa8033d00 0x7f1aa8006540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f71177fe640 1 -- 172.21.15.133:0/735598244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71100052a0 con 0x7f712c0dfee0 2024-04-25T10:02:21.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1aa7fff640 1 -- 172.21.15.133:0/952057663 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639675726 0 0) 0x7f1aa8054120 con 0x7f1aa800aae0 2024-04-25T10:02:21.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1aa7fff640 1 -- 172.21.15.133:0/952057663 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a9c003530 con 0x7f1aa800aae0 2024-04-25T10:02:21.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab49eb640 1 -- 172.21.15.133:0/952057663 >> v2:172.21.15.162:3300/0 conn(0x7f1aa8033d00 msgr2=0x7f1aa8006540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab49eb640 1 --2- 172.21.15.133:0/952057663 >> v2:172.21.15.162:3300/0 conn(0x7f1aa8033d00 0x7f1aa8006540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab49eb640 1 -- 172.21.15.133:0/952057663 >> v1:172.21.15.162:6789/0 conn(0x7f1aa800aae0 legacy=0x7f1aa804faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f1ab49eb640 1 -- 172.21.15.133:0/952057663 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa8006020 con 0x7f1aa80501b0 2024-04-25T10:02:21.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f7129ffb640 1 -- 172.21.15.133:0/735598244 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f713409a070 con 0x7f712c0dfee0 2024-04-25T10:02:21.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f7129ffb640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7118066fb0 0x7f7118069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.124+0000 7f7139374640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7118066fb0 0x7f7118069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f7139374640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7118066fb0 0x7f7118069470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7130009a40 tx=0x7f7130009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f7129ffb640 1 -- 172.21.15.133:0/735598244 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f71340ff590 con 0x7f712c0dfee0 2024-04-25T10:02:21.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1ab49eb640 1 --2- 172.21.15.133:0/952057663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1aa80501b0 0x7f1aa80df2b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009dc0 tx=0x7f1aac01c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce78ce3b807773fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1aa7fff640 1 -- 172.21.15.133:0/952057663 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0b2070 con 0x7f1aa80501b0 2024-04-25T10:02:21.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1aa7fff640 1 -- 172.21.15.133:0/952057663 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac0a5710 con 0x7f1aa80501b0 2024-04-25T10:02:21.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1aa7fff640 1 -- 172.21.15.133:0/952057663 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0a5b00 con 0x7f1aa80501b0 2024-04-25T10:02:21.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1ab61ee640 1 -- 172.21.15.133:0/952057663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1aa80501b0 msgr2=0x7f1aa80df2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1ab61ee640 1 --2- 172.21.15.133:0/952057663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1aa80501b0 0x7f1aa80df2b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009dc0 tx=0x7f1aac01c850 comp rx=0 tx=0).stop 2024-04-25T10:02:21.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1ab61ee640 1 -- 172.21.15.133:0/952057663 shutdown_connections 2024-04-25T10:02:21.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1ab61ee640 1 --2- 172.21.15.133:0/952057663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1aa80501b0 0x7f1aa80df2b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1ab61ee640 1 --2- 172.21.15.133:0/952057663 >> v2:172.21.15.162:3300/0 conn(0x7f1aa8033d00 0x7f1aa8006540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1ab61ee640 1 -- 172.21.15.133:0/952057663 >> 172.21.15.133:0/952057663 conn(0x7f1aa8009fc0 msgr2=0x7f1aa800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1ab61ee640 1 -- 172.21.15.133:0/952057663 shutdown_connections 2024-04-25T10:02:21.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.128+0000 7f1ab61ee640 1 -- 172.21.15.133:0/952057663 wait complete. 2024-04-25T10:02:21.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab61ee640 1 Processor -- start 2024-04-25T10:02:21.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab61ee640 1 -- start start 2024-04-25T10:02:21.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab61ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1aa800aae0 0x7f1aa804e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab61ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1aa8033d00 0x7f1aa80d5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab61ee640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1aa80501b0 0x7f1aa80d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab61ee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa8007410 con 0x7f1aa8033d00 2024-04-25T10:02:21.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab61ee640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa8007290 con 0x7f1aa80501b0 2024-04-25T10:02:21.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab61ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa8007590 con 0x7f1aa800aae0 2024-04-25T10:02:21.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab49eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1aa8033d00 0x7f1aa80d5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab49eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1aa8033d00 0x7f1aa80d5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48710/0 (socket says 172.21.15.133:48710) 2024-04-25T10:02:21.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab59ed640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1aa80501b0 0x7f1aa80d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab49eb640 1 -- 172.21.15.133:0/1429651667 learned_addr learned my addr 172.21.15.133:0/1429651667 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:21.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab51ec640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1aa800aae0 0x7f1aa804e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab49eb640 1 -- 172.21.15.133:0/1429651667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1aa800aae0 msgr2=0x7f1aa804e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab49eb640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1aa800aae0 0x7f1aa804e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab49eb640 1 -- 172.21.15.133:0/1429651667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1aa80501b0 msgr2=0x7f1aa80d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab49eb640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1aa80501b0 0x7f1aa80d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab49eb640 1 -- 172.21.15.133:0/1429651667 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa804bdf0 con 0x7f1aa8033d00 2024-04-25T10:02:21.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.132+0000 7f1ab49eb640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1aa8033d00 0x7f1aa80d5a40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1aac01a480 tx=0x7f1aac0195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.136+0000 7f7129ffb640 1 -- 172.21.15.133:0/735598244 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f713408f070 con 0x7f712c0dfee0 2024-04-25T10:02:21.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.136+0000 7f1aa5ffb640 1 -- 172.21.15.133:0/1429651667 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0bb030 con 0x7f1aa8033d00 2024-04-25T10:02:21.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.136+0000 7f1ab61ee640 1 -- 172.21.15.133:0/1429651667 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa804c0d0 con 0x7f1aa8033d00 2024-04-25T10:02:21.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.136+0000 7f1ab61ee640 1 -- 172.21.15.133:0/1429651667 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa804c690 con 0x7f1aa8033d00 2024-04-25T10:02:21.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.136+0000 7f1ab61ee640 1 -- 172.21.15.133:0/1429651667 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa80089e0 con 0x7f1aa8033d00 2024-04-25T10:02:21.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.136+0000 7f1aa5ffb640 1 -- 172.21.15.133:0/1429651667 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac009a30 con 0x7f1aa8033d00 2024-04-25T10:02:21.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.136+0000 7f1aa5ffb640 1 -- 172.21.15.133:0/1429651667 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0b2040 con 0x7f1aa8033d00 2024-04-25T10:02:21.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.140+0000 7f1aa5ffb640 1 -- 172.21.15.133:0/1429651667 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1aac0be070 con 0x7f1aa8033d00 2024-04-25T10:02:21.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.140+0000 7f1aa5ffb640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a94066f90 0x7f1a94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.140+0000 7f1ab51ec640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a94066f90 0x7f1a94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.140+0000 7f1aa5ffb640 1 -- 172.21.15.133:0/1429651667 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1aac11fa20 con 0x7f1aa8033d00 2024-04-25T10:02:21.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.140+0000 7f1ab51ec640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a94066f90 0x7f1a94069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0007fd0 tx=0x7f1aa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.148+0000 7f1aa5ffb640 1 -- 172.21.15.133:0/1429651667 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1aac0c5070 con 0x7f1aa8033d00 2024-04-25T10:02:21.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.212+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/832909462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4b680b0fc0 con 0x7f4b68152a90 2024-04-25T10:02:21.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.212+0000 7f4b4f7fe640 1 -- 172.21.15.133:0/832909462 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b60016ff0 con 0x7f4b68152a90 2024-04-25T10:02:21.219 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.212+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/832909462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 msgr2=0x7f4b54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.216+0000 7f4b6d6bd640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 0x7f4b54069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009f50 tx=0x7f4b58009190 comp rx=0 tx=0).stop 2024-04-25T10:02:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.216+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/832909462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b68152a90 msgr2=0x7f4b680cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.216+0000 7f4b6d6bd640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b68152a90 0x7f4b680cce80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b600aa000 tx=0x7f4b50005840 comp rx=0 tx=0).stop 2024-04-25T10:02:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.216+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/832909462 shutdown_connections 2024-04-25T10:02:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.216+0000 7f4b6d6bd640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b68152a90 0x7f4b680cce80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.216+0000 7f4b6d6bd640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b680c8050 0x7f4b681ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.216+0000 7f4b6d6bd640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 0x7f4b54069420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.216+0000 7f4b6d6bd640 1 --2- 172.21.15.133:0/832909462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b500051d0 0x7f4b681fc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.216+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/832909462 >> 172.21.15.133:0/832909462 conn(0x7f4b680999d0 msgr2=0x7f4b680afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.216+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/832909462 shutdown_connections 2024-04-25T10:02:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.216+0000 7f4b6d6bd640 1 -- 172.21.15.133:0/832909462 wait complete. 2024-04-25T10:02:21.238 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-04-25T10:02:21.238 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.288+0000 7f8529ae3640 1 -- 172.21.15.133:0/4291964483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f852406fc80 con 0x7f852415e1b0 2024-04-25T10:02:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.288+0000 7f850bfff640 1 -- 172.21.15.133:0/4291964483 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f851c01aeb0 con 0x7f852415e1b0 2024-04-25T10:02:21.297 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:21.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 -- 172.21.15.133:0/4291964483 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8510066fb0 msgr2=0x7f8510069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8510066fb0 0x7f8510069470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f85241e73b0 tx=0x7f8518009190 comp rx=0 tx=0).stop 2024-04-25T10:02:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 -- 172.21.15.133:0/4291964483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f852415e1b0 msgr2=0x7f8524201220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f852415e1b0 0x7f8524201220 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f851c021c80 tx=0x7f851c01e180 comp rx=0 tx=0).stop 2024-04-25T10:02:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 -- 172.21.15.133:0/4291964483 shutdown_connections 2024-04-25T10:02:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8510066fb0 0x7f8510069470 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f852415e1b0 0x7f8524201220 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f852415ddd0 0x7f85241fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 --2- 172.21.15.133:0/4291964483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85240cc250 0x7f85241e7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 -- 172.21.15.133:0/4291964483 >> 172.21.15.133:0/4291964483 conn(0x7f85240999d0 msgr2=0x7f852406eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 -- 172.21.15.133:0/4291964483 shutdown_connections 2024-04-25T10:02:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.292+0000 7f8529ae3640 1 -- 172.21.15.133:0/4291964483 wait complete. 2024-04-25T10:02:21.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-25T10:02:21.314 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:21.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.324+0000 7f4cfeffd640 1 -- 172.21.15.133:0/292457343 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4cf0005860 con 0x7f4d180ce8e0 2024-04-25T10:02:21.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.324+0000 7f4d157fa640 1 -- 172.21.15.133:0/292457343 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4d0c00f2e0 con 0x7f4d180ce8e0 2024-04-25T10:02:21.333 INFO:teuthology.orchestra.run.smithi133.stdout:55834574860 2024-04-25T10:02:21.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 -- 172.21.15.133:0/292457343 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d04066f90 msgr2=0x7f4d04069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d04066f90 0x7f4d04069450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000e250 tx=0x7f4d1000b040 comp rx=0 tx=0).stop 2024-04-25T10:02:21.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 -- 172.21.15.133:0/292457343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d180ce8e0 msgr2=0x7f4d1815c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d180ce8e0 0x7f4d1815c2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00b410 tx=0x7f4d0c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 -- 172.21.15.133:0/292457343 shutdown_connections 2024-04-25T10:02:21.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d180ce8e0 0x7f4d1815c2a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d180ce450 0x7f4d180b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d04066f90 0x7f4d04069450 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 --2- 172.21.15.133:0/292457343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1806e970 0x7f4d18154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 -- 172.21.15.133:0/292457343 >> 172.21.15.133:0/292457343 conn(0x7f4d18066f70 msgr2=0x7f4d1806d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 -- 172.21.15.133:0/292457343 shutdown_connections 2024-04-25T10:02:21.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.328+0000 7f4cfeffd640 1 -- 172.21.15.133:0/292457343 wait complete. 2024-04-25T10:02:21.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.0 2024-04-25T10:02:21.350 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:21.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.368+0000 7f1865c89640 1 -- 172.21.15.133:0/1692340739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1840005590 con 0x7f18600d1d90 2024-04-25T10:02:21.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.368+0000 7f1847fff640 1 -- 172.21.15.133:0/1692340739 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f18540f2360 con 0x7f18600d1d90 2024-04-25T10:02:21.376 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 -- 172.21.15.133:0/1692340739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f184c066fb0 msgr2=0x7f184c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f184c066fb0 0x7f184c069470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1850000f70 tx=0x7f1850007e90 comp rx=0 tx=0).stop 2024-04-25T10:02:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 -- 172.21.15.133:0/1692340739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18600d1d90 msgr2=0x7f18600c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18600d1d90 0x7f18600c88c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f18540b3040 tx=0x7f185401ba30 comp rx=0 tx=0).stop 2024-04-25T10:02:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 -- 172.21.15.133:0/1692340739 shutdown_connections 2024-04-25T10:02:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f186015d6d0 0x7f18600b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f184c066fb0 0x7f184c069470 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18600d1d90 0x7f18600c88c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 --2- 172.21.15.133:0/1692340739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18600b9590 0x7f18601f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 -- 172.21.15.133:0/1692340739 >> 172.21.15.133:0/1692340739 conn(0x7f18600727c0 msgr2=0x7f18600cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 -- 172.21.15.133:0/1692340739 shutdown_connections 2024-04-25T10:02:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.372+0000 7f1865c89640 1 -- 172.21.15.133:0/1692340739 wait complete. 2024-04-25T10:02:21.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-04-25T10:02:21.393 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:21.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.412+0000 7fb69b56b640 1 -- 172.21.15.133:0/1138273332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb6940b1830 con 0x7fb6940c4d80 2024-04-25T10:02:21.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.412+0000 7fb689ffb640 1 -- 172.21.15.133:0/1138273332 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6840eede0 con 0x7fb6940c4d80 2024-04-25T10:02:21.419 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:21.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 -- 172.21.15.133:0/1138273332 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6800673d0 msgr2=0x7fb680069890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6800673d0 0x7fb680069890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb69415e3b0 tx=0x7fb690001720 comp rx=0 tx=0).stop 2024-04-25T10:02:21.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 -- 172.21.15.133:0/1138273332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6940c4d80 msgr2=0x7fb694164ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6940c4d80 0x7fb694164ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6840ae040 tx=0x7fb68401abd0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 -- 172.21.15.133:0/1138273332 shutdown_connections 2024-04-25T10:02:21.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb694166410 0x7fb694162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6940d0c50 0x7fb694160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6800673d0 0x7fb680069890 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 --2- 172.21.15.133:0/1138273332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6940c4d80 0x7fb694164ea0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 -- 172.21.15.133:0/1138273332 >> 172.21.15.133:0/1138273332 conn(0x7fb6940d16a0 msgr2=0x7fb6940b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 -- 172.21.15.133:0/1138273332 shutdown_connections 2024-04-25T10:02:21.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7fb69b56b640 1 -- 172.21.15.133:0/1138273332 wait complete. 2024-04-25T10:02:21.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7f1ab61ee640 1 -- 172.21.15.133:0/1429651667 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1aa8052cf0 con 0x7f1aa8033d00 2024-04-25T10:02:21.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.416+0000 7f1aa5ffb640 1 -- 172.21.15.133:0/1429651667 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1aac016560 con 0x7f1aa8033d00 2024-04-25T10:02:21.424 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 -- 172.21.15.133:0/1429651667 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a94066f90 msgr2=0x7f1a94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a94066f90 0x7f1a94069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0007fd0 tx=0x7f1aa0009040 comp rx=0 tx=0).stop 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 -- 172.21.15.133:0/1429651667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1aa8033d00 msgr2=0x7f1aa80d5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1aa8033d00 0x7f1aa80d5a40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1aac01a480 tx=0x7f1aac0195f0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 -- 172.21.15.133:0/1429651667 shutdown_connections 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1aa80501b0 0x7f1aa80d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a94066f90 0x7f1a94069450 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1aa8033d00 0x7f1aa80d5a40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 --2- 172.21.15.133:0/1429651667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1aa800aae0 0x7f1aa804e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 -- 172.21.15.133:0/1429651667 >> 172.21.15.133:0/1429651667 conn(0x7f1aa8009fc0 msgr2=0x7f1aa8004a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 -- 172.21.15.133:0/1429651667 shutdown_connections 2024-04-25T10:02:21.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.420+0000 7f1ab61ee640 1 -- 172.21.15.133:0/1429651667 wait complete. 2024-04-25T10:02:21.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-25T10:02:21.442 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:21.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-04-25T10:02:21.447 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.444+0000 7f71177fe640 1 -- 172.21.15.133:0/735598244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7110005590 con 0x7f712c0dfee0 2024-04-25T10:02:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.444+0000 7f7129ffb640 1 -- 172.21.15.133:0/735598244 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f713406ce20 con 0x7f712c0dfee0 2024-04-25T10:02:21.452 INFO:teuthology.orchestra.run.smithi133.stdout:55834574859 2024-04-25T10:02:21.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 -- 172.21.15.133:0/735598244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7118066fb0 msgr2=0x7f7118069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7118066fb0 0x7f7118069470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7130009a40 tx=0x7f7130009190 comp rx=0 tx=0).stop 2024-04-25T10:02:21.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 -- 172.21.15.133:0/735598244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f712c0dfee0 msgr2=0x7f712c0d7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f712c0dfee0 0x7f712c0d7ce0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f71340751a0 tx=0x7f7134068040 comp rx=0 tx=0).stop 2024-04-25T10:02:21.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 -- 172.21.15.133:0/735598244 shutdown_connections 2024-04-25T10:02:21.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7118066fb0 0x7f7118069470 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f712c0dfee0 0x7f712c0d7ce0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f712c050920 0x7f712c0e1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 --2- 172.21.15.133:0/735598244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f712c03b570 0x7f712c0df130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 -- 172.21.15.133:0/735598244 >> 172.21.15.133:0/735598244 conn(0x7f712c055be0 msgr2=0x7f712c008a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 -- 172.21.15.133:0/735598244 shutdown_connections 2024-04-25T10:02:21.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.448+0000 7f71177fe640 1 -- 172.21.15.133:0/735598244 wait complete. 2024-04-25T10:02:21.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-04-25T10:02:21.469 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:21.469 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:02:21.469 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:21.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.612+0000 7f6a69cbb640 1 Processor -- start 2024-04-25T10:02:21.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.612+0000 7f6a69cbb640 1 -- start start 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a69cbb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6a640cc1f0 0x7f6a64066df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a69cbb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a640cba80 con 0x7f6a6406fcc0 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a69cbb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a640cb7a0 con 0x7f6a6415bb00 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a69cbb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a640cc660 con 0x7f6a640cc1f0 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a62ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6a6406fcc0 0x7f6a6406b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52072/0 (socket says 172.21.15.133:52072) 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a62ffd640 1 -- 172.21.15.133:0/1763447947 learned_addr learned my addr 172.21.15.133:0/1763447947 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353551556 0 0) 0x7f6a640cba80 con 0x7f6a6406fcc0 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a63fff640 1 --2- 172.21.15.133:0/1763447947 >> v2:172.21.15.162:3300/0 conn(0x7f6a640cc1f0 0x7f6a64066df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a4c003530 con 0x7f6a6406fcc0 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3337317882 0 0) 0x7f6a4c003530 con 0x7f6a6406fcc0 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 >> v2:172.21.15.162:3300/0 conn(0x7f6a640cc1f0 msgr2=0x7f6a64066df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 --2- 172.21.15.133:0/1763447947 >> v2:172.21.15.162:3300/0 conn(0x7f6a640cc1f0 0x7f6a64066df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 >> v1:172.21.15.162:6789/0 conn(0x7f6a6415bb00 legacy=0x7f6a6406f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a640b5990 con 0x7f6a6406fcc0 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6a58001680 con 0x7f6a6406fcc0 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 >> v1:172.21.15.133:6789/0 conn(0x7f6a6406fcc0 legacy=0x7f6a6406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 --2- 172.21.15.133:0/1763447947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 0x7f6a4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 --2- 172.21.15.133:0/1763447947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a4c004810 0x7f6a4c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 --2- 172.21.15.133:0/1763447947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a4c0052f0 0x7f6a4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640cc660 con 0x7f6a4c0039c0 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c00c1f0 con 0x7f6a4c004810 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c00c3f0 con 0x7f6a4c0052f0 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a637fe640 1 --2- 172.21.15.133:0/1763447947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 0x7f6a4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a62ffd640 1 --2- 172.21.15.133:0/1763447947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a4c004810 0x7f6a4c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a63fff640 1 --2- 172.21.15.133:0/1763447947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a4c0052f0 0x7f6a4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a637fe640 1 -- 172.21.15.133:0/1763447947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a4c0052f0 msgr2=0x7f6a4c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a637fe640 1 --2- 172.21.15.133:0/1763447947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a4c0052f0 0x7f6a4c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a637fe640 1 -- 172.21.15.133:0/1763447947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a4c004810 msgr2=0x7f6a4c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a637fe640 1 --2- 172.21.15.133:0/1763447947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a4c004810 0x7f6a4c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a637fe640 1 -- 172.21.15.133:0/1763447947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c003530 con 0x7f6a4c0039c0 2024-04-25T10:02:21.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a63fff640 1 -- 172.21.15.133:0/1763447947 reap_dead start 2024-04-25T10:02:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.616+0000 7f6a637fe640 1 --2- 172.21.15.133:0/1763447947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 0x7f6a4c003e00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a54001750 tx=0x7f6a5401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdeed83e3fb59fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a540013a0 con 0x7f6a4c0039c0 2024-04-25T10:02:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a540052f0 con 0x7f6a4c0039c0 2024-04-25T10:02:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a627fc640 1 -- 172.21.15.133:0/1763447947 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54003840 con 0x7f6a4c0039c0 2024-04-25T10:02:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1763447947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 msgr2=0x7f6a4c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 --2- 172.21.15.133:0/1763447947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 0x7f6a4c003e00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a54001750 tx=0x7f6a5401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1763447947 shutdown_connections 2024-04-25T10:02:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 --2- 172.21.15.133:0/1763447947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 0x7f6a4c003e00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1763447947 >> 172.21.15.133:0/1763447947 conn(0x7f6a640cc8e0 msgr2=0x7f6a640cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1763447947 shutdown_connections 2024-04-25T10:02:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1763447947 wait complete. 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 Processor -- start 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 -- start start 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 0x7f6a64157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64158d60 0x7f6a641591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a640c9600 0x7f6a640c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6406c240 con 0x7f6a4c0039c0 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6406c0c0 con 0x7f6a640c9600 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a64067200 con 0x7f6a64158d60 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a637fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 0x7f6a64157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a637fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 0x7f6a64157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48724/0 (socket says 172.21.15.133:48724) 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a637fe640 1 -- 172.21.15.133:0/1264074101 learned_addr learned my addr 172.21.15.133:0/1264074101 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a62ffd640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64158d60 0x7f6a641591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a63fff640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a640c9600 0x7f6a640c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a637fe640 1 -- 172.21.15.133:0/1264074101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64158d60 msgr2=0x7f6a641591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a637fe640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64158d60 0x7f6a641591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a637fe640 1 -- 172.21.15.133:0/1264074101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a640c9600 msgr2=0x7f6a640c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a637fe640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a640c9600 0x7f6a640c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a637fe640 1 -- 172.21.15.133:0/1264074101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a640c8170 con 0x7f6a4c0039c0 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a63fff640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a640c9600 0x7f6a640c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a637fe640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 0x7f6a64157c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a5401c1f0 tx=0x7f6a4c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a4bfff640 1 -- 172.21.15.133:0/1264074101 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a540b8070 con 0x7f6a4c0039c0 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a4bfff640 1 -- 172.21.15.133:0/1264074101 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a540054a0 con 0x7f6a4c0039c0 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1264074101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a640c9fc0 con 0x7f6a4c0039c0 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a4bfff640 1 -- 172.21.15.133:0/1264074101 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54007a40 con 0x7f6a4c0039c0 2024-04-25T10:02:21.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.620+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1264074101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a640c73b0 con 0x7f6a4c0039c0 2024-04-25T10:02:21.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.624+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1264074101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a64067c70 con 0x7f6a4c0039c0 2024-04-25T10:02:21.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.624+0000 7f6a4bfff640 1 -- 172.21.15.133:0/1264074101 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a54007260 con 0x7f6a4c0039c0 2024-04-25T10:02:21.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.624+0000 7f6a4bfff640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a50066f60 0x7f6a50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.624+0000 7f6a4bfff640 1 -- 172.21.15.133:0/1264074101 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6a5411f430 con 0x7f6a4c0039c0 2024-04-25T10:02:21.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.624+0000 7f6a62ffd640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a50066f60 0x7f6a50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.624+0000 7f6a62ffd640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a50066f60 0x7f6a50069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6a58001ec0 tx=0x7f6a58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.628+0000 7f6a4bfff640 1 -- 172.21.15.133:0/1264074101 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a540160d0 con 0x7f6a4c0039c0 2024-04-25T10:02:21.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.772+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1264074101 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a640b4690 con 0x7f6a50066f60 2024-04-25T10:02:21.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.780+0000 7f6a4bfff640 1 -- 172.21.15.133:0/1264074101 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7f6a640b4690 con 0x7f6a50066f60 2024-04-25T10:02:21.790 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:21.792 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:02:21.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1264074101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a50066f60 msgr2=0x7f6a50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a50066f60 0x7f6a50069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6a58001ec0 tx=0x7f6a58009040 comp rx=0 tx=0).stop 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1264074101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 msgr2=0x7f6a64157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 0x7f6a64157c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a5401c1f0 tx=0x7f6a4c005840 comp rx=0 tx=0).stop 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1264074101 shutdown_connections 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a640c9600 0x7f6a640c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64158d60 0x7f6a641591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a50066f60 0x7f6a50069420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 --2- 172.21.15.133:0/1264074101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a4c0039c0 0x7f6a64157c80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1264074101 >> 172.21.15.133:0/1264074101 conn(0x7f6a640cc8e0 msgr2=0x7f6a640b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1264074101 shutdown_connections 2024-04-25T10:02:21.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.784+0000 7f6a69cbb640 1 -- 172.21.15.133:0/1264074101 wait complete. 2024-04-25T10:02:21.807 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T10:02:21.712799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219388,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755332,"statfs":{"total":773094113280,"available":772869459968,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004689"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631430+0000","last_change":"2024-04-25T10:01:50.953534+0000","last_active":"2024-04-25T10:01:57.631430+0000","last_peered":"2024-04-25T10:01:57.631430+0000","last_clean":"2024-04-25T10:01:57.631430+0000","last_became_active":"2024-04-25T10:01:50.953054+0000","last_became_peered":"2024-04-25T10:01:50.953054+0000","last_unstale":"2024-04-25T10:01:57.631430+0000","last_undegraded":"2024-04-25T10:01:57.631430+0000","last_fullsized":"2024-04-25T10:01:57.631430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:19.695230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633973+0000","last_change":"2024-04-25T10:01:51.958018+0000","last_active":"2024-04-25T10:01:57.633973+0000","last_peered":"2024-04-25T10:01:57.633973+0000","last_clean":"2024-04-25T10:01:57.633973+0000","last_became_active":"2024-04-25T10:01:51.957717+0000","last_became_peered":"2024-04-25T10:01:51.957717+0000","last_unstale":"2024-04-25T10:01:57.633973+0000","last_undegraded":"2024-04-25T10:01:57.633973+0000","last_fullsized":"2024-04-25T10:01:57.633973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:55.810277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631489+0000","last_change":"2024-04-25T10:01:56.111100+0000","last_active":"2024-04-25T10:01:57.631489+0000","last_peered":"2024-04-25T10:01:57.631489+0000","last_clean":"2024-04-25T10:01:57.631489+0000","last_became_active":"2024-04-25T10:01:56.110327+0000","last_became_peered":"2024-04-25T10:01:56.110327+0000","last_unstale":"2024-04-25T10:01:57.631489+0000","last_undegraded":"2024-04-25T10:01:57.631489+0000","last_fullsized":"2024-04-25T10:01:57.631489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634369+0000","last_change":"2024-04-25T10:01:50.940954+0000","last_active":"2024-04-25T10:01:57.634369+0000","last_peered":"2024-04-25T10:01:57.634369+0000","last_clean":"2024-04-25T10:01:57.634369+0000","last_became_active":"2024-04-25T10:01:50.940601+0000","last_became_peered":"2024-04-25T10:01:50.940601+0000","last_unstale":"2024-04-25T10:01:57.634369+0000","last_undegraded":"2024-04-25T10:01:57.634369+0000","last_fullsized":"2024-04-25T10:01:57.634369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631026+0000","last_change":"2024-04-25T10:01:51.994632+0000","last_active":"2024-04-25T10:01:57.631026+0000","last_peered":"2024-04-25T10:01:57.631026+0000","last_clean":"2024-04-25T10:01:57.631026+0000","last_became_active":"2024-04-25T10:01:51.994229+0000","last_became_peered":"2024-04-25T10:01:51.994229+0000","last_unstale":"2024-04-25T10:01:57.631026+0000","last_undegraded":"2024-04-25T10:01:57.631026+0000","last_fullsized":"2024-04-25T10:01:57.631026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005703+0000","last_change":"2024-04-25T10:01:56.024604+0000","last_active":"2024-04-25T10:01:57.005703+0000","last_peered":"2024-04-25T10:01:57.005703+0000","last_clean":"2024-04-25T10:01:57.005703+0000","last_became_active":"2024-04-25T10:01:56.023667+0000","last_became_peered":"2024-04-25T10:01:56.023667+0000","last_unstale":"2024-04-25T10:01:57.005703+0000","last_undegraded":"2024-04-25T10:01:57.005703+0000","last_fullsized":"2024-04-25T10:01:57.005703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:50.018687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348063+0000","last_change":"2024-04-25T10:01:50.957056+0000","last_active":"2024-04-25T10:01:57.348063+0000","last_peered":"2024-04-25T10:01:57.348063+0000","last_clean":"2024-04-25T10:01:57.348063+0000","last_became_active":"2024-04-25T10:01:50.956522+0000","last_became_peered":"2024-04-25T10:01:50.956522+0000","last_unstale":"2024-04-25T10:01:57.348063+0000","last_undegraded":"2024-04-25T10:01:57.348063+0000","last_fullsized":"2024-04-25T10:01:57.348063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:25.085344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101183+0000","last_change":"2024-04-25T10:01:51.987821+0000","last_active":"2024-04-25T10:01:57.101183+0000","last_peered":"2024-04-25T10:01:57.101183+0000","last_clean":"2024-04-25T10:01:57.101183+0000","last_became_active":"2024-04-25T10:01:51.978435+0000","last_became_peered":"2024-04-25T10:01:51.978435+0000","last_unstale":"2024-04-25T10:01:57.101183+0000","last_undegraded":"2024-04-25T10:01:57.101183+0000","last_fullsized":"2024-04-25T10:01:57.101183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.714429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009007+0000","last_change":"2024-04-25T10:01:56.024538+0000","last_active":"2024-04-25T10:01:57.009007+0000","last_peered":"2024-04-25T10:01:57.009007+0000","last_clean":"2024-04-25T10:01:57.009007+0000","last_became_active":"2024-04-25T10:01:56.023560+0000","last_became_peered":"2024-04-25T10:01:56.023560+0000","last_unstale":"2024-04-25T10:01:57.009007+0000","last_undegraded":"2024-04-25T10:01:57.009007+0000","last_fullsized":"2024-04-25T10:01:57.009007+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.432097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347087+0000","last_change":"2024-04-25T10:01:50.954944+0000","last_active":"2024-04-25T10:01:57.347087+0000","last_peered":"2024-04-25T10:01:57.347087+0000","last_clean":"2024-04-25T10:01:57.347087+0000","last_became_active":"2024-04-25T10:01:50.954098+0000","last_became_peered":"2024-04-25T10:01:50.954098+0000","last_unstale":"2024-04-25T10:01:57.347087+0000","last_undegraded":"2024-04-25T10:01:57.347087+0000","last_fullsized":"2024-04-25T10:01:57.347087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:51.011075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101791+0000","last_change":"2024-04-25T10:01:51.976877+0000","last_active":"2024-04-25T10:01:57.101791+0000","last_peered":"2024-04-25T10:01:57.101791+0000","last_clean":"2024-04-25T10:01:57.101791+0000","last_became_active":"2024-04-25T10:01:51.967649+0000","last_became_peered":"2024-04-25T10:01:51.967649+0000","last_unstale":"2024-04-25T10:01:57.101791+0000","last_undegraded":"2024-04-25T10:01:57.101791+0000","last_fullsized":"2024-04-25T10:01:57.101791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006978+0000","last_change":"2024-04-25T10:01:56.018179+0000","last_active":"2024-04-25T10:01:57.006978+0000","last_peered":"2024-04-25T10:01:57.006978+0000","last_clean":"2024-04-25T10:01:57.006978+0000","last_became_active":"2024-04-25T10:01:56.017199+0000","last_became_peered":"2024-04-25T10:01:56.017199+0000","last_unstale":"2024-04-25T10:01:57.006978+0000","last_undegraded":"2024-04-25T10:01:57.006978+0000","last_fullsized":"2024-04-25T10:01:57.006978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:26.814142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007115+0000","last_change":"2024-04-25T10:01:51.935248+0000","last_active":"2024-04-25T10:01:57.007115+0000","last_peered":"2024-04-25T10:01:57.007115+0000","last_clean":"2024-04-25T10:01:57.007115+0000","last_became_active":"2024-04-25T10:01:51.932825+0000","last_became_peered":"2024-04-25T10:01:51.932825+0000","last_unstale":"2024-04-25T10:01:57.007115+0000","last_undegraded":"2024-04-25T10:01:57.007115+0000","last_fullsized":"2024-04-25T10:01:57.007115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347924+0000","last_change":"2024-04-25T10:01:50.956325+0000","last_active":"2024-04-25T10:01:57.347924+0000","last_peered":"2024-04-25T10:01:57.347924+0000","last_clean":"2024-04-25T10:01:57.347924+0000","last_became_active":"2024-04-25T10:01:50.954154+0000","last_became_peered":"2024-04-25T10:01:50.954154+0000","last_unstale":"2024-04-25T10:01:57.347924+0000","last_undegraded":"2024-04-25T10:01:57.347924+0000","last_fullsized":"2024-04-25T10:01:57.347924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:32.182247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008038+0000","last_change":"2024-04-25T10:01:56.024877+0000","last_active":"2024-04-25T10:01:57.008038+0000","last_peered":"2024-04-25T10:01:57.008038+0000","last_clean":"2024-04-25T10:01:57.008038+0000","last_became_active":"2024-04-25T10:01:56.023788+0000","last_became_peered":"2024-04-25T10:01:56.023788+0000","last_unstale":"2024-04-25T10:01:57.008038+0000","last_undegraded":"2024-04-25T10:01:57.008038+0000","last_fullsized":"2024-04-25T10:01:57.008038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:52.776865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633155+0000","last_change":"2024-04-25T10:01:51.952977+0000","last_active":"2024-04-25T10:01:57.633155+0000","last_peered":"2024-04-25T10:01:57.633155+0000","last_clean":"2024-04-25T10:01:57.633155+0000","last_became_active":"2024-04-25T10:01:51.951977+0000","last_became_peered":"2024-04-25T10:01:51.951977+0000","last_unstale":"2024-04-25T10:01:57.633155+0000","last_undegraded":"2024-04-25T10:01:57.633155+0000","last_fullsized":"2024-04-25T10:01:57.633155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.714429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630222+0000","last_change":"2024-04-25T10:01:50.934823+0000","last_active":"2024-04-25T10:01:57.630222+0000","last_peered":"2024-04-25T10:01:57.630222+0000","last_clean":"2024-04-25T10:01:57.630222+0000","last_became_active":"2024-04-25T10:01:50.934535+0000","last_became_peered":"2024-04-25T10:01:50.934535+0000","last_unstale":"2024-04-25T10:01:57.630222+0000","last_undegraded":"2024-04-25T10:01:57.630222+0000","last_fullsized":"2024-04-25T10:01:57.630222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007823+0000","last_change":"2024-04-25T10:01:56.017858+0000","last_active":"2024-04-25T10:01:57.007823+0000","last_peered":"2024-04-25T10:01:57.007823+0000","last_clean":"2024-04-25T10:01:57.007823+0000","last_became_active":"2024-04-25T10:01:56.016904+0000","last_became_peered":"2024-04-25T10:01:56.016904+0000","last_unstale":"2024-04-25T10:01:57.007823+0000","last_undegraded":"2024-04-25T10:01:57.007823+0000","last_fullsized":"2024-04-25T10:01:57.007823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:38.512242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631872+0000","last_change":"2024-04-25T10:01:51.948751+0000","last_active":"2024-04-25T10:01:57.631872+0000","last_peered":"2024-04-25T10:01:57.631872+0000","last_clean":"2024-04-25T10:01:57.631872+0000","last_became_active":"2024-04-25T10:01:51.946530+0000","last_became_peered":"2024-04-25T10:01:51.946530+0000","last_unstale":"2024-04-25T10:01:57.631872+0000","last_undegraded":"2024-04-25T10:01:57.631872+0000","last_fullsized":"2024-04-25T10:01:57.631872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627984+0000","last_change":"2024-04-25T10:01:50.953620+0000","last_active":"2024-04-25T10:01:57.627984+0000","last_peered":"2024-04-25T10:01:57.627984+0000","last_clean":"2024-04-25T10:01:57.627984+0000","last_became_active":"2024-04-25T10:01:50.953363+0000","last_became_peered":"2024-04-25T10:01:50.953363+0000","last_unstale":"2024-04-25T10:01:57.627984+0000","last_undegraded":"2024-04-25T10:01:57.627984+0000","last_fullsized":"2024-04-25T10:01:57.627984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629574+0000","last_change":"2024-04-25T10:01:56.111935+0000","last_active":"2024-04-25T10:01:57.629574+0000","last_peered":"2024-04-25T10:01:57.629574+0000","last_clean":"2024-04-25T10:01:57.629574+0000","last_became_active":"2024-04-25T10:01:56.111568+0000","last_became_peered":"2024-04-25T10:01:56.111568+0000","last_unstale":"2024-04-25T10:01:57.629574+0000","last_undegraded":"2024-04-25T10:01:57.629574+0000","last_fullsized":"2024-04-25T10:01:57.629574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:18.254251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006627+0000","last_change":"2024-04-25T10:01:51.965350+0000","last_active":"2024-04-25T10:01:57.006627+0000","last_peered":"2024-04-25T10:01:57.006627+0000","last_clean":"2024-04-25T10:01:57.006627+0000","last_became_active":"2024-04-25T10:01:51.964456+0000","last_became_peered":"2024-04-25T10:01:51.964456+0000","last_unstale":"2024-04-25T10:01:57.006627+0000","last_undegraded":"2024-04-25T10:01:57.006627+0000","last_fullsized":"2024-04-25T10:01:57.006627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:59.032520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346651+0000","last_change":"2024-04-25T10:01:50.954332+0000","last_active":"2024-04-25T10:01:57.346651+0000","last_peered":"2024-04-25T10:01:57.346651+0000","last_clean":"2024-04-25T10:01:57.346651+0000","last_became_active":"2024-04-25T10:01:50.953854+0000","last_became_peered":"2024-04-25T10:01:50.953854+0000","last_unstale":"2024-04-25T10:01:57.346651+0000","last_undegraded":"2024-04-25T10:01:57.346651+0000","last_fullsized":"2024-04-25T10:01:57.346651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:38.278178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008718+0000","last_change":"2024-04-25T10:01:56.017972+0000","last_active":"2024-04-25T10:01:57.008718+0000","last_peered":"2024-04-25T10:01:57.008718+0000","last_clean":"2024-04-25T10:01:57.008718+0000","last_became_active":"2024-04-25T10:01:56.016953+0000","last_became_peered":"2024-04-25T10:01:56.016953+0000","last_unstale":"2024-04-25T10:01:57.008718+0000","last_undegraded":"2024-04-25T10:01:57.008718+0000","last_fullsized":"2024-04-25T10:01:57.008718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:59.872712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630912+0000","last_change":"2024-04-25T10:01:51.944937+0000","last_active":"2024-04-25T10:01:57.630912+0000","last_peered":"2024-04-25T10:01:57.630912+0000","last_clean":"2024-04-25T10:01:57.630912+0000","last_became_active":"2024-04-25T10:01:51.944599+0000","last_became_peered":"2024-04-25T10:01:51.944599+0000","last_unstale":"2024-04-25T10:01:57.630912+0000","last_undegraded":"2024-04-25T10:01:57.630912+0000","last_fullsized":"2024-04-25T10:01:57.630912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008241+0000","last_change":"2024-04-25T10:01:50.955661+0000","last_active":"2024-04-25T10:01:57.008241+0000","last_peered":"2024-04-25T10:01:57.008241+0000","last_clean":"2024-04-25T10:01:57.008241+0000","last_became_active":"2024-04-25T10:01:50.954955+0000","last_became_peered":"2024-04-25T10:01:50.954955+0000","last_unstale":"2024-04-25T10:01:57.008241+0000","last_undegraded":"2024-04-25T10:01:57.008241+0000","last_fullsized":"2024-04-25T10:01:57.008241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:33.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006345+0000","last_change":"2024-04-25T10:01:56.108996+0000","last_active":"2024-04-25T10:01:57.006345+0000","last_peered":"2024-04-25T10:01:57.006345+0000","last_clean":"2024-04-25T10:01:57.006345+0000","last_became_active":"2024-04-25T10:01:56.107964+0000","last_became_peered":"2024-04-25T10:01:56.107964+0000","last_unstale":"2024-04-25T10:01:57.006345+0000","last_undegraded":"2024-04-25T10:01:57.006345+0000","last_fullsized":"2024-04-25T10:01:57.006345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.917422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634815+0000","last_change":"2024-04-25T10:01:51.959144+0000","last_active":"2024-04-25T10:01:57.634815+0000","last_peered":"2024-04-25T10:01:57.634815+0000","last_clean":"2024-04-25T10:01:57.634815+0000","last_became_active":"2024-04-25T10:01:51.958032+0000","last_became_peered":"2024-04-25T10:01:51.958032+0000","last_unstale":"2024-04-25T10:01:57.634815+0000","last_undegraded":"2024-04-25T10:01:57.634815+0000","last_fullsized":"2024-04-25T10:01:57.634815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:34.449807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629847+0000","last_change":"2024-04-25T10:01:50.951871+0000","last_active":"2024-04-25T10:01:57.629847+0000","last_peered":"2024-04-25T10:01:57.629847+0000","last_clean":"2024-04-25T10:01:57.629847+0000","last_became_active":"2024-04-25T10:01:50.951527+0000","last_became_peered":"2024-04-25T10:01:50.951527+0000","last_unstale":"2024-04-25T10:01:57.629847+0000","last_undegraded":"2024-04-25T10:01:57.629847+0000","last_fullsized":"2024-04-25T10:01:57.629847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:38.278178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630769+0000","last_change":"2024-04-25T10:01:56.101023+0000","last_active":"2024-04-25T10:01:57.630769+0000","last_peered":"2024-04-25T10:01:57.630769+0000","last_clean":"2024-04-25T10:01:57.630769+0000","last_became_active":"2024-04-25T10:01:56.100400+0000","last_became_peered":"2024-04-25T10:01:56.100400+0000","last_unstale":"2024-04-25T10:01:57.630769+0000","last_undegraded":"2024-04-25T10:01:57.630769+0000","last_fullsized":"2024-04-25T10:01:57.630769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:40.633384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628600+0000","last_change":"2024-04-25T10:01:51.951713+0000","last_active":"2024-04-25T10:01:57.628600+0000","last_peered":"2024-04-25T10:01:57.628600+0000","last_clean":"2024-04-25T10:01:57.628600+0000","last_became_active":"2024-04-25T10:01:51.951438+0000","last_became_peered":"2024-04-25T10:01:51.951438+0000","last_unstale":"2024-04-25T10:01:57.628600+0000","last_undegraded":"2024-04-25T10:01:57.628600+0000","last_fullsized":"2024-04-25T10:01:57.628600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348330+0000","last_change":"2024-04-25T10:01:50.956823+0000","last_active":"2024-04-25T10:01:57.348330+0000","last_peered":"2024-04-25T10:01:57.348330+0000","last_clean":"2024-04-25T10:01:57.348330+0000","last_became_active":"2024-04-25T10:01:50.956170+0000","last_became_peered":"2024-04-25T10:01:50.956170+0000","last_unstale":"2024-04-25T10:01:57.348330+0000","last_undegraded":"2024-04-25T10:01:57.348330+0000","last_fullsized":"2024-04-25T10:01:57.348330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:43.784003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629539+0000","last_change":"2024-04-25T10:01:56.101938+0000","last_active":"2024-04-25T10:01:57.629539+0000","last_peered":"2024-04-25T10:01:57.629539+0000","last_clean":"2024-04-25T10:01:57.629539+0000","last_became_active":"2024-04-25T10:01:56.101501+0000","last_became_peered":"2024-04-25T10:01:56.101501+0000","last_unstale":"2024-04-25T10:01:57.629539+0000","last_undegraded":"2024-04-25T10:01:57.629539+0000","last_fullsized":"2024-04-25T10:01:57.629539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:03.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.102065+0000","last_change":"2024-04-25T10:01:51.977453+0000","last_active":"2024-04-25T10:01:57.102065+0000","last_peered":"2024-04-25T10:01:57.102065+0000","last_clean":"2024-04-25T10:01:57.102065+0000","last_became_active":"2024-04-25T10:01:51.968804+0000","last_became_peered":"2024-04-25T10:01:51.968804+0000","last_unstale":"2024-04-25T10:01:57.102065+0000","last_undegraded":"2024-04-25T10:01:57.102065+0000","last_fullsized":"2024-04-25T10:01:57.102065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:34.449807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629477+0000","last_change":"2024-04-25T10:01:50.934637+0000","last_active":"2024-04-25T10:01:57.629477+0000","last_peered":"2024-04-25T10:01:57.629477+0000","last_clean":"2024-04-25T10:01:57.629477+0000","last_became_active":"2024-04-25T10:01:50.933810+0000","last_became_peered":"2024-04-25T10:01:50.933810+0000","last_unstale":"2024-04-25T10:01:57.629477+0000","last_undegraded":"2024-04-25T10:01:57.629477+0000","last_fullsized":"2024-04-25T10:01:57.629477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:21.571798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627408+0000","last_change":"2024-04-25T10:01:56.030222+0000","last_active":"2024-04-25T10:01:57.627408+0000","last_peered":"2024-04-25T10:01:57.627408+0000","last_clean":"2024-04-25T10:01:57.627408+0000","last_became_active":"2024-04-25T10:01:56.023359+0000","last_became_peered":"2024-04-25T10:01:56.023359+0000","last_unstale":"2024-04-25T10:01:57.627408+0000","last_undegraded":"2024-04-25T10:01:57.627408+0000","last_fullsized":"2024-04-25T10:01:57.627408+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009639+0000","last_change":"2024-04-25T10:01:51.941059+0000","last_active":"2024-04-25T10:01:57.009639+0000","last_peered":"2024-04-25T10:01:57.009639+0000","last_clean":"2024-04-25T10:01:57.009639+0000","last_became_active":"2024-04-25T10:01:51.938978+0000","last_became_peered":"2024-04-25T10:01:51.938978+0000","last_unstale":"2024-04-25T10:01:57.009639+0000","last_undegraded":"2024-04-25T10:01:57.009639+0000","last_fullsized":"2024-04-25T10:01:57.009639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:39.550415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101806+0000","last_change":"2024-04-25T10:01:50.952791+0000","last_active":"2024-04-25T10:01:57.101806+0000","last_peered":"2024-04-25T10:01:57.101806+0000","last_clean":"2024-04-25T10:01:57.101806+0000","last_became_active":"2024-04-25T10:01:50.950155+0000","last_became_peered":"2024-04-25T10:01:50.950155+0000","last_unstale":"2024-04-25T10:01:57.101806+0000","last_undegraded":"2024-04-25T10:01:57.101806+0000","last_fullsized":"2024-04-25T10:01:57.101806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629324+0000","last_change":"2024-04-25T10:01:56.016879+0000","last_active":"2024-04-25T10:01:57.629324+0000","last_peered":"2024-04-25T10:01:57.629324+0000","last_clean":"2024-04-25T10:01:57.629324+0000","last_became_active":"2024-04-25T10:01:56.016202+0000","last_became_peered":"2024-04-25T10:01:56.016202+0000","last_unstale":"2024-04-25T10:01:57.629324+0000","last_undegraded":"2024-04-25T10:01:57.629324+0000","last_fullsized":"2024-04-25T10:01:57.629324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:48.574592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627562+0000","last_change":"2024-04-25T10:01:51.952368+0000","last_active":"2024-04-25T10:01:57.627562+0000","last_peered":"2024-04-25T10:01:57.627562+0000","last_clean":"2024-04-25T10:01:57.627562+0000","last_became_active":"2024-04-25T10:01:51.952095+0000","last_became_peered":"2024-04-25T10:01:51.952095+0000","last_unstale":"2024-04-25T10:01:57.627562+0000","last_undegraded":"2024-04-25T10:01:57.627562+0000","last_fullsized":"2024-04-25T10:01:57.627562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004117+0000","last_change":"2024-04-25T10:01:50.940514+0000","last_active":"2024-04-25T10:01:57.004117+0000","last_peered":"2024-04-25T10:01:57.004117+0000","last_clean":"2024-04-25T10:01:57.004117+0000","last_became_active":"2024-04-25T10:01:50.935395+0000","last_became_peered":"2024-04-25T10:01:50.935395+0000","last_unstale":"2024-04-25T10:01:57.004117+0000","last_undegraded":"2024-04-25T10:01:57.004117+0000","last_fullsized":"2024-04-25T10:01:57.004117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.010630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629857+0000","last_change":"2024-04-25T10:01:56.015853+0000","last_active":"2024-04-25T10:01:57.629857+0000","last_peered":"2024-04-25T10:01:57.629857+0000","last_clean":"2024-04-25T10:01:57.629857+0000","last_became_active":"2024-04-25T10:01:56.015382+0000","last_became_peered":"2024-04-25T10:01:56.015382+0000","last_unstale":"2024-04-25T10:01:57.629857+0000","last_undegraded":"2024-04-25T10:01:57.629857+0000","last_fullsized":"2024-04-25T10:01:57.629857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:32.289340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628148+0000","last_change":"2024-04-25T10:01:51.939122+0000","last_active":"2024-04-25T10:01:57.628148+0000","last_peered":"2024-04-25T10:01:57.628148+0000","last_clean":"2024-04-25T10:01:57.628148+0000","last_became_active":"2024-04-25T10:01:51.938342+0000","last_became_peered":"2024-04-25T10:01:51.938342+0000","last_unstale":"2024-04-25T10:01:57.628148+0000","last_undegraded":"2024-04-25T10:01:57.628148+0000","last_fullsized":"2024-04-25T10:01:57.628148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:33.192217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346666+0000","last_change":"2024-04-25T10:01:50.957985+0000","last_active":"2024-04-25T10:01:57.346666+0000","last_peered":"2024-04-25T10:01:57.346666+0000","last_clean":"2024-04-25T10:01:57.346666+0000","last_became_active":"2024-04-25T10:01:50.957155+0000","last_became_peered":"2024-04-25T10:01:50.957155+0000","last_unstale":"2024-04-25T10:01:57.346666+0000","last_undegraded":"2024-04-25T10:01:57.346666+0000","last_fullsized":"2024-04-25T10:01:57.346666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:21.571798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631051+0000","last_change":"2024-04-25T10:01:56.105198+0000","last_active":"2024-04-25T10:01:57.631051+0000","last_peered":"2024-04-25T10:01:57.631051+0000","last_clean":"2024-04-25T10:01:57.631051+0000","last_became_active":"2024-04-25T10:01:56.104858+0000","last_became_peered":"2024-04-25T10:01:56.104858+0000","last_unstale":"2024-04-25T10:01:57.631051+0000","last_undegraded":"2024-04-25T10:01:57.631051+0000","last_fullsized":"2024-04-25T10:01:57.631051+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:48.574592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632717+0000","last_change":"2024-04-25T10:01:51.994401+0000","last_active":"2024-04-25T10:01:57.632717+0000","last_peered":"2024-04-25T10:01:57.632717+0000","last_clean":"2024-04-25T10:01:57.632717+0000","last_became_active":"2024-04-25T10:01:51.994086+0000","last_became_peered":"2024-04-25T10:01:51.994086+0000","last_unstale":"2024-04-25T10:01:57.632717+0000","last_undegraded":"2024-04-25T10:01:57.632717+0000","last_fullsized":"2024-04-25T10:01:57.632717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101375+0000","last_change":"2024-04-25T10:01:50.950324+0000","last_active":"2024-04-25T10:01:57.101375+0000","last_peered":"2024-04-25T10:01:57.101375+0000","last_clean":"2024-04-25T10:01:57.101375+0000","last_became_active":"2024-04-25T10:01:50.949810+0000","last_became_peered":"2024-04-25T10:01:50.949810+0000","last_unstale":"2024-04-25T10:01:57.101375+0000","last_undegraded":"2024-04-25T10:01:57.101375+0000","last_fullsized":"2024-04-25T10:01:57.101375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346935+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:01:57.346935+0000","last_peered":"2024-04-25T10:01:57.346935+0000","last_clean":"2024-04-25T10:01:57.346935+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:01:57.346935+0000","last_undegraded":"2024-04-25T10:01:57.346935+0000","last_fullsized":"2024-04-25T10:01:57.346935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007399+0000","last_change":"2024-04-25T10:01:56.108134+0000","last_active":"2024-04-25T10:01:57.007399+0000","last_peered":"2024-04-25T10:01:57.007399+0000","last_clean":"2024-04-25T10:01:57.007399+0000","last_became_active":"2024-04-25T10:01:56.107123+0000","last_became_peered":"2024-04-25T10:01:56.107123+0000","last_unstale":"2024-04-25T10:01:57.007399+0000","last_undegraded":"2024-04-25T10:01:57.007399+0000","last_fullsized":"2024-04-25T10:01:57.007399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:03.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347779+0000","last_change":"2024-04-25T10:01:51.983906+0000","last_active":"2024-04-25T10:01:57.347779+0000","last_peered":"2024-04-25T10:01:57.347779+0000","last_clean":"2024-04-25T10:01:57.347779+0000","last_became_active":"2024-04-25T10:01:51.982118+0000","last_became_peered":"2024-04-25T10:01:51.982118+0000","last_unstale":"2024-04-25T10:01:57.347779+0000","last_undegraded":"2024-04-25T10:01:57.347779+0000","last_fullsized":"2024-04-25T10:01:57.347779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:14.191816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004595+0000","last_change":"2024-04-25T10:01:50.946054+0000","last_active":"2024-04-25T10:01:57.004595+0000","last_peered":"2024-04-25T10:01:57.004595+0000","last_clean":"2024-04-25T10:01:57.004595+0000","last_became_active":"2024-04-25T10:01:50.945744+0000","last_became_peered":"2024-04-25T10:01:50.945744+0000","last_unstale":"2024-04-25T10:01:57.004595+0000","last_undegraded":"2024-04-25T10:01:57.004595+0000","last_fullsized":"2024-04-25T10:01:57.004595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006974+0000","last_change":"2024-04-25T10:01:56.104590+0000","last_active":"2024-04-25T10:01:57.006974+0000","last_peered":"2024-04-25T10:01:57.006974+0000","last_clean":"2024-04-25T10:01:57.006974+0000","last_became_active":"2024-04-25T10:01:56.103498+0000","last_became_peered":"2024-04-25T10:01:56.103498+0000","last_unstale":"2024-04-25T10:01:57.006974+0000","last_undegraded":"2024-04-25T10:01:57.006974+0000","last_fullsized":"2024-04-25T10:01:57.006974+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101453+0000","last_change":"2024-04-25T10:01:51.968651+0000","last_active":"2024-04-25T10:01:57.101453+0000","last_peered":"2024-04-25T10:01:57.101453+0000","last_clean":"2024-04-25T10:01:57.101453+0000","last_became_active":"2024-04-25T10:01:51.962327+0000","last_became_peered":"2024-04-25T10:01:51.962327+0000","last_unstale":"2024-04-25T10:01:57.101453+0000","last_undegraded":"2024-04-25T10:01:57.101453+0000","last_fullsized":"2024-04-25T10:01:57.101453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:55.810277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629450+0000","last_change":"2024-04-25T10:01:50.942416+0000","last_active":"2024-04-25T10:01:57.629450+0000","last_peered":"2024-04-25T10:01:57.629450+0000","last_clean":"2024-04-25T10:01:57.629450+0000","last_became_active":"2024-04-25T10:01:50.942009+0000","last_became_peered":"2024-04-25T10:01:50.942009+0000","last_unstale":"2024-04-25T10:01:57.629450+0000","last_undegraded":"2024-04-25T10:01:57.629450+0000","last_fullsized":"2024-04-25T10:01:57.629450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:25.085344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630028+0000","last_change":"2024-04-25T10:01:56.023691+0000","last_active":"2024-04-25T10:01:57.630028+0000","last_peered":"2024-04-25T10:01:57.630028+0000","last_clean":"2024-04-25T10:01:57.630028+0000","last_became_active":"2024-04-25T10:01:56.022988+0000","last_became_peered":"2024-04-25T10:01:56.022988+0000","last_unstale":"2024-04-25T10:01:57.630028+0000","last_undegraded":"2024-04-25T10:01:57.630028+0000","last_fullsized":"2024-04-25T10:01:57.630028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347762+0000","last_change":"2024-04-25T10:01:51.987406+0000","last_active":"2024-04-25T10:01:57.347762+0000","last_peered":"2024-04-25T10:01:57.347762+0000","last_clean":"2024-04-25T10:01:57.347762+0000","last_became_active":"2024-04-25T10:01:51.986930+0000","last_became_peered":"2024-04-25T10:01:51.986930+0000","last_unstale":"2024-04-25T10:01:57.347762+0000","last_undegraded":"2024-04-25T10:01:57.347762+0000","last_fullsized":"2024-04-25T10:01:57.347762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:27.239681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004599+0000","last_change":"2024-04-25T10:01:50.932604+0000","last_active":"2024-04-25T10:01:57.004599+0000","last_peered":"2024-04-25T10:01:57.004599+0000","last_clean":"2024-04-25T10:01:57.004599+0000","last_became_active":"2024-04-25T10:01:50.932205+0000","last_became_peered":"2024-04-25T10:01:50.932205+0000","last_unstale":"2024-04-25T10:01:57.004599+0000","last_undegraded":"2024-04-25T10:01:57.004599+0000","last_fullsized":"2024-04-25T10:01:57.004599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:35.306064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100705+0000","last_change":"2024-04-25T10:01:56.105972+0000","last_active":"2024-04-25T10:01:57.100705+0000","last_peered":"2024-04-25T10:01:57.100705+0000","last_clean":"2024-04-25T10:01:57.100705+0000","last_became_active":"2024-04-25T10:01:56.104834+0000","last_became_peered":"2024-04-25T10:01:56.104834+0000","last_unstale":"2024-04-25T10:01:57.100705+0000","last_undegraded":"2024-04-25T10:01:57.100705+0000","last_fullsized":"2024-04-25T10:01:57.100705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347690+0000","last_change":"2024-04-25T10:01:51.991881+0000","last_active":"2024-04-25T10:01:57.347690+0000","last_peered":"2024-04-25T10:01:57.347690+0000","last_clean":"2024-04-25T10:01:57.347690+0000","last_became_active":"2024-04-25T10:01:51.991266+0000","last_became_peered":"2024-04-25T10:01:51.991266+0000","last_unstale":"2024-04-25T10:01:57.347690+0000","last_undegraded":"2024-04-25T10:01:57.347690+0000","last_fullsized":"2024-04-25T10:01:57.347690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:39.074042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009212+0000","last_change":"2024-04-25T10:01:50.945437+0000","last_active":"2024-04-25T10:01:57.009212+0000","last_peered":"2024-04-25T10:01:57.009212+0000","last_clean":"2024-04-25T10:01:57.009212+0000","last_became_active":"2024-04-25T10:01:50.944786+0000","last_became_peered":"2024-04-25T10:01:50.944786+0000","last_unstale":"2024-04-25T10:01:57.009212+0000","last_undegraded":"2024-04-25T10:01:57.009212+0000","last_fullsized":"2024-04-25T10:01:57.009212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:53.181549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005662+0000","last_change":"2024-04-25T10:01:56.000803+0000","last_active":"2024-04-25T10:01:57.005662+0000","last_peered":"2024-04-25T10:01:57.005662+0000","last_clean":"2024-04-25T10:01:57.005662+0000","last_became_active":"2024-04-25T10:01:55.999071+0000","last_became_peered":"2024-04-25T10:01:55.999071+0000","last_unstale":"2024-04-25T10:01:57.005662+0000","last_undegraded":"2024-04-25T10:01:57.005662+0000","last_fullsized":"2024-04-25T10:01:57.005662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:50.609054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347419+0000","last_change":"2024-04-25T10:01:51.982017+0000","last_active":"2024-04-25T10:01:57.347419+0000","last_peered":"2024-04-25T10:01:57.347419+0000","last_clean":"2024-04-25T10:01:57.347419+0000","last_became_active":"2024-04-25T10:01:51.981340+0000","last_became_peered":"2024-04-25T10:01:51.981340+0000","last_unstale":"2024-04-25T10:01:57.347419+0000","last_undegraded":"2024-04-25T10:01:57.347419+0000","last_fullsized":"2024-04-25T10:01:57.347419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:22.999554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004023+0000","last_change":"2024-04-25T10:01:50.934868+0000","last_active":"2024-04-25T10:01:57.004023+0000","last_peered":"2024-04-25T10:01:57.004023+0000","last_clean":"2024-04-25T10:01:57.004023+0000","last_became_active":"2024-04-25T10:01:50.934560+0000","last_became_peered":"2024-04-25T10:01:50.934560+0000","last_unstale":"2024-04-25T10:01:57.004023+0000","last_undegraded":"2024-04-25T10:01:57.004023+0000","last_fullsized":"2024-04-25T10:01:57.004023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630724+0000","last_change":"2024-04-25T10:01:56.012427+0000","last_active":"2024-04-25T10:01:57.630724+0000","last_peered":"2024-04-25T10:01:57.630724+0000","last_clean":"2024-04-25T10:01:57.630724+0000","last_became_active":"2024-04-25T10:01:56.008358+0000","last_became_peered":"2024-04-25T10:01:56.008358+0000","last_unstale":"2024-04-25T10:01:57.630724+0000","last_undegraded":"2024-04-25T10:01:57.630724+0000","last_fullsized":"2024-04-25T10:01:57.630724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:56.864362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006288+0000","last_change":"2024-04-25T10:01:51.947557+0000","last_active":"2024-04-25T10:01:57.006288+0000","last_peered":"2024-04-25T10:01:57.006288+0000","last_clean":"2024-04-25T10:01:57.006288+0000","last_became_active":"2024-04-25T10:01:51.942206+0000","last_became_peered":"2024-04-25T10:01:51.942206+0000","last_unstale":"2024-04-25T10:01:57.006288+0000","last_undegraded":"2024-04-25T10:01:57.006288+0000","last_fullsized":"2024-04-25T10:01:57.006288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006959+0000","last_change":"2024-04-25T10:01:50.930663+0000","last_active":"2024-04-25T10:01:57.006959+0000","last_peered":"2024-04-25T10:01:57.006959+0000","last_clean":"2024-04-25T10:01:57.006959+0000","last_became_active":"2024-04-25T10:01:50.930224+0000","last_became_peered":"2024-04-25T10:01:50.930224+0000","last_unstale":"2024-04-25T10:01:57.006959+0000","last_undegraded":"2024-04-25T10:01:57.006959+0000","last_fullsized":"2024-04-25T10:01:57.006959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634460+0000","last_change":"2024-04-25T10:01:56.110077+0000","last_active":"2024-04-25T10:01:57.634460+0000","last_peered":"2024-04-25T10:01:57.634460+0000","last_clean":"2024-04-25T10:01:57.634460+0000","last_became_active":"2024-04-25T10:01:56.109686+0000","last_became_peered":"2024-04-25T10:01:56.109686+0000","last_unstale":"2024-04-25T10:01:57.634460+0000","last_undegraded":"2024-04-25T10:01:57.634460+0000","last_fullsized":"2024-04-25T10:01:57.634460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:20.441998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347395+0000","last_change":"2024-04-25T10:01:51.981399+0000","last_active":"2024-04-25T10:01:57.347395+0000","last_peered":"2024-04-25T10:01:57.347395+0000","last_clean":"2024-04-25T10:01:57.347395+0000","last_became_active":"2024-04-25T10:01:51.980512+0000","last_became_peered":"2024-04-25T10:01:51.980512+0000","last_unstale":"2024-04-25T10:01:57.347395+0000","last_undegraded":"2024-04-25T10:01:57.347395+0000","last_fullsized":"2024-04-25T10:01:57.347395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:52.801927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101702+0000","last_change":"2024-04-25T10:01:50.949394+0000","last_active":"2024-04-25T10:01:57.101702+0000","last_peered":"2024-04-25T10:01:57.101702+0000","last_clean":"2024-04-25T10:01:57.101702+0000","last_became_active":"2024-04-25T10:01:50.948515+0000","last_became_peered":"2024-04-25T10:01:50.948515+0000","last_unstale":"2024-04-25T10:01:57.101702+0000","last_undegraded":"2024-04-25T10:01:57.101702+0000","last_fullsized":"2024-04-25T10:01:57.101702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633739+0000","last_change":"2024-04-25T10:01:56.103664+0000","last_active":"2024-04-25T10:01:57.633739+0000","last_peered":"2024-04-25T10:01:57.633739+0000","last_clean":"2024-04-25T10:01:57.633739+0000","last_became_active":"2024-04-25T10:01:56.103259+0000","last_became_peered":"2024-04-25T10:01:56.103259+0000","last_unstale":"2024-04-25T10:01:57.633739+0000","last_undegraded":"2024-04-25T10:01:57.633739+0000","last_fullsized":"2024-04-25T10:01:57.633739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:19.687438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629659+0000","last_change":"2024-04-25T10:01:51.946200+0000","last_active":"2024-04-25T10:01:57.629659+0000","last_peered":"2024-04-25T10:01:57.629659+0000","last_clean":"2024-04-25T10:01:57.629659+0000","last_became_active":"2024-04-25T10:01:51.945344+0000","last_became_peered":"2024-04-25T10:01:51.945344+0000","last_unstale":"2024-04-25T10:01:57.629659+0000","last_undegraded":"2024-04-25T10:01:57.629659+0000","last_fullsized":"2024-04-25T10:01:57.629659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:13.244438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629997+0000","last_change":"2024-04-25T10:01:50.943130+0000","last_active":"2024-04-25T10:01:57.629997+0000","last_peered":"2024-04-25T10:01:57.629997+0000","last_clean":"2024-04-25T10:01:57.629997+0000","last_became_active":"2024-04-25T10:01:50.942800+0000","last_became_peered":"2024-04-25T10:01:50.942800+0000","last_unstale":"2024-04-25T10:01:57.629997+0000","last_undegraded":"2024-04-25T10:01:57.629997+0000","last_fullsized":"2024-04-25T10:01:57.629997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:58.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.102211+0000","last_change":"2024-04-25T10:01:56.103005+0000","last_active":"2024-04-25T10:01:57.102211+0000","last_peered":"2024-04-25T10:01:57.102211+0000","last_clean":"2024-04-25T10:01:57.102211+0000","last_became_active":"2024-04-25T10:01:56.101534+0000","last_became_peered":"2024-04-25T10:01:56.101534+0000","last_unstale":"2024-04-25T10:01:57.102211+0000","last_undegraded":"2024-04-25T10:01:57.102211+0000","last_fullsized":"2024-04-25T10:01:57.102211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:31.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347218+0000","last_change":"2024-04-25T10:01:51.982629+0000","last_active":"2024-04-25T10:01:57.347218+0000","last_peered":"2024-04-25T10:01:57.347218+0000","last_clean":"2024-04-25T10:01:57.347218+0000","last_became_active":"2024-04-25T10:01:51.981703+0000","last_became_peered":"2024-04-25T10:01:51.981703+0000","last_unstale":"2024-04-25T10:01:57.347218+0000","last_undegraded":"2024-04-25T10:01:57.347218+0000","last_fullsized":"2024-04-25T10:01:57.347218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:44.512157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:02.186202+0000","last_change":"2024-04-25T10:01:50.936030+0000","last_active":"2024-04-25T10:02:02.186202+0000","last_peered":"2024-04-25T10:02:02.186202+0000","last_clean":"2024-04-25T10:02:02.186202+0000","last_became_active":"2024-04-25T10:01:50.935276+0000","last_became_peered":"2024-04-25T10:01:50.935276+0000","last_unstale":"2024-04-25T10:02:02.186202+0000","last_undegraded":"2024-04-25T10:02:02.186202+0000","last_fullsized":"2024-04-25T10:02:02.186202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627939+0000","last_change":"2024-04-25T10:01:56.029517+0000","last_active":"2024-04-25T10:01:57.627939+0000","last_peered":"2024-04-25T10:01:57.627939+0000","last_clean":"2024-04-25T10:01:57.627939+0000","last_became_active":"2024-04-25T10:01:56.018301+0000","last_became_peered":"2024-04-25T10:01:56.018301+0000","last_unstale":"2024-04-25T10:01:57.627939+0000","last_undegraded":"2024-04-25T10:01:57.627939+0000","last_fullsized":"2024-04-25T10:01:57.627939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:56.864362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008458+0000","last_change":"2024-04-25T10:01:51.952954+0000","last_active":"2024-04-25T10:01:57.008458+0000","last_peered":"2024-04-25T10:01:57.008458+0000","last_clean":"2024-04-25T10:01:57.008458+0000","last_became_active":"2024-04-25T10:01:51.952204+0000","last_became_peered":"2024-04-25T10:01:51.952204+0000","last_unstale":"2024-04-25T10:01:57.008458+0000","last_undegraded":"2024-04-25T10:01:57.008458+0000","last_fullsized":"2024-04-25T10:01:57.008458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:39.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629335+0000","last_change":"2024-04-25T10:01:50.950696+0000","last_active":"2024-04-25T10:01:57.629335+0000","last_peered":"2024-04-25T10:01:57.629335+0000","last_clean":"2024-04-25T10:01:57.629335+0000","last_became_active":"2024-04-25T10:01:50.950369+0000","last_became_peered":"2024-04-25T10:01:50.950369+0000","last_unstale":"2024-04-25T10:01:57.629335+0000","last_undegraded":"2024-04-25T10:01:57.629335+0000","last_fullsized":"2024-04-25T10:01:57.629335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:38.540445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009378+0000","last_change":"2024-04-25T10:01:56.025446+0000","last_active":"2024-04-25T10:01:57.009378+0000","last_peered":"2024-04-25T10:01:57.009378+0000","last_clean":"2024-04-25T10:01:57.009378+0000","last_became_active":"2024-04-25T10:01:56.023974+0000","last_became_peered":"2024-04-25T10:01:56.023974+0000","last_unstale":"2024-04-25T10:01:57.009378+0000","last_undegraded":"2024-04-25T10:01:57.009378+0000","last_fullsized":"2024-04-25T10:01:57.009378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631278+0000","last_change":"2024-04-25T10:01:51.969210+0000","last_active":"2024-04-25T10:01:57.631278+0000","last_peered":"2024-04-25T10:01:57.631278+0000","last_clean":"2024-04-25T10:01:57.631278+0000","last_became_active":"2024-04-25T10:01:51.968830+0000","last_became_peered":"2024-04-25T10:01:51.968830+0000","last_unstale":"2024-04-25T10:01:57.631278+0000","last_undegraded":"2024-04-25T10:01:57.631278+0000","last_fullsized":"2024-04-25T10:01:57.631278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101496+0000","last_change":"2024-04-25T10:01:50.948707+0000","last_active":"2024-04-25T10:01:57.101496+0000","last_peered":"2024-04-25T10:01:57.101496+0000","last_clean":"2024-04-25T10:01:57.101496+0000","last_became_active":"2024-04-25T10:01:50.947988+0000","last_became_peered":"2024-04-25T10:01:50.947988+0000","last_unstale":"2024-04-25T10:01:57.101496+0000","last_undegraded":"2024-04-25T10:01:57.101496+0000","last_fullsized":"2024-04-25T10:01:57.101496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631528+0000","last_change":"2024-04-25T10:01:56.017001+0000","last_active":"2024-04-25T10:01:57.631528+0000","last_peered":"2024-04-25T10:01:57.631528+0000","last_clean":"2024-04-25T10:01:57.631528+0000","last_became_active":"2024-04-25T10:01:56.016220+0000","last_became_peered":"2024-04-25T10:01:56.016220+0000","last_unstale":"2024-04-25T10:01:57.631528+0000","last_undegraded":"2024-04-25T10:01:57.631528+0000","last_fullsized":"2024-04-25T10:01:57.631528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:31.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101724+0000","last_change":"2024-04-25T10:01:51.978944+0000","last_active":"2024-04-25T10:01:57.101724+0000","last_peered":"2024-04-25T10:01:57.101724+0000","last_clean":"2024-04-25T10:01:57.101724+0000","last_became_active":"2024-04-25T10:01:51.978133+0000","last_became_peered":"2024-04-25T10:01:51.978133+0000","last_unstale":"2024-04-25T10:01:57.101724+0000","last_undegraded":"2024-04-25T10:01:57.101724+0000","last_fullsized":"2024-04-25T10:01:57.101724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007998+0000","last_change":"2024-04-25T10:01:50.953423+0000","last_active":"2024-04-25T10:01:57.007998+0000","last_peered":"2024-04-25T10:01:57.007998+0000","last_clean":"2024-04-25T10:01:57.007998+0000","last_became_active":"2024-04-25T10:01:50.952221+0000","last_became_peered":"2024-04-25T10:01:50.952221+0000","last_unstale":"2024-04-25T10:01:57.007998+0000","last_undegraded":"2024-04-25T10:01:57.007998+0000","last_fullsized":"2024-04-25T10:01:57.007998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631879+0000","last_change":"2024-04-25T10:01:56.110756+0000","last_active":"2024-04-25T10:01:57.631879+0000","last_peered":"2024-04-25T10:01:57.631879+0000","last_clean":"2024-04-25T10:01:57.631879+0000","last_became_active":"2024-04-25T10:01:56.109749+0000","last_became_peered":"2024-04-25T10:01:56.109749+0000","last_unstale":"2024-04-25T10:01:57.631879+0000","last_undegraded":"2024-04-25T10:01:57.631879+0000","last_fullsized":"2024-04-25T10:01:57.631879+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:48.763078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628771+0000","last_change":"2024-04-25T10:01:56.106331+0000","last_active":"2024-04-25T10:01:57.628771+0000","last_peered":"2024-04-25T10:01:57.628771+0000","last_clean":"2024-04-25T10:01:57.628771+0000","last_became_active":"2024-04-25T10:01:56.105560+0000","last_became_peered":"2024-04-25T10:01:56.105560+0000","last_unstale":"2024-04-25T10:01:57.628771+0000","last_undegraded":"2024-04-25T10:01:57.628771+0000","last_fullsized":"2024-04-25T10:01:57.628771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627589+0000","last_change":"2024-04-25T10:01:51.959889+0000","last_active":"2024-04-25T10:01:57.627589+0000","last_peered":"2024-04-25T10:01:57.627589+0000","last_clean":"2024-04-25T10:01:57.627589+0000","last_became_active":"2024-04-25T10:01:51.959609+0000","last_became_peered":"2024-04-25T10:01:51.959609+0000","last_unstale":"2024-04-25T10:01:57.627589+0000","last_undegraded":"2024-04-25T10:01:57.627589+0000","last_fullsized":"2024-04-25T10:01:57.627589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101160+0000","last_change":"2024-04-25T10:01:50.947616+0000","last_active":"2024-04-25T10:01:57.101160+0000","last_peered":"2024-04-25T10:01:57.101160+0000","last_clean":"2024-04-25T10:01:57.101160+0000","last_became_active":"2024-04-25T10:01:50.946930+0000","last_became_peered":"2024-04-25T10:01:50.946930+0000","last_unstale":"2024-04-25T10:01:57.101160+0000","last_undegraded":"2024-04-25T10:01:57.101160+0000","last_fullsized":"2024-04-25T10:01:57.101160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:33.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007713+0000","last_change":"2024-04-25T10:01:56.018502+0000","last_active":"2024-04-25T10:01:57.007713+0000","last_peered":"2024-04-25T10:01:57.007713+0000","last_clean":"2024-04-25T10:01:57.007713+0000","last_became_active":"2024-04-25T10:01:56.017379+0000","last_became_peered":"2024-04-25T10:01:56.017379+0000","last_unstale":"2024-04-25T10:01:57.007713+0000","last_undegraded":"2024-04-25T10:01:57.007713+0000","last_fullsized":"2024-04-25T10:01:57.007713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631645+0000","last_change":"2024-04-25T10:01:51.953615+0000","last_active":"2024-04-25T10:01:57.631645+0000","last_peered":"2024-04-25T10:01:57.631645+0000","last_clean":"2024-04-25T10:01:57.631645+0000","last_became_active":"2024-04-25T10:01:51.953362+0000","last_became_peered":"2024-04-25T10:01:51.953362+0000","last_unstale":"2024-04-25T10:01:57.631645+0000","last_undegraded":"2024-04-25T10:01:57.631645+0000","last_fullsized":"2024-04-25T10:01:57.631645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632491+0000","last_change":"2024-04-25T10:01:50.945460+0000","last_active":"2024-04-25T10:01:57.632491+0000","last_peered":"2024-04-25T10:01:57.632491+0000","last_clean":"2024-04-25T10:01:57.632491+0000","last_became_active":"2024-04-25T10:01:50.945086+0000","last_became_peered":"2024-04-25T10:01:50.945086+0000","last_unstale":"2024-04-25T10:01:57.632491+0000","last_undegraded":"2024-04-25T10:01:57.632491+0000","last_fullsized":"2024-04-25T10:01:57.632491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.010630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007381+0000","last_change":"2024-04-25T10:01:50.944204+0000","last_active":"2024-04-25T10:01:57.007381+0000","last_peered":"2024-04-25T10:01:57.007381+0000","last_clean":"2024-04-25T10:01:57.007381+0000","last_became_active":"2024-04-25T10:01:50.943478+0000","last_became_peered":"2024-04-25T10:01:50.943478+0000","last_unstale":"2024-04-25T10:01:57.007381+0000","last_undegraded":"2024-04-25T10:01:57.007381+0000","last_fullsized":"2024-04-25T10:01:57.007381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629902+0000","last_change":"2024-04-25T10:01:51.939383+0000","last_active":"2024-04-25T10:01:57.629902+0000","last_peered":"2024-04-25T10:01:57.629902+0000","last_clean":"2024-04-25T10:01:57.629902+0000","last_became_active":"2024-04-25T10:01:51.938491+0000","last_became_peered":"2024-04-25T10:01:51.938491+0000","last_unstale":"2024-04-25T10:01:57.629902+0000","last_undegraded":"2024-04-25T10:01:57.629902+0000","last_fullsized":"2024-04-25T10:01:57.629902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004475+0000","last_change":"2024-04-25T10:01:56.024292+0000","last_active":"2024-04-25T10:01:57.004475+0000","last_peered":"2024-04-25T10:01:57.004475+0000","last_clean":"2024-04-25T10:01:57.004475+0000","last_became_active":"2024-04-25T10:01:56.023778+0000","last_became_peered":"2024-04-25T10:01:56.023778+0000","last_unstale":"2024-04-25T10:01:57.004475+0000","last_undegraded":"2024-04-25T10:01:57.004475+0000","last_fullsized":"2024-04-25T10:01:57.004475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:30.157749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633372+0000","last_change":"2024-04-25T10:01:50.954588+0000","last_active":"2024-04-25T10:01:57.633372+0000","last_peered":"2024-04-25T10:01:57.633372+0000","last_clean":"2024-04-25T10:01:57.633372+0000","last_became_active":"2024-04-25T10:01:50.954119+0000","last_became_peered":"2024-04-25T10:01:50.954119+0000","last_unstale":"2024-04-25T10:01:57.633372+0000","last_undegraded":"2024-04-25T10:01:57.633372+0000","last_fullsized":"2024-04-25T10:01:57.633372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:35.306064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007283+0000","last_change":"2024-04-25T10:01:51.941227+0000","last_active":"2024-04-25T10:01:57.007283+0000","last_peered":"2024-04-25T10:01:57.007283+0000","last_clean":"2024-04-25T10:01:57.007283+0000","last_became_active":"2024-04-25T10:01:51.940116+0000","last_became_peered":"2024-04-25T10:01:51.940116+0000","last_unstale":"2024-04-25T10:01:57.007283+0000","last_undegraded":"2024-04-25T10:01:57.007283+0000","last_fullsized":"2024-04-25T10:01:57.007283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:49.915603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633711+0000","last_change":"2024-04-25T10:01:56.024226+0000","last_active":"2024-04-25T10:01:57.633711+0000","last_peered":"2024-04-25T10:01:57.633711+0000","last_clean":"2024-04-25T10:01:57.633711+0000","last_became_active":"2024-04-25T10:01:56.023568+0000","last_became_peered":"2024-04-25T10:01:56.023568+0000","last_unstale":"2024-04-25T10:01:57.633711+0000","last_undegraded":"2024-04-25T10:01:57.633711+0000","last_fullsized":"2024-04-25T10:01:57.633711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.434943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005947+0000","last_change":"2024-04-25T10:01:51.948178+0000","last_active":"2024-04-25T10:01:57.005947+0000","last_peered":"2024-04-25T10:01:57.005947+0000","last_clean":"2024-04-25T10:01:57.005947+0000","last_became_active":"2024-04-25T10:01:51.943159+0000","last_became_peered":"2024-04-25T10:01:51.943159+0000","last_unstale":"2024-04-25T10:01:57.005947+0000","last_undegraded":"2024-04-25T10:01:57.005947+0000","last_fullsized":"2024-04-25T10:01:57.005947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:13.244438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004086+0000","last_change":"2024-04-25T10:01:51.977923+0000","last_active":"2024-04-25T10:01:57.004086+0000","last_peered":"2024-04-25T10:01:57.004086+0000","last_clean":"2024-04-25T10:01:57.004086+0000","last_became_active":"2024-04-25T10:01:51.977601+0000","last_became_peered":"2024-04-25T10:01:51.977601+0000","last_unstale":"2024-04-25T10:01:57.004086+0000","last_undegraded":"2024-04-25T10:01:57.004086+0000","last_fullsized":"2024-04-25T10:01:57.004086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:39.550415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101440+0000","last_change":"2024-04-25T10:01:51.977323+0000","last_active":"2024-04-25T10:01:57.101440+0000","last_peered":"2024-04-25T10:01:57.101440+0000","last_clean":"2024-04-25T10:01:57.101440+0000","last_became_active":"2024-04-25T10:01:51.976662+0000","last_became_peered":"2024-04-25T10:01:51.976662+0000","last_unstale":"2024-04-25T10:01:57.101440+0000","last_undegraded":"2024-04-25T10:01:57.101440+0000","last_fullsized":"2024-04-25T10:01:57.101440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:52.801927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629808+0000","last_change":"2024-04-25T10:01:51.962727+0000","last_active":"2024-04-25T10:01:57.629808+0000","last_peered":"2024-04-25T10:01:57.629808+0000","last_clean":"2024-04-25T10:01:57.629808+0000","last_became_active":"2024-04-25T10:01:51.962048+0000","last_became_peered":"2024-04-25T10:01:51.962048+0000","last_unstale":"2024-04-25T10:01:57.629808+0000","last_undegraded":"2024-04-25T10:01:57.629808+0000","last_fullsized":"2024-04-25T10:01:57.629808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633852+0000","last_change":"2024-04-25T10:01:51.956859+0000","last_active":"2024-04-25T10:01:57.633852+0000","last_peered":"2024-04-25T10:01:57.633852+0000","last_clean":"2024-04-25T10:01:57.633852+0000","last_became_active":"2024-04-25T10:01:51.956531+0000","last_became_peered":"2024-04-25T10:01:51.956531+0000","last_unstale":"2024-04-25T10:01:57.633852+0000","last_undegraded":"2024-04-25T10:01:57.633852+0000","last_fullsized":"2024-04-25T10:01:57.633852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628823+0000","last_change":"2024-04-25T10:01:51.960245+0000","last_active":"2024-04-25T10:01:57.628823+0000","last_peered":"2024-04-25T10:01:57.628823+0000","last_clean":"2024-04-25T10:01:57.628823+0000","last_became_active":"2024-04-25T10:01:51.959954+0000","last_became_peered":"2024-04-25T10:01:51.959954+0000","last_unstale":"2024-04-25T10:01:57.628823+0000","last_undegraded":"2024-04-25T10:01:57.628823+0000","last_fullsized":"2024-04-25T10:01:57.628823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008265+0000","last_change":"2024-04-25T10:01:51.951215+0000","last_active":"2024-04-25T10:01:57.008265+0000","last_peered":"2024-04-25T10:01:57.008265+0000","last_clean":"2024-04-25T10:01:57.008265+0000","last_became_active":"2024-04-25T10:01:51.950168+0000","last_became_peered":"2024-04-25T10:01:51.950168+0000","last_unstale":"2024-04-25T10:01:57.008265+0000","last_undegraded":"2024-04-25T10:01:57.008265+0000","last_fullsized":"2024-04-25T10:01:57.008265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634588+0000","last_change":"2024-04-25T10:01:51.959477+0000","last_active":"2024-04-25T10:01:57.634588+0000","last_peered":"2024-04-25T10:01:57.634588+0000","last_clean":"2024-04-25T10:01:57.634588+0000","last_became_active":"2024-04-25T10:01:51.958740+0000","last_became_peered":"2024-04-25T10:01:51.958740+0000","last_unstale":"2024-04-25T10:01:57.634588+0000","last_undegraded":"2024-04-25T10:01:57.634588+0000","last_fullsized":"2024-04-25T10:01:57.634588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100898+0000","last_change":"2024-04-25T10:01:51.987810+0000","last_active":"2024-04-25T10:01:57.100898+0000","last_peered":"2024-04-25T10:01:57.100898+0000","last_clean":"2024-04-25T10:01:57.100898+0000","last_became_active":"2024-04-25T10:01:51.978504+0000","last_became_peered":"2024-04-25T10:01:51.978504+0000","last_unstale":"2024-04-25T10:01:57.100898+0000","last_undegraded":"2024-04-25T10:01:57.100898+0000","last_fullsized":"2024-04-25T10:01:57.100898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:44.512157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632135+0000","last_change":"2024-04-25T10:01:51.949190+0000","last_active":"2024-04-25T10:01:57.632135+0000","last_peered":"2024-04-25T10:01:57.632135+0000","last_clean":"2024-04-25T10:01:57.632135+0000","last_became_active":"2024-04-25T10:01:51.948886+0000","last_became_peered":"2024-04-25T10:01:51.948886+0000","last_unstale":"2024-04-25T10:01:57.632135+0000","last_undegraded":"2024-04-25T10:01:57.632135+0000","last_fullsized":"2024-04-25T10:01:57.632135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101158+0000","last_change":"2024-04-25T10:01:51.987833+0000","last_active":"2024-04-25T10:01:57.101158+0000","last_peered":"2024-04-25T10:01:57.101158+0000","last_clean":"2024-04-25T10:01:57.101158+0000","last_became_active":"2024-04-25T10:01:51.978154+0000","last_became_peered":"2024-04-25T10:01:51.978154+0000","last_unstale":"2024-04-25T10:01:57.101158+0000","last_undegraded":"2024-04-25T10:01:57.101158+0000","last_fullsized":"2024-04-25T10:01:57.101158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:22.999554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629659+0000","last_change":"2024-04-25T10:01:51.947940+0000","last_active":"2024-04-25T10:01:57.629659+0000","last_peered":"2024-04-25T10:01:57.629659+0000","last_clean":"2024-04-25T10:01:57.629659+0000","last_became_active":"2024-04-25T10:01:51.947672+0000","last_became_peered":"2024-04-25T10:01:51.947672+0000","last_unstale":"2024-04-25T10:01:57.629659+0000","last_undegraded":"2024-04-25T10:01:57.629659+0000","last_fullsized":"2024-04-25T10:01:57.629659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:39.087518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007150+0000","last_change":"2024-04-25T10:01:51.936013+0000","last_active":"2024-04-25T10:01:57.007150+0000","last_peered":"2024-04-25T10:01:57.007150+0000","last_clean":"2024-04-25T10:01:57.007150+0000","last_became_active":"2024-04-25T10:01:51.933077+0000","last_became_peered":"2024-04-25T10:01:51.933077+0000","last_unstale":"2024-04-25T10:01:57.007150+0000","last_undegraded":"2024-04-25T10:01:57.007150+0000","last_fullsized":"2024-04-25T10:01:57.007150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348064+0000","last_change":"2024-04-25T10:01:51.987425+0000","last_active":"2024-04-25T10:01:57.348064+0000","last_peered":"2024-04-25T10:01:57.348064+0000","last_clean":"2024-04-25T10:01:57.348064+0000","last_became_active":"2024-04-25T10:01:51.986698+0000","last_became_peered":"2024-04-25T10:01:51.986698+0000","last_unstale":"2024-04-25T10:01:57.348064+0000","last_undegraded":"2024-04-25T10:01:57.348064+0000","last_fullsized":"2024-04-25T10:01:57.348064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:28.226905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004602+0000","last_change":"2024-04-25T10:01:51.962633+0000","last_active":"2024-04-25T10:01:57.004602+0000","last_peered":"2024-04-25T10:01:57.004602+0000","last_clean":"2024-04-25T10:01:57.004602+0000","last_became_active":"2024-04-25T10:01:51.962235+0000","last_became_peered":"2024-04-25T10:01:51.962235+0000","last_unstale":"2024-04-25T10:01:57.004602+0000","last_undegraded":"2024-04-25T10:01:57.004602+0000","last_fullsized":"2024-04-25T10:01:57.004602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:39.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101861+0000","last_change":"2024-04-25T10:01:51.992557+0000","last_active":"2024-04-25T10:01:57.101861+0000","last_peered":"2024-04-25T10:01:57.101861+0000","last_clean":"2024-04-25T10:01:57.101861+0000","last_became_active":"2024-04-25T10:01:51.991982+0000","last_became_peered":"2024-04-25T10:01:51.991982+0000","last_unstale":"2024-04-25T10:01:57.101861+0000","last_undegraded":"2024-04-25T10:01:57.101861+0000","last_fullsized":"2024-04-25T10:01:57.101861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100581+0000","last_change":"2024-04-25T10:01:51.988776+0000","last_active":"2024-04-25T10:01:57.100581+0000","last_peered":"2024-04-25T10:01:57.100581+0000","last_clean":"2024-04-25T10:01:57.100581+0000","last_became_active":"2024-04-25T10:01:51.987983+0000","last_became_peered":"2024-04-25T10:01:51.987983+0000","last_unstale":"2024-04-25T10:01:57.100581+0000","last_undegraded":"2024-04-25T10:01:57.100581+0000","last_fullsized":"2024-04-25T10:01:57.100581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:39.074042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627816+0000","last_change":"2024-04-25T10:01:51.946481+0000","last_active":"2024-04-25T10:01:57.627816+0000","last_peered":"2024-04-25T10:01:57.627816+0000","last_clean":"2024-04-25T10:01:57.627816+0000","last_became_active":"2024-04-25T10:01:51.946161+0000","last_became_peered":"2024-04-25T10:01:51.946161+0000","last_unstale":"2024-04-25T10:01:57.627816+0000","last_undegraded":"2024-04-25T10:01:57.627816+0000","last_fullsized":"2024-04-25T10:01:57.627816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629440+0000","last_change":"2024-04-25T10:01:51.946480+0000","last_active":"2024-04-25T10:01:57.629440+0000","last_peered":"2024-04-25T10:01:57.629440+0000","last_clean":"2024-04-25T10:01:57.629440+0000","last_became_active":"2024-04-25T10:01:51.945872+0000","last_became_peered":"2024-04-25T10:01:51.945872+0000","last_unstale":"2024-04-25T10:01:57.629440+0000","last_undegraded":"2024-04-25T10:01:57.629440+0000","last_fullsized":"2024-04-25T10:01:57.629440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:44.793972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633094+0000","last_change":"2024-04-25T10:01:51.958336+0000","last_active":"2024-04-25T10:01:57.633094+0000","last_peered":"2024-04-25T10:01:57.633094+0000","last_clean":"2024-04-25T10:01:57.633094+0000","last_became_active":"2024-04-25T10:01:51.957993+0000","last_became_peered":"2024-04-25T10:01:51.957993+0000","last_unstale":"2024-04-25T10:01:57.633094+0000","last_undegraded":"2024-04-25T10:01:57.633094+0000","last_fullsized":"2024-04-25T10:01:57.633094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008236+0000","last_change":"2024-04-25T10:01:51.943174+0000","last_active":"2024-04-25T10:01:57.008236+0000","last_peered":"2024-04-25T10:01:57.008236+0000","last_clean":"2024-04-25T10:01:57.008236+0000","last_became_active":"2024-04-25T10:01:51.942327+0000","last_became_peered":"2024-04-25T10:01:51.942327+0000","last_unstale":"2024-04-25T10:01:57.008236+0000","last_undegraded":"2024-04-25T10:01:57.008236+0000","last_fullsized":"2024-04-25T10:01:57.008236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:22.581767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007882+0000","last_change":"2024-04-25T10:01:51.963532+0000","last_active":"2024-04-25T10:01:57.007882+0000","last_peered":"2024-04-25T10:01:57.007882+0000","last_clean":"2024-04-25T10:01:57.007882+0000","last_became_active":"2024-04-25T10:01:51.962793+0000","last_became_peered":"2024-04-25T10:01:51.962793+0000","last_unstale":"2024-04-25T10:01:57.007882+0000","last_undegraded":"2024-04-25T10:01:57.007882+0000","last_fullsized":"2024-04-25T10:01:57.007882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008103+0000","last_change":"2024-04-25T10:01:51.970144+0000","last_active":"2024-04-25T10:01:57.008103+0000","last_peered":"2024-04-25T10:01:57.008103+0000","last_clean":"2024-04-25T10:01:57.008103+0000","last_became_active":"2024-04-25T10:01:51.967189+0000","last_became_peered":"2024-04-25T10:01:51.967189+0000","last_unstale":"2024-04-25T10:01:57.008103+0000","last_undegraded":"2024-04-25T10:01:57.008103+0000","last_fullsized":"2024-04-25T10:01:57.008103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347824+0000","last_change":"2024-04-25T10:01:51.995155+0000","last_active":"2024-04-25T10:01:57.347824+0000","last_peered":"2024-04-25T10:01:57.347824+0000","last_clean":"2024-04-25T10:01:57.347824+0000","last_became_active":"2024-04-25T10:01:51.994514+0000","last_became_peered":"2024-04-25T10:01:51.994514+0000","last_unstale":"2024-04-25T10:01:57.347824+0000","last_undegraded":"2024-04-25T10:01:57.347824+0000","last_fullsized":"2024-04-25T10:01:57.347824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:36.570949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631050+0000","last_change":"2024-04-25T10:01:51.977705+0000","last_active":"2024-04-25T10:01:57.631050+0000","last_peered":"2024-04-25T10:01:57.631050+0000","last_clean":"2024-04-25T10:01:57.631050+0000","last_became_active":"2024-04-25T10:01:51.977373+0000","last_became_peered":"2024-04-25T10:01:51.977373+0000","last_unstale":"2024-04-25T10:01:57.631050+0000","last_undegraded":"2024-04-25T10:01:57.631050+0000","last_fullsized":"2024-04-25T10:01:57.631050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347155+0000","last_change":"2024-04-25T10:01:51.958773+0000","last_active":"2024-04-25T10:01:57.347155+0000","last_peered":"2024-04-25T10:01:57.347155+0000","last_clean":"2024-04-25T10:01:57.347155+0000","last_became_active":"2024-04-25T10:01:51.958015+0000","last_became_peered":"2024-04-25T10:01:51.958015+0000","last_unstale":"2024-04-25T10:01:57.347155+0000","last_undegraded":"2024-04-25T10:01:57.347155+0000","last_fullsized":"2024-04-25T10:01:57.347155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100869+0000","last_change":"2024-04-25T10:01:51.994172+0000","last_active":"2024-04-25T10:01:57.100869+0000","last_peered":"2024-04-25T10:01:57.100869+0000","last_clean":"2024-04-25T10:01:57.100869+0000","last_became_active":"2024-04-25T10:01:51.993498+0000","last_became_peered":"2024-04-25T10:01:51.993498+0000","last_unstale":"2024-04-25T10:01:57.100869+0000","last_undegraded":"2024-04-25T10:01:57.100869+0000","last_fullsized":"2024-04-25T10:01:57.100869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:59.098969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346909+0000","last_change":"2024-04-25T10:01:51.958621+0000","last_active":"2024-04-25T10:01:57.346909+0000","last_peered":"2024-04-25T10:01:57.346909+0000","last_clean":"2024-04-25T10:01:57.346909+0000","last_became_active":"2024-04-25T10:01:51.957937+0000","last_became_peered":"2024-04-25T10:01:51.957937+0000","last_unstale":"2024-04-25T10:01:57.346909+0000","last_undegraded":"2024-04-25T10:01:57.346909+0000","last_fullsized":"2024-04-25T10:01:57.346909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:59.098969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101663+0000","last_change":"2024-04-25T10:01:51.967673+0000","last_active":"2024-04-25T10:01:57.101663+0000","last_peered":"2024-04-25T10:01:57.101663+0000","last_clean":"2024-04-25T10:01:57.101663+0000","last_became_active":"2024-04-25T10:01:51.966329+0000","last_became_peered":"2024-04-25T10:01:51.966329+0000","last_unstale":"2024-04-25T10:01:57.101663+0000","last_undegraded":"2024-04-25T10:01:57.101663+0000","last_fullsized":"2024-04-25T10:01:57.101663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630321+0000","last_change":"2024-04-25T10:01:51.950122+0000","last_active":"2024-04-25T10:01:57.630321+0000","last_peered":"2024-04-25T10:01:57.630321+0000","last_clean":"2024-04-25T10:01:57.630321+0000","last_became_active":"2024-04-25T10:01:51.949559+0000","last_became_peered":"2024-04-25T10:01:51.949559+0000","last_unstale":"2024-04-25T10:01:57.630321+0000","last_undegraded":"2024-04-25T10:01:57.630321+0000","last_fullsized":"2024-04-25T10:01:57.630321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:52.021045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629583+0000","last_change":"2024-04-25T10:01:51.943127+0000","last_active":"2024-04-25T10:01:57.629583+0000","last_peered":"2024-04-25T10:01:57.629583+0000","last_clean":"2024-04-25T10:01:57.629583+0000","last_became_active":"2024-04-25T10:01:51.942304+0000","last_became_peered":"2024-04-25T10:01:51.942304+0000","last_unstale":"2024-04-25T10:01:57.629583+0000","last_undegraded":"2024-04-25T10:01:57.629583+0000","last_fullsized":"2024-04-25T10:01:57.629583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347294+0000","last_change":"2024-04-25T10:01:51.987276+0000","last_active":"2024-04-25T10:01:57.347294+0000","last_peered":"2024-04-25T10:01:57.347294+0000","last_clean":"2024-04-25T10:01:57.347294+0000","last_became_active":"2024-04-25T10:01:51.986766+0000","last_became_peered":"2024-04-25T10:01:51.986766+0000","last_unstale":"2024-04-25T10:01:57.347294+0000","last_undegraded":"2024-04-25T10:01:57.347294+0000","last_fullsized":"2024-04-25T10:01:57.347294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:57.724919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:21.811 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c10d02640 1 Processor -- start 2024-04-25T10:02:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c10d02640 1 -- start start 2024-04-25T10:02:21.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c10d02640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2c0c158110 0x7f2c0c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c10d02640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c1584f0 0x7f2c0c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c10d02640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0c0cbe60 con 0x7f2c0c06e460 2024-04-25T10:02:21.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c10d02640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0c0b19b0 con 0x7f2c0c158110 2024-04-25T10:02:21.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c10d02640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c15b360 con 0x7f2c0c1584f0 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c0ad9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2c0c06e460 0x7f2c0c15e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52074/0 (socket says 172.21.15.133:52074) 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c0ad9d640 1 -- 172.21.15.133:0/2894067533 learned_addr learned my addr 172.21.15.133:0/2894067533 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614538068 0 0) 0x7f2c0c0cbe60 con 0x7f2c0c06e460 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c0a59c640 1 --2- 172.21.15.133:0/2894067533 >> v2:172.21.15.162:3300/0 conn(0x7f2c0c158110 0x7f2c0c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c03fff640 1 --2- 172.21.15.133:0/2894067533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c1584f0 0x7f2c0c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bf8005740 con 0x7f2c0c06e460 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1251132012 0 0) 0x7f2bf8005740 con 0x7f2c0c06e460 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c1584f0 msgr2=0x7f2c0c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 --2- 172.21.15.133:0/2894067533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c1584f0 0x7f2c0c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 >> v2:172.21.15.162:3300/0 conn(0x7f2c0c158110 msgr2=0x7f2c0c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 --2- 172.21.15.133:0/2894067533 >> v2:172.21.15.162:3300/0 conn(0x7f2c0c158110 0x7f2c0c0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c15b640 con 0x7f2c0c06e460 2024-04-25T10:02:21.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2c04001a00 con 0x7f2c0c06e460 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 >> v1:172.21.15.133:6789/0 conn(0x7f2c0c06e460 legacy=0x7f2c0c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 --2- 172.21.15.133:0/2894067533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c1584f0 0x7f2bf8005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 --2- 172.21.15.133:0/2894067533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 0x7f2bf8006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 --2- 172.21.15.133:0/2894067533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf8006f60 0x7f2bf80073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0b19b0 con 0x7f2bf8006150 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c15b360 con 0x7f2bf8006f60 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf800dc30 con 0x7f2c0c1584f0 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c0a59c640 1 --2- 172.21.15.133:0/2894067533 >> v2:172.21.15.162:3300/0 conn(0x7f2c0c158110 0x7f2c0c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c03fff640 1 --2- 172.21.15.133:0/2894067533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 0x7f2bf8006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c0ad9d640 1 --2- 172.21.15.133:0/2894067533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf8006f60 0x7f2bf80073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c03fff640 1 -- 172.21.15.133:0/2894067533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c1584f0 msgr2=0x7f2bf8005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c03fff640 1 --2- 172.21.15.133:0/2894067533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c1584f0 0x7f2bf8005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c03fff640 1 -- 172.21.15.133:0/2894067533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf8006f60 msgr2=0x7f2bf80073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c03fff640 1 --2- 172.21.15.133:0/2894067533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf8006f60 0x7f2bf80073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.956+0000 7f2c03fff640 1 -- 172.21.15.133:0/2894067533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf8005740 con 0x7f2bf8006150 2024-04-25T10:02:21.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c03fff640 1 --2- 172.21.15.133:0/2894067533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 0x7f2bf8006a20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009ae0 tx=0x7f2bf401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80870df8435c911 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40b2070 con 0x7f2bf8006150 2024-04-25T10:02:21.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf40a5590 con 0x7f2bf8006150 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c09d9b640 1 -- 172.21.15.133:0/2894067533 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40096b0 con 0x7f2bf8006150 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 -- 172.21.15.133:0/2894067533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 msgr2=0x7f2bf8006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 --2- 172.21.15.133:0/2894067533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 0x7f2bf8006a20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009ae0 tx=0x7f2bf401cab0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0ad9d640 1 -- 172.21.15.133:0/2894067533 reap_dead start 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 -- 172.21.15.133:0/2894067533 shutdown_connections 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 -- 172.21.15.133:0/2894067533 >> 172.21.15.133:0/2894067533 conn(0x7f2c0c067000 msgr2=0x7f2c0c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 -- 172.21.15.133:0/2894067533 shutdown_connections 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 -- 172.21.15.133:0/2894067533 wait complete. 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 Processor -- start 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 -- start start 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 0x7f2c0c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c14b2b0 0x7f2c0c14d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c0c14d7f0 0x7f2c0c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0cd670 con 0x7f2bf8006150 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c15bc00 con 0x7f2c0c14d7f0 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c10d02640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0cd7f0 con 0x7f2c0c14b2b0 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 0x7f2c0c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 0x7f2c0c153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48740/0 (socket says 172.21.15.133:48740) 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0a59c640 1 -- 172.21.15.133:0/4256141988 learned_addr learned my addr 172.21.15.133:0/4256141988 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0ad9d640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c0c14d7f0 0x7f2c0c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0a59c640 1 -- 172.21.15.133:0/4256141988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c14b2b0 msgr2=0x7f2c0c14d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c03fff640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c14b2b0 0x7f2c0c14d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0a59c640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c14b2b0 0x7f2c0c14d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0a59c640 1 -- 172.21.15.133:0/4256141988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c0c14d7f0 msgr2=0x7f2c0c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0a59c640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c0c14d7f0 0x7f2c0c0b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0a59c640 1 -- 172.21.15.133:0/4256141988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c1f3aa0 con 0x7f2bf8006150 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0ad9d640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c0c14d7f0 0x7f2c0c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c03fff640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c14b2b0 0x7f2c0c14d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:21.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c0a59c640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 0x7f2c0c153d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00e810 tx=0x7f2bfc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.960+0000 7f2c02ffd640 1 -- 172.21.15.133:0/4256141988 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc017070 con 0x7f2bf8006150 2024-04-25T10:02:21.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.964+0000 7f2c02ffd640 1 -- 172.21.15.133:0/4256141988 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc00c870 con 0x7f2bf8006150 2024-04-25T10:02:21.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.964+0000 7f2c02ffd640 1 -- 172.21.15.133:0/4256141988 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc00f040 con 0x7f2bf8006150 2024-04-25T10:02:21.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.964+0000 7f2c10d02640 1 -- 172.21.15.133:0/4256141988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c1f3d80 con 0x7f2bf8006150 2024-04-25T10:02:21.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.964+0000 7f2c10d02640 1 -- 172.21.15.133:0/4256141988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0c1f4340 con 0x7f2bf8006150 2024-04-25T10:02:21.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.964+0000 7f2c10d02640 1 -- 172.21.15.133:0/4256141988 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c14dc00 con 0x7f2bf8006150 2024-04-25T10:02:21.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.964+0000 7f2c02ffd640 1 -- 172.21.15.133:0/4256141988 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bfc01a070 con 0x7f2bf8006150 2024-04-25T10:02:21.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.964+0000 7f2c02ffd640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bec066f90 0x7f2bec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.964+0000 7f2c02ffd640 1 -- 172.21.15.133:0/4256141988 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2bfc084f70 con 0x7f2bf8006150 2024-04-25T10:02:21.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.964+0000 7f2c03fff640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bec066f90 0x7f2bec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.964+0000 7f2c03fff640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bec066f90 0x7f2bec069450 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40af000 tx=0x7f2bf40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:21.968+0000 7f2c02ffd640 1 -- 172.21.15.133:0/4256141988 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c0c14dc00 con 0x7f2bf8006150 2024-04-25T10:02:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.112+0000 7f2c10d02640 1 -- 172.21.15.133:0/4256141988 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c0c1554e0 con 0x7f2bec066f90 2024-04-25T10:02:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.132+0000 7f2c02ffd640 1 -- 172.21.15.133:0/4256141988 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7f2bf410d310 con 0x7f2bec066f90 2024-04-25T10:02:22.144 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:22.146 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 -- 172.21.15.133:0/4256141988 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bec066f90 msgr2=0x7f2bec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bec066f90 0x7f2bec069450 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40af000 tx=0x7f2bf40046d0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 -- 172.21.15.133:0/4256141988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 msgr2=0x7f2c0c153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 0x7f2c0c153d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00e810 tx=0x7f2bfc00ecd0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 -- 172.21.15.133:0/4256141988 shutdown_connections 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c0c14d7f0 0x7f2c0c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c0c14b2b0 0x7f2c0c14d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bec066f90 0x7f2bec069450 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 --2- 172.21.15.133:0/4256141988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf8006150 0x7f2c0c153d50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 -- 172.21.15.133:0/4256141988 >> 172.21.15.133:0/4256141988 conn(0x7f2c0c067000 msgr2=0x7f2c0c0cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 -- 172.21.15.133:0/4256141988 shutdown_connections 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.140+0000 7f2c10d02640 1 -- 172.21.15.133:0/4256141988 wait complete. 2024-04-25T10:02:22.161 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T10:02:21.712799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219388,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755332,"statfs":{"total":773094113280,"available":772869459968,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004689"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631430+0000","last_change":"2024-04-25T10:01:50.953534+0000","last_active":"2024-04-25T10:01:57.631430+0000","last_peered":"2024-04-25T10:01:57.631430+0000","last_clean":"2024-04-25T10:01:57.631430+0000","last_became_active":"2024-04-25T10:01:50.953054+0000","last_became_peered":"2024-04-25T10:01:50.953054+0000","last_unstale":"2024-04-25T10:01:57.631430+0000","last_undegraded":"2024-04-25T10:01:57.631430+0000","last_fullsized":"2024-04-25T10:01:57.631430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:19.695230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633973+0000","last_change":"2024-04-25T10:01:51.958018+0000","last_active":"2024-04-25T10:01:57.633973+0000","last_peered":"2024-04-25T10:01:57.633973+0000","last_clean":"2024-04-25T10:01:57.633973+0000","last_became_active":"2024-04-25T10:01:51.957717+0000","last_became_peered":"2024-04-25T10:01:51.957717+0000","last_unstale":"2024-04-25T10:01:57.633973+0000","last_undegraded":"2024-04-25T10:01:57.633973+0000","last_fullsized":"2024-04-25T10:01:57.633973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:55.810277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631489+0000","last_change":"2024-04-25T10:01:56.111100+0000","last_active":"2024-04-25T10:01:57.631489+0000","last_peered":"2024-04-25T10:01:57.631489+0000","last_clean":"2024-04-25T10:01:57.631489+0000","last_became_active":"2024-04-25T10:01:56.110327+0000","last_became_peered":"2024-04-25T10:01:56.110327+0000","last_unstale":"2024-04-25T10:01:57.631489+0000","last_undegraded":"2024-04-25T10:01:57.631489+0000","last_fullsized":"2024-04-25T10:01:57.631489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634369+0000","last_change":"2024-04-25T10:01:50.940954+0000","last_active":"2024-04-25T10:01:57.634369+0000","last_peered":"2024-04-25T10:01:57.634369+0000","last_clean":"2024-04-25T10:01:57.634369+0000","last_became_active":"2024-04-25T10:01:50.940601+0000","last_became_peered":"2024-04-25T10:01:50.940601+0000","last_unstale":"2024-04-25T10:01:57.634369+0000","last_undegraded":"2024-04-25T10:01:57.634369+0000","last_fullsized":"2024-04-25T10:01:57.634369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631026+0000","last_change":"2024-04-25T10:01:51.994632+0000","last_active":"2024-04-25T10:01:57.631026+0000","last_peered":"2024-04-25T10:01:57.631026+0000","last_clean":"2024-04-25T10:01:57.631026+0000","last_became_active":"2024-04-25T10:01:51.994229+0000","last_became_peered":"2024-04-25T10:01:51.994229+0000","last_unstale":"2024-04-25T10:01:57.631026+0000","last_undegraded":"2024-04-25T10:01:57.631026+0000","last_fullsized":"2024-04-25T10:01:57.631026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005703+0000","last_change":"2024-04-25T10:01:56.024604+0000","last_active":"2024-04-25T10:01:57.005703+0000","last_peered":"2024-04-25T10:01:57.005703+0000","last_clean":"2024-04-25T10:01:57.005703+0000","last_became_active":"2024-04-25T10:01:56.023667+0000","last_became_peered":"2024-04-25T10:01:56.023667+0000","last_unstale":"2024-04-25T10:01:57.005703+0000","last_undegraded":"2024-04-25T10:01:57.005703+0000","last_fullsized":"2024-04-25T10:01:57.005703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:50.018687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348063+0000","last_change":"2024-04-25T10:01:50.957056+0000","last_active":"2024-04-25T10:01:57.348063+0000","last_peered":"2024-04-25T10:01:57.348063+0000","last_clean":"2024-04-25T10:01:57.348063+0000","last_became_active":"2024-04-25T10:01:50.956522+0000","last_became_peered":"2024-04-25T10:01:50.956522+0000","last_unstale":"2024-04-25T10:01:57.348063+0000","last_undegraded":"2024-04-25T10:01:57.348063+0000","last_fullsized":"2024-04-25T10:01:57.348063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:25.085344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101183+0000","last_change":"2024-04-25T10:01:51.987821+0000","last_active":"2024-04-25T10:01:57.101183+0000","last_peered":"2024-04-25T10:01:57.101183+0000","last_clean":"2024-04-25T10:01:57.101183+0000","last_became_active":"2024-04-25T10:01:51.978435+0000","last_became_peered":"2024-04-25T10:01:51.978435+0000","last_unstale":"2024-04-25T10:01:57.101183+0000","last_undegraded":"2024-04-25T10:01:57.101183+0000","last_fullsized":"2024-04-25T10:01:57.101183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.714429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009007+0000","last_change":"2024-04-25T10:01:56.024538+0000","last_active":"2024-04-25T10:01:57.009007+0000","last_peered":"2024-04-25T10:01:57.009007+0000","last_clean":"2024-04-25T10:01:57.009007+0000","last_became_active":"2024-04-25T10:01:56.023560+0000","last_became_peered":"2024-04-25T10:01:56.023560+0000","last_unstale":"2024-04-25T10:01:57.009007+0000","last_undegraded":"2024-04-25T10:01:57.009007+0000","last_fullsized":"2024-04-25T10:01:57.009007+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.432097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347087+0000","last_change":"2024-04-25T10:01:50.954944+0000","last_active":"2024-04-25T10:01:57.347087+0000","last_peered":"2024-04-25T10:01:57.347087+0000","last_clean":"2024-04-25T10:01:57.347087+0000","last_became_active":"2024-04-25T10:01:50.954098+0000","last_became_peered":"2024-04-25T10:01:50.954098+0000","last_unstale":"2024-04-25T10:01:57.347087+0000","last_undegraded":"2024-04-25T10:01:57.347087+0000","last_fullsized":"2024-04-25T10:01:57.347087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:51.011075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101791+0000","last_change":"2024-04-25T10:01:51.976877+0000","last_active":"2024-04-25T10:01:57.101791+0000","last_peered":"2024-04-25T10:01:57.101791+0000","last_clean":"2024-04-25T10:01:57.101791+0000","last_became_active":"2024-04-25T10:01:51.967649+0000","last_became_peered":"2024-04-25T10:01:51.967649+0000","last_unstale":"2024-04-25T10:01:57.101791+0000","last_undegraded":"2024-04-25T10:01:57.101791+0000","last_fullsized":"2024-04-25T10:01:57.101791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006978+0000","last_change":"2024-04-25T10:01:56.018179+0000","last_active":"2024-04-25T10:01:57.006978+0000","last_peered":"2024-04-25T10:01:57.006978+0000","last_clean":"2024-04-25T10:01:57.006978+0000","last_became_active":"2024-04-25T10:01:56.017199+0000","last_became_peered":"2024-04-25T10:01:56.017199+0000","last_unstale":"2024-04-25T10:01:57.006978+0000","last_undegraded":"2024-04-25T10:01:57.006978+0000","last_fullsized":"2024-04-25T10:01:57.006978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:26.814142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007115+0000","last_change":"2024-04-25T10:01:51.935248+0000","last_active":"2024-04-25T10:01:57.007115+0000","last_peered":"2024-04-25T10:01:57.007115+0000","last_clean":"2024-04-25T10:01:57.007115+0000","last_became_active":"2024-04-25T10:01:51.932825+0000","last_became_peered":"2024-04-25T10:01:51.932825+0000","last_unstale":"2024-04-25T10:01:57.007115+0000","last_undegraded":"2024-04-25T10:01:57.007115+0000","last_fullsized":"2024-04-25T10:01:57.007115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347924+0000","last_change":"2024-04-25T10:01:50.956325+0000","last_active":"2024-04-25T10:01:57.347924+0000","last_peered":"2024-04-25T10:01:57.347924+0000","last_clean":"2024-04-25T10:01:57.347924+0000","last_became_active":"2024-04-25T10:01:50.954154+0000","last_became_peered":"2024-04-25T10:01:50.954154+0000","last_unstale":"2024-04-25T10:01:57.347924+0000","last_undegraded":"2024-04-25T10:01:57.347924+0000","last_fullsized":"2024-04-25T10:01:57.347924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:32.182247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008038+0000","last_change":"2024-04-25T10:01:56.024877+0000","last_active":"2024-04-25T10:01:57.008038+0000","last_peered":"2024-04-25T10:01:57.008038+0000","last_clean":"2024-04-25T10:01:57.008038+0000","last_became_active":"2024-04-25T10:01:56.023788+0000","last_became_peered":"2024-04-25T10:01:56.023788+0000","last_unstale":"2024-04-25T10:01:57.008038+0000","last_undegraded":"2024-04-25T10:01:57.008038+0000","last_fullsized":"2024-04-25T10:01:57.008038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:52.776865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633155+0000","last_change":"2024-04-25T10:01:51.952977+0000","last_active":"2024-04-25T10:01:57.633155+0000","last_peered":"2024-04-25T10:01:57.633155+0000","last_clean":"2024-04-25T10:01:57.633155+0000","last_became_active":"2024-04-25T10:01:51.951977+0000","last_became_peered":"2024-04-25T10:01:51.951977+0000","last_unstale":"2024-04-25T10:01:57.633155+0000","last_undegraded":"2024-04-25T10:01:57.633155+0000","last_fullsized":"2024-04-25T10:01:57.633155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.714429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630222+0000","last_change":"2024-04-25T10:01:50.934823+0000","last_active":"2024-04-25T10:01:57.630222+0000","last_peered":"2024-04-25T10:01:57.630222+0000","last_clean":"2024-04-25T10:01:57.630222+0000","last_became_active":"2024-04-25T10:01:50.934535+0000","last_became_peered":"2024-04-25T10:01:50.934535+0000","last_unstale":"2024-04-25T10:01:57.630222+0000","last_undegraded":"2024-04-25T10:01:57.630222+0000","last_fullsized":"2024-04-25T10:01:57.630222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007823+0000","last_change":"2024-04-25T10:01:56.017858+0000","last_active":"2024-04-25T10:01:57.007823+0000","last_peered":"2024-04-25T10:01:57.007823+0000","last_clean":"2024-04-25T10:01:57.007823+0000","last_became_active":"2024-04-25T10:01:56.016904+0000","last_became_peered":"2024-04-25T10:01:56.016904+0000","last_unstale":"2024-04-25T10:01:57.007823+0000","last_undegraded":"2024-04-25T10:01:57.007823+0000","last_fullsized":"2024-04-25T10:01:57.007823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:38.512242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631872+0000","last_change":"2024-04-25T10:01:51.948751+0000","last_active":"2024-04-25T10:01:57.631872+0000","last_peered":"2024-04-25T10:01:57.631872+0000","last_clean":"2024-04-25T10:01:57.631872+0000","last_became_active":"2024-04-25T10:01:51.946530+0000","last_became_peered":"2024-04-25T10:01:51.946530+0000","last_unstale":"2024-04-25T10:01:57.631872+0000","last_undegraded":"2024-04-25T10:01:57.631872+0000","last_fullsized":"2024-04-25T10:01:57.631872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627984+0000","last_change":"2024-04-25T10:01:50.953620+0000","last_active":"2024-04-25T10:01:57.627984+0000","last_peered":"2024-04-25T10:01:57.627984+0000","last_clean":"2024-04-25T10:01:57.627984+0000","last_became_active":"2024-04-25T10:01:50.953363+0000","last_became_peered":"2024-04-25T10:01:50.953363+0000","last_unstale":"2024-04-25T10:01:57.627984+0000","last_undegraded":"2024-04-25T10:01:57.627984+0000","last_fullsized":"2024-04-25T10:01:57.627984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629574+0000","last_change":"2024-04-25T10:01:56.111935+0000","last_active":"2024-04-25T10:01:57.629574+0000","last_peered":"2024-04-25T10:01:57.629574+0000","last_clean":"2024-04-25T10:01:57.629574+0000","last_became_active":"2024-04-25T10:01:56.111568+0000","last_became_peered":"2024-04-25T10:01:56.111568+0000","last_unstale":"2024-04-25T10:01:57.629574+0000","last_undegraded":"2024-04-25T10:01:57.629574+0000","last_fullsized":"2024-04-25T10:01:57.629574+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:18.254251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006627+0000","last_change":"2024-04-25T10:01:51.965350+0000","last_active":"2024-04-25T10:01:57.006627+0000","last_peered":"2024-04-25T10:01:57.006627+0000","last_clean":"2024-04-25T10:01:57.006627+0000","last_became_active":"2024-04-25T10:01:51.964456+0000","last_became_peered":"2024-04-25T10:01:51.964456+0000","last_unstale":"2024-04-25T10:01:57.006627+0000","last_undegraded":"2024-04-25T10:01:57.006627+0000","last_fullsized":"2024-04-25T10:01:57.006627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:59.032520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346651+0000","last_change":"2024-04-25T10:01:50.954332+0000","last_active":"2024-04-25T10:01:57.346651+0000","last_peered":"2024-04-25T10:01:57.346651+0000","last_clean":"2024-04-25T10:01:57.346651+0000","last_became_active":"2024-04-25T10:01:50.953854+0000","last_became_peered":"2024-04-25T10:01:50.953854+0000","last_unstale":"2024-04-25T10:01:57.346651+0000","last_undegraded":"2024-04-25T10:01:57.346651+0000","last_fullsized":"2024-04-25T10:01:57.346651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:38.278178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008718+0000","last_change":"2024-04-25T10:01:56.017972+0000","last_active":"2024-04-25T10:01:57.008718+0000","last_peered":"2024-04-25T10:01:57.008718+0000","last_clean":"2024-04-25T10:01:57.008718+0000","last_became_active":"2024-04-25T10:01:56.016953+0000","last_became_peered":"2024-04-25T10:01:56.016953+0000","last_unstale":"2024-04-25T10:01:57.008718+0000","last_undegraded":"2024-04-25T10:01:57.008718+0000","last_fullsized":"2024-04-25T10:01:57.008718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:59.872712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630912+0000","last_change":"2024-04-25T10:01:51.944937+0000","last_active":"2024-04-25T10:01:57.630912+0000","last_peered":"2024-04-25T10:01:57.630912+0000","last_clean":"2024-04-25T10:01:57.630912+0000","last_became_active":"2024-04-25T10:01:51.944599+0000","last_became_peered":"2024-04-25T10:01:51.944599+0000","last_unstale":"2024-04-25T10:01:57.630912+0000","last_undegraded":"2024-04-25T10:01:57.630912+0000","last_fullsized":"2024-04-25T10:01:57.630912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008241+0000","last_change":"2024-04-25T10:01:50.955661+0000","last_active":"2024-04-25T10:01:57.008241+0000","last_peered":"2024-04-25T10:01:57.008241+0000","last_clean":"2024-04-25T10:01:57.008241+0000","last_became_active":"2024-04-25T10:01:50.954955+0000","last_became_peered":"2024-04-25T10:01:50.954955+0000","last_unstale":"2024-04-25T10:01:57.008241+0000","last_undegraded":"2024-04-25T10:01:57.008241+0000","last_fullsized":"2024-04-25T10:01:57.008241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:33.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006345+0000","last_change":"2024-04-25T10:01:56.108996+0000","last_active":"2024-04-25T10:01:57.006345+0000","last_peered":"2024-04-25T10:01:57.006345+0000","last_clean":"2024-04-25T10:01:57.006345+0000","last_became_active":"2024-04-25T10:01:56.107964+0000","last_became_peered":"2024-04-25T10:01:56.107964+0000","last_unstale":"2024-04-25T10:01:57.006345+0000","last_undegraded":"2024-04-25T10:01:57.006345+0000","last_fullsized":"2024-04-25T10:01:57.006345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.917422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634815+0000","last_change":"2024-04-25T10:01:51.959144+0000","last_active":"2024-04-25T10:01:57.634815+0000","last_peered":"2024-04-25T10:01:57.634815+0000","last_clean":"2024-04-25T10:01:57.634815+0000","last_became_active":"2024-04-25T10:01:51.958032+0000","last_became_peered":"2024-04-25T10:01:51.958032+0000","last_unstale":"2024-04-25T10:01:57.634815+0000","last_undegraded":"2024-04-25T10:01:57.634815+0000","last_fullsized":"2024-04-25T10:01:57.634815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:34.449807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629847+0000","last_change":"2024-04-25T10:01:50.951871+0000","last_active":"2024-04-25T10:01:57.629847+0000","last_peered":"2024-04-25T10:01:57.629847+0000","last_clean":"2024-04-25T10:01:57.629847+0000","last_became_active":"2024-04-25T10:01:50.951527+0000","last_became_peered":"2024-04-25T10:01:50.951527+0000","last_unstale":"2024-04-25T10:01:57.629847+0000","last_undegraded":"2024-04-25T10:01:57.629847+0000","last_fullsized":"2024-04-25T10:01:57.629847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:38.278178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630769+0000","last_change":"2024-04-25T10:01:56.101023+0000","last_active":"2024-04-25T10:01:57.630769+0000","last_peered":"2024-04-25T10:01:57.630769+0000","last_clean":"2024-04-25T10:01:57.630769+0000","last_became_active":"2024-04-25T10:01:56.100400+0000","last_became_peered":"2024-04-25T10:01:56.100400+0000","last_unstale":"2024-04-25T10:01:57.630769+0000","last_undegraded":"2024-04-25T10:01:57.630769+0000","last_fullsized":"2024-04-25T10:01:57.630769+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:40.633384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628600+0000","last_change":"2024-04-25T10:01:51.951713+0000","last_active":"2024-04-25T10:01:57.628600+0000","last_peered":"2024-04-25T10:01:57.628600+0000","last_clean":"2024-04-25T10:01:57.628600+0000","last_became_active":"2024-04-25T10:01:51.951438+0000","last_became_peered":"2024-04-25T10:01:51.951438+0000","last_unstale":"2024-04-25T10:01:57.628600+0000","last_undegraded":"2024-04-25T10:01:57.628600+0000","last_fullsized":"2024-04-25T10:01:57.628600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348330+0000","last_change":"2024-04-25T10:01:50.956823+0000","last_active":"2024-04-25T10:01:57.348330+0000","last_peered":"2024-04-25T10:01:57.348330+0000","last_clean":"2024-04-25T10:01:57.348330+0000","last_became_active":"2024-04-25T10:01:50.956170+0000","last_became_peered":"2024-04-25T10:01:50.956170+0000","last_unstale":"2024-04-25T10:01:57.348330+0000","last_undegraded":"2024-04-25T10:01:57.348330+0000","last_fullsized":"2024-04-25T10:01:57.348330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:43.784003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629539+0000","last_change":"2024-04-25T10:01:56.101938+0000","last_active":"2024-04-25T10:01:57.629539+0000","last_peered":"2024-04-25T10:01:57.629539+0000","last_clean":"2024-04-25T10:01:57.629539+0000","last_became_active":"2024-04-25T10:01:56.101501+0000","last_became_peered":"2024-04-25T10:01:56.101501+0000","last_unstale":"2024-04-25T10:01:57.629539+0000","last_undegraded":"2024-04-25T10:01:57.629539+0000","last_fullsized":"2024-04-25T10:01:57.629539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:03.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.102065+0000","last_change":"2024-04-25T10:01:51.977453+0000","last_active":"2024-04-25T10:01:57.102065+0000","last_peered":"2024-04-25T10:01:57.102065+0000","last_clean":"2024-04-25T10:01:57.102065+0000","last_became_active":"2024-04-25T10:01:51.968804+0000","last_became_peered":"2024-04-25T10:01:51.968804+0000","last_unstale":"2024-04-25T10:01:57.102065+0000","last_undegraded":"2024-04-25T10:01:57.102065+0000","last_fullsized":"2024-04-25T10:01:57.102065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:34.449807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629477+0000","last_change":"2024-04-25T10:01:50.934637+0000","last_active":"2024-04-25T10:01:57.629477+0000","last_peered":"2024-04-25T10:01:57.629477+0000","last_clean":"2024-04-25T10:01:57.629477+0000","last_became_active":"2024-04-25T10:01:50.933810+0000","last_became_peered":"2024-04-25T10:01:50.933810+0000","last_unstale":"2024-04-25T10:01:57.629477+0000","last_undegraded":"2024-04-25T10:01:57.629477+0000","last_fullsized":"2024-04-25T10:01:57.629477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:21.571798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627408+0000","last_change":"2024-04-25T10:01:56.030222+0000","last_active":"2024-04-25T10:01:57.627408+0000","last_peered":"2024-04-25T10:01:57.627408+0000","last_clean":"2024-04-25T10:01:57.627408+0000","last_became_active":"2024-04-25T10:01:56.023359+0000","last_became_peered":"2024-04-25T10:01:56.023359+0000","last_unstale":"2024-04-25T10:01:57.627408+0000","last_undegraded":"2024-04-25T10:01:57.627408+0000","last_fullsized":"2024-04-25T10:01:57.627408+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009639+0000","last_change":"2024-04-25T10:01:51.941059+0000","last_active":"2024-04-25T10:01:57.009639+0000","last_peered":"2024-04-25T10:01:57.009639+0000","last_clean":"2024-04-25T10:01:57.009639+0000","last_became_active":"2024-04-25T10:01:51.938978+0000","last_became_peered":"2024-04-25T10:01:51.938978+0000","last_unstale":"2024-04-25T10:01:57.009639+0000","last_undegraded":"2024-04-25T10:01:57.009639+0000","last_fullsized":"2024-04-25T10:01:57.009639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:39.550415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101806+0000","last_change":"2024-04-25T10:01:50.952791+0000","last_active":"2024-04-25T10:01:57.101806+0000","last_peered":"2024-04-25T10:01:57.101806+0000","last_clean":"2024-04-25T10:01:57.101806+0000","last_became_active":"2024-04-25T10:01:50.950155+0000","last_became_peered":"2024-04-25T10:01:50.950155+0000","last_unstale":"2024-04-25T10:01:57.101806+0000","last_undegraded":"2024-04-25T10:01:57.101806+0000","last_fullsized":"2024-04-25T10:01:57.101806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629324+0000","last_change":"2024-04-25T10:01:56.016879+0000","last_active":"2024-04-25T10:01:57.629324+0000","last_peered":"2024-04-25T10:01:57.629324+0000","last_clean":"2024-04-25T10:01:57.629324+0000","last_became_active":"2024-04-25T10:01:56.016202+0000","last_became_peered":"2024-04-25T10:01:56.016202+0000","last_unstale":"2024-04-25T10:01:57.629324+0000","last_undegraded":"2024-04-25T10:01:57.629324+0000","last_fullsized":"2024-04-25T10:01:57.629324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:48.574592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627562+0000","last_change":"2024-04-25T10:01:51.952368+0000","last_active":"2024-04-25T10:01:57.627562+0000","last_peered":"2024-04-25T10:01:57.627562+0000","last_clean":"2024-04-25T10:01:57.627562+0000","last_became_active":"2024-04-25T10:01:51.952095+0000","last_became_peered":"2024-04-25T10:01:51.952095+0000","last_unstale":"2024-04-25T10:01:57.627562+0000","last_undegraded":"2024-04-25T10:01:57.627562+0000","last_fullsized":"2024-04-25T10:01:57.627562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004117+0000","last_change":"2024-04-25T10:01:50.940514+0000","last_active":"2024-04-25T10:01:57.004117+0000","last_peered":"2024-04-25T10:01:57.004117+0000","last_clean":"2024-04-25T10:01:57.004117+0000","last_became_active":"2024-04-25T10:01:50.935395+0000","last_became_peered":"2024-04-25T10:01:50.935395+0000","last_unstale":"2024-04-25T10:01:57.004117+0000","last_undegraded":"2024-04-25T10:01:57.004117+0000","last_fullsized":"2024-04-25T10:01:57.004117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.010630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629857+0000","last_change":"2024-04-25T10:01:56.015853+0000","last_active":"2024-04-25T10:01:57.629857+0000","last_peered":"2024-04-25T10:01:57.629857+0000","last_clean":"2024-04-25T10:01:57.629857+0000","last_became_active":"2024-04-25T10:01:56.015382+0000","last_became_peered":"2024-04-25T10:01:56.015382+0000","last_unstale":"2024-04-25T10:01:57.629857+0000","last_undegraded":"2024-04-25T10:01:57.629857+0000","last_fullsized":"2024-04-25T10:01:57.629857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:32.289340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628148+0000","last_change":"2024-04-25T10:01:51.939122+0000","last_active":"2024-04-25T10:01:57.628148+0000","last_peered":"2024-04-25T10:01:57.628148+0000","last_clean":"2024-04-25T10:01:57.628148+0000","last_became_active":"2024-04-25T10:01:51.938342+0000","last_became_peered":"2024-04-25T10:01:51.938342+0000","last_unstale":"2024-04-25T10:01:57.628148+0000","last_undegraded":"2024-04-25T10:01:57.628148+0000","last_fullsized":"2024-04-25T10:01:57.628148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:33.192217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346666+0000","last_change":"2024-04-25T10:01:50.957985+0000","last_active":"2024-04-25T10:01:57.346666+0000","last_peered":"2024-04-25T10:01:57.346666+0000","last_clean":"2024-04-25T10:01:57.346666+0000","last_became_active":"2024-04-25T10:01:50.957155+0000","last_became_peered":"2024-04-25T10:01:50.957155+0000","last_unstale":"2024-04-25T10:01:57.346666+0000","last_undegraded":"2024-04-25T10:01:57.346666+0000","last_fullsized":"2024-04-25T10:01:57.346666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:21.571798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631051+0000","last_change":"2024-04-25T10:01:56.105198+0000","last_active":"2024-04-25T10:01:57.631051+0000","last_peered":"2024-04-25T10:01:57.631051+0000","last_clean":"2024-04-25T10:01:57.631051+0000","last_became_active":"2024-04-25T10:01:56.104858+0000","last_became_peered":"2024-04-25T10:01:56.104858+0000","last_unstale":"2024-04-25T10:01:57.631051+0000","last_undegraded":"2024-04-25T10:01:57.631051+0000","last_fullsized":"2024-04-25T10:01:57.631051+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:48.574592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632717+0000","last_change":"2024-04-25T10:01:51.994401+0000","last_active":"2024-04-25T10:01:57.632717+0000","last_peered":"2024-04-25T10:01:57.632717+0000","last_clean":"2024-04-25T10:01:57.632717+0000","last_became_active":"2024-04-25T10:01:51.994086+0000","last_became_peered":"2024-04-25T10:01:51.994086+0000","last_unstale":"2024-04-25T10:01:57.632717+0000","last_undegraded":"2024-04-25T10:01:57.632717+0000","last_fullsized":"2024-04-25T10:01:57.632717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101375+0000","last_change":"2024-04-25T10:01:50.950324+0000","last_active":"2024-04-25T10:01:57.101375+0000","last_peered":"2024-04-25T10:01:57.101375+0000","last_clean":"2024-04-25T10:01:57.101375+0000","last_became_active":"2024-04-25T10:01:50.949810+0000","last_became_peered":"2024-04-25T10:01:50.949810+0000","last_unstale":"2024-04-25T10:01:57.101375+0000","last_undegraded":"2024-04-25T10:01:57.101375+0000","last_fullsized":"2024-04-25T10:01:57.101375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":177,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346935+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:01:57.346935+0000","last_peered":"2024-04-25T10:01:57.346935+0000","last_clean":"2024-04-25T10:01:57.346935+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:01:57.346935+0000","last_undegraded":"2024-04-25T10:01:57.346935+0000","last_fullsized":"2024-04-25T10:01:57.346935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007399+0000","last_change":"2024-04-25T10:01:56.108134+0000","last_active":"2024-04-25T10:01:57.007399+0000","last_peered":"2024-04-25T10:01:57.007399+0000","last_clean":"2024-04-25T10:01:57.007399+0000","last_became_active":"2024-04-25T10:01:56.107123+0000","last_became_peered":"2024-04-25T10:01:56.107123+0000","last_unstale":"2024-04-25T10:01:57.007399+0000","last_undegraded":"2024-04-25T10:01:57.007399+0000","last_fullsized":"2024-04-25T10:01:57.007399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:03.161404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347779+0000","last_change":"2024-04-25T10:01:51.983906+0000","last_active":"2024-04-25T10:01:57.347779+0000","last_peered":"2024-04-25T10:01:57.347779+0000","last_clean":"2024-04-25T10:01:57.347779+0000","last_became_active":"2024-04-25T10:01:51.982118+0000","last_became_peered":"2024-04-25T10:01:51.982118+0000","last_unstale":"2024-04-25T10:01:57.347779+0000","last_undegraded":"2024-04-25T10:01:57.347779+0000","last_fullsized":"2024-04-25T10:01:57.347779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:14.191816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004595+0000","last_change":"2024-04-25T10:01:50.946054+0000","last_active":"2024-04-25T10:01:57.004595+0000","last_peered":"2024-04-25T10:01:57.004595+0000","last_clean":"2024-04-25T10:01:57.004595+0000","last_became_active":"2024-04-25T10:01:50.945744+0000","last_became_peered":"2024-04-25T10:01:50.945744+0000","last_unstale":"2024-04-25T10:01:57.004595+0000","last_undegraded":"2024-04-25T10:01:57.004595+0000","last_fullsized":"2024-04-25T10:01:57.004595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006974+0000","last_change":"2024-04-25T10:01:56.104590+0000","last_active":"2024-04-25T10:01:57.006974+0000","last_peered":"2024-04-25T10:01:57.006974+0000","last_clean":"2024-04-25T10:01:57.006974+0000","last_became_active":"2024-04-25T10:01:56.103498+0000","last_became_peered":"2024-04-25T10:01:56.103498+0000","last_unstale":"2024-04-25T10:01:57.006974+0000","last_undegraded":"2024-04-25T10:01:57.006974+0000","last_fullsized":"2024-04-25T10:01:57.006974+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101453+0000","last_change":"2024-04-25T10:01:51.968651+0000","last_active":"2024-04-25T10:01:57.101453+0000","last_peered":"2024-04-25T10:01:57.101453+0000","last_clean":"2024-04-25T10:01:57.101453+0000","last_became_active":"2024-04-25T10:01:51.962327+0000","last_became_peered":"2024-04-25T10:01:51.962327+0000","last_unstale":"2024-04-25T10:01:57.101453+0000","last_undegraded":"2024-04-25T10:01:57.101453+0000","last_fullsized":"2024-04-25T10:01:57.101453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:55.810277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629450+0000","last_change":"2024-04-25T10:01:50.942416+0000","last_active":"2024-04-25T10:01:57.629450+0000","last_peered":"2024-04-25T10:01:57.629450+0000","last_clean":"2024-04-25T10:01:57.629450+0000","last_became_active":"2024-04-25T10:01:50.942009+0000","last_became_peered":"2024-04-25T10:01:50.942009+0000","last_unstale":"2024-04-25T10:01:57.629450+0000","last_undegraded":"2024-04-25T10:01:57.629450+0000","last_fullsized":"2024-04-25T10:01:57.629450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:25.085344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630028+0000","last_change":"2024-04-25T10:01:56.023691+0000","last_active":"2024-04-25T10:01:57.630028+0000","last_peered":"2024-04-25T10:01:57.630028+0000","last_clean":"2024-04-25T10:01:57.630028+0000","last_became_active":"2024-04-25T10:01:56.022988+0000","last_became_peered":"2024-04-25T10:01:56.022988+0000","last_unstale":"2024-04-25T10:01:57.630028+0000","last_undegraded":"2024-04-25T10:01:57.630028+0000","last_fullsized":"2024-04-25T10:01:57.630028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347762+0000","last_change":"2024-04-25T10:01:51.987406+0000","last_active":"2024-04-25T10:01:57.347762+0000","last_peered":"2024-04-25T10:01:57.347762+0000","last_clean":"2024-04-25T10:01:57.347762+0000","last_became_active":"2024-04-25T10:01:51.986930+0000","last_became_peered":"2024-04-25T10:01:51.986930+0000","last_unstale":"2024-04-25T10:01:57.347762+0000","last_undegraded":"2024-04-25T10:01:57.347762+0000","last_fullsized":"2024-04-25T10:01:57.347762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:27.239681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004599+0000","last_change":"2024-04-25T10:01:50.932604+0000","last_active":"2024-04-25T10:01:57.004599+0000","last_peered":"2024-04-25T10:01:57.004599+0000","last_clean":"2024-04-25T10:01:57.004599+0000","last_became_active":"2024-04-25T10:01:50.932205+0000","last_became_peered":"2024-04-25T10:01:50.932205+0000","last_unstale":"2024-04-25T10:01:57.004599+0000","last_undegraded":"2024-04-25T10:01:57.004599+0000","last_fullsized":"2024-04-25T10:01:57.004599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:35.306064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100705+0000","last_change":"2024-04-25T10:01:56.105972+0000","last_active":"2024-04-25T10:01:57.100705+0000","last_peered":"2024-04-25T10:01:57.100705+0000","last_clean":"2024-04-25T10:01:57.100705+0000","last_became_active":"2024-04-25T10:01:56.104834+0000","last_became_peered":"2024-04-25T10:01:56.104834+0000","last_unstale":"2024-04-25T10:01:57.100705+0000","last_undegraded":"2024-04-25T10:01:57.100705+0000","last_fullsized":"2024-04-25T10:01:57.100705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:01.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347690+0000","last_change":"2024-04-25T10:01:51.991881+0000","last_active":"2024-04-25T10:01:57.347690+0000","last_peered":"2024-04-25T10:01:57.347690+0000","last_clean":"2024-04-25T10:01:57.347690+0000","last_became_active":"2024-04-25T10:01:51.991266+0000","last_became_peered":"2024-04-25T10:01:51.991266+0000","last_unstale":"2024-04-25T10:01:57.347690+0000","last_undegraded":"2024-04-25T10:01:57.347690+0000","last_fullsized":"2024-04-25T10:01:57.347690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:39.074042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009212+0000","last_change":"2024-04-25T10:01:50.945437+0000","last_active":"2024-04-25T10:01:57.009212+0000","last_peered":"2024-04-25T10:01:57.009212+0000","last_clean":"2024-04-25T10:01:57.009212+0000","last_became_active":"2024-04-25T10:01:50.944786+0000","last_became_peered":"2024-04-25T10:01:50.944786+0000","last_unstale":"2024-04-25T10:01:57.009212+0000","last_undegraded":"2024-04-25T10:01:57.009212+0000","last_fullsized":"2024-04-25T10:01:57.009212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:53.181549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005662+0000","last_change":"2024-04-25T10:01:56.000803+0000","last_active":"2024-04-25T10:01:57.005662+0000","last_peered":"2024-04-25T10:01:57.005662+0000","last_clean":"2024-04-25T10:01:57.005662+0000","last_became_active":"2024-04-25T10:01:55.999071+0000","last_became_peered":"2024-04-25T10:01:55.999071+0000","last_unstale":"2024-04-25T10:01:57.005662+0000","last_undegraded":"2024-04-25T10:01:57.005662+0000","last_fullsized":"2024-04-25T10:01:57.005662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:50.609054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347419+0000","last_change":"2024-04-25T10:01:51.982017+0000","last_active":"2024-04-25T10:01:57.347419+0000","last_peered":"2024-04-25T10:01:57.347419+0000","last_clean":"2024-04-25T10:01:57.347419+0000","last_became_active":"2024-04-25T10:01:51.981340+0000","last_became_peered":"2024-04-25T10:01:51.981340+0000","last_unstale":"2024-04-25T10:01:57.347419+0000","last_undegraded":"2024-04-25T10:01:57.347419+0000","last_fullsized":"2024-04-25T10:01:57.347419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:22.999554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004023+0000","last_change":"2024-04-25T10:01:50.934868+0000","last_active":"2024-04-25T10:01:57.004023+0000","last_peered":"2024-04-25T10:01:57.004023+0000","last_clean":"2024-04-25T10:01:57.004023+0000","last_became_active":"2024-04-25T10:01:50.934560+0000","last_became_peered":"2024-04-25T10:01:50.934560+0000","last_unstale":"2024-04-25T10:01:57.004023+0000","last_undegraded":"2024-04-25T10:01:57.004023+0000","last_fullsized":"2024-04-25T10:01:57.004023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630724+0000","last_change":"2024-04-25T10:01:56.012427+0000","last_active":"2024-04-25T10:01:57.630724+0000","last_peered":"2024-04-25T10:01:57.630724+0000","last_clean":"2024-04-25T10:01:57.630724+0000","last_became_active":"2024-04-25T10:01:56.008358+0000","last_became_peered":"2024-04-25T10:01:56.008358+0000","last_unstale":"2024-04-25T10:01:57.630724+0000","last_undegraded":"2024-04-25T10:01:57.630724+0000","last_fullsized":"2024-04-25T10:01:57.630724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:56.864362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006288+0000","last_change":"2024-04-25T10:01:51.947557+0000","last_active":"2024-04-25T10:01:57.006288+0000","last_peered":"2024-04-25T10:01:57.006288+0000","last_clean":"2024-04-25T10:01:57.006288+0000","last_became_active":"2024-04-25T10:01:51.942206+0000","last_became_peered":"2024-04-25T10:01:51.942206+0000","last_unstale":"2024-04-25T10:01:57.006288+0000","last_undegraded":"2024-04-25T10:01:57.006288+0000","last_fullsized":"2024-04-25T10:01:57.006288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.006959+0000","last_change":"2024-04-25T10:01:50.930663+0000","last_active":"2024-04-25T10:01:57.006959+0000","last_peered":"2024-04-25T10:01:57.006959+0000","last_clean":"2024-04-25T10:01:57.006959+0000","last_became_active":"2024-04-25T10:01:50.930224+0000","last_became_peered":"2024-04-25T10:01:50.930224+0000","last_unstale":"2024-04-25T10:01:57.006959+0000","last_undegraded":"2024-04-25T10:01:57.006959+0000","last_fullsized":"2024-04-25T10:01:57.006959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:25.317835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634460+0000","last_change":"2024-04-25T10:01:56.110077+0000","last_active":"2024-04-25T10:01:57.634460+0000","last_peered":"2024-04-25T10:01:57.634460+0000","last_clean":"2024-04-25T10:01:57.634460+0000","last_became_active":"2024-04-25T10:01:56.109686+0000","last_became_peered":"2024-04-25T10:01:56.109686+0000","last_unstale":"2024-04-25T10:01:57.634460+0000","last_undegraded":"2024-04-25T10:01:57.634460+0000","last_fullsized":"2024-04-25T10:01:57.634460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:20.441998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347395+0000","last_change":"2024-04-25T10:01:51.981399+0000","last_active":"2024-04-25T10:01:57.347395+0000","last_peered":"2024-04-25T10:01:57.347395+0000","last_clean":"2024-04-25T10:01:57.347395+0000","last_became_active":"2024-04-25T10:01:51.980512+0000","last_became_peered":"2024-04-25T10:01:51.980512+0000","last_unstale":"2024-04-25T10:01:57.347395+0000","last_undegraded":"2024-04-25T10:01:57.347395+0000","last_fullsized":"2024-04-25T10:01:57.347395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:52.801927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101702+0000","last_change":"2024-04-25T10:01:50.949394+0000","last_active":"2024-04-25T10:01:57.101702+0000","last_peered":"2024-04-25T10:01:57.101702+0000","last_clean":"2024-04-25T10:01:57.101702+0000","last_became_active":"2024-04-25T10:01:50.948515+0000","last_became_peered":"2024-04-25T10:01:50.948515+0000","last_unstale":"2024-04-25T10:01:57.101702+0000","last_undegraded":"2024-04-25T10:01:57.101702+0000","last_fullsized":"2024-04-25T10:01:57.101702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633739+0000","last_change":"2024-04-25T10:01:56.103664+0000","last_active":"2024-04-25T10:01:57.633739+0000","last_peered":"2024-04-25T10:01:57.633739+0000","last_clean":"2024-04-25T10:01:57.633739+0000","last_became_active":"2024-04-25T10:01:56.103259+0000","last_became_peered":"2024-04-25T10:01:56.103259+0000","last_unstale":"2024-04-25T10:01:57.633739+0000","last_undegraded":"2024-04-25T10:01:57.633739+0000","last_fullsized":"2024-04-25T10:01:57.633739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:19.687438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629659+0000","last_change":"2024-04-25T10:01:51.946200+0000","last_active":"2024-04-25T10:01:57.629659+0000","last_peered":"2024-04-25T10:01:57.629659+0000","last_clean":"2024-04-25T10:01:57.629659+0000","last_became_active":"2024-04-25T10:01:51.945344+0000","last_became_peered":"2024-04-25T10:01:51.945344+0000","last_unstale":"2024-04-25T10:01:57.629659+0000","last_undegraded":"2024-04-25T10:01:57.629659+0000","last_fullsized":"2024-04-25T10:01:57.629659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:13.244438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629997+0000","last_change":"2024-04-25T10:01:50.943130+0000","last_active":"2024-04-25T10:01:57.629997+0000","last_peered":"2024-04-25T10:01:57.629997+0000","last_clean":"2024-04-25T10:01:57.629997+0000","last_became_active":"2024-04-25T10:01:50.942800+0000","last_became_peered":"2024-04-25T10:01:50.942800+0000","last_unstale":"2024-04-25T10:01:57.629997+0000","last_undegraded":"2024-04-25T10:01:57.629997+0000","last_fullsized":"2024-04-25T10:01:57.629997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:58.597257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.102211+0000","last_change":"2024-04-25T10:01:56.103005+0000","last_active":"2024-04-25T10:01:57.102211+0000","last_peered":"2024-04-25T10:01:57.102211+0000","last_clean":"2024-04-25T10:01:57.102211+0000","last_became_active":"2024-04-25T10:01:56.101534+0000","last_became_peered":"2024-04-25T10:01:56.101534+0000","last_unstale":"2024-04-25T10:01:57.102211+0000","last_undegraded":"2024-04-25T10:01:57.102211+0000","last_fullsized":"2024-04-25T10:01:57.102211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:31.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347218+0000","last_change":"2024-04-25T10:01:51.982629+0000","last_active":"2024-04-25T10:01:57.347218+0000","last_peered":"2024-04-25T10:01:57.347218+0000","last_clean":"2024-04-25T10:01:57.347218+0000","last_became_active":"2024-04-25T10:01:51.981703+0000","last_became_peered":"2024-04-25T10:01:51.981703+0000","last_unstale":"2024-04-25T10:01:57.347218+0000","last_undegraded":"2024-04-25T10:01:57.347218+0000","last_fullsized":"2024-04-25T10:01:57.347218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:44.512157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:02.186202+0000","last_change":"2024-04-25T10:01:50.936030+0000","last_active":"2024-04-25T10:02:02.186202+0000","last_peered":"2024-04-25T10:02:02.186202+0000","last_clean":"2024-04-25T10:02:02.186202+0000","last_became_active":"2024-04-25T10:01:50.935276+0000","last_became_peered":"2024-04-25T10:01:50.935276+0000","last_unstale":"2024-04-25T10:02:02.186202+0000","last_undegraded":"2024-04-25T10:02:02.186202+0000","last_fullsized":"2024-04-25T10:02:02.186202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.022550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627939+0000","last_change":"2024-04-25T10:01:56.029517+0000","last_active":"2024-04-25T10:01:57.627939+0000","last_peered":"2024-04-25T10:01:57.627939+0000","last_clean":"2024-04-25T10:01:57.627939+0000","last_became_active":"2024-04-25T10:01:56.018301+0000","last_became_peered":"2024-04-25T10:01:56.018301+0000","last_unstale":"2024-04-25T10:01:57.627939+0000","last_undegraded":"2024-04-25T10:01:57.627939+0000","last_fullsized":"2024-04-25T10:01:57.627939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:56.864362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008458+0000","last_change":"2024-04-25T10:01:51.952954+0000","last_active":"2024-04-25T10:01:57.008458+0000","last_peered":"2024-04-25T10:01:57.008458+0000","last_clean":"2024-04-25T10:01:57.008458+0000","last_became_active":"2024-04-25T10:01:51.952204+0000","last_became_peered":"2024-04-25T10:01:51.952204+0000","last_unstale":"2024-04-25T10:01:57.008458+0000","last_undegraded":"2024-04-25T10:01:57.008458+0000","last_fullsized":"2024-04-25T10:01:57.008458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:39.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629335+0000","last_change":"2024-04-25T10:01:50.950696+0000","last_active":"2024-04-25T10:01:57.629335+0000","last_peered":"2024-04-25T10:01:57.629335+0000","last_clean":"2024-04-25T10:01:57.629335+0000","last_became_active":"2024-04-25T10:01:50.950369+0000","last_became_peered":"2024-04-25T10:01:50.950369+0000","last_unstale":"2024-04-25T10:01:57.629335+0000","last_undegraded":"2024-04-25T10:01:57.629335+0000","last_fullsized":"2024-04-25T10:01:57.629335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:38.540445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.009378+0000","last_change":"2024-04-25T10:01:56.025446+0000","last_active":"2024-04-25T10:01:57.009378+0000","last_peered":"2024-04-25T10:01:57.009378+0000","last_clean":"2024-04-25T10:01:57.009378+0000","last_became_active":"2024-04-25T10:01:56.023974+0000","last_became_peered":"2024-04-25T10:01:56.023974+0000","last_unstale":"2024-04-25T10:01:57.009378+0000","last_undegraded":"2024-04-25T10:01:57.009378+0000","last_fullsized":"2024-04-25T10:01:57.009378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631278+0000","last_change":"2024-04-25T10:01:51.969210+0000","last_active":"2024-04-25T10:01:57.631278+0000","last_peered":"2024-04-25T10:01:57.631278+0000","last_clean":"2024-04-25T10:01:57.631278+0000","last_became_active":"2024-04-25T10:01:51.968830+0000","last_became_peered":"2024-04-25T10:01:51.968830+0000","last_unstale":"2024-04-25T10:01:57.631278+0000","last_undegraded":"2024-04-25T10:01:57.631278+0000","last_fullsized":"2024-04-25T10:01:57.631278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101496+0000","last_change":"2024-04-25T10:01:50.948707+0000","last_active":"2024-04-25T10:01:57.101496+0000","last_peered":"2024-04-25T10:01:57.101496+0000","last_clean":"2024-04-25T10:01:57.101496+0000","last_became_active":"2024-04-25T10:01:50.947988+0000","last_became_peered":"2024-04-25T10:01:50.947988+0000","last_unstale":"2024-04-25T10:01:57.101496+0000","last_undegraded":"2024-04-25T10:01:57.101496+0000","last_fullsized":"2024-04-25T10:01:57.101496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631528+0000","last_change":"2024-04-25T10:01:56.017001+0000","last_active":"2024-04-25T10:01:57.631528+0000","last_peered":"2024-04-25T10:01:57.631528+0000","last_clean":"2024-04-25T10:01:57.631528+0000","last_became_active":"2024-04-25T10:01:56.016220+0000","last_became_peered":"2024-04-25T10:01:56.016220+0000","last_unstale":"2024-04-25T10:01:57.631528+0000","last_undegraded":"2024-04-25T10:01:57.631528+0000","last_fullsized":"2024-04-25T10:01:57.631528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:31.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101724+0000","last_change":"2024-04-25T10:01:51.978944+0000","last_active":"2024-04-25T10:01:57.101724+0000","last_peered":"2024-04-25T10:01:57.101724+0000","last_clean":"2024-04-25T10:01:57.101724+0000","last_became_active":"2024-04-25T10:01:51.978133+0000","last_became_peered":"2024-04-25T10:01:51.978133+0000","last_unstale":"2024-04-25T10:01:57.101724+0000","last_undegraded":"2024-04-25T10:01:57.101724+0000","last_fullsized":"2024-04-25T10:01:57.101724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007998+0000","last_change":"2024-04-25T10:01:50.953423+0000","last_active":"2024-04-25T10:01:57.007998+0000","last_peered":"2024-04-25T10:01:57.007998+0000","last_clean":"2024-04-25T10:01:57.007998+0000","last_became_active":"2024-04-25T10:01:50.952221+0000","last_became_peered":"2024-04-25T10:01:50.952221+0000","last_unstale":"2024-04-25T10:01:57.007998+0000","last_undegraded":"2024-04-25T10:01:57.007998+0000","last_fullsized":"2024-04-25T10:01:57.007998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:48.905633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631879+0000","last_change":"2024-04-25T10:01:56.110756+0000","last_active":"2024-04-25T10:01:57.631879+0000","last_peered":"2024-04-25T10:01:57.631879+0000","last_clean":"2024-04-25T10:01:57.631879+0000","last_became_active":"2024-04-25T10:01:56.109749+0000","last_became_peered":"2024-04-25T10:01:56.109749+0000","last_unstale":"2024-04-25T10:01:57.631879+0000","last_undegraded":"2024-04-25T10:01:57.631879+0000","last_fullsized":"2024-04-25T10:01:57.631879+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:48.763078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628771+0000","last_change":"2024-04-25T10:01:56.106331+0000","last_active":"2024-04-25T10:01:57.628771+0000","last_peered":"2024-04-25T10:01:57.628771+0000","last_clean":"2024-04-25T10:01:57.628771+0000","last_became_active":"2024-04-25T10:01:56.105560+0000","last_became_peered":"2024-04-25T10:01:56.105560+0000","last_unstale":"2024-04-25T10:01:57.628771+0000","last_undegraded":"2024-04-25T10:01:57.628771+0000","last_fullsized":"2024-04-25T10:01:57.628771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.061989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627589+0000","last_change":"2024-04-25T10:01:51.959889+0000","last_active":"2024-04-25T10:01:57.627589+0000","last_peered":"2024-04-25T10:01:57.627589+0000","last_clean":"2024-04-25T10:01:57.627589+0000","last_became_active":"2024-04-25T10:01:51.959609+0000","last_became_peered":"2024-04-25T10:01:51.959609+0000","last_unstale":"2024-04-25T10:01:57.627589+0000","last_undegraded":"2024-04-25T10:01:57.627589+0000","last_fullsized":"2024-04-25T10:01:57.627589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101160+0000","last_change":"2024-04-25T10:01:50.947616+0000","last_active":"2024-04-25T10:01:57.101160+0000","last_peered":"2024-04-25T10:01:57.101160+0000","last_clean":"2024-04-25T10:01:57.101160+0000","last_became_active":"2024-04-25T10:01:50.946930+0000","last_became_peered":"2024-04-25T10:01:50.946930+0000","last_unstale":"2024-04-25T10:01:57.101160+0000","last_undegraded":"2024-04-25T10:01:57.101160+0000","last_fullsized":"2024-04-25T10:01:57.101160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:33.425960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007713+0000","last_change":"2024-04-25T10:01:56.018502+0000","last_active":"2024-04-25T10:01:57.007713+0000","last_peered":"2024-04-25T10:01:57.007713+0000","last_clean":"2024-04-25T10:01:57.007713+0000","last_became_active":"2024-04-25T10:01:56.017379+0000","last_became_peered":"2024-04-25T10:01:56.017379+0000","last_unstale":"2024-04-25T10:01:57.007713+0000","last_undegraded":"2024-04-25T10:01:57.007713+0000","last_fullsized":"2024-04-25T10:01:57.007713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:43.136478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631645+0000","last_change":"2024-04-25T10:01:51.953615+0000","last_active":"2024-04-25T10:01:57.631645+0000","last_peered":"2024-04-25T10:01:57.631645+0000","last_clean":"2024-04-25T10:01:57.631645+0000","last_became_active":"2024-04-25T10:01:51.953362+0000","last_became_peered":"2024-04-25T10:01:51.953362+0000","last_unstale":"2024-04-25T10:01:57.631645+0000","last_undegraded":"2024-04-25T10:01:57.631645+0000","last_fullsized":"2024-04-25T10:01:57.631645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632491+0000","last_change":"2024-04-25T10:01:50.945460+0000","last_active":"2024-04-25T10:01:57.632491+0000","last_peered":"2024-04-25T10:01:57.632491+0000","last_clean":"2024-04-25T10:01:57.632491+0000","last_became_active":"2024-04-25T10:01:50.945086+0000","last_became_peered":"2024-04-25T10:01:50.945086+0000","last_unstale":"2024-04-25T10:01:57.632491+0000","last_undegraded":"2024-04-25T10:01:57.632491+0000","last_fullsized":"2024-04-25T10:01:57.632491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.010630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007381+0000","last_change":"2024-04-25T10:01:50.944204+0000","last_active":"2024-04-25T10:01:57.007381+0000","last_peered":"2024-04-25T10:01:57.007381+0000","last_clean":"2024-04-25T10:01:57.007381+0000","last_became_active":"2024-04-25T10:01:50.943478+0000","last_became_peered":"2024-04-25T10:01:50.943478+0000","last_unstale":"2024-04-25T10:01:57.007381+0000","last_undegraded":"2024-04-25T10:01:57.007381+0000","last_fullsized":"2024-04-25T10:01:57.007381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:41.951630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629902+0000","last_change":"2024-04-25T10:01:51.939383+0000","last_active":"2024-04-25T10:01:57.629902+0000","last_peered":"2024-04-25T10:01:57.629902+0000","last_clean":"2024-04-25T10:01:57.629902+0000","last_became_active":"2024-04-25T10:01:51.938491+0000","last_became_peered":"2024-04-25T10:01:51.938491+0000","last_unstale":"2024-04-25T10:01:57.629902+0000","last_undegraded":"2024-04-25T10:01:57.629902+0000","last_fullsized":"2024-04-25T10:01:57.629902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004475+0000","last_change":"2024-04-25T10:01:56.024292+0000","last_active":"2024-04-25T10:01:57.004475+0000","last_peered":"2024-04-25T10:01:57.004475+0000","last_clean":"2024-04-25T10:01:57.004475+0000","last_became_active":"2024-04-25T10:01:56.023778+0000","last_became_peered":"2024-04-25T10:01:56.023778+0000","last_unstale":"2024-04-25T10:01:57.004475+0000","last_undegraded":"2024-04-25T10:01:57.004475+0000","last_fullsized":"2024-04-25T10:01:57.004475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:30.157749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633372+0000","last_change":"2024-04-25T10:01:50.954588+0000","last_active":"2024-04-25T10:01:57.633372+0000","last_peered":"2024-04-25T10:01:57.633372+0000","last_clean":"2024-04-25T10:01:57.633372+0000","last_became_active":"2024-04-25T10:01:50.954119+0000","last_became_peered":"2024-04-25T10:01:50.954119+0000","last_unstale":"2024-04-25T10:01:57.633372+0000","last_undegraded":"2024-04-25T10:01:57.633372+0000","last_fullsized":"2024-04-25T10:01:57.633372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:49.892857+0000","last_clean_scrub_stamp":"2024-04-25T10:01:49.892857+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:35.306064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007283+0000","last_change":"2024-04-25T10:01:51.941227+0000","last_active":"2024-04-25T10:01:57.007283+0000","last_peered":"2024-04-25T10:01:57.007283+0000","last_clean":"2024-04-25T10:01:57.007283+0000","last_became_active":"2024-04-25T10:01:51.940116+0000","last_became_peered":"2024-04-25T10:01:51.940116+0000","last_unstale":"2024-04-25T10:01:57.007283+0000","last_undegraded":"2024-04-25T10:01:57.007283+0000","last_fullsized":"2024-04-25T10:01:57.007283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:49.915603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633711+0000","last_change":"2024-04-25T10:01:56.024226+0000","last_active":"2024-04-25T10:01:57.633711+0000","last_peered":"2024-04-25T10:01:57.633711+0000","last_clean":"2024-04-25T10:01:57.633711+0000","last_became_active":"2024-04-25T10:01:56.023568+0000","last_became_peered":"2024-04-25T10:01:56.023568+0000","last_unstale":"2024-04-25T10:01:57.633711+0000","last_undegraded":"2024-04-25T10:01:57.633711+0000","last_fullsized":"2024-04-25T10:01:57.633711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:54.965262+0000","last_clean_scrub_stamp":"2024-04-25T10:01:54.965262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.434943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.005947+0000","last_change":"2024-04-25T10:01:51.948178+0000","last_active":"2024-04-25T10:01:57.005947+0000","last_peered":"2024-04-25T10:01:57.005947+0000","last_clean":"2024-04-25T10:01:57.005947+0000","last_became_active":"2024-04-25T10:01:51.943159+0000","last_became_peered":"2024-04-25T10:01:51.943159+0000","last_unstale":"2024-04-25T10:01:57.005947+0000","last_undegraded":"2024-04-25T10:01:57.005947+0000","last_fullsized":"2024-04-25T10:01:57.005947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:13.244438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004086+0000","last_change":"2024-04-25T10:01:51.977923+0000","last_active":"2024-04-25T10:01:57.004086+0000","last_peered":"2024-04-25T10:01:57.004086+0000","last_clean":"2024-04-25T10:01:57.004086+0000","last_became_active":"2024-04-25T10:01:51.977601+0000","last_became_peered":"2024-04-25T10:01:51.977601+0000","last_unstale":"2024-04-25T10:01:57.004086+0000","last_undegraded":"2024-04-25T10:01:57.004086+0000","last_fullsized":"2024-04-25T10:01:57.004086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:39.550415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101440+0000","last_change":"2024-04-25T10:01:51.977323+0000","last_active":"2024-04-25T10:01:57.101440+0000","last_peered":"2024-04-25T10:01:57.101440+0000","last_clean":"2024-04-25T10:01:57.101440+0000","last_became_active":"2024-04-25T10:01:51.976662+0000","last_became_peered":"2024-04-25T10:01:51.976662+0000","last_unstale":"2024-04-25T10:01:57.101440+0000","last_undegraded":"2024-04-25T10:01:57.101440+0000","last_fullsized":"2024-04-25T10:01:57.101440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:52.801927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629808+0000","last_change":"2024-04-25T10:01:51.962727+0000","last_active":"2024-04-25T10:01:57.629808+0000","last_peered":"2024-04-25T10:01:57.629808+0000","last_clean":"2024-04-25T10:01:57.629808+0000","last_became_active":"2024-04-25T10:01:51.962048+0000","last_became_peered":"2024-04-25T10:01:51.962048+0000","last_unstale":"2024-04-25T10:01:57.629808+0000","last_undegraded":"2024-04-25T10:01:57.629808+0000","last_fullsized":"2024-04-25T10:01:57.629808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633852+0000","last_change":"2024-04-25T10:01:51.956859+0000","last_active":"2024-04-25T10:01:57.633852+0000","last_peered":"2024-04-25T10:01:57.633852+0000","last_clean":"2024-04-25T10:01:57.633852+0000","last_became_active":"2024-04-25T10:01:51.956531+0000","last_became_peered":"2024-04-25T10:01:51.956531+0000","last_unstale":"2024-04-25T10:01:57.633852+0000","last_undegraded":"2024-04-25T10:01:57.633852+0000","last_fullsized":"2024-04-25T10:01:57.633852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.628823+0000","last_change":"2024-04-25T10:01:51.960245+0000","last_active":"2024-04-25T10:01:57.628823+0000","last_peered":"2024-04-25T10:01:57.628823+0000","last_clean":"2024-04-25T10:01:57.628823+0000","last_became_active":"2024-04-25T10:01:51.959954+0000","last_became_peered":"2024-04-25T10:01:51.959954+0000","last_unstale":"2024-04-25T10:01:57.628823+0000","last_undegraded":"2024-04-25T10:01:57.628823+0000","last_fullsized":"2024-04-25T10:01:57.628823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008265+0000","last_change":"2024-04-25T10:01:51.951215+0000","last_active":"2024-04-25T10:01:57.008265+0000","last_peered":"2024-04-25T10:01:57.008265+0000","last_clean":"2024-04-25T10:01:57.008265+0000","last_became_active":"2024-04-25T10:01:51.950168+0000","last_became_peered":"2024-04-25T10:01:51.950168+0000","last_unstale":"2024-04-25T10:01:57.008265+0000","last_undegraded":"2024-04-25T10:01:57.008265+0000","last_fullsized":"2024-04-25T10:01:57.008265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.634588+0000","last_change":"2024-04-25T10:01:51.959477+0000","last_active":"2024-04-25T10:01:57.634588+0000","last_peered":"2024-04-25T10:01:57.634588+0000","last_clean":"2024-04-25T10:01:57.634588+0000","last_became_active":"2024-04-25T10:01:51.958740+0000","last_became_peered":"2024-04-25T10:01:51.958740+0000","last_unstale":"2024-04-25T10:01:57.634588+0000","last_undegraded":"2024-04-25T10:01:57.634588+0000","last_fullsized":"2024-04-25T10:01:57.634588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100898+0000","last_change":"2024-04-25T10:01:51.987810+0000","last_active":"2024-04-25T10:01:57.100898+0000","last_peered":"2024-04-25T10:01:57.100898+0000","last_clean":"2024-04-25T10:01:57.100898+0000","last_became_active":"2024-04-25T10:01:51.978504+0000","last_became_peered":"2024-04-25T10:01:51.978504+0000","last_unstale":"2024-04-25T10:01:57.100898+0000","last_undegraded":"2024-04-25T10:01:57.100898+0000","last_fullsized":"2024-04-25T10:01:57.100898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:44.512157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.632135+0000","last_change":"2024-04-25T10:01:51.949190+0000","last_active":"2024-04-25T10:01:57.632135+0000","last_peered":"2024-04-25T10:01:57.632135+0000","last_clean":"2024-04-25T10:01:57.632135+0000","last_became_active":"2024-04-25T10:01:51.948886+0000","last_became_peered":"2024-04-25T10:01:51.948886+0000","last_unstale":"2024-04-25T10:01:57.632135+0000","last_undegraded":"2024-04-25T10:01:57.632135+0000","last_fullsized":"2024-04-25T10:01:57.632135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101158+0000","last_change":"2024-04-25T10:01:51.987833+0000","last_active":"2024-04-25T10:01:57.101158+0000","last_peered":"2024-04-25T10:01:57.101158+0000","last_clean":"2024-04-25T10:01:57.101158+0000","last_became_active":"2024-04-25T10:01:51.978154+0000","last_became_peered":"2024-04-25T10:01:51.978154+0000","last_unstale":"2024-04-25T10:01:57.101158+0000","last_undegraded":"2024-04-25T10:01:57.101158+0000","last_fullsized":"2024-04-25T10:01:57.101158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:22.999554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629659+0000","last_change":"2024-04-25T10:01:51.947940+0000","last_active":"2024-04-25T10:01:57.629659+0000","last_peered":"2024-04-25T10:01:57.629659+0000","last_clean":"2024-04-25T10:01:57.629659+0000","last_became_active":"2024-04-25T10:01:51.947672+0000","last_became_peered":"2024-04-25T10:01:51.947672+0000","last_unstale":"2024-04-25T10:01:57.629659+0000","last_undegraded":"2024-04-25T10:01:57.629659+0000","last_fullsized":"2024-04-25T10:01:57.629659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:39.087518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007150+0000","last_change":"2024-04-25T10:01:51.936013+0000","last_active":"2024-04-25T10:01:57.007150+0000","last_peered":"2024-04-25T10:01:57.007150+0000","last_clean":"2024-04-25T10:01:57.007150+0000","last_became_active":"2024-04-25T10:01:51.933077+0000","last_became_peered":"2024-04-25T10:01:51.933077+0000","last_unstale":"2024-04-25T10:01:57.007150+0000","last_undegraded":"2024-04-25T10:01:57.007150+0000","last_fullsized":"2024-04-25T10:01:57.007150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:46.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.348064+0000","last_change":"2024-04-25T10:01:51.987425+0000","last_active":"2024-04-25T10:01:57.348064+0000","last_peered":"2024-04-25T10:01:57.348064+0000","last_clean":"2024-04-25T10:01:57.348064+0000","last_became_active":"2024-04-25T10:01:51.986698+0000","last_became_peered":"2024-04-25T10:01:51.986698+0000","last_unstale":"2024-04-25T10:01:57.348064+0000","last_undegraded":"2024-04-25T10:01:57.348064+0000","last_fullsized":"2024-04-25T10:01:57.348064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:28.226905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.004602+0000","last_change":"2024-04-25T10:01:51.962633+0000","last_active":"2024-04-25T10:01:57.004602+0000","last_peered":"2024-04-25T10:01:57.004602+0000","last_clean":"2024-04-25T10:01:57.004602+0000","last_became_active":"2024-04-25T10:01:51.962235+0000","last_became_peered":"2024-04-25T10:01:51.962235+0000","last_unstale":"2024-04-25T10:01:57.004602+0000","last_undegraded":"2024-04-25T10:01:57.004602+0000","last_fullsized":"2024-04-25T10:01:57.004602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:39.288148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101861+0000","last_change":"2024-04-25T10:01:51.992557+0000","last_active":"2024-04-25T10:01:57.101861+0000","last_peered":"2024-04-25T10:01:57.101861+0000","last_clean":"2024-04-25T10:01:57.101861+0000","last_became_active":"2024-04-25T10:01:51.991982+0000","last_became_peered":"2024-04-25T10:01:51.991982+0000","last_unstale":"2024-04-25T10:01:57.101861+0000","last_undegraded":"2024-04-25T10:01:57.101861+0000","last_fullsized":"2024-04-25T10:01:57.101861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100581+0000","last_change":"2024-04-25T10:01:51.988776+0000","last_active":"2024-04-25T10:01:57.100581+0000","last_peered":"2024-04-25T10:01:57.100581+0000","last_clean":"2024-04-25T10:01:57.100581+0000","last_became_active":"2024-04-25T10:01:51.987983+0000","last_became_peered":"2024-04-25T10:01:51.987983+0000","last_unstale":"2024-04-25T10:01:57.100581+0000","last_undegraded":"2024-04-25T10:01:57.100581+0000","last_fullsized":"2024-04-25T10:01:57.100581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:39.074042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.627816+0000","last_change":"2024-04-25T10:01:51.946481+0000","last_active":"2024-04-25T10:01:57.627816+0000","last_peered":"2024-04-25T10:01:57.627816+0000","last_clean":"2024-04-25T10:01:57.627816+0000","last_became_active":"2024-04-25T10:01:51.946161+0000","last_became_peered":"2024-04-25T10:01:51.946161+0000","last_unstale":"2024-04-25T10:01:57.627816+0000","last_undegraded":"2024-04-25T10:01:57.627816+0000","last_fullsized":"2024-04-25T10:01:57.627816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:09.432872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629440+0000","last_change":"2024-04-25T10:01:51.946480+0000","last_active":"2024-04-25T10:01:57.629440+0000","last_peered":"2024-04-25T10:01:57.629440+0000","last_clean":"2024-04-25T10:01:57.629440+0000","last_became_active":"2024-04-25T10:01:51.945872+0000","last_became_peered":"2024-04-25T10:01:51.945872+0000","last_unstale":"2024-04-25T10:01:57.629440+0000","last_undegraded":"2024-04-25T10:01:57.629440+0000","last_fullsized":"2024-04-25T10:01:57.629440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:44.793972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.633094+0000","last_change":"2024-04-25T10:01:51.958336+0000","last_active":"2024-04-25T10:01:57.633094+0000","last_peered":"2024-04-25T10:01:57.633094+0000","last_clean":"2024-04-25T10:01:57.633094+0000","last_became_active":"2024-04-25T10:01:51.957993+0000","last_became_peered":"2024-04-25T10:01:51.957993+0000","last_unstale":"2024-04-25T10:01:57.633094+0000","last_undegraded":"2024-04-25T10:01:57.633094+0000","last_fullsized":"2024-04-25T10:01:57.633094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.854987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008236+0000","last_change":"2024-04-25T10:01:51.943174+0000","last_active":"2024-04-25T10:01:57.008236+0000","last_peered":"2024-04-25T10:01:57.008236+0000","last_clean":"2024-04-25T10:01:57.008236+0000","last_became_active":"2024-04-25T10:01:51.942327+0000","last_became_peered":"2024-04-25T10:01:51.942327+0000","last_unstale":"2024-04-25T10:01:57.008236+0000","last_undegraded":"2024-04-25T10:01:57.008236+0000","last_fullsized":"2024-04-25T10:01:57.008236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:22.581767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.007882+0000","last_change":"2024-04-25T10:01:51.963532+0000","last_active":"2024-04-25T10:01:57.007882+0000","last_peered":"2024-04-25T10:01:57.007882+0000","last_clean":"2024-04-25T10:01:57.007882+0000","last_became_active":"2024-04-25T10:01:51.962793+0000","last_became_peered":"2024-04-25T10:01:51.962793+0000","last_unstale":"2024-04-25T10:01:57.007882+0000","last_undegraded":"2024-04-25T10:01:57.007882+0000","last_fullsized":"2024-04-25T10:01:57.007882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:21.610429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.008103+0000","last_change":"2024-04-25T10:01:51.970144+0000","last_active":"2024-04-25T10:01:57.008103+0000","last_peered":"2024-04-25T10:01:57.008103+0000","last_clean":"2024-04-25T10:01:57.008103+0000","last_became_active":"2024-04-25T10:01:51.967189+0000","last_became_peered":"2024-04-25T10:01:51.967189+0000","last_unstale":"2024-04-25T10:01:57.008103+0000","last_undegraded":"2024-04-25T10:01:57.008103+0000","last_fullsized":"2024-04-25T10:01:57.008103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:45.956252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347824+0000","last_change":"2024-04-25T10:01:51.995155+0000","last_active":"2024-04-25T10:01:57.347824+0000","last_peered":"2024-04-25T10:01:57.347824+0000","last_clean":"2024-04-25T10:01:57.347824+0000","last_became_active":"2024-04-25T10:01:51.994514+0000","last_became_peered":"2024-04-25T10:01:51.994514+0000","last_unstale":"2024-04-25T10:01:57.347824+0000","last_undegraded":"2024-04-25T10:01:57.347824+0000","last_fullsized":"2024-04-25T10:01:57.347824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:36.570949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.631050+0000","last_change":"2024-04-25T10:01:51.977705+0000","last_active":"2024-04-25T10:01:57.631050+0000","last_peered":"2024-04-25T10:01:57.631050+0000","last_clean":"2024-04-25T10:01:57.631050+0000","last_became_active":"2024-04-25T10:01:51.977373+0000","last_became_peered":"2024-04-25T10:01:51.977373+0000","last_unstale":"2024-04-25T10:01:57.631050+0000","last_undegraded":"2024-04-25T10:01:57.631050+0000","last_fullsized":"2024-04-25T10:01:57.631050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:02.911542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347155+0000","last_change":"2024-04-25T10:01:51.958773+0000","last_active":"2024-04-25T10:01:57.347155+0000","last_peered":"2024-04-25T10:01:57.347155+0000","last_clean":"2024-04-25T10:01:57.347155+0000","last_became_active":"2024-04-25T10:01:51.958015+0000","last_became_peered":"2024-04-25T10:01:51.958015+0000","last_unstale":"2024-04-25T10:01:57.347155+0000","last_undegraded":"2024-04-25T10:01:57.347155+0000","last_fullsized":"2024-04-25T10:01:57.347155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.369662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.100869+0000","last_change":"2024-04-25T10:01:51.994172+0000","last_active":"2024-04-25T10:01:57.100869+0000","last_peered":"2024-04-25T10:01:57.100869+0000","last_clean":"2024-04-25T10:01:57.100869+0000","last_became_active":"2024-04-25T10:01:51.993498+0000","last_became_peered":"2024-04-25T10:01:51.993498+0000","last_unstale":"2024-04-25T10:01:57.100869+0000","last_undegraded":"2024-04-25T10:01:57.100869+0000","last_fullsized":"2024-04-25T10:01:57.100869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:59.098969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.346909+0000","last_change":"2024-04-25T10:01:51.958621+0000","last_active":"2024-04-25T10:01:57.346909+0000","last_peered":"2024-04-25T10:01:57.346909+0000","last_clean":"2024-04-25T10:01:57.346909+0000","last_became_active":"2024-04-25T10:01:51.957937+0000","last_became_peered":"2024-04-25T10:01:51.957937+0000","last_unstale":"2024-04-25T10:01:57.346909+0000","last_undegraded":"2024-04-25T10:01:57.346909+0000","last_fullsized":"2024-04-25T10:01:57.346909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:59.098969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.101663+0000","last_change":"2024-04-25T10:01:51.967673+0000","last_active":"2024-04-25T10:01:57.101663+0000","last_peered":"2024-04-25T10:01:57.101663+0000","last_clean":"2024-04-25T10:01:57.101663+0000","last_became_active":"2024-04-25T10:01:51.966329+0000","last_became_peered":"2024-04-25T10:01:51.966329+0000","last_unstale":"2024-04-25T10:01:57.101663+0000","last_undegraded":"2024-04-25T10:01:57.101663+0000","last_fullsized":"2024-04-25T10:01:57.101663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.630321+0000","last_change":"2024-04-25T10:01:51.950122+0000","last_active":"2024-04-25T10:01:57.630321+0000","last_peered":"2024-04-25T10:01:57.630321+0000","last_clean":"2024-04-25T10:01:57.630321+0000","last_became_active":"2024-04-25T10:01:51.949559+0000","last_became_peered":"2024-04-25T10:01:51.949559+0000","last_unstale":"2024-04-25T10:01:57.630321+0000","last_undegraded":"2024-04-25T10:01:57.630321+0000","last_fullsized":"2024-04-25T10:01:57.630321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:52.021045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.629583+0000","last_change":"2024-04-25T10:01:51.943127+0000","last_active":"2024-04-25T10:01:57.629583+0000","last_peered":"2024-04-25T10:01:57.629583+0000","last_clean":"2024-04-25T10:01:57.629583+0000","last_became_active":"2024-04-25T10:01:51.942304+0000","last_became_peered":"2024-04-25T10:01:51.942304+0000","last_unstale":"2024-04-25T10:01:57.629583+0000","last_undegraded":"2024-04-25T10:01:57.629583+0000","last_fullsized":"2024-04-25T10:01:57.629583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:22.751707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:01:57.347294+0000","last_change":"2024-04-25T10:01:51.987276+0000","last_active":"2024-04-25T10:01:57.347294+0000","last_peered":"2024-04-25T10:01:57.347294+0000","last_clean":"2024-04-25T10:01:57.347294+0000","last_became_active":"2024-04-25T10:01:51.986766+0000","last_became_peered":"2024-04-25T10:01:51.986766+0000","last_unstale":"2024-04-25T10:01:57.347294+0000","last_undegraded":"2024-04-25T10:01:57.347294+0000","last_fullsized":"2024-04-25T10:01:57.347294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:50.902827+0000","last_clean_scrub_stamp":"2024-04-25T10:01:50.902827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:57.724919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:22.166 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:02:22.166 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:02:22.166 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:02:22.166 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:02:22.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.304+0000 7f5dd372e640 1 Processor -- start 2024-04-25T10:02:22.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.304+0000 7f5dd372e640 1 -- start start 2024-04-25T10:02:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.304+0000 7f5dd372e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5dcc158110 0x7f5dcc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.304+0000 7f5dd372e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dcc1584f0 0x7f5dcc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd372e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5dcc06e460 0x7f5dcc06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd372e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc06edf0 con 0x7f5dcc06e460 2024-04-25T10:02:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd372e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc0cbe60 con 0x7f5dcc158110 2024-04-25T10:02:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd372e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0b19b0 con 0x7f5dcc1584f0 2024-04-25T10:02:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd1ccb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5dcc06e460 0x7f5dcc06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd1ccb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5dcc06e460 0x7f5dcc06e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48744/0 (socket says 172.21.15.133:48744) 2024-04-25T10:02:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd1ccb640 1 -- 172.21.15.133:0/2920576969 learned_addr learned my addr 172.21.15.133:0/2920576969 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd14ca640 1 --2- 172.21.15.133:0/2920576969 >> v2:172.21.15.162:3300/0 conn(0x7f5dcc158110 0x7f5dcc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd0cc9640 1 --2- 172.21.15.133:0/2920576969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dcc1584f0 0x7f5dcc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd1ccb640 1 -- 172.21.15.133:0/2920576969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dcc1584f0 msgr2=0x7f5dcc15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd1ccb640 1 --2- 172.21.15.133:0/2920576969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dcc1584f0 0x7f5dcc15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd1ccb640 1 -- 172.21.15.133:0/2920576969 >> v2:172.21.15.162:3300/0 conn(0x7f5dcc158110 msgr2=0x7f5dcc0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd1ccb640 1 --2- 172.21.15.133:0/2920576969 >> v2:172.21.15.162:3300/0 conn(0x7f5dcc158110 0x7f5dcc0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd1ccb640 1 -- 172.21.15.133:0/2920576969 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc15b360 con 0x7f5dcc06e460 2024-04-25T10:02:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd14ca640 1 --2- 172.21.15.133:0/2920576969 >> v2:172.21.15.162:3300/0 conn(0x7f5dcc158110 0x7f5dcc0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:22.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dd1ccb640 1 --2- 172.21.15.133:0/2920576969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dcc06e460 0x7f5dcc06e8b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0b4d40 tx=0x7f5dc401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833b420441eb734f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dc3fff640 1 -- 172.21.15.133:0/2920576969 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40a5840 con 0x7f5dcc06e460 2024-04-25T10:02:22.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dc3fff640 1 -- 172.21.15.133:0/2920576969 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc40a59e0 con 0x7f5dcc06e460 2024-04-25T10:02:22.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.308+0000 7f5dc3fff640 1 -- 172.21.15.133:0/2920576969 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40a5db0 con 0x7f5dcc06e460 2024-04-25T10:02:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- 172.21.15.133:0/2920576969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dcc06e460 msgr2=0x7f5dcc06e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 --2- 172.21.15.133:0/2920576969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dcc06e460 0x7f5dcc06e8b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0b4d40 tx=0x7f5dc401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- 172.21.15.133:0/2920576969 shutdown_connections 2024-04-25T10:02:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 --2- 172.21.15.133:0/2920576969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dcc06e460 0x7f5dcc06e8b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 --2- 172.21.15.133:0/2920576969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dcc1584f0 0x7f5dcc15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 --2- 172.21.15.133:0/2920576969 >> v2:172.21.15.162:3300/0 conn(0x7f5dcc158110 0x7f5dcc0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- 172.21.15.133:0/2920576969 >> 172.21.15.133:0/2920576969 conn(0x7f5dcc067000 msgr2=0x7f5dcc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- 172.21.15.133:0/2920576969 shutdown_connections 2024-04-25T10:02:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- 172.21.15.133:0/2920576969 wait complete. 2024-04-25T10:02:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 Processor -- start 2024-04-25T10:02:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- start start 2024-04-25T10:02:22.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dcc06e460 0x7f5dcc0b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dcc158110 0x7f5dcc150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dcc1584f0 0x7f5dcc15c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0cd7f0 con 0x7f5dcc06e460 2024-04-25T10:02:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0cd670 con 0x7f5dcc158110 2024-04-25T10:02:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc15eb10 con 0x7f5dcc1584f0 2024-04-25T10:02:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd14ca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dcc06e460 0x7f5dcc0b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd14ca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dcc06e460 0x7f5dcc0b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48750/0 (socket says 172.21.15.133:48750) 2024-04-25T10:02:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd14ca640 1 -- 172.21.15.133:0/1491139481 learned_addr learned my addr 172.21.15.133:0/1491139481 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd0cc9640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dcc158110 0x7f5dcc150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd1ccb640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dcc1584f0 0x7f5dcc15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd14ca640 1 -- 172.21.15.133:0/1491139481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dcc1584f0 msgr2=0x7f5dcc15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd14ca640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dcc1584f0 0x7f5dcc15c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd14ca640 1 -- 172.21.15.133:0/1491139481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dcc158110 msgr2=0x7f5dcc150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd14ca640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dcc158110 0x7f5dcc150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd14ca640 1 -- 172.21.15.133:0/1491139481 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc15c8b0 con 0x7f5dcc06e460 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd1ccb640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dcc1584f0 0x7f5dcc15c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd14ca640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dcc06e460 0x7f5dcc0b3fa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c7f0 tx=0x7f5dbc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dc1ffb640 1 -- 172.21.15.133:0/1491139481 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc015070 con 0x7f5dcc06e460 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dc1ffb640 1 -- 172.21.15.133:0/1491139481 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc007a30 con 0x7f5dcc06e460 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- 172.21.15.133:0/1491139481 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc15cb50 con 0x7f5dcc06e460 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dc1ffb640 1 -- 172.21.15.133:0/1491139481 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc007e00 con 0x7f5dcc06e460 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- 172.21.15.133:0/1491139481 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc15d110 con 0x7f5dcc06e460 2024-04-25T10:02:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.312+0000 7f5dd372e640 1 -- 172.21.15.133:0/1491139481 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc06bc20 con 0x7f5dcc06e460 2024-04-25T10:02:22.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.316+0000 7f5dc1ffb640 1 -- 172.21.15.133:0/1491139481 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5dbc018070 con 0x7f5dcc06e460 2024-04-25T10:02:22.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.316+0000 7f5dc1ffb640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5db0066f60 0x7f5db0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.316+0000 7f5dc1ffb640 1 -- 172.21.15.133:0/1491139481 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5dbc085250 con 0x7f5dcc06e460 2024-04-25T10:02:22.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.316+0000 7f5dd0cc9640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5db0066f60 0x7f5db0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.316+0000 7f5dd0cc9640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5db0066f60 0x7f5db0069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0ca880 tx=0x7f5dc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.320+0000 7f5dc1ffb640 1 -- 172.21.15.133:0/1491139481 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5dbc055f20 con 0x7f5dcc06e460 2024-04-25T10:02:22.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.496+0000 7f5dd372e640 1 -- 172.21.15.133:0/1491139481 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5dcc068ef0 con 0x7f5dcc06e460 2024-04-25T10:02:22.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.496+0000 7f5dc1ffb640 1 -- 172.21.15.133:0/1491139481 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5dbc055820 con 0x7f5dcc06e460 2024-04-25T10:02:22.505 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:22.505 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:02:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 -- 172.21.15.133:0/1491139481 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5db0066f60 msgr2=0x7f5db0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5db0066f60 0x7f5db0069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0ca880 tx=0x7f5dc8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 -- 172.21.15.133:0/1491139481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dcc06e460 msgr2=0x7f5dcc0b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dcc06e460 0x7f5dcc0b3fa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c7f0 tx=0x7f5dbc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 -- 172.21.15.133:0/1491139481 shutdown_connections 2024-04-25T10:02:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5dcc1584f0 0x7f5dcc15c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5dcc158110 0x7f5dcc150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5db0066f60 0x7f5db0069420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 --2- 172.21.15.133:0/1491139481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dcc06e460 0x7f5dcc0b3fa0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 -- 172.21.15.133:0/1491139481 >> 172.21.15.133:0/1491139481 conn(0x7f5dcc067000 msgr2=0x7f5dcc150830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 -- 172.21.15.133:0/1491139481 shutdown_connections 2024-04-25T10:02:22.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.500+0000 7f5dd372e640 1 -- 172.21.15.133:0/1491139481 wait complete. 2024-04-25T10:02:22.522 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:02:22.523 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:22.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa843162640 1 Processor -- start 2024-04-25T10:02:22.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa843162640 1 -- start start 2024-04-25T10:02:22.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa843162640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa83c160b90 0x7fa83c0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa843162640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa83c0b31c0 0x7fa83c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa843162640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83c0cba70 con 0x7fa83c0b3b50 2024-04-25T10:02:22.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa843162640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83c0cae10 con 0x7fa83c0b31c0 2024-04-25T10:02:22.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa843162640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c0cb2f0 con 0x7fa83c160b90 2024-04-25T10:02:22.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa84195f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa83c0b31c0 0x7fa83c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa84195f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa83c0b31c0 0x7fa83c0b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48756/0 (socket says 172.21.15.133:48756) 2024-04-25T10:02:22.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa84195f640 1 -- 172.21.15.133:0/3036544329 learned_addr learned my addr 172.21.15.133:0/3036544329 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:22.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa842160640 1 --2- 172.21.15.133:0/3036544329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa83c160b90 0x7fa83c0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa84195f640 1 -- 172.21.15.133:0/3036544329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa83c160b90 msgr2=0x7fa83c0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa84195f640 1 --2- 172.21.15.133:0/3036544329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa83c160b90 0x7fa83c0b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa84195f640 1 -- 172.21.15.133:0/3036544329 >> v1:172.21.15.133:6789/0 conn(0x7fa83c0b3b50 legacy=0x7fa83c161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa84195f640 1 -- 172.21.15.133:0/3036544329 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c0c9920 con 0x7fa83c0b31c0 2024-04-25T10:02:22.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa842160640 1 --2- 172.21.15.133:0/3036544329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa83c160b90 0x7fa83c0b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:22.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa84195f640 1 --2- 172.21.15.133:0/3036544329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c0b31c0 0x7fa83c0b3610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0098a0 tx=0x7fa82c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4da8ff83edce019 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.632+0000 7fa84115e640 1 -- 172.21.15.133:0/3036544329 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0b0070 con 0x7fa83c0b31c0 2024-04-25T10:02:22.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa84115e640 1 -- 172.21.15.133:0/3036544329 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c004230 con 0x7fa83c0b31c0 2024-04-25T10:02:22.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa84115e640 1 -- 172.21.15.133:0/3036544329 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c003620 con 0x7fa83c0b31c0 2024-04-25T10:02:22.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 -- 172.21.15.133:0/3036544329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c0b31c0 msgr2=0x7fa83c0b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 --2- 172.21.15.133:0/3036544329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c0b31c0 0x7fa83c0b3610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0098a0 tx=0x7fa82c01c540 comp rx=0 tx=0).stop 2024-04-25T10:02:22.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 -- 172.21.15.133:0/3036544329 shutdown_connections 2024-04-25T10:02:22.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 --2- 172.21.15.133:0/3036544329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c0b31c0 0x7fa83c0b3610 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 --2- 172.21.15.133:0/3036544329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa83c160b90 0x7fa83c0b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 -- 172.21.15.133:0/3036544329 >> 172.21.15.133:0/3036544329 conn(0x7fa83c0b1030 msgr2=0x7fa83c0b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 -- 172.21.15.133:0/3036544329 shutdown_connections 2024-04-25T10:02:22.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 -- 172.21.15.133:0/3036544329 wait complete. 2024-04-25T10:02:22.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 Processor -- start 2024-04-25T10:02:22.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 -- start start 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa83c0b31c0 0x7fa83c0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c0b3b50 0x7fa83c0c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa83c160b90 0x7fa83c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c160f70 con 0x7fa83c0b3b50 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c0cc580 con 0x7fa83c160b90 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c1610f0 con 0x7fa83c0b31c0 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa84195f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c0b3b50 0x7fa83c0c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa84195f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c0b3b50 0x7fa83c0c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48764/0 (socket says 172.21.15.133:48764) 2024-04-25T10:02:22.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa84195f640 1 -- 172.21.15.133:0/3182549428 learned_addr learned my addr 172.21.15.133:0/3182549428 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:22.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa842160640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa83c0b31c0 0x7fa83c0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa84195f640 1 -- 172.21.15.133:0/3182549428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa83c0b31c0 msgr2=0x7fa83c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa84195f640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa83c0b31c0 0x7fa83c0c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa84195f640 1 -- 172.21.15.133:0/3182549428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa83c160b90 msgr2=0x7fa83c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa84195f640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa83c160b90 0x7fa83c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa84195f640 1 -- 172.21.15.133:0/3182549428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c0c6e30 con 0x7fa83c0b3b50 2024-04-25T10:02:22.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa84195f640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c0b3b50 0x7fa83c0c6300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa82c01a170 tx=0x7fa82c0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa832ffd640 1 -- 172.21.15.133:0/3182549428 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0b0040 con 0x7fa83c0b3b50 2024-04-25T10:02:22.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa832ffd640 1 -- 172.21.15.133:0/3182549428 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c0bb070 con 0x7fa83c0b3b50 2024-04-25T10:02:22.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 -- 172.21.15.133:0/3182549428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c0c6b50 con 0x7fa83c0b3b50 2024-04-25T10:02:22.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa832ffd640 1 -- 172.21.15.133:0/3182549428 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0b6440 con 0x7fa83c0b3b50 2024-04-25T10:02:22.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.636+0000 7fa843162640 1 -- 172.21.15.133:0/3182549428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c06d390 con 0x7fa83c0b3b50 2024-04-25T10:02:22.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.640+0000 7fa843162640 1 -- 172.21.15.133:0/3182549428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83c068e80 con 0x7fa83c0b3b50 2024-04-25T10:02:22.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.640+0000 7fa832ffd640 1 -- 172.21.15.133:0/3182549428 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa82c0b65e0 con 0x7fa83c0b3b50 2024-04-25T10:02:22.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.640+0000 7fa832ffd640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa820066f60 0x7fa820069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.640+0000 7fa842160640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa820066f60 0x7fa820069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.640+0000 7fa832ffd640 1 -- 172.21.15.133:0/3182549428 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa82c11f9b0 con 0x7fa83c0b3b50 2024-04-25T10:02:22.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.640+0000 7fa842160640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa820066f60 0x7fa820069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa838007510 tx=0x7fa8380074a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.644+0000 7fa832ffd640 1 -- 172.21.15.133:0/3182549428 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa82c016130 con 0x7fa83c0b3b50 2024-04-25T10:02:22.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.788+0000 7fa843162640 1 -- 172.21.15.133:0/3182549428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa83c0b15b0 con 0x7fa83c0b3b50 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.792+0000 7fa832ffd640 1 -- 172.21.15.133:0/3182549428 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7fa82c0f0830 con 0x7fa83c0b3b50 2024-04-25T10:02:22.800 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:22.800 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 -- 172.21.15.133:0/3182549428 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa820066f60 msgr2=0x7fa820069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa820066f60 0x7fa820069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa838007510 tx=0x7fa8380074a0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 -- 172.21.15.133:0/3182549428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c0b3b50 msgr2=0x7fa83c0c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c0b3b50 0x7fa83c0c6300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa82c01a170 tx=0x7fa82c0192e0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 -- 172.21.15.133:0/3182549428 shutdown_connections 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa83c160b90 0x7fa83c0cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa820066f60 0x7fa820069420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c0b3b50 0x7fa83c0c6300 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 --2- 172.21.15.133:0/3182549428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa83c0b31c0 0x7fa83c0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 -- 172.21.15.133:0/3182549428 >> 172.21.15.133:0/3182549428 conn(0x7fa83c0b1030 msgr2=0x7fa83c0b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 -- 172.21.15.133:0/3182549428 shutdown_connections 2024-04-25T10:02:22.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.796+0000 7fa843162640 1 -- 172.21.15.133:0/3182549428 wait complete. 2024-04-25T10:02:22.817 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:22.818 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:22.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.928+0000 7feb8e89d640 1 Processor -- start 2024-04-25T10:02:22.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.928+0000 7feb8e89d640 1 -- start start 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feb880cd820 0x7feb880cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feb8815ced0 0x7feb8815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb880ce2e0 con 0x7feb88145170 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8806db00 con 0x7feb880cd820 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb88145550 con 0x7feb8815ced0 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb7ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feb880cd820 0x7feb880cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb7ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feb880cd820 0x7feb880cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48766/0 (socket says 172.21.15.133:48766) 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb7ffff640 1 -- 172.21.15.133:0/2097657316 learned_addr learned my addr 172.21.15.133:0/2097657316 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8ce3a640 1 --2- 172.21.15.133:0/2097657316 >> v2:172.21.15.162:3300/0 conn(0x7feb8815ced0 0x7feb8815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb7ffff640 1 -- 172.21.15.133:0/2097657316 >> v2:172.21.15.162:3300/0 conn(0x7feb8815ced0 msgr2=0x7feb8815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb7ffff640 1 --2- 172.21.15.133:0/2097657316 >> v2:172.21.15.162:3300/0 conn(0x7feb8815ced0 0x7feb8815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb7ffff640 1 -- 172.21.15.133:0/2097657316 >> v1:172.21.15.162:6789/0 conn(0x7feb88145170 legacy=0x7feb8806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb7ffff640 1 -- 172.21.15.133:0/2097657316 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8806d4b0 con 0x7feb880cd820 2024-04-25T10:02:22.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb7ffff640 1 --2- 172.21.15.133:0/2097657316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb880cd820 0x7feb880cdca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feb880b27f0 tx=0x7feb7001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824c35d72996e658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb877fe640 1 -- 172.21.15.133:0/2097657316 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb700b0070 con 0x7feb880cd820 2024-04-25T10:02:22.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb877fe640 1 -- 172.21.15.133:0/2097657316 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb700ab600 con 0x7feb880cd820 2024-04-25T10:02:22.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb877fe640 1 -- 172.21.15.133:0/2097657316 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb700ab9d0 con 0x7feb880cd820 2024-04-25T10:02:22.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 -- 172.21.15.133:0/2097657316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb880cd820 msgr2=0x7feb880cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 --2- 172.21.15.133:0/2097657316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb880cd820 0x7feb880cdca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feb880b27f0 tx=0x7feb7001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 -- 172.21.15.133:0/2097657316 shutdown_connections 2024-04-25T10:02:22.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 --2- 172.21.15.133:0/2097657316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb880cd820 0x7feb880cdca0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 --2- 172.21.15.133:0/2097657316 >> v2:172.21.15.162:3300/0 conn(0x7feb8815ced0 0x7feb8815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 -- 172.21.15.133:0/2097657316 >> 172.21.15.133:0/2097657316 conn(0x7feb88066f70 msgr2=0x7feb8806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 -- 172.21.15.133:0/2097657316 shutdown_connections 2024-04-25T10:02:22.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.932+0000 7feb8e89d640 1 -- 172.21.15.133:0/2097657316 wait complete. 2024-04-25T10:02:22.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8e89d640 1 Processor -- start 2024-04-25T10:02:22.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8e89d640 1 -- start start 2024-04-25T10:02:22.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8e89d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb880cd820 0x7feb881f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8e89d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb88145170 0x7feb88152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8e89d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb8815ced0 0x7feb8815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8e89d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb88168610 con 0x7feb88145170 2024-04-25T10:02:22.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8e89d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb88168490 con 0x7feb880cd820 2024-04-25T10:02:22.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8e89d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb88168790 con 0x7feb8815ced0 2024-04-25T10:02:22.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb7ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb88145170 0x7feb88152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb7ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb88145170 0x7feb88152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48774/0 (socket says 172.21.15.133:48774) 2024-04-25T10:02:22.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb7ffff640 1 -- 172.21.15.133:0/625426941 learned_addr learned my addr 172.21.15.133:0/625426941 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:22.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb87fff640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb880cd820 0x7feb881f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8ce3a640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb8815ced0 0x7feb8815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb7ffff640 1 -- 172.21.15.133:0/625426941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb8815ced0 msgr2=0x7feb8815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb7ffff640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb8815ced0 0x7feb8815ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb7ffff640 1 -- 172.21.15.133:0/625426941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb880cd820 msgr2=0x7feb881f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb7ffff640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb880cd820 0x7feb881f6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb7ffff640 1 -- 172.21.15.133:0/625426941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb88141c00 con 0x7feb88145170 2024-04-25T10:02:22.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8ce3a640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb8815ced0 0x7feb8815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:22.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb7ffff640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb88145170 0x7feb88152160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feb700ae040 tx=0x7feb7001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb857fa640 1 -- 172.21.15.133:0/625426941 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb700ba020 con 0x7feb88145170 2024-04-25T10:02:22.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb857fa640 1 -- 172.21.15.133:0/625426941 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb700b6420 con 0x7feb88145170 2024-04-25T10:02:22.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8e89d640 1 -- 172.21.15.133:0/625426941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb88141ee0 con 0x7feb88145170 2024-04-25T10:02:22.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb857fa640 1 -- 172.21.15.133:0/625426941 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb700b0040 con 0x7feb88145170 2024-04-25T10:02:22.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb8e89d640 1 -- 172.21.15.133:0/625426941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb881424a0 con 0x7feb88145170 2024-04-25T10:02:22.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.936+0000 7feb7e7fc640 1 -- 172.21.15.133:0/625426941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb600052a0 con 0x7feb88145170 2024-04-25T10:02:22.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.940+0000 7feb857fa640 1 -- 172.21.15.133:0/625426941 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb700bd020 con 0x7feb88145170 2024-04-25T10:02:22.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.940+0000 7feb857fa640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb68066f90 0x7feb68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.940+0000 7feb857fa640 1 -- 172.21.15.133:0/625426941 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7feb7011f4b0 con 0x7feb88145170 2024-04-25T10:02:22.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.940+0000 7feb87fff640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb68066f90 0x7feb68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.940+0000 7feb87fff640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb68066f90 0x7feb68069450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feb78004e30 tx=0x7feb78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:22.944+0000 7feb857fa640 1 -- 172.21.15.133:0/625426941 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb700c4030 con 0x7feb88145170 2024-04-25T10:02:23.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.092+0000 7feb7e7fc640 1 -- 172.21.15.133:0/625426941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb60005860 con 0x7feb88145170 2024-04-25T10:02:23.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.092+0000 7feb857fa640 1 -- 172.21.15.133:0/625426941 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb700160d0 con 0x7feb88145170 2024-04-25T10:02:23.099 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:23.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 -- 172.21.15.133:0/625426941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb68066f90 msgr2=0x7feb68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb68066f90 0x7feb68069450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feb78004e30 tx=0x7feb78009040 comp rx=0 tx=0).stop 2024-04-25T10:02:23.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 -- 172.21.15.133:0/625426941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb88145170 msgr2=0x7feb88152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb88145170 0x7feb88152160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feb700ae040 tx=0x7feb7001ad20 comp rx=0 tx=0).stop 2024-04-25T10:02:23.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 -- 172.21.15.133:0/625426941 shutdown_connections 2024-04-25T10:02:23.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb8815ced0 0x7feb8815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb68066f90 0x7feb68069450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb88145170 0x7feb88152160 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 --2- 172.21.15.133:0/625426941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb880cd820 0x7feb881f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 -- 172.21.15.133:0/625426941 >> 172.21.15.133:0/625426941 conn(0x7feb88066f70 msgr2=0x7feb8806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 -- 172.21.15.133:0/625426941 shutdown_connections 2024-04-25T10:02:23.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.096+0000 7feb7e7fc640 1 -- 172.21.15.133:0/625426941 wait complete. 2024-04-25T10:02:23.117 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:23.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.228+0000 7f54de997640 1 Processor -- start 2024-04-25T10:02:23.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.228+0000 7f54de997640 1 -- start start 2024-04-25T10:02:23.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54de997640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f54d80b2150 0x7f54d806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54de997640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d815ac50 con 0x7f54d8143ea0 2024-04-25T10:02:23.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54de997640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d80cd180 con 0x7f54d815b9b0 2024-04-25T10:02:23.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54de997640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d80ccf20 con 0x7f54d80b2150 2024-04-25T10:02:23.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f54d8143ea0 0x7f54d80cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52094/0 (socket says 172.21.15.133:52094) 2024-04-25T10:02:23.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d7fff640 1 -- 172.21.15.133:0/3828809889 learned_addr learned my addr 172.21.15.133:0/3828809889 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:23.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277941648 0 0) 0x7f54c8001120 con 0x7f54d8143ea0 2024-04-25T10:02:23.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d77fe640 1 --2- 172.21.15.133:0/3828809889 >> v2:172.21.15.162:3300/0 conn(0x7f54d80b2150 0x7f54d806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54c0003530 con 0x7f54d8143ea0 2024-04-25T10:02:23.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 680891541 0 0) 0x7f54c0003530 con 0x7f54d8143ea0 2024-04-25T10:02:23.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 >> v2:172.21.15.162:3300/0 conn(0x7f54d80b2150 msgr2=0x7f54d806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 --2- 172.21.15.133:0/3828809889 >> v2:172.21.15.162:3300/0 conn(0x7f54d80b2150 0x7f54d806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 >> v1:172.21.15.162:6789/0 conn(0x7f54d815b9b0 legacy=0x7f54d80b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d815aed0 con 0x7f54d8143ea0 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f54c8001850 con 0x7f54d8143ea0 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 >> v1:172.21.15.133:6789/0 conn(0x7f54d8143ea0 legacy=0x7f54d80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 --2- 172.21.15.133:0/3828809889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54c00039c0 0x7f54c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 --2- 172.21.15.133:0/3828809889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54c0004810 0x7f54c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 --2- 172.21.15.133:0/3828809889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54c00052f0 0x7f54c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d80ccf20 con 0x7f54c00052f0 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c000c1c0 con 0x7f54c0004810 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c000c3c0 con 0x7f54c00039c0 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54dcf34640 1 --2- 172.21.15.133:0/3828809889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54c00052f0 0x7f54c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d77fe640 1 --2- 172.21.15.133:0/3828809889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54c0004810 0x7f54c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54d7fff640 1 --2- 172.21.15.133:0/3828809889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54c00039c0 0x7f54c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54dcf34640 1 -- 172.21.15.133:0/3828809889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54c00039c0 msgr2=0x7f54c0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54dcf34640 1 --2- 172.21.15.133:0/3828809889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54c00039c0 0x7f54c0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54dcf34640 1 -- 172.21.15.133:0/3828809889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54c0004810 msgr2=0x7f54c0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54dcf34640 1 --2- 172.21.15.133:0/3828809889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54c0004810 0x7f54c0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.232+0000 7f54dcf34640 1 -- 172.21.15.133:0/3828809889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c0003530 con 0x7f54c00052f0 2024-04-25T10:02:23.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54dcf34640 1 -- 172.21.15.133:0/3828809889 reap_dead start 2024-04-25T10:02:23.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54dcf34640 1 --2- 172.21.15.133:0/3828809889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54c00052f0 0x7f54c000b9d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f54d815bdc0 tx=0x7f54d001c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbccbf234f01dcb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0001a90 con 0x7f54c00052f0 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d0003b80 con 0x7f54c00052f0 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54d6ffd640 1 -- 172.21.15.133:0/3828809889 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0005020 con 0x7f54c00052f0 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 -- 172.21.15.133:0/3828809889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54c00052f0 msgr2=0x7f54c000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 --2- 172.21.15.133:0/3828809889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54c00052f0 0x7f54c000b9d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f54d815bdc0 tx=0x7f54d001c720 comp rx=0 tx=0).stop 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 -- 172.21.15.133:0/3828809889 shutdown_connections 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 --2- 172.21.15.133:0/3828809889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54c00052f0 0x7f54c000b9d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 -- 172.21.15.133:0/3828809889 >> 172.21.15.133:0/3828809889 conn(0x7f54d8066f70 msgr2=0x7f54d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 -- 172.21.15.133:0/3828809889 shutdown_connections 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 -- 172.21.15.133:0/3828809889 wait complete. 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 Processor -- start 2024-04-25T10:02:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 -- start start 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54c00052f0 0x7f54d813fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54d81e4ef0 0x7f54d81539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54d81e5370 0x7f54d8153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d8144280 con 0x7f54d81e5370 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d815b490 con 0x7f54d81e4ef0 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d8144400 con 0x7f54c00052f0 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54dcf34640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54d81e5370 0x7f54d8153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54dcf34640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54d81e5370 0x7f54d8153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48786/0 (socket says 172.21.15.133:48786) 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54dcf34640 1 -- 172.21.15.133:0/3150991118 learned_addr learned my addr 172.21.15.133:0/3150991118 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54d77fe640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54d81e4ef0 0x7f54d81539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54d7fff640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54c00052f0 0x7f54d813fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54dcf34640 1 -- 172.21.15.133:0/3150991118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54c00052f0 msgr2=0x7f54d813fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54dcf34640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54c00052f0 0x7f54d813fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54dcf34640 1 -- 172.21.15.133:0/3150991118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54d81e4ef0 msgr2=0x7f54d81539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54dcf34640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54d81e4ef0 0x7f54d81539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54dcf34640 1 -- 172.21.15.133:0/3150991118 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d81e69e0 con 0x7f54d81e5370 2024-04-25T10:02:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54dcf34640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54d81e5370 0x7f54d8153f30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f54d001cc00 tx=0x7f54c0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54d4ff9640 1 -- 172.21.15.133:0/3150991118 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0001260 con 0x7f54d81e5370 2024-04-25T10:02:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54d4ff9640 1 -- 172.21.15.133:0/3150991118 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d0003980 con 0x7f54d81e5370 2024-04-25T10:02:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 -- 172.21.15.133:0/3150991118 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d81e5940 con 0x7f54d81e5370 2024-04-25T10:02:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54d4ff9640 1 -- 172.21.15.133:0/3150991118 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0006da0 con 0x7f54d81e5370 2024-04-25T10:02:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.236+0000 7f54de997640 1 -- 172.21.15.133:0/3150991118 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d81e5f00 con 0x7f54d81e5370 2024-04-25T10:02:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.240+0000 7f54de997640 1 -- 172.21.15.133:0/3150991118 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d8068e80 con 0x7f54d81e5370 2024-04-25T10:02:23.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.240+0000 7f54d4ff9640 1 -- 172.21.15.133:0/3150991118 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54d0005070 con 0x7f54d81e5370 2024-04-25T10:02:23.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.240+0000 7f54d4ff9640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54c4067100 0x7f54c40695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.240+0000 7f54d4ff9640 1 -- 172.21.15.133:0/3150991118 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f54d011e520 con 0x7f54d81e5370 2024-04-25T10:02:23.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.240+0000 7f54d7fff640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54c4067100 0x7f54c40695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.240+0000 7f54d7fff640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54c4067100 0x7f54c40695c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54c80012c0 tx=0x7f54c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.244+0000 7f54d4ff9640 1 -- 172.21.15.133:0/3150991118 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54d0016430 con 0x7f54d81e5370 2024-04-25T10:02:23.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.388+0000 7f54de997640 1 -- 172.21.15.133:0/3150991118 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f54d80b3200 con 0x7f54d81e5370 2024-04-25T10:02:23.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54d4ff9640 1 -- 172.21.15.133:0/3150991118 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f54d00ef1f0 con 0x7f54d81e5370 2024-04-25T10:02:23.398 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 -- 172.21.15.133:0/3150991118 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54c4067100 msgr2=0x7f54c40695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54c4067100 0x7f54c40695c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54c80012c0 tx=0x7f54c8009040 comp rx=0 tx=0).stop 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 -- 172.21.15.133:0/3150991118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54d81e5370 msgr2=0x7f54d8153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54d81e5370 0x7f54d8153f30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f54d001cc00 tx=0x7f54c0005840 comp rx=0 tx=0).stop 2024-04-25T10:02:23.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 -- 172.21.15.133:0/3150991118 shutdown_connections 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54c4067100 0x7f54c40695c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54d81e5370 0x7f54d8153f30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f54d81e4ef0 0x7f54d81539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 --2- 172.21.15.133:0/3150991118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f54c00052f0 0x7f54d813fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 -- 172.21.15.133:0/3150991118 >> 172.21.15.133:0/3150991118 conn(0x7f54d8066f70 msgr2=0x7f54d815d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 -- 172.21.15.133:0/3150991118 shutdown_connections 2024-04-25T10:02:23.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.392+0000 7f54de997640 1 -- 172.21.15.133:0/3150991118 wait complete. 2024-04-25T10:02:23.415 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:23.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f5371d640 1 Processor -- start 2024-04-25T10:02:23.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f5371d640 1 -- start start 2024-04-25T10:02:23.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f5371d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0f4c0cd820 0x7f0f4c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f5371d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4c0ce2e0 con 0x7f0f4c145170 2024-04-25T10:02:23.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f5371d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4c06d4b0 con 0x7f0f4c15ced0 2024-04-25T10:02:23.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f5371d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4c06db00 con 0x7f0f4c0cd820 2024-04-25T10:02:23.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f50cb8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0f4c0cd820 0x7f0f4c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f50cb8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0f4c0cd820 0x7f0f4c0cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48796/0 (socket says 172.21.15.133:48796) 2024-04-25T10:02:23.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f50cb8640 1 -- 172.21.15.133:0/3407837651 learned_addr learned my addr 172.21.15.133:0/3407837651 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:23.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f50cb8640 1 -- 172.21.15.133:0/3407837651 >> v1:172.21.15.162:6789/0 conn(0x7f0f4c15ced0 legacy=0x7f0f4c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f50cb8640 1 -- 172.21.15.133:0/3407837651 >> v1:172.21.15.133:6789/0 conn(0x7f0f4c145170 legacy=0x7f0f4c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f50cb8640 1 -- 172.21.15.133:0/3407837651 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c06d790 con 0x7f0f4c0cd820 2024-04-25T10:02:23.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.520+0000 7f0f50cb8640 1 --2- 172.21.15.133:0/3407837651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f4c0cd820 0x7f0f4c0cdca0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009870 tx=0x7f0f3c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8088aaa416d58ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f43fff640 1 -- 172.21.15.133:0/3407837651 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c0b0070 con 0x7f0f4c0cd820 2024-04-25T10:02:23.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f43fff640 1 -- 172.21.15.133:0/3407837651 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c0ab600 con 0x7f0f4c0cd820 2024-04-25T10:02:23.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f43fff640 1 -- 172.21.15.133:0/3407837651 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c0ab9d0 con 0x7f0f4c0cd820 2024-04-25T10:02:23.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 -- 172.21.15.133:0/3407837651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f4c0cd820 msgr2=0x7f0f4c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 --2- 172.21.15.133:0/3407837651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f4c0cd820 0x7f0f4c0cdca0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009870 tx=0x7f0f3c01c520 comp rx=0 tx=0).stop 2024-04-25T10:02:23.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 -- 172.21.15.133:0/3407837651 shutdown_connections 2024-04-25T10:02:23.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 --2- 172.21.15.133:0/3407837651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f4c0cd820 0x7f0f4c0cdca0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 -- 172.21.15.133:0/3407837651 >> 172.21.15.133:0/3407837651 conn(0x7f0f4c066f70 msgr2=0x7f0f4c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 -- 172.21.15.133:0/3407837651 shutdown_connections 2024-04-25T10:02:23.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 -- 172.21.15.133:0/3407837651 wait complete. 2024-04-25T10:02:23.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 Processor -- start 2024-04-25T10:02:23.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 -- start start 2024-04-25T10:02:23.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f4c0cd820 0x7f0f4c151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f4c145170 0x7f0f4c133910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f4c15ced0 0x7f0f4c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c15d320 con 0x7f0f4c15ced0 2024-04-25T10:02:23.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c145550 con 0x7f0f4c145170 2024-04-25T10:02:23.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c15d4a0 con 0x7f0f4c0cd820 2024-04-25T10:02:23.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f51cba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f4c15ced0 0x7f0f4c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f51cba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f4c15ced0 0x7f0f4c133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48810/0 (socket says 172.21.15.133:48810) 2024-04-25T10:02:23.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f51cba640 1 -- 172.21.15.133:0/4162124747 learned_addr learned my addr 172.21.15.133:0/4162124747 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:23.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f51cba640 1 -- 172.21.15.133:0/4162124747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f4c0cd820 msgr2=0x7f0f4c151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f50cb8640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f4c145170 0x7f0f4c133910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f514b9640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f4c0cd820 0x7f0f4c151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f51cba640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f4c0cd820 0x7f0f4c151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f51cba640 1 -- 172.21.15.133:0/4162124747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f4c145170 msgr2=0x7f0f4c133910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f51cba640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f4c145170 0x7f0f4c133910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f51cba640 1 -- 172.21.15.133:0/4162124747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c06e470 con 0x7f0f4c15ced0 2024-04-25T10:02:23.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f51cba640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f4c15ced0 0x7f0f4c133e50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400b9c0 tx=0x7f0f4400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f41ffb640 1 -- 172.21.15.133:0/4162124747 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44019070 con 0x7f0f4c15ced0 2024-04-25T10:02:23.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f41ffb640 1 -- 172.21.15.133:0/4162124747 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4400cba0 con 0x7f0f4c15ced0 2024-04-25T10:02:23.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 -- 172.21.15.133:0/4162124747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4c135c70 con 0x7f0f4c15ced0 2024-04-25T10:02:23.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f41ffb640 1 -- 172.21.15.133:0/4162124747 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44014440 con 0x7f0f4c15ced0 2024-04-25T10:02:23.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.524+0000 7f0f5371d640 1 -- 172.21.15.133:0/4162124747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4c1356b0 con 0x7f0f4c15ced0 2024-04-25T10:02:23.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.528+0000 7f0f5371d640 1 -- 172.21.15.133:0/4162124747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4c068e80 con 0x7f0f4c15ced0 2024-04-25T10:02:23.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.528+0000 7f0f41ffb640 1 -- 172.21.15.133:0/4162124747 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f4401c070 con 0x7f0f4c15ced0 2024-04-25T10:02:23.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.528+0000 7f0f41ffb640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f30066f60 0x7f0f30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.528+0000 7f0f41ffb640 1 -- 172.21.15.133:0/4162124747 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0f44085040 con 0x7f0f4c15ced0 2024-04-25T10:02:23.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.528+0000 7f0f514b9640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f30066f60 0x7f0f30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.528+0000 7f0f514b9640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f30066f60 0x7f0f30069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c06e400 tx=0x7f0f48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.532+0000 7f0f41ffb640 1 -- 172.21.15.133:0/4162124747 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f44055da0 con 0x7f0f4c15ced0 2024-04-25T10:02:23.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.676+0000 7f0f5371d640 1 -- 172.21.15.133:0/4162124747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f4c0b2b80 con 0x7f0f4c15ced0 2024-04-25T10:02:23.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.676+0000 7f0f41ffb640 1 -- 172.21.15.133:0/4162124747 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f440556a0 con 0x7f0f4c15ced0 2024-04-25T10:02:23.684 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 -- 172.21.15.133:0/4162124747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f30066f60 msgr2=0x7f0f30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f30066f60 0x7f0f30069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c06e400 tx=0x7f0f48009040 comp rx=0 tx=0).stop 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 -- 172.21.15.133:0/4162124747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f4c15ced0 msgr2=0x7f0f4c133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f4c15ced0 0x7f0f4c133e50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400b9c0 tx=0x7f0f4400be80 comp rx=0 tx=0).stop 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 -- 172.21.15.133:0/4162124747 shutdown_connections 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f30066f60 0x7f0f30069420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f4c15ced0 0x7f0f4c133e50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f4c145170 0x7f0f4c133910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 --2- 172.21.15.133:0/4162124747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f4c0cd820 0x7f0f4c151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 -- 172.21.15.133:0/4162124747 >> 172.21.15.133:0/4162124747 conn(0x7f0f4c066f70 msgr2=0x7f0f4c15ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 -- 172.21.15.133:0/4162124747 shutdown_connections 2024-04-25T10:02:23.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.680+0000 7f0f5371d640 1 -- 172.21.15.133:0/4162124747 wait complete. 2024-04-25T10:02:23.702 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:23.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607e984640 1 Processor -- start 2024-04-25T10:02:23.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607e984640 1 -- start start 2024-04-25T10:02:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607e984640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f607815bbb0 0x7f60780b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607e984640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60780afe00 con 0x7f60780cc790 2024-04-25T10:02:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607e984640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60780b00e0 con 0x7f60780cc330 2024-04-25T10:02:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607e984640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60780b0730 con 0x7f607815bbb0 2024-04-25T10:02:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607cf21640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f60780cc790 0x7f60780ccf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52118/0 (socket says 172.21.15.133:52118) 2024-04-25T10:02:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607cf21640 1 -- 172.21.15.133:0/983563953 learned_addr learned my addr 172.21.15.133:0/983563953 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6077fff640 1 --2- 172.21.15.133:0/983563953 >> v2:172.21.15.162:3300/0 conn(0x7f607815bbb0 0x7f60780b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25886505 0 0) 0x7f60780afe00 con 0x7f60780cc790 2024-04-25T10:02:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6060003530 con 0x7f60780cc790 2024-04-25T10:02:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3843939418 0 0) 0x7f6060003530 con 0x7f60780cc790 2024-04-25T10:02:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 >> v2:172.21.15.162:3300/0 conn(0x7f607815bbb0 msgr2=0x7f60780b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 --2- 172.21.15.133:0/983563953 >> v2:172.21.15.162:3300/0 conn(0x7f607815bbb0 0x7f60780b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 >> v1:172.21.15.162:6789/0 conn(0x7f60780cc330 legacy=0x7f60780ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60780b03c0 con 0x7f60780cc790 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6070001a00 con 0x7f60780cc790 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 >> v1:172.21.15.133:6789/0 conn(0x7f60780cc790 legacy=0x7f60780ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60600039c0 0x7f6060003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6060004810 0x7f6060004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60600052f0 0x7f606000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780b0730 con 0x7f60600052f0 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606000c1c0 con 0x7f60600039c0 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606000c3c0 con 0x7f6060004810 2024-04-25T10:02:23.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607cf21640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60600052f0 0x7f606000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f6077fff640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60600039c0 0x7f6060003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f60777fe640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6060004810 0x7f6060004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607cf21640 1 -- 172.21.15.133:0/983563953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6060004810 msgr2=0x7f6060004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607cf21640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6060004810 0x7f6060004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607cf21640 1 -- 172.21.15.133:0/983563953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60600039c0 msgr2=0x7f6060003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607cf21640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60600039c0 0x7f6060003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f607cf21640 1 -- 172.21.15.133:0/983563953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6060003530 con 0x7f60600052f0 2024-04-25T10:02:23.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.820+0000 7f60777fe640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6060004810 0x7f6060004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:23.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607cf21640 1 -- 172.21.15.133:0/983563953 reap_dead start 2024-04-25T10:02:23.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607cf21640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60600052f0 0x7f606000b9d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f60700013e0 tx=0x7f607001d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb9457b7a1db0fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607000a4a0 con 0x7f60600052f0 2024-04-25T10:02:23.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60700ac420 con 0x7f60600052f0 2024-04-25T10:02:23.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f6076ffd640 1 -- 172.21.15.133:0/983563953 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700ac7f0 con 0x7f60600052f0 2024-04-25T10:02:23.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 -- 172.21.15.133:0/983563953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60600052f0 msgr2=0x7f606000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60600052f0 0x7f606000b9d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f60700013e0 tx=0x7f607001d3c0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 -- 172.21.15.133:0/983563953 shutdown_connections 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 --2- 172.21.15.133:0/983563953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60600052f0 0x7f606000b9d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 -- 172.21.15.133:0/983563953 >> 172.21.15.133:0/983563953 conn(0x7f60780d0570 msgr2=0x7f60780d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 -- 172.21.15.133:0/983563953 shutdown_connections 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 -- 172.21.15.133:0/983563953 wait complete. 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 Processor -- start 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 -- start start 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60600052f0 0x7f60780c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6078155300 0x7f60780adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6078155780 0x7f60780ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780b0f30 con 0x7f6078155300 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780b0db0 con 0x7f6078155780 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f607e984640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60780ad1c0 con 0x7f60600052f0 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f60777fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6078155300 0x7f60780adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f60777fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6078155300 0x7f60780adce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48822/0 (socket says 172.21.15.133:48822) 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.824+0000 7f60777fe640 1 -- 172.21.15.133:0/1729878865 learned_addr learned my addr 172.21.15.133:0/1729878865 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f607cf21640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6078155780 0x7f60780ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f6077fff640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60600052f0 0x7f60780c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f60777fe640 1 -- 172.21.15.133:0/1729878865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60600052f0 msgr2=0x7f60780c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f60777fe640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60600052f0 0x7f60780c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f60777fe640 1 -- 172.21.15.133:0/1729878865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6078155780 msgr2=0x7f60780ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f60777fe640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6078155780 0x7f60780ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f60777fe640 1 -- 172.21.15.133:0/1729878865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60780c95d0 con 0x7f6078155300 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f60777fe640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6078155300 0x7f60780adce0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f606c00d540 tx=0x7f606c00da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f6074ff9640 1 -- 172.21.15.133:0/1729878865 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c003620 con 0x7f6078155300 2024-04-25T10:02:23.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f6074ff9640 1 -- 172.21.15.133:0/1729878865 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c017070 con 0x7f6078155300 2024-04-25T10:02:23.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f607e984640 1 -- 172.21.15.133:0/1729878865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6078155d50 con 0x7f6078155300 2024-04-25T10:02:23.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f6074ff9640 1 -- 172.21.15.133:0/1729878865 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c0039e0 con 0x7f6078155300 2024-04-25T10:02:23.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f607e984640 1 -- 172.21.15.133:0/1729878865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6078156310 con 0x7f6078155300 2024-04-25T10:02:23.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f607e984640 1 -- 172.21.15.133:0/1729878865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6078067c70 con 0x7f6078155300 2024-04-25T10:02:23.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f6074ff9640 1 -- 172.21.15.133:0/1729878865 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f606c003b80 con 0x7f6078155300 2024-04-25T10:02:23.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f6074ff9640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6064066f90 0x7f6064069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f6074ff9640 1 -- 172.21.15.133:0/1729878865 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f606c0850f0 con 0x7f6078155300 2024-04-25T10:02:23.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f6077fff640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6064066f90 0x7f6064069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.828+0000 7f6077fff640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6064066f90 0x7f6064069450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f60680099b0 tx=0x7f6068008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.832+0000 7f6074ff9640 1 -- 172.21.15.133:0/1729878865 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f606c055f10 con 0x7f6078155300 2024-04-25T10:02:23.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.992+0000 7f607e984640 1 -- 172.21.15.133:0/1729878865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f60780caa30 con 0x7f6078155300 2024-04-25T10:02:23.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.992+0000 7f6074ff9640 1 -- 172.21.15.133:0/1729878865 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f606c055810 con 0x7f6078155300 2024-04-25T10:02:23.999 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:24.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 -- 172.21.15.133:0/1729878865 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6064066f90 msgr2=0x7f6064069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6064066f90 0x7f6064069450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f60680099b0 tx=0x7f6068008040 comp rx=0 tx=0).stop 2024-04-25T10:02:24.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 -- 172.21.15.133:0/1729878865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6078155300 msgr2=0x7f60780adce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6078155300 0x7f60780adce0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f606c00d540 tx=0x7f606c00da00 comp rx=0 tx=0).stop 2024-04-25T10:02:24.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 -- 172.21.15.133:0/1729878865 shutdown_connections 2024-04-25T10:02:24.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6078155780 0x7f60780ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6064066f90 0x7f6064069450 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6078155300 0x7f60780adce0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 --2- 172.21.15.133:0/1729878865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60600052f0 0x7f60780c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 -- 172.21.15.133:0/1729878865 >> 172.21.15.133:0/1729878865 conn(0x7f60780d0570 msgr2=0x7f607815adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 -- 172.21.15.133:0/1729878865 shutdown_connections 2024-04-25T10:02:24.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:23.996+0000 7f607e984640 1 -- 172.21.15.133:0/1729878865 wait complete. 2024-04-25T10:02:24.017 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:24.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.124+0000 7fc8244f5640 1 Processor -- start 2024-04-25T10:02:24.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.124+0000 7fc8244f5640 1 -- start start 2024-04-25T10:02:24.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc8244f5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81c0b1f70 0x7fc81c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc8244f5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc81c0ce330 0x7fc81c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc8244f5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81c0c61d0 con 0x7fc81c15c9a0 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc8244f5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81c0cece0 con 0x7fc81c0ce330 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc8244f5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c0c64b0 con 0x7fc81c0b1f70 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc822a92640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc81c0ce330 0x7fc81c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc822a92640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc81c0ce330 0x7fc81c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48824/0 (socket says 172.21.15.133:48824) 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc822a92640 1 -- 172.21.15.133:0/3077180912 learned_addr learned my addr 172.21.15.133:0/3077180912 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc821a90640 1 --2- 172.21.15.133:0/3077180912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81c0b1f70 0x7fc81c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc822a92640 1 -- 172.21.15.133:0/3077180912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81c0b1f70 msgr2=0x7fc81c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc822a92640 1 --2- 172.21.15.133:0/3077180912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81c0b1f70 0x7fc81c06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc822a92640 1 -- 172.21.15.133:0/3077180912 >> v1:172.21.15.162:6789/0 conn(0x7fc81c15c9a0 legacy=0x7fc81c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc822a92640 1 -- 172.21.15.133:0/3077180912 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c06d4f0 con 0x7fc81c0ce330 2024-04-25T10:02:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc822a92640 1 --2- 172.21.15.133:0/3077180912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c0ce330 0x7fc81c0ce7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc814009e50 tx=0x7fc81401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dae7b25fd75c3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc82128f640 1 -- 172.21.15.133:0/3077180912 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8140a57e0 con 0x7fc81c0ce330 2024-04-25T10:02:24.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc82128f640 1 -- 172.21.15.133:0/3077180912 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8140a5980 con 0x7fc81c0ce330 2024-04-25T10:02:24.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc82128f640 1 -- 172.21.15.133:0/3077180912 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8140a5d50 con 0x7fc81c0ce330 2024-04-25T10:02:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc8244f5640 1 -- 172.21.15.133:0/3077180912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c0ce330 msgr2=0x7fc81c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.128+0000 7fc8244f5640 1 --2- 172.21.15.133:0/3077180912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c0ce330 0x7fc81c0ce7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc814009e50 tx=0x7fc81401ca00 comp rx=0 tx=0).stop 2024-04-25T10:02:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 -- 172.21.15.133:0/3077180912 shutdown_connections 2024-04-25T10:02:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 --2- 172.21.15.133:0/3077180912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c0ce330 0x7fc81c0ce7a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 --2- 172.21.15.133:0/3077180912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81c0b1f70 0x7fc81c06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 -- 172.21.15.133:0/3077180912 >> 172.21.15.133:0/3077180912 conn(0x7fc81c067000 msgr2=0x7fc81c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 -- 172.21.15.133:0/3077180912 shutdown_connections 2024-04-25T10:02:24.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 -- 172.21.15.133:0/3077180912 wait complete. 2024-04-25T10:02:24.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 Processor -- start 2024-04-25T10:02:24.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 -- start start 2024-04-25T10:02:24.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81c0b1f70 0x7fc81c149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc81c0ce330 0x7fc81c149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c15c9a0 0x7fc81c14bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c15dad0 con 0x7fc81c15c9a0 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c15d950 con 0x7fc81c0ce330 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c15dc50 con 0x7fc81c0b1f70 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc822a92640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c15c9a0 0x7fc81c14bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc822a92640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c15c9a0 0x7fc81c14bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48830/0 (socket says 172.21.15.133:48830) 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc822a92640 1 -- 172.21.15.133:0/4145790750 learned_addr learned my addr 172.21.15.133:0/4145790750 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc821a90640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc81c0ce330 0x7fc81c149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc822291640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81c0b1f70 0x7fc81c149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc822a92640 1 -- 172.21.15.133:0/4145790750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81c0b1f70 msgr2=0x7fc81c149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc822a92640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81c0b1f70 0x7fc81c149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc822a92640 1 -- 172.21.15.133:0/4145790750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc81c0ce330 msgr2=0x7fc81c149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc822a92640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc81c0ce330 0x7fc81c149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc822a92640 1 -- 172.21.15.133:0/4145790750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c14a9a0 con 0x7fc81c15c9a0 2024-04-25T10:02:24.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc822a92640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c15c9a0 0x7fc81c14bdf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc81401a630 tx=0x7fc8140197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc812ffd640 1 -- 172.21.15.133:0/4145790750 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8140b8670 con 0x7fc81c15c9a0 2024-04-25T10:02:24.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc812ffd640 1 -- 172.21.15.133:0/4145790750 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8140be070 con 0x7fc81c15c9a0 2024-04-25T10:02:24.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 -- 172.21.15.133:0/4145790750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c14ac80 con 0x7fc81c15c9a0 2024-04-25T10:02:24.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc812ffd640 1 -- 172.21.15.133:0/4145790750 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8140b8a50 con 0x7fc81c15c9a0 2024-04-25T10:02:24.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.132+0000 7fc8244f5640 1 -- 172.21.15.133:0/4145790750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c14b240 con 0x7fc81c15c9a0 2024-04-25T10:02:24.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.136+0000 7fc8244f5640 1 -- 172.21.15.133:0/4145790750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c06bc20 con 0x7fc81c15c9a0 2024-04-25T10:02:24.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.136+0000 7fc812ffd640 1 -- 172.21.15.133:0/4145790750 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8140b8bf0 con 0x7fc81c15c9a0 2024-04-25T10:02:24.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.136+0000 7fc812ffd640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc800066f60 0x7fc800069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.136+0000 7fc822291640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc800066f60 0x7fc800069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.136+0000 7fc812ffd640 1 -- 172.21.15.133:0/4145790750 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc814120490 con 0x7fc81c15c9a0 2024-04-25T10:02:24.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.136+0000 7fc822291640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc800066f60 0x7fc800069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc81c157810 tx=0x7fc818009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.140+0000 7fc812ffd640 1 -- 172.21.15.133:0/4145790750 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc814016620 con 0x7fc81c15c9a0 2024-04-25T10:02:24.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc8244f5640 1 -- 172.21.15.133:0/4145790750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc81c068ef0 con 0x7fc81c15c9a0 2024-04-25T10:02:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc812ffd640 1 -- 172.21.15.133:0/4145790750 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7fc8140f1280 con 0x7fc81c15c9a0 2024-04-25T10:02:24.327 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:24.327 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2230831968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2230831968},{"type":"v1","addr":"172.21.15.162:6835","nonce":2230831968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1456396097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1456396097},{"type":"v1","addr":"172.21.15.133:6836","nonce":1456396097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:51.276631+0000","modified":"2024-04-25T10:02:14.440426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2462154167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2462154167},{"type":"v1","addr":"172.21.15.133:6837","nonce":2462154167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 9 2024-04-25T10:02:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc8244f5640 1 -- 172.21.15.133:0/4145790750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc800066f60 msgr2=0x7fc800069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc8244f5640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc800066f60 0x7fc800069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc81c157810 tx=0x7fc818009040 comp rx=0 tx=0).stop 2024-04-25T10:02:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc8244f5640 1 -- 172.21.15.133:0/4145790750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c15c9a0 msgr2=0x7fc81c14bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc8244f5640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c15c9a0 0x7fc81c14bdf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc81401a630 tx=0x7fc8140197a0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc8244f5640 1 -- 172.21.15.133:0/4145790750 shutdown_connections 2024-04-25T10:02:24.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc8244f5640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc800066f60 0x7fc800069420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc8244f5640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c15c9a0 0x7fc81c14bdf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc8244f5640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc81c0ce330 0x7fc81c149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc8244f5640 1 --2- 172.21.15.133:0/4145790750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81c0b1f70 0x7fc81c149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.320+0000 7fc8244f5640 1 -- 172.21.15.133:0/4145790750 >> 172.21.15.133:0/4145790750 conn(0x7fc81c067000 msgr2=0x7fc81c15afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.324+0000 7fc8244f5640 1 -- 172.21.15.133:0/4145790750 shutdown_connections 2024-04-25T10:02:24.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.324+0000 7fc8244f5640 1 -- 172.21.15.133:0/4145790750 wait complete. 2024-04-25T10:02:24.347 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:24.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59d4fc9640 1 Processor -- start 2024-04-25T10:02:24.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59d4fc9640 1 -- start start 2024-04-25T10:02:24.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59d4fc9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f59d00cc860 0x7f59d00cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59d4fc9640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d006e640 con 0x7f59d0161cb0 2024-04-25T10:02:24.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59d4fc9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d006e920 con 0x7f59d0162090 2024-04-25T10:02:24.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59d4fc9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d006ec00 con 0x7f59d00cc860 2024-04-25T10:02:24.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59ce59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f59d0161cb0 0x7f59d006dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52134/0 (socket says 172.21.15.133:52134) 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59ce59c640 1 -- 172.21.15.133:0/1269884041 learned_addr learned my addr 172.21.15.133:0/1269884041 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59c5d9b640 1 --2- 172.21.15.133:0/1269884041 >> v2:172.21.15.162:3300/0 conn(0x7f59d00cc860 0x7f59d00cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184288697 0 0) 0x7f59d006e640 con 0x7f59d0161cb0 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59bc003530 con 0x7f59d0161cb0 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4075131352 0 0) 0x7f59bc003530 con 0x7f59d0161cb0 2024-04-25T10:02:24.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 >> v2:172.21.15.162:3300/0 conn(0x7f59d00cc860 msgr2=0x7f59d00cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 --2- 172.21.15.133:0/1269884041 >> v2:172.21.15.162:3300/0 conn(0x7f59d00cc860 0x7f59d00cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 >> v1:172.21.15.162:6789/0 conn(0x7f59d0162090 legacy=0x7f59d0162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d01625b0 con 0x7f59d0161cb0 2024-04-25T10:02:24.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f59c0001680 con 0x7f59d0161cb0 2024-04-25T10:02:24.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 >> v1:172.21.15.133:6789/0 conn(0x7f59d0161cb0 legacy=0x7f59d006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 --2- 172.21.15.133:0/1269884041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 0x7f59bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 --2- 172.21.15.133:0/1269884041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59bc004810 0x7f59bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 --2- 172.21.15.133:0/1269884041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59bc0051d0 0x7f59bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d006ec00 con 0x7f59bc0039c0 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc00c0a0 con 0x7f59bc004810 2024-04-25T10:02:24.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc00c2a0 con 0x7f59bc0051d0 2024-04-25T10:02:24.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59ce59c640 1 --2- 172.21.15.133:0/1269884041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 0x7f59bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59c5d9b640 1 --2- 172.21.15.133:0/1269884041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59bc004810 0x7f59bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59ced9d640 1 --2- 172.21.15.133:0/1269884041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59bc0051d0 0x7f59bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59ce59c640 1 -- 172.21.15.133:0/1269884041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59bc0051d0 msgr2=0x7f59bc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59ce59c640 1 --2- 172.21.15.133:0/1269884041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59bc0051d0 0x7f59bc00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59ce59c640 1 -- 172.21.15.133:0/1269884041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59bc004810 msgr2=0x7f59bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59ce59c640 1 --2- 172.21.15.133:0/1269884041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59bc004810 0x7f59bc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59ce59c640 1 -- 172.21.15.133:0/1269884041 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc003530 con 0x7f59bc0039c0 2024-04-25T10:02:24.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.460+0000 7f59ced9d640 1 -- 172.21.15.133:0/1269884041 reap_dead start 2024-04-25T10:02:24.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59ce59c640 1 --2- 172.21.15.133:0/1269884041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 0x7f59bc003e00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59c000c1c0 tx=0x7f59c001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794205f9d6266ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0007d00 con 0x7f59bc0039c0 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c00a9020 con 0x7f59bc0039c0 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59cdd9b640 1 -- 172.21.15.133:0/1269884041 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0005b20 con 0x7f59bc0039c0 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 -- 172.21.15.133:0/1269884041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 msgr2=0x7f59bc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 --2- 172.21.15.133:0/1269884041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 0x7f59bc003e00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59c000c1c0 tx=0x7f59c001cc00 comp rx=0 tx=0).stop 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 -- 172.21.15.133:0/1269884041 shutdown_connections 2024-04-25T10:02:24.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 --2- 172.21.15.133:0/1269884041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 0x7f59bc003e00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 -- 172.21.15.133:0/1269884041 >> 172.21.15.133:0/1269884041 conn(0x7f59d0067000 msgr2=0x7f59d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 -- 172.21.15.133:0/1269884041 shutdown_connections 2024-04-25T10:02:24.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 -- 172.21.15.133:0/1269884041 wait complete. 2024-04-25T10:02:24.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 Processor -- start 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 -- start start 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 0x7f59d01582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d01571f0 0x7f59d0149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.464+0000 7f59d4fc9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d015ace0 0x7f59d015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59d4fc9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d00b0dc0 con 0x7f59bc0039c0 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59d4fc9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d00b0c40 con 0x7f59d01571f0 2024-04-25T10:02:24.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59d4fc9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d00b0f40 con 0x7f59d015ace0 2024-04-25T10:02:24.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59ce59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 0x7f59d01582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59ce59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 0x7f59d01582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48858/0 (socket says 172.21.15.133:48858) 2024-04-25T10:02:24.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59ce59c640 1 -- 172.21.15.133:0/626715962 learned_addr learned my addr 172.21.15.133:0/626715962 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:24.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59ced9d640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d015ace0 0x7f59d015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59c5d9b640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d01571f0 0x7f59d0149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59ce59c640 1 -- 172.21.15.133:0/626715962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d015ace0 msgr2=0x7f59d015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59ce59c640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d015ace0 0x7f59d015b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59ce59c640 1 -- 172.21.15.133:0/626715962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d01571f0 msgr2=0x7f59d0149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59ce59c640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d01571f0 0x7f59d0149e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59ce59c640 1 -- 172.21.15.133:0/626715962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d0156430 con 0x7f59bc0039c0 2024-04-25T10:02:24.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59c5d9b640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d01571f0 0x7f59d0149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:24.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59ce59c640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 0x7f59d01582c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f59c0000c10 tx=0x7f59bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59c77fe640 1 -- 172.21.15.133:0/626715962 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00bb070 con 0x7f59bc0039c0 2024-04-25T10:02:24.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59c77fe640 1 -- 172.21.15.133:0/626715962 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c0001650 con 0x7f59bc0039c0 2024-04-25T10:02:24.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59d4fc9640 1 -- 172.21.15.133:0/626715962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d0156710 con 0x7f59bc0039c0 2024-04-25T10:02:24.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59c77fe640 1 -- 172.21.15.133:0/626715962 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0007d00 con 0x7f59bc0039c0 2024-04-25T10:02:24.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59d4fc9640 1 -- 172.21.15.133:0/626715962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d0156cd0 con 0x7f59bc0039c0 2024-04-25T10:02:24.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59d4fc9640 1 -- 172.21.15.133:0/626715962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d0068ef0 con 0x7f59bc0039c0 2024-04-25T10:02:24.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59c77fe640 1 -- 172.21.15.133:0/626715962 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59c00be070 con 0x7f59bc0039c0 2024-04-25T10:02:24.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59c77fe640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59b0066f60 0x7f59b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59c77fe640 1 -- 172.21.15.133:0/626715962 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f59c011e620 con 0x7f59bc0039c0 2024-04-25T10:02:24.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59c5d9b640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59b0066f60 0x7f59b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.468+0000 7f59c5d9b640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59b0066f60 0x7f59b0069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f59d01654a0 tx=0x7f59b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.472+0000 7f59c77fe640 1 -- 172.21.15.133:0/626715962 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59c0016970 con 0x7f59bc0039c0 2024-04-25T10:02:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.632+0000 7f59d4fc9640 1 -- 172.21.15.133:0/626715962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59d00b23e0 con 0x7f59bc0039c0 2024-04-25T10:02:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.632+0000 7f59c77fe640 1 -- 172.21.15.133:0/626715962 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7f59c00ef2f0 con 0x7f59bc0039c0 2024-04-25T10:02:24.640 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:24.640 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:24.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 -- 172.21.15.133:0/626715962 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59b0066f60 msgr2=0x7f59b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59b0066f60 0x7f59b0069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f59d01654a0 tx=0x7f59b8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 -- 172.21.15.133:0/626715962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 msgr2=0x7f59d01582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 0x7f59d01582c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f59c0000c10 tx=0x7f59bc005840 comp rx=0 tx=0).stop 2024-04-25T10:02:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 -- 172.21.15.133:0/626715962 shutdown_connections 2024-04-25T10:02:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d015ace0 0x7f59d015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d01571f0 0x7f59d0149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59b0066f60 0x7f59b0069420 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 --2- 172.21.15.133:0/626715962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59bc0039c0 0x7f59d01582c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 -- 172.21.15.133:0/626715962 >> 172.21.15.133:0/626715962 conn(0x7f59d0067000 msgr2=0x7f59d0165990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 -- 172.21.15.133:0/626715962 shutdown_connections 2024-04-25T10:02:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.636+0000 7f59d4fc9640 1 -- 172.21.15.133:0/626715962 wait complete. 2024-04-25T10:02:24.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:24.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:24.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff464489640 1 Processor -- start 2024-04-25T10:02:24.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff464489640 1 -- start start 2024-04-25T10:02:24.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff464489640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff45c145170 0x7ff45c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff464489640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c15ced0 0x7ff45c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff464489640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45c0ce2e0 con 0x7ff45c06e4d0 2024-04-25T10:02:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff464489640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45c06db00 con 0x7ff45c145170 2024-04-25T10:02:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff464489640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c145550 con 0x7ff45c15ced0 2024-04-25T10:02:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff462225640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff45c145170 0x7ff45c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff462a26640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c15ced0 0x7ff45c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff462225640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff45c145170 0x7ff45c06df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:56034/0 (socket says 172.21.15.133:56034) 2024-04-25T10:02:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff462225640 1 -- 172.21.15.133:0/1103971666 learned_addr learned my addr 172.21.15.133:0/1103971666 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff462a26640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c15ced0 0x7ff45c15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39980/0 (socket says 172.21.15.133:39980) 2024-04-25T10:02:24.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff462a26640 1 -- 172.21.15.133:0/1103971666 >> v2:172.21.15.162:3300/0 conn(0x7ff45c145170 msgr2=0x7ff45c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff462a26640 1 --2- 172.21.15.133:0/1103971666 >> v2:172.21.15.162:3300/0 conn(0x7ff45c145170 0x7ff45c06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff462a26640 1 -- 172.21.15.133:0/1103971666 >> v1:172.21.15.162:6789/0 conn(0x7ff45c06e4d0 legacy=0x7ff45c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff462a26640 1 -- 172.21.15.133:0/1103971666 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c06d4b0 con 0x7ff45c15ced0 2024-04-25T10:02:24.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.780+0000 7ff462225640 1 --2- 172.21.15.133:0/1103971666 >> v2:172.21.15.162:3300/0 conn(0x7ff45c145170 0x7ff45c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:24.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff462a26640 1 --2- 172.21.15.133:0/1103971666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c15ced0 0x7ff45c15d360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff454009d90 tx=0x7ff45401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a681dde516190b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff461223640 1 -- 172.21.15.133:0/1103971666 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4540b2070 con 0x7ff45c15ced0 2024-04-25T10:02:24.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff461223640 1 -- 172.21.15.133:0/1103971666 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4540a5830 con 0x7ff45c15ced0 2024-04-25T10:02:24.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff461223640 1 -- 172.21.15.133:0/1103971666 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4540a5c00 con 0x7ff45c15ced0 2024-04-25T10:02:24.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff464489640 1 -- 172.21.15.133:0/1103971666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c15ced0 msgr2=0x7ff45c15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff464489640 1 --2- 172.21.15.133:0/1103971666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c15ced0 0x7ff45c15d360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff454009d90 tx=0x7ff45401c970 comp rx=0 tx=0).stop 2024-04-25T10:02:24.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff464489640 1 -- 172.21.15.133:0/1103971666 shutdown_connections 2024-04-25T10:02:24.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff464489640 1 --2- 172.21.15.133:0/1103971666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c15ced0 0x7ff45c15d360 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff464489640 1 --2- 172.21.15.133:0/1103971666 >> v2:172.21.15.162:3300/0 conn(0x7ff45c145170 0x7ff45c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff464489640 1 -- 172.21.15.133:0/1103971666 >> 172.21.15.133:0/1103971666 conn(0x7ff45c066f70 msgr2=0x7ff45c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff464489640 1 -- 172.21.15.133:0/1103971666 shutdown_connections 2024-04-25T10:02:24.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff464489640 1 -- 172.21.15.133:0/1103971666 wait complete. 2024-04-25T10:02:24.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff464489640 1 Processor -- start 2024-04-25T10:02:24.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.784+0000 7ff464489640 1 -- start start 2024-04-25T10:02:24.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff464489640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff45c06e4d0 0x7ff45c13abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff464489640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c145170 0x7ff45c13cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff464489640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff45c15ced0 0x7ff45c133900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff464489640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c168610 con 0x7ff45c06e4d0 2024-04-25T10:02:24.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff464489640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c168490 con 0x7ff45c15ced0 2024-04-25T10:02:24.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff464489640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c168790 con 0x7ff45c145170 2024-04-25T10:02:24.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff462225640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff45c06e4d0 0x7ff45c13abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff462225640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff45c06e4d0 0x7ff45c13abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48872/0 (socket says 172.21.15.133:48872) 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff462225640 1 -- 172.21.15.133:0/1773959765 learned_addr learned my addr 172.21.15.133:0/1773959765 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff462a26640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff45c15ced0 0x7ff45c133900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff461a24640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c145170 0x7ff45c13cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff462225640 1 -- 172.21.15.133:0/1773959765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c145170 msgr2=0x7ff45c13cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff462225640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c145170 0x7ff45c13cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff462225640 1 -- 172.21.15.133:0/1773959765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff45c15ced0 msgr2=0x7ff45c133900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff462225640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff45c15ced0 0x7ff45c133900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff462225640 1 -- 172.21.15.133:0/1773959765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c13bcf0 con 0x7ff45c06e4d0 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff462a26640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff45c15ced0 0x7ff45c133900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff462225640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff45c06e4d0 0x7ff45c13abe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff45800ea00 tx=0x7ff45800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff452ffd640 1 -- 172.21.15.133:0/1773959765 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458017070 con 0x7ff45c06e4d0 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff452ffd640 1 -- 172.21.15.133:0/1773959765 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45800ca60 con 0x7ff45c06e4d0 2024-04-25T10:02:24.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff464489640 1 -- 172.21.15.133:0/1773959765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45c13bfd0 con 0x7ff45c06e4d0 2024-04-25T10:02:24.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff452ffd640 1 -- 172.21.15.133:0/1773959765 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45800ce10 con 0x7ff45c06e4d0 2024-04-25T10:02:24.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff464489640 1 -- 172.21.15.133:0/1773959765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45c1345e0 con 0x7ff45c06e4d0 2024-04-25T10:02:24.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff464489640 1 -- 172.21.15.133:0/1773959765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45c068e80 con 0x7ff45c06e4d0 2024-04-25T10:02:24.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff452ffd640 1 -- 172.21.15.133:0/1773959765 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff45801a070 con 0x7ff45c06e4d0 2024-04-25T10:02:24.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff452ffd640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff440066f60 0x7ff440069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff452ffd640 1 -- 172.21.15.133:0/1773959765 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff458085940 con 0x7ff45c06e4d0 2024-04-25T10:02:24.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff461a24640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff440066f60 0x7ff440069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.788+0000 7ff461a24640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff440066f60 0x7ff440069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff44c007fd0 tx=0x7ff44c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.792+0000 7ff452ffd640 1 -- 172.21.15.133:0/1773959765 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff458056780 con 0x7ff45c06e4d0 2024-04-25T10:02:24.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.952+0000 7ff464489640 1 -- 172.21.15.133:0/1773959765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff45c13ee20 con 0x7ff45c06e4d0 2024-04-25T10:02:24.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.952+0000 7ff452ffd640 1 -- 172.21.15.133:0/1773959765 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff458056080 con 0x7ff45c06e4d0 2024-04-25T10:02:24.960 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:24.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 -- 172.21.15.133:0/1773959765 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff440066f60 msgr2=0x7ff440069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff440066f60 0x7ff440069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff44c007fd0 tx=0x7ff44c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:24.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 -- 172.21.15.133:0/1773959765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff45c06e4d0 msgr2=0x7ff45c13abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff45c06e4d0 0x7ff45c13abe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff45800ea00 tx=0x7ff45800eec0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 -- 172.21.15.133:0/1773959765 shutdown_connections 2024-04-25T10:02:24.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff45c15ced0 0x7ff45c133900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff45c145170 0x7ff45c13cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff440066f60 0x7ff440069420 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 --2- 172.21.15.133:0/1773959765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff45c06e4d0 0x7ff45c13abe0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 -- 172.21.15.133:0/1773959765 >> 172.21.15.133:0/1773959765 conn(0x7ff45c066f70 msgr2=0x7ff45c160670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 -- 172.21.15.133:0/1773959765 shutdown_connections 2024-04-25T10:02:24.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:24.956+0000 7ff464489640 1 -- 172.21.15.133:0/1773959765 wait complete. 2024-04-25T10:02:24.979 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:25.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a31af1640 1 Processor -- start 2024-04-25T10:02:25.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a31af1640 1 -- start start 2024-04-25T10:02:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a31af1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5a2c15d5e0 0x7f5a2c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a31af1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2c0ce2e0 con 0x7f5a2c145170 2024-04-25T10:02:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a31af1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2c06d4b0 con 0x7f5a2c0cd820 2024-04-25T10:02:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a31af1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2c06db00 con 0x7f5a2c15d5e0 2024-04-25T10:02:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a2b7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5a2c145170 0x7f5a2c06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52142/0 (socket says 172.21.15.133:52142) 2024-04-25T10:02:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a2b7fe640 1 -- 172.21.15.133:0/168794177 learned_addr learned my addr 172.21.15.133:0/168794177 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a2bfff640 1 --2- 172.21.15.133:0/168794177 >> v2:172.21.15.133:3300/0 conn(0x7f5a2c15d5e0 0x7f5a2c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a2bfff640 1 -- 172.21.15.133:0/168794177 >> v1:172.21.15.162:6789/0 conn(0x7f5a2c0cd820 legacy=0x7f5a2c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a2bfff640 1 -- 172.21.15.133:0/168794177 >> v1:172.21.15.133:6789/0 conn(0x7f5a2c145170 legacy=0x7f5a2c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.092+0000 7f5a2bfff640 1 -- 172.21.15.133:0/168794177 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c06d790 con 0x7f5a2c15d5e0 2024-04-25T10:02:25.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2bfff640 1 --2- 172.21.15.133:0/168794177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c15d5e0 0x7f5a2c0cdca0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009d90 tx=0x7f5a2401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cb92fd2d13b408 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2affd640 1 -- 172.21.15.133:0/168794177 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a240b2070 con 0x7f5a2c15d5e0 2024-04-25T10:02:25.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2affd640 1 -- 172.21.15.133:0/168794177 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a240a5830 con 0x7f5a2c15d5e0 2024-04-25T10:02:25.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2affd640 1 -- 172.21.15.133:0/168794177 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a240a5c00 con 0x7f5a2c15d5e0 2024-04-25T10:02:25.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 -- 172.21.15.133:0/168794177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c15d5e0 msgr2=0x7f5a2c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 --2- 172.21.15.133:0/168794177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c15d5e0 0x7f5a2c0cdca0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009d90 tx=0x7f5a2401c970 comp rx=0 tx=0).stop 2024-04-25T10:02:25.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 -- 172.21.15.133:0/168794177 shutdown_connections 2024-04-25T10:02:25.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 --2- 172.21.15.133:0/168794177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c15d5e0 0x7f5a2c0cdca0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 -- 172.21.15.133:0/168794177 >> 172.21.15.133:0/168794177 conn(0x7f5a2c066f70 msgr2=0x7f5a2c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 -- 172.21.15.133:0/168794177 shutdown_connections 2024-04-25T10:02:25.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 -- 172.21.15.133:0/168794177 wait complete. 2024-04-25T10:02:25.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 Processor -- start 2024-04-25T10:02:25.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 -- start start 2024-04-25T10:02:25.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c0cd820 0x7f5a2c141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a2c145170 0x7f5a2c0b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a2c0b3ae0 0x7f5a2c139820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c1696b0 con 0x7f5a2c0cd820 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c145550 con 0x7f5a2c145170 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a31af1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c169830 con 0x7f5a2c0b3ae0 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c0cd820 0x7f5a2c141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c0cd820 0x7f5a2c141300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48888/0 (socket says 172.21.15.133:48888) 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2b7fe640 1 -- 172.21.15.133:0/2020089016 learned_addr learned my addr 172.21.15.133:0/2020089016 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a22ffd640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a2c145170 0x7f5a2c0b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2bfff640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a2c0b3ae0 0x7f5a2c139820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2b7fe640 1 -- 172.21.15.133:0/2020089016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a2c0b3ae0 msgr2=0x7f5a2c139820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2b7fe640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a2c0b3ae0 0x7f5a2c139820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2b7fe640 1 -- 172.21.15.133:0/2020089016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a2c145170 msgr2=0x7f5a2c0b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.096+0000 7f5a2b7fe640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a2c145170 0x7f5a2c0b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a2b7fe640 1 -- 172.21.15.133:0/2020089016 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c13d640 con 0x7f5a2c0cd820 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a22ffd640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a2c145170 0x7f5a2c0b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a2bfff640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a2c0b3ae0 0x7f5a2c139820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a2b7fe640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c0cd820 0x7f5a2c141300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c500 tx=0x7f5a1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a28ff9640 1 -- 172.21.15.133:0/2020089016 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c017070 con 0x7f5a2c0cd820 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a28ff9640 1 -- 172.21.15.133:0/2020089016 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c005cf0 con 0x7f5a2c0cd820 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a31af1640 1 -- 172.21.15.133:0/2020089016 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2c13d8e0 con 0x7f5a2c0cd820 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a28ff9640 1 -- 172.21.15.133:0/2020089016 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0072f0 con 0x7f5a2c0cd820 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a31af1640 1 -- 172.21.15.133:0/2020089016 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c0b4450 con 0x7f5a2c0cd820 2024-04-25T10:02:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a31af1640 1 -- 172.21.15.133:0/2020089016 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2c068e80 con 0x7f5a2c0cd820 2024-04-25T10:02:25.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a28ff9640 1 -- 172.21.15.133:0/2020089016 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a1c01a070 con 0x7f5a2c0cd820 2024-04-25T10:02:25.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a28ff9640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a0c066f90 0x7f5a0c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a28ff9640 1 -- 172.21.15.133:0/2020089016 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5a1c0845d0 con 0x7f5a2c0cd820 2024-04-25T10:02:25.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a22ffd640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a0c066f90 0x7f5a0c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.100+0000 7f5a22ffd640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a0c066f90 0x7f5a0c069450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c15e330 tx=0x7f5a14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.104+0000 7f5a28ff9640 1 -- 172.21.15.133:0/2020089016 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a1c054ce0 con 0x7f5a2c0cd820 2024-04-25T10:02:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.264+0000 7f5a31af1640 1 -- 172.21.15.133:0/2020089016 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a2c0b4be0 con 0x7f5a2c0cd820 2024-04-25T10:02:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.264+0000 7f5a28ff9640 1 -- 172.21.15.133:0/2020089016 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a1c058d80 con 0x7f5a2c0cd820 2024-04-25T10:02:25.271 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:25.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 -- 172.21.15.133:0/2020089016 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a0c066f90 msgr2=0x7f5a0c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a0c066f90 0x7f5a0c069450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c15e330 tx=0x7f5a14009040 comp rx=0 tx=0).stop 2024-04-25T10:02:25.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 -- 172.21.15.133:0/2020089016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c0cd820 msgr2=0x7f5a2c141300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c0cd820 0x7f5a2c141300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c500 tx=0x7f5a1c00c9c0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 -- 172.21.15.133:0/2020089016 shutdown_connections 2024-04-25T10:02:25.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5a2c0b3ae0 0x7f5a2c139820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a2c145170 0x7f5a2c0b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a0c066f90 0x7f5a0c069450 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 --2- 172.21.15.133:0/2020089016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c0cd820 0x7f5a2c141300 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 -- 172.21.15.133:0/2020089016 >> 172.21.15.133:0/2020089016 conn(0x7f5a2c066f70 msgr2=0x7f5a2c15e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 -- 172.21.15.133:0/2020089016 shutdown_connections 2024-04-25T10:02:25.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.268+0000 7f5a31af1640 1 -- 172.21.15.133:0/2020089016 wait complete. 2024-04-25T10:02:25.291 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.400+0000 7fa527b00640 1 Processor -- start 2024-04-25T10:02:25.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.400+0000 7fa527b00640 1 -- start start 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.400+0000 7fa527b00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5200d21c0 0x7fa5200cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.400+0000 7fa527b00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa52006f8f0 0x7fa52006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.400+0000 7fa527b00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5200cefa0 con 0x7fa52015f020 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.400+0000 7fa527b00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5200cd690 con 0x7fa5200d21c0 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.400+0000 7fa527b00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200ccdf0 con 0x7fa52006f8f0 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.400+0000 7fa52589c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa52015f020 0x7fa5200cf270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52156/0 (socket says 172.21.15.133:52156) 2024-04-25T10:02:25.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.400+0000 7fa52589c640 1 -- 172.21.15.133:0/2648416842 learned_addr learned my addr 172.21.15.133:0/2648416842 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.400+0000 7fa52509b640 1 --2- 172.21.15.133:0/2648416842 >> v2:172.21.15.133:3300/0 conn(0x7fa5200d21c0 0x7fa5200cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.400+0000 7fa52609d640 1 --2- 172.21.15.133:0/2648416842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa52006f8f0 0x7fa52006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa52509b640 1 -- 172.21.15.133:0/2648416842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa52006f8f0 msgr2=0x7fa52006fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa52509b640 1 --2- 172.21.15.133:0/2648416842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa52006f8f0 0x7fa52006fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa52509b640 1 -- 172.21.15.133:0/2648416842 >> v1:172.21.15.133:6789/0 conn(0x7fa52015f020 legacy=0x7fa5200cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa52509b640 1 -- 172.21.15.133:0/2648416842 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200cd0d0 con 0x7fa5200d21c0 2024-04-25T10:02:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa52489a640 1 -- 172.21.15.133:0/2648416842 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348094461 0 0) 0x7fa5180014a0 con 0x7fa52015f020 2024-04-25T10:02:25.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa52509b640 1 --2- 172.21.15.133:0/2648416842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5200d21c0 0x7fa5200cf980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa5100097c0 tx=0x7fa51001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a99c2521ee322c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa52489a640 1 -- 172.21.15.133:0/2648416842 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100b1070 con 0x7fa5200d21c0 2024-04-25T10:02:25.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa52489a640 1 -- 172.21.15.133:0/2648416842 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5100ac410 con 0x7fa5200d21c0 2024-04-25T10:02:25.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa52489a640 1 -- 172.21.15.133:0/2648416842 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100ac7e0 con 0x7fa5200d21c0 2024-04-25T10:02:25.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa527b00640 1 -- 172.21.15.133:0/2648416842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5200d21c0 msgr2=0x7fa5200cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa527b00640 1 --2- 172.21.15.133:0/2648416842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5200d21c0 0x7fa5200cf980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa5100097c0 tx=0x7fa51001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa527b00640 1 -- 172.21.15.133:0/2648416842 shutdown_connections 2024-04-25T10:02:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa527b00640 1 --2- 172.21.15.133:0/2648416842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa52006f8f0 0x7fa52006fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa527b00640 1 --2- 172.21.15.133:0/2648416842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5200d21c0 0x7fa5200cf980 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa527b00640 1 -- 172.21.15.133:0/2648416842 >> 172.21.15.133:0/2648416842 conn(0x7fa5200d16a0 msgr2=0x7fa5200d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa527b00640 1 -- 172.21.15.133:0/2648416842 shutdown_connections 2024-04-25T10:02:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.404+0000 7fa527b00640 1 -- 172.21.15.133:0/2648416842 wait complete. 2024-04-25T10:02:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa527b00640 1 Processor -- start 2024-04-25T10:02:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa527b00640 1 -- start start 2024-04-25T10:02:25.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa527b00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa52006f8f0 0x7fa5200c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa527b00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5200d21c0 0x7fa5201512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa527b00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa52015f020 0x7fa5200c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa527b00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5201686e0 con 0x7fa52006f8f0 2024-04-25T10:02:25.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa527b00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa520168560 con 0x7fa52015f020 2024-04-25T10:02:25.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa527b00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa520168860 con 0x7fa5200d21c0 2024-04-25T10:02:25.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa52589c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa52006f8f0 0x7fa5200c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa52589c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa52006f8f0 0x7fa5200c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48902/0 (socket says 172.21.15.133:48902) 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa52589c640 1 -- 172.21.15.133:0/1347949500 learned_addr learned my addr 172.21.15.133:0/1347949500 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa52509b640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5200d21c0 0x7fa5201512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa52609d640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa52015f020 0x7fa5200c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa52589c640 1 -- 172.21.15.133:0/1347949500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5200d21c0 msgr2=0x7fa5201512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa52589c640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5200d21c0 0x7fa5201512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa52589c640 1 -- 172.21.15.133:0/1347949500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa52015f020 msgr2=0x7fa5200c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa52589c640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa52015f020 0x7fa5200c51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa52589c640 1 -- 172.21.15.133:0/1347949500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200c70f0 con 0x7fa52006f8f0 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa52589c640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa52006f8f0 0x7fa5200c7c50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa51800c8f0 tx=0x7fa51800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa5167fc640 1 -- 172.21.15.133:0/1347949500 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518018070 con 0x7fa52006f8f0 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa5167fc640 1 -- 172.21.15.133:0/1347949500 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51800a5e0 con 0x7fa52006f8f0 2024-04-25T10:02:25.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa527b00640 1 -- 172.21.15.133:0/1347949500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5200c73d0 con 0x7fa52006f8f0 2024-04-25T10:02:25.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa5167fc640 1 -- 172.21.15.133:0/1347949500 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51800aa10 con 0x7fa52006f8f0 2024-04-25T10:02:25.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa527b00640 1 -- 172.21.15.133:0/1347949500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5200c6dc0 con 0x7fa52006f8f0 2024-04-25T10:02:25.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa527b00640 1 -- 172.21.15.133:0/1347949500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa520067610 con 0x7fa52006f8f0 2024-04-25T10:02:25.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.408+0000 7fa5167fc640 1 -- 172.21.15.133:0/1347949500 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa51801b070 con 0x7fa52006f8f0 2024-04-25T10:02:25.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.412+0000 7fa5167fc640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa504066f90 0x7fa504069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.412+0000 7fa52509b640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa504066f90 0x7fa504069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.412+0000 7fa5167fc640 1 -- 172.21.15.133:0/1347949500 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa5180849a0 con 0x7fa52006f8f0 2024-04-25T10:02:25.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.412+0000 7fa52509b640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa504066f90 0x7fa504069450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa5100ae000 tx=0x7fa510008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.416+0000 7fa5167fc640 1 -- 172.21.15.133:0/1347949500 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5200cefa0 con 0x7fa52006f8f0 2024-04-25T10:02:25.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.568+0000 7fa527b00640 1 -- 172.21.15.133:0/1347949500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa520067c70 con 0x7fa52006f8f0 2024-04-25T10:02:25.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.568+0000 7fa5167fc640 1 -- 172.21.15.133:0/1347949500 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa518055700 con 0x7fa52006f8f0 2024-04-25T10:02:25.576 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:25.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 -- 172.21.15.133:0/1347949500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa504066f90 msgr2=0x7fa504069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa504066f90 0x7fa504069450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa5100ae000 tx=0x7fa510008040 comp rx=0 tx=0).stop 2024-04-25T10:02:25.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 -- 172.21.15.133:0/1347949500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa52006f8f0 msgr2=0x7fa5200c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa52006f8f0 0x7fa5200c7c50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa51800c8f0 tx=0x7fa51800cdb0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 -- 172.21.15.133:0/1347949500 shutdown_connections 2024-04-25T10:02:25.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa52015f020 0x7fa5200c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5200d21c0 0x7fa5201512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa504066f90 0x7fa504069450 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 --2- 172.21.15.133:0/1347949500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa52006f8f0 0x7fa5200c7c50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 -- 172.21.15.133:0/1347949500 >> 172.21.15.133:0/1347949500 conn(0x7fa5200d16a0 msgr2=0x7fa5200cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 -- 172.21.15.133:0/1347949500 shutdown_connections 2024-04-25T10:02:25.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.572+0000 7fa527b00640 1 -- 172.21.15.133:0/1347949500 wait complete. 2024-04-25T10:02:25.594 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f111cf84640 1 Processor -- start 2024-04-25T10:02:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f111cf84640 1 -- start start 2024-04-25T10:02:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f111cf84640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f11180b2150 0x7f111806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f111cf84640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f111815b9b0 0x7f111806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f111cf84640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111815ac50 con 0x7f1118143ea0 2024-04-25T10:02:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f111cf84640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11180ccf20 con 0x7f11180b2150 2024-04-25T10:02:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f111cf84640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11180cd180 con 0x7f111815b9b0 2024-04-25T10:02:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f1115d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f11180b2150 0x7f111806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f1115d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f11180b2150 0x7f111806e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48904/0 (socket says 172.21.15.133:48904) 2024-04-25T10:02:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f1115d9b640 1 -- 172.21.15.133:0/1634019887 learned_addr learned my addr 172.21.15.133:0/1634019887 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f1116d9d640 1 --2- 172.21.15.133:0/1634019887 >> v2:172.21.15.162:3300/0 conn(0x7f111815b9b0 0x7f111806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f1115d9b640 1 -- 172.21.15.133:0/1634019887 >> v2:172.21.15.162:3300/0 conn(0x7f111815b9b0 msgr2=0x7f111806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f1115d9b640 1 --2- 172.21.15.133:0/1634019887 >> v2:172.21.15.162:3300/0 conn(0x7f111815b9b0 0x7f111806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f1115d9b640 1 -- 172.21.15.133:0/1634019887 >> v1:172.21.15.162:6789/0 conn(0x7f1118143ea0 legacy=0x7f11180cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.708+0000 7f1115d9b640 1 -- 172.21.15.133:0/1634019887 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111815aed0 con 0x7f11180b2150 2024-04-25T10:02:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f1115d9b640 1 --2- 172.21.15.133:0/1634019887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11180b2150 0x7f111806e3b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f11180cd7f0 tx=0x7f110c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689ec06efa5e20a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111559a640 1 -- 172.21.15.133:0/1634019887 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c0b0070 con 0x7f11180b2150 2024-04-25T10:02:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111559a640 1 -- 172.21.15.133:0/1634019887 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c0ab600 con 0x7f11180b2150 2024-04-25T10:02:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111559a640 1 -- 172.21.15.133:0/1634019887 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c0ab9d0 con 0x7f11180b2150 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 -- 172.21.15.133:0/1634019887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11180b2150 msgr2=0x7f111806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 --2- 172.21.15.133:0/1634019887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11180b2150 0x7f111806e3b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f11180cd7f0 tx=0x7f110c01c490 comp rx=0 tx=0).stop 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 -- 172.21.15.133:0/1634019887 shutdown_connections 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 --2- 172.21.15.133:0/1634019887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11180b2150 0x7f111806e3b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 --2- 172.21.15.133:0/1634019887 >> v2:172.21.15.162:3300/0 conn(0x7f111815b9b0 0x7f111806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 -- 172.21.15.133:0/1634019887 >> 172.21.15.133:0/1634019887 conn(0x7f1118066f70 msgr2=0x7f111806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 -- 172.21.15.133:0/1634019887 shutdown_connections 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 -- 172.21.15.133:0/1634019887 wait complete. 2024-04-25T10:02:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 Processor -- start 2024-04-25T10:02:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 -- start start 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11180b2150 0x7f111813b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1118143ea0 0x7f1118152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f111815b9b0 0x7f11181429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11180b1270 con 0x7f111815b9b0 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11180b10f0 con 0x7f1118143ea0 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.712+0000 7f111cf84640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11180b13f0 con 0x7f11180b2150 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1116d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f111815b9b0 0x7f11181429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1116d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f111815b9b0 0x7f11181429e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48912/0 (socket says 172.21.15.133:48912) 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1116d9d640 1 -- 172.21.15.133:0/1885637192 learned_addr learned my addr 172.21.15.133:0/1885637192 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1115d9b640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1118143ea0 0x7f1118152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f111659c640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11180b2150 0x7f111813b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1116d9d640 1 -- 172.21.15.133:0/1885637192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11180b2150 msgr2=0x7f111813b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1116d9d640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11180b2150 0x7f111813b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1116d9d640 1 -- 172.21.15.133:0/1885637192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1118143ea0 msgr2=0x7f1118152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1116d9d640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1118143ea0 0x7f1118152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1116d9d640 1 -- 172.21.15.133:0/1885637192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1118152b80 con 0x7f111815b9b0 2024-04-25T10:02:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1116d9d640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f111815b9b0 0x7f11181429e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f111000efc0 tx=0x7f111000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1106ffd640 1 -- 172.21.15.133:0/1885637192 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110018020 con 0x7f111815b9b0 2024-04-25T10:02:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1106ffd640 1 -- 172.21.15.133:0/1885637192 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1110014480 con 0x7f111815b9b0 2024-04-25T10:02:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1106ffd640 1 -- 172.21.15.133:0/1885637192 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110009e60 con 0x7f111815b9b0 2024-04-25T10:02:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f111cf84640 1 -- 172.21.15.133:0/1885637192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1118152e60 con 0x7f111815b9b0 2024-04-25T10:02:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f111cf84640 1 -- 172.21.15.133:0/1885637192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1118153420 con 0x7f111815b9b0 2024-04-25T10:02:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f111cf84640 1 -- 172.21.15.133:0/1885637192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1118068e80 con 0x7f111815b9b0 2024-04-25T10:02:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1106ffd640 1 -- 172.21.15.133:0/1885637192 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f111001b020 con 0x7f111815b9b0 2024-04-25T10:02:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1106ffd640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f8066f60 0x7f10f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f1106ffd640 1 -- 172.21.15.133:0/1885637192 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1110086030 con 0x7f111815b9b0 2024-04-25T10:02:25.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f111659c640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f8066f60 0x7f10f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.716+0000 7f111659c640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f8066f60 0x7f10f8069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f111806ed40 tx=0x7f1108009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.720+0000 7f1106ffd640 1 -- 172.21.15.133:0/1885637192 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1110022030 con 0x7f111815b9b0 2024-04-25T10:02:25.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.876+0000 7f111cf84640 1 -- 172.21.15.133:0/1885637192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f11180b04b0 con 0x7f111815b9b0 2024-04-25T10:02:25.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f1106ffd640 1 -- 172.21.15.133:0/1885637192 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1110010040 con 0x7f111815b9b0 2024-04-25T10:02:25.885 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 -- 172.21.15.133:0/1885637192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f8066f60 msgr2=0x7f10f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f8066f60 0x7f10f8069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f111806ed40 tx=0x7f1108009040 comp rx=0 tx=0).stop 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 -- 172.21.15.133:0/1885637192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f111815b9b0 msgr2=0x7f11181429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f111815b9b0 0x7f11181429e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f111000efc0 tx=0x7f111000c490 comp rx=0 tx=0).stop 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 -- 172.21.15.133:0/1885637192 shutdown_connections 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10f8066f60 0x7f10f8069420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f111815b9b0 0x7f11181429e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1118143ea0 0x7f1118152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 --2- 172.21.15.133:0/1885637192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11180b2150 0x7f111813b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 -- 172.21.15.133:0/1885637192 >> 172.21.15.133:0/1885637192 conn(0x7f1118066f70 msgr2=0x7f111815c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 -- 172.21.15.133:0/1885637192 shutdown_connections 2024-04-25T10:02:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:25.880+0000 7f111cf84640 1 -- 172.21.15.133:0/1885637192 wait complete. 2024-04-25T10:02:25.904 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:26.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.016+0000 7fd8a3159640 1 Processor -- start 2024-04-25T10:02:26.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.016+0000 7fd8a3159640 1 -- start start 2024-04-25T10:02:26.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a3159640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd89c15b9c0 0x7fd89c072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a3159640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd89c0704a0 0x7fd89c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a3159640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89c15bda0 con 0x7fd89c0ccf90 2024-04-25T10:02:26.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a3159640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c0cd450 con 0x7fd89c0704a0 2024-04-25T10:02:26.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a3159640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c0cd730 con 0x7fd89c15b9c0 2024-04-25T10:02:26.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a16f6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd89c0704a0 0x7fd89c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a16f6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd89c0704a0 0x7fd89c0cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48918/0 (socket says 172.21.15.133:48918) 2024-04-25T10:02:26.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a16f6640 1 -- 172.21.15.133:0/3642001329 learned_addr learned my addr 172.21.15.133:0/3642001329 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:26.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a0ef5640 1 --2- 172.21.15.133:0/3642001329 >> v2:172.21.15.162:3300/0 conn(0x7fd89c15b9c0 0x7fd89c072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a16f6640 1 -- 172.21.15.133:0/3642001329 >> v2:172.21.15.162:3300/0 conn(0x7fd89c15b9c0 msgr2=0x7fd89c072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a16f6640 1 --2- 172.21.15.133:0/3642001329 >> v2:172.21.15.162:3300/0 conn(0x7fd89c15b9c0 0x7fd89c072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a16f6640 1 -- 172.21.15.133:0/3642001329 >> v1:172.21.15.133:6789/0 conn(0x7fd89c0ccf90 legacy=0x7fd89c06fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a16f6640 1 -- 172.21.15.133:0/3642001329 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c0cda10 con 0x7fd89c0704a0 2024-04-25T10:02:26.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a0ef5640 1 --2- 172.21.15.133:0/3642001329 >> v2:172.21.15.162:3300/0 conn(0x7fd89c15b9c0 0x7fd89c072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:26.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd8a16f6640 1 --2- 172.21.15.133:0/3642001329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd89c0704a0 0x7fd89c0cf950 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0d16a0 tx=0x7fd89401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84eee085a9f38afd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd89b7fe640 1 -- 172.21.15.133:0/3642001329 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8940a5730 con 0x7fd89c0704a0 2024-04-25T10:02:26.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd89b7fe640 1 -- 172.21.15.133:0/3642001329 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8940a58d0 con 0x7fd89c0704a0 2024-04-25T10:02:26.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.020+0000 7fd89b7fe640 1 -- 172.21.15.133:0/3642001329 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8940a5ca0 con 0x7fd89c0704a0 2024-04-25T10:02:26.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 -- 172.21.15.133:0/3642001329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd89c0704a0 msgr2=0x7fd89c0cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 --2- 172.21.15.133:0/3642001329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd89c0704a0 0x7fd89c0cf950 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0d16a0 tx=0x7fd89401c960 comp rx=0 tx=0).stop 2024-04-25T10:02:26.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 -- 172.21.15.133:0/3642001329 shutdown_connections 2024-04-25T10:02:26.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 --2- 172.21.15.133:0/3642001329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd89c0704a0 0x7fd89c0cf950 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 --2- 172.21.15.133:0/3642001329 >> v2:172.21.15.162:3300/0 conn(0x7fd89c15b9c0 0x7fd89c072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 -- 172.21.15.133:0/3642001329 >> 172.21.15.133:0/3642001329 conn(0x7fd89c0b2370 msgr2=0x7fd89c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 -- 172.21.15.133:0/3642001329 shutdown_connections 2024-04-25T10:02:26.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 -- 172.21.15.133:0/3642001329 wait complete. 2024-04-25T10:02:26.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 Processor -- start 2024-04-25T10:02:26.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 -- start start 2024-04-25T10:02:26.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd89c0704a0 0x7fd89c13f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd89c0ccf90 0x7fd89c14eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd89c15b9c0 0x7fd89c1505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0d0a30 con 0x7fd89c0ccf90 2024-04-25T10:02:26.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0d08b0 con 0x7fd89c0704a0 2024-04-25T10:02:26.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0d0bb0 con 0x7fd89c15b9c0 2024-04-25T10:02:26.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd89bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd89c0ccf90 0x7fd89c14eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd89bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd89c0ccf90 0x7fd89c14eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48934/0 (socket says 172.21.15.133:48934) 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd89bfff640 1 -- 172.21.15.133:0/739908954 learned_addr learned my addr 172.21.15.133:0/739908954 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a0ef5640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd89c0704a0 0x7fd89c13f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a16f6640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd89c15b9c0 0x7fd89c1505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd89bfff640 1 -- 172.21.15.133:0/739908954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd89c15b9c0 msgr2=0x7fd89c1505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd89bfff640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd89c15b9c0 0x7fd89c1505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd89bfff640 1 -- 172.21.15.133:0/739908954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd89c0704a0 msgr2=0x7fd89c13f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd89bfff640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd89c0704a0 0x7fd89c13f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd89bfff640 1 -- 172.21.15.133:0/739908954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c15e870 con 0x7fd89c0ccf90 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a16f6640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd89c15b9c0 0x7fd89c1505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a0ef5640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd89c0704a0 0x7fd89c13f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd89bfff640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd89c0ccf90 0x7fd89c14eb40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00c390 tx=0x7fd88c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8997fa640 1 -- 172.21.15.133:0/739908954 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c017070 con 0x7fd89c0ccf90 2024-04-25T10:02:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8997fa640 1 -- 172.21.15.133:0/739908954 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c0052f0 con 0x7fd89c0ccf90 2024-04-25T10:02:26.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8997fa640 1 -- 172.21.15.133:0/739908954 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c003840 con 0x7fd89c0ccf90 2024-04-25T10:02:26.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 -- 172.21.15.133:0/739908954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c15eb50 con 0x7fd89c0ccf90 2024-04-25T10:02:26.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.024+0000 7fd8a3159640 1 -- 172.21.15.133:0/739908954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c14f080 con 0x7fd89c0ccf90 2024-04-25T10:02:26.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.028+0000 7fd8a3159640 1 -- 172.21.15.133:0/739908954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c067c70 con 0x7fd89c0ccf90 2024-04-25T10:02:26.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.028+0000 7fd8997fa640 1 -- 172.21.15.133:0/739908954 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd88c01a070 con 0x7fd89c0ccf90 2024-04-25T10:02:26.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.028+0000 7fd8997fa640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd888066f90 0x7fd888069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.028+0000 7fd8997fa640 1 -- 172.21.15.133:0/739908954 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd88c084620 con 0x7fd89c0ccf90 2024-04-25T10:02:26.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.028+0000 7fd8a0ef5640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd888066f90 0x7fd888069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.028+0000 7fd8a0ef5640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd888066f90 0x7fd888069450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd890008930 tx=0x7fd890008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.032+0000 7fd8997fa640 1 -- 172.21.15.133:0/739908954 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd88c0552f0 con 0x7fd89c0ccf90 2024-04-25T10:02:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 -- 172.21.15.133:0/739908954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd89c070ba0 con 0x7fd89c0ccf90 2024-04-25T10:02:26.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8997fa640 1 -- 172.21.15.133:0/739908954 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7fd88c054bf0 con 0x7fd89c0ccf90 2024-04-25T10:02:26.226 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:26.227 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2230831968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2230831968},{"type":"v1","addr":"172.21.15.162:6835","nonce":2230831968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1456396097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1456396097},{"type":"v1","addr":"172.21.15.133:6836","nonce":1456396097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:51.276631+0000","modified":"2024-04-25T10:02:14.440426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2462154167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2462154167},{"type":"v1","addr":"172.21.15.133:6837","nonce":2462154167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:26.227 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 9 2024-04-25T10:02:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 -- 172.21.15.133:0/739908954 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd888066f90 msgr2=0x7fd888069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd888066f90 0x7fd888069450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd890008930 tx=0x7fd890008040 comp rx=0 tx=0).stop 2024-04-25T10:02:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 -- 172.21.15.133:0/739908954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd89c0ccf90 msgr2=0x7fd89c14eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd89c0ccf90 0x7fd89c14eb40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00c390 tx=0x7fd88c00c850 comp rx=0 tx=0).stop 2024-04-25T10:02:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 -- 172.21.15.133:0/739908954 shutdown_connections 2024-04-25T10:02:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd89c15b9c0 0x7fd89c1505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd888066f90 0x7fd888069450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd89c0ccf90 0x7fd89c14eb40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 --2- 172.21.15.133:0/739908954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd89c0704a0 0x7fd89c13f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 -- 172.21.15.133:0/739908954 >> 172.21.15.133:0/739908954 conn(0x7fd89c0b2370 msgr2=0x7fd89c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 -- 172.21.15.133:0/739908954 shutdown_connections 2024-04-25T10:02:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.220+0000 7fd8a3159640 1 -- 172.21.15.133:0/739908954 wait complete. 2024-04-25T10:02:26.244 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:26.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.356+0000 7f7955acd640 1 Processor -- start 2024-04-25T10:02:26.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.356+0000 7f7955acd640 1 -- start start 2024-04-25T10:02:26.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f7955acd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f795015c9a0 0x7f79500c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f7955acd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79500ce330 0x7f79500ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f7955acd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79500c61d0 con 0x7f79500b1f70 2024-04-25T10:02:26.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f7955acd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79500cecc0 con 0x7f795015c9a0 2024-04-25T10:02:26.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f7955acd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79500c64b0 con 0x7f79500ce330 2024-04-25T10:02:26.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794effd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f79500b1f70 0x7f795006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33996/0 (socket says 172.21.15.133:33996) 2024-04-25T10:02:26.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794effd640 1 -- 172.21.15.133:0/3231509005 learned_addr learned my addr 172.21.15.133:0/3231509005 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:26.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794f7fe640 1 --2- 172.21.15.133:0/3231509005 >> v2:172.21.15.162:3300/0 conn(0x7f795015c9a0 0x7f79500c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794ffff640 1 --2- 172.21.15.133:0/3231509005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79500ce330 0x7f79500ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794e7fc640 1 -- 172.21.15.133:0/3231509005 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226666724 0 0) 0x7f79500c61d0 con 0x7f79500b1f70 2024-04-25T10:02:26.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794e7fc640 1 -- 172.21.15.133:0/3231509005 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7938003530 con 0x7f79500b1f70 2024-04-25T10:02:26.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794f7fe640 1 -- 172.21.15.133:0/3231509005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79500ce330 msgr2=0x7f79500ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794f7fe640 1 --2- 172.21.15.133:0/3231509005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79500ce330 0x7f79500ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794f7fe640 1 -- 172.21.15.133:0/3231509005 >> v1:172.21.15.133:6789/0 conn(0x7f79500b1f70 legacy=0x7f795006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794e7fc640 1 -- 172.21.15.133:0/3231509005 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1931696112 0 0) 0x7f7938003530 con 0x7f79500b1f70 2024-04-25T10:02:26.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794f7fe640 1 -- 172.21.15.133:0/3231509005 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795006d4f0 con 0x7f795015c9a0 2024-04-25T10:02:26.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.360+0000 7f794f7fe640 1 --2- 172.21.15.133:0/3231509005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f795015c9a0 0x7f79500c5980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79400097c0 tx=0x7f794001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca1b1bc44b0eb274 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794e7fc640 1 -- 172.21.15.133:0/3231509005 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79400b0070 con 0x7f795015c9a0 2024-04-25T10:02:26.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794e7fc640 1 -- 172.21.15.133:0/3231509005 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79400ab400 con 0x7f795015c9a0 2024-04-25T10:02:26.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794e7fc640 1 -- 172.21.15.133:0/3231509005 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944001680 con 0x7f795015c9a0 2024-04-25T10:02:26.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 -- 172.21.15.133:0/3231509005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f795015c9a0 msgr2=0x7f79500c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 --2- 172.21.15.133:0/3231509005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f795015c9a0 0x7f79500c5980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79400097c0 tx=0x7f794001c3c0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 -- 172.21.15.133:0/3231509005 shutdown_connections 2024-04-25T10:02:26.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 --2- 172.21.15.133:0/3231509005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79500ce330 0x7f79500ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 --2- 172.21.15.133:0/3231509005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f795015c9a0 0x7f79500c5980 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 -- 172.21.15.133:0/3231509005 >> 172.21.15.133:0/3231509005 conn(0x7f7950067000 msgr2=0x7f795006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 -- 172.21.15.133:0/3231509005 shutdown_connections 2024-04-25T10:02:26.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 -- 172.21.15.133:0/3231509005 wait complete. 2024-04-25T10:02:26.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 Processor -- start 2024-04-25T10:02:26.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 -- start start 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79500b1f70 0x7f795013e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79500ce330 0x7f795015f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f795015c9a0 0x7f795015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795015dad0 con 0x7f795015c9a0 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795015d950 con 0x7f79500ce330 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795015dc50 con 0x7f79500b1f70 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f795015c9a0 0x7f795015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f795015c9a0 0x7f795015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50248/0 (socket says 172.21.15.133:50248) 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794ffff640 1 -- 172.21.15.133:0/3934001880 learned_addr learned my addr 172.21.15.133:0/3934001880 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794f7fe640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79500b1f70 0x7f795013e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794effd640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79500ce330 0x7f795015f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794ffff640 1 -- 172.21.15.133:0/3934001880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79500b1f70 msgr2=0x7f795013e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794ffff640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79500b1f70 0x7f795013e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794ffff640 1 -- 172.21.15.133:0/3934001880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79500ce330 msgr2=0x7f795015f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794ffff640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79500ce330 0x7f795015f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794ffff640 1 -- 172.21.15.133:0/3934001880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795015a660 con 0x7f795015c9a0 2024-04-25T10:02:26.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f794ffff640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f795015c9a0 0x7f795015de00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f794800efc0 tx=0x7f794800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7937fff640 1 -- 172.21.15.133:0/3934001880 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948018020 con 0x7f795015c9a0 2024-04-25T10:02:26.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7937fff640 1 -- 172.21.15.133:0/3934001880 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7948014480 con 0x7f795015c9a0 2024-04-25T10:02:26.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 -- 172.21.15.133:0/3934001880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795015a940 con 0x7f795015c9a0 2024-04-25T10:02:26.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7937fff640 1 -- 172.21.15.133:0/3934001880 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948014870 con 0x7f795015c9a0 2024-04-25T10:02:26.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.364+0000 7f7955acd640 1 -- 172.21.15.133:0/3934001880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795015af00 con 0x7f795015c9a0 2024-04-25T10:02:26.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.368+0000 7f7955acd640 1 -- 172.21.15.133:0/3934001880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7950068ef0 con 0x7f795015c9a0 2024-04-25T10:02:26.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.368+0000 7f7937fff640 1 -- 172.21.15.133:0/3934001880 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f794801b020 con 0x7f795015c9a0 2024-04-25T10:02:26.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.368+0000 7f7937fff640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f793c066f60 0x7f793c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.368+0000 7f7937fff640 1 -- 172.21.15.133:0/3934001880 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7948085d80 con 0x7f795015c9a0 2024-04-25T10:02:26.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.368+0000 7f794f7fe640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f793c066f60 0x7f793c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.368+0000 7f794f7fe640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f793c066f60 0x7f793c069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f79400ad000 tx=0x7f79400a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.372+0000 7f7937fff640 1 -- 172.21.15.133:0/3934001880 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7948022070 con 0x7f795015c9a0 2024-04-25T10:02:26.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.516+0000 7f7955acd640 1 -- 172.21.15.133:0/3934001880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79501591d0 con 0x7f795015c9a0 2024-04-25T10:02:26.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.520+0000 7f7937fff640 1 -- 172.21.15.133:0/3934001880 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18538 (secure 0 0 0) 0x7f7948010040 con 0x7f795015c9a0 2024-04-25T10:02:26.528 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:26.528 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:01:56.349210+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:26.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 -- 172.21.15.133:0/3934001880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f793c066f60 msgr2=0x7f793c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f793c066f60 0x7f793c069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f79400ad000 tx=0x7f79400a6000 comp rx=0 tx=0).stop 2024-04-25T10:02:26.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 -- 172.21.15.133:0/3934001880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f795015c9a0 msgr2=0x7f795015de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f795015c9a0 0x7f795015de00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f794800efc0 tx=0x7f794800c490 comp rx=0 tx=0).stop 2024-04-25T10:02:26.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 -- 172.21.15.133:0/3934001880 shutdown_connections 2024-04-25T10:02:26.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f793c066f60 0x7f793c069420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f795015c9a0 0x7f795015de00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79500ce330 0x7f795015f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 --2- 172.21.15.133:0/3934001880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79500b1f70 0x7f795013e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 -- 172.21.15.133:0/3934001880 >> 172.21.15.133:0/3934001880 conn(0x7f7950067000 msgr2=0x7f79500cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 -- 172.21.15.133:0/3934001880 shutdown_connections 2024-04-25T10:02:26.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.524+0000 7f7955acd640 1 -- 172.21.15.133:0/3934001880 wait complete. 2024-04-25T10:02:26.546 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:26.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c403c640 1 Processor -- start 2024-04-25T10:02:26.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c403c640 1 -- start start 2024-04-25T10:02:26.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c403c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86bc15db00 0x7f86bc0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c403c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f86bc0d0770 0x7f86bc0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c403c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc0ceb20 con 0x7f86bc1614c0 2024-04-25T10:02:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c403c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86bc0d1110 con 0x7f86bc15db00 2024-04-25T10:02:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c403c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86bc0b7740 con 0x7f86bc0d0770 2024-04-25T10:02:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c1dd8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86bc15db00 0x7f86bc0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c1dd8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86bc15db00 0x7f86bc0b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50262/0 (socket says 172.21.15.133:50262) 2024-04-25T10:02:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c1dd8640 1 -- 172.21.15.133:0/2186892158 learned_addr learned my addr 172.21.15.133:0/2186892158 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c15d7640 1 --2- 172.21.15.133:0/2186892158 >> v2:172.21.15.162:3300/0 conn(0x7f86bc0d0770 0x7f86bc0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c1dd8640 1 -- 172.21.15.133:0/2186892158 >> v2:172.21.15.162:3300/0 conn(0x7f86bc0d0770 msgr2=0x7f86bc0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c1dd8640 1 --2- 172.21.15.133:0/2186892158 >> v2:172.21.15.162:3300/0 conn(0x7f86bc0d0770 0x7f86bc0d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c1dd8640 1 -- 172.21.15.133:0/2186892158 >> v1:172.21.15.133:6789/0 conn(0x7f86bc1614c0 legacy=0x7f86bc161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.648+0000 7f86c1dd8640 1 -- 172.21.15.133:0/2186892158 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc0b97f0 con 0x7f86bc15db00 2024-04-25T10:02:26.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c1dd8640 1 --2- 172.21.15.133:0/2186892158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86bc15db00 0x7f86bc0b2990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0d0200 tx=0x7f86b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9762330102a874d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c0dd6640 1 -- 172.21.15.133:0/2186892158 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b80aa650 con 0x7f86bc15db00 2024-04-25T10:02:26.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c0dd6640 1 -- 172.21.15.133:0/2186892158 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b80af070 con 0x7f86bc15db00 2024-04-25T10:02:26.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c0dd6640 1 -- 172.21.15.133:0/2186892158 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b80aaa10 con 0x7f86bc15db00 2024-04-25T10:02:26.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 -- 172.21.15.133:0/2186892158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86bc15db00 msgr2=0x7f86bc0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 --2- 172.21.15.133:0/2186892158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86bc15db00 0x7f86bc0b2990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0d0200 tx=0x7f86b801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 -- 172.21.15.133:0/2186892158 shutdown_connections 2024-04-25T10:02:26.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 --2- 172.21.15.133:0/2186892158 >> v2:172.21.15.162:3300/0 conn(0x7f86bc0d0770 0x7f86bc0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 --2- 172.21.15.133:0/2186892158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86bc15db00 0x7f86bc0b2990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 -- 172.21.15.133:0/2186892158 >> 172.21.15.133:0/2186892158 conn(0x7f86bc0727c0 msgr2=0x7f86bc072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 -- 172.21.15.133:0/2186892158 shutdown_connections 2024-04-25T10:02:26.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 -- 172.21.15.133:0/2186892158 wait complete. 2024-04-25T10:02:26.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 Processor -- start 2024-04-25T10:02:26.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 -- start start 2024-04-25T10:02:26.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86bc0d0770 0x7f86bc159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86bc15db00 0x7f86bc160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86bc1614c0 0x7f86bc160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0b7430 con 0x7f86bc0d0770 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0b82c0 con 0x7f86bc15db00 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0d18a0 con 0x7f86bc1614c0 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c1dd8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86bc0d0770 0x7f86bc159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c1dd8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86bc0d0770 0x7f86bc159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50278/0 (socket says 172.21.15.133:50278) 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c1dd8640 1 -- 172.21.15.133:0/2304050254 learned_addr learned my addr 172.21.15.133:0/2304050254 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c1dd8640 1 -- 172.21.15.133:0/2304050254 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86bc1614c0 msgr2=0x7f86bc160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c1dd8640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86bc1614c0 0x7f86bc160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c1dd8640 1 -- 172.21.15.133:0/2304050254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86bc15db00 msgr2=0x7f86bc160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c1dd8640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86bc15db00 0x7f86bc160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c1dd8640 1 -- 172.21.15.133:0/2304050254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc150730 con 0x7f86bc0d0770 2024-04-25T10:02:26.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c1dd8640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86bc0d0770 0x7f86bc159ff0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86b8019100 tx=0x7f86b801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86b27fc640 1 -- 172.21.15.133:0/2304050254 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b80b3800 con 0x7f86bc0d0770 2024-04-25T10:02:26.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86b27fc640 1 -- 172.21.15.133:0/2304050254 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b80a6040 con 0x7f86bc0d0770 2024-04-25T10:02:26.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 -- 172.21.15.133:0/2304050254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86bc150450 con 0x7f86bc0d0770 2024-04-25T10:02:26.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86b27fc640 1 -- 172.21.15.133:0/2304050254 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b80b3c00 con 0x7f86bc0d0770 2024-04-25T10:02:26.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.652+0000 7f86c403c640 1 -- 172.21.15.133:0/2304050254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc158310 con 0x7f86bc0d0770 2024-04-25T10:02:26.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.656+0000 7f86c403c640 1 -- 172.21.15.133:0/2304050254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86bc068ef0 con 0x7f86bc0d0770 2024-04-25T10:02:26.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.656+0000 7f86b27fc640 1 -- 172.21.15.133:0/2304050254 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86b80c6390 con 0x7f86bc0d0770 2024-04-25T10:02:26.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.656+0000 7f86b27fc640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86a0066f60 0x7f86a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.656+0000 7f86b27fc640 1 -- 172.21.15.133:0/2304050254 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f86b811f9d0 con 0x7f86bc0d0770 2024-04-25T10:02:26.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.656+0000 7f86c15d7640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86a0066f60 0x7f86a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.656+0000 7f86c15d7640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86a0066f60 0x7f86a0069420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86bc157340 tx=0x7f86ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.660+0000 7f86b27fc640 1 -- 172.21.15.133:0/2304050254 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86b80c3060 con 0x7f86bc0d0770 2024-04-25T10:02:26.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.840+0000 7f86c403c640 1 -- 172.21.15.133:0/2304050254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86bc068ef0 con 0x7f86bc0d0770 2024-04-25T10:02:26.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.840+0000 7f86b27fc640 1 -- 172.21.15.133:0/2304050254 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f86b8016110 con 0x7f86bc0d0770 2024-04-25T10:02:26.847 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:26.847 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2230831968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2230831968},{"type":"v1","addr":"172.21.15.162:6835","nonce":2230831968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1456396097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1456396097},{"type":"v1","addr":"172.21.15.133:6836","nonce":1456396097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:51.276631+0000","modified":"2024-04-25T10:02:14.440426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2462154167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2462154167},{"type":"v1","addr":"172.21.15.133:6837","nonce":2462154167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:26.847 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 9 2024-04-25T10:02:26.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 -- 172.21.15.133:0/2304050254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86a0066f60 msgr2=0x7f86a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86a0066f60 0x7f86a0069420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86bc157340 tx=0x7f86ac008040 comp rx=0 tx=0).stop 2024-04-25T10:02:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 -- 172.21.15.133:0/2304050254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86bc0d0770 msgr2=0x7f86bc159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86bc0d0770 0x7f86bc159ff0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86b8019100 tx=0x7f86b801ad30 comp rx=0 tx=0).stop 2024-04-25T10:02:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 -- 172.21.15.133:0/2304050254 shutdown_connections 2024-04-25T10:02:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86bc1614c0 0x7f86bc160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86bc15db00 0x7f86bc160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86a0066f60 0x7f86a0069420 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 --2- 172.21.15.133:0/2304050254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86bc0d0770 0x7f86bc159ff0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 -- 172.21.15.133:0/2304050254 >> 172.21.15.133:0/2304050254 conn(0x7f86bc0727c0 msgr2=0x7f86bc0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 -- 172.21.15.133:0/2304050254 shutdown_connections 2024-04-25T10:02:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:26.844+0000 7f86c403c640 1 -- 172.21.15.133:0/2304050254 wait complete. 2024-04-25T10:02:27.267 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:01:51.276631+0000', 'modified': '2024-04-25T10:02:14.440426+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/2462154167', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 2462154167}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 2462154167}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:27.267 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:27.268 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:02:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6ebc1640 1 Processor -- start 2024-04-25T10:02:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6ebc1640 1 -- start start 2024-04-25T10:02:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6ebc1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e6806bd80 0x7f9e681685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6ebc1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e680ce0b0 con 0x7f9e6815dff0 2024-04-25T10:02:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6ebc1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e680ce350 con 0x7f9e680cb410 2024-04-25T10:02:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6ebc1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e680ce630 con 0x7f9e6806bd80 2024-04-25T10:02:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6d15e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e6806bd80 0x7f9e681685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6d15e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e6806bd80 0x7f9e681685b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50282/0 (socket says 172.21.15.133:50282) 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6d15e640 1 -- 172.21.15.133:0/3639587092 learned_addr learned my addr 172.21.15.133:0/3639587092 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6d15e640 1 -- 172.21.15.133:0/3639587092 >> v1:172.21.15.162:6789/0 conn(0x7f9e680cb410 legacy=0x7f9e6815e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6d15e640 1 -- 172.21.15.133:0/3639587092 >> v1:172.21.15.133:6789/0 conn(0x7f9e6815dff0 legacy=0x7f9e680cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.396+0000 7f9e6d15e640 1 -- 172.21.15.133:0/3639587092 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680ce910 con 0x7f9e6806bd80 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e6d15e640 1 --2- 172.21.15.133:0/3639587092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e6806bd80 0x7f9e681685b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009e50 tx=0x7f9e6001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0675e1e030de485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e677fe640 1 -- 172.21.15.133:0/3639587092 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e600a5620 con 0x7f9e6806bd80 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e677fe640 1 -- 172.21.15.133:0/3639587092 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e600a57c0 con 0x7f9e6806bd80 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e677fe640 1 -- 172.21.15.133:0/3639587092 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e600a5b90 con 0x7f9e6806bd80 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/3639587092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e6806bd80 msgr2=0x7f9e681685b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e6ebc1640 1 --2- 172.21.15.133:0/3639587092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e6806bd80 0x7f9e681685b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009e50 tx=0x7f9e6001c960 comp rx=0 tx=0).stop 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/3639587092 shutdown_connections 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e6ebc1640 1 --2- 172.21.15.133:0/3639587092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e6806bd80 0x7f9e681685b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/3639587092 >> 172.21.15.133:0/3639587092 conn(0x7f9e680cc790 msgr2=0x7f9e680ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/3639587092 shutdown_connections 2024-04-25T10:02:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/3639587092 wait complete. 2024-04-25T10:02:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e6ebc1640 1 Processor -- start 2024-04-25T10:02:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.400+0000 7f9e6ebc1640 1 -- start start 2024-04-25T10:02:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6ebc1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e6806bd80 0x7f9e680b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6ebc1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e680cb410 0x7f9e6806e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6ebc1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e6815dff0 0x7f9e6806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6ebc1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6816bab0 con 0x7f9e680cb410 2024-04-25T10:02:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6ebc1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6816b930 con 0x7f9e6815dff0 2024-04-25T10:02:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6ebc1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6816bc30 con 0x7f9e6806bd80 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e680cb410 0x7f9e6806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e680cb410 0x7f9e6806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50290/0 (socket says 172.21.15.133:50290) 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e67fff640 1 -- 172.21.15.133:0/612296460 learned_addr learned my addr 172.21.15.133:0/612296460 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6d15e640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e6815dff0 0x7f9e6806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e6806bd80 0x7f9e680b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e67fff640 1 -- 172.21.15.133:0/612296460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e6806bd80 msgr2=0x7f9e680b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e67fff640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e6806bd80 0x7f9e680b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e67fff640 1 -- 172.21.15.133:0/612296460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e6815dff0 msgr2=0x7f9e6806e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e67fff640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e6815dff0 0x7f9e6806e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e67fff640 1 -- 172.21.15.133:0/612296460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680c9a20 con 0x7f9e680cb410 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6d15e640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e6815dff0 0x7f9e6806e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e67fff640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e680cb410 0x7f9e6806e330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800c450 tx=0x7f9e5800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e657fa640 1 -- 172.21.15.133:0/612296460 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58012830 con 0x7f9e680cb410 2024-04-25T10:02:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e657fa640 1 -- 172.21.15.133:0/612296460 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e58017070 con 0x7f9e680cb410 2024-04-25T10:02:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e657fa640 1 -- 172.21.15.133:0/612296460 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58012c30 con 0x7f9e680cb410 2024-04-25T10:02:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/612296460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e680b3140 con 0x7f9e680cb410 2024-04-25T10:02:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/612296460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e680b3700 con 0x7f9e680cb410 2024-04-25T10:02:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e657fa640 1 -- 172.21.15.133:0/612296460 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e5801c460 con 0x7f9e680cb410 2024-04-25T10:02:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e657fa640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e48066f90 0x7f9e48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e657fa640 1 -- 172.21.15.133:0/612296460 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9e58085140 con 0x7f9e680cb410 2024-04-25T10:02:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.404+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e48066f90 0x7f9e48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e48066f90 0x7f9e48069450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9e68158da0 tx=0x7f9e50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 Processor -- start 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 -- start start 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a4920 0x562bfb6a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb6a5240 0x562bfb65a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb65af30 0x562bfb65b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfb5425f0 con 0x562bfb6a5240 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfb4a9da0 con 0x562bfb65af30 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bfb432150 con 0x562bfb6a4920 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb6a5240 0x562bfb65a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb6a5240 0x562bfb65a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50294/0 (socket says 172.21.15.133:50294) 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e67fff640 1 -- 172.21.15.133:0/3203112662 learned_addr learned my addr 172.21.15.133:0/3203112662 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/3203112662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a4920 0x562bfb6a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e67fff640 1 -- 172.21.15.133:0/3203112662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a4920 msgr2=0x562bfb6a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e6d15e640 1 --2- 172.21.15.133:0/3203112662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb65af30 0x562bfb65b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e67fff640 1 --2- 172.21.15.133:0/3203112662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a4920 0x562bfb6a4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e67fff640 1 -- 172.21.15.133:0/3203112662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb65af30 msgr2=0x562bfb65b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e67fff640 1 --2- 172.21.15.133:0/3203112662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb65af30 0x562bfb65b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e67fff640 1 -- 172.21.15.133:0/3203112662 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bfb44bca0 con 0x562bfb6a5240 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e67fff640 1 --2- 172.21.15.133:0/3203112662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb6a5240 0x562bfb65a9f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9e58051b10 tx=0x7f9e58055700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469a8314f9bef217 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e5e7fc640 1 -- 172.21.15.133:0/3203112662 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5806d1d0 con 0x562bfb6a5240 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e5e7fc640 1 -- 172.21.15.133:0/3203112662 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e58053a00 con 0x562bfb6a5240 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e5e7fc640 1 -- 172.21.15.133:0/3203112662 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5805d980 con 0x562bfb6a5240 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 -- 172.21.15.133:0/3203112662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb6a5240 msgr2=0x562bfb65a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/3203112662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb6a5240 0x562bfb65a9f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9e58051b10 tx=0x7f9e58055700 comp rx=0 tx=0).stop 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 -- 172.21.15.133:0/3203112662 shutdown_connections 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/3203112662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb65af30 0x562bfb65b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/3203112662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb6a5240 0x562bfb65a9f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/3203112662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a4920 0x562bfb6a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 -- 172.21.15.133:0/3203112662 >> 172.21.15.133:0/3203112662 conn(0x562bfb6612a0 msgr2=0x562bfb6592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 -- 172.21.15.133:0/3203112662 shutdown_connections 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 -- 172.21.15.133:0/3203112662 wait complete. 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 -- start start 2024-04-25T10:02:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.408+0000 7f9e71d2e000 1 Processor -- start 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6abc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 0x562bfb6ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 0x562bfb65eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfb464990 con 0x562bfb65af30 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfb4a5e40 con 0x562bfb6a4920 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bfb435900 con 0x562bfb6a5240 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e6c95d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 10 client.0 _ll_register_callbacks cb 0x562bfb55d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e6c95d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6abc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50296/0 (socket says 172.21.15.133:50296) 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e6c95d640 1 -- 172.21.15.133:0/1472834523 learned_addr learned my addr 172.21.15.133:0/1472834523 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e6c95d640 1 -- 172.21.15.133:0/1472834523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 msgr2=0x562bfb65eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/1472834523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 0x562bfb65eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e6c95d640 1 -- 172.21.15.133:0/1472834523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 msgr2=0x562bfb6ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/1472834523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 0x562bfb6ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e6c95d640 1 -- 172.21.15.133:0/1472834523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bfb462630 con 0x562bfb65af30 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/1472834523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6abc80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9e500038d0 tx=0x7f9e50000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb27905fd856fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e477fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e477fe640 1 -- 172.21.15.133:0/1472834523 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000dc20 con 0x562bfb65af30 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e477fe640 1 -- 172.21.15.133:0/1472834523 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9e5000ddc0 con 0x562bfb65af30 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 -- 172.21.15.133:0/1472834523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e58053a00 con 0x562bfb65af30 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e477fe640 1 -- 172.21.15.133:0/1472834523 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e50012410 con 0x562bfb65af30 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 10 client.4869 fetch_fsmap learned FSMap version 9 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 -- 172.21.15.133:0/1472834523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x562bfb4323a0 con 0x562bfb65af30 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e477fe640 1 -- 172.21.15.133:0/1472834523 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e58053a00 con 0x562bfb65af30 2024-04-25T10:02:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e477fe640 1 -- 172.21.15.133:0/1472834523 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f9e5002a050 con 0x562bfb65af30 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 10 client.4869 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 10 client.4869 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 20 client.4869 populate_metadata read hostname 'smithi133' 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/1472834523 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x562bfb65f4b0 0x562bfb65f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 4 client.4869 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 -- 172.21.15.133:0/1472834523 --> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bfb445a50 con 0x562bfb65f4b0 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e67fff640 1 --2- 172.21.15.133:0/1472834523 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x562bfb65f4b0 0x562bfb65f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e67fff640 1 --2- 172.21.15.133:0/1472834523 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x562bfb65f4b0 0x562bfb65f890 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d6c50de2ebd885 server_cookie=524d72db6c2ae0f8 in_seq=0 out_seq=0 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e477fe640 10 client.4869 ms_handle_connect on v2:172.21.15.133:6835/2462154167 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e477fe640 1 -- 172.21.15.133:0/1472834523 <== mds.0 v2:172.21.15.133:6835/2462154167 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562bfb445a50 con 0x562bfb65f4b0 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e477fe640 10 client.4869 handle_command_reply: tid=0 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 client.4869 shutdown 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 -- 172.21.15.133:0/1472834523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 msgr2=0x562bfb6abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/1472834523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6abc80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9e500038d0 tx=0x7f9e50000ee0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 -- 172.21.15.133:0/1472834523 shutdown_connections 2024-04-25T10:02:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/1472834523 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x562bfb65f4b0 0x562bfb65f890 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/1472834523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 0x562bfb65eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/1472834523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 0x562bfb6ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e477fe640 0 client.4869 ms_handle_reset on v2:172.21.15.133:6835/2462154167 2024-04-25T10:02:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/1472834523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6abc80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 -- 172.21.15.133:0/1472834523 >> 172.21.15.133:0/1472834523 conn(0x562bfb6612a0 msgr2=0x562bfb6599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 -- 172.21.15.133:0/1472834523 shutdown_connections 2024-04-25T10:02:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 1 -- 172.21.15.133:0/1472834523 wait complete. 2024-04-25T10:02:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.412+0000 7f9e71d2e000 20 client.4869 trim_cache size 0 max 16384 2024-04-25T10:02:27.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e71d2e000 1 Processor -- start 2024-04-25T10:02:27.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e71d2e000 1 -- start start 2024-04-25T10:02:27.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 0x562bfb7004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 0x562bfb7009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68203be0 con 0x562bfb65af30 2024-04-25T10:02:27.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68202fe0 con 0x562bfb6a4920 2024-04-25T10:02:27.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e682035e0 con 0x562bfb6a5240 2024-04-25T10:02:27.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e6c95d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e6c95d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6fff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50298/0 (socket says 172.21.15.133:50298) 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e6c95d640 1 -- 172.21.15.133:0/3822891194 learned_addr learned my addr 172.21.15.133:0/3822891194 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e6c95d640 1 -- 172.21.15.133:0/3822891194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 msgr2=0x562bfb7009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/3822891194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 0x562bfb7009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e6c95d640 1 -- 172.21.15.133:0/3822891194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 msgr2=0x562bfb7004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/3822891194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 0x562bfb7004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.428+0000 7f9e6c95d640 1 -- 172.21.15.133:0/3822891194 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bfb6a5070 con 0x562bfb65af30 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/3822891194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6fff60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e500038a0 tx=0x7f9e50012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9195d313c054a0a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e5dffb640 1 -- 172.21.15.133:0/3822891194 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000daf0 con 0x562bfb65af30 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e5dffb640 1 -- 172.21.15.133:0/3822891194 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50019040 con 0x562bfb65af30 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e5dffb640 1 -- 172.21.15.133:0/3822891194 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50012830 con 0x562bfb65af30 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- 172.21.15.133:0/3822891194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 msgr2=0x562bfb6fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/3822891194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6fff60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e500038a0 tx=0x7f9e50012000 comp rx=0 tx=0).stop 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- 172.21.15.133:0/3822891194 shutdown_connections 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/3822891194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 0x562bfb7009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/3822891194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 0x562bfb7004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/3822891194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6fff60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- 172.21.15.133:0/3822891194 >> 172.21.15.133:0/3822891194 conn(0x562bfb6612a0 msgr2=0x562bfb65e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- 172.21.15.133:0/3822891194 shutdown_connections 2024-04-25T10:02:27.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- 172.21.15.133:0/3822891194 wait complete. 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- start start 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 Processor -- start 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 0x562bfb6e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 0x562bfb6e9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfb4ae3d0 con 0x562bfb65af30 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bfb565d50 con 0x562bfb6a4920 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bfb444980 con 0x562bfb6a5240 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 10 client.0 _ll_register_callbacks cb 0x7f9e580554d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e6c95d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e6c95d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50310/0 (socket says 172.21.15.133:50310) 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e6c95d640 1 -- 172.21.15.133:0/674487470 learned_addr learned my addr 172.21.15.133:0/674487470 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e6c95d640 1 -- 172.21.15.133:0/674487470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 msgr2=0x562bfb6e9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 0x562bfb6e9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e6c95d640 1 -- 172.21.15.133:0/674487470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 msgr2=0x562bfb6e97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e67fff640 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 0x562bfb6e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 0x562bfb6e97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e6c95d640 1 -- 172.21.15.133:0/674487470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bfb52b600 con 0x562bfb65af30 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e6c95d640 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6b1580 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e500127b0 tx=0x7f9e50011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92edf009ba9ee326 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e477fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e477fe640 1 -- 172.21.15.133:0/674487470 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5003b070 con 0x562bfb65af30 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e477fe640 1 -- 172.21.15.133:0/674487470 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9e50012d00 con 0x562bfb65af30 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- 172.21.15.133:0/674487470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e50019040 con 0x562bfb65af30 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e477fe640 1 -- 172.21.15.133:0/674487470 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e5002c470 con 0x562bfb65af30 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 10 client.4875 fetch_fsmap learned FSMap version 9 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- 172.21.15.133:0/674487470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x562bfb43e9b0 con 0x562bfb65af30 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e477fe640 1 -- 172.21.15.133:0/674487470 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50019040 con 0x562bfb65af30 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e477fe640 1 -- 172.21.15.133:0/674487470 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f9e50035030 con 0x562bfb65af30 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 10 client.4875 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 10 client.4875 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 20 client.4875 populate_metadata read hostname 'smithi133' 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x562bfb6b1ac0 0x562bfb6ea250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 4 client.4875 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e71d2e000 1 -- 172.21.15.133:0/674487470 --> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562bfb4b72e0 con 0x562bfb6b1ac0 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e67fff640 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x562bfb6b1ac0 0x562bfb6ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.432+0000 7f9e477fe640 10 client.4875 ms_handle_connect on v2:172.21.15.133:6835/2462154167 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e67fff640 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x562bfb6b1ac0 0x562bfb6ea250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2579489be57068d server_cookie=f759dfbe7d7cf86c in_seq=0 out_seq=0 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e477fe640 1 -- 172.21.15.133:0/674487470 <== mds.0 v2:172.21.15.133:6835/2462154167 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x562bfb4b72e0 con 0x562bfb6b1ac0 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e477fe640 10 client.4875 handle_command_reply: tid=0 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 1 client.4875 shutdown 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 1 -- 172.21.15.133:0/674487470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 msgr2=0x562bfb6b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6b1580 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e500127b0 tx=0x7f9e50011000 comp rx=0 tx=0).stop 2024-04-25T10:02:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 1 -- 172.21.15.133:0/674487470 shutdown_connections 2024-04-25T10:02:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.133:6835/2462154167,v1:172.21.15.133:6837/2462154167] conn(0x562bfb6b1ac0 0x562bfb6ea250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562bfb6a5240 0x562bfb6e9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562bfb6a4920 0x562bfb6e97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 1 --2- 172.21.15.133:0/674487470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562bfb65af30 0x562bfb6b1580 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e477fe640 0 client.4875 ms_handle_reset on v2:172.21.15.133:6835/2462154167 2024-04-25T10:02:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 1 -- 172.21.15.133:0/674487470 >> 172.21.15.133:0/674487470 conn(0x562bfb6612a0 msgr2=0x562bfb6ac840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 1 -- 172.21.15.133:0/674487470 shutdown_connections 2024-04-25T10:02:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 1 -- 172.21.15.133:0/674487470 wait complete. 2024-04-25T10:02:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e71d2e000 20 client.4875 trim_cache size 0 max 16384 2024-04-25T10:02:27.447 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4296, 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 35.495309425000002, 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 9, 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 24, 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 41.973206804 2024-04-25T10:02:27.448 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:02:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/612296460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e48066f90 msgr2=0x7f9e48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e48066f90 0x7f9e48069450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9e68158da0 tx=0x7f9e50009040 comp rx=0 tx=0).stop 2024-04-25T10:02:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/612296460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e680cb410 msgr2=0x7f9e6806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e680cb410 0x7f9e6806e330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800c450 tx=0x7f9e5800c910 comp rx=0 tx=0).stop 2024-04-25T10:02:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/612296460 shutdown_connections 2024-04-25T10:02:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e6815dff0 0x7f9e6806e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e48066f90 0x7f9e48069450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e680cb410 0x7f9e6806e330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 --2- 172.21.15.133:0/612296460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e6806bd80 0x7f9e680b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/612296460 >> 172.21.15.133:0/612296460 conn(0x7f9e680cc790 msgr2=0x7f9e680b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/612296460 shutdown_connections 2024-04-25T10:02:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.436+0000 7f9e6ebc1640 1 -- 172.21.15.133:0/612296460 wait complete. 2024-04-25T10:02:27.458 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T10:02:27.469 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T10:02:27.469 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T10:02:27.469 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T10:02:27.469 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:27.469 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:27.469 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T10:02:27.469 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:27.470 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:27.470 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T10:02:27.470 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:27.470 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:27.470 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T10:02:27.470 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:27.470 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:27.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:27.470 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-04-25T10:02:27.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:27.477 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link delete ceph-brx 2024-04-25T10:02:27.537 INFO:teuthology.orchestra.run.smithi162.stderr:Cannot find device "ceph-brx" 2024-04-25T10:02:27.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:02:27.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:27.539 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:02:27.545 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T10:02:27.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:27.545 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:27.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:27.602 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-04-25T10:02:27.791 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T10:02:27.791 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.916+0000 7f27c4c4b640 1 Processor -- start 2024-04-25T10:02:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.916+0000 7f27c4c4b640 1 -- start start 2024-04-25T10:02:27.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.916+0000 7f27c4c4b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f27c015db00 0x7f27c00b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.916+0000 7f27c4c4b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f27c00d0770 0x7f27c00d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.916+0000 7f27c4c4b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c00ceb20 con 0x7f27c01614c0 2024-04-25T10:02:27.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.916+0000 7f27c4c4b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c00d1110 con 0x7f27c015db00 2024-04-25T10:02:27.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.916+0000 7f27c4c4b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c00b7740 con 0x7f27c00d0770 2024-04-25T10:02:27.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.916+0000 7f27be59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f27c015db00 0x7f27c00b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.916+0000 7f27be59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f27c015db00 0x7f27c00b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50312/0 (socket says 172.21.15.133:50312) 2024-04-25T10:02:27.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.916+0000 7f27bed9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f27c01614c0 0x7f27c0161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34014/0 (socket says 172.21.15.133:34014) 2024-04-25T10:02:27.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.916+0000 7f27be59c640 1 -- 172.21.15.133:0/2286433854 learned_addr learned my addr 172.21.15.133:0/2286433854 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:27.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27be59c640 1 -- 172.21.15.133:0/2286433854 >> v2:172.21.15.162:3300/0 conn(0x7f27c00d0770 msgr2=0x7f27c00d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27be59c640 1 --2- 172.21.15.133:0/2286433854 >> v2:172.21.15.162:3300/0 conn(0x7f27c00d0770 0x7f27c00d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27be59c640 1 -- 172.21.15.133:0/2286433854 >> v1:172.21.15.133:6789/0 conn(0x7f27c01614c0 legacy=0x7f27c0161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27be59c640 1 -- 172.21.15.133:0/2286433854 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c00b97f0 con 0x7f27c015db00 2024-04-25T10:02:27.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27be59c640 1 --2- 172.21.15.133:0/2286433854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27c015db00 0x7f27c00b2990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f27c00d0200 tx=0x7f27b001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf8b59aa4d32f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27bdd9b640 1 -- 172.21.15.133:0/2286433854 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b00aa650 con 0x7f27c015db00 2024-04-25T10:02:27.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27bdd9b640 1 -- 172.21.15.133:0/2286433854 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b00af070 con 0x7f27c015db00 2024-04-25T10:02:27.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27bdd9b640 1 -- 172.21.15.133:0/2286433854 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b00aaa10 con 0x7f27c015db00 2024-04-25T10:02:27.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2286433854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27c015db00 msgr2=0x7f27c00b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27c4c4b640 1 --2- 172.21.15.133:0/2286433854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27c015db00 0x7f27c00b2990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f27c00d0200 tx=0x7f27b001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2286433854 shutdown_connections 2024-04-25T10:02:27.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27c4c4b640 1 --2- 172.21.15.133:0/2286433854 >> v2:172.21.15.162:3300/0 conn(0x7f27c00d0770 0x7f27c00d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27c4c4b640 1 --2- 172.21.15.133:0/2286433854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27c015db00 0x7f27c00b2990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2286433854 >> 172.21.15.133:0/2286433854 conn(0x7f27c00727c0 msgr2=0x7f27c0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2286433854 shutdown_connections 2024-04-25T10:02:27.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.920+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2286433854 wait complete. 2024-04-25T10:02:27.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27c4c4b640 1 Processor -- start 2024-04-25T10:02:27.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27c4c4b640 1 -- start start 2024-04-25T10:02:27.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27c4c4b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c00d0770 0x7f27c014c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27c4c4b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27c015db00 0x7f27c014e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27c4c4b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c01614c0 0x7f27c015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27c4c4b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c00b7430 con 0x7f27c015db00 2024-04-25T10:02:27.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27c4c4b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c00b82c0 con 0x7f27c00d0770 2024-04-25T10:02:27.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27c4c4b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c00d18a0 con 0x7f27c01614c0 2024-04-25T10:02:27.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27c015db00 0x7f27c014e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27c015db00 0x7f27c014e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50328/0 (socket says 172.21.15.133:50328) 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b5d9b640 1 -- 172.21.15.133:0/2997177706 learned_addr learned my addr 172.21.15.133:0/2997177706 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27be59c640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c00d0770 0x7f27c014c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b5d9b640 1 -- 172.21.15.133:0/2997177706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c01614c0 msgr2=0x7f27c015ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b5d9b640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c01614c0 0x7f27c015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b5d9b640 1 -- 172.21.15.133:0/2997177706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c00d0770 msgr2=0x7f27c014c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b5d9b640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c00d0770 0x7f27c014c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b5d9b640 1 -- 172.21.15.133:0/2997177706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c0150980 con 0x7f27c015db00 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27be59c640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c00d0770 0x7f27c014c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b5d9b640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27c015db00 0x7f27c014e340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f27a800b410 tx=0x7f27a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b77fe640 1 -- 172.21.15.133:0/2997177706 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8015020 con 0x7f27c015db00 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b77fe640 1 -- 172.21.15.133:0/2997177706 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a8011830 con 0x7f27c015db00 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2997177706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c01502c0 con 0x7f27c015db00 2024-04-25T10:02:27.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b77fe640 1 -- 172.21.15.133:0/2997177706 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8011c20 con 0x7f27c015db00 2024-04-25T10:02:27.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2997177706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c015f620 con 0x7f27c015db00 2024-04-25T10:02:27.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2997177706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c0068ef0 con 0x7f27c015db00 2024-04-25T10:02:27.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b77fe640 1 -- 172.21.15.133:0/2997177706 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27a8018020 con 0x7f27c015db00 2024-04-25T10:02:27.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b77fe640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f27a0066f90 0x7f27a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27b77fe640 1 -- 172.21.15.133:0/2997177706 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f27a8085190 con 0x7f27c015db00 2024-04-25T10:02:27.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.924+0000 7f27be59c640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f27a0066f90 0x7f27a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.928+0000 7f27be59c640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f27a0066f90 0x7f27a0069450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f27b00ae000 tx=0x7f27b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:27.932+0000 7f27b77fe640 1 -- 172.21.15.133:0/2997177706 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27a801f020 con 0x7f27c015db00 2024-04-25T10:02:28.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.116+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2997177706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f27c00b2750 con 0x7f27c015db00 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27b77fe640 1 -- 172.21.15.133:0/2997177706 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f27a80172c0 con 0x7f27c015db00 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2997177706 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f27a0066f90 msgr2=0x7f27a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f27a0066f90 0x7f27a0069450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f27b00ae000 tx=0x7f27b0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2997177706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27c015db00 msgr2=0x7f27c014e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27c015db00 0x7f27c014e340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f27a800b410 tx=0x7f27a800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2997177706 shutdown_connections 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27c01614c0 0x7f27c015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f27a0066f90 0x7f27a0069450 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27c015db00 0x7f27c014e340 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 --2- 172.21.15.133:0/2997177706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f27c00d0770 0x7f27c014c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2997177706 >> 172.21.15.133:0/2997177706 conn(0x7f27c00727c0 msgr2=0x7f27c00b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2997177706 shutdown_connections 2024-04-25T10:02:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:28.120+0000 7f27c4c4b640 1 -- 172.21.15.133:0/2997177706 wait complete. 2024-04-25T10:02:28.143 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:28.143 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:28.143 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:02:28.143 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:02:28.143 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:28.143 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:28.143 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:28.143 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:28.143 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:28.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.144 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:02:28.149 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:28.149 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:28.149 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:28.150 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:28.151 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42483sec preferred_lft 42483sec 2024-04-25T10:02:28.151 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:02:28.151 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:28.151 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:02:28.151 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:28.151 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:02:28.151 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:28.151 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:02:28.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.151 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:02:28.151 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:02:28.151 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:02:28.152 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2024-04-25T10:02:28.152 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2024-04-25T10:02:28.152 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:02:28.152 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:28.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.234 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:28.286 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:02:28.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.288 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:02:28.337 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:28.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.339 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:02:28.339 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:02:28.339 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:28.339 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:28.339 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:28.339 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:28.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.412 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:02:28.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.458 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-04-25T10:02:28.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.505 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:02:28.505 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:02:28.506 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:28.506 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:02:28.506 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:28.574 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:02:28.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.574 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:02:28.574 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:02:28.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:02:28.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:28.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:02:28.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:02:28.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:02:28.574 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:28.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.668 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:02:28.668 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:02:28.668 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:02:28.668 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:28.668 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:28.687 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:02:28.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:28.687 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:28.733 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:28.733 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:28.733 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:28.782 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:02:28.839 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:28.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.890 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:02:28.938 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:02:28.938 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:02:28.982 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:02:28.997 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:29.047 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:02:29.047 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:02:29.047 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:02:29.047 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:02:29.047 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:02:29.047 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:02:29.048 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:29.049 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:02:29.049 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:02:29.049 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:02:29.049 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:29.049 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:29.049 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:29.049 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:29.049 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:02:29.049 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:29.049 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:29.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:29.049 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:02:29.066 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:02:29.060+0000 7f1befc22580 -1 init, newargv = 0x556c04cb0e00 newargc=15 2024-04-25T10:02:29.066 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[36088]: starting ceph client 2024-04-25T10:02:29.072 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[36088]: starting fuse 2024-04-25T10:02:30.172 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:02:30.187 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:30.235 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:02:30.235 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:02:30.235 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:02:30.235 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:02:30.235 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:02:30.235 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:02:30.235 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:02:30.235 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:02:30.235 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:02:30.235 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:30.236 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:02:30.237 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:02:30.237 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:02:30.237 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:30.237 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:30.237 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:30.237 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:30.237 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:02:30.237 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:30.237 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:30.237 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:30.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:30.238 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:02:30.286 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:02:30.286 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:02:30.287 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:02:30.288 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:02:30.410 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.36088.asok 2024-04-25T10:02:30.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:30.422 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36088.asok 2024-04-25T10:02:30.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:30.422 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36088.asok status 2024-04-25T10:02:30.573 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "36088", 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4893, 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 4893 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2315595507 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2315595507 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.4893 192.168.0.1:0/2315595507", 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/2315595507", 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 9, 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 24, 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:02:30.577 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:02:30.587 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:30.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.700+0000 7f905ab43640 1 Processor -- start 2024-04-25T10:02:30.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.700+0000 7f905ab43640 1 -- start start 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f905ab43640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90540b1f70 0x7f905406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f905ab43640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90540c61d0 con 0x7f90540ce330 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f905ab43640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90540c64b0 con 0x7f905415c9a0 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f905ab43640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905406d4f0 con 0x7f90540b1f70 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90590e0640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f90540ce330 0x7f905415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34022/0 (socket says 172.21.15.133:34022) 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90590e0640 1 -- 172.21.15.133:0/1976522565 learned_addr learned my addr 172.21.15.133:0/1976522565 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f9053fff640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90540b1f70 0x7f905406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621662380 0 0) 0x7f90540c61d0 con 0x7f90540ce330 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f903c003530 con 0x7f90540ce330 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 87686127 0 0) 0x7f903c003530 con 0x7f90540ce330 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90540b1f70 msgr2=0x7f905406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90540b1f70 0x7f905406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 >> v1:172.21.15.162:6789/0 conn(0x7f905415c9a0 legacy=0x7f90540c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905406d7d0 con 0x7f90540ce330 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f904c001a00 con 0x7f90540ce330 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 >> v1:172.21.15.133:6789/0 conn(0x7f90540ce330 legacy=0x7f905415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 0x7f903c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90540b1f70 0x7f903c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903c004d50 0x7f903c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905406d4f0 con 0x7f903c0039f0 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c00ba80 con 0x7f903c004d50 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903c00bc80 con 0x7f90540b1f70 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90588df640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 0x7f903c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f9053fff640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90540b1f70 0x7f903c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90590e0640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903c004d50 0x7f903c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90588df640 1 -- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90540b1f70 msgr2=0x7f903c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90588df640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90540b1f70 0x7f903c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90588df640 1 -- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903c004d50 msgr2=0x7f903c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90588df640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903c004d50 0x7f903c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90588df640 1 -- 172.21.15.133:0/1976522565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c003530 con 0x7f903c0039f0 2024-04-25T10:02:30.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.704+0000 7f90590e0640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f903c004d50 0x7f903c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:30.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f90588df640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 0x7f903c003e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9044001750 tx=0x7f904401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422db0d402b2e3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90440013a0 con 0x7f903c0039f0 2024-04-25T10:02:30.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90440052f0 con 0x7f903c0039f0 2024-04-25T10:02:30.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f90537fe640 1 -- 172.21.15.133:0/1976522565 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044003840 con 0x7f903c0039f0 2024-04-25T10:02:30.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 -- 172.21.15.133:0/1976522565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 msgr2=0x7f903c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 --2- 172.21.15.133:0/1976522565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 0x7f903c003e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9044001750 tx=0x7f904401c3e0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f90590e0640 1 -- 172.21.15.133:0/1976522565 reap_dead start 2024-04-25T10:02:30.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 -- 172.21.15.133:0/1976522565 shutdown_connections 2024-04-25T10:02:30.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 -- 172.21.15.133:0/1976522565 >> 172.21.15.133:0/1976522565 conn(0x7f9054067000 msgr2=0x7f905406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 -- 172.21.15.133:0/1976522565 shutdown_connections 2024-04-25T10:02:30.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 -- 172.21.15.133:0/1976522565 wait complete. 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 Processor -- start 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 -- start start 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 0x7f9054166100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9054156220 0x7f905414dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90541645a0 0x7f90541552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9054167740 con 0x7f903c0039f0 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90541675c0 con 0x7f9054156220 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f905ab43640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90541678c0 con 0x7f90541645a0 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f90588df640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 0x7f9054166100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f90588df640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 0x7f9054166100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50348/0 (socket says 172.21.15.133:50348) 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f90588df640 1 -- 172.21.15.133:0/2713004679 learned_addr learned my addr 172.21.15.133:0/2713004679 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.708+0000 7f9053fff640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9054156220 0x7f905414dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90588df640 1 -- 172.21.15.133:0/2713004679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90541645a0 msgr2=0x7f90541552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90588df640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90541645a0 0x7f90541552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90588df640 1 -- 172.21.15.133:0/2713004679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9054156220 msgr2=0x7f905414dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90588df640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9054156220 0x7f905414dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90588df640 1 -- 172.21.15.133:0/2713004679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90541666e0 con 0x7f903c0039f0 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f9053fff640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9054156220 0x7f905414dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90588df640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 0x7f9054166100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9044005040 tx=0x7f903c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90517fa640 1 -- 172.21.15.133:0/2713004679 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044007a40 con 0x7f903c0039f0 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90517fa640 1 -- 172.21.15.133:0/2713004679 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90440bb070 con 0x7f903c0039f0 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f905ab43640 1 -- 172.21.15.133:0/2713004679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9054154c10 con 0x7f903c0039f0 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90517fa640 1 -- 172.21.15.133:0/2713004679 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90440013a0 con 0x7f903c0039f0 2024-04-25T10:02:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f905ab43640 1 -- 172.21.15.133:0/2713004679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905414e500 con 0x7f903c0039f0 2024-04-25T10:02:30.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f905ab43640 1 -- 172.21.15.133:0/2713004679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9054068ef0 con 0x7f903c0039f0 2024-04-25T10:02:30.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90517fa640 1 -- 172.21.15.133:0/2713004679 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90440b4040 con 0x7f903c0039f0 2024-04-25T10:02:30.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90517fa640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9040066f60 0x7f9040069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f90517fa640 1 -- 172.21.15.133:0/2713004679 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f904411f320 con 0x7f903c0039f0 2024-04-25T10:02:30.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f9053fff640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9040066f60 0x7f9040069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.712+0000 7f9053fff640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9040066f60 0x7f9040069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9054157cb0 tx=0x7f9048008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.716+0000 7f90517fa640 1 -- 172.21.15.133:0/2713004679 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90440160f0 con 0x7f903c0039f0 2024-04-25T10:02:30.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 -- 172.21.15.133:0/2713004679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9054157ce0 con 0x7f903c0039f0 2024-04-25T10:02:30.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f90517fa640 1 -- 172.21.15.133:0/2713004679 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f90440f0110 con 0x7f903c0039f0 2024-04-25T10:02:30.919 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 -- 172.21.15.133:0/2713004679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9040066f60 msgr2=0x7f9040069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9040066f60 0x7f9040069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9054157cb0 tx=0x7f9048008040 comp rx=0 tx=0).stop 2024-04-25T10:02:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 -- 172.21.15.133:0/2713004679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 msgr2=0x7f9054166100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 0x7f9054166100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9044005040 tx=0x7f903c005840 comp rx=0 tx=0).stop 2024-04-25T10:02:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 -- 172.21.15.133:0/2713004679 shutdown_connections 2024-04-25T10:02:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90541645a0 0x7f90541552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9054156220 0x7f905414dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9040066f60 0x7f9040069420 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 --2- 172.21.15.133:0/2713004679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c0039f0 0x7f9054166100 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 -- 172.21.15.133:0/2713004679 >> 172.21.15.133:0/2713004679 conn(0x7f9054067000 msgr2=0x7f905406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 -- 172.21.15.133:0/2713004679 shutdown_connections 2024-04-25T10:02:30.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:30.912+0000 7f905ab43640 1 -- 172.21.15.133:0/2713004679 wait complete. 2024-04-25T10:02:30.937 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:30.937 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:30.937 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:02:30.937 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T10:02:30.938 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:30.938 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:30.938 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:30.938 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:30.938 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:02:30.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:30.938 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42481sec preferred_lft 42481sec 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:30.946 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:30.947 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:30.996 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:02:30.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:30.997 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:02:31.050 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:02:31.050 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:31.050 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:31.051 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:02:31.052 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:02:31.052 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:31.052 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:31.052 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:31.052 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:31.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:31.127 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:02:31.178 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:02:31.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:31.179 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-04-25T10:02:31.229 INFO:teuthology.orchestra.run.smithi133.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T10:02:31.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:31.230 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:02:31.230 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:02:31.230 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:31.230 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T10:02:31.230 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:31.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:31.299 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:31.353 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi133.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether d2:51:2f:87:e3:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::d051:2fff:fe87:e378/64 scope link 2024-04-25T10:02:31.354 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:31.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:31.355 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:02:31.408 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:31.408 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:31.408 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:31.409 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:31.409 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:31.409 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:31.409 INFO:teuthology.orchestra.run.smithi133.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:31.409 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether d2:51:2f:87:e3:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:31.409 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:31.409 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:31.409 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::d051:2fff:fe87:e378/64 scope link 2024-04-25T10:02:31.409 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:31.410 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T10:02:31.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:31.411 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:02:31.411 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:02:31.411 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T10:02:31.411 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:31.411 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T10:02:31.411 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T10:02:31.411 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T10:02:31.411 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:31.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:31.510 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:02:31.510 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:02:31.510 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.1 up 2024-04-25T10:02:31.510 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:02:31.510 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:02:31.529 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T10:02:31.530 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:31.530 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:31.577 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:02:31.578 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:31.578 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:31.629 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:02:31.686 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:31.735 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:02:31.735 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:02:31.735 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:02:31.735 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:02:31.735 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:02:31.735 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:02:31.735 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:02:31.735 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:02:31.735 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:02:31.735 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:31.736 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:31.737 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:31.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:31.739 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:02:31.792 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:02:31.792 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T10:02:31.792 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T10:02:31.835 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:02:31.847 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:31.896 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:31.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:31.896 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:02:31.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi133.stderr:2024-04-25T10:02:31.904+0000 7fbdb3ec3580 -1 init, newargv = 0x564033aa7660 newargc=15 2024-04-25T10:02:31.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi133.stderr:ceph-fuse[36229]: starting ceph client 2024-04-25T10:02:31.913 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:02:31.915 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi133.stderr:ceph-fuse[36229]: starting fuse 2024-04-25T10:02:33.015 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:02:33.031 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:33.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:33.086 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:02:33.134 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:02:33.134 INFO:teuthology.orchestra.run.smithi133.stdout:46 2024-04-25T10:02:33.134 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T10:02:33.134 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:33.134 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:33.135 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:33.136 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:33.136 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:02:33.136 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:02:33.136 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:02:33.136 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:33.136 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:02:33.136 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.2")) 2024-04-25T10:02:33.136 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:02:33.260 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.2.36229.asok 2024-04-25T10:02:33.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:33.271 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.36229.asok 2024-04-25T10:02:33.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:33.272 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.36229.asok status 2024-04-25T10:02:33.428 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:02:33.428 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:02:33.428 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:33.428 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "2", 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "36229", 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4905, 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 4905 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:33.429 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3037300100 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3037300100 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.4905 192.168.0.2:0/3037300100", 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.2:0/3037300100", 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:02:33.430 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 9, 2024-04-25T10:02:33.431 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 25, 2024-04-25T10:02:33.431 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:33.431 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:02:33.431 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:02:33.431 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:02:33.443 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:33.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb59d5ce640 1 Processor -- start 2024-04-25T10:02:33.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb59d5ce640 1 -- start start 2024-04-25T10:02:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb59d5ce640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb59815c9a0 0x7fb5980c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb59d5ce640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5980c61d0 con 0x7fb5980ce330 2024-04-25T10:02:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb59d5ce640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5980c64b0 con 0x7fb5980b1f70 2024-04-25T10:02:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb59d5ce640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59806d4f0 con 0x7fb59815c9a0 2024-04-25T10:02:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb5977fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb5980ce330 0x7fb59815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34032/0 (socket says 172.21.15.133:34032) 2024-04-25T10:02:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb5977fe640 1 -- 172.21.15.133:0/953382800 learned_addr learned my addr 172.21.15.133:0/953382800 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb596ffd640 1 --2- 172.21.15.133:0/953382800 >> v2:172.21.15.133:3300/0 conn(0x7fb59815c9a0 0x7fb5980c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb595ffb640 1 -- 172.21.15.133:0/953382800 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742975286 0 0) 0x7fb5980c61d0 con 0x7fb5980ce330 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb595ffb640 1 -- 172.21.15.133:0/953382800 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb580003530 con 0x7fb5980ce330 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb596ffd640 1 -- 172.21.15.133:0/953382800 >> v1:172.21.15.162:6789/0 conn(0x7fb5980b1f70 legacy=0x7fb59806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb596ffd640 1 -- 172.21.15.133:0/953382800 >> v1:172.21.15.133:6789/0 conn(0x7fb5980ce330 legacy=0x7fb59815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.556+0000 7fb596ffd640 1 -- 172.21.15.133:0/953382800 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59806d7d0 con 0x7fb59815c9a0 2024-04-25T10:02:33.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb596ffd640 1 --2- 172.21.15.133:0/953382800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb59815c9a0 0x7fb5980c5980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb5880097c0 tx=0x7fb58801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f33247e3081759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb595ffb640 1 -- 172.21.15.133:0/953382800 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5880b0070 con 0x7fb59815c9a0 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb595ffb640 1 -- 172.21.15.133:0/953382800 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5880ab600 con 0x7fb59815c9a0 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb595ffb640 1 -- 172.21.15.133:0/953382800 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5880ab9d0 con 0x7fb59815c9a0 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 -- 172.21.15.133:0/953382800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb59815c9a0 msgr2=0x7fb5980c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 --2- 172.21.15.133:0/953382800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb59815c9a0 0x7fb5980c5980 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb5880097c0 tx=0x7fb58801c460 comp rx=0 tx=0).stop 2024-04-25T10:02:33.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 -- 172.21.15.133:0/953382800 shutdown_connections 2024-04-25T10:02:33.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 --2- 172.21.15.133:0/953382800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb59815c9a0 0x7fb5980c5980 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 -- 172.21.15.133:0/953382800 >> 172.21.15.133:0/953382800 conn(0x7fb598067000 msgr2=0x7fb59806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 -- 172.21.15.133:0/953382800 shutdown_connections 2024-04-25T10:02:33.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 -- 172.21.15.133:0/953382800 wait complete. 2024-04-25T10:02:33.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 Processor -- start 2024-04-25T10:02:33.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 -- start start 2024-04-25T10:02:33.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5980b1f70 0x7fb598156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5980ce330 0x7fb5980c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb59815c9a0 0x7fb598159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59815cf00 con 0x7fb5980ce330 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59815cd80 con 0x7fb59815c9a0 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb59d5ce640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb598169de0 con 0x7fb5980b1f70 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb5967fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5980ce330 0x7fb5980c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb5967fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5980ce330 0x7fb5980c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50370/0 (socket says 172.21.15.133:50370) 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb5967fc640 1 -- 172.21.15.133:0/2701517860 learned_addr learned my addr 172.21.15.133:0/2701517860 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb5967fc640 1 -- 172.21.15.133:0/2701517860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5980b1f70 msgr2=0x7fb598156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb5967fc640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5980b1f70 0x7fb598156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb5967fc640 1 -- 172.21.15.133:0/2701517860 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb59815c9a0 msgr2=0x7fb598159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb5977fe640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb59815c9a0 0x7fb598159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb5967fc640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb59815c9a0 0x7fb598159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb5967fc640 1 -- 172.21.15.133:0/2701517860 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5980c83a0 con 0x7fb5980ce330 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.560+0000 7fb5967fc640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5980ce330 0x7fb5980c7ce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00b440 tx=0x7fb58c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.564+0000 7fb57f7fe640 1 -- 172.21.15.133:0/2701517860 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c015020 con 0x7fb5980ce330 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.564+0000 7fb57f7fe640 1 -- 172.21.15.133:0/2701517860 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c011890 con 0x7fb5980ce330 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.564+0000 7fb57f7fe640 1 -- 172.21.15.133:0/2701517860 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c011c80 con 0x7fb5980ce330 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.564+0000 7fb59d5ce640 1 -- 172.21.15.133:0/2701517860 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5980c8680 con 0x7fb5980ce330 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.564+0000 7fb59d5ce640 1 -- 172.21.15.133:0/2701517860 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5980c8c40 con 0x7fb5980ce330 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.564+0000 7fb59d5ce640 1 -- 172.21.15.133:0/2701517860 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb598068ef0 con 0x7fb5980ce330 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.564+0000 7fb57f7fe640 1 -- 172.21.15.133:0/2701517860 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb58c018020 con 0x7fb5980ce330 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.564+0000 7fb57f7fe640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb584066f60 0x7fb584069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.564+0000 7fb596ffd640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb584066f60 0x7fb584069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.564+0000 7fb57f7fe640 1 -- 172.21.15.133:0/2701517860 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb58c0852c0 con 0x7fb5980ce330 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.564+0000 7fb596ffd640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb584066f60 0x7fb584069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5880ad000 tx=0x7fb588008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.568+0000 7fb57f7fe640 1 -- 172.21.15.133:0/2701517860 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb58c01f030 con 0x7fb5980ce330 2024-04-25T10:02:33.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.764+0000 7fb59d5ce640 1 -- 172.21.15.133:0/2701517860 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb5980cd750 con 0x7fb5980ce330 2024-04-25T10:02:33.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.764+0000 7fb57f7fe640 1 -- 172.21.15.133:0/2701517860 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb58c00d040 con 0x7fb5980ce330 2024-04-25T10:02:33.771 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:33.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.764+0000 7fb59d5ce640 1 -- 172.21.15.133:0/2701517860 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb584066f60 msgr2=0x7fb584069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.764+0000 7fb59d5ce640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb584066f60 0x7fb584069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5880ad000 tx=0x7fb588008040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.764+0000 7fb59d5ce640 1 -- 172.21.15.133:0/2701517860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5980ce330 msgr2=0x7fb5980c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.764+0000 7fb59d5ce640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5980ce330 0x7fb5980c7ce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00b440 tx=0x7fb58c00b900 comp rx=0 tx=0).stop 2024-04-25T10:02:33.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.768+0000 7fb59d5ce640 1 -- 172.21.15.133:0/2701517860 shutdown_connections 2024-04-25T10:02:33.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.768+0000 7fb59d5ce640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb59815c9a0 0x7fb598159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.768+0000 7fb59d5ce640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb584066f60 0x7fb584069420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.768+0000 7fb59d5ce640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5980ce330 0x7fb5980c7ce0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.768+0000 7fb59d5ce640 1 --2- 172.21.15.133:0/2701517860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5980b1f70 0x7fb598156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.768+0000 7fb59d5ce640 1 -- 172.21.15.133:0/2701517860 >> 172.21.15.133:0/2701517860 conn(0x7fb598067000 msgr2=0x7fb59815d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.768+0000 7fb59d5ce640 1 -- 172.21.15.133:0/2701517860 shutdown_connections 2024-04-25T10:02:33.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:33.768+0000 7fb59d5ce640 1 -- 172.21.15.133:0/2701517860 wait complete. 2024-04-25T10:02:33.791 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:33.791 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:33.791 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-04-25T10:02:33.791 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T10:02:33.791 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:33.791 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:33.791 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:33.791 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:33.791 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:02:33.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:33.791 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-04-25T10:02:33.798 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:33.798 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:33.799 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:33.799 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:33.799 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:33.800 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42487sec preferred_lft 42487sec 2024-04-25T10:02:33.801 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2024-04-25T10:02:33.801 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:33.801 INFO:teuthology.orchestra.run.smithi162.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:02:33.801 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 02:42:32:89:d0:d4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:33.801 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:02:33.801 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:33.801 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:02:33.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:33.801 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-04-25T10:02:33.801 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-25T10:02:33.802 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:02:33.802 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip addr flush dev ceph-brx 2024-04-25T10:02:33.802 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set ceph-brx up 2024-04-25T10:02:33.802 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:02:33.802 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-25T10:02:33.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:33.882 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:33.937 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-04-25T10:02:33.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:33.939 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-04-25T10:02:34.003 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-04-25T10:02:34.004 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:34.004 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:02:34.004 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:34.004 INFO:teuthology.orchestra.run.smithi162.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:02:34.004 INFO:teuthology.orchestra.run.smithi162.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:02:34.004 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:34.004 INFO:teuthology.orchestra.run.smithi162.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:34.004 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:34.004 INFO:teuthology.orchestra.run.smithi162.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:34.004 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:34.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:34.005 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-04-25T10:02:34.005 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-25T10:02:34.005 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:34.005 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:34.005 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:34.005 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-25T10:02:34.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:34.076 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-04-25T10:02:34.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:34.126 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list-id 2024-04-25T10:02:34.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:34.173 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-04-25T10:02:34.173 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-25T10:02:34.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:34.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T10:02:34.173 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-25T10:02:34.241 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T10:02:34.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:34.241 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-04-25T10:02:34.241 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-25T10:02:34.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T10:02:34.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:34.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T10:02:34.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T10:02:34.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T10:02:34.241 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-25T10:02:34.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:34.351 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-04-25T10:02:34.351 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-25T10:02:34.351 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.0 up 2024-04-25T10:02:34.351 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:34.351 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-25T10:02:34.370 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T10:02:34.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:34.370 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:34.416 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:02:34.416 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:34.417 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:34.464 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2024-04-25T10:02:34.526 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:34.578 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2024-04-25T10:02:34.578 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2024-04-25T10:02:34.578 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2024-04-25T10:02:34.578 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2024-04-25T10:02:34.578 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2024-04-25T10:02:34.578 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2024-04-25T10:02:34.578 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2024-04-25T10:02:34.578 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2024-04-25T10:02:34.578 INFO:teuthology.orchestra.run.smithi162.stdout:/run/lock 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/fs/nfsd 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:34.579 INFO:teuthology.orchestra.run.smithi162.stdout:/run/rpc_pipefs 2024-04-25T10:02:34.580 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2024-04-25T10:02:34.580 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2024-04-25T10:02:34.580 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:34.580 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:34.580 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:34.580 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:34.580 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2024-04-25T10:02:34.580 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:34.580 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:34.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:34.581 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2024-04-25T10:02:34.633 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:02:34.633 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T10:02:34.678 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2024-04-25T10:02:34.693 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:34.741 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2024-04-25T10:02:34.741 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2024-04-25T10:02:34.741 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2024-04-25T10:02:34.741 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2024-04-25T10:02:34.741 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2024-04-25T10:02:34.741 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/run/lock 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/fs/nfsd 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:34.742 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2024-04-25T10:02:34.743 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:34.743 INFO:teuthology.orchestra.run.smithi162.stdout:/run/rpc_pipefs 2024-04-25T10:02:34.743 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2024-04-25T10:02:34.743 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2024-04-25T10:02:34.743 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:34.743 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:34.743 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:34.743 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:34.743 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2024-04-25T10:02:34.743 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:34.743 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:34.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:34.744 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2024-04-25T10:02:34.758 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:2024-04-25T10:02:34.752+0000 7fea91c5e580 -1 init, newargv = 0x55d4f9b51f90 newargc=15 2024-04-25T10:02:34.758 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:ceph-fuse[33406]: starting ceph client 2024-04-25T10:02:34.765 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi162.stderr:ceph-fuse[33406]: starting fuse 2024-04-25T10:02:35.863 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2024-04-25T10:02:35.879 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:35.927 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2024-04-25T10:02:35.927 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2024-04-25T10:02:35.927 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2024-04-25T10:02:35.927 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2024-04-25T10:02:35.927 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/run/lock 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/fs/nfsd 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2024-04-25T10:02:35.928 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:/run/rpc_pipefs 2024-04-25T10:02:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2024-04-25T10:02:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2024-04-25T10:02:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2024-04-25T10:02:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:35.929 INFO:teuthology.orchestra.run.smithi162.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:35.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:35.930 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi162.stdout:45 2024-04-25T10:02:35.977 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:02:35.977 DEBUG:teuthology.orchestra.run.smithi162:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:35.977 DEBUG:teuthology.orchestra.run.smithi162:> import glob 2024-04-25T10:02:35.977 DEBUG:teuthology.orchestra.run.smithi162:> import re 2024-04-25T10:02:35.977 DEBUG:teuthology.orchestra.run.smithi162:> import os 2024-04-25T10:02:35.977 DEBUG:teuthology.orchestra.run.smithi162:> import subprocess 2024-04-25T10:02:35.977 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> def _find_admin_socket(client_name): 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> files = glob.glob(asok_path) 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> # Given a non-glob path, it better be there 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> if "*" not in asok_path: 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> assert(len(files) == 1) 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> return files[0] 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> for f in files: 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:35.978 DEBUG:teuthology.orchestra.run.smithi162:> contents = proc_f.read() 2024-04-25T10:02:35.979 DEBUG:teuthology.orchestra.run.smithi162:> if mountpoint in contents: 2024-04-25T10:02:35.979 DEBUG:teuthology.orchestra.run.smithi162:> return f 2024-04-25T10:02:35.979 DEBUG:teuthology.orchestra.run.smithi162:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:35.979 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-25T10:02:35.979 DEBUG:teuthology.orchestra.run.smithi162:> print(_find_admin_socket("client.1")) 2024-04-25T10:02:35.979 DEBUG:teuthology.orchestra.run.smithi162:> ' 2024-04-25T10:02:36.102 INFO:teuthology.orchestra.run.smithi162.stdout:/var/run/ceph/ceph-client.1.33406.asok 2024-04-25T10:02:36.106 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:02:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:36.115 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33406.asok 2024-04-25T10:02:36.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:36.115 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33406.asok status 2024-04-25T10:02:36.277 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-25T10:02:36.277 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": { 2024-04-25T10:02:36.277 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:36.277 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:36.277 INFO:teuthology.orchestra.run.smithi162.stdout: "entity_id": "1", 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: "hostname": "smithi162", 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: "pid": "33406", 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: "root": "/" 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_count": 0, 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 4556, 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: "inst": { 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: "name": { 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "client", 2024-04-25T10:02:36.278 INFO:teuthology.orchestra.run.smithi162.stdout: "num": 4556 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 3724178043 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 3724178043 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-25T10:02:36.279 INFO:teuthology.orchestra.run.smithi162.stdout: "inst_str": "client.4556 192.168.0.1:0/3724178043", 2024-04-25T10:02:36.280 INFO:teuthology.orchestra.run.smithi162.stdout: "addr_str": "192.168.0.1:0/3724178043", 2024-04-25T10:02:36.280 INFO:teuthology.orchestra.run.smithi162.stdout: "inode_count": 1, 2024-04-25T10:02:36.280 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_epoch": 9, 2024-04-25T10:02:36.280 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch": 26, 2024-04-25T10:02:36.280 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:36.280 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklisted": false, 2024-04-25T10:02:36.280 INFO:teuthology.orchestra.run.smithi162.stdout: "fs_name": "cephfs" 2024-04-25T10:02:36.280 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-25T10:02:36.291 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b2d8b640 1 Processor -- start 2024-04-25T10:02:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b2d8b640 1 -- start start 2024-04-25T10:02:36.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b2d8b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f20ac0ce330 0x7f20ac0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b2d8b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac0c61d0 con 0x7f20ac15c9a0 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b2d8b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac0c64b0 con 0x7f20ac0b1f70 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b2d8b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ac0cecc0 con 0x7f20ac0ce330 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b0b27640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f20ac15c9a0 0x7f20ac0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53628/0 (socket says 172.21.15.133:53628) 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b0b27640 1 -- 172.21.15.133:0/3733044337 learned_addr learned my addr 172.21.15.133:0/3733044337 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b1328640 1 --2- 172.21.15.133:0/3733044337 >> v2:172.21.15.162:3300/0 conn(0x7f20ac0ce330 0x7f20ac0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632881344 0 0) 0x7f20ac0c61d0 con 0x7f20ac15c9a0 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2094003530 con 0x7f20ac15c9a0 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1645133941 0 0) 0x7f2094003530 con 0x7f20ac15c9a0 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 >> v2:172.21.15.162:3300/0 conn(0x7f20ac0ce330 msgr2=0x7f20ac0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 --2- 172.21.15.133:0/3733044337 >> v2:172.21.15.162:3300/0 conn(0x7f20ac0ce330 0x7f20ac0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 >> v1:172.21.15.162:6789/0 conn(0x7f20ac0b1f70 legacy=0x7f20ac06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac06d4f0 con 0x7f20ac15c9a0 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f209c001680 con 0x7f20ac15c9a0 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 >> v1:172.21.15.133:6789/0 conn(0x7f20ac15c9a0 legacy=0x7f20ac0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 --2- 172.21.15.133:0/3733044337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20940039c0 0x7f2094003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 --2- 172.21.15.133:0/3733044337 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2094004810 0x7f2094004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 --2- 172.21.15.133:0/3733044337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20940051d0 0x7f209400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0cecc0 con 0x7f20940051d0 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209400c070 con 0x7f20940039c0 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209400c270 con 0x7f2094004810 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b1328640 1 --2- 172.21.15.133:0/3733044337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20940051d0 0x7f209400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20abfff640 1 --2- 172.21.15.133:0/3733044337 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2094004810 0x7f2094004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b0b27640 1 --2- 172.21.15.133:0/3733044337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20940039c0 0x7f2094003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b1328640 1 -- 172.21.15.133:0/3733044337 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2094004810 msgr2=0x7f2094004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b1328640 1 --2- 172.21.15.133:0/3733044337 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2094004810 0x7f2094004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b1328640 1 -- 172.21.15.133:0/3733044337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20940039c0 msgr2=0x7f2094003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b1328640 1 --2- 172.21.15.133:0/3733044337 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20940039c0 0x7f2094003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.416+0000 7f20b1328640 1 -- 172.21.15.133:0/3733044337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2094003530 con 0x7f20940051d0 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b1328640 1 -- 172.21.15.133:0/3733044337 reap_dead start 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b1328640 1 --2- 172.21.15.133:0/3733044337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20940051d0 0x7f209400b9d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009fa0 tx=0x7f20a401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828ae625cfee08c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a40b2070 con 0x7f20940051d0 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a40a5d50 con 0x7f20940051d0 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20ab7fe640 1 -- 172.21.15.133:0/3733044337 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4009bf0 con 0x7f20940051d0 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 -- 172.21.15.133:0/3733044337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20940051d0 msgr2=0x7f209400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 --2- 172.21.15.133:0/3733044337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20940051d0 0x7f209400b9d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009fa0 tx=0x7f20a401cc80 comp rx=0 tx=0).stop 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 -- 172.21.15.133:0/3733044337 shutdown_connections 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 --2- 172.21.15.133:0/3733044337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20940051d0 0x7f209400b9d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 -- 172.21.15.133:0/3733044337 >> 172.21.15.133:0/3733044337 conn(0x7f20ac067000 msgr2=0x7f20ac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 -- 172.21.15.133:0/3733044337 shutdown_connections 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 -- 172.21.15.133:0/3733044337 wait complete. 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 Processor -- start 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 -- start start 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20940051d0 0x7f20ac152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20ac136f20 0x7f20ac13d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ac13d640 0x7f20ac15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac15cf00 con 0x7f20ac136f20 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.420+0000 7f20b2d8b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac15cd80 con 0x7f20940051d0 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20b2d8b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0ce780 con 0x7f20ac13d640 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20ac136f20 0x7f20ac13d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20ac136f20 0x7f20ac13d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39574/0 (socket says 172.21.15.133:39574) 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20abfff640 1 -- 172.21.15.133:0/10324920 learned_addr learned my addr 172.21.15.133:0/10324920 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20b1328640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ac13d640 0x7f20ac15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20b0b27640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20940051d0 0x7f20ac152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20abfff640 1 -- 172.21.15.133:0/10324920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ac13d640 msgr2=0x7f20ac15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20abfff640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ac13d640 0x7f20ac15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20abfff640 1 -- 172.21.15.133:0/10324920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20940051d0 msgr2=0x7f20ac152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20abfff640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20940051d0 0x7f20ac152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20abfff640 1 -- 172.21.15.133:0/10324920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac15d0f0 con 0x7f20ac136f20 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20b0b27640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20940051d0 0x7f20ac152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20abfff640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20ac136f20 0x7f20ac13d100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f20a000e470 tx=0x7f20a000e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20a97fa640 1 -- 172.21.15.133:0/10324920 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00052e0 con 0x7f20ac136f20 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20a97fa640 1 -- 172.21.15.133:0/10324920 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a0018070 con 0x7f20ac136f20 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20a97fa640 1 -- 172.21.15.133:0/10324920 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0003820 con 0x7f20ac136f20 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20b2d8b640 1 -- 172.21.15.133:0/10324920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ac1376d0 con 0x7f20ac136f20 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20b2d8b640 1 -- 172.21.15.133:0/10324920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ac137c90 con 0x7f20ac136f20 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20b2d8b640 1 -- 172.21.15.133:0/10324920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ac068ef0 con 0x7f20ac136f20 2024-04-25T10:02:36.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20a97fa640 1 -- 172.21.15.133:0/10324920 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20a00039c0 con 0x7f20ac136f20 2024-04-25T10:02:36.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20a97fa640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2098066f90 0x7f2098069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20a97fa640 1 -- 172.21.15.133:0/10324920 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f20a0084fe0 con 0x7f20ac136f20 2024-04-25T10:02:36.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20b0b27640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2098066f90 0x7f2098069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.424+0000 7f20b0b27640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2098066f90 0x7f2098069450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f209c001210 tx=0x7f209c001160 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.428+0000 7f20a97fa640 1 -- 172.21.15.133:0/10324920 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20a0055c80 con 0x7f20ac136f20 2024-04-25T10:02:36.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 -- 172.21.15.133:0/10324920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f20ac13e2b0 con 0x7f20ac136f20 2024-04-25T10:02:36.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20a97fa640 1 -- 172.21.15.133:0/10324920 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f20a0005480 con 0x7f20ac136f20 2024-04-25T10:02:36.626 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:36.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 -- 172.21.15.133:0/10324920 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2098066f90 msgr2=0x7f2098069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2098066f90 0x7f2098069450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f209c001210 tx=0x7f209c001160 comp rx=0 tx=0).stop 2024-04-25T10:02:36.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 -- 172.21.15.133:0/10324920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20ac136f20 msgr2=0x7f20ac13d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20ac136f20 0x7f20ac13d100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f20a000e470 tx=0x7f20a000e930 comp rx=0 tx=0).stop 2024-04-25T10:02:36.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 -- 172.21.15.133:0/10324920 shutdown_connections 2024-04-25T10:02:36.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20ac13d640 0x7f20ac15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2098066f90 0x7f2098069450 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20ac136f20 0x7f20ac13d100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 --2- 172.21.15.133:0/10324920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20940051d0 0x7f20ac152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 -- 172.21.15.133:0/10324920 >> 172.21.15.133:0/10324920 conn(0x7f20ac067000 msgr2=0x7f20ac15b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 -- 172.21.15.133:0/10324920 shutdown_connections 2024-04-25T10:02:36.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:36.620+0000 7f20b2d8b640 1 -- 172.21.15.133:0/10324920 wait complete. 2024-04-25T10:02:36.643 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:36.643 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:36.643 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi162.front.sepia.ceph.com 2024-04-25T10:02:36.643 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T10:02:36.643 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:36.643 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:36.644 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:36.644 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:36.644 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:02:36.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:36.644 DEBUG:teuthology.orchestra.run.smithi162:> ip addr 2024-04-25T10:02:36.651 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:36.651 INFO:teuthology.orchestra.run.smithi162.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f0 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:d9:37:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: altname enp7s0f1 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:40 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0c:c4:7a:88:7f:41 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.21.15.162/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft 42484sec preferred_lft 42484sec 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::ec4:7aff:fe88:7f41/64 scope link 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 02:42:32:89:d0:d4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run.smithi162.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run.smithi162.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:36.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:36.653 DEBUG:teuthology.orchestra.run.smithi162:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:36.708 INFO:teuthology.orchestra.run.smithi162.stdout:1 2024-04-25T10:02:36.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:36.710 DEBUG:teuthology.orchestra.run.smithi162:> route 2024-04-25T10:02:36.766 INFO:teuthology.orchestra.run.smithi162.stdout:Kernel IP routing table 2024-04-25T10:02:36.766 INFO:teuthology.orchestra.run.smithi162.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:36.766 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:02:36.766 INFO:teuthology.orchestra.run.smithi162.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:36.766 INFO:teuthology.orchestra.run.smithi162.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:02:36.766 INFO:teuthology.orchestra.run.smithi162.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:02:36.766 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:36.766 INFO:teuthology.orchestra.run.smithi162.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:36.767 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:36.767 INFO:teuthology.orchestra.run.smithi162.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:36.767 INFO:teuthology.orchestra.run.smithi162.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:36.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:36.767 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-04-25T10:02:36.767 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-25T10:02:36.767 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:36.767 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:36.767 DEBUG:teuthology.orchestra.run.smithi162:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:36.767 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-25T10:02:36.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:36.844 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-04-25T10:02:36.897 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:02:36.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:36.898 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list-id 2024-04-25T10:02:36.945 INFO:teuthology.orchestra.run.smithi162.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T10:02:36.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:36.946 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-04-25T10:02:36.946 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-25T10:02:36.946 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:36.946 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T10:02:36.946 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-25T10:02:37.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:37.015 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:02:37.068 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:37.068 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:37.069 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:37.069 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:37.069 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:37.069 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:37.069 INFO:teuthology.orchestra.run.smithi162.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:37.069 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether ba:cc:eb:bf:c1:41 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:37.069 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:37.069 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:37.069 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::b8cc:ebff:febf:c141/64 scope link 2024-04-25T10:02:37.069 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:37.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:37.070 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:02:37.125 INFO:teuthology.orchestra.run.smithi162.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:37.125 INFO:teuthology.orchestra.run.smithi162.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:37.125 INFO:teuthology.orchestra.run.smithi162.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:37.125 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:37.125 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:37.126 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:37.126 INFO:teuthology.orchestra.run.smithi162.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:37.126 INFO:teuthology.orchestra.run.smithi162.stdout: link/ether ba:cc:eb:bf:c1:41 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:37.126 INFO:teuthology.orchestra.run.smithi162.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:37.126 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:37.126 INFO:teuthology.orchestra.run.smithi162.stdout: inet6 fe80::b8cc:ebff:febf:c141/64 scope link 2024-04-25T10:02:37.126 INFO:teuthology.orchestra.run.smithi162.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:37.127 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T10:02:37.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:37.127 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-04-25T10:02:37.127 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-25T10:02:37.127 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T10:02:37.127 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:37.127 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T10:02:37.128 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T10:02:37.128 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T10:02:37.128 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-25T10:02:37.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:37.221 DEBUG:teuthology.orchestra.run.smithi162:> (cd / && exec bash -c ' 2024-04-25T10:02:37.221 DEBUG:teuthology.orchestra.run.smithi162:> set -e 2024-04-25T10:02:37.221 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set brx.1 up 2024-04-25T10:02:37.221 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:02:37.221 DEBUG:teuthology.orchestra.run.smithi162:> ') 2024-04-25T10:02:37.240 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T10:02:37.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:37.241 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:37.288 INFO:teuthology.orchestra.run.smithi162.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:02:37.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:37.288 DEBUG:teuthology.orchestra.run.smithi162:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:37.336 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2024-04-25T10:02:37.391 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:37.438 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/run/lock 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:37.439 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/fs/nfsd 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/run/rpc_pipefs 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2024-04-25T10:02:37.440 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:37.441 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:37.441 INFO:teuthology.orchestra.run.smithi162.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:37.441 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:37.441 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:37.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:37.442 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2024-04-25T10:02:37.488 INFO:teuthology.orchestra.run.smithi162.stdout:45 2024-04-25T10:02:37.488 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T10:02:37.488 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T10:02:37.533 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2024-04-25T10:02:37.547 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/run/lock 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/fs/nfsd 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/run/rpc_pipefs 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:37.594 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:37.595 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:37.595 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2024-04-25T10:02:37.595 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:37.595 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:37.595 INFO:teuthology.orchestra.run.smithi162.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:37.595 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:37.595 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:37.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:37.596 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2024-04-25T10:02:37.609 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi162.stderr:2024-04-25T10:02:37.604+0000 7f510d684580 -1 init, newargv = 0x55beb47df830 newargc=15 2024-04-25T10:02:37.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi162.stderr:ceph-fuse[33522]: starting ceph client 2024-04-25T10:02:37.614 INFO:teuthology.orchestra.run.smithi162.stdout:45 2024-04-25T10:02:37.616 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi162.stderr:ceph-fuse[33522]: starting fuse 2024-04-25T10:02:38.716 DEBUG:teuthology.orchestra.run.smithi162:> sudo modprobe fuse 2024-04-25T10:02:38.733 DEBUG:teuthology.orchestra.run.smithi162:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:38.782 INFO:teuthology.orchestra.run.smithi162.stdout:/sys 2024-04-25T10:02:38.782 INFO:teuthology.orchestra.run.smithi162.stdout:/proc 2024-04-25T10:02:38.782 INFO:teuthology.orchestra.run.smithi162.stdout:/dev 2024-04-25T10:02:38.782 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/pts 2024-04-25T10:02:38.782 INFO:teuthology.orchestra.run.smithi162.stdout:/run 2024-04-25T10:02:38.782 INFO:teuthology.orchestra.run.smithi162.stdout:/ 2024-04-25T10:02:38.782 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/security 2024-04-25T10:02:38.782 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/shm 2024-04-25T10:02:38.782 INFO:teuthology.orchestra.run.smithi162.stdout:/run/lock 2024-04-25T10:02:38.782 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/cgroup 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/pstore 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/bpf 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/hugepages 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/dev/mqueue 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/debug 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/tracing 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/proc/fs/nfsd 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/sys/kernel/config 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/run/rpc_pipefs 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/run/user/1000 2024-04-25T10:02:38.783 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph 2024-04-25T10:02:38.784 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:38.784 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:38.784 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:38.784 INFO:teuthology.orchestra.run.smithi162.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:38.784 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns 2024-04-25T10:02:38.784 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:38.784 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:38.784 INFO:teuthology.orchestra.run.smithi162.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:38.784 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:38.784 INFO:teuthology.orchestra.run.smithi162.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:38.784 INFO:teuthology.orchestra.run.smithi162.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:38.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:38.785 DEBUG:teuthology.orchestra.run.smithi162:> ls /sys/fs/fuse/connections 2024-04-25T10:02:38.833 INFO:teuthology.orchestra.run.smithi162.stdout:45 2024-04-25T10:02:38.833 INFO:teuthology.orchestra.run.smithi162.stdout:46 2024-04-25T10:02:38.833 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T10:02:38.833 DEBUG:teuthology.orchestra.run.smithi162:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:38.833 DEBUG:teuthology.orchestra.run.smithi162:> import glob 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> import re 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> import os 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> import subprocess 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> def _find_admin_socket(client_name): 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> files = glob.glob(asok_path) 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> # Given a non-glob path, it better be there 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> if "*" not in asok_path: 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> assert(len(files) == 1) 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> return files[0] 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> for f in files: 2024-04-25T10:02:38.834 DEBUG:teuthology.orchestra.run.smithi162:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:38.835 DEBUG:teuthology.orchestra.run.smithi162:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:38.835 DEBUG:teuthology.orchestra.run.smithi162:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:38.835 DEBUG:teuthology.orchestra.run.smithi162:> contents = proc_f.read() 2024-04-25T10:02:38.835 DEBUG:teuthology.orchestra.run.smithi162:> if mountpoint in contents: 2024-04-25T10:02:38.835 DEBUG:teuthology.orchestra.run.smithi162:> return f 2024-04-25T10:02:38.835 DEBUG:teuthology.orchestra.run.smithi162:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:38.835 DEBUG:teuthology.orchestra.run.smithi162:> 2024-04-25T10:02:38.835 DEBUG:teuthology.orchestra.run.smithi162:> print(_find_admin_socket("client.3")) 2024-04-25T10:02:38.835 DEBUG:teuthology.orchestra.run.smithi162:> ' 2024-04-25T10:02:38.957 INFO:teuthology.orchestra.run.smithi162.stdout:/var/run/ceph/ceph-client.3.33522.asok 2024-04-25T10:02:38.961 INFO:teuthology.orchestra.run.smithi162.stderr:2024-04-25T10:02:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:38.968 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33522.asok 2024-04-25T10:02:38.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:38.969 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33522.asok status 2024-04-25T10:02:39.118 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "metadata": { 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "entity_id": "3", 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "hostname": "smithi162", 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "pid": "33522", 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "root": "/" 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_count": 0, 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "id": 4651, 2024-04-25T10:02:39.119 INFO:teuthology.orchestra.run.smithi162.stdout: "inst": { 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "name": { 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "client", 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "num": 4651 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 3588560128 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": { 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:02:39.120 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 3588560128 2024-04-25T10:02:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-04-25T10:02:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "inst_str": "client.4651 192.168.0.2:0/3588560128", 2024-04-25T10:02:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "addr_str": "192.168.0.2:0/3588560128", 2024-04-25T10:02:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "inode_count": 1, 2024-04-25T10:02:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_epoch": 9, 2024-04-25T10:02:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch": 26, 2024-04-25T10:02:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklisted": false, 2024-04-25T10:02:39.121 INFO:teuthology.orchestra.run.smithi162.stdout: "fs_name": "cephfs" 2024-04-25T10:02:39.121 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-04-25T10:02:39.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.132 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:02:39.132 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:39.140 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:02:39.140 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:39.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.140 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:39.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.195 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:02:39.195 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:39.242 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:02:39.243 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:39.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.243 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:39.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.299 DEBUG:teuthology.orchestra.run.smithi162:> stat --file-system '--printf=%T 2024-04-25T10:02:39.299 DEBUG:teuthology.orchestra.run.smithi162:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:39.306 INFO:teuthology.orchestra.run.smithi162.stdout:fuseblk 2024-04-25T10:02:39.306 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:39.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.306 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:39.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.361 DEBUG:teuthology.orchestra.run.smithi162:> stat --file-system '--printf=%T 2024-04-25T10:02:39.361 DEBUG:teuthology.orchestra.run.smithi162:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:39.410 INFO:teuthology.orchestra.run.smithi162.stdout:fuseblk 2024-04-25T10:02:39.410 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:39.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.410 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:39.468 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T10:02:39.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:39.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.604+0000 7ff60b397640 1 Processor -- start 2024-04-25T10:02:39.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff60b397640 1 -- start start 2024-04-25T10:02:39.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff60b397640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff60415a9d0 0x7ff6040c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff60b397640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6040cc5a0 con 0x7ff60406eb70 2024-04-25T10:02:39.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff60b397640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6040cc880 con 0x7ff60415a5f0 2024-04-25T10:02:39.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff60b397640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6040ccb60 con 0x7ff60415a9d0 2024-04-25T10:02:39.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff608932640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff60406eb70 0x7ff6040cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53640/0 (socket says 172.21.15.133:53640) 2024-04-25T10:02:39.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff608932640 1 -- 172.21.15.133:0/565683596 learned_addr learned my addr 172.21.15.133:0/565683596 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:39.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626116768 0 0) 0x7ff6040cc5a0 con 0x7ff60406eb70 2024-04-25T10:02:39.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff609934640 1 --2- 172.21.15.133:0/565683596 >> v2:172.21.15.162:3300/0 conn(0x7ff60415a9d0 0x7ff6040c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f0005740 con 0x7ff60406eb70 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1138244236 0 0) 0x7ff5f0005740 con 0x7ff60406eb70 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 >> v2:172.21.15.162:3300/0 conn(0x7ff60415a9d0 msgr2=0x7ff6040c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 --2- 172.21.15.133:0/565683596 >> v2:172.21.15.162:3300/0 conn(0x7ff60415a9d0 0x7ff6040c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 >> v1:172.21.15.162:6789/0 conn(0x7ff60415a5f0 legacy=0x7ff60406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6040cce40 con 0x7ff60406eb70 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff5f4001680 con 0x7ff60406eb70 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 >> v1:172.21.15.133:6789/0 conn(0x7ff60406eb70 legacy=0x7ff6040cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5f0005bd0 0x7ff5f0005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5f0006a20 0x7ff5f0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5f00073e0 0x7ff5f000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040ccb60 con 0x7ff5f0006a20 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f000e280 con 0x7ff5f0005bd0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f000e480 con 0x7ff5f00073e0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff608932640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5f0006a20 0x7ff5f0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff609133640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5f0005bd0 0x7ff5f0005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff609934640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5f00073e0 0x7ff5f000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff608932640 1 -- 172.21.15.133:0/565683596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5f00073e0 msgr2=0x7ff5f000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff608932640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5f00073e0 0x7ff5f000dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff608932640 1 -- 172.21.15.133:0/565683596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5f0005bd0 msgr2=0x7ff5f0005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff608932640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5f0005bd0 0x7ff5f0005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff608932640 1 -- 172.21.15.133:0/565683596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f0005740 con 0x7ff5f0006a20 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff609934640 1 -- 172.21.15.133:0/565683596 reap_dead start 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.608+0000 7ff609133640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5f0005bd0 0x7ff5f0005ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff608932640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5f0006a20 0x7ff5f0006ea0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400c1c0 tx=0x7ff5f401f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce0fbbc39dd08ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4007d00 con 0x7ff5f0006a20 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f40ac020 con 0x7ff5f0006a20 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff5fbfff640 1 -- 172.21.15.133:0/565683596 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f40058e0 con 0x7ff5f0006a20 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 -- 172.21.15.133:0/565683596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5f0006a20 msgr2=0x7ff5f0006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5f0006a20 0x7ff5f0006ea0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400c1c0 tx=0x7ff5f401f2d0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 -- 172.21.15.133:0/565683596 shutdown_connections 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 --2- 172.21.15.133:0/565683596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5f0006a20 0x7ff5f0006ea0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 -- 172.21.15.133:0/565683596 >> 172.21.15.133:0/565683596 conn(0x7ff604067000 msgr2=0x7ff60406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 -- 172.21.15.133:0/565683596 shutdown_connections 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 -- 172.21.15.133:0/565683596 wait complete. 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 Processor -- start 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 -- start start 2024-04-25T10:02:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5f0006a20 0x7ff60414f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff60413d860 0x7ff604152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6041400c0 0x7ff604140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040c55f0 con 0x7ff60413d860 2024-04-25T10:02:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040c5470 con 0x7ff5f0006a20 2024-04-25T10:02:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff60b397640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040cbe50 con 0x7ff6041400c0 2024-04-25T10:02:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff608932640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff60413d860 0x7ff604152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff608932640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff60413d860 0x7ff604152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39580/0 (socket says 172.21.15.133:39580) 2024-04-25T10:02:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.612+0000 7ff608932640 1 -- 172.21.15.133:0/3230602390 learned_addr learned my addr 172.21.15.133:0/3230602390 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:39.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff609133640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5f0006a20 0x7ff60414f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff609934640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6041400c0 0x7ff604140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff608932640 1 -- 172.21.15.133:0/3230602390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6041400c0 msgr2=0x7ff604140540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff608932640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6041400c0 0x7ff604140540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff608932640 1 -- 172.21.15.133:0/3230602390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5f0006a20 msgr2=0x7ff60414f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff608932640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5f0006a20 0x7ff60414f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff608932640 1 -- 172.21.15.133:0/3230602390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60414fe60 con 0x7ff60413d860 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff608932640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff60413d860 0x7ff604152850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40ae040 tx=0x7ff5f0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff5f9ffb640 1 -- 172.21.15.133:0/3230602390 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4001c40 con 0x7ff60413d860 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff5f9ffb640 1 -- 172.21.15.133:0/3230602390 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f40049f0 con 0x7ff60413d860 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff60b397640 1 -- 172.21.15.133:0/3230602390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff604155010 con 0x7ff60413d860 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff5f9ffb640 1 -- 172.21.15.133:0/3230602390 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4004de0 con 0x7ff60413d860 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff60b397640 1 -- 172.21.15.133:0/3230602390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6041555d0 con 0x7ff60413d860 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff60b397640 1 -- 172.21.15.133:0/3230602390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff604068ef0 con 0x7ff60413d860 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff5f9ffb640 1 -- 172.21.15.133:0/3230602390 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5f40070c0 con 0x7ff60413d860 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff5f9ffb640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5e8066f60 0x7ff5e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff609133640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5e8066f60 0x7ff5e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff5f9ffb640 1 -- 172.21.15.133:0/3230602390 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff5f41221f0 con 0x7ff60413d860 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.616+0000 7ff609133640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5e8066f60 0x7ff5e8069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff600007fd0 tx=0x7ff600009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.620+0000 7ff5f9ffb640 1 -- 172.21.15.133:0/3230602390 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5f4018fe0 con 0x7ff60413d860 2024-04-25T10:02:39.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.780+0000 7ff60b397640 1 -- 172.21.15.133:0/3230602390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6040cf3d0 con 0x7ff60413d860 2024-04-25T10:02:39.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.784+0000 7ff5f9ffb640 1 -- 172.21.15.133:0/3230602390 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7ff5f40f2e00 con 0x7ff60413d860 2024-04-25T10:02:39.792 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:39.792 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:39.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 -- 172.21.15.133:0/3230602390 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5e8066f60 msgr2=0x7ff5e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5e8066f60 0x7ff5e8069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff600007fd0 tx=0x7ff600009040 comp rx=0 tx=0).stop 2024-04-25T10:02:39.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 -- 172.21.15.133:0/3230602390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff60413d860 msgr2=0x7ff604152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff60413d860 0x7ff604152850 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40ae040 tx=0x7ff5f0007a50 comp rx=0 tx=0).stop 2024-04-25T10:02:39.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 -- 172.21.15.133:0/3230602390 shutdown_connections 2024-04-25T10:02:39.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6041400c0 0x7ff604140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5e8066f60 0x7ff5e8069420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff60413d860 0x7ff604152850 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 --2- 172.21.15.133:0/3230602390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5f0006a20 0x7ff60414f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 -- 172.21.15.133:0/3230602390 >> 172.21.15.133:0/3230602390 conn(0x7ff604067000 msgr2=0x7ff6040ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 -- 172.21.15.133:0/3230602390 shutdown_connections 2024-04-25T10:02:39.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.788+0000 7ff60b397640 1 -- 172.21.15.133:0/3230602390 wait complete. 2024-04-25T10:02:39.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:39.812 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:39.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c83d8640 1 Processor -- start 2024-04-25T10:02:39.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c83d8640 1 -- start start 2024-04-25T10:02:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c83d8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87c015eb40 0x7f87c00b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c83d8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f87c00aebc0 0x7f87c00af010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c83d8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c01471f0 con 0x7f87c006e8b0 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c83d8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c00af550 con 0x7f87c015eb40 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c83d8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c0147490 con 0x7f87c00aebc0 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c5973640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87c015eb40 0x7f87c00b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c6174640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f87c006e8b0 0x7f87c0146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53650/0 (socket says 172.21.15.133:53650) 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c5973640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87c015eb40 0x7f87c00b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39582/0 (socket says 172.21.15.133:39582) 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c6174640 1 -- 172.21.15.133:0/53567544 learned_addr learned my addr 172.21.15.133:0/53567544 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c6975640 1 --2- 172.21.15.133:0/53567544 >> v2:172.21.15.162:3300/0 conn(0x7f87c00aebc0 0x7f87c00af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c5973640 1 -- 172.21.15.133:0/53567544 >> v2:172.21.15.162:3300/0 conn(0x7f87c00aebc0 msgr2=0x7f87c00af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c5973640 1 --2- 172.21.15.133:0/53567544 >> v2:172.21.15.162:3300/0 conn(0x7f87c00aebc0 0x7f87c00af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c5973640 1 -- 172.21.15.133:0/53567544 >> v1:172.21.15.133:6789/0 conn(0x7f87c006e8b0 legacy=0x7f87c0146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c5973640 1 -- 172.21.15.133:0/53567544 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c00b1090 con 0x7f87c015eb40 2024-04-25T10:02:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.932+0000 7f87c6975640 1 --2- 172.21.15.133:0/53567544 >> v2:172.21.15.162:3300/0 conn(0x7f87c00aebc0 0x7f87c00af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:39.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c5973640 1 --2- 172.21.15.133:0/53567544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c015eb40 0x7f87c00b1e50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f87c00b27f0 tx=0x7f87b001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fd26ed281c427a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c5172640 1 -- 172.21.15.133:0/53567544 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00b0070 con 0x7f87c015eb40 2024-04-25T10:02:39.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c5172640 1 -- 172.21.15.133:0/53567544 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b00ab600 con 0x7f87c015eb40 2024-04-25T10:02:39.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c5172640 1 -- 172.21.15.133:0/53567544 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00ab9d0 con 0x7f87c015eb40 2024-04-25T10:02:39.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c83d8640 1 -- 172.21.15.133:0/53567544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c015eb40 msgr2=0x7f87c00b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c83d8640 1 --2- 172.21.15.133:0/53567544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c015eb40 0x7f87c00b1e50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f87c00b27f0 tx=0x7f87b001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c83d8640 1 -- 172.21.15.133:0/53567544 shutdown_connections 2024-04-25T10:02:39.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c83d8640 1 --2- 172.21.15.133:0/53567544 >> v2:172.21.15.162:3300/0 conn(0x7f87c00aebc0 0x7f87c00af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c83d8640 1 --2- 172.21.15.133:0/53567544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c015eb40 0x7f87c00b1e50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c83d8640 1 -- 172.21.15.133:0/53567544 >> 172.21.15.133:0/53567544 conn(0x7f87c0066f70 msgr2=0x7f87c006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c83d8640 1 -- 172.21.15.133:0/53567544 shutdown_connections 2024-04-25T10:02:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.936+0000 7f87c83d8640 1 -- 172.21.15.133:0/53567544 wait complete. 2024-04-25T10:02:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c83d8640 1 Processor -- start 2024-04-25T10:02:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c83d8640 1 -- start start 2024-04-25T10:02:39.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c83d8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87c006e8b0 0x7f87c015cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c83d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c00aebc0 0x7f87c013ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c83d8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87c015eb40 0x7f87c0139c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c83d8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c006de40 con 0x7f87c00aebc0 2024-04-25T10:02:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c83d8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c006dcc0 con 0x7f87c006e8b0 2024-04-25T10:02:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c83d8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00b2390 con 0x7f87c015eb40 2024-04-25T10:02:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c5973640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c00aebc0 0x7f87c013ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c5973640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c00aebc0 0x7f87c013ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39588/0 (socket says 172.21.15.133:39588) 2024-04-25T10:02:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c5973640 1 -- 172.21.15.133:0/3934417631 learned_addr learned my addr 172.21.15.133:0/3934417631 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c6174640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87c006e8b0 0x7f87c015cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c5973640 1 -- 172.21.15.133:0/3934417631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87c015eb40 msgr2=0x7f87c0139c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c5973640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87c015eb40 0x7f87c0139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c5973640 1 -- 172.21.15.133:0/3934417631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87c006e8b0 msgr2=0x7f87c015cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c5973640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87c006e8b0 0x7f87c015cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c5973640 1 -- 172.21.15.133:0/3934417631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c015d530 con 0x7f87c00aebc0 2024-04-25T10:02:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c6174640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87c006e8b0 0x7f87c015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c5973640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c00aebc0 0x7f87c013ec20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f87b001a020 tx=0x7f87b0019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87b6ffd640 1 -- 172.21.15.133:0/3934417631 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00ba070 con 0x7f87c00aebc0 2024-04-25T10:02:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87b6ffd640 1 -- 172.21.15.133:0/3934417631 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b00b5680 con 0x7f87c00aebc0 2024-04-25T10:02:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c83d8640 1 -- 172.21.15.133:0/3934417631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c013c0d0 con 0x7f87c00aebc0 2024-04-25T10:02:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87b6ffd640 1 -- 172.21.15.133:0/3934417631 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00b0040 con 0x7f87c00aebc0 2024-04-25T10:02:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c83d8640 1 -- 172.21.15.133:0/3934417631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c013c690 con 0x7f87c00aebc0 2024-04-25T10:02:39.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87c83d8640 1 -- 172.21.15.133:0/3934417631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c0068e80 con 0x7f87c00aebc0 2024-04-25T10:02:39.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.940+0000 7f87b6ffd640 1 -- 172.21.15.133:0/3934417631 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87b00bd070 con 0x7f87c00aebc0 2024-04-25T10:02:39.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.944+0000 7f87b6ffd640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87a4066f60 0x7f87a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.944+0000 7f87b6ffd640 1 -- 172.21.15.133:0/3934417631 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f87b011f990 con 0x7f87c00aebc0 2024-04-25T10:02:39.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.944+0000 7f87c6174640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87a4066f60 0x7f87a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.944+0000 7f87c6174640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87a4066f60 0x7f87a4069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f87bc005890 tx=0x7f87bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:39.948+0000 7f87b6ffd640 1 -- 172.21.15.133:0/3934417631 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87b0016070 con 0x7f87c00aebc0 2024-04-25T10:02:40.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87c83d8640 1 -- 172.21.15.133:0/3934417631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87c006b960 con 0x7f87c00aebc0 2024-04-25T10:02:40.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87b6ffd640 1 -- 172.21.15.133:0/3934417631 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87b00f0630 con 0x7f87c00aebc0 2024-04-25T10:02:40.107 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:40.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87c83d8640 1 -- 172.21.15.133:0/3934417631 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87a4066f60 msgr2=0x7f87a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87c83d8640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87a4066f60 0x7f87a4069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f87bc005890 tx=0x7f87bc009040 comp rx=0 tx=0).stop 2024-04-25T10:02:40.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87c83d8640 1 -- 172.21.15.133:0/3934417631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c00aebc0 msgr2=0x7f87c013ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87c83d8640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c00aebc0 0x7f87c013ec20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f87b001a020 tx=0x7f87b0019190 comp rx=0 tx=0).stop 2024-04-25T10:02:40.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87c83d8640 1 -- 172.21.15.133:0/3934417631 shutdown_connections 2024-04-25T10:02:40.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87c83d8640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87c015eb40 0x7f87c0139c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87c83d8640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87a4066f60 0x7f87a4069420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87c83d8640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c00aebc0 0x7f87c013ec20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87c83d8640 1 --2- 172.21.15.133:0/3934417631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87c006e8b0 0x7f87c015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.100+0000 7f87c83d8640 1 -- 172.21.15.133:0/3934417631 >> 172.21.15.133:0/3934417631 conn(0x7f87c0066f70 msgr2=0x7f87c013fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.104+0000 7f87c83d8640 1 -- 172.21.15.133:0/3934417631 shutdown_connections 2024-04-25T10:02:40.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.104+0000 7f87c83d8640 1 -- 172.21.15.133:0/3934417631 wait complete. 2024-04-25T10:02:40.125 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:40.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.236+0000 7fc253f11640 1 Processor -- start 2024-04-25T10:02:40.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.236+0000 7fc253f11640 1 -- start start 2024-04-25T10:02:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc253f11640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc24c15caa0 0x7fc24c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc253f11640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc24c06b1d0 0x7fc24c15f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc253f11640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c06b800 con 0x7fc24c0cc420 2024-04-25T10:02:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc253f11640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c159890 con 0x7fc24c06b1d0 2024-04-25T10:02:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc253f11640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c06bae0 con 0x7fc24c15caa0 2024-04-25T10:02:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc2524ae640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc24c06b1d0 0x7fc24c15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc251cad640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc24c0cc420 0x7fc24c0cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53666/0 (socket says 172.21.15.133:53666) 2024-04-25T10:02:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc251cad640 1 -- 172.21.15.133:0/2742715623 learned_addr learned my addr 172.21.15.133:0/2742715623 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc2514ac640 1 --2- 172.21.15.133:0/2742715623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc24c15caa0 0x7fc24c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc2524ae640 1 -- 172.21.15.133:0/2742715623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc24c15caa0 msgr2=0x7fc24c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc2524ae640 1 --2- 172.21.15.133:0/2742715623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc24c15caa0 0x7fc24c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc2524ae640 1 -- 172.21.15.133:0/2742715623 >> v1:172.21.15.133:6789/0 conn(0x7fc24c0cc420 legacy=0x7fc24c0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc2524ae640 1 -- 172.21.15.133:0/2742715623 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c15bfc0 con 0x7fc24c06b1d0 2024-04-25T10:02:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc250cab640 1 -- 172.21.15.133:0/2742715623 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535673961 0 0) 0x7fc24c06b800 con 0x7fc24c0cc420 2024-04-25T10:02:40.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc2524ae640 1 --2- 172.21.15.133:0/2742715623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24c06b1d0 0x7fc24c15f410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc244009d90 tx=0x7fc24401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ef72d732fe42a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc250cab640 1 -- 172.21.15.133:0/2742715623 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2440b2070 con 0x7fc24c06b1d0 2024-04-25T10:02:40.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc250cab640 1 -- 172.21.15.133:0/2742715623 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2440a5830 con 0x7fc24c06b1d0 2024-04-25T10:02:40.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc250cab640 1 -- 172.21.15.133:0/2742715623 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2440a5c00 con 0x7fc24c06b1d0 2024-04-25T10:02:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc253f11640 1 -- 172.21.15.133:0/2742715623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24c06b1d0 msgr2=0x7fc24c15f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.240+0000 7fc253f11640 1 --2- 172.21.15.133:0/2742715623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24c06b1d0 0x7fc24c15f410 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc244009d90 tx=0x7fc24401c970 comp rx=0 tx=0).stop 2024-04-25T10:02:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 -- 172.21.15.133:0/2742715623 shutdown_connections 2024-04-25T10:02:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 --2- 172.21.15.133:0/2742715623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24c06b1d0 0x7fc24c15f410 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 --2- 172.21.15.133:0/2742715623 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc24c15caa0 0x7fc24c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 -- 172.21.15.133:0/2742715623 >> 172.21.15.133:0/2742715623 conn(0x7fc24c0999d0 msgr2=0x7fc24c0d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 -- 172.21.15.133:0/2742715623 shutdown_connections 2024-04-25T10:02:40.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 -- 172.21.15.133:0/2742715623 wait complete. 2024-04-25T10:02:40.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 Processor -- start 2024-04-25T10:02:40.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 -- start start 2024-04-25T10:02:40.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24c06b1d0 0x7fc24c157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc24c0cc420 0x7fc24c144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc24c15caa0 0x7fc24c155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c168b40 con 0x7fc24c06b1d0 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c1689c0 con 0x7fc24c0cc420 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c168cc0 con 0x7fc24c15caa0 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc251cad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24c06b1d0 0x7fc24c157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc251cad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24c06b1d0 0x7fc24c157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39606/0 (socket says 172.21.15.133:39606) 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc251cad640 1 -- 172.21.15.133:0/1587957473 learned_addr learned my addr 172.21.15.133:0/1587957473 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc251cad640 1 -- 172.21.15.133:0/1587957473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc24c15caa0 msgr2=0x7fc24c155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc251cad640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc24c15caa0 0x7fc24c155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc251cad640 1 -- 172.21.15.133:0/1587957473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc24c0cc420 msgr2=0x7fc24c144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc251cad640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc24c0cc420 0x7fc24c144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc251cad640 1 -- 172.21.15.133:0/1587957473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c1454b0 con 0x7fc24c06b1d0 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc251cad640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24c06b1d0 0x7fc24c157e50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c520 tx=0x7fc24800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc2427fc640 1 -- 172.21.15.133:0/1587957473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248018070 con 0x7fc24c06b1d0 2024-04-25T10:02:40.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc2427fc640 1 -- 172.21.15.133:0/1587957473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2480074e0 con 0x7fc24c06b1d0 2024-04-25T10:02:40.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc2427fc640 1 -- 172.21.15.133:0/1587957473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2480049f0 con 0x7fc24c06b1d0 2024-04-25T10:02:40.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 -- 172.21.15.133:0/1587957473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24c145790 con 0x7fc24c06b1d0 2024-04-25T10:02:40.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.244+0000 7fc253f11640 1 -- 172.21.15.133:0/1587957473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c15bb30 con 0x7fc24c06b1d0 2024-04-25T10:02:40.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.248+0000 7fc253f11640 1 -- 172.21.15.133:0/1587957473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24c142980 con 0x7fc24c06b1d0 2024-04-25T10:02:40.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.248+0000 7fc2427fc640 1 -- 172.21.15.133:0/1587957473 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc24801b070 con 0x7fc24c06b1d0 2024-04-25T10:02:40.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.248+0000 7fc2427fc640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc230066f60 0x7fc230069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.248+0000 7fc2427fc640 1 -- 172.21.15.133:0/1587957473 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc248084860 con 0x7fc24c06b1d0 2024-04-25T10:02:40.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.248+0000 7fc2514ac640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc230066f60 0x7fc230069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.248+0000 7fc2514ac640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc230066f60 0x7fc230069420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0cc900 tx=0x7fc23c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.252+0000 7fc2427fc640 1 -- 172.21.15.133:0/1587957473 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2480554e0 con 0x7fc24c06b1d0 2024-04-25T10:02:40.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 -- 172.21.15.133:0/1587957473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc24c099f20 con 0x7fc24c06b1d0 2024-04-25T10:02:40.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc2427fc640 1 -- 172.21.15.133:0/1587957473 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc248054de0 con 0x7fc24c06b1d0 2024-04-25T10:02:40.418 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:40.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 -- 172.21.15.133:0/1587957473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc230066f60 msgr2=0x7fc230069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc230066f60 0x7fc230069420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0cc900 tx=0x7fc23c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:40.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 -- 172.21.15.133:0/1587957473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24c06b1d0 msgr2=0x7fc24c157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24c06b1d0 0x7fc24c157e50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c520 tx=0x7fc24800c9e0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 -- 172.21.15.133:0/1587957473 shutdown_connections 2024-04-25T10:02:40.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc24c15caa0 0x7fc24c155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc24c0cc420 0x7fc24c144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc230066f60 0x7fc230069420 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 --2- 172.21.15.133:0/1587957473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc24c06b1d0 0x7fc24c157e50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 -- 172.21.15.133:0/1587957473 >> 172.21.15.133:0/1587957473 conn(0x7fc24c0999d0 msgr2=0x7fc24c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 -- 172.21.15.133:0/1587957473 shutdown_connections 2024-04-25T10:02:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.412+0000 7fc253f11640 1 -- 172.21.15.133:0/1587957473 wait complete. 2024-04-25T10:02:40.436 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:40.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f8690781640 1 Processor -- start 2024-04-25T10:02:40.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f8690781640 1 -- start start 2024-04-25T10:02:40.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f8690781640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f868815ea40 0x7f86880b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f8690781640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868815f1b0 con 0x7f86880cf240 2024-04-25T10:02:40.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f8690781640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86880ce480 con 0x7f868815deb0 2024-04-25T10:02:40.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f8690781640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86880ce230 con 0x7f868815ea40 2024-04-25T10:02:40.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f868ed1e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f868815ea40 0x7f86880b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f868dd1c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f86880cf240 0x7f868815e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53670/0 (socket says 172.21.15.133:53670) 2024-04-25T10:02:40.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f868dd1c640 1 -- 172.21.15.133:0/3724876395 learned_addr learned my addr 172.21.15.133:0/3724876395 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:40.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f868ed1e640 1 -- 172.21.15.133:0/3724876395 >> v1:172.21.15.162:6789/0 conn(0x7f868815deb0 legacy=0x7f86880cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f868ed1e640 1 -- 172.21.15.133:0/3724876395 >> v1:172.21.15.133:6789/0 conn(0x7f86880cf240 legacy=0x7f868815e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f868ed1e640 1 -- 172.21.15.133:0/3724876395 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86880ce760 con 0x7f868815ea40 2024-04-25T10:02:40.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.556+0000 7f868d51b640 1 -- 172.21.15.133:0/3724876395 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564610152 0 0) 0x7f868815f1b0 con 0x7f86880cf240 2024-04-25T10:02:40.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868ed1e640 1 --2- 172.21.15.133:0/3724876395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868815ea40 0x7f86880b3410 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8680009d90 tx=0x7f868001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7534f4c7fe227f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868d51b640 1 -- 172.21.15.133:0/3724876395 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86800b2070 con 0x7f868815ea40 2024-04-25T10:02:40.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868d51b640 1 -- 172.21.15.133:0/3724876395 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86800a5870 con 0x7f868815ea40 2024-04-25T10:02:40.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868d51b640 1 -- 172.21.15.133:0/3724876395 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86800a5c40 con 0x7f868815ea40 2024-04-25T10:02:40.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 -- 172.21.15.133:0/3724876395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868815ea40 msgr2=0x7f86880b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 --2- 172.21.15.133:0/3724876395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868815ea40 0x7f86880b3410 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8680009d90 tx=0x7f868001c8a0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 -- 172.21.15.133:0/3724876395 shutdown_connections 2024-04-25T10:02:40.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 --2- 172.21.15.133:0/3724876395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868815ea40 0x7f86880b3410 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 -- 172.21.15.133:0/3724876395 >> 172.21.15.133:0/3724876395 conn(0x7f86880999d0 msgr2=0x7f86880d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 -- 172.21.15.133:0/3724876395 shutdown_connections 2024-04-25T10:02:40.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 -- 172.21.15.133:0/3724876395 wait complete. 2024-04-25T10:02:40.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 Processor -- start 2024-04-25T10:02:40.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 -- start start 2024-04-25T10:02:40.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86880cf240 0x7f868815d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868815deb0 0x7f868815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868815ea40 0x7f8688150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86880cc570 con 0x7f868815deb0 2024-04-25T10:02:40.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86880cedb0 con 0x7f86880cf240 2024-04-25T10:02:40.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f8690781640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86881688f0 con 0x7f868815ea40 2024-04-25T10:02:40.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868dd1c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868815deb0 0x7f868815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868dd1c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868815deb0 0x7f868815ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39612/0 (socket says 172.21.15.133:39612) 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868dd1c640 1 -- 172.21.15.133:0/2725135950 learned_addr learned my addr 172.21.15.133:0/2725135950 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868ed1e640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868815ea40 0x7f8688150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868e51d640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86880cf240 0x7f868815d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868dd1c640 1 -- 172.21.15.133:0/2725135950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868815ea40 msgr2=0x7f8688150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868dd1c640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868815ea40 0x7f8688150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868dd1c640 1 -- 172.21.15.133:0/2725135950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86880cf240 msgr2=0x7f868815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868dd1c640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86880cf240 0x7f868815d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868dd1c640 1 -- 172.21.15.133:0/2725135950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868815c110 con 0x7f868815deb0 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868ed1e640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868815ea40 0x7f8688150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.560+0000 7f868dd1c640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868815deb0 0x7f868815ba80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f867800c520 tx=0x7f867800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.564+0000 7f867effd640 1 -- 172.21.15.133:0/2725135950 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678018070 con 0x7f868815deb0 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.564+0000 7f867effd640 1 -- 172.21.15.133:0/2725135950 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86780074e0 con 0x7f868815deb0 2024-04-25T10:02:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.564+0000 7f867effd640 1 -- 172.21.15.133:0/2725135950 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780049f0 con 0x7f868815deb0 2024-04-25T10:02:40.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.564+0000 7f8690781640 1 -- 172.21.15.133:0/2725135950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868815c540 con 0x7f868815deb0 2024-04-25T10:02:40.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.564+0000 7f8690781640 1 -- 172.21.15.133:0/2725135950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868815cac0 con 0x7f868815deb0 2024-04-25T10:02:40.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.564+0000 7f8690781640 1 -- 172.21.15.133:0/2725135950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8688067c70 con 0x7f868815deb0 2024-04-25T10:02:40.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.564+0000 7f867effd640 1 -- 172.21.15.133:0/2725135950 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f867801b070 con 0x7f868815deb0 2024-04-25T10:02:40.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.564+0000 7f867effd640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f866c066f60 0x7f866c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.564+0000 7f867effd640 1 -- 172.21.15.133:0/2725135950 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8678084840 con 0x7f868815deb0 2024-04-25T10:02:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.564+0000 7f868e51d640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f866c066f60 0x7f866c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.564+0000 7f868e51d640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f866c066f60 0x7f866c069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f86880cdf10 tx=0x7f8684009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.568+0000 7f867effd640 1 -- 172.21.15.133:0/2725135950 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86780554e0 con 0x7f868815deb0 2024-04-25T10:02:40.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.724+0000 7f8690781640 1 -- 172.21.15.133:0/2725135950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86880ccdb0 con 0x7f868815deb0 2024-04-25T10:02:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.724+0000 7f867effd640 1 -- 172.21.15.133:0/2725135950 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8678054de0 con 0x7f868815deb0 2024-04-25T10:02:40.733 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 -- 172.21.15.133:0/2725135950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f866c066f60 msgr2=0x7f866c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f866c066f60 0x7f866c069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f86880cdf10 tx=0x7f8684009040 comp rx=0 tx=0).stop 2024-04-25T10:02:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 -- 172.21.15.133:0/2725135950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868815deb0 msgr2=0x7f868815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868815deb0 0x7f868815ba80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f867800c520 tx=0x7f867800c9e0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 -- 172.21.15.133:0/2725135950 shutdown_connections 2024-04-25T10:02:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f868815ea40 0x7f8688150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f866c066f60 0x7f866c069420 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868815deb0 0x7f868815ba80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 --2- 172.21.15.133:0/2725135950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86880cf240 0x7f868815d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 -- 172.21.15.133:0/2725135950 >> 172.21.15.133:0/2725135950 conn(0x7f86880999d0 msgr2=0x7f86880b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 -- 172.21.15.133:0/2725135950 shutdown_connections 2024-04-25T10:02:40.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.728+0000 7f8690781640 1 -- 172.21.15.133:0/2725135950 wait complete. 2024-04-25T10:02:40.752 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:40.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd09a8a8640 1 Processor -- start 2024-04-25T10:02:40.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd09a8a8640 1 -- start start 2024-04-25T10:02:40.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd09a8a8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd09415dab0 0x7fd0940b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd09a8a8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0940b72d0 con 0x7fd09415d6d0 2024-04-25T10:02:40.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd09a8a8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0940b75b0 con 0x7fd0940b9590 2024-04-25T10:02:40.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd09a8a8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0940b7890 con 0x7fd09415dab0 2024-04-25T10:02:40.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd093fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd09415d6d0 0x7fd0940b6880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53682/0 (socket says 172.21.15.133:53682) 2024-04-25T10:02:40.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd093fff640 1 -- 172.21.15.133:0/3742771585 learned_addr learned my addr 172.21.15.133:0/3742771585 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:40.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd098e45640 1 --2- 172.21.15.133:0/3742771585 >> v2:172.21.15.162:3300/0 conn(0x7fd09415dab0 0x7fd0940b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155187045 0 0) 0x7fd0940b72d0 con 0x7fd09415d6d0 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd07c003530 con 0x7fd09415d6d0 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 608394027 0 0) 0x7fd07c003530 con 0x7fd09415d6d0 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 >> v2:172.21.15.162:3300/0 conn(0x7fd09415dab0 msgr2=0x7fd0940b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 --2- 172.21.15.133:0/3742771585 >> v2:172.21.15.162:3300/0 conn(0x7fd09415dab0 0x7fd0940b8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 >> v1:172.21.15.162:6789/0 conn(0x7fd0940b9590 legacy=0x7fd0940d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940b7b70 con 0x7fd09415d6d0 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fd088001680 con 0x7fd09415d6d0 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 >> v1:172.21.15.133:6789/0 conn(0x7fd09415d6d0 legacy=0x7fd0940b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 --2- 172.21.15.133:0/3742771585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c0039c0 0x7fd07c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 --2- 172.21.15.133:0/3742771585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07c004810 0x7fd07c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 --2- 172.21.15.133:0/3742771585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c0051d0 0x7fd07c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940b7890 con 0x7fd07c004810 2024-04-25T10:02:40.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c00c080 con 0x7fd07c0051d0 2024-04-25T10:02:40.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c00c280 con 0x7fd07c0039c0 2024-04-25T10:02:40.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd0937fe640 1 --2- 172.21.15.133:0/3742771585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07c004810 0x7fd07c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd098e45640 1 --2- 172.21.15.133:0/3742771585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c0051d0 0x7fd07c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd0937fe640 1 -- 172.21.15.133:0/3742771585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c0039c0 msgr2=0x7fd07c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd0937fe640 1 --2- 172.21.15.133:0/3742771585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07c0039c0 0x7fd07c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd0937fe640 1 -- 172.21.15.133:0/3742771585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c0051d0 msgr2=0x7fd07c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd0937fe640 1 --2- 172.21.15.133:0/3742771585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c0051d0 0x7fd07c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd0937fe640 1 -- 172.21.15.133:0/3742771585 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c003530 con 0x7fd07c004810 2024-04-25T10:02:40.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd098e45640 1 --2- 172.21.15.133:0/3742771585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c0051d0 0x7fd07c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:40.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.868+0000 7fd098e45640 1 -- 172.21.15.133:0/3742771585 reap_dead start 2024-04-25T10:02:40.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd0937fe640 1 --2- 172.21.15.133:0/3742771585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07c004810 0x7fd07c004c90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd084001780 tx=0x7fd08401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672907c7d0d9083a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840013a0 con 0x7fd07c004810 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0840052f0 con 0x7fd07c004810 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd092ffd640 1 -- 172.21.15.133:0/3742771585 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084003840 con 0x7fd07c004810 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd09a8a8640 1 -- 172.21.15.133:0/3742771585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07c004810 msgr2=0x7fd07c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd09a8a8640 1 --2- 172.21.15.133:0/3742771585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07c004810 0x7fd07c004c90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd084001780 tx=0x7fd08401c400 comp rx=0 tx=0).stop 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd09a8a8640 1 -- 172.21.15.133:0/3742771585 shutdown_connections 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd09a8a8640 1 --2- 172.21.15.133:0/3742771585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07c004810 0x7fd07c004c90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd09a8a8640 1 -- 172.21.15.133:0/3742771585 >> 172.21.15.133:0/3742771585 conn(0x7fd0940727c0 msgr2=0x7fd094072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd09a8a8640 1 -- 172.21.15.133:0/3742771585 shutdown_connections 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd09a8a8640 1 -- 172.21.15.133:0/3742771585 wait complete. 2024-04-25T10:02:40.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd09a8a8640 1 Processor -- start 2024-04-25T10:02:40.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.872+0000 7fd09a8a8640 1 -- start start 2024-04-25T10:02:40.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd09a8a8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c004810 0x7fd0940c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd09a8a8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd094156df0 0x7fd0940ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd09a8a8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0940cae20 0x7fd0940d3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd09a8a8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940b6a00 con 0x7fd094156df0 2024-04-25T10:02:40.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd09a8a8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940b6880 con 0x7fd07c004810 2024-04-25T10:02:40.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd09a8a8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940b6b80 con 0x7fd0940cae20 2024-04-25T10:02:40.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd0937fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd094156df0 0x7fd0940ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd0937fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd094156df0 0x7fd0940ca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39624/0 (socket says 172.21.15.133:39624) 2024-04-25T10:02:40.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd0937fe640 1 -- 172.21.15.133:0/14848098 learned_addr learned my addr 172.21.15.133:0/14848098 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd093fff640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c004810 0x7fd0940c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd098e45640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0940cae20 0x7fd0940d3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd0937fe640 1 -- 172.21.15.133:0/14848098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0940cae20 msgr2=0x7fd0940d3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd0937fe640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0940cae20 0x7fd0940d3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd0937fe640 1 -- 172.21.15.133:0/14848098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c004810 msgr2=0x7fd0940c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd0937fe640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c004810 0x7fd0940c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd0937fe640 1 -- 172.21.15.133:0/14848098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940d2f20 con 0x7fd094156df0 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd0937fe640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd094156df0 0x7fd0940ca8e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd08401c230 tx=0x7fd07c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd090ff9640 1 -- 172.21.15.133:0/14848098 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084007a40 con 0x7fd094156df0 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd090ff9640 1 -- 172.21.15.133:0/14848098 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0840ba070 con 0x7fd094156df0 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd090ff9640 1 -- 172.21.15.133:0/14848098 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084007250 con 0x7fd094156df0 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd09a8a8640 1 -- 172.21.15.133:0/14848098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0940d3200 con 0x7fd094156df0 2024-04-25T10:02:40.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd09a8a8640 1 -- 172.21.15.133:0/14848098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0940d37c0 con 0x7fd094156df0 2024-04-25T10:02:40.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd09a8a8640 1 -- 172.21.15.133:0/14848098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd094068e80 con 0x7fd094156df0 2024-04-25T10:02:40.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd090ff9640 1 -- 172.21.15.133:0/14848098 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0840052f0 con 0x7fd094156df0 2024-04-25T10:02:40.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd090ff9640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd080066f90 0x7fd080069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd090ff9640 1 -- 172.21.15.133:0/14848098 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd08411f7e0 con 0x7fd094156df0 2024-04-25T10:02:40.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.876+0000 7fd093fff640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd080066f90 0x7fd080069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.880+0000 7fd093fff640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd080066f90 0x7fd080069450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd0880011f0 tx=0x7fd088001580 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:40.884+0000 7fd090ff9640 1 -- 172.21.15.133:0/14848098 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd084016110 con 0x7fd094156df0 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.040+0000 7fd09a8a8640 1 -- 172.21.15.133:0/14848098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd0940c7610 con 0x7fd094156df0 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.040+0000 7fd090ff9640 1 -- 172.21.15.133:0/14848098 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd0840f0400 con 0x7fd094156df0 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:41.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 -- 172.21.15.133:0/14848098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd080066f90 msgr2=0x7fd080069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd080066f90 0x7fd080069450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd0880011f0 tx=0x7fd088001580 comp rx=0 tx=0).stop 2024-04-25T10:02:41.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 -- 172.21.15.133:0/14848098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd094156df0 msgr2=0x7fd0940ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd094156df0 0x7fd0940ca8e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd08401c230 tx=0x7fd07c005840 comp rx=0 tx=0).stop 2024-04-25T10:02:41.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 -- 172.21.15.133:0/14848098 shutdown_connections 2024-04-25T10:02:41.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0940cae20 0x7fd0940d3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd080066f90 0x7fd080069450 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd094156df0 0x7fd0940ca8e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 --2- 172.21.15.133:0/14848098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07c004810 0x7fd0940c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 -- 172.21.15.133:0/14848098 >> 172.21.15.133:0/14848098 conn(0x7fd0940727c0 msgr2=0x7fd0941601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 -- 172.21.15.133:0/14848098 shutdown_connections 2024-04-25T10:02:41.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.044+0000 7fd09a8a8640 1 -- 172.21.15.133:0/14848098 wait complete. 2024-04-25T10:02:41.067 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:41.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.172+0000 7f2059430640 1 Processor -- start 2024-04-25T10:02:41.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.172+0000 7f2059430640 1 -- start start 2024-04-25T10:02:41.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f2059430640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f205415db00 0x7f20540b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f2059430640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f20540d0bd0 0x7f2054161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f2059430640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20540ceb20 con 0x7f20540d0770 2024-04-25T10:02:41.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f2059430640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20540b7740 con 0x7f20540d0bd0 2024-04-25T10:02:41.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f2059430640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20540b7a20 con 0x7f205415db00 2024-04-25T10:02:41.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f20537fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f20540d0bd0 0x7f2054161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f2052ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f205415db00 0x7f20540b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f204a7fc640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f20540d0770 0x7f20541614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:44570/0 (socket says 172.21.15.133:44570) 2024-04-25T10:02:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f204a7fc640 1 -- 172.21.15.133:0/1640279447 learned_addr learned my addr 172.21.15.133:0/1640279447 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f2052ffd640 1 -- 172.21.15.133:0/1640279447 >> v2:172.21.15.162:3300/0 conn(0x7f20540d0bd0 msgr2=0x7f2054161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f2052ffd640 1 --2- 172.21.15.133:0/1640279447 >> v2:172.21.15.162:3300/0 conn(0x7f20540d0bd0 0x7f2054161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f2052ffd640 1 -- 172.21.15.133:0/1640279447 >> v1:172.21.15.162:6789/0 conn(0x7f20540d0770 legacy=0x7f20541614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f2052ffd640 1 -- 172.21.15.133:0/1640279447 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20540b97f0 con 0x7f205415db00 2024-04-25T10:02:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.176+0000 7f20537fe640 1 --2- 172.21.15.133:0/1640279447 >> v2:172.21.15.162:3300/0 conn(0x7f20540d0bd0 0x7f2054161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f2052ffd640 1 --2- 172.21.15.133:0/1640279447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f205415db00 0x7f20540b2990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f20440097c0 tx=0x7f204401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9ad216d442a8465 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f20527fc640 1 -- 172.21.15.133:0/1640279447 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20440a7070 con 0x7f205415db00 2024-04-25T10:02:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f20527fc640 1 -- 172.21.15.133:0/1640279447 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20440b0070 con 0x7f205415db00 2024-04-25T10:02:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f20527fc640 1 -- 172.21.15.133:0/1640279447 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20440ab890 con 0x7f205415db00 2024-04-25T10:02:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f2059430640 1 -- 172.21.15.133:0/1640279447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f205415db00 msgr2=0x7f20540b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f2059430640 1 --2- 172.21.15.133:0/1640279447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f205415db00 0x7f20540b2990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f20440097c0 tx=0x7f204401c470 comp rx=0 tx=0).stop 2024-04-25T10:02:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f2059430640 1 -- 172.21.15.133:0/1640279447 shutdown_connections 2024-04-25T10:02:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f2059430640 1 --2- 172.21.15.133:0/1640279447 >> v2:172.21.15.162:3300/0 conn(0x7f20540d0bd0 0x7f2054161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f2059430640 1 --2- 172.21.15.133:0/1640279447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f205415db00 0x7f20540b2990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f2059430640 1 -- 172.21.15.133:0/1640279447 >> 172.21.15.133:0/1640279447 conn(0x7f20540727c0 msgr2=0x7f2054072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f2059430640 1 -- 172.21.15.133:0/1640279447 shutdown_connections 2024-04-25T10:02:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f2059430640 1 -- 172.21.15.133:0/1640279447 wait complete. 2024-04-25T10:02:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f2059430640 1 Processor -- start 2024-04-25T10:02:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.180+0000 7f2059430640 1 -- start start 2024-04-25T10:02:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2059430640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20540d0770 0x7f205415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2059430640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20540d0bd0 0x7f205414d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2059430640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f205415db00 0x7f20540c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2059430640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20540d0fe0 con 0x7f205415db00 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2059430640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20540b7430 con 0x7f20540d0770 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2059430640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20540d18a0 con 0x7f20540d0bd0 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f20537fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f205415db00 0x7f20540c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f20537fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f205415db00 0x7f20540c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39640/0 (socket says 172.21.15.133:39640) 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f20537fe640 1 -- 172.21.15.133:0/3235600726 learned_addr learned my addr 172.21.15.133:0/3235600726 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2052ffd640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20540d0770 0x7f205415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f204a7fc640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20540d0bd0 0x7f205414d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f20537fe640 1 -- 172.21.15.133:0/3235600726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20540d0bd0 msgr2=0x7f205414d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f20537fe640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20540d0bd0 0x7f205414d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f20537fe640 1 -- 172.21.15.133:0/3235600726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20540d0770 msgr2=0x7f205415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f20537fe640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20540d0770 0x7f205415dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f20537fe640 1 -- 172.21.15.133:0/3235600726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205414eda0 con 0x7f205415db00 2024-04-25T10:02:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f20537fe640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f205415db00 0x7f20540c97c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f204c00efc0 tx=0x7f204c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f204bfff640 1 -- 172.21.15.133:0/3235600726 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c018030 con 0x7f205415db00 2024-04-25T10:02:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f204bfff640 1 -- 172.21.15.133:0/3235600726 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c00cde0 con 0x7f205415db00 2024-04-25T10:02:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2059430640 1 -- 172.21.15.133:0/3235600726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205414f080 con 0x7f205415db00 2024-04-25T10:02:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f204bfff640 1 -- 172.21.15.133:0/3235600726 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c003bc0 con 0x7f205415db00 2024-04-25T10:02:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2059430640 1 -- 172.21.15.133:0/3235600726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205414def0 con 0x7f205415db00 2024-04-25T10:02:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2059430640 1 -- 172.21.15.133:0/3235600726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205414f530 con 0x7f205415db00 2024-04-25T10:02:41.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f204bfff640 1 -- 172.21.15.133:0/3235600726 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f204c01b070 con 0x7f205415db00 2024-04-25T10:02:41.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f204bfff640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2034066f60 0x7f2034069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f204bfff640 1 -- 172.21.15.133:0/3235600726 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f204c085eb0 con 0x7f205415db00 2024-04-25T10:02:41.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2052ffd640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2034066f60 0x7f2034069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.184+0000 7f2052ffd640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2034066f60 0x7f2034069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f20440af000 tx=0x7f20440a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.188+0000 7f204bfff640 1 -- 172.21.15.133:0/3235600726 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f204c010040 con 0x7f205415db00 2024-04-25T10:02:41.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.340+0000 7f2059430640 1 -- 172.21.15.133:0/3235600726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2054068ef0 con 0x7f205415db00 2024-04-25T10:02:41.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.344+0000 7f204bfff640 1 -- 172.21.15.133:0/3235600726 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f204c056ad0 con 0x7f205415db00 2024-04-25T10:02:41.350 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:41.350 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:41.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.344+0000 7f2059430640 1 -- 172.21.15.133:0/3235600726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2034066f60 msgr2=0x7f2034069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.344+0000 7f2059430640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2034066f60 0x7f2034069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f20440af000 tx=0x7f20440a7000 comp rx=0 tx=0).stop 2024-04-25T10:02:41.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.344+0000 7f2059430640 1 -- 172.21.15.133:0/3235600726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f205415db00 msgr2=0x7f20540c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.344+0000 7f2059430640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f205415db00 0x7f20540c97c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f204c00efc0 tx=0x7f204c009f90 comp rx=0 tx=0).stop 2024-04-25T10:02:41.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.348+0000 7f2059430640 1 -- 172.21.15.133:0/3235600726 shutdown_connections 2024-04-25T10:02:41.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.348+0000 7f2059430640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2034066f60 0x7f2034069420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.348+0000 7f2059430640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f205415db00 0x7f20540c97c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.348+0000 7f2059430640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20540d0bd0 0x7f205414d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.348+0000 7f2059430640 1 --2- 172.21.15.133:0/3235600726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20540d0770 0x7f205415dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.348+0000 7f2059430640 1 -- 172.21.15.133:0/3235600726 >> 172.21.15.133:0/3235600726 conn(0x7f20540727c0 msgr2=0x7f20541679d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.348+0000 7f2059430640 1 -- 172.21.15.133:0/3235600726 shutdown_connections 2024-04-25T10:02:41.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.348+0000 7f2059430640 1 -- 172.21.15.133:0/3235600726 wait complete. 2024-04-25T10:02:41.369 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:41.370 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:41.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.480+0000 7ff18ca93640 1 Processor -- start 2024-04-25T10:02:41.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.480+0000 7ff18ca93640 1 -- start start 2024-04-25T10:02:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18ca93640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff188145170 0x7ff18806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18ca93640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1880ce2e0 con 0x7ff18815ced0 2024-04-25T10:02:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18ca93640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18806d4b0 con 0x7ff18806e4d0 2024-04-25T10:02:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18ca93640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff18806db00 con 0x7ff188145170 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff186d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff18815ced0 0x7ff1880b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53692/0 (socket says 172.21.15.133:53692) 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff186d9d640 1 -- 172.21.15.133:0/1477571647 learned_addr learned my addr 172.21.15.133:0/1477571647 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477980011 0 0) 0x7ff1880ce2e0 con 0x7ff18815ced0 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff170003530 con 0x7ff18815ced0 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18659c640 1 --2- 172.21.15.133:0/1477571647 >> v2:172.21.15.162:3300/0 conn(0x7ff188145170 0x7ff18806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1049881035 0 0) 0x7ff170003530 con 0x7ff18815ced0 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 >> v2:172.21.15.162:3300/0 conn(0x7ff188145170 msgr2=0x7ff18806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 --2- 172.21.15.133:0/1477571647 >> v2:172.21.15.162:3300/0 conn(0x7ff188145170 0x7ff18806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 >> v1:172.21.15.162:6789/0 conn(0x7ff18806e4d0 legacy=0x7ff1880cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18806d790 con 0x7ff18815ced0 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff180001a00 con 0x7ff18815ced0 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 >> v1:172.21.15.133:6789/0 conn(0x7ff18815ced0 legacy=0x7ff1880b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 --2- 172.21.15.133:0/1477571647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1700039c0 0x7ff170003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 --2- 172.21.15.133:0/1477571647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff170004810 0x7ff170004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 --2- 172.21.15.133:0/1477571647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1700051d0 0x7ff17000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18806db00 con 0x7ff1700051d0 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17000c070 con 0x7ff1700039c0 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17000c270 con 0x7ff170004810 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff186d9d640 1 --2- 172.21.15.133:0/1477571647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1700051d0 0x7ff17000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff18659c640 1 --2- 172.21.15.133:0/1477571647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1700039c0 0x7ff170003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff185d9b640 1 --2- 172.21.15.133:0/1477571647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff170004810 0x7ff170004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff186d9d640 1 -- 172.21.15.133:0/1477571647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff170004810 msgr2=0x7ff170004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff186d9d640 1 --2- 172.21.15.133:0/1477571647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff170004810 0x7ff170004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff186d9d640 1 -- 172.21.15.133:0/1477571647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1700039c0 msgr2=0x7ff170003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff186d9d640 1 --2- 172.21.15.133:0/1477571647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1700039c0 0x7ff170003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff186d9d640 1 -- 172.21.15.133:0/1477571647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff170003530 con 0x7ff1700051d0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff186d9d640 1 -- 172.21.15.133:0/1477571647 reap_dead start 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.484+0000 7ff186d9d640 1 --2- 172.21.15.133:0/1477571647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1700051d0 0x7ff17000b9d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff18000a850 tx=0x7ff18001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcbf009b8c94394 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18000a4a0 con 0x7ff1700051d0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1800ad650 con 0x7ff1700051d0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18559a640 1 -- 172.21.15.133:0/1477571647 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1800ada40 con 0x7ff1700051d0 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 -- 172.21.15.133:0/1477571647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1700051d0 msgr2=0x7ff17000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 --2- 172.21.15.133:0/1477571647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1700051d0 0x7ff17000b9d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff18000a850 tx=0x7ff18001d3a0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 -- 172.21.15.133:0/1477571647 shutdown_connections 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 --2- 172.21.15.133:0/1477571647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1700051d0 0x7ff17000b9d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 -- 172.21.15.133:0/1477571647 >> 172.21.15.133:0/1477571647 conn(0x7ff188066f70 msgr2=0x7ff18806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 -- 172.21.15.133:0/1477571647 shutdown_connections 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 -- 172.21.15.133:0/1477571647 wait complete. 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 Processor -- start 2024-04-25T10:02:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 -- start start 2024-04-25T10:02:41.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1700051d0 0x7ff1881519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18813ecb0 0x7ff18813f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff18813b190 0x7ff18813b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18806df90 con 0x7ff18813ecb0 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff188145550 con 0x7ff18813b190 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18806e110 con 0x7ff1700051d0 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff185d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18813ecb0 0x7ff18813f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff185d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18813ecb0 0x7ff18813f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39652/0 (socket says 172.21.15.133:39652) 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff185d9b640 1 -- 172.21.15.133:0/2380364609 learned_addr learned my addr 172.21.15.133:0/2380364609 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18659c640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1700051d0 0x7ff1881519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff186d9d640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff18813b190 0x7ff18813b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff185d9b640 1 -- 172.21.15.133:0/2380364609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1700051d0 msgr2=0x7ff1881519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff185d9b640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1700051d0 0x7ff1881519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff185d9b640 1 -- 172.21.15.133:0/2380364609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff18813b190 msgr2=0x7ff18813b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff185d9b640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff18813b190 0x7ff18813b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff185d9b640 1 -- 172.21.15.133:0/2380364609 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff188151f30 con 0x7ff18813ecb0 2024-04-25T10:02:41.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff186d9d640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff18813b190 0x7ff18813b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff185d9b640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18813ecb0 0x7ff18813f130 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00e440 tx=0x7ff17c00e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff176ffd640 1 -- 172.21.15.133:0/2380364609 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c019070 con 0x7ff18813ecb0 2024-04-25T10:02:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff176ffd640 1 -- 172.21.15.133:0/2380364609 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c0052e0 con 0x7ff18813ecb0 2024-04-25T10:02:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 -- 172.21.15.133:0/2380364609 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18813a0f0 con 0x7ff18813ecb0 2024-04-25T10:02:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff176ffd640 1 -- 172.21.15.133:0/2380364609 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c003820 con 0x7ff18813ecb0 2024-04-25T10:02:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.488+0000 7ff18ca93640 1 -- 172.21.15.133:0/2380364609 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18813a6b0 con 0x7ff18813ecb0 2024-04-25T10:02:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.492+0000 7ff18ca93640 1 -- 172.21.15.133:0/2380364609 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff188068e80 con 0x7ff18813ecb0 2024-04-25T10:02:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.492+0000 7ff176ffd640 1 -- 172.21.15.133:0/2380364609 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff17c01c070 con 0x7ff18813ecb0 2024-04-25T10:02:41.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.492+0000 7ff176ffd640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f90 0x7ff168069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.492+0000 7ff18659c640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f90 0x7ff168069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.492+0000 7ff176ffd640 1 -- 172.21.15.133:0/2380364609 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff17c0848e0 con 0x7ff18813ecb0 2024-04-25T10:02:41.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.492+0000 7ff18659c640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f90 0x7ff168069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff178009a60 tx=0x7ff178008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.496+0000 7ff176ffd640 1 -- 172.21.15.133:0/2380364609 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff17c054ea0 con 0x7ff18813ecb0 2024-04-25T10:02:41.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.652+0000 7ff18ca93640 1 -- 172.21.15.133:0/2380364609 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff188143c80 con 0x7ff18813ecb0 2024-04-25T10:02:41.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.652+0000 7ff176ffd640 1 -- 172.21.15.133:0/2380364609 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff17c058f40 con 0x7ff18813ecb0 2024-04-25T10:02:41.659 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 -- 172.21.15.133:0/2380364609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f90 msgr2=0x7ff168069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f90 0x7ff168069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff178009a60 tx=0x7ff178008040 comp rx=0 tx=0).stop 2024-04-25T10:02:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 -- 172.21.15.133:0/2380364609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18813ecb0 msgr2=0x7ff18813f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18813ecb0 0x7ff18813f130 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00e440 tx=0x7ff17c00e840 comp rx=0 tx=0).stop 2024-04-25T10:02:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 -- 172.21.15.133:0/2380364609 shutdown_connections 2024-04-25T10:02:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff18813b190 0x7ff18813b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f90 0x7ff168069450 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18813ecb0 0x7ff18813f130 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 --2- 172.21.15.133:0/2380364609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1700051d0 0x7ff1881519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 -- 172.21.15.133:0/2380364609 >> 172.21.15.133:0/2380364609 conn(0x7ff188066f70 msgr2=0x7ff188154e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 -- 172.21.15.133:0/2380364609 shutdown_connections 2024-04-25T10:02:41.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.656+0000 7ff18ca93640 1 -- 172.21.15.133:0/2380364609 wait complete. 2024-04-25T10:02:41.678 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:41.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd120507640 1 Processor -- start 2024-04-25T10:02:41.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd120507640 1 -- start start 2024-04-25T10:02:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd120507640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd11815d9a0 0x7fd1180d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd120507640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd1180ce600 0x7fd1180cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd120507640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1180d1920 0x7fd1180d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd120507640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1180cefa0 con 0x7fd1180d1920 2024-04-25T10:02:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd120507640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd11815ebd0 con 0x7fd1180ce600 2024-04-25T10:02:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd120507640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11815eeb0 con 0x7fd11815d9a0 2024-04-25T10:02:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11eaa4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1180d1920 0x7fd1180d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11eaa4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1180d1920 0x7fd1180d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39654/0 (socket says 172.21.15.133:39654) 2024-04-25T10:02:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11eaa4640 1 -- 172.21.15.133:0/530858741 learned_addr learned my addr 172.21.15.133:0/530858741 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11daa2640 1 --2- 172.21.15.133:0/530858741 >> v2:172.21.15.162:3300/0 conn(0x7fd1180ce600 0x7fd1180cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11e2a3640 1 --2- 172.21.15.133:0/530858741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd11815d9a0 0x7fd1180d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11eaa4640 1 -- 172.21.15.133:0/530858741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd11815d9a0 msgr2=0x7fd1180d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11eaa4640 1 --2- 172.21.15.133:0/530858741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd11815d9a0 0x7fd1180d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11eaa4640 1 -- 172.21.15.133:0/530858741 >> v2:172.21.15.162:3300/0 conn(0x7fd1180ce600 msgr2=0x7fd1180cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:41.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11eaa4640 1 --2- 172.21.15.133:0/530858741 >> v2:172.21.15.162:3300/0 conn(0x7fd1180ce600 0x7fd1180cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11eaa4640 1 -- 172.21.15.133:0/530858741 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1180d0330 con 0x7fd1180d1920 2024-04-25T10:02:41.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11e2a3640 1 --2- 172.21.15.133:0/530858741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd11815d9a0 0x7fd1180d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:41.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.796+0000 7fd11daa2640 1 --2- 172.21.15.133:0/530858741 >> v2:172.21.15.162:3300/0 conn(0x7fd1180ce600 0x7fd1180cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:41.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd11eaa4640 1 --2- 172.21.15.133:0/530858741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1180d1920 0x7fd1180d1db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd1180b6420 tx=0x7fd11001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71d3655c143346a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd11d2a1640 1 -- 172.21.15.133:0/530858741 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1100a5790 con 0x7fd1180d1920 2024-04-25T10:02:41.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd11d2a1640 1 -- 172.21.15.133:0/530858741 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1100a5930 con 0x7fd1180d1920 2024-04-25T10:02:41.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd11d2a1640 1 -- 172.21.15.133:0/530858741 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1100a5d00 con 0x7fd1180d1920 2024-04-25T10:02:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd120507640 1 -- 172.21.15.133:0/530858741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1180d1920 msgr2=0x7fd1180d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd120507640 1 --2- 172.21.15.133:0/530858741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1180d1920 0x7fd1180d1db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd1180b6420 tx=0x7fd11001c930 comp rx=0 tx=0).stop 2024-04-25T10:02:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd120507640 1 -- 172.21.15.133:0/530858741 shutdown_connections 2024-04-25T10:02:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd120507640 1 --2- 172.21.15.133:0/530858741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1180d1920 0x7fd1180d1db0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd120507640 1 --2- 172.21.15.133:0/530858741 >> v2:172.21.15.162:3300/0 conn(0x7fd1180ce600 0x7fd1180cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd120507640 1 --2- 172.21.15.133:0/530858741 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd11815d9a0 0x7fd1180d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd120507640 1 -- 172.21.15.133:0/530858741 >> 172.21.15.133:0/530858741 conn(0x7fd11806bf80 msgr2=0x7fd11806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd120507640 1 -- 172.21.15.133:0/530858741 shutdown_connections 2024-04-25T10:02:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd120507640 1 -- 172.21.15.133:0/530858741 wait complete. 2024-04-25T10:02:41.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd120507640 1 Processor -- start 2024-04-25T10:02:41.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.800+0000 7fd120507640 1 -- start start 2024-04-25T10:02:41.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd120507640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1180ce600 0x7fd118154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd120507640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1180d1920 0x7fd118157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd120507640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd11815d9a0 0x7fd1181579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd120507640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180b9820 con 0x7fd1180d1920 2024-04-25T10:02:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd120507640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180b96a0 con 0x7fd11815d9a0 2024-04-25T10:02:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd120507640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180d22f0 con 0x7fd1180ce600 2024-04-25T10:02:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11daa2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1180d1920 0x7fd118157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11daa2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1180d1920 0x7fd118157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39656/0 (socket says 172.21.15.133:39656) 2024-04-25T10:02:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11daa2640 1 -- 172.21.15.133:0/548312895 learned_addr learned my addr 172.21.15.133:0/548312895 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11eaa4640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd11815d9a0 0x7fd1181579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11e2a3640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1180ce600 0x7fd118154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11daa2640 1 -- 172.21.15.133:0/548312895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1180ce600 msgr2=0x7fd118154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11daa2640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1180ce600 0x7fd118154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11daa2640 1 -- 172.21.15.133:0/548312895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd11815d9a0 msgr2=0x7fd1181579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11daa2640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd11815d9a0 0x7fd1181579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11daa2640 1 -- 172.21.15.133:0/548312895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1180c93a0 con 0x7fd1180d1920 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11e2a3640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1180ce600 0x7fd118154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11eaa4640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd11815d9a0 0x7fd1181579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11daa2640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1180d1920 0x7fd118157460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd11400c7f0 tx=0x7fd11400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd10effd640 1 -- 172.21.15.133:0/548312895 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd114015070 con 0x7fd1180d1920 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd10effd640 1 -- 172.21.15.133:0/548312895 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd114007820 con 0x7fd1180d1920 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd120507640 1 -- 172.21.15.133:0/548312895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1180c9680 con 0x7fd1180d1920 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd10effd640 1 -- 172.21.15.133:0/548312895 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd114007c10 con 0x7fd1180d1920 2024-04-25T10:02:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd120507640 1 -- 172.21.15.133:0/548312895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1180c9c40 con 0x7fd1180d1920 2024-04-25T10:02:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd120507640 1 -- 172.21.15.133:0/548312895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd118068e80 con 0x7fd1180d1920 2024-04-25T10:02:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd10effd640 1 -- 172.21.15.133:0/548312895 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd114018070 con 0x7fd1180d1920 2024-04-25T10:02:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd10effd640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd0fc066f60 0x7fd0fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd10effd640 1 -- 172.21.15.133:0/548312895 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd114085590 con 0x7fd1180d1920 2024-04-25T10:02:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11e2a3640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd0fc066f60 0x7fd0fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.804+0000 7fd11e2a3640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd0fc066f60 0x7fd0fc069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd1180c72d0 tx=0x7fd108008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:41.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.808+0000 7fd10effd640 1 -- 172.21.15.133:0/548312895 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd114056110 con 0x7fd1180d1920 2024-04-25T10:02:41.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.956+0000 7fd120507640 1 -- 172.21.15.133:0/548312895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1180d3300 con 0x7fd1180d1920 2024-04-25T10:02:41.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd10effd640 1 -- 172.21.15.133:0/548312895 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd114055a10 con 0x7fd1180d1920 2024-04-25T10:02:41.965 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:41.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 -- 172.21.15.133:0/548312895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd0fc066f60 msgr2=0x7fd0fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd0fc066f60 0x7fd0fc069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd1180c72d0 tx=0x7fd108008040 comp rx=0 tx=0).stop 2024-04-25T10:02:41.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 -- 172.21.15.133:0/548312895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1180d1920 msgr2=0x7fd118157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:41.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1180d1920 0x7fd118157460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd11400c7f0 tx=0x7fd11400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 -- 172.21.15.133:0/548312895 shutdown_connections 2024-04-25T10:02:41.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd11815d9a0 0x7fd1181579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd0fc066f60 0x7fd0fc069420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1180d1920 0x7fd118157460 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 --2- 172.21.15.133:0/548312895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1180ce600 0x7fd118154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:41.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 -- 172.21.15.133:0/548312895 >> 172.21.15.133:0/548312895 conn(0x7fd11806bf80 msgr2=0x7fd1180735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:41.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 -- 172.21.15.133:0/548312895 shutdown_connections 2024-04-25T10:02:41.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:41.960+0000 7fd120507640 1 -- 172.21.15.133:0/548312895 wait complete. 2024-04-25T10:02:41.983 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:42.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e3c66640 1 Processor -- start 2024-04-25T10:02:42.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e3c66640 1 -- start start 2024-04-25T10:02:42.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e3c66640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51dc143ea0 0x7f51dc0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e3c66640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51dc0b2150 0x7f51dc06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e3c66640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc15ac50 con 0x7f51dc15b9b0 2024-04-25T10:02:42.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e3c66640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc0cd180 con 0x7f51dc0b2150 2024-04-25T10:02:42.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e3c66640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc15aed0 con 0x7f51dc143ea0 2024-04-25T10:02:42.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e1201640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51dc0b2150 0x7f51dc06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e1201640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51dc0b2150 0x7f51dc06e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39658/0 (socket says 172.21.15.133:39658) 2024-04-25T10:02:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e1201640 1 -- 172.21.15.133:0/747837748 learned_addr learned my addr 172.21.15.133:0/747837748 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e1201640 1 -- 172.21.15.133:0/747837748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51dc143ea0 msgr2=0x7f51dc0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e1a02640 1 --2- 172.21.15.133:0/747837748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51dc143ea0 0x7f51dc0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e1201640 1 --2- 172.21.15.133:0/747837748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51dc143ea0 0x7f51dc0cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e1201640 1 -- 172.21.15.133:0/747837748 >> v1:172.21.15.162:6789/0 conn(0x7f51dc15b9b0 legacy=0x7f51dc15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.096+0000 7f51e1201640 1 -- 172.21.15.133:0/747837748 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc15b1b0 con 0x7f51dc0b2150 2024-04-25T10:02:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e1201640 1 --2- 172.21.15.133:0/747837748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51dc0b2150 0x7f51dc06e3b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009870 tx=0x7f51cc01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7058a16bd43aa8b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e0a00640 1 -- 172.21.15.133:0/747837748 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0b0070 con 0x7f51dc0b2150 2024-04-25T10:02:42.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e0a00640 1 -- 172.21.15.133:0/747837748 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc0ab600 con 0x7f51dc0b2150 2024-04-25T10:02:42.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e0a00640 1 -- 172.21.15.133:0/747837748 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0ab9d0 con 0x7f51dc0b2150 2024-04-25T10:02:42.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 -- 172.21.15.133:0/747837748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51dc0b2150 msgr2=0x7f51dc06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 --2- 172.21.15.133:0/747837748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51dc0b2150 0x7f51dc06e3b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009870 tx=0x7f51cc01c340 comp rx=0 tx=0).stop 2024-04-25T10:02:42.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 -- 172.21.15.133:0/747837748 shutdown_connections 2024-04-25T10:02:42.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 --2- 172.21.15.133:0/747837748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51dc0b2150 0x7f51dc06e3b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 --2- 172.21.15.133:0/747837748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51dc143ea0 0x7f51dc0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 -- 172.21.15.133:0/747837748 >> 172.21.15.133:0/747837748 conn(0x7f51dc066f70 msgr2=0x7f51dc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:42.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 -- 172.21.15.133:0/747837748 shutdown_connections 2024-04-25T10:02:42.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 -- 172.21.15.133:0/747837748 wait complete. 2024-04-25T10:02:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 Processor -- start 2024-04-25T10:02:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 -- start start 2024-04-25T10:02:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51dc0b2150 0x7f51dc134af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51dc143ea0 0x7f51dc151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51dc15b9b0 0x7f51dc163780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc1675e0 con 0x7f51dc143ea0 2024-04-25T10:02:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc06ed00 con 0x7f51dc0b2150 2024-04-25T10:02:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.100+0000 7f51e3c66640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc167760 con 0x7f51dc15b9b0 2024-04-25T10:02:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1201640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51dc143ea0 0x7f51dc151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1201640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51dc143ea0 0x7f51dc151d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39666/0 (socket says 172.21.15.133:39666) 2024-04-25T10:02:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1201640 1 -- 172.21.15.133:0/2706542574 learned_addr learned my addr 172.21.15.133:0/2706542574 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1a02640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51dc0b2150 0x7f51dc134af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e2203640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51dc15b9b0 0x7f51dc163780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1201640 1 -- 172.21.15.133:0/2706542574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51dc15b9b0 msgr2=0x7f51dc163780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1201640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51dc15b9b0 0x7f51dc163780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1201640 1 -- 172.21.15.133:0/2706542574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51dc0b2150 msgr2=0x7f51dc134af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1201640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51dc0b2150 0x7f51dc134af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1201640 1 -- 172.21.15.133:0/2706542574 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc13e800 con 0x7f51dc143ea0 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1201640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51dc143ea0 0x7f51dc151d70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f51cc01a9f0 tx=0x7f51cc0190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51d27fc640 1 -- 172.21.15.133:0/2706542574 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0b0040 con 0x7f51dc143ea0 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51d27fc640 1 -- 172.21.15.133:0/2706542574 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc0b4440 con 0x7f51dc143ea0 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51d27fc640 1 -- 172.21.15.133:0/2706542574 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0b4830 con 0x7f51dc143ea0 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e3c66640 1 -- 172.21.15.133:0/2706542574 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc13eae0 con 0x7f51dc143ea0 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e3c66640 1 -- 172.21.15.133:0/2706542574 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc13f0a0 con 0x7f51dc143ea0 2024-04-25T10:02:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e3c66640 1 -- 172.21.15.133:0/2706542574 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc068e80 con 0x7f51dc143ea0 2024-04-25T10:02:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51d27fc640 1 -- 172.21.15.133:0/2706542574 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51cc0bb020 con 0x7f51dc143ea0 2024-04-25T10:02:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51d27fc640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f90 0x7f51c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51d27fc640 1 -- 172.21.15.133:0/2706542574 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f51cc11e8d0 con 0x7f51dc143ea0 2024-04-25T10:02:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1a02640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f90 0x7f51c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.104+0000 7f51e1a02640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f90 0x7f51c0069450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51dc164830 tx=0x7f51d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.108+0000 7f51d27fc640 1 -- 172.21.15.133:0/2706542574 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51cc0c2020 con 0x7f51dc143ea0 2024-04-25T10:02:42.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 -- 172.21.15.133:0/2706542574 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51dc15e730 con 0x7f51dc143ea0 2024-04-25T10:02:42.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51d27fc640 1 -- 172.21.15.133:0/2706542574 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51dc15ac50 con 0x7f51dc143ea0 2024-04-25T10:02:42.262 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 -- 172.21.15.133:0/2706542574 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f90 msgr2=0x7f51c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f90 0x7f51c0069450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51dc164830 tx=0x7f51d8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 -- 172.21.15.133:0/2706542574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51dc143ea0 msgr2=0x7f51dc151d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51dc143ea0 0x7f51dc151d70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f51cc01a9f0 tx=0x7f51cc0190e0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 -- 172.21.15.133:0/2706542574 shutdown_connections 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51dc15b9b0 0x7f51dc163780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f90 0x7f51c0069450 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51dc143ea0 0x7f51dc151d70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 --2- 172.21.15.133:0/2706542574 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51dc0b2150 0x7f51dc134af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 -- 172.21.15.133:0/2706542574 >> 172.21.15.133:0/2706542574 conn(0x7f51dc066f70 msgr2=0x7f51dc142fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 -- 172.21.15.133:0/2706542574 shutdown_connections 2024-04-25T10:02:42.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.256+0000 7f51e3c66640 1 -- 172.21.15.133:0/2706542574 wait complete. 2024-04-25T10:02:42.279 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:42.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.380+0000 7fe8b2187640 1 Processor -- start 2024-04-25T10:02:42.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.380+0000 7fe8b2187640 1 -- start start 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8b2187640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe8ac15f030 0x7fe8ac09a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8b2187640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b2550 0x7fe8ac0999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8b2187640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac0cf320 con 0x7fe8ac099f10 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8b2187640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac0d07e0 con 0x7fe8ac15f030 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8b2187640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0ccdd0 con 0x7fe8ac0b2550 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8abfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe8ac099f10 0x7fe8ac15f6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53694/0 (socket says 172.21.15.133:53694) 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8abfff640 1 -- 172.21.15.133:0/1039818981 learned_addr learned my addr 172.21.15.133:0/1039818981 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8ab7fe640 1 --2- 172.21.15.133:0/1039818981 >> v2:172.21.15.162:3300/0 conn(0x7fe8ac15f030 0x7fe8ac09a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602262112 0 0) 0x7fe8ac0cf320 con 0x7fe8ac099f10 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aaffd640 1 --2- 172.21.15.133:0/1039818981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b2550 0x7fe8ac0999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe894003530 con 0x7fe8ac099f10 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 356929550 0 0) 0x7fe894003530 con 0x7fe8ac099f10 2024-04-25T10:02:42.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b2550 msgr2=0x7fe8ac0999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 --2- 172.21.15.133:0/1039818981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b2550 0x7fe8ac0999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 >> v2:172.21.15.162:3300/0 conn(0x7fe8ac15f030 msgr2=0x7fe8ac09a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 --2- 172.21.15.133:0/1039818981 >> v2:172.21.15.162:3300/0 conn(0x7fe8ac15f030 0x7fe8ac09a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac15f410 con 0x7fe8ac099f10 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe8a4001a00 con 0x7fe8ac099f10 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 >> v1:172.21.15.133:6789/0 conn(0x7fe8ac099f10 legacy=0x7fe8ac15f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 --2- 172.21.15.133:0/1039818981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8940039c0 0x7fe894003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 --2- 172.21.15.133:0/1039818981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b2550 0x7fe894004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 --2- 172.21.15.133:0/1039818981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe894004d50 0x7fe8940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0ccdd0 con 0x7fe8940039c0 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89400ba00 con 0x7fe894004d50 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89400bc00 con 0x7fe8ac0b2550 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8ab7fe640 1 --2- 172.21.15.133:0/1039818981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8940039c0 0x7fe894003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8ab7fe640 1 -- 172.21.15.133:0/1039818981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b2550 msgr2=0x7fe894004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8ab7fe640 1 --2- 172.21.15.133:0/1039818981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b2550 0x7fe894004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8ab7fe640 1 -- 172.21.15.133:0/1039818981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe894004d50 msgr2=0x7fe8940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8ab7fe640 1 --2- 172.21.15.133:0/1039818981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe894004d50 0x7fe8940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8ab7fe640 1 -- 172.21.15.133:0/1039818981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe894003530 con 0x7fe8940039c0 2024-04-25T10:02:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8ab7fe640 1 --2- 172.21.15.133:0/1039818981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8940039c0 0x7fe894003de0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009ae0 tx=0x7fe89c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f2d58eb97db9d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0096b0 con 0x7fe8940039c0 2024-04-25T10:02:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c0a6490 con 0x7fe8940039c0 2024-04-25T10:02:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8aa7fc640 1 -- 172.21.15.133:0/1039818981 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0a6880 con 0x7fe8940039c0 2024-04-25T10:02:42.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8b2187640 1 -- 172.21.15.133:0/1039818981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8940039c0 msgr2=0x7fe894003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.384+0000 7fe8b2187640 1 --2- 172.21.15.133:0/1039818981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8940039c0 0x7fe894003de0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009ae0 tx=0x7fe89c01cab0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8abfff640 1 -- 172.21.15.133:0/1039818981 reap_dead start 2024-04-25T10:02:42.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 -- 172.21.15.133:0/1039818981 shutdown_connections 2024-04-25T10:02:42.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 -- 172.21.15.133:0/1039818981 >> 172.21.15.133:0/1039818981 conn(0x7fe8ac0b31a0 msgr2=0x7fe8ac0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:42.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 -- 172.21.15.133:0/1039818981 shutdown_connections 2024-04-25T10:02:42.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 -- 172.21.15.133:0/1039818981 wait complete. 2024-04-25T10:02:42.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 Processor -- start 2024-04-25T10:02:42.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 -- start start 2024-04-25T10:02:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8940039c0 0x7fe8ac0b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b42d0 0x7fe8ac14eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ac157a40 0x7fe8ac157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0c5380 con 0x7fe8ac157a40 2024-04-25T10:02:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0c5200 con 0x7fe8940039c0 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b2930 con 0x7fe8ac0b42d0 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ac157a40 0x7fe8ac157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ac157a40 0x7fe8ac157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39690/0 (socket says 172.21.15.133:39690) 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8abfff640 1 -- 172.21.15.133:0/1924575218 learned_addr learned my addr 172.21.15.133:0/1924575218 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8ab7fe640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8940039c0 0x7fe8ac0b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8aaffd640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b42d0 0x7fe8ac14eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8abfff640 1 -- 172.21.15.133:0/1924575218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b42d0 msgr2=0x7fe8ac14eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8abfff640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b42d0 0x7fe8ac14eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8abfff640 1 -- 172.21.15.133:0/1924575218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8940039c0 msgr2=0x7fe8ac0b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8abfff640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8940039c0 0x7fe8ac0b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8abfff640 1 -- 172.21.15.133:0/1924575218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac14f050 con 0x7fe8ac157a40 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8abfff640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ac157a40 0x7fe8ac157ec0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4000f90 tx=0x7fe8a400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe893fff640 1 -- 172.21.15.133:0/1924575218 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4007110 con 0x7fe8ac157a40 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe893fff640 1 -- 172.21.15.133:0/1924575218 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a40072d0 con 0x7fe8ac157a40 2024-04-25T10:02:42.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 -- 172.21.15.133:0/1924575218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac0b4a40 con 0x7fe8ac157a40 2024-04-25T10:02:42.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe893fff640 1 -- 172.21.15.133:0/1924575218 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40076c0 con 0x7fe8ac157a40 2024-04-25T10:02:42.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.388+0000 7fe8b2187640 1 -- 172.21.15.133:0/1924575218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac0b5000 con 0x7fe8ac157a40 2024-04-25T10:02:42.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.392+0000 7fe8b2187640 1 -- 172.21.15.133:0/1924575218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac0b5650 con 0x7fe8ac157a40 2024-04-25T10:02:42.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.392+0000 7fe893fff640 1 -- 172.21.15.133:0/1924575218 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8a4011070 con 0x7fe8ac157a40 2024-04-25T10:02:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.392+0000 7fe893fff640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe898066f90 0x7fe898069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.392+0000 7fe8ab7fe640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe898066f90 0x7fe898069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.392+0000 7fe893fff640 1 -- 172.21.15.133:0/1924575218 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe8a4084aa0 con 0x7fe8ac157a40 2024-04-25T10:02:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.392+0000 7fe8ab7fe640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe898066f90 0x7fe898069450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0ad000 tx=0x7fe89c0b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.396+0000 7fe893fff640 1 -- 172.21.15.133:0/1924575218 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8a401a060 con 0x7fe8ac157a40 2024-04-25T10:02:42.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.540+0000 7fe8b2187640 1 -- 172.21.15.133:0/1924575218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe8ac068e80 con 0x7fe8ac157a40 2024-04-25T10:02:42.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.540+0000 7fe893fff640 1 -- 172.21.15.133:0/1924575218 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe8a40550f0 con 0x7fe8ac157a40 2024-04-25T10:02:42.549 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:42.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 -- 172.21.15.133:0/1924575218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe898066f90 msgr2=0x7fe898069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe898066f90 0x7fe898069450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0ad000 tx=0x7fe89c0b2040 comp rx=0 tx=0).stop 2024-04-25T10:02:42.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 -- 172.21.15.133:0/1924575218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ac157a40 msgr2=0x7fe8ac157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ac157a40 0x7fe8ac157ec0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4000f90 tx=0x7fe8a400ccd0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 -- 172.21.15.133:0/1924575218 shutdown_connections 2024-04-25T10:02:42.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe898066f90 0x7fe898069450 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ac157a40 0x7fe8ac157ec0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8ac0b42d0 0x7fe8ac14eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 --2- 172.21.15.133:0/1924575218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8940039c0 0x7fe8ac0b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 -- 172.21.15.133:0/1924575218 >> 172.21.15.133:0/1924575218 conn(0x7fe8ac0b31a0 msgr2=0x7fe8ac1615b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:42.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 -- 172.21.15.133:0/1924575218 shutdown_connections 2024-04-25T10:02:42.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.544+0000 7fe8b2187640 1 -- 172.21.15.133:0/1924575218 wait complete. 2024-04-25T10:02:42.566 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:42.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.664+0000 7f2d69f73640 1 Processor -- start 2024-04-25T10:02:42.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.664+0000 7f2d69f73640 1 -- start start 2024-04-25T10:02:42.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d69f73640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2d640d0770 0x7f2d640d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d69f73640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d640ceb20 con 0x7f2d640b87e0 2024-04-25T10:02:42.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d69f73640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d640b30a0 con 0x7f2d6415db00 2024-04-25T10:02:42.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d69f73640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d640d1110 con 0x7f2d640d0770 2024-04-25T10:02:42.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d63fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2d640b87e0 0x7f2d640b8c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53700/0 (socket says 172.21.15.133:53700) 2024-04-25T10:02:42.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d63fff640 1 -- 172.21.15.133:0/4150415218 learned_addr learned my addr 172.21.15.133:0/4150415218 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:42.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d62ffd640 1 --2- 172.21.15.133:0/4150415218 >> v2:172.21.15.162:3300/0 conn(0x7f2d640d0770 0x7f2d640d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320927788 0 0) 0x7f2d640ceb20 con 0x7f2d640b87e0 2024-04-25T10:02:42.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d4c003530 con 0x7f2d640b87e0 2024-04-25T10:02:42.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2375922567 0 0) 0x7f2d4c003530 con 0x7f2d640b87e0 2024-04-25T10:02:42.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 >> v2:172.21.15.162:3300/0 conn(0x7f2d640d0770 msgr2=0x7f2d640d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 --2- 172.21.15.133:0/4150415218 >> v2:172.21.15.162:3300/0 conn(0x7f2d640d0770 0x7f2d640d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 >> v1:172.21.15.162:6789/0 conn(0x7f2d6415db00 legacy=0x7f2d640b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640b7740 con 0x7f2d640b87e0 2024-04-25T10:02:42.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2d5c001a00 con 0x7f2d640b87e0 2024-04-25T10:02:42.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 >> v1:172.21.15.133:6789/0 conn(0x7f2d640b87e0 legacy=0x7f2d640b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 --2- 172.21.15.133:0/4150415218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d4c0039c0 0x7f2d4c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 --2- 172.21.15.133:0/4150415218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4c004810 0x7f2d4c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 --2- 172.21.15.133:0/4150415218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d4c0051d0 0x7f2d4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c00c040 con 0x7f2d4c0039c0 2024-04-25T10:02:42.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c00c1f0 con 0x7f2d4c0051d0 2024-04-25T10:02:42.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c00c3f0 con 0x7f2d4c004810 2024-04-25T10:02:42.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d637fe640 1 --2- 172.21.15.133:0/4150415218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d4c0039c0 0x7f2d4c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d63fff640 1 --2- 172.21.15.133:0/4150415218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d4c0051d0 0x7f2d4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d637fe640 1 -- 172.21.15.133:0/4150415218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4c004810 msgr2=0x7f2d4c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d637fe640 1 --2- 172.21.15.133:0/4150415218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4c004810 0x7f2d4c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d637fe640 1 -- 172.21.15.133:0/4150415218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d4c0051d0 msgr2=0x7f2d4c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d637fe640 1 --2- 172.21.15.133:0/4150415218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d4c0051d0 0x7f2d4c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d63fff640 1 -- 172.21.15.133:0/4150415218 reap_dead start 2024-04-25T10:02:42.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d637fe640 1 -- 172.21.15.133:0/4150415218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c003530 con 0x7f2d4c0039c0 2024-04-25T10:02:42.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.668+0000 7f2d637fe640 1 --2- 172.21.15.133:0/4150415218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d4c0039c0 0x7f2d4c003de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001780 tx=0x7f2d5801c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2e5f835e49a0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580013a0 con 0x7f2d4c0039c0 2024-04-25T10:02:42.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d580052f0 con 0x7f2d4c0039c0 2024-04-25T10:02:42.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d627fc640 1 -- 172.21.15.133:0/4150415218 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58003840 con 0x7f2d4c0039c0 2024-04-25T10:02:42.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 -- 172.21.15.133:0/4150415218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d4c0039c0 msgr2=0x7f2d4c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 --2- 172.21.15.133:0/4150415218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d4c0039c0 0x7f2d4c003de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001780 tx=0x7f2d5801c1f0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 -- 172.21.15.133:0/4150415218 shutdown_connections 2024-04-25T10:02:42.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 --2- 172.21.15.133:0/4150415218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d4c0039c0 0x7f2d4c003de0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 -- 172.21.15.133:0/4150415218 >> 172.21.15.133:0/4150415218 conn(0x7f2d640727c0 msgr2=0x7f2d64072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:42.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 -- 172.21.15.133:0/4150415218 shutdown_connections 2024-04-25T10:02:42.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 -- 172.21.15.133:0/4150415218 wait complete. 2024-04-25T10:02:42.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 Processor -- start 2024-04-25T10:02:42.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 -- start start 2024-04-25T10:02:42.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4c0039c0 0x7f2d640b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d641fa600 0x7f2d640d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d641faa80 0x7f2d640d2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640b7430 con 0x7f2d641fa600 2024-04-25T10:02:42.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640b82c0 con 0x7f2d641faa80 2024-04-25T10:02:42.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640d18a0 con 0x7f2d4c0039c0 2024-04-25T10:02:42.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d62ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d641fa600 0x7f2d640d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d62ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d641fa600 0x7f2d640d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39716/0 (socket says 172.21.15.133:39716) 2024-04-25T10:02:42.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d62ffd640 1 -- 172.21.15.133:0/3759861228 learned_addr learned my addr 172.21.15.133:0/3759861228 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:42.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d63fff640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d641faa80 0x7f2d640d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d637fe640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4c0039c0 0x7f2d640b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d62ffd640 1 -- 172.21.15.133:0/3759861228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4c0039c0 msgr2=0x7f2d640b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d62ffd640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4c0039c0 0x7f2d640b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d62ffd640 1 -- 172.21.15.133:0/3759861228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d641faa80 msgr2=0x7f2d640d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d62ffd640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d641faa80 0x7f2d640d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d62ffd640 1 -- 172.21.15.133:0/3759861228 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d641fafe0 con 0x7f2d641fa600 2024-04-25T10:02:42.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d62ffd640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d641fa600 0x7f2d640d22e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400e7d0 tx=0x7f2d5400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d4bfff640 1 -- 172.21.15.133:0/3759861228 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d54017070 con 0x7f2d641fa600 2024-04-25T10:02:42.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d4bfff640 1 -- 172.21.15.133:0/3759861228 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5400ee80 con 0x7f2d641fa600 2024-04-25T10:02:42.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 -- 172.21.15.133:0/3759861228 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d641fb2c0 con 0x7f2d641fa600 2024-04-25T10:02:42.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d4bfff640 1 -- 172.21.15.133:0/3759861228 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5400ca80 con 0x7f2d641fa600 2024-04-25T10:02:42.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.672+0000 7f2d69f73640 1 -- 172.21.15.133:0/3759861228 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d640b2010 con 0x7f2d641fa600 2024-04-25T10:02:42.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.676+0000 7f2d69f73640 1 -- 172.21.15.133:0/3759861228 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d64068ef0 con 0x7f2d641fa600 2024-04-25T10:02:42.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.676+0000 7f2d4bfff640 1 -- 172.21.15.133:0/3759861228 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d5401a070 con 0x7f2d641fa600 2024-04-25T10:02:42.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.676+0000 7f2d4bfff640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d50066f90 0x7f2d50069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.676+0000 7f2d4bfff640 1 -- 172.21.15.133:0/3759861228 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2d54085080 con 0x7f2d641fa600 2024-04-25T10:02:42.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.676+0000 7f2d637fe640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d50066f90 0x7f2d50069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.676+0000 7f2d637fe640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d50066f90 0x7f2d50069450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001750 tx=0x7f2d580011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.680+0000 7f2d4bfff640 1 -- 172.21.15.133:0/3759861228 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d54055c90 con 0x7f2d641fa600 2024-04-25T10:02:42.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.840+0000 7f2d69f73640 1 -- 172.21.15.133:0/3759861228 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d64161e60 con 0x7f2d641fa600 2024-04-25T10:02:42.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.844+0000 7f2d4bfff640 1 -- 172.21.15.133:0/3759861228 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f2d54055590 con 0x7f2d641fa600 2024-04-25T10:02:42.852 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:42.852 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:42.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 -- 172.21.15.133:0/3759861228 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d50066f90 msgr2=0x7f2d50069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d50066f90 0x7f2d50069450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001750 tx=0x7f2d580011a0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 -- 172.21.15.133:0/3759861228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d641fa600 msgr2=0x7f2d640d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d641fa600 0x7f2d640d22e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400e7d0 tx=0x7f2d5400ec90 comp rx=0 tx=0).stop 2024-04-25T10:02:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 -- 172.21.15.133:0/3759861228 shutdown_connections 2024-04-25T10:02:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d641faa80 0x7f2d640d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d50066f90 0x7f2d50069450 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d641fa600 0x7f2d640d22e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 --2- 172.21.15.133:0/3759861228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d4c0039c0 0x7f2d640b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 -- 172.21.15.133:0/3759861228 >> 172.21.15.133:0/3759861228 conn(0x7f2d640727c0 msgr2=0x7f2d640c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 -- 172.21.15.133:0/3759861228 shutdown_connections 2024-04-25T10:02:42.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.848+0000 7f2d69f73640 1 -- 172.21.15.133:0/3759861228 wait complete. 2024-04-25T10:02:42.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:42.870 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:42.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc7aa7640 1 Processor -- start 2024-04-25T10:02:42.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc7aa7640 1 -- start start 2024-04-25T10:02:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc7aa7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0145170 0x7f3fc006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc7aa7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3fc006e4d0 0x7f3fc00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc7aa7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc00ce2e0 con 0x7f3fc00cdd60 2024-04-25T10:02:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc7aa7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc006db00 con 0x7f3fc006e4d0 2024-04-25T10:02:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc7aa7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc0145550 con 0x7f3fc0145170 2024-04-25T10:02:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc6044640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3fc00cdd60 0x7f3fc015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53706/0 (socket says 172.21.15.133:53706) 2024-04-25T10:02:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc6044640 1 -- 172.21.15.133:0/2814872033 learned_addr learned my addr 172.21.15.133:0/2814872033 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc5843640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0145170 0x7f3fc006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc5042640 1 --2- 172.21.15.133:0/2814872033 >> v2:172.21.15.162:3300/0 conn(0x7f3fc006e4d0 0x7f3fc00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816439737 0 0) 0x7f3fc00ce2e0 con 0x7f3fc00cdd60 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fac003530 con 0x7f3fc00cdd60 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 270206774 0 0) 0x7f3fac003530 con 0x7f3fc00cdd60 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0145170 msgr2=0x7f3fc006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0145170 0x7f3fc006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 >> v2:172.21.15.162:3300/0 conn(0x7f3fc006e4d0 msgr2=0x7f3fc00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 --2- 172.21.15.133:0/2814872033 >> v2:172.21.15.162:3300/0 conn(0x7f3fc006e4d0 0x7f3fc00cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc006d4b0 con 0x7f3fc00cdd60 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f3fb8001a00 con 0x7f3fc00cdd60 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 >> v1:172.21.15.133:6789/0 conn(0x7f3fc00cdd60 legacy=0x7f3fc015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fac0039c0 0x7f3fac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0145170 0x7f3fac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac004d50 0x7f3fac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc006db00 con 0x7f3fac004d50 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc0145550 con 0x7f3fac0039c0 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac00b9d0 con 0x7f3fc0145170 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc6044640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac004d50 0x7f3fac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc5042640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0145170 0x7f3fac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc6044640 1 -- 172.21.15.133:0/2814872033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0145170 msgr2=0x7f3fac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc6044640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc0145170 0x7f3fac004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc6044640 1 -- 172.21.15.133:0/2814872033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fac0039c0 msgr2=0x7f3fac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc5843640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fac0039c0 0x7f3fac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc6044640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fac0039c0 0x7f3fac003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.976+0000 7f3fc6044640 1 -- 172.21.15.133:0/2814872033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac003530 con 0x7f3fac004d50 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc6044640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac004d50 0x7f3fac0051d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80013e0 tx=0x7f3fb801d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a621f65166f87f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800a4e0 con 0x7f3fac004d50 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb80072f0 con 0x7f3fac004d50 2024-04-25T10:02:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc4841640 1 -- 172.21.15.133:0/2814872033 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80076e0 con 0x7f3fac004d50 2024-04-25T10:02:42.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2814872033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac004d50 msgr2=0x7f3fac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 --2- 172.21.15.133:0/2814872033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fac004d50 0x7f3fac0051d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80013e0 tx=0x7f3fb801d3a0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc6044640 1 -- 172.21.15.133:0/2814872033 reap_dead start 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2814872033 shutdown_connections 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2814872033 >> 172.21.15.133:0/2814872033 conn(0x7f3fc0066f70 msgr2=0x7f3fc006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2814872033 shutdown_connections 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2814872033 wait complete. 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 Processor -- start 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 -- start start 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fac004d50 0x7f3fc01f09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc013d1b0 0x7f3fc012b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc013f1d0 0x7f3fc013f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc015d5e0 con 0x7f3fc013d1b0 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc015cb70 con 0x7f3fac004d50 2024-04-25T10:02:42.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc7aa7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc015d760 con 0x7f3fc013f1d0 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc5042640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc013d1b0 0x7f3fc012b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc5042640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc013d1b0 0x7f3fc012b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39736/0 (socket says 172.21.15.133:39736) 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.980+0000 7f3fc5042640 1 -- 172.21.15.133:0/2407532452 learned_addr learned my addr 172.21.15.133:0/2407532452 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fc5042640 1 -- 172.21.15.133:0/2407532452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc013f1d0 msgr2=0x7f3fc013f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fc5042640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc013f1d0 0x7f3fc013f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fc5042640 1 -- 172.21.15.133:0/2407532452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fac004d50 msgr2=0x7f3fc01f09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fc5042640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fac004d50 0x7f3fc01f09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fc5042640 1 -- 172.21.15.133:0/2407532452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc013d630 con 0x7f3fc013d1b0 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fc5042640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc013d1b0 0x7f3fc012b0e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0010890 tx=0x7f3fb0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fb67fc640 1 -- 172.21.15.133:0/2407532452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb000f040 con 0x7f3fc013d1b0 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fb67fc640 1 -- 172.21.15.133:0/2407532452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb000ea80 con 0x7f3fc013d1b0 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2407532452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc012b620 con 0x7f3fc013d1b0 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fb67fc640 1 -- 172.21.15.133:0/2407532452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb000ee70 con 0x7f3fc013d1b0 2024-04-25T10:02:42.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2407532452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc013ae20 con 0x7f3fc013d1b0 2024-04-25T10:02:42.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2407532452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc0068e80 con 0x7f3fc013d1b0 2024-04-25T10:02:42.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fb67fc640 1 -- 172.21.15.133:0/2407532452 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3fb001c400 con 0x7f3fc013d1b0 2024-04-25T10:02:42.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fb67fc640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3fa4066f90 0x7f3fa4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fb67fc640 1 -- 172.21.15.133:0/2407532452 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3fb00858a0 con 0x7f3fc013d1b0 2024-04-25T10:02:42.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fc5843640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3fa4066f90 0x7f3fa4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.984+0000 7f3fc5843640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3fa4066f90 0x7f3fa4069450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009970 tx=0x7f3fbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:42.988+0000 7f3fb67fc640 1 -- 172.21.15.133:0/2407532452 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3fc012b620 con 0x7f3fc013d1b0 2024-04-25T10:02:43.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.136+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2407532452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fc015dcb0 con 0x7f3fc013d1b0 2024-04-25T10:02:43.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.136+0000 7f3fb67fc640 1 -- 172.21.15.133:0/2407532452 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fb005ae20 con 0x7f3fc013d1b0 2024-04-25T10:02:43.144 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:43.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2407532452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3fa4066f90 msgr2=0x7f3fa4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3fa4066f90 0x7f3fa4069450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009970 tx=0x7f3fbc008040 comp rx=0 tx=0).stop 2024-04-25T10:02:43.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2407532452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc013d1b0 msgr2=0x7f3fc012b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc013d1b0 0x7f3fc012b0e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0010890 tx=0x7f3fb0010d50 comp rx=0 tx=0).stop 2024-04-25T10:02:43.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2407532452 shutdown_connections 2024-04-25T10:02:43.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3fc013f1d0 0x7f3fc013f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3fa4066f90 0x7f3fa4069450 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fc013d1b0 0x7f3fc012b0e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 --2- 172.21.15.133:0/2407532452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fac004d50 0x7f3fc01f09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2407532452 >> 172.21.15.133:0/2407532452 conn(0x7f3fc0066f70 msgr2=0x7f3fc015f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2407532452 shutdown_connections 2024-04-25T10:02:43.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.140+0000 7f3fc7aa7640 1 -- 172.21.15.133:0/2407532452 wait complete. 2024-04-25T10:02:43.163 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.264+0000 7f9e2e134640 1 Processor -- start 2024-04-25T10:02:43.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.264+0000 7f9e2e134640 1 -- start start 2024-04-25T10:02:43.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e2e134640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e28142920 0x7f9e28143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e2e134640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e28140ea0 con 0x7f9e2815d9f0 2024-04-25T10:02:43.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e2e134640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e280cf280 con 0x7f9e28140a40 2024-04-25T10:02:43.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e2e134640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e2806d440 con 0x7f9e28142920 2024-04-25T10:02:43.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e277fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e28142920 0x7f9e28143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e27fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9e2815d9f0 0x7f9e2806df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53714/0 (socket says 172.21.15.133:53714) 2024-04-25T10:02:43.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e27fff640 1 -- 172.21.15.133:0/1506280056 learned_addr learned my addr 172.21.15.133:0/1506280056 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:43.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e267fc640 1 -- 172.21.15.133:0/1506280056 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933937978 0 0) 0x7f9e28140ea0 con 0x7f9e2815d9f0 2024-04-25T10:02:43.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e267fc640 1 -- 172.21.15.133:0/1506280056 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e10003530 con 0x7f9e2815d9f0 2024-04-25T10:02:43.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e277fe640 1 -- 172.21.15.133:0/1506280056 >> v1:172.21.15.162:6789/0 conn(0x7f9e28140a40 legacy=0x7f9e280ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e277fe640 1 -- 172.21.15.133:0/1506280056 >> v1:172.21.15.133:6789/0 conn(0x7f9e2815d9f0 legacy=0x7f9e2806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e277fe640 1 -- 172.21.15.133:0/1506280056 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2806d720 con 0x7f9e28142920 2024-04-25T10:02:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e277fe640 1 --2- 172.21.15.133:0/1506280056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e28142920 0x7f9e28143560 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9e180097c0 tx=0x7f9e1801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477f47f64c9eff0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e267fc640 1 -- 172.21.15.133:0/1506280056 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e180b0070 con 0x7f9e28142920 2024-04-25T10:02:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e267fc640 1 -- 172.21.15.133:0/1506280056 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e180ab600 con 0x7f9e28142920 2024-04-25T10:02:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.268+0000 7f9e267fc640 1 -- 172.21.15.133:0/1506280056 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e180ab9d0 con 0x7f9e28142920 2024-04-25T10:02:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 -- 172.21.15.133:0/1506280056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e28142920 msgr2=0x7f9e28143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 --2- 172.21.15.133:0/1506280056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e28142920 0x7f9e28143560 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9e180097c0 tx=0x7f9e1801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 -- 172.21.15.133:0/1506280056 shutdown_connections 2024-04-25T10:02:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 --2- 172.21.15.133:0/1506280056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e28142920 0x7f9e28143560 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 -- 172.21.15.133:0/1506280056 >> 172.21.15.133:0/1506280056 conn(0x7f9e28066f70 msgr2=0x7f9e2806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 -- 172.21.15.133:0/1506280056 shutdown_connections 2024-04-25T10:02:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 -- 172.21.15.133:0/1506280056 wait complete. 2024-04-25T10:02:43.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 Processor -- start 2024-04-25T10:02:43.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 -- start start 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e28140a40 0x7f9e28151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e28142920 0x7f9e2815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e2815d9f0 0x7f9e2815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28142e80 con 0x7f9e28142920 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28142d00 con 0x7f9e28140a40 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28167da0 con 0x7f9e2815d9f0 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e26ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e28142920 0x7f9e2815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e26ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e28142920 0x7f9e2815ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39750/0 (socket says 172.21.15.133:39750) 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e26ffd640 1 -- 172.21.15.133:0/1854968400 learned_addr learned my addr 172.21.15.133:0/1854968400 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e26ffd640 1 -- 172.21.15.133:0/1854968400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e2815d9f0 msgr2=0x7f9e2815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e26ffd640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e2815d9f0 0x7f9e2815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e26ffd640 1 -- 172.21.15.133:0/1854968400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e28140a40 msgr2=0x7f9e28151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e277fe640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e28140a40 0x7f9e28151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e26ffd640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e28140a40 0x7f9e28151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e26ffd640 1 -- 172.21.15.133:0/1854968400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2815ba00 con 0x7f9e28142920 2024-04-25T10:02:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e26ffd640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e28142920 0x7f9e2815ae80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c480 tx=0x7f9e1c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e0ffff640 1 -- 172.21.15.133:0/1854968400 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c012830 con 0x7f9e28142920 2024-04-25T10:02:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e0ffff640 1 -- 172.21.15.133:0/1854968400 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c017070 con 0x7f9e28142920 2024-04-25T10:02:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 -- 172.21.15.133:0/1854968400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2815bce0 con 0x7f9e28142920 2024-04-25T10:02:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e0ffff640 1 -- 172.21.15.133:0/1854968400 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c012c30 con 0x7f9e28142920 2024-04-25T10:02:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.272+0000 7f9e2e134640 1 -- 172.21.15.133:0/1854968400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2815c2a0 con 0x7f9e28142920 2024-04-25T10:02:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.276+0000 7f9e2e134640 1 -- 172.21.15.133:0/1854968400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e28068e80 con 0x7f9e28142920 2024-04-25T10:02:43.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.276+0000 7f9e0ffff640 1 -- 172.21.15.133:0/1854968400 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e1c01c460 con 0x7f9e28142920 2024-04-25T10:02:43.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.276+0000 7f9e0ffff640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e14066f60 0x7f9e14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.276+0000 7f9e0ffff640 1 -- 172.21.15.133:0/1854968400 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9e1c085250 con 0x7f9e28142920 2024-04-25T10:02:43.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.276+0000 7f9e277fe640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e14066f60 0x7f9e14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.276+0000 7f9e277fe640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e14066f60 0x7f9e14069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e180ad000 tx=0x7f9e18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.280+0000 7f9e0ffff640 1 -- 172.21.15.133:0/1854968400 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e1c055ef0 con 0x7f9e28142920 2024-04-25T10:02:43.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.432+0000 7f9e2e134640 1 -- 172.21.15.133:0/1854968400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e2815d380 con 0x7f9e28142920 2024-04-25T10:02:43.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e0ffff640 1 -- 172.21.15.133:0/1854968400 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e1c0557f0 con 0x7f9e28142920 2024-04-25T10:02:43.441 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:43.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 -- 172.21.15.133:0/1854968400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e14066f60 msgr2=0x7f9e14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e14066f60 0x7f9e14069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e180ad000 tx=0x7f9e18008040 comp rx=0 tx=0).stop 2024-04-25T10:02:43.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 -- 172.21.15.133:0/1854968400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e28142920 msgr2=0x7f9e2815ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e28142920 0x7f9e2815ae80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c480 tx=0x7f9e1c00c940 comp rx=0 tx=0).stop 2024-04-25T10:02:43.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 -- 172.21.15.133:0/1854968400 shutdown_connections 2024-04-25T10:02:43.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e2815d9f0 0x7f9e2815b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e14066f60 0x7f9e14069420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e28142920 0x7f9e2815ae80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 --2- 172.21.15.133:0/1854968400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e28140a40 0x7f9e28151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 -- 172.21.15.133:0/1854968400 >> 172.21.15.133:0/1854968400 conn(0x7f9e28066f70 msgr2=0x7f9e28141680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 -- 172.21.15.133:0/1854968400 shutdown_connections 2024-04-25T10:02:43.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.436+0000 7f9e2e134640 1 -- 172.21.15.133:0/1854968400 wait complete. 2024-04-25T10:02:43.459 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:43.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.560+0000 7fab6ba79640 1 Processor -- start 2024-04-25T10:02:43.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.560+0000 7fab6ba79640 1 -- start start 2024-04-25T10:02:43.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab6ba79640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab6415dc70 0x7fab6406fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab6ba79640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab641622a0 0x7fab64161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab6ba79640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab640cc320 con 0x7fab64161c40 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab6ba79640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab641620b0 con 0x7fab6415dc70 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab6ba79640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6415e050 con 0x7fab641622a0 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab69815640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab6415dc70 0x7fab6406fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab6a016640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fab64161c40 0x7fab6406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53728/0 (socket says 172.21.15.133:53728) 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab6a016640 1 -- 172.21.15.133:0/2863143686 learned_addr learned my addr 172.21.15.133:0/2863143686 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab69014640 1 --2- 172.21.15.133:0/2863143686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab641622a0 0x7fab64161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab69815640 1 -- 172.21.15.133:0/2863143686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab641622a0 msgr2=0x7fab64161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab69815640 1 --2- 172.21.15.133:0/2863143686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab641622a0 0x7fab64161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab69815640 1 -- 172.21.15.133:0/2863143686 >> v1:172.21.15.133:6789/0 conn(0x7fab64161c40 legacy=0x7fab6406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab68813640 1 -- 172.21.15.133:0/2863143686 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562801156 0 0) 0x7fab640cc320 con 0x7fab64161c40 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab69815640 1 -- 172.21.15.133:0/2863143686 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab640ca500 con 0x7fab6415dc70 2024-04-25T10:02:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab69014640 1 --2- 172.21.15.133:0/2863143686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab641622a0 0x7fab64161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:43.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab69815640 1 --2- 172.21.15.133:0/2863143686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab6415dc70 0x7fab6406fd90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fab600097c0 tx=0x7fab6001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63098c2b037e3343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab68813640 1 -- 172.21.15.133:0/2863143686 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab600b0070 con 0x7fab6415dc70 2024-04-25T10:02:43.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab68813640 1 -- 172.21.15.133:0/2863143686 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab600ab600 con 0x7fab6415dc70 2024-04-25T10:02:43.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.564+0000 7fab68813640 1 -- 172.21.15.133:0/2863143686 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab600ab9d0 con 0x7fab6415dc70 2024-04-25T10:02:43.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- 172.21.15.133:0/2863143686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab6415dc70 msgr2=0x7fab6406fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 --2- 172.21.15.133:0/2863143686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab6415dc70 0x7fab6406fd90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fab600097c0 tx=0x7fab6001c410 comp rx=0 tx=0).stop 2024-04-25T10:02:43.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- 172.21.15.133:0/2863143686 shutdown_connections 2024-04-25T10:02:43.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 --2- 172.21.15.133:0/2863143686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab641622a0 0x7fab64161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 --2- 172.21.15.133:0/2863143686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab6415dc70 0x7fab6406fd90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- 172.21.15.133:0/2863143686 >> 172.21.15.133:0/2863143686 conn(0x7fab640b1800 msgr2=0x7fab640b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- 172.21.15.133:0/2863143686 shutdown_connections 2024-04-25T10:02:43.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- 172.21.15.133:0/2863143686 wait complete. 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 Processor -- start 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- start start 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab6415dc70 0x7fab64154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab64161c40 0x7fab641554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab641622a0 0x7fab641568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6416add0 con 0x7fab641622a0 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6416ac50 con 0x7fab6415dc70 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6416af50 con 0x7fab64161c40 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6a016640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab641622a0 0x7fab641568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6a016640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab641622a0 0x7fab641568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39758/0 (socket says 172.21.15.133:39758) 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6a016640 1 -- 172.21.15.133:0/1802800504 learned_addr learned my addr 172.21.15.133:0/1802800504 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:43.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6a016640 1 -- 172.21.15.133:0/1802800504 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab64161c40 msgr2=0x7fab641554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6a016640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab64161c40 0x7fab641554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6a016640 1 -- 172.21.15.133:0/1802800504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab6415dc70 msgr2=0x7fab64154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6a016640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab6415dc70 0x7fab64154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6a016640 1 -- 172.21.15.133:0/1802800504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab640c86c0 con 0x7fab641622a0 2024-04-25T10:02:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6a016640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab641622a0 0x7fab641568b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c730 tx=0x7fab5c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab5a7fc640 1 -- 172.21.15.133:0/1802800504 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c018070 con 0x7fab641622a0 2024-04-25T10:02:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab5a7fc640 1 -- 172.21.15.133:0/1802800504 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c00ce40 con 0x7fab641622a0 2024-04-25T10:02:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- 172.21.15.133:0/1802800504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab640c89a0 con 0x7fab641622a0 2024-04-25T10:02:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab5a7fc640 1 -- 172.21.15.133:0/1802800504 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00a860 con 0x7fab641622a0 2024-04-25T10:02:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- 172.21.15.133:0/1802800504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6415d430 con 0x7fab641622a0 2024-04-25T10:02:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.568+0000 7fab6ba79640 1 -- 172.21.15.133:0/1802800504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab64067c70 con 0x7fab641622a0 2024-04-25T10:02:43.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.572+0000 7fab5a7fc640 1 -- 172.21.15.133:0/1802800504 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab5c01b070 con 0x7fab641622a0 2024-04-25T10:02:43.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.572+0000 7fab5a7fc640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 0x7fab48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.572+0000 7fab5a7fc640 1 -- 172.21.15.133:0/1802800504 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fab5c084900 con 0x7fab641622a0 2024-04-25T10:02:43.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.572+0000 7fab69815640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 0x7fab48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.572+0000 7fab69815640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 0x7fab48069450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab600ad000 tx=0x7fab60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.576+0000 7fab5a7fc640 1 -- 172.21.15.133:0/1802800504 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab5c0555a0 con 0x7fab641622a0 2024-04-25T10:02:43.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 -- 172.21.15.133:0/1802800504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab6406afd0 con 0x7fab641622a0 2024-04-25T10:02:43.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab5a7fc640 1 -- 172.21.15.133:0/1802800504 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab5c054ea0 con 0x7fab641622a0 2024-04-25T10:02:43.726 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:43.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 -- 172.21.15.133:0/1802800504 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 msgr2=0x7fab48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 0x7fab48069450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab600ad000 tx=0x7fab60008040 comp rx=0 tx=0).stop 2024-04-25T10:02:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 -- 172.21.15.133:0/1802800504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab641622a0 msgr2=0x7fab641568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab641622a0 0x7fab641568b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c730 tx=0x7fab5c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 -- 172.21.15.133:0/1802800504 shutdown_connections 2024-04-25T10:02:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 0x7fab48069450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab641622a0 0x7fab641568b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab64161c40 0x7fab641554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 --2- 172.21.15.133:0/1802800504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab6415dc70 0x7fab64154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 -- 172.21.15.133:0/1802800504 >> 172.21.15.133:0/1802800504 conn(0x7fab640b1800 msgr2=0x7fab640b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 -- 172.21.15.133:0/1802800504 shutdown_connections 2024-04-25T10:02:43.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.720+0000 7fab6ba79640 1 -- 172.21.15.133:0/1802800504 wait complete. 2024-04-25T10:02:43.744 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:43.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56c9f08640 1 Processor -- start 2024-04-25T10:02:43.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56c9f08640 1 -- start start 2024-04-25T10:02:43.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56c9f08640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c406ac90 0x7f56c415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56c9f08640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56c40b2770 0x7f56c406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56c9f08640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f56c415f260 0x7f56c415f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56c9f08640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c40b2f80 con 0x7f56c40b2770 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56c9f08640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c415cc30 con 0x7f56c415f260 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56c9f08640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c406b5b0 con 0x7f56c406ac90 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56bbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56c40b2770 0x7f56c406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56bbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56c40b2770 0x7f56c406b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39768/0 (socket says 172.21.15.133:39768) 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56bbfff640 1 -- 172.21.15.133:0/1236886035 learned_addr learned my addr 172.21.15.133:0/1236886035 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56c3fff640 1 --2- 172.21.15.133:0/1236886035 >> v2:172.21.15.162:3300/0 conn(0x7f56c415f260 0x7f56c415f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56c37fe640 1 --2- 172.21.15.133:0/1236886035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c406ac90 0x7f56c415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56bbfff640 1 -- 172.21.15.133:0/1236886035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c406ac90 msgr2=0x7f56c415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56bbfff640 1 --2- 172.21.15.133:0/1236886035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c406ac90 0x7f56c415c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56bbfff640 1 -- 172.21.15.133:0/1236886035 >> v2:172.21.15.162:3300/0 conn(0x7f56c415f260 msgr2=0x7f56c415f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56bbfff640 1 --2- 172.21.15.133:0/1236886035 >> v2:172.21.15.162:3300/0 conn(0x7f56c415f260 0x7f56c415f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56bbfff640 1 -- 172.21.15.133:0/1236886035 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c40b2ca0 con 0x7f56c40b2770 2024-04-25T10:02:43.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.860+0000 7f56c3fff640 1 --2- 172.21.15.133:0/1236886035 >> v2:172.21.15.162:3300/0 conn(0x7f56c415f260 0x7f56c415f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:43.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56bbfff640 1 --2- 172.21.15.133:0/1236886035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c40b2770 0x7f56c406b070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f56c4072220 tx=0x7f56ac01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e0ccd8e5d21119 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c2ffd640 1 -- 172.21.15.133:0/1236886035 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0aa610 con 0x7f56c40b2770 2024-04-25T10:02:43.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c2ffd640 1 -- 172.21.15.133:0/1236886035 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac0af070 con 0x7f56c40b2770 2024-04-25T10:02:43.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c2ffd640 1 -- 172.21.15.133:0/1236886035 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0aa9d0 con 0x7f56c40b2770 2024-04-25T10:02:43.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 -- 172.21.15.133:0/1236886035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c40b2770 msgr2=0x7f56c406b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 --2- 172.21.15.133:0/1236886035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c40b2770 0x7f56c406b070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f56c4072220 tx=0x7f56ac01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 -- 172.21.15.133:0/1236886035 shutdown_connections 2024-04-25T10:02:43.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 --2- 172.21.15.133:0/1236886035 >> v2:172.21.15.162:3300/0 conn(0x7f56c415f260 0x7f56c415f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 --2- 172.21.15.133:0/1236886035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c40b2770 0x7f56c406b070 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 --2- 172.21.15.133:0/1236886035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c406ac90 0x7f56c415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 -- 172.21.15.133:0/1236886035 >> 172.21.15.133:0/1236886035 conn(0x7f56c40999d0 msgr2=0x7f56c40b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 -- 172.21.15.133:0/1236886035 shutdown_connections 2024-04-25T10:02:43.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 -- 172.21.15.133:0/1236886035 wait complete. 2024-04-25T10:02:43.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 Processor -- start 2024-04-25T10:02:43.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 -- start start 2024-04-25T10:02:43.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c406ac90 0x7f56c40d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56c40b2770 0x7f56c40c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c415f260 0x7f56c40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c415d040 con 0x7f56c406ac90 2024-04-25T10:02:43.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c40cc910 con 0x7f56c40b2770 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c9f08640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c415d1c0 con 0x7f56c415f260 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c406ac90 0x7f56c40d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c406ac90 0x7f56c40d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39774/0 (socket says 172.21.15.133:39774) 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c37fe640 1 -- 172.21.15.133:0/893863515 learned_addr learned my addr 172.21.15.133:0/893863515 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56bbfff640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56c40b2770 0x7f56c40c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c37fe640 1 -- 172.21.15.133:0/893863515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c415f260 msgr2=0x7f56c40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c37fe640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c415f260 0x7f56c40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c37fe640 1 -- 172.21.15.133:0/893863515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56c40b2770 msgr2=0x7f56c40c3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c37fe640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56c40b2770 0x7f56c40c3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c37fe640 1 -- 172.21.15.133:0/893863515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c40b4eb0 con 0x7f56c406ac90 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c37fe640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c406ac90 0x7f56c40d0b10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f56b400d5f0 tx=0x7f56b400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.864+0000 7f56c0ff9640 1 -- 172.21.15.133:0/893863515 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b40125f0 con 0x7f56c406ac90 2024-04-25T10:02:43.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.868+0000 7f56c0ff9640 1 -- 172.21.15.133:0/893863515 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b4019070 con 0x7f56c406ac90 2024-04-25T10:02:43.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.868+0000 7f56c9f08640 1 -- 172.21.15.133:0/893863515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c40b5190 con 0x7f56c406ac90 2024-04-25T10:02:43.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.868+0000 7f56c0ff9640 1 -- 172.21.15.133:0/893863515 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b40129d0 con 0x7f56c406ac90 2024-04-25T10:02:43.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.868+0000 7f56c9f08640 1 -- 172.21.15.133:0/893863515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c40c4190 con 0x7f56c406ac90 2024-04-25T10:02:43.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.868+0000 7f56c9f08640 1 -- 172.21.15.133:0/893863515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c4068e80 con 0x7f56c406ac90 2024-04-25T10:02:43.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.868+0000 7f56c0ff9640 1 -- 172.21.15.133:0/893863515 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56b4012b70 con 0x7f56c406ac90 2024-04-25T10:02:43.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.868+0000 7f56c0ff9640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a4066f60 0x7f56a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.868+0000 7f56c0ff9640 1 -- 172.21.15.133:0/893863515 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f56b4085830 con 0x7f56c406ac90 2024-04-25T10:02:43.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.868+0000 7f56bbfff640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a4066f60 0x7f56a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.868+0000 7f56bbfff640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a4066f60 0x7f56a4069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0ae000 tx=0x7f56ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:43.872+0000 7f56c0ff9640 1 -- 172.21.15.133:0/893863515 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f56b40564d0 con 0x7f56c406ac90 2024-04-25T10:02:44.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.020+0000 7f56c9f08640 1 -- 172.21.15.133:0/893863515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f56c415fa10 con 0x7f56c406ac90 2024-04-25T10:02:44.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.020+0000 7f56c0ff9640 1 -- 172.21.15.133:0/893863515 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f56b4055dd0 con 0x7f56c406ac90 2024-04-25T10:02:44.027 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 -- 172.21.15.133:0/893863515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a4066f60 msgr2=0x7f56a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a4066f60 0x7f56a4069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0ae000 tx=0x7f56ac008040 comp rx=0 tx=0).stop 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 -- 172.21.15.133:0/893863515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c406ac90 msgr2=0x7f56c40d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c406ac90 0x7f56c40d0b10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f56b400d5f0 tx=0x7f56b400dab0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 -- 172.21.15.133:0/893863515 shutdown_connections 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c415f260 0x7f56c40c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56c40b2770 0x7f56c40c3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a4066f60 0x7f56a4069420 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 --2- 172.21.15.133:0/893863515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c406ac90 0x7f56c40d0b10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 -- 172.21.15.133:0/893863515 >> 172.21.15.133:0/893863515 conn(0x7f56c40999d0 msgr2=0x7f56c41554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 -- 172.21.15.133:0/893863515 shutdown_connections 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.024+0000 7f56c9f08640 1 -- 172.21.15.133:0/893863515 wait complete. 2024-04-25T10:02:44.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:44.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.152+0000 7fb95f3fa640 1 Processor -- start 2024-04-25T10:02:44.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.152+0000 7fb95f3fa640 1 -- start start 2024-04-25T10:02:44.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95f3fa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb95806eb70 0x7fb9580b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95f3fa640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb9581584f0 0x7fb95815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95f3fa640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9580cbe60 con 0x7fb958158110 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95f3fa640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9580b19b0 con 0x7fb9581584f0 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95f3fa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95815b360 con 0x7fb95806eb70 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95d196640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb958158110 0x7fb95806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53734/0 (socket says 172.21.15.133:53734) 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95d196640 1 -- 172.21.15.133:0/3002703309 learned_addr learned my addr 172.21.15.133:0/3002703309 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95c995640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb95806eb70 0x7fb9580b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95d997640 1 --2- 172.21.15.133:0/3002703309 >> v2:172.21.15.162:3300/0 conn(0x7fb9581584f0 0x7fb95815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455012031 0 0) 0x7fb9580cbe60 con 0x7fb958158110 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb944003530 con 0x7fb958158110 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3993067731 0 0) 0x7fb944003530 con 0x7fb958158110 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb95806eb70 msgr2=0x7fb9580b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb95806eb70 0x7fb9580b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 >> v2:172.21.15.162:3300/0 conn(0x7fb9581584f0 msgr2=0x7fb95815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 --2- 172.21.15.133:0/3002703309 >> v2:172.21.15.162:3300/0 conn(0x7fb9581584f0 0x7fb95815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95815b640 con 0x7fb958158110 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb948001680 con 0x7fb958158110 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 >> v1:172.21.15.133:6789/0 conn(0x7fb958158110 legacy=0x7fb95806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb95806eb70 0x7fb9440039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb944003f40 0x7fb944004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb944004d50 0x7fb9440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580b19b0 con 0x7fb944003f40 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95815b360 con 0x7fb944004d50 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94400b9d0 con 0x7fb95806eb70 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95c995640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb944003f40 0x7fb944004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95d196640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb95806eb70 0x7fb9440039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95d997640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb944004d50 0x7fb9440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95c995640 1 -- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb95806eb70 msgr2=0x7fb9440039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95c995640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb95806eb70 0x7fb9440039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95c995640 1 -- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb944004d50 msgr2=0x7fb9440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95c995640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb944004d50 0x7fb9440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95c995640 1 -- 172.21.15.133:0/3002703309 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb944003530 con 0x7fb944003f40 2024-04-25T10:02:44.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95d196640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb95806eb70 0x7fb9440039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:44.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.156+0000 7fb95d997640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb944004d50 0x7fb9440051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:44.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95c995640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb944003f40 0x7fb944004810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb954009ac0 tx=0x7fb95401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f31392f1493f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9540b2070 con 0x7fb944003f40 2024-04-25T10:02:44.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9540a5640 con 0x7fb944003f40 2024-04-25T10:02:44.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb94ffff640 1 -- 172.21.15.133:0/3002703309 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954009690 con 0x7fb944003f40 2024-04-25T10:02:44.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3002703309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb944003f40 msgr2=0x7fb944004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 --2- 172.21.15.133:0/3002703309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb944003f40 0x7fb944004810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb954009ac0 tx=0x7fb95401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95d997640 1 -- 172.21.15.133:0/3002703309 reap_dead start 2024-04-25T10:02:44.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3002703309 shutdown_connections 2024-04-25T10:02:44.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3002703309 >> 172.21.15.133:0/3002703309 conn(0x7fb958067000 msgr2=0x7fb95806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3002703309 shutdown_connections 2024-04-25T10:02:44.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3002703309 wait complete. 2024-04-25T10:02:44.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 Processor -- start 2024-04-25T10:02:44.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 -- start start 2024-04-25T10:02:44.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb944003f40 0x7fb95815cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb95813d830 0x7fb95815e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb95814e640 0x7fb958150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580cd670 con 0x7fb95813d830 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95815bc00 con 0x7fb95814e640 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580cd7f0 con 0x7fb944003f40 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95c995640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb95813d830 0x7fb95815e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95c995640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb95813d830 0x7fb95815e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39796/0 (socket says 172.21.15.133:39796) 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95c995640 1 -- 172.21.15.133:0/3097039110 learned_addr learned my addr 172.21.15.133:0/3097039110 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95d997640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb95814e640 0x7fb958150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95d196640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb944003f40 0x7fb95815cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95c995640 1 -- 172.21.15.133:0/3097039110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb944003f40 msgr2=0x7fb95815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95c995640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb944003f40 0x7fb95815cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95c995640 1 -- 172.21.15.133:0/3097039110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb95814e640 msgr2=0x7fb958150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95c995640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb95814e640 0x7fb958150080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95c995640 1 -- 172.21.15.133:0/3097039110 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb958067d60 con 0x7fb95813d830 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95c995640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb95813d830 0x7fb95815e640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb9540b0040 tx=0x7fb944005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb94dffb640 1 -- 172.21.15.133:0/3097039110 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9540b2040 con 0x7fb95813d830 2024-04-25T10:02:44.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb94dffb640 1 -- 172.21.15.133:0/3097039110 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9540ad440 con 0x7fb95813d830 2024-04-25T10:02:44.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3097039110 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95815d5a0 con 0x7fb95813d830 2024-04-25T10:02:44.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb94dffb640 1 -- 172.21.15.133:0/3097039110 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954009690 con 0x7fb95813d830 2024-04-25T10:02:44.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.160+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3097039110 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95815db60 con 0x7fb95813d830 2024-04-25T10:02:44.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.164+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3097039110 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95806bc20 con 0x7fb95813d830 2024-04-25T10:02:44.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.164+0000 7fb94dffb640 1 -- 172.21.15.133:0/3097039110 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9540b5070 con 0x7fb95813d830 2024-04-25T10:02:44.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.164+0000 7fb94dffb640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb93c066f90 0x7fb93c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.164+0000 7fb94dffb640 1 -- 172.21.15.133:0/3097039110 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb95411f990 con 0x7fb95813d830 2024-04-25T10:02:44.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.164+0000 7fb95d196640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb93c066f90 0x7fb93c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.164+0000 7fb95d196640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb93c066f90 0x7fb93c069450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb948001210 tx=0x7fb948001a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.168+0000 7fb94dffb640 1 -- 172.21.15.133:0/3097039110 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9540bc020 con 0x7fb95813d830 2024-04-25T10:02:44.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.328+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3097039110 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb958068ef0 con 0x7fb95813d830 2024-04-25T10:02:44.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb94dffb640 1 -- 172.21.15.133:0/3097039110 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7fb954016760 con 0x7fb95813d830 2024-04-25T10:02:44.338 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:44.338 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3097039110 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb93c066f90 msgr2=0x7fb93c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb93c066f90 0x7fb93c069450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb948001210 tx=0x7fb948001a50 comp rx=0 tx=0).stop 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3097039110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb95813d830 msgr2=0x7fb95815e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb95813d830 0x7fb95815e640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb9540b0040 tx=0x7fb944005840 comp rx=0 tx=0).stop 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3097039110 shutdown_connections 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb95814e640 0x7fb958150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb93c066f90 0x7fb93c069450 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb95813d830 0x7fb95815e640 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 --2- 172.21.15.133:0/3097039110 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb944003f40 0x7fb95815cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3097039110 >> 172.21.15.133:0/3097039110 conn(0x7fb958067000 msgr2=0x7fb958157430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3097039110 shutdown_connections 2024-04-25T10:02:44.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.332+0000 7fb95f3fa640 1 -- 172.21.15.133:0/3097039110 wait complete. 2024-04-25T10:02:44.356 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:44.357 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:44.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6eb1a640 1 Processor -- start 2024-04-25T10:02:44.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6eb1a640 1 -- start start 2024-04-25T10:02:44.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6eb1a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fde6806e8b0 0x7fde6815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6eb1a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fde680b1e50 0x7fde68146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6eb1a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde680b1090 con 0x7fde680aebc0 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6eb1a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde68147490 con 0x7fde6806e8b0 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6eb1a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde680b19c0 con 0x7fde680b1e50 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6c8b6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fde6806e8b0 0x7fde6815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6d0b7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fde680aebc0 0x7fde6806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53736/0 (socket says 172.21.15.133:53736) 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6d0b7640 1 -- 172.21.15.133:0/3499215513 learned_addr learned my addr 172.21.15.133:0/3499215513 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde5ffff640 1 --2- 172.21.15.133:0/3499215513 >> v2:172.21.15.162:3300/0 conn(0x7fde680b1e50 0x7fde68146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6c8b6640 1 -- 172.21.15.133:0/3499215513 >> v2:172.21.15.162:3300/0 conn(0x7fde680b1e50 msgr2=0x7fde68146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6c8b6640 1 --2- 172.21.15.133:0/3499215513 >> v2:172.21.15.162:3300/0 conn(0x7fde680b1e50 0x7fde68146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6c8b6640 1 -- 172.21.15.133:0/3499215513 >> v1:172.21.15.133:6789/0 conn(0x7fde680aebc0 legacy=0x7fde6806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6c8b6640 1 -- 172.21.15.133:0/3499215513 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde680b1370 con 0x7fde6806e8b0 2024-04-25T10:02:44.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde6c8b6640 1 --2- 172.21.15.133:0/3499215513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde6806e8b0 0x7fde6815eb40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fde680b4e30 tx=0x7fde6401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7491718a20755b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde5f7fe640 1 -- 172.21.15.133:0/3499215513 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde640b0070 con 0x7fde6806e8b0 2024-04-25T10:02:44.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde5f7fe640 1 -- 172.21.15.133:0/3499215513 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde640ab600 con 0x7fde6806e8b0 2024-04-25T10:02:44.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.472+0000 7fde5f7fe640 1 -- 172.21.15.133:0/3499215513 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde640ab9d0 con 0x7fde6806e8b0 2024-04-25T10:02:44.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 -- 172.21.15.133:0/3499215513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde6806e8b0 msgr2=0x7fde6815eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 --2- 172.21.15.133:0/3499215513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde6806e8b0 0x7fde6815eb40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fde680b4e30 tx=0x7fde6401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 -- 172.21.15.133:0/3499215513 shutdown_connections 2024-04-25T10:02:44.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 --2- 172.21.15.133:0/3499215513 >> v2:172.21.15.162:3300/0 conn(0x7fde680b1e50 0x7fde68146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 --2- 172.21.15.133:0/3499215513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde6806e8b0 0x7fde6815eb40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 -- 172.21.15.133:0/3499215513 >> 172.21.15.133:0/3499215513 conn(0x7fde68066f70 msgr2=0x7fde6806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 -- 172.21.15.133:0/3499215513 shutdown_connections 2024-04-25T10:02:44.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 -- 172.21.15.133:0/3499215513 wait complete. 2024-04-25T10:02:44.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 Processor -- start 2024-04-25T10:02:44.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 -- start start 2024-04-25T10:02:44.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde6806e8b0 0x7fde6813ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde680aebc0 0x7fde6815df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde680b1e50 0x7fde6815e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde680b23b0 con 0x7fde680aebc0 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde680b2230 con 0x7fde680b1e50 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde68147020 con 0x7fde6806e8b0 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde680aebc0 0x7fde6815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde680aebc0 0x7fde6815df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39808/0 (socket says 172.21.15.133:39808) 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5ffff640 1 -- 172.21.15.133:0/4001637167 learned_addr learned my addr 172.21.15.133:0/4001637167 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6c8b6640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde6806e8b0 0x7fde6813ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6d0b7640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde680b1e50 0x7fde6815e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5ffff640 1 -- 172.21.15.133:0/4001637167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde6806e8b0 msgr2=0x7fde6813ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5ffff640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde6806e8b0 0x7fde6813ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5ffff640 1 -- 172.21.15.133:0/4001637167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde680b1e50 msgr2=0x7fde6815e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5ffff640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde680b1e50 0x7fde6815e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5ffff640 1 -- 172.21.15.133:0/4001637167 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde68144600 con 0x7fde680aebc0 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6c8b6640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde6806e8b0 0x7fde6813ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6d0b7640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde680b1e50 0x7fde6815e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5ffff640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde680aebc0 0x7fde6815df20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fde5800c7f0 tx=0x7fde5800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5d7fa640 1 -- 172.21.15.133:0/4001637167 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58015070 con 0x7fde680aebc0 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5d7fa640 1 -- 172.21.15.133:0/4001637167 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58004240 con 0x7fde680aebc0 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 -- 172.21.15.133:0/4001637167 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde68145b30 con 0x7fde680aebc0 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde5d7fa640 1 -- 172.21.15.133:0/4001637167 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58003620 con 0x7fde680aebc0 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.476+0000 7fde6eb1a640 1 -- 172.21.15.133:0/4001637167 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde68144b60 con 0x7fde680aebc0 2024-04-25T10:02:44.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.480+0000 7fde6eb1a640 1 -- 172.21.15.133:0/4001637167 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6813ab40 con 0x7fde680aebc0 2024-04-25T10:02:44.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.480+0000 7fde5d7fa640 1 -- 172.21.15.133:0/4001637167 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde580039d0 con 0x7fde680aebc0 2024-04-25T10:02:44.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.480+0000 7fde5d7fa640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde54066f60 0x7fde54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.480+0000 7fde5d7fa640 1 -- 172.21.15.133:0/4001637167 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fde58085480 con 0x7fde680aebc0 2024-04-25T10:02:44.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.480+0000 7fde6c8b6640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde54066f60 0x7fde54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.480+0000 7fde6c8b6640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde54066f60 0x7fde54069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde640ad000 tx=0x7fde64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.484+0000 7fde5d7fa640 1 -- 172.21.15.133:0/4001637167 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde68144600 con 0x7fde680aebc0 2024-04-25T10:02:44.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.640+0000 7fde6eb1a640 1 -- 172.21.15.133:0/4001637167 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde6806b960 con 0x7fde680aebc0 2024-04-25T10:02:44.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.640+0000 7fde5d7fa640 1 -- 172.21.15.133:0/4001637167 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde58056120 con 0x7fde680aebc0 2024-04-25T10:02:44.649 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 -- 172.21.15.133:0/4001637167 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde54066f60 msgr2=0x7fde54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde54066f60 0x7fde54069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde640ad000 tx=0x7fde64008040 comp rx=0 tx=0).stop 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 -- 172.21.15.133:0/4001637167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde680aebc0 msgr2=0x7fde6815df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde680aebc0 0x7fde6815df20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fde5800c7f0 tx=0x7fde5800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 -- 172.21.15.133:0/4001637167 shutdown_connections 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde680b1e50 0x7fde6815e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde54066f60 0x7fde54069420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde680aebc0 0x7fde6815df20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 --2- 172.21.15.133:0/4001637167 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde6806e8b0 0x7fde6813ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 -- 172.21.15.133:0/4001637167 >> 172.21.15.133:0/4001637167 conn(0x7fde68066f70 msgr2=0x7fde6813fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 -- 172.21.15.133:0/4001637167 shutdown_connections 2024-04-25T10:02:44.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.644+0000 7fde6eb1a640 1 -- 172.21.15.133:0/4001637167 wait complete. 2024-04-25T10:02:44.667 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:44.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecfb1b640 1 Processor -- start 2024-04-25T10:02:44.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecfb1b640 1 -- start start 2024-04-25T10:02:44.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecfb1b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ec815eb40 0x7f3ec80b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecfb1b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec81471f0 con 0x7f3ec806e8b0 2024-04-25T10:02:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecfb1b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec80b1090 con 0x7f3ec80aebc0 2024-04-25T10:02:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecfb1b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec8147490 con 0x7f3ec815eb40 2024-04-25T10:02:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecd8b7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3ec806e8b0 0x7f3ec8146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53738/0 (socket says 172.21.15.133:53738) 2024-04-25T10:02:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecd8b7640 1 -- 172.21.15.133:0/1122999429 learned_addr learned my addr 172.21.15.133:0/1122999429 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecd0b6640 1 --2- 172.21.15.133:0/1122999429 >> v2:172.21.15.133:3300/0 conn(0x7f3ec815eb40 0x7f3ec80b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecc8b5640 1 -- 172.21.15.133:0/1122999429 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925312074 0 0) 0x7f3ec81471f0 con 0x7f3ec806e8b0 2024-04-25T10:02:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecc8b5640 1 -- 172.21.15.133:0/1122999429 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3eb0003530 con 0x7f3ec806e8b0 2024-04-25T10:02:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecd0b6640 1 -- 172.21.15.133:0/1122999429 >> v1:172.21.15.162:6789/0 conn(0x7f3ec80aebc0 legacy=0x7f3ec806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecd0b6640 1 -- 172.21.15.133:0/1122999429 >> v1:172.21.15.133:6789/0 conn(0x7f3ec806e8b0 legacy=0x7f3ec8146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.780+0000 7f3ecd0b6640 1 -- 172.21.15.133:0/1122999429 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec80b1370 con 0x7f3ec815eb40 2024-04-25T10:02:44.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecd0b6640 1 --2- 172.21.15.133:0/1122999429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ec815eb40 0x7f3ec80b1e50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0097c0 tx=0x7f3ebc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6745251d3bfc438e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecc8b5640 1 -- 172.21.15.133:0/1122999429 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0b0070 con 0x7f3ec815eb40 2024-04-25T10:02:44.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecc8b5640 1 -- 172.21.15.133:0/1122999429 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc0ab600 con 0x7f3ec815eb40 2024-04-25T10:02:44.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecc8b5640 1 -- 172.21.15.133:0/1122999429 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0ab9d0 con 0x7f3ec815eb40 2024-04-25T10:02:44.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1122999429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ec815eb40 msgr2=0x7f3ec80b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecfb1b640 1 --2- 172.21.15.133:0/1122999429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ec815eb40 0x7f3ec80b1e50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0097c0 tx=0x7f3ebc01c410 comp rx=0 tx=0).stop 2024-04-25T10:02:44.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1122999429 shutdown_connections 2024-04-25T10:02:44.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecfb1b640 1 --2- 172.21.15.133:0/1122999429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ec815eb40 0x7f3ec80b1e50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1122999429 >> 172.21.15.133:0/1122999429 conn(0x7f3ec8066f70 msgr2=0x7f3ec806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1122999429 shutdown_connections 2024-04-25T10:02:44.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1122999429 wait complete. 2024-04-25T10:02:44.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecfb1b640 1 Processor -- start 2024-04-25T10:02:44.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.784+0000 7f3ecfb1b640 1 -- start start 2024-04-25T10:02:44.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecfb1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ec806e8b0 0x7f3ec81f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecfb1b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec80aebc0 0x7f3ec80accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecfb1b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ec815eb40 0x7f3ec80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecfb1b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec816c0b0 con 0x7f3ec806e8b0 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecfb1b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec80b2390 con 0x7f3ec80aebc0 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecfb1b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec816c230 con 0x7f3ec815eb40 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd8b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ec806e8b0 0x7f3ec81f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd8b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ec806e8b0 0x7f3ec81f80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39826/0 (socket says 172.21.15.133:39826) 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd8b7640 1 -- 172.21.15.133:0/1184367630 learned_addr learned my addr 172.21.15.133:0/1184367630 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd0b6640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec80aebc0 0x7f3ec80accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd8b7640 1 -- 172.21.15.133:0/1184367630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ec815eb40 msgr2=0x7f3ec80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd8b7640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ec815eb40 0x7f3ec80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd8b7640 1 -- 172.21.15.133:0/1184367630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec80aebc0 msgr2=0x7f3ec80accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd8b7640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec80aebc0 0x7f3ec80accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd8b7640 1 -- 172.21.15.133:0/1184367630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec8142d50 con 0x7f3ec806e8b0 2024-04-25T10:02:44.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd0b6640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec80aebc0 0x7f3ec80accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:44.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd8b7640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ec806e8b0 0x7f3ec81f80e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800cbf0 tx=0x7f3eb800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ec67fc640 1 -- 172.21.15.133:0/1184367630 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8017070 con 0x7f3ec806e8b0 2024-04-25T10:02:44.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ec67fc640 1 -- 172.21.15.133:0/1184367630 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb800ad50 con 0x7f3ec806e8b0 2024-04-25T10:02:44.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1184367630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec8143030 con 0x7f3ec806e8b0 2024-04-25T10:02:44.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ec67fc640 1 -- 172.21.15.133:0/1184367630 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8012620 con 0x7f3ec806e8b0 2024-04-25T10:02:44.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1184367630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec81435f0 con 0x7f3ec806e8b0 2024-04-25T10:02:44.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1184367630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec8068e80 con 0x7f3ec806e8b0 2024-04-25T10:02:44.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ec67fc640 1 -- 172.21.15.133:0/1184367630 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3eb801a070 con 0x7f3ec806e8b0 2024-04-25T10:02:44.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ec67fc640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eb4066f60 0x7f3eb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ec67fc640 1 -- 172.21.15.133:0/1184367630 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3eb80852d0 con 0x7f3ec806e8b0 2024-04-25T10:02:44.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd0b6640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eb4066f60 0x7f3eb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.788+0000 7f3ecd0b6640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eb4066f60 0x7f3eb4069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0ad000 tx=0x7f3ebc0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.792+0000 7f3ec67fc640 1 -- 172.21.15.133:0/1184367630 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3eb8055ee0 con 0x7f3ec806e8b0 2024-04-25T10:02:44.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.948+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1184367630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ec806d790 con 0x7f3ec806e8b0 2024-04-25T10:02:44.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ec67fc640 1 -- 172.21.15.133:0/1184367630 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3eb80850f0 con 0x7f3ec806e8b0 2024-04-25T10:02:44.958 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1184367630 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eb4066f60 msgr2=0x7f3eb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eb4066f60 0x7f3eb4069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0ad000 tx=0x7f3ebc0a6000 comp rx=0 tx=0).stop 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1184367630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ec806e8b0 msgr2=0x7f3ec81f80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ec806e8b0 0x7f3ec81f80e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800cbf0 tx=0x7f3eb800a430 comp rx=0 tx=0).stop 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1184367630 shutdown_connections 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ec815eb40 0x7f3ec80ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ec80aebc0 0x7f3ec80accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eb4066f60 0x7f3eb4069420 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 --2- 172.21.15.133:0/1184367630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ec806e8b0 0x7f3ec81f80e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1184367630 >> 172.21.15.133:0/1184367630 conn(0x7f3ec8066f70 msgr2=0x7f3ec815f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1184367630 shutdown_connections 2024-04-25T10:02:44.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:44.952+0000 7f3ecfb1b640 1 -- 172.21.15.133:0/1184367630 wait complete. 2024-04-25T10:02:44.976 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:45.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.088+0000 7f803a154640 1 Processor -- start 2024-04-25T10:02:45.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.088+0000 7f803a154640 1 -- start start 2024-04-25T10:02:45.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f803a154640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f80340cd820 0x7f80340cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f803a154640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f803415ced0 0x7f803415d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f803a154640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340ce2e0 con 0x7f8034145170 2024-04-25T10:02:45.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f803a154640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803406db00 con 0x7f80340cd820 2024-04-25T10:02:45.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f803a154640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8034145550 con 0x7f803415ced0 2024-04-25T10:02:45.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f8032ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f80340cd820 0x7f80340cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f8032ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f80340cd820 0x7f80340cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39830/0 (socket says 172.21.15.133:39830) 2024-04-25T10:02:45.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f8032ffd640 1 -- 172.21.15.133:0/3749856881 learned_addr learned my addr 172.21.15.133:0/3749856881 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:45.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f8033fff640 1 --2- 172.21.15.133:0/3749856881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f803415ced0 0x7f803415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f8032ffd640 1 -- 172.21.15.133:0/3749856881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f803415ced0 msgr2=0x7f803415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f8032ffd640 1 --2- 172.21.15.133:0/3749856881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f803415ced0 0x7f803415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f8032ffd640 1 -- 172.21.15.133:0/3749856881 >> v1:172.21.15.162:6789/0 conn(0x7f8034145170 legacy=0x7f803406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f8032ffd640 1 -- 172.21.15.133:0/3749856881 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803406d4b0 con 0x7f80340cd820 2024-04-25T10:02:45.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f8032ffd640 1 --2- 172.21.15.133:0/3749856881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80340cd820 0x7f80340cdca0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8028009870 tx=0x7f802801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d8358173e21a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f80327fc640 1 -- 172.21.15.133:0/3749856881 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280b0070 con 0x7f80340cd820 2024-04-25T10:02:45.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f80327fc640 1 -- 172.21.15.133:0/3749856881 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80280ab600 con 0x7f80340cd820 2024-04-25T10:02:45.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.092+0000 7f80327fc640 1 -- 172.21.15.133:0/3749856881 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280ab9d0 con 0x7f80340cd820 2024-04-25T10:02:45.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- 172.21.15.133:0/3749856881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80340cd820 msgr2=0x7f80340cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 --2- 172.21.15.133:0/3749856881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80340cd820 0x7f80340cdca0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8028009870 tx=0x7f802801c520 comp rx=0 tx=0).stop 2024-04-25T10:02:45.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- 172.21.15.133:0/3749856881 shutdown_connections 2024-04-25T10:02:45.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 --2- 172.21.15.133:0/3749856881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f803415ced0 0x7f803415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 --2- 172.21.15.133:0/3749856881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80340cd820 0x7f80340cdca0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- 172.21.15.133:0/3749856881 >> 172.21.15.133:0/3749856881 conn(0x7f8034066f70 msgr2=0x7f803406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- 172.21.15.133:0/3749856881 shutdown_connections 2024-04-25T10:02:45.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- 172.21.15.133:0/3749856881 wait complete. 2024-04-25T10:02:45.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 Processor -- start 2024-04-25T10:02:45.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- start start 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80340cd820 0x7f803413da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8034145170 0x7f8034162190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f803415ced0 0x7f80341431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8034168610 con 0x7f80340cd820 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8034168490 con 0x7f8034145170 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8034168790 con 0x7f803415ced0 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f80337fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80340cd820 0x7f803413da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f80337fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80340cd820 0x7f803413da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39832/0 (socket says 172.21.15.133:39832) 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f80337fe640 1 -- 172.21.15.133:0/1569480012 learned_addr learned my addr 172.21.15.133:0/1569480012 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f8032ffd640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8034145170 0x7f8034162190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f8033fff640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f803415ced0 0x7f80341431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f80337fe640 1 -- 172.21.15.133:0/1569480012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f803415ced0 msgr2=0x7f80341431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f80337fe640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f803415ced0 0x7f80341431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f80337fe640 1 -- 172.21.15.133:0/1569480012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8034145170 msgr2=0x7f8034162190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f80337fe640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8034145170 0x7f8034162190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f80337fe640 1 -- 172.21.15.133:0/1569480012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8034162c50 con 0x7f80340cd820 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f8033fff640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f803415ced0 0x7f80341431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f80337fe640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80340cd820 0x7f803413da70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f802400b410 tx=0x7f802400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f801bfff640 1 -- 172.21.15.133:0/1569480012 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024015030 con 0x7f80340cd820 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f801bfff640 1 -- 172.21.15.133:0/1569480012 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8024004020 con 0x7f80340cd820 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- 172.21.15.133:0/1569480012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8034162f30 con 0x7f80340cd820 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f801bfff640 1 -- 172.21.15.133:0/1569480012 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024004410 con 0x7f80340cd820 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- 172.21.15.133:0/1569480012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80341634f0 con 0x7f80340cd820 2024-04-25T10:02:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.096+0000 7f803a154640 1 -- 172.21.15.133:0/1569480012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8034068e80 con 0x7f80340cd820 2024-04-25T10:02:45.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.100+0000 7f801bfff640 1 -- 172.21.15.133:0/1569480012 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8024018070 con 0x7f80340cd820 2024-04-25T10:02:45.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.100+0000 7f801bfff640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8020066f60 0x7f8020069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.100+0000 7f801bfff640 1 -- 172.21.15.133:0/1569480012 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8024084fd0 con 0x7f80340cd820 2024-04-25T10:02:45.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.100+0000 7f8032ffd640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8020066f60 0x7f8020069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.100+0000 7f8032ffd640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8020066f60 0x7f8020069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f80280ad000 tx=0x7f80280a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.104+0000 7f801bfff640 1 -- 172.21.15.133:0/1569480012 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8024055be0 con 0x7f80340cd820 2024-04-25T10:02:45.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 -- 172.21.15.133:0/1569480012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f803406e250 con 0x7f80340cd820 2024-04-25T10:02:45.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f801bfff640 1 -- 172.21.15.133:0/1569480012 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80240554e0 con 0x7f80340cd820 2024-04-25T10:02:45.263 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:45.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 -- 172.21.15.133:0/1569480012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8020066f60 msgr2=0x7f8020069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8020066f60 0x7f8020069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f80280ad000 tx=0x7f80280a6000 comp rx=0 tx=0).stop 2024-04-25T10:02:45.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 -- 172.21.15.133:0/1569480012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80340cd820 msgr2=0x7f803413da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80340cd820 0x7f803413da70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f802400b410 tx=0x7f802400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 -- 172.21.15.133:0/1569480012 shutdown_connections 2024-04-25T10:02:45.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f803415ced0 0x7f80341431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8034145170 0x7f8034162190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8020066f60 0x7f8020069420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 --2- 172.21.15.133:0/1569480012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80340cd820 0x7f803413da70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 -- 172.21.15.133:0/1569480012 >> 172.21.15.133:0/1569480012 conn(0x7f8034066f70 msgr2=0x7f803415eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 -- 172.21.15.133:0/1569480012 shutdown_connections 2024-04-25T10:02:45.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.256+0000 7f803a154640 1 -- 172.21.15.133:0/1569480012 wait complete. 2024-04-25T10:02:45.280 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:45.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630c683640 1 Processor -- start 2024-04-25T10:02:45.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630c683640 1 -- start start 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630c683640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f630409a7c0 0x7f630415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630c683640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f630409a0e0 0x7f63040cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630c683640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63040cc5a0 con 0x7f630415bab0 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630c683640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63040cc880 con 0x7f630409a0e0 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630c683640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63040ccb60 con 0x7f630409a7c0 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630ac20640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f630409a0e0 0x7f63040cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630ac20640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f630409a0e0 0x7f63040cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39840/0 (socket says 172.21.15.133:39840) 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630ac20640 1 -- 172.21.15.133:0/1323611225 learned_addr learned my addr 172.21.15.133:0/1323611225 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630ac20640 1 -- 172.21.15.133:0/1323611225 >> v2:172.21.15.162:3300/0 conn(0x7f630409a7c0 msgr2=0x7f630415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f6309c1e640 1 --2- 172.21.15.133:0/1323611225 >> v2:172.21.15.162:3300/0 conn(0x7f630409a7c0 0x7f630415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630ac20640 1 --2- 172.21.15.133:0/1323611225 >> v2:172.21.15.162:3300/0 conn(0x7f630409a7c0 0x7f630415c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630ac20640 1 -- 172.21.15.133:0/1323611225 >> v1:172.21.15.162:6789/0 conn(0x7f630415bab0 legacy=0x7f63040999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f630ac20640 1 -- 172.21.15.133:0/1323611225 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63040cce40 con 0x7f630409a0e0 2024-04-25T10:02:45.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.384+0000 7f6309c1e640 1 --2- 172.21.15.133:0/1323611225 >> v2:172.21.15.162:3300/0 conn(0x7f630409a7c0 0x7f630415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:45.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630ac20640 1 --2- 172.21.15.133:0/1323611225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630409a0e0 0x7f63040cf280 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f63040732f0 tx=0x7f62fc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac08eca9cbb1e326 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630941d640 1 -- 172.21.15.133:0/1323611225 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0b0070 con 0x7f630409a0e0 2024-04-25T10:02:45.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630941d640 1 -- 172.21.15.133:0/1323611225 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc0ab600 con 0x7f630409a0e0 2024-04-25T10:02:45.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630941d640 1 -- 172.21.15.133:0/1323611225 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0ab9d0 con 0x7f630409a0e0 2024-04-25T10:02:45.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630c683640 1 -- 172.21.15.133:0/1323611225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630409a0e0 msgr2=0x7f63040cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630c683640 1 --2- 172.21.15.133:0/1323611225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630409a0e0 0x7f63040cf280 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f63040732f0 tx=0x7f62fc01c450 comp rx=0 tx=0).stop 2024-04-25T10:02:45.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630c683640 1 -- 172.21.15.133:0/1323611225 shutdown_connections 2024-04-25T10:02:45.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630c683640 1 --2- 172.21.15.133:0/1323611225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630409a0e0 0x7f63040cf280 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630c683640 1 --2- 172.21.15.133:0/1323611225 >> v2:172.21.15.162:3300/0 conn(0x7f630409a7c0 0x7f630415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630c683640 1 -- 172.21.15.133:0/1323611225 >> 172.21.15.133:0/1323611225 conn(0x7f63040b31a0 msgr2=0x7f63040b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630c683640 1 -- 172.21.15.133:0/1323611225 shutdown_connections 2024-04-25T10:02:45.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630c683640 1 -- 172.21.15.133:0/1323611225 wait complete. 2024-04-25T10:02:45.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630c683640 1 Processor -- start 2024-04-25T10:02:45.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.388+0000 7f630c683640 1 -- start start 2024-04-25T10:02:45.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630c683640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630409a0e0 0x7f63040c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630c683640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630409a7c0 0x7f63040cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630c683640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630415bab0 0x7f63040c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630c683640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63040d0360 con 0x7f630409a0e0 2024-04-25T10:02:45.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630c683640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63040d01e0 con 0x7f630415bab0 2024-04-25T10:02:45.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630c683640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63041687d0 con 0x7f630409a7c0 2024-04-25T10:02:45.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630a41f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630409a0e0 0x7f63040c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630a41f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630409a0e0 0x7f63040c4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39852/0 (socket says 172.21.15.133:39852) 2024-04-25T10:02:45.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630a41f640 1 -- 172.21.15.133:0/3994902956 learned_addr learned my addr 172.21.15.133:0/3994902956 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:45.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630a41f640 1 -- 172.21.15.133:0/3994902956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630409a7c0 msgr2=0x7f63040cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630a41f640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630409a7c0 0x7f63040cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630a41f640 1 -- 172.21.15.133:0/3994902956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630415bab0 msgr2=0x7f63040c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630a41f640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630415bab0 0x7f63040c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630a41f640 1 -- 172.21.15.133:0/3994902956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304157fd0 con 0x7f630409a0e0 2024-04-25T10:02:45.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630a41f640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630409a0e0 0x7f63040c4b80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f630000ba50 tx=0x7f630000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f62faffd640 1 -- 172.21.15.133:0/3994902956 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300019070 con 0x7f630409a0e0 2024-04-25T10:02:45.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f62faffd640 1 -- 172.21.15.133:0/3994902956 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630000cbc0 con 0x7f630409a0e0 2024-04-25T10:02:45.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f62faffd640 1 -- 172.21.15.133:0/3994902956 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300014440 con 0x7f630409a0e0 2024-04-25T10:02:45.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630c683640 1 -- 172.21.15.133:0/3994902956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63040c7e60 con 0x7f630409a0e0 2024-04-25T10:02:45.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630c683640 1 -- 172.21.15.133:0/3994902956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6304158270 con 0x7f630409a0e0 2024-04-25T10:02:45.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f630c683640 1 -- 172.21.15.133:0/3994902956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6304068e80 con 0x7f630409a0e0 2024-04-25T10:02:45.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f62faffd640 1 -- 172.21.15.133:0/3994902956 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f630001c070 con 0x7f630409a0e0 2024-04-25T10:02:45.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f62faffd640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f90 0x7f62e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f62faffd640 1 -- 172.21.15.133:0/3994902956 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6300085060 con 0x7f630409a0e0 2024-04-25T10:02:45.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.392+0000 7f6309c1e640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f90 0x7f62e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.396+0000 7f6309c1e640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f90 0x7f62e8069450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f630415cde0 tx=0x7f62f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.396+0000 7f62faffd640 1 -- 172.21.15.133:0/3994902956 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6300055610 con 0x7f630409a0e0 2024-04-25T10:02:45.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.544+0000 7f630c683640 1 -- 172.21.15.133:0/3994902956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f63040b7630 con 0x7f630409a0e0 2024-04-25T10:02:45.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.544+0000 7f62faffd640 1 -- 172.21.15.133:0/3994902956 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f63000596b0 con 0x7f630409a0e0 2024-04-25T10:02:45.552 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 -- 172.21.15.133:0/3994902956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f90 msgr2=0x7f62e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f90 0x7f62e8069450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f630415cde0 tx=0x7f62f4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 -- 172.21.15.133:0/3994902956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630409a0e0 msgr2=0x7f63040c4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630409a0e0 0x7f63040c4b80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f630000ba50 tx=0x7f630000bf10 comp rx=0 tx=0).stop 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 -- 172.21.15.133:0/3994902956 shutdown_connections 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f630415bab0 0x7f63040c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630409a7c0 0x7f63040cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f90 0x7f62e8069450 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 --2- 172.21.15.133:0/3994902956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630409a0e0 0x7f63040c4b80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 -- 172.21.15.133:0/3994902956 >> 172.21.15.133:0/3994902956 conn(0x7f63040b31a0 msgr2=0x7f63040b5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 -- 172.21.15.133:0/3994902956 shutdown_connections 2024-04-25T10:02:45.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.548+0000 7f630c683640 1 -- 172.21.15.133:0/3994902956 wait complete. 2024-04-25T10:02:45.581 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-25T10:02:45.581 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:02:45.581 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:45.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff2fb3640 1 Processor -- start 2024-04-25T10:02:45.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff2fb3640 1 -- start start 2024-04-25T10:02:45.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff2fb3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcfec15b9c0 0x7fcfec072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff2fb3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcfec0704a0 0x7fcfec0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff2fb3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfec15bda0 con 0x7fcfec0ccf90 2024-04-25T10:02:45.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff2fb3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfec0cd450 con 0x7fcfec15b9c0 2024-04-25T10:02:45.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff2fb3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfec0cd730 con 0x7fcfec0704a0 2024-04-25T10:02:45.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff0d4f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcfec15b9c0 0x7fcfec072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff0d4f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcfec15b9c0 0x7fcfec072cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39860/0 (socket says 172.21.15.133:39860) 2024-04-25T10:02:45.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff0d4f640 1 -- 172.21.15.133:0/2296617983 learned_addr learned my addr 172.21.15.133:0/2296617983 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:45.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff1550640 1 --2- 172.21.15.133:0/2296617983 >> v2:172.21.15.162:3300/0 conn(0x7fcfec0704a0 0x7fcfec0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff0d4f640 1 -- 172.21.15.133:0/2296617983 >> v2:172.21.15.162:3300/0 conn(0x7fcfec0704a0 msgr2=0x7fcfec0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff0d4f640 1 --2- 172.21.15.133:0/2296617983 >> v2:172.21.15.162:3300/0 conn(0x7fcfec0704a0 0x7fcfec0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff0d4f640 1 -- 172.21.15.133:0/2296617983 >> v1:172.21.15.162:6789/0 conn(0x7fcfec0ccf90 legacy=0x7fcfec06fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.692+0000 7fcff0d4f640 1 -- 172.21.15.133:0/2296617983 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0cda10 con 0x7fcfec15b9c0 2024-04-25T10:02:45.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff0d4f640 1 --2- 172.21.15.133:0/2296617983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec15b9c0 0x7fcfec072cb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0b3720 tx=0x7fcfdc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e4c124cef92d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcfeb7fe640 1 -- 172.21.15.133:0/2296617983 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0b1070 con 0x7fcfec15b9c0 2024-04-25T10:02:45.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcfeb7fe640 1 -- 172.21.15.133:0/2296617983 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfdc0ac600 con 0x7fcfec15b9c0 2024-04-25T10:02:45.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcfeb7fe640 1 -- 172.21.15.133:0/2296617983 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0ac9d0 con 0x7fcfec15b9c0 2024-04-25T10:02:45.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 -- 172.21.15.133:0/2296617983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec15b9c0 msgr2=0x7fcfec072cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 --2- 172.21.15.133:0/2296617983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec15b9c0 0x7fcfec072cb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0b3720 tx=0x7fcfdc01c490 comp rx=0 tx=0).stop 2024-04-25T10:02:45.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 -- 172.21.15.133:0/2296617983 shutdown_connections 2024-04-25T10:02:45.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 --2- 172.21.15.133:0/2296617983 >> v2:172.21.15.162:3300/0 conn(0x7fcfec0704a0 0x7fcfec0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 --2- 172.21.15.133:0/2296617983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec15b9c0 0x7fcfec072cb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 -- 172.21.15.133:0/2296617983 >> 172.21.15.133:0/2296617983 conn(0x7fcfec0b2370 msgr2=0x7fcfec0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 -- 172.21.15.133:0/2296617983 shutdown_connections 2024-04-25T10:02:45.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 -- 172.21.15.133:0/2296617983 wait complete. 2024-04-25T10:02:45.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 Processor -- start 2024-04-25T10:02:45.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 -- start start 2024-04-25T10:02:45.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfec0704a0 0x7fcfec14be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec0ccf90 0x7fcfec150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfec15b9c0 0x7fcfec153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0d0a30 con 0x7fcfec0ccf90 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0d08b0 con 0x7fcfec15b9c0 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.696+0000 7fcff2fb3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0d0bb0 con 0x7fcfec0704a0 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec0ccf90 0x7fcfec150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec0ccf90 0x7fcfec150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39868/0 (socket says 172.21.15.133:39868) 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfebfff640 1 -- 172.21.15.133:0/1057446085 learned_addr learned my addr 172.21.15.133:0/1057446085 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcff0d4f640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfec0704a0 0x7fcfec14be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcff1550640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfec15b9c0 0x7fcfec153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfebfff640 1 -- 172.21.15.133:0/1057446085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfec0704a0 msgr2=0x7fcfec14be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfebfff640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfec0704a0 0x7fcfec14be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfebfff640 1 -- 172.21.15.133:0/1057446085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfec15b9c0 msgr2=0x7fcfec153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfebfff640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfec15b9c0 0x7fcfec153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfebfff640 1 -- 172.21.15.133:0/1057446085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec14fea0 con 0x7fcfec0ccf90 2024-04-25T10:02:45.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcff0d4f640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfec0704a0 0x7fcfec14be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:45.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcff1550640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfec15b9c0 0x7fcfec153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:45.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfebfff640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec0ccf90 0x7fcfec150910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c390 tx=0x7fcfe000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfe97fa640 1 -- 172.21.15.133:0/1057446085 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0017070 con 0x7fcfec0ccf90 2024-04-25T10:02:45.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfe97fa640 1 -- 172.21.15.133:0/1057446085 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe00052f0 con 0x7fcfec0ccf90 2024-04-25T10:02:45.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcff2fb3640 1 -- 172.21.15.133:0/1057446085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec150160 con 0x7fcfec0ccf90 2024-04-25T10:02:45.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfe97fa640 1 -- 172.21.15.133:0/1057446085 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0003840 con 0x7fcfec0ccf90 2024-04-25T10:02:45.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcff2fb3640 1 -- 172.21.15.133:0/1057446085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfec14bb40 con 0x7fcfec0ccf90 2024-04-25T10:02:45.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcff2fb3640 1 -- 172.21.15.133:0/1057446085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfec067c70 con 0x7fcfec0ccf90 2024-04-25T10:02:45.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfe97fa640 1 -- 172.21.15.133:0/1057446085 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfe001a070 con 0x7fcfec0ccf90 2024-04-25T10:02:45.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfe97fa640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfd8066f60 0x7fcfd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcff0d4f640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfd8066f60 0x7fcfd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcfe97fa640 1 -- 172.21.15.133:0/1057446085 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcfe0084690 con 0x7fcfec0ccf90 2024-04-25T10:02:45.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.700+0000 7fcff0d4f640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfd8066f60 0x7fcfd8069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0ae000 tx=0x7fcfdc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.704+0000 7fcfe97fa640 1 -- 172.21.15.133:0/1057446085 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfe0055330 con 0x7fcfec0ccf90 2024-04-25T10:02:45.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.864+0000 7fcff2fb3640 1 -- 172.21.15.133:0/1057446085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfec070ba0 con 0x7fcfec0ccf90 2024-04-25T10:02:45.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.868+0000 7fcfe97fa640 1 -- 172.21.15.133:0/1057446085 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7fcfe0081a00 con 0x7fcfec0ccf90 2024-04-25T10:02:45.875 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:45.875 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:45.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 -- 172.21.15.133:0/1057446085 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfd8066f60 msgr2=0x7fcfd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfd8066f60 0x7fcfd8069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0ae000 tx=0x7fcfdc008040 comp rx=0 tx=0).stop 2024-04-25T10:02:45.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 -- 172.21.15.133:0/1057446085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec0ccf90 msgr2=0x7fcfec150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec0ccf90 0x7fcfec150910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c390 tx=0x7fcfe000c850 comp rx=0 tx=0).stop 2024-04-25T10:02:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 -- 172.21.15.133:0/1057446085 shutdown_connections 2024-04-25T10:02:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfec15b9c0 0x7fcfec153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfd8066f60 0x7fcfd8069420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfec0ccf90 0x7fcfec150910 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 --2- 172.21.15.133:0/1057446085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfec0704a0 0x7fcfec14be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 -- 172.21.15.133:0/1057446085 >> 172.21.15.133:0/1057446085 conn(0x7fcfec0b2370 msgr2=0x7fcfec0b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 -- 172.21.15.133:0/1057446085 shutdown_connections 2024-04-25T10:02:45.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:45.872+0000 7fcff2fb3640 1 -- 172.21.15.133:0/1057446085 wait complete. 2024-04-25T10:02:45.894 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:45.894 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:46.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.012+0000 7f9075e1c640 1 Processor -- start 2024-04-25T10:02:46.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.012+0000 7f9075e1c640 1 -- start start 2024-04-25T10:02:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f9075e1c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90700b5ef0 0x7f90700ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f9075e1c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90700d0300 con 0x7f90700d1060 2024-04-25T10:02:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f9075e1c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90700cfaa0 con 0x7f907006b3a0 2024-04-25T10:02:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f9075e1c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90700cfd40 con 0x7f90700b5ef0 2024-04-25T10:02:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f906effd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f90700d1060 0x7f907006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53754/0 (socket says 172.21.15.133:53754) 2024-04-25T10:02:46.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f906effd640 1 -- 172.21.15.133:0/4194102218 learned_addr learned my addr 172.21.15.133:0/4194102218 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:46.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f906f7fe640 1 --2- 172.21.15.133:0/4194102218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90700b5ef0 0x7f90700ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f906e7fc640 1 -- 172.21.15.133:0/4194102218 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211009260 0 0) 0x7f90700d0300 con 0x7f90700d1060 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f906e7fc640 1 -- 172.21.15.133:0/4194102218 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9058003530 con 0x7f90700d1060 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f906e7fc640 1 -- 172.21.15.133:0/4194102218 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1877375682 0 0) 0x7f9058003530 con 0x7f90700d1060 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f906f7fe640 1 -- 172.21.15.133:0/4194102218 >> v1:172.21.15.162:6789/0 conn(0x7f907006b3a0 legacy=0x7f90700cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f906f7fe640 1 -- 172.21.15.133:0/4194102218 >> v1:172.21.15.133:6789/0 conn(0x7f90700d1060 legacy=0x7f907006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f906f7fe640 1 -- 172.21.15.133:0/4194102218 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700d0020 con 0x7f90700b5ef0 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.016+0000 7f906f7fe640 1 --2- 172.21.15.133:0/4194102218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90700b5ef0 0x7f90700ce140 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90600097f0 tx=0x7f906001c2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=538f72f28f4c798 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906e7fc640 1 -- 172.21.15.133:0/4194102218 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90600b0070 con 0x7f90700b5ef0 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906e7fc640 1 -- 172.21.15.133:0/4194102218 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90600ab400 con 0x7f90700b5ef0 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906e7fc640 1 -- 172.21.15.133:0/4194102218 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064001680 con 0x7f90700b5ef0 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 -- 172.21.15.133:0/4194102218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90700b5ef0 msgr2=0x7f90700ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 --2- 172.21.15.133:0/4194102218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90700b5ef0 0x7f90700ce140 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f90600097f0 tx=0x7f906001c2b0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 -- 172.21.15.133:0/4194102218 shutdown_connections 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 --2- 172.21.15.133:0/4194102218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90700b5ef0 0x7f90700ce140 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 -- 172.21.15.133:0/4194102218 >> 172.21.15.133:0/4194102218 conn(0x7f90700d0570 msgr2=0x7f90700d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 -- 172.21.15.133:0/4194102218 shutdown_connections 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 -- 172.21.15.133:0/4194102218 wait complete. 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 Processor -- start 2024-04-25T10:02:46.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 -- start start 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f907006b3a0 0x7f907015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90700b5ef0 0x7f90701f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90700d1060 0x7f907015e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90700b33c0 con 0x7f90700b5ef0 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90700cafb0 con 0x7f907006b3a0 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90700b62d0 con 0x7f90700d1060 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90700b5ef0 0x7f90701f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90700b5ef0 0x7f90701f3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39878/0 (socket says 172.21.15.133:39878) 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906effd640 1 -- 172.21.15.133:0/943035113 learned_addr learned my addr 172.21.15.133:0/943035113 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906ffff640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90700d1060 0x7f907015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906f7fe640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f907006b3a0 0x7f907015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906effd640 1 -- 172.21.15.133:0/943035113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90700d1060 msgr2=0x7f907015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906effd640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90700d1060 0x7f907015e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906effd640 1 -- 172.21.15.133:0/943035113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f907006b3a0 msgr2=0x7f907015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906effd640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f907006b3a0 0x7f907015a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906effd640 1 -- 172.21.15.133:0/943035113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9070160070 con 0x7f90700b5ef0 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f906effd640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90700b5ef0 0x7f90701f3c50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f906400c4d0 tx=0x7f906400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9057fff640 1 -- 172.21.15.133:0/943035113 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064018070 con 0x7f90700b5ef0 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9057fff640 1 -- 172.21.15.133:0/943035113 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9064007500 con 0x7f90700b5ef0 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 -- 172.21.15.133:0/943035113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9070160350 con 0x7f90700b5ef0 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9057fff640 1 -- 172.21.15.133:0/943035113 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064004a30 con 0x7f90700b5ef0 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.020+0000 7f9075e1c640 1 -- 172.21.15.133:0/943035113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9070160910 con 0x7f90700b5ef0 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.024+0000 7f9075e1c640 1 -- 172.21.15.133:0/943035113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9070067c70 con 0x7f90700b5ef0 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.024+0000 7f9057fff640 1 -- 172.21.15.133:0/943035113 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f906401b070 con 0x7f90700b5ef0 2024-04-25T10:02:46.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.024+0000 7f9057fff640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f905c066f60 0x7f905c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.024+0000 7f9057fff640 1 -- 172.21.15.133:0/943035113 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f90640847c0 con 0x7f90700b5ef0 2024-04-25T10:02:46.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.024+0000 7f906f7fe640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f905c066f60 0x7f905c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.024+0000 7f906f7fe640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f905c066f60 0x7f905c069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90600ad000 tx=0x7f90600a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.028+0000 7f9057fff640 1 -- 172.21.15.133:0/943035113 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9064054ce0 con 0x7f90700b5ef0 2024-04-25T10:02:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.184+0000 7f9075e1c640 1 -- 172.21.15.133:0/943035113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90700b03b0 con 0x7f90700b5ef0 2024-04-25T10:02:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9057fff640 1 -- 172.21.15.133:0/943035113 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9064058d80 con 0x7f90700b5ef0 2024-04-25T10:02:46.194 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:46.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 -- 172.21.15.133:0/943035113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f905c066f60 msgr2=0x7f905c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f905c066f60 0x7f905c069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90600ad000 tx=0x7f90600a6000 comp rx=0 tx=0).stop 2024-04-25T10:02:46.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 -- 172.21.15.133:0/943035113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90700b5ef0 msgr2=0x7f90701f3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90700b5ef0 0x7f90701f3c50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f906400c4d0 tx=0x7f906400c990 comp rx=0 tx=0).stop 2024-04-25T10:02:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 -- 172.21.15.133:0/943035113 shutdown_connections 2024-04-25T10:02:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90700d1060 0x7f907015e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f905c066f60 0x7f905c069420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90700b5ef0 0x7f90701f3c50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 --2- 172.21.15.133:0/943035113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f907006b3a0 0x7f907015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 -- 172.21.15.133:0/943035113 >> 172.21.15.133:0/943035113 conn(0x7f90700d0570 msgr2=0x7f907006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 -- 172.21.15.133:0/943035113 shutdown_connections 2024-04-25T10:02:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.188+0000 7f9075e1c640 1 -- 172.21.15.133:0/943035113 wait complete. 2024-04-25T10:02:46.213 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:46.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff0a1a96640 1 Processor -- start 2024-04-25T10:02:46.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff0a1a96640 1 -- start start 2024-04-25T10:02:46.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff0a1a96640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff09c142920 0x7ff09c143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff0a1a96640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff09c140a40 0x7ff09c0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff0a1a96640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c0cf000 con 0x7ff09c15d9f0 2024-04-25T10:02:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff0a1a96640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c140ea0 con 0x7ff09c142920 2024-04-25T10:02:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff0a1a96640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0cf280 con 0x7ff09c140a40 2024-04-25T10:02:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff09b7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff09c142920 0x7ff09c143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff09affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff09c140a40 0x7ff09c0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff09bfff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff09c15d9f0 0x7ff09c06df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:44268/0 (socket says 172.21.15.133:44268) 2024-04-25T10:02:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff09bfff640 1 -- 172.21.15.133:0/1102567258 learned_addr learned my addr 172.21.15.133:0/1102567258 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff09affd640 1 -- 172.21.15.133:0/1102567258 >> v2:172.21.15.162:3300/0 conn(0x7ff09c142920 msgr2=0x7ff09c143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff09affd640 1 --2- 172.21.15.133:0/1102567258 >> v2:172.21.15.162:3300/0 conn(0x7ff09c142920 0x7ff09c143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff09affd640 1 -- 172.21.15.133:0/1102567258 >> v1:172.21.15.162:6789/0 conn(0x7ff09c15d9f0 legacy=0x7ff09c06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.328+0000 7ff09affd640 1 -- 172.21.15.133:0/1102567258 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c06d440 con 0x7ff09c140a40 2024-04-25T10:02:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff09affd640 1 --2- 172.21.15.133:0/1102567258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff09c140a40 0x7ff09c0ceac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff0840097c0 tx=0x7ff08401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a72381137b5302c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff09a7fc640 1 -- 172.21.15.133:0/1102567258 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840b0070 con 0x7ff09c140a40 2024-04-25T10:02:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff09a7fc640 1 -- 172.21.15.133:0/1102567258 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0840ab600 con 0x7ff09c140a40 2024-04-25T10:02:46.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff09a7fc640 1 -- 172.21.15.133:0/1102567258 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840ab9d0 con 0x7ff09c140a40 2024-04-25T10:02:46.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff0a1a96640 1 -- 172.21.15.133:0/1102567258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff09c140a40 msgr2=0x7ff09c0ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff0a1a96640 1 --2- 172.21.15.133:0/1102567258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff09c140a40 0x7ff09c0ceac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff0840097c0 tx=0x7ff08401c410 comp rx=0 tx=0).stop 2024-04-25T10:02:46.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff0a1a96640 1 -- 172.21.15.133:0/1102567258 shutdown_connections 2024-04-25T10:02:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff0a1a96640 1 --2- 172.21.15.133:0/1102567258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff09c140a40 0x7ff09c0ceac0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff0a1a96640 1 --2- 172.21.15.133:0/1102567258 >> v2:172.21.15.162:3300/0 conn(0x7ff09c142920 0x7ff09c143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff0a1a96640 1 -- 172.21.15.133:0/1102567258 >> 172.21.15.133:0/1102567258 conn(0x7ff09c066f70 msgr2=0x7ff09c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff0a1a96640 1 -- 172.21.15.133:0/1102567258 shutdown_connections 2024-04-25T10:02:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff0a1a96640 1 -- 172.21.15.133:0/1102567258 wait complete. 2024-04-25T10:02:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff0a1a96640 1 Processor -- start 2024-04-25T10:02:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.332+0000 7ff0a1a96640 1 -- start start 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff0a1a96640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff09c140a40 0x7ff09c1f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff0a1a96640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff09c142920 0x7ff09c1383d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff0a1a96640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff09c15d9f0 0x7ff09c138910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff0a1a96640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c06ebc0 con 0x7ff09c15d9f0 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff0a1a96640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c06ea40 con 0x7ff09c140a40 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff0a1a96640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c142d00 con 0x7ff09c142920 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff09c15d9f0 0x7ff09c138910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff09c15d9f0 0x7ff09c138910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46788/0 (socket says 172.21.15.133:46788) 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09bfff640 1 -- 172.21.15.133:0/4104111377 learned_addr learned my addr 172.21.15.133:0/4104111377 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09affd640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff09c142920 0x7ff09c1383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09b7fe640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff09c140a40 0x7ff09c1f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09bfff640 1 -- 172.21.15.133:0/4104111377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff09c142920 msgr2=0x7ff09c1383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09bfff640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff09c142920 0x7ff09c1383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09bfff640 1 -- 172.21.15.133:0/4104111377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff09c140a40 msgr2=0x7ff09c1f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09bfff640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff09c140a40 0x7ff09c1f6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09bfff640 1 -- 172.21.15.133:0/4104111377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0b0dd0 con 0x7ff09c15d9f0 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09bfff640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff09c15d9f0 0x7ff09c138910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff09400caa0 tx=0x7ff09400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff093fff640 1 -- 172.21.15.133:0/4104111377 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094017070 con 0x7ff09c15d9f0 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff093fff640 1 -- 172.21.15.133:0/4104111377 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09400a6d0 con 0x7ff09c15d9f0 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff0a1a96640 1 -- 172.21.15.133:0/4104111377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c0afd30 con 0x7ff09c15d9f0 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff093fff640 1 -- 172.21.15.133:0/4104111377 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09400aac0 con 0x7ff09c15d9f0 2024-04-25T10:02:46.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff0a1a96640 1 -- 172.21.15.133:0/4104111377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c0b05d0 con 0x7ff09c15d9f0 2024-04-25T10:02:46.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff0a1a96640 1 -- 172.21.15.133:0/4104111377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c068e80 con 0x7ff09c15d9f0 2024-04-25T10:02:46.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff093fff640 1 -- 172.21.15.133:0/4104111377 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff09401a070 con 0x7ff09c15d9f0 2024-04-25T10:02:46.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff093fff640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff07c066f90 0x7ff07c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff09b7fe640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff07c066f90 0x7ff07c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.336+0000 7ff093fff640 1 -- 172.21.15.133:0/4104111377 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff094084b90 con 0x7ff09c15d9f0 2024-04-25T10:02:46.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.340+0000 7ff09b7fe640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff07c066f90 0x7ff07c069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009a50 tx=0x7ff08c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.340+0000 7ff093fff640 1 -- 172.21.15.133:0/4104111377 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff09c0cf000 con 0x7ff09c15d9f0 2024-04-25T10:02:46.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.500+0000 7ff0a1a96640 1 -- 172.21.15.133:0/4104111377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff09c0afa70 con 0x7ff09c15d9f0 2024-04-25T10:02:46.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.500+0000 7ff093fff640 1 -- 172.21.15.133:0/4104111377 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0940550c0 con 0x7ff09c15d9f0 2024-04-25T10:02:46.508 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 -- 172.21.15.133:0/4104111377 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff07c066f90 msgr2=0x7ff07c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff07c066f90 0x7ff07c069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009a50 tx=0x7ff08c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 -- 172.21.15.133:0/4104111377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff09c15d9f0 msgr2=0x7ff09c138910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff09c15d9f0 0x7ff09c138910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff09400caa0 tx=0x7ff09400cf60 comp rx=0 tx=0).stop 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 -- 172.21.15.133:0/4104111377 shutdown_connections 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff07c066f90 0x7ff07c069450 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff09c15d9f0 0x7ff09c138910 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff09c142920 0x7ff09c1383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 --2- 172.21.15.133:0/4104111377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff09c140a40 0x7ff09c1f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 -- 172.21.15.133:0/4104111377 >> 172.21.15.133:0/4104111377 conn(0x7ff09c066f70 msgr2=0x7ff09c141950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 -- 172.21.15.133:0/4104111377 shutdown_connections 2024-04-25T10:02:46.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.504+0000 7ff0a1a96640 1 -- 172.21.15.133:0/4104111377 wait complete. 2024-04-25T10:02:46.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:46.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc1287af640 1 Processor -- start 2024-04-25T10:02:46.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc1287af640 1 -- start start 2024-04-25T10:02:46.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc1287af640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc12015d9a0 0x7fc1200d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc1287af640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1200ce600 0x7fc1200cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc1287af640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc1200d1920 0x7fc1200d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc1287af640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1200cefa0 con 0x7fc12015d9a0 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc1287af640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12015ebd0 con 0x7fc1200d1920 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc1287af640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12015eeb0 con 0x7fc1200ce600 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc12654b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc12015d9a0 0x7fc1200d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc12654b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc12015d9a0 0x7fc1200d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46802/0 (socket says 172.21.15.133:46802) 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc12654b640 1 -- 172.21.15.133:0/2018282863 learned_addr learned my addr 172.21.15.133:0/2018282863 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc126d4c640 1 --2- 172.21.15.133:0/2018282863 >> v2:172.21.15.162:3300/0 conn(0x7fc1200d1920 0x7fc1200d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc12654b640 1 -- 172.21.15.133:0/2018282863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1200ce600 msgr2=0x7fc1200cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc125d4a640 1 --2- 172.21.15.133:0/2018282863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1200ce600 0x7fc1200cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc12654b640 1 --2- 172.21.15.133:0/2018282863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1200ce600 0x7fc1200cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc12654b640 1 -- 172.21.15.133:0/2018282863 >> v2:172.21.15.162:3300/0 conn(0x7fc1200d1920 msgr2=0x7fc1200d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc12654b640 1 --2- 172.21.15.133:0/2018282863 >> v2:172.21.15.162:3300/0 conn(0x7fc1200d1920 0x7fc1200d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc12654b640 1 -- 172.21.15.133:0/2018282863 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1200d0330 con 0x7fc12015d9a0 2024-04-25T10:02:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc126d4c640 1 --2- 172.21.15.133:0/2018282863 >> v2:172.21.15.162:3300/0 conn(0x7fc1200d1920 0x7fc1200d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.640+0000 7fc125d4a640 1 --2- 172.21.15.133:0/2018282863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1200ce600 0x7fc1200cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:46.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc12654b640 1 --2- 172.21.15.133:0/2018282863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc12015d9a0 0x7fc1200d0b30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc1201615d0 tx=0x7fc11001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92275fb6e55eb15c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc125549640 1 -- 172.21.15.133:0/2018282863 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1100ab600 con 0x7fc12015d9a0 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc125549640 1 -- 172.21.15.133:0/2018282863 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1100b0070 con 0x7fc12015d9a0 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc125549640 1 -- 172.21.15.133:0/2018282863 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1100ab9c0 con 0x7fc12015d9a0 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 -- 172.21.15.133:0/2018282863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc12015d9a0 msgr2=0x7fc1200d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 --2- 172.21.15.133:0/2018282863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc12015d9a0 0x7fc1200d0b30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc1201615d0 tx=0x7fc11001c460 comp rx=0 tx=0).stop 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 -- 172.21.15.133:0/2018282863 shutdown_connections 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 --2- 172.21.15.133:0/2018282863 >> v2:172.21.15.162:3300/0 conn(0x7fc1200d1920 0x7fc1200d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 --2- 172.21.15.133:0/2018282863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1200ce600 0x7fc1200cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 --2- 172.21.15.133:0/2018282863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc12015d9a0 0x7fc1200d0b30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 -- 172.21.15.133:0/2018282863 >> 172.21.15.133:0/2018282863 conn(0x7fc12006bf80 msgr2=0x7fc12006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 -- 172.21.15.133:0/2018282863 shutdown_connections 2024-04-25T10:02:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 -- 172.21.15.133:0/2018282863 wait complete. 2024-04-25T10:02:46.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 Processor -- start 2024-04-25T10:02:46.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 -- start start 2024-04-25T10:02:46.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1200ce600 0x7fc120167630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1200d1920 0x7fc12015c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc12015d9a0 0x7fc120160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1200b9820 con 0x7fc12015d9a0 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1200b96a0 con 0x7fc1200ce600 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc1287af640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1200d22f0 con 0x7fc1200d1920 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc126d4c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc12015d9a0 0x7fc120160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc126d4c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc12015d9a0 0x7fc120160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46816/0 (socket says 172.21.15.133:46816) 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc126d4c640 1 -- 172.21.15.133:0/1526889097 learned_addr learned my addr 172.21.15.133:0/1526889097 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc126d4c640 1 -- 172.21.15.133:0/1526889097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1200d1920 msgr2=0x7fc12015c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc126d4c640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1200d1920 0x7fc12015c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc126d4c640 1 -- 172.21.15.133:0/1526889097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1200ce600 msgr2=0x7fc120167630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc126d4c640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1200ce600 0x7fc120167630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc126d4c640 1 -- 172.21.15.133:0/1526889097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1200b5f90 con 0x7fc12015d9a0 2024-04-25T10:02:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc126d4c640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc12015d9a0 0x7fc120160930 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc11800cd50 tx=0x7fc11800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc116ffd640 1 -- 172.21.15.133:0/1526889097 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11800f680 con 0x7fc12015d9a0 2024-04-25T10:02:46.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc116ffd640 1 -- 172.21.15.133:0/1526889097 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11800f840 con 0x7fc12015d9a0 2024-04-25T10:02:46.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.644+0000 7fc116ffd640 1 -- 172.21.15.133:0/1526889097 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11800fc30 con 0x7fc12015d9a0 2024-04-25T10:02:46.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.648+0000 7fc1287af640 1 -- 172.21.15.133:0/1526889097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1200b59d0 con 0x7fc12015d9a0 2024-04-25T10:02:46.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.648+0000 7fc1287af640 1 -- 172.21.15.133:0/1526889097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1200ca9a0 con 0x7fc12015d9a0 2024-04-25T10:02:46.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.648+0000 7fc1287af640 1 -- 172.21.15.133:0/1526889097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc120068e80 con 0x7fc12015d9a0 2024-04-25T10:02:46.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.648+0000 7fc116ffd640 1 -- 172.21.15.133:0/1526889097 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc11800e3e0 con 0x7fc12015d9a0 2024-04-25T10:02:46.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.648+0000 7fc116ffd640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc104066f60 0x7fc104069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.648+0000 7fc116ffd640 1 -- 172.21.15.133:0/1526889097 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc118086250 con 0x7fc12015d9a0 2024-04-25T10:02:46.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.648+0000 7fc12654b640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc104066f60 0x7fc104069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.648+0000 7fc12654b640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc104066f60 0x7fc104069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1100af000 tx=0x7fc110004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.652+0000 7fc116ffd640 1 -- 172.21.15.133:0/1526889097 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc118014030 con 0x7fc12015d9a0 2024-04-25T10:02:46.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc1287af640 1 -- 172.21.15.133:0/1526889097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc120167b70 con 0x7fc12015d9a0 2024-04-25T10:02:46.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc116ffd640 1 -- 172.21.15.133:0/1526889097 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc118056dc0 con 0x7fc12015d9a0 2024-04-25T10:02:46.807 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:46.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc1287af640 1 -- 172.21.15.133:0/1526889097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc104066f60 msgr2=0x7fc104069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc1287af640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc104066f60 0x7fc104069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1100af000 tx=0x7fc110004740 comp rx=0 tx=0).stop 2024-04-25T10:02:46.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc1287af640 1 -- 172.21.15.133:0/1526889097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc12015d9a0 msgr2=0x7fc120160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc1287af640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc12015d9a0 0x7fc120160930 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc11800cd50 tx=0x7fc11800a650 comp rx=0 tx=0).stop 2024-04-25T10:02:46.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc1287af640 1 -- 172.21.15.133:0/1526889097 shutdown_connections 2024-04-25T10:02:46.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc1287af640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc104066f60 0x7fc104069420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc1287af640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc12015d9a0 0x7fc120160930 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc1287af640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc1200d1920 0x7fc12015c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc1287af640 1 --2- 172.21.15.133:0/1526889097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc1200ce600 0x7fc120167630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.800+0000 7fc1287af640 1 -- 172.21.15.133:0/1526889097 >> 172.21.15.133:0/1526889097 conn(0x7fc12006bf80 msgr2=0x7fc12006c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.804+0000 7fc1287af640 1 -- 172.21.15.133:0/1526889097 shutdown_connections 2024-04-25T10:02:46.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.804+0000 7fc1287af640 1 -- 172.21.15.133:0/1526889097 wait complete. 2024-04-25T10:02:46.825 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:46.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f4bbe7640 1 Processor -- start 2024-04-25T10:02:46.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f4bbe7640 1 -- start start 2024-04-25T10:02:46.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f4bbe7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8f44161180 0x7f8f440cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f4bbe7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f440cf840 con 0x7f8f4409a5e0 2024-04-25T10:02:46.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f4bbe7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4406c850 con 0x7f8f4409a0e0 2024-04-25T10:02:46.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f4bbe7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f44161560 con 0x7f8f44161180 2024-04-25T10:02:46.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f49182640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8f4409a5e0 0x7f8f440999d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47030/0 (socket says 172.21.15.133:47030) 2024-04-25T10:02:46.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f49182640 1 -- 172.21.15.133:0/3928170556 learned_addr learned my addr 172.21.15.133:0/3928170556 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:46.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f49983640 1 --2- 172.21.15.133:0/3928170556 >> v2:172.21.15.162:3300/0 conn(0x7f8f44161180 0x7f8f440cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107183130 0 0) 0x7f8f440cf840 con 0x7f8f4409a5e0 2024-04-25T10:02:46.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f30003530 con 0x7f8f4409a5e0 2024-04-25T10:02:46.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3268126090 0 0) 0x7f8f30003530 con 0x7f8f4409a5e0 2024-04-25T10:02:46.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 >> v2:172.21.15.162:3300/0 conn(0x7f8f44161180 msgr2=0x7f8f440cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 --2- 172.21.15.133:0/3928170556 >> v2:172.21.15.162:3300/0 conn(0x7f8f44161180 0x7f8f440cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 >> v1:172.21.15.162:6789/0 conn(0x7f8f4409a0e0 legacy=0x7f8f440d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4406cb30 con 0x7f8f4409a5e0 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f8f34001680 con 0x7f8f4409a5e0 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 >> v1:172.21.15.133:6789/0 conn(0x7f8f4409a5e0 legacy=0x7f8f440999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 --2- 172.21.15.133:0/3928170556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f300039c0 0x7f8f30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 --2- 172.21.15.133:0/3928170556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f30004810 0x7f8f30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 --2- 172.21.15.133:0/3928170556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f300051d0 0x7f8f3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44161560 con 0x7f8f300039c0 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3000c070 con 0x7f8f300051d0 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3000c270 con 0x7f8f30004810 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f49983640 1 --2- 172.21.15.133:0/3928170556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f300039c0 0x7f8f30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f49983640 1 -- 172.21.15.133:0/3928170556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f30004810 msgr2=0x7f8f30004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f49983640 1 --2- 172.21.15.133:0/3928170556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f30004810 0x7f8f30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f49983640 1 -- 172.21.15.133:0/3928170556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f300051d0 msgr2=0x7f8f3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f49983640 1 --2- 172.21.15.133:0/3928170556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f300051d0 0x7f8f3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f49983640 1 -- 172.21.15.133:0/3928170556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f30003530 con 0x7f8f300039c0 2024-04-25T10:02:46.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.944+0000 7f8f4a184640 1 -- 172.21.15.133:0/3928170556 reap_dead start 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f49983640 1 --2- 172.21.15.133:0/3928170556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f300039c0 0x7f8f30003de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009fc0 tx=0x7f8f4001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d76ce5f4ffeb1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400b2070 con 0x7f8f300039c0 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400a5ab0 con 0x7f8f300039c0 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f48981640 1 -- 172.21.15.133:0/3928170556 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40009c10 con 0x7f8f300039c0 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/3928170556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f300039c0 msgr2=0x7f8f30003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 --2- 172.21.15.133:0/3928170556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f300039c0 0x7f8f30003de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009fc0 tx=0x7f8f4001ccc0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/3928170556 shutdown_connections 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 --2- 172.21.15.133:0/3928170556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f300039c0 0x7f8f30003de0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/3928170556 >> 172.21.15.133:0/3928170556 conn(0x7f8f440b31a0 msgr2=0x7f8f440b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/3928170556 shutdown_connections 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/3928170556 wait complete. 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 Processor -- start 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 -- start start 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f300039c0 0x7f8f4414ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f440ca6f0 0x7f8f440cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f441584f0 0x7f8f44158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440d07f0 con 0x7f8f441584f0 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4409add0 con 0x7f8f300039c0 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4bbe7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440d0970 con 0x7f8f440ca6f0 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4a184640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f441584f0 0x7f8f44158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4a184640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f441584f0 0x7f8f44158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46834/0 (socket says 172.21.15.133:46834) 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4a184640 1 -- 172.21.15.133:0/2870341358 learned_addr learned my addr 172.21.15.133:0/2870341358 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f49983640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f300039c0 0x7f8f4414ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4a184640 1 -- 172.21.15.133:0/2870341358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f440ca6f0 msgr2=0x7f8f440cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4a184640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f440ca6f0 0x7f8f440cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4a184640 1 -- 172.21.15.133:0/2870341358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f300039c0 msgr2=0x7f8f4414ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4a184640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f300039c0 0x7f8f4414ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4a184640 1 -- 172.21.15.133:0/2870341358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440c5e10 con 0x7f8f441584f0 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.948+0000 7f8f4a184640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f441584f0 0x7f8f44158970 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f440d2620 tx=0x7f8f3c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.952+0000 7f8f3a7fc640 1 -- 172.21.15.133:0/2870341358 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c007270 con 0x7f8f441584f0 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.952+0000 7f8f3a7fc640 1 -- 172.21.15.133:0/2870341358 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c007410 con 0x7f8f441584f0 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.952+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/2870341358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f440c60f0 con 0x7f8f441584f0 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.952+0000 7f8f3a7fc640 1 -- 172.21.15.133:0/2870341358 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0057d0 con 0x7f8f441584f0 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.952+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/2870341358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f440c66b0 con 0x7f8f441584f0 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.952+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/2870341358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f440c68b0 con 0x7f8f441584f0 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.952+0000 7f8f3a7fc640 1 -- 172.21.15.133:0/2870341358 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f3c011070 con 0x7f8f441584f0 2024-04-25T10:02:46.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.952+0000 7f8f3a7fc640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f28066f60 0x7f8f28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.952+0000 7f8f49983640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f28066f60 0x7f8f28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.952+0000 7f8f3a7fc640 1 -- 172.21.15.133:0/2870341358 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8f3c0843b0 con 0x7f8f441584f0 2024-04-25T10:02:46.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.952+0000 7f8f49983640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f28066f60 0x7f8f28069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f400af000 tx=0x7f8f400a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:46.956+0000 7f8f3a7fc640 1 -- 172.21.15.133:0/2870341358 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f440c68b0 con 0x7f8f441584f0 2024-04-25T10:02:47.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/2870341358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8f44068e80 con 0x7f8f441584f0 2024-04-25T10:02:47.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f3a7fc640 1 -- 172.21.15.133:0/2870341358 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8f3c019060 con 0x7f8f441584f0 2024-04-25T10:02:47.122 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:47.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/2870341358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f28066f60 msgr2=0x7f8f28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f28066f60 0x7f8f28069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f400af000 tx=0x7f8f400a7040 comp rx=0 tx=0).stop 2024-04-25T10:02:47.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/2870341358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f441584f0 msgr2=0x7f8f44158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f441584f0 0x7f8f44158970 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f440d2620 tx=0x7f8f3c007f90 comp rx=0 tx=0).stop 2024-04-25T10:02:47.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/2870341358 shutdown_connections 2024-04-25T10:02:47.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f28066f60 0x7f8f28069420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f441584f0 0x7f8f44158970 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f440ca6f0 0x7f8f440cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 --2- 172.21.15.133:0/2870341358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f300039c0 0x7f8f4414ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/2870341358 >> 172.21.15.133:0/2870341358 conn(0x7f8f440b31a0 msgr2=0x7f8f44070630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/2870341358 shutdown_connections 2024-04-25T10:02:47.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.116+0000 7f8f4bbe7640 1 -- 172.21.15.133:0/2870341358 wait complete. 2024-04-25T10:02:47.140 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T10:02:47.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed9a0f3640 1 Processor -- start 2024-04-25T10:02:47.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed9a0f3640 1 -- start start 2024-04-25T10:02:47.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed9a0f3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed9415c9a0 0x7fed940c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed9a0f3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fed940b1f70 0x7fed9406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed9a0f3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940c61d0 con 0x7fed940ce330 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed9a0f3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed940c64b0 con 0x7fed9415c9a0 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed9a0f3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9406d4f0 con 0x7fed940b1f70 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed937fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed9415c9a0 0x7fed940c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed93fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fed940ce330 0x7fed9415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47038/0 (socket says 172.21.15.133:47038) 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed93fff640 1 -- 172.21.15.133:0/3099213460 learned_addr learned my addr 172.21.15.133:0/3099213460 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed8affd640 1 --2- 172.21.15.133:0/3099213460 >> v2:172.21.15.162:3300/0 conn(0x7fed940b1f70 0x7fed9406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed937fe640 1 -- 172.21.15.133:0/3099213460 >> v2:172.21.15.162:3300/0 conn(0x7fed940b1f70 msgr2=0x7fed9406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed937fe640 1 --2- 172.21.15.133:0/3099213460 >> v2:172.21.15.162:3300/0 conn(0x7fed940b1f70 0x7fed9406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed937fe640 1 -- 172.21.15.133:0/3099213460 >> v1:172.21.15.133:6789/0 conn(0x7fed940ce330 legacy=0x7fed9415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed937fe640 1 -- 172.21.15.133:0/3099213460 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9406d7d0 con 0x7fed9415c9a0 2024-04-25T10:02:47.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.248+0000 7fed92ffd640 1 -- 172.21.15.133:0/3099213460 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762385094 0 0) 0x7fed940c61d0 con 0x7fed940ce330 2024-04-25T10:02:47.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed937fe640 1 --2- 172.21.15.133:0/3099213460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed9415c9a0 0x7fed940c5980 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fed94072670 tx=0x7fed8401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca6608a4b81e84c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed92ffd640 1 -- 172.21.15.133:0/3099213460 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed840b0070 con 0x7fed9415c9a0 2024-04-25T10:02:47.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed92ffd640 1 -- 172.21.15.133:0/3099213460 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed840ab600 con 0x7fed9415c9a0 2024-04-25T10:02:47.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed92ffd640 1 -- 172.21.15.133:0/3099213460 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed840ab9d0 con 0x7fed9415c9a0 2024-04-25T10:02:47.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3099213460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed9415c9a0 msgr2=0x7fed940c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed9a0f3640 1 --2- 172.21.15.133:0/3099213460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed9415c9a0 0x7fed940c5980 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fed94072670 tx=0x7fed8401c420 comp rx=0 tx=0).stop 2024-04-25T10:02:47.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3099213460 shutdown_connections 2024-04-25T10:02:47.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed9a0f3640 1 --2- 172.21.15.133:0/3099213460 >> v2:172.21.15.162:3300/0 conn(0x7fed940b1f70 0x7fed9406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed9a0f3640 1 --2- 172.21.15.133:0/3099213460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed9415c9a0 0x7fed940c5980 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3099213460 >> 172.21.15.133:0/3099213460 conn(0x7fed94067000 msgr2=0x7fed9406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3099213460 shutdown_connections 2024-04-25T10:02:47.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3099213460 wait complete. 2024-04-25T10:02:47.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.252+0000 7fed9a0f3640 1 Processor -- start 2024-04-25T10:02:47.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed9a0f3640 1 -- start start 2024-04-25T10:02:47.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed9a0f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed940b1f70 0x7fed94154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed9a0f3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed940ce330 0x7fed94155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed9a0f3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed9414fb50 0x7fed9414ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed9a0f3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940ceb50 con 0x7fed940b1f70 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed9a0f3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9415cd80 con 0x7fed940ce330 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed9a0f3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed94150520 con 0x7fed9414fb50 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed937fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed940b1f70 0x7fed94154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed937fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed940b1f70 0x7fed94154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46864/0 (socket says 172.21.15.133:46864) 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed937fe640 1 -- 172.21.15.133:0/3089005993 learned_addr learned my addr 172.21.15.133:0/3089005993 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed937fe640 1 -- 172.21.15.133:0/3089005993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed9414fb50 msgr2=0x7fed9414ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed937fe640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed9414fb50 0x7fed9414ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed937fe640 1 -- 172.21.15.133:0/3089005993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed940ce330 msgr2=0x7fed94155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed937fe640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed940ce330 0x7fed94155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed937fe640 1 -- 172.21.15.133:0/3089005993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed940b3490 con 0x7fed940b1f70 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed937fe640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed940b1f70 0x7fed94154e10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fed8401a0b0 tx=0x7fed8401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed90ff9640 1 -- 172.21.15.133:0/3089005993 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed840b0050 con 0x7fed940b1f70 2024-04-25T10:02:47.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed90ff9640 1 -- 172.21.15.133:0/3089005993 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed840bc070 con 0x7fed940b1f70 2024-04-25T10:02:47.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3089005993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed94148360 con 0x7fed940b1f70 2024-04-25T10:02:47.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed90ff9640 1 -- 172.21.15.133:0/3089005993 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed840b5810 con 0x7fed940b1f70 2024-04-25T10:02:47.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3089005993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed94148920 con 0x7fed940b1f70 2024-04-25T10:02:47.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3089005993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9406bc20 con 0x7fed940b1f70 2024-04-25T10:02:47.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed90ff9640 1 -- 172.21.15.133:0/3089005993 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed840b59b0 con 0x7fed940b1f70 2024-04-25T10:02:47.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed90ff9640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed74066f90 0x7fed74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed90ff9640 1 -- 172.21.15.133:0/3089005993 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fed8411fba0 con 0x7fed940b1f70 2024-04-25T10:02:47.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed8affd640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed74066f90 0x7fed74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.256+0000 7fed8affd640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed74066f90 0x7fed74069450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fed94155e10 tx=0x7fed7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.260+0000 7fed90ff9640 1 -- 172.21.15.133:0/3089005993 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed84016070 con 0x7fed940b1f70 2024-04-25T10:02:47.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.476+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3089005993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7fed94068ef0 con 0x7fed940b1f70 2024-04-25T10:02:47.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.848+0000 7fed90ff9640 1 -- 172.21.15.133:0/3089005993 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v72) v1 ==== 151+0+0 (secure 0 0 0) 0x7fed840f6030 con 0x7fed940b1f70 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3089005993 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed74066f90 msgr2=0x7fed74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed74066f90 0x7fed74069450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fed94155e10 tx=0x7fed7c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3089005993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed940b1f70 msgr2=0x7fed94154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed940b1f70 0x7fed94154e10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fed8401a0b0 tx=0x7fed8401acc0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3089005993 shutdown_connections 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed9414fb50 0x7fed9414ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed940ce330 0x7fed94155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed74066f90 0x7fed74069450 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 --2- 172.21.15.133:0/3089005993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed940b1f70 0x7fed94154e10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3089005993 >> 172.21.15.133:0/3089005993 conn(0x7fed94067000 msgr2=0x7fed940b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3089005993 shutdown_connections 2024-04-25T10:02:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:47.852+0000 7fed9a0f3640 1 -- 172.21.15.133:0/3089005993 wait complete. 2024-04-25T10:02:47.881 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:02:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ec5a0640 1 Processor -- start 2024-04-25T10:02:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ec5a0640 1 -- start start 2024-04-25T10:02:48.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ec5a0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76e415ee80 0x7f76e40af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ec5a0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e415d5c0 con 0x7f76e40acf20 2024-04-25T10:02:48.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ec5a0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e415cbf0 con 0x7f76e415c770 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ec5a0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4156200 con 0x7f76e415ee80 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76eab3d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f76e40acf20 0x7f76e406bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47054/0 (socket says 172.21.15.133:47054) 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76eab3d640 1 -- 172.21.15.133:0/2190570476 learned_addr learned my addr 172.21.15.133:0/2190570476 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ea33c640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76e415ee80 0x7f76e40af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403093300 0 0) 0x7f76e415d5c0 con 0x7f76e40acf20 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76cc003530 con 0x7f76e40acf20 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1481552429 0 0) 0x7f76cc003530 con 0x7f76e40acf20 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76e415ee80 msgr2=0x7f76e40af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76e415ee80 0x7f76e40af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 >> v1:172.21.15.162:6789/0 conn(0x7f76e415c770 legacy=0x7f76e415ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e406ae30 con 0x7f76e40acf20 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f76dc001680 con 0x7f76e40acf20 2024-04-25T10:02:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 >> v1:172.21.15.133:6789/0 conn(0x7f76e40acf20 legacy=0x7f76e406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76cc0039f0 0x7f76cc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76cc004810 0x7f76cc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76cc005190 0x7f76cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4156200 con 0x7f76cc0039f0 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc00bf10 con 0x7f76cc005190 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc00c0e0 con 0x7f76cc004810 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ea33c640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76cc0039f0 0x7f76cc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76e9b3b640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76cc004810 0x7f76cc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76eab3d640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76cc005190 0x7f76cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ea33c640 1 -- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76cc004810 msgr2=0x7f76cc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ea33c640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76cc004810 0x7f76cc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ea33c640 1 -- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76cc005190 msgr2=0x7f76cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ea33c640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76cc005190 0x7f76cc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76ea33c640 1 -- 172.21.15.133:0/2190570476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc003530 con 0x7f76cc0039f0 2024-04-25T10:02:48.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.004+0000 7f76eab3d640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76cc005190 0x7f76cc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:48.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ea33c640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76cc0039f0 0x7f76cc003e10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009fc0 tx=0x7f76d401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7dd1e5c8c97984 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d40b2070 con 0x7f76cc0039f0 2024-04-25T10:02:48.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d40a5b60 con 0x7f76cc0039f0 2024-04-25T10:02:48.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76e933a640 1 -- 172.21.15.133:0/2190570476 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4009c10 con 0x7f76cc0039f0 2024-04-25T10:02:48.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2190570476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76cc0039f0 msgr2=0x7f76cc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 --2- 172.21.15.133:0/2190570476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76cc0039f0 0x7f76cc003e10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009fc0 tx=0x7f76d401cc80 comp rx=0 tx=0).stop 2024-04-25T10:02:48.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76eab3d640 1 -- 172.21.15.133:0/2190570476 reap_dead start 2024-04-25T10:02:48.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2190570476 shutdown_connections 2024-04-25T10:02:48.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2190570476 >> 172.21.15.133:0/2190570476 conn(0x7f76e40cfd50 msgr2=0x7f76e40d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2190570476 shutdown_connections 2024-04-25T10:02:48.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2190570476 wait complete. 2024-04-25T10:02:48.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 Processor -- start 2024-04-25T10:02:48.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 -- start start 2024-04-25T10:02:48.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76cc0039f0 0x7f76e415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e406d7b0 0x7f76e406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76e414fef0 0x7f76e4150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e40ad330 con 0x7f76e406d7b0 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e406b6d0 con 0x7f76cc0039f0 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76ec5a0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e40ad4b0 con 0x7f76e414fef0 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.008+0000 7f76e9b3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e406d7b0 0x7f76e406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e9b3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e406d7b0 0x7f76e406dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46880/0 (socket says 172.21.15.133:46880) 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e9b3b640 1 -- 172.21.15.133:0/2189724752 learned_addr learned my addr 172.21.15.133:0/2189724752 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76eab3d640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76e414fef0 0x7f76e4150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76ea33c640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76cc0039f0 0x7f76e415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e9b3b640 1 -- 172.21.15.133:0/2189724752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76e414fef0 msgr2=0x7f76e4150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e9b3b640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76e414fef0 0x7f76e4150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e9b3b640 1 -- 172.21.15.133:0/2189724752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76cc0039f0 msgr2=0x7f76e415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e9b3b640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76cc0039f0 0x7f76e415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e9b3b640 1 -- 172.21.15.133:0/2189724752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e406e320 con 0x7f76e406d7b0 2024-04-25T10:02:48.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76ea33c640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76cc0039f0 0x7f76e415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e9b3b640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e406d7b0 0x7f76e406dc30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f76d800e4d0 tx=0x7f76d800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e2ffd640 1 -- 172.21.15.133:0/2189724752 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d80053a0 con 0x7f76e406d7b0 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e2ffd640 1 -- 172.21.15.133:0/2189724752 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8018070 con 0x7f76e406d7b0 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2189724752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e406e600 con 0x7f76e406d7b0 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e2ffd640 1 -- 172.21.15.133:0/2189724752 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8003820 con 0x7f76e406d7b0 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2189724752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e40cdd10 con 0x7f76e406d7b0 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2189724752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e4067c70 con 0x7f76e406d7b0 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e2ffd640 1 -- 172.21.15.133:0/2189724752 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76d80039c0 con 0x7f76e406d7b0 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e2ffd640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76d0066f90 0x7f76d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76e2ffd640 1 -- 172.21.15.133:0/2189724752 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f76d8085090 con 0x7f76e406d7b0 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76ea33c640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76d0066f90 0x7f76d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.012+0000 7f76ea33c640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76d0066f90 0x7f76d0069450 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f76d40af000 tx=0x7f76d40a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.016+0000 7f76e2ffd640 1 -- 172.21.15.133:0/2189724752 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76d8055d30 con 0x7f76e406d7b0 2024-04-25T10:02:48.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.176+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2189724752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f76e4158650 con 0x7f76e406d7b0 2024-04-25T10:02:48.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76e2ffd640 1 -- 172.21.15.133:0/2189724752 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f76d8055630 con 0x7f76e406d7b0 2024-04-25T10:02:48.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2189724752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76d0066f90 msgr2=0x7f76d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76d0066f90 0x7f76d0069450 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f76d40af000 tx=0x7f76d40a7040 comp rx=0 tx=0).stop 2024-04-25T10:02:48.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2189724752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e406d7b0 msgr2=0x7f76e406dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e406d7b0 0x7f76e406dc30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f76d800e4d0 tx=0x7f76d800e990 comp rx=0 tx=0).stop 2024-04-25T10:02:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2189724752 shutdown_connections 2024-04-25T10:02:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76e414fef0 0x7f76e4150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76d0066f90 0x7f76d0069450 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76e406d7b0 0x7f76e406dc30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 --2- 172.21.15.133:0/2189724752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76cc0039f0 0x7f76e415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2189724752 >> 172.21.15.133:0/2189724752 conn(0x7f76e40cfd50 msgr2=0x7f76e4166920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2189724752 shutdown_connections 2024-04-25T10:02:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.180+0000 7f76ec5a0640 1 -- 172.21.15.133:0/2189724752 wait complete. 2024-04-25T10:02:48.204 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:02:48.204 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:48.243 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:48.303 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:02:48.303 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:02:48.303 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:48.308 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:02:48.308 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:48.353 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:48.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6092586640 1 Processor -- start 2024-04-25T10:02:48.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6092586640 1 -- start start 2024-04-25T10:02:48.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6092586640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 0x7f608c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6092586640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f608c1584f0 0x7f608c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6092586640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c0cbe60 con 0x7f608c158110 2024-04-25T10:02:48.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6092586640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c0b19b0 con 0x7f608c1584f0 2024-04-25T10:02:48.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6092586640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c15b360 con 0x7f608c06eb70 2024-04-25T10:02:48.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6090b23640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f608c1584f0 0x7f608c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6090b23640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f608c1584f0 0x7f608c15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46886/0 (socket says 172.21.15.133:46886) 2024-04-25T10:02:48.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6090b23640 1 -- 172.21.15.133:0/1648793665 learned_addr learned my addr 172.21.15.133:0/1648793665 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:48.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6083fff640 1 --2- 172.21.15.133:0/1648793665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 0x7f608c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6090b23640 1 -- 172.21.15.133:0/1648793665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 msgr2=0x7f608c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6090b23640 1 --2- 172.21.15.133:0/1648793665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 0x7f608c0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6090b23640 1 -- 172.21.15.133:0/1648793665 >> v1:172.21.15.162:6789/0 conn(0x7f608c158110 legacy=0x7f608c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6090b23640 1 -- 172.21.15.133:0/1648793665 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c15b640 con 0x7f608c1584f0 2024-04-25T10:02:48.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.512+0000 7f6083fff640 1 --2- 172.21.15.133:0/1648793665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 0x7f608c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:48.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f6090b23640 1 --2- 172.21.15.133:0/1648793665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f608c1584f0 0x7f608c15ad10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6084009e50 tx=0x7f608401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1b45d4329eb0f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f608b7fe640 1 -- 172.21.15.133:0/1648793665 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60840a5860 con 0x7f608c1584f0 2024-04-25T10:02:48.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f608b7fe640 1 -- 172.21.15.133:0/1648793665 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60840a5a00 con 0x7f608c1584f0 2024-04-25T10:02:48.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f608b7fe640 1 -- 172.21.15.133:0/1648793665 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60840a5dd0 con 0x7f608c1584f0 2024-04-25T10:02:48.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f6092586640 1 -- 172.21.15.133:0/1648793665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f608c1584f0 msgr2=0x7f608c15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f6092586640 1 --2- 172.21.15.133:0/1648793665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f608c1584f0 0x7f608c15ad10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6084009e50 tx=0x7f608401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f6092586640 1 -- 172.21.15.133:0/1648793665 shutdown_connections 2024-04-25T10:02:48.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f6092586640 1 --2- 172.21.15.133:0/1648793665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f608c1584f0 0x7f608c15ad10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f6092586640 1 --2- 172.21.15.133:0/1648793665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 0x7f608c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f6092586640 1 -- 172.21.15.133:0/1648793665 >> 172.21.15.133:0/1648793665 conn(0x7f608c067000 msgr2=0x7f608c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f6092586640 1 -- 172.21.15.133:0/1648793665 shutdown_connections 2024-04-25T10:02:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f6092586640 1 -- 172.21.15.133:0/1648793665 wait complete. 2024-04-25T10:02:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f6092586640 1 Processor -- start 2024-04-25T10:02:48.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.516+0000 7f6092586640 1 -- start start 2024-04-25T10:02:48.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6092586640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 0x7f608c1f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6092586640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c158110 0x7f608c14ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6092586640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f608c1584f0 0x7f608c150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6092586640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0ccca0 con 0x7f608c1584f0 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6092586640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0ccb20 con 0x7f608c158110 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6092586640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0cce20 con 0x7f608c06eb70 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6090b23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f608c1584f0 0x7f608c150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6090b23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f608c1584f0 0x7f608c150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46888/0 (socket says 172.21.15.133:46888) 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6090b23640 1 -- 172.21.15.133:0/2070698302 learned_addr learned my addr 172.21.15.133:0/2070698302 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f608bfff640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 0x7f608c1f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6083fff640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c158110 0x7f608c14ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6090b23640 1 -- 172.21.15.133:0/2070698302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 msgr2=0x7f608c1f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6090b23640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 0x7f608c1f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6090b23640 1 -- 172.21.15.133:0/2070698302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c158110 msgr2=0x7f608c14ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6090b23640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c158110 0x7f608c14ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6090b23640 1 -- 172.21.15.133:0/2070698302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0b2e20 con 0x7f608c1584f0 2024-04-25T10:02:48.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f608bfff640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 0x7f608c1f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:48.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6090b23640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f608c1584f0 0x7f608c150930 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f60840af040 tx=0x7f6084019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f60897fa640 1 -- 172.21.15.133:0/2070698302 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60840b8880 con 0x7f608c1584f0 2024-04-25T10:02:48.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f60897fa640 1 -- 172.21.15.133:0/2070698302 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60840be070 con 0x7f608c1584f0 2024-04-25T10:02:48.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6092586640 1 -- 172.21.15.133:0/2070698302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c0b30d0 con 0x7f608c1584f0 2024-04-25T10:02:48.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f60897fa640 1 -- 172.21.15.133:0/2070698302 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60840b8c80 con 0x7f608c1584f0 2024-04-25T10:02:48.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6092586640 1 -- 172.21.15.133:0/2070698302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c0b2640 con 0x7f608c1584f0 2024-04-25T10:02:48.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f6092586640 1 -- 172.21.15.133:0/2070698302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608c06bc20 con 0x7f608c1584f0 2024-04-25T10:02:48.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f60897fa640 1 -- 172.21.15.133:0/2070698302 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60840c3460 con 0x7f608c1584f0 2024-04-25T10:02:48.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f60897fa640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f606c066f60 0x7f606c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f60897fa640 1 -- 172.21.15.133:0/2070698302 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f60841211d0 con 0x7f608c1584f0 2024-04-25T10:02:48.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f608bfff640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f606c066f60 0x7f606c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.520+0000 7f608bfff640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f606c066f60 0x7f606c069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f607c003e40 tx=0x7f607c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.524+0000 7f60897fa640 1 -- 172.21.15.133:0/2070698302 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60840165e0 con 0x7f608c1584f0 2024-04-25T10:02:48.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.672+0000 7f6092586640 1 -- 172.21.15.133:0/2070698302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f608c068ef0 con 0x7f608c1584f0 2024-04-25T10:02:48.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.672+0000 7f60897fa640 1 -- 172.21.15.133:0/2070698302 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f60840b8e20 con 0x7f608c1584f0 2024-04-25T10:02:48.681 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:48.681 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:48.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 -- 172.21.15.133:0/2070698302 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f606c066f60 msgr2=0x7f606c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f606c066f60 0x7f606c069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f607c003e40 tx=0x7f607c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:48.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 -- 172.21.15.133:0/2070698302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f608c1584f0 msgr2=0x7f608c150930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f608c1584f0 0x7f608c150930 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f60840af040 tx=0x7f6084019760 comp rx=0 tx=0).stop 2024-04-25T10:02:48.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 -- 172.21.15.133:0/2070698302 shutdown_connections 2024-04-25T10:02:48.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f606c066f60 0x7f606c069420 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f608c1584f0 0x7f608c150930 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f608c158110 0x7f608c14ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 --2- 172.21.15.133:0/2070698302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f608c06eb70 0x7f608c1f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 -- 172.21.15.133:0/2070698302 >> 172.21.15.133:0/2070698302 conn(0x7f608c067000 msgr2=0x7f608c06e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 -- 172.21.15.133:0/2070698302 shutdown_connections 2024-04-25T10:02:48.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.676+0000 7f6092586640 1 -- 172.21.15.133:0/2070698302 wait complete. 2024-04-25T10:02:48.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:48.700 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:48.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.804+0000 7f990eb65640 1 Processor -- start 2024-04-25T10:02:48.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990eb65640 1 -- start start 2024-04-25T10:02:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990eb65640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f990815d9a0 0x7f99080d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990eb65640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f99080d2030 0x7f99080cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990eb65640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990815ebd0 con 0x7f99080ce600 2024-04-25T10:02:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990eb65640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990815eeb0 con 0x7f99080d2030 2024-04-25T10:02:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990eb65640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99080d0330 con 0x7f990815d9a0 2024-04-25T10:02:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f9907fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f99080ce600 0x7f99080d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47070/0 (socket says 172.21.15.133:47070) 2024-04-25T10:02:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f9907fff640 1 -- 172.21.15.133:0/2269575235 learned_addr learned my addr 172.21.15.133:0/2269575235 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990d102640 1 --2- 172.21.15.133:0/2269575235 >> v2:172.21.15.133:3300/0 conn(0x7f99080d2030 0x7f99080cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990c901640 1 --2- 172.21.15.133:0/2269575235 >> v2:172.21.15.162:3300/0 conn(0x7f990815d9a0 0x7f99080d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f99077fe640 1 -- 172.21.15.133:0/2269575235 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467378766 0 0) 0x7f990815ebd0 con 0x7f99080ce600 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f99077fe640 1 -- 172.21.15.133:0/2269575235 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f0003530 con 0x7f99080ce600 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990d102640 1 -- 172.21.15.133:0/2269575235 >> v2:172.21.15.162:3300/0 conn(0x7f990815d9a0 msgr2=0x7f99080d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990d102640 1 --2- 172.21.15.133:0/2269575235 >> v2:172.21.15.162:3300/0 conn(0x7f990815d9a0 0x7f99080d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990d102640 1 -- 172.21.15.133:0/2269575235 >> v1:172.21.15.133:6789/0 conn(0x7f99080ce600 legacy=0x7f99080d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.808+0000 7f990d102640 1 -- 172.21.15.133:0/2269575235 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99080d0610 con 0x7f99080d2030 2024-04-25T10:02:48.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990d102640 1 --2- 172.21.15.133:0/2269575235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99080d2030 0x7f99080cea90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f99080d4750 tx=0x7f990001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be974a9882ed3c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f99077fe640 1 -- 172.21.15.133:0/2269575235 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99000b0070 con 0x7f99080d2030 2024-04-25T10:02:48.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f99077fe640 1 -- 172.21.15.133:0/2269575235 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99000ab600 con 0x7f99080d2030 2024-04-25T10:02:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f99077fe640 1 -- 172.21.15.133:0/2269575235 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99000ab9b0 con 0x7f99080d2030 2024-04-25T10:02:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 -- 172.21.15.133:0/2269575235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99080d2030 msgr2=0x7f99080cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 --2- 172.21.15.133:0/2269575235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99080d2030 0x7f99080cea90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f99080d4750 tx=0x7f990001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 -- 172.21.15.133:0/2269575235 shutdown_connections 2024-04-25T10:02:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 --2- 172.21.15.133:0/2269575235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99080d2030 0x7f99080cea90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 --2- 172.21.15.133:0/2269575235 >> v2:172.21.15.162:3300/0 conn(0x7f990815d9a0 0x7f99080d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 -- 172.21.15.133:0/2269575235 >> 172.21.15.133:0/2269575235 conn(0x7f990806bf80 msgr2=0x7f990806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 -- 172.21.15.133:0/2269575235 shutdown_connections 2024-04-25T10:02:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 -- 172.21.15.133:0/2269575235 wait complete. 2024-04-25T10:02:48.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 Processor -- start 2024-04-25T10:02:48.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 -- start start 2024-04-25T10:02:48.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99080ce600 0x7f990815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99080d2030 0x7f99080c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f990815d9a0 0x7f99080c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99080b9820 con 0x7f99080d2030 2024-04-25T10:02:48.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99080b96a0 con 0x7f99080ce600 2024-04-25T10:02:48.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990815e590 con 0x7f990815d9a0 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f9907fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99080d2030 0x7f99080c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f9907fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99080d2030 0x7f99080c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46904/0 (socket says 172.21.15.133:46904) 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f9907fff640 1 -- 172.21.15.133:0/1186896255 learned_addr learned my addr 172.21.15.133:0/1186896255 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990d102640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f990815d9a0 0x7f99080c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f9907fff640 1 -- 172.21.15.133:0/1186896255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f990815d9a0 msgr2=0x7f99080c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f9907fff640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f990815d9a0 0x7f99080c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f9907fff640 1 -- 172.21.15.133:0/1186896255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99080ce600 msgr2=0x7f990815ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f9907fff640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99080ce600 0x7f990815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f9907fff640 1 -- 172.21.15.133:0/1186896255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990815a6e0 con 0x7f99080d2030 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f9907fff640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99080d2030 0x7f99080c9630 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00cb40 tx=0x7f98fc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f99057fa640 1 -- 172.21.15.133:0/1186896255 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc017070 con 0x7f99080d2030 2024-04-25T10:02:48.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f99057fa640 1 -- 172.21.15.133:0/1186896255 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc00aca0 con 0x7f99080d2030 2024-04-25T10:02:48.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.812+0000 7f990eb65640 1 -- 172.21.15.133:0/1186896255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990815a9a0 con 0x7f99080d2030 2024-04-25T10:02:48.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.816+0000 7f99057fa640 1 -- 172.21.15.133:0/1186896255 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc012620 con 0x7f99080d2030 2024-04-25T10:02:48.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.816+0000 7f990eb65640 1 -- 172.21.15.133:0/1186896255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990815b240 con 0x7f99080d2030 2024-04-25T10:02:48.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.816+0000 7f990eb65640 1 -- 172.21.15.133:0/1186896255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9908068e80 con 0x7f99080d2030 2024-04-25T10:02:48.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.816+0000 7f99057fa640 1 -- 172.21.15.133:0/1186896255 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98fc01a070 con 0x7f99080d2030 2024-04-25T10:02:48.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.816+0000 7f99057fa640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98f4066f90 0x7f98f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.816+0000 7f99057fa640 1 -- 172.21.15.133:0/1186896255 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f98fc085160 con 0x7f99080d2030 2024-04-25T10:02:48.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.816+0000 7f990c901640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98f4066f90 0x7f98f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.816+0000 7f990c901640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98f4066f90 0x7f98f4069450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f98f8006aa0 tx=0x7f98f8006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.820+0000 7f99057fa640 1 -- 172.21.15.133:0/1186896255 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98fc055e00 con 0x7f99080d2030 2024-04-25T10:02:48.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 -- 172.21.15.133:0/1186896255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99080c70e0 con 0x7f99080d2030 2024-04-25T10:02:48.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f99057fa640 1 -- 172.21.15.133:0/1186896255 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98fc055700 con 0x7f99080d2030 2024-04-25T10:02:48.975 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:48.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 -- 172.21.15.133:0/1186896255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98f4066f90 msgr2=0x7f98f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98f4066f90 0x7f98f4069450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f98f8006aa0 tx=0x7f98f8006a30 comp rx=0 tx=0).stop 2024-04-25T10:02:48.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 -- 172.21.15.133:0/1186896255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99080d2030 msgr2=0x7f99080c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99080d2030 0x7f99080c9630 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00cb40 tx=0x7f98fc00a060 comp rx=0 tx=0).stop 2024-04-25T10:02:48.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 -- 172.21.15.133:0/1186896255 shutdown_connections 2024-04-25T10:02:48.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f990815d9a0 0x7f99080c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98f4066f90 0x7f98f4069450 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99080d2030 0x7f99080c9630 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 --2- 172.21.15.133:0/1186896255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99080ce600 0x7f990815ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 -- 172.21.15.133:0/1186896255 >> 172.21.15.133:0/1186896255 conn(0x7f990806bf80 msgr2=0x7f99080d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 -- 172.21.15.133:0/1186896255 shutdown_connections 2024-04-25T10:02:48.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:48.968+0000 7f990eb65640 1 -- 172.21.15.133:0/1186896255 wait complete. 2024-04-25T10:02:48.992 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:49.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e34593640 1 Processor -- start 2024-04-25T10:02:49.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e34593640 1 -- start start 2024-04-25T10:02:49.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e34593640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5e2c15b980 0x7f5e2c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e34593640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2c0ce6a0 0x7f5e2c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e34593640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c0d02f0 con 0x7f5e2c0d2100 2024-04-25T10:02:49.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e34593640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2c15bd60 con 0x7f5e2c15b980 2024-04-25T10:02:49.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e34593640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0cd770 con 0x7f5e2c0ce6a0 2024-04-25T10:02:49.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e31b2e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5e2c0d2100 0x7f5e2c0cdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47076/0 (socket says 172.21.15.133:47076) 2024-04-25T10:02:49.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e31b2e640 1 -- 172.21.15.133:0/1585307178 learned_addr learned my addr 172.21.15.133:0/1585307178 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:49.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655116556 0 0) 0x7f5e2c0d02f0 con 0x7f5e2c0d2100 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e18003530 con 0x7f5e2c0d2100 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e32b30640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2c0ce6a0 0x7f5e2c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3232f640 1 --2- 172.21.15.133:0/1585307178 >> v2:172.21.15.162:3300/0 conn(0x7f5e2c15b980 0x7f5e2c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 246590941 0 0) 0x7f5e18003530 con 0x7f5e2c0d2100 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2c0ce6a0 msgr2=0x7f5e2c06fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2c0ce6a0 0x7f5e2c06fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 >> v2:172.21.15.162:3300/0 conn(0x7f5e2c15b980 msgr2=0x7f5e2c0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 --2- 172.21.15.133:0/1585307178 >> v2:172.21.15.162:3300/0 conn(0x7f5e2c15b980 0x7f5e2c0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0d05d0 con 0x7f5e2c0d2100 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5e1c001680 con 0x7f5e2c0d2100 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 >> v1:172.21.15.133:6789/0 conn(0x7f5e2c0d2100 legacy=0x7f5e2c0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2c0ce6a0 0x7f5e180039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e18003ff0 0x7f5e18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e18004d50 0x7f5e180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1800ba80 con 0x7f5e18004d50 2024-04-25T10:02:49.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1800bc00 con 0x7f5e18003ff0 2024-04-25T10:02:49.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.108+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1800bdd0 con 0x7f5e2c0ce6a0 2024-04-25T10:02:49.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e32b30640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e18004d50 0x7f5e180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e3232f640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2c0ce6a0 0x7f5e180039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e31b2e640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e18003ff0 0x7f5e18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e32b30640 1 -- 172.21.15.133:0/1585307178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2c0ce6a0 msgr2=0x7f5e180039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:49.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e32b30640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e2c0ce6a0 0x7f5e180039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e32b30640 1 -- 172.21.15.133:0/1585307178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e18003ff0 msgr2=0x7f5e18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:49.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e32b30640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e18003ff0 0x7f5e18004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e32b30640 1 -- 172.21.15.133:0/1585307178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e18003530 con 0x7f5e18004d50 2024-04-25T10:02:49.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e32b30640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e18004d50 0x7f5e180051d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0d15e0 tx=0x7f5e2401cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4f6512edacb4bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e240b2070 con 0x7f5e18004d50 2024-04-25T10:02:49.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e240a5cc0 con 0x7f5e18004d50 2024-04-25T10:02:49.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e3132d640 1 -- 172.21.15.133:0/1585307178 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24009be0 con 0x7f5e18004d50 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e34593640 1 -- 172.21.15.133:0/1585307178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e18004d50 msgr2=0x7f5e180051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e34593640 1 --2- 172.21.15.133:0/1585307178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e18004d50 0x7f5e180051d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0d15e0 tx=0x7f5e2401cdc0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e32b30640 1 -- 172.21.15.133:0/1585307178 reap_dead start 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e34593640 1 -- 172.21.15.133:0/1585307178 shutdown_connections 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e34593640 1 -- 172.21.15.133:0/1585307178 >> 172.21.15.133:0/1585307178 conn(0x7f5e2c0d1610 msgr2=0x7f5e2c0d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e34593640 1 -- 172.21.15.133:0/1585307178 shutdown_connections 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.112+0000 7f5e34593640 1 -- 172.21.15.133:0/1585307178 wait complete. 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e34593640 1 Processor -- start 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e34593640 1 -- start start 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e34593640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e18004d50 0x7f5e2c15a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e34593640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c06cc10 0x7f5e2c153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e34593640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2c0b5d70 0x7f5e2c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e34593640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c06f800 con 0x7f5e2c06cc10 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e34593640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c06f240 con 0x7f5e2c0b5d70 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e34593640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c06fc90 con 0x7f5e18004d50 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e31b2e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c06cc10 0x7f5e2c153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e31b2e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c06cc10 0x7f5e2c153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46908/0 (socket says 172.21.15.133:46908) 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e31b2e640 1 -- 172.21.15.133:0/1549520351 learned_addr learned my addr 172.21.15.133:0/1549520351 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e32b30640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2c0b5d70 0x7f5e2c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e3232f640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e18004d50 0x7f5e2c15a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e31b2e640 1 -- 172.21.15.133:0/1549520351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e18004d50 msgr2=0x7f5e2c15a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e31b2e640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e18004d50 0x7f5e2c15a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e31b2e640 1 -- 172.21.15.133:0/1549520351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2c0b5d70 msgr2=0x7f5e2c0b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e31b2e640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2c0b5d70 0x7f5e2c0b61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e31b2e640 1 -- 172.21.15.133:0/1549520351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c155350 con 0x7f5e2c06cc10 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e32b30640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2c0b5d70 0x7f5e2c0b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:49.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e31b2e640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c06cc10 0x7f5e2c153a30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e880 tx=0x7f5e1c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e22ffd640 1 -- 172.21.15.133:0/1549520351 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00c8d0 con 0x7f5e2c06cc10 2024-04-25T10:02:49.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e22ffd640 1 -- 172.21.15.133:0/1549520351 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c018070 con 0x7f5e2c06cc10 2024-04-25T10:02:49.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e22ffd640 1 -- 172.21.15.133:0/1549520351 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00ccd0 con 0x7f5e2c06cc10 2024-04-25T10:02:49.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e34593640 1 -- 172.21.15.133:0/1549520351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c06d460 con 0x7f5e2c06cc10 2024-04-25T10:02:49.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e34593640 1 -- 172.21.15.133:0/1549520351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c06da20 con 0x7f5e2c06cc10 2024-04-25T10:02:49.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e34593640 1 -- 172.21.15.133:0/1549520351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c067c70 con 0x7f5e2c06cc10 2024-04-25T10:02:49.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e22ffd640 1 -- 172.21.15.133:0/1549520351 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e1c01d460 con 0x7f5e2c06cc10 2024-04-25T10:02:49.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.116+0000 7f5e22ffd640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e10066f60 0x7f5e10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.120+0000 7f5e22ffd640 1 -- 172.21.15.133:0/1549520351 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5e1c085410 con 0x7f5e2c06cc10 2024-04-25T10:02:49.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.120+0000 7f5e3232f640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e10066f60 0x7f5e10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.120+0000 7f5e3232f640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e10066f60 0x7f5e10069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800ba30 tx=0x7f5e28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.124+0000 7f5e22ffd640 1 -- 172.21.15.133:0/1549520351 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e1c056020 con 0x7f5e2c06cc10 2024-04-25T10:02:49.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.268+0000 7f5e34593640 1 -- 172.21.15.133:0/1549520351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e2c1e1030 con 0x7f5e2c06cc10 2024-04-25T10:02:49.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.268+0000 7f5e22ffd640 1 -- 172.21.15.133:0/1549520351 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e1c055920 con 0x7f5e2c06cc10 2024-04-25T10:02:49.275 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:49.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.268+0000 7f5e34593640 1 -- 172.21.15.133:0/1549520351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e10066f60 msgr2=0x7f5e10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.268+0000 7f5e34593640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e10066f60 0x7f5e10069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800ba30 tx=0x7f5e28008040 comp rx=0 tx=0).stop 2024-04-25T10:02:49.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.268+0000 7f5e34593640 1 -- 172.21.15.133:0/1549520351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c06cc10 msgr2=0x7f5e2c153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.268+0000 7f5e34593640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c06cc10 0x7f5e2c153a30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e880 tx=0x7f5e1c00ed40 comp rx=0 tx=0).stop 2024-04-25T10:02:49.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.268+0000 7f5e34593640 1 -- 172.21.15.133:0/1549520351 shutdown_connections 2024-04-25T10:02:49.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.268+0000 7f5e34593640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e2c0b5d70 0x7f5e2c0b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.268+0000 7f5e34593640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e10066f60 0x7f5e10069420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.268+0000 7f5e34593640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e2c06cc10 0x7f5e2c153a30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.268+0000 7f5e34593640 1 --2- 172.21.15.133:0/1549520351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5e18004d50 0x7f5e2c15a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.272+0000 7f5e34593640 1 -- 172.21.15.133:0/1549520351 >> 172.21.15.133:0/1549520351 conn(0x7f5e2c0d1610 msgr2=0x7f5e2c0cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.272+0000 7f5e34593640 1 -- 172.21.15.133:0/1549520351 shutdown_connections 2024-04-25T10:02:49.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.272+0000 7f5e34593640 1 -- 172.21.15.133:0/1549520351 wait complete. 2024-04-25T10:02:49.292 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:49.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdac6a59640 1 Processor -- start 2024-04-25T10:02:49.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdac6a59640 1 -- start start 2024-04-25T10:02:49.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdac6a59640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdac0143ea0 0x7fdac00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdac6a59640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac006eac0 0x7fdac00b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdac6a59640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac015ac50 con 0x7fdac00b2150 2024-04-25T10:02:49.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdac6a59640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac00cd180 con 0x7fdac0143ea0 2024-04-25T10:02:49.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdac6a59640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac015aed0 con 0x7fdac006eac0 2024-04-25T10:02:49.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdabffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdac0143ea0 0x7fdac00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdabffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdac0143ea0 0x7fdac00cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46912/0 (socket says 172.21.15.133:46912) 2024-04-25T10:02:49.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdabffff640 1 -- 172.21.15.133:0/953272625 learned_addr learned my addr 172.21.15.133:0/953272625 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:49.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdac4ff6640 1 --2- 172.21.15.133:0/953272625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac006eac0 0x7fdac00b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdabffff640 1 -- 172.21.15.133:0/953272625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac006eac0 msgr2=0x7fdac00b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:49.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdabffff640 1 --2- 172.21.15.133:0/953272625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac006eac0 0x7fdac00b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdabffff640 1 -- 172.21.15.133:0/953272625 >> v1:172.21.15.162:6789/0 conn(0x7fdac00b2150 legacy=0x7fdac006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:49.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdabffff640 1 -- 172.21.15.133:0/953272625 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac015b1b0 con 0x7fdac0143ea0 2024-04-25T10:02:49.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.412+0000 7fdabffff640 1 --2- 172.21.15.133:0/953272625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdac0143ea0 0x7fdac00cc810 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdab000bb10 tx=0x7fdab001ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab8c951c86f84c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdabf7fe640 1 -- 172.21.15.133:0/953272625 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab00b4070 con 0x7fdac0143ea0 2024-04-25T10:02:49.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdabf7fe640 1 -- 172.21.15.133:0/953272625 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab00a79b0 con 0x7fdac0143ea0 2024-04-25T10:02:49.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdabf7fe640 1 -- 172.21.15.133:0/953272625 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab00a7d80 con 0x7fdac0143ea0 2024-04-25T10:02:49.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 -- 172.21.15.133:0/953272625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdac0143ea0 msgr2=0x7fdac00cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 --2- 172.21.15.133:0/953272625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdac0143ea0 0x7fdac00cc810 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdab000bb10 tx=0x7fdab001ea10 comp rx=0 tx=0).stop 2024-04-25T10:02:49.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 -- 172.21.15.133:0/953272625 shutdown_connections 2024-04-25T10:02:49.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 --2- 172.21.15.133:0/953272625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac006eac0 0x7fdac00b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 --2- 172.21.15.133:0/953272625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdac0143ea0 0x7fdac00cc810 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 -- 172.21.15.133:0/953272625 >> 172.21.15.133:0/953272625 conn(0x7fdac0066f70 msgr2=0x7fdac006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 -- 172.21.15.133:0/953272625 shutdown_connections 2024-04-25T10:02:49.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 -- 172.21.15.133:0/953272625 wait complete. 2024-04-25T10:02:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 Processor -- start 2024-04-25T10:02:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 -- start start 2024-04-25T10:02:49.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006eac0 0x7fdac0157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b2150 0x7fdac0152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdac0143ea0 0x7fdac01529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac006d420 con 0x7fdac0143ea0 2024-04-25T10:02:49.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac006d2a0 con 0x7fdac006eac0 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac006d5a0 con 0x7fdac00b2150 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac4ff6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdac0143ea0 0x7fdac01529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac4ff6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdac0143ea0 0x7fdac01529b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46914/0 (socket says 172.21.15.133:46914) 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac4ff6640 1 -- 172.21.15.133:0/3620687762 learned_addr learned my addr 172.21.15.133:0/3620687762 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdabffff640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006eac0 0x7fdac0157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdab77fe640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b2150 0x7fdac0152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac4ff6640 1 -- 172.21.15.133:0/3620687762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b2150 msgr2=0x7fdac0152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac4ff6640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b2150 0x7fdac0152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac4ff6640 1 -- 172.21.15.133:0/3620687762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006eac0 msgr2=0x7fdac0157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac4ff6640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006eac0 0x7fdac0157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac4ff6640 1 -- 172.21.15.133:0/3620687762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac0156fb0 con 0x7fdac0143ea0 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdabffff640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006eac0 0x7fdac0157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac4ff6640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdac0143ea0 0x7fdac01529b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdab8007e30 tx=0x7fdab800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdabd7fa640 1 -- 172.21.15.133:0/3620687762 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8014620 con 0x7fdac0143ea0 2024-04-25T10:02:49.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdabd7fa640 1 -- 172.21.15.133:0/3620687762 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab8019070 con 0x7fdac0143ea0 2024-04-25T10:02:49.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 -- 172.21.15.133:0/3620687762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac0157290 con 0x7fdac0143ea0 2024-04-25T10:02:49.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdabd7fa640 1 -- 172.21.15.133:0/3620687762 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8014a00 con 0x7fdac0143ea0 2024-04-25T10:02:49.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.416+0000 7fdac6a59640 1 -- 172.21.15.133:0/3620687762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac0157850 con 0x7fdac0143ea0 2024-04-25T10:02:49.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.420+0000 7fdac6a59640 1 -- 172.21.15.133:0/3620687762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac0068e80 con 0x7fdac0143ea0 2024-04-25T10:02:49.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.420+0000 7fdabd7fa640 1 -- 172.21.15.133:0/3620687762 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdab8014ba0 con 0x7fdac0143ea0 2024-04-25T10:02:49.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.420+0000 7fdabd7fa640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdaa0066f60 0x7fdaa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.420+0000 7fdabd7fa640 1 -- 172.21.15.133:0/3620687762 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdab8085cc0 con 0x7fdac0143ea0 2024-04-25T10:02:49.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.420+0000 7fdabffff640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdaa0066f60 0x7fdaa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.420+0000 7fdabffff640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdaa0066f60 0x7fdaa0069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdab00b1000 tx=0x7fdab0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.424+0000 7fdabd7fa640 1 -- 172.21.15.133:0/3620687762 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdab80568d0 con 0x7fdac0143ea0 2024-04-25T10:02:49.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 -- 172.21.15.133:0/3620687762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdac00ce990 con 0x7fdac0143ea0 2024-04-25T10:02:49.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdabd7fa640 1 -- 172.21.15.133:0/3620687762 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdab80561d0 con 0x7fdac0143ea0 2024-04-25T10:02:49.578 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:49.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 -- 172.21.15.133:0/3620687762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdaa0066f60 msgr2=0x7fdaa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdaa0066f60 0x7fdaa0069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdab00b1000 tx=0x7fdab0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:49.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 -- 172.21.15.133:0/3620687762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdac0143ea0 msgr2=0x7fdac01529b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdac0143ea0 0x7fdac01529b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdab8007e30 tx=0x7fdab800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 -- 172.21.15.133:0/3620687762 shutdown_connections 2024-04-25T10:02:49.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdaa0066f60 0x7fdaa0069420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdac0143ea0 0x7fdac01529b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdac00b2150 0x7fdac0152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 --2- 172.21.15.133:0/3620687762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdac006eac0 0x7fdac0157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 -- 172.21.15.133:0/3620687762 >> 172.21.15.133:0/3620687762 conn(0x7fdac0066f70 msgr2=0x7fdac0158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 -- 172.21.15.133:0/3620687762 shutdown_connections 2024-04-25T10:02:49.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.572+0000 7fdac6a59640 1 -- 172.21.15.133:0/3620687762 wait complete. 2024-04-25T10:02:49.596 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:49.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b8614640 1 Processor -- start 2024-04-25T10:02:49.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b8614640 1 -- start start 2024-04-25T10:02:49.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b8614640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f69b0145170 0x7f69b006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b8614640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f69b006e4d0 0x7f69b00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b8614640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b00ce2e0 con 0x7f69b00cdd60 2024-04-25T10:02:49.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b8614640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b006db00 con 0x7f69b006e4d0 2024-04-25T10:02:49.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b8614640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b0145550 con 0x7f69b0145170 2024-04-25T10:02:49.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b5baf640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f69b006e4d0 0x7f69b00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b6bb1640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f69b00cdd60 0x7f69b015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47090/0 (socket says 172.21.15.133:47090) 2024-04-25T10:02:49.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b6bb1640 1 -- 172.21.15.133:0/758667004 learned_addr learned my addr 172.21.15.133:0/758667004 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:49.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b63b0640 1 --2- 172.21.15.133:0/758667004 >> v2:172.21.15.162:3300/0 conn(0x7f69b0145170 0x7f69b006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b5baf640 1 -- 172.21.15.133:0/758667004 >> v2:172.21.15.162:3300/0 conn(0x7f69b0145170 msgr2=0x7f69b006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:49.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b5baf640 1 --2- 172.21.15.133:0/758667004 >> v2:172.21.15.162:3300/0 conn(0x7f69b0145170 0x7f69b006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b5baf640 1 -- 172.21.15.133:0/758667004 >> v1:172.21.15.133:6789/0 conn(0x7f69b00cdd60 legacy=0x7f69b015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b53ae640 1 -- 172.21.15.133:0/758667004 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618231343 0 0) 0x7f69b00ce2e0 con 0x7f69b00cdd60 2024-04-25T10:02:49.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.708+0000 7f69b5baf640 1 -- 172.21.15.133:0/758667004 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b006d4b0 con 0x7f69b006e4d0 2024-04-25T10:02:49.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b5baf640 1 --2- 172.21.15.133:0/758667004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69b006e4d0 0x7f69b00cd820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f69b006b790 tx=0x7f69a001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5de6304bf0cf39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b53ae640 1 -- 172.21.15.133:0/758667004 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a00b0070 con 0x7f69b006e4d0 2024-04-25T10:02:49.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b53ae640 1 -- 172.21.15.133:0/758667004 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a00ab600 con 0x7f69b006e4d0 2024-04-25T10:02:49.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b53ae640 1 -- 172.21.15.133:0/758667004 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a00ab9d0 con 0x7f69b006e4d0 2024-04-25T10:02:49.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 -- 172.21.15.133:0/758667004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69b006e4d0 msgr2=0x7f69b00cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 --2- 172.21.15.133:0/758667004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69b006e4d0 0x7f69b00cd820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f69b006b790 tx=0x7f69a001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 -- 172.21.15.133:0/758667004 shutdown_connections 2024-04-25T10:02:49.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 --2- 172.21.15.133:0/758667004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69b006e4d0 0x7f69b00cd820 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 --2- 172.21.15.133:0/758667004 >> v2:172.21.15.162:3300/0 conn(0x7f69b0145170 0x7f69b006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 -- 172.21.15.133:0/758667004 >> 172.21.15.133:0/758667004 conn(0x7f69b0066f70 msgr2=0x7f69b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 -- 172.21.15.133:0/758667004 shutdown_connections 2024-04-25T10:02:49.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 -- 172.21.15.133:0/758667004 wait complete. 2024-04-25T10:02:49.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 Processor -- start 2024-04-25T10:02:49.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 -- start start 2024-04-25T10:02:49.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69b006e4d0 0x7f69b01344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69b00cdd60 0x7f69b013cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69b0145170 0x7f69b00b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b0168490 con 0x7f69b0145170 2024-04-25T10:02:49.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b015d7f0 con 0x7f69b00cdd60 2024-04-25T10:02:49.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b8614640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b0168610 con 0x7f69b006e4d0 2024-04-25T10:02:49.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b6bb1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69b0145170 0x7f69b00b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b6bb1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69b0145170 0x7f69b00b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46934/0 (socket says 172.21.15.133:46934) 2024-04-25T10:02:49.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b6bb1640 1 -- 172.21.15.133:0/2860673215 learned_addr learned my addr 172.21.15.133:0/2860673215 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:49.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b5baf640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69b00cdd60 0x7f69b013cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b63b0640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69b006e4d0 0x7f69b01344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b6bb1640 1 -- 172.21.15.133:0/2860673215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69b006e4d0 msgr2=0x7f69b01344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b6bb1640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69b006e4d0 0x7f69b01344b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b6bb1640 1 -- 172.21.15.133:0/2860673215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69b00cdd60 msgr2=0x7f69b013cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b6bb1640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69b00cdd60 0x7f69b013cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.712+0000 7f69b6bb1640 1 -- 172.21.15.133:0/2860673215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b0154ea0 con 0x7f69b0145170 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69b6bb1640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69b0145170 0x7f69b00b3c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f69a800c7a0 tx=0x7f69a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69a6ffd640 1 -- 172.21.15.133:0/2860673215 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8018070 con 0x7f69b0145170 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69a6ffd640 1 -- 172.21.15.133:0/2860673215 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a800a640 con 0x7f69b0145170 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69b8614640 1 -- 172.21.15.133:0/2860673215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b0155180 con 0x7f69b0145170 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69a6ffd640 1 -- 172.21.15.133:0/2860673215 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a800aab0 con 0x7f69b0145170 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69b8614640 1 -- 172.21.15.133:0/2860673215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b0155740 con 0x7f69b0145170 2024-04-25T10:02:49.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69b8614640 1 -- 172.21.15.133:0/2860673215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b0068e80 con 0x7f69b0145170 2024-04-25T10:02:49.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69a6ffd640 1 -- 172.21.15.133:0/2860673215 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69a801b070 con 0x7f69b0145170 2024-04-25T10:02:49.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69a6ffd640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6994066f90 0x7f6994069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69a6ffd640 1 -- 172.21.15.133:0/2860673215 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f69a8084a20 con 0x7f69b0145170 2024-04-25T10:02:49.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69b63b0640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6994066f90 0x7f6994069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.716+0000 7f69b63b0640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6994066f90 0x7f6994069450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009790 tx=0x7f69ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.720+0000 7f69a6ffd640 1 -- 172.21.15.133:0/2860673215 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f69a8055630 con 0x7f69b0145170 2024-04-25T10:02:49.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69b8614640 1 -- 172.21.15.133:0/2860673215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f69b013e590 con 0x7f69b0145170 2024-04-25T10:02:49.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69a6ffd640 1 -- 172.21.15.133:0/2860673215 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f69a8054f30 con 0x7f69b0145170 2024-04-25T10:02:49.883 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69b8614640 1 -- 172.21.15.133:0/2860673215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6994066f90 msgr2=0x7f6994069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69b8614640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6994066f90 0x7f6994069450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009790 tx=0x7f69ac008040 comp rx=0 tx=0).stop 2024-04-25T10:02:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69b8614640 1 -- 172.21.15.133:0/2860673215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69b0145170 msgr2=0x7f69b00b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69b8614640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69b0145170 0x7f69b00b3c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f69a800c7a0 tx=0x7f69a800cc60 comp rx=0 tx=0).stop 2024-04-25T10:02:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69b8614640 1 -- 172.21.15.133:0/2860673215 shutdown_connections 2024-04-25T10:02:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69b8614640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6994066f90 0x7f6994069450 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69b8614640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69b0145170 0x7f69b00b3c90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69b8614640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69b00cdd60 0x7f69b013cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69b8614640 1 --2- 172.21.15.133:0/2860673215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f69b006e4d0 0x7f69b01344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.876+0000 7f69b8614640 1 -- 172.21.15.133:0/2860673215 >> 172.21.15.133:0/2860673215 conn(0x7f69b0066f70 msgr2=0x7f69b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.880+0000 7f69b8614640 1 -- 172.21.15.133:0/2860673215 shutdown_connections 2024-04-25T10:02:49.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:49.880+0000 7f69b8614640 1 -- 172.21.15.133:0/2860673215 wait complete. 2024-04-25T10:02:49.900 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:50.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9c4be4640 1 Processor -- start 2024-04-25T10:02:50.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9c4be4640 1 -- start start 2024-04-25T10:02:50.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9c4be4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9c015ba20 0x7fb9c015f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9c4be4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb9c015ff00 0x7fb9c006f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9c4be4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c015be00 con 0x7fb9c015faa0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9c4be4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c00cfb40 con 0x7fb9c015ff00 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9c4be4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c006b0a0 con 0x7fb9c015ba20 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9bdd9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb9c015faa0 0x7fb9c006e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47092/0 (socket says 172.21.15.133:47092) 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9bdd9b640 1 -- 172.21.15.133:0/216145782 learned_addr learned my addr 172.21.15.133:0/216145782 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9bed9d640 1 --2- 172.21.15.133:0/216145782 >> v2:172.21.15.133:3300/0 conn(0x7fb9c015ff00 0x7fb9c006f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9be59c640 1 --2- 172.21.15.133:0/216145782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9c015ba20 0x7fb9c015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9bd59a640 1 -- 172.21.15.133:0/216145782 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329220178 0 0) 0x7fb9b8001120 con 0x7fb9c015faa0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9bd59a640 1 -- 172.21.15.133:0/216145782 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ac003530 con 0x7fb9c015faa0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9bed9d640 1 -- 172.21.15.133:0/216145782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9c015ba20 msgr2=0x7fb9c015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9bd59a640 1 -- 172.21.15.133:0/216145782 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2822426397 0 0) 0x7fb9ac003530 con 0x7fb9c015faa0 2024-04-25T10:02:50.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9bed9d640 1 --2- 172.21.15.133:0/216145782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9c015ba20 0x7fb9c015f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9bed9d640 1 -- 172.21.15.133:0/216145782 >> v1:172.21.15.133:6789/0 conn(0x7fb9c015faa0 legacy=0x7fb9c006e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.004+0000 7fb9bed9d640 1 -- 172.21.15.133:0/216145782 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c0160390 con 0x7fb9c015ff00 2024-04-25T10:02:50.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9bed9d640 1 --2- 172.21.15.133:0/216145782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9c015ff00 0x7fb9c006f0b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80097c0 tx=0x7fb9a801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f706e0996cef4865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9bd59a640 1 -- 172.21.15.133:0/216145782 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a80b0070 con 0x7fb9c015ff00 2024-04-25T10:02:50.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9bd59a640 1 -- 172.21.15.133:0/216145782 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a80ab400 con 0x7fb9c015ff00 2024-04-25T10:02:50.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9bd59a640 1 -- 172.21.15.133:0/216145782 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8001850 con 0x7fb9c015ff00 2024-04-25T10:02:50.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9c4be4640 1 -- 172.21.15.133:0/216145782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9c015ff00 msgr2=0x7fb9c006f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9c4be4640 1 --2- 172.21.15.133:0/216145782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9c015ff00 0x7fb9c006f0b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80097c0 tx=0x7fb9a801c330 comp rx=0 tx=0).stop 2024-04-25T10:02:50.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9c4be4640 1 -- 172.21.15.133:0/216145782 shutdown_connections 2024-04-25T10:02:50.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9c4be4640 1 --2- 172.21.15.133:0/216145782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9c015ff00 0x7fb9c006f0b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9c4be4640 1 --2- 172.21.15.133:0/216145782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9c015ba20 0x7fb9c015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9c4be4640 1 -- 172.21.15.133:0/216145782 >> 172.21.15.133:0/216145782 conn(0x7fb9c00cc750 msgr2=0x7fb9c00ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9c4be4640 1 -- 172.21.15.133:0/216145782 shutdown_connections 2024-04-25T10:02:50.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9c4be4640 1 -- 172.21.15.133:0/216145782 wait complete. 2024-04-25T10:02:50.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9c4be4640 1 Processor -- start 2024-04-25T10:02:50.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.012+0000 7fb9c4be4640 1 -- start start 2024-04-25T10:02:50.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9c4be4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9c015ba20 0x7fb9c0151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9c4be4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9c015faa0 0x7fb9c0150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9c4be4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9c015ff00 0x7fb9c0153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9c4be4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00669a0 con 0x7fb9c015faa0 2024-04-25T10:02:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9c4be4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00cdbd0 con 0x7fb9c015ba20 2024-04-25T10:02:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9c4be4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0066b20 con 0x7fb9c015ff00 2024-04-25T10:02:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9bdd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9c015faa0 0x7fb9c0150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9bdd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9c015faa0 0x7fb9c0150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46954/0 (socket says 172.21.15.133:46954) 2024-04-25T10:02:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9bdd9b640 1 -- 172.21.15.133:0/963703727 learned_addr learned my addr 172.21.15.133:0/963703727 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9be59c640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9c015ba20 0x7fb9c0151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9bdd9b640 1 -- 172.21.15.133:0/963703727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9c015ff00 msgr2=0x7fb9c0153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9bed9d640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9c015ff00 0x7fb9c0153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9bdd9b640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9c015ff00 0x7fb9c0153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9bdd9b640 1 -- 172.21.15.133:0/963703727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9c015ba20 msgr2=0x7fb9c0151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9bdd9b640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9c015ba20 0x7fb9c0151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9bdd9b640 1 -- 172.21.15.133:0/963703727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c01569f0 con 0x7fb9c015faa0 2024-04-25T10:02:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9be59c640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9c015ba20 0x7fb9c0151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9bed9d640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9c015ff00 0x7fb9c0153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9bdd9b640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9c015faa0 0x7fb9c0150a70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800c750 tx=0x7fb9b800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9b6ffd640 1 -- 172.21.15.133:0/963703727 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8005ae0 con 0x7fb9c015faa0 2024-04-25T10:02:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9b6ffd640 1 -- 172.21.15.133:0/963703727 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b8005c80 con 0x7fb9c015faa0 2024-04-25T10:02:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9c4be4640 1 -- 172.21.15.133:0/963703727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c0156cd0 con 0x7fb9c015faa0 2024-04-25T10:02:50.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9b6ffd640 1 -- 172.21.15.133:0/963703727 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b80072b0 con 0x7fb9c015faa0 2024-04-25T10:02:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9c4be4640 1 -- 172.21.15.133:0/963703727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c0151bb0 con 0x7fb9c015faa0 2024-04-25T10:02:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9c4be4640 1 -- 172.21.15.133:0/963703727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c0067c70 con 0x7fb9c015faa0 2024-04-25T10:02:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9b6ffd640 1 -- 172.21.15.133:0/963703727 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9b8007450 con 0x7fb9c015faa0 2024-04-25T10:02:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9b6ffd640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9a0066f60 0x7fb9a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9b6ffd640 1 -- 172.21.15.133:0/963703727 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb9b8085270 con 0x7fb9c015faa0 2024-04-25T10:02:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.016+0000 7fb9be59c640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9a0066f60 0x7fb9a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.020+0000 7fb9be59c640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9a0066f60 0x7fb9a0069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0157e30 tx=0x7fb9b000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.020+0000 7fb9b6ffd640 1 -- 172.21.15.133:0/963703727 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9c015be00 con 0x7fb9c015faa0 2024-04-25T10:02:50.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 -- 172.21.15.133:0/963703727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb9c01576b0 con 0x7fb9c015faa0 2024-04-25T10:02:50.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9b6ffd640 1 -- 172.21.15.133:0/963703727 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb9b8010300 con 0x7fb9c015faa0 2024-04-25T10:02:50.190 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:02:50.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 -- 172.21.15.133:0/963703727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9a0066f60 msgr2=0x7fb9a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9a0066f60 0x7fb9a0069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0157e30 tx=0x7fb9b000b040 comp rx=0 tx=0).stop 2024-04-25T10:02:50.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 -- 172.21.15.133:0/963703727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9c015faa0 msgr2=0x7fb9c0150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9c015faa0 0x7fb9c0150a70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800c750 tx=0x7fb9b800cc10 comp rx=0 tx=0).stop 2024-04-25T10:02:50.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 -- 172.21.15.133:0/963703727 shutdown_connections 2024-04-25T10:02:50.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9c015ff00 0x7fb9c0153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9a0066f60 0x7fb9a0069420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9c015faa0 0x7fb9c0150a70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 --2- 172.21.15.133:0/963703727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9c015ba20 0x7fb9c0151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 -- 172.21.15.133:0/963703727 >> 172.21.15.133:0/963703727 conn(0x7fb9c00cc750 msgr2=0x7fb9c006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 -- 172.21.15.133:0/963703727 shutdown_connections 2024-04-25T10:02:50.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.184+0000 7fb9c4be4640 1 -- 172.21.15.133:0/963703727 wait complete. 2024-04-25T10:02:50.209 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:02:50.209 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:50.214 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:50.313 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:50.315 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:50.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.356+0000 7f080710d640 1 Processor -- start 2024-04-25T10:02:50.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.356+0000 7f080710d640 1 -- start start 2024-04-25T10:02:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.356+0000 7f080710d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0800099de0 0x7f08000cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.356+0000 7f080710d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000b29c0 0x7f08000b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.356+0000 7f080710d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08000afc10 con 0x7f08000cd2b0 2024-04-25T10:02:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.356+0000 7f080710d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08000afef0 con 0x7f0800099de0 2024-04-25T10:02:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.356+0000 7f080710d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000cdcb0 con 0x7f08000b29c0 2024-04-25T10:02:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.356+0000 7f0804ea9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f08000cd2b0 0x7f08000b53c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47094/0 (socket says 172.21.15.133:47094) 2024-04-25T10:02:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.356+0000 7f0804ea9640 1 -- 172.21.15.133:0/1909486937 learned_addr learned my addr 172.21.15.133:0/1909486937 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07fffff640 1 --2- 172.21.15.133:0/1909486937 >> v2:172.21.15.162:3300/0 conn(0x7f0800099de0 0x7f08000cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f08056aa640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000b29c0 0x7f08000b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215657520 0 0) 0x7f08000afc10 con 0x7f08000cd2b0 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e8003530 con 0x7f08000cd2b0 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 941796697 0 0) 0x7f07e8003530 con 0x7f08000cd2b0 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000b29c0 msgr2=0x7f08000b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000b29c0 0x7f08000b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 >> v2:172.21.15.162:3300/0 conn(0x7f0800099de0 msgr2=0x7f08000cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 --2- 172.21.15.133:0/1909486937 >> v2:172.21.15.162:3300/0 conn(0x7f0800099de0 0x7f08000cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000cdf90 con 0x7f08000cd2b0 2024-04-25T10:02:50.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f07f4001680 con 0x7f08000cd2b0 2024-04-25T10:02:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 >> v1:172.21.15.133:6789/0 conn(0x7f08000cd2b0 legacy=0x7f08000b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000b29c0 0x7f07e80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 0x7f07e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e8004d50 0x7f07e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08000afef0 con 0x7f07e8003f40 2024-04-25T10:02:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08000cdcb0 con 0x7f07e8004d50 2024-04-25T10:02:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e800ba20 con 0x7f08000b29c0 2024-04-25T10:02:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07fffff640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 0x7f07e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f0804ea9640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000b29c0 0x7f07e80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f08056aa640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e8004d50 0x7f07e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07fffff640 1 -- 172.21.15.133:0/1909486937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000b29c0 msgr2=0x7f07e80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07fffff640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000b29c0 0x7f07e80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07fffff640 1 -- 172.21.15.133:0/1909486937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e8004d50 msgr2=0x7f07e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07fffff640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e8004d50 0x7f07e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07fffff640 1 -- 172.21.15.133:0/1909486937 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8003530 con 0x7f07e8003f40 2024-04-25T10:02:50.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07fffff640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 0x7f07e8004810 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009ae0 tx=0x7f07f001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee681b783052623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00096b0 con 0x7f07e8003f40 2024-04-25T10:02:50.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f00a9470 con 0x7f07e8003f40 2024-04-25T10:02:50.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f07ff7fe640 1 -- 172.21.15.133:0/1909486937 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00a9860 con 0x7f07e8003f40 2024-04-25T10:02:50.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f080710d640 1 -- 172.21.15.133:0/1909486937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 msgr2=0x7f07e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f080710d640 1 --2- 172.21.15.133:0/1909486937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 0x7f07e8004810 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009ae0 tx=0x7f07f001ca70 comp rx=0 tx=0).stop 2024-04-25T10:02:50.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f08056aa640 1 -- 172.21.15.133:0/1909486937 reap_dead start 2024-04-25T10:02:50.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f080710d640 1 -- 172.21.15.133:0/1909486937 shutdown_connections 2024-04-25T10:02:50.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f080710d640 1 -- 172.21.15.133:0/1909486937 >> 172.21.15.133:0/1909486937 conn(0x7f08000999d0 msgr2=0x7f08000b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f080710d640 1 -- 172.21.15.133:0/1909486937 shutdown_connections 2024-04-25T10:02:50.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.360+0000 7f080710d640 1 -- 172.21.15.133:0/1909486937 wait complete. 2024-04-25T10:02:50.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f080710d640 1 Processor -- start 2024-04-25T10:02:50.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f080710d640 1 -- start start 2024-04-25T10:02:50.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f080710d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 0x7f080006cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f080710d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000addf0 0x7f08000ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f080710d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0800157d50 0x7f08001581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f080710d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08000c58b0 con 0x7f07e8003f40 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f080710d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08000c5730 con 0x7f0800157d50 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f080710d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000c5a30 con 0x7f08000addf0 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f0804ea9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 0x7f080006cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f0804ea9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 0x7f080006cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46964/0 (socket says 172.21.15.133:46964) 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f0804ea9640 1 -- 172.21.15.133:0/3657248172 learned_addr learned my addr 172.21.15.133:0/3657248172 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f07fffff640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000addf0 0x7f08000ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f08056aa640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0800157d50 0x7f08001581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f0804ea9640 1 -- 172.21.15.133:0/3657248172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000addf0 msgr2=0x7f08000ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f0804ea9640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000addf0 0x7f08000ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f0804ea9640 1 -- 172.21.15.133:0/3657248172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0800157d50 msgr2=0x7f08001581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f0804ea9640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0800157d50 0x7f08001581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f0804ea9640 1 -- 172.21.15.133:0/3657248172 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080006d490 con 0x7f07e8003f40 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f0804ea9640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 0x7f080006cf50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f07f4000fc0 tx=0x7f07f40077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f07fd7fa640 1 -- 172.21.15.133:0/3657248172 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f400f930 con 0x7f07e8003f40 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f07fd7fa640 1 -- 172.21.15.133:0/3657248172 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f4018070 con 0x7f07e8003f40 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f07fd7fa640 1 -- 172.21.15.133:0/3657248172 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f400fd50 con 0x7f07e8003f40 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f080710d640 1 -- 172.21.15.133:0/3657248172 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0800158ac0 con 0x7f07e8003f40 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f080710d640 1 -- 172.21.15.133:0/3657248172 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0800159080 con 0x7f07e8003f40 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.364+0000 7f080710d640 1 -- 172.21.15.133:0/3657248172 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0800068e80 con 0x7f07e8003f40 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.368+0000 7f07fd7fa640 1 -- 172.21.15.133:0/3657248172 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07f401d460 con 0x7f07e8003f40 2024-04-25T10:02:50.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.368+0000 7f07fd7fa640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07ec066f90 0x7f07ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.368+0000 7f07fd7fa640 1 -- 172.21.15.133:0/3657248172 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f07f4085400 con 0x7f07e8003f40 2024-04-25T10:02:50.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.368+0000 7f07fffff640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07ec066f90 0x7f07ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.368+0000 7f07fffff640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07ec066f90 0x7f07ec069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07f00ae000 tx=0x7f07f00b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.376+0000 7f07fd7fa640 1 -- 172.21.15.133:0/3657248172 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07f4056010 con 0x7f07e8003f40 2024-04-25T10:02:50.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.532+0000 7f080710d640 1 -- 172.21.15.133:0/3657248172 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08000b5460 con 0x7f07e8003f40 2024-04-25T10:02:50.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.536+0000 7f07fd7fa640 1 -- 172.21.15.133:0/3657248172 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f07f4055910 con 0x7f07e8003f40 2024-04-25T10:02:50.544 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:50.544 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 -- 172.21.15.133:0/3657248172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07ec066f90 msgr2=0x7f07ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07ec066f90 0x7f07ec069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f07f00ae000 tx=0x7f07f00b3040 comp rx=0 tx=0).stop 2024-04-25T10:02:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 -- 172.21.15.133:0/3657248172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 msgr2=0x7f080006cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 0x7f080006cf50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f07f4000fc0 tx=0x7f07f40077a0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 -- 172.21.15.133:0/3657248172 shutdown_connections 2024-04-25T10:02:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0800157d50 0x7f08001581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08000addf0 0x7f08000ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07ec066f90 0x7f07ec069450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 --2- 172.21.15.133:0/3657248172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e8003f40 0x7f080006cf50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 -- 172.21.15.133:0/3657248172 >> 172.21.15.133:0/3657248172 conn(0x7f08000999d0 msgr2=0x7f080015d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 -- 172.21.15.133:0/3657248172 shutdown_connections 2024-04-25T10:02:50.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.540+0000 7f080710d640 1 -- 172.21.15.133:0/3657248172 wait complete. 2024-04-25T10:02:50.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:50.562 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:50.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.716+0000 7efcb2336640 1 Processor -- start 2024-04-25T10:02:50.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.716+0000 7efcb2336640 1 -- start start 2024-04-25T10:02:50.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcb2336640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcac06e8b0 0x7efcac15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcb2336640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efcac0aebc0 0x7efcac0af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcb2336640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac1471f0 con 0x7efcac0b1e50 2024-04-25T10:02:50.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcb2336640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcac147490 con 0x7efcac0aebc0 2024-04-25T10:02:50.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcb2336640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0b19c0 con 0x7efcac06e8b0 2024-04-25T10:02:50.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcb08d3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efcac0aebc0 0x7efcac0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcab7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7efcac0b1e50 0x7efcac146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47106/0 (socket says 172.21.15.133:47106) 2024-04-25T10:02:50.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcab7fe640 1 -- 172.21.15.133:0/3003717328 learned_addr learned my addr 172.21.15.133:0/3003717328 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:50.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcabfff640 1 --2- 172.21.15.133:0/3003717328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcac06e8b0 0x7efcac15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcb08d3640 1 -- 172.21.15.133:0/3003717328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcac06e8b0 msgr2=0x7efcac15eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcb08d3640 1 --2- 172.21.15.133:0/3003717328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcac06e8b0 0x7efcac15eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcb08d3640 1 -- 172.21.15.133:0/3003717328 >> v1:172.21.15.133:6789/0 conn(0x7efcac0b1e50 legacy=0x7efcac146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcaaffd640 1 -- 172.21.15.133:0/3003717328 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622642476 0 0) 0x7efcac1471f0 con 0x7efcac0b1e50 2024-04-25T10:02:50.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcb08d3640 1 -- 172.21.15.133:0/3003717328 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac0b1090 con 0x7efcac0aebc0 2024-04-25T10:02:50.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcabfff640 1 --2- 172.21.15.133:0/3003717328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcac06e8b0 0x7efcac15eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:50.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcb08d3640 1 --2- 172.21.15.133:0/3003717328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcac0aebc0 0x7efcac0af050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efca4009dc0 tx=0x7efca401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e78a0f1a93acd46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcaaffd640 1 -- 172.21.15.133:0/3003717328 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40b2070 con 0x7efcac0aebc0 2024-04-25T10:02:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcaaffd640 1 -- 172.21.15.133:0/3003717328 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca40a5780 con 0x7efcac0aebc0 2024-04-25T10:02:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.720+0000 7efcaaffd640 1 -- 172.21.15.133:0/3003717328 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40a5b50 con 0x7efcac0aebc0 2024-04-25T10:02:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- 172.21.15.133:0/3003717328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcac0aebc0 msgr2=0x7efcac0af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 --2- 172.21.15.133:0/3003717328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcac0aebc0 0x7efcac0af050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efca4009dc0 tx=0x7efca401c8c0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- 172.21.15.133:0/3003717328 shutdown_connections 2024-04-25T10:02:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 --2- 172.21.15.133:0/3003717328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcac0aebc0 0x7efcac0af050 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 --2- 172.21.15.133:0/3003717328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcac06e8b0 0x7efcac15eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- 172.21.15.133:0/3003717328 >> 172.21.15.133:0/3003717328 conn(0x7efcac066f70 msgr2=0x7efcac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- 172.21.15.133:0/3003717328 shutdown_connections 2024-04-25T10:02:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- 172.21.15.133:0/3003717328 wait complete. 2024-04-25T10:02:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 Processor -- start 2024-04-25T10:02:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- start start 2024-04-25T10:02:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcac06e8b0 0x7efcac13cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcac0aebc0 0x7efcac144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcac0b1e50 0x7efcac15e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac06dc30 con 0x7efcac0b1e50 2024-04-25T10:02:50.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac06dab0 con 0x7efcac06e8b0 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac06ddb0 con 0x7efcac0aebc0 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb08d3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcac0b1e50 0x7efcac15e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb08d3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcac0b1e50 0x7efcac15e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46968/0 (socket says 172.21.15.133:46968) 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb08d3640 1 -- 172.21.15.133:0/251270728 learned_addr learned my addr 172.21.15.133:0/251270728 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb08d3640 1 -- 172.21.15.133:0/251270728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcac0aebc0 msgr2=0x7efcac144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb08d3640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcac0aebc0 0x7efcac144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb08d3640 1 -- 172.21.15.133:0/251270728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcac06e8b0 msgr2=0x7efcac13cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb08d3640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcac06e8b0 0x7efcac13cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb08d3640 1 -- 172.21.15.133:0/251270728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac0b4150 con 0x7efcac0b1e50 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb08d3640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcac0b1e50 0x7efcac15e520 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efca40b0040 tx=0x7efca4019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efca8ff9640 1 -- 172.21.15.133:0/251270728 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40b2050 con 0x7efcac0b1e50 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efca8ff9640 1 -- 172.21.15.133:0/251270728 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca40b3040 con 0x7efcac0b1e50 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efca8ff9640 1 -- 172.21.15.133:0/251270728 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40ad650 con 0x7efcac0b1e50 2024-04-25T10:02:50.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- 172.21.15.133:0/251270728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcac0b4430 con 0x7efcac0b1e50 2024-04-25T10:02:50.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- 172.21.15.133:0/251270728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac15cc40 con 0x7efcac0b1e50 2024-04-25T10:02:50.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.724+0000 7efcb2336640 1 -- 172.21.15.133:0/251270728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcac143c20 con 0x7efcac0b1e50 2024-04-25T10:02:50.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.728+0000 7efca8ff9640 1 -- 172.21.15.133:0/251270728 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efca40c0020 con 0x7efcac0b1e50 2024-04-25T10:02:50.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.728+0000 7efca8ff9640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc98066f60 0x7efc98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.728+0000 7efca8ff9640 1 -- 172.21.15.133:0/251270728 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7efca411fd70 con 0x7efcac0b1e50 2024-04-25T10:02:50.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.728+0000 7efcabfff640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc98066f60 0x7efc98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.728+0000 7efcabfff640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc98066f60 0x7efc98069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc9c006fd0 tx=0x7efc9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.732+0000 7efca8ff9640 1 -- 172.21.15.133:0/251270728 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efca4016540 con 0x7efcac0b1e50 2024-04-25T10:02:50.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.880+0000 7efcb2336640 1 -- 172.21.15.133:0/251270728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcac068e80 con 0x7efcac0b1e50 2024-04-25T10:02:50.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.880+0000 7efca8ff9640 1 -- 172.21.15.133:0/251270728 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7efca40f0a10 con 0x7efcac0b1e50 2024-04-25T10:02:50.888 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 -- 172.21.15.133:0/251270728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc98066f60 msgr2=0x7efc98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc98066f60 0x7efc98069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc9c006fd0 tx=0x7efc9c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 -- 172.21.15.133:0/251270728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcac0b1e50 msgr2=0x7efcac15e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcac0b1e50 0x7efcac15e520 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efca40b0040 tx=0x7efca4019660 comp rx=0 tx=0).stop 2024-04-25T10:02:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 -- 172.21.15.133:0/251270728 shutdown_connections 2024-04-25T10:02:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc98066f60 0x7efc98069420 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcac0b1e50 0x7efcac15e520 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efcac0aebc0 0x7efcac144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 --2- 172.21.15.133:0/251270728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcac06e8b0 0x7efcac13cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 -- 172.21.15.133:0/251270728 >> 172.21.15.133:0/251270728 conn(0x7efcac066f70 msgr2=0x7efcac0adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 -- 172.21.15.133:0/251270728 shutdown_connections 2024-04-25T10:02:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:50.884+0000 7efcb2336640 1 -- 172.21.15.133:0/251270728 wait complete. 2024-04-25T10:02:50.905 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:51.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.016+0000 7f8cee1e1640 1 Processor -- start 2024-04-25T10:02:51.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.016+0000 7f8cee1e1640 1 -- start start 2024-04-25T10:02:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8ce8142920 0x7f8ce8143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ce8140a40 0x7f8ce80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce80cf000 con 0x7f8ce815d9f0 2024-04-25T10:02:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce8140ea0 con 0x7f8ce8142920 2024-04-25T10:02:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce80cf280 con 0x7f8ce8140a40 2024-04-25T10:02:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8ce77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8ce8142920 0x7f8ce8143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8ce77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8ce8142920 0x7f8ce8143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46978/0 (socket says 172.21.15.133:46978) 2024-04-25T10:02:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8ce77fe640 1 -- 172.21.15.133:0/1131170355 learned_addr learned my addr 172.21.15.133:0/1131170355 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cdffff640 1 --2- 172.21.15.133:0/1131170355 >> v2:172.21.15.162:3300/0 conn(0x7f8ce8140a40 0x7f8ce80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8ce77fe640 1 -- 172.21.15.133:0/1131170355 >> v2:172.21.15.162:3300/0 conn(0x7f8ce8140a40 msgr2=0x7f8ce80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8ce77fe640 1 --2- 172.21.15.133:0/1131170355 >> v2:172.21.15.162:3300/0 conn(0x7f8ce8140a40 0x7f8ce80ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8ce77fe640 1 -- 172.21.15.133:0/1131170355 >> v1:172.21.15.162:6789/0 conn(0x7f8ce815d9f0 legacy=0x7f8ce806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8ce77fe640 1 -- 172.21.15.133:0/1131170355 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce806d440 con 0x7f8ce8142920 2024-04-25T10:02:51.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8ce77fe640 1 --2- 172.21.15.133:0/1131170355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ce8142920 0x7f8ce8143560 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8ce806f440 tx=0x7f8cd801c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab20740812c45f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8ce6ffd640 1 -- 172.21.15.133:0/1131170355 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80b0070 con 0x7f8ce8142920 2024-04-25T10:02:51.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8ce6ffd640 1 -- 172.21.15.133:0/1131170355 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd80ab600 con 0x7f8ce8142920 2024-04-25T10:02:51.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8ce6ffd640 1 -- 172.21.15.133:0/1131170355 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80ab9d0 con 0x7f8ce8142920 2024-04-25T10:02:51.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 -- 172.21.15.133:0/1131170355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ce8142920 msgr2=0x7f8ce8143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 --2- 172.21.15.133:0/1131170355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ce8142920 0x7f8ce8143560 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8ce806f440 tx=0x7f8cd801c270 comp rx=0 tx=0).stop 2024-04-25T10:02:51.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 -- 172.21.15.133:0/1131170355 shutdown_connections 2024-04-25T10:02:51.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 --2- 172.21.15.133:0/1131170355 >> v2:172.21.15.162:3300/0 conn(0x7f8ce8140a40 0x7f8ce80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 --2- 172.21.15.133:0/1131170355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ce8142920 0x7f8ce8143560 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 -- 172.21.15.133:0/1131170355 >> 172.21.15.133:0/1131170355 conn(0x7f8ce8066f70 msgr2=0x7f8ce806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 -- 172.21.15.133:0/1131170355 shutdown_connections 2024-04-25T10:02:51.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.020+0000 7f8cee1e1640 1 -- 172.21.15.133:0/1131170355 wait complete. 2024-04-25T10:02:51.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cee1e1640 1 Processor -- start 2024-04-25T10:02:51.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cee1e1640 1 -- start start 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cee1e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ce8140a40 0x7f8ce80b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cee1e1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ce8142920 0x7f8ce813acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cee1e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ce815d9f0 0x7f8ce813d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cee1e1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce806ebc0 con 0x7f8ce8142920 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cee1e1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce806ea40 con 0x7f8ce815d9f0 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cee1e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce8142d00 con 0x7f8ce8140a40 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cdffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ce8142920 0x7f8ce813acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cdffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ce8142920 0x7f8ce813acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46982/0 (socket says 172.21.15.133:46982) 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cdffff640 1 -- 172.21.15.133:0/2224867090 learned_addr learned my addr 172.21.15.133:0/2224867090 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8ce77fe640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ce8140a40 0x7f8ce80b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8ce7fff640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ce815d9f0 0x7f8ce813d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cdffff640 1 -- 172.21.15.133:0/2224867090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ce8140a40 msgr2=0x7f8ce80b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cdffff640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ce8140a40 0x7f8ce80b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cdffff640 1 -- 172.21.15.133:0/2224867090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ce815d9f0 msgr2=0x7f8ce813d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cdffff640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ce815d9f0 0x7f8ce813d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cdffff640 1 -- 172.21.15.133:0/2224867090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce80afe10 con 0x7f8ce8142920 2024-04-25T10:02:51.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cdffff640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ce8142920 0x7f8ce813acf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000c7f0 tx=0x7f8cd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8ce4ff9640 1 -- 172.21.15.133:0/2224867090 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0015070 con 0x7f8ce8142920 2024-04-25T10:02:51.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8ce4ff9640 1 -- 172.21.15.133:0/2224867090 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd0004240 con 0x7f8ce8142920 2024-04-25T10:02:51.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cee1e1640 1 -- 172.21.15.133:0/2224867090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce80b00f0 con 0x7f8ce8142920 2024-04-25T10:02:51.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8ce4ff9640 1 -- 172.21.15.133:0/2224867090 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0003620 con 0x7f8ce8142920 2024-04-25T10:02:51.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cee1e1640 1 -- 172.21.15.133:0/2224867090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce813dc40 con 0x7f8ce8142920 2024-04-25T10:02:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.024+0000 7f8cee1e1640 1 -- 172.21.15.133:0/2224867090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce8068e80 con 0x7f8ce8142920 2024-04-25T10:02:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.028+0000 7f8ce4ff9640 1 -- 172.21.15.133:0/2224867090 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8cd00037c0 con 0x7f8ce8142920 2024-04-25T10:02:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.028+0000 7f8ce4ff9640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8cc8066f90 0x7f8cc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.028+0000 7f8ce77fe640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8cc8066f90 0x7f8cc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.028+0000 7f8ce4ff9640 1 -- 172.21.15.133:0/2224867090 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8cd00852a0 con 0x7f8ce8142920 2024-04-25T10:02:51.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.028+0000 7f8ce77fe640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8cc8066f90 0x7f8cc8069450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80ad000 tx=0x7f8cd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.032+0000 7f8ce4ff9640 1 -- 172.21.15.133:0/2224867090 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8cd0055eb0 con 0x7f8ce8142920 2024-04-25T10:02:51.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.176+0000 7f8cee1e1640 1 -- 172.21.15.133:0/2224867090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ce8155fe0 con 0x7f8ce8142920 2024-04-25T10:02:51.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.176+0000 7f8ce4ff9640 1 -- 172.21.15.133:0/2224867090 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8cd00557b0 con 0x7f8ce8142920 2024-04-25T10:02:51.183 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:51.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.176+0000 7f8cee1e1640 1 -- 172.21.15.133:0/2224867090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8cc8066f90 msgr2=0x7f8cc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.176+0000 7f8cee1e1640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8cc8066f90 0x7f8cc8069450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80ad000 tx=0x7f8cd8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.180+0000 7f8cee1e1640 1 -- 172.21.15.133:0/2224867090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ce8142920 msgr2=0x7f8ce813acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.180+0000 7f8cee1e1640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ce8142920 0x7f8ce813acf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000c7f0 tx=0x7f8cd000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.180+0000 7f8cee1e1640 1 -- 172.21.15.133:0/2224867090 shutdown_connections 2024-04-25T10:02:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.180+0000 7f8cee1e1640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ce815d9f0 0x7f8ce813d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.180+0000 7f8cee1e1640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8cc8066f90 0x7f8cc8069450 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.180+0000 7f8cee1e1640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ce8142920 0x7f8ce813acf0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.180+0000 7f8cee1e1640 1 --2- 172.21.15.133:0/2224867090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ce8140a40 0x7f8ce80b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.180+0000 7f8cee1e1640 1 -- 172.21.15.133:0/2224867090 >> 172.21.15.133:0/2224867090 conn(0x7f8ce8066f70 msgr2=0x7f8ce80ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.180+0000 7f8cee1e1640 1 -- 172.21.15.133:0/2224867090 shutdown_connections 2024-04-25T10:02:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.180+0000 7f8cee1e1640 1 -- 172.21.15.133:0/2224867090 wait complete. 2024-04-25T10:02:51.200 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:51.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.304+0000 7f59b2393640 1 Processor -- start 2024-04-25T10:02:51.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.304+0000 7f59b2393640 1 -- start start 2024-04-25T10:02:51.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59b2393640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f59ac143560 0x7f59ac140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59b2393640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f59ac15d9f0 0x7f59ac15de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59b2393640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ac0cf280 con 0x7f59ac142920 2024-04-25T10:02:51.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59b2393640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ac15e370 con 0x7f59ac143560 2024-04-25T10:02:51.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59b2393640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ac06d440 con 0x7f59ac15d9f0 2024-04-25T10:02:51.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59ab7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f59ac143560 0x7f59ac140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59ab7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f59ac143560 0x7f59ac140a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46994/0 (socket says 172.21.15.133:46994) 2024-04-25T10:02:51.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59ab7fe640 1 -- 172.21.15.133:0/3066820701 learned_addr learned my addr 172.21.15.133:0/3066820701 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:51.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59b0930640 1 --2- 172.21.15.133:0/3066820701 >> v2:172.21.15.162:3300/0 conn(0x7f59ac15d9f0 0x7f59ac15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59ab7fe640 1 -- 172.21.15.133:0/3066820701 >> v2:172.21.15.162:3300/0 conn(0x7f59ac15d9f0 msgr2=0x7f59ac15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59ab7fe640 1 --2- 172.21.15.133:0/3066820701 >> v2:172.21.15.162:3300/0 conn(0x7f59ac15d9f0 0x7f59ac15de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59ab7fe640 1 -- 172.21.15.133:0/3066820701 >> v1:172.21.15.162:6789/0 conn(0x7f59ac142920 legacy=0x7f59ac0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59ab7fe640 1 -- 172.21.15.133:0/3066820701 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac06d720 con 0x7f59ac143560 2024-04-25T10:02:51.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59ab7fe640 1 --2- 172.21.15.133:0/3066820701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac143560 0x7f59ac140a40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f59ac06f5d0 tx=0x7f59a001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98e43414f8d757e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59aaffd640 1 -- 172.21.15.133:0/3066820701 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a00b2070 con 0x7f59ac143560 2024-04-25T10:02:51.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59aaffd640 1 -- 172.21.15.133:0/3066820701 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a00a9070 con 0x7f59ac143560 2024-04-25T10:02:51.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.308+0000 7f59aaffd640 1 -- 172.21.15.133:0/3066820701 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a00ad880 con 0x7f59ac143560 2024-04-25T10:02:51.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 -- 172.21.15.133:0/3066820701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac143560 msgr2=0x7f59ac140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 --2- 172.21.15.133:0/3066820701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac143560 0x7f59ac140a40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f59ac06f5d0 tx=0x7f59a001c450 comp rx=0 tx=0).stop 2024-04-25T10:02:51.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 -- 172.21.15.133:0/3066820701 shutdown_connections 2024-04-25T10:02:51.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 --2- 172.21.15.133:0/3066820701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac143560 0x7f59ac140a40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 --2- 172.21.15.133:0/3066820701 >> v2:172.21.15.162:3300/0 conn(0x7f59ac15d9f0 0x7f59ac15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 -- 172.21.15.133:0/3066820701 >> 172.21.15.133:0/3066820701 conn(0x7f59ac066f70 msgr2=0x7f59ac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 -- 172.21.15.133:0/3066820701 shutdown_connections 2024-04-25T10:02:51.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 -- 172.21.15.133:0/3066820701 wait complete. 2024-04-25T10:02:51.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 Processor -- start 2024-04-25T10:02:51.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 -- start start 2024-04-25T10:02:51.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac142920 0x7f59ac0b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59ac143560 0x7f59ac13f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59ac15d9f0 0x7f59ac156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac06eb40 con 0x7f59ac142920 2024-04-25T10:02:51.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac06e9c0 con 0x7f59ac143560 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac06b3a0 con 0x7f59ac15d9f0 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac142920 0x7f59ac0b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac142920 0x7f59ac0b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47008/0 (socket says 172.21.15.133:47008) 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59abfff640 1 -- 172.21.15.133:0/3348835033 learned_addr learned my addr 172.21.15.133:0/3348835033 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b0930640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59ac15d9f0 0x7f59ac156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59ab7fe640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59ac143560 0x7f59ac13f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59abfff640 1 -- 172.21.15.133:0/3348835033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59ac15d9f0 msgr2=0x7f59ac156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59abfff640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59ac15d9f0 0x7f59ac156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59abfff640 1 -- 172.21.15.133:0/3348835033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59ac143560 msgr2=0x7f59ac13f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59abfff640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59ac143560 0x7f59ac13f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59abfff640 1 -- 172.21.15.133:0/3348835033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac0b2da0 con 0x7f59ac142920 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b0930640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59ac15d9f0 0x7f59ac156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59ab7fe640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59ac143560 0x7f59ac13f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59abfff640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac142920 0x7f59ac0b2550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c450 tx=0x7f599c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59a8ff9640 1 -- 172.21.15.133:0/3348835033 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c012830 con 0x7f59ac142920 2024-04-25T10:02:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59a8ff9640 1 -- 172.21.15.133:0/3348835033 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c017070 con 0x7f59ac142920 2024-04-25T10:02:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 -- 172.21.15.133:0/3348835033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ac0b3080 con 0x7f59ac142920 2024-04-25T10:02:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59a8ff9640 1 -- 172.21.15.133:0/3348835033 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c012c30 con 0x7f59ac142920 2024-04-25T10:02:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.312+0000 7f59b2393640 1 -- 172.21.15.133:0/3348835033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac15b4f0 con 0x7f59ac142920 2024-04-25T10:02:51.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.316+0000 7f59b2393640 1 -- 172.21.15.133:0/3348835033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ac068e80 con 0x7f59ac142920 2024-04-25T10:02:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.316+0000 7f59a8ff9640 1 -- 172.21.15.133:0/3348835033 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f599c01c460 con 0x7f59ac142920 2024-04-25T10:02:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.316+0000 7f59a8ff9640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5998066f60 0x7f5998069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.316+0000 7f59ab7fe640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5998066f60 0x7f5998069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.316+0000 7f59a8ff9640 1 -- 172.21.15.133:0/3348835033 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f599c0851e0 con 0x7f59ac142920 2024-04-25T10:02:51.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.316+0000 7f59ab7fe640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5998066f60 0x7f5998069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f59a00af000 tx=0x7f59a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.320+0000 7f59a8ff9640 1 -- 172.21.15.133:0/3348835033 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f599c055e80 con 0x7f59ac142920 2024-04-25T10:02:51.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.468+0000 7f59b2393640 1 -- 172.21.15.133:0/3348835033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59ac0afa70 con 0x7f59ac142920 2024-04-25T10:02:51.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.468+0000 7f59a8ff9640 1 -- 172.21.15.133:0/3348835033 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f599c055780 con 0x7f59ac142920 2024-04-25T10:02:51.475 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:51.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.468+0000 7f59b2393640 1 -- 172.21.15.133:0/3348835033 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5998066f60 msgr2=0x7f5998069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.468+0000 7f59b2393640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5998066f60 0x7f5998069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f59a00af000 tx=0x7f59a0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:51.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.468+0000 7f59b2393640 1 -- 172.21.15.133:0/3348835033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac142920 msgr2=0x7f59ac0b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.468+0000 7f59b2393640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac142920 0x7f59ac0b2550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c450 tx=0x7f599c00c910 comp rx=0 tx=0).stop 2024-04-25T10:02:51.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.472+0000 7f59b2393640 1 -- 172.21.15.133:0/3348835033 shutdown_connections 2024-04-25T10:02:51.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.472+0000 7f59b2393640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59ac15d9f0 0x7f59ac156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.472+0000 7f59b2393640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59ac143560 0x7f59ac13f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.472+0000 7f59b2393640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5998066f60 0x7f5998069420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.472+0000 7f59b2393640 1 --2- 172.21.15.133:0/3348835033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac142920 0x7f59ac0b2550 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.472+0000 7f59b2393640 1 -- 172.21.15.133:0/3348835033 >> 172.21.15.133:0/3348835033 conn(0x7f59ac066f70 msgr2=0x7f59ac0735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.472+0000 7f59b2393640 1 -- 172.21.15.133:0/3348835033 shutdown_connections 2024-04-25T10:02:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.472+0000 7f59b2393640 1 -- 172.21.15.133:0/3348835033 wait complete. 2024-04-25T10:02:51.492 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:51.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.596+0000 7f59d5a20640 1 Processor -- start 2024-04-25T10:02:51.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59d5a20640 1 -- start start 2024-04-25T10:02:51.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59d5a20640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00cd820 0x7f59d00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59d5a20640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f59d015ced0 0x7f59d015d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59d5a20640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d00ce2e0 con 0x7f59d0145170 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59d5a20640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d006db00 con 0x7f59d015ced0 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59d5a20640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d0145550 con 0x7f59d00cd820 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59ceffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f59d0145170 0x7f59d006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47108/0 (socket says 172.21.15.133:47108) 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59ceffd640 1 -- 172.21.15.133:0/1655041631 learned_addr learned my addr 172.21.15.133:0/1655041631 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856056245 0 0) 0x7f59d00ce2e0 con 0x7f59d0145170 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b8003530 con 0x7f59d0145170 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59ce7fc640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00cd820 0x7f59d00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cf7fe640 1 --2- 172.21.15.133:0/1655041631 >> v2:172.21.15.162:3300/0 conn(0x7f59d015ced0 0x7f59d015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 23829267 0 0) 0x7f59b8003530 con 0x7f59d0145170 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00cd820 msgr2=0x7f59d00cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00cd820 0x7f59d00cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 >> v2:172.21.15.162:3300/0 conn(0x7f59d015ced0 msgr2=0x7f59d015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 --2- 172.21.15.133:0/1655041631 >> v2:172.21.15.162:3300/0 conn(0x7f59d015ced0 0x7f59d015d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d006d4b0 con 0x7f59d0145170 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f59c0001680 con 0x7f59d0145170 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cf7fe640 1 --2- 172.21.15.133:0/1655041631 >> v2:172.21.15.162:3300/0 conn(0x7f59d015ced0 0x7f59d015d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 >> v1:172.21.15.133:6789/0 conn(0x7f59d0145170 legacy=0x7f59d006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b80039c0 0x7f59b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b8004810 0x7f59b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00cd820 0x7f59b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d006db00 con 0x7f59b8004810 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d0145550 con 0x7f59b80039c0 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b800ba20 con 0x7f59d00cd820 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59ce7fc640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b8004810 0x7f59b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59cf7fe640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00cd820 0x7f59b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59ceffd640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b80039c0 0x7f59b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59ce7fc640 1 -- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00cd820 msgr2=0x7f59b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59ce7fc640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00cd820 0x7f59b80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59ce7fc640 1 -- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b80039c0 msgr2=0x7f59b8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59ce7fc640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b80039c0 0x7f59b8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59ce7fc640 1 -- 172.21.15.133:0/1655041631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b8003530 con 0x7f59b8004810 2024-04-25T10:02:51.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.600+0000 7f59ceffd640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59b80039c0 0x7f59b8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:51.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ce7fc640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b8004810 0x7f59b8004c90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009ae0 tx=0x7f59c401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8409f8714505f6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40b2070 con 0x7f59b8004810 2024-04-25T10:02:51.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c40a55a0 con 0x7f59b8004810 2024-04-25T10:02:51.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59cdffb640 1 -- 172.21.15.133:0/1655041631 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40096b0 con 0x7f59b8004810 2024-04-25T10:02:51.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 -- 172.21.15.133:0/1655041631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b8004810 msgr2=0x7f59b8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 --2- 172.21.15.133:0/1655041631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b8004810 0x7f59b8004c90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009ae0 tx=0x7f59c401ca90 comp rx=0 tx=0).stop 2024-04-25T10:02:51.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59cf7fe640 1 -- 172.21.15.133:0/1655041631 reap_dead start 2024-04-25T10:02:51.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 -- 172.21.15.133:0/1655041631 shutdown_connections 2024-04-25T10:02:51.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 -- 172.21.15.133:0/1655041631 >> 172.21.15.133:0/1655041631 conn(0x7f59d0066f70 msgr2=0x7f59d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 -- 172.21.15.133:0/1655041631 shutdown_connections 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 -- 172.21.15.133:0/1655041631 wait complete. 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 Processor -- start 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 -- start start 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d013c770 0x7f59d00ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59d00ad830 0x7f59d00b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00b0c10 0x7f59d00af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d015cb70 con 0x7f59d00ad830 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d006e6a0 con 0x7f59d013c770 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d00cdca0 con 0x7f59d00b0c10 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ce7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59d00ad830 0x7f59d00b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ce7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59d00ad830 0x7f59d00b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47030/0 (socket says 172.21.15.133:47030) 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ce7fc640 1 -- 172.21.15.133:0/701663758 learned_addr learned my addr 172.21.15.133:0/701663758 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:51.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ceffd640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d013c770 0x7f59d00ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ce7fc640 1 -- 172.21.15.133:0/701663758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00b0c10 msgr2=0x7f59d00af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59cf7fe640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00b0c10 0x7f59d00af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ce7fc640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00b0c10 0x7f59d00af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ce7fc640 1 -- 172.21.15.133:0/701663758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d013c770 msgr2=0x7f59d00ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ce7fc640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d013c770 0x7f59d00ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ce7fc640 1 -- 172.21.15.133:0/701663758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d013ada0 con 0x7f59d00ad830 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ceffd640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d013c770 0x7f59d00ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59ce7fc640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59d00ad830 0x7f59d00b06d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009ae0 tx=0x7f59c4019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59b7fff640 1 -- 172.21.15.133:0/701663758 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40bc030 con 0x7f59d00ad830 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59b7fff640 1 -- 172.21.15.133:0/701663758 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c400f000 con 0x7f59d00ad830 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.604+0000 7f59d5a20640 1 -- 172.21.15.133:0/701663758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d00b13c0 con 0x7f59d00ad830 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.608+0000 7f59b7fff640 1 -- 172.21.15.133:0/701663758 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40b2040 con 0x7f59d00ad830 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.608+0000 7f59d5a20640 1 -- 172.21.15.133:0/701663758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d013b340 con 0x7f59d00ad830 2024-04-25T10:02:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.608+0000 7f59d5a20640 1 -- 172.21.15.133:0/701663758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d0068e80 con 0x7f59d00ad830 2024-04-25T10:02:51.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.608+0000 7f59b7fff640 1 -- 172.21.15.133:0/701663758 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59c40bf070 con 0x7f59d00ad830 2024-04-25T10:02:51.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.608+0000 7f59b7fff640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59bc066f90 0x7f59bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.608+0000 7f59b7fff640 1 -- 172.21.15.133:0/701663758 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f59c411f0b0 con 0x7f59d00ad830 2024-04-25T10:02:51.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.608+0000 7f59ceffd640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59bc066f90 0x7f59bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.608+0000 7f59ceffd640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59bc066f90 0x7f59bc069450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f59c00017f0 tx=0x7f59c0001440 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.612+0000 7f59b7fff640 1 -- 172.21.15.133:0/701663758 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59c40c7020 con 0x7f59d00ad830 2024-04-25T10:02:51.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.756+0000 7f59d5a20640 1 -- 172.21.15.133:0/701663758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f59d00ae730 con 0x7f59d00ad830 2024-04-25T10:02:51.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.756+0000 7f59b7fff640 1 -- 172.21.15.133:0/701663758 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f59c40167a0 con 0x7f59d00ad830 2024-04-25T10:02:51.765 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 -- 172.21.15.133:0/701663758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59bc066f90 msgr2=0x7f59bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59bc066f90 0x7f59bc069450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f59c00017f0 tx=0x7f59c0001440 comp rx=0 tx=0).stop 2024-04-25T10:02:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 -- 172.21.15.133:0/701663758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59d00ad830 msgr2=0x7f59d00b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59d00ad830 0x7f59d00b06d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009ae0 tx=0x7f59c4019830 comp rx=0 tx=0).stop 2024-04-25T10:02:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 -- 172.21.15.133:0/701663758 shutdown_connections 2024-04-25T10:02:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59d00b0c10 0x7f59d00af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59bc066f90 0x7f59bc069450 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59d00ad830 0x7f59d00b06d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 --2- 172.21.15.133:0/701663758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59d013c770 0x7f59d00ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 -- 172.21.15.133:0/701663758 >> 172.21.15.133:0/701663758 conn(0x7f59d0066f70 msgr2=0x7f59d00b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 -- 172.21.15.133:0/701663758 shutdown_connections 2024-04-25T10:02:51.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.760+0000 7f59d5a20640 1 -- 172.21.15.133:0/701663758 wait complete. 2024-04-25T10:02:51.783 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:51.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb03d806640 1 Processor -- start 2024-04-25T10:02:51.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb03d806640 1 -- start start 2024-04-25T10:02:51.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb03d806640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb03815daf0 0x7fb0380cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb03d806640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03815ded0 con 0x7fb0380cec10 2024-04-25T10:02:51.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb03d806640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0380d10d0 con 0x7fb0380b91c0 2024-04-25T10:02:51.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb03d806640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0380d1da0 con 0x7fb03815daf0 2024-04-25T10:02:51.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb036ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb03815daf0 0x7fb0380cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb0377fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb0380cec10 0x7fb0380d2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47116/0 (socket says 172.21.15.133:47116) 2024-04-25T10:02:51.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb0377fe640 1 -- 172.21.15.133:0/362172250 learned_addr learned my addr 172.21.15.133:0/362172250 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:51.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb036ffd640 1 -- 172.21.15.133:0/362172250 >> v1:172.21.15.162:6789/0 conn(0x7fb0380b91c0 legacy=0x7fb0380ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb036ffd640 1 -- 172.21.15.133:0/362172250 >> v1:172.21.15.133:6789/0 conn(0x7fb0380cec10 legacy=0x7fb0380d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb036ffd640 1 -- 172.21.15.133:0/362172250 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0380d0830 con 0x7fb03815daf0 2024-04-25T10:02:51.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb036ffd640 1 --2- 172.21.15.133:0/362172250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb03815daf0 0x7fb0380cffd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb0280097c0 tx=0x7fb02801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3696c500f09f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.896+0000 7fb035ffb640 1 -- 172.21.15.133:0/362172250 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0280b0070 con 0x7fb03815daf0 2024-04-25T10:02:51.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb035ffb640 1 -- 172.21.15.133:0/362172250 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0280ab600 con 0x7fb03815daf0 2024-04-25T10:02:51.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb035ffb640 1 -- 172.21.15.133:0/362172250 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0280ab9d0 con 0x7fb03815daf0 2024-04-25T10:02:51.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 -- 172.21.15.133:0/362172250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb03815daf0 msgr2=0x7fb0380cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 --2- 172.21.15.133:0/362172250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb03815daf0 0x7fb0380cffd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb0280097c0 tx=0x7fb02801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 -- 172.21.15.133:0/362172250 shutdown_connections 2024-04-25T10:02:51.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 --2- 172.21.15.133:0/362172250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb03815daf0 0x7fb0380cffd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 -- 172.21.15.133:0/362172250 >> 172.21.15.133:0/362172250 conn(0x7fb03806bf80 msgr2=0x7fb03806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 -- 172.21.15.133:0/362172250 shutdown_connections 2024-04-25T10:02:51.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 -- 172.21.15.133:0/362172250 wait complete. 2024-04-25T10:02:51.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 Processor -- start 2024-04-25T10:02:51.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 -- start start 2024-04-25T10:02:51.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0380b91c0 0x7fb0380c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0380cec10 0x7fb03815f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb03815daf0 0x7fb03815f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0380d3320 con 0x7fb0380cec10 2024-04-25T10:02:51.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0380d31a0 con 0x7fb03815daf0 2024-04-25T10:02:51.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0380d34a0 con 0x7fb0380b91c0 2024-04-25T10:02:51.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb0367fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0380cec10 0x7fb03815f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb0367fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0380cec10 0x7fb03815f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47038/0 (socket says 172.21.15.133:47038) 2024-04-25T10:02:51.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb0367fc640 1 -- 172.21.15.133:0/825780822 learned_addr learned my addr 172.21.15.133:0/825780822 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb036ffd640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0380b91c0 0x7fb0380c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb0377fe640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb03815daf0 0x7fb03815f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb0367fc640 1 -- 172.21.15.133:0/825780822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0380b91c0 msgr2=0x7fb0380c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb0367fc640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0380b91c0 0x7fb0380c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb0367fc640 1 -- 172.21.15.133:0/825780822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb03815daf0 msgr2=0x7fb03815f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb0367fc640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb03815daf0 0x7fb03815f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb0367fc640 1 -- 172.21.15.133:0/825780822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0380c9300 con 0x7fb0380cec10 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb0367fc640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0380cec10 0x7fb03815f0b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00b410 tx=0x7fb02c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb01ffff640 1 -- 172.21.15.133:0/825780822 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c015030 con 0x7fb0380cec10 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb01ffff640 1 -- 172.21.15.133:0/825780822 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c004020 con 0x7fb0380cec10 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 -- 172.21.15.133:0/825780822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03815b230 con 0x7fb0380cec10 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb01ffff640 1 -- 172.21.15.133:0/825780822 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c004410 con 0x7fb0380cec10 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.900+0000 7fb03d806640 1 -- 172.21.15.133:0/825780822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03815b7f0 con 0x7fb0380cec10 2024-04-25T10:02:51.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.904+0000 7fb03d806640 1 -- 172.21.15.133:0/825780822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb038068e80 con 0x7fb0380cec10 2024-04-25T10:02:51.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.904+0000 7fb01ffff640 1 -- 172.21.15.133:0/825780822 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb02c018070 con 0x7fb0380cec10 2024-04-25T10:02:51.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.904+0000 7fb01ffff640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb024066f60 0x7fb024069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.904+0000 7fb01ffff640 1 -- 172.21.15.133:0/825780822 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb02c084fd0 con 0x7fb0380cec10 2024-04-25T10:02:51.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.904+0000 7fb036ffd640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb024066f60 0x7fb024069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.904+0000 7fb036ffd640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb024066f60 0x7fb024069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb0280ad000 tx=0x7fb0280a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:51.908+0000 7fb01ffff640 1 -- 172.21.15.133:0/825780822 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0380c9300 con 0x7fb0380cec10 2024-04-25T10:02:52.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.068+0000 7fb03d806640 1 -- 172.21.15.133:0/825780822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb038161760 con 0x7fb0380cec10 2024-04-25T10:02:52.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.068+0000 7fb01ffff640 1 -- 172.21.15.133:0/825780822 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb02c055be0 con 0x7fb0380cec10 2024-04-25T10:02:52.077 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 -- 172.21.15.133:0/825780822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb024066f60 msgr2=0x7fb024069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb024066f60 0x7fb024069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb0280ad000 tx=0x7fb0280a6000 comp rx=0 tx=0).stop 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 -- 172.21.15.133:0/825780822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0380cec10 msgr2=0x7fb03815f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0380cec10 0x7fb03815f0b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00b410 tx=0x7fb02c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 -- 172.21.15.133:0/825780822 shutdown_connections 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb03815daf0 0x7fb03815f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb024066f60 0x7fb024069420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0380cec10 0x7fb03815f0b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 --2- 172.21.15.133:0/825780822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb0380b91c0 0x7fb0380c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 -- 172.21.15.133:0/825780822 >> 172.21.15.133:0/825780822 conn(0x7fb03806bf80 msgr2=0x7fb03814d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 -- 172.21.15.133:0/825780822 shutdown_connections 2024-04-25T10:02:52.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.072+0000 7fb03d806640 1 -- 172.21.15.133:0/825780822 wait complete. 2024-04-25T10:02:52.093 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:02:52.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:52.094 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:52.106 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:52.106 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:02:52.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:52.106 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:02:52.156 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:02:52.156 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:52.163 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:02:52.163 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:52.214 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:52.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.324+0000 7f9d0b573640 1 Processor -- start 2024-04-25T10:02:52.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.324+0000 7f9d0b573640 1 -- start start 2024-04-25T10:02:52.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0b573640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9d0415d9b0 0x7f9d040cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0b573640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d040cee80 0x7f9d0406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0b573640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d040ca9b0 con 0x7f9d0415dd90 2024-04-25T10:02:52.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0b573640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d040cbe30 con 0x7f9d0415d9b0 2024-04-25T10:02:52.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0b573640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040cac90 con 0x7f9d040cee80 2024-04-25T10:02:52.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0930f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9d0415d9b0 0x7f9d040cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d09b10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d040cee80 0x7f9d0406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d09b10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d040cee80 0x7f9d0406d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39666/0 (socket says 172.21.15.133:39666) 2024-04-25T10:02:52.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d08b0e640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9d0415dd90 0x7f9d040ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:44340/0 (socket says 172.21.15.133:44340) 2024-04-25T10:02:52.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d09b10640 1 -- 172.21.15.133:0/2307574073 learned_addr learned my addr 172.21.15.133:0/2307574073 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:52.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0930f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9d0415d9b0 0x7f9d040cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:34468/0 (socket says 172.21.15.133:34468) 2024-04-25T10:02:52.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0930f640 1 -- 172.21.15.133:0/2307574073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d040cee80 msgr2=0x7f9d0406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0930f640 1 --2- 172.21.15.133:0/2307574073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d040cee80 0x7f9d0406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0930f640 1 -- 172.21.15.133:0/2307574073 >> v1:172.21.15.162:6789/0 conn(0x7f9d0415dd90 legacy=0x7f9d040ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0930f640 1 -- 172.21.15.133:0/2307574073 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d040caf70 con 0x7f9d0415d9b0 2024-04-25T10:02:52.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.328+0000 7f9d0930f640 1 --2- 172.21.15.133:0/2307574073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d0415d9b0 0x7f9d040cb5e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9d040cbfb0 tx=0x7f9cfc01c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b01a0a563aee6c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9cfbfff640 1 -- 172.21.15.133:0/2307574073 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0b1070 con 0x7f9d0415d9b0 2024-04-25T10:02:52.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9cfbfff640 1 -- 172.21.15.133:0/2307574073 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc0ac630 con 0x7f9d0415d9b0 2024-04-25T10:02:52.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9cfbfff640 1 -- 172.21.15.133:0/2307574073 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0a8050 con 0x7f9d0415d9b0 2024-04-25T10:02:52.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 -- 172.21.15.133:0/2307574073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d0415d9b0 msgr2=0x7f9d040cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 --2- 172.21.15.133:0/2307574073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d0415d9b0 0x7f9d040cb5e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9d040cbfb0 tx=0x7f9cfc01c3b0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 -- 172.21.15.133:0/2307574073 shutdown_connections 2024-04-25T10:02:52.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 --2- 172.21.15.133:0/2307574073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d040cee80 0x7f9d0406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 --2- 172.21.15.133:0/2307574073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d0415d9b0 0x7f9d040cb5e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 -- 172.21.15.133:0/2307574073 >> 172.21.15.133:0/2307574073 conn(0x7f9d04067000 msgr2=0x7f9d0406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 -- 172.21.15.133:0/2307574073 shutdown_connections 2024-04-25T10:02:52.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 -- 172.21.15.133:0/2307574073 wait complete. 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 Processor -- start 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 -- start start 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d040cee80 0x7f9d0413db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d0415d9b0 0x7f9d040b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d0415dd90 0x7f9d0413b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0406e7d0 con 0x7f9d0415d9b0 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0406e650 con 0x7f9d0415dd90 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0406e950 con 0x7f9d040cee80 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d08b0e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d0415d9b0 0x7f9d040b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d08b0e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d0415d9b0 0x7f9d040b1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47042/0 (socket says 172.21.15.133:47042) 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d08b0e640 1 -- 172.21.15.133:0/3497891961 learned_addr learned my addr 172.21.15.133:0/3497891961 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d09b10640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d0415dd90 0x7f9d0413b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0930f640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d040cee80 0x7f9d0413db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d08b0e640 1 -- 172.21.15.133:0/3497891961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d040cee80 msgr2=0x7f9d0413db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d08b0e640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d040cee80 0x7f9d0413db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d08b0e640 1 -- 172.21.15.133:0/3497891961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d0415dd90 msgr2=0x7f9d0413b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d08b0e640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d0415dd90 0x7f9d0413b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d08b0e640 1 -- 172.21.15.133:0/3497891961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0413ba30 con 0x7f9d0415d9b0 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d08b0e640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d0415d9b0 0x7f9d040b1190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400c370 tx=0x7f9cf400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/3497891961 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40074e0 con 0x7f9d0415d9b0 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/3497891961 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf4007680 con 0x7f9d0415d9b0 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 -- 172.21.15.133:0/3497891961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0413bd10 con 0x7f9d0415d9b0 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/3497891961 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4004a30 con 0x7f9d0415d9b0 2024-04-25T10:02:52.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.332+0000 7f9d0b573640 1 -- 172.21.15.133:0/3497891961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0413c2d0 con 0x7f9d0415d9b0 2024-04-25T10:02:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.336+0000 7f9d0b573640 1 -- 172.21.15.133:0/3497891961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d04068ef0 con 0x7f9d0415d9b0 2024-04-25T10:02:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.336+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/3497891961 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9cf401a020 con 0x7f9d0415d9b0 2024-04-25T10:02:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.336+0000 7f9cf9ffb640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce8066f60 0x7f9ce8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.336+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/3497891961 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9cf4084ad0 con 0x7f9d0415d9b0 2024-04-25T10:02:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.336+0000 7f9d0930f640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce8066f60 0x7f9ce8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.336+0000 7f9d0930f640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce8066f60 0x7f9ce8069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0ae000 tx=0x7f9cfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.340+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/3497891961 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9cf4055770 con 0x7f9d0415d9b0 2024-04-25T10:02:52.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.500+0000 7f9d0b573640 1 -- 172.21.15.133:0/3497891961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d04153850 con 0x7f9d0415d9b0 2024-04-25T10:02:52.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9cf9ffb640 1 -- 172.21.15.133:0/3497891961 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f9cf4055070 con 0x7f9d0415d9b0 2024-04-25T10:02:52.510 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:52.510 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:52.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 -- 172.21.15.133:0/3497891961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce8066f60 msgr2=0x7f9ce8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce8066f60 0x7f9ce8069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0ae000 tx=0x7f9cfc008040 comp rx=0 tx=0).stop 2024-04-25T10:02:52.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 -- 172.21.15.133:0/3497891961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d0415d9b0 msgr2=0x7f9d040b1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d0415d9b0 0x7f9d040b1190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400c370 tx=0x7f9cf400c830 comp rx=0 tx=0).stop 2024-04-25T10:02:52.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 -- 172.21.15.133:0/3497891961 shutdown_connections 2024-04-25T10:02:52.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d0415dd90 0x7f9d0413b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce8066f60 0x7f9ce8069420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d0415d9b0 0x7f9d040b1190 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 --2- 172.21.15.133:0/3497891961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d040cee80 0x7f9d0413db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 -- 172.21.15.133:0/3497891961 >> 172.21.15.133:0/3497891961 conn(0x7f9d04067000 msgr2=0x7f9d040b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 -- 172.21.15.133:0/3497891961 shutdown_connections 2024-04-25T10:02:52.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.504+0000 7f9d0b573640 1 -- 172.21.15.133:0/3497891961 wait complete. 2024-04-25T10:02:52.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:52.528 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:52.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.636+0000 7fc340308640 1 Processor -- start 2024-04-25T10:02:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.636+0000 7fc340308640 1 -- start start 2024-04-25T10:02:52.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc340308640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc33806e8b0 0x7fc33815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc340308640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3381471f0 con 0x7fc3380b1e50 2024-04-25T10:02:52.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc340308640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3380b1090 con 0x7fc3380aebc0 2024-04-25T10:02:52.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc340308640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc338147490 con 0x7fc33806e8b0 2024-04-25T10:02:52.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d8a3640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc3380b1e50 0x7fc338146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47124/0 (socket says 172.21.15.133:47124) 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d8a3640 1 -- 172.21.15.133:0/1377622552 learned_addr learned my addr 172.21.15.133:0/1377622552 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865498439 0 0) 0x7fc3381471f0 con 0x7fc3380b1e50 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33e0a4640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc33806e8b0 0x7fc33815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc324003530 con 0x7fc3380b1e50 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 505666857 0 0) 0x7fc324003530 con 0x7fc3380b1e50 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc33806e8b0 msgr2=0x7fc33815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc33806e8b0 0x7fc33815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 >> v1:172.21.15.162:6789/0 conn(0x7fc3380aebc0 legacy=0x7fc33806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380b1370 con 0x7fc3380b1e50 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fc328001680 con 0x7fc3380b1e50 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 >> v1:172.21.15.133:6789/0 conn(0x7fc3380b1e50 legacy=0x7fc338146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3240039f0 0x7fc324003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc33806e8b0 0x7fc324004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc324004d50 0x7fc3240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338147490 con 0x7fc324004d50 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32400bb00 con 0x7fc3240039f0 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32400bd00 con 0x7fc33806e8b0 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33e8a5640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc324004d50 0x7fc3240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33e0a4640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3240039f0 0x7fc324003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d8a3640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc33806e8b0 0x7fc324004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33e8a5640 1 -- 172.21.15.133:0/1377622552 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc33806e8b0 msgr2=0x7fc324004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33e8a5640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc33806e8b0 0x7fc324004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33e8a5640 1 -- 172.21.15.133:0/1377622552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3240039f0 msgr2=0x7fc324003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33e8a5640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3240039f0 0x7fc324003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33e8a5640 1 -- 172.21.15.133:0/1377622552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc324003530 con 0x7fc324004d50 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33e8a5640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc324004d50 0x7fc3240051d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc330001e70 tx=0x7fc33001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921b846bd0b9eca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.640+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330001a90 con 0x7fc324004d50 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc330003b80 con 0x7fc324004d50 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33d0a2640 1 -- 172.21.15.133:0/1377622552 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330005080 con 0x7fc324004d50 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 -- 172.21.15.133:0/1377622552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc324004d50 msgr2=0x7fc3240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 --2- 172.21.15.133:0/1377622552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc324004d50 0x7fc3240051d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc330001e70 tx=0x7fc33001c780 comp rx=0 tx=0).stop 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33e8a5640 1 -- 172.21.15.133:0/1377622552 reap_dead start 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 -- 172.21.15.133:0/1377622552 shutdown_connections 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 -- 172.21.15.133:0/1377622552 >> 172.21.15.133:0/1377622552 conn(0x7fc338066f70 msgr2=0x7fc33806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 -- 172.21.15.133:0/1377622552 shutdown_connections 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 -- 172.21.15.133:0/1377622552 wait complete. 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 Processor -- start 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 -- start start 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc324004d50 0x7fc33815d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380c9b90 0x7fc3380ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3380c4900 0x7fc3380c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380b22b0 con 0x7fc3380c9b90 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380b19c0 con 0x7fc324004d50 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33815eb40 con 0x7fc3380c4900 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33d8a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380c9b90 0x7fc3380ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33d8a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380c9b90 0x7fc3380ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47056/0 (socket says 172.21.15.133:47056) 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33d8a3640 1 -- 172.21.15.133:0/3770752090 learned_addr learned my addr 172.21.15.133:0/3770752090 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33e0a4640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc324004d50 0x7fc33815d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33e8a5640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3380c4900 0x7fc3380c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33d8a3640 1 -- 172.21.15.133:0/3770752090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3380c4900 msgr2=0x7fc3380c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33d8a3640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3380c4900 0x7fc3380c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33d8a3640 1 -- 172.21.15.133:0/3770752090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc324004d50 msgr2=0x7fc33815d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33d8a3640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc324004d50 0x7fc33815d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33d8a3640 1 -- 172.21.15.133:0/3770752090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380c5670 con 0x7fc3380c9b90 2024-04-25T10:02:52.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc33d8a3640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380c9b90 0x7fc3380ca010 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc328001060 tx=0x7fc328011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc32effd640 1 -- 172.21.15.133:0/3770752090 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328010040 con 0x7fc3380c9b90 2024-04-25T10:02:52.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc32effd640 1 -- 172.21.15.133:0/3770752090 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32801a070 con 0x7fc3380c9b90 2024-04-25T10:02:52.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 -- 172.21.15.133:0/3770752090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3380c5950 con 0x7fc3380c9b90 2024-04-25T10:02:52.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc32effd640 1 -- 172.21.15.133:0/3770752090 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32800fde0 con 0x7fc3380c9b90 2024-04-25T10:02:52.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.644+0000 7fc340308640 1 -- 172.21.15.133:0/3770752090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3380c5f10 con 0x7fc3380c9b90 2024-04-25T10:02:52.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.648+0000 7fc340308640 1 -- 172.21.15.133:0/3770752090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc338068e80 con 0x7fc3380c9b90 2024-04-25T10:02:52.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.648+0000 7fc32effd640 1 -- 172.21.15.133:0/3770752090 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc32801f460 con 0x7fc3380c9b90 2024-04-25T10:02:52.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.648+0000 7fc32effd640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc31c066f60 0x7fc31c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.648+0000 7fc32effd640 1 -- 172.21.15.133:0/3770752090 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc328088760 con 0x7fc3380c9b90 2024-04-25T10:02:52.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.648+0000 7fc33e0a4640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc31c066f60 0x7fc31c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.648+0000 7fc33e0a4640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc31c066f60 0x7fc31c069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc334009a30 tx=0x7fc334008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.652+0000 7fc32effd640 1 -- 172.21.15.133:0/3770752090 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc328059370 con 0x7fc3380c9b90 2024-04-25T10:02:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.796+0000 7fc340308640 1 -- 172.21.15.133:0/3770752090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc33806b960 con 0x7fc3380c9b90 2024-04-25T10:02:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.800+0000 7fc32effd640 1 -- 172.21.15.133:0/3770752090 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc328058c70 con 0x7fc3380c9b90 2024-04-25T10:02:52.806 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:52.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.800+0000 7fc340308640 1 -- 172.21.15.133:0/3770752090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc31c066f60 msgr2=0x7fc31c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.800+0000 7fc340308640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc31c066f60 0x7fc31c069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc334009a30 tx=0x7fc334008040 comp rx=0 tx=0).stop 2024-04-25T10:02:52.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.800+0000 7fc340308640 1 -- 172.21.15.133:0/3770752090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380c9b90 msgr2=0x7fc3380ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.800+0000 7fc340308640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380c9b90 0x7fc3380ca010 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc328001060 tx=0x7fc328011d70 comp rx=0 tx=0).stop 2024-04-25T10:02:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.800+0000 7fc340308640 1 -- 172.21.15.133:0/3770752090 shutdown_connections 2024-04-25T10:02:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.800+0000 7fc340308640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3380c4900 0x7fc3380c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.804+0000 7fc340308640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc31c066f60 0x7fc31c069420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.804+0000 7fc340308640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380c9b90 0x7fc3380ca010 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.804+0000 7fc340308640 1 --2- 172.21.15.133:0/3770752090 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc324004d50 0x7fc33815d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.804+0000 7fc340308640 1 -- 172.21.15.133:0/3770752090 >> 172.21.15.133:0/3770752090 conn(0x7fc338066f70 msgr2=0x7fc33806d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.804+0000 7fc340308640 1 -- 172.21.15.133:0/3770752090 shutdown_connections 2024-04-25T10:02:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.804+0000 7fc340308640 1 -- 172.21.15.133:0/3770752090 wait complete. 2024-04-25T10:02:52.824 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:52.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a7aa95640 1 Processor -- start 2024-04-25T10:02:52.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a7aa95640 1 -- start start 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a7aa95640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8a74145170 0x7f8a7406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a7aa95640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8a7415ced0 0x7f8a7415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a7aa95640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a740ce2e0 con 0x7f8a7406e4d0 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a7aa95640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7406db00 con 0x7f8a7415ced0 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a7aa95640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a74145550 con 0x7f8a74145170 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a79032640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8a7415ced0 0x7f8a7415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a79032640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8a7415ced0 0x7f8a7415d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47070/0 (socket says 172.21.15.133:47070) 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a79032640 1 -- 172.21.15.133:0/4160463656 learned_addr learned my addr 172.21.15.133:0/4160463656 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a78831640 1 --2- 172.21.15.133:0/4160463656 >> v2:172.21.15.162:3300/0 conn(0x7f8a74145170 0x7f8a7406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a79032640 1 -- 172.21.15.133:0/4160463656 >> v2:172.21.15.162:3300/0 conn(0x7f8a74145170 msgr2=0x7f8a7406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a79032640 1 --2- 172.21.15.133:0/4160463656 >> v2:172.21.15.162:3300/0 conn(0x7f8a74145170 0x7f8a7406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a79032640 1 -- 172.21.15.133:0/4160463656 >> v1:172.21.15.162:6789/0 conn(0x7f8a7406e4d0 legacy=0x7f8a740cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.940+0000 7f8a79032640 1 -- 172.21.15.133:0/4160463656 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7406d4b0 con 0x7f8a7415ced0 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a79032640 1 --2- 172.21.15.133:0/4160463656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a7415ced0 0x7f8a7415d320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a7406b790 tx=0x7f8a6c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b99dc4456326feb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a737fe640 1 -- 172.21.15.133:0/4160463656 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0a5670 con 0x7f8a7415ced0 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a737fe640 1 -- 172.21.15.133:0/4160463656 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c0a5810 con 0x7f8a7415ced0 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a737fe640 1 -- 172.21.15.133:0/4160463656 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0a5be0 con 0x7f8a7415ced0 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 -- 172.21.15.133:0/4160463656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a7415ced0 msgr2=0x7f8a7415d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 --2- 172.21.15.133:0/4160463656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a7415ced0 0x7f8a7415d320 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a7406b790 tx=0x7f8a6c01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 -- 172.21.15.133:0/4160463656 shutdown_connections 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 --2- 172.21.15.133:0/4160463656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a7415ced0 0x7f8a7415d320 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 --2- 172.21.15.133:0/4160463656 >> v2:172.21.15.162:3300/0 conn(0x7f8a74145170 0x7f8a7406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 -- 172.21.15.133:0/4160463656 >> 172.21.15.133:0/4160463656 conn(0x7f8a74066f70 msgr2=0x7f8a7406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 -- 172.21.15.133:0/4160463656 shutdown_connections 2024-04-25T10:02:52.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 -- 172.21.15.133:0/4160463656 wait complete. 2024-04-25T10:02:52.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 Processor -- start 2024-04-25T10:02:52.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 -- start start 2024-04-25T10:02:52.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a7406e4d0 0x7f8a74156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a74145170 0x7f8a7413c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a7415ced0 0x7f8a7413ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a74168610 con 0x7f8a74145170 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a74168490 con 0x7f8a7415ced0 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.944+0000 7f8a7aa95640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a74168790 con 0x7f8a7406e4d0 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a74145170 0x7f8a7413c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a78831640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a7406e4d0 0x7f8a74156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a74145170 0x7f8a7413c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47076/0 (socket says 172.21.15.133:47076) 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a73fff640 1 -- 172.21.15.133:0/3503582792 learned_addr learned my addr 172.21.15.133:0/3503582792 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a79032640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a7415ced0 0x7f8a7413ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a73fff640 1 -- 172.21.15.133:0/3503582792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a7406e4d0 msgr2=0x7f8a74156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a73fff640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a7406e4d0 0x7f8a74156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a73fff640 1 -- 172.21.15.133:0/3503582792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a7415ced0 msgr2=0x7f8a7413ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a73fff640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a7415ced0 0x7f8a7413ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a73fff640 1 -- 172.21.15.133:0/3503582792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7413fc10 con 0x7f8a74145170 2024-04-25T10:02:52.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a73fff640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a74145170 0x7f8a7413c010 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800b3e0 tx=0x7f8a6800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a717fa640 1 -- 172.21.15.133:0/3503582792 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68015020 con 0x7f8a74145170 2024-04-25T10:02:52.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a717fa640 1 -- 172.21.15.133:0/3503582792 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a68011830 con 0x7f8a74145170 2024-04-25T10:02:52.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a7aa95640 1 -- 172.21.15.133:0/3503582792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7413fef0 con 0x7f8a74145170 2024-04-25T10:02:52.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a717fa640 1 -- 172.21.15.133:0/3503582792 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68011c60 con 0x7f8a74145170 2024-04-25T10:02:52.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a7aa95640 1 -- 172.21.15.133:0/3503582792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7413c550 con 0x7f8a74145170 2024-04-25T10:02:52.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a7aa95640 1 -- 172.21.15.133:0/3503582792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a74068e80 con 0x7f8a74145170 2024-04-25T10:02:52.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a717fa640 1 -- 172.21.15.133:0/3503582792 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8a68018020 con 0x7f8a74145170 2024-04-25T10:02:52.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a717fa640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8a60066f60 0x7f8a60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a717fa640 1 -- 172.21.15.133:0/3503582792 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8a68084f60 con 0x7f8a74145170 2024-04-25T10:02:52.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a78831640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8a60066f60 0x7f8a60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.948+0000 7f8a78831640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8a60066f60 0x7f8a60069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a64006fd0 tx=0x7f8a64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:52.952+0000 7f8a717fa640 1 -- 172.21.15.133:0/3503582792 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a6801f030 con 0x7f8a74145170 2024-04-25T10:02:53.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 -- 172.21.15.133:0/3503582792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a7415c0f0 con 0x7f8a74145170 2024-04-25T10:02:53.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a717fa640 1 -- 172.21.15.133:0/3503582792 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a68055520 con 0x7f8a74145170 2024-04-25T10:02:53.106 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 -- 172.21.15.133:0/3503582792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8a60066f60 msgr2=0x7f8a60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8a60066f60 0x7f8a60069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a64006fd0 tx=0x7f8a64008040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 -- 172.21.15.133:0/3503582792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a74145170 msgr2=0x7f8a7413c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a74145170 0x7f8a7413c010 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800b3e0 tx=0x7f8a6800b8a0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 -- 172.21.15.133:0/3503582792 shutdown_connections 2024-04-25T10:02:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a7415ced0 0x7f8a7413ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8a60066f60 0x7f8a60069420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a74145170 0x7f8a7413c010 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 --2- 172.21.15.133:0/3503582792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8a7406e4d0 0x7f8a74156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 -- 172.21.15.133:0/3503582792 >> 172.21.15.133:0/3503582792 conn(0x7f8a74066f70 msgr2=0x7f8a7415d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 -- 172.21.15.133:0/3503582792 shutdown_connections 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.100+0000 7f8a7aa95640 1 -- 172.21.15.133:0/3503582792 wait complete. 2024-04-25T10:02:53.123 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:53.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a82497640 1 Processor -- start 2024-04-25T10:02:53.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a82497640 1 -- start start 2024-04-25T10:02:53.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a82497640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a7c143ea0 0x7f1a7c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a82497640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1a7c06eac0 0x7f1a7c0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a82497640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7c15ac50 con 0x7f1a7c0b2150 2024-04-25T10:02:53.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a82497640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a7c0cd180 con 0x7f1a7c06eac0 2024-04-25T10:02:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a82497640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c15aed0 con 0x7f1a7c143ea0 2024-04-25T10:02:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a80a34640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1a7c06eac0 0x7f1a7c0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a80a34640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1a7c06eac0 0x7f1a7c0b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47082/0 (socket says 172.21.15.133:47082) 2024-04-25T10:02:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a80a34640 1 -- 172.21.15.133:0/3102770408 learned_addr learned my addr 172.21.15.133:0/3102770408 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a80a34640 1 -- 172.21.15.133:0/3102770408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a7c143ea0 msgr2=0x7f1a7c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a7bfff640 1 --2- 172.21.15.133:0/3102770408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a7c143ea0 0x7f1a7c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a80a34640 1 --2- 172.21.15.133:0/3102770408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a7c143ea0 0x7f1a7c0cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a80a34640 1 -- 172.21.15.133:0/3102770408 >> v1:172.21.15.162:6789/0 conn(0x7f1a7c0b2150 legacy=0x7f1a7c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.228+0000 7f1a80a34640 1 -- 172.21.15.133:0/3102770408 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c15b1b0 con 0x7f1a7c06eac0 2024-04-25T10:02:53.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a80a34640 1 --2- 172.21.15.133:0/3102770408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a7c06eac0 0x7f1a7c0b06d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009e50 tx=0x7f1a7401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05f80415a3f598a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a7affd640 1 -- 172.21.15.133:0/3102770408 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a740a55a0 con 0x7f1a7c06eac0 2024-04-25T10:02:53.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a7affd640 1 -- 172.21.15.133:0/3102770408 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a740a5740 con 0x7f1a7c06eac0 2024-04-25T10:02:53.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a7affd640 1 -- 172.21.15.133:0/3102770408 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a740a5b10 con 0x7f1a7c06eac0 2024-04-25T10:02:53.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 -- 172.21.15.133:0/3102770408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a7c06eac0 msgr2=0x7f1a7c0b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 --2- 172.21.15.133:0/3102770408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a7c06eac0 0x7f1a7c0b06d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009e50 tx=0x7f1a7401c810 comp rx=0 tx=0).stop 2024-04-25T10:02:53.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 -- 172.21.15.133:0/3102770408 shutdown_connections 2024-04-25T10:02:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 --2- 172.21.15.133:0/3102770408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a7c06eac0 0x7f1a7c0b06d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 --2- 172.21.15.133:0/3102770408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a7c143ea0 0x7f1a7c0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 -- 172.21.15.133:0/3102770408 >> 172.21.15.133:0/3102770408 conn(0x7f1a7c066f70 msgr2=0x7f1a7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 -- 172.21.15.133:0/3102770408 shutdown_connections 2024-04-25T10:02:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 -- 172.21.15.133:0/3102770408 wait complete. 2024-04-25T10:02:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 Processor -- start 2024-04-25T10:02:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 -- start start 2024-04-25T10:02:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a7c06eac0 0x7f1a7c164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a7c0b2150 0x7f1a7c15f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a7c143ea0 0x7f1a7c15f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c167550 con 0x7f1a7c143ea0 2024-04-25T10:02:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c1673d0 con 0x7f1a7c0b2150 2024-04-25T10:02:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c1676d0 con 0x7f1a7c06eac0 2024-04-25T10:02:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a80a34640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a7c143ea0 0x7f1a7c15f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a80a34640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a7c143ea0 0x7f1a7c15f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47094/0 (socket says 172.21.15.133:47094) 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a80a34640 1 -- 172.21.15.133:0/1694820138 learned_addr learned my addr 172.21.15.133:0/1694820138 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a7b7fe640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a7c0b2150 0x7f1a7c15f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a7bfff640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a7c06eac0 0x7f1a7c164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a80a34640 1 -- 172.21.15.133:0/1694820138 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a7c06eac0 msgr2=0x7f1a7c164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a80a34640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a7c06eac0 0x7f1a7c164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a80a34640 1 -- 172.21.15.133:0/1694820138 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a7c0b2150 msgr2=0x7f1a7c15f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a80a34640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a7c0b2150 0x7f1a7c15f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a80a34640 1 -- 172.21.15.133:0/1694820138 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c13fb10 con 0x7f1a7c143ea0 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a7b7fe640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a7c0b2150 0x7f1a7c15f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a80a34640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a7c143ea0 0x7f1a7c15f8b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009e50 tx=0x7f1a740195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a78ff9640 1 -- 172.21.15.133:0/1694820138 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a740bc020 con 0x7f1a7c143ea0 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a78ff9640 1 -- 172.21.15.133:0/1694820138 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a740b8670 con 0x7f1a7c143ea0 2024-04-25T10:02:53.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 -- 172.21.15.133:0/1694820138 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7c13fdf0 con 0x7f1a7c143ea0 2024-04-25T10:02:53.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a78ff9640 1 -- 172.21.15.133:0/1694820138 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74009510 con 0x7f1a7c143ea0 2024-04-25T10:02:53.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.232+0000 7f1a82497640 1 -- 172.21.15.133:0/1694820138 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7c1403b0 con 0x7f1a7c143ea0 2024-04-25T10:02:53.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.236+0000 7f1a82497640 1 -- 172.21.15.133:0/1694820138 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7c068e80 con 0x7f1a7c143ea0 2024-04-25T10:02:53.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.236+0000 7f1a78ff9640 1 -- 172.21.15.133:0/1694820138 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a740bf020 con 0x7f1a7c143ea0 2024-04-25T10:02:53.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.236+0000 7f1a78ff9640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 0x7f1a68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.236+0000 7f1a78ff9640 1 -- 172.21.15.133:0/1694820138 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1a74120450 con 0x7f1a7c143ea0 2024-04-25T10:02:53.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.236+0000 7f1a7bfff640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 0x7f1a68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.236+0000 7f1a7bfff640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 0x7f1a68069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000b800 tx=0x7f1a70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.240+0000 7f1a78ff9640 1 -- 172.21.15.133:0/1694820138 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a740c6030 con 0x7f1a7c143ea0 2024-04-25T10:02:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.384+0000 7f1a82497640 1 -- 172.21.15.133:0/1694820138 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a7c06e6f0 con 0x7f1a7c143ea0 2024-04-25T10:02:53.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.384+0000 7f1a78ff9640 1 -- 172.21.15.133:0/1694820138 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a7c15ac50 con 0x7f1a7c143ea0 2024-04-25T10:02:53.393 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:53.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 -- 172.21.15.133:0/1694820138 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 msgr2=0x7f1a68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 0x7f1a68069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000b800 tx=0x7f1a70008040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 -- 172.21.15.133:0/1694820138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a7c143ea0 msgr2=0x7f1a7c15f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a7c143ea0 0x7f1a7c15f8b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009e50 tx=0x7f1a740195b0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 -- 172.21.15.133:0/1694820138 shutdown_connections 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 0x7f1a68069420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a7c143ea0 0x7f1a7c15f8b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a7c0b2150 0x7f1a7c15f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 --2- 172.21.15.133:0/1694820138 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a7c06eac0 0x7f1a7c164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 -- 172.21.15.133:0/1694820138 >> 172.21.15.133:0/1694820138 conn(0x7f1a7c066f70 msgr2=0x7f1a7c15c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 -- 172.21.15.133:0/1694820138 shutdown_connections 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.388+0000 7f1a82497640 1 -- 172.21.15.133:0/1694820138 wait complete. 2024-04-25T10:02:53.414 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.516+0000 7f0df1983640 1 Processor -- start 2024-04-25T10:02:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.516+0000 7f0df1983640 1 -- start start 2024-04-25T10:02:53.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0df1983640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0dec0ce600 0x7f0dec0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0df1983640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0dec0d1920 0x7f0dec0d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0df1983640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec15ebd0 con 0x7f0dec15d9a0 2024-04-25T10:02:53.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0df1983640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec15eeb0 con 0x7f0dec0d1920 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0df1983640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec0d0330 con 0x7f0dec0ce600 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0deaffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0dec15d9a0 0x7f0dec0d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47126/0 (socket says 172.21.15.133:47126) 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0deaffd640 1 -- 172.21.15.133:0/662238049 learned_addr learned my addr 172.21.15.133:0/662238049 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0deb7fe640 1 --2- 172.21.15.133:0/662238049 >> v2:172.21.15.133:3300/0 conn(0x7f0dec0d1920 0x7f0dec0d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0de9ffb640 1 -- 172.21.15.133:0/662238049 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957289193 0 0) 0x7f0dec15ebd0 con 0x7f0dec15d9a0 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0de9ffb640 1 -- 172.21.15.133:0/662238049 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd8003530 con 0x7f0dec15d9a0 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0dea7fc640 1 --2- 172.21.15.133:0/662238049 >> v2:172.21.15.162:3300/0 conn(0x7f0dec0ce600 0x7f0dec0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0deb7fe640 1 -- 172.21.15.133:0/662238049 >> v2:172.21.15.162:3300/0 conn(0x7f0dec0ce600 msgr2=0x7f0dec0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0deb7fe640 1 --2- 172.21.15.133:0/662238049 >> v2:172.21.15.162:3300/0 conn(0x7f0dec0ce600 0x7f0dec0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0deb7fe640 1 -- 172.21.15.133:0/662238049 >> v1:172.21.15.133:6789/0 conn(0x7f0dec15d9a0 legacy=0x7f0dec0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0deb7fe640 1 -- 172.21.15.133:0/662238049 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec0d0610 con 0x7f0dec0d1920 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0dea7fc640 1 --2- 172.21.15.133:0/662238049 >> v2:172.21.15.162:3300/0 conn(0x7f0dec0ce600 0x7f0dec0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0deb7fe640 1 --2- 172.21.15.133:0/662238049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dec0d1920 0x7f0dec0d1d70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0ce080 tx=0x7f0ddc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7775f63be51c692 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0de9ffb640 1 -- 172.21.15.133:0/662238049 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0b0070 con 0x7f0dec0d1920 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0de9ffb640 1 -- 172.21.15.133:0/662238049 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc0ab600 con 0x7f0dec0d1920 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0de9ffb640 1 -- 172.21.15.133:0/662238049 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0ab9d0 con 0x7f0dec0d1920 2024-04-25T10:02:53.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0df1983640 1 -- 172.21.15.133:0/662238049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dec0d1920 msgr2=0x7f0dec0d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.520+0000 7f0df1983640 1 --2- 172.21.15.133:0/662238049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dec0d1920 0x7f0dec0d1d70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0ce080 tx=0x7f0ddc01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 -- 172.21.15.133:0/662238049 shutdown_connections 2024-04-25T10:02:53.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 --2- 172.21.15.133:0/662238049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dec0d1920 0x7f0dec0d1d70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 --2- 172.21.15.133:0/662238049 >> v2:172.21.15.162:3300/0 conn(0x7f0dec0ce600 0x7f0dec0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 -- 172.21.15.133:0/662238049 >> 172.21.15.133:0/662238049 conn(0x7f0dec06bf80 msgr2=0x7f0dec06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 -- 172.21.15.133:0/662238049 shutdown_connections 2024-04-25T10:02:53.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 -- 172.21.15.133:0/662238049 wait complete. 2024-04-25T10:02:53.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 Processor -- start 2024-04-25T10:02:53.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 -- start start 2024-04-25T10:02:53.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dec0ce600 0x7f0dec14a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dec0d1920 0x7f0dec147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dec15d9a0 0x7f0dec0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec15e960 con 0x7f0dec15d9a0 2024-04-25T10:02:53.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec15e7e0 con 0x7f0dec0ce600 2024-04-25T10:02:53.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec16a7e0 con 0x7f0dec0d1920 2024-04-25T10:02:53.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0deb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dec15d9a0 0x7f0dec0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0deb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dec15d9a0 0x7f0dec0cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47106/0 (socket says 172.21.15.133:47106) 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0deb7fe640 1 -- 172.21.15.133:0/1827129067 learned_addr learned my addr 172.21.15.133:0/1827129067 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0deaffd640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dec0ce600 0x7f0dec14a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0deb7fe640 1 -- 172.21.15.133:0/1827129067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dec0d1920 msgr2=0x7f0dec147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0deb7fe640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dec0d1920 0x7f0dec147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0deb7fe640 1 -- 172.21.15.133:0/1827129067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dec0ce600 msgr2=0x7f0dec14a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0deb7fe640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dec0ce600 0x7f0dec14a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0deb7fe640 1 -- 172.21.15.133:0/1827129067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec0cda30 con 0x7f0dec15d9a0 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0deb7fe640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dec15d9a0 0x7f0dec0cb080 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc019000 tx=0x7f0ddc019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0de3fff640 1 -- 172.21.15.133:0/1827129067 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0abe60 con 0x7f0dec15d9a0 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0de3fff640 1 -- 172.21.15.133:0/1827129067 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc0b9070 con 0x7f0dec15d9a0 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 -- 172.21.15.133:0/1827129067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec0cdcf0 con 0x7f0dec15d9a0 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0de3fff640 1 -- 172.21.15.133:0/1827129067 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0b0040 con 0x7f0dec15d9a0 2024-04-25T10:02:53.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 -- 172.21.15.133:0/1827129067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec0c8f60 con 0x7f0dec15d9a0 2024-04-25T10:02:53.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.524+0000 7f0df1983640 1 -- 172.21.15.133:0/1827129067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec0c8d90 con 0x7f0dec15d9a0 2024-04-25T10:02:53.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.528+0000 7f0de3fff640 1 -- 172.21.15.133:0/1827129067 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ddc0b4650 con 0x7f0dec15d9a0 2024-04-25T10:02:53.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.528+0000 7f0de3fff640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0dcc066f60 0x7f0dcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.528+0000 7f0de3fff640 1 -- 172.21.15.133:0/1827129067 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0ddc11f0d0 con 0x7f0dec15d9a0 2024-04-25T10:02:53.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.528+0000 7f0deaffd640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0dcc066f60 0x7f0dcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.528+0000 7f0deaffd640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0dcc066f60 0x7f0dcc069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0de40012f0 tx=0x7f0de4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.532+0000 7f0de3fff640 1 -- 172.21.15.133:0/1827129067 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ddc016070 con 0x7f0dec15d9a0 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.676+0000 7f0df1983640 1 -- 172.21.15.133:0/1827129067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0dec0cb820 con 0x7f0dec15d9a0 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.676+0000 7f0de3fff640 1 -- 172.21.15.133:0/1827129067 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0ddc0efce0 con 0x7f0dec15d9a0 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 -- 172.21.15.133:0/1827129067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0dcc066f60 msgr2=0x7f0dcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0dcc066f60 0x7f0dcc069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0de40012f0 tx=0x7f0de4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 -- 172.21.15.133:0/1827129067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dec15d9a0 msgr2=0x7f0dec0cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dec15d9a0 0x7f0dec0cb080 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc019000 tx=0x7f0ddc019190 comp rx=0 tx=0).stop 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 -- 172.21.15.133:0/1827129067 shutdown_connections 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0dcc066f60 0x7f0dcc069420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dec15d9a0 0x7f0dec0cb080 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0dec0d1920 0x7f0dec147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 --2- 172.21.15.133:0/1827129067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0dec0ce600 0x7f0dec14a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 -- 172.21.15.133:0/1827129067 >> 172.21.15.133:0/1827129067 conn(0x7f0dec06bf80 msgr2=0x7f0dec161620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 -- 172.21.15.133:0/1827129067 shutdown_connections 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.680+0000 7f0df1983640 1 -- 172.21.15.133:0/1827129067 wait complete. 2024-04-25T10:02:53.703 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:53.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8de2626640 1 Processor -- start 2024-04-25T10:02:53.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8de2626640 1 -- start start 2024-04-25T10:02:53.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8de2626640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8ddc15eb40 0x7f8ddc0b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8de2626640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8ddc0aebc0 0x7f8ddc0af010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8de2626640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc1471f0 con 0x7f8ddc06e8b0 2024-04-25T10:02:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8de2626640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc0af550 con 0x7f8ddc15eb40 2024-04-25T10:02:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8de2626640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc147490 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8ddb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8ddc15eb40 0x7f8ddc0b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8ddb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8ddc15eb40 0x7f8ddc0b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47112/0 (socket says 172.21.15.133:47112) 2024-04-25T10:02:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8ddb7fe640 1 -- 172.21.15.133:0/134084010 learned_addr learned my addr 172.21.15.133:0/134084010 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8de0bc3640 1 --2- 172.21.15.133:0/134084010 >> v2:172.21.15.162:3300/0 conn(0x7f8ddc0aebc0 0x7f8ddc0af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8ddb7fe640 1 -- 172.21.15.133:0/134084010 >> v2:172.21.15.162:3300/0 conn(0x7f8ddc0aebc0 msgr2=0x7f8ddc0af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8ddb7fe640 1 --2- 172.21.15.133:0/134084010 >> v2:172.21.15.162:3300/0 conn(0x7f8ddc0aebc0 0x7f8ddc0af010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8ddb7fe640 1 -- 172.21.15.133:0/134084010 >> v1:172.21.15.162:6789/0 conn(0x7f8ddc06e8b0 legacy=0x7f8ddc146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.808+0000 7f8ddb7fe640 1 -- 172.21.15.133:0/134084010 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0b1090 con 0x7f8ddc15eb40 2024-04-25T10:02:53.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddb7fe640 1 --2- 172.21.15.133:0/134084010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc15eb40 0x7f8ddc0b1e50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0b27f0 tx=0x7f8dd001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2018174d9bbe44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddaffd640 1 -- 172.21.15.133:0/134084010 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd00b0070 con 0x7f8ddc15eb40 2024-04-25T10:02:53.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddaffd640 1 -- 172.21.15.133:0/134084010 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd00ab600 con 0x7f8ddc15eb40 2024-04-25T10:02:53.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddaffd640 1 -- 172.21.15.133:0/134084010 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd00ab9d0 con 0x7f8ddc15eb40 2024-04-25T10:02:53.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 -- 172.21.15.133:0/134084010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc15eb40 msgr2=0x7f8ddc0b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 --2- 172.21.15.133:0/134084010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc15eb40 0x7f8ddc0b1e50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0b27f0 tx=0x7f8dd001c450 comp rx=0 tx=0).stop 2024-04-25T10:02:53.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 -- 172.21.15.133:0/134084010 shutdown_connections 2024-04-25T10:02:53.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 --2- 172.21.15.133:0/134084010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc15eb40 0x7f8ddc0b1e50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 --2- 172.21.15.133:0/134084010 >> v2:172.21.15.162:3300/0 conn(0x7f8ddc0aebc0 0x7f8ddc0af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 -- 172.21.15.133:0/134084010 >> 172.21.15.133:0/134084010 conn(0x7f8ddc066f70 msgr2=0x7f8ddc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 -- 172.21.15.133:0/134084010 shutdown_connections 2024-04-25T10:02:53.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 -- 172.21.15.133:0/134084010 wait complete. 2024-04-25T10:02:53.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 Processor -- start 2024-04-25T10:02:53.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 -- start start 2024-04-25T10:02:53.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ddc06e8b0 0x7f8ddc1460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc0aebc0 0x7f8ddc13eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ddc15eb40 0x7f8ddc13d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc06de40 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc06dcc0 con 0x7f8ddc06e8b0 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8de2626640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0b2390 con 0x7f8ddc15eb40 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc0aebc0 0x7f8ddc13eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc0aebc0 0x7f8ddc13eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47126/0 (socket says 172.21.15.133:47126) 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddb7fe640 1 -- 172.21.15.133:0/279396975 learned_addr learned my addr 172.21.15.133:0/279396975 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddb7fe640 1 -- 172.21.15.133:0/279396975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ddc15eb40 msgr2=0x7f8ddc13d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddb7fe640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ddc15eb40 0x7f8ddc13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddb7fe640 1 -- 172.21.15.133:0/279396975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ddc06e8b0 msgr2=0x7f8ddc1460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddb7fe640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ddc06e8b0 0x7f8ddc1460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddb7fe640 1 -- 172.21.15.133:0/279396975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc15d2d0 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.812+0000 7f8ddb7fe640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc0aebc0 0x7f8ddc13eed0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00ae040 tx=0x7f8dd00191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.816+0000 7f8dd8ff9640 1 -- 172.21.15.133:0/279396975 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd00b0050 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.816+0000 7f8dd8ff9640 1 -- 172.21.15.133:0/279396975 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd00bc070 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.816+0000 7f8de2626640 1 -- 172.21.15.133:0/279396975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc13deb0 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.816+0000 7f8dd8ff9640 1 -- 172.21.15.133:0/279396975 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd00b5a70 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.816+0000 7f8de2626640 1 -- 172.21.15.133:0/279396975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc1458c0 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.816+0000 7f8de2626640 1 -- 172.21.15.133:0/279396975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc068e80 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.816+0000 7f8dd8ff9640 1 -- 172.21.15.133:0/279396975 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8dd00b5c10 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.816+0000 7f8dd8ff9640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.816+0000 7f8dd8ff9640 1 -- 172.21.15.133:0/279396975 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8dd011fb30 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.816+0000 7f8ddbfff640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.816+0000 7f8ddbfff640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8dc8066f60 0x7f8dc8069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc003cf0 tx=0x7f8dcc0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.820+0000 7f8dd8ff9640 1 -- 172.21.15.133:0/279396975 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8dd00160d0 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.976+0000 7f8de2626640 1 -- 172.21.15.133:0/279396975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8ddc06b960 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8dd8ff9640 1 -- 172.21.15.133:0/279396975 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8dd00f08f0 con 0x7f8ddc0aebc0 2024-04-25T10:02:53.985 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:02:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 -- 172.21.15.133:0/279396975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8dc8066f60 msgr2=0x7f8dc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8dc8066f60 0x7f8dc8069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc003cf0 tx=0x7f8dcc0079c0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 -- 172.21.15.133:0/279396975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc0aebc0 msgr2=0x7f8ddc13eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc0aebc0 0x7f8ddc13eed0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00ae040 tx=0x7f8dd00191f0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 -- 172.21.15.133:0/279396975 shutdown_connections 2024-04-25T10:02:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ddc15eb40 0x7f8ddc13d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ddc0aebc0 0x7f8ddc13eed0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 --2- 172.21.15.133:0/279396975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ddc06e8b0 0x7f8ddc1460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 -- 172.21.15.133:0/279396975 >> 172.21.15.133:0/279396975 conn(0x7f8ddc066f70 msgr2=0x7f8ddc15fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 -- 172.21.15.133:0/279396975 shutdown_connections 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:53.980+0000 7f8de2626640 1 -- 172.21.15.133:0/279396975 wait complete. 2024-04-25T10:02:54.003 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:02:54.003 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:54.010 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:54.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f25675c8640 1 Processor -- start 2024-04-25T10:02:54.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f25675c8640 1 -- start start 2024-04-25T10:02:54.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f25675c8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f256015c9a0 0x7f25600c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f25675c8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25600b1f70 0x7f256006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f25675c8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f25600ce330 0x7f25600ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f25675c8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25600cece0 con 0x7f25600ce330 2024-04-25T10:02:54.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f25675c8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25600c61d0 con 0x7f256015c9a0 2024-04-25T10:02:54.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f25675c8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25600c64b0 con 0x7f25600b1f70 2024-04-25T10:02:54.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f2565b65640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f25600ce330 0x7f25600ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f2565b65640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f25600ce330 0x7f25600ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47138/0 (socket says 172.21.15.133:47138) 2024-04-25T10:02:54.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f2565b65640 1 -- 172.21.15.133:0/88870221 learned_addr learned my addr 172.21.15.133:0/88870221 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:54.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f2564b63640 1 --2- 172.21.15.133:0/88870221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25600b1f70 0x7f256006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f2565364640 1 --2- 172.21.15.133:0/88870221 >> v2:172.21.15.162:3300/0 conn(0x7f256015c9a0 0x7f25600c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f2565b65640 1 -- 172.21.15.133:0/88870221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25600b1f70 msgr2=0x7f256006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f2565b65640 1 --2- 172.21.15.133:0/88870221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25600b1f70 0x7f256006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f2565b65640 1 -- 172.21.15.133:0/88870221 >> v2:172.21.15.162:3300/0 conn(0x7f256015c9a0 msgr2=0x7f25600c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f2565b65640 1 --2- 172.21.15.133:0/88870221 >> v2:172.21.15.162:3300/0 conn(0x7f256015c9a0 0x7f25600c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f2565b65640 1 -- 172.21.15.133:0/88870221 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256006d4f0 con 0x7f25600ce330 2024-04-25T10:02:54.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.124+0000 7f2565b65640 1 --2- 172.21.15.133:0/88870221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25600ce330 0x7f25600ce7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f256006f5f0 tx=0x7f255801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994659c34e4765c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f255ffff640 1 -- 172.21.15.133:0/88870221 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580b1070 con 0x7f25600ce330 2024-04-25T10:02:54.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f255ffff640 1 -- 172.21.15.133:0/88870221 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25580ac620 con 0x7f25600ce330 2024-04-25T10:02:54.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f255ffff640 1 -- 172.21.15.133:0/88870221 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580ac9f0 con 0x7f25600ce330 2024-04-25T10:02:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 -- 172.21.15.133:0/88870221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25600ce330 msgr2=0x7f25600ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 --2- 172.21.15.133:0/88870221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25600ce330 0x7f25600ce7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f256006f5f0 tx=0x7f255801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 -- 172.21.15.133:0/88870221 shutdown_connections 2024-04-25T10:02:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 --2- 172.21.15.133:0/88870221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25600ce330 0x7f25600ce7a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 --2- 172.21.15.133:0/88870221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f25600b1f70 0x7f256006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 --2- 172.21.15.133:0/88870221 >> v2:172.21.15.162:3300/0 conn(0x7f256015c9a0 0x7f25600c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 -- 172.21.15.133:0/88870221 >> 172.21.15.133:0/88870221 conn(0x7f2560067000 msgr2=0x7f256006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 -- 172.21.15.133:0/88870221 shutdown_connections 2024-04-25T10:02:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 -- 172.21.15.133:0/88870221 wait complete. 2024-04-25T10:02:54.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 Processor -- start 2024-04-25T10:02:54.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 -- start start 2024-04-25T10:02:54.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25600b1f70 0x7f256015dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25600ce330 0x7f2560139cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f256015c9a0 0x7f256013a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256015cf00 con 0x7f25600ce330 2024-04-25T10:02:54.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256015cd80 con 0x7f25600b1f70 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256015d950 con 0x7f256015c9a0 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2564b63640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25600ce330 0x7f2560139cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2564b63640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25600ce330 0x7f2560139cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47154/0 (socket says 172.21.15.133:47154) 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2564b63640 1 -- 172.21.15.133:0/893773221 learned_addr learned my addr 172.21.15.133:0/893773221 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2565b65640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f256015c9a0 0x7f256013a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2565364640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25600b1f70 0x7f256015dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2564b63640 1 -- 172.21.15.133:0/893773221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f256015c9a0 msgr2=0x7f256013a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2564b63640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f256015c9a0 0x7f256013a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2564b63640 1 -- 172.21.15.133:0/893773221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25600b1f70 msgr2=0x7f256015dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2564b63640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25600b1f70 0x7f256015dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2564b63640 1 -- 172.21.15.133:0/893773221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256014ec60 con 0x7f25600ce330 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2565b65640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f256015c9a0 0x7f256013a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f2564b63640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25600ce330 0x7f2560139cd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f255400d830 tx=0x7f255400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f255dffb640 1 -- 172.21.15.133:0/893773221 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554018020 con 0x7f25600ce330 2024-04-25T10:02:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f255dffb640 1 -- 172.21.15.133:0/893773221 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25540096c0 con 0x7f25600ce330 2024-04-25T10:02:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 -- 172.21.15.133:0/893773221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256014ef40 con 0x7f25600ce330 2024-04-25T10:02:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f255dffb640 1 -- 172.21.15.133:0/893773221 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554009ab0 con 0x7f25600ce330 2024-04-25T10:02:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.128+0000 7f25675c8640 1 -- 172.21.15.133:0/893773221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256014f500 con 0x7f25600ce330 2024-04-25T10:02:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.132+0000 7f25675c8640 1 -- 172.21.15.133:0/893773221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256006bc20 con 0x7f25600ce330 2024-04-25T10:02:54.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.132+0000 7f255dffb640 1 -- 172.21.15.133:0/893773221 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f255401b020 con 0x7f25600ce330 2024-04-25T10:02:54.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.132+0000 7f255dffb640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f254c066f60 0x7f254c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.132+0000 7f255dffb640 1 -- 172.21.15.133:0/893773221 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2554085990 con 0x7f25600ce330 2024-04-25T10:02:54.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.132+0000 7f2565364640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f254c066f60 0x7f254c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.132+0000 7f2565364640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f254c066f60 0x7f254c069420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2550003fd0 tx=0x7f25500039a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.136+0000 7f255dffb640 1 -- 172.21.15.133:0/893773221 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2554022020 con 0x7f25600ce330 2024-04-25T10:02:54.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.296+0000 7f25675c8640 1 -- 172.21.15.133:0/893773221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f256014f6d0 con 0x7f25600ce330 2024-04-25T10:02:54.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.296+0000 7f255dffb640 1 -- 172.21.15.133:0/893773221 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f2554056630 con 0x7f25600ce330 2024-04-25T10:02:54.303 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:54.303 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.296+0000 7f25675c8640 1 -- 172.21.15.133:0/893773221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f254c066f60 msgr2=0x7f254c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.296+0000 7f25675c8640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f254c066f60 0x7f254c069420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2550003fd0 tx=0x7f25500039a0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.296+0000 7f25675c8640 1 -- 172.21.15.133:0/893773221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25600ce330 msgr2=0x7f2560139cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.296+0000 7f25675c8640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25600ce330 0x7f2560139cd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f255400d830 tx=0x7f255400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.300+0000 7f25675c8640 1 -- 172.21.15.133:0/893773221 shutdown_connections 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.300+0000 7f25675c8640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f256015c9a0 0x7f256013a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.300+0000 7f25675c8640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f254c066f60 0x7f254c069420 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.300+0000 7f25675c8640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25600ce330 0x7f2560139cd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.300+0000 7f25675c8640 1 --2- 172.21.15.133:0/893773221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25600b1f70 0x7f256015dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.300+0000 7f25675c8640 1 -- 172.21.15.133:0/893773221 >> 172.21.15.133:0/893773221 conn(0x7f2560067000 msgr2=0x7f256015b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.300+0000 7f25675c8640 1 -- 172.21.15.133:0/893773221 shutdown_connections 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.300+0000 7f25675c8640 1 -- 172.21.15.133:0/893773221 wait complete. 2024-04-25T10:02:54.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:54.323 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f77279d8640 1 Processor -- start 2024-04-25T10:02:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f77279d8640 1 -- start start 2024-04-25T10:02:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f77279d8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f772015e100 0x7f7720140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f77279d8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77200cf280 con 0x7f7720142920 2024-04-25T10:02:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f77279d8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772006d440 con 0x7f7720143560 2024-04-25T10:02:54.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f77279d8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772006d720 con 0x7f772015e100 2024-04-25T10:02:54.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f7725f75640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f772015e100 0x7f7720140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f7725774640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7720142920 0x7f77200ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47132/0 (socket says 172.21.15.133:47132) 2024-04-25T10:02:54.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f7725774640 1 -- 172.21.15.133:0/1208543642 learned_addr learned my addr 172.21.15.133:0/1208543642 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:54.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f7725f75640 1 -- 172.21.15.133:0/1208543642 >> v1:172.21.15.162:6789/0 conn(0x7f7720143560 legacy=0x7f772015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f7725f75640 1 -- 172.21.15.133:0/1208543642 >> v1:172.21.15.133:6789/0 conn(0x7f7720142920 legacy=0x7f77200ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.436+0000 7f7725f75640 1 -- 172.21.15.133:0/1208543642 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772006da00 con 0x7f772015e100 2024-04-25T10:02:54.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f7725f75640 1 --2- 172.21.15.133:0/1208543642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772015e100 0x7f7720140a40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f77180097c0 tx=0x7f771801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87612ba3db9a3d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f771ffff640 1 -- 172.21.15.133:0/1208543642 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77180b0070 con 0x7f772015e100 2024-04-25T10:02:54.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f771ffff640 1 -- 172.21.15.133:0/1208543642 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77180ab600 con 0x7f772015e100 2024-04-25T10:02:54.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f771ffff640 1 -- 172.21.15.133:0/1208543642 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77180ab9d0 con 0x7f772015e100 2024-04-25T10:02:54.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 -- 172.21.15.133:0/1208543642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772015e100 msgr2=0x7f7720140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 --2- 172.21.15.133:0/1208543642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772015e100 0x7f7720140a40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f77180097c0 tx=0x7f771801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 -- 172.21.15.133:0/1208543642 shutdown_connections 2024-04-25T10:02:54.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 --2- 172.21.15.133:0/1208543642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772015e100 0x7f7720140a40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 -- 172.21.15.133:0/1208543642 >> 172.21.15.133:0/1208543642 conn(0x7f7720066f70 msgr2=0x7f772006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 -- 172.21.15.133:0/1208543642 shutdown_connections 2024-04-25T10:02:54.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 -- 172.21.15.133:0/1208543642 wait complete. 2024-04-25T10:02:54.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 Processor -- start 2024-04-25T10:02:54.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 -- start start 2024-04-25T10:02:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7720142920 0x7f7720139340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7720143560 0x7f772013dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f772015e100 0x7f772013e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772006eb40 con 0x7f7720142920 2024-04-25T10:02:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772006e9c0 con 0x7f7720143560 2024-04-25T10:02:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f77279d8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7720167da0 con 0x7f772015e100 2024-04-25T10:02:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f7725774640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7720142920 0x7f7720139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f7725774640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7720142920 0x7f7720139340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47164/0 (socket says 172.21.15.133:47164) 2024-04-25T10:02:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f7725774640 1 -- 172.21.15.133:0/1478564720 learned_addr learned my addr 172.21.15.133:0/1478564720 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f7724f73640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7720143560 0x7f772013dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.440+0000 7f7725f75640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f772015e100 0x7f772013e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f7725774640 1 -- 172.21.15.133:0/1478564720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f772015e100 msgr2=0x7f772013e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f7725774640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f772015e100 0x7f772013e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f7725774640 1 -- 172.21.15.133:0/1478564720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7720143560 msgr2=0x7f772013dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f7725774640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7720143560 0x7f772013dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f7725774640 1 -- 172.21.15.133:0/1478564720 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7720154530 con 0x7f7720142920 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f7724f73640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7720143560 0x7f772013dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f7725774640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7720142920 0x7f7720139340 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f771000cad0 tx=0x7f771000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f771dffb640 1 -- 172.21.15.133:0/1478564720 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710017070 con 0x7f7720142920 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f771dffb640 1 -- 172.21.15.133:0/1478564720 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771000a7c0 con 0x7f7720142920 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f771dffb640 1 -- 172.21.15.133:0/1478564720 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771000ab90 con 0x7f7720142920 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f77279d8640 1 -- 172.21.15.133:0/1478564720 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7720154810 con 0x7f7720142920 2024-04-25T10:02:54.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f77279d8640 1 -- 172.21.15.133:0/1478564720 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7720154dd0 con 0x7f7720142920 2024-04-25T10:02:54.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f77279d8640 1 -- 172.21.15.133:0/1478564720 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7720068e80 con 0x7f7720142920 2024-04-25T10:02:54.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f771dffb640 1 -- 172.21.15.133:0/1478564720 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f771001a070 con 0x7f7720142920 2024-04-25T10:02:54.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f771dffb640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f770c066f60 0x7f770c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f7724f73640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f770c066f60 0x7f770c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f771dffb640 1 -- 172.21.15.133:0/1478564720 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7710084d20 con 0x7f7720142920 2024-04-25T10:02:54.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.444+0000 7f7724f73640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f770c066f60 0x7f770c069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7720067db0 tx=0x7f7714009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.448+0000 7f771dffb640 1 -- 172.21.15.133:0/1478564720 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7710055930 con 0x7f7720142920 2024-04-25T10:02:54.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.608+0000 7f77279d8640 1 -- 172.21.15.133:0/1478564720 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77200b0d50 con 0x7f7720142920 2024-04-25T10:02:54.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.608+0000 7f771dffb640 1 -- 172.21.15.133:0/1478564720 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7710055230 con 0x7f7720142920 2024-04-25T10:02:54.616 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:54.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 -- 172.21.15.133:0/1478564720 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f770c066f60 msgr2=0x7f770c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f770c066f60 0x7f770c069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7720067db0 tx=0x7f7714009040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 -- 172.21.15.133:0/1478564720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7720142920 msgr2=0x7f7720139340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7720142920 0x7f7720139340 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f771000cad0 tx=0x7f771000cf90 comp rx=0 tx=0).stop 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 -- 172.21.15.133:0/1478564720 shutdown_connections 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f772015e100 0x7f772013e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7720143560 0x7f772013dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f770c066f60 0x7f770c069420 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 --2- 172.21.15.133:0/1478564720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7720142920 0x7f7720139340 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 -- 172.21.15.133:0/1478564720 >> 172.21.15.133:0/1478564720 conn(0x7f7720066f70 msgr2=0x7f77200afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 -- 172.21.15.133:0/1478564720 shutdown_connections 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.612+0000 7f77279d8640 1 -- 172.21.15.133:0/1478564720 wait complete. 2024-04-25T10:02:54.635 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:54.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53bc68640 1 Processor -- start 2024-04-25T10:02:54.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53bc68640 1 -- start start 2024-04-25T10:02:54.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53bc68640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa534145170 0x7fa53406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53bc68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa53415ced0 0x7fa53415d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53bc68640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5340ce2e0 con 0x7fa53406e4d0 2024-04-25T10:02:54.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53bc68640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53406db00 con 0x7fa534145170 2024-04-25T10:02:54.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53bc68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa534145550 con 0x7fa53415ced0 2024-04-25T10:02:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa539203640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa53406e4d0 0x7fa5340cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:44396/0 (socket says 172.21.15.133:44396) 2024-04-25T10:02:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53a205640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa53415ced0 0x7fa53415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa539203640 1 -- 172.21.15.133:0/1259608366 learned_addr learned my addr 172.21.15.133:0/1259608366 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa539a04640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa534145170 0x7fa53406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa538a02640 1 -- 172.21.15.133:0/1259608366 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598846721 0 0) 0x7fa5340ce2e0 con 0x7fa53406e4d0 2024-04-25T10:02:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa538a02640 1 -- 172.21.15.133:0/1259608366 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa520003530 con 0x7fa53406e4d0 2024-04-25T10:02:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53a205640 1 -- 172.21.15.133:0/1259608366 >> v2:172.21.15.162:3300/0 conn(0x7fa534145170 msgr2=0x7fa53406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53a205640 1 --2- 172.21.15.133:0/1259608366 >> v2:172.21.15.162:3300/0 conn(0x7fa534145170 0x7fa53406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53a205640 1 -- 172.21.15.133:0/1259608366 >> v1:172.21.15.162:6789/0 conn(0x7fa53406e4d0 legacy=0x7fa5340cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa53a205640 1 -- 172.21.15.133:0/1259608366 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53406d4b0 con 0x7fa53415ced0 2024-04-25T10:02:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.752+0000 7fa539a04640 1 --2- 172.21.15.133:0/1259608366 >> v2:172.21.15.162:3300/0 conn(0x7fa534145170 0x7fa53406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa53a205640 1 --2- 172.21.15.133:0/1259608366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa53415ced0 0x7fa53415d360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa530009d90 tx=0x7fa53001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf5ecb80538c6622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa538a02640 1 -- 172.21.15.133:0/1259608366 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300b2070 con 0x7fa53415ced0 2024-04-25T10:02:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa538a02640 1 -- 172.21.15.133:0/1259608366 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5300a5830 con 0x7fa53415ced0 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa538a02640 1 -- 172.21.15.133:0/1259608366 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300a5c00 con 0x7fa53415ced0 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa53bc68640 1 -- 172.21.15.133:0/1259608366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa53415ced0 msgr2=0x7fa53415d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa53bc68640 1 --2- 172.21.15.133:0/1259608366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa53415ced0 0x7fa53415d360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa530009d90 tx=0x7fa53001c970 comp rx=0 tx=0).stop 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa53bc68640 1 -- 172.21.15.133:0/1259608366 shutdown_connections 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa53bc68640 1 --2- 172.21.15.133:0/1259608366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa53415ced0 0x7fa53415d360 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa53bc68640 1 --2- 172.21.15.133:0/1259608366 >> v2:172.21.15.162:3300/0 conn(0x7fa534145170 0x7fa53406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa53bc68640 1 -- 172.21.15.133:0/1259608366 >> 172.21.15.133:0/1259608366 conn(0x7fa534066f70 msgr2=0x7fa53406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa53bc68640 1 -- 172.21.15.133:0/1259608366 shutdown_connections 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa53bc68640 1 -- 172.21.15.133:0/1259608366 wait complete. 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa53bc68640 1 Processor -- start 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.756+0000 7fa53bc68640 1 -- start start 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa53bc68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa53406e4d0 0x7fa534133b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa53bc68640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534145170 0x7fa53406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa53bc68640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53415ced0 0x7fa53406b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa53bc68640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa534168610 con 0x7fa534145170 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa53bc68640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa534168490 con 0x7fa53415ced0 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa53bc68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa534168790 con 0x7fa53406e4d0 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa539203640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534145170 0x7fa53406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa539203640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534145170 0x7fa53406ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47166/0 (socket says 172.21.15.133:47166) 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa539203640 1 -- 172.21.15.133:0/1449414279 learned_addr learned my addr 172.21.15.133:0/1449414279 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa539a04640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa53406e4d0 0x7fa534133b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa53a205640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53415ced0 0x7fa53406b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa539203640 1 -- 172.21.15.133:0/1449414279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa53406e4d0 msgr2=0x7fa534133b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa539203640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa53406e4d0 0x7fa534133b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa539203640 1 -- 172.21.15.133:0/1449414279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53415ced0 msgr2=0x7fa53406b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa539203640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53415ced0 0x7fa53406b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa539203640 1 -- 172.21.15.133:0/1449414279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa534135dc0 con 0x7fa534145170 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa539203640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534145170 0x7fa53406ac90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa524001ec0 tx=0x7fa52400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa52a7fc640 1 -- 172.21.15.133:0/1449414279 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524017070 con 0x7fa534145170 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa52a7fc640 1 -- 172.21.15.133:0/1449414279 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa524005d50 con 0x7fa534145170 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa53bc68640 1 -- 172.21.15.133:0/1449414279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53415fa50 con 0x7fa534145170 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa52a7fc640 1 -- 172.21.15.133:0/1449414279 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524007270 con 0x7fa534145170 2024-04-25T10:02:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa53bc68640 1 -- 172.21.15.133:0/1449414279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa534136340 con 0x7fa534145170 2024-04-25T10:02:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa53bc68640 1 -- 172.21.15.133:0/1449414279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa534068e80 con 0x7fa534145170 2024-04-25T10:02:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa52a7fc640 1 -- 172.21.15.133:0/1449414279 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa52401a070 con 0x7fa534145170 2024-04-25T10:02:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa52a7fc640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa518066f90 0x7fa518069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa52a7fc640 1 -- 172.21.15.133:0/1449414279 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa524084800 con 0x7fa534145170 2024-04-25T10:02:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.760+0000 7fa539a04640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa518066f90 0x7fa518069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.764+0000 7fa539a04640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa518066f90 0x7fa518069450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa5340ae990 tx=0x7fa52c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.764+0000 7fa52a7fc640 1 -- 172.21.15.133:0/1449414279 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5240554d0 con 0x7fa534145170 2024-04-25T10:02:54.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.920+0000 7fa53bc68640 1 -- 172.21.15.133:0/1449414279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa53415c5a0 con 0x7fa534145170 2024-04-25T10:02:54.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa52a7fc640 1 -- 172.21.15.133:0/1449414279 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa524054dd0 con 0x7fa534145170 2024-04-25T10:02:54.929 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:54.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 -- 172.21.15.133:0/1449414279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa518066f90 msgr2=0x7fa518069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa518066f90 0x7fa518069450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa5340ae990 tx=0x7fa52c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 -- 172.21.15.133:0/1449414279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534145170 msgr2=0x7fa53406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534145170 0x7fa53406ac90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa524001ec0 tx=0x7fa52400c960 comp rx=0 tx=0).stop 2024-04-25T10:02:54.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 -- 172.21.15.133:0/1449414279 shutdown_connections 2024-04-25T10:02:54.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa53415ced0 0x7fa53406b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa518066f90 0x7fa518069450 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534145170 0x7fa53406ac90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 --2- 172.21.15.133:0/1449414279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa53406e4d0 0x7fa534133b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 -- 172.21.15.133:0/1449414279 >> 172.21.15.133:0/1449414279 conn(0x7fa534066f70 msgr2=0x7fa5341402f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 -- 172.21.15.133:0/1449414279 shutdown_connections 2024-04-25T10:02:54.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:54.924+0000 7fa53bc68640 1 -- 172.21.15.133:0/1449414279 wait complete. 2024-04-25T10:02:54.948 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:55.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.060+0000 7fd07ac6e640 1 Processor -- start 2024-04-25T10:02:55.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.060+0000 7fd07ac6e640 1 -- start start 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd07ac6e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd074145170 0x7fd07406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd07ac6e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07406e4d0 0x7fd0740cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd07ac6e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0740ce2e0 con 0x7fd0740cdd60 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd07ac6e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07406db00 con 0x7fd074145170 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd07ac6e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd074145550 con 0x7fd07406e4d0 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd078a0a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd074145170 0x7fd07406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd078a0a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd074145170 0x7fd07406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47168/0 (socket says 172.21.15.133:47168) 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd078a0a640 1 -- 172.21.15.133:0/2390495298 learned_addr learned my addr 172.21.15.133:0/2390495298 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd06bfff640 1 --2- 172.21.15.133:0/2390495298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07406e4d0 0x7fd0740cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd078a0a640 1 -- 172.21.15.133:0/2390495298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07406e4d0 msgr2=0x7fd0740cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd078a0a640 1 --2- 172.21.15.133:0/2390495298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07406e4d0 0x7fd0740cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd078a0a640 1 -- 172.21.15.133:0/2390495298 >> v1:172.21.15.162:6789/0 conn(0x7fd0740cdd60 legacy=0x7fd07415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd078a0a640 1 -- 172.21.15.133:0/2390495298 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07406d4b0 con 0x7fd074145170 2024-04-25T10:02:55.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd078a0a640 1 --2- 172.21.15.133:0/2390495298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd074145170 0x7fd07406df90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd070009870 tx=0x7fd07001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620d8ffdc89c0499 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd06b7fe640 1 -- 172.21.15.133:0/2390495298 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0700b0070 con 0x7fd074145170 2024-04-25T10:02:55.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd06b7fe640 1 -- 172.21.15.133:0/2390495298 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0700ab600 con 0x7fd074145170 2024-04-25T10:02:55.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.064+0000 7fd06b7fe640 1 -- 172.21.15.133:0/2390495298 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0700ab9d0 con 0x7fd074145170 2024-04-25T10:02:55.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 -- 172.21.15.133:0/2390495298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd074145170 msgr2=0x7fd07406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 --2- 172.21.15.133:0/2390495298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd074145170 0x7fd07406df90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd070009870 tx=0x7fd07001c520 comp rx=0 tx=0).stop 2024-04-25T10:02:55.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 -- 172.21.15.133:0/2390495298 shutdown_connections 2024-04-25T10:02:55.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 --2- 172.21.15.133:0/2390495298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07406e4d0 0x7fd0740cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 --2- 172.21.15.133:0/2390495298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd074145170 0x7fd07406df90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 -- 172.21.15.133:0/2390495298 >> 172.21.15.133:0/2390495298 conn(0x7fd074066f70 msgr2=0x7fd07406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 -- 172.21.15.133:0/2390495298 shutdown_connections 2024-04-25T10:02:55.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 -- 172.21.15.133:0/2390495298 wait complete. 2024-04-25T10:02:55.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 Processor -- start 2024-04-25T10:02:55.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 -- start start 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07406e4d0 0x7fd07413cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0740cdd60 0x7fd074160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd074145170 0x7fd0741610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd074168490 con 0x7fd074145170 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07415d7f0 con 0x7fd07406e4d0 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd074168610 con 0x7fd0740cdd60 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07920b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd074145170 0x7fd0741610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07920b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd074145170 0x7fd0741610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47170/0 (socket says 172.21.15.133:47170) 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07920b640 1 -- 172.21.15.133:0/681546975 learned_addr learned my addr 172.21.15.133:0/681546975 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd06bfff640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0740cdd60 0x7fd074160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd078a0a640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07406e4d0 0x7fd07413cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07920b640 1 -- 172.21.15.133:0/681546975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0740cdd60 msgr2=0x7fd074160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07920b640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0740cdd60 0x7fd074160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07920b640 1 -- 172.21.15.133:0/681546975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07406e4d0 msgr2=0x7fd07413cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07920b640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07406e4d0 0x7fd07413cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07920b640 1 -- 172.21.15.133:0/681546975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07413bee0 con 0x7fd074145170 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07920b640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd074145170 0x7fd0741610e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00c7b0 tx=0x7fd06c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd0697fa640 1 -- 172.21.15.133:0/681546975 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c004b70 con 0x7fd074145170 2024-04-25T10:02:55.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd0697fa640 1 -- 172.21.15.133:0/681546975 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c004d10 con 0x7fd074145170 2024-04-25T10:02:55.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 -- 172.21.15.133:0/681546975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07413c1c0 con 0x7fd074145170 2024-04-25T10:02:55.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd0697fa640 1 -- 172.21.15.133:0/681546975 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c00aa90 con 0x7fd074145170 2024-04-25T10:02:55.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.068+0000 7fd07ac6e640 1 -- 172.21.15.133:0/681546975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07413c780 con 0x7fd074145170 2024-04-25T10:02:55.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.072+0000 7fd07ac6e640 1 -- 172.21.15.133:0/681546975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd074068e80 con 0x7fd074145170 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.072+0000 7fd0697fa640 1 -- 172.21.15.133:0/681546975 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd06c00ac30 con 0x7fd074145170 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.072+0000 7fd0697fa640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd060066f90 0x7fd060069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.072+0000 7fd0697fa640 1 -- 172.21.15.133:0/681546975 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd06c085720 con 0x7fd074145170 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.072+0000 7fd078a0a640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd060066f90 0x7fd060069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.072+0000 7fd078a0a640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd060066f90 0x7fd060069450 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd0700ad000 tx=0x7fd0700a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.076+0000 7fd0697fa640 1 -- 172.21.15.133:0/681546975 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd06c020070 con 0x7fd074145170 2024-04-25T10:02:55.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.232+0000 7fd07ac6e640 1 -- 172.21.15.133:0/681546975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd074160680 con 0x7fd074145170 2024-04-25T10:02:55.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.232+0000 7fd0697fa640 1 -- 172.21.15.133:0/681546975 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd06c0562a0 con 0x7fd074145170 2024-04-25T10:02:55.240 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:55.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 -- 172.21.15.133:0/681546975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd060066f90 msgr2=0x7fd060069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd060066f90 0x7fd060069450 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd0700ad000 tx=0x7fd0700a6000 comp rx=0 tx=0).stop 2024-04-25T10:02:55.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 -- 172.21.15.133:0/681546975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd074145170 msgr2=0x7fd0741610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd074145170 0x7fd0741610e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00c7b0 tx=0x7fd06c00cc70 comp rx=0 tx=0).stop 2024-04-25T10:02:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 -- 172.21.15.133:0/681546975 shutdown_connections 2024-04-25T10:02:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd060066f90 0x7fd060069450 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd074145170 0x7fd0741610e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0740cdd60 0x7fd074160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 --2- 172.21.15.133:0/681546975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd07406e4d0 0x7fd07413cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 -- 172.21.15.133:0/681546975 >> 172.21.15.133:0/681546975 conn(0x7fd074066f70 msgr2=0x7fd0740b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 -- 172.21.15.133:0/681546975 shutdown_connections 2024-04-25T10:02:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.236+0000 7fd07ac6e640 1 -- 172.21.15.133:0/681546975 wait complete. 2024-04-25T10:02:55.258 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:55.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.364+0000 7f56328ab640 1 Processor -- start 2024-04-25T10:02:55.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.364+0000 7f56328ab640 1 -- start start 2024-04-25T10:02:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f56328ab640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f562c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f56328ab640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f562c0b2150 0x7f562c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f56328ab640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c15ac50 con 0x7f562c15b9b0 2024-04-25T10:02:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f56328ab640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562c0cd180 con 0x7f562c0b2150 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f56328ab640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15aed0 con 0x7f562c143ea0 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f5630e48640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f562c15b9b0 0x7f562c15e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47134/0 (socket says 172.21.15.133:47134) 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f5630e48640 1 -- 172.21.15.133:0/3020275556 learned_addr learned my addr 172.21.15.133:0/3020275556 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207624844 0 0) 0x7f56240014a0 con 0x7f562c15b9b0 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562bfff640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f562c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5618003530 con 0x7f562c15b9b0 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f5623fff640 1 --2- 172.21.15.133:0/3020275556 >> v2:172.21.15.162:3300/0 conn(0x7f562c0b2150 0x7f562c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2831680243 0 0) 0x7f5618003530 con 0x7f562c15b9b0 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 msgr2=0x7f562c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f562c0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 >> v2:172.21.15.162:3300/0 conn(0x7f562c0b2150 msgr2=0x7f562c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 --2- 172.21.15.133:0/3020275556 >> v2:172.21.15.162:3300/0 conn(0x7f562c0b2150 0x7f562c06e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c15b1b0 con 0x7f562c15b9b0 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5624001bd0 con 0x7f562c15b9b0 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 >> v1:172.21.15.133:6789/0 conn(0x7f562c15b9b0 legacy=0x7f562c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039c0 0x7f5618003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5618004810 0x7f5618004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f56180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0cd180 con 0x7f56180039c0 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15aed0 con 0x7f5618004810 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561800ba80 con 0x7f562c143ea0 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562bfff640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039c0 0x7f5618003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f5623fff640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5618004810 0x7f5618004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562bfff640 1 -- 172.21.15.133:0/3020275556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 msgr2=0x7f56180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562bfff640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f56180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562bfff640 1 -- 172.21.15.133:0/3020275556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5618004810 msgr2=0x7f5618004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562bfff640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5618004810 0x7f5618004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562bfff640 1 -- 172.21.15.133:0/3020275556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5618003530 con 0x7f56180039c0 2024-04-25T10:02:55.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f5623fff640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5618004810 0x7f5618004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562bfff640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039c0 0x7f5618003de0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f561c00b9e0 tx=0x7f561c01c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a31b128e00f3ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0b3070 con 0x7f56180039c0 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c0aa050 con 0x7f56180039c0 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.368+0000 7f562b7fe640 1 -- 172.21.15.133:0/3020275556 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0a8030 con 0x7f56180039c0 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- 172.21.15.133:0/3020275556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039c0 msgr2=0x7f5618003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 --2- 172.21.15.133:0/3020275556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039c0 0x7f5618003de0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f561c00b9e0 tx=0x7f561c01c660 comp rx=0 tx=0).stop 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f5630e48640 1 -- 172.21.15.133:0/3020275556 reap_dead start 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- 172.21.15.133:0/3020275556 shutdown_connections 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- 172.21.15.133:0/3020275556 >> 172.21.15.133:0/3020275556 conn(0x7f562c066f70 msgr2=0x7f562c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- 172.21.15.133:0/3020275556 shutdown_connections 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- 172.21.15.133:0/3020275556 wait complete. 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 Processor -- start 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- start start 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56180039c0 0x7f562c135290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f562c141860 0x7f562c1663d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c167d70 0x7f562c140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15ece0 con 0x7f562c141860 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15eb60 con 0x7f56180039c0 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15ee60 con 0x7f562c167d70 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f5623fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f562c141860 0x7f562c1663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f5623fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f562c141860 0x7f562c1663d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47192/0 (socket says 172.21.15.133:47192) 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f5623fff640 1 -- 172.21.15.133:0/687035573 learned_addr learned my addr 172.21.15.133:0/687035573 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f5630e48640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c167d70 0x7f562c140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f562bfff640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56180039c0 0x7f562c135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f5623fff640 1 -- 172.21.15.133:0/687035573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c167d70 msgr2=0x7f562c140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f5623fff640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c167d70 0x7f562c140b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f5623fff640 1 -- 172.21.15.133:0/687035573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56180039c0 msgr2=0x7f562c135290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f5623fff640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56180039c0 0x7f562c135290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f5623fff640 1 -- 172.21.15.133:0/687035573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c166cd0 con 0x7f562c141860 2024-04-25T10:02:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f5623fff640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f562c141860 0x7f562c1663d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f561400e810 tx=0x7f561400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56297fa640 1 -- 172.21.15.133:0/687035573 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614017070 con 0x7f562c141860 2024-04-25T10:02:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56297fa640 1 -- 172.21.15.133:0/687035573 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561400c8d0 con 0x7f562c141860 2024-04-25T10:02:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- 172.21.15.133:0/687035573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c166fb0 con 0x7f562c141860 2024-04-25T10:02:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56297fa640 1 -- 172.21.15.133:0/687035573 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561400f040 con 0x7f562c141860 2024-04-25T10:02:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- 172.21.15.133:0/687035573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c167570 con 0x7f562c141860 2024-04-25T10:02:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.372+0000 7f56328ab640 1 -- 172.21.15.133:0/687035573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c068e80 con 0x7f562c141860 2024-04-25T10:02:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.376+0000 7f56297fa640 1 -- 172.21.15.133:0/687035573 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f561401a070 con 0x7f562c141860 2024-04-25T10:02:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.376+0000 7f56297fa640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f560c066f60 0x7f560c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.376+0000 7f562bfff640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f560c066f60 0x7f560c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.376+0000 7f56297fa640 1 -- 172.21.15.133:0/687035573 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5614085070 con 0x7f562c141860 2024-04-25T10:02:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.376+0000 7f562bfff640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f560c066f60 0x7f560c069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f561c0b0000 tx=0x7f561c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.380+0000 7f56297fa640 1 -- 172.21.15.133:0/687035573 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5614055be0 con 0x7f562c141860 2024-04-25T10:02:55.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.524+0000 7f56328ab640 1 -- 172.21.15.133:0/687035573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f562c1357d0 con 0x7f562c141860 2024-04-25T10:02:55.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.524+0000 7f56297fa640 1 -- 172.21.15.133:0/687035573 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f56140554e0 con 0x7f562c141860 2024-04-25T10:02:55.532 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:55.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 -- 172.21.15.133:0/687035573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f560c066f60 msgr2=0x7f560c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f560c066f60 0x7f560c069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f561c0b0000 tx=0x7f561c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 -- 172.21.15.133:0/687035573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f562c141860 msgr2=0x7f562c1663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f562c141860 0x7f562c1663d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f561400e810 tx=0x7f561400ecd0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 -- 172.21.15.133:0/687035573 shutdown_connections 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c167d70 0x7f562c140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f560c066f60 0x7f560c069420 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f562c141860 0x7f562c1663d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 --2- 172.21.15.133:0/687035573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56180039c0 0x7f562c135290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 -- 172.21.15.133:0/687035573 >> 172.21.15.133:0/687035573 conn(0x7f562c066f70 msgr2=0x7f562c143060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 -- 172.21.15.133:0/687035573 shutdown_connections 2024-04-25T10:02:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.528+0000 7f56328ab640 1 -- 172.21.15.133:0/687035573 wait complete. 2024-04-25T10:02:55.550 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:55.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08d66e2640 1 Processor -- start 2024-04-25T10:02:55.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08d66e2640 1 -- start start 2024-04-25T10:02:55.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08d66e2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f08d00b2150 0x7f08d006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08d66e2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d015ac50 con 0x7f08d015b9b0 2024-04-25T10:02:55.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08d66e2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d00cd180 con 0x7f08d0143ea0 2024-04-25T10:02:55.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08d66e2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d00ccf20 con 0x7f08d00b2150 2024-04-25T10:02:55.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08cf7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f08d00b2150 0x7f08d006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08d4c7f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f08d015b9b0 0x7f08d00b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47136/0 (socket says 172.21.15.133:47136) 2024-04-25T10:02:55.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08d4c7f640 1 -- 172.21.15.133:0/3687492940 learned_addr learned my addr 172.21.15.133:0/3687492940 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:55.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08cf7fe640 1 -- 172.21.15.133:0/3687492940 >> v1:172.21.15.162:6789/0 conn(0x7f08d0143ea0 legacy=0x7f08d00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08cf7fe640 1 -- 172.21.15.133:0/3687492940 >> v1:172.21.15.133:6789/0 conn(0x7f08d015b9b0 legacy=0x7f08d00b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08cf7fe640 1 -- 172.21.15.133:0/3687492940 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d015aed0 con 0x7f08d00b2150 2024-04-25T10:02:55.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.660+0000 7f08ceffd640 1 -- 172.21.15.133:0/3687492940 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678705784 0 0) 0x7f08c00014a0 con 0x7f08d015b9b0 2024-04-25T10:02:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08cf7fe640 1 --2- 172.21.15.133:0/3687492940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08d00b2150 0x7f08d006e3b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f08b80097c0 tx=0x7f08b801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c9ea4b0b437739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08ceffd640 1 -- 172.21.15.133:0/3687492940 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b80b0070 con 0x7f08d00b2150 2024-04-25T10:02:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08ceffd640 1 -- 172.21.15.133:0/3687492940 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b80ab600 con 0x7f08d00b2150 2024-04-25T10:02:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08ceffd640 1 -- 172.21.15.133:0/3687492940 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b80ab9d0 con 0x7f08d00b2150 2024-04-25T10:02:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 -- 172.21.15.133:0/3687492940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08d00b2150 msgr2=0x7f08d006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 --2- 172.21.15.133:0/3687492940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08d00b2150 0x7f08d006e3b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f08b80097c0 tx=0x7f08b801c410 comp rx=0 tx=0).stop 2024-04-25T10:02:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 -- 172.21.15.133:0/3687492940 shutdown_connections 2024-04-25T10:02:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 --2- 172.21.15.133:0/3687492940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08d00b2150 0x7f08d006e3b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 -- 172.21.15.133:0/3687492940 >> 172.21.15.133:0/3687492940 conn(0x7f08d0066f70 msgr2=0x7f08d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 -- 172.21.15.133:0/3687492940 shutdown_connections 2024-04-25T10:02:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 -- 172.21.15.133:0/3687492940 wait complete. 2024-04-25T10:02:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 Processor -- start 2024-04-25T10:02:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 -- start start 2024-04-25T10:02:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08d00b2150 0x7f08d01361d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08d0143ea0 0x7f08d01397e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08d015b9b0 0x7f08d0139d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d01685f0 con 0x7f08d015b9b0 2024-04-25T10:02:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d006ed00 con 0x7f08d00b2150 2024-04-25T10:02:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d66e2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d0168770 con 0x7f08d0143ea0 2024-04-25T10:02:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d4c7f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08d015b9b0 0x7f08d0139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d4c7f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08d015b9b0 0x7f08d0139d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47214/0 (socket says 172.21.15.133:47214) 2024-04-25T10:02:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d4c7f640 1 -- 172.21.15.133:0/1093927502 learned_addr learned my addr 172.21.15.133:0/1093927502 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08cffff640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08d00b2150 0x7f08d01361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08cf7fe640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08d0143ea0 0x7f08d01397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d4c7f640 1 -- 172.21.15.133:0/1093927502 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08d0143ea0 msgr2=0x7f08d01397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d4c7f640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08d0143ea0 0x7f08d01397e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d4c7f640 1 -- 172.21.15.133:0/1093927502 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08d00b2150 msgr2=0x7f08d01361d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d4c7f640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08d00b2150 0x7f08d01361d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08d4c7f640 1 -- 172.21.15.133:0/1093927502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d0138280 con 0x7f08d015b9b0 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.664+0000 7f08cffff640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08d00b2150 0x7f08d01361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08d4c7f640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08d015b9b0 0x7f08d0139d20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f08c000c6d0 tx=0x7f08c000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08ccff9640 1 -- 172.21.15.133:0/1093927502 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0018070 con 0x7f08d015b9b0 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08ccff9640 1 -- 172.21.15.133:0/1093927502 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c0004d70 con 0x7f08d015b9b0 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08d66e2640 1 -- 172.21.15.133:0/1093927502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d0137600 con 0x7f08d015b9b0 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08ccff9640 1 -- 172.21.15.133:0/1093927502 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c000a880 con 0x7f08d015b9b0 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08d66e2640 1 -- 172.21.15.133:0/1093927502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d0137bc0 con 0x7f08d015b9b0 2024-04-25T10:02:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08d66e2640 1 -- 172.21.15.133:0/1093927502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d0068e80 con 0x7f08d015b9b0 2024-04-25T10:02:55.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08ccff9640 1 -- 172.21.15.133:0/1093927502 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08c001b070 con 0x7f08d015b9b0 2024-04-25T10:02:55.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08ccff9640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08b0066f60 0x7f08b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08ccff9640 1 -- 172.21.15.133:0/1093927502 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f08c0084b50 con 0x7f08d015b9b0 2024-04-25T10:02:55.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08cffff640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08b0066f60 0x7f08b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.668+0000 7f08cffff640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08b0066f60 0x7f08b0069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f08c8000f70 tx=0x7f08c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.672+0000 7f08ccff9640 1 -- 172.21.15.133:0/1093927502 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08d0138280 con 0x7f08d015b9b0 2024-04-25T10:02:55.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08d66e2640 1 -- 172.21.15.133:0/1093927502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f08d01412f0 con 0x7f08d015b9b0 2024-04-25T10:02:55.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08ccff9640 1 -- 172.21.15.133:0/1093927502 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f08d015ac50 con 0x7f08d015b9b0 2024-04-25T10:02:55.850 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:02:55.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08d66e2640 1 -- 172.21.15.133:0/1093927502 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08b0066f60 msgr2=0x7f08b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08d66e2640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08b0066f60 0x7f08b0069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f08c8000f70 tx=0x7f08c8009040 comp rx=0 tx=0).stop 2024-04-25T10:02:55.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08d66e2640 1 -- 172.21.15.133:0/1093927502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08d015b9b0 msgr2=0x7f08d0139d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08d66e2640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08d015b9b0 0x7f08d0139d20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f08c000c6d0 tx=0x7f08c000cb90 comp rx=0 tx=0).stop 2024-04-25T10:02:55.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08d66e2640 1 -- 172.21.15.133:0/1093927502 shutdown_connections 2024-04-25T10:02:55.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08d66e2640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08b0066f60 0x7f08b0069420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08d66e2640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08d015b9b0 0x7f08d0139d20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08d66e2640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08d0143ea0 0x7f08d01397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08d66e2640 1 --2- 172.21.15.133:0/1093927502 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08d00b2150 0x7f08d01361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.844+0000 7f08d66e2640 1 -- 172.21.15.133:0/1093927502 >> 172.21.15.133:0/1093927502 conn(0x7f08d0066f70 msgr2=0x7f08d015ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.848+0000 7f08d66e2640 1 -- 172.21.15.133:0/1093927502 shutdown_connections 2024-04-25T10:02:55.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:55.848+0000 7f08d66e2640 1 -- 172.21.15.133:0/1093927502 wait complete. 2024-04-25T10:02:55.870 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi162.front.sepia.ceph.com... 2024-04-25T10:02:55.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:55.870 DEBUG:teuthology.orchestra.run.smithi162:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:55.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:02:55.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:55.897 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:02:55.956 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:02:55.956 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:55.963 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:02:55.963 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:56.010 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:56.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.180+0000 7f53511b0640 1 Processor -- start 2024-04-25T10:02:56.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.180+0000 7f53511b0640 1 -- start start 2024-04-25T10:02:56.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.180+0000 7f53511b0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f534c158090 0x7f534c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.180+0000 7f53511b0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534c15e1b0 con 0x7f534c15ec50 2024-04-25T10:02:56.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.180+0000 7f53511b0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534c0b4cb0 con 0x7f534c06b3a0 2024-04-25T10:02:56.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.180+0000 7f53511b0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534c15df50 con 0x7f534c158090 2024-04-25T10:02:56.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.180+0000 7f534ad9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f534c158090 0x7f534c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f534ad9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f534c158090 0x7f534c0b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54794/0 (socket says 172.21.15.133:54794) 2024-04-25T10:02:56.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f534ad9d640 1 -- 172.21.15.133:0/1395741637 learned_addr learned my addr 172.21.15.133:0/1395741637 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:56.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f534ad9d640 1 -- 172.21.15.133:0/1395741637 >> v1:172.21.15.162:6789/0 conn(0x7f534c06b3a0 legacy=0x7f534c161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f534ad9d640 1 -- 172.21.15.133:0/1395741637 >> v1:172.21.15.133:6789/0 conn(0x7f534c15ec50 legacy=0x7f534c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f534ad9d640 1 -- 172.21.15.133:0/1395741637 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c0b49d0 con 0x7f534c158090 2024-04-25T10:02:56.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f534ad9d640 1 --2- 172.21.15.133:0/1395741637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f534c158090 0x7f534c0b4180 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f533c009870 tx=0x7f533c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86514234e86097a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f5349d9b640 1 -- 172.21.15.133:0/1395741637 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0b0070 con 0x7f534c158090 2024-04-25T10:02:56.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f5349d9b640 1 -- 172.21.15.133:0/1395741637 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c0ab600 con 0x7f534c158090 2024-04-25T10:02:56.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f5349d9b640 1 -- 172.21.15.133:0/1395741637 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0ab9d0 con 0x7f534c158090 2024-04-25T10:02:56.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f53511b0640 1 -- 172.21.15.133:0/1395741637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f534c158090 msgr2=0x7f534c0b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f53511b0640 1 --2- 172.21.15.133:0/1395741637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f534c158090 0x7f534c0b4180 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f533c009870 tx=0x7f533c01c520 comp rx=0 tx=0).stop 2024-04-25T10:02:56.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f53511b0640 1 -- 172.21.15.133:0/1395741637 shutdown_connections 2024-04-25T10:02:56.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f53511b0640 1 --2- 172.21.15.133:0/1395741637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f534c158090 0x7f534c0b4180 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f53511b0640 1 -- 172.21.15.133:0/1395741637 >> 172.21.15.133:0/1395741637 conn(0x7f534c0b2370 msgr2=0x7f534c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f53511b0640 1 -- 172.21.15.133:0/1395741637 shutdown_connections 2024-04-25T10:02:56.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.184+0000 7f53511b0640 1 -- 172.21.15.133:0/1395741637 wait complete. 2024-04-25T10:02:56.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53511b0640 1 Processor -- start 2024-04-25T10:02:56.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53511b0640 1 -- start start 2024-04-25T10:02:56.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53511b0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f534c06b3a0 0x7f534c0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53511b0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f534c158090 0x7f534c0caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53511b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f534c15ec50 0x7f534c0caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53511b0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c16b890 con 0x7f534c06b3a0 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53511b0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c16b710 con 0x7f534c158090 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53511b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534c16ba10 con 0x7f534c15ec50 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f534ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f534c06b3a0 0x7f534c0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f534ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f534c06b3a0 0x7f534c0d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54796/0 (socket says 172.21.15.133:54796) 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f534ad9d640 1 -- 172.21.15.133:0/31674453 learned_addr learned my addr 172.21.15.133:0/31674453 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f534a59c640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f534c158090 0x7f534c0caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f534b59e640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f534c15ec50 0x7f534c0caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f534ad9d640 1 -- 172.21.15.133:0/31674453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f534c15ec50 msgr2=0x7f534c0caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f534ad9d640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f534c15ec50 0x7f534c0caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f534ad9d640 1 -- 172.21.15.133:0/31674453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f534c158090 msgr2=0x7f534c0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f534ad9d640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f534c158090 0x7f534c0caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f534ad9d640 1 -- 172.21.15.133:0/31674453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c1542a0 con 0x7f534c06b3a0 2024-04-25T10:02:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f534ad9d640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f534c06b3a0 0x7f534c0d0210 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f533c01abd0 tx=0x7f533c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53337fe640 1 -- 172.21.15.133:0/31674453 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0ba020 con 0x7f534c06b3a0 2024-04-25T10:02:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53337fe640 1 -- 172.21.15.133:0/31674453 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c0b6620 con 0x7f534c06b3a0 2024-04-25T10:02:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53337fe640 1 -- 172.21.15.133:0/31674453 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0b0040 con 0x7f534c06b3a0 2024-04-25T10:02:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53511b0640 1 -- 172.21.15.133:0/31674453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534c154580 con 0x7f534c06b3a0 2024-04-25T10:02:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53511b0640 1 -- 172.21.15.133:0/31674453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c14eb00 con 0x7f534c06b3a0 2024-04-25T10:02:56.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.188+0000 7f53511b0640 1 -- 172.21.15.133:0/31674453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534c067c70 con 0x7f534c06b3a0 2024-04-25T10:02:56.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.192+0000 7f53337fe640 1 -- 172.21.15.133:0/31674453 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f533c0bd020 con 0x7f534c06b3a0 2024-04-25T10:02:56.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.192+0000 7f53337fe640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5338066f60 0x7f5338069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.192+0000 7f53337fe640 1 -- 172.21.15.133:0/31674453 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f533c11f6f0 con 0x7f534c06b3a0 2024-04-25T10:02:56.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.192+0000 7f534a59c640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5338066f60 0x7f5338069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.192+0000 7f534a59c640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5338066f60 0x7f5338069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f534c0cd5e0 tx=0x7f5340009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.196+0000 7f53337fe640 1 -- 172.21.15.133:0/31674453 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f533c0c4030 con 0x7f534c06b3a0 2024-04-25T10:02:56.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.340+0000 7f53511b0640 1 -- 172.21.15.133:0/31674453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f534c14e5d0 con 0x7f534c06b3a0 2024-04-25T10:02:56.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.344+0000 7f53337fe640 1 -- 172.21.15.133:0/31674453 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f533c016110 con 0x7f534c06b3a0 2024-04-25T10:02:56.351 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:56.351 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:56.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 -- 172.21.15.133:0/31674453 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5338066f60 msgr2=0x7f5338069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5338066f60 0x7f5338069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f534c0cd5e0 tx=0x7f5340009040 comp rx=0 tx=0).stop 2024-04-25T10:02:56.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 -- 172.21.15.133:0/31674453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f534c06b3a0 msgr2=0x7f534c0d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f534c06b3a0 0x7f534c0d0210 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f533c01abd0 tx=0x7f533c01ad60 comp rx=0 tx=0).stop 2024-04-25T10:02:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 -- 172.21.15.133:0/31674453 shutdown_connections 2024-04-25T10:02:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f534c15ec50 0x7f534c0caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f534c158090 0x7f534c0caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5338066f60 0x7f5338069420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 --2- 172.21.15.133:0/31674453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f534c06b3a0 0x7f534c0d0210 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 -- 172.21.15.133:0/31674453 >> 172.21.15.133:0/31674453 conn(0x7f534c0b2370 msgr2=0x7f534c15d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 -- 172.21.15.133:0/31674453 shutdown_connections 2024-04-25T10:02:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.348+0000 7f53511b0640 1 -- 172.21.15.133:0/31674453 wait complete. 2024-04-25T10:02:56.369 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:56.370 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:56.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f07315640 1 Processor -- start 2024-04-25T10:02:56.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f07315640 1 -- start start 2024-04-25T10:02:56.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f07315640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3f0015d9a0 0x7f3f000d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f07315640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3f000ce600 0x7f3f000cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f07315640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0015ebd0 con 0x7f3f000d1920 2024-04-25T10:02:56.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f07315640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0015eeb0 con 0x7f3f000ce600 2024-04-25T10:02:56.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f07315640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f000d0330 con 0x7f3f0015d9a0 2024-04-25T10:02:56.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f048b0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3f000ce600 0x7f3f000cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f048b0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3f000ce600 0x7f3f000cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54804/0 (socket says 172.21.15.133:54804) 2024-04-25T10:02:56.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f048b0640 1 -- 172.21.15.133:0/2217789794 learned_addr learned my addr 172.21.15.133:0/2217789794 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:56.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f050b1640 1 --2- 172.21.15.133:0/2217789794 >> v2:172.21.15.162:3300/0 conn(0x7f3f0015d9a0 0x7f3f000d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f048b0640 1 -- 172.21.15.133:0/2217789794 >> v2:172.21.15.162:3300/0 conn(0x7f3f0015d9a0 msgr2=0x7f3f000d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f048b0640 1 --2- 172.21.15.133:0/2217789794 >> v2:172.21.15.162:3300/0 conn(0x7f3f0015d9a0 0x7f3f000d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f048b0640 1 -- 172.21.15.133:0/2217789794 >> v1:172.21.15.133:6789/0 conn(0x7f3f000d1920 legacy=0x7f3f000d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.476+0000 7f3f048b0640 1 -- 172.21.15.133:0/2217789794 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f000d0610 con 0x7f3f000ce600 2024-04-25T10:02:56.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f048b0640 1 --2- 172.21.15.133:0/2217789794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f000ce600 0x7f3f000cea90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3f000d4750 tx=0x7f3ef401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937691d3cce80616 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3efffff640 1 -- 172.21.15.133:0/2217789794 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40b2070 con 0x7f3f000ce600 2024-04-25T10:02:56.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3efffff640 1 -- 172.21.15.133:0/2217789794 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef40a9070 con 0x7f3f000ce600 2024-04-25T10:02:56.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3efffff640 1 -- 172.21.15.133:0/2217789794 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40ad880 con 0x7f3f000ce600 2024-04-25T10:02:56.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 -- 172.21.15.133:0/2217789794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f000ce600 msgr2=0x7f3f000cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 --2- 172.21.15.133:0/2217789794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f000ce600 0x7f3f000cea90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3f000d4750 tx=0x7f3ef401c490 comp rx=0 tx=0).stop 2024-04-25T10:02:56.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 -- 172.21.15.133:0/2217789794 shutdown_connections 2024-04-25T10:02:56.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 --2- 172.21.15.133:0/2217789794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f000ce600 0x7f3f000cea90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 --2- 172.21.15.133:0/2217789794 >> v2:172.21.15.162:3300/0 conn(0x7f3f0015d9a0 0x7f3f000d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 -- 172.21.15.133:0/2217789794 >> 172.21.15.133:0/2217789794 conn(0x7f3f0006bf80 msgr2=0x7f3f0006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 -- 172.21.15.133:0/2217789794 shutdown_connections 2024-04-25T10:02:56.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 -- 172.21.15.133:0/2217789794 wait complete. 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 Processor -- start 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 -- start start 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f000ce600 0x7f3f00159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f000d1920 0x7f3f001541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f0015d9a0 0x7f3f000b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000b9820 con 0x7f3f000ce600 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000b96a0 con 0x7f3f000d1920 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0015e590 con 0x7f3f0015d9a0 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f050b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f000ce600 0x7f3f00159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f050b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f000ce600 0x7f3f00159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54818/0 (socket says 172.21.15.133:54818) 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f050b1640 1 -- 172.21.15.133:0/3413232851 learned_addr learned my addr 172.21.15.133:0/3413232851 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:56.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f048b0640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f000d1920 0x7f3f001541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f050b1640 1 -- 172.21.15.133:0/3413232851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f0015d9a0 msgr2=0x7f3f000b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f050b1640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f0015d9a0 0x7f3f000b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f050b1640 1 -- 172.21.15.133:0/3413232851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f000d1920 msgr2=0x7f3f001541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f050b1640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f000d1920 0x7f3f001541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f050b1640 1 -- 172.21.15.133:0/3413232851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f00154730 con 0x7f3f000ce600 2024-04-25T10:02:56.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f050b1640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f000ce600 0x7f3f00159700 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800d5f0 tx=0x7f3ef800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3efdffb640 1 -- 172.21.15.133:0/3413232851 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80125f0 con 0x7f3f000ce600 2024-04-25T10:02:56.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3efdffb640 1 -- 172.21.15.133:0/3413232851 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef8019070 con 0x7f3f000ce600 2024-04-25T10:02:56.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 -- 172.21.15.133:0/3413232851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f000b4e90 con 0x7f3f000ce600 2024-04-25T10:02:56.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3efdffb640 1 -- 172.21.15.133:0/3413232851 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80129b0 con 0x7f3f000ce600 2024-04-25T10:02:56.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.480+0000 7f3f07315640 1 -- 172.21.15.133:0/3413232851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f00158f00 con 0x7f3f000ce600 2024-04-25T10:02:56.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.484+0000 7f3f07315640 1 -- 172.21.15.133:0/3413232851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f00068e80 con 0x7f3f000ce600 2024-04-25T10:02:56.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.484+0000 7f3efdffb640 1 -- 172.21.15.133:0/3413232851 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ef8012b50 con 0x7f3f000ce600 2024-04-25T10:02:56.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.484+0000 7f3efdffb640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eec066f60 0x7f3eec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.484+0000 7f3efdffb640 1 -- 172.21.15.133:0/3413232851 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3ef8085870 con 0x7f3f000ce600 2024-04-25T10:02:56.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.484+0000 7f3f048b0640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eec066f60 0x7f3eec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.484+0000 7f3f048b0640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eec066f60 0x7f3eec069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40af000 tx=0x7f3ef4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.488+0000 7f3efdffb640 1 -- 172.21.15.133:0/3413232851 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ef8056480 con 0x7f3f000ce600 2024-04-25T10:02:56.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.644+0000 7f3f07315640 1 -- 172.21.15.133:0/3413232851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3f00167e50 con 0x7f3f000ce600 2024-04-25T10:02:56.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.644+0000 7f3efdffb640 1 -- 172.21.15.133:0/3413232851 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ef8055d80 con 0x7f3f000ce600 2024-04-25T10:02:56.651 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 -- 172.21.15.133:0/3413232851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eec066f60 msgr2=0x7f3eec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eec066f60 0x7f3eec069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40af000 tx=0x7f3ef4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 -- 172.21.15.133:0/3413232851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f000ce600 msgr2=0x7f3f00159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f000ce600 0x7f3f00159700 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800d5f0 tx=0x7f3ef800dab0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 -- 172.21.15.133:0/3413232851 shutdown_connections 2024-04-25T10:02:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f0015d9a0 0x7f3f000b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f000d1920 0x7f3f001541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3eec066f60 0x7f3eec069420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 --2- 172.21.15.133:0/3413232851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f000ce600 0x7f3f00159700 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 -- 172.21.15.133:0/3413232851 >> 172.21.15.133:0/3413232851 conn(0x7f3f0006bf80 msgr2=0x7f3f000b78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 -- 172.21.15.133:0/3413232851 shutdown_connections 2024-04-25T10:02:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.648+0000 7f3f07315640 1 -- 172.21.15.133:0/3413232851 wait complete. 2024-04-25T10:02:56.669 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:56.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.772+0000 7fa1e8d7d640 1 Processor -- start 2024-04-25T10:02:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.772+0000 7fa1e8d7d640 1 -- start start 2024-04-25T10:02:56.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e8d7d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa1e415e010 0x7fa1e40d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e8d7d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa1e40b4460 0x7fa1e40ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e8d7d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e40cfba0 con 0x7fa1e40d0670 2024-04-25T10:02:56.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e8d7d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e40d11c0 con 0x7fa1e40b4460 2024-04-25T10:02:56.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e8d7d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e40d2300 con 0x7fa1e415e010 2024-04-25T10:02:56.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e1d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa1e40d0670 0x7fa1e40d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35808/0 (socket says 172.21.15.133:35808) 2024-04-25T10:02:56.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e1d9b640 1 -- 172.21.15.133:0/2201621213 learned_addr learned my addr 172.21.15.133:0/2201621213 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:56.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e2d9d640 1 --2- 172.21.15.133:0/2201621213 >> v2:172.21.15.133:3300/0 conn(0x7fa1e40b4460 0x7fa1e40ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e259c640 1 --2- 172.21.15.133:0/2201621213 >> v2:172.21.15.162:3300/0 conn(0x7fa1e415e010 0x7fa1e40d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e2d9d640 1 -- 172.21.15.133:0/2201621213 >> v2:172.21.15.162:3300/0 conn(0x7fa1e415e010 msgr2=0x7fa1e40d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e2d9d640 1 --2- 172.21.15.133:0/2201621213 >> v2:172.21.15.162:3300/0 conn(0x7fa1e415e010 0x7fa1e40d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e2d9d640 1 -- 172.21.15.133:0/2201621213 >> v1:172.21.15.133:6789/0 conn(0x7fa1e40d0670 legacy=0x7fa1e40d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e2d9d640 1 -- 172.21.15.133:0/2201621213 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e40d25e0 con 0x7fa1e40b4460 2024-04-25T10:02:56.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e2d9d640 1 --2- 172.21.15.133:0/2201621213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e40b4460 0x7fa1e40ce510 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4069160 tx=0x7fa1dc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdb4032bf3c0c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e159a640 1 -- 172.21.15.133:0/2201621213 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0b2070 con 0x7fa1e40b4460 2024-04-25T10:02:56.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e159a640 1 -- 172.21.15.133:0/2201621213 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc0a5810 con 0x7fa1e40b4460 2024-04-25T10:02:56.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.776+0000 7fa1e159a640 1 -- 172.21.15.133:0/2201621213 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0a5be0 con 0x7fa1e40b4460 2024-04-25T10:02:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/2201621213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e40b4460 msgr2=0x7fa1e40ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 --2- 172.21.15.133:0/2201621213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e40b4460 0x7fa1e40ce510 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4069160 tx=0x7fa1dc01c930 comp rx=0 tx=0).stop 2024-04-25T10:02:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/2201621213 shutdown_connections 2024-04-25T10:02:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 --2- 172.21.15.133:0/2201621213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e40b4460 0x7fa1e40ce510 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 --2- 172.21.15.133:0/2201621213 >> v2:172.21.15.162:3300/0 conn(0x7fa1e415e010 0x7fa1e40d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/2201621213 >> 172.21.15.133:0/2201621213 conn(0x7fa1e408af60 msgr2=0x7fa1e40d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/2201621213 shutdown_connections 2024-04-25T10:02:56.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/2201621213 wait complete. 2024-04-25T10:02:56.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 Processor -- start 2024-04-25T10:02:56.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 -- start start 2024-04-25T10:02:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1e40b4460 0x7fa1e40b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1e40d0670 0x7fa1e414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e415e010 0x7fa1e414d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e40cf670 con 0x7fa1e415e010 2024-04-25T10:02:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e40cf4f0 con 0x7fa1e40d0670 2024-04-25T10:02:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e40cf7f0 con 0x7fa1e40b4460 2024-04-25T10:02:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e415e010 0x7fa1e414d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e415e010 0x7fa1e414d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54832/0 (socket says 172.21.15.133:54832) 2024-04-25T10:02:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e2d9d640 1 -- 172.21.15.133:0/1526963141 learned_addr learned my addr 172.21.15.133:0/1526963141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e2d9d640 1 -- 172.21.15.133:0/1526963141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1e40b4460 msgr2=0x7fa1e40b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e2d9d640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1e40b4460 0x7fa1e40b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e2d9d640 1 -- 172.21.15.133:0/1526963141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1e40d0670 msgr2=0x7fa1e414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e2d9d640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1e40d0670 0x7fa1e414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e2d9d640 1 -- 172.21.15.133:0/1526963141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e414e2b0 con 0x7fa1e415e010 2024-04-25T10:02:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e2d9d640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e415e010 0x7fa1e414d3a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0b0040 tx=0x7fa1dc0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1d2ffd640 1 -- 172.21.15.133:0/1526963141 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0b2050 con 0x7fa1e415e010 2024-04-25T10:02:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1d2ffd640 1 -- 172.21.15.133:0/1526963141 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc0be070 con 0x7fa1e415e010 2024-04-25T10:02:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1d2ffd640 1 -- 172.21.15.133:0/1526963141 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0ad870 con 0x7fa1e415e010 2024-04-25T10:02:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/1526963141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e414e550 con 0x7fa1e415e010 2024-04-25T10:02:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.780+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/1526963141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e414eb10 con 0x7fa1e415e010 2024-04-25T10:02:56.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.784+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/1526963141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e4067c70 con 0x7fa1e415e010 2024-04-25T10:02:56.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.784+0000 7fa1d2ffd640 1 -- 172.21.15.133:0/1526963141 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1dc0ada10 con 0x7fa1e415e010 2024-04-25T10:02:56.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.784+0000 7fa1d2ffd640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1c4066f60 0x7fa1c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.784+0000 7fa1d2ffd640 1 -- 172.21.15.133:0/1526963141 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa1dc120610 con 0x7fa1e415e010 2024-04-25T10:02:56.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.784+0000 7fa1e259c640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1c4066f60 0x7fa1c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.784+0000 7fa1e259c640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1c4066f60 0x7fa1c4069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80098f0 tx=0x7fa1d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.788+0000 7fa1d2ffd640 1 -- 172.21.15.133:0/1526963141 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1dc016520 con 0x7fa1e415e010 2024-04-25T10:02:56.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.944+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/1526963141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1e40b2a50 con 0x7fa1e415e010 2024-04-25T10:02:56.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.944+0000 7fa1d2ffd640 1 -- 172.21.15.133:0/1526963141 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1dc0f1210 con 0x7fa1e415e010 2024-04-25T10:02:56.953 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:56.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/1526963141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1c4066f60 msgr2=0x7fa1c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1c4066f60 0x7fa1c4069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80098f0 tx=0x7fa1d8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/1526963141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e415e010 msgr2=0x7fa1e414d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e415e010 0x7fa1e414d3a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0b0040 tx=0x7fa1dc0196d0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/1526963141 shutdown_connections 2024-04-25T10:02:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1c4066f60 0x7fa1c4069420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1e415e010 0x7fa1e414d3a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1e40d0670 0x7fa1e414dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 --2- 172.21.15.133:0/1526963141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1e40b4460 0x7fa1e40b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/1526963141 >> 172.21.15.133:0/1526963141 conn(0x7fa1e408af60 msgr2=0x7fa1e406cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/1526963141 shutdown_connections 2024-04-25T10:02:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:56.948+0000 7fa1e8d7d640 1 -- 172.21.15.133:0/1526963141 wait complete. 2024-04-25T10:02:56.972 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feb04f8f640 1 Processor -- start 2024-04-25T10:02:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feb04f8f640 1 -- start start 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feb04f8f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0015d9a0 0x7feb000d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feb04f8f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feb000d2030 0x7feb000cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feb04f8f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb000d24c0 con 0x7feb000ce600 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feb04f8f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb000cefa0 con 0x7feb000d2030 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feb04f8f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0015ebd0 con 0x7feb0015d9a0 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafdd9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7feb000ce600 0x7feb000d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35812/0 (socket says 172.21.15.133:35812) 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafdd9b640 1 -- 172.21.15.133:0/2437264887 learned_addr learned my addr 172.21.15.133:0/2437264887 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafe59c640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0015d9a0 0x7feb000d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafed9d640 1 --2- 172.21.15.133:0/2437264887 >> v2:172.21.15.162:3300/0 conn(0x7feb000d2030 0x7feb000cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415084921 0 0) 0x7feb000d24c0 con 0x7feb000ce600 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feae8003530 con 0x7feb000ce600 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2141570631 0 0) 0x7feae8003530 con 0x7feb000ce600 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0015d9a0 msgr2=0x7feb000d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0015d9a0 0x7feb000d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 >> v2:172.21.15.162:3300/0 conn(0x7feb000d2030 msgr2=0x7feb000cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 --2- 172.21.15.133:0/2437264887 >> v2:172.21.15.162:3300/0 conn(0x7feb000d2030 0x7feb000cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0015eeb0 con 0x7feb000ce600 2024-04-25T10:02:57.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7feaf4001680 con 0x7feb000ce600 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 >> v1:172.21.15.133:6789/0 conn(0x7feb000ce600 legacy=0x7feb000d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 0x7feae8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feae8004810 0x7feae8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0015d9a0 0x7feae80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0015ebd0 con 0x7feae80039c0 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae800b9d0 con 0x7feae8004810 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae800bb50 con 0x7feb0015d9a0 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafe59c640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 0x7feae8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafed9d640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0015d9a0 0x7feae80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafdd9b640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feae8004810 0x7feae8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafe59c640 1 -- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0015d9a0 msgr2=0x7feae80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafe59c640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb0015d9a0 0x7feae80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafe59c640 1 -- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feae8004810 msgr2=0x7feae8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafe59c640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feae8004810 0x7feae8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafe59c640 1 -- 172.21.15.133:0/2437264887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8003530 con 0x7feae80039c0 2024-04-25T10:02:57.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.076+0000 7feafdd9b640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feae8004810 0x7feae8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafe59c640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 0x7feae8003de0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009b10 tx=0x7feaf001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25336ed11293a4df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00b2070 con 0x7feae80039c0 2024-04-25T10:02:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf00a5530 con 0x7feae80039c0 2024-04-25T10:02:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafd59a640 1 -- 172.21.15.133:0/2437264887 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00096b0 con 0x7feae80039c0 2024-04-25T10:02:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 -- 172.21.15.133:0/2437264887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 msgr2=0x7feae8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 --2- 172.21.15.133:0/2437264887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 0x7feae8003de0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009b10 tx=0x7feaf001ca40 comp rx=0 tx=0).stop 2024-04-25T10:02:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafed9d640 1 -- 172.21.15.133:0/2437264887 reap_dead start 2024-04-25T10:02:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 -- 172.21.15.133:0/2437264887 shutdown_connections 2024-04-25T10:02:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 -- 172.21.15.133:0/2437264887 >> 172.21.15.133:0/2437264887 conn(0x7feb0006bf80 msgr2=0x7feb0006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 -- 172.21.15.133:0/2437264887 shutdown_connections 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 -- 172.21.15.133:0/2437264887 wait complete. 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 Processor -- start 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 -- start start 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 0x7feb001586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb000c97a0 0x7feb000c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb00148ea0 0x7feb00149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb000b9820 con 0x7feae80039c0 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb000b96a0 con 0x7feb000c97a0 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feb04f8f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb000d0f40 con 0x7feb00148ea0 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafe59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 0x7feb001586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafe59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 0x7feb001586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54864/0 (socket says 172.21.15.133:54864) 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafe59c640 1 -- 172.21.15.133:0/2038950435 learned_addr learned my addr 172.21.15.133:0/2038950435 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafed9d640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb00148ea0 0x7feb00149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafdd9b640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb000c97a0 0x7feb000c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafe59c640 1 -- 172.21.15.133:0/2038950435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb00148ea0 msgr2=0x7feb00149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafe59c640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb00148ea0 0x7feb00149320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafe59c640 1 -- 172.21.15.133:0/2038950435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb000c97a0 msgr2=0x7feb000c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafe59c640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb000c97a0 0x7feb000c9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafe59c640 1 -- 172.21.15.133:0/2038950435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb00158e70 con 0x7feae80039c0 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafed9d640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb00148ea0 0x7feb00149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.080+0000 7feafe59c640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 0x7feb001586d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7feaf001c870 tx=0x7feae8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.084+0000 7feaeeffd640 1 -- 172.21.15.133:0/2038950435 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00bc030 con 0x7feae80039c0 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.084+0000 7feaeeffd640 1 -- 172.21.15.133:0/2038950435 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf000f000 con 0x7feae80039c0 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.084+0000 7feb04f8f640 1 -- 172.21.15.133:0/2038950435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb00159150 con 0x7feae80039c0 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.084+0000 7feaeeffd640 1 -- 172.21.15.133:0/2038950435 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00b2040 con 0x7feae80039c0 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.084+0000 7feb04f8f640 1 -- 172.21.15.133:0/2038950435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb00159710 con 0x7feae80039c0 2024-04-25T10:02:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.084+0000 7feb04f8f640 1 -- 172.21.15.133:0/2038950435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb00068e80 con 0x7feae80039c0 2024-04-25T10:02:57.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.084+0000 7feaeeffd640 1 -- 172.21.15.133:0/2038950435 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feaf00bf070 con 0x7feae80039c0 2024-04-25T10:02:57.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.084+0000 7feaeeffd640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feae0066f60 0x7feae0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.084+0000 7feaeeffd640 1 -- 172.21.15.133:0/2038950435 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7feaf011f0c0 con 0x7feae80039c0 2024-04-25T10:02:57.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.084+0000 7feafdd9b640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feae0066f60 0x7feae0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.084+0000 7feafdd9b640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feae0066f60 0x7feae0069420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000f70 tx=0x7feaf4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.088+0000 7feaeeffd640 1 -- 172.21.15.133:0/2038950435 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feaf00c7020 con 0x7feae80039c0 2024-04-25T10:02:57.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.244+0000 7feb04f8f640 1 -- 172.21.15.133:0/2038950435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feb00073b90 con 0x7feae80039c0 2024-04-25T10:02:57.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.244+0000 7feaeeffd640 1 -- 172.21.15.133:0/2038950435 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7feaf00167c0 con 0x7feae80039c0 2024-04-25T10:02:57.251 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 -- 172.21.15.133:0/2038950435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feae0066f60 msgr2=0x7feae0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feae0066f60 0x7feae0069420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000f70 tx=0x7feaf4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 -- 172.21.15.133:0/2038950435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 msgr2=0x7feb001586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 0x7feb001586d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7feaf001c870 tx=0x7feae8005840 comp rx=0 tx=0).stop 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 -- 172.21.15.133:0/2038950435 shutdown_connections 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb00148ea0 0x7feb00149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb000c97a0 0x7feb000c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feae0066f60 0x7feae0069420 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 --2- 172.21.15.133:0/2038950435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae80039c0 0x7feb001586d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 -- 172.21.15.133:0/2038950435 >> 172.21.15.133:0/2038950435 conn(0x7feb0006bf80 msgr2=0x7feb000d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 -- 172.21.15.133:0/2038950435 shutdown_connections 2024-04-25T10:02:57.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.248+0000 7feb04f8f640 1 -- 172.21.15.133:0/2038950435 wait complete. 2024-04-25T10:02:57.270 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:57.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a40fd640 1 Processor -- start 2024-04-25T10:02:57.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a40fd640 1 -- start start 2024-04-25T10:02:57.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a40fd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f309c15bc90 0x7f309c0cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a40fd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c0d01d0 0x7f309c0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a40fd640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309c0cc6a0 con 0x7f309c15b8b0 2024-04-25T10:02:57.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a40fd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c0d0600 con 0x7f309c15bc90 2024-04-25T10:02:57.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a40fd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0cc980 con 0x7f309c0d01d0 2024-04-25T10:02:57.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a269a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c0d01d0 0x7f309c0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a1e99640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f309c15b8b0 0x7f309c0ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:58938/0 (socket says 172.21.15.133:58938) 2024-04-25T10:02:57.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a1698640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f309c15bc90 0x7f309c0cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a1e99640 1 -- 172.21.15.133:0/170750769 learned_addr learned my addr 172.21.15.133:0/170750769 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:57.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a269a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c0d01d0 0x7f309c0b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:57840/0 (socket says 172.21.15.133:57840) 2024-04-25T10:02:57.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a0e97640 1 -- 172.21.15.133:0/170750769 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632194477 0 0) 0x7f309c0cc6a0 con 0x7f309c15b8b0 2024-04-25T10:02:57.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a0e97640 1 -- 172.21.15.133:0/170750769 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3088003530 con 0x7f309c15b8b0 2024-04-25T10:02:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a1698640 1 -- 172.21.15.133:0/170750769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c0d01d0 msgr2=0x7f309c0b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a1698640 1 --2- 172.21.15.133:0/170750769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c0d01d0 0x7f309c0b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a1698640 1 -- 172.21.15.133:0/170750769 >> v1:172.21.15.162:6789/0 conn(0x7f309c15b8b0 legacy=0x7f309c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a1698640 1 -- 172.21.15.133:0/170750769 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0ccc60 con 0x7f309c15bc90 2024-04-25T10:02:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.380+0000 7f30a269a640 1 --2- 172.21.15.133:0/170750769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c0d01d0 0x7f309c0b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:57.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a1698640 1 --2- 172.21.15.133:0/170750769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f309c15bc90 0x7f309c0cfc90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f308c0097c0 tx=0x7f308c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f933f5a99fd5a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a0e97640 1 -- 172.21.15.133:0/170750769 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c0b0070 con 0x7f309c15bc90 2024-04-25T10:02:57.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a0e97640 1 -- 172.21.15.133:0/170750769 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c0ab600 con 0x7f309c15bc90 2024-04-25T10:02:57.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a0e97640 1 -- 172.21.15.133:0/170750769 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c0ab9b0 con 0x7f309c15bc90 2024-04-25T10:02:57.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a40fd640 1 -- 172.21.15.133:0/170750769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f309c15bc90 msgr2=0x7f309c0cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a40fd640 1 --2- 172.21.15.133:0/170750769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f309c15bc90 0x7f309c0cfc90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f308c0097c0 tx=0x7f308c01c410 comp rx=0 tx=0).stop 2024-04-25T10:02:57.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a40fd640 1 -- 172.21.15.133:0/170750769 shutdown_connections 2024-04-25T10:02:57.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a40fd640 1 --2- 172.21.15.133:0/170750769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c0d01d0 0x7f309c0b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a40fd640 1 --2- 172.21.15.133:0/170750769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f309c15bc90 0x7f309c0cfc90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a40fd640 1 -- 172.21.15.133:0/170750769 >> 172.21.15.133:0/170750769 conn(0x7f309c09a990 msgr2=0x7f309c0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a40fd640 1 -- 172.21.15.133:0/170750769 shutdown_connections 2024-04-25T10:02:57.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a40fd640 1 -- 172.21.15.133:0/170750769 wait complete. 2024-04-25T10:02:57.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a40fd640 1 Processor -- start 2024-04-25T10:02:57.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.384+0000 7f30a40fd640 1 -- start start 2024-04-25T10:02:57.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a40fd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f309c0d01d0 0x7f309c14ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a40fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f309c15b8b0 0x7f309c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a40fd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c15bc90 0x7f309c156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a40fd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0b17f0 con 0x7f309c15b8b0 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a40fd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0cc440 con 0x7f309c0d01d0 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a40fd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0b1970 con 0x7f309c15bc90 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1698640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f309c15b8b0 0x7f309c15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1698640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f309c15b8b0 0x7f309c15a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54870/0 (socket says 172.21.15.133:54870) 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1698640 1 -- 172.21.15.133:0/2248991080 learned_addr learned my addr 172.21.15.133:0/2248991080 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a269a640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c15bc90 0x7f309c156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1e99640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f309c0d01d0 0x7f309c14ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1698640 1 -- 172.21.15.133:0/2248991080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c15bc90 msgr2=0x7f309c156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1698640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c15bc90 0x7f309c156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1698640 1 -- 172.21.15.133:0/2248991080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f309c0d01d0 msgr2=0x7f309c14ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1698640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f309c0d01d0 0x7f309c14ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1698640 1 -- 172.21.15.133:0/2248991080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c14b0a0 con 0x7f309c15b8b0 2024-04-25T10:02:57.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1698640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f309c15b8b0 0x7f309c15a5c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f308c0ae040 tx=0x7f308c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30927fc640 1 -- 172.21.15.133:0/2248991080 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c0b0040 con 0x7f309c15b8b0 2024-04-25T10:02:57.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30927fc640 1 -- 172.21.15.133:0/2248991080 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c0ab400 con 0x7f309c15b8b0 2024-04-25T10:02:57.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30927fc640 1 -- 172.21.15.133:0/2248991080 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c006e20 con 0x7f309c15b8b0 2024-04-25T10:02:57.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a40fd640 1 -- 172.21.15.133:0/2248991080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c15e130 con 0x7f309c15b8b0 2024-04-25T10:02:57.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a40fd640 1 -- 172.21.15.133:0/2248991080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c15e6f0 con 0x7f309c15b8b0 2024-04-25T10:02:57.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a40fd640 1 -- 172.21.15.133:0/2248991080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c067c70 con 0x7f309c15b8b0 2024-04-25T10:02:57.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30927fc640 1 -- 172.21.15.133:0/2248991080 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f308c0bb020 con 0x7f309c15b8b0 2024-04-25T10:02:57.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30927fc640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3080066f60 0x7f3080069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30927fc640 1 -- 172.21.15.133:0/2248991080 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f308c11e9f0 con 0x7f309c15b8b0 2024-04-25T10:02:57.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1e99640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3080066f60 0x7f3080069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.388+0000 7f30a1e99640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3080066f60 0x7f3080069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3098005890 tx=0x7f3098009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.392+0000 7f30927fc640 1 -- 172.21.15.133:0/2248991080 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f308c016090 con 0x7f309c15b8b0 2024-04-25T10:02:57.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 -- 172.21.15.133:0/2248991080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f309c0b0860 con 0x7f309c15b8b0 2024-04-25T10:02:57.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30927fc640 1 -- 172.21.15.133:0/2248991080 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f308c0ef600 con 0x7f309c15b8b0 2024-04-25T10:02:57.546 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:57.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 -- 172.21.15.133:0/2248991080 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3080066f60 msgr2=0x7f3080069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3080066f60 0x7f3080069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3098005890 tx=0x7f3098009040 comp rx=0 tx=0).stop 2024-04-25T10:02:57.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 -- 172.21.15.133:0/2248991080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f309c15b8b0 msgr2=0x7f309c15a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f309c15b8b0 0x7f309c15a5c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f308c0ae040 tx=0x7f308c0191b0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 -- 172.21.15.133:0/2248991080 shutdown_connections 2024-04-25T10:02:57.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f309c15bc90 0x7f309c156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3080066f60 0x7f3080069420 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f309c15b8b0 0x7f309c15a5c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 --2- 172.21.15.133:0/2248991080 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f309c0d01d0 0x7f309c14ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 -- 172.21.15.133:0/2248991080 >> 172.21.15.133:0/2248991080 conn(0x7f309c09a990 msgr2=0x7f309c06f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 -- 172.21.15.133:0/2248991080 shutdown_connections 2024-04-25T10:02:57.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.540+0000 7f30a40fd640 1 -- 172.21.15.133:0/2248991080 wait complete. 2024-04-25T10:02:57.563 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:57.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90fe92640 1 Processor -- start 2024-04-25T10:02:57.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90fe92640 1 -- start start 2024-04-25T10:02:57.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90fe92640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe908143ea0 0x7fe9080cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90fe92640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe9080b2150 0x7fe90806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90fe92640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe90815b9b0 0x7fe90806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90fe92640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90815ac50 con 0x7fe9080b2150 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90fe92640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9080cd180 con 0x7fe908143ea0 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90fe92640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90815aed0 con 0x7fe90815b9b0 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90d42d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe9080b2150 0x7fe90806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90d42d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe9080b2150 0x7fe90806e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54884/0 (socket says 172.21.15.133:54884) 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90d42d640 1 -- 172.21.15.133:0/2615387655 learned_addr learned my addr 172.21.15.133:0/2615387655 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90e42f640 1 --2- 172.21.15.133:0/2615387655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe90815b9b0 0x7fe90806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90dc2e640 1 --2- 172.21.15.133:0/2615387655 >> v2:172.21.15.162:3300/0 conn(0x7fe908143ea0 0x7fe9080cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90d42d640 1 -- 172.21.15.133:0/2615387655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe90815b9b0 msgr2=0x7fe90806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90d42d640 1 --2- 172.21.15.133:0/2615387655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe90815b9b0 0x7fe90806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90d42d640 1 -- 172.21.15.133:0/2615387655 >> v2:172.21.15.162:3300/0 conn(0x7fe908143ea0 msgr2=0x7fe9080cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90d42d640 1 --2- 172.21.15.133:0/2615387655 >> v2:172.21.15.162:3300/0 conn(0x7fe908143ea0 0x7fe9080cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.664+0000 7fe90d42d640 1 -- 172.21.15.133:0/2615387655 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90815b1b0 con 0x7fe9080b2150 2024-04-25T10:02:57.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90d42d640 1 --2- 172.21.15.133:0/2615387655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080b2150 0x7fe90806e3b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe9080b4b20 tx=0x7fe8f801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6465522f5e597dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90cc2c640 1 -- 172.21.15.133:0/2615387655 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f80ab5f0 con 0x7fe9080b2150 2024-04-25T10:02:57.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90cc2c640 1 -- 172.21.15.133:0/2615387655 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f80b0070 con 0x7fe9080b2150 2024-04-25T10:02:57.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90cc2c640 1 -- 172.21.15.133:0/2615387655 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f80ab9b0 con 0x7fe9080b2150 2024-04-25T10:02:57.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 -- 172.21.15.133:0/2615387655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080b2150 msgr2=0x7fe90806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 --2- 172.21.15.133:0/2615387655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080b2150 0x7fe90806e3b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe9080b4b20 tx=0x7fe8f801c520 comp rx=0 tx=0).stop 2024-04-25T10:02:57.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 -- 172.21.15.133:0/2615387655 shutdown_connections 2024-04-25T10:02:57.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 --2- 172.21.15.133:0/2615387655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe90815b9b0 0x7fe90806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 --2- 172.21.15.133:0/2615387655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080b2150 0x7fe90806e3b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 --2- 172.21.15.133:0/2615387655 >> v2:172.21.15.162:3300/0 conn(0x7fe908143ea0 0x7fe9080cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 -- 172.21.15.133:0/2615387655 >> 172.21.15.133:0/2615387655 conn(0x7fe908066f70 msgr2=0x7fe90806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 -- 172.21.15.133:0/2615387655 shutdown_connections 2024-04-25T10:02:57.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 -- 172.21.15.133:0/2615387655 wait complete. 2024-04-25T10:02:57.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 Processor -- start 2024-04-25T10:02:57.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 -- start start 2024-04-25T10:02:57.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080b2150 0x7fe90813d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe908143ea0 0x7fe90813be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe90815b9b0 0x7fe90815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90815ede0 con 0x7fe9080b2150 2024-04-25T10:02:57.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90815ec60 con 0x7fe908143ea0 2024-04-25T10:02:57.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90fe92640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90815ef60 con 0x7fe90815b9b0 2024-04-25T10:02:57.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90dc2e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080b2150 0x7fe90813d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90dc2e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080b2150 0x7fe90813d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54890/0 (socket says 172.21.15.133:54890) 2024-04-25T10:02:57.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90dc2e640 1 -- 172.21.15.133:0/3075089537 learned_addr learned my addr 172.21.15.133:0/3075089537 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:57.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90e42f640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe90815b9b0 0x7fe90815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90dc2e640 1 -- 172.21.15.133:0/3075089537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe90815b9b0 msgr2=0x7fe90815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90dc2e640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe90815b9b0 0x7fe90815d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90dc2e640 1 -- 172.21.15.133:0/3075089537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe908143ea0 msgr2=0x7fe90813be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90d42d640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe908143ea0 0x7fe90813be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90dc2e640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe908143ea0 0x7fe90813be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.668+0000 7fe90dc2e640 1 -- 172.21.15.133:0/3075089537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90813c3b0 con 0x7fe9080b2150 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe90dc2e640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080b2150 0x7fe90813d720 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe90400d830 tx=0x7fe90400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe8fe7fc640 1 -- 172.21.15.133:0/3075089537 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90400b6c0 con 0x7fe9080b2150 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe8fe7fc640 1 -- 172.21.15.133:0/3075089537 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe904019070 con 0x7fe9080b2150 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe90fe92640 1 -- 172.21.15.133:0/3075089537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90813c680 con 0x7fe9080b2150 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe8fe7fc640 1 -- 172.21.15.133:0/3075089537 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90400ba60 con 0x7fe9080b2150 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe90fe92640 1 -- 172.21.15.133:0/3075089537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90813cc40 con 0x7fe9080b2150 2024-04-25T10:02:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe90fe92640 1 -- 172.21.15.133:0/3075089537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe908068e80 con 0x7fe9080b2150 2024-04-25T10:02:57.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe8fe7fc640 1 -- 172.21.15.133:0/3075089537 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe90401e460 con 0x7fe9080b2150 2024-04-25T10:02:57.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe8fe7fc640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8ec066f60 0x7fe8ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe8fe7fc640 1 -- 172.21.15.133:0/3075089537 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe904085b10 con 0x7fe9080b2150 2024-04-25T10:02:57.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe90d42d640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8ec066f60 0x7fe8ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.672+0000 7fe90d42d640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8ec066f60 0x7fe8ec069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80af000 tx=0x7fe8f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.676+0000 7fe8fe7fc640 1 -- 172.21.15.133:0/3075089537 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe904056720 con 0x7fe9080b2150 2024-04-25T10:02:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.832+0000 7fe90fe92640 1 -- 172.21.15.133:0/3075089537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe90806eaa0 con 0x7fe9080b2150 2024-04-25T10:02:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe8fe7fc640 1 -- 172.21.15.133:0/3075089537 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe904056020 con 0x7fe9080b2150 2024-04-25T10:02:57.842 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:02:57.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 -- 172.21.15.133:0/3075089537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8ec066f60 msgr2=0x7fe8ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8ec066f60 0x7fe8ec069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80af000 tx=0x7fe8f8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:57.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 -- 172.21.15.133:0/3075089537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080b2150 msgr2=0x7fe90813d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080b2150 0x7fe90813d720 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe90400d830 tx=0x7fe90400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 -- 172.21.15.133:0/3075089537 shutdown_connections 2024-04-25T10:02:57.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe90815b9b0 0x7fe90815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe908143ea0 0x7fe90813be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8ec066f60 0x7fe8ec069420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 --2- 172.21.15.133:0/3075089537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080b2150 0x7fe90813d720 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 -- 172.21.15.133:0/3075089537 >> 172.21.15.133:0/3075089537 conn(0x7fe908066f70 msgr2=0x7fe9080735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 -- 172.21.15.133:0/3075089537 shutdown_connections 2024-04-25T10:02:57.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:57.836+0000 7fe90fe92640 1 -- 172.21.15.133:0/3075089537 wait complete. 2024-04-25T10:02:57.859 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:02:57.859 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:57.865 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:58.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.008+0000 7f373fc4e640 1 Processor -- start 2024-04-25T10:02:58.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373fc4e640 1 -- start start 2024-04-25T10:02:58.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373fc4e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f373815a5f0 0x7f37380c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373fc4e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f373815a9d0 0x7f37380cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373fc4e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37380cc5a0 con 0x7f373806e460 2024-04-25T10:02:58.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373fc4e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37380cc880 con 0x7f373815a5f0 2024-04-25T10:02:58.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373fc4e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37380ccb60 con 0x7f373815a9d0 2024-04-25T10:02:58.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373e1eb640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f373806e460 0x7f373815e380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35814/0 (socket says 172.21.15.133:35814) 2024-04-25T10:02:58.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373e1eb640 1 -- 172.21.15.133:0/1760423325 learned_addr learned my addr 172.21.15.133:0/1760423325 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455210980 0 0) 0x7f37380cc5a0 con 0x7f373806e460 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373d9ea640 1 --2- 172.21.15.133:0/1760423325 >> v2:172.21.15.162:3300/0 conn(0x7f373815a5f0 0x7f37380c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373d1e9640 1 --2- 172.21.15.133:0/1760423325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f373815a9d0 0x7f37380cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3724003530 con 0x7f373806e460 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1534253131 0 0) 0x7f3724003530 con 0x7f373806e460 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f373815a9d0 msgr2=0x7f37380cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 --2- 172.21.15.133:0/1760423325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f373815a9d0 0x7f37380cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 >> v2:172.21.15.162:3300/0 conn(0x7f373815a5f0 msgr2=0x7f37380c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 --2- 172.21.15.133:0/1760423325 >> v2:172.21.15.162:3300/0 conn(0x7f373815a5f0 0x7f37380c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37380cce40 con 0x7f373806e460 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f372c001a00 con 0x7f373806e460 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 >> v1:172.21.15.133:6789/0 conn(0x7f373806e460 legacy=0x7f373815e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 --2- 172.21.15.133:0/1760423325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37240039c0 0x7f3724003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 --2- 172.21.15.133:0/1760423325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724004810 0x7f3724004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 --2- 172.21.15.133:0/1760423325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37240051d0 0x7f372400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37380cc880 con 0x7f37240039c0 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37380ccb60 con 0x7f37240051d0 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372400bf40 con 0x7f3724004810 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373d1e9640 1 --2- 172.21.15.133:0/1760423325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f373815a9d0 0x7f37380cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373d9ea640 1 --2- 172.21.15.133:0/1760423325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37240039c0 0x7f3724003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373d9ea640 1 -- 172.21.15.133:0/1760423325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724004810 msgr2=0x7f3724004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373d9ea640 1 --2- 172.21.15.133:0/1760423325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3724004810 0x7f3724004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373d9ea640 1 -- 172.21.15.133:0/1760423325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37240051d0 msgr2=0x7f372400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373d9ea640 1 --2- 172.21.15.133:0/1760423325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37240051d0 0x7f372400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.012+0000 7f373d9ea640 1 -- 172.21.15.133:0/1760423325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3724003530 con 0x7f37240039c0 2024-04-25T10:02:58.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373d9ea640 1 --2- 172.21.15.133:0/1760423325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37240039c0 0x7f3724003de0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3730009ae0 tx=0x7f373001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d515b41fb375dc50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37300b2070 con 0x7f37240039c0 2024-04-25T10:02:58.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37300a5590 con 0x7f37240039c0 2024-04-25T10:02:58.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373c9e8640 1 -- 172.21.15.133:0/1760423325 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37300096b0 con 0x7f37240039c0 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 -- 172.21.15.133:0/1760423325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37240039c0 msgr2=0x7f3724003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 --2- 172.21.15.133:0/1760423325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37240039c0 0x7f3724003de0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3730009ae0 tx=0x7f373001cab0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373e1eb640 1 -- 172.21.15.133:0/1760423325 reap_dead start 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 -- 172.21.15.133:0/1760423325 shutdown_connections 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 -- 172.21.15.133:0/1760423325 >> 172.21.15.133:0/1760423325 conn(0x7f3738067000 msgr2=0x7f373806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 -- 172.21.15.133:0/1760423325 shutdown_connections 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 -- 172.21.15.133:0/1760423325 wait complete. 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 Processor -- start 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 -- start start 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37240039c0 0x7f373814cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3738147f10 0x7f373814d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f373814db60 0x7f37380b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373806ec20 con 0x7f373814db60 2024-04-25T10:02:58.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373806eaa0 con 0x7f37240039c0 2024-04-25T10:02:58.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373806eda0 con 0x7f3738147f10 2024-04-25T10:02:58.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373e1eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f373814db60 0x7f37380b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373e1eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f373814db60 0x7f37380b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54914/0 (socket says 172.21.15.133:54914) 2024-04-25T10:02:58.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373e1eb640 1 -- 172.21.15.133:0/447799661 learned_addr learned my addr 172.21.15.133:0/447799661 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:58.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373d1e9640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3738147f10 0x7f373814d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373d9ea640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37240039c0 0x7f373814cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373e1eb640 1 -- 172.21.15.133:0/447799661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3738147f10 msgr2=0x7f373814d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373e1eb640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3738147f10 0x7f373814d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373e1eb640 1 -- 172.21.15.133:0/447799661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37240039c0 msgr2=0x7f373814cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373e1eb640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37240039c0 0x7f373814cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373e1eb640 1 -- 172.21.15.133:0/447799661 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373814e0c0 con 0x7f373814db60 2024-04-25T10:02:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373e1eb640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f373814db60 0x7f37380b37d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f37380cb860 tx=0x7f372c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f372a7fc640 1 -- 172.21.15.133:0/447799661 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c007110 con 0x7f373814db60 2024-04-25T10:02:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f372a7fc640 1 -- 172.21.15.133:0/447799661 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c0072d0 con 0x7f373814db60 2024-04-25T10:02:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 -- 172.21.15.133:0/447799661 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373814be50 con 0x7f373814db60 2024-04-25T10:02:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f372a7fc640 1 -- 172.21.15.133:0/447799661 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0076c0 con 0x7f373814db60 2024-04-25T10:02:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.016+0000 7f373fc4e640 1 -- 172.21.15.133:0/447799661 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373814c410 con 0x7f373814db60 2024-04-25T10:02:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.020+0000 7f373fc4e640 1 -- 172.21.15.133:0/447799661 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3738068ef0 con 0x7f373814db60 2024-04-25T10:02:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.020+0000 7f372a7fc640 1 -- 172.21.15.133:0/447799661 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f372c007860 con 0x7f373814db60 2024-04-25T10:02:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.020+0000 7f372a7fc640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f371c066f60 0x7f371c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.020+0000 7f372a7fc640 1 -- 172.21.15.133:0/447799661 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f372c084d20 con 0x7f373814db60 2024-04-25T10:02:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.020+0000 7f373d9ea640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f371c066f60 0x7f371c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.020+0000 7f373d9ea640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f371c066f60 0x7f371c069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3730009ae0 tx=0x7f37300046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.024+0000 7f372a7fc640 1 -- 172.21.15.133:0/447799661 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f372c0559c0 con 0x7f373814db60 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.172+0000 7f373fc4e640 1 -- 172.21.15.133:0/447799661 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f373806d5a0 con 0x7f373814db60 2024-04-25T10:02:58.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.172+0000 7f372a7fc640 1 -- 172.21.15.133:0/447799661 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f372c0552c0 con 0x7f373814db60 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:58.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 -- 172.21.15.133:0/447799661 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f371c066f60 msgr2=0x7f371c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f371c066f60 0x7f371c069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3730009ae0 tx=0x7f37300046d0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 -- 172.21.15.133:0/447799661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f373814db60 msgr2=0x7f37380b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f373814db60 0x7f37380b37d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f37380cb860 tx=0x7f372c00cf00 comp rx=0 tx=0).stop 2024-04-25T10:02:58.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 -- 172.21.15.133:0/447799661 shutdown_connections 2024-04-25T10:02:58.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f371c066f60 0x7f371c069420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f373814db60 0x7f37380b37d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3738147f10 0x7f373814d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 --2- 172.21.15.133:0/447799661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37240039c0 0x7f373814cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 -- 172.21.15.133:0/447799661 >> 172.21.15.133:0/447799661 conn(0x7f3738067000 msgr2=0x7f373815f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 -- 172.21.15.133:0/447799661 shutdown_connections 2024-04-25T10:02:58.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.176+0000 7f373fc4e640 1 -- 172.21.15.133:0/447799661 wait complete. 2024-04-25T10:02:58.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:58.198 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:58.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7feca49b4640 1 Processor -- start 2024-04-25T10:02:58.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7feca49b4640 1 -- start start 2024-04-25T10:02:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7feca49b4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feca015ba80 0x7feca00cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7feca49b4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca00ccd80 con 0x7feca00b0470 2024-04-25T10:02:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7feca49b4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca00cadc0 con 0x7feca015cb80 2024-04-25T10:02:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7feca49b4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca00cc870 con 0x7feca015ba80 2024-04-25T10:02:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7fec9e59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feca015ba80 0x7feca00cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7fec9e59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feca015ba80 0x7feca00cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54916/0 (socket says 172.21.15.133:54916) 2024-04-25T10:02:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7fec9ed9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7feca00b0470 0x7feca006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35820/0 (socket says 172.21.15.133:35820) 2024-04-25T10:02:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7fec9e59c640 1 -- 172.21.15.133:0/2060254883 learned_addr learned my addr 172.21.15.133:0/2060254883 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7fec9e59c640 1 -- 172.21.15.133:0/2060254883 >> v1:172.21.15.162:6789/0 conn(0x7feca015cb80 legacy=0x7feca00afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7fec9e59c640 1 -- 172.21.15.133:0/2060254883 >> v1:172.21.15.133:6789/0 conn(0x7feca00b0470 legacy=0x7feca006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.304+0000 7fec9e59c640 1 -- 172.21.15.133:0/2060254883 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca00ccaa0 con 0x7feca015ba80 2024-04-25T10:02:58.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9e59c640 1 --2- 172.21.15.133:0/2060254883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca015ba80 0x7feca00cc330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fec88009870 tx=0x7fec8801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58e6527b8f03185 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9dd9b640 1 -- 172.21.15.133:0/2060254883 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880ab650 con 0x7feca015ba80 2024-04-25T10:02:58.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9dd9b640 1 -- 172.21.15.133:0/2060254883 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec880b0070 con 0x7feca015ba80 2024-04-25T10:02:58.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9dd9b640 1 -- 172.21.15.133:0/2060254883 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880aba10 con 0x7feca015ba80 2024-04-25T10:02:58.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 -- 172.21.15.133:0/2060254883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca015ba80 msgr2=0x7feca00cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 --2- 172.21.15.133:0/2060254883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca015ba80 0x7feca00cc330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fec88009870 tx=0x7fec8801c560 comp rx=0 tx=0).stop 2024-04-25T10:02:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 -- 172.21.15.133:0/2060254883 shutdown_connections 2024-04-25T10:02:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 --2- 172.21.15.133:0/2060254883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca015ba80 0x7feca00cc330 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 -- 172.21.15.133:0/2060254883 >> 172.21.15.133:0/2060254883 conn(0x7feca00d0570 msgr2=0x7feca00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 -- 172.21.15.133:0/2060254883 shutdown_connections 2024-04-25T10:02:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 -- 172.21.15.133:0/2060254883 wait complete. 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 Processor -- start 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 -- start start 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00b0470 0x7feca01f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feca015ba80 0x7feca01599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feca015cb80 0x7feca0159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca01696d0 con 0x7feca00b0470 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca0169550 con 0x7feca015cb80 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca0169850 con 0x7feca015ba80 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00b0470 0x7feca01f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00b0470 0x7feca01f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54920/0 (socket says 172.21.15.133:54920) 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9e59c640 1 -- 172.21.15.133:0/299517186 learned_addr learned my addr 172.21.15.133:0/299517186 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:58.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9ed9d640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feca015cb80 0x7feca0159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9e59c640 1 -- 172.21.15.133:0/299517186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feca015ba80 msgr2=0x7feca01599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9e59c640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feca015ba80 0x7feca01599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9e59c640 1 -- 172.21.15.133:0/299517186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feca015cb80 msgr2=0x7feca0159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9e59c640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feca015cb80 0x7feca0159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9e59c640 1 -- 172.21.15.133:0/299517186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca00b33c0 con 0x7feca00b0470 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec9e59c640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00b0470 0x7feca01f5250 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fec880ad040 tx=0x7fec8801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec96ffd640 1 -- 172.21.15.133:0/299517186 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880b4800 con 0x7feca00b0470 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec96ffd640 1 -- 172.21.15.133:0/299517186 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec880b49c0 con 0x7feca00b0470 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 -- 172.21.15.133:0/299517186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca01f5830 con 0x7feca00b0470 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7fec96ffd640 1 -- 172.21.15.133:0/299517186 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880b4db0 con 0x7feca00b0470 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.308+0000 7feca49b4640 1 -- 172.21.15.133:0/299517186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca01f5f20 con 0x7feca00b0470 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.312+0000 7feca49b4640 1 -- 172.21.15.133:0/299517186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca0067c70 con 0x7feca00b0470 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.312+0000 7fec96ffd640 1 -- 172.21.15.133:0/299517186 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec880c7390 con 0x7feca00b0470 2024-04-25T10:02:58.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.312+0000 7fec96ffd640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec84066f60 0x7fec84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.312+0000 7fec96ffd640 1 -- 172.21.15.133:0/299517186 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fec8811fce0 con 0x7feca00b0470 2024-04-25T10:02:58.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.312+0000 7fec97fff640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec84066f60 0x7fec84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.312+0000 7fec97fff640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec84066f60 0x7fec84069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fec90007fd0 tx=0x7fec90007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.316+0000 7fec96ffd640 1 -- 172.21.15.133:0/299517186 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec880c4060 con 0x7feca00b0470 2024-04-25T10:02:58.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.460+0000 7feca49b4640 1 -- 172.21.15.133:0/299517186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feca0157760 con 0x7feca00b0470 2024-04-25T10:02:58.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7fec96ffd640 1 -- 172.21.15.133:0/299517186 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec88016150 con 0x7feca00b0470 2024-04-25T10:02:58.469 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:02:58.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 -- 172.21.15.133:0/299517186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec84066f60 msgr2=0x7fec84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec84066f60 0x7fec84069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fec90007fd0 tx=0x7fec90007980 comp rx=0 tx=0).stop 2024-04-25T10:02:58.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 -- 172.21.15.133:0/299517186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00b0470 msgr2=0x7feca01f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00b0470 0x7feca01f5250 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fec880ad040 tx=0x7fec8801ada0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 -- 172.21.15.133:0/299517186 shutdown_connections 2024-04-25T10:02:58.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feca015cb80 0x7feca0159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feca015ba80 0x7feca01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec84066f60 0x7fec84069420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 --2- 172.21.15.133:0/299517186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00b0470 0x7feca01f5250 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 -- 172.21.15.133:0/299517186 >> 172.21.15.133:0/299517186 conn(0x7feca00d0570 msgr2=0x7feca00d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 -- 172.21.15.133:0/299517186 shutdown_connections 2024-04-25T10:02:58.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.464+0000 7feca49b4640 1 -- 172.21.15.133:0/299517186 wait complete. 2024-04-25T10:02:58.486 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:58.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.588+0000 7fa0be554640 1 Processor -- start 2024-04-25T10:02:58.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0be554640 1 -- start start 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0be554640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa0b815d9a0 0x7fa0b80d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0be554640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b815ebd0 con 0x7fa0b80ce600 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0be554640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b815eeb0 con 0x7fa0b80d2030 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0be554640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b80d0330 con 0x7fa0b815d9a0 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0affff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa0b80ce600 0x7fa0b80d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35824/0 (socket says 172.21.15.133:35824) 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0affff640 1 -- 172.21.15.133:0/3679254804 learned_addr learned my addr 172.21.15.133:0/3679254804 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517709904 0 0) 0x7fa0b815ebd0 con 0x7fa0b80ce600 2024-04-25T10:02:58.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b7fff640 1 --2- 172.21.15.133:0/3679254804 >> v2:172.21.15.162:3300/0 conn(0x7fa0b815d9a0 0x7fa0b80d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a4003530 con 0x7fa0b80ce600 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2158088871 0 0) 0x7fa0a4003530 con 0x7fa0b80ce600 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 >> v2:172.21.15.162:3300/0 conn(0x7fa0b815d9a0 msgr2=0x7fa0b80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 --2- 172.21.15.133:0/3679254804 >> v2:172.21.15.162:3300/0 conn(0x7fa0b815d9a0 0x7fa0b80d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 >> v1:172.21.15.162:6789/0 conn(0x7fa0b80d2030 legacy=0x7fa0b815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b80d0610 con 0x7fa0b80ce600 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa0a0001680 con 0x7fa0b80ce600 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 >> v1:172.21.15.133:6789/0 conn(0x7fa0b80ce600 legacy=0x7fa0b80d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 --2- 172.21.15.133:0/3679254804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0a40039c0 0x7fa0a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 --2- 172.21.15.133:0/3679254804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a4004810 0x7fa0a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 --2- 172.21.15.133:0/3679254804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0a40051d0 0x7fa0a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80d0330 con 0x7fa0a4004810 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a400c170 con 0x7fa0a40039c0 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a400c370 con 0x7fa0a40051d0 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0affff640 1 --2- 172.21.15.133:0/3679254804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a4004810 0x7fa0a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0b7fff640 1 --2- 172.21.15.133:0/3679254804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0a40039c0 0x7fa0a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0affff640 1 -- 172.21.15.133:0/3679254804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0a40051d0 msgr2=0x7fa0a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0affff640 1 --2- 172.21.15.133:0/3679254804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0a40051d0 0x7fa0a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0affff640 1 -- 172.21.15.133:0/3679254804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0a40039c0 msgr2=0x7fa0a4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0affff640 1 --2- 172.21.15.133:0/3679254804 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0a40039c0 0x7fa0a4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0affff640 1 -- 172.21.15.133:0/3679254804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a4003530 con 0x7fa0a4004810 2024-04-25T10:02:58.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.592+0000 7fa0bcaf1640 1 -- 172.21.15.133:0/3679254804 reap_dead start 2024-04-25T10:02:58.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0affff640 1 --2- 172.21.15.133:0/3679254804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a4004810 0x7fa0a4004c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c1c0 tx=0x7fa0a001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dc56f630351829 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0007d00 con 0x7fa0a4004810 2024-04-25T10:02:58.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a00a9020 con 0x7fa0a4004810 2024-04-25T10:02:58.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0b77fe640 1 -- 172.21.15.133:0/3679254804 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00a5b50 con 0x7fa0a4004810 2024-04-25T10:02:58.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 -- 172.21.15.133:0/3679254804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a4004810 msgr2=0x7fa0a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 --2- 172.21.15.133:0/3679254804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a4004810 0x7fa0a4004c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c1c0 tx=0x7fa0a001cc30 comp rx=0 tx=0).stop 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 -- 172.21.15.133:0/3679254804 shutdown_connections 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 --2- 172.21.15.133:0/3679254804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0a4004810 0x7fa0a4004c90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 -- 172.21.15.133:0/3679254804 >> 172.21.15.133:0/3679254804 conn(0x7fa0b806bf80 msgr2=0x7fa0b806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 -- 172.21.15.133:0/3679254804 shutdown_connections 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 -- 172.21.15.133:0/3679254804 wait complete. 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 Processor -- start 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 -- start start 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0a4004810 0x7fa0b80cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0b81572d0 0x7fa0b815fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b81601f0 0x7fa0b80cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80b9820 con 0x7fa0b81601f0 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80b96a0 con 0x7fa0a4004810 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80d1d30 con 0x7fa0b81572d0 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0bcaf1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b81601f0 0x7fa0b80cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0bcaf1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b81601f0 0x7fa0b80cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54930/0 (socket says 172.21.15.133:54930) 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0bcaf1640 1 -- 172.21.15.133:0/1174201036 learned_addr learned my addr 172.21.15.133:0/1174201036 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0affff640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0b81572d0 0x7fa0b815fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0bcaf1640 1 -- 172.21.15.133:0/1174201036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0b81572d0 msgr2=0x7fa0b815fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0bcaf1640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0b81572d0 0x7fa0b815fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0bcaf1640 1 -- 172.21.15.133:0/1174201036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0a4004810 msgr2=0x7fa0b80cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0bcaf1640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0a4004810 0x7fa0b80cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0bcaf1640 1 -- 172.21.15.133:0/1174201036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b81fa320 con 0x7fa0b81601f0 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0affff640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0b81572d0 0x7fa0b815fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0bcaf1640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b81601f0 0x7fa0b80cb950 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80b63c0 tx=0x7fa0b000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0b57fa640 1 -- 172.21.15.133:0/1174201036 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0005d50 con 0x7fa0b81601f0 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0b57fa640 1 -- 172.21.15.133:0/1174201036 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b0003980 con 0x7fa0b81601f0 2024-04-25T10:02:58.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 -- 172.21.15.133:0/1174201036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b81fa600 con 0x7fa0b81601f0 2024-04-25T10:02:58.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0b57fa640 1 -- 172.21.15.133:0/1174201036 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0003d70 con 0x7fa0b81601f0 2024-04-25T10:02:58.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.596+0000 7fa0be554640 1 -- 172.21.15.133:0/1174201036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b81fabc0 con 0x7fa0b81601f0 2024-04-25T10:02:58.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.600+0000 7fa0be554640 1 -- 172.21.15.133:0/1174201036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b8068e80 con 0x7fa0b81601f0 2024-04-25T10:02:58.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.600+0000 7fa0b57fa640 1 -- 172.21.15.133:0/1174201036 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0b000a5e0 con 0x7fa0b81601f0 2024-04-25T10:02:58.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.600+0000 7fa0b57fa640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa098066f60 0x7fa098069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.600+0000 7fa0b57fa640 1 -- 172.21.15.133:0/1174201036 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa0b0084780 con 0x7fa0b81601f0 2024-04-25T10:02:58.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.600+0000 7fa0b7fff640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa098066f60 0x7fa098069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.600+0000 7fa0b7fff640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa098066f60 0x7fa098069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80099b0 tx=0x7fa0a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.604+0000 7fa0b57fa640 1 -- 172.21.15.133:0/1174201036 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0b0055390 con 0x7fa0b81601f0 2024-04-25T10:02:58.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.748+0000 7fa0be554640 1 -- 172.21.15.133:0/1174201036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0b815a910 con 0x7fa0b81601f0 2024-04-25T10:02:58.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.748+0000 7fa0b57fa640 1 -- 172.21.15.133:0/1174201036 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0b0054c90 con 0x7fa0b81601f0 2024-04-25T10:02:58.757 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 -- 172.21.15.133:0/1174201036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa098066f60 msgr2=0x7fa098069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa098066f60 0x7fa098069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80099b0 tx=0x7fa0a8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 -- 172.21.15.133:0/1174201036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b81601f0 msgr2=0x7fa0b80cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b81601f0 0x7fa0b80cb950 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80b63c0 tx=0x7fa0b000cb90 comp rx=0 tx=0).stop 2024-04-25T10:02:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 -- 172.21.15.133:0/1174201036 shutdown_connections 2024-04-25T10:02:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa098066f60 0x7fa098069420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b81601f0 0x7fa0b80cb950 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0b81572d0 0x7fa0b815fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 --2- 172.21.15.133:0/1174201036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0a4004810 0x7fa0b80cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 -- 172.21.15.133:0/1174201036 >> 172.21.15.133:0/1174201036 conn(0x7fa0b806bf80 msgr2=0x7fa0b815f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 -- 172.21.15.133:0/1174201036 shutdown_connections 2024-04-25T10:02:58.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.752+0000 7fa0be554640 1 -- 172.21.15.133:0/1174201036 wait complete. 2024-04-25T10:02:58.774 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:58.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d42a66640 1 Processor -- start 2024-04-25T10:02:58.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d42a66640 1 -- start start 2024-04-25T10:02:58.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d42a66640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1d3c0ce600 0x7f1d3c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d42a66640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1d3c0d1920 0x7f1d3c0d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d42a66640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3c15ebd0 con 0x7f1d3c15d9a0 2024-04-25T10:02:58.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d42a66640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3c15eeb0 con 0x7f1d3c0ce600 2024-04-25T10:02:58.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d42a66640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3c0d0330 con 0x7f1d3c0d1920 2024-04-25T10:02:58.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d3bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1d3c0ce600 0x7f1d3c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d3bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1d3c0ce600 0x7f1d3c0cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54938/0 (socket says 172.21.15.133:54938) 2024-04-25T10:02:58.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d3bfff640 1 -- 172.21.15.133:0/4174330381 learned_addr learned my addr 172.21.15.133:0/4174330381 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:58.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d3bfff640 1 -- 172.21.15.133:0/4174330381 >> v2:172.21.15.162:3300/0 conn(0x7f1d3c0d1920 msgr2=0x7f1d3c0d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d3bfff640 1 --2- 172.21.15.133:0/4174330381 >> v2:172.21.15.162:3300/0 conn(0x7f1d3c0d1920 0x7f1d3c0d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d3bfff640 1 -- 172.21.15.133:0/4174330381 >> v1:172.21.15.133:6789/0 conn(0x7f1d3c15d9a0 legacy=0x7f1d3c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.880+0000 7f1d3bfff640 1 -- 172.21.15.133:0/4174330381 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c0d0610 con 0x7f1d3c0ce600 2024-04-25T10:02:58.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d3bfff640 1 --2- 172.21.15.133:0/4174330381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d3c0ce600 0x7f1d3c0cea90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c08ccc0 tx=0x7f1d3001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d716f788ed5215b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d3b7fe640 1 -- 172.21.15.133:0/4174330381 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d300b0070 con 0x7f1d3c0ce600 2024-04-25T10:02:58.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d3b7fe640 1 -- 172.21.15.133:0/4174330381 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d300ab600 con 0x7f1d3c0ce600 2024-04-25T10:02:58.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d3b7fe640 1 -- 172.21.15.133:0/4174330381 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d300ab9d0 con 0x7f1d3c0ce600 2024-04-25T10:02:58.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 -- 172.21.15.133:0/4174330381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d3c0ce600 msgr2=0x7f1d3c0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 --2- 172.21.15.133:0/4174330381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d3c0ce600 0x7f1d3c0cea90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c08ccc0 tx=0x7f1d3001c490 comp rx=0 tx=0).stop 2024-04-25T10:02:58.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 -- 172.21.15.133:0/4174330381 shutdown_connections 2024-04-25T10:02:58.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 --2- 172.21.15.133:0/4174330381 >> v2:172.21.15.162:3300/0 conn(0x7f1d3c0d1920 0x7f1d3c0d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 --2- 172.21.15.133:0/4174330381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d3c0ce600 0x7f1d3c0cea90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 -- 172.21.15.133:0/4174330381 >> 172.21.15.133:0/4174330381 conn(0x7f1d3c06bf80 msgr2=0x7f1d3c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 -- 172.21.15.133:0/4174330381 shutdown_connections 2024-04-25T10:02:58.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 -- 172.21.15.133:0/4174330381 wait complete. 2024-04-25T10:02:58.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 Processor -- start 2024-04-25T10:02:58.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 -- start start 2024-04-25T10:02:58.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d3c0ce600 0x7f1d3c148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d3c0d1920 0x7f1d3c15af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d3c15d9a0 0x7f1d3c0b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c15e960 con 0x7f1d3c15d9a0 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c15e7e0 con 0x7f1d3c0ce600 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c16a7e0 con 0x7f1d3c0d1920 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d41003640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d3c15d9a0 0x7f1d3c0b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d41003640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d3c15d9a0 0x7f1d3c0b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54946/0 (socket says 172.21.15.133:54946) 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d41003640 1 -- 172.21.15.133:0/991277424 learned_addr learned my addr 172.21.15.133:0/991277424 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d3bfff640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d3c0d1920 0x7f1d3c15af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d40802640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d3c0ce600 0x7f1d3c148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d41003640 1 -- 172.21.15.133:0/991277424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d3c0d1920 msgr2=0x7f1d3c15af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d41003640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d3c0d1920 0x7f1d3c15af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d41003640 1 -- 172.21.15.133:0/991277424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d3c0ce600 msgr2=0x7f1d3c148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d41003640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d3c0ce600 0x7f1d3c148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d41003640 1 -- 172.21.15.133:0/991277424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c15a1c0 con 0x7f1d3c15d9a0 2024-04-25T10:02:58.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d40802640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d3c0ce600 0x7f1d3c148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:58.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d41003640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d3c15d9a0 0x7f1d3c0b75f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400b970 tx=0x7f1d3400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d397fa640 1 -- 172.21.15.133:0/991277424 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34019070 con 0x7f1d3c15d9a0 2024-04-25T10:02:58.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d397fa640 1 -- 172.21.15.133:0/991277424 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3400cb60 con 0x7f1d3c15d9a0 2024-04-25T10:02:58.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d397fa640 1 -- 172.21.15.133:0/991277424 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34014440 con 0x7f1d3c15d9a0 2024-04-25T10:02:58.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.884+0000 7f1d42a66640 1 -- 172.21.15.133:0/991277424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3c15b480 con 0x7f1d3c15d9a0 2024-04-25T10:02:58.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.888+0000 7f1d42a66640 1 -- 172.21.15.133:0/991277424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c15a740 con 0x7f1d3c15d9a0 2024-04-25T10:02:58.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.888+0000 7f1d42a66640 1 -- 172.21.15.133:0/991277424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c068e80 con 0x7f1d3c15d9a0 2024-04-25T10:02:58.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.888+0000 7f1d397fa640 1 -- 172.21.15.133:0/991277424 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d3401c070 con 0x7f1d3c15d9a0 2024-04-25T10:02:58.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.888+0000 7f1d397fa640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d28066f60 0x7f1d28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.888+0000 7f1d397fa640 1 -- 172.21.15.133:0/991277424 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1d34085190 con 0x7f1d3c15d9a0 2024-04-25T10:02:58.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.888+0000 7f1d40802640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d28066f60 0x7f1d28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.888+0000 7f1d40802640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d28066f60 0x7f1d28069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c1fa290 tx=0x7f1d2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:58.892+0000 7f1d397fa640 1 -- 172.21.15.133:0/991277424 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d34055da0 con 0x7f1d3c15d9a0 2024-04-25T10:02:59.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 -- 172.21.15.133:0/991277424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d3c073820 con 0x7f1d3c15d9a0 2024-04-25T10:02:59.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d397fa640 1 -- 172.21.15.133:0/991277424 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d340556a0 con 0x7f1d3c15d9a0 2024-04-25T10:02:59.047 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:02:59.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 -- 172.21.15.133:0/991277424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d28066f60 msgr2=0x7f1d28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d28066f60 0x7f1d28069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c1fa290 tx=0x7f1d2c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:59.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 -- 172.21.15.133:0/991277424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d3c15d9a0 msgr2=0x7f1d3c0b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d3c15d9a0 0x7f1d3c0b75f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400b970 tx=0x7f1d3400be30 comp rx=0 tx=0).stop 2024-04-25T10:02:59.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 -- 172.21.15.133:0/991277424 shutdown_connections 2024-04-25T10:02:59.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d28066f60 0x7f1d28069420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d3c15d9a0 0x7f1d3c0b75f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d3c0d1920 0x7f1d3c15af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 --2- 172.21.15.133:0/991277424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d3c0ce600 0x7f1d3c148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 -- 172.21.15.133:0/991277424 >> 172.21.15.133:0/991277424 conn(0x7f1d3c06bf80 msgr2=0x7f1d3c15c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 -- 172.21.15.133:0/991277424 shutdown_connections 2024-04-25T10:02:59.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.040+0000 7f1d42a66640 1 -- 172.21.15.133:0/991277424 wait complete. 2024-04-25T10:02:59.065 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.168+0000 7fbfb9097640 1 Processor -- start 2024-04-25T10:02:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.168+0000 7fbfb9097640 1 -- start start 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb9097640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbfb406e8b0 0x7fbfb415eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb9097640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfb40b1e50 0x7fbfb4146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb9097640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbfb40aebc0 0x7fbfb40af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb9097640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb4147490 con 0x7fbfb406e8b0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb9097640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb40b19c0 con 0x7fbfb40aebc0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb9097640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40b1090 con 0x7fbfb40b1e50 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb2d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbfb406e8b0 0x7fbfb415eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb2d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbfb406e8b0 0x7fbfb415eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54962/0 (socket says 172.21.15.133:54962) 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb2d9d640 1 -- 172.21.15.133:0/2589858794 learned_addr learned my addr 172.21.15.133:0/2589858794 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb359e640 1 --2- 172.21.15.133:0/2589858794 >> v2:172.21.15.162:3300/0 conn(0x7fbfb40aebc0 0x7fbfb40af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb2d9d640 1 -- 172.21.15.133:0/2589858794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfb40b1e50 msgr2=0x7fbfb4146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb259c640 1 --2- 172.21.15.133:0/2589858794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfb40b1e50 0x7fbfb4146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb2d9d640 1 --2- 172.21.15.133:0/2589858794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfb40b1e50 0x7fbfb4146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb2d9d640 1 -- 172.21.15.133:0/2589858794 >> v2:172.21.15.162:3300/0 conn(0x7fbfb40aebc0 msgr2=0x7fbfb40af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb2d9d640 1 --2- 172.21.15.133:0/2589858794 >> v2:172.21.15.162:3300/0 conn(0x7fbfb40aebc0 0x7fbfb40af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb2d9d640 1 -- 172.21.15.133:0/2589858794 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb40b1370 con 0x7fbfb406e8b0 2024-04-25T10:02:59.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb359e640 1 --2- 172.21.15.133:0/2589858794 >> v2:172.21.15.162:3300/0 conn(0x7fbfb40aebc0 0x7fbfb40af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:59.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.172+0000 7fbfb2d9d640 1 --2- 172.21.15.133:0/2589858794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb406e8b0 0x7fbfb415eb40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009870 tx=0x7fbfa401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff0566af7b38768 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb1d9b640 1 -- 172.21.15.133:0/2589858794 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40b0070 con 0x7fbfb406e8b0 2024-04-25T10:02:59.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb1d9b640 1 -- 172.21.15.133:0/2589858794 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa40ab600 con 0x7fbfb406e8b0 2024-04-25T10:02:59.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb1d9b640 1 -- 172.21.15.133:0/2589858794 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40ab9d0 con 0x7fbfb406e8b0 2024-04-25T10:02:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 -- 172.21.15.133:0/2589858794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb406e8b0 msgr2=0x7fbfb415eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 --2- 172.21.15.133:0/2589858794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb406e8b0 0x7fbfb415eb40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009870 tx=0x7fbfa401c490 comp rx=0 tx=0).stop 2024-04-25T10:02:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 -- 172.21.15.133:0/2589858794 shutdown_connections 2024-04-25T10:02:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 --2- 172.21.15.133:0/2589858794 >> v2:172.21.15.162:3300/0 conn(0x7fbfb40aebc0 0x7fbfb40af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 --2- 172.21.15.133:0/2589858794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfb40b1e50 0x7fbfb4146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 --2- 172.21.15.133:0/2589858794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb406e8b0 0x7fbfb415eb40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 -- 172.21.15.133:0/2589858794 >> 172.21.15.133:0/2589858794 conn(0x7fbfb4066f70 msgr2=0x7fbfb406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 -- 172.21.15.133:0/2589858794 shutdown_connections 2024-04-25T10:02:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 -- 172.21.15.133:0/2589858794 wait complete. 2024-04-25T10:02:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 Processor -- start 2024-04-25T10:02:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 -- start start 2024-04-25T10:02:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfb406e8b0 0x7fbfb4136c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfb40aebc0 0x7fbfb4138f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40b1e50 0x7fbfb40ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb406dab0 con 0x7fbfb40b1e50 2024-04-25T10:02:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4147230 con 0x7fbfb406e8b0 2024-04-25T10:02:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb9097640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb406dc30 con 0x7fbfb40aebc0 2024-04-25T10:02:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40b1e50 0x7fbfb40ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40b1e50 0x7fbfb40ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54964/0 (socket says 172.21.15.133:54964) 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb359e640 1 -- 172.21.15.133:0/1935865864 learned_addr learned my addr 172.21.15.133:0/1935865864 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb2d9d640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfb406e8b0 0x7fbfb4136c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb259c640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfb40aebc0 0x7fbfb4138f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb359e640 1 -- 172.21.15.133:0/1935865864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfb40aebc0 msgr2=0x7fbfb4138f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb359e640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfb40aebc0 0x7fbfb4138f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb359e640 1 -- 172.21.15.133:0/1935865864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfb406e8b0 msgr2=0x7fbfb4136c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb359e640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfb406e8b0 0x7fbfb4136c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb359e640 1 -- 172.21.15.133:0/1935865864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4141600 con 0x7fbfb40b1e50 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfb359e640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40b1e50 0x7fbfb40ade30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00cef0 tx=0x7fbfac00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfab7fe640 1 -- 172.21.15.133:0/1935865864 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac01a070 con 0x7fbfb40b1e50 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfab7fe640 1 -- 172.21.15.133:0/1935865864 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac00d860 con 0x7fbfb40b1e50 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.176+0000 7fbfab7fe640 1 -- 172.21.15.133:0/1935865864 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00dc50 con 0x7fbfb40b1e50 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.180+0000 7fbfb9097640 1 -- 172.21.15.133:0/1935865864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb4139810 con 0x7fbfb40b1e50 2024-04-25T10:02:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.180+0000 7fbfb9097640 1 -- 172.21.15.133:0/1935865864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb4141b80 con 0x7fbfb40b1e50 2024-04-25T10:02:59.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.180+0000 7fbfb9097640 1 -- 172.21.15.133:0/1935865864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb4141d80 con 0x7fbfb40b1e50 2024-04-25T10:02:59.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.180+0000 7fbfab7fe640 1 -- 172.21.15.133:0/1935865864 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfac01d070 con 0x7fbfb40b1e50 2024-04-25T10:02:59.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.180+0000 7fbfab7fe640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbf9c066f60 0x7fbf9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.180+0000 7fbfab7fe640 1 -- 172.21.15.133:0/1935865864 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbfac085ce0 con 0x7fbfb40b1e50 2024-04-25T10:02:59.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.180+0000 7fbfb2d9d640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbf9c066f60 0x7fbf9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.180+0000 7fbfb2d9d640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbf9c066f60 0x7fbf9c069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40ad000 tx=0x7fbfa4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.184+0000 7fbfab7fe640 1 -- 172.21.15.133:0/1935865864 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfb4141d80 con 0x7fbfb40b1e50 2024-04-25T10:02:59.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.340+0000 7fbfb9097640 1 -- 172.21.15.133:0/1935865864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbfb4068e80 con 0x7fbfb40b1e50 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.340+0000 7fbfab7fe640 1 -- 172.21.15.133:0/1935865864 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbfac0569b0 con 0x7fbfb40b1e50 2024-04-25T10:02:59.348 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:02:59.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 -- 172.21.15.133:0/1935865864 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbf9c066f60 msgr2=0x7fbf9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbf9c066f60 0x7fbf9c069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40ad000 tx=0x7fbfa4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:59.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 -- 172.21.15.133:0/1935865864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40b1e50 msgr2=0x7fbfb40ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40b1e50 0x7fbfb40ade30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00cef0 tx=0x7fbfac00a710 comp rx=0 tx=0).stop 2024-04-25T10:02:59.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 -- 172.21.15.133:0/1935865864 shutdown_connections 2024-04-25T10:02:59.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbf9c066f60 0x7fbf9c069420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfb40b1e50 0x7fbfb40ade30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfb40aebc0 0x7fbfb4138f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 --2- 172.21.15.133:0/1935865864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfb406e8b0 0x7fbfb4136c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 -- 172.21.15.133:0/1935865864 >> 172.21.15.133:0/1935865864 conn(0x7fbfb4066f70 msgr2=0x7fbfb4140110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 -- 172.21.15.133:0/1935865864 shutdown_connections 2024-04-25T10:02:59.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.344+0000 7fbfb9097640 1 -- 172.21.15.133:0/1935865864 wait complete. 2024-04-25T10:02:59.367 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56b291640 1 Processor -- start 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56b291640 1 -- start start 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56b291640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe564145170 0x7fe56406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56b291640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe56406e4d0 0x7fe5640cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56b291640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe5640cdd60 0x7fe56415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56b291640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56406db00 con 0x7fe56406e4d0 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56b291640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe564145550 con 0x7fe5640cdd60 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56b291640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640ce2e0 con 0x7fe564145170 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56882c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe56406e4d0 0x7fe5640cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56882c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe56406e4d0 0x7fe5640cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54976/0 (socket says 172.21.15.133:54976) 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56882c640 1 -- 172.21.15.133:0/1918978576 learned_addr learned my addr 172.21.15.133:0/1918978576 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56882c640 1 -- 172.21.15.133:0/1918978576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe564145170 msgr2=0x7fe56406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56882c640 1 --2- 172.21.15.133:0/1918978576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe564145170 0x7fe56406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56882c640 1 -- 172.21.15.133:0/1918978576 >> v2:172.21.15.162:3300/0 conn(0x7fe5640cdd60 msgr2=0x7fe56415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56882c640 1 --2- 172.21.15.133:0/1918978576 >> v2:172.21.15.162:3300/0 conn(0x7fe5640cdd60 0x7fe56415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.472+0000 7fe56882c640 1 -- 172.21.15.133:0/1918978576 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56406d4b0 con 0x7fe56406e4d0 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56882c640 1 --2- 172.21.15.133:0/1918978576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe56406e4d0 0x7fe5640cd820 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe5640a8550 tx=0x7fe55401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459c8895b92549b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe55bfff640 1 -- 172.21.15.133:0/1918978576 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5540b0070 con 0x7fe56406e4d0 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe55bfff640 1 -- 172.21.15.133:0/1918978576 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5540ab600 con 0x7fe56406e4d0 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe55bfff640 1 -- 172.21.15.133:0/1918978576 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5540ab9d0 con 0x7fe56406e4d0 2024-04-25T10:02:59.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 -- 172.21.15.133:0/1918978576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe56406e4d0 msgr2=0x7fe5640cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 --2- 172.21.15.133:0/1918978576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe56406e4d0 0x7fe5640cd820 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe5640a8550 tx=0x7fe55401c520 comp rx=0 tx=0).stop 2024-04-25T10:02:59.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 -- 172.21.15.133:0/1918978576 shutdown_connections 2024-04-25T10:02:59.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 --2- 172.21.15.133:0/1918978576 >> v2:172.21.15.162:3300/0 conn(0x7fe5640cdd60 0x7fe56415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 --2- 172.21.15.133:0/1918978576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe56406e4d0 0x7fe5640cd820 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 --2- 172.21.15.133:0/1918978576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe564145170 0x7fe56406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 -- 172.21.15.133:0/1918978576 >> 172.21.15.133:0/1918978576 conn(0x7fe564066f70 msgr2=0x7fe56406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 -- 172.21.15.133:0/1918978576 shutdown_connections 2024-04-25T10:02:59.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 -- 172.21.15.133:0/1918978576 wait complete. 2024-04-25T10:02:59.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 Processor -- start 2024-04-25T10:02:59.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 -- start start 2024-04-25T10:02:59.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe56406e4d0 0x7fe56413f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5640cdd60 0x7fe56413e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe564145170 0x7fe5640ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56415f690 con 0x7fe564145170 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56415f510 con 0x7fe5640cdd60 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56b291640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56415f810 con 0x7fe56406e4d0 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56982e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe564145170 0x7fe5640ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56982e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe564145170 0x7fe5640ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54990/0 (socket says 172.21.15.133:54990) 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56982e640 1 -- 172.21.15.133:0/1964107451 learned_addr learned my addr 172.21.15.133:0/1964107451 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56902d640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe56406e4d0 0x7fe56413f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56882c640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5640cdd60 0x7fe56413e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56982e640 1 -- 172.21.15.133:0/1964107451 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe56406e4d0 msgr2=0x7fe56413f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56982e640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe56406e4d0 0x7fe56413f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56982e640 1 -- 172.21.15.133:0/1964107451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5640cdd60 msgr2=0x7fe56413e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56982e640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5640cdd60 0x7fe56413e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56982e640 1 -- 172.21.15.133:0/1964107451 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56413f8a0 con 0x7fe564145170 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56882c640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5640cdd60 0x7fe56413e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.476+0000 7fe56982e640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe564145170 0x7fe5640ad8a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00b970 tx=0x7fe55c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.480+0000 7fe559ffb640 1 -- 172.21.15.133:0/1964107451 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c019070 con 0x7fe564145170 2024-04-25T10:02:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.480+0000 7fe559ffb640 1 -- 172.21.15.133:0/1964107451 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c00cb60 con 0x7fe564145170 2024-04-25T10:02:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.480+0000 7fe56b291640 1 -- 172.21.15.133:0/1964107451 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5640ac800 con 0x7fe564145170 2024-04-25T10:02:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.480+0000 7fe559ffb640 1 -- 172.21.15.133:0/1964107451 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c014440 con 0x7fe564145170 2024-04-25T10:02:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.480+0000 7fe56b291640 1 -- 172.21.15.133:0/1964107451 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5640acdc0 con 0x7fe564145170 2024-04-25T10:02:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.480+0000 7fe56b291640 1 -- 172.21.15.133:0/1964107451 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe564068e80 con 0x7fe564145170 2024-04-25T10:02:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.480+0000 7fe559ffb640 1 -- 172.21.15.133:0/1964107451 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe55c01c070 con 0x7fe564145170 2024-04-25T10:02:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.480+0000 7fe559ffb640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe548066f60 0x7fe548069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.480+0000 7fe559ffb640 1 -- 172.21.15.133:0/1964107451 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe55c085190 con 0x7fe564145170 2024-04-25T10:02:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.480+0000 7fe56902d640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe548066f60 0x7fe548069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.480+0000 7fe56902d640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe548066f60 0x7fe548069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe560006fd0 tx=0x7fe560008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.484+0000 7fe559ffb640 1 -- 172.21.15.133:0/1964107451 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe55c055da0 con 0x7fe564145170 2024-04-25T10:02:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.644+0000 7fe56b291640 1 -- 172.21.15.133:0/1964107451 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe564142780 con 0x7fe564145170 2024-04-25T10:02:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.644+0000 7fe559ffb640 1 -- 172.21.15.133:0/1964107451 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe55c0556a0 con 0x7fe564145170 2024-04-25T10:02:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:02:59.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 -- 172.21.15.133:0/1964107451 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe548066f60 msgr2=0x7fe548069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe548066f60 0x7fe548069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe560006fd0 tx=0x7fe560008040 comp rx=0 tx=0).stop 2024-04-25T10:02:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 -- 172.21.15.133:0/1964107451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe564145170 msgr2=0x7fe5640ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe564145170 0x7fe5640ad8a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00b970 tx=0x7fe55c00be30 comp rx=0 tx=0).stop 2024-04-25T10:02:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 -- 172.21.15.133:0/1964107451 shutdown_connections 2024-04-25T10:02:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe548066f60 0x7fe548069420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe564145170 0x7fe5640ad8a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5640cdd60 0x7fe56413e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 --2- 172.21.15.133:0/1964107451 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe56406e4d0 0x7fe56413f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 -- 172.21.15.133:0/1964107451 >> 172.21.15.133:0/1964107451 conn(0x7fe564066f70 msgr2=0x7fe5640b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 -- 172.21.15.133:0/1964107451 shutdown_connections 2024-04-25T10:02:59.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.648+0000 7fe56b291640 1 -- 172.21.15.133:0/1964107451 wait complete. 2024-04-25T10:02:59.670 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:02:59.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.670 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:59.682 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:59.682 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:02:59.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.683 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:02:59.730 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:02:59.730 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:59.737 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:02:59.737 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:59.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:59.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.896+0000 7f83d554a640 1 Processor -- start 2024-04-25T10:02:59.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.896+0000 7f83d554a640 1 -- start start 2024-04-25T10:02:59.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.900+0000 7f83d554a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f83d015a9d0 0x7f83d00c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.900+0000 7f83d554a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d00cc5a0 con 0x7f83d006eb70 2024-04-25T10:02:59.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.900+0000 7f83d554a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d00cc880 con 0x7f83d015a5f0 2024-04-25T10:02:59.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.900+0000 7f83d554a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d00ccb60 con 0x7f83d015a9d0 2024-04-25T10:02:59.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.900+0000 7f83ce7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f83d006eb70 0x7f83d00cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35838/0 (socket says 172.21.15.133:35838) 2024-04-25T10:02:59.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.900+0000 7f83ce7fc640 1 -- 172.21.15.133:0/2536440296 learned_addr learned my addr 172.21.15.133:0/2536440296 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:02:59.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.900+0000 7f83cf7fe640 1 --2- 172.21.15.133:0/2536440296 >> v2:172.21.15.162:3300/0 conn(0x7f83d015a9d0 0x7f83d00c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.900+0000 7f83cf7fe640 1 -- 172.21.15.133:0/2536440296 >> v1:172.21.15.162:6789/0 conn(0x7f83d015a5f0 legacy=0x7f83d006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.900+0000 7f83cf7fe640 1 -- 172.21.15.133:0/2536440296 >> v1:172.21.15.133:6789/0 conn(0x7f83d006eb70 legacy=0x7f83d00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.900+0000 7f83cf7fe640 1 -- 172.21.15.133:0/2536440296 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d00cce40 con 0x7f83d015a9d0 2024-04-25T10:02:59.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83cf7fe640 1 --2- 172.21.15.133:0/2536440296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d015a9d0 0x7f83d00c5260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009d90 tx=0x7f83c801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae0452f803b7a360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83cdffb640 1 -- 172.21.15.133:0/2536440296 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c80b2070 con 0x7f83d015a9d0 2024-04-25T10:02:59.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83cdffb640 1 -- 172.21.15.133:0/2536440296 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c80a5940 con 0x7f83d015a9d0 2024-04-25T10:02:59.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83cdffb640 1 -- 172.21.15.133:0/2536440296 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c80a5d10 con 0x7f83d015a9d0 2024-04-25T10:02:59.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83d554a640 1 -- 172.21.15.133:0/2536440296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d015a9d0 msgr2=0x7f83d00c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83d554a640 1 --2- 172.21.15.133:0/2536440296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d015a9d0 0x7f83d00c5260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009d90 tx=0x7f83c801c970 comp rx=0 tx=0).stop 2024-04-25T10:02:59.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83d554a640 1 -- 172.21.15.133:0/2536440296 shutdown_connections 2024-04-25T10:02:59.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83d554a640 1 --2- 172.21.15.133:0/2536440296 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d015a9d0 0x7f83d00c5260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83d554a640 1 -- 172.21.15.133:0/2536440296 >> 172.21.15.133:0/2536440296 conn(0x7f83d0067000 msgr2=0x7f83d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83d554a640 1 -- 172.21.15.133:0/2536440296 shutdown_connections 2024-04-25T10:02:59.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83d554a640 1 -- 172.21.15.133:0/2536440296 wait complete. 2024-04-25T10:02:59.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83d554a640 1 Processor -- start 2024-04-25T10:02:59.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.904+0000 7f83d554a640 1 -- start start 2024-04-25T10:02:59.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83d554a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d006eb70 0x7f83d0149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83d554a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d015a5f0 0x7f83d0153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83d554a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d015a9d0 0x7f83d00b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83d554a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00ceb60 con 0x7f83d006eb70 2024-04-25T10:02:59.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83d554a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00ce9e0 con 0x7f83d015a9d0 2024-04-25T10:02:59.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83d554a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d0168500 con 0x7f83d015a5f0 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83ceffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d006eb70 0x7f83d0149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83ceffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d006eb70 0x7f83d0149090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54996/0 (socket says 172.21.15.133:54996) 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83ceffd640 1 -- 172.21.15.133:0/2287336721 learned_addr learned my addr 172.21.15.133:0/2287336721 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83cf7fe640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d015a9d0 0x7f83d00b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83ce7fc640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d015a5f0 0x7f83d0153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83ceffd640 1 -- 172.21.15.133:0/2287336721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d015a5f0 msgr2=0x7f83d0153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83ceffd640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d015a5f0 0x7f83d0153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83ceffd640 1 -- 172.21.15.133:0/2287336721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d015a9d0 msgr2=0x7f83d00b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83ceffd640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d015a9d0 0x7f83d00b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83ceffd640 1 -- 172.21.15.133:0/2287336721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d01528d0 con 0x7f83d006eb70 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83ce7fc640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d015a5f0 0x7f83d0153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:59.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83ceffd640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d006eb70 0x7f83d0149090 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c420 tx=0x7f83c000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83b77fe640 1 -- 172.21.15.133:0/2287336721 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c00052d0 con 0x7f83d006eb70 2024-04-25T10:02:59.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83b77fe640 1 -- 172.21.15.133:0/2287336721 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c0005470 con 0x7f83d006eb70 2024-04-25T10:02:59.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83d554a640 1 -- 172.21.15.133:0/2287336721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d0152bb0 con 0x7f83d006eb70 2024-04-25T10:02:59.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83b77fe640 1 -- 172.21.15.133:0/2287336721 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0007260 con 0x7f83d006eb70 2024-04-25T10:02:59.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83d554a640 1 -- 172.21.15.133:0/2287336721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d0151dc0 con 0x7f83d006eb70 2024-04-25T10:02:59.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83d554a640 1 -- 172.21.15.133:0/2287336721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d0068ef0 con 0x7f83d006eb70 2024-04-25T10:02:59.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83b77fe640 1 -- 172.21.15.133:0/2287336721 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83c0007400 con 0x7f83d006eb70 2024-04-25T10:02:59.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.908+0000 7f83b77fe640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83bc066f60 0x7f83bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.912+0000 7f83b77fe640 1 -- 172.21.15.133:0/2287336721 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f83c00848e0 con 0x7f83d006eb70 2024-04-25T10:02:59.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.912+0000 7f83ce7fc640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83bc066f60 0x7f83bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.912+0000 7f83ce7fc640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83bc066f60 0x7f83bc069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f83d00b3df0 tx=0x7f83c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:02:59.916+0000 7f83b77fe640 1 -- 172.21.15.133:0/2287336721 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83c00554f0 con 0x7f83d006eb70 2024-04-25T10:03:00.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.064+0000 7f83d554a640 1 -- 172.21.15.133:0/2287336721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83d0152130 con 0x7f83d006eb70 2024-04-25T10:03:00.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83b77fe640 1 -- 172.21.15.133:0/2287336721 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f83c0054df0 con 0x7f83d006eb70 2024-04-25T10:03:00.073 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:00.073 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 -- 172.21.15.133:0/2287336721 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83bc066f60 msgr2=0x7f83bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83bc066f60 0x7f83bc069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f83d00b3df0 tx=0x7f83c4009040 comp rx=0 tx=0).stop 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 -- 172.21.15.133:0/2287336721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d006eb70 msgr2=0x7f83d0149090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d006eb70 0x7f83d0149090 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c420 tx=0x7f83c000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 -- 172.21.15.133:0/2287336721 shutdown_connections 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d015a9d0 0x7f83d00b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d015a5f0 0x7f83d0153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83bc066f60 0x7f83bc069420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 --2- 172.21.15.133:0/2287336721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d006eb70 0x7f83d0149090 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 -- 172.21.15.133:0/2287336721 >> 172.21.15.133:0/2287336721 conn(0x7f83d0067000 msgr2=0x7f83d015d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 -- 172.21.15.133:0/2287336721 shutdown_connections 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.068+0000 7f83d554a640 1 -- 172.21.15.133:0/2287336721 wait complete. 2024-04-25T10:03:00.092 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:00.092 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:00.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.208+0000 7f2bfd9dc640 1 Processor -- start 2024-04-25T10:03:00.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.208+0000 7f2bfd9dc640 1 -- start start 2024-04-25T10:03:00.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bfd9dc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2bf8143ea0 0x7f2bf80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bfd9dc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2bf80b2150 0x7f2bf806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bfd9dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf815b9b0 0x7f2bf806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bfd9dc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf815ac50 con 0x7f2bf80b2150 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bfd9dc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf80cd180 con 0x7f2bf8143ea0 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bfd9dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf815aed0 con 0x7f2bf815b9b0 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bee7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2bf80b2150 0x7f2bf806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bee7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2bf80b2150 0x7f2bf806e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55000/0 (socket says 172.21.15.133:55000) 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bee7fc640 1 -- 172.21.15.133:0/3633431089 learned_addr learned my addr 172.21.15.133:0/3633431089 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bf6ffd640 1 --2- 172.21.15.133:0/3633431089 >> v2:172.21.15.162:3300/0 conn(0x7f2bf8143ea0 0x7f2bf80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bf77fe640 1 --2- 172.21.15.133:0/3633431089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf815b9b0 0x7f2bf806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bee7fc640 1 -- 172.21.15.133:0/3633431089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf815b9b0 msgr2=0x7f2bf806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bee7fc640 1 --2- 172.21.15.133:0/3633431089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf815b9b0 0x7f2bf806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bee7fc640 1 -- 172.21.15.133:0/3633431089 >> v2:172.21.15.162:3300/0 conn(0x7f2bf8143ea0 msgr2=0x7f2bf80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bee7fc640 1 --2- 172.21.15.133:0/3633431089 >> v2:172.21.15.162:3300/0 conn(0x7f2bf8143ea0 0x7f2bf80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bee7fc640 1 -- 172.21.15.133:0/3633431089 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf815b1b0 con 0x7f2bf80b2150 2024-04-25T10:03:00.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bee7fc640 1 --2- 172.21.15.133:0/3633431089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf80b2150 0x7f2bf806e3b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80b4b20 tx=0x7f2be001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31438ebdcf9332c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bf67fc640 1 -- 172.21.15.133:0/3633431089 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be00ab5f0 con 0x7f2bf80b2150 2024-04-25T10:03:00.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bf67fc640 1 -- 172.21.15.133:0/3633431089 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be00b0070 con 0x7f2bf80b2150 2024-04-25T10:03:00.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.212+0000 7f2bf67fc640 1 -- 172.21.15.133:0/3633431089 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be00ab9b0 con 0x7f2bf80b2150 2024-04-25T10:03:00.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3633431089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf80b2150 msgr2=0x7f2bf806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 --2- 172.21.15.133:0/3633431089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf80b2150 0x7f2bf806e3b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80b4b20 tx=0x7f2be001c520 comp rx=0 tx=0).stop 2024-04-25T10:03:00.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3633431089 shutdown_connections 2024-04-25T10:03:00.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 --2- 172.21.15.133:0/3633431089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf815b9b0 0x7f2bf806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 --2- 172.21.15.133:0/3633431089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf80b2150 0x7f2bf806e3b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 --2- 172.21.15.133:0/3633431089 >> v2:172.21.15.162:3300/0 conn(0x7f2bf8143ea0 0x7f2bf80cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3633431089 >> 172.21.15.133:0/3633431089 conn(0x7f2bf8066f70 msgr2=0x7f2bf806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3633431089 shutdown_connections 2024-04-25T10:03:00.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3633431089 wait complete. 2024-04-25T10:03:00.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 Processor -- start 2024-04-25T10:03:00.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 -- start start 2024-04-25T10:03:00.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf80b2150 0x7f2bf813db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf8143ea0 0x7f2bf8137110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf815b9b0 0x7f2bf8137650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf815ede0 con 0x7f2bf815b9b0 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf815ec60 con 0x7f2bf8143ea0 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf815ef60 con 0x7f2bf80b2150 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf815b9b0 0x7f2bf8137650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf815b9b0 0x7f2bf8137650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55016/0 (socket says 172.21.15.133:55016) 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf77fe640 1 -- 172.21.15.133:0/3028981634 learned_addr learned my addr 172.21.15.133:0/3028981634 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bee7fc640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf8143ea0 0x7f2bf8137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf6ffd640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf80b2150 0x7f2bf813db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf77fe640 1 -- 172.21.15.133:0/3028981634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf80b2150 msgr2=0x7f2bf813db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf77fe640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf80b2150 0x7f2bf813db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf77fe640 1 -- 172.21.15.133:0/3028981634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf8143ea0 msgr2=0x7f2bf8137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf77fe640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf8143ea0 0x7f2bf8137110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf77fe640 1 -- 172.21.15.133:0/3028981634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf815dfd0 con 0x7f2bf815b9b0 2024-04-25T10:03:00.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf77fe640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf815b9b0 0x7f2bf8137650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0007de0 tx=0x7f2bf000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf4ff9640 1 -- 172.21.15.133:0/3028981634 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0014620 con 0x7f2bf815b9b0 2024-04-25T10:03:00.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf4ff9640 1 -- 172.21.15.133:0/3028981634 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf0019070 con 0x7f2bf815b9b0 2024-04-25T10:03:00.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3028981634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf813f850 con 0x7f2bf815b9b0 2024-04-25T10:03:00.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bf4ff9640 1 -- 172.21.15.133:0/3028981634 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0014a20 con 0x7f2bf815b9b0 2024-04-25T10:03:00.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.216+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3028981634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf8135090 con 0x7f2bf815b9b0 2024-04-25T10:03:00.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.220+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3028981634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf8068e80 con 0x7f2bf815b9b0 2024-04-25T10:03:00.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.220+0000 7f2bf4ff9640 1 -- 172.21.15.133:0/3028981634 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bf0014bc0 con 0x7f2bf815b9b0 2024-04-25T10:03:00.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.220+0000 7f2bf4ff9640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bd8066f60 0x7f2bd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.220+0000 7f2bf4ff9640 1 -- 172.21.15.133:0/3028981634 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2bf0085b50 con 0x7f2bf815b9b0 2024-04-25T10:03:00.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.220+0000 7f2bf6ffd640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bd8066f60 0x7f2bd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.220+0000 7f2bf6ffd640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bd8066f60 0x7f2bd8069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8139820 tx=0x7f2be8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.224+0000 7f2bf4ff9640 1 -- 172.21.15.133:0/3028981634 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bf0056110 con 0x7f2bf815b9b0 2024-04-25T10:03:00.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.380+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3028981634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2bf80b4830 con 0x7f2bf815b9b0 2024-04-25T10:03:00.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.380+0000 7f2bf4ff9640 1 -- 172.21.15.133:0/3028981634 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2bf005a1b0 con 0x7f2bf815b9b0 2024-04-25T10:03:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:03:00.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3028981634 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bd8066f60 msgr2=0x7f2bd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bd8066f60 0x7f2bd8069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8139820 tx=0x7f2be8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:00.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3028981634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf815b9b0 msgr2=0x7f2bf8137650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf815b9b0 0x7f2bf8137650 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0007de0 tx=0x7f2bf000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3028981634 shutdown_connections 2024-04-25T10:03:00.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bd8066f60 0x7f2bd8069420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf815b9b0 0x7f2bf8137650 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bf8143ea0 0x7f2bf8137110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 --2- 172.21.15.133:0/3028981634 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bf80b2150 0x7f2bf813db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3028981634 >> 172.21.15.133:0/3028981634 conn(0x7f2bf8066f70 msgr2=0x7f2bf806e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3028981634 shutdown_connections 2024-04-25T10:03:00.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.384+0000 7f2bfd9dc640 1 -- 172.21.15.133:0/3028981634 wait complete. 2024-04-25T10:03:00.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:00.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f0717d74640 1 Processor -- start 2024-04-25T10:03:00.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f0717d74640 1 -- start start 2024-04-25T10:03:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f0717d74640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07100ce600 0x7f07100cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f0717d74640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071015ebd0 con 0x7f071015d9a0 2024-04-25T10:03:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f0717d74640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071015eeb0 con 0x7f07100d1920 2024-04-25T10:03:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f0717d74640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07100d0330 con 0x7f07100ce600 2024-04-25T10:03:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f071530f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07100ce600 0x7f07100cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f071530f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07100ce600 0x7f07100cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55022/0 (socket says 172.21.15.133:55022) 2024-04-25T10:03:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f071530f640 1 -- 172.21.15.133:0/3885343035 learned_addr learned my addr 172.21.15.133:0/3885343035 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f071530f640 1 -- 172.21.15.133:0/3885343035 >> v1:172.21.15.162:6789/0 conn(0x7f07100d1920 legacy=0x7f07100d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f071530f640 1 -- 172.21.15.133:0/3885343035 >> v1:172.21.15.133:6789/0 conn(0x7f071015d9a0 legacy=0x7f07100d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.520+0000 7f071530f640 1 -- 172.21.15.133:0/3885343035 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07100d0610 con 0x7f07100ce600 2024-04-25T10:03:00.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f071530f640 1 --2- 172.21.15.133:0/3885343035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07100ce600 0x7f07100cea90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0700009870 tx=0x7f070001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74a54f004f62693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0714b0e640 1 -- 172.21.15.133:0/3885343035 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000b1070 con 0x7f07100ce600 2024-04-25T10:03:00.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0714b0e640 1 -- 172.21.15.133:0/3885343035 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07000ac600 con 0x7f07100ce600 2024-04-25T10:03:00.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0714b0e640 1 -- 172.21.15.133:0/3885343035 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000ac9d0 con 0x7f07100ce600 2024-04-25T10:03:00.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 -- 172.21.15.133:0/3885343035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07100ce600 msgr2=0x7f07100cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 --2- 172.21.15.133:0/3885343035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07100ce600 0x7f07100cea90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0700009870 tx=0x7f070001c490 comp rx=0 tx=0).stop 2024-04-25T10:03:00.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 -- 172.21.15.133:0/3885343035 shutdown_connections 2024-04-25T10:03:00.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 --2- 172.21.15.133:0/3885343035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07100ce600 0x7f07100cea90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 -- 172.21.15.133:0/3885343035 >> 172.21.15.133:0/3885343035 conn(0x7f071006bf80 msgr2=0x7f071006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 -- 172.21.15.133:0/3885343035 shutdown_connections 2024-04-25T10:03:00.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 -- 172.21.15.133:0/3885343035 wait complete. 2024-04-25T10:03:00.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 Processor -- start 2024-04-25T10:03:00.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 -- start start 2024-04-25T10:03:00.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07100ce600 0x7f0710155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07100d1920 0x7f07100b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071015d9a0 0x7f07100b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071015e710 con 0x7f071015d9a0 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071015e590 con 0x7f07100ce600 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071015e890 con 0x7f07100d1920 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0716311640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071015d9a0 0x7f07100b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0716311640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071015d9a0 0x7f07100b4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55024/0 (socket says 172.21.15.133:55024) 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0716311640 1 -- 172.21.15.133:0/3671591685 learned_addr learned my addr 172.21.15.133:0/3671591685 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0715b10640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07100ce600 0x7f0710155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f071530f640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07100d1920 0x7f07100b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0716311640 1 -- 172.21.15.133:0/3671591685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07100d1920 msgr2=0x7f07100b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0716311640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07100d1920 0x7f07100b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0716311640 1 -- 172.21.15.133:0/3671591685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07100ce600 msgr2=0x7f0710155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0716311640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07100ce600 0x7f0710155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0716311640 1 -- 172.21.15.133:0/3671591685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07100c8370 con 0x7f071015d9a0 2024-04-25T10:03:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0716311640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071015d9a0 0x7f07100b4b30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f070800b9a0 tx=0x7f070800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f07067fc640 1 -- 172.21.15.133:0/3671591685 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708019070 con 0x7f071015d9a0 2024-04-25T10:03:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f07067fc640 1 -- 172.21.15.133:0/3671591685 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070800cba0 con 0x7f071015d9a0 2024-04-25T10:03:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 -- 172.21.15.133:0/3671591685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07100c8650 con 0x7f071015d9a0 2024-04-25T10:03:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f07067fc640 1 -- 172.21.15.133:0/3671591685 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708014440 con 0x7f071015d9a0 2024-04-25T10:03:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.524+0000 7f0717d74640 1 -- 172.21.15.133:0/3671591685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07100cb640 con 0x7f071015d9a0 2024-04-25T10:03:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.528+0000 7f0717d74640 1 -- 172.21.15.133:0/3671591685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0710068e80 con 0x7f071015d9a0 2024-04-25T10:03:00.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.528+0000 7f07067fc640 1 -- 172.21.15.133:0/3671591685 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f070801c070 con 0x7f071015d9a0 2024-04-25T10:03:00.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.528+0000 7f07067fc640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06f4066f60 0x7f06f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.528+0000 7f07067fc640 1 -- 172.21.15.133:0/3671591685 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f07080850c0 con 0x7f071015d9a0 2024-04-25T10:03:00.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.528+0000 7f0715b10640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06f4066f60 0x7f06f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.528+0000 7f0715b10640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06f4066f60 0x7f06f4069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f070c001650 tx=0x7f070c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.532+0000 7f07067fc640 1 -- 172.21.15.133:0/3671591685 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0708055c70 con 0x7f071015d9a0 2024-04-25T10:03:00.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.688+0000 7f0717d74640 1 -- 172.21.15.133:0/3671591685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07100cdf20 con 0x7f071015d9a0 2024-04-25T10:03:00.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.688+0000 7f07067fc640 1 -- 172.21.15.133:0/3671591685 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0708055570 con 0x7f071015d9a0 2024-04-25T10:03:00.697 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:03:00.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 -- 172.21.15.133:0/3671591685 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06f4066f60 msgr2=0x7f06f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06f4066f60 0x7f06f4069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f070c001650 tx=0x7f070c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:00.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 -- 172.21.15.133:0/3671591685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071015d9a0 msgr2=0x7f07100b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071015d9a0 0x7f07100b4b30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f070800b9a0 tx=0x7f070800be60 comp rx=0 tx=0).stop 2024-04-25T10:03:00.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 -- 172.21.15.133:0/3671591685 shutdown_connections 2024-04-25T10:03:00.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06f4066f60 0x7f06f4069420 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f071015d9a0 0x7f07100b4b30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07100d1920 0x7f07100b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 --2- 172.21.15.133:0/3671591685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07100ce600 0x7f0710155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 -- 172.21.15.133:0/3671591685 >> 172.21.15.133:0/3671591685 conn(0x7f071006bf80 msgr2=0x7f07100b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 -- 172.21.15.133:0/3671591685 shutdown_connections 2024-04-25T10:03:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.692+0000 7f0717d74640 1 -- 172.21.15.133:0/3671591685 wait complete. 2024-04-25T10:03:00.715 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:00.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b1a9f3640 1 Processor -- start 2024-04-25T10:03:00.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b1a9f3640 1 -- start start 2024-04-25T10:03:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b1a9f3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b1415bc90 0x7f8b140cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b1a9f3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b140d01d0 0x7f8b140b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b1a9f3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b140cc6a0 con 0x7f8b1415b8b0 2024-04-25T10:03:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b1a9f3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b140d0640 con 0x7f8b140d01d0 2024-04-25T10:03:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b1a9f3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140cc980 con 0x7f8b1415bc90 2024-04-25T10:03:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b18f90640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b140d01d0 0x7f8b140b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b18f90640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b140d01d0 0x7f8b140b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55034/0 (socket says 172.21.15.133:55034) 2024-04-25T10:03:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b18f90640 1 -- 172.21.15.133:0/3175040279 learned_addr learned my addr 172.21.15.133:0/3175040279 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b137fe640 1 --2- 172.21.15.133:0/3175040279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b1415bc90 0x7f8b140cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b18f90640 1 -- 172.21.15.133:0/3175040279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b1415bc90 msgr2=0x7f8b140cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b18f90640 1 --2- 172.21.15.133:0/3175040279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b1415bc90 0x7f8b140cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b18f90640 1 -- 172.21.15.133:0/3175040279 >> v1:172.21.15.162:6789/0 conn(0x7f8b1415b8b0 legacy=0x7f8b140ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.820+0000 7f8b18f90640 1 -- 172.21.15.133:0/3175040279 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b140ccc60 con 0x7f8b140d01d0 2024-04-25T10:03:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b18f90640 1 --2- 172.21.15.133:0/3175040279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b140d01d0 0x7f8b140b0aa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009e50 tx=0x7f8b0c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc945968eca561fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b12ffd640 1 -- 172.21.15.133:0/3175040279 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0a5690 con 0x7f8b140d01d0 2024-04-25T10:03:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b12ffd640 1 -- 172.21.15.133:0/3175040279 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c0a5830 con 0x7f8b140d01d0 2024-04-25T10:03:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b12ffd640 1 -- 172.21.15.133:0/3175040279 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0a5c00 con 0x7f8b140d01d0 2024-04-25T10:03:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3175040279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b140d01d0 msgr2=0x7f8b140b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 --2- 172.21.15.133:0/3175040279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b140d01d0 0x7f8b140b0aa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009e50 tx=0x7f8b0c01c960 comp rx=0 tx=0).stop 2024-04-25T10:03:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3175040279 shutdown_connections 2024-04-25T10:03:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 --2- 172.21.15.133:0/3175040279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b140d01d0 0x7f8b140b0aa0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 --2- 172.21.15.133:0/3175040279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b1415bc90 0x7f8b140cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3175040279 >> 172.21.15.133:0/3175040279 conn(0x7f8b1409a990 msgr2=0x7f8b140d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3175040279 shutdown_connections 2024-04-25T10:03:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3175040279 wait complete. 2024-04-25T10:03:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 Processor -- start 2024-04-25T10:03:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 -- start start 2024-04-25T10:03:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b140d01d0 0x7f8b1414f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1415b8b0 0x7f8b14155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b1415bc90 0x7f8b140c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140b17f0 con 0x7f8b1415b8b0 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140cc440 con 0x7f8b1415bc90 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.824+0000 7f8b1a9f3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140b1970 con 0x7f8b140d01d0 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1415b8b0 0x7f8b14155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1415b8b0 0x7f8b14155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55048/0 (socket says 172.21.15.133:55048) 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b137fe640 1 -- 172.21.15.133:0/3713401295 learned_addr learned my addr 172.21.15.133:0/3713401295 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b13fff640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b140d01d0 0x7f8b1414f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b137fe640 1 -- 172.21.15.133:0/3713401295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b140d01d0 msgr2=0x7f8b1414f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b137fe640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b140d01d0 0x7f8b1414f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b18f90640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b1415bc90 0x7f8b140c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b137fe640 1 -- 172.21.15.133:0/3713401295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b1415bc90 msgr2=0x7f8b140c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b137fe640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b1415bc90 0x7f8b140c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b137fe640 1 -- 172.21.15.133:0/3713401295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b14154ed0 con 0x7f8b1415b8b0 2024-04-25T10:03:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b137fe640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1415b8b0 0x7f8b14155f70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800d5f0 tx=0x7f8b0800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b10ff9640 1 -- 172.21.15.133:0/3713401295 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080125f0 con 0x7f8b1415b8b0 2024-04-25T10:03:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b10ff9640 1 -- 172.21.15.133:0/3713401295 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b08019070 con 0x7f8b1415b8b0 2024-04-25T10:03:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3713401295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b141551b0 con 0x7f8b1415b8b0 2024-04-25T10:03:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b10ff9640 1 -- 172.21.15.133:0/3713401295 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080129b0 con 0x7f8b1415b8b0 2024-04-25T10:03:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3713401295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b14155a50 con 0x7f8b1415b8b0 2024-04-25T10:03:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3713401295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b14067c70 con 0x7f8b1415b8b0 2024-04-25T10:03:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b10ff9640 1 -- 172.21.15.133:0/3713401295 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b08012b50 con 0x7f8b1415b8b0 2024-04-25T10:03:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b10ff9640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b00066f60 0x7f8b00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b10ff9640 1 -- 172.21.15.133:0/3713401295 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8b08085850 con 0x7f8b1415b8b0 2024-04-25T10:03:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b13fff640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b00066f60 0x7f8b00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.828+0000 7f8b13fff640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b00066f60 0x7f8b00069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8b14166890 tx=0x7f8b04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.832+0000 7f8b10ff9640 1 -- 172.21.15.133:0/3713401295 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b08056460 con 0x7f8b1415b8b0 2024-04-25T10:03:00.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.980+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3713401295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b140d1dc0 con 0x7f8b1415b8b0 2024-04-25T10:03:00.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.980+0000 7f8b10ff9640 1 -- 172.21.15.133:0/3713401295 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8b08055d60 con 0x7f8b1415b8b0 2024-04-25T10:03:00.988 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:03:00.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3713401295 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b00066f60 msgr2=0x7f8b00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b00066f60 0x7f8b00069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8b14166890 tx=0x7f8b04009040 comp rx=0 tx=0).stop 2024-04-25T10:03:00.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3713401295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1415b8b0 msgr2=0x7f8b14155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1415b8b0 0x7f8b14155f70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800d5f0 tx=0x7f8b0800dab0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3713401295 shutdown_connections 2024-04-25T10:03:00.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b1415bc90 0x7f8b140c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b00066f60 0x7f8b00069420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b1415b8b0 0x7f8b14155f70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 --2- 172.21.15.133:0/3713401295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b140d01d0 0x7f8b1414f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3713401295 >> 172.21.15.133:0/3713401295 conn(0x7f8b1409a990 msgr2=0x7f8b1406f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3713401295 shutdown_connections 2024-04-25T10:03:00.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:00.984+0000 7f8b1a9f3640 1 -- 172.21.15.133:0/3713401295 wait complete. 2024-04-25T10:03:01.006 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:01.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc237fc640 1 Processor -- start 2024-04-25T10:03:01.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc237fc640 1 -- start start 2024-04-25T10:03:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc237fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc1c0cc330 0x7fdc1c166530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc237fc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c0ccd80 con 0x7fdc1c15ba80 2024-04-25T10:03:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc237fc640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c0cadc0 con 0x7fdc1c15cb80 2024-04-25T10:03:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc237fc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c15be60 con 0x7fdc1c0cc330 2024-04-25T10:03:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc21598640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdc1c15ba80 0x7fdc1c0afd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35852/0 (socket says 172.21.15.133:35852) 2024-04-25T10:03:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc21598640 1 -- 172.21.15.133:0/156683174 learned_addr learned my addr 172.21.15.133:0/156683174 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc21d99640 1 --2- 172.21.15.133:0/156683174 >> v2:172.21.15.133:3300/0 conn(0x7fdc1c0cc330 0x7fdc1c166530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc21d99640 1 -- 172.21.15.133:0/156683174 >> v1:172.21.15.162:6789/0 conn(0x7fdc1c15cb80 legacy=0x7fdc1c0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc21d99640 1 -- 172.21.15.133:0/156683174 >> v1:172.21.15.133:6789/0 conn(0x7fdc1c15ba80 legacy=0x7fdc1c0afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.116+0000 7fdc21d99640 1 -- 172.21.15.133:0/156683174 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0ccaa0 con 0x7fdc1c0cc330 2024-04-25T10:03:01.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc21d99640 1 --2- 172.21.15.133:0/156683174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1c0cc330 0x7fdc1c166530 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009d90 tx=0x7fdc1401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471f203cfc7fd82f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc13fff640 1 -- 172.21.15.133:0/156683174 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140b2070 con 0x7fdc1c0cc330 2024-04-25T10:03:01.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc13fff640 1 -- 172.21.15.133:0/156683174 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc140a58e0 con 0x7fdc1c0cc330 2024-04-25T10:03:01.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc13fff640 1 -- 172.21.15.133:0/156683174 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140a5cb0 con 0x7fdc1c0cc330 2024-04-25T10:03:01.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 -- 172.21.15.133:0/156683174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1c0cc330 msgr2=0x7fdc1c166530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 --2- 172.21.15.133:0/156683174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1c0cc330 0x7fdc1c166530 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009d90 tx=0x7fdc1401c930 comp rx=0 tx=0).stop 2024-04-25T10:03:01.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 -- 172.21.15.133:0/156683174 shutdown_connections 2024-04-25T10:03:01.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 --2- 172.21.15.133:0/156683174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1c0cc330 0x7fdc1c166530 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 -- 172.21.15.133:0/156683174 >> 172.21.15.133:0/156683174 conn(0x7fdc1c0d0570 msgr2=0x7fdc1c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 -- 172.21.15.133:0/156683174 shutdown_connections 2024-04-25T10:03:01.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 -- 172.21.15.133:0/156683174 wait complete. 2024-04-25T10:03:01.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 Processor -- start 2024-04-25T10:03:01.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 -- start start 2024-04-25T10:03:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1c0cc330 0x7fdc1c152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc1c15ba80 0x7fdc1c155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc1c15cb80 0x7fdc1c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1696a0 con 0x7fdc1c0cc330 2024-04-25T10:03:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cc740 con 0x7fdc1c15cb80 2024-04-25T10:03:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c169820 con 0x7fdc1c15ba80 2024-04-25T10:03:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc21598640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1c0cc330 0x7fdc1c152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc21598640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1c0cc330 0x7fdc1c152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55070/0 (socket says 172.21.15.133:55070) 2024-04-25T10:03:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc21598640 1 -- 172.21.15.133:0/3043424853 learned_addr learned my addr 172.21.15.133:0/3043424853 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc21598640 1 -- 172.21.15.133:0/3043424853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc1c15ba80 msgr2=0x7fdc1c155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc21598640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc1c15ba80 0x7fdc1c155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc21598640 1 -- 172.21.15.133:0/3043424853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc1c15cb80 msgr2=0x7fdc1c0b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc21598640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc1c15cb80 0x7fdc1c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc21598640 1 -- 172.21.15.133:0/3043424853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c1560e0 con 0x7fdc1c0cc330 2024-04-25T10:03:01.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc21598640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1c0cc330 0x7fdc1c152d40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800c450 tx=0x7fdc1800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc11ffb640 1 -- 172.21.15.133:0/3043424853 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18017070 con 0x7fdc1c0cc330 2024-04-25T10:03:01.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc11ffb640 1 -- 172.21.15.133:0/3043424853 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc18005cf0 con 0x7fdc1c0cc330 2024-04-25T10:03:01.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 -- 172.21.15.133:0/3043424853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c1563c0 con 0x7fdc1c0cc330 2024-04-25T10:03:01.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc11ffb640 1 -- 172.21.15.133:0/3043424853 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18007290 con 0x7fdc1c0cc330 2024-04-25T10:03:01.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.120+0000 7fdc237fc640 1 -- 172.21.15.133:0/3043424853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c156980 con 0x7fdc1c0cc330 2024-04-25T10:03:01.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.124+0000 7fdc237fc640 1 -- 172.21.15.133:0/3043424853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c067c70 con 0x7fdc1c0cc330 2024-04-25T10:03:01.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.124+0000 7fdc11ffb640 1 -- 172.21.15.133:0/3043424853 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc1801a070 con 0x7fdc1c0cc330 2024-04-25T10:03:01.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.124+0000 7fdc11ffb640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc00066f90 0x7fdc00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.124+0000 7fdc11ffb640 1 -- 172.21.15.133:0/3043424853 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdc18084720 con 0x7fdc1c0cc330 2024-04-25T10:03:01.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.124+0000 7fdc20d97640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc00066f90 0x7fdc00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.124+0000 7fdc20d97640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc00066f90 0x7fdc00069450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009670 tx=0x7fdc0c006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.128+0000 7fdc11ffb640 1 -- 172.21.15.133:0/3043424853 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc18055330 con 0x7fdc1c0cc330 2024-04-25T10:03:01.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.272+0000 7fdc237fc640 1 -- 172.21.15.133:0/3043424853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdc1c06e480 con 0x7fdc1c0cc330 2024-04-25T10:03:01.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.272+0000 7fdc11ffb640 1 -- 172.21.15.133:0/3043424853 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdc18054c30 con 0x7fdc1c0cc330 2024-04-25T10:03:01.280 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:03:01.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 -- 172.21.15.133:0/3043424853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc00066f90 msgr2=0x7fdc00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc00066f90 0x7fdc00069450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009670 tx=0x7fdc0c006bd0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 -- 172.21.15.133:0/3043424853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1c0cc330 msgr2=0x7fdc1c152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1c0cc330 0x7fdc1c152d40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800c450 tx=0x7fdc1800c910 comp rx=0 tx=0).stop 2024-04-25T10:03:01.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 -- 172.21.15.133:0/3043424853 shutdown_connections 2024-04-25T10:03:01.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc1c15cb80 0x7fdc1c0b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc1c15ba80 0x7fdc1c155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc00066f90 0x7fdc00069450 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 --2- 172.21.15.133:0/3043424853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1c0cc330 0x7fdc1c152d40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 -- 172.21.15.133:0/3043424853 >> 172.21.15.133:0/3043424853 conn(0x7fdc1c0d0570 msgr2=0x7fdc1c0ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 -- 172.21.15.133:0/3043424853 shutdown_connections 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.276+0000 7fdc237fc640 1 -- 172.21.15.133:0/3043424853 wait complete. 2024-04-25T10:03:01.297 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21defee640 1 Processor -- start 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21defee640 1 -- start start 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21defee640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f21d80afb10 0x7f21d80cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21defee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d80cc000 0x7f21d80cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21defee640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d806ffc0 con 0x7f21d80cbc20 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21defee640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d806e360 con 0x7f21d80afb10 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21defee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d806e640 con 0x7f21d80cc000 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21dcd8a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f21d80cbc20 0x7f21d806ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35858/0 (socket says 172.21.15.133:35858) 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21dcd8a640 1 -- 172.21.15.133:0/2109792629 learned_addr learned my addr 172.21.15.133:0/2109792629 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d7fff640 1 --2- 172.21.15.133:0/2109792629 >> v2:172.21.15.162:3300/0 conn(0x7f21d80afb10 0x7f21d80cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21dd58b640 1 --2- 172.21.15.133:0/2109792629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d80cc000 0x7f21d80cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525389285 0 0) 0x7f21d806ffc0 con 0x7f21d80cbc20 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c0003530 con 0x7f21d80cbc20 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014433690 0 0) 0x7f21c0003530 con 0x7f21d80cbc20 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d80cc000 msgr2=0x7f21d80cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 --2- 172.21.15.133:0/2109792629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d80cc000 0x7f21d80cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 >> v2:172.21.15.162:3300/0 conn(0x7f21d80afb10 msgr2=0x7f21d80cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 --2- 172.21.15.133:0/2109792629 >> v2:172.21.15.162:3300/0 conn(0x7f21d80afb10 0x7f21d80cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d806e920 con 0x7f21d80cbc20 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f21cc001680 con 0x7f21d80cbc20 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 >> v1:172.21.15.133:6789/0 conn(0x7f21d80cbc20 legacy=0x7f21d806ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 --2- 172.21.15.133:0/2109792629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c00039c0 0x7f21c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 --2- 172.21.15.133:0/2109792629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d80cc000 0x7f21c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 --2- 172.21.15.133:0/2109792629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21c0004d50 0x7f21c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d806e360 con 0x7f21c00039c0 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d806e640 con 0x7f21c0004d50 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c000b9d0 con 0x7f21d80cc000 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21dcd8a640 1 --2- 172.21.15.133:0/2109792629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c00039c0 0x7f21c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21dcd8a640 1 -- 172.21.15.133:0/2109792629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d80cc000 msgr2=0x7f21c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21d7fff640 1 --2- 172.21.15.133:0/2109792629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d80cc000 0x7f21c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21dcd8a640 1 --2- 172.21.15.133:0/2109792629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d80cc000 0x7f21c0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21dcd8a640 1 -- 172.21.15.133:0/2109792629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21c0004d50 msgr2=0x7f21c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21dcd8a640 1 --2- 172.21.15.133:0/2109792629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21c0004d50 0x7f21c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.396+0000 7f21dcd8a640 1 -- 172.21.15.133:0/2109792629 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c0003530 con 0x7f21c00039c0 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21dcd8a640 1 --2- 172.21.15.133:0/2109792629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c00039c0 0x7f21c0003de0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f21cc001060 tx=0x7f21cc01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47427dbb9f91e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc007d80 con 0x7f21c00039c0 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc0a9020 con 0x7f21c00039c0 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21d77fe640 1 -- 172.21.15.133:0/2109792629 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0a5b30 con 0x7f21c00039c0 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21defee640 1 -- 172.21.15.133:0/2109792629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c00039c0 msgr2=0x7f21c0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21defee640 1 --2- 172.21.15.133:0/2109792629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c00039c0 0x7f21c0003de0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f21cc001060 tx=0x7f21cc01cca0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21dd58b640 1 -- 172.21.15.133:0/2109792629 reap_dead start 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21defee640 1 -- 172.21.15.133:0/2109792629 shutdown_connections 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21defee640 1 -- 172.21.15.133:0/2109792629 >> 172.21.15.133:0/2109792629 conn(0x7f21d80999d0 msgr2=0x7f21d80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21defee640 1 -- 172.21.15.133:0/2109792629 shutdown_connections 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21defee640 1 -- 172.21.15.133:0/2109792629 wait complete. 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21defee640 1 Processor -- start 2024-04-25T10:03:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.400+0000 7f21defee640 1 -- start start 2024-04-25T10:03:01.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21defee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21c00039c0 0x7f21d80b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21defee640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21d80c9550 0x7f21d80c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21defee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d80b4190 0x7f21d80b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21defee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80d0120 con 0x7f21d80b4190 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21defee640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80cffa0 con 0x7f21d80c9550 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21defee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d815d1a0 con 0x7f21c00039c0 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dd58b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d80b4190 0x7f21d80b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dd58b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d80b4190 0x7f21d80b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55092/0 (socket says 172.21.15.133:55092) 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dd58b640 1 -- 172.21.15.133:0/1908238215 learned_addr learned my addr 172.21.15.133:0/1908238215 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dcd8a640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21c00039c0 0x7f21d80b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21d7fff640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21d80c9550 0x7f21d80c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dd58b640 1 -- 172.21.15.133:0/1908238215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21c00039c0 msgr2=0x7f21d80b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dd58b640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21c00039c0 0x7f21d80b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dd58b640 1 -- 172.21.15.133:0/1908238215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21d80c9550 msgr2=0x7f21d80c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dd58b640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21d80c9550 0x7f21d80c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dd58b640 1 -- 172.21.15.133:0/1908238215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8161240 con 0x7f21d80b4190 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dd58b640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d80b4190 0x7f21d80b45f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f21d000ee50 tx=0x7f21d000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21d57fa640 1 -- 172.21.15.133:0/1908238215 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000bc50 con 0x7f21d80b4190 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21d57fa640 1 -- 172.21.15.133:0/1908238215 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d001a070 con 0x7f21d80b4190 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21defee640 1 -- 172.21.15.133:0/1908238215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d80c75f0 con 0x7f21d80b4190 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21d57fa640 1 -- 172.21.15.133:0/1908238215 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d0014400 con 0x7f21d80b4190 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21defee640 1 -- 172.21.15.133:0/1908238215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d80c7bb0 con 0x7f21d80b4190 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21defee640 1 -- 172.21.15.133:0/1908238215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d8067c70 con 0x7f21d80b4190 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21d57fa640 1 -- 172.21.15.133:0/1908238215 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21d0013070 con 0x7f21d80b4190 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21d57fa640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c4066f60 0x7f21c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21d57fa640 1 -- 172.21.15.133:0/1908238215 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f21d00854c0 con 0x7f21d80b4190 2024-04-25T10:03:01.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dcd8a640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c4066f60 0x7f21c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.404+0000 7f21dcd8a640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c4066f60 0x7f21c4069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0aa000 tx=0x7f21cc001a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.408+0000 7f21d57fa640 1 -- 172.21.15.133:0/1908238215 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21d00122c0 con 0x7f21d80b4190 2024-04-25T10:03:01.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.572+0000 7f21defee640 1 -- 172.21.15.133:0/1908238215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f21d80b64a0 con 0x7f21d80b4190 2024-04-25T10:03:01.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.572+0000 7f21d57fa640 1 -- 172.21.15.133:0/1908238215 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f21d00560d0 con 0x7f21d80b4190 2024-04-25T10:03:01.581 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:03:01.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 -- 172.21.15.133:0/1908238215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c4066f60 msgr2=0x7f21c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c4066f60 0x7f21c4069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0aa000 tx=0x7f21cc001a50 comp rx=0 tx=0).stop 2024-04-25T10:03:01.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 -- 172.21.15.133:0/1908238215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d80b4190 msgr2=0x7f21d80b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d80b4190 0x7f21d80b45f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f21d000ee50 tx=0x7f21d000b480 comp rx=0 tx=0).stop 2024-04-25T10:03:01.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 -- 172.21.15.133:0/1908238215 shutdown_connections 2024-04-25T10:03:01.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c4066f60 0x7f21c4069420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d80b4190 0x7f21d80b45f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21d80c9550 0x7f21d80c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 --2- 172.21.15.133:0/1908238215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21c00039c0 0x7f21d80b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 -- 172.21.15.133:0/1908238215 >> 172.21.15.133:0/1908238215 conn(0x7f21d80999d0 msgr2=0x7f21d806ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 -- 172.21.15.133:0/1908238215 shutdown_connections 2024-04-25T10:03:01.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.576+0000 7f21defee640 1 -- 172.21.15.133:0/1908238215 wait complete. 2024-04-25T10:03:01.599 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:03:01.599 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:01.608 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:01.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc84297640 1 Processor -- start 2024-04-25T10:03:01.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc84297640 1 -- start start 2024-04-25T10:03:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc84297640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdc7c158110 0x7fdc7c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc84297640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc7c1584f0 0x7fdc7c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc84297640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc7c06e460 0x7fdc7c06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc84297640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7c06edf0 con 0x7fdc7c06e460 2024-04-25T10:03:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc84297640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7c0cbe60 con 0x7fdc7c158110 2024-04-25T10:03:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc84297640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0b19b0 con 0x7fdc7c1584f0 2024-04-25T10:03:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc82834640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc7c06e460 0x7fdc7c06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc82834640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc7c06e460 0x7fdc7c06e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55102/0 (socket says 172.21.15.133:55102) 2024-04-25T10:03:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc82834640 1 -- 172.21.15.133:0/3342688910 learned_addr learned my addr 172.21.15.133:0/3342688910 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc81832640 1 --2- 172.21.15.133:0/3342688910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc7c1584f0 0x7fdc7c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc82033640 1 --2- 172.21.15.133:0/3342688910 >> v2:172.21.15.162:3300/0 conn(0x7fdc7c158110 0x7fdc7c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc82834640 1 -- 172.21.15.133:0/3342688910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc7c1584f0 msgr2=0x7fdc7c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc82834640 1 --2- 172.21.15.133:0/3342688910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc7c1584f0 0x7fdc7c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc82834640 1 -- 172.21.15.133:0/3342688910 >> v2:172.21.15.162:3300/0 conn(0x7fdc7c158110 msgr2=0x7fdc7c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc82834640 1 --2- 172.21.15.133:0/3342688910 >> v2:172.21.15.162:3300/0 conn(0x7fdc7c158110 0x7fdc7c0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc82834640 1 -- 172.21.15.133:0/3342688910 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c15b360 con 0x7fdc7c06e460 2024-04-25T10:03:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc81832640 1 --2- 172.21.15.133:0/3342688910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc7c1584f0 0x7fdc7c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc82834640 1 --2- 172.21.15.133:0/3342688910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc7c06e460 0x7fdc7c06e8b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0b4d40 tx=0x7fdc7401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abeb9758408e84f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.716+0000 7fdc81031640 1 -- 172.21.15.133:0/3342688910 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc740aa610 con 0x7fdc7c06e460 2024-04-25T10:03:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc81031640 1 -- 172.21.15.133:0/3342688910 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc740af070 con 0x7fdc7c06e460 2024-04-25T10:03:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc81031640 1 -- 172.21.15.133:0/3342688910 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc740aa9d0 con 0x7fdc7c06e460 2024-04-25T10:03:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 -- 172.21.15.133:0/3342688910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc7c06e460 msgr2=0x7fdc7c06e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 --2- 172.21.15.133:0/3342688910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc7c06e460 0x7fdc7c06e8b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0b4d40 tx=0x7fdc7401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 -- 172.21.15.133:0/3342688910 shutdown_connections 2024-04-25T10:03:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 --2- 172.21.15.133:0/3342688910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc7c06e460 0x7fdc7c06e8b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 --2- 172.21.15.133:0/3342688910 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc7c1584f0 0x7fdc7c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 --2- 172.21.15.133:0/3342688910 >> v2:172.21.15.162:3300/0 conn(0x7fdc7c158110 0x7fdc7c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 -- 172.21.15.133:0/3342688910 >> 172.21.15.133:0/3342688910 conn(0x7fdc7c067000 msgr2=0x7fdc7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 -- 172.21.15.133:0/3342688910 shutdown_connections 2024-04-25T10:03:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 -- 172.21.15.133:0/3342688910 wait complete. 2024-04-25T10:03:01.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 Processor -- start 2024-04-25T10:03:01.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 -- start start 2024-04-25T10:03:01.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc7c06e460 0x7fdc7c13f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc7c158110 0x7fdc7c0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc7c1584f0 0x7fdc7c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0cd7f0 con 0x7fdc7c06e460 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0cd670 con 0x7fdc7c1584f0 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc84297640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c15eb10 con 0x7fdc7c158110 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc82033640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc7c06e460 0x7fdc7c13f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc82033640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc7c06e460 0x7fdc7c13f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55114/0 (socket says 172.21.15.133:55114) 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc82033640 1 -- 172.21.15.133:0/3755152769 learned_addr learned my addr 172.21.15.133:0/3755152769 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc81832640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc7c158110 0x7fdc7c0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc82834640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc7c1584f0 0x7fdc7c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc82033640 1 -- 172.21.15.133:0/3755152769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc7c158110 msgr2=0x7fdc7c0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc82033640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc7c158110 0x7fdc7c0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc82033640 1 -- 172.21.15.133:0/3755152769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc7c1584f0 msgr2=0x7fdc7c0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc82033640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc7c1584f0 0x7fdc7c0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc82033640 1 -- 172.21.15.133:0/3755152769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c0c8100 con 0x7fdc7c06e460 2024-04-25T10:03:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc82834640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc7c1584f0 0x7fdc7c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.720+0000 7fdc82033640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc7c06e460 0x7fdc7c13f720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00d830 tx=0x7fdc6c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.724+0000 7fdc7affd640 1 -- 172.21.15.133:0/3755152769 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c018020 con 0x7fdc7c06e460 2024-04-25T10:03:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.724+0000 7fdc7affd640 1 -- 172.21.15.133:0/3755152769 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6c0096a0 con 0x7fdc7c06e460 2024-04-25T10:03:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.724+0000 7fdc84297640 1 -- 172.21.15.133:0/3755152769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c0c83e0 con 0x7fdc7c06e460 2024-04-25T10:03:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.724+0000 7fdc7affd640 1 -- 172.21.15.133:0/3755152769 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c009a50 con 0x7fdc7c06e460 2024-04-25T10:03:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.724+0000 7fdc84297640 1 -- 172.21.15.133:0/3755152769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c0c89a0 con 0x7fdc7c06e460 2024-04-25T10:03:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.724+0000 7fdc84297640 1 -- 172.21.15.133:0/3755152769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c06bc20 con 0x7fdc7c06e460 2024-04-25T10:03:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.724+0000 7fdc7affd640 1 -- 172.21.15.133:0/3755152769 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc6c01b020 con 0x7fdc7c06e460 2024-04-25T10:03:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.724+0000 7fdc7affd640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc68066f60 0x7fdc68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.724+0000 7fdc81832640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc68066f60 0x7fdc68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.724+0000 7fdc7affd640 1 -- 172.21.15.133:0/3755152769 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdc6c085c10 con 0x7fdc7c06e460 2024-04-25T10:03:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.724+0000 7fdc81832640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc68066f60 0x7fdc68069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc70007510 tx=0x7fdc700074a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.728+0000 7fdc7affd640 1 -- 172.21.15.133:0/3755152769 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc6c022020 con 0x7fdc7c06e460 2024-04-25T10:03:01.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.880+0000 7fdc84297640 1 -- 172.21.15.133:0/3755152769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc7c15b040 con 0x7fdc7c06e460 2024-04-25T10:03:01.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.880+0000 7fdc7affd640 1 -- 172.21.15.133:0/3755152769 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7fdc6c011340 con 0x7fdc7c06e460 2024-04-25T10:03:01.887 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:01.888 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:01.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 -- 172.21.15.133:0/3755152769 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc68066f60 msgr2=0x7fdc68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc68066f60 0x7fdc68069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc70007510 tx=0x7fdc700074a0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 -- 172.21.15.133:0/3755152769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc7c06e460 msgr2=0x7fdc7c13f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc7c06e460 0x7fdc7c13f720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00d830 tx=0x7fdc6c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 -- 172.21.15.133:0/3755152769 shutdown_connections 2024-04-25T10:03:01.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc7c1584f0 0x7fdc7c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc7c158110 0x7fdc7c0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc68066f60 0x7fdc68069420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 --2- 172.21.15.133:0/3755152769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc7c06e460 0x7fdc7c13f720 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 -- 172.21.15.133:0/3755152769 >> 172.21.15.133:0/3755152769 conn(0x7fdc7c067000 msgr2=0x7fdc7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 -- 172.21.15.133:0/3755152769 shutdown_connections 2024-04-25T10:03:01.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:01.884+0000 7fdc84297640 1 -- 172.21.15.133:0/3755152769 wait complete. 2024-04-25T10:03:01.906 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:01.907 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054ec08640 1 Processor -- start 2024-04-25T10:03:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054ec08640 1 -- start start 2024-04-25T10:03:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054ec08640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05480b2b60 0x7f05480d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054ec08640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f05480b2fe0 0x7f05480d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054ec08640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054815e3a0 con 0x7f05481548b0 2024-04-25T10:03:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054ec08640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054815e680 con 0x7f05480b2fe0 2024-04-25T10:03:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054ec08640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054815e960 con 0x7f05480b2b60 2024-04-25T10:03:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054c9a4640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f05481548b0 0x7f05480d1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35862/0 (socket says 172.21.15.133:35862) 2024-04-25T10:03:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054c9a4640 1 -- 172.21.15.133:0/3140045377 learned_addr learned my addr 172.21.15.133:0/3140045377 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327281345 0 0) 0x7f054815e3a0 con 0x7f05481548b0 2024-04-25T10:03:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f0547fff640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05480b2b60 0x7f05480d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054d1a5640 1 --2- 172.21.15.133:0/3140045377 >> v2:172.21.15.162:3300/0 conn(0x7f05480b2fe0 0x7f05480d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0538003530 con 0x7f05481548b0 2024-04-25T10:03:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4242389521 0 0) 0x7f0538003530 con 0x7f05481548b0 2024-04-25T10:03:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05480b2b60 msgr2=0x7f05480d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05480b2b60 0x7f05480d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 >> v2:172.21.15.162:3300/0 conn(0x7f05480b2fe0 msgr2=0x7f05480d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 --2- 172.21.15.133:0/3140045377 >> v2:172.21.15.162:3300/0 conn(0x7f05480b2fe0 0x7f05480d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054815ec40 con 0x7f05481548b0 2024-04-25T10:03:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f053c001680 con 0x7f05481548b0 2024-04-25T10:03:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 >> v1:172.21.15.133:6789/0 conn(0x7f05481548b0 legacy=0x7f05480d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05380039c0 0x7f0538003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05480b2b60 0x7f0538004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0538004d50 0x7f05380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054815e680 con 0x7f0538004d50 2024-04-25T10:03:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054815e960 con 0x7f05380039c0 2024-04-25T10:03:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053800b9d0 con 0x7f05480b2b60 2024-04-25T10:03:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054d1a5640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0538004d50 0x7f05380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054c9a4640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05380039c0 0x7f0538003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f0547fff640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05480b2b60 0x7f0538004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054d1a5640 1 -- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05480b2b60 msgr2=0x7f0538004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054d1a5640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05480b2b60 0x7f0538004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054d1a5640 1 -- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05380039c0 msgr2=0x7f0538003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054d1a5640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05380039c0 0x7f0538003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f054d1a5640 1 -- 172.21.15.133:0/3140045377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0538003530 con 0x7f0538004d50 2024-04-25T10:03:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.020+0000 7f0547fff640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05480b2b60 0x7f0538004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:02.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054d1a5640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0538004d50 0x7f05380051d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0540009fc0 tx=0x7f054001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80a445653f27c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400b2070 con 0x7f0538004d50 2024-04-25T10:03:02.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05400a5cd0 con 0x7f0538004d50 2024-04-25T10:03:02.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f05477fe640 1 -- 172.21.15.133:0/3140045377 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540009c10 con 0x7f0538004d50 2024-04-25T10:03:02.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 -- 172.21.15.133:0/3140045377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0538004d50 msgr2=0x7f05380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 --2- 172.21.15.133:0/3140045377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0538004d50 0x7f05380051d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0540009fc0 tx=0x7f054001ccc0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054d1a5640 1 -- 172.21.15.133:0/3140045377 reap_dead start 2024-04-25T10:03:02.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 -- 172.21.15.133:0/3140045377 shutdown_connections 2024-04-25T10:03:02.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 -- 172.21.15.133:0/3140045377 >> 172.21.15.133:0/3140045377 conn(0x7f05480727c0 msgr2=0x7f0548072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 -- 172.21.15.133:0/3140045377 shutdown_connections 2024-04-25T10:03:02.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 -- 172.21.15.133:0/3140045377 wait complete. 2024-04-25T10:03:02.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 Processor -- start 2024-04-25T10:03:02.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 -- start start 2024-04-25T10:03:02.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0538004d50 0x7f0548151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05480d1660 0x7f05480d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05480b4710 0x7f05480b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054815e0f0 con 0x7f05480b4710 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054815df70 con 0x7f05480d1660 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054ec08640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05480d0a00 con 0x7f0538004d50 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054d1a5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05480b4710 0x7f05480b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054d1a5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05480b4710 0x7f05480b4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55134/0 (socket says 172.21.15.133:55134) 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054d1a5640 1 -- 172.21.15.133:0/609668318 learned_addr learned my addr 172.21.15.133:0/609668318 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054c9a4640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0538004d50 0x7f0548151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f0547fff640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05480d1660 0x7f05480d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054d1a5640 1 -- 172.21.15.133:0/609668318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0538004d50 msgr2=0x7f0548151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054d1a5640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0538004d50 0x7f0548151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054d1a5640 1 -- 172.21.15.133:0/609668318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05480d1660 msgr2=0x7f05480d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054d1a5640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05480d1660 0x7f05480d1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.024+0000 7f054d1a5640 1 -- 172.21.15.133:0/609668318 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05480b5440 con 0x7f05480b4710 2024-04-25T10:03:02.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f054d1a5640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05480b4710 0x7f05480b4b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f05400b0040 tx=0x7f0540009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f05457fa640 1 -- 172.21.15.133:0/609668318 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400bd070 con 0x7f05480b4710 2024-04-25T10:03:02.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f05457fa640 1 -- 172.21.15.133:0/609668318 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05400a5ad0 con 0x7f05480b4710 2024-04-25T10:03:02.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f054ec08640 1 -- 172.21.15.133:0/609668318 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05480b5720 con 0x7f05480b4710 2024-04-25T10:03:02.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f05457fa640 1 -- 172.21.15.133:0/609668318 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400b2040 con 0x7f05480b4710 2024-04-25T10:03:02.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f054ec08640 1 -- 172.21.15.133:0/609668318 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05480b5ce0 con 0x7f05480b4710 2024-04-25T10:03:02.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f054ec08640 1 -- 172.21.15.133:0/609668318 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0548068e80 con 0x7f05480b4710 2024-04-25T10:03:02.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f05457fa640 1 -- 172.21.15.133:0/609668318 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05400c0070 con 0x7f05480b4710 2024-04-25T10:03:02.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f05457fa640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f052c066f90 0x7f052c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f05457fa640 1 -- 172.21.15.133:0/609668318 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f054011f330 con 0x7f05480b4710 2024-04-25T10:03:02.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f054c9a4640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f052c066f90 0x7f052c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.028+0000 7f054c9a4640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f052c066f90 0x7f052c069450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f053c000f70 tx=0x7f053c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.032+0000 7f05457fa640 1 -- 172.21.15.133:0/609668318 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05400169d0 con 0x7f05480b4710 2024-04-25T10:03:02.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.176+0000 7f054ec08640 1 -- 172.21.15.133:0/609668318 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05480d3750 con 0x7f05480b4710 2024-04-25T10:03:02.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f05457fa640 1 -- 172.21.15.133:0/609668318 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05400ef8f0 con 0x7f05480b4710 2024-04-25T10:03:02.185 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:03:02.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 -- 172.21.15.133:0/609668318 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f052c066f90 msgr2=0x7f052c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f052c066f90 0x7f052c069450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f053c000f70 tx=0x7f053c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:02.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 -- 172.21.15.133:0/609668318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05480b4710 msgr2=0x7f05480b4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05480b4710 0x7f05480b4b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f05400b0040 tx=0x7f0540009750 comp rx=0 tx=0).stop 2024-04-25T10:03:02.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 -- 172.21.15.133:0/609668318 shutdown_connections 2024-04-25T10:03:02.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f052c066f90 0x7f052c069450 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05480b4710 0x7f05480b4b90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05480d1660 0x7f05480d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 --2- 172.21.15.133:0/609668318 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0538004d50 0x7f0548151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 -- 172.21.15.133:0/609668318 >> 172.21.15.133:0/609668318 conn(0x7f05480727c0 msgr2=0x7f05480b8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 -- 172.21.15.133:0/609668318 shutdown_connections 2024-04-25T10:03:02.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.180+0000 7f054ec08640 1 -- 172.21.15.133:0/609668318 wait complete. 2024-04-25T10:03:02.203 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:02.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff0854d1640 1 Processor -- start 2024-04-25T10:03:02.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff0854d1640 1 -- start start 2024-04-25T10:03:02.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff0854d1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff080145170 0x7ff08006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff0854d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08006e4d0 0x7ff0800cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff0854d1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0800ce2e0 con 0x7ff0800cdd60 2024-04-25T10:03:02.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff0854d1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08006db00 con 0x7ff080145170 2024-04-25T10:03:02.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff0854d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff080145550 con 0x7ff08006e4d0 2024-04-25T10:03:02.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff0800cdd60 0x7ff08015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35864/0 (socket says 172.21.15.133:35864) 2024-04-25T10:03:02.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07f7fe640 1 -- 172.21.15.133:0/154987981 learned_addr learned my addr 172.21.15.133:0/154987981 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:02.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07effd640 1 --2- 172.21.15.133:0/154987981 >> v2:172.21.15.162:3300/0 conn(0x7ff080145170 0x7ff08006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff077fff640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08006e4d0 0x7ff0800cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979098163 0 0) 0x7ff0800ce2e0 con 0x7ff0800cdd60 2024-04-25T10:03:02.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff06c003530 con 0x7ff0800cdd60 2024-04-25T10:03:02.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 947056414 0 0) 0x7ff06c003530 con 0x7ff0800cdd60 2024-04-25T10:03:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08006e4d0 msgr2=0x7ff0800cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08006e4d0 0x7ff0800cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 >> v2:172.21.15.162:3300/0 conn(0x7ff080145170 msgr2=0x7ff08006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 --2- 172.21.15.133:0/154987981 >> v2:172.21.15.162:3300/0 conn(0x7ff080145170 0x7ff08006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08006d4b0 con 0x7ff0800cdd60 2024-04-25T10:03:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff078001a00 con 0x7ff0800cdd60 2024-04-25T10:03:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 >> v1:172.21.15.133:6789/0 conn(0x7ff0800cdd60 legacy=0x7ff08015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff06c0039c0 0x7ff06c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08006e4d0 0x7ff06c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff06c004d50 0x7ff06c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08006db00 con 0x7ff06c0039c0 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff080145550 con 0x7ff06c004d50 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c00b9d0 con 0x7ff08006e4d0 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07effd640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff06c0039c0 0x7ff06c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07effd640 1 -- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08006e4d0 msgr2=0x7ff06c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff077fff640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08006e4d0 0x7ff06c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07effd640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08006e4d0 0x7ff06c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07f7fe640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff06c004d50 0x7ff06c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07effd640 1 -- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff06c004d50 msgr2=0x7ff06c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07effd640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff06c004d50 0x7ff06c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff07effd640 1 -- 172.21.15.133:0/154987981 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c003530 con 0x7ff06c0039c0 2024-04-25T10:03:02.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.316+0000 7ff077fff640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08006e4d0 0x7ff06c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff07effd640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff06c0039c0 0x7ff06c003de0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff070009ae0 tx=0x7ff07001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30275001094e57fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0700096b0 con 0x7ff06c0039c0 2024-04-25T10:03:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0700a6490 con 0x7ff06c0039c0 2024-04-25T10:03:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff07e7fc640 1 -- 172.21.15.133:0/154987981 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0700a6880 con 0x7ff06c0039c0 2024-04-25T10:03:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 -- 172.21.15.133:0/154987981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff06c0039c0 msgr2=0x7ff06c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 --2- 172.21.15.133:0/154987981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff06c0039c0 0x7ff06c003de0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff070009ae0 tx=0x7ff07001cab0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff07f7fe640 1 -- 172.21.15.133:0/154987981 reap_dead start 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 -- 172.21.15.133:0/154987981 shutdown_connections 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 -- 172.21.15.133:0/154987981 >> 172.21.15.133:0/154987981 conn(0x7ff080066f70 msgr2=0x7ff08006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 -- 172.21.15.133:0/154987981 shutdown_connections 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 -- 172.21.15.133:0/154987981 wait complete. 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 Processor -- start 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 -- start start 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff06c0039c0 0x7ff0800b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0800b2f20 0x7ff080136a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08015f2e0 0x7ff08015f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff080166480 con 0x7ff0800b2f20 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08015d7f0 con 0x7ff06c0039c0 2024-04-25T10:03:02.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0854d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff080166600 con 0x7ff08015f2e0 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff077fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0800b2f20 0x7ff080136a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff077fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0800b2f20 0x7ff080136a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55150/0 (socket says 172.21.15.133:55150) 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff077fff640 1 -- 172.21.15.133:0/3884050691 learned_addr learned my addr 172.21.15.133:0/3884050691 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff07effd640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff06c0039c0 0x7ff0800b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff07f7fe640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08015f2e0 0x7ff08015f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff077fff640 1 -- 172.21.15.133:0/3884050691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08015f2e0 msgr2=0x7ff08015f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff077fff640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08015f2e0 0x7ff08015f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff077fff640 1 -- 172.21.15.133:0/3884050691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff06c0039c0 msgr2=0x7ff0800b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff077fff640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff06c0039c0 0x7ff0800b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff077fff640 1 -- 172.21.15.133:0/3884050691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080144060 con 0x7ff0800b2f20 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff077fff640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0800b2f20 0x7ff080136a00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff068009fd0 tx=0x7ff068010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0777fe640 1 -- 172.21.15.133:0/3884050691 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06800f040 con 0x7ff0800b2f20 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.320+0000 7ff0777fe640 1 -- 172.21.15.133:0/3884050691 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068004220 con 0x7ff0800b2f20 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.324+0000 7ff0854d1640 1 -- 172.21.15.133:0/3884050691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff080144340 con 0x7ff0800b2f20 2024-04-25T10:03:02.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.324+0000 7ff0777fe640 1 -- 172.21.15.133:0/3884050691 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06800e900 con 0x7ff0800b2f20 2024-04-25T10:03:02.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.324+0000 7ff0854d1640 1 -- 172.21.15.133:0/3884050691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff080138380 con 0x7ff0800b2f20 2024-04-25T10:03:02.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.324+0000 7ff0854d1640 1 -- 172.21.15.133:0/3884050691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff080068e80 con 0x7ff0800b2f20 2024-04-25T10:03:02.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.324+0000 7ff0777fe640 1 -- 172.21.15.133:0/3884050691 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff06800eaa0 con 0x7ff0800b2f20 2024-04-25T10:03:02.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.324+0000 7ff0777fe640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff060066f90 0x7ff060069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.324+0000 7ff0777fe640 1 -- 172.21.15.133:0/3884050691 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff068085720 con 0x7ff0800b2f20 2024-04-25T10:03:02.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.324+0000 7ff07effd640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff060066f90 0x7ff060069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.324+0000 7ff07effd640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff060066f90 0x7ff060069450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff0700ad000 tx=0x7ff0700046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.328+0000 7ff0777fe640 1 -- 172.21.15.133:0/3884050691 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff068018030 con 0x7ff0800b2f20 2024-04-25T10:03:02.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.472+0000 7ff0854d1640 1 -- 172.21.15.133:0/3884050691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff08006e000 con 0x7ff0800b2f20 2024-04-25T10:03:02.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.472+0000 7ff0777fe640 1 -- 172.21.15.133:0/3884050691 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0680563c0 con 0x7ff0800b2f20 2024-04-25T10:03:02.480 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:03:02.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 -- 172.21.15.133:0/3884050691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff060066f90 msgr2=0x7ff060069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff060066f90 0x7ff060069450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff0700ad000 tx=0x7ff0700046d0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 -- 172.21.15.133:0/3884050691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0800b2f20 msgr2=0x7ff080136a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0800b2f20 0x7ff080136a00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff068009fd0 tx=0x7ff068010d60 comp rx=0 tx=0).stop 2024-04-25T10:03:02.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 -- 172.21.15.133:0/3884050691 shutdown_connections 2024-04-25T10:03:02.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff08015f2e0 0x7ff08015f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff060066f90 0x7ff060069450 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0800b2f20 0x7ff080136a00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 --2- 172.21.15.133:0/3884050691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff06c0039c0 0x7ff0800b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 -- 172.21.15.133:0/3884050691 >> 172.21.15.133:0/3884050691 conn(0x7ff080066f70 msgr2=0x7ff0800b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 -- 172.21.15.133:0/3884050691 shutdown_connections 2024-04-25T10:03:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.476+0000 7ff0854d1640 1 -- 172.21.15.133:0/3884050691 wait complete. 2024-04-25T10:03:02.497 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.600+0000 7f49420db640 1 Processor -- start 2024-04-25T10:03:02.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.600+0000 7f49420db640 1 -- start start 2024-04-25T10:03:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f49420db640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f493c15dd00 0x7f493c15e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f49420db640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f493c06bda0 0x7f493c0999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f49420db640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c15e750 con 0x7f493c099f10 2024-04-25T10:03:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f49420db640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c09a360 con 0x7f493c15dd00 2024-04-25T10:03:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f49420db640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c15ea30 con 0x7f493c06bda0 2024-04-25T10:03:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f493b7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f493c15dd00 0x7f493c15e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f493affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f493c06bda0 0x7f493c0999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f493bfff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f493c099f10 0x7f493c161690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:59006/0 (socket says 172.21.15.133:59006) 2024-04-25T10:03:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f493bfff640 1 -- 172.21.15.133:0/62692633 learned_addr learned my addr 172.21.15.133:0/62692633 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f493affd640 1 -- 172.21.15.133:0/62692633 >> v2:172.21.15.162:3300/0 conn(0x7f493c15dd00 msgr2=0x7f493c15e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f493affd640 1 --2- 172.21.15.133:0/62692633 >> v2:172.21.15.162:3300/0 conn(0x7f493c15dd00 0x7f493c15e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f493affd640 1 -- 172.21.15.133:0/62692633 >> v1:172.21.15.162:6789/0 conn(0x7f493c099f10 legacy=0x7f493c161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f493affd640 1 -- 172.21.15.133:0/62692633 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c15ed10 con 0x7f493c06bda0 2024-04-25T10:03:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f493b7fe640 1 --2- 172.21.15.133:0/62692633 >> v2:172.21.15.162:3300/0 conn(0x7f493c15dd00 0x7f493c15e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:02.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f493affd640 1 --2- 172.21.15.133:0/62692633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f493c06bda0 0x7f493c0999d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49300097c0 tx=0x7f493001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=457cef3f048bb316 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.604+0000 7f493a7fc640 1 -- 172.21.15.133:0/62692633 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49300b0070 con 0x7f493c06bda0 2024-04-25T10:03:02.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493a7fc640 1 -- 172.21.15.133:0/62692633 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49300ab600 con 0x7f493c06bda0 2024-04-25T10:03:02.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493a7fc640 1 -- 172.21.15.133:0/62692633 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49300ab9d0 con 0x7f493c06bda0 2024-04-25T10:03:02.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 -- 172.21.15.133:0/62692633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f493c06bda0 msgr2=0x7f493c0999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 --2- 172.21.15.133:0/62692633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f493c06bda0 0x7f493c0999d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49300097c0 tx=0x7f493001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 -- 172.21.15.133:0/62692633 shutdown_connections 2024-04-25T10:03:02.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 --2- 172.21.15.133:0/62692633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f493c06bda0 0x7f493c0999d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 --2- 172.21.15.133:0/62692633 >> v2:172.21.15.162:3300/0 conn(0x7f493c15dd00 0x7f493c15e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 -- 172.21.15.133:0/62692633 >> 172.21.15.133:0/62692633 conn(0x7f493c0b31a0 msgr2=0x7f493c0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 -- 172.21.15.133:0/62692633 shutdown_connections 2024-04-25T10:03:02.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 -- 172.21.15.133:0/62692633 wait complete. 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 Processor -- start 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 -- start start 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c06bda0 0x7f493c153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f493c099f10 0x7f493c151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f493c15dd00 0x7f493c151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0d1140 con 0x7f493c06bda0 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0d0fc0 con 0x7f493c15dd00 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c16a9d0 con 0x7f493c099f10 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c06bda0 0x7f493c153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c06bda0 0x7f493c153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55158/0 (socket says 172.21.15.133:55158) 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493b7fe640 1 -- 172.21.15.133:0/203151654 learned_addr learned my addr 172.21.15.133:0/203151654 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:02.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493bfff640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f493c15dd00 0x7f493c151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493affd640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f493c099f10 0x7f493c151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493b7fe640 1 -- 172.21.15.133:0/203151654 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f493c099f10 msgr2=0x7f493c151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493b7fe640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f493c099f10 0x7f493c151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493b7fe640 1 -- 172.21.15.133:0/203151654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f493c15dd00 msgr2=0x7f493c151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493b7fe640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f493c15dd00 0x7f493c151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493b7fe640 1 -- 172.21.15.133:0/203151654 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c159950 con 0x7f493c06bda0 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f493b7fe640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c06bda0 0x7f493c153770 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f492c00ec10 tx=0x7f492c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f4923fff640 1 -- 172.21.15.133:0/203151654 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c017070 con 0x7f493c06bda0 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f4923fff640 1 -- 172.21.15.133:0/203151654 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c00cbd0 con 0x7f493c06bda0 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 -- 172.21.15.133:0/203151654 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c1551b0 con 0x7f493c06bda0 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f4923fff640 1 -- 172.21.15.133:0/203151654 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c013470 con 0x7f493c06bda0 2024-04-25T10:03:02.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.608+0000 7f49420db640 1 -- 172.21.15.133:0/203151654 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c159390 con 0x7f493c06bda0 2024-04-25T10:03:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.612+0000 7f49420db640 1 -- 172.21.15.133:0/203151654 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c068e80 con 0x7f493c06bda0 2024-04-25T10:03:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.612+0000 7f4923fff640 1 -- 172.21.15.133:0/203151654 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f492c01a070 con 0x7f493c06bda0 2024-04-25T10:03:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.612+0000 7f4923fff640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4928066f60 0x7f4928069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.612+0000 7f493affd640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4928066f60 0x7f4928069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.612+0000 7f4923fff640 1 -- 172.21.15.133:0/203151654 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f492c085370 con 0x7f493c06bda0 2024-04-25T10:03:02.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.612+0000 7f493affd640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4928066f60 0x7f4928069420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f49300ad000 tx=0x7f4930008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.616+0000 7f4923fff640 1 -- 172.21.15.133:0/203151654 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f492c055f80 con 0x7f493c06bda0 2024-04-25T10:03:02.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.768+0000 7f49420db640 1 -- 172.21.15.133:0/203151654 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f493c0b7630 con 0x7f493c06bda0 2024-04-25T10:03:02.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.768+0000 7f4923fff640 1 -- 172.21.15.133:0/203151654 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f492c055880 con 0x7f493c06bda0 2024-04-25T10:03:02.775 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:03:02.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 -- 172.21.15.133:0/203151654 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4928066f60 msgr2=0x7f4928069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4928066f60 0x7f4928069420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f49300ad000 tx=0x7f4930008040 comp rx=0 tx=0).stop 2024-04-25T10:03:02.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 -- 172.21.15.133:0/203151654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c06bda0 msgr2=0x7f493c153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c06bda0 0x7f493c153770 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f492c00ec10 tx=0x7f492c00c520 comp rx=0 tx=0).stop 2024-04-25T10:03:02.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 -- 172.21.15.133:0/203151654 shutdown_connections 2024-04-25T10:03:02.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f493c15dd00 0x7f493c151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f493c099f10 0x7f493c151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4928066f60 0x7f4928069420 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 --2- 172.21.15.133:0/203151654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f493c06bda0 0x7f493c153770 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 -- 172.21.15.133:0/203151654 >> 172.21.15.133:0/203151654 conn(0x7f493c0b31a0 msgr2=0x7f493c0ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 -- 172.21.15.133:0/203151654 shutdown_connections 2024-04-25T10:03:02.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.772+0000 7f49420db640 1 -- 172.21.15.133:0/203151654 wait complete. 2024-04-25T10:03:02.792 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:02.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.896+0000 7fd2d2ca5640 1 Processor -- start 2024-04-25T10:03:02.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.896+0000 7fd2d2ca5640 1 -- start start 2024-04-25T10:03:02.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2d2ca5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2cc0b2150 0x7fd2cc06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2d2ca5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2cc15b9b0 0x7fd2cc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2d2ca5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc15ac50 con 0x7fd2cc143ea0 2024-04-25T10:03:02.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2d2ca5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2cc0ccf20 con 0x7fd2cc0b2150 2024-04-25T10:03:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2d2ca5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0cd180 con 0x7fd2cc15b9b0 2024-04-25T10:03:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2d0a41640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd2cc143ea0 0x7fd2cc0cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35868/0 (socket says 172.21.15.133:35868) 2024-04-25T10:03:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2d0a41640 1 -- 172.21.15.133:0/2601079875 learned_addr learned my addr 172.21.15.133:0/2601079875 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2cbfff640 1 --2- 172.21.15.133:0/2601079875 >> v2:172.21.15.133:3300/0 conn(0x7fd2cc0b2150 0x7fd2cc06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2d1242640 1 --2- 172.21.15.133:0/2601079875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2cc15b9b0 0x7fd2cc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2cb7fe640 1 -- 172.21.15.133:0/2601079875 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876326216 0 0) 0x7fd2bc001120 con 0x7fd2cc143ea0 2024-04-25T10:03:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2cb7fe640 1 -- 172.21.15.133:0/2601079875 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2b4005740 con 0x7fd2cc143ea0 2024-04-25T10:03:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2cbfff640 1 -- 172.21.15.133:0/2601079875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2cc15b9b0 msgr2=0x7fd2cc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2cbfff640 1 --2- 172.21.15.133:0/2601079875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2cc15b9b0 0x7fd2cc06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2cbfff640 1 -- 172.21.15.133:0/2601079875 >> v1:172.21.15.133:6789/0 conn(0x7fd2cc143ea0 legacy=0x7fd2cc0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.900+0000 7fd2cbfff640 1 -- 172.21.15.133:0/2601079875 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc15aed0 con 0x7fd2cc0b2150 2024-04-25T10:03:02.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2cbfff640 1 --2- 172.21.15.133:0/2601079875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2cc0b2150 0x7fd2cc06e3b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00097c0 tx=0x7fd2c001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a13c78048e901a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2cb7fe640 1 -- 172.21.15.133:0/2601079875 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00b2070 con 0x7fd2cc0b2150 2024-04-25T10:03:02.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2cb7fe640 1 -- 172.21.15.133:0/2601079875 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c00ad640 con 0x7fd2cc0b2150 2024-04-25T10:03:02.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2cb7fe640 1 -- 172.21.15.133:0/2601079875 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00ada10 con 0x7fd2cc0b2150 2024-04-25T10:03:02.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/2601079875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2cc0b2150 msgr2=0x7fd2cc06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 --2- 172.21.15.133:0/2601079875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2cc0b2150 0x7fd2cc06e3b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00097c0 tx=0x7fd2c001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/2601079875 shutdown_connections 2024-04-25T10:03:02.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 --2- 172.21.15.133:0/2601079875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2cc15b9b0 0x7fd2cc06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 --2- 172.21.15.133:0/2601079875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2cc0b2150 0x7fd2cc06e3b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/2601079875 >> 172.21.15.133:0/2601079875 conn(0x7fd2cc066f70 msgr2=0x7fd2cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/2601079875 shutdown_connections 2024-04-25T10:03:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/2601079875 wait complete. 2024-04-25T10:03:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 Processor -- start 2024-04-25T10:03:02.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 -- start start 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc0b2150 0x7fd2cc142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2cc143ea0 0x7fd2cc15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2cc15b9b0 0x7fd2cc160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0b1270 con 0x7fd2cc143ea0 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0b10f0 con 0x7fd2cc0b2150 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2d2ca5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0b13f0 con 0x7fd2cc15b9b0 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2cc143ea0 0x7fd2cc15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2cc143ea0 0x7fd2cc15c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55180/0 (socket says 172.21.15.133:55180) 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.904+0000 7fd2cbfff640 1 -- 172.21.15.133:0/1439483891 learned_addr learned my addr 172.21.15.133:0/1439483891 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2cbfff640 1 -- 172.21.15.133:0/1439483891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2cc15b9b0 msgr2=0x7fd2cc160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2cbfff640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2cc15b9b0 0x7fd2cc160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2d0a41640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc0b2150 0x7fd2cc142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2cbfff640 1 -- 172.21.15.133:0/1439483891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc0b2150 msgr2=0x7fd2cc142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2cbfff640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc0b2150 0x7fd2cc142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2cbfff640 1 -- 172.21.15.133:0/1439483891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc1432e0 con 0x7fd2cc143ea0 2024-04-25T10:03:02.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2cbfff640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2cc143ea0 0x7fd2cc15c660 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd2c001a0d0 tx=0x7fd2c0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2c97fa640 1 -- 172.21.15.133:0/1439483891 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00ba020 con 0x7fd2cc143ea0 2024-04-25T10:03:02.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2c97fa640 1 -- 172.21.15.133:0/1439483891 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c00b6410 con 0x7fd2cc143ea0 2024-04-25T10:03:02.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2c97fa640 1 -- 172.21.15.133:0/1439483891 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00b2040 con 0x7fd2cc143ea0 2024-04-25T10:03:02.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/1439483891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc13f750 con 0x7fd2cc143ea0 2024-04-25T10:03:02.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/1439483891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc1425a0 con 0x7fd2cc143ea0 2024-04-25T10:03:02.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/1439483891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc1423b0 con 0x7fd2cc143ea0 2024-04-25T10:03:02.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2c97fa640 1 -- 172.21.15.133:0/1439483891 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2c00bd020 con 0x7fd2cc143ea0 2024-04-25T10:03:02.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2c97fa640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b8066f60 0x7fd2b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2d0a41640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b8066f60 0x7fd2b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2c97fa640 1 -- 172.21.15.133:0/1439483891 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd2c011f190 con 0x7fd2cc143ea0 2024-04-25T10:03:02.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.908+0000 7fd2d0a41640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b8066f60 0x7fd2b8069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0012c0 tx=0x7fd2bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:02.912+0000 7fd2c97fa640 1 -- 172.21.15.133:0/1439483891 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2c00c4030 con 0x7fd2cc143ea0 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.060+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/1439483891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd2cc000910 con 0x7fd2cc143ea0 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2c97fa640 1 -- 172.21.15.133:0/1439483891 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd2c0016090 con 0x7fd2cc143ea0 2024-04-25T10:03:03.069 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/1439483891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b8066f60 msgr2=0x7fd2b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b8066f60 0x7fd2b8069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0012c0 tx=0x7fd2bc009040 comp rx=0 tx=0).stop 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/1439483891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2cc143ea0 msgr2=0x7fd2cc15c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2cc143ea0 0x7fd2cc15c660 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd2c001a0d0 tx=0x7fd2c0019240 comp rx=0 tx=0).stop 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/1439483891 shutdown_connections 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2cc15b9b0 0x7fd2cc160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b8066f60 0x7fd2b8069420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2cc143ea0 0x7fd2cc15c660 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 --2- 172.21.15.133:0/1439483891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2cc0b2150 0x7fd2cc142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/1439483891 >> 172.21.15.133:0/1439483891 conn(0x7fd2cc066f70 msgr2=0x7fd2cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/1439483891 shutdown_connections 2024-04-25T10:03:03.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.064+0000 7fd2d2ca5640 1 -- 172.21.15.133:0/1439483891 wait complete. 2024-04-25T10:03:03.087 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1259f8c640 1 Processor -- start 2024-04-25T10:03:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1259f8c640 1 -- start start 2024-04-25T10:03:03.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1259f8c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1254162590 0x7f125415efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1259f8c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f12540ce850 0x7f12540d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1259f8c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12540cf0c0 con 0x7f12540d02b0 2024-04-25T10:03:03.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1259f8c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12540cf3a0 con 0x7f12540ce850 2024-04-25T10:03:03.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1259f8c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1254161ab0 con 0x7f1254162590 2024-04-25T10:03:03.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f12537fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f12540d02b0 0x7f12540d0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35874/0 (socket says 172.21.15.133:35874) 2024-04-25T10:03:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f12537fe640 1 -- 172.21.15.133:0/3984190437 learned_addr learned my addr 172.21.15.133:0/3984190437 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1253fff640 1 --2- 172.21.15.133:0/3984190437 >> v2:172.21.15.133:3300/0 conn(0x7f12540ce850 0x7f12540d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f12527fc640 1 -- 172.21.15.133:0/3984190437 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931108722 0 0) 0x7f12540cf0c0 con 0x7f12540d02b0 2024-04-25T10:03:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f12527fc640 1 -- 172.21.15.133:0/3984190437 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f123c003530 con 0x7f12540d02b0 2024-04-25T10:03:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1252ffd640 1 --2- 172.21.15.133:0/3984190437 >> v2:172.21.15.162:3300/0 conn(0x7f1254162590 0x7f125415efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1253fff640 1 -- 172.21.15.133:0/3984190437 >> v2:172.21.15.162:3300/0 conn(0x7f1254162590 msgr2=0x7f125415efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1253fff640 1 --2- 172.21.15.133:0/3984190437 >> v2:172.21.15.162:3300/0 conn(0x7f1254162590 0x7f125415efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1253fff640 1 -- 172.21.15.133:0/3984190437 >> v1:172.21.15.133:6789/0 conn(0x7f12540d02b0 legacy=0x7f12540d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.188+0000 7f1253fff640 1 -- 172.21.15.133:0/3984190437 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1254161d90 con 0x7f12540ce850 2024-04-25T10:03:03.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1253fff640 1 --2- 172.21.15.133:0/3984190437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12540ce850 0x7f12540d2090 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f12540ce1e0 tx=0x7f124401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2a26b415fea0c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f12527fc640 1 -- 172.21.15.133:0/3984190437 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440b0070 con 0x7f12540ce850 2024-04-25T10:03:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f12527fc640 1 -- 172.21.15.133:0/3984190437 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12440ab600 con 0x7f12540ce850 2024-04-25T10:03:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f12527fc640 1 -- 172.21.15.133:0/3984190437 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440ab9d0 con 0x7f12540ce850 2024-04-25T10:03:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 -- 172.21.15.133:0/3984190437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12540ce850 msgr2=0x7f12540d2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 --2- 172.21.15.133:0/3984190437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12540ce850 0x7f12540d2090 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f12540ce1e0 tx=0x7f124401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 -- 172.21.15.133:0/3984190437 shutdown_connections 2024-04-25T10:03:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 --2- 172.21.15.133:0/3984190437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12540ce850 0x7f12540d2090 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 --2- 172.21.15.133:0/3984190437 >> v2:172.21.15.162:3300/0 conn(0x7f1254162590 0x7f125415efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 -- 172.21.15.133:0/3984190437 >> 172.21.15.133:0/3984190437 conn(0x7f125406bf30 msgr2=0x7f125406c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 -- 172.21.15.133:0/3984190437 shutdown_connections 2024-04-25T10:03:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 -- 172.21.15.133:0/3984190437 wait complete. 2024-04-25T10:03:03.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 Processor -- start 2024-04-25T10:03:03.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 -- start start 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12540ce850 0x7f125415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12540d02b0 0x7f1254151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1254162590 0x7f1254157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125416ae50 con 0x7f1254162590 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125415bed0 con 0x7f12540d02b0 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125416afd0 con 0x7f12540ce850 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1253fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1254162590 0x7f1254157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1253fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1254162590 0x7f1254157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55212/0 (socket says 172.21.15.133:55212) 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1253fff640 1 -- 172.21.15.133:0/3224823367 learned_addr learned my addr 172.21.15.133:0/3224823367 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1253fff640 1 -- 172.21.15.133:0/3224823367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12540ce850 msgr2=0x7f125415ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1253fff640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12540ce850 0x7f125415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1253fff640 1 -- 172.21.15.133:0/3224823367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12540d02b0 msgr2=0x7f1254151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1253fff640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12540d02b0 0x7f1254151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1253fff640 1 -- 172.21.15.133:0/3224823367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12541531d0 con 0x7f1254162590 2024-04-25T10:03:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1253fff640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1254162590 0x7f1254157930 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f12440ae040 tx=0x7f1244019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f123bfff640 1 -- 172.21.15.133:0/3224823367 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440b8020 con 0x7f1254162590 2024-04-25T10:03:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f123bfff640 1 -- 172.21.15.133:0/3224823367 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12440b4410 con 0x7f1254162590 2024-04-25T10:03:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 -- 172.21.15.133:0/3224823367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12541534b0 con 0x7f1254162590 2024-04-25T10:03:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f123bfff640 1 -- 172.21.15.133:0/3224823367 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440b0040 con 0x7f1254162590 2024-04-25T10:03:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.192+0000 7f1259f8c640 1 -- 172.21.15.133:0/3224823367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12540cabc0 con 0x7f1254162590 2024-04-25T10:03:03.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.196+0000 7f1259f8c640 1 -- 172.21.15.133:0/3224823367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1254067c70 con 0x7f1254162590 2024-04-25T10:03:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.196+0000 7f123bfff640 1 -- 172.21.15.133:0/3224823367 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12440bb020 con 0x7f1254162590 2024-04-25T10:03:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.196+0000 7f123bfff640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1240066f60 0x7f1240069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.196+0000 7f123bfff640 1 -- 172.21.15.133:0/3224823367 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f124411ef40 con 0x7f1254162590 2024-04-25T10:03:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.196+0000 7f12537fe640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1240066f60 0x7f1240069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.196+0000 7f12537fe640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1240066f60 0x7f1240069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f124c0012f0 tx=0x7f124c0015c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.200+0000 7f123bfff640 1 -- 172.21.15.133:0/3224823367 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12440c2030 con 0x7f1254162590 2024-04-25T10:03:03.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.356+0000 7f1259f8c640 1 -- 172.21.15.133:0/3224823367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f12540d1a10 con 0x7f1254162590 2024-04-25T10:03:03.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.356+0000 7f123bfff640 1 -- 172.21.15.133:0/3224823367 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1244016070 con 0x7f1254162590 2024-04-25T10:03:03.364 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:03:03.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 -- 172.21.15.133:0/3224823367 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1240066f60 msgr2=0x7f1240069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1240066f60 0x7f1240069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f124c0012f0 tx=0x7f124c0015c0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 -- 172.21.15.133:0/3224823367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1254162590 msgr2=0x7f1254157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1254162590 0x7f1254157930 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f12440ae040 tx=0x7f1244019190 comp rx=0 tx=0).stop 2024-04-25T10:03:03.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 -- 172.21.15.133:0/3224823367 shutdown_connections 2024-04-25T10:03:03.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1240066f60 0x7f1240069420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1254162590 0x7f1254157930 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12540d02b0 0x7f1254151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 --2- 172.21.15.133:0/3224823367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12540ce850 0x7f125415ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 -- 172.21.15.133:0/3224823367 >> 172.21.15.133:0/3224823367 conn(0x7f125406bf30 msgr2=0x7f125406c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 -- 172.21.15.133:0/3224823367 shutdown_connections 2024-04-25T10:03:03.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.360+0000 7f1259f8c640 1 -- 172.21.15.133:0/3224823367 wait complete. 2024-04-25T10:03:03.382 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi162.front.sepia.ceph.com... 2024-04-25T10:03:03.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:03.382 DEBUG:teuthology.orchestra.run.smithi162:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T10:03:03.397 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:03.397 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:03:03.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:03.398 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:03:03.446 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:03:03.447 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:03.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd427991640 1 Processor -- start 2024-04-25T10:03:03.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd427991640 1 -- start start 2024-04-25T10:03:03.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd427991640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd42015d9b0 0x7fd4200cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd427991640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4200ca9b0 con 0x7fd42015dd90 2024-04-25T10:03:03.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd427991640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4200cac90 con 0x7fd4200cee80 2024-04-25T10:03:03.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd427991640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4200cbe30 con 0x7fd42015d9b0 2024-04-25T10:03:03.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd42572d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd42015d9b0 0x7fd4200cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd42572d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd42015d9b0 0x7fd4200cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55226/0 (socket says 172.21.15.133:55226) 2024-04-25T10:03:03.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd42572d640 1 -- 172.21.15.133:0/1503128731 learned_addr learned my addr 172.21.15.133:0/1503128731 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:03.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd424f2c640 1 --1- 172.21.15.133:0/1503128731 >> v1:172.21.15.133:6789/0 conn(0x7fd42015dd90 0x7fd4200ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35882/0 (socket says 172.21.15.133:35882) 2024-04-25T10:03:03.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd42572d640 1 -- 172.21.15.133:0/1503128731 >> v1:172.21.15.162:6789/0 conn(0x7fd4200cee80 legacy=0x7fd42006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd42572d640 1 -- 172.21.15.133:0/1503128731 >> v1:172.21.15.133:6789/0 conn(0x7fd42015dd90 legacy=0x7fd4200ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.560+0000 7fd42572d640 1 -- 172.21.15.133:0/1503128731 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4200caf70 con 0x7fd42015d9b0 2024-04-25T10:03:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.564+0000 7fd42572d640 1 --2- 172.21.15.133:0/1503128731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd42015d9b0 0x7fd4200cb5e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009870 tx=0x7fd41c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eadfe2247597b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.564+0000 7fd417fff640 1 -- 172.21.15.133:0/1503128731 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0ab650 con 0x7fd42015d9b0 2024-04-25T10:03:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.564+0000 7fd417fff640 1 -- 172.21.15.133:0/1503128731 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41c0b0070 con 0x7fd42015d9b0 2024-04-25T10:03:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.564+0000 7fd417fff640 1 -- 172.21.15.133:0/1503128731 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0aba10 con 0x7fd42015d9b0 2024-04-25T10:03:03.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.564+0000 7fd427991640 1 -- 172.21.15.133:0/1503128731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd42015d9b0 msgr2=0x7fd4200cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.564+0000 7fd427991640 1 --2- 172.21.15.133:0/1503128731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd42015d9b0 0x7fd4200cb5e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009870 tx=0x7fd41c01c430 comp rx=0 tx=0).stop 2024-04-25T10:03:03.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.564+0000 7fd427991640 1 -- 172.21.15.133:0/1503128731 shutdown_connections 2024-04-25T10:03:03.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.564+0000 7fd427991640 1 --2- 172.21.15.133:0/1503128731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd42015d9b0 0x7fd4200cb5e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.564+0000 7fd427991640 1 -- 172.21.15.133:0/1503128731 >> 172.21.15.133:0/1503128731 conn(0x7fd420067000 msgr2=0x7fd42006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.564+0000 7fd427991640 1 -- 172.21.15.133:0/1503128731 shutdown_connections 2024-04-25T10:03:03.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.564+0000 7fd427991640 1 -- 172.21.15.133:0/1503128731 wait complete. 2024-04-25T10:03:03.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd427991640 1 Processor -- start 2024-04-25T10:03:03.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd427991640 1 -- start start 2024-04-25T10:03:03.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd427991640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4200cee80 0x7fd420146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd427991640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd42015d9b0 0x7fd42015fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd427991640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd42015dd90 0x7fd42014e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd427991640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42006e9a0 con 0x7fd4200cee80 2024-04-25T10:03:03.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd427991640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42006e820 con 0x7fd42015d9b0 2024-04-25T10:03:03.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd427991640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42016b870 con 0x7fd42015dd90 2024-04-25T10:03:03.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd42572d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4200cee80 0x7fd420146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd42572d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4200cee80 0x7fd420146920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55242/0 (socket says 172.21.15.133:55242) 2024-04-25T10:03:03.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd42572d640 1 -- 172.21.15.133:0/3337693178 learned_addr learned my addr 172.21.15.133:0/3337693178 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd424f2c640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd42015d9b0 0x7fd42015fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd425f2e640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd42015dd90 0x7fd42014e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd42572d640 1 -- 172.21.15.133:0/3337693178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd42015dd90 msgr2=0x7fd42014e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd42572d640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd42015dd90 0x7fd42014e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd42572d640 1 -- 172.21.15.133:0/3337693178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd42015d9b0 msgr2=0x7fd42015fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd42572d640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd42015d9b0 0x7fd42015fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd42572d640 1 -- 172.21.15.133:0/3337693178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd420146f90 con 0x7fd4200cee80 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd424f2c640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd42015d9b0 0x7fd42015fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd42572d640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4200cee80 0x7fd420146920 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0190a0 tx=0x7fd41c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd415ffb640 1 -- 172.21.15.133:0/3337693178 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0b4800 con 0x7fd4200cee80 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd415ffb640 1 -- 172.21.15.133:0/3337693178 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41c0b49c0 con 0x7fd4200cee80 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd427991640 1 -- 172.21.15.133:0/3337693178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd420147270 con 0x7fd4200cee80 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd415ffb640 1 -- 172.21.15.133:0/3337693178 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0b4db0 con 0x7fd4200cee80 2024-04-25T10:03:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd427991640 1 -- 172.21.15.133:0/3337693178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd420147830 con 0x7fd4200cee80 2024-04-25T10:03:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd427991640 1 -- 172.21.15.133:0/3337693178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd420068ef0 con 0x7fd4200cee80 2024-04-25T10:03:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.568+0000 7fd415ffb640 1 -- 172.21.15.133:0/3337693178 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd41c0c7440 con 0x7fd4200cee80 2024-04-25T10:03:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.572+0000 7fd415ffb640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd404066f60 0x7fd404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.572+0000 7fd415ffb640 1 -- 172.21.15.133:0/3337693178 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd41c1201e0 con 0x7fd4200cee80 2024-04-25T10:03:03.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.572+0000 7fd424f2c640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd404066f60 0x7fd404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.572+0000 7fd424f2c640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd404066f60 0x7fd404069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd420148fd0 tx=0x7fd410009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.576+0000 7fd415ffb640 1 -- 172.21.15.133:0/3337693178 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd41c016110 con 0x7fd4200cee80 2024-04-25T10:03:03.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.752+0000 7fd427991640 1 -- 172.21.15.133:0/3337693178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4201528f0 con 0x7fd4200cee80 2024-04-25T10:03:03.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.752+0000 7fd415ffb640 1 -- 172.21.15.133:0/3337693178 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7fd41c0f0e00 con 0x7fd4200cee80 2024-04-25T10:03:03.761 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:03.761 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2230831968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2230831968},{"type":"v1","addr":"172.21.15.162:6835","nonce":2230831968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1456396097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1456396097},{"type":"v1","addr":"172.21.15.133:6836","nonce":1456396097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:51.276631+0000","modified":"2024-04-25T10:02:14.440426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2462154167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2462154167},{"type":"v1","addr":"172.21.15.133:6837","nonce":2462154167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:03.761 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 9 2024-04-25T10:03:03.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 -- 172.21.15.133:0/3337693178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd404066f60 msgr2=0x7fd404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd404066f60 0x7fd404069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd420148fd0 tx=0x7fd410009040 comp rx=0 tx=0).stop 2024-04-25T10:03:03.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 -- 172.21.15.133:0/3337693178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4200cee80 msgr2=0x7fd420146920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4200cee80 0x7fd420146920 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0190a0 tx=0x7fd41c01acd0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 -- 172.21.15.133:0/3337693178 shutdown_connections 2024-04-25T10:03:03.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd42015dd90 0x7fd42014e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd42015d9b0 0x7fd42015fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd404066f60 0x7fd404069420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 --2- 172.21.15.133:0/3337693178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4200cee80 0x7fd420146920 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 -- 172.21.15.133:0/3337693178 >> 172.21.15.133:0/3337693178 conn(0x7fd420067000 msgr2=0x7fd4200cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 -- 172.21.15.133:0/3337693178 shutdown_connections 2024-04-25T10:03:03.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.756+0000 7fd427991640 1 -- 172.21.15.133:0/3337693178 wait complete. 2024-04-25T10:03:03.778 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:03.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.888+0000 7fda4d172640 1 Processor -- start 2024-04-25T10:03:03.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda4d172640 1 -- start start 2024-04-25T10:03:03.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda4d172640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fda4815c9a0 0x7fda480c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda4d172640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda480b1f70 0x7fda4806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda4d172640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda480c61d0 con 0x7fda480ce330 2024-04-25T10:03:03.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda4d172640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda480c64b0 con 0x7fda4815c9a0 2024-04-25T10:03:03.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda4d172640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4806d4f0 con 0x7fda480b1f70 2024-04-25T10:03:03.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda46d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fda4815c9a0 0x7fda480c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda4659c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda480b1f70 0x7fda4806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda4759e640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fda480ce330 0x7fda4815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:59024/0 (socket says 172.21.15.133:59024) 2024-04-25T10:03:03.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda4759e640 1 -- 172.21.15.133:0/1846437972 learned_addr learned my addr 172.21.15.133:0/1846437972 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:03.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda46d9d640 1 -- 172.21.15.133:0/1846437972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda480b1f70 msgr2=0x7fda4806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda46d9d640 1 --2- 172.21.15.133:0/1846437972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda480b1f70 0x7fda4806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda46d9d640 1 -- 172.21.15.133:0/1846437972 >> v1:172.21.15.162:6789/0 conn(0x7fda480ce330 legacy=0x7fda4815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda46d9d640 1 -- 172.21.15.133:0/1846437972 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4806d7d0 con 0x7fda4815c9a0 2024-04-25T10:03:03.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.892+0000 7fda4659c640 1 --2- 172.21.15.133:0/1846437972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda480b1f70 0x7fda4806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:03.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda46d9d640 1 --2- 172.21.15.133:0/1846437972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda4815c9a0 0x7fda480c5980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fda380097c0 tx=0x7fda3801c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ad804ca0ce5800c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda45d9b640 1 -- 172.21.15.133:0/1846437972 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda380b0070 con 0x7fda4815c9a0 2024-04-25T10:03:03.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda45d9b640 1 -- 172.21.15.133:0/1846437972 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda38004230 con 0x7fda4815c9a0 2024-04-25T10:03:03.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda45d9b640 1 -- 172.21.15.133:0/1846437972 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda38003620 con 0x7fda4815c9a0 2024-04-25T10:03:03.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 -- 172.21.15.133:0/1846437972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda4815c9a0 msgr2=0x7fda480c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 --2- 172.21.15.133:0/1846437972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda4815c9a0 0x7fda480c5980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fda380097c0 tx=0x7fda3801c440 comp rx=0 tx=0).stop 2024-04-25T10:03:03.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 -- 172.21.15.133:0/1846437972 shutdown_connections 2024-04-25T10:03:03.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 --2- 172.21.15.133:0/1846437972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda480b1f70 0x7fda4806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 --2- 172.21.15.133:0/1846437972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda4815c9a0 0x7fda480c5980 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 -- 172.21.15.133:0/1846437972 >> 172.21.15.133:0/1846437972 conn(0x7fda48067000 msgr2=0x7fda4806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 -- 172.21.15.133:0/1846437972 shutdown_connections 2024-04-25T10:03:03.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 -- 172.21.15.133:0/1846437972 wait complete. 2024-04-25T10:03:03.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 Processor -- start 2024-04-25T10:03:03.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 -- start start 2024-04-25T10:03:03.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda480b1f70 0x7fda48150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda480ce330 0x7fda48157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4815c9a0 0x7fda480c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda480ceb50 con 0x7fda4815c9a0 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4815cd80 con 0x7fda480b1f70 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4d172640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda481580d0 con 0x7fda480ce330 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4815c9a0 0x7fda480c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4815c9a0 0x7fda480c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55244/0 (socket says 172.21.15.133:55244) 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4759e640 1 -- 172.21.15.133:0/1982845866 learned_addr learned my addr 172.21.15.133:0/1982845866 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda46d9d640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda480b1f70 0x7fda48150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.896+0000 7fda4659c640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda480ce330 0x7fda48157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda4759e640 1 -- 172.21.15.133:0/1982845866 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda480ce330 msgr2=0x7fda48157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda4759e640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda480ce330 0x7fda48157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda4759e640 1 -- 172.21.15.133:0/1982845866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda480b1f70 msgr2=0x7fda48150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda4759e640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda480b1f70 0x7fda48150080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda4759e640 1 -- 172.21.15.133:0/1982845866 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda480ca740 con 0x7fda4815c9a0 2024-04-25T10:03:03.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda4759e640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4815c9a0 0x7fda480c8560 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fda4000ca80 tx=0x7fda4000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda3f7fe640 1 -- 172.21.15.133:0/1982845866 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40017070 con 0x7fda4815c9a0 2024-04-25T10:03:03.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda3f7fe640 1 -- 172.21.15.133:0/1982845866 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4000ac30 con 0x7fda4815c9a0 2024-04-25T10:03:03.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda3f7fe640 1 -- 172.21.15.133:0/1982845866 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40012600 con 0x7fda4815c9a0 2024-04-25T10:03:03.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda4d172640 1 -- 172.21.15.133:0/1982845866 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda480c8aa0 con 0x7fda4815c9a0 2024-04-25T10:03:03.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda4d172640 1 -- 172.21.15.133:0/1982845866 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda480ca9e0 con 0x7fda4815c9a0 2024-04-25T10:03:03.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda4d172640 1 -- 172.21.15.133:0/1982845866 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4806bc20 con 0x7fda4815c9a0 2024-04-25T10:03:03.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda3f7fe640 1 -- 172.21.15.133:0/1982845866 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda4001a070 con 0x7fda4815c9a0 2024-04-25T10:03:03.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda3f7fe640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda28066f60 0x7fda28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda3f7fe640 1 -- 172.21.15.133:0/1982845866 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fda40084f90 con 0x7fda4815c9a0 2024-04-25T10:03:03.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda46d9d640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda28066f60 0x7fda28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.900+0000 7fda46d9d640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda28066f60 0x7fda28069420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda380ad000 tx=0x7fda38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:03.904+0000 7fda3f7fe640 1 -- 172.21.15.133:0/1982845866 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda4000ae20 con 0x7fda4815c9a0 2024-04-25T10:03:04.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.068+0000 7fda4d172640 1 -- 172.21.15.133:0/1982845866 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda48068ef0 con 0x7fda4815c9a0 2024-04-25T10:03:04.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda3f7fe640 1 -- 172.21.15.133:0/1982845866 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7fda400555e0 con 0x7fda4815c9a0 2024-04-25T10:03:04.078 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:04.079 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:04.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 -- 172.21.15.133:0/1982845866 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda28066f60 msgr2=0x7fda28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda28066f60 0x7fda28069420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda380ad000 tx=0x7fda38008040 comp rx=0 tx=0).stop 2024-04-25T10:03:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 -- 172.21.15.133:0/1982845866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4815c9a0 msgr2=0x7fda480c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4815c9a0 0x7fda480c8560 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fda4000ca80 tx=0x7fda4000cf40 comp rx=0 tx=0).stop 2024-04-25T10:03:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 -- 172.21.15.133:0/1982845866 shutdown_connections 2024-04-25T10:03:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda28066f60 0x7fda28069420 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4815c9a0 0x7fda480c8560 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda480ce330 0x7fda48157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 --2- 172.21.15.133:0/1982845866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda480b1f70 0x7fda48150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 -- 172.21.15.133:0/1982845866 >> 172.21.15.133:0/1982845866 conn(0x7fda48067000 msgr2=0x7fda481537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 -- 172.21.15.133:0/1982845866 shutdown_connections 2024-04-25T10:03:04.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.072+0000 7fda4d172640 1 -- 172.21.15.133:0/1982845866 wait complete. 2024-04-25T10:03:04.098 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:49.812095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:50.262173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:01:54.010420+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:04.099 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:04.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e91db9640 1 Processor -- start 2024-04-25T10:03:04.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e91db9640 1 -- start start 2024-04-25T10:03:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e91db9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c06e8b0 0x7f2e8c15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e91db9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e8c0b1e50 0x7f2e8c146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e91db9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8c0b1090 con 0x7f2e8c0aebc0 2024-04-25T10:03:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e91db9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8c147490 con 0x7f2e8c0b1e50 2024-04-25T10:03:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e91db9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c0b19c0 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e8b7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c06e8b0 0x7f2e8c15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e8affd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2e8c0b1e50 0x7f2e8c146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e8b7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c06e8b0 0x7f2e8c15eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:58186/0 (socket says 172.21.15.133:58186) 2024-04-25T10:03:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e8b7fe640 1 -- 172.21.15.133:0/550532417 learned_addr learned my addr 172.21.15.133:0/550532417 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e8b7fe640 1 -- 172.21.15.133:0/550532417 >> v2:172.21.15.162:3300/0 conn(0x7f2e8c0b1e50 msgr2=0x7f2e8c146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e8b7fe640 1 --2- 172.21.15.133:0/550532417 >> v2:172.21.15.162:3300/0 conn(0x7f2e8c0b1e50 0x7f2e8c146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e8b7fe640 1 -- 172.21.15.133:0/550532417 >> v1:172.21.15.162:6789/0 conn(0x7f2e8c0aebc0 legacy=0x7f2e8c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e8b7fe640 1 -- 172.21.15.133:0/550532417 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0b1370 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.216+0000 7f2e8affd640 1 --2- 172.21.15.133:0/550532417 >> v2:172.21.15.162:3300/0 conn(0x7f2e8c0b1e50 0x7f2e8c146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:04.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e8b7fe640 1 --2- 172.21.15.133:0/550532417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c06e8b0 0x7f2e8c15eb40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0098a0 tx=0x7f2e7c01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d532dfa8ac1c75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e8a7fc640 1 -- 172.21.15.133:0/550532417 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0b1070 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e8a7fc640 1 -- 172.21.15.133:0/550532417 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c0ac620 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e8a7fc640 1 -- 172.21.15.133:0/550532417 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0ac9f0 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e91db9640 1 -- 172.21.15.133:0/550532417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c06e8b0 msgr2=0x7f2e8c15eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e91db9640 1 --2- 172.21.15.133:0/550532417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c06e8b0 0x7f2e8c15eb40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0098a0 tx=0x7f2e7c01c4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e91db9640 1 -- 172.21.15.133:0/550532417 shutdown_connections 2024-04-25T10:03:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e91db9640 1 --2- 172.21.15.133:0/550532417 >> v2:172.21.15.162:3300/0 conn(0x7f2e8c0b1e50 0x7f2e8c146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e91db9640 1 --2- 172.21.15.133:0/550532417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c06e8b0 0x7f2e8c15eb40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e91db9640 1 -- 172.21.15.133:0/550532417 >> 172.21.15.133:0/550532417 conn(0x7f2e8c066f70 msgr2=0x7f2e8c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e91db9640 1 -- 172.21.15.133:0/550532417 shutdown_connections 2024-04-25T10:03:04.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.220+0000 7f2e91db9640 1 -- 172.21.15.133:0/550532417 wait complete. 2024-04-25T10:03:04.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e91db9640 1 Processor -- start 2024-04-25T10:03:04.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e91db9640 1 -- start start 2024-04-25T10:03:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e91db9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e8c06e8b0 0x7f2e8c15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e91db9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e8c0aebc0 0x7f2e8c13f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e91db9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c0b1e50 0x7f2e8c0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e91db9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c0b23b0 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e91db9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c0b2230 con 0x7f2e8c0aebc0 2024-04-25T10:03:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e91db9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c147020 con 0x7f2e8c0b1e50 2024-04-25T10:03:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e8c06e8b0 0x7f2e8c15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e8c06e8b0 0x7f2e8c15c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55254/0 (socket says 172.21.15.133:55254) 2024-04-25T10:03:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8b7fe640 1 -- 172.21.15.133:0/3276303042 learned_addr learned my addr 172.21.15.133:0/3276303042 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8affd640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e8c0aebc0 0x7f2e8c13f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8bfff640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c0b1e50 0x7f2e8c0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8b7fe640 1 -- 172.21.15.133:0/3276303042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c0b1e50 msgr2=0x7f2e8c0ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8b7fe640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c0b1e50 0x7f2e8c0ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8b7fe640 1 -- 172.21.15.133:0/3276303042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e8c0aebc0 msgr2=0x7f2e8c13f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8b7fe640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e8c0aebc0 0x7f2e8c13f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8b7fe640 1 -- 172.21.15.133:0/3276303042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c13fb10 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8affd640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e8c0aebc0 0x7f2e8c13f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e8b7fe640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e8c06e8b0 0x7f2e8c15c7c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0af040 tx=0x7f2e7c01acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e73fff640 1 -- 172.21.15.133:0/3276303042 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0b1040 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e73fff640 1 -- 172.21.15.133:0/3276303042 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c0ac420 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e91db9640 1 -- 172.21.15.133:0/3276303042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c13fdf0 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e73fff640 1 -- 172.21.15.133:0/3276303042 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0b7660 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e91db9640 1 -- 172.21.15.133:0/3276303042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8c15cd00 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e91db9640 1 -- 172.21.15.133:0/3276303042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8c1f6e90 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.224+0000 7f2e73fff640 1 -- 172.21.15.133:0/3276303042 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e7c0bf030 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.228+0000 7f2e73fff640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2e78066f60 0x7f2e78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.228+0000 7f2e8affd640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2e78066f60 0x7f2e78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.228+0000 7f2e73fff640 1 -- 172.21.15.133:0/3276303042 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2e7c11ed30 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.228+0000 7f2e8affd640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2e78066f60 0x7f2e78069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c15ec80 tx=0x7f2e80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.232+0000 7f2e73fff640 1 -- 172.21.15.133:0/3276303042 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e8c1f6e90 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.388+0000 7f2e91db9640 1 -- 172.21.15.133:0/3276303042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e8c06b960 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.388+0000 7f2e73fff640 1 -- 172.21.15.133:0/3276303042 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e7c016130 con 0x7f2e8c06e8b0 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 -- 172.21.15.133:0/3276303042 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2e78066f60 msgr2=0x7f2e78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2e78066f60 0x7f2e78069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c15ec80 tx=0x7f2e80008040 comp rx=0 tx=0).stop 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 -- 172.21.15.133:0/3276303042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e8c06e8b0 msgr2=0x7f2e8c15c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e8c06e8b0 0x7f2e8c15c7c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0af040 tx=0x7f2e7c01acf0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 -- 172.21.15.133:0/3276303042 shutdown_connections 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2e8c0b1e50 0x7f2e8c0ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e8c0aebc0 0x7f2e8c13f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2e78066f60 0x7f2e78069420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 --2- 172.21.15.133:0/3276303042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e8c06e8b0 0x7f2e8c15c7c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 -- 172.21.15.133:0/3276303042 >> 172.21.15.133:0/3276303042 conn(0x7f2e8c066f70 msgr2=0x7f2e8c0b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 -- 172.21.15.133:0/3276303042 shutdown_connections 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.392+0000 7f2e91db9640 1 -- 172.21.15.133:0/3276303042 wait complete. 2024-04-25T10:03:04.414 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.524+0000 7f434ccd2640 1 Processor -- start 2024-04-25T10:03:04.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.524+0000 7f434ccd2640 1 -- start start 2024-04-25T10:03:04.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.528+0000 7f434ccd2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f434806e8b0 0x7f434815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.528+0000 7f434ccd2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43481471f0 con 0x7f43480b1e50 2024-04-25T10:03:04.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.528+0000 7f434ccd2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43480b1090 con 0x7f43480aebc0 2024-04-25T10:03:04.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.528+0000 7f434ccd2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4348147490 con 0x7f434806e8b0 2024-04-25T10:03:04.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.528+0000 7f434659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f434806e8b0 0x7f434815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.528+0000 7f434659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f434806e8b0 0x7f434815eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55262/0 (socket says 172.21.15.133:55262) 2024-04-25T10:03:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.528+0000 7f434659c640 1 -- 172.21.15.133:0/3756737384 learned_addr learned my addr 172.21.15.133:0/3756737384 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.528+0000 7f434659c640 1 -- 172.21.15.133:0/3756737384 >> v1:172.21.15.162:6789/0 conn(0x7f43480aebc0 legacy=0x7f434806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.528+0000 7f434659c640 1 -- 172.21.15.133:0/3756737384 >> v1:172.21.15.133:6789/0 conn(0x7f43480b1e50 legacy=0x7f4348146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.528+0000 7f434659c640 1 -- 172.21.15.133:0/3756737384 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43480b1370 con 0x7f434806e8b0 2024-04-25T10:03:04.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.528+0000 7f434659c640 1 --2- 172.21.15.133:0/3756737384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f434806e8b0 0x7f434815eb40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4338009870 tx=0x7f433801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee74dd973b301961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4345d9b640 1 -- 172.21.15.133:0/3756737384 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43380b0070 con 0x7f434806e8b0 2024-04-25T10:03:04.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4345d9b640 1 -- 172.21.15.133:0/3756737384 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43380ab600 con 0x7f434806e8b0 2024-04-25T10:03:04.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4345d9b640 1 -- 172.21.15.133:0/3756737384 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43380ab9d0 con 0x7f434806e8b0 2024-04-25T10:03:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 -- 172.21.15.133:0/3756737384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f434806e8b0 msgr2=0x7f434815eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 --2- 172.21.15.133:0/3756737384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f434806e8b0 0x7f434815eb40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4338009870 tx=0x7f433801c490 comp rx=0 tx=0).stop 2024-04-25T10:03:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 -- 172.21.15.133:0/3756737384 shutdown_connections 2024-04-25T10:03:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 --2- 172.21.15.133:0/3756737384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f434806e8b0 0x7f434815eb40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 -- 172.21.15.133:0/3756737384 >> 172.21.15.133:0/3756737384 conn(0x7f4348066f70 msgr2=0x7f434806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 -- 172.21.15.133:0/3756737384 shutdown_connections 2024-04-25T10:03:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 -- 172.21.15.133:0/3756737384 wait complete. 2024-04-25T10:03:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 Processor -- start 2024-04-25T10:03:04.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 -- start start 2024-04-25T10:03:04.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f434806e8b0 0x7f434815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43480aebc0 0x7f4348139f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43480b1e50 0x7f434813ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43480b23b0 con 0x7f43480b1e50 2024-04-25T10:03:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43480b2230 con 0x7f434806e8b0 2024-04-25T10:03:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434816c0b0 con 0x7f43480aebc0 2024-04-25T10:03:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4346d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43480b1e50 0x7f434813ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4346d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43480b1e50 0x7f434813ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55272/0 (socket says 172.21.15.133:55272) 2024-04-25T10:03:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4346d9d640 1 -- 172.21.15.133:0/847404977 learned_addr learned my addr 172.21.15.133:0/847404977 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4346d9d640 1 -- 172.21.15.133:0/847404977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43480aebc0 msgr2=0x7f4348139f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434659c640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f434806e8b0 0x7f434815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f433dd9b640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43480aebc0 0x7f4348139f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4346d9d640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43480aebc0 0x7f4348139f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4346d9d640 1 -- 172.21.15.133:0/847404977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f434806e8b0 msgr2=0x7f434815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4346d9d640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f434806e8b0 0x7f434815d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4346d9d640 1 -- 172.21.15.133:0/847404977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434813a9b0 con 0x7f43480b1e50 2024-04-25T10:03:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f4346d9d640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43480b1e50 0x7f434813ead0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f434000c7b0 tx=0x7f434000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f433f7fe640 1 -- 172.21.15.133:0/847404977 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340004b70 con 0x7f43480b1e50 2024-04-25T10:03:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f433f7fe640 1 -- 172.21.15.133:0/847404977 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4340004d10 con 0x7f43480b1e50 2024-04-25T10:03:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 -- 172.21.15.133:0/847404977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434813ac90 con 0x7f43480b1e50 2024-04-25T10:03:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f433f7fe640 1 -- 172.21.15.133:0/847404977 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000aa90 con 0x7f43480b1e50 2024-04-25T10:03:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.532+0000 7f434ccd2640 1 -- 172.21.15.133:0/847404977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434813b250 con 0x7f43480b1e50 2024-04-25T10:03:04.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.536+0000 7f434ccd2640 1 -- 172.21.15.133:0/847404977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4348068e80 con 0x7f43480b1e50 2024-04-25T10:03:04.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.536+0000 7f433f7fe640 1 -- 172.21.15.133:0/847404977 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f434000ac30 con 0x7f43480b1e50 2024-04-25T10:03:04.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.536+0000 7f433f7fe640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4328066f60 0x7f4328069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.536+0000 7f433f7fe640 1 -- 172.21.15.133:0/847404977 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4340085650 con 0x7f43480b1e50 2024-04-25T10:03:04.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.536+0000 7f434659c640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4328066f60 0x7f4328069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.536+0000 7f434659c640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4328066f60 0x7f4328069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43380ad000 tx=0x7f4338008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.540+0000 7f433f7fe640 1 -- 172.21.15.133:0/847404977 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4340020070 con 0x7f43480b1e50 2024-04-25T10:03:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.688+0000 7f434ccd2640 1 -- 172.21.15.133:0/847404977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43481429a0 con 0x7f43480b1e50 2024-04-25T10:03:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.688+0000 7f433f7fe640 1 -- 172.21.15.133:0/847404977 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43400562f0 con 0x7f43480b1e50 2024-04-25T10:03:04.695 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:03:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 -- 172.21.15.133:0/847404977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4328066f60 msgr2=0x7f4328069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4328066f60 0x7f4328069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43380ad000 tx=0x7f4338008040 comp rx=0 tx=0).stop 2024-04-25T10:03:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 -- 172.21.15.133:0/847404977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43480b1e50 msgr2=0x7f434813ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43480b1e50 0x7f434813ead0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f434000c7b0 tx=0x7f434000cc70 comp rx=0 tx=0).stop 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 -- 172.21.15.133:0/847404977 shutdown_connections 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4328066f60 0x7f4328069420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43480b1e50 0x7f434813ead0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43480aebc0 0x7f4348139f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 --2- 172.21.15.133:0/847404977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f434806e8b0 0x7f434815d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 -- 172.21.15.133:0/847404977 >> 172.21.15.133:0/847404977 conn(0x7f4348066f70 msgr2=0x7f43480ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 -- 172.21.15.133:0/847404977 shutdown_connections 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.692+0000 7f434ccd2640 1 -- 172.21.15.133:0/847404977 wait complete. 2024-04-25T10:03:04.714 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:04.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b3c1c5640 1 Processor -- start 2024-04-25T10:03:04.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b3c1c5640 1 -- start start 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b3c1c5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b340b2150 0x7f0b3406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b3c1c5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3415ac50 con 0x7f0b34143ea0 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b3c1c5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b340cd180 con 0x7f0b3415b9b0 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b3c1c5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340ccf20 con 0x7f0b340b2150 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39f61640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0b34143ea0 0x7f0b340cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35904/0 (socket says 172.21.15.133:35904) 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39f61640 1 -- 172.21.15.133:0/2573285543 learned_addr learned my addr 172.21.15.133:0/2573285543 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355681569 0 0) 0x7f0b30001120 con 0x7f0b34143ea0 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b20003530 con 0x7f0b34143ea0 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39760640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b340b2150 0x7f0b3406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1749227624 0 0) 0x7f0b20003530 con 0x7f0b34143ea0 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b340b2150 msgr2=0x7f0b3406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b340b2150 0x7f0b3406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 >> v1:172.21.15.162:6789/0 conn(0x7f0b3415b9b0 legacy=0x7f0b340b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3415aed0 con 0x7f0b34143ea0 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0b30001850 con 0x7f0b34143ea0 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39760640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b340b2150 0x7f0b3406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 >> v1:172.21.15.133:6789/0 conn(0x7f0b34143ea0 legacy=0x7f0b340cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 0x7f0b20003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b20004810 0x7f0b20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b340b2150 0x7f0b200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340ccf20 con 0x7f0b200039f0 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2000bad0 con 0x7f0b20004810 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2000bcd0 con 0x7f0b340b2150 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39f61640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 0x7f0b20003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39760640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b20004810 0x7f0b20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b3a762640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b340b2150 0x7f0b200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39f61640 1 -- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b340b2150 msgr2=0x7f0b200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39f61640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b340b2150 0x7f0b200051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39f61640 1 -- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b20004810 msgr2=0x7f0b20004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39f61640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b20004810 0x7f0b20004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39f61640 1 -- 172.21.15.133:0/2573285543 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b20003530 con 0x7f0b200039f0 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.824+0000 7f0b39760640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b20004810 0x7f0b20004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b39f61640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 0x7f0b20003e10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000c4c0 tx=0x7f0b3001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66da6cab80f9488 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000c0e0 con 0x7f0b200039f0 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b300a9070 con 0x7f0b200039f0 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b38f5f640 1 -- 172.21.15.133:0/2573285543 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30005d40 con 0x7f0b200039f0 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/2573285543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 msgr2=0x7f0b20003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b3c1c5640 1 --2- 172.21.15.133:0/2573285543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 0x7f0b20003e10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000c4c0 tx=0x7f0b3001cc80 comp rx=0 tx=0).stop 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b3a762640 1 -- 172.21.15.133:0/2573285543 reap_dead start 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/2573285543 shutdown_connections 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/2573285543 >> 172.21.15.133:0/2573285543 conn(0x7f0b34066f70 msgr2=0x7f0b3406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/2573285543 shutdown_connections 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/2573285543 wait complete. 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b3c1c5640 1 Processor -- start 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.828+0000 7f0b3c1c5640 1 -- start start 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b3c1c5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 0x7f0b341eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b3c1c5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b34157d30 0x7f0b34142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b3c1c5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b3415cb10 0x7f0b3415cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b3c1c5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3406e740 con 0x7f0b200039f0 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b3c1c5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3406e5c0 con 0x7f0b34157d30 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b3c1c5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3406e8c0 con 0x7f0b3415cb10 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39f61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 0x7f0b341eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39f61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 0x7f0b341eb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55292/0 (socket says 172.21.15.133:55292) 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39f61640 1 -- 172.21.15.133:0/3187415918 learned_addr learned my addr 172.21.15.133:0/3187415918 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b3a762640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b3415cb10 0x7f0b3415cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39760640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b34157d30 0x7f0b34142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39f61640 1 -- 172.21.15.133:0/3187415918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b3415cb10 msgr2=0x7f0b3415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39f61640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b3415cb10 0x7f0b3415cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39f61640 1 -- 172.21.15.133:0/3187415918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b34157d30 msgr2=0x7f0b34142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39f61640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b34157d30 0x7f0b34142c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39f61640 1 -- 172.21.15.133:0/3187415918 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34154c20 con 0x7f0b200039f0 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39f61640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 0x7f0b341eb7d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0b30001b50 tx=0x7f0b20005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b2a7fc640 1 -- 172.21.15.133:0/3187415918 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30001120 con 0x7f0b200039f0 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b2a7fc640 1 -- 172.21.15.133:0/3187415918 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b30005d20 con 0x7f0b200039f0 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/3187415918 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b341581b0 con 0x7f0b200039f0 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b2a7fc640 1 -- 172.21.15.133:0/3187415918 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b300049f0 con 0x7f0b200039f0 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/3187415918 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b34154940 con 0x7f0b200039f0 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/3187415918 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b34068e80 con 0x7f0b200039f0 2024-04-25T10:03:04.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b2a7fc640 1 -- 172.21.15.133:0/3187415918 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b30004b90 con 0x7f0b200039f0 2024-04-25T10:03:04.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b2a7fc640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b18066f60 0x7f0b18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b2a7fc640 1 -- 172.21.15.133:0/3187415918 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0b3011ee90 con 0x7f0b200039f0 2024-04-25T10:03:04.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39760640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b18066f60 0x7f0b18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.832+0000 7f0b39760640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b18066f60 0x7f0b18069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0b340b0d20 tx=0x7f0b24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.836+0000 7f0b2a7fc640 1 -- 172.21.15.133:0/3187415918 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b3415ac50 con 0x7f0b200039f0 2024-04-25T10:03:04.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/3187415918 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b3415bdf0 con 0x7f0b200039f0 2024-04-25T10:03:04.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b2a7fc640 1 -- 172.21.15.133:0/3187415918 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b300efb30 con 0x7f0b200039f0 2024-04-25T10:03:04.991 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:03:04.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/3187415918 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b18066f60 msgr2=0x7f0b18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b18066f60 0x7f0b18069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0b340b0d20 tx=0x7f0b24008040 comp rx=0 tx=0).stop 2024-04-25T10:03:04.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/3187415918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 msgr2=0x7f0b341eb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 0x7f0b341eb7d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0b30001b50 tx=0x7f0b20005840 comp rx=0 tx=0).stop 2024-04-25T10:03:04.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/3187415918 shutdown_connections 2024-04-25T10:03:04.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b3415cb10 0x7f0b3415cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b34157d30 0x7f0b34142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b18066f60 0x7f0b18069420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 --2- 172.21.15.133:0/3187415918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b200039f0 0x7f0b341eb7d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/3187415918 >> 172.21.15.133:0/3187415918 conn(0x7f0b34066f70 msgr2=0x7f0b34160670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/3187415918 shutdown_connections 2024-04-25T10:03:04.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:04.984+0000 7f0b3c1c5640 1 -- 172.21.15.133:0/3187415918 wait complete. 2024-04-25T10:03:05.008 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:05.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.124+0000 7ff04e494640 1 Processor -- start 2024-04-25T10:03:05.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.124+0000 7ff04e494640 1 -- start start 2024-04-25T10:03:05.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff04e494640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04815dff0 0x7ff0480cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff04e494640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff04806bd80 0x7ff0481675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff04e494640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0480ce0b0 con 0x7ff0480cb240 2024-04-25T10:03:05.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff04e494640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0480cb680 con 0x7ff04806bd80 2024-04-25T10:03:05.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff04e494640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480ce350 con 0x7ff04815dff0 2024-04-25T10:03:05.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff04ca31640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff04806bd80 0x7ff0481675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff047fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04815dff0 0x7ff0480cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff04ca31640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff04806bd80 0x7ff0481675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:41470/0 (socket says 172.21.15.133:41470) 2024-04-25T10:03:05.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff04ca31640 1 -- 172.21.15.133:0/4219922608 learned_addr learned my addr 172.21.15.133:0/4219922608 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:05.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff047fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04815dff0 0x7ff0480cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:58240/0 (socket says 172.21.15.133:58240) 2024-04-25T10:03:05.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff0477fe640 1 --1- 172.21.15.133:0/4219922608 >> v1:172.21.15.162:6789/0 conn(0x7ff0480cb240 0x7ff04815e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:59060/0 (socket says 172.21.15.133:59060) 2024-04-25T10:03:05.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff047fff640 1 -- 172.21.15.133:0/4219922608 >> v2:172.21.15.162:3300/0 conn(0x7ff04806bd80 msgr2=0x7ff0481675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff047fff640 1 --2- 172.21.15.133:0/4219922608 >> v2:172.21.15.162:3300/0 conn(0x7ff04806bd80 0x7ff0481675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff047fff640 1 -- 172.21.15.133:0/4219922608 >> v1:172.21.15.162:6789/0 conn(0x7ff0480cb240 legacy=0x7ff04815e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.128+0000 7ff047fff640 1 -- 172.21.15.133:0/4219922608 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0480ce630 con 0x7ff04815dff0 2024-04-25T10:03:05.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff047fff640 1 --2- 172.21.15.133:0/4219922608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04815dff0 0x7ff0480cad00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff0380097c0 tx=0x7ff03801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c502846dbc81aab9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff046ffd640 1 -- 172.21.15.133:0/4219922608 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380b0070 con 0x7ff04815dff0 2024-04-25T10:03:05.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff046ffd640 1 -- 172.21.15.133:0/4219922608 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0380ab600 con 0x7ff04815dff0 2024-04-25T10:03:05.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff046ffd640 1 -- 172.21.15.133:0/4219922608 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380ab9d0 con 0x7ff04815dff0 2024-04-25T10:03:05.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff04e494640 1 -- 172.21.15.133:0/4219922608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04815dff0 msgr2=0x7ff0480cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff04e494640 1 --2- 172.21.15.133:0/4219922608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04815dff0 0x7ff0480cad00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff0380097c0 tx=0x7ff03801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff04e494640 1 -- 172.21.15.133:0/4219922608 shutdown_connections 2024-04-25T10:03:05.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff04e494640 1 --2- 172.21.15.133:0/4219922608 >> v2:172.21.15.162:3300/0 conn(0x7ff04806bd80 0x7ff0481675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff04e494640 1 --2- 172.21.15.133:0/4219922608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff04815dff0 0x7ff0480cad00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff04e494640 1 -- 172.21.15.133:0/4219922608 >> 172.21.15.133:0/4219922608 conn(0x7ff0480cc790 msgr2=0x7ff0480ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff04e494640 1 -- 172.21.15.133:0/4219922608 shutdown_connections 2024-04-25T10:03:05.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff04e494640 1 -- 172.21.15.133:0/4219922608 wait complete. 2024-04-25T10:03:05.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff04e494640 1 Processor -- start 2024-04-25T10:03:05.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.132+0000 7ff04e494640 1 -- start start 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff04e494640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff04806bd80 0x7ff048159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff04e494640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0480cb240 0x7ff048142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff04e494640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff04815dff0 0x7ff04815cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff04e494640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04816a480 con 0x7ff04806bd80 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff04e494640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04816a300 con 0x7ff04815dff0 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff04e494640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04816a600 con 0x7ff0480cb240 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff047fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff04806bd80 0x7ff048159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff047fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff04806bd80 0x7ff048159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55296/0 (socket says 172.21.15.133:55296) 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff047fff640 1 -- 172.21.15.133:0/3937461976 learned_addr learned my addr 172.21.15.133:0/3937461976 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff04ca31640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff04815dff0 0x7ff04815cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff0477fe640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0480cb240 0x7ff048142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff047fff640 1 -- 172.21.15.133:0/3937461976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0480cb240 msgr2=0x7ff048142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff047fff640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0480cb240 0x7ff048142010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff047fff640 1 -- 172.21.15.133:0/3937461976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff04815dff0 msgr2=0x7ff04815cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff047fff640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff04815dff0 0x7ff04815cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff047fff640 1 -- 172.21.15.133:0/3937461976 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04815a7e0 con 0x7ff04806bd80 2024-04-25T10:03:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff047fff640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff04806bd80 0x7ff048159f90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff03801a0d0 tx=0x7ff038019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff044ff9640 1 -- 172.21.15.133:0/3937461976 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380b0040 con 0x7ff04806bd80 2024-04-25T10:03:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff044ff9640 1 -- 172.21.15.133:0/3937461976 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0380bd070 con 0x7ff04806bd80 2024-04-25T10:03:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff04e494640 1 -- 172.21.15.133:0/3937461976 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04815aac0 con 0x7ff04806bd80 2024-04-25T10:03:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff044ff9640 1 -- 172.21.15.133:0/3937461976 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380b5680 con 0x7ff04806bd80 2024-04-25T10:03:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff04e494640 1 -- 172.21.15.133:0/3937461976 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04815b080 con 0x7ff04806bd80 2024-04-25T10:03:05.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff04e494640 1 -- 172.21.15.133:0/3937461976 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff048068e80 con 0x7ff04806bd80 2024-04-25T10:03:05.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff044ff9640 1 -- 172.21.15.133:0/3937461976 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0380b3070 con 0x7ff04806bd80 2024-04-25T10:03:05.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff044ff9640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff034066f60 0x7ff034069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff044ff9640 1 -- 172.21.15.133:0/3937461976 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff03811f540 con 0x7ff04806bd80 2024-04-25T10:03:05.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff0477fe640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff034066f60 0x7ff034069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.136+0000 7ff0477fe640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff034066f60 0x7ff034069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff048157f60 tx=0x7ff03c007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.140+0000 7ff044ff9640 1 -- 172.21.15.133:0/3937461976 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff038016090 con 0x7ff04806bd80 2024-04-25T10:03:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 -- 172.21.15.133:0/3937461976 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff04815f400 con 0x7ff04806bd80 2024-04-25T10:03:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff044ff9640 1 -- 172.21.15.133:0/3937461976 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff0380f00d0 con 0x7ff04806bd80 2024-04-25T10:03:05.306 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:03:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 -- 172.21.15.133:0/3937461976 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff034066f60 msgr2=0x7ff034069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff034066f60 0x7ff034069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff048157f60 tx=0x7ff03c007cd0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 -- 172.21.15.133:0/3937461976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff04806bd80 msgr2=0x7ff048159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff04806bd80 0x7ff048159f90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff03801a0d0 tx=0x7ff038019240 comp rx=0 tx=0).stop 2024-04-25T10:03:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 -- 172.21.15.133:0/3937461976 shutdown_connections 2024-04-25T10:03:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff04815dff0 0x7ff04815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0480cb240 0x7ff048142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff034066f60 0x7ff034069420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 --2- 172.21.15.133:0/3937461976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff04806bd80 0x7ff048159f90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 -- 172.21.15.133:0/3937461976 >> 172.21.15.133:0/3937461976 conn(0x7ff0480cc790 msgr2=0x7ff04815ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 -- 172.21.15.133:0/3937461976 shutdown_connections 2024-04-25T10:03:05.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.300+0000 7ff04e494640 1 -- 172.21.15.133:0/3937461976 wait complete. 2024-04-25T10:03:05.325 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.436+0000 7f4d86b28640 1 Processor -- start 2024-04-25T10:03:05.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.436+0000 7f4d86b28640 1 -- start start 2024-04-25T10:03:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.436+0000 7f4d86b28640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d8015d9b0 0x7f4d800cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.436+0000 7f4d86b28640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d8015dd90 0x7f4d800ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.436+0000 7f4d86b28640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d800b19f0 0x7f4d800cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.436+0000 7f4d86b28640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d800cbe30 con 0x7f4d8015d9b0 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.436+0000 7f4d86b28640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d800ca9b0 con 0x7f4d800b19f0 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.436+0000 7f4d86b28640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800cac90 con 0x7f4d8015dd90 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.436+0000 7f4d848c4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d8015d9b0 0x7f4d800cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.436+0000 7f4d848c4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d8015d9b0 0x7f4d800cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55300/0 (socket says 172.21.15.133:55300) 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.436+0000 7f4d848c4640 1 -- 172.21.15.133:0/3918877675 learned_addr learned my addr 172.21.15.133:0/3918877675 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d7ffff640 1 --2- 172.21.15.133:0/3918877675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d8015dd90 0x7f4d800ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d850c5640 1 --2- 172.21.15.133:0/3918877675 >> v2:172.21.15.162:3300/0 conn(0x7f4d800b19f0 0x7f4d800cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d848c4640 1 -- 172.21.15.133:0/3918877675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d8015dd90 msgr2=0x7f4d800ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d848c4640 1 --2- 172.21.15.133:0/3918877675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d8015dd90 0x7f4d800ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d848c4640 1 -- 172.21.15.133:0/3918877675 >> v2:172.21.15.162:3300/0 conn(0x7f4d800b19f0 msgr2=0x7f4d800cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d848c4640 1 --2- 172.21.15.133:0/3918877675 >> v2:172.21.15.162:3300/0 conn(0x7f4d800b19f0 0x7f4d800cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d848c4640 1 -- 172.21.15.133:0/3918877675 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800caf70 con 0x7f4d8015d9b0 2024-04-25T10:03:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d7ffff640 1 --2- 172.21.15.133:0/3918877675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d8015dd90 0x7f4d800ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:05.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d848c4640 1 --2- 172.21.15.133:0/3918877675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d8015d9b0 0x7f4d800cb5e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4d8006b4e0 tx=0x7f4d7001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7018ed75a8b97fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d7f7fe640 1 -- 172.21.15.133:0/3918877675 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d700ab610 con 0x7f4d8015d9b0 2024-04-25T10:03:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d7f7fe640 1 -- 172.21.15.133:0/3918877675 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d700b0070 con 0x7f4d8015d9b0 2024-04-25T10:03:05.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d7f7fe640 1 -- 172.21.15.133:0/3918877675 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d700ab9d0 con 0x7f4d8015d9b0 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d86b28640 1 -- 172.21.15.133:0/3918877675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d8015d9b0 msgr2=0x7f4d800cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d86b28640 1 --2- 172.21.15.133:0/3918877675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d8015d9b0 0x7f4d800cb5e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4d8006b4e0 tx=0x7f4d7001c430 comp rx=0 tx=0).stop 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d86b28640 1 -- 172.21.15.133:0/3918877675 shutdown_connections 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d86b28640 1 --2- 172.21.15.133:0/3918877675 >> v2:172.21.15.162:3300/0 conn(0x7f4d800b19f0 0x7f4d800cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d86b28640 1 --2- 172.21.15.133:0/3918877675 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d8015dd90 0x7f4d800ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d86b28640 1 --2- 172.21.15.133:0/3918877675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d8015d9b0 0x7f4d800cb5e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d86b28640 1 -- 172.21.15.133:0/3918877675 >> 172.21.15.133:0/3918877675 conn(0x7f4d80067000 msgr2=0x7f4d8006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d86b28640 1 -- 172.21.15.133:0/3918877675 shutdown_connections 2024-04-25T10:03:05.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.440+0000 7f4d86b28640 1 -- 172.21.15.133:0/3918877675 wait complete. 2024-04-25T10:03:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d86b28640 1 Processor -- start 2024-04-25T10:03:05.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d86b28640 1 -- start start 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d86b28640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d800b19f0 0x7f4d8014fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d86b28640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d8015d9b0 0x7f4d80146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d86b28640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d8015dd90 0x7f4d8015fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d86b28640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80162290 con 0x7f4d800b19f0 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d86b28640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80162110 con 0x7f4d8015dd90 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d86b28640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80162410 con 0x7f4d8015d9b0 2024-04-25T10:03:05.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d848c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d800b19f0 0x7f4d8014fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d848c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d800b19f0 0x7f4d8014fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55316/0 (socket says 172.21.15.133:55316) 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d848c4640 1 -- 172.21.15.133:0/585764582 learned_addr learned my addr 172.21.15.133:0/585764582 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d7ffff640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d8015d9b0 0x7f4d80146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d850c5640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d8015dd90 0x7f4d8015fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d848c4640 1 -- 172.21.15.133:0/585764582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d8015d9b0 msgr2=0x7f4d80146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d848c4640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d8015d9b0 0x7f4d80146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d848c4640 1 -- 172.21.15.133:0/585764582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d8015dd90 msgr2=0x7f4d8015fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d848c4640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d8015dd90 0x7f4d8015fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d848c4640 1 -- 172.21.15.133:0/585764582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d80153c70 con 0x7f4d800b19f0 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d850c5640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d8015dd90 0x7f4d8015fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d848c4640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d800b19f0 0x7f4d8014fe00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4d7001ab40 tx=0x7f4d7001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d7d7fa640 1 -- 172.21.15.133:0/585764582 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d700b4800 con 0x7f4d800b19f0 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d7d7fa640 1 -- 172.21.15.133:0/585764582 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d700b49a0 con 0x7f4d800b19f0 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d86b28640 1 -- 172.21.15.133:0/585764582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d80154350 con 0x7f4d800b19f0 2024-04-25T10:03:05.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d7d7fa640 1 -- 172.21.15.133:0/585764582 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d700b4d90 con 0x7f4d800b19f0 2024-04-25T10:03:05.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d86b28640 1 -- 172.21.15.133:0/585764582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d80154910 con 0x7f4d800b19f0 2024-04-25T10:03:05.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.444+0000 7f4d86b28640 1 -- 172.21.15.133:0/585764582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d80068ef0 con 0x7f4d800b19f0 2024-04-25T10:03:05.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.448+0000 7f4d7d7fa640 1 -- 172.21.15.133:0/585764582 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d700c7440 con 0x7f4d800b19f0 2024-04-25T10:03:05.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.448+0000 7f4d7d7fa640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d6c066f60 0x7f4d6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.448+0000 7f4d7d7fa640 1 -- 172.21.15.133:0/585764582 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4d7011ff00 con 0x7f4d800b19f0 2024-04-25T10:03:05.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.448+0000 7f4d7ffff640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d6c066f60 0x7f4d6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.448+0000 7f4d7ffff640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d6c066f60 0x7f4d6c069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4d801540d0 tx=0x7f4d74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.452+0000 7f4d7d7fa640 1 -- 172.21.15.133:0/585764582 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d700a8050 con 0x7f4d800b19f0 2024-04-25T10:03:05.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.628+0000 7f4d86b28640 1 -- 172.21.15.133:0/585764582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d8006dc70 con 0x7f4d800b19f0 2024-04-25T10:03:05.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.628+0000 7f4d7d7fa640 1 -- 172.21.15.133:0/585764582 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f4d70016110 con 0x7f4d800b19f0 2024-04-25T10:03:05.636 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:05.636 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2230831968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2230831968},{"type":"v1","addr":"172.21.15.162:6835","nonce":2230831968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1456396097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1456396097},{"type":"v1","addr":"172.21.15.133:6836","nonce":1456396097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:51.276631+0000","modified":"2024-04-25T10:02:14.440426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2462154167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2462154167},{"type":"v1","addr":"172.21.15.133:6837","nonce":2462154167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:05.637 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 9 2024-04-25T10:03:05.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 -- 172.21.15.133:0/585764582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d6c066f60 msgr2=0x7f4d6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d6c066f60 0x7f4d6c069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4d801540d0 tx=0x7f4d74008040 comp rx=0 tx=0).stop 2024-04-25T10:03:05.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 -- 172.21.15.133:0/585764582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d800b19f0 msgr2=0x7f4d8014fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d800b19f0 0x7f4d8014fe00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4d7001ab40 tx=0x7f4d7001acd0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 -- 172.21.15.133:0/585764582 shutdown_connections 2024-04-25T10:03:05.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d8015dd90 0x7f4d8015fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d8015d9b0 0x7f4d80146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d6c066f60 0x7f4d6c069420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 --2- 172.21.15.133:0/585764582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d800b19f0 0x7f4d8014fe00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 -- 172.21.15.133:0/585764582 >> 172.21.15.133:0/585764582 conn(0x7f4d80067000 msgr2=0x7f4d8006e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 -- 172.21.15.133:0/585764582 shutdown_connections 2024-04-25T10:03:05.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.632+0000 7f4d86b28640 1 -- 172.21.15.133:0/585764582 wait complete. 2024-04-25T10:03:05.654 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:05.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.764+0000 7f7fce2b1640 1 Processor -- start 2024-04-25T10:03:05.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.764+0000 7f7fce2b1640 1 -- start start 2024-04-25T10:03:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.764+0000 7f7fce2b1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc8152b30 0x7f7fc80b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.764+0000 7f7fce2b1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc806d350 con 0x7f7fc80cea40 2024-04-25T10:03:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.764+0000 7f7fce2b1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc815bf90 con 0x7f7fc8152750 2024-04-25T10:03:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.764+0000 7f7fce2b1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc806d5d0 con 0x7f7fc8152b30 2024-04-25T10:03:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.764+0000 7f7fcc84e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7fc80cea40 0x7f7fc80b1cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35916/0 (socket says 172.21.15.133:35916) 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.764+0000 7f7fcc84e640 1 -- 172.21.15.133:0/3839575466 learned_addr learned my addr 172.21.15.133:0/3839575466 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.764+0000 7f7fc77fe640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc8152b30 0x7f7fc80b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305866751 0 0) 0x7f7fc0001120 con 0x7f7fc80cea40 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fb4003530 con 0x7f7fc80cea40 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2498762090 0 0) 0x7f7fb4003530 con 0x7f7fc80cea40 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc8152b30 msgr2=0x7f7fc80b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc8152b30 0x7f7fc80b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 >> v1:172.21.15.162:6789/0 conn(0x7f7fc8152750 legacy=0x7f7fc80ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc815c230 con 0x7f7fc80cea40 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7fc0001850 con 0x7f7fc80cea40 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 >> v1:172.21.15.133:6789/0 conn(0x7f7fc80cea40 legacy=0x7f7fc80b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc8152b30 0x7f7fb40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fb4003f70 0x7f7fb4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fb4004d50 0x7f7fb40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc806d5d0 con 0x7f7fb4004d50 2024-04-25T10:03:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb400bab0 con 0x7f7fb4003f70 2024-04-25T10:03:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb400bcb0 con 0x7f7fc8152b30 2024-04-25T10:03:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fcc84e640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fb4004d50 0x7f7fb40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc7fff640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc8152b30 0x7f7fb40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc77fe640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fb4003f70 0x7f7fb4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fcc84e640 1 -- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc8152b30 msgr2=0x7f7fb40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fcc84e640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc8152b30 0x7f7fb40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fcc84e640 1 -- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fb4003f70 msgr2=0x7f7fb4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fcc84e640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fb4003f70 0x7f7fb4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fcc84e640 1 -- 172.21.15.133:0/3839575466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb4003530 con 0x7f7fb4004d50 2024-04-25T10:03:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc77fe640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fb4003f70 0x7f7fb4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:05.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fcc84e640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fb4004d50 0x7f7fb40051d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7fc815f3d0 tx=0x7f7fc001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4145577759ce76f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000c0e0 con 0x7f7fb4004d50 2024-04-25T10:03:05.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc00a9070 con 0x7f7fb4004d50 2024-04-25T10:03:05.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fc6ffd640 1 -- 172.21.15.133:0/3839575466 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00a5dc0 con 0x7f7fb4004d50 2024-04-25T10:03:05.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3839575466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fb4004d50 msgr2=0x7f7fb40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fce2b1640 1 --2- 172.21.15.133:0/3839575466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fb4004d50 0x7f7fb40051d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7fc815f3d0 tx=0x7f7fc001cc20 comp rx=0 tx=0).stop 2024-04-25T10:03:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fcc84e640 1 -- 172.21.15.133:0/3839575466 reap_dead start 2024-04-25T10:03:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3839575466 shutdown_connections 2024-04-25T10:03:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3839575466 >> 172.21.15.133:0/3839575466 conn(0x7f7fc8067000 msgr2=0x7f7fc806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3839575466 shutdown_connections 2024-04-25T10:03:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.768+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3839575466 wait complete. 2024-04-25T10:03:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fce2b1640 1 Processor -- start 2024-04-25T10:03:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fce2b1640 1 -- start start 2024-04-25T10:03:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fce2b1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8143f40 0x7f7fc80c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fce2b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fc8157e00 0x7f7fc8158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fce2b1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80c99c0 0x7f7fc80c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fce2b1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80b17b0 con 0x7f7fc8157e00 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fce2b1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc815c7f0 con 0x7f7fc8143f40 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fce2b1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80b1930 con 0x7f7fc80c99c0 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fc8157e00 0x7f7fc8158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fc8157e00 0x7f7fc8158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55322/0 (socket says 172.21.15.133:55322) 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc77fe640 1 -- 172.21.15.133:0/3891796315 learned_addr learned my addr 172.21.15.133:0/3891796315 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fcc84e640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80c99c0 0x7f7fc80c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc7fff640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8143f40 0x7f7fc80c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc77fe640 1 -- 172.21.15.133:0/3891796315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80c99c0 msgr2=0x7f7fc80c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc77fe640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80c99c0 0x7f7fc80c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc77fe640 1 -- 172.21.15.133:0/3891796315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8143f40 msgr2=0x7f7fc80c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc77fe640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8143f40 0x7f7fc80c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc77fe640 1 -- 172.21.15.133:0/3891796315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc8157040 con 0x7f7fc8157e00 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc77fe640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fc8157e00 0x7f7fc8158bf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00108c0 tx=0x7f7fb0010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc4ff9640 1 -- 172.21.15.133:0/3891796315 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000f040 con 0x7f7fc8157e00 2024-04-25T10:03:05.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc4ff9640 1 -- 172.21.15.133:0/3891796315 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb0004220 con 0x7f7fc8157e00 2024-04-25T10:03:05.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3891796315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc8157320 con 0x7f7fc8157e00 2024-04-25T10:03:05.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fc4ff9640 1 -- 172.21.15.133:0/3891796315 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000e960 con 0x7f7fc8157e00 2024-04-25T10:03:05.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3891796315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc81578e0 con 0x7f7fc8157e00 2024-04-25T10:03:05.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.772+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3891796315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc8068ef0 con 0x7f7fc8157e00 2024-04-25T10:03:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.776+0000 7f7fc4ff9640 1 -- 172.21.15.133:0/3891796315 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fb000eb00 con 0x7f7fc8157e00 2024-04-25T10:03:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.776+0000 7f7fc4ff9640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fa8066f60 0x7f7fa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.776+0000 7f7fc4ff9640 1 -- 172.21.15.133:0/3891796315 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7fb0085770 con 0x7f7fc8157e00 2024-04-25T10:03:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.776+0000 7f7fc7fff640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fa8066f60 0x7f7fa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.776+0000 7f7fc7fff640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fa8066f60 0x7f7fa8069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8001d20 tx=0x7f7fb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.780+0000 7f7fc4ff9640 1 -- 172.21.15.133:0/3891796315 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7fb0018030 con 0x7f7fc8157e00 2024-04-25T10:03:05.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.940+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3891796315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fc80cb4e0 con 0x7f7fc8157e00 2024-04-25T10:03:05.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.940+0000 7f7fc4ff9640 1 -- 172.21.15.133:0/3891796315 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f7fb0056380 con 0x7f7fc8157e00 2024-04-25T10:03:05.949 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:05.949 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:05.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3891796315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fa8066f60 msgr2=0x7f7fa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fa8066f60 0x7f7fa8069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8001d20 tx=0x7f7fb8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:05.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3891796315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fc8157e00 msgr2=0x7f7fc8158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fc8157e00 0x7f7fc8158bf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00108c0 tx=0x7f7fb0010d80 comp rx=0 tx=0).stop 2024-04-25T10:03:05.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3891796315 shutdown_connections 2024-04-25T10:03:05.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fc80c99c0 0x7f7fc80c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fa8066f60 0x7f7fa8069420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fc8157e00 0x7f7fc8158bf0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 --2- 172.21.15.133:0/3891796315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fc8143f40 0x7f7fc80c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3891796315 >> 172.21.15.133:0/3891796315 conn(0x7f7fc8067000 msgr2=0x7f7fc80c58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3891796315 shutdown_connections 2024-04-25T10:03:05.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:05.944+0000 7f7fce2b1640 1 -- 172.21.15.133:0/3891796315 wait complete. 2024-04-25T10:03:05.968 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:03:05.969 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:06.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba3c5e640 1 Processor -- start 2024-04-25T10:03:06.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba3c5e640 1 -- start start 2024-04-25T10:03:06.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba3c5e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b9c0ce6e0 0x7f8b9c0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba3c5e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8b9c15de60 0x7f8b9c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba3c5e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b9c15eb50 con 0x7f8b9c15da80 2024-04-25T10:03:06.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba3c5e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9c15f110 con 0x7f8b9c0ce6e0 2024-04-25T10:03:06.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba3c5e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9c161c00 con 0x7f8b9c15de60 2024-04-25T10:03:06.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba19fa640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8b9c15da80 0x7f8b9c162090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35930/0 (socket says 172.21.15.133:35930) 2024-04-25T10:03:06.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba19fa640 1 -- 172.21.15.133:0/4065703768 learned_addr learned my addr 172.21.15.133:0/4065703768 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:06.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba11f9640 1 --2- 172.21.15.133:0/4065703768 >> v2:172.21.15.133:3300/0 conn(0x7f8b9c0ce6e0 0x7f8b9c0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba21fb640 1 --2- 172.21.15.133:0/4065703768 >> v2:172.21.15.162:3300/0 conn(0x7f8b9c15de60 0x7f8b9c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba11f9640 1 -- 172.21.15.133:0/4065703768 >> v2:172.21.15.162:3300/0 conn(0x7f8b9c15de60 msgr2=0x7f8b9c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba11f9640 1 --2- 172.21.15.133:0/4065703768 >> v2:172.21.15.162:3300/0 conn(0x7f8b9c15de60 0x7f8b9c15e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba11f9640 1 -- 172.21.15.133:0/4065703768 >> v1:172.21.15.133:6789/0 conn(0x7f8b9c15da80 legacy=0x7f8b9c162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba11f9640 1 -- 172.21.15.133:0/4065703768 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c15ee30 con 0x7f8b9c0ce6e0 2024-04-25T10:03:06.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba21fb640 1 --2- 172.21.15.133:0/4065703768 >> v2:172.21.15.162:3300/0 conn(0x7f8b9c15de60 0x7f8b9c15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.076+0000 7f8ba11f9640 1 --2- 172.21.15.133:0/4065703768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b9c0ce6e0 0x7f8b9c0ceb90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0d1290 tx=0x7f8b8c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4245349f11efdd89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba09f8640 1 -- 172.21.15.133:0/4065703768 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0b1070 con 0x7f8b9c0ce6e0 2024-04-25T10:03:06.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba09f8640 1 -- 172.21.15.133:0/4065703768 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c0ac5f0 con 0x7f8b9c0ce6e0 2024-04-25T10:03:06.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba09f8640 1 -- 172.21.15.133:0/4065703768 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0ac9c0 con 0x7f8b9c0ce6e0 2024-04-25T10:03:06.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/4065703768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b9c0ce6e0 msgr2=0x7f8b9c0ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 --2- 172.21.15.133:0/4065703768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b9c0ce6e0 0x7f8b9c0ceb90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0d1290 tx=0x7f8b8c01c480 comp rx=0 tx=0).stop 2024-04-25T10:03:06.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/4065703768 shutdown_connections 2024-04-25T10:03:06.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 --2- 172.21.15.133:0/4065703768 >> v2:172.21.15.162:3300/0 conn(0x7f8b9c15de60 0x7f8b9c15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 --2- 172.21.15.133:0/4065703768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b9c0ce6e0 0x7f8b9c0ceb90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/4065703768 >> 172.21.15.133:0/4065703768 conn(0x7f8b9c06c0d0 msgr2=0x7f8b9c06c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/4065703768 shutdown_connections 2024-04-25T10:03:06.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/4065703768 wait complete. 2024-04-25T10:03:06.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 Processor -- start 2024-04-25T10:03:06.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 -- start start 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b9c0ce6e0 0x7f8b9c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b9c15da80 0x7f8b9c14b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b9c15de60 0x7f8b9c1602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0d25b0 con 0x7f8b9c15da80 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0b9660 con 0x7f8b9c15de60 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0d2730 con 0x7f8b9c0ce6e0 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba11f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b9c15da80 0x7f8b9c14b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba11f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b9c15da80 0x7f8b9c14b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55346/0 (socket says 172.21.15.133:55346) 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba11f9640 1 -- 172.21.15.133:0/3177235279 learned_addr learned my addr 172.21.15.133:0/3177235279 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba21fb640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b9c15de60 0x7f8b9c1602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba19fa640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b9c0ce6e0 0x7f8b9c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba11f9640 1 -- 172.21.15.133:0/3177235279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b9c0ce6e0 msgr2=0x7f8b9c0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba11f9640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b9c0ce6e0 0x7f8b9c0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba11f9640 1 -- 172.21.15.133:0/3177235279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b9c15de60 msgr2=0x7f8b9c1602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba11f9640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b9c15de60 0x7f8b9c1602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba11f9640 1 -- 172.21.15.133:0/3177235279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c155ad0 con 0x7f8b9c15da80 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba21fb640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b9c15de60 0x7f8b9c1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba19fa640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b9c0ce6e0 0x7f8b9c0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba11f9640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b9c15da80 0x7f8b9c14b360 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c01ab30 tx=0x7f8b8c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8b927fc640 1 -- 172.21.15.133:0/3177235279 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0b1040 con 0x7f8b9c15da80 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8b927fc640 1 -- 172.21.15.133:0/3177235279 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c0bb070 con 0x7f8b9c15da80 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/3177235279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9c155db0 con 0x7f8b9c15da80 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8b927fc640 1 -- 172.21.15.133:0/3177235279 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0b5650 con 0x7f8b9c15da80 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.080+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/3177235279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9c14cc40 con 0x7f8b9c15da80 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.084+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/3177235279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9c067c70 con 0x7f8b9c15da80 2024-04-25T10:03:06.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.084+0000 7f8b927fc640 1 -- 172.21.15.133:0/3177235279 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b8c0b57f0 con 0x7f8b9c15da80 2024-04-25T10:03:06.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.084+0000 7f8b927fc640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b80066f60 0x7f8b80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.084+0000 7f8b927fc640 1 -- 172.21.15.133:0/3177235279 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8b8c11f0c0 con 0x7f8b9c15da80 2024-04-25T10:03:06.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.084+0000 7f8ba19fa640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b80066f60 0x7f8b80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.084+0000 7f8ba19fa640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b80066f60 0x7f8b80069420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c1688e0 tx=0x7f8b98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.088+0000 7f8b927fc640 1 -- 172.21.15.133:0/3177235279 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b8c0a9070 con 0x7f8b9c15da80 2024-04-25T10:03:06.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.268+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/3177235279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b9c0d1670 con 0x7f8b9c15da80 2024-04-25T10:03:06.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.268+0000 7f8b927fc640 1 -- 172.21.15.133:0/3177235279 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f8b8c016070 con 0x7f8b9c15da80 2024-04-25T10:03:06.276 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:06.277 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2230831968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2230831968},{"type":"v1","addr":"172.21.15.162:6835","nonce":2230831968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1456396097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1456396097},{"type":"v1","addr":"172.21.15.133:6836","nonce":1456396097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:51.276631+0000","modified":"2024-04-25T10:02:14.440426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2462154167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2462154167},{"type":"v1","addr":"172.21.15.133:6837","nonce":2462154167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:06.277 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 9 2024-04-25T10:03:06.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/3177235279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b80066f60 msgr2=0x7f8b80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b80066f60 0x7f8b80069420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c1688e0 tx=0x7f8b98009040 comp rx=0 tx=0).stop 2024-04-25T10:03:06.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/3177235279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b9c15da80 msgr2=0x7f8b9c14b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b9c15da80 0x7f8b9c14b360 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c01ab30 tx=0x7f8b8c019220 comp rx=0 tx=0).stop 2024-04-25T10:03:06.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/3177235279 shutdown_connections 2024-04-25T10:03:06.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8b9c15de60 0x7f8b9c1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8b80066f60 0x7f8b80069420 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b9c15da80 0x7f8b9c14b360 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 --2- 172.21.15.133:0/3177235279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8b9c0ce6e0 0x7f8b9c0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/3177235279 >> 172.21.15.133:0/3177235279 conn(0x7f8b9c06c0d0 msgr2=0x7f8b9c156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/3177235279 shutdown_connections 2024-04-25T10:03:06.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.272+0000 7f8ba3c5e640 1 -- 172.21.15.133:0/3177235279 wait complete. 2024-04-25T10:03:06.294 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:06.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.392+0000 7fef6fc22640 1 Processor -- start 2024-04-25T10:03:06.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.392+0000 7fef6fc22640 1 -- start start 2024-04-25T10:03:06.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6fc22640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fef681584f0 0x7fef680b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6fc22640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef680cbe60 con 0x7fef68158110 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6fc22640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef680b19b0 con 0x7fef6806eb70 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6fc22640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6815b360 con 0x7fef681584f0 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6d9be640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fef68158110 0x7fef6806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50284/0 (socket says 172.21.15.133:50284) 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6d9be640 1 -- 172.21.15.133:0/3141917288 learned_addr learned my addr 172.21.15.133:0/3141917288 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6e1bf640 1 --2- 172.21.15.133:0/3141917288 >> v2:172.21.15.162:3300/0 conn(0x7fef681584f0 0x7fef680b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455259788 0 0) 0x7fef680cbe60 con 0x7fef68158110 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef50003530 con 0x7fef68158110 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 353676813 0 0) 0x7fef50003530 con 0x7fef68158110 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 >> v2:172.21.15.162:3300/0 conn(0x7fef681584f0 msgr2=0x7fef680b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 --2- 172.21.15.133:0/3141917288 >> v2:172.21.15.162:3300/0 conn(0x7fef681584f0 0x7fef680b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 >> v1:172.21.15.162:6789/0 conn(0x7fef6806eb70 legacy=0x7fef680cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6815b640 con 0x7fef68158110 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fef60001680 con 0x7fef68158110 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 >> v1:172.21.15.133:6789/0 conn(0x7fef68158110 legacy=0x7fef6806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef500039c0 0x7fef50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef50004810 0x7fef50004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef500051d0 0x7fef5000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6815b360 con 0x7fef50004810 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5000c070 con 0x7fef500051d0 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5000c270 con 0x7fef500039c0 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6d1bd640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef50004810 0x7fef50004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6e1bf640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef500051d0 0x7fef5000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6d1bd640 1 -- 172.21.15.133:0/3141917288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef500039c0 msgr2=0x7fef50003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6d9be640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef500039c0 0x7fef50003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6d1bd640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef500039c0 0x7fef50003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6d1bd640 1 -- 172.21.15.133:0/3141917288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef500051d0 msgr2=0x7fef5000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6d1bd640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef500051d0 0x7fef5000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6d1bd640 1 -- 172.21.15.133:0/3141917288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef50003530 con 0x7fef50004810 2024-04-25T10:03:06.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6e1bf640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef500051d0 0x7fef5000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6e1bf640 1 -- 172.21.15.133:0/3141917288 reap_dead start 2024-04-25T10:03:06.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.396+0000 7fef6d9be640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef500039c0 0x7fef50003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6d1bd640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef50004810 0x7fef50004c90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fef5c001750 tx=0x7fef5c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1943926ae1b0290 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0013a0 con 0x7fef50004810 2024-04-25T10:03:06.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c0052f0 con 0x7fef50004810 2024-04-25T10:03:06.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6c9bc640 1 -- 172.21.15.133:0/3141917288 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60001d50 con 0x7fef50004810 2024-04-25T10:03:06.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 -- 172.21.15.133:0/3141917288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef50004810 msgr2=0x7fef50004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef50004810 0x7fef50004c90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fef5c001750 tx=0x7fef5c01c3c0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 -- 172.21.15.133:0/3141917288 shutdown_connections 2024-04-25T10:03:06.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 --2- 172.21.15.133:0/3141917288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef50004810 0x7fef50004c90 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 -- 172.21.15.133:0/3141917288 >> 172.21.15.133:0/3141917288 conn(0x7fef68067000 msgr2=0x7fef6806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 -- 172.21.15.133:0/3141917288 shutdown_connections 2024-04-25T10:03:06.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 -- 172.21.15.133:0/3141917288 wait complete. 2024-04-25T10:03:06.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 Processor -- start 2024-04-25T10:03:06.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 -- start start 2024-04-25T10:03:06.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef50004810 0x7fef68146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef680cc210 0x7fef680cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef6815e320 0x7fef6815e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef680cd670 con 0x7fef6815e320 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6815bc00 con 0x7fef680cc210 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6fc22640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef680cd7f0 con 0x7fef50004810 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6e1bf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef6815e320 0x7fef6815e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6e1bf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef6815e320 0x7fef6815e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57370/0 (socket says 172.21.15.133:57370) 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6e1bf640 1 -- 172.21.15.133:0/1330795294 learned_addr learned my addr 172.21.15.133:0/1330795294 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6d9be640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef50004810 0x7fef68146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6e1bf640 1 -- 172.21.15.133:0/1330795294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef50004810 msgr2=0x7fef68146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6e1bf640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef50004810 0x7fef68146690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6e1bf640 1 -- 172.21.15.133:0/1330795294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef680cc210 msgr2=0x7fef680cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6d1bd640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef680cc210 0x7fef680cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6e1bf640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef680cc210 0x7fef680cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6e1bf640 1 -- 172.21.15.133:0/1330795294 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6815bfb0 con 0x7fef6815e320 2024-04-25T10:03:06.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef6e1bf640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef6815e320 0x7fef6815e7a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fef68158900 tx=0x7fef5800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef667fc640 1 -- 172.21.15.133:0/1330795294 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58016420 con 0x7fef6815e320 2024-04-25T10:03:06.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef667fc640 1 -- 172.21.15.133:0/1330795294 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5801b070 con 0x7fef6815e320 2024-04-25T10:03:06.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.400+0000 7fef667fc640 1 -- 172.21.15.133:0/1330795294 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef580167c0 con 0x7fef6815e320 2024-04-25T10:03:06.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.404+0000 7fef6fc22640 1 -- 172.21.15.133:0/1330795294 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6815c290 con 0x7fef6815e320 2024-04-25T10:03:06.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.404+0000 7fef6fc22640 1 -- 172.21.15.133:0/1330795294 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6815c850 con 0x7fef6815e320 2024-04-25T10:03:06.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.404+0000 7fef6fc22640 1 -- 172.21.15.133:0/1330795294 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6806bc20 con 0x7fef6815e320 2024-04-25T10:03:06.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.404+0000 7fef667fc640 1 -- 172.21.15.133:0/1330795294 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef58016960 con 0x7fef6815e320 2024-04-25T10:03:06.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.404+0000 7fef667fc640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef54066f90 0x7fef54069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.404+0000 7fef667fc640 1 -- 172.21.15.133:0/1330795294 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fef580862b0 con 0x7fef6815e320 2024-04-25T10:03:06.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.404+0000 7fef6d9be640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef54066f90 0x7fef54069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.404+0000 7fef6d9be640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef54066f90 0x7fef54069450 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef60004880 tx=0x7fef60000f20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.408+0000 7fef667fc640 1 -- 172.21.15.133:0/1330795294 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef58025020 con 0x7fef6815e320 2024-04-25T10:03:06.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef6fc22640 1 -- 172.21.15.133:0/1330795294 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef68068ef0 con 0x7fef6815e320 2024-04-25T10:03:06.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef667fc640 1 -- 172.21.15.133:0/1330795294 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7fef58056ec0 con 0x7fef6815e320 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2230831968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2230831968},{"type":"v1","addr":"172.21.15.162:6835","nonce":2230831968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/537130868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":537130868},{"type":"v1","addr":"172.21.15.162:6833","nonce":537130868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4293,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1456396097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1456396097},{"type":"v1","addr":"172.21.15.133:6836","nonce":1456396097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:51.276631+0000","modified":"2024-04-25T10:02:14.440426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2462154167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2462154167},{"type":"v1","addr":"172.21.15.133:6837","nonce":2462154167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 9 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef6fc22640 1 -- 172.21.15.133:0/1330795294 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef54066f90 msgr2=0x7fef54069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef6fc22640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef54066f90 0x7fef54069450 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef60004880 tx=0x7fef60000f20 comp rx=0 tx=0).stop 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef6fc22640 1 -- 172.21.15.133:0/1330795294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef6815e320 msgr2=0x7fef6815e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef6fc22640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef6815e320 0x7fef6815e7a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fef68158900 tx=0x7fef5800f520 comp rx=0 tx=0).stop 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef6fc22640 1 -- 172.21.15.133:0/1330795294 shutdown_connections 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef6fc22640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef54066f90 0x7fef54069450 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef6fc22640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef6815e320 0x7fef6815e7a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef6fc22640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef680cc210 0x7fef680cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef6fc22640 1 --2- 172.21.15.133:0/1330795294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef50004810 0x7fef68146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.600+0000 7fef6fc22640 1 -- 172.21.15.133:0/1330795294 >> 172.21.15.133:0/1330795294 conn(0x7fef68067000 msgr2=0x7fef680ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.604+0000 7fef6fc22640 1 -- 172.21.15.133:0/1330795294 shutdown_connections 2024-04-25T10:03:06.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.604+0000 7fef6fc22640 1 -- 172.21.15.133:0/1330795294 wait complete. 2024-04-25T10:03:06.624 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:06.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.724+0000 7f510fa26640 1 Processor -- start 2024-04-25T10:03:06.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.724+0000 7f510fa26640 1 -- start start 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51080d0770 0x7f51080d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51080b87e0 0x7f51080b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51080ceb20 con 0x7f510815db00 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51080d1110 con 0x7f51080d0770 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080b30a0 con 0x7f51080b87e0 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510cfc1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51080d0770 0x7f51080d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510cfc1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51080d0770 0x7f51080d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57372/0 (socket says 172.21.15.133:57372) 2024-04-25T10:03:06.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510cfc1640 1 -- 172.21.15.133:0/1789624685 learned_addr learned my addr 172.21.15.133:0/1789624685 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:06.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510cfc1640 1 -- 172.21.15.133:0/1789624685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51080b87e0 msgr2=0x7f51080b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510cfc1640 1 --2- 172.21.15.133:0/1789624685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51080b87e0 0x7f51080b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510cfc1640 1 -- 172.21.15.133:0/1789624685 >> v1:172.21.15.133:6789/0 conn(0x7f510815db00 legacy=0x7f51080b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510cfc1640 1 -- 172.21.15.133:0/1789624685 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080b7740 con 0x7f51080d0770 2024-04-25T10:03:06.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510cfc1640 1 --2- 172.21.15.133:0/1789624685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080d0770 0x7f51080d0bd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009870 tx=0x7f50f801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae111d7f1321441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f50fffff640 1 -- 172.21.15.133:0/1789624685 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80a95f0 con 0x7f51080d0770 2024-04-25T10:03:06.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f50fffff640 1 -- 172.21.15.133:0/1789624685 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f80a9790 con 0x7f51080d0770 2024-04-25T10:03:06.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f50fffff640 1 -- 172.21.15.133:0/1789624685 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80a9b60 con 0x7f51080d0770 2024-04-25T10:03:06.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 -- 172.21.15.133:0/1789624685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080d0770 msgr2=0x7f51080d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 --2- 172.21.15.133:0/1789624685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080d0770 0x7f51080d0bd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009870 tx=0x7f50f801c530 comp rx=0 tx=0).stop 2024-04-25T10:03:06.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 -- 172.21.15.133:0/1789624685 shutdown_connections 2024-04-25T10:03:06.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 --2- 172.21.15.133:0/1789624685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51080b87e0 0x7f51080b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 --2- 172.21.15.133:0/1789624685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080d0770 0x7f51080d0bd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 -- 172.21.15.133:0/1789624685 >> 172.21.15.133:0/1789624685 conn(0x7f51080727c0 msgr2=0x7f5108072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 -- 172.21.15.133:0/1789624685 shutdown_connections 2024-04-25T10:03:06.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.728+0000 7f510fa26640 1 -- 172.21.15.133:0/1789624685 wait complete. 2024-04-25T10:03:06.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510fa26640 1 Processor -- start 2024-04-25T10:03:06.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510fa26640 1 -- start start 2024-04-25T10:03:06.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510fa26640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080b87e0 0x7f5108153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510fa26640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51080d0770 0x7f51081525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510fa26640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510815db00 0x7f5108159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510fa26640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080b7430 con 0x7f51080b87e0 2024-04-25T10:03:06.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510fa26640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080b82c0 con 0x7f510815db00 2024-04-25T10:03:06.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510fa26640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080d18a0 con 0x7f51080d0770 2024-04-25T10:03:06.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510d7c2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080b87e0 0x7f5108153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510d7c2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080b87e0 0x7f5108153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57374/0 (socket says 172.21.15.133:57374) 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510d7c2640 1 -- 172.21.15.133:0/3656526633 learned_addr learned my addr 172.21.15.133:0/3656526633 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510d7c2640 1 -- 172.21.15.133:0/3656526633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51080d0770 msgr2=0x7f51081525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510d7c2640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51080d0770 0x7f51081525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510d7c2640 1 -- 172.21.15.133:0/3656526633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510815db00 msgr2=0x7f5108159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510d7c2640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510815db00 0x7f5108159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510d7c2640 1 -- 172.21.15.133:0/3656526633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5108159a30 con 0x7f51080b87e0 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510d7c2640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080b87e0 0x7f5108153d40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f510400c370 tx=0x7f510400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f50fdffb640 1 -- 172.21.15.133:0/3656526633 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51040052d0 con 0x7f51080b87e0 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f50fdffb640 1 -- 172.21.15.133:0/3656526633 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5104005470 con 0x7f51080b87e0 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510fa26640 1 -- 172.21.15.133:0/3656526633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5108152f80 con 0x7f51080b87e0 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f50fdffb640 1 -- 172.21.15.133:0/3656526633 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104007260 con 0x7f51080b87e0 2024-04-25T10:03:06.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510fa26640 1 -- 172.21.15.133:0/3656526633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5108153540 con 0x7f51080b87e0 2024-04-25T10:03:06.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f510fa26640 1 -- 172.21.15.133:0/3656526633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5108068ef0 con 0x7f51080b87e0 2024-04-25T10:03:06.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.732+0000 7f50fdffb640 1 -- 172.21.15.133:0/3656526633 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5104007400 con 0x7f51080b87e0 2024-04-25T10:03:06.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.736+0000 7f50fdffb640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f50ec066f60 0x7f50ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.736+0000 7f50fdffb640 1 -- 172.21.15.133:0/3656526633 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5104084890 con 0x7f51080b87e0 2024-04-25T10:03:06.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.736+0000 7f510cfc1640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f50ec066f60 0x7f50ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.736+0000 7f510cfc1640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f50ec066f60 0x7f50ec069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f50f80ae000 tx=0x7f50f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.740+0000 7f50fdffb640 1 -- 172.21.15.133:0/3656526633 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51040554a0 con 0x7f51080b87e0 2024-04-25T10:03:06.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.888+0000 7f510fa26640 1 -- 172.21.15.133:0/3656526633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51080b2750 con 0x7f51080b87e0 2024-04-25T10:03:06.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f50fdffb640 1 -- 172.21.15.133:0/3656526633 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18927 (secure 0 0 0) 0x7f5104054da0 con 0x7f51080b87e0 2024-04-25T10:03:06.897 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:06.897 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:02:32.781962+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:49.812095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:50.262173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:01:54.010420+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:06.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 -- 172.21.15.133:0/3656526633 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f50ec066f60 msgr2=0x7f50ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f50ec066f60 0x7f50ec069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f50f80ae000 tx=0x7f50f8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:06.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 -- 172.21.15.133:0/3656526633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080b87e0 msgr2=0x7f5108153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080b87e0 0x7f5108153d40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f510400c370 tx=0x7f510400c830 comp rx=0 tx=0).stop 2024-04-25T10:03:06.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 -- 172.21.15.133:0/3656526633 shutdown_connections 2024-04-25T10:03:06.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f510815db00 0x7f5108159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51080d0770 0x7f51081525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f50ec066f60 0x7f50ec069420 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 --2- 172.21.15.133:0/3656526633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080b87e0 0x7f5108153d40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 -- 172.21.15.133:0/3656526633 >> 172.21.15.133:0/3656526633 conn(0x7f51080727c0 msgr2=0x7f51080d2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 -- 172.21.15.133:0/3656526633 shutdown_connections 2024-04-25T10:03:06.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:06.892+0000 7f510fa26640 1 -- 172.21.15.133:0/3656526633 wait complete. 2024-04-25T10:03:06.915 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:03:07.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.016+0000 7f671f5aa640 1 Processor -- start 2024-04-25T10:03:07.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.016+0000 7f671f5aa640 1 -- start start 2024-04-25T10:03:07.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671f5aa640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f67180cd820 0x7f67180cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671f5aa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671815ced0 0x7f671815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671f5aa640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67180ce2e0 con 0x7f6718145170 2024-04-25T10:03:07.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671f5aa640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671806db00 con 0x7f67180cd820 2024-04-25T10:03:07.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671f5aa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6718145550 con 0x7f671815ced0 2024-04-25T10:03:07.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671d346640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6718145170 0x7f671806df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50302/0 (socket says 172.21.15.133:50302) 2024-04-25T10:03:07.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671d346640 1 -- 172.21.15.133:0/2058553577 learned_addr learned my addr 172.21.15.133:0/2058553577 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:07.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671cb45640 1 --2- 172.21.15.133:0/2058553577 >> v2:172.21.15.162:3300/0 conn(0x7f67180cd820 0x7f67180cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671db47640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671815ced0 0x7f671815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005220064 0 0) 0x7f67180ce2e0 con 0x7f6718145170 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6704003530 con 0x7f6718145170 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2473964246 0 0) 0x7f6704003530 con 0x7f6718145170 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671815ced0 msgr2=0x7f671815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671815ced0 0x7f671815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 >> v2:172.21.15.162:3300/0 conn(0x7f67180cd820 msgr2=0x7f67180cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 --2- 172.21.15.133:0/2058553577 >> v2:172.21.15.162:3300/0 conn(0x7f67180cd820 0x7f67180cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671806d4b0 con 0x7f6718145170 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6714001680 con 0x7f6718145170 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 >> v1:172.21.15.133:6789/0 conn(0x7f6718145170 legacy=0x7f671806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67040039c0 0x7f6704003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671815ced0 0x7f6704004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6704004d50 0x7f67040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671806db00 con 0x7f67040039c0 2024-04-25T10:03:07.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6718145550 con 0x7f6704004d50 2024-04-25T10:03:07.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670400b9d0 con 0x7f671815ced0 2024-04-25T10:03:07.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671d346640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67040039c0 0x7f6704003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671cb45640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671815ced0 0x7f6704004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671db47640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6704004d50 0x7f67040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671d346640 1 -- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671815ced0 msgr2=0x7f6704004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671d346640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671815ced0 0x7f6704004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671d346640 1 -- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6704004d50 msgr2=0x7f67040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671d346640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6704004d50 0x7f67040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671d346640 1 -- 172.21.15.133:0/2058553577 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6704003530 con 0x7f67040039c0 2024-04-25T10:03:07.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671db47640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6704004d50 0x7f67040051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:07.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671cb45640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671815ced0 0x7f6704004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f671d346640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67040039c0 0x7f6704003de0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f671400c190 tx=0x7f671401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dda384a4be2c54f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714007c90 con 0x7f67040039c0 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67140a9020 con 0x7f67040039c0 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.020+0000 7f670ffff640 1 -- 172.21.15.133:0/2058553577 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714005ae0 con 0x7f67040039c0 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- 172.21.15.133:0/2058553577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67040039c0 msgr2=0x7f6704003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 --2- 172.21.15.133:0/2058553577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67040039c0 0x7f6704003de0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f671400c190 tx=0x7f671401cc00 comp rx=0 tx=0).stop 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671db47640 1 -- 172.21.15.133:0/2058553577 reap_dead start 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- 172.21.15.133:0/2058553577 shutdown_connections 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- 172.21.15.133:0/2058553577 >> 172.21.15.133:0/2058553577 conn(0x7f6718066f70 msgr2=0x7f671806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- 172.21.15.133:0/2058553577 shutdown_connections 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- 172.21.15.133:0/2058553577 wait complete. 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 Processor -- start 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- start start 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67040039c0 0x7f6718137ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671813c5d0 0x7f671813e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671813ca50 0x7f671813e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6718166480 con 0x7f671813ca50 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671815cb70 con 0x7f67040039c0 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6718166600 con 0x7f671813c5d0 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671db47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671813ca50 0x7f671813e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671db47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671813ca50 0x7f671813e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57378/0 (socket says 172.21.15.133:57378) 2024-04-25T10:03:07.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671db47640 1 -- 172.21.15.133:0/486079996 learned_addr learned my addr 172.21.15.133:0/486079996 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671cb45640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671813c5d0 0x7f671813e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671d346640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67040039c0 0x7f6718137ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671db47640 1 -- 172.21.15.133:0/486079996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671813c5d0 msgr2=0x7f671813e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671db47640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671813c5d0 0x7f671813e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671db47640 1 -- 172.21.15.133:0/486079996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67040039c0 msgr2=0x7f6718137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671db47640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67040039c0 0x7f6718137ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671db47640 1 -- 172.21.15.133:0/486079996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671813cfb0 con 0x7f671813ca50 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671d346640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67040039c0 0x7f6718137ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671db47640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671813ca50 0x7f671813e9a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f67180b23f0 tx=0x7f671000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f670dffb640 1 -- 172.21.15.133:0/486079996 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671001c070 con 0x7f671813ca50 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f670dffb640 1 -- 172.21.15.133:0/486079996 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6710017400 con 0x7f671813ca50 2024-04-25T10:03:07.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- 172.21.15.133:0/486079996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671813d290 con 0x7f671813ca50 2024-04-25T10:03:07.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f670dffb640 1 -- 172.21.15.133:0/486079996 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67100177d0 con 0x7f671813ca50 2024-04-25T10:03:07.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- 172.21.15.133:0/486079996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671813d850 con 0x7f671813ca50 2024-04-25T10:03:07.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.024+0000 7f671f5aa640 1 -- 172.21.15.133:0/486079996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6718068e80 con 0x7f671813ca50 2024-04-25T10:03:07.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.028+0000 7f670dffb640 1 -- 172.21.15.133:0/486079996 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f671001f070 con 0x7f671813ca50 2024-04-25T10:03:07.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.028+0000 7f670dffb640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66fc066f90 0x7f66fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.028+0000 7f670dffb640 1 -- 172.21.15.133:0/486079996 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f67100854b0 con 0x7f671813ca50 2024-04-25T10:03:07.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.028+0000 7f671d346640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66fc066f90 0x7f66fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.028+0000 7f671d346640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66fc066f90 0x7f66fc069450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f67140aa000 tx=0x7f67140019f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.032+0000 7f670dffb640 1 -- 172.21.15.133:0/486079996 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6710012070 con 0x7f671813ca50 2024-04-25T10:03:07.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.208+0000 7f671f5aa640 1 -- 172.21.15.133:0/486079996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f671813e240 con 0x7f671813ca50 2024-04-25T10:03:07.216 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:03:07.208+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:03:07.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.212+0000 7f670dffb640 1 -- 172.21.15.133:0/486079996 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6710056150 con 0x7f671813ca50 2024-04-25T10:03:07.219 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:03:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 -- 172.21.15.133:0/486079996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66fc066f90 msgr2=0x7f66fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66fc066f90 0x7f66fc069450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f67140aa000 tx=0x7f67140019f0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 -- 172.21.15.133:0/486079996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671813ca50 msgr2=0x7f671813e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671813ca50 0x7f671813e9a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f67180b23f0 tx=0x7f671000f4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 -- 172.21.15.133:0/486079996 shutdown_connections 2024-04-25T10:03:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66fc066f90 0x7f66fc069450 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671813ca50 0x7f671813e9a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671813c5d0 0x7f671813e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 --2- 172.21.15.133:0/486079996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67040039c0 0x7f6718137ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 -- 172.21.15.133:0/486079996 >> 172.21.15.133:0/486079996 conn(0x7f6718066f70 msgr2=0x7f67180adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 -- 172.21.15.133:0/486079996 shutdown_connections 2024-04-25T10:03:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.216+0000 7f671f5aa640 1 -- 172.21.15.133:0/486079996 wait complete. 2024-04-25T10:03:07.226 INFO:tasks.ceph.mds.c.smithi133.stderr: -4780> 2024-04-25T10:01:57.409+0000 7f6220dec640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:07.242 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:03:07.261 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:03:07.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa87d640640 1 Processor -- start 2024-04-25T10:03:07.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa87d640640 1 -- start start 2024-04-25T10:03:07.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa87d640640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8780ce600 0x7fa8780d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa87d640640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa8780d0a70 0x7fa87806c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa87d640640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8780b1220 con 0x7fa87806ccf0 2024-04-25T10:03:07.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa87d640640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8780b2ff0 con 0x7fa8780d0a70 2024-04-25T10:03:07.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa87d640640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780b1500 con 0x7fa8780ce600 2024-04-25T10:03:07.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa8767fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa8780d0a70 0x7fa87806c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa8767fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa8780d0a70 0x7fa87806c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57394/0 (socket says 172.21.15.133:57394) 2024-04-25T10:03:07.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa8767fc640 1 -- 172.21.15.133:0/3671494284 learned_addr learned my addr 172.21.15.133:0/3671494284 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:07.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa876ffd640 1 --2- 172.21.15.133:0/3671494284 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8780ce600 0x7fa8780d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa8767fc640 1 -- 172.21.15.133:0/3671494284 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8780ce600 msgr2=0x7fa8780d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa8767fc640 1 --2- 172.21.15.133:0/3671494284 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8780ce600 0x7fa8780d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa8767fc640 1 -- 172.21.15.133:0/3671494284 >> v1:172.21.15.162:6789/0 conn(0x7fa87806ccf0 legacy=0x7fa87815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.360+0000 7fa8767fc640 1 -- 172.21.15.133:0/3671494284 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8780b17e0 con 0x7fa8780d0a70 2024-04-25T10:03:07.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa8767fc640 1 --2- 172.21.15.133:0/3671494284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8780d0a70 0x7fa87806c7b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009870 tx=0x7fa86c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e83e736b187945 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa875ffb640 1 -- 172.21.15.133:0/3671494284 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0b0070 con 0x7fa8780d0a70 2024-04-25T10:03:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa875ffb640 1 -- 172.21.15.133:0/3671494284 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c0ab600 con 0x7fa8780d0a70 2024-04-25T10:03:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa875ffb640 1 -- 172.21.15.133:0/3671494284 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0ab9d0 con 0x7fa8780d0a70 2024-04-25T10:03:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 -- 172.21.15.133:0/3671494284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8780d0a70 msgr2=0x7fa87806c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 --2- 172.21.15.133:0/3671494284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8780d0a70 0x7fa87806c7b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009870 tx=0x7fa86c01c520 comp rx=0 tx=0).stop 2024-04-25T10:03:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 -- 172.21.15.133:0/3671494284 shutdown_connections 2024-04-25T10:03:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 --2- 172.21.15.133:0/3671494284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8780d0a70 0x7fa87806c7b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 --2- 172.21.15.133:0/3671494284 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8780ce600 0x7fa8780d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 -- 172.21.15.133:0/3671494284 >> 172.21.15.133:0/3671494284 conn(0x7fa87809a990 msgr2=0x7fa8780d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 -- 172.21.15.133:0/3671494284 shutdown_connections 2024-04-25T10:03:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 -- 172.21.15.133:0/3671494284 wait complete. 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 Processor -- start 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 -- start start 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87806ccf0 0x7fa878168ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8780ce600 0x7fa8780c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8780d0a70 0x7fa878160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780ae280 con 0x7fa87806ccf0 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780d21a0 con 0x7fa8780d0a70 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa87d640640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa878156ce0 con 0x7fa8780ce600 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa876ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87806ccf0 0x7fa878168ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa876ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87806ccf0 0x7fa878168ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57396/0 (socket says 172.21.15.133:57396) 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa876ffd640 1 -- 172.21.15.133:0/2431438682 learned_addr learned my addr 172.21.15.133:0/2431438682 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa876ffd640 1 -- 172.21.15.133:0/2431438682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8780ce600 msgr2=0x7fa8780c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa8767fc640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8780ce600 0x7fa8780c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa876ffd640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8780ce600 0x7fa8780c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa8777fe640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8780d0a70 0x7fa878160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa876ffd640 1 -- 172.21.15.133:0/2431438682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8780d0a70 msgr2=0x7fa878160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa876ffd640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8780d0a70 0x7fa878160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa876ffd640 1 -- 172.21.15.133:0/2431438682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa878156a00 con 0x7fa87806ccf0 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa876ffd640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87806ccf0 0x7fa878168ed0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa86800d800 tx=0x7fa86800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.364+0000 7fa85f7fe640 1 -- 172.21.15.133:0/2431438682 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86800b6c0 con 0x7fa87806ccf0 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.368+0000 7fa85f7fe640 1 -- 172.21.15.133:0/2431438682 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa868019070 con 0x7fa87806ccf0 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.368+0000 7fa87d640640 1 -- 172.21.15.133:0/2431438682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8780c7670 con 0x7fa87806ccf0 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.368+0000 7fa85f7fe640 1 -- 172.21.15.133:0/2431438682 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86800ba60 con 0x7fa87806ccf0 2024-04-25T10:03:07.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.368+0000 7fa87d640640 1 -- 172.21.15.133:0/2431438682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8780c7c30 con 0x7fa87806ccf0 2024-04-25T10:03:07.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.368+0000 7fa87d640640 1 -- 172.21.15.133:0/2431438682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa878067c70 con 0x7fa87806ccf0 2024-04-25T10:03:07.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.368+0000 7fa85f7fe640 1 -- 172.21.15.133:0/2431438682 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa86800bc00 con 0x7fa87806ccf0 2024-04-25T10:03:07.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.368+0000 7fa85f7fe640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa864066f60 0x7fa864069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.368+0000 7fa8767fc640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa864066f60 0x7fa864069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.368+0000 7fa85f7fe640 1 -- 172.21.15.133:0/2431438682 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6821+0+0 (secure 0 0 0) 0x7fa868085860 con 0x7fa87806ccf0 2024-04-25T10:03:07.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.368+0000 7fa8767fc640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa864066f60 0x7fa864069420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0ad000 tx=0x7fa86c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.372+0000 7fa85f7fe640 1 -- 172.21.15.133:0/2431438682 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8680564a0 con 0x7fa87806ccf0 2024-04-25T10:03:07.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:07.552+0000 7fa87d640640 1 -- 172.21.15.133:0/2431438682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa87806c590 con 0x7fa87806ccf0 2024-04-25T10:03:08.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa85f7fe640 1 -- 172.21.15.133:0/2431438682 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa868055da0 con 0x7fa87806ccf0 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 -- 172.21.15.133:0/2431438682 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa864066f60 msgr2=0x7fa864069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa864066f60 0x7fa864069420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0ad000 tx=0x7fa86c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 -- 172.21.15.133:0/2431438682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87806ccf0 msgr2=0x7fa878168ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87806ccf0 0x7fa878168ed0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa86800d800 tx=0x7fa86800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 -- 172.21.15.133:0/2431438682 shutdown_connections 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8780d0a70 0x7fa878160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8780ce600 0x7fa8780c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa864066f60 0x7fa864069420 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 --2- 172.21.15.133:0/2431438682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa87806ccf0 0x7fa878168ed0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 -- 172.21.15.133:0/2431438682 >> 172.21.15.133:0/2431438682 conn(0x7fa87809a990 msgr2=0x7fa87815a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 -- 172.21.15.133:0/2431438682 shutdown_connections 2024-04-25T10:03:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.216+0000 7fa87d640640 1 -- 172.21.15.133:0/2431438682 wait complete. 2024-04-25T10:03:08.245 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:03:08.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb2a2739640 1 Processor -- start 2024-04-25T10:03:08.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb2a2739640 1 -- start start 2024-04-25T10:03:08.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb2a2739640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c0cd3e0 0x7fb29c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb2a2739640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29c0d0610 con 0x7fb29c160f10 2024-04-25T10:03:08.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb2a2739640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29c1612f0 con 0x7fb29c06b1d0 2024-04-25T10:03:08.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb2a2739640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0d08b0 con 0x7fb29c0cd3e0 2024-04-25T10:03:08.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb29c160f10 0x7fb29c0b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50304/0 (socket says 172.21.15.133:50304) 2024-04-25T10:03:08.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29bfff640 1 -- 172.21.15.133:0/264670247 learned_addr learned my addr 172.21.15.133:0/264670247 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:08.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29b7fe640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c0cd3e0 0x7fb29c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143719844 0 0) 0x7fb29c0d0610 con 0x7fb29c160f10 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb284003530 con 0x7fb29c160f10 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2530477328 0 0) 0x7fb284003530 con 0x7fb29c160f10 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c0cd3e0 msgr2=0x7fb29c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c0cd3e0 0x7fb29c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 >> v1:172.21.15.162:6789/0 conn(0x7fb29c06b1d0 legacy=0x7fb29c161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c0d0b90 con 0x7fb29c160f10 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb28c001680 con 0x7fb29c160f10 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 >> v1:172.21.15.133:6789/0 conn(0x7fb29c160f10 legacy=0x7fb29c0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c0cd3e0 0x7fb2840039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb284003f70 0x7fb284004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb284004d50 0x7fb2840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0d08b0 con 0x7fb284003f70 2024-04-25T10:03:08.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28400bc10 con 0x7fb284004d50 2024-04-25T10:03:08.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28400be10 con 0x7fb29c0cd3e0 2024-04-25T10:03:08.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29b7fe640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb284003f70 0x7fb284004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29bfff640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c0cd3e0 0x7fb2840039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb2a0cd6640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb284004d50 0x7fb2840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29b7fe640 1 -- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c0cd3e0 msgr2=0x7fb2840039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29b7fe640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c0cd3e0 0x7fb2840039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29b7fe640 1 -- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb284004d50 msgr2=0x7fb2840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29b7fe640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb284004d50 0x7fb2840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb29b7fe640 1 -- 172.21.15.133:0/264670247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb284003530 con 0x7fb284003f70 2024-04-25T10:03:08.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.416+0000 7fb2a0cd6640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb284004d50 0x7fb2840051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:08.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.420+0000 7fb29b7fe640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb284003f70 0x7fb284004810 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb290009ae0 tx=0x7fb29001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0e32629fe3fa85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.420+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2900096b0 con 0x7fb284003f70 2024-04-25T10:03:08.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.420+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2900a6450 con 0x7fb284003f70 2024-04-25T10:03:08.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.420+0000 7fb29affd640 1 -- 172.21.15.133:0/264670247 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c162530 con 0x7fb284003f70 2024-04-25T10:03:08.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.420+0000 7fb2a2739640 1 -- 172.21.15.133:0/264670247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb284003f70 msgr2=0x7fb284004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.420+0000 7fb2a2739640 1 --2- 172.21.15.133:0/264670247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb284003f70 0x7fb284004810 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb290009ae0 tx=0x7fb29001ca70 comp rx=0 tx=0).stop 2024-04-25T10:03:08.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.420+0000 7fb2a0cd6640 1 -- 172.21.15.133:0/264670247 reap_dead start 2024-04-25T10:03:08.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.420+0000 7fb2a2739640 1 -- 172.21.15.133:0/264670247 shutdown_connections 2024-04-25T10:03:08.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.420+0000 7fb2a2739640 1 -- 172.21.15.133:0/264670247 >> 172.21.15.133:0/264670247 conn(0x7fb29c06e480 msgr2=0x7fb29c06e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.420+0000 7fb2a2739640 1 -- 172.21.15.133:0/264670247 shutdown_connections 2024-04-25T10:03:08.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.420+0000 7fb2a2739640 1 -- 172.21.15.133:0/264670247 wait complete. 2024-04-25T10:03:08.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a2739640 1 Processor -- start 2024-04-25T10:03:08.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a2739640 1 -- start start 2024-04-25T10:03:08.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a2739640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb284003f70 0x7fb29c1637e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a2739640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c168990 0x7fb29c168e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a2739640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb29c0b1360 0x7fb29c0b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a2739640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c169730 con 0x7fb29c0b1360 2024-04-25T10:03:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a2739640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0afdf0 con 0x7fb284003f70 2024-04-25T10:03:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a2739640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c1698b0 con 0x7fb29c168990 2024-04-25T10:03:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a0cd6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb29c0b1360 0x7fb29c0b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a0cd6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb29c0b1360 0x7fb29c0b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57416/0 (socket says 172.21.15.133:57416) 2024-04-25T10:03:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a0cd6640 1 -- 172.21.15.133:0/1407783523 learned_addr learned my addr 172.21.15.133:0/1407783523 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb29bfff640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb284003f70 0x7fb29c1637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb29b7fe640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c168990 0x7fb29c168e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a0cd6640 1 -- 172.21.15.133:0/1407783523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c168990 msgr2=0x7fb29c168e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a0cd6640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c168990 0x7fb29c168e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a0cd6640 1 -- 172.21.15.133:0/1407783523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb284003f70 msgr2=0x7fb29c1637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a0cd6640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb284003f70 0x7fb29c1637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a0cd6640 1 -- 172.21.15.133:0/1407783523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c0b20d0 con 0x7fb29c0b1360 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a0cd6640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb29c0b1360 0x7fb29c0b17e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0725f0 tx=0x7fb29400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb298ff9640 1 -- 172.21.15.133:0/1407783523 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294006d60 con 0x7fb29c0b1360 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb298ff9640 1 -- 172.21.15.133:0/1407783523 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb294003980 con 0x7fb29c0b1360 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb298ff9640 1 -- 172.21.15.133:0/1407783523 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294003d70 con 0x7fb29c0b1360 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a2739640 1 -- 172.21.15.133:0/1407783523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c0b23b0 con 0x7fb29c0b1360 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a2739640 1 -- 172.21.15.133:0/1407783523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c0b2970 con 0x7fb29c0b1360 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb2a2739640 1 -- 172.21.15.133:0/1407783523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29c069000 con 0x7fb29c0b1360 2024-04-25T10:03:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.424+0000 7fb298ff9640 1 -- 172.21.15.133:0/1407783523 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb29401b020 con 0x7fb29c0b1360 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.428+0000 7fb298ff9640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb288066f60 0x7fb288069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.428+0000 7fb298ff9640 1 -- 172.21.15.133:0/1407783523 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6821+0+0 (secure 0 0 0) 0x7fb2940850e0 con 0x7fb29c0b1360 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.428+0000 7fb29bfff640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb288066f60 0x7fb288069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.428+0000 7fb29bfff640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb288066f60 0x7fb288069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb28c001ec0 tx=0x7fb28c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.432+0000 7fb298ff9640 1 -- 172.21.15.133:0/1407783523 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb29401ce40 con 0x7fb29c0b1360 2024-04-25T10:03:08.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:08.580+0000 7fb2a2739640 1 -- 172.21.15.133:0/1407783523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb29c0732d0 con 0x7fb29c0b1360 2024-04-25T10:03:09.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.224+0000 7fb298ff9640 1 -- 172.21.15.133:0/1407783523 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb294055c90 con 0x7fb29c0b1360 2024-04-25T10:03:09.233 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:03:09.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.228+0000 7fb2a2739640 1 -- 172.21.15.133:0/1407783523 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb288066f60 msgr2=0x7fb288069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.228+0000 7fb2a2739640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb288066f60 0x7fb288069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb28c001ec0 tx=0x7fb28c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:09.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.228+0000 7fb2a2739640 1 -- 172.21.15.133:0/1407783523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb29c0b1360 msgr2=0x7fb29c0b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.228+0000 7fb2a2739640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb29c0b1360 0x7fb29c0b17e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0725f0 tx=0x7fb29400dc00 comp rx=0 tx=0).stop 2024-04-25T10:03:09.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.228+0000 7fb2a2739640 1 -- 172.21.15.133:0/1407783523 shutdown_connections 2024-04-25T10:03:09.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.228+0000 7fb2a2739640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb288066f60 0x7fb288069420 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.228+0000 7fb2a2739640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb29c0b1360 0x7fb29c0b17e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.228+0000 7fb2a2739640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb29c168990 0x7fb29c168e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.228+0000 7fb2a2739640 1 --2- 172.21.15.133:0/1407783523 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb284003f70 0x7fb29c1637e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.228+0000 7fb2a2739640 1 -- 172.21.15.133:0/1407783523 >> 172.21.15.133:0/1407783523 conn(0x7fb29c06e480 msgr2=0x7fb29c0cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.232+0000 7fb2a2739640 1 -- 172.21.15.133:0/1407783523 shutdown_connections 2024-04-25T10:03:09.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.232+0000 7fb2a2739640 1 -- 172.21.15.133:0/1407783523 wait complete. 2024-04-25T10:03:09.259 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:03:09.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f4816225640 1 Processor -- start 2024-04-25T10:03:09.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f4816225640 1 -- start start 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f4816225640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f48100b4940 0x7f48100b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f4816225640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f481015c420 0x7f48100cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f4816225640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48100b5260 con 0x7f4810071cb0 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f4816225640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4810072a90 con 0x7f481015c420 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f4816225640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48100a85d0 con 0x7f48100b4940 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f480ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f481015c420 0x7f48100cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f480ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f481015c420 0x7f48100cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57418/0 (socket says 172.21.15.133:57418) 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f480ffff640 1 -- 172.21.15.133:0/496902865 learned_addr learned my addr 172.21.15.133:0/496902865 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f480f7fe640 1 --2- 172.21.15.133:0/496902865 >> v2:172.21.15.162:3300/0 conn(0x7f48100b4940 0x7f48100b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f480ffff640 1 -- 172.21.15.133:0/496902865 >> v2:172.21.15.162:3300/0 conn(0x7f48100b4940 msgr2=0x7f48100b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f480ffff640 1 --2- 172.21.15.133:0/496902865 >> v2:172.21.15.162:3300/0 conn(0x7f48100b4940 0x7f48100b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f480ffff640 1 -- 172.21.15.133:0/496902865 >> v1:172.21.15.162:6789/0 conn(0x7f4810071cb0 legacy=0x7f481015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.384+0000 7f480ffff640 1 -- 172.21.15.133:0/496902865 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481015c9b0 con 0x7f481015c420 2024-04-25T10:03:09.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f480ffff640 1 --2- 172.21.15.133:0/496902865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f481015c420 0x7f48100cc010 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4810070a40 tx=0x7f480001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151ae39adc40c20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f480e7fc640 1 -- 172.21.15.133:0/496902865 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000aa610 con 0x7f481015c420 2024-04-25T10:03:09.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f480e7fc640 1 -- 172.21.15.133:0/496902865 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48000af070 con 0x7f481015c420 2024-04-25T10:03:09.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f480e7fc640 1 -- 172.21.15.133:0/496902865 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000aa9d0 con 0x7f481015c420 2024-04-25T10:03:09.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f4816225640 1 -- 172.21.15.133:0/496902865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f481015c420 msgr2=0x7f48100cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f4816225640 1 --2- 172.21.15.133:0/496902865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f481015c420 0x7f48100cc010 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4810070a40 tx=0x7f480001c4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f4816225640 1 -- 172.21.15.133:0/496902865 shutdown_connections 2024-04-25T10:03:09.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f4816225640 1 --2- 172.21.15.133:0/496902865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f481015c420 0x7f48100cc010 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f4816225640 1 --2- 172.21.15.133:0/496902865 >> v2:172.21.15.162:3300/0 conn(0x7f48100b4940 0x7f48100b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f4816225640 1 -- 172.21.15.133:0/496902865 >> 172.21.15.133:0/496902865 conn(0x7f481006d370 msgr2=0x7f481006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f4816225640 1 -- 172.21.15.133:0/496902865 shutdown_connections 2024-04-25T10:03:09.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f4816225640 1 -- 172.21.15.133:0/496902865 wait complete. 2024-04-25T10:03:09.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.388+0000 7f4816225640 1 Processor -- start 2024-04-25T10:03:09.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f4816225640 1 -- start start 2024-04-25T10:03:09.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f4816225640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4810071cb0 0x7f481013af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f4816225640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48100b4940 0x7f481013c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f4816225640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f481015c420 0x7f48100ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f4816225640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100cc960 con 0x7f4810071cb0 2024-04-25T10:03:09.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f4816225640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4810071790 con 0x7f481015c420 2024-04-25T10:03:09.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f4816225640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48100ccae0 con 0x7f48100b4940 2024-04-25T10:03:09.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4810071cb0 0x7f481013af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4810071cb0 0x7f481013af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57424/0 (socket says 172.21.15.133:57424) 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480f7fe640 1 -- 172.21.15.133:0/4261479193 learned_addr learned my addr 172.21.15.133:0/4261479193 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480ffff640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f481015c420 0x7f48100ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480f7fe640 1 -- 172.21.15.133:0/4261479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48100b4940 msgr2=0x7f481013c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480effd640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48100b4940 0x7f481013c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480f7fe640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48100b4940 0x7f481013c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480f7fe640 1 -- 172.21.15.133:0/4261479193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f481015c420 msgr2=0x7f48100ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480f7fe640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f481015c420 0x7f48100ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480f7fe640 1 -- 172.21.15.133:0/4261479193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481013c080 con 0x7f4810071cb0 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480ffff640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f481015c420 0x7f48100ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480effd640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48100b4940 0x7f481013c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480f7fe640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4810071cb0 0x7f481013af30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f480800d830 tx=0x7f480800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f47f7fff640 1 -- 172.21.15.133:0/4261479193 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480800b6c0 con 0x7f4810071cb0 2024-04-25T10:03:09.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f47f7fff640 1 -- 172.21.15.133:0/4261479193 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4808019070 con 0x7f4810071cb0 2024-04-25T10:03:09.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f4816225640 1 -- 172.21.15.133:0/4261479193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481013c360 con 0x7f4810071cb0 2024-04-25T10:03:09.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f47f7fff640 1 -- 172.21.15.133:0/4261479193 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480800ba60 con 0x7f4810071cb0 2024-04-25T10:03:09.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f4816225640 1 -- 172.21.15.133:0/4261479193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48100ae570 con 0x7f4810071cb0 2024-04-25T10:03:09.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f4816225640 1 -- 172.21.15.133:0/4261479193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4810069000 con 0x7f4810071cb0 2024-04-25T10:03:09.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f47f7fff640 1 -- 172.21.15.133:0/4261479193 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f480801e460 con 0x7f4810071cb0 2024-04-25T10:03:09.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f47f7fff640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47fc066f60 0x7f47fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f47f7fff640 1 -- 172.21.15.133:0/4261479193 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6383+0+0 (secure 0 0 0) 0x7f4808085960 con 0x7f4810071cb0 2024-04-25T10:03:09.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.392+0000 7f480effd640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47fc066f60 0x7f47fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.396+0000 7f480effd640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47fc066f60 0x7f47fc069420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4804007fd0 tx=0x7f4804009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.400+0000 7f47f7fff640 1 -- 172.21.15.133:0/4261479193 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48080566d0 con 0x7f4810071cb0 2024-04-25T10:03:09.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:09.560+0000 7f4816225640 1 -- 172.21.15.133:0/4261479193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f481013e560 con 0x7f4810071cb0 2024-04-25T10:03:10.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.228+0000 7f47f7fff640 1 -- 172.21.15.133:0/4261479193 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4808055fd0 con 0x7f4810071cb0 2024-04-25T10:03:10.236 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:03:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 -- 172.21.15.133:0/4261479193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47fc066f60 msgr2=0x7f47fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47fc066f60 0x7f47fc069420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4804007fd0 tx=0x7f4804009040 comp rx=0 tx=0).stop 2024-04-25T10:03:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 -- 172.21.15.133:0/4261479193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4810071cb0 msgr2=0x7f481013af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4810071cb0 0x7f481013af30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f480800d830 tx=0x7f480800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 -- 172.21.15.133:0/4261479193 shutdown_connections 2024-04-25T10:03:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f481015c420 0x7f48100ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48100b4940 0x7f481013c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47fc066f60 0x7f47fc069420 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 --2- 172.21.15.133:0/4261479193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4810071cb0 0x7f481013af30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 -- 172.21.15.133:0/4261479193 >> 172.21.15.133:0/4261479193 conn(0x7f481006d370 msgr2=0x7f481006d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 -- 172.21.15.133:0/4261479193 shutdown_connections 2024-04-25T10:03:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.232+0000 7f4816225640 1 -- 172.21.15.133:0/4261479193 wait complete. 2024-04-25T10:03:10.266 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T10:03:10.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb80af6c640 1 Processor -- start 2024-04-25T10:03:10.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb80af6c640 1 -- start start 2024-04-25T10:03:10.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb80af6c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb804157970 0x7fb804140270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb80af6c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb80415da20 0x7fb80415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb80af6c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8040cc070 con 0x7fb8040719c0 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb80af6c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8040cc910 con 0x7fb804157970 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb80af6c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040cc350 con 0x7fb80415da20 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb808d08640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb804157970 0x7fb804140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb808d08640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb804157970 0x7fb804140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57432/0 (socket says 172.21.15.133:57432) 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb808d08640 1 -- 172.21.15.133:0/761686298 learned_addr learned my addr 172.21.15.133:0/761686298 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb808d08640 1 -- 172.21.15.133:0/761686298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb80415da20 msgr2=0x7fb80415deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb808d08640 1 --2- 172.21.15.133:0/761686298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb80415da20 0x7fb80415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb808d08640 1 -- 172.21.15.133:0/761686298 >> v1:172.21.15.162:6789/0 conn(0x7fb8040719c0 legacy=0x7fb8040cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.384+0000 7fb808d08640 1 -- 172.21.15.133:0/761686298 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040cc630 con 0x7fb804157970 2024-04-25T10:03:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb808d08640 1 --2- 172.21.15.133:0/761686298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb804157970 0x7fb804140270 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009e50 tx=0x7fb7f401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd733cfa3b869f6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb8037fe640 1 -- 172.21.15.133:0/761686298 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40a55f0 con 0x7fb804157970 2024-04-25T10:03:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb8037fe640 1 -- 172.21.15.133:0/761686298 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f40a5790 con 0x7fb804157970 2024-04-25T10:03:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb8037fe640 1 -- 172.21.15.133:0/761686298 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40a5b60 con 0x7fb804157970 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb80af6c640 1 -- 172.21.15.133:0/761686298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb804157970 msgr2=0x7fb804140270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb80af6c640 1 --2- 172.21.15.133:0/761686298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb804157970 0x7fb804140270 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009e50 tx=0x7fb7f401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb80af6c640 1 -- 172.21.15.133:0/761686298 shutdown_connections 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb80af6c640 1 --2- 172.21.15.133:0/761686298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb80415da20 0x7fb80415deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb80af6c640 1 --2- 172.21.15.133:0/761686298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb804157970 0x7fb804140270 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb80af6c640 1 -- 172.21.15.133:0/761686298 >> 172.21.15.133:0/761686298 conn(0x7fb80406d370 msgr2=0x7fb80406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb80af6c640 1 -- 172.21.15.133:0/761686298 shutdown_connections 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb80af6c640 1 -- 172.21.15.133:0/761686298 wait complete. 2024-04-25T10:03:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb80af6c640 1 Processor -- start 2024-04-25T10:03:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.388+0000 7fb80af6c640 1 -- start start 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb80af6c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8040719c0 0x7fb804146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb80af6c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb804157970 0x7fb80414f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb80af6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb80415da20 0x7fb80415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb80af6c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80415a800 con 0x7fb80415da20 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb80af6c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80415a680 con 0x7fb804157970 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb80af6c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80415a980 con 0x7fb8040719c0 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb809509640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb80415da20 0x7fb80415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb809509640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb80415da20 0x7fb80415bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57446/0 (socket says 172.21.15.133:57446) 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb809509640 1 -- 172.21.15.133:0/740916499 learned_addr learned my addr 172.21.15.133:0/740916499 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb808d08640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8040719c0 0x7fb804146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb803fff640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb804157970 0x7fb80414f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb809509640 1 -- 172.21.15.133:0/740916499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8040719c0 msgr2=0x7fb804146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb809509640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8040719c0 0x7fb804146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb809509640 1 -- 172.21.15.133:0/740916499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb804157970 msgr2=0x7fb80414f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb809509640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb804157970 0x7fb80414f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb809509640 1 -- 172.21.15.133:0/740916499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804154150 con 0x7fb80415da20 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb809509640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb80415da20 0x7fb80415bdf0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00b3c0 tx=0x7fb7fc00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb8017fa640 1 -- 172.21.15.133:0/740916499 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc011830 con 0x7fb80415da20 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb8017fa640 1 -- 172.21.15.133:0/740916499 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc016070 con 0x7fb80415da20 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb80af6c640 1 -- 172.21.15.133:0/740916499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8041466b0 con 0x7fb80415da20 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb8017fa640 1 -- 172.21.15.133:0/740916499 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc011c10 con 0x7fb80415da20 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb80af6c640 1 -- 172.21.15.133:0/740916499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8041543c0 con 0x7fb80415da20 2024-04-25T10:03:10.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb80af6c640 1 -- 172.21.15.133:0/740916499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb804069000 con 0x7fb80415da20 2024-04-25T10:03:10.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb8017fa640 1 -- 172.21.15.133:0/740916499 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7fc01b460 con 0x7fb80415da20 2024-04-25T10:03:10.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb8017fa640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7f0066f60 0x7fb7f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.392+0000 7fb8017fa640 1 -- 172.21.15.133:0/740916499 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb7fc085870 con 0x7fb80415da20 2024-04-25T10:03:10.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.396+0000 7fb808d08640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7f0066f60 0x7fb7f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.396+0000 7fb808d08640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7f0066f60 0x7fb7f0069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40ae000 tx=0x7fb7f40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.396+0000 7fb8017fa640 1 -- 172.21.15.133:0/740916499 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7fc056810 con 0x7fb80415da20 2024-04-25T10:03:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:10.560+0000 7fb80af6c640 1 -- 172.21.15.133:0/740916499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb8040b56c0 con 0x7fb80415da20 2024-04-25T10:03:11.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.252+0000 7fb8017fa640 1 -- 172.21.15.133:0/740916499 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7fb7fc056110 con 0x7fb80415da20 2024-04-25T10:03:11.262 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data_ec' removed 2024-04-25T10:03:11.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 -- 172.21.15.133:0/740916499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7f0066f60 msgr2=0x7fb7f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7f0066f60 0x7fb7f0069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40ae000 tx=0x7fb7f40a7070 comp rx=0 tx=0).stop 2024-04-25T10:03:11.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 -- 172.21.15.133:0/740916499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb80415da20 msgr2=0x7fb80415bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb80415da20 0x7fb80415bdf0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00b3c0 tx=0x7fb7fc00b880 comp rx=0 tx=0).stop 2024-04-25T10:03:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 -- 172.21.15.133:0/740916499 shutdown_connections 2024-04-25T10:03:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7f0066f60 0x7fb7f0069420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb80415da20 0x7fb80415bdf0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb804157970 0x7fb80414f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 --2- 172.21.15.133:0/740916499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8040719c0 0x7fb804146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 -- 172.21.15.133:0/740916499 >> 172.21.15.133:0/740916499 conn(0x7fb80406d370 msgr2=0x7fb804157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 -- 172.21.15.133:0/740916499 shutdown_connections 2024-04-25T10:03:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.260+0000 7fb80af6c640 1 -- 172.21.15.133:0/740916499 wait complete. 2024-04-25T10:03:11.297 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:03:11.297 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:03:11.297 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:03:11.298 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:11.298 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:03:11.298 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:03:11.298 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:03:11.298 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:11.298 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:03:11.298 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:03:11.299 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:03:11.299 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:11.299 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:03:11.299 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:03:11.299 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:03:11.299 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:11.308 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:03:11.308 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:03:11.310 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:03:11.310 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:03:11.315 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:03:11.316 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:03:11.317 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:03:11.317 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:03:11.354 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:03:11.354 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:11.355 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:03:11.355 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:11.362 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:03:11.362 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:11.363 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:03:11.363 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:11.363 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:03:11.420 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:03:11.420 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:03:11.421 INFO:tasks.ceph.mds.c.smithi133.stderr:2024-04-25T10:03:11.416+0000 7fe93cc12640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6835/0: (98) Address already in use 2024-04-25T10:03:11.432 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:03:11.436 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:03:11.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48f201640 1 Processor -- start 2024-04-25T10:03:11.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48f201640 1 -- start start 2024-04-25T10:03:11.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48f201640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd48815d9a0 0x7fd4880d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48f201640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4880ce600 0x7fd4880cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48f201640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4880d24c0 con 0x7fd4880d1920 2024-04-25T10:03:11.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48f201640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4880cefa0 con 0x7fd48815d9a0 2024-04-25T10:03:11.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48f201640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48815ebd0 con 0x7fd4880ce600 2024-04-25T10:03:11.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48cf9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd48815d9a0 0x7fd4880d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48cf9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd48815d9a0 0x7fd4880d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57462/0 (socket says 172.21.15.133:57462) 2024-04-25T10:03:11.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48cf9d640 1 -- 172.21.15.133:0/181076202 learned_addr learned my addr 172.21.15.133:0/181076202 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:11.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd487fff640 1 --2- 172.21.15.133:0/181076202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4880ce600 0x7fd4880cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48cf9d640 1 -- 172.21.15.133:0/181076202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4880ce600 msgr2=0x7fd4880cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48cf9d640 1 --2- 172.21.15.133:0/181076202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4880ce600 0x7fd4880cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48cf9d640 1 -- 172.21.15.133:0/181076202 >> v1:172.21.15.162:6789/0 conn(0x7fd4880d1920 legacy=0x7fd4880d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.472+0000 7fd48cf9d640 1 -- 172.21.15.133:0/181076202 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48815eeb0 con 0x7fd48815d9a0 2024-04-25T10:03:11.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd48cf9d640 1 --2- 172.21.15.133:0/181076202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48815d9a0 0x7fd4880d0b30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd478009870 tx=0x7fd47801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd0e253e234c54b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd4877fe640 1 -- 172.21.15.133:0/181076202 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780aa600 con 0x7fd48815d9a0 2024-04-25T10:03:11.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd4877fe640 1 -- 172.21.15.133:0/181076202 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4780af070 con 0x7fd48815d9a0 2024-04-25T10:03:11.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd4877fe640 1 -- 172.21.15.133:0/181076202 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780aa9c0 con 0x7fd48815d9a0 2024-04-25T10:03:11.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd48f201640 1 -- 172.21.15.133:0/181076202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48815d9a0 msgr2=0x7fd4880d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd48f201640 1 --2- 172.21.15.133:0/181076202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48815d9a0 0x7fd4880d0b30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd478009870 tx=0x7fd47801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd48f201640 1 -- 172.21.15.133:0/181076202 shutdown_connections 2024-04-25T10:03:11.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd48f201640 1 --2- 172.21.15.133:0/181076202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4880ce600 0x7fd4880cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd48f201640 1 --2- 172.21.15.133:0/181076202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48815d9a0 0x7fd4880d0b30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd48f201640 1 -- 172.21.15.133:0/181076202 >> 172.21.15.133:0/181076202 conn(0x7fd48806bf80 msgr2=0x7fd48806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd48f201640 1 -- 172.21.15.133:0/181076202 shutdown_connections 2024-04-25T10:03:11.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd48f201640 1 -- 172.21.15.133:0/181076202 wait complete. 2024-04-25T10:03:11.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd48f201640 1 Processor -- start 2024-04-25T10:03:11.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.476+0000 7fd48f201640 1 -- start start 2024-04-25T10:03:11.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48f201640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4880ce600 0x7fd4880b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48f201640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4880d1920 0x7fd48815c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48f201640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd48815d9a0 0x7fd4880b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48f201640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880b96a0 con 0x7fd4880ce600 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48f201640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880cffd0 con 0x7fd48815d9a0 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48f201640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880b9820 con 0x7fd4880d1920 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48cf9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4880ce600 0x7fd4880b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48cf9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4880ce600 0x7fd4880b7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57466/0 (socket says 172.21.15.133:57466) 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48cf9d640 1 -- 172.21.15.133:0/2429220055 learned_addr learned my addr 172.21.15.133:0/2429220055 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48d79e640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd48815d9a0 0x7fd4880b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd487fff640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4880d1920 0x7fd48815c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48cf9d640 1 -- 172.21.15.133:0/2429220055 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4880d1920 msgr2=0x7fd48815c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48cf9d640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4880d1920 0x7fd48815c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48cf9d640 1 -- 172.21.15.133:0/2429220055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd48815d9a0 msgr2=0x7fd4880b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48cf9d640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd48815d9a0 0x7fd4880b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48cf9d640 1 -- 172.21.15.133:0/2429220055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4881677a0 con 0x7fd4880ce600 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48cf9d640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4880ce600 0x7fd4880b7e00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd4780ac040 tx=0x7fd47801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd4857fa640 1 -- 172.21.15.133:0/2429220055 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780bd5f0 con 0x7fd4880ce600 2024-04-25T10:03:11.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd4857fa640 1 -- 172.21.15.133:0/2429220055 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4780bd790 con 0x7fd4880ce600 2024-04-25T10:03:11.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd4857fa640 1 -- 172.21.15.133:0/2429220055 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780bdb80 con 0x7fd4880ce600 2024-04-25T10:03:11.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48f201640 1 -- 172.21.15.133:0/2429220055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd488167a80 con 0x7fd4880ce600 2024-04-25T10:03:11.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48f201640 1 -- 172.21.15.133:0/2429220055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4880ca900 con 0x7fd4880ce600 2024-04-25T10:03:11.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd48f201640 1 -- 172.21.15.133:0/2429220055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd488068e80 con 0x7fd4880ce600 2024-04-25T10:03:11.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd4857fa640 1 -- 172.21.15.133:0/2429220055 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4780a9070 con 0x7fd4880ce600 2024-04-25T10:03:11.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd4857fa640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd474066f60 0x7fd474069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd4857fa640 1 -- 172.21.15.133:0/2429220055 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd47811f4e0 con 0x7fd4880ce600 2024-04-25T10:03:11.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd487fff640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd474066f60 0x7fd474069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.480+0000 7fd487fff640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd474066f60 0x7fd474069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd4880b2620 tx=0x7fd47c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.484+0000 7fd4857fa640 1 -- 172.21.15.133:0/2429220055 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd478016150 con 0x7fd4880ce600 2024-04-25T10:03:11.519 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T10:03:11.519 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T10:03:11.519 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:03:11.520 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:03:11.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:11.664+0000 7fd48f201640 1 -- 172.21.15.133:0/2429220055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd488073750 con 0x7fd4880ce600 2024-04-25T10:03:12.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.304+0000 7fd4857fa640 1 -- 172.21.15.133:0/2429220055 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd4780f00a0 con 0x7fd4880ce600 2024-04-25T10:03:12.310 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:03:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 -- 172.21.15.133:0/2429220055 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd474066f60 msgr2=0x7fd474069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd474066f60 0x7fd474069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd4880b2620 tx=0x7fd47c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 -- 172.21.15.133:0/2429220055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4880ce600 msgr2=0x7fd4880b7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4880ce600 0x7fd4880b7e00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd4780ac040 tx=0x7fd47801ace0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 -- 172.21.15.133:0/2429220055 shutdown_connections 2024-04-25T10:03:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd48815d9a0 0x7fd4880b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4880d1920 0x7fd48815c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd474066f60 0x7fd474069420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 --2- 172.21.15.133:0/2429220055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4880ce600 0x7fd4880b7e00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 -- 172.21.15.133:0/2429220055 >> 172.21.15.133:0/2429220055 conn(0x7fd48806bf80 msgr2=0x7fd4880b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 -- 172.21.15.133:0/2429220055 shutdown_connections 2024-04-25T10:03:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.308+0000 7fd48f201640 1 -- 172.21.15.133:0/2429220055 wait complete. 2024-04-25T10:03:12.337 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:03:12.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d2e6a640 1 Processor -- start 2024-04-25T10:03:12.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d2e6a640 1 -- start start 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d2e6a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58cc0b1f70 0x7f58cc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d2e6a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f58cc0ce330 0x7f58cc0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d2e6a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58cc0c61d0 con 0x7f58cc15c9a0 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d2e6a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58cc0cece0 con 0x7f58cc0ce330 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d2e6a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc0c64b0 con 0x7f58cc0b1f70 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d1407640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f58cc0ce330 0x7f58cc0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d1407640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f58cc0ce330 0x7f58cc0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57482/0 (socket says 172.21.15.133:57482) 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d1407640 1 -- 172.21.15.133:0/3393144793 learned_addr learned my addr 172.21.15.133:0/3393144793 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58cbfff640 1 --2- 172.21.15.133:0/3393144793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58cc0b1f70 0x7f58cc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d1407640 1 -- 172.21.15.133:0/3393144793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58cc0b1f70 msgr2=0x7f58cc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d1407640 1 --2- 172.21.15.133:0/3393144793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58cc0b1f70 0x7f58cc06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d1407640 1 -- 172.21.15.133:0/3393144793 >> v1:172.21.15.162:6789/0 conn(0x7f58cc15c9a0 legacy=0x7f58cc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.448+0000 7f58d1407640 1 -- 172.21.15.133:0/3393144793 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc06d4f0 con 0x7f58cc0ce330 2024-04-25T10:03:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d1407640 1 --2- 172.21.15.133:0/3393144793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58cc0ce330 0x7f58cc0ce7a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009e50 tx=0x7f58c401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6423d6fb49618cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58cb7fe640 1 -- 172.21.15.133:0/3393144793 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c40a57e0 con 0x7f58cc0ce330 2024-04-25T10:03:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58cb7fe640 1 -- 172.21.15.133:0/3393144793 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c40a5980 con 0x7f58cc0ce330 2024-04-25T10:03:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58cb7fe640 1 -- 172.21.15.133:0/3393144793 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c40a5d50 con 0x7f58cc0ce330 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3393144793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58cc0ce330 msgr2=0x7f58cc0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 --2- 172.21.15.133:0/3393144793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58cc0ce330 0x7f58cc0ce7a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009e50 tx=0x7f58c401ca00 comp rx=0 tx=0).stop 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3393144793 shutdown_connections 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 --2- 172.21.15.133:0/3393144793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58cc0ce330 0x7f58cc0ce7a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 --2- 172.21.15.133:0/3393144793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58cc0b1f70 0x7f58cc06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3393144793 >> 172.21.15.133:0/3393144793 conn(0x7f58cc067000 msgr2=0x7f58cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3393144793 shutdown_connections 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3393144793 wait complete. 2024-04-25T10:03:12.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 Processor -- start 2024-04-25T10:03:12.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 -- start start 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58cc0b1f70 0x7f58cc145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58cc0ce330 0x7f58cc14ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58cc15c9a0 0x7f58cc14f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc15d950 con 0x7f58cc15c9a0 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc0ce130 con 0x7f58cc0b1f70 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc15dad0 con 0x7f58cc0ce330 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d1407640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58cc15c9a0 0x7f58cc14f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d1407640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58cc15c9a0 0x7f58cc14f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57498/0 (socket says 172.21.15.133:57498) 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d1407640 1 -- 172.21.15.133:0/3534021872 learned_addr learned my addr 172.21.15.133:0/3534021872 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d0c06640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58cc0b1f70 0x7f58cc145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58cbfff640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58cc0ce330 0x7f58cc14ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d1407640 1 -- 172.21.15.133:0/3534021872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58cc0ce330 msgr2=0x7f58cc14ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d1407640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58cc0ce330 0x7f58cc14ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d1407640 1 -- 172.21.15.133:0/3534021872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58cc0b1f70 msgr2=0x7f58cc145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d1407640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58cc0b1f70 0x7f58cc145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d1407640 1 -- 172.21.15.133:0/3534021872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc146970 con 0x7f58cc15c9a0 2024-04-25T10:03:12.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d1407640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58cc15c9a0 0x7f58cc14f220 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f58c40af040 tx=0x7f58c40197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58c97fa640 1 -- 172.21.15.133:0/3534021872 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c40b8670 con 0x7f58cc15c9a0 2024-04-25T10:03:12.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58c97fa640 1 -- 172.21.15.133:0/3534021872 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c40be070 con 0x7f58cc15c9a0 2024-04-25T10:03:12.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3534021872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58cc146c50 con 0x7f58cc15c9a0 2024-04-25T10:03:12.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58c97fa640 1 -- 172.21.15.133:0/3534021872 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c40b8a90 con 0x7f58cc15c9a0 2024-04-25T10:03:12.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.452+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3534021872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58cc147210 con 0x7f58cc15c9a0 2024-04-25T10:03:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.456+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3534021872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58cc06bc20 con 0x7f58cc15c9a0 2024-04-25T10:03:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.456+0000 7f58c97fa640 1 -- 172.21.15.133:0/3534021872 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58c40b8c30 con 0x7f58cc15c9a0 2024-04-25T10:03:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.456+0000 7f58c97fa640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f58b0066f60 0x7f58b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.456+0000 7f58c97fa640 1 -- 172.21.15.133:0/3534021872 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f58c4120150 con 0x7f58cc15c9a0 2024-04-25T10:03:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.456+0000 7f58d0c06640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f58b0066f60 0x7f58b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.456+0000 7f58d0c06640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f58b0066f60 0x7f58b0069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f58c00015b0 tx=0x7f58c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.460+0000 7f58c97fa640 1 -- 172.21.15.133:0/3534021872 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58c4016620 con 0x7f58cc15c9a0 2024-04-25T10:03:12.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.672+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3534021872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f58cc068ef0 con 0x7f58cc15c9a0 2024-04-25T10:03:12.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.672+0000 7f58c97fa640 1 -- 172.21.15.133:0/3534021872 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f58c40f1290 con 0x7f58cc15c9a0 2024-04-25T10:03:12.680 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:12.680 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:03:12.680 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:03:12.680 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.680 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:03:12.680 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:03:12.680 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.680 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:03:12.680 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:03:12.680 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.681 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:03:12.682 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.683 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.684 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.685 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:12.686 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.687 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:03:12.688 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.689 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:03:12.690 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.691 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-mds", 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQD3KSpmeyvYBBAAjORHjpUIyy8RiUpEeCWdJw==", 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQD3KSpmpkDYBBAAh71BTfhE4qHirVjSZ/7YyA==", 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.692 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-osd", 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQD3KSpmnlHYBBAAcKiPYcgaO5TGC787F4SMYw==", 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQD3KSpms2HYBBAAWgUa3qlGp6i/CDfk+8YJEw==", 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.693 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQD3KSpmpnLYBBAA61r8N85dyYBcz9sqeWB89A==", 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQD3KSpmuIPYBBAA6WFM6mamnkazoQK2t9FLvA==", 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.694 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:03:12.695 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.696 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:12.696 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:03:12.696 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:03:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3534021872 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f58b0066f60 msgr2=0x7f58b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f58b0066f60 0x7f58b0069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f58c00015b0 tx=0x7f58c0009040 comp rx=0 tx=0).stop 2024-04-25T10:03:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3534021872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58cc15c9a0 msgr2=0x7f58cc14f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58cc15c9a0 0x7f58cc14f220 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f58c40af040 tx=0x7f58c40197a0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3534021872 shutdown_connections 2024-04-25T10:03:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f58b0066f60 0x7f58b0069420 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58cc15c9a0 0x7f58cc14f220 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f58cc0ce330 0x7f58cc14ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 --2- 172.21.15.133:0/3534021872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f58cc0b1f70 0x7f58cc145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3534021872 >> 172.21.15.133:0/3534021872 conn(0x7f58cc067000 msgr2=0x7f58cc14f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3534021872 shutdown_connections 2024-04-25T10:03:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.676+0000 7f58d2e6a640 1 -- 172.21.15.133:0/3534021872 wait complete. 2024-04-25T10:03:12.699 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T10:03:12.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.800+0000 7f38850a4640 1 Processor -- start 2024-04-25T10:03:12.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.800+0000 7f38850a4640 1 -- start start 2024-04-25T10:03:12.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f38850a4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f388015b850 0x7f38800cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f38850a4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f38800d0100 0x7f38800b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f38850a4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38800cc530 con 0x7f38800b2cf0 2024-04-25T10:03:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f38850a4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38800b3160 con 0x7f38800d0100 2024-04-25T10:03:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f38850a4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38800cd140 con 0x7f388015b850 2024-04-25T10:03:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f3877fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f38800d0100 0x7f38800b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f3877fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f38800d0100 0x7f38800b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57502/0 (socket says 172.21.15.133:57502) 2024-04-25T10:03:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f3877fff640 1 -- 172.21.15.133:0/2249698908 learned_addr learned my addr 172.21.15.133:0/2249698908 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f387ed9d640 1 --2- 172.21.15.133:0/2249698908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f388015b850 0x7f38800cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f3877fff640 1 -- 172.21.15.133:0/2249698908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f388015b850 msgr2=0x7f38800cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f3877fff640 1 --2- 172.21.15.133:0/2249698908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f388015b850 0x7f38800cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f3877fff640 1 -- 172.21.15.133:0/2249698908 >> v1:172.21.15.162:6789/0 conn(0x7f38800b2cf0 legacy=0x7f388008af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f3877fff640 1 -- 172.21.15.133:0/2249698908 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38800cc810 con 0x7f38800d0100 2024-04-25T10:03:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.804+0000 7f3877fff640 1 --2- 172.21.15.133:0/2249698908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38800d0100 0x7f38800b27b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3868009870 tx=0x7f386801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89877d5e167f066 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f387e59c640 1 -- 172.21.15.133:0/2249698908 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38680ab650 con 0x7f38800d0100 2024-04-25T10:03:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f387e59c640 1 -- 172.21.15.133:0/2249698908 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38680b0070 con 0x7f38800d0100 2024-04-25T10:03:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f387e59c640 1 -- 172.21.15.133:0/2249698908 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38680aba10 con 0x7f38800d0100 2024-04-25T10:03:12.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 -- 172.21.15.133:0/2249698908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38800d0100 msgr2=0x7f38800b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 --2- 172.21.15.133:0/2249698908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38800d0100 0x7f38800b27b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3868009870 tx=0x7f386801c560 comp rx=0 tx=0).stop 2024-04-25T10:03:12.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 -- 172.21.15.133:0/2249698908 shutdown_connections 2024-04-25T10:03:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 --2- 172.21.15.133:0/2249698908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38800d0100 0x7f38800b27b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 --2- 172.21.15.133:0/2249698908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f388015b850 0x7f38800cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 -- 172.21.15.133:0/2249698908 >> 172.21.15.133:0/2249698908 conn(0x7f388009ab90 msgr2=0x7f38800d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 -- 172.21.15.133:0/2249698908 shutdown_connections 2024-04-25T10:03:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 -- 172.21.15.133:0/2249698908 wait complete. 2024-04-25T10:03:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 Processor -- start 2024-04-25T10:03:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 -- start start 2024-04-25T10:03:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38800b2cf0 0x7f388014a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38800d0100 0x7f3880156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f388015b850 0x7f3880156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388015bdb0 con 0x7f388015b850 2024-04-25T10:03:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388015bc30 con 0x7f38800d0100 2024-04-25T10:03:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f38850a4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38801684b0 con 0x7f38800b2cf0 2024-04-25T10:03:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f387f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f388015b850 0x7f3880156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f387f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f388015b850 0x7f3880156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57514/0 (socket says 172.21.15.133:57514) 2024-04-25T10:03:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f387f59e640 1 -- 172.21.15.133:0/1524592651 learned_addr learned my addr 172.21.15.133:0/1524592651 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f387ed9d640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38800b2cf0 0x7f388014a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.808+0000 7f3877fff640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38800d0100 0x7f3880156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f387f59e640 1 -- 172.21.15.133:0/1524592651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38800b2cf0 msgr2=0x7f388014a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f387f59e640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38800b2cf0 0x7f388014a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f387f59e640 1 -- 172.21.15.133:0/1524592651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38800d0100 msgr2=0x7f3880156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f387f59e640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38800d0100 0x7f3880156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f387f59e640 1 -- 172.21.15.133:0/1524592651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3880157620 con 0x7f388015b850 2024-04-25T10:03:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f3877fff640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38800d0100 0x7f3880156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:12.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f387f59e640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f388015b850 0x7f3880156fa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f387800c760 tx=0x7f387800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f38777fe640 1 -- 172.21.15.133:0/1524592651 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878003ba0 con 0x7f388015b850 2024-04-25T10:03:12.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f38777fe640 1 -- 172.21.15.133:0/1524592651 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3878003d40 con 0x7f388015b850 2024-04-25T10:03:12.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f38850a4640 1 -- 172.21.15.133:0/1524592651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3880157900 con 0x7f388015b850 2024-04-25T10:03:12.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f38777fe640 1 -- 172.21.15.133:0/1524592651 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387800aa90 con 0x7f388015b850 2024-04-25T10:03:12.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f38850a4640 1 -- 172.21.15.133:0/1524592651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3880157ec0 con 0x7f388015b850 2024-04-25T10:03:12.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f38850a4640 1 -- 172.21.15.133:0/1524592651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3880068e80 con 0x7f388015b850 2024-04-25T10:03:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f38777fe640 1 -- 172.21.15.133:0/1524592651 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f387800ac30 con 0x7f388015b850 2024-04-25T10:03:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f38777fe640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3860066f60 0x7f3860069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f38777fe640 1 -- 172.21.15.133:0/1524592651 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f38780857c0 con 0x7f388015b850 2024-04-25T10:03:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f387ed9d640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3860066f60 0x7f3860069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.812+0000 7f387ed9d640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3860066f60 0x7f3860069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3880071500 tx=0x7f3870008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:12.816+0000 7f38777fe640 1 -- 172.21.15.133:0/1524592651 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3878056990 con 0x7f388015b850 2024-04-25T10:03:13.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.016+0000 7f38850a4640 1 -- 172.21.15.133:0/1524592651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f38800b1fa0 con 0x7f388015b850 2024-04-25T10:03:13.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38777fe640 1 -- 172.21.15.133:0/1524592651 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3878056290 con 0x7f388015b850 2024-04-25T10:03:13.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 -- 172.21.15.133:0/1524592651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3860066f60 msgr2=0x7f3860069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3860066f60 0x7f3860069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3880071500 tx=0x7f3870008040 comp rx=0 tx=0).stop 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 -- 172.21.15.133:0/1524592651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f388015b850 msgr2=0x7f3880156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f388015b850 0x7f3880156fa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f387800c760 tx=0x7f387800cc20 comp rx=0 tx=0).stop 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 -- 172.21.15.133:0/1524592651 shutdown_connections 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3860066f60 0x7f3860069420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f388015b850 0x7f3880156fa0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38800d0100 0x7f3880156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 --2- 172.21.15.133:0/1524592651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38800b2cf0 0x7f388014a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 -- 172.21.15.133:0/1524592651 >> 172.21.15.133:0/1524592651 conn(0x7f388009ab90 msgr2=0x7f3880073790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 -- 172.21.15.133:0/1524592651 shutdown_connections 2024-04-25T10:03:13.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.020+0000 7f38850a4640 1 -- 172.21.15.133:0/1524592651 wait complete. 2024-04-25T10:03:13.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T10:03:13.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb7f73640 1 Processor -- start 2024-04-25T10:03:13.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb7f73640 1 -- start start 2024-04-25T10:03:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb7f73640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefb00cd820 0x7fefb00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb7f73640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fefb015ced0 0x7fefb015d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb7f73640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb00ce2e0 con 0x7fefb0145170 2024-04-25T10:03:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb7f73640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb006db00 con 0x7fefb015ced0 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb7f73640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0145550 con 0x7fefb00cd820 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb6510640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fefb015ced0 0x7fefb015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb5d0f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fefb0145170 0x7fefb006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50336/0 (socket says 172.21.15.133:50336) 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb6510640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fefb015ced0 0x7fefb015d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57520/0 (socket says 172.21.15.133:57520) 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb6510640 1 -- 172.21.15.133:0/503450418 learned_addr learned my addr 172.21.15.133:0/503450418 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb550e640 1 --2- 172.21.15.133:0/503450418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefb00cd820 0x7fefb00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb6510640 1 -- 172.21.15.133:0/503450418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefb00cd820 msgr2=0x7fefb00cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb6510640 1 --2- 172.21.15.133:0/503450418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefb00cd820 0x7fefb00cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb6510640 1 -- 172.21.15.133:0/503450418 >> v1:172.21.15.133:6789/0 conn(0x7fefb0145170 legacy=0x7fefb006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb4d0d640 1 -- 172.21.15.133:0/503450418 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744714787 0 0) 0x7fefb00ce2e0 con 0x7fefb0145170 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb6510640 1 -- 172.21.15.133:0/503450418 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb006d4b0 con 0x7fefb015ced0 2024-04-25T10:03:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.156+0000 7fefb550e640 1 --2- 172.21.15.133:0/503450418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefb00cd820 0x7fefb00cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:13.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb6510640 1 --2- 172.21.15.133:0/503450418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb015ced0 0x7fefb015d360 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fefac009e50 tx=0x7fefac01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94813e0b101df40d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb4d0d640 1 -- 172.21.15.133:0/503450418 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac0a5700 con 0x7fefb015ced0 2024-04-25T10:03:13.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb4d0d640 1 -- 172.21.15.133:0/503450418 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac0a58a0 con 0x7fefb015ced0 2024-04-25T10:03:13.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb4d0d640 1 -- 172.21.15.133:0/503450418 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac0a5c70 con 0x7fefb015ced0 2024-04-25T10:03:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 -- 172.21.15.133:0/503450418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb015ced0 msgr2=0x7fefb015d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 --2- 172.21.15.133:0/503450418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb015ced0 0x7fefb015d360 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fefac009e50 tx=0x7fefac01ca30 comp rx=0 tx=0).stop 2024-04-25T10:03:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 -- 172.21.15.133:0/503450418 shutdown_connections 2024-04-25T10:03:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 --2- 172.21.15.133:0/503450418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb015ced0 0x7fefb015d360 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 --2- 172.21.15.133:0/503450418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefb00cd820 0x7fefb00cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 -- 172.21.15.133:0/503450418 >> 172.21.15.133:0/503450418 conn(0x7fefb0066f70 msgr2=0x7fefb006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 -- 172.21.15.133:0/503450418 shutdown_connections 2024-04-25T10:03:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 -- 172.21.15.133:0/503450418 wait complete. 2024-04-25T10:03:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 Processor -- start 2024-04-25T10:03:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 -- start start 2024-04-25T10:03:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefb00cd820 0x7fefb013cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefb0145170 0x7fefb0163880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb015ced0 0x7fefb01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0168610 con 0x7fefb015ced0 2024-04-25T10:03:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0168490 con 0x7fefb0145170 2024-04-25T10:03:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb7f73640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0168790 con 0x7fefb00cd820 2024-04-25T10:03:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb6510640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb015ced0 0x7fefb01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb6510640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb015ced0 0x7fefb01609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57528/0 (socket says 172.21.15.133:57528) 2024-04-25T10:03:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb6510640 1 -- 172.21.15.133:0/3321102433 learned_addr learned my addr 172.21.15.133:0/3321102433 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb6510640 1 -- 172.21.15.133:0/3321102433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefb00cd820 msgr2=0x7fefb013cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb6510640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefb00cd820 0x7fefb013cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb6510640 1 -- 172.21.15.133:0/3321102433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefb0145170 msgr2=0x7fefb0163880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb6510640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefb0145170 0x7fefb0163880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb6510640 1 -- 172.21.15.133:0/3321102433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb013d3b0 con 0x7fefb015ced0 2024-04-25T10:03:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.160+0000 7fefb6510640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb015ced0 0x7fefb01609c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fefac009e50 tx=0x7fefac0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.164+0000 7fefa67fc640 1 -- 172.21.15.133:0/3321102433 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac0bd020 con 0x7fefb015ced0 2024-04-25T10:03:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.164+0000 7fefa67fc640 1 -- 172.21.15.133:0/3321102433 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac0b8670 con 0x7fefb015ced0 2024-04-25T10:03:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.164+0000 7fefb7f73640 1 -- 172.21.15.133:0/3321102433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb00ae790 con 0x7fefb015ced0 2024-04-25T10:03:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.164+0000 7fefa67fc640 1 -- 172.21.15.133:0/3321102433 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac0b8a40 con 0x7fefb015ced0 2024-04-25T10:03:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.164+0000 7fefb7f73640 1 -- 172.21.15.133:0/3321102433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb0141480 con 0x7fefb015ced0 2024-04-25T10:03:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.164+0000 7fefb7f73640 1 -- 172.21.15.133:0/3321102433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb0068e80 con 0x7fefb015ced0 2024-04-25T10:03:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.164+0000 7fefa67fc640 1 -- 172.21.15.133:0/3321102433 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fefac0c0070 con 0x7fefb015ced0 2024-04-25T10:03:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.164+0000 7fefa67fc640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef94066f60 0x7fef94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.164+0000 7fefa67fc640 1 -- 172.21.15.133:0/3321102433 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fefac11f5c0 con 0x7fefb015ced0 2024-04-25T10:03:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.164+0000 7fefb5d0f640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef94066f60 0x7fef94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.164+0000 7fefb5d0f640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef94066f60 0x7fef94069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fefa80011f0 tx=0x7fefa8007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.168+0000 7fefa67fc640 1 -- 172.21.15.133:0/3321102433 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fefac016620 con 0x7fefb015ced0 2024-04-25T10:03:13.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.372+0000 7fefb7f73640 1 -- 172.21.15.133:0/3321102433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fefb00cdda0 con 0x7fefb015ced0 2024-04-25T10:03:13.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefa67fc640 1 -- 172.21.15.133:0/3321102433 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fefac0a5870 con 0x7fefb015ced0 2024-04-25T10:03:13.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 -- 172.21.15.133:0/3321102433 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef94066f60 msgr2=0x7fef94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef94066f60 0x7fef94069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fefa80011f0 tx=0x7fefa8007930 comp rx=0 tx=0).stop 2024-04-25T10:03:13.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 -- 172.21.15.133:0/3321102433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb015ced0 msgr2=0x7fefb01609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb015ced0 0x7fefb01609c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fefac009e50 tx=0x7fefac0197d0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 -- 172.21.15.133:0/3321102433 shutdown_connections 2024-04-25T10:03:13.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef94066f60 0x7fef94069420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb015ced0 0x7fefb01609c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefb0145170 0x7fefb0163880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 --2- 172.21.15.133:0/3321102433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefb00cd820 0x7fefb013cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 -- 172.21.15.133:0/3321102433 >> 172.21.15.133:0/3321102433 conn(0x7fefb0066f70 msgr2=0x7fefb013ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 -- 172.21.15.133:0/3321102433 shutdown_connections 2024-04-25T10:03:13.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.376+0000 7fefb7f73640 1 -- 172.21.15.133:0/3321102433 wait complete. 2024-04-25T10:03:13.399 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T10:03:13.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8bd017640 1 Processor -- start 2024-04-25T10:03:13.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8bd017640 1 -- start start 2024-04-25T10:03:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8bd017640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8b8142920 0x7fc8b8143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8bd017640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b8140ea0 con 0x7fc8b815d9f0 2024-04-25T10:03:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8bd017640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b80cf280 con 0x7fc8b8140a40 2024-04-25T10:03:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8bd017640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b806d440 con 0x7fc8b8142920 2024-04-25T10:03:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8b759e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc8b815d9f0 0x7fc8b806df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50348/0 (socket says 172.21.15.133:50348) 2024-04-25T10:03:13.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8b759e640 1 -- 172.21.15.133:0/1060142563 learned_addr learned my addr 172.21.15.133:0/1060142563 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833551623 0 0) 0x7fc8b8140ea0 con 0x7fc8b815d9f0 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8b6d9d640 1 --2- 172.21.15.133:0/1060142563 >> v2:172.21.15.162:3300/0 conn(0x7fc8b8142920 0x7fc8b8143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8a4005740 con 0x7fc8b815d9f0 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.504+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2005465933 0 0) 0x7fc8a4005740 con 0x7fc8b815d9f0 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 >> v2:172.21.15.162:3300/0 conn(0x7fc8b8142920 msgr2=0x7fc8b8143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 --2- 172.21.15.133:0/1060142563 >> v2:172.21.15.162:3300/0 conn(0x7fc8b8142920 0x7fc8b8143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 >> v1:172.21.15.162:6789/0 conn(0x7fc8b8140a40 legacy=0x7fc8b80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b806d720 con 0x7fc8b815d9f0 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fc8b0001a20 con 0x7fc8b815d9f0 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 >> v1:172.21.15.133:6789/0 conn(0x7fc8b815d9f0 legacy=0x7fc8b806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 --2- 172.21.15.133:0/1060142563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a4005bd0 0x7fc8a4005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 --2- 172.21.15.133:0/1060142563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a4006a20 0x7fc8a4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 --2- 172.21.15.133:0/1060142563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a40073e0 0x7fc8a400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b806d440 con 0x7fc8a4006a20 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a400e380 con 0x7fc8a4005bd0 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a400e580 con 0x7fc8a40073e0 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8ae59c640 1 --2- 172.21.15.133:0/1060142563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a4006a20 0x7fc8a4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b759e640 1 --2- 172.21.15.133:0/1060142563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a40073e0 0x7fc8a400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b6d9d640 1 --2- 172.21.15.133:0/1060142563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a4005bd0 0x7fc8a4005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8ae59c640 1 -- 172.21.15.133:0/1060142563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a40073e0 msgr2=0x7fc8a400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8ae59c640 1 --2- 172.21.15.133:0/1060142563 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a40073e0 0x7fc8a400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8ae59c640 1 -- 172.21.15.133:0/1060142563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a4005bd0 msgr2=0x7fc8a4005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8ae59c640 1 --2- 172.21.15.133:0/1060142563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8a4005bd0 0x7fc8a4005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b759e640 1 -- 172.21.15.133:0/1060142563 reap_dead start 2024-04-25T10:03:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8ae59c640 1 -- 172.21.15.133:0/1060142563 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a4005740 con 0x7fc8a4006a20 2024-04-25T10:03:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8ae59c640 1 --2- 172.21.15.133:0/1060142563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a4006a20 0x7fc8a4006ea0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0001780 tx=0x7fc8a001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aba2de0bf4c2594 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a00013a0 con 0x7fc8a4006a20 2024-04-25T10:03:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a00052f0 con 0x7fc8a4006a20 2024-04-25T10:03:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8b659c640 1 -- 172.21.15.133:0/1060142563 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0003840 con 0x7fc8a4006a20 2024-04-25T10:03:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8bd017640 1 -- 172.21.15.133:0/1060142563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a4006a20 msgr2=0x7fc8a4006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8bd017640 1 --2- 172.21.15.133:0/1060142563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a4006a20 0x7fc8a4006ea0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0001780 tx=0x7fc8a001c400 comp rx=0 tx=0).stop 2024-04-25T10:03:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8bd017640 1 -- 172.21.15.133:0/1060142563 shutdown_connections 2024-04-25T10:03:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8bd017640 1 --2- 172.21.15.133:0/1060142563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a4006a20 0x7fc8a4006ea0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8bd017640 1 -- 172.21.15.133:0/1060142563 >> 172.21.15.133:0/1060142563 conn(0x7fc8b8066f70 msgr2=0x7fc8b806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8bd017640 1 -- 172.21.15.133:0/1060142563 shutdown_connections 2024-04-25T10:03:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.508+0000 7fc8bd017640 1 -- 172.21.15.133:0/1060142563 wait complete. 2024-04-25T10:03:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8bd017640 1 Processor -- start 2024-04-25T10:03:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8bd017640 1 -- start start 2024-04-25T10:03:13.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8bd017640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a4006a20 0x7fc8b813fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8bd017640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b80acf50 0x7fc8b80ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8bd017640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8b8154080 0x7fc8b80b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8bd017640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b815df50 con 0x7fc8b8154080 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8bd017640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b815ddd0 con 0x7fc8b80acf50 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8bd017640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b815e0d0 con 0x7fc8a4006a20 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8b759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8b8154080 0x7fc8b80b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8b759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8b8154080 0x7fc8b80b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57550/0 (socket says 172.21.15.133:57550) 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8b759e640 1 -- 172.21.15.133:0/1334590963 learned_addr learned my addr 172.21.15.133:0/1334590963 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8b6d9d640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a4006a20 0x7fc8b813fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8ae59c640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b80acf50 0x7fc8b80ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8b759e640 1 -- 172.21.15.133:0/1334590963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a4006a20 msgr2=0x7fc8b813fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8b759e640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a4006a20 0x7fc8b813fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8b759e640 1 -- 172.21.15.133:0/1334590963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b80acf50 msgr2=0x7fc8b80ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8b759e640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b80acf50 0x7fc8b80ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8b759e640 1 -- 172.21.15.133:0/1334590963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b80ad3f0 con 0x7fc8b8154080 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8ae59c640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b80acf50 0x7fc8b80ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8b759e640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8b8154080 0x7fc8b80b25b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8143ad0 tx=0x7fc8b000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8affff640 1 -- 172.21.15.133:0/1334590963 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000ec10 con 0x7fc8b8154080 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8affff640 1 -- 172.21.15.133:0/1334590963 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0018070 con 0x7fc8b8154080 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8bd017640 1 -- 172.21.15.133:0/1334590963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b81535a0 con 0x7fc8b8154080 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8affff640 1 -- 172.21.15.133:0/1334590963 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0013420 con 0x7fc8b8154080 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8bd017640 1 -- 172.21.15.133:0/1334590963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b8140470 con 0x7fc8b8154080 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8bd017640 1 -- 172.21.15.133:0/1334590963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b80ad6d0 con 0x7fc8b8154080 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.512+0000 7fc8affff640 1 -- 172.21.15.133:0/1334590963 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8b00135c0 con 0x7fc8b8154080 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.516+0000 7fc8affff640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc898066f60 0x7fc898069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.516+0000 7fc8b6d9d640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc898066f60 0x7fc898069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.516+0000 7fc8affff640 1 -- 172.21.15.133:0/1334590963 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc8b0085370 con 0x7fc8b8154080 2024-04-25T10:03:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.516+0000 7fc8b6d9d640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc898066f60 0x7fc898069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8b804f450 tx=0x7fc8a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.520+0000 7fc8affff640 1 -- 172.21.15.133:0/1334590963 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc8b00564b0 con 0x7fc8b8154080 2024-04-25T10:03:13.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.728+0000 7fc8bd017640 1 -- 172.21.15.133:0/1334590963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fc8b8068e80 con 0x7fc8b8154080 2024-04-25T10:03:13.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8affff640 1 -- 172.21.15.133:0/1334590963 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc8b0055db0 con 0x7fc8b8154080 2024-04-25T10:03:13.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 -- 172.21.15.133:0/1334590963 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc898066f60 msgr2=0x7fc898069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc898066f60 0x7fc898069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8b804f450 tx=0x7fc8a8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:13.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 -- 172.21.15.133:0/1334590963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8b8154080 msgr2=0x7fc8b80b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8b8154080 0x7fc8b80b25b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8143ad0 tx=0x7fc8b000e530 comp rx=0 tx=0).stop 2024-04-25T10:03:13.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 -- 172.21.15.133:0/1334590963 shutdown_connections 2024-04-25T10:03:13.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc898066f60 0x7fc898069420 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8b8154080 0x7fc8b80b25b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8b80acf50 0x7fc8b80ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 --2- 172.21.15.133:0/1334590963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8a4006a20 0x7fc8b813fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 -- 172.21.15.133:0/1334590963 >> 172.21.15.133:0/1334590963 conn(0x7fc8b8066f70 msgr2=0x7fc8b815c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 -- 172.21.15.133:0/1334590963 shutdown_connections 2024-04-25T10:03:13.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.732+0000 7fc8bd017640 1 -- 172.21.15.133:0/1334590963 wait complete. 2024-04-25T10:03:13.755 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T10:03:13.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9e0d3c640 1 Processor -- start 2024-04-25T10:03:13.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9e0d3c640 1 -- start start 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9e0d3c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa9dc15d9a0 0x7fa9dc0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9e0d3c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9dc0ce600 0x7fa9dc0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9e0d3c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa9dc0d1920 0x7fa9dc0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9e0d3c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9dc0cefa0 con 0x7fa9dc15d9a0 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9e0d3c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9dc15ebd0 con 0x7fa9dc0d1920 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9e0d3c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc15eeb0 con 0x7fa9dc0ce600 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9da59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa9dc15d9a0 0x7fa9dc0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9da59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa9dc15d9a0 0x7fa9dc0d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57562/0 (socket says 172.21.15.133:57562) 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9da59c640 1 -- 172.21.15.133:0/2972339222 learned_addr learned my addr 172.21.15.133:0/2972339222 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9d9d9b640 1 --2- 172.21.15.133:0/2972339222 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9dc0ce600 0x7fa9dc0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9dad9d640 1 --2- 172.21.15.133:0/2972339222 >> v2:172.21.15.162:3300/0 conn(0x7fa9dc0d1920 0x7fa9dc0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9da59c640 1 -- 172.21.15.133:0/2972339222 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9dc0ce600 msgr2=0x7fa9dc0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9da59c640 1 --2- 172.21.15.133:0/2972339222 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9dc0ce600 0x7fa9dc0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9da59c640 1 -- 172.21.15.133:0/2972339222 >> v2:172.21.15.162:3300/0 conn(0x7fa9dc0d1920 msgr2=0x7fa9dc0d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9da59c640 1 --2- 172.21.15.133:0/2972339222 >> v2:172.21.15.162:3300/0 conn(0x7fa9dc0d1920 0x7fa9dc0d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.856+0000 7fa9da59c640 1 -- 172.21.15.133:0/2972339222 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc0d0330 con 0x7fa9dc15d9a0 2024-04-25T10:03:13.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9da59c640 1 --2- 172.21.15.133:0/2972339222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc15d9a0 0x7fa9dc0d0b30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc1615d0 tx=0x7fa9cc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d162223bbe7205 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9d959a640 1 -- 172.21.15.133:0/2972339222 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0ab600 con 0x7fa9dc15d9a0 2024-04-25T10:03:13.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9d959a640 1 -- 172.21.15.133:0/2972339222 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0b0070 con 0x7fa9dc15d9a0 2024-04-25T10:03:13.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9d959a640 1 -- 172.21.15.133:0/2972339222 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0ab9c0 con 0x7fa9dc15d9a0 2024-04-25T10:03:13.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/2972339222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc15d9a0 msgr2=0x7fa9dc0d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9e0d3c640 1 --2- 172.21.15.133:0/2972339222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc15d9a0 0x7fa9dc0d0b30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc1615d0 tx=0x7fa9cc01c460 comp rx=0 tx=0).stop 2024-04-25T10:03:13.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/2972339222 shutdown_connections 2024-04-25T10:03:13.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9e0d3c640 1 --2- 172.21.15.133:0/2972339222 >> v2:172.21.15.162:3300/0 conn(0x7fa9dc0d1920 0x7fa9dc0d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9e0d3c640 1 --2- 172.21.15.133:0/2972339222 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9dc0ce600 0x7fa9dc0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9e0d3c640 1 --2- 172.21.15.133:0/2972339222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc15d9a0 0x7fa9dc0d0b30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/2972339222 >> 172.21.15.133:0/2972339222 conn(0x7fa9dc06bf80 msgr2=0x7fa9dc06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/2972339222 shutdown_connections 2024-04-25T10:03:13.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.860+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/2972339222 wait complete. 2024-04-25T10:03:13.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9e0d3c640 1 Processor -- start 2024-04-25T10:03:13.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9e0d3c640 1 -- start start 2024-04-25T10:03:13.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9e0d3c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9dc0ce600 0x7fa9dc162810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9e0d3c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc0d1920 0x7fa9dc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9e0d3c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9dc15d9a0 0x7fa9dc15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9e0d3c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0b9820 con 0x7fa9dc0d1920 2024-04-25T10:03:13.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9e0d3c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0b96a0 con 0x7fa9dc15d9a0 2024-04-25T10:03:13.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9e0d3c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0d22f0 con 0x7fa9dc0ce600 2024-04-25T10:03:13.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9d9d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc0d1920 0x7fa9dc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9d9d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc0d1920 0x7fa9dc15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57574/0 (socket says 172.21.15.133:57574) 2024-04-25T10:03:13.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9d9d9b640 1 -- 172.21.15.133:0/4220380460 learned_addr learned my addr 172.21.15.133:0/4220380460 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9da59c640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9dc0ce600 0x7fa9dc162810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9d9d9b640 1 -- 172.21.15.133:0/4220380460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9dc0ce600 msgr2=0x7fa9dc162810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9d9d9b640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9dc0ce600 0x7fa9dc162810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9d9d9b640 1 -- 172.21.15.133:0/4220380460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9dc15d9a0 msgr2=0x7fa9dc15bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9dad9d640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9dc15d9a0 0x7fa9dc15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9d9d9b640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9dc15d9a0 0x7fa9dc15bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9d9d9b640 1 -- 172.21.15.133:0/4220380460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc0c9cf0 con 0x7fa9dc0d1920 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9da59c640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9dc0ce600 0x7fa9dc162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9d9d9b640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc0d1920 0x7fa9dc15b6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000d5f0 tx=0x7fa9d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9caffd640 1 -- 172.21.15.133:0/4220380460 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0018020 con 0x7fa9dc0d1920 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9caffd640 1 -- 172.21.15.133:0/4220380460 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0014620 con 0x7fa9dc0d1920 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/4220380460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9dc0c9a10 con 0x7fa9dc0d1920 2024-04-25T10:03:13.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9caffd640 1 -- 172.21.15.133:0/4220380460 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0014a10 con 0x7fa9dc0d1920 2024-04-25T10:03:13.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/4220380460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc0cb5a0 con 0x7fa9dc0d1920 2024-04-25T10:03:13.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/4220380460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9dc068e80 con 0x7fa9dc0d1920 2024-04-25T10:03:13.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9caffd640 1 -- 172.21.15.133:0/4220380460 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9d001b020 con 0x7fa9dc0d1920 2024-04-25T10:03:13.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.864+0000 7fa9caffd640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9bc066f60 0x7fa9bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.868+0000 7fa9da59c640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9bc066f60 0x7fa9bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.868+0000 7fa9caffd640 1 -- 172.21.15.133:0/4220380460 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa9d00852e0 con 0x7fa9dc0d1920 2024-04-25T10:03:13.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.868+0000 7fa9da59c640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9bc066f60 0x7fa9bc069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0af000 tx=0x7fa9cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:13.868+0000 7fa9caffd640 1 -- 172.21.15.133:0/4220380460 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9d0022020 con 0x7fa9dc0d1920 2024-04-25T10:03:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.076+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/4220380460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fa9dc0b2290 con 0x7fa9dc0d1920 2024-04-25T10:03:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.080+0000 7fa9caffd640 1 -- 172.21.15.133:0/4220380460 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa9d0056540 con 0x7fa9dc0d1920 2024-04-25T10:03:14.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/4220380460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9bc066f60 msgr2=0x7fa9bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9bc066f60 0x7fa9bc069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0af000 tx=0x7fa9cc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:14.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/4220380460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc0d1920 msgr2=0x7fa9dc15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc0d1920 0x7fa9dc15b6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000d5f0 tx=0x7fa9d000dab0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/4220380460 shutdown_connections 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9dc15d9a0 0x7fa9dc15bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9bc066f60 0x7fa9bc069420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9dc0d1920 0x7fa9dc15b6a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 --2- 172.21.15.133:0/4220380460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9dc0ce600 0x7fa9dc162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/4220380460 >> 172.21.15.133:0/4220380460 conn(0x7fa9dc06bf80 msgr2=0x7fa9dc0b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/4220380460 shutdown_connections 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.084+0000 7fa9e0d3c640 1 -- 172.21.15.133:0/4220380460 wait complete. 2024-04-25T10:03:14.106 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021c8b4640 1 Processor -- start 2024-04-25T10:03:14.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021c8b4640 1 -- start start 2024-04-25T10:03:14.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021c8b4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0218143ea0 0x7f02180cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021c8b4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021815ac50 con 0x7f021806eac0 2024-04-25T10:03:14.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021c8b4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02180cd180 con 0x7f02180b2150 2024-04-25T10:03:14.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021c8b4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021815aed0 con 0x7f0218143ea0 2024-04-25T10:03:14.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f0216d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f021806eac0 0x7f02180b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50354/0 (socket says 172.21.15.133:50354) 2024-04-25T10:03:14.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f0216d9d640 1 -- 172.21.15.133:0/2000093732 learned_addr learned my addr 172.21.15.133:0/2000093732 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021659c640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0218143ea0 0x7f02180cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226309584 0 0) 0x7f02100014a0 con 0x7f021806eac0 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0200003530 con 0x7f021806eac0 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2025669513 0 0) 0x7f0200003530 con 0x7f021806eac0 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0218143ea0 msgr2=0x7f02180cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0218143ea0 0x7f02180cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 >> v1:172.21.15.162:6789/0 conn(0x7f02180b2150 legacy=0x7f021806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021815b1b0 con 0x7f021806eac0 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0210001bd0 con 0x7f021806eac0 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 >> v1:172.21.15.133:6789/0 conn(0x7f021806eac0 legacy=0x7f02180b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0218143ea0 0x7f02000039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0200003f70 0x7f0200004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0200004d50 0x7f02000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021815aed0 con 0x7f0200003f70 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020000bab0 con 0x7f0200004d50 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020000bcb0 con 0x7f0218143ea0 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f0215d9b640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0200003f70 0x7f0200004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f021659c640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0218143ea0 0x7f02000039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f0216d9d640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0200004d50 0x7f02000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f0215d9b640 1 -- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0218143ea0 msgr2=0x7f02000039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f0215d9b640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0218143ea0 0x7f02000039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f0215d9b640 1 -- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0200004d50 msgr2=0x7f02000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f0215d9b640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0200004d50 0x7f02000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f0215d9b640 1 -- 172.21.15.133:0/2000093732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0200003530 con 0x7f0200003f70 2024-04-25T10:03:14.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.208+0000 7f0216d9d640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0200004d50 0x7f02000051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:14.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f0215d9b640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0200003f70 0x7f0200004810 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f020c001780 tx=0x7f020c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97178615b66c6ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c0013a0 con 0x7f0200003f70 2024-04-25T10:03:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c0052f0 con 0x7f0200003f70 2024-04-25T10:03:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f021559a640 1 -- 172.21.15.133:0/2000093732 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c003840 con 0x7f0200003f70 2024-04-25T10:03:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f021c8b4640 1 -- 172.21.15.133:0/2000093732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0200003f70 msgr2=0x7f0200004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f021c8b4640 1 --2- 172.21.15.133:0/2000093732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0200003f70 0x7f0200004810 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f020c001780 tx=0x7f020c01c420 comp rx=0 tx=0).stop 2024-04-25T10:03:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f0216d9d640 1 -- 172.21.15.133:0/2000093732 reap_dead start 2024-04-25T10:03:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f021c8b4640 1 -- 172.21.15.133:0/2000093732 shutdown_connections 2024-04-25T10:03:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f021c8b4640 1 -- 172.21.15.133:0/2000093732 >> 172.21.15.133:0/2000093732 conn(0x7f0218066f70 msgr2=0x7f021806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f021c8b4640 1 -- 172.21.15.133:0/2000093732 shutdown_connections 2024-04-25T10:03:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f021c8b4640 1 -- 172.21.15.133:0/2000093732 wait complete. 2024-04-25T10:03:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f021c8b4640 1 Processor -- start 2024-04-25T10:03:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.212+0000 7f021c8b4640 1 -- start start 2024-04-25T10:03:14.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021c8b4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0200003f70 0x7f02181f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021c8b4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0218140340 0x7f0218139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021c8b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f021813d240 0x7f021813d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021c8b4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021815bb30 con 0x7f021813d240 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021c8b4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021815b9b0 con 0x7f0218140340 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021c8b4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0218144280 con 0x7f0200003f70 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0216d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f021813d240 0x7f021813d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0216d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f021813d240 0x7f021813d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57592/0 (socket says 172.21.15.133:57592) 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0216d9d640 1 -- 172.21.15.133:0/450224098 learned_addr learned my addr 172.21.15.133:0/450224098 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021659c640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0200003f70 0x7f02181f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0216d9d640 1 -- 172.21.15.133:0/450224098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0200003f70 msgr2=0x7f02181f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0216d9d640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0200003f70 0x7f02181f42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0216d9d640 1 -- 172.21.15.133:0/450224098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0218140340 msgr2=0x7f0218139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0216d9d640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0218140340 0x7f0218139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0216d9d640 1 -- 172.21.15.133:0/450224098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021813a630 con 0x7f021813d240 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0216d9d640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f021813d240 0x7f021813d6f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f02180cddb0 tx=0x7f0210010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0206ffd640 1 -- 172.21.15.133:0/450224098 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021001a070 con 0x7f021813d240 2024-04-25T10:03:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0206ffd640 1 -- 172.21.15.133:0/450224098 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0210011040 con 0x7f021813d240 2024-04-25T10:03:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0206ffd640 1 -- 172.21.15.133:0/450224098 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021000ea90 con 0x7f021813d240 2024-04-25T10:03:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021c8b4640 1 -- 172.21.15.133:0/450224098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021813a910 con 0x7f021813d240 2024-04-25T10:03:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021c8b4640 1 -- 172.21.15.133:0/450224098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021813aed0 con 0x7f021813d240 2024-04-25T10:03:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021c8b4640 1 -- 172.21.15.133:0/450224098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0218068e80 con 0x7f021813d240 2024-04-25T10:03:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0206ffd640 1 -- 172.21.15.133:0/450224098 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f021000ec30 con 0x7f021813d240 2024-04-25T10:03:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0206ffd640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f8066f90 0x7f01f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021659c640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f8066f90 0x7f01f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f0206ffd640 1 -- 172.21.15.133:0/450224098 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f02100871e0 con 0x7f021813d240 2024-04-25T10:03:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.216+0000 7f021659c640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f8066f90 0x7f01f8069450 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f021815c860 tx=0x7f0208008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.220+0000 7f0206ffd640 1 -- 172.21.15.133:0/450224098 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f021815ac50 con 0x7f021813d240 2024-04-25T10:03:14.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.424+0000 7f021c8b4640 1 -- 172.21.15.133:0/450224098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f0218159db0 con 0x7f021813d240 2024-04-25T10:03:14.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.428+0000 7f0206ffd640 1 -- 172.21.15.133:0/450224098 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0210058440 con 0x7f021813d240 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 -- 172.21.15.133:0/450224098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f8066f90 msgr2=0x7f01f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f8066f90 0x7f01f8069450 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f021815c860 tx=0x7f0208008040 comp rx=0 tx=0).stop 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 -- 172.21.15.133:0/450224098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f021813d240 msgr2=0x7f021813d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f021813d240 0x7f021813d6f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f02180cddb0 tx=0x7f0210010c30 comp rx=0 tx=0).stop 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 -- 172.21.15.133:0/450224098 shutdown_connections 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f8066f90 0x7f01f8069450 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f021813d240 0x7f021813d6f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0218140340 0x7f0218139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 --2- 172.21.15.133:0/450224098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0200003f70 0x7f02181f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 -- 172.21.15.133:0/450224098 >> 172.21.15.133:0/450224098 conn(0x7f0218066f70 msgr2=0x7f021806d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 -- 172.21.15.133:0/450224098 shutdown_connections 2024-04-25T10:03:14.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.432+0000 7f021c8b4640 1 -- 172.21.15.133:0/450224098 wait complete. 2024-04-25T10:03:14.453 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T10:03:14.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae5693e640 1 Processor -- start 2024-04-25T10:03:14.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae5693e640 1 -- start start 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae5693e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae5006b3a0 0x7fae5006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae5693e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fae5006f020 0x7fae500ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae5693e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae500701a0 con 0x7fae5015db70 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae5693e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5006eb90 con 0x7fae5006b3a0 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae5693e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5006e540 con 0x7fae5006f020 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae4f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae5006b3a0 0x7fae5006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae4f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae5006b3a0 0x7fae5006fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57608/0 (socket says 172.21.15.133:57608) 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae4f7fe640 1 -- 172.21.15.133:0/2811464955 learned_addr learned my addr 172.21.15.133:0/2811464955 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae54edb640 1 --2- 172.21.15.133:0/2811464955 >> v2:172.21.15.162:3300/0 conn(0x7fae5006f020 0x7fae500ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae4f7fe640 1 -- 172.21.15.133:0/2811464955 >> v2:172.21.15.162:3300/0 conn(0x7fae5006f020 msgr2=0x7fae500ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae4f7fe640 1 --2- 172.21.15.133:0/2811464955 >> v2:172.21.15.162:3300/0 conn(0x7fae5006f020 0x7fae500ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae4f7fe640 1 -- 172.21.15.133:0/2811464955 >> v1:172.21.15.162:6789/0 conn(0x7fae5015db70 legacy=0x7fae5006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.564+0000 7fae4f7fe640 1 -- 172.21.15.133:0/2811464955 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5006e820 con 0x7fae5006b3a0 2024-04-25T10:03:14.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae4f7fe640 1 --2- 172.21.15.133:0/2811464955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae5006b3a0 0x7fae5006fbc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fae5006ed10 tx=0x7fae3801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de309e2f9b03be47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae4effd640 1 -- 172.21.15.133:0/2811464955 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380ab5f0 con 0x7fae5006b3a0 2024-04-25T10:03:14.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae4effd640 1 -- 172.21.15.133:0/2811464955 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae380b0070 con 0x7fae5006b3a0 2024-04-25T10:03:14.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae4effd640 1 -- 172.21.15.133:0/2811464955 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380ab9b0 con 0x7fae5006b3a0 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae5693e640 1 -- 172.21.15.133:0/2811464955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae5006b3a0 msgr2=0x7fae5006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae5693e640 1 --2- 172.21.15.133:0/2811464955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae5006b3a0 0x7fae5006fbc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fae5006ed10 tx=0x7fae3801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae5693e640 1 -- 172.21.15.133:0/2811464955 shutdown_connections 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae5693e640 1 --2- 172.21.15.133:0/2811464955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae5006b3a0 0x7fae5006fbc0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae5693e640 1 --2- 172.21.15.133:0/2811464955 >> v2:172.21.15.162:3300/0 conn(0x7fae5006f020 0x7fae500ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae5693e640 1 -- 172.21.15.133:0/2811464955 >> 172.21.15.133:0/2811464955 conn(0x7fae500999d0 msgr2=0x7fae500b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae5693e640 1 -- 172.21.15.133:0/2811464955 shutdown_connections 2024-04-25T10:03:14.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae5693e640 1 -- 172.21.15.133:0/2811464955 wait complete. 2024-04-25T10:03:14.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae5693e640 1 Processor -- start 2024-04-25T10:03:14.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.568+0000 7fae5693e640 1 -- start start 2024-04-25T10:03:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae5693e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae5006b3a0 0x7fae501517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae5693e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae5006f020 0x7fae500cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae5693e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae5015db70 0x7fae5015e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae5693e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5016ac40 con 0x7fae5006f020 2024-04-25T10:03:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae5693e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5016aac0 con 0x7fae5015db70 2024-04-25T10:03:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae5693e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5016adc0 con 0x7fae5006b3a0 2024-04-25T10:03:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae5006f020 0x7fae500cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae5006f020 0x7fae500cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57624/0 (socket says 172.21.15.133:57624) 2024-04-25T10:03:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4f7fe640 1 -- 172.21.15.133:0/3450504517 learned_addr learned my addr 172.21.15.133:0/3450504517 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4ffff640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae5006b3a0 0x7fae501517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae54edb640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae5015db70 0x7fae5015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4f7fe640 1 -- 172.21.15.133:0/3450504517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae5006b3a0 msgr2=0x7fae501517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4f7fe640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae5006b3a0 0x7fae501517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4f7fe640 1 -- 172.21.15.133:0/3450504517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae5015db70 msgr2=0x7fae5015e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4f7fe640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae5015db70 0x7fae5015e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4f7fe640 1 -- 172.21.15.133:0/3450504517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae500cb5e0 con 0x7fae5006f020 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae54edb640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae5015db70 0x7fae5015e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4f7fe640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae5006f020 0x7fae500cab20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fae3801cd80 tx=0x7fae3801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4cff9640 1 -- 172.21.15.133:0/3450504517 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380b4800 con 0x7fae5006f020 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4cff9640 1 -- 172.21.15.133:0/3450504517 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae380b49a0 con 0x7fae5006f020 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4cff9640 1 -- 172.21.15.133:0/3450504517 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380b4d90 con 0x7fae5006f020 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae5693e640 1 -- 172.21.15.133:0/3450504517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae500cb8c0 con 0x7fae5006f020 2024-04-25T10:03:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae5693e640 1 -- 172.21.15.133:0/3450504517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae500cbe80 con 0x7fae5006f020 2024-04-25T10:03:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae5693e640 1 -- 172.21.15.133:0/3450504517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae50068e80 con 0x7fae5006f020 2024-04-25T10:03:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4cff9640 1 -- 172.21.15.133:0/3450504517 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae380b3380 con 0x7fae5006f020 2024-04-25T10:03:14.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4cff9640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae30066f60 0x7fae30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4cff9640 1 -- 172.21.15.133:0/3450504517 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fae3811f0a0 con 0x7fae5006f020 2024-04-25T10:03:14.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.572+0000 7fae4ffff640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae30066f60 0x7fae30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.576+0000 7fae4ffff640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae30066f60 0x7fae30069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fae500cdb30 tx=0x7fae400094d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.580+0000 7fae4cff9640 1 -- 172.21.15.133:0/3450504517 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae380160d0 con 0x7fae5006f020 2024-04-25T10:03:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.780+0000 7fae5693e640 1 -- 172.21.15.133:0/3450504517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fae5006ac90 con 0x7fae5006f020 2024-04-25T10:03:14.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae4cff9640 1 -- 172.21.15.133:0/3450504517 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fae380efd40 con 0x7fae5006f020 2024-04-25T10:03:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 -- 172.21.15.133:0/3450504517 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae30066f60 msgr2=0x7fae30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae30066f60 0x7fae30069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fae500cdb30 tx=0x7fae400094d0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 -- 172.21.15.133:0/3450504517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae5006f020 msgr2=0x7fae500cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae5006f020 0x7fae500cab20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fae3801cd80 tx=0x7fae3801ad20 comp rx=0 tx=0).stop 2024-04-25T10:03:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 -- 172.21.15.133:0/3450504517 shutdown_connections 2024-04-25T10:03:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae5015db70 0x7fae5015e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae30066f60 0x7fae30069420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae5006f020 0x7fae500cab20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 --2- 172.21.15.133:0/3450504517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae5006b3a0 0x7fae501517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 -- 172.21.15.133:0/3450504517 >> 172.21.15.133:0/3450504517 conn(0x7fae500999d0 msgr2=0x7fae500d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 -- 172.21.15.133:0/3450504517 shutdown_connections 2024-04-25T10:03:14.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.784+0000 7fae5693e640 1 -- 172.21.15.133:0/3450504517 wait complete. 2024-04-25T10:03:14.809 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:14.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.916+0000 7f432e895640 1 Processor -- start 2024-04-25T10:03:14.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.916+0000 7f432e895640 1 -- start start 2024-04-25T10:03:14.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f432e895640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328099d10 0x7f43280cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f432e895640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43280cd830 con 0x7f432815f560 2024-04-25T10:03:14.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f432e895640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43280d0490 con 0x7f43280b08b0 2024-04-25T10:03:14.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f432e895640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43280d0240 con 0x7f4328099d10 2024-04-25T10:03:14.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f432ce32640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f432815f560 0x7f43280b16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50360/0 (socket says 172.21.15.133:50360) 2024-04-25T10:03:14.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f432ce32640 1 -- 172.21.15.133:0/1377466126 learned_addr learned my addr 172.21.15.133:0/1377466126 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:14.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4327fff640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328099d10 0x7f43280cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542053253 0 0) 0x7f43280cd830 con 0x7f432815f560 2024-04-25T10:03:14.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4310003530 con 0x7f432815f560 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1276299299 0 0) 0x7f4310003530 con 0x7f432815f560 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328099d10 msgr2=0x7f43280cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328099d10 0x7f43280cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 >> v1:172.21.15.162:6789/0 conn(0x7f43280b08b0 legacy=0x7f432815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280d0770 con 0x7f432815f560 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4320001a00 con 0x7f432815f560 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 >> v1:172.21.15.133:6789/0 conn(0x7f432815f560 legacy=0x7f43280b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43100039f0 0x7f4310003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328099d10 0x7f4310004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4310004d50 0x7f43100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43280d0240 con 0x7f43100039f0 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431000bbb0 con 0x7f4310004d50 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431000bdb0 con 0x7f4328099d10 2024-04-25T10:03:14.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4327fff640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43100039f0 0x7f4310003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f43277fe640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328099d10 0x7f4310004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f432ce32640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4310004d50 0x7f43100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4327fff640 1 -- 172.21.15.133:0/1377466126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328099d10 msgr2=0x7f4310004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4327fff640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328099d10 0x7f4310004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4327fff640 1 -- 172.21.15.133:0/1377466126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4310004d50 msgr2=0x7f43100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4327fff640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4310004d50 0x7f43100051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4327fff640 1 -- 172.21.15.133:0/1377466126 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310003530 con 0x7f43100039f0 2024-04-25T10:03:14.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4327fff640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43100039f0 0x7f4310003e10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f431800ba30 tx=0x7f431801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31466a8b470ecbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43180b3070 con 0x7f43100039f0 2024-04-25T10:03:14.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43180ae630 con 0x7f43100039f0 2024-04-25T10:03:14.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.920+0000 7f4326ffd640 1 -- 172.21.15.133:0/1377466126 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43180aa050 con 0x7f43100039f0 2024-04-25T10:03:14.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 -- 172.21.15.133:0/1377466126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43100039f0 msgr2=0x7f4310003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 --2- 172.21.15.133:0/1377466126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43100039f0 0x7f4310003e10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f431800ba30 tx=0x7f431801c6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432ce32640 1 -- 172.21.15.133:0/1377466126 reap_dead start 2024-04-25T10:03:14.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 -- 172.21.15.133:0/1377466126 shutdown_connections 2024-04-25T10:03:14.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 -- 172.21.15.133:0/1377466126 >> 172.21.15.133:0/1377466126 conn(0x7f43280b3420 msgr2=0x7f43280b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 -- 172.21.15.133:0/1377466126 shutdown_connections 2024-04-25T10:03:14.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 -- 172.21.15.133:0/1377466126 wait complete. 2024-04-25T10:03:14.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 Processor -- start 2024-04-25T10:03:14.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 -- start start 2024-04-25T10:03:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328167230 0x7f432814c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43280b5800 0x7f43280b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43280b5c80 0x7f43280b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43280b29d0 con 0x7f43280b5800 2024-04-25T10:03:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43280b2850 con 0x7f43280b5c80 2024-04-25T10:03:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4328167a70 con 0x7f4328167230 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f43277fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43280b5800 0x7f43280b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f43277fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43280b5800 0x7f43280b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57648/0 (socket says 172.21.15.133:57648) 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f43277fe640 1 -- 172.21.15.133:0/3467959330 learned_addr learned my addr 172.21.15.133:0/3467959330 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432ce32640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43280b5c80 0x7f43280b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f4327fff640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328167230 0x7f432814c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f43277fe640 1 -- 172.21.15.133:0/3467959330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328167230 msgr2=0x7f432814c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f43277fe640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328167230 0x7f432814c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f43277fe640 1 -- 172.21.15.133:0/3467959330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43280b5c80 msgr2=0x7f43280b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f43277fe640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43280b5c80 0x7f43280b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f43277fe640 1 -- 172.21.15.133:0/3467959330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432809b9f0 con 0x7f43280b5800 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f43277fe640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43280b5800 0x7f43280b6820 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f431c00d5a0 tx=0x7f431c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f4324ff9640 1 -- 172.21.15.133:0/3467959330 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c005480 con 0x7f43280b5800 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f4324ff9640 1 -- 172.21.15.133:0/3467959330 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c017070 con 0x7f43280b5800 2024-04-25T10:03:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 -- 172.21.15.133:0/3467959330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432809bcd0 con 0x7f43280b5800 2024-04-25T10:03:14.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f4324ff9640 1 -- 172.21.15.133:0/3467959330 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c003820 con 0x7f43280b5800 2024-04-25T10:03:14.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.924+0000 7f432e895640 1 -- 172.21.15.133:0/3467959330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432809c290 con 0x7f43280b5800 2024-04-25T10:03:14.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.928+0000 7f432e895640 1 -- 172.21.15.133:0/3467959330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4328067c70 con 0x7f43280b5800 2024-04-25T10:03:14.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.928+0000 7f4324ff9640 1 -- 172.21.15.133:0/3467959330 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f431c0039c0 con 0x7f43280b5800 2024-04-25T10:03:14.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.928+0000 7f4324ff9640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4314066f90 0x7f4314069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.928+0000 7f4327fff640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4314066f90 0x7f4314069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.928+0000 7f4324ff9640 1 -- 172.21.15.133:0/3467959330 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f431c084960 con 0x7f43280b5800 2024-04-25T10:03:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.928+0000 7f4327fff640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4314066f90 0x7f4314069450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f43180b0000 tx=0x7f43180046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:14.932+0000 7f4324ff9640 1 -- 172.21.15.133:0/3467959330 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f431c021020 con 0x7f43280b5800 2024-04-25T10:03:15.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.088+0000 7f432e895640 1 -- 172.21.15.133:0/3467959330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43280aef80 con 0x7f43280b5800 2024-04-25T10:03:15.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.088+0000 7f4324ff9640 1 -- 172.21.15.133:0/3467959330 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13188 (secure 0 0 0) 0x7f431c019310 con 0x7f43280b5800 2024-04-25T10:03:15.097 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:15.097 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":34,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:03:12.322895+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:15.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 -- 172.21.15.133:0/3467959330 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4314066f90 msgr2=0x7f4314069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4314066f90 0x7f4314069450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f43180b0000 tx=0x7f43180046d0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 -- 172.21.15.133:0/3467959330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43280b5800 msgr2=0x7f43280b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43280b5800 0x7f43280b6820 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f431c00d5a0 tx=0x7f431c00da60 comp rx=0 tx=0).stop 2024-04-25T10:03:15.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 -- 172.21.15.133:0/3467959330 shutdown_connections 2024-04-25T10:03:15.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43280b5c80 0x7f43280b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4314066f90 0x7f4314069450 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43280b5800 0x7f43280b6820 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 --2- 172.21.15.133:0/3467959330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4328167230 0x7f432814c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 -- 172.21.15.133:0/3467959330 >> 172.21.15.133:0/3467959330 conn(0x7f43280b3420 msgr2=0x7f43280cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 -- 172.21.15.133:0/3467959330 shutdown_connections 2024-04-25T10:03:15.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.092+0000 7f432e895640 1 -- 172.21.15.133:0/3467959330 wait complete. 2024-04-25T10:03:15.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:15.116 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:15.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faf01ec8640 1 Processor -- start 2024-04-25T10:03:15.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faf01ec8640 1 -- start start 2024-04-25T10:03:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faf01ec8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faefc0cd820 0x7faefc0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faf01ec8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faefc15ced0 0x7faefc15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faf01ec8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc0ce2e0 con 0x7faefc145170 2024-04-25T10:03:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faf01ec8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faefc06db00 con 0x7faefc15ced0 2024-04-25T10:03:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faf01ec8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc145550 con 0x7faefc0cd820 2024-04-25T10:03:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faefbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faefc15ced0 0x7faefc15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faefb7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7faefc145170 0x7faefc06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50364/0 (socket says 172.21.15.133:50364) 2024-04-25T10:03:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faefbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faefc15ced0 0x7faefc15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57664/0 (socket says 172.21.15.133:57664) 2024-04-25T10:03:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faefb7fe640 1 -- 172.21.15.133:0/85273944 learned_addr learned my addr 172.21.15.133:0/85273944 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:15.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faefaffd640 1 --2- 172.21.15.133:0/85273944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faefc0cd820 0x7faefc0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faefbfff640 1 -- 172.21.15.133:0/85273944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faefc0cd820 msgr2=0x7faefc0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faefbfff640 1 --2- 172.21.15.133:0/85273944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faefc0cd820 0x7faefc0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faefbfff640 1 -- 172.21.15.133:0/85273944 >> v1:172.21.15.133:6789/0 conn(0x7faefc145170 legacy=0x7faefc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.232+0000 7faefbfff640 1 -- 172.21.15.133:0/85273944 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc06d4b0 con 0x7faefc15ced0 2024-04-25T10:03:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefbfff640 1 --2- 172.21.15.133:0/85273944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc15ced0 0x7faefc15d360 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faef4009d90 tx=0x7faef401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819884ae81fdf66b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefa7fc640 1 -- 172.21.15.133:0/85273944 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef40b2070 con 0x7faefc15ced0 2024-04-25T10:03:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefa7fc640 1 -- 172.21.15.133:0/85273944 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef40a5830 con 0x7faefc15ced0 2024-04-25T10:03:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefa7fc640 1 -- 172.21.15.133:0/85273944 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef40a5c00 con 0x7faefc15ced0 2024-04-25T10:03:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 -- 172.21.15.133:0/85273944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc15ced0 msgr2=0x7faefc15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 --2- 172.21.15.133:0/85273944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc15ced0 0x7faefc15d360 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faef4009d90 tx=0x7faef401c970 comp rx=0 tx=0).stop 2024-04-25T10:03:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 -- 172.21.15.133:0/85273944 shutdown_connections 2024-04-25T10:03:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 --2- 172.21.15.133:0/85273944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc15ced0 0x7faefc15d360 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 --2- 172.21.15.133:0/85273944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faefc0cd820 0x7faefc0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 -- 172.21.15.133:0/85273944 >> 172.21.15.133:0/85273944 conn(0x7faefc066f70 msgr2=0x7faefc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 -- 172.21.15.133:0/85273944 shutdown_connections 2024-04-25T10:03:15.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 -- 172.21.15.133:0/85273944 wait complete. 2024-04-25T10:03:15.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 Processor -- start 2024-04-25T10:03:15.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 -- start start 2024-04-25T10:03:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faefc0cd820 0x7faefc164e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc145170 0x7faefc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc15ced0 0x7faefc13ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc168610 con 0x7faefc145170 2024-04-25T10:03:15.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc168490 con 0x7faefc15ced0 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faf01ec8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc168790 con 0x7faefc0cd820 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc145170 0x7faefc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc145170 0x7faefc13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57676/0 (socket says 172.21.15.133:57676) 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefaffd640 1 -- 172.21.15.133:0/232125885 learned_addr learned my addr 172.21.15.133:0/232125885 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefbfff640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc15ced0 0x7faefc13ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefb7fe640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faefc0cd820 0x7faefc164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefaffd640 1 -- 172.21.15.133:0/232125885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faefc0cd820 msgr2=0x7faefc164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefaffd640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faefc0cd820 0x7faefc164e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefaffd640 1 -- 172.21.15.133:0/232125885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc15ced0 msgr2=0x7faefc13ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefaffd640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc15ced0 0x7faefc13ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefaffd640 1 -- 172.21.15.133:0/232125885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc0b12b0 con 0x7faefc145170 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.236+0000 7faefaffd640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc145170 0x7faefc13b550 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faef000d830 tx=0x7faef000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.240+0000 7faee3fff640 1 -- 172.21.15.133:0/232125885 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000b6c0 con 0x7faefc145170 2024-04-25T10:03:15.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.240+0000 7faee3fff640 1 -- 172.21.15.133:0/232125885 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef0019070 con 0x7faefc145170 2024-04-25T10:03:15.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.240+0000 7faf01ec8640 1 -- 172.21.15.133:0/232125885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc0af650 con 0x7faefc145170 2024-04-25T10:03:15.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.240+0000 7faee3fff640 1 -- 172.21.15.133:0/232125885 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000ba60 con 0x7faefc145170 2024-04-25T10:03:15.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.240+0000 7faf01ec8640 1 -- 172.21.15.133:0/232125885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc0afc10 con 0x7faefc145170 2024-04-25T10:03:15.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.240+0000 7faf01ec8640 1 -- 172.21.15.133:0/232125885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faefc068e80 con 0x7faefc145170 2024-04-25T10:03:15.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.240+0000 7faee3fff640 1 -- 172.21.15.133:0/232125885 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faef001e460 con 0x7faefc145170 2024-04-25T10:03:15.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.240+0000 7faee3fff640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faee8066f60 0x7faee8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.240+0000 7faee3fff640 1 -- 172.21.15.133:0/232125885 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7faef00860d0 con 0x7faefc145170 2024-04-25T10:03:15.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.240+0000 7faefb7fe640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faee8066f60 0x7faee8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.240+0000 7faefb7fe640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faee8066f60 0x7faee8069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faeec000f70 tx=0x7faeec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.244+0000 7faee3fff640 1 -- 172.21.15.133:0/232125885 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faef00572a0 con 0x7faefc145170 2024-04-25T10:03:15.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.396+0000 7faf01ec8640 1 -- 172.21.15.133:0/232125885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faefc15de30 con 0x7faefc145170 2024-04-25T10:03:15.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faee3fff640 1 -- 172.21.15.133:0/232125885 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7faef0056ba0 con 0x7faefc145170 2024-04-25T10:03:15.405 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 -- 172.21.15.133:0/232125885 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faee8066f60 msgr2=0x7faee8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faee8066f60 0x7faee8069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faeec000f70 tx=0x7faeec009040 comp rx=0 tx=0).stop 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 -- 172.21.15.133:0/232125885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc145170 msgr2=0x7faefc13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc145170 0x7faefc13b550 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faef000d830 tx=0x7faef000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 -- 172.21.15.133:0/232125885 shutdown_connections 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faefc15ced0 0x7faefc13ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faee8066f60 0x7faee8069420 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc145170 0x7faefc13b550 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 --2- 172.21.15.133:0/232125885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faefc0cd820 0x7faefc164e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 -- 172.21.15.133:0/232125885 >> 172.21.15.133:0/232125885 conn(0x7faefc066f70 msgr2=0x7faefc15e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 -- 172.21.15.133:0/232125885 shutdown_connections 2024-04-25T10:03:15.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.400+0000 7faf01ec8640 1 -- 172.21.15.133:0/232125885 wait complete. 2024-04-25T10:03:15.423 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:03:15.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f371381e640 1 Processor -- start 2024-04-25T10:03:15.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f371381e640 1 -- start start 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f371381e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f370c0b1f70 0x7f370c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f371381e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f370c0ce330 0x7f370c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f371381e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c0c61d0 con 0x7f370c15c9a0 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f371381e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c0cece0 con 0x7f370c0ce330 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f371381e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0c64b0 con 0x7f370c0b1f70 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f3711dbb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f370c0ce330 0x7f370c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f3711dbb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f370c0ce330 0x7f370c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57682/0 (socket says 172.21.15.133:57682) 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f3711dbb640 1 -- 172.21.15.133:0/2813175803 learned_addr learned my addr 172.21.15.133:0/2813175803 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f3710db9640 1 --2- 172.21.15.133:0/2813175803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f370c0b1f70 0x7f370c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f3711dbb640 1 -- 172.21.15.133:0/2813175803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f370c0b1f70 msgr2=0x7f370c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f3711dbb640 1 --2- 172.21.15.133:0/2813175803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f370c0b1f70 0x7f370c06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f3711dbb640 1 -- 172.21.15.133:0/2813175803 >> v1:172.21.15.162:6789/0 conn(0x7f370c15c9a0 legacy=0x7f370c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.524+0000 7f3711dbb640 1 -- 172.21.15.133:0/2813175803 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c06d4f0 con 0x7f370c0ce330 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f3711dbb640 1 --2- 172.21.15.133:0/2813175803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c0ce330 0x7f370c0ce7a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3704009e50 tx=0x7f370401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6368475f98ac2e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f3703fff640 1 -- 172.21.15.133:0/2813175803 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040a5750 con 0x7f370c0ce330 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f3703fff640 1 -- 172.21.15.133:0/2813175803 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37040a58f0 con 0x7f370c0ce330 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f3703fff640 1 -- 172.21.15.133:0/2813175803 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040a5cc0 con 0x7f370c0ce330 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 -- 172.21.15.133:0/2813175803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c0ce330 msgr2=0x7f370c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 --2- 172.21.15.133:0/2813175803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c0ce330 0x7f370c0ce7a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3704009e50 tx=0x7f370401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 -- 172.21.15.133:0/2813175803 shutdown_connections 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 --2- 172.21.15.133:0/2813175803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c0ce330 0x7f370c0ce7a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 --2- 172.21.15.133:0/2813175803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f370c0b1f70 0x7f370c06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 -- 172.21.15.133:0/2813175803 >> 172.21.15.133:0/2813175803 conn(0x7f370c067000 msgr2=0x7f370c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 -- 172.21.15.133:0/2813175803 shutdown_connections 2024-04-25T10:03:15.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 -- 172.21.15.133:0/2813175803 wait complete. 2024-04-25T10:03:15.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 Processor -- start 2024-04-25T10:03:15.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 -- start start 2024-04-25T10:03:15.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c0b1f70 0x7f370c165400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f370c0ce330 0x7f370c148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c15c9a0 0x7f370c163910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c15dad0 con 0x7f370c0b1f70 2024-04-25T10:03:15.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c15d950 con 0x7f370c15c9a0 2024-04-25T10:03:15.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c15dc50 con 0x7f370c0ce330 2024-04-25T10:03:15.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f37115ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c0b1f70 0x7f370c165400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f37115ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c0b1f70 0x7f370c165400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57696/0 (socket says 172.21.15.133:57696) 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f37115ba640 1 -- 172.21.15.133:0/1466745266 learned_addr learned my addr 172.21.15.133:0/1466745266 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f3710db9640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f370c0ce330 0x7f370c148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f3711dbb640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c15c9a0 0x7f370c163910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f37115ba640 1 -- 172.21.15.133:0/1466745266 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f370c0ce330 msgr2=0x7f370c148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f37115ba640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f370c0ce330 0x7f370c148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f37115ba640 1 -- 172.21.15.133:0/1466745266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c15c9a0 msgr2=0x7f370c163910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f37115ba640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c15c9a0 0x7f370c163910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f37115ba640 1 -- 172.21.15.133:0/1466745266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c165940 con 0x7f370c0b1f70 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f3710db9640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f370c0ce330 0x7f370c148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f3711dbb640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c15c9a0 0x7f370c163910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f37115ba640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c0b1f70 0x7f370c165400 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f370800c450 tx=0x7f370800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f3701ffb640 1 -- 172.21.15.133:0/1466745266 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708012830 con 0x7f370c0b1f70 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f3701ffb640 1 -- 172.21.15.133:0/1466745266 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3708017070 con 0x7f370c0b1f70 2024-04-25T10:03:15.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f3701ffb640 1 -- 172.21.15.133:0/1466745266 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708012c30 con 0x7f370c0b1f70 2024-04-25T10:03:15.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 -- 172.21.15.133:0/1466745266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c164360 con 0x7f370c0b1f70 2024-04-25T10:03:15.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.528+0000 7f371381e640 1 -- 172.21.15.133:0/1466745266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c164920 con 0x7f370c0b1f70 2024-04-25T10:03:15.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.532+0000 7f371381e640 1 -- 172.21.15.133:0/1466745266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c068ef0 con 0x7f370c0b1f70 2024-04-25T10:03:15.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.532+0000 7f3701ffb640 1 -- 172.21.15.133:0/1466745266 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f370801c460 con 0x7f370c0b1f70 2024-04-25T10:03:15.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.532+0000 7f3701ffb640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f36f0066f60 0x7f36f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.532+0000 7f3701ffb640 1 -- 172.21.15.133:0/1466745266 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3708084c80 con 0x7f370c0b1f70 2024-04-25T10:03:15.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.532+0000 7f3710db9640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f36f0066f60 0x7f36f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.532+0000 7f3710db9640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f36f0066f60 0x7f36f0069420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f36fc006fd0 tx=0x7f36fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.536+0000 7f3701ffb640 1 -- 172.21.15.133:0/1466745266 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3708055db0 con 0x7f370c0b1f70 2024-04-25T10:03:15.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 -- 172.21.15.133:0/1466745266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f370c06b630 con 0x7f370c0b1f70 2024-04-25T10:03:15.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f3701ffb640 1 -- 172.21.15.133:0/1466745266 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+65 (secure 0 0 0) 0x7f37080556b0 con 0x7f370c0b1f70 2024-04-25T10:03:15.690 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:15.690 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:03:15.690 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:03:15.690 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:03:15.690 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:03:15.690 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:15.690 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:03:15.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 -- 172.21.15.133:0/1466745266 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f36f0066f60 msgr2=0x7f36f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f36f0066f60 0x7f36f0069420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f36fc006fd0 tx=0x7f36fc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:15.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 -- 172.21.15.133:0/1466745266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c0b1f70 msgr2=0x7f370c165400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c0b1f70 0x7f370c165400 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f370800c450 tx=0x7f370800c910 comp rx=0 tx=0).stop 2024-04-25T10:03:15.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 -- 172.21.15.133:0/1466745266 shutdown_connections 2024-04-25T10:03:15.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370c15c9a0 0x7f370c163910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f370c0ce330 0x7f370c148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f36f0066f60 0x7f36f0069420 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 --2- 172.21.15.133:0/1466745266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f370c0b1f70 0x7f370c165400 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 -- 172.21.15.133:0/1466745266 >> 172.21.15.133:0/1466745266 conn(0x7f370c067000 msgr2=0x7f370c15a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 -- 172.21.15.133:0/1466745266 shutdown_connections 2024-04-25T10:03:15.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.684+0000 7f371381e640 1 -- 172.21.15.133:0/1466745266 wait complete. 2024-04-25T10:03:15.707 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:03:15.707 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:03:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d59cff640 1 Processor -- start 2024-04-25T10:03:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d59cff640 1 -- start start 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d59cff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d5415aac0 0x7f6d540ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d59cff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6d540d01f0 0x7f6d540d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d59cff640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d540d0bc0 con 0x7f6d540b0220 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d59cff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5415b7a0 con 0x7f6d540d01f0 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d59cff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5415ba80 con 0x7f6d5415aac0 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d52ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6d540b0220 0x7f6d5415aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50366/0 (socket says 172.21.15.133:50366) 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d52ffd640 1 -- 172.21.15.133:0/2705294403 learned_addr learned my addr 172.21.15.133:0/2705294403 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d53fff640 1 --2- 172.21.15.133:0/2705294403 >> v2:172.21.15.162:3300/0 conn(0x7f6d540d01f0 0x7f6d540d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d537fe640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d5415aac0 0x7f6d540ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488978254 0 0) 0x7f6d540d0bc0 con 0x7f6d540b0220 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d3c003530 con 0x7f6d540b0220 2024-04-25T10:03:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2902659986 0 0) 0x7f6d3c003530 con 0x7f6d540b0220 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d5415aac0 msgr2=0x7f6d540ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d5415aac0 0x7f6d540ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 >> v2:172.21.15.162:3300/0 conn(0x7f6d540d01f0 msgr2=0x7f6d540d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 --2- 172.21.15.133:0/2705294403 >> v2:172.21.15.162:3300/0 conn(0x7f6d540d01f0 0x7f6d540d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5415bd60 con 0x7f6d540b0220 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6d4c001680 con 0x7f6d540b0220 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 >> v1:172.21.15.133:6789/0 conn(0x7f6d540b0220 legacy=0x7f6d5415aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d5415aac0 0x7f6d3c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d3c003f40 0x7f6d3c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d3c004d50 0x7f6d3c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5415b7a0 con 0x7f6d3c004d50 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5415ba80 con 0x7f6d3c003f40 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c00ba20 con 0x7f6d5415aac0 2024-04-25T10:03:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d53fff640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d3c004d50 0x7f6d3c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d537fe640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d5415aac0 0x7f6d3c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d52ffd640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d3c003f40 0x7f6d3c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d53fff640 1 -- 172.21.15.133:0/2705294403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d5415aac0 msgr2=0x7f6d3c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d53fff640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d5415aac0 0x7f6d3c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d53fff640 1 -- 172.21.15.133:0/2705294403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d3c003f40 msgr2=0x7f6d3c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d53fff640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d3c003f40 0x7f6d3c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.824+0000 7f6d53fff640 1 -- 172.21.15.133:0/2705294403 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c003530 con 0x7f6d3c004d50 2024-04-25T10:03:15.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.828+0000 7f6d53fff640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d3c004d50 0x7f6d3c0051d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009b10 tx=0x7f6d4801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156769671ad48461 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.828+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d480b2070 con 0x7f6d3c004d50 2024-04-25T10:03:15.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.828+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d480a56a0 con 0x7f6d3c004d50 2024-04-25T10:03:15.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.828+0000 7f6d527fc640 1 -- 172.21.15.133:0/2705294403 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d480096b0 con 0x7f6d3c004d50 2024-04-25T10:03:15.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.828+0000 7f6d59cff640 1 -- 172.21.15.133:0/2705294403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d3c004d50 msgr2=0x7f6d3c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.828+0000 7f6d59cff640 1 --2- 172.21.15.133:0/2705294403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d3c004d50 0x7f6d3c0051d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009b10 tx=0x7f6d4801cab0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.828+0000 7f6d53fff640 1 -- 172.21.15.133:0/2705294403 reap_dead start 2024-04-25T10:03:15.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.828+0000 7f6d59cff640 1 -- 172.21.15.133:0/2705294403 shutdown_connections 2024-04-25T10:03:15.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.828+0000 7f6d59cff640 1 -- 172.21.15.133:0/2705294403 >> 172.21.15.133:0/2705294403 conn(0x7f6d540726f0 msgr2=0x7f6d540999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.828+0000 7f6d59cff640 1 -- 172.21.15.133:0/2705294403 shutdown_connections 2024-04-25T10:03:15.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.828+0000 7f6d59cff640 1 -- 172.21.15.133:0/2705294403 wait complete. 2024-04-25T10:03:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d59cff640 1 Processor -- start 2024-04-25T10:03:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d59cff640 1 -- start start 2024-04-25T10:03:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d59cff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d540ccaa0 0x7f6d540b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d59cff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d540cce80 0x7f6d540b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d59cff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d54158960 0x7f6d54158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d59cff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d540ce5a0 con 0x7f6d540cce80 2024-04-25T10:03:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d59cff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5415c040 con 0x7f6d540ccaa0 2024-04-25T10:03:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d59cff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d540ce720 con 0x7f6d54158960 2024-04-25T10:03:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d52ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d540cce80 0x7f6d540b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d52ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d540cce80 0x7f6d540b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57710/0 (socket says 172.21.15.133:57710) 2024-04-25T10:03:15.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d52ffd640 1 -- 172.21.15.133:0/3601663443 learned_addr learned my addr 172.21.15.133:0/3601663443 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d537fe640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d540ccaa0 0x7f6d540b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d53fff640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d54158960 0x7f6d54158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d52ffd640 1 -- 172.21.15.133:0/3601663443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d54158960 msgr2=0x7f6d54158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d52ffd640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d54158960 0x7f6d54158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d52ffd640 1 -- 172.21.15.133:0/3601663443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d540ccaa0 msgr2=0x7f6d540b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d52ffd640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d540ccaa0 0x7f6d540b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d52ffd640 1 -- 172.21.15.133:0/3601663443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d54159300 con 0x7f6d540cce80 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d53fff640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d54158960 0x7f6d54158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d52ffd640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d540cce80 0x7f6d540b4670 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c011820 tx=0x7f6d4c011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d3bfff640 1 -- 172.21.15.133:0/3601663443 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c010040 con 0x7f6d540cce80 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d3bfff640 1 -- 172.21.15.133:0/3601663443 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c01a070 con 0x7f6d540cce80 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d59cff640 1 -- 172.21.15.133:0/3601663443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d540b4cb0 con 0x7f6d540cce80 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d3bfff640 1 -- 172.21.15.133:0/3601663443 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c00fde0 con 0x7f6d540cce80 2024-04-25T10:03:15.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d59cff640 1 -- 172.21.15.133:0/3601663443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d540b5270 con 0x7f6d540cce80 2024-04-25T10:03:15.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d59cff640 1 -- 172.21.15.133:0/3601663443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d54069000 con 0x7f6d540cce80 2024-04-25T10:03:15.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d3bfff640 1 -- 172.21.15.133:0/3601663443 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d4c01f460 con 0x7f6d540cce80 2024-04-25T10:03:15.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d3bfff640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d40066f30 0x7f6d400693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d3bfff640 1 -- 172.21.15.133:0/3601663443 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6d4c088140 con 0x7f6d540cce80 2024-04-25T10:03:15.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d537fe640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d40066f30 0x7f6d400693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.832+0000 7f6d537fe640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d40066f30 0x7f6d400693f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6d44007380 tx=0x7f6d4400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.836+0000 7f6d3bfff640 1 -- 172.21.15.133:0/3601663443 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d4c0593a0 con 0x7f6d540cce80 2024-04-25T10:03:15.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:15.992+0000 7f6d59cff640 1 -- 172.21.15.133:0/3601663443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6d540cf8b0 con 0x7f6d540cce80 2024-04-25T10:03:16.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.388+0000 7f6d3bfff640 1 -- 172.21.15.133:0/3601663443 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6d4c058ca0 con 0x7f6d540cce80 2024-04-25T10:03:16.396 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:03:16.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 -- 172.21.15.133:0/3601663443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d40066f30 msgr2=0x7f6d400693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d40066f30 0x7f6d400693f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6d44007380 tx=0x7f6d4400b040 comp rx=0 tx=0).stop 2024-04-25T10:03:16.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 -- 172.21.15.133:0/3601663443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d540cce80 msgr2=0x7f6d540b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d540cce80 0x7f6d540b4670 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c011820 tx=0x7f6d4c011ce0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 -- 172.21.15.133:0/3601663443 shutdown_connections 2024-04-25T10:03:16.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6d54158960 0x7f6d54158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d40066f30 0x7f6d400693f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d540cce80 0x7f6d540b4670 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 --2- 172.21.15.133:0/3601663443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6d540ccaa0 0x7f6d540b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 -- 172.21.15.133:0/3601663443 >> 172.21.15.133:0/3601663443 conn(0x7f6d540726f0 msgr2=0x7f6d540ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 -- 172.21.15.133:0/3601663443 shutdown_connections 2024-04-25T10:03:16.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.392+0000 7f6d59cff640 1 -- 172.21.15.133:0/3601663443 wait complete. 2024-04-25T10:03:16.428 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:03:16.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.544+0000 7f57c6374640 1 Processor -- start 2024-04-25T10:03:16.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.544+0000 7f57c6374640 1 -- start start 2024-04-25T10:03:16.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.544+0000 7f57c6374640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f57c0145170 0x7f57c006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57c015ced0 0x7f57c015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c00ce2e0 con 0x7f57c006e4d0 2024-04-25T10:03:16.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c006db00 con 0x7f57c015ced0 2024-04-25T10:03:16.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c0145550 con 0x7f57c0145170 2024-04-25T10:03:16.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c4911640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57c015ced0 0x7f57c015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c4911640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57c015ced0 0x7f57c015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60560/0 (socket says 172.21.15.133:60560) 2024-04-25T10:03:16.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c4911640 1 -- 172.21.15.133:0/1093761427 learned_addr learned my addr 172.21.15.133:0/1093761427 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:16.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57bffff640 1 --2- 172.21.15.133:0/1093761427 >> v2:172.21.15.162:3300/0 conn(0x7f57c0145170 0x7f57c006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c4911640 1 -- 172.21.15.133:0/1093761427 >> v2:172.21.15.162:3300/0 conn(0x7f57c0145170 msgr2=0x7f57c006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c4911640 1 --2- 172.21.15.133:0/1093761427 >> v2:172.21.15.162:3300/0 conn(0x7f57c0145170 0x7f57c006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c4911640 1 -- 172.21.15.133:0/1093761427 >> v1:172.21.15.162:6789/0 conn(0x7f57c006e4d0 legacy=0x7f57c00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c4911640 1 -- 172.21.15.133:0/1093761427 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c006d4b0 con 0x7f57c015ced0 2024-04-25T10:03:16.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c4911640 1 --2- 172.21.15.133:0/1093761427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c015ced0 0x7f57c015d320 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f57c006b790 tx=0x7f57b801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9320159df28e14b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57beffd640 1 -- 172.21.15.133:0/1093761427 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80a5670 con 0x7f57c015ced0 2024-04-25T10:03:16.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57beffd640 1 -- 172.21.15.133:0/1093761427 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b80a5810 con 0x7f57c015ced0 2024-04-25T10:03:16.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57beffd640 1 -- 172.21.15.133:0/1093761427 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80a5be0 con 0x7f57c015ced0 2024-04-25T10:03:16.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 -- 172.21.15.133:0/1093761427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c015ced0 msgr2=0x7f57c015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 --2- 172.21.15.133:0/1093761427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c015ced0 0x7f57c015d320 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f57c006b790 tx=0x7f57b801c9b0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 -- 172.21.15.133:0/1093761427 shutdown_connections 2024-04-25T10:03:16.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 --2- 172.21.15.133:0/1093761427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c015ced0 0x7f57c015d320 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 --2- 172.21.15.133:0/1093761427 >> v2:172.21.15.162:3300/0 conn(0x7f57c0145170 0x7f57c006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 -- 172.21.15.133:0/1093761427 >> 172.21.15.133:0/1093761427 conn(0x7f57c0066f70 msgr2=0x7f57c006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 -- 172.21.15.133:0/1093761427 shutdown_connections 2024-04-25T10:03:16.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.548+0000 7f57c6374640 1 -- 172.21.15.133:0/1093761427 wait complete. 2024-04-25T10:03:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c6374640 1 Processor -- start 2024-04-25T10:03:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c6374640 1 -- start start 2024-04-25T10:03:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c6374640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c006e4d0 0x7f57c0152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c6374640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57c0145170 0x7f57c015a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c6374640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c015ced0 0x7f57c015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c6374640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c0168610 con 0x7f57c015ced0 2024-04-25T10:03:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c6374640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c0168490 con 0x7f57c0145170 2024-04-25T10:03:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c6374640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c0168790 con 0x7f57c006e4d0 2024-04-25T10:03:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c4911640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c015ced0 0x7f57c015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c4911640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c015ced0 0x7f57c015ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60562/0 (socket says 172.21.15.133:60562) 2024-04-25T10:03:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c4911640 1 -- 172.21.15.133:0/2408212503 learned_addr learned my addr 172.21.15.133:0/2408212503 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57bffff640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c006e4d0 0x7f57c0152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57bf7fe640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57c0145170 0x7f57c015a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c4911640 1 -- 172.21.15.133:0/2408212503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c006e4d0 msgr2=0x7f57c0152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c4911640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c006e4d0 0x7f57c0152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c4911640 1 -- 172.21.15.133:0/2408212503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57c0145170 msgr2=0x7f57c015a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c4911640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57c0145170 0x7f57c015a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c4911640 1 -- 172.21.15.133:0/2408212503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c0155ae0 con 0x7f57c015ced0 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c4911640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c015ced0 0x7f57c015ad00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f57b801a5e0 tx=0x7f57b8019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57bcff9640 1 -- 172.21.15.133:0/2408212503 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8009510 con 0x7f57c015ced0 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57bcff9640 1 -- 172.21.15.133:0/2408212503 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b80be070 con 0x7f57c015ced0 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c6374640 1 -- 172.21.15.133:0/2408212503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c0155dc0 con 0x7f57c015ced0 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57bcff9640 1 -- 172.21.15.133:0/2408212503 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80b88b0 con 0x7f57c015ced0 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c6374640 1 -- 172.21.15.133:0/2408212503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c00af550 con 0x7f57c015ced0 2024-04-25T10:03:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.552+0000 7f57c6374640 1 -- 172.21.15.133:0/2408212503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c0068e80 con 0x7f57c015ced0 2024-04-25T10:03:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.556+0000 7f57bcff9640 1 -- 172.21.15.133:0/2408212503 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57b80b8a50 con 0x7f57c015ced0 2024-04-25T10:03:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.556+0000 7f57bcff9640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f57ac066f90 0x7f57ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.556+0000 7f57bcff9640 1 -- 172.21.15.133:0/2408212503 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f57b81206d0 con 0x7f57c015ced0 2024-04-25T10:03:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.556+0000 7f57bffff640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f57ac066f90 0x7f57ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.556+0000 7f57bffff640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f57ac066f90 0x7f57ac069450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f57b000b930 tx=0x7f57b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.560+0000 7f57bcff9640 1 -- 172.21.15.133:0/2408212503 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f57b80165a0 con 0x7f57c015ced0 2024-04-25T10:03:16.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:16.716+0000 7f57c6374640 1 -- 172.21.15.133:0/2408212503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f57c00b2c20 con 0x7f57c015ced0 2024-04-25T10:03:17.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.400+0000 7f57bcff9640 1 -- 172.21.15.133:0/2408212503 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7f57b80f1810 con 0x7f57c015ced0 2024-04-25T10:03:17.409 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:03:17.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.404+0000 7f57c6374640 1 -- 172.21.15.133:0/2408212503 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f57ac066f90 msgr2=0x7f57ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.404+0000 7f57c6374640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f57ac066f90 0x7f57ac069450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f57b000b930 tx=0x7f57b0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:17.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.404+0000 7f57c6374640 1 -- 172.21.15.133:0/2408212503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c015ced0 msgr2=0x7f57c015ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.404+0000 7f57c6374640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c015ced0 0x7f57c015ad00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f57b801a5e0 tx=0x7f57b8019750 comp rx=0 tx=0).stop 2024-04-25T10:03:17.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.408+0000 7f57c6374640 1 -- 172.21.15.133:0/2408212503 shutdown_connections 2024-04-25T10:03:17.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.408+0000 7f57c6374640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f57ac066f90 0x7f57ac069450 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.408+0000 7f57c6374640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c015ced0 0x7f57c015ad00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.408+0000 7f57c6374640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57c0145170 0x7f57c015a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.408+0000 7f57c6374640 1 --2- 172.21.15.133:0/2408212503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57c006e4d0 0x7f57c0152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.408+0000 7f57c6374640 1 -- 172.21.15.133:0/2408212503 >> 172.21.15.133:0/2408212503 conn(0x7f57c0066f70 msgr2=0x7f57c015d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.408+0000 7f57c6374640 1 -- 172.21.15.133:0/2408212503 shutdown_connections 2024-04-25T10:03:17.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.408+0000 7f57c6374640 1 -- 172.21.15.133:0/2408212503 wait complete. 2024-04-25T10:03:17.445 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:03:17.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2cda86640 1 Processor -- start 2024-04-25T10:03:17.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2cda86640 1 -- start start 2024-04-25T10:03:17.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2cda86640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd2c8145170 0x7fd2c806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2cda86640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2c815ced0 0x7fd2c815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2cda86640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c80ce2e0 con 0x7fd2c806e4d0 2024-04-25T10:03:17.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2cda86640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c806db00 con 0x7fd2c815ced0 2024-04-25T10:03:17.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2cda86640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c8145550 con 0x7fd2c8145170 2024-04-25T10:03:17.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2c7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2c815ced0 0x7fd2c815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2c6ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd2c806e4d0 0x7fd2c80cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40780/0 (socket says 172.21.15.133:40780) 2024-04-25T10:03:17.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2c6ffd640 1 -- 172.21.15.133:0/592889031 learned_addr learned my addr 172.21.15.133:0/592889031 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:17.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2c7fff640 1 -- 172.21.15.133:0/592889031 >> v2:172.21.15.162:3300/0 conn(0x7fd2c8145170 msgr2=0x7fd2c806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2c77fe640 1 --2- 172.21.15.133:0/592889031 >> v2:172.21.15.162:3300/0 conn(0x7fd2c8145170 0x7fd2c806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2c7fff640 1 --2- 172.21.15.133:0/592889031 >> v2:172.21.15.162:3300/0 conn(0x7fd2c8145170 0x7fd2c806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2c7fff640 1 -- 172.21.15.133:0/592889031 >> v1:172.21.15.133:6789/0 conn(0x7fd2c806e4d0 legacy=0x7fd2c80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2c67fc640 1 -- 172.21.15.133:0/592889031 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653948216 0 0) 0x7fd2c80ce2e0 con 0x7fd2c806e4d0 2024-04-25T10:03:17.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2c7fff640 1 -- 172.21.15.133:0/592889031 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c806d4b0 con 0x7fd2c815ced0 2024-04-25T10:03:17.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.560+0000 7fd2c77fe640 1 --2- 172.21.15.133:0/592889031 >> v2:172.21.15.162:3300/0 conn(0x7fd2c8145170 0x7fd2c806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:17.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c7fff640 1 --2- 172.21.15.133:0/592889031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c815ced0 0x7fd2c815d320 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd2c806b790 tx=0x7fd2c001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a042897a5fbed77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c67fc640 1 -- 172.21.15.133:0/592889031 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00b2070 con 0x7fd2c815ced0 2024-04-25T10:03:17.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c67fc640 1 -- 172.21.15.133:0/592889031 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c00a57f0 con 0x7fd2c815ced0 2024-04-25T10:03:17.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c67fc640 1 -- 172.21.15.133:0/592889031 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00a5bc0 con 0x7fd2c815ced0 2024-04-25T10:03:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 -- 172.21.15.133:0/592889031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c815ced0 msgr2=0x7fd2c815d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 --2- 172.21.15.133:0/592889031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c815ced0 0x7fd2c815d320 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd2c806b790 tx=0x7fd2c001c930 comp rx=0 tx=0).stop 2024-04-25T10:03:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 -- 172.21.15.133:0/592889031 shutdown_connections 2024-04-25T10:03:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 --2- 172.21.15.133:0/592889031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c815ced0 0x7fd2c815d320 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 --2- 172.21.15.133:0/592889031 >> v2:172.21.15.162:3300/0 conn(0x7fd2c8145170 0x7fd2c806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 -- 172.21.15.133:0/592889031 >> 172.21.15.133:0/592889031 conn(0x7fd2c8066f70 msgr2=0x7fd2c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 -- 172.21.15.133:0/592889031 shutdown_connections 2024-04-25T10:03:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 -- 172.21.15.133:0/592889031 wait complete. 2024-04-25T10:03:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 Processor -- start 2024-04-25T10:03:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 -- start start 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c806e4d0 0x7fd2c8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c8145170 0x7fd2c8152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2c815ced0 0x7fd2c8153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8168610 con 0x7fd2c8145170 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8168490 con 0x7fd2c806e4d0 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8168790 con 0x7fd2c815ced0 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c8145170 0x7fd2c8152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c8145170 0x7fd2c8152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60578/0 (socket says 172.21.15.133:60578) 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c6ffd640 1 -- 172.21.15.133:0/3305086257 learned_addr learned my addr 172.21.15.133:0/3305086257 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c77fe640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c806e4d0 0x7fd2c8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c7fff640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2c815ced0 0x7fd2c8153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c6ffd640 1 -- 172.21.15.133:0/3305086257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2c815ced0 msgr2=0x7fd2c8153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c6ffd640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2c815ced0 0x7fd2c8153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c6ffd640 1 -- 172.21.15.133:0/3305086257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c806e4d0 msgr2=0x7fd2c8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c6ffd640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c806e4d0 0x7fd2c8133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c6ffd640 1 -- 172.21.15.133:0/3305086257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c8151770 con 0x7fd2c8145170 2024-04-25T10:03:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c7fff640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2c815ced0 0x7fd2c8153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c77fe640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c806e4d0 0x7fd2c8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2c6ffd640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c8145170 0x7fd2c8152e30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c480 tx=0x7fd2bc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2affff640 1 -- 172.21.15.133:0/3305086257 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc018070 con 0x7fd2c8145170 2024-04-25T10:03:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2affff640 1 -- 172.21.15.133:0/3305086257 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc0074e0 con 0x7fd2c8145170 2024-04-25T10:03:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2affff640 1 -- 172.21.15.133:0/3305086257 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc004a30 con 0x7fd2c8145170 2024-04-25T10:03:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 -- 172.21.15.133:0/3305086257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c8151a50 con 0x7fd2c8145170 2024-04-25T10:03:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.564+0000 7fd2cda86640 1 -- 172.21.15.133:0/3305086257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c8154200 con 0x7fd2c8145170 2024-04-25T10:03:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.568+0000 7fd2cda86640 1 -- 172.21.15.133:0/3305086257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c8068e80 con 0x7fd2c8145170 2024-04-25T10:03:17.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.568+0000 7fd2affff640 1 -- 172.21.15.133:0/3305086257 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2bc01b070 con 0x7fd2c8145170 2024-04-25T10:03:17.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.568+0000 7fd2affff640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b4066f60 0x7fd2b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.568+0000 7fd2affff640 1 -- 172.21.15.133:0/3305086257 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd2bc0843e0 con 0x7fd2c8145170 2024-04-25T10:03:17.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.568+0000 7fd2c77fe640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b4066f60 0x7fd2b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.568+0000 7fd2c77fe640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b4066f60 0x7fd2b4069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8008930 tx=0x7fd2b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.572+0000 7fd2affff640 1 -- 172.21.15.133:0/3305086257 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2bc055330 con 0x7fd2c8145170 2024-04-25T10:03:17.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:17.768+0000 7fd2cda86640 1 -- 172.21.15.133:0/3305086257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd2c813dbd0 con 0x7fd2c8145170 2024-04-25T10:03:18.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T10:03:18.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T10:03:18.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:03:18.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:03:18.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.444+0000 7fd2affff640 1 -- 172.21.15.133:0/3305086257 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd2bc054c30 con 0x7fd2c8145170 2024-04-25T10:03:18.451 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T10:03:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 -- 172.21.15.133:0/3305086257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b4066f60 msgr2=0x7fd2b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b4066f60 0x7fd2b4069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8008930 tx=0x7fd2b8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 -- 172.21.15.133:0/3305086257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c8145170 msgr2=0x7fd2c8152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c8145170 0x7fd2c8152e30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c480 tx=0x7fd2bc00c940 comp rx=0 tx=0).stop 2024-04-25T10:03:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 -- 172.21.15.133:0/3305086257 shutdown_connections 2024-04-25T10:03:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2c815ced0 0x7fd2c8153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b4066f60 0x7fd2b4069420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c8145170 0x7fd2c8152e30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 --2- 172.21.15.133:0/3305086257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c806e4d0 0x7fd2c8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 -- 172.21.15.133:0/3305086257 >> 172.21.15.133:0/3305086257 conn(0x7fd2c8066f70 msgr2=0x7fd2c8141120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 -- 172.21.15.133:0/3305086257 shutdown_connections 2024-04-25T10:03:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.452+0000 7fd2cda86640 1 -- 172.21.15.133:0/3305086257 wait complete. 2024-04-25T10:03:18.487 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:18.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.592+0000 7f935bab2640 1 Processor -- start 2024-04-25T10:03:18.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.592+0000 7f935bab2640 1 -- start start 2024-04-25T10:03:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935bab2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9354158110 0x7f93540b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935bab2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93541584f0 0x7f935415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935bab2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f935406e460 0x7f935406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935bab2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935406edf0 con 0x7f9354158110 2024-04-25T10:03:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935bab2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93540cbe60 con 0x7f935406e460 2024-04-25T10:03:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935bab2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93540b19b0 con 0x7f93541584f0 2024-04-25T10:03:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935984e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9354158110 0x7f93540b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935984e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9354158110 0x7f93540b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60582/0 (socket says 172.21.15.133:60582) 2024-04-25T10:03:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935984e640 1 -- 172.21.15.133:0/673778562 learned_addr learned my addr 172.21.15.133:0/673778562 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935904d640 1 --2- 172.21.15.133:0/673778562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93541584f0 0x7f935415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935a04f640 1 --2- 172.21.15.133:0/673778562 >> v2:172.21.15.162:3300/0 conn(0x7f935406e460 0x7f935406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935984e640 1 -- 172.21.15.133:0/673778562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93541584f0 msgr2=0x7f935415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935984e640 1 --2- 172.21.15.133:0/673778562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93541584f0 0x7f935415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935984e640 1 -- 172.21.15.133:0/673778562 >> v2:172.21.15.162:3300/0 conn(0x7f935406e460 msgr2=0x7f935406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935984e640 1 --2- 172.21.15.133:0/673778562 >> v2:172.21.15.162:3300/0 conn(0x7f935406e460 0x7f935406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935984e640 1 -- 172.21.15.133:0/673778562 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935415b360 con 0x7f9354158110 2024-04-25T10:03:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935984e640 1 --2- 172.21.15.133:0/673778562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9354158110 0x7f93540b1f00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f935415f3a0 tx=0x7f935001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148e127124817e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935884c640 1 -- 172.21.15.133:0/673778562 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500aa610 con 0x7f9354158110 2024-04-25T10:03:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935884c640 1 -- 172.21.15.133:0/673778562 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93500af070 con 0x7f9354158110 2024-04-25T10:03:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935884c640 1 -- 172.21.15.133:0/673778562 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500aa9d0 con 0x7f9354158110 2024-04-25T10:03:18.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935bab2640 1 -- 172.21.15.133:0/673778562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9354158110 msgr2=0x7f93540b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.596+0000 7f935bab2640 1 --2- 172.21.15.133:0/673778562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9354158110 0x7f93540b1f00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f935415f3a0 tx=0x7f935001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 -- 172.21.15.133:0/673778562 shutdown_connections 2024-04-25T10:03:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 --2- 172.21.15.133:0/673778562 >> v2:172.21.15.162:3300/0 conn(0x7f935406e460 0x7f935406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 --2- 172.21.15.133:0/673778562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93541584f0 0x7f935415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 --2- 172.21.15.133:0/673778562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9354158110 0x7f93540b1f00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 -- 172.21.15.133:0/673778562 >> 172.21.15.133:0/673778562 conn(0x7f9354067000 msgr2=0x7f935406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 -- 172.21.15.133:0/673778562 shutdown_connections 2024-04-25T10:03:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 -- 172.21.15.133:0/673778562 wait complete. 2024-04-25T10:03:18.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 Processor -- start 2024-04-25T10:03:18.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 -- start start 2024-04-25T10:03:18.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935406e460 0x7f9354163d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9354158110 0x7f93540cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93541584f0 0x7f935406aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93540cd7f0 con 0x7f9354158110 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93540cd670 con 0x7f935406e460 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935415eb10 con 0x7f93541584f0 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935904d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9354158110 0x7f93540cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935904d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9354158110 0x7f93540cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60588/0 (socket says 172.21.15.133:60588) 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935904d640 1 -- 172.21.15.133:0/1824591578 learned_addr learned my addr 172.21.15.133:0/1824591578 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935a04f640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93541584f0 0x7f935406aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935984e640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935406e460 0x7f9354163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935904d640 1 -- 172.21.15.133:0/1824591578 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93541584f0 msgr2=0x7f935406aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935904d640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93541584f0 0x7f935406aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935904d640 1 -- 172.21.15.133:0/1824591578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935406e460 msgr2=0x7f9354163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935904d640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935406e460 0x7f9354163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935904d640 1 -- 172.21.15.133:0/1824591578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9354154ab0 con 0x7f9354158110 2024-04-25T10:03:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935904d640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9354158110 0x7f93540cc860 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f934400d5f0 tx=0x7f934400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f934a7fc640 1 -- 172.21.15.133:0/1824591578 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440125f0 con 0x7f9354158110 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f934a7fc640 1 -- 172.21.15.133:0/1824591578 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9344019070 con 0x7f9354158110 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f934a7fc640 1 -- 172.21.15.133:0/1824591578 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440129b0 con 0x7f9354158110 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 -- 172.21.15.133:0/1824591578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9354152280 con 0x7f9354158110 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.600+0000 7f935bab2640 1 -- 172.21.15.133:0/1824591578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93540cc340 con 0x7f9354158110 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.604+0000 7f935bab2640 1 -- 172.21.15.133:0/1824591578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935406bc20 con 0x7f9354158110 2024-04-25T10:03:18.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.604+0000 7f934a7fc640 1 -- 172.21.15.133:0/1824591578 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9344012b50 con 0x7f9354158110 2024-04-25T10:03:18.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.604+0000 7f934a7fc640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9338066f90 0x7f9338069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.604+0000 7f934a7fc640 1 -- 172.21.15.133:0/1824591578 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9344086030 con 0x7f9354158110 2024-04-25T10:03:18.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.604+0000 7f935984e640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9338066f90 0x7f9338069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.604+0000 7f935984e640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9338066f90 0x7f9338069450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f93500ae000 tx=0x7f93500a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.608+0000 7f934a7fc640 1 -- 172.21.15.133:0/1824591578 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9344056f20 con 0x7f9354158110 2024-04-25T10:03:18.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.768+0000 7f935bab2640 1 -- 172.21.15.133:0/1824591578 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f935415d7c0 con 0x7f9354158110 2024-04-25T10:03:18.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.768+0000 7f934a7fc640 1 -- 172.21.15.133:0/1824591578 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16979 (secure 0 0 0) 0x7f9344056820 con 0x7f9354158110 2024-04-25T10:03:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:18.777 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":37,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:03:17.773075+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:16.032957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:16.760653+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 -- 172.21.15.133:0/1824591578 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9338066f90 msgr2=0x7f9338069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9338066f90 0x7f9338069450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f93500ae000 tx=0x7f93500a7000 comp rx=0 tx=0).stop 2024-04-25T10:03:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 -- 172.21.15.133:0/1824591578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9354158110 msgr2=0x7f93540cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9354158110 0x7f93540cc860 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f934400d5f0 tx=0x7f934400dab0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 -- 172.21.15.133:0/1824591578 shutdown_connections 2024-04-25T10:03:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f93541584f0 0x7f935406aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9338066f90 0x7f9338069450 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9354158110 0x7f93540cc860 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 --2- 172.21.15.133:0/1824591578 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f935406e460 0x7f9354163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 -- 172.21.15.133:0/1824591578 >> 172.21.15.133:0/1824591578 conn(0x7f9354067000 msgr2=0x7f9354158fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 -- 172.21.15.133:0/1824591578 shutdown_connections 2024-04-25T10:03:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.772+0000 7f935bab2640 1 -- 172.21.15.133:0/1824591578 wait complete. 2024-04-25T10:03:18.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:18.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecbcb63640 1 Processor -- start 2024-04-25T10:03:18.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecbcb63640 1 -- start start 2024-04-25T10:03:18.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecbcb63640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fecb80ce310 0x7fecb80cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecbcb63640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb806de10 con 0x7fecb80cdf30 2024-04-25T10:03:18.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecbcb63640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb806e0f0 con 0x7fecb80cbb80 2024-04-25T10:03:18.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecbcb63640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb80cbff0 con 0x7fecb80ce310 2024-04-25T10:03:18.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecb5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fecb80ce310 0x7fecb80cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecb5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fecb80ce310 0x7fecb80cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60590/0 (socket says 172.21.15.133:60590) 2024-04-25T10:03:18.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecb5d9b640 1 -- 172.21.15.133:0/3003033124 learned_addr learned my addr 172.21.15.133:0/3003033124 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:18.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecb659c640 1 --1- 172.21.15.133:0/3003033124 >> v1:172.21.15.133:6789/0 conn(0x7fecb80cdf30 0x7fecb806d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40784/0 (socket says 172.21.15.133:40784) 2024-04-25T10:03:18.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecb5d9b640 1 -- 172.21.15.133:0/3003033124 >> v1:172.21.15.162:6789/0 conn(0x7fecb80cbb80 legacy=0x7fecb80cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecb5d9b640 1 -- 172.21.15.133:0/3003033124 >> v1:172.21.15.133:6789/0 conn(0x7fecb80cdf30 legacy=0x7fecb806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecb5d9b640 1 -- 172.21.15.133:0/3003033124 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb806e3d0 con 0x7fecb80ce310 2024-04-25T10:03:18.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecb559a640 1 -- 172.21.15.133:0/3003033124 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141173176 0 0) 0x7fecb806de10 con 0x7fecb80cdf30 2024-04-25T10:03:18.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecb5d9b640 1 --2- 172.21.15.133:0/3003033124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecb80ce310 0x7fecb80cb640 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fecac009870 tx=0x7fecac01cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969c2f60f566bb12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.912+0000 7fecb559a640 1 -- 172.21.15.133:0/3003033124 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0ab430 con 0x7fecb80ce310 2024-04-25T10:03:18.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb559a640 1 -- 172.21.15.133:0/3003033124 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac0b0070 con 0x7fecb80ce310 2024-04-25T10:03:18.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb559a640 1 -- 172.21.15.133:0/3003033124 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0ab7f0 con 0x7fecb80ce310 2024-04-25T10:03:18.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 -- 172.21.15.133:0/3003033124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecb80ce310 msgr2=0x7fecb80cb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 --2- 172.21.15.133:0/3003033124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecb80ce310 0x7fecb80cb640 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fecac009870 tx=0x7fecac01cf80 comp rx=0 tx=0).stop 2024-04-25T10:03:18.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 -- 172.21.15.133:0/3003033124 shutdown_connections 2024-04-25T10:03:18.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 --2- 172.21.15.133:0/3003033124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecb80ce310 0x7fecb80cb640 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 -- 172.21.15.133:0/3003033124 >> 172.21.15.133:0/3003033124 conn(0x7fecb8067000 msgr2=0x7fecb806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 -- 172.21.15.133:0/3003033124 shutdown_connections 2024-04-25T10:03:18.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 -- 172.21.15.133:0/3003033124 wait complete. 2024-04-25T10:03:18.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 Processor -- start 2024-04-25T10:03:18.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 -- start start 2024-04-25T10:03:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecb80cbb80 0x7fecb8142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecb80cdf30 0x7fecb813eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecb80ce310 0x7fecb8158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb80cd350 con 0x7fecb80cdf30 2024-04-25T10:03:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb80cd1d0 con 0x7fecb80cbb80 2024-04-25T10:03:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb80cd4d0 con 0x7fecb80ce310 2024-04-25T10:03:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecb80cdf30 0x7fecb813eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecb80cdf30 0x7fecb813eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60606/0 (socket says 172.21.15.133:60606) 2024-04-25T10:03:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb5d9b640 1 -- 172.21.15.133:0/1430919787 learned_addr learned my addr 172.21.15.133:0/1430919787 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb659c640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecb80cbb80 0x7fecb8142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb6d9d640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecb80ce310 0x7fecb8158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb5d9b640 1 -- 172.21.15.133:0/1430919787 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecb80ce310 msgr2=0x7fecb8158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb5d9b640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecb80ce310 0x7fecb8158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb5d9b640 1 -- 172.21.15.133:0/1430919787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecb80cbb80 msgr2=0x7fecb8142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb5d9b640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecb80cbb80 0x7fecb8142810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb5d9b640 1 -- 172.21.15.133:0/1430919787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb80675c0 con 0x7fecb80cdf30 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb659c640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecb80cbb80 0x7fecb8142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecb5d9b640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecb80cdf30 0x7fecb813eca0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fecac01a150 tx=0x7fecac0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7feca6ffd640 1 -- 172.21.15.133:0/1430919787 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0ba070 con 0x7fecb80cdf30 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7feca6ffd640 1 -- 172.21.15.133:0/1430919787 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac0abce0 con 0x7fecb80cdf30 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 -- 172.21.15.133:0/1430919787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb8142d50 con 0x7fecb80cdf30 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7feca6ffd640 1 -- 172.21.15.133:0/1430919787 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac004220 con 0x7fecb80cdf30 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.916+0000 7fecbcb63640 1 -- 172.21.15.133:0/1430919787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb80c8210 con 0x7fecb80cdf30 2024-04-25T10:03:18.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.920+0000 7fecbcb63640 1 -- 172.21.15.133:0/1430919787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb8068ef0 con 0x7fecb80cdf30 2024-04-25T10:03:18.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.920+0000 7feca6ffd640 1 -- 172.21.15.133:0/1430919787 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fecac003620 con 0x7fecb80cdf30 2024-04-25T10:03:18.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.920+0000 7feca6ffd640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec98066f60 0x7fec98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.920+0000 7feca6ffd640 1 -- 172.21.15.133:0/1430919787 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fecac0a8030 con 0x7fecb80cdf30 2024-04-25T10:03:18.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.920+0000 7fecb659c640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec98066f60 0x7fec98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.920+0000 7fecb659c640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec98066f60 0x7fec98069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feca8000f70 tx=0x7feca8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:18.924+0000 7feca6ffd640 1 -- 172.21.15.133:0/1430919787 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fecac0c3060 con 0x7fecb80cdf30 2024-04-25T10:03:19.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.084+0000 7fecbcb63640 1 -- 172.21.15.133:0/1430919787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecb8143300 con 0x7fecb80cdf30 2024-04-25T10:03:19.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.084+0000 7feca6ffd640 1 -- 172.21.15.133:0/1430919787 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16979 (secure 0 0 0) 0x7fecac016110 con 0x7fecb80cdf30 2024-04-25T10:03:19.093 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:19.093 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":37,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:03:17.773075+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:16.032957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:16.760653+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 -- 172.21.15.133:0/1430919787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec98066f60 msgr2=0x7fec98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec98066f60 0x7fec98069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feca8000f70 tx=0x7feca8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 -- 172.21.15.133:0/1430919787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecb80cdf30 msgr2=0x7fecb813eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecb80cdf30 0x7fecb813eca0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fecac01a150 tx=0x7fecac0192c0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 -- 172.21.15.133:0/1430919787 shutdown_connections 2024-04-25T10:03:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fecb80ce310 0x7fecb8158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec98066f60 0x7fec98069420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecb80cdf30 0x7fecb813eca0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 --2- 172.21.15.133:0/1430919787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecb80cbb80 0x7fecb8142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 -- 172.21.15.133:0/1430919787 >> 172.21.15.133:0/1430919787 conn(0x7fecb8067000 msgr2=0x7fecb806d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 -- 172.21.15.133:0/1430919787 shutdown_connections 2024-04-25T10:03:19.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.088+0000 7fecbcb63640 1 -- 172.21.15.133:0/1430919787 wait complete. 2024-04-25T10:03:19.112 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:03:19.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f658ab27640 1 Processor -- start 2024-04-25T10:03:19.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f658ab27640 1 -- start start 2024-04-25T10:03:19.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f658ab27640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65840cd820 0x7f65840cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f658ab27640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65840ce2e0 con 0x7f6584145170 2024-04-25T10:03:19.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f658ab27640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658406d4b0 con 0x7f658415ced0 2024-04-25T10:03:19.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f658ab27640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658406db00 con 0x7f65840cd820 2024-04-25T10:03:19.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65888c3640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6584145170 0x7f658406df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40800/0 (socket says 172.21.15.133:40800) 2024-04-25T10:03:19.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65888c3640 1 -- 172.21.15.133:0/2444267206 learned_addr learned my addr 172.21.15.133:0/2444267206 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841772038 0 0) 0x7f65840ce2e0 con 0x7f6584145170 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f6583fff640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65840cd820 0x7f65840cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6574003530 con 0x7f6584145170 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3971285846 0 0) 0x7f6574003530 con 0x7f6584145170 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65840cd820 msgr2=0x7f65840cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65840cd820 0x7f65840cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 >> v1:172.21.15.162:6789/0 conn(0x7f658415ced0 legacy=0x7f65840b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658406d790 con 0x7f6584145170 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6578001680 con 0x7f6584145170 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f6583fff640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65840cd820 0x7f65840cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 >> v1:172.21.15.133:6789/0 conn(0x7f6584145170 legacy=0x7f658406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65840cd820 0x7f65740039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6574003fe0 0x7f6574004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6574004d50 0x7f65740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658406db00 con 0x7f6574003fe0 2024-04-25T10:03:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657400bb30 con 0x7f6574004d50 2024-04-25T10:03:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657400bd30 con 0x7f65840cd820 2024-04-25T10:03:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f6583fff640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6574003fe0 0x7f6574004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65888c3640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65840cd820 0x7f65740039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65890c4640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6574004d50 0x7f65740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f6583fff640 1 -- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65840cd820 msgr2=0x7f65740039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f6583fff640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65840cd820 0x7f65740039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f6583fff640 1 -- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6574004d50 msgr2=0x7f65740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f6583fff640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6574004d50 0x7f65740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f6583fff640 1 -- 172.21.15.133:0/2444267206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6574003530 con 0x7f6574003fe0 2024-04-25T10:03:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.228+0000 7f65888c3640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65840cd820 0x7f65740039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f6583fff640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6574003fe0 0x7f6574004810 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f65700098d0 tx=0x7f657001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd145bc892de38d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000f040 con 0x7f6574003fe0 2024-04-25T10:03:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65700a9620 con 0x7f6574003fe0 2024-04-25T10:03:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f65837fe640 1 -- 172.21.15.133:0/2444267206 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700a99d0 con 0x7f6574003fe0 2024-04-25T10:03:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f658ab27640 1 -- 172.21.15.133:0/2444267206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6574003fe0 msgr2=0x7f6574004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f658ab27640 1 --2- 172.21.15.133:0/2444267206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6574003fe0 0x7f6574004810 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f65700098d0 tx=0x7f657001c510 comp rx=0 tx=0).stop 2024-04-25T10:03:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f65890c4640 1 -- 172.21.15.133:0/2444267206 reap_dead start 2024-04-25T10:03:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f658ab27640 1 -- 172.21.15.133:0/2444267206 shutdown_connections 2024-04-25T10:03:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f658ab27640 1 -- 172.21.15.133:0/2444267206 >> 172.21.15.133:0/2444267206 conn(0x7f6584066f70 msgr2=0x7f658406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f658ab27640 1 -- 172.21.15.133:0/2444267206 shutdown_connections 2024-04-25T10:03:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f658ab27640 1 -- 172.21.15.133:0/2444267206 wait complete. 2024-04-25T10:03:19.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f658ab27640 1 Processor -- start 2024-04-25T10:03:19.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.232+0000 7f658ab27640 1 -- start start 2024-04-25T10:03:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f658ab27640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6574003fe0 0x7f658413a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f658ab27640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6584137c10 0x7f6584152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f658ab27640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6584142b10 0x7f6584142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f658ab27640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658415cb70 con 0x7f6584142b10 2024-04-25T10:03:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f658ab27640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6584145550 con 0x7f6584137c10 2024-04-25T10:03:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f658ab27640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65840b1dd0 con 0x7f6574003fe0 2024-04-25T10:03:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65890c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6584142b10 0x7f6584142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65890c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6584142b10 0x7f6584142f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60612/0 (socket says 172.21.15.133:60612) 2024-04-25T10:03:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65890c4640 1 -- 172.21.15.133:0/877730498 learned_addr learned my addr 172.21.15.133:0/877730498 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f6583fff640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6584137c10 0x7f6584152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65890c4640 1 -- 172.21.15.133:0/877730498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6574003fe0 msgr2=0x7f658413a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65890c4640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6574003fe0 0x7f658413a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65890c4640 1 -- 172.21.15.133:0/877730498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6584137c10 msgr2=0x7f6584152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65890c4640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6584137c10 0x7f6584152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65890c4640 1 -- 172.21.15.133:0/877730498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584138090 con 0x7f6584142b10 2024-04-25T10:03:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65890c4640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6584142b10 0x7f6584142f90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f657c00e700 tx=0x7f657c00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65817fa640 1 -- 172.21.15.133:0/877730498 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c003980 con 0x7f6584142b10 2024-04-25T10:03:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65817fa640 1 -- 172.21.15.133:0/877730498 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657c018070 con 0x7f6584142b10 2024-04-25T10:03:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f658ab27640 1 -- 172.21.15.133:0/877730498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6584138360 con 0x7f6584142b10 2024-04-25T10:03:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65817fa640 1 -- 172.21.15.133:0/877730498 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c003d60 con 0x7f6584142b10 2024-04-25T10:03:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f658ab27640 1 -- 172.21.15.133:0/877730498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6584138920 con 0x7f6584142b10 2024-04-25T10:03:19.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f658ab27640 1 -- 172.21.15.133:0/877730498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6584068e80 con 0x7f6584142b10 2024-04-25T10:03:19.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65817fa640 1 -- 172.21.15.133:0/877730498 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f657c0050d0 con 0x7f6584142b10 2024-04-25T10:03:19.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65817fa640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6568066f60 0x7f6568069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65817fa640 1 -- 172.21.15.133:0/877730498 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f657c084e10 con 0x7f6584142b10 2024-04-25T10:03:19.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.236+0000 7f65888c3640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6568066f60 0x7f6568069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.240+0000 7f65888c3640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6568066f60 0x7f6568069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6578000c10 tx=0x7f6578001b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.240+0000 7f65817fa640 1 -- 172.21.15.133:0/877730498 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f657c055640 con 0x7f6584142b10 2024-04-25T10:03:19.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.436+0000 7f658ab27640 1 -- 172.21.15.133:0/877730498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6584152640 con 0x7f6584142b10 2024-04-25T10:03:19.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f65817fa640 1 -- 172.21.15.133:0/877730498 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f657c0596e0 con 0x7f6584142b10 2024-04-25T10:03:19.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 -- 172.21.15.133:0/877730498 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6568066f60 msgr2=0x7f6568069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6568066f60 0x7f6568069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6578000c10 tx=0x7f6578001b70 comp rx=0 tx=0).stop 2024-04-25T10:03:19.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 -- 172.21.15.133:0/877730498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6584142b10 msgr2=0x7f6584142f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6584142b10 0x7f6584142f90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f657c00e700 tx=0x7f657c00ebc0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 -- 172.21.15.133:0/877730498 shutdown_connections 2024-04-25T10:03:19.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6568066f60 0x7f6568069420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6584142b10 0x7f6584142f90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6584137c10 0x7f6584152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 --2- 172.21.15.133:0/877730498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6574003fe0 0x7f658413a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 -- 172.21.15.133:0/877730498 >> 172.21.15.133:0/877730498 conn(0x7f6584066f70 msgr2=0x7f65840b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 -- 172.21.15.133:0/877730498 shutdown_connections 2024-04-25T10:03:19.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.452+0000 7f658ab27640 1 -- 172.21.15.133:0/877730498 wait complete. 2024-04-25T10:03:19.477 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:03:19.477 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:03:19.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f087db4d640 1 Processor -- start 2024-04-25T10:03:19.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f087db4d640 1 -- start start 2024-04-25T10:03:19.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f087db4d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f087806bca0 0x7f08780b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f087db4d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f08780669a0 0x7f087806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f087db4d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087806c5c0 con 0x7f08780acd10 2024-04-25T10:03:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f087db4d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0878072580 con 0x7f087806bca0 2024-04-25T10:03:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f087db4d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0878066e60 con 0x7f08780669a0 2024-04-25T10:03:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f08777fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f087806bca0 0x7f08780b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f08777fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f087806bca0 0x7f08780b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60622/0 (socket says 172.21.15.133:60622) 2024-04-25T10:03:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f08777fe640 1 -- 172.21.15.133:0/1669716799 learned_addr learned my addr 172.21.15.133:0/1669716799 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f0876ffd640 1 --2- 172.21.15.133:0/1669716799 >> v2:172.21.15.162:3300/0 conn(0x7f08780669a0 0x7f087806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f08777fe640 1 -- 172.21.15.133:0/1669716799 >> v2:172.21.15.162:3300/0 conn(0x7f08780669a0 msgr2=0x7f087806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f08777fe640 1 --2- 172.21.15.133:0/1669716799 >> v2:172.21.15.162:3300/0 conn(0x7f08780669a0 0x7f087806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f08777fe640 1 -- 172.21.15.133:0/1669716799 >> v1:172.21.15.162:6789/0 conn(0x7f08780acd10 legacy=0x7f087806dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.592+0000 7f08777fe640 1 -- 172.21.15.133:0/1669716799 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780727d0 con 0x7f087806bca0 2024-04-25T10:03:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f08777fe640 1 --2- 172.21.15.133:0/1669716799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087806bca0 0x7f08780b3710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f08780cc4e0 tx=0x7f086801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d610e6628721df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f08767fc640 1 -- 172.21.15.133:0/1669716799 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680a5620 con 0x7f087806bca0 2024-04-25T10:03:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f08767fc640 1 -- 172.21.15.133:0/1669716799 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08680a57c0 con 0x7f087806bca0 2024-04-25T10:03:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f08767fc640 1 -- 172.21.15.133:0/1669716799 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680a5b90 con 0x7f087806bca0 2024-04-25T10:03:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f087db4d640 1 -- 172.21.15.133:0/1669716799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087806bca0 msgr2=0x7f08780b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f087db4d640 1 --2- 172.21.15.133:0/1669716799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087806bca0 0x7f08780b3710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f08780cc4e0 tx=0x7f086801c890 comp rx=0 tx=0).stop 2024-04-25T10:03:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f087db4d640 1 -- 172.21.15.133:0/1669716799 shutdown_connections 2024-04-25T10:03:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f087db4d640 1 --2- 172.21.15.133:0/1669716799 >> v2:172.21.15.162:3300/0 conn(0x7f08780669a0 0x7f087806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f087db4d640 1 --2- 172.21.15.133:0/1669716799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087806bca0 0x7f08780b3710 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f087db4d640 1 -- 172.21.15.133:0/1669716799 >> 172.21.15.133:0/1669716799 conn(0x7f0878071850 msgr2=0x7f0878071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f087db4d640 1 -- 172.21.15.133:0/1669716799 shutdown_connections 2024-04-25T10:03:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f087db4d640 1 -- 172.21.15.133:0/1669716799 wait complete. 2024-04-25T10:03:19.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f087db4d640 1 Processor -- start 2024-04-25T10:03:19.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.596+0000 7f087db4d640 1 -- start start 2024-04-25T10:03:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f087db4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08780669a0 0x7f08781525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f087db4d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f087806bca0 0x7f08781552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f087db4d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780acd10 0x7f0878156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f087db4d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08780ad320 con 0x7f08780669a0 2024-04-25T10:03:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f087db4d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08780ad1a0 con 0x7f08780acd10 2024-04-25T10:03:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f087db4d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087816af10 con 0x7f087806bca0 2024-04-25T10:03:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f08777fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08780669a0 0x7f08781525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f08777fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08780669a0 0x7f08781525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60630/0 (socket says 172.21.15.133:60630) 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f08777fe640 1 -- 172.21.15.133:0/653651135 learned_addr learned my addr 172.21.15.133:0/653651135 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f0876ffd640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f087806bca0 0x7f08781552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f0877fff640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780acd10 0x7f0878156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f08777fe640 1 -- 172.21.15.133:0/653651135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f087806bca0 msgr2=0x7f08781552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f08777fe640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f087806bca0 0x7f08781552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f08777fe640 1 -- 172.21.15.133:0/653651135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780acd10 msgr2=0x7f0878156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f08777fe640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780acd10 0x7f0878156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f08777fe640 1 -- 172.21.15.133:0/653651135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780cdfc0 con 0x7f08780669a0 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f08777fe640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08780669a0 0x7f08781525b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f08680af040 tx=0x7f0868019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f086ffff640 1 -- 172.21.15.133:0/653651135 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680bc020 con 0x7f08780669a0 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f086ffff640 1 -- 172.21.15.133:0/653651135 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08680b8670 con 0x7f08780669a0 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f086ffff640 1 -- 172.21.15.133:0/653651135 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868009510 con 0x7f08780669a0 2024-04-25T10:03:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f087db4d640 1 -- 172.21.15.133:0/653651135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08780ce2a0 con 0x7f08780669a0 2024-04-25T10:03:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f087db4d640 1 -- 172.21.15.133:0/653651135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087815c440 con 0x7f08780669a0 2024-04-25T10:03:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f087db4d640 1 -- 172.21.15.133:0/653651135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0878067c70 con 0x7f08780669a0 2024-04-25T10:03:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f086ffff640 1 -- 172.21.15.133:0/653651135 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08680bf020 con 0x7f08780669a0 2024-04-25T10:03:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f086ffff640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0858066f60 0x7f0858069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f086ffff640 1 -- 172.21.15.133:0/653651135 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0868120150 con 0x7f08780669a0 2024-04-25T10:03:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.600+0000 7f0876ffd640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0858066f60 0x7f0858069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.604+0000 7f0876ffd640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0858066f60 0x7f0858069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0878167f10 tx=0x7f0860008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.608+0000 7f086ffff640 1 -- 172.21.15.133:0/653651135 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08680c6030 con 0x7f08780669a0 2024-04-25T10:03:19.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:19.788+0000 7f087db4d640 1 -- 172.21.15.133:0/653651135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f087815a780 con 0x7f08780669a0 2024-04-25T10:03:20.426 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:20.429 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:20.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.452+0000 7f086ffff640 1 -- 172.21.15.133:0/653651135 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f08680165a0 con 0x7f08780669a0 2024-04-25T10:03:20.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 -- 172.21.15.133:0/653651135 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0858066f60 msgr2=0x7f0858069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0858066f60 0x7f0858069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0878167f10 tx=0x7f0860008040 comp rx=0 tx=0).stop 2024-04-25T10:03:20.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 -- 172.21.15.133:0/653651135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08780669a0 msgr2=0x7f08781525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08780669a0 0x7f08781525b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f08680af040 tx=0x7f0868019630 comp rx=0 tx=0).stop 2024-04-25T10:03:20.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 -- 172.21.15.133:0/653651135 shutdown_connections 2024-04-25T10:03:20.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08780acd10 0x7f0878156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f087806bca0 0x7f08781552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0858066f60 0x7f0858069420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 --2- 172.21.15.133:0/653651135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08780669a0 0x7f08781525b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 -- 172.21.15.133:0/653651135 >> 172.21.15.133:0/653651135 conn(0x7f0878071850 msgr2=0x7f08780ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 -- 172.21.15.133:0/653651135 shutdown_connections 2024-04-25T10:03:20.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.456+0000 7f087db4d640 1 -- 172.21.15.133:0/653651135 wait complete. 2024-04-25T10:03:20.485 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:03:20.486 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:20.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.640+0000 7f8c46957640 1 Processor -- start 2024-04-25T10:03:20.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.640+0000 7f8c46957640 1 -- start start 2024-04-25T10:03:20.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c46957640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4015c9a0 0x7f8c400c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c46957640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8c400ce330 0x7f8c400ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c46957640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c400c61d0 con 0x7f8c400b1f70 2024-04-25T10:03:20.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c46957640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c400cece0 con 0x7f8c400ce330 2024-04-25T10:03:20.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c46957640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c400c64b0 con 0x7f8c4015c9a0 2024-04-25T10:03:20.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c44ef4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8c400ce330 0x7f8c400ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c3ffff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4015c9a0 0x7f8c400c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c3f7fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f8c400b1f70 0x7f8c4006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:55246/0 (socket says 172.21.15.133:55246) 2024-04-25T10:03:20.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c3f7fe640 1 -- 172.21.15.133:0/1794943593 learned_addr learned my addr 172.21.15.133:0/1794943593 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:20.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c44ef4640 1 -- 172.21.15.133:0/1794943593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4015c9a0 msgr2=0x7f8c400c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c44ef4640 1 --2- 172.21.15.133:0/1794943593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4015c9a0 0x7f8c400c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c44ef4640 1 -- 172.21.15.133:0/1794943593 >> v1:172.21.15.162:6789/0 conn(0x7f8c400b1f70 legacy=0x7f8c4006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c3effd640 1 -- 172.21.15.133:0/1794943593 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253439050 0 0) 0x7f8c400c61d0 con 0x7f8c400b1f70 2024-04-25T10:03:20.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c44ef4640 1 -- 172.21.15.133:0/1794943593 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4006d4f0 con 0x7f8c400ce330 2024-04-25T10:03:20.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.644+0000 7f8c3ffff640 1 --2- 172.21.15.133:0/1794943593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4015c9a0 0x7f8c400c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:20.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c44ef4640 1 --2- 172.21.15.133:0/1794943593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c400ce330 0x7f8c400ce7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009d90 tx=0x7f8c3801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2be35cd4083a0199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c3effd640 1 -- 172.21.15.133:0/1794943593 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c380b2070 con 0x7f8c400ce330 2024-04-25T10:03:20.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c3effd640 1 -- 172.21.15.133:0/1794943593 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c380a58e0 con 0x7f8c400ce330 2024-04-25T10:03:20.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c3effd640 1 -- 172.21.15.133:0/1794943593 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c380a5cb0 con 0x7f8c400ce330 2024-04-25T10:03:20.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 -- 172.21.15.133:0/1794943593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c400ce330 msgr2=0x7f8c400ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 --2- 172.21.15.133:0/1794943593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c400ce330 0x7f8c400ce7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009d90 tx=0x7f8c3801c970 comp rx=0 tx=0).stop 2024-04-25T10:03:20.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 -- 172.21.15.133:0/1794943593 shutdown_connections 2024-04-25T10:03:20.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 --2- 172.21.15.133:0/1794943593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c400ce330 0x7f8c400ce7a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 --2- 172.21.15.133:0/1794943593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4015c9a0 0x7f8c400c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 -- 172.21.15.133:0/1794943593 >> 172.21.15.133:0/1794943593 conn(0x7f8c40067000 msgr2=0x7f8c4006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 -- 172.21.15.133:0/1794943593 shutdown_connections 2024-04-25T10:03:20.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 -- 172.21.15.133:0/1794943593 wait complete. 2024-04-25T10:03:20.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 Processor -- start 2024-04-25T10:03:20.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 -- start start 2024-04-25T10:03:20.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c400b1f70 0x7f8c4014c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400ce330 0x7f8c40145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c4015c9a0 0x7f8c4015e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4015dad0 con 0x7f8c400b1f70 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4015d950 con 0x7f8c4015c9a0 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c46957640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4015dc50 con 0x7f8c400ce330 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c3ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c400b1f70 0x7f8c4014c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c3ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c400b1f70 0x7f8c4014c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60642/0 (socket says 172.21.15.133:60642) 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.648+0000 7f8c3ffff640 1 -- 172.21.15.133:0/3005328087 learned_addr learned my addr 172.21.15.133:0/3005328087 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3f7fe640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400ce330 0x7f8c40145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c44ef4640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c4015c9a0 0x7f8c4015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3ffff640 1 -- 172.21.15.133:0/3005328087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400ce330 msgr2=0x7f8c40145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3ffff640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400ce330 0x7f8c40145300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3ffff640 1 -- 172.21.15.133:0/3005328087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c4015c9a0 msgr2=0x7f8c4015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3ffff640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c4015c9a0 0x7f8c4015e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3ffff640 1 -- 172.21.15.133:0/3005328087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c40145f90 con 0x7f8c400b1f70 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c44ef4640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c4015c9a0 0x7f8c4015e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3f7fe640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400ce330 0x7f8c40145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3ffff640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c400b1f70 0x7f8c4014c470 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000ec10 tx=0x7f8c3000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3cff9640 1 -- 172.21.15.133:0/3005328087 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30017070 con 0x7f8c400b1f70 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3cff9640 1 -- 172.21.15.133:0/3005328087 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3000cc30 con 0x7f8c400b1f70 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c46957640 1 -- 172.21.15.133:0/3005328087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c40146270 con 0x7f8c400b1f70 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3cff9640 1 -- 172.21.15.133:0/3005328087 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30013470 con 0x7f8c400b1f70 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c46957640 1 -- 172.21.15.133:0/3005328087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c40146830 con 0x7f8c400b1f70 2024-04-25T10:03:20.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c46957640 1 -- 172.21.15.133:0/3005328087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4006bc20 con 0x7f8c400b1f70 2024-04-25T10:03:20.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3cff9640 1 -- 172.21.15.133:0/3005328087 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c3001a070 con 0x7f8c400b1f70 2024-04-25T10:03:20.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3cff9640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c2c066f60 0x7f8c2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3cff9640 1 -- 172.21.15.133:0/3005328087 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c300855a0 con 0x7f8c400b1f70 2024-04-25T10:03:20.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3f7fe640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c2c066f60 0x7f8c2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.652+0000 7f8c3f7fe640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c2c066f60 0x7f8c2c069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8c34007fd0 tx=0x7f8c34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.656+0000 7f8c3cff9640 1 -- 172.21.15.133:0/3005328087 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c300563a0 con 0x7f8c400b1f70 2024-04-25T10:03:20.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.852+0000 7f8c46957640 1 -- 172.21.15.133:0/3005328087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c40068ef0 con 0x7f8c400b1f70 2024-04-25T10:03:20.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.852+0000 7f8c3cff9640 1 -- 172.21.15.133:0/3005328087 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f8c30055ca0 con 0x7f8c400b1f70 2024-04-25T10:03:20.859 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:20.859 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2887512873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2887512873},{"type":"v1","addr":"172.21.15.162:6833","nonce":2887512873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1044882076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1044882076},{"type":"v1","addr":"172.21.15.162:6835","nonce":1044882076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1965748675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1965748675},{"type":"v1","addr":"172.21.15.133:6836","nonce":1965748675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:17.773574+0000","modified":"2024-04-25T10:03:19.793202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/205640021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":205640021},{"type":"v1","addr":"172.21.15.133:6837","nonce":205640021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:20.860 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 16 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 -- 172.21.15.133:0/3005328087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c2c066f60 msgr2=0x7f8c2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c2c066f60 0x7f8c2c069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8c34007fd0 tx=0x7f8c34009040 comp rx=0 tx=0).stop 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 -- 172.21.15.133:0/3005328087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c400b1f70 msgr2=0x7f8c4014c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c400b1f70 0x7f8c4014c470 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000ec10 tx=0x7f8c3000c520 comp rx=0 tx=0).stop 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 -- 172.21.15.133:0/3005328087 shutdown_connections 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c4015c9a0 0x7f8c4015e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400ce330 0x7f8c40145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c2c066f60 0x7f8c2c069420 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 --2- 172.21.15.133:0/3005328087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c400b1f70 0x7f8c4014c470 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 -- 172.21.15.133:0/3005328087 >> 172.21.15.133:0/3005328087 conn(0x7f8c40067000 msgr2=0x7f8c401544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 -- 172.21.15.133:0/3005328087 shutdown_connections 2024-04-25T10:03:20.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.856+0000 7f8c46957640 1 -- 172.21.15.133:0/3005328087 wait complete. 2024-04-25T10:03:20.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:20.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.980+0000 7f36858a9640 1 Processor -- start 2024-04-25T10:03:20.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.980+0000 7f36858a9640 1 -- start start 2024-04-25T10:03:20.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f36858a9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 0x7f36800b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f36858a9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f36801584f0 0x7f368015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f36858a9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36800cbe60 con 0x7f3680158110 2024-04-25T10:03:20.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f36858a9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36800b19b0 con 0x7f36801584f0 2024-04-25T10:03:20.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f36858a9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368015b360 con 0x7f368006eb70 2024-04-25T10:03:20.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 0x7f36800b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f367f7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f36801584f0 0x7f368015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 0x7f36800b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:53004/0 (socket says 172.21.15.133:53004) 2024-04-25T10:03:20.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f367e7fc640 1 -- 172.21.15.133:0/1024647031 learned_addr learned my addr 172.21.15.133:0/1024647031 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:20.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f367effd640 1 --1- 172.21.15.133:0/1024647031 >> v1:172.21.15.162:6789/0 conn(0x7f3680158110 0x7f368006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:55258/0 (socket says 172.21.15.133:55258) 2024-04-25T10:03:20.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f367f7fe640 1 -- 172.21.15.133:0/1024647031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 msgr2=0x7f36800b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f367f7fe640 1 --2- 172.21.15.133:0/1024647031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 0x7f36800b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f367f7fe640 1 -- 172.21.15.133:0/1024647031 >> v1:172.21.15.162:6789/0 conn(0x7f3680158110 legacy=0x7f368006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f367f7fe640 1 -- 172.21.15.133:0/1024647031 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368015b640 con 0x7f36801584f0 2024-04-25T10:03:20.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f367dffb640 1 -- 172.21.15.133:0/1024647031 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489539810 0 0) 0x7f36800cbe60 con 0x7f3680158110 2024-04-25T10:03:20.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.984+0000 7f367e7fc640 1 --2- 172.21.15.133:0/1024647031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 0x7f36800b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:20.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367f7fe640 1 --2- 172.21.15.133:0/1024647031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36801584f0 0x7f368015ad10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f367800bfa0 tx=0x7f367801eb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daa952c35a4f67cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367dffb640 1 -- 172.21.15.133:0/1024647031 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800b2d0 con 0x7f36801584f0 2024-04-25T10:03:20.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367dffb640 1 -- 172.21.15.133:0/1024647031 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36780a7930 con 0x7f36801584f0 2024-04-25T10:03:20.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367dffb640 1 -- 172.21.15.133:0/1024647031 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36780a7d00 con 0x7f36801584f0 2024-04-25T10:03:20.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 -- 172.21.15.133:0/1024647031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36801584f0 msgr2=0x7f368015ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 --2- 172.21.15.133:0/1024647031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36801584f0 0x7f368015ad10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f367800bfa0 tx=0x7f367801eb40 comp rx=0 tx=0).stop 2024-04-25T10:03:20.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 -- 172.21.15.133:0/1024647031 shutdown_connections 2024-04-25T10:03:20.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 --2- 172.21.15.133:0/1024647031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 0x7f36800b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 --2- 172.21.15.133:0/1024647031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36801584f0 0x7f368015ad10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 -- 172.21.15.133:0/1024647031 >> 172.21.15.133:0/1024647031 conn(0x7f3680067000 msgr2=0x7f368006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 -- 172.21.15.133:0/1024647031 shutdown_connections 2024-04-25T10:03:20.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 -- 172.21.15.133:0/1024647031 wait complete. 2024-04-25T10:03:20.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 Processor -- start 2024-04-25T10:03:20.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 -- start start 2024-04-25T10:03:20.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 0x7f3680155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3680158110 0x7f36801f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36801584f0 0x7f36800c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368006dce0 con 0x7f3680158110 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368006db60 con 0x7f36801584f0 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368006de60 con 0x7f368006eb70 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3680158110 0x7f36801f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3680158110 0x7f36801f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60644/0 (socket says 172.21.15.133:60644) 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367e7fc640 1 -- 172.21.15.133:0/3138133465 learned_addr learned my addr 172.21.15.133:0/3138133465 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367effd640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 0x7f3680155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367e7fc640 1 -- 172.21.15.133:0/3138133465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 msgr2=0x7f3680155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367f7fe640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36801584f0 0x7f36800c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367e7fc640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 0x7f3680155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367e7fc640 1 -- 172.21.15.133:0/3138133465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36801584f0 msgr2=0x7f36800c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367e7fc640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36801584f0 0x7f36800c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367e7fc640 1 -- 172.21.15.133:0/3138133465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36801f5fc0 con 0x7f3680158110 2024-04-25T10:03:20.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f367e7fc640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3680158110 0x7f36801f25f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f367400ea00 tx=0x7f367400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f3667fff640 1 -- 172.21.15.133:0/3138133465 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674017070 con 0x7f3680158110 2024-04-25T10:03:20.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f3667fff640 1 -- 172.21.15.133:0/3138133465 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367400ca60 con 0x7f3680158110 2024-04-25T10:03:20.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 -- 172.21.15.133:0/3138133465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36801f5200 con 0x7f3680158110 2024-04-25T10:03:20.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f3667fff640 1 -- 172.21.15.133:0/3138133465 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367400ce10 con 0x7f3680158110 2024-04-25T10:03:20.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.988+0000 7f36858a9640 1 -- 172.21.15.133:0/3138133465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36801f57c0 con 0x7f3680158110 2024-04-25T10:03:20.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.992+0000 7f36858a9640 1 -- 172.21.15.133:0/3138133465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3680068ef0 con 0x7f3680158110 2024-04-25T10:03:21.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.992+0000 7f3667fff640 1 -- 172.21.15.133:0/3138133465 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f367401a070 con 0x7f3680158110 2024-04-25T10:03:21.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.992+0000 7f3667fff640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f366c066f60 0x7f366c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.992+0000 7f3667fff640 1 -- 172.21.15.133:0/3138133465 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3674085090 con 0x7f3680158110 2024-04-25T10:03:21.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.992+0000 7f367effd640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f366c066f60 0x7f366c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.992+0000 7f367effd640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f366c066f60 0x7f366c069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3670000f70 tx=0x7f3670007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:20.996+0000 7f3667fff640 1 -- 172.21.15.133:0/3138133465 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3674055fa0 con 0x7f3680158110 2024-04-25T10:03:21.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.152+0000 7f36858a9640 1 -- 172.21.15.133:0/3138133465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f368006e600 con 0x7f3680158110 2024-04-25T10:03:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f3667fff640 1 -- 172.21.15.133:0/3138133465 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16966 (secure 0 0 0) 0x7f367407ce20 con 0x7f3680158110 2024-04-25T10:03:21.163 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:21.163 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":38,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:03:19.409104+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:16.032957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:16.760653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 -- 172.21.15.133:0/3138133465 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f366c066f60 msgr2=0x7f366c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f366c066f60 0x7f366c069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3670000f70 tx=0x7f3670007b60 comp rx=0 tx=0).stop 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 -- 172.21.15.133:0/3138133465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3680158110 msgr2=0x7f36801f25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3680158110 0x7f36801f25f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f367400ea00 tx=0x7f367400eec0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 -- 172.21.15.133:0/3138133465 shutdown_connections 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f36801584f0 0x7f36800c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f366c066f60 0x7f366c069420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3680158110 0x7f36801f25f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 --2- 172.21.15.133:0/3138133465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f368006eb70 0x7f3680155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 -- 172.21.15.133:0/3138133465 >> 172.21.15.133:0/3138133465 conn(0x7f3680067000 msgr2=0x7f3680158dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 -- 172.21.15.133:0/3138133465 shutdown_connections 2024-04-25T10:03:21.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.156+0000 7f36858a9640 1 -- 172.21.15.133:0/3138133465 wait complete. 2024-04-25T10:03:21.182 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:03:21.182 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893c21e640 1 Processor -- start 2024-04-25T10:03:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893c21e640 1 -- start start 2024-04-25T10:03:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893c21e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893415db00 0x7f89340b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893c21e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f89340d0bd0 0x7f8934161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893c21e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89340ceb20 con 0x7f89340d0770 2024-04-25T10:03:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893c21e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89340b7740 con 0x7f89340d0bd0 2024-04-25T10:03:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893c21e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340b7a20 con 0x7f893415db00 2024-04-25T10:03:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f89397b9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f89340d0770 0x7f89341614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40806/0 (socket says 172.21.15.133:40806) 2024-04-25T10:03:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f89397b9640 1 -- 172.21.15.133:0/3487112866 learned_addr learned my addr 172.21.15.133:0/3487112866 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893a7bb640 1 --2- 172.21.15.133:0/3487112866 >> v2:172.21.15.162:3300/0 conn(0x7f89340d0bd0 0x7f8934161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8939fba640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893415db00 0x7f89340b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083455346 0 0) 0x7f89340ceb20 con 0x7f89340d0770 2024-04-25T10:03:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8920003530 con 0x7f89340d0770 2024-04-25T10:03:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1219837717 0 0) 0x7f8920003530 con 0x7f89340d0770 2024-04-25T10:03:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893415db00 msgr2=0x7f89340b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893415db00 0x7f89340b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 >> v2:172.21.15.162:3300/0 conn(0x7f89340d0bd0 msgr2=0x7f8934161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 --2- 172.21.15.133:0/3487112866 >> v2:172.21.15.162:3300/0 conn(0x7f89340d0bd0 0x7f8934161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89340b97f0 con 0x7f89340d0770 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f8924001680 con 0x7f89340d0770 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 >> v1:172.21.15.133:6789/0 conn(0x7f89340d0770 legacy=0x7f89341614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89200039c0 0x7f8920003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893415db00 0x7f8920004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8920004d50 0x7f89200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340b7740 con 0x7f8920004d50 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340b7a20 con 0x7f89200039c0 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892000b9d0 con 0x7f893415db00 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893a7bb640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8920004d50 0x7f89200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f8939fba640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89200039c0 0x7f8920003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f89397b9640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893415db00 0x7f8920004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893a7bb640 1 -- 172.21.15.133:0/3487112866 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893415db00 msgr2=0x7f8920004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893a7bb640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f893415db00 0x7f8920004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893a7bb640 1 -- 172.21.15.133:0/3487112866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89200039c0 msgr2=0x7f8920003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893a7bb640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89200039c0 0x7f8920003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.296+0000 7f893a7bb640 1 -- 172.21.15.133:0/3487112866 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8920003530 con 0x7f8920004d50 2024-04-25T10:03:21.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f893a7bb640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8920004d50 0x7f89200051d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f892c009fc0 tx=0x7f892c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1054b91113318451 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c0b2070 con 0x7f8920004d50 2024-04-25T10:03:21.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c0a5cd0 con 0x7f8920004d50 2024-04-25T10:03:21.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f8938fb8640 1 -- 172.21.15.133:0/3487112866 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c009c10 con 0x7f8920004d50 2024-04-25T10:03:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f893c21e640 1 -- 172.21.15.133:0/3487112866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8920004d50 msgr2=0x7f89200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f893c21e640 1 --2- 172.21.15.133:0/3487112866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8920004d50 0x7f89200051d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f892c009fc0 tx=0x7f892c01ccc0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f893a7bb640 1 -- 172.21.15.133:0/3487112866 reap_dead start 2024-04-25T10:03:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f893c21e640 1 -- 172.21.15.133:0/3487112866 shutdown_connections 2024-04-25T10:03:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f893c21e640 1 -- 172.21.15.133:0/3487112866 >> 172.21.15.133:0/3487112866 conn(0x7f89340727c0 msgr2=0x7f8934072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f893c21e640 1 -- 172.21.15.133:0/3487112866 shutdown_connections 2024-04-25T10:03:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f893c21e640 1 -- 172.21.15.133:0/3487112866 wait complete. 2024-04-25T10:03:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f893c21e640 1 Processor -- start 2024-04-25T10:03:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.300+0000 7f893c21e640 1 -- start start 2024-04-25T10:03:21.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f893c21e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8920004d50 0x7f8934151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f893c21e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89340d1dc0 0x7f89340cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f893c21e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89340c8c40 0x7f89340c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f893c21e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340b82c0 con 0x7f89340d1dc0 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f893c21e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340b7fe0 con 0x7f8920004d50 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f893c21e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340d1060 con 0x7f89340c8c40 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f89397b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89340d1dc0 0x7f89340cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f89397b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89340d1dc0 0x7f89340cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60664/0 (socket says 172.21.15.133:60664) 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f89397b9640 1 -- 172.21.15.133:0/714504722 learned_addr learned my addr 172.21.15.133:0/714504722 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f8939fba640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8920004d50 0x7f8934151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f893a7bb640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89340c8c40 0x7f89340c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f89397b9640 1 -- 172.21.15.133:0/714504722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89340c8c40 msgr2=0x7f89340c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f89397b9640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89340c8c40 0x7f89340c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f89397b9640 1 -- 172.21.15.133:0/714504722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8920004d50 msgr2=0x7f8934151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f89397b9640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8920004d50 0x7f8934151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f89397b9640 1 -- 172.21.15.133:0/714504722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89340cc2c0 con 0x7f89340d1dc0 2024-04-25T10:03:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f89397b9640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89340d1dc0 0x7f89340cbd80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f892400e880 tx=0x7f892400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f892a7fc640 1 -- 172.21.15.133:0/714504722 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892400c8d0 con 0x7f89340d1dc0 2024-04-25T10:03:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f892a7fc640 1 -- 172.21.15.133:0/714504722 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8924018070 con 0x7f89340d1dc0 2024-04-25T10:03:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f893c21e640 1 -- 172.21.15.133:0/714504722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89340d2ee0 con 0x7f89340d1dc0 2024-04-25T10:03:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f892a7fc640 1 -- 172.21.15.133:0/714504722 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892400ccb0 con 0x7f89340d1dc0 2024-04-25T10:03:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f893c21e640 1 -- 172.21.15.133:0/714504722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89340d12e0 con 0x7f89340d1dc0 2024-04-25T10:03:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f893c21e640 1 -- 172.21.15.133:0/714504722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89340c9600 con 0x7f89340d1dc0 2024-04-25T10:03:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f892a7fc640 1 -- 172.21.15.133:0/714504722 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f892401d460 con 0x7f89340d1dc0 2024-04-25T10:03:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f892a7fc640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8918066f60 0x7f8918069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f892a7fc640 1 -- 172.21.15.133:0/714504722 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89240851f0 con 0x7f89340d1dc0 2024-04-25T10:03:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f8939fba640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8918066f60 0x7f8918069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.304+0000 7f8939fba640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8918066f60 0x7f8918069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f89300099b0 tx=0x7f8930008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.308+0000 7f892a7fc640 1 -- 172.21.15.133:0/714504722 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8924056070 con 0x7f89340d1dc0 2024-04-25T10:03:21.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.452+0000 7f893c21e640 1 -- 172.21.15.133:0/714504722 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8934068ef0 con 0x7f8918066f60 2024-04-25T10:03:21.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.456+0000 7f892a7fc640 1 -- 172.21.15.133:0/714504722 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237200 (secure 0 0 0) 0x7f8934068ef0 con 0x7f8918066f60 2024-04-25T10:03:21.467 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:21.469 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:03:21.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.460+0000 7f893c21e640 1 -- 172.21.15.133:0/714504722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8918066f60 msgr2=0x7f8918069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.460+0000 7f893c21e640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8918066f60 0x7f8918069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f89300099b0 tx=0x7f8930008040 comp rx=0 tx=0).stop 2024-04-25T10:03:21.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.460+0000 7f893c21e640 1 -- 172.21.15.133:0/714504722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89340d1dc0 msgr2=0x7f89340cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.460+0000 7f893c21e640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89340d1dc0 0x7f89340cbd80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f892400e880 tx=0x7f892400ed40 comp rx=0 tx=0).stop 2024-04-25T10:03:21.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.460+0000 7f893c21e640 1 -- 172.21.15.133:0/714504722 shutdown_connections 2024-04-25T10:03:21.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.460+0000 7f893c21e640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f89340c8c40 0x7f89340c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.460+0000 7f893c21e640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8918066f60 0x7f8918069420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.460+0000 7f893c21e640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89340d1dc0 0x7f89340cbd80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.460+0000 7f893c21e640 1 --2- 172.21.15.133:0/714504722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8920004d50 0x7f8934151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.460+0000 7f893c21e640 1 -- 172.21.15.133:0/714504722 >> 172.21.15.133:0/714504722 conn(0x7f89340727c0 msgr2=0x7f8934161800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.460+0000 7f893c21e640 1 -- 172.21.15.133:0/714504722 shutdown_connections 2024-04-25T10:03:21.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.464+0000 7f893c21e640 1 -- 172.21.15.133:0/714504722 wait complete. 2024-04-25T10:03:21.485 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-04-25T10:03:19.731234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220432,"kb_used_data":4200,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754288,"statfs":{"total":773094113280,"available":772868390912,"internally_reserved":0,"allocated":4300800,"data_stored":2487080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001076"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:17.394052+0000","last_change":"2024-04-25T10:03:17.394052+0000","last_active":"2024-04-25T10:03:17.394052+0000","last_peered":"2024-04-25T10:03:17.394052+0000","last_clean":"2024-04-25T10:03:17.394052+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:17.394052+0000","last_undegraded":"2024-04-25T10:03:17.394052+0000","last_fullsized":"2024-04-25T10:03:17.394052+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:16.383467+0000","last_change":"2024-04-25T10:03:16.383467+0000","last_active":"2024-04-25T10:03:16.383467+0000","last_peered":"2024-04-25T10:03:16.383467+0000","last_clean":"2024-04-25T10:03:16.383467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:16.383467+0000","last_undegraded":"2024-04-25T10:03:16.383467+0000","last_fullsized":"2024-04-25T10:03:16.383467+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:03:12.349036+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:03:12.349036+0000","last_peered":"2024-04-25T10:03:12.349036+0000","last_clean":"2024-04-25T10:03:12.349036+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:03:12.349036+0000","last_undegraded":"2024-04-25T10:03:12.349036+0000","last_fullsized":"2024-04-25T10:03:12.349036+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":753661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27420,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":389120,"data_stored":163293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":163293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":163293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":163293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":163293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":163293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":983040,"data_stored":753661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:21.489 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:21.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7e8b9640 1 Processor -- start 2024-04-25T10:03:21.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7e8b9640 1 -- start start 2024-04-25T10:03:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7e8b9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdf78158110 0x7fdf780b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7e8b9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdf7806eb70 0x7fdf7815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7e8b9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf780cbe60 con 0x7fdf781584f0 2024-04-25T10:03:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7e8b9640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf780b19b0 con 0x7fdf7806eb70 2024-04-25T10:03:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7e8b9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7815b360 con 0x7fdf78158110 2024-04-25T10:03:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7ce56640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdf7806eb70 0x7fdf7815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7ce56640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdf7806eb70 0x7fdf7815ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60672/0 (socket says 172.21.15.133:60672) 2024-04-25T10:03:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7ce56640 1 -- 172.21.15.133:0/1768597101 learned_addr learned my addr 172.21.15.133:0/1768597101 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf77fff640 1 --2- 172.21.15.133:0/1768597101 >> v2:172.21.15.162:3300/0 conn(0x7fdf78158110 0x7fdf780b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7ce56640 1 -- 172.21.15.133:0/1768597101 >> v2:172.21.15.162:3300/0 conn(0x7fdf78158110 msgr2=0x7fdf780b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7ce56640 1 --2- 172.21.15.133:0/1768597101 >> v2:172.21.15.162:3300/0 conn(0x7fdf78158110 0x7fdf780b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7ce56640 1 -- 172.21.15.133:0/1768597101 >> v1:172.21.15.162:6789/0 conn(0x7fdf781584f0 legacy=0x7fdf7806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7ce56640 1 -- 172.21.15.133:0/1768597101 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7815b640 con 0x7fdf7806eb70 2024-04-25T10:03:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf7ce56640 1 --2- 172.21.15.133:0/1768597101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf7806eb70 0x7fdf7815ad10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdf780b4d40 tx=0x7fdf7001eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64dab7f51394e14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.640+0000 7fdf777fe640 1 -- 172.21.15.133:0/1768597101 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000b2d0 con 0x7fdf7806eb70 2024-04-25T10:03:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf777fe640 1 -- 172.21.15.133:0/1768597101 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7000b470 con 0x7fdf7806eb70 2024-04-25T10:03:21.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf777fe640 1 -- 172.21.15.133:0/1768597101 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700a7b90 con 0x7fdf7806eb70 2024-04-25T10:03:21.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1768597101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf7806eb70 msgr2=0x7fdf7815ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 --2- 172.21.15.133:0/1768597101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf7806eb70 0x7fdf7815ad10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdf780b4d40 tx=0x7fdf7001eb80 comp rx=0 tx=0).stop 2024-04-25T10:03:21.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1768597101 shutdown_connections 2024-04-25T10:03:21.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 --2- 172.21.15.133:0/1768597101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf7806eb70 0x7fdf7815ad10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 --2- 172.21.15.133:0/1768597101 >> v2:172.21.15.162:3300/0 conn(0x7fdf78158110 0x7fdf780b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1768597101 >> 172.21.15.133:0/1768597101 conn(0x7fdf78067000 msgr2=0x7fdf7806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1768597101 shutdown_connections 2024-04-25T10:03:21.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1768597101 wait complete. 2024-04-25T10:03:21.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 Processor -- start 2024-04-25T10:03:21.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 -- start start 2024-04-25T10:03:21.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdf7806eb70 0x7fdf7814e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf78158110 0x7fdf780c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf781584f0 0x7fdf78156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7806dce0 con 0x7fdf781584f0 2024-04-25T10:03:21.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7806db60 con 0x7fdf78158110 2024-04-25T10:03:21.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7806de60 con 0x7fdf7806eb70 2024-04-25T10:03:21.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7ce56640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf781584f0 0x7fdf78156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7ce56640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf781584f0 0x7fdf78156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60686/0 (socket says 172.21.15.133:60686) 2024-04-25T10:03:21.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7ce56640 1 -- 172.21.15.133:0/1623137316 learned_addr learned my addr 172.21.15.133:0/1623137316 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:21.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf77fff640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdf7806eb70 0x7fdf7814e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf6f7fe640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf78158110 0x7fdf780c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7ce56640 1 -- 172.21.15.133:0/1623137316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdf7806eb70 msgr2=0x7fdf7814e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7ce56640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdf7806eb70 0x7fdf7814e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7ce56640 1 -- 172.21.15.133:0/1623137316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf78158110 msgr2=0x7fdf780c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7ce56640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf78158110 0x7fdf780c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7ce56640 1 -- 172.21.15.133:0/1623137316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7814ed00 con 0x7fdf781584f0 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf77fff640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdf7806eb70 0x7fdf7814e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf6f7fe640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf78158110 0x7fdf780c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7ce56640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf781584f0 0x7fdf78156990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdf7001c7b0 tx=0x7fdf7001b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf757fa640 1 -- 172.21.15.133:0/1623137316 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70009800 con 0x7fdf781584f0 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf757fa640 1 -- 172.21.15.133:0/1623137316 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf700c0070 con 0x7fdf781584f0 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1623137316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7815beb0 con 0x7fdf781584f0 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf757fa640 1 -- 172.21.15.133:0/1623137316 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70009c00 con 0x7fdf781584f0 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.644+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1623137316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7815c470 con 0x7fdf781584f0 2024-04-25T10:03:21.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.648+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1623137316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf78068ef0 con 0x7fdf781584f0 2024-04-25T10:03:21.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.648+0000 7fdf757fa640 1 -- 172.21.15.133:0/1623137316 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf7000b2e0 con 0x7fdf781584f0 2024-04-25T10:03:21.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.648+0000 7fdf757fa640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdf58066f60 0x7fdf58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.648+0000 7fdf757fa640 1 -- 172.21.15.133:0/1623137316 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdf70122fa0 con 0x7fdf781584f0 2024-04-25T10:03:21.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.648+0000 7fdf77fff640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdf58066f60 0x7fdf58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.648+0000 7fdf77fff640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdf58066f60 0x7fdf58069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf780b4790 tx=0x7fdf68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.652+0000 7fdf757fa640 1 -- 172.21.15.133:0/1623137316 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf700187a0 con 0x7fdf781584f0 2024-04-25T10:03:21.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.792+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1623137316 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf78158930 con 0x7fdf58066f60 2024-04-25T10:03:21.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.800+0000 7fdf757fa640 1 -- 172.21.15.133:0/1623137316 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251914 (secure 0 0 0) 0x7fdf78158930 con 0x7fdf58066f60 2024-04-25T10:03:21.809 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:03:21.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1623137316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdf58066f60 msgr2=0x7fdf58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdf58066f60 0x7fdf58069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf780b4790 tx=0x7fdf68008040 comp rx=0 tx=0).stop 2024-04-25T10:03:21.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1623137316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf781584f0 msgr2=0x7fdf78156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf781584f0 0x7fdf78156990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdf7001c7b0 tx=0x7fdf7001b920 comp rx=0 tx=0).stop 2024-04-25T10:03:21.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1623137316 shutdown_connections 2024-04-25T10:03:21.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdf58066f60 0x7fdf58069420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf781584f0 0x7fdf78156990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf78158110 0x7fdf780c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 --2- 172.21.15.133:0/1623137316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdf7806eb70 0x7fdf7814e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1623137316 >> 172.21.15.133:0/1623137316 conn(0x7fdf78067000 msgr2=0x7fdf78152c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1623137316 shutdown_connections 2024-04-25T10:03:21.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.804+0000 7fdf7e8b9640 1 -- 172.21.15.133:0/1623137316 wait complete. 2024-04-25T10:03:21.827 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-04-25T10:03:21.731806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220792,"kb_used_data":4576,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753928,"statfs":{"total":773094113280,"available":772868022272,"internally_reserved":0,"allocated":4685824,"data_stored":2665628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":53,"apply_latency_ms":53,"commit_latency_ns":53000000,"apply_latency_ns":53000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.335198"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.461434+0000","last_change":"2024-04-25T10:03:18.455252+0000","last_active":"2024-04-25T10:03:18.461434+0000","last_peered":"2024-04-25T10:03:18.461434+0000","last_clean":"2024-04-25T10:03:18.461434+0000","last_became_active":"2024-04-25T10:03:18.454478+0000","last_became_peered":"2024-04-25T10:03:18.454478+0000","last_unstale":"2024-04-25T10:03:18.461434+0000","last_undegraded":"2024-04-25T10:03:18.461434+0000","last_fullsized":"2024-04-25T10:03:18.461434+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:06.842871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.461299+0000","last_change":"2024-04-25T10:03:18.453281+0000","last_active":"2024-04-25T10:03:18.461299+0000","last_peered":"2024-04-25T10:03:18.461299+0000","last_clean":"2024-04-25T10:03:18.461299+0000","last_became_active":"2024-04-25T10:03:18.452444+0000","last_became_peered":"2024-04-25T10:03:18.452444+0000","last_unstale":"2024-04-25T10:03:18.461299+0000","last_undegraded":"2024-04-25T10:03:18.461299+0000","last_fullsized":"2024-04-25T10:03:18.461299+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:29.161301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.456161+0000","last_change":"2024-04-25T10:03:18.453585+0000","last_active":"2024-04-25T10:03:18.456161+0000","last_peered":"2024-04-25T10:03:18.456161+0000","last_clean":"2024-04-25T10:03:18.456161+0000","last_became_active":"2024-04-25T10:03:18.453057+0000","last_became_peered":"2024-04-25T10:03:18.453057+0000","last_unstale":"2024-04-25T10:03:18.456161+0000","last_undegraded":"2024-04-25T10:03:18.456161+0000","last_fullsized":"2024-04-25T10:03:18.456161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:21.056790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.417802+0000","last_change":"2024-04-25T10:03:18.439642+0000","last_active":"2024-04-25T10:03:19.417802+0000","last_peered":"2024-04-25T10:03:19.417802+0000","last_clean":"2024-04-25T10:03:19.417802+0000","last_became_active":"2024-04-25T10:03:18.438834+0000","last_became_peered":"2024-04-25T10:03:18.438834+0000","last_unstale":"2024-04-25T10:03:19.417802+0000","last_undegraded":"2024-04-25T10:03:19.417802+0000","last_fullsized":"2024-04-25T10:03:19.417802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:33.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.415324+0000","last_change":"2024-04-25T10:03:18.435457+0000","last_active":"2024-04-25T10:03:19.415324+0000","last_peered":"2024-04-25T10:03:19.415324+0000","last_clean":"2024-04-25T10:03:19.415324+0000","last_became_active":"2024-04-25T10:03:18.434745+0000","last_became_peered":"2024-04-25T10:03:18.434745+0000","last_unstale":"2024-04-25T10:03:19.415324+0000","last_undegraded":"2024-04-25T10:03:19.415324+0000","last_fullsized":"2024-04-25T10:03:19.415324+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:28.675784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.459691+0000","last_change":"2024-04-25T10:03:18.458896+0000","last_active":"2024-04-25T10:03:18.459691+0000","last_peered":"2024-04-25T10:03:18.459691+0000","last_clean":"2024-04-25T10:03:18.459691+0000","last_became_active":"2024-04-25T10:03:18.458602+0000","last_became_peered":"2024-04-25T10:03:18.458602+0000","last_unstale":"2024-04-25T10:03:18.459691+0000","last_undegraded":"2024-04-25T10:03:18.459691+0000","last_fullsized":"2024-04-25T10:03:18.459691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:40.331322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.477958+0000","last_change":"2024-04-25T10:03:18.460521+0000","last_active":"2024-04-25T10:03:18.477958+0000","last_peered":"2024-04-25T10:03:18.477958+0000","last_clean":"2024-04-25T10:03:18.477958+0000","last_became_active":"2024-04-25T10:03:18.459923+0000","last_became_peered":"2024-04-25T10:03:18.459923+0000","last_unstale":"2024-04-25T10:03:18.477958+0000","last_undegraded":"2024-04-25T10:03:18.477958+0000","last_fullsized":"2024-04-25T10:03:18.477958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:12.950362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.460735+0000","last_change":"2024-04-25T10:03:18.452167+0000","last_active":"2024-04-25T10:03:18.460735+0000","last_peered":"2024-04-25T10:03:18.460735+0000","last_clean":"2024-04-25T10:03:18.460735+0000","last_became_active":"2024-04-25T10:03:18.451448+0000","last_became_peered":"2024-04-25T10:03:18.451448+0000","last_unstale":"2024-04-25T10:03:18.460735+0000","last_undegraded":"2024-04-25T10:03:18.460735+0000","last_fullsized":"2024-04-25T10:03:18.460735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:57.787758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.456197+0000","last_change":"2024-04-25T10:03:18.453561+0000","last_active":"2024-04-25T10:03:18.456197+0000","last_peered":"2024-04-25T10:03:18.456197+0000","last_clean":"2024-04-25T10:03:18.456197+0000","last_became_active":"2024-04-25T10:03:18.452854+0000","last_became_peered":"2024-04-25T10:03:18.452854+0000","last_unstale":"2024-04-25T10:03:18.456197+0000","last_undegraded":"2024-04-25T10:03:18.456197+0000","last_fullsized":"2024-04-25T10:03:18.456197+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:38.115465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.473024+0000","last_change":"2024-04-25T10:03:18.440971+0000","last_active":"2024-04-25T10:03:18.473024+0000","last_peered":"2024-04-25T10:03:18.473024+0000","last_clean":"2024-04-25T10:03:18.473024+0000","last_became_active":"2024-04-25T10:03:18.440192+0000","last_became_peered":"2024-04-25T10:03:18.440192+0000","last_unstale":"2024-04-25T10:03:18.473024+0000","last_undegraded":"2024-04-25T10:03:18.473024+0000","last_fullsized":"2024-04-25T10:03:18.473024+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:26.718807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.477681+0000","last_change":"2024-04-25T10:03:18.454644+0000","last_active":"2024-04-25T10:03:18.477681+0000","last_peered":"2024-04-25T10:03:18.477681+0000","last_clean":"2024-04-25T10:03:18.477681+0000","last_became_active":"2024-04-25T10:03:18.453993+0000","last_became_peered":"2024-04-25T10:03:18.453993+0000","last_unstale":"2024-04-25T10:03:18.477681+0000","last_undegraded":"2024-04-25T10:03:18.477681+0000","last_fullsized":"2024-04-25T10:03:18.477681+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:04.585731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.445447+0000","last_change":"2024-04-25T10:03:18.442672+0000","last_active":"2024-04-25T10:03:18.445447+0000","last_peered":"2024-04-25T10:03:18.445447+0000","last_clean":"2024-04-25T10:03:18.445447+0000","last_became_active":"2024-04-25T10:03:18.441911+0000","last_became_peered":"2024-04-25T10:03:18.441911+0000","last_unstale":"2024-04-25T10:03:18.445447+0000","last_undegraded":"2024-04-25T10:03:18.445447+0000","last_fullsized":"2024-04-25T10:03:18.445447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:00.791407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.478803+0000","last_change":"2024-04-25T10:03:18.471929+0000","last_active":"2024-04-25T10:03:18.478803+0000","last_peered":"2024-04-25T10:03:18.478803+0000","last_clean":"2024-04-25T10:03:18.478803+0000","last_became_active":"2024-04-25T10:03:18.470613+0000","last_became_peered":"2024-04-25T10:03:18.470613+0000","last_unstale":"2024-04-25T10:03:18.478803+0000","last_undegraded":"2024-04-25T10:03:18.478803+0000","last_fullsized":"2024-04-25T10:03:18.478803+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:52.383214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.462847+0000","last_change":"2024-04-25T10:03:18.444789+0000","last_active":"2024-04-25T10:03:18.462847+0000","last_peered":"2024-04-25T10:03:18.462847+0000","last_clean":"2024-04-25T10:03:18.462847+0000","last_became_active":"2024-04-25T10:03:18.437842+0000","last_became_peered":"2024-04-25T10:03:18.437842+0000","last_unstale":"2024-04-25T10:03:18.462847+0000","last_undegraded":"2024-04-25T10:03:18.462847+0000","last_fullsized":"2024-04-25T10:03:18.462847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.427033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.474561+0000","last_change":"2024-04-25T10:03:18.448270+0000","last_active":"2024-04-25T10:03:18.474561+0000","last_peered":"2024-04-25T10:03:18.474561+0000","last_clean":"2024-04-25T10:03:18.474561+0000","last_became_active":"2024-04-25T10:03:18.440945+0000","last_became_peered":"2024-04-25T10:03:18.440945+0000","last_unstale":"2024-04-25T10:03:18.474561+0000","last_undegraded":"2024-04-25T10:03:18.474561+0000","last_fullsized":"2024-04-25T10:03:18.474561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:57.787758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.417253+0000","last_change":"2024-04-25T10:03:18.457259+0000","last_active":"2024-04-25T10:03:19.417253+0000","last_peered":"2024-04-25T10:03:19.417253+0000","last_clean":"2024-04-25T10:03:19.417253+0000","last_became_active":"2024-04-25T10:03:18.456340+0000","last_became_peered":"2024-04-25T10:03:18.456340+0000","last_unstale":"2024-04-25T10:03:19.417253+0000","last_undegraded":"2024-04-25T10:03:19.417253+0000","last_fullsized":"2024-04-25T10:03:19.417253+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:06.842871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.419698+0000","last_change":"2024-04-25T10:03:18.475461+0000","last_active":"2024-04-25T10:03:19.419698+0000","last_peered":"2024-04-25T10:03:19.419698+0000","last_clean":"2024-04-25T10:03:19.419698+0000","last_became_active":"2024-04-25T10:03:18.473847+0000","last_became_peered":"2024-04-25T10:03:18.473847+0000","last_unstale":"2024-04-25T10:03:19.419698+0000","last_undegraded":"2024-04-25T10:03:19.419698+0000","last_fullsized":"2024-04-25T10:03:19.419698+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:30.756228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.419306+0000","last_change":"2024-04-25T10:03:18.445321+0000","last_active":"2024-04-25T10:03:19.419306+0000","last_peered":"2024-04-25T10:03:19.419306+0000","last_clean":"2024-04-25T10:03:19.419306+0000","last_became_active":"2024-04-25T10:03:18.437701+0000","last_became_peered":"2024-04-25T10:03:18.437701+0000","last_unstale":"2024-04-25T10:03:19.419306+0000","last_undegraded":"2024-04-25T10:03:19.419306+0000","last_fullsized":"2024-04-25T10:03:19.419306+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:14.854586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.415670+0000","last_change":"2024-04-25T10:03:18.439505+0000","last_active":"2024-04-25T10:03:19.415670+0000","last_peered":"2024-04-25T10:03:19.415670+0000","last_clean":"2024-04-25T10:03:19.415670+0000","last_became_active":"2024-04-25T10:03:18.438632+0000","last_became_peered":"2024-04-25T10:03:18.438632+0000","last_unstale":"2024-04-25T10:03:19.415670+0000","last_undegraded":"2024-04-25T10:03:19.415670+0000","last_fullsized":"2024-04-25T10:03:19.415670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:33.500013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.418809+0000","last_change":"2024-04-25T10:03:18.436280+0000","last_active":"2024-04-25T10:03:19.418809+0000","last_peered":"2024-04-25T10:03:19.418809+0000","last_clean":"2024-04-25T10:03:19.418809+0000","last_became_active":"2024-04-25T10:03:18.435469+0000","last_became_peered":"2024-04-25T10:03:18.435469+0000","last_unstale":"2024-04-25T10:03:19.418809+0000","last_undegraded":"2024-04-25T10:03:19.418809+0000","last_fullsized":"2024-04-25T10:03:19.418809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:21.056790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.477215+0000","last_change":"2024-04-25T10:03:18.450547+0000","last_active":"2024-04-25T10:03:18.477215+0000","last_peered":"2024-04-25T10:03:18.477215+0000","last_clean":"2024-04-25T10:03:18.477215+0000","last_became_active":"2024-04-25T10:03:18.449869+0000","last_became_peered":"2024-04-25T10:03:18.449869+0000","last_unstale":"2024-04-25T10:03:18.477215+0000","last_undegraded":"2024-04-25T10:03:18.477215+0000","last_fullsized":"2024-04-25T10:03:18.477215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:29.161301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.463228+0000","last_change":"2024-04-25T10:03:18.446066+0000","last_active":"2024-04-25T10:03:18.463228+0000","last_peered":"2024-04-25T10:03:18.463228+0000","last_clean":"2024-04-25T10:03:18.463228+0000","last_became_active":"2024-04-25T10:03:18.445407+0000","last_became_peered":"2024-04-25T10:03:18.445407+0000","last_unstale":"2024-04-25T10:03:18.463228+0000","last_undegraded":"2024-04-25T10:03:18.463228+0000","last_fullsized":"2024-04-25T10:03:18.463228+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:38.115465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.418177+0000","last_change":"2024-04-25T10:03:18.443656+0000","last_active":"2024-04-25T10:03:19.418177+0000","last_peered":"2024-04-25T10:03:19.418177+0000","last_clean":"2024-04-25T10:03:19.418177+0000","last_became_active":"2024-04-25T10:03:18.442931+0000","last_became_peered":"2024-04-25T10:03:18.442931+0000","last_unstale":"2024-04-25T10:03:19.418177+0000","last_undegraded":"2024-04-25T10:03:19.418177+0000","last_fullsized":"2024-04-25T10:03:19.418177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:14.650267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.415335+0000","last_change":"2024-04-25T10:03:18.438601+0000","last_active":"2024-04-25T10:03:19.415335+0000","last_peered":"2024-04-25T10:03:19.415335+0000","last_clean":"2024-04-25T10:03:19.415335+0000","last_became_active":"2024-04-25T10:03:18.437808+0000","last_became_peered":"2024-04-25T10:03:18.437808+0000","last_unstale":"2024-04-25T10:03:19.415335+0000","last_undegraded":"2024-04-25T10:03:19.415335+0000","last_fullsized":"2024-04-25T10:03:19.415335+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:28.983976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.419189+0000","last_change":"2024-04-25T10:03:18.446224+0000","last_active":"2024-04-25T10:03:19.419189+0000","last_peered":"2024-04-25T10:03:19.419189+0000","last_clean":"2024-04-25T10:03:19.419189+0000","last_became_active":"2024-04-25T10:03:18.445244+0000","last_became_peered":"2024-04-25T10:03:18.445244+0000","last_unstale":"2024-04-25T10:03:19.419189+0000","last_undegraded":"2024-04-25T10:03:19.419189+0000","last_fullsized":"2024-04-25T10:03:19.419189+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:14.452300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.458033+0000","last_change":"2024-04-25T10:03:18.452477+0000","last_active":"2024-04-25T10:03:18.458033+0000","last_peered":"2024-04-25T10:03:18.458033+0000","last_clean":"2024-04-25T10:03:18.458033+0000","last_became_active":"2024-04-25T10:03:18.452144+0000","last_became_peered":"2024-04-25T10:03:18.452144+0000","last_unstale":"2024-04-25T10:03:18.458033+0000","last_undegraded":"2024-04-25T10:03:18.458033+0000","last_fullsized":"2024-04-25T10:03:18.458033+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:26.718807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.419053+0000","last_change":"2024-04-25T10:03:18.429703+0000","last_active":"2024-04-25T10:03:19.419053+0000","last_peered":"2024-04-25T10:03:19.419053+0000","last_clean":"2024-04-25T10:03:19.419053+0000","last_became_active":"2024-04-25T10:03:18.428109+0000","last_became_peered":"2024-04-25T10:03:18.428109+0000","last_unstale":"2024-04-25T10:03:19.419053+0000","last_undegraded":"2024-04-25T10:03:19.419053+0000","last_fullsized":"2024-04-25T10:03:19.419053+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:31.249126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.478650+0000","last_change":"2024-04-25T10:03:18.460712+0000","last_active":"2024-04-25T10:03:18.478650+0000","last_peered":"2024-04-25T10:03:18.478650+0000","last_clean":"2024-04-25T10:03:18.478650+0000","last_became_active":"2024-04-25T10:03:18.459966+0000","last_became_peered":"2024-04-25T10:03:18.459966+0000","last_unstale":"2024-04-25T10:03:18.478650+0000","last_undegraded":"2024-04-25T10:03:18.478650+0000","last_fullsized":"2024-04-25T10:03:18.478650+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:42.091286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.418216+0000","last_change":"2024-04-25T10:03:18.452350+0000","last_active":"2024-04-25T10:03:19.418216+0000","last_peered":"2024-04-25T10:03:19.418216+0000","last_clean":"2024-04-25T10:03:19.418216+0000","last_became_active":"2024-04-25T10:03:18.447714+0000","last_became_peered":"2024-04-25T10:03:18.447714+0000","last_unstale":"2024-04-25T10:03:19.418216+0000","last_undegraded":"2024-04-25T10:03:19.418216+0000","last_fullsized":"2024-04-25T10:03:19.418216+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:38.115465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.417202+0000","last_change":"2024-04-25T10:03:18.455930+0000","last_active":"2024-04-25T10:03:19.417202+0000","last_peered":"2024-04-25T10:03:19.417202+0000","last_clean":"2024-04-25T10:03:19.417202+0000","last_became_active":"2024-04-25T10:03:18.455147+0000","last_became_peered":"2024-04-25T10:03:18.455147+0000","last_unstale":"2024-04-25T10:03:19.417202+0000","last_undegraded":"2024-04-25T10:03:19.417202+0000","last_fullsized":"2024-04-25T10:03:19.417202+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.427033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.418059+0000","last_change":"2024-04-25T10:03:18.457286+0000","last_active":"2024-04-25T10:03:19.418059+0000","last_peered":"2024-04-25T10:03:19.418059+0000","last_clean":"2024-04-25T10:03:19.418059+0000","last_became_active":"2024-04-25T10:03:18.443869+0000","last_became_peered":"2024-04-25T10:03:18.443869+0000","last_unstale":"2024-04-25T10:03:19.418059+0000","last_undegraded":"2024-04-25T10:03:19.418059+0000","last_fullsized":"2024-04-25T10:03:19.418059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:52.383214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.444074+0000","last_change":"2024-04-25T10:03:18.431355+0000","last_active":"2024-04-25T10:03:18.444074+0000","last_peered":"2024-04-25T10:03:18.444074+0000","last_clean":"2024-04-25T10:03:18.444074+0000","last_became_active":"2024-04-25T10:03:18.430626+0000","last_became_peered":"2024-04-25T10:03:18.430626+0000","last_unstale":"2024-04-25T10:03:18.444074+0000","last_undegraded":"2024-04-25T10:03:18.444074+0000","last_fullsized":"2024-04-25T10:03:18.444074+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:31.249126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.419896+0000","last_change":"2024-04-25T10:03:18.475080+0000","last_active":"2024-04-25T10:03:19.419896+0000","last_peered":"2024-04-25T10:03:19.419896+0000","last_clean":"2024-04-25T10:03:19.419896+0000","last_became_active":"2024-04-25T10:03:18.474468+0000","last_became_peered":"2024-04-25T10:03:18.474468+0000","last_unstale":"2024-04-25T10:03:19.419896+0000","last_undegraded":"2024-04-25T10:03:19.419896+0000","last_fullsized":"2024-04-25T10:03:19.419896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:27.834303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.419274+0000","last_change":"2024-04-25T10:03:17.435411+0000","last_active":"2024-04-25T10:03:19.419274+0000","last_peered":"2024-04-25T10:03:19.419274+0000","last_clean":"2024-04-25T10:03:19.419274+0000","last_became_active":"2024-04-25T10:03:17.434687+0000","last_became_peered":"2024-04-25T10:03:17.434687+0000","last_unstale":"2024-04-25T10:03:19.419274+0000","last_undegraded":"2024-04-25T10:03:19.419274+0000","last_fullsized":"2024-04-25T10:03:19.419274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:32.572630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.458139+0000","last_change":"2024-04-25T10:03:18.453275+0000","last_active":"2024-04-25T10:03:18.458139+0000","last_peered":"2024-04-25T10:03:18.458139+0000","last_clean":"2024-04-25T10:03:18.458139+0000","last_became_active":"2024-04-25T10:03:18.452430+0000","last_became_peered":"2024-04-25T10:03:18.452430+0000","last_unstale":"2024-04-25T10:03:18.458139+0000","last_undegraded":"2024-04-25T10:03:18.458139+0000","last_fullsized":"2024-04-25T10:03:18.458139+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:39.174456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.484393+0000","last_change":"2024-04-25T10:03:17.432006+0000","last_active":"2024-04-25T10:03:18.484393+0000","last_peered":"2024-04-25T10:03:18.484393+0000","last_clean":"2024-04-25T10:03:18.484393+0000","last_became_active":"2024-04-25T10:03:17.431718+0000","last_became_peered":"2024-04-25T10:03:17.431718+0000","last_unstale":"2024-04-25T10:03:18.484393+0000","last_undegraded":"2024-04-25T10:03:18.484393+0000","last_fullsized":"2024-04-25T10:03:18.484393+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:00.127480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.453347+0000","last_change":"2024-04-25T10:03:18.436945+0000","last_active":"2024-04-25T10:03:18.453347+0000","last_peered":"2024-04-25T10:03:18.453347+0000","last_clean":"2024-04-25T10:03:18.453347+0000","last_became_active":"2024-04-25T10:03:18.436078+0000","last_became_peered":"2024-04-25T10:03:18.436078+0000","last_unstale":"2024-04-25T10:03:18.453347+0000","last_undegraded":"2024-04-25T10:03:18.453347+0000","last_fullsized":"2024-04-25T10:03:18.453347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:01.138064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.481114+0000","last_change":"2024-04-25T10:03:17.442029+0000","last_active":"2024-04-25T10:03:18.481114+0000","last_peered":"2024-04-25T10:03:18.481114+0000","last_clean":"2024-04-25T10:03:18.481114+0000","last_became_active":"2024-04-25T10:03:17.441125+0000","last_became_peered":"2024-04-25T10:03:17.441125+0000","last_unstale":"2024-04-25T10:03:18.481114+0000","last_undegraded":"2024-04-25T10:03:18.481114+0000","last_fullsized":"2024-04-25T10:03:18.481114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:20.046205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.461447+0000","last_change":"2024-04-25T10:03:18.457641+0000","last_active":"2024-04-25T10:03:18.461447+0000","last_peered":"2024-04-25T10:03:18.461447+0000","last_clean":"2024-04-25T10:03:18.461447+0000","last_became_active":"2024-04-25T10:03:18.454714+0000","last_became_peered":"2024-04-25T10:03:18.454714+0000","last_unstale":"2024-04-25T10:03:18.461447+0000","last_undegraded":"2024-04-25T10:03:18.461447+0000","last_fullsized":"2024-04-25T10:03:18.461447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:14.650267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.478436+0000","last_change":"2024-04-25T10:03:17.441384+0000","last_active":"2024-04-25T10:03:18.478436+0000","last_peered":"2024-04-25T10:03:18.478436+0000","last_clean":"2024-04-25T10:03:18.478436+0000","last_became_active":"2024-04-25T10:03:17.440139+0000","last_became_peered":"2024-04-25T10:03:17.440139+0000","last_unstale":"2024-04-25T10:03:18.478436+0000","last_undegraded":"2024-04-25T10:03:18.478436+0000","last_fullsized":"2024-04-25T10:03:18.478436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:11.956244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.448122+0000","last_change":"2024-04-25T10:03:18.446063+0000","last_active":"2024-04-25T10:03:18.448122+0000","last_peered":"2024-04-25T10:03:18.448122+0000","last_clean":"2024-04-25T10:03:18.448122+0000","last_became_active":"2024-04-25T10:03:18.445311+0000","last_became_peered":"2024-04-25T10:03:18.445311+0000","last_unstale":"2024-04-25T10:03:18.448122+0000","last_undegraded":"2024-04-25T10:03:18.448122+0000","last_fullsized":"2024-04-25T10:03:18.448122+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:35.388687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.478697+0000","last_change":"2024-04-25T10:03:17.453589+0000","last_active":"2024-04-25T10:03:18.478697+0000","last_peered":"2024-04-25T10:03:18.478697+0000","last_clean":"2024-04-25T10:03:18.478697+0000","last_became_active":"2024-04-25T10:03:17.447710+0000","last_became_peered":"2024-04-25T10:03:17.447710+0000","last_unstale":"2024-04-25T10:03:18.478697+0000","last_undegraded":"2024-04-25T10:03:18.478697+0000","last_fullsized":"2024-04-25T10:03:18.478697+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.238541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.474529+0000","last_change":"2024-04-25T10:03:18.448215+0000","last_active":"2024-04-25T10:03:18.474529+0000","last_peered":"2024-04-25T10:03:18.474529+0000","last_clean":"2024-04-25T10:03:18.474529+0000","last_became_active":"2024-04-25T10:03:18.440838+0000","last_became_peered":"2024-04-25T10:03:18.440838+0000","last_unstale":"2024-04-25T10:03:18.474529+0000","last_undegraded":"2024-04-25T10:03:18.474529+0000","last_fullsized":"2024-04-25T10:03:18.474529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:57.380955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.419205+0000","last_change":"2024-04-25T10:03:17.450306+0000","last_active":"2024-04-25T10:03:19.419205+0000","last_peered":"2024-04-25T10:03:19.419205+0000","last_clean":"2024-04-25T10:03:19.419205+0000","last_became_active":"2024-04-25T10:03:17.449619+0000","last_became_peered":"2024-04-25T10:03:17.449619+0000","last_unstale":"2024-04-25T10:03:19.419205+0000","last_undegraded":"2024-04-25T10:03:19.419205+0000","last_fullsized":"2024-04-25T10:03:19.419205+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.238541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.463280+0000","last_change":"2024-04-25T10:03:18.446507+0000","last_active":"2024-04-25T10:03:18.463280+0000","last_peered":"2024-04-25T10:03:18.463280+0000","last_clean":"2024-04-25T10:03:18.463280+0000","last_became_active":"2024-04-25T10:03:18.445714+0000","last_became_peered":"2024-04-25T10:03:18.445714+0000","last_unstale":"2024-04-25T10:03:18.463280+0000","last_undegraded":"2024-04-25T10:03:18.463280+0000","last_fullsized":"2024-04-25T10:03:18.463280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:21.056790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.426257+0000","last_change":"2024-04-25T10:03:17.458394+0000","last_active":"2024-04-25T10:03:18.426257+0000","last_peered":"2024-04-25T10:03:18.426257+0000","last_clean":"2024-04-25T10:03:18.426257+0000","last_became_active":"2024-04-25T10:03:17.458153+0000","last_became_peered":"2024-04-25T10:03:17.458153+0000","last_unstale":"2024-04-25T10:03:18.426257+0000","last_undegraded":"2024-04-25T10:03:18.426257+0000","last_fullsized":"2024-04-25T10:03:18.426257+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:37.104880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.461517+0000","last_change":"2024-04-25T10:03:18.454715+0000","last_active":"2024-04-25T10:03:18.461517+0000","last_peered":"2024-04-25T10:03:18.461517+0000","last_clean":"2024-04-25T10:03:18.461517+0000","last_became_active":"2024-04-25T10:03:18.454192+0000","last_became_peered":"2024-04-25T10:03:18.454192+0000","last_unstale":"2024-04-25T10:03:18.461517+0000","last_undegraded":"2024-04-25T10:03:18.461517+0000","last_fullsized":"2024-04-25T10:03:18.461517+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:12.950362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.422918+0000","last_change":"2024-04-25T10:03:17.445479+0000","last_active":"2024-04-25T10:03:18.422918+0000","last_peered":"2024-04-25T10:03:18.422918+0000","last_clean":"2024-04-25T10:03:18.422918+0000","last_became_active":"2024-04-25T10:03:17.441982+0000","last_became_peered":"2024-04-25T10:03:17.441982+0000","last_unstale":"2024-04-25T10:03:18.422918+0000","last_undegraded":"2024-04-25T10:03:18.422918+0000","last_fullsized":"2024-04-25T10:03:18.422918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:32.572630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.417999+0000","last_change":"2024-04-25T10:03:18.438848+0000","last_active":"2024-04-25T10:03:19.417999+0000","last_peered":"2024-04-25T10:03:19.417999+0000","last_clean":"2024-04-25T10:03:19.417999+0000","last_became_active":"2024-04-25T10:03:18.438088+0000","last_became_peered":"2024-04-25T10:03:18.438088+0000","last_unstale":"2024-04-25T10:03:19.417999+0000","last_undegraded":"2024-04-25T10:03:19.417999+0000","last_fullsized":"2024-04-25T10:03:19.417999+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:12.966828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.451657+0000","last_change":"2024-04-25T10:03:17.425211+0000","last_active":"2024-04-25T10:03:18.451657+0000","last_peered":"2024-04-25T10:03:18.451657+0000","last_clean":"2024-04-25T10:03:18.451657+0000","last_became_active":"2024-04-25T10:03:17.424787+0000","last_became_peered":"2024-04-25T10:03:17.424787+0000","last_unstale":"2024-04-25T10:03:18.451657+0000","last_undegraded":"2024-04-25T10:03:18.451657+0000","last_fullsized":"2024-04-25T10:03:18.451657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:39.320738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.420116+0000","last_change":"2024-04-25T10:03:18.468300+0000","last_active":"2024-04-25T10:03:19.420116+0000","last_peered":"2024-04-25T10:03:19.420116+0000","last_clean":"2024-04-25T10:03:19.420116+0000","last_became_active":"2024-04-25T10:03:18.458083+0000","last_became_peered":"2024-04-25T10:03:18.458083+0000","last_unstale":"2024-04-25T10:03:19.420116+0000","last_undegraded":"2024-04-25T10:03:19.420116+0000","last_fullsized":"2024-04-25T10:03:19.420116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:26.329005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.440649+0000","last_change":"2024-04-25T10:03:17.457867+0000","last_active":"2024-04-25T10:03:18.440649+0000","last_peered":"2024-04-25T10:03:18.440649+0000","last_clean":"2024-04-25T10:03:18.440649+0000","last_became_active":"2024-04-25T10:03:17.456720+0000","last_became_peered":"2024-04-25T10:03:17.456720+0000","last_unstale":"2024-04-25T10:03:18.440649+0000","last_undegraded":"2024-04-25T10:03:18.440649+0000","last_fullsized":"2024-04-25T10:03:18.440649+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:13.844001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.479191+0000","last_change":"2024-04-25T10:03:18.478120+0000","last_active":"2024-04-25T10:03:18.479191+0000","last_peered":"2024-04-25T10:03:18.479191+0000","last_clean":"2024-04-25T10:03:18.479191+0000","last_became_active":"2024-04-25T10:03:18.477898+0000","last_became_peered":"2024-04-25T10:03:18.477898+0000","last_unstale":"2024-04-25T10:03:18.479191+0000","last_undegraded":"2024-04-25T10:03:18.479191+0000","last_fullsized":"2024-04-25T10:03:18.479191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:14.854586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.478551+0000","last_change":"2024-04-25T10:03:17.457494+0000","last_active":"2024-04-25T10:03:18.478551+0000","last_peered":"2024-04-25T10:03:18.478551+0000","last_clean":"2024-04-25T10:03:18.478551+0000","last_became_active":"2024-04-25T10:03:17.456716+0000","last_became_peered":"2024-04-25T10:03:17.456716+0000","last_unstale":"2024-04-25T10:03:18.478551+0000","last_undegraded":"2024-04-25T10:03:18.478551+0000","last_fullsized":"2024-04-25T10:03:18.478551+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:13.441715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.453410+0000","last_change":"2024-04-25T10:03:18.437911+0000","last_active":"2024-04-25T10:03:18.453410+0000","last_peered":"2024-04-25T10:03:18.453410+0000","last_clean":"2024-04-25T10:03:18.453410+0000","last_became_active":"2024-04-25T10:03:18.435871+0000","last_became_peered":"2024-04-25T10:03:18.435871+0000","last_unstale":"2024-04-25T10:03:18.453410+0000","last_undegraded":"2024-04-25T10:03:18.453410+0000","last_fullsized":"2024-04-25T10:03:18.453410+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:00.791407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.487095+0000","last_change":"2024-04-25T10:03:17.437879+0000","last_active":"2024-04-25T10:03:18.487095+0000","last_peered":"2024-04-25T10:03:18.487095+0000","last_clean":"2024-04-25T10:03:18.487095+0000","last_became_active":"2024-04-25T10:03:17.437535+0000","last_became_peered":"2024-04-25T10:03:17.437535+0000","last_unstale":"2024-04-25T10:03:18.487095+0000","last_undegraded":"2024-04-25T10:03:18.487095+0000","last_fullsized":"2024-04-25T10:03:18.487095+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:56.370370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.456071+0000","last_change":"2024-04-25T10:03:18.452688+0000","last_active":"2024-04-25T10:03:18.456071+0000","last_peered":"2024-04-25T10:03:18.456071+0000","last_clean":"2024-04-25T10:03:18.456071+0000","last_became_active":"2024-04-25T10:03:18.452385+0000","last_became_peered":"2024-04-25T10:03:18.452385+0000","last_unstale":"2024-04-25T10:03:18.456071+0000","last_undegraded":"2024-04-25T10:03:18.456071+0000","last_fullsized":"2024-04-25T10:03:18.456071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:24.427033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.470110+0000","last_change":"2024-04-25T10:03:17.435194+0000","last_active":"2024-04-25T10:03:18.470110+0000","last_peered":"2024-04-25T10:03:18.470110+0000","last_clean":"2024-04-25T10:03:18.470110+0000","last_became_active":"2024-04-25T10:03:17.434688+0000","last_became_peered":"2024-04-25T10:03:17.434688+0000","last_unstale":"2024-04-25T10:03:18.470110+0000","last_undegraded":"2024-04-25T10:03:18.470110+0000","last_fullsized":"2024-04-25T10:03:18.470110+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:38.163871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.461570+0000","last_change":"2024-04-25T10:03:18.455249+0000","last_active":"2024-04-25T10:03:18.461570+0000","last_peered":"2024-04-25T10:03:18.461570+0000","last_clean":"2024-04-25T10:03:18.461570+0000","last_became_active":"2024-04-25T10:03:18.454997+0000","last_became_peered":"2024-04-25T10:03:18.454997+0000","last_unstale":"2024-04-25T10:03:18.461570+0000","last_undegraded":"2024-04-25T10:03:18.461570+0000","last_fullsized":"2024-04-25T10:03:18.461570+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:42.091286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.424098+0000","last_change":"2024-04-25T10:03:17.455462+0000","last_active":"2024-04-25T10:03:18.424098+0000","last_peered":"2024-04-25T10:03:18.424098+0000","last_clean":"2024-04-25T10:03:18.424098+0000","last_became_active":"2024-04-25T10:03:17.455143+0000","last_became_peered":"2024-04-25T10:03:17.455143+0000","last_unstale":"2024-04-25T10:03:18.424098+0000","last_undegraded":"2024-04-25T10:03:18.424098+0000","last_fullsized":"2024-04-25T10:03:18.424098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:23.416449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.418811+0000","last_change":"2024-04-25T10:03:18.456418+0000","last_active":"2024-04-25T10:03:19.418811+0000","last_peered":"2024-04-25T10:03:19.418811+0000","last_clean":"2024-04-25T10:03:19.418811+0000","last_became_active":"2024-04-25T10:03:18.443995+0000","last_became_peered":"2024-04-25T10:03:18.443995+0000","last_unstale":"2024-04-25T10:03:19.418811+0000","last_undegraded":"2024-04-25T10:03:19.418811+0000","last_fullsized":"2024-04-25T10:03:19.418811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:03.664967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.416918+0000","last_change":"2024-04-25T10:03:17.443110+0000","last_active":"2024-04-25T10:03:19.416918+0000","last_peered":"2024-04-25T10:03:19.416918+0000","last_clean":"2024-04-25T10:03:19.416918+0000","last_became_active":"2024-04-25T10:03:17.442481+0000","last_became_peered":"2024-04-25T10:03:17.442481+0000","last_unstale":"2024-04-25T10:03:19.416918+0000","last_undegraded":"2024-04-25T10:03:19.416918+0000","last_fullsized":"2024-04-25T10:03:19.416918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:25.784994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.461497+0000","last_change":"2024-04-25T10:03:18.454441+0000","last_active":"2024-04-25T10:03:18.461497+0000","last_peered":"2024-04-25T10:03:18.461497+0000","last_clean":"2024-04-25T10:03:18.461497+0000","last_became_active":"2024-04-25T10:03:18.453893+0000","last_became_peered":"2024-04-25T10:03:18.453893+0000","last_unstale":"2024-04-25T10:03:18.461497+0000","last_undegraded":"2024-04-25T10:03:18.461497+0000","last_fullsized":"2024-04-25T10:03:18.461497+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.205127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.417565+0000","last_change":"2024-04-25T10:03:17.442773+0000","last_active":"2024-04-25T10:03:19.417565+0000","last_peered":"2024-04-25T10:03:19.417565+0000","last_clean":"2024-04-25T10:03:19.417565+0000","last_became_active":"2024-04-25T10:03:17.441085+0000","last_became_peered":"2024-04-25T10:03:17.441085+0000","last_unstale":"2024-04-25T10:03:19.417565+0000","last_undegraded":"2024-04-25T10:03:19.417565+0000","last_fullsized":"2024-04-25T10:03:19.417565+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:13.441715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.461417+0000","last_change":"2024-04-25T10:03:18.457074+0000","last_active":"2024-04-25T10:03:18.461417+0000","last_peered":"2024-04-25T10:03:18.461417+0000","last_clean":"2024-04-25T10:03:18.461417+0000","last_became_active":"2024-04-25T10:03:18.456778+0000","last_became_peered":"2024-04-25T10:03:18.456778+0000","last_unstale":"2024-04-25T10:03:18.461417+0000","last_undegraded":"2024-04-25T10:03:18.461417+0000","last_fullsized":"2024-04-25T10:03:18.461417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:52.383214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.480303+0000","last_change":"2024-04-25T10:03:17.457523+0000","last_active":"2024-04-25T10:03:18.480303+0000","last_peered":"2024-04-25T10:03:18.480303+0000","last_clean":"2024-04-25T10:03:18.480303+0000","last_became_active":"2024-04-25T10:03:17.456667+0000","last_became_peered":"2024-04-25T10:03:17.456667+0000","last_unstale":"2024-04-25T10:03:18.480303+0000","last_undegraded":"2024-04-25T10:03:18.480303+0000","last_fullsized":"2024-04-25T10:03:18.480303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:39.320738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.477209+0000","last_change":"2024-04-25T10:03:18.449605+0000","last_active":"2024-04-25T10:03:18.477209+0000","last_peered":"2024-04-25T10:03:18.477209+0000","last_clean":"2024-04-25T10:03:18.477209+0000","last_became_active":"2024-04-25T10:03:18.448882+0000","last_became_peered":"2024-04-25T10:03:18.448882+0000","last_unstale":"2024-04-25T10:03:18.477209+0000","last_undegraded":"2024-04-25T10:03:18.477209+0000","last_fullsized":"2024-04-25T10:03:18.477209+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:48.097429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.481805+0000","last_change":"2024-04-25T10:03:17.459251+0000","last_active":"2024-04-25T10:03:18.481805+0000","last_peered":"2024-04-25T10:03:18.481805+0000","last_clean":"2024-04-25T10:03:18.481805+0000","last_became_active":"2024-04-25T10:03:17.458392+0000","last_became_peered":"2024-04-25T10:03:17.458392+0000","last_unstale":"2024-04-25T10:03:18.481805+0000","last_undegraded":"2024-04-25T10:03:18.481805+0000","last_fullsized":"2024-04-25T10:03:18.481805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:57.881722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.453907+0000","last_change":"2024-04-25T10:03:18.443162+0000","last_active":"2024-04-25T10:03:18.453907+0000","last_peered":"2024-04-25T10:03:18.453907+0000","last_clean":"2024-04-25T10:03:18.453907+0000","last_became_active":"2024-04-25T10:03:18.438347+0000","last_became_peered":"2024-04-25T10:03:18.438347+0000","last_unstale":"2024-04-25T10:03:18.453907+0000","last_undegraded":"2024-04-25T10:03:18.453907+0000","last_fullsized":"2024-04-25T10:03:18.453907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:35.388687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.415818+0000","last_change":"2024-04-25T10:03:17.428581+0000","last_active":"2024-04-25T10:03:19.415818+0000","last_peered":"2024-04-25T10:03:19.415818+0000","last_clean":"2024-04-25T10:03:19.415818+0000","last_became_active":"2024-04-25T10:03:17.428212+0000","last_became_peered":"2024-04-25T10:03:17.428212+0000","last_unstale":"2024-04-25T10:03:19.415818+0000","last_undegraded":"2024-04-25T10:03:19.415818+0000","last_fullsized":"2024-04-25T10:03:19.415818+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:23.181440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.416272+0000","last_change":"2024-04-25T10:03:18.458382+0000","last_active":"2024-04-25T10:03:19.416272+0000","last_peered":"2024-04-25T10:03:19.416272+0000","last_clean":"2024-04-25T10:03:19.416272+0000","last_became_active":"2024-04-25T10:03:18.458073+0000","last_became_peered":"2024-04-25T10:03:18.458073+0000","last_unstale":"2024-04-25T10:03:19.416272+0000","last_undegraded":"2024-04-25T10:03:19.416272+0000","last_fullsized":"2024-04-25T10:03:19.416272+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:26.795579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.422172+0000","last_change":"2024-04-25T10:03:17.443401+0000","last_active":"2024-04-25T10:03:18.422172+0000","last_peered":"2024-04-25T10:03:18.422172+0000","last_clean":"2024-04-25T10:03:18.422172+0000","last_became_active":"2024-04-25T10:03:17.443122+0000","last_became_peered":"2024-04-25T10:03:17.443122+0000","last_unstale":"2024-04-25T10:03:18.422172+0000","last_undegraded":"2024-04-25T10:03:18.422172+0000","last_fullsized":"2024-04-25T10:03:18.422172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:34.378103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.418346+0000","last_change":"2024-04-25T10:03:18.442493+0000","last_active":"2024-04-25T10:03:19.418346+0000","last_peered":"2024-04-25T10:03:19.418346+0000","last_clean":"2024-04-25T10:03:19.418346+0000","last_became_active":"2024-04-25T10:03:18.441694+0000","last_became_peered":"2024-04-25T10:03:18.441694+0000","last_unstale":"2024-04-25T10:03:19.418346+0000","last_undegraded":"2024-04-25T10:03:19.418346+0000","last_fullsized":"2024-04-25T10:03:19.418346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:18.432775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.416016+0000","last_change":"2024-04-25T10:03:17.429462+0000","last_active":"2024-04-25T10:03:19.416016+0000","last_peered":"2024-04-25T10:03:19.416016+0000","last_clean":"2024-04-25T10:03:19.416016+0000","last_became_active":"2024-04-25T10:03:17.429143+0000","last_became_peered":"2024-04-25T10:03:17.429143+0000","last_unstale":"2024-04-25T10:03:19.416016+0000","last_undegraded":"2024-04-25T10:03:19.416016+0000","last_fullsized":"2024-04-25T10:03:19.416016+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:24.149573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.476862+0000","last_change":"2024-04-25T10:03:18.476348+0000","last_active":"2024-04-25T10:03:18.476862+0000","last_peered":"2024-04-25T10:03:18.476862+0000","last_clean":"2024-04-25T10:03:18.476862+0000","last_became_active":"2024-04-25T10:03:18.475991+0000","last_became_peered":"2024-04-25T10:03:18.475991+0000","last_unstale":"2024-04-25T10:03:18.476862+0000","last_undegraded":"2024-04-25T10:03:18.476862+0000","last_fullsized":"2024-04-25T10:03:18.476862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:14.452300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.415997+0000","last_change":"2024-04-25T10:03:17.450750+0000","last_active":"2024-04-25T10:03:19.415997+0000","last_peered":"2024-04-25T10:03:19.415997+0000","last_clean":"2024-04-25T10:03:19.415997+0000","last_became_active":"2024-04-25T10:03:17.450236+0000","last_became_peered":"2024-04-25T10:03:17.450236+0000","last_unstale":"2024-04-25T10:03:19.415997+0000","last_undegraded":"2024-04-25T10:03:19.415997+0000","last_fullsized":"2024-04-25T10:03:19.415997+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:34.996769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.474600+0000","last_change":"2024-04-25T10:03:18.441132+0000","last_active":"2024-04-25T10:03:18.474600+0000","last_peered":"2024-04-25T10:03:18.474600+0000","last_clean":"2024-04-25T10:03:18.474600+0000","last_became_active":"2024-04-25T10:03:18.440065+0000","last_became_peered":"2024-04-25T10:03:18.440065+0000","last_unstale":"2024-04-25T10:03:18.474600+0000","last_undegraded":"2024-04-25T10:03:18.474600+0000","last_fullsized":"2024-04-25T10:03:18.474600+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:39.174456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.418510+0000","last_change":"2024-04-25T10:03:17.432244+0000","last_active":"2024-04-25T10:03:19.418510+0000","last_peered":"2024-04-25T10:03:19.418510+0000","last_clean":"2024-04-25T10:03:19.418510+0000","last_became_active":"2024-04-25T10:03:17.431512+0000","last_became_peered":"2024-04-25T10:03:17.431512+0000","last_unstale":"2024-04-25T10:03:19.418510+0000","last_undegraded":"2024-04-25T10:03:19.418510+0000","last_fullsized":"2024-04-25T10:03:19.418510+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:13.844001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.461477+0000","last_change":"2024-04-25T10:03:18.453105+0000","last_active":"2024-04-25T10:03:18.461477+0000","last_peered":"2024-04-25T10:03:18.461477+0000","last_clean":"2024-04-25T10:03:18.461477+0000","last_became_active":"2024-04-25T10:03:18.452782+0000","last_became_peered":"2024-04-25T10:03:18.452782+0000","last_unstale":"2024-04-25T10:03:18.461477+0000","last_undegraded":"2024-04-25T10:03:18.461477+0000","last_fullsized":"2024-04-25T10:03:18.461477+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:48.097429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.415098+0000","last_change":"2024-04-25T10:03:17.431646+0000","last_active":"2024-04-25T10:03:19.415098+0000","last_peered":"2024-04-25T10:03:19.415098+0000","last_clean":"2024-04-25T10:03:19.415098+0000","last_became_active":"2024-04-25T10:03:17.431361+0000","last_became_peered":"2024-04-25T10:03:17.431361+0000","last_unstale":"2024-04-25T10:03:19.415098+0000","last_undegraded":"2024-04-25T10:03:19.415098+0000","last_fullsized":"2024-04-25T10:03:19.415098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:47.045635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.457914+0000","last_change":"2024-04-25T10:03:18.451666+0000","last_active":"2024-04-25T10:03:18.457914+0000","last_peered":"2024-04-25T10:03:18.457914+0000","last_clean":"2024-04-25T10:03:18.457914+0000","last_became_active":"2024-04-25T10:03:18.451387+0000","last_became_peered":"2024-04-25T10:03:18.451387+0000","last_unstale":"2024-04-25T10:03:18.457914+0000","last_undegraded":"2024-04-25T10:03:18.457914+0000","last_fullsized":"2024-04-25T10:03:18.457914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:14.854586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.478437+0000","last_change":"2024-04-25T10:03:17.454618+0000","last_active":"2024-04-25T10:03:18.478437+0000","last_peered":"2024-04-25T10:03:18.478437+0000","last_clean":"2024-04-25T10:03:18.478437+0000","last_became_active":"2024-04-25T10:03:17.454120+0000","last_became_peered":"2024-04-25T10:03:17.454120+0000","last_unstale":"2024-04-25T10:03:18.478437+0000","last_undegraded":"2024-04-25T10:03:18.478437+0000","last_fullsized":"2024-04-25T10:03:18.478437+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:17.194543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.420234+0000","last_change":"2024-04-25T10:03:18.468947+0000","last_active":"2024-04-25T10:03:19.420234+0000","last_peered":"2024-04-25T10:03:19.420234+0000","last_clean":"2024-04-25T10:03:19.420234+0000","last_became_active":"2024-04-25T10:03:18.456699+0000","last_became_peered":"2024-04-25T10:03:18.456699+0000","last_unstale":"2024-04-25T10:03:19.420234+0000","last_undegraded":"2024-04-25T10:03:19.420234+0000","last_fullsized":"2024-04-25T10:03:19.420234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:09.332558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.415835+0000","last_change":"2024-04-25T10:03:17.459415+0000","last_active":"2024-04-25T10:03:19.415835+0000","last_peered":"2024-04-25T10:03:19.415835+0000","last_clean":"2024-04-25T10:03:19.415835+0000","last_became_active":"2024-04-25T10:03:17.459149+0000","last_became_peered":"2024-04-25T10:03:17.459149+0000","last_unstale":"2024-04-25T10:03:19.415835+0000","last_undegraded":"2024-04-25T10:03:19.415835+0000","last_fullsized":"2024-04-25T10:03:19.415835+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:48.163833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.461346+0000","last_change":"2024-04-25T10:03:18.453615+0000","last_active":"2024-04-25T10:03:18.461346+0000","last_peered":"2024-04-25T10:03:18.461346+0000","last_clean":"2024-04-25T10:03:18.461346+0000","last_became_active":"2024-04-25T10:03:18.452082+0000","last_became_peered":"2024-04-25T10:03:18.452082+0000","last_unstale":"2024-04-25T10:03:18.461346+0000","last_undegraded":"2024-04-25T10:03:18.461346+0000","last_fullsized":"2024-04-25T10:03:18.461346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:04.585731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.425408+0000","last_change":"2024-04-25T10:03:17.432340+0000","last_active":"2024-04-25T10:03:18.425408+0000","last_peered":"2024-04-25T10:03:18.425408+0000","last_clean":"2024-04-25T10:03:18.425408+0000","last_became_active":"2024-04-25T10:03:17.431997+0000","last_became_peered":"2024-04-25T10:03:17.431997+0000","last_unstale":"2024-04-25T10:03:18.425408+0000","last_undegraded":"2024-04-25T10:03:18.425408+0000","last_fullsized":"2024-04-25T10:03:18.425408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:57.881722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.461618+0000","last_change":"2024-04-25T10:03:18.459690+0000","last_active":"2024-04-25T10:03:18.461618+0000","last_peered":"2024-04-25T10:03:18.461618+0000","last_clean":"2024-04-25T10:03:18.461618+0000","last_became_active":"2024-04-25T10:03:18.459147+0000","last_became_peered":"2024-04-25T10:03:18.459147+0000","last_unstale":"2024-04-25T10:03:18.461618+0000","last_undegraded":"2024-04-25T10:03:18.461618+0000","last_fullsized":"2024-04-25T10:03:18.461618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:31.249126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.430554+0000","last_change":"2024-04-25T10:03:17.433077+0000","last_active":"2024-04-25T10:03:18.430554+0000","last_peered":"2024-04-25T10:03:18.430554+0000","last_clean":"2024-04-25T10:03:18.430554+0000","last_became_active":"2024-04-25T10:03:17.432492+0000","last_became_peered":"2024-04-25T10:03:17.432492+0000","last_unstale":"2024-04-25T10:03:18.430554+0000","last_undegraded":"2024-04-25T10:03:18.430554+0000","last_fullsized":"2024-04-25T10:03:18.430554+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:40.768753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.443935+0000","last_change":"2024-04-25T10:03:18.436977+0000","last_active":"2024-04-25T10:03:18.443935+0000","last_peered":"2024-04-25T10:03:18.443935+0000","last_clean":"2024-04-25T10:03:18.443935+0000","last_became_active":"2024-04-25T10:03:18.436040+0000","last_became_peered":"2024-04-25T10:03:18.436040+0000","last_unstale":"2024-04-25T10:03:18.443935+0000","last_undegraded":"2024-04-25T10:03:18.443935+0000","last_fullsized":"2024-04-25T10:03:18.443935+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:14.452300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.418002+0000","last_change":"2024-04-25T10:03:17.434985+0000","last_active":"2024-04-25T10:03:19.418002+0000","last_peered":"2024-04-25T10:03:19.418002+0000","last_clean":"2024-04-25T10:03:19.418002+0000","last_became_active":"2024-04-25T10:03:17.434216+0000","last_became_peered":"2024-04-25T10:03:17.434216+0000","last_unstale":"2024-04-25T10:03:19.418002+0000","last_undegraded":"2024-04-25T10:03:19.418002+0000","last_fullsized":"2024-04-25T10:03:19.418002+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:32.489428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.464792+0000","last_change":"2024-04-25T10:03:18.462832+0000","last_active":"2024-04-25T10:03:18.464792+0000","last_peered":"2024-04-25T10:03:18.464792+0000","last_clean":"2024-04-25T10:03:18.464792+0000","last_became_active":"2024-04-25T10:03:18.462181+0000","last_became_peered":"2024-04-25T10:03:18.462181+0000","last_unstale":"2024-04-25T10:03:18.464792+0000","last_undegraded":"2024-04-25T10:03:18.464792+0000","last_fullsized":"2024-04-25T10:03:18.464792+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:31.249126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.425408+0000","last_change":"2024-04-25T10:03:17.439937+0000","last_active":"2024-04-25T10:03:18.425408+0000","last_peered":"2024-04-25T10:03:18.425408+0000","last_clean":"2024-04-25T10:03:18.425408+0000","last_became_active":"2024-04-25T10:03:17.439062+0000","last_became_peered":"2024-04-25T10:03:17.439062+0000","last_unstale":"2024-04-25T10:03:18.425408+0000","last_undegraded":"2024-04-25T10:03:18.425408+0000","last_fullsized":"2024-04-25T10:03:18.425408+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:40.768753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.419984+0000","last_change":"2024-04-25T10:03:18.457340+0000","last_active":"2024-04-25T10:03:19.419984+0000","last_peered":"2024-04-25T10:03:19.419984+0000","last_clean":"2024-04-25T10:03:19.419984+0000","last_became_active":"2024-04-25T10:03:18.456537+0000","last_became_peered":"2024-04-25T10:03:18.456537+0000","last_unstale":"2024-04-25T10:03:19.419984+0000","last_undegraded":"2024-04-25T10:03:19.419984+0000","last_fullsized":"2024-04-25T10:03:19.419984+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:19.854110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.448569+0000","last_change":"2024-04-25T10:03:17.425097+0000","last_active":"2024-04-25T10:03:18.448569+0000","last_peered":"2024-04-25T10:03:18.448569+0000","last_clean":"2024-04-25T10:03:18.448569+0000","last_became_active":"2024-04-25T10:03:17.424686+0000","last_became_peered":"2024-04-25T10:03:17.424686+0000","last_unstale":"2024-04-25T10:03:18.448569+0000","last_undegraded":"2024-04-25T10:03:18.448569+0000","last_fullsized":"2024-04-25T10:03:18.448569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:25.708222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:18.478188+0000","last_change":"2024-04-25T10:03:18.465204+0000","last_active":"2024-04-25T10:03:18.478188+0000","last_peered":"2024-04-25T10:03:18.478188+0000","last_clean":"2024-04-25T10:03:18.478188+0000","last_became_active":"2024-04-25T10:03:18.464678+0000","last_became_peered":"2024-04-25T10:03:18.464678+0000","last_unstale":"2024-04-25T10:03:18.478188+0000","last_undegraded":"2024-04-25T10:03:18.478188+0000","last_fullsized":"2024-04-25T10:03:18.478188+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:17.394052+0000","last_clean_scrub_stamp":"2024-04-25T10:03:17.394052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:06.842871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.419899+0000","last_change":"2024-04-25T10:03:17.434355+0000","last_active":"2024-04-25T10:03:19.419899+0000","last_peered":"2024-04-25T10:03:19.419899+0000","last_clean":"2024-04-25T10:03:19.419899+0000","last_became_active":"2024-04-25T10:03:17.433544+0000","last_became_peered":"2024-04-25T10:03:17.433544+0000","last_unstale":"2024-04-25T10:03:19.419899+0000","last_undegraded":"2024-04-25T10:03:19.419899+0000","last_fullsized":"2024-04-25T10:03:19.419899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:16.383467+0000","last_clean_scrub_stamp":"2024-04-25T10:03:16.383467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:25.708222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:19.417753+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:03:19.417753+0000","last_peered":"2024-04-25T10:03:19.417753+0000","last_clean":"2024-04-25T10:03:19.417753+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:03:19.417753+0000","last_undegraded":"2024-04-25T10:03:19.417753+0000","last_fullsized":"2024-04-25T10:03:19.417753+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":782078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":183448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":183940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":184982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":190098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1024000,"data_stored":773738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:21.831 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:03:21.831 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:03:21.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.992+0000 7f6c96b72640 1 Processor -- start 2024-04-25T10:03:21.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.992+0000 7f6c96b72640 1 -- start start 2024-04-25T10:03:22.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.992+0000 7f6c96b72640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6c90143ea0 0x7f6c900cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.992+0000 7f6c96b72640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c9006eac0 0x7f6c900b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.992+0000 7f6c96b72640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c9015ac50 con 0x7f6c900b2150 2024-04-25T10:03:22.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.992+0000 7f6c96b72640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c900cd180 con 0x7f6c90143ea0 2024-04-25T10:03:22.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.992+0000 7f6c96b72640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9015aed0 con 0x7f6c9006eac0 2024-04-25T10:03:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.992+0000 7f6c9490e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6c90143ea0 0x7f6c900cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.992+0000 7f6c9510f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c9006eac0 0x7f6c900b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.992+0000 7f6c8ffff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6c900b2150 0x7f6c9006e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:55280/0 (socket says 172.21.15.133:55280) 2024-04-25T10:03:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.992+0000 7f6c8ffff640 1 -- 172.21.15.133:0/3334336449 learned_addr learned my addr 172.21.15.133:0/3334336449 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c9490e640 1 -- 172.21.15.133:0/3334336449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c9006eac0 msgr2=0x7f6c900b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c9490e640 1 --2- 172.21.15.133:0/3334336449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c9006eac0 0x7f6c900b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c9490e640 1 -- 172.21.15.133:0/3334336449 >> v1:172.21.15.162:6789/0 conn(0x7f6c900b2150 legacy=0x7f6c9006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c9490e640 1 -- 172.21.15.133:0/3334336449 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9015b1b0 con 0x7f6c90143ea0 2024-04-25T10:03:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c9510f640 1 --2- 172.21.15.133:0/3334336449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c9006eac0 0x7f6c900b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c8f7fe640 1 -- 172.21.15.133:0/3334336449 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312496702 0 0) 0x7f6c7c001120 con 0x7f6c900b2150 2024-04-25T10:03:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c9490e640 1 --2- 172.21.15.133:0/3334336449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c90143ea0 0x7f6c900cc810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6c840097c0 tx=0x7f6c8401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c1cd320e0604f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c8f7fe640 1 -- 172.21.15.133:0/3334336449 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c840b1070 con 0x7f6c90143ea0 2024-04-25T10:03:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c8f7fe640 1 -- 172.21.15.133:0/3334336449 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c840ac410 con 0x7f6c90143ea0 2024-04-25T10:03:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c8f7fe640 1 -- 172.21.15.133:0/3334336449 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c840ac7e0 con 0x7f6c90143ea0 2024-04-25T10:03:22.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c96b72640 1 -- 172.21.15.133:0/3334336449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c90143ea0 msgr2=0x7f6c900cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c96b72640 1 --2- 172.21.15.133:0/3334336449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c90143ea0 0x7f6c900cc810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6c840097c0 tx=0x7f6c8401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c96b72640 1 -- 172.21.15.133:0/3334336449 shutdown_connections 2024-04-25T10:03:22.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c96b72640 1 --2- 172.21.15.133:0/3334336449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c9006eac0 0x7f6c900b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c96b72640 1 --2- 172.21.15.133:0/3334336449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c90143ea0 0x7f6c900cc810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c96b72640 1 -- 172.21.15.133:0/3334336449 >> 172.21.15.133:0/3334336449 conn(0x7f6c90066f70 msgr2=0x7f6c9006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c96b72640 1 -- 172.21.15.133:0/3334336449 shutdown_connections 2024-04-25T10:03:22.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:21.996+0000 7f6c96b72640 1 -- 172.21.15.133:0/3334336449 wait complete. 2024-04-25T10:03:22.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c96b72640 1 Processor -- start 2024-04-25T10:03:22.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c96b72640 1 -- start start 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c96b72640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9006eac0 0x7f6c90156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c96b72640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c900b2150 0x7f6c90156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c96b72640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c90143ea0 0x7f6c90156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c96b72640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c90167550 con 0x7f6c9006eac0 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c96b72640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c901673d0 con 0x7f6c90143ea0 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c96b72640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c901676d0 con 0x7f6c900b2150 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c9490e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9006eac0 0x7f6c90156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c9490e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9006eac0 0x7f6c90156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60702/0 (socket says 172.21.15.133:60702) 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c9490e640 1 -- 172.21.15.133:0/3310624007 learned_addr learned my addr 172.21.15.133:0/3310624007 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c9510f640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c90143ea0 0x7f6c90156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c8ffff640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c900b2150 0x7f6c90156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c9490e640 1 -- 172.21.15.133:0/3310624007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c900b2150 msgr2=0x7f6c90156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c9490e640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c900b2150 0x7f6c90156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c9490e640 1 -- 172.21.15.133:0/3310624007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c90143ea0 msgr2=0x7f6c90156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c9490e640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c90143ea0 0x7f6c90156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c9490e640 1 -- 172.21.15.133:0/3310624007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c901575f0 con 0x7f6c9006eac0 2024-04-25T10:03:22.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c9490e640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9006eac0 0x7f6c90156560 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6c840097c0 tx=0x7f6c8401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c8d7fa640 1 -- 172.21.15.133:0/3310624007 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c840b1050 con 0x7f6c9006eac0 2024-04-25T10:03:22.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c8d7fa640 1 -- 172.21.15.133:0/3310624007 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c840b2040 con 0x7f6c9006eac0 2024-04-25T10:03:22.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c8d7fa640 1 -- 172.21.15.133:0/3310624007 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84006e20 con 0x7f6c9006eac0 2024-04-25T10:03:22.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c96b72640 1 -- 172.21.15.133:0/3310624007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c901578d0 con 0x7f6c9006eac0 2024-04-25T10:03:22.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c96b72640 1 -- 172.21.15.133:0/3310624007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c90162390 con 0x7f6c9006eac0 2024-04-25T10:03:22.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.000+0000 7f6c96b72640 1 -- 172.21.15.133:0/3310624007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c90068e80 con 0x7f6c9006eac0 2024-04-25T10:03:22.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.004+0000 7f6c8d7fa640 1 -- 172.21.15.133:0/3310624007 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c840bf060 con 0x7f6c9006eac0 2024-04-25T10:03:22.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.004+0000 7f6c8d7fa640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c74066f60 0x7f6c74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.004+0000 7f6c8d7fa640 1 -- 172.21.15.133:0/3310624007 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c8411eae0 con 0x7f6c9006eac0 2024-04-25T10:03:22.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.004+0000 7f6c8ffff640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c74066f60 0x7f6c74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.004+0000 7f6c8ffff640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c74066f60 0x7f6c74069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6c901586e0 tx=0x7f6c7c007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.008+0000 7f6c8d7fa640 1 -- 172.21.15.133:0/3310624007 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c84016090 con 0x7f6c9006eac0 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.224+0000 7f6c96b72640 1 -- 172.21.15.133:0/3310624007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6c90157aa0 con 0x7f6c9006eac0 2024-04-25T10:03:22.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c8d7fa640 1 -- 172.21.15.133:0/3310624007 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6c840ef960 con 0x7f6c9006eac0 2024-04-25T10:03:22.234 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:03:22.234 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:03:22.234 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:03:22.234 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:03:22.234 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:03:22.234 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:03:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 -- 172.21.15.133:0/3310624007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c74066f60 msgr2=0x7f6c74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c74066f60 0x7f6c74069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6c901586e0 tx=0x7f6c7c007980 comp rx=0 tx=0).stop 2024-04-25T10:03:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 -- 172.21.15.133:0/3310624007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9006eac0 msgr2=0x7f6c90156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9006eac0 0x7f6c90156560 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6c840097c0 tx=0x7f6c8401ace0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 -- 172.21.15.133:0/3310624007 shutdown_connections 2024-04-25T10:03:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c90143ea0 0x7f6c90156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c900b2150 0x7f6c90156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c74066f60 0x7f6c74069420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 --2- 172.21.15.133:0/3310624007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9006eac0 0x7f6c90156560 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 -- 172.21.15.133:0/3310624007 >> 172.21.15.133:0/3310624007 conn(0x7f6c90066f70 msgr2=0x7f6c9015cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 -- 172.21.15.133:0/3310624007 shutdown_connections 2024-04-25T10:03:22.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.228+0000 7f6c96b72640 1 -- 172.21.15.133:0/3310624007 wait complete. 2024-04-25T10:03:22.253 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:22.316 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-04-25T10:03:22.308+0000 7f891ab7a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:22.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f672192c640 1 Processor -- start 2024-04-25T10:03:22.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f672192c640 1 -- start start 2024-04-25T10:03:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f672192c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671c158110 0x7f671c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f672192c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f671c1584f0 0x7f671c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f672192c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671c0cbe60 con 0x7f671c06e460 2024-04-25T10:03:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f672192c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671c0b19b0 con 0x7f671c1584f0 2024-04-25T10:03:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f672192c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671c15b360 con 0x7f671c158110 2024-04-25T10:03:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f671a7fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f671c1584f0 0x7f671c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f671affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671c158110 0x7f671c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f671a7fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f671c1584f0 0x7f671c15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:36560/0 (socket says 172.21.15.133:36560) 2024-04-25T10:03:22.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f671a7fc640 1 -- 172.21.15.133:0/1599187334 learned_addr learned my addr 172.21.15.133:0/1599187334 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:22.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.360+0000 7f671b7fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f671c06e460 0x7f671c15e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:55294/0 (socket says 172.21.15.133:55294) 2024-04-25T10:03:22.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f671affd640 1 -- 172.21.15.133:0/1599187334 >> v2:172.21.15.162:3300/0 conn(0x7f671c1584f0 msgr2=0x7f671c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f671affd640 1 --2- 172.21.15.133:0/1599187334 >> v2:172.21.15.162:3300/0 conn(0x7f671c1584f0 0x7f671c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f671affd640 1 -- 172.21.15.133:0/1599187334 >> v1:172.21.15.162:6789/0 conn(0x7f671c06e460 legacy=0x7f671c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f6719ffb640 1 -- 172.21.15.133:0/1599187334 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487251161 0 0) 0x7f671c0cbe60 con 0x7f671c06e460 2024-04-25T10:03:22.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f671affd640 1 -- 172.21.15.133:0/1599187334 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c15b640 con 0x7f671c158110 2024-04-25T10:03:22.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f671a7fc640 1 --2- 172.21.15.133:0/1599187334 >> v2:172.21.15.162:3300/0 conn(0x7f671c1584f0 0x7f671c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:22.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f671affd640 1 --2- 172.21.15.133:0/1599187334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671c158110 0x7f671c0b1f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f670c0097c0 tx=0x7f670c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9549e15775ad4e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f6719ffb640 1 -- 172.21.15.133:0/1599187334 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c0b1070 con 0x7f671c158110 2024-04-25T10:03:22.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f6719ffb640 1 -- 172.21.15.133:0/1599187334 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c0ac410 con 0x7f671c158110 2024-04-25T10:03:22.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f6719ffb640 1 -- 172.21.15.133:0/1599187334 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c0ac7e0 con 0x7f671c158110 2024-04-25T10:03:22.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f672192c640 1 -- 172.21.15.133:0/1599187334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671c158110 msgr2=0x7f671c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f672192c640 1 --2- 172.21.15.133:0/1599187334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671c158110 0x7f671c0b1f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f670c0097c0 tx=0x7f670c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f672192c640 1 -- 172.21.15.133:0/1599187334 shutdown_connections 2024-04-25T10:03:22.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f672192c640 1 --2- 172.21.15.133:0/1599187334 >> v2:172.21.15.162:3300/0 conn(0x7f671c1584f0 0x7f671c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.364+0000 7f672192c640 1 --2- 172.21.15.133:0/1599187334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671c158110 0x7f671c0b1f00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 -- 172.21.15.133:0/1599187334 >> 172.21.15.133:0/1599187334 conn(0x7f671c067000 msgr2=0x7f671c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 -- 172.21.15.133:0/1599187334 shutdown_connections 2024-04-25T10:03:22.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 -- 172.21.15.133:0/1599187334 wait complete. 2024-04-25T10:03:22.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 Processor -- start 2024-04-25T10:03:22.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 -- start start 2024-04-25T10:03:22.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671c06e460 0x7f671c15da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f671c158110 0x7f671c15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671c1584f0 0x7f671c143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0cd7f0 con 0x7f671c1584f0 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0cd670 con 0x7f671c158110 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671c167470 con 0x7f671c06e460 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f671b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671c1584f0 0x7f671c143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f671b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671c1584f0 0x7f671c143030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60708/0 (socket says 172.21.15.133:60708) 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f671b7fe640 1 -- 172.21.15.133:0/528443278 learned_addr learned my addr 172.21.15.133:0/528443278 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f671affd640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671c06e460 0x7f671c15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f671a7fc640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f671c158110 0x7f671c15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f671b7fe640 1 -- 172.21.15.133:0/528443278 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671c06e460 msgr2=0x7f671c15da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f671b7fe640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671c06e460 0x7f671c15da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f671b7fe640 1 -- 172.21.15.133:0/528443278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f671c158110 msgr2=0x7f671c15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f671b7fe640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f671c158110 0x7f671c15beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f671b7fe640 1 -- 172.21.15.133:0/528443278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c151600 con 0x7f671c1584f0 2024-04-25T10:03:22.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f671b7fe640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671c1584f0 0x7f671c143030 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6714001570 tx=0x7f671400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f6703fff640 1 -- 172.21.15.133:0/528443278 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714018070 con 0x7f671c1584f0 2024-04-25T10:03:22.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f6703fff640 1 -- 172.21.15.133:0/528443278 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671400a640 con 0x7f671c1584f0 2024-04-25T10:03:22.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f6703fff640 1 -- 172.21.15.133:0/528443278 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671400aa30 con 0x7f671c1584f0 2024-04-25T10:03:22.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 -- 172.21.15.133:0/528443278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671c151040 con 0x7f671c1584f0 2024-04-25T10:03:22.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.368+0000 7f672192c640 1 -- 172.21.15.133:0/528443278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c1643b0 con 0x7f671c1584f0 2024-04-25T10:03:22.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.372+0000 7f672192c640 1 -- 172.21.15.133:0/528443278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671c06bc20 con 0x7f671c1584f0 2024-04-25T10:03:22.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.372+0000 7f6703fff640 1 -- 172.21.15.133:0/528443278 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f671400abd0 con 0x7f671c1584f0 2024-04-25T10:03:22.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.372+0000 7f6703fff640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6708066f60 0x7f6708069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.372+0000 7f6703fff640 1 -- 172.21.15.133:0/528443278 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6714084980 con 0x7f671c1584f0 2024-04-25T10:03:22.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.372+0000 7f671affd640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6708066f60 0x7f6708069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.372+0000 7f671affd640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6708066f60 0x7f6708069420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f670c0ae000 tx=0x7f670c0a6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.376+0000 7f6703fff640 1 -- 172.21.15.133:0/528443278 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67140556e0 con 0x7f671c1584f0 2024-04-25T10:03:22.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.556+0000 7f672192c640 1 -- 172.21.15.133:0/528443278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f671c068ef0 con 0x7f671c1584f0 2024-04-25T10:03:22.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.556+0000 7f6703fff640 1 -- 172.21.15.133:0/528443278 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f6714054fe0 con 0x7f671c1584f0 2024-04-25T10:03:22.565 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:03:22.565 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2887512873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2887512873},{"type":"v1","addr":"172.21.15.162:6833","nonce":2887512873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1044882076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1044882076},{"type":"v1","addr":"172.21.15.162:6835","nonce":1044882076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1965748675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1965748675},{"type":"v1","addr":"172.21.15.133:6836","nonce":1965748675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:17.773574+0000","modified":"2024-04-25T10:03:19.793202+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/205640021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":205640021},{"type":"v1","addr":"172.21.15.133:6837","nonce":205640021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:22.565 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 16 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 -- 172.21.15.133:0/528443278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6708066f60 msgr2=0x7f6708069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6708066f60 0x7f6708069420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f670c0ae000 tx=0x7f670c0a6040 comp rx=0 tx=0).stop 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 -- 172.21.15.133:0/528443278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671c1584f0 msgr2=0x7f671c143030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671c1584f0 0x7f671c143030 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6714001570 tx=0x7f671400cca0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 -- 172.21.15.133:0/528443278 shutdown_connections 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6708066f60 0x7f6708069420 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f671c1584f0 0x7f671c143030 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f671c158110 0x7f671c15beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 --2- 172.21.15.133:0/528443278 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f671c06e460 0x7f671c15da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 -- 172.21.15.133:0/528443278 >> 172.21.15.133:0/528443278 conn(0x7f671c067000 msgr2=0x7f671c06d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 -- 172.21.15.133:0/528443278 shutdown_connections 2024-04-25T10:03:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:22.560+0000 7f672192c640 1 -- 172.21.15.133:0/528443278 wait complete. 2024-04-25T10:03:22.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:03:17.773574+0000', 'modified': '2024-04-25T10:03:19.793202+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5532': {'gid': 5532, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/205640021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 205640021}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 205640021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:22.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:03:22.984 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:03:23.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.104+0000 7fb27b0d5640 1 Processor -- start 2024-04-25T10:03:23.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.104+0000 7fb27b0d5640 1 -- start start 2024-04-25T10:03:23.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.104+0000 7fb27b0d5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2740d0a30 0x7fb2740b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.104+0000 7fb27b0d5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2740b1820 con 0x7fb2740ce080 2024-04-25T10:03:23.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.104+0000 7fb27b0d5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2740cdc40 con 0x7fb27406cc80 2024-04-25T10:03:23.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.104+0000 7fb27b0d5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2740b1ac0 con 0x7fb2740d0a30 2024-04-25T10:03:23.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb279672640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2740d0a30 0x7fb2740b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb278e71640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb2740ce080 0x7fb2740ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40820/0 (socket says 172.21.15.133:40820) 2024-04-25T10:03:23.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb278e71640 1 -- 172.21.15.133:0/1002073975 learned_addr learned my addr 172.21.15.133:0/1002073975 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:03:23.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb279672640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2740d0a30 0x7fb2740b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60724/0 (socket says 172.21.15.133:60724) 2024-04-25T10:03:23.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb279672640 1 -- 172.21.15.133:0/1002073975 >> v1:172.21.15.162:6789/0 conn(0x7fb27406cc80 legacy=0x7fb2741604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb279672640 1 -- 172.21.15.133:0/1002073975 >> v1:172.21.15.133:6789/0 conn(0x7fb2740ce080 legacy=0x7fb2740ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb279672640 1 -- 172.21.15.133:0/1002073975 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740b1da0 con 0x7fb2740d0a30 2024-04-25T10:03:23.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb279672640 1 --2- 172.21.15.133:0/1002073975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2740d0a30 0x7fb2740b1160 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb2740d20c0 tx=0x7fb26c01c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfbbb312790383b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb2737fe640 1 -- 172.21.15.133:0/1002073975 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0b2070 con 0x7fb2740d0a30 2024-04-25T10:03:23.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb2737fe640 1 -- 172.21.15.133:0/1002073975 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0a58f0 con 0x7fb2740d0a30 2024-04-25T10:03:23.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb2737fe640 1 -- 172.21.15.133:0/1002073975 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0a5cc0 con 0x7fb2740d0a30 2024-04-25T10:03:23.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb27b0d5640 1 -- 172.21.15.133:0/1002073975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2740d0a30 msgr2=0x7fb2740b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb27b0d5640 1 --2- 172.21.15.133:0/1002073975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2740d0a30 0x7fb2740b1160 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb2740d20c0 tx=0x7fb26c01c830 comp rx=0 tx=0).stop 2024-04-25T10:03:23.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb27b0d5640 1 -- 172.21.15.133:0/1002073975 shutdown_connections 2024-04-25T10:03:23.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb27b0d5640 1 --2- 172.21.15.133:0/1002073975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2740d0a30 0x7fb2740b1160 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb27b0d5640 1 -- 172.21.15.133:0/1002073975 >> 172.21.15.133:0/1002073975 conn(0x7fb27409ab90 msgr2=0x7fb2740d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb27b0d5640 1 -- 172.21.15.133:0/1002073975 shutdown_connections 2024-04-25T10:03:23.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.108+0000 7fb27b0d5640 1 -- 172.21.15.133:0/1002073975 wait complete. 2024-04-25T10:03:23.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb27b0d5640 1 Processor -- start 2024-04-25T10:03:23.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb27b0d5640 1 -- start start 2024-04-25T10:03:23.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb27b0d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb27406cc80 0x7fb27414f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb27b0d5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2740ce080 0x7fb2740b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb27b0d5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2740d0a30 0x7fb27414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb27b0d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27416b6a0 con 0x7fb27406cc80 2024-04-25T10:03:23.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb27b0d5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27416b520 con 0x7fb2740d0a30 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb27b0d5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27416b820 con 0x7fb2740ce080 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb278e71640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb27406cc80 0x7fb27414f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb278e71640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb27406cc80 0x7fb27414f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60726/0 (socket says 172.21.15.133:60726) 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb278e71640 1 -- 172.21.15.133:0/3074067286 learned_addr learned my addr 172.21.15.133:0/3074067286 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb278e71640 1 -- 172.21.15.133:0/3074067286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2740ce080 msgr2=0x7fb2740b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb278e71640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2740ce080 0x7fb2740b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb278e71640 1 -- 172.21.15.133:0/3074067286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2740d0a30 msgr2=0x7fb27414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb278e71640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2740d0a30 0x7fb27414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb278e71640 1 -- 172.21.15.133:0/3074067286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb274159b00 con 0x7fb27406cc80 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb278e71640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb27406cc80 0x7fb27414f330 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c400 tx=0x7fb26800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb2717fa640 1 -- 172.21.15.133:0/3074067286 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268017070 con 0x7fb27406cc80 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb2717fa640 1 -- 172.21.15.133:0/3074067286 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2680074e0 con 0x7fb27406cc80 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb27b0d5640 1 -- 172.21.15.133:0/3074067286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb274157180 con 0x7fb27406cc80 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb2717fa640 1 -- 172.21.15.133:0/3074067286 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2680059d0 con 0x7fb27406cc80 2024-04-25T10:03:23.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb27b0d5640 1 -- 172.21.15.133:0/3074067286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2740afb10 con 0x7fb27406cc80 2024-04-25T10:03:23.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb2717fa640 1 -- 172.21.15.133:0/3074067286 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb26801a070 con 0x7fb27406cc80 2024-04-25T10:03:23.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.112+0000 7fb2717fa640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb258066f60 0x7fb258069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb2717fa640 1 -- 172.21.15.133:0/3074067286 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb268084540 con 0x7fb27406cc80 2024-04-25T10:03:23.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb273fff640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb258066f60 0x7fb258069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb273fff640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb258066f60 0x7fb258069420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb27415cf60 tx=0x7fb2600079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 Processor -- start 2024-04-25T10:03:23.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 -- start start 2024-04-25T10:03:23.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0eca1920 0x55be0eca1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca2240 0x55be0ec579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0ec57f30 0x55be0ec58310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0eb3f5f0 con 0x55be0ec57f30 2024-04-25T10:03:23.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0eaa6da0 con 0x55be0eca2240 2024-04-25T10:03:23.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be0ea2f150 con 0x55be0eca1920 2024-04-25T10:03:23.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb279672640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0ec57f30 0x55be0ec58310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb279672640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0ec57f30 0x55be0ec58310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60742/0 (socket says 172.21.15.133:60742) 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb279672640 1 -- 172.21.15.133:0/1978320849 learned_addr learned my addr 172.21.15.133:0/1978320849 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb278e71640 1 --2- 172.21.15.133:0/1978320849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0eca1920 0x55be0eca1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb279672640 1 -- 172.21.15.133:0/1978320849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0eca1920 msgr2=0x55be0eca1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb273fff640 1 --2- 172.21.15.133:0/1978320849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca2240 0x55be0ec579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb279672640 1 --2- 172.21.15.133:0/1978320849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0eca1920 0x55be0eca1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb279672640 1 -- 172.21.15.133:0/1978320849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca2240 msgr2=0x55be0ec579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb279672640 1 --2- 172.21.15.133:0/1978320849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca2240 0x55be0ec579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb279672640 1 -- 172.21.15.133:0/1978320849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be0ea48ca0 con 0x55be0ec57f30 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb279672640 1 --2- 172.21.15.133:0/1978320849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0ec57f30 0x55be0ec58310 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb26c019440 tx=0x7fb26c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68723d47a663008c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb25e7fc640 1 -- 172.21.15.133:0/1978320849 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0b2040 con 0x55be0ec57f30 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb25e7fc640 1 -- 172.21.15.133:0/1978320849 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0be070 con 0x55be0ec57f30 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb25e7fc640 1 -- 172.21.15.133:0/1978320849 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0a5b10 con 0x55be0ec57f30 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 -- 172.21.15.133:0/1978320849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0ec57f30 msgr2=0x55be0ec58310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 --2- 172.21.15.133:0/1978320849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0ec57f30 0x55be0ec58310 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb26c019440 tx=0x7fb26c004a50 comp rx=0 tx=0).stop 2024-04-25T10:03:23.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 -- 172.21.15.133:0/1978320849 shutdown_connections 2024-04-25T10:03:23.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 --2- 172.21.15.133:0/1978320849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0ec57f30 0x55be0ec58310 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 --2- 172.21.15.133:0/1978320849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca2240 0x55be0ec579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 --2- 172.21.15.133:0/1978320849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0eca1920 0x55be0eca1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 -- 172.21.15.133:0/1978320849 >> 172.21.15.133:0/1978320849 conn(0x55be0ec5e2a0 msgr2=0x55be0ec562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 -- 172.21.15.133:0/1978320849 shutdown_connections 2024-04-25T10:03:23.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 -- 172.21.15.133:0/1978320849 wait complete. 2024-04-25T10:03:23.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 -- start start 2024-04-25T10:03:23.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.116+0000 7fb27e242000 1 Processor -- start 2024-04-25T10:03:23.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0ec57f30 0x55be0eca8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0eca1920 0x55be0eca91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ec5baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0ea61990 con 0x55be0eca2240 2024-04-25T10:03:23.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0eaa2e40 con 0x55be0ec57f30 2024-04-25T10:03:23.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be0ea32900 con 0x55be0eca1920 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 10 client.0 _ll_register_callbacks cb 0x55be0eb5a1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb279672640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ec5baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb279672640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ec5baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60752/0 (socket says 172.21.15.133:60752) 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb279672640 1 -- 172.21.15.133:0/2008589061 learned_addr learned my addr 172.21.15.133:0/2008589061 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb278e71640 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0ec57f30 0x55be0eca8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb273fff640 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0eca1920 0x55be0eca91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb279672640 1 -- 172.21.15.133:0/2008589061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0eca1920 msgr2=0x55be0eca91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb279672640 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0eca1920 0x55be0eca91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb279672640 1 -- 172.21.15.133:0/2008589061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0ec57f30 msgr2=0x55be0eca8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb279672640 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0ec57f30 0x55be0eca8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb279672640 1 -- 172.21.15.133:0/2008589061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be0ea5f630 con 0x55be0eca2240 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb279672640 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ec5baa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb26c019440 tx=0x7fb26c01b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400b7b16e34f604b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb2537fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:03:23.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb2537fe640 1 -- 172.21.15.133:0/2008589061 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0b2040 con 0x55be0eca2240 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb2537fe640 1 -- 172.21.15.133:0/2008589061 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb26c0a59b0 con 0x55be0eca2240 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 -- 172.21.15.133:0/2008589061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb26c0be070 con 0x55be0eca2240 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb2537fe640 1 -- 172.21.15.133:0/2008589061 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb26c0d2050 con 0x55be0eca2240 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 10 client.5703 fetch_fsmap learned FSMap version 16 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 -- 172.21.15.133:0/2008589061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55be0ea2f3a0 con 0x55be0eca2240 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb2537fe640 1 -- 172.21.15.133:0/2008589061 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0be070 con 0x55be0eca2240 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb2537fe640 1 -- 172.21.15.133:0/2008589061 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb26c0d5020 con 0x55be0eca2240 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 10 client.5703 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 10 client.5703 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 20 client.5703 populate_metadata read hostname 'smithi133' 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55be0ec5d160 0x55be0ecb2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 4 client.5703 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 -- 172.21.15.133:0/2008589061 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be0ea42a50 con 0x55be0ec5d160 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb278e71640 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55be0ec5d160 0x55be0ecb2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb2537fe640 10 client.5703 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb278e71640 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55be0ec5d160 0x55be0ecb2ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614a9d21cdfb3c39 server_cookie=8bb12bf18ab5a324 in_seq=0 out_seq=0 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb2537fe640 1 -- 172.21.15.133:0/2008589061 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be0ea42a50 con 0x55be0ec5d160 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb2537fe640 10 client.5703 handle_command_reply: tid=0 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 client.5703 shutdown 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 -- 172.21.15.133:0/2008589061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 msgr2=0x55be0ec5baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ec5baa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb26c019440 tx=0x7fb26c01b070 comp rx=0 tx=0).stop 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 -- 172.21.15.133:0/2008589061 shutdown_connections 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55be0ec5d160 0x55be0ecb2ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ec5baa0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0eca1920 0x55be0eca91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb2537fe640 0 client.5703 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 --2- 172.21.15.133:0/2008589061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0ec57f30 0x55be0eca8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 -- 172.21.15.133:0/2008589061 >> 172.21.15.133:0/2008589061 conn(0x55be0ec5e2a0 msgr2=0x55be0ec569e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 -- 172.21.15.133:0/2008589061 shutdown_connections 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 1 -- 172.21.15.133:0/2008589061 wait complete. 2024-04-25T10:03:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.120+0000 7fb27e242000 20 client.5703 trim_cache size 0 max 16384 2024-04-25T10:03:23.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb27e242000 1 Processor -- start 2024-04-25T10:03:23.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb27e242000 1 -- start start 2024-04-25T10:03:23.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb27e242000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0ec57f30 0x55be0eca01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb27e242000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca1920 0x55be0eca06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb27e242000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ecab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb27e242000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740af920 con 0x55be0eca2240 2024-04-25T10:03:23.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb27e242000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb274157e80 con 0x55be0eca1920 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb27e242000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740af320 con 0x55be0ec57f30 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb279672640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ecab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb279672640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ecab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60754/0 (socket says 172.21.15.133:60754) 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb279672640 1 -- 172.21.15.133:0/703318689 learned_addr learned my addr 172.21.15.133:0/703318689 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb278e71640 1 --2- 172.21.15.133:0/703318689 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0ec57f30 0x55be0eca01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb273fff640 1 --2- 172.21.15.133:0/703318689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca1920 0x55be0eca06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb279672640 1 -- 172.21.15.133:0/703318689 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0ec57f30 msgr2=0x55be0eca01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb279672640 1 --2- 172.21.15.133:0/703318689 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0ec57f30 0x55be0eca01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb279672640 1 -- 172.21.15.133:0/703318689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca1920 msgr2=0x55be0eca06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb279672640 1 --2- 172.21.15.133:0/703318689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca1920 0x55be0eca06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb279672640 1 -- 172.21.15.133:0/703318689 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be0eca2070 con 0x55be0eca2240 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb273fff640 1 --2- 172.21.15.133:0/703318689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca1920 0x55be0eca06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:23.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb279672640 1 --2- 172.21.15.133:0/703318689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ecab6f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb26c01c870 tx=0x7fb26c0c3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee63f90b915aa67d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb25dffb640 1 -- 172.21.15.133:0/703318689 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0b2070 con 0x55be0eca2240 2024-04-25T10:03:23.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb25dffb640 1 -- 172.21.15.133:0/703318689 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0be040 con 0x55be0eca2240 2024-04-25T10:03:23.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb25dffb640 1 -- 172.21.15.133:0/703318689 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c003f20 con 0x55be0eca2240 2024-04-25T10:03:23.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb27e242000 1 -- 172.21.15.133:0/703318689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 msgr2=0x55be0ecab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.136+0000 7fb27e242000 1 --2- 172.21.15.133:0/703318689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ecab6f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb26c01c870 tx=0x7fb26c0c3000 comp rx=0 tx=0).stop 2024-04-25T10:03:23.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/703318689 shutdown_connections 2024-04-25T10:03:23.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- 172.21.15.133:0/703318689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ecab6f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- 172.21.15.133:0/703318689 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca1920 0x55be0eca06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- 172.21.15.133:0/703318689 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0ec57f30 0x55be0eca01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/703318689 >> 172.21.15.133:0/703318689 conn(0x55be0ec5e2a0 msgr2=0x55be0ec5b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/703318689 shutdown_connections 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/703318689 wait complete. 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- start start 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 Processor -- start 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0ec57f30 0x55be0ecae570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca1920 0x55be0ece4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ece5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0eaab3d0 con 0x55be0eca2240 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0eb62d50 con 0x55be0eca1920 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be0ea41980 con 0x55be0ec57f30 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 10 client.0 _ll_register_callbacks cb 0x7fb26c0b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb279672640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ece5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb279672640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ece5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60770/0 (socket says 172.21.15.133:60770) 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb279672640 1 -- 172.21.15.133:0/1733417614 learned_addr learned my addr 172.21.15.133:0/1733417614 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:23.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb273fff640 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca1920 0x55be0ece4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb278e71640 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0ec57f30 0x55be0ecae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb279672640 1 -- 172.21.15.133:0/1733417614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0ec57f30 msgr2=0x55be0ecae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb279672640 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0ec57f30 0x55be0ecae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb279672640 1 -- 172.21.15.133:0/1733417614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca1920 msgr2=0x55be0ece4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb279672640 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca1920 0x55be0ece4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb279672640 1 -- 172.21.15.133:0/1733417614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be0eb28600 con 0x55be0eca2240 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb279672640 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ece5360 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb26c019440 tx=0x7fb26c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefd74a6c7f8f48e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb2537fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb2537fe640 1 -- 172.21.15.133:0/1733417614 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0dc070 con 0x55be0eca2240 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb2537fe640 1 -- 172.21.15.133:0/1733417614 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb26c004330 con 0x55be0eca2240 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/1733417614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb26c0be040 con 0x55be0eca2240 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb2537fe640 1 -- 172.21.15.133:0/1733417614 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb26c0b2050 con 0x55be0eca2240 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 10 client.5709 fetch_fsmap learned FSMap version 16 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/1733417614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55be0ea3b9b0 con 0x55be0eca2240 2024-04-25T10:03:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb2537fe640 1 -- 172.21.15.133:0/1733417614 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0be040 con 0x55be0eca2240 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb2537fe640 1 -- 172.21.15.133:0/1733417614 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb26c0d5070 con 0x55be0eca2240 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 10 client.5709 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 10 client.5709 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 20 client.5709 populate_metadata read hostname 'smithi133' 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55be0ec5de60 0x55be0ece5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 4 client.5709 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/1733417614 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55be0eab42e0 con 0x55be0ec5de60 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb278e71640 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55be0ec5de60 0x55be0ece5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb2537fe640 10 client.5709 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb278e71640 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55be0ec5de60 0x55be0ece5c00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f812f565e911f215 server_cookie=28673393fd3a5186 in_seq=0 out_seq=0 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb2537fe640 1 -- 172.21.15.133:0/1733417614 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55be0eab42e0 con 0x55be0ec5de60 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb2537fe640 10 client.5709 handle_command_reply: tid=0 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 client.5709 shutdown 2024-04-25T10:03:23.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/1733417614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 msgr2=0x55be0ece5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ece5360 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb26c019440 tx=0x7fb26c004ac0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/1733417614 shutdown_connections 2024-04-25T10:03:23.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55be0ec5de60 0x55be0ece5c00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be0eca2240 0x55be0ece5360 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55be0eca1920 0x55be0ece4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb2537fe640 0 client.5709 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:03:23.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 --2- 172.21.15.133:0/1733417614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55be0ec57f30 0x55be0ecae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/1733417614 >> 172.21.15.133:0/1733417614 conn(0x55be0ec5e2a0 msgr2=0x55be0eca96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/1733417614 shutdown_connections 2024-04-25T10:03:23.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 1 -- 172.21.15.133:0/1733417614 wait complete. 2024-04-25T10:03:23.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.140+0000 7fb27e242000 20 client.5709 trim_cache size 0 max 16384 2024-04-25T10:03:23.156 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:03:23.156 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:03:23.156 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:03:23.157 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 5532, 2024-04-25T10:03:23.157 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:03:23.157 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:03:23.157 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:03:23.157 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 4.6964506869999996, 2024-04-25T10:03:23.157 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 16, 2024-04-25T10:03:23.157 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 38, 2024-04-25T10:03:23.158 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 38, 2024-04-25T10:03:23.158 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 11.725760528 2024-04-25T10:03:23.158 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:03:23.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 -- 172.21.15.133:0/3074067286 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb258066f60 msgr2=0x7fb258069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb258066f60 0x7fb258069420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb27415cf60 tx=0x7fb2600079c0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 -- 172.21.15.133:0/3074067286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb27406cc80 msgr2=0x7fb27414f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb27406cc80 0x7fb27414f330 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c400 tx=0x7fb26800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 -- 172.21.15.133:0/3074067286 shutdown_connections 2024-04-25T10:03:23.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2740d0a30 0x7fb27414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2740ce080 0x7fb2740b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb258066f60 0x7fb258069420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 --2- 172.21.15.133:0/3074067286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb27406cc80 0x7fb27414f330 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 -- 172.21.15.133:0/3074067286 >> 172.21.15.133:0/3074067286 conn(0x7fb27409ab90 msgr2=0x7fb2741593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 -- 172.21.15.133:0/3074067286 shutdown_connections 2024-04-25T10:03:23.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.144+0000 7fb27b0d5640 1 -- 172.21.15.133:0/3074067286 wait complete. 2024-04-25T10:03:23.164 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:03:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.256+0000 7fc631daf640 1 Processor -- start 2024-04-25T10:03:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.256+0000 7fc631daf640 1 -- start start 2024-04-25T10:03:23.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.256+0000 7fc631daf640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc62c06eb70 0x7fc62c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.256+0000 7fc631daf640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc62c1584f0 0x7fc62c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.256+0000 7fc631daf640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc62c0cbe60 con 0x7fc62c158110 2024-04-25T10:03:23.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.256+0000 7fc631daf640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62c0b19b0 con 0x7fc62c06eb70 2024-04-25T10:03:23.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.256+0000 7fc631daf640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62c15b360 con 0x7fc62c1584f0 2024-04-25T10:03:23.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.256+0000 7fc62affd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc62c06eb70 0x7fc62c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.256+0000 7fc62affd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc62c06eb70 0x7fc62c0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60778/0 (socket says 172.21.15.133:60778) 2024-04-25T10:03:23.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.256+0000 7fc62affd640 1 -- 172.21.15.133:0/3593647107 learned_addr learned my addr 172.21.15.133:0/3593647107 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:23.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc62bfff640 1 --2- 172.21.15.133:0/3593647107 >> v2:172.21.15.162:3300/0 conn(0x7fc62c1584f0 0x7fc62c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc62affd640 1 -- 172.21.15.133:0/3593647107 >> v2:172.21.15.162:3300/0 conn(0x7fc62c1584f0 msgr2=0x7fc62c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc62affd640 1 --2- 172.21.15.133:0/3593647107 >> v2:172.21.15.162:3300/0 conn(0x7fc62c1584f0 0x7fc62c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc62affd640 1 -- 172.21.15.133:0/3593647107 >> v1:172.21.15.162:6789/0 conn(0x7fc62c158110 legacy=0x7fc62c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc62affd640 1 -- 172.21.15.133:0/3593647107 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c15b640 con 0x7fc62c06eb70 2024-04-25T10:03:23.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc62affd640 1 --2- 172.21.15.133:0/3593647107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c06eb70 0x7fc62c0b1f00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0b2470 tx=0x7fc62001eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df83eba49dbb0275 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc62a7fc640 1 -- 172.21.15.133:0/3593647107 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200b4070 con 0x7fc62c06eb70 2024-04-25T10:03:23.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc62a7fc640 1 -- 172.21.15.133:0/3593647107 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6200a7b70 con 0x7fc62c06eb70 2024-04-25T10:03:23.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc62a7fc640 1 -- 172.21.15.133:0/3593647107 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200af440 con 0x7fc62c06eb70 2024-04-25T10:03:23.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc631daf640 1 -- 172.21.15.133:0/3593647107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c06eb70 msgr2=0x7fc62c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc631daf640 1 --2- 172.21.15.133:0/3593647107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c06eb70 0x7fc62c0b1f00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0b2470 tx=0x7fc62001eb80 comp rx=0 tx=0).stop 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc631daf640 1 -- 172.21.15.133:0/3593647107 shutdown_connections 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc631daf640 1 --2- 172.21.15.133:0/3593647107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c06eb70 0x7fc62c0b1f00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc631daf640 1 --2- 172.21.15.133:0/3593647107 >> v2:172.21.15.162:3300/0 conn(0x7fc62c1584f0 0x7fc62c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc631daf640 1 -- 172.21.15.133:0/3593647107 >> 172.21.15.133:0/3593647107 conn(0x7fc62c067000 msgr2=0x7fc62c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc631daf640 1 -- 172.21.15.133:0/3593647107 shutdown_connections 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.260+0000 7fc631daf640 1 -- 172.21.15.133:0/3593647107 wait complete. 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc631daf640 1 Processor -- start 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc631daf640 1 -- start start 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc631daf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc62c06eb70 0x7fc62c14f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc631daf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c158110 0x7fc62c0c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc631daf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c1584f0 0x7fc62c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc631daf640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c06dce0 con 0x7fc62c158110 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc631daf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c06db60 con 0x7fc62c1584f0 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc631daf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c06de60 con 0x7fc62c06eb70 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c158110 0x7fc62c0c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c158110 0x7fc62c0c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60790/0 (socket says 172.21.15.133:60790) 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62affd640 1 -- 172.21.15.133:0/2650039198 learned_addr learned my addr 172.21.15.133:0/2650039198 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62bfff640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c1584f0 0x7fc62c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62b7fe640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc62c06eb70 0x7fc62c14f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62affd640 1 -- 172.21.15.133:0/2650039198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc62c06eb70 msgr2=0x7fc62c14f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62affd640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc62c06eb70 0x7fc62c14f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62affd640 1 -- 172.21.15.133:0/2650039198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c1584f0 msgr2=0x7fc62c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62affd640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c1584f0 0x7fc62c156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62affd640 1 -- 172.21.15.133:0/2650039198 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c152030 con 0x7fc62c158110 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62bfff640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c1584f0 0x7fc62c156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:23.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62affd640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c158110 0x7fc62c0c6af0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc62000b2c0 tx=0x7fc62001b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc613fff640 1 -- 172.21.15.133:0/2650039198 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200b4040 con 0x7fc62c158110 2024-04-25T10:03:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc613fff640 1 -- 172.21.15.133:0/2650039198 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6200c1070 con 0x7fc62c158110 2024-04-25T10:03:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc631daf640 1 -- 172.21.15.133:0/2650039198 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62c152310 con 0x7fc62c158110 2024-04-25T10:03:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc613fff640 1 -- 172.21.15.133:0/2650039198 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200b9460 con 0x7fc62c158110 2024-04-25T10:03:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc631daf640 1 -- 172.21.15.133:0/2650039198 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c1528d0 con 0x7fc62c158110 2024-04-25T10:03:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc631daf640 1 -- 172.21.15.133:0/2650039198 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c068ef0 con 0x7fc62c158110 2024-04-25T10:03:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc613fff640 1 -- 172.21.15.133:0/2650039198 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6200b7070 con 0x7fc62c158110 2024-04-25T10:03:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc613fff640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc618066f60 0x7fc618069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc613fff640 1 -- 172.21.15.133:0/2650039198 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc620122210 con 0x7fc62c158110 2024-04-25T10:03:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.264+0000 7fc62b7fe640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc618066f60 0x7fc618069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.268+0000 7fc62b7fe640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc618066f60 0x7fc618069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc61c003d60 tx=0x7fc61c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.268+0000 7fc613fff640 1 -- 172.21.15.133:0/2650039198 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc6200187a0 con 0x7fc62c158110 2024-04-25T10:03:23.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.448+0000 7fc631daf640 1 -- 172.21.15.133:0/2650039198 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc62c06e500 con 0x7fc62c158110 2024-04-25T10:03:23.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.448+0000 7fc613fff640 1 -- 172.21.15.133:0/2650039198 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc6200ad340 con 0x7fc62c158110 2024-04-25T10:03:23.456 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:03:23.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 -- 172.21.15.133:0/2650039198 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc618066f60 msgr2=0x7fc618069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc618066f60 0x7fc618069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc61c003d60 tx=0x7fc61c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:23.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 -- 172.21.15.133:0/2650039198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c158110 msgr2=0x7fc62c0c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c158110 0x7fc62c0c6af0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc62000b2c0 tx=0x7fc62001b920 comp rx=0 tx=0).stop 2024-04-25T10:03:23.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 -- 172.21.15.133:0/2650039198 shutdown_connections 2024-04-25T10:03:23.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c1584f0 0x7fc62c156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc618066f60 0x7fc618069420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c158110 0x7fc62c0c6af0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 --2- 172.21.15.133:0/2650039198 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc62c06eb70 0x7fc62c14f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 -- 172.21.15.133:0/2650039198 >> 172.21.15.133:0/2650039198 conn(0x7fc62c067000 msgr2=0x7fc62c15ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 -- 172.21.15.133:0/2650039198 shutdown_connections 2024-04-25T10:03:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:23.452+0000 7fc631daf640 1 -- 172.21.15.133:0/2650039198 wait complete. 2024-04-25T10:03:23.473 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:03:23.473 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:03:23.473 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:03:23.473 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:03:23.473 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:23.473 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:03:23.473 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:03:23.473 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:03:23.473 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:23.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:23.474 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:03:23.479 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:03:23.479 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:03:23.479 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:03:23.479 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:23.479 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:03:23.479 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:23.479 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:03:23.479 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:23.479 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:03:23.479 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:03:23.479 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42428sec preferred_lft 42428sec 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:03:23.480 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:03:23.481 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:23.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:23.482 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:03:23.531 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:03:23.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:23.533 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:03:23.584 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:03:23.584 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:03:23.584 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:03:23.584 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:03:23.584 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:03:23.584 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:03:23.584 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:03:23.585 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:03:23.585 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:03:23.585 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:03:23.585 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:03:23.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:23.585 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:03:23.585 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:03:23.585 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:03:23.585 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:03:23.585 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:03:23.585 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:03:23.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:23.656 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:03:23.701 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:03:23.702 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:03:23.702 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:23.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:23.702 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:03:23.757 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:03:23.758 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:23.758 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:23.805 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:23.806 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:23.806 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:23.853 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:03:23.911 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:03:23.964 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:03:23.965 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:03:23.965 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:03:23.965 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:03:23.965 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:03:23.965 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:03:23.965 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:03:23.965 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:03:23.966 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:03:23.966 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:03:23.966 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:03:23.966 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:03:23.966 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:03:23.966 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:03:23.966 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:03:23.966 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:03:23.966 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:03:23.966 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:03:23.966 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:03:23.967 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:03:23.967 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:03:23.967 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:03:23.967 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:23.967 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:23.967 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:23.967 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:23.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:23.968 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:03:24.014 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:03:24.014 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:03:24.059 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:03:24.073 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:03:24.121 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:03:24.122 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:03:24.122 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:03:24.122 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:03:24.122 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:03:24.122 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:03:24.122 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:03:24.122 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:03:24.122 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:03:24.122 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:03:24.122 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:03:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:03:24.124 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:03:24.124 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:03:24.124 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:03:24.124 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:24.124 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:24.124 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:24.124 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:24.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:24.125 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:03:24.138 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:03:24.132+0000 7f550ab29580 -1 init, newargv = 0x55d96144ebb0 newargc=15 2024-04-25T10:03:24.138 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[39538]: starting ceph client 2024-04-25T10:03:24.144 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[39538]: starting fuse 2024-04-25T10:03:25.243 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:03:25.259 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:03:25.306 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:03:25.307 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:03:25.308 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:03:25.309 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:25.309 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:25.309 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:25.309 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:25.309 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:25.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.310 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:03:25.358 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:03:25.358 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:03:25.358 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:03:25.358 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:03:25.359 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:03:25.360 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:03:25.483 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.39538.asok 2024-04-25T10:03:25.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:25.494 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39538.asok 2024-04-25T10:03:25.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.495 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39538.asok status 2024-04-25T10:03:25.649 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "39538", 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:03:25.650 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 5724, 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 5724 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3425871221 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:03:25.651 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3425871221 2024-04-25T10:03:25.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:03:25.652 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.5724 192.168.0.1:0/3425871221", 2024-04-25T10:03:25.652 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/3425871221", 2024-04-25T10:03:25.652 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:03:25.652 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 16, 2024-04-25T10:03:25.652 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 38, 2024-04-25T10:03:25.652 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:03:25.652 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:03:25.652 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:03:25.652 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:03:25.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.664 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:03:25.664 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:03:25.670 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:25.670 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:25.723 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T10:03:25.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca4d8e0640 1 Processor -- start 2024-04-25T10:03:25.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca4d8e0640 1 -- start start 2024-04-25T10:03:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca4d8e0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fca480b4940 0x7fca480b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca4d8e0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fca4815c420 0x7fca480cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca4d8e0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca480b5260 con 0x7fca48071cb0 2024-04-25T10:03:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca4d8e0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca48072a90 con 0x7fca480b4940 2024-04-25T10:03:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca4d8e0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca480a85d0 con 0x7fca4815c420 2024-04-25T10:03:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca46ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fca480b4940 0x7fca480b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca46ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fca480b4940 0x7fca480b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60796/0 (socket says 172.21.15.133:60796) 2024-04-25T10:03:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca467fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fca48071cb0 0x7fca4815bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40830/0 (socket says 172.21.15.133:40830) 2024-04-25T10:03:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca46ffd640 1 -- 172.21.15.133:0/3540643559 learned_addr learned my addr 172.21.15.133:0/3540643559 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca477fe640 1 --2- 172.21.15.133:0/3540643559 >> v2:172.21.15.162:3300/0 conn(0x7fca4815c420 0x7fca480cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca46ffd640 1 -- 172.21.15.133:0/3540643559 >> v2:172.21.15.162:3300/0 conn(0x7fca4815c420 msgr2=0x7fca480cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca46ffd640 1 --2- 172.21.15.133:0/3540643559 >> v2:172.21.15.162:3300/0 conn(0x7fca4815c420 0x7fca480cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca46ffd640 1 -- 172.21.15.133:0/3540643559 >> v1:172.21.15.133:6789/0 conn(0x7fca48071cb0 legacy=0x7fca4815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca45ffb640 1 -- 172.21.15.133:0/3540643559 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460409516 0 0) 0x7fca480b5260 con 0x7fca48071cb0 2024-04-25T10:03:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.888+0000 7fca46ffd640 1 -- 172.21.15.133:0/3540643559 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4815c9b0 con 0x7fca480b4940 2024-04-25T10:03:25.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca46ffd640 1 --2- 172.21.15.133:0/3540643559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca480b4940 0x7fca480b4d20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fca480cbfc0 tx=0x7fca3801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21ff332b6b1d796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca45ffb640 1 -- 172.21.15.133:0/3540643559 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380aa650 con 0x7fca480b4940 2024-04-25T10:03:25.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca45ffb640 1 -- 172.21.15.133:0/3540643559 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca380af070 con 0x7fca480b4940 2024-04-25T10:03:25.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca45ffb640 1 -- 172.21.15.133:0/3540643559 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380aaa10 con 0x7fca480b4940 2024-04-25T10:03:25.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca4d8e0640 1 -- 172.21.15.133:0/3540643559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca480b4940 msgr2=0x7fca480b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca4d8e0640 1 --2- 172.21.15.133:0/3540643559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca480b4940 0x7fca480b4d20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fca480cbfc0 tx=0x7fca3801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca4d8e0640 1 -- 172.21.15.133:0/3540643559 shutdown_connections 2024-04-25T10:03:25.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca4d8e0640 1 --2- 172.21.15.133:0/3540643559 >> v2:172.21.15.162:3300/0 conn(0x7fca4815c420 0x7fca480cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca4d8e0640 1 --2- 172.21.15.133:0/3540643559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca480b4940 0x7fca480b4d20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca4d8e0640 1 -- 172.21.15.133:0/3540643559 >> 172.21.15.133:0/3540643559 conn(0x7fca4806d370 msgr2=0x7fca4806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca4d8e0640 1 -- 172.21.15.133:0/3540643559 shutdown_connections 2024-04-25T10:03:25.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca4d8e0640 1 -- 172.21.15.133:0/3540643559 wait complete. 2024-04-25T10:03:25.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca4d8e0640 1 Processor -- start 2024-04-25T10:03:25.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.892+0000 7fca4d8e0640 1 -- start start 2024-04-25T10:03:25.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca4d8e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca48071cb0 0x7fca480aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca4d8e0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca480b4940 0x7fca480ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca4d8e0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca4815c420 0x7fca4813e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca4d8e0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca480cc960 con 0x7fca48071cb0 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca4d8e0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca48071790 con 0x7fca4815c420 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca4d8e0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca480ccae0 con 0x7fca480b4940 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca46ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca48071cb0 0x7fca480aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca46ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca48071cb0 0x7fca480aba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60800/0 (socket says 172.21.15.133:60800) 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca46ffd640 1 -- 172.21.15.133:0/329730768 learned_addr learned my addr 172.21.15.133:0/329730768 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca477fe640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca4815c420 0x7fca4813e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca467fc640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca480b4940 0x7fca480ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca46ffd640 1 -- 172.21.15.133:0/329730768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca480b4940 msgr2=0x7fca480ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca46ffd640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca480b4940 0x7fca480ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca46ffd640 1 -- 172.21.15.133:0/329730768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca4815c420 msgr2=0x7fca4813e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca46ffd640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca4815c420 0x7fca4813e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca46ffd640 1 -- 172.21.15.133:0/329730768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca48159170 con 0x7fca48071cb0 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca477fe640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca4815c420 0x7fca4813e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca46ffd640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca48071cb0 0x7fca480aba90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fca3801a120 tx=0x7fca3801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca2ffff640 1 -- 172.21.15.133:0/329730768 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380bd5f0 con 0x7fca48071cb0 2024-04-25T10:03:25.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca2ffff640 1 -- 172.21.15.133:0/329730768 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca380a6040 con 0x7fca48071cb0 2024-04-25T10:03:25.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca4d8e0640 1 -- 172.21.15.133:0/329730768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca48158bb0 con 0x7fca48071cb0 2024-04-25T10:03:25.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca2ffff640 1 -- 172.21.15.133:0/329730768 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380bd9b0 con 0x7fca48071cb0 2024-04-25T10:03:25.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca4d8e0640 1 -- 172.21.15.133:0/329730768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca480ab570 con 0x7fca48071cb0 2024-04-25T10:03:25.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca4d8e0640 1 -- 172.21.15.133:0/329730768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca48069000 con 0x7fca48071cb0 2024-04-25T10:03:25.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca2ffff640 1 -- 172.21.15.133:0/329730768 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca380bdb50 con 0x7fca48071cb0 2024-04-25T10:03:25.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca2ffff640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fca34066f60 0x7fca34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca2ffff640 1 -- 172.21.15.133:0/329730768 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fca3811f5b0 con 0x7fca48071cb0 2024-04-25T10:03:25.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca467fc640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fca34066f60 0x7fca34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.896+0000 7fca467fc640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fca34066f60 0x7fca34069420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fca480ae5d0 tx=0x7fca3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:25.900+0000 7fca2ffff640 1 -- 172.21.15.133:0/329730768 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca380c3060 con 0x7fca48071cb0 2024-04-25T10:03:26.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.092+0000 7fca4d8e0640 1 -- 172.21.15.133:0/329730768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fca48158f40 con 0x7fca48071cb0 2024-04-25T10:03:26.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.792+0000 7fca2ffff640 1 -- 172.21.15.133:0/329730768 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v17) v1 ==== 167+0+0 (secure 0 0 0) 0x7fca38016110 con 0x7fca48071cb0 2024-04-25T10:03:26.801 INFO:teuthology.orchestra.run.smithi133.stderr:enabled new snapshots 2024-04-25T10:03:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 -- 172.21.15.133:0/329730768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fca34066f60 msgr2=0x7fca34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fca34066f60 0x7fca34069420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fca480ae5d0 tx=0x7fca3c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 -- 172.21.15.133:0/329730768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca48071cb0 msgr2=0x7fca480aba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca48071cb0 0x7fca480aba90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fca3801a120 tx=0x7fca3801ad30 comp rx=0 tx=0).stop 2024-04-25T10:03:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 -- 172.21.15.133:0/329730768 shutdown_connections 2024-04-25T10:03:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca4815c420 0x7fca4813e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fca480b4940 0x7fca480ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fca34066f60 0x7fca34069420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 --2- 172.21.15.133:0/329730768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca48071cb0 0x7fca480aba90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 -- 172.21.15.133:0/329730768 >> 172.21.15.133:0/329730768 conn(0x7fca4806d370 msgr2=0x7fca4815d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 -- 172.21.15.133:0/329730768 shutdown_connections 2024-04-25T10:03:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:26.796+0000 7fca4d8e0640 1 -- 172.21.15.133:0/329730768 wait complete. 2024-04-25T10:03:26.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:26.824 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T10:03:26.836 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:03:26.836 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:26.886 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-04-25T10:03:26.886 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:03:26.886 DEBUG:teuthology.orchestra.run.smithi133:> import uuid 2024-04-25T10:03:26.886 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:03:26.886 DEBUG:teuthology.orchestra.run.smithi133:> n = 2000 2024-04-25T10:03:26.886 DEBUG:teuthology.orchestra.run.smithi133:> create_hard_links = False 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> if 3 > 0: 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> create_hard_links = True 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> dpath = os.path.dirname(path) 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> fnameprefix = os.path.basename(path) 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> try: 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> for i in range(n): 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:03:26.887 DEBUG:teuthology.orchestra.run.smithi133:> f.write(f"{i}") 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> f.flush() 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(f.fileno()) 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> os.unlink(fpath) 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(dirfd) 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> if create_hard_links: 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> for j in range(3): 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(dirfd) 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> finally: 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> os.close(dirfd) 2024-04-25T10:03:26.888 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:03:49.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:50.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:50.003 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:03:50.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:50.088 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T10:03:50.482 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:50.484 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:20.525 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:20.529 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:49.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:49.230 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:04:49.291 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:04:49.291 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T10:04:49.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b75184640 1 Processor -- start 2024-04-25T10:04:49.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b75184640 1 -- start start 2024-04-25T10:04:49.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b75184640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2b70145170 0x7f2b7006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b75184640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2b7015ced0 0x7f2b7015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b75184640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b700ce2e0 con 0x7f2b7006e4d0 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b75184640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7006db00 con 0x7f2b70145170 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b75184640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b70145550 con 0x7f2b7015ced0 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b6ed9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2b70145170 0x7f2b7006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b6e59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2b7006e4d0 0x7f2b700cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49306/0 (socket says 172.21.15.133:49306) 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b6e59c640 1 -- 172.21.15.133:0/4006024587 learned_addr learned my addr 172.21.15.133:0/4006024587 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b6f59e640 1 --2- 172.21.15.133:0/4006024587 >> v2:172.21.15.162:3300/0 conn(0x7f2b7015ced0 0x7f2b7015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b6ed9d640 1 -- 172.21.15.133:0/4006024587 >> v2:172.21.15.162:3300/0 conn(0x7f2b7015ced0 msgr2=0x7f2b7015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/4006024587 >> v2:172.21.15.162:3300/0 conn(0x7f2b7015ced0 0x7f2b7015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b6ed9d640 1 -- 172.21.15.133:0/4006024587 >> v1:172.21.15.133:6789/0 conn(0x7f2b7006e4d0 legacy=0x7f2b700cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b6ed9d640 1 -- 172.21.15.133:0/4006024587 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7006d4b0 con 0x7f2b70145170 2024-04-25T10:04:49.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b6dd9b640 1 -- 172.21.15.133:0/4006024587 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312729750 0 0) 0x7f2b700ce2e0 con 0x7f2b7006e4d0 2024-04-25T10:04:49.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.415+0000 7f2b6f59e640 1 --2- 172.21.15.133:0/4006024587 >> v2:172.21.15.162:3300/0 conn(0x7f2b7015ced0 0x7f2b7015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:49.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/4006024587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b70145170 0x7f2b7006df90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2b700b27f0 tx=0x7f2b6001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521df20f0f87002a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6dd9b640 1 -- 172.21.15.133:0/4006024587 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b600b0070 con 0x7f2b70145170 2024-04-25T10:04:49.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6dd9b640 1 -- 172.21.15.133:0/4006024587 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b600ab600 con 0x7f2b70145170 2024-04-25T10:04:49.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6dd9b640 1 -- 172.21.15.133:0/4006024587 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b600ab9d0 con 0x7f2b70145170 2024-04-25T10:04:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 -- 172.21.15.133:0/4006024587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b70145170 msgr2=0x7f2b7006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 --2- 172.21.15.133:0/4006024587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b70145170 0x7f2b7006df90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2b700b27f0 tx=0x7f2b6001c480 comp rx=0 tx=0).stop 2024-04-25T10:04:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 -- 172.21.15.133:0/4006024587 shutdown_connections 2024-04-25T10:04:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 --2- 172.21.15.133:0/4006024587 >> v2:172.21.15.162:3300/0 conn(0x7f2b7015ced0 0x7f2b7015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 --2- 172.21.15.133:0/4006024587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b70145170 0x7f2b7006df90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 -- 172.21.15.133:0/4006024587 >> 172.21.15.133:0/4006024587 conn(0x7f2b70066f70 msgr2=0x7f2b7006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 -- 172.21.15.133:0/4006024587 shutdown_connections 2024-04-25T10:04:49.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 -- 172.21.15.133:0/4006024587 wait complete. 2024-04-25T10:04:49.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 Processor -- start 2024-04-25T10:04:49.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 -- start start 2024-04-25T10:04:49.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b7006e4d0 0x7f2b701405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b70145170 0x7f2b7013ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b7015ced0 0x7f2b701540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70168610 con 0x7f2b70145170 2024-04-25T10:04:49.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70168490 con 0x7f2b7006e4d0 2024-04-25T10:04:49.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b75184640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70168790 con 0x7f2b7015ced0 2024-04-25T10:04:49.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b70145170 0x7f2b7013ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b70145170 0x7f2b7013ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47376/0 (socket says 172.21.15.133:47376) 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3108010320 learned_addr learned my addr 172.21.15.133:0/3108010320 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b7006e4d0 0x7f2b701405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3108010320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b7015ced0 msgr2=0x7f2b701540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b7015ced0 0x7f2b701540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3108010320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b7006e4d0 msgr2=0x7f2b701405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b7006e4d0 0x7f2b701405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3108010320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7013f450 con 0x7f2b70145170 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b70145170 0x7f2b7013ea70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400c4e0 tx=0x7f2b6400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.419+0000 7f2b577fe640 1 -- 172.21.15.133:0/3108010320 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64018070 con 0x7f2b70145170 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.423+0000 7f2b577fe640 1 -- 172.21.15.133:0/3108010320 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b64007500 con 0x7f2b70145170 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.423+0000 7f2b75184640 1 -- 172.21.15.133:0/3108010320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7013f730 con 0x7f2b70145170 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.423+0000 7f2b577fe640 1 -- 172.21.15.133:0/3108010320 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64004a30 con 0x7f2b70145170 2024-04-25T10:04:49.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.423+0000 7f2b75184640 1 -- 172.21.15.133:0/3108010320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7013fcf0 con 0x7f2b70145170 2024-04-25T10:04:49.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.423+0000 7f2b577fe640 1 -- 172.21.15.133:0/3108010320 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b6401b070 con 0x7f2b70145170 2024-04-25T10:04:49.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.423+0000 7f2b577fe640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b5c066f60 0x7f2b5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.423+0000 7f2b577fe640 1 -- 172.21.15.133:0/3108010320 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2b640845c0 con 0x7f2b70145170 2024-04-25T10:04:49.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.423+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b5c066f60 0x7f2b5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.423+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b5c066f60 0x7f2b5c069420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2b600ad000 tx=0x7f2b60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.423+0000 7f2b782f1000 1 Processor -- start 2024-04-25T10:04:49.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.423+0000 7f2b782f1000 1 -- start start 2024-04-25T10:04:49.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c7a40 0x563db23c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c8360 0x563db2382850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db2383170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db22651f0 con 0x563db23c8360 2024-04-25T10:04:49.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db21cbe90 con 0x563db23c7a40 2024-04-25T10:04:49.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563db2153f50 con 0x563db2382d90 2024-04-25T10:04:49.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c8360 0x563db2382850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c8360 0x563db2382850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47390/0 (socket says 172.21.15.133:47390) 2024-04-25T10:04:49.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6e59c640 1 -- 172.21.15.133:0/1627894413 learned_addr learned my addr 172.21.15.133:0/1627894413 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/1627894413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c7a40 0x563db23c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6e59c640 1 -- 172.21.15.133:0/1627894413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 msgr2=0x563db2383170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/1627894413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db2383170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6e59c640 1 -- 172.21.15.133:0/1627894413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c7a40 msgr2=0x563db23c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/1627894413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c7a40 0x563db23c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6e59c640 1 -- 172.21.15.133:0/1627894413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563db21a38b0 con 0x563db23c8360 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/1627894413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c8360 0x563db2382850 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400c4b0 tx=0x7f2b64054e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c952e49f9d0948b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b54ff9640 1 -- 172.21.15.133:0/1627894413 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64053050 con 0x563db23c8360 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b54ff9640 1 -- 172.21.15.133:0/1627894413 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b640513c0 con 0x563db23c8360 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b54ff9640 1 -- 172.21.15.133:0/1627894413 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6405d2a0 con 0x563db23c8360 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- 172.21.15.133:0/1627894413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c8360 msgr2=0x563db2382850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 --2- 172.21.15.133:0/1627894413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c8360 0x563db2382850 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400c4b0 tx=0x7f2b64054e90 comp rx=0 tx=0).stop 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- 172.21.15.133:0/1627894413 shutdown_connections 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 --2- 172.21.15.133:0/1627894413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db2383170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 --2- 172.21.15.133:0/1627894413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c8360 0x563db2382850 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 --2- 172.21.15.133:0/1627894413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c7a40 0x563db23c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- 172.21.15.133:0/1627894413 >> 172.21.15.133:0/1627894413 conn(0x563db237ed40 msgr2=0x563db2381130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- 172.21.15.133:0/1627894413 shutdown_connections 2024-04-25T10:04:49.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- 172.21.15.133:0/1627894413 wait complete. 2024-04-25T10:04:49.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- start start 2024-04-25T10:04:49.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 Processor -- start 2024-04-25T10:04:49.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db2382d90 0x563db23cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c7a40 0x563db23cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db23c8360 0x563db2388e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db2186540 con 0x563db2382d90 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db21c7f30 con 0x563db23c7a40 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563db2172c00 con 0x563db23c8360 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 10 client.0 _ll_register_callbacks cb 0x563db2280180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db2382d90 0x563db23cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db2382d90 0x563db23cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47404/0 (socket says 172.21.15.133:47404) 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6ed9d640 1 -- 172.21.15.133:0/4129930706 learned_addr learned my addr 172.21.15.133:0/4129930706 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c7a40 0x563db23cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6f59e640 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db23c8360 0x563db2388e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6ed9d640 1 -- 172.21.15.133:0/4129930706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db23c8360 msgr2=0x563db2388e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db23c8360 0x563db2388e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6ed9d640 1 -- 172.21.15.133:0/4129930706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c7a40 msgr2=0x563db23cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c7a40 0x563db23cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6ed9d640 1 -- 172.21.15.133:0/4129930706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563db21541a0 con 0x563db2382d90 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db2382d90 0x563db23cf1c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2b600ae040 tx=0x7f2b600046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e889fa0bd52110 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b4dffb640 1 -- 172.21.15.133:0/4129930706 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b600c7060 con 0x563db2382d90 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b4dffb640 1 -- 172.21.15.133:0/4129930706 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2b600b7770 con 0x563db2382d90 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.427+0000 7f2b782f1000 1 -- 172.21.15.133:0/4129930706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b640513c0 con 0x563db2382d90 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b4dffb640 1 -- 172.21.15.133:0/4129930706 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b600cd050 con 0x563db2382d90 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 10 client.5748 fetch_fsmap learned FSMap version 17 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 -- 172.21.15.133:0/4129930706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x563db224e840 con 0x563db2382d90 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b4dffb640 1 -- 172.21.15.133:0/4129930706 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b640513c0 con 0x563db2382d90 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b4dffb640 1 -- 172.21.15.133:0/4129930706 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b600d0020 con 0x563db2382d90 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 10 client.5748 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 10 client.5748 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 20 client.5748 populate_metadata read hostname 'smithi133' 2024-04-25T10:04:49.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x563db2389c00 0x563db2389fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 4 client.5748 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 -- 172.21.15.133:0/4129930706 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563db2167be0 con 0x563db2389c00 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x563db2389c00 0x563db2389fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b4dffb640 10 client.5748 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x563db2389c00 0x563db2389fe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c115d26bfdd006 server_cookie=9c39b0c2ae234d29 in_seq=0 out_seq=0 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b4dffb640 1 -- 172.21.15.133:0/4129930706 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563db2167be0 con 0x563db2389c00 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b4dffb640 10 client.5748 handle_command_reply: tid=0 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 client.5748 shutdown 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 -- 172.21.15.133:0/4129930706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db2382d90 msgr2=0x563db23cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db2382d90 0x563db23cf1c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2b600ae040 tx=0x7f2b600046d0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 -- 172.21.15.133:0/4129930706 shutdown_connections 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x563db2389c00 0x563db2389fe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db23c8360 0x563db2388e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c7a40 0x563db23cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b4dffb640 0 client.5748 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 --2- 172.21.15.133:0/4129930706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db2382d90 0x563db23cf1c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 -- 172.21.15.133:0/4129930706 >> 172.21.15.133:0/4129930706 conn(0x563db237ed40 msgr2=0x563db23819c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 -- 172.21.15.133:0/4129930706 shutdown_connections 2024-04-25T10:04:49.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 1 -- 172.21.15.133:0/4129930706 wait complete. 2024-04-25T10:04:49.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.431+0000 7f2b782f1000 20 client.5748 trim_cache size 0 max 16384 2024-04-25T10:04:49.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 Processor -- start 2024-04-25T10:04:49.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- start start 2024-04-25T10:04:49.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db2424080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db24245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c8360 0x563db2424b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b702010c0 con 0x563db23c7a40 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b702002a0 con 0x563db23c8360 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70136390 con 0x563db2382d90 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db24245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db24245c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47416/0 (socket says 172.21.15.133:47416) 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3038762051 learned_addr learned my addr 172.21.15.133:0/3038762051 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/3038762051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db2424080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3038762051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 msgr2=0x563db2424080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/3038762051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db2424080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3038762051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c8360 msgr2=0x563db2424b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/3038762051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c8360 0x563db2424b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3038762051 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563db2225a10 con 0x563db23c7a40 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/3038762051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db24245c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2b640516d0 tx=0x7f2b64084000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff73dc3e215e858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b4ffff640 1 -- 172.21.15.133:0/3038762051 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640513c0 con 0x563db23c7a40 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b4ffff640 1 -- 172.21.15.133:0/3038762051 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6405d060 con 0x563db23c7a40 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b4ffff640 1 -- 172.21.15.133:0/3038762051 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640541a0 con 0x563db23c7a40 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- 172.21.15.133:0/3038762051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 msgr2=0x563db24245c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 --2- 172.21.15.133:0/3038762051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db24245c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2b640516d0 tx=0x7f2b64084000 comp rx=0 tx=0).stop 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- 172.21.15.133:0/3038762051 shutdown_connections 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 --2- 172.21.15.133:0/3038762051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c8360 0x563db2424b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 --2- 172.21.15.133:0/3038762051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db24245c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 --2- 172.21.15.133:0/3038762051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db2424080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- 172.21.15.133:0/3038762051 >> 172.21.15.133:0/3038762051 conn(0x563db237ed40 msgr2=0x563db23d58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- 172.21.15.133:0/3038762051 shutdown_connections 2024-04-25T10:04:49.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- 172.21.15.133:0/3038762051 wait complete. 2024-04-25T10:04:49.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- start start 2024-04-25T10:04:49.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 Processor -- start 2024-04-25T10:04:49.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db23ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db23ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c8360 0x563db23cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db21c7f30 con 0x563db23c7a40 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70136990 con 0x563db23c8360 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563db2288d10 con 0x563db2382d90 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b782f1000 10 client.0 _ll_register_callbacks cb 0x563db218ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db23ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db23ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47424/0 (socket says 172.21.15.133:47424) 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3334249966 learned_addr learned my addr 172.21.15.133:0/3334249966 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6f59e640 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c8360 0x563db23cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.447+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db23ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3334249966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 msgr2=0x563db23ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db23ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3334249966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c8360 msgr2=0x563db23cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c8360 0x563db23cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b6e59c640 1 -- 172.21.15.133:0/3334249966 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563db2160b00 con 0x563db23c7a40 2024-04-25T10:04:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b6f59e640 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c8360 0x563db23cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db23ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b6e59c640 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db23ccfe0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400cea0 tx=0x7f2b64014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfc161720303eee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:04:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b4dffb640 1 -- 172.21.15.133:0/3334249966 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640513c0 con 0x563db23c7a40 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b4dffb640 1 -- 172.21.15.133:0/3334249966 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2b640541a0 con 0x563db23c7a40 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 -- 172.21.15.133:0/3334249966 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b6405d060 con 0x563db23c7a40 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b4dffb640 1 -- 172.21.15.133:0/3334249966 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b640543c0 con 0x563db23c7a40 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 10 client.5754 fetch_fsmap learned FSMap version 17 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 -- 172.21.15.133:0/3334249966 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x563db2172f80 con 0x563db23c7a40 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b4dffb640 1 -- 172.21.15.133:0/3334249966 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6405d060 con 0x563db23c7a40 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b4dffb640 1 -- 172.21.15.133:0/3334249966 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b64050aa0 con 0x563db23c7a40 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 10 client.5754 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 10 client.5754 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 20 client.5754 populate_metadata read hostname 'smithi133' 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x563db242dcd0 0x563db24300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 4 client.5754 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:04:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 -- 172.21.15.133:0/3334249966 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x563db21d93d0 con 0x563db242dcd0 2024-04-25T10:04:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x563db242dcd0 0x563db24300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b6ed9d640 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x563db242dcd0 0x563db24300c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f775c7c5cd49e134 server_cookie=f39d2841d70b351a in_seq=0 out_seq=0 2024-04-25T10:04:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b4dffb640 10 client.5754 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b4dffb640 1 -- 172.21.15.133:0/3334249966 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17929 (crc 0 0 0) 0x563db21d93d0 con 0x563db242dcd0 2024-04-25T10:04:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b4dffb640 10 client.5754 handle_command_reply: tid=0 2024-04-25T10:04:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 client.5754 shutdown 2024-04-25T10:04:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 -- 172.21.15.133:0/3334249966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 msgr2=0x563db23ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db23ccfe0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400cea0 tx=0x7f2b64014070 comp rx=0 tx=0).stop 2024-04-25T10:04:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 -- 172.21.15.133:0/3334249966 shutdown_connections 2024-04-25T10:04:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x563db242dcd0 0x563db24300c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563db23c8360 0x563db23cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563db23c7a40 0x563db23ccfe0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 --2- 172.21.15.133:0/3334249966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563db2382d90 0x563db23ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 -- 172.21.15.133:0/3334249966 >> 172.21.15.133:0/3334249966 conn(0x563db237ed40 msgr2=0x563db23c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b4dffb640 0 client.5754 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 -- 172.21.15.133:0/3334249966 shutdown_connections 2024-04-25T10:04:49.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 1 -- 172.21.15.133:0/3334249966 wait complete. 2024-04-25T10:04:49.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b782f1000 20 client.5754 trim_cache size 0 max 16384 2024-04-25T10:04:49.466 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:04:49.466 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 4030, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 4031, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1002446, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 39168409, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 9, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.469569295, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.209803926, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.090108364, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.139056680, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4029, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.494518417, 2024-04-25T10:04:49.467 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000122739 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2, 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000297, 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000148 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2845, 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2841, 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 801772, 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 17696999, 2024-04-25T10:04:49.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 5, 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.285128584, 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.115587719, 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.064628569, 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.088929335, 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2841, 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.397780667, 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000140014 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 115552, 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 23824 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 34237, 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 60472, 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 12382587, 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 58971795, 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 8, 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 4.750510948, 2024-04-25T10:04:49.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 2.375913502, 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 2.839003582, 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.098331687, 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 60471, 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.882915674, 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000031137 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 87501, 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.020080683, 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000229 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 119696 2024-04-25T10:04:49.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8736, 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 23.249700612, 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002661366 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-04-25T10:04:49.472 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1917, 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.218961098, 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000635869 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 26425, 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 26425, 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 26425, 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 432.619534874, 2024-04-25T10:04:49.473 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.016371600 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 12, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 19884, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 19884, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-04-25T10:04:49.474 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 19884, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 386, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 386, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 2, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 42639, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 42192, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 141, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-04-25T10:04:49.475 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:04:49.476 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 7798, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 6340, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 7798, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:04:49.477 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 45, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 17976 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 2000, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 230, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 4140, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 1616, 2024-04-25T10:04:49.478 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 2224, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-04-25T10:04:49.479 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 23785, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 23785, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 31, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 31, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 3, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 85675937, 2024-04-25T10:04:49.480 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6854, 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 118.629896302, 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.017308126 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 2014, 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 2014, 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 0, 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 13, 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 13, 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 0, 2024-04-25T10:04:49.481 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 19884, 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 19884, 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 0, 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 156, 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 2002, 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 1846, 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 101804, 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215340 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 35551, 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 26425, 2024-04-25T10:04:49.482 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 7, 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2000, 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.344601492, 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000672300 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 260, 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.082059183, 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000315612 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6000, 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 9.525815933, 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001587635 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7928, 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 14.913641301, 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001881135 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.484 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3, 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.032204813, 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.010734937 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-04-25T10:04:49.486 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.003627006, 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003627006 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.009036052, 2024-04-25T10:04:49.487 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.009036052 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6, 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.160938609, 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.026823101 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2224, 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 378.677113062, 2024-04-25T10:04:49.488 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.170268486 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-04-25T10:04:49.489 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.006591237, 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.006591237 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.490 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000299273, 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000299273 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.491 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.492 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8000, 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 27.863606913, 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003482950 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-04-25T10:04:49.493 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 1, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 0, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 1, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 15254, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 15254, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 82, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:04:49.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-25T10:04:49.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-25T10:04:49.496 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 16771108, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 4231, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-25T10:04:49.497 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 5520, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 78, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 20721501, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 807811, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.498 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 8863, 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 81768234, 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 37, 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 8863, 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8863, 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 43.539548880, 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004912506 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-04-25T10:04:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8863, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 9009 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 8863, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 99, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 8764, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 0, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 11, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 99, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 6953, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 58, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-04-25T10:04:49.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 1616, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 31, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 22, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-04-25T10:04:49.501 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-04-25T10:04:49.502 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 40, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 3, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 8, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 8, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 175, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 0, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 44 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2024-04-25T10:04:49.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 32, 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 1616, 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 1616, 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.504 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 41108, 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 10481779, 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 41108, 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 41108, 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 10481779, 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:04:49.505 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:04:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 8863, 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 81944378, 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 8863, 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 81944378, 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-04-25T10:04:49.507 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 8863, 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 8863, 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 8863, 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 8863, 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:04:49.508 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 1616, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 176144, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 1616, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 99, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 176144, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.509 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55a5d831a960": { 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 84, 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 23785, 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 81481633, 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 23785, 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 6854, 2024-04-25T10:04:49.510 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 81481549, 2024-04-25T10:04:49.511 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:04:49.511 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:04:49.511 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:04:49.511 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:04:49.511 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:04:49.511 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:04:49.511 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:04:49.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 -- 172.21.15.133:0/3108010320 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b5c066f60 msgr2=0x7f2b5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b5c066f60 0x7f2b5c069420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2b600ad000 tx=0x7f2b60008040 comp rx=0 tx=0).stop 2024-04-25T10:04:49.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 -- 172.21.15.133:0/3108010320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b70145170 msgr2=0x7f2b7013ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b70145170 0x7f2b7013ea70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400c4e0 tx=0x7f2b6400c9a0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 -- 172.21.15.133:0/3108010320 shutdown_connections 2024-04-25T10:04:49.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b7015ced0 0x7f2b701540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b5c066f60 0x7f2b5c069420 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b70145170 0x7f2b7013ea70 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 --2- 172.21.15.133:0/3108010320 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b7006e4d0 0x7f2b701405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 -- 172.21.15.133:0/3108010320 >> 172.21.15.133:0/3108010320 conn(0x7f2b70066f70 msgr2=0x7f2b7015f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 -- 172.21.15.133:0/3108010320 shutdown_connections 2024-04-25T10:04:49.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.451+0000 7f2b75184640 1 -- 172.21.15.133:0/3108010320 wait complete. 2024-04-25T10:04:49.514 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T10:04:49.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:49.515 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T10:04:49.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.623+0000 7f2219309640 1 Processor -- start 2024-04-25T10:04:49.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.623+0000 7f2219309640 1 -- start start 2024-04-25T10:04:49.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f2219309640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22140ceda0 0x7f2214165490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f2219309640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f221415bf40 con 0x7f221415b910 2024-04-25T10:04:49.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f2219309640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f221415c200 con 0x7f221415b530 2024-04-25T10:04:49.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f2219309640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22141589c0 con 0x7f22140ceda0 2024-04-25T10:04:49.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f220bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f221415b910 0x7f22140ce690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49308/0 (socket says 172.21.15.133:49308) 2024-04-25T10:04:49.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f220bfff640 1 -- 172.21.15.133:0/1734821155 learned_addr learned my addr 172.21.15.133:0/1734821155 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:04:49.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267081821 0 0) 0x7f221415bf40 con 0x7f221415b910 2024-04-25T10:04:49.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2200003530 con 0x7f221415b910 2024-04-25T10:04:49.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22137fe640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22140ceda0 0x7f2214165490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 192643933 0 0) 0x7f2200003530 con 0x7f221415b910 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22140ceda0 msgr2=0x7f2214165490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22140ceda0 0x7f2214165490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 >> v1:172.21.15.162:6789/0 conn(0x7f221415b530 legacy=0x7f22140cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221415c4e0 con 0x7f221415b910 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f21fc001680 con 0x7f221415b910 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 >> v1:172.21.15.133:6789/0 conn(0x7f221415b910 legacy=0x7f22140ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22000039f0 0x7f2200003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2200004810 0x7f2200004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22140ceda0 0x7f220000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2200005500 con 0x7f2200004810 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220000bf10 con 0x7f22000039f0 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220000c090 con 0x7f22140ceda0 2024-04-25T10:04:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f220bfff640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2200004810 0x7f2200004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f2212ffd640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22000039f0 0x7f2200003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f22137fe640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22140ceda0 0x7f220000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f220bfff640 1 -- 172.21.15.133:0/1734821155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22140ceda0 msgr2=0x7f220000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f220bfff640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22140ceda0 0x7f220000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f220bfff640 1 -- 172.21.15.133:0/1734821155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22000039f0 msgr2=0x7f2200003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f220bfff640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22000039f0 0x7f2200003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.627+0000 7f220bfff640 1 -- 172.21.15.133:0/1734821155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2200003530 con 0x7f2200004810 2024-04-25T10:04:49.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f220bfff640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2200004810 0x7f2200004c90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c210 tx=0x7f21fc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc59220f2e34e76f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc007d20 con 0x7f2200004810 2024-04-25T10:04:49.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc0a9020 con 0x7f2200004810 2024-04-25T10:04:49.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22127fc640 1 -- 172.21.15.133:0/1734821155 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc005b00 con 0x7f2200004810 2024-04-25T10:04:49.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 -- 172.21.15.133:0/1734821155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2200004810 msgr2=0x7f2200004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 --2- 172.21.15.133:0/1734821155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2200004810 0x7f2200004c90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c210 tx=0x7f21fc01cc60 comp rx=0 tx=0).stop 2024-04-25T10:04:49.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22137fe640 1 -- 172.21.15.133:0/1734821155 reap_dead start 2024-04-25T10:04:49.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 -- 172.21.15.133:0/1734821155 shutdown_connections 2024-04-25T10:04:49.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 -- 172.21.15.133:0/1734821155 >> 172.21.15.133:0/1734821155 conn(0x7f221406d370 msgr2=0x7f221406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 -- 172.21.15.133:0/1734821155 shutdown_connections 2024-04-25T10:04:49.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 -- 172.21.15.133:0/1734821155 wait complete. 2024-04-25T10:04:49.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 Processor -- start 2024-04-25T10:04:49.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 -- start start 2024-04-25T10:04:49.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2200004810 0x7f221414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22141503b0 0x7f2214162470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22141f6330 0x7f22141568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2214165ca0 con 0x7f22141f6330 2024-04-25T10:04:49.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221415bd80 con 0x7f2200004810 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2214165e20 con 0x7f22141503b0 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22141f6330 0x7f22141568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22141f6330 0x7f22141568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47444/0 (socket says 172.21.15.133:47444) 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22137fe640 1 -- 172.21.15.133:0/3639549478 learned_addr learned my addr 172.21.15.133:0/3639549478 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f220bfff640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22141503b0 0x7f2214162470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2212ffd640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2200004810 0x7f221414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22137fe640 1 -- 172.21.15.133:0/3639549478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22141503b0 msgr2=0x7f2214162470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22137fe640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22141503b0 0x7f2214162470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22137fe640 1 -- 172.21.15.133:0/3639549478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2200004810 msgr2=0x7f221414fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22137fe640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2200004810 0x7f221414fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22137fe640 1 -- 172.21.15.133:0/3639549478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22141592c0 con 0x7f22141f6330 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f22137fe640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22141f6330 0x7f22141568e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f220c0049e0 tx=0x7f220c009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f220b7fe640 1 -- 172.21.15.133:0/3639549478 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c007de0 con 0x7f22141f6330 2024-04-25T10:04:49.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f220b7fe640 1 -- 172.21.15.133:0/3639549478 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c019070 con 0x7f22141f6330 2024-04-25T10:04:49.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 -- 172.21.15.133:0/3639549478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2214150c20 con 0x7f22141f6330 2024-04-25T10:04:49.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f220b7fe640 1 -- 172.21.15.133:0/3639549478 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c014650 con 0x7f22141f6330 2024-04-25T10:04:49.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.631+0000 7f2219309640 1 -- 172.21.15.133:0/3639549478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2214160c80 con 0x7f22141f6330 2024-04-25T10:04:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f220b7fe640 1 -- 172.21.15.133:0/3639549478 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f220c0147f0 con 0x7f22141f6330 2024-04-25T10:04:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f220b7fe640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21f4066f60 0x7f21f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f220b7fe640 1 -- 172.21.15.133:0/3639549478 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f220c0857a0 con 0x7f22141f6330 2024-04-25T10:04:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f2212ffd640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21f4066f60 0x7f21f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f2212ffd640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21f4066f60 0x7f21f4069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2204000c10 tx=0x7f2204009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f221c476000 1 Processor -- start 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f221c476000 1 -- start start 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f221c476000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026520 0x55740b026900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f221c476000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026e40 0x55740afdd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f221c476000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740afddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f221c476000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740ae2ad00 con 0x55740afdd970 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f221c476000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740adb2df0 con 0x55740b026e40 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f221c476000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55740ae26da0 con 0x55740b026520 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f22137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740afddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f22137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740afddd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47456/0 (socket says 172.21.15.133:47456) 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.635+0000 7f22137fe640 1 -- 172.21.15.133:0/3647877079 learned_addr learned my addr 172.21.15.133:0/3647877079 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2212ffd640 1 --2- 172.21.15.133:0/3647877079 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026520 0x55740b026900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f220bfff640 1 --2- 172.21.15.133:0/3647877079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026e40 0x55740afdd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f22137fe640 1 -- 172.21.15.133:0/3647877079 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026520 msgr2=0x55740b026900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f22137fe640 1 --2- 172.21.15.133:0/3647877079 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026520 0x55740b026900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f22137fe640 1 -- 172.21.15.133:0/3647877079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026e40 msgr2=0x55740afdd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f22137fe640 1 --2- 172.21.15.133:0/3647877079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026e40 0x55740afdd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f22137fe640 1 -- 172.21.15.133:0/3647877079 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55740ade5580 con 0x55740afdd970 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f220bfff640 1 --2- 172.21.15.133:0/3647877079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026e40 0x55740afdd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f22137fe640 1 --2- 172.21.15.133:0/3647877079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740afddd50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f220c05e0e0 tx=0x7f220c055b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e416e94328007277 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2208ff9640 1 -- 172.21.15.133:0/3647877079 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c055f00 con 0x55740afdd970 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2208ff9640 1 -- 172.21.15.133:0/3647877079 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c05a050 con 0x55740afdd970 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2208ff9640 1 -- 172.21.15.133:0/3647877079 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c05e610 con 0x55740afdd970 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 -- 172.21.15.133:0/3647877079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 msgr2=0x55740afddd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 --2- 172.21.15.133:0/3647877079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740afddd50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f220c05e0e0 tx=0x7f220c055b70 comp rx=0 tx=0).stop 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 -- 172.21.15.133:0/3647877079 shutdown_connections 2024-04-25T10:04:49.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 --2- 172.21.15.133:0/3647877079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740afddd50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 --2- 172.21.15.133:0/3647877079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026e40 0x55740afdd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 --2- 172.21.15.133:0/3647877079 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026520 0x55740b026900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 -- 172.21.15.133:0/3647877079 >> 172.21.15.133:0/3647877079 conn(0x55740aec4c10 msgr2=0x55740afdbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 -- 172.21.15.133:0/3647877079 shutdown_connections 2024-04-25T10:04:49.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 -- 172.21.15.133:0/3647877079 wait complete. 2024-04-25T10:04:49.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 -- start start 2024-04-25T10:04:49.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 Processor -- start 2024-04-25T10:04:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b025e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026520 0x55740afe1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026e40 0x55740afe2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740adc36d0 con 0x55740afdd970 2024-04-25T10:04:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740ade5950 con 0x55740b026520 2024-04-25T10:04:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55740ae2a010 con 0x55740b026e40 2024-04-25T10:04:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 10 client.0 _ll_register_callbacks cb 0x55740afc9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b025e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b025e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47466/0 (socket says 172.21.15.133:47466) 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2212ffd640 1 -- 172.21.15.133:0/2013402853 learned_addr learned my addr 172.21.15.133:0/2013402853 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f22137fe640 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026e40 0x55740afe2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f220bfff640 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026520 0x55740afe1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2212ffd640 1 -- 172.21.15.133:0/2013402853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026e40 msgr2=0x55740afe2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2212ffd640 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026e40 0x55740afe2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2212ffd640 1 -- 172.21.15.133:0/2013402853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026520 msgr2=0x55740afe1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2212ffd640 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026520 0x55740afe1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2212ffd640 1 -- 172.21.15.133:0/2013402853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55740adc7d40 con 0x55740afdd970 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f2212ffd640 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b025e70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2204001b60 tx=0x7f22040011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eb6455f2635f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f21f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f21f1ffb640 1 -- 172.21.15.133:0/2013402853 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204028060 con 0x55740afdd970 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f21f1ffb640 1 -- 172.21.15.133:0/2013402853 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2204023c60 con 0x55740afdd970 2024-04-25T10:04:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 -- 172.21.15.133:0/2013402853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f220c05a050 con 0x55740afdd970 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f21f1ffb640 1 -- 172.21.15.133:0/2013402853 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22040215a0 con 0x55740afdd970 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 10 client.5766 fetch_fsmap learned FSMap version 17 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f221c476000 1 -- 172.21.15.133:0/2013402853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55740ae25cf0 con 0x55740afdd970 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.639+0000 7f21f1ffb640 1 -- 172.21.15.133:0/2013402853 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c05a050 con 0x55740afdd970 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f21f1ffb640 1 -- 172.21.15.133:0/2013402853 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2204021740 con 0x55740afdd970 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 10 client.5766 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 10 client.5766 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 20 client.5766 populate_metadata read hostname 'smithi133' 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55740afe30a0 0x55740b037840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 4 client.5766 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 -- 172.21.15.133:0/2013402853 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55740adc69d0 con 0x55740afe30a0 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f220bfff640 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55740afe30a0 0x55740b037840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f21f1ffb640 10 client.5766 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f220bfff640 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55740afe30a0 0x55740b037840 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12767de9f8e48d8 server_cookie=c3132d9b33e4d864 in_seq=0 out_seq=0 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f21f1ffb640 1 -- 172.21.15.133:0/2013402853 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55740adc69d0 con 0x55740afe30a0 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f21f1ffb640 10 client.5766 handle_command_reply: tid=0 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 client.5766 shutdown 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 -- 172.21.15.133:0/2013402853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 msgr2=0x55740b025e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b025e70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2204001b60 tx=0x7f22040011a0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 -- 172.21.15.133:0/2013402853 shutdown_connections 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55740afe30a0 0x55740b037840 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026e40 0x55740afe2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026520 0x55740afe1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f21f1ffb640 0 client.5766 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 --2- 172.21.15.133:0/2013402853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b025e70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 -- 172.21.15.133:0/2013402853 >> 172.21.15.133:0/2013402853 conn(0x55740aec4c10 msgr2=0x55740afdc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 -- 172.21.15.133:0/2013402853 shutdown_connections 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 1 -- 172.21.15.133:0/2013402853 wait complete. 2024-04-25T10:04:49.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.643+0000 7f221c476000 20 client.5766 trim_cache size 0 max 16384 2024-04-25T10:04:49.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 Processor -- start 2024-04-25T10:04:49.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 -- start start 2024-04-25T10:04:49.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b036840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026520 0x55740b036d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026e40 0x55740b0372c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2214203740 con 0x55740afdd970 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2214202900 con 0x55740b026520 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2214201d00 con 0x55740b026e40 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b036840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b036840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47482/0 (socket says 172.21.15.133:47482) 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f2212ffd640 1 -- 172.21.15.133:0/2014884011 learned_addr learned my addr 172.21.15.133:0/2014884011 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f2212ffd640 1 -- 172.21.15.133:0/2014884011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026e40 msgr2=0x55740b0372c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f220bfff640 1 --2- 172.21.15.133:0/2014884011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026520 0x55740b036d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f2212ffd640 1 --2- 172.21.15.133:0/2014884011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026e40 0x55740b0372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f2212ffd640 1 -- 172.21.15.133:0/2014884011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026520 msgr2=0x55740b036d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f2212ffd640 1 --2- 172.21.15.133:0/2014884011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026520 0x55740b036d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f2212ffd640 1 -- 172.21.15.133:0/2014884011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55740b026c70 con 0x55740afdd970 2024-04-25T10:04:49.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f2212ffd640 1 --2- 172.21.15.133:0/2014884011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b036840 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f22040011e0 tx=0x7f2204021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90bedd6b82ae237 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f21f3fff640 1 -- 172.21.15.133:0/2014884011 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204001760 con 0x55740afdd970 2024-04-25T10:04:49.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f21f3fff640 1 -- 172.21.15.133:0/2014884011 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2204014ba0 con 0x55740afdd970 2024-04-25T10:04:49.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f21f3fff640 1 -- 172.21.15.133:0/2014884011 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204021c40 con 0x55740afdd970 2024-04-25T10:04:49.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 -- 172.21.15.133:0/2014884011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 msgr2=0x55740b036840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 --2- 172.21.15.133:0/2014884011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b036840 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f22040011e0 tx=0x7f2204021000 comp rx=0 tx=0).stop 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 -- 172.21.15.133:0/2014884011 shutdown_connections 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 --2- 172.21.15.133:0/2014884011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740b026e40 0x55740b0372c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 --2- 172.21.15.133:0/2014884011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026520 0x55740b036d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 --2- 172.21.15.133:0/2014884011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740afdd970 0x55740b036840 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 -- 172.21.15.133:0/2014884011 >> 172.21.15.133:0/2014884011 conn(0x55740aec4c10 msgr2=0x55740afdec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 -- 172.21.15.133:0/2014884011 shutdown_connections 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 -- 172.21.15.133:0/2014884011 wait complete. 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 -- start start 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.659+0000 7f221c476000 1 Processor -- start 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740afdd970 0x55740b06ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740b026520 0x55740b06b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026e40 0x55740b06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2214202300 con 0x55740b026520 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740aee7a90 con 0x55740b026e40 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55740adef210 con 0x55740afdd970 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 10 client.0 _ll_register_callbacks cb 0x7f220c009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f220bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740b026520 0x55740b06b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f220bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740b026520 0x55740b06b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47496/0 (socket says 172.21.15.133:47496) 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f220bfff640 1 -- 172.21.15.133:0/12063127 learned_addr learned my addr 172.21.15.133:0/12063127 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f220bfff640 1 -- 172.21.15.133:0/12063127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740afdd970 msgr2=0x55740b06ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f220bfff640 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740afdd970 0x55740b06ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f22137fe640 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026e40 0x55740b06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f220bfff640 1 -- 172.21.15.133:0/12063127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026e40 msgr2=0x55740b06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f220bfff640 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026e40 0x55740b06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f220bfff640 1 -- 172.21.15.133:0/12063127 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55740aeadd30 con 0x55740b026520 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f220bfff640 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740b026520 0x55740b06b6b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f21fc01d500 tx=0x7f21fc019a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58367185dbd5c3d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f21f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f21f1ffb640 1 -- 172.21.15.133:0/12063127 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0016c0 con 0x55740b026520 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f21f1ffb640 1 -- 172.21.15.133:0/12063127 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f21fc0ba410 con 0x55740b026520 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 1 -- 172.21.15.133:0/12063127 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2204014ba0 con 0x55740b026520 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f21f1ffb640 1 -- 172.21.15.133:0/12063127 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21fc0ba630 con 0x55740b026520 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 10 client.5772 fetch_fsmap learned FSMap version 17 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 1 -- 172.21.15.133:0/12063127 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55740adf75a0 con 0x55740b026520 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f21f1ffb640 1 -- 172.21.15.133:0/12063127 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2204014ba0 con 0x55740b026520 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f21f1ffb640 1 -- 172.21.15.133:0/12063127 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f21fc0047f0 con 0x55740b026520 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 10 client.5772 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 10 client.5772 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:04:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 20 client.5772 populate_metadata read hostname 'smithi133' 2024-04-25T10:04:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55740b06c130 0x55740b06c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 4 client.5772 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T10:04:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f221c476000 1 -- 172.21.15.133:0/12063127 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x55740afe26c0 con 0x55740b06c130 2024-04-25T10:04:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f2212ffd640 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55740b06c130 0x55740b06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f21f1ffb640 10 client.5772 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.663+0000 7f2212ffd640 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55740b06c130 0x55740b06c510 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ebe09fb220d0393 server_cookie=28e8807fccf7109d in_seq=0 out_seq=0 2024-04-25T10:04:49.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f21f1ffb640 1 -- 172.21.15.133:0/12063127 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55740afe26c0 con 0x55740b06c130 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f21f1ffb640 10 client.5772 handle_command_reply: tid=0 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f221c476000 1 client.5772 shutdown 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f221c476000 1 -- 172.21.15.133:0/12063127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740b026520 msgr2=0x55740b06b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f221c476000 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740b026520 0x55740b06b6b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f21fc01d500 tx=0x7f21fc019a00 comp rx=0 tx=0).stop 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f221c476000 1 -- 172.21.15.133:0/12063127 shutdown_connections 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f221c476000 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55740b06c130 0x55740b06c510 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f221c476000 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55740b026e40 0x55740b06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f221c476000 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55740b026520 0x55740b06b6b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f21f1ffb640 0 client.5772 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f221c476000 1 --2- 172.21.15.133:0/12063127 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55740afdd970 0x55740b06ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f221c476000 1 -- 172.21.15.133:0/12063127 >> 172.21.15.133:0/12063127 conn(0x55740aec4c10 msgr2=0x55740afdf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f221c476000 1 -- 172.21.15.133:0/12063127 shutdown_connections 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.671+0000 7f221c476000 1 -- 172.21.15.133:0/12063127 wait complete. 2024-04-25T10:04:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f221c476000 20 client.5772 trim_cache size 0 max 16384 2024-04-25T10:04:49.682 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:04:49.682 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:04:49.682 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "0aee2ea0-c999-444b-8b79-f68b42f1fb6a", 2024-04-25T10:04:49.682 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:04:49.682 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:04:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 -- 172.21.15.133:0/3639549478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21f4066f60 msgr2=0x7f21f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21f4066f60 0x7f21f4069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2204000c10 tx=0x7f2204009040 comp rx=0 tx=0).stop 2024-04-25T10:04:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 -- 172.21.15.133:0/3639549478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22141f6330 msgr2=0x7f22141568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22141f6330 0x7f22141568e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f220c0049e0 tx=0x7f220c009dc0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 -- 172.21.15.133:0/3639549478 shutdown_connections 2024-04-25T10:04:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21f4066f60 0x7f21f4069420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22141f6330 0x7f22141568e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22141503b0 0x7f2214162470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 --2- 172.21.15.133:0/3639549478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2200004810 0x7f221414fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 -- 172.21.15.133:0/3639549478 >> 172.21.15.133:0/3639549478 conn(0x7f221406d370 msgr2=0x7f22140ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 -- 172.21.15.133:0/3639549478 shutdown_connections 2024-04-25T10:04:49.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.675+0000 7f2219309640 1 -- 172.21.15.133:0/3639549478 wait complete. 2024-04-25T10:04:49.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:49.700 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T10:04:49.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f439c1c5640 1 Processor -- start 2024-04-25T10:04:49.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f439c1c5640 1 -- start start 2024-04-25T10:04:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f439c1c5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43940cc490 0x7f439406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f439c1c5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f43940d0060 0x7f439406b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f439c1c5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439406b710 con 0x7f43940cc0b0 2024-04-25T10:04:49.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f439c1c5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43941623d0 con 0x7f43940d0060 2024-04-25T10:04:49.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f439c1c5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43941617c0 con 0x7f43940cc490 2024-04-25T10:04:49.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f439a762640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f43940d0060 0x7f439406b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f439a762640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f43940d0060 0x7f439406b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:36532/0 (socket says 172.21.15.133:36532) 2024-04-25T10:04:49.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f4399f61640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f43940cc0b0 0x7f43940cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:49876/0 (socket says 172.21.15.133:49876) 2024-04-25T10:04:49.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f439a762640 1 -- 172.21.15.133:0/3111552373 learned_addr learned my addr 172.21.15.133:0/3111552373 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.827+0000 7f4399760640 1 --2- 172.21.15.133:0/3111552373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43940cc490 0x7f439406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.831+0000 7f4398f5f640 1 -- 172.21.15.133:0/3111552373 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444150822 0 0) 0x7f439406b710 con 0x7f43940cc0b0 2024-04-25T10:04:49.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.831+0000 7f4398f5f640 1 -- 172.21.15.133:0/3111552373 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4380003530 con 0x7f43940cc0b0 2024-04-25T10:04:49.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.831+0000 7f4399760640 1 -- 172.21.15.133:0/3111552373 >> v2:172.21.15.162:3300/0 conn(0x7f43940d0060 msgr2=0x7f439406b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.831+0000 7f4399760640 1 --2- 172.21.15.133:0/3111552373 >> v2:172.21.15.162:3300/0 conn(0x7f43940d0060 0x7f439406b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.831+0000 7f4399760640 1 -- 172.21.15.133:0/3111552373 >> v1:172.21.15.162:6789/0 conn(0x7f43940cc0b0 legacy=0x7f43940cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.831+0000 7f4399760640 1 -- 172.21.15.133:0/3111552373 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4394161aa0 con 0x7f43940cc490 2024-04-25T10:04:49.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.831+0000 7f4399760640 1 --2- 172.21.15.133:0/3111552373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43940cc490 0x7f439406ac90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43840097c0 tx=0x7f438401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b53ab0196b286cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f4398f5f640 1 -- 172.21.15.133:0/3111552373 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43840b2070 con 0x7f43940cc490 2024-04-25T10:04:49.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f4398f5f640 1 -- 172.21.15.133:0/3111552373 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43840ad680 con 0x7f43940cc490 2024-04-25T10:04:49.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f4398f5f640 1 -- 172.21.15.133:0/3111552373 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43840ada30 con 0x7f43940cc490 2024-04-25T10:04:49.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 -- 172.21.15.133:0/3111552373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43940cc490 msgr2=0x7f439406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 --2- 172.21.15.133:0/3111552373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43940cc490 0x7f439406ac90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43840097c0 tx=0x7f438401c470 comp rx=0 tx=0).stop 2024-04-25T10:04:49.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 -- 172.21.15.133:0/3111552373 shutdown_connections 2024-04-25T10:04:49.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 --2- 172.21.15.133:0/3111552373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43940cc490 0x7f439406ac90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 --2- 172.21.15.133:0/3111552373 >> v2:172.21.15.162:3300/0 conn(0x7f43940d0060 0x7f439406b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 -- 172.21.15.133:0/3111552373 >> 172.21.15.133:0/3111552373 conn(0x7f43940999d0 msgr2=0x7f43940d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 -- 172.21.15.133:0/3111552373 shutdown_connections 2024-04-25T10:04:49.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 -- 172.21.15.133:0/3111552373 wait complete. 2024-04-25T10:04:49.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 Processor -- start 2024-04-25T10:04:49.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 -- start start 2024-04-25T10:04:49.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43940cc0b0 0x7f43940b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43940cc490 0x7f439414c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43940d0060 0x7f43940ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439415de60 con 0x7f43940cc490 2024-04-25T10:04:49.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439415dce0 con 0x7f43940d0060 2024-04-25T10:04:49.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f439c1c5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439415dfe0 con 0x7f43940cc0b0 2024-04-25T10:04:49.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f4399760640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43940cc490 0x7f439414c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f4399760640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43940cc490 0x7f439414c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47502/0 (socket says 172.21.15.133:47502) 2024-04-25T10:04:49.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f4399760640 1 -- 172.21.15.133:0/1656422829 learned_addr learned my addr 172.21.15.133:0/1656422829 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.835+0000 7f4399f61640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43940cc0b0 0x7f43940b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f439a762640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43940d0060 0x7f43940ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f4399760640 1 -- 172.21.15.133:0/1656422829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43940cc0b0 msgr2=0x7f43940b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f4399760640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43940cc0b0 0x7f43940b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f4399760640 1 -- 172.21.15.133:0/1656422829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43940d0060 msgr2=0x7f43940ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f4399760640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43940d0060 0x7f43940ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f4399760640 1 -- 172.21.15.133:0/1656422829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43940b3e20 con 0x7f43940cc490 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f439a762640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43940d0060 0x7f43940ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f4399760640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43940cc490 0x7f439414c130 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f43840097c0 tx=0x7f4384019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f438a7fc640 1 -- 172.21.15.133:0/1656422829 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43840bd070 con 0x7f43940cc490 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f438a7fc640 1 -- 172.21.15.133:0/1656422829 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4384004230 con 0x7f43940cc490 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f438a7fc640 1 -- 172.21.15.133:0/1656422829 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43840b2040 con 0x7f43940cc490 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f439c1c5640 1 -- 172.21.15.133:0/1656422829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43940b4100 con 0x7f43940cc490 2024-04-25T10:04:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f439c1c5640 1 -- 172.21.15.133:0/1656422829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43940b46c0 con 0x7f43940cc490 2024-04-25T10:04:49.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f438a7fc640 1 -- 172.21.15.133:0/1656422829 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43840ad680 con 0x7f43940cc490 2024-04-25T10:04:49.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f438a7fc640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f60 0x7f4378069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f438a7fc640 1 -- 172.21.15.133:0/1656422829 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f438411e1b0 con 0x7f43940cc490 2024-04-25T10:04:49.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.839+0000 7f4399f61640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f60 0x7f4378069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f4399f61640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f60 0x7f4378069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f439406e730 tx=0x7f4390007d00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 Processor -- start 2024-04-25T10:04:49.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 -- start start 2024-04-25T10:04:49.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836019d0 0x55ad83601db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad836022f0 0x55ad835b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad835b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad83404ff0 con 0x55ad835b80a0 2024-04-25T10:04:49.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8338d150 con 0x55ad836019d0 2024-04-25T10:04:49.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad83401090 con 0x55ad836022f0 2024-04-25T10:04:49.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439a762640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad835b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439a762640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad835b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47512/0 (socket says 172.21.15.133:47512) 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439a762640 1 -- 172.21.15.133:0/1034120769 learned_addr learned my addr 172.21.15.133:0/1034120769 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f4399f61640 1 --2- 172.21.15.133:0/1034120769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836019d0 0x55ad83601db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f4399760640 1 --2- 172.21.15.133:0/1034120769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad836022f0 0x55ad835b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439a762640 1 -- 172.21.15.133:0/1034120769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad836022f0 msgr2=0x55ad835b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439a762640 1 --2- 172.21.15.133:0/1034120769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad836022f0 0x55ad835b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439a762640 1 -- 172.21.15.133:0/1034120769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836019d0 msgr2=0x55ad83601db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439a762640 1 --2- 172.21.15.133:0/1034120769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836019d0 0x55ad83601db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439a762640 1 -- 172.21.15.133:0/1034120769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad8359b0d0 con 0x55ad835b80a0 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439a762640 1 --2- 172.21.15.133:0/1034120769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad835b8480 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f438c009f20 tx=0x7f438c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9dfb322335841eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f43777fe640 1 -- 172.21.15.133:0/1034120769 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c0147e0 con 0x55ad835b80a0 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f43777fe640 1 -- 172.21.15.133:0/1034120769 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438c01a070 con 0x55ad835b80a0 2024-04-25T10:04:49.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f43777fe640 1 -- 172.21.15.133:0/1034120769 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c014b80 con 0x55ad835b80a0 2024-04-25T10:04:49.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 -- 172.21.15.133:0/1034120769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 msgr2=0x55ad835b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 --2- 172.21.15.133:0/1034120769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad835b8480 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f438c009f20 tx=0x7f438c009810 comp rx=0 tx=0).stop 2024-04-25T10:04:49.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 -- 172.21.15.133:0/1034120769 shutdown_connections 2024-04-25T10:04:49.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 --2- 172.21.15.133:0/1034120769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad835b8480 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 --2- 172.21.15.133:0/1034120769 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad836022f0 0x55ad835b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 --2- 172.21.15.133:0/1034120769 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836019d0 0x55ad83601db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 -- 172.21.15.133:0/1034120769 >> 172.21.15.133:0/1034120769 conn(0x55ad835b4020 msgr2=0x55ad835b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 -- 172.21.15.133:0/1034120769 shutdown_connections 2024-04-25T10:04:49.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.843+0000 7f439f332000 1 -- 172.21.15.133:0/1034120769 wait complete. 2024-04-25T10:04:49.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 -- start start 2024-04-25T10:04:49.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 Processor -- start 2024-04-25T10:04:49.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83603930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad836019d0 0x55ad83603e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399f61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83603930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836022f0 0x55ad836043b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad833c9250 con 0x55ad835b80a0 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399f61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83603930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47528/0 (socket says 172.21.15.133:47528) 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 -- 172.21.15.133:0/2872495360 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad833bf690 con 0x55ad836022f0 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399f61640 1 -- 172.21.15.133:0/2872495360 learned_addr learned my addr 172.21.15.133:0/2872495360 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 -- 172.21.15.133:0/2872495360 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad8338da40 con 0x55ad836019d0 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 10 client.0 _ll_register_callbacks cb 0x55ad835a2e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439a762640 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836022f0 0x55ad836043b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399760640 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad836019d0 0x55ad83603e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399f61640 1 -- 172.21.15.133:0/2872495360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad836019d0 msgr2=0x55ad83603e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399f61640 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad836019d0 0x55ad83603e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399f61640 1 -- 172.21.15.133:0/2872495360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836022f0 msgr2=0x55ad836043b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399f61640 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836022f0 0x55ad836043b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399f61640 1 -- 172.21.15.133:0/2872495360 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad834b21c0 con 0x55ad835b80a0 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399760640 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad836019d0 0x55ad83603e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399f61640 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83603930 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4390013670 tx=0x7f4390000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2434103d1ce56f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4374ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4374ff9640 1 -- 172.21.15.133:0/2872495360 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4390020580 con 0x55ad835b80a0 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4374ff9640 1 -- 172.21.15.133:0/2872495360 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4390018960 con 0x55ad835b80a0 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 -- 172.21.15.133:0/2872495360 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f438c01a070 con 0x55ad835b80a0 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4374ff9640 1 -- 172.21.15.133:0/2872495360 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4390018b60 con 0x55ad835b80a0 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 10 client.5781 fetch_fsmap learned FSMap version 17 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 -- 172.21.15.133:0/2872495360 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55ad834ae170 con 0x55ad835b80a0 2024-04-25T10:04:49.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4374ff9640 1 -- 172.21.15.133:0/2872495360 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4390026070 con 0x55ad835b80a0 2024-04-25T10:04:49.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4374ff9640 1 -- 172.21.15.133:0/2872495360 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4390020720 con 0x55ad835b80a0 2024-04-25T10:04:49.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 10 client.5781 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:04:49.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 10 client.5781 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:04:49.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 20 client.5781 populate_metadata read hostname 'smithi133' 2024-04-25T10:04:49.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55ad83604dc0 0x55ad8361c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 4 client.5781 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:49.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f439f332000 1 -- 172.21.15.133:0/2872495360 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f438c01a070 con 0x55ad83604dc0 2024-04-25T10:04:49.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.847+0000 7f4399760640 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55ad83604dc0 0x55ad8361c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f4399760640 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55ad83604dc0 0x55ad8361c780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d91ebcc97da2b5f server_cookie=ce2fd6ec14d24834 in_seq=0 out_seq=0 2024-04-25T10:04:49.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f4374ff9640 10 client.5781 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f4374ff9640 1 -- 172.21.15.133:0/2872495360 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f438c01a070 con 0x55ad83604dc0 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f4374ff9640 10 client.5781 handle_command_reply: tid=0 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 1 client.5781 shutdown 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 1 -- 172.21.15.133:0/2872495360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 msgr2=0x55ad83603930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83603930 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4390013670 tx=0x7f4390000ee0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 1 -- 172.21.15.133:0/2872495360 shutdown_connections 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55ad83604dc0 0x55ad8361c780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836022f0 0x55ad836043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad836019d0 0x55ad83603e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 1 --2- 172.21.15.133:0/2872495360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83603930 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 1 -- 172.21.15.133:0/2872495360 >> 172.21.15.133:0/2872495360 conn(0x55ad835b4020 msgr2=0x55ad835b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f4374ff9640 0 client.5781 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 1 -- 172.21.15.133:0/2872495360 shutdown_connections 2024-04-25T10:04:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 1 -- 172.21.15.133:0/2872495360 wait complete. 2024-04-25T10:04:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.851+0000 7f439f332000 20 client.5781 trim_cache size 0 max 16384 2024-04-25T10:04:49.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f439f332000 1 Processor -- start 2024-04-25T10:04:49.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f439f332000 1 -- start start 2024-04-25T10:04:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f439f332000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad835b4020 0x55ad83606dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f439f332000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83607310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f439f332000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836019d0 0x55ad83607850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f439f332000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43940b89c0 con 0x55ad835b80a0 2024-04-25T10:04:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f439f332000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43940b7b80 con 0x55ad836019d0 2024-04-25T10:04:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f439f332000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43940ad5f0 con 0x55ad835b4020 2024-04-25T10:04:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4399760640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83607310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4399760640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83607310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47544/0 (socket says 172.21.15.133:47544) 2024-04-25T10:04:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4399760640 1 -- 172.21.15.133:0/1518916699 learned_addr learned my addr 172.21.15.133:0/1518916699 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f439a762640 1 --2- 172.21.15.133:0/1518916699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836019d0 0x55ad83607850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4399760640 1 -- 172.21.15.133:0/1518916699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad835b4020 msgr2=0x55ad83606dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4399760640 1 --2- 172.21.15.133:0/1518916699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad835b4020 0x55ad83606dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4399760640 1 -- 172.21.15.133:0/1518916699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836019d0 msgr2=0x55ad83607850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4399760640 1 --2- 172.21.15.133:0/1518916699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836019d0 0x55ad83607850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4399760640 1 -- 172.21.15.133:0/1518916699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad8344bde0 con 0x55ad835b80a0 2024-04-25T10:04:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4399760640 1 --2- 172.21.15.133:0/1518916699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83607310 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4384097e50 tx=0x7f43840f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82aedae2408f3ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4376ffd640 1 -- 172.21.15.133:0/1518916699 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43840f7020 con 0x55ad835b80a0 2024-04-25T10:04:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4376ffd640 1 -- 172.21.15.133:0/1518916699 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43840eca60 con 0x55ad835b80a0 2024-04-25T10:04:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.875+0000 7f4376ffd640 1 -- 172.21.15.133:0/1518916699 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43840eb030 con 0x55ad835b80a0 2024-04-25T10:04:49.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 -- 172.21.15.133:0/1518916699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 msgr2=0x55ad83607310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 --2- 172.21.15.133:0/1518916699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83607310 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4384097e50 tx=0x7f43840f1040 comp rx=0 tx=0).stop 2024-04-25T10:04:49.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 -- 172.21.15.133:0/1518916699 shutdown_connections 2024-04-25T10:04:49.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 --2- 172.21.15.133:0/1518916699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad836019d0 0x55ad83607850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 --2- 172.21.15.133:0/1518916699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad835b80a0 0x55ad83607310 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 --2- 172.21.15.133:0/1518916699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad835b4020 0x55ad83606dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 -- 172.21.15.133:0/1518916699 >> 172.21.15.133:0/1518916699 conn(0x55ad835e47e0 msgr2=0x55ad835b6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 -- 172.21.15.133:0/1518916699 shutdown_connections 2024-04-25T10:04:49.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 -- 172.21.15.133:0/1518916699 wait complete. 2024-04-25T10:04:49.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 -- start start 2024-04-25T10:04:49.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 Processor -- start 2024-04-25T10:04:49.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad835b4020 0x55ad8361ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad835b80a0 0x55ad8361cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad836019d0 0x55ad8361d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad8339fe00 con 0x55ad836019d0 2024-04-25T10:04:49.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43940adbf0 con 0x55ad835b80a0 2024-04-25T10:04:49.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad83404300 con 0x55ad835b4020 2024-04-25T10:04:49.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439f332000 10 client.0 _ll_register_callbacks cb 0x55ad83583040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:49.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439a762640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad836019d0 0x55ad8361d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439a762640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad836019d0 0x55ad8361d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47556/0 (socket says 172.21.15.133:47556) 2024-04-25T10:04:49.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439a762640 1 -- 172.21.15.133:0/3263975216 learned_addr learned my addr 172.21.15.133:0/3263975216 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:04:49.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f4399f61640 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad835b4020 0x55ad8361ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f4399760640 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad835b80a0 0x55ad8361cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439a762640 1 -- 172.21.15.133:0/3263975216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad835b4020 msgr2=0x55ad8361ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439a762640 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad835b4020 0x55ad8361ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439a762640 1 -- 172.21.15.133:0/3263975216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad835b80a0 msgr2=0x55ad8361cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439a762640 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad835b80a0 0x55ad8361cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439a762640 1 -- 172.21.15.133:0/3263975216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad833a68e0 con 0x55ad836019d0 2024-04-25T10:04:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f4399f61640 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad835b4020 0x55ad8361ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.879+0000 7f439a762640 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad836019d0 0x55ad8361d510 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f438c014b60 tx=0x7f438c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a993c329a4367c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4374ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:04:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4374ff9640 1 -- 172.21.15.133:0/3263975216 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c003ba0 con 0x55ad836019d0 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4374ff9640 1 -- 172.21.15.133:0/3263975216 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f438c003d40 con 0x55ad836019d0 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 -- 172.21.15.133:0/3263975216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43840eca60 con 0x55ad836019d0 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4374ff9640 1 -- 172.21.15.133:0/3263975216 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f438c0094d0 con 0x55ad836019d0 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 10 client.5787 fetch_fsmap learned FSMap version 17 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 -- 172.21.15.133:0/3263975216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55ad8345e8f0 con 0x55ad836019d0 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4374ff9640 1 -- 172.21.15.133:0/3263975216 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43840eca60 con 0x55ad836019d0 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4374ff9640 1 -- 172.21.15.133:0/3263975216 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f438c004020 con 0x55ad836019d0 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 10 client.5787 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 10 client.5787 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 20 client.5787 populate_metadata read hostname 'smithi133' 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55ad836022f0 0x55ad8361da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 4 client.5787 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 -- 172.21.15.133:0/3263975216 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ad833a0c60 con 0x55ad836022f0 2024-04-25T10:04:49.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4399f61640 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55ad836022f0 0x55ad8361da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:49.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4374ff9640 10 client.5787 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4399f61640 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55ad836022f0 0x55ad8361da50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50084f9ce325b313 server_cookie=d09de51519e4716 in_seq=0 out_seq=0 2024-04-25T10:04:49.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4374ff9640 1 -- 172.21.15.133:0/3263975216 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+274 (crc 0 0 0) 0x55ad833a0c60 con 0x55ad836022f0 2024-04-25T10:04:49.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4374ff9640 10 client.5787 handle_command_reply: tid=0 2024-04-25T10:04:49.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 client.5787 shutdown 2024-04-25T10:04:49.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 -- 172.21.15.133:0/3263975216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad836019d0 msgr2=0x55ad8361d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad836019d0 0x55ad8361d510 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f438c014b60 tx=0x7f438c012000 comp rx=0 tx=0).stop 2024-04-25T10:04:49.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 -- 172.21.15.133:0/3263975216 shutdown_connections 2024-04-25T10:04:49.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55ad836022f0 0x55ad8361da50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad836019d0 0x55ad8361d510 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ad835b80a0 0x55ad8361cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 --2- 172.21.15.133:0/3263975216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ad835b4020 0x55ad8361ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f4374ff9640 0 client.5787 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:04:49.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 -- 172.21.15.133:0/3263975216 >> 172.21.15.133:0/3263975216 conn(0x55ad835e47e0 msgr2=0x55ad835b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 -- 172.21.15.133:0/3263975216 shutdown_connections 2024-04-25T10:04:49.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 1 -- 172.21.15.133:0/3263975216 wait complete. 2024-04-25T10:04:49.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439f332000 20 client.5787 trim_cache size 0 max 16384 2024-04-25T10:04:49.898 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:04:49.898 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (969 inodes in the stack)", 2024-04-25T10:04:49.898 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-04-25T10:04:49.899 INFO:teuthology.orchestra.run.smithi133.stdout: "0aee2ea0-c999-444b-8b79-f68b42f1fb6a": { 2024-04-25T10:04:49.899 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-04-25T10:04:49.899 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "0aee2ea0-c999-444b-8b79-f68b42f1fb6a", 2024-04-25T10:04:49.899 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T10:04:49.899 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:04:49.899 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:04:49.899 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:04:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 -- 172.21.15.133:0/1656422829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f60 msgr2=0x7f4378069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f60 0x7f4378069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f439406e730 tx=0x7f4390007d00 comp rx=0 tx=0).stop 2024-04-25T10:04:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 -- 172.21.15.133:0/1656422829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43940cc490 msgr2=0x7f439414c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43940cc490 0x7f439414c130 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f43840097c0 tx=0x7f4384019210 comp rx=0 tx=0).stop 2024-04-25T10:04:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 -- 172.21.15.133:0/1656422829 shutdown_connections 2024-04-25T10:04:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43940d0060 0x7f43940ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f60 0x7f4378069420 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43940cc490 0x7f439414c130 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 --2- 172.21.15.133:0/1656422829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43940cc0b0 0x7f43940b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 -- 172.21.15.133:0/1656422829 >> 172.21.15.133:0/1656422829 conn(0x7f43940999d0 msgr2=0x7f43940b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:49.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 -- 172.21.15.133:0/1656422829 shutdown_connections 2024-04-25T10:04:49.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:04:49.883+0000 7f439c1c5640 1 -- 172.21.15.133:0/1656422829 wait complete. 2024-04-25T10:04:49.914 INFO:tasks.cephfs.filesystem:scrub status for tag:0aee2ea0-c999-444b-8b79-f68b42f1fb6a - {'path': '/', 'tag': '0aee2ea0-c999-444b-8b79-f68b42f1fb6a', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T10:04:50.654 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:50.657 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:19.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:19.915 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T10:05:20.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.038+0000 7f41e6ab6640 1 Processor -- start 2024-04-25T10:05:20.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.038+0000 7f41e6ab6640 1 -- start start 2024-04-25T10:05:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e6ab6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f41e0142920 0x7f41e0143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e6ab6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41e0140a40 0x7f41e00ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e6ab6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e00cf000 con 0x7f41e015d9f0 2024-04-25T10:05:20.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e6ab6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e0140ea0 con 0x7f41e0142920 2024-04-25T10:05:20.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e6ab6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00cf280 con 0x7f41e0140a40 2024-04-25T10:05:20.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e4852640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f41e0142920 0x7f41e0143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e4852640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f41e0142920 0x7f41e0143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55368/0 (socket says 172.21.15.133:55368) 2024-04-25T10:05:20.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e4852640 1 -- 172.21.15.133:0/2544185774 learned_addr learned my addr 172.21.15.133:0/2544185774 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41d7fff640 1 --2- 172.21.15.133:0/2544185774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41e0140a40 0x7f41e00ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e4852640 1 -- 172.21.15.133:0/2544185774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41e0140a40 msgr2=0x7f41e00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e4852640 1 --2- 172.21.15.133:0/2544185774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41e0140a40 0x7f41e00ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e4852640 1 -- 172.21.15.133:0/2544185774 >> v1:172.21.15.162:6789/0 conn(0x7f41e015d9f0 legacy=0x7f41e006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e4852640 1 -- 172.21.15.133:0/2544185774 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e006d440 con 0x7f41e0142920 2024-04-25T10:05:20.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41e4852640 1 --2- 172.21.15.133:0/2544185774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e0142920 0x7f41e0143560 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009870 tx=0x7f41dc01c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9593de8ee7a3e74f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41d77fe640 1 -- 172.21.15.133:0/2544185774 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0b0070 con 0x7f41e0142920 2024-04-25T10:05:20.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41d77fe640 1 -- 172.21.15.133:0/2544185774 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc0ab600 con 0x7f41e0142920 2024-04-25T10:05:20.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.042+0000 7f41d77fe640 1 -- 172.21.15.133:0/2544185774 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0ab9d0 con 0x7f41e0142920 2024-04-25T10:05:20.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 -- 172.21.15.133:0/2544185774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e0142920 msgr2=0x7f41e0143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 --2- 172.21.15.133:0/2544185774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e0142920 0x7f41e0143560 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009870 tx=0x7f41dc01c2b0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 -- 172.21.15.133:0/2544185774 shutdown_connections 2024-04-25T10:05:20.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 --2- 172.21.15.133:0/2544185774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41e0140a40 0x7f41e00ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 --2- 172.21.15.133:0/2544185774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e0142920 0x7f41e0143560 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 -- 172.21.15.133:0/2544185774 >> 172.21.15.133:0/2544185774 conn(0x7f41e0066f70 msgr2=0x7f41e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 -- 172.21.15.133:0/2544185774 shutdown_connections 2024-04-25T10:05:20.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 -- 172.21.15.133:0/2544185774 wait complete. 2024-04-25T10:05:20.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 Processor -- start 2024-04-25T10:05:20.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 -- start start 2024-04-25T10:05:20.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e0140a40 0x7f41e0159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41e0142920 0x7f41e015cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41e015d9f0 0x7f41e015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e006ebc0 con 0x7f41e0140a40 2024-04-25T10:05:20.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e006ea40 con 0x7f41e015d9f0 2024-04-25T10:05:20.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e6ab6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0142d00 con 0x7f41e0142920 2024-04-25T10:05:20.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e4852640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e0140a40 0x7f41e0159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e4852640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e0140a40 0x7f41e0159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55370/0 (socket says 172.21.15.133:55370) 2024-04-25T10:05:20.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e4852640 1 -- 172.21.15.133:0/473464366 learned_addr learned my addr 172.21.15.133:0/473464366 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41d7fff640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41e0142920 0x7f41e015cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e5053640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41e015d9f0 0x7f41e015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e4852640 1 -- 172.21.15.133:0/473464366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41e0142920 msgr2=0x7f41e015cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e4852640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41e0142920 0x7f41e015cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e4852640 1 -- 172.21.15.133:0/473464366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41e015d9f0 msgr2=0x7f41e015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e4852640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41e015d9f0 0x7f41e015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e4852640 1 -- 172.21.15.133:0/473464366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e0159c70 con 0x7f41e0140a40 2024-04-25T10:05:20.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.046+0000 7f41e4852640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e0140a40 0x7f41e0159730 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0ae040 tx=0x7f41dc01aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.050+0000 7f41d57fa640 1 -- 172.21.15.133:0/473464366 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0ba020 con 0x7f41e0140a40 2024-04-25T10:05:20.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.050+0000 7f41d57fa640 1 -- 172.21.15.133:0/473464366 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc0b6620 con 0x7f41e0140a40 2024-04-25T10:05:20.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.050+0000 7f41e6ab6640 1 -- 172.21.15.133:0/473464366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e00b1700 con 0x7f41e0140a40 2024-04-25T10:05:20.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.050+0000 7f41d57fa640 1 -- 172.21.15.133:0/473464366 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0b0040 con 0x7f41e0140a40 2024-04-25T10:05:20.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.050+0000 7f41e6ab6640 1 -- 172.21.15.133:0/473464366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e00b1cc0 con 0x7f41e0140a40 2024-04-25T10:05:20.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.050+0000 7f41d57fa640 1 -- 172.21.15.133:0/473464366 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f41dc0bd020 con 0x7f41e0140a40 2024-04-25T10:05:20.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.050+0000 7f41d57fa640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f41cc066f60 0x7f41cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.050+0000 7f41d57fa640 1 -- 172.21.15.133:0/473464366 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f41dc11f400 con 0x7f41e0140a40 2024-04-25T10:05:20.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.050+0000 7f41d7fff640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f41cc066f60 0x7f41cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.050+0000 7f41d7fff640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f41cc066f60 0x7f41cc069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f41d000b930 tx=0x7f41d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 Processor -- start 2024-04-25T10:05:20.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 -- start start 2024-04-25T10:05:20.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d909d0 0x55b8a0d90db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d912f0 0x55b8a0d46b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d47480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a0b93ff0 con 0x55b8a0d470a0 2024-04-25T10:05:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a0b1c150 con 0x55b8a0d912f0 2024-04-25T10:05:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8a0b90090 con 0x55b8a0d909d0 2024-04-25T10:05:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e5053640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d47480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e5053640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d47480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55386/0 (socket says 172.21.15.133:55386) 2024-04-25T10:05:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e5053640 1 -- 172.21.15.133:0/2291454148 learned_addr learned my addr 172.21.15.133:0/2291454148 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41d7fff640 1 --2- 172.21.15.133:0/2291454148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d912f0 0x55b8a0d46b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e4852640 1 --2- 172.21.15.133:0/2291454148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d909d0 0x55b8a0d90db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e5053640 1 -- 172.21.15.133:0/2291454148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d909d0 msgr2=0x55b8a0d90db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e5053640 1 --2- 172.21.15.133:0/2291454148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d909d0 0x55b8a0d90db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e5053640 1 -- 172.21.15.133:0/2291454148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d912f0 msgr2=0x55b8a0d46b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e5053640 1 --2- 172.21.15.133:0/2291454148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d912f0 0x55b8a0d46b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e5053640 1 -- 172.21.15.133:0/2291454148 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8a0d2a0d0 con 0x55b8a0d470a0 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41d7fff640 1 --2- 172.21.15.133:0/2291454148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d912f0 0x55b8a0d46b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e5053640 1 --2- 172.21.15.133:0/2291454148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d47480 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009f00 tx=0x7f41d800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f54dbc09afbbafe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41c67fc640 1 -- 172.21.15.133:0/2291454148 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d801a070 con 0x55b8a0d470a0 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41c67fc640 1 -- 172.21.15.133:0/2291454148 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d800a5f0 con 0x55b8a0d470a0 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41c67fc640 1 -- 172.21.15.133:0/2291454148 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d800a9c0 con 0x55b8a0d470a0 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 -- 172.21.15.133:0/2291454148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 msgr2=0x55b8a0d47480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 --2- 172.21.15.133:0/2291454148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d47480 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009f00 tx=0x7f41d800c7d0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 -- 172.21.15.133:0/2291454148 shutdown_connections 2024-04-25T10:05:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 --2- 172.21.15.133:0/2291454148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d47480 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 --2- 172.21.15.133:0/2291454148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d912f0 0x55b8a0d46b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 --2- 172.21.15.133:0/2291454148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d909d0 0x55b8a0d90db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 -- 172.21.15.133:0/2291454148 >> 172.21.15.133:0/2291454148 conn(0x55b8a0d43020 msgr2=0x55b8a0d45440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 -- 172.21.15.133:0/2291454148 shutdown_connections 2024-04-25T10:05:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 -- 172.21.15.133:0/2291454148 wait complete. 2024-04-25T10:05:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 -- start start 2024-04-25T10:05:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.054+0000 7f41e9c23000 1 Processor -- start 2024-04-25T10:05:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d470a0 0x55b8a0d757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d909d0 0x55b8a0d75d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d912f0 0x55b8a0d76270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a0b58250 con 0x55b8a0d912f0 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a0b4e690 con 0x55b8a0d470a0 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8a0b1ca40 con 0x55b8a0d909d0 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 10 client.0 _ll_register_callbacks cb 0x55b8a0d31e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e5053640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d912f0 0x55b8a0d76270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e5053640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d912f0 0x55b8a0d76270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55394/0 (socket says 172.21.15.133:55394) 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e5053640 1 -- 172.21.15.133:0/3136626473 learned_addr learned my addr 172.21.15.133:0/3136626473 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e4852640 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d470a0 0x55b8a0d757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41d7fff640 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d909d0 0x55b8a0d75d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e5053640 1 -- 172.21.15.133:0/3136626473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d909d0 msgr2=0x55b8a0d75d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e5053640 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d909d0 0x55b8a0d75d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e5053640 1 -- 172.21.15.133:0/3136626473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d470a0 msgr2=0x55b8a0d757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e5053640 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d470a0 0x55b8a0d757f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e5053640 1 -- 172.21.15.133:0/3136626473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8a0c411c0 con 0x55b8a0d912f0 2024-04-25T10:05:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e5053640 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d912f0 0x55b8a0d76270 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f41d800c810 tx=0x7f41d8004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18548f1e4dae54ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3136626473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d8025020 con 0x55b8a0d912f0 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3136626473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f41d8020670 con 0x55b8a0d912f0 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 -- 172.21.15.133:0/3136626473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41d800a5f0 con 0x55b8a0d912f0 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3136626473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41d802f050 con 0x55b8a0d912f0 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 10 client.5799 fetch_fsmap learned FSMap version 17 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 -- 172.21.15.133:0/3136626473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55b8a0c3d170 con 0x55b8a0d912f0 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3136626473 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d800a5f0 con 0x55b8a0d912f0 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3136626473 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f41d8032020 con 0x55b8a0d912f0 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 10 client.5799 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 10 client.5799 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 20 client.5799 populate_metadata read hostname 'smithi133' 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b8a0da0c80 0x55b8a0da3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 4 client.5799 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 -- 172.21.15.133:0/3136626473 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8a0b2fc60 con 0x55b8a0da0c80 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e4852640 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b8a0da0c80 0x55b8a0da3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41bf7fe640 10 client.5799 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e4852640 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b8a0da0c80 0x55b8a0da3070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11217b9f15f06c2f server_cookie=8993d233694d760d in_seq=0 out_seq=0 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3136626473 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8a0b2fc60 con 0x55b8a0da0c80 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41bf7fe640 10 client.5799 handle_command_reply: tid=0 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 client.5799 shutdown 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 -- 172.21.15.133:0/3136626473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d912f0 msgr2=0x55b8a0d76270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d912f0 0x55b8a0d76270 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f41d800c810 tx=0x7f41d8004a10 comp rx=0 tx=0).stop 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 -- 172.21.15.133:0/3136626473 shutdown_connections 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b8a0da0c80 0x55b8a0da3070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d912f0 0x55b8a0d76270 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d909d0 0x55b8a0d75d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41bf7fe640 0 client.5799 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:05:20.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3136626473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d470a0 0x55b8a0d757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 -- 172.21.15.133:0/3136626473 >> 172.21.15.133:0/3136626473 conn(0x55b8a0d43020 msgr2=0x55b8a0d45ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 -- 172.21.15.133:0/3136626473 shutdown_connections 2024-04-25T10:05:20.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 1 -- 172.21.15.133:0/3136626473 wait complete. 2024-04-25T10:05:20.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.058+0000 7f41e9c23000 20 client.5799 trim_cache size 0 max 16384 2024-04-25T10:05:20.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e9c23000 1 Processor -- start 2024-04-25T10:05:20.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e9c23000 1 -- start start 2024-04-25T10:05:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d8fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d909d0 0x55b8a0d46420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d912f0 0x55b8a0d46960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e9c23000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0203950 con 0x55b8a0d470a0 2024-04-25T10:05:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e9c23000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0202b30 con 0x55b8a0d909d0 2024-04-25T10:05:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e9c23000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e013bf10 con 0x55b8a0d912f0 2024-04-25T10:05:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e4852640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d8fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e4852640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d8fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55402/0 (socket says 172.21.15.133:55402) 2024-04-25T10:05:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e4852640 1 -- 172.21.15.133:0/463393209 learned_addr learned my addr 172.21.15.133:0/463393209 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e4852640 1 -- 172.21.15.133:0/463393209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d912f0 msgr2=0x55b8a0d46960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e4852640 1 --2- 172.21.15.133:0/463393209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d912f0 0x55b8a0d46960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e4852640 1 -- 172.21.15.133:0/463393209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d909d0 msgr2=0x55b8a0d46420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e4852640 1 --2- 172.21.15.133:0/463393209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d909d0 0x55b8a0d46420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e4852640 1 -- 172.21.15.133:0/463393209 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8a0bdade0 con 0x55b8a0d470a0 2024-04-25T10:05:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41e4852640 1 --2- 172.21.15.133:0/463393209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d8fe10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f41dc01c790 tx=0x7f41dc11d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0eba57a59ce99a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41c5ffb640 1 -- 172.21.15.133:0/463393209 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0efbe0 con 0x55b8a0d470a0 2024-04-25T10:05:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41c5ffb640 1 -- 172.21.15.133:0/463393209 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc0f3d30 con 0x55b8a0d470a0 2024-04-25T10:05:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.074+0000 7f41c5ffb640 1 -- 172.21.15.133:0/463393209 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0f82f0 con 0x55b8a0d470a0 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/463393209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 msgr2=0x55b8a0d8fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- 172.21.15.133:0/463393209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d8fe10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f41dc01c790 tx=0x7f41dc11d6d0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/463393209 shutdown_connections 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- 172.21.15.133:0/463393209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d912f0 0x55b8a0d46960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- 172.21.15.133:0/463393209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d909d0 0x55b8a0d46420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- 172.21.15.133:0/463393209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d470a0 0x55b8a0d8fe10 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/463393209 >> 172.21.15.133:0/463393209 conn(0x55b8a0d43020 msgr2=0x55b8a0d8f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/463393209 shutdown_connections 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/463393209 wait complete. 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- start start 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 Processor -- start 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d470a0 0x55b8a0df9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d909d0 0x55b8a0dfa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d912f0 0x55b8a0da3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a0b2ee00 con 0x55b8a0d909d0 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e013c510 con 0x55b8a0d912f0 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b8a0b93300 con 0x55b8a0d470a0 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 10 client.0 _ll_register_callbacks cb 0x55b8a0d12040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d909d0 0x55b8a0dfa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41d7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d909d0 0x55b8a0dfa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55414/0 (socket says 172.21.15.133:55414) 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41d7fff640 1 -- 172.21.15.133:0/3174463441 learned_addr learned my addr 172.21.15.133:0/3174463441 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41d7fff640 1 -- 172.21.15.133:0/3174463441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d470a0 msgr2=0x55b8a0df9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41d7fff640 1 --2- 172.21.15.133:0/3174463441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d470a0 0x55b8a0df9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41d7fff640 1 -- 172.21.15.133:0/3174463441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d912f0 msgr2=0x55b8a0da3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41d7fff640 1 --2- 172.21.15.133:0/3174463441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d912f0 0x55b8a0da3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41d7fff640 1 -- 172.21.15.133:0/3174463441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8a0b358e0 con 0x55b8a0d909d0 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41d7fff640 1 --2- 172.21.15.133:0/3174463441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d909d0 0x55b8a0dfa060 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f41d0010010 tx=0x7f41d00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bce6861a94e637 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3174463441 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0015820 con 0x55b8a0d909d0 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3174463441 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f41d00159c0 con 0x55b8a0d909d0 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/3174463441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41dc0f3d30 con 0x55b8a0d909d0 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3174463441 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41d002e050 con 0x55b8a0d909d0 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 10 client.5805 fetch_fsmap learned FSMap version 17 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/3174463441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55b8a0bed8f0 con 0x55b8a0d909d0 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3174463441 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc0f3d30 con 0x55b8a0d909d0 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3174463441 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f41d0031020 con 0x55b8a0d909d0 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 10 client.5805 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 10 client.5805 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 20 client.5805 populate_metadata read hostname 'smithi133' 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3174463441 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b8a0da3ff0 0x55b8a0da4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 4 client.5805 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/3174463441 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b8a0ba1530 con 0x55b8a0da3ff0 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e4852640 1 --2- 172.21.15.133:0/3174463441 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b8a0da3ff0 0x55b8a0da4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e4852640 1 --2- 172.21.15.133:0/3174463441 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b8a0da3ff0 0x55b8a0da4400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7f2e2cbd66ca07 server_cookie=6a29086c2f2d685b in_seq=0 out_seq=0 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41bf7fe640 10 client.5805 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41bf7fe640 1 -- 172.21.15.133:0/3174463441 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b8a0ba1530 con 0x55b8a0da3ff0 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41bf7fe640 10 client.5805 handle_command_reply: tid=0 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 client.5805 shutdown 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/3174463441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d909d0 msgr2=0x55b8a0dfa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3174463441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d909d0 0x55b8a0dfa060 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f41d0010010 tx=0x7f41d00046d0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/3174463441 shutdown_connections 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3174463441 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b8a0da3ff0 0x55b8a0da4400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3174463441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b8a0d912f0 0x55b8a0da3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3174463441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8a0d909d0 0x55b8a0dfa060 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 --2- 172.21.15.133:0/3174463441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b8a0d470a0 0x55b8a0df9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/3174463441 >> 172.21.15.133:0/3174463441 conn(0x55b8a0d43020 msgr2=0x55b8a0d8f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41bf7fe640 0 client.5805 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/3174463441 shutdown_connections 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 1 -- 172.21.15.133:0/3174463441 wait complete. 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.078+0000 7f41e9c23000 20 client.5805 trim_cache size 0 max 16384 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-04-25T10:05:20.095 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:05:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 -- 172.21.15.133:0/473464366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f41cc066f60 msgr2=0x7f41cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f41cc066f60 0x7f41cc069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f41d000b930 tx=0x7f41d0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 -- 172.21.15.133:0/473464366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e0140a40 msgr2=0x7f41e0159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e0140a40 0x7f41e0159730 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0ae040 tx=0x7f41dc01aaf0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 -- 172.21.15.133:0/473464366 shutdown_connections 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41e015d9f0 0x7f41e015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41e0142920 0x7f41e015cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f41cc066f60 0x7f41cc069420 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 --2- 172.21.15.133:0/473464366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e0140a40 0x7f41e0159730 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 -- 172.21.15.133:0/473464366 >> 172.21.15.133:0/473464366 conn(0x7f41e0066f70 msgr2=0x7f41e00ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 -- 172.21.15.133:0/473464366 shutdown_connections 2024-04-25T10:05:20.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.082+0000 7f41e6ab6640 1 -- 172.21.15.133:0/473464366 wait complete. 2024-04-25T10:05:20.105 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:05:20.105 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:05:20.105 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.214+0000 7f79275da640 1 Processor -- start 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.214+0000 7f79275da640 1 -- start start 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.214+0000 7f79275da640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f792015d9a0 0x7f79200d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.214+0000 7f79275da640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79200d2030 0x7f79200cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.214+0000 7f79275da640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79200d24c0 con 0x7f79200ce600 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.214+0000 7f79275da640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79200cefa0 con 0x7f79200d2030 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.214+0000 7f79275da640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792015ebd0 con 0x7f792015d9a0 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7925b77640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79200d2030 0x7f79200cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7925b77640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79200d2030 0x7f79200cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55428/0 (socket says 172.21.15.133:55428) 2024-04-25T10:05:20.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7925b77640 1 -- 172.21.15.133:0/864009136 learned_addr learned my addr 172.21.15.133:0/864009136 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7924b75640 1 --1- 172.21.15.133:0/864009136 >> v1:172.21.15.133:6789/0 conn(0x7f79200ce600 0x7f79200d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49756/0 (socket says 172.21.15.133:49756) 2024-04-25T10:05:20.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7925376640 1 --2- 172.21.15.133:0/864009136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f792015d9a0 0x7f79200d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7925b77640 1 -- 172.21.15.133:0/864009136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f792015d9a0 msgr2=0x7f79200d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7925b77640 1 --2- 172.21.15.133:0/864009136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f792015d9a0 0x7f79200d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7925b77640 1 -- 172.21.15.133:0/864009136 >> v1:172.21.15.133:6789/0 conn(0x7f79200ce600 legacy=0x7f79200d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7925b77640 1 -- 172.21.15.133:0/864009136 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792015eeb0 con 0x7f79200d2030 2024-04-25T10:05:20.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7925b77640 1 --2- 172.21.15.133:0/864009136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79200d2030 0x7f79200cea60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7918009e50 tx=0x7f791801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4e1c93c7346de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7917fff640 1 -- 172.21.15.133:0/864009136 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79180a5830 con 0x7f79200d2030 2024-04-25T10:05:20.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7917fff640 1 -- 172.21.15.133:0/864009136 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79180a59d0 con 0x7f79200d2030 2024-04-25T10:05:20.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f7917fff640 1 -- 172.21.15.133:0/864009136 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79180a5da0 con 0x7f79200d2030 2024-04-25T10:05:20.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f79275da640 1 -- 172.21.15.133:0/864009136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79200d2030 msgr2=0x7f79200cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f79275da640 1 --2- 172.21.15.133:0/864009136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79200d2030 0x7f79200cea60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7918009e50 tx=0x7f791801c970 comp rx=0 tx=0).stop 2024-04-25T10:05:20.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f79275da640 1 -- 172.21.15.133:0/864009136 shutdown_connections 2024-04-25T10:05:20.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f79275da640 1 --2- 172.21.15.133:0/864009136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79200d2030 0x7f79200cea60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f79275da640 1 --2- 172.21.15.133:0/864009136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f792015d9a0 0x7f79200d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f79275da640 1 -- 172.21.15.133:0/864009136 >> 172.21.15.133:0/864009136 conn(0x7f792006bf80 msgr2=0x7f792006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f79275da640 1 -- 172.21.15.133:0/864009136 shutdown_connections 2024-04-25T10:05:20.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.218+0000 7f79275da640 1 -- 172.21.15.133:0/864009136 wait complete. 2024-04-25T10:05:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f79275da640 1 Processor -- start 2024-04-25T10:05:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f79275da640 1 -- start start 2024-04-25T10:05:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f79275da640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79200ce600 0x7f792015cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f79275da640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79200d2030 0x7f79200c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f79275da640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f792015d9a0 0x7f792015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f79275da640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79200b96a0 con 0x7f79200ce600 2024-04-25T10:05:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f79275da640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79200cffd0 con 0x7f792015d9a0 2024-04-25T10:05:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f79275da640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79200b9820 con 0x7f79200d2030 2024-04-25T10:05:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7925376640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79200ce600 0x7f792015cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7925376640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79200ce600 0x7f792015cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55444/0 (socket says 172.21.15.133:55444) 2024-04-25T10:05:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7925376640 1 -- 172.21.15.133:0/2261515780 learned_addr learned my addr 172.21.15.133:0/2261515780 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7924b75640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79200d2030 0x7f79200c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7925b77640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f792015d9a0 0x7f792015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7925376640 1 -- 172.21.15.133:0/2261515780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79200d2030 msgr2=0x7f79200c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7925376640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79200d2030 0x7f79200c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7925376640 1 -- 172.21.15.133:0/2261515780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f792015d9a0 msgr2=0x7f792015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7925376640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f792015d9a0 0x7f792015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7925376640 1 -- 172.21.15.133:0/2261515780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200c7dd0 con 0x7f79200ce600 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7925376640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79200ce600 0x7f792015cbb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f791c00d620 tx=0x7f791c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7915ffb640 1 -- 172.21.15.133:0/2261515780 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c00b620 con 0x7f79200ce600 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7915ffb640 1 -- 172.21.15.133:0/2261515780 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c019070 con 0x7f79200ce600 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f79275da640 1 -- 172.21.15.133:0/2261515780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79200c9840 con 0x7f79200ce600 2024-04-25T10:05:20.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f7915ffb640 1 -- 172.21.15.133:0/2261515780 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c00b9e0 con 0x7f79200ce600 2024-04-25T10:05:20.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.222+0000 7f79275da640 1 -- 172.21.15.133:0/2261515780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79200c8070 con 0x7f79200ce600 2024-04-25T10:05:20.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7915ffb640 1 -- 172.21.15.133:0/2261515780 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f791c00bb80 con 0x7f79200ce600 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7915ffb640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7904066f60 0x7f7904069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7915ffb640 1 -- 172.21.15.133:0/2261515780 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f791c085600 con 0x7f79200ce600 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7924b75640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7904066f60 0x7f7904069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7924b75640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7904066f60 0x7f7904069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f79200cb6c0 tx=0x7f7910007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 Processor -- start 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 -- start start 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fcdfa40 0x55b24fcdfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fce0360 0x55b24fc9a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fc9ad90 0x55b24fc9b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b24fb7d1f0 con 0x55b24fcdfa40 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b24fae3e90 con 0x55b24fc9ad90 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b24fa6bf50 con 0x55b24fce0360 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7925376640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fcdfa40 0x55b24fcdfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7925376640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fcdfa40 0x55b24fcdfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55448/0 (socket says 172.21.15.133:55448) 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7925376640 1 -- 172.21.15.133:0/4014850853 learned_addr learned my addr 172.21.15.133:0/4014850853 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7924b75640 1 --2- 172.21.15.133:0/4014850853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fce0360 0x55b24fc9a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7925b77640 1 --2- 172.21.15.133:0/4014850853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fc9ad90 0x55b24fc9b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7925376640 1 -- 172.21.15.133:0/4014850853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fce0360 msgr2=0x55b24fc9a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7925376640 1 --2- 172.21.15.133:0/4014850853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fce0360 0x55b24fc9a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7925376640 1 -- 172.21.15.133:0/4014850853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fc9ad90 msgr2=0x55b24fc9b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7925376640 1 --2- 172.21.15.133:0/4014850853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fc9ad90 0x55b24fc9b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7925376640 1 -- 172.21.15.133:0/4014850853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b24fabb8b0 con 0x55b24fcdfa40 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7924b75640 1 --2- 172.21.15.133:0/4014850853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fce0360 0x55b24fc9a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7925b77640 1 --2- 172.21.15.133:0/4014850853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fc9ad90 0x55b24fc9b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f7925376640 1 --2- 172.21.15.133:0/4014850853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fcdfa40 0x55b24fcdfe20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f791c00b0e0 tx=0x7f791c007440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5223e73a9db52143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f790affd640 1 -- 172.21.15.133:0/4014850853 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c059ea0 con 0x55b24fcdfa40 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f790affd640 1 -- 172.21.15.133:0/4014850853 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c05e460 con 0x55b24fcdfa40 2024-04-25T10:05:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f790affd640 1 -- 172.21.15.133:0/4014850853 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c054050 con 0x55b24fcdfa40 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 -- 172.21.15.133:0/4014850853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fcdfa40 msgr2=0x55b24fcdfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 --2- 172.21.15.133:0/4014850853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fcdfa40 0x55b24fcdfe20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f791c00b0e0 tx=0x7f791c007440 comp rx=0 tx=0).stop 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 -- 172.21.15.133:0/4014850853 shutdown_connections 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 --2- 172.21.15.133:0/4014850853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fc9ad90 0x55b24fc9b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 --2- 172.21.15.133:0/4014850853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fce0360 0x55b24fc9a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 --2- 172.21.15.133:0/4014850853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fcdfa40 0x55b24fcdfe20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 -- 172.21.15.133:0/4014850853 >> 172.21.15.133:0/4014850853 conn(0x55b24fc96d40 msgr2=0x55b24fc99130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 -- 172.21.15.133:0/4014850853 shutdown_connections 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.226+0000 7f792a747000 1 -- 172.21.15.133:0/4014850853 wait complete. 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 -- start start 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 Processor -- start 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fc9ad90 0x55b24fce71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 0x55b24fce7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fce0360 0x55b24fca0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b24fa9e540 con 0x55b24fce0360 2024-04-25T10:05:20.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b24fadff30 con 0x55b24fc9ad90 2024-04-25T10:05:20.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b24fa8f260 con 0x55b24fcdfa40 2024-04-25T10:05:20.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 10 client.0 _ll_register_callbacks cb 0x55b24fb98180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:20.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925b77640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fce0360 0x55b24fca0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925b77640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fce0360 0x55b24fca0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55450/0 (socket says 172.21.15.133:55450) 2024-04-25T10:05:20.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925b77640 1 -- 172.21.15.133:0/122045215 learned_addr learned my addr 172.21.15.133:0/122045215 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925376640 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fc9ad90 0x55b24fce71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7924b75640 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 0x55b24fce7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925b77640 1 -- 172.21.15.133:0/122045215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 msgr2=0x55b24fce7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925b77640 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 0x55b24fce7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925b77640 1 -- 172.21.15.133:0/122045215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fc9ad90 msgr2=0x55b24fce71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925b77640 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fc9ad90 0x55b24fce71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925b77640 1 -- 172.21.15.133:0/122045215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b24fa6c1a0 con 0x55b24fce0360 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7924b75640 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 0x55b24fce7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925b77640 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fce0360 0x55b24fca0e80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f791801b020 tx=0x7f791801b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548b3ea5315029a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f78fffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f78fffff640 1 -- 172.21.15.133:0/122045215 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79180a5bf0 con 0x55b24fce0360 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f78fffff640 1 -- 172.21.15.133:0/122045215 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f79180a5d90 con 0x55b24fce0360 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 -- 172.21.15.133:0/122045215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f791c05e460 con 0x55b24fce0360 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f78fffff640 1 -- 172.21.15.133:0/122045215 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79180c8050 con 0x55b24fce0360 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 10 client.5820 fetch_fsmap learned FSMap version 17 2024-04-25T10:05:20.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 -- 172.21.15.133:0/122045215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55b24fb66840 con 0x55b24fce0360 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f78fffff640 1 -- 172.21.15.133:0/122045215 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c05e460 con 0x55b24fce0360 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f78fffff640 1 -- 172.21.15.133:0/122045215 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f79180cb020 con 0x55b24fce0360 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 10 client.5820 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 10 client.5820 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 20 client.5820 populate_metadata read hostname 'smithi133' 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b24fca1c20 0x55b24fca2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 4 client.5820 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 -- 172.21.15.133:0/122045215 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b24fa7fbe0 con 0x55b24fca1c20 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925376640 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b24fca1c20 0x55b24fca2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f7925376640 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b24fca1c20 0x55b24fca2030 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1bf3f299eba2be server_cookie=2d92cdfe4bd9791d in_seq=0 out_seq=0 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f78fffff640 10 client.5820 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f78fffff640 1 -- 172.21.15.133:0/122045215 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b24fa7fbe0 con 0x55b24fca1c20 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f78fffff640 10 client.5820 handle_command_reply: tid=0 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 client.5820 shutdown 2024-04-25T10:05:20.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 -- 172.21.15.133:0/122045215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fce0360 msgr2=0x55b24fca0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fce0360 0x55b24fca0e80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f791801b020 tx=0x7f791801b1b0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 -- 172.21.15.133:0/122045215 shutdown_connections 2024-04-25T10:05:20.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b24fca1c20 0x55b24fca2030 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fce0360 0x55b24fca0e80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 0x55b24fce7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 --2- 172.21.15.133:0/122045215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fc9ad90 0x55b24fce71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f78fffff640 0 client.5820 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:05:20.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.230+0000 7f792a747000 1 -- 172.21.15.133:0/122045215 >> 172.21.15.133:0/122045215 conn(0x55b24fc96d40 msgr2=0x55b24fc999c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.234+0000 7f792a747000 1 -- 172.21.15.133:0/122045215 shutdown_connections 2024-04-25T10:05:20.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.234+0000 7f792a747000 1 -- 172.21.15.133:0/122045215 wait complete. 2024-04-25T10:05:20.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.234+0000 7f792a747000 20 client.5820 trim_cache size 0 max 16384 2024-04-25T10:05:20.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 Processor -- start 2024-04-25T10:05:20.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 -- start start 2024-04-25T10:05:20.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd44890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 0x55b24fd44dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fce0360 0x55b24fd45310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920203110 con 0x55b24fc9ad90 2024-04-25T10:05:20.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920202310 con 0x55b24fce0360 2024-04-25T10:05:20.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7920158020 con 0x55b24fcdfa40 2024-04-25T10:05:20.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f7925376640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd44890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f7925376640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd44890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55456/0 (socket says 172.21.15.133:55456) 2024-04-25T10:05:20.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f7925376640 1 -- 172.21.15.133:0/3154235083 learned_addr learned my addr 172.21.15.133:0/3154235083 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f7925b77640 1 --2- 172.21.15.133:0/3154235083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fce0360 0x55b24fd45310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f7925376640 1 -- 172.21.15.133:0/3154235083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 msgr2=0x55b24fd44dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f7925376640 1 --2- 172.21.15.133:0/3154235083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 0x55b24fd44dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f7925376640 1 -- 172.21.15.133:0/3154235083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fce0360 msgr2=0x55b24fd45310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f7925376640 1 --2- 172.21.15.133:0/3154235083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fce0360 0x55b24fd45310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f7925376640 1 -- 172.21.15.133:0/3154235083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b24fb3da10 con 0x55b24fc9ad90 2024-04-25T10:05:20.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f7925376640 1 --2- 172.21.15.133:0/3154235083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd44890 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f791c0839c0 tx=0x7f791c0074b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0134e28a6970930 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f790a7fc640 1 -- 172.21.15.133:0/3154235083 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c06d040 con 0x55b24fc9ad90 2024-04-25T10:05:20.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f790a7fc640 1 -- 172.21.15.133:0/3154235083 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c072070 con 0x55b24fc9ad90 2024-04-25T10:05:20.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f790a7fc640 1 -- 172.21.15.133:0/3154235083 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c05e460 con 0x55b24fc9ad90 2024-04-25T10:05:20.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 -- 172.21.15.133:0/3154235083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 msgr2=0x55b24fd44890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 --2- 172.21.15.133:0/3154235083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd44890 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f791c0839c0 tx=0x7f791c0074b0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 -- 172.21.15.133:0/3154235083 shutdown_connections 2024-04-25T10:05:20.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 --2- 172.21.15.133:0/3154235083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fce0360 0x55b24fd45310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 --2- 172.21.15.133:0/3154235083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 0x55b24fd44dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 --2- 172.21.15.133:0/3154235083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd44890 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 -- 172.21.15.133:0/3154235083 >> 172.21.15.133:0/3154235083 conn(0x55b24fc96d40 msgr2=0x55b24fced8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 -- 172.21.15.133:0/3154235083 shutdown_connections 2024-04-25T10:05:20.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.246+0000 7f792a747000 1 -- 172.21.15.133:0/3154235083 wait complete. 2024-04-25T10:05:20.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- start start 2024-04-25T10:05:20.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 Processor -- start 2024-04-25T10:05:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 0x55b24fd48de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fce0360 0x55b24fce4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b24fa9cdf0 con 0x55b24fc9ad90 2024-04-25T10:05:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920201d40 con 0x55b24fce0360 2024-04-25T10:05:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b24fa98cb0 con 0x55b24fcdfa40 2024-04-25T10:05:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 10 client.0 _ll_register_callbacks cb 0x55b24faa6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f7925376640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f7925376640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd488a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55464/0 (socket says 172.21.15.133:55464) 2024-04-25T10:05:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f7925376640 1 -- 172.21.15.133:0/3560273906 learned_addr learned my addr 172.21.15.133:0/3560273906 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f7925376640 1 -- 172.21.15.133:0/3560273906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 msgr2=0x55b24fd48de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f7925376640 1 --2- 172.21.15.133:0/3560273906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 0x55b24fd48de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f7925376640 1 -- 172.21.15.133:0/3560273906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fce0360 msgr2=0x55b24fce4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f7925376640 1 --2- 172.21.15.133:0/3560273906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fce0360 0x55b24fce4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f7925376640 1 -- 172.21.15.133:0/3560273906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b24fa78b00 con 0x55b24fc9ad90 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f7925376640 1 --2- 172.21.15.133:0/3560273906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd488a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f791c083970 tx=0x7f791c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2b1c2c73e4bf29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f78fffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f78fffff640 1 -- 172.21.15.133:0/3560273906 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c06d040 con 0x55b24fc9ad90 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f78fffff640 1 -- 172.21.15.133:0/3560273906 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f791c054050 con 0x55b24fc9ad90 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- 172.21.15.133:0/3560273906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f791c072070 con 0x55b24fc9ad90 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f78fffff640 1 -- 172.21.15.133:0/3560273906 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f791c063050 con 0x55b24fc9ad90 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 10 client.5826 fetch_fsmap learned FSMap version 17 2024-04-25T10:05:20.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- 172.21.15.133:0/3560273906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55b24fa8af80 con 0x55b24fc9ad90 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f78fffff640 1 -- 172.21.15.133:0/3560273906 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c072070 con 0x55b24fc9ad90 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f78fffff640 1 -- 172.21.15.133:0/3560273906 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f791c066080 con 0x55b24fc9ad90 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 10 client.5826 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 10 client.5826 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 20 client.5826 populate_metadata read hostname 'smithi133' 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 --2- 172.21.15.133:0/3560273906 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b24fce53a0 0x55b24fce57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 4 client.5826 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- 172.21.15.133:0/3560273906 --> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55b24faf13d0 con 0x55b24fce53a0 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f7924b75640 1 --2- 172.21.15.133:0/3560273906 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b24fce53a0 0x55b24fce57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f78fffff640 10 client.5826 ms_handle_connect on v2:172.21.15.133:6835/205640021 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f7924b75640 1 --2- 172.21.15.133:0/3560273906 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b24fce53a0 0x55b24fce57b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa9089be64515b1 server_cookie=b7ec477b12e36467 in_seq=0 out_seq=0 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f78fffff640 1 -- 172.21.15.133:0/3560273906 <== mds.0 v2:172.21.15.133:6835/205640021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17954 (crc 0 0 0) 0x55b24faf13d0 con 0x55b24fce53a0 2024-04-25T10:05:20.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f78fffff640 10 client.5826 handle_command_reply: tid=0 2024-04-25T10:05:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 client.5826 shutdown 2024-04-25T10:05:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- 172.21.15.133:0/3560273906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 msgr2=0x55b24fd488a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 --2- 172.21.15.133:0/3560273906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd488a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f791c083970 tx=0x7f791c010000 comp rx=0 tx=0).stop 2024-04-25T10:05:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- 172.21.15.133:0/3560273906 shutdown_connections 2024-04-25T10:05:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 --2- 172.21.15.133:0/3560273906 >> [v2:172.21.15.133:6835/205640021,v1:172.21.15.133:6837/205640021] conn(0x55b24fce53a0 0x55b24fce57b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 --2- 172.21.15.133:0/3560273906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b24fce0360 0x55b24fce4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 --2- 172.21.15.133:0/3560273906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b24fcdfa40 0x55b24fd48de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f78fffff640 0 client.5826 ms_handle_reset on v2:172.21.15.133:6835/205640021 2024-04-25T10:05:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 --2- 172.21.15.133:0/3560273906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b24fc9ad90 0x55b24fd488a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- 172.21.15.133:0/3560273906 >> 172.21.15.133:0/3560273906 conn(0x55b24fc96d40 msgr2=0x55b24fcde1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- 172.21.15.133:0/3560273906 shutdown_connections 2024-04-25T10:05:20.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 1 -- 172.21.15.133:0/3560273906 wait complete. 2024-04-25T10:05:20.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.250+0000 7f792a747000 20 client.5826 trim_cache size 0 max 16384 2024-04-25T10:05:20.268 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:05:20.268 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:05:20.268 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 5646, 2024-04-25T10:05:20.268 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 5647, 2024-04-25T10:05:20.268 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1413128, 2024-04-25T10:05:20.268 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 39795218, 2024-04-25T10:05:20.268 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 17, 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.579982519, 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.243425020, 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.113962978, 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.172766611, 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 5645, 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.597525856, 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000105850 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 10, 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000001529, 2024-04-25T10:05:20.269 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000152 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 4463, 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 4459, 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1213470, 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 18231785, 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 5, 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-04-25T10:05:20.270 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.456461922, 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.203903996, 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.094145746, 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.133088462, 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4459, 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.645811068, 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000144833 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 119696, 2024-04-25T10:05:20.271 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 30672 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 35096, 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 61306, 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 12811667, 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 60672056, 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 8, 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 4.829949803, 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 2.405608790, 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 2.853330965, 2024-04-25T10:05:20.272 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.122628936, 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 61305, 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.945259997, 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000031730 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 87534, 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.020088690, 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000229 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 130496 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-25T10:05:20.273 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 10360, 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 23.636449574, 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002281510 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T10:05:20.274 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3235, 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.639344113, 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000506752 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 26425, 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 26425, 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 26425, 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 432.619534874, 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.016371600 2024-04-25T10:05:20.275 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.276 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-04-25T10:05:20.276 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-04-25T10:05:20.276 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-04-25T10:05:20.276 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-04-25T10:05:20.276 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 12, 2024-04-25T10:05:20.276 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-04-25T10:05:20.276 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-04-25T10:05:20.276 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 17884, 2024-04-25T10:05:20.276 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-04-25T10:05:20.276 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 17884, 2024-04-25T10:05:20.277 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-04-25T10:05:20.277 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 17884, 2024-04-25T10:05:20.277 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-04-25T10:05:20.277 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 2, 2024-04-25T10:05:20.277 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 2, 2024-04-25T10:05:20.277 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 2, 2024-04-25T10:05:20.277 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 42640, 2024-04-25T10:05:20.278 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 42193, 2024-04-25T10:05:20.278 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-04-25T10:05:20.278 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-04-25T10:05:20.278 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:05:20.278 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-04-25T10:05:20.278 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 141, 2024-04-25T10:05:20.279 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-04-25T10:05:20.279 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-04-25T10:05:20.279 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-04-25T10:05:20.279 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-04-25T10:05:20.279 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-04-25T10:05:20.279 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-04-25T10:05:20.280 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-04-25T10:05:20.280 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 2013, 2024-04-25T10:05:20.280 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-04-25T10:05:20.280 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 0, 2024-04-25T10:05:20.280 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 0, 2024-04-25T10:05:20.280 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-04-25T10:05:20.280 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 2013, 2024-04-25T10:05:20.280 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-04-25T10:05:20.280 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:05:20.280 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 13, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 2000, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 7798, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 6340, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 7798, 2024-04-25T10:05:20.281 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 46, 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 17976 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-04-25T10:05:20.282 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 4140, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 3616, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 2224, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-04-25T10:05:20.283 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:05:20.284 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-04-25T10:05:20.284 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-04-25T10:05:20.284 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.284 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-04-25T10:05:20.284 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-04-25T10:05:20.284 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 25793, 2024-04-25T10:05:20.284 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-04-25T10:05:20.284 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-04-25T10:05:20.284 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 25793, 2024-04-25T10:05:20.284 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-04-25T10:05:20.284 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-04-25T10:05:20.285 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 34, 2024-04-25T10:05:20.285 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-04-25T10:05:20.285 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-04-25T10:05:20.285 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 34, 2024-04-25T10:05:20.285 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 4, 2024-04-25T10:05:20.285 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-04-25T10:05:20.286 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-04-25T10:05:20.286 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-04-25T10:05:20.286 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 86891319, 2024-04-25T10:05:20.286 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2024-04-25T10:05:20.286 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-04-25T10:05:20.287 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6858, 2024-04-25T10:05:20.287 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 118.669125205, 2024-04-25T10:05:20.287 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.017303751 2024-04-25T10:05:20.287 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.287 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2024-04-25T10:05:20.287 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.287 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-04-25T10:05:20.288 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 14, 2024-04-25T10:05:20.288 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 2016, 2024-04-25T10:05:20.288 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 2002, 2024-04-25T10:05:20.288 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 13, 2024-04-25T10:05:20.288 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 13, 2024-04-25T10:05:20.289 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 0, 2024-04-25T10:05:20.289 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 17884, 2024-04-25T10:05:20.289 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 19884, 2024-04-25T10:05:20.289 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 2000, 2024-04-25T10:05:20.289 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 2, 2024-04-25T10:05:20.289 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 2002, 2024-04-25T10:05:20.290 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 2000, 2024-04-25T10:05:20.290 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 93252, 2024-04-25T10:05:20.290 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215340 2024-04-25T10:05:20.290 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.290 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-04-25T10:05:20.290 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 35551, 2024-04-25T10:05:20.290 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-04-25T10:05:20.290 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 26425, 2024-04-25T10:05:20.291 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 8, 2024-04-25T10:05:20.291 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-04-25T10:05:20.291 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-04-25T10:05:20.291 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2000, 2024-04-25T10:05:20.291 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.344601492, 2024-04-25T10:05:20.291 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000672300 2024-04-25T10:05:20.291 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.291 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-04-25T10:05:20.292 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 260, 2024-04-25T10:05:20.292 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.082059183, 2024-04-25T10:05:20.292 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000315612 2024-04-25T10:05:20.292 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.292 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-04-25T10:05:20.292 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.292 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.293 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.293 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.293 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-04-25T10:05:20.293 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6000, 2024-04-25T10:05:20.293 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 9.525815933, 2024-04-25T10:05:20.293 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001587635 2024-04-25T10:05:20.293 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7928, 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 14.913641301, 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001881135 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.294 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-04-25T10:05:20.295 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3, 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.032204813, 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.010734937 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.003627006, 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003627006 2024-04-25T10:05:20.296 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.009036052, 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.009036052 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.297 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6, 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.160938609, 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.026823101 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2224, 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 378.677113062, 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.170268486 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.298 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.006591237, 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.006591237 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000299273, 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000299273 2024-04-25T10:05:20.300 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.301 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-04-25T10:05:20.302 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8000, 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 27.863606913, 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003482950 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-04-25T10:05:20.303 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-04-25T10:05:20.304 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 1, 2024-04-25T10:05:20.304 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 0, 2024-04-25T10:05:20.304 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 1, 2024-04-25T10:05:20.304 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-04-25T10:05:20.304 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 15254, 2024-04-25T10:05:20.304 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 15254, 2024-04-25T10:05:20.304 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 112, 2024-04-25T10:05:20.304 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-04-25T10:05:20.304 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.304 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-25T10:05:20.304 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:05:20.305 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-25T10:05:20.306 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 251650, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 86, 2024-04-25T10:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:05:20.308 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-04-25T10:05:20.309 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-25T10:05:20.309 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-25T10:05:20.309 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:05:20.309 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-25T10:05:20.309 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:05:20.309 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-25T10:05:20.310 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 5624, 2024-04-25T10:05:20.310 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 80, 2024-04-25T10:05:20.310 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:05:20.310 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:05:20.310 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-25T10:05:20.310 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-25T10:05:20.310 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 13466100, 2024-04-25T10:05:20.310 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 661331, 2024-04-25T10:05:20.311 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-25T10:05:20.311 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-25T10:05:20.311 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-25T10:05:20.311 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-25T10:05:20.311 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.311 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-04-25T10:05:20.312 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-04-25T10:05:20.312 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-04-25T10:05:20.312 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 12902, 2024-04-25T10:05:20.312 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 83200254, 2024-04-25T10:05:20.312 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 37, 2024-04-25T10:05:20.312 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 12902, 2024-04-25T10:05:20.313 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-04-25T10:05:20.313 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 12902, 2024-04-25T10:05:20.313 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 56.459416475, 2024-04-25T10:05:20.313 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004376020 2024-04-25T10:05:20.313 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.313 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-04-25T10:05:20.313 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-04-25T10:05:20.314 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 12902, 2024-04-25T10:05:20.314 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 13053 2024-04-25T10:05:20.314 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.314 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 12902, 2024-04-25T10:05:20.314 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 2122, 2024-04-25T10:05:20.314 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 10780, 2024-04-25T10:05:20.315 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-04-25T10:05:20.315 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-04-25T10:05:20.315 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 0, 2024-04-25T10:05:20.315 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 11, 2024-04-25T10:05:20.315 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 107, 2024-04-25T10:05:20.315 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 6963, 2024-04-25T10:05:20.315 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 61, 2024-04-25T10:05:20.315 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-04-25T10:05:20.316 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-04-25T10:05:20.316 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 1616, 2024-04-25T10:05:20.316 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-04-25T10:05:20.316 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 2031, 2024-04-25T10:05:20.316 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-04-25T10:05:20.317 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-04-25T10:05:20.317 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-04-25T10:05:20.317 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 2015, 2024-04-25T10:05:20.317 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 22, 2024-04-25T10:05:20.317 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:05:20.317 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-04-25T10:05:20.318 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:05:20.318 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-04-25T10:05:20.318 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-04-25T10:05:20.318 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-04-25T10:05:20.318 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:05:20.318 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-04-25T10:05:20.318 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:05:20.318 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-04-25T10:05:20.318 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-04-25T10:05:20.319 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 42, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 5, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 8, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 8, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 180, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 0, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 47 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-04-25T10:05:20.320 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 32, 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 3616, 2024-04-25T10:05:20.321 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 3616, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 45201, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 11364196, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 45201, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 45201, 2024-04-25T10:05:20.322 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 11364196, 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T10:05:20.323 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 12902, 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 83578398, 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 12902, 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 83578398, 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-04-25T10:05:20.324 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 12902, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 12902, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 12902, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 12902, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.325 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 3616, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 378144, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 3616, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 105, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 378144, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.326 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55a5d831a960": { 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 25793, 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 82697015, 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 25793, 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:05:20.327 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 6858, 2024-04-25T10:05:20.328 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 82697015, 2024-04-25T10:05:20.328 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:05:20.328 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:05:20.328 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:05:20.328 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:05:20.328 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:20.328 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:20.328 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:05:20.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 -- 172.21.15.133:0/2261515780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7904066f60 msgr2=0x7f7904069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7904066f60 0x7f7904069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f79200cb6c0 tx=0x7f7910007cd0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 -- 172.21.15.133:0/2261515780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79200ce600 msgr2=0x7f792015cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79200ce600 0x7f792015cbb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f791c00d620 tx=0x7f791c00dae0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 -- 172.21.15.133:0/2261515780 shutdown_connections 2024-04-25T10:05:20.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f792015d9a0 0x7f792015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79200d2030 0x7f79200c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7904066f60 0x7f7904069420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 --2- 172.21.15.133:0/2261515780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79200ce600 0x7f792015cbb0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 -- 172.21.15.133:0/2261515780 >> 172.21.15.133:0/2261515780 conn(0x7f792006bf80 msgr2=0x7f792015a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 -- 172.21.15.133:0/2261515780 shutdown_connections 2024-04-25T10:05:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.254+0000 7f79275da640 1 -- 172.21.15.133:0/2261515780 wait complete. 2024-04-25T10:05:20.331 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:20.332 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:20.370 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:20.419 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:20.420 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:05:20.420 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:20.426 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:20.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e93d4640 1 Processor -- start 2024-04-25T10:05:20.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e93d4640 1 -- start start 2024-04-25T10:05:20.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e93d4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e4158090 0x7fa9e40b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e93d4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa9e406b3a0 0x7fa9e4161590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e93d4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e415e1b0 con 0x7fa9e415ec50 2024-04-25T10:05:20.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e93d4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e415df50 con 0x7fa9e406b3a0 2024-04-25T10:05:20.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e93d4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40b4cb0 con 0x7fa9e4158090 2024-04-25T10:05:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e37fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa9e406b3a0 0x7fa9e4161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e2ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e4158090 0x7fa9e40b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9da7fc640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa9e415ec50 0x7fa9e406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:53400/0 (socket says 172.21.15.133:53400) 2024-04-25T10:05:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9da7fc640 1 -- 172.21.15.133:0/2451760635 learned_addr learned my addr 172.21.15.133:0/2451760635 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e27fc640 1 -- 172.21.15.133:0/2451760635 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776851512 0 0) 0x7fa9e415e1b0 con 0x7fa9e415ec50 2024-04-25T10:05:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e27fc640 1 -- 172.21.15.133:0/2451760635 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d0003530 con 0x7fa9e415ec50 2024-04-25T10:05:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e2ffd640 1 -- 172.21.15.133:0/2451760635 >> v2:172.21.15.162:3300/0 conn(0x7fa9e406b3a0 msgr2=0x7fa9e4161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e2ffd640 1 --2- 172.21.15.133:0/2451760635 >> v2:172.21.15.162:3300/0 conn(0x7fa9e406b3a0 0x7fa9e4161590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e2ffd640 1 -- 172.21.15.133:0/2451760635 >> v1:172.21.15.162:6789/0 conn(0x7fa9e415ec50 legacy=0x7fa9e406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e2ffd640 1 -- 172.21.15.133:0/2451760635 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e40b49d0 con 0x7fa9e4158090 2024-04-25T10:05:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.586+0000 7fa9e37fe640 1 --2- 172.21.15.133:0/2451760635 >> v2:172.21.15.162:3300/0 conn(0x7fa9e406b3a0 0x7fa9e4161590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:20.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e2ffd640 1 --2- 172.21.15.133:0/2451760635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e4158090 0x7fa9e40b4180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40097c0 tx=0x7fa9d401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28d2019bc387b69e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e27fc640 1 -- 172.21.15.133:0/2451760635 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40b0070 con 0x7fa9e4158090 2024-04-25T10:05:20.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e27fc640 1 -- 172.21.15.133:0/2451760635 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d40ab600 con 0x7fa9e4158090 2024-04-25T10:05:20.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e27fc640 1 -- 172.21.15.133:0/2451760635 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40ab9d0 con 0x7fa9e4158090 2024-04-25T10:05:20.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2451760635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e4158090 msgr2=0x7fa9e40b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 --2- 172.21.15.133:0/2451760635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e4158090 0x7fa9e40b4180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40097c0 tx=0x7fa9d401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2451760635 shutdown_connections 2024-04-25T10:05:20.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 --2- 172.21.15.133:0/2451760635 >> v2:172.21.15.162:3300/0 conn(0x7fa9e406b3a0 0x7fa9e4161590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 --2- 172.21.15.133:0/2451760635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e4158090 0x7fa9e40b4180 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2451760635 >> 172.21.15.133:0/2451760635 conn(0x7fa9e40b2370 msgr2=0x7fa9e40b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2451760635 shutdown_connections 2024-04-25T10:05:20.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2451760635 wait complete. 2024-04-25T10:05:20.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 Processor -- start 2024-04-25T10:05:20.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 -- start start 2024-04-25T10:05:20.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e406b3a0 0x7fa9e41558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e4158090 0x7fa9e40b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e415ec50 0x7fa9e40b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e416a880 con 0x7fa9e4158090 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e416a700 con 0x7fa9e415ec50 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9e93d4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e416aa00 con 0x7fa9e406b3a0 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9da7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e4158090 0x7fa9e40b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9da7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e4158090 0x7fa9e40b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55470/0 (socket says 172.21.15.133:55470) 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.590+0000 7fa9da7fc640 1 -- 172.21.15.133:0/2098485358 learned_addr learned my addr 172.21.15.133:0/2098485358 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9e2ffd640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e406b3a0 0x7fa9e41558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9e37fe640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e415ec50 0x7fa9e40b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9da7fc640 1 -- 172.21.15.133:0/2098485358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e406b3a0 msgr2=0x7fa9e41558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9da7fc640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e406b3a0 0x7fa9e41558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9da7fc640 1 -- 172.21.15.133:0/2098485358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e415ec50 msgr2=0x7fa9e40b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9da7fc640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e415ec50 0x7fa9e40b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9da7fc640 1 -- 172.21.15.133:0/2098485358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e40cac50 con 0x7fa9e4158090 2024-04-25T10:05:20.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9e2ffd640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e406b3a0 0x7fa9e41558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:20.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9e37fe640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e415ec50 0x7fa9e40b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:20.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9da7fc640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e4158090 0x7fa9e40b0dc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c480 tx=0x7fa9cc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9dbfff640 1 -- 172.21.15.133:0/2098485358 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc017070 con 0x7fa9e4158090 2024-04-25T10:05:20.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9dbfff640 1 -- 172.21.15.133:0/2098485358 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc005cf0 con 0x7fa9e4158090 2024-04-25T10:05:20.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9dbfff640 1 -- 172.21.15.133:0/2098485358 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc007290 con 0x7fa9e4158090 2024-04-25T10:05:20.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2098485358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e40caf30 con 0x7fa9e4158090 2024-04-25T10:05:20.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2098485358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e4155e10 con 0x7fa9e4158090 2024-04-25T10:05:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2098485358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e4067c70 con 0x7fa9e4158090 2024-04-25T10:05:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9dbfff640 1 -- 172.21.15.133:0/2098485358 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9cc01a070 con 0x7fa9e4158090 2024-04-25T10:05:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9dbfff640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 0x7fa9c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9dbfff640 1 -- 172.21.15.133:0/2098485358 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa9cc0845a0 con 0x7fa9e4158090 2024-04-25T10:05:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9e2ffd640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 0x7fa9c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.594+0000 7fa9e2ffd640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 0x7fa9c4069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40ad000 tx=0x7fa9d40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.598+0000 7fa9dbfff640 1 -- 172.21.15.133:0/2098485358 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9cc055480 con 0x7fa9e4158090 2024-04-25T10:05:20.758 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:20.760 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:20.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.758+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2098485358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9e415f6f0 con 0x7fa9e4158090 2024-04-25T10:05:20.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9dbfff640 1 -- 172.21.15.133:0/2098485358 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17080 (secure 0 0 0) 0x7fa9cc054d80 con 0x7fa9e4158090 2024-04-25T10:05:20.770 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:20.770 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":42,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:04:56.820759+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:16.032957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:16.760653+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2098485358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 msgr2=0x7fa9c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9e93d4640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 0x7fa9c4069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40ad000 tx=0x7fa9d40a6000 comp rx=0 tx=0).stop 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2098485358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e4158090 msgr2=0x7fa9e40b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9e93d4640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e4158090 0x7fa9e40b0dc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c480 tx=0x7fa9cc00c940 comp rx=0 tx=0).stop 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2098485358 shutdown_connections 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9e93d4640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9e415ec50 0x7fa9e40b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9e93d4640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 0x7fa9c4069420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9e93d4640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9e4158090 0x7fa9e40b0dc0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9e93d4640 1 --2- 172.21.15.133:0/2098485358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9e406b3a0 0x7fa9e41558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2098485358 >> 172.21.15.133:0/2098485358 conn(0x7fa9e40b2370 msgr2=0x7fa9e40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.762+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2098485358 shutdown_connections 2024-04-25T10:05:20.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.766+0000 7fa9e93d4640 1 -- 172.21.15.133:0/2098485358 wait complete. 2024-04-25T10:05:20.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:16.032957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:16.760653+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 41, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:20.789 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:20.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5c19f7640 1 Processor -- start 2024-04-25T10:05:20.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5c19f7640 1 -- start start 2024-04-25T10:05:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5c19f7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5bc0cd820 0x7ff5bc0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5c19f7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff5bc15ced0 0x7ff5bc15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5c19f7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5bc0ce2e0 con 0x7ff5bc145170 2024-04-25T10:05:20.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5c19f7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc06db00 con 0x7ff5bc15ced0 2024-04-25T10:05:20.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5c19f7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc145550 con 0x7ff5bc0cd820 2024-04-25T10:05:20.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5bb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff5bc15ced0 0x7ff5bc15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5bb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff5bc15ced0 0x7ff5bc15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55478/0 (socket says 172.21.15.133:55478) 2024-04-25T10:05:20.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5baffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff5bc145170 0x7ff5bc06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49772/0 (socket says 172.21.15.133:49772) 2024-04-25T10:05:20.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5bb7fe640 1 -- 172.21.15.133:0/234793974 learned_addr learned my addr 172.21.15.133:0/234793974 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5ba7fc640 1 --2- 172.21.15.133:0/234793974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5bc0cd820 0x7ff5bc0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5bb7fe640 1 -- 172.21.15.133:0/234793974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5bc0cd820 msgr2=0x7ff5bc0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5bb7fe640 1 --2- 172.21.15.133:0/234793974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5bc0cd820 0x7ff5bc0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5bb7fe640 1 -- 172.21.15.133:0/234793974 >> v1:172.21.15.133:6789/0 conn(0x7ff5bc145170 legacy=0x7ff5bc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5bb7fe640 1 -- 172.21.15.133:0/234793974 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc06d4b0 con 0x7ff5bc15ced0 2024-04-25T10:05:20.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5b9ffb640 1 -- 172.21.15.133:0/234793974 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247865485 0 0) 0x7ff5bc0ce2e0 con 0x7ff5bc145170 2024-04-25T10:05:20.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5bb7fe640 1 --2- 172.21.15.133:0/234793974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc15ced0 0x7ff5bc15d360 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009870 tx=0x7ff5b401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48725aab757548fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.958+0000 7ff5b9ffb640 1 -- 172.21.15.133:0/234793974 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40b2070 con 0x7ff5bc15ced0 2024-04-25T10:05:20.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5b9ffb640 1 -- 172.21.15.133:0/234793974 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b40a9070 con 0x7ff5bc15ced0 2024-04-25T10:05:20.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5b9ffb640 1 -- 172.21.15.133:0/234793974 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40ad880 con 0x7ff5bc15ced0 2024-04-25T10:05:20.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 -- 172.21.15.133:0/234793974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc15ced0 msgr2=0x7ff5bc15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 --2- 172.21.15.133:0/234793974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc15ced0 0x7ff5bc15d360 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009870 tx=0x7ff5b401c560 comp rx=0 tx=0).stop 2024-04-25T10:05:20.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 -- 172.21.15.133:0/234793974 shutdown_connections 2024-04-25T10:05:20.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 --2- 172.21.15.133:0/234793974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc15ced0 0x7ff5bc15d360 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 --2- 172.21.15.133:0/234793974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5bc0cd820 0x7ff5bc0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 -- 172.21.15.133:0/234793974 >> 172.21.15.133:0/234793974 conn(0x7ff5bc066f70 msgr2=0x7ff5bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 -- 172.21.15.133:0/234793974 shutdown_connections 2024-04-25T10:05:20.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 -- 172.21.15.133:0/234793974 wait complete. 2024-04-25T10:05:20.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 Processor -- start 2024-04-25T10:05:20.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 -- start start 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5bc0cd820 0x7ff5bc13ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc145170 0x7ff5bc15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5bc15ced0 0x7ff5bc13b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc168610 con 0x7ff5bc145170 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc168490 con 0x7ff5bc15ced0 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc168790 con 0x7ff5bc0cd820 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5ba7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc145170 0x7ff5bc15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5ba7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc145170 0x7ff5bc15d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55488/0 (socket says 172.21.15.133:55488) 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5ba7fc640 1 -- 172.21.15.133:0/1755930887 learned_addr learned my addr 172.21.15.133:0/1755930887 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5baffd640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5bc0cd820 0x7ff5bc13ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5bb7fe640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5bc15ced0 0x7ff5bc13b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5ba7fc640 1 -- 172.21.15.133:0/1755930887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5bc0cd820 msgr2=0x7ff5bc13ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5ba7fc640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5bc0cd820 0x7ff5bc13ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5ba7fc640 1 -- 172.21.15.133:0/1755930887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5bc15ced0 msgr2=0x7ff5bc13b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5ba7fc640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5bc15ced0 0x7ff5bc13b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5ba7fc640 1 -- 172.21.15.133:0/1755930887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc1448d0 con 0x7ff5bc145170 2024-04-25T10:05:20.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5baffd640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5bc0cd820 0x7ff5bc13ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5ba7fc640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc145170 0x7ff5bc15d9b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00d830 tx=0x7ff5ac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5b3fff640 1 -- 172.21.15.133:0/1755930887 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac00b6c0 con 0x7ff5bc145170 2024-04-25T10:05:20.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5b3fff640 1 -- 172.21.15.133:0/1755930887 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ac019070 con 0x7ff5bc145170 2024-04-25T10:05:20.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 -- 172.21.15.133:0/1755930887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc0ab8d0 con 0x7ff5bc145170 2024-04-25T10:05:20.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5b3fff640 1 -- 172.21.15.133:0/1755930887 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac00ba60 con 0x7ff5bc145170 2024-04-25T10:05:20.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.962+0000 7ff5c19f7640 1 -- 172.21.15.133:0/1755930887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc1440d0 con 0x7ff5bc145170 2024-04-25T10:05:20.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.966+0000 7ff5c19f7640 1 -- 172.21.15.133:0/1755930887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5bc068e80 con 0x7ff5bc145170 2024-04-25T10:05:20.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.966+0000 7ff5b3fff640 1 -- 172.21.15.133:0/1755930887 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5ac01e460 con 0x7ff5bc145170 2024-04-25T10:05:20.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.966+0000 7ff5b3fff640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff59c066f60 0x7ff59c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.966+0000 7ff5b3fff640 1 -- 172.21.15.133:0/1755930887 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7ff5ac085840 con 0x7ff5bc145170 2024-04-25T10:05:20.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.966+0000 7ff5baffd640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff59c066f60 0x7ff59c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.966+0000 7ff5baffd640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff59c066f60 0x7ff59c069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4007fd0 tx=0x7ff5a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:20.970+0000 7ff5b3fff640 1 -- 172.21.15.133:0/1755930887 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5ac056720 con 0x7ff5bc145170 2024-04-25T10:05:21.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 -- 172.21.15.133:0/1755930887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5bc13e440 con 0x7ff5bc145170 2024-04-25T10:05:21.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5b3fff640 1 -- 172.21.15.133:0/1755930887 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5ac056020 con 0x7ff5bc145170 2024-04-25T10:05:21.141 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:05:21.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 -- 172.21.15.133:0/1755930887 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff59c066f60 msgr2=0x7ff59c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff59c066f60 0x7ff59c069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4007fd0 tx=0x7ff5a4009040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 -- 172.21.15.133:0/1755930887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc145170 msgr2=0x7ff5bc15d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc145170 0x7ff5bc15d9b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00d830 tx=0x7ff5ac00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 -- 172.21.15.133:0/1755930887 shutdown_connections 2024-04-25T10:05:21.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5bc15ced0 0x7ff5bc13b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff59c066f60 0x7ff59c069420 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc145170 0x7ff5bc15d9b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 --2- 172.21.15.133:0/1755930887 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5bc0cd820 0x7ff5bc13ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 -- 172.21.15.133:0/1755930887 >> 172.21.15.133:0/1755930887 conn(0x7ff5bc066f70 msgr2=0x7ff5bc13d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 -- 172.21.15.133:0/1755930887 shutdown_connections 2024-04-25T10:05:21.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.134+0000 7ff5c19f7640 1 -- 172.21.15.133:0/1755930887 wait complete. 2024-04-25T10:05:21.159 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:21.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0c290e640 1 Processor -- start 2024-04-25T10:05:21.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0c290e640 1 -- start start 2024-04-25T10:05:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0c290e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 0x7ff0bc0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0c290e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff0bc0d1920 0x7ff0bc0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0c290e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc15ebd0 con 0x7ff0bc15d9a0 2024-04-25T10:05:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0c290e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0bc0cefa0 con 0x7ff0bc0d1920 2024-04-25T10:05:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0c290e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc15eeb0 con 0x7ff0bc0ce600 2024-04-25T10:05:21.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0c0eab640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff0bc0d1920 0x7ff0bc0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0bb7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 0x7ff0bc0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0bb7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 0x7ff0bc0cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:59202/0 (socket says 172.21.15.133:59202) 2024-04-25T10:05:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0bb7fe640 1 -- 172.21.15.133:0/1635996790 learned_addr learned my addr 172.21.15.133:0/1635996790 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0baffd640 1 -- 172.21.15.133:0/1635996790 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948060726 0 0) 0x7ff0bc15ebd0 con 0x7ff0bc15d9a0 2024-04-25T10:05:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0baffd640 1 -- 172.21.15.133:0/1635996790 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0a4003530 con 0x7ff0bc15d9a0 2024-04-25T10:05:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0bb7fe640 1 -- 172.21.15.133:0/1635996790 >> v2:172.21.15.162:3300/0 conn(0x7ff0bc0d1920 msgr2=0x7ff0bc0d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0bb7fe640 1 --2- 172.21.15.133:0/1635996790 >> v2:172.21.15.162:3300/0 conn(0x7ff0bc0d1920 0x7ff0bc0d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0bb7fe640 1 -- 172.21.15.133:0/1635996790 >> v1:172.21.15.162:6789/0 conn(0x7ff0bc15d9a0 legacy=0x7ff0bc0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.274+0000 7ff0bb7fe640 1 -- 172.21.15.133:0/1635996790 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc0d0330 con 0x7ff0bc0ce600 2024-04-25T10:05:21.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0bb7fe640 1 --2- 172.21.15.133:0/1635996790 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 0x7ff0bc0cea60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009870 tx=0x7ff0b001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e442d219acd78425 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0baffd640 1 -- 172.21.15.133:0/1635996790 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00aa620 con 0x7ff0bc0ce600 2024-04-25T10:05:21.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0baffd640 1 -- 172.21.15.133:0/1635996790 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b00af070 con 0x7ff0bc0ce600 2024-04-25T10:05:21.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0baffd640 1 -- 172.21.15.133:0/1635996790 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00aa9e0 con 0x7ff0bc0ce600 2024-04-25T10:05:21.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0c290e640 1 -- 172.21.15.133:0/1635996790 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 msgr2=0x7ff0bc0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0c290e640 1 --2- 172.21.15.133:0/1635996790 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 0x7ff0bc0cea60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009870 tx=0x7ff0b001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0c290e640 1 -- 172.21.15.133:0/1635996790 shutdown_connections 2024-04-25T10:05:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0c290e640 1 --2- 172.21.15.133:0/1635996790 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 0x7ff0bc0cea60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0c290e640 1 --2- 172.21.15.133:0/1635996790 >> v2:172.21.15.162:3300/0 conn(0x7ff0bc0d1920 0x7ff0bc0d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0c290e640 1 -- 172.21.15.133:0/1635996790 >> 172.21.15.133:0/1635996790 conn(0x7ff0bc06bf80 msgr2=0x7ff0bc06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0c290e640 1 -- 172.21.15.133:0/1635996790 shutdown_connections 2024-04-25T10:05:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0c290e640 1 -- 172.21.15.133:0/1635996790 wait complete. 2024-04-25T10:05:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0c290e640 1 Processor -- start 2024-04-25T10:05:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.278+0000 7ff0c290e640 1 -- start start 2024-04-25T10:05:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0c290e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 0x7ff0bc155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0c290e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0bc0d1920 0x7ff0bc15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0c290e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0bc15d9a0 0x7ff0bc14d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0c290e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0d2470 con 0x7ff0bc0d1920 2024-04-25T10:05:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0c290e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0d22f0 con 0x7ff0bc15d9a0 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0c290e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc15e590 con 0x7ff0bc0ce600 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0bc0d1920 0x7ff0bc15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0bc0d1920 0x7ff0bc15b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55492/0 (socket says 172.21.15.133:55492) 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bb7fe640 1 -- 172.21.15.133:0/2901624387 learned_addr learned my addr 172.21.15.133:0/2901624387 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bbfff640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 0x7ff0bc155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0c0eab640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0bc15d9a0 0x7ff0bc14d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bb7fe640 1 -- 172.21.15.133:0/2901624387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 msgr2=0x7ff0bc155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bb7fe640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 0x7ff0bc155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bb7fe640 1 -- 172.21.15.133:0/2901624387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0bc15d9a0 msgr2=0x7ff0bc14d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bb7fe640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0bc15d9a0 0x7ff0bc14d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bb7fe640 1 -- 172.21.15.133:0/2901624387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc160cd0 con 0x7ff0bc0d1920 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bb7fe640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0bc0d1920 0x7ff0bc15b860 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00ac040 tx=0x7ff0b001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0b8ff9640 1 -- 172.21.15.133:0/2901624387 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00aab80 con 0x7ff0bc0d1920 2024-04-25T10:05:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0b8ff9640 1 -- 172.21.15.133:0/2901624387 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b00aad20 con 0x7ff0bc0d1920 2024-04-25T10:05:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0c290e640 1 -- 172.21.15.133:0/2901624387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0bc160fb0 con 0x7ff0bc0d1920 2024-04-25T10:05:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0b8ff9640 1 -- 172.21.15.133:0/2901624387 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00bd630 con 0x7ff0bc0d1920 2024-04-25T10:05:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0c290e640 1 -- 172.21.15.133:0/2901624387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0bc155bd0 con 0x7ff0bc0d1920 2024-04-25T10:05:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0c290e640 1 -- 172.21.15.133:0/2901624387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0bc068e80 con 0x7ff0bc0d1920 2024-04-25T10:05:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0b8ff9640 1 -- 172.21.15.133:0/2901624387 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0b00c6430 con 0x7ff0bc0d1920 2024-04-25T10:05:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0b8ff9640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a8066f60 0x7ff0a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bbfff640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a8066f60 0x7ff0a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0b8ff9640 1 -- 172.21.15.133:0/2901624387 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7ff0b011f6d0 con 0x7ff0bc0d1920 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.282+0000 7ff0bbfff640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a8066f60 0x7ff0a8069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0015e0 tx=0x7ff0ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.286+0000 7ff0b8ff9640 1 -- 172.21.15.133:0/2901624387 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0b0016150 con 0x7ff0bc0d1920 2024-04-25T10:05:21.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.434+0000 7ff0c290e640 1 -- 172.21.15.133:0/2901624387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff0bc0cda60 con 0x7ff0bc0d1920 2024-04-25T10:05:21.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.434+0000 7ff0b8ff9640 1 -- 172.21.15.133:0/2901624387 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0b00f05b0 con 0x7ff0bc0d1920 2024-04-25T10:05:21.443 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:05:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 -- 172.21.15.133:0/2901624387 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a8066f60 msgr2=0x7ff0a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a8066f60 0x7ff0a8069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0015e0 tx=0x7ff0ac009040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 -- 172.21.15.133:0/2901624387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0bc0d1920 msgr2=0x7ff0bc15b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0bc0d1920 0x7ff0bc15b860 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00ac040 tx=0x7ff0b001ace0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 -- 172.21.15.133:0/2901624387 shutdown_connections 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0bc15d9a0 0x7ff0bc14d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a8066f60 0x7ff0a8069420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0bc0d1920 0x7ff0bc15b860 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 --2- 172.21.15.133:0/2901624387 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0bc0ce600 0x7ff0bc155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 -- 172.21.15.133:0/2901624387 >> 172.21.15.133:0/2901624387 conn(0x7ff0bc06bf80 msgr2=0x7ff0bc0b79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 -- 172.21.15.133:0/2901624387 shutdown_connections 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.438+0000 7ff0c290e640 1 -- 172.21.15.133:0/2901624387 wait complete. 2024-04-25T10:05:21.460 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:21.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.562+0000 7f7d4902b640 1 Processor -- start 2024-04-25T10:05:21.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.562+0000 7f7d4902b640 1 -- start start 2024-04-25T10:05:21.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d4902b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d44145170 0x7f7d4406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d4902b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7d4406e4d0 0x7f7d440cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d4902b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d440ce2e0 con 0x7f7d440cdd60 2024-04-25T10:05:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d4902b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4406db00 con 0x7f7d4406e4d0 2024-04-25T10:05:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d4902b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d44145550 con 0x7f7d44145170 2024-04-25T10:05:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d3bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7d4406e4d0 0x7f7d440cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d3bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7d4406e4d0 0x7f7d440cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55496/0 (socket says 172.21.15.133:55496) 2024-04-25T10:05:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d3bfff640 1 -- 172.21.15.133:0/685975137 learned_addr learned my addr 172.21.15.133:0/685975137 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d3bfff640 1 -- 172.21.15.133:0/685975137 >> v2:172.21.15.162:3300/0 conn(0x7f7d44145170 msgr2=0x7f7d4406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d3bfff640 1 --2- 172.21.15.133:0/685975137 >> v2:172.21.15.162:3300/0 conn(0x7f7d44145170 0x7f7d4406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d3bfff640 1 -- 172.21.15.133:0/685975137 >> v1:172.21.15.162:6789/0 conn(0x7f7d440cdd60 legacy=0x7f7d4415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d3bfff640 1 -- 172.21.15.133:0/685975137 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4406d4b0 con 0x7f7d4406e4d0 2024-04-25T10:05:21.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d3bfff640 1 --2- 172.21.15.133:0/685975137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d4406e4d0 0x7f7d440cd820 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7d4406b790 tx=0x7f7d2c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec4ba35b029f46c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d4259c640 1 -- 172.21.15.133:0/685975137 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0b0070 con 0x7f7d4406e4d0 2024-04-25T10:05:21.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d4259c640 1 -- 172.21.15.133:0/685975137 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c0ab600 con 0x7f7d4406e4d0 2024-04-25T10:05:21.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d4259c640 1 -- 172.21.15.133:0/685975137 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0ab9d0 con 0x7f7d4406e4d0 2024-04-25T10:05:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d4902b640 1 -- 172.21.15.133:0/685975137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d4406e4d0 msgr2=0x7f7d440cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.566+0000 7f7d4902b640 1 --2- 172.21.15.133:0/685975137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d4406e4d0 0x7f7d440cd820 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7d4406b790 tx=0x7f7d2c01c450 comp rx=0 tx=0).stop 2024-04-25T10:05:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 -- 172.21.15.133:0/685975137 shutdown_connections 2024-04-25T10:05:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 --2- 172.21.15.133:0/685975137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d4406e4d0 0x7f7d440cd820 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 --2- 172.21.15.133:0/685975137 >> v2:172.21.15.162:3300/0 conn(0x7f7d44145170 0x7f7d4406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 -- 172.21.15.133:0/685975137 >> 172.21.15.133:0/685975137 conn(0x7f7d44066f70 msgr2=0x7f7d4406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 -- 172.21.15.133:0/685975137 shutdown_connections 2024-04-25T10:05:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 -- 172.21.15.133:0/685975137 wait complete. 2024-04-25T10:05:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 Processor -- start 2024-04-25T10:05:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 -- start start 2024-04-25T10:05:21.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d4406e4d0 0x7f7d440b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d440cdd60 0x7f7d440ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d44145170 0x7f7d44153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44168490 con 0x7f7d440cdd60 2024-04-25T10:05:21.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4415d830 con 0x7f7d4406e4d0 2024-04-25T10:05:21.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44168610 con 0x7f7d44145170 2024-04-25T10:05:21.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d440cdd60 0x7f7d440ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d440cdd60 0x7f7d440ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55498/0 (socket says 172.21.15.133:55498) 2024-04-25T10:05:21.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3bfff640 1 -- 172.21.15.133:0/19516593 learned_addr learned my addr 172.21.15.133:0/19516593 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:21.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3bfff640 1 -- 172.21.15.133:0/19516593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d44145170 msgr2=0x7f7d44153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3bfff640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d44145170 0x7f7d44153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3bfff640 1 -- 172.21.15.133:0/19516593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d4406e4d0 msgr2=0x7f7d440b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3bfff640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d4406e4d0 0x7f7d440b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3bfff640 1 -- 172.21.15.133:0/19516593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d440b46e0 con 0x7f7d440cdd60 2024-04-25T10:05:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3bfff640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d440cdd60 0x7f7d440ae030 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c01ab00 tx=0x7f7d2c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3b7fe640 1 -- 172.21.15.133:0/19516593 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0ba020 con 0x7f7d440cdd60 2024-04-25T10:05:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3b7fe640 1 -- 172.21.15.133:0/19516593 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c0b6640 con 0x7f7d440cdd60 2024-04-25T10:05:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 -- 172.21.15.133:0/19516593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d440acf90 con 0x7f7d440cdd60 2024-04-25T10:05:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d3b7fe640 1 -- 172.21.15.133:0/19516593 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0b0040 con 0x7f7d440cdd60 2024-04-25T10:05:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.570+0000 7f7d4902b640 1 -- 172.21.15.133:0/19516593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d440ad550 con 0x7f7d440cdd60 2024-04-25T10:05:21.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.574+0000 7f7d4902b640 1 -- 172.21.15.133:0/19516593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d44068e80 con 0x7f7d440cdd60 2024-04-25T10:05:21.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.574+0000 7f7d3b7fe640 1 -- 172.21.15.133:0/19516593 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d2c0bd020 con 0x7f7d440cdd60 2024-04-25T10:05:21.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.574+0000 7f7d3b7fe640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d24066f60 0x7f7d24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.574+0000 7f7d3b7fe640 1 -- 172.21.15.133:0/19516593 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f7d2c11f550 con 0x7f7d440cdd60 2024-04-25T10:05:21.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.574+0000 7f7d42d9d640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d24066f60 0x7f7d24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.574+0000 7f7d42d9d640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d24066f60 0x7f7d24069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7d440ae960 tx=0x7f7d34006c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.578+0000 7f7d3b7fe640 1 -- 172.21.15.133:0/19516593 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d2c0c4030 con 0x7f7d440cdd60 2024-04-25T10:05:21.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.722+0000 7f7d4902b640 1 -- 172.21.15.133:0/19516593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d4406e260 con 0x7f7d440cdd60 2024-04-25T10:05:21.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.722+0000 7f7d3b7fe640 1 -- 172.21.15.133:0/19516593 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d2c0f4e20 con 0x7f7d440cdd60 2024-04-25T10:05:21.732 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:05:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 -- 172.21.15.133:0/19516593 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d24066f60 msgr2=0x7f7d24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d24066f60 0x7f7d24069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7d440ae960 tx=0x7f7d34006c00 comp rx=0 tx=0).stop 2024-04-25T10:05:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 -- 172.21.15.133:0/19516593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d440cdd60 msgr2=0x7f7d440ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d440cdd60 0x7f7d440ae030 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c01ab00 tx=0x7f7d2c01ac90 comp rx=0 tx=0).stop 2024-04-25T10:05:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 -- 172.21.15.133:0/19516593 shutdown_connections 2024-04-25T10:05:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d44145170 0x7f7d44153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d24066f60 0x7f7d24069420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d440cdd60 0x7f7d440ae030 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 --2- 172.21.15.133:0/19516593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d4406e4d0 0x7f7d440b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 -- 172.21.15.133:0/19516593 >> 172.21.15.133:0/19516593 conn(0x7f7d44066f70 msgr2=0x7f7d4415d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 -- 172.21.15.133:0/19516593 shutdown_connections 2024-04-25T10:05:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.726+0000 7f7d4902b640 1 -- 172.21.15.133:0/19516593 wait complete. 2024-04-25T10:05:21.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:21.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f9025bc3640 1 Processor -- start 2024-04-25T10:05:21.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f9025bc3640 1 -- start start 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f9025bc3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f90200729b0 0x7f90200b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f9025bc3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f90200af4a0 0x7f90200cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f9025bc3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90200b43e0 con 0x7f902006dd70 2024-04-25T10:05:21.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f9025bc3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90200cd8f0 con 0x7f90200729b0 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f9025bc3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f902006c0e0 con 0x7f90200af4a0 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f901effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f90200729b0 0x7f90200b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f901effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f90200729b0 0x7f90200b46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55512/0 (socket says 172.21.15.133:55512) 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f901effd640 1 -- 172.21.15.133:0/2829943538 learned_addr learned my addr 172.21.15.133:0/2829943538 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f901ffff640 1 --2- 172.21.15.133:0/2829943538 >> v2:172.21.15.162:3300/0 conn(0x7f90200af4a0 0x7f90200cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f901effd640 1 -- 172.21.15.133:0/2829943538 >> v2:172.21.15.162:3300/0 conn(0x7f90200af4a0 msgr2=0x7f90200cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f901effd640 1 --2- 172.21.15.133:0/2829943538 >> v2:172.21.15.162:3300/0 conn(0x7f90200af4a0 0x7f90200cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f901effd640 1 -- 172.21.15.133:0/2829943538 >> v1:172.21.15.162:6789/0 conn(0x7f902006dd70 legacy=0x7f902006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.862+0000 7f901effd640 1 -- 172.21.15.133:0/2829943538 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902006c3c0 con 0x7f90200729b0 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f901effd640 1 --2- 172.21.15.133:0/2829943538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90200729b0 0x7f90200b46c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f902006d100 tx=0x7f900801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfdac54b084b58e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f901e7fc640 1 -- 172.21.15.133:0/2829943538 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90080b0070 con 0x7f90200729b0 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f901e7fc640 1 -- 172.21.15.133:0/2829943538 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90080a4e50 con 0x7f90200729b0 2024-04-25T10:05:21.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f901e7fc640 1 -- 172.21.15.133:0/2829943538 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90080ab820 con 0x7f90200729b0 2024-04-25T10:05:21.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 -- 172.21.15.133:0/2829943538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90200729b0 msgr2=0x7f90200b46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 --2- 172.21.15.133:0/2829943538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90200729b0 0x7f90200b46c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f902006d100 tx=0x7f900801c3c0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 -- 172.21.15.133:0/2829943538 shutdown_connections 2024-04-25T10:05:21.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 --2- 172.21.15.133:0/2829943538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90200729b0 0x7f90200b46c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 --2- 172.21.15.133:0/2829943538 >> v2:172.21.15.162:3300/0 conn(0x7f90200af4a0 0x7f90200cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 -- 172.21.15.133:0/2829943538 >> 172.21.15.133:0/2829943538 conn(0x7f90200cc750 msgr2=0x7f90200ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 -- 172.21.15.133:0/2829943538 shutdown_connections 2024-04-25T10:05:21.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 -- 172.21.15.133:0/2829943538 wait complete. 2024-04-25T10:05:21.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 Processor -- start 2024-04-25T10:05:21.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 -- start start 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f902006dd70 0x7f902015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90200729b0 0x7f90200aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90200af4a0 0x7f902014d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90200cf130 con 0x7f902006dd70 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90200cefb0 con 0x7f90200af4a0 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f9025bc3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902006ac90 con 0x7f90200729b0 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f901f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f902006dd70 0x7f902015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f901f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f902006dd70 0x7f902015aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55528/0 (socket says 172.21.15.133:55528) 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f901f7fe640 1 -- 172.21.15.133:0/3821231727 learned_addr learned my addr 172.21.15.133:0/3821231727 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f901ffff640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90200af4a0 0x7f902014d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.866+0000 7f901effd640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90200729b0 0x7f90200aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f901f7fe640 1 -- 172.21.15.133:0/3821231727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90200729b0 msgr2=0x7f90200aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f901f7fe640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90200729b0 0x7f90200aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f901f7fe640 1 -- 172.21.15.133:0/3821231727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90200af4a0 msgr2=0x7f902014d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f901f7fe640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90200af4a0 0x7f902014d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f901f7fe640 1 -- 172.21.15.133:0/3821231727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9020159970 con 0x7f902006dd70 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f901effd640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90200729b0 0x7f90200aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f901ffff640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90200af4a0 0x7f902014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f901f7fe640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f902006dd70 0x7f902015aa10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f901800c7c0 tx=0x7f901800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f9017fff640 1 -- 172.21.15.133:0/3821231727 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018016020 con 0x7f902006dd70 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f9017fff640 1 -- 172.21.15.133:0/3821231727 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901800a9f0 con 0x7f902006dd70 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f9025bc3640 1 -- 172.21.15.133:0/3821231727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9020159c50 con 0x7f902006dd70 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f9017fff640 1 -- 172.21.15.133:0/3821231727 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901800ae20 con 0x7f902006dd70 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f9025bc3640 1 -- 172.21.15.133:0/3821231727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902015a210 con 0x7f902006dd70 2024-04-25T10:05:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f9025bc3640 1 -- 172.21.15.133:0/3821231727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9020067c70 con 0x7f902006dd70 2024-04-25T10:05:21.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f9017fff640 1 -- 172.21.15.133:0/3821231727 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9018019020 con 0x7f902006dd70 2024-04-25T10:05:21.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f9017fff640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 0x7f9000069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f9017fff640 1 -- 172.21.15.133:0/3821231727 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f9018085500 con 0x7f902006dd70 2024-04-25T10:05:21.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f901effd640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 0x7f9000069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.870+0000 7f901effd640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 0x7f9000069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f90080ad000 tx=0x7f9008008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:21.874+0000 7f9017fff640 1 -- 172.21.15.133:0/3821231727 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9018020020 con 0x7f902006dd70 2024-04-25T10:05:22.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.034+0000 7f9025bc3640 1 -- 172.21.15.133:0/3821231727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9020167930 con 0x7f902006dd70 2024-04-25T10:05:22.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.034+0000 7f9017fff640 1 -- 172.21.15.133:0/3821231727 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f90180563e0 con 0x7f902006dd70 2024-04-25T10:05:22.043 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:05:22.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 -- 172.21.15.133:0/3821231727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 msgr2=0x7f9000069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 0x7f9000069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f90080ad000 tx=0x7f9008008040 comp rx=0 tx=0).stop 2024-04-25T10:05:22.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 -- 172.21.15.133:0/3821231727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f902006dd70 msgr2=0x7f902015aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f902006dd70 0x7f902015aa10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f901800c7c0 tx=0x7f901800cc80 comp rx=0 tx=0).stop 2024-04-25T10:05:22.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 -- 172.21.15.133:0/3821231727 shutdown_connections 2024-04-25T10:05:22.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f90200af4a0 0x7f902014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90200729b0 0x7f90200aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9000066f60 0x7f9000069420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 --2- 172.21.15.133:0/3821231727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f902006dd70 0x7f902015aa10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 -- 172.21.15.133:0/3821231727 >> 172.21.15.133:0/3821231727 conn(0x7f90200cc750 msgr2=0x7f90200b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 -- 172.21.15.133:0/3821231727 shutdown_connections 2024-04-25T10:05:22.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.038+0000 7f9025bc3640 1 -- 172.21.15.133:0/3821231727 wait complete. 2024-04-25T10:05:22.060 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:05:22.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:22.061 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:22.071 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[39538]: fuse finished with error 0 and tester_r 0 2024-04-25T10:05:22.074 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:05:22.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:22.075 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:05:22.243 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:05:22.244 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:22.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:22.250 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:05:22.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:22.251 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:05:22.298 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:05:22.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:22.299 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:05:22.299 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:22.305 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:22.305 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:05:22.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:22.306 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:05:22.353 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:05:22.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:22.354 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:05:22.354 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:22.360 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:22.360 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:05:22.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:22.360 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:05:22.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:22.407 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:22.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 Processor -- start 2024-04-25T10:05:22.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 -- start start 2024-04-25T10:05:22.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff8cc15c9a0 0x7ff8cc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8cc0b1f70 0x7ff8cc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8cc0c61d0 con 0x7ff8cc0ce330 2024-04-25T10:05:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8cc0c64b0 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc06d4f0 con 0x7ff8cc0b1f70 2024-04-25T10:05:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d0850640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff8cc15c9a0 0x7ff8cc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d0850640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff8cc15c9a0 0x7ff8cc0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55530/0 (socket says 172.21.15.133:55530) 2024-04-25T10:05:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d0850640 1 -- 172.21.15.133:0/3548997585 learned_addr learned my addr 172.21.15.133:0/3548997585 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8cbfff640 1 --2- 172.21.15.133:0/3548997585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8cc0b1f70 0x7ff8cc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d0850640 1 -- 172.21.15.133:0/3548997585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8cc0b1f70 msgr2=0x7ff8cc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d0850640 1 --2- 172.21.15.133:0/3548997585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8cc0b1f70 0x7ff8cc06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d0850640 1 -- 172.21.15.133:0/3548997585 >> v1:172.21.15.162:6789/0 conn(0x7ff8cc0ce330 legacy=0x7ff8cc15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d0850640 1 -- 172.21.15.133:0/3548997585 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc06d7d0 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d0850640 1 --2- 172.21.15.133:0/3548997585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc15c9a0 0x7ff8cc0c5980 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009870 tx=0x7ff8bc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af4b697e42eea04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8cb7fe640 1 -- 172.21.15.133:0/3548997585 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0ab610 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8cb7fe640 1 -- 172.21.15.133:0/3548997585 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc0b0070 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8cb7fe640 1 -- 172.21.15.133:0/3548997585 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0ab9d0 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3548997585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc15c9a0 msgr2=0x7ff8cc0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 --2- 172.21.15.133:0/3548997585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc15c9a0 0x7ff8cc0c5980 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009870 tx=0x7ff8bc01c500 comp rx=0 tx=0).stop 2024-04-25T10:05:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3548997585 shutdown_connections 2024-04-25T10:05:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 --2- 172.21.15.133:0/3548997585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8cc0b1f70 0x7ff8cc06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 --2- 172.21.15.133:0/3548997585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc15c9a0 0x7ff8cc0c5980 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3548997585 >> 172.21.15.133:0/3548997585 conn(0x7ff8cc067000 msgr2=0x7ff8cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3548997585 shutdown_connections 2024-04-25T10:05:22.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3548997585 wait complete. 2024-04-25T10:05:22.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 Processor -- start 2024-04-25T10:05:22.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.534+0000 7ff8d2ab4640 1 -- start start 2024-04-25T10:05:22.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d2ab4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8cc0b1f70 0x7ff8cc155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d2ab4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8cc0ce330 0x7ff8cc144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d2ab4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc15c9a0 0x7ff8cc15fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d2ab4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0ceb50 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d2ab4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc15cd80 con 0x7ff8cc0ce330 2024-04-25T10:05:22.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d2ab4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc154a30 con 0x7ff8cc0b1f70 2024-04-25T10:05:22.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d1051640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc15c9a0 0x7ff8cc15fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d1051640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc15c9a0 0x7ff8cc15fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55538/0 (socket says 172.21.15.133:55538) 2024-04-25T10:05:22.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d1051640 1 -- 172.21.15.133:0/3566752447 learned_addr learned my addr 172.21.15.133:0/3566752447 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:22.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8cbfff640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8cc0ce330 0x7ff8cc144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d0850640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8cc0b1f70 0x7ff8cc155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d1051640 1 -- 172.21.15.133:0/3566752447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8cc0b1f70 msgr2=0x7ff8cc155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d1051640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8cc0b1f70 0x7ff8cc155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d1051640 1 -- 172.21.15.133:0/3566752447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8cc0ce330 msgr2=0x7ff8cc144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d1051640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8cc0ce330 0x7ff8cc144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d1051640 1 -- 172.21.15.133:0/3566752447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc154d10 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8cbfff640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8cc0ce330 0x7ff8cc144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d1051640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc15c9a0 0x7ff8cc15fbd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400c730 tx=0x7ff8c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8c97fa640 1 -- 172.21.15.133:0/3566752447 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4003b80 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8c97fa640 1 -- 172.21.15.133:0/3566752447 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c4003d20 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3566752447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc154ff0 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8c97fa640 1 -- 172.21.15.133:0/3566752447 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c400aa30 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3566752447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc1555b0 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3566752447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8cc06bc20 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8c97fa640 1 -- 172.21.15.133:0/3566752447 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8c400abd0 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8c97fa640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8b8066f60 0x7ff8b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8c97fa640 1 -- 172.21.15.133:0/3566752447 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7ff8c4085050 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d0850640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8b8066f60 0x7ff8b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.538+0000 7ff8d0850640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8b8066f60 0x7ff8b8069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0af000 tx=0x7ff8bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.542+0000 7ff8c97fa640 1 -- 172.21.15.133:0/3566752447 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8c4055ea0 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.746+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3566752447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8cc068ef0 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.746+0000 7ff8c97fa640 1 -- 172.21.15.133:0/3566752447 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4947 (secure 0 0 0) 0x7ff8c40557a0 con 0x7ff8cc15c9a0 2024-04-25T10:05:22.754 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:22.754 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2887512873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2887512873},{"type":"v1","addr":"172.21.15.162:6833","nonce":2887512873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1044882076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1044882076},{"type":"v1","addr":"172.21.15.162:6835","nonce":1044882076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1965748675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1965748675},{"type":"v1","addr":"172.21.15.133:6836","nonce":1965748675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:03:17.773574+0000","modified":"2024-04-25T10:03:26.098972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/205640021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":205640021},{"type":"v1","addr":"172.21.15.133:6837","nonce":205640021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:05:22.755 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 17 2024-04-25T10:05:22.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3566752447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8b8066f60 msgr2=0x7ff8b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8b8066f60 0x7ff8b8069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0af000 tx=0x7ff8bc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:22.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3566752447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc15c9a0 msgr2=0x7ff8cc15fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc15c9a0 0x7ff8cc15fbd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400c730 tx=0x7ff8c400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3566752447 shutdown_connections 2024-04-25T10:05:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8b8066f60 0x7ff8b8069420 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc15c9a0 0x7ff8cc15fbd0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8cc0ce330 0x7ff8cc144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 --2- 172.21.15.133:0/3566752447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8cc0b1f70 0x7ff8cc155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3566752447 >> 172.21.15.133:0/3566752447 conn(0x7ff8cc067000 msgr2=0x7ff8cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3566752447 shutdown_connections 2024-04-25T10:05:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.750+0000 7ff8d2ab4640 1 -- 172.21.15.133:0/3566752447 wait complete. 2024-04-25T10:05:22.774 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:22.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.874+0000 7fec95b02640 1 Processor -- start 2024-04-25T10:05:22.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.874+0000 7fec95b02640 1 -- start start 2024-04-25T10:05:22.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.874+0000 7fec95b02640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 0x7fec9006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.874+0000 7fec95b02640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fec900ce330 0x7fec900ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.874+0000 7fec95b02640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec900c61d0 con 0x7fec9015c9a0 2024-04-25T10:05:22.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.874+0000 7fec95b02640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec900cece0 con 0x7fec900ce330 2024-04-25T10:05:22.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.874+0000 7fec95b02640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec900c64b0 con 0x7fec900b1f70 2024-04-25T10:05:22.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.878+0000 7fec8ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fec900ce330 0x7fec900ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.878+0000 7fec8f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fec9015c9a0 0x7fec900c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49782/0 (socket says 172.21.15.133:49782) 2024-04-25T10:05:22.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.878+0000 7fec8f7fe640 1 -- 172.21.15.133:0/3152714405 learned_addr learned my addr 172.21.15.133:0/3152714405 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:22.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.878+0000 7fec8effd640 1 --2- 172.21.15.133:0/3152714405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 0x7fec9006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.878+0000 7fec8ffff640 1 -- 172.21.15.133:0/3152714405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 msgr2=0x7fec9006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.878+0000 7fec8e7fc640 1 -- 172.21.15.133:0/3152714405 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606742229 0 0) 0x7fec900c61d0 con 0x7fec9015c9a0 2024-04-25T10:05:22.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.878+0000 7fec8ffff640 1 --2- 172.21.15.133:0/3152714405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 0x7fec9006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.878+0000 7fec8ffff640 1 -- 172.21.15.133:0/3152714405 >> v1:172.21.15.133:6789/0 conn(0x7fec9015c9a0 legacy=0x7fec900c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.878+0000 7fec8ffff640 1 -- 172.21.15.133:0/3152714405 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9006d4f0 con 0x7fec900ce330 2024-04-25T10:05:22.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.878+0000 7fec8effd640 1 --2- 172.21.15.133:0/3152714405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 0x7fec9006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:22.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec8ffff640 1 --2- 172.21.15.133:0/3152714405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec900ce330 0x7fec900ce7a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fec88009d90 tx=0x7fec8801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfd36cb2a3e64e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec8e7fc640 1 -- 172.21.15.133:0/3152714405 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880b2070 con 0x7fec900ce330 2024-04-25T10:05:22.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec8e7fc640 1 -- 172.21.15.133:0/3152714405 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec880a5910 con 0x7fec900ce330 2024-04-25T10:05:22.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec8e7fc640 1 -- 172.21.15.133:0/3152714405 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880a5ce0 con 0x7fec900ce330 2024-04-25T10:05:22.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 -- 172.21.15.133:0/3152714405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec900ce330 msgr2=0x7fec900ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 --2- 172.21.15.133:0/3152714405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec900ce330 0x7fec900ce7a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fec88009d90 tx=0x7fec8801c940 comp rx=0 tx=0).stop 2024-04-25T10:05:22.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 -- 172.21.15.133:0/3152714405 shutdown_connections 2024-04-25T10:05:22.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 --2- 172.21.15.133:0/3152714405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec900ce330 0x7fec900ce7a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 --2- 172.21.15.133:0/3152714405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 0x7fec9006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 -- 172.21.15.133:0/3152714405 >> 172.21.15.133:0/3152714405 conn(0x7fec90067000 msgr2=0x7fec9006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 -- 172.21.15.133:0/3152714405 shutdown_connections 2024-04-25T10:05:22.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 -- 172.21.15.133:0/3152714405 wait complete. 2024-04-25T10:05:22.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 Processor -- start 2024-04-25T10:05:22.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 -- start start 2024-04-25T10:05:22.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 0x7fec90157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec900ce330 0x7fec901466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec9015c9a0 0x7fec90159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9015dad0 con 0x7fec900ce330 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9015d950 con 0x7fec9015c9a0 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.882+0000 7fec95b02640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9015dc50 con 0x7fec900b1f70 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec900ce330 0x7fec901466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec900ce330 0x7fec901466b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55556/0 (socket says 172.21.15.133:55556) 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8effd640 1 -- 172.21.15.133:0/1003495346 learned_addr learned my addr 172.21.15.133:0/1003495346 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8f7fe640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 0x7fec90157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8ffff640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec9015c9a0 0x7fec90159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8effd640 1 -- 172.21.15.133:0/1003495346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 msgr2=0x7fec90157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8effd640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 0x7fec90157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8effd640 1 -- 172.21.15.133:0/1003495346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec9015c9a0 msgr2=0x7fec90159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8effd640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec9015c9a0 0x7fec90159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8effd640 1 -- 172.21.15.133:0/1003495346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9006b720 con 0x7fec900ce330 2024-04-25T10:05:22.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8f7fe640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 0x7fec90157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8ffff640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec9015c9a0 0x7fec90159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8effd640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec900ce330 0x7fec901466b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fec8000c7f0 tx=0x7fec8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec87fff640 1 -- 172.21.15.133:0/1003495346 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80015070 con 0x7fec900ce330 2024-04-25T10:05:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec87fff640 1 -- 172.21.15.133:0/1003495346 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec80007820 con 0x7fec900ce330 2024-04-25T10:05:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec95b02640 1 -- 172.21.15.133:0/1003495346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec90159b90 con 0x7fec900ce330 2024-04-25T10:05:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec87fff640 1 -- 172.21.15.133:0/1003495346 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80007c10 con 0x7fec900ce330 2024-04-25T10:05:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec95b02640 1 -- 172.21.15.133:0/1003495346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec901585b0 con 0x7fec900ce330 2024-04-25T10:05:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec95b02640 1 -- 172.21.15.133:0/1003495346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9006bc20 con 0x7fec900ce330 2024-04-25T10:05:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec87fff640 1 -- 172.21.15.133:0/1003495346 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec80018070 con 0x7fec900ce330 2024-04-25T10:05:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec87fff640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec70066f60 0x7fec70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec87fff640 1 -- 172.21.15.133:0/1003495346 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fec800850d0 con 0x7fec900ce330 2024-04-25T10:05:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8f7fe640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec70066f60 0x7fec70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.886+0000 7fec8f7fe640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec70066f60 0x7fec70069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fec78000f70 tx=0x7fec78007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:22.890+0000 7fec87fff640 1 -- 172.21.15.133:0/1003495346 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec90159b90 con 0x7fec900ce330 2024-04-25T10:05:23.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.038+0000 7fec95b02640 1 -- 172.21.15.133:0/1003495346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec90068ef0 con 0x7fec900ce330 2024-04-25T10:05:23.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.038+0000 7fec87fff640 1 -- 172.21.15.133:0/1003495346 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17080 (secure 0 0 0) 0x7fec80055f20 con 0x7fec900ce330 2024-04-25T10:05:23.048 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:23.048 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":42,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:04:56.820759+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:16.032957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:16.760653+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:23.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 -- 172.21.15.133:0/1003495346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec70066f60 msgr2=0x7fec70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec70066f60 0x7fec70069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fec78000f70 tx=0x7fec78007cb0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 -- 172.21.15.133:0/1003495346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec900ce330 msgr2=0x7fec901466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec900ce330 0x7fec901466b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fec8000c7f0 tx=0x7fec8000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 -- 172.21.15.133:0/1003495346 shutdown_connections 2024-04-25T10:05:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec9015c9a0 0x7fec90159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fec70066f60 0x7fec70069420 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fec78000f70 tx=0x7fec78007cb0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec900ce330 0x7fec901466b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 --2- 172.21.15.133:0/1003495346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec900b1f70 0x7fec90157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 -- 172.21.15.133:0/1003495346 >> 172.21.15.133:0/1003495346 conn(0x7fec90067000 msgr2=0x7fec9015b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 -- 172.21.15.133:0/1003495346 shutdown_connections 2024-04-25T10:05:23.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.042+0000 7fec95b02640 1 -- 172.21.15.133:0/1003495346 wait complete. 2024-04-25T10:05:23.065 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:16.032957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:16.760653+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 41, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:23.065 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:23.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa06a449640 1 Processor -- start 2024-04-25T10:05:23.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa06a449640 1 -- start start 2024-04-25T10:05:23.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa06a449640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa06406bca0 0x7fa06406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa06a449640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa06415bb00 0x7fa06415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa06a449640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa064072b40 con 0x7fa06406f3b0 2024-04-25T10:05:23.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa06a449640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa06406c5c0 con 0x7fa06415bb00 2024-04-25T10:05:23.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa06a449640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa064160160 con 0x7fa06406bca0 2024-04-25T10:05:23.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa0689e6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa06415bb00 0x7fa06415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa0637fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa06406f3b0 0x7fa0640af140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49788/0 (socket says 172.21.15.133:49788) 2024-04-25T10:05:23.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa0637fe640 1 -- 172.21.15.133:0/1523799501 learned_addr learned my addr 172.21.15.133:0/1523799501 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:23.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa063fff640 1 --2- 172.21.15.133:0/1523799501 >> v2:172.21.15.162:3300/0 conn(0x7fa06406bca0 0x7fa06406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa062ffd640 1 -- 172.21.15.133:0/1523799501 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593074618 0 0) 0x7fa064072b40 con 0x7fa06406f3b0 2024-04-25T10:05:23.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa0689e6640 1 -- 172.21.15.133:0/1523799501 >> v2:172.21.15.162:3300/0 conn(0x7fa06406bca0 msgr2=0x7fa06406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa0689e6640 1 --2- 172.21.15.133:0/1523799501 >> v2:172.21.15.162:3300/0 conn(0x7fa06406bca0 0x7fa06406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa0689e6640 1 -- 172.21.15.133:0/1523799501 >> v1:172.21.15.133:6789/0 conn(0x7fa06406f3b0 legacy=0x7fa0640af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa0689e6640 1 -- 172.21.15.133:0/1523799501 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064072580 con 0x7fa06415bb00 2024-04-25T10:05:23.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.178+0000 7fa063fff640 1 --2- 172.21.15.133:0/1523799501 >> v2:172.21.15.162:3300/0 conn(0x7fa06406bca0 0x7fa06406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa0689e6640 1 --2- 172.21.15.133:0/1523799501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa06415bb00 0x7fa06415bfb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa0640b5100 tx=0x7fa05c01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadd5b5849be848b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa062ffd640 1 -- 172.21.15.133:0/1523799501 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0a66e0 con 0x7fa06415bb00 2024-04-25T10:05:23.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa062ffd640 1 -- 172.21.15.133:0/1523799501 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c0a6880 con 0x7fa06415bb00 2024-04-25T10:05:23.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa062ffd640 1 -- 172.21.15.133:0/1523799501 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0a6c50 con 0x7fa06415bb00 2024-04-25T10:05:23.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 -- 172.21.15.133:0/1523799501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa06415bb00 msgr2=0x7fa06415bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 --2- 172.21.15.133:0/1523799501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa06415bb00 0x7fa06415bfb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa0640b5100 tx=0x7fa05c01c810 comp rx=0 tx=0).stop 2024-04-25T10:05:23.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 -- 172.21.15.133:0/1523799501 shutdown_connections 2024-04-25T10:05:23.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 --2- 172.21.15.133:0/1523799501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa06415bb00 0x7fa06415bfb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 --2- 172.21.15.133:0/1523799501 >> v2:172.21.15.162:3300/0 conn(0x7fa06406bca0 0x7fa06406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 -- 172.21.15.133:0/1523799501 >> 172.21.15.133:0/1523799501 conn(0x7fa064071850 msgr2=0x7fa064071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 -- 172.21.15.133:0/1523799501 shutdown_connections 2024-04-25T10:05:23.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 -- 172.21.15.133:0/1523799501 wait complete. 2024-04-25T10:05:23.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 Processor -- start 2024-04-25T10:05:23.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 -- start start 2024-04-25T10:05:23.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06406bca0 0x7fa064144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa06406f3b0 0x7fa0640ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa06415bb00 0x7fa064157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640adc20 con 0x7fa06406f3b0 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640adaa0 con 0x7fa06406bca0 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.182+0000 7fa06a449640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa064168740 con 0x7fa06415bb00 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa0637fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa06406f3b0 0x7fa0640ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa0637fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa06406f3b0 0x7fa0640ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55568/0 (socket says 172.21.15.133:55568) 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa0637fe640 1 -- 172.21.15.133:0/864416325 learned_addr learned my addr 172.21.15.133:0/864416325 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa063fff640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06406bca0 0x7fa064144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa0689e6640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa06415bb00 0x7fa064157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa0637fe640 1 -- 172.21.15.133:0/864416325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa06415bb00 msgr2=0x7fa064157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa0637fe640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa06415bb00 0x7fa064157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa0637fe640 1 -- 172.21.15.133:0/864416325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06406bca0 msgr2=0x7fa064144590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa0637fe640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06406bca0 0x7fa064144590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa0637fe640 1 -- 172.21.15.133:0/864416325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0640b2f80 con 0x7fa06406f3b0 2024-04-25T10:05:23.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa0689e6640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa06415bb00 0x7fa064157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa0637fe640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa06406f3b0 0x7fa0640ccfb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa05800c480 tx=0x7fa05800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa060ff9640 1 -- 172.21.15.133:0/864416325 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058018070 con 0x7fa06406f3b0 2024-04-25T10:05:23.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa060ff9640 1 -- 172.21.15.133:0/864416325 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0580074e0 con 0x7fa06406f3b0 2024-04-25T10:05:23.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa060ff9640 1 -- 172.21.15.133:0/864416325 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0580049f0 con 0x7fa06406f3b0 2024-04-25T10:05:23.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa06a449640 1 -- 172.21.15.133:0/864416325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0640b3260 con 0x7fa06406f3b0 2024-04-25T10:05:23.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa06a449640 1 -- 172.21.15.133:0/864416325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0640b3820 con 0x7fa06406f3b0 2024-04-25T10:05:23.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa06a449640 1 -- 172.21.15.133:0/864416325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa064067c70 con 0x7fa06406f3b0 2024-04-25T10:05:23.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa060ff9640 1 -- 172.21.15.133:0/864416325 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa058004b90 con 0x7fa06406f3b0 2024-04-25T10:05:23.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa060ff9640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa050066f60 0x7fa050069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa060ff9640 1 -- 172.21.15.133:0/864416325 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa0580844e0 con 0x7fa06406f3b0 2024-04-25T10:05:23.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa063fff640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa050066f60 0x7fa050069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.186+0000 7fa063fff640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa050066f60 0x7fa050069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa054006fd0 tx=0x7fa054008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.190+0000 7fa060ff9640 1 -- 172.21.15.133:0/864416325 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0580553c0 con 0x7fa06406f3b0 2024-04-25T10:05:23.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.350+0000 7fa06a449640 1 -- 172.21.15.133:0/864416325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa06406ee50 con 0x7fa06406f3b0 2024-04-25T10:05:23.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa060ff9640 1 -- 172.21.15.133:0/864416325 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa058054cc0 con 0x7fa06406f3b0 2024-04-25T10:05:23.360 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:05:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 -- 172.21.15.133:0/864416325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa050066f60 msgr2=0x7fa050069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa050066f60 0x7fa050069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa054006fd0 tx=0x7fa054008040 comp rx=0 tx=0).stop 2024-04-25T10:05:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 -- 172.21.15.133:0/864416325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa06406f3b0 msgr2=0x7fa0640ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa06406f3b0 0x7fa0640ccfb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa05800c480 tx=0x7fa05800c940 comp rx=0 tx=0).stop 2024-04-25T10:05:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 -- 172.21.15.133:0/864416325 shutdown_connections 2024-04-25T10:05:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa06415bb00 0x7fa064157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa050066f60 0x7fa050069420 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa06406f3b0 0x7fa0640ccfb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 --2- 172.21.15.133:0/864416325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa06406bca0 0x7fa064144590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 -- 172.21.15.133:0/864416325 >> 172.21.15.133:0/864416325 conn(0x7fa064071850 msgr2=0x7fa06415a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 -- 172.21.15.133:0/864416325 shutdown_connections 2024-04-25T10:05:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.354+0000 7fa06a449640 1 -- 172.21.15.133:0/864416325 wait complete. 2024-04-25T10:05:23.380 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:23.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c0a1fe640 1 Processor -- start 2024-04-25T10:05:23.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c0a1fe640 1 -- start start 2024-04-25T10:05:23.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c0a1fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040ce600 0x7f5c040cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c0a1fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5c040d1920 0x7f5c040d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c0a1fe640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0415ebd0 con 0x7f5c0415d9a0 2024-04-25T10:05:23.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c0a1fe640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c040cefa0 con 0x7f5c040d1920 2024-04-25T10:05:23.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c0a1fe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0415eeb0 con 0x7f5c040ce600 2024-04-25T10:05:23.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c037fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5c0415d9a0 0x7f5c040d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49794/0 (socket says 172.21.15.133:49794) 2024-04-25T10:05:23.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c037fe640 1 -- 172.21.15.133:0/1921736322 learned_addr learned my addr 172.21.15.133:0/1921736322 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:23.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c03fff640 1 --2- 172.21.15.133:0/1921736322 >> v2:172.21.15.162:3300/0 conn(0x7f5c040d1920 0x7f5c040d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c02ffd640 1 --2- 172.21.15.133:0/1921736322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040ce600 0x7f5c040cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773110436 0 0) 0x7f5c0415ebd0 con 0x7f5c0415d9a0 2024-04-25T10:05:23.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bec003530 con 0x7f5c0415d9a0 2024-04-25T10:05:23.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3996819295 0 0) 0x7f5bec003530 con 0x7f5c0415d9a0 2024-04-25T10:05:23.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040ce600 msgr2=0x7f5c040cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 --2- 172.21.15.133:0/1921736322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040ce600 0x7f5c040cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 >> v2:172.21.15.162:3300/0 conn(0x7f5c040d1920 msgr2=0x7f5c040d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 --2- 172.21.15.133:0/1921736322 >> v2:172.21.15.162:3300/0 conn(0x7f5c040d1920 0x7f5c040d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c040d0330 con 0x7f5c0415d9a0 2024-04-25T10:05:23.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5bf4001680 con 0x7f5c0415d9a0 2024-04-25T10:05:23.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 >> v1:172.21.15.133:6789/0 conn(0x7f5c0415d9a0 legacy=0x7f5c040d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 --2- 172.21.15.133:0/1921736322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bec0039c0 0x7f5bec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 --2- 172.21.15.133:0/1921736322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040ce600 0x7f5bec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 --2- 172.21.15.133:0/1921736322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bec004d50 0x7f5bec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0415eeb0 con 0x7f5bec0039c0 2024-04-25T10:05:23.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec00b9d0 con 0x7f5bec004d50 2024-04-25T10:05:23.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec00bb50 con 0x7f5c040ce600 2024-04-25T10:05:23.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c037fe640 1 --2- 172.21.15.133:0/1921736322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bec0039c0 0x7f5bec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c02ffd640 1 --2- 172.21.15.133:0/1921736322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040ce600 0x7f5bec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c037fe640 1 -- 172.21.15.133:0/1921736322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040ce600 msgr2=0x7f5bec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c037fe640 1 --2- 172.21.15.133:0/1921736322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040ce600 0x7f5bec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c037fe640 1 -- 172.21.15.133:0/1921736322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bec004d50 msgr2=0x7f5bec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c037fe640 1 --2- 172.21.15.133:0/1921736322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bec004d50 0x7f5bec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.494+0000 7f5c037fe640 1 -- 172.21.15.133:0/1921736322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec003530 con 0x7f5bec0039c0 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c037fe640 1 --2- 172.21.15.133:0/1921736322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bec0039c0 0x7f5bec003de0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400c240 tx=0x7f5bf401cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65071cdfd7d97ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4007d40 con 0x7f5bec0039c0 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf40a9020 con 0x7f5bec0039c0 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c027fc640 1 -- 172.21.15.133:0/1921736322 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4005b00 con 0x7f5bec0039c0 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/1921736322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bec0039c0 msgr2=0x7f5bec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 --2- 172.21.15.133:0/1921736322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bec0039c0 0x7f5bec003de0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400c240 tx=0x7f5bf401cc70 comp rx=0 tx=0).stop 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c03fff640 1 -- 172.21.15.133:0/1921736322 reap_dead start 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/1921736322 shutdown_connections 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/1921736322 >> 172.21.15.133:0/1921736322 conn(0x7f5c0406bf80 msgr2=0x7f5c0406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/1921736322 shutdown_connections 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/1921736322 wait complete. 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 Processor -- start 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 -- start start 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bec0039c0 0x7f5c04157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c040d25f0 0x7f5c040d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040b7bc0 0x7f5c040b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c040d2470 con 0x7f5c040d25f0 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c040d22f0 con 0x7f5bec0039c0 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c0a1fe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0415e590 con 0x7f5c040b7bc0 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c02ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c040d25f0 0x7f5c040d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c02ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c040d25f0 0x7f5c040d2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55576/0 (socket says 172.21.15.133:55576) 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.498+0000 7f5c02ffd640 1 -- 172.21.15.133:0/2383619785 learned_addr learned my addr 172.21.15.133:0/2383619785 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c03fff640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040b7bc0 0x7f5c040b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c037fe640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bec0039c0 0x7f5c04157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c02ffd640 1 -- 172.21.15.133:0/2383619785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040b7bc0 msgr2=0x7f5c040b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c02ffd640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040b7bc0 0x7f5c040b8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c02ffd640 1 -- 172.21.15.133:0/2383619785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bec0039c0 msgr2=0x7f5c04157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c02ffd640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bec0039c0 0x7f5c04157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c02ffd640 1 -- 172.21.15.133:0/2383619785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c040b7810 con 0x7f5c040d25f0 2024-04-25T10:05:23.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c037fe640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bec0039c0 0x7f5c04157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c02ffd640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c040d25f0 0x7f5c040d2a70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8011560 tx=0x7f5bf8011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5bebfff640 1 -- 172.21.15.133:0/2383619785 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8015930 con 0x7f5c040d25f0 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5bebfff640 1 -- 172.21.15.133:0/2383619785 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8015ad0 con 0x7f5c040d25f0 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/2383619785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c040b8980 con 0x7f5c040d25f0 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5bebfff640 1 -- 172.21.15.133:0/2383619785 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf800f460 con 0x7f5c040d25f0 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/2383619785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c040b6d30 con 0x7f5c040d25f0 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/2383619785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c04068e80 con 0x7f5c040d25f0 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5bebfff640 1 -- 172.21.15.133:0/2383619785 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bf800f600 con 0x7f5c040d25f0 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5bebfff640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5bf0066f90 0x7f5bf0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c037fe640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5bf0066f90 0x7f5bf0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5bebfff640 1 -- 172.21.15.133:0/2383619785 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f5bf8085ab0 con 0x7f5c040d25f0 2024-04-25T10:05:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.502+0000 7f5c037fe640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5bf0066f90 0x7f5bf0069450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40aa000 tx=0x7f5c0415eda0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.506+0000 7f5bebfff640 1 -- 172.21.15.133:0/2383619785 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bf8019030 con 0x7f5c040d25f0 2024-04-25T10:05:23.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.654+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/2383619785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c040b5a60 con 0x7f5c040d25f0 2024-04-25T10:05:23.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.654+0000 7f5bebfff640 1 -- 172.21.15.133:0/2383619785 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5bf80568f0 con 0x7f5c040d25f0 2024-04-25T10:05:23.663 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:05:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/2383619785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5bf0066f90 msgr2=0x7f5bf0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5bf0066f90 0x7f5bf0069450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40aa000 tx=0x7f5c0415eda0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/2383619785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c040d25f0 msgr2=0x7f5c040d2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c040d25f0 0x7f5c040d2a70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8011560 tx=0x7f5bf8011a20 comp rx=0 tx=0).stop 2024-04-25T10:05:23.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/2383619785 shutdown_connections 2024-04-25T10:05:23.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c040b7bc0 0x7f5c040b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5bf0066f90 0x7f5bf0069450 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c040d25f0 0x7f5c040d2a70 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 --2- 172.21.15.133:0/2383619785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bec0039c0 0x7f5c04157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/2383619785 >> 172.21.15.133:0/2383619785 conn(0x7f5c0406bf80 msgr2=0x7f5c0415c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/2383619785 shutdown_connections 2024-04-25T10:05:23.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.658+0000 7f5c0a1fe640 1 -- 172.21.15.133:0/2383619785 wait complete. 2024-04-25T10:05:23.681 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:23.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbdad1d640 1 Processor -- start 2024-04-25T10:05:23.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbdad1d640 1 -- start start 2024-04-25T10:05:23.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbdad1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd4145170 0x7ffbd406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbdad1d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffbd406e4d0 0x7ffbd40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbdad1d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd40ce2e0 con 0x7ffbd40cdd60 2024-04-25T10:05:23.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbdad1d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd406db00 con 0x7ffbd406e4d0 2024-04-25T10:05:23.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbdad1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4145550 con 0x7ffbd4145170 2024-04-25T10:05:23.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd92ba640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ffbd40cdd60 0x7ffbd415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49800/0 (socket says 172.21.15.133:49800) 2024-04-25T10:05:23.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd92ba640 1 -- 172.21.15.133:0/812788048 learned_addr learned my addr 172.21.15.133:0/812788048 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:23.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246500053 0 0) 0x7ffbd40ce2e0 con 0x7ffbd40cdd60 2024-04-25T10:05:23.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbbc003530 con 0x7ffbd40cdd60 2024-04-25T10:05:23.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd3fff640 1 --2- 172.21.15.133:0/812788048 >> v2:172.21.15.162:3300/0 conn(0x7ffbd406e4d0 0x7ffbd40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd8ab9640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd4145170 0x7ffbd406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2780704771 0 0) 0x7ffbbc003530 con 0x7ffbd40cdd60 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd4145170 msgr2=0x7ffbd406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd4145170 0x7ffbd406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 >> v2:172.21.15.162:3300/0 conn(0x7ffbd406e4d0 msgr2=0x7ffbd40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 --2- 172.21.15.133:0/812788048 >> v2:172.21.15.162:3300/0 conn(0x7ffbd406e4d0 0x7ffbd40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd406d4b0 con 0x7ffbd40cdd60 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ffbc4001a00 con 0x7ffbd40cdd60 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd3fff640 1 --2- 172.21.15.133:0/812788048 >> v2:172.21.15.162:3300/0 conn(0x7ffbd406e4d0 0x7ffbd40cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 >> v1:172.21.15.133:6789/0 conn(0x7ffbd40cdd60 legacy=0x7ffbd415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 0x7ffbbc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd4145170 0x7ffbbc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbbc004d50 0x7ffbbc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd406db00 con 0x7ffbbc0039c0 2024-04-25T10:05:23.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4145550 con 0x7ffbbc004d50 2024-04-25T10:05:23.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc00ba80 con 0x7ffbd4145170 2024-04-25T10:05:23.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd8ab9640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 0x7ffbbc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd3fff640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd4145170 0x7ffbbc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd92ba640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbbc004d50 0x7ffbbc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd8ab9640 1 -- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd4145170 msgr2=0x7ffbbc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd8ab9640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd4145170 0x7ffbbc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd8ab9640 1 -- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbbc004d50 msgr2=0x7ffbbc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd8ab9640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbbc004d50 0x7ffbbc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd8ab9640 1 -- 172.21.15.133:0/812788048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc003530 con 0x7ffbbc0039c0 2024-04-25T10:05:23.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.786+0000 7ffbd92ba640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbbc004d50 0x7ffbbc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbd8ab9640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 0x7ffbbc003de0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00ba40 tx=0x7ffbcc01c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf4199c27ad9dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b3070 con 0x7ffbbc0039c0 2024-04-25T10:05:23.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc0a9070 con 0x7ffbbc0039c0 2024-04-25T10:05:23.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbd37fe640 1 -- 172.21.15.133:0/812788048 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00b5e0 con 0x7ffbbc0039c0 2024-04-25T10:05:23.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 -- 172.21.15.133:0/812788048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 msgr2=0x7ffbbc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 --2- 172.21.15.133:0/812788048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 0x7ffbbc003de0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00ba40 tx=0x7ffbcc01c660 comp rx=0 tx=0).stop 2024-04-25T10:05:23.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbd92ba640 1 -- 172.21.15.133:0/812788048 reap_dead start 2024-04-25T10:05:23.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 -- 172.21.15.133:0/812788048 shutdown_connections 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 -- 172.21.15.133:0/812788048 >> 172.21.15.133:0/812788048 conn(0x7ffbd4066f70 msgr2=0x7ffbd406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 -- 172.21.15.133:0/812788048 shutdown_connections 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 -- 172.21.15.133:0/812788048 wait complete. 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 Processor -- start 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 -- start start 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 0x7ffbd413edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd41f6900 0x7ffbd4162e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd41f9660 0x7ffbd41646d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4166480 con 0x7ffbbc0039c0 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd415d7f0 con 0x7ffbd41f6900 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbdad1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd4166600 con 0x7ffbd41f9660 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbd8ab9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 0x7ffbd413edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbd8ab9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 0x7ffbd413edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55584/0 (socket says 172.21.15.133:55584) 2024-04-25T10:05:23.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.790+0000 7ffbd8ab9640 1 -- 172.21.15.133:0/1328210412 learned_addr learned my addr 172.21.15.133:0/1328210412 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd3fff640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd41f6900 0x7ffbd4162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd8ab9640 1 -- 172.21.15.133:0/1328210412 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd41f9660 msgr2=0x7ffbd41646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd8ab9640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd41f9660 0x7ffbd41646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd8ab9640 1 -- 172.21.15.133:0/1328210412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd41f6900 msgr2=0x7ffbd4162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd8ab9640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd41f6900 0x7ffbd4162e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd8ab9640 1 -- 172.21.15.133:0/1328210412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd41633b0 con 0x7ffbbc0039c0 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd3fff640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd41f6900 0x7ffbd4162e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd8ab9640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 0x7ffbd413edc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00ba40 tx=0x7ffbbc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd17fa640 1 -- 172.21.15.133:0/1328210412 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b3040 con 0x7ffbbc0039c0 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd17fa640 1 -- 172.21.15.133:0/1328210412 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc0aec90 con 0x7ffbbc0039c0 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbdad1d640 1 -- 172.21.15.133:0/1328210412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd4163630 con 0x7ffbbc0039c0 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd17fa640 1 -- 172.21.15.133:0/1328210412 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b8660 con 0x7ffbbc0039c0 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbdad1d640 1 -- 172.21.15.133:0/1328210412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd4163bf0 con 0x7ffbbc0039c0 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbdad1d640 1 -- 172.21.15.133:0/1328210412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd4068e80 con 0x7ffbbc0039c0 2024-04-25T10:05:23.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd17fa640 1 -- 172.21.15.133:0/1328210412 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffbcc0b6070 con 0x7ffbbc0039c0 2024-04-25T10:05:23.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd17fa640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbc0066f60 0x7ffbc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd3fff640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbc0066f60 0x7ffbc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd17fa640 1 -- 172.21.15.133:0/1328210412 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7ffbcc11fa10 con 0x7ffbbc0039c0 2024-04-25T10:05:23.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.794+0000 7ffbd3fff640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbc0066f60 0x7ffbc0069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40b05a0 tx=0x7ffbc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.798+0000 7ffbd17fa640 1 -- 172.21.15.133:0/1328210412 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffbd41633b0 con 0x7ffbbc0039c0 2024-04-25T10:05:23.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 -- 172.21.15.133:0/1328210412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffbd40b0aa0 con 0x7ffbbc0039c0 2024-04-25T10:05:23.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbd17fa640 1 -- 172.21.15.133:0/1328210412 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffbcc0bd020 con 0x7ffbbc0039c0 2024-04-25T10:05:23.953 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:05:23.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 -- 172.21.15.133:0/1328210412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbc0066f60 msgr2=0x7ffbc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbc0066f60 0x7ffbc0069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40b05a0 tx=0x7ffbc8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:23.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 -- 172.21.15.133:0/1328210412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 msgr2=0x7ffbd413edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 0x7ffbd413edc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00ba40 tx=0x7ffbbc005840 comp rx=0 tx=0).stop 2024-04-25T10:05:23.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 -- 172.21.15.133:0/1328210412 shutdown_connections 2024-04-25T10:05:23.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbd41f9660 0x7ffbd41646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbd41f6900 0x7ffbd4162e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbc0066f60 0x7ffbc0069420 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 --2- 172.21.15.133:0/1328210412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbbc0039c0 0x7ffbd413edc0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 -- 172.21.15.133:0/1328210412 >> 172.21.15.133:0/1328210412 conn(0x7ffbd4066f70 msgr2=0x7ffbd4144670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 -- 172.21.15.133:0/1328210412 shutdown_connections 2024-04-25T10:05:23.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:23.946+0000 7ffbdad1d640 1 -- 172.21.15.133:0/1328210412 wait complete. 2024-04-25T10:05:23.971 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:24.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbcd821640 1 Processor -- start 2024-04-25T10:05:24.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbcd821640 1 -- start start 2024-04-25T10:05:24.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbcd821640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffbc81591d0 0x7ffbc806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbcd821640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 0x7ffbc815d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbcd821640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc815cb60 con 0x7ffbc815c6f0 2024-04-25T10:05:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbcd821640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc806d9e0 con 0x7ffbc81591d0 2024-04-25T10:05:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbcd821640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc806e150 con 0x7ffbc806c3b0 2024-04-25T10:05:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbc6ffd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffbc81591d0 0x7ffbc806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbc77fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 0x7ffbc815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbc6ffd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffbc81591d0 0x7ffbc806fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:46458/0 (socket says 172.21.15.133:46458) 2024-04-25T10:05:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbc6ffd640 1 -- 172.21.15.133:0/4239398943 learned_addr learned my addr 172.21.15.133:0/4239398943 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbc67fc640 1 --1- 172.21.15.133:0/4239398943 >> v1:172.21.15.162:6789/0 conn(0x7ffbc815c6f0 0x7ffbc806bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:53452/0 (socket says 172.21.15.133:53452) 2024-04-25T10:05:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbc77fe640 1 -- 172.21.15.133:0/4239398943 >> v2:172.21.15.162:3300/0 conn(0x7ffbc81591d0 msgr2=0x7ffbc806fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbc77fe640 1 --2- 172.21.15.133:0/4239398943 >> v2:172.21.15.162:3300/0 conn(0x7ffbc81591d0 0x7ffbc806fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbc77fe640 1 -- 172.21.15.133:0/4239398943 >> v1:172.21.15.162:6789/0 conn(0x7ffbc815c6f0 legacy=0x7ffbc806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbc77fe640 1 -- 172.21.15.133:0/4239398943 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc806e410 con 0x7ffbc806c3b0 2024-04-25T10:05:24.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.074+0000 7ffbc6ffd640 1 --2- 172.21.15.133:0/4239398943 >> v2:172.21.15.162:3300/0 conn(0x7ffbc81591d0 0x7ffbc806fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:24.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbc77fe640 1 --2- 172.21.15.133:0/4239398943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 0x7ffbc815d560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009d90 tx=0x7ffbc001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d1a1a84451edde6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbc5ffb640 1 -- 172.21.15.133:0/4239398943 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00b2070 con 0x7ffbc806c3b0 2024-04-25T10:05:24.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbc5ffb640 1 -- 172.21.15.133:0/4239398943 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc00a5830 con 0x7ffbc806c3b0 2024-04-25T10:05:24.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbc5ffb640 1 -- 172.21.15.133:0/4239398943 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00a5c00 con 0x7ffbc806c3b0 2024-04-25T10:05:24.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbcd821640 1 -- 172.21.15.133:0/4239398943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 msgr2=0x7ffbc815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbcd821640 1 --2- 172.21.15.133:0/4239398943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 0x7ffbc815d560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009d90 tx=0x7ffbc001c970 comp rx=0 tx=0).stop 2024-04-25T10:05:24.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbcd821640 1 -- 172.21.15.133:0/4239398943 shutdown_connections 2024-04-25T10:05:24.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbcd821640 1 --2- 172.21.15.133:0/4239398943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 0x7ffbc815d560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbcd821640 1 --2- 172.21.15.133:0/4239398943 >> v2:172.21.15.162:3300/0 conn(0x7ffbc81591d0 0x7ffbc806fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbcd821640 1 -- 172.21.15.133:0/4239398943 >> 172.21.15.133:0/4239398943 conn(0x7ffbc8071aa0 msgr2=0x7ffbc8071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbcd821640 1 -- 172.21.15.133:0/4239398943 shutdown_connections 2024-04-25T10:05:24.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbcd821640 1 -- 172.21.15.133:0/4239398943 wait complete. 2024-04-25T10:05:24.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbcd821640 1 Processor -- start 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.078+0000 7ffbcd821640 1 -- start start 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbcd821640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 0x7ffbc81588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbcd821640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbc81591d0 0x7ffbc80ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbcd821640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc815c6f0 0x7ffbc81502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbcd821640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc815e430 con 0x7ffbc815c6f0 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbcd821640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc815e2b0 con 0x7ffbc81591d0 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbcd821640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc815e5b0 con 0x7ffbc806c3b0 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc815c6f0 0x7ffbc81502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc815c6f0 0x7ffbc81502e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55592/0 (socket says 172.21.15.133:55592) 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc77fe640 1 -- 172.21.15.133:0/368367408 learned_addr learned my addr 172.21.15.133:0/368367408 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc67fc640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbc81591d0 0x7ffbc80ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc6ffd640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 0x7ffbc81588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc77fe640 1 -- 172.21.15.133:0/368367408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 msgr2=0x7ffbc81588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc77fe640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 0x7ffbc81588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc77fe640 1 -- 172.21.15.133:0/368367408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbc81591d0 msgr2=0x7ffbc80ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc77fe640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbc81591d0 0x7ffbc80ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc77fe640 1 -- 172.21.15.133:0/368367408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8150e30 con 0x7ffbc815c6f0 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc6ffd640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 0x7ffbc81588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc77fe640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc815c6f0 0x7ffbc81502e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00b0040 tx=0x7ffbc0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbaffff640 1 -- 172.21.15.133:0/368367408 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00b2050 con 0x7ffbc815c6f0 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbaffff640 1 -- 172.21.15.133:0/368367408 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc00be070 con 0x7ffbc815c6f0 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbcd821640 1 -- 172.21.15.133:0/368367408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc80cd550 con 0x7ffbc815c6f0 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbaffff640 1 -- 172.21.15.133:0/368367408 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00ada90 con 0x7ffbc815c6f0 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbcd821640 1 -- 172.21.15.133:0/368367408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc80cdb10 con 0x7ffbc815c6f0 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbcd821640 1 -- 172.21.15.133:0/368367408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc8067c70 con 0x7ffbc815c6f0 2024-04-25T10:05:24.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbaffff640 1 -- 172.21.15.133:0/368367408 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffbc00a56b0 con 0x7ffbc815c6f0 2024-04-25T10:05:24.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbaffff640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbb4066f60 0x7ffbb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbaffff640 1 -- 172.21.15.133:0/368367408 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7ffbc01202d0 con 0x7ffbc815c6f0 2024-04-25T10:05:24.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc6ffd640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbb4066f60 0x7ffbb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.082+0000 7ffbc6ffd640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbb4066f60 0x7ffbb4069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffbc815eb70 tx=0x7ffbb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.086+0000 7ffbaffff640 1 -- 172.21.15.133:0/368367408 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffbc0016560 con 0x7ffbc815c6f0 2024-04-25T10:05:24.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.282+0000 7ffbcd821640 1 -- 172.21.15.133:0/368367408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbc8072ca0 con 0x7ffbc815c6f0 2024-04-25T10:05:24.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.282+0000 7ffbaffff640 1 -- 172.21.15.133:0/368367408 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4947 (secure 0 0 0) 0x7ffbc00ab340 con 0x7ffbc815c6f0 2024-04-25T10:05:24.291 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:24.291 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2887512873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2887512873},{"type":"v1","addr":"172.21.15.162:6833","nonce":2887512873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1044882076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1044882076},{"type":"v1","addr":"172.21.15.162:6835","nonce":1044882076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1965748675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1965748675},{"type":"v1","addr":"172.21.15.133:6836","nonce":1965748675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:03:17.773574+0000","modified":"2024-04-25T10:03:26.098972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/205640021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":205640021},{"type":"v1","addr":"172.21.15.133:6837","nonce":205640021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:05:24.291 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 17 2024-04-25T10:05:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 -- 172.21.15.133:0/368367408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbb4066f60 msgr2=0x7ffbb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbb4066f60 0x7ffbb4069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffbc815eb70 tx=0x7ffbb8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 -- 172.21.15.133:0/368367408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc815c6f0 msgr2=0x7ffbc81502e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc815c6f0 0x7ffbc81502e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00b0040 tx=0x7ffbc0019710 comp rx=0 tx=0).stop 2024-04-25T10:05:24.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 -- 172.21.15.133:0/368367408 shutdown_connections 2024-04-25T10:05:24.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffbb4066f60 0x7ffbb4069420 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc815c6f0 0x7ffbc81502e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffbc81591d0 0x7ffbc80ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 --2- 172.21.15.133:0/368367408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffbc806c3b0 0x7ffbc81588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 -- 172.21.15.133:0/368367408 >> 172.21.15.133:0/368367408 conn(0x7ffbc8071aa0 msgr2=0x7ffbc815abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 -- 172.21.15.133:0/368367408 shutdown_connections 2024-04-25T10:05:24.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.286+0000 7ffbcd821640 1 -- 172.21.15.133:0/368367408 wait complete. 2024-04-25T10:05:24.309 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:24.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc051652640 1 Processor -- start 2024-04-25T10:05:24.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc051652640 1 -- start start 2024-04-25T10:05:24.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc051652640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc04c15a1a0 0x7fc04c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc051652640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04c15e460 con 0x7fc04c15e050 2024-04-25T10:05:24.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc051652640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04c0ce880 con 0x7fc04c159dc0 2024-04-25T10:05:24.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc051652640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04c0c49e0 con 0x7fc04c15a1a0 2024-04-25T10:05:24.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc0427fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc04c15a1a0 0x7fc04c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc04b7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc04c15e050 0x7fc04c06d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49804/0 (socket says 172.21.15.133:49804) 2024-04-25T10:05:24.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc04b7fe640 1 -- 172.21.15.133:0/1169413794 learned_addr learned my addr 172.21.15.133:0/1169413794 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:24.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc0427fc640 1 -- 172.21.15.133:0/1169413794 >> v1:172.21.15.162:6789/0 conn(0x7fc04c159dc0 legacy=0x7fc04c0cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc0427fc640 1 -- 172.21.15.133:0/1169413794 >> v1:172.21.15.133:6789/0 conn(0x7fc04c15e050 legacy=0x7fc04c06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc0427fc640 1 -- 172.21.15.133:0/1169413794 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c0ceb20 con 0x7fc04c15a1a0 2024-04-25T10:05:24.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc04a7fc640 1 -- 172.21.15.133:0/1169413794 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830511803 0 0) 0x7fc0440014a0 con 0x7fc04c15e050 2024-04-25T10:05:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc0427fc640 1 --2- 172.21.15.133:0/1169413794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc04c15a1a0 0x7fc04c15db10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc0340097c0 tx=0x7fc03401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b143a186d9d918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc04a7fc640 1 -- 172.21.15.133:0/1169413794 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340b0070 con 0x7fc04c15a1a0 2024-04-25T10:05:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc04a7fc640 1 -- 172.21.15.133:0/1169413794 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0340ab600 con 0x7fc04c15a1a0 2024-04-25T10:05:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.410+0000 7fc04a7fc640 1 -- 172.21.15.133:0/1169413794 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340ab9d0 con 0x7fc04c15a1a0 2024-04-25T10:05:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- 172.21.15.133:0/1169413794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc04c15a1a0 msgr2=0x7fc04c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 --2- 172.21.15.133:0/1169413794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc04c15a1a0 0x7fc04c15db10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc0340097c0 tx=0x7fc03401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- 172.21.15.133:0/1169413794 shutdown_connections 2024-04-25T10:05:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 --2- 172.21.15.133:0/1169413794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc04c15a1a0 0x7fc04c15db10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- 172.21.15.133:0/1169413794 >> 172.21.15.133:0/1169413794 conn(0x7fc04c067000 msgr2=0x7fc04c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- 172.21.15.133:0/1169413794 shutdown_connections 2024-04-25T10:05:24.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- 172.21.15.133:0/1169413794 wait complete. 2024-04-25T10:05:24.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 Processor -- start 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- start start 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc04c159dc0 0x7fc04c14bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc04c15e050 0x7fc04c156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc04c14cd20 0x7fc04c156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c06b520 con 0x7fc04c159dc0 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c06b3a0 con 0x7fc04c14cd20 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c06b6a0 con 0x7fc04c15e050 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc04affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc04c159dc0 0x7fc04c14bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc04affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc04c159dc0 0x7fc04c14bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55606/0 (socket says 172.21.15.133:55606) 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc04affd640 1 -- 172.21.15.133:0/607797300 learned_addr learned my addr 172.21.15.133:0/607797300 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc04b7fe640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc04c14cd20 0x7fc04c156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc0427fc640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc04c15e050 0x7fc04c156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc04affd640 1 -- 172.21.15.133:0/607797300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc04c15e050 msgr2=0x7fc04c156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc04affd640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc04c15e050 0x7fc04c156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc04affd640 1 -- 172.21.15.133:0/607797300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc04c14cd20 msgr2=0x7fc04c156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc04affd640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc04c14cd20 0x7fc04c156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc04affd640 1 -- 172.21.15.133:0/607797300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c14c4c0 con 0x7fc04c159dc0 2024-04-25T10:05:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc04affd640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc04c159dc0 0x7fc04c14bf80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00c530 tx=0x7fc03c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc043fff640 1 -- 172.21.15.133:0/607797300 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c016020 con 0x7fc04c159dc0 2024-04-25T10:05:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc043fff640 1 -- 172.21.15.133:0/607797300 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c005220 con 0x7fc04c159dc0 2024-04-25T10:05:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- 172.21.15.133:0/607797300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04c150870 con 0x7fc04c159dc0 2024-04-25T10:05:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc043fff640 1 -- 172.21.15.133:0/607797300 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c007030 con 0x7fc04c159dc0 2024-04-25T10:05:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- 172.21.15.133:0/607797300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04c154420 con 0x7fc04c159dc0 2024-04-25T10:05:24.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.414+0000 7fc051652640 1 -- 172.21.15.133:0/607797300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04c068ef0 con 0x7fc04c159dc0 2024-04-25T10:05:24.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.418+0000 7fc043fff640 1 -- 172.21.15.133:0/607797300 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc03c0071d0 con 0x7fc04c159dc0 2024-04-25T10:05:24.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.418+0000 7fc043fff640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc02c066f60 0x7fc02c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.418+0000 7fc0427fc640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc02c066f60 0x7fc02c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.418+0000 7fc043fff640 1 -- 172.21.15.133:0/607797300 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fc03c084b40 con 0x7fc04c159dc0 2024-04-25T10:05:24.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.418+0000 7fc0427fc640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc02c066f60 0x7fc02c069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc0340ad000 tx=0x7fc0340a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.422+0000 7fc043fff640 1 -- 172.21.15.133:0/607797300 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc03c020020 con 0x7fc04c159dc0 2024-04-25T10:05:24.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.570+0000 7fc051652640 1 -- 172.21.15.133:0/607797300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc04c0cdaf0 con 0x7fc04c159dc0 2024-04-25T10:05:24.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.570+0000 7fc043fff640 1 -- 172.21.15.133:0/607797300 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17080 (secure 0 0 0) 0x7fc03c0074c0 con 0x7fc04c159dc0 2024-04-25T10:05:24.580 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:24.580 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":42,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:04:56.820759+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:16.032957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:16.760653+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 -- 172.21.15.133:0/607797300 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc02c066f60 msgr2=0x7fc02c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc02c066f60 0x7fc02c069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc0340ad000 tx=0x7fc0340a6000 comp rx=0 tx=0).stop 2024-04-25T10:05:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 -- 172.21.15.133:0/607797300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc04c159dc0 msgr2=0x7fc04c14bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc04c159dc0 0x7fc04c14bf80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00c530 tx=0x7fc03c00c9f0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 -- 172.21.15.133:0/607797300 shutdown_connections 2024-04-25T10:05:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc04c14cd20 0x7fc04c156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc04c15e050 0x7fc04c156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc02c066f60 0x7fc02c069420 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 --2- 172.21.15.133:0/607797300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc04c159dc0 0x7fc04c14bf80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 -- 172.21.15.133:0/607797300 >> 172.21.15.133:0/607797300 conn(0x7fc04c067000 msgr2=0x7fc04c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 -- 172.21.15.133:0/607797300 shutdown_connections 2024-04-25T10:05:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.574+0000 7fc051652640 1 -- 172.21.15.133:0/607797300 wait complete. 2024-04-25T10:05:24.598 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:05:24.598 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f866a794640 1 Processor -- start 2024-04-25T10:05:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f866a794640 1 -- start start 2024-04-25T10:05:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f866a794640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f866415c9a0 0x7f86640c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f866a794640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86640ce330 0x7f86640ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f866a794640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86640c61d0 con 0x7f86640b1f70 2024-04-25T10:05:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f866a794640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86640cecc0 con 0x7f86640ce330 2024-04-25T10:05:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f866a794640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86640c64b0 con 0x7f866415c9a0 2024-04-25T10:05:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f8668d31640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86640ce330 0x7f86640ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f8668d31640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86640ce330 0x7f86640ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55608/0 (socket says 172.21.15.133:55608) 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f8668d31640 1 -- 172.21.15.133:0/3176510755 learned_addr learned my addr 172.21.15.133:0/3176510755 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f8663fff640 1 --2- 172.21.15.133:0/3176510755 >> v2:172.21.15.162:3300/0 conn(0x7f866415c9a0 0x7f86640c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f8668d31640 1 -- 172.21.15.133:0/3176510755 >> v2:172.21.15.162:3300/0 conn(0x7f866415c9a0 msgr2=0x7f86640c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f8668d31640 1 --2- 172.21.15.133:0/3176510755 >> v2:172.21.15.162:3300/0 conn(0x7f866415c9a0 0x7f86640c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f8668d31640 1 -- 172.21.15.133:0/3176510755 >> v1:172.21.15.162:6789/0 conn(0x7f86640b1f70 legacy=0x7f866406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.710+0000 7f8668d31640 1 -- 172.21.15.133:0/3176510755 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866406d4f0 con 0x7f86640ce330 2024-04-25T10:05:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f8668d31640 1 --2- 172.21.15.133:0/3176510755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86640ce330 0x7f86640ce780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f866406f5f0 tx=0x7f865c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f07b07ba7dc11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f8662ffd640 1 -- 172.21.15.133:0/3176510755 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0a5720 con 0x7f86640ce330 2024-04-25T10:05:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f8662ffd640 1 -- 172.21.15.133:0/3176510755 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c0a58c0 con 0x7f86640ce330 2024-04-25T10:05:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f8662ffd640 1 -- 172.21.15.133:0/3176510755 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0a5c90 con 0x7f86640ce330 2024-04-25T10:05:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 -- 172.21.15.133:0/3176510755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86640ce330 msgr2=0x7f86640ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 --2- 172.21.15.133:0/3176510755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86640ce330 0x7f86640ce780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f866406f5f0 tx=0x7f865c01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 -- 172.21.15.133:0/3176510755 shutdown_connections 2024-04-25T10:05:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 --2- 172.21.15.133:0/3176510755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86640ce330 0x7f86640ce780 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 --2- 172.21.15.133:0/3176510755 >> v2:172.21.15.162:3300/0 conn(0x7f866415c9a0 0x7f86640c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 -- 172.21.15.133:0/3176510755 >> 172.21.15.133:0/3176510755 conn(0x7f8664067000 msgr2=0x7f866406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 -- 172.21.15.133:0/3176510755 shutdown_connections 2024-04-25T10:05:24.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 -- 172.21.15.133:0/3176510755 wait complete. 2024-04-25T10:05:24.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 Processor -- start 2024-04-25T10:05:24.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 -- start start 2024-04-25T10:05:24.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86640b1f70 0x7f86640cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86640ce330 0x7f8664153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f866415c9a0 0x7f86640cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866415dad0 con 0x7f86640ce330 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866415d950 con 0x7f86640b1f70 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f866a794640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866415dc50 con 0x7f866415c9a0 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f86637fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86640ce330 0x7f8664153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f86637fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86640ce330 0x7f8664153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55618/0 (socket says 172.21.15.133:55618) 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f86637fe640 1 -- 172.21.15.133:0/2621964341 learned_addr learned my addr 172.21.15.133:0/2621964341 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f8663fff640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86640b1f70 0x7f86640cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f8668d31640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f866415c9a0 0x7f86640cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f86637fe640 1 -- 172.21.15.133:0/2621964341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f866415c9a0 msgr2=0x7f86640cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f86637fe640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f866415c9a0 0x7f86640cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f86637fe640 1 -- 172.21.15.133:0/2621964341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86640b1f70 msgr2=0x7f86640cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f86637fe640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86640b1f70 0x7f86640cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f86637fe640 1 -- 172.21.15.133:0/2621964341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640ce120 con 0x7f86640ce330 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f8663fff640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86640b1f70 0x7f86640cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:24.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f8668d31640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f866415c9a0 0x7f86640cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:24.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f86637fe640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86640ce330 0x7f8664153950 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f865800b410 tx=0x7f865800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f8660ff9640 1 -- 172.21.15.133:0/2621964341 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658015020 con 0x7f86640ce330 2024-04-25T10:05:24.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f8660ff9640 1 -- 172.21.15.133:0/2621964341 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8658011830 con 0x7f86640ce330 2024-04-25T10:05:24.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.714+0000 7f8660ff9640 1 -- 172.21.15.133:0/2621964341 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658011c20 con 0x7f86640ce330 2024-04-25T10:05:24.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.718+0000 7f866a794640 1 -- 172.21.15.133:0/2621964341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8664150480 con 0x7f86640ce330 2024-04-25T10:05:24.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.718+0000 7f866a794640 1 -- 172.21.15.133:0/2621964341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866414e720 con 0x7f86640ce330 2024-04-25T10:05:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.718+0000 7f866a794640 1 -- 172.21.15.133:0/2621964341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8664068ef0 con 0x7f86640ce330 2024-04-25T10:05:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.718+0000 7f8660ff9640 1 -- 172.21.15.133:0/2621964341 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8658018020 con 0x7f86640ce330 2024-04-25T10:05:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.718+0000 7f8660ff9640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8650066f60 0x7f8650069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.718+0000 7f8660ff9640 1 -- 172.21.15.133:0/2621964341 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f8658085120 con 0x7f86640ce330 2024-04-25T10:05:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.718+0000 7f8663fff640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8650066f60 0x7f8650069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.718+0000 7f8663fff640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8650066f60 0x7f8650069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8654006fd0 tx=0x7f8654008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.722+0000 7f8660ff9640 1 -- 172.21.15.133:0/2621964341 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f865801f020 con 0x7f86640ce330 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 -- 172.21.15.133:0/2621964341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f866406c2d0 con 0x7f86640ce330 2024-04-25T10:05:24.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f8660ff9640 1 -- 172.21.15.133:0/2621964341 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f86580172c0 con 0x7f86640ce330 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2887512873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2887512873},{"type":"v1","addr":"172.21.15.162:6833","nonce":2887512873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1044882076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1044882076},{"type":"v1","addr":"172.21.15.162:6835","nonce":1044882076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1965748675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1965748675},{"type":"v1","addr":"172.21.15.133:6836","nonce":1965748675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:03:17.773574+0000","modified":"2024-04-25T10:03:26.098972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/205640021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":205640021},{"type":"v1","addr":"172.21.15.133:6837","nonce":205640021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:05:24.926 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 17 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 -- 172.21.15.133:0/2621964341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8650066f60 msgr2=0x7f8650069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8650066f60 0x7f8650069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8654006fd0 tx=0x7f8654008040 comp rx=0 tx=0).stop 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 -- 172.21.15.133:0/2621964341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86640ce330 msgr2=0x7f8664153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86640ce330 0x7f8664153950 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f865800b410 tx=0x7f865800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 -- 172.21.15.133:0/2621964341 shutdown_connections 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f866415c9a0 0x7f86640cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8650066f60 0x7f8650069420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86640ce330 0x7f8664153950 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 --2- 172.21.15.133:0/2621964341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86640b1f70 0x7f86640cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 -- 172.21.15.133:0/2621964341 >> 172.21.15.133:0/2621964341 conn(0x7f8664067000 msgr2=0x7f86641513e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 -- 172.21.15.133:0/2621964341 shutdown_connections 2024-04-25T10:05:24.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:24.918+0000 7f866a794640 1 -- 172.21.15.133:0/2621964341 wait complete. 2024-04-25T10:05:24.945 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:25.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb029e1640 1 Processor -- start 2024-04-25T10:05:25.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb029e1640 1 -- start start 2024-04-25T10:05:25.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb029e1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcafc0b1f70 0x7fcafc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb029e1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcafc0c61d0 con 0x7fcafc0ce330 2024-04-25T10:05:25.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb029e1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcafc0c64b0 con 0x7fcafc15c9a0 2024-04-25T10:05:25.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb029e1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcafc06d4f0 con 0x7fcafc0b1f70 2024-04-25T10:05:25.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb00f7e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcafc0ce330 0x7fcafc15d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49814/0 (socket says 172.21.15.133:49814) 2024-04-25T10:05:25.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb00f7e640 1 -- 172.21.15.133:0/3432196300 learned_addr learned my addr 172.21.15.133:0/3432196300 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:25.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafb7fe640 1 --2- 172.21.15.133:0/3432196300 >> v2:172.21.15.162:3300/0 conn(0x7fcafc0b1f70 0x7fcafc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784840855 0 0) 0x7fcafc0c61d0 con 0x7fcafc0ce330 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae4003530 con 0x7fcafc0ce330 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1033191611 0 0) 0x7fcae4003530 con 0x7fcafc0ce330 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 >> v2:172.21.15.162:3300/0 conn(0x7fcafc0b1f70 msgr2=0x7fcafc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 --2- 172.21.15.133:0/3432196300 >> v2:172.21.15.162:3300/0 conn(0x7fcafc0b1f70 0x7fcafc06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 >> v1:172.21.15.162:6789/0 conn(0x7fcafc15c9a0 legacy=0x7fcafc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc06d7d0 con 0x7fcafc0ce330 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fcaf4001a00 con 0x7fcafc0ce330 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 >> v1:172.21.15.133:6789/0 conn(0x7fcafc0ce330 legacy=0x7fcafc15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 --2- 172.21.15.133:0/3432196300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae40039c0 0x7fcae4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 --2- 172.21.15.133:0/3432196300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae4004810 0x7fcae4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 --2- 172.21.15.133:0/3432196300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40051d0 0x7fcae400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc06d4f0 con 0x7fcae40051d0 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae400c070 con 0x7fcae4004810 2024-04-25T10:05:25.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae400c270 con 0x7fcae40039c0 2024-04-25T10:05:25.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb00f7e640 1 --2- 172.21.15.133:0/3432196300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40051d0 0x7fcae400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafb7fe640 1 --2- 172.21.15.133:0/3432196300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae4004810 0x7fcae4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcafbfff640 1 --2- 172.21.15.133:0/3432196300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae40039c0 0x7fcae4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb00f7e640 1 -- 172.21.15.133:0/3432196300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae40039c0 msgr2=0x7fcae4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb00f7e640 1 --2- 172.21.15.133:0/3432196300 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae40039c0 0x7fcae4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb00f7e640 1 -- 172.21.15.133:0/3432196300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae4004810 msgr2=0x7fcae4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb00f7e640 1 --2- 172.21.15.133:0/3432196300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae4004810 0x7fcae4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.050+0000 7fcb00f7e640 1 -- 172.21.15.133:0/3432196300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae4003530 con 0x7fcae40051d0 2024-04-25T10:05:25.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcb00f7e640 1 -- 172.21.15.133:0/3432196300 reap_dead start 2024-04-25T10:05:25.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcb00f7e640 1 --2- 172.21.15.133:0/3432196300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40051d0 0x7fcae400b9d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400a860 tx=0x7fcaf401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a003598f77cec1a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf400a480 con 0x7fcae40051d0 2024-04-25T10:05:25.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf40ad630 con 0x7fcae40051d0 2024-04-25T10:05:25.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcafaffd640 1 -- 172.21.15.133:0/3432196300 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf40ada20 con 0x7fcae40051d0 2024-04-25T10:05:25.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcb029e1640 1 -- 172.21.15.133:0/3432196300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40051d0 msgr2=0x7fcae400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcb029e1640 1 --2- 172.21.15.133:0/3432196300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40051d0 0x7fcae400b9d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400a860 tx=0x7fcaf401d380 comp rx=0 tx=0).stop 2024-04-25T10:05:25.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcb029e1640 1 -- 172.21.15.133:0/3432196300 shutdown_connections 2024-04-25T10:05:25.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcb029e1640 1 --2- 172.21.15.133:0/3432196300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40051d0 0x7fcae400b9d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcb029e1640 1 -- 172.21.15.133:0/3432196300 >> 172.21.15.133:0/3432196300 conn(0x7fcafc067000 msgr2=0x7fcafc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcb029e1640 1 -- 172.21.15.133:0/3432196300 shutdown_connections 2024-04-25T10:05:25.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcb029e1640 1 -- 172.21.15.133:0/3432196300 wait complete. 2024-04-25T10:05:25.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcb029e1640 1 Processor -- start 2024-04-25T10:05:25.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.054+0000 7fcb029e1640 1 -- start start 2024-04-25T10:05:25.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcb029e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae40051d0 0x7fcafc145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcb029e1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcafc14f280 0x7fcafc14fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcb029e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcafc15bad0 0x7fcafc0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcb029e1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0ce780 con 0x7fcafc14f280 2024-04-25T10:05:25.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcb029e1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0ce900 con 0x7fcae40051d0 2024-04-25T10:05:25.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcb029e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0cea80 con 0x7fcafc15bad0 2024-04-25T10:05:25.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcafc14f280 0x7fcafc14fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcafc14f280 0x7fcafc14fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55640/0 (socket says 172.21.15.133:55640) 2024-04-25T10:05:25.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafb7fe640 1 -- 172.21.15.133:0/2022359014 learned_addr learned my addr 172.21.15.133:0/2022359014 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafbfff640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae40051d0 0x7fcafc145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcb00f7e640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcafc15bad0 0x7fcafc0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafb7fe640 1 -- 172.21.15.133:0/2022359014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcafc15bad0 msgr2=0x7fcafc0b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafb7fe640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcafc15bad0 0x7fcafc0b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafb7fe640 1 -- 172.21.15.133:0/2022359014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae40051d0 msgr2=0x7fcafc145e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafb7fe640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae40051d0 0x7fcafc145e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafb7fe640 1 -- 172.21.15.133:0/2022359014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc15aa30 con 0x7fcafc14f280 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafb7fe640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcafc14f280 0x7fcafc14fde0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0011960 tx=0x7fcaf0011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcaf8ff9640 1 -- 172.21.15.133:0/2022359014 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000faf0 con 0x7fcafc14f280 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcaf8ff9640 1 -- 172.21.15.133:0/2022359014 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf000fc90 con 0x7fcafc14f280 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcb029e1640 1 -- 172.21.15.133:0/2022359014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcafc15ad10 con 0x7fcafc14f280 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcaf8ff9640 1 -- 172.21.15.133:0/2022359014 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0014680 con 0x7fcafc14f280 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcb029e1640 1 -- 172.21.15.133:0/2022359014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcafc15b2d0 con 0x7fcafc14f280 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcb029e1640 1 -- 172.21.15.133:0/2022359014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcafc068ef0 con 0x7fcafc14f280 2024-04-25T10:05:25.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcaf8ff9640 1 -- 172.21.15.133:0/2022359014 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcaf0012040 con 0x7fcafc14f280 2024-04-25T10:05:25.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcaf8ff9640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcae8066f60 0x7fcae8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcaf8ff9640 1 -- 172.21.15.133:0/2022359014 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fcaf00857c0 con 0x7fcafc14f280 2024-04-25T10:05:25.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafbfff640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcae8066f60 0x7fcae8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.058+0000 7fcafbfff640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcae8066f60 0x7fcae8069420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000c10 tx=0x7fcaec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.062+0000 7fcaf8ff9640 1 -- 172.21.15.133:0/2022359014 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcaf0019030 con 0x7fcafc14f280 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.246+0000 7fcb029e1640 1 -- 172.21.15.133:0/2022359014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcafc0b4b50 con 0x7fcafc14f280 2024-04-25T10:05:25.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.246+0000 7fcaf8ff9640 1 -- 172.21.15.133:0/2022359014 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fcaf00566a0 con 0x7fcafc14f280 2024-04-25T10:05:25.251 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:25.252 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2887512873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2887512873},{"type":"v1","addr":"172.21.15.162:6833","nonce":2887512873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1044882076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1044882076},{"type":"v1","addr":"172.21.15.162:6835","nonce":1044882076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1965748675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1965748675},{"type":"v1","addr":"172.21.15.133:6836","nonce":1965748675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:03:17.773574+0000","modified":"2024-04-25T10:03:26.098972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/205640021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":205640021},{"type":"v1","addr":"172.21.15.133:6837","nonce":205640021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:05:25.252 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 17 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 -- 172.21.15.133:0/2022359014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcae8066f60 msgr2=0x7fcae8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcae8066f60 0x7fcae8069420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000c10 tx=0x7fcaec009040 comp rx=0 tx=0).stop 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 -- 172.21.15.133:0/2022359014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcafc14f280 msgr2=0x7fcafc14fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcafc14f280 0x7fcafc14fde0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0011960 tx=0x7fcaf0011e20 comp rx=0 tx=0).stop 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 -- 172.21.15.133:0/2022359014 shutdown_connections 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcafc15bad0 0x7fcafc0b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcae8066f60 0x7fcae8069420 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcafc14f280 0x7fcafc14fde0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 --2- 172.21.15.133:0/2022359014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae40051d0 0x7fcafc145e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 -- 172.21.15.133:0/2022359014 >> 172.21.15.133:0/2022359014 conn(0x7fcafc067000 msgr2=0x7fcafc158430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 -- 172.21.15.133:0/2022359014 shutdown_connections 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.250+0000 7fcb029e1640 1 -- 172.21.15.133:0/2022359014 wait complete. 2024-04-25T10:05:25.269 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:25.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796e59a640 1 Processor -- start 2024-04-25T10:05:25.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796e59a640 1 -- start start 2024-04-25T10:05:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796e59a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f79680d0770 0x7f79680d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796e59a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79680b87e0 0x7f79680b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796e59a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79680ceb20 con 0x7f796815db00 2024-04-25T10:05:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796e59a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79680d1110 con 0x7f79680b87e0 2024-04-25T10:05:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796e59a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79680b30a0 con 0x7f79680d0770 2024-04-25T10:05:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796cb37640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79680b87e0 0x7f79680b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796cb37640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79680b87e0 0x7f79680b8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55644/0 (socket says 172.21.15.133:55644) 2024-04-25T10:05:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796cb37640 1 -- 172.21.15.133:0/217899895 learned_addr learned my addr 172.21.15.133:0/217899895 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f79677fe640 1 --2- 172.21.15.133:0/217899895 >> v2:172.21.15.162:3300/0 conn(0x7f79680d0770 0x7f79680d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796cb37640 1 -- 172.21.15.133:0/217899895 >> v2:172.21.15.162:3300/0 conn(0x7f79680d0770 msgr2=0x7f79680d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796cb37640 1 --2- 172.21.15.133:0/217899895 >> v2:172.21.15.162:3300/0 conn(0x7f79680d0770 0x7f79680d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796cb37640 1 -- 172.21.15.133:0/217899895 >> v1:172.21.15.162:6789/0 conn(0x7f796815db00 legacy=0x7f79680b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.374+0000 7f796cb37640 1 -- 172.21.15.133:0/217899895 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680b7740 con 0x7f79680b87e0 2024-04-25T10:05:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796cb37640 1 --2- 172.21.15.133:0/217899895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79680b87e0 0x7f79680b8c70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f79680d0200 tx=0x7f796001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749b354ba124ca2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f7966ffd640 1 -- 172.21.15.133:0/217899895 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79600a5840 con 0x7f79680b87e0 2024-04-25T10:05:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f7966ffd640 1 -- 172.21.15.133:0/217899895 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79600a59e0 con 0x7f79680b87e0 2024-04-25T10:05:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f7966ffd640 1 -- 172.21.15.133:0/217899895 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79600a5db0 con 0x7f79680b87e0 2024-04-25T10:05:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 -- 172.21.15.133:0/217899895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79680b87e0 msgr2=0x7f79680b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 --2- 172.21.15.133:0/217899895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79680b87e0 0x7f79680b8c70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f79680d0200 tx=0x7f796001c980 comp rx=0 tx=0).stop 2024-04-25T10:05:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 -- 172.21.15.133:0/217899895 shutdown_connections 2024-04-25T10:05:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 --2- 172.21.15.133:0/217899895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79680b87e0 0x7f79680b8c70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 --2- 172.21.15.133:0/217899895 >> v2:172.21.15.162:3300/0 conn(0x7f79680d0770 0x7f79680d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 -- 172.21.15.133:0/217899895 >> 172.21.15.133:0/217899895 conn(0x7f79680727c0 msgr2=0x7f7968072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 -- 172.21.15.133:0/217899895 shutdown_connections 2024-04-25T10:05:25.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 -- 172.21.15.133:0/217899895 wait complete. 2024-04-25T10:05:25.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 Processor -- start 2024-04-25T10:05:25.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 -- start start 2024-04-25T10:05:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79680b87e0 0x7f79680ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79680d0770 0x7f796815f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f796815db00 0x7f79680cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79680b7430 con 0x7f796815db00 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79680b82c0 con 0x7f79680d0770 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796e59a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79680d18a0 con 0x7f79680b87e0 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796cb37640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f796815db00 0x7f79680cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796cb37640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f796815db00 0x7f79680cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55646/0 (socket says 172.21.15.133:55646) 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.378+0000 7f796cb37640 1 -- 172.21.15.133:0/3365731817 learned_addr learned my addr 172.21.15.133:0/3365731817 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f79677fe640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79680d0770 0x7f796815f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f7967fff640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79680b87e0 0x7f79680ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f796cb37640 1 -- 172.21.15.133:0/3365731817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79680b87e0 msgr2=0x7f79680ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f796cb37640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79680b87e0 0x7f79680ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f796cb37640 1 -- 172.21.15.133:0/3365731817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79680d0770 msgr2=0x7f796815f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f796cb37640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79680d0770 0x7f796815f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f796cb37640 1 -- 172.21.15.133:0/3365731817 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968160890 con 0x7f796815db00 2024-04-25T10:05:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f796cb37640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f796815db00 0x7f79680cb410 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f796001a5b0 tx=0x7f7960019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f7964ff9640 1 -- 172.21.15.133:0/3365731817 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79600bc020 con 0x7f796815db00 2024-04-25T10:05:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f7964ff9640 1 -- 172.21.15.133:0/3365731817 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79600b8690 con 0x7f796815db00 2024-04-25T10:05:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f796e59a640 1 -- 172.21.15.133:0/3365731817 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7968160b70 con 0x7f796815db00 2024-04-25T10:05:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f7964ff9640 1 -- 172.21.15.133:0/3365731817 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79600b8a80 con 0x7f796815db00 2024-04-25T10:05:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f796e59a640 1 -- 172.21.15.133:0/3365731817 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79680c9f50 con 0x7f796815db00 2024-04-25T10:05:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f796e59a640 1 -- 172.21.15.133:0/3365731817 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7968068ef0 con 0x7f796815db00 2024-04-25T10:05:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f7964ff9640 1 -- 172.21.15.133:0/3365731817 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79600bf020 con 0x7f796815db00 2024-04-25T10:05:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f7964ff9640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7954066f60 0x7f7954069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f7964ff9640 1 -- 172.21.15.133:0/3365731817 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f7960120420 con 0x7f796815db00 2024-04-25T10:05:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f7967fff640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7954066f60 0x7f7954069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.382+0000 7f7967fff640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7954066f60 0x7f7954069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7958004e30 tx=0x7f7958009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.386+0000 7f7964ff9640 1 -- 172.21.15.133:0/3365731817 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79600c6030 con 0x7f796815db00 2024-04-25T10:05:25.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.542+0000 7f796e59a640 1 -- 172.21.15.133:0/3365731817 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79680b2750 con 0x7f796815db00 2024-04-25T10:05:25.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f7964ff9640 1 -- 172.21.15.133:0/3365731817 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17080 (secure 0 0 0) 0x7f79600165a0 con 0x7f796815db00 2024-04-25T10:05:25.549 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:25.549 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":42,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:04:56.820759+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:16.032957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:16.760653+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:25.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 -- 172.21.15.133:0/3365731817 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7954066f60 msgr2=0x7f7954069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7954066f60 0x7f7954069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7958004e30 tx=0x7f7958009040 comp rx=0 tx=0).stop 2024-04-25T10:05:25.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 -- 172.21.15.133:0/3365731817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f796815db00 msgr2=0x7f79680cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f796815db00 0x7f79680cb410 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f796001a5b0 tx=0x7f7960019720 comp rx=0 tx=0).stop 2024-04-25T10:05:25.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 -- 172.21.15.133:0/3365731817 shutdown_connections 2024-04-25T10:05:25.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7954066f60 0x7f7954069420 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f796815db00 0x7f79680cb410 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79680d0770 0x7f796815f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 --2- 172.21.15.133:0/3365731817 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79680b87e0 0x7f79680ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 -- 172.21.15.133:0/3365731817 >> 172.21.15.133:0/3365731817 conn(0x7f79680727c0 msgr2=0x7f7968072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 -- 172.21.15.133:0/3365731817 shutdown_connections 2024-04-25T10:05:25.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.546+0000 7f796e59a640 1 -- 172.21.15.133:0/3365731817 wait complete. 2024-04-25T10:05:25.567 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:05:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f474819f640 1 Processor -- start 2024-04-25T10:05:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f474819f640 1 -- start start 2024-04-25T10:05:25.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f474819f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f47400cd820 0x7f47400cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f474819f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f474015ced0 0x7f474015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f474819f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47400ce2e0 con 0x7f4740145170 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f474819f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474006db00 con 0x7f47400cd820 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f474819f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4740145550 con 0x7f474015ced0 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f4745f3b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4740145170 0x7f474006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49816/0 (socket says 172.21.15.133:49816) 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f4745f3b640 1 -- 172.21.15.133:0/2610751362 learned_addr learned my addr 172.21.15.133:0/2610751362 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f474573a640 1 --2- 172.21.15.133:0/2610751362 >> v2:172.21.15.133:3300/0 conn(0x7f47400cd820 0x7f47400cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f474673c640 1 --2- 172.21.15.133:0/2610751362 >> v2:172.21.15.162:3300/0 conn(0x7f474015ced0 0x7f474015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f4744f39640 1 -- 172.21.15.133:0/2610751362 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213791956 0 0) 0x7f47400ce2e0 con 0x7f4740145170 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.666+0000 7f4744f39640 1 -- 172.21.15.133:0/2610751362 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f472c003530 con 0x7f4740145170 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474573a640 1 -- 172.21.15.133:0/2610751362 >> v2:172.21.15.162:3300/0 conn(0x7f474015ced0 msgr2=0x7f474015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474573a640 1 --2- 172.21.15.133:0/2610751362 >> v2:172.21.15.162:3300/0 conn(0x7f474015ced0 0x7f474015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474573a640 1 -- 172.21.15.133:0/2610751362 >> v1:172.21.15.133:6789/0 conn(0x7f4740145170 legacy=0x7f474006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474573a640 1 -- 172.21.15.133:0/2610751362 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474006d4b0 con 0x7f47400cd820 2024-04-25T10:05:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474573a640 1 --2- 172.21.15.133:0/2610751362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47400cd820 0x7f47400cdca0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f47400b27f0 tx=0x7f473001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da24ea6bf2aa1bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f4744f39640 1 -- 172.21.15.133:0/2610751362 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47300b0070 con 0x7f47400cd820 2024-04-25T10:05:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f4744f39640 1 -- 172.21.15.133:0/2610751362 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47300ab600 con 0x7f47400cd820 2024-04-25T10:05:25.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f4744f39640 1 -- 172.21.15.133:0/2610751362 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47300ab9d0 con 0x7f47400cd820 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474819f640 1 -- 172.21.15.133:0/2610751362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47400cd820 msgr2=0x7f47400cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474819f640 1 --2- 172.21.15.133:0/2610751362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47400cd820 0x7f47400cdca0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f47400b27f0 tx=0x7f473001c480 comp rx=0 tx=0).stop 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474819f640 1 -- 172.21.15.133:0/2610751362 shutdown_connections 2024-04-25T10:05:25.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474819f640 1 --2- 172.21.15.133:0/2610751362 >> v2:172.21.15.162:3300/0 conn(0x7f474015ced0 0x7f474015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474819f640 1 --2- 172.21.15.133:0/2610751362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47400cd820 0x7f47400cdca0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474819f640 1 -- 172.21.15.133:0/2610751362 >> 172.21.15.133:0/2610751362 conn(0x7f4740066f70 msgr2=0x7f474006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474819f640 1 -- 172.21.15.133:0/2610751362 shutdown_connections 2024-04-25T10:05:25.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474819f640 1 -- 172.21.15.133:0/2610751362 wait complete. 2024-04-25T10:05:25.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474819f640 1 Processor -- start 2024-04-25T10:05:25.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.670+0000 7f474819f640 1 -- start start 2024-04-25T10:05:25.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f474819f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47400cd820 0x7f4740134070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f474819f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4740145170 0x7f47400b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f474819f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f474015ced0 0x7f47401362d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f474819f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4740168610 con 0x7f47400cd820 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f474819f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4740168490 con 0x7f4740145170 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f474819f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4740168790 con 0x7f474015ced0 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f4745f3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47400cd820 0x7f4740134070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f4745f3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47400cd820 0x7f4740134070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55652/0 (socket says 172.21.15.133:55652) 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f4745f3b640 1 -- 172.21.15.133:0/383373840 learned_addr learned my addr 172.21.15.133:0/383373840 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f4745f3b640 1 -- 172.21.15.133:0/383373840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f474015ced0 msgr2=0x7f47401362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f4745f3b640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f474015ced0 0x7f47401362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f4745f3b640 1 -- 172.21.15.133:0/383373840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4740145170 msgr2=0x7f47400b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f4745f3b640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4740145170 0x7f47400b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f4745f3b640 1 -- 172.21.15.133:0/383373840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47401442e0 con 0x7f47400cd820 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f4745f3b640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47400cd820 0x7f4740134070 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c450 tx=0x7f473c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f47367fc640 1 -- 172.21.15.133:0/383373840 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c017070 con 0x7f47400cd820 2024-04-25T10:05:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f47367fc640 1 -- 172.21.15.133:0/383373840 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c005db0 con 0x7f47400cd820 2024-04-25T10:05:25.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f474819f640 1 -- 172.21.15.133:0/383373840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47401445c0 con 0x7f47400cd820 2024-04-25T10:05:25.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f47367fc640 1 -- 172.21.15.133:0/383373840 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c007290 con 0x7f47400cd820 2024-04-25T10:05:25.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f474819f640 1 -- 172.21.15.133:0/383373840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4740140120 con 0x7f47400cd820 2024-04-25T10:05:25.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f474819f640 1 -- 172.21.15.133:0/383373840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4740068e80 con 0x7f47400cd820 2024-04-25T10:05:25.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f47367fc640 1 -- 172.21.15.133:0/383373840 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f473c01a070 con 0x7f47400cd820 2024-04-25T10:05:25.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f47367fc640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4724066f90 0x7f4724069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f47367fc640 1 -- 172.21.15.133:0/383373840 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f473c084510 con 0x7f47400cd820 2024-04-25T10:05:25.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f474573a640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4724066f90 0x7f4724069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.674+0000 7f474573a640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4724066f90 0x7f4724069450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47300ad000 tx=0x7f4730004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.678+0000 7f47367fc640 1 -- 172.21.15.133:0/383373840 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f473c0553f0 con 0x7f47400cd820 2024-04-25T10:05:25.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.858+0000 7f474819f640 1 -- 172.21.15.133:0/383373840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4740141a00 con 0x7f47400cd820 2024-04-25T10:05:25.862 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:05:25.858+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:05:25.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.862+0000 7f47367fc640 1 -- 172.21.15.133:0/383373840 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f473c054cf0 con 0x7f47400cd820 2024-04-25T10:05:25.865 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:05:25.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 -- 172.21.15.133:0/383373840 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4724066f90 msgr2=0x7f4724069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4724066f90 0x7f4724069450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47300ad000 tx=0x7f4730004740 comp rx=0 tx=0).stop 2024-04-25T10:05:25.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 -- 172.21.15.133:0/383373840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47400cd820 msgr2=0x7f4740134070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47400cd820 0x7f4740134070 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c450 tx=0x7f473c00c910 comp rx=0 tx=0).stop 2024-04-25T10:05:25.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 -- 172.21.15.133:0/383373840 shutdown_connections 2024-04-25T10:05:25.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f474015ced0 0x7f47401362d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4740145170 0x7f47400b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4724066f90 0x7f4724069450 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 --2- 172.21.15.133:0/383373840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47400cd820 0x7f4740134070 secure :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c450 tx=0x7f473c00c910 comp rx=0 tx=0).stop 2024-04-25T10:05:25.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 -- 172.21.15.133:0/383373840 >> 172.21.15.133:0/383373840 conn(0x7f4740066f70 msgr2=0x7f474013d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 -- 172.21.15.133:0/383373840 shutdown_connections 2024-04-25T10:05:25.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:25.866+0000 7f474819f640 1 -- 172.21.15.133:0/383373840 wait complete. 2024-04-25T10:05:25.892 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:05:25.917 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:05:26.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.006+0000 7f1fd090c640 1 Processor -- start 2024-04-25T10:05:26.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.006+0000 7f1fd090c640 1 -- start start 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.006+0000 7f1fd090c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1fcc142920 0x7f1fcc143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.006+0000 7f1fd090c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1fcc140a40 0x7f1fcc0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.006+0000 7f1fd090c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc0cf000 con 0x7f1fcc15d9f0 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.006+0000 7f1fd090c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc140ea0 con 0x7f1fcc140a40 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.006+0000 7f1fd090c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc0cf280 con 0x7f1fcc142920 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fc1d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1fcc140a40 0x7f1fcc0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fcad9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1fcc15d9f0 0x7f1fcc06df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49822/0 (socket says 172.21.15.133:49822) 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fcad9d640 1 -- 172.21.15.133:0/153935586 learned_addr learned my addr 172.21.15.133:0/153935586 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fca59c640 1 --2- 172.21.15.133:0/153935586 >> v2:172.21.15.162:3300/0 conn(0x7f1fcc142920 0x7f1fcc143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fc1d9b640 1 -- 172.21.15.133:0/153935586 >> v2:172.21.15.162:3300/0 conn(0x7f1fcc142920 msgr2=0x7f1fcc143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fc1d9b640 1 --2- 172.21.15.133:0/153935586 >> v2:172.21.15.162:3300/0 conn(0x7f1fcc142920 0x7f1fcc143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fc1d9b640 1 -- 172.21.15.133:0/153935586 >> v1:172.21.15.133:6789/0 conn(0x7f1fcc15d9f0 legacy=0x7f1fcc06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fc9d9b640 1 -- 172.21.15.133:0/153935586 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757887572 0 0) 0x7f1fc40014a0 con 0x7f1fcc15d9f0 2024-04-25T10:05:26.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fc1d9b640 1 -- 172.21.15.133:0/153935586 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc06d440 con 0x7f1fcc140a40 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fc1d9b640 1 --2- 172.21.15.133:0/153935586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fcc140a40 0x7f1fcc0ceac0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc06ee50 tx=0x7f1fb401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfda58ae308598b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fc9d9b640 1 -- 172.21.15.133:0/153935586 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40b0070 con 0x7f1fcc140a40 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fc9d9b640 1 -- 172.21.15.133:0/153935586 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb40ab600 con 0x7f1fcc140a40 2024-04-25T10:05:26.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fc9d9b640 1 -- 172.21.15.133:0/153935586 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40ab9d0 con 0x7f1fcc140a40 2024-04-25T10:05:26.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fd090c640 1 -- 172.21.15.133:0/153935586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fcc140a40 msgr2=0x7f1fcc0ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fd090c640 1 --2- 172.21.15.133:0/153935586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fcc140a40 0x7f1fcc0ceac0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc06ee50 tx=0x7f1fb401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fd090c640 1 -- 172.21.15.133:0/153935586 shutdown_connections 2024-04-25T10:05:26.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fd090c640 1 --2- 172.21.15.133:0/153935586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fcc140a40 0x7f1fcc0ceac0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fd090c640 1 --2- 172.21.15.133:0/153935586 >> v2:172.21.15.162:3300/0 conn(0x7f1fcc142920 0x7f1fcc143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fd090c640 1 -- 172.21.15.133:0/153935586 >> 172.21.15.133:0/153935586 conn(0x7f1fcc066f70 msgr2=0x7f1fcc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fd090c640 1 -- 172.21.15.133:0/153935586 shutdown_connections 2024-04-25T10:05:26.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.010+0000 7f1fd090c640 1 -- 172.21.15.133:0/153935586 wait complete. 2024-04-25T10:05:26.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fd090c640 1 Processor -- start 2024-04-25T10:05:26.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fd090c640 1 -- start start 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fd090c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fcc140a40 0x7f1fcc1537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fd090c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fcc142920 0x7f1fcc14ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fd090c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fcc15d9f0 0x7f1fcc13e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fd090c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc06ebc0 con 0x7f1fcc142920 2024-04-25T10:05:26.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fd090c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc06ea40 con 0x7f1fcc15d9f0 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fd090c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc142d00 con 0x7f1fcc140a40 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fcc142920 0x7f1fcc14ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fcc142920 0x7f1fcc14ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55688/0 (socket says 172.21.15.133:55688) 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc1d9b640 1 -- 172.21.15.133:0/2794318446 learned_addr learned my addr 172.21.15.133:0/2794318446 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fca59c640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fcc140a40 0x7f1fcc1537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fcad9d640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fcc15d9f0 0x7f1fcc13e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc1d9b640 1 -- 172.21.15.133:0/2794318446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fcc140a40 msgr2=0x7f1fcc1537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc1d9b640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fcc140a40 0x7f1fcc1537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc1d9b640 1 -- 172.21.15.133:0/2794318446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fcc15d9f0 msgr2=0x7f1fcc13e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc1d9b640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fcc15d9f0 0x7f1fcc13e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc1d9b640 1 -- 172.21.15.133:0/2794318446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc13f620 con 0x7f1fcc142920 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fcad9d640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fcc15d9f0 0x7f1fcc13e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc1d9b640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fcc142920 0x7f1fcc14ed60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40ae040 tx=0x7f1fb4019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc37fe640 1 -- 172.21.15.133:0/2794318446 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40b8020 con 0x7f1fcc142920 2024-04-25T10:05:26.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc37fe640 1 -- 172.21.15.133:0/2794318446 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb40b4410 con 0x7f1fcc142920 2024-04-25T10:05:26.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fd090c640 1 -- 172.21.15.133:0/2794318446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc13f900 con 0x7f1fcc142920 2024-04-25T10:05:26.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fc37fe640 1 -- 172.21.15.133:0/2794318446 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40b0040 con 0x7f1fcc142920 2024-04-25T10:05:26.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fd090c640 1 -- 172.21.15.133:0/2794318446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc13fec0 con 0x7f1fcc142920 2024-04-25T10:05:26.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.014+0000 7f1fd090c640 1 -- 172.21.15.133:0/2794318446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc068e80 con 0x7f1fcc142920 2024-04-25T10:05:26.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.018+0000 7f1fc37fe640 1 -- 172.21.15.133:0/2794318446 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1fb40bb020 con 0x7f1fcc142920 2024-04-25T10:05:26.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.018+0000 7f1fc37fe640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1fac066f90 0x7f1fac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.018+0000 7f1fca59c640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1fac066f90 0x7f1fac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.018+0000 7f1fc37fe640 1 -- 172.21.15.133:0/2794318446 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f1fb411edd0 con 0x7f1fcc142920 2024-04-25T10:05:26.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.018+0000 7f1fca59c640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1fac066f90 0x7f1fac069450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc1570c0 tx=0x7f1fbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.022+0000 7f1fc37fe640 1 -- 172.21.15.133:0/2794318446 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1fb40c2030 con 0x7f1fcc142920 2024-04-25T10:05:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.222+0000 7f1fd090c640 1 -- 172.21.15.133:0/2794318446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1fcc0afa70 con 0x7f1fcc142920 2024-04-25T10:05:26.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.862+0000 7f1fc37fe640 1 -- 172.21.15.133:0/2794318446 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1fb41200a0 con 0x7f1fcc142920 2024-04-25T10:05:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 -- 172.21.15.133:0/2794318446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1fac066f90 msgr2=0x7f1fac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1fac066f90 0x7f1fac069450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc1570c0 tx=0x7f1fbc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 -- 172.21.15.133:0/2794318446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fcc142920 msgr2=0x7f1fcc14ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fcc142920 0x7f1fcc14ed60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40ae040 tx=0x7f1fb4019190 comp rx=0 tx=0).stop 2024-04-25T10:05:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 -- 172.21.15.133:0/2794318446 shutdown_connections 2024-04-25T10:05:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fcc15d9f0 0x7f1fcc13e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1fac066f90 0x7f1fac069450 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fcc142920 0x7f1fcc14ed60 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 --2- 172.21.15.133:0/2794318446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1fcc140a40 0x7f1fcc1537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 -- 172.21.15.133:0/2794318446 >> 172.21.15.133:0/2794318446 conn(0x7f1fcc066f70 msgr2=0x7f1fcc0cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 -- 172.21.15.133:0/2794318446 shutdown_connections 2024-04-25T10:05:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:26.866+0000 7f1fd090c640 1 -- 172.21.15.133:0/2794318446 wait complete. 2024-04-25T10:05:26.894 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:05:27.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb835e26640 1 Processor -- start 2024-04-25T10:05:27.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb835e26640 1 -- start start 2024-04-25T10:05:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb835e26640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb8300d3360 0x7fb8300ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb835e26640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8300d22c0 con 0x7fb8300d1d00 2024-04-25T10:05:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb835e26640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8300d25a0 con 0x7fb83015d9d0 2024-04-25T10:05:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb835e26640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8300d2880 con 0x7fb8300d3360 2024-04-25T10:05:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb82effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb8300d3360 0x7fb8300ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb82ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb8300d1d00 0x7fb8300b6a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36966/0 (socket says 172.21.15.133:36966) 2024-04-25T10:05:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb82ffff640 1 -- 172.21.15.133:0/1762596299 learned_addr learned my addr 172.21.15.133:0/1762596299 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb82effd640 1 -- 172.21.15.133:0/1762596299 >> v1:172.21.15.162:6789/0 conn(0x7fb83015d9d0 legacy=0x7fb8300b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb82effd640 1 -- 172.21.15.133:0/1762596299 >> v1:172.21.15.133:6789/0 conn(0x7fb8300d1d00 legacy=0x7fb8300b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.062+0000 7fb82effd640 1 -- 172.21.15.133:0/1762596299 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8300d2b60 con 0x7fb8300d3360 2024-04-25T10:05:27.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb82effd640 1 --2- 172.21.15.133:0/1762596299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8300d3360 0x7fb8300ce600 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb8240097c0 tx=0x7fb82401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff4b3d5d4c4dd11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb82e7fc640 1 -- 172.21.15.133:0/1762596299 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8240b0070 con 0x7fb8300d3360 2024-04-25T10:05:27.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb82e7fc640 1 -- 172.21.15.133:0/1762596299 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8240ab600 con 0x7fb8300d3360 2024-04-25T10:05:27.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb82e7fc640 1 -- 172.21.15.133:0/1762596299 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8240ab9d0 con 0x7fb8300d3360 2024-04-25T10:05:27.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 -- 172.21.15.133:0/1762596299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8300d3360 msgr2=0x7fb8300ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 --2- 172.21.15.133:0/1762596299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8300d3360 0x7fb8300ce600 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb8240097c0 tx=0x7fb82401c430 comp rx=0 tx=0).stop 2024-04-25T10:05:27.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 -- 172.21.15.133:0/1762596299 shutdown_connections 2024-04-25T10:05:27.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 --2- 172.21.15.133:0/1762596299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8300d3360 0x7fb8300ce600 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 -- 172.21.15.133:0/1762596299 >> 172.21.15.133:0/1762596299 conn(0x7fb83006c080 msgr2=0x7fb83006c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 -- 172.21.15.133:0/1762596299 shutdown_connections 2024-04-25T10:05:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 -- 172.21.15.133:0/1762596299 wait complete. 2024-04-25T10:05:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 Processor -- start 2024-04-25T10:05:27.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 -- start start 2024-04-25T10:05:27.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8300d1d00 0x7fb8300c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8300d3360 0x7fb8300c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83015d9d0 0x7fb8300c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83016b800 con 0x7fb83015d9d0 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300cef50 con 0x7fb8300d3360 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.066+0000 7fb835e26640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83016b980 con 0x7fb8300d1d00 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83015d9d0 0x7fb8300c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83015d9d0 0x7fb8300c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39742/0 (socket says 172.21.15.133:39742) 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82ffff640 1 -- 172.21.15.133:0/153734690 learned_addr learned my addr 172.21.15.133:0/153734690 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82effd640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8300d3360 0x7fb8300c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82ffff640 1 -- 172.21.15.133:0/153734690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8300d1d00 msgr2=0x7fb8300c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82ffff640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8300d1d00 0x7fb8300c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82ffff640 1 -- 172.21.15.133:0/153734690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8300d3360 msgr2=0x7fb8300c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82ffff640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8300d3360 0x7fb8300c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82ffff640 1 -- 172.21.15.133:0/153734690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8300c69f0 con 0x7fb83015d9d0 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82ffff640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83015d9d0 0x7fb8300c9b70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb82800ca70 tx=0x7fb82800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb817fff640 1 -- 172.21.15.133:0/153734690 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828017070 con 0x7fb83015d9d0 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb817fff640 1 -- 172.21.15.133:0/153734690 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82800abd0 con 0x7fb83015d9d0 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb835e26640 1 -- 172.21.15.133:0/153734690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83015a650 con 0x7fb83015d9d0 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb817fff640 1 -- 172.21.15.133:0/153734690 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828012400 con 0x7fb83015d9d0 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb835e26640 1 -- 172.21.15.133:0/153734690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83015eab0 con 0x7fb83015d9d0 2024-04-25T10:05:27.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb835e26640 1 -- 172.21.15.133:0/153734690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb830069000 con 0x7fb83015d9d0 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb817fff640 1 -- 172.21.15.133:0/153734690 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb82801a070 con 0x7fb83015d9d0 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb817fff640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb81c066f60 0x7fb81c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb817fff640 1 -- 172.21.15.133:0/153734690 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fb828084d20 con 0x7fb83015d9d0 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82f7fe640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb81c066f60 0x7fb81c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.070+0000 7fb82f7fe640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb81c066f60 0x7fb81c069420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb830161060 tx=0x7fb820006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.074+0000 7fb817fff640 1 -- 172.21.15.133:0/153734690 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb828055570 con 0x7fb83015d9d0 2024-04-25T10:05:27.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:27.238+0000 7fb835e26640 1 -- 172.21.15.133:0/153734690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb8300b8300 con 0x7fb83015d9d0 2024-04-25T10:05:28.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.098+0000 7fb817fff640 1 -- 172.21.15.133:0/153734690 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb828059610 con 0x7fb83015d9d0 2024-04-25T10:05:28.101 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:05:28.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 -- 172.21.15.133:0/153734690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb81c066f60 msgr2=0x7fb81c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb81c066f60 0x7fb81c069420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb830161060 tx=0x7fb820006bd0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 -- 172.21.15.133:0/153734690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83015d9d0 msgr2=0x7fb8300c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83015d9d0 0x7fb8300c9b70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb82800ca70 tx=0x7fb82800cf30 comp rx=0 tx=0).stop 2024-04-25T10:05:28.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 -- 172.21.15.133:0/153734690 shutdown_connections 2024-04-25T10:05:28.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb81c066f60 0x7fb81c069420 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb83015d9d0 0x7fb8300c9b70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8300d3360 0x7fb8300c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 --2- 172.21.15.133:0/153734690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8300d1d00 0x7fb8300c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 -- 172.21.15.133:0/153734690 >> 172.21.15.133:0/153734690 conn(0x7fb83006c080 msgr2=0x7fb83006c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 -- 172.21.15.133:0/153734690 shutdown_connections 2024-04-25T10:05:28.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.102+0000 7fb835e26640 1 -- 172.21.15.133:0/153734690 wait complete. 2024-04-25T10:05:28.129 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:05:28.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5faad7a640 1 Processor -- start 2024-04-25T10:05:28.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5faad7a640 1 -- start start 2024-04-25T10:05:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5faad7a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5fa40ced10 0x7f5fa41685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5faad7a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa40d22c0 con 0x7f5fa415d9d0 2024-04-25T10:05:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5faad7a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa40d25a0 con 0x7f5fa40d3360 2024-04-25T10:05:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5faad7a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa40d2880 con 0x7f5fa40ced10 2024-04-25T10:05:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa8b16640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5fa415d9d0 0x7f5fa40b92e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36968/0 (socket says 172.21.15.133:36968) 2024-04-25T10:05:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa8b16640 1 -- 172.21.15.133:0/853434257 learned_addr learned my addr 172.21.15.133:0/853434257 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa9317640 1 --2- 172.21.15.133:0/853434257 >> v2:172.21.15.162:3300/0 conn(0x7f5fa40ced10 0x7f5fa41685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656229097 0 0) 0x7f5fa40d22c0 con 0x7f5fa415d9d0 2024-04-25T10:05:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f8c003530 con 0x7f5fa415d9d0 2024-04-25T10:05:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4034505954 0 0) 0x7f5f8c003530 con 0x7f5fa415d9d0 2024-04-25T10:05:28.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 >> v2:172.21.15.162:3300/0 conn(0x7f5fa40ced10 msgr2=0x7f5fa41685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 --2- 172.21.15.133:0/853434257 >> v2:172.21.15.162:3300/0 conn(0x7f5fa40ced10 0x7f5fa41685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 >> v1:172.21.15.162:6789/0 conn(0x7f5fa40d3360 legacy=0x7f5fa40ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa40d2b60 con 0x7f5fa415d9d0 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5f9c001a00 con 0x7f5fa415d9d0 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 >> v1:172.21.15.133:6789/0 conn(0x7f5fa415d9d0 legacy=0x7f5fa40b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 --2- 172.21.15.133:0/853434257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f8c0039c0 0x7f5f8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 --2- 172.21.15.133:0/853434257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f8c004810 0x7f5f8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 --2- 172.21.15.133:0/853434257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 0x7f5f8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa40d2880 con 0x7f5f8c0051d0 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c00c070 con 0x7f5f8c004810 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c00c270 con 0x7f5f8c0039c0 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa9317640 1 --2- 172.21.15.133:0/853434257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 0x7f5f8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa3fff640 1 --2- 172.21.15.133:0/853434257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f8c004810 0x7f5f8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa8b16640 1 --2- 172.21.15.133:0/853434257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f8c0039c0 0x7f5f8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa9317640 1 -- 172.21.15.133:0/853434257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f8c0039c0 msgr2=0x7f5f8c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa9317640 1 --2- 172.21.15.133:0/853434257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f8c0039c0 0x7f5f8c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa9317640 1 -- 172.21.15.133:0/853434257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f8c004810 msgr2=0x7f5f8c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa9317640 1 --2- 172.21.15.133:0/853434257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f8c004810 0x7f5f8c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.250+0000 7f5fa9317640 1 -- 172.21.15.133:0/853434257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c003530 con 0x7f5f8c0051d0 2024-04-25T10:05:28.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5fa9317640 1 -- 172.21.15.133:0/853434257 reap_dead start 2024-04-25T10:05:28.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5fa9317640 1 --2- 172.21.15.133:0/853434257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 0x7f5f8c00b9d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009a90 tx=0x7f5f9401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd0871e076d239c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94009690 con 0x7f5f8c0051d0 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f940a5660 con 0x7f5f8c0051d0 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5fa37fe640 1 -- 172.21.15.133:0/853434257 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f940a5a30 con 0x7f5f8c0051d0 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5faad7a640 1 -- 172.21.15.133:0/853434257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 msgr2=0x7f5f8c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5faad7a640 1 --2- 172.21.15.133:0/853434257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 0x7f5f8c00b9d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009a90 tx=0x7f5f9401ca70 comp rx=0 tx=0).stop 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5faad7a640 1 -- 172.21.15.133:0/853434257 shutdown_connections 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5faad7a640 1 --2- 172.21.15.133:0/853434257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 0x7f5f8c00b9d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5faad7a640 1 -- 172.21.15.133:0/853434257 >> 172.21.15.133:0/853434257 conn(0x7f5fa406c080 msgr2=0x7f5fa406c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5faad7a640 1 -- 172.21.15.133:0/853434257 shutdown_connections 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.254+0000 7f5faad7a640 1 -- 172.21.15.133:0/853434257 wait complete. 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5faad7a640 1 Processor -- start 2024-04-25T10:05:28.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5faad7a640 1 -- start start 2024-04-25T10:05:28.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5faad7a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 0x7f5fa40cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5faad7a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa40c9ee0 0x7f5fa40cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5faad7a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fa4153110 0x7f5fa4153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5faad7a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa40d1d00 con 0x7f5f8c0051d0 2024-04-25T10:05:28.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5faad7a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa415ddb0 con 0x7f5fa40c9ee0 2024-04-25T10:05:28.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5faad7a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa40d1e80 con 0x7f5fa4153110 2024-04-25T10:05:28.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa8b16640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 0x7f5fa40cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa8b16640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 0x7f5fa40cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39766/0 (socket says 172.21.15.133:39766) 2024-04-25T10:05:28.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa8b16640 1 -- 172.21.15.133:0/475781774 learned_addr learned my addr 172.21.15.133:0/475781774 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:28.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa3fff640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa40c9ee0 0x7f5fa40cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa9317640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fa4153110 0x7f5fa4153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa8b16640 1 -- 172.21.15.133:0/475781774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fa4153110 msgr2=0x7f5fa4153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa8b16640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fa4153110 0x7f5fa4153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa8b16640 1 -- 172.21.15.133:0/475781774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa40c9ee0 msgr2=0x7f5fa40cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa8b16640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa40c9ee0 0x7f5fa40cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa8b16640 1 -- 172.21.15.133:0/475781774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa4152810 con 0x7f5f8c0051d0 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa9317640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fa4153110 0x7f5fa4153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa3fff640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa40c9ee0 0x7f5fa40cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa8b16640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 0x7f5fa40cbf90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c001ab0 tx=0x7f5f9c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa17fa640 1 -- 172.21.15.133:0/475781774 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c019070 con 0x7f5f8c0051d0 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa17fa640 1 -- 172.21.15.133:0/475781774 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c00bb50 con 0x7f5f8c0051d0 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa17fa640 1 -- 172.21.15.133:0/475781774 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0070d0 con 0x7f5f8c0051d0 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5faad7a640 1 -- 172.21.15.133:0/475781774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa4152af0 con 0x7f5f8c0051d0 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5faad7a640 1 -- 172.21.15.133:0/475781774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa40cb790 con 0x7f5f8c0051d0 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5faad7a640 1 -- 172.21.15.133:0/475781774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa40ca2c0 con 0x7f5f8c0051d0 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa17fa640 1 -- 172.21.15.133:0/475781774 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f9c01c070 con 0x7f5f8c0051d0 2024-04-25T10:05:28.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa17fa640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f90066f90 0x7f5f90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa17fa640 1 -- 172.21.15.133:0/475781774 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f5f9c085030 con 0x7f5f8c0051d0 2024-04-25T10:05:28.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.258+0000 7f5fa3fff640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f90066f90 0x7f5f90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.262+0000 7f5fa3fff640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f90066f90 0x7f5f90069450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40ccf40 tx=0x7f5f98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.266+0000 7f5fa17fa640 1 -- 172.21.15.133:0/475781774 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f9c055f60 con 0x7f5f8c0051d0 2024-04-25T10:05:28.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:28.446+0000 7f5faad7a640 1 -- 172.21.15.133:0/475781774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5fa4069000 con 0x7f5f8c0051d0 2024-04-25T10:05:29.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.098+0000 7f5fa17fa640 1 -- 172.21.15.133:0/475781774 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5f9c055860 con 0x7f5f8c0051d0 2024-04-25T10:05:29.103 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:05:29.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 -- 172.21.15.133:0/475781774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f90066f90 msgr2=0x7f5f90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f90066f90 0x7f5f90069450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40ccf40 tx=0x7f5f98009040 comp rx=0 tx=0).stop 2024-04-25T10:05:29.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 -- 172.21.15.133:0/475781774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 msgr2=0x7f5fa40cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 0x7f5fa40cbf90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c001ab0 tx=0x7f5f9c00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 -- 172.21.15.133:0/475781774 shutdown_connections 2024-04-25T10:05:29.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fa4153110 0x7f5fa4153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fa40c9ee0 0x7f5fa40cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f90066f90 0x7f5f90069450 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 --2- 172.21.15.133:0/475781774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f8c0051d0 0x7f5fa40cbf90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 -- 172.21.15.133:0/475781774 >> 172.21.15.133:0/475781774 conn(0x7f5fa406c080 msgr2=0x7f5fa406c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 -- 172.21.15.133:0/475781774 shutdown_connections 2024-04-25T10:05:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.102+0000 7f5faad7a640 1 -- 172.21.15.133:0/475781774 wait complete. 2024-04-25T10:05:29.134 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:05:29.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.254+0000 7fc81cf3f640 1 Processor -- start 2024-04-25T10:05:29.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.254+0000 7fc81cf3f640 1 -- start start 2024-04-25T10:05:29.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81cf3f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8180cd820 0x7fc8180cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81cf3f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc81815ced0 0x7fc81815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81cf3f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8180ce2e0 con 0x7fc818145170 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81cf3f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81806db00 con 0x7fc81815ced0 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81cf3f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc818145550 con 0x7fc8180cd820 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81659c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc818145170 0x7fc81806df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36976/0 (socket says 172.21.15.133:36976) 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81659c640 1 -- 172.21.15.133:0/208271825 learned_addr learned my addr 172.21.15.133:0/208271825 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150444388 0 0) 0x7fc8180ce2e0 con 0x7fc818145170 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc815d9b640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8180cd820 0x7fc8180cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc800003530 con 0x7fc818145170 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc816d9d640 1 --2- 172.21.15.133:0/208271825 >> v2:172.21.15.162:3300/0 conn(0x7fc81815ced0 0x7fc81815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2007933795 0 0) 0x7fc800003530 con 0x7fc818145170 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8180cd820 msgr2=0x7fc8180cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8180cd820 0x7fc8180cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 >> v2:172.21.15.162:3300/0 conn(0x7fc81815ced0 msgr2=0x7fc81815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 --2- 172.21.15.133:0/208271825 >> v2:172.21.15.162:3300/0 conn(0x7fc81815ced0 0x7fc81815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81806d4b0 con 0x7fc818145170 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fc808001680 con 0x7fc818145170 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 >> v1:172.21.15.133:6789/0 conn(0x7fc818145170 legacy=0x7fc81806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8180cd820 0x7fc8000039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc800003f40 0x7fc800004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc800004d50 0x7fc8000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806db00 con 0x7fc800004d50 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818145550 con 0x7fc800003f40 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80000ba20 con 0x7fc8180cd820 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc816d9d640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc800004d50 0x7fc8000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81659c640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8180cd820 0x7fc8000039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc815d9b640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc800003f40 0x7fc800004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc816d9d640 1 -- 172.21.15.133:0/208271825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8180cd820 msgr2=0x7fc8000039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc816d9d640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8180cd820 0x7fc8000039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc816d9d640 1 -- 172.21.15.133:0/208271825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc800003f40 msgr2=0x7fc800004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc816d9d640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc800003f40 0x7fc800004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc816d9d640 1 -- 172.21.15.133:0/208271825 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc800003530 con 0x7fc800004d50 2024-04-25T10:05:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc816d9d640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc800004d50 0x7fc8000051d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc810007360 tx=0x7fc81001ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27aa4cd81c8d6a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8100b2070 con 0x7fc800004d50 2024-04-25T10:05:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8100a5d80 con 0x7fc800004d50 2024-04-25T10:05:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.258+0000 7fc81559a640 1 -- 172.21.15.133:0/208271825 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810009be0 con 0x7fc800004d50 2024-04-25T10:05:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 -- 172.21.15.133:0/208271825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc800004d50 msgr2=0x7fc8000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 --2- 172.21.15.133:0/208271825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc800004d50 0x7fc8000051d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc810007360 tx=0x7fc81001ce80 comp rx=0 tx=0).stop 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc816d9d640 1 -- 172.21.15.133:0/208271825 reap_dead start 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 -- 172.21.15.133:0/208271825 shutdown_connections 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 -- 172.21.15.133:0/208271825 >> 172.21.15.133:0/208271825 conn(0x7fc818066f70 msgr2=0x7fc81806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 -- 172.21.15.133:0/208271825 shutdown_connections 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 -- 172.21.15.133:0/208271825 wait complete. 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 Processor -- start 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 -- start start 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc800004d50 0x7fc818141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8180ac960 0x7fc8180acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81815d4c0 0x7fc81815d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81815cb70 con 0x7fc81815d4c0 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806e6a0 con 0x7fc8180ac960 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180cdca0 con 0x7fc800004d50 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc816d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81815d4c0 0x7fc81815d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc816d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81815d4c0 0x7fc81815d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39782/0 (socket says 172.21.15.133:39782) 2024-04-25T10:05:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc816d9d640 1 -- 172.21.15.133:0/1654649656 learned_addr learned my addr 172.21.15.133:0/1654649656 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81659c640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc800004d50 0x7fc818141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc815d9b640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8180ac960 0x7fc8180acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc816d9d640 1 -- 172.21.15.133:0/1654649656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc800004d50 msgr2=0x7fc818141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc816d9d640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc800004d50 0x7fc818141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc816d9d640 1 -- 172.21.15.133:0/1654649656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8180ac960 msgr2=0x7fc8180acde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc816d9d640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8180ac960 0x7fc8180acde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc816d9d640 1 -- 172.21.15.133:0/1654649656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8180ad3c0 con 0x7fc81815d4c0 2024-04-25T10:05:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc816d9d640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81815d4c0 0x7fc81815d940 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc8100b0040 tx=0x7fc800005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc806ffd640 1 -- 172.21.15.133:0/1654649656 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8100b2040 con 0x7fc81815d4c0 2024-04-25T10:05:29.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc806ffd640 1 -- 172.21.15.133:0/1654649656 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8100adbd0 con 0x7fc81815d4c0 2024-04-25T10:05:29.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 -- 172.21.15.133:0/1654649656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8180ad6a0 con 0x7fc81815d4c0 2024-04-25T10:05:29.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc806ffd640 1 -- 172.21.15.133:0/1654649656 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810009be0 con 0x7fc81815d4c0 2024-04-25T10:05:29.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.262+0000 7fc81cf3f640 1 -- 172.21.15.133:0/1654649656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8180adc60 con 0x7fc81815d4c0 2024-04-25T10:05:29.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.266+0000 7fc81cf3f640 1 -- 172.21.15.133:0/1654649656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc818068e80 con 0x7fc81815d4c0 2024-04-25T10:05:29.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.266+0000 7fc806ffd640 1 -- 172.21.15.133:0/1654649656 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8100bf060 con 0x7fc81815d4c0 2024-04-25T10:05:29.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.266+0000 7fc806ffd640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7f8066f60 0x7fc7f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.266+0000 7fc806ffd640 1 -- 172.21.15.133:0/1654649656 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc81011f2c0 con 0x7fc81815d4c0 2024-04-25T10:05:29.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.266+0000 7fc81659c640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7f8066f60 0x7fc7f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.266+0000 7fc81659c640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7f8066f60 0x7fc7f8069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc808000c10 tx=0x7fc808009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.270+0000 7fc806ffd640 1 -- 172.21.15.133:0/1654649656 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc810016b90 con 0x7fc81815d4c0 2024-04-25T10:05:29.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.430+0000 7fc81cf3f640 1 -- 172.21.15.133:0/1654649656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc8180b49d0 con 0x7fc81815d4c0 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc806ffd640 1 -- 172.21.15.133:0/1654649656 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc8100f0310 con 0x7fc81815d4c0 2024-04-25T10:05:29.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 -- 172.21.15.133:0/1654649656 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7f8066f60 msgr2=0x7fc7f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7f8066f60 0x7fc7f8069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc808000c10 tx=0x7fc808009040 comp rx=0 tx=0).stop 2024-04-25T10:05:29.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 -- 172.21.15.133:0/1654649656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81815d4c0 msgr2=0x7fc81815d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81815d4c0 0x7fc81815d940 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc8100b0040 tx=0x7fc800005840 comp rx=0 tx=0).stop 2024-04-25T10:05:29.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 -- 172.21.15.133:0/1654649656 shutdown_connections 2024-04-25T10:05:29.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7f8066f60 0x7fc7f8069420 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81815d4c0 0x7fc81815d940 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8180ac960 0x7fc8180acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 --2- 172.21.15.133:0/1654649656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc800004d50 0x7fc818141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 -- 172.21.15.133:0/1654649656 >> 172.21.15.133:0/1654649656 conn(0x7fc818066f70 msgr2=0x7fc81815e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 -- 172.21.15.133:0/1654649656 shutdown_connections 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.434+0000 7fc81cf3f640 1 -- 172.21.15.133:0/1654649656 wait complete. 2024-04-25T10:05:29.457 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T10:05:29.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb170f83640 1 Processor -- start 2024-04-25T10:05:29.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb170f83640 1 -- start start 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb170f83640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb16c0b1f70 0x7fb16c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb170f83640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb16c0ce330 0x7fb16c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb170f83640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb16c0c61d0 con 0x7fb16c15c9a0 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb170f83640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16c0cecc0 con 0x7fb16c0b1f70 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb170f83640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16c0c64b0 con 0x7fb16c0ce330 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb169d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb16c0b1f70 0x7fb16c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb169d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb16c0b1f70 0x7fb16c06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39792/0 (socket says 172.21.15.133:39792) 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb169d9b640 1 -- 172.21.15.133:0/2632665413 learned_addr learned my addr 172.21.15.133:0/2632665413 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb16ad9d640 1 --2- 172.21.15.133:0/2632665413 >> v2:172.21.15.162:3300/0 conn(0x7fb16c0ce330 0x7fb16c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb169d9b640 1 -- 172.21.15.133:0/2632665413 >> v2:172.21.15.162:3300/0 conn(0x7fb16c0ce330 msgr2=0x7fb16c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb169d9b640 1 --2- 172.21.15.133:0/2632665413 >> v2:172.21.15.162:3300/0 conn(0x7fb16c0ce330 0x7fb16c0ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb169d9b640 1 -- 172.21.15.133:0/2632665413 >> v1:172.21.15.162:6789/0 conn(0x7fb16c15c9a0 legacy=0x7fb16c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.574+0000 7fb169d9b640 1 -- 172.21.15.133:0/2632665413 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c06d4f0 con 0x7fb16c0b1f70 2024-04-25T10:05:29.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb169d9b640 1 --2- 172.21.15.133:0/2632665413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb16c0b1f70 0x7fb16c06dcf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0cc0f0 tx=0x7fb16001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5588d629c0c947 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb16959a640 1 -- 172.21.15.133:0/2632665413 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1600aa430 con 0x7fb16c0b1f70 2024-04-25T10:05:29.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb16959a640 1 -- 172.21.15.133:0/2632665413 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1600af070 con 0x7fb16c0b1f70 2024-04-25T10:05:29.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb16959a640 1 -- 172.21.15.133:0/2632665413 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1600aa7f0 con 0x7fb16c0b1f70 2024-04-25T10:05:29.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 -- 172.21.15.133:0/2632665413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb16c0b1f70 msgr2=0x7fb16c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 --2- 172.21.15.133:0/2632665413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb16c0b1f70 0x7fb16c06dcf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0cc0f0 tx=0x7fb16001c4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 -- 172.21.15.133:0/2632665413 shutdown_connections 2024-04-25T10:05:29.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 --2- 172.21.15.133:0/2632665413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb16c0b1f70 0x7fb16c06dcf0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 --2- 172.21.15.133:0/2632665413 >> v2:172.21.15.162:3300/0 conn(0x7fb16c0ce330 0x7fb16c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 -- 172.21.15.133:0/2632665413 >> 172.21.15.133:0/2632665413 conn(0x7fb16c067000 msgr2=0x7fb16c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 -- 172.21.15.133:0/2632665413 shutdown_connections 2024-04-25T10:05:29.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 -- 172.21.15.133:0/2632665413 wait complete. 2024-04-25T10:05:29.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 Processor -- start 2024-04-25T10:05:29.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 -- start start 2024-04-25T10:05:29.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb16c0b1f70 0x7fb16c158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16c0ce330 0x7fb16c141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb16c15c9a0 0x7fb16c15a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c15dad0 con 0x7fb16c15c9a0 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c15d950 con 0x7fb16c0b1f70 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb170f83640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c15dc50 con 0x7fb16c0ce330 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb16ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb16c15c9a0 0x7fb16c15a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb16ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb16c15c9a0 0x7fb16c15a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39794/0 (socket says 172.21.15.133:39794) 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.578+0000 7fb16ad9d640 1 -- 172.21.15.133:0/665562473 learned_addr learned my addr 172.21.15.133:0/665562473 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb16a59c640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb16c0b1f70 0x7fb16c158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb169d9b640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16c0ce330 0x7fb16c141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb16ad9d640 1 -- 172.21.15.133:0/665562473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16c0ce330 msgr2=0x7fb16c141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb16ad9d640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16c0ce330 0x7fb16c141f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb16ad9d640 1 -- 172.21.15.133:0/665562473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb16c0b1f70 msgr2=0x7fb16c158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb16ad9d640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb16c0b1f70 0x7fb16c158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb16ad9d640 1 -- 172.21.15.133:0/665562473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c159550 con 0x7fb16c15c9a0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb16a59c640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb16c0b1f70 0x7fb16c158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb16ad9d640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb16c15c9a0 0x7fb16c15a5f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb16400cc30 tx=0x7fb16400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb15affd640 1 -- 172.21.15.133:0/665562473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16400ce40 con 0x7fb16c15c9a0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb15affd640 1 -- 172.21.15.133:0/665562473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1640039a0 con 0x7fb16c15c9a0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb170f83640 1 -- 172.21.15.133:0/665562473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c159830 con 0x7fb16c15c9a0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb15affd640 1 -- 172.21.15.133:0/665562473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164003d50 con 0x7fb16c15c9a0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb170f83640 1 -- 172.21.15.133:0/665562473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c159df0 con 0x7fb16c15c9a0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb170f83640 1 -- 172.21.15.133:0/665562473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c06bc20 con 0x7fb16c15c9a0 2024-04-25T10:05:29.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb15affd640 1 -- 172.21.15.133:0/665562473 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb164003f70 con 0x7fb16c15c9a0 2024-04-25T10:05:29.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb15affd640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb14c066f60 0x7fb14c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb15affd640 1 -- 172.21.15.133:0/665562473 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb1640853f0 con 0x7fb16c15c9a0 2024-04-25T10:05:29.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb16a59c640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb14c066f60 0x7fb14c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.582+0000 7fb16a59c640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb14c066f60 0x7fb14c069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00a850 tx=0x7fb15c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.586+0000 7fb15affd640 1 -- 172.21.15.133:0/665562473 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb164014030 con 0x7fb16c15c9a0 2024-04-25T10:05:29.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:29.802+0000 7fb170f83640 1 -- 172.21.15.133:0/665562473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7fb16c068ef0 con 0x7fb16c15c9a0 2024-04-25T10:05:30.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.118+0000 7fb15affd640 1 -- 172.21.15.133:0/665562473 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v178) v1 ==== 148+0+0 (secure 0 0 0) 0x7fb1640564e0 con 0x7fb16c15c9a0 2024-04-25T10:05:30.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.118+0000 7fb170f83640 1 -- 172.21.15.133:0/665562473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb14c066f60 msgr2=0x7fb14c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.118+0000 7fb170f83640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb14c066f60 0x7fb14c069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00a850 tx=0x7fb15c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:30.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.118+0000 7fb170f83640 1 -- 172.21.15.133:0/665562473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb16c15c9a0 msgr2=0x7fb16c15a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.118+0000 7fb170f83640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb16c15c9a0 0x7fb16c15a5f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb16400cc30 tx=0x7fb16400a5c0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.118+0000 7fb170f83640 1 -- 172.21.15.133:0/665562473 shutdown_connections 2024-04-25T10:05:30.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.118+0000 7fb170f83640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb14c066f60 0x7fb14c069420 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.118+0000 7fb170f83640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb16c15c9a0 0x7fb16c15a5f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.122+0000 7fb170f83640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb16c0ce330 0x7fb16c141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.122+0000 7fb170f83640 1 --2- 172.21.15.133:0/665562473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb16c0b1f70 0x7fb16c158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.122+0000 7fb170f83640 1 -- 172.21.15.133:0/665562473 >> 172.21.15.133:0/665562473 conn(0x7fb16c067000 msgr2=0x7fb16c0c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.122+0000 7fb170f83640 1 -- 172.21.15.133:0/665562473 shutdown_connections 2024-04-25T10:05:30.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.122+0000 7fb170f83640 1 -- 172.21.15.133:0/665562473 wait complete. 2024-04-25T10:05:30.147 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T10:05:30.148 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:05:30.149 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:30.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.266+0000 7fb800d9f640 1 Processor -- start 2024-04-25T10:05:30.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb800d9f640 1 -- start start 2024-04-25T10:05:30.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb800d9f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7fc15b8c0 0x7fb7fc09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb800d9f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7fc0cd360 0x7fb7fc15f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb800d9f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7fc0999d0 con 0x7fb7fc15f4a0 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb800d9f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7fc15fe40 con 0x7fb7fc0cd360 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb800d9f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc0d0530 con 0x7fb7fc15b8c0 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb7fad9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7fc0cd360 0x7fb7fc15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb7fad9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7fc0cd360 0x7fb7fc15f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39802/0 (socket says 172.21.15.133:39802) 2024-04-25T10:05:30.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb7fad9d640 1 -- 172.21.15.133:0/3226343917 learned_addr learned my addr 172.21.15.133:0/3226343917 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:30.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb7fa59c640 1 --2- 172.21.15.133:0/3226343917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7fc15b8c0 0x7fb7fc09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb7fad9d640 1 -- 172.21.15.133:0/3226343917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7fc15b8c0 msgr2=0x7fb7fc09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb7fad9d640 1 --2- 172.21.15.133:0/3226343917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7fc15b8c0 0x7fb7fc09aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb7fad9d640 1 -- 172.21.15.133:0/3226343917 >> v1:172.21.15.162:6789/0 conn(0x7fb7fc15f4a0 legacy=0x7fb7fc0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.270+0000 7fb7fad9d640 1 -- 172.21.15.133:0/3226343917 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc099cb0 con 0x7fb7fc0cd360 2024-04-25T10:05:30.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7fad9d640 1 --2- 172.21.15.133:0/3226343917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc0cd360 0x7fb7fc15f900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009870 tx=0x7fb7f401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454857af8bc974e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7f959a640 1 -- 172.21.15.133:0/3226343917 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40aa610 con 0x7fb7fc0cd360 2024-04-25T10:05:30.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7f959a640 1 -- 172.21.15.133:0/3226343917 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f40af070 con 0x7fb7fc0cd360 2024-04-25T10:05:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7f959a640 1 -- 172.21.15.133:0/3226343917 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f40aa9d0 con 0x7fb7fc0cd360 2024-04-25T10:05:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 -- 172.21.15.133:0/3226343917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc0cd360 msgr2=0x7fb7fc15f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 --2- 172.21.15.133:0/3226343917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc0cd360 0x7fb7fc15f900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009870 tx=0x7fb7f401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 -- 172.21.15.133:0/3226343917 shutdown_connections 2024-04-25T10:05:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 --2- 172.21.15.133:0/3226343917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc0cd360 0x7fb7fc15f900 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 --2- 172.21.15.133:0/3226343917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7fc15b8c0 0x7fb7fc09aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 -- 172.21.15.133:0/3226343917 >> 172.21.15.133:0/3226343917 conn(0x7fb7fc0b3120 msgr2=0x7fb7fc0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 -- 172.21.15.133:0/3226343917 shutdown_connections 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 -- 172.21.15.133:0/3226343917 wait complete. 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 Processor -- start 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 -- start start 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc0cd360 0x7fb7fc15bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7fc15b8c0 0x7fb7fc154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7fc15f4a0 0x7fb7fc0af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc15be20 con 0x7fb7fc0cd360 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc15bca0 con 0x7fb7fc15f4a0 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc0b17a0 con 0x7fb7fc15b8c0 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc0cd360 0x7fb7fc15bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7fa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc0cd360 0x7fb7fc15bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39806/0 (socket says 172.21.15.133:39806) 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7fa59c640 1 -- 172.21.15.133:0/2699639605 learned_addr learned my addr 172.21.15.133:0/2699639605 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7f9d9b640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7fc15b8c0 0x7fb7fc154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7fad9d640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7fc15f4a0 0x7fb7fc0af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7fa59c640 1 -- 172.21.15.133:0/2699639605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7fc15b8c0 msgr2=0x7fb7fc154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7fa59c640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7fc15b8c0 0x7fb7fc154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7fa59c640 1 -- 172.21.15.133:0/2699639605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7fc15f4a0 msgr2=0x7fb7fc0af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7fa59c640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7fc15f4a0 0x7fb7fc0af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7fa59c640 1 -- 172.21.15.133:0/2699639605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc159d70 con 0x7fb7fc0cd360 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7fa59c640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc0cd360 0x7fb7fc15bfa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00d800 tx=0x7fb7ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7f2ffd640 1 -- 172.21.15.133:0/2699639605 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00b6c0 con 0x7fb7fc0cd360 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7f2ffd640 1 -- 172.21.15.133:0/2699639605 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec019070 con 0x7fb7fc0cd360 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb7f2ffd640 1 -- 172.21.15.133:0/2699639605 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00ba60 con 0x7fb7fc0cd360 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 -- 172.21.15.133:0/2699639605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc15e520 con 0x7fb7fc0cd360 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.274+0000 7fb800d9f640 1 -- 172.21.15.133:0/2699639605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc15a010 con 0x7fb7fc0cd360 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.278+0000 7fb800d9f640 1 -- 172.21.15.133:0/2699639605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc067c70 con 0x7fb7fc0cd360 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.278+0000 7fb7f2ffd640 1 -- 172.21.15.133:0/2699639605 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7ec00bc00 con 0x7fb7fc0cd360 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.278+0000 7fb7f2ffd640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7e0066f60 0x7fb7e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.278+0000 7fb7f9d9b640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7e0066f60 0x7fb7e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.278+0000 7fb7f2ffd640 1 -- 172.21.15.133:0/2699639605 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb7ec085320 con 0x7fb7fc0cd360 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.278+0000 7fb7f9d9b640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7e0066f60 0x7fb7e0069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0cf010 tx=0x7fb7e40094d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.282+0000 7fb7f2ffd640 1 -- 172.21.15.133:0/2699639605 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7ec0564a0 con 0x7fb7fc0cd360 2024-04-25T10:05:30.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.442+0000 7fb800d9f640 1 -- 172.21.15.133:0/2699639605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7fc073dd0 con 0x7fb7fc0cd360 2024-04-25T10:05:30.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.442+0000 7fb7f2ffd640 1 -- 172.21.15.133:0/2699639605 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+13317 (secure 0 0 0) 0x7fb7ec055da0 con 0x7fb7fc0cd360 2024-04-25T10:05:30.448 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:30.449 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":45,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:05:29.098756+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/205640021":"2024-04-26T10:05:25.860637+0000","172.21.15.133:6835/205640021":"2024-04-26T10:05:25.860637+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 -- 172.21.15.133:0/2699639605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7e0066f60 msgr2=0x7fb7e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7e0066f60 0x7fb7e0069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0cf010 tx=0x7fb7e40094d0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 -- 172.21.15.133:0/2699639605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc0cd360 msgr2=0x7fb7fc15bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc0cd360 0x7fb7fc15bfa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00d800 tx=0x7fb7ec00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 -- 172.21.15.133:0/2699639605 shutdown_connections 2024-04-25T10:05:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7fc15f4a0 0x7fb7fc0af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7fc15b8c0 0x7fb7fc154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7e0066f60 0x7fb7e0069420 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 --2- 172.21.15.133:0/2699639605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7fc0cd360 0x7fb7fc15bfa0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 -- 172.21.15.133:0/2699639605 >> 172.21.15.133:0/2699639605 conn(0x7fb7fc0b3120 msgr2=0x7fb7fc15aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 -- 172.21.15.133:0/2699639605 shutdown_connections 2024-04-25T10:05:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.446+0000 7fb800d9f640 1 -- 172.21.15.133:0/2699639605 wait complete. 2024-04-25T10:05:30.469 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:30.469 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.586+0000 7feceeb98640 1 Processor -- start 2024-04-25T10:05:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.586+0000 7feceeb98640 1 -- start start 2024-04-25T10:05:30.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7feceeb98640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece815dff0 0x7fece80cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7feceeb98640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fece80cb240 0x7fece815e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7feceeb98640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece80ce0b0 con 0x7fece815e910 2024-04-25T10:05:30.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7feceeb98640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece80cb680 con 0x7fece80cb240 2024-04-25T10:05:30.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7feceeb98640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece80ce350 con 0x7fece815dff0 2024-04-25T10:05:30.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7feced135640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fece815e910 0x7fece806ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36982/0 (socket says 172.21.15.133:36982) 2024-04-25T10:05:30.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7feced135640 1 -- 172.21.15.133:0/375981975 learned_addr learned my addr 172.21.15.133:0/375981975 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:30.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fecec934640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece815dff0 0x7fece80cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359486534 0 0) 0x7fece80ce0b0 con 0x7fece815e910 2024-04-25T10:05:30.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece7fff640 1 --2- 172.21.15.133:0/375981975 >> v2:172.21.15.162:3300/0 conn(0x7fece80cb240 0x7fece815e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd0003530 con 0x7fece815e910 2024-04-25T10:05:30.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1125006237 0 0) 0x7fecd0003530 con 0x7fece815e910 2024-04-25T10:05:30.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece815dff0 msgr2=0x7fece80cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece815dff0 0x7fece80cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 >> v2:172.21.15.162:3300/0 conn(0x7fece80cb240 msgr2=0x7fece815e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 --2- 172.21.15.133:0/375981975 >> v2:172.21.15.162:3300/0 conn(0x7fece80cb240 0x7fece815e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece80ce630 con 0x7fece815e910 2024-04-25T10:05:30.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fecd8001680 con 0x7fece815e910 2024-04-25T10:05:30.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 >> v1:172.21.15.133:6789/0 conn(0x7fece815e910 legacy=0x7fece806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece815dff0 0x7fecd00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecd0003f40 0x7fecd0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd0004d50 0x7fecd00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece80cb680 con 0x7fecd0003f40 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece80ce350 con 0x7fecd0004d50 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd000b9d0 con 0x7fece815dff0 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece7fff640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecd0003f40 0x7fecd0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fecec934640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece815dff0 0x7fecd00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7feced135640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd0004d50 0x7fecd00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece7fff640 1 -- 172.21.15.133:0/375981975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece815dff0 msgr2=0x7fecd00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece7fff640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece815dff0 0x7fecd00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece7fff640 1 -- 172.21.15.133:0/375981975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd0004d50 msgr2=0x7fecd00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece7fff640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd0004d50 0x7fecd00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.590+0000 7fece7fff640 1 -- 172.21.15.133:0/375981975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0003530 con 0x7fecd0003f40 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece7fff640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecd0003f40 0x7fecd0004810 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009ae0 tx=0x7fecdc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58272a9986203de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0b2070 con 0x7fecd0003f40 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc0a56a0 con 0x7fecd0003f40 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece77fe640 1 -- 172.21.15.133:0/375981975 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0096b0 con 0x7fecd0003f40 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 -- 172.21.15.133:0/375981975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecd0003f40 msgr2=0x7fecd0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 --2- 172.21.15.133:0/375981975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fecd0003f40 0x7fecd0004810 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009ae0 tx=0x7fecdc01ca50 comp rx=0 tx=0).stop 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feced135640 1 -- 172.21.15.133:0/375981975 reap_dead start 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 -- 172.21.15.133:0/375981975 shutdown_connections 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 -- 172.21.15.133:0/375981975 >> 172.21.15.133:0/375981975 conn(0x7fece80cc790 msgr2=0x7fece80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 -- 172.21.15.133:0/375981975 shutdown_connections 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 -- 172.21.15.133:0/375981975 wait complete. 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 Processor -- start 2024-04-25T10:05:30.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 -- start start 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd0003f40 0x7fece814d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fece815ae40 0x7fece815b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece80ae1f0 0x7fece80ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece80ceed0 con 0x7fece815ae40 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece80cebf0 con 0x7fecd0003f40 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece815ed80 con 0x7fece80ae1f0 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fece815ae40 0x7fece815b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fece815ae40 0x7fece815b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39816/0 (socket says 172.21.15.133:39816) 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece7fff640 1 -- 172.21.15.133:0/3395973349 learned_addr learned my addr 172.21.15.133:0/3395973349 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fecec934640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd0003f40 0x7fece814d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feced135640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece80ae1f0 0x7fece80ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece7fff640 1 -- 172.21.15.133:0/3395973349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece80ae1f0 msgr2=0x7fece80ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece7fff640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece80ae1f0 0x7fece80ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece7fff640 1 -- 172.21.15.133:0/3395973349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd0003f40 msgr2=0x7fece814d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece7fff640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd0003f40 0x7fece814d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece7fff640 1 -- 172.21.15.133:0/3395973349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece80ad150 con 0x7fece815ae40 2024-04-25T10:05:30.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece7fff640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fece815ae40 0x7fece815b2c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fecdc01c880 tx=0x7fecd0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece57fa640 1 -- 172.21.15.133:0/3395973349 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0bc030 con 0x7fece815ae40 2024-04-25T10:05:30.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece57fa640 1 -- 172.21.15.133:0/3395973349 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc00f000 con 0x7fece815ae40 2024-04-25T10:05:30.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7fece57fa640 1 -- 172.21.15.133:0/3395973349 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0b2040 con 0x7fece815ae40 2024-04-25T10:05:30.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 -- 172.21.15.133:0/3395973349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece80ad430 con 0x7fece815ae40 2024-04-25T10:05:30.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.594+0000 7feceeb98640 1 -- 172.21.15.133:0/3395973349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece80ad9f0 con 0x7fece815ae40 2024-04-25T10:05:30.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.598+0000 7feceeb98640 1 -- 172.21.15.133:0/3395973349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece8068e80 con 0x7fece815ae40 2024-04-25T10:05:30.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.598+0000 7fece57fa640 1 -- 172.21.15.133:0/3395973349 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fecdc0bf070 con 0x7fece815ae40 2024-04-25T10:05:30.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.598+0000 7fece57fa640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fecd4066f60 0x7fecd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.598+0000 7fece57fa640 1 -- 172.21.15.133:0/3395973349 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fecdc11ebb0 con 0x7fece815ae40 2024-04-25T10:05:30.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.598+0000 7fecec934640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fecd4066f60 0x7fecd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.598+0000 7fecec934640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fecd4066f60 0x7fecd4069420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fece0007440 tx=0x7fece000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.602+0000 7fece57fa640 1 -- 172.21.15.133:0/3395973349 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fecdc0167c0 con 0x7fece815ae40 2024-04-25T10:05:30.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.758+0000 7feceeb98640 1 -- 172.21.15.133:0/3395973349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fece806e340 con 0x7fece815ae40 2024-04-25T10:05:30.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.758+0000 7fece57fa640 1 -- 172.21.15.133:0/3395973349 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7fecdc0ef770 con 0x7fece815ae40 2024-04-25T10:05:30.763 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 -- 172.21.15.133:0/3395973349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fecd4066f60 msgr2=0x7fecd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fecd4066f60 0x7fecd4069420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fece0007440 tx=0x7fece000b040 comp rx=0 tx=0).stop 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 -- 172.21.15.133:0/3395973349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fece815ae40 msgr2=0x7fece815b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fece815ae40 0x7fece815b2c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fecdc01c880 tx=0x7fecd0005840 comp rx=0 tx=0).stop 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 -- 172.21.15.133:0/3395973349 shutdown_connections 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fece80ae1f0 0x7fece80ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fecd4066f60 0x7fecd4069420 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fece815ae40 0x7fece815b2c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 --2- 172.21.15.133:0/3395973349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecd0003f40 0x7fece814d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 -- 172.21.15.133:0/3395973349 >> 172.21.15.133:0/3395973349 conn(0x7fece80cc790 msgr2=0x7fece815f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 -- 172.21.15.133:0/3395973349 shutdown_connections 2024-04-25T10:05:30.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.762+0000 7feceeb98640 1 -- 172.21.15.133:0/3395973349 wait complete. 2024-04-25T10:05:30.781 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T10:05:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f2274c52640 1 Processor -- start 2024-04-25T10:05:30.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f2274c52640 1 -- start start 2024-04-25T10:05:30.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f2274c52640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22700b1f70 0x7f227006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f2274c52640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22700c61d0 con 0x7f227015c9a0 2024-04-25T10:05:30.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f2274c52640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22700c64b0 con 0x7f22700ce330 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f2274c52640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227006d4f0 con 0x7f22700b1f70 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226e59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f227015c9a0 0x7f22700c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36986/0 (socket says 172.21.15.133:36986) 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226e59c640 1 -- 172.21.15.133:0/175751685 learned_addr learned my addr 172.21.15.133:0/175751685 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f2265d9b640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22700b1f70 0x7f227006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577806951 0 0) 0x7f22700c61d0 con 0x7f227015c9a0 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f225c003530 con 0x7f227015c9a0 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1398115097 0 0) 0x7f225c003530 con 0x7f227015c9a0 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22700b1f70 msgr2=0x7f227006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22700b1f70 0x7f227006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 >> v1:172.21.15.162:6789/0 conn(0x7f22700ce330 legacy=0x7f227015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227006d7d0 con 0x7f227015c9a0 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2260001680 con 0x7f227015c9a0 2024-04-25T10:05:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 >> v1:172.21.15.133:6789/0 conn(0x7f227015c9a0 legacy=0x7f22700c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225c0039f0 0x7f225c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22700b1f70 0x7f225c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f225c004d50 0x7f225c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227006d4f0 con 0x7f225c004d50 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c00ba80 con 0x7f225c0039f0 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c00bc80 con 0x7f22700b1f70 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226ed9d640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f225c004d50 0x7f225c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f2265d9b640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22700b1f70 0x7f225c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226e59c640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225c0039f0 0x7f225c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226ed9d640 1 -- 172.21.15.133:0/175751685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22700b1f70 msgr2=0x7f225c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226ed9d640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22700b1f70 0x7f225c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226ed9d640 1 -- 172.21.15.133:0/175751685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225c0039f0 msgr2=0x7f225c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226ed9d640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225c0039f0 0x7f225c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.882+0000 7f226ed9d640 1 -- 172.21.15.133:0/175751685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c003530 con 0x7f225c004d50 2024-04-25T10:05:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f226ed9d640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f225c004d50 0x7f225c0051d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2268001e40 tx=0x7f226801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a1af6bcb82b4ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268001a90 con 0x7f225c004d50 2024-04-25T10:05:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2268003b80 con 0x7f225c004d50 2024-04-25T10:05:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f226dd9b640 1 -- 172.21.15.133:0/175751685 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268005080 con 0x7f225c004d50 2024-04-25T10:05:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f2274c52640 1 -- 172.21.15.133:0/175751685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f225c004d50 msgr2=0x7f225c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f2274c52640 1 --2- 172.21.15.133:0/175751685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f225c004d50 0x7f225c0051d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2268001e40 tx=0x7f226801c760 comp rx=0 tx=0).stop 2024-04-25T10:05:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f226ed9d640 1 -- 172.21.15.133:0/175751685 reap_dead start 2024-04-25T10:05:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f2274c52640 1 -- 172.21.15.133:0/175751685 shutdown_connections 2024-04-25T10:05:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f2274c52640 1 -- 172.21.15.133:0/175751685 >> 172.21.15.133:0/175751685 conn(0x7f2270067000 msgr2=0x7f227006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f2274c52640 1 -- 172.21.15.133:0/175751685 shutdown_connections 2024-04-25T10:05:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f2274c52640 1 -- 172.21.15.133:0/175751685 wait complete. 2024-04-25T10:05:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f2274c52640 1 Processor -- start 2024-04-25T10:05:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.886+0000 7f2274c52640 1 -- start start 2024-04-25T10:05:30.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f2274c52640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225c004d50 0x7f2270156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f2274c52640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22701f3b30 0x7f22700b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f2274c52640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22700b39f0 0x7f2270156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f2274c52640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227015d9d0 con 0x7f22700b39f0 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f2274c52640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227015d850 con 0x7f225c004d50 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f2274c52640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227015db50 con 0x7f22701f3b30 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22700b39f0 0x7f2270156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22700b39f0 0x7f2270156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39842/0 (socket says 172.21.15.133:39842) 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226ed9d640 1 -- 172.21.15.133:0/1702349762 learned_addr learned my addr 172.21.15.133:0/1702349762 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f2265d9b640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22701f3b30 0x7f22700b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226e59c640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225c004d50 0x7f2270156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226ed9d640 1 -- 172.21.15.133:0/1702349762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22701f3b30 msgr2=0x7f22700b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226ed9d640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22701f3b30 0x7f22700b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226ed9d640 1 -- 172.21.15.133:0/1702349762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225c004d50 msgr2=0x7f2270156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226ed9d640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225c004d50 0x7f2270156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226ed9d640 1 -- 172.21.15.133:0/1702349762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22701f4310 con 0x7f22700b39f0 2024-04-25T10:05:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226ed9d640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22700b39f0 0x7f2270156820 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f226801cc40 tx=0x7f225c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f22677fe640 1 -- 172.21.15.133:0/1702349762 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268005840 con 0x7f22700b39f0 2024-04-25T10:05:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f22677fe640 1 -- 172.21.15.133:0/1702349762 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22680012e0 con 0x7f22700b39f0 2024-04-25T10:05:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f2274c52640 1 -- 172.21.15.133:0/1702349762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2270157e20 con 0x7f22700b39f0 2024-04-25T10:05:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f22677fe640 1 -- 172.21.15.133:0/1702349762 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268001a90 con 0x7f22700b39f0 2024-04-25T10:05:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f2274c52640 1 -- 172.21.15.133:0/1702349762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2270158610 con 0x7f22700b39f0 2024-04-25T10:05:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f2274c52640 1 -- 172.21.15.133:0/1702349762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2270068ef0 con 0x7f22700b39f0 2024-04-25T10:05:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f22677fe640 1 -- 172.21.15.133:0/1702349762 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22680bb020 con 0x7f22700b39f0 2024-04-25T10:05:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f22677fe640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2250066f90 0x7f2250069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f22677fe640 1 -- 172.21.15.133:0/1702349762 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f226811f550 con 0x7f22700b39f0 2024-04-25T10:05:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226e59c640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2250066f90 0x7f2250069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.890+0000 7f226e59c640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2250066f90 0x7f2250069450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2260000f70 tx=0x7f2260000ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:30.894+0000 7f22677fe640 1 -- 172.21.15.133:0/1702349762 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22680ac020 con 0x7f22700b39f0 2024-04-25T10:05:31.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.114+0000 7f2274c52640 1 -- 172.21.15.133:0/1702349762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f227015ea40 con 0x7f22700b39f0 2024-04-25T10:05:31.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.118+0000 7f22677fe640 1 -- 172.21.15.133:0/1702349762 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v179) v1 ==== 146+0+0 (secure 0 0 0) 0x7f2268016470 con 0x7f22700b39f0 2024-04-25T10:05:31.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 -- 172.21.15.133:0/1702349762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2250066f90 msgr2=0x7f2250069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2250066f90 0x7f2250069450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2260000f70 tx=0x7f2260000ee0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 -- 172.21.15.133:0/1702349762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22700b39f0 msgr2=0x7f2270156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22700b39f0 0x7f2270156820 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f226801cc40 tx=0x7f225c005840 comp rx=0 tx=0).stop 2024-04-25T10:05:31.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 -- 172.21.15.133:0/1702349762 shutdown_connections 2024-04-25T10:05:31.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2250066f90 0x7f2250069450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22700b39f0 0x7f2270156820 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22701f3b30 0x7f22700b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 --2- 172.21.15.133:0/1702349762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f225c004d50 0x7f2270156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 -- 172.21.15.133:0/1702349762 >> 172.21.15.133:0/1702349762 conn(0x7f2270067000 msgr2=0x7f227015b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 -- 172.21.15.133:0/1702349762 shutdown_connections 2024-04-25T10:05:31.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.122+0000 7f2274c52640 1 -- 172.21.15.133:0/1702349762 wait complete. 2024-04-25T10:05:31.142 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:05:31.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f0301d54640 1 Processor -- start 2024-04-25T10:05:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f0301d54640 1 -- start start 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f0301d54640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f02fc0d2030 0x7f02fc0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f0301d54640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02fc15ebd0 con 0x7f02fc15d9a0 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f0301d54640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02fc15eeb0 con 0x7f02fc0ce600 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f0301d54640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02fc0d0330 con 0x7f02fc0d2030 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02fb7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f02fc15d9a0 0x7f02fc0d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36992/0 (socket says 172.21.15.133:36992) 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02fb7fe640 1 -- 172.21.15.133:0/3833530723 learned_addr learned my addr 172.21.15.133:0/3833530723 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080543979 0 0) 0x7f02fc15ebd0 con 0x7f02fc15d9a0 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02e8003530 con 0x7f02fc15d9a0 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02fbfff640 1 --2- 172.21.15.133:0/3833530723 >> v2:172.21.15.162:3300/0 conn(0x7f02fc0d2030 0x7f02fc0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1966563990 0 0) 0x7f02e8003530 con 0x7f02fc15d9a0 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 >> v2:172.21.15.162:3300/0 conn(0x7f02fc0d2030 msgr2=0x7f02fc0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 --2- 172.21.15.133:0/3833530723 >> v2:172.21.15.162:3300/0 conn(0x7f02fc0d2030 0x7f02fc0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 >> v1:172.21.15.162:6789/0 conn(0x7f02fc0ce600 legacy=0x7f02fc0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc0d0610 con 0x7f02fc15d9a0 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f02ec001680 con 0x7f02fc15d9a0 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 >> v1:172.21.15.133:6789/0 conn(0x7f02fc15d9a0 legacy=0x7f02fc0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 --2- 172.21.15.133:0/3833530723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 0x7f02e8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 --2- 172.21.15.133:0/3833530723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02e8004810 0x7f02e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 --2- 172.21.15.133:0/3833530723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02e8005220 0x7f02e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0d0330 con 0x7f02e8003a10 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e800c170 con 0x7f02e8004810 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e800c370 con 0x7f02e8005220 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02fb7fe640 1 --2- 172.21.15.133:0/3833530723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 0x7f02e8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02f3fff640 1 --2- 172.21.15.133:0/3833530723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02e8004810 0x7f02e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02fbfff640 1 --2- 172.21.15.133:0/3833530723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02e8005220 0x7f02e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02fb7fe640 1 -- 172.21.15.133:0/3833530723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02e8005220 msgr2=0x7f02e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02fb7fe640 1 --2- 172.21.15.133:0/3833530723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02e8005220 0x7f02e800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02fb7fe640 1 -- 172.21.15.133:0/3833530723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02e8004810 msgr2=0x7f02e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02fb7fe640 1 --2- 172.21.15.133:0/3833530723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02e8004810 0x7f02e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02fbfff640 1 -- 172.21.15.133:0/3833530723 reap_dead start 2024-04-25T10:05:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.258+0000 7f02fb7fe640 1 -- 172.21.15.133:0/3833530723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e8003530 con 0x7f02e8003a10 2024-04-25T10:05:31.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02fb7fe640 1 --2- 172.21.15.133:0/3833530723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 0x7f02e8003e30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00c0e0 tx=0x7f02ec01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891e72e7b79c059c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec007c30 con 0x7f02e8003a10 2024-04-25T10:05:31.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ec0a9020 con 0x7f02e8003a10 2024-04-25T10:05:31.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02faffd640 1 -- 172.21.15.133:0/3833530723 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec005ae0 con 0x7f02e8003a10 2024-04-25T10:05:31.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 -- 172.21.15.133:0/3833530723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 msgr2=0x7f02e8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 --2- 172.21.15.133:0/3833530723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 0x7f02e8003e30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00c0e0 tx=0x7f02ec01cab0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 -- 172.21.15.133:0/3833530723 shutdown_connections 2024-04-25T10:05:31.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 --2- 172.21.15.133:0/3833530723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 0x7f02e8003e30 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 -- 172.21.15.133:0/3833530723 >> 172.21.15.133:0/3833530723 conn(0x7f02fc06bf80 msgr2=0x7f02fc06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 -- 172.21.15.133:0/3833530723 shutdown_connections 2024-04-25T10:05:31.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 -- 172.21.15.133:0/3833530723 wait complete. 2024-04-25T10:05:31.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 Processor -- start 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 -- start start 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 0x7f02fc0c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02fc0c7360 0x7f02fc14bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02fc0c9690 0x7f02fc0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0d1eb0 con 0x7f02e8003a10 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0d1d30 con 0x7f02fc0c9690 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f0301d54640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0cedf0 con 0x7f02fc0c7360 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02fb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 0x7f02fc0c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02fb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 0x7f02fc0c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39862/0 (socket says 172.21.15.133:39862) 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02fb7fe640 1 -- 172.21.15.133:0/1113468485 learned_addr learned my addr 172.21.15.133:0/1113468485 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02f3fff640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02fc0c7360 0x7f02fc14bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02fb7fe640 1 -- 172.21.15.133:0/1113468485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02fc0c7360 msgr2=0x7f02fc14bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02fbfff640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02fc0c9690 0x7f02fc0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02fb7fe640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02fc0c7360 0x7f02fc14bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02fb7fe640 1 -- 172.21.15.133:0/1113468485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02fc0c9690 msgr2=0x7f02fc0c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02fb7fe640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02fc0c9690 0x7f02fc0c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02fb7fe640 1 -- 172.21.15.133:0/1113468485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc0c78b0 con 0x7f02e8003a10 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.262+0000 7f02f3fff640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02fc0c7360 0x7f02fc14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f02fb7fe640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 0x7f02fc0c8950 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00c0e0 tx=0x7f02e8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f02f8ff9640 1 -- 172.21.15.133:0/1113468485 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec004ab0 con 0x7f02e8003a10 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f02f8ff9640 1 -- 172.21.15.133:0/1113468485 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ec0b2070 con 0x7f02e8003a10 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f0301d54640 1 -- 172.21.15.133:0/1113468485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02fc0c7b90 con 0x7f02e8003a10 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f02f8ff9640 1 -- 172.21.15.133:0/1113468485 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec0070d0 con 0x7f02e8003a10 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f0301d54640 1 -- 172.21.15.133:0/1113468485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02fc0c8150 con 0x7f02e8003a10 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f0301d54640 1 -- 172.21.15.133:0/1113468485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02fc068e80 con 0x7f02e8003a10 2024-04-25T10:05:31.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f02f8ff9640 1 -- 172.21.15.133:0/1113468485 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02ec007270 con 0x7f02e8003a10 2024-04-25T10:05:31.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f02f8ff9640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02dc066f60 0x7f02dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f02f8ff9640 1 -- 172.21.15.133:0/1113468485 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f02ec11e240 con 0x7f02e8003a10 2024-04-25T10:05:31.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f02f3fff640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02dc066f60 0x7f02dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.266+0000 7f02f3fff640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02dc066f60 0x7f02dc069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f02e4007fd0 tx=0x7f02e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.270+0000 7f02f8ff9640 1 -- 172.21.15.133:0/1113468485 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f02ec016830 con 0x7f02e8003a10 2024-04-25T10:05:31.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.418+0000 7f0301d54640 1 -- 172.21.15.133:0/1113468485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f02fc0b6180 con 0x7f02e8003a10 2024-04-25T10:05:31.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f02f8ff9640 1 -- 172.21.15.133:0/1113468485 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f02ec0ef4e0 con 0x7f02e8003a10 2024-04-25T10:05:31.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 -- 172.21.15.133:0/1113468485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02dc066f60 msgr2=0x7f02dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02dc066f60 0x7f02dc069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f02e4007fd0 tx=0x7f02e4009040 comp rx=0 tx=0).stop 2024-04-25T10:05:31.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 -- 172.21.15.133:0/1113468485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 msgr2=0x7f02fc0c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 0x7f02fc0c8950 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00c0e0 tx=0x7f02e8005840 comp rx=0 tx=0).stop 2024-04-25T10:05:31.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 -- 172.21.15.133:0/1113468485 shutdown_connections 2024-04-25T10:05:31.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02fc0c9690 0x7f02fc0c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02fc0c7360 0x7f02fc14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02dc066f60 0x7f02dc069420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 --2- 172.21.15.133:0/1113468485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02e8003a10 0x7f02fc0c8950 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 -- 172.21.15.133:0/1113468485 >> 172.21.15.133:0/1113468485 conn(0x7f02fc06bf80 msgr2=0x7f02fc0b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 -- 172.21.15.133:0/1113468485 shutdown_connections 2024-04-25T10:05:31.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.422+0000 7f0301d54640 1 -- 172.21.15.133:0/1113468485 wait complete. 2024-04-25T10:05:31.444 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:31.444 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:31.487 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:31.535 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:31.536 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:05:31.536 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:31.542 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:31.542 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:05:31.542 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:31.548 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:31.549 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:05:31.549 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:31.588 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:31.589 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:05:31.589 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:31.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:31.595 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:05:31.596 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:31.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.746+0000 7f7e2f13b640 1 Processor -- start 2024-04-25T10:05:31.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.746+0000 7f7e2f13b640 1 -- start start 2024-04-25T10:05:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2f13b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e2815b6d0 0x7f7e280af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2f13b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7e2806c3b0 0x7f7e2806ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2f13b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e280ada90 con 0x7f7e280ad520 2024-04-25T10:05:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2f13b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2806e390 con 0x7f7e2806c3b0 2024-04-25T10:05:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2f13b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2806e670 con 0x7f7e2815b6d0 2024-04-25T10:05:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2d6d8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7e2806c3b0 0x7f7e2806ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2d6d8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7e2806c3b0 0x7f7e2806ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39876/0 (socket says 172.21.15.133:39876) 2024-04-25T10:05:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2d6d8640 1 -- 172.21.15.133:0/556121720 learned_addr learned my addr 172.21.15.133:0/556121720 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:31.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2d6d8640 1 -- 172.21.15.133:0/556121720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e2815b6d0 msgr2=0x7f7e280af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2d6d8640 1 --2- 172.21.15.133:0/556121720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e2815b6d0 0x7f7e280af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2d6d8640 1 -- 172.21.15.133:0/556121720 >> v1:172.21.15.162:6789/0 conn(0x7f7e280ad520 legacy=0x7f7e2806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2d6d8640 1 -- 172.21.15.133:0/556121720 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2806e950 con 0x7f7e2806c3b0 2024-04-25T10:05:31.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2d6d8640 1 --2- 172.21.15.133:0/556121720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e2806c3b0 0x7f7e2806ee50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009e50 tx=0x7f7e2001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6e1c114e33703a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e277fe640 1 -- 172.21.15.133:0/556121720 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e200a57d0 con 0x7f7e2806c3b0 2024-04-25T10:05:31.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e277fe640 1 -- 172.21.15.133:0/556121720 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e200a5970 con 0x7f7e2806c3b0 2024-04-25T10:05:31.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e277fe640 1 -- 172.21.15.133:0/556121720 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e200a5d40 con 0x7f7e2806c3b0 2024-04-25T10:05:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2f13b640 1 -- 172.21.15.133:0/556121720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e2806c3b0 msgr2=0x7f7e2806ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2f13b640 1 --2- 172.21.15.133:0/556121720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e2806c3b0 0x7f7e2806ee50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009e50 tx=0x7f7e2001c930 comp rx=0 tx=0).stop 2024-04-25T10:05:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2f13b640 1 -- 172.21.15.133:0/556121720 shutdown_connections 2024-04-25T10:05:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2f13b640 1 --2- 172.21.15.133:0/556121720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e2806c3b0 0x7f7e2806ee50 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2f13b640 1 --2- 172.21.15.133:0/556121720 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e2815b6d0 0x7f7e280af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.750+0000 7f7e2f13b640 1 -- 172.21.15.133:0/556121720 >> 172.21.15.133:0/556121720 conn(0x7f7e28071a80 msgr2=0x7f7e28071e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 -- 172.21.15.133:0/556121720 shutdown_connections 2024-04-25T10:05:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 -- 172.21.15.133:0/556121720 wait complete. 2024-04-25T10:05:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 Processor -- start 2024-04-25T10:05:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 -- start start 2024-04-25T10:05:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e2806c3b0 0x7f7e2814f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e280ad520 0x7f7e2813b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e2815b6d0 0x7f7e2813b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e28168760 con 0x7f7e280ad520 2024-04-25T10:05:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2815c550 con 0x7f7e2806c3b0 2024-04-25T10:05:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e281688e0 con 0x7f7e2815b6d0 2024-04-25T10:05:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e280ad520 0x7f7e2813b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e280ad520 0x7f7e2813b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39878/0 (socket says 172.21.15.133:39878) 2024-04-25T10:05:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e27fff640 1 -- 172.21.15.133:0/2928672533 learned_addr learned my addr 172.21.15.133:0/2928672533 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e27fff640 1 -- 172.21.15.133:0/2928672533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e2815b6d0 msgr2=0x7f7e2813b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e27fff640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e2815b6d0 0x7f7e2813b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e27fff640 1 -- 172.21.15.133:0/2928672533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e2806c3b0 msgr2=0x7f7e2814f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e27fff640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e2806c3b0 0x7f7e2814f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e27fff640 1 -- 172.21.15.133:0/2928672533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2814d680 con 0x7f7e280ad520 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e27fff640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e280ad520 0x7f7e2813b300 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400b3e0 tx=0x7f7e1400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e257fa640 1 -- 172.21.15.133:0/2928672533 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14015020 con 0x7f7e280ad520 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e257fa640 1 -- 172.21.15.133:0/2928672533 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e14011850 con 0x7f7e280ad520 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 -- 172.21.15.133:0/2928672533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2814d960 con 0x7f7e280ad520 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e257fa640 1 -- 172.21.15.133:0/2928672533 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14011c40 con 0x7f7e280ad520 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 -- 172.21.15.133:0/2928672533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2814df20 con 0x7f7e280ad520 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.754+0000 7f7e2f13b640 1 -- 172.21.15.133:0/2928672533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e28068ef0 con 0x7f7e280ad520 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.758+0000 7f7e257fa640 1 -- 172.21.15.133:0/2928672533 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e14018020 con 0x7f7e280ad520 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.758+0000 7f7e257fa640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e0c066f60 0x7f7e0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.758+0000 7f7e257fa640 1 -- 172.21.15.133:0/2928672533 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7e14084de0 con 0x7f7e280ad520 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.758+0000 7f7e2ced7640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e0c066f60 0x7f7e0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.758+0000 7f7e2ced7640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e0c066f60 0x7f7e0c069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009690 tx=0x7f7e1c006c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.762+0000 7f7e257fa640 1 -- 172.21.15.133:0/2928672533 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e1401f020 con 0x7f7e280ad520 2024-04-25T10:05:31.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.938+0000 7f7e2f13b640 1 -- 172.21.15.133:0/2928672533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e28067080 con 0x7f7e280ad520 2024-04-25T10:05:31.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.938+0000 7f7e257fa640 1 -- 172.21.15.133:0/2928672533 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f7e140172c0 con 0x7f7e280ad520 2024-04-25T10:05:31.944 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:31.945 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2887512873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2887512873},{"type":"v1","addr":"172.21.15.162:6833","nonce":2887512873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1044882076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1044882076},{"type":"v1","addr":"172.21.15.162:6835","nonce":1044882076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1965748675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1965748675},{"type":"v1","addr":"172.21.15.133:6836","nonce":1965748675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2776713217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2776713217},{"type":"v1","addr":"172.21.15.133:6837","nonce":2776713217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2024-04-25T10:05:31.945 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 19 2024-04-25T10:05:31.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 -- 172.21.15.133:0/2928672533 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e0c066f60 msgr2=0x7f7e0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e0c066f60 0x7f7e0c069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009690 tx=0x7f7e1c006c00 comp rx=0 tx=0).stop 2024-04-25T10:05:31.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 -- 172.21.15.133:0/2928672533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e280ad520 msgr2=0x7f7e2813b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e280ad520 0x7f7e2813b300 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400b3e0 tx=0x7f7e1400b8a0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 -- 172.21.15.133:0/2928672533 shutdown_connections 2024-04-25T10:05:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e2815b6d0 0x7f7e2813b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e0c066f60 0x7f7e0c069420 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e280ad520 0x7f7e2813b300 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 --2- 172.21.15.133:0/2928672533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e2806c3b0 0x7f7e2814f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 -- 172.21.15.133:0/2928672533 >> 172.21.15.133:0/2928672533 conn(0x7f7e28071a80 msgr2=0x7f7e2815cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 -- 172.21.15.133:0/2928672533 shutdown_connections 2024-04-25T10:05:31.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:31.942+0000 7f7e2f13b640 1 -- 172.21.15.133:0/2928672533 wait complete. 2024-04-25T10:05:31.966 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:05:31.966 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:05:31.966 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:05:31.966 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:31.966 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:05:31.966 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:05:31.967 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:05:31.967 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:31.967 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:05:31.967 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:05:31.967 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:05:31.967 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:31.967 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:05:31.968 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:05:31.968 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:05:31.968 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:31.981 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:05:31.981 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:05:31.982 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:05:31.982 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:05:31.983 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:05:31.983 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:05:31.984 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:05:31.984 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:05:32.027 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:05:32.028 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:05:32.028 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:32.029 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:32.030 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:05:32.030 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:05:32.030 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:32.031 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:32.031 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:05:32.089 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:05:32.090 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:05:32.100 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:05:32.101 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:05:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.134+0000 7f86fc61f640 1 Processor -- start 2024-04-25T10:05:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.134+0000 7f86fc61f640 1 -- start start 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fc61f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f86f415d9a0 0x7f86f40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fc61f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86f40d2030 0x7f86f40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fc61f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f415ebd0 con 0x7f86f40ce600 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fc61f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f415eeb0 con 0x7f86f40d2030 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fc61f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f40d0330 con 0x7f86f415d9a0 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fabbc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86f40d2030 0x7f86f40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fabbc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86f40d2030 0x7f86f40cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39908/0 (socket says 172.21.15.133:39908) 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fabbc640 1 -- 172.21.15.133:0/1776138448 learned_addr learned my addr 172.21.15.133:0/1776138448 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fa3bb640 1 --2- 172.21.15.133:0/1776138448 >> v2:172.21.15.162:3300/0 conn(0x7f86f415d9a0 0x7f86f40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fabbc640 1 -- 172.21.15.133:0/1776138448 >> v2:172.21.15.162:3300/0 conn(0x7f86f415d9a0 msgr2=0x7f86f40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fabbc640 1 --2- 172.21.15.133:0/1776138448 >> v2:172.21.15.162:3300/0 conn(0x7f86f415d9a0 0x7f86f40d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fabbc640 1 -- 172.21.15.133:0/1776138448 >> v1:172.21.15.162:6789/0 conn(0x7f86f40ce600 legacy=0x7f86f40d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fabbc640 1 -- 172.21.15.133:0/1776138448 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f40d0610 con 0x7f86f40d2030 2024-04-25T10:05:32.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fabbc640 1 --2- 172.21.15.133:0/1776138448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86f40d2030 0x7f86f40cea90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f86f40d4750 tx=0x7f86ec01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86772cc1da84ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86f93b9640 1 -- 172.21.15.133:0/1776138448 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0a56f0 con 0x7f86f40d2030 2024-04-25T10:05:32.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86f93b9640 1 -- 172.21.15.133:0/1776138448 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ec0a5890 con 0x7f86f40d2030 2024-04-25T10:05:32.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86f93b9640 1 -- 172.21.15.133:0/1776138448 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0a5c60 con 0x7f86f40d2030 2024-04-25T10:05:32.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fc61f640 1 -- 172.21.15.133:0/1776138448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86f40d2030 msgr2=0x7f86f40cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.138+0000 7f86fc61f640 1 --2- 172.21.15.133:0/1776138448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86f40d2030 0x7f86f40cea90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f86f40d4750 tx=0x7f86ec01c960 comp rx=0 tx=0).stop 2024-04-25T10:05:32.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 -- 172.21.15.133:0/1776138448 shutdown_connections 2024-04-25T10:05:32.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 --2- 172.21.15.133:0/1776138448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86f40d2030 0x7f86f40cea90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 --2- 172.21.15.133:0/1776138448 >> v2:172.21.15.162:3300/0 conn(0x7f86f415d9a0 0x7f86f40d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 -- 172.21.15.133:0/1776138448 >> 172.21.15.133:0/1776138448 conn(0x7f86f406bf80 msgr2=0x7f86f406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 -- 172.21.15.133:0/1776138448 shutdown_connections 2024-04-25T10:05:32.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 -- 172.21.15.133:0/1776138448 wait complete. 2024-04-25T10:05:32.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 Processor -- start 2024-04-25T10:05:32.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 -- start start 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f40ce600 0x7f86f40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86f40d2030 0x7f86f4154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f415d9a0 0x7f86f4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f40b9820 con 0x7f86f40d2030 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f40b96a0 con 0x7f86f415d9a0 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f415e590 con 0x7f86f40ce600 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86f9bba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86f40d2030 0x7f86f4154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86f9bba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86f40d2030 0x7f86f4154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39924/0 (socket says 172.21.15.133:39924) 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86f9bba640 1 -- 172.21.15.133:0/3744688472 learned_addr learned my addr 172.21.15.133:0/3744688472 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fa3bb640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f40ce600 0x7f86f40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fabbc640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f415d9a0 0x7f86f4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86f9bba640 1 -- 172.21.15.133:0/3744688472 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f40ce600 msgr2=0x7f86f40cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86f9bba640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f40ce600 0x7f86f40cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86f9bba640 1 -- 172.21.15.133:0/3744688472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f415d9a0 msgr2=0x7f86f4154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86f9bba640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f415d9a0 0x7f86f4154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86f9bba640 1 -- 172.21.15.133:0/3744688472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f4156a80 con 0x7f86f40d2030 2024-04-25T10:05:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fa3bb640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f40ce600 0x7f86f40cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86f9bba640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86f40d2030 0x7f86f4154310 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c450 tx=0x7f86e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86eaffd640 1 -- 172.21.15.133:0/3744688472 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4016020 con 0x7f86f40d2030 2024-04-25T10:05:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86eaffd640 1 -- 172.21.15.133:0/3744688472 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e40052d0 con 0x7f86f40d2030 2024-04-25T10:05:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 -- 172.21.15.133:0/3744688472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f4156d20 con 0x7f86f40d2030 2024-04-25T10:05:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86eaffd640 1 -- 172.21.15.133:0/3744688472 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4007260 con 0x7f86f40d2030 2024-04-25T10:05:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 -- 172.21.15.133:0/3744688472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f41572e0 con 0x7f86f40d2030 2024-04-25T10:05:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.142+0000 7f86fc61f640 1 -- 172.21.15.133:0/3744688472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f4068e80 con 0x7f86f40d2030 2024-04-25T10:05:32.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.146+0000 7f86eaffd640 1 -- 172.21.15.133:0/3744688472 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86e4007400 con 0x7f86f40d2030 2024-04-25T10:05:32.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.146+0000 7f86eaffd640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86d8066f60 0x7f86d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.146+0000 7f86eaffd640 1 -- 172.21.15.133:0/3744688472 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f86e4084720 con 0x7f86f40d2030 2024-04-25T10:05:32.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.146+0000 7f86fa3bb640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86d8066f60 0x7f86d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.146+0000 7f86fa3bb640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86d8066f60 0x7f86d8069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f86f40b52a0 tx=0x7f86f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.150+0000 7f86eaffd640 1 -- 172.21.15.133:0/3744688472 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86e4020020 con 0x7f86f40d2030 2024-04-25T10:05:32.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T10:05:32.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T10:05:32.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:05:32.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:05:32.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:32.302+0000 7f86fc61f640 1 -- 172.21.15.133:0/3744688472 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f86f40c70e0 con 0x7f86f40d2030 2024-04-25T10:05:33.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.134+0000 7f86eaffd640 1 -- 172.21.15.133:0/3744688472 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7f86e4005480 con 0x7f86f40d2030 2024-04-25T10:05:33.138 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:05:33.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 -- 172.21.15.133:0/3744688472 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86d8066f60 msgr2=0x7f86d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86d8066f60 0x7f86d8069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f86f40b52a0 tx=0x7f86f0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:33.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 -- 172.21.15.133:0/3744688472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86f40d2030 msgr2=0x7f86f4154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86f40d2030 0x7f86f4154310 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c450 tx=0x7f86e400c910 comp rx=0 tx=0).stop 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 -- 172.21.15.133:0/3744688472 shutdown_connections 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86f415d9a0 0x7f86f4154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86d8066f60 0x7f86d8069420 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86f40d2030 0x7f86f4154310 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 --2- 172.21.15.133:0/3744688472 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86f40ce600 0x7f86f40cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 -- 172.21.15.133:0/3744688472 >> 172.21.15.133:0/3744688472 conn(0x7f86f406bf80 msgr2=0x7f86f4162030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 -- 172.21.15.133:0/3744688472 shutdown_connections 2024-04-25T10:05:33.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.138+0000 7f86fc61f640 1 -- 172.21.15.133:0/3744688472 wait complete. 2024-04-25T10:05:33.162 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:05:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fb21c9640 1 Processor -- start 2024-04-25T10:05:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fb21c9640 1 -- start start 2024-04-25T10:05:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fb21c9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2fac06eb70 0x7f2fac0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fb21c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac1584f0 0x7f2fac15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fb21c9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac0cbe60 con 0x7f2fac158110 2024-04-25T10:05:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fb21c9640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac0b19b0 con 0x7f2fac06eb70 2024-04-25T10:05:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fb21c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac15b360 con 0x7f2fac1584f0 2024-04-25T10:05:33.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fa3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2fac06eb70 0x7f2fac0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fa3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2fac06eb70 0x7f2fac0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39934/0 (socket says 172.21.15.133:39934) 2024-04-25T10:05:33.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fa3fff640 1 -- 172.21.15.133:0/2527474863 learned_addr learned my addr 172.21.15.133:0/2527474863 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:33.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fabfff640 1 --2- 172.21.15.133:0/2527474863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac1584f0 0x7f2fac15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fa3fff640 1 -- 172.21.15.133:0/2527474863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac1584f0 msgr2=0x7f2fac15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.282+0000 7f2fa3fff640 1 --2- 172.21.15.133:0/2527474863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac1584f0 0x7f2fac15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2fa3fff640 1 -- 172.21.15.133:0/2527474863 >> v1:172.21.15.162:6789/0 conn(0x7f2fac158110 legacy=0x7f2fac06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2fa3fff640 1 -- 172.21.15.133:0/2527474863 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac15b640 con 0x7f2fac06eb70 2024-04-25T10:05:33.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2fa3fff640 1 --2- 172.21.15.133:0/2527474863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac06eb70 0x7f2fac0b1f00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400bb10 tx=0x7f2f9401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7440c497bf545b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2faaffd640 1 -- 172.21.15.133:0/2527474863 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f940b4070 con 0x7f2fac06eb70 2024-04-25T10:05:33.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2faaffd640 1 -- 172.21.15.133:0/2527474863 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f940a7b70 con 0x7f2fac06eb70 2024-04-25T10:05:33.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2faaffd640 1 -- 172.21.15.133:0/2527474863 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f940af440 con 0x7f2fac06eb70 2024-04-25T10:05:33.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2fb21c9640 1 -- 172.21.15.133:0/2527474863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac06eb70 msgr2=0x7f2fac0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2fb21c9640 1 --2- 172.21.15.133:0/2527474863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac06eb70 0x7f2fac0b1f00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400bb10 tx=0x7f2f9401ebc0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2fb21c9640 1 -- 172.21.15.133:0/2527474863 shutdown_connections 2024-04-25T10:05:33.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2fb21c9640 1 --2- 172.21.15.133:0/2527474863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac1584f0 0x7f2fac15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2fb21c9640 1 --2- 172.21.15.133:0/2527474863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac06eb70 0x7f2fac0b1f00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2fb21c9640 1 -- 172.21.15.133:0/2527474863 >> 172.21.15.133:0/2527474863 conn(0x7f2fac067000 msgr2=0x7f2fac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2fb21c9640 1 -- 172.21.15.133:0/2527474863 shutdown_connections 2024-04-25T10:05:33.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.286+0000 7f2fb21c9640 1 -- 172.21.15.133:0/2527474863 wait complete. 2024-04-25T10:05:33.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fb21c9640 1 Processor -- start 2024-04-25T10:05:33.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fb21c9640 1 -- start start 2024-04-25T10:05:33.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fb21c9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fac06eb70 0x7f2fac150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fb21c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac158110 0x7f2fac14c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fb21c9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac1584f0 0x7f2fac164850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fb21c9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac06dce0 con 0x7f2fac1584f0 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fb21c9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac06db60 con 0x7f2fac06eb70 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fb21c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac06de60 con 0x7f2fac158110 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac1584f0 0x7f2fac164850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac1584f0 0x7f2fac164850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39946/0 (socket says 172.21.15.133:39946) 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fabfff640 1 -- 172.21.15.133:0/1328192770 learned_addr learned my addr 172.21.15.133:0/1328192770 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fa3fff640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac158110 0x7f2fac14c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fab7fe640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fac06eb70 0x7f2fac150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fabfff640 1 -- 172.21.15.133:0/1328192770 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac158110 msgr2=0x7f2fac14c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fabfff640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac158110 0x7f2fac14c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fabfff640 1 -- 172.21.15.133:0/1328192770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fac06eb70 msgr2=0x7f2fac150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fabfff640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fac06eb70 0x7f2fac150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fabfff640 1 -- 172.21.15.133:0/1328192770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac14cd50 con 0x7f2fac1584f0 2024-04-25T10:05:33.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fa3fff640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac158110 0x7f2fac14c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:33.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fabfff640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac1584f0 0x7f2fac164850 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400cc00 tx=0x7f2fa400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fa8ff9640 1 -- 172.21.15.133:0/1328192770 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400ce10 con 0x7f2fac1584f0 2024-04-25T10:05:33.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fa8ff9640 1 -- 172.21.15.133:0/1328192770 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa40039a0 con 0x7f2fac1584f0 2024-04-25T10:05:33.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fb21c9640 1 -- 172.21.15.133:0/1328192770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac165150 con 0x7f2fac1584f0 2024-04-25T10:05:33.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fa8ff9640 1 -- 172.21.15.133:0/1328192770 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4003d70 con 0x7f2fac1584f0 2024-04-25T10:05:33.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fb21c9640 1 -- 172.21.15.133:0/1328192770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac163dc0 con 0x7f2fac1584f0 2024-04-25T10:05:33.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fb21c9640 1 -- 172.21.15.133:0/1328192770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac06bc20 con 0x7f2fac1584f0 2024-04-25T10:05:33.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fa8ff9640 1 -- 172.21.15.133:0/1328192770 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2fa4003f10 con 0x7f2fac1584f0 2024-04-25T10:05:33.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.290+0000 7f2fa8ff9640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f8c066f60 0x7f2f8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.294+0000 7f2fa8ff9640 1 -- 172.21.15.133:0/1328192770 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2fa40852c0 con 0x7f2fac1584f0 2024-04-25T10:05:33.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.294+0000 7f2fab7fe640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f8c066f60 0x7f2f8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.294+0000 7f2fab7fe640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f8c066f60 0x7f2f8c069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c007fd0 tx=0x7f2f9c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.298+0000 7f2fa8ff9640 1 -- 172.21.15.133:0/1328192770 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2fa4014030 con 0x7f2fac1584f0 2024-04-25T10:05:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.506+0000 7f2fb21c9640 1 -- 172.21.15.133:0/1328192770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2fac0c47e0 con 0x7f2fac1584f0 2024-04-25T10:05:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.506+0000 7f2fa8ff9640 1 -- 172.21.15.133:0/1328192770 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2fa4056490 con 0x7f2fac1584f0 2024-04-25T10:05:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:05:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:05:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:05:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:05:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:05:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:05:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:05:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:05:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:05:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:05:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:05:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:05:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:05:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:05:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:05:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:05:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.519 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:33.519 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:05:33.519 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.519 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.519 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.519 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.519 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:05:33.519 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:05:33.519 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.519 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:33.519 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:05:33.520 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.520 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.520 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.521 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.521 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.522 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.523 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.524 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:05:33.525 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:05:33.527 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:33.528 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:05:33.529 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:05:33.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 -- 172.21.15.133:0/1328192770 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f8c066f60 msgr2=0x7f2f8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f8c066f60 0x7f2f8c069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c007fd0 tx=0x7f2f9c009040 comp rx=0 tx=0).stop 2024-04-25T10:05:33.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 -- 172.21.15.133:0/1328192770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac1584f0 msgr2=0x7f2fac164850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac1584f0 0x7f2fac164850 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400cc00 tx=0x7f2fa400a5c0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 -- 172.21.15.133:0/1328192770 shutdown_connections 2024-04-25T10:05:33.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f8c066f60 0x7f2f8c069420 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fac1584f0 0x7f2fac164850 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fac158110 0x7f2fac14c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 --2- 172.21.15.133:0/1328192770 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fac06eb70 0x7f2fac150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 -- 172.21.15.133:0/1328192770 >> 172.21.15.133:0/1328192770 conn(0x7f2fac067000 msgr2=0x7f2fac0ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 -- 172.21.15.133:0/1328192770 shutdown_connections 2024-04-25T10:05:33.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.510+0000 7f2fb21c9640 1 -- 172.21.15.133:0/1328192770 wait complete. 2024-04-25T10:05:33.531 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:33.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.642+0000 7fed51f4d640 1 Processor -- start 2024-04-25T10:05:33.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.642+0000 7fed51f4d640 1 -- start start 2024-04-25T10:05:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed51f4d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed4c15c9a0 0x7fed4c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed51f4d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4c0c61d0 con 0x7fed4c0b1f70 2024-04-25T10:05:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed51f4d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4c0c64b0 con 0x7fed4c0ce330 2024-04-25T10:05:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed51f4d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4c06d4f0 con 0x7fed4c15c9a0 2024-04-25T10:05:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4affd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fed4c0b1f70 0x7fed4c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37006/0 (socket says 172.21.15.133:37006) 2024-04-25T10:05:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4affd640 1 -- 172.21.15.133:0/3331142219 learned_addr learned my addr 172.21.15.133:0/3331142219 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4b7fe640 1 --2- 172.21.15.133:0/3331142219 >> v2:172.21.15.133:3300/0 conn(0x7fed4c15c9a0 0x7fed4c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4a7fc640 1 -- 172.21.15.133:0/3331142219 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817061646 0 0) 0x7fed4c0c61d0 con 0x7fed4c0b1f70 2024-04-25T10:05:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4a7fc640 1 -- 172.21.15.133:0/3331142219 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed38003530 con 0x7fed4c0b1f70 2024-04-25T10:05:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4b7fe640 1 -- 172.21.15.133:0/3331142219 >> v1:172.21.15.162:6789/0 conn(0x7fed4c0ce330 legacy=0x7fed4c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4b7fe640 1 -- 172.21.15.133:0/3331142219 >> v1:172.21.15.133:6789/0 conn(0x7fed4c0b1f70 legacy=0x7fed4c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4b7fe640 1 -- 172.21.15.133:0/3331142219 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c06d7d0 con 0x7fed4c15c9a0 2024-04-25T10:05:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4b7fe640 1 --2- 172.21.15.133:0/3331142219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed4c15c9a0 0x7fed4c0c5980 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fed340097c0 tx=0x7fed3401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69023d88804ee89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4a7fc640 1 -- 172.21.15.133:0/3331142219 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340b0070 con 0x7fed4c15c9a0 2024-04-25T10:05:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4a7fc640 1 -- 172.21.15.133:0/3331142219 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed340ab600 con 0x7fed4c15c9a0 2024-04-25T10:05:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.646+0000 7fed4a7fc640 1 -- 172.21.15.133:0/3331142219 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340ab9d0 con 0x7fed4c15c9a0 2024-04-25T10:05:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 -- 172.21.15.133:0/3331142219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed4c15c9a0 msgr2=0x7fed4c0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 --2- 172.21.15.133:0/3331142219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed4c15c9a0 0x7fed4c0c5980 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fed340097c0 tx=0x7fed3401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 -- 172.21.15.133:0/3331142219 shutdown_connections 2024-04-25T10:05:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 --2- 172.21.15.133:0/3331142219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed4c15c9a0 0x7fed4c0c5980 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 -- 172.21.15.133:0/3331142219 >> 172.21.15.133:0/3331142219 conn(0x7fed4c067000 msgr2=0x7fed4c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 -- 172.21.15.133:0/3331142219 shutdown_connections 2024-04-25T10:05:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 -- 172.21.15.133:0/3331142219 wait complete. 2024-04-25T10:05:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 Processor -- start 2024-04-25T10:05:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 -- start start 2024-04-25T10:05:33.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed4c0b1f70 0x7fed4c149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed4c0ce330 0x7fed4c14c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed4c15c9a0 0x7fed4c14b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c15cf00 con 0x7fed4c0ce330 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c15cd80 con 0x7fed4c15c9a0 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c169de0 con 0x7fed4c0b1f70 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed4affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed4c0ce330 0x7fed4c14c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed4affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed4c0ce330 0x7fed4c14c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39966/0 (socket says 172.21.15.133:39966) 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed4affd640 1 -- 172.21.15.133:0/2858071179 learned_addr learned my addr 172.21.15.133:0/2858071179 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed4bfff640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed4c15c9a0 0x7fed4c14b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed4affd640 1 -- 172.21.15.133:0/2858071179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed4c0b1f70 msgr2=0x7fed4c149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed4affd640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed4c0b1f70 0x7fed4c149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed4affd640 1 -- 172.21.15.133:0/2858071179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed4c15c9a0 msgr2=0x7fed4c14b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed4affd640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed4c15c9a0 0x7fed4c14b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed4affd640 1 -- 172.21.15.133:0/2858071179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c0c94b0 con 0x7fed4c0ce330 2024-04-25T10:05:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed4affd640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed4c0ce330 0x7fed4c14c180 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00c450 tx=0x7fed3c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed43fff640 1 -- 172.21.15.133:0/2858071179 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c017070 con 0x7fed4c0ce330 2024-04-25T10:05:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed43fff640 1 -- 172.21.15.133:0/2858071179 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3c005cf0 con 0x7fed4c0ce330 2024-04-25T10:05:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed43fff640 1 -- 172.21.15.133:0/2858071179 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c007270 con 0x7fed4c0ce330 2024-04-25T10:05:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 -- 172.21.15.133:0/2858071179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4c0c9790 con 0x7fed4c0ce330 2024-04-25T10:05:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.650+0000 7fed51f4d640 1 -- 172.21.15.133:0/2858071179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4c0c9d50 con 0x7fed4c0ce330 2024-04-25T10:05:33.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.654+0000 7fed51f4d640 1 -- 172.21.15.133:0/2858071179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed4c068ef0 con 0x7fed4c0ce330 2024-04-25T10:05:33.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.654+0000 7fed43fff640 1 -- 172.21.15.133:0/2858071179 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed3c01a070 con 0x7fed4c0ce330 2024-04-25T10:05:33.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.654+0000 7fed43fff640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed2c066f60 0x7fed2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.654+0000 7fed4b7fe640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed2c066f60 0x7fed2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.654+0000 7fed43fff640 1 -- 172.21.15.133:0/2858071179 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fed3c0840e0 con 0x7fed4c0ce330 2024-04-25T10:05:33.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.654+0000 7fed4b7fe640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed2c066f60 0x7fed2c069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fed340ad000 tx=0x7fed34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.658+0000 7fed43fff640 1 -- 172.21.15.133:0/2858071179 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed3c054c60 con 0x7fed4c0ce330 2024-04-25T10:05:33.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.818+0000 7fed51f4d640 1 -- 172.21.15.133:0/2858071179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed4c152170 con 0x7fed4c0ce330 2024-04-25T10:05:33.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.818+0000 7fed43fff640 1 -- 172.21.15.133:0/2858071179 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13188 (secure 0 0 0) 0x7fed3c058d00 con 0x7fed4c0ce330 2024-04-25T10:05:33.824 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:33.824 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":50,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:05:33.133914+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:33.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 -- 172.21.15.133:0/2858071179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed2c066f60 msgr2=0x7fed2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed2c066f60 0x7fed2c069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fed340ad000 tx=0x7fed34008040 comp rx=0 tx=0).stop 2024-04-25T10:05:33.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 -- 172.21.15.133:0/2858071179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed4c0ce330 msgr2=0x7fed4c14c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed4c0ce330 0x7fed4c14c180 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00c450 tx=0x7fed3c00c910 comp rx=0 tx=0).stop 2024-04-25T10:05:33.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 -- 172.21.15.133:0/2858071179 shutdown_connections 2024-04-25T10:05:33.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed4c15c9a0 0x7fed4c14b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed2c066f60 0x7fed2c069420 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed4c0ce330 0x7fed4c14c180 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 --2- 172.21.15.133:0/2858071179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed4c0b1f70 0x7fed4c149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 -- 172.21.15.133:0/2858071179 >> 172.21.15.133:0/2858071179 conn(0x7fed4c067000 msgr2=0x7fed4c0b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 -- 172.21.15.133:0/2858071179 shutdown_connections 2024-04-25T10:05:33.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.822+0000 7fed51f4d640 1 -- 172.21.15.133:0/2858071179 wait complete. 2024-04-25T10:05:33.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:33.845 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.962+0000 7f55f8270640 1 Processor -- start 2024-04-25T10:05:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.962+0000 7f55f8270640 1 -- start start 2024-04-25T10:05:33.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f8270640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f55f00d2030 0x7f55f00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f8270640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f015ebd0 con 0x7f55f00ce600 2024-04-25T10:05:33.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f8270640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f015eeb0 con 0x7f55f015d9a0 2024-04-25T10:05:33.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f8270640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f00d0330 con 0x7f55f00d2030 2024-04-25T10:05:33.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f580b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f55f00ce600 0x7f55f00d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37012/0 (socket says 172.21.15.133:37012) 2024-04-25T10:05:33.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f580b640 1 -- 172.21.15.133:0/3779364005 learned_addr learned my addr 172.21.15.133:0/3779364005 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:33.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f680d640 1 --2- 172.21.15.133:0/3779364005 >> v2:172.21.15.162:3300/0 conn(0x7f55f00d2030 0x7f55f00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957100063 0 0) 0x7f55f015ebd0 con 0x7f55f00ce600 2024-04-25T10:05:33.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55dc003530 con 0x7f55f00ce600 2024-04-25T10:05:33.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2893827204 0 0) 0x7f55dc003530 con 0x7f55f00ce600 2024-04-25T10:05:33.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 >> v2:172.21.15.162:3300/0 conn(0x7f55f00d2030 msgr2=0x7f55f00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 --2- 172.21.15.133:0/3779364005 >> v2:172.21.15.162:3300/0 conn(0x7f55f00d2030 0x7f55f00cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 >> v1:172.21.15.162:6789/0 conn(0x7f55f015d9a0 legacy=0x7f55f00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00d0610 con 0x7f55f00ce600 2024-04-25T10:05:33.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f55e0001680 con 0x7f55f00ce600 2024-04-25T10:05:33.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 >> v1:172.21.15.133:6789/0 conn(0x7f55f00ce600 legacy=0x7f55f00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 --2- 172.21.15.133:0/3779364005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc0039c0 0x7f55dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 --2- 172.21.15.133:0/3779364005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55dc004810 0x7f55dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 --2- 172.21.15.133:0/3779364005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55dc0051d0 0x7f55dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00d0330 con 0x7f55dc004810 2024-04-25T10:05:33.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc00c170 con 0x7f55dc0039c0 2024-04-25T10:05:33.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc00c370 con 0x7f55dc0051d0 2024-04-25T10:05:33.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f580b640 1 --2- 172.21.15.133:0/3779364005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55dc004810 0x7f55dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f600c640 1 --2- 172.21.15.133:0/3779364005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc0039c0 0x7f55dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f680d640 1 --2- 172.21.15.133:0/3779364005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55dc0051d0 0x7f55dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f580b640 1 -- 172.21.15.133:0/3779364005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55dc0051d0 msgr2=0x7f55dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f580b640 1 --2- 172.21.15.133:0/3779364005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55dc0051d0 0x7f55dc00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f580b640 1 -- 172.21.15.133:0/3779364005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc0039c0 msgr2=0x7f55dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f580b640 1 --2- 172.21.15.133:0/3779364005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc0039c0 0x7f55dc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f580b640 1 -- 172.21.15.133:0/3779364005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc003530 con 0x7f55dc004810 2024-04-25T10:05:33.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f680d640 1 -- 172.21.15.133:0/3779364005 reap_dead start 2024-04-25T10:05:33.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f580b640 1 --2- 172.21.15.133:0/3779364005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55dc004810 0x7f55dc004c90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f55e000c1c0 tx=0x7f55e001cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9fbe8f90b13300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0007d00 con 0x7f55dc004810 2024-04-25T10:05:33.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e00a9020 con 0x7f55dc004810 2024-04-25T10:05:33.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.966+0000 7f55f500a640 1 -- 172.21.15.133:0/3779364005 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0005ae0 con 0x7f55dc004810 2024-04-25T10:05:33.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 -- 172.21.15.133:0/3779364005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55dc004810 msgr2=0x7f55dc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 --2- 172.21.15.133:0/3779364005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55dc004810 0x7f55dc004c90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f55e000c1c0 tx=0x7f55e001cb70 comp rx=0 tx=0).stop 2024-04-25T10:05:33.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 -- 172.21.15.133:0/3779364005 shutdown_connections 2024-04-25T10:05:33.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 --2- 172.21.15.133:0/3779364005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55dc004810 0x7f55dc004c90 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 -- 172.21.15.133:0/3779364005 >> 172.21.15.133:0/3779364005 conn(0x7f55f006bf80 msgr2=0x7f55f006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 -- 172.21.15.133:0/3779364005 shutdown_connections 2024-04-25T10:05:33.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 -- 172.21.15.133:0/3779364005 wait complete. 2024-04-25T10:05:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 Processor -- start 2024-04-25T10:05:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 -- start start 2024-04-25T10:05:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc004810 0x7f55f00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55f00cd000 0x7f55f00cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f015ff60 0x7f55f01603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00d1eb0 con 0x7f55f015ff60 2024-04-25T10:05:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00d1d30 con 0x7f55dc004810 2024-04-25T10:05:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00cee60 con 0x7f55f00cd000 2024-04-25T10:05:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f680d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f015ff60 0x7f55f01603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f680d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f015ff60 0x7f55f01603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39994/0 (socket says 172.21.15.133:39994) 2024-04-25T10:05:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f680d640 1 -- 172.21.15.133:0/1434336284 learned_addr learned my addr 172.21.15.133:0/1434336284 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f600c640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc004810 0x7f55f00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f580b640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55f00cd000 0x7f55f00cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f680d640 1 -- 172.21.15.133:0/1434336284 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55f00cd000 msgr2=0x7f55f00cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f680d640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55f00cd000 0x7f55f00cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f680d640 1 -- 172.21.15.133:0/1434336284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc004810 msgr2=0x7f55f00c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f680d640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc004810 0x7f55f00c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f680d640 1 -- 172.21.15.133:0/1434336284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00ca900 con 0x7f55f015ff60 2024-04-25T10:05:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f600c640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc004810 0x7f55f00c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f680d640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f015ff60 0x7f55f01603f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f55f0067810 tx=0x7f55e800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55e6ffd640 1 -- 172.21.15.133:0/1434336284 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8016620 con 0x7f55f015ff60 2024-04-25T10:05:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55e6ffd640 1 -- 172.21.15.133:0/1434336284 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e801b070 con 0x7f55f015ff60 2024-04-25T10:05:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 -- 172.21.15.133:0/1434336284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f0160cd0 con 0x7f55f015ff60 2024-04-25T10:05:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55e6ffd640 1 -- 172.21.15.133:0/1434336284 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8016a20 con 0x7f55f015ff60 2024-04-25T10:05:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.970+0000 7f55f8270640 1 -- 172.21.15.133:0/1434336284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f00cabc0 con 0x7f55f015ff60 2024-04-25T10:05:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.974+0000 7f55f8270640 1 -- 172.21.15.133:0/1434336284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f0068e80 con 0x7f55f015ff60 2024-04-25T10:05:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.974+0000 7f55e6ffd640 1 -- 172.21.15.133:0/1434336284 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55e8016bc0 con 0x7f55f015ff60 2024-04-25T10:05:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.974+0000 7f55e6ffd640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55d4066f60 0x7f55d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.974+0000 7f55e6ffd640 1 -- 172.21.15.133:0/1434336284 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f55e8085db0 con 0x7f55f015ff60 2024-04-25T10:05:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.974+0000 7f55f600c640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55d4066f60 0x7f55d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.974+0000 7f55f600c640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55d4066f60 0x7f55d4069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f55ec000c10 tx=0x7f55ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:33.978+0000 7f55e6ffd640 1 -- 172.21.15.133:0/1434336284 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55e8025020 con 0x7f55f015ff60 2024-04-25T10:05:34.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.138+0000 7f55f8270640 1 -- 172.21.15.133:0/1434336284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55f00b6f10 con 0x7f55f015ff60 2024-04-25T10:05:34.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.138+0000 7f55e6ffd640 1 -- 172.21.15.133:0/1434336284 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55e8057010 con 0x7f55f015ff60 2024-04-25T10:05:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:05:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 -- 172.21.15.133:0/1434336284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55d4066f60 msgr2=0x7f55d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55d4066f60 0x7f55d4069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f55ec000c10 tx=0x7f55ec009040 comp rx=0 tx=0).stop 2024-04-25T10:05:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 -- 172.21.15.133:0/1434336284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f015ff60 msgr2=0x7f55f01603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f015ff60 0x7f55f01603f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f55f0067810 tx=0x7f55e800f520 comp rx=0 tx=0).stop 2024-04-25T10:05:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 -- 172.21.15.133:0/1434336284 shutdown_connections 2024-04-25T10:05:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55d4066f60 0x7f55d4069420 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55f015ff60 0x7f55f01603f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55f00cd000 0x7f55f00cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 --2- 172.21.15.133:0/1434336284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55dc004810 0x7f55f00c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 -- 172.21.15.133:0/1434336284 >> 172.21.15.133:0/1434336284 conn(0x7f55f006bf80 msgr2=0x7f55f00b5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 -- 172.21.15.133:0/1434336284 shutdown_connections 2024-04-25T10:05:34.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.142+0000 7f55f8270640 1 -- 172.21.15.133:0/1434336284 wait complete. 2024-04-25T10:05:34.163 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:05:34.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.274+0000 7f21cacae640 1 Processor -- start 2024-04-25T10:05:34.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.274+0000 7f21cacae640 1 -- start start 2024-04-25T10:05:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f21c406eb70 0x7f21c40c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f21c415a9d0 0x7f21c40cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c40cc5a0 con 0x7f21c415a5f0 2024-04-25T10:05:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c40cc880 con 0x7f21c406eb70 2024-04-25T10:05:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c40ccb60 con 0x7f21c415a9d0 2024-04-25T10:05:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f21c406eb70 0x7f21c40c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f21c406eb70 0x7f21c40c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39998/0 (socket says 172.21.15.133:39998) 2024-04-25T10:05:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c3fff640 1 -- 172.21.15.133:0/2551285217 learned_addr learned my addr 172.21.15.133:0/2551285217 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c924b640 1 --2- 172.21.15.133:0/2551285217 >> v2:172.21.15.162:3300/0 conn(0x7f21c415a9d0 0x7f21c40cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c3fff640 1 -- 172.21.15.133:0/2551285217 >> v2:172.21.15.162:3300/0 conn(0x7f21c415a9d0 msgr2=0x7f21c40cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c3fff640 1 --2- 172.21.15.133:0/2551285217 >> v2:172.21.15.162:3300/0 conn(0x7f21c415a9d0 0x7f21c40cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c3fff640 1 -- 172.21.15.133:0/2551285217 >> v1:172.21.15.162:6789/0 conn(0x7f21c415a5f0 legacy=0x7f21c406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c3fff640 1 -- 172.21.15.133:0/2551285217 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40cce40 con 0x7f21c406eb70 2024-04-25T10:05:34.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c3fff640 1 --2- 172.21.15.133:0/2551285217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c406eb70 0x7f21c40c5260 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f21c40cd490 tx=0x7f21b401ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbedface1a24f927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c37fe640 1 -- 172.21.15.133:0/2551285217 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b40b4070 con 0x7f21c406eb70 2024-04-25T10:05:34.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c37fe640 1 -- 172.21.15.133:0/2551285217 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b40a7980 con 0x7f21c406eb70 2024-04-25T10:05:34.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21c37fe640 1 -- 172.21.15.133:0/2551285217 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b40a7d50 con 0x7f21c406eb70 2024-04-25T10:05:34.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 -- 172.21.15.133:0/2551285217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c406eb70 msgr2=0x7f21c40c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 --2- 172.21.15.133:0/2551285217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c406eb70 0x7f21c40c5260 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f21c40cd490 tx=0x7f21b401ebb0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 -- 172.21.15.133:0/2551285217 shutdown_connections 2024-04-25T10:05:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 --2- 172.21.15.133:0/2551285217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c406eb70 0x7f21c40c5260 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 --2- 172.21.15.133:0/2551285217 >> v2:172.21.15.162:3300/0 conn(0x7f21c415a9d0 0x7f21c40cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 -- 172.21.15.133:0/2551285217 >> 172.21.15.133:0/2551285217 conn(0x7f21c4067000 msgr2=0x7f21c406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 -- 172.21.15.133:0/2551285217 shutdown_connections 2024-04-25T10:05:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.278+0000 7f21cacae640 1 -- 172.21.15.133:0/2551285217 wait complete. 2024-04-25T10:05:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21cacae640 1 Processor -- start 2024-04-25T10:05:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21cacae640 1 -- start start 2024-04-25T10:05:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21cacae640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21c406eb70 0x7f21c414e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21cacae640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21c415a5f0 0x7f21c41580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21cacae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c415a9d0 0x7f21c4158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21cacae640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c406df30 con 0x7f21c415a9d0 2024-04-25T10:05:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21cacae640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c406ddb0 con 0x7f21c406eb70 2024-04-25T10:05:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21cacae640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c406e0b0 con 0x7f21c415a5f0 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c924b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c415a9d0 0x7f21c4158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c924b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c415a9d0 0x7f21c4158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40002/0 (socket says 172.21.15.133:40002) 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c924b640 1 -- 172.21.15.133:0/291830793 learned_addr learned my addr 172.21.15.133:0/291830793 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c8a4a640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21c406eb70 0x7f21c414e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c3fff640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21c415a5f0 0x7f21c41580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c924b640 1 -- 172.21.15.133:0/291830793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21c415a5f0 msgr2=0x7f21c41580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c924b640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21c415a5f0 0x7f21c41580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c924b640 1 -- 172.21.15.133:0/291830793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21c406eb70 msgr2=0x7f21c414e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c924b640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21c406eb70 0x7f21c414e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c924b640 1 -- 172.21.15.133:0/291830793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40b4440 con 0x7f21c415a9d0 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c8a4a640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21c406eb70 0x7f21c414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c924b640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c415a9d0 0x7f21c4158620 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00cc00 tx=0x7f21bc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c17fa640 1 -- 172.21.15.133:0/291830793 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc00ce10 con 0x7f21c415a9d0 2024-04-25T10:05:34.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c17fa640 1 -- 172.21.15.133:0/291830793 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc0039a0 con 0x7f21c415a9d0 2024-04-25T10:05:34.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21cacae640 1 -- 172.21.15.133:0/291830793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c4153840 con 0x7f21c415a9d0 2024-04-25T10:05:34.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21c17fa640 1 -- 172.21.15.133:0/291830793 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc003d70 con 0x7f21c415a9d0 2024-04-25T10:05:34.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21cacae640 1 -- 172.21.15.133:0/291830793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c4153e00 con 0x7f21c415a9d0 2024-04-25T10:05:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.282+0000 7f21cacae640 1 -- 172.21.15.133:0/291830793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c4153fd0 con 0x7f21c415a9d0 2024-04-25T10:05:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.286+0000 7f21c17fa640 1 -- 172.21.15.133:0/291830793 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21bc003f50 con 0x7f21c415a9d0 2024-04-25T10:05:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.286+0000 7f21c17fa640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21a4066f60 0x7f21a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.286+0000 7f21c8a4a640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21a4066f60 0x7f21a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.286+0000 7f21c17fa640 1 -- 172.21.15.133:0/291830793 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f21bc085220 con 0x7f21c415a9d0 2024-04-25T10:05:34.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.286+0000 7f21c8a4a640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21a4066f60 0x7f21a4069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0016e0 tx=0x7f21ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.290+0000 7f21c17fa640 1 -- 172.21.15.133:0/291830793 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21bc014030 con 0x7f21c415a9d0 2024-04-25T10:05:34.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.442+0000 7f21cacae640 1 -- 172.21.15.133:0/291830793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f21c4073ba0 con 0x7f21c415a9d0 2024-04-25T10:05:34.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.442+0000 7f21c17fa640 1 -- 172.21.15.133:0/291830793 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+65 (secure 0 0 0) 0x7f21bc056480 con 0x7f21c415a9d0 2024-04-25T10:05:34.447 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:34.447 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:05:34.447 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:05:34.447 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:05:34.447 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:05:34.447 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:34.447 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:05:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 -- 172.21.15.133:0/291830793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21a4066f60 msgr2=0x7f21a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21a4066f60 0x7f21a4069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0016e0 tx=0x7f21ac009040 comp rx=0 tx=0).stop 2024-04-25T10:05:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 -- 172.21.15.133:0/291830793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c415a9d0 msgr2=0x7f21c4158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c415a9d0 0x7f21c4158620 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00cc00 tx=0x7f21bc00a5c0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 -- 172.21.15.133:0/291830793 shutdown_connections 2024-04-25T10:05:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21a4066f60 0x7f21a4069420 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c415a9d0 0x7f21c4158620 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21c415a5f0 0x7f21c41580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 --2- 172.21.15.133:0/291830793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21c406eb70 0x7f21c414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 -- 172.21.15.133:0/291830793 >> 172.21.15.133:0/291830793 conn(0x7f21c4067000 msgr2=0x7f21c40b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 -- 172.21.15.133:0/291830793 shutdown_connections 2024-04-25T10:05:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.446+0000 7f21cacae640 1 -- 172.21.15.133:0/291830793 wait complete. 2024-04-25T10:05:34.465 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:05:34.465 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:05:34.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66cb65f640 1 Processor -- start 2024-04-25T10:05:34.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66cb65f640 1 -- start start 2024-04-25T10:05:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66cb65f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f66c40d3360 0x7f66c40ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66cb65f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f66c40d1d00 0x7f66c40ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66cb65f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c40d22c0 con 0x7f66c415d9d0 2024-04-25T10:05:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66cb65f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c40d25a0 con 0x7f66c40d1d00 2024-04-25T10:05:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66cb65f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c40d2880 con 0x7f66c40d3360 2024-04-25T10:05:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66c9bfc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f66c40d1d00 0x7f66c40ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66c93fb640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f66c415d9d0 0x7f66c40b92e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37014/0 (socket says 172.21.15.133:37014) 2024-04-25T10:05:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66c93fb640 1 -- 172.21.15.133:0/1165905270 learned_addr learned my addr 172.21.15.133:0/1165905270 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66c8bfa640 1 --2- 172.21.15.133:0/1165905270 >> v2:172.21.15.162:3300/0 conn(0x7f66c40d3360 0x7f66c40ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66c9bfc640 1 -- 172.21.15.133:0/1165905270 >> v2:172.21.15.162:3300/0 conn(0x7f66c40d3360 msgr2=0x7f66c40ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66c9bfc640 1 --2- 172.21.15.133:0/1165905270 >> v2:172.21.15.162:3300/0 conn(0x7f66c40d3360 0x7f66c40ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66c9bfc640 1 -- 172.21.15.133:0/1165905270 >> v1:172.21.15.133:6789/0 conn(0x7f66c415d9d0 legacy=0x7f66c40b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66c9bfc640 1 -- 172.21.15.133:0/1165905270 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c40d2b60 con 0x7f66c40d1d00 2024-04-25T10:05:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66bbfff640 1 -- 172.21.15.133:0/1165905270 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474082420 0 0) 0x7f66c40d22c0 con 0x7f66c415d9d0 2024-04-25T10:05:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.586+0000 7f66c8bfa640 1 --2- 172.21.15.133:0/1165905270 >> v2:172.21.15.162:3300/0 conn(0x7f66c40d3360 0x7f66c40ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:34.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66c9bfc640 1 --2- 172.21.15.133:0/1165905270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66c40d1d00 0x7f66c40ceb40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f66c41614d0 tx=0x7f66bc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83a1a121c261d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66bbfff640 1 -- 172.21.15.133:0/1165905270 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0a7070 con 0x7f66c40d1d00 2024-04-25T10:05:34.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66bbfff640 1 -- 172.21.15.133:0/1165905270 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc0b0070 con 0x7f66c40d1d00 2024-04-25T10:05:34.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66bbfff640 1 -- 172.21.15.133:0/1165905270 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0ab890 con 0x7f66c40d1d00 2024-04-25T10:05:34.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66cb65f640 1 -- 172.21.15.133:0/1165905270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66c40d1d00 msgr2=0x7f66c40ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66cb65f640 1 --2- 172.21.15.133:0/1165905270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66c40d1d00 0x7f66c40ceb40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f66c41614d0 tx=0x7f66bc01c450 comp rx=0 tx=0).stop 2024-04-25T10:05:34.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66cb65f640 1 -- 172.21.15.133:0/1165905270 shutdown_connections 2024-04-25T10:05:34.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66cb65f640 1 --2- 172.21.15.133:0/1165905270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66c40d1d00 0x7f66c40ceb40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66cb65f640 1 --2- 172.21.15.133:0/1165905270 >> v2:172.21.15.162:3300/0 conn(0x7f66c40d3360 0x7f66c40ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66cb65f640 1 -- 172.21.15.133:0/1165905270 >> 172.21.15.133:0/1165905270 conn(0x7f66c406c080 msgr2=0x7f66c406c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66cb65f640 1 -- 172.21.15.133:0/1165905270 shutdown_connections 2024-04-25T10:05:34.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66cb65f640 1 -- 172.21.15.133:0/1165905270 wait complete. 2024-04-25T10:05:34.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66cb65f640 1 Processor -- start 2024-04-25T10:05:34.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.590+0000 7f66cb65f640 1 -- start start 2024-04-25T10:05:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66cb65f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66c40d1d00 0x7f66c4157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66cb65f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66c40d3360 0x7f66c40c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66cb65f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66c415d9d0 0x7f66c414d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66cb65f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c40b7600 con 0x7f66c415d9d0 2024-04-25T10:05:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66cb65f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c40b7480 con 0x7f66c40d1d00 2024-04-25T10:05:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66cb65f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c40b7780 con 0x7f66c40d3360 2024-04-25T10:05:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c9bfc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66c415d9d0 0x7f66c414d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c9bfc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66c415d9d0 0x7f66c414d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40016/0 (socket says 172.21.15.133:40016) 2024-04-25T10:05:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c9bfc640 1 -- 172.21.15.133:0/3454791594 learned_addr learned my addr 172.21.15.133:0/3454791594 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c93fb640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66c40d1d00 0x7f66c4157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c8bfa640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66c40d3360 0x7f66c40c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c9bfc640 1 -- 172.21.15.133:0/3454791594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66c40d3360 msgr2=0x7f66c40c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c9bfc640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66c40d3360 0x7f66c40c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c9bfc640 1 -- 172.21.15.133:0/3454791594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66c40d1d00 msgr2=0x7f66c4157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c9bfc640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66c40d1d00 0x7f66c4157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c9bfc640 1 -- 172.21.15.133:0/3454791594 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c40c8e00 con 0x7f66c415d9d0 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c93fb640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66c40d1d00 0x7f66c4157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c8bfa640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66c40d3360 0x7f66c40c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c9bfc640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66c415d9d0 0x7f66c414d7a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0ad040 tx=0x7f66bc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66b9ffb640 1 -- 172.21.15.133:0/3454791594 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0bf610 con 0x7f66c415d9d0 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66b9ffb640 1 -- 172.21.15.133:0/3454791594 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc0bf7d0 con 0x7f66c415d9d0 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66cb65f640 1 -- 172.21.15.133:0/3454791594 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c40c88a0 con 0x7f66c415d9d0 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66b9ffb640 1 -- 172.21.15.133:0/3454791594 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0bfbc0 con 0x7f66c415d9d0 2024-04-25T10:05:34.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66cb65f640 1 -- 172.21.15.133:0/3454791594 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c40c9380 con 0x7f66c415d9d0 2024-04-25T10:05:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66cb65f640 1 -- 172.21.15.133:0/3454791594 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c4069000 con 0x7f66c415d9d0 2024-04-25T10:05:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66b9ffb640 1 -- 172.21.15.133:0/3454791594 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66bc0bd020 con 0x7f66c415d9d0 2024-04-25T10:05:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66b9ffb640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66a8066f90 0x7f66a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c93fb640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66a8066f90 0x7f66a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66b9ffb640 1 -- 172.21.15.133:0/3454791594 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f66bc11fa40 con 0x7f66c415d9d0 2024-04-25T10:05:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.594+0000 7f66c93fb640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66a8066f90 0x7f66a8069450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f66c00012f0 tx=0x7f66c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.598+0000 7f66b9ffb640 1 -- 172.21.15.133:0/3454791594 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f66bc016070 con 0x7f66c415d9d0 2024-04-25T10:05:34.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:34.766+0000 7f66cb65f640 1 -- 172.21.15.133:0/3454791594 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f66c41522b0 con 0x7f66c415d9d0 2024-04-25T10:05:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.154+0000 7f66b9ffb640 1 -- 172.21.15.133:0/3454791594 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7f66bc0f0c10 con 0x7f66c415d9d0 2024-04-25T10:05:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:05:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 -- 172.21.15.133:0/3454791594 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66a8066f90 msgr2=0x7f66a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66a8066f90 0x7f66a8069450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f66c00012f0 tx=0x7f66c0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 -- 172.21.15.133:0/3454791594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66c415d9d0 msgr2=0x7f66c414d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66c415d9d0 0x7f66c414d7a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0ad040 tx=0x7f66bc01ac90 comp rx=0 tx=0).stop 2024-04-25T10:05:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 -- 172.21.15.133:0/3454791594 shutdown_connections 2024-04-25T10:05:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f66a8066f90 0x7f66a8069450 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66c415d9d0 0x7f66c414d7a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66c40d3360 0x7f66c40c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 --2- 172.21.15.133:0/3454791594 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66c40d1d00 0x7f66c4157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 -- 172.21.15.133:0/3454791594 >> 172.21.15.133:0/3454791594 conn(0x7f66c406c080 msgr2=0x7f66c406c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 -- 172.21.15.133:0/3454791594 shutdown_connections 2024-04-25T10:05:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.158+0000 7f66cb65f640 1 -- 172.21.15.133:0/3454791594 wait complete. 2024-04-25T10:05:35.187 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:05:35.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab8a114640 1 Processor -- start 2024-04-25T10:05:35.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab8a114640 1 -- start start 2024-04-25T10:05:35.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab8a114640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab840b2150 0x7fab8406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab8a114640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fab8415b9b0 0x7fab8406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab8a114640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab8415ac50 con 0x7fab84143ea0 2024-04-25T10:05:35.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab8a114640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab840ccf20 con 0x7fab8415b9b0 2024-04-25T10:05:35.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab8a114640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab840cd180 con 0x7fab840b2150 2024-04-25T10:05:35.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab83fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fab8415b9b0 0x7fab8406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab82ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab840b2150 0x7fab8406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab837fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fab84143ea0 0x7fab840cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:43760/0 (socket says 172.21.15.133:43760) 2024-04-25T10:05:35.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab837fe640 1 -- 172.21.15.133:0/2566914663 learned_addr learned my addr 172.21.15.133:0/2566914663 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:35.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab82ffd640 1 -- 172.21.15.133:0/2566914663 >> v2:172.21.15.162:3300/0 conn(0x7fab8415b9b0 msgr2=0x7fab8406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab82ffd640 1 --2- 172.21.15.133:0/2566914663 >> v2:172.21.15.162:3300/0 conn(0x7fab8415b9b0 0x7fab8406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab82ffd640 1 -- 172.21.15.133:0/2566914663 >> v1:172.21.15.162:6789/0 conn(0x7fab84143ea0 legacy=0x7fab840cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.310+0000 7fab82ffd640 1 -- 172.21.15.133:0/2566914663 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8415aed0 con 0x7fab840b2150 2024-04-25T10:05:35.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab82ffd640 1 --2- 172.21.15.133:0/2566914663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab840b2150 0x7fab8406e3b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fab780097c0 tx=0x7fab7801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eff6491e64a8c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab827fc640 1 -- 172.21.15.133:0/2566914663 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab780b1070 con 0x7fab840b2150 2024-04-25T10:05:35.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab827fc640 1 -- 172.21.15.133:0/2566914663 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab780ac640 con 0x7fab840b2150 2024-04-25T10:05:35.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab827fc640 1 -- 172.21.15.133:0/2566914663 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab780aca10 con 0x7fab840b2150 2024-04-25T10:05:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab8a114640 1 -- 172.21.15.133:0/2566914663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab840b2150 msgr2=0x7fab8406e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab8a114640 1 --2- 172.21.15.133:0/2566914663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab840b2150 0x7fab8406e3b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fab780097c0 tx=0x7fab7801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab8a114640 1 -- 172.21.15.133:0/2566914663 shutdown_connections 2024-04-25T10:05:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab8a114640 1 --2- 172.21.15.133:0/2566914663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab840b2150 0x7fab8406e3b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab8a114640 1 --2- 172.21.15.133:0/2566914663 >> v2:172.21.15.162:3300/0 conn(0x7fab8415b9b0 0x7fab8406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab8a114640 1 -- 172.21.15.133:0/2566914663 >> 172.21.15.133:0/2566914663 conn(0x7fab84066f70 msgr2=0x7fab8406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab8a114640 1 -- 172.21.15.133:0/2566914663 shutdown_connections 2024-04-25T10:05:35.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.314+0000 7fab8a114640 1 -- 172.21.15.133:0/2566914663 wait complete. 2024-04-25T10:05:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab8a114640 1 Processor -- start 2024-04-25T10:05:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab8a114640 1 -- start start 2024-04-25T10:05:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab8a114640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab840b2150 0x7fab84137290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab8a114640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab84143ea0 0x7fab84163780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab8a114640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab8415b9b0 0x7fab84163cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab8a114640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab840b1270 con 0x7fab8415b9b0 2024-04-25T10:05:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab8a114640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab840b10f0 con 0x7fab84143ea0 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab8a114640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab840b13f0 con 0x7fab840b2150 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab83fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab8415b9b0 0x7fab84163cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab83fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab8415b9b0 0x7fab84163cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40028/0 (socket says 172.21.15.133:40028) 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab83fff640 1 -- 172.21.15.133:0/3064924996 learned_addr learned my addr 172.21.15.133:0/3064924996 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab837fe640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab840b2150 0x7fab84137290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab82ffd640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab84143ea0 0x7fab84163780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab83fff640 1 -- 172.21.15.133:0/3064924996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab840b2150 msgr2=0x7fab84137290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab83fff640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab840b2150 0x7fab84137290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab83fff640 1 -- 172.21.15.133:0/3064924996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab84143ea0 msgr2=0x7fab84163780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab83fff640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab84143ea0 0x7fab84163780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab83fff640 1 -- 172.21.15.133:0/3064924996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab841436c0 con 0x7fab8415b9b0 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab82ffd640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab84143ea0 0x7fab84163780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab83fff640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab8415b9b0 0x7fab84163cc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00efc0 tx=0x7fab7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab6bfff640 1 -- 172.21.15.133:0/3064924996 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c018020 con 0x7fab8415b9b0 2024-04-25T10:05:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab6bfff640 1 -- 172.21.15.133:0/3064924996 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c014480 con 0x7fab8415b9b0 2024-04-25T10:05:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab8a114640 1 -- 172.21.15.133:0/3064924996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab84143100 con 0x7fab8415b9b0 2024-04-25T10:05:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab6bfff640 1 -- 172.21.15.133:0/3064924996 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c009e60 con 0x7fab8415b9b0 2024-04-25T10:05:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab8a114640 1 -- 172.21.15.133:0/3064924996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab841588f0 con 0x7fab8415b9b0 2024-04-25T10:05:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab8a114640 1 -- 172.21.15.133:0/3064924996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab84068e80 con 0x7fab8415b9b0 2024-04-25T10:05:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.318+0000 7fab6bfff640 1 -- 172.21.15.133:0/3064924996 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab7c01b020 con 0x7fab8415b9b0 2024-04-25T10:05:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.322+0000 7fab6bfff640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab70066f60 0x7fab70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.322+0000 7fab837fe640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab70066f60 0x7fab70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.322+0000 7fab6bfff640 1 -- 172.21.15.133:0/3064924996 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fab7c086320 con 0x7fab8415b9b0 2024-04-25T10:05:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.322+0000 7fab837fe640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab70066f60 0x7fab70069420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fab8415d4f0 tx=0x7fab74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.326+0000 7fab6bfff640 1 -- 172.21.15.133:0/3064924996 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab7c022030 con 0x7fab8415b9b0 2024-04-25T10:05:35.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:35.470+0000 7fab8a114640 1 -- 172.21.15.133:0/3064924996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fab840b04b0 con 0x7fab8415b9b0 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.158+0000 7fab6bfff640 1 -- 172.21.15.133:0/3064924996 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) v1 ==== 170+0+0 (secure 0 0 0) 0x7fab7c010040 con 0x7fab8415b9b0 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 -- 172.21.15.133:0/3064924996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab70066f60 msgr2=0x7fab70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab70066f60 0x7fab70069420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fab8415d4f0 tx=0x7fab74009040 comp rx=0 tx=0).stop 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 -- 172.21.15.133:0/3064924996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab8415b9b0 msgr2=0x7fab84163cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab8415b9b0 0x7fab84163cc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00efc0 tx=0x7fab7c00c490 comp rx=0 tx=0).stop 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 -- 172.21.15.133:0/3064924996 shutdown_connections 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab70066f60 0x7fab70069420 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab8415b9b0 0x7fab84163cc0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab84143ea0 0x7fab84163780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 --2- 172.21.15.133:0/3064924996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab840b2150 0x7fab84137290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 -- 172.21.15.133:0/3064924996 >> 172.21.15.133:0/3064924996 conn(0x7fab84066f70 msgr2=0x7fab8415ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 -- 172.21.15.133:0/3064924996 shutdown_connections 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.162+0000 7fab8a114640 1 -- 172.21.15.133:0/3064924996 wait complete. 2024-04-25T10:05:36.202 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:05:36.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c15aac640 1 Processor -- start 2024-04-25T10:05:36.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c15aac640 1 -- start start 2024-04-25T10:05:36.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c15aac640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0c10145170 0x7f0c1006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c15aac640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0c1006e4d0 0x7f0c100cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c15aac640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c100ce2e0 con 0x7f0c100cdd60 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c15aac640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1006db00 con 0x7f0c10145170 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c15aac640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c10145550 con 0x7f0c1006e4d0 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c0f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0c10145170 0x7f0c1006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c0ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0c100cdd60 0x7f0c1015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40054/0 (socket says 172.21.15.133:40054) 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c0ffff640 1 -- 172.21.15.133:0/727447570 learned_addr learned my addr 172.21.15.133:0/727447570 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c07fff640 1 --2- 172.21.15.133:0/727447570 >> v2:172.21.15.162:3300/0 conn(0x7f0c1006e4d0 0x7f0c100cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c0f7fe640 1 -- 172.21.15.133:0/727447570 >> v2:172.21.15.162:3300/0 conn(0x7f0c1006e4d0 msgr2=0x7f0c100cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c0f7fe640 1 --2- 172.21.15.133:0/727447570 >> v2:172.21.15.162:3300/0 conn(0x7f0c1006e4d0 0x7f0c100cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c0f7fe640 1 -- 172.21.15.133:0/727447570 >> v1:172.21.15.133:6789/0 conn(0x7f0c100cdd60 legacy=0x7f0c1015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c0f7fe640 1 -- 172.21.15.133:0/727447570 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1006d4b0 con 0x7f0c10145170 2024-04-25T10:05:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.322+0000 7f0c0effd640 1 -- 172.21.15.133:0/727447570 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390798215 0 0) 0x7f0c100ce2e0 con 0x7f0c100cdd60 2024-04-25T10:05:36.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0f7fe640 1 --2- 172.21.15.133:0/727447570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c10145170 0x7f0c1006df90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0c100b4dd0 tx=0x7f0c0001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aada3d8d10a2737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0effd640 1 -- 172.21.15.133:0/727447570 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c000b0070 con 0x7f0c10145170 2024-04-25T10:05:36.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0effd640 1 -- 172.21.15.133:0/727447570 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c000ab600 con 0x7f0c10145170 2024-04-25T10:05:36.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0effd640 1 -- 172.21.15.133:0/727447570 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c000ab9d0 con 0x7f0c10145170 2024-04-25T10:05:36.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 -- 172.21.15.133:0/727447570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c10145170 msgr2=0x7f0c1006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 --2- 172.21.15.133:0/727447570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c10145170 0x7f0c1006df90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0c100b4dd0 tx=0x7f0c0001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 -- 172.21.15.133:0/727447570 shutdown_connections 2024-04-25T10:05:36.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 --2- 172.21.15.133:0/727447570 >> v2:172.21.15.162:3300/0 conn(0x7f0c1006e4d0 0x7f0c100cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 --2- 172.21.15.133:0/727447570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c10145170 0x7f0c1006df90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 -- 172.21.15.133:0/727447570 >> 172.21.15.133:0/727447570 conn(0x7f0c10066f70 msgr2=0x7f0c1006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 -- 172.21.15.133:0/727447570 shutdown_connections 2024-04-25T10:05:36.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 -- 172.21.15.133:0/727447570 wait complete. 2024-04-25T10:05:36.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 Processor -- start 2024-04-25T10:05:36.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 -- start start 2024-04-25T10:05:36.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006e4d0 0x7f0c101329a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c100cdd60 0x7f0c10151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c10145170 0x7f0c100b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c10168490 con 0x7f0c10145170 2024-04-25T10:05:36.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1015d7f0 con 0x7f0c1006e4d0 2024-04-25T10:05:36.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c15aac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c10168610 con 0x7f0c100cdd60 2024-04-25T10:05:36.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c10145170 0x7f0c100b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c10145170 0x7f0c100b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43398/0 (socket says 172.21.15.133:43398) 2024-04-25T10:05:36.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0ffff640 1 -- 172.21.15.133:0/4292032435 learned_addr learned my addr 172.21.15.133:0/4292032435 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0f7fe640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006e4d0 0x7f0c101329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c07fff640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c100cdd60 0x7f0c10151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0ffff640 1 -- 172.21.15.133:0/4292032435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c100cdd60 msgr2=0x7f0c10151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0ffff640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c100cdd60 0x7f0c10151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0ffff640 1 -- 172.21.15.133:0/4292032435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006e4d0 msgr2=0x7f0c101329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0ffff640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006e4d0 0x7f0c101329a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0ffff640 1 -- 172.21.15.133:0/4292032435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c10132f80 con 0x7f0c10145170 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.326+0000 7f0c0f7fe640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006e4d0 0x7f0c101329a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c0ffff640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c10145170 0x7f0c100b3f40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800c7d0 tx=0x7f0c0800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c0cff9640 1 -- 172.21.15.133:0/4292032435 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08018070 con 0x7f0c10145170 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c0cff9640 1 -- 172.21.15.133:0/4292032435 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0800a640 con 0x7f0c10145170 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c15aac640 1 -- 172.21.15.133:0/4292032435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c10133260 con 0x7f0c10145170 2024-04-25T10:05:36.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c0cff9640 1 -- 172.21.15.133:0/4292032435 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0800aa70 con 0x7f0c10145170 2024-04-25T10:05:36.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c15aac640 1 -- 172.21.15.133:0/4292032435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c10133820 con 0x7f0c10145170 2024-04-25T10:05:36.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c15aac640 1 -- 172.21.15.133:0/4292032435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c10068e80 con 0x7f0c10145170 2024-04-25T10:05:36.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c0cff9640 1 -- 172.21.15.133:0/4292032435 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c0801b070 con 0x7f0c10145170 2024-04-25T10:05:36.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c0cff9640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bf4066f60 0x7f0bf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c0cff9640 1 -- 172.21.15.133:0/4292032435 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f0c080846d0 con 0x7f0c10145170 2024-04-25T10:05:36.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c0f7fe640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bf4066f60 0x7f0bf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.330+0000 7f0c0f7fe640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bf4066f60 0x7f0bf4069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0c000ad000 tx=0x7f0c00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.334+0000 7f0c0cff9640 1 -- 172.21.15.133:0/4292032435 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c08055640 con 0x7f0c10145170 2024-04-25T10:05:36.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:36.526+0000 7f0c15aac640 1 -- 172.21.15.133:0/4292032435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0c1015de50 con 0x7f0c10145170 2024-04-25T10:05:37.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.182+0000 7f0c0cff9640 1 -- 172.21.15.133:0/4292032435 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0c08054f40 con 0x7f0c10145170 2024-04-25T10:05:37.188 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 -- 172.21.15.133:0/4292032435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bf4066f60 msgr2=0x7f0bf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bf4066f60 0x7f0bf4069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0c000ad000 tx=0x7f0c00008040 comp rx=0 tx=0).stop 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 -- 172.21.15.133:0/4292032435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c10145170 msgr2=0x7f0c100b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c10145170 0x7f0c100b3f40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800c7d0 tx=0x7f0c0800cc90 comp rx=0 tx=0).stop 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 -- 172.21.15.133:0/4292032435 shutdown_connections 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bf4066f60 0x7f0bf4069420 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c10145170 0x7f0c100b3f40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c100cdd60 0x7f0c10151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 --2- 172.21.15.133:0/4292032435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c1006e4d0 0x7f0c101329a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 -- 172.21.15.133:0/4292032435 >> 172.21.15.133:0/4292032435 conn(0x7f0c10066f70 msgr2=0x7f0c1013dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 -- 172.21.15.133:0/4292032435 shutdown_connections 2024-04-25T10:05:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.186+0000 7f0c15aac640 1 -- 172.21.15.133:0/4292032435 wait complete. 2024-04-25T10:05:37.237 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:37.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.346+0000 7f1ef058c640 1 Processor -- start 2024-04-25T10:05:37.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.346+0000 7f1ef058c640 1 -- start start 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.346+0000 7f1ef058c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ee815eec0 0x7f1ee80999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.346+0000 7f1ef058c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1ee80b2bd0 0x7f1ee8099f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.346+0000 7f1ef058c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee815f2a0 con 0x7f1ee80b0db0 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.346+0000 7f1ef058c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee80b2050 con 0x7f1ee80b2bd0 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.346+0000 7f1ef058c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee80b2330 con 0x7f1ee815eec0 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.346+0000 7f1eedb27640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1ee80b2bd0 0x7f1ee8099f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.346+0000 7f1eedb27640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1ee80b2bd0 0x7f1ee8099f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43414/0 (socket says 172.21.15.133:43414) 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.346+0000 7f1eedb27640 1 -- 172.21.15.133:0/2032936794 learned_addr learned my addr 172.21.15.133:0/2032936794 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1eedb27640 1 -- 172.21.15.133:0/2032936794 >> v2:172.21.15.162:3300/0 conn(0x7f1ee815eec0 msgr2=0x7f1ee80999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1eedb27640 1 --2- 172.21.15.133:0/2032936794 >> v2:172.21.15.162:3300/0 conn(0x7f1ee815eec0 0x7f1ee80999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1eedb27640 1 -- 172.21.15.133:0/2032936794 >> v1:172.21.15.162:6789/0 conn(0x7f1ee80b0db0 legacy=0x7f1ee80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1eedb27640 1 -- 172.21.15.133:0/2032936794 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee80b2610 con 0x7f1ee80b2bd0 2024-04-25T10:05:37.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1eedb27640 1 --2- 172.21.15.133:0/2032936794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ee80b2bd0 0x7f1ee8099f10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8072db0 tx=0x7f1ee401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1949e9df220b54b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1eed326640 1 -- 172.21.15.133:0/2032936794 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee40b1070 con 0x7f1ee80b2bd0 2024-04-25T10:05:37.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1eed326640 1 -- 172.21.15.133:0/2032936794 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee40ac630 con 0x7f1ee80b2bd0 2024-04-25T10:05:37.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1eed326640 1 -- 172.21.15.133:0/2032936794 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee40a8050 con 0x7f1ee80b2bd0 2024-04-25T10:05:37.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1ef058c640 1 -- 172.21.15.133:0/2032936794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ee80b2bd0 msgr2=0x7f1ee8099f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1ef058c640 1 --2- 172.21.15.133:0/2032936794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ee80b2bd0 0x7f1ee8099f10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8072db0 tx=0x7f1ee401c4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1ef058c640 1 -- 172.21.15.133:0/2032936794 shutdown_connections 2024-04-25T10:05:37.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1ef058c640 1 --2- 172.21.15.133:0/2032936794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ee80b2bd0 0x7f1ee8099f10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1ef058c640 1 --2- 172.21.15.133:0/2032936794 >> v2:172.21.15.162:3300/0 conn(0x7f1ee815eec0 0x7f1ee80999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1ef058c640 1 -- 172.21.15.133:0/2032936794 >> 172.21.15.133:0/2032936794 conn(0x7f1ee80b31a0 msgr2=0x7f1ee80b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1ef058c640 1 -- 172.21.15.133:0/2032936794 shutdown_connections 2024-04-25T10:05:37.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.350+0000 7f1ef058c640 1 -- 172.21.15.133:0/2032936794 wait complete. 2024-04-25T10:05:37.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1ef058c640 1 Processor -- start 2024-04-25T10:05:37.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1ef058c640 1 -- start start 2024-04-25T10:05:37.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1ef058c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee80b0db0 0x7f1ee8155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1ef058c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee80b2bd0 0x7f1ee80b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1ef058c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ee815eec0 0x7f1ee8165cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1ef058c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee81686f0 con 0x7f1ee815eec0 2024-04-25T10:05:37.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1ef058c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee8168570 con 0x7f1ee80b0db0 2024-04-25T10:05:37.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1ef058c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee8168870 con 0x7f1ee80b2bd0 2024-04-25T10:05:37.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1eeeb29640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ee815eec0 0x7f1ee8165cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1eeeb29640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ee815eec0 0x7f1ee8165cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43422/0 (socket says 172.21.15.133:43422) 2024-04-25T10:05:37.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1eeeb29640 1 -- 172.21.15.133:0/2093002934 learned_addr learned my addr 172.21.15.133:0/2093002934 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1eee328640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee80b0db0 0x7f1ee8155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1eedb27640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee80b2bd0 0x7f1ee80b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1eeeb29640 1 -- 172.21.15.133:0/2093002934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee80b2bd0 msgr2=0x7f1ee80b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1eeeb29640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee80b2bd0 0x7f1ee80b6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1eeeb29640 1 -- 172.21.15.133:0/2093002934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee80b0db0 msgr2=0x7f1ee8155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1eeeb29640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee80b0db0 0x7f1ee8155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1eeeb29640 1 -- 172.21.15.133:0/2093002934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee8164c20 con 0x7f1ee815eec0 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1eeeb29640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ee815eec0 0x7f1ee8165cc0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800c410 tx=0x7f1ed800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1edeffd640 1 -- 172.21.15.133:0/2093002934 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8017070 con 0x7f1ee815eec0 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1edeffd640 1 -- 172.21.15.133:0/2093002934 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed8003600 con 0x7f1ee815eec0 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1edeffd640 1 -- 172.21.15.133:0/2093002934 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed80039f0 con 0x7f1ee815eec0 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1ef058c640 1 -- 172.21.15.133:0/2093002934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee8164f00 con 0x7f1ee815eec0 2024-04-25T10:05:37.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1ef058c640 1 -- 172.21.15.133:0/2093002934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee81654c0 con 0x7f1ee815eec0 2024-04-25T10:05:37.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1ef058c640 1 -- 172.21.15.133:0/2093002934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee8068ef0 con 0x7f1ee815eec0 2024-04-25T10:05:37.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.354+0000 7f1edeffd640 1 -- 172.21.15.133:0/2093002934 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ed8003b90 con 0x7f1ee815eec0 2024-04-25T10:05:37.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.358+0000 7f1edeffd640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ecc066f60 0x7f1ecc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.358+0000 7f1eee328640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ecc066f60 0x7f1ecc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.358+0000 7f1edeffd640 1 -- 172.21.15.133:0/2093002934 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ed80843d0 con 0x7f1ee815eec0 2024-04-25T10:05:37.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.358+0000 7f1eee328640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ecc066f60 0x7f1ecc069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009f20 tx=0x7f1ee00094f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.362+0000 7f1edeffd640 1 -- 172.21.15.133:0/2093002934 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ed80552e0 con 0x7f1ee815eec0 2024-04-25T10:05:37.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.510+0000 7f1ef058c640 1 -- 172.21.15.133:0/2093002934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ee815e550 con 0x7f1ee815eec0 2024-04-25T10:05:37.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1edeffd640 1 -- 172.21.15.133:0/2093002934 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16979 (secure 0 0 0) 0x7f1ed8054be0 con 0x7f1ee815eec0 2024-04-25T10:05:37.518 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:37.519 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":53,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:05:36.528355+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:34.805104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:35.512057+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:37.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 -- 172.21.15.133:0/2093002934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ecc066f60 msgr2=0x7f1ecc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ecc066f60 0x7f1ecc069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009f20 tx=0x7f1ee00094f0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 -- 172.21.15.133:0/2093002934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ee815eec0 msgr2=0x7f1ee8165cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ee815eec0 0x7f1ee8165cc0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800c410 tx=0x7f1ed800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 -- 172.21.15.133:0/2093002934 shutdown_connections 2024-04-25T10:05:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ecc066f60 0x7f1ecc069420 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ee815eec0 0x7f1ee8165cc0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ee80b2bd0 0x7f1ee80b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 --2- 172.21.15.133:0/2093002934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ee80b0db0 0x7f1ee8155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 -- 172.21.15.133:0/2093002934 >> 172.21.15.133:0/2093002934 conn(0x7f1ee80b31a0 msgr2=0x7f1ee809a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 -- 172.21.15.133:0/2093002934 shutdown_connections 2024-04-25T10:05:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.514+0000 7f1ef058c640 1 -- 172.21.15.133:0/2093002934 wait complete. 2024-04-25T10:05:37.537 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:37.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb8292a640 1 Processor -- start 2024-04-25T10:05:37.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb8292a640 1 -- start start 2024-04-25T10:05:37.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb8292a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdb7c152750 0x7fdb7c0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb8292a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7c06d350 con 0x7fdb7c152b30 2024-04-25T10:05:37.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb8292a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7c15bf90 con 0x7fdb7c0ce870 2024-04-25T10:05:37.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb8292a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7c0cecb0 con 0x7fdb7c152750 2024-04-25T10:05:37.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb73fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdb7c152b30 0x7fdb7c0b17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40064/0 (socket says 172.21.15.133:40064) 2024-04-25T10:05:37.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb73fff640 1 -- 172.21.15.133:0/1086610084 learned_addr learned my addr 172.21.15.133:0/1086610084 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:37.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7bfff640 1 --2- 172.21.15.133:0/1086610084 >> v2:172.21.15.162:3300/0 conn(0x7fdb7c152750 0x7fdb7c0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767488675 0 0) 0x7fdb64001120 con 0x7fdb7c152b30 2024-04-25T10:05:37.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb68003530 con 0x7fdb7c152b30 2024-04-25T10:05:37.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3756289017 0 0) 0x7fdb68003530 con 0x7fdb7c152b30 2024-04-25T10:05:37.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 >> v2:172.21.15.162:3300/0 conn(0x7fdb7c152750 msgr2=0x7fdb7c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 --2- 172.21.15.133:0/1086610084 >> v2:172.21.15.162:3300/0 conn(0x7fdb7c152750 0x7fdb7c0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 >> v1:172.21.15.162:6789/0 conn(0x7fdb7c0ce870 legacy=0x7fdb7c15f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c06d5d0 con 0x7fdb7c152b30 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fdb64001850 con 0x7fdb7c152b30 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 >> v1:172.21.15.133:6789/0 conn(0x7fdb7c152b30 legacy=0x7fdb7c0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 --2- 172.21.15.133:0/1086610084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb680039c0 0x7fdb68003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 --2- 172.21.15.133:0/1086610084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb68004810 0x7fdb68004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 --2- 172.21.15.133:0/1086610084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb680051d0 0x7fdb6800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c0cecb0 con 0x7fdb68004810 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6800c070 con 0x7fdb680039c0 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6800c270 con 0x7fdb680051d0 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb73fff640 1 --2- 172.21.15.133:0/1086610084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb68004810 0x7fdb68004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb7bfff640 1 --2- 172.21.15.133:0/1086610084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb680039c0 0x7fdb68003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb73fff640 1 -- 172.21.15.133:0/1086610084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb680051d0 msgr2=0x7fdb6800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb73fff640 1 --2- 172.21.15.133:0/1086610084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb680051d0 0x7fdb6800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb73fff640 1 -- 172.21.15.133:0/1086610084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb680039c0 msgr2=0x7fdb68003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb73fff640 1 --2- 172.21.15.133:0/1086610084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb680039c0 0x7fdb68003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb73fff640 1 -- 172.21.15.133:0/1086610084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb68003530 con 0x7fdb68004810 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.642+0000 7fdb80ec7640 1 -- 172.21.15.133:0/1086610084 reap_dead start 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb73fff640 1 --2- 172.21.15.133:0/1086610084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb68004810 0x7fdb68004c90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400c4c0 tx=0x7fdb6401f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17d553d50db8d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6400c0e0 con 0x7fdb68004810 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb64005b00 con 0x7fdb68004810 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb7b7fe640 1 -- 172.21.15.133:0/1086610084 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64007070 con 0x7fdb68004810 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 -- 172.21.15.133:0/1086610084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb68004810 msgr2=0x7fdb68004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 --2- 172.21.15.133:0/1086610084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb68004810 0x7fdb68004c90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400c4c0 tx=0x7fdb6401f420 comp rx=0 tx=0).stop 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 -- 172.21.15.133:0/1086610084 shutdown_connections 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 --2- 172.21.15.133:0/1086610084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb68004810 0x7fdb68004c90 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 -- 172.21.15.133:0/1086610084 >> 172.21.15.133:0/1086610084 conn(0x7fdb7c067000 msgr2=0x7fdb7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 -- 172.21.15.133:0/1086610084 shutdown_connections 2024-04-25T10:05:37.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 -- 172.21.15.133:0/1086610084 wait complete. 2024-04-25T10:05:37.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 Processor -- start 2024-04-25T10:05:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 -- start start 2024-04-25T10:05:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb68004810 0x7fdb7c156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb7c1569d0 0x7fdb7c141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb7c142500 0x7fdb7c14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c15c7f0 con 0x7fdb7c1569d0 2024-04-25T10:05:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c15c510 con 0x7fdb7c142500 2024-04-25T10:05:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.646+0000 7fdb8292a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c0ce330 con 0x7fdb68004810 2024-04-25T10:05:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb73fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb7c1569d0 0x7fdb7c141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb73fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb7c1569d0 0x7fdb7c141fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43438/0 (socket says 172.21.15.133:43438) 2024-04-25T10:05:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb73fff640 1 -- 172.21.15.133:0/1810684151 learned_addr learned my addr 172.21.15.133:0/1810684151 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb7bfff640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb68004810 0x7fdb7c156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb73fff640 1 -- 172.21.15.133:0/1810684151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb68004810 msgr2=0x7fdb7c156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb73fff640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb68004810 0x7fdb7c156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb73fff640 1 -- 172.21.15.133:0/1810684151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb7c142500 msgr2=0x7fdb7c14f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb80ec7640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb7c142500 0x7fdb7c14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb73fff640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb7c142500 0x7fdb7c14f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb73fff640 1 -- 172.21.15.133:0/1810684151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c06e920 con 0x7fdb7c1569d0 2024-04-25T10:05:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb7bfff640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb68004810 0x7fdb7c156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:37.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb73fff640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb7c1569d0 0x7fdb7c141fc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fdb64001b50 tx=0x7fdb68005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb797fa640 1 -- 172.21.15.133:0/1810684151 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64001120 con 0x7fdb7c1569d0 2024-04-25T10:05:37.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb797fa640 1 -- 172.21.15.133:0/1810684151 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb640b6070 con 0x7fdb7c1569d0 2024-04-25T10:05:37.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb8292a640 1 -- 172.21.15.133:0/1810684151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c0c9770 con 0x7fdb7c1569d0 2024-04-25T10:05:37.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb797fa640 1 -- 172.21.15.133:0/1810684151 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64001850 con 0x7fdb7c1569d0 2024-04-25T10:05:37.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb8292a640 1 -- 172.21.15.133:0/1810684151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7c0c9d30 con 0x7fdb7c1569d0 2024-04-25T10:05:37.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb8292a640 1 -- 172.21.15.133:0/1810684151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7c068ef0 con 0x7fdb7c1569d0 2024-04-25T10:05:37.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb797fa640 1 -- 172.21.15.133:0/1810684151 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb640047f0 con 0x7fdb7c1569d0 2024-04-25T10:05:37.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb797fa640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb5c066f60 0x7fdb5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb797fa640 1 -- 172.21.15.133:0/1810684151 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdb641228c0 con 0x7fdb7c1569d0 2024-04-25T10:05:37.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb7bfff640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb5c066f60 0x7fdb5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.650+0000 7fdb7bfff640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb5c066f60 0x7fdb5c069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009aa0 tx=0x7fdb6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.654+0000 7fdb797fa640 1 -- 172.21.15.133:0/1810684151 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdb7c06d350 con 0x7fdb7c1569d0 2024-04-25T10:05:37.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.802+0000 7fdb8292a640 1 -- 172.21.15.133:0/1810684151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb7c1428e0 con 0x7fdb7c1569d0 2024-04-25T10:05:37.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.802+0000 7fdb797fa640 1 -- 172.21.15.133:0/1810684151 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16979 (secure 0 0 0) 0x7fdb640f37d0 con 0x7fdb7c1569d0 2024-04-25T10:05:37.808 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:37.808 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":53,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:05:36.528355+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:34.805104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:35.512057+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:37.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 -- 172.21.15.133:0/1810684151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb5c066f60 msgr2=0x7fdb5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb5c066f60 0x7fdb5c069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009aa0 tx=0x7fdb6c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:37.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 -- 172.21.15.133:0/1810684151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb7c1569d0 msgr2=0x7fdb7c141fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb7c1569d0 0x7fdb7c141fc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fdb64001b50 tx=0x7fdb68005840 comp rx=0 tx=0).stop 2024-04-25T10:05:37.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 -- 172.21.15.133:0/1810684151 shutdown_connections 2024-04-25T10:05:37.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb7c142500 0x7fdb7c14f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb5c066f60 0x7fdb5c069420 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb7c1569d0 0x7fdb7c141fc0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 --2- 172.21.15.133:0/1810684151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb68004810 0x7fdb7c156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 -- 172.21.15.133:0/1810684151 >> 172.21.15.133:0/1810684151 conn(0x7fdb7c067000 msgr2=0x7fdb7c15a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 -- 172.21.15.133:0/1810684151 shutdown_connections 2024-04-25T10:05:37.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.806+0000 7fdb8292a640 1 -- 172.21.15.133:0/1810684151 wait complete. 2024-04-25T10:05:37.826 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:05:37.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213ebfe640 1 Processor -- start 2024-04-25T10:05:37.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213ebfe640 1 -- start start 2024-04-25T10:05:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213ebfe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f213815bc90 0x7f21380cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213ebfe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f21380d01d0 0x7f21380b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213ebfe640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21380cc6a0 con 0x7f213815b8b0 2024-04-25T10:05:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213ebfe640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21380d0640 con 0x7f21380d01d0 2024-04-25T10:05:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213ebfe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21380cc980 con 0x7f213815bc90 2024-04-25T10:05:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213c99a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f213815b8b0 0x7f21380ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40070/0 (socket says 172.21.15.133:40070) 2024-04-25T10:05:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213c99a640 1 -- 172.21.15.133:0/2249116237 learned_addr learned my addr 172.21.15.133:0/2249116237 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f2137fff640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f213815bc90 0x7f21380cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213d19b640 1 --2- 172.21.15.133:0/2249116237 >> v2:172.21.15.162:3300/0 conn(0x7f21380d01d0 0x7f21380b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212333970 0 0) 0x7f21380cc6a0 con 0x7f213815b8b0 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2124003530 con 0x7f213815b8b0 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2972576026 0 0) 0x7f2124003530 con 0x7f213815b8b0 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f213815bc90 msgr2=0x7f21380cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f213815bc90 0x7f21380cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 >> v2:172.21.15.162:3300/0 conn(0x7f21380d01d0 msgr2=0x7f21380b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 --2- 172.21.15.133:0/2249116237 >> v2:172.21.15.162:3300/0 conn(0x7f21380d01d0 0x7f21380b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21380ccc60 con 0x7f213815b8b0 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2128001680 con 0x7f213815b8b0 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 >> v1:172.21.15.133:6789/0 conn(0x7f213815b8b0 legacy=0x7f21380ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21240039c0 0x7f2124003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2124004810 0x7f2124004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f213815bc90 0x7f21240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380d0640 con 0x7f21240039c0 2024-04-25T10:05:37.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380cc980 con 0x7f2124004810 2024-04-25T10:05:37.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212400ba20 con 0x7f213815bc90 2024-04-25T10:05:37.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213c99a640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21240039c0 0x7f2124003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213d19b640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f213815bc90 0x7f21240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f2137fff640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2124004810 0x7f2124004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213c99a640 1 -- 172.21.15.133:0/2249116237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f213815bc90 msgr2=0x7f21240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213c99a640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f213815bc90 0x7f21240051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213c99a640 1 -- 172.21.15.133:0/2249116237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2124004810 msgr2=0x7f2124004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213c99a640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2124004810 0x7f2124004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.946+0000 7f213c99a640 1 -- 172.21.15.133:0/2249116237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2124003530 con 0x7f21240039c0 2024-04-25T10:05:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.950+0000 7f213c99a640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21240039c0 0x7f2124003de0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f212800c2f0 tx=0x7f212801cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c98fce0edab918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.950+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128007d60 con 0x7f21240039c0 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.950+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21280a9020 con 0x7f21240039c0 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.950+0000 7f21377fe640 1 -- 172.21.15.133:0/2249116237 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128005b00 con 0x7f21240039c0 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.950+0000 7f213ebfe640 1 -- 172.21.15.133:0/2249116237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21240039c0 msgr2=0x7f2124003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.950+0000 7f213ebfe640 1 --2- 172.21.15.133:0/2249116237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21240039c0 0x7f2124003de0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f212800c2f0 tx=0x7f212801cd40 comp rx=0 tx=0).stop 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.950+0000 7f213d19b640 1 -- 172.21.15.133:0/2249116237 reap_dead start 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.950+0000 7f213ebfe640 1 -- 172.21.15.133:0/2249116237 shutdown_connections 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.950+0000 7f213ebfe640 1 -- 172.21.15.133:0/2249116237 >> 172.21.15.133:0/2249116237 conn(0x7f213809a990 msgr2=0x7f21380d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.950+0000 7f213ebfe640 1 -- 172.21.15.133:0/2249116237 shutdown_connections 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.950+0000 7f213ebfe640 1 -- 172.21.15.133:0/2249116237 wait complete. 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213ebfe640 1 Processor -- start 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213ebfe640 1 -- start start 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213ebfe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21240039c0 0x7f21381f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213ebfe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21380b2d50 0x7f21380b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213ebfe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f213815c310 0x7f213815c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213ebfe640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380b17f0 con 0x7f21380b2d50 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213ebfe640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380cc440 con 0x7f213815c310 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213ebfe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21380b1970 con 0x7f21240039c0 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f2137fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21380b2d50 0x7f21380b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f2137fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21380b2d50 0x7f21380b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43466/0 (socket says 172.21.15.133:43466) 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f2137fff640 1 -- 172.21.15.133:0/3437707221 learned_addr learned my addr 172.21.15.133:0/3437707221 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213d19b640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f213815c310 0x7f213815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213c99a640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21240039c0 0x7f21381f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f2137fff640 1 -- 172.21.15.133:0/3437707221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21240039c0 msgr2=0x7f21381f7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f2137fff640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21240039c0 0x7f21381f7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f2137fff640 1 -- 172.21.15.133:0/3437707221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f213815c310 msgr2=0x7f213815c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f2137fff640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f213815c310 0x7f213815c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f2137fff640 1 -- 172.21.15.133:0/3437707221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213814c8e0 con 0x7f21380b2d50 2024-04-25T10:05:37.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f2137fff640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21380b2d50 0x7f21380b31b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2130009fb0 tx=0x7f21300115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f21357fa640 1 -- 172.21.15.133:0/3437707221 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213001a020 con 0x7f21380b2d50 2024-04-25T10:05:37.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f21357fa640 1 -- 172.21.15.133:0/3437707221 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2130016470 con 0x7f21380b2d50 2024-04-25T10:05:37.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213ebfe640 1 -- 172.21.15.133:0/3437707221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213814cbc0 con 0x7f21380b2d50 2024-04-25T10:05:37.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f21357fa640 1 -- 172.21.15.133:0/3437707221 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130016860 con 0x7f21380b2d50 2024-04-25T10:05:37.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213ebfe640 1 -- 172.21.15.133:0/3437707221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213814d180 con 0x7f21380b2d50 2024-04-25T10:05:37.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213ebfe640 1 -- 172.21.15.133:0/3437707221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2138067c70 con 0x7f21380b2d50 2024-04-25T10:05:37.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f21357fa640 1 -- 172.21.15.133:0/3437707221 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f213001d020 con 0x7f21380b2d50 2024-04-25T10:05:37.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f21357fa640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2118066f90 0x7f2118069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f21357fa640 1 -- 172.21.15.133:0/3437707221 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2130086050 con 0x7f21380b2d50 2024-04-25T10:05:37.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.954+0000 7f213c99a640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2118066f90 0x7f2118069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.958+0000 7f213c99a640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2118066f90 0x7f2118069450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f21280aa000 tx=0x7f2128001a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:37.958+0000 7f21357fa640 1 -- 172.21.15.133:0/3437707221 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2130024020 con 0x7f21380b2d50 2024-04-25T10:05:38.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.154+0000 7f213ebfe640 1 -- 172.21.15.133:0/3437707221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f21380b0860 con 0x7f21380b2d50 2024-04-25T10:05:38.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.218+0000 7f21357fa640 1 -- 172.21.15.133:0/3437707221 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2130056ed0 con 0x7f21380b2d50 2024-04-25T10:05:38.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 -- 172.21.15.133:0/3437707221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2118066f90 msgr2=0x7f2118069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2118066f90 0x7f2118069450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f21280aa000 tx=0x7f2128001a50 comp rx=0 tx=0).stop 2024-04-25T10:05:38.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 -- 172.21.15.133:0/3437707221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21380b2d50 msgr2=0x7f21380b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21380b2d50 0x7f21380b31b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2130009fb0 tx=0x7f21300115a0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 -- 172.21.15.133:0/3437707221 shutdown_connections 2024-04-25T10:05:38.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f213815c310 0x7f213815c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2118066f90 0x7f2118069450 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21380b2d50 0x7f21380b31b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 --2- 172.21.15.133:0/3437707221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21240039c0 0x7f21381f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 -- 172.21.15.133:0/3437707221 >> 172.21.15.133:0/3437707221 conn(0x7f213809a990 msgr2=0x7f213806f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 -- 172.21.15.133:0/3437707221 shutdown_connections 2024-04-25T10:05:38.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.222+0000 7f213ebfe640 1 -- 172.21.15.133:0/3437707221 wait complete. 2024-04-25T10:05:38.245 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:05:38.246 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:05:38.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bd6544640 1 Processor -- start 2024-04-25T10:05:38.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bd6544640 1 -- start start 2024-04-25T10:05:38.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bd6544640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3bd015d9a0 0x7f3bd00d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bd6544640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd015ebd0 con 0x7f3bd00ce600 2024-04-25T10:05:38.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bd6544640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd015eeb0 con 0x7f3bd00d2030 2024-04-25T10:05:38.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bd6544640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00d0330 con 0x7f3bd015d9a0 2024-04-25T10:05:38.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bcffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3bd015d9a0 0x7f3bd00d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bcf7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3bd00ce600 0x7f3bd00d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40072/0 (socket says 172.21.15.133:40072) 2024-04-25T10:05:38.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bcf7fe640 1 -- 172.21.15.133:0/2701167117 learned_addr learned my addr 172.21.15.133:0/2701167117 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:38.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bcffff640 1 -- 172.21.15.133:0/2701167117 >> v1:172.21.15.162:6789/0 conn(0x7f3bd00d2030 legacy=0x7f3bd015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:38.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bcffff640 1 -- 172.21.15.133:0/2701167117 >> v1:172.21.15.133:6789/0 conn(0x7f3bd00ce600 legacy=0x7f3bd00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.362+0000 7f3bcffff640 1 -- 172.21.15.133:0/2701167117 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00d0610 con 0x7f3bd015d9a0 2024-04-25T10:05:38.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bcffff640 1 --2- 172.21.15.133:0/2701167117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd015d9a0 0x7f3bd00d0b30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00097c0 tx=0x7f3bc001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b03e6d00e3de7f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bceffd640 1 -- 172.21.15.133:0/2701167117 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc00b2070 con 0x7f3bd015d9a0 2024-04-25T10:05:38.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bceffd640 1 -- 172.21.15.133:0/2701167117 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc00ad680 con 0x7f3bd015d9a0 2024-04-25T10:05:38.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bceffd640 1 -- 172.21.15.133:0/2701167117 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc00ada50 con 0x7f3bd015d9a0 2024-04-25T10:05:38.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bd6544640 1 -- 172.21.15.133:0/2701167117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd015d9a0 msgr2=0x7f3bd00d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bd6544640 1 --2- 172.21.15.133:0/2701167117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd015d9a0 0x7f3bd00d0b30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00097c0 tx=0x7f3bc001c3a0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bd6544640 1 -- 172.21.15.133:0/2701167117 shutdown_connections 2024-04-25T10:05:38.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bd6544640 1 --2- 172.21.15.133:0/2701167117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd015d9a0 0x7f3bd00d0b30 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bd6544640 1 -- 172.21.15.133:0/2701167117 >> 172.21.15.133:0/2701167117 conn(0x7f3bd006bf80 msgr2=0x7f3bd006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bd6544640 1 -- 172.21.15.133:0/2701167117 shutdown_connections 2024-04-25T10:05:38.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bd6544640 1 -- 172.21.15.133:0/2701167117 wait complete. 2024-04-25T10:05:38.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bd6544640 1 Processor -- start 2024-04-25T10:05:38.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.366+0000 7f3bd6544640 1 -- start start 2024-04-25T10:05:38.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd6544640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd00ce600 0x7f3bd00c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd6544640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd00d2030 0x7f3bd00c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd6544640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd015d9a0 0x7f3bd01578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd6544640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00b96a0 con 0x7f3bd015d9a0 2024-04-25T10:05:38.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd6544640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00cffd0 con 0x7f3bd00ce600 2024-04-25T10:05:38.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd6544640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00b9820 con 0x7f3bd00d2030 2024-04-25T10:05:38.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd4ae1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd015d9a0 0x7f3bd01578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd4ae1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd015d9a0 0x7f3bd01578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43490/0 (socket says 172.21.15.133:43490) 2024-04-25T10:05:38.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd4ae1640 1 -- 172.21.15.133:0/3537225982 learned_addr learned my addr 172.21.15.133:0/3537225982 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:38.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bcffff640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd00ce600 0x7f3bd00c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd4ae1640 1 -- 172.21.15.133:0/3537225982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd00d2030 msgr2=0x7f3bd00c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd4ae1640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd00d2030 0x7f3bd00c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd4ae1640 1 -- 172.21.15.133:0/3537225982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd00ce600 msgr2=0x7f3bd00c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd4ae1640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd00ce600 0x7f3bd00c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd4ae1640 1 -- 172.21.15.133:0/3537225982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00c8ef0 con 0x7f3bd015d9a0 2024-04-25T10:05:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd4ae1640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd015d9a0 0x7f3bd01578e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800c780 tx=0x7f3bc800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bccff9640 1 -- 172.21.15.133:0/3537225982 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8004b70 con 0x7f3bd015d9a0 2024-04-25T10:05:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bccff9640 1 -- 172.21.15.133:0/3537225982 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc8004d10 con 0x7f3bd015d9a0 2024-04-25T10:05:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd6544640 1 -- 172.21.15.133:0/3537225982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd00c91d0 con 0x7f3bd015d9a0 2024-04-25T10:05:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bccff9640 1 -- 172.21.15.133:0/3537225982 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800aa90 con 0x7f3bd015d9a0 2024-04-25T10:05:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd6544640 1 -- 172.21.15.133:0/3537225982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd00c9790 con 0x7f3bd015d9a0 2024-04-25T10:05:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bd6544640 1 -- 172.21.15.133:0/3537225982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd0068e80 con 0x7f3bd015d9a0 2024-04-25T10:05:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bccff9640 1 -- 172.21.15.133:0/3537225982 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3bc800ac30 con 0x7f3bd015d9a0 2024-04-25T10:05:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bccff9640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bbc066f60 0x7f3bbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bcffff640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bbc066f60 0x7f3bbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bccff9640 1 -- 172.21.15.133:0/3537225982 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3bc8085460 con 0x7f3bd015d9a0 2024-04-25T10:05:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.370+0000 7f3bcffff640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bbc066f60 0x7f3bbc069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00af000 tx=0x7f3bc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.374+0000 7f3bccff9640 1 -- 172.21.15.133:0/3537225982 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3bc8020070 con 0x7f3bd015d9a0 2024-04-25T10:05:38.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:38.562+0000 7f3bd6544640 1 -- 172.21.15.133:0/3537225982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3bd015e190 con 0x7f3bd015d9a0 2024-04-25T10:05:38.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T10:05:38.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T10:05:38.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:05:38.899 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:05:39.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.222+0000 7f3bccff9640 1 -- 172.21.15.133:0/3537225982 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3bc80562e0 con 0x7f3bd015d9a0 2024-04-25T10:05:39.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 -- 172.21.15.133:0/3537225982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bbc066f60 msgr2=0x7f3bbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bbc066f60 0x7f3bbc069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00af000 tx=0x7f3bc0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:39.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 -- 172.21.15.133:0/3537225982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd015d9a0 msgr2=0x7f3bd01578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd015d9a0 0x7f3bd01578e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800c780 tx=0x7f3bc800cc40 comp rx=0 tx=0).stop 2024-04-25T10:05:39.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 -- 172.21.15.133:0/3537225982 shutdown_connections 2024-04-25T10:05:39.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3bbc066f60 0x7f3bbc069420 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd015d9a0 0x7f3bd01578e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bd00d2030 0x7f3bd00c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 --2- 172.21.15.133:0/3537225982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bd00ce600 0x7f3bd00c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 -- 172.21.15.133:0/3537225982 >> 172.21.15.133:0/3537225982 conn(0x7f3bd006bf80 msgr2=0x7f3bd00b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 -- 172.21.15.133:0/3537225982 shutdown_connections 2024-04-25T10:05:39.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.226+0000 7f3bd6544640 1 -- 172.21.15.133:0/3537225982 wait complete. 2024-04-25T10:05:39.257 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:05:39.258 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.370+0000 7f3dbae53640 1 Processor -- start 2024-04-25T10:05:39.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.370+0000 7f3dbae53640 1 -- start start 2024-04-25T10:05:39.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3dbae53640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3db415d9b0 0x7f3db40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3dbae53640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db40ca9b0 con 0x7f3db415dd90 2024-04-25T10:05:39.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3dbae53640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db40cac90 con 0x7f3db40cee80 2024-04-25T10:05:39.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3dbae53640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db40cbe30 con 0x7f3db415d9b0 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db3fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3db415dd90 0x7f3db40ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40084/0 (socket says 172.21.15.133:40084) 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db3fff640 1 -- 172.21.15.133:0/3449764908 learned_addr learned my addr 172.21.15.133:0/3449764908 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db8bef640 1 --2- 172.21.15.133:0/3449764908 >> v2:172.21.15.162:3300/0 conn(0x7f3db415d9b0 0x7f3db40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792273567 0 0) 0x7f3db40ca9b0 con 0x7f3db415dd90 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3da0003530 con 0x7f3db415dd90 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 761217861 0 0) 0x7f3da0003530 con 0x7f3db415dd90 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 >> v2:172.21.15.162:3300/0 conn(0x7f3db415d9b0 msgr2=0x7f3db40cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 --2- 172.21.15.133:0/3449764908 >> v2:172.21.15.162:3300/0 conn(0x7f3db415d9b0 0x7f3db40cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 >> v1:172.21.15.162:6789/0 conn(0x7f3db40cee80 legacy=0x7f3db406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db40caf70 con 0x7f3db415dd90 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f3d9c0016a0 con 0x7f3db415dd90 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 >> v1:172.21.15.133:6789/0 conn(0x7f3db415dd90 legacy=0x7f3db40ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 --2- 172.21.15.133:0/3449764908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da00039c0 0x7f3da0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 --2- 172.21.15.133:0/3449764908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3da0004810 0x7f3da0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 --2- 172.21.15.133:0/3449764908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3da00051d0 0x7f3da000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40cbe30 con 0x7f3da00039c0 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da000c170 con 0x7f3da00051d0 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da000c370 con 0x7f3da0004810 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db8bef640 1 --2- 172.21.15.133:0/3449764908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da00039c0 0x7f3da0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db93f0640 1 --2- 172.21.15.133:0/3449764908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3da00051d0 0x7f3da000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db3fff640 1 --2- 172.21.15.133:0/3449764908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3da0004810 0x7f3da0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db8bef640 1 -- 172.21.15.133:0/3449764908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3da0004810 msgr2=0x7f3da0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db8bef640 1 --2- 172.21.15.133:0/3449764908 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3da0004810 0x7f3da0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db8bef640 1 -- 172.21.15.133:0/3449764908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3da00051d0 msgr2=0x7f3da000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db8bef640 1 --2- 172.21.15.133:0/3449764908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3da00051d0 0x7f3da000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db8bef640 1 -- 172.21.15.133:0/3449764908 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da0003530 con 0x7f3da00039c0 2024-04-25T10:05:39.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.374+0000 7f3db93f0640 1 -- 172.21.15.133:0/3449764908 reap_dead start 2024-04-25T10:05:39.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db8bef640 1 --2- 172.21.15.133:0/3449764908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da00039c0 0x7f3da0003de0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009fc0 tx=0x7f3da401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce978ec945efd4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da40b2070 con 0x7f3da00039c0 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da40a5ba0 con 0x7f3da00039c0 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db37fe640 1 -- 172.21.15.133:0/3449764908 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4009c10 con 0x7f3da00039c0 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 -- 172.21.15.133:0/3449764908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da00039c0 msgr2=0x7f3da0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 --2- 172.21.15.133:0/3449764908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da00039c0 0x7f3da0003de0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009fc0 tx=0x7f3da401cc80 comp rx=0 tx=0).stop 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 -- 172.21.15.133:0/3449764908 shutdown_connections 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 --2- 172.21.15.133:0/3449764908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da00039c0 0x7f3da0003de0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 -- 172.21.15.133:0/3449764908 >> 172.21.15.133:0/3449764908 conn(0x7f3db4067000 msgr2=0x7f3db406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 -- 172.21.15.133:0/3449764908 shutdown_connections 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 -- 172.21.15.133:0/3449764908 wait complete. 2024-04-25T10:05:39.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 Processor -- start 2024-04-25T10:05:39.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 -- start start 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3da00039c0 0x7f3db415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db41529d0 0x7f3db4152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3db41f6e30 0x7f3db41f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4161f00 con 0x7f3db41529d0 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4162080 con 0x7f3da00039c0 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4162200 con 0x7f3db41f6e30 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db41529d0 0x7f3db4152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db41529d0 0x7f3db4152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43508/0 (socket says 172.21.15.133:43508) 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db3fff640 1 -- 172.21.15.133:0/921128192 learned_addr learned my addr 172.21.15.133:0/921128192 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db93f0640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3db41f6e30 0x7f3db41f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db8bef640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3da00039c0 0x7f3db415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db3fff640 1 -- 172.21.15.133:0/921128192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3db41f6e30 msgr2=0x7f3db41f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db3fff640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3db41f6e30 0x7f3db41f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db3fff640 1 -- 172.21.15.133:0/921128192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3da00039c0 msgr2=0x7f3db415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db3fff640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3da00039c0 0x7f3db415bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db3fff640 1 -- 172.21.15.133:0/921128192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db4153620 con 0x7f3db41529d0 2024-04-25T10:05:39.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db93f0640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3db41f6e30 0x7f3db41f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:39.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db3fff640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db41529d0 0x7f3db4152e50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00e790 tx=0x7f3d9c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db17fa640 1 -- 172.21.15.133:0/921128192 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00c8d0 con 0x7f3db41529d0 2024-04-25T10:05:39.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db17fa640 1 -- 172.21.15.133:0/921128192 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c018070 con 0x7f3db41529d0 2024-04-25T10:05:39.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 -- 172.21.15.133:0/921128192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db4153390 con 0x7f3db41529d0 2024-04-25T10:05:39.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3db17fa640 1 -- 172.21.15.133:0/921128192 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00ccb0 con 0x7f3db41529d0 2024-04-25T10:05:39.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.378+0000 7f3dbae53640 1 -- 172.21.15.133:0/921128192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db415a200 con 0x7f3db41529d0 2024-04-25T10:05:39.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.382+0000 7f3dbae53640 1 -- 172.21.15.133:0/921128192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db4068ef0 con 0x7f3db41529d0 2024-04-25T10:05:39.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.382+0000 7f3db17fa640 1 -- 172.21.15.133:0/921128192 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d9c01d460 con 0x7f3db41529d0 2024-04-25T10:05:39.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.382+0000 7f3db17fa640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3d94066f60 0x7f3d94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.382+0000 7f3db17fa640 1 -- 172.21.15.133:0/921128192 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d9c085170 con 0x7f3db41529d0 2024-04-25T10:05:39.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.382+0000 7f3db8bef640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3d94066f60 0x7f3d94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.382+0000 7f3db8bef640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3d94066f60 0x7f3d94069420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3da40af000 tx=0x7f3da40a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.386+0000 7f3db17fa640 1 -- 172.21.15.133:0/921128192 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d9c055f50 con 0x7f3db41529d0 2024-04-25T10:05:39.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.574+0000 7f3dbae53640 1 -- 172.21.15.133:0/921128192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3db406c2d0 con 0x7f3db41529d0 2024-04-25T10:05:39.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.574+0000 7f3db17fa640 1 -- 172.21.15.133:0/921128192 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f3d9c055850 con 0x7f3db41529d0 2024-04-25T10:05:39.580 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:39.580 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:05:38.566660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:39.580 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2024-04-25T10:05:39.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 -- 172.21.15.133:0/921128192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3d94066f60 msgr2=0x7f3d94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3d94066f60 0x7f3d94069420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3da40af000 tx=0x7f3da40a7040 comp rx=0 tx=0).stop 2024-04-25T10:05:39.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 -- 172.21.15.133:0/921128192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db41529d0 msgr2=0x7f3db4152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db41529d0 0x7f3db4152e50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00e790 tx=0x7f3d9c00ec50 comp rx=0 tx=0).stop 2024-04-25T10:05:39.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 -- 172.21.15.133:0/921128192 shutdown_connections 2024-04-25T10:05:39.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3db41f6e30 0x7f3db41f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3d94066f60 0x7f3d94069420 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db41529d0 0x7f3db4152e50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 --2- 172.21.15.133:0/921128192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3da00039c0 0x7f3db415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 -- 172.21.15.133:0/921128192 >> 172.21.15.133:0/921128192 conn(0x7f3db4067000 msgr2=0x7f3db40b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 -- 172.21.15.133:0/921128192 shutdown_connections 2024-04-25T10:05:39.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.578+0000 7f3dbae53640 1 -- 172.21.15.133:0/921128192 wait complete. 2024-04-25T10:05:39.598 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:39.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.706+0000 7fcd80927640 1 Processor -- start 2024-04-25T10:05:39.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.706+0000 7fcd80927640 1 -- start start 2024-04-25T10:05:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.706+0000 7fcd80927640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcd7c0b1f70 0x7fcd7c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.706+0000 7fcd80927640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c0c61d0 con 0x7fcd7c0ce330 2024-04-25T10:05:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.706+0000 7fcd80927640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c0c64b0 con 0x7fcd7c15c9a0 2024-04-25T10:05:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.706+0000 7fcd80927640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c06d4f0 con 0x7fcd7c0b1f70 2024-04-25T10:05:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd7ad9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcd7c0ce330 0x7fcd7c15d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40096/0 (socket says 172.21.15.133:40096) 2024-04-25T10:05:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd7ad9d640 1 -- 172.21.15.133:0/3235135784 learned_addr learned my addr 172.21.15.133:0/3235135784 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd79d9b640 1 --2- 172.21.15.133:0/3235135784 >> v2:172.21.15.133:3300/0 conn(0x7fcd7c0b1f70 0x7fcd7c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd7959a640 1 -- 172.21.15.133:0/3235135784 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759255408 0 0) 0x7fcd7c0c61d0 con 0x7fcd7c0ce330 2024-04-25T10:05:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd7959a640 1 -- 172.21.15.133:0/3235135784 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd64003530 con 0x7fcd7c0ce330 2024-04-25T10:05:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd79d9b640 1 -- 172.21.15.133:0/3235135784 >> v1:172.21.15.162:6789/0 conn(0x7fcd7c15c9a0 legacy=0x7fcd7c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd79d9b640 1 -- 172.21.15.133:0/3235135784 >> v1:172.21.15.133:6789/0 conn(0x7fcd7c0ce330 legacy=0x7fcd7c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd7959a640 1 -- 172.21.15.133:0/3235135784 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2999965188 0 0) 0x7fcd64003530 con 0x7fcd7c0ce330 2024-04-25T10:05:39.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd79d9b640 1 -- 172.21.15.133:0/3235135784 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c06d7d0 con 0x7fcd7c0b1f70 2024-04-25T10:05:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd79d9b640 1 --2- 172.21.15.133:0/3235135784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd7c0b1f70 0x7fcd7c06dcf0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcd700097c0 tx=0x7fcd7001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c5f3f26310b341 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd7959a640 1 -- 172.21.15.133:0/3235135784 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd700b2070 con 0x7fcd7c0b1f70 2024-04-25T10:05:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd7959a640 1 -- 172.21.15.133:0/3235135784 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd700a9070 con 0x7fcd7c0b1f70 2024-04-25T10:05:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd7959a640 1 -- 172.21.15.133:0/3235135784 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c001680 con 0x7fcd7c0b1f70 2024-04-25T10:05:39.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd80927640 1 -- 172.21.15.133:0/3235135784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd7c0b1f70 msgr2=0x7fcd7c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd80927640 1 --2- 172.21.15.133:0/3235135784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd7c0b1f70 0x7fcd7c06dcf0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcd700097c0 tx=0x7fcd7001c3c0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd80927640 1 -- 172.21.15.133:0/3235135784 shutdown_connections 2024-04-25T10:05:39.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd80927640 1 --2- 172.21.15.133:0/3235135784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd7c0b1f70 0x7fcd7c06dcf0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd80927640 1 -- 172.21.15.133:0/3235135784 >> 172.21.15.133:0/3235135784 conn(0x7fcd7c067000 msgr2=0x7fcd7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd80927640 1 -- 172.21.15.133:0/3235135784 shutdown_connections 2024-04-25T10:05:39.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.710+0000 7fcd80927640 1 -- 172.21.15.133:0/3235135784 wait complete. 2024-04-25T10:05:39.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd80927640 1 Processor -- start 2024-04-25T10:05:39.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd80927640 1 -- start start 2024-04-25T10:05:39.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd80927640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd7c0b1f70 0x7fcd7c145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd80927640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd7c0ce330 0x7fcd7c158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd80927640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd7c15c9a0 0x7fcd7c142f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd80927640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c15cf00 con 0x7fcd7c0ce330 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd80927640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c15cd80 con 0x7fcd7c15c9a0 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd80927640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c169de0 con 0x7fcd7c0b1f70 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd79d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd7c0ce330 0x7fcd7c158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd79d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd7c0ce330 0x7fcd7c158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43530/0 (socket says 172.21.15.133:43530) 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd79d9b640 1 -- 172.21.15.133:0/171630304 learned_addr learned my addr 172.21.15.133:0/171630304 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd7a59c640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd7c0b1f70 0x7fcd7c145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd7ad9d640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd7c15c9a0 0x7fcd7c142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd79d9b640 1 -- 172.21.15.133:0/171630304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd7c0b1f70 msgr2=0x7fcd7c145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd79d9b640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd7c0b1f70 0x7fcd7c145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd79d9b640 1 -- 172.21.15.133:0/171630304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd7c15c9a0 msgr2=0x7fcd7c142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd79d9b640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd7c15c9a0 0x7fcd7c142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd79d9b640 1 -- 172.21.15.133:0/171630304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c153f30 con 0x7fcd7c0ce330 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd7a59c640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd7c0b1f70 0x7fcd7c145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd7ad9d640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd7c15c9a0 0x7fcd7c142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:39.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd79d9b640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd7c0ce330 0x7fcd7c158e80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcd70018fd0 tx=0x7fcd70019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd6affd640 1 -- 172.21.15.133:0/171630304 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd700b2040 con 0x7fcd7c0ce330 2024-04-25T10:05:39.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd6affd640 1 -- 172.21.15.133:0/171630304 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd700adb60 con 0x7fcd7c0ce330 2024-04-25T10:05:39.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd6affd640 1 -- 172.21.15.133:0/171630304 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70006e20 con 0x7fcd7c0ce330 2024-04-25T10:05:39.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd80927640 1 -- 172.21.15.133:0/171630304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c157bd0 con 0x7fcd7c0ce330 2024-04-25T10:05:39.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd80927640 1 -- 172.21.15.133:0/171630304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c159510 con 0x7fcd7c0ce330 2024-04-25T10:05:39.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.714+0000 7fcd80927640 1 -- 172.21.15.133:0/171630304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c068ef0 con 0x7fcd7c0ce330 2024-04-25T10:05:39.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.718+0000 7fcd6affd640 1 -- 172.21.15.133:0/171630304 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd700be020 con 0x7fcd7c0ce330 2024-04-25T10:05:39.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.718+0000 7fcd6affd640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd5c066f60 0x7fcd5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.718+0000 7fcd6affd640 1 -- 172.21.15.133:0/171630304 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd7011e870 con 0x7fcd7c0ce330 2024-04-25T10:05:39.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.718+0000 7fcd7a59c640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd5c066f60 0x7fcd5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.718+0000 7fcd7a59c640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd5c066f60 0x7fcd5c069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcd74007fd0 tx=0x7fcd74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.722+0000 7fcd6affd640 1 -- 172.21.15.133:0/171630304 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd70015fb0 con 0x7fcd7c0ce330 2024-04-25T10:05:39.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.870+0000 7fcd80927640 1 -- 172.21.15.133:0/171630304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd7c0c6bd0 con 0x7fcd7c0ce330 2024-04-25T10:05:39.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd6affd640 1 -- 172.21.15.133:0/171630304 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16966 (secure 0 0 0) 0x7fcd700ef770 con 0x7fcd7c0ce330 2024-04-25T10:05:39.877 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:39.877 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":54,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:05:38.186046+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:34.805104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:35.512057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:39.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 -- 172.21.15.133:0/171630304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd5c066f60 msgr2=0x7fcd5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd5c066f60 0x7fcd5c069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcd74007fd0 tx=0x7fcd74009040 comp rx=0 tx=0).stop 2024-04-25T10:05:39.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 -- 172.21.15.133:0/171630304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd7c0ce330 msgr2=0x7fcd7c158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd7c0ce330 0x7fcd7c158e80 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcd70018fd0 tx=0x7fcd70019160 comp rx=0 tx=0).stop 2024-04-25T10:05:39.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 -- 172.21.15.133:0/171630304 shutdown_connections 2024-04-25T10:05:39.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd7c15c9a0 0x7fcd7c142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd5c066f60 0x7fcd5c069420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd7c0ce330 0x7fcd7c158e80 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 --2- 172.21.15.133:0/171630304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd7c0b1f70 0x7fcd7c145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 -- 172.21.15.133:0/171630304 >> 172.21.15.133:0/171630304 conn(0x7fcd7c067000 msgr2=0x7fcd7c15b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 -- 172.21.15.133:0/171630304 shutdown_connections 2024-04-25T10:05:39.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:39.874+0000 7fcd80927640 1 -- 172.21.15.133:0/171630304 wait complete. 2024-04-25T10:05:39.896 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:05:39.896 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:40.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.014+0000 7fb8e7b74640 1 Processor -- start 2024-04-25T10:05:40.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.014+0000 7fb8e7b74640 1 -- start start 2024-04-25T10:05:40.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e7b74640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb8e015c9a0 0x7fb8e00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e7b74640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb8e00b1f70 0x7fb8e006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e7b74640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e00c61d0 con 0x7fb8e00ce330 2024-04-25T10:05:40.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e7b74640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e00c64b0 con 0x7fb8e00b1f70 2024-04-25T10:05:40.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e7b74640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e006d4f0 con 0x7fb8e015c9a0 2024-04-25T10:05:40.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e510f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb8e00b1f70 0x7fb8e006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e510f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb8e00b1f70 0x7fb8e006dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43546/0 (socket says 172.21.15.133:43546) 2024-04-25T10:05:40.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e510f640 1 -- 172.21.15.133:0/4034832322 learned_addr learned my addr 172.21.15.133:0/4034832322 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:40.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e5910640 1 --2- 172.21.15.133:0/4034832322 >> v2:172.21.15.162:3300/0 conn(0x7fb8e015c9a0 0x7fb8e00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e510f640 1 -- 172.21.15.133:0/4034832322 >> v2:172.21.15.162:3300/0 conn(0x7fb8e015c9a0 msgr2=0x7fb8e00c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e510f640 1 --2- 172.21.15.133:0/4034832322 >> v2:172.21.15.162:3300/0 conn(0x7fb8e015c9a0 0x7fb8e00c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e510f640 1 -- 172.21.15.133:0/4034832322 >> v1:172.21.15.162:6789/0 conn(0x7fb8e00ce330 legacy=0x7fb8e015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e510f640 1 -- 172.21.15.133:0/4034832322 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e006d7d0 con 0x7fb8e00b1f70 2024-04-25T10:05:40.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e510f640 1 --2- 172.21.15.133:0/4034832322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e00b1f70 0x7fb8e006dcf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb8e006f5f0 tx=0x7fb8d401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fb0424f5b08b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e490e640 1 -- 172.21.15.133:0/4034832322 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d40aa600 con 0x7fb8e00b1f70 2024-04-25T10:05:40.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e490e640 1 -- 172.21.15.133:0/4034832322 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d40af070 con 0x7fb8e00b1f70 2024-04-25T10:05:40.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e490e640 1 -- 172.21.15.133:0/4034832322 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d40aa9c0 con 0x7fb8e00b1f70 2024-04-25T10:05:40.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e7b74640 1 -- 172.21.15.133:0/4034832322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e00b1f70 msgr2=0x7fb8e006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e7b74640 1 --2- 172.21.15.133:0/4034832322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e00b1f70 0x7fb8e006dcf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb8e006f5f0 tx=0x7fb8d401c480 comp rx=0 tx=0).stop 2024-04-25T10:05:40.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e7b74640 1 -- 172.21.15.133:0/4034832322 shutdown_connections 2024-04-25T10:05:40.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e7b74640 1 --2- 172.21.15.133:0/4034832322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e00b1f70 0x7fb8e006dcf0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e7b74640 1 --2- 172.21.15.133:0/4034832322 >> v2:172.21.15.162:3300/0 conn(0x7fb8e015c9a0 0x7fb8e00c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.018+0000 7fb8e7b74640 1 -- 172.21.15.133:0/4034832322 >> 172.21.15.133:0/4034832322 conn(0x7fb8e0067000 msgr2=0x7fb8e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 -- 172.21.15.133:0/4034832322 shutdown_connections 2024-04-25T10:05:40.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 -- 172.21.15.133:0/4034832322 wait complete. 2024-04-25T10:05:40.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 Processor -- start 2024-04-25T10:05:40.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 -- start start 2024-04-25T10:05:40.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e00b1f70 0x7fb8e0146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8e00ce330 0x7fb8e014a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e015c9a0 0x7fb8e0146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015cd80 con 0x7fb8e00b1f70 2024-04-25T10:05:40.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e006e230 con 0x7fb8e015c9a0 2024-04-25T10:05:40.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015cf00 con 0x7fb8e00ce330 2024-04-25T10:05:40.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e5910640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e00b1f70 0x7fb8e0146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e5910640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e00b1f70 0x7fb8e0146ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43552/0 (socket says 172.21.15.133:43552) 2024-04-25T10:05:40.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e5910640 1 -- 172.21.15.133:0/331843535 learned_addr learned my addr 172.21.15.133:0/331843535 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e6111640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e015c9a0 0x7fb8e0146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e510f640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8e00ce330 0x7fb8e014a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e5910640 1 -- 172.21.15.133:0/331843535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8e00ce330 msgr2=0x7fb8e014a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e5910640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8e00ce330 0x7fb8e014a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e5910640 1 -- 172.21.15.133:0/331843535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e015c9a0 msgr2=0x7fb8e0146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e5910640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e015c9a0 0x7fb8e0146700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e5910640 1 -- 172.21.15.133:0/331843535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e0145940 con 0x7fb8e00b1f70 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e510f640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8e00ce330 0x7fb8e014a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e5910640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e00b1f70 0x7fb8e0146ce0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00c7f0 tx=0x7fb8dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8d27fc640 1 -- 172.21.15.133:0/331843535 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc015070 con 0x7fb8e00b1f70 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8d27fc640 1 -- 172.21.15.133:0/331843535 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc004240 con 0x7fb8e00b1f70 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 -- 172.21.15.133:0/331843535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e0145c20 con 0x7fb8e00b1f70 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8d27fc640 1 -- 172.21.15.133:0/331843535 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc003620 con 0x7fb8e00b1f70 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 -- 172.21.15.133:0/331843535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e01461e0 con 0x7fb8e00b1f70 2024-04-25T10:05:40.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.022+0000 7fb8e7b74640 1 -- 172.21.15.133:0/331843535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e006bc20 con 0x7fb8e00b1f70 2024-04-25T10:05:40.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.026+0000 7fb8d27fc640 1 -- 172.21.15.133:0/331843535 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8dc0039d0 con 0x7fb8e00b1f70 2024-04-25T10:05:40.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.026+0000 7fb8d27fc640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8c4066f60 0x7fb8c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.026+0000 7fb8d27fc640 1 -- 172.21.15.133:0/331843535 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8dc085280 con 0x7fb8e00b1f70 2024-04-25T10:05:40.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.026+0000 7fb8e510f640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8c4066f60 0x7fb8c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.026+0000 7fb8e510f640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8c4066f60 0x7fb8c4069420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40ae000 tx=0x7fb8d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.030+0000 7fb8d27fc640 1 -- 172.21.15.133:0/331843535 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8dc056100 con 0x7fb8e00b1f70 2024-04-25T10:05:40.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.182+0000 7fb8e7b74640 1 -- 172.21.15.133:0/331843535 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8e0068ed0 con 0x7fb8c4066f60 2024-04-25T10:05:40.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.194+0000 7fb8d27fc640 1 -- 172.21.15.133:0/331843535 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250102 (secure 0 0 0) 0x7fb8e0068ed0 con 0x7fb8c4066f60 2024-04-25T10:05:40.200 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:05:40.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 -- 172.21.15.133:0/331843535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8c4066f60 msgr2=0x7fb8c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8c4066f60 0x7fb8c4069420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40ae000 tx=0x7fb8d4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 -- 172.21.15.133:0/331843535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e00b1f70 msgr2=0x7fb8e0146ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e00b1f70 0x7fb8e0146ce0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00c7f0 tx=0x7fb8dc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 -- 172.21.15.133:0/331843535 shutdown_connections 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8e015c9a0 0x7fb8e0146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8e00ce330 0x7fb8e014a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8c4066f60 0x7fb8c4069420 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 --2- 172.21.15.133:0/331843535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e00b1f70 0x7fb8e0146ce0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 -- 172.21.15.133:0/331843535 >> 172.21.15.133:0/331843535 conn(0x7fb8e0067000 msgr2=0x7fb8e015e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 -- 172.21.15.133:0/331843535 shutdown_connections 2024-04-25T10:05:40.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.198+0000 7fb8e7b74640 1 -- 172.21.15.133:0/331843535 wait complete. 2024-04-25T10:05:40.218 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-04-25T10:05:39.776113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":347252,"kb_used_data":6192,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754627468,"statfs":{"total":773094113280,"available":772738527232,"internally_reserved":0,"allocated":6340608,"data_stored":3471849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":349097521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.619070"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193794+0000","last_change":"2024-04-25T10:05:37.228346+0000","last_active":"2024-04-25T10:05:38.193794+0000","last_peered":"2024-04-25T10:05:38.193794+0000","last_clean":"2024-04-25T10:05:38.193794+0000","last_became_active":"2024-04-25T10:05:37.228062+0000","last_became_peered":"2024-04-25T10:05:37.228062+0000","last_unstale":"2024-04-25T10:05:38.193794+0000","last_undegraded":"2024-04-25T10:05:38.193794+0000","last_fullsized":"2024-04-25T10:05:38.193794+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.807237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551210+0000","last_change":"2024-04-25T10:05:37.244108+0000","last_active":"2024-04-25T10:05:38.551210+0000","last_peered":"2024-04-25T10:05:38.551210+0000","last_clean":"2024-04-25T10:05:38.551210+0000","last_became_active":"2024-04-25T10:05:37.243540+0000","last_became_peered":"2024-04-25T10:05:37.243540+0000","last_unstale":"2024-04-25T10:05:38.551210+0000","last_undegraded":"2024-04-25T10:05:38.551210+0000","last_fullsized":"2024-04-25T10:05:38.551210+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:57.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553820+0000","last_change":"2024-04-25T10:05:37.215389+0000","last_active":"2024-04-25T10:05:38.553820+0000","last_peered":"2024-04-25T10:05:38.553820+0000","last_clean":"2024-04-25T10:05:38.553820+0000","last_became_active":"2024-04-25T10:05:37.214066+0000","last_became_peered":"2024-04-25T10:05:37.214066+0000","last_unstale":"2024-04-25T10:05:38.553820+0000","last_undegraded":"2024-04-25T10:05:38.553820+0000","last_fullsized":"2024-04-25T10:05:38.553820+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:16.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550681+0000","last_change":"2024-04-25T10:05:37.243719+0000","last_active":"2024-04-25T10:05:38.550681+0000","last_peered":"2024-04-25T10:05:38.550681+0000","last_clean":"2024-04-25T10:05:38.550681+0000","last_became_active":"2024-04-25T10:05:37.242856+0000","last_became_peered":"2024-04-25T10:05:37.242856+0000","last_unstale":"2024-04-25T10:05:38.550681+0000","last_undegraded":"2024-04-25T10:05:38.550681+0000","last_fullsized":"2024-04-25T10:05:38.550681+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549470+0000","last_change":"2024-04-25T10:05:37.241776+0000","last_active":"2024-04-25T10:05:38.549470+0000","last_peered":"2024-04-25T10:05:38.549470+0000","last_clean":"2024-04-25T10:05:38.549470+0000","last_became_active":"2024-04-25T10:05:37.240967+0000","last_became_peered":"2024-04-25T10:05:37.240967+0000","last_unstale":"2024-04-25T10:05:38.549470+0000","last_undegraded":"2024-04-25T10:05:38.549470+0000","last_fullsized":"2024-04-25T10:05:38.549470+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.790321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193759+0000","last_change":"2024-04-25T10:05:37.228488+0000","last_active":"2024-04-25T10:05:38.193759+0000","last_peered":"2024-04-25T10:05:38.193759+0000","last_clean":"2024-04-25T10:05:38.193759+0000","last_became_active":"2024-04-25T10:05:37.228212+0000","last_became_peered":"2024-04-25T10:05:37.228212+0000","last_unstale":"2024-04-25T10:05:38.193759+0000","last_undegraded":"2024-04-25T10:05:38.193759+0000","last_fullsized":"2024-04-25T10:05:38.193759+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:40.538309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551627+0000","last_change":"2024-04-25T10:05:37.216471+0000","last_active":"2024-04-25T10:05:38.551627+0000","last_peered":"2024-04-25T10:05:38.551627+0000","last_clean":"2024-04-25T10:05:38.551627+0000","last_became_active":"2024-04-25T10:05:37.216024+0000","last_became_peered":"2024-04-25T10:05:37.216024+0000","last_unstale":"2024-04-25T10:05:38.551627+0000","last_undegraded":"2024-04-25T10:05:38.551627+0000","last_fullsized":"2024-04-25T10:05:38.551627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551113+0000","last_change":"2024-04-25T10:05:37.215547+0000","last_active":"2024-04-25T10:05:38.551113+0000","last_peered":"2024-04-25T10:05:38.551113+0000","last_clean":"2024-04-25T10:05:38.551113+0000","last_became_active":"2024-04-25T10:05:37.215035+0000","last_became_peered":"2024-04-25T10:05:37.215035+0000","last_unstale":"2024-04-25T10:05:38.551113+0000","last_undegraded":"2024-04-25T10:05:38.551113+0000","last_fullsized":"2024-04-25T10:05:38.551113+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:02.441654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551909+0000","last_change":"2024-04-25T10:05:37.219302+0000","last_active":"2024-04-25T10:05:38.551909+0000","last_peered":"2024-04-25T10:05:38.551909+0000","last_clean":"2024-04-25T10:05:38.551909+0000","last_became_active":"2024-04-25T10:05:37.218837+0000","last_became_peered":"2024-04-25T10:05:37.218837+0000","last_unstale":"2024-04-25T10:05:38.551909+0000","last_undegraded":"2024-04-25T10:05:38.551909+0000","last_fullsized":"2024-04-25T10:05:38.551909+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:22.308378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193584+0000","last_change":"2024-04-25T10:05:37.230769+0000","last_active":"2024-04-25T10:05:38.193584+0000","last_peered":"2024-04-25T10:05:38.193584+0000","last_clean":"2024-04-25T10:05:38.193584+0000","last_became_active":"2024-04-25T10:05:37.229183+0000","last_became_peered":"2024-04-25T10:05:37.229183+0000","last_unstale":"2024-04-25T10:05:38.193584+0000","last_undegraded":"2024-04-25T10:05:38.193584+0000","last_fullsized":"2024-04-25T10:05:38.193584+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:50.039535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.555020+0000","last_change":"2024-04-25T10:05:37.241984+0000","last_active":"2024-04-25T10:05:38.555020+0000","last_peered":"2024-04-25T10:05:38.555020+0000","last_clean":"2024-04-25T10:05:38.555020+0000","last_became_active":"2024-04-25T10:05:37.241700+0000","last_became_peered":"2024-04-25T10:05:37.241700+0000","last_unstale":"2024-04-25T10:05:38.555020+0000","last_undegraded":"2024-04-25T10:05:38.555020+0000","last_fullsized":"2024-04-25T10:05:38.555020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:02.887132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551320+0000","last_change":"2024-04-25T10:05:37.240777+0000","last_active":"2024-04-25T10:05:38.551320+0000","last_peered":"2024-04-25T10:05:38.551320+0000","last_clean":"2024-04-25T10:05:38.551320+0000","last_became_active":"2024-04-25T10:05:37.240486+0000","last_became_peered":"2024-04-25T10:05:37.240486+0000","last_unstale":"2024-04-25T10:05:38.551320+0000","last_undegraded":"2024-04-25T10:05:38.551320+0000","last_fullsized":"2024-04-25T10:05:38.551320+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:05.759734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194117+0000","last_change":"2024-04-25T10:05:37.228825+0000","last_active":"2024-04-25T10:05:38.194117+0000","last_peered":"2024-04-25T10:05:38.194117+0000","last_clean":"2024-04-25T10:05:38.194117+0000","last_became_active":"2024-04-25T10:05:37.228567+0000","last_became_peered":"2024-04-25T10:05:37.228567+0000","last_unstale":"2024-04-25T10:05:38.194117+0000","last_undegraded":"2024-04-25T10:05:38.194117+0000","last_fullsized":"2024-04-25T10:05:38.194117+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:42.164656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552001+0000","last_change":"2024-04-25T10:05:37.216195+0000","last_active":"2024-04-25T10:05:38.552001+0000","last_peered":"2024-04-25T10:05:38.552001+0000","last_clean":"2024-04-25T10:05:38.552001+0000","last_became_active":"2024-04-25T10:05:37.215910+0000","last_became_peered":"2024-04-25T10:05:37.215910+0000","last_unstale":"2024-04-25T10:05:38.552001+0000","last_undegraded":"2024-04-25T10:05:38.552001+0000","last_fullsized":"2024-04-25T10:05:38.552001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549428+0000","last_change":"2024-04-25T10:05:37.244630+0000","last_active":"2024-04-25T10:05:38.549428+0000","last_peered":"2024-04-25T10:05:38.549428+0000","last_clean":"2024-04-25T10:05:38.549428+0000","last_became_active":"2024-04-25T10:05:37.244120+0000","last_became_peered":"2024-04-25T10:05:37.244120+0000","last_unstale":"2024-04-25T10:05:38.549428+0000","last_undegraded":"2024-04-25T10:05:38.549428+0000","last_fullsized":"2024-04-25T10:05:38.549428+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193338+0000","last_change":"2024-04-25T10:05:37.229142+0000","last_active":"2024-04-25T10:05:38.193338+0000","last_peered":"2024-04-25T10:05:38.193338+0000","last_clean":"2024-04-25T10:05:38.193338+0000","last_became_active":"2024-04-25T10:05:37.228852+0000","last_became_peered":"2024-04-25T10:05:37.228852+0000","last_unstale":"2024-04-25T10:05:38.193338+0000","last_undegraded":"2024-04-25T10:05:38.193338+0000","last_fullsized":"2024-04-25T10:05:38.193338+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:56.021983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552832+0000","last_change":"2024-04-25T10:05:37.219710+0000","last_active":"2024-04-25T10:05:38.552832+0000","last_peered":"2024-04-25T10:05:38.552832+0000","last_clean":"2024-04-25T10:05:38.552832+0000","last_became_active":"2024-04-25T10:05:37.218987+0000","last_became_peered":"2024-04-25T10:05:37.218987+0000","last_unstale":"2024-04-25T10:05:38.552832+0000","last_undegraded":"2024-04-25T10:05:38.552832+0000","last_fullsized":"2024-04-25T10:05:38.552832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551559+0000","last_change":"2024-04-25T10:05:37.244343+0000","last_active":"2024-04-25T10:05:38.551559+0000","last_peered":"2024-04-25T10:05:38.551559+0000","last_clean":"2024-04-25T10:05:38.551559+0000","last_became_active":"2024-04-25T10:05:37.243655+0000","last_became_peered":"2024-04-25T10:05:37.243655+0000","last_unstale":"2024-04-25T10:05:38.551559+0000","last_undegraded":"2024-04-25T10:05:38.551559+0000","last_fullsized":"2024-04-25T10:05:38.551559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554321+0000","last_change":"2024-04-25T10:05:37.222335+0000","last_active":"2024-04-25T10:05:38.554321+0000","last_peered":"2024-04-25T10:05:38.554321+0000","last_clean":"2024-04-25T10:05:38.554321+0000","last_became_active":"2024-04-25T10:05:37.220912+0000","last_became_peered":"2024-04-25T10:05:37.220912+0000","last_unstale":"2024-04-25T10:05:38.554321+0000","last_undegraded":"2024-04-25T10:05:38.554321+0000","last_fullsized":"2024-04-25T10:05:38.554321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:02.441654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550628+0000","last_change":"2024-04-25T10:05:37.212566+0000","last_active":"2024-04-25T10:05:38.550628+0000","last_peered":"2024-04-25T10:05:38.550628+0000","last_clean":"2024-04-25T10:05:38.550628+0000","last_became_active":"2024-04-25T10:05:37.211952+0000","last_became_peered":"2024-04-25T10:05:37.211952+0000","last_unstale":"2024-04-25T10:05:38.550628+0000","last_undegraded":"2024-04-25T10:05:38.550628+0000","last_fullsized":"2024-04-25T10:05:38.550628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:00.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554521+0000","last_change":"2024-04-25T10:05:37.238454+0000","last_active":"2024-04-25T10:05:38.554521+0000","last_peered":"2024-04-25T10:05:38.554521+0000","last_clean":"2024-04-25T10:05:38.554521+0000","last_became_active":"2024-04-25T10:05:37.238147+0000","last_became_peered":"2024-04-25T10:05:37.238147+0000","last_unstale":"2024-04-25T10:05:38.554521+0000","last_undegraded":"2024-04-25T10:05:38.554521+0000","last_fullsized":"2024-04-25T10:05:38.554521+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:10.077596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550991+0000","last_change":"2024-04-25T10:05:37.216233+0000","last_active":"2024-04-25T10:05:38.550991+0000","last_peered":"2024-04-25T10:05:38.550991+0000","last_clean":"2024-04-25T10:05:38.550991+0000","last_became_active":"2024-04-25T10:05:37.214054+0000","last_became_peered":"2024-04-25T10:05:37.214054+0000","last_unstale":"2024-04-25T10:05:38.550991+0000","last_undegraded":"2024-04-25T10:05:38.550991+0000","last_fullsized":"2024-04-25T10:05:38.550991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:30.598917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553345+0000","last_change":"2024-04-25T10:05:37.219854+0000","last_active":"2024-04-25T10:05:38.553345+0000","last_peered":"2024-04-25T10:05:38.553345+0000","last_clean":"2024-04-25T10:05:38.553345+0000","last_became_active":"2024-04-25T10:05:37.219447+0000","last_became_peered":"2024-04-25T10:05:37.219447+0000","last_unstale":"2024-04-25T10:05:38.553345+0000","last_undegraded":"2024-04-25T10:05:38.553345+0000","last_fullsized":"2024-04-25T10:05:38.553345+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:00.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551069+0000","last_change":"2024-04-25T10:05:37.203182+0000","last_active":"2024-04-25T10:05:38.551069+0000","last_peered":"2024-04-25T10:05:38.551069+0000","last_clean":"2024-04-25T10:05:38.551069+0000","last_became_active":"2024-04-25T10:05:37.202893+0000","last_became_peered":"2024-04-25T10:05:37.202893+0000","last_unstale":"2024-04-25T10:05:38.551069+0000","last_undegraded":"2024-04-25T10:05:38.551069+0000","last_fullsized":"2024-04-25T10:05:38.551069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:12.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194246+0000","last_change":"2024-04-25T10:05:37.205817+0000","last_active":"2024-04-25T10:05:38.194246+0000","last_peered":"2024-04-25T10:05:38.194246+0000","last_clean":"2024-04-25T10:05:38.194246+0000","last_became_active":"2024-04-25T10:05:37.205572+0000","last_became_peered":"2024-04-25T10:05:37.205572+0000","last_unstale":"2024-04-25T10:05:38.194246+0000","last_undegraded":"2024-04-25T10:05:38.194246+0000","last_fullsized":"2024-04-25T10:05:38.194246+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:36.414900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553104+0000","last_change":"2024-04-25T10:05:37.240878+0000","last_active":"2024-04-25T10:05:38.553104+0000","last_peered":"2024-04-25T10:05:38.553104+0000","last_clean":"2024-04-25T10:05:38.553104+0000","last_became_active":"2024-04-25T10:05:37.240620+0000","last_became_peered":"2024-04-25T10:05:37.240620+0000","last_unstale":"2024-04-25T10:05:38.553104+0000","last_undegraded":"2024-04-25T10:05:38.553104+0000","last_fullsized":"2024-04-25T10:05:38.553104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550361+0000","last_change":"2024-04-25T10:05:37.246366+0000","last_active":"2024-04-25T10:05:38.550361+0000","last_peered":"2024-04-25T10:05:38.550361+0000","last_clean":"2024-04-25T10:05:38.550361+0000","last_became_active":"2024-04-25T10:05:37.239714+0000","last_became_peered":"2024-04-25T10:05:37.239714+0000","last_unstale":"2024-04-25T10:05:38.550361+0000","last_undegraded":"2024-04-25T10:05:38.550361+0000","last_fullsized":"2024-04-25T10:05:38.550361+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552009+0000","last_change":"2024-04-25T10:05:37.216294+0000","last_active":"2024-04-25T10:05:38.552009+0000","last_peered":"2024-04-25T10:05:38.552009+0000","last_clean":"2024-04-25T10:05:38.552009+0000","last_became_active":"2024-04-25T10:05:37.215738+0000","last_became_peered":"2024-04-25T10:05:37.215738+0000","last_unstale":"2024-04-25T10:05:38.552009+0000","last_undegraded":"2024-04-25T10:05:38.552009+0000","last_fullsized":"2024-04-25T10:05:38.552009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191560+0000","last_change":"2024-04-25T10:05:37.221885+0000","last_active":"2024-04-25T10:05:38.191560+0000","last_peered":"2024-04-25T10:05:38.191560+0000","last_clean":"2024-04-25T10:05:38.191560+0000","last_became_active":"2024-04-25T10:05:37.219364+0000","last_became_peered":"2024-04-25T10:05:37.219364+0000","last_unstale":"2024-04-25T10:05:38.191560+0000","last_undegraded":"2024-04-25T10:05:38.191560+0000","last_fullsized":"2024-04-25T10:05:38.191560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549409+0000","last_change":"2024-04-25T10:05:37.229695+0000","last_active":"2024-04-25T10:05:38.549409+0000","last_peered":"2024-04-25T10:05:38.549409+0000","last_clean":"2024-04-25T10:05:38.549409+0000","last_became_active":"2024-04-25T10:05:37.228961+0000","last_became_peered":"2024-04-25T10:05:37.228961+0000","last_unstale":"2024-04-25T10:05:38.549409+0000","last_undegraded":"2024-04-25T10:05:38.549409+0000","last_fullsized":"2024-04-25T10:05:38.549409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191931+0000","last_change":"2024-04-25T10:05:37.230425+0000","last_active":"2024-04-25T10:05:38.191931+0000","last_peered":"2024-04-25T10:05:38.191931+0000","last_clean":"2024-04-25T10:05:38.191931+0000","last_became_active":"2024-04-25T10:05:37.229462+0000","last_became_peered":"2024-04-25T10:05:37.229462+0000","last_unstale":"2024-04-25T10:05:38.191931+0000","last_undegraded":"2024-04-25T10:05:38.191931+0000","last_fullsized":"2024-04-25T10:05:38.191931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194406+0000","last_change":"2024-04-25T10:05:36.192412+0000","last_active":"2024-04-25T10:05:38.194406+0000","last_peered":"2024-04-25T10:05:38.194406+0000","last_clean":"2024-04-25T10:05:38.194406+0000","last_became_active":"2024-04-25T10:05:36.191754+0000","last_became_peered":"2024-04-25T10:05:36.191754+0000","last_unstale":"2024-04-25T10:05:38.194406+0000","last_undegraded":"2024-04-25T10:05:38.194406+0000","last_fullsized":"2024-04-25T10:05:38.194406+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:33.604952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194384+0000","last_change":"2024-04-25T10:05:36.197170+0000","last_active":"2024-04-25T10:05:38.194384+0000","last_peered":"2024-04-25T10:05:38.194384+0000","last_clean":"2024-04-25T10:05:38.194384+0000","last_became_active":"2024-04-25T10:05:36.196555+0000","last_became_peered":"2024-04-25T10:05:36.196555+0000","last_unstale":"2024-04-25T10:05:38.194384+0000","last_undegraded":"2024-04-25T10:05:38.194384+0000","last_fullsized":"2024-04-25T10:05:38.194384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:43.120025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550712+0000","last_change":"2024-04-25T10:05:37.206507+0000","last_active":"2024-04-25T10:05:38.550712+0000","last_peered":"2024-04-25T10:05:38.550712+0000","last_clean":"2024-04-25T10:05:38.550712+0000","last_became_active":"2024-04-25T10:05:37.204792+0000","last_became_peered":"2024-04-25T10:05:37.204792+0000","last_unstale":"2024-04-25T10:05:38.550712+0000","last_undegraded":"2024-04-25T10:05:38.550712+0000","last_fullsized":"2024-04-25T10:05:38.550712+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:16.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192596+0000","last_change":"2024-04-25T10:05:36.201693+0000","last_active":"2024-04-25T10:05:38.192596+0000","last_peered":"2024-04-25T10:05:38.192596+0000","last_clean":"2024-04-25T10:05:38.192596+0000","last_became_active":"2024-04-25T10:05:36.200944+0000","last_became_peered":"2024-04-25T10:05:36.200944+0000","last_unstale":"2024-04-25T10:05:38.192596+0000","last_undegraded":"2024-04-25T10:05:38.192596+0000","last_fullsized":"2024-04-25T10:05:38.192596+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550050+0000","last_change":"2024-04-25T10:05:37.222859+0000","last_active":"2024-04-25T10:05:38.550050+0000","last_peered":"2024-04-25T10:05:38.550050+0000","last_clean":"2024-04-25T10:05:38.550050+0000","last_became_active":"2024-04-25T10:05:37.222162+0000","last_became_peered":"2024-04-25T10:05:37.222162+0000","last_unstale":"2024-04-25T10:05:38.550050+0000","last_undegraded":"2024-04-25T10:05:38.550050+0000","last_fullsized":"2024-04-25T10:05:38.550050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:44.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553382+0000","last_change":"2024-04-25T10:05:36.193782+0000","last_active":"2024-04-25T10:05:38.553382+0000","last_peered":"2024-04-25T10:05:38.553382+0000","last_clean":"2024-04-25T10:05:38.553382+0000","last_became_active":"2024-04-25T10:05:36.190455+0000","last_became_peered":"2024-04-25T10:05:36.190455+0000","last_unstale":"2024-04-25T10:05:38.553382+0000","last_undegraded":"2024-04-25T10:05:38.553382+0000","last_fullsized":"2024-04-25T10:05:38.553382+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:32.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550845+0000","last_change":"2024-04-25T10:05:37.246868+0000","last_active":"2024-04-25T10:05:38.550845+0000","last_peered":"2024-04-25T10:05:38.550845+0000","last_clean":"2024-04-25T10:05:38.550845+0000","last_became_active":"2024-04-25T10:05:37.246095+0000","last_became_peered":"2024-04-25T10:05:37.246095+0000","last_unstale":"2024-04-25T10:05:38.550845+0000","last_undegraded":"2024-04-25T10:05:38.550845+0000","last_fullsized":"2024-04-25T10:05:38.550845+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548980+0000","last_change":"2024-04-25T10:05:36.197860+0000","last_active":"2024-04-25T10:05:38.548980+0000","last_peered":"2024-04-25T10:05:38.548980+0000","last_clean":"2024-04-25T10:05:38.548980+0000","last_became_active":"2024-04-25T10:05:36.197137+0000","last_became_peered":"2024-04-25T10:05:36.197137+0000","last_unstale":"2024-04-25T10:05:38.548980+0000","last_undegraded":"2024-04-25T10:05:38.548980+0000","last_fullsized":"2024-04-25T10:05:38.548980+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:30.726291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551788+0000","last_change":"2024-04-25T10:05:37.206314+0000","last_active":"2024-04-25T10:05:38.551788+0000","last_peered":"2024-04-25T10:05:38.551788+0000","last_clean":"2024-04-25T10:05:38.551788+0000","last_became_active":"2024-04-25T10:05:37.205748+0000","last_became_peered":"2024-04-25T10:05:37.205748+0000","last_unstale":"2024-04-25T10:05:38.551788+0000","last_undegraded":"2024-04-25T10:05:38.551788+0000","last_fullsized":"2024-04-25T10:05:38.551788+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:52.894883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550829+0000","last_change":"2024-04-25T10:05:37.240137+0000","last_active":"2024-04-25T10:05:38.550829+0000","last_peered":"2024-04-25T10:05:38.550829+0000","last_clean":"2024-04-25T10:05:38.550829+0000","last_became_active":"2024-04-25T10:05:37.239387+0000","last_became_peered":"2024-04-25T10:05:37.239387+0000","last_unstale":"2024-04-25T10:05:38.550829+0000","last_undegraded":"2024-04-25T10:05:38.550829+0000","last_fullsized":"2024-04-25T10:05:38.550829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551084+0000","last_change":"2024-04-25T10:05:36.197600+0000","last_active":"2024-04-25T10:05:38.551084+0000","last_peered":"2024-04-25T10:05:38.551084+0000","last_clean":"2024-04-25T10:05:38.551084+0000","last_became_active":"2024-04-25T10:05:36.197265+0000","last_became_peered":"2024-04-25T10:05:36.197265+0000","last_unstale":"2024-04-25T10:05:38.551084+0000","last_undegraded":"2024-04-25T10:05:38.551084+0000","last_fullsized":"2024-04-25T10:05:38.551084+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:24.137684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548851+0000","last_change":"2024-04-25T10:05:37.224800+0000","last_active":"2024-04-25T10:05:38.548851+0000","last_peered":"2024-04-25T10:05:38.548851+0000","last_clean":"2024-04-25T10:05:38.548851+0000","last_became_active":"2024-04-25T10:05:37.224068+0000","last_became_peered":"2024-04-25T10:05:37.224068+0000","last_unstale":"2024-04-25T10:05:38.548851+0000","last_undegraded":"2024-04-25T10:05:38.548851+0000","last_fullsized":"2024-04-25T10:05:38.548851+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:25.148179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551076+0000","last_change":"2024-04-25T10:05:36.195068+0000","last_active":"2024-04-25T10:05:38.551076+0000","last_peered":"2024-04-25T10:05:38.551076+0000","last_clean":"2024-04-25T10:05:38.551076+0000","last_became_active":"2024-04-25T10:05:36.194281+0000","last_became_peered":"2024-04-25T10:05:36.194281+0000","last_unstale":"2024-04-25T10:05:38.551076+0000","last_undegraded":"2024-04-25T10:05:38.551076+0000","last_fullsized":"2024-04-25T10:05:38.551076+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:41.154160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191829+0000","last_change":"2024-04-25T10:05:37.223331+0000","last_active":"2024-04-25T10:05:38.191829+0000","last_peered":"2024-04-25T10:05:38.191829+0000","last_clean":"2024-04-25T10:05:38.191829+0000","last_became_active":"2024-04-25T10:05:37.222613+0000","last_became_peered":"2024-04-25T10:05:37.222613+0000","last_unstale":"2024-04-25T10:05:38.191829+0000","last_undegraded":"2024-04-25T10:05:38.191829+0000","last_fullsized":"2024-04-25T10:05:38.191829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549616+0000","last_change":"2024-04-25T10:05:36.200167+0000","last_active":"2024-04-25T10:05:38.549616+0000","last_peered":"2024-04-25T10:05:38.549616+0000","last_clean":"2024-04-25T10:05:38.549616+0000","last_became_active":"2024-04-25T10:05:36.199467+0000","last_became_peered":"2024-04-25T10:05:36.199467+0000","last_unstale":"2024-04-25T10:05:38.549616+0000","last_undegraded":"2024-04-25T10:05:38.549616+0000","last_fullsized":"2024-04-25T10:05:38.549616+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:47.830431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552373+0000","last_change":"2024-04-25T10:05:37.241002+0000","last_active":"2024-04-25T10:05:38.552373+0000","last_peered":"2024-04-25T10:05:38.552373+0000","last_clean":"2024-04-25T10:05:38.552373+0000","last_became_active":"2024-04-25T10:05:37.240740+0000","last_became_peered":"2024-04-25T10:05:37.240740+0000","last_unstale":"2024-04-25T10:05:38.552373+0000","last_undegraded":"2024-04-25T10:05:38.552373+0000","last_fullsized":"2024-04-25T10:05:38.552373+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551231+0000","last_change":"2024-04-25T10:05:36.195424+0000","last_active":"2024-04-25T10:05:38.551231+0000","last_peered":"2024-04-25T10:05:38.551231+0000","last_clean":"2024-04-25T10:05:38.551231+0000","last_became_active":"2024-04-25T10:05:36.193306+0000","last_became_peered":"2024-04-25T10:05:36.193306+0000","last_unstale":"2024-04-25T10:05:38.551231+0000","last_undegraded":"2024-04-25T10:05:38.551231+0000","last_fullsized":"2024-04-25T10:05:38.551231+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549522+0000","last_change":"2024-04-25T10:05:37.220528+0000","last_active":"2024-04-25T10:05:38.549522+0000","last_peered":"2024-04-25T10:05:38.549522+0000","last_clean":"2024-04-25T10:05:38.549522+0000","last_became_active":"2024-04-25T10:05:37.219586+0000","last_became_peered":"2024-04-25T10:05:37.219586+0000","last_unstale":"2024-04-25T10:05:38.549522+0000","last_undegraded":"2024-04-25T10:05:38.549522+0000","last_fullsized":"2024-04-25T10:05:38.549522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:55.011916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551578+0000","last_change":"2024-04-25T10:05:36.199745+0000","last_active":"2024-04-25T10:05:38.551578+0000","last_peered":"2024-04-25T10:05:38.551578+0000","last_clean":"2024-04-25T10:05:38.551578+0000","last_became_active":"2024-04-25T10:05:36.191871+0000","last_became_peered":"2024-04-25T10:05:36.191871+0000","last_unstale":"2024-04-25T10:05:38.551578+0000","last_undegraded":"2024-04-25T10:05:38.551578+0000","last_fullsized":"2024-04-25T10:05:38.551578+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191388+0000","last_change":"2024-04-25T10:05:37.230868+0000","last_active":"2024-04-25T10:05:38.191388+0000","last_peered":"2024-04-25T10:05:38.191388+0000","last_clean":"2024-04-25T10:05:38.191388+0000","last_became_active":"2024-04-25T10:05:37.229549+0000","last_became_peered":"2024-04-25T10:05:37.229549+0000","last_unstale":"2024-04-25T10:05:38.191388+0000","last_undegraded":"2024-04-25T10:05:38.191388+0000","last_fullsized":"2024-04-25T10:05:38.191388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:04.305032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553620+0000","last_change":"2024-04-25T10:05:36.201220+0000","last_active":"2024-04-25T10:05:38.553620+0000","last_peered":"2024-04-25T10:05:38.553620+0000","last_clean":"2024-04-25T10:05:38.553620+0000","last_became_active":"2024-04-25T10:05:36.200820+0000","last_became_peered":"2024-04-25T10:05:36.200820+0000","last_unstale":"2024-04-25T10:05:38.553620+0000","last_undegraded":"2024-04-25T10:05:38.553620+0000","last_fullsized":"2024-04-25T10:05:38.553620+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:54.001421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554271+0000","last_change":"2024-04-25T10:05:37.215939+0000","last_active":"2024-04-25T10:05:38.554271+0000","last_peered":"2024-04-25T10:05:38.554271+0000","last_clean":"2024-04-25T10:05:38.554271+0000","last_became_active":"2024-04-25T10:05:37.215693+0000","last_became_peered":"2024-04-25T10:05:37.215693+0000","last_unstale":"2024-04-25T10:05:38.554271+0000","last_undegraded":"2024-04-25T10:05:38.554271+0000","last_fullsized":"2024-04-25T10:05:38.554271+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:12.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552870+0000","last_change":"2024-04-25T10:05:37.238447+0000","last_active":"2024-04-25T10:05:38.552870+0000","last_peered":"2024-04-25T10:05:38.552870+0000","last_clean":"2024-04-25T10:05:38.552870+0000","last_became_active":"2024-04-25T10:05:37.238106+0000","last_became_peered":"2024-04-25T10:05:37.238106+0000","last_unstale":"2024-04-25T10:05:38.552870+0000","last_undegraded":"2024-04-25T10:05:38.552870+0000","last_fullsized":"2024-04-25T10:05:38.552870+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:06.779179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549834+0000","last_change":"2024-04-25T10:05:36.198280+0000","last_active":"2024-04-25T10:05:38.549834+0000","last_peered":"2024-04-25T10:05:38.549834+0000","last_clean":"2024-04-25T10:05:38.549834+0000","last_became_active":"2024-04-25T10:05:36.197484+0000","last_became_peered":"2024-04-25T10:05:36.197484+0000","last_unstale":"2024-04-25T10:05:38.549834+0000","last_undegraded":"2024-04-25T10:05:38.549834+0000","last_fullsized":"2024-04-25T10:05:38.549834+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.796742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552446+0000","last_change":"2024-04-25T10:05:37.219004+0000","last_active":"2024-04-25T10:05:38.552446+0000","last_peered":"2024-04-25T10:05:38.552446+0000","last_clean":"2024-04-25T10:05:38.552446+0000","last_became_active":"2024-04-25T10:05:37.216939+0000","last_became_peered":"2024-04-25T10:05:37.216939+0000","last_unstale":"2024-04-25T10:05:38.552446+0000","last_undegraded":"2024-04-25T10:05:38.552446+0000","last_fullsized":"2024-04-25T10:05:38.552446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551470+0000","last_change":"2024-04-25T10:05:36.198305+0000","last_active":"2024-04-25T10:05:38.551470+0000","last_peered":"2024-04-25T10:05:38.551470+0000","last_clean":"2024-04-25T10:05:38.551470+0000","last_became_active":"2024-04-25T10:05:36.191908+0000","last_became_peered":"2024-04-25T10:05:36.191908+0000","last_unstale":"2024-04-25T10:05:38.551470+0000","last_undegraded":"2024-04-25T10:05:38.551470+0000","last_fullsized":"2024-04-25T10:05:38.551470+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:52.819131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553781+0000","last_change":"2024-04-25T10:05:37.224631+0000","last_active":"2024-04-25T10:05:38.553781+0000","last_peered":"2024-04-25T10:05:38.553781+0000","last_clean":"2024-04-25T10:05:38.553781+0000","last_became_active":"2024-04-25T10:05:37.224014+0000","last_became_peered":"2024-04-25T10:05:37.224014+0000","last_unstale":"2024-04-25T10:05:38.553781+0000","last_undegraded":"2024-04-25T10:05:38.553781+0000","last_fullsized":"2024-04-25T10:05:38.553781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:08.906935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553276+0000","last_change":"2024-04-25T10:05:36.203863+0000","last_active":"2024-04-25T10:05:38.553276+0000","last_peered":"2024-04-25T10:05:38.553276+0000","last_clean":"2024-04-25T10:05:38.553276+0000","last_became_active":"2024-04-25T10:05:36.203473+0000","last_became_peered":"2024-04-25T10:05:36.203473+0000","last_unstale":"2024-04-25T10:05:38.553276+0000","last_undegraded":"2024-04-25T10:05:38.553276+0000","last_fullsized":"2024-04-25T10:05:38.553276+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:24.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193575+0000","last_change":"2024-04-25T10:05:37.228514+0000","last_active":"2024-04-25T10:05:38.193575+0000","last_peered":"2024-04-25T10:05:38.193575+0000","last_clean":"2024-04-25T10:05:38.193575+0000","last_became_active":"2024-04-25T10:05:37.213429+0000","last_became_peered":"2024-04-25T10:05:37.213429+0000","last_unstale":"2024-04-25T10:05:38.193575+0000","last_undegraded":"2024-04-25T10:05:38.193575+0000","last_fullsized":"2024-04-25T10:05:38.193575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:07.257110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.555115+0000","last_change":"2024-04-25T10:05:36.192160+0000","last_active":"2024-04-25T10:05:38.555115+0000","last_peered":"2024-04-25T10:05:38.555115+0000","last_clean":"2024-04-25T10:05:38.555115+0000","last_became_active":"2024-04-25T10:05:36.191475+0000","last_became_peered":"2024-04-25T10:05:36.191475+0000","last_unstale":"2024-04-25T10:05:38.555115+0000","last_undegraded":"2024-04-25T10:05:38.555115+0000","last_fullsized":"2024-04-25T10:05:38.555115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:06.047245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548936+0000","last_change":"2024-04-25T10:05:37.218759+0000","last_active":"2024-04-25T10:05:38.548936+0000","last_peered":"2024-04-25T10:05:38.548936+0000","last_clean":"2024-04-25T10:05:38.548936+0000","last_became_active":"2024-04-25T10:05:37.217992+0000","last_became_peered":"2024-04-25T10:05:37.217992+0000","last_unstale":"2024-04-25T10:05:38.548936+0000","last_undegraded":"2024-04-25T10:05:38.548936+0000","last_fullsized":"2024-04-25T10:05:38.548936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:59.463557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194500+0000","last_change":"2024-04-25T10:05:37.227960+0000","last_active":"2024-04-25T10:05:38.194500+0000","last_peered":"2024-04-25T10:05:38.194500+0000","last_clean":"2024-04-25T10:05:38.194500+0000","last_became_active":"2024-04-25T10:05:37.213414+0000","last_became_peered":"2024-04-25T10:05:37.213414+0000","last_unstale":"2024-04-25T10:05:38.194500+0000","last_undegraded":"2024-04-25T10:05:38.194500+0000","last_fullsized":"2024-04-25T10:05:38.194500+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:01.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191893+0000","last_change":"2024-04-25T10:05:37.224756+0000","last_active":"2024-04-25T10:05:38.191893+0000","last_peered":"2024-04-25T10:05:38.191893+0000","last_clean":"2024-04-25T10:05:38.191893+0000","last_became_active":"2024-04-25T10:05:37.223895+0000","last_became_peered":"2024-04-25T10:05:37.223895+0000","last_unstale":"2024-04-25T10:05:38.191893+0000","last_undegraded":"2024-04-25T10:05:38.191893+0000","last_fullsized":"2024-04-25T10:05:38.191893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194604+0000","last_change":"2024-04-25T10:05:36.186378+0000","last_active":"2024-04-25T10:05:38.194604+0000","last_peered":"2024-04-25T10:05:38.194604+0000","last_clean":"2024-04-25T10:05:38.194604+0000","last_became_active":"2024-04-25T10:05:36.185733+0000","last_became_peered":"2024-04-25T10:05:36.185733+0000","last_unstale":"2024-04-25T10:05:38.194604+0000","last_undegraded":"2024-04-25T10:05:38.194604+0000","last_fullsized":"2024-04-25T10:05:38.194604+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:43.722766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551744+0000","last_change":"2024-04-25T10:05:36.199741+0000","last_active":"2024-04-25T10:05:38.551744+0000","last_peered":"2024-04-25T10:05:38.551744+0000","last_clean":"2024-04-25T10:05:38.551744+0000","last_became_active":"2024-04-25T10:05:36.199311+0000","last_became_peered":"2024-04-25T10:05:36.199311+0000","last_unstale":"2024-04-25T10:05:38.551744+0000","last_undegraded":"2024-04-25T10:05:38.551744+0000","last_fullsized":"2024-04-25T10:05:38.551744+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551563+0000","last_change":"2024-04-25T10:05:37.240101+0000","last_active":"2024-04-25T10:05:38.551563+0000","last_peered":"2024-04-25T10:05:38.551563+0000","last_clean":"2024-04-25T10:05:38.551563+0000","last_became_active":"2024-04-25T10:05:37.239789+0000","last_became_peered":"2024-04-25T10:05:37.239789+0000","last_unstale":"2024-04-25T10:05:38.551563+0000","last_undegraded":"2024-04-25T10:05:38.551563+0000","last_fullsized":"2024-04-25T10:05:38.551563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:04.305032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552440+0000","last_change":"2024-04-25T10:05:36.194819+0000","last_active":"2024-04-25T10:05:38.552440+0000","last_peered":"2024-04-25T10:05:38.552440+0000","last_clean":"2024-04-25T10:05:38.552440+0000","last_became_active":"2024-04-25T10:05:36.194511+0000","last_became_peered":"2024-04-25T10:05:36.194511+0000","last_unstale":"2024-04-25T10:05:38.552440+0000","last_undegraded":"2024-04-25T10:05:38.552440+0000","last_fullsized":"2024-04-25T10:05:38.552440+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:43.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194192+0000","last_change":"2024-04-25T10:05:37.204849+0000","last_active":"2024-04-25T10:05:38.194192+0000","last_peered":"2024-04-25T10:05:38.194192+0000","last_clean":"2024-04-25T10:05:38.194192+0000","last_became_active":"2024-04-25T10:05:37.204310+0000","last_became_peered":"2024-04-25T10:05:37.204310+0000","last_unstale":"2024-04-25T10:05:38.194192+0000","last_undegraded":"2024-04-25T10:05:38.194192+0000","last_fullsized":"2024-04-25T10:05:38.194192+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:49.155350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554104+0000","last_change":"2024-04-25T10:05:36.189798+0000","last_active":"2024-04-25T10:05:38.554104+0000","last_peered":"2024-04-25T10:05:38.554104+0000","last_clean":"2024-04-25T10:05:38.554104+0000","last_became_active":"2024-04-25T10:05:36.189079+0000","last_became_peered":"2024-04-25T10:05:36.189079+0000","last_unstale":"2024-04-25T10:05:38.554104+0000","last_undegraded":"2024-04-25T10:05:38.554104+0000","last_fullsized":"2024-04-25T10:05:38.554104+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:06.607042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192449+0000","last_change":"2024-04-25T10:05:37.213816+0000","last_active":"2024-04-25T10:05:38.192449+0000","last_peered":"2024-04-25T10:05:38.192449+0000","last_clean":"2024-04-25T10:05:38.192449+0000","last_became_active":"2024-04-25T10:05:37.213082+0000","last_became_peered":"2024-04-25T10:05:37.213082+0000","last_unstale":"2024-04-25T10:05:38.192449+0000","last_undegraded":"2024-04-25T10:05:38.192449+0000","last_fullsized":"2024-04-25T10:05:38.192449+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193639+0000","last_change":"2024-04-25T10:05:37.199321+0000","last_active":"2024-04-25T10:05:38.193639+0000","last_peered":"2024-04-25T10:05:38.193639+0000","last_clean":"2024-04-25T10:05:38.193639+0000","last_became_active":"2024-04-25T10:05:37.199011+0000","last_became_peered":"2024-04-25T10:05:37.199011+0000","last_unstale":"2024-04-25T10:05:38.193639+0000","last_undegraded":"2024-04-25T10:05:38.193639+0000","last_fullsized":"2024-04-25T10:05:38.193639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.753836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551360+0000","last_change":"2024-04-25T10:05:36.198586+0000","last_active":"2024-04-25T10:05:38.551360+0000","last_peered":"2024-04-25T10:05:38.551360+0000","last_clean":"2024-04-25T10:05:38.551360+0000","last_became_active":"2024-04-25T10:05:36.197856+0000","last_became_peered":"2024-04-25T10:05:36.197856+0000","last_unstale":"2024-04-25T10:05:38.551360+0000","last_undegraded":"2024-04-25T10:05:38.551360+0000","last_fullsized":"2024-04-25T10:05:38.551360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550497+0000","last_change":"2024-04-25T10:05:37.225133+0000","last_active":"2024-04-25T10:05:38.550497+0000","last_peered":"2024-04-25T10:05:38.550497+0000","last_clean":"2024-04-25T10:05:38.550497+0000","last_became_active":"2024-04-25T10:05:37.224655+0000","last_became_peered":"2024-04-25T10:05:37.224655+0000","last_unstale":"2024-04-25T10:05:38.550497+0000","last_undegraded":"2024-04-25T10:05:38.550497+0000","last_fullsized":"2024-04-25T10:05:38.550497+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553244+0000","last_change":"2024-04-25T10:05:36.189704+0000","last_active":"2024-04-25T10:05:38.553244+0000","last_peered":"2024-04-25T10:05:38.553244+0000","last_clean":"2024-04-25T10:05:38.553244+0000","last_became_active":"2024-04-25T10:05:36.188969+0000","last_became_peered":"2024-04-25T10:05:36.188969+0000","last_unstale":"2024-04-25T10:05:38.553244+0000","last_undegraded":"2024-04-25T10:05:38.553244+0000","last_fullsized":"2024-04-25T10:05:38.553244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:58.453061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553005+0000","last_change":"2024-04-25T10:05:37.219793+0000","last_active":"2024-04-25T10:05:38.553005+0000","last_peered":"2024-04-25T10:05:38.553005+0000","last_clean":"2024-04-25T10:05:38.553005+0000","last_became_active":"2024-04-25T10:05:37.219521+0000","last_became_peered":"2024-04-25T10:05:37.219521+0000","last_unstale":"2024-04-25T10:05:38.553005+0000","last_undegraded":"2024-04-25T10:05:38.553005+0000","last_fullsized":"2024-04-25T10:05:38.553005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:57.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550594+0000","last_change":"2024-04-25T10:05:36.198953+0000","last_active":"2024-04-25T10:05:38.550594+0000","last_peered":"2024-04-25T10:05:38.550594+0000","last_clean":"2024-04-25T10:05:38.550594+0000","last_became_active":"2024-04-25T10:05:36.197976+0000","last_became_peered":"2024-04-25T10:05:36.197976+0000","last_unstale":"2024-04-25T10:05:38.550594+0000","last_undegraded":"2024-04-25T10:05:38.550594+0000","last_fullsized":"2024-04-25T10:05:38.550594+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:09.514786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191968+0000","last_change":"2024-04-25T10:05:37.217261+0000","last_active":"2024-04-25T10:05:38.191968+0000","last_peered":"2024-04-25T10:05:38.191968+0000","last_clean":"2024-04-25T10:05:38.191968+0000","last_became_active":"2024-04-25T10:05:37.216539+0000","last_became_peered":"2024-04-25T10:05:37.216539+0000","last_unstale":"2024-04-25T10:05:38.191968+0000","last_undegraded":"2024-04-25T10:05:38.191968+0000","last_fullsized":"2024-04-25T10:05:38.191968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551865+0000","last_change":"2024-04-25T10:05:36.191359+0000","last_active":"2024-04-25T10:05:38.551865+0000","last_peered":"2024-04-25T10:05:38.551865+0000","last_clean":"2024-04-25T10:05:38.551865+0000","last_became_active":"2024-04-25T10:05:36.190991+0000","last_became_peered":"2024-04-25T10:05:36.190991+0000","last_unstale":"2024-04-25T10:05:38.551865+0000","last_undegraded":"2024-04-25T10:05:38.551865+0000","last_fullsized":"2024-04-25T10:05:38.551865+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:00.276908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551375+0000","last_change":"2024-04-25T10:05:37.215467+0000","last_active":"2024-04-25T10:05:38.551375+0000","last_peered":"2024-04-25T10:05:38.551375+0000","last_clean":"2024-04-25T10:05:38.551375+0000","last_became_active":"2024-04-25T10:05:37.212611+0000","last_became_peered":"2024-04-25T10:05:37.212611+0000","last_unstale":"2024-04-25T10:05:38.551375+0000","last_undegraded":"2024-04-25T10:05:38.551375+0000","last_fullsized":"2024-04-25T10:05:38.551375+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:29.558721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548739+0000","last_change":"2024-04-25T10:05:36.191311+0000","last_active":"2024-04-25T10:05:38.548739+0000","last_peered":"2024-04-25T10:05:38.548739+0000","last_clean":"2024-04-25T10:05:38.548739+0000","last_became_active":"2024-04-25T10:05:36.190535+0000","last_became_peered":"2024-04-25T10:05:36.190535+0000","last_unstale":"2024-04-25T10:05:38.548739+0000","last_undegraded":"2024-04-25T10:05:38.548739+0000","last_fullsized":"2024-04-25T10:05:38.548739+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.118904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552275+0000","last_change":"2024-04-25T10:05:37.209994+0000","last_active":"2024-04-25T10:05:38.552275+0000","last_peered":"2024-04-25T10:05:38.552275+0000","last_clean":"2024-04-25T10:05:38.552275+0000","last_became_active":"2024-04-25T10:05:37.209686+0000","last_became_peered":"2024-04-25T10:05:37.209686+0000","last_unstale":"2024-04-25T10:05:38.552275+0000","last_undegraded":"2024-04-25T10:05:38.552275+0000","last_fullsized":"2024-04-25T10:05:38.552275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:36.861585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192702+0000","last_change":"2024-04-25T10:05:36.184629+0000","last_active":"2024-04-25T10:05:38.192702+0000","last_peered":"2024-04-25T10:05:38.192702+0000","last_clean":"2024-04-25T10:05:38.192702+0000","last_became_active":"2024-04-25T10:05:36.183892+0000","last_became_peered":"2024-04-25T10:05:36.183892+0000","last_unstale":"2024-04-25T10:05:38.192702+0000","last_undegraded":"2024-04-25T10:05:38.192702+0000","last_fullsized":"2024-04-25T10:05:38.192702+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:39.205026+0000","last_change":"2024-04-25T10:05:36.201261+0000","last_active":"2024-04-25T10:05:39.205026+0000","last_peered":"2024-04-25T10:05:39.205026+0000","last_clean":"2024-04-25T10:05:39.205026+0000","last_became_active":"2024-04-25T10:05:36.199919+0000","last_became_peered":"2024-04-25T10:05:36.199919+0000","last_unstale":"2024-04-25T10:05:39.205026+0000","last_undegraded":"2024-04-25T10:05:39.205026+0000","last_fullsized":"2024-04-25T10:05:39.205026+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:37.214569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549551+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:05:38.549551+0000","last_peered":"2024-04-25T10:05:38.549551+0000","last_clean":"2024-04-25T10:05:38.549551+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:05:38.549551+0000","last_undegraded":"2024-04-25T10:05:38.549551+0000","last_fullsized":"2024-04-25T10:05:38.549551+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":52,"acting":52,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46284,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325556,"statfs":{"total":96636764160,"available":96589369344,"internally_reserved":0,"allocated":1245184,"data_stored":879702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40096,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331744,"statfs":{"total":96636764160,"available":96595705856,"internally_reserved":0,"allocated":593920,"data_stored":262475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":643072,"data_stored":289210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45700,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326140,"statfs":{"total":96636764160,"available":96589967360,"internally_reserved":0,"allocated":647168,"data_stored":289792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44308,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327532,"statfs":{"total":96636764160,"available":96591392768,"internally_reserved":0,"allocated":659456,"data_stored":290844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":655360,"data_stored":290822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":651264,"data_stored":289370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50452,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321388,"statfs":{"total":96636764160,"available":96585101312,"internally_reserved":0,"allocated":1245184,"data_stored":879634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:40.223 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:40.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f411c005640 1 Processor -- start 2024-04-25T10:05:40.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f411c005640 1 -- start start 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f411c005640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f411415d9b0 0x7f41140cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f411c005640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f41140cee80 0x7f411406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f411c005640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41140ca9b0 con 0x7f411415dd90 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f411c005640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41140cbe30 con 0x7f41140cee80 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f411c005640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41140cac90 con 0x7f411415d9b0 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f41195a0640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f411415dd90 0x7f41140ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40106/0 (socket says 172.21.15.133:40106) 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f41195a0640 1 -- 172.21.15.133:0/3799045169 learned_addr learned my addr 172.21.15.133:0/3799045169 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f411a5a2640 1 --2- 172.21.15.133:0/3799045169 >> v2:172.21.15.162:3300/0 conn(0x7f41140cee80 0x7f411406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4119da1640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f411415d9b0 0x7f41140cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209105132 0 0) 0x7f41140ca9b0 con 0x7f411415dd90 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4100003530 con 0x7f411415dd90 2024-04-25T10:05:40.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1028597023 0 0) 0x7f4100003530 con 0x7f411415dd90 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f411415d9b0 msgr2=0x7f41140cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f411415d9b0 0x7f41140cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 >> v2:172.21.15.162:3300/0 conn(0x7f41140cee80 msgr2=0x7f411406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 --2- 172.21.15.133:0/3799045169 >> v2:172.21.15.162:3300/0 conn(0x7f41140cee80 0x7f411406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140caf70 con 0x7f411415dd90 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4104001680 con 0x7f411415dd90 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 >> v1:172.21.15.133:6789/0 conn(0x7f411415dd90 legacy=0x7f41140ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 0x7f4100003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4100004810 0x7f4100004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f411415d9b0 0x7f41000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41140cbe30 con 0x7f41000039c0 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41140cac90 con 0x7f4100004810 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410000ba00 con 0x7f411415d9b0 2024-04-25T10:05:40.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4119da1640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 0x7f4100003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f41195a0640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4100004810 0x7f4100004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4119da1640 1 -- 172.21.15.133:0/3799045169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f411415d9b0 msgr2=0x7f41000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4119da1640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f411415d9b0 0x7f41000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4119da1640 1 -- 172.21.15.133:0/3799045169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4100004810 msgr2=0x7f4100004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4119da1640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4100004810 0x7f4100004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f4119da1640 1 -- 172.21.15.133:0/3799045169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100003530 con 0x7f41000039c0 2024-04-25T10:05:40.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.378+0000 7f41195a0640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4100004810 0x7f4100004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:40.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f4119da1640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 0x7f4100003e00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f410c009ae0 tx=0x7f410c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cff323bf87ba5ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0b2070 con 0x7f41000039c0 2024-04-25T10:05:40.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c0a55c0 con 0x7f41000039c0 2024-04-25T10:05:40.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f4118d9f640 1 -- 172.21.15.133:0/3799045169 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0096b0 con 0x7f41000039c0 2024-04-25T10:05:40.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 -- 172.21.15.133:0/3799045169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 msgr2=0x7f4100003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 --2- 172.21.15.133:0/3799045169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 0x7f4100003e00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f410c009ae0 tx=0x7f410c01cab0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411a5a2640 1 -- 172.21.15.133:0/3799045169 reap_dead start 2024-04-25T10:05:40.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 -- 172.21.15.133:0/3799045169 shutdown_connections 2024-04-25T10:05:40.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 -- 172.21.15.133:0/3799045169 >> 172.21.15.133:0/3799045169 conn(0x7f4114067000 msgr2=0x7f411406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 -- 172.21.15.133:0/3799045169 shutdown_connections 2024-04-25T10:05:40.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 -- 172.21.15.133:0/3799045169 wait complete. 2024-04-25T10:05:40.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 Processor -- start 2024-04-25T10:05:40.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 -- start start 2024-04-25T10:05:40.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 0x7f411415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4114145e30 0x7f4114152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41141462b0 0x7f4114152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411406e7d0 con 0x7f41000039c0 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411406e650 con 0x7f4114145e30 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.382+0000 7f411c005640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411406e950 con 0x7f41141462b0 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f4119da1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 0x7f411415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f4119da1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 0x7f411415c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43564/0 (socket says 172.21.15.133:43564) 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f4119da1640 1 -- 172.21.15.133:0/1334982408 learned_addr learned my addr 172.21.15.133:0/1334982408 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f41195a0640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4114145e30 0x7f4114152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f411a5a2640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41141462b0 0x7f4114152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f4119da1640 1 -- 172.21.15.133:0/1334982408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41141462b0 msgr2=0x7f4114152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f4119da1640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41141462b0 0x7f4114152eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f4119da1640 1 -- 172.21.15.133:0/1334982408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4114145e30 msgr2=0x7f4114152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f4119da1640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4114145e30 0x7f4114152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f4119da1640 1 -- 172.21.15.133:0/1334982408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411415c7e0 con 0x7f41000039c0 2024-04-25T10:05:40.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f41195a0640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4114145e30 0x7f4114152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:40.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f4119da1640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 0x7f411415c2a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f410c0b0040 tx=0x7f4100005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f410a7fc640 1 -- 172.21.15.133:0/1334982408 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0bc030 con 0x7f41000039c0 2024-04-25T10:05:40.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f410a7fc640 1 -- 172.21.15.133:0/1334982408 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c00f000 con 0x7f41000039c0 2024-04-25T10:05:40.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f410a7fc640 1 -- 172.21.15.133:0/1334982408 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0b2040 con 0x7f41000039c0 2024-04-25T10:05:40.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f411c005640 1 -- 172.21.15.133:0/1334982408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41141518d0 con 0x7f41000039c0 2024-04-25T10:05:40.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f411c005640 1 -- 172.21.15.133:0/1334982408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4114151e90 con 0x7f41000039c0 2024-04-25T10:05:40.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f411c005640 1 -- 172.21.15.133:0/1334982408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4114068ef0 con 0x7f41000039c0 2024-04-25T10:05:40.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f410a7fc640 1 -- 172.21.15.133:0/1334982408 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f410c0bf070 con 0x7f41000039c0 2024-04-25T10:05:40.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f410a7fc640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40f8066f60 0x7f40f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f41195a0640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40f8066f60 0x7f40f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f410a7fc640 1 -- 172.21.15.133:0/1334982408 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f410c11ef40 con 0x7f41000039c0 2024-04-25T10:05:40.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.386+0000 7f41195a0640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40f8066f60 0x7f40f8069420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f41141fa0b0 tx=0x7f4104009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.390+0000 7f410a7fc640 1 -- 172.21.15.133:0/1334982408 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f410c0c7020 con 0x7f41000039c0 2024-04-25T10:05:40.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.546+0000 7f411c005640 1 -- 172.21.15.133:0/1334982408 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4114158ef0 con 0x7f40f8066f60 2024-04-25T10:05:40.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.562+0000 7f410a7fc640 1 -- 172.21.15.133:0/1334982408 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250102 (secure 0 0 0) 0x7f4114158ef0 con 0x7f40f8066f60 2024-04-25T10:05:40.569 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 -- 172.21.15.133:0/1334982408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40f8066f60 msgr2=0x7f40f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40f8066f60 0x7f40f8069420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f41141fa0b0 tx=0x7f4104009040 comp rx=0 tx=0).stop 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 -- 172.21.15.133:0/1334982408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 msgr2=0x7f411415c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 0x7f411415c2a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f410c0b0040 tx=0x7f4100005840 comp rx=0 tx=0).stop 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 -- 172.21.15.133:0/1334982408 shutdown_connections 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41141462b0 0x7f4114152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4114145e30 0x7f4114152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40f8066f60 0x7f40f8069420 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 --2- 172.21.15.133:0/1334982408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000039c0 0x7f411415c2a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 -- 172.21.15.133:0/1334982408 >> 172.21.15.133:0/1334982408 conn(0x7f4114067000 msgr2=0x7f41140ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 -- 172.21.15.133:0/1334982408 shutdown_connections 2024-04-25T10:05:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.566+0000 7f411c005640 1 -- 172.21.15.133:0/1334982408 wait complete. 2024-04-25T10:05:40.588 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-04-25T10:05:39.776113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":347252,"kb_used_data":6192,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754627468,"statfs":{"total":773094113280,"available":772738527232,"internally_reserved":0,"allocated":6340608,"data_stored":3471849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":349097521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.619070"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193794+0000","last_change":"2024-04-25T10:05:37.228346+0000","last_active":"2024-04-25T10:05:38.193794+0000","last_peered":"2024-04-25T10:05:38.193794+0000","last_clean":"2024-04-25T10:05:38.193794+0000","last_became_active":"2024-04-25T10:05:37.228062+0000","last_became_peered":"2024-04-25T10:05:37.228062+0000","last_unstale":"2024-04-25T10:05:38.193794+0000","last_undegraded":"2024-04-25T10:05:38.193794+0000","last_fullsized":"2024-04-25T10:05:38.193794+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.807237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551210+0000","last_change":"2024-04-25T10:05:37.244108+0000","last_active":"2024-04-25T10:05:38.551210+0000","last_peered":"2024-04-25T10:05:38.551210+0000","last_clean":"2024-04-25T10:05:38.551210+0000","last_became_active":"2024-04-25T10:05:37.243540+0000","last_became_peered":"2024-04-25T10:05:37.243540+0000","last_unstale":"2024-04-25T10:05:38.551210+0000","last_undegraded":"2024-04-25T10:05:38.551210+0000","last_fullsized":"2024-04-25T10:05:38.551210+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:57.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553820+0000","last_change":"2024-04-25T10:05:37.215389+0000","last_active":"2024-04-25T10:05:38.553820+0000","last_peered":"2024-04-25T10:05:38.553820+0000","last_clean":"2024-04-25T10:05:38.553820+0000","last_became_active":"2024-04-25T10:05:37.214066+0000","last_became_peered":"2024-04-25T10:05:37.214066+0000","last_unstale":"2024-04-25T10:05:38.553820+0000","last_undegraded":"2024-04-25T10:05:38.553820+0000","last_fullsized":"2024-04-25T10:05:38.553820+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:16.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550681+0000","last_change":"2024-04-25T10:05:37.243719+0000","last_active":"2024-04-25T10:05:38.550681+0000","last_peered":"2024-04-25T10:05:38.550681+0000","last_clean":"2024-04-25T10:05:38.550681+0000","last_became_active":"2024-04-25T10:05:37.242856+0000","last_became_peered":"2024-04-25T10:05:37.242856+0000","last_unstale":"2024-04-25T10:05:38.550681+0000","last_undegraded":"2024-04-25T10:05:38.550681+0000","last_fullsized":"2024-04-25T10:05:38.550681+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549470+0000","last_change":"2024-04-25T10:05:37.241776+0000","last_active":"2024-04-25T10:05:38.549470+0000","last_peered":"2024-04-25T10:05:38.549470+0000","last_clean":"2024-04-25T10:05:38.549470+0000","last_became_active":"2024-04-25T10:05:37.240967+0000","last_became_peered":"2024-04-25T10:05:37.240967+0000","last_unstale":"2024-04-25T10:05:38.549470+0000","last_undegraded":"2024-04-25T10:05:38.549470+0000","last_fullsized":"2024-04-25T10:05:38.549470+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.790321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193759+0000","last_change":"2024-04-25T10:05:37.228488+0000","last_active":"2024-04-25T10:05:38.193759+0000","last_peered":"2024-04-25T10:05:38.193759+0000","last_clean":"2024-04-25T10:05:38.193759+0000","last_became_active":"2024-04-25T10:05:37.228212+0000","last_became_peered":"2024-04-25T10:05:37.228212+0000","last_unstale":"2024-04-25T10:05:38.193759+0000","last_undegraded":"2024-04-25T10:05:38.193759+0000","last_fullsized":"2024-04-25T10:05:38.193759+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:40.538309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551627+0000","last_change":"2024-04-25T10:05:37.216471+0000","last_active":"2024-04-25T10:05:38.551627+0000","last_peered":"2024-04-25T10:05:38.551627+0000","last_clean":"2024-04-25T10:05:38.551627+0000","last_became_active":"2024-04-25T10:05:37.216024+0000","last_became_peered":"2024-04-25T10:05:37.216024+0000","last_unstale":"2024-04-25T10:05:38.551627+0000","last_undegraded":"2024-04-25T10:05:38.551627+0000","last_fullsized":"2024-04-25T10:05:38.551627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551113+0000","last_change":"2024-04-25T10:05:37.215547+0000","last_active":"2024-04-25T10:05:38.551113+0000","last_peered":"2024-04-25T10:05:38.551113+0000","last_clean":"2024-04-25T10:05:38.551113+0000","last_became_active":"2024-04-25T10:05:37.215035+0000","last_became_peered":"2024-04-25T10:05:37.215035+0000","last_unstale":"2024-04-25T10:05:38.551113+0000","last_undegraded":"2024-04-25T10:05:38.551113+0000","last_fullsized":"2024-04-25T10:05:38.551113+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:02.441654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551909+0000","last_change":"2024-04-25T10:05:37.219302+0000","last_active":"2024-04-25T10:05:38.551909+0000","last_peered":"2024-04-25T10:05:38.551909+0000","last_clean":"2024-04-25T10:05:38.551909+0000","last_became_active":"2024-04-25T10:05:37.218837+0000","last_became_peered":"2024-04-25T10:05:37.218837+0000","last_unstale":"2024-04-25T10:05:38.551909+0000","last_undegraded":"2024-04-25T10:05:38.551909+0000","last_fullsized":"2024-04-25T10:05:38.551909+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:22.308378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193584+0000","last_change":"2024-04-25T10:05:37.230769+0000","last_active":"2024-04-25T10:05:38.193584+0000","last_peered":"2024-04-25T10:05:38.193584+0000","last_clean":"2024-04-25T10:05:38.193584+0000","last_became_active":"2024-04-25T10:05:37.229183+0000","last_became_peered":"2024-04-25T10:05:37.229183+0000","last_unstale":"2024-04-25T10:05:38.193584+0000","last_undegraded":"2024-04-25T10:05:38.193584+0000","last_fullsized":"2024-04-25T10:05:38.193584+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:50.039535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.555020+0000","last_change":"2024-04-25T10:05:37.241984+0000","last_active":"2024-04-25T10:05:38.555020+0000","last_peered":"2024-04-25T10:05:38.555020+0000","last_clean":"2024-04-25T10:05:38.555020+0000","last_became_active":"2024-04-25T10:05:37.241700+0000","last_became_peered":"2024-04-25T10:05:37.241700+0000","last_unstale":"2024-04-25T10:05:38.555020+0000","last_undegraded":"2024-04-25T10:05:38.555020+0000","last_fullsized":"2024-04-25T10:05:38.555020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:02.887132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551320+0000","last_change":"2024-04-25T10:05:37.240777+0000","last_active":"2024-04-25T10:05:38.551320+0000","last_peered":"2024-04-25T10:05:38.551320+0000","last_clean":"2024-04-25T10:05:38.551320+0000","last_became_active":"2024-04-25T10:05:37.240486+0000","last_became_peered":"2024-04-25T10:05:37.240486+0000","last_unstale":"2024-04-25T10:05:38.551320+0000","last_undegraded":"2024-04-25T10:05:38.551320+0000","last_fullsized":"2024-04-25T10:05:38.551320+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:05.759734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194117+0000","last_change":"2024-04-25T10:05:37.228825+0000","last_active":"2024-04-25T10:05:38.194117+0000","last_peered":"2024-04-25T10:05:38.194117+0000","last_clean":"2024-04-25T10:05:38.194117+0000","last_became_active":"2024-04-25T10:05:37.228567+0000","last_became_peered":"2024-04-25T10:05:37.228567+0000","last_unstale":"2024-04-25T10:05:38.194117+0000","last_undegraded":"2024-04-25T10:05:38.194117+0000","last_fullsized":"2024-04-25T10:05:38.194117+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:42.164656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552001+0000","last_change":"2024-04-25T10:05:37.216195+0000","last_active":"2024-04-25T10:05:38.552001+0000","last_peered":"2024-04-25T10:05:38.552001+0000","last_clean":"2024-04-25T10:05:38.552001+0000","last_became_active":"2024-04-25T10:05:37.215910+0000","last_became_peered":"2024-04-25T10:05:37.215910+0000","last_unstale":"2024-04-25T10:05:38.552001+0000","last_undegraded":"2024-04-25T10:05:38.552001+0000","last_fullsized":"2024-04-25T10:05:38.552001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549428+0000","last_change":"2024-04-25T10:05:37.244630+0000","last_active":"2024-04-25T10:05:38.549428+0000","last_peered":"2024-04-25T10:05:38.549428+0000","last_clean":"2024-04-25T10:05:38.549428+0000","last_became_active":"2024-04-25T10:05:37.244120+0000","last_became_peered":"2024-04-25T10:05:37.244120+0000","last_unstale":"2024-04-25T10:05:38.549428+0000","last_undegraded":"2024-04-25T10:05:38.549428+0000","last_fullsized":"2024-04-25T10:05:38.549428+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193338+0000","last_change":"2024-04-25T10:05:37.229142+0000","last_active":"2024-04-25T10:05:38.193338+0000","last_peered":"2024-04-25T10:05:38.193338+0000","last_clean":"2024-04-25T10:05:38.193338+0000","last_became_active":"2024-04-25T10:05:37.228852+0000","last_became_peered":"2024-04-25T10:05:37.228852+0000","last_unstale":"2024-04-25T10:05:38.193338+0000","last_undegraded":"2024-04-25T10:05:38.193338+0000","last_fullsized":"2024-04-25T10:05:38.193338+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:56.021983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552832+0000","last_change":"2024-04-25T10:05:37.219710+0000","last_active":"2024-04-25T10:05:38.552832+0000","last_peered":"2024-04-25T10:05:38.552832+0000","last_clean":"2024-04-25T10:05:38.552832+0000","last_became_active":"2024-04-25T10:05:37.218987+0000","last_became_peered":"2024-04-25T10:05:37.218987+0000","last_unstale":"2024-04-25T10:05:38.552832+0000","last_undegraded":"2024-04-25T10:05:38.552832+0000","last_fullsized":"2024-04-25T10:05:38.552832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551559+0000","last_change":"2024-04-25T10:05:37.244343+0000","last_active":"2024-04-25T10:05:38.551559+0000","last_peered":"2024-04-25T10:05:38.551559+0000","last_clean":"2024-04-25T10:05:38.551559+0000","last_became_active":"2024-04-25T10:05:37.243655+0000","last_became_peered":"2024-04-25T10:05:37.243655+0000","last_unstale":"2024-04-25T10:05:38.551559+0000","last_undegraded":"2024-04-25T10:05:38.551559+0000","last_fullsized":"2024-04-25T10:05:38.551559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554321+0000","last_change":"2024-04-25T10:05:37.222335+0000","last_active":"2024-04-25T10:05:38.554321+0000","last_peered":"2024-04-25T10:05:38.554321+0000","last_clean":"2024-04-25T10:05:38.554321+0000","last_became_active":"2024-04-25T10:05:37.220912+0000","last_became_peered":"2024-04-25T10:05:37.220912+0000","last_unstale":"2024-04-25T10:05:38.554321+0000","last_undegraded":"2024-04-25T10:05:38.554321+0000","last_fullsized":"2024-04-25T10:05:38.554321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:02.441654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550628+0000","last_change":"2024-04-25T10:05:37.212566+0000","last_active":"2024-04-25T10:05:38.550628+0000","last_peered":"2024-04-25T10:05:38.550628+0000","last_clean":"2024-04-25T10:05:38.550628+0000","last_became_active":"2024-04-25T10:05:37.211952+0000","last_became_peered":"2024-04-25T10:05:37.211952+0000","last_unstale":"2024-04-25T10:05:38.550628+0000","last_undegraded":"2024-04-25T10:05:38.550628+0000","last_fullsized":"2024-04-25T10:05:38.550628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:00.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554521+0000","last_change":"2024-04-25T10:05:37.238454+0000","last_active":"2024-04-25T10:05:38.554521+0000","last_peered":"2024-04-25T10:05:38.554521+0000","last_clean":"2024-04-25T10:05:38.554521+0000","last_became_active":"2024-04-25T10:05:37.238147+0000","last_became_peered":"2024-04-25T10:05:37.238147+0000","last_unstale":"2024-04-25T10:05:38.554521+0000","last_undegraded":"2024-04-25T10:05:38.554521+0000","last_fullsized":"2024-04-25T10:05:38.554521+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:10.077596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550991+0000","last_change":"2024-04-25T10:05:37.216233+0000","last_active":"2024-04-25T10:05:38.550991+0000","last_peered":"2024-04-25T10:05:38.550991+0000","last_clean":"2024-04-25T10:05:38.550991+0000","last_became_active":"2024-04-25T10:05:37.214054+0000","last_became_peered":"2024-04-25T10:05:37.214054+0000","last_unstale":"2024-04-25T10:05:38.550991+0000","last_undegraded":"2024-04-25T10:05:38.550991+0000","last_fullsized":"2024-04-25T10:05:38.550991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:30.598917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553345+0000","last_change":"2024-04-25T10:05:37.219854+0000","last_active":"2024-04-25T10:05:38.553345+0000","last_peered":"2024-04-25T10:05:38.553345+0000","last_clean":"2024-04-25T10:05:38.553345+0000","last_became_active":"2024-04-25T10:05:37.219447+0000","last_became_peered":"2024-04-25T10:05:37.219447+0000","last_unstale":"2024-04-25T10:05:38.553345+0000","last_undegraded":"2024-04-25T10:05:38.553345+0000","last_fullsized":"2024-04-25T10:05:38.553345+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:00.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551069+0000","last_change":"2024-04-25T10:05:37.203182+0000","last_active":"2024-04-25T10:05:38.551069+0000","last_peered":"2024-04-25T10:05:38.551069+0000","last_clean":"2024-04-25T10:05:38.551069+0000","last_became_active":"2024-04-25T10:05:37.202893+0000","last_became_peered":"2024-04-25T10:05:37.202893+0000","last_unstale":"2024-04-25T10:05:38.551069+0000","last_undegraded":"2024-04-25T10:05:38.551069+0000","last_fullsized":"2024-04-25T10:05:38.551069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:12.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194246+0000","last_change":"2024-04-25T10:05:37.205817+0000","last_active":"2024-04-25T10:05:38.194246+0000","last_peered":"2024-04-25T10:05:38.194246+0000","last_clean":"2024-04-25T10:05:38.194246+0000","last_became_active":"2024-04-25T10:05:37.205572+0000","last_became_peered":"2024-04-25T10:05:37.205572+0000","last_unstale":"2024-04-25T10:05:38.194246+0000","last_undegraded":"2024-04-25T10:05:38.194246+0000","last_fullsized":"2024-04-25T10:05:38.194246+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:36.414900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553104+0000","last_change":"2024-04-25T10:05:37.240878+0000","last_active":"2024-04-25T10:05:38.553104+0000","last_peered":"2024-04-25T10:05:38.553104+0000","last_clean":"2024-04-25T10:05:38.553104+0000","last_became_active":"2024-04-25T10:05:37.240620+0000","last_became_peered":"2024-04-25T10:05:37.240620+0000","last_unstale":"2024-04-25T10:05:38.553104+0000","last_undegraded":"2024-04-25T10:05:38.553104+0000","last_fullsized":"2024-04-25T10:05:38.553104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550361+0000","last_change":"2024-04-25T10:05:37.246366+0000","last_active":"2024-04-25T10:05:38.550361+0000","last_peered":"2024-04-25T10:05:38.550361+0000","last_clean":"2024-04-25T10:05:38.550361+0000","last_became_active":"2024-04-25T10:05:37.239714+0000","last_became_peered":"2024-04-25T10:05:37.239714+0000","last_unstale":"2024-04-25T10:05:38.550361+0000","last_undegraded":"2024-04-25T10:05:38.550361+0000","last_fullsized":"2024-04-25T10:05:38.550361+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552009+0000","last_change":"2024-04-25T10:05:37.216294+0000","last_active":"2024-04-25T10:05:38.552009+0000","last_peered":"2024-04-25T10:05:38.552009+0000","last_clean":"2024-04-25T10:05:38.552009+0000","last_became_active":"2024-04-25T10:05:37.215738+0000","last_became_peered":"2024-04-25T10:05:37.215738+0000","last_unstale":"2024-04-25T10:05:38.552009+0000","last_undegraded":"2024-04-25T10:05:38.552009+0000","last_fullsized":"2024-04-25T10:05:38.552009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191560+0000","last_change":"2024-04-25T10:05:37.221885+0000","last_active":"2024-04-25T10:05:38.191560+0000","last_peered":"2024-04-25T10:05:38.191560+0000","last_clean":"2024-04-25T10:05:38.191560+0000","last_became_active":"2024-04-25T10:05:37.219364+0000","last_became_peered":"2024-04-25T10:05:37.219364+0000","last_unstale":"2024-04-25T10:05:38.191560+0000","last_undegraded":"2024-04-25T10:05:38.191560+0000","last_fullsized":"2024-04-25T10:05:38.191560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549409+0000","last_change":"2024-04-25T10:05:37.229695+0000","last_active":"2024-04-25T10:05:38.549409+0000","last_peered":"2024-04-25T10:05:38.549409+0000","last_clean":"2024-04-25T10:05:38.549409+0000","last_became_active":"2024-04-25T10:05:37.228961+0000","last_became_peered":"2024-04-25T10:05:37.228961+0000","last_unstale":"2024-04-25T10:05:38.549409+0000","last_undegraded":"2024-04-25T10:05:38.549409+0000","last_fullsized":"2024-04-25T10:05:38.549409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191931+0000","last_change":"2024-04-25T10:05:37.230425+0000","last_active":"2024-04-25T10:05:38.191931+0000","last_peered":"2024-04-25T10:05:38.191931+0000","last_clean":"2024-04-25T10:05:38.191931+0000","last_became_active":"2024-04-25T10:05:37.229462+0000","last_became_peered":"2024-04-25T10:05:37.229462+0000","last_unstale":"2024-04-25T10:05:38.191931+0000","last_undegraded":"2024-04-25T10:05:38.191931+0000","last_fullsized":"2024-04-25T10:05:38.191931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194406+0000","last_change":"2024-04-25T10:05:36.192412+0000","last_active":"2024-04-25T10:05:38.194406+0000","last_peered":"2024-04-25T10:05:38.194406+0000","last_clean":"2024-04-25T10:05:38.194406+0000","last_became_active":"2024-04-25T10:05:36.191754+0000","last_became_peered":"2024-04-25T10:05:36.191754+0000","last_unstale":"2024-04-25T10:05:38.194406+0000","last_undegraded":"2024-04-25T10:05:38.194406+0000","last_fullsized":"2024-04-25T10:05:38.194406+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:33.604952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194384+0000","last_change":"2024-04-25T10:05:36.197170+0000","last_active":"2024-04-25T10:05:38.194384+0000","last_peered":"2024-04-25T10:05:38.194384+0000","last_clean":"2024-04-25T10:05:38.194384+0000","last_became_active":"2024-04-25T10:05:36.196555+0000","last_became_peered":"2024-04-25T10:05:36.196555+0000","last_unstale":"2024-04-25T10:05:38.194384+0000","last_undegraded":"2024-04-25T10:05:38.194384+0000","last_fullsized":"2024-04-25T10:05:38.194384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:43.120025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550712+0000","last_change":"2024-04-25T10:05:37.206507+0000","last_active":"2024-04-25T10:05:38.550712+0000","last_peered":"2024-04-25T10:05:38.550712+0000","last_clean":"2024-04-25T10:05:38.550712+0000","last_became_active":"2024-04-25T10:05:37.204792+0000","last_became_peered":"2024-04-25T10:05:37.204792+0000","last_unstale":"2024-04-25T10:05:38.550712+0000","last_undegraded":"2024-04-25T10:05:38.550712+0000","last_fullsized":"2024-04-25T10:05:38.550712+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:16.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192596+0000","last_change":"2024-04-25T10:05:36.201693+0000","last_active":"2024-04-25T10:05:38.192596+0000","last_peered":"2024-04-25T10:05:38.192596+0000","last_clean":"2024-04-25T10:05:38.192596+0000","last_became_active":"2024-04-25T10:05:36.200944+0000","last_became_peered":"2024-04-25T10:05:36.200944+0000","last_unstale":"2024-04-25T10:05:38.192596+0000","last_undegraded":"2024-04-25T10:05:38.192596+0000","last_fullsized":"2024-04-25T10:05:38.192596+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550050+0000","last_change":"2024-04-25T10:05:37.222859+0000","last_active":"2024-04-25T10:05:38.550050+0000","last_peered":"2024-04-25T10:05:38.550050+0000","last_clean":"2024-04-25T10:05:38.550050+0000","last_became_active":"2024-04-25T10:05:37.222162+0000","last_became_peered":"2024-04-25T10:05:37.222162+0000","last_unstale":"2024-04-25T10:05:38.550050+0000","last_undegraded":"2024-04-25T10:05:38.550050+0000","last_fullsized":"2024-04-25T10:05:38.550050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:44.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553382+0000","last_change":"2024-04-25T10:05:36.193782+0000","last_active":"2024-04-25T10:05:38.553382+0000","last_peered":"2024-04-25T10:05:38.553382+0000","last_clean":"2024-04-25T10:05:38.553382+0000","last_became_active":"2024-04-25T10:05:36.190455+0000","last_became_peered":"2024-04-25T10:05:36.190455+0000","last_unstale":"2024-04-25T10:05:38.553382+0000","last_undegraded":"2024-04-25T10:05:38.553382+0000","last_fullsized":"2024-04-25T10:05:38.553382+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:32.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550845+0000","last_change":"2024-04-25T10:05:37.246868+0000","last_active":"2024-04-25T10:05:38.550845+0000","last_peered":"2024-04-25T10:05:38.550845+0000","last_clean":"2024-04-25T10:05:38.550845+0000","last_became_active":"2024-04-25T10:05:37.246095+0000","last_became_peered":"2024-04-25T10:05:37.246095+0000","last_unstale":"2024-04-25T10:05:38.550845+0000","last_undegraded":"2024-04-25T10:05:38.550845+0000","last_fullsized":"2024-04-25T10:05:38.550845+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548980+0000","last_change":"2024-04-25T10:05:36.197860+0000","last_active":"2024-04-25T10:05:38.548980+0000","last_peered":"2024-04-25T10:05:38.548980+0000","last_clean":"2024-04-25T10:05:38.548980+0000","last_became_active":"2024-04-25T10:05:36.197137+0000","last_became_peered":"2024-04-25T10:05:36.197137+0000","last_unstale":"2024-04-25T10:05:38.548980+0000","last_undegraded":"2024-04-25T10:05:38.548980+0000","last_fullsized":"2024-04-25T10:05:38.548980+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:30.726291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551788+0000","last_change":"2024-04-25T10:05:37.206314+0000","last_active":"2024-04-25T10:05:38.551788+0000","last_peered":"2024-04-25T10:05:38.551788+0000","last_clean":"2024-04-25T10:05:38.551788+0000","last_became_active":"2024-04-25T10:05:37.205748+0000","last_became_peered":"2024-04-25T10:05:37.205748+0000","last_unstale":"2024-04-25T10:05:38.551788+0000","last_undegraded":"2024-04-25T10:05:38.551788+0000","last_fullsized":"2024-04-25T10:05:38.551788+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:52.894883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550829+0000","last_change":"2024-04-25T10:05:37.240137+0000","last_active":"2024-04-25T10:05:38.550829+0000","last_peered":"2024-04-25T10:05:38.550829+0000","last_clean":"2024-04-25T10:05:38.550829+0000","last_became_active":"2024-04-25T10:05:37.239387+0000","last_became_peered":"2024-04-25T10:05:37.239387+0000","last_unstale":"2024-04-25T10:05:38.550829+0000","last_undegraded":"2024-04-25T10:05:38.550829+0000","last_fullsized":"2024-04-25T10:05:38.550829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551084+0000","last_change":"2024-04-25T10:05:36.197600+0000","last_active":"2024-04-25T10:05:38.551084+0000","last_peered":"2024-04-25T10:05:38.551084+0000","last_clean":"2024-04-25T10:05:38.551084+0000","last_became_active":"2024-04-25T10:05:36.197265+0000","last_became_peered":"2024-04-25T10:05:36.197265+0000","last_unstale":"2024-04-25T10:05:38.551084+0000","last_undegraded":"2024-04-25T10:05:38.551084+0000","last_fullsized":"2024-04-25T10:05:38.551084+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:24.137684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548851+0000","last_change":"2024-04-25T10:05:37.224800+0000","last_active":"2024-04-25T10:05:38.548851+0000","last_peered":"2024-04-25T10:05:38.548851+0000","last_clean":"2024-04-25T10:05:38.548851+0000","last_became_active":"2024-04-25T10:05:37.224068+0000","last_became_peered":"2024-04-25T10:05:37.224068+0000","last_unstale":"2024-04-25T10:05:38.548851+0000","last_undegraded":"2024-04-25T10:05:38.548851+0000","last_fullsized":"2024-04-25T10:05:38.548851+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:25.148179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551076+0000","last_change":"2024-04-25T10:05:36.195068+0000","last_active":"2024-04-25T10:05:38.551076+0000","last_peered":"2024-04-25T10:05:38.551076+0000","last_clean":"2024-04-25T10:05:38.551076+0000","last_became_active":"2024-04-25T10:05:36.194281+0000","last_became_peered":"2024-04-25T10:05:36.194281+0000","last_unstale":"2024-04-25T10:05:38.551076+0000","last_undegraded":"2024-04-25T10:05:38.551076+0000","last_fullsized":"2024-04-25T10:05:38.551076+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:41.154160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191829+0000","last_change":"2024-04-25T10:05:37.223331+0000","last_active":"2024-04-25T10:05:38.191829+0000","last_peered":"2024-04-25T10:05:38.191829+0000","last_clean":"2024-04-25T10:05:38.191829+0000","last_became_active":"2024-04-25T10:05:37.222613+0000","last_became_peered":"2024-04-25T10:05:37.222613+0000","last_unstale":"2024-04-25T10:05:38.191829+0000","last_undegraded":"2024-04-25T10:05:38.191829+0000","last_fullsized":"2024-04-25T10:05:38.191829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549616+0000","last_change":"2024-04-25T10:05:36.200167+0000","last_active":"2024-04-25T10:05:38.549616+0000","last_peered":"2024-04-25T10:05:38.549616+0000","last_clean":"2024-04-25T10:05:38.549616+0000","last_became_active":"2024-04-25T10:05:36.199467+0000","last_became_peered":"2024-04-25T10:05:36.199467+0000","last_unstale":"2024-04-25T10:05:38.549616+0000","last_undegraded":"2024-04-25T10:05:38.549616+0000","last_fullsized":"2024-04-25T10:05:38.549616+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:47.830431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552373+0000","last_change":"2024-04-25T10:05:37.241002+0000","last_active":"2024-04-25T10:05:38.552373+0000","last_peered":"2024-04-25T10:05:38.552373+0000","last_clean":"2024-04-25T10:05:38.552373+0000","last_became_active":"2024-04-25T10:05:37.240740+0000","last_became_peered":"2024-04-25T10:05:37.240740+0000","last_unstale":"2024-04-25T10:05:38.552373+0000","last_undegraded":"2024-04-25T10:05:38.552373+0000","last_fullsized":"2024-04-25T10:05:38.552373+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551231+0000","last_change":"2024-04-25T10:05:36.195424+0000","last_active":"2024-04-25T10:05:38.551231+0000","last_peered":"2024-04-25T10:05:38.551231+0000","last_clean":"2024-04-25T10:05:38.551231+0000","last_became_active":"2024-04-25T10:05:36.193306+0000","last_became_peered":"2024-04-25T10:05:36.193306+0000","last_unstale":"2024-04-25T10:05:38.551231+0000","last_undegraded":"2024-04-25T10:05:38.551231+0000","last_fullsized":"2024-04-25T10:05:38.551231+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549522+0000","last_change":"2024-04-25T10:05:37.220528+0000","last_active":"2024-04-25T10:05:38.549522+0000","last_peered":"2024-04-25T10:05:38.549522+0000","last_clean":"2024-04-25T10:05:38.549522+0000","last_became_active":"2024-04-25T10:05:37.219586+0000","last_became_peered":"2024-04-25T10:05:37.219586+0000","last_unstale":"2024-04-25T10:05:38.549522+0000","last_undegraded":"2024-04-25T10:05:38.549522+0000","last_fullsized":"2024-04-25T10:05:38.549522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:55.011916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551578+0000","last_change":"2024-04-25T10:05:36.199745+0000","last_active":"2024-04-25T10:05:38.551578+0000","last_peered":"2024-04-25T10:05:38.551578+0000","last_clean":"2024-04-25T10:05:38.551578+0000","last_became_active":"2024-04-25T10:05:36.191871+0000","last_became_peered":"2024-04-25T10:05:36.191871+0000","last_unstale":"2024-04-25T10:05:38.551578+0000","last_undegraded":"2024-04-25T10:05:38.551578+0000","last_fullsized":"2024-04-25T10:05:38.551578+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191388+0000","last_change":"2024-04-25T10:05:37.230868+0000","last_active":"2024-04-25T10:05:38.191388+0000","last_peered":"2024-04-25T10:05:38.191388+0000","last_clean":"2024-04-25T10:05:38.191388+0000","last_became_active":"2024-04-25T10:05:37.229549+0000","last_became_peered":"2024-04-25T10:05:37.229549+0000","last_unstale":"2024-04-25T10:05:38.191388+0000","last_undegraded":"2024-04-25T10:05:38.191388+0000","last_fullsized":"2024-04-25T10:05:38.191388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:04.305032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553620+0000","last_change":"2024-04-25T10:05:36.201220+0000","last_active":"2024-04-25T10:05:38.553620+0000","last_peered":"2024-04-25T10:05:38.553620+0000","last_clean":"2024-04-25T10:05:38.553620+0000","last_became_active":"2024-04-25T10:05:36.200820+0000","last_became_peered":"2024-04-25T10:05:36.200820+0000","last_unstale":"2024-04-25T10:05:38.553620+0000","last_undegraded":"2024-04-25T10:05:38.553620+0000","last_fullsized":"2024-04-25T10:05:38.553620+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:54.001421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554271+0000","last_change":"2024-04-25T10:05:37.215939+0000","last_active":"2024-04-25T10:05:38.554271+0000","last_peered":"2024-04-25T10:05:38.554271+0000","last_clean":"2024-04-25T10:05:38.554271+0000","last_became_active":"2024-04-25T10:05:37.215693+0000","last_became_peered":"2024-04-25T10:05:37.215693+0000","last_unstale":"2024-04-25T10:05:38.554271+0000","last_undegraded":"2024-04-25T10:05:38.554271+0000","last_fullsized":"2024-04-25T10:05:38.554271+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:12.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552870+0000","last_change":"2024-04-25T10:05:37.238447+0000","last_active":"2024-04-25T10:05:38.552870+0000","last_peered":"2024-04-25T10:05:38.552870+0000","last_clean":"2024-04-25T10:05:38.552870+0000","last_became_active":"2024-04-25T10:05:37.238106+0000","last_became_peered":"2024-04-25T10:05:37.238106+0000","last_unstale":"2024-04-25T10:05:38.552870+0000","last_undegraded":"2024-04-25T10:05:38.552870+0000","last_fullsized":"2024-04-25T10:05:38.552870+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:06.779179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549834+0000","last_change":"2024-04-25T10:05:36.198280+0000","last_active":"2024-04-25T10:05:38.549834+0000","last_peered":"2024-04-25T10:05:38.549834+0000","last_clean":"2024-04-25T10:05:38.549834+0000","last_became_active":"2024-04-25T10:05:36.197484+0000","last_became_peered":"2024-04-25T10:05:36.197484+0000","last_unstale":"2024-04-25T10:05:38.549834+0000","last_undegraded":"2024-04-25T10:05:38.549834+0000","last_fullsized":"2024-04-25T10:05:38.549834+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.796742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552446+0000","last_change":"2024-04-25T10:05:37.219004+0000","last_active":"2024-04-25T10:05:38.552446+0000","last_peered":"2024-04-25T10:05:38.552446+0000","last_clean":"2024-04-25T10:05:38.552446+0000","last_became_active":"2024-04-25T10:05:37.216939+0000","last_became_peered":"2024-04-25T10:05:37.216939+0000","last_unstale":"2024-04-25T10:05:38.552446+0000","last_undegraded":"2024-04-25T10:05:38.552446+0000","last_fullsized":"2024-04-25T10:05:38.552446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551470+0000","last_change":"2024-04-25T10:05:36.198305+0000","last_active":"2024-04-25T10:05:38.551470+0000","last_peered":"2024-04-25T10:05:38.551470+0000","last_clean":"2024-04-25T10:05:38.551470+0000","last_became_active":"2024-04-25T10:05:36.191908+0000","last_became_peered":"2024-04-25T10:05:36.191908+0000","last_unstale":"2024-04-25T10:05:38.551470+0000","last_undegraded":"2024-04-25T10:05:38.551470+0000","last_fullsized":"2024-04-25T10:05:38.551470+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:52.819131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553781+0000","last_change":"2024-04-25T10:05:37.224631+0000","last_active":"2024-04-25T10:05:38.553781+0000","last_peered":"2024-04-25T10:05:38.553781+0000","last_clean":"2024-04-25T10:05:38.553781+0000","last_became_active":"2024-04-25T10:05:37.224014+0000","last_became_peered":"2024-04-25T10:05:37.224014+0000","last_unstale":"2024-04-25T10:05:38.553781+0000","last_undegraded":"2024-04-25T10:05:38.553781+0000","last_fullsized":"2024-04-25T10:05:38.553781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:08.906935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553276+0000","last_change":"2024-04-25T10:05:36.203863+0000","last_active":"2024-04-25T10:05:38.553276+0000","last_peered":"2024-04-25T10:05:38.553276+0000","last_clean":"2024-04-25T10:05:38.553276+0000","last_became_active":"2024-04-25T10:05:36.203473+0000","last_became_peered":"2024-04-25T10:05:36.203473+0000","last_unstale":"2024-04-25T10:05:38.553276+0000","last_undegraded":"2024-04-25T10:05:38.553276+0000","last_fullsized":"2024-04-25T10:05:38.553276+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:24.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193575+0000","last_change":"2024-04-25T10:05:37.228514+0000","last_active":"2024-04-25T10:05:38.193575+0000","last_peered":"2024-04-25T10:05:38.193575+0000","last_clean":"2024-04-25T10:05:38.193575+0000","last_became_active":"2024-04-25T10:05:37.213429+0000","last_became_peered":"2024-04-25T10:05:37.213429+0000","last_unstale":"2024-04-25T10:05:38.193575+0000","last_undegraded":"2024-04-25T10:05:38.193575+0000","last_fullsized":"2024-04-25T10:05:38.193575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:07.257110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.555115+0000","last_change":"2024-04-25T10:05:36.192160+0000","last_active":"2024-04-25T10:05:38.555115+0000","last_peered":"2024-04-25T10:05:38.555115+0000","last_clean":"2024-04-25T10:05:38.555115+0000","last_became_active":"2024-04-25T10:05:36.191475+0000","last_became_peered":"2024-04-25T10:05:36.191475+0000","last_unstale":"2024-04-25T10:05:38.555115+0000","last_undegraded":"2024-04-25T10:05:38.555115+0000","last_fullsized":"2024-04-25T10:05:38.555115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:06.047245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548936+0000","last_change":"2024-04-25T10:05:37.218759+0000","last_active":"2024-04-25T10:05:38.548936+0000","last_peered":"2024-04-25T10:05:38.548936+0000","last_clean":"2024-04-25T10:05:38.548936+0000","last_became_active":"2024-04-25T10:05:37.217992+0000","last_became_peered":"2024-04-25T10:05:37.217992+0000","last_unstale":"2024-04-25T10:05:38.548936+0000","last_undegraded":"2024-04-25T10:05:38.548936+0000","last_fullsized":"2024-04-25T10:05:38.548936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:59.463557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194500+0000","last_change":"2024-04-25T10:05:37.227960+0000","last_active":"2024-04-25T10:05:38.194500+0000","last_peered":"2024-04-25T10:05:38.194500+0000","last_clean":"2024-04-25T10:05:38.194500+0000","last_became_active":"2024-04-25T10:05:37.213414+0000","last_became_peered":"2024-04-25T10:05:37.213414+0000","last_unstale":"2024-04-25T10:05:38.194500+0000","last_undegraded":"2024-04-25T10:05:38.194500+0000","last_fullsized":"2024-04-25T10:05:38.194500+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:01.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191893+0000","last_change":"2024-04-25T10:05:37.224756+0000","last_active":"2024-04-25T10:05:38.191893+0000","last_peered":"2024-04-25T10:05:38.191893+0000","last_clean":"2024-04-25T10:05:38.191893+0000","last_became_active":"2024-04-25T10:05:37.223895+0000","last_became_peered":"2024-04-25T10:05:37.223895+0000","last_unstale":"2024-04-25T10:05:38.191893+0000","last_undegraded":"2024-04-25T10:05:38.191893+0000","last_fullsized":"2024-04-25T10:05:38.191893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194604+0000","last_change":"2024-04-25T10:05:36.186378+0000","last_active":"2024-04-25T10:05:38.194604+0000","last_peered":"2024-04-25T10:05:38.194604+0000","last_clean":"2024-04-25T10:05:38.194604+0000","last_became_active":"2024-04-25T10:05:36.185733+0000","last_became_peered":"2024-04-25T10:05:36.185733+0000","last_unstale":"2024-04-25T10:05:38.194604+0000","last_undegraded":"2024-04-25T10:05:38.194604+0000","last_fullsized":"2024-04-25T10:05:38.194604+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:43.722766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551744+0000","last_change":"2024-04-25T10:05:36.199741+0000","last_active":"2024-04-25T10:05:38.551744+0000","last_peered":"2024-04-25T10:05:38.551744+0000","last_clean":"2024-04-25T10:05:38.551744+0000","last_became_active":"2024-04-25T10:05:36.199311+0000","last_became_peered":"2024-04-25T10:05:36.199311+0000","last_unstale":"2024-04-25T10:05:38.551744+0000","last_undegraded":"2024-04-25T10:05:38.551744+0000","last_fullsized":"2024-04-25T10:05:38.551744+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551563+0000","last_change":"2024-04-25T10:05:37.240101+0000","last_active":"2024-04-25T10:05:38.551563+0000","last_peered":"2024-04-25T10:05:38.551563+0000","last_clean":"2024-04-25T10:05:38.551563+0000","last_became_active":"2024-04-25T10:05:37.239789+0000","last_became_peered":"2024-04-25T10:05:37.239789+0000","last_unstale":"2024-04-25T10:05:38.551563+0000","last_undegraded":"2024-04-25T10:05:38.551563+0000","last_fullsized":"2024-04-25T10:05:38.551563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:04.305032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552440+0000","last_change":"2024-04-25T10:05:36.194819+0000","last_active":"2024-04-25T10:05:38.552440+0000","last_peered":"2024-04-25T10:05:38.552440+0000","last_clean":"2024-04-25T10:05:38.552440+0000","last_became_active":"2024-04-25T10:05:36.194511+0000","last_became_peered":"2024-04-25T10:05:36.194511+0000","last_unstale":"2024-04-25T10:05:38.552440+0000","last_undegraded":"2024-04-25T10:05:38.552440+0000","last_fullsized":"2024-04-25T10:05:38.552440+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:43.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194192+0000","last_change":"2024-04-25T10:05:37.204849+0000","last_active":"2024-04-25T10:05:38.194192+0000","last_peered":"2024-04-25T10:05:38.194192+0000","last_clean":"2024-04-25T10:05:38.194192+0000","last_became_active":"2024-04-25T10:05:37.204310+0000","last_became_peered":"2024-04-25T10:05:37.204310+0000","last_unstale":"2024-04-25T10:05:38.194192+0000","last_undegraded":"2024-04-25T10:05:38.194192+0000","last_fullsized":"2024-04-25T10:05:38.194192+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:49.155350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554104+0000","last_change":"2024-04-25T10:05:36.189798+0000","last_active":"2024-04-25T10:05:38.554104+0000","last_peered":"2024-04-25T10:05:38.554104+0000","last_clean":"2024-04-25T10:05:38.554104+0000","last_became_active":"2024-04-25T10:05:36.189079+0000","last_became_peered":"2024-04-25T10:05:36.189079+0000","last_unstale":"2024-04-25T10:05:38.554104+0000","last_undegraded":"2024-04-25T10:05:38.554104+0000","last_fullsized":"2024-04-25T10:05:38.554104+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:06.607042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192449+0000","last_change":"2024-04-25T10:05:37.213816+0000","last_active":"2024-04-25T10:05:38.192449+0000","last_peered":"2024-04-25T10:05:38.192449+0000","last_clean":"2024-04-25T10:05:38.192449+0000","last_became_active":"2024-04-25T10:05:37.213082+0000","last_became_peered":"2024-04-25T10:05:37.213082+0000","last_unstale":"2024-04-25T10:05:38.192449+0000","last_undegraded":"2024-04-25T10:05:38.192449+0000","last_fullsized":"2024-04-25T10:05:38.192449+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193639+0000","last_change":"2024-04-25T10:05:37.199321+0000","last_active":"2024-04-25T10:05:38.193639+0000","last_peered":"2024-04-25T10:05:38.193639+0000","last_clean":"2024-04-25T10:05:38.193639+0000","last_became_active":"2024-04-25T10:05:37.199011+0000","last_became_peered":"2024-04-25T10:05:37.199011+0000","last_unstale":"2024-04-25T10:05:38.193639+0000","last_undegraded":"2024-04-25T10:05:38.193639+0000","last_fullsized":"2024-04-25T10:05:38.193639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.753836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551360+0000","last_change":"2024-04-25T10:05:36.198586+0000","last_active":"2024-04-25T10:05:38.551360+0000","last_peered":"2024-04-25T10:05:38.551360+0000","last_clean":"2024-04-25T10:05:38.551360+0000","last_became_active":"2024-04-25T10:05:36.197856+0000","last_became_peered":"2024-04-25T10:05:36.197856+0000","last_unstale":"2024-04-25T10:05:38.551360+0000","last_undegraded":"2024-04-25T10:05:38.551360+0000","last_fullsized":"2024-04-25T10:05:38.551360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550497+0000","last_change":"2024-04-25T10:05:37.225133+0000","last_active":"2024-04-25T10:05:38.550497+0000","last_peered":"2024-04-25T10:05:38.550497+0000","last_clean":"2024-04-25T10:05:38.550497+0000","last_became_active":"2024-04-25T10:05:37.224655+0000","last_became_peered":"2024-04-25T10:05:37.224655+0000","last_unstale":"2024-04-25T10:05:38.550497+0000","last_undegraded":"2024-04-25T10:05:38.550497+0000","last_fullsized":"2024-04-25T10:05:38.550497+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553244+0000","last_change":"2024-04-25T10:05:36.189704+0000","last_active":"2024-04-25T10:05:38.553244+0000","last_peered":"2024-04-25T10:05:38.553244+0000","last_clean":"2024-04-25T10:05:38.553244+0000","last_became_active":"2024-04-25T10:05:36.188969+0000","last_became_peered":"2024-04-25T10:05:36.188969+0000","last_unstale":"2024-04-25T10:05:38.553244+0000","last_undegraded":"2024-04-25T10:05:38.553244+0000","last_fullsized":"2024-04-25T10:05:38.553244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:58.453061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553005+0000","last_change":"2024-04-25T10:05:37.219793+0000","last_active":"2024-04-25T10:05:38.553005+0000","last_peered":"2024-04-25T10:05:38.553005+0000","last_clean":"2024-04-25T10:05:38.553005+0000","last_became_active":"2024-04-25T10:05:37.219521+0000","last_became_peered":"2024-04-25T10:05:37.219521+0000","last_unstale":"2024-04-25T10:05:38.553005+0000","last_undegraded":"2024-04-25T10:05:38.553005+0000","last_fullsized":"2024-04-25T10:05:38.553005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:57.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550594+0000","last_change":"2024-04-25T10:05:36.198953+0000","last_active":"2024-04-25T10:05:38.550594+0000","last_peered":"2024-04-25T10:05:38.550594+0000","last_clean":"2024-04-25T10:05:38.550594+0000","last_became_active":"2024-04-25T10:05:36.197976+0000","last_became_peered":"2024-04-25T10:05:36.197976+0000","last_unstale":"2024-04-25T10:05:38.550594+0000","last_undegraded":"2024-04-25T10:05:38.550594+0000","last_fullsized":"2024-04-25T10:05:38.550594+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:09.514786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191968+0000","last_change":"2024-04-25T10:05:37.217261+0000","last_active":"2024-04-25T10:05:38.191968+0000","last_peered":"2024-04-25T10:05:38.191968+0000","last_clean":"2024-04-25T10:05:38.191968+0000","last_became_active":"2024-04-25T10:05:37.216539+0000","last_became_peered":"2024-04-25T10:05:37.216539+0000","last_unstale":"2024-04-25T10:05:38.191968+0000","last_undegraded":"2024-04-25T10:05:38.191968+0000","last_fullsized":"2024-04-25T10:05:38.191968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551865+0000","last_change":"2024-04-25T10:05:36.191359+0000","last_active":"2024-04-25T10:05:38.551865+0000","last_peered":"2024-04-25T10:05:38.551865+0000","last_clean":"2024-04-25T10:05:38.551865+0000","last_became_active":"2024-04-25T10:05:36.190991+0000","last_became_peered":"2024-04-25T10:05:36.190991+0000","last_unstale":"2024-04-25T10:05:38.551865+0000","last_undegraded":"2024-04-25T10:05:38.551865+0000","last_fullsized":"2024-04-25T10:05:38.551865+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:00.276908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551375+0000","last_change":"2024-04-25T10:05:37.215467+0000","last_active":"2024-04-25T10:05:38.551375+0000","last_peered":"2024-04-25T10:05:38.551375+0000","last_clean":"2024-04-25T10:05:38.551375+0000","last_became_active":"2024-04-25T10:05:37.212611+0000","last_became_peered":"2024-04-25T10:05:37.212611+0000","last_unstale":"2024-04-25T10:05:38.551375+0000","last_undegraded":"2024-04-25T10:05:38.551375+0000","last_fullsized":"2024-04-25T10:05:38.551375+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:29.558721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548739+0000","last_change":"2024-04-25T10:05:36.191311+0000","last_active":"2024-04-25T10:05:38.548739+0000","last_peered":"2024-04-25T10:05:38.548739+0000","last_clean":"2024-04-25T10:05:38.548739+0000","last_became_active":"2024-04-25T10:05:36.190535+0000","last_became_peered":"2024-04-25T10:05:36.190535+0000","last_unstale":"2024-04-25T10:05:38.548739+0000","last_undegraded":"2024-04-25T10:05:38.548739+0000","last_fullsized":"2024-04-25T10:05:38.548739+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.118904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552275+0000","last_change":"2024-04-25T10:05:37.209994+0000","last_active":"2024-04-25T10:05:38.552275+0000","last_peered":"2024-04-25T10:05:38.552275+0000","last_clean":"2024-04-25T10:05:38.552275+0000","last_became_active":"2024-04-25T10:05:37.209686+0000","last_became_peered":"2024-04-25T10:05:37.209686+0000","last_unstale":"2024-04-25T10:05:38.552275+0000","last_undegraded":"2024-04-25T10:05:38.552275+0000","last_fullsized":"2024-04-25T10:05:38.552275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:36.861585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192702+0000","last_change":"2024-04-25T10:05:36.184629+0000","last_active":"2024-04-25T10:05:38.192702+0000","last_peered":"2024-04-25T10:05:38.192702+0000","last_clean":"2024-04-25T10:05:38.192702+0000","last_became_active":"2024-04-25T10:05:36.183892+0000","last_became_peered":"2024-04-25T10:05:36.183892+0000","last_unstale":"2024-04-25T10:05:38.192702+0000","last_undegraded":"2024-04-25T10:05:38.192702+0000","last_fullsized":"2024-04-25T10:05:38.192702+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:39.205026+0000","last_change":"2024-04-25T10:05:36.201261+0000","last_active":"2024-04-25T10:05:39.205026+0000","last_peered":"2024-04-25T10:05:39.205026+0000","last_clean":"2024-04-25T10:05:39.205026+0000","last_became_active":"2024-04-25T10:05:36.199919+0000","last_became_peered":"2024-04-25T10:05:36.199919+0000","last_unstale":"2024-04-25T10:05:39.205026+0000","last_undegraded":"2024-04-25T10:05:39.205026+0000","last_fullsized":"2024-04-25T10:05:39.205026+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:37.214569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549551+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:05:38.549551+0000","last_peered":"2024-04-25T10:05:38.549551+0000","last_clean":"2024-04-25T10:05:38.549551+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:05:38.549551+0000","last_undegraded":"2024-04-25T10:05:38.549551+0000","last_fullsized":"2024-04-25T10:05:38.549551+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":52,"acting":52,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46284,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325556,"statfs":{"total":96636764160,"available":96589369344,"internally_reserved":0,"allocated":1245184,"data_stored":879702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40096,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331744,"statfs":{"total":96636764160,"available":96595705856,"internally_reserved":0,"allocated":593920,"data_stored":262475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":643072,"data_stored":289210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45700,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326140,"statfs":{"total":96636764160,"available":96589967360,"internally_reserved":0,"allocated":647168,"data_stored":289792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44308,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327532,"statfs":{"total":96636764160,"available":96591392768,"internally_reserved":0,"allocated":659456,"data_stored":290844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":655360,"data_stored":290822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":651264,"data_stored":289370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50452,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321388,"statfs":{"total":96636764160,"available":96585101312,"internally_reserved":0,"allocated":1245184,"data_stored":879634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:40.592 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:40.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.742+0000 7f2c7072b640 1 Processor -- start 2024-04-25T10:05:40.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.742+0000 7f2c7072b640 1 -- start start 2024-04-25T10:05:40.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c7072b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2c680cee80 0x7f2c68168850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c7072b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680ca9b0 con 0x7f2c6815dd90 2024-04-25T10:05:40.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c7072b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680cac90 con 0x7f2c6815d9b0 2024-04-25T10:05:40.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c7072b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c680cbe30 con 0x7f2c680cee80 2024-04-25T10:05:40.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6dcc6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2c6815dd90 0x7f2c680ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40122/0 (socket says 172.21.15.133:40122) 2024-04-25T10:05:40.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6dcc6640 1 -- 172.21.15.133:0/2862424326 learned_addr learned my addr 172.21.15.133:0/2862424326 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6ecc8640 1 --2- 172.21.15.133:0/2862424326 >> v2:172.21.15.162:3300/0 conn(0x7f2c680cee80 0x7f2c68168850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053866140 0 0) 0x7f2c680ca9b0 con 0x7f2c6815dd90 2024-04-25T10:05:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c54003530 con 0x7f2c6815dd90 2024-04-25T10:05:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1141464332 0 0) 0x7f2c54003530 con 0x7f2c6815dd90 2024-04-25T10:05:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 >> v2:172.21.15.162:3300/0 conn(0x7f2c680cee80 msgr2=0x7f2c68168850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 --2- 172.21.15.133:0/2862424326 >> v2:172.21.15.162:3300/0 conn(0x7f2c680cee80 0x7f2c68168850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 >> v1:172.21.15.162:6789/0 conn(0x7f2c6815d9b0 legacy=0x7f2c680cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680caf70 con 0x7f2c6815dd90 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2c580016a0 con 0x7f2c6815dd90 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 >> v1:172.21.15.133:6789/0 conn(0x7f2c6815dd90 legacy=0x7f2c680ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c540039c0 0x7f2c54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c54004810 0x7f2c54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c540051d0 0x7f2c5400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680cbe30 con 0x7f2c54004810 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5400c170 con 0x7f2c540039c0 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5400c370 con 0x7f2c540051d0 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6dcc6640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c54004810 0x7f2c54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6e4c7640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c540039c0 0x7f2c54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6ecc8640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c540051d0 0x7f2c5400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6dcc6640 1 -- 172.21.15.133:0/2862424326 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c540051d0 msgr2=0x7f2c5400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6dcc6640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c540051d0 0x7f2c5400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6dcc6640 1 -- 172.21.15.133:0/2862424326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c540039c0 msgr2=0x7f2c54003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6dcc6640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c540039c0 0x7f2c54003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6dcc6640 1 -- 172.21.15.133:0/2862424326 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c54003530 con 0x7f2c54004810 2024-04-25T10:05:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6ecc8640 1 -- 172.21.15.133:0/2862424326 reap_dead start 2024-04-25T10:05:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.746+0000 7f2c6e4c7640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c540039c0 0x7f2c54003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6dcc6640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c54004810 0x7f2c54004c90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800c210 tx=0x7f2c5801cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21153a256d1dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58007d20 con 0x7f2c54004810 2024-04-25T10:05:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c580a9020 con 0x7f2c54004810 2024-04-25T10:05:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6d4c5640 1 -- 172.21.15.133:0/2862424326 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58005b20 con 0x7f2c54004810 2024-04-25T10:05:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 -- 172.21.15.133:0/2862424326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c54004810 msgr2=0x7f2c54004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c54004810 0x7f2c54004c90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800c210 tx=0x7f2c5801cb80 comp rx=0 tx=0).stop 2024-04-25T10:05:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 -- 172.21.15.133:0/2862424326 shutdown_connections 2024-04-25T10:05:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 --2- 172.21.15.133:0/2862424326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c54004810 0x7f2c54004c90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 -- 172.21.15.133:0/2862424326 >> 172.21.15.133:0/2862424326 conn(0x7f2c68067000 msgr2=0x7f2c6806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 -- 172.21.15.133:0/2862424326 shutdown_connections 2024-04-25T10:05:40.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 -- 172.21.15.133:0/2862424326 wait complete. 2024-04-25T10:05:40.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 Processor -- start 2024-04-25T10:05:40.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 -- start start 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c54004810 0x7f2c68142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c68144470 0x7f2c68142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c68142ec0 0x7f2c681564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68162080 con 0x7f2c68142ec0 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68161f00 con 0x7f2c68144470 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c7072b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68162200 con 0x7f2c54004810 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6ecc8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c68142ec0 0x7f2c681564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6ecc8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c68142ec0 0x7f2c681564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43572/0 (socket says 172.21.15.133:43572) 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6ecc8640 1 -- 172.21.15.133:0/3805450535 learned_addr learned my addr 172.21.15.133:0/3805450535 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6dcc6640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c68144470 0x7f2c68142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6ecc8640 1 -- 172.21.15.133:0/3805450535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c54004810 msgr2=0x7f2c68142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6ecc8640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c54004810 0x7f2c68142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6ecc8640 1 -- 172.21.15.133:0/3805450535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c68144470 msgr2=0x7f2c68142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6ecc8640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c68144470 0x7f2c68142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6ecc8640 1 -- 172.21.15.133:0/3805450535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c681433d0 con 0x7f2c68142ec0 2024-04-25T10:05:40.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c6ecc8640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c68142ec0 0x7f2c681564d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009fd0 tx=0x7f2c6000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c5effd640 1 -- 172.21.15.133:0/3805450535 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60016620 con 0x7f2c68142ec0 2024-04-25T10:05:40.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c5effd640 1 -- 172.21.15.133:0/3805450535 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6001b070 con 0x7f2c68142ec0 2024-04-25T10:05:40.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.750+0000 7f2c5effd640 1 -- 172.21.15.133:0/3805450535 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60016a20 con 0x7f2c68142ec0 2024-04-25T10:05:40.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.754+0000 7f2c7072b640 1 -- 172.21.15.133:0/3805450535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c681436b0 con 0x7f2c68142ec0 2024-04-25T10:05:40.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.754+0000 7f2c7072b640 1 -- 172.21.15.133:0/3805450535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c68143c70 con 0x7f2c68142ec0 2024-04-25T10:05:40.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.754+0000 7f2c7072b640 1 -- 172.21.15.133:0/3805450535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c68068ef0 con 0x7f2c68142ec0 2024-04-25T10:05:40.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.754+0000 7f2c5effd640 1 -- 172.21.15.133:0/3805450535 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c60016bc0 con 0x7f2c68142ec0 2024-04-25T10:05:40.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.754+0000 7f2c5effd640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c4c066f60 0x7f2c4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.754+0000 7f2c5effd640 1 -- 172.21.15.133:0/3805450535 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2c60085df0 con 0x7f2c68142ec0 2024-04-25T10:05:40.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.754+0000 7f2c6e4c7640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c4c066f60 0x7f2c4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.754+0000 7f2c6e4c7640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c4c066f60 0x7f2c4c069420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2c6804f450 tx=0x7f2c64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.758+0000 7f2c5effd640 1 -- 172.21.15.133:0/3805450535 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c60025020 con 0x7f2c68142ec0 2024-04-25T10:05:40.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.910+0000 7f2c7072b640 1 -- 172.21.15.133:0/3805450535 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c6806e1a0 con 0x7f2c4c066f60 2024-04-25T10:05:40.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.922+0000 7f2c5effd640 1 -- 172.21.15.133:0/3805450535 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250102 (secure 0 0 0) 0x7f2c6806e1a0 con 0x7f2c4c066f60 2024-04-25T10:05:40.927 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:40.928 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:05:40.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.922+0000 7f2c7072b640 1 -- 172.21.15.133:0/3805450535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c4c066f60 msgr2=0x7f2c4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.922+0000 7f2c7072b640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c4c066f60 0x7f2c4c069420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2c6804f450 tx=0x7f2c64009040 comp rx=0 tx=0).stop 2024-04-25T10:05:40.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.922+0000 7f2c7072b640 1 -- 172.21.15.133:0/3805450535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c68142ec0 msgr2=0x7f2c681564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.922+0000 7f2c7072b640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c68142ec0 0x7f2c681564d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009fd0 tx=0x7f2c6000f520 comp rx=0 tx=0).stop 2024-04-25T10:05:40.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.922+0000 7f2c7072b640 1 -- 172.21.15.133:0/3805450535 shutdown_connections 2024-04-25T10:05:40.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.922+0000 7f2c7072b640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c4c066f60 0x7f2c4c069420 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.922+0000 7f2c7072b640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c68142ec0 0x7f2c681564d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.922+0000 7f2c7072b640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2c68144470 0x7f2c68142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.922+0000 7f2c7072b640 1 --2- 172.21.15.133:0/3805450535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2c54004810 0x7f2c68142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.922+0000 7f2c7072b640 1 -- 172.21.15.133:0/3805450535 >> 172.21.15.133:0/3805450535 conn(0x7f2c68067000 msgr2=0x7f2c680cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.926+0000 7f2c7072b640 1 -- 172.21.15.133:0/3805450535 shutdown_connections 2024-04-25T10:05:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:40.926+0000 7f2c7072b640 1 -- 172.21.15.133:0/3805450535 wait complete. 2024-04-25T10:05:40.944 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-04-25T10:05:39.776113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":347252,"kb_used_data":6192,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754627468,"statfs":{"total":773094113280,"available":772738527232,"internally_reserved":0,"allocated":6340608,"data_stored":3471849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":349097521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.619070"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193794+0000","last_change":"2024-04-25T10:05:37.228346+0000","last_active":"2024-04-25T10:05:38.193794+0000","last_peered":"2024-04-25T10:05:38.193794+0000","last_clean":"2024-04-25T10:05:38.193794+0000","last_became_active":"2024-04-25T10:05:37.228062+0000","last_became_peered":"2024-04-25T10:05:37.228062+0000","last_unstale":"2024-04-25T10:05:38.193794+0000","last_undegraded":"2024-04-25T10:05:38.193794+0000","last_fullsized":"2024-04-25T10:05:38.193794+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.807237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551210+0000","last_change":"2024-04-25T10:05:37.244108+0000","last_active":"2024-04-25T10:05:38.551210+0000","last_peered":"2024-04-25T10:05:38.551210+0000","last_clean":"2024-04-25T10:05:38.551210+0000","last_became_active":"2024-04-25T10:05:37.243540+0000","last_became_peered":"2024-04-25T10:05:37.243540+0000","last_unstale":"2024-04-25T10:05:38.551210+0000","last_undegraded":"2024-04-25T10:05:38.551210+0000","last_fullsized":"2024-04-25T10:05:38.551210+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:57.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553820+0000","last_change":"2024-04-25T10:05:37.215389+0000","last_active":"2024-04-25T10:05:38.553820+0000","last_peered":"2024-04-25T10:05:38.553820+0000","last_clean":"2024-04-25T10:05:38.553820+0000","last_became_active":"2024-04-25T10:05:37.214066+0000","last_became_peered":"2024-04-25T10:05:37.214066+0000","last_unstale":"2024-04-25T10:05:38.553820+0000","last_undegraded":"2024-04-25T10:05:38.553820+0000","last_fullsized":"2024-04-25T10:05:38.553820+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:16.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550681+0000","last_change":"2024-04-25T10:05:37.243719+0000","last_active":"2024-04-25T10:05:38.550681+0000","last_peered":"2024-04-25T10:05:38.550681+0000","last_clean":"2024-04-25T10:05:38.550681+0000","last_became_active":"2024-04-25T10:05:37.242856+0000","last_became_peered":"2024-04-25T10:05:37.242856+0000","last_unstale":"2024-04-25T10:05:38.550681+0000","last_undegraded":"2024-04-25T10:05:38.550681+0000","last_fullsized":"2024-04-25T10:05:38.550681+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549470+0000","last_change":"2024-04-25T10:05:37.241776+0000","last_active":"2024-04-25T10:05:38.549470+0000","last_peered":"2024-04-25T10:05:38.549470+0000","last_clean":"2024-04-25T10:05:38.549470+0000","last_became_active":"2024-04-25T10:05:37.240967+0000","last_became_peered":"2024-04-25T10:05:37.240967+0000","last_unstale":"2024-04-25T10:05:38.549470+0000","last_undegraded":"2024-04-25T10:05:38.549470+0000","last_fullsized":"2024-04-25T10:05:38.549470+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.790321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193759+0000","last_change":"2024-04-25T10:05:37.228488+0000","last_active":"2024-04-25T10:05:38.193759+0000","last_peered":"2024-04-25T10:05:38.193759+0000","last_clean":"2024-04-25T10:05:38.193759+0000","last_became_active":"2024-04-25T10:05:37.228212+0000","last_became_peered":"2024-04-25T10:05:37.228212+0000","last_unstale":"2024-04-25T10:05:38.193759+0000","last_undegraded":"2024-04-25T10:05:38.193759+0000","last_fullsized":"2024-04-25T10:05:38.193759+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:40.538309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551627+0000","last_change":"2024-04-25T10:05:37.216471+0000","last_active":"2024-04-25T10:05:38.551627+0000","last_peered":"2024-04-25T10:05:38.551627+0000","last_clean":"2024-04-25T10:05:38.551627+0000","last_became_active":"2024-04-25T10:05:37.216024+0000","last_became_peered":"2024-04-25T10:05:37.216024+0000","last_unstale":"2024-04-25T10:05:38.551627+0000","last_undegraded":"2024-04-25T10:05:38.551627+0000","last_fullsized":"2024-04-25T10:05:38.551627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551113+0000","last_change":"2024-04-25T10:05:37.215547+0000","last_active":"2024-04-25T10:05:38.551113+0000","last_peered":"2024-04-25T10:05:38.551113+0000","last_clean":"2024-04-25T10:05:38.551113+0000","last_became_active":"2024-04-25T10:05:37.215035+0000","last_became_peered":"2024-04-25T10:05:37.215035+0000","last_unstale":"2024-04-25T10:05:38.551113+0000","last_undegraded":"2024-04-25T10:05:38.551113+0000","last_fullsized":"2024-04-25T10:05:38.551113+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:02.441654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551909+0000","last_change":"2024-04-25T10:05:37.219302+0000","last_active":"2024-04-25T10:05:38.551909+0000","last_peered":"2024-04-25T10:05:38.551909+0000","last_clean":"2024-04-25T10:05:38.551909+0000","last_became_active":"2024-04-25T10:05:37.218837+0000","last_became_peered":"2024-04-25T10:05:37.218837+0000","last_unstale":"2024-04-25T10:05:38.551909+0000","last_undegraded":"2024-04-25T10:05:38.551909+0000","last_fullsized":"2024-04-25T10:05:38.551909+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:22.308378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193584+0000","last_change":"2024-04-25T10:05:37.230769+0000","last_active":"2024-04-25T10:05:38.193584+0000","last_peered":"2024-04-25T10:05:38.193584+0000","last_clean":"2024-04-25T10:05:38.193584+0000","last_became_active":"2024-04-25T10:05:37.229183+0000","last_became_peered":"2024-04-25T10:05:37.229183+0000","last_unstale":"2024-04-25T10:05:38.193584+0000","last_undegraded":"2024-04-25T10:05:38.193584+0000","last_fullsized":"2024-04-25T10:05:38.193584+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:50.039535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.555020+0000","last_change":"2024-04-25T10:05:37.241984+0000","last_active":"2024-04-25T10:05:38.555020+0000","last_peered":"2024-04-25T10:05:38.555020+0000","last_clean":"2024-04-25T10:05:38.555020+0000","last_became_active":"2024-04-25T10:05:37.241700+0000","last_became_peered":"2024-04-25T10:05:37.241700+0000","last_unstale":"2024-04-25T10:05:38.555020+0000","last_undegraded":"2024-04-25T10:05:38.555020+0000","last_fullsized":"2024-04-25T10:05:38.555020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:02.887132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551320+0000","last_change":"2024-04-25T10:05:37.240777+0000","last_active":"2024-04-25T10:05:38.551320+0000","last_peered":"2024-04-25T10:05:38.551320+0000","last_clean":"2024-04-25T10:05:38.551320+0000","last_became_active":"2024-04-25T10:05:37.240486+0000","last_became_peered":"2024-04-25T10:05:37.240486+0000","last_unstale":"2024-04-25T10:05:38.551320+0000","last_undegraded":"2024-04-25T10:05:38.551320+0000","last_fullsized":"2024-04-25T10:05:38.551320+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:05.759734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194117+0000","last_change":"2024-04-25T10:05:37.228825+0000","last_active":"2024-04-25T10:05:38.194117+0000","last_peered":"2024-04-25T10:05:38.194117+0000","last_clean":"2024-04-25T10:05:38.194117+0000","last_became_active":"2024-04-25T10:05:37.228567+0000","last_became_peered":"2024-04-25T10:05:37.228567+0000","last_unstale":"2024-04-25T10:05:38.194117+0000","last_undegraded":"2024-04-25T10:05:38.194117+0000","last_fullsized":"2024-04-25T10:05:38.194117+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:42.164656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552001+0000","last_change":"2024-04-25T10:05:37.216195+0000","last_active":"2024-04-25T10:05:38.552001+0000","last_peered":"2024-04-25T10:05:38.552001+0000","last_clean":"2024-04-25T10:05:38.552001+0000","last_became_active":"2024-04-25T10:05:37.215910+0000","last_became_peered":"2024-04-25T10:05:37.215910+0000","last_unstale":"2024-04-25T10:05:38.552001+0000","last_undegraded":"2024-04-25T10:05:38.552001+0000","last_fullsized":"2024-04-25T10:05:38.552001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549428+0000","last_change":"2024-04-25T10:05:37.244630+0000","last_active":"2024-04-25T10:05:38.549428+0000","last_peered":"2024-04-25T10:05:38.549428+0000","last_clean":"2024-04-25T10:05:38.549428+0000","last_became_active":"2024-04-25T10:05:37.244120+0000","last_became_peered":"2024-04-25T10:05:37.244120+0000","last_unstale":"2024-04-25T10:05:38.549428+0000","last_undegraded":"2024-04-25T10:05:38.549428+0000","last_fullsized":"2024-04-25T10:05:38.549428+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193338+0000","last_change":"2024-04-25T10:05:37.229142+0000","last_active":"2024-04-25T10:05:38.193338+0000","last_peered":"2024-04-25T10:05:38.193338+0000","last_clean":"2024-04-25T10:05:38.193338+0000","last_became_active":"2024-04-25T10:05:37.228852+0000","last_became_peered":"2024-04-25T10:05:37.228852+0000","last_unstale":"2024-04-25T10:05:38.193338+0000","last_undegraded":"2024-04-25T10:05:38.193338+0000","last_fullsized":"2024-04-25T10:05:38.193338+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:56.021983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552832+0000","last_change":"2024-04-25T10:05:37.219710+0000","last_active":"2024-04-25T10:05:38.552832+0000","last_peered":"2024-04-25T10:05:38.552832+0000","last_clean":"2024-04-25T10:05:38.552832+0000","last_became_active":"2024-04-25T10:05:37.218987+0000","last_became_peered":"2024-04-25T10:05:37.218987+0000","last_unstale":"2024-04-25T10:05:38.552832+0000","last_undegraded":"2024-04-25T10:05:38.552832+0000","last_fullsized":"2024-04-25T10:05:38.552832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551559+0000","last_change":"2024-04-25T10:05:37.244343+0000","last_active":"2024-04-25T10:05:38.551559+0000","last_peered":"2024-04-25T10:05:38.551559+0000","last_clean":"2024-04-25T10:05:38.551559+0000","last_became_active":"2024-04-25T10:05:37.243655+0000","last_became_peered":"2024-04-25T10:05:37.243655+0000","last_unstale":"2024-04-25T10:05:38.551559+0000","last_undegraded":"2024-04-25T10:05:38.551559+0000","last_fullsized":"2024-04-25T10:05:38.551559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554321+0000","last_change":"2024-04-25T10:05:37.222335+0000","last_active":"2024-04-25T10:05:38.554321+0000","last_peered":"2024-04-25T10:05:38.554321+0000","last_clean":"2024-04-25T10:05:38.554321+0000","last_became_active":"2024-04-25T10:05:37.220912+0000","last_became_peered":"2024-04-25T10:05:37.220912+0000","last_unstale":"2024-04-25T10:05:38.554321+0000","last_undegraded":"2024-04-25T10:05:38.554321+0000","last_fullsized":"2024-04-25T10:05:38.554321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:02.441654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550628+0000","last_change":"2024-04-25T10:05:37.212566+0000","last_active":"2024-04-25T10:05:38.550628+0000","last_peered":"2024-04-25T10:05:38.550628+0000","last_clean":"2024-04-25T10:05:38.550628+0000","last_became_active":"2024-04-25T10:05:37.211952+0000","last_became_peered":"2024-04-25T10:05:37.211952+0000","last_unstale":"2024-04-25T10:05:38.550628+0000","last_undegraded":"2024-04-25T10:05:38.550628+0000","last_fullsized":"2024-04-25T10:05:38.550628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:00.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554521+0000","last_change":"2024-04-25T10:05:37.238454+0000","last_active":"2024-04-25T10:05:38.554521+0000","last_peered":"2024-04-25T10:05:38.554521+0000","last_clean":"2024-04-25T10:05:38.554521+0000","last_became_active":"2024-04-25T10:05:37.238147+0000","last_became_peered":"2024-04-25T10:05:37.238147+0000","last_unstale":"2024-04-25T10:05:38.554521+0000","last_undegraded":"2024-04-25T10:05:38.554521+0000","last_fullsized":"2024-04-25T10:05:38.554521+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:10.077596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550991+0000","last_change":"2024-04-25T10:05:37.216233+0000","last_active":"2024-04-25T10:05:38.550991+0000","last_peered":"2024-04-25T10:05:38.550991+0000","last_clean":"2024-04-25T10:05:38.550991+0000","last_became_active":"2024-04-25T10:05:37.214054+0000","last_became_peered":"2024-04-25T10:05:37.214054+0000","last_unstale":"2024-04-25T10:05:38.550991+0000","last_undegraded":"2024-04-25T10:05:38.550991+0000","last_fullsized":"2024-04-25T10:05:38.550991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:30.598917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553345+0000","last_change":"2024-04-25T10:05:37.219854+0000","last_active":"2024-04-25T10:05:38.553345+0000","last_peered":"2024-04-25T10:05:38.553345+0000","last_clean":"2024-04-25T10:05:38.553345+0000","last_became_active":"2024-04-25T10:05:37.219447+0000","last_became_peered":"2024-04-25T10:05:37.219447+0000","last_unstale":"2024-04-25T10:05:38.553345+0000","last_undegraded":"2024-04-25T10:05:38.553345+0000","last_fullsized":"2024-04-25T10:05:38.553345+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:00.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551069+0000","last_change":"2024-04-25T10:05:37.203182+0000","last_active":"2024-04-25T10:05:38.551069+0000","last_peered":"2024-04-25T10:05:38.551069+0000","last_clean":"2024-04-25T10:05:38.551069+0000","last_became_active":"2024-04-25T10:05:37.202893+0000","last_became_peered":"2024-04-25T10:05:37.202893+0000","last_unstale":"2024-04-25T10:05:38.551069+0000","last_undegraded":"2024-04-25T10:05:38.551069+0000","last_fullsized":"2024-04-25T10:05:38.551069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:12.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194246+0000","last_change":"2024-04-25T10:05:37.205817+0000","last_active":"2024-04-25T10:05:38.194246+0000","last_peered":"2024-04-25T10:05:38.194246+0000","last_clean":"2024-04-25T10:05:38.194246+0000","last_became_active":"2024-04-25T10:05:37.205572+0000","last_became_peered":"2024-04-25T10:05:37.205572+0000","last_unstale":"2024-04-25T10:05:38.194246+0000","last_undegraded":"2024-04-25T10:05:38.194246+0000","last_fullsized":"2024-04-25T10:05:38.194246+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:36.414900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553104+0000","last_change":"2024-04-25T10:05:37.240878+0000","last_active":"2024-04-25T10:05:38.553104+0000","last_peered":"2024-04-25T10:05:38.553104+0000","last_clean":"2024-04-25T10:05:38.553104+0000","last_became_active":"2024-04-25T10:05:37.240620+0000","last_became_peered":"2024-04-25T10:05:37.240620+0000","last_unstale":"2024-04-25T10:05:38.553104+0000","last_undegraded":"2024-04-25T10:05:38.553104+0000","last_fullsized":"2024-04-25T10:05:38.553104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550361+0000","last_change":"2024-04-25T10:05:37.246366+0000","last_active":"2024-04-25T10:05:38.550361+0000","last_peered":"2024-04-25T10:05:38.550361+0000","last_clean":"2024-04-25T10:05:38.550361+0000","last_became_active":"2024-04-25T10:05:37.239714+0000","last_became_peered":"2024-04-25T10:05:37.239714+0000","last_unstale":"2024-04-25T10:05:38.550361+0000","last_undegraded":"2024-04-25T10:05:38.550361+0000","last_fullsized":"2024-04-25T10:05:38.550361+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552009+0000","last_change":"2024-04-25T10:05:37.216294+0000","last_active":"2024-04-25T10:05:38.552009+0000","last_peered":"2024-04-25T10:05:38.552009+0000","last_clean":"2024-04-25T10:05:38.552009+0000","last_became_active":"2024-04-25T10:05:37.215738+0000","last_became_peered":"2024-04-25T10:05:37.215738+0000","last_unstale":"2024-04-25T10:05:38.552009+0000","last_undegraded":"2024-04-25T10:05:38.552009+0000","last_fullsized":"2024-04-25T10:05:38.552009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191560+0000","last_change":"2024-04-25T10:05:37.221885+0000","last_active":"2024-04-25T10:05:38.191560+0000","last_peered":"2024-04-25T10:05:38.191560+0000","last_clean":"2024-04-25T10:05:38.191560+0000","last_became_active":"2024-04-25T10:05:37.219364+0000","last_became_peered":"2024-04-25T10:05:37.219364+0000","last_unstale":"2024-04-25T10:05:38.191560+0000","last_undegraded":"2024-04-25T10:05:38.191560+0000","last_fullsized":"2024-04-25T10:05:38.191560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549409+0000","last_change":"2024-04-25T10:05:37.229695+0000","last_active":"2024-04-25T10:05:38.549409+0000","last_peered":"2024-04-25T10:05:38.549409+0000","last_clean":"2024-04-25T10:05:38.549409+0000","last_became_active":"2024-04-25T10:05:37.228961+0000","last_became_peered":"2024-04-25T10:05:37.228961+0000","last_unstale":"2024-04-25T10:05:38.549409+0000","last_undegraded":"2024-04-25T10:05:38.549409+0000","last_fullsized":"2024-04-25T10:05:38.549409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191931+0000","last_change":"2024-04-25T10:05:37.230425+0000","last_active":"2024-04-25T10:05:38.191931+0000","last_peered":"2024-04-25T10:05:38.191931+0000","last_clean":"2024-04-25T10:05:38.191931+0000","last_became_active":"2024-04-25T10:05:37.229462+0000","last_became_peered":"2024-04-25T10:05:37.229462+0000","last_unstale":"2024-04-25T10:05:38.191931+0000","last_undegraded":"2024-04-25T10:05:38.191931+0000","last_fullsized":"2024-04-25T10:05:38.191931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194406+0000","last_change":"2024-04-25T10:05:36.192412+0000","last_active":"2024-04-25T10:05:38.194406+0000","last_peered":"2024-04-25T10:05:38.194406+0000","last_clean":"2024-04-25T10:05:38.194406+0000","last_became_active":"2024-04-25T10:05:36.191754+0000","last_became_peered":"2024-04-25T10:05:36.191754+0000","last_unstale":"2024-04-25T10:05:38.194406+0000","last_undegraded":"2024-04-25T10:05:38.194406+0000","last_fullsized":"2024-04-25T10:05:38.194406+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:33.604952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194384+0000","last_change":"2024-04-25T10:05:36.197170+0000","last_active":"2024-04-25T10:05:38.194384+0000","last_peered":"2024-04-25T10:05:38.194384+0000","last_clean":"2024-04-25T10:05:38.194384+0000","last_became_active":"2024-04-25T10:05:36.196555+0000","last_became_peered":"2024-04-25T10:05:36.196555+0000","last_unstale":"2024-04-25T10:05:38.194384+0000","last_undegraded":"2024-04-25T10:05:38.194384+0000","last_fullsized":"2024-04-25T10:05:38.194384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:43.120025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550712+0000","last_change":"2024-04-25T10:05:37.206507+0000","last_active":"2024-04-25T10:05:38.550712+0000","last_peered":"2024-04-25T10:05:38.550712+0000","last_clean":"2024-04-25T10:05:38.550712+0000","last_became_active":"2024-04-25T10:05:37.204792+0000","last_became_peered":"2024-04-25T10:05:37.204792+0000","last_unstale":"2024-04-25T10:05:38.550712+0000","last_undegraded":"2024-04-25T10:05:38.550712+0000","last_fullsized":"2024-04-25T10:05:38.550712+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:16.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192596+0000","last_change":"2024-04-25T10:05:36.201693+0000","last_active":"2024-04-25T10:05:38.192596+0000","last_peered":"2024-04-25T10:05:38.192596+0000","last_clean":"2024-04-25T10:05:38.192596+0000","last_became_active":"2024-04-25T10:05:36.200944+0000","last_became_peered":"2024-04-25T10:05:36.200944+0000","last_unstale":"2024-04-25T10:05:38.192596+0000","last_undegraded":"2024-04-25T10:05:38.192596+0000","last_fullsized":"2024-04-25T10:05:38.192596+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550050+0000","last_change":"2024-04-25T10:05:37.222859+0000","last_active":"2024-04-25T10:05:38.550050+0000","last_peered":"2024-04-25T10:05:38.550050+0000","last_clean":"2024-04-25T10:05:38.550050+0000","last_became_active":"2024-04-25T10:05:37.222162+0000","last_became_peered":"2024-04-25T10:05:37.222162+0000","last_unstale":"2024-04-25T10:05:38.550050+0000","last_undegraded":"2024-04-25T10:05:38.550050+0000","last_fullsized":"2024-04-25T10:05:38.550050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:44.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553382+0000","last_change":"2024-04-25T10:05:36.193782+0000","last_active":"2024-04-25T10:05:38.553382+0000","last_peered":"2024-04-25T10:05:38.553382+0000","last_clean":"2024-04-25T10:05:38.553382+0000","last_became_active":"2024-04-25T10:05:36.190455+0000","last_became_peered":"2024-04-25T10:05:36.190455+0000","last_unstale":"2024-04-25T10:05:38.553382+0000","last_undegraded":"2024-04-25T10:05:38.553382+0000","last_fullsized":"2024-04-25T10:05:38.553382+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:32.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550845+0000","last_change":"2024-04-25T10:05:37.246868+0000","last_active":"2024-04-25T10:05:38.550845+0000","last_peered":"2024-04-25T10:05:38.550845+0000","last_clean":"2024-04-25T10:05:38.550845+0000","last_became_active":"2024-04-25T10:05:37.246095+0000","last_became_peered":"2024-04-25T10:05:37.246095+0000","last_unstale":"2024-04-25T10:05:38.550845+0000","last_undegraded":"2024-04-25T10:05:38.550845+0000","last_fullsized":"2024-04-25T10:05:38.550845+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548980+0000","last_change":"2024-04-25T10:05:36.197860+0000","last_active":"2024-04-25T10:05:38.548980+0000","last_peered":"2024-04-25T10:05:38.548980+0000","last_clean":"2024-04-25T10:05:38.548980+0000","last_became_active":"2024-04-25T10:05:36.197137+0000","last_became_peered":"2024-04-25T10:05:36.197137+0000","last_unstale":"2024-04-25T10:05:38.548980+0000","last_undegraded":"2024-04-25T10:05:38.548980+0000","last_fullsized":"2024-04-25T10:05:38.548980+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:30.726291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551788+0000","last_change":"2024-04-25T10:05:37.206314+0000","last_active":"2024-04-25T10:05:38.551788+0000","last_peered":"2024-04-25T10:05:38.551788+0000","last_clean":"2024-04-25T10:05:38.551788+0000","last_became_active":"2024-04-25T10:05:37.205748+0000","last_became_peered":"2024-04-25T10:05:37.205748+0000","last_unstale":"2024-04-25T10:05:38.551788+0000","last_undegraded":"2024-04-25T10:05:38.551788+0000","last_fullsized":"2024-04-25T10:05:38.551788+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:52.894883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550829+0000","last_change":"2024-04-25T10:05:37.240137+0000","last_active":"2024-04-25T10:05:38.550829+0000","last_peered":"2024-04-25T10:05:38.550829+0000","last_clean":"2024-04-25T10:05:38.550829+0000","last_became_active":"2024-04-25T10:05:37.239387+0000","last_became_peered":"2024-04-25T10:05:37.239387+0000","last_unstale":"2024-04-25T10:05:38.550829+0000","last_undegraded":"2024-04-25T10:05:38.550829+0000","last_fullsized":"2024-04-25T10:05:38.550829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551084+0000","last_change":"2024-04-25T10:05:36.197600+0000","last_active":"2024-04-25T10:05:38.551084+0000","last_peered":"2024-04-25T10:05:38.551084+0000","last_clean":"2024-04-25T10:05:38.551084+0000","last_became_active":"2024-04-25T10:05:36.197265+0000","last_became_peered":"2024-04-25T10:05:36.197265+0000","last_unstale":"2024-04-25T10:05:38.551084+0000","last_undegraded":"2024-04-25T10:05:38.551084+0000","last_fullsized":"2024-04-25T10:05:38.551084+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:24.137684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548851+0000","last_change":"2024-04-25T10:05:37.224800+0000","last_active":"2024-04-25T10:05:38.548851+0000","last_peered":"2024-04-25T10:05:38.548851+0000","last_clean":"2024-04-25T10:05:38.548851+0000","last_became_active":"2024-04-25T10:05:37.224068+0000","last_became_peered":"2024-04-25T10:05:37.224068+0000","last_unstale":"2024-04-25T10:05:38.548851+0000","last_undegraded":"2024-04-25T10:05:38.548851+0000","last_fullsized":"2024-04-25T10:05:38.548851+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:25.148179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551076+0000","last_change":"2024-04-25T10:05:36.195068+0000","last_active":"2024-04-25T10:05:38.551076+0000","last_peered":"2024-04-25T10:05:38.551076+0000","last_clean":"2024-04-25T10:05:38.551076+0000","last_became_active":"2024-04-25T10:05:36.194281+0000","last_became_peered":"2024-04-25T10:05:36.194281+0000","last_unstale":"2024-04-25T10:05:38.551076+0000","last_undegraded":"2024-04-25T10:05:38.551076+0000","last_fullsized":"2024-04-25T10:05:38.551076+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:41.154160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191829+0000","last_change":"2024-04-25T10:05:37.223331+0000","last_active":"2024-04-25T10:05:38.191829+0000","last_peered":"2024-04-25T10:05:38.191829+0000","last_clean":"2024-04-25T10:05:38.191829+0000","last_became_active":"2024-04-25T10:05:37.222613+0000","last_became_peered":"2024-04-25T10:05:37.222613+0000","last_unstale":"2024-04-25T10:05:38.191829+0000","last_undegraded":"2024-04-25T10:05:38.191829+0000","last_fullsized":"2024-04-25T10:05:38.191829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549616+0000","last_change":"2024-04-25T10:05:36.200167+0000","last_active":"2024-04-25T10:05:38.549616+0000","last_peered":"2024-04-25T10:05:38.549616+0000","last_clean":"2024-04-25T10:05:38.549616+0000","last_became_active":"2024-04-25T10:05:36.199467+0000","last_became_peered":"2024-04-25T10:05:36.199467+0000","last_unstale":"2024-04-25T10:05:38.549616+0000","last_undegraded":"2024-04-25T10:05:38.549616+0000","last_fullsized":"2024-04-25T10:05:38.549616+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:47.830431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552373+0000","last_change":"2024-04-25T10:05:37.241002+0000","last_active":"2024-04-25T10:05:38.552373+0000","last_peered":"2024-04-25T10:05:38.552373+0000","last_clean":"2024-04-25T10:05:38.552373+0000","last_became_active":"2024-04-25T10:05:37.240740+0000","last_became_peered":"2024-04-25T10:05:37.240740+0000","last_unstale":"2024-04-25T10:05:38.552373+0000","last_undegraded":"2024-04-25T10:05:38.552373+0000","last_fullsized":"2024-04-25T10:05:38.552373+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551231+0000","last_change":"2024-04-25T10:05:36.195424+0000","last_active":"2024-04-25T10:05:38.551231+0000","last_peered":"2024-04-25T10:05:38.551231+0000","last_clean":"2024-04-25T10:05:38.551231+0000","last_became_active":"2024-04-25T10:05:36.193306+0000","last_became_peered":"2024-04-25T10:05:36.193306+0000","last_unstale":"2024-04-25T10:05:38.551231+0000","last_undegraded":"2024-04-25T10:05:38.551231+0000","last_fullsized":"2024-04-25T10:05:38.551231+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549522+0000","last_change":"2024-04-25T10:05:37.220528+0000","last_active":"2024-04-25T10:05:38.549522+0000","last_peered":"2024-04-25T10:05:38.549522+0000","last_clean":"2024-04-25T10:05:38.549522+0000","last_became_active":"2024-04-25T10:05:37.219586+0000","last_became_peered":"2024-04-25T10:05:37.219586+0000","last_unstale":"2024-04-25T10:05:38.549522+0000","last_undegraded":"2024-04-25T10:05:38.549522+0000","last_fullsized":"2024-04-25T10:05:38.549522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:55.011916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551578+0000","last_change":"2024-04-25T10:05:36.199745+0000","last_active":"2024-04-25T10:05:38.551578+0000","last_peered":"2024-04-25T10:05:38.551578+0000","last_clean":"2024-04-25T10:05:38.551578+0000","last_became_active":"2024-04-25T10:05:36.191871+0000","last_became_peered":"2024-04-25T10:05:36.191871+0000","last_unstale":"2024-04-25T10:05:38.551578+0000","last_undegraded":"2024-04-25T10:05:38.551578+0000","last_fullsized":"2024-04-25T10:05:38.551578+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191388+0000","last_change":"2024-04-25T10:05:37.230868+0000","last_active":"2024-04-25T10:05:38.191388+0000","last_peered":"2024-04-25T10:05:38.191388+0000","last_clean":"2024-04-25T10:05:38.191388+0000","last_became_active":"2024-04-25T10:05:37.229549+0000","last_became_peered":"2024-04-25T10:05:37.229549+0000","last_unstale":"2024-04-25T10:05:38.191388+0000","last_undegraded":"2024-04-25T10:05:38.191388+0000","last_fullsized":"2024-04-25T10:05:38.191388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:04.305032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553620+0000","last_change":"2024-04-25T10:05:36.201220+0000","last_active":"2024-04-25T10:05:38.553620+0000","last_peered":"2024-04-25T10:05:38.553620+0000","last_clean":"2024-04-25T10:05:38.553620+0000","last_became_active":"2024-04-25T10:05:36.200820+0000","last_became_peered":"2024-04-25T10:05:36.200820+0000","last_unstale":"2024-04-25T10:05:38.553620+0000","last_undegraded":"2024-04-25T10:05:38.553620+0000","last_fullsized":"2024-04-25T10:05:38.553620+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:54.001421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554271+0000","last_change":"2024-04-25T10:05:37.215939+0000","last_active":"2024-04-25T10:05:38.554271+0000","last_peered":"2024-04-25T10:05:38.554271+0000","last_clean":"2024-04-25T10:05:38.554271+0000","last_became_active":"2024-04-25T10:05:37.215693+0000","last_became_peered":"2024-04-25T10:05:37.215693+0000","last_unstale":"2024-04-25T10:05:38.554271+0000","last_undegraded":"2024-04-25T10:05:38.554271+0000","last_fullsized":"2024-04-25T10:05:38.554271+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:12.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552870+0000","last_change":"2024-04-25T10:05:37.238447+0000","last_active":"2024-04-25T10:05:38.552870+0000","last_peered":"2024-04-25T10:05:38.552870+0000","last_clean":"2024-04-25T10:05:38.552870+0000","last_became_active":"2024-04-25T10:05:37.238106+0000","last_became_peered":"2024-04-25T10:05:37.238106+0000","last_unstale":"2024-04-25T10:05:38.552870+0000","last_undegraded":"2024-04-25T10:05:38.552870+0000","last_fullsized":"2024-04-25T10:05:38.552870+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:06.779179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549834+0000","last_change":"2024-04-25T10:05:36.198280+0000","last_active":"2024-04-25T10:05:38.549834+0000","last_peered":"2024-04-25T10:05:38.549834+0000","last_clean":"2024-04-25T10:05:38.549834+0000","last_became_active":"2024-04-25T10:05:36.197484+0000","last_became_peered":"2024-04-25T10:05:36.197484+0000","last_unstale":"2024-04-25T10:05:38.549834+0000","last_undegraded":"2024-04-25T10:05:38.549834+0000","last_fullsized":"2024-04-25T10:05:38.549834+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.796742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552446+0000","last_change":"2024-04-25T10:05:37.219004+0000","last_active":"2024-04-25T10:05:38.552446+0000","last_peered":"2024-04-25T10:05:38.552446+0000","last_clean":"2024-04-25T10:05:38.552446+0000","last_became_active":"2024-04-25T10:05:37.216939+0000","last_became_peered":"2024-04-25T10:05:37.216939+0000","last_unstale":"2024-04-25T10:05:38.552446+0000","last_undegraded":"2024-04-25T10:05:38.552446+0000","last_fullsized":"2024-04-25T10:05:38.552446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551470+0000","last_change":"2024-04-25T10:05:36.198305+0000","last_active":"2024-04-25T10:05:38.551470+0000","last_peered":"2024-04-25T10:05:38.551470+0000","last_clean":"2024-04-25T10:05:38.551470+0000","last_became_active":"2024-04-25T10:05:36.191908+0000","last_became_peered":"2024-04-25T10:05:36.191908+0000","last_unstale":"2024-04-25T10:05:38.551470+0000","last_undegraded":"2024-04-25T10:05:38.551470+0000","last_fullsized":"2024-04-25T10:05:38.551470+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:52.819131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553781+0000","last_change":"2024-04-25T10:05:37.224631+0000","last_active":"2024-04-25T10:05:38.553781+0000","last_peered":"2024-04-25T10:05:38.553781+0000","last_clean":"2024-04-25T10:05:38.553781+0000","last_became_active":"2024-04-25T10:05:37.224014+0000","last_became_peered":"2024-04-25T10:05:37.224014+0000","last_unstale":"2024-04-25T10:05:38.553781+0000","last_undegraded":"2024-04-25T10:05:38.553781+0000","last_fullsized":"2024-04-25T10:05:38.553781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:08.906935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553276+0000","last_change":"2024-04-25T10:05:36.203863+0000","last_active":"2024-04-25T10:05:38.553276+0000","last_peered":"2024-04-25T10:05:38.553276+0000","last_clean":"2024-04-25T10:05:38.553276+0000","last_became_active":"2024-04-25T10:05:36.203473+0000","last_became_peered":"2024-04-25T10:05:36.203473+0000","last_unstale":"2024-04-25T10:05:38.553276+0000","last_undegraded":"2024-04-25T10:05:38.553276+0000","last_fullsized":"2024-04-25T10:05:38.553276+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:24.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193575+0000","last_change":"2024-04-25T10:05:37.228514+0000","last_active":"2024-04-25T10:05:38.193575+0000","last_peered":"2024-04-25T10:05:38.193575+0000","last_clean":"2024-04-25T10:05:38.193575+0000","last_became_active":"2024-04-25T10:05:37.213429+0000","last_became_peered":"2024-04-25T10:05:37.213429+0000","last_unstale":"2024-04-25T10:05:38.193575+0000","last_undegraded":"2024-04-25T10:05:38.193575+0000","last_fullsized":"2024-04-25T10:05:38.193575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:07.257110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.555115+0000","last_change":"2024-04-25T10:05:36.192160+0000","last_active":"2024-04-25T10:05:38.555115+0000","last_peered":"2024-04-25T10:05:38.555115+0000","last_clean":"2024-04-25T10:05:38.555115+0000","last_became_active":"2024-04-25T10:05:36.191475+0000","last_became_peered":"2024-04-25T10:05:36.191475+0000","last_unstale":"2024-04-25T10:05:38.555115+0000","last_undegraded":"2024-04-25T10:05:38.555115+0000","last_fullsized":"2024-04-25T10:05:38.555115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:06.047245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548936+0000","last_change":"2024-04-25T10:05:37.218759+0000","last_active":"2024-04-25T10:05:38.548936+0000","last_peered":"2024-04-25T10:05:38.548936+0000","last_clean":"2024-04-25T10:05:38.548936+0000","last_became_active":"2024-04-25T10:05:37.217992+0000","last_became_peered":"2024-04-25T10:05:37.217992+0000","last_unstale":"2024-04-25T10:05:38.548936+0000","last_undegraded":"2024-04-25T10:05:38.548936+0000","last_fullsized":"2024-04-25T10:05:38.548936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:59.463557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194500+0000","last_change":"2024-04-25T10:05:37.227960+0000","last_active":"2024-04-25T10:05:38.194500+0000","last_peered":"2024-04-25T10:05:38.194500+0000","last_clean":"2024-04-25T10:05:38.194500+0000","last_became_active":"2024-04-25T10:05:37.213414+0000","last_became_peered":"2024-04-25T10:05:37.213414+0000","last_unstale":"2024-04-25T10:05:38.194500+0000","last_undegraded":"2024-04-25T10:05:38.194500+0000","last_fullsized":"2024-04-25T10:05:38.194500+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:01.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191893+0000","last_change":"2024-04-25T10:05:37.224756+0000","last_active":"2024-04-25T10:05:38.191893+0000","last_peered":"2024-04-25T10:05:38.191893+0000","last_clean":"2024-04-25T10:05:38.191893+0000","last_became_active":"2024-04-25T10:05:37.223895+0000","last_became_peered":"2024-04-25T10:05:37.223895+0000","last_unstale":"2024-04-25T10:05:38.191893+0000","last_undegraded":"2024-04-25T10:05:38.191893+0000","last_fullsized":"2024-04-25T10:05:38.191893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194604+0000","last_change":"2024-04-25T10:05:36.186378+0000","last_active":"2024-04-25T10:05:38.194604+0000","last_peered":"2024-04-25T10:05:38.194604+0000","last_clean":"2024-04-25T10:05:38.194604+0000","last_became_active":"2024-04-25T10:05:36.185733+0000","last_became_peered":"2024-04-25T10:05:36.185733+0000","last_unstale":"2024-04-25T10:05:38.194604+0000","last_undegraded":"2024-04-25T10:05:38.194604+0000","last_fullsized":"2024-04-25T10:05:38.194604+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:43.722766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551744+0000","last_change":"2024-04-25T10:05:36.199741+0000","last_active":"2024-04-25T10:05:38.551744+0000","last_peered":"2024-04-25T10:05:38.551744+0000","last_clean":"2024-04-25T10:05:38.551744+0000","last_became_active":"2024-04-25T10:05:36.199311+0000","last_became_peered":"2024-04-25T10:05:36.199311+0000","last_unstale":"2024-04-25T10:05:38.551744+0000","last_undegraded":"2024-04-25T10:05:38.551744+0000","last_fullsized":"2024-04-25T10:05:38.551744+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551563+0000","last_change":"2024-04-25T10:05:37.240101+0000","last_active":"2024-04-25T10:05:38.551563+0000","last_peered":"2024-04-25T10:05:38.551563+0000","last_clean":"2024-04-25T10:05:38.551563+0000","last_became_active":"2024-04-25T10:05:37.239789+0000","last_became_peered":"2024-04-25T10:05:37.239789+0000","last_unstale":"2024-04-25T10:05:38.551563+0000","last_undegraded":"2024-04-25T10:05:38.551563+0000","last_fullsized":"2024-04-25T10:05:38.551563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:04.305032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552440+0000","last_change":"2024-04-25T10:05:36.194819+0000","last_active":"2024-04-25T10:05:38.552440+0000","last_peered":"2024-04-25T10:05:38.552440+0000","last_clean":"2024-04-25T10:05:38.552440+0000","last_became_active":"2024-04-25T10:05:36.194511+0000","last_became_peered":"2024-04-25T10:05:36.194511+0000","last_unstale":"2024-04-25T10:05:38.552440+0000","last_undegraded":"2024-04-25T10:05:38.552440+0000","last_fullsized":"2024-04-25T10:05:38.552440+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:43.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194192+0000","last_change":"2024-04-25T10:05:37.204849+0000","last_active":"2024-04-25T10:05:38.194192+0000","last_peered":"2024-04-25T10:05:38.194192+0000","last_clean":"2024-04-25T10:05:38.194192+0000","last_became_active":"2024-04-25T10:05:37.204310+0000","last_became_peered":"2024-04-25T10:05:37.204310+0000","last_unstale":"2024-04-25T10:05:38.194192+0000","last_undegraded":"2024-04-25T10:05:38.194192+0000","last_fullsized":"2024-04-25T10:05:38.194192+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:49.155350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554104+0000","last_change":"2024-04-25T10:05:36.189798+0000","last_active":"2024-04-25T10:05:38.554104+0000","last_peered":"2024-04-25T10:05:38.554104+0000","last_clean":"2024-04-25T10:05:38.554104+0000","last_became_active":"2024-04-25T10:05:36.189079+0000","last_became_peered":"2024-04-25T10:05:36.189079+0000","last_unstale":"2024-04-25T10:05:38.554104+0000","last_undegraded":"2024-04-25T10:05:38.554104+0000","last_fullsized":"2024-04-25T10:05:38.554104+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:06.607042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192449+0000","last_change":"2024-04-25T10:05:37.213816+0000","last_active":"2024-04-25T10:05:38.192449+0000","last_peered":"2024-04-25T10:05:38.192449+0000","last_clean":"2024-04-25T10:05:38.192449+0000","last_became_active":"2024-04-25T10:05:37.213082+0000","last_became_peered":"2024-04-25T10:05:37.213082+0000","last_unstale":"2024-04-25T10:05:38.192449+0000","last_undegraded":"2024-04-25T10:05:38.192449+0000","last_fullsized":"2024-04-25T10:05:38.192449+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193639+0000","last_change":"2024-04-25T10:05:37.199321+0000","last_active":"2024-04-25T10:05:38.193639+0000","last_peered":"2024-04-25T10:05:38.193639+0000","last_clean":"2024-04-25T10:05:38.193639+0000","last_became_active":"2024-04-25T10:05:37.199011+0000","last_became_peered":"2024-04-25T10:05:37.199011+0000","last_unstale":"2024-04-25T10:05:38.193639+0000","last_undegraded":"2024-04-25T10:05:38.193639+0000","last_fullsized":"2024-04-25T10:05:38.193639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.753836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551360+0000","last_change":"2024-04-25T10:05:36.198586+0000","last_active":"2024-04-25T10:05:38.551360+0000","last_peered":"2024-04-25T10:05:38.551360+0000","last_clean":"2024-04-25T10:05:38.551360+0000","last_became_active":"2024-04-25T10:05:36.197856+0000","last_became_peered":"2024-04-25T10:05:36.197856+0000","last_unstale":"2024-04-25T10:05:38.551360+0000","last_undegraded":"2024-04-25T10:05:38.551360+0000","last_fullsized":"2024-04-25T10:05:38.551360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550497+0000","last_change":"2024-04-25T10:05:37.225133+0000","last_active":"2024-04-25T10:05:38.550497+0000","last_peered":"2024-04-25T10:05:38.550497+0000","last_clean":"2024-04-25T10:05:38.550497+0000","last_became_active":"2024-04-25T10:05:37.224655+0000","last_became_peered":"2024-04-25T10:05:37.224655+0000","last_unstale":"2024-04-25T10:05:38.550497+0000","last_undegraded":"2024-04-25T10:05:38.550497+0000","last_fullsized":"2024-04-25T10:05:38.550497+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553244+0000","last_change":"2024-04-25T10:05:36.189704+0000","last_active":"2024-04-25T10:05:38.553244+0000","last_peered":"2024-04-25T10:05:38.553244+0000","last_clean":"2024-04-25T10:05:38.553244+0000","last_became_active":"2024-04-25T10:05:36.188969+0000","last_became_peered":"2024-04-25T10:05:36.188969+0000","last_unstale":"2024-04-25T10:05:38.553244+0000","last_undegraded":"2024-04-25T10:05:38.553244+0000","last_fullsized":"2024-04-25T10:05:38.553244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:58.453061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553005+0000","last_change":"2024-04-25T10:05:37.219793+0000","last_active":"2024-04-25T10:05:38.553005+0000","last_peered":"2024-04-25T10:05:38.553005+0000","last_clean":"2024-04-25T10:05:38.553005+0000","last_became_active":"2024-04-25T10:05:37.219521+0000","last_became_peered":"2024-04-25T10:05:37.219521+0000","last_unstale":"2024-04-25T10:05:38.553005+0000","last_undegraded":"2024-04-25T10:05:38.553005+0000","last_fullsized":"2024-04-25T10:05:38.553005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:57.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550594+0000","last_change":"2024-04-25T10:05:36.198953+0000","last_active":"2024-04-25T10:05:38.550594+0000","last_peered":"2024-04-25T10:05:38.550594+0000","last_clean":"2024-04-25T10:05:38.550594+0000","last_became_active":"2024-04-25T10:05:36.197976+0000","last_became_peered":"2024-04-25T10:05:36.197976+0000","last_unstale":"2024-04-25T10:05:38.550594+0000","last_undegraded":"2024-04-25T10:05:38.550594+0000","last_fullsized":"2024-04-25T10:05:38.550594+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:09.514786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191968+0000","last_change":"2024-04-25T10:05:37.217261+0000","last_active":"2024-04-25T10:05:38.191968+0000","last_peered":"2024-04-25T10:05:38.191968+0000","last_clean":"2024-04-25T10:05:38.191968+0000","last_became_active":"2024-04-25T10:05:37.216539+0000","last_became_peered":"2024-04-25T10:05:37.216539+0000","last_unstale":"2024-04-25T10:05:38.191968+0000","last_undegraded":"2024-04-25T10:05:38.191968+0000","last_fullsized":"2024-04-25T10:05:38.191968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551865+0000","last_change":"2024-04-25T10:05:36.191359+0000","last_active":"2024-04-25T10:05:38.551865+0000","last_peered":"2024-04-25T10:05:38.551865+0000","last_clean":"2024-04-25T10:05:38.551865+0000","last_became_active":"2024-04-25T10:05:36.190991+0000","last_became_peered":"2024-04-25T10:05:36.190991+0000","last_unstale":"2024-04-25T10:05:38.551865+0000","last_undegraded":"2024-04-25T10:05:38.551865+0000","last_fullsized":"2024-04-25T10:05:38.551865+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:00.276908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551375+0000","last_change":"2024-04-25T10:05:37.215467+0000","last_active":"2024-04-25T10:05:38.551375+0000","last_peered":"2024-04-25T10:05:38.551375+0000","last_clean":"2024-04-25T10:05:38.551375+0000","last_became_active":"2024-04-25T10:05:37.212611+0000","last_became_peered":"2024-04-25T10:05:37.212611+0000","last_unstale":"2024-04-25T10:05:38.551375+0000","last_undegraded":"2024-04-25T10:05:38.551375+0000","last_fullsized":"2024-04-25T10:05:38.551375+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:29.558721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548739+0000","last_change":"2024-04-25T10:05:36.191311+0000","last_active":"2024-04-25T10:05:38.548739+0000","last_peered":"2024-04-25T10:05:38.548739+0000","last_clean":"2024-04-25T10:05:38.548739+0000","last_became_active":"2024-04-25T10:05:36.190535+0000","last_became_peered":"2024-04-25T10:05:36.190535+0000","last_unstale":"2024-04-25T10:05:38.548739+0000","last_undegraded":"2024-04-25T10:05:38.548739+0000","last_fullsized":"2024-04-25T10:05:38.548739+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.118904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552275+0000","last_change":"2024-04-25T10:05:37.209994+0000","last_active":"2024-04-25T10:05:38.552275+0000","last_peered":"2024-04-25T10:05:38.552275+0000","last_clean":"2024-04-25T10:05:38.552275+0000","last_became_active":"2024-04-25T10:05:37.209686+0000","last_became_peered":"2024-04-25T10:05:37.209686+0000","last_unstale":"2024-04-25T10:05:38.552275+0000","last_undegraded":"2024-04-25T10:05:38.552275+0000","last_fullsized":"2024-04-25T10:05:38.552275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:36.861585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192702+0000","last_change":"2024-04-25T10:05:36.184629+0000","last_active":"2024-04-25T10:05:38.192702+0000","last_peered":"2024-04-25T10:05:38.192702+0000","last_clean":"2024-04-25T10:05:38.192702+0000","last_became_active":"2024-04-25T10:05:36.183892+0000","last_became_peered":"2024-04-25T10:05:36.183892+0000","last_unstale":"2024-04-25T10:05:38.192702+0000","last_undegraded":"2024-04-25T10:05:38.192702+0000","last_fullsized":"2024-04-25T10:05:38.192702+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:39.205026+0000","last_change":"2024-04-25T10:05:36.201261+0000","last_active":"2024-04-25T10:05:39.205026+0000","last_peered":"2024-04-25T10:05:39.205026+0000","last_clean":"2024-04-25T10:05:39.205026+0000","last_became_active":"2024-04-25T10:05:36.199919+0000","last_became_peered":"2024-04-25T10:05:36.199919+0000","last_unstale":"2024-04-25T10:05:39.205026+0000","last_undegraded":"2024-04-25T10:05:39.205026+0000","last_fullsized":"2024-04-25T10:05:39.205026+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:37.214569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549551+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:05:38.549551+0000","last_peered":"2024-04-25T10:05:38.549551+0000","last_clean":"2024-04-25T10:05:38.549551+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:05:38.549551+0000","last_undegraded":"2024-04-25T10:05:38.549551+0000","last_fullsized":"2024-04-25T10:05:38.549551+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":52,"acting":52,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46284,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325556,"statfs":{"total":96636764160,"available":96589369344,"internally_reserved":0,"allocated":1245184,"data_stored":879702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40096,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331744,"statfs":{"total":96636764160,"available":96595705856,"internally_reserved":0,"allocated":593920,"data_stored":262475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":643072,"data_stored":289210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45700,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326140,"statfs":{"total":96636764160,"available":96589967360,"internally_reserved":0,"allocated":647168,"data_stored":289792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44308,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327532,"statfs":{"total":96636764160,"available":96591392768,"internally_reserved":0,"allocated":659456,"data_stored":290844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":655360,"data_stored":290822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":651264,"data_stored":289370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50452,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321388,"statfs":{"total":96636764160,"available":96585101312,"internally_reserved":0,"allocated":1245184,"data_stored":879634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:40.948 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-04-25T10:05:40.948 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:36.155259+0000', 'last_change': '2024-04-25T10:05:36.155259+0000', 'last_active': '2024-04-25T10:05:36.155259+0000', 'last_peered': '2024-04-25T10:05:36.155259+0000', 'last_clean': '2024-04-25T10:05:36.155259+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:36.155259+0000', 'last_undegraded': '2024-04-25T10:05:36.155259+0000', 'last_fullsized': '2024-04-25T10:05:36.155259+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.948 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-04-25T10:05:40.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:36.155259+0000', 'last_change': '2024-04-25T10:05:36.155259+0000', 'last_active': '2024-04-25T10:05:36.155259+0000', 'last_peered': '2024-04-25T10:05:36.155259+0000', 'last_clean': '2024-04-25T10:05:36.155259+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:36.155259+0000', 'last_undegraded': '2024-04-25T10:05:36.155259+0000', 'last_fullsized': '2024-04-25T10:05:36.155259+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-04-25T10:05:40.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:36.155259+0000', 'last_change': '2024-04-25T10:05:36.155259+0000', 'last_active': '2024-04-25T10:05:36.155259+0000', 'last_peered': '2024-04-25T10:05:36.155259+0000', 'last_clean': '2024-04-25T10:05:36.155259+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:36.155259+0000', 'last_undegraded': '2024-04-25T10:05:36.155259+0000', 'last_fullsized': '2024-04-25T10:05:36.155259+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-25T10:05:40.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:35.144764+0000', 'last_change': '2024-04-25T10:05:35.144764+0000', 'last_active': '2024-04-25T10:05:35.144764+0000', 'last_peered': '2024-04-25T10:05:35.144764+0000', 'last_clean': '2024-04-25T10:05:35.144764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:35.144764+0000', 'last_undegraded': '2024-04-25T10:05:35.144764+0000', 'last_fullsized': '2024-04-25T10:05:35.144764+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-04-25T10:05:40.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:36.155259+0000', 'last_change': '2024-04-25T10:05:36.155259+0000', 'last_active': '2024-04-25T10:05:36.155259+0000', 'last_peered': '2024-04-25T10:05:36.155259+0000', 'last_clean': '2024-04-25T10:05:36.155259+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:36.155259+0000', 'last_undegraded': '2024-04-25T10:05:36.155259+0000', 'last_fullsized': '2024-04-25T10:05:36.155259+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-25T10:05:40.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:35.144764+0000', 'last_change': '2024-04-25T10:05:35.144764+0000', 'last_active': '2024-04-25T10:05:35.144764+0000', 'last_peered': '2024-04-25T10:05:35.144764+0000', 'last_clean': '2024-04-25T10:05:35.144764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:35.144764+0000', 'last_undegraded': '2024-04-25T10:05:35.144764+0000', 'last_fullsized': '2024-04-25T10:05:35.144764+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-25T10:05:40.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:35.144764+0000', 'last_change': '2024-04-25T10:05:35.144764+0000', 'last_active': '2024-04-25T10:05:35.144764+0000', 'last_peered': '2024-04-25T10:05:35.144764+0000', 'last_clean': '2024-04-25T10:05:35.144764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:35.144764+0000', 'last_undegraded': '2024-04-25T10:05:35.144764+0000', 'last_fullsized': '2024-04-25T10:05:35.144764+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-25T10:05:40.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:35.144764+0000', 'last_change': '2024-04-25T10:05:35.144764+0000', 'last_active': '2024-04-25T10:05:35.144764+0000', 'last_peered': '2024-04-25T10:05:35.144764+0000', 'last_clean': '2024-04-25T10:05:35.144764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:35.144764+0000', 'last_undegraded': '2024-04-25T10:05:35.144764+0000', 'last_fullsized': '2024-04-25T10:05:35.144764+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-25T10:05:40.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:35.144764+0000', 'last_change': '2024-04-25T10:05:35.144764+0000', 'last_active': '2024-04-25T10:05:35.144764+0000', 'last_peered': '2024-04-25T10:05:35.144764+0000', 'last_clean': '2024-04-25T10:05:35.144764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:35.144764+0000', 'last_undegraded': '2024-04-25T10:05:35.144764+0000', 'last_fullsized': '2024-04-25T10:05:35.144764+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-04-25T10:05:40.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:36.155259+0000', 'last_change': '2024-04-25T10:05:36.155259+0000', 'last_active': '2024-04-25T10:05:36.155259+0000', 'last_peered': '2024-04-25T10:05:36.155259+0000', 'last_clean': '2024-04-25T10:05:36.155259+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:36.155259+0000', 'last_undegraded': '2024-04-25T10:05:36.155259+0000', 'last_fullsized': '2024-04-25T10:05:36.155259+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-25T10:05:40.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:35.144764+0000', 'last_change': '2024-04-25T10:05:35.144764+0000', 'last_active': '2024-04-25T10:05:35.144764+0000', 'last_peered': '2024-04-25T10:05:35.144764+0000', 'last_clean': '2024-04-25T10:05:35.144764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:35.144764+0000', 'last_undegraded': '2024-04-25T10:05:35.144764+0000', 'last_fullsized': '2024-04-25T10:05:35.144764+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:35.144764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-04-25T10:05:40.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:05:36.155259+0000', 'last_change': '2024-04-25T10:05:36.155259+0000', 'last_active': '2024-04-25T10:05:36.155259+0000', 'last_peered': '2024-04-25T10:05:36.155259+0000', 'last_clean': '2024-04-25T10:05:36.155259+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:05:36.155259+0000', 'last_undegraded': '2024-04-25T10:05:36.155259+0000', 'last_fullsized': '2024-04-25T10:05:36.155259+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'last_clean_scrub_stamp': '2024-04-25T10:05:36.155259+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:05:40.951 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:05:41.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.102+0000 7f0f2a944640 1 Processor -- start 2024-04-25T10:05:41.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.102+0000 7f0f2a944640 1 -- start start 2024-04-25T10:05:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f2a944640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0f2415c9a0 0x7f0f240c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f2a944640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0f240b1f70 0x7f0f2406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f2a944640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f240c61d0 con 0x7f0f240ce330 2024-04-25T10:05:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f2a944640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f240c64b0 con 0x7f0f240b1f70 2024-04-25T10:05:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f2a944640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2406d4f0 con 0x7f0f2415c9a0 2024-04-25T10:05:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f1b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0f240b1f70 0x7f0f2406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f1b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0f240b1f70 0x7f0f2406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43574/0 (socket says 172.21.15.133:43574) 2024-04-25T10:05:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f1b7fe640 1 -- 172.21.15.133:0/3101872069 learned_addr learned my addr 172.21.15.133:0/3101872069 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f23fff640 1 --2- 172.21.15.133:0/3101872069 >> v2:172.21.15.162:3300/0 conn(0x7f0f2415c9a0 0x7f0f240c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f1b7fe640 1 -- 172.21.15.133:0/3101872069 >> v2:172.21.15.162:3300/0 conn(0x7f0f2415c9a0 msgr2=0x7f0f240c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f1b7fe640 1 --2- 172.21.15.133:0/3101872069 >> v2:172.21.15.162:3300/0 conn(0x7f0f2415c9a0 0x7f0f240c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f1b7fe640 1 -- 172.21.15.133:0/3101872069 >> v1:172.21.15.162:6789/0 conn(0x7f0f240ce330 legacy=0x7f0f2415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f1b7fe640 1 -- 172.21.15.133:0/3101872069 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2406d7d0 con 0x7f0f240b1f70 2024-04-25T10:05:41.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f23fff640 1 --2- 172.21.15.133:0/3101872069 >> v2:172.21.15.162:3300/0 conn(0x7f0f2415c9a0 0x7f0f240c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:41.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f1b7fe640 1 --2- 172.21.15.133:0/3101872069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f240b1f70 0x7f0f2406dcf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0f2406f5f0 tx=0x7f0f0c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deea1637390c47b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f237fe640 1 -- 172.21.15.133:0/3101872069 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0aa600 con 0x7f0f240b1f70 2024-04-25T10:05:41.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f237fe640 1 -- 172.21.15.133:0/3101872069 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c0af070 con 0x7f0f240b1f70 2024-04-25T10:05:41.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.106+0000 7f0f237fe640 1 -- 172.21.15.133:0/3101872069 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0aa9c0 con 0x7f0f240b1f70 2024-04-25T10:05:41.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 -- 172.21.15.133:0/3101872069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f240b1f70 msgr2=0x7f0f2406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 --2- 172.21.15.133:0/3101872069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f240b1f70 0x7f0f2406dcf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0f2406f5f0 tx=0x7f0f0c01c480 comp rx=0 tx=0).stop 2024-04-25T10:05:41.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 -- 172.21.15.133:0/3101872069 shutdown_connections 2024-04-25T10:05:41.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 --2- 172.21.15.133:0/3101872069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f240b1f70 0x7f0f2406dcf0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 --2- 172.21.15.133:0/3101872069 >> v2:172.21.15.162:3300/0 conn(0x7f0f2415c9a0 0x7f0f240c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 -- 172.21.15.133:0/3101872069 >> 172.21.15.133:0/3101872069 conn(0x7f0f24067000 msgr2=0x7f0f2406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 -- 172.21.15.133:0/3101872069 shutdown_connections 2024-04-25T10:05:41.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 -- 172.21.15.133:0/3101872069 wait complete. 2024-04-25T10:05:41.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 Processor -- start 2024-04-25T10:05:41.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 -- start start 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f240b1f70 0x7f0f2414f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f240ce330 0x7f0f240b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f2415c9a0 0x7f0f240b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2415cd80 con 0x7f0f2415c9a0 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2406e230 con 0x7f0f240b1f70 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2415cf00 con 0x7f0f240ce330 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f28ee1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f2415c9a0 0x7f0f240b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f28ee1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f2415c9a0 0x7f0f240b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43590/0 (socket says 172.21.15.133:43590) 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f28ee1640 1 -- 172.21.15.133:0/3465808187 learned_addr learned my addr 172.21.15.133:0/3465808187 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f23fff640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f240b1f70 0x7f0f2414f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f1b7fe640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f240ce330 0x7f0f240b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f28ee1640 1 -- 172.21.15.133:0/3465808187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f240ce330 msgr2=0x7f0f240b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f28ee1640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f240ce330 0x7f0f240b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f28ee1640 1 -- 172.21.15.133:0/3465808187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f240b1f70 msgr2=0x7f0f2414f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f28ee1640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f240b1f70 0x7f0f2414f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f28ee1640 1 -- 172.21.15.133:0/3465808187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2414feb0 con 0x7f0f2415c9a0 2024-04-25T10:05:41.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f23fff640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f240b1f70 0x7f0f2414f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:41.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f28ee1640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f2415c9a0 0x7f0f240b39f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00b9a0 tx=0x7f0f1c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f217fa640 1 -- 172.21.15.133:0/3465808187 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c019070 con 0x7f0f2415c9a0 2024-04-25T10:05:41.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f217fa640 1 -- 172.21.15.133:0/3465808187 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c00cba0 con 0x7f0f2415c9a0 2024-04-25T10:05:41.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f217fa640 1 -- 172.21.15.133:0/3465808187 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c014440 con 0x7f0f2415c9a0 2024-04-25T10:05:41.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 -- 172.21.15.133:0/3465808187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f24153290 con 0x7f0f2415c9a0 2024-04-25T10:05:41.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.110+0000 7f0f2a944640 1 -- 172.21.15.133:0/3465808187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f24153850 con 0x7f0f2415c9a0 2024-04-25T10:05:41.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.114+0000 7f0f2a944640 1 -- 172.21.15.133:0/3465808187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2406bc20 con 0x7f0f2415c9a0 2024-04-25T10:05:41.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.114+0000 7f0f217fa640 1 -- 172.21.15.133:0/3465808187 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f1c01c070 con 0x7f0f2415c9a0 2024-04-25T10:05:41.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.114+0000 7f0f217fa640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f04066f60 0x7f0f04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.114+0000 7f0f217fa640 1 -- 172.21.15.133:0/3465808187 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f1c084da0 con 0x7f0f2415c9a0 2024-04-25T10:05:41.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.114+0000 7f0f23fff640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f04066f60 0x7f0f04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.114+0000 7f0f23fff640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f04066f60 0x7f0f04069420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0f14008930 tx=0x7f0f14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.118+0000 7f0f217fa640 1 -- 172.21.15.133:0/3465808187 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f1c055cb0 con 0x7f0f2415c9a0 2024-04-25T10:05:41.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 -- 172.21.15.133:0/3465808187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0f24068ef0 con 0x7f0f2415c9a0 2024-04-25T10:05:41.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f217fa640 1 -- 172.21.15.133:0/3465808187 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1251 (secure 0 0 0) 0x7f0f1c0555b0 con 0x7f0f2415c9a0 2024-04-25T10:05:41.342 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:41.342 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":254,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":85},{"state_name":"unknown","count":12}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592126,"bytes_used":355586048,"bytes_avail":772738527232,"bytes_total":773094113280,"unknown_pgs_ratio":0.12371134012937546,"write_bytes_sec":2660,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":23,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":6012}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-04-25T10:04:57.762162+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:05:41.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 -- 172.21.15.133:0/3465808187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f04066f60 msgr2=0x7f0f04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f04066f60 0x7f0f04069420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0f14008930 tx=0x7f0f14008040 comp rx=0 tx=0).stop 2024-04-25T10:05:41.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 -- 172.21.15.133:0/3465808187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f2415c9a0 msgr2=0x7f0f240b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f2415c9a0 0x7f0f240b39f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00b9a0 tx=0x7f0f1c00be60 comp rx=0 tx=0).stop 2024-04-25T10:05:41.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 -- 172.21.15.133:0/3465808187 shutdown_connections 2024-04-25T10:05:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f04066f60 0x7f0f04069420 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f2415c9a0 0x7f0f240b39f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f240ce330 0x7f0f240b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 --2- 172.21.15.133:0/3465808187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f240b1f70 0x7f0f2414f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 -- 172.21.15.133:0/3465808187 >> 172.21.15.133:0/3465808187 conn(0x7f0f24067000 msgr2=0x7f0f2415b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 -- 172.21.15.133:0/3465808187 shutdown_connections 2024-04-25T10:05:41.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.338+0000 7f0f2a944640 1 -- 172.21.15.133:0/3465808187 wait complete. 2024-04-25T10:05:41.361 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:05:41.362 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:41.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d0e24f640 1 Processor -- start 2024-04-25T10:05:41.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d0e24f640 1 -- start start 2024-04-25T10:05:41.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d0e24f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1d08159dc0 0x7f1d080cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d0e24f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1d0815e220 0x7f1d0806d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d0e24f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080ce880 con 0x7f1d0815a1a0 2024-04-25T10:05:41.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d0e24f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080c49e0 con 0x7f1d08159dc0 2024-04-25T10:05:41.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d0e24f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080ceb20 con 0x7f1d0815e220 2024-04-25T10:05:41.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d077fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1d08159dc0 0x7f1d080cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d06ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1d0815a1a0 0x7f1d0815db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40134/0 (socket says 172.21.15.133:40134) 2024-04-25T10:05:41.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d06ffd640 1 -- 172.21.15.133:0/1473287636 learned_addr learned my addr 172.21.15.133:0/1473287636 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:41.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d07fff640 1 --2- 172.21.15.133:0/1473287636 >> v2:172.21.15.162:3300/0 conn(0x7f1d0815e220 0x7f1d0806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d077fe640 1 -- 172.21.15.133:0/1473287636 >> v2:172.21.15.162:3300/0 conn(0x7f1d0815e220 msgr2=0x7f1d0806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:41.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d077fe640 1 --2- 172.21.15.133:0/1473287636 >> v2:172.21.15.162:3300/0 conn(0x7f1d0815e220 0x7f1d0806d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d077fe640 1 -- 172.21.15.133:0/1473287636 >> v1:172.21.15.133:6789/0 conn(0x7f1d0815a1a0 legacy=0x7f1d0815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d067fc640 1 -- 172.21.15.133:0/1473287636 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476329129 0 0) 0x7f1d080ce880 con 0x7f1d0815a1a0 2024-04-25T10:05:41.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.478+0000 7f1d077fe640 1 -- 172.21.15.133:0/1473287636 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080cbfa0 con 0x7f1d08159dc0 2024-04-25T10:05:41.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d077fe640 1 --2- 172.21.15.133:0/1473287636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08159dc0 0x7f1d080cba60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1d080cee00 tx=0x7f1cf801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a569ad566441d3b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d067fc640 1 -- 172.21.15.133:0/1473287636 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80b0070 con 0x7f1d08159dc0 2024-04-25T10:05:41.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d067fc640 1 -- 172.21.15.133:0/1473287636 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf80ab600 con 0x7f1d08159dc0 2024-04-25T10:05:41.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d067fc640 1 -- 172.21.15.133:0/1473287636 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80ab9d0 con 0x7f1d08159dc0 2024-04-25T10:05:41.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d0e24f640 1 -- 172.21.15.133:0/1473287636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08159dc0 msgr2=0x7f1d080cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d0e24f640 1 --2- 172.21.15.133:0/1473287636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08159dc0 0x7f1d080cba60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1d080cee00 tx=0x7f1cf801c480 comp rx=0 tx=0).stop 2024-04-25T10:05:41.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d0e24f640 1 -- 172.21.15.133:0/1473287636 shutdown_connections 2024-04-25T10:05:41.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d0e24f640 1 --2- 172.21.15.133:0/1473287636 >> v2:172.21.15.162:3300/0 conn(0x7f1d0815e220 0x7f1d0806d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d0e24f640 1 --2- 172.21.15.133:0/1473287636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08159dc0 0x7f1d080cba60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d0e24f640 1 -- 172.21.15.133:0/1473287636 >> 172.21.15.133:0/1473287636 conn(0x7f1d08067000 msgr2=0x7f1d0806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d0e24f640 1 -- 172.21.15.133:0/1473287636 shutdown_connections 2024-04-25T10:05:41.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d0e24f640 1 -- 172.21.15.133:0/1473287636 wait complete. 2024-04-25T10:05:41.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d0e24f640 1 Processor -- start 2024-04-25T10:05:41.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.482+0000 7f1d0e24f640 1 -- start start 2024-04-25T10:05:41.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d0e24f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08159dc0 0x7f1d0814fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d0e24f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d0815e220 0x7f1d08149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d0e24f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d08146ce0 0x7f1d08144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d0e24f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0806da10 con 0x7f1d08159dc0 2024-04-25T10:05:41.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d0e24f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0806d890 con 0x7f1d08146ce0 2024-04-25T10:05:41.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d0e24f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0806db90 con 0x7f1d0815e220 2024-04-25T10:05:41.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08159dc0 0x7f1d0814fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08159dc0 0x7f1d0814fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43610/0 (socket says 172.21.15.133:43610) 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d077fe640 1 -- 172.21.15.133:0/279459595 learned_addr learned my addr 172.21.15.133:0/279459595 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d077fe640 1 -- 172.21.15.133:0/279459595 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d0815e220 msgr2=0x7f1d08149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d077fe640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d0815e220 0x7f1d08149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d077fe640 1 -- 172.21.15.133:0/279459595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d08146ce0 msgr2=0x7f1d08144770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d077fe640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d08146ce0 0x7f1d08144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d077fe640 1 -- 172.21.15.133:0/279459595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0815d380 con 0x7f1d08159dc0 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d077fe640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08159dc0 0x7f1d0814fa10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8019090 tx=0x7f1cf8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1ceffff640 1 -- 172.21.15.133:0/279459595 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80b0040 con 0x7f1d08159dc0 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1ceffff640 1 -- 172.21.15.133:0/279459595 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf80b9070 con 0x7f1d08159dc0 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d0e24f640 1 -- 172.21.15.133:0/279459595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0815c010 con 0x7f1d08159dc0 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1ceffff640 1 -- 172.21.15.133:0/279459595 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80b4610 con 0x7f1d08159dc0 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d0e24f640 1 -- 172.21.15.133:0/279459595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d081493d0 con 0x7f1d08159dc0 2024-04-25T10:05:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d0e24f640 1 -- 172.21.15.133:0/279459595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d08068ef0 con 0x7f1d08159dc0 2024-04-25T10:05:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1ceffff640 1 -- 172.21.15.133:0/279459595 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cf80b47b0 con 0x7f1d08159dc0 2024-04-25T10:05:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1ceffff640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf4066f90 0x7f1cf4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1ceffff640 1 -- 172.21.15.133:0/279459595 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1cf811eef0 con 0x7f1d08159dc0 2024-04-25T10:05:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d06ffd640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf4066f90 0x7f1cf4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.486+0000 7f1d06ffd640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf4066f90 0x7f1cf4069450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc000c10 tx=0x7f1cfc007910 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:41.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.490+0000 7f1ceffff640 1 -- 172.21.15.133:0/279459595 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cf8016070 con 0x7f1d08159dc0 2024-04-25T10:05:41.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.642+0000 7f1d0e24f640 1 -- 172.21.15.133:0/279459595 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d080b31e0 con 0x7f1cf4066f90 2024-04-25T10:05:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.658+0000 7f1ceffff640 1 -- 172.21.15.133:0/279459595 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250102 (secure 0 0 0) 0x7f1d080b31e0 con 0x7f1cf4066f90 2024-04-25T10:05:41.665 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:41.666 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:05:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 -- 172.21.15.133:0/279459595 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf4066f90 msgr2=0x7f1cf4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf4066f90 0x7f1cf4069450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc000c10 tx=0x7f1cfc007910 comp rx=0 tx=0).stop 2024-04-25T10:05:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 -- 172.21.15.133:0/279459595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08159dc0 msgr2=0x7f1d0814fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08159dc0 0x7f1d0814fa10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8019090 tx=0x7f1cf8019220 comp rx=0 tx=0).stop 2024-04-25T10:05:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 -- 172.21.15.133:0/279459595 shutdown_connections 2024-04-25T10:05:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d08146ce0 0x7f1d08144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d0815e220 0x7f1d08149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf4066f90 0x7f1cf4069450 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 --2- 172.21.15.133:0/279459595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d08159dc0 0x7f1d0814fa10 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 -- 172.21.15.133:0/279459595 >> 172.21.15.133:0/279459595 conn(0x7f1d08067000 msgr2=0x7f1d0815aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 -- 172.21.15.133:0/279459595 shutdown_connections 2024-04-25T10:05:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:41.662+0000 7f1d0e24f640 1 -- 172.21.15.133:0/279459595 wait complete. 2024-04-25T10:05:41.683 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-04-25T10:05:39.776113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":347252,"kb_used_data":6192,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754627468,"statfs":{"total":773094113280,"available":772738527232,"internally_reserved":0,"allocated":6340608,"data_stored":3471849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":349097521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.619070"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193794+0000","last_change":"2024-04-25T10:05:37.228346+0000","last_active":"2024-04-25T10:05:38.193794+0000","last_peered":"2024-04-25T10:05:38.193794+0000","last_clean":"2024-04-25T10:05:38.193794+0000","last_became_active":"2024-04-25T10:05:37.228062+0000","last_became_peered":"2024-04-25T10:05:37.228062+0000","last_unstale":"2024-04-25T10:05:38.193794+0000","last_undegraded":"2024-04-25T10:05:38.193794+0000","last_fullsized":"2024-04-25T10:05:38.193794+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.807237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551210+0000","last_change":"2024-04-25T10:05:37.244108+0000","last_active":"2024-04-25T10:05:38.551210+0000","last_peered":"2024-04-25T10:05:38.551210+0000","last_clean":"2024-04-25T10:05:38.551210+0000","last_became_active":"2024-04-25T10:05:37.243540+0000","last_became_peered":"2024-04-25T10:05:37.243540+0000","last_unstale":"2024-04-25T10:05:38.551210+0000","last_undegraded":"2024-04-25T10:05:38.551210+0000","last_fullsized":"2024-04-25T10:05:38.551210+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:57.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553820+0000","last_change":"2024-04-25T10:05:37.215389+0000","last_active":"2024-04-25T10:05:38.553820+0000","last_peered":"2024-04-25T10:05:38.553820+0000","last_clean":"2024-04-25T10:05:38.553820+0000","last_became_active":"2024-04-25T10:05:37.214066+0000","last_became_peered":"2024-04-25T10:05:37.214066+0000","last_unstale":"2024-04-25T10:05:38.553820+0000","last_undegraded":"2024-04-25T10:05:38.553820+0000","last_fullsized":"2024-04-25T10:05:38.553820+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:16.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550681+0000","last_change":"2024-04-25T10:05:37.243719+0000","last_active":"2024-04-25T10:05:38.550681+0000","last_peered":"2024-04-25T10:05:38.550681+0000","last_clean":"2024-04-25T10:05:38.550681+0000","last_became_active":"2024-04-25T10:05:37.242856+0000","last_became_peered":"2024-04-25T10:05:37.242856+0000","last_unstale":"2024-04-25T10:05:38.550681+0000","last_undegraded":"2024-04-25T10:05:38.550681+0000","last_fullsized":"2024-04-25T10:05:38.550681+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549470+0000","last_change":"2024-04-25T10:05:37.241776+0000","last_active":"2024-04-25T10:05:38.549470+0000","last_peered":"2024-04-25T10:05:38.549470+0000","last_clean":"2024-04-25T10:05:38.549470+0000","last_became_active":"2024-04-25T10:05:37.240967+0000","last_became_peered":"2024-04-25T10:05:37.240967+0000","last_unstale":"2024-04-25T10:05:38.549470+0000","last_undegraded":"2024-04-25T10:05:38.549470+0000","last_fullsized":"2024-04-25T10:05:38.549470+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.790321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193759+0000","last_change":"2024-04-25T10:05:37.228488+0000","last_active":"2024-04-25T10:05:38.193759+0000","last_peered":"2024-04-25T10:05:38.193759+0000","last_clean":"2024-04-25T10:05:38.193759+0000","last_became_active":"2024-04-25T10:05:37.228212+0000","last_became_peered":"2024-04-25T10:05:37.228212+0000","last_unstale":"2024-04-25T10:05:38.193759+0000","last_undegraded":"2024-04-25T10:05:38.193759+0000","last_fullsized":"2024-04-25T10:05:38.193759+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:40.538309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551627+0000","last_change":"2024-04-25T10:05:37.216471+0000","last_active":"2024-04-25T10:05:38.551627+0000","last_peered":"2024-04-25T10:05:38.551627+0000","last_clean":"2024-04-25T10:05:38.551627+0000","last_became_active":"2024-04-25T10:05:37.216024+0000","last_became_peered":"2024-04-25T10:05:37.216024+0000","last_unstale":"2024-04-25T10:05:38.551627+0000","last_undegraded":"2024-04-25T10:05:38.551627+0000","last_fullsized":"2024-04-25T10:05:38.551627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551113+0000","last_change":"2024-04-25T10:05:37.215547+0000","last_active":"2024-04-25T10:05:38.551113+0000","last_peered":"2024-04-25T10:05:38.551113+0000","last_clean":"2024-04-25T10:05:38.551113+0000","last_became_active":"2024-04-25T10:05:37.215035+0000","last_became_peered":"2024-04-25T10:05:37.215035+0000","last_unstale":"2024-04-25T10:05:38.551113+0000","last_undegraded":"2024-04-25T10:05:38.551113+0000","last_fullsized":"2024-04-25T10:05:38.551113+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:02.441654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551909+0000","last_change":"2024-04-25T10:05:37.219302+0000","last_active":"2024-04-25T10:05:38.551909+0000","last_peered":"2024-04-25T10:05:38.551909+0000","last_clean":"2024-04-25T10:05:38.551909+0000","last_became_active":"2024-04-25T10:05:37.218837+0000","last_became_peered":"2024-04-25T10:05:37.218837+0000","last_unstale":"2024-04-25T10:05:38.551909+0000","last_undegraded":"2024-04-25T10:05:38.551909+0000","last_fullsized":"2024-04-25T10:05:38.551909+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:22.308378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193584+0000","last_change":"2024-04-25T10:05:37.230769+0000","last_active":"2024-04-25T10:05:38.193584+0000","last_peered":"2024-04-25T10:05:38.193584+0000","last_clean":"2024-04-25T10:05:38.193584+0000","last_became_active":"2024-04-25T10:05:37.229183+0000","last_became_peered":"2024-04-25T10:05:37.229183+0000","last_unstale":"2024-04-25T10:05:38.193584+0000","last_undegraded":"2024-04-25T10:05:38.193584+0000","last_fullsized":"2024-04-25T10:05:38.193584+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:50.039535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.555020+0000","last_change":"2024-04-25T10:05:37.241984+0000","last_active":"2024-04-25T10:05:38.555020+0000","last_peered":"2024-04-25T10:05:38.555020+0000","last_clean":"2024-04-25T10:05:38.555020+0000","last_became_active":"2024-04-25T10:05:37.241700+0000","last_became_peered":"2024-04-25T10:05:37.241700+0000","last_unstale":"2024-04-25T10:05:38.555020+0000","last_undegraded":"2024-04-25T10:05:38.555020+0000","last_fullsized":"2024-04-25T10:05:38.555020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:02.887132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551320+0000","last_change":"2024-04-25T10:05:37.240777+0000","last_active":"2024-04-25T10:05:38.551320+0000","last_peered":"2024-04-25T10:05:38.551320+0000","last_clean":"2024-04-25T10:05:38.551320+0000","last_became_active":"2024-04-25T10:05:37.240486+0000","last_became_peered":"2024-04-25T10:05:37.240486+0000","last_unstale":"2024-04-25T10:05:38.551320+0000","last_undegraded":"2024-04-25T10:05:38.551320+0000","last_fullsized":"2024-04-25T10:05:38.551320+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:05.759734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194117+0000","last_change":"2024-04-25T10:05:37.228825+0000","last_active":"2024-04-25T10:05:38.194117+0000","last_peered":"2024-04-25T10:05:38.194117+0000","last_clean":"2024-04-25T10:05:38.194117+0000","last_became_active":"2024-04-25T10:05:37.228567+0000","last_became_peered":"2024-04-25T10:05:37.228567+0000","last_unstale":"2024-04-25T10:05:38.194117+0000","last_undegraded":"2024-04-25T10:05:38.194117+0000","last_fullsized":"2024-04-25T10:05:38.194117+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:42.164656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552001+0000","last_change":"2024-04-25T10:05:37.216195+0000","last_active":"2024-04-25T10:05:38.552001+0000","last_peered":"2024-04-25T10:05:38.552001+0000","last_clean":"2024-04-25T10:05:38.552001+0000","last_became_active":"2024-04-25T10:05:37.215910+0000","last_became_peered":"2024-04-25T10:05:37.215910+0000","last_unstale":"2024-04-25T10:05:38.552001+0000","last_undegraded":"2024-04-25T10:05:38.552001+0000","last_fullsized":"2024-04-25T10:05:38.552001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549428+0000","last_change":"2024-04-25T10:05:37.244630+0000","last_active":"2024-04-25T10:05:38.549428+0000","last_peered":"2024-04-25T10:05:38.549428+0000","last_clean":"2024-04-25T10:05:38.549428+0000","last_became_active":"2024-04-25T10:05:37.244120+0000","last_became_peered":"2024-04-25T10:05:37.244120+0000","last_unstale":"2024-04-25T10:05:38.549428+0000","last_undegraded":"2024-04-25T10:05:38.549428+0000","last_fullsized":"2024-04-25T10:05:38.549428+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193338+0000","last_change":"2024-04-25T10:05:37.229142+0000","last_active":"2024-04-25T10:05:38.193338+0000","last_peered":"2024-04-25T10:05:38.193338+0000","last_clean":"2024-04-25T10:05:38.193338+0000","last_became_active":"2024-04-25T10:05:37.228852+0000","last_became_peered":"2024-04-25T10:05:37.228852+0000","last_unstale":"2024-04-25T10:05:38.193338+0000","last_undegraded":"2024-04-25T10:05:38.193338+0000","last_fullsized":"2024-04-25T10:05:38.193338+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:56.021983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552832+0000","last_change":"2024-04-25T10:05:37.219710+0000","last_active":"2024-04-25T10:05:38.552832+0000","last_peered":"2024-04-25T10:05:38.552832+0000","last_clean":"2024-04-25T10:05:38.552832+0000","last_became_active":"2024-04-25T10:05:37.218987+0000","last_became_peered":"2024-04-25T10:05:37.218987+0000","last_unstale":"2024-04-25T10:05:38.552832+0000","last_undegraded":"2024-04-25T10:05:38.552832+0000","last_fullsized":"2024-04-25T10:05:38.552832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551559+0000","last_change":"2024-04-25T10:05:37.244343+0000","last_active":"2024-04-25T10:05:38.551559+0000","last_peered":"2024-04-25T10:05:38.551559+0000","last_clean":"2024-04-25T10:05:38.551559+0000","last_became_active":"2024-04-25T10:05:37.243655+0000","last_became_peered":"2024-04-25T10:05:37.243655+0000","last_unstale":"2024-04-25T10:05:38.551559+0000","last_undegraded":"2024-04-25T10:05:38.551559+0000","last_fullsized":"2024-04-25T10:05:38.551559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554321+0000","last_change":"2024-04-25T10:05:37.222335+0000","last_active":"2024-04-25T10:05:38.554321+0000","last_peered":"2024-04-25T10:05:38.554321+0000","last_clean":"2024-04-25T10:05:38.554321+0000","last_became_active":"2024-04-25T10:05:37.220912+0000","last_became_peered":"2024-04-25T10:05:37.220912+0000","last_unstale":"2024-04-25T10:05:38.554321+0000","last_undegraded":"2024-04-25T10:05:38.554321+0000","last_fullsized":"2024-04-25T10:05:38.554321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:02.441654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550628+0000","last_change":"2024-04-25T10:05:37.212566+0000","last_active":"2024-04-25T10:05:38.550628+0000","last_peered":"2024-04-25T10:05:38.550628+0000","last_clean":"2024-04-25T10:05:38.550628+0000","last_became_active":"2024-04-25T10:05:37.211952+0000","last_became_peered":"2024-04-25T10:05:37.211952+0000","last_unstale":"2024-04-25T10:05:38.550628+0000","last_undegraded":"2024-04-25T10:05:38.550628+0000","last_fullsized":"2024-04-25T10:05:38.550628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:00.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554521+0000","last_change":"2024-04-25T10:05:37.238454+0000","last_active":"2024-04-25T10:05:38.554521+0000","last_peered":"2024-04-25T10:05:38.554521+0000","last_clean":"2024-04-25T10:05:38.554521+0000","last_became_active":"2024-04-25T10:05:37.238147+0000","last_became_peered":"2024-04-25T10:05:37.238147+0000","last_unstale":"2024-04-25T10:05:38.554521+0000","last_undegraded":"2024-04-25T10:05:38.554521+0000","last_fullsized":"2024-04-25T10:05:38.554521+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:10.077596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550991+0000","last_change":"2024-04-25T10:05:37.216233+0000","last_active":"2024-04-25T10:05:38.550991+0000","last_peered":"2024-04-25T10:05:38.550991+0000","last_clean":"2024-04-25T10:05:38.550991+0000","last_became_active":"2024-04-25T10:05:37.214054+0000","last_became_peered":"2024-04-25T10:05:37.214054+0000","last_unstale":"2024-04-25T10:05:38.550991+0000","last_undegraded":"2024-04-25T10:05:38.550991+0000","last_fullsized":"2024-04-25T10:05:38.550991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:30.598917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553345+0000","last_change":"2024-04-25T10:05:37.219854+0000","last_active":"2024-04-25T10:05:38.553345+0000","last_peered":"2024-04-25T10:05:38.553345+0000","last_clean":"2024-04-25T10:05:38.553345+0000","last_became_active":"2024-04-25T10:05:37.219447+0000","last_became_peered":"2024-04-25T10:05:37.219447+0000","last_unstale":"2024-04-25T10:05:38.553345+0000","last_undegraded":"2024-04-25T10:05:38.553345+0000","last_fullsized":"2024-04-25T10:05:38.553345+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:00.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551069+0000","last_change":"2024-04-25T10:05:37.203182+0000","last_active":"2024-04-25T10:05:38.551069+0000","last_peered":"2024-04-25T10:05:38.551069+0000","last_clean":"2024-04-25T10:05:38.551069+0000","last_became_active":"2024-04-25T10:05:37.202893+0000","last_became_peered":"2024-04-25T10:05:37.202893+0000","last_unstale":"2024-04-25T10:05:38.551069+0000","last_undegraded":"2024-04-25T10:05:38.551069+0000","last_fullsized":"2024-04-25T10:05:38.551069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:12.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194246+0000","last_change":"2024-04-25T10:05:37.205817+0000","last_active":"2024-04-25T10:05:38.194246+0000","last_peered":"2024-04-25T10:05:38.194246+0000","last_clean":"2024-04-25T10:05:38.194246+0000","last_became_active":"2024-04-25T10:05:37.205572+0000","last_became_peered":"2024-04-25T10:05:37.205572+0000","last_unstale":"2024-04-25T10:05:38.194246+0000","last_undegraded":"2024-04-25T10:05:38.194246+0000","last_fullsized":"2024-04-25T10:05:38.194246+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:36.414900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553104+0000","last_change":"2024-04-25T10:05:37.240878+0000","last_active":"2024-04-25T10:05:38.553104+0000","last_peered":"2024-04-25T10:05:38.553104+0000","last_clean":"2024-04-25T10:05:38.553104+0000","last_became_active":"2024-04-25T10:05:37.240620+0000","last_became_peered":"2024-04-25T10:05:37.240620+0000","last_unstale":"2024-04-25T10:05:38.553104+0000","last_undegraded":"2024-04-25T10:05:38.553104+0000","last_fullsized":"2024-04-25T10:05:38.553104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550361+0000","last_change":"2024-04-25T10:05:37.246366+0000","last_active":"2024-04-25T10:05:38.550361+0000","last_peered":"2024-04-25T10:05:38.550361+0000","last_clean":"2024-04-25T10:05:38.550361+0000","last_became_active":"2024-04-25T10:05:37.239714+0000","last_became_peered":"2024-04-25T10:05:37.239714+0000","last_unstale":"2024-04-25T10:05:38.550361+0000","last_undegraded":"2024-04-25T10:05:38.550361+0000","last_fullsized":"2024-04-25T10:05:38.550361+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552009+0000","last_change":"2024-04-25T10:05:37.216294+0000","last_active":"2024-04-25T10:05:38.552009+0000","last_peered":"2024-04-25T10:05:38.552009+0000","last_clean":"2024-04-25T10:05:38.552009+0000","last_became_active":"2024-04-25T10:05:37.215738+0000","last_became_peered":"2024-04-25T10:05:37.215738+0000","last_unstale":"2024-04-25T10:05:38.552009+0000","last_undegraded":"2024-04-25T10:05:38.552009+0000","last_fullsized":"2024-04-25T10:05:38.552009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191560+0000","last_change":"2024-04-25T10:05:37.221885+0000","last_active":"2024-04-25T10:05:38.191560+0000","last_peered":"2024-04-25T10:05:38.191560+0000","last_clean":"2024-04-25T10:05:38.191560+0000","last_became_active":"2024-04-25T10:05:37.219364+0000","last_became_peered":"2024-04-25T10:05:37.219364+0000","last_unstale":"2024-04-25T10:05:38.191560+0000","last_undegraded":"2024-04-25T10:05:38.191560+0000","last_fullsized":"2024-04-25T10:05:38.191560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549409+0000","last_change":"2024-04-25T10:05:37.229695+0000","last_active":"2024-04-25T10:05:38.549409+0000","last_peered":"2024-04-25T10:05:38.549409+0000","last_clean":"2024-04-25T10:05:38.549409+0000","last_became_active":"2024-04-25T10:05:37.228961+0000","last_became_peered":"2024-04-25T10:05:37.228961+0000","last_unstale":"2024-04-25T10:05:38.549409+0000","last_undegraded":"2024-04-25T10:05:38.549409+0000","last_fullsized":"2024-04-25T10:05:38.549409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191931+0000","last_change":"2024-04-25T10:05:37.230425+0000","last_active":"2024-04-25T10:05:38.191931+0000","last_peered":"2024-04-25T10:05:38.191931+0000","last_clean":"2024-04-25T10:05:38.191931+0000","last_became_active":"2024-04-25T10:05:37.229462+0000","last_became_peered":"2024-04-25T10:05:37.229462+0000","last_unstale":"2024-04-25T10:05:38.191931+0000","last_undegraded":"2024-04-25T10:05:38.191931+0000","last_fullsized":"2024-04-25T10:05:38.191931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194406+0000","last_change":"2024-04-25T10:05:36.192412+0000","last_active":"2024-04-25T10:05:38.194406+0000","last_peered":"2024-04-25T10:05:38.194406+0000","last_clean":"2024-04-25T10:05:38.194406+0000","last_became_active":"2024-04-25T10:05:36.191754+0000","last_became_peered":"2024-04-25T10:05:36.191754+0000","last_unstale":"2024-04-25T10:05:38.194406+0000","last_undegraded":"2024-04-25T10:05:38.194406+0000","last_fullsized":"2024-04-25T10:05:38.194406+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:33.604952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194384+0000","last_change":"2024-04-25T10:05:36.197170+0000","last_active":"2024-04-25T10:05:38.194384+0000","last_peered":"2024-04-25T10:05:38.194384+0000","last_clean":"2024-04-25T10:05:38.194384+0000","last_became_active":"2024-04-25T10:05:36.196555+0000","last_became_peered":"2024-04-25T10:05:36.196555+0000","last_unstale":"2024-04-25T10:05:38.194384+0000","last_undegraded":"2024-04-25T10:05:38.194384+0000","last_fullsized":"2024-04-25T10:05:38.194384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:43.120025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550712+0000","last_change":"2024-04-25T10:05:37.206507+0000","last_active":"2024-04-25T10:05:38.550712+0000","last_peered":"2024-04-25T10:05:38.550712+0000","last_clean":"2024-04-25T10:05:38.550712+0000","last_became_active":"2024-04-25T10:05:37.204792+0000","last_became_peered":"2024-04-25T10:05:37.204792+0000","last_unstale":"2024-04-25T10:05:38.550712+0000","last_undegraded":"2024-04-25T10:05:38.550712+0000","last_fullsized":"2024-04-25T10:05:38.550712+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:16.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192596+0000","last_change":"2024-04-25T10:05:36.201693+0000","last_active":"2024-04-25T10:05:38.192596+0000","last_peered":"2024-04-25T10:05:38.192596+0000","last_clean":"2024-04-25T10:05:38.192596+0000","last_became_active":"2024-04-25T10:05:36.200944+0000","last_became_peered":"2024-04-25T10:05:36.200944+0000","last_unstale":"2024-04-25T10:05:38.192596+0000","last_undegraded":"2024-04-25T10:05:38.192596+0000","last_fullsized":"2024-04-25T10:05:38.192596+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550050+0000","last_change":"2024-04-25T10:05:37.222859+0000","last_active":"2024-04-25T10:05:38.550050+0000","last_peered":"2024-04-25T10:05:38.550050+0000","last_clean":"2024-04-25T10:05:38.550050+0000","last_became_active":"2024-04-25T10:05:37.222162+0000","last_became_peered":"2024-04-25T10:05:37.222162+0000","last_unstale":"2024-04-25T10:05:38.550050+0000","last_undegraded":"2024-04-25T10:05:38.550050+0000","last_fullsized":"2024-04-25T10:05:38.550050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:44.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553382+0000","last_change":"2024-04-25T10:05:36.193782+0000","last_active":"2024-04-25T10:05:38.553382+0000","last_peered":"2024-04-25T10:05:38.553382+0000","last_clean":"2024-04-25T10:05:38.553382+0000","last_became_active":"2024-04-25T10:05:36.190455+0000","last_became_peered":"2024-04-25T10:05:36.190455+0000","last_unstale":"2024-04-25T10:05:38.553382+0000","last_undegraded":"2024-04-25T10:05:38.553382+0000","last_fullsized":"2024-04-25T10:05:38.553382+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:32.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550845+0000","last_change":"2024-04-25T10:05:37.246868+0000","last_active":"2024-04-25T10:05:38.550845+0000","last_peered":"2024-04-25T10:05:38.550845+0000","last_clean":"2024-04-25T10:05:38.550845+0000","last_became_active":"2024-04-25T10:05:37.246095+0000","last_became_peered":"2024-04-25T10:05:37.246095+0000","last_unstale":"2024-04-25T10:05:38.550845+0000","last_undegraded":"2024-04-25T10:05:38.550845+0000","last_fullsized":"2024-04-25T10:05:38.550845+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548980+0000","last_change":"2024-04-25T10:05:36.197860+0000","last_active":"2024-04-25T10:05:38.548980+0000","last_peered":"2024-04-25T10:05:38.548980+0000","last_clean":"2024-04-25T10:05:38.548980+0000","last_became_active":"2024-04-25T10:05:36.197137+0000","last_became_peered":"2024-04-25T10:05:36.197137+0000","last_unstale":"2024-04-25T10:05:38.548980+0000","last_undegraded":"2024-04-25T10:05:38.548980+0000","last_fullsized":"2024-04-25T10:05:38.548980+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:30.726291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551788+0000","last_change":"2024-04-25T10:05:37.206314+0000","last_active":"2024-04-25T10:05:38.551788+0000","last_peered":"2024-04-25T10:05:38.551788+0000","last_clean":"2024-04-25T10:05:38.551788+0000","last_became_active":"2024-04-25T10:05:37.205748+0000","last_became_peered":"2024-04-25T10:05:37.205748+0000","last_unstale":"2024-04-25T10:05:38.551788+0000","last_undegraded":"2024-04-25T10:05:38.551788+0000","last_fullsized":"2024-04-25T10:05:38.551788+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:52.894883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550829+0000","last_change":"2024-04-25T10:05:37.240137+0000","last_active":"2024-04-25T10:05:38.550829+0000","last_peered":"2024-04-25T10:05:38.550829+0000","last_clean":"2024-04-25T10:05:38.550829+0000","last_became_active":"2024-04-25T10:05:37.239387+0000","last_became_peered":"2024-04-25T10:05:37.239387+0000","last_unstale":"2024-04-25T10:05:38.550829+0000","last_undegraded":"2024-04-25T10:05:38.550829+0000","last_fullsized":"2024-04-25T10:05:38.550829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551084+0000","last_change":"2024-04-25T10:05:36.197600+0000","last_active":"2024-04-25T10:05:38.551084+0000","last_peered":"2024-04-25T10:05:38.551084+0000","last_clean":"2024-04-25T10:05:38.551084+0000","last_became_active":"2024-04-25T10:05:36.197265+0000","last_became_peered":"2024-04-25T10:05:36.197265+0000","last_unstale":"2024-04-25T10:05:38.551084+0000","last_undegraded":"2024-04-25T10:05:38.551084+0000","last_fullsized":"2024-04-25T10:05:38.551084+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:24.137684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548851+0000","last_change":"2024-04-25T10:05:37.224800+0000","last_active":"2024-04-25T10:05:38.548851+0000","last_peered":"2024-04-25T10:05:38.548851+0000","last_clean":"2024-04-25T10:05:38.548851+0000","last_became_active":"2024-04-25T10:05:37.224068+0000","last_became_peered":"2024-04-25T10:05:37.224068+0000","last_unstale":"2024-04-25T10:05:38.548851+0000","last_undegraded":"2024-04-25T10:05:38.548851+0000","last_fullsized":"2024-04-25T10:05:38.548851+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:25.148179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551076+0000","last_change":"2024-04-25T10:05:36.195068+0000","last_active":"2024-04-25T10:05:38.551076+0000","last_peered":"2024-04-25T10:05:38.551076+0000","last_clean":"2024-04-25T10:05:38.551076+0000","last_became_active":"2024-04-25T10:05:36.194281+0000","last_became_peered":"2024-04-25T10:05:36.194281+0000","last_unstale":"2024-04-25T10:05:38.551076+0000","last_undegraded":"2024-04-25T10:05:38.551076+0000","last_fullsized":"2024-04-25T10:05:38.551076+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:41.154160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191829+0000","last_change":"2024-04-25T10:05:37.223331+0000","last_active":"2024-04-25T10:05:38.191829+0000","last_peered":"2024-04-25T10:05:38.191829+0000","last_clean":"2024-04-25T10:05:38.191829+0000","last_became_active":"2024-04-25T10:05:37.222613+0000","last_became_peered":"2024-04-25T10:05:37.222613+0000","last_unstale":"2024-04-25T10:05:38.191829+0000","last_undegraded":"2024-04-25T10:05:38.191829+0000","last_fullsized":"2024-04-25T10:05:38.191829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549616+0000","last_change":"2024-04-25T10:05:36.200167+0000","last_active":"2024-04-25T10:05:38.549616+0000","last_peered":"2024-04-25T10:05:38.549616+0000","last_clean":"2024-04-25T10:05:38.549616+0000","last_became_active":"2024-04-25T10:05:36.199467+0000","last_became_peered":"2024-04-25T10:05:36.199467+0000","last_unstale":"2024-04-25T10:05:38.549616+0000","last_undegraded":"2024-04-25T10:05:38.549616+0000","last_fullsized":"2024-04-25T10:05:38.549616+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:47.830431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552373+0000","last_change":"2024-04-25T10:05:37.241002+0000","last_active":"2024-04-25T10:05:38.552373+0000","last_peered":"2024-04-25T10:05:38.552373+0000","last_clean":"2024-04-25T10:05:38.552373+0000","last_became_active":"2024-04-25T10:05:37.240740+0000","last_became_peered":"2024-04-25T10:05:37.240740+0000","last_unstale":"2024-04-25T10:05:38.552373+0000","last_undegraded":"2024-04-25T10:05:38.552373+0000","last_fullsized":"2024-04-25T10:05:38.552373+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551231+0000","last_change":"2024-04-25T10:05:36.195424+0000","last_active":"2024-04-25T10:05:38.551231+0000","last_peered":"2024-04-25T10:05:38.551231+0000","last_clean":"2024-04-25T10:05:38.551231+0000","last_became_active":"2024-04-25T10:05:36.193306+0000","last_became_peered":"2024-04-25T10:05:36.193306+0000","last_unstale":"2024-04-25T10:05:38.551231+0000","last_undegraded":"2024-04-25T10:05:38.551231+0000","last_fullsized":"2024-04-25T10:05:38.551231+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549522+0000","last_change":"2024-04-25T10:05:37.220528+0000","last_active":"2024-04-25T10:05:38.549522+0000","last_peered":"2024-04-25T10:05:38.549522+0000","last_clean":"2024-04-25T10:05:38.549522+0000","last_became_active":"2024-04-25T10:05:37.219586+0000","last_became_peered":"2024-04-25T10:05:37.219586+0000","last_unstale":"2024-04-25T10:05:38.549522+0000","last_undegraded":"2024-04-25T10:05:38.549522+0000","last_fullsized":"2024-04-25T10:05:38.549522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:55.011916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551578+0000","last_change":"2024-04-25T10:05:36.199745+0000","last_active":"2024-04-25T10:05:38.551578+0000","last_peered":"2024-04-25T10:05:38.551578+0000","last_clean":"2024-04-25T10:05:38.551578+0000","last_became_active":"2024-04-25T10:05:36.191871+0000","last_became_peered":"2024-04-25T10:05:36.191871+0000","last_unstale":"2024-04-25T10:05:38.551578+0000","last_undegraded":"2024-04-25T10:05:38.551578+0000","last_fullsized":"2024-04-25T10:05:38.551578+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191388+0000","last_change":"2024-04-25T10:05:37.230868+0000","last_active":"2024-04-25T10:05:38.191388+0000","last_peered":"2024-04-25T10:05:38.191388+0000","last_clean":"2024-04-25T10:05:38.191388+0000","last_became_active":"2024-04-25T10:05:37.229549+0000","last_became_peered":"2024-04-25T10:05:37.229549+0000","last_unstale":"2024-04-25T10:05:38.191388+0000","last_undegraded":"2024-04-25T10:05:38.191388+0000","last_fullsized":"2024-04-25T10:05:38.191388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:04.305032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553620+0000","last_change":"2024-04-25T10:05:36.201220+0000","last_active":"2024-04-25T10:05:38.553620+0000","last_peered":"2024-04-25T10:05:38.553620+0000","last_clean":"2024-04-25T10:05:38.553620+0000","last_became_active":"2024-04-25T10:05:36.200820+0000","last_became_peered":"2024-04-25T10:05:36.200820+0000","last_unstale":"2024-04-25T10:05:38.553620+0000","last_undegraded":"2024-04-25T10:05:38.553620+0000","last_fullsized":"2024-04-25T10:05:38.553620+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:54.001421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554271+0000","last_change":"2024-04-25T10:05:37.215939+0000","last_active":"2024-04-25T10:05:38.554271+0000","last_peered":"2024-04-25T10:05:38.554271+0000","last_clean":"2024-04-25T10:05:38.554271+0000","last_became_active":"2024-04-25T10:05:37.215693+0000","last_became_peered":"2024-04-25T10:05:37.215693+0000","last_unstale":"2024-04-25T10:05:38.554271+0000","last_undegraded":"2024-04-25T10:05:38.554271+0000","last_fullsized":"2024-04-25T10:05:38.554271+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:12.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552870+0000","last_change":"2024-04-25T10:05:37.238447+0000","last_active":"2024-04-25T10:05:38.552870+0000","last_peered":"2024-04-25T10:05:38.552870+0000","last_clean":"2024-04-25T10:05:38.552870+0000","last_became_active":"2024-04-25T10:05:37.238106+0000","last_became_peered":"2024-04-25T10:05:37.238106+0000","last_unstale":"2024-04-25T10:05:38.552870+0000","last_undegraded":"2024-04-25T10:05:38.552870+0000","last_fullsized":"2024-04-25T10:05:38.552870+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:06.779179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549834+0000","last_change":"2024-04-25T10:05:36.198280+0000","last_active":"2024-04-25T10:05:38.549834+0000","last_peered":"2024-04-25T10:05:38.549834+0000","last_clean":"2024-04-25T10:05:38.549834+0000","last_became_active":"2024-04-25T10:05:36.197484+0000","last_became_peered":"2024-04-25T10:05:36.197484+0000","last_unstale":"2024-04-25T10:05:38.549834+0000","last_undegraded":"2024-04-25T10:05:38.549834+0000","last_fullsized":"2024-04-25T10:05:38.549834+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.796742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552446+0000","last_change":"2024-04-25T10:05:37.219004+0000","last_active":"2024-04-25T10:05:38.552446+0000","last_peered":"2024-04-25T10:05:38.552446+0000","last_clean":"2024-04-25T10:05:38.552446+0000","last_became_active":"2024-04-25T10:05:37.216939+0000","last_became_peered":"2024-04-25T10:05:37.216939+0000","last_unstale":"2024-04-25T10:05:38.552446+0000","last_undegraded":"2024-04-25T10:05:38.552446+0000","last_fullsized":"2024-04-25T10:05:38.552446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551470+0000","last_change":"2024-04-25T10:05:36.198305+0000","last_active":"2024-04-25T10:05:38.551470+0000","last_peered":"2024-04-25T10:05:38.551470+0000","last_clean":"2024-04-25T10:05:38.551470+0000","last_became_active":"2024-04-25T10:05:36.191908+0000","last_became_peered":"2024-04-25T10:05:36.191908+0000","last_unstale":"2024-04-25T10:05:38.551470+0000","last_undegraded":"2024-04-25T10:05:38.551470+0000","last_fullsized":"2024-04-25T10:05:38.551470+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:52.819131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553781+0000","last_change":"2024-04-25T10:05:37.224631+0000","last_active":"2024-04-25T10:05:38.553781+0000","last_peered":"2024-04-25T10:05:38.553781+0000","last_clean":"2024-04-25T10:05:38.553781+0000","last_became_active":"2024-04-25T10:05:37.224014+0000","last_became_peered":"2024-04-25T10:05:37.224014+0000","last_unstale":"2024-04-25T10:05:38.553781+0000","last_undegraded":"2024-04-25T10:05:38.553781+0000","last_fullsized":"2024-04-25T10:05:38.553781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:08.906935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553276+0000","last_change":"2024-04-25T10:05:36.203863+0000","last_active":"2024-04-25T10:05:38.553276+0000","last_peered":"2024-04-25T10:05:38.553276+0000","last_clean":"2024-04-25T10:05:38.553276+0000","last_became_active":"2024-04-25T10:05:36.203473+0000","last_became_peered":"2024-04-25T10:05:36.203473+0000","last_unstale":"2024-04-25T10:05:38.553276+0000","last_undegraded":"2024-04-25T10:05:38.553276+0000","last_fullsized":"2024-04-25T10:05:38.553276+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:24.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193575+0000","last_change":"2024-04-25T10:05:37.228514+0000","last_active":"2024-04-25T10:05:38.193575+0000","last_peered":"2024-04-25T10:05:38.193575+0000","last_clean":"2024-04-25T10:05:38.193575+0000","last_became_active":"2024-04-25T10:05:37.213429+0000","last_became_peered":"2024-04-25T10:05:37.213429+0000","last_unstale":"2024-04-25T10:05:38.193575+0000","last_undegraded":"2024-04-25T10:05:38.193575+0000","last_fullsized":"2024-04-25T10:05:38.193575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:07.257110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.555115+0000","last_change":"2024-04-25T10:05:36.192160+0000","last_active":"2024-04-25T10:05:38.555115+0000","last_peered":"2024-04-25T10:05:38.555115+0000","last_clean":"2024-04-25T10:05:38.555115+0000","last_became_active":"2024-04-25T10:05:36.191475+0000","last_became_peered":"2024-04-25T10:05:36.191475+0000","last_unstale":"2024-04-25T10:05:38.555115+0000","last_undegraded":"2024-04-25T10:05:38.555115+0000","last_fullsized":"2024-04-25T10:05:38.555115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:06.047245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548936+0000","last_change":"2024-04-25T10:05:37.218759+0000","last_active":"2024-04-25T10:05:38.548936+0000","last_peered":"2024-04-25T10:05:38.548936+0000","last_clean":"2024-04-25T10:05:38.548936+0000","last_became_active":"2024-04-25T10:05:37.217992+0000","last_became_peered":"2024-04-25T10:05:37.217992+0000","last_unstale":"2024-04-25T10:05:38.548936+0000","last_undegraded":"2024-04-25T10:05:38.548936+0000","last_fullsized":"2024-04-25T10:05:38.548936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:59.463557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194500+0000","last_change":"2024-04-25T10:05:37.227960+0000","last_active":"2024-04-25T10:05:38.194500+0000","last_peered":"2024-04-25T10:05:38.194500+0000","last_clean":"2024-04-25T10:05:38.194500+0000","last_became_active":"2024-04-25T10:05:37.213414+0000","last_became_peered":"2024-04-25T10:05:37.213414+0000","last_unstale":"2024-04-25T10:05:38.194500+0000","last_undegraded":"2024-04-25T10:05:38.194500+0000","last_fullsized":"2024-04-25T10:05:38.194500+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:01.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191893+0000","last_change":"2024-04-25T10:05:37.224756+0000","last_active":"2024-04-25T10:05:38.191893+0000","last_peered":"2024-04-25T10:05:38.191893+0000","last_clean":"2024-04-25T10:05:38.191893+0000","last_became_active":"2024-04-25T10:05:37.223895+0000","last_became_peered":"2024-04-25T10:05:37.223895+0000","last_unstale":"2024-04-25T10:05:38.191893+0000","last_undegraded":"2024-04-25T10:05:38.191893+0000","last_fullsized":"2024-04-25T10:05:38.191893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194604+0000","last_change":"2024-04-25T10:05:36.186378+0000","last_active":"2024-04-25T10:05:38.194604+0000","last_peered":"2024-04-25T10:05:38.194604+0000","last_clean":"2024-04-25T10:05:38.194604+0000","last_became_active":"2024-04-25T10:05:36.185733+0000","last_became_peered":"2024-04-25T10:05:36.185733+0000","last_unstale":"2024-04-25T10:05:38.194604+0000","last_undegraded":"2024-04-25T10:05:38.194604+0000","last_fullsized":"2024-04-25T10:05:38.194604+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:43.722766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551744+0000","last_change":"2024-04-25T10:05:36.199741+0000","last_active":"2024-04-25T10:05:38.551744+0000","last_peered":"2024-04-25T10:05:38.551744+0000","last_clean":"2024-04-25T10:05:38.551744+0000","last_became_active":"2024-04-25T10:05:36.199311+0000","last_became_peered":"2024-04-25T10:05:36.199311+0000","last_unstale":"2024-04-25T10:05:38.551744+0000","last_undegraded":"2024-04-25T10:05:38.551744+0000","last_fullsized":"2024-04-25T10:05:38.551744+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551563+0000","last_change":"2024-04-25T10:05:37.240101+0000","last_active":"2024-04-25T10:05:38.551563+0000","last_peered":"2024-04-25T10:05:38.551563+0000","last_clean":"2024-04-25T10:05:38.551563+0000","last_became_active":"2024-04-25T10:05:37.239789+0000","last_became_peered":"2024-04-25T10:05:37.239789+0000","last_unstale":"2024-04-25T10:05:38.551563+0000","last_undegraded":"2024-04-25T10:05:38.551563+0000","last_fullsized":"2024-04-25T10:05:38.551563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:04.305032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552440+0000","last_change":"2024-04-25T10:05:36.194819+0000","last_active":"2024-04-25T10:05:38.552440+0000","last_peered":"2024-04-25T10:05:38.552440+0000","last_clean":"2024-04-25T10:05:38.552440+0000","last_became_active":"2024-04-25T10:05:36.194511+0000","last_became_peered":"2024-04-25T10:05:36.194511+0000","last_unstale":"2024-04-25T10:05:38.552440+0000","last_undegraded":"2024-04-25T10:05:38.552440+0000","last_fullsized":"2024-04-25T10:05:38.552440+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:43.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194192+0000","last_change":"2024-04-25T10:05:37.204849+0000","last_active":"2024-04-25T10:05:38.194192+0000","last_peered":"2024-04-25T10:05:38.194192+0000","last_clean":"2024-04-25T10:05:38.194192+0000","last_became_active":"2024-04-25T10:05:37.204310+0000","last_became_peered":"2024-04-25T10:05:37.204310+0000","last_unstale":"2024-04-25T10:05:38.194192+0000","last_undegraded":"2024-04-25T10:05:38.194192+0000","last_fullsized":"2024-04-25T10:05:38.194192+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:49.155350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554104+0000","last_change":"2024-04-25T10:05:36.189798+0000","last_active":"2024-04-25T10:05:38.554104+0000","last_peered":"2024-04-25T10:05:38.554104+0000","last_clean":"2024-04-25T10:05:38.554104+0000","last_became_active":"2024-04-25T10:05:36.189079+0000","last_became_peered":"2024-04-25T10:05:36.189079+0000","last_unstale":"2024-04-25T10:05:38.554104+0000","last_undegraded":"2024-04-25T10:05:38.554104+0000","last_fullsized":"2024-04-25T10:05:38.554104+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:06.607042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192449+0000","last_change":"2024-04-25T10:05:37.213816+0000","last_active":"2024-04-25T10:05:38.192449+0000","last_peered":"2024-04-25T10:05:38.192449+0000","last_clean":"2024-04-25T10:05:38.192449+0000","last_became_active":"2024-04-25T10:05:37.213082+0000","last_became_peered":"2024-04-25T10:05:37.213082+0000","last_unstale":"2024-04-25T10:05:38.192449+0000","last_undegraded":"2024-04-25T10:05:38.192449+0000","last_fullsized":"2024-04-25T10:05:38.192449+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:35.144764+0000","last_change":"2024-04-25T10:05:35.144764+0000","last_active":"2024-04-25T10:05:35.144764+0000","last_peered":"2024-04-25T10:05:35.144764+0000","last_clean":"2024-04-25T10:05:35.144764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:35.144764+0000","last_undegraded":"2024-04-25T10:05:35.144764+0000","last_fullsized":"2024-04-25T10:05:35.144764+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193639+0000","last_change":"2024-04-25T10:05:37.199321+0000","last_active":"2024-04-25T10:05:38.193639+0000","last_peered":"2024-04-25T10:05:38.193639+0000","last_clean":"2024-04-25T10:05:38.193639+0000","last_became_active":"2024-04-25T10:05:37.199011+0000","last_became_peered":"2024-04-25T10:05:37.199011+0000","last_unstale":"2024-04-25T10:05:38.193639+0000","last_undegraded":"2024-04-25T10:05:38.193639+0000","last_fullsized":"2024-04-25T10:05:38.193639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.753836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551360+0000","last_change":"2024-04-25T10:05:36.198586+0000","last_active":"2024-04-25T10:05:38.551360+0000","last_peered":"2024-04-25T10:05:38.551360+0000","last_clean":"2024-04-25T10:05:38.551360+0000","last_became_active":"2024-04-25T10:05:36.197856+0000","last_became_peered":"2024-04-25T10:05:36.197856+0000","last_unstale":"2024-04-25T10:05:38.551360+0000","last_undegraded":"2024-04-25T10:05:38.551360+0000","last_fullsized":"2024-04-25T10:05:38.551360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550497+0000","last_change":"2024-04-25T10:05:37.225133+0000","last_active":"2024-04-25T10:05:38.550497+0000","last_peered":"2024-04-25T10:05:38.550497+0000","last_clean":"2024-04-25T10:05:38.550497+0000","last_became_active":"2024-04-25T10:05:37.224655+0000","last_became_peered":"2024-04-25T10:05:37.224655+0000","last_unstale":"2024-04-25T10:05:38.550497+0000","last_undegraded":"2024-04-25T10:05:38.550497+0000","last_fullsized":"2024-04-25T10:05:38.550497+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553244+0000","last_change":"2024-04-25T10:05:36.189704+0000","last_active":"2024-04-25T10:05:38.553244+0000","last_peered":"2024-04-25T10:05:38.553244+0000","last_clean":"2024-04-25T10:05:38.553244+0000","last_became_active":"2024-04-25T10:05:36.188969+0000","last_became_peered":"2024-04-25T10:05:36.188969+0000","last_unstale":"2024-04-25T10:05:38.553244+0000","last_undegraded":"2024-04-25T10:05:38.553244+0000","last_fullsized":"2024-04-25T10:05:38.553244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:58.453061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553005+0000","last_change":"2024-04-25T10:05:37.219793+0000","last_active":"2024-04-25T10:05:38.553005+0000","last_peered":"2024-04-25T10:05:38.553005+0000","last_clean":"2024-04-25T10:05:38.553005+0000","last_became_active":"2024-04-25T10:05:37.219521+0000","last_became_peered":"2024-04-25T10:05:37.219521+0000","last_unstale":"2024-04-25T10:05:38.553005+0000","last_undegraded":"2024-04-25T10:05:38.553005+0000","last_fullsized":"2024-04-25T10:05:38.553005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:57.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550594+0000","last_change":"2024-04-25T10:05:36.198953+0000","last_active":"2024-04-25T10:05:38.550594+0000","last_peered":"2024-04-25T10:05:38.550594+0000","last_clean":"2024-04-25T10:05:38.550594+0000","last_became_active":"2024-04-25T10:05:36.197976+0000","last_became_peered":"2024-04-25T10:05:36.197976+0000","last_unstale":"2024-04-25T10:05:38.550594+0000","last_undegraded":"2024-04-25T10:05:38.550594+0000","last_fullsized":"2024-04-25T10:05:38.550594+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:09.514786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191968+0000","last_change":"2024-04-25T10:05:37.217261+0000","last_active":"2024-04-25T10:05:38.191968+0000","last_peered":"2024-04-25T10:05:38.191968+0000","last_clean":"2024-04-25T10:05:38.191968+0000","last_became_active":"2024-04-25T10:05:37.216539+0000","last_became_peered":"2024-04-25T10:05:37.216539+0000","last_unstale":"2024-04-25T10:05:38.191968+0000","last_undegraded":"2024-04-25T10:05:38.191968+0000","last_fullsized":"2024-04-25T10:05:38.191968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551865+0000","last_change":"2024-04-25T10:05:36.191359+0000","last_active":"2024-04-25T10:05:38.551865+0000","last_peered":"2024-04-25T10:05:38.551865+0000","last_clean":"2024-04-25T10:05:38.551865+0000","last_became_active":"2024-04-25T10:05:36.190991+0000","last_became_peered":"2024-04-25T10:05:36.190991+0000","last_unstale":"2024-04-25T10:05:38.551865+0000","last_undegraded":"2024-04-25T10:05:38.551865+0000","last_fullsized":"2024-04-25T10:05:38.551865+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:00.276908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551375+0000","last_change":"2024-04-25T10:05:37.215467+0000","last_active":"2024-04-25T10:05:38.551375+0000","last_peered":"2024-04-25T10:05:38.551375+0000","last_clean":"2024-04-25T10:05:38.551375+0000","last_became_active":"2024-04-25T10:05:37.212611+0000","last_became_peered":"2024-04-25T10:05:37.212611+0000","last_unstale":"2024-04-25T10:05:38.551375+0000","last_undegraded":"2024-04-25T10:05:38.551375+0000","last_fullsized":"2024-04-25T10:05:38.551375+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:29.558721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548739+0000","last_change":"2024-04-25T10:05:36.191311+0000","last_active":"2024-04-25T10:05:38.548739+0000","last_peered":"2024-04-25T10:05:38.548739+0000","last_clean":"2024-04-25T10:05:38.548739+0000","last_became_active":"2024-04-25T10:05:36.190535+0000","last_became_peered":"2024-04-25T10:05:36.190535+0000","last_unstale":"2024-04-25T10:05:38.548739+0000","last_undegraded":"2024-04-25T10:05:38.548739+0000","last_fullsized":"2024-04-25T10:05:38.548739+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.118904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552275+0000","last_change":"2024-04-25T10:05:37.209994+0000","last_active":"2024-04-25T10:05:38.552275+0000","last_peered":"2024-04-25T10:05:38.552275+0000","last_clean":"2024-04-25T10:05:38.552275+0000","last_became_active":"2024-04-25T10:05:37.209686+0000","last_became_peered":"2024-04-25T10:05:37.209686+0000","last_unstale":"2024-04-25T10:05:38.552275+0000","last_undegraded":"2024-04-25T10:05:38.552275+0000","last_fullsized":"2024-04-25T10:05:38.552275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:36.861585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192702+0000","last_change":"2024-04-25T10:05:36.184629+0000","last_active":"2024-04-25T10:05:38.192702+0000","last_peered":"2024-04-25T10:05:38.192702+0000","last_clean":"2024-04-25T10:05:38.192702+0000","last_became_active":"2024-04-25T10:05:36.183892+0000","last_became_peered":"2024-04-25T10:05:36.183892+0000","last_unstale":"2024-04-25T10:05:38.192702+0000","last_undegraded":"2024-04-25T10:05:38.192702+0000","last_fullsized":"2024-04-25T10:05:38.192702+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:05:36.155259+0000","last_change":"2024-04-25T10:05:36.155259+0000","last_active":"2024-04-25T10:05:36.155259+0000","last_peered":"2024-04-25T10:05:36.155259+0000","last_clean":"2024-04-25T10:05:36.155259+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:05:36.155259+0000","last_undegraded":"2024-04-25T10:05:36.155259+0000","last_fullsized":"2024-04-25T10:05:36.155259+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:39.205026+0000","last_change":"2024-04-25T10:05:36.201261+0000","last_active":"2024-04-25T10:05:39.205026+0000","last_peered":"2024-04-25T10:05:39.205026+0000","last_clean":"2024-04-25T10:05:39.205026+0000","last_became_active":"2024-04-25T10:05:36.199919+0000","last_became_peered":"2024-04-25T10:05:36.199919+0000","last_unstale":"2024-04-25T10:05:39.205026+0000","last_undegraded":"2024-04-25T10:05:39.205026+0000","last_fullsized":"2024-04-25T10:05:39.205026+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:37.214569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549551+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:05:38.549551+0000","last_peered":"2024-04-25T10:05:38.549551+0000","last_clean":"2024-04-25T10:05:38.549551+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:05:38.549551+0000","last_undegraded":"2024-04-25T10:05:38.549551+0000","last_fullsized":"2024-04-25T10:05:38.549551+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":52,"acting":52,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46284,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325556,"statfs":{"total":96636764160,"available":96589369344,"internally_reserved":0,"allocated":1245184,"data_stored":879702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40096,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331744,"statfs":{"total":96636764160,"available":96595705856,"internally_reserved":0,"allocated":593920,"data_stored":262475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":643072,"data_stored":289210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45700,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326140,"statfs":{"total":96636764160,"available":96589967360,"internally_reserved":0,"allocated":647168,"data_stored":289792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44308,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327532,"statfs":{"total":96636764160,"available":96591392768,"internally_reserved":0,"allocated":659456,"data_stored":290844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":655360,"data_stored":290822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":651264,"data_stored":289370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50452,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321388,"statfs":{"total":96636764160,"available":96585101312,"internally_reserved":0,"allocated":1245184,"data_stored":879634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:42.445 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-04-25T10:05:42.442+0000 7f00dac3d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:44.688 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:44.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.814+0000 7fcbb83d7640 1 Processor -- start 2024-04-25T10:05:44.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.814+0000 7fcbb83d7640 1 -- start start 2024-04-25T10:05:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb83d7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcbb0152750 0x7fcbb00ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb83d7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb006d350 con 0x7fcbb00ce870 2024-04-25T10:05:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb83d7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb015bf90 con 0x7fcbb0152b30 2024-04-25T10:05:44.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb83d7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb00cecb0 con 0x7fcbb0152750 2024-04-25T10:05:44.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb6173640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcbb0152750 0x7fcbb00ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb6974640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcbb00ce870 0x7fcbb015f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40150/0 (socket says 172.21.15.133:40150) 2024-04-25T10:05:44.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb6974640 1 -- 172.21.15.133:0/4012223070 learned_addr learned my addr 172.21.15.133:0/4012223070 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:44.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb6173640 1 -- 172.21.15.133:0/4012223070 >> v1:172.21.15.162:6789/0 conn(0x7fcbb0152b30 legacy=0x7fcbb00b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:44.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb6173640 1 -- 172.21.15.133:0/4012223070 >> v1:172.21.15.133:6789/0 conn(0x7fcbb00ce870 legacy=0x7fcbb015f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb6173640 1 -- 172.21.15.133:0/4012223070 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb006d5d0 con 0x7fcbb0152750 2024-04-25T10:05:44.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb6173640 1 --2- 172.21.15.133:0/4012223070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb0152750 0x7fcbb00ce330 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0097c0 tx=0x7fcbac01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0539217bbee09f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:44.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb5171640 1 -- 172.21.15.133:0/4012223070 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0b0070 con 0x7fcbb0152750 2024-04-25T10:05:44.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb5171640 1 -- 172.21.15.133:0/4012223070 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac0ab600 con 0x7fcbb0152750 2024-04-25T10:05:44.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.818+0000 7fcbb5171640 1 -- 172.21.15.133:0/4012223070 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0ab9d0 con 0x7fcbb0152750 2024-04-25T10:05:44.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 -- 172.21.15.133:0/4012223070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb0152750 msgr2=0x7fcbb00ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 --2- 172.21.15.133:0/4012223070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb0152750 0x7fcbb00ce330 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0097c0 tx=0x7fcbac01c460 comp rx=0 tx=0).stop 2024-04-25T10:05:44.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 -- 172.21.15.133:0/4012223070 shutdown_connections 2024-04-25T10:05:44.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 --2- 172.21.15.133:0/4012223070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb0152750 0x7fcbb00ce330 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 -- 172.21.15.133:0/4012223070 >> 172.21.15.133:0/4012223070 conn(0x7fcbb0067000 msgr2=0x7fcbb006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:44.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 -- 172.21.15.133:0/4012223070 shutdown_connections 2024-04-25T10:05:44.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 -- 172.21.15.133:0/4012223070 wait complete. 2024-04-25T10:05:44.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 Processor -- start 2024-04-25T10:05:44.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 -- start start 2024-04-25T10:05:44.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbb00ce870 0x7fcbb014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb0152750 0x7fcbb00ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbb0152b30 0x7fcbb00ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0169770 con 0x7fcbb0152750 2024-04-25T10:05:44.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01695f0 con 0x7fcbb0152b30 2024-04-25T10:05:44.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01698f0 con 0x7fcbb00ce870 2024-04-25T10:05:44.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb5972640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb0152750 0x7fcbb00ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb5972640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb0152750 0x7fcbb00ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43636/0 (socket says 172.21.15.133:43636) 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb5972640 1 -- 172.21.15.133:0/2655273548 learned_addr learned my addr 172.21.15.133:0/2655273548 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb6173640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbb00ce870 0x7fcbb014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb5972640 1 -- 172.21.15.133:0/2655273548 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbb00ce870 msgr2=0x7fcbb014f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb6974640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbb0152b30 0x7fcbb00ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb5972640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbb00ce870 0x7fcbb014f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb5972640 1 -- 172.21.15.133:0/2655273548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbb0152b30 msgr2=0x7fcbb00ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb5972640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbb0152b30 0x7fcbb00ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb5972640 1 -- 172.21.15.133:0/2655273548 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb015afd0 con 0x7fcbb0152750 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb6974640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbb0152b30 0x7fcbb00ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb5972640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb0152750 0x7fcbb00ca310 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c480 tx=0x7fcba000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcba6ffd640 1 -- 172.21.15.133:0/2655273548 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0012830 con 0x7fcbb0152750 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcba6ffd640 1 -- 172.21.15.133:0/2655273548 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba0017070 con 0x7fcbb0152750 2024-04-25T10:05:44.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 -- 172.21.15.133:0/2655273548 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb015b2b0 con 0x7fcbb0152750 2024-04-25T10:05:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcba6ffd640 1 -- 172.21.15.133:0/2655273548 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0012c30 con 0x7fcbb0152750 2024-04-25T10:05:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.822+0000 7fcbb83d7640 1 -- 172.21.15.133:0/2655273548 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb00c8080 con 0x7fcbb0152750 2024-04-25T10:05:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.826+0000 7fcbb83d7640 1 -- 172.21.15.133:0/2655273548 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb0068ef0 con 0x7fcbb0152750 2024-04-25T10:05:44.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.826+0000 7fcba6ffd640 1 -- 172.21.15.133:0/2655273548 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcba001c460 con 0x7fcbb0152750 2024-04-25T10:05:44.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.826+0000 7fcba6ffd640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb94066f60 0x7fcb94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:44.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.826+0000 7fcba6ffd640 1 -- 172.21.15.133:0/2655273548 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcba0084f70 con 0x7fcbb0152750 2024-04-25T10:05:44.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.826+0000 7fcbb6173640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb94066f60 0x7fcb94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:44.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.826+0000 7fcbb6173640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb94066f60 0x7fcb94069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0ad000 tx=0x7fcbac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:44.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.830+0000 7fcba6ffd640 1 -- 172.21.15.133:0/2655273548 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcba0055e80 con 0x7fcbb0152750 2024-04-25T10:05:44.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.982+0000 7fcbb83d7640 1 -- 172.21.15.133:0/2655273548 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcbb00c8250 con 0x7fcb94066f60 2024-04-25T10:05:44.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.990+0000 7fcba6ffd640 1 -- 172.21.15.133:0/2655273548 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251960 (secure 0 0 0) 0x7fcbb00c8250 con 0x7fcb94066f60 2024-04-25T10:05:44.998 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:45.000 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:05:45.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.994+0000 7fcbb83d7640 1 -- 172.21.15.133:0/2655273548 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb94066f60 msgr2=0x7fcb94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.994+0000 7fcbb83d7640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb94066f60 0x7fcb94069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0ad000 tx=0x7fcbac008040 comp rx=0 tx=0).stop 2024-04-25T10:05:45.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.994+0000 7fcbb83d7640 1 -- 172.21.15.133:0/2655273548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb0152750 msgr2=0x7fcbb00ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.994+0000 7fcbb83d7640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb0152750 0x7fcbb00ca310 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c480 tx=0x7fcba000c940 comp rx=0 tx=0).stop 2024-04-25T10:05:45.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.994+0000 7fcbb83d7640 1 -- 172.21.15.133:0/2655273548 shutdown_connections 2024-04-25T10:05:45.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.994+0000 7fcbb83d7640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbb0152b30 0x7fcbb00ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.994+0000 7fcbb83d7640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb94066f60 0x7fcb94069420 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.994+0000 7fcbb83d7640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbb0152750 0x7fcbb00ca310 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.994+0000 7fcbb83d7640 1 --2- 172.21.15.133:0/2655273548 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbb00ce870 0x7fcbb014f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.994+0000 7fcbb83d7640 1 -- 172.21.15.133:0/2655273548 >> 172.21.15.133:0/2655273548 conn(0x7fcbb0067000 msgr2=0x7fcbb006d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.998+0000 7fcbb83d7640 1 -- 172.21.15.133:0/2655273548 shutdown_connections 2024-04-25T10:05:45.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:44.998+0000 7fcbb83d7640 1 -- 172.21.15.133:0/2655273548 wait complete. 2024-04-25T10:05:45.016 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-04-25T10:05:43.777510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":347296,"kb_used_data":6248,"kb_used_omap":12,"kb_used_meta":340915,"kb_avail":754627424,"statfs":{"total":773094113280,"available":772738482176,"internally_reserved":0,"allocated":6397952,"data_stored":3499236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":349097521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.598167"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193794+0000","last_change":"2024-04-25T10:05:37.228346+0000","last_active":"2024-04-25T10:05:38.193794+0000","last_peered":"2024-04-25T10:05:38.193794+0000","last_clean":"2024-04-25T10:05:38.193794+0000","last_became_active":"2024-04-25T10:05:37.228062+0000","last_became_peered":"2024-04-25T10:05:37.228062+0000","last_unstale":"2024-04-25T10:05:38.193794+0000","last_undegraded":"2024-04-25T10:05:38.193794+0000","last_fullsized":"2024-04-25T10:05:38.193794+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.807237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551210+0000","last_change":"2024-04-25T10:05:37.244108+0000","last_active":"2024-04-25T10:05:38.551210+0000","last_peered":"2024-04-25T10:05:38.551210+0000","last_clean":"2024-04-25T10:05:38.551210+0000","last_became_active":"2024-04-25T10:05:37.243540+0000","last_became_peered":"2024-04-25T10:05:37.243540+0000","last_unstale":"2024-04-25T10:05:38.551210+0000","last_undegraded":"2024-04-25T10:05:38.551210+0000","last_fullsized":"2024-04-25T10:05:38.551210+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:57.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553820+0000","last_change":"2024-04-25T10:05:37.215389+0000","last_active":"2024-04-25T10:05:38.553820+0000","last_peered":"2024-04-25T10:05:38.553820+0000","last_clean":"2024-04-25T10:05:38.553820+0000","last_became_active":"2024-04-25T10:05:37.214066+0000","last_became_peered":"2024-04-25T10:05:37.214066+0000","last_unstale":"2024-04-25T10:05:38.553820+0000","last_undegraded":"2024-04-25T10:05:38.553820+0000","last_fullsized":"2024-04-25T10:05:38.553820+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:16.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550681+0000","last_change":"2024-04-25T10:05:37.243719+0000","last_active":"2024-04-25T10:05:38.550681+0000","last_peered":"2024-04-25T10:05:38.550681+0000","last_clean":"2024-04-25T10:05:38.550681+0000","last_became_active":"2024-04-25T10:05:37.242856+0000","last_became_peered":"2024-04-25T10:05:37.242856+0000","last_unstale":"2024-04-25T10:05:38.550681+0000","last_undegraded":"2024-04-25T10:05:38.550681+0000","last_fullsized":"2024-04-25T10:05:38.550681+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549470+0000","last_change":"2024-04-25T10:05:37.241776+0000","last_active":"2024-04-25T10:05:38.549470+0000","last_peered":"2024-04-25T10:05:38.549470+0000","last_clean":"2024-04-25T10:05:38.549470+0000","last_became_active":"2024-04-25T10:05:37.240967+0000","last_became_peered":"2024-04-25T10:05:37.240967+0000","last_unstale":"2024-04-25T10:05:38.549470+0000","last_undegraded":"2024-04-25T10:05:38.549470+0000","last_fullsized":"2024-04-25T10:05:38.549470+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.790321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193759+0000","last_change":"2024-04-25T10:05:37.228488+0000","last_active":"2024-04-25T10:05:38.193759+0000","last_peered":"2024-04-25T10:05:38.193759+0000","last_clean":"2024-04-25T10:05:38.193759+0000","last_became_active":"2024-04-25T10:05:37.228212+0000","last_became_peered":"2024-04-25T10:05:37.228212+0000","last_unstale":"2024-04-25T10:05:38.193759+0000","last_undegraded":"2024-04-25T10:05:38.193759+0000","last_fullsized":"2024-04-25T10:05:38.193759+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:40.538309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551627+0000","last_change":"2024-04-25T10:05:37.216471+0000","last_active":"2024-04-25T10:05:38.551627+0000","last_peered":"2024-04-25T10:05:38.551627+0000","last_clean":"2024-04-25T10:05:38.551627+0000","last_became_active":"2024-04-25T10:05:37.216024+0000","last_became_peered":"2024-04-25T10:05:37.216024+0000","last_unstale":"2024-04-25T10:05:38.551627+0000","last_undegraded":"2024-04-25T10:05:38.551627+0000","last_fullsized":"2024-04-25T10:05:38.551627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551113+0000","last_change":"2024-04-25T10:05:37.215547+0000","last_active":"2024-04-25T10:05:38.551113+0000","last_peered":"2024-04-25T10:05:38.551113+0000","last_clean":"2024-04-25T10:05:38.551113+0000","last_became_active":"2024-04-25T10:05:37.215035+0000","last_became_peered":"2024-04-25T10:05:37.215035+0000","last_unstale":"2024-04-25T10:05:38.551113+0000","last_undegraded":"2024-04-25T10:05:38.551113+0000","last_fullsized":"2024-04-25T10:05:38.551113+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:02.441654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551909+0000","last_change":"2024-04-25T10:05:37.219302+0000","last_active":"2024-04-25T10:05:38.551909+0000","last_peered":"2024-04-25T10:05:38.551909+0000","last_clean":"2024-04-25T10:05:38.551909+0000","last_became_active":"2024-04-25T10:05:37.218837+0000","last_became_peered":"2024-04-25T10:05:37.218837+0000","last_unstale":"2024-04-25T10:05:38.551909+0000","last_undegraded":"2024-04-25T10:05:38.551909+0000","last_fullsized":"2024-04-25T10:05:38.551909+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:22.308378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193584+0000","last_change":"2024-04-25T10:05:37.230769+0000","last_active":"2024-04-25T10:05:38.193584+0000","last_peered":"2024-04-25T10:05:38.193584+0000","last_clean":"2024-04-25T10:05:38.193584+0000","last_became_active":"2024-04-25T10:05:37.229183+0000","last_became_peered":"2024-04-25T10:05:37.229183+0000","last_unstale":"2024-04-25T10:05:38.193584+0000","last_undegraded":"2024-04-25T10:05:38.193584+0000","last_fullsized":"2024-04-25T10:05:38.193584+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:50.039535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.555020+0000","last_change":"2024-04-25T10:05:37.241984+0000","last_active":"2024-04-25T10:05:38.555020+0000","last_peered":"2024-04-25T10:05:38.555020+0000","last_clean":"2024-04-25T10:05:38.555020+0000","last_became_active":"2024-04-25T10:05:37.241700+0000","last_became_peered":"2024-04-25T10:05:37.241700+0000","last_unstale":"2024-04-25T10:05:38.555020+0000","last_undegraded":"2024-04-25T10:05:38.555020+0000","last_fullsized":"2024-04-25T10:05:38.555020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:02.887132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551320+0000","last_change":"2024-04-25T10:05:37.240777+0000","last_active":"2024-04-25T10:05:38.551320+0000","last_peered":"2024-04-25T10:05:38.551320+0000","last_clean":"2024-04-25T10:05:38.551320+0000","last_became_active":"2024-04-25T10:05:37.240486+0000","last_became_peered":"2024-04-25T10:05:37.240486+0000","last_unstale":"2024-04-25T10:05:38.551320+0000","last_undegraded":"2024-04-25T10:05:38.551320+0000","last_fullsized":"2024-04-25T10:05:38.551320+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:05.759734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194117+0000","last_change":"2024-04-25T10:05:37.228825+0000","last_active":"2024-04-25T10:05:38.194117+0000","last_peered":"2024-04-25T10:05:38.194117+0000","last_clean":"2024-04-25T10:05:38.194117+0000","last_became_active":"2024-04-25T10:05:37.228567+0000","last_became_peered":"2024-04-25T10:05:37.228567+0000","last_unstale":"2024-04-25T10:05:38.194117+0000","last_undegraded":"2024-04-25T10:05:38.194117+0000","last_fullsized":"2024-04-25T10:05:38.194117+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:42.164656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552001+0000","last_change":"2024-04-25T10:05:37.216195+0000","last_active":"2024-04-25T10:05:38.552001+0000","last_peered":"2024-04-25T10:05:38.552001+0000","last_clean":"2024-04-25T10:05:38.552001+0000","last_became_active":"2024-04-25T10:05:37.215910+0000","last_became_peered":"2024-04-25T10:05:37.215910+0000","last_unstale":"2024-04-25T10:05:38.552001+0000","last_undegraded":"2024-04-25T10:05:38.552001+0000","last_fullsized":"2024-04-25T10:05:38.552001+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549428+0000","last_change":"2024-04-25T10:05:37.244630+0000","last_active":"2024-04-25T10:05:38.549428+0000","last_peered":"2024-04-25T10:05:38.549428+0000","last_clean":"2024-04-25T10:05:38.549428+0000","last_became_active":"2024-04-25T10:05:37.244120+0000","last_became_peered":"2024-04-25T10:05:37.244120+0000","last_unstale":"2024-04-25T10:05:38.549428+0000","last_undegraded":"2024-04-25T10:05:38.549428+0000","last_fullsized":"2024-04-25T10:05:38.549428+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193338+0000","last_change":"2024-04-25T10:05:37.229142+0000","last_active":"2024-04-25T10:05:38.193338+0000","last_peered":"2024-04-25T10:05:38.193338+0000","last_clean":"2024-04-25T10:05:38.193338+0000","last_became_active":"2024-04-25T10:05:37.228852+0000","last_became_peered":"2024-04-25T10:05:37.228852+0000","last_unstale":"2024-04-25T10:05:38.193338+0000","last_undegraded":"2024-04-25T10:05:38.193338+0000","last_fullsized":"2024-04-25T10:05:38.193338+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:56.021983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552832+0000","last_change":"2024-04-25T10:05:37.219710+0000","last_active":"2024-04-25T10:05:38.552832+0000","last_peered":"2024-04-25T10:05:38.552832+0000","last_clean":"2024-04-25T10:05:38.552832+0000","last_became_active":"2024-04-25T10:05:37.218987+0000","last_became_peered":"2024-04-25T10:05:37.218987+0000","last_unstale":"2024-04-25T10:05:38.552832+0000","last_undegraded":"2024-04-25T10:05:38.552832+0000","last_fullsized":"2024-04-25T10:05:38.552832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551559+0000","last_change":"2024-04-25T10:05:37.244343+0000","last_active":"2024-04-25T10:05:38.551559+0000","last_peered":"2024-04-25T10:05:38.551559+0000","last_clean":"2024-04-25T10:05:38.551559+0000","last_became_active":"2024-04-25T10:05:37.243655+0000","last_became_peered":"2024-04-25T10:05:37.243655+0000","last_unstale":"2024-04-25T10:05:38.551559+0000","last_undegraded":"2024-04-25T10:05:38.551559+0000","last_fullsized":"2024-04-25T10:05:38.551559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554321+0000","last_change":"2024-04-25T10:05:37.222335+0000","last_active":"2024-04-25T10:05:38.554321+0000","last_peered":"2024-04-25T10:05:38.554321+0000","last_clean":"2024-04-25T10:05:38.554321+0000","last_became_active":"2024-04-25T10:05:37.220912+0000","last_became_peered":"2024-04-25T10:05:37.220912+0000","last_unstale":"2024-04-25T10:05:38.554321+0000","last_undegraded":"2024-04-25T10:05:38.554321+0000","last_fullsized":"2024-04-25T10:05:38.554321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:02.441654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193217+0000","last_change":"2024-04-25T10:05:37.221227+0000","last_active":"2024-04-25T10:05:38.193217+0000","last_peered":"2024-04-25T10:05:38.193217+0000","last_clean":"2024-04-25T10:05:38.193217+0000","last_became_active":"2024-04-25T10:05:37.220479+0000","last_became_peered":"2024-04-25T10:05:37.220479+0000","last_unstale":"2024-04-25T10:05:38.193217+0000","last_undegraded":"2024-04-25T10:05:38.193217+0000","last_fullsized":"2024-04-25T10:05:38.193217+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:55.011916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193039+0000","last_change":"2024-04-25T10:05:37.216269+0000","last_active":"2024-04-25T10:05:38.193039+0000","last_peered":"2024-04-25T10:05:38.193039+0000","last_clean":"2024-04-25T10:05:38.193039+0000","last_became_active":"2024-04-25T10:05:37.215487+0000","last_became_peered":"2024-04-25T10:05:37.215487+0000","last_unstale":"2024-04-25T10:05:38.193039+0000","last_undegraded":"2024-04-25T10:05:38.193039+0000","last_fullsized":"2024-04-25T10:05:38.193039+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.790321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550628+0000","last_change":"2024-04-25T10:05:37.212566+0000","last_active":"2024-04-25T10:05:38.550628+0000","last_peered":"2024-04-25T10:05:38.550628+0000","last_clean":"2024-04-25T10:05:38.550628+0000","last_became_active":"2024-04-25T10:05:37.211952+0000","last_became_peered":"2024-04-25T10:05:37.211952+0000","last_unstale":"2024-04-25T10:05:38.550628+0000","last_undegraded":"2024-04-25T10:05:38.550628+0000","last_fullsized":"2024-04-25T10:05:38.550628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:00.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554521+0000","last_change":"2024-04-25T10:05:37.238454+0000","last_active":"2024-04-25T10:05:38.554521+0000","last_peered":"2024-04-25T10:05:38.554521+0000","last_clean":"2024-04-25T10:05:38.554521+0000","last_became_active":"2024-04-25T10:05:37.238147+0000","last_became_peered":"2024-04-25T10:05:37.238147+0000","last_unstale":"2024-04-25T10:05:38.554521+0000","last_undegraded":"2024-04-25T10:05:38.554521+0000","last_fullsized":"2024-04-25T10:05:38.554521+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:10.077596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550991+0000","last_change":"2024-04-25T10:05:37.216233+0000","last_active":"2024-04-25T10:05:38.550991+0000","last_peered":"2024-04-25T10:05:38.550991+0000","last_clean":"2024-04-25T10:05:38.550991+0000","last_became_active":"2024-04-25T10:05:37.214054+0000","last_became_peered":"2024-04-25T10:05:37.214054+0000","last_unstale":"2024-04-25T10:05:38.550991+0000","last_undegraded":"2024-04-25T10:05:38.550991+0000","last_fullsized":"2024-04-25T10:05:38.550991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:30.598917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553345+0000","last_change":"2024-04-25T10:05:37.219854+0000","last_active":"2024-04-25T10:05:38.553345+0000","last_peered":"2024-04-25T10:05:38.553345+0000","last_clean":"2024-04-25T10:05:38.553345+0000","last_became_active":"2024-04-25T10:05:37.219447+0000","last_became_peered":"2024-04-25T10:05:37.219447+0000","last_unstale":"2024-04-25T10:05:38.553345+0000","last_undegraded":"2024-04-25T10:05:38.553345+0000","last_fullsized":"2024-04-25T10:05:38.553345+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:00.371870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551069+0000","last_change":"2024-04-25T10:05:37.203182+0000","last_active":"2024-04-25T10:05:38.551069+0000","last_peered":"2024-04-25T10:05:38.551069+0000","last_clean":"2024-04-25T10:05:38.551069+0000","last_became_active":"2024-04-25T10:05:37.202893+0000","last_became_peered":"2024-04-25T10:05:37.202893+0000","last_unstale":"2024-04-25T10:05:38.551069+0000","last_undegraded":"2024-04-25T10:05:38.551069+0000","last_fullsized":"2024-04-25T10:05:38.551069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:12.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194246+0000","last_change":"2024-04-25T10:05:37.205817+0000","last_active":"2024-04-25T10:05:38.194246+0000","last_peered":"2024-04-25T10:05:38.194246+0000","last_clean":"2024-04-25T10:05:38.194246+0000","last_became_active":"2024-04-25T10:05:37.205572+0000","last_became_peered":"2024-04-25T10:05:37.205572+0000","last_unstale":"2024-04-25T10:05:38.194246+0000","last_undegraded":"2024-04-25T10:05:38.194246+0000","last_fullsized":"2024-04-25T10:05:38.194246+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:36.414900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193407+0000","last_change":"2024-04-25T10:05:37.198956+0000","last_active":"2024-04-25T10:05:38.193407+0000","last_peered":"2024-04-25T10:05:38.193407+0000","last_clean":"2024-04-25T10:05:38.193407+0000","last_became_active":"2024-04-25T10:05:37.198432+0000","last_became_peered":"2024-04-25T10:05:37.198432+0000","last_unstale":"2024-04-25T10:05:38.193407+0000","last_undegraded":"2024-04-25T10:05:38.193407+0000","last_fullsized":"2024-04-25T10:05:38.193407+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:44.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553104+0000","last_change":"2024-04-25T10:05:37.240878+0000","last_active":"2024-04-25T10:05:38.553104+0000","last_peered":"2024-04-25T10:05:38.553104+0000","last_clean":"2024-04-25T10:05:38.553104+0000","last_became_active":"2024-04-25T10:05:37.240620+0000","last_became_peered":"2024-04-25T10:05:37.240620+0000","last_unstale":"2024-04-25T10:05:38.553104+0000","last_undegraded":"2024-04-25T10:05:38.553104+0000","last_fullsized":"2024-04-25T10:05:38.553104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550361+0000","last_change":"2024-04-25T10:05:37.246366+0000","last_active":"2024-04-25T10:05:38.550361+0000","last_peered":"2024-04-25T10:05:38.550361+0000","last_clean":"2024-04-25T10:05:38.550361+0000","last_became_active":"2024-04-25T10:05:37.239714+0000","last_became_peered":"2024-04-25T10:05:37.239714+0000","last_unstale":"2024-04-25T10:05:38.550361+0000","last_undegraded":"2024-04-25T10:05:38.550361+0000","last_fullsized":"2024-04-25T10:05:38.550361+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552009+0000","last_change":"2024-04-25T10:05:37.216294+0000","last_active":"2024-04-25T10:05:38.552009+0000","last_peered":"2024-04-25T10:05:38.552009+0000","last_clean":"2024-04-25T10:05:38.552009+0000","last_became_active":"2024-04-25T10:05:37.215738+0000","last_became_peered":"2024-04-25T10:05:37.215738+0000","last_unstale":"2024-04-25T10:05:38.552009+0000","last_undegraded":"2024-04-25T10:05:38.552009+0000","last_fullsized":"2024-04-25T10:05:38.552009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191560+0000","last_change":"2024-04-25T10:05:37.221885+0000","last_active":"2024-04-25T10:05:38.191560+0000","last_peered":"2024-04-25T10:05:38.191560+0000","last_clean":"2024-04-25T10:05:38.191560+0000","last_became_active":"2024-04-25T10:05:37.219364+0000","last_became_peered":"2024-04-25T10:05:37.219364+0000","last_unstale":"2024-04-25T10:05:38.191560+0000","last_undegraded":"2024-04-25T10:05:38.191560+0000","last_fullsized":"2024-04-25T10:05:38.191560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549409+0000","last_change":"2024-04-25T10:05:37.229695+0000","last_active":"2024-04-25T10:05:38.549409+0000","last_peered":"2024-04-25T10:05:38.549409+0000","last_clean":"2024-04-25T10:05:38.549409+0000","last_became_active":"2024-04-25T10:05:37.228961+0000","last_became_peered":"2024-04-25T10:05:37.228961+0000","last_unstale":"2024-04-25T10:05:38.549409+0000","last_undegraded":"2024-04-25T10:05:38.549409+0000","last_fullsized":"2024-04-25T10:05:38.549409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193343+0000","last_change":"2024-04-25T10:05:36.193138+0000","last_active":"2024-04-25T10:05:38.193343+0000","last_peered":"2024-04-25T10:05:38.193343+0000","last_clean":"2024-04-25T10:05:38.193343+0000","last_became_active":"2024-04-25T10:05:36.192404+0000","last_became_peered":"2024-04-25T10:05:36.192404+0000","last_unstale":"2024-04-25T10:05:38.193343+0000","last_undegraded":"2024-04-25T10:05:38.193343+0000","last_fullsized":"2024-04-25T10:05:38.193343+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:30.726291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191931+0000","last_change":"2024-04-25T10:05:37.230425+0000","last_active":"2024-04-25T10:05:38.191931+0000","last_peered":"2024-04-25T10:05:38.191931+0000","last_clean":"2024-04-25T10:05:38.191931+0000","last_became_active":"2024-04-25T10:05:37.229462+0000","last_became_peered":"2024-04-25T10:05:37.229462+0000","last_unstale":"2024-04-25T10:05:38.191931+0000","last_undegraded":"2024-04-25T10:05:38.191931+0000","last_fullsized":"2024-04-25T10:05:38.191931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194406+0000","last_change":"2024-04-25T10:05:36.192412+0000","last_active":"2024-04-25T10:05:38.194406+0000","last_peered":"2024-04-25T10:05:38.194406+0000","last_clean":"2024-04-25T10:05:38.194406+0000","last_became_active":"2024-04-25T10:05:36.191754+0000","last_became_peered":"2024-04-25T10:05:36.191754+0000","last_unstale":"2024-04-25T10:05:38.194406+0000","last_undegraded":"2024-04-25T10:05:38.194406+0000","last_fullsized":"2024-04-25T10:05:38.194406+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:33.604952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193012+0000","last_change":"2024-04-25T10:05:37.237694+0000","last_active":"2024-04-25T10:05:38.193012+0000","last_peered":"2024-04-25T10:05:38.193012+0000","last_clean":"2024-04-25T10:05:38.193012+0000","last_became_active":"2024-04-25T10:05:37.236741+0000","last_became_peered":"2024-04-25T10:05:37.236741+0000","last_unstale":"2024-04-25T10:05:38.193012+0000","last_undegraded":"2024-04-25T10:05:38.193012+0000","last_fullsized":"2024-04-25T10:05:38.193012+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:59.463557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194384+0000","last_change":"2024-04-25T10:05:36.197170+0000","last_active":"2024-04-25T10:05:38.194384+0000","last_peered":"2024-04-25T10:05:38.194384+0000","last_clean":"2024-04-25T10:05:38.194384+0000","last_became_active":"2024-04-25T10:05:36.196555+0000","last_became_peered":"2024-04-25T10:05:36.196555+0000","last_unstale":"2024-04-25T10:05:38.194384+0000","last_undegraded":"2024-04-25T10:05:38.194384+0000","last_fullsized":"2024-04-25T10:05:38.194384+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:43.120025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550712+0000","last_change":"2024-04-25T10:05:37.206507+0000","last_active":"2024-04-25T10:05:38.550712+0000","last_peered":"2024-04-25T10:05:38.550712+0000","last_clean":"2024-04-25T10:05:38.550712+0000","last_became_active":"2024-04-25T10:05:37.204792+0000","last_became_peered":"2024-04-25T10:05:37.204792+0000","last_unstale":"2024-04-25T10:05:38.550712+0000","last_undegraded":"2024-04-25T10:05:38.550712+0000","last_fullsized":"2024-04-25T10:05:38.550712+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:16.000616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192596+0000","last_change":"2024-04-25T10:05:36.201693+0000","last_active":"2024-04-25T10:05:38.192596+0000","last_peered":"2024-04-25T10:05:38.192596+0000","last_clean":"2024-04-25T10:05:38.192596+0000","last_became_active":"2024-04-25T10:05:36.200944+0000","last_became_peered":"2024-04-25T10:05:36.200944+0000","last_unstale":"2024-04-25T10:05:38.192596+0000","last_undegraded":"2024-04-25T10:05:38.192596+0000","last_fullsized":"2024-04-25T10:05:38.192596+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550050+0000","last_change":"2024-04-25T10:05:37.222859+0000","last_active":"2024-04-25T10:05:38.550050+0000","last_peered":"2024-04-25T10:05:38.550050+0000","last_clean":"2024-04-25T10:05:38.550050+0000","last_became_active":"2024-04-25T10:05:37.222162+0000","last_became_peered":"2024-04-25T10:05:37.222162+0000","last_unstale":"2024-04-25T10:05:38.550050+0000","last_undegraded":"2024-04-25T10:05:38.550050+0000","last_fullsized":"2024-04-25T10:05:38.550050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:44.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553382+0000","last_change":"2024-04-25T10:05:36.193782+0000","last_active":"2024-04-25T10:05:38.553382+0000","last_peered":"2024-04-25T10:05:38.553382+0000","last_clean":"2024-04-25T10:05:38.553382+0000","last_became_active":"2024-04-25T10:05:36.190455+0000","last_became_peered":"2024-04-25T10:05:36.190455+0000","last_unstale":"2024-04-25T10:05:38.553382+0000","last_undegraded":"2024-04-25T10:05:38.553382+0000","last_fullsized":"2024-04-25T10:05:38.553382+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:32.779826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550845+0000","last_change":"2024-04-25T10:05:37.246868+0000","last_active":"2024-04-25T10:05:38.550845+0000","last_peered":"2024-04-25T10:05:38.550845+0000","last_clean":"2024-04-25T10:05:38.550845+0000","last_became_active":"2024-04-25T10:05:37.246095+0000","last_became_peered":"2024-04-25T10:05:37.246095+0000","last_unstale":"2024-04-25T10:05:38.550845+0000","last_undegraded":"2024-04-25T10:05:38.550845+0000","last_fullsized":"2024-04-25T10:05:38.550845+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548980+0000","last_change":"2024-04-25T10:05:36.197860+0000","last_active":"2024-04-25T10:05:38.548980+0000","last_peered":"2024-04-25T10:05:38.548980+0000","last_clean":"2024-04-25T10:05:38.548980+0000","last_became_active":"2024-04-25T10:05:36.197137+0000","last_became_peered":"2024-04-25T10:05:36.197137+0000","last_unstale":"2024-04-25T10:05:38.548980+0000","last_undegraded":"2024-04-25T10:05:38.548980+0000","last_fullsized":"2024-04-25T10:05:38.548980+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:30.726291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551788+0000","last_change":"2024-04-25T10:05:37.206314+0000","last_active":"2024-04-25T10:05:38.551788+0000","last_peered":"2024-04-25T10:05:38.551788+0000","last_clean":"2024-04-25T10:05:38.551788+0000","last_became_active":"2024-04-25T10:05:37.205748+0000","last_became_peered":"2024-04-25T10:05:37.205748+0000","last_unstale":"2024-04-25T10:05:38.551788+0000","last_undegraded":"2024-04-25T10:05:38.551788+0000","last_fullsized":"2024-04-25T10:05:38.551788+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:52.894883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191224+0000","last_change":"2024-04-25T10:05:36.190351+0000","last_active":"2024-04-25T10:05:38.191224+0000","last_peered":"2024-04-25T10:05:38.191224+0000","last_clean":"2024-04-25T10:05:38.191224+0000","last_became_active":"2024-04-25T10:05:36.189429+0000","last_became_peered":"2024-04-25T10:05:36.189429+0000","last_unstale":"2024-04-25T10:05:38.191224+0000","last_undegraded":"2024-04-25T10:05:38.191224+0000","last_fullsized":"2024-04-25T10:05:38.191224+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:55.218399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550829+0000","last_change":"2024-04-25T10:05:37.240137+0000","last_active":"2024-04-25T10:05:38.550829+0000","last_peered":"2024-04-25T10:05:38.550829+0000","last_clean":"2024-04-25T10:05:38.550829+0000","last_became_active":"2024-04-25T10:05:37.239387+0000","last_became_peered":"2024-04-25T10:05:37.239387+0000","last_unstale":"2024-04-25T10:05:38.550829+0000","last_undegraded":"2024-04-25T10:05:38.550829+0000","last_fullsized":"2024-04-25T10:05:38.550829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551084+0000","last_change":"2024-04-25T10:05:36.197600+0000","last_active":"2024-04-25T10:05:38.551084+0000","last_peered":"2024-04-25T10:05:38.551084+0000","last_clean":"2024-04-25T10:05:38.551084+0000","last_became_active":"2024-04-25T10:05:36.197265+0000","last_became_peered":"2024-04-25T10:05:36.197265+0000","last_unstale":"2024-04-25T10:05:38.551084+0000","last_undegraded":"2024-04-25T10:05:38.551084+0000","last_fullsized":"2024-04-25T10:05:38.551084+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:24.137684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548851+0000","last_change":"2024-04-25T10:05:37.224800+0000","last_active":"2024-04-25T10:05:38.548851+0000","last_peered":"2024-04-25T10:05:38.548851+0000","last_clean":"2024-04-25T10:05:38.548851+0000","last_became_active":"2024-04-25T10:05:37.224068+0000","last_became_peered":"2024-04-25T10:05:37.224068+0000","last_unstale":"2024-04-25T10:05:38.548851+0000","last_undegraded":"2024-04-25T10:05:38.548851+0000","last_fullsized":"2024-04-25T10:05:38.548851+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:25.148179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551076+0000","last_change":"2024-04-25T10:05:36.195068+0000","last_active":"2024-04-25T10:05:38.551076+0000","last_peered":"2024-04-25T10:05:38.551076+0000","last_clean":"2024-04-25T10:05:38.551076+0000","last_became_active":"2024-04-25T10:05:36.194281+0000","last_became_peered":"2024-04-25T10:05:36.194281+0000","last_unstale":"2024-04-25T10:05:38.551076+0000","last_undegraded":"2024-04-25T10:05:38.551076+0000","last_fullsized":"2024-04-25T10:05:38.551076+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:41.154160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191829+0000","last_change":"2024-04-25T10:05:37.223331+0000","last_active":"2024-04-25T10:05:38.191829+0000","last_peered":"2024-04-25T10:05:38.191829+0000","last_clean":"2024-04-25T10:05:38.191829+0000","last_became_active":"2024-04-25T10:05:37.222613+0000","last_became_peered":"2024-04-25T10:05:37.222613+0000","last_unstale":"2024-04-25T10:05:38.191829+0000","last_undegraded":"2024-04-25T10:05:38.191829+0000","last_fullsized":"2024-04-25T10:05:38.191829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:28.018909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549616+0000","last_change":"2024-04-25T10:05:36.200167+0000","last_active":"2024-04-25T10:05:38.549616+0000","last_peered":"2024-04-25T10:05:38.549616+0000","last_clean":"2024-04-25T10:05:38.549616+0000","last_became_active":"2024-04-25T10:05:36.199467+0000","last_became_peered":"2024-04-25T10:05:36.199467+0000","last_unstale":"2024-04-25T10:05:38.549616+0000","last_undegraded":"2024-04-25T10:05:38.549616+0000","last_fullsized":"2024-04-25T10:05:38.549616+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:47.830431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552373+0000","last_change":"2024-04-25T10:05:37.241002+0000","last_active":"2024-04-25T10:05:38.552373+0000","last_peered":"2024-04-25T10:05:38.552373+0000","last_clean":"2024-04-25T10:05:38.552373+0000","last_became_active":"2024-04-25T10:05:37.240740+0000","last_became_peered":"2024-04-25T10:05:37.240740+0000","last_unstale":"2024-04-25T10:05:38.552373+0000","last_undegraded":"2024-04-25T10:05:38.552373+0000","last_fullsized":"2024-04-25T10:05:38.552373+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:08.151039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551231+0000","last_change":"2024-04-25T10:05:36.195424+0000","last_active":"2024-04-25T10:05:38.551231+0000","last_peered":"2024-04-25T10:05:38.551231+0000","last_clean":"2024-04-25T10:05:38.551231+0000","last_became_active":"2024-04-25T10:05:36.193306+0000","last_became_peered":"2024-04-25T10:05:36.193306+0000","last_unstale":"2024-04-25T10:05:38.551231+0000","last_undegraded":"2024-04-25T10:05:38.551231+0000","last_fullsized":"2024-04-25T10:05:38.551231+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549522+0000","last_change":"2024-04-25T10:05:37.220528+0000","last_active":"2024-04-25T10:05:38.549522+0000","last_peered":"2024-04-25T10:05:38.549522+0000","last_clean":"2024-04-25T10:05:38.549522+0000","last_became_active":"2024-04-25T10:05:37.219586+0000","last_became_peered":"2024-04-25T10:05:37.219586+0000","last_unstale":"2024-04-25T10:05:38.549522+0000","last_undegraded":"2024-04-25T10:05:38.549522+0000","last_fullsized":"2024-04-25T10:05:38.549522+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:55.011916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551578+0000","last_change":"2024-04-25T10:05:36.199745+0000","last_active":"2024-04-25T10:05:38.551578+0000","last_peered":"2024-04-25T10:05:38.551578+0000","last_clean":"2024-04-25T10:05:38.551578+0000","last_became_active":"2024-04-25T10:05:36.191871+0000","last_became_peered":"2024-04-25T10:05:36.191871+0000","last_unstale":"2024-04-25T10:05:38.551578+0000","last_undegraded":"2024-04-25T10:05:38.551578+0000","last_fullsized":"2024-04-25T10:05:38.551578+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191388+0000","last_change":"2024-04-25T10:05:37.230868+0000","last_active":"2024-04-25T10:05:38.191388+0000","last_peered":"2024-04-25T10:05:38.191388+0000","last_clean":"2024-04-25T10:05:38.191388+0000","last_became_active":"2024-04-25T10:05:37.229549+0000","last_became_peered":"2024-04-25T10:05:37.229549+0000","last_unstale":"2024-04-25T10:05:38.191388+0000","last_undegraded":"2024-04-25T10:05:38.191388+0000","last_fullsized":"2024-04-25T10:05:38.191388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:04.305032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553620+0000","last_change":"2024-04-25T10:05:36.201220+0000","last_active":"2024-04-25T10:05:38.553620+0000","last_peered":"2024-04-25T10:05:38.553620+0000","last_clean":"2024-04-25T10:05:38.553620+0000","last_became_active":"2024-04-25T10:05:36.200820+0000","last_became_peered":"2024-04-25T10:05:36.200820+0000","last_unstale":"2024-04-25T10:05:38.553620+0000","last_undegraded":"2024-04-25T10:05:38.553620+0000","last_fullsized":"2024-04-25T10:05:38.553620+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:54.001421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554271+0000","last_change":"2024-04-25T10:05:37.215939+0000","last_active":"2024-04-25T10:05:38.554271+0000","last_peered":"2024-04-25T10:05:38.554271+0000","last_clean":"2024-04-25T10:05:38.554271+0000","last_became_active":"2024-04-25T10:05:37.215693+0000","last_became_peered":"2024-04-25T10:05:37.215693+0000","last_unstale":"2024-04-25T10:05:38.554271+0000","last_undegraded":"2024-04-25T10:05:38.554271+0000","last_fullsized":"2024-04-25T10:05:38.554271+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:12.932945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192769+0000","last_change":"2024-04-25T10:05:36.195354+0000","last_active":"2024-04-25T10:05:38.192769+0000","last_peered":"2024-04-25T10:05:38.192769+0000","last_clean":"2024-04-25T10:05:38.192769+0000","last_became_active":"2024-04-25T10:05:36.194358+0000","last_became_peered":"2024-04-25T10:05:36.194358+0000","last_unstale":"2024-04-25T10:05:38.192769+0000","last_undegraded":"2024-04-25T10:05:38.192769+0000","last_fullsized":"2024-04-25T10:05:38.192769+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:47.830431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552870+0000","last_change":"2024-04-25T10:05:37.238447+0000","last_active":"2024-04-25T10:05:38.552870+0000","last_peered":"2024-04-25T10:05:38.552870+0000","last_clean":"2024-04-25T10:05:38.552870+0000","last_became_active":"2024-04-25T10:05:37.238106+0000","last_became_peered":"2024-04-25T10:05:37.238106+0000","last_unstale":"2024-04-25T10:05:38.552870+0000","last_undegraded":"2024-04-25T10:05:38.552870+0000","last_fullsized":"2024-04-25T10:05:38.552870+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:06.779179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549834+0000","last_change":"2024-04-25T10:05:36.198280+0000","last_active":"2024-04-25T10:05:38.549834+0000","last_peered":"2024-04-25T10:05:38.549834+0000","last_clean":"2024-04-25T10:05:38.549834+0000","last_became_active":"2024-04-25T10:05:36.197484+0000","last_became_peered":"2024-04-25T10:05:36.197484+0000","last_unstale":"2024-04-25T10:05:38.549834+0000","last_undegraded":"2024-04-25T10:05:38.549834+0000","last_fullsized":"2024-04-25T10:05:38.549834+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.796742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552446+0000","last_change":"2024-04-25T10:05:37.219004+0000","last_active":"2024-04-25T10:05:38.552446+0000","last_peered":"2024-04-25T10:05:38.552446+0000","last_clean":"2024-04-25T10:05:38.552446+0000","last_became_active":"2024-04-25T10:05:37.216939+0000","last_became_peered":"2024-04-25T10:05:37.216939+0000","last_unstale":"2024-04-25T10:05:38.552446+0000","last_undegraded":"2024-04-25T10:05:38.552446+0000","last_fullsized":"2024-04-25T10:05:38.552446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551470+0000","last_change":"2024-04-25T10:05:36.198305+0000","last_active":"2024-04-25T10:05:38.551470+0000","last_peered":"2024-04-25T10:05:38.551470+0000","last_clean":"2024-04-25T10:05:38.551470+0000","last_became_active":"2024-04-25T10:05:36.191908+0000","last_became_peered":"2024-04-25T10:05:36.191908+0000","last_unstale":"2024-04-25T10:05:38.551470+0000","last_undegraded":"2024-04-25T10:05:38.551470+0000","last_fullsized":"2024-04-25T10:05:38.551470+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:52.819131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553781+0000","last_change":"2024-04-25T10:05:37.224631+0000","last_active":"2024-04-25T10:05:38.553781+0000","last_peered":"2024-04-25T10:05:38.553781+0000","last_clean":"2024-04-25T10:05:38.553781+0000","last_became_active":"2024-04-25T10:05:37.224014+0000","last_became_peered":"2024-04-25T10:05:37.224014+0000","last_unstale":"2024-04-25T10:05:38.553781+0000","last_undegraded":"2024-04-25T10:05:38.553781+0000","last_fullsized":"2024-04-25T10:05:38.553781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:08.906935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553276+0000","last_change":"2024-04-25T10:05:36.203863+0000","last_active":"2024-04-25T10:05:38.553276+0000","last_peered":"2024-04-25T10:05:38.553276+0000","last_clean":"2024-04-25T10:05:38.553276+0000","last_became_active":"2024-04-25T10:05:36.203473+0000","last_became_peered":"2024-04-25T10:05:36.203473+0000","last_unstale":"2024-04-25T10:05:38.553276+0000","last_undegraded":"2024-04-25T10:05:38.553276+0000","last_fullsized":"2024-04-25T10:05:38.553276+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:24.903882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193575+0000","last_change":"2024-04-25T10:05:37.228514+0000","last_active":"2024-04-25T10:05:38.193575+0000","last_peered":"2024-04-25T10:05:38.193575+0000","last_clean":"2024-04-25T10:05:38.193575+0000","last_became_active":"2024-04-25T10:05:37.213429+0000","last_became_peered":"2024-04-25T10:05:37.213429+0000","last_unstale":"2024-04-25T10:05:38.193575+0000","last_undegraded":"2024-04-25T10:05:38.193575+0000","last_fullsized":"2024-04-25T10:05:38.193575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:07.257110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.555115+0000","last_change":"2024-04-25T10:05:36.192160+0000","last_active":"2024-04-25T10:05:38.555115+0000","last_peered":"2024-04-25T10:05:38.555115+0000","last_clean":"2024-04-25T10:05:38.555115+0000","last_became_active":"2024-04-25T10:05:36.191475+0000","last_became_peered":"2024-04-25T10:05:36.191475+0000","last_unstale":"2024-04-25T10:05:38.555115+0000","last_undegraded":"2024-04-25T10:05:38.555115+0000","last_fullsized":"2024-04-25T10:05:38.555115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:06.047245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548936+0000","last_change":"2024-04-25T10:05:37.218759+0000","last_active":"2024-04-25T10:05:38.548936+0000","last_peered":"2024-04-25T10:05:38.548936+0000","last_clean":"2024-04-25T10:05:38.548936+0000","last_became_active":"2024-04-25T10:05:37.217992+0000","last_became_peered":"2024-04-25T10:05:37.217992+0000","last_unstale":"2024-04-25T10:05:38.548936+0000","last_undegraded":"2024-04-25T10:05:38.548936+0000","last_fullsized":"2024-04-25T10:05:38.548936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:59.463557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193462+0000","last_change":"2024-04-25T10:05:36.194967+0000","last_active":"2024-04-25T10:05:38.193462+0000","last_peered":"2024-04-25T10:05:38.193462+0000","last_clean":"2024-04-25T10:05:38.193462+0000","last_became_active":"2024-04-25T10:05:36.194063+0000","last_became_peered":"2024-04-25T10:05:36.194063+0000","last_unstale":"2024-04-25T10:05:38.193462+0000","last_undegraded":"2024-04-25T10:05:38.193462+0000","last_fullsized":"2024-04-25T10:05:38.193462+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:07.834554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194500+0000","last_change":"2024-04-25T10:05:37.227960+0000","last_active":"2024-04-25T10:05:38.194500+0000","last_peered":"2024-04-25T10:05:38.194500+0000","last_clean":"2024-04-25T10:05:38.194500+0000","last_became_active":"2024-04-25T10:05:37.213414+0000","last_became_peered":"2024-04-25T10:05:37.213414+0000","last_unstale":"2024-04-25T10:05:38.194500+0000","last_undegraded":"2024-04-25T10:05:38.194500+0000","last_fullsized":"2024-04-25T10:05:38.194500+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:01.287403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193444+0000","last_change":"2024-04-25T10:05:36.199649+0000","last_active":"2024-04-25T10:05:38.193444+0000","last_peered":"2024-04-25T10:05:38.193444+0000","last_clean":"2024-04-25T10:05:38.193444+0000","last_became_active":"2024-04-25T10:05:36.198586+0000","last_became_peered":"2024-04-25T10:05:36.198586+0000","last_unstale":"2024-04-25T10:05:38.193444+0000","last_undegraded":"2024-04-25T10:05:38.193444+0000","last_fullsized":"2024-04-25T10:05:38.193444+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:32.934523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191893+0000","last_change":"2024-04-25T10:05:37.224756+0000","last_active":"2024-04-25T10:05:38.191893+0000","last_peered":"2024-04-25T10:05:38.191893+0000","last_clean":"2024-04-25T10:05:38.191893+0000","last_became_active":"2024-04-25T10:05:37.223895+0000","last_became_peered":"2024-04-25T10:05:37.223895+0000","last_unstale":"2024-04-25T10:05:38.191893+0000","last_undegraded":"2024-04-25T10:05:38.191893+0000","last_fullsized":"2024-04-25T10:05:38.191893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:12.280273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194604+0000","last_change":"2024-04-25T10:05:36.186378+0000","last_active":"2024-04-25T10:05:38.194604+0000","last_peered":"2024-04-25T10:05:38.194604+0000","last_clean":"2024-04-25T10:05:38.194604+0000","last_became_active":"2024-04-25T10:05:36.185733+0000","last_became_peered":"2024-04-25T10:05:36.185733+0000","last_unstale":"2024-04-25T10:05:38.194604+0000","last_undegraded":"2024-04-25T10:05:38.194604+0000","last_fullsized":"2024-04-25T10:05:38.194604+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:43.722766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193577+0000","last_change":"2024-04-25T10:05:37.219044+0000","last_active":"2024-04-25T10:05:38.193577+0000","last_peered":"2024-04-25T10:05:38.193577+0000","last_clean":"2024-04-25T10:05:38.193577+0000","last_became_active":"2024-04-25T10:05:37.214643+0000","last_became_peered":"2024-04-25T10:05:37.214643+0000","last_unstale":"2024-04-25T10:05:38.193577+0000","last_undegraded":"2024-04-25T10:05:38.193577+0000","last_fullsized":"2024-04-25T10:05:38.193577+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:38.225064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551744+0000","last_change":"2024-04-25T10:05:36.199741+0000","last_active":"2024-04-25T10:05:38.551744+0000","last_peered":"2024-04-25T10:05:38.551744+0000","last_clean":"2024-04-25T10:05:38.551744+0000","last_became_active":"2024-04-25T10:05:36.199311+0000","last_became_peered":"2024-04-25T10:05:36.199311+0000","last_unstale":"2024-04-25T10:05:38.551744+0000","last_undegraded":"2024-04-25T10:05:38.551744+0000","last_fullsized":"2024-04-25T10:05:38.551744+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551563+0000","last_change":"2024-04-25T10:05:37.240101+0000","last_active":"2024-04-25T10:05:38.551563+0000","last_peered":"2024-04-25T10:05:38.551563+0000","last_clean":"2024-04-25T10:05:38.551563+0000","last_became_active":"2024-04-25T10:05:37.239789+0000","last_became_peered":"2024-04-25T10:05:37.239789+0000","last_unstale":"2024-04-25T10:05:38.551563+0000","last_undegraded":"2024-04-25T10:05:38.551563+0000","last_fullsized":"2024-04-25T10:05:38.551563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:04.305032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552440+0000","last_change":"2024-04-25T10:05:36.194819+0000","last_active":"2024-04-25T10:05:38.552440+0000","last_peered":"2024-04-25T10:05:38.552440+0000","last_clean":"2024-04-25T10:05:38.552440+0000","last_became_active":"2024-04-25T10:05:36.194511+0000","last_became_peered":"2024-04-25T10:05:36.194511+0000","last_unstale":"2024-04-25T10:05:38.552440+0000","last_undegraded":"2024-04-25T10:05:38.552440+0000","last_fullsized":"2024-04-25T10:05:38.552440+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:43.644444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.194192+0000","last_change":"2024-04-25T10:05:37.204849+0000","last_active":"2024-04-25T10:05:38.194192+0000","last_peered":"2024-04-25T10:05:38.194192+0000","last_clean":"2024-04-25T10:05:38.194192+0000","last_became_active":"2024-04-25T10:05:37.204310+0000","last_became_peered":"2024-04-25T10:05:37.204310+0000","last_unstale":"2024-04-25T10:05:38.194192+0000","last_undegraded":"2024-04-25T10:05:38.194192+0000","last_fullsized":"2024-04-25T10:05:38.194192+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:49.155350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.554104+0000","last_change":"2024-04-25T10:05:36.189798+0000","last_active":"2024-04-25T10:05:38.554104+0000","last_peered":"2024-04-25T10:05:38.554104+0000","last_clean":"2024-04-25T10:05:38.554104+0000","last_became_active":"2024-04-25T10:05:36.189079+0000","last_became_peered":"2024-04-25T10:05:36.189079+0000","last_unstale":"2024-04-25T10:05:38.554104+0000","last_undegraded":"2024-04-25T10:05:38.554104+0000","last_fullsized":"2024-04-25T10:05:38.554104+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:06.607042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192449+0000","last_change":"2024-04-25T10:05:37.213816+0000","last_active":"2024-04-25T10:05:38.192449+0000","last_peered":"2024-04-25T10:05:38.192449+0000","last_clean":"2024-04-25T10:05:38.192449+0000","last_became_active":"2024-04-25T10:05:37.213082+0000","last_became_peered":"2024-04-25T10:05:37.213082+0000","last_unstale":"2024-04-25T10:05:38.192449+0000","last_undegraded":"2024-04-25T10:05:38.192449+0000","last_fullsized":"2024-04-25T10:05:38.192449+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193601+0000","last_change":"2024-04-25T10:05:36.184974+0000","last_active":"2024-04-25T10:05:38.193601+0000","last_peered":"2024-04-25T10:05:38.193601+0000","last_clean":"2024-04-25T10:05:38.193601+0000","last_became_active":"2024-04-25T10:05:36.184298+0000","last_became_peered":"2024-04-25T10:05:36.184298+0000","last_unstale":"2024-04-25T10:05:38.193601+0000","last_undegraded":"2024-04-25T10:05:38.193601+0000","last_fullsized":"2024-04-25T10:05:38.193601+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:09.255166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.193639+0000","last_change":"2024-04-25T10:05:37.199321+0000","last_active":"2024-04-25T10:05:38.193639+0000","last_peered":"2024-04-25T10:05:38.193639+0000","last_clean":"2024-04-25T10:05:38.193639+0000","last_became_active":"2024-04-25T10:05:37.199011+0000","last_became_peered":"2024-04-25T10:05:37.199011+0000","last_unstale":"2024-04-25T10:05:38.193639+0000","last_undegraded":"2024-04-25T10:05:38.193639+0000","last_fullsized":"2024-04-25T10:05:38.193639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:22.753836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551360+0000","last_change":"2024-04-25T10:05:36.198586+0000","last_active":"2024-04-25T10:05:38.551360+0000","last_peered":"2024-04-25T10:05:38.551360+0000","last_clean":"2024-04-25T10:05:38.551360+0000","last_became_active":"2024-04-25T10:05:36.197856+0000","last_became_peered":"2024-04-25T10:05:36.197856+0000","last_unstale":"2024-04-25T10:05:38.551360+0000","last_undegraded":"2024-04-25T10:05:38.551360+0000","last_fullsized":"2024-04-25T10:05:38.551360+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:35.404404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550497+0000","last_change":"2024-04-25T10:05:37.225133+0000","last_active":"2024-04-25T10:05:38.550497+0000","last_peered":"2024-04-25T10:05:38.550497+0000","last_clean":"2024-04-25T10:05:38.550497+0000","last_became_active":"2024-04-25T10:05:37.224655+0000","last_became_peered":"2024-04-25T10:05:37.224655+0000","last_unstale":"2024-04-25T10:05:38.550497+0000","last_undegraded":"2024-04-25T10:05:38.550497+0000","last_fullsized":"2024-04-25T10:05:38.550497+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:04.433285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553244+0000","last_change":"2024-04-25T10:05:36.189704+0000","last_active":"2024-04-25T10:05:38.553244+0000","last_peered":"2024-04-25T10:05:38.553244+0000","last_clean":"2024-04-25T10:05:38.553244+0000","last_became_active":"2024-04-25T10:05:36.188969+0000","last_became_peered":"2024-04-25T10:05:36.188969+0000","last_unstale":"2024-04-25T10:05:38.553244+0000","last_undegraded":"2024-04-25T10:05:38.553244+0000","last_fullsized":"2024-04-25T10:05:38.553244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:58.453061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.553005+0000","last_change":"2024-04-25T10:05:37.219793+0000","last_active":"2024-04-25T10:05:38.553005+0000","last_peered":"2024-04-25T10:05:38.553005+0000","last_clean":"2024-04-25T10:05:38.553005+0000","last_became_active":"2024-04-25T10:05:37.219521+0000","last_became_peered":"2024-04-25T10:05:37.219521+0000","last_unstale":"2024-04-25T10:05:38.553005+0000","last_undegraded":"2024-04-25T10:05:38.553005+0000","last_fullsized":"2024-04-25T10:05:38.553005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:57.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.550594+0000","last_change":"2024-04-25T10:05:36.198953+0000","last_active":"2024-04-25T10:05:38.550594+0000","last_peered":"2024-04-25T10:05:38.550594+0000","last_clean":"2024-04-25T10:05:38.550594+0000","last_became_active":"2024-04-25T10:05:36.197976+0000","last_became_peered":"2024-04-25T10:05:36.197976+0000","last_unstale":"2024-04-25T10:05:38.550594+0000","last_undegraded":"2024-04-25T10:05:38.550594+0000","last_fullsized":"2024-04-25T10:05:38.550594+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:09.514786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191968+0000","last_change":"2024-04-25T10:05:37.217261+0000","last_active":"2024-04-25T10:05:38.191968+0000","last_peered":"2024-04-25T10:05:38.191968+0000","last_clean":"2024-04-25T10:05:38.191968+0000","last_became_active":"2024-04-25T10:05:37.216539+0000","last_became_peered":"2024-04-25T10:05:37.216539+0000","last_unstale":"2024-04-25T10:05:38.191968+0000","last_undegraded":"2024-04-25T10:05:38.191968+0000","last_fullsized":"2024-04-25T10:05:38.191968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.124987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551865+0000","last_change":"2024-04-25T10:05:36.191359+0000","last_active":"2024-04-25T10:05:38.551865+0000","last_peered":"2024-04-25T10:05:38.551865+0000","last_clean":"2024-04-25T10:05:38.551865+0000","last_became_active":"2024-04-25T10:05:36.190991+0000","last_became_peered":"2024-04-25T10:05:36.190991+0000","last_unstale":"2024-04-25T10:05:38.551865+0000","last_undegraded":"2024-04-25T10:05:38.551865+0000","last_fullsized":"2024-04-25T10:05:38.551865+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:00.276908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.551375+0000","last_change":"2024-04-25T10:05:37.215467+0000","last_active":"2024-04-25T10:05:38.551375+0000","last_peered":"2024-04-25T10:05:38.551375+0000","last_clean":"2024-04-25T10:05:38.551375+0000","last_became_active":"2024-04-25T10:05:37.212611+0000","last_became_peered":"2024-04-25T10:05:37.212611+0000","last_unstale":"2024-04-25T10:05:38.551375+0000","last_undegraded":"2024-04-25T10:05:38.551375+0000","last_fullsized":"2024-04-25T10:05:38.551375+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:29.558721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.548739+0000","last_change":"2024-04-25T10:05:36.191311+0000","last_active":"2024-04-25T10:05:38.548739+0000","last_peered":"2024-04-25T10:05:38.548739+0000","last_clean":"2024-04-25T10:05:38.548739+0000","last_became_active":"2024-04-25T10:05:36.190535+0000","last_became_peered":"2024-04-25T10:05:36.190535+0000","last_unstale":"2024-04-25T10:05:38.548739+0000","last_undegraded":"2024-04-25T10:05:38.548739+0000","last_fullsized":"2024-04-25T10:05:38.548739+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.118904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.552275+0000","last_change":"2024-04-25T10:05:37.209994+0000","last_active":"2024-04-25T10:05:38.552275+0000","last_peered":"2024-04-25T10:05:38.552275+0000","last_clean":"2024-04-25T10:05:38.552275+0000","last_became_active":"2024-04-25T10:05:37.209686+0000","last_became_peered":"2024-04-25T10:05:37.209686+0000","last_unstale":"2024-04-25T10:05:38.552275+0000","last_undegraded":"2024-04-25T10:05:38.552275+0000","last_fullsized":"2024-04-25T10:05:38.552275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:36.861585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.192702+0000","last_change":"2024-04-25T10:05:36.184629+0000","last_active":"2024-04-25T10:05:38.192702+0000","last_peered":"2024-04-25T10:05:38.192702+0000","last_clean":"2024-04-25T10:05:38.192702+0000","last_became_active":"2024-04-25T10:05:36.183892+0000","last_became_peered":"2024-04-25T10:05:36.183892+0000","last_unstale":"2024-04-25T10:05:38.192702+0000","last_undegraded":"2024-04-25T10:05:38.192702+0000","last_fullsized":"2024-04-25T10:05:38.192702+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:48.144855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.191335+0000","last_change":"2024-04-25T10:05:37.235328+0000","last_active":"2024-04-25T10:05:38.191335+0000","last_peered":"2024-04-25T10:05:38.191335+0000","last_clean":"2024-04-25T10:05:38.191335+0000","last_became_active":"2024-04-25T10:05:37.234620+0000","last_became_peered":"2024-04-25T10:05:37.234620+0000","last_unstale":"2024-04-25T10:05:38.191335+0000","last_undegraded":"2024-04-25T10:05:38.191335+0000","last_fullsized":"2024-04-25T10:05:38.191335+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:36.155259+0000","last_clean_scrub_stamp":"2024-04-25T10:05:36.155259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:25.148179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:39.205026+0000","last_change":"2024-04-25T10:05:36.201261+0000","last_active":"2024-04-25T10:05:39.205026+0000","last_peered":"2024-04-25T10:05:39.205026+0000","last_clean":"2024-04-25T10:05:39.205026+0000","last_became_active":"2024-04-25T10:05:36.199919+0000","last_became_peered":"2024-04-25T10:05:36.199919+0000","last_unstale":"2024-04-25T10:05:39.205026+0000","last_undegraded":"2024-04-25T10:05:39.205026+0000","last_fullsized":"2024-04-25T10:05:39.205026+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:35.144764+0000","last_clean_scrub_stamp":"2024-04-25T10:05:35.144764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:37.214569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:05:38.549551+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:05:38.549551+0000","last_peered":"2024-04-25T10:05:38.549551+0000","last_clean":"2024-04-25T10:05:38.549551+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:05:38.549551+0000","last_undegraded":"2024-04-25T10:05:38.549551+0000","last_fullsized":"2024-04-25T10:05:38.549551+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46284,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325556,"statfs":{"total":96636764160,"available":96589369344,"internally_reserved":0,"allocated":1245184,"data_stored":879702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":651264,"data_stored":289862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":643072,"data_stored":289210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45700,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326140,"statfs":{"total":96636764160,"available":96589967360,"internally_reserved":0,"allocated":647168,"data_stored":289792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44308,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327532,"statfs":{"total":96636764160,"available":96591392768,"internally_reserved":0,"allocated":659456,"data_stored":290844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":655360,"data_stored":290822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":651264,"data_stored":289370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50452,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321388,"statfs":{"total":96636764160,"available":96585101312,"internally_reserved":0,"allocated":1245184,"data_stored":879634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:45.020 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:05:45.021 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:05:45.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcadd51f640 1 Processor -- start 2024-04-25T10:05:45.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcadd51f640 1 -- start start 2024-04-25T10:05:45.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcadd51f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcad8145170 0x7fcad806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcadd51f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad806e4d0 0x7fcad80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcadd51f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcad80cdd60 0x7fcad815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcadd51f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad806db00 con 0x7fcad80cdd60 2024-04-25T10:05:45.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcadd51f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad8145550 con 0x7fcad8145170 2024-04-25T10:05:45.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcadd51f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80ce2e0 con 0x7fcad806e4d0 2024-04-25T10:05:45.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcad77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcad80cdd60 0x7fcad815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcad77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcad80cdd60 0x7fcad815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43652/0 (socket says 172.21.15.133:43652) 2024-04-25T10:05:45.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcad77fe640 1 -- 172.21.15.133:0/188936959 learned_addr learned my addr 172.21.15.133:0/188936959 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:45.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcad77fe640 1 -- 172.21.15.133:0/188936959 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad806e4d0 msgr2=0x7fcad80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:45.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcad77fe640 1 --2- 172.21.15.133:0/188936959 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad806e4d0 0x7fcad80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcad77fe640 1 -- 172.21.15.133:0/188936959 >> v2:172.21.15.162:3300/0 conn(0x7fcad8145170 msgr2=0x7fcad806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:45.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcad77fe640 1 --2- 172.21.15.133:0/188936959 >> v2:172.21.15.162:3300/0 conn(0x7fcad8145170 0x7fcad806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.182+0000 7fcad77fe640 1 -- 172.21.15.133:0/188936959 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad806d4b0 con 0x7fcad80cdd60 2024-04-25T10:05:45.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad77fe640 1 --2- 172.21.15.133:0/188936959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad80cdd60 0x7fcad815ced0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcad806b790 tx=0x7fcad001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546a40ffc5fb2730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad5ffb640 1 -- 172.21.15.133:0/188936959 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00a5670 con 0x7fcad80cdd60 2024-04-25T10:05:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad5ffb640 1 -- 172.21.15.133:0/188936959 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad00a5810 con 0x7fcad80cdd60 2024-04-25T10:05:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad5ffb640 1 -- 172.21.15.133:0/188936959 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00a5be0 con 0x7fcad80cdd60 2024-04-25T10:05:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 -- 172.21.15.133:0/188936959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad80cdd60 msgr2=0x7fcad815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 --2- 172.21.15.133:0/188936959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad80cdd60 0x7fcad815ced0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcad806b790 tx=0x7fcad001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 -- 172.21.15.133:0/188936959 shutdown_connections 2024-04-25T10:05:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 --2- 172.21.15.133:0/188936959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad80cdd60 0x7fcad815ced0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 --2- 172.21.15.133:0/188936959 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad806e4d0 0x7fcad80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 --2- 172.21.15.133:0/188936959 >> v2:172.21.15.162:3300/0 conn(0x7fcad8145170 0x7fcad806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 -- 172.21.15.133:0/188936959 >> 172.21.15.133:0/188936959 conn(0x7fcad8066f70 msgr2=0x7fcad806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 -- 172.21.15.133:0/188936959 shutdown_connections 2024-04-25T10:05:45.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 -- 172.21.15.133:0/188936959 wait complete. 2024-04-25T10:05:45.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 Processor -- start 2024-04-25T10:05:45.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 -- start start 2024-04-25T10:05:45.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcad806e4d0 0x7fcad8138160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad80cdd60 0x7fcad813d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad8145170 0x7fcad813c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad815f690 con 0x7fcad8145170 2024-04-25T10:05:45.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad815f510 con 0x7fcad806e4d0 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcadd51f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad815f810 con 0x7fcad80cdd60 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad8145170 0x7fcad813c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad8145170 0x7fcad813c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43664/0 (socket says 172.21.15.133:43664) 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad77fe640 1 -- 172.21.15.133:0/2206861781 learned_addr learned my addr 172.21.15.133:0/2206861781 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad67fc640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad80cdd60 0x7fcad813d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad6ffd640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcad806e4d0 0x7fcad8138160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad77fe640 1 -- 172.21.15.133:0/2206861781 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad80cdd60 msgr2=0x7fcad813d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad77fe640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad80cdd60 0x7fcad813d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad77fe640 1 -- 172.21.15.133:0/2206861781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcad806e4d0 msgr2=0x7fcad8138160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad77fe640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcad806e4d0 0x7fcad8138160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad77fe640 1 -- 172.21.15.133:0/2206861781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad813bcd0 con 0x7fcad8145170 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad6ffd640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcad806e4d0 0x7fcad8138160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:45.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.186+0000 7fcad77fe640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad8145170 0x7fcad813c460 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcad001b0a0 tx=0x7fcad0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.190+0000 7fcabf7fe640 1 -- 172.21.15.133:0/2206861781 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00bc020 con 0x7fcad8145170 2024-04-25T10:05:45.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.190+0000 7fcabf7fe640 1 -- 172.21.15.133:0/2206861781 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad00b8690 con 0x7fcad8145170 2024-04-25T10:05:45.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.190+0000 7fcadd51f640 1 -- 172.21.15.133:0/2206861781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad815dfd0 con 0x7fcad8145170 2024-04-25T10:05:45.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.190+0000 7fcabf7fe640 1 -- 172.21.15.133:0/2206861781 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00b8a80 con 0x7fcad8145170 2024-04-25T10:05:45.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.190+0000 7fcadd51f640 1 -- 172.21.15.133:0/2206861781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad815e2b0 con 0x7fcad8145170 2024-04-25T10:05:45.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.190+0000 7fcadd51f640 1 -- 172.21.15.133:0/2206861781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad8068e80 con 0x7fcad8145170 2024-04-25T10:05:45.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.190+0000 7fcabf7fe640 1 -- 172.21.15.133:0/2206861781 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcad00bf020 con 0x7fcad8145170 2024-04-25T10:05:45.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.190+0000 7fcabf7fe640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcac4066f60 0x7fcac4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.190+0000 7fcabf7fe640 1 -- 172.21.15.133:0/2206861781 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcad0120700 con 0x7fcad8145170 2024-04-25T10:05:45.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.190+0000 7fcad6ffd640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcac4066f60 0x7fcac4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.190+0000 7fcad6ffd640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcac4066f60 0x7fcac4069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcad8144b00 tx=0x7fcac8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.194+0000 7fcabf7fe640 1 -- 172.21.15.133:0/2206861781 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcad00c6030 con 0x7fcad8145170 2024-04-25T10:05:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.402+0000 7fcadd51f640 1 -- 172.21.15.133:0/2206861781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcad815e910 con 0x7fcad8145170 2024-04-25T10:05:45.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.406+0000 7fcabf7fe640 1 -- 172.21.15.133:0/2206861781 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fcad00165e0 con 0x7fcad8145170 2024-04-25T10:05:45.411 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:05:45.411 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:05:45.411 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:05:45.411 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:05:45.411 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:45.411 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 -- 172.21.15.133:0/2206861781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcac4066f60 msgr2=0x7fcac4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcac4066f60 0x7fcac4069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcad8144b00 tx=0x7fcac8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 -- 172.21.15.133:0/2206861781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad8145170 msgr2=0x7fcad813c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad8145170 0x7fcad813c460 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcad001b0a0 tx=0x7fcad0019790 comp rx=0 tx=0).stop 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 -- 172.21.15.133:0/2206861781 shutdown_connections 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcac4066f60 0x7fcac4069420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad8145170 0x7fcad813c460 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad80cdd60 0x7fcad813d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 --2- 172.21.15.133:0/2206861781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcad806e4d0 0x7fcad8138160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 -- 172.21.15.133:0/2206861781 >> 172.21.15.133:0/2206861781 conn(0x7fcad8066f70 msgr2=0x7fcad8152e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 -- 172.21.15.133:0/2206861781 shutdown_connections 2024-04-25T10:05:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.410+0000 7fcadd51f640 1 -- 172.21.15.133:0/2206861781 wait complete. 2024-04-25T10:05:45.430 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:45.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.530+0000 7f34474ad640 1 Processor -- start 2024-04-25T10:05:45.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.530+0000 7f34474ad640 1 -- start start 2024-04-25T10:05:45.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f34474ad640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f34400b1f70 0x7f344006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f34474ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400ce330 0x7f34400ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f34474ad640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34400c61d0 con 0x7f344015c9a0 2024-04-25T10:05:45.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f34474ad640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34400cecc0 con 0x7f34400b1f70 2024-04-25T10:05:45.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f34474ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34400c64b0 con 0x7f34400ce330 2024-04-25T10:05:45.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f3445249640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f344015c9a0 0x7f34400c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40166/0 (socket says 172.21.15.133:40166) 2024-04-25T10:05:45.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f3444a48640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f34400b1f70 0x7f344006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f3445249640 1 -- 172.21.15.133:0/2594756047 learned_addr learned my addr 172.21.15.133:0/2594756047 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f3444a48640 1 -- 172.21.15.133:0/2594756047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400ce330 msgr2=0x7f34400ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f3444a48640 1 --2- 172.21.15.133:0/2594756047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400ce330 0x7f34400ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f3444a48640 1 -- 172.21.15.133:0/2594756047 >> v1:172.21.15.133:6789/0 conn(0x7f344015c9a0 legacy=0x7f34400c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f3444a48640 1 -- 172.21.15.133:0/2594756047 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344006d4f0 con 0x7f34400b1f70 2024-04-25T10:05:45.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f3444a48640 1 --2- 172.21.15.133:0/2594756047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34400b1f70 0x7f344006dcf0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f34300097c0 tx=0x7f343001c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34de3ce0816533ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f3437fff640 1 -- 172.21.15.133:0/2594756047 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300b0070 con 0x7f34400b1f70 2024-04-25T10:05:45.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f3437fff640 1 -- 172.21.15.133:0/2594756047 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34300ab600 con 0x7f34400b1f70 2024-04-25T10:05:45.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.534+0000 7f3437fff640 1 -- 172.21.15.133:0/2594756047 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300ab9d0 con 0x7f34400b1f70 2024-04-25T10:05:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 -- 172.21.15.133:0/2594756047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34400b1f70 msgr2=0x7f344006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 --2- 172.21.15.133:0/2594756047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34400b1f70 0x7f344006dcf0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f34300097c0 tx=0x7f343001c300 comp rx=0 tx=0).stop 2024-04-25T10:05:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 -- 172.21.15.133:0/2594756047 shutdown_connections 2024-04-25T10:05:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 --2- 172.21.15.133:0/2594756047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400ce330 0x7f34400ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 --2- 172.21.15.133:0/2594756047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34400b1f70 0x7f344006dcf0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 -- 172.21.15.133:0/2594756047 >> 172.21.15.133:0/2594756047 conn(0x7f3440067000 msgr2=0x7f344006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 -- 172.21.15.133:0/2594756047 shutdown_connections 2024-04-25T10:05:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 -- 172.21.15.133:0/2594756047 wait complete. 2024-04-25T10:05:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 Processor -- start 2024-04-25T10:05:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 -- start start 2024-04-25T10:05:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34400b1f70 0x7f344014cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34400ce330 0x7f344014b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f344015c9a0 0x7f344014d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344015dad0 con 0x7f34400ce330 2024-04-25T10:05:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344015d950 con 0x7f34400b1f70 2024-04-25T10:05:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344015dc50 con 0x7f344015c9a0 2024-04-25T10:05:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3444a48640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34400ce330 0x7f344014b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3444a48640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34400ce330 0x7f344014b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43682/0 (socket says 172.21.15.133:43682) 2024-04-25T10:05:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3444a48640 1 -- 172.21.15.133:0/3434443197 learned_addr learned my addr 172.21.15.133:0/3434443197 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3444a48640 1 -- 172.21.15.133:0/3434443197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f344015c9a0 msgr2=0x7f344014d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3444a48640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f344015c9a0 0x7f344014d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3444a48640 1 -- 172.21.15.133:0/3434443197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34400b1f70 msgr2=0x7f344014cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3444a48640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34400b1f70 0x7f344014cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3444a48640 1 -- 172.21.15.133:0/3434443197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34401543c0 con 0x7f34400ce330 2024-04-25T10:05:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3444a48640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34400ce330 0x7f344014b470 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f343001a9b0 tx=0x7f34300190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3435ffb640 1 -- 172.21.15.133:0/3434443197 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300b8020 con 0x7f34400ce330 2024-04-25T10:05:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3435ffb640 1 -- 172.21.15.133:0/3434443197 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34300b4610 con 0x7f34400ce330 2024-04-25T10:05:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 -- 172.21.15.133:0/3434443197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3440152220 con 0x7f34400ce330 2024-04-25T10:05:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f3435ffb640 1 -- 172.21.15.133:0/3434443197 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300b0040 con 0x7f34400ce330 2024-04-25T10:05:45.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.538+0000 7f34474ad640 1 -- 172.21.15.133:0/3434443197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3440154920 con 0x7f34400ce330 2024-04-25T10:05:45.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.542+0000 7f34474ad640 1 -- 172.21.15.133:0/3434443197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3440068ef0 con 0x7f34400ce330 2024-04-25T10:05:45.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.542+0000 7f3435ffb640 1 -- 172.21.15.133:0/3434443197 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34300bb020 con 0x7f34400ce330 2024-04-25T10:05:45.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.542+0000 7f3435ffb640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3424066f60 0x7f3424069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:45.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.542+0000 7f3435ffb640 1 -- 172.21.15.133:0/3434443197 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f343011ee40 con 0x7f34400ce330 2024-04-25T10:05:45.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.542+0000 7f3445249640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3424066f60 0x7f3424069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:45.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.542+0000 7f3445249640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3424066f60 0x7f3424069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34401502f0 tx=0x7f343c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:45.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.546+0000 7f3435ffb640 1 -- 172.21.15.133:0/3434443197 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f34300c2030 con 0x7f34400ce330 2024-04-25T10:05:45.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.722+0000 7f34474ad640 1 -- 172.21.15.133:0/3434443197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f344006df00 con 0x7f34400ce330 2024-04-25T10:05:45.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f3435ffb640 1 -- 172.21.15.133:0/3434443197 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f34401543c0 con 0x7f34400ce330 2024-04-25T10:05:45.729 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:05:45.729 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:05:38.566660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:45.729 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 -- 172.21.15.133:0/3434443197 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3424066f60 msgr2=0x7f3424069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3424066f60 0x7f3424069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34401502f0 tx=0x7f343c0079b0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 -- 172.21.15.133:0/3434443197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34400ce330 msgr2=0x7f344014b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34400ce330 0x7f344014b470 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f343001a9b0 tx=0x7f34300190a0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 -- 172.21.15.133:0/3434443197 shutdown_connections 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f344015c9a0 0x7f344014d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3424066f60 0x7f3424069420 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34400ce330 0x7f344014b470 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 --2- 172.21.15.133:0/3434443197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34400b1f70 0x7f344014cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 -- 172.21.15.133:0/3434443197 >> 172.21.15.133:0/3434443197 conn(0x7f3440067000 msgr2=0x7f34400c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 -- 172.21.15.133:0/3434443197 shutdown_connections 2024-04-25T10:05:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:45.726+0000 7f34474ad640 1 -- 172.21.15.133:0/3434443197 wait complete. 2024-04-25T10:05:46.149 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:36.528839+0000', 'modified': '2024-04-25T10:05:38.566660+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6012}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6012': {'gid': 6012, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/1425600691', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 1425600691}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 1425600691}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:05:46.150 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:05:46.150 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:05:46.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99e5b97640 1 Processor -- start 2024-04-25T10:05:46.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99e5b97640 1 -- start start 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99e5b97640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f99e015db20 0x7f99e00d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99e5b97640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e00cfaf0 con 0x7f99e015df00 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99e5b97640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e00cfdd0 con 0x7f99e009a710 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99e5b97640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e00d00b0 con 0x7f99e015db20 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99deffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f99e015df00 0x7f99e006c9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40406/0 (socket says 172.21.15.133:40406) 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99deffd640 1 -- 172.21.15.133:0/2728498390 learned_addr learned my addr 172.21.15.133:0/2728498390 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2728498390 >> v2:172.21.15.162:3300/0 conn(0x7f99e015db20 0x7f99e00d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336872398 0 0) 0x7f99e00cfaf0 con 0x7f99e015df00 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99cc003530 con 0x7f99e015df00 2024-04-25T10:05:46.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2732702050 0 0) 0x7f99cc003530 con 0x7f99e015df00 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 >> v2:172.21.15.162:3300/0 conn(0x7f99e015db20 msgr2=0x7f99e00d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 --2- 172.21.15.133:0/2728498390 >> v2:172.21.15.162:3300/0 conn(0x7f99e015db20 0x7f99e00d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 >> v1:172.21.15.162:6789/0 conn(0x7f99e009a710 legacy=0x7f99e00999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00d0390 con 0x7f99e015df00 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f99c8001680 con 0x7f99e015df00 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 >> v1:172.21.15.133:6789/0 conn(0x7f99e015df00 legacy=0x7f99e006c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 0x7f99cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99cc004810 0x7f99cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99cc0051d0 0x7f99cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e00d00b0 con 0x7f99cc0039c0 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc00c170 con 0x7f99cc004810 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc00c370 con 0x7f99cc0051d0 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.278+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 0x7f99cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99deffd640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99cc004810 0x7f99cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99df7fe640 1 -- 172.21.15.133:0/2728498390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99cc0051d0 msgr2=0x7f99cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99dffff640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99cc0051d0 0x7f99cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99cc0051d0 0x7f99cc00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99df7fe640 1 -- 172.21.15.133:0/2728498390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99cc004810 msgr2=0x7f99cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99cc004810 0x7f99cc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99df7fe640 1 -- 172.21.15.133:0/2728498390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc003530 con 0x7f99cc0039c0 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99dffff640 1 -- 172.21.15.133:0/2728498390 reap_dead start 2024-04-25T10:05:46.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99deffd640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99cc004810 0x7f99cc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:46.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 0x7f99cc003de0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009fc0 tx=0x7f99d801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4933aa88993476a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d80b2070 con 0x7f99cc0039c0 2024-04-25T10:05:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d80a5a90 con 0x7f99cc0039c0 2024-04-25T10:05:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99de7fc640 1 -- 172.21.15.133:0/2728498390 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8009c10 con 0x7f99cc0039c0 2024-04-25T10:05:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99e5b97640 1 -- 172.21.15.133:0/2728498390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 msgr2=0x7f99cc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99e5b97640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 0x7f99cc003de0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009fc0 tx=0x7f99d801cbc0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99e5b97640 1 -- 172.21.15.133:0/2728498390 shutdown_connections 2024-04-25T10:05:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99e5b97640 1 --2- 172.21.15.133:0/2728498390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 0x7f99cc003de0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99e5b97640 1 -- 172.21.15.133:0/2728498390 >> 172.21.15.133:0/2728498390 conn(0x7f99e00b3170 msgr2=0x7f99e00b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99e5b97640 1 -- 172.21.15.133:0/2728498390 shutdown_connections 2024-04-25T10:05:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.282+0000 7f99e5b97640 1 -- 172.21.15.133:0/2728498390 wait complete. 2024-04-25T10:05:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99e5b97640 1 Processor -- start 2024-04-25T10:05:46.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99e5b97640 1 -- start start 2024-04-25T10:05:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99e5b97640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 0x7f99e0156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99e5b97640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e00c77e0 0x7f99e00c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99e5b97640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e015fa10 0x7f99e015fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99e5b97640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e006cdd0 con 0x7f99cc0039c0 2024-04-25T10:05:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99e5b97640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e006cf50 con 0x7f99e015fa10 2024-04-25T10:05:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99e5b97640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e00cbcd0 con 0x7f99e00c77e0 2024-04-25T10:05:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99df7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 0x7f99e0156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99df7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 0x7f99e0156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60164/0 (socket says 172.21.15.133:60164) 2024-04-25T10:05:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99df7fe640 1 -- 172.21.15.133:0/2184668535 learned_addr learned my addr 172.21.15.133:0/2184668535 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99deffd640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e00c77e0 0x7f99e00c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99dffff640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e015fa10 0x7f99e015fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99df7fe640 1 -- 172.21.15.133:0/2184668535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e00c77e0 msgr2=0x7f99e00c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e00c77e0 0x7f99e00c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99df7fe640 1 -- 172.21.15.133:0/2184668535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e015fa10 msgr2=0x7f99e015fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e015fa10 0x7f99e015fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99df7fe640 1 -- 172.21.15.133:0/2184668535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e01f7570 con 0x7f99cc0039c0 2024-04-25T10:05:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 0x7f99e0156930 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f99d80b0040 tx=0x7f99d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99d7fff640 1 -- 172.21.15.133:0/2184668535 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d80b2040 con 0x7f99cc0039c0 2024-04-25T10:05:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99d7fff640 1 -- 172.21.15.133:0/2184668535 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d80a5890 con 0x7f99cc0039c0 2024-04-25T10:05:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99d7fff640 1 -- 172.21.15.133:0/2184668535 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d80adce0 con 0x7f99cc0039c0 2024-04-25T10:05:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99e5b97640 1 -- 172.21.15.133:0/2184668535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e0160780 con 0x7f99cc0039c0 2024-04-25T10:05:46.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99e5b97640 1 -- 172.21.15.133:0/2184668535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e01f7ae0 con 0x7f99cc0039c0 2024-04-25T10:05:46.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.286+0000 7f99d7fff640 1 -- 172.21.15.133:0/2184668535 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99d80bf030 con 0x7f99cc0039c0 2024-04-25T10:05:46.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99d7fff640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99c0066f60 0x7f99c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99d7fff640 1 -- 172.21.15.133:0/2184668535 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99d811f480 con 0x7f99cc0039c0 2024-04-25T10:05:46.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99deffd640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99c0066f60 0x7f99c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99deffd640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99c0066f60 0x7f99c0069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f99c80017f0 tx=0x7f99c8001b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 Processor -- start 2024-04-25T10:05:46.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 -- start start 2024-04-25T10:05:46.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b370920 0x55da7b370d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b371240 0x55da7b3269f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b327310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7b20e5f0 con 0x55da7b326f30 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7b175da0 con 0x55da7b371240 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da7b0fe150 con 0x55da7b370920 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99dffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b327310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99dffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b327310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60170/0 (socket says 172.21.15.133:60170) 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99dffff640 1 -- 172.21.15.133:0/774347204 learned_addr learned my addr 172.21.15.133:0/774347204 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99deffd640 1 --2- 172.21.15.133:0/774347204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b371240 0x55da7b3269f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99df7fe640 1 --2- 172.21.15.133:0/774347204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b370920 0x55da7b370d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99dffff640 1 -- 172.21.15.133:0/774347204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b370920 msgr2=0x55da7b370d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99dffff640 1 --2- 172.21.15.133:0/774347204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b370920 0x55da7b370d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99dffff640 1 -- 172.21.15.133:0/774347204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b371240 msgr2=0x55da7b3269f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99dffff640 1 --2- 172.21.15.133:0/774347204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b371240 0x55da7b3269f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99dffff640 1 -- 172.21.15.133:0/774347204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da7b117ca0 con 0x55da7b326f30 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99df7fe640 1 --2- 172.21.15.133:0/774347204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b370920 0x55da7b370d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99dffff640 1 --2- 172.21.15.133:0/774347204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b327310 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f99e015e2e0 tx=0x7f99d0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2ccdb7683a13b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99d57fa640 1 -- 172.21.15.133:0/774347204 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d00050d0 con 0x55da7b326f30 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99d57fa640 1 -- 172.21.15.133:0/774347204 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d0018070 con 0x55da7b326f30 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99d57fa640 1 -- 172.21.15.133:0/774347204 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0005470 con 0x55da7b326f30 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 -- 172.21.15.133:0/774347204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 msgr2=0x55da7b327310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 --2- 172.21.15.133:0/774347204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b327310 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f99e015e2e0 tx=0x7f99d0001a80 comp rx=0 tx=0).stop 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 -- 172.21.15.133:0/774347204 shutdown_connections 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 --2- 172.21.15.133:0/774347204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b327310 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 --2- 172.21.15.133:0/774347204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b371240 0x55da7b3269f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 --2- 172.21.15.133:0/774347204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b370920 0x55da7b370d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 -- 172.21.15.133:0/774347204 >> 172.21.15.133:0/774347204 conn(0x55da7b32d2a0 msgr2=0x55da7b3252d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 -- 172.21.15.133:0/774347204 shutdown_connections 2024-04-25T10:05:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.290+0000 7f99e8d04000 1 -- 172.21.15.133:0/774347204 wait complete. 2024-04-25T10:05:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- start start 2024-04-25T10:05:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 Processor -- start 2024-04-25T10:05:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b326f30 0x55da7b377c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b3781c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b371240 0x55da7b32aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7b130990 con 0x55da7b370920 2024-04-25T10:05:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7b171e40 con 0x55da7b326f30 2024-04-25T10:05:46.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da7b101900 con 0x55da7b371240 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 10 client.0 _ll_register_callbacks cb 0x55da7b2291c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99deffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b3781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99deffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b3781c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60172/0 (socket says 172.21.15.133:60172) 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99deffd640 1 -- 172.21.15.133:0/2700642328 learned_addr learned my addr 172.21.15.133:0/2700642328 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b326f30 0x55da7b377c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99dffff640 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b371240 0x55da7b32aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99deffd640 1 -- 172.21.15.133:0/2700642328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b371240 msgr2=0x55da7b32aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99deffd640 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b371240 0x55da7b32aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99deffd640 1 -- 172.21.15.133:0/2700642328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b326f30 msgr2=0x55da7b377c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99deffd640 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b326f30 0x55da7b377c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99deffd640 1 -- 172.21.15.133:0/2700642328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da7b12e630 con 0x55da7b370920 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99deffd640 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b3781c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f99e0062830 tx=0x7f99c8001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821ff9a079fd9aea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99be7fc640 1 -- 172.21.15.133:0/2700642328 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8012560 con 0x55da7b370920 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99be7fc640 1 -- 172.21.15.133:0/2700642328 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99c8010040 con 0x55da7b370920 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- 172.21.15.133:0/2700642328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99d0018070 con 0x55da7b370920 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99be7fc640 1 -- 172.21.15.133:0/2700642328 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99c8012700 con 0x55da7b370920 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 10 client.6192 fetch_fsmap learned FSMap version 23 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- 172.21.15.133:0/2700642328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55da7b0fe3a0 con 0x55da7b370920 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99be7fc640 1 -- 172.21.15.133:0/2700642328 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d0018070 con 0x55da7b370920 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99be7fc640 1 -- 172.21.15.133:0/2700642328 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99c80128a0 con 0x55da7b370920 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 10 client.6192 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 10 client.6192 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 20 client.6192 populate_metadata read hostname 'smithi133' 2024-04-25T10:05:46.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55da7b32c160 0x55da7b381ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 4 client.6192 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- 172.21.15.133:0/2700642328 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da7b111a50 con 0x55da7b32c160 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55da7b32c160 0x55da7b381ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55da7b32c160 0x55da7b381ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4dd660573380afc server_cookie=6ae031e49e580f6c in_seq=0 out_seq=0 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99be7fc640 10 client.6192 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99be7fc640 1 -- 172.21.15.133:0/2700642328 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55da7b111a50 con 0x55da7b32c160 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99be7fc640 10 client.6192 handle_command_reply: tid=0 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 client.6192 shutdown 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- 172.21.15.133:0/2700642328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 msgr2=0x55da7b3781c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b3781c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f99e0062830 tx=0x7f99c8001a50 comp rx=0 tx=0).stop 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- 172.21.15.133:0/2700642328 shutdown_connections 2024-04-25T10:05:46.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55da7b32c160 0x55da7b381ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b371240 0x55da7b32aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99be7fc640 0 client.6192 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:05:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b3781c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 --2- 172.21.15.133:0/2700642328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b326f30 0x55da7b377c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- 172.21.15.133:0/2700642328 >> 172.21.15.133:0/2700642328 conn(0x55da7b32d2a0 msgr2=0x55da7b3259e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- 172.21.15.133:0/2700642328 shutdown_connections 2024-04-25T10:05:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 1 -- 172.21.15.133:0/2700642328 wait complete. 2024-04-25T10:05:46.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.294+0000 7f99e8d04000 20 client.6192 trim_cache size 0 max 16384 2024-04-25T10:05:46.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99e8d04000 1 Processor -- start 2024-04-25T10:05:46.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99e8d04000 1 -- start start 2024-04-25T10:05:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b32a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b370920 0x55da7b32a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b371240 0x55da7b3797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99e8d04000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0203280 con 0x55da7b326f30 2024-04-25T10:05:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99e8d04000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e01fb5a0 con 0x55da7b370920 2024-04-25T10:05:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99e8d04000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0202d50 con 0x55da7b371240 2024-04-25T10:05:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99df7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b32a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99df7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b32a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60188/0 (socket says 172.21.15.133:60188) 2024-04-25T10:05:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99df7fe640 1 -- 172.21.15.133:0/2823341297 learned_addr learned my addr 172.21.15.133:0/2823341297 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:46.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99df7fe640 1 -- 172.21.15.133:0/2823341297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b371240 msgr2=0x55da7b3797e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2823341297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b371240 0x55da7b3797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99df7fe640 1 -- 172.21.15.133:0/2823341297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b370920 msgr2=0x55da7b32a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2823341297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b370920 0x55da7b32a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.310+0000 7f99df7fe640 1 -- 172.21.15.133:0/2823341297 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da7b371070 con 0x55da7b326f30 2024-04-25T10:05:46.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99df7fe640 1 --2- 172.21.15.133:0/2823341297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b32a2c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f99d801c630 tx=0x7f99d811f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba1c09d4253aa9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99d4ff9640 1 -- 172.21.15.133:0/2823341297 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d80a5a60 con 0x55da7b326f30 2024-04-25T10:05:46.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99d4ff9640 1 -- 172.21.15.133:0/2823341297 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d80efc90 con 0x55da7b326f30 2024-04-25T10:05:46.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99d4ff9640 1 -- 172.21.15.133:0/2823341297 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d80f3de0 con 0x55da7b326f30 2024-04-25T10:05:46.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- 172.21.15.133:0/2823341297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 msgr2=0x55da7b32a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 --2- 172.21.15.133:0/2823341297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b32a2c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f99d801c630 tx=0x7f99d811f040 comp rx=0 tx=0).stop 2024-04-25T10:05:46.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- 172.21.15.133:0/2823341297 shutdown_connections 2024-04-25T10:05:46.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 --2- 172.21.15.133:0/2823341297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b371240 0x55da7b3797e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 --2- 172.21.15.133:0/2823341297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b370920 0x55da7b32a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 --2- 172.21.15.133:0/2823341297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b326f30 0x55da7b32a2c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- 172.21.15.133:0/2823341297 >> 172.21.15.133:0/2823341297 conn(0x55da7b32d2a0 msgr2=0x55da7b329bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- 172.21.15.133:0/2823341297 shutdown_connections 2024-04-25T10:05:46.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- 172.21.15.133:0/2823341297 wait complete. 2024-04-25T10:05:46.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- start start 2024-04-25T10:05:46.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 Processor -- start 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b326f30 0x55da7b37cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b37d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b371240 0x55da7b38b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7b17a3d0 con 0x55da7b370920 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7b231d50 con 0x55da7b371240 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da7b110980 con 0x55da7b326f30 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 10 client.0 _ll_register_callbacks cb 0x7f99d0007b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99deffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b37d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99deffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b37d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60190/0 (socket says 172.21.15.133:60190) 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99deffd640 1 -- 172.21.15.133:0/160136690 learned_addr learned my addr 172.21.15.133:0/160136690 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99deffd640 1 -- 172.21.15.133:0/160136690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b326f30 msgr2=0x55da7b37cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99deffd640 1 --2- 172.21.15.133:0/160136690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b326f30 0x55da7b37cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99deffd640 1 -- 172.21.15.133:0/160136690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b371240 msgr2=0x55da7b38b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99deffd640 1 --2- 172.21.15.133:0/160136690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b371240 0x55da7b38b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99deffd640 1 -- 172.21.15.133:0/160136690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da7b1f7600 con 0x55da7b370920 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99deffd640 1 --2- 172.21.15.133:0/160136690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b37d310 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99c800fbf0 tx=0x7f99c800b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46351dbff995281d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99be7fc640 1 -- 172.21.15.133:0/160136690 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c803b060 con 0x55da7b370920 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99be7fc640 1 -- 172.21.15.133:0/160136690 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99c8036ca0 con 0x55da7b370920 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- 172.21.15.133:0/160136690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99d80efc90 con 0x55da7b370920 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99be7fc640 1 -- 172.21.15.133:0/160136690 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99c8012560 con 0x55da7b370920 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 10 client.6198 fetch_fsmap learned FSMap version 23 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- 172.21.15.133:0/160136690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55da7b10a9b0 con 0x55da7b370920 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99be7fc640 1 -- 172.21.15.133:0/160136690 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d80efc90 con 0x55da7b370920 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99be7fc640 1 -- 172.21.15.133:0/160136690 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99c802c620 con 0x55da7b370920 2024-04-25T10:05:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 10 client.6198 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 10 client.6198 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 20 client.6198 populate_metadata read hostname 'smithi133' 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 --2- 172.21.15.133:0/160136690 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55da7b38c160 0x55da7b3d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 4 client.6198 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99e8d04000 1 -- 172.21.15.133:0/160136690 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55da7b3785f0 con 0x55da7b38c160 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99df7fe640 1 --2- 172.21.15.133:0/160136690 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55da7b38c160 0x55da7b3d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99be7fc640 10 client.6198 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99df7fe640 1 --2- 172.21.15.133:0/160136690 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55da7b38c160 0x55da7b3d1420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c8bea2b397fb2e server_cookie=96740311db1d7f13 in_seq=0 out_seq=0 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.314+0000 7f99be7fc640 1 -- 172.21.15.133:0/160136690 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55da7b3785f0 con 0x55da7b38c160 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99be7fc640 10 client.6198 handle_command_reply: tid=0 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 1 client.6198 shutdown 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 1 -- 172.21.15.133:0/160136690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 msgr2=0x55da7b37d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 1 --2- 172.21.15.133:0/160136690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b37d310 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99c800fbf0 tx=0x7f99c800b000 comp rx=0 tx=0).stop 2024-04-25T10:05:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 1 -- 172.21.15.133:0/160136690 shutdown_connections 2024-04-25T10:05:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 1 --2- 172.21.15.133:0/160136690 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55da7b38c160 0x55da7b3d1420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 1 --2- 172.21.15.133:0/160136690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da7b371240 0x55da7b38b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99be7fc640 0 client.6198 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:05:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 1 --2- 172.21.15.133:0/160136690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da7b370920 0x55da7b37d310 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 1 --2- 172.21.15.133:0/160136690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da7b326f30 0x55da7b37cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 1 -- 172.21.15.133:0/160136690 >> 172.21.15.133:0/160136690 conn(0x55da7b32d2a0 msgr2=0x55da7b32a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 1 -- 172.21.15.133:0/160136690 shutdown_connections 2024-04-25T10:05:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 1 -- 172.21.15.133:0/160136690 wait complete. 2024-04-25T10:05:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e8d04000 20 client.6198 trim_cache size 0 max 16384 2024-04-25T10:05:46.327 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 6012, 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 9.131321797, 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 23, 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 54, 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 54, 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 14.229580481999999 2024-04-25T10:05:46.328 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:05:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 -- 172.21.15.133:0/2184668535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99c0066f60 msgr2=0x7f99c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99c0066f60 0x7f99c0069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f99c80017f0 tx=0x7f99c8001b70 comp rx=0 tx=0).stop 2024-04-25T10:05:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 -- 172.21.15.133:0/2184668535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 msgr2=0x7f99e0156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 0x7f99e0156930 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f99d80b0040 tx=0x7f99d8009750 comp rx=0 tx=0).stop 2024-04-25T10:05:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 -- 172.21.15.133:0/2184668535 shutdown_connections 2024-04-25T10:05:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e015fa10 0x7f99e015fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e00c77e0 0x7f99e00c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99c0066f60 0x7f99c0069420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 --2- 172.21.15.133:0/2184668535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc0039c0 0x7f99e0156930 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 -- 172.21.15.133:0/2184668535 >> 172.21.15.133:0/2184668535 conn(0x7f99e00b3170 msgr2=0x7f99e00ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 -- 172.21.15.133:0/2184668535 shutdown_connections 2024-04-25T10:05:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.318+0000 7f99e5b97640 1 -- 172.21.15.133:0/2184668535 wait complete. 2024-04-25T10:05:46.337 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:46.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad178e9640 1 Processor -- start 2024-04-25T10:05:46.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad178e9640 1 -- start start 2024-04-25T10:05:46.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad178e9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fad1015db00 0x7fad100b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad178e9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad100d0770 0x7fad100d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad178e9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad101614c0 0x7fad10161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad178e9640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad100d1110 con 0x7fad100d0770 2024-04-25T10:05:46.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad178e9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad100ceb20 con 0x7fad1015db00 2024-04-25T10:05:46.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad178e9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad100b7740 con 0x7fad101614c0 2024-04-25T10:05:46.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad14e84640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad100d0770 0x7fad100d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad14e84640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad100d0770 0x7fad100d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60198/0 (socket says 172.21.15.133:60198) 2024-04-25T10:05:46.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad14e84640 1 -- 172.21.15.133:0/828467801 learned_addr learned my addr 172.21.15.133:0/828467801 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:46.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad15685640 1 --2- 172.21.15.133:0/828467801 >> v2:172.21.15.162:3300/0 conn(0x7fad1015db00 0x7fad100b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad15e86640 1 --2- 172.21.15.133:0/828467801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad101614c0 0x7fad10161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad14e84640 1 -- 172.21.15.133:0/828467801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad101614c0 msgr2=0x7fad10161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad14e84640 1 --2- 172.21.15.133:0/828467801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad101614c0 0x7fad10161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad14e84640 1 -- 172.21.15.133:0/828467801 >> v2:172.21.15.162:3300/0 conn(0x7fad1015db00 msgr2=0x7fad100b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:46.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad14e84640 1 --2- 172.21.15.133:0/828467801 >> v2:172.21.15.162:3300/0 conn(0x7fad1015db00 0x7fad100b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad14e84640 1 -- 172.21.15.133:0/828467801 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad100b7a20 con 0x7fad100d0770 2024-04-25T10:05:46.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad14e84640 1 --2- 172.21.15.133:0/828467801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad100d0770 0x7fad100d0bd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fad100b53d0 tx=0x7fad0001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5f5fbcb592e8a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad07fff640 1 -- 172.21.15.133:0/828467801 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000aa610 con 0x7fad100d0770 2024-04-25T10:05:46.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad07fff640 1 -- 172.21.15.133:0/828467801 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad000af070 con 0x7fad100d0770 2024-04-25T10:05:46.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.438+0000 7fad07fff640 1 -- 172.21.15.133:0/828467801 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000aa9d0 con 0x7fad100d0770 2024-04-25T10:05:46.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 -- 172.21.15.133:0/828467801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad100d0770 msgr2=0x7fad100d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 --2- 172.21.15.133:0/828467801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad100d0770 0x7fad100d0bd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fad100b53d0 tx=0x7fad0001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 -- 172.21.15.133:0/828467801 shutdown_connections 2024-04-25T10:05:46.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 --2- 172.21.15.133:0/828467801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad101614c0 0x7fad10161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 --2- 172.21.15.133:0/828467801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad100d0770 0x7fad100d0bd0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 --2- 172.21.15.133:0/828467801 >> v2:172.21.15.162:3300/0 conn(0x7fad1015db00 0x7fad100b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 -- 172.21.15.133:0/828467801 >> 172.21.15.133:0/828467801 conn(0x7fad100727c0 msgr2=0x7fad10072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 -- 172.21.15.133:0/828467801 shutdown_connections 2024-04-25T10:05:46.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 -- 172.21.15.133:0/828467801 wait complete. 2024-04-25T10:05:46.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 Processor -- start 2024-04-25T10:05:46.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 -- start start 2024-04-25T10:05:46.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad100d0770 0x7fad101608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad1015db00 0x7fad10158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad101614c0 0x7fad10158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad100b7430 con 0x7fad100d0770 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad100b82c0 con 0x7fad1015db00 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad100d18a0 con 0x7fad101614c0 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad15685640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad100d0770 0x7fad101608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad15685640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad100d0770 0x7fad101608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60210/0 (socket says 172.21.15.133:60210) 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad15685640 1 -- 172.21.15.133:0/2414504261 learned_addr learned my addr 172.21.15.133:0/2414504261 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad15e86640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad101614c0 0x7fad10158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad15685640 1 -- 172.21.15.133:0/2414504261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad101614c0 msgr2=0x7fad10158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad15685640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad101614c0 0x7fad10158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad15685640 1 -- 172.21.15.133:0/2414504261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad1015db00 msgr2=0x7fad10158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad14e84640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad1015db00 0x7fad10158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad15685640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad1015db00 0x7fad10158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad15685640 1 -- 172.21.15.133:0/2414504261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad10140b40 con 0x7fad100d0770 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad15685640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad100d0770 0x7fad101608a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00c5c0 tx=0x7fad0c00ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad05ffb640 1 -- 172.21.15.133:0/2414504261 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c015070 con 0x7fad100d0770 2024-04-25T10:05:46.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad05ffb640 1 -- 172.21.15.133:0/2414504261 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c011800 con 0x7fad100d0770 2024-04-25T10:05:46.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 -- 172.21.15.133:0/2414504261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad10159670 con 0x7fad100d0770 2024-04-25T10:05:46.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad05ffb640 1 -- 172.21.15.133:0/2414504261 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c011bd0 con 0x7fad100d0770 2024-04-25T10:05:46.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.442+0000 7fad178e9640 1 -- 172.21.15.133:0/2414504261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad10159390 con 0x7fad100d0770 2024-04-25T10:05:46.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.446+0000 7fad178e9640 1 -- 172.21.15.133:0/2414504261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad10068ef0 con 0x7fad100d0770 2024-04-25T10:05:46.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.446+0000 7fad05ffb640 1 -- 172.21.15.133:0/2414504261 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad0c018070 con 0x7fad100d0770 2024-04-25T10:05:46.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.446+0000 7fad05ffb640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:46.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.446+0000 7fad05ffb640 1 -- 172.21.15.133:0/2414504261 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fad0c084ed0 con 0x7fad100d0770 2024-04-25T10:05:46.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.446+0000 7fad14e84640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:46.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.446+0000 7fad14e84640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7facf4066f60 0x7facf4069420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fad000ae000 tx=0x7fad00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.450+0000 7fad05ffb640 1 -- 172.21.15.133:0/2414504261 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad0c055de0 con 0x7fad100d0770 2024-04-25T10:05:46.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 -- 172.21.15.133:0/2414504261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fad1015c620 con 0x7fad100d0770 2024-04-25T10:05:46.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad05ffb640 1 -- 172.21.15.133:0/2414504261 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fad0c0556e0 con 0x7fad100d0770 2024-04-25T10:05:46.634 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:46.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 -- 172.21.15.133:0/2414504261 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7facf4066f60 msgr2=0x7facf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7facf4066f60 0x7facf4069420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fad000ae000 tx=0x7fad00008040 comp rx=0 tx=0).stop 2024-04-25T10:05:46.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 -- 172.21.15.133:0/2414504261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad100d0770 msgr2=0x7fad101608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:46.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad100d0770 0x7fad101608a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00c5c0 tx=0x7fad0c00ca80 comp rx=0 tx=0).stop 2024-04-25T10:05:46.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 -- 172.21.15.133:0/2414504261 shutdown_connections 2024-04-25T10:05:46.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad101614c0 0x7fad10158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad1015db00 0x7fad10158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 --2- 172.21.15.133:0/2414504261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad100d0770 0x7fad101608a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:46.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 -- 172.21.15.133:0/2414504261 >> 172.21.15.133:0/2414504261 conn(0x7fad100727c0 msgr2=0x7fad1015ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:46.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 -- 172.21.15.133:0/2414504261 shutdown_connections 2024-04-25T10:05:46.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:46.630+0000 7fad178e9640 1 -- 172.21.15.133:0/2414504261 wait complete. 2024-04-25T10:05:46.651 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:05:46.652 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:46.652 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:05:46.652 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:05:46.652 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:46.652 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:05:46.652 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:05:46.652 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:46.652 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:46.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.653 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:05:46.659 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:46.659 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:46.659 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42285sec preferred_lft 42285sec 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:05:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:46.661 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:46.662 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:05:46.662 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:46.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.662 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:46.711 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:05:46.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.713 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:05:46.764 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:05:46.764 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:46.764 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:05:46.764 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:05:46.764 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:05:46.764 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:05:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:05:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:46.765 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:46.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.765 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:05:46.765 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:05:46.765 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:05:46.765 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:05:46.766 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:05:46.766 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:05:46.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.843 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:05:46.893 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:05:46.893 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:05:46.893 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:46.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:46.894 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:05:46.949 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:05:46.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:46.949 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:46.997 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:46.997 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:46.997 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:47.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:05:47.102 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:47.150 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:05:47.150 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:05:47.150 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:05:47.150 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:05:47.150 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:05:47.150 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:05:47.150 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:05:47.150 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:05:47.150 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:05:47.150 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:05:47.151 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:05:47.152 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:47.152 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:47.152 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:47.152 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:47.152 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:05:47.152 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:47.152 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:47.152 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:47.152 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:47.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:47.153 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:05:47.201 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:05:47.201 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:05:47.246 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:05:47.262 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:47.309 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:05:47.309 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:05:47.310 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:47.311 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:47.312 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:47.312 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:47.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:47.312 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:05:47.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:05:47.318+0000 7fa4d22b8580 -1 init, newargv = 0x563bbd373cc0 newargc=15 2024-04-25T10:05:47.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[53351]: starting ceph client 2024-04-25T10:05:47.328 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[53351]: starting fuse 2024-04-25T10:05:48.432 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:05:48.449 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:48.498 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:05:48.498 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:05:48.498 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:05:48.498 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:05:48.498 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:05:48.498 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:05:48.498 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:05:48.499 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:48.500 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:48.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.501 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:05:48.549 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:05:48.549 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:05:48.549 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:05:48.550 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:05:48.551 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:05:48.551 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:05:48.551 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:05:48.551 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:05:48.551 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:05:48.551 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:05:48.551 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:05:48.551 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:05:48.551 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:05:48.551 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:05:48.551 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:05:48.674 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.53351.asok 2024-04-25T10:05:48.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:05:48.685 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53351.asok 2024-04-25T10:05:48.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.686 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53351.asok status 2024-04-25T10:05:48.836 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:05:48.836 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:05:48.837 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:05:48.837 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:05:48.837 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:05:48.837 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:05:48.837 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:05:48.837 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "53351", 2024-04-25T10:05:48.838 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:05:48.838 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:48.838 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:05:48.838 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:05:48.838 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 6207, 2024-04-25T10:05:48.838 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:05:48.839 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:05:48.839 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:05:48.839 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 6207 2024-04-25T10:05:48.839 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:48.839 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:05:48.839 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:05:48.840 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:05:48.840 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3299924918 2024-04-25T10:05:48.840 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:05:48.840 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:48.840 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:05:48.840 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:05:48.840 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:05:48.841 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3299924918 2024-04-25T10:05:48.841 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:05:48.841 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.6207 192.168.0.1:0/3299924918", 2024-04-25T10:05:48.841 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/3299924918", 2024-04-25T10:05:48.841 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:05:48.841 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 23, 2024-04-25T10:05:48.842 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 54, 2024-04-25T10:05:48.842 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:05:48.842 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:05:48.842 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:05:48.842 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:05:48.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.852 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:05:48.852 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:48.860 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:05:48.860 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:48.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.861 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:48.918 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:48.918 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T10:05:48.918 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T10:05:48.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:48.918 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T10:05:48.974 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-04-25T10:05:48.974 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:05:48.974 DEBUG:teuthology.orchestra.run.smithi133:> import stat 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> import json 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> import sys 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> try: 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> except OSError as e: 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> sys.exit(e.errno) 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> print(json.dumps( 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> indent=2)) 2024-04-25T10:05:48.975 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:05:49.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:05:49 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T10:05:49.095 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T10:05:49.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:49.096 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T10:05:49.156 INFO:teuthology.orchestra.run.smithi133.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T10:05:49.501 INFO:teuthology.orchestra.run.smithi133.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T10:05:50.802 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:50.805 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:59.980 INFO:teuthology.orchestra.run.smithi133.stderr:Updating files: 12% (181/1398) Updating files: 13% (182/1398) Updating files: 14% (196/1398) Updating files: 15% (210/1398) Updating files: 16% (224/1398) Updating files: 17% (238/1398) Updating files: 18% (252/1398) Updating files: 19% (266/1398) Updating files: 20% (280/1398) Updating files: 21% (294/1398) Updating files: 22% (308/1398) Updating files: 23% (322/1398) Updating files: 24% (336/1398) Updating files: 25% (350/1398) Updating files: 26% (364/1398) Updating files: 26% (368/1398) Updating files: 27% (378/1398) Updating files: 28% (392/1398) Updating files: 29% (406/1398) Updating files: 30% (420/1398) Updating files: 31% (434/1398) Updating files: 32% (448/1398) Updating files: 33% (462/1398) Updating files: 34% (476/1398) Updating files: 35% (490/1398) Updating files: 36% (504/1398) Updating files: 37% (518/1398) Updating files: 38% (532/1398) Updating files: 39% (546/1398) Updating files: 40% (560/1398) Updating files: 41% (574/1398) Updating files: 42% (588/1398) Updating files: 43% (602/1398) Updating files: 44% (616/1398) Updating files: 45% (630/1398) Updating files: 46% (644/1398) Updating files: 46% (652/1398) Updating files: 47% (658/1398) Updating files: 48% (672/1398) Updating files: 49% (686/1398) Updating files: 50% (699/1398) Updating files: 51% (713/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 67% (945/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 87% (1224/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T10:05:59.988 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T10:05:59.988 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T10:05:59.988 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:00.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.082+0000 7fcea9019640 1 Processor -- start 2024-04-25T10:06:00.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.082+0000 7fcea9019640 1 -- start start 2024-04-25T10:06:00.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea9019640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcea415db40 0x7fcea40d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea9019640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea40d23c0 con 0x7fcea40b5a60 2024-04-25T10:06:00.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea9019640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea40d26a0 con 0x7fcea40d1f20 2024-04-25T10:06:00.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea9019640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40d2fd0 con 0x7fcea415db40 2024-04-25T10:06:00.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea2d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcea415db40 0x7fcea40d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea2d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcea415db40 0x7fcea40d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50658/0 (socket says 172.21.15.133:50658) 2024-04-25T10:06:00.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea359e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcea40b5a60 0x7fcea40b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60214/0 (socket says 172.21.15.133:60214) 2024-04-25T10:06:00.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea2d9d640 1 -- 172.21.15.133:0/1895417734 learned_addr learned my addr 172.21.15.133:0/1895417734 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:00.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea2d9d640 1 -- 172.21.15.133:0/1895417734 >> v1:172.21.15.162:6789/0 conn(0x7fcea40d1f20 legacy=0x7fcea40b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea2d9d640 1 -- 172.21.15.133:0/1895417734 >> v1:172.21.15.133:6789/0 conn(0x7fcea40b5a60 legacy=0x7fcea40b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea2d9d640 1 -- 172.21.15.133:0/1895417734 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea40d2980 con 0x7fcea415db40 2024-04-25T10:06:00.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea1d9b640 1 -- 172.21.15.133:0/1895417734 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76516057 0 0) 0x7fcea40d23c0 con 0x7fcea40b5a60 2024-04-25T10:06:00.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea2d9d640 1 --2- 172.21.15.133:0/1895417734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea415db40 0x7fcea40d19e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009870 tx=0x7fce9c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c660edacf1a796fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea1d9b640 1 -- 172.21.15.133:0/1895417734 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0ab610 con 0x7fcea415db40 2024-04-25T10:06:00.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea1d9b640 1 -- 172.21.15.133:0/1895417734 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c0b0070 con 0x7fcea415db40 2024-04-25T10:06:00.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea1d9b640 1 -- 172.21.15.133:0/1895417734 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0ab9d0 con 0x7fcea415db40 2024-04-25T10:06:00.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea9019640 1 -- 172.21.15.133:0/1895417734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea415db40 msgr2=0x7fcea40d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea9019640 1 --2- 172.21.15.133:0/1895417734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea415db40 0x7fcea40d19e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009870 tx=0x7fce9c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea9019640 1 -- 172.21.15.133:0/1895417734 shutdown_connections 2024-04-25T10:06:00.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea9019640 1 --2- 172.21.15.133:0/1895417734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea415db40 0x7fcea40d19e0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea9019640 1 -- 172.21.15.133:0/1895417734 >> 172.21.15.133:0/1895417734 conn(0x7fcea40d0350 msgr2=0x7fcea40d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea9019640 1 -- 172.21.15.133:0/1895417734 shutdown_connections 2024-04-25T10:06:00.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.086+0000 7fcea9019640 1 -- 172.21.15.133:0/1895417734 wait complete. 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea9019640 1 Processor -- start 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea9019640 1 -- start start 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea9019640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcea40b5a60 0x7fcea41535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea9019640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcea40d1f20 0x7fcea4154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea9019640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea415db40 0x7fcea40c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea9019640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40ce400 con 0x7fcea415db40 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea9019640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40ce280 con 0x7fcea40b5a60 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea9019640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea416b7a0 con 0x7fcea40d1f20 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea415db40 0x7fcea40c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea415db40 0x7fcea40c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50666/0 (socket says 172.21.15.133:50666) 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea359e640 1 -- 172.21.15.133:0/176177724 learned_addr learned my addr 172.21.15.133:0/176177724 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea2d9d640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcea40b5a60 0x7fcea41535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea259c640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcea40d1f20 0x7fcea4154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea359e640 1 -- 172.21.15.133:0/176177724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcea40d1f20 msgr2=0x7fcea4154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea359e640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcea40d1f20 0x7fcea4154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea359e640 1 -- 172.21.15.133:0/176177724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcea40b5a60 msgr2=0x7fcea41535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea359e640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcea40b5a60 0x7fcea41535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea359e640 1 -- 172.21.15.133:0/176177724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea40b4370 con 0x7fcea415db40 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea259c640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcea40d1f20 0x7fcea4154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea359e640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea415db40 0x7fcea40c97c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fce9400c480 tx=0x7fce9400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fce8b7fe640 1 -- 172.21.15.133:0/176177724 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94018070 con 0x7fcea415db40 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fce8b7fe640 1 -- 172.21.15.133:0/176177724 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce940074e0 con 0x7fcea415db40 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea9019640 1 -- 172.21.15.133:0/176177724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea4155230 con 0x7fcea415db40 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fce8b7fe640 1 -- 172.21.15.133:0/176177724 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94004a30 con 0x7fcea415db40 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea9019640 1 -- 172.21.15.133:0/176177724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea41557f0 con 0x7fcea415db40 2024-04-25T10:06:00.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.090+0000 7fcea9019640 1 -- 172.21.15.133:0/176177724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea4155fb0 con 0x7fcea415db40 2024-04-25T10:06:00.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.094+0000 7fce8b7fe640 1 -- 172.21.15.133:0/176177724 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce9401b070 con 0x7fcea415db40 2024-04-25T10:06:00.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.094+0000 7fce8b7fe640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f90 0x7fce90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.094+0000 7fce8b7fe640 1 -- 172.21.15.133:0/176177724 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce94084380 con 0x7fcea415db40 2024-04-25T10:06:00.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.094+0000 7fcea2d9d640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f90 0x7fce90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.094+0000 7fcea2d9d640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f90 0x7fce90069450 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0af000 tx=0x7fce9c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.098+0000 7fce8b7fe640 1 -- 172.21.15.133:0/176177724 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcea4155fb0 con 0x7fcea415db40 2024-04-25T10:06:00.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.278+0000 7fcea9019640 1 -- 172.21.15.133:0/176177724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcea4068ef0 con 0x7fcea415db40 2024-04-25T10:06:00.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.278+0000 7fce8b7fe640 1 -- 172.21.15.133:0/176177724 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fce94055290 con 0x7fcea415db40 2024-04-25T10:06:00.283 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:00.283 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:05:38.566660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:00.283 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2024-04-25T10:06:00.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 -- 172.21.15.133:0/176177724 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f90 msgr2=0x7fce90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f90 0x7fce90069450 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0af000 tx=0x7fce9c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:06:00.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 -- 172.21.15.133:0/176177724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea415db40 msgr2=0x7fcea40c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea415db40 0x7fcea40c97c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fce9400c480 tx=0x7fce9400c940 comp rx=0 tx=0).stop 2024-04-25T10:06:00.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 -- 172.21.15.133:0/176177724 shutdown_connections 2024-04-25T10:06:00.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f90 0x7fce90069450 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea415db40 0x7fcea40c97c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcea40d1f20 0x7fcea4154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 --2- 172.21.15.133:0/176177724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcea40b5a60 0x7fcea41535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 -- 172.21.15.133:0/176177724 >> 172.21.15.133:0/176177724 conn(0x7fcea40d0350 msgr2=0x7fcea40b89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 -- 172.21.15.133:0/176177724 shutdown_connections 2024-04-25T10:06:00.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.282+0000 7fcea9019640 1 -- 172.21.15.133:0/176177724 wait complete. 2024-04-25T10:06:00.300 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T10:06:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:00.422 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": -2 2024-04-25T10:06:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:00.435 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T10:06:00.435 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T10:06:00.435 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:00.436 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T10:06:00.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f37f7d640 1 Processor -- start 2024-04-25T10:06:00.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f37f7d640 1 -- start start 2024-04-25T10:06:00.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f37f7d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1f30143ea0 0x7f1f300cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f37f7d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f300b2150 0x7f1f3006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f37f7d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1f3015b9b0 0x7f1f3006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f37f7d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f3015ac50 con 0x7f1f30143ea0 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f37f7d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f300cd180 con 0x7f1f3015b9b0 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f37f7d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3015aed0 con 0x7f1f300b2150 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f35d19640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1f30143ea0 0x7f1f300cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f35d19640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1f30143ea0 0x7f1f300cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50672/0 (socket says 172.21.15.133:50672) 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f35d19640 1 -- 172.21.15.133:0/2367448137 learned_addr learned my addr 172.21.15.133:0/2367448137 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f35518640 1 --2- 172.21.15.133:0/2367448137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f300b2150 0x7f1f3006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f3651a640 1 --2- 172.21.15.133:0/2367448137 >> v2:172.21.15.162:3300/0 conn(0x7f1f3015b9b0 0x7f1f3006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f35d19640 1 -- 172.21.15.133:0/2367448137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f300b2150 msgr2=0x7f1f3006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f35d19640 1 --2- 172.21.15.133:0/2367448137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f300b2150 0x7f1f3006e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f35d19640 1 -- 172.21.15.133:0/2367448137 >> v2:172.21.15.162:3300/0 conn(0x7f1f3015b9b0 msgr2=0x7f1f3006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f35d19640 1 --2- 172.21.15.133:0/2367448137 >> v2:172.21.15.162:3300/0 conn(0x7f1f3015b9b0 0x7f1f3006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f35d19640 1 -- 172.21.15.133:0/2367448137 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3015b1b0 con 0x7f1f30143ea0 2024-04-25T10:06:00.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f35518640 1 --2- 172.21.15.133:0/2367448137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f300b2150 0x7f1f3006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:00.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.542+0000 7f1f3651a640 1 --2- 172.21.15.133:0/2367448137 >> v2:172.21.15.162:3300/0 conn(0x7f1f3015b9b0 0x7f1f3006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:00.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f35d19640 1 --2- 172.21.15.133:0/2367448137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f30143ea0 0x7f1f300cc810 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1f30151d40 tx=0x7f1f2c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e35013c3b12607 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f34d17640 1 -- 172.21.15.133:0/2367448137 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0ab5f0 con 0x7f1f30143ea0 2024-04-25T10:06:00.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f34d17640 1 -- 172.21.15.133:0/2367448137 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c0b0070 con 0x7f1f30143ea0 2024-04-25T10:06:00.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f34d17640 1 -- 172.21.15.133:0/2367448137 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0ab9b0 con 0x7f1f30143ea0 2024-04-25T10:06:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2367448137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f30143ea0 msgr2=0x7f1f300cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f37f7d640 1 --2- 172.21.15.133:0/2367448137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f30143ea0 0x7f1f300cc810 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1f30151d40 tx=0x7f1f2c01c520 comp rx=0 tx=0).stop 2024-04-25T10:06:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2367448137 shutdown_connections 2024-04-25T10:06:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f37f7d640 1 --2- 172.21.15.133:0/2367448137 >> v2:172.21.15.162:3300/0 conn(0x7f1f3015b9b0 0x7f1f3006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f37f7d640 1 --2- 172.21.15.133:0/2367448137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f300b2150 0x7f1f3006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f37f7d640 1 --2- 172.21.15.133:0/2367448137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f30143ea0 0x7f1f300cc810 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2367448137 >> 172.21.15.133:0/2367448137 conn(0x7f1f30066f70 msgr2=0x7f1f3006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2367448137 shutdown_connections 2024-04-25T10:06:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2367448137 wait complete. 2024-04-25T10:06:00.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f37f7d640 1 Processor -- start 2024-04-25T10:06:00.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.546+0000 7f1f37f7d640 1 -- start start 2024-04-25T10:06:00.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f37f7d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f300b2150 0x7f1f3013fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f37f7d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f30143ea0 0x7f1f30135ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f37f7d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f3015b9b0 0x7f1f3013ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f37f7d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3015ede0 con 0x7f1f30143ea0 2024-04-25T10:06:00.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f37f7d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3015ec60 con 0x7f1f300b2150 2024-04-25T10:06:00.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f37f7d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3015ef60 con 0x7f1f3015b9b0 2024-04-25T10:06:00.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35518640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f30143ea0 0x7f1f30135ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35518640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f30143ea0 0x7f1f30135ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50682/0 (socket says 172.21.15.133:50682) 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35518640 1 -- 172.21.15.133:0/2874201466 learned_addr learned my addr 172.21.15.133:0/2874201466 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35d19640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f300b2150 0x7f1f3013fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f3651a640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f3015b9b0 0x7f1f3013ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35518640 1 -- 172.21.15.133:0/2874201466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f3015b9b0 msgr2=0x7f1f3013ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35518640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f3015b9b0 0x7f1f3013ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35518640 1 -- 172.21.15.133:0/2874201466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f300b2150 msgr2=0x7f1f3013fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35518640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f300b2150 0x7f1f3013fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35518640 1 -- 172.21.15.133:0/2874201466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3013e960 con 0x7f1f30143ea0 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f3651a640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f3015b9b0 0x7f1f3013ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35518640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f30143ea0 0x7f1f30135ff0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400c7c0 tx=0x7f1f2400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f227fc640 1 -- 172.21.15.133:0/2874201466 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24007a30 con 0x7f1f30143ea0 2024-04-25T10:06:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f227fc640 1 -- 172.21.15.133:0/2874201466 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f24007bd0 con 0x7f1f30143ea0 2024-04-25T10:06:00.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2874201466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3013b270 con 0x7f1f30143ea0 2024-04-25T10:06:00.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f227fc640 1 -- 172.21.15.133:0/2874201466 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240113f0 con 0x7f1f30143ea0 2024-04-25T10:06:00.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2874201466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3013eee0 con 0x7f1f30143ea0 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f227fc640 1 -- 172.21.15.133:0/2874201466 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f24011590 con 0x7f1f30143ea0 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f227fc640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f14066f60 0x7f1f14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35d19640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f14066f60 0x7f1f14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f227fc640 1 -- 172.21.15.133:0/2874201466 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f240853d0 con 0x7f1f30143ea0 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f35d19640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f14066f60 0x7f1f14069420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0af000 tx=0x7f1f2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f3b0ea000 1 Processor -- start 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.550+0000 7f1f3b0ea000 1 -- start start 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d772970 0x564c7d772d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d773290 0x564c7d728790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7290b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7d53e8a0 con 0x564c7d728cd0 2024-04-25T10:06:00.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7d576d10 con 0x564c7d772970 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c7d4ff2d0 con 0x564c7d773290 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7290b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7290b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50684/0 (socket says 172.21.15.133:50684) 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 -- 172.21.15.133:0/1830454678 learned_addr learned my addr 172.21.15.133:0/1830454678 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f35d19640 1 --2- 172.21.15.133:0/1830454678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d772970 0x564c7d772d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f35518640 1 --2- 172.21.15.133:0/1830454678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d773290 0x564c7d728790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 -- 172.21.15.133:0/1830454678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d773290 msgr2=0x564c7d728790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 --2- 172.21.15.133:0/1830454678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d773290 0x564c7d728790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 -- 172.21.15.133:0/1830454678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d772970 msgr2=0x564c7d772d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 --2- 172.21.15.133:0/1830454678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d772970 0x564c7d772d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 -- 172.21.15.133:0/1830454678 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c7d5bd870 con 0x564c7d728cd0 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f35d19640 1 --2- 172.21.15.133:0/1830454678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d772970 0x564c7d772d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 --2- 172.21.15.133:0/1830454678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7290b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1f28007eb0 tx=0x7f1f2800d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8a974081766a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f137fe640 1 -- 172.21.15.133:0/1830454678 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800f860 con 0x564c7d728cd0 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f137fe640 1 -- 172.21.15.133:0/1830454678 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2800fa00 con 0x564c7d728cd0 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f137fe640 1 -- 172.21.15.133:0/1830454678 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800fdd0 con 0x564c7d728cd0 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/1830454678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 msgr2=0x564c7d7290b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/1830454678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7290b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1f28007eb0 tx=0x7f1f2800d740 comp rx=0 tx=0).stop 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/1830454678 shutdown_connections 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/1830454678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7290b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/1830454678 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d773290 0x564c7d728790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/1830454678 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d772970 0x564c7d772d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/1830454678 >> 172.21.15.133:0/1830454678 conn(0x564c7d72f540 msgr2=0x564c7d72f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/1830454678 shutdown_connections 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/1830454678 wait complete. 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- start start 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 Processor -- start 2024-04-25T10:06:00.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d728cd0 0x564c7d779cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d772970 0x564c7d77a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d773290 0x564c7d72bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7d531b50 con 0x564c7d773290 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7d572db0 con 0x564c7d772970 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c7d538700 con 0x564c7d728cd0 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 10 client.0 _ll_register_callbacks cb 0x564c7d62afb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d773290 0x564c7d72bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d773290 0x564c7d72bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50698/0 (socket says 172.21.15.133:50698) 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 -- 172.21.15.133:0/3053587005 learned_addr learned my addr 172.21.15.133:0/3053587005 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f35518640 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d772970 0x564c7d77a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f35d19640 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d728cd0 0x564c7d779cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 -- 172.21.15.133:0/3053587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d728cd0 msgr2=0x564c7d779cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d728cd0 0x564c7d779cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 -- 172.21.15.133:0/3053587005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d772970 msgr2=0x564c7d77a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d772970 0x564c7d77a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 -- 172.21.15.133:0/3053587005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c7d4ff520 con 0x564c7d773290 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3651a640 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d773290 0x564c7d72bfe0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1f28000f90 tx=0x7f1f28004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8825982917bd4e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f10ff9640 1 -- 172.21.15.133:0/3053587005 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28027060 con 0x564c7d773290 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f10ff9640 1 -- 172.21.15.133:0/3053587005 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f28018890 con 0x564c7d773290 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.554+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3053587005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f2800fa00 con 0x564c7d773290 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f10ff9640 1 -- 172.21.15.133:0/3053587005 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f2802d050 con 0x564c7d773290 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 10 client.6228 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3053587005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x564c7d571d00 con 0x564c7d773290 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f10ff9640 1 -- 172.21.15.133:0/3053587005 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2800fa00 con 0x564c7d773290 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f10ff9640 1 -- 172.21.15.133:0/3053587005 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1f28030020 con 0x564c7d773290 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 10 client.6228 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 10 client.6228 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:00.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 20 client.6228 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x564c7d72cd50 0x564c7d72d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 4 client.6228 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3053587005 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c7d512b10 con 0x564c7d72cd50 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f35d19640 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x564c7d72cd50 0x564c7d72d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f35d19640 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x564c7d72cd50 0x564c7d72d130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f064f7c6a7fdfb99 server_cookie=57dd74cfb8cea8b in_seq=0 out_seq=0 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f10ff9640 10 client.6228 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f10ff9640 1 -- 172.21.15.133:0/3053587005 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c7d512b10 con 0x564c7d72cd50 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f10ff9640 10 client.6228 handle_command_reply: tid=0 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 client.6228 shutdown 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3053587005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d773290 msgr2=0x564c7d72bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d773290 0x564c7d72bfe0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1f28000f90 tx=0x7f1f28004ac0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3053587005 shutdown_connections 2024-04-25T10:06:00.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x564c7d72cd50 0x564c7d72d130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d773290 0x564c7d72bfe0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d772970 0x564c7d77a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f10ff9640 0 client.6228 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:00.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/3053587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d728cd0 0x564c7d779cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3053587005 >> 172.21.15.133:0/3053587005 conn(0x564c7d72f540 msgr2=0x564c7d725770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3053587005 shutdown_connections 2024-04-25T10:06:00.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3053587005 wait complete. 2024-04-25T10:06:00.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.558+0000 7f1f3b0ea000 20 client.6228 trim_cache size 0 max 16384 2024-04-25T10:06:00.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.570+0000 7f1f3b0ea000 1 Processor -- start 2024-04-25T10:06:00.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.570+0000 7f1f3b0ea000 1 -- start start 2024-04-25T10:06:00.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d728cd0 0x564c7d7ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d772970 0x564c7d7ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d773290 0x564c7d7cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30203510 con 0x564c7d772970 2024-04-25T10:06:00.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30202690 con 0x564c7d728cd0 2024-04-25T10:06:00.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30201ac0 con 0x564c7d773290 2024-04-25T10:06:00.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35518640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d772970 0x564c7d7ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35518640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d772970 0x564c7d7ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50704/0 (socket says 172.21.15.133:50704) 2024-04-25T10:06:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35518640 1 -- 172.21.15.133:0/3214301078 learned_addr learned my addr 172.21.15.133:0/3214301078 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35518640 1 -- 172.21.15.133:0/3214301078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d773290 msgr2=0x564c7d7cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3651a640 1 --2- 172.21.15.133:0/3214301078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d773290 0x564c7d7cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35d19640 1 --2- 172.21.15.133:0/3214301078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d728cd0 0x564c7d7ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35518640 1 --2- 172.21.15.133:0/3214301078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d773290 0x564c7d7cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35518640 1 -- 172.21.15.133:0/3214301078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d728cd0 msgr2=0x564c7d7ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35518640 1 --2- 172.21.15.133:0/3214301078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d728cd0 0x564c7d7ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35518640 1 -- 172.21.15.133:0/3214301078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c7d7730c0 con 0x564c7d772970 2024-04-25T10:06:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3651a640 1 --2- 172.21.15.133:0/3214301078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d773290 0x564c7d7cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35518640 1 --2- 172.21.15.133:0/3214301078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d772970 0x564c7d7ceec0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400a000 tx=0x7f1f2401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a904555a13080c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f12ffd640 1 -- 172.21.15.133:0/3214301078 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2401e040 con 0x564c7d772970 2024-04-25T10:06:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f12ffd640 1 -- 172.21.15.133:0/3214301078 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f24053e40 con 0x564c7d772970 2024-04-25T10:06:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f12ffd640 1 -- 172.21.15.133:0/3214301078 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240521b0 con 0x564c7d772970 2024-04-25T10:06:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3214301078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d772970 msgr2=0x564c7d7ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/3214301078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d772970 0x564c7d7ceec0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400a000 tx=0x7f1f2401c000 comp rx=0 tx=0).stop 2024-04-25T10:06:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3214301078 shutdown_connections 2024-04-25T10:06:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/3214301078 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d773290 0x564c7d7cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/3214301078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d772970 0x564c7d7ceec0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/3214301078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d728cd0 0x564c7d7ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3214301078 >> 172.21.15.133:0/3214301078 conn(0x564c7d72f540 msgr2=0x564c7d726ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3214301078 shutdown_connections 2024-04-25T10:06:00.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/3214301078 wait complete. 2024-04-25T10:06:00.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- start start 2024-04-25T10:06:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 Processor -- start 2024-04-25T10:06:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7d3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d772970 0x564c7d7d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d773290 0x564c7d7779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7d572db0 con 0x564c7d728cd0 2024-04-25T10:06:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7d53b1c0 con 0x564c7d773290 2024-04-25T10:06:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c7d540870 con 0x564c7d772970 2024-04-25T10:06:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 10 client.0 _ll_register_callbacks cb 0x7f1f28007650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35d19640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35d19640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7d3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50708/0 (socket says 172.21.15.133:50708) 2024-04-25T10:06:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35d19640 1 -- 172.21.15.133:0/730043070 learned_addr learned my addr 172.21.15.133:0/730043070 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35d19640 1 -- 172.21.15.133:0/730043070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d772970 msgr2=0x564c7d7d35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35d19640 1 --2- 172.21.15.133:0/730043070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d772970 0x564c7d7d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35d19640 1 -- 172.21.15.133:0/730043070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d773290 msgr2=0x564c7d7779a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35d19640 1 --2- 172.21.15.133:0/730043070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d773290 0x564c7d7779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35d19640 1 -- 172.21.15.133:0/730043070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c7d5fa920 con 0x564c7d728cd0 2024-04-25T10:06:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f35d19640 1 --2- 172.21.15.133:0/730043070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7d3070 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c008010 tx=0x7f1f2c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2592064c956fa173 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f10ff9640 1 -- 172.21.15.133:0/730043070 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0c9030 con 0x564c7d728cd0 2024-04-25T10:06:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f10ff9640 1 -- 172.21.15.133:0/730043070 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f2c0c5110 con 0x564c7d728cd0 2024-04-25T10:06:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/730043070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f24053e40 con 0x564c7d728cd0 2024-04-25T10:06:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f10ff9640 1 -- 172.21.15.133:0/730043070 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f2c0d5050 con 0x564c7d728cd0 2024-04-25T10:06:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 10 client.6234 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/730043070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x564c7d5d0a30 con 0x564c7d728cd0 2024-04-25T10:06:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.574+0000 7f1f10ff9640 1 -- 172.21.15.133:0/730043070 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f24053e40 con 0x564c7d728cd0 2024-04-25T10:06:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f10ff9640 1 -- 172.21.15.133:0/730043070 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1f2c0d8020 con 0x564c7d728cd0 2024-04-25T10:06:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 10 client.6234 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 10 client.6234 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 20 client.6234 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/730043070 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x564c7d778360 0x564c7d778770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 4 client.6234 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T10:06:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/730043070 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x564c7d584250 con 0x564c7d778360 2024-04-25T10:06:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f35518640 1 --2- 172.21.15.133:0/730043070 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x564c7d778360 0x564c7d778770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f35518640 1 --2- 172.21.15.133:0/730043070 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x564c7d778360 0x564c7d778770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254f05c338cb1cef server_cookie=30e0557e6c992780 in_seq=0 out_seq=0 2024-04-25T10:06:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f10ff9640 10 client.6234 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:00.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f10ff9640 1 -- 172.21.15.133:0/730043070 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x564c7d584250 con 0x564c7d778360 2024-04-25T10:06:00.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f10ff9640 10 client.6234 handle_command_reply: tid=0 2024-04-25T10:06:00.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 client.6234 shutdown 2024-04-25T10:06:00.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/730043070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 msgr2=0x564c7d7d3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/730043070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7d3070 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c008010 tx=0x7f1f2c01ad60 comp rx=0 tx=0).stop 2024-04-25T10:06:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/730043070 shutdown_connections 2024-04-25T10:06:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/730043070 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x564c7d778360 0x564c7d778770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/730043070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564c7d773290 0x564c7d7779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/730043070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564c7d772970 0x564c7d7d35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f10ff9640 0 client.6234 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 --2- 172.21.15.133:0/730043070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564c7d728cd0 0x564c7d7d3070 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/730043070 >> 172.21.15.133:0/730043070 conn(0x564c7d72f540 msgr2=0x564c7d727780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/730043070 shutdown_connections 2024-04-25T10:06:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 1 -- 172.21.15.133:0/730043070 wait complete. 2024-04-25T10:06:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f3b0ea000 20 client.6234 trim_cache size 0 max 16384 2024-04-25T10:06:00.603 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:00.603 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": -2 2024-04-25T10:06:00.603 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:00.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2874201466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f14066f60 msgr2=0x7f1f14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f14066f60 0x7f1f14069420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0af000 tx=0x7f1f2c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:00.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2874201466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f30143ea0 msgr2=0x7f1f30135ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f30143ea0 0x7f1f30135ff0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400c7c0 tx=0x7f1f2400cc80 comp rx=0 tx=0).stop 2024-04-25T10:06:00.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2874201466 shutdown_connections 2024-04-25T10:06:00.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f3015b9b0 0x7f1f3013ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f14066f60 0x7f1f14069420 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f30143ea0 0x7f1f30135ff0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 --2- 172.21.15.133:0/2874201466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f300b2150 0x7f1f3013fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2874201466 >> 172.21.15.133:0/2874201466 conn(0x7f1f30066f70 msgr2=0x7f1f300b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2874201466 shutdown_connections 2024-04-25T10:06:00.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.578+0000 7f1f37f7d640 1 -- 172.21.15.133:0/2874201466 wait complete. 2024-04-25T10:06:00.608 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T10:06:00.608 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:06:00.608 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:06:00.609 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:00.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3ac4b1640 1 Processor -- start 2024-04-25T10:06:00.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3ac4b1640 1 -- start start 2024-04-25T10:06:00.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3ac4b1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3a406eb70 0x7ff3a40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3ac4b1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff3a41584f0 0x7ff3a415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3ac4b1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a40cbe60 con 0x7ff3a4158110 2024-04-25T10:06:00.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3ac4b1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a40b19b0 con 0x7ff3a41584f0 2024-04-25T10:06:00.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3ac4b1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a415b360 con 0x7ff3a406eb70 2024-04-25T10:06:00.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3aaa4e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff3a41584f0 0x7ff3a415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3aaa4e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff3a41584f0 0x7ff3a415ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50714/0 (socket says 172.21.15.133:50714) 2024-04-25T10:06:00.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3aaa4e640 1 -- 172.21.15.133:0/93802737 learned_addr learned my addr 172.21.15.133:0/93802737 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:00.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3a9a4c640 1 --2- 172.21.15.133:0/93802737 >> v2:172.21.15.162:3300/0 conn(0x7ff3a406eb70 0x7ff3a40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3aaa4e640 1 -- 172.21.15.133:0/93802737 >> v2:172.21.15.162:3300/0 conn(0x7ff3a406eb70 msgr2=0x7ff3a40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3aaa4e640 1 --2- 172.21.15.133:0/93802737 >> v2:172.21.15.162:3300/0 conn(0x7ff3a406eb70 0x7ff3a40b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3aaa4e640 1 -- 172.21.15.133:0/93802737 >> v1:172.21.15.162:6789/0 conn(0x7ff3a4158110 legacy=0x7ff3a406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:00.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.722+0000 7ff3aaa4e640 1 -- 172.21.15.133:0/93802737 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a415b640 con 0x7ff3a41584f0 2024-04-25T10:06:00.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3aaa4e640 1 --2- 172.21.15.133:0/93802737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3a41584f0 0x7ff3a415ad10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40b4da0 tx=0x7ff39c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263927064b7e043d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a924b640 1 -- 172.21.15.133:0/93802737 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0a57a0 con 0x7ff3a41584f0 2024-04-25T10:06:00.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a924b640 1 -- 172.21.15.133:0/93802737 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c0a5940 con 0x7ff3a41584f0 2024-04-25T10:06:00.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a924b640 1 -- 172.21.15.133:0/93802737 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0a5d10 con 0x7ff3a41584f0 2024-04-25T10:06:00.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/93802737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3a41584f0 msgr2=0x7ff3a415ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 --2- 172.21.15.133:0/93802737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3a41584f0 0x7ff3a415ad10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40b4da0 tx=0x7ff39c01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/93802737 shutdown_connections 2024-04-25T10:06:00.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 --2- 172.21.15.133:0/93802737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3a41584f0 0x7ff3a415ad10 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 --2- 172.21.15.133:0/93802737 >> v2:172.21.15.162:3300/0 conn(0x7ff3a406eb70 0x7ff3a40b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/93802737 >> 172.21.15.133:0/93802737 conn(0x7ff3a4067000 msgr2=0x7ff3a406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/93802737 shutdown_connections 2024-04-25T10:06:00.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/93802737 wait complete. 2024-04-25T10:06:00.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 Processor -- start 2024-04-25T10:06:00.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 -- start start 2024-04-25T10:06:00.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a406eb70 0x7ff3a40c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3a4158110 0x7ff3a40c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3a41584f0 0x7ff3a4151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a406dce0 con 0x7ff3a4158110 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a406db60 con 0x7ff3a41584f0 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a406de60 con 0x7ff3a406eb70 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a9a4c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3a4158110 0x7ff3a40c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a9a4c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3a4158110 0x7ff3a40c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50728/0 (socket says 172.21.15.133:50728) 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a9a4c640 1 -- 172.21.15.133:0/3774433896 learned_addr learned my addr 172.21.15.133:0/3774433896 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3aa24d640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a406eb70 0x7ff3a40c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3aaa4e640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3a41584f0 0x7ff3a4151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a9a4c640 1 -- 172.21.15.133:0/3774433896 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a406eb70 msgr2=0x7ff3a40c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a9a4c640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a406eb70 0x7ff3a40c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a9a4c640 1 -- 172.21.15.133:0/3774433896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3a41584f0 msgr2=0x7ff3a4151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a9a4c640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3a41584f0 0x7ff3a4151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a9a4c640 1 -- 172.21.15.133:0/3774433896 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40c9a70 con 0x7ff3a4158110 2024-04-25T10:06:00.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3aa24d640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a406eb70 0x7ff3a40c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:00.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3a9a4c640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3a4158110 0x7ff3a40c7940 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff39400d800 tx=0x7ff39400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff39affd640 1 -- 172.21.15.133:0/3774433896 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39400b6c0 con 0x7ff3a4158110 2024-04-25T10:06:00.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff39affd640 1 -- 172.21.15.133:0/3774433896 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff394019070 con 0x7ff3a4158110 2024-04-25T10:06:00.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/3774433896 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a40c9d50 con 0x7ff3a4158110 2024-04-25T10:06:00.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff39affd640 1 -- 172.21.15.133:0/3774433896 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39400baa0 con 0x7ff3a4158110 2024-04-25T10:06:00.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.726+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/3774433896 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a40ca5f0 con 0x7ff3a4158110 2024-04-25T10:06:00.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.730+0000 7ff39affd640 1 -- 172.21.15.133:0/3774433896 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff39400bc40 con 0x7ff3a4158110 2024-04-25T10:06:00.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.730+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/3774433896 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a40c9f20 con 0x7ff3a4158110 2024-04-25T10:06:00.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.730+0000 7ff39affd640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff388066f60 0x7ff388069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:00.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.730+0000 7ff39affd640 1 -- 172.21.15.133:0/3774433896 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff394085650 con 0x7ff3a4158110 2024-04-25T10:06:00.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.730+0000 7ff3aa24d640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff388066f60 0x7ff388069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:00.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.730+0000 7ff3aa24d640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff388066f60 0x7ff388069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00016f0 tx=0x7ff3a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:00.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.734+0000 7ff39affd640 1 -- 172.21.15.133:0/3774433896 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3940564d0 con 0x7ff3a4158110 2024-04-25T10:06:00.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.926+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/3774433896 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3a406bc20 con 0x7ff3a4158110 2024-04-25T10:06:00.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.926+0000 7ff39affd640 1 -- 172.21.15.133:0/3774433896 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7ff394055dd0 con 0x7ff3a4158110 2024-04-25T10:06:00.932 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:00.933 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:05:38.566660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:00.933 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2024-04-25T10:06:00.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/3774433896 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff388066f60 msgr2=0x7ff388069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff388066f60 0x7ff388069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00016f0 tx=0x7ff3a0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:00.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/3774433896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3a4158110 msgr2=0x7ff3a40c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:00.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3a4158110 0x7ff3a40c7940 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff39400d800 tx=0x7ff39400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/3774433896 shutdown_connections 2024-04-25T10:06:00.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3a41584f0 0x7ff3a4151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff388066f60 0x7ff388069420 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3a4158110 0x7ff3a40c7940 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 --2- 172.21.15.133:0/3774433896 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3a406eb70 0x7ff3a40c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:00.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/3774433896 >> 172.21.15.133:0/3774433896 conn(0x7ff3a4067000 msgr2=0x7ff3a406e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:00.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/3774433896 shutdown_connections 2024-04-25T10:06:00.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:00.930+0000 7ff3ac4b1640 1 -- 172.21.15.133:0/3774433896 wait complete. 2024-04-25T10:06:00.951 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:06:01.157 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:01.158 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:06:01.158 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:01.171 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:01.172 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:06:01.172 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:01.172 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:06:01.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f00414ae640 1 Processor -- start 2024-04-25T10:06:01.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f00414ae640 1 -- start start 2024-04-25T10:06:01.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f00414ae640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f003c06cc80 0x7f003c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f00414ae640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c0b1820 con 0x7f003c0ce080 2024-04-25T10:06:01.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f00414ae640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c0cdc40 con 0x7f003c1604a0 2024-04-25T10:06:01.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f00414ae640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0b1ac0 con 0x7f003c06cc80 2024-04-25T10:06:01.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003affd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f003c0ce080 0x7f003c0ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60218/0 (socket says 172.21.15.133:60218) 2024-04-25T10:06:01.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003affd640 1 -- 172.21.15.133:0/3223762761 learned_addr learned my addr 172.21.15.133:0/3223762761 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:01.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003a7fc640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f003c06cc80 0x7f003c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305188983 0 0) 0x7f003c0b1820 con 0x7f003c0ce080 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0024003530 con 0x7f003c0ce080 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1633861583 0 0) 0x7f0024003530 con 0x7f003c0ce080 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f003c06cc80 msgr2=0x7f003c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f003c06cc80 0x7f003c0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 >> v1:172.21.15.162:6789/0 conn(0x7f003c1604a0 legacy=0x7f003c160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c0b1da0 con 0x7f003c0ce080 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f002c001800 con 0x7f003c0ce080 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 >> v1:172.21.15.133:6789/0 conn(0x7f003c0ce080 legacy=0x7f003c0ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f003c06cc80 0x7f00240039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0024003f70 0x7f0024004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0024004d50 0x7f00240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0b1ac0 con 0x7f0024003f70 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002400bab0 con 0x7f0024004d50 2024-04-25T10:06:01.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002400bcb0 con 0x7f003c06cc80 2024-04-25T10:06:01.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003a7fc640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0024003f70 0x7f0024004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003affd640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f003c06cc80 0x7f00240039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003b7fe640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0024004d50 0x7f00240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003a7fc640 1 -- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f003c06cc80 msgr2=0x7f00240039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003a7fc640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f003c06cc80 0x7f00240039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003a7fc640 1 -- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0024004d50 msgr2=0x7f00240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003a7fc640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0024004d50 0x7f00240051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003a7fc640 1 -- 172.21.15.133:0/3223762761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0024003530 con 0x7f0024003f70 2024-04-25T10:06:01.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.294+0000 7f003b7fe640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0024004d50 0x7f00240051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:01.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003a7fc640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0024003f70 0x7f0024004810 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0030009ae0 tx=0x7f003001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89575fe8e9584cc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00300b2070 con 0x7f0024003f70 2024-04-25T10:06:01.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00300a5660 con 0x7f0024003f70 2024-04-25T10:06:01.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f0039ffb640 1 -- 172.21.15.133:0/3223762761 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00300096b0 con 0x7f0024003f70 2024-04-25T10:06:01.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 -- 172.21.15.133:0/3223762761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0024003f70 msgr2=0x7f0024004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 --2- 172.21.15.133:0/3223762761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0024003f70 0x7f0024004810 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0030009ae0 tx=0x7f003001ca70 comp rx=0 tx=0).stop 2024-04-25T10:06:01.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003b7fe640 1 -- 172.21.15.133:0/3223762761 reap_dead start 2024-04-25T10:06:01.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 -- 172.21.15.133:0/3223762761 shutdown_connections 2024-04-25T10:06:01.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 -- 172.21.15.133:0/3223762761 >> 172.21.15.133:0/3223762761 conn(0x7f003c09ab90 msgr2=0x7f003c0d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 -- 172.21.15.133:0/3223762761 shutdown_connections 2024-04-25T10:06:01.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 -- 172.21.15.133:0/3223762761 wait complete. 2024-04-25T10:06:01.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 Processor -- start 2024-04-25T10:06:01.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 -- start start 2024-04-25T10:06:01.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0024003f70 0x7f003c154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f003c0ca120 0x7f003c15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f003c0c8da0 0x7f003c0af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0b2640 con 0x7f003c0ca120 2024-04-25T10:06:01.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0b2360 con 0x7f003c0c8da0 2024-04-25T10:06:01.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f00414ae640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c161040 con 0x7f0024003f70 2024-04-25T10:06:01.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f003c0ca120 0x7f003c15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f003c0ca120 0x7f003c15e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50738/0 (socket says 172.21.15.133:50738) 2024-04-25T10:06:01.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003a7fc640 1 -- 172.21.15.133:0/846315476 learned_addr learned my addr 172.21.15.133:0/846315476 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003affd640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0024003f70 0x7f003c154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003b7fe640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f003c0c8da0 0x7f003c0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003a7fc640 1 -- 172.21.15.133:0/846315476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0024003f70 msgr2=0x7f003c154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003a7fc640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0024003f70 0x7f003c154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003a7fc640 1 -- 172.21.15.133:0/846315476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f003c0c8da0 msgr2=0x7f003c0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003a7fc640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f003c0c8da0 0x7f003c0af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003a7fc640 1 -- 172.21.15.133:0/846315476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c0c9220 con 0x7f003c0ca120 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003b7fe640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f003c0c8da0 0x7f003c0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.298+0000 7f003affd640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0024003f70 0x7f003c154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f003a7fc640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f003c0ca120 0x7f003c15e4f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0030009ae0 tx=0x7f0024005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f00237fe640 1 -- 172.21.15.133:0/846315476 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00300bc030 con 0x7f003c0ca120 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f00237fe640 1 -- 172.21.15.133:0/846315476 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0030004020 con 0x7f003c0ca120 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f00414ae640 1 -- 172.21.15.133:0/846315476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c0c94d0 con 0x7f003c0ca120 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f00237fe640 1 -- 172.21.15.133:0/846315476 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00300b2040 con 0x7f003c0ca120 2024-04-25T10:06:01.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f00414ae640 1 -- 172.21.15.133:0/846315476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003c0c9a90 con 0x7f003c0ca120 2024-04-25T10:06:01.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f00237fe640 1 -- 172.21.15.133:0/846315476 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00300bf070 con 0x7f003c0ca120 2024-04-25T10:06:01.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f00237fe640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0028066f90 0x7f0028069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f003affd640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0028066f90 0x7f0028069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f00237fe640 1 -- 172.21.15.133:0/846315476 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f003011ee80 con 0x7f003c0ca120 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f003affd640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0028066f90 0x7f0028069450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f002c000f00 tx=0x7f002c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f004461b000 1 Processor -- start 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.302+0000 7f004461b000 1 -- start start 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9afea0 0x557fec9b0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9b07c0 0x557fec9661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557fec966af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec7b4db0 con 0x557fec966710 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec73d230 con 0x557fec9b07c0 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fec7b0e50 con 0x557fec9afea0 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557fec966af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557fec966af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50748/0 (socket says 172.21.15.133:50748) 2024-04-25T10:06:01.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003b7fe640 1 -- 172.21.15.133:0/751037819 learned_addr learned my addr 172.21.15.133:0/751037819 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003a7fc640 1 --2- 172.21.15.133:0/751037819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9b07c0 0x557fec9661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003b7fe640 1 -- 172.21.15.133:0/751037819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9afea0 msgr2=0x557fec9b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003b7fe640 1 --2- 172.21.15.133:0/751037819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9afea0 0x557fec9b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003b7fe640 1 -- 172.21.15.133:0/751037819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9b07c0 msgr2=0x557fec9661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003b7fe640 1 --2- 172.21.15.133:0/751037819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9b07c0 0x557fec9661d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003b7fe640 1 -- 172.21.15.133:0/751037819 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fec776750 con 0x557fec966710 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003b7fe640 1 --2- 172.21.15.133:0/751037819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557fec966af0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0034000f90 tx=0x7f0034002070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98beb847bfa5170 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f0020ff9640 1 -- 172.21.15.133:0/751037819 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034003d90 con 0x557fec966710 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f0020ff9640 1 -- 172.21.15.133:0/751037819 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003400e830 con 0x557fec966710 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f0020ff9640 1 -- 172.21.15.133:0/751037819 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003400ec40 con 0x557fec966710 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- 172.21.15.133:0/751037819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 msgr2=0x557fec966af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 --2- 172.21.15.133:0/751037819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557fec966af0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0034000f90 tx=0x7f0034002070 comp rx=0 tx=0).stop 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- 172.21.15.133:0/751037819 shutdown_connections 2024-04-25T10:06:01.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 --2- 172.21.15.133:0/751037819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557fec966af0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 --2- 172.21.15.133:0/751037819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9b07c0 0x557fec9661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 --2- 172.21.15.133:0/751037819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9afea0 0x557fec9b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- 172.21.15.133:0/751037819 >> 172.21.15.133:0/751037819 conn(0x557fec9626c0 msgr2=0x557fec964ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- 172.21.15.133:0/751037819 shutdown_connections 2024-04-25T10:06:01.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- 172.21.15.133:0/751037819 wait complete. 2024-04-25T10:06:01.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- start start 2024-04-25T10:06:01.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 Processor -- start 2024-04-25T10:06:01.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec966710 0x557fec994ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9afea0 0x557fec995410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9b07c0 0x557fec995950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec76fbf0 con 0x557fec9afea0 2024-04-25T10:06:01.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec7b1e20 con 0x557fec966710 2024-04-25T10:06:01.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fec76ee40 con 0x557fec9b07c0 2024-04-25T10:06:01.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 10 client.0 _ll_register_callbacks cb 0x557fec951790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:01.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9afea0 0x557fec995410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9afea0 0x557fec995410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50764/0 (socket says 172.21.15.133:50764) 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003a7fc640 1 -- 172.21.15.133:0/641657530 learned_addr learned my addr 172.21.15.133:0/641657530 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003affd640 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec966710 0x557fec994ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003b7fe640 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9b07c0 0x557fec995950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003a7fc640 1 -- 172.21.15.133:0/641657530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9b07c0 msgr2=0x557fec995950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003a7fc640 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9b07c0 0x557fec995950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003a7fc640 1 -- 172.21.15.133:0/641657530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec966710 msgr2=0x557fec994ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003a7fc640 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec966710 0x557fec994ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003a7fc640 1 -- 172.21.15.133:0/641657530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fec757640 con 0x557fec9afea0 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f003a7fc640 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9afea0 0x557fec995410 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f00300ef630 tx=0x7f00300ef0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645411e10574fd32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f0019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f0019ffb640 1 -- 172.21.15.133:0/641657530 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00300f7d00 con 0x557fec9afea0 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f0019ffb640 1 -- 172.21.15.133:0/641657530 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0030104df0 con 0x557fec9afea0 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.306+0000 7f004461b000 1 -- 172.21.15.133:0/641657530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f003400e830 con 0x557fec9afea0 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f0019ffb640 1 -- 172.21.15.133:0/641657530 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00300f7870 con 0x557fec9afea0 2024-04-25T10:06:01.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 10 client.6252 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 -- 172.21.15.133:0/641657530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x557fec73d480 con 0x557fec9afea0 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f0019ffb640 1 -- 172.21.15.133:0/641657530 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003400e830 con 0x557fec9afea0 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f0019ffb640 1 -- 172.21.15.133:0/641657530 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f00300b5030 con 0x557fec9afea0 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 10 client.6252 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 10 client.6252 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 20 client.6252 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x557fec9bff40 0x557fec9c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 4 client.6252 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 -- 172.21.15.133:0/641657530 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fec750bb0 con 0x557fec9bff40 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f003affd640 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x557fec9bff40 0x557fec9c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f0019ffb640 10 client.6252 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f003affd640 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x557fec9bff40 0x557fec9c2330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cdc2a2441359dc7 server_cookie=7aad05ea800c7b5d in_seq=0 out_seq=0 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f0019ffb640 1 -- 172.21.15.133:0/641657530 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557fec750bb0 con 0x557fec9bff40 2024-04-25T10:06:01.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f0019ffb640 10 client.6252 handle_command_reply: tid=0 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 client.6252 shutdown 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 -- 172.21.15.133:0/641657530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9afea0 msgr2=0x557fec995410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9afea0 0x557fec995410 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f00300ef630 tx=0x7f00300ef0c0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 -- 172.21.15.133:0/641657530 shutdown_connections 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x557fec9bff40 0x557fec9c2330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9b07c0 0x557fec995950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f0019ffb640 0 client.6252 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9afea0 0x557fec995410 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 --2- 172.21.15.133:0/641657530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec966710 0x557fec994ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 -- 172.21.15.133:0/641657530 >> 172.21.15.133:0/641657530 conn(0x557fec9626c0 msgr2=0x557fec965340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 -- 172.21.15.133:0/641657530 shutdown_connections 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 1 -- 172.21.15.133:0/641657530 wait complete. 2024-04-25T10:06:01.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.310+0000 7f004461b000 20 client.6252 trim_cache size 0 max 16384 2024-04-25T10:06:01.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 Processor -- start 2024-04-25T10:06:01.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 -- start start 2024-04-25T10:06:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec966710 0x557fec965ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9afea0 0x557fec966000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9b07c0 0x557fec9c0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c200c10 con 0x557fec9b07c0 2024-04-25T10:06:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c1ffe40 con 0x557fec9afea0 2024-04-25T10:06:01.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c1ff240 con 0x557fec966710 2024-04-25T10:06:01.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f003b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9b07c0 0x557fec9c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f003b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9b07c0 0x557fec9c0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50766/0 (socket says 172.21.15.133:50766) 2024-04-25T10:06:01.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f003b7fe640 1 -- 172.21.15.133:0/4206877322 learned_addr learned my addr 172.21.15.133:0/4206877322 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:01.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f003a7fc640 1 --2- 172.21.15.133:0/4206877322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9afea0 0x557fec966000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f003affd640 1 --2- 172.21.15.133:0/4206877322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec966710 0x557fec965ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f003b7fe640 1 -- 172.21.15.133:0/4206877322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec966710 msgr2=0x557fec965ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f003b7fe640 1 --2- 172.21.15.133:0/4206877322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec966710 0x557fec965ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f003b7fe640 1 -- 172.21.15.133:0/4206877322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9afea0 msgr2=0x557fec966000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f003b7fe640 1 --2- 172.21.15.133:0/4206877322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9afea0 0x557fec966000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f003b7fe640 1 -- 172.21.15.133:0/4206877322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fec836ef0 con 0x557fec9b07c0 2024-04-25T10:06:01.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f003b7fe640 1 --2- 172.21.15.133:0/4206877322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9b07c0 0x557fec9c0320 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f00340020b0 tx=0x7f003400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abe5c54a4479340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f001bfff640 1 -- 172.21.15.133:0/4206877322 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003400e830 con 0x557fec9b07c0 2024-04-25T10:06:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f001bfff640 1 -- 172.21.15.133:0/4206877322 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003400e9f0 con 0x557fec9b07c0 2024-04-25T10:06:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f001bfff640 1 -- 172.21.15.133:0/4206877322 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003400ede0 con 0x557fec9b07c0 2024-04-25T10:06:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 -- 172.21.15.133:0/4206877322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9b07c0 msgr2=0x557fec9c0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 --2- 172.21.15.133:0/4206877322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9b07c0 0x557fec9c0320 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f00340020b0 tx=0x7f003400c000 comp rx=0 tx=0).stop 2024-04-25T10:06:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 -- 172.21.15.133:0/4206877322 shutdown_connections 2024-04-25T10:06:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 --2- 172.21.15.133:0/4206877322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec9b07c0 0x557fec9c0320 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 --2- 172.21.15.133:0/4206877322 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9afea0 0x557fec966000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 --2- 172.21.15.133:0/4206877322 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec966710 0x557fec965ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 -- 172.21.15.133:0/4206877322 >> 172.21.15.133:0/4206877322 conn(0x557fec9626c0 msgr2=0x557fec9ae7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 -- 172.21.15.133:0/4206877322 shutdown_connections 2024-04-25T10:06:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 -- 172.21.15.133:0/4206877322 wait complete. 2024-04-25T10:06:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 -- start start 2024-04-25T10:06:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.326+0000 7f004461b000 1 Processor -- start 2024-04-25T10:06:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557feca116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9afea0 0x557feca11c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9b07c0 0x557fec9c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fec76fbf0 con 0x557fec966710 2024-04-25T10:06:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c1ff840 con 0x557fec9b07c0 2024-04-25T10:06:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fec96a6e0 con 0x557fec9afea0 2024-04-25T10:06:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 10 client.0 _ll_register_callbacks cb 0x7f0034007eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557feca116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557feca116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50778/0 (socket says 172.21.15.133:50778) 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003affd640 1 -- 172.21.15.133:0/2672487565 learned_addr learned my addr 172.21.15.133:0/2672487565 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003a7fc640 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9afea0 0x557feca11c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003b7fe640 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9b07c0 0x557fec9c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003affd640 1 -- 172.21.15.133:0/2672487565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9afea0 msgr2=0x557feca11c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003affd640 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9afea0 0x557feca11c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003affd640 1 -- 172.21.15.133:0/2672487565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9b07c0 msgr2=0x557fec9c2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003affd640 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9b07c0 0x557fec9c2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003affd640 1 -- 172.21.15.133:0/2672487565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fec7749a0 con 0x557fec966710 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003a7fc640 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9afea0 0x557feca11c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003affd640 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557feca116c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f002c013b10 tx=0x7f002c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1947d52f1486e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f0019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f0019ffb640 1 -- 172.21.15.133:0/2672487565 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c01c030 con 0x557fec966710 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f0019ffb640 1 -- 172.21.15.133:0/2672487565 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f002c02db00 con 0x557fec966710 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 -- 172.21.15.133:0/2672487565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f003400e9f0 con 0x557fec966710 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f0019ffb640 1 -- 172.21.15.133:0/2672487565 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f002c036070 con 0x557fec966710 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 10 client.6258 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 -- 172.21.15.133:0/2672487565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x557fec4715a0 con 0x557fec966710 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f0019ffb640 1 -- 172.21.15.133:0/2672487565 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003400e9f0 con 0x557fec966710 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f0019ffb640 1 -- 172.21.15.133:0/2672487565 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f002c022040 con 0x557fec966710 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 10 client.6258 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 10 client.6258 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 20 client.6258 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x557fec9c34f0 0x557fec9c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 4 client.6258 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T10:06:01.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 -- 172.21.15.133:0/2672487565 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x557fec764080 con 0x557fec9c34f0 2024-04-25T10:06:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003a7fc640 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x557fec9c34f0 0x557fec9c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f0019ffb640 10 client.6258 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f003a7fc640 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x557fec9c34f0 0x557fec9c3900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b79d78c4a8591f3 server_cookie=35c20f5cd7f5c253 in_seq=0 out_seq=0 2024-04-25T10:06:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f0019ffb640 1 -- 172.21.15.133:0/2672487565 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557fec764080 con 0x557fec9c34f0 2024-04-25T10:06:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f0019ffb640 10 client.6258 handle_command_reply: tid=0 2024-04-25T10:06:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 client.6258 shutdown 2024-04-25T10:06:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 -- 172.21.15.133:0/2672487565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 msgr2=0x557feca116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557feca116c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f002c013b10 tx=0x7f002c000bd0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 -- 172.21.15.133:0/2672487565 shutdown_connections 2024-04-25T10:06:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x557fec9c34f0 0x557fec9c3900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557fec9b07c0 0x557fec9c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x557fec9afea0 0x557feca11c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 --2- 172.21.15.133:0/2672487565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557fec966710 0x557feca116c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f0019ffb640 0 client.6258 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 -- 172.21.15.133:0/2672487565 >> 172.21.15.133:0/2672487565 conn(0x557fec9626c0 msgr2=0x557fec9af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 -- 172.21.15.133:0/2672487565 shutdown_connections 2024-04-25T10:06:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 1 -- 172.21.15.133:0/2672487565 wait complete. 2024-04-25T10:06:01.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f004461b000 20 client.6258 trim_cache size 0 max 16384 2024-04-25T10:06:01.349 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:01.349 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:06:01.350 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "18369933-0d57-426d-8bc2-2b2964311939", 2024-04-25T10:06:01.350 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:06:01.350 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:01.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 -- 172.21.15.133:0/846315476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0028066f90 msgr2=0x7f0028069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0028066f90 0x7f0028069450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f002c000f00 tx=0x7f002c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:01.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 -- 172.21.15.133:0/846315476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f003c0ca120 msgr2=0x7f003c15e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f003c0ca120 0x7f003c15e4f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0030009ae0 tx=0x7f0024005840 comp rx=0 tx=0).stop 2024-04-25T10:06:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 -- 172.21.15.133:0/846315476 shutdown_connections 2024-04-25T10:06:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f003c0c8da0 0x7f003c0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0028066f90 0x7f0028069450 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f003c0ca120 0x7f003c15e4f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 --2- 172.21.15.133:0/846315476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0024003f70 0x7f003c154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 -- 172.21.15.133:0/846315476 >> 172.21.15.133:0/846315476 conn(0x7f003c09ab90 msgr2=0x7f003c0d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 -- 172.21.15.133:0/846315476 shutdown_connections 2024-04-25T10:06:01.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.330+0000 7f00414ae640 1 -- 172.21.15.133:0/846315476 wait complete. 2024-04-25T10:06:01.354 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '18369933-0d57-426d-8bc2-2b2964311939', 'mode': 'asynchronous'}' 2024-04-25T10:06:01.354 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:06:01.354 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:06:01.355 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.462+0000 7ff7f4d5e640 1 Processor -- start 2024-04-25T10:06:01.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.462+0000 7ff7f4d5e640 1 -- start start 2024-04-25T10:06:01.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7f4d5e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff7f015db00 0x7ff7f00b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7f4d5e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7f00d0770 0x7ff7f00d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7f4d5e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff7f01614c0 0x7ff7f0161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7f4d5e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f00d1110 con 0x7ff7f015db00 2024-04-25T10:06:01.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7f4d5e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f00ceb20 con 0x7ff7f01614c0 2024-04-25T10:06:01.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7f4d5e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00b7740 con 0x7ff7f00d0770 2024-04-25T10:06:01.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7ee59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff7f015db00 0x7ff7f00b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7ee59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff7f015db00 0x7ff7f00b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50782/0 (socket says 172.21.15.133:50782) 2024-04-25T10:06:01.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7ee59c640 1 -- 172.21.15.133:0/864831655 learned_addr learned my addr 172.21.15.133:0/864831655 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:01.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7eed9d640 1 --2- 172.21.15.133:0/864831655 >> v2:172.21.15.162:3300/0 conn(0x7ff7f01614c0 0x7ff7f0161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7e7fff640 1 --2- 172.21.15.133:0/864831655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7f00d0770 0x7ff7f00d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7ee59c640 1 -- 172.21.15.133:0/864831655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7f00d0770 msgr2=0x7ff7f00d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7ee59c640 1 --2- 172.21.15.133:0/864831655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7f00d0770 0x7ff7f00d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7ee59c640 1 -- 172.21.15.133:0/864831655 >> v2:172.21.15.162:3300/0 conn(0x7ff7f01614c0 msgr2=0x7ff7f0161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7ee59c640 1 --2- 172.21.15.133:0/864831655 >> v2:172.21.15.162:3300/0 conn(0x7ff7f01614c0 0x7ff7f0161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7ee59c640 1 -- 172.21.15.133:0/864831655 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f00b7a20 con 0x7ff7f015db00 2024-04-25T10:06:01.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7eed9d640 1 --2- 172.21.15.133:0/864831655 >> v2:172.21.15.162:3300/0 conn(0x7ff7f01614c0 0x7ff7f0161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:01.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7ee59c640 1 --2- 172.21.15.133:0/864831655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7f015db00 0x7ff7f00b2990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff7f006cf70 tx=0x7ff7e001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747b69fdd2eb5d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7edd9b640 1 -- 172.21.15.133:0/864831655 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00aa610 con 0x7ff7f015db00 2024-04-25T10:06:01.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7edd9b640 1 -- 172.21.15.133:0/864831655 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e00af070 con 0x7ff7f015db00 2024-04-25T10:06:01.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.466+0000 7ff7edd9b640 1 -- 172.21.15.133:0/864831655 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00aa9d0 con 0x7ff7f015db00 2024-04-25T10:06:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/864831655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7f015db00 msgr2=0x7ff7f00b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 --2- 172.21.15.133:0/864831655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7f015db00 0x7ff7f00b2990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff7f006cf70 tx=0x7ff7e001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/864831655 shutdown_connections 2024-04-25T10:06:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 --2- 172.21.15.133:0/864831655 >> v2:172.21.15.162:3300/0 conn(0x7ff7f01614c0 0x7ff7f0161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 --2- 172.21.15.133:0/864831655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7f00d0770 0x7ff7f00d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 --2- 172.21.15.133:0/864831655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7f015db00 0x7ff7f00b2990 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/864831655 >> 172.21.15.133:0/864831655 conn(0x7ff7f00727c0 msgr2=0x7ff7f0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/864831655 shutdown_connections 2024-04-25T10:06:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/864831655 wait complete. 2024-04-25T10:06:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 Processor -- start 2024-04-25T10:06:01.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 -- start start 2024-04-25T10:06:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00d0770 0x7ff7f015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7f015db00 0x7ff7f015e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7f01614c0 0x7ff7f015eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00b7430 con 0x7ff7f01614c0 2024-04-25T10:06:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00b82c0 con 0x7ff7f00d0770 2024-04-25T10:06:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00d18a0 con 0x7ff7f015db00 2024-04-25T10:06:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7eed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7f01614c0 0x7ff7f015eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7eed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7f01614c0 0x7ff7f015eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50786/0 (socket says 172.21.15.133:50786) 2024-04-25T10:06:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7eed9d640 1 -- 172.21.15.133:0/3885932178 learned_addr learned my addr 172.21.15.133:0/3885932178 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7ee59c640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00d0770 0x7ff7f015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7e7fff640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7f015db00 0x7ff7f015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7eed9d640 1 -- 172.21.15.133:0/3885932178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7f015db00 msgr2=0x7ff7f015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7eed9d640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7f015db00 0x7ff7f015e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7eed9d640 1 -- 172.21.15.133:0/3885932178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00d0770 msgr2=0x7ff7f015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7eed9d640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00d0770 0x7ff7f015a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7eed9d640 1 -- 172.21.15.133:0/3885932178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f015fac0 con 0x7ff7f01614c0 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7eed9d640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7f01614c0 0x7ff7f015eed0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800cef0 tx=0x7ff7e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7e6ffd640 1 -- 172.21.15.133:0/3885932178 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e801a070 con 0x7ff7f01614c0 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7e6ffd640 1 -- 172.21.15.133:0/3885932178 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e800d860 con 0x7ff7f01614c0 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/3885932178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f015f560 con 0x7ff7f01614c0 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7e6ffd640 1 -- 172.21.15.133:0/3885932178 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800dc50 con 0x7ff7f01614c0 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.470+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/3885932178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f01f6c00 con 0x7ff7f01614c0 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.474+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/3885932178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f0068ef0 con 0x7ff7f01614c0 2024-04-25T10:06:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.474+0000 7ff7e6ffd640 1 -- 172.21.15.133:0/3885932178 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7e801d070 con 0x7ff7f01614c0 2024-04-25T10:06:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.474+0000 7ff7e6ffd640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7d0066f60 0x7ff7d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.474+0000 7ff7e6ffd640 1 -- 172.21.15.133:0/3885932178 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7e8085a70 con 0x7ff7f01614c0 2024-04-25T10:06:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.474+0000 7ff7ee59c640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7d0066f60 0x7ff7d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.474+0000 7ff7ee59c640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7d0066f60 0x7ff7d0069420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00ae000 tx=0x7ff7e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.478+0000 7ff7e6ffd640 1 -- 172.21.15.133:0/3885932178 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7f015f560 con 0x7ff7f01614c0 2024-04-25T10:06:01.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.670+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/3885932178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7f00b2750 con 0x7ff7f01614c0 2024-04-25T10:06:01.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.670+0000 7ff7e6ffd640 1 -- 172.21.15.133:0/3885932178 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7ff7e80569b0 con 0x7ff7f01614c0 2024-04-25T10:06:01.675 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:01.675 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:05:38.566660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:01.675 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2024-04-25T10:06:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/3885932178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7d0066f60 msgr2=0x7ff7d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7d0066f60 0x7ff7d0069420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00ae000 tx=0x7ff7e0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/3885932178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7f01614c0 msgr2=0x7ff7f015eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7f01614c0 0x7ff7f015eed0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800cef0 tx=0x7ff7e800a710 comp rx=0 tx=0).stop 2024-04-25T10:06:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/3885932178 shutdown_connections 2024-04-25T10:06:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7d0066f60 0x7ff7d0069420 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7f01614c0 0x7ff7f015eed0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7f015db00 0x7ff7f015e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 --2- 172.21.15.133:0/3885932178 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7f00d0770 0x7ff7f015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/3885932178 >> 172.21.15.133:0/3885932178 conn(0x7ff7f00727c0 msgr2=0x7ff7f00b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/3885932178 shutdown_connections 2024-04-25T10:06:01.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.674+0000 7ff7f4d5e640 1 -- 172.21.15.133:0/3885932178 wait complete. 2024-04-25T10:06:01.693 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:06:01.847 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:01.848 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:06:01.848 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:01.861 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:01.861 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:06:01.862 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:01.862 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:06:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f91473a4640 1 Processor -- start 2024-04-25T10:06:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f91473a4640 1 -- start start 2024-04-25T10:06:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f91473a4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f914015abf0 0x7f91400ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f91473a4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91400cebd0 0x7f91400cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f91473a4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91400cc410 con 0x7f914015a810 2024-04-25T10:06:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f91473a4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91400b00b0 con 0x7f914015abf0 2024-04-25T10:06:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f91473a4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91400b0390 con 0x7f91400cebd0 2024-04-25T10:06:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f914493f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f914015abf0 0x7f91400ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f914493f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f914015abf0 0x7f91400ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50792/0 (socket says 172.21.15.133:50792) 2024-04-25T10:06:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f9145140640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f914015a810 0x7f91400cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60230/0 (socket says 172.21.15.133:60230) 2024-04-25T10:06:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f914493f640 1 -- 172.21.15.133:0/3616082141 learned_addr learned my addr 172.21.15.133:0/3616082141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f9145941640 1 --2- 172.21.15.133:0/3616082141 >> v2:172.21.15.162:3300/0 conn(0x7f91400cebd0 0x7f91400cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f914493f640 1 -- 172.21.15.133:0/3616082141 >> v2:172.21.15.162:3300/0 conn(0x7f91400cebd0 msgr2=0x7f91400cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f914493f640 1 --2- 172.21.15.133:0/3616082141 >> v2:172.21.15.162:3300/0 conn(0x7f91400cebd0 0x7f91400cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f914493f640 1 -- 172.21.15.133:0/3616082141 >> v1:172.21.15.133:6789/0 conn(0x7f914015a810 legacy=0x7f91400cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.982+0000 7f914493f640 1 -- 172.21.15.133:0/3616082141 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91400b0670 con 0x7f914015abf0 2024-04-25T10:06:01.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f914493f640 1 --2- 172.21.15.133:0/3616082141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f914015abf0 0x7f91400ce5a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f91400d16e0 tx=0x7f913001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e667c2cbb2d5a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9137fff640 1 -- 172.21.15.133:0/3616082141 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300ab430 con 0x7f914015abf0 2024-04-25T10:06:01.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9137fff640 1 -- 172.21.15.133:0/3616082141 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91300b0070 con 0x7f914015abf0 2024-04-25T10:06:01.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9137fff640 1 -- 172.21.15.133:0/3616082141 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300ab7f0 con 0x7f914015abf0 2024-04-25T10:06:01.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 -- 172.21.15.133:0/3616082141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f914015abf0 msgr2=0x7f91400ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 --2- 172.21.15.133:0/3616082141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f914015abf0 0x7f91400ce5a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f91400d16e0 tx=0x7f913001c520 comp rx=0 tx=0).stop 2024-04-25T10:06:01.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 -- 172.21.15.133:0/3616082141 shutdown_connections 2024-04-25T10:06:01.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 --2- 172.21.15.133:0/3616082141 >> v2:172.21.15.162:3300/0 conn(0x7f91400cebd0 0x7f91400cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 --2- 172.21.15.133:0/3616082141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f914015abf0 0x7f91400ce5a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 -- 172.21.15.133:0/3616082141 >> 172.21.15.133:0/3616082141 conn(0x7f91400d0610 msgr2=0x7f91400d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:01.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 -- 172.21.15.133:0/3616082141 shutdown_connections 2024-04-25T10:06:01.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 -- 172.21.15.133:0/3616082141 wait complete. 2024-04-25T10:06:01.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 Processor -- start 2024-04-25T10:06:01.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 -- start start 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91400cebd0 0x7f9140153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f914015a810 0x7f914006aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914015abf0 0x7f914014cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91400b17b0 con 0x7f91400cebd0 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91400b1630 con 0x7f914015a810 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f91473a4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91400b1930 con 0x7f914015abf0 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9145140640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91400cebd0 0x7f9140153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9145140640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91400cebd0 0x7f9140153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50806/0 (socket says 172.21.15.133:50806) 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9145140640 1 -- 172.21.15.133:0/3273932490 learned_addr learned my addr 172.21.15.133:0/3273932490 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f914493f640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f914015a810 0x7f914006aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9145941640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914015abf0 0x7f914014cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9145140640 1 -- 172.21.15.133:0/3273932490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914015abf0 msgr2=0x7f914014cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9145140640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914015abf0 0x7f914014cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9145140640 1 -- 172.21.15.133:0/3273932490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f914015a810 msgr2=0x7f914006aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9145140640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f914015a810 0x7f914006aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.986+0000 7f9145140640 1 -- 172.21.15.133:0/3273932490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91400b5160 con 0x7f91400cebd0 2024-04-25T10:06:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f9145140640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91400cebd0 0x7f9140153500 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f913c00c790 tx=0x7f913c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f9135ffb640 1 -- 172.21.15.133:0/3273932490 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c017070 con 0x7f91400cebd0 2024-04-25T10:06:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f9135ffb640 1 -- 172.21.15.133:0/3273932490 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c00a640 con 0x7f91400cebd0 2024-04-25T10:06:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f91473a4640 1 -- 172.21.15.133:0/3273932490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91400b5990 con 0x7f91400cebd0 2024-04-25T10:06:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f9135ffb640 1 -- 172.21.15.133:0/3273932490 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c00a9f0 con 0x7f91400cebd0 2024-04-25T10:06:01.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f91473a4640 1 -- 172.21.15.133:0/3273932490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91400b56b0 con 0x7f91400cebd0 2024-04-25T10:06:01.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f9135ffb640 1 -- 172.21.15.133:0/3273932490 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f913c01a070 con 0x7f91400cebd0 2024-04-25T10:06:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f9135ffb640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9124066f90 0x7f9124069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f9135ffb640 1 -- 172.21.15.133:0/3273932490 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f913c0846b0 con 0x7f91400cebd0 2024-04-25T10:06:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f914493f640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9124066f90 0x7f9124069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f914493f640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9124066f90 0x7f9124069450 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f91300af000 tx=0x7f9130008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f914a511000 1 Processor -- start 2024-04-25T10:06:01.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.990+0000 7f914a511000 1 -- start start 2024-04-25T10:06:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873540200 0x55c8735405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873524a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873524f60 0x55c873525340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8733dd4a0 con 0x55c873540b20 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c873345210 con 0x55c873524f60 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8732cd980 con 0x55c873540200 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914493f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873524a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914493f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873524a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50816/0 (socket says 172.21.15.133:50816) 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914493f640 1 -- 172.21.15.133:0/2504774044 learned_addr learned my addr 172.21.15.133:0/2504774044 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145140640 1 --2- 172.21.15.133:0/2504774044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873540200 0x55c8735405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914493f640 1 -- 172.21.15.133:0/2504774044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873540200 msgr2=0x55c8735405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914493f640 1 --2- 172.21.15.133:0/2504774044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873540200 0x55c8735405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914493f640 1 -- 172.21.15.133:0/2504774044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873524f60 msgr2=0x55c873525340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914493f640 1 --2- 172.21.15.133:0/2504774044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873524f60 0x55c873525340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:01.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914493f640 1 -- 172.21.15.133:0/2504774044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c873340200 con 0x55c873540b20 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145140640 1 --2- 172.21.15.133:0/2504774044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873540200 0x55c8735405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914493f640 1 --2- 172.21.15.133:0/2504774044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873524a20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9130019130 tx=0x7f9130004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa43993aef2d6ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f912affd640 1 -- 172.21.15.133:0/2504774044 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300b4ac0 con 0x55c873540b20 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f912affd640 1 -- 172.21.15.133:0/2504774044 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91300c4070 con 0x55c873540b20 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f912affd640 1 -- 172.21.15.133:0/2504774044 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300b4e60 con 0x55c873540b20 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- 172.21.15.133:0/2504774044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 msgr2=0x55c873524a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 --2- 172.21.15.133:0/2504774044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873524a20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9130019130 tx=0x7f9130004740 comp rx=0 tx=0).stop 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- 172.21.15.133:0/2504774044 shutdown_connections 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 --2- 172.21.15.133:0/2504774044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873524f60 0x55c873525340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 --2- 172.21.15.133:0/2504774044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873524a20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 --2- 172.21.15.133:0/2504774044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873540200 0x55c8735405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- 172.21.15.133:0/2504774044 >> 172.21.15.133:0/2504774044 conn(0x55c8734ed4f0 msgr2=0x55c8734fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- 172.21.15.133:0/2504774044 shutdown_connections 2024-04-25T10:06:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- 172.21.15.133:0/2504774044 wait complete. 2024-04-25T10:06:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- start start 2024-04-25T10:06:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 Processor -- start 2024-04-25T10:06:02.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873524f60 0x55c873547980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873540200 0x55c873547ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873525340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8732ffd80 con 0x55c873540b20 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8733412b0 con 0x55c873524f60 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c873306830 con 0x55c873540200 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 10 client.0 _ll_register_callbacks cb 0x55c8733f8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145941640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873525340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145941640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873525340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50826/0 (socket says 172.21.15.133:50826) 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145941640 1 -- 172.21.15.133:0/2691095683 learned_addr learned my addr 172.21.15.133:0/2691095683 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914493f640 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873540200 0x55c873547ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145140640 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873524f60 0x55c873547980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145941640 1 -- 172.21.15.133:0/2691095683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873540200 msgr2=0x55c873547ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145941640 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873540200 0x55c873547ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145941640 1 -- 172.21.15.133:0/2691095683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873524f60 msgr2=0x55c873547980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145941640 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873524f60 0x55c873547980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145941640 1 -- 172.21.15.133:0/2691095683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c873311130 con 0x55c873540b20 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f9145941640 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873525340 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f913800bf60 tx=0x7f913800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9380f2299f5c23d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f911ffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f911ffff640 1 -- 172.21.15.133:0/2691095683 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138016070 con 0x55c873540b20 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f911ffff640 1 -- 172.21.15.133:0/2691095683 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9138004220 con 0x55c873540b20 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.994+0000 7f914a511000 1 -- 172.21.15.133:0/2691095683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91300c4070 con 0x55c873540b20 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f911ffff640 1 -- 172.21.15.133:0/2691095683 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9138020050 con 0x55c873540b20 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 10 client.6279 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 -- 172.21.15.133:0/2691095683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55c8733c6490 con 0x55c873540b20 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f911ffff640 1 -- 172.21.15.133:0/2691095683 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91300c4070 con 0x55c873540b20 2024-04-25T10:06:02.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f911ffff640 1 -- 172.21.15.133:0/2691095683 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9138023020 con 0x55c873540b20 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 10 client.6279 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 10 client.6279 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 20 client.6279 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55c8735260e0 0x55c8735264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 4 client.6279 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 -- 172.21.15.133:0/2691095683 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8732e1020 con 0x55c8735260e0 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f9145140640 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55c8735260e0 0x55c8735264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f911ffff640 10 client.6279 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f9145140640 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55c8735260e0 0x55c8735264c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e354d0c19d5b17c1 server_cookie=58a2de3bf6ce52dc in_seq=0 out_seq=0 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f911ffff640 1 -- 172.21.15.133:0/2691095683 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c8732e1020 con 0x55c8735260e0 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f911ffff640 10 client.6279 handle_command_reply: tid=0 2024-04-25T10:06:02.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 client.6279 shutdown 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 -- 172.21.15.133:0/2691095683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 msgr2=0x55c873525340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873525340 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f913800bf60 tx=0x7f913800bf90 comp rx=0 tx=0).stop 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 -- 172.21.15.133:0/2691095683 shutdown_connections 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55c8735260e0 0x55c8735264c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c873525340 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873540200 0x55c873547ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f911ffff640 0 client.6279 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 --2- 172.21.15.133:0/2691095683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873524f60 0x55c873547980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 -- 172.21.15.133:0/2691095683 >> 172.21.15.133:0/2691095683 conn(0x55c8734ed4f0 msgr2=0x55c8734f4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 -- 172.21.15.133:0/2691095683 shutdown_connections 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 1 -- 172.21.15.133:0/2691095683 wait complete. 2024-04-25T10:06:02.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:01.998+0000 7f914a511000 20 client.6279 trim_cache size 0 max 16384 2024-04-25T10:06:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 Processor -- start 2024-04-25T10:06:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 -- start start 2024-04-25T10:06:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873524f60 0x55c8734faf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873540200 0x55c8734fb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c8734fb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9140200a50 con 0x55c873540b20 2024-04-25T10:06:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91401ffbd0 con 0x55c873540200 2024-04-25T10:06:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914015c010 con 0x55c873524f60 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f9145941640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c8734fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f9145941640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c8734fb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50838/0 (socket says 172.21.15.133:50838) 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f9145941640 1 -- 172.21.15.133:0/165884682 learned_addr learned my addr 172.21.15.133:0/165884682 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f9145941640 1 -- 172.21.15.133:0/165884682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873524f60 msgr2=0x55c8734faf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f9145941640 1 --2- 172.21.15.133:0/165884682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873524f60 0x55c8734faf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f9145941640 1 -- 172.21.15.133:0/165884682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873540200 msgr2=0x55c8734fb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f9145941640 1 --2- 172.21.15.133:0/165884682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873540200 0x55c8734fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f9145941640 1 -- 172.21.15.133:0/165884682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c873304a30 con 0x55c873540b20 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f9145941640 1 --2- 172.21.15.133:0/165884682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c8734fb980 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9138006f10 tx=0x7f9138012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d78585dfa0401d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f912a7fc640 1 -- 172.21.15.133:0/165884682 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138028020 con 0x55c873540b20 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f912a7fc640 1 -- 172.21.15.133:0/165884682 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9138004230 con 0x55c873540b20 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f912a7fc640 1 -- 172.21.15.133:0/165884682 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138016050 con 0x55c873540b20 2024-04-25T10:06:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 -- 172.21.15.133:0/165884682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 msgr2=0x55c8734fb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 --2- 172.21.15.133:0/165884682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c8734fb980 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9138006f10 tx=0x7f9138012000 comp rx=0 tx=0).stop 2024-04-25T10:06:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 -- 172.21.15.133:0/165884682 shutdown_connections 2024-04-25T10:06:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 --2- 172.21.15.133:0/165884682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540b20 0x55c8734fb980 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 --2- 172.21.15.133:0/165884682 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873540200 0x55c8734fb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 --2- 172.21.15.133:0/165884682 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873524f60 0x55c8734faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 -- 172.21.15.133:0/165884682 >> 172.21.15.133:0/165884682 conn(0x55c8734ed4f0 msgr2=0x55c8735236f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 -- 172.21.15.133:0/165884682 shutdown_connections 2024-04-25T10:06:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 -- 172.21.15.133:0/165884682 wait complete. 2024-04-25T10:06:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 -- start start 2024-04-25T10:06:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.014+0000 7f914a511000 1 Processor -- start 2024-04-25T10:06:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873524f60 0x55c873562630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540200 0x55c873562b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873540b20 0x55c8734f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8732d1190 con 0x55c873540200 2024-04-25T10:06:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914015c610 con 0x55c873540b20 2024-04-25T10:06:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c873349840 con 0x55c873524f60 2024-04-25T10:06:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 10 client.0 _ll_register_callbacks cb 0x7f91300096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914493f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540200 0x55c873562b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914493f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540200 0x55c873562b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50846/0 (socket says 172.21.15.133:50846) 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914493f640 1 -- 172.21.15.133:0/1647037161 learned_addr learned my addr 172.21.15.133:0/1647037161 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f9145140640 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873524f60 0x55c873562630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914493f640 1 -- 172.21.15.133:0/1647037161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873524f60 msgr2=0x55c873562630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f9145941640 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873540b20 0x55c8734f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914493f640 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873524f60 0x55c873562630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914493f640 1 -- 172.21.15.133:0/1647037161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873540b20 msgr2=0x55c8734f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914493f640 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873540b20 0x55c8734f5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914493f640 1 -- 172.21.15.133:0/1647037161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c87330d040 con 0x55c873540200 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914493f640 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540200 0x55c873562b70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f91300c1040 tx=0x7f91300192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e73012b66eff42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f911ffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f911ffff640 1 -- 172.21.15.133:0/1647037161 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130003820 con 0x55c873540200 2024-04-25T10:06:02.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f911ffff640 1 -- 172.21.15.133:0/1647037161 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91300039c0 con 0x55c873540200 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 -- 172.21.15.133:0/1647037161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9138004230 con 0x55c873540200 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f911ffff640 1 -- 172.21.15.133:0/1647037161 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91300d8050 con 0x55c873540200 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 10 client.6285 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 -- 172.21.15.133:0/1647037161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55c873001900 con 0x55c873540200 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f911ffff640 1 -- 172.21.15.133:0/1647037161 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9138004230 con 0x55c873540200 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f911ffff640 1 -- 172.21.15.133:0/1647037161 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91300aa070 con 0x55c873540200 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 10 client.6285 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 10 client.6285 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 20 client.6285 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55c8734f57a0 0x55c8734f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 4 client.6285 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 -- 172.21.15.133:0/1647037161 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x55c873352750 con 0x55c8734f57a0 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f9145140640 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55c8734f57a0 0x55c8734f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f911ffff640 10 client.6285 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f9145140640 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55c8734f57a0 0x55c8734f5b80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79317f7fdc8f1f8f server_cookie=30ab61c644938c1a in_seq=0 out_seq=0 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f911ffff640 1 -- 172.21.15.133:0/1647037161 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c873352750 con 0x55c8734f57a0 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f911ffff640 10 client.6285 handle_command_reply: tid=0 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 client.6285 shutdown 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 -- 172.21.15.133:0/1647037161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540200 msgr2=0x55c873562b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540200 0x55c873562b70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f91300c1040 tx=0x7f91300192c0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 -- 172.21.15.133:0/1647037161 shutdown_connections 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55c8734f57a0 0x55c8734f5b80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c873540b20 0x55c8734f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c873540200 0x55c873562b70 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 --2- 172.21.15.133:0/1647037161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c873524f60 0x55c873562630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 -- 172.21.15.133:0/1647037161 >> 172.21.15.133:0/1647037161 conn(0x55c8734ed4f0 msgr2=0x55c8734f7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f911ffff640 0 client.6285 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:02.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 -- 172.21.15.133:0/1647037161 shutdown_connections 2024-04-25T10:06:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 1 -- 172.21.15.133:0/1647037161 wait complete. 2024-04-25T10:06:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f914a511000 20 client.6285 trim_cache size 0 max 16384 2024-04-25T10:06:02.029 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:02.029 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:06:02.029 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "eee14728-fd00-4bfb-ab1a-cb75f0e9d170", 2024-04-25T10:06:02.029 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:06:02.029 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 -- 172.21.15.133:0/3273932490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9124066f90 msgr2=0x7f9124069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9124066f90 0x7f9124069450 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f91300af000 tx=0x7f9130008040 comp rx=0 tx=0).stop 2024-04-25T10:06:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 -- 172.21.15.133:0/3273932490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91400cebd0 msgr2=0x7f9140153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91400cebd0 0x7f9140153500 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f913c00c790 tx=0x7f913c00cc50 comp rx=0 tx=0).stop 2024-04-25T10:06:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 -- 172.21.15.133:0/3273932490 shutdown_connections 2024-04-25T10:06:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f914015abf0 0x7f914014cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f914015a810 0x7f914006aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9124066f90 0x7f9124069450 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 --2- 172.21.15.133:0/3273932490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91400cebd0 0x7f9140153500 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 -- 172.21.15.133:0/3273932490 >> 172.21.15.133:0/3273932490 conn(0x7f91400d0610 msgr2=0x7f9140157990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 -- 172.21.15.133:0/3273932490 shutdown_connections 2024-04-25T10:06:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.018+0000 7f91473a4640 1 -- 172.21.15.133:0/3273932490 wait complete. 2024-04-25T10:06:02.039 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'eee14728-fd00-4bfb-ab1a-cb75f0e9d170', 'mode': 'asynchronous'}' 2024-04-25T10:06:02.039 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T10:06:02.039 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:06:02.040 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:02.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb966ae5640 1 Processor -- start 2024-04-25T10:06:02.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb966ae5640 1 -- start start 2024-04-25T10:06:02.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb966ae5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb96015d910 0x7fb960161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb966ae5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb96015dcf0 0x7fb9600ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb966ae5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb9600b92e0 0x7fb9600ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb966ae5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb960161e70 con 0x7fb96015dcf0 2024-04-25T10:06:02.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb966ae5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb960162150 con 0x7fb9600b92e0 2024-04-25T10:06:02.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb966ae5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb960162430 con 0x7fb96015d910 2024-04-25T10:06:02.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb95ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb96015dcf0 0x7fb9600ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb95ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb96015dcf0 0x7fb9600ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50862/0 (socket says 172.21.15.133:50862) 2024-04-25T10:06:02.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb95ffff640 1 -- 172.21.15.133:0/1459485842 learned_addr learned my addr 172.21.15.133:0/1459485842 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb95ffff640 1 -- 172.21.15.133:0/1459485842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb96015d910 msgr2=0x7fb960161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb95ffff640 1 --2- 172.21.15.133:0/1459485842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb96015d910 0x7fb960161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb95ffff640 1 -- 172.21.15.133:0/1459485842 >> v2:172.21.15.162:3300/0 conn(0x7fb9600b92e0 msgr2=0x7fb9600ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb965082640 1 --2- 172.21.15.133:0/1459485842 >> v2:172.21.15.162:3300/0 conn(0x7fb9600b92e0 0x7fb9600ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb95ffff640 1 --2- 172.21.15.133:0/1459485842 >> v2:172.21.15.162:3300/0 conn(0x7fb9600b92e0 0x7fb9600ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.166+0000 7fb95ffff640 1 -- 172.21.15.133:0/1459485842 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9600d1d20 con 0x7fb96015dcf0 2024-04-25T10:06:02.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb95ffff640 1 --2- 172.21.15.133:0/1459485842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb96015dcf0 0x7fb9600ce620 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb9600d4a00 tx=0x7fb94c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ac40491eb14d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb95f7fe640 1 -- 172.21.15.133:0/1459485842 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0a7070 con 0x7fb96015dcf0 2024-04-25T10:06:02.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb95f7fe640 1 -- 172.21.15.133:0/1459485842 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c0b0070 con 0x7fb96015dcf0 2024-04-25T10:06:02.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb95f7fe640 1 -- 172.21.15.133:0/1459485842 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0ab890 con 0x7fb96015dcf0 2024-04-25T10:06:02.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 -- 172.21.15.133:0/1459485842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb96015dcf0 msgr2=0x7fb9600ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 --2- 172.21.15.133:0/1459485842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb96015dcf0 0x7fb9600ce620 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb9600d4a00 tx=0x7fb94c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 -- 172.21.15.133:0/1459485842 shutdown_connections 2024-04-25T10:06:02.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 --2- 172.21.15.133:0/1459485842 >> v2:172.21.15.162:3300/0 conn(0x7fb9600b92e0 0x7fb9600ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 --2- 172.21.15.133:0/1459485842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb96015dcf0 0x7fb9600ce620 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 --2- 172.21.15.133:0/1459485842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb96015d910 0x7fb960161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 -- 172.21.15.133:0/1459485842 >> 172.21.15.133:0/1459485842 conn(0x7fb9600d03d0 msgr2=0x7fb9600d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 -- 172.21.15.133:0/1459485842 shutdown_connections 2024-04-25T10:06:02.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 -- 172.21.15.133:0/1459485842 wait complete. 2024-04-25T10:06:02.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 Processor -- start 2024-04-25T10:06:02.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 -- start start 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9600b92e0 0x7fb960160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb96015d910 0x7fb96015e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb96015dcf0 0x7fb960154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb960072d50 con 0x7fb9600b92e0 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb960072bd0 con 0x7fb96015d910 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.170+0000 7fb966ae5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb960072ed0 con 0x7fb96015dcf0 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb964881640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9600b92e0 0x7fb960160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb964881640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9600b92e0 0x7fb960160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50870/0 (socket says 172.21.15.133:50870) 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb964881640 1 -- 172.21.15.133:0/1625304372 learned_addr learned my addr 172.21.15.133:0/1625304372 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb965082640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb96015dcf0 0x7fb960154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb95ffff640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb96015d910 0x7fb96015e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb964881640 1 -- 172.21.15.133:0/1625304372 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb96015dcf0 msgr2=0x7fb960154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb964881640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb96015dcf0 0x7fb960154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb964881640 1 -- 172.21.15.133:0/1625304372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb96015d910 msgr2=0x7fb96015e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb964881640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb96015d910 0x7fb96015e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb964881640 1 -- 172.21.15.133:0/1625304372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9601553c0 con 0x7fb9600b92e0 2024-04-25T10:06:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb964881640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9600b92e0 0x7fb960160cf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb95400b550 tx=0x7fb95400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb95d7fa640 1 -- 172.21.15.133:0/1625304372 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954014070 con 0x7fb9600b92e0 2024-04-25T10:06:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb95d7fa640 1 -- 172.21.15.133:0/1625304372 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb954010890 con 0x7fb9600b92e0 2024-04-25T10:06:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb966ae5640 1 -- 172.21.15.133:0/1625304372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96015e840 con 0x7fb9600b92e0 2024-04-25T10:06:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb95d7fa640 1 -- 172.21.15.133:0/1625304372 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954010c80 con 0x7fb9600b92e0 2024-04-25T10:06:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb966ae5640 1 -- 172.21.15.133:0/1625304372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96015ee00 con 0x7fb9600b92e0 2024-04-25T10:06:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb966ae5640 1 -- 172.21.15.133:0/1625304372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb960068ef0 con 0x7fb9600b92e0 2024-04-25T10:06:02.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb95d7fa640 1 -- 172.21.15.133:0/1625304372 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb954017070 con 0x7fb9600b92e0 2024-04-25T10:06:02.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb95d7fa640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb944066f90 0x7fb944069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb95d7fa640 1 -- 172.21.15.133:0/1625304372 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb954084ff0 con 0x7fb9600b92e0 2024-04-25T10:06:02.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb95ffff640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb944066f90 0x7fb944069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.174+0000 7fb95ffff640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb944066f90 0x7fb944069450 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0af000 tx=0x7fb94c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.178+0000 7fb95d7fa640 1 -- 172.21.15.133:0/1625304372 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb954055e10 con 0x7fb9600b92e0 2024-04-25T10:06:02.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.358+0000 7fb966ae5640 1 -- 172.21.15.133:0/1625304372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9600d15f0 con 0x7fb9600b92e0 2024-04-25T10:06:02.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.358+0000 7fb95d7fa640 1 -- 172.21.15.133:0/1625304372 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fb954010e20 con 0x7fb9600b92e0 2024-04-25T10:06:02.363 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:02.363 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:05:38.566660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:02.363 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2024-04-25T10:06:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 -- 172.21.15.133:0/1625304372 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb944066f90 msgr2=0x7fb944069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb944066f90 0x7fb944069450 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0af000 tx=0x7fb94c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 -- 172.21.15.133:0/1625304372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9600b92e0 msgr2=0x7fb960160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9600b92e0 0x7fb960160cf0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb95400b550 tx=0x7fb95400ba10 comp rx=0 tx=0).stop 2024-04-25T10:06:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 -- 172.21.15.133:0/1625304372 shutdown_connections 2024-04-25T10:06:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb96015dcf0 0x7fb960154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb96015d910 0x7fb96015e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb944066f90 0x7fb944069450 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 --2- 172.21.15.133:0/1625304372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9600b92e0 0x7fb960160cf0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 -- 172.21.15.133:0/1625304372 >> 172.21.15.133:0/1625304372 conn(0x7fb9600d03d0 msgr2=0x7fb9600b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 -- 172.21.15.133:0/1625304372 shutdown_connections 2024-04-25T10:06:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.362+0000 7fb966ae5640 1 -- 172.21.15.133:0/1625304372 wait complete. 2024-04-25T10:06:02.381 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:02.515 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:02.516 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T10:06:02.516 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:02.516 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T10:06:02.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a0e2d7640 1 Processor -- start 2024-04-25T10:06:02.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a0e2d7640 1 -- start start 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a0e2d7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0a0815fd20 0x7f0a080ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a0e2d7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a08162ba0 0x7f0a080c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a0e2d7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a080ce0b0 con 0x7f0a080c4680 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a0e2d7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a08163030 con 0x7f0a0815fd20 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a0e2d7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a08163280 con 0x7f0a08162ba0 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a07fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0a0815fd20 0x7f0a080ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a077fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0a080c4680 0x7f0a08162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60234/0 (socket says 172.21.15.133:60234) 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a07fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0a0815fd20 0x7f0a080ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50884/0 (socket says 172.21.15.133:50884) 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a07fff640 1 -- 172.21.15.133:0/3840604484 learned_addr learned my addr 172.21.15.133:0/3840604484 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a0c874640 1 --2- 172.21.15.133:0/3840604484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a08162ba0 0x7f0a080c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a07fff640 1 -- 172.21.15.133:0/3840604484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a08162ba0 msgr2=0x7f0a080c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a07fff640 1 --2- 172.21.15.133:0/3840604484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a08162ba0 0x7f0a080c4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a07fff640 1 -- 172.21.15.133:0/3840604484 >> v1:172.21.15.133:6789/0 conn(0x7f0a080c4680 legacy=0x7f0a08162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a06ffd640 1 -- 172.21.15.133:0/3840604484 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748308315 0 0) 0x7f0a080ce0b0 con 0x7f0a080c4680 2024-04-25T10:06:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a07fff640 1 -- 172.21.15.133:0/3840604484 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a080ce370 con 0x7f0a0815fd20 2024-04-25T10:06:02.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.626+0000 7f0a0c874640 1 --2- 172.21.15.133:0/3840604484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a08162ba0 0x7f0a080c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a07fff640 1 --2- 172.21.15.133:0/3840604484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a0815fd20 0x7f0a080ceb70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009e50 tx=0x7f0a0001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d891fb4c4dce3840 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a06ffd640 1 -- 172.21.15.133:0/3840604484 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000a56e0 con 0x7f0a0815fd20 2024-04-25T10:06:02.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a06ffd640 1 -- 172.21.15.133:0/3840604484 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a000a5880 con 0x7f0a0815fd20 2024-04-25T10:06:02.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a06ffd640 1 -- 172.21.15.133:0/3840604484 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000a5c50 con 0x7f0a0815fd20 2024-04-25T10:06:02.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/3840604484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a0815fd20 msgr2=0x7f0a080ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 --2- 172.21.15.133:0/3840604484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a0815fd20 0x7f0a080ceb70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009e50 tx=0x7f0a0001c910 comp rx=0 tx=0).stop 2024-04-25T10:06:02.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/3840604484 shutdown_connections 2024-04-25T10:06:02.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 --2- 172.21.15.133:0/3840604484 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a08162ba0 0x7f0a080c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 --2- 172.21.15.133:0/3840604484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a0815fd20 0x7f0a080ceb70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/3840604484 >> 172.21.15.133:0/3840604484 conn(0x7f0a0809a990 msgr2=0x7f0a080d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/3840604484 shutdown_connections 2024-04-25T10:06:02.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/3840604484 wait complete. 2024-04-25T10:06:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 Processor -- start 2024-04-25T10:06:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 -- start start 2024-04-25T10:06:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a080c4680 0x7f0a0815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a0815fd20 0x7f0a08157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a08162ba0 0x7f0a080aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080cc8e0 con 0x7f0a080c4680 2024-04-25T10:06:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080cc760 con 0x7f0a08162ba0 2024-04-25T10:06:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a0e2d7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0816ec40 con 0x7f0a0815fd20 2024-04-25T10:06:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a080c4680 0x7f0a0815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a080c4680 0x7f0a0815ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50890/0 (socket says 172.21.15.133:50890) 2024-04-25T10:06:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.630+0000 7f0a07fff640 1 -- 172.21.15.133:0/622111785 learned_addr learned my addr 172.21.15.133:0/622111785 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a07fff640 1 -- 172.21.15.133:0/622111785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a0815fd20 msgr2=0x7f0a08157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a07fff640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a0815fd20 0x7f0a08157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a07fff640 1 -- 172.21.15.133:0/622111785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a08162ba0 msgr2=0x7f0a080aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a07fff640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a08162ba0 0x7f0a080aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a07fff640 1 -- 172.21.15.133:0/622111785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a08158ee0 con 0x7f0a080c4680 2024-04-25T10:06:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a07fff640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a080c4680 0x7f0a0815ebb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0a0001a540 tx=0x7f0a000196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a04ff9640 1 -- 172.21.15.133:0/622111785 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000bc020 con 0x7f0a080c4680 2024-04-25T10:06:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a04ff9640 1 -- 172.21.15.133:0/622111785 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a000b8880 con 0x7f0a080c4680 2024-04-25T10:06:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/622111785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a08158090 con 0x7f0a080c4680 2024-04-25T10:06:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a04ff9640 1 -- 172.21.15.133:0/622111785 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000b8c70 con 0x7f0a080c4680 2024-04-25T10:06:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/622111785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a08158640 con 0x7f0a080c4680 2024-04-25T10:06:02.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a04ff9640 1 -- 172.21.15.133:0/622111785 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a000bf020 con 0x7f0a080c4680 2024-04-25T10:06:02.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a04ff9640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09e8066f60 0x7f09e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a04ff9640 1 -- 172.21.15.133:0/622111785 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0a00120400 con 0x7f0a080c4680 2024-04-25T10:06:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a077fe640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09e8066f60 0x7f09e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a077fe640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09e8066f60 0x7f09e8069420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0a08157b00 tx=0x7f09f0007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a11444000 1 Processor -- start 2024-04-25T10:06:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.634+0000 7f0a11444000 1 -- start start 2024-04-25T10:06:02.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c32ae000 0x55f0c32ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c3263ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0c30b1f40 con 0x55f0c3264410 2024-04-25T10:06:02.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0c3039fc0 con 0x55f0c32ae000 2024-04-25T10:06:02.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0c30adfe0 con 0x55f0c32ae920 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a0c874640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a0c874640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50898/0 (socket says 172.21.15.133:50898) 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a0c874640 1 -- 172.21.15.133:0/1920459142 learned_addr learned my addr 172.21.15.133:0/1920459142 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a07fff640 1 --2- 172.21.15.133:0/1920459142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c32ae000 0x55f0c32ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a0c874640 1 -- 172.21.15.133:0/1920459142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 msgr2=0x55f0c3263ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a077fe640 1 --2- 172.21.15.133:0/1920459142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c3263ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a0c874640 1 --2- 172.21.15.133:0/1920459142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c3263ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a0c874640 1 -- 172.21.15.133:0/1920459142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c32ae000 msgr2=0x55f0c32ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a0c874640 1 --2- 172.21.15.133:0/1920459142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c32ae000 0x55f0c32ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a0c874640 1 -- 172.21.15.133:0/1920459142 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0c30f9a50 con 0x55f0c3264410 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a0c874640 1 --2- 172.21.15.133:0/1920459142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32647f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f09f800a8f0 tx=0x7f09f800adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547ded955cfb8afb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f09fdffb640 1 -- 172.21.15.133:0/1920459142 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8015070 con 0x55f0c3264410 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f09fdffb640 1 -- 172.21.15.133:0/1920459142 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f800cb00 con 0x55f0c3264410 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f09fdffb640 1 -- 172.21.15.133:0/1920459142 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80113f0 con 0x55f0c3264410 2024-04-25T10:06:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- 172.21.15.133:0/1920459142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 msgr2=0x55f0c32647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 --2- 172.21.15.133:0/1920459142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32647f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f09f800a8f0 tx=0x7f09f800adb0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- 172.21.15.133:0/1920459142 shutdown_connections 2024-04-25T10:06:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 --2- 172.21.15.133:0/1920459142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32647f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 --2- 172.21.15.133:0/1920459142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c3263ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 --2- 172.21.15.133:0/1920459142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c32ae000 0x55f0c32ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- 172.21.15.133:0/1920459142 >> 172.21.15.133:0/1920459142 conn(0x55f0c32603c0 msgr2=0x55f0c32627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- 172.21.15.133:0/1920459142 shutdown_connections 2024-04-25T10:06:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- 172.21.15.133:0/1920459142 wait complete. 2024-04-25T10:06:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- start start 2024-04-25T10:06:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 Processor -- start 2024-04-25T10:06:02.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c3264410 0x55f0c326fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c3270340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c3270880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0c30b6570 con 0x55f0c32ae000 2024-04-25T10:06:02.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0c306c5f0 con 0x55f0c3264410 2024-04-25T10:06:02.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0c306e0a0 con 0x55f0c32ae920 2024-04-25T10:06:02.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 10 client.0 _ll_register_callbacks cb 0x55f0c31667a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c3270340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c3270340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50900/0 (socket says 172.21.15.133:50900) 2024-04-25T10:06:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a077fe640 1 -- 172.21.15.133:0/1762107305 learned_addr learned my addr 172.21.15.133:0/1762107305 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a07fff640 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c3264410 0x55f0c326fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a077fe640 1 -- 172.21.15.133:0/1762107305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 msgr2=0x55f0c3270880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a077fe640 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c3270880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a077fe640 1 -- 172.21.15.133:0/1762107305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c3264410 msgr2=0x55f0c326fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a077fe640 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c3264410 0x55f0c326fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a077fe640 1 -- 172.21.15.133:0/1762107305 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0c303a210 con 0x55f0c32ae000 2024-04-25T10:06:02.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a07fff640 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c3264410 0x55f0c326fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a077fe640 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c3270340 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f09f0005b50 tx=0x7f09f0001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8312834e5d588c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f09e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:02.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f09e6ffd640 1 -- 172.21.15.133:0/1762107305 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0012440 con 0x55f0c32ae000 2024-04-25T10:06:02.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f09e6ffd640 1 -- 172.21.15.133:0/1762107305 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09f0022c60 con 0x55f0c32ae000 2024-04-25T10:06:02.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- 172.21.15.133:0/1762107305 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09f800cb00 con 0x55f0c32ae000 2024-04-25T10:06:02.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f09e6ffd640 1 -- 172.21.15.133:0/1762107305 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09f0022e60 con 0x55f0c32ae000 2024-04-25T10:06:02.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 10 client.6306 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:02.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f0a11444000 1 -- 172.21.15.133:0/1762107305 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55f0c3136400 con 0x55f0c32ae000 2024-04-25T10:06:02.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.638+0000 7f09e6ffd640 1 -- 172.21.15.133:0/1762107305 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f800cb00 con 0x55f0c32ae000 2024-04-25T10:06:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f09e6ffd640 1 -- 172.21.15.133:0/1762107305 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f09f0029050 con 0x55f0c32ae000 2024-04-25T10:06:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 10 client.6306 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 10 client.6306 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 20 client.6306 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55f0c3271290 0x55f0c32c02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 4 client.6306 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:02.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 -- 172.21.15.133:0/1762107305 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0c304dc90 con 0x55f0c3271290 2024-04-25T10:06:02.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a07fff640 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55f0c3271290 0x55f0c32c02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f09e6ffd640 10 client.6306 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:02.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a07fff640 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55f0c3271290 0x55f0c32c02c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=469a534a613c4068 server_cookie=815acc2bdabb974d in_seq=0 out_seq=0 2024-04-25T10:06:02.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f09e6ffd640 1 -- 172.21.15.133:0/1762107305 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0c304dc90 con 0x55f0c3271290 2024-04-25T10:06:02.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f09e6ffd640 10 client.6306 handle_command_reply: tid=0 2024-04-25T10:06:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 client.6306 shutdown 2024-04-25T10:06:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 -- 172.21.15.133:0/1762107305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 msgr2=0x55f0c3270340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c3270340 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f09f0005b50 tx=0x7f09f0001090 comp rx=0 tx=0).stop 2024-04-25T10:06:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 -- 172.21.15.133:0/1762107305 shutdown_connections 2024-04-25T10:06:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55f0c3271290 0x55f0c32c02c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c3270880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c3270340 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f09e6ffd640 0 client.6306 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:02.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 --2- 172.21.15.133:0/1762107305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c3264410 0x55f0c326fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 -- 172.21.15.133:0/1762107305 >> 172.21.15.133:0/1762107305 conn(0x55f0c32603c0 msgr2=0x55f0c3263040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 -- 172.21.15.133:0/1762107305 shutdown_connections 2024-04-25T10:06:02.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 1 -- 172.21.15.133:0/1762107305 wait complete. 2024-04-25T10:06:02.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.642+0000 7f0a11444000 20 client.6306 trim_cache size 0 max 16384 2024-04-25T10:06:02.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 Processor -- start 2024-04-25T10:06:02.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- start start 2024-04-25T10:06:02.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c32ae000 0x55f0c3263d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c32c02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a082071d0 con 0x55f0c3264410 2024-04-25T10:06:02.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080b20f0 con 0x55f0c32ae000 2024-04-25T10:06:02.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080b58e0 con 0x55f0c32ae920 2024-04-25T10:06:02.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32637c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50902/0 (socket says 172.21.15.133:50902) 2024-04-25T10:06:02.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a07fff640 1 -- 172.21.15.133:0/3489850218 learned_addr learned my addr 172.21.15.133:0/3489850218 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a07fff640 1 -- 172.21.15.133:0/3489850218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 msgr2=0x55f0c32c02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a07fff640 1 --2- 172.21.15.133:0/3489850218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c32c02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a07fff640 1 -- 172.21.15.133:0/3489850218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c32ae000 msgr2=0x55f0c3263d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a07fff640 1 --2- 172.21.15.133:0/3489850218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c32ae000 0x55f0c3263d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a07fff640 1 -- 172.21.15.133:0/3489850218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0c310cc10 con 0x55f0c3264410 2024-04-25T10:06:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a07fff640 1 --2- 172.21.15.133:0/3489850218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32637c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0a0001ce80 tx=0x7f0a0011e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bd557a957c23ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f09fd7fa640 1 -- 172.21.15.133:0/3489850218 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000f0b90 con 0x55f0c3264410 2024-04-25T10:06:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f09fd7fa640 1 -- 172.21.15.133:0/3489850218 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a000f4ce0 con 0x55f0c3264410 2024-04-25T10:06:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f09fd7fa640 1 -- 172.21.15.133:0/3489850218 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000f92a0 con 0x55f0c3264410 2024-04-25T10:06:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- 172.21.15.133:0/3489850218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 msgr2=0x55f0c32637c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 --2- 172.21.15.133:0/3489850218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32637c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0a0001ce80 tx=0x7f0a0011e700 comp rx=0 tx=0).stop 2024-04-25T10:06:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- 172.21.15.133:0/3489850218 shutdown_connections 2024-04-25T10:06:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 --2- 172.21.15.133:0/3489850218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c32c02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 --2- 172.21.15.133:0/3489850218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c32ae000 0x55f0c3263d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 --2- 172.21.15.133:0/3489850218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c3264410 0x55f0c32637c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- 172.21.15.133:0/3489850218 >> 172.21.15.133:0/3489850218 conn(0x55f0c32603c0 msgr2=0x55f0c326fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- 172.21.15.133:0/3489850218 shutdown_connections 2024-04-25T10:06:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- 172.21.15.133:0/3489850218 wait complete. 2024-04-25T10:06:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- start start 2024-04-25T10:06:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 Processor -- start 2024-04-25T10:06:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c3264410 0x55f0c32c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c32cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c32cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0c30b1250 con 0x55f0c32ae000 2024-04-25T10:06:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080b1af0 con 0x55f0c3264410 2024-04-25T10:06:02.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0c30efe80 con 0x55f0c32ae920 2024-04-25T10:06:02.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a11444000 10 client.0 _ll_register_callbacks cb 0x55f0c31042c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:02.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c32cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c32cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50918/0 (socket says 172.21.15.133:50918) 2024-04-25T10:06:02.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.658+0000 7f0a077fe640 1 -- 172.21.15.133:0/2828307009 learned_addr learned my addr 172.21.15.133:0/2828307009 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0c874640 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c32cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a07fff640 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c3264410 0x55f0c32c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a077fe640 1 -- 172.21.15.133:0/2828307009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 msgr2=0x55f0c32cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a077fe640 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c32cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a077fe640 1 -- 172.21.15.133:0/2828307009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c3264410 msgr2=0x55f0c32c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a077fe640 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c3264410 0x55f0c32c22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a077fe640 1 -- 172.21.15.133:0/2828307009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0c3046bb0 con 0x55f0c32ae000 2024-04-25T10:06:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0c874640 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c32cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a077fe640 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c32cad90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f09f00012d0 tx=0x7f09f0007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337a9c9b9f958f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f09e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:02.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f09e6ffd640 1 -- 172.21.15.133:0/2828307009 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0012440 con 0x55f0c32ae000 2024-04-25T10:06:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 -- 172.21.15.133:0/2828307009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a000f4ce0 con 0x55f0c32ae000 2024-04-25T10:06:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f09e6ffd640 1 -- 172.21.15.133:0/2828307009 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09f002b470 con 0x55f0c32ae000 2024-04-25T10:06:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f09e6ffd640 1 -- 172.21.15.133:0/2828307009 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09f00125e0 con 0x55f0c32ae000 2024-04-25T10:06:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 10 client.6312 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 -- 172.21.15.133:0/2828307009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55f0c3059030 con 0x55f0c32ae000 2024-04-25T10:06:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f09e6ffd640 1 -- 172.21.15.133:0/2828307009 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a000f4ce0 con 0x55f0c32ae000 2024-04-25T10:06:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f09e6ffd640 1 -- 172.21.15.133:0/2828307009 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f09f003f080 con 0x55f0c32ae000 2024-04-25T10:06:02.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 10 client.6312 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 10 client.6312 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 20 client.6312 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55f0c330e5e0 0x55f0c330e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 4 client.6312 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T10:06:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 -- 172.21.15.133:0/2828307009 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55f0c30bf480 con 0x55f0c330e5e0 2024-04-25T10:06:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a07fff640 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55f0c330e5e0 0x55f0c330e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f09e6ffd640 10 client.6312 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a07fff640 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55f0c330e5e0 0x55f0c330e9f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cb21fb1f1b6a522 server_cookie=1e7d5f7778181ff8 in_seq=0 out_seq=0 2024-04-25T10:06:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f09e6ffd640 1 -- 172.21.15.133:0/2828307009 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f0c30bf480 con 0x55f0c330e5e0 2024-04-25T10:06:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f09e6ffd640 10 client.6312 handle_command_reply: tid=0 2024-04-25T10:06:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 client.6312 shutdown 2024-04-25T10:06:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 -- 172.21.15.133:0/2828307009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 msgr2=0x55f0c32cad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c32cad90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f09f00012d0 tx=0x7f09f0007000 comp rx=0 tx=0).stop 2024-04-25T10:06:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 -- 172.21.15.133:0/2828307009 shutdown_connections 2024-04-25T10:06:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55f0c330e5e0 0x55f0c330e9f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f0c32ae920 0x55f0c32cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f09e6ffd640 0 client.6312 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0c32ae000 0x55f0c32cad90 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 --2- 172.21.15.133:0/2828307009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f0c3264410 0x55f0c32c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 -- 172.21.15.133:0/2828307009 >> 172.21.15.133:0/2828307009 conn(0x55f0c32603c0 msgr2=0x55f0c3270600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 -- 172.21.15.133:0/2828307009 shutdown_connections 2024-04-25T10:06:02.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 1 -- 172.21.15.133:0/2828307009 wait complete. 2024-04-25T10:06:02.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a11444000 20 client.6312 trim_cache size 0 max 16384 2024-04-25T10:06:02.682 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:02.682 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:06:02.682 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "49cfd23c-4ad6-4086-b030-cb4d274d07e5", 2024-04-25T10:06:02.683 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:06:02.683 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:02.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/622111785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09e8066f60 msgr2=0x7f09e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09e8066f60 0x7f09e8069420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0a08157b00 tx=0x7f09f0007970 comp rx=0 tx=0).stop 2024-04-25T10:06:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/622111785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a080c4680 msgr2=0x7f0a0815ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a080c4680 0x7f0a0815ebb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0a0001a540 tx=0x7f0a000196b0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/622111785 shutdown_connections 2024-04-25T10:06:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a08162ba0 0x7f0a080aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a0815fd20 0x7f0a08157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09e8066f60 0x7f09e8069420 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 --2- 172.21.15.133:0/622111785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a080c4680 0x7f0a0815ebb0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/622111785 >> 172.21.15.133:0/622111785 conn(0x7f0a0809a990 msgr2=0x7f0a080d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/622111785 shutdown_connections 2024-04-25T10:06:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.662+0000 7f0a0e2d7640 1 -- 172.21.15.133:0/622111785 wait complete. 2024-04-25T10:06:02.688 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '49cfd23c-4ad6-4086-b030-cb4d274d07e5', 'mode': 'asynchronous'}' 2024-04-25T10:06:02.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:02.689 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T10:06:02.697 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T10:06:02.697 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:02.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f110476a640 1 Processor -- start 2024-04-25T10:06:02.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f110476a640 1 -- start start 2024-04-25T10:06:02.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f110476a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f10fc15db00 0x7f10fc0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f110476a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc0d0770 0x7f10fc0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f110476a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10fc0ceb20 con 0x7f10fc1614c0 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f110476a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10fc0d1110 con 0x7f10fc15db00 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f110476a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0b7740 con 0x7f10fc0d0770 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1102d07640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f10fc1614c0 0x7f10fc161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60248/0 (socket says 172.21.15.133:60248) 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1102d07640 1 -- 172.21.15.133:0/1223227656 learned_addr learned my addr 172.21.15.133:0/1223227656 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101d05640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc0d0770 0x7f10fc0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1102506640 1 --2- 172.21.15.133:0/1223227656 >> v2:172.21.15.162:3300/0 conn(0x7f10fc15db00 0x7f10fc0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498671154 0 0) 0x7f10fc0ceb20 con 0x7f10fc1614c0 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e8003530 con 0x7f10fc1614c0 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4217525941 0 0) 0x7f10e8003530 con 0x7f10fc1614c0 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc0d0770 msgr2=0x7f10fc0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc0d0770 0x7f10fc0d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 >> v2:172.21.15.162:3300/0 conn(0x7f10fc15db00 msgr2=0x7f10fc0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 --2- 172.21.15.133:0/1223227656 >> v2:172.21.15.162:3300/0 conn(0x7f10fc15db00 0x7f10fc0b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc0b97f0 con 0x7f10fc1614c0 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f10f4001a00 con 0x7f10fc1614c0 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 >> v1:172.21.15.133:6789/0 conn(0x7f10fc1614c0 legacy=0x7f10fc161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e80039c0 0x7f10e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 0x7f10e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc0d0770 0x7f10e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0b7740 con 0x7f10e8004810 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e800b9d0 con 0x7f10e80039c0 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.854+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e800bb50 con 0x7f10fc0d0770 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1101d05640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 0x7f10e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1102506640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e80039c0 0x7f10e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1102d07640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc0d0770 0x7f10e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1101d05640 1 -- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc0d0770 msgr2=0x7f10e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1101d05640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc0d0770 0x7f10e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1101d05640 1 -- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e80039c0 msgr2=0x7f10e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1101d05640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e80039c0 0x7f10e8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1101d05640 1 -- 172.21.15.133:0/1223227656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e8003530 con 0x7f10e8004810 2024-04-25T10:06:02.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1102506640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10e80039c0 0x7f10e8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1102d07640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc0d0770 0x7f10e80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1101d05640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 0x7f10e8004c90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009ae0 tx=0x7f10f801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951c330c88488b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f80b2070 con 0x7f10e8004810 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f80a5580 con 0x7f10e8004810 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1101504640 1 -- 172.21.15.133:0/1223227656 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f80096b0 con 0x7f10e8004810 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f110476a640 1 -- 172.21.15.133:0/1223227656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 msgr2=0x7f10e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f110476a640 1 --2- 172.21.15.133:0/1223227656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 0x7f10e8004c90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009ae0 tx=0x7f10f801ca70 comp rx=0 tx=0).stop 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f1102d07640 1 -- 172.21.15.133:0/1223227656 reap_dead start 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f110476a640 1 -- 172.21.15.133:0/1223227656 shutdown_connections 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f110476a640 1 -- 172.21.15.133:0/1223227656 >> 172.21.15.133:0/1223227656 conn(0x7f10fc0727c0 msgr2=0x7f10fc072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f110476a640 1 -- 172.21.15.133:0/1223227656 shutdown_connections 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.858+0000 7f110476a640 1 -- 172.21.15.133:0/1223227656 wait complete. 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f110476a640 1 Processor -- start 2024-04-25T10:06:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f110476a640 1 -- start start 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f110476a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 0x7f10fc156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f110476a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc15f770 0x7f10fc151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f110476a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10fc0c8f40 0x7f10fc15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f110476a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0b7fe0 con 0x7f10e8004810 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f110476a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0b7d00 con 0x7f10fc0c8f40 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f110476a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc0b82c0 con 0x7f10fc15f770 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f1102506640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 0x7f10fc156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f1102506640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 0x7f10fc156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50936/0 (socket says 172.21.15.133:50936) 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f1102506640 1 -- 172.21.15.133:0/1328314476 learned_addr learned my addr 172.21.15.133:0/1328314476 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f1102506640 1 -- 172.21.15.133:0/1328314476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc15f770 msgr2=0x7f10fc151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f1102506640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc15f770 0x7f10fc151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f1102506640 1 -- 172.21.15.133:0/1328314476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10fc0c8f40 msgr2=0x7f10fc15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f1102506640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10fc0c8f40 0x7f10fc15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f1102506640 1 -- 172.21.15.133:0/1328314476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc15fc70 con 0x7f10e8004810 2024-04-25T10:06:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f1102506640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 0x7f10fc156630 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f10ec010950 tx=0x7f10ec010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f10f2ffd640 1 -- 172.21.15.133:0/1328314476 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec00f040 con 0x7f10e8004810 2024-04-25T10:06:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f10f2ffd640 1 -- 172.21.15.133:0/1328314476 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec00ec20 con 0x7f10e8004810 2024-04-25T10:06:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f10f2ffd640 1 -- 172.21.15.133:0/1328314476 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec004020 con 0x7f10e8004810 2024-04-25T10:06:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f110476a640 1 -- 172.21.15.133:0/1328314476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10fc15ff50 con 0x7f10e8004810 2024-04-25T10:06:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f110476a640 1 -- 172.21.15.133:0/1328314476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10fc15df80 con 0x7f10e8004810 2024-04-25T10:06:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f110476a640 1 -- 172.21.15.133:0/1328314476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e40052a0 con 0x7f10e8004810 2024-04-25T10:06:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.862+0000 7f10f2ffd640 1 -- 172.21.15.133:0/1328314476 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10ec0041c0 con 0x7f10e8004810 2024-04-25T10:06:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.866+0000 7f10f2ffd640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 0x7f10e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.866+0000 7f10f2ffd640 1 -- 172.21.15.133:0/1328314476 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10ec085600 con 0x7f10e8004810 2024-04-25T10:06:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.866+0000 7f1101d05640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 0x7f10e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.866+0000 7f1101d05640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 0x7f10e0069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f10f80af000 tx=0x7f10f8004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:02.870+0000 7f10f2ffd640 1 -- 172.21.15.133:0/1328314476 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10ec018030 con 0x7f10e8004810 2024-04-25T10:06:03.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 -- 172.21.15.133:0/1328314476 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10e4005860 con 0x7f10e8004810 2024-04-25T10:06:03.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f10f2ffd640 1 -- 172.21.15.133:0/1328314476 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f10ec0044b0 con 0x7f10e8004810 2024-04-25T10:06:03.066 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:03.066 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:05:38.566660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:03.066 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2024-04-25T10:06:03.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 -- 172.21.15.133:0/1328314476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 msgr2=0x7f10e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 0x7f10e0069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f10f80af000 tx=0x7f10f8004740 comp rx=0 tx=0).stop 2024-04-25T10:06:03.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 -- 172.21.15.133:0/1328314476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 msgr2=0x7f10fc156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 0x7f10fc156630 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f10ec010950 tx=0x7f10ec010e10 comp rx=0 tx=0).stop 2024-04-25T10:06:03.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 -- 172.21.15.133:0/1328314476 shutdown_connections 2024-04-25T10:06:03.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f10fc0c8f40 0x7f10fc15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f10fc15f770 0x7f10fc151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 0x7f10e0069420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 --2- 172.21.15.133:0/1328314476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e8004810 0x7f10fc156630 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 -- 172.21.15.133:0/1328314476 >> 172.21.15.133:0/1328314476 conn(0x7f10fc0727c0 msgr2=0x7f10fc0d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 -- 172.21.15.133:0/1328314476 shutdown_connections 2024-04-25T10:06:03.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.062+0000 7f110476a640 1 -- 172.21.15.133:0/1328314476 wait complete. 2024-04-25T10:06:03.085 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T10:06:03.209 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:03.209 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:06:03.210 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:03.224 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:03.224 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:06:03.224 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:03.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:03.230 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T10:06:03.295 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:06:03.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:06:03.296 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131348 s, 79.8 MB/s 2024-04-25T10:06:03.297 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T10:06:03.297 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:03.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d9b90640 1 Processor -- start 2024-04-25T10:06:03.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d9b90640 1 -- start start 2024-04-25T10:06:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d9b90640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f21d415dbf0 0x7f21d40afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d9b90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d40d0f80 0x7f21d406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d9b90640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d40b3e40 con 0x7f21d406ac90 2024-04-25T10:06:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d9b90640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d406f280 con 0x7f21d415dbf0 2024-04-25T10:06:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d9b90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40b3880 con 0x7f21d40d0f80 2024-04-25T10:06:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d37fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f21d415dbf0 0x7f21d40afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d37fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f21d415dbf0 0x7f21d40afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50948/0 (socket says 172.21.15.133:50948) 2024-04-25T10:06:03.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d37fe640 1 -- 172.21.15.133:0/1564569240 learned_addr learned my addr 172.21.15.133:0/1564569240 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:03.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d2ffd640 1 --2- 172.21.15.133:0/1564569240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d40d0f80 0x7f21d406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d37fe640 1 -- 172.21.15.133:0/1564569240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d40d0f80 msgr2=0x7f21d406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d37fe640 1 --2- 172.21.15.133:0/1564569240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d40d0f80 0x7f21d406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d37fe640 1 -- 172.21.15.133:0/1564569240 >> v1:172.21.15.162:6789/0 conn(0x7f21d406ac90 legacy=0x7f21d41616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.474+0000 7f21d37fe640 1 -- 172.21.15.133:0/1564569240 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d40b3b60 con 0x7f21d415dbf0 2024-04-25T10:06:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d37fe640 1 --2- 172.21.15.133:0/1564569240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d415dbf0 0x7f21d40afc00 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009e50 tx=0x7f21c401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a701af40233034d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d27fc640 1 -- 172.21.15.133:0/1564569240 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40a5660 con 0x7f21d415dbf0 2024-04-25T10:06:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d27fc640 1 -- 172.21.15.133:0/1564569240 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c40a5800 con 0x7f21d415dbf0 2024-04-25T10:06:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d27fc640 1 -- 172.21.15.133:0/1564569240 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40a5bd0 con 0x7f21d415dbf0 2024-04-25T10:06:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d9b90640 1 -- 172.21.15.133:0/1564569240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d415dbf0 msgr2=0x7f21d40afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d9b90640 1 --2- 172.21.15.133:0/1564569240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d415dbf0 0x7f21d40afc00 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009e50 tx=0x7f21c401c9a0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d9b90640 1 -- 172.21.15.133:0/1564569240 shutdown_connections 2024-04-25T10:06:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d9b90640 1 --2- 172.21.15.133:0/1564569240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d40d0f80 0x7f21d406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d9b90640 1 --2- 172.21.15.133:0/1564569240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d415dbf0 0x7f21d40afc00 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d9b90640 1 -- 172.21.15.133:0/1564569240 >> 172.21.15.133:0/1564569240 conn(0x7f21d4072430 msgr2=0x7f21d4072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d9b90640 1 -- 172.21.15.133:0/1564569240 shutdown_connections 2024-04-25T10:06:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d9b90640 1 -- 172.21.15.133:0/1564569240 wait complete. 2024-04-25T10:06:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d9b90640 1 Processor -- start 2024-04-25T10:06:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.478+0000 7f21d9b90640 1 -- start start 2024-04-25T10:06:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d9b90640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d406ac90 0x7f21d415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d9b90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d40d0f80 0x7f21d4160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d9b90640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21d415dbf0 0x7f21d40cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d9b90640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d416b3d0 con 0x7f21d406ac90 2024-04-25T10:06:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d9b90640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d416b250 con 0x7f21d415dbf0 2024-04-25T10:06:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d9b90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d416b550 con 0x7f21d40d0f80 2024-04-25T10:06:03.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d406ac90 0x7f21d415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d406ac90 0x7f21d415ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50950/0 (socket says 172.21.15.133:50950) 2024-04-25T10:06:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d37fe640 1 -- 172.21.15.133:0/2524211604 learned_addr learned my addr 172.21.15.133:0/2524211604 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d3fff640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21d415dbf0 0x7f21d40cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d2ffd640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d40d0f80 0x7f21d4160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d37fe640 1 -- 172.21.15.133:0/2524211604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d40d0f80 msgr2=0x7f21d4160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d37fe640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d40d0f80 0x7f21d4160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d37fe640 1 -- 172.21.15.133:0/2524211604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21d415dbf0 msgr2=0x7f21d40cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d37fe640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21d415dbf0 0x7f21d40cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d37fe640 1 -- 172.21.15.133:0/2524211604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d415ba80 con 0x7f21d406ac90 2024-04-25T10:06:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d3fff640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21d415dbf0 0x7f21d40cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d37fe640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d406ac90 0x7f21d415ee20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f21c401a5d0 tx=0x7f21c4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21bbfff640 1 -- 172.21.15.133:0/2524211604 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40bc020 con 0x7f21d406ac90 2024-04-25T10:06:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21bbfff640 1 -- 172.21.15.133:0/2524211604 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c40b8670 con 0x7f21d406ac90 2024-04-25T10:06:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21bbfff640 1 -- 172.21.15.133:0/2524211604 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4009510 con 0x7f21d406ac90 2024-04-25T10:06:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d9b90640 1 -- 172.21.15.133:0/2524211604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d415bd60 con 0x7f21d406ac90 2024-04-25T10:06:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d9b90640 1 -- 172.21.15.133:0/2524211604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d4160c70 con 0x7f21d406ac90 2024-04-25T10:06:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d9b90640 1 -- 172.21.15.133:0/2524211604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d4067c70 con 0x7f21d406ac90 2024-04-25T10:06:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21bbfff640 1 -- 172.21.15.133:0/2524211604 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21c40bf020 con 0x7f21d406ac90 2024-04-25T10:06:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21bbfff640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c0066f60 0x7f21c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21bbfff640 1 -- 172.21.15.133:0/2524211604 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f21c41200c0 con 0x7f21d406ac90 2024-04-25T10:06:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.482+0000 7f21d2ffd640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c0066f60 0x7f21c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.486+0000 7f21d2ffd640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c0066f60 0x7f21c0069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f21d4168980 tx=0x7f21c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.486+0000 7f21bbfff640 1 -- 172.21.15.133:0/2524211604 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21c40c6030 con 0x7f21d406ac90 2024-04-25T10:06:03.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.690+0000 7f21d9b90640 1 -- 172.21.15.133:0/2524211604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21d406f060 con 0x7f21d406ac90 2024-04-25T10:06:03.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.690+0000 7f21bbfff640 1 -- 172.21.15.133:0/2524211604 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f21c4016620 con 0x7f21d406ac90 2024-04-25T10:06:03.695 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:03.695 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:05:38.566660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:03.695 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2024-04-25T10:06:03.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 -- 172.21.15.133:0/2524211604 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c0066f60 msgr2=0x7f21c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c0066f60 0x7f21c0069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f21d4168980 tx=0x7f21c8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:03.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 -- 172.21.15.133:0/2524211604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d406ac90 msgr2=0x7f21d415ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d406ac90 0x7f21d415ee20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f21c401a5d0 tx=0x7f21c4019740 comp rx=0 tx=0).stop 2024-04-25T10:06:03.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 -- 172.21.15.133:0/2524211604 shutdown_connections 2024-04-25T10:06:03.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21d415dbf0 0x7f21d40cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21d40d0f80 0x7f21d4160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f21c0066f60 0x7f21c0069420 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 --2- 172.21.15.133:0/2524211604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21d406ac90 0x7f21d415ee20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 -- 172.21.15.133:0/2524211604 >> 172.21.15.133:0/2524211604 conn(0x7f21d4072430 msgr2=0x7f21d4156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 -- 172.21.15.133:0/2524211604 shutdown_connections 2024-04-25T10:06:03.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:03.694+0000 7f21d9b90640 1 -- 172.21.15.133:0/2524211604 wait complete. 2024-04-25T10:06:03.714 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T10:06:03.947 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:03.947 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:06:03.947 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:03.965 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:03.966 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-04-25T10:06:03.966 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:06:03.966 DEBUG:teuthology.orchestra.run.smithi133:> import stat 2024-04-25T10:06:03.966 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:06:03.966 DEBUG:teuthology.orchestra.run.smithi133:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T10:06:03.966 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:06:04.032 INFO:teuthology.orchestra.run.smithi133.stdout:1099511630409 2024-04-25T10:06:04.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:04.042 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:04.042 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:06:04.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.206+0000 7faea773d640 1 Processor -- start 2024-04-25T10:06:04.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.206+0000 7faea773d640 1 -- start start 2024-04-25T10:06:04.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea773d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faea015d9a0 0x7faea00d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea773d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faea00ce600 0x7faea00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea773d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea015ebd0 con 0x7faea00d1920 2024-04-25T10:06:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea773d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea015eeb0 con 0x7faea015d9a0 2024-04-25T10:06:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea773d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea00d0330 con 0x7faea00ce600 2024-04-25T10:06:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea54d9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faea015d9a0 0x7faea00d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea54d9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faea015d9a0 0x7faea00d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50952/0 (socket says 172.21.15.133:50952) 2024-04-25T10:06:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea54d9640 1 -- 172.21.15.133:0/2656936646 learned_addr learned my addr 172.21.15.133:0/2656936646 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea4cd8640 1 --2- 172.21.15.133:0/2656936646 >> v2:172.21.15.162:3300/0 conn(0x7faea00ce600 0x7faea00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea54d9640 1 -- 172.21.15.133:0/2656936646 >> v2:172.21.15.162:3300/0 conn(0x7faea00ce600 msgr2=0x7faea00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea54d9640 1 --2- 172.21.15.133:0/2656936646 >> v2:172.21.15.162:3300/0 conn(0x7faea00ce600 0x7faea00cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea54d9640 1 -- 172.21.15.133:0/2656936646 >> v1:172.21.15.162:6789/0 conn(0x7faea00d1920 legacy=0x7faea00d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea54d9640 1 -- 172.21.15.133:0/2656936646 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea00d0610 con 0x7faea015d9a0 2024-04-25T10:06:04.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7faea54d9640 1 --2- 172.21.15.133:0/2656936646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea015d9a0 0x7faea00d0b30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faea00ce080 tx=0x7fae9c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff35a5ca397d370 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7fae97fff640 1 -- 172.21.15.133:0/2656936646 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0b0070 con 0x7faea015d9a0 2024-04-25T10:06:04.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7fae97fff640 1 -- 172.21.15.133:0/2656936646 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c0ab600 con 0x7faea015d9a0 2024-04-25T10:06:04.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.210+0000 7fae97fff640 1 -- 172.21.15.133:0/2656936646 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0ab9d0 con 0x7faea015d9a0 2024-04-25T10:06:04.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 -- 172.21.15.133:0/2656936646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea015d9a0 msgr2=0x7faea00d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 --2- 172.21.15.133:0/2656936646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea015d9a0 0x7faea00d0b30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faea00ce080 tx=0x7fae9c01c450 comp rx=0 tx=0).stop 2024-04-25T10:06:04.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 -- 172.21.15.133:0/2656936646 shutdown_connections 2024-04-25T10:06:04.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 --2- 172.21.15.133:0/2656936646 >> v2:172.21.15.162:3300/0 conn(0x7faea00ce600 0x7faea00cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 --2- 172.21.15.133:0/2656936646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea015d9a0 0x7faea00d0b30 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 -- 172.21.15.133:0/2656936646 >> 172.21.15.133:0/2656936646 conn(0x7faea006bf80 msgr2=0x7faea006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 -- 172.21.15.133:0/2656936646 shutdown_connections 2024-04-25T10:06:04.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 -- 172.21.15.133:0/2656936646 wait complete. 2024-04-25T10:06:04.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 Processor -- start 2024-04-25T10:06:04.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 -- start start 2024-04-25T10:06:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faea00ce600 0x7faea00cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faea00d1920 0x7faea00c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea015d9a0 0x7faea00cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00b9820 con 0x7faea015d9a0 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00b96a0 con 0x7faea00ce600 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea015e590 con 0x7faea00d1920 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea5cda640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea015d9a0 0x7faea00cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea5cda640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea015d9a0 0x7faea00cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50958/0 (socket says 172.21.15.133:50958) 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea5cda640 1 -- 172.21.15.133:0/2546496792 learned_addr learned my addr 172.21.15.133:0/2546496792 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea54d9640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faea00ce600 0x7faea00cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea4cd8640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faea00d1920 0x7faea00c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea5cda640 1 -- 172.21.15.133:0/2546496792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faea00d1920 msgr2=0x7faea00c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea5cda640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faea00d1920 0x7faea00c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea5cda640 1 -- 172.21.15.133:0/2546496792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faea00ce600 msgr2=0x7faea00cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea5cda640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faea00ce600 0x7faea00cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea5cda640 1 -- 172.21.15.133:0/2546496792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea01f6870 con 0x7faea015d9a0 2024-04-25T10:06:04.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea5cda640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea015d9a0 0x7faea00cbbe0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c7b0 tx=0x7fae9800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7fae95ffb640 1 -- 172.21.15.133:0/2546496792 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98004b70 con 0x7faea015d9a0 2024-04-25T10:06:04.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7fae95ffb640 1 -- 172.21.15.133:0/2546496792 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae98004d10 con 0x7faea015d9a0 2024-04-25T10:06:04.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7fae95ffb640 1 -- 172.21.15.133:0/2546496792 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9800aa30 con 0x7faea015d9a0 2024-04-25T10:06:04.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 -- 172.21.15.133:0/2546496792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea00c9840 con 0x7faea015d9a0 2024-04-25T10:06:04.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.214+0000 7faea773d640 1 -- 172.21.15.133:0/2546496792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea00cb3e0 con 0x7faea015d9a0 2024-04-25T10:06:04.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7fae95ffb640 1 -- 172.21.15.133:0/2546496792 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae9800abd0 con 0x7faea015d9a0 2024-04-25T10:06:04.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7fae95ffb640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae84066f60 0x7fae84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faea54d9640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae84066f60 0x7fae84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7fae95ffb640 1 -- 172.21.15.133:0/2546496792 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae98085360 con 0x7faea015d9a0 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faea54d9640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae84066f60 0x7fae84069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0ad000 tx=0x7fae9c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faeaa8aa000 1 Processor -- start 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faeaa8aa000 1 -- start start 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963d11180 0x563963d11560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11aa0 0x563963cc70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963cc7600 0x563963cc79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563963b14f20 con 0x563963d11180 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563963a9d000 con 0x563963d11aa0 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563963b10fc0 con 0x563963cc7600 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faea54d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963d11180 0x563963d11560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faea54d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963d11180 0x563963d11560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50966/0 (socket says 172.21.15.133:50966) 2024-04-25T10:06:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faea54d9640 1 -- 172.21.15.133:0/972738734 learned_addr learned my addr 172.21.15.133:0/972738734 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faea4cd8640 1 --2- 172.21.15.133:0/972738734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11aa0 0x563963cc70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.218+0000 7faea5cda640 1 --2- 172.21.15.133:0/972738734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963cc7600 0x563963cc79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 -- 172.21.15.133:0/972738734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963cc7600 msgr2=0x563963cc79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 --2- 172.21.15.133:0/972738734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963cc7600 0x563963cc79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 -- 172.21.15.133:0/972738734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11aa0 msgr2=0x563963cc70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 --2- 172.21.15.133:0/972738734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11aa0 0x563963cc70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 -- 172.21.15.133:0/972738734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563963af0ba0 con 0x563963d11180 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea5cda640 1 --2- 172.21.15.133:0/972738734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963cc7600 0x563963cc79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea4cd8640 1 --2- 172.21.15.133:0/972738734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11aa0 0x563963cc70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 --2- 172.21.15.133:0/972738734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963d11180 0x563963d11560 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fae9c000c10 tx=0x7fae9c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed94c31bcd121f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7fae82ffd640 1 -- 172.21.15.133:0/972738734 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0c7070 con 0x563963d11180 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7fae82ffd640 1 -- 172.21.15.133:0/972738734 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c0c2470 con 0x563963d11180 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7fae82ffd640 1 -- 172.21.15.133:0/972738734 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0b0040 con 0x563963d11180 2024-04-25T10:06:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 -- 172.21.15.133:0/972738734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963d11180 msgr2=0x563963d11560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/972738734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963d11180 0x563963d11560 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fae9c000c10 tx=0x7fae9c004740 comp rx=0 tx=0).stop 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 -- 172.21.15.133:0/972738734 shutdown_connections 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/972738734 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963cc7600 0x563963cc79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/972738734 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11aa0 0x563963cc70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/972738734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963d11180 0x563963d11560 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 -- 172.21.15.133:0/972738734 >> 172.21.15.133:0/972738734 conn(0x563963ccd760 msgr2=0x563963cc59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 -- 172.21.15.133:0/972738734 shutdown_connections 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 -- 172.21.15.133:0/972738734 wait complete. 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 -- start start 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 Processor -- start 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963ccc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11180 0x563963cccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963d11aa0 0x563963cff450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563963bd29c0 con 0x563963cc7600 2024-04-25T10:06:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563963acf5d0 con 0x563963d11180 2024-04-25T10:06:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563963ad6ee0 con 0x563963d11aa0 2024-04-25T10:06:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 10 client.0 _ll_register_callbacks cb 0x563963cb2a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963ccc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963ccc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50972/0 (socket says 172.21.15.133:50972) 2024-04-25T10:06:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 -- 172.21.15.133:0/3731536559 learned_addr learned my addr 172.21.15.133:0/3731536559 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea5cda640 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963d11aa0 0x563963cff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea4cd8640 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11180 0x563963cccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 -- 172.21.15.133:0/3731536559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963d11aa0 msgr2=0x563963cff450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963d11aa0 0x563963cff450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 -- 172.21.15.133:0/3731536559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11180 msgr2=0x563963cccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11180 0x563963cccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 -- 172.21.15.133:0/3731536559 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563963b5b5a0 con 0x563963cc7600 2024-04-25T10:06:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea5cda640 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963d11aa0 0x563963cff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faea54d9640 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963ccc6a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0ae040 tx=0x7fae9c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6640b4102ad9edc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7fae7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7fae7bfff640 1 -- 172.21.15.133:0/3731536559 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0d0060 con 0x563963cc7600 2024-04-25T10:06:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7fae7bfff640 1 -- 172.21.15.133:0/3731536559 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae9c0c2bc0 con 0x563963cc7600 2024-04-25T10:06:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 -- 172.21.15.133:0/3731536559 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae9c0c2470 con 0x563963cc7600 2024-04-25T10:06:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7fae7bfff640 1 -- 172.21.15.133:0/3731536559 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae9c0c7070 con 0x563963cc7600 2024-04-25T10:06:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 10 client.6336 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7faeaa8aa000 1 -- 172.21.15.133:0/3731536559 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x563963aec2b0 con 0x563963cc7600 2024-04-25T10:06:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.222+0000 7fae7bfff640 1 -- 172.21.15.133:0/3731536559 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c0c2470 con 0x563963cc7600 2024-04-25T10:06:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7fae7bfff640 1 -- 172.21.15.133:0/3731536559 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fae9c0b5430 con 0x563963cc7600 2024-04-25T10:06:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 10 client.6336 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 10 client.6336 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 20 client.6336 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563963cfee60 0x563963cffe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 4 client.6336 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 -- 172.21.15.133:0/3731536559 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563963ab0c70 con 0x563963cfee60 2024-04-25T10:06:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faea4cd8640 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563963cfee60 0x563963cffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faea4cd8640 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563963cfee60 0x563963cffe60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17d153fb00a55b1 server_cookie=ea3c54b594cd1f5e in_seq=0 out_seq=0 2024-04-25T10:06:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7fae7bfff640 10 client.6336 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7fae7bfff640 1 -- 172.21.15.133:0/3731536559 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563963ab0c70 con 0x563963cfee60 2024-04-25T10:06:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7fae7bfff640 10 client.6336 handle_command_reply: tid=0 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 client.6336 shutdown 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 -- 172.21.15.133:0/3731536559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 msgr2=0x563963ccc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963ccc6a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0ae040 tx=0x7fae9c01ac90 comp rx=0 tx=0).stop 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 -- 172.21.15.133:0/3731536559 shutdown_connections 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563963cfee60 0x563963cffe60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963d11aa0 0x563963cff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7fae7bfff640 0 client.6336 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11180 0x563963cccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/3731536559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963ccc6a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 -- 172.21.15.133:0/3731536559 >> 172.21.15.133:0/3731536559 conn(0x563963ccd760 msgr2=0x563963cc60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 -- 172.21.15.133:0/3731536559 shutdown_connections 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 1 -- 172.21.15.133:0/3731536559 wait complete. 2024-04-25T10:06:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.226+0000 7faeaa8aa000 20 client.6336 trim_cache size 0 max 16384 2024-04-25T10:06:04.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 Processor -- start 2024-04-25T10:06:04.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 -- start start 2024-04-25T10:06:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963ccd760 0x563963d105f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963cc7600 0x563963d10b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11180 0x563963cc6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea02030d0 con 0x563963ccd760 2024-04-25T10:06:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0202270 con 0x563963d11180 2024-04-25T10:06:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea015bfc0 con 0x563963cc7600 2024-04-25T10:06:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faea54d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963ccd760 0x563963d105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faea54d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963ccd760 0x563963d105f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50974/0 (socket says 172.21.15.133:50974) 2024-04-25T10:06:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faea54d9640 1 -- 172.21.15.133:0/1313531538 learned_addr learned my addr 172.21.15.133:0/1313531538 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faea54d9640 1 -- 172.21.15.133:0/1313531538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963cc7600 msgr2=0x563963d10b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faea54d9640 1 --2- 172.21.15.133:0/1313531538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963cc7600 0x563963d10b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faea54d9640 1 -- 172.21.15.133:0/1313531538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11180 msgr2=0x563963cc6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faea54d9640 1 --2- 172.21.15.133:0/1313531538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11180 0x563963cc6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faea54d9640 1 -- 172.21.15.133:0/1313531538 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563963a9d250 con 0x563963ccd760 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faea54d9640 1 --2- 172.21.15.133:0/1313531538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963ccd760 0x563963d105f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fae9c01c9c0 tx=0x7fae9c00b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b710a6027014c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7fae827fc640 1 -- 172.21.15.133:0/1313531538 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0b0040 con 0x563963ccd760 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7fae827fc640 1 -- 172.21.15.133:0/1313531538 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c006a30 con 0x563963ccd760 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7fae827fc640 1 -- 172.21.15.133:0/1313531538 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0b5430 con 0x563963ccd760 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 -- 172.21.15.133:0/1313531538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963ccd760 msgr2=0x563963d105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/1313531538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963ccd760 0x563963d105f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fae9c01c9c0 tx=0x7fae9c00b050 comp rx=0 tx=0).stop 2024-04-25T10:06:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 -- 172.21.15.133:0/1313531538 shutdown_connections 2024-04-25T10:06:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/1313531538 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963d11180 0x563963cc6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/1313531538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963cc7600 0x563963d10b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/1313531538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963ccd760 0x563963d105f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 -- 172.21.15.133:0/1313531538 >> 172.21.15.133:0/1313531538 conn(0x563963cff5b0 msgr2=0x563963d0fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 -- 172.21.15.133:0/1313531538 shutdown_connections 2024-04-25T10:06:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.238+0000 7faeaa8aa000 1 -- 172.21.15.133:0/1313531538 wait complete. 2024-04-25T10:06:04.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- start start 2024-04-25T10:06:04.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 Processor -- start 2024-04-25T10:06:04.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963d260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963ccd760 0x563963d26630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963d11180 0x563963d26b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563963b19550 con 0x563963cc7600 2024-04-25T10:06:04.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea015c5c0 con 0x563963ccd760 2024-04-25T10:06:04.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563963bd29c0 con 0x563963d11180 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 10 client.0 _ll_register_callbacks cb 0x563963c93600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea54d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963d260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea54d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963d260f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50978/0 (socket says 172.21.15.133:50978) 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea54d9640 1 -- 172.21.15.133:0/918638343 learned_addr learned my addr 172.21.15.133:0/918638343 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea5cda640 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963d11180 0x563963d26b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea54d9640 1 -- 172.21.15.133:0/918638343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963d11180 msgr2=0x563963d26b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea4cd8640 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963ccd760 0x563963d26630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea54d9640 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963d11180 0x563963d26b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea54d9640 1 -- 172.21.15.133:0/918638343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963ccd760 msgr2=0x563963d26630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea54d9640 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963ccd760 0x563963d26630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea54d9640 1 -- 172.21.15.133:0/918638343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563963aa9b90 con 0x563963cc7600 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea54d9640 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963d260f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00b090 tx=0x7fae9c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c2bbde7e6d7ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7fae7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7fae7bfff640 1 -- 172.21.15.133:0/918638343 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0b0070 con 0x563963cc7600 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7fae7bfff640 1 -- 172.21.15.133:0/918638343 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae9c0d3af0 con 0x563963cc7600 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- 172.21.15.133:0/918638343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae9c006a30 con 0x563963cc7600 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7fae7bfff640 1 -- 172.21.15.133:0/918638343 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae9c0d0070 con 0x563963cc7600 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 10 client.6342 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- 172.21.15.133:0/918638343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x563963abc010 con 0x563963cc7600 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7fae7bfff640 1 -- 172.21.15.133:0/918638343 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c006a30 con 0x563963cc7600 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7fae7bfff640 1 -- 172.21.15.133:0/918638343 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fae9c0c2930 con 0x563963cc7600 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 10 client.6342 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:04.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 10 client.6342 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:04.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 20 client.6342 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:04.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563963d11aa0 0x563963d78df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 4 client.6342 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:06:04.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- 172.21.15.133:0/918638343 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563963b22460 con 0x563963d11aa0 2024-04-25T10:06:04.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea4cd8640 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563963d11aa0 0x563963d78df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7fae7bfff640 10 client.6342 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea4cd8640 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563963d11aa0 0x563963d78df0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea966b0a4844d1a7 server_cookie=78bbd49c84194ed3 in_seq=0 out_seq=0 2024-04-25T10:06:04.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7fae7bfff640 1 -- 172.21.15.133:0/918638343 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563963b22460 con 0x563963d11aa0 2024-04-25T10:06:04.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7fae7bfff640 10 client.6342 handle_command_reply: tid=0 2024-04-25T10:06:04.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 client.6342 shutdown 2024-04-25T10:06:04.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- 172.21.15.133:0/918638343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 msgr2=0x563963d260f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963d260f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00b090 tx=0x7fae9c009240 comp rx=0 tx=0).stop 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- 172.21.15.133:0/918638343 shutdown_connections 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563963d11aa0 0x563963d78df0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563963d11180 0x563963d26b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563963ccd760 0x563963d26630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 --2- 172.21.15.133:0/918638343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563963cc7600 0x563963d260f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7fae7bfff640 0 client.6342 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- 172.21.15.133:0/918638343 >> 172.21.15.133:0/918638343 conn(0x563963cff5b0 msgr2=0x563963d10770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- 172.21.15.133:0/918638343 shutdown_connections 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 1 -- 172.21.15.133:0/918638343 wait complete. 2024-04-25T10:06:04.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faeaa8aa000 20 client.6342 trim_cache size 0 max 16384 2024-04-25T10:06:04.255 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea773d640 1 -- 172.21.15.133:0/2546496792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae84066f60 msgr2=0x7fae84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea773d640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae84066f60 0x7fae84069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0ad000 tx=0x7fae9c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea773d640 1 -- 172.21.15.133:0/2546496792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea015d9a0 msgr2=0x7faea00cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea773d640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea015d9a0 0x7faea00cbbe0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c7b0 tx=0x7fae9800cc70 comp rx=0 tx=0).stop 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea773d640 1 -- 172.21.15.133:0/2546496792 shutdown_connections 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea773d640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae84066f60 0x7fae84069420 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea773d640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faea015d9a0 0x7faea00cbbe0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea773d640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faea00d1920 0x7faea00c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea773d640 1 --2- 172.21.15.133:0/2546496792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faea00ce600 0x7faea00cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.242+0000 7faea773d640 1 -- 172.21.15.133:0/2546496792 >> 172.21.15.133:0/2546496792 conn(0x7faea006bf80 msgr2=0x7faea00735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.246+0000 7faea773d640 1 -- 172.21.15.133:0/2546496792 shutdown_connections 2024-04-25T10:06:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.246+0000 7faea773d640 1 -- 172.21.15.133:0/2546496792 wait complete. 2024-04-25T10:06:04.264 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a49.00000000 parent 2024-04-25T10:06:04.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.290+0000 7f8f291e7080 1 Processor -- start 2024-04-25T10:06:04.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.290+0000 7f8f291e7080 1 -- start start 2024-04-25T10:06:04.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c7d1310 0x55da6c8a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da6c818330 0x55da6c7cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da6c8a2110 0x55da6c89f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6c7749f0 con 0x55da6c7d1310 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6c778950 con 0x55da6c818330 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da6c8a36a0 con 0x55da6c8a2110 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f26d05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c7d1310 0x55da6c8a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f26d05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c7d1310 0x55da6c8a24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50988/0 (socket says 172.21.15.133:50988) 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f26d05640 1 -- 172.21.15.133:0/1495854728 learned_addr learned my addr 172.21.15.133:0/1495854728 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f26d05640 1 -- 172.21.15.133:0/1495854728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da6c8a2110 msgr2=0x55da6c89f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f26d05640 1 --2- 172.21.15.133:0/1495854728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da6c8a2110 0x55da6c89f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f26d05640 1 -- 172.21.15.133:0/1495854728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da6c818330 msgr2=0x55da6c7cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f26d05640 1 --2- 172.21.15.133:0/1495854728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da6c818330 0x55da6c7cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f26d05640 1 -- 172.21.15.133:0/1495854728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da6c7f8410 con 0x55da6c7d1310 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f26d05640 1 --2- 172.21.15.133:0/1495854728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c7d1310 0x55da6c8a24f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800a590 tx=0x7f8f18003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58063e6ca3c1b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f25502640 1 -- 172.21.15.133:0/1495854728 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f180aa940 con 0x55da6c7d1310 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f25502640 1 -- 172.21.15.133:0/1495854728 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f180aab00 con 0x55da6c7d1310 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f25502640 1 -- 172.21.15.133:0/1495854728 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f180bc400 con 0x55da6c7d1310 2024-04-25T10:06:04.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- 172.21.15.133:0/1495854728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c7d1310 msgr2=0x55da6c8a24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 --2- 172.21.15.133:0/1495854728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c7d1310 0x55da6c8a24f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800a590 tx=0x7f8f18003030 comp rx=0 tx=0).stop 2024-04-25T10:06:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- 172.21.15.133:0/1495854728 shutdown_connections 2024-04-25T10:06:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 --2- 172.21.15.133:0/1495854728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da6c8a2110 0x55da6c89f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 --2- 172.21.15.133:0/1495854728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da6c818330 0x55da6c7cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 --2- 172.21.15.133:0/1495854728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c7d1310 0x55da6c8a24f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- 172.21.15.133:0/1495854728 >> 172.21.15.133:0/1495854728 conn(0x55da6c7cda00 msgr2=0x55da6c7cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- 172.21.15.133:0/1495854728 shutdown_connections 2024-04-25T10:06:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- 172.21.15.133:0/1495854728 wait complete. 2024-04-25T10:06:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 Processor -- start 2024-04-25T10:06:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- start start 2024-04-25T10:06:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da6c7d1310 0x55da6c896470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da6c818330 0x55da6c8a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c8a2110 0x55da6c8934f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6c7f86b0 con 0x55da6c8a2110 2024-04-25T10:06:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6c8a4a80 con 0x55da6c818330 2024-04-25T10:06:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da6c8a4ce0 con 0x55da6c7d1310 2024-04-25T10:06:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f27506640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c8a2110 0x55da6c8934f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f27506640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c8a2110 0x55da6c8934f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50996/0 (socket says 172.21.15.133:50996) 2024-04-25T10:06:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f27506640 1 -- 172.21.15.133:0/1995822310 learned_addr learned my addr 172.21.15.133:0/1995822310 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f27506640 1 -- 172.21.15.133:0/1995822310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da6c7d1310 msgr2=0x55da6c896470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f26504640 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da6c818330 0x55da6c8a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f26d05640 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da6c7d1310 0x55da6c896470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f27506640 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55da6c7d1310 0x55da6c896470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f27506640 1 -- 172.21.15.133:0/1995822310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da6c818330 msgr2=0x55da6c8a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f27506640 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55da6c818330 0x55da6c8a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f27506640 1 -- 172.21.15.133:0/1995822310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da6c899450 con 0x55da6c8a2110 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f27506640 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c8a2110 0x55da6c8934f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8f20003100 tx=0x7f8f2000b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f0ffff640 1 -- 172.21.15.133:0/1995822310 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20011620 con 0x55da6c8a2110 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f0ffff640 1 -- 172.21.15.133:0/1995822310 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f20016070 con 0x55da6c8a2110 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- 172.21.15.133:0/1995822310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55da6c8996f0 con 0x55da6c8a2110 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f0ffff640 1 -- 172.21.15.133:0/1995822310 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20011a20 con 0x55da6c8a2110 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.294+0000 7f8f291e7080 1 -- 172.21.15.133:0/1995822310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da6c899c30 con 0x55da6c8a2110 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.298+0000 7f8f0ffff640 1 -- 172.21.15.133:0/1995822310 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f20011bc0 con 0x55da6c8a2110 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.298+0000 7f8f0ffff640 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f14065970 0x7f8f14067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.298+0000 7f8f0ffff640 1 -- 172.21.15.133:0/1995822310 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f200839c0 con 0x55da6c8a2110 2024-04-25T10:06:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.298+0000 7f8f26d05640 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f14065970 0x7f8f14067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.298+0000 7f8f291e7080 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x55da6c8a1020 0x55da6c898160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.298+0000 7f8f291e7080 1 -- 172.21.15.133:0/1995822310 --> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] -- osd_op(unknown.0.0:1 8.1f 8:f95eeb57:::10000000a49.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) v9 -- 0x55da6c936140 con 0x55da6c8a1020 2024-04-25T10:06:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.298+0000 7f8f26504640 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x55da6c8a1020 0x55da6c898160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.298+0000 7f8f26d05640 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f14065970 0x7f8f14067e30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8f180b5000 tx=0x7f8f180a8000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.298+0000 7f8f26504640 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x55da6c8a1020 0x55da6c898160 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f26504640 1 -- 172.21.15.133:0/1995822310 <== osd.0 v2:172.21.15.133:6816/136834454 1 ==== osd_op_reply(1 10000000a49.00000000 [rmxattr] v54'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8f1c00c3f0 con 0x55da6c8a1020 2024-04-25T10:06:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f291e7080 1 -- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x55da6c8a1020 msgr2=0x55da6c898160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f291e7080 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x55da6c8a1020 0x55da6c898160 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f291e7080 1 -- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f14065970 msgr2=0x7f8f14067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f291e7080 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f14065970 0x7f8f14067e30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8f180b5000 tx=0x7f8f180a8000 comp rx=0 tx=0).stop 2024-04-25T10:06:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f291e7080 1 -- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c8a2110 msgr2=0x55da6c8934f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f291e7080 1 --2- 172.21.15.133:0/1995822310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55da6c8a2110 0x55da6c8934f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8f20003100 tx=0x7f8f2000b4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f27506640 1 -- 172.21.15.133:0/1995822310 reap_dead start 2024-04-25T10:06:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f291e7080 1 -- 172.21.15.133:0/1995822310 shutdown_connections 2024-04-25T10:06:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f291e7080 1 -- 172.21.15.133:0/1995822310 >> 172.21.15.133:0/1995822310 conn(0x55da6c7cda00 msgr2=0x55da6c7cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f291e7080 1 -- 172.21.15.133:0/1995822310 shutdown_connections 2024-04-25T10:06:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.302+0000 7f8f291e7080 1 -- 172.21.15.133:0/1995822310 wait complete. 2024-04-25T10:06:04.308 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T10:06:04.308 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:04.308 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T10:06:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b0df84640 1 Processor -- start 2024-04-25T10:06:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b0df84640 1 -- start start 2024-04-25T10:06:04.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b0df84640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 0x7f9b0815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b0df84640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9b080b33d0 0x7f9b081655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b0df84640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0815bc20 con 0x7f9b080b4700 2024-04-25T10:06:04.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b0df84640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b08155160 con 0x7f9b080b33d0 2024-04-25T10:06:04.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b0df84640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b080af360 con 0x7f9b080b4320 2024-04-25T10:06:04.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b077fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 0x7f9b0815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b07fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9b080b33d0 0x7f9b081655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b077fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 0x7f9b0815b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:47946/0 (socket says 172.21.15.133:47946) 2024-04-25T10:06:04.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b077fe640 1 -- 172.21.15.133:0/3201397115 learned_addr learned my addr 172.21.15.133:0/3201397115 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b077fe640 1 -- 172.21.15.133:0/3201397115 >> v2:172.21.15.162:3300/0 conn(0x7f9b080b33d0 msgr2=0x7f9b081655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b077fe640 1 --2- 172.21.15.133:0/3201397115 >> v2:172.21.15.162:3300/0 conn(0x7f9b080b33d0 0x7f9b081655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b077fe640 1 -- 172.21.15.133:0/3201397115 >> v1:172.21.15.162:6789/0 conn(0x7f9b080b4700 legacy=0x7f9b080b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.478+0000 7f9b077fe640 1 -- 172.21.15.133:0/3201397115 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0815e580 con 0x7f9b080b4320 2024-04-25T10:06:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b077fe640 1 --2- 172.21.15.133:0/3201397115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 0x7f9b0815b6e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009870 tx=0x7f9af801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a9e80568d7f2286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b067fc640 1 -- 172.21.15.133:0/3201397115 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80a95f0 con 0x7f9b080b4320 2024-04-25T10:06:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b067fc640 1 -- 172.21.15.133:0/3201397115 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af80a9790 con 0x7f9b080b4320 2024-04-25T10:06:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b067fc640 1 -- 172.21.15.133:0/3201397115 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80a9b60 con 0x7f9b080b4320 2024-04-25T10:06:04.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b0df84640 1 -- 172.21.15.133:0/3201397115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 msgr2=0x7f9b0815b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b0df84640 1 --2- 172.21.15.133:0/3201397115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 0x7f9b0815b6e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009870 tx=0x7f9af801c530 comp rx=0 tx=0).stop 2024-04-25T10:06:04.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b0df84640 1 -- 172.21.15.133:0/3201397115 shutdown_connections 2024-04-25T10:06:04.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b0df84640 1 --2- 172.21.15.133:0/3201397115 >> v2:172.21.15.162:3300/0 conn(0x7f9b080b33d0 0x7f9b081655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b0df84640 1 --2- 172.21.15.133:0/3201397115 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 0x7f9b0815b6e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b0df84640 1 -- 172.21.15.133:0/3201397115 >> 172.21.15.133:0/3201397115 conn(0x7f9b080cfd50 msgr2=0x7f9b080d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b0df84640 1 -- 172.21.15.133:0/3201397115 shutdown_connections 2024-04-25T10:06:04.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b0df84640 1 -- 172.21.15.133:0/3201397115 wait complete. 2024-04-25T10:06:04.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b0df84640 1 Processor -- start 2024-04-25T10:06:04.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.482+0000 7f9b0df84640 1 -- start start 2024-04-25T10:06:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b0df84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b080b33d0 0x7f9b08153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b0df84640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 0x7f9b0814cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b0df84640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b080b4700 0x7f9b0806e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b0df84640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0815f100 con 0x7f9b080b33d0 2024-04-25T10:06:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b0df84640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0815e250 con 0x7f9b080b4700 2024-04-25T10:06:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b0df84640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b08066c10 con 0x7f9b080b4320 2024-04-25T10:06:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b080b33d0 0x7f9b08153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b080b33d0 0x7f9b08153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51006/0 (socket says 172.21.15.133:51006) 2024-04-25T10:06:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b077fe640 1 -- 172.21.15.133:0/3340841839 learned_addr learned my addr 172.21.15.133:0/3340841839 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b07fff640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b080b4700 0x7f9b0806e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b06ffd640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 0x7f9b0814cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b077fe640 1 -- 172.21.15.133:0/3340841839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 msgr2=0x7f9b0814cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b077fe640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 0x7f9b0814cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b077fe640 1 -- 172.21.15.133:0/3340841839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b080b4700 msgr2=0x7f9b0806e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b077fe640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b080b4700 0x7f9b0806e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b077fe640 1 -- 172.21.15.133:0/3340841839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b081536e0 con 0x7f9b080b33d0 2024-04-25T10:06:04.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b077fe640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b080b33d0 0x7f9b08153070 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9af801a160 tx=0x7f9af80192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9aeffff640 1 -- 172.21.15.133:0/3340841839 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80bb600 con 0x7f9b080b33d0 2024-04-25T10:06:04.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9aeffff640 1 -- 172.21.15.133:0/3340841839 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af80bb7a0 con 0x7f9b080b33d0 2024-04-25T10:06:04.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9aeffff640 1 -- 172.21.15.133:0/3340841839 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80bbb90 con 0x7f9b080b33d0 2024-04-25T10:06:04.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b0df84640 1 -- 172.21.15.133:0/3340841839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b081539c0 con 0x7f9b080b33d0 2024-04-25T10:06:04.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9b0df84640 1 -- 172.21.15.133:0/3340841839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b08153f80 con 0x7f9b080b33d0 2024-04-25T10:06:04.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.486+0000 7f9aeffff640 1 -- 172.21.15.133:0/3340841839 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9af80c43f0 con 0x7f9b080b33d0 2024-04-25T10:06:04.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9aeffff640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9af4066f60 0x7f9af4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9aeffff640 1 -- 172.21.15.133:0/3340841839 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9af811fb50 con 0x7f9b080b33d0 2024-04-25T10:06:04.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b06ffd640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9af4066f60 0x7f9af4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 Processor -- start 2024-04-25T10:06:04.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b06ffd640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9af4066f60 0x7f9af4069420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9b0815c2e0 tx=0x7f9afc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 -- start start 2024-04-25T10:06:04.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 0x55e4156fabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4156a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4154f9ce0 con 0x55e4156a9920 2024-04-25T10:06:04.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e415482530 con 0x55e4156fa7f0 2024-04-25T10:06:04.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4154f5d80 con 0x55e4156fb110 2024-04-25T10:06:04.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156a9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51016/0 (socket says 172.21.15.133:51016) 2024-04-25T10:06:04.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b07fff640 1 -- 172.21.15.133:0/3706822513 learned_addr learned my addr 172.21.15.133:0/3706822513 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b06ffd640 1 --2- 172.21.15.133:0/3706822513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4156a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b077fe640 1 --2- 172.21.15.133:0/3706822513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 0x55e4156fabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b07fff640 1 -- 172.21.15.133:0/3706822513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 msgr2=0x55e4156a93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b07fff640 1 --2- 172.21.15.133:0/3706822513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4156a93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b07fff640 1 -- 172.21.15.133:0/3706822513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 msgr2=0x55e4156fabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b07fff640 1 --2- 172.21.15.133:0/3706822513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 0x55e4156fabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b07fff640 1 -- 172.21.15.133:0/3706822513 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4154f4cd0 con 0x55e4156a9920 2024-04-25T10:06:04.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b07fff640 1 --2- 172.21.15.133:0/3706822513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156a9d00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9b00007460 tx=0x7f9b00009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee886d2fc352a50a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9aed7fa640 1 -- 172.21.15.133:0/3706822513 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00019020 con 0x55e4156a9920 2024-04-25T10:06:04.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9aed7fa640 1 -- 172.21.15.133:0/3706822513 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b00015600 con 0x55e4156a9920 2024-04-25T10:06:04.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9aed7fa640 1 -- 172.21.15.133:0/3706822513 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b000159b0 con 0x55e4156a9920 2024-04-25T10:06:04.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 -- 172.21.15.133:0/3706822513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 msgr2=0x55e4156a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 --2- 172.21.15.133:0/3706822513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156a9d00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9b00007460 tx=0x7f9b00009750 comp rx=0 tx=0).stop 2024-04-25T10:06:04.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 -- 172.21.15.133:0/3706822513 shutdown_connections 2024-04-25T10:06:04.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 --2- 172.21.15.133:0/3706822513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156a9d00 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 --2- 172.21.15.133:0/3706822513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4156a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 --2- 172.21.15.133:0/3706822513 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 0x55e4156fabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 -- 172.21.15.133:0/3706822513 >> 172.21.15.133:0/3706822513 conn(0x55e4156d6f90 msgr2=0x55e4156d9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 -- 172.21.15.133:0/3706822513 shutdown_connections 2024-04-25T10:06:04.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.490+0000 7f9b110f1000 1 -- 172.21.15.133:0/3706822513 wait complete. 2024-04-25T10:06:04.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 -- start start 2024-04-25T10:06:04.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 Processor -- start 2024-04-25T10:06:04.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156a9920 0x55e4156b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fa7f0 0x55e4156b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156fb110 0x55e4156b8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4154b4bc0 con 0x55e4156fb110 2024-04-25T10:06:04.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4154f8ff0 con 0x55e4156a9920 2024-04-25T10:06:04.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4154c60b0 con 0x55e4156fa7f0 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 10 client.0 _ll_register_callbacks cb 0x55e4156974a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156fb110 0x55e4156b8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156fb110 0x55e4156b8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51022/0 (socket says 172.21.15.133:51022) 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b07fff640 1 -- 172.21.15.133:0/2078345662 learned_addr learned my addr 172.21.15.133:0/2078345662 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b06ffd640 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fa7f0 0x55e4156b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b077fe640 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156a9920 0x55e4156b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b07fff640 1 -- 172.21.15.133:0/2078345662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fa7f0 msgr2=0x55e4156b7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b07fff640 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fa7f0 0x55e4156b7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b07fff640 1 -- 172.21.15.133:0/2078345662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156a9920 msgr2=0x55e4156b79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b07fff640 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156a9920 0x55e4156b79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b07fff640 1 -- 172.21.15.133:0/2078345662 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e415507950 con 0x55e4156fb110 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b06ffd640 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fa7f0 0x55e4156b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b07fff640 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156fb110 0x55e4156b8440 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9b00000f90 tx=0x7f9b00004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970d116159ae512b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9ae67fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:04.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9ae67fc640 1 -- 172.21.15.133:0/2078345662 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00029680 con 0x55e4156fb110 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9ae67fc640 1 -- 172.21.15.133:0/2078345662 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b00029820 con 0x55e4156fb110 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 -- 172.21.15.133:0/2078345662 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b00015600 con 0x55e4156fb110 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9ae67fc640 1 -- 172.21.15.133:0/2078345662 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b00029a20 con 0x55e4156fb110 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 10 client.6363 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 -- 172.21.15.133:0/2078345662 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55e415576790 con 0x55e4156fb110 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9ae67fc640 1 -- 172.21.15.133:0/2078345662 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b00015600 con 0x55e4156fb110 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9ae67fc640 1 -- 172.21.15.133:0/2078345662 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9b00029bc0 con 0x55e4156fb110 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 10 client.6363 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 10 client.6363 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 20 client.6363 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e415709880 0x55e41570bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 4 client.6363 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b110f1000 1 -- 172.21.15.133:0/2078345662 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e415495b80 con 0x55e415709880 2024-04-25T10:06:04.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.494+0000 7f9b077fe640 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e415709880 0x55e41570bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9ae67fc640 10 client.6363 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b077fe640 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e415709880 0x55e41570bc70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0f4c9e84a1a1cd server_cookie=c3214fab7368e112 in_seq=0 out_seq=0 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9ae67fc640 1 -- 172.21.15.133:0/2078345662 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e415495b80 con 0x55e415709880 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9ae67fc640 10 client.6363 handle_command_reply: tid=0 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 1 client.6363 shutdown 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 1 -- 172.21.15.133:0/2078345662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156fb110 msgr2=0x55e4156b8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156fb110 0x55e4156b8440 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9b00000f90 tx=0x7f9b00004ac0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 1 -- 172.21.15.133:0/2078345662 shutdown_connections 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e415709880 0x55e41570bc70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156fb110 0x55e4156b8440 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9ae67fc640 0 client.6363 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fa7f0 0x55e4156b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 1 --2- 172.21.15.133:0/2078345662 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156a9920 0x55e4156b79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 1 -- 172.21.15.133:0/2078345662 >> 172.21.15.133:0/2078345662 conn(0x55e4156d6f90 msgr2=0x55e4156d9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 1 -- 172.21.15.133:0/2078345662 shutdown_connections 2024-04-25T10:06:04.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 1 -- 172.21.15.133:0/2078345662 wait complete. 2024-04-25T10:06:04.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.498+0000 7f9b110f1000 20 client.6363 trim_cache size 0 max 16384 2024-04-25T10:06:04.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 Processor -- start 2024-04-25T10:06:04.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 -- start start 2024-04-25T10:06:04.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156aae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 0x55e4156a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4156aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b081feed0 con 0x55e4156a9920 2024-04-25T10:06:04.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0806b890 con 0x55e4156fa7f0 2024-04-25T10:06:04.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0806ac90 con 0x55e4156fb110 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156aae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51038/0 (socket says 172.21.15.133:51038) 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b077fe640 1 -- 172.21.15.133:0/285624547 learned_addr learned my addr 172.21.15.133:0/285624547 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b077fe640 1 -- 172.21.15.133:0/285624547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 msgr2=0x55e4156aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b077fe640 1 --2- 172.21.15.133:0/285624547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4156aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b077fe640 1 -- 172.21.15.133:0/285624547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 msgr2=0x55e4156a9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b077fe640 1 --2- 172.21.15.133:0/285624547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 0x55e4156a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b077fe640 1 -- 172.21.15.133:0/285624547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4154bf6f0 con 0x55e4156a9920 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b077fe640 1 --2- 172.21.15.133:0/285624547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156aae60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9af80ec770 tx=0x7f9af80f3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f982370bc1d2df8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9aecff9640 1 -- 172.21.15.133:0/285624547 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80f8a70 con 0x55e4156a9920 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9aecff9640 1 -- 172.21.15.133:0/285624547 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af80ee400 con 0x55e4156a9920 2024-04-25T10:06:04.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9aecff9640 1 -- 172.21.15.133:0/285624547 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80f4120 con 0x55e4156a9920 2024-04-25T10:06:04.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 -- 172.21.15.133:0/285624547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 msgr2=0x55e4156aae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 --2- 172.21.15.133:0/285624547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156aae60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9af80ec770 tx=0x7f9af80f3040 comp rx=0 tx=0).stop 2024-04-25T10:06:04.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 -- 172.21.15.133:0/285624547 shutdown_connections 2024-04-25T10:06:04.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 --2- 172.21.15.133:0/285624547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4156aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 --2- 172.21.15.133:0/285624547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 0x55e4156a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 --2- 172.21.15.133:0/285624547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e4156aae60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 -- 172.21.15.133:0/285624547 >> 172.21.15.133:0/285624547 conn(0x55e4156d6f90 msgr2=0x55e4156d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 -- 172.21.15.133:0/285624547 shutdown_connections 2024-04-25T10:06:04.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.514+0000 7f9b110f1000 1 -- 172.21.15.133:0/285624547 wait complete. 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 -- start start 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 Processor -- start 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e41575a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 0x55e41575a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4157156f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4154fe310 con 0x55e4156a9920 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0806b290 con 0x55e4156fa7f0 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4154f8ff0 con 0x55e4156fb110 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 10 client.0 _ll_register_callbacks cb 0x7f9b00009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e41575a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e41575a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51054/0 (socket says 172.21.15.133:51054) 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b077fe640 1 -- 172.21.15.133:0/754915169 learned_addr learned my addr 172.21.15.133:0/754915169 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b07fff640 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4157156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b077fe640 1 -- 172.21.15.133:0/754915169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 msgr2=0x55e4157156f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b077fe640 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4157156f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b06ffd640 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 0x55e41575a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b077fe640 1 -- 172.21.15.133:0/754915169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 msgr2=0x55e41575a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b077fe640 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 0x55e41575a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b077fe640 1 -- 172.21.15.133:0/754915169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4154b5c80 con 0x55e4156a9920 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b07fff640 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4157156f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b077fe640 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e41575a280 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9af80f02f0 tx=0x7f9b00015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302bb44d49e67b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9ae67fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9ae67fc640 1 -- 172.21.15.133:0/754915169 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80f8a70 con 0x55e4156a9920 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9ae67fc640 1 -- 172.21.15.133:0/754915169 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9af80f4170 con 0x55e4156a9920 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 -- 172.21.15.133:0/754915169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9af80ee400 con 0x55e4156a9920 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9ae67fc640 1 -- 172.21.15.133:0/754915169 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9af80fd050 con 0x55e4156a9920 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 10 client.6369 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 -- 172.21.15.133:0/754915169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55e4151b6600 con 0x55e4156a9920 2024-04-25T10:06:04.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9ae67fc640 1 -- 172.21.15.133:0/754915169 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af80ee400 con 0x55e4156a9920 2024-04-25T10:06:04.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9ae67fc640 1 -- 172.21.15.133:0/754915169 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9af80ec9d0 con 0x55e4156a9920 2024-04-25T10:06:04.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 10 client.6369 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:04.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 10 client.6369 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:04.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 20 client.6369 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:04.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e415716110 0x55e415716520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 4 client.6369 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T10:06:04.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b110f1000 1 -- 172.21.15.133:0/754915169 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x55e415507220 con 0x55e415716110 2024-04-25T10:06:04.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b06ffd640 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e415716110 0x55e415716520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9ae67fc640 10 client.6369 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.518+0000 7f9b06ffd640 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e415716110 0x55e415716520 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b0edbd6ecd5b3d server_cookie=8f86c431f35cab38 in_seq=0 out_seq=0 2024-04-25T10:06:04.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9ae67fc640 1 -- 172.21.15.133:0/754915169 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e415507220 con 0x55e415716110 2024-04-25T10:06:04.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9ae67fc640 10 client.6369 handle_command_reply: tid=0 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 1 client.6369 shutdown 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 1 -- 172.21.15.133:0/754915169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 msgr2=0x55e41575a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e41575a280 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9af80f02f0 tx=0x7f9b00015000 comp rx=0 tx=0).stop 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 1 -- 172.21.15.133:0/754915169 shutdown_connections 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e415716110 0x55e415716520 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e4156fb110 0x55e4157156f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e4156fa7f0 0x55e41575a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 1 --2- 172.21.15.133:0/754915169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4156a9920 0x55e41575a280 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 1 -- 172.21.15.133:0/754915169 >> 172.21.15.133:0/754915169 conn(0x55e4156d6f90 msgr2=0x55e4156a88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9ae67fc640 0 client.6369 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 1 -- 172.21.15.133:0/754915169 shutdown_connections 2024-04-25T10:06:04.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 1 -- 172.21.15.133:0/754915169 wait complete. 2024-04-25T10:06:04.530 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f00d6c35640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a49 [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v549 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6207=0-4194304@1} caps={6207=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6207 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55607c8b5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a49:[<0x10000000001/new_file_0 v549>,<0x10000000000/ceph-qa-suite v69>,<0x1/scrub_test_path v32>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T10:06:04.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b110f1000 20 client.6369 trim_cache size 0 max 16384 2024-04-25T10:06:04.530 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:04.530 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:06:04.530 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "25d50f4a-fd77-4d52-bbb3-682cd0a4eb30", 2024-04-25T10:06:04.530 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:06:04.531 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:04.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 -- 172.21.15.133:0/3340841839 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9af4066f60 msgr2=0x7f9af4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9af4066f60 0x7f9af4069420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9b0815c2e0 tx=0x7f9afc009040 comp rx=0 tx=0).stop 2024-04-25T10:06:04.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 -- 172.21.15.133:0/3340841839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b080b33d0 msgr2=0x7f9b08153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b080b33d0 0x7f9b08153070 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9af801a160 tx=0x7f9af80192d0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 -- 172.21.15.133:0/3340841839 shutdown_connections 2024-04-25T10:06:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b080b4700 0x7f9b0806e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b080b4320 0x7f9b0814cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9af4066f60 0x7f9af4069420 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 --2- 172.21.15.133:0/3340841839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b080b33d0 0x7f9b08153070 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 -- 172.21.15.133:0/3340841839 >> 172.21.15.133:0/3340841839 conn(0x7f9b080cfd50 msgr2=0x7f9b080af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 -- 172.21.15.133:0/3340841839 shutdown_connections 2024-04-25T10:06:04.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.522+0000 7f9b0df84640 1 -- 172.21.15.133:0/3340841839 wait complete. 2024-04-25T10:06:04.543 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '25d50f4a-fd77-4d52-bbb3-682cd0a4eb30', 'mode': 'asynchronous'}' 2024-04-25T10:06:04.543 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:06:04.543 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:06:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd80dc57640 1 Processor -- start 2024-04-25T10:06:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd80dc57640 1 -- start start 2024-04-25T10:06:04.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd80dc57640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd80815d9a0 0x7fd8080d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd80dc57640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8080ce600 0x7fd8080cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd80dc57640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8080d1920 0x7fd8080d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd80dc57640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080cefa0 con 0x7fd8080d1920 2024-04-25T10:06:04.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd80dc57640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd80815ebd0 con 0x7fd8080ce600 2024-04-25T10:06:04.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd80dc57640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80815eeb0 con 0x7fd80815d9a0 2024-04-25T10:06:04.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd807fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8080d1920 0x7fd8080d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd807fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8080d1920 0x7fd8080d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51068/0 (socket says 172.21.15.133:51068) 2024-04-25T10:06:04.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd807fff640 1 -- 172.21.15.133:0/684746684 learned_addr learned my addr 172.21.15.133:0/684746684 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd806ffd640 1 --2- 172.21.15.133:0/684746684 >> v2:172.21.15.162:3300/0 conn(0x7fd8080ce600 0x7fd8080cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd8077fe640 1 --2- 172.21.15.133:0/684746684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd80815d9a0 0x7fd8080d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd807fff640 1 -- 172.21.15.133:0/684746684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd80815d9a0 msgr2=0x7fd8080d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd807fff640 1 --2- 172.21.15.133:0/684746684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd80815d9a0 0x7fd8080d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd807fff640 1 -- 172.21.15.133:0/684746684 >> v2:172.21.15.162:3300/0 conn(0x7fd8080ce600 msgr2=0x7fd8080cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd807fff640 1 --2- 172.21.15.133:0/684746684 >> v2:172.21.15.162:3300/0 conn(0x7fd8080ce600 0x7fd8080cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd807fff640 1 -- 172.21.15.133:0/684746684 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8080d0330 con 0x7fd8080d1920 2024-04-25T10:06:04.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.654+0000 7fd807fff640 1 --2- 172.21.15.133:0/684746684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080d1920 0x7fd8080d1db0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd8080b6420 tx=0x7fd80001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d34920eb82c77f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd8067fc640 1 -- 172.21.15.133:0/684746684 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8000a5790 con 0x7fd8080d1920 2024-04-25T10:06:04.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd8067fc640 1 -- 172.21.15.133:0/684746684 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8000a5930 con 0x7fd8080d1920 2024-04-25T10:06:04.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd8067fc640 1 -- 172.21.15.133:0/684746684 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8000a5d00 con 0x7fd8080d1920 2024-04-25T10:06:04.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 -- 172.21.15.133:0/684746684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080d1920 msgr2=0x7fd8080d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 --2- 172.21.15.133:0/684746684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080d1920 0x7fd8080d1db0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd8080b6420 tx=0x7fd80001c930 comp rx=0 tx=0).stop 2024-04-25T10:06:04.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 -- 172.21.15.133:0/684746684 shutdown_connections 2024-04-25T10:06:04.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 --2- 172.21.15.133:0/684746684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080d1920 0x7fd8080d1db0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 --2- 172.21.15.133:0/684746684 >> v2:172.21.15.162:3300/0 conn(0x7fd8080ce600 0x7fd8080cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 --2- 172.21.15.133:0/684746684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd80815d9a0 0x7fd8080d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 -- 172.21.15.133:0/684746684 >> 172.21.15.133:0/684746684 conn(0x7fd80806bf80 msgr2=0x7fd80806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 -- 172.21.15.133:0/684746684 shutdown_connections 2024-04-25T10:06:04.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 -- 172.21.15.133:0/684746684 wait complete. 2024-04-25T10:06:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 Processor -- start 2024-04-25T10:06:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 -- start start 2024-04-25T10:06:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8080ce600 0x7fd8080cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080d1920 0x7fd8081594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd80815d9a0 0x7fd808158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080b9820 con 0x7fd80815d9a0 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080b96a0 con 0x7fd8080d1920 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080d22f0 con 0x7fd8080ce600 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd807fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd80815d9a0 0x7fd808158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd807fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd80815d9a0 0x7fd808158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51076/0 (socket says 172.21.15.133:51076) 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd807fff640 1 -- 172.21.15.133:0/2689011749 learned_addr learned my addr 172.21.15.133:0/2689011749 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd8077fe640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8080ce600 0x7fd8080cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd806ffd640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080d1920 0x7fd8081594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd807fff640 1 -- 172.21.15.133:0/2689011749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8080ce600 msgr2=0x7fd8080cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd807fff640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8080ce600 0x7fd8080cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd807fff640 1 -- 172.21.15.133:0/2689011749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080d1920 msgr2=0x7fd8081594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd807fff640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080d1920 0x7fd8081594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd807fff640 1 -- 172.21.15.133:0/2689011749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80815a7a0 con 0x7fd80815d9a0 2024-04-25T10:06:04.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd806ffd640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080d1920 0x7fd8081594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd807fff640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd80815d9a0 0x7fd808158910 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd8000af040 tx=0x7fd8000196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd7effff640 1 -- 172.21.15.133:0/2689011749 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8000bc020 con 0x7fd80815d9a0 2024-04-25T10:06:04.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd7effff640 1 -- 172.21.15.133:0/2689011749 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8000b8690 con 0x7fd80815d9a0 2024-04-25T10:06:04.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 -- 172.21.15.133:0/2689011749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd808159f00 con 0x7fd80815d9a0 2024-04-25T10:06:04.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd7effff640 1 -- 172.21.15.133:0/2689011749 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8000b8a80 con 0x7fd80815d9a0 2024-04-25T10:06:04.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.658+0000 7fd80dc57640 1 -- 172.21.15.133:0/2689011749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80815a4c0 con 0x7fd80815d9a0 2024-04-25T10:06:04.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd7effff640 1 -- 172.21.15.133:0/2689011749 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8000bf020 con 0x7fd80815d9a0 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd7effff640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 0x7fd7f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd7effff640 1 -- 172.21.15.133:0/2689011749 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd800120560 con 0x7fd80815d9a0 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd8077fe640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 0x7fd7f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd8077fe640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 0x7fd7f4069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800b9d0 tx=0x7fd7f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd810dc4000 1 Processor -- start 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd810dc4000 1 -- start start 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 0x556b64cf1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64ca7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64ca8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd810dc4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b64af4ff0 con 0x556b64ca80a0 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd810dc4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b64a7d150 con 0x556b64cf22f0 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd810dc4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b64af1090 con 0x556b64cf19d0 2024-04-25T10:06:04.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd807fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64ca8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd807fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64ca8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51086/0 (socket says 172.21.15.133:51086) 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd807fff640 1 -- 172.21.15.133:0/962327327 learned_addr learned my addr 172.21.15.133:0/962327327 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd8077fe640 1 --2- 172.21.15.133:0/962327327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 0x556b64cf1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd806ffd640 1 --2- 172.21.15.133:0/962327327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64ca7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd807fff640 1 -- 172.21.15.133:0/962327327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 msgr2=0x556b64cf1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd807fff640 1 --2- 172.21.15.133:0/962327327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 0x556b64cf1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd807fff640 1 -- 172.21.15.133:0/962327327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 msgr2=0x556b64ca7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd807fff640 1 --2- 172.21.15.133:0/962327327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64ca7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd807fff640 1 -- 172.21.15.133:0/962327327 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b64c8b0d0 con 0x556b64ca80a0 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd806ffd640 1 --2- 172.21.15.133:0/962327327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64ca7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd8077fe640 1 --2- 172.21.15.133:0/962327327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 0x556b64cf1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.662+0000 7fd807fff640 1 --2- 172.21.15.133:0/962327327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64ca8480 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd8000ed290 tx=0x7fd8000f0850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271805209791c84b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd7ed7fa640 1 -- 172.21.15.133:0/962327327 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8000f0e30 con 0x556b64ca80a0 2024-04-25T10:06:04.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd7ed7fa640 1 -- 172.21.15.133:0/962327327 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8000f4f80 con 0x556b64ca80a0 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd7ed7fa640 1 -- 172.21.15.133:0/962327327 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8000f9540 con 0x556b64ca80a0 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- 172.21.15.133:0/962327327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 msgr2=0x556b64ca8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 --2- 172.21.15.133:0/962327327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64ca8480 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd8000ed290 tx=0x7fd8000f0850 comp rx=0 tx=0).stop 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- 172.21.15.133:0/962327327 shutdown_connections 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 --2- 172.21.15.133:0/962327327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64ca8480 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 --2- 172.21.15.133:0/962327327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64ca7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 --2- 172.21.15.133:0/962327327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 0x556b64cf1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- 172.21.15.133:0/962327327 >> 172.21.15.133:0/962327327 conn(0x556b64ca4020 msgr2=0x556b64ca6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- 172.21.15.133:0/962327327 shutdown_connections 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- 172.21.15.133:0/962327327 wait complete. 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- start start 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 Processor -- start 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64ca80a0 0x556b64cd67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cd6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64cd7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b64ab9250 con 0x556b64cf19d0 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b64aaf690 con 0x556b64cf22f0 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b64a7da40 con 0x556b64ca80a0 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 10 client.0 _ll_register_callbacks cb 0x556b64c92e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd806ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cd6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd806ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cd6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51096/0 (socket says 172.21.15.133:51096) 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd806ffd640 1 -- 172.21.15.133:0/3143292120 learned_addr learned my addr 172.21.15.133:0/3143292120 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd807fff640 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64cd7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd8077fe640 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64ca80a0 0x556b64cd67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd806ffd640 1 -- 172.21.15.133:0/3143292120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64ca80a0 msgr2=0x556b64cd67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd806ffd640 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64ca80a0 0x556b64cd67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd806ffd640 1 -- 172.21.15.133:0/3143292120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 msgr2=0x556b64cd7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd806ffd640 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64cd7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd806ffd640 1 -- 172.21.15.133:0/3143292120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b64ba21c0 con 0x556b64cf19d0 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd8077fe640 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64ca80a0 0x556b64cd67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd806ffd640 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cd6d30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd8080ce3f0 tx=0x7fd7fc00f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772bf4bcf0f3381e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd7e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd7e67fc640 1 -- 172.21.15.133:0/3143292120 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc018070 con 0x556b64cf19d0 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- 172.21.15.133:0/3143292120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8000f4f80 con 0x556b64cf19d0 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd7e67fc640 1 -- 172.21.15.133:0/3143292120 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7fc014680 con 0x556b64cf19d0 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd7e67fc640 1 -- 172.21.15.133:0/3143292120 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7fc023050 con 0x556b64cf19d0 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 10 client.6381 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- 172.21.15.133:0/3143292120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x556b64b9e170 con 0x556b64cf19d0 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd7e67fc640 1 -- 172.21.15.133:0/3143292120 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8000f4f80 con 0x556b64cf19d0 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd7e67fc640 1 -- 172.21.15.133:0/3143292120 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd7fc026020 con 0x556b64cf19d0 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 10 client.6381 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 10 client.6381 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:04.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 20 client.6381 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556b64d01c80 0x556b64d04070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 4 client.6381 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd810dc4000 1 -- 172.21.15.133:0/3143292120 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b64a90c60 con 0x556b64d01c80 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd8077fe640 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556b64d01c80 0x556b64d04070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd7e67fc640 10 client.6381 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.666+0000 7fd8077fe640 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556b64d01c80 0x556b64d04070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be67252fd1db205f server_cookie=73c1cafc20e55fd3 in_seq=0 out_seq=0 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd7e67fc640 1 -- 172.21.15.133:0/3143292120 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b64a90c60 con 0x556b64d01c80 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd7e67fc640 10 client.6381 handle_command_reply: tid=0 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 1 client.6381 shutdown 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 1 -- 172.21.15.133:0/3143292120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 msgr2=0x556b64cd6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cd6d30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd8080ce3f0 tx=0x7fd7fc00f5b0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 1 -- 172.21.15.133:0/3143292120 shutdown_connections 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556b64d01c80 0x556b64d04070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64cd7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cd6d30 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd7e67fc640 0 client.6381 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 1 --2- 172.21.15.133:0/3143292120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64ca80a0 0x556b64cd67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 1 -- 172.21.15.133:0/3143292120 >> 172.21.15.133:0/3143292120 conn(0x556b64ca4020 msgr2=0x556b64ca6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 1 -- 172.21.15.133:0/3143292120 shutdown_connections 2024-04-25T10:06:04.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 1 -- 172.21.15.133:0/3143292120 wait complete. 2024-04-25T10:06:04.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.670+0000 7fd810dc4000 20 client.6381 trim_cache size 0 max 16384 2024-04-25T10:06:04.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.682+0000 7fd810dc4000 1 Processor -- start 2024-04-25T10:06:04.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.682+0000 7fd810dc4000 1 -- start start 2024-04-25T10:06:04.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64ca80a0 0x556b64ca7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cfee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64cff380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd808202f20 con 0x556b64cf19d0 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080b4cb0 con 0x556b64cf22f0 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8081fad40 con 0x556b64ca80a0 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd806ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cfee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd806ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cfee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51102/0 (socket says 172.21.15.133:51102) 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd806ffd640 1 -- 172.21.15.133:0/2283393086 learned_addr learned my addr 172.21.15.133:0/2283393086 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd806ffd640 1 -- 172.21.15.133:0/2283393086 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64ca80a0 msgr2=0x556b64ca7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd806ffd640 1 --2- 172.21.15.133:0/2283393086 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64ca80a0 0x556b64ca7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd806ffd640 1 -- 172.21.15.133:0/2283393086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 msgr2=0x556b64cff380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd806ffd640 1 --2- 172.21.15.133:0/2283393086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64cff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd806ffd640 1 -- 172.21.15.133:0/2283393086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b64b3bde0 con 0x556b64cf19d0 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd806ffd640 1 --2- 172.21.15.133:0/2283393086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cfee40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc000c10 tx=0x7fd7fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15747e02d05cdf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd7ecff9640 1 -- 172.21.15.133:0/2283393086 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc018040 con 0x556b64cf19d0 2024-04-25T10:06:04.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd7ecff9640 1 -- 172.21.15.133:0/2283393086 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc00fab0 con 0x556b64cf19d0 2024-04-25T10:06:04.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd7ecff9640 1 -- 172.21.15.133:0/2283393086 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc014b30 con 0x556b64cf19d0 2024-04-25T10:06:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- 172.21.15.133:0/2283393086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 msgr2=0x556b64cfee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 --2- 172.21.15.133:0/2283393086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cfee40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc000c10 tx=0x7fd7fc014000 comp rx=0 tx=0).stop 2024-04-25T10:06:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- 172.21.15.133:0/2283393086 shutdown_connections 2024-04-25T10:06:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 --2- 172.21.15.133:0/2283393086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64cff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 --2- 172.21.15.133:0/2283393086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64cf19d0 0x556b64cfee40 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 --2- 172.21.15.133:0/2283393086 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64ca80a0 0x556b64ca7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- 172.21.15.133:0/2283393086 >> 172.21.15.133:0/2283393086 conn(0x556b64ca4020 msgr2=0x556b64cfe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- 172.21.15.133:0/2283393086 shutdown_connections 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- 172.21.15.133:0/2283393086 wait complete. 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- start start 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 Processor -- start 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64d59d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 0x556b64d5a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64d03a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b64a8fe00 con 0x556b64ca80a0 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080b46b0 con 0x556b64cf22f0 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b64af4300 con 0x556b64cf19d0 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd810dc4000 10 client.0 _ll_register_callbacks cb 0x556b64c73040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd8077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64d59d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd8077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64d59d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51108/0 (socket says 172.21.15.133:51108) 2024-04-25T10:06:04.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd8077fe640 1 -- 172.21.15.133:0/2385556665 learned_addr learned my addr 172.21.15.133:0/2385556665 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd807fff640 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64d03a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd8077fe640 1 -- 172.21.15.133:0/2385556665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 msgr2=0x556b64d5a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd806ffd640 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 0x556b64d5a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd8077fe640 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 0x556b64d5a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd8077fe640 1 -- 172.21.15.133:0/2385556665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 msgr2=0x556b64d03a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd8077fe640 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64d03a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd8077fe640 1 -- 172.21.15.133:0/2385556665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b64a968e0 con 0x556b64ca80a0 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd807fff640 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64d03a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.686+0000 7fd806ffd640 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 0x556b64d5a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd8077fe640 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64d59d20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800fb80 tx=0x7fd7f80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c1b09f496401b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd7e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd7e67fc640 1 -- 172.21.15.133:0/2385556665 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8026350 con 0x556b64ca80a0 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd7e67fc640 1 -- 172.21.15.133:0/2385556665 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7f80264f0 con 0x556b64ca80a0 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 -- 172.21.15.133:0/2385556665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7fc00fab0 con 0x556b64ca80a0 2024-04-25T10:06:04.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd7e67fc640 1 -- 172.21.15.133:0/2385556665 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7f8035050 con 0x556b64ca80a0 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 10 client.6387 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 -- 172.21.15.133:0/2385556665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x556b64b4e8f0 con 0x556b64ca80a0 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd7e67fc640 1 -- 172.21.15.133:0/2385556665 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc00fab0 con 0x556b64ca80a0 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd7e67fc640 1 -- 172.21.15.133:0/2385556665 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd7f8038020 con 0x556b64ca80a0 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 10 client.6387 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 10 client.6387 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 20 client.6387 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556b64d04610 0x556b64d04a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 4 client.6387 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 -- 172.21.15.133:0/2385556665 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556b64b02530 con 0x556b64d04610 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd806ffd640 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556b64d04610 0x556b64d04a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd7e67fc640 10 client.6387 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd806ffd640 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556b64d04610 0x556b64d04a20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba2aa87d248fb8ee server_cookie=370d0359927a8023 in_seq=0 out_seq=0 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd7e67fc640 1 -- 172.21.15.133:0/2385556665 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+455 (crc 0 0 0) 0x556b64b02530 con 0x556b64d04610 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd7e67fc640 10 client.6387 handle_command_reply: tid=0 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 client.6387 shutdown 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 -- 172.21.15.133:0/2385556665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 msgr2=0x556b64d59d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64d59d20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800fb80 tx=0x7fd7f80046d0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 -- 172.21.15.133:0/2385556665 shutdown_connections 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556b64d04610 0x556b64d04a20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556b64cf22f0 0x556b64d03a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556b64cf19d0 0x556b64d5a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 --2- 172.21.15.133:0/2385556665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b64ca80a0 0x556b64d59d20 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd7e67fc640 0 client.6387 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 -- 172.21.15.133:0/2385556665 >> 172.21.15.133:0/2385556665 conn(0x556b64ca4020 msgr2=0x556b64cfefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 -- 172.21.15.133:0/2385556665 shutdown_connections 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 1 -- 172.21.15.133:0/2385556665 wait complete. 2024-04-25T10:06:04.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd810dc4000 20 client.6387 trim_cache size 0 max 16384 2024-04-25T10:06:04.700 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:04.700 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:06:04.700 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-04-25T10:06:04.700 INFO:teuthology.orchestra.run.smithi133.stdout: "25d50f4a-fd77-4d52-bbb3-682cd0a4eb30": { 2024-04-25T10:06:04.700 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T10:06:04.700 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "25d50f4a-fd77-4d52-bbb3-682cd0a4eb30", 2024-04-25T10:06:04.700 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "" 2024-04-25T10:06:04.700 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:04.700 INFO:teuthology.orchestra.run.smithi133.stdout: "49cfd23c-4ad6-4086-b030-cb4d274d07e5": { 2024-04-25T10:06:04.700 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-04-25T10:06:04.700 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "49cfd23c-4ad6-4086-b030-cb4d274d07e5", 2024-04-25T10:06:04.701 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "" 2024-04-25T10:06:04.701 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:04.701 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:04.701 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:04.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 -- 172.21.15.133:0/2689011749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 msgr2=0x7fd7f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 0x7fd7f4069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800b9d0 tx=0x7fd7f8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:04.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 -- 172.21.15.133:0/2689011749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd80815d9a0 msgr2=0x7fd808158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd80815d9a0 0x7fd808158910 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd8000af040 tx=0x7fd8000196d0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 -- 172.21.15.133:0/2689011749 shutdown_connections 2024-04-25T10:06:04.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 0x7fd7f4069420 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd80815d9a0 0x7fd808158910 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080d1920 0x7fd8081594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 --2- 172.21.15.133:0/2689011749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8080ce600 0x7fd8080cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 -- 172.21.15.133:0/2689011749 >> 172.21.15.133:0/2689011749 conn(0x7fd80806bf80 msgr2=0x7fd8080b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 -- 172.21.15.133:0/2689011749 shutdown_connections 2024-04-25T10:06:04.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:04.690+0000 7fd80dc57640 1 -- 172.21.15.133:0/2689011749 wait complete. 2024-04-25T10:06:09.712 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:06:09.712 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:06:09.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.842+0000 7fcdee046640 1 Processor -- start 2024-04-25T10:06:09.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.842+0000 7fcdee046640 1 -- start start 2024-04-25T10:06:09.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.842+0000 7fcdee046640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcde815d9a0 0x7fcde80d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.842+0000 7fcdee046640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcde80d2030 0x7fcde80cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.842+0000 7fcdee046640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde815ebd0 con 0x7fcde80ce600 2024-04-25T10:06:09.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.842+0000 7fcdee046640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde815eeb0 con 0x7fcde815d9a0 2024-04-25T10:06:09.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.842+0000 7fcdee046640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde80d0330 con 0x7fcde80d2030 2024-04-25T10:06:09.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.842+0000 7fcde77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcde815d9a0 0x7fcde80d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcde815d9a0 0x7fcde80d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40080/0 (socket says 172.21.15.133:40080) 2024-04-25T10:06:09.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde77fe640 1 -- 172.21.15.133:0/3798201859 learned_addr learned my addr 172.21.15.133:0/3798201859 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:09.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde7fff640 1 --2- 172.21.15.133:0/3798201859 >> v2:172.21.15.162:3300/0 conn(0x7fcde80d2030 0x7fcde80cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde77fe640 1 -- 172.21.15.133:0/3798201859 >> v2:172.21.15.162:3300/0 conn(0x7fcde80d2030 msgr2=0x7fcde80cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde77fe640 1 --2- 172.21.15.133:0/3798201859 >> v2:172.21.15.162:3300/0 conn(0x7fcde80d2030 0x7fcde80cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde77fe640 1 -- 172.21.15.133:0/3798201859 >> v1:172.21.15.133:6789/0 conn(0x7fcde80ce600 legacy=0x7fcde80d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde67fc640 1 -- 172.21.15.133:0/3798201859 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058140720 0 0) 0x7fcde815ebd0 con 0x7fcde80ce600 2024-04-25T10:06:09.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde77fe640 1 -- 172.21.15.133:0/3798201859 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80d0610 con 0x7fcde815d9a0 2024-04-25T10:06:09.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde77fe640 1 --2- 172.21.15.133:0/3798201859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde815d9a0 0x7fcde80d0b30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcde808ccc0 tx=0x7fcdd801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef34673175f7f04f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde67fc640 1 -- 172.21.15.133:0/3798201859 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80b2070 con 0x7fcde815d9a0 2024-04-25T10:06:09.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde67fc640 1 -- 172.21.15.133:0/3798201859 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd80a9070 con 0x7fcde815d9a0 2024-04-25T10:06:09.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcde67fc640 1 -- 172.21.15.133:0/3798201859 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80ad880 con 0x7fcde815d9a0 2024-04-25T10:06:09.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcdee046640 1 -- 172.21.15.133:0/3798201859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde815d9a0 msgr2=0x7fcde80d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcdee046640 1 --2- 172.21.15.133:0/3798201859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde815d9a0 0x7fcde80d0b30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcde808ccc0 tx=0x7fcdd801c490 comp rx=0 tx=0).stop 2024-04-25T10:06:09.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcdee046640 1 -- 172.21.15.133:0/3798201859 shutdown_connections 2024-04-25T10:06:09.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcdee046640 1 --2- 172.21.15.133:0/3798201859 >> v2:172.21.15.162:3300/0 conn(0x7fcde80d2030 0x7fcde80cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcdee046640 1 --2- 172.21.15.133:0/3798201859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde815d9a0 0x7fcde80d0b30 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcdee046640 1 -- 172.21.15.133:0/3798201859 >> 172.21.15.133:0/3798201859 conn(0x7fcde806bf80 msgr2=0x7fcde806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcdee046640 1 -- 172.21.15.133:0/3798201859 shutdown_connections 2024-04-25T10:06:09.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.846+0000 7fcdee046640 1 -- 172.21.15.133:0/3798201859 wait complete. 2024-04-25T10:06:09.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdee046640 1 Processor -- start 2024-04-25T10:06:09.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdee046640 1 -- start start 2024-04-25T10:06:09.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdee046640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde80ce600 0x7fcde80cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdee046640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcde80d2030 0x7fcde8167ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdee046640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcde815d9a0 0x7fcde80c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdee046640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80b9820 con 0x7fcde80ce600 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdee046640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80b96a0 con 0x7fcde815d9a0 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdee046640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde815e590 con 0x7fcde80d2030 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde80ce600 0x7fcde80cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde80ce600 0x7fcde80cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40082/0 (socket says 172.21.15.133:40082) 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde77fe640 1 -- 172.21.15.133:0/2977757922 learned_addr learned my addr 172.21.15.133:0/2977757922 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:09.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde7fff640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcde815d9a0 0x7fcde80c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde77fe640 1 -- 172.21.15.133:0/2977757922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcde80d2030 msgr2=0x7fcde8167ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde6ffd640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcde80d2030 0x7fcde8167ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde77fe640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcde80d2030 0x7fcde8167ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde77fe640 1 -- 172.21.15.133:0/2977757922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcde815d9a0 msgr2=0x7fcde80c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde77fe640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcde815d9a0 0x7fcde80c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde77fe640 1 -- 172.21.15.133:0/2977757922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80cad60 con 0x7fcde80ce600 2024-04-25T10:06:09.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde7fff640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcde815d9a0 0x7fcde80c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:09.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcde77fe640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde80ce600 0x7fcde80cd220 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80b0040 tx=0x7fcdd801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdcffff640 1 -- 172.21.15.133:0/2977757922 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80bc020 con 0x7fcde80ce600 2024-04-25T10:06:09.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdcffff640 1 -- 172.21.15.133:0/2977757922 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd80b8420 con 0x7fcde80ce600 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdcffff640 1 -- 172.21.15.133:0/2977757922 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80b2040 con 0x7fcde80ce600 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdee046640 1 -- 172.21.15.133:0/2977757922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde80c7400 con 0x7fcde80ce600 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdee046640 1 -- 172.21.15.133:0/2977757922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde80cb040 con 0x7fcde80ce600 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.850+0000 7fcdcffff640 1 -- 172.21.15.133:0/2977757922 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcdd80bf020 con 0x7fcde80ce600 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdcffff640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdd4066f60 0x7fcdd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdcffff640 1 -- 172.21.15.133:0/2977757922 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcdd811f340 con 0x7fcde80ce600 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcde6ffd640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdd4066f60 0x7fcdd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcde6ffd640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdd4066f60 0x7fcdd4069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcddc007fd0 tx=0x7fcddc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 Processor -- start 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 -- start start 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973df1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df22f0 0x563973da7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973da8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973bf4ff0 con 0x563973df19d0 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973b7d150 con 0x563973da80a0 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563973bf1090 con 0x563973df22f0 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcde77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973df1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcde77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973df1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40096/0 (socket says 172.21.15.133:40096) 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcde77fe640 1 -- 172.21.15.133:0/2417223419 learned_addr learned my addr 172.21.15.133:0/2417223419 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcde77fe640 1 -- 172.21.15.133:0/2417223419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df22f0 msgr2=0x563973da7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcde77fe640 1 --2- 172.21.15.133:0/2417223419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df22f0 0x563973da7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcde77fe640 1 -- 172.21.15.133:0/2417223419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 msgr2=0x563973da8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcde77fe640 1 --2- 172.21.15.133:0/2417223419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973da8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcde77fe640 1 -- 172.21.15.133:0/2417223419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563973d8b0d0 con 0x563973df19d0 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcde77fe640 1 --2- 172.21.15.133:0/2417223419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973df1db0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80ebfb0 tx=0x7fcdd8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5cc788eaef2733 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdcd7fa640 1 -- 172.21.15.133:0/2417223419 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80f3ca0 con 0x563973df19d0 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdcd7fa640 1 -- 172.21.15.133:0/2417223419 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd80a9070 con 0x563973df19d0 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdcd7fa640 1 -- 172.21.15.133:0/2417223419 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80f8260 con 0x563973df19d0 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 -- 172.21.15.133:0/2417223419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 msgr2=0x563973df1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/2417223419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973df1db0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80ebfb0 tx=0x7fcdd8009240 comp rx=0 tx=0).stop 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 -- 172.21.15.133:0/2417223419 shutdown_connections 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/2417223419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973da8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/2417223419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df22f0 0x563973da7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/2417223419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973df1db0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 -- 172.21.15.133:0/2417223419 >> 172.21.15.133:0/2417223419 conn(0x563973da4020 msgr2=0x563973da6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 -- 172.21.15.133:0/2417223419 shutdown_connections 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.854+0000 7fcdf11b3000 1 -- 172.21.15.133:0/2417223419 wait complete. 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- start start 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 Processor -- start 2024-04-25T10:06:09.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973dd67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df19d0 0x563973dd6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df22f0 0x563973dd7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973bb9250 con 0x563973df22f0 2024-04-25T10:06:09.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973baf690 con 0x563973da80a0 2024-04-25T10:06:09.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563973b7da40 con 0x563973df19d0 2024-04-25T10:06:09.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 10 client.0 _ll_register_callbacks cb 0x563973d92e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:09.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df22f0 0x563973dd7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df22f0 0x563973dd7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40100/0 (socket says 172.21.15.133:40100) 2024-04-25T10:06:09.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde7fff640 1 -- 172.21.15.133:0/3907699033 learned_addr learned my addr 172.21.15.133:0/3907699033 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde77fe640 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973dd67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde6ffd640 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df19d0 0x563973dd6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde7fff640 1 -- 172.21.15.133:0/3907699033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df19d0 msgr2=0x563973dd6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde7fff640 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df19d0 0x563973dd6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde7fff640 1 -- 172.21.15.133:0/3907699033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 msgr2=0x563973dd67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde7fff640 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973dd67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde7fff640 1 -- 172.21.15.133:0/3907699033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563973ca21c0 con 0x563973df22f0 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde77fe640 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973dd67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde7fff640 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df22f0 0x563973dd7270 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fcde00079f0 tx=0x7fcde0007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e1447ac12beaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdc67fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdc67fc640 1 -- 172.21.15.133:0/3907699033 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0018020 con 0x563973df22f0 2024-04-25T10:06:09.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdc67fc640 1 -- 172.21.15.133:0/3907699033 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcde000fba0 con 0x563973df22f0 2024-04-25T10:06:09.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3907699033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdd80a9070 con 0x563973df22f0 2024-04-25T10:06:09.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdc67fc640 1 -- 172.21.15.133:0/3907699033 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcde0022050 con 0x563973df22f0 2024-04-25T10:06:09.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 10 client.6402 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:09.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3907699033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x563973c9e170 con 0x563973df22f0 2024-04-25T10:06:09.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdc67fc640 1 -- 172.21.15.133:0/3907699033 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd80a9070 con 0x563973df22f0 2024-04-25T10:06:09.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdc67fc640 1 -- 172.21.15.133:0/3907699033 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcde0025020 con 0x563973df22f0 2024-04-25T10:06:09.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 10 client.6402 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:09.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 10 client.6402 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:09.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 20 client.6402 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:09.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563973e01c80 0x563973e04070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 4 client.6402 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:09.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3907699033 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563973b90c60 con 0x563973e01c80 2024-04-25T10:06:09.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde77fe640 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563973e01c80 0x563973e04070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcde77fe640 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563973e01c80 0x563973e04070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7a59faa16be0a2 server_cookie=2792e72b1cc27f52 in_seq=0 out_seq=0 2024-04-25T10:06:09.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdc67fc640 10 client.6402 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:09.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdc67fc640 1 -- 172.21.15.133:0/3907699033 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563973b90c60 con 0x563973e01c80 2024-04-25T10:06:09.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdc67fc640 10 client.6402 handle_command_reply: tid=0 2024-04-25T10:06:09.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 client.6402 shutdown 2024-04-25T10:06:09.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3907699033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df22f0 msgr2=0x563973dd7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df22f0 0x563973dd7270 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fcde00079f0 tx=0x7fcde0007eb0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3907699033 shutdown_connections 2024-04-25T10:06:09.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563973e01c80 0x563973e04070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df22f0 0x563973dd7270 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df19d0 0x563973dd6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdc67fc640 0 client.6402 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:09.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/3907699033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973dd67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3907699033 >> 172.21.15.133:0/3907699033 conn(0x563973da4020 msgr2=0x563973da6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3907699033 shutdown_connections 2024-04-25T10:06:09.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3907699033 wait complete. 2024-04-25T10:06:09.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.858+0000 7fcdf11b3000 20 client.6402 trim_cache size 0 max 16384 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.874+0000 7fcdf11b3000 1 Processor -- start 2024-04-25T10:06:09.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.874+0000 7fcdf11b3000 1 -- start start 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.874+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973e4e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.874+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973e4e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.874+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df22f0 0x563973e4eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.874+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde8203010 con 0x563973df19d0 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.874+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde8202200 con 0x563973da80a0 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.874+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde8156970 con 0x563973df22f0 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.874+0000 7fcde6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973e4e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.874+0000 7fcde6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973e4e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40114/0 (socket says 172.21.15.133:40114) 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde6ffd640 1 -- 172.21.15.133:0/3412728345 learned_addr learned my addr 172.21.15.133:0/3412728345 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde7fff640 1 --2- 172.21.15.133:0/3412728345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df22f0 0x563973e4eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde6ffd640 1 -- 172.21.15.133:0/3412728345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df22f0 msgr2=0x563973e4eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde6ffd640 1 --2- 172.21.15.133:0/3412728345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df22f0 0x563973e4eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde77fe640 1 --2- 172.21.15.133:0/3412728345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973e4e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde6ffd640 1 -- 172.21.15.133:0/3412728345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 msgr2=0x563973e4e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde6ffd640 1 --2- 172.21.15.133:0/3412728345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973e4e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde6ffd640 1 -- 172.21.15.133:0/3412728345 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563973c3bde0 con 0x563973df19d0 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde6ffd640 1 --2- 172.21.15.133:0/3412728345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973e4e630 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fcde808b5e0 tx=0x7fcddc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6022db752061f9ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdccff9640 1 -- 172.21.15.133:0/3412728345 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc026070 con 0x563973df19d0 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdccff9640 1 -- 172.21.15.133:0/3412728345 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc005480 con 0x563973df19d0 2024-04-25T10:06:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdccff9640 1 -- 172.21.15.133:0/3412728345 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00d8a0 con 0x563973df19d0 2024-04-25T10:06:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3412728345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 msgr2=0x563973e4e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/3412728345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973e4e630 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fcde808b5e0 tx=0x7fcddc010000 comp rx=0 tx=0).stop 2024-04-25T10:06:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3412728345 shutdown_connections 2024-04-25T10:06:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/3412728345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df22f0 0x563973e4eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/3412728345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973df19d0 0x563973e4e630 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/3412728345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973da80a0 0x563973e4e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3412728345 >> 172.21.15.133:0/3412728345 conn(0x563973da4020 msgr2=0x563973dfefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3412728345 shutdown_connections 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 -- 172.21.15.133:0/3412728345 wait complete. 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 -- start start 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 Processor -- start 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973da80a0 0x563973e04380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df19d0 0x563973e048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973df22f0 0x563973e04e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563973b8fe00 con 0x563973da80a0 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde8156f70 con 0x563973df22f0 2024-04-25T10:06:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563973bf4300 con 0x563973df19d0 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 10 client.0 _ll_register_callbacks cb 0x563973d73040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973da80a0 0x563973e04380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973da80a0 0x563973e04380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40128/0 (socket says 172.21.15.133:40128) 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde77fe640 1 -- 172.21.15.133:0/379324788 learned_addr learned my addr 172.21.15.133:0/379324788 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde77fe640 1 -- 172.21.15.133:0/379324788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df19d0 msgr2=0x563973e048c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde7fff640 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973df22f0 0x563973e04e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde6ffd640 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df19d0 0x563973e048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde77fe640 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df19d0 0x563973e048c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde77fe640 1 -- 172.21.15.133:0/379324788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973df22f0 msgr2=0x563973e04e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde77fe640 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973df22f0 0x563973e04e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde77fe640 1 -- 172.21.15.133:0/379324788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563973b968e0 con 0x563973da80a0 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcde77fe640 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973da80a0 0x563973e04380 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fcdd811d740 tx=0x7fcdd811d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cab3a6848636f4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdc67fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdc67fc640 1 -- 172.21.15.133:0/379324788 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd811f070 con 0x563973da80a0 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdc67fc640 1 -- 172.21.15.133:0/379324788 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcdd80ec1c0 con 0x563973da80a0 2024-04-25T10:06:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 -- 172.21.15.133:0/379324788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563973c02530 con 0x563973da80a0 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdc67fc640 1 -- 172.21.15.133:0/379324788 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdd8119020 con 0x563973da80a0 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 10 client.6408 fetch_fsmap learned FSMap version 23 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdf11b3000 1 -- 172.21.15.133:0/379324788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x563973c4e8f0 con 0x563973da80a0 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.878+0000 7fcdc67fc640 1 -- 172.21.15.133:0/379324788 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563973c02530 con 0x563973da80a0 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdc67fc640 1 -- 172.21.15.133:0/379324788 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcdd80b5030 con 0x563973da80a0 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 10 client.6408 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 10 client.6408 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 20 client.6408 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563973e057f0 0x563973e59f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 4 client.6408 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 -- 172.21.15.133:0/379324788 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563973bfc880 con 0x563973e057f0 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcde6ffd640 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563973e057f0 0x563973e59f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdc67fc640 10 client.6408 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcde6ffd640 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563973e057f0 0x563973e59f60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a410ea8814ea18cb server_cookie=5d327b21459b2261 in_seq=0 out_seq=0 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdc67fc640 1 -- 172.21.15.133:0/379324788 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563973bfc880 con 0x563973e057f0 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdc67fc640 10 client.6408 handle_command_reply: tid=0 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 client.6408 shutdown 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 -- 172.21.15.133:0/379324788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973da80a0 msgr2=0x563973e04380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973da80a0 0x563973e04380 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fcdd811d740 tx=0x7fcdd811d640 comp rx=0 tx=0).stop 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 -- 172.21.15.133:0/379324788 shutdown_connections 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563973e057f0 0x563973e59f60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563973df22f0 0x563973e04e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563973df19d0 0x563973e048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdc67fc640 0 client.6408 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 --2- 172.21.15.133:0/379324788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563973da80a0 0x563973e04380 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 -- 172.21.15.133:0/379324788 >> 172.21.15.133:0/379324788 conn(0x563973da4020 msgr2=0x563973df0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 -- 172.21.15.133:0/379324788 shutdown_connections 2024-04-25T10:06:09.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 1 -- 172.21.15.133:0/379324788 wait complete. 2024-04-25T10:06:09.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdf11b3000 20 client.6408 trim_cache size 0 max 16384 2024-04-25T10:06:09.891 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:09.891 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-04-25T10:06:09.891 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:06:09.891 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 -- 172.21.15.133:0/2977757922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdd4066f60 msgr2=0x7fcdd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdd4066f60 0x7fcdd4069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcddc007fd0 tx=0x7fcddc009040 comp rx=0 tx=0).stop 2024-04-25T10:06:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 -- 172.21.15.133:0/2977757922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde80ce600 msgr2=0x7fcde80cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde80ce600 0x7fcde80cd220 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80b0040 tx=0x7fcdd801acd0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 -- 172.21.15.133:0/2977757922 shutdown_connections 2024-04-25T10:06:09.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcde815d9a0 0x7fcde80c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcde80d2030 0x7fcde8167ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdd4066f60 0x7fcdd4069420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 --2- 172.21.15.133:0/2977757922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde80ce600 0x7fcde80cd220 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 -- 172.21.15.133:0/2977757922 >> 172.21.15.133:0/2977757922 conn(0x7fcde806bf80 msgr2=0x7fcde80b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 -- 172.21.15.133:0/2977757922 shutdown_connections 2024-04-25T10:06:09.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:09.882+0000 7fcdee046640 1 -- 172.21.15.133:0/2977757922 wait complete. 2024-04-25T10:06:09.900 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:06:09.898+0000 7fded4882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:06:09.902 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:06:09.902 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:09.902 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:06:10.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.014+0000 7f70138de640 1 Processor -- start 2024-04-25T10:06:10.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.014+0000 7f70138de640 1 -- start start 2024-04-25T10:06:10.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f700c142920 0x7f700c143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f700c140a40 0x7f700c0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0cf000 con 0x7f700c15d9f0 2024-04-25T10:06:10.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c140ea0 con 0x7f700c142920 2024-04-25T10:06:10.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0cf280 con 0x7f700c140a40 2024-04-25T10:06:10.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f701167a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f700c142920 0x7f700c143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f701167a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f700c142920 0x7f700c143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40142/0 (socket says 172.21.15.133:40142) 2024-04-25T10:06:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f701167a640 1 -- 172.21.15.133:0/586901453 learned_addr learned my addr 172.21.15.133:0/586901453 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f7010e79640 1 --2- 172.21.15.133:0/586901453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f700c140a40 0x7f700c0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f701167a640 1 -- 172.21.15.133:0/586901453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f700c140a40 msgr2=0x7f700c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f701167a640 1 --2- 172.21.15.133:0/586901453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f700c140a40 0x7f700c0ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f701167a640 1 -- 172.21.15.133:0/586901453 >> v1:172.21.15.162:6789/0 conn(0x7f700c15d9f0 legacy=0x7f700c06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f701167a640 1 -- 172.21.15.133:0/586901453 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c06d440 con 0x7f700c142920 2024-04-25T10:06:10.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f701167a640 1 --2- 172.21.15.133:0/586901453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c142920 0x7f700c143560 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f700800bb10 tx=0x7f700801e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ad9314a472615f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f7003fff640 1 -- 172.21.15.133:0/586901453 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080b4070 con 0x7f700c142920 2024-04-25T10:06:10.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f7003fff640 1 -- 172.21.15.133:0/586901453 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70080a7900 con 0x7f700c142920 2024-04-25T10:06:10.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f7003fff640 1 -- 172.21.15.133:0/586901453 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080a7cd0 con 0x7f700c142920 2024-04-25T10:06:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 -- 172.21.15.133:0/586901453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c142920 msgr2=0x7f700c143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 --2- 172.21.15.133:0/586901453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c142920 0x7f700c143560 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f700800bb10 tx=0x7f700801e980 comp rx=0 tx=0).stop 2024-04-25T10:06:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 -- 172.21.15.133:0/586901453 shutdown_connections 2024-04-25T10:06:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 --2- 172.21.15.133:0/586901453 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f700c140a40 0x7f700c0ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 --2- 172.21.15.133:0/586901453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c142920 0x7f700c143560 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 -- 172.21.15.133:0/586901453 >> 172.21.15.133:0/586901453 conn(0x7f700c066f70 msgr2=0x7f700c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 -- 172.21.15.133:0/586901453 shutdown_connections 2024-04-25T10:06:10.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.018+0000 7f70138de640 1 -- 172.21.15.133:0/586901453 wait complete. 2024-04-25T10:06:10.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f70138de640 1 Processor -- start 2024-04-25T10:06:10.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f70138de640 1 -- start start 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f70138de640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f700c140a40 0x7f700c13a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f70138de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f700c142920 0x7f700c1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f70138de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c15d9f0 0x7f700c15bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f70138de640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c06ebc0 con 0x7f700c15d9f0 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f70138de640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c06ea40 con 0x7f700c140a40 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f70138de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c142d00 con 0x7f700c142920 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7011e7b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c15d9f0 0x7f700c15bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7011e7b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c15d9f0 0x7f700c15bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40152/0 (socket says 172.21.15.133:40152) 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7011e7b640 1 -- 172.21.15.133:0/2055498806 learned_addr learned my addr 172.21.15.133:0/2055498806 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7011e7b640 1 -- 172.21.15.133:0/2055498806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f700c142920 msgr2=0x7f700c1563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7010e79640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f700c142920 0x7f700c1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f701167a640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f700c140a40 0x7f700c13a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7011e7b640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f700c142920 0x7f700c1563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7011e7b640 1 -- 172.21.15.133:0/2055498806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f700c140a40 msgr2=0x7f700c13a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7011e7b640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f700c140a40 0x7f700c13a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7011e7b640 1 -- 172.21.15.133:0/2055498806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c15ced0 con 0x7f700c15d9f0 2024-04-25T10:06:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f701167a640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f700c140a40 0x7f700c13a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7011e7b640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c15d9f0 0x7f700c15bd90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f700400b9a0 tx=0x7f700400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7001ffb640 1 -- 172.21.15.133:0/2055498806 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004019070 con 0x7f700c15d9f0 2024-04-25T10:06:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7001ffb640 1 -- 172.21.15.133:0/2055498806 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700400cba0 con 0x7f700c15d9f0 2024-04-25T10:06:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f70138de640 1 -- 172.21.15.133:0/2055498806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c154bf0 con 0x7f700c15d9f0 2024-04-25T10:06:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f7001ffb640 1 -- 172.21.15.133:0/2055498806 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004014440 con 0x7f700c15d9f0 2024-04-25T10:06:10.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.022+0000 7f70138de640 1 -- 172.21.15.133:0/2055498806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c15c910 con 0x7f700c15d9f0 2024-04-25T10:06:10.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7001ffb640 1 -- 172.21.15.133:0/2055498806 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f700401c070 con 0x7f700c15d9f0 2024-04-25T10:06:10.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7001ffb640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ff0066f60 0x7f6ff0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7001ffb640 1 -- 172.21.15.133:0/2055498806 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7004084d90 con 0x7f700c15d9f0 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f701167a640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ff0066f60 0x7f6ff0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f701167a640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ff0066f60 0x7f6ff0069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70080b1000 tx=0x7f70080a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7016a4b000 1 Processor -- start 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7016a4b000 1 -- start start 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 0x56549fd7d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fd7d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7016a4b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549fbcaf20 con 0x56549fdc7180 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7016a4b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549fb53000 con 0x56549fd7d600 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7016a4b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56549fbc6fc0 con 0x56549fdc7aa0 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f701167a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f701167a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40154/0 (socket says 172.21.15.133:40154) 2024-04-25T10:06:10.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f701167a640 1 -- 172.21.15.133:0/78405328 learned_addr learned my addr 172.21.15.133:0/78405328 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7011e7b640 1 --2- 172.21.15.133:0/78405328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fd7d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7010e79640 1 --2- 172.21.15.133:0/78405328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 0x56549fd7d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f701167a640 1 -- 172.21.15.133:0/78405328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 msgr2=0x56549fd7d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f701167a640 1 --2- 172.21.15.133:0/78405328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 0x56549fd7d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f701167a640 1 -- 172.21.15.133:0/78405328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 msgr2=0x56549fd7d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f701167a640 1 --2- 172.21.15.133:0/78405328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fd7d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f701167a640 1 -- 172.21.15.133:0/78405328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56549fba6ba0 con 0x56549fdc7180 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7011e7b640 1 --2- 172.21.15.133:0/78405328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fd7d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f701167a640 1 --2- 172.21.15.133:0/78405328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc7560 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7008006db0 tx=0x7f7008004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891b12557c376565 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f6ff6ffd640 1 -- 172.21.15.133:0/78405328 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080cb070 con 0x56549fdc7180 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f6ff6ffd640 1 -- 172.21.15.133:0/78405328 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70080c6470 con 0x56549fdc7180 2024-04-25T10:06:10.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f6ff6ffd640 1 -- 172.21.15.133:0/78405328 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080b4040 con 0x56549fdc7180 2024-04-25T10:06:10.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7016a4b000 1 -- 172.21.15.133:0/78405328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 msgr2=0x56549fdc7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.026+0000 7f7016a4b000 1 --2- 172.21.15.133:0/78405328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc7560 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7008006db0 tx=0x7f7008004740 comp rx=0 tx=0).stop 2024-04-25T10:06:10.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 -- 172.21.15.133:0/78405328 shutdown_connections 2024-04-25T10:06:10.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 --2- 172.21.15.133:0/78405328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fd7d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 --2- 172.21.15.133:0/78405328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 0x56549fd7d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 --2- 172.21.15.133:0/78405328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc7560 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 -- 172.21.15.133:0/78405328 >> 172.21.15.133:0/78405328 conn(0x56549fd83760 msgr2=0x56549fd7b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 -- 172.21.15.133:0/78405328 shutdown_connections 2024-04-25T10:06:10.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 -- 172.21.15.133:0/78405328 wait complete. 2024-04-25T10:06:10.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 -- start start 2024-04-25T10:06:10.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 Processor -- start 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fd7d600 0x56549fd826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fdc7180 0x56549fd82c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 0x56549fdb5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549fc889c0 con 0x56549fd7d600 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549fb855d0 con 0x56549fdc7180 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56549fb87080 con 0x56549fdc7aa0 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 10 client.0 _ll_register_callbacks cb 0x56549fd68a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f701167a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fd7d600 0x56549fd826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f701167a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fd7d600 0x56549fd826e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40166/0 (socket says 172.21.15.133:40166) 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f701167a640 1 -- 172.21.15.133:0/697501329 learned_addr learned my addr 172.21.15.133:0/697501329 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7010e79640 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fdc7180 0x56549fd82c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f701167a640 1 -- 172.21.15.133:0/697501329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 msgr2=0x56549fdb5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f701167a640 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 0x56549fdb5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f701167a640 1 -- 172.21.15.133:0/697501329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fdc7180 msgr2=0x56549fd82c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f701167a640 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fdc7180 0x56549fd82c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f701167a640 1 -- 172.21.15.133:0/697501329 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56549fc115a0 con 0x56549fd7d600 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7010e79640 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fdc7180 0x56549fd82c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f701167a640 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fd7d600 0x56549fd826e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f700809a420 tx=0x7f700801d1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e163218ae49774 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f6febfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f6febfff640 1 -- 172.21.15.133:0/697501329 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080b4070 con 0x56549fd7d600 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f6febfff640 1 -- 172.21.15.133:0/697501329 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f70080c6620 con 0x56549fd7d600 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 -- 172.21.15.133:0/697501329 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70080c6470 con 0x56549fd7d600 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f6febfff640 1 -- 172.21.15.133:0/697501329 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70080d9050 con 0x56549fd7d600 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 10 client.6420 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 -- 172.21.15.133:0/697501329 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56549fba22b0 con 0x56549fd7d600 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f6febfff640 1 -- 172.21.15.133:0/697501329 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70080c6470 con 0x56549fd7d600 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f6febfff640 1 -- 172.21.15.133:0/697501329 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f70080dc020 con 0x56549fd7d600 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 10 client.6420 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 10 client.6420 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 20 client.6420 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56549fdb4e60 0x56549fdb5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 4 client.6420 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7016a4b000 1 -- 172.21.15.133:0/697501329 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56549fb66c70 con 0x56549fdb4e60 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7010e79640 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56549fdb4e60 0x56549fdb5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f7010e79640 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56549fdb4e60 0x56549fdb5e60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82362f435ddb4379 server_cookie=875253743e82d1e1 in_seq=0 out_seq=0 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f6febfff640 10 client.6420 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f6febfff640 1 -- 172.21.15.133:0/697501329 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56549fb66c70 con 0x56549fdb4e60 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.030+0000 7f6febfff640 10 client.6420 handle_command_reply: tid=0 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 1 client.6420 shutdown 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 1 -- 172.21.15.133:0/697501329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fd7d600 msgr2=0x56549fd826e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fd7d600 0x56549fd826e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f700809a420 tx=0x7f700801d1c0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 1 -- 172.21.15.133:0/697501329 shutdown_connections 2024-04-25T10:06:10.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56549fdb4e60 0x56549fdb5e60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 0x56549fdb5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fdc7180 0x56549fd82c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f6febfff640 0 client.6420 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:10.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 1 --2- 172.21.15.133:0/697501329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fd7d600 0x56549fd826e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 1 -- 172.21.15.133:0/697501329 >> 172.21.15.133:0/697501329 conn(0x56549fd83760 msgr2=0x56549fd7c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 1 -- 172.21.15.133:0/697501329 shutdown_connections 2024-04-25T10:06:10.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 1 -- 172.21.15.133:0/697501329 wait complete. 2024-04-25T10:06:10.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.034+0000 7f7016a4b000 20 client.6420 trim_cache size 0 max 16384 2024-04-25T10:06:10.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 Processor -- start 2024-04-25T10:06:10.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 -- start start 2024-04-25T10:06:10.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fdc65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 0x56549fd7c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c1ff5e0 con 0x56549fdc7180 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c1fe810 con 0x56549fd7d600 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c138fa0 con 0x56549fdc7aa0 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7010e79640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7010e79640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40172/0 (socket says 172.21.15.133:40172) 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7010e79640 1 -- 172.21.15.133:0/3593329411 learned_addr learned my addr 172.21.15.133:0/3593329411 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7010e79640 1 -- 172.21.15.133:0/3593329411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 msgr2=0x56549fd7c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7010e79640 1 --2- 172.21.15.133:0/3593329411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 0x56549fd7c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7010e79640 1 -- 172.21.15.133:0/3593329411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 msgr2=0x56549fdc65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7010e79640 1 --2- 172.21.15.133:0/3593329411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fdc65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7010e79640 1 -- 172.21.15.133:0/3593329411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56549fb53250 con 0x56549fdc7180 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7010e79640 1 --2- 172.21.15.133:0/3593329411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc6b30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00d9a0 tx=0x7f6ffc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc0cb7ed7e0323a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f6ff67fc640 1 -- 172.21.15.133:0/3593329411 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00dca0 con 0x56549fdc7180 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f6ff67fc640 1 -- 172.21.15.133:0/3593329411 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc00de40 con 0x56549fdc7180 2024-04-25T10:06:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f6ff67fc640 1 -- 172.21.15.133:0/3593329411 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc004230 con 0x56549fdc7180 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 -- 172.21.15.133:0/3593329411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 msgr2=0x56549fdc6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 --2- 172.21.15.133:0/3593329411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc6b30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00d9a0 tx=0x7f6ffc008040 comp rx=0 tx=0).stop 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 -- 172.21.15.133:0/3593329411 shutdown_connections 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 --2- 172.21.15.133:0/3593329411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7aa0 0x56549fd7c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 --2- 172.21.15.133:0/3593329411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7180 0x56549fdc6b30 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 --2- 172.21.15.133:0/3593329411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fdc65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 -- 172.21.15.133:0/3593329411 >> 172.21.15.133:0/3593329411 conn(0x56549fd83760 msgr2=0x56549fdc5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 -- 172.21.15.133:0/3593329411 shutdown_connections 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 -- 172.21.15.133:0/3593329411 wait complete. 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 -- start start 2024-04-25T10:06:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 Processor -- start 2024-04-25T10:06:10.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.046+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fddbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7180 0x56549fddc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7aa0 0x56549fddc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549fc809d0 con 0x56549fdc7aa0 2024-04-25T10:06:10.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c1395a0 con 0x56549fd7d600 2024-04-25T10:06:10.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56549fbcf550 con 0x56549fdc7180 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 10 client.0 _ll_register_callbacks cb 0x56549fd49600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7011e7b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7aa0 0x56549fddc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7011e7b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7aa0 0x56549fddc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40178/0 (socket says 172.21.15.133:40178) 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7011e7b640 1 -- 172.21.15.133:0/2233136957 learned_addr learned my addr 172.21.15.133:0/2233136957 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f701167a640 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fddbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7010e79640 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7180 0x56549fddc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7011e7b640 1 -- 172.21.15.133:0/2233136957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7180 msgr2=0x56549fddc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7011e7b640 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7180 0x56549fddc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7011e7b640 1 -- 172.21.15.133:0/2233136957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 msgr2=0x56549fddbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7011e7b640 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fddbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7011e7b640 1 -- 172.21.15.133:0/2233136957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56549fb5fb90 con 0x56549fdc7aa0 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7010e79640 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7180 0x56549fddc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7011e7b640 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7aa0 0x56549fddc780 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f70040552a0 tx=0x7f70040556e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516386d374c48db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f6febfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f6febfff640 1 -- 172.21.15.133:0/2233136957 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700405da20 con 0x56549fdc7aa0 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f6febfff640 1 -- 172.21.15.133:0/2233136957 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f70040549f0 con 0x56549fdc7aa0 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 -- 172.21.15.133:0/2233136957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ffc00de40 con 0x56549fdc7aa0 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f6febfff640 1 -- 172.21.15.133:0/2233136957 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7004054c10 con 0x56549fdc7aa0 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 10 client.6426 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 -- 172.21.15.133:0/2233136957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56549fb72010 con 0x56549fdc7aa0 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f6febfff640 1 -- 172.21.15.133:0/2233136957 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc00de40 con 0x56549fdc7aa0 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f6febfff640 1 -- 172.21.15.133:0/2233136957 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f700407cb20 con 0x56549fdc7aa0 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 10 client.6426 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 10 client.6426 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 20 client.6426 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56549fe355f0 0x56549fe379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 4 client.6426 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 -- 172.21.15.133:0/2233136957 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56549fdb57e0 con 0x56549fe355f0 2024-04-25T10:06:10.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f701167a640 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56549fe355f0 0x56549fe379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f6febfff640 10 client.6426 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f701167a640 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56549fe355f0 0x56549fe379e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c811d09d72ea43bd server_cookie=a589775031cd241d in_seq=0 out_seq=0 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f6febfff640 1 -- 172.21.15.133:0/2233136957 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x56549fdb57e0 con 0x56549fe355f0 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f6febfff640 10 client.6426 handle_command_reply: tid=0 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 client.6426 shutdown 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 -- 172.21.15.133:0/2233136957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7aa0 msgr2=0x56549fddc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7aa0 0x56549fddc780 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f70040552a0 tx=0x7f70040556e0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 -- 172.21.15.133:0/2233136957 shutdown_connections 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56549fe355f0 0x56549fe379e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56549fdc7aa0 0x56549fddc780 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56549fdc7180 0x56549fddc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f6febfff640 0 client.6426 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 --2- 172.21.15.133:0/2233136957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56549fd7d600 0x56549fddbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 -- 172.21.15.133:0/2233136957 >> 172.21.15.133:0/2233136957 conn(0x56549fd83760 msgr2=0x56549fdc6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 -- 172.21.15.133:0/2233136957 shutdown_connections 2024-04-25T10:06:10.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 1 -- 172.21.15.133:0/2233136957 wait complete. 2024-04-25T10:06:10.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f7016a4b000 20 client.6426 trim_cache size 0 max 16384 2024-04-25T10:06:10.061 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:06:10.061 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:10.061 INFO:teuthology.orchestra.run.smithi133.stdout: "damage_type": "backtrace", 2024-04-25T10:06:10.061 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 1195972095, 2024-04-25T10:06:10.061 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 1099511630409, 2024-04-25T10:06:10.061 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T10:06:10.061 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:10.062 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:06:10.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 -- 172.21.15.133:0/2055498806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ff0066f60 msgr2=0x7f6ff0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ff0066f60 0x7f6ff0069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70080b1000 tx=0x7f70080a9070 comp rx=0 tx=0).stop 2024-04-25T10:06:10.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 -- 172.21.15.133:0/2055498806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c15d9f0 msgr2=0x7f700c15bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c15d9f0 0x7f700c15bd90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f700400b9a0 tx=0x7f700400be60 comp rx=0 tx=0).stop 2024-04-25T10:06:10.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 -- 172.21.15.133:0/2055498806 shutdown_connections 2024-04-25T10:06:10.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ff0066f60 0x7f6ff0069420 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c15d9f0 0x7f700c15bd90 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f700c142920 0x7f700c1563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 --2- 172.21.15.133:0/2055498806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f700c140a40 0x7f700c13a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 -- 172.21.15.133:0/2055498806 >> 172.21.15.133:0/2055498806 conn(0x7f700c066f70 msgr2=0x7f700c1415d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 -- 172.21.15.133:0/2055498806 shutdown_connections 2024-04-25T10:06:10.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.050+0000 7f70138de640 1 -- 172.21.15.133:0/2055498806 wait complete. 2024-04-25T10:06:10.071 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:10.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.178+0000 7f7c5b4a0640 1 Processor -- start 2024-04-25T10:06:10.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.178+0000 7f7c5b4a0640 1 -- start start 2024-04-25T10:06:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c5b4a0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c5415c9a0 0x7f7c540c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c5b4a0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c540b1f70 0x7f7c5406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c5b4a0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c540ce330 0x7f7c540ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c5b4a0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c540cecc0 con 0x7f7c540b1f70 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c5b4a0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c540c61d0 con 0x7f7c5415c9a0 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c5b4a0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c540c64b0 con 0x7f7c540ce330 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c58a3b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c540b1f70 0x7f7c5406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c58a3b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c540b1f70 0x7f7c5406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40190/0 (socket says 172.21.15.133:40190) 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c58a3b640 1 -- 172.21.15.133:0/1747752073 learned_addr learned my addr 172.21.15.133:0/1747752073 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c59a3d640 1 --2- 172.21.15.133:0/1747752073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c540ce330 0x7f7c540ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c5923c640 1 --2- 172.21.15.133:0/1747752073 >> v2:172.21.15.162:3300/0 conn(0x7f7c5415c9a0 0x7f7c540c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c58a3b640 1 -- 172.21.15.133:0/1747752073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c540ce330 msgr2=0x7f7c540ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c58a3b640 1 --2- 172.21.15.133:0/1747752073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c540ce330 0x7f7c540ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c58a3b640 1 -- 172.21.15.133:0/1747752073 >> v2:172.21.15.162:3300/0 conn(0x7f7c5415c9a0 msgr2=0x7f7c540c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c58a3b640 1 --2- 172.21.15.133:0/1747752073 >> v2:172.21.15.162:3300/0 conn(0x7f7c5415c9a0 0x7f7c540c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c58a3b640 1 -- 172.21.15.133:0/1747752073 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5406d4f0 con 0x7f7c540b1f70 2024-04-25T10:06:10.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c58a3b640 1 --2- 172.21.15.133:0/1747752073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c540b1f70 0x7f7c5406dcf0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7c5406f5f0 tx=0x7f7c4401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86d4a6a7caf70c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c4bfff640 1 -- 172.21.15.133:0/1747752073 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c440a7070 con 0x7f7c540b1f70 2024-04-25T10:06:10.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c4bfff640 1 -- 172.21.15.133:0/1747752073 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c440b0070 con 0x7f7c540b1f70 2024-04-25T10:06:10.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.182+0000 7f7c4bfff640 1 -- 172.21.15.133:0/1747752073 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c440ab890 con 0x7f7c540b1f70 2024-04-25T10:06:10.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/1747752073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c540b1f70 msgr2=0x7f7c5406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 --2- 172.21.15.133:0/1747752073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c540b1f70 0x7f7c5406dcf0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7c5406f5f0 tx=0x7f7c4401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/1747752073 shutdown_connections 2024-04-25T10:06:10.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 --2- 172.21.15.133:0/1747752073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c540ce330 0x7f7c540ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 --2- 172.21.15.133:0/1747752073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c540b1f70 0x7f7c5406dcf0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 --2- 172.21.15.133:0/1747752073 >> v2:172.21.15.162:3300/0 conn(0x7f7c5415c9a0 0x7f7c540c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/1747752073 >> 172.21.15.133:0/1747752073 conn(0x7f7c54067000 msgr2=0x7f7c5406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/1747752073 shutdown_connections 2024-04-25T10:06:10.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/1747752073 wait complete. 2024-04-25T10:06:10.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 Processor -- start 2024-04-25T10:06:10.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 -- start start 2024-04-25T10:06:10.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c540b1f70 0x7f7c54158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c540ce330 0x7f7c54156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5415c9a0 0x7f7c54155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5415cf00 con 0x7f7c540ce330 2024-04-25T10:06:10.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5415cd80 con 0x7f7c540b1f70 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5415d950 con 0x7f7c5415c9a0 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c58a3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c540ce330 0x7f7c54156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c58a3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c540ce330 0x7f7c54156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40194/0 (socket says 172.21.15.133:40194) 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c58a3b640 1 -- 172.21.15.133:0/895497439 learned_addr learned my addr 172.21.15.133:0/895497439 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5923c640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c540b1f70 0x7f7c54158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c59a3d640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5415c9a0 0x7f7c54155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c58a3b640 1 -- 172.21.15.133:0/895497439 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5415c9a0 msgr2=0x7f7c54155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c58a3b640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5415c9a0 0x7f7c54155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c58a3b640 1 -- 172.21.15.133:0/895497439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c540b1f70 msgr2=0x7f7c54158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c58a3b640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c540b1f70 0x7f7c54158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c58a3b640 1 -- 172.21.15.133:0/895497439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5415a810 con 0x7f7c540ce330 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c58a3b640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c540ce330 0x7f7c54156bb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7c4401aba0 tx=0x7f7c4401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c49ffb640 1 -- 172.21.15.133:0/895497439 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c440b4800 con 0x7f7c540ce330 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c49ffb640 1 -- 172.21.15.133:0/895497439 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c440b49a0 con 0x7f7c540ce330 2024-04-25T10:06:10.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/895497439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5415aaf0 con 0x7f7c540ce330 2024-04-25T10:06:10.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c49ffb640 1 -- 172.21.15.133:0/895497439 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c440b4d90 con 0x7f7c540ce330 2024-04-25T10:06:10.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.186+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/895497439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c541461e0 con 0x7f7c540ce330 2024-04-25T10:06:10.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.190+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/895497439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5406bc20 con 0x7f7c540ce330 2024-04-25T10:06:10.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.190+0000 7f7c49ffb640 1 -- 172.21.15.133:0/895497439 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c440c7390 con 0x7f7c540ce330 2024-04-25T10:06:10.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.190+0000 7f7c49ffb640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 0x7f7c38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.190+0000 7f7c49ffb640 1 -- 172.21.15.133:0/895497439 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7c4411faf0 con 0x7f7c540ce330 2024-04-25T10:06:10.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.190+0000 7f7c5923c640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 0x7f7c38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.190+0000 7f7c5923c640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 0x7f7c38069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000a8d0 tx=0x7f7c50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.194+0000 7f7c49ffb640 1 -- 172.21.15.133:0/895497439 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c440c4060 con 0x7f7c540ce330 2024-04-25T10:06:10.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/895497439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c54068ef0 con 0x7f7c540ce330 2024-04-25T10:06:10.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c49ffb640 1 -- 172.21.15.133:0/895497439 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f7c44016110 con 0x7f7c540ce330 2024-04-25T10:06:10.390 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:10.390 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:06:09.247920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:10.390 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 24 2024-04-25T10:06:10.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/895497439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 msgr2=0x7f7c38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 0x7f7c38069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000a8d0 tx=0x7f7c50008040 comp rx=0 tx=0).stop 2024-04-25T10:06:10.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/895497439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c540ce330 msgr2=0x7f7c54156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c540ce330 0x7f7c54156bb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7c4401aba0 tx=0x7f7c4401ad30 comp rx=0 tx=0).stop 2024-04-25T10:06:10.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/895497439 shutdown_connections 2024-04-25T10:06:10.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5415c9a0 0x7f7c54155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 0x7f7c38069420 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c540ce330 0x7f7c54156bb0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 --2- 172.21.15.133:0/895497439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c540b1f70 0x7f7c54158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/895497439 >> 172.21.15.133:0/895497439 conn(0x7f7c54067000 msgr2=0x7f7c5415afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/895497439 shutdown_connections 2024-04-25T10:06:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.386+0000 7f7c5b4a0640 1 -- 172.21.15.133:0/895497439 wait complete. 2024-04-25T10:06:10.407 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 1195972095 2024-04-25T10:06:10.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.518+0000 7f6852876640 1 Processor -- start 2024-04-25T10:06:10.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.518+0000 7f6852876640 1 -- start start 2024-04-25T10:06:10.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6852876640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0d0a30 0x7f684c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6852876640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0b1820 con 0x7f684c0ce080 2024-04-25T10:06:10.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6852876640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0cdc40 con 0x7f684c06cc80 2024-04-25T10:06:10.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6852876640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0b1ac0 con 0x7f684c0d0a30 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f684c0ce080 0x7f684c0ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34470/0 (socket says 172.21.15.133:34470) 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684bfff640 1 -- 172.21.15.133:0/2452147514 learned_addr learned my addr 172.21.15.133:0/2452147514 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217542788 0 0) 0x7f684c0b1820 con 0x7f684c0ce080 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6834003530 con 0x7f684c0ce080 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6850e13640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0d0a30 0x7f684c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2061992034 0 0) 0x7f6834003530 con 0x7f684c0ce080 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0d0a30 msgr2=0x7f684c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0d0a30 0x7f684c0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 >> v1:172.21.15.162:6789/0 conn(0x7f684c06cc80 legacy=0x7f684c1604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0b1da0 con 0x7f684c0ce080 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f683c001820 con 0x7f684c0ce080 2024-04-25T10:06:10.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6850e13640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0d0a30 0x7f684c0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:10.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 >> v1:172.21.15.133:6789/0 conn(0x7f684c0ce080 legacy=0x7f684c0ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0d0a30 0x7f68340039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6834003fe0 0x7f6834004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6834004d50 0x7f68340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0b1ac0 con 0x7f6834004d50 2024-04-25T10:06:10.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683400bb30 con 0x7f6834003fe0 2024-04-25T10:06:10.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683400bd30 con 0x7f684c0d0a30 2024-04-25T10:06:10.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6850e13640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6834004d50 0x7f68340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684bfff640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0d0a30 0x7f68340039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6850e13640 1 -- 172.21.15.133:0/2452147514 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0d0a30 msgr2=0x7f68340039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6850e13640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0d0a30 0x7f68340039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6850e13640 1 -- 172.21.15.133:0/2452147514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6834003fe0 msgr2=0x7f6834004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6850e13640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6834003fe0 0x7f6834004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6850e13640 1 -- 172.21.15.133:0/2452147514 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6834003530 con 0x7f6834004d50 2024-04-25T10:06:10.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f6850e13640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6834004d50 0x7f68340051d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f684c0d0e30 tx=0x7f684401ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a362a604b57aff95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844009c80 con 0x7f6834004d50 2024-04-25T10:06:10.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68440a5d60 con 0x7f6834004d50 2024-04-25T10:06:10.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.522+0000 7f684affd640 1 -- 172.21.15.133:0/2452147514 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844003ba0 con 0x7f6834004d50 2024-04-25T10:06:10.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 -- 172.21.15.133:0/2452147514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6834004d50 msgr2=0x7f68340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 --2- 172.21.15.133:0/2452147514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6834004d50 0x7f68340051d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f684c0d0e30 tx=0x7f684401ce60 comp rx=0 tx=0).stop 2024-04-25T10:06:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6850e13640 1 -- 172.21.15.133:0/2452147514 reap_dead start 2024-04-25T10:06:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 -- 172.21.15.133:0/2452147514 shutdown_connections 2024-04-25T10:06:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 -- 172.21.15.133:0/2452147514 >> 172.21.15.133:0/2452147514 conn(0x7f684c09ab90 msgr2=0x7f684c0d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 -- 172.21.15.133:0/2452147514 shutdown_connections 2024-04-25T10:06:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 -- 172.21.15.133:0/2452147514 wait complete. 2024-04-25T10:06:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 Processor -- start 2024-04-25T10:06:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 -- start start 2024-04-25T10:06:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f684c1f3580 0x7f684c0b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f684c1f3960 0x7f684c0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0b4820 0x7f684c0b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0b2640 con 0x7f684c1f3580 2024-04-25T10:06:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0b2360 con 0x7f684c1f3960 2024-04-25T10:06:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0b1160 con 0x7f684c0b4820 2024-04-25T10:06:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f684bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f684c1f3580 0x7f684c0b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f684bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f684c1f3580 0x7f684c0b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40210/0 (socket says 172.21.15.133:40210) 2024-04-25T10:06:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f684bfff640 1 -- 172.21.15.133:0/4025477668 learned_addr learned my addr 172.21.15.133:0/4025477668 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f684b7fe640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f684c1f3960 0x7f684c0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6850e13640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0b4820 0x7f684c0b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f684bfff640 1 -- 172.21.15.133:0/4025477668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0b4820 msgr2=0x7f684c0b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f684bfff640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0b4820 0x7f684c0b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f684bfff640 1 -- 172.21.15.133:0/4025477668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f684c1f3960 msgr2=0x7f684c0c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f684bfff640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f684c1f3960 0x7f684c0c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f684bfff640 1 -- 172.21.15.133:0/4025477668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0c5ee0 con 0x7f684c1f3580 2024-04-25T10:06:10.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6850e13640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0b4820 0x7f684c0b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f684b7fe640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f684c1f3960 0x7f684c0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f684bfff640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f684c1f3580 0x7f684c0b6ec0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f683c001ec0 tx=0x7f683c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6848ff9640 1 -- 172.21.15.133:0/4025477668 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c0072e0 con 0x7f684c1f3580 2024-04-25T10:06:10.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6848ff9640 1 -- 172.21.15.133:0/4025477668 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c018070 con 0x7f684c1f3580 2024-04-25T10:06:10.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 -- 172.21.15.133:0/4025477668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c0c6180 con 0x7f684c1f3580 2024-04-25T10:06:10.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6848ff9640 1 -- 172.21.15.133:0/4025477668 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c0057b0 con 0x7f684c1f3580 2024-04-25T10:06:10.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.526+0000 7f6852876640 1 -- 172.21.15.133:0/4025477668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c0c6740 con 0x7f684c1f3580 2024-04-25T10:06:10.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6848ff9640 1 -- 172.21.15.133:0/4025477668 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f683c005950 con 0x7f684c1f3580 2024-04-25T10:06:10.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6848ff9640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6838066f60 0x7f6838069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6848ff9640 1 -- 172.21.15.133:0/4025477668 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f683c085100 con 0x7f684c1f3580 2024-04-25T10:06:10.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f684b7fe640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6838066f60 0x7f6838069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f684b7fe640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6838066f60 0x7f6838069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f684c1f72a0 tx=0x7f6840009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f68559e3000 1 Processor -- start 2024-04-25T10:06:10.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f68559e3000 1 -- start start 2024-04-25T10:06:10.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f68559e3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 0x55e170b14450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f68559e3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170ac76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f68559e3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f68559e3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e170917f00 con 0x55e170ac7be0 2024-04-25T10:06:10.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f68559e3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1708a04a0 con 0x55e170b14990 2024-04-25T10:06:10.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f68559e3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e170913fa0 con 0x55e170b14070 2024-04-25T10:06:10.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6850e13640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6850e13640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40216/0 (socket says 172.21.15.133:40216) 2024-04-25T10:06:10.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6850e13640 1 -- 172.21.15.133:0/2442100600 learned_addr learned my addr 172.21.15.133:0/2442100600 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f684bfff640 1 --2- 172.21.15.133:0/2442100600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 0x55e170b14450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f684b7fe640 1 --2- 172.21.15.133:0/2442100600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170ac76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6850e13640 1 -- 172.21.15.133:0/2442100600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 msgr2=0x55e170b14450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6850e13640 1 --2- 172.21.15.133:0/2442100600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 0x55e170b14450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6850e13640 1 -- 172.21.15.133:0/2442100600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 msgr2=0x55e170ac76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6850e13640 1 --2- 172.21.15.133:0/2442100600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170ac76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6850e13640 1 -- 172.21.15.133:0/2442100600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e17089dd00 con 0x55e170ac7be0 2024-04-25T10:06:10.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f684b7fe640 1 --2- 172.21.15.133:0/2442100600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170ac76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6850e13640 1 --2- 172.21.15.133:0/2442100600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac7fc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6844009760 tx=0x7f6844004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6e47d426a4ecee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6831ffb640 1 -- 172.21.15.133:0/2442100600 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68440b2040 con 0x55e170ac7be0 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6831ffb640 1 -- 172.21.15.133:0/2442100600 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844009c80 con 0x55e170ac7be0 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.530+0000 7f6831ffb640 1 -- 172.21.15.133:0/2442100600 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68440a5d60 con 0x55e170ac7be0 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- 172.21.15.133:0/2442100600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 msgr2=0x55e170ac7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 --2- 172.21.15.133:0/2442100600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac7fc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6844009760 tx=0x7f6844004ac0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- 172.21.15.133:0/2442100600 shutdown_connections 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 --2- 172.21.15.133:0/2442100600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac7fc0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 --2- 172.21.15.133:0/2442100600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170ac76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 --2- 172.21.15.133:0/2442100600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 0x55e170b14450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- 172.21.15.133:0/2442100600 >> 172.21.15.133:0/2442100600 conn(0x55e170af4e10 msgr2=0x55e170af7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- 172.21.15.133:0/2442100600 shutdown_connections 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- 172.21.15.133:0/2442100600 wait complete. 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- start start 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 Processor -- start 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170ac7be0 0x55e170b1b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170b14070 0x55e170b1b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170acdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e17091c530 con 0x55e170b14070 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1708d2ae0 con 0x55e170b14990 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1708d6e10 con 0x55e170ac7be0 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 10 client.0 _ll_register_callbacks cb 0x55e170ab5750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170b14070 0x55e170b1b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170b14070 0x55e170b1b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40226/0 (socket says 172.21.15.133:40226) 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684b7fe640 1 -- 172.21.15.133:0/2972171964 learned_addr learned my addr 172.21.15.133:0/2972171964 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f6850e13640 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170acdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684bfff640 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170ac7be0 0x55e170b1b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684b7fe640 1 -- 172.21.15.133:0/2972171964 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170ac7be0 msgr2=0x55e170b1b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684b7fe640 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170ac7be0 0x55e170b1b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684b7fe640 1 -- 172.21.15.133:0/2972171964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 msgr2=0x55e170acdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684b7fe640 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170acdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684b7fe640 1 -- 172.21.15.133:0/2972171964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1708baa10 con 0x55e170b14070 2024-04-25T10:06:10.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684b7fe640 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170b14070 0x55e170b1b910 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6840001e80 tx=0x7f6840001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d3f52ed1bf66c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f682affd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:10.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f682affd640 1 -- 172.21.15.133:0/2972171964 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840003c80 con 0x55e170b14070 2024-04-25T10:06:10.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f682affd640 1 -- 172.21.15.133:0/2972171964 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6840023c60 con 0x55e170b14070 2024-04-25T10:06:10.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- 172.21.15.133:0/2972171964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6844009c80 con 0x55e170b14070 2024-04-25T10:06:10.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f682affd640 1 -- 172.21.15.133:0/2972171964 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6840023e60 con 0x55e170b14070 2024-04-25T10:06:10.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 10 client.6444 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:10.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- 172.21.15.133:0/2972171964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e170912ef0 con 0x55e170b14070 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f682affd640 1 -- 172.21.15.133:0/2972171964 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844009c80 con 0x55e170b14070 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f682affd640 1 -- 172.21.15.133:0/2972171964 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f684002a050 con 0x55e170b14070 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 10 client.6444 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 10 client.6444 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 20 client.6444 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e170ace810 0x55e170acebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 4 client.6444 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 -- 172.21.15.133:0/2972171964 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1708b3b20 con 0x55e170ace810 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684bfff640 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e170ace810 0x55e170acebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f682affd640 10 client.6444 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f684bfff640 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e170ace810 0x55e170acebf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d2c8128150fc83 server_cookie=77d339c2399efce1 in_seq=0 out_seq=0 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f682affd640 1 -- 172.21.15.133:0/2972171964 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e1708b3b20 con 0x55e170ace810 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f682affd640 10 client.6444 handle_command_reply: tid=0 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.534+0000 7f68559e3000 1 client.6444 shutdown 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f68559e3000 1 -- 172.21.15.133:0/2972171964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170b14070 msgr2=0x55e170b1b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f68559e3000 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170b14070 0x55e170b1b910 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6840001e80 tx=0x7f6840001090 comp rx=0 tx=0).stop 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f68559e3000 1 -- 172.21.15.133:0/2972171964 shutdown_connections 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f68559e3000 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e170ace810 0x55e170acebf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f68559e3000 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170acdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f68559e3000 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170b14070 0x55e170b1b910 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f68559e3000 1 --2- 172.21.15.133:0/2972171964 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170ac7be0 0x55e170b1b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f68559e3000 1 -- 172.21.15.133:0/2972171964 >> 172.21.15.133:0/2972171964 conn(0x55e170af4e10 msgr2=0x55e170ac6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f682affd640 0 client.6444 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f68559e3000 1 -- 172.21.15.133:0/2972171964 shutdown_connections 2024-04-25T10:06:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f68559e3000 1 -- 172.21.15.133:0/2972171964 wait complete. 2024-04-25T10:06:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.538+0000 7f68559e3000 20 client.6444 trim_cache size 0 max 16384 2024-04-25T10:06:10.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.546+0000 7f68559e3000 1 Processor -- start 2024-04-25T10:06:10.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- start start 2024-04-25T10:06:10.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170acebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 0x55e170acf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170acf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1ff220 con 0x55e170ac7be0 2024-04-25T10:06:10.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1ff820 con 0x55e170b14990 2024-04-25T10:06:10.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1708e4b90 con 0x55e170b14070 2024-04-25T10:06:10.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170acebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170acebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40240/0 (socket says 172.21.15.133:40240) 2024-04-25T10:06:10.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 -- 172.21.15.133:0/1965892321 learned_addr learned my addr 172.21.15.133:0/1965892321 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 -- 172.21.15.133:0/1965892321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 msgr2=0x55e170acf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 --2- 172.21.15.133:0/1965892321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 0x55e170acf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 -- 172.21.15.133:0/1965892321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 msgr2=0x55e170acf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 --2- 172.21.15.133:0/1965892321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170acf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 -- 172.21.15.133:0/1965892321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e170b147c0 con 0x55e170ac7be0 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 --2- 172.21.15.133:0/1965892321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170acebf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f683c0834a0 tx=0x7f683c083370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff664935ab0ba3b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68317fa640 1 -- 172.21.15.133:0/1965892321 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c055880 con 0x55e170ac7be0 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68317fa640 1 -- 172.21.15.133:0/1965892321 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c06d070 con 0x55e170ac7be0 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68317fa640 1 -- 172.21.15.133:0/1965892321 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c0599d0 con 0x55e170ac7be0 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- 172.21.15.133:0/1965892321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 msgr2=0x55e170acebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 --2- 172.21.15.133:0/1965892321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170acebf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f683c0834a0 tx=0x7f683c083370 comp rx=0 tx=0).stop 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- 172.21.15.133:0/1965892321 shutdown_connections 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 --2- 172.21.15.133:0/1965892321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170acf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 --2- 172.21.15.133:0/1965892321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 0x55e170acf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 --2- 172.21.15.133:0/1965892321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170acebf0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- 172.21.15.133:0/1965892321 >> 172.21.15.133:0/1965892321 conn(0x55e170af4e10 msgr2=0x55e170acd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- 172.21.15.133:0/1965892321 shutdown_connections 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- 172.21.15.133:0/1965892321 wait complete. 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- start start 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 Processor -- start 2024-04-25T10:06:10.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 0x55e170b1eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170b1f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1708d5700 con 0x55e170ac7be0 2024-04-25T10:06:10.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e170ac7750 con 0x55e170b14990 2024-04-25T10:06:10.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e170ad31c0 con 0x55e170b14070 2024-04-25T10:06:10.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f68559e3000 10 client.0 _ll_register_callbacks cb 0x7f6834003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40246/0 (socket says 172.21.15.133:40246) 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 -- 172.21.15.133:0/2836201911 learned_addr learned my addr 172.21.15.133:0/2836201911 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 -- 172.21.15.133:0/2836201911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 msgr2=0x55e170b1eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 --2- 172.21.15.133:0/2836201911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 0x55e170b1eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 -- 172.21.15.133:0/2836201911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 msgr2=0x55e170b1f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 --2- 172.21.15.133:0/2836201911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170b1f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 -- 172.21.15.133:0/2836201911 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1708d7790 con 0x55e170ac7be0 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f684bfff640 1 --2- 172.21.15.133:0/2836201911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac6900 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f683c055580 tx=0x7f683c000f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d396eecf65d8a002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f682affd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:10.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.550+0000 7f682affd640 1 -- 172.21.15.133:0/2836201911 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c080070 con 0x55e170ac7be0 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f682affd640 1 -- 172.21.15.133:0/2836201911 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f683c053b80 con 0x55e170ac7be0 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 -- 172.21.15.133:0/2836201911 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f683c06d070 con 0x55e170ac7be0 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f682affd640 1 -- 172.21.15.133:0/2836201911 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f683c055880 con 0x55e170ac7be0 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 10 client.6450 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 -- 172.21.15.133:0/2836201911 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e1708dfda0 con 0x55e170ac7be0 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f682affd640 1 -- 172.21.15.133:0/2836201911 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c06d070 con 0x55e170ac7be0 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f682affd640 1 -- 172.21.15.133:0/2836201911 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f683c0830b0 con 0x55e170ac7be0 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 10 client.6450 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 10 client.6450 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 20 client.6450 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 --2- 172.21.15.133:0/2836201911 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e170b74510 0x55e170b748f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 4 client.6450 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 1195972095}] 2024-04-25T10:06:10.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 -- 172.21.15.133:0/2836201911 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1195972095}) v1 -- 0x55e170925440 con 0x55e170b74510 2024-04-25T10:06:10.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f684b7fe640 1 --2- 172.21.15.133:0/2836201911 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e170b74510 0x55e170b748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f682affd640 10 client.6450 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:10.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f684b7fe640 1 --2- 172.21.15.133:0/2836201911 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e170b74510 0x55e170b748f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bbbada84e1f335f server_cookie=4339b11a744f0cb7 in_seq=0 out_seq=0 2024-04-25T10:06:10.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f682affd640 1 -- 172.21.15.133:0/2836201911 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55e170925440 con 0x55e170b74510 2024-04-25T10:06:10.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f682affd640 10 client.6450 handle_command_reply: tid=0 2024-04-25T10:06:10.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 client.6450 shutdown 2024-04-25T10:06:10.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 -- 172.21.15.133:0/2836201911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 msgr2=0x55e170ac6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 --2- 172.21.15.133:0/2836201911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac6900 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f683c055580 tx=0x7f683c000f80 comp rx=0 tx=0).stop 2024-04-25T10:06:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 -- 172.21.15.133:0/2836201911 shutdown_connections 2024-04-25T10:06:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 --2- 172.21.15.133:0/2836201911 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55e170b74510 0x55e170b748f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 --2- 172.21.15.133:0/2836201911 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e170b14990 0x55e170b1f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 --2- 172.21.15.133:0/2836201911 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e170b14070 0x55e170b1eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f682affd640 0 client.6450 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 --2- 172.21.15.133:0/2836201911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e170ac7be0 0x55e170ac6900 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 -- 172.21.15.133:0/2836201911 >> 172.21.15.133:0/2836201911 conn(0x55e170af4e10 msgr2=0x55e170ac5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 -- 172.21.15.133:0/2836201911 shutdown_connections 2024-04-25T10:06:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 1 -- 172.21.15.133:0/2836201911 wait complete. 2024-04-25T10:06:10.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f68559e3000 20 client.6450 trim_cache size 0 max 16384 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 -- 172.21.15.133:0/4025477668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6838066f60 msgr2=0x7f6838069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6838066f60 0x7f6838069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f684c1f72a0 tx=0x7f6840009040 comp rx=0 tx=0).stop 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 -- 172.21.15.133:0/4025477668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f684c1f3580 msgr2=0x7f684c0b6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f684c1f3580 0x7f684c0b6ec0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f683c001ec0 tx=0x7f683c00ec80 comp rx=0 tx=0).stop 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 -- 172.21.15.133:0/4025477668 shutdown_connections 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f684c0b4820 0x7f684c0b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f684c1f3960 0x7f684c0c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6838066f60 0x7f6838069420 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 --2- 172.21.15.133:0/4025477668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f684c1f3580 0x7f684c0b6ec0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 -- 172.21.15.133:0/4025477668 >> 172.21.15.133:0/4025477668 conn(0x7f684c09ab90 msgr2=0x7f684c0b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 -- 172.21.15.133:0/4025477668 shutdown_connections 2024-04-25T10:06:10.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.554+0000 7f6852876640 1 -- 172.21.15.133:0/4025477668 wait complete. 2024-04-25T10:06:10.573 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:06:10.573 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:10.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.678+0000 7f19511a2640 1 Processor -- start 2024-04-25T10:06:10.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.678+0000 7f19511a2640 1 -- start start 2024-04-25T10:06:10.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f19511a2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194c15c9a0 0x7f194c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f19511a2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f194c0ce330 0x7f194c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f19511a2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0c61d0 con 0x7f194c0b1f70 2024-04-25T10:06:10.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f19511a2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c0cece0 con 0x7f194c0ce330 2024-04-25T10:06:10.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f19511a2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0c64b0 con 0x7f194c15c9a0 2024-04-25T10:06:10.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f194b59e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f194c0ce330 0x7f194c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f194a59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f194c0b1f70 0x7f194c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34482/0 (socket says 172.21.15.133:34482) 2024-04-25T10:06:10.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f194a59c640 1 -- 172.21.15.133:0/889285082 learned_addr learned my addr 172.21.15.133:0/889285082 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:10.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f194ad9d640 1 --2- 172.21.15.133:0/889285082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194c15c9a0 0x7f194c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f194b59e640 1 -- 172.21.15.133:0/889285082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194c15c9a0 msgr2=0x7f194c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f194b59e640 1 --2- 172.21.15.133:0/889285082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194c15c9a0 0x7f194c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f194b59e640 1 -- 172.21.15.133:0/889285082 >> v1:172.21.15.133:6789/0 conn(0x7f194c0b1f70 legacy=0x7f194c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f194b59e640 1 -- 172.21.15.133:0/889285082 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c06d4f0 con 0x7f194c0ce330 2024-04-25T10:06:10.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f1949d9b640 1 -- 172.21.15.133:0/889285082 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128074968 0 0) 0x7f194c0c61d0 con 0x7f194c0b1f70 2024-04-25T10:06:10.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f194b59e640 1 --2- 172.21.15.133:0/889285082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f194c0ce330 0x7f194c0ce7a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f193c009d90 tx=0x7f193c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a17601200f5999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f1949d9b640 1 -- 172.21.15.133:0/889285082 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c0b2070 con 0x7f194c0ce330 2024-04-25T10:06:10.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f1949d9b640 1 -- 172.21.15.133:0/889285082 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c0a58e0 con 0x7f194c0ce330 2024-04-25T10:06:10.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.682+0000 7f1949d9b640 1 -- 172.21.15.133:0/889285082 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c0a5cb0 con 0x7f194c0ce330 2024-04-25T10:06:10.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 -- 172.21.15.133:0/889285082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f194c0ce330 msgr2=0x7f194c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 --2- 172.21.15.133:0/889285082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f194c0ce330 0x7f194c0ce7a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f193c009d90 tx=0x7f193c01c970 comp rx=0 tx=0).stop 2024-04-25T10:06:10.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 -- 172.21.15.133:0/889285082 shutdown_connections 2024-04-25T10:06:10.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 --2- 172.21.15.133:0/889285082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f194c0ce330 0x7f194c0ce7a0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 --2- 172.21.15.133:0/889285082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194c15c9a0 0x7f194c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 -- 172.21.15.133:0/889285082 >> 172.21.15.133:0/889285082 conn(0x7f194c067000 msgr2=0x7f194c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 -- 172.21.15.133:0/889285082 shutdown_connections 2024-04-25T10:06:10.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 -- 172.21.15.133:0/889285082 wait complete. 2024-04-25T10:06:10.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 Processor -- start 2024-04-25T10:06:10.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 -- start start 2024-04-25T10:06:10.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f194c0b1f70 0x7f194c14ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194c0ce330 0x7f194c1494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f194c15c9a0 0x7f194c149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c15dad0 con 0x7f194c15c9a0 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c15d950 con 0x7f194c0b1f70 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c15dc50 con 0x7f194c0ce330 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f194b59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f194c15c9a0 0x7f194c149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f194b59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f194c15c9a0 0x7f194c149a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40278/0 (socket says 172.21.15.133:40278) 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f194b59e640 1 -- 172.21.15.133:0/326935536 learned_addr learned my addr 172.21.15.133:0/326935536 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f194ad9d640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f194c0b1f70 0x7f194c14ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f194b59e640 1 -- 172.21.15.133:0/326935536 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194c0ce330 msgr2=0x7f194c1494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f194b59e640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194c0ce330 0x7f194c1494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f194b59e640 1 -- 172.21.15.133:0/326935536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f194c0b1f70 msgr2=0x7f194c14ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f194b59e640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f194c0b1f70 0x7f194c14ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f194b59e640 1 -- 172.21.15.133:0/326935536 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c14e350 con 0x7f194c15c9a0 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f194b59e640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f194c15c9a0 0x7f194c149a20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f193c0b0040 tx=0x7f193c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19437fe640 1 -- 172.21.15.133:0/326935536 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c0b2050 con 0x7f194c15c9a0 2024-04-25T10:06:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19437fe640 1 -- 172.21.15.133:0/326935536 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c0b3040 con 0x7f194c15c9a0 2024-04-25T10:06:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19437fe640 1 -- 172.21.15.133:0/326935536 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c0ad850 con 0x7f194c15c9a0 2024-04-25T10:06:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 -- 172.21.15.133:0/326935536 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c0b2530 con 0x7f194c15c9a0 2024-04-25T10:06:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.686+0000 7f19511a2640 1 -- 172.21.15.133:0/326935536 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c149f60 con 0x7f194c15c9a0 2024-04-25T10:06:10.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.690+0000 7f19511a2640 1 -- 172.21.15.133:0/326935536 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c06bc20 con 0x7f194c15c9a0 2024-04-25T10:06:10.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.690+0000 7f19437fe640 1 -- 172.21.15.133:0/326935536 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f193c0c0020 con 0x7f194c15c9a0 2024-04-25T10:06:10.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.690+0000 7f19437fe640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f192c066f60 0x7f192c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.690+0000 7f194ad9d640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f192c066f60 0x7f192c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.690+0000 7f19437fe640 1 -- 172.21.15.133:0/326935536 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f193c11fbc0 con 0x7f194c15c9a0 2024-04-25T10:06:10.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.690+0000 7f194ad9d640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f192c066f60 0x7f192c069420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f194c154730 tx=0x7f1944008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.694+0000 7f19437fe640 1 -- 172.21.15.133:0/326935536 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f193c016560 con 0x7f194c15c9a0 2024-04-25T10:06:10.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.886+0000 7f19511a2640 1 -- 172.21.15.133:0/326935536 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f194c068ef0 con 0x7f194c15c9a0 2024-04-25T10:06:10.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.886+0000 7f19437fe640 1 -- 172.21.15.133:0/326935536 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f193c0f0ad0 con 0x7f194c15c9a0 2024-04-25T10:06:10.891 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:10.891 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:06:09.247920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:10.892 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 24 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 -- 172.21.15.133:0/326935536 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f192c066f60 msgr2=0x7f192c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f192c066f60 0x7f192c069420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f194c154730 tx=0x7f1944008040 comp rx=0 tx=0).stop 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 -- 172.21.15.133:0/326935536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f194c15c9a0 msgr2=0x7f194c149a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f194c15c9a0 0x7f194c149a20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f193c0b0040 tx=0x7f193c019710 comp rx=0 tx=0).stop 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 -- 172.21.15.133:0/326935536 shutdown_connections 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f192c066f60 0x7f192c069420 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f194c15c9a0 0x7f194c149a20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f194c0ce330 0x7f194c1494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 --2- 172.21.15.133:0/326935536 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f194c0b1f70 0x7f194c14ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 -- 172.21.15.133:0/326935536 >> 172.21.15.133:0/326935536 conn(0x7f194c067000 msgr2=0x7f194c154780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 -- 172.21.15.133:0/326935536 shutdown_connections 2024-04-25T10:06:10.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:10.890+0000 7f19511a2640 1 -- 172.21.15.133:0/326935536 wait complete. 2024-04-25T10:06:10.910 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T10:06:11.030 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:11.030 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:06:11.030 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:11.044 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:11.044 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:11.044 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T10:06:11.044 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T10:06:11.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:11.044 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T10:06:11.052 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-04-25T10:06:11.052 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:06:11.052 DEBUG:teuthology.orchestra.run.smithi133:> import stat 2024-04-25T10:06:11.052 DEBUG:teuthology.orchestra.run.smithi133:> import json 2024-04-25T10:06:11.052 DEBUG:teuthology.orchestra.run.smithi133:> import sys 2024-04-25T10:06:11.052 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:06:11.053 DEBUG:teuthology.orchestra.run.smithi133:> try: 2024-04-25T10:06:11.053 DEBUG:teuthology.orchestra.run.smithi133:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T10:06:11.053 DEBUG:teuthology.orchestra.run.smithi133:> except OSError as e: 2024-04-25T10:06:11.053 DEBUG:teuthology.orchestra.run.smithi133:> sys.exit(e.errno) 2024-04-25T10:06:11.053 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:06:11.053 DEBUG:teuthology.orchestra.run.smithi133:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:06:11.053 DEBUG:teuthology.orchestra.run.smithi133:> print(json.dumps( 2024-04-25T10:06:11.053 DEBUG:teuthology.orchestra.run.smithi133:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:06:11.053 DEBUG:teuthology.orchestra.run.smithi133:> indent=2)) 2024-04-25T10:06:11.053 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:06:11.159 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:11.159 INFO:teuthology.orchestra.run.smithi133.stdout: "st_mode": 16893, 2024-04-25T10:06:11.159 INFO:teuthology.orchestra.run.smithi133.stdout: "st_ino": 1099511627777, 2024-04-25T10:06:11.159 INFO:teuthology.orchestra.run.smithi133.stdout: "st_dev": 45, 2024-04-25T10:06:11.160 INFO:teuthology.orchestra.run.smithi133.stdout: "st_nlink": 16, 2024-04-25T10:06:11.160 INFO:teuthology.orchestra.run.smithi133.stdout: "st_uid": 1000, 2024-04-25T10:06:11.160 INFO:teuthology.orchestra.run.smithi133.stdout: "st_gid": 1271, 2024-04-25T10:06:11.160 INFO:teuthology.orchestra.run.smithi133.stdout: "st_size": 9171037, 2024-04-25T10:06:11.160 INFO:teuthology.orchestra.run.smithi133.stdout: "st_atime": 1714039549.1507015, 2024-04-25T10:06:11.160 INFO:teuthology.orchestra.run.smithi133.stdout: "st_mtime": 1714039563.2785697, 2024-04-25T10:06:11.160 INFO:teuthology.orchestra.run.smithi133.stdout: "st_ctime": 1714039563.2785697 2024-04-25T10:06:11.160 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:11.168 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T10:06:11.168 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T10:06:11.169 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b5a11e640 1 Processor -- start 2024-04-25T10:06:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b5a11e640 1 -- start start 2024-04-25T10:06:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b5a11e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b540ce330 0x7f6b540ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b5a11e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b540c61d0 con 0x7f6b540b1f70 2024-04-25T10:06:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b5a11e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b540c64b0 con 0x7f6b5415c9a0 2024-04-25T10:06:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b5a11e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b540cecc0 con 0x7f6b540ce330 2024-04-25T10:06:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b52ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6b540b1f70 0x7f6b5406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34496/0 (socket says 172.21.15.133:34496) 2024-04-25T10:06:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b52ffd640 1 -- 172.21.15.133:0/1913004325 learned_addr learned my addr 172.21.15.133:0/1913004325 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b53fff640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b540ce330 0x7f6b540ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248382194 0 0) 0x7f6b540c61d0 con 0x7f6b540b1f70 2024-04-25T10:06:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b40003530 con 0x7f6b540b1f70 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1988390791 0 0) 0x7f6b40003530 con 0x7f6b540b1f70 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b540ce330 msgr2=0x7f6b540ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b540ce330 0x7f6b540ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 >> v1:172.21.15.162:6789/0 conn(0x7f6b5415c9a0 legacy=0x7f6b540c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5406d4f0 con 0x7f6b540b1f70 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6b3c001680 con 0x7f6b540b1f70 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 >> v1:172.21.15.133:6789/0 conn(0x7f6b540b1f70 legacy=0x7f6b5406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b540ce330 0x7f6b400039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b40003f70 0x7f6b40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b40004d50 0x7f6b400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b540cecc0 con 0x7f6b40003f70 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4000bab0 con 0x7f6b40004d50 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4000bcb0 con 0x7f6b540ce330 2024-04-25T10:06:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b52ffd640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b40003f70 0x7f6b40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b53fff640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b40004d50 0x7f6b400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b537fe640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b540ce330 0x7f6b400039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b52ffd640 1 -- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b540ce330 msgr2=0x7f6b400039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b52ffd640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b540ce330 0x7f6b400039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b52ffd640 1 -- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b40004d50 msgr2=0x7f6b400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b52ffd640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b40004d50 0x7f6b400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b52ffd640 1 -- 172.21.15.133:0/1913004325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b40003530 con 0x7f6b40003f70 2024-04-25T10:06:11.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.330+0000 7f6b53fff640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b40004d50 0x7f6b400051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b52ffd640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b40003f70 0x7f6b40004810 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c240 tx=0x7f6b3c01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b429085d1336ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c007d20 con 0x7f6b40003f70 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c005ae0 con 0x7f6b40003f70 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b527fc640 1 -- 172.21.15.133:0/1913004325 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c007070 con 0x7f6b40003f70 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b5a11e640 1 -- 172.21.15.133:0/1913004325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b40003f70 msgr2=0x7f6b40004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b5a11e640 1 --2- 172.21.15.133:0/1913004325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b40003f70 0x7f6b40004810 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c240 tx=0x7f6b3c01f3f0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b53fff640 1 -- 172.21.15.133:0/1913004325 reap_dead start 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b5a11e640 1 -- 172.21.15.133:0/1913004325 shutdown_connections 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b5a11e640 1 -- 172.21.15.133:0/1913004325 >> 172.21.15.133:0/1913004325 conn(0x7f6b54067000 msgr2=0x7f6b5406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b5a11e640 1 -- 172.21.15.133:0/1913004325 shutdown_connections 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b5a11e640 1 -- 172.21.15.133:0/1913004325 wait complete. 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b5a11e640 1 Processor -- start 2024-04-25T10:06:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.334+0000 7f6b5a11e640 1 -- start start 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b5a11e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b40003f70 0x7f6b540b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b5a11e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b54154580 0x7f6b540cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b5a11e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b540c7ce0 0x7f6b540c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b5a11e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5415cf00 con 0x7f6b54154580 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b5a11e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5415cd80 con 0x7f6b540c7ce0 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b5a11e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b540ce780 con 0x7f6b40003f70 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b52ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b54154580 0x7f6b540cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b52ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b54154580 0x7f6b540cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40298/0 (socket says 172.21.15.133:40298) 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b52ffd640 1 -- 172.21.15.133:0/2282517004 learned_addr learned my addr 172.21.15.133:0/2282517004 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b537fe640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b40003f70 0x7f6b540b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b53fff640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b540c7ce0 0x7f6b540c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b52ffd640 1 -- 172.21.15.133:0/2282517004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b40003f70 msgr2=0x7f6b540b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b52ffd640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b40003f70 0x7f6b540b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b52ffd640 1 -- 172.21.15.133:0/2282517004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b540c7ce0 msgr2=0x7f6b540c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b52ffd640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b540c7ce0 0x7f6b540c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b52ffd640 1 -- 172.21.15.133:0/2282517004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b540cca70 con 0x7f6b54154580 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b53fff640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b540c7ce0 0x7f6b540c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b537fe640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b40003f70 0x7f6b540b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b52ffd640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b54154580 0x7f6b540cc120 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c240 tx=0x7f6b40005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b4bfff640 1 -- 172.21.15.133:0/2282517004 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c007d20 con 0x7f6b54154580 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b4bfff640 1 -- 172.21.15.133:0/2282517004 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c0047f0 con 0x7f6b54154580 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b5a11e640 1 -- 172.21.15.133:0/2282517004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b540ccd50 con 0x7f6b54154580 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b4bfff640 1 -- 172.21.15.133:0/2282517004 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c004bc0 con 0x7f6b54154580 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b5a11e640 1 -- 172.21.15.133:0/2282517004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b540cd310 con 0x7f6b54154580 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b5a11e640 1 -- 172.21.15.133:0/2282517004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b54068ef0 con 0x7f6b54154580 2024-04-25T10:06:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b4bfff640 1 -- 172.21.15.133:0/2282517004 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b3c007070 con 0x7f6b54154580 2024-04-25T10:06:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b4bfff640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b34066f90 0x7f6b34069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b4bfff640 1 -- 172.21.15.133:0/2282517004 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b3c122870 con 0x7f6b54154580 2024-04-25T10:06:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b537fe640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b34066f90 0x7f6b34069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.338+0000 7f6b537fe640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b34066f90 0x7f6b34069450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6b44000c10 tx=0x7f6b44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.342+0000 7f6b4bfff640 1 -- 172.21.15.133:0/2282517004 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b3c0c9080 con 0x7f6b54154580 2024-04-25T10:06:11.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.530+0000 7f6b5a11e640 1 -- 172.21.15.133:0/2282517004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b541549f0 con 0x7f6b54154580 2024-04-25T10:06:11.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b4bfff640 1 -- 172.21.15.133:0/2282517004 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f6b3c0b7070 con 0x7f6b54154580 2024-04-25T10:06:11.537 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:11.537 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:06:09.247920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:11.538 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 24 2024-04-25T10:06:11.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 -- 172.21.15.133:0/2282517004 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b34066f90 msgr2=0x7f6b34069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b34066f90 0x7f6b34069450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6b44000c10 tx=0x7f6b44009040 comp rx=0 tx=0).stop 2024-04-25T10:06:11.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 -- 172.21.15.133:0/2282517004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b54154580 msgr2=0x7f6b540cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b54154580 0x7f6b540cc120 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c240 tx=0x7f6b40005840 comp rx=0 tx=0).stop 2024-04-25T10:06:11.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 -- 172.21.15.133:0/2282517004 shutdown_connections 2024-04-25T10:06:11.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b540c7ce0 0x7f6b540c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b34066f90 0x7f6b34069450 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b54154580 0x7f6b540cc120 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 --2- 172.21.15.133:0/2282517004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b40003f70 0x7f6b540b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 -- 172.21.15.133:0/2282517004 >> 172.21.15.133:0/2282517004 conn(0x7f6b54067000 msgr2=0x7f6b5406dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 -- 172.21.15.133:0/2282517004 shutdown_connections 2024-04-25T10:06:11.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.534+0000 7f6b5a11e640 1 -- 172.21.15.133:0/2282517004 wait complete. 2024-04-25T10:06:11.555 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T10:06:11.677 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:11.677 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": -2 2024-04-25T10:06:11.677 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:11.691 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T10:06:11.691 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T10:06:11.691 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:11.691 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T10:06:11.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc4400a8640 1 Processor -- start 2024-04-25T10:06:11.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc4400a8640 1 -- start start 2024-04-25T10:06:11.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc4400a8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4380cd820 0x7fc4380cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc4400a8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc43815ced0 0x7fc43815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc4400a8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4380ce2e0 con 0x7fc438145170 2024-04-25T10:06:11.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc4400a8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43806db00 con 0x7fc4380cd820 2024-04-25T10:06:11.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc4400a8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc438145550 con 0x7fc43815ced0 2024-04-25T10:06:11.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc43d643640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4380cd820 0x7fc4380cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc43d643640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4380cd820 0x7fc4380cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40312/0 (socket says 172.21.15.133:40312) 2024-04-25T10:06:11.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc43d643640 1 -- 172.21.15.133:0/2488880407 learned_addr learned my addr 172.21.15.133:0/2488880407 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:11.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc43e645640 1 --2- 172.21.15.133:0/2488880407 >> v2:172.21.15.162:3300/0 conn(0x7fc43815ced0 0x7fc43815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc43d643640 1 -- 172.21.15.133:0/2488880407 >> v2:172.21.15.162:3300/0 conn(0x7fc43815ced0 msgr2=0x7fc43815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc43d643640 1 --2- 172.21.15.133:0/2488880407 >> v2:172.21.15.162:3300/0 conn(0x7fc43815ced0 0x7fc43815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc43d643640 1 -- 172.21.15.133:0/2488880407 >> v1:172.21.15.162:6789/0 conn(0x7fc438145170 legacy=0x7fc43806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.810+0000 7fc43d643640 1 -- 172.21.15.133:0/2488880407 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43806d4b0 con 0x7fc4380cd820 2024-04-25T10:06:11.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43d643640 1 --2- 172.21.15.133:0/2488880407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4380cd820 0x7fc4380cdca0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc4380b27f0 tx=0x7fc42801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff2d2493dce313c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43ce42640 1 -- 172.21.15.133:0/2488880407 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280b0070 con 0x7fc4380cd820 2024-04-25T10:06:11.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43ce42640 1 -- 172.21.15.133:0/2488880407 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4280ab600 con 0x7fc4380cd820 2024-04-25T10:06:11.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43ce42640 1 -- 172.21.15.133:0/2488880407 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280ab9d0 con 0x7fc4380cd820 2024-04-25T10:06:11.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 -- 172.21.15.133:0/2488880407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4380cd820 msgr2=0x7fc4380cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 --2- 172.21.15.133:0/2488880407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4380cd820 0x7fc4380cdca0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc4380b27f0 tx=0x7fc42801c520 comp rx=0 tx=0).stop 2024-04-25T10:06:11.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 -- 172.21.15.133:0/2488880407 shutdown_connections 2024-04-25T10:06:11.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 --2- 172.21.15.133:0/2488880407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4380cd820 0x7fc4380cdca0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 --2- 172.21.15.133:0/2488880407 >> v2:172.21.15.162:3300/0 conn(0x7fc43815ced0 0x7fc43815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 -- 172.21.15.133:0/2488880407 >> 172.21.15.133:0/2488880407 conn(0x7fc438066f70 msgr2=0x7fc43806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 -- 172.21.15.133:0/2488880407 shutdown_connections 2024-04-25T10:06:11.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 -- 172.21.15.133:0/2488880407 wait complete. 2024-04-25T10:06:11.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 Processor -- start 2024-04-25T10:06:11.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 -- start start 2024-04-25T10:06:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4380cd820 0x7fc438142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc438145170 0x7fc4381343b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc43815ced0 0x7fc438144450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc438168610 con 0x7fc438145170 2024-04-25T10:06:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc438168490 con 0x7fc43815ced0 2024-04-25T10:06:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc438168790 con 0x7fc4380cd820 2024-04-25T10:06:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43d643640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc438145170 0x7fc4381343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43d643640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc438145170 0x7fc4381343b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40324/0 (socket says 172.21.15.133:40324) 2024-04-25T10:06:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43d643640 1 -- 172.21.15.133:0/2118564861 learned_addr learned my addr 172.21.15.133:0/2118564861 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43de44640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4380cd820 0x7fc438142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43d643640 1 -- 172.21.15.133:0/2118564861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4380cd820 msgr2=0x7fc438142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43d643640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4380cd820 0x7fc438142c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43e645640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc43815ced0 0x7fc438144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43d643640 1 -- 172.21.15.133:0/2118564861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc43815ced0 msgr2=0x7fc438144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43d643640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc43815ced0 0x7fc438144450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43d643640 1 -- 172.21.15.133:0/2118564861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc438144990 con 0x7fc438145170 2024-04-25T10:06:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc43d643640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc438145170 0x7fc4381343b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fc42801ca00 tx=0x7fc4280192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc42e7fc640 1 -- 172.21.15.133:0/2118564861 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280b0040 con 0x7fc438145170 2024-04-25T10:06:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc42e7fc640 1 -- 172.21.15.133:0/2118564861 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4280b9070 con 0x7fc438145170 2024-04-25T10:06:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc4400a8640 1 -- 172.21.15.133:0/2118564861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4381433b0 con 0x7fc438145170 2024-04-25T10:06:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.814+0000 7fc42e7fc640 1 -- 172.21.15.133:0/2118564861 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280b4660 con 0x7fc438145170 2024-04-25T10:06:11.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc4400a8640 1 -- 172.21.15.133:0/2118564861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc438143970 con 0x7fc438145170 2024-04-25T10:06:11.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc42e7fc640 1 -- 172.21.15.133:0/2118564861 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4280b4800 con 0x7fc438145170 2024-04-25T10:06:11.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc42e7fc640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc41c066f90 0x7fc41c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc42e7fc640 1 -- 172.21.15.133:0/2118564861 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc42811ee40 con 0x7fc438145170 2024-04-25T10:06:11.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc43de44640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc41c066f90 0x7fc41c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc43de44640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc41c066f90 0x7fc41c069450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc434001650 tx=0x7fc434009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc443215000 1 Processor -- start 2024-04-25T10:06:11.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc443215000 1 -- start start 2024-04-25T10:06:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc443215000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x556706858d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc443215000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 0x55670680e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc443215000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670680ecd0 0x55670680f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc443215000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567066248a0 con 0x556706858970 2024-04-25T10:06:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc443215000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670665cd10 con 0x55670680ecd0 2024-04-25T10:06:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc443215000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567065e52d0 con 0x556706859290 2024-04-25T10:06:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc43de44640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x556706858d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc43de44640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x556706858d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40330/0 (socket says 172.21.15.133:40330) 2024-04-25T10:06:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.818+0000 7fc43de44640 1 -- 172.21.15.133:0/317963952 learned_addr learned my addr 172.21.15.133:0/317963952 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 -- 172.21.15.133:0/317963952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 msgr2=0x55670680e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43d643640 1 --2- 172.21.15.133:0/317963952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 0x55670680e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43e645640 1 --2- 172.21.15.133:0/317963952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670680ecd0 0x55670680f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 --2- 172.21.15.133:0/317963952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 0x55670680e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 -- 172.21.15.133:0/317963952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670680ecd0 msgr2=0x55670680f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 --2- 172.21.15.133:0/317963952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670680ecd0 0x55670680f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 -- 172.21.15.133:0/317963952 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567066a3870 con 0x556706858970 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43d643640 1 --2- 172.21.15.133:0/317963952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 0x55670680e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 --2- 172.21.15.133:0/317963952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x556706858d50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc4340052e0 tx=0x7fc434011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271393a2d0357fda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc41b7fe640 1 -- 172.21.15.133:0/317963952 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43400e530 con 0x556706858970 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc41b7fe640 1 -- 172.21.15.133:0/317963952 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43400e6d0 con 0x556706858970 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc41b7fe640 1 -- 172.21.15.133:0/317963952 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43400ea80 con 0x556706858970 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- 172.21.15.133:0/317963952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 msgr2=0x556706858d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 --2- 172.21.15.133:0/317963952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x556706858d50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc4340052e0 tx=0x7fc434011000 comp rx=0 tx=0).stop 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- 172.21.15.133:0/317963952 shutdown_connections 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 --2- 172.21.15.133:0/317963952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55670680ecd0 0x55670680f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 --2- 172.21.15.133:0/317963952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 0x55670680e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 --2- 172.21.15.133:0/317963952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x556706858d50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- 172.21.15.133:0/317963952 >> 172.21.15.133:0/317963952 conn(0x556706815540 msgr2=0x556706815920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- 172.21.15.133:0/317963952 shutdown_connections 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- 172.21.15.133:0/317963952 wait complete. 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- start start 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 Processor -- start 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55670680ecd0 0x55670685fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706858970 0x556706860210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 0x556706812100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556706617b50 con 0x55670680ecd0 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556706658db0 con 0x556706858970 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556706661340 con 0x556706859290 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 10 client.0 _ll_register_callbacks cb 0x556706710fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:11.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55670680ecd0 0x55670685fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55670680ecd0 0x55670685fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40334/0 (socket says 172.21.15.133:40334) 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 -- 172.21.15.133:0/4050487960 learned_addr learned my addr 172.21.15.133:0/4050487960 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43e645640 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 0x556706812100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 -- 172.21.15.133:0/4050487960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 msgr2=0x556706812100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43d643640 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706858970 0x556706860210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 0x556706812100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 -- 172.21.15.133:0/4050487960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706858970 msgr2=0x556706860210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706858970 0x556706860210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 -- 172.21.15.133:0/4050487960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567065e5520 con 0x55670680ecd0 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43e645640 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 0x556706812100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43de44640 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55670680ecd0 0x55670685fcd0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc434008fd0 tx=0x7fc434000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9367f2b84d42f594 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc418ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc418ff9640 1 -- 172.21.15.133:0/4050487960 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434021450 con 0x55670680ecd0 2024-04-25T10:06:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc418ff9640 1 -- 172.21.15.133:0/4050487960 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc4340215f0 con 0x55670680ecd0 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- 172.21.15.133:0/4050487960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc43400e6d0 con 0x55670680ecd0 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc418ff9640 1 -- 172.21.15.133:0/4050487960 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc434033050 con 0x55670680ecd0 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 10 client.6477 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- 172.21.15.133:0/4050487960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x556706657d00 con 0x55670680ecd0 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc418ff9640 1 -- 172.21.15.133:0/4050487960 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43400e6d0 con 0x55670680ecd0 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc418ff9640 1 -- 172.21.15.133:0/4050487960 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc434036020 con 0x55670680ecd0 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 10 client.6477 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 10 client.6477 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 20 client.6477 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556706812ea0 0x5567068132b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 4 client.6477 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc443215000 1 -- 172.21.15.133:0/4050487960 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567065f8b10 con 0x556706812ea0 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.822+0000 7fc43d643640 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556706812ea0 0x5567068132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc418ff9640 10 client.6477 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc43d643640 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556706812ea0 0x5567068132b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1694fb7343bd22b4 server_cookie=3d2c92857ddd6d23 in_seq=0 out_seq=0 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc418ff9640 1 -- 172.21.15.133:0/4050487960 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567065f8b10 con 0x556706812ea0 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc418ff9640 10 client.6477 handle_command_reply: tid=0 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 1 client.6477 shutdown 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 1 -- 172.21.15.133:0/4050487960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55670680ecd0 msgr2=0x55670685fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55670680ecd0 0x55670685fcd0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc434008fd0 tx=0x7fc434000bd0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 1 -- 172.21.15.133:0/4050487960 shutdown_connections 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x556706812ea0 0x5567068132b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556706859290 0x556706812100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706858970 0x556706860210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc418ff9640 0 client.6477 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 1 --2- 172.21.15.133:0/4050487960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55670680ecd0 0x55670685fcd0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 1 -- 172.21.15.133:0/4050487960 >> 172.21.15.133:0/4050487960 conn(0x556706815540 msgr2=0x55670680b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 1 -- 172.21.15.133:0/4050487960 shutdown_connections 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 1 -- 172.21.15.133:0/4050487960 wait complete. 2024-04-25T10:06:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.826+0000 7fc443215000 20 client.6477 trim_cache size 0 max 16384 2024-04-25T10:06:11.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.838+0000 7fc443215000 1 Processor -- start 2024-04-25T10:06:11.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.838+0000 7fc443215000 1 -- start start 2024-04-25T10:06:11.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55670680ecd0 0x5567068b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706859290 0x5567068b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc438201130 con 0x556706858970 2024-04-25T10:06:11.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc438200360 con 0x556706859290 2024-04-25T10:06:11.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc438138cf0 con 0x55670680ecd0 2024-04-25T10:06:11.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40346/0 (socket says 172.21.15.133:40346) 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 -- 172.21.15.133:0/1661434238 learned_addr learned my addr 172.21.15.133:0/1661434238 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 -- 172.21.15.133:0/1661434238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55670680ecd0 msgr2=0x5567068b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 --2- 172.21.15.133:0/1661434238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55670680ecd0 0x5567068b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 -- 172.21.15.133:0/1661434238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706859290 msgr2=0x5567068b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 --2- 172.21.15.133:0/1661434238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706859290 0x5567068b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 -- 172.21.15.133:0/1661434238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567068590c0 con 0x556706858970 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 --2- 172.21.15.133:0/1661434238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b50d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc4280ef550 tx=0x7fc4280eb910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac112548831ed9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc41affd640 1 -- 172.21.15.133:0/1661434238 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280f3710 con 0x556706858970 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc41affd640 1 -- 172.21.15.133:0/1661434238 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4280f7cd0 con 0x556706858970 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc41affd640 1 -- 172.21.15.133:0/1661434238 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280ed8c0 con 0x556706858970 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- 172.21.15.133:0/1661434238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 msgr2=0x5567068b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 --2- 172.21.15.133:0/1661434238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b50d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc4280ef550 tx=0x7fc4280eb910 comp rx=0 tx=0).stop 2024-04-25T10:06:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- 172.21.15.133:0/1661434238 shutdown_connections 2024-04-25T10:06:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 --2- 172.21.15.133:0/1661434238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706859290 0x5567068b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 --2- 172.21.15.133:0/1661434238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b50d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 --2- 172.21.15.133:0/1661434238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55670680ecd0 0x5567068b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- 172.21.15.133:0/1661434238 >> 172.21.15.133:0/1661434238 conn(0x556706815540 msgr2=0x556706813280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- 172.21.15.133:0/1661434238 shutdown_connections 2024-04-25T10:06:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- 172.21.15.133:0/1661434238 wait complete. 2024-04-25T10:06:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- start start 2024-04-25T10:06:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 Processor -- start 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55670680ecd0 0x5567068b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706859290 0x55670685cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556706620550 con 0x556706858970 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc438135af0 con 0x556706859290 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567066e0490 con 0x55670680ecd0 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 10 client.0 _ll_register_callbacks cb 0x7fc434001370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40356/0 (socket says 172.21.15.133:40356) 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 -- 172.21.15.133:0/903257965 learned_addr learned my addr 172.21.15.133:0/903257965 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 -- 172.21.15.133:0/903257965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55670680ecd0 msgr2=0x5567068b8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 --2- 172.21.15.133:0/903257965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55670680ecd0 0x5567068b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 -- 172.21.15.133:0/903257965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706859290 msgr2=0x55670685cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 --2- 172.21.15.133:0/903257965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706859290 0x55670685cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 -- 172.21.15.133:0/903257965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567066e0920 con 0x556706858970 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc43d643640 1 --2- 172.21.15.133:0/903257965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b90e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc42811d230 tx=0x7fc4280c0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4931d923aa874070 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc418ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc418ff9640 1 -- 172.21.15.133:0/903257965 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280f3710 con 0x556706858970 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc418ff9640 1 -- 172.21.15.133:0/903257965 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc428107040 con 0x556706858970 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- 172.21.15.133:0/903257965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4280f7cd0 con 0x556706858970 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc418ff9640 1 -- 172.21.15.133:0/903257965 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc428112050 con 0x556706858970 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 10 client.6483 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc443215000 1 -- 172.21.15.133:0/903257965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5567066b6a30 con 0x556706858970 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.842+0000 7fc418ff9640 1 -- 172.21.15.133:0/903257965 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4280f7cd0 con 0x556706858970 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc418ff9640 1 -- 172.21.15.133:0/903257965 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc4380b1ed0 con 0x556706858970 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 10 client.6483 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 10 client.6483 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 20 client.6483 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 --2- 172.21.15.133:0/903257965 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55670685d630 0x55670685da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 4 client.6483 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 -- 172.21.15.133:0/903257965 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55670666a250 con 0x55670685d630 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc43de44640 1 --2- 172.21.15.133:0/903257965 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55670685d630 0x55670685da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc418ff9640 10 client.6483 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc43de44640 1 --2- 172.21.15.133:0/903257965 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55670685d630 0x55670685da10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df88d109a0a4360 server_cookie=a9994837764601b3 in_seq=0 out_seq=0 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc418ff9640 1 -- 172.21.15.133:0/903257965 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55670666a250 con 0x55670685d630 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc418ff9640 10 client.6483 handle_command_reply: tid=0 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 client.6483 shutdown 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 -- 172.21.15.133:0/903257965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 msgr2=0x5567068b90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 --2- 172.21.15.133:0/903257965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b90e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc42811d230 tx=0x7fc4280c0040 comp rx=0 tx=0).stop 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 -- 172.21.15.133:0/903257965 shutdown_connections 2024-04-25T10:06:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 --2- 172.21.15.133:0/903257965 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55670685d630 0x55670685da10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 --2- 172.21.15.133:0/903257965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556706859290 0x55670685cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc418ff9640 0 client.6483 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 --2- 172.21.15.133:0/903257965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556706858970 0x5567068b90e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 --2- 172.21.15.133:0/903257965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55670680ecd0 0x5567068b8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 -- 172.21.15.133:0/903257965 >> 172.21.15.133:0/903257965 conn(0x556706815540 msgr2=0x556706813280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 -- 172.21.15.133:0/903257965 shutdown_connections 2024-04-25T10:06:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 1 -- 172.21.15.133:0/903257965 wait complete. 2024-04-25T10:06:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc443215000 20 client.6483 trim_cache size 0 max 16384 2024-04-25T10:06:11.855 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:11.856 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": -2 2024-04-25T10:06:11.856 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 -- 172.21.15.133:0/2118564861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc41c066f90 msgr2=0x7fc41c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc41c066f90 0x7fc41c069450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc434001650 tx=0x7fc434009040 comp rx=0 tx=0).stop 2024-04-25T10:06:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 -- 172.21.15.133:0/2118564861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc438145170 msgr2=0x7fc4381343b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc438145170 0x7fc4381343b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fc42801ca00 tx=0x7fc4280192c0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 -- 172.21.15.133:0/2118564861 shutdown_connections 2024-04-25T10:06:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc43815ced0 0x7fc438144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc41c066f90 0x7fc41c069450 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc438145170 0x7fc4381343b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 --2- 172.21.15.133:0/2118564861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4380cd820 0x7fc438142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 -- 172.21.15.133:0/2118564861 >> 172.21.15.133:0/2118564861 conn(0x7fc438066f70 msgr2=0x7fc438140f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 -- 172.21.15.133:0/2118564861 shutdown_connections 2024-04-25T10:06:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.846+0000 7fc4400a8640 1 -- 172.21.15.133:0/2118564861 wait complete. 2024-04-25T10:06:11.865 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T10:06:11.865 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:06:11.865 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:11.866 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:06:11.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53eb18e640 1 Processor -- start 2024-04-25T10:06:11.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53eb18e640 1 -- start start 2024-04-25T10:06:11.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53eb18e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 0x7f53e40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53eb18e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53e406eac0 0x7f53e40b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53eb18e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e415ac50 con 0x7f53e40b2150 2024-04-25T10:06:11.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53eb18e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40cd180 con 0x7f53e406eac0 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53eb18e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e415aed0 con 0x7f53e4143ea0 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53e972b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53e406eac0 0x7f53e40b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53dbfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f53e40b2150 0x7f53e406e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34498/0 (socket says 172.21.15.133:34498) 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53dbfff640 1 -- 172.21.15.133:0/3447628321 learned_addr learned my addr 172.21.15.133:0/3447628321 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53e972b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53e406eac0 0x7f53e40b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40360/0 (socket says 172.21.15.133:40360) 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/3447628321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 0x7f53e40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53e972b640 1 -- 172.21.15.133:0/3447628321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 msgr2=0x7f53e40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53e972b640 1 --2- 172.21.15.133:0/3447628321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 0x7f53e40cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53e972b640 1 -- 172.21.15.133:0/3447628321 >> v1:172.21.15.133:6789/0 conn(0x7f53e40b2150 legacy=0x7f53e406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53e972b640 1 -- 172.21.15.133:0/3447628321 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e415b1b0 con 0x7f53e406eac0 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53db7fe640 1 -- 172.21.15.133:0/3447628321 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263524069 0 0) 0x7f53d4001120 con 0x7f53e40b2150 2024-04-25T10:06:11.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.966+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/3447628321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 0x7f53e40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53e972b640 1 --2- 172.21.15.133:0/3447628321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e406eac0 0x7f53e40b06d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5d6137d1c1441a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53db7fe640 1 -- 172.21.15.133:0/3447628321 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0b2070 con 0x7f53e406eac0 2024-04-25T10:06:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53db7fe640 1 -- 172.21.15.133:0/3447628321 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc0a5940 con 0x7f53e406eac0 2024-04-25T10:06:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53db7fe640 1 -- 172.21.15.133:0/3447628321 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0a5d10 con 0x7f53e406eac0 2024-04-25T10:06:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53eb18e640 1 -- 172.21.15.133:0/3447628321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e406eac0 msgr2=0x7f53e40b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53eb18e640 1 --2- 172.21.15.133:0/3447628321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e406eac0 0x7f53e40b06d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc01c970 comp rx=0 tx=0).stop 2024-04-25T10:06:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53eb18e640 1 -- 172.21.15.133:0/3447628321 shutdown_connections 2024-04-25T10:06:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53eb18e640 1 --2- 172.21.15.133:0/3447628321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e406eac0 0x7f53e40b06d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53eb18e640 1 --2- 172.21.15.133:0/3447628321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 0x7f53e40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53eb18e640 1 -- 172.21.15.133:0/3447628321 >> 172.21.15.133:0/3447628321 conn(0x7f53e4066f70 msgr2=0x7f53e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53eb18e640 1 -- 172.21.15.133:0/3447628321 shutdown_connections 2024-04-25T10:06:11.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53eb18e640 1 -- 172.21.15.133:0/3447628321 wait complete. 2024-04-25T10:06:11.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53eb18e640 1 Processor -- start 2024-04-25T10:06:11.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.970+0000 7f53eb18e640 1 -- start start 2024-04-25T10:06:11.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53eb18e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e406eac0 0x7f53e4166de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53eb18e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53e40b2150 0x7f53e413f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53eb18e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 0x7f53e41639e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53eb18e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4165540 con 0x7f53e406eac0 2024-04-25T10:06:11.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53eb18e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e41653c0 con 0x7f53e40b2150 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53eb18e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e41656c0 con 0x7f53e4143ea0 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53e8f2a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e406eac0 0x7f53e4166de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53e8f2a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e406eac0 0x7f53e4166de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40374/0 (socket says 172.21.15.133:40374) 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53e8f2a640 1 -- 172.21.15.133:0/2690977105 learned_addr learned my addr 172.21.15.133:0/2690977105 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53e972b640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 0x7f53e41639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53dbfff640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53e40b2150 0x7f53e413f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53e8f2a640 1 -- 172.21.15.133:0/2690977105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 msgr2=0x7f53e41639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 0x7f53e41639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53e8f2a640 1 -- 172.21.15.133:0/2690977105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53e40b2150 msgr2=0x7f53e413f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53e40b2150 0x7f53e413f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53e8f2a640 1 -- 172.21.15.133:0/2690977105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e4167dd0 con 0x7f53e406eac0 2024-04-25T10:06:11.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53e972b640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 0x7f53e41639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e406eac0 0x7f53e4166de0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c820 tx=0x7f53e000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53d97fa640 1 -- 172.21.15.133:0/2690977105 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0015070 con 0x7f53e406eac0 2024-04-25T10:06:11.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53d97fa640 1 -- 172.21.15.133:0/2690977105 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0007a30 con 0x7f53e406eac0 2024-04-25T10:06:11.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53d97fa640 1 -- 172.21.15.133:0/2690977105 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0007e40 con 0x7f53e406eac0 2024-04-25T10:06:11.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53eb18e640 1 -- 172.21.15.133:0/2690977105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e41680b0 con 0x7f53e406eac0 2024-04-25T10:06:11.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53eb18e640 1 -- 172.21.15.133:0/2690977105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e413ecb0 con 0x7f53e406eac0 2024-04-25T10:06:11.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53d97fa640 1 -- 172.21.15.133:0/2690977105 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53e0018070 con 0x7f53e406eac0 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53d97fa640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53d0066f60 0x7f53d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53d97fa640 1 -- 172.21.15.133:0/2690977105 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53e00850f0 con 0x7f53e406eac0 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.974+0000 7f53dbfff640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53d0066f60 0x7f53d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53dbfff640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53d0066f60 0x7f53d0069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f53e4142610 tx=0x7f53d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 Processor -- start 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- start start 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537eeaa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 0x56537ee60af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56537ecaedb0 con 0x56537eeaa7c0 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56537ec37230 con 0x56537ee60710 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56537ecaae50 con 0x56537eea9ea0 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53dbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53dbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40390/0 (socket says 172.21.15.133:40390) 2024-04-25T10:06:11.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53dbfff640 1 -- 172.21.15.133:0/599271063 learned_addr learned my addr 172.21.15.133:0/599271063 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/599271063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537eeaa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53e972b640 1 --2- 172.21.15.133:0/599271063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 0x56537ee60af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53dbfff640 1 -- 172.21.15.133:0/599271063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 msgr2=0x56537eeaa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53dbfff640 1 --2- 172.21.15.133:0/599271063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537eeaa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53dbfff640 1 -- 172.21.15.133:0/599271063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 msgr2=0x56537ee60af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53dbfff640 1 --2- 172.21.15.133:0/599271063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 0x56537ee60af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53dbfff640 1 -- 172.21.15.133:0/599271063 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56537ec70750 con 0x56537eeaa7c0 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53e972b640 1 --2- 172.21.15.133:0/599271063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 0x56537ee60af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53dbfff640 1 --2- 172.21.15.133:0/599271063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee601d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f53d40071d0 tx=0x7f53d4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b4c98b1364bc1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ca7fc640 1 -- 172.21.15.133:0/599271063 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400d9a0 con 0x56537eeaa7c0 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ca7fc640 1 -- 172.21.15.133:0/599271063 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d400db40 con 0x56537eeaa7c0 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ca7fc640 1 -- 172.21.15.133:0/599271063 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d4020450 con 0x56537eeaa7c0 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- 172.21.15.133:0/599271063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 msgr2=0x56537ee601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/599271063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee601d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f53d40071d0 tx=0x7f53d4010000 comp rx=0 tx=0).stop 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- 172.21.15.133:0/599271063 shutdown_connections 2024-04-25T10:06:11.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/599271063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 0x56537ee60af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/599271063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee601d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/599271063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537eeaa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- 172.21.15.133:0/599271063 >> 172.21.15.133:0/599271063 conn(0x56537ee5c6c0 msgr2=0x56537ee5eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- 172.21.15.133:0/599271063 shutdown_connections 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- 172.21.15.133:0/599271063 wait complete. 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- start start 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 Processor -- start 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 0x56537ee8eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537ee8f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee8f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56537ec69bf0 con 0x56537eeaa7c0 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56537ec63d40 con 0x56537ee60710 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.978+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56537ec33990 con 0x56537eea9ea0 2024-04-25T10:06:11.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 10 client.0 _ll_register_callbacks cb 0x56537ee4b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e972b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee8f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e972b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee8f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40398/0 (socket says 172.21.15.133:40398) 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e972b640 1 -- 172.21.15.133:0/2424647373 learned_addr learned my addr 172.21.15.133:0/2424647373 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 0x56537ee8eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53dbfff640 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537ee8f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e972b640 1 -- 172.21.15.133:0/2424647373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 msgr2=0x56537ee8f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e972b640 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537ee8f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e972b640 1 -- 172.21.15.133:0/2424647373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 msgr2=0x56537ee8eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e972b640 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 0x56537ee8eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e972b640 1 -- 172.21.15.133:0/2424647373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56537ec51640 con 0x56537eeaa7c0 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 0x56537ee8eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e972b640 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee8f950 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f53dc01b020 tx=0x7f53dc01b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb8f19651172e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:11.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53c37fe640 1 -- 172.21.15.133:0/2424647373 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0bd070 con 0x56537eeaa7c0 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53c37fe640 1 -- 172.21.15.133:0/2424647373 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53dc0b8440 con 0x56537eeaa7c0 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 -- 172.21.15.133:0/2424647373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53d400db40 con 0x56537eeaa7c0 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53c37fe640 1 -- 172.21.15.133:0/2424647373 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53dc0c7050 con 0x56537eeaa7c0 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 10 client.6498 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 -- 172.21.15.133:0/2424647373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56537ec37480 con 0x56537eeaa7c0 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53c37fe640 1 -- 172.21.15.133:0/2424647373 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d400db40 con 0x56537eeaa7c0 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53c37fe640 1 -- 172.21.15.133:0/2424647373 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53dc0b8660 con 0x56537eeaa7c0 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 10 client.6498 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 10 client.6498 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 20 client.6498 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56537eeb9f40 0x56537eebc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 4 client.6498 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 -- 172.21.15.133:0/2424647373 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56537ec4abb0 con 0x56537eeb9f40 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56537eeb9f40 0x56537eebc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53c37fe640 10 client.6498 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56537eeb9f40 0x56537eebc330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc95f82fc166998 server_cookie=722508c47232d3a1 in_seq=0 out_seq=0 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53c37fe640 1 -- 172.21.15.133:0/2424647373 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56537ec4abb0 con 0x56537eeb9f40 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53c37fe640 10 client.6498 handle_command_reply: tid=0 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 client.6498 shutdown 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 -- 172.21.15.133:0/2424647373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 msgr2=0x56537ee8f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee8f950 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f53dc01b020 tx=0x7f53dc01b1b0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 -- 172.21.15.133:0/2424647373 shutdown_connections 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56537eeb9f40 0x56537eebc330 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537eeaa7c0 0x56537ee8f950 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537ee8f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53c37fe640 0 client.6498 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/2424647373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537ee60710 0x56537ee8eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 -- 172.21.15.133:0/2424647373 >> 172.21.15.133:0/2424647373 conn(0x56537ee5c6c0 msgr2=0x56537ee5f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 -- 172.21.15.133:0/2424647373 shutdown_connections 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 1 -- 172.21.15.133:0/2424647373 wait complete. 2024-04-25T10:06:11.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.982+0000 7f53ee2fb000 20 client.6498 trim_cache size 0 max 16384 2024-04-25T10:06:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 Processor -- start 2024-04-25T10:06:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 -- start start 2024-04-25T10:06:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537ef069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537eea9ea0 0x56537ef06f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eeaa7c0 0x56537ef07460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4203940 con 0x56537ee60710 2024-04-25T10:06:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4202b40 con 0x56537eea9ea0 2024-04-25T10:06:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e41fb300 con 0x56537eeaa7c0 2024-04-25T10:06:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53e8f2a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537ef069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53e8f2a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537ef069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40400/0 (socket says 172.21.15.133:40400) 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53e8f2a640 1 -- 172.21.15.133:0/1946761357 learned_addr learned my addr 172.21.15.133:0/1946761357 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53e8f2a640 1 -- 172.21.15.133:0/1946761357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eeaa7c0 msgr2=0x56537ef07460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/1946761357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eeaa7c0 0x56537ef07460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53e8f2a640 1 -- 172.21.15.133:0/1946761357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537eea9ea0 msgr2=0x56537ef06f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/1946761357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537eea9ea0 0x56537ef06f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53e8f2a640 1 -- 172.21.15.133:0/1946761357 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56537ed30ef0 con 0x56537ee60710 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/1946761357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537ef069e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f53e0007410 tx=0x7f53e0085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb87e79cc0e0fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53c9ffb640 1 -- 172.21.15.133:0/1946761357 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0053910 con 0x56537ee60710 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53c9ffb640 1 -- 172.21.15.133:0/1946761357 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0053b70 con 0x56537ee60710 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53c9ffb640 1 -- 172.21.15.133:0/1946761357 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0056ba0 con 0x56537ee60710 2024-04-25T10:06:12.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1946761357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 msgr2=0x56537ef069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/1946761357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537ef069e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f53e0007410 tx=0x7f53e0085040 comp rx=0 tx=0).stop 2024-04-25T10:06:12.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1946761357 shutdown_connections 2024-04-25T10:06:12.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/1946761357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eeaa7c0 0x56537ef07460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/1946761357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537eea9ea0 0x56537ef06f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/1946761357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537ef069e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1946761357 >> 172.21.15.133:0/1946761357 conn(0x56537ee5c6c0 msgr2=0x56537eeb7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1946761357 shutdown_connections 2024-04-25T10:06:12.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1946761357 wait complete. 2024-04-25T10:06:12.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 -- start start 2024-04-25T10:06:12.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:11.998+0000 7f53ee2fb000 1 Processor -- start 2024-04-25T10:06:12.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537eebc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537eebcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537eeaa7c0 0x56537eebd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56537ec69bf0 con 0x56537ee60710 2024-04-25T10:06:12.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4143260 con 0x56537eeaa7c0 2024-04-25T10:06:12.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56537ee667e0 con 0x56537eea9ea0 2024-04-25T10:06:12.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 10 client.0 _ll_register_callbacks cb 0x7f53d4007a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:12.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53e8f2a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537eebc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53e8f2a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537eebc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40408/0 (socket says 172.21.15.133:40408) 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53e8f2a640 1 -- 172.21.15.133:0/1845103298 learned_addr learned my addr 172.21.15.133:0/1845103298 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53e972b640 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537eeaa7c0 0x56537eebd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53e8f2a640 1 -- 172.21.15.133:0/1845103298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 msgr2=0x56537eebcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53dbfff640 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537eebcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537eebcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53e8f2a640 1 -- 172.21.15.133:0/1845103298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537eeaa7c0 msgr2=0x56537eebd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537eeaa7c0 0x56537eebd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53e8f2a640 1 -- 172.21.15.133:0/1845103298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56537ec6e9a0 con 0x56537ee60710 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53e972b640 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537eeaa7c0 0x56537eebd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53e8f2a640 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537eebc640 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f53e00834e0 tx=0x7f53e0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5165390053bcca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53c37fe640 1 -- 172.21.15.133:0/1845103298 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0080070 con 0x56537ee60710 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53c37fe640 1 -- 172.21.15.133:0/1845103298 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53e0051ee0 con 0x56537ee60710 2024-04-25T10:06:12.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1845103298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53e0053b70 con 0x56537ee60710 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53c37fe640 1 -- 172.21.15.133:0/1845103298 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e0056ba0 con 0x56537ee60710 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 10 client.6504 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1845103298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56537e96b5a0 con 0x56537ee60710 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53c37fe640 1 -- 172.21.15.133:0/1845103298 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0053b70 con 0x56537ee60710 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53c37fe640 1 -- 172.21.15.133:0/1845103298 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53e0054cc0 con 0x56537ee60710 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 10 client.6504 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 10 client.6504 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 20 client.6504 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56537ef07860 0x56537eebd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 4 client.6504 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1845103298 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x56537ec5e080 con 0x56537ef07860 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53dbfff640 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56537ef07860 0x56537eebd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53dbfff640 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56537ef07860 0x56537eebd600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a893d88862951e3 server_cookie=dfcab653ebe92653 in_seq=0 out_seq=0 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53c37fe640 10 client.6504 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53c37fe640 1 -- 172.21.15.133:0/1845103298 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56537ec5e080 con 0x56537ef07860 2024-04-25T10:06:12.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53c37fe640 10 client.6504 handle_command_reply: tid=0 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 client.6504 shutdown 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1845103298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 msgr2=0x56537eebc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537eebc640 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f53e00834e0 tx=0x7f53e0013040 comp rx=0 tx=0).stop 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1845103298 shutdown_connections 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x56537ef07860 0x56537eebd600 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56537eeaa7c0 0x56537eebd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53c37fe640 0 client.6504 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56537eea9ea0 0x56537eebcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 --2- 172.21.15.133:0/1845103298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56537ee60710 0x56537eebc640 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1845103298 >> 172.21.15.133:0/1845103298 conn(0x56537ee5c6c0 msgr2=0x56537eea87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1845103298 shutdown_connections 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 1 -- 172.21.15.133:0/1845103298 wait complete. 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53ee2fb000 20 client.6504 trim_cache size 0 max 16384 2024-04-25T10:06:12.012 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:12.013 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:06:12.013 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "a207c01c-30cd-422e-853f-21e8efe3dfa8", 2024-04-25T10:06:12.013 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:06:12.013 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:12.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 -- 172.21.15.133:0/2690977105 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53d0066f60 msgr2=0x7f53d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53d0066f60 0x7f53d0069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f53e4142610 tx=0x7f53d4009040 comp rx=0 tx=0).stop 2024-04-25T10:06:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 -- 172.21.15.133:0/2690977105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e406eac0 msgr2=0x7f53e4166de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e406eac0 0x7f53e4166de0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c820 tx=0x7f53e000cce0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 -- 172.21.15.133:0/2690977105 shutdown_connections 2024-04-25T10:06:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53e4143ea0 0x7f53e41639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f53e40b2150 0x7f53e413f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53d0066f60 0x7f53d0069420 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 --2- 172.21.15.133:0/2690977105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e406eac0 0x7f53e4166de0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 -- 172.21.15.133:0/2690977105 >> 172.21.15.133:0/2690977105 conn(0x7f53e4066f70 msgr2=0x7f53e406d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 -- 172.21.15.133:0/2690977105 shutdown_connections 2024-04-25T10:06:12.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.002+0000 7f53eb18e640 1 -- 172.21.15.133:0/2690977105 wait complete. 2024-04-25T10:06:12.022 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'a207c01c-30cd-422e-853f-21e8efe3dfa8', 'mode': 'asynchronous'}' 2024-04-25T10:06:12.022 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:06:12.023 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:12.023 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:06:12.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe6900f3640 1 Processor -- start 2024-04-25T10:06:12.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe6900f3640 1 -- start start 2024-04-25T10:06:12.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe6900f3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe68815de60 0x7fe68815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe6900f3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68815e300 con 0x7fe68815da80 2024-04-25T10:06:12.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe6900f3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6880cabb0 con 0x7fe68815f000 2024-04-25T10:06:12.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe6900f3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6880cae90 con 0x7fe68815de60 2024-04-25T10:06:12.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68de8f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe68815da80 0x7fe6880ca360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34502/0 (socket says 172.21.15.133:34502) 2024-04-25T10:06:12.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68de8f640 1 -- 172.21.15.133:0/1660155963 learned_addr learned my addr 172.21.15.133:0/1660155963 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:12.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68d68e640 1 --2- 172.21.15.133:0/1660155963 >> v2:172.21.15.162:3300/0 conn(0x7fe68815de60 0x7fe68815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108831665 0 0) 0x7fe68815e300 con 0x7fe68815da80 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe674003530 con 0x7fe68815da80 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3542485099 0 0) 0x7fe674003530 con 0x7fe68815da80 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 >> v2:172.21.15.162:3300/0 conn(0x7fe68815de60 msgr2=0x7fe68815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 --2- 172.21.15.133:0/1660155963 >> v2:172.21.15.162:3300/0 conn(0x7fe68815de60 0x7fe68815eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 >> v1:172.21.15.162:6789/0 conn(0x7fe68815f000 legacy=0x7fe68806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880cb170 con 0x7fe68815da80 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe6840016a0 con 0x7fe68815da80 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 >> v1:172.21.15.133:6789/0 conn(0x7fe68815da80 legacy=0x7fe6880ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 --2- 172.21.15.133:0/1660155963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6740039c0 0x7fe674003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 --2- 172.21.15.133:0/1660155963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe674004810 0x7fe674004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 --2- 172.21.15.133:0/1660155963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6740051d0 0x7fe67400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880cae90 con 0x7fe6740051d0 2024-04-25T10:06:12.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400c070 con 0x7fe674004810 2024-04-25T10:06:12.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400c270 con 0x7fe6740039c0 2024-04-25T10:06:12.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68e690640 1 --2- 172.21.15.133:0/1660155963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6740051d0 0x7fe67400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68d68e640 1 --2- 172.21.15.133:0/1660155963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe674004810 0x7fe674004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68e690640 1 -- 172.21.15.133:0/1660155963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6740039c0 msgr2=0x7fe674003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68e690640 1 --2- 172.21.15.133:0/1660155963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6740039c0 0x7fe674003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68e690640 1 -- 172.21.15.133:0/1660155963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe674004810 msgr2=0x7fe674004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68e690640 1 --2- 172.21.15.133:0/1660155963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe674004810 0x7fe674004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.142+0000 7fe68e690640 1 -- 172.21.15.133:0/1660155963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe674003530 con 0x7fe6740051d0 2024-04-25T10:06:12.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68e690640 1 -- 172.21.15.133:0/1660155963 reap_dead start 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68e690640 1 --2- 172.21.15.133:0/1660155963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6740051d0 0x7fe67400b9d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe680001a50 tx=0x7fe68001e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb860343a86c3fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680001a90 con 0x7fe6740051d0 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe680003b80 con 0x7fe6740051d0 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68ce8d640 1 -- 172.21.15.133:0/1660155963 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6800050d0 con 0x7fe6740051d0 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 -- 172.21.15.133:0/1660155963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6740051d0 msgr2=0x7fe67400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 --2- 172.21.15.133:0/1660155963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6740051d0 0x7fe67400b9d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe680001a50 tx=0x7fe68001e970 comp rx=0 tx=0).stop 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 -- 172.21.15.133:0/1660155963 shutdown_connections 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 --2- 172.21.15.133:0/1660155963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6740051d0 0x7fe67400b9d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 -- 172.21.15.133:0/1660155963 >> 172.21.15.133:0/1660155963 conn(0x7fe6880cc790 msgr2=0x7fe6880ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 -- 172.21.15.133:0/1660155963 shutdown_connections 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 -- 172.21.15.133:0/1660155963 wait complete. 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 Processor -- start 2024-04-25T10:06:12.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 -- start start 2024-04-25T10:06:12.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6740051d0 0x7fe688146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6880af930 0x7fe68809d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6880b4e60 0x7fe6880b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880cf750 con 0x7fe6880af930 2024-04-25T10:06:12.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880cf5d0 con 0x7fe6880b4e60 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe6900f3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880cf8d0 con 0x7fe6740051d0 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68d68e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6880af930 0x7fe68809d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68d68e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6880af930 0x7fe68809d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40422/0 (socket says 172.21.15.133:40422) 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68d68e640 1 -- 172.21.15.133:0/58805425 learned_addr learned my addr 172.21.15.133:0/58805425 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68de8f640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6740051d0 0x7fe688146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68e690640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6880b4e60 0x7fe6880b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68d68e640 1 -- 172.21.15.133:0/58805425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6740051d0 msgr2=0x7fe688146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68d68e640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6740051d0 0x7fe688146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68d68e640 1 -- 172.21.15.133:0/58805425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6880b4e60 msgr2=0x7fe6880b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68d68e640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6880b4e60 0x7fe6880b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.146+0000 7fe68d68e640 1 -- 172.21.15.133:0/58805425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880b00f0 con 0x7fe6880af930 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe68d68e640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6880af930 0x7fe68809d1c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe678011560 tx=0x7fe678011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe67e7fc640 1 -- 172.21.15.133:0/58805425 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678015930 con 0x7fe6880af930 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe67e7fc640 1 -- 172.21.15.133:0/58805425 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe678015ad0 con 0x7fe6880af930 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe6900f3640 1 -- 172.21.15.133:0/58805425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6880b03d0 con 0x7fe6880af930 2024-04-25T10:06:12.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe67e7fc640 1 -- 172.21.15.133:0/58805425 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800f460 con 0x7fe6880af930 2024-04-25T10:06:12.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe6900f3640 1 -- 172.21.15.133:0/58805425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6880b0990 con 0x7fe6880af930 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe67e7fc640 1 -- 172.21.15.133:0/58805425 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe67800f600 con 0x7fe6880af930 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe67e7fc640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe66c066f60 0x7fe66c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe67e7fc640 1 -- 172.21.15.133:0/58805425 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe678085a70 con 0x7fe6880af930 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe68de8f640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe66c066f60 0x7fe66c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe68de8f640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe66c066f60 0x7fe66c069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe684001890 tx=0x7fe6840016a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe693260000 1 Processor -- start 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe693260000 1 -- start start 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe693260000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe693260000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b50b20 0x561bb3b34a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe693260000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b34f60 0x561bb3b35340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe693260000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb39ed4a0 con 0x561bb3b50200 2024-04-25T10:06:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe693260000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb3955210 con 0x561bb3b34f60 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe693260000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561bb38dd980 con 0x561bb3b50b20 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe68de8f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe68de8f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40432/0 (socket says 172.21.15.133:40432) 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.150+0000 7fe68de8f640 1 -- 172.21.15.133:0/4069281551 learned_addr learned my addr 172.21.15.133:0/4069281551 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68d68e640 1 --2- 172.21.15.133:0/4069281551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b50b20 0x561bb3b34a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68de8f640 1 -- 172.21.15.133:0/4069281551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b50b20 msgr2=0x561bb3b34a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68de8f640 1 --2- 172.21.15.133:0/4069281551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b50b20 0x561bb3b34a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68de8f640 1 -- 172.21.15.133:0/4069281551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b34f60 msgr2=0x561bb3b35340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68de8f640 1 --2- 172.21.15.133:0/4069281551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b34f60 0x561bb3b35340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68de8f640 1 -- 172.21.15.133:0/4069281551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bb3950200 con 0x561bb3b50200 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68d68e640 1 --2- 172.21.15.133:0/4069281551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b50b20 0x561bb3b34a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68de8f640 1 --2- 172.21.15.133:0/4069281551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b505e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe684004a70 tx=0x7fe684011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9f5a63db520f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe66b7fe640 1 -- 172.21.15.133:0/4069281551 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684005260 con 0x561bb3b50200 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe66b7fe640 1 -- 172.21.15.133:0/4069281551 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe684005400 con 0x561bb3b50200 2024-04-25T10:06:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe66b7fe640 1 -- 172.21.15.133:0/4069281551 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684021450 con 0x561bb3b50200 2024-04-25T10:06:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- 172.21.15.133:0/4069281551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 msgr2=0x561bb3b505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 --2- 172.21.15.133:0/4069281551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b505e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe684004a70 tx=0x7fe684011000 comp rx=0 tx=0).stop 2024-04-25T10:06:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- 172.21.15.133:0/4069281551 shutdown_connections 2024-04-25T10:06:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 --2- 172.21.15.133:0/4069281551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b34f60 0x561bb3b35340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 --2- 172.21.15.133:0/4069281551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b50b20 0x561bb3b34a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 --2- 172.21.15.133:0/4069281551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b505e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- 172.21.15.133:0/4069281551 >> 172.21.15.133:0/4069281551 conn(0x561bb3afd4f0 msgr2=0x561bb3b0ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- 172.21.15.133:0/4069281551 shutdown_connections 2024-04-25T10:06:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- 172.21.15.133:0/4069281551 wait complete. 2024-04-25T10:06:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- start start 2024-04-25T10:06:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 Processor -- start 2024-04-25T10:06:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b34f60 0x561bb3b57980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b50200 0x561bb3b57ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b35340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb390fd80 con 0x561bb3b50b20 2024-04-25T10:06:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb39512b0 con 0x561bb3b50200 2024-04-25T10:06:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561bb3916830 con 0x561bb3b34f60 2024-04-25T10:06:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 10 client.0 _ll_register_callbacks cb 0x561bb3a08070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68e690640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b35340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68e690640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b35340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40448/0 (socket says 172.21.15.133:40448) 2024-04-25T10:06:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68e690640 1 -- 172.21.15.133:0/2006713722 learned_addr learned my addr 172.21.15.133:0/2006713722 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68de8f640 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b34f60 0x561bb3b57980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68e690640 1 -- 172.21.15.133:0/2006713722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b34f60 msgr2=0x561bb3b57980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68d68e640 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b50200 0x561bb3b57ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68e690640 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b34f60 0x561bb3b57980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68e690640 1 -- 172.21.15.133:0/2006713722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b50200 msgr2=0x561bb3b57ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68e690640 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b50200 0x561bb3b57ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68e690640 1 -- 172.21.15.133:0/2006713722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bb3921130 con 0x561bb3b50b20 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68e690640 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b35340 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe68001f210 tx=0x7fe68001b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c6a891b5b2e94f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe668ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe668ff9640 1 -- 172.21.15.133:0/2006713722 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680003e60 con 0x561bb3b50b20 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe668ff9640 1 -- 172.21.15.133:0/2006713722 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe680006d60 con 0x561bb3b50b20 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- 172.21.15.133:0/2006713722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe684005400 con 0x561bb3b50b20 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe668ff9640 1 -- 172.21.15.133:0/2006713722 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6800050d0 con 0x561bb3b50b20 2024-04-25T10:06:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 10 client.6516 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- 172.21.15.133:0/2006713722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x561bb39d6490 con 0x561bb3b50b20 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe668ff9640 1 -- 172.21.15.133:0/2006713722 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe684005400 con 0x561bb3b50b20 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe668ff9640 1 -- 172.21.15.133:0/2006713722 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe680005270 con 0x561bb3b50b20 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 10 client.6516 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 10 client.6516 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 20 client.6516 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x561bb3b360b0 0x561bb3b364c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 4 client.6516 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe693260000 1 -- 172.21.15.133:0/2006713722 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bb38f1020 con 0x561bb3b360b0 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.154+0000 7fe68de8f640 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x561bb3b360b0 0x561bb3b364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe668ff9640 10 client.6516 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe68de8f640 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x561bb3b360b0 0x561bb3b364c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39926c7191d22ba9 server_cookie=f47371463c599ec3 in_seq=0 out_seq=0 2024-04-25T10:06:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe668ff9640 1 -- 172.21.15.133:0/2006713722 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561bb38f1020 con 0x561bb3b360b0 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe668ff9640 10 client.6516 handle_command_reply: tid=0 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 1 client.6516 shutdown 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 1 -- 172.21.15.133:0/2006713722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 msgr2=0x561bb3b35340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b35340 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe68001f210 tx=0x7fe68001b710 comp rx=0 tx=0).stop 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 1 -- 172.21.15.133:0/2006713722 shutdown_connections 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x561bb3b360b0 0x561bb3b364c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b35340 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b50200 0x561bb3b57ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe668ff9640 0 client.6516 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 1 --2- 172.21.15.133:0/2006713722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b34f60 0x561bb3b57980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 1 -- 172.21.15.133:0/2006713722 >> 172.21.15.133:0/2006713722 conn(0x561bb3afd4f0 msgr2=0x561bb3b04aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 1 -- 172.21.15.133:0/2006713722 shutdown_connections 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 1 -- 172.21.15.133:0/2006713722 wait complete. 2024-04-25T10:06:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.158+0000 7fe693260000 20 client.6516 trim_cache size 0 max 16384 2024-04-25T10:06:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.170+0000 7fe693260000 1 Processor -- start 2024-04-25T10:06:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.170+0000 7fe693260000 1 -- start start 2024-04-25T10:06:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b34f60 0x561bb3b4f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b50200 0x561bb3b4e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b4ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6881fc5b0 con 0x561bb3b50b20 2024-04-25T10:06:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6881fb790 con 0x561bb3b50200 2024-04-25T10:06:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6881fac20 con 0x561bb3b34f60 2024-04-25T10:06:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68e690640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b4ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68e690640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b4ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40450/0 (socket says 172.21.15.133:40450) 2024-04-25T10:06:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68e690640 1 -- 172.21.15.133:0/1951957666 learned_addr learned my addr 172.21.15.133:0/1951957666 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68e690640 1 -- 172.21.15.133:0/1951957666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b34f60 msgr2=0x561bb3b4f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68e690640 1 --2- 172.21.15.133:0/1951957666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b34f60 0x561bb3b4f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68e690640 1 -- 172.21.15.133:0/1951957666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b50200 msgr2=0x561bb3b4e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68e690640 1 --2- 172.21.15.133:0/1951957666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b50200 0x561bb3b4e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68e690640 1 -- 172.21.15.133:0/1951957666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bb3914a30 con 0x561bb3b50b20 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68e690640 1 --2- 172.21.15.133:0/1951957666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b4ea90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe68001e9b0 tx=0x7fe6800b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00ed72f5c6af88c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe66affd640 1 -- 172.21.15.133:0/1951957666 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680006b60 con 0x561bb3b50b20 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe66affd640 1 -- 172.21.15.133:0/1951957666 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe680006d00 con 0x561bb3b50b20 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe66affd640 1 -- 172.21.15.133:0/1951957666 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6800a7870 con 0x561bb3b50b20 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- 172.21.15.133:0/1951957666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 msgr2=0x561bb3b4ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 --2- 172.21.15.133:0/1951957666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b4ea90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe68001e9b0 tx=0x7fe6800b4000 comp rx=0 tx=0).stop 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- 172.21.15.133:0/1951957666 shutdown_connections 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 --2- 172.21.15.133:0/1951957666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50b20 0x561bb3b4ea90 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 --2- 172.21.15.133:0/1951957666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b50200 0x561bb3b4e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 --2- 172.21.15.133:0/1951957666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b34f60 0x561bb3b4f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- 172.21.15.133:0/1951957666 >> 172.21.15.133:0/1951957666 conn(0x561bb3afd4f0 msgr2=0x561bb3b336f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- 172.21.15.133:0/1951957666 shutdown_connections 2024-04-25T10:06:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- 172.21.15.133:0/1951957666 wait complete. 2024-04-25T10:06:12.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- start start 2024-04-25T10:06:12.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 Processor -- start 2024-04-25T10:06:12.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b34f60 0x561bb3b347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b05260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b50b20 0x561bb3b057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb38e1190 con 0x561bb3b50200 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6881fb190 con 0x561bb3b34f60 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561bb3954520 con 0x561bb3b50b20 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 10 client.0 _ll_register_callbacks cb 0x7fe6840047f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68d68e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b05260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68d68e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b05260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40456/0 (socket says 172.21.15.133:40456) 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68d68e640 1 -- 172.21.15.133:0/3377866875 learned_addr learned my addr 172.21.15.133:0/3377866875 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68d68e640 1 -- 172.21.15.133:0/3377866875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b50b20 msgr2=0x561bb3b057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68d68e640 1 --2- 172.21.15.133:0/3377866875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b50b20 0x561bb3b057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68d68e640 1 -- 172.21.15.133:0/3377866875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b34f60 msgr2=0x561bb3b347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68d68e640 1 --2- 172.21.15.133:0/3377866875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b34f60 0x561bb3b347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68d68e640 1 -- 172.21.15.133:0/3377866875 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bb391d040 con 0x561bb3b50200 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe68d68e640 1 --2- 172.21.15.133:0/3377866875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b05260 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe678009fd0 tx=0x7fe678055ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e71caa407bf232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe668ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:12.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe668ff9640 1 -- 172.21.15.133:0/3377866875 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67805a3d0 con 0x561bb3b50200 2024-04-25T10:06:12.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe668ff9640 1 -- 172.21.15.133:0/3377866875 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe6780556d0 con 0x561bb3b50200 2024-04-25T10:06:12.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- 172.21.15.133:0/3377866875 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe680006d00 con 0x561bb3b50200 2024-04-25T10:06:12.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe668ff9640 1 -- 172.21.15.133:0/3377866875 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe67805e990 con 0x561bb3b50200 2024-04-25T10:06:12.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 10 client.6522 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:12.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.174+0000 7fe693260000 1 -- 172.21.15.133:0/3377866875 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x561bb3611900 con 0x561bb3b50200 2024-04-25T10:06:12.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe668ff9640 1 -- 172.21.15.133:0/3377866875 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe680006d00 con 0x561bb3b50200 2024-04-25T10:06:12.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe668ff9640 1 -- 172.21.15.133:0/3377866875 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe67805c040 con 0x561bb3b50200 2024-04-25T10:06:12.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 10 client.6522 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:12.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 10 client.6522 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 20 client.6522 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 --2- 172.21.15.133:0/3377866875 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x561bb3bb6490 0x561bb3bb8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 4 client.6522 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 -- 172.21.15.133:0/3377866875 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x561bb3962750 con 0x561bb3bb6490 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe68de8f640 1 --2- 172.21.15.133:0/3377866875 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x561bb3bb6490 0x561bb3bb8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe668ff9640 10 client.6522 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe68de8f640 1 --2- 172.21.15.133:0/3377866875 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x561bb3bb6490 0x561bb3bb8880 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327791d7c6326781 server_cookie=bc0375861e269a6d in_seq=0 out_seq=0 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe668ff9640 1 -- 172.21.15.133:0/3377866875 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561bb3962750 con 0x561bb3bb6490 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe668ff9640 10 client.6522 handle_command_reply: tid=0 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 client.6522 shutdown 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 -- 172.21.15.133:0/3377866875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 msgr2=0x561bb3b05260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 --2- 172.21.15.133:0/3377866875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b05260 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe678009fd0 tx=0x7fe678055ce0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 -- 172.21.15.133:0/3377866875 shutdown_connections 2024-04-25T10:06:12.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 --2- 172.21.15.133:0/3377866875 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x561bb3bb6490 0x561bb3bb8880 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 --2- 172.21.15.133:0/3377866875 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561bb3b50b20 0x561bb3b057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 --2- 172.21.15.133:0/3377866875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bb3b50200 0x561bb3b05260 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 --2- 172.21.15.133:0/3377866875 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561bb3b34f60 0x561bb3b347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 -- 172.21.15.133:0/3377866875 >> 172.21.15.133:0/3377866875 conn(0x561bb3afd4f0 msgr2=0x561bb3b07c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe668ff9640 0 client.6522 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:12.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 -- 172.21.15.133:0/3377866875 shutdown_connections 2024-04-25T10:06:12.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 1 -- 172.21.15.133:0/3377866875 wait complete. 2024-04-25T10:06:12.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe693260000 20 client.6522 trim_cache size 0 max 16384 2024-04-25T10:06:12.188 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:12.189 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:06:12.189 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "8b8a07f7-ce80-435b-8bda-d06916484062", 2024-04-25T10:06:12.189 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:06:12.189 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:12.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 -- 172.21.15.133:0/58805425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe66c066f60 msgr2=0x7fe66c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe66c066f60 0x7fe66c069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe684001890 tx=0x7fe6840016a0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 -- 172.21.15.133:0/58805425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6880af930 msgr2=0x7fe68809d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6880af930 0x7fe68809d1c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe678011560 tx=0x7fe678011a20 comp rx=0 tx=0).stop 2024-04-25T10:06:12.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 -- 172.21.15.133:0/58805425 shutdown_connections 2024-04-25T10:06:12.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe6880b4e60 0x7fe6880b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe66c066f60 0x7fe66c069420 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6880af930 0x7fe68809d1c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 --2- 172.21.15.133:0/58805425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe6740051d0 0x7fe688146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 -- 172.21.15.133:0/58805425 >> 172.21.15.133:0/58805425 conn(0x7fe6880cc790 msgr2=0x7fe6880aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 -- 172.21.15.133:0/58805425 shutdown_connections 2024-04-25T10:06:12.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.178+0000 7fe6900f3640 1 -- 172.21.15.133:0/58805425 wait complete. 2024-04-25T10:06:12.197 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '8b8a07f7-ce80-435b-8bda-d06916484062', 'mode': 'asynchronous'}' 2024-04-25T10:06:12.197 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T10:06:12.197 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:12.198 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T10:06:12.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.298+0000 7fb8b4975640 1 Processor -- start 2024-04-25T10:06:12.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.298+0000 7fb8b4975640 1 -- start start 2024-04-25T10:06:12.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.298+0000 7fb8b4975640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8b00b2150 0x7fb8b006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.298+0000 7fb8b4975640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb8b015b9b0 0x7fb8b006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.298+0000 7fb8b4975640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b015ac50 con 0x7fb8b0143ea0 2024-04-25T10:06:12.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.298+0000 7fb8b4975640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b00ccf20 con 0x7fb8b015b9b0 2024-04-25T10:06:12.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.298+0000 7fb8b4975640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00cd180 con 0x7fb8b00b2150 2024-04-25T10:06:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8aed9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb8b015b9b0 0x7fb8b006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8ae59c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb8b0143ea0 0x7fb8b00cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:59868/0 (socket says 172.21.15.133:59868) 2024-04-25T10:06:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8aed9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb8b015b9b0 0x7fb8b006e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:46342/0 (socket says 172.21.15.133:46342) 2024-04-25T10:06:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8ae59c640 1 -- 172.21.15.133:0/1280832380 learned_addr learned my addr 172.21.15.133:0/1280832380 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8add9b640 1 --2- 172.21.15.133:0/1280832380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8b00b2150 0x7fb8b006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8aed9d640 1 -- 172.21.15.133:0/1280832380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8b00b2150 msgr2=0x7fb8b006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8aed9d640 1 --2- 172.21.15.133:0/1280832380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8b00b2150 0x7fb8b006e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8aed9d640 1 -- 172.21.15.133:0/1280832380 >> v1:172.21.15.162:6789/0 conn(0x7fb8b0143ea0 legacy=0x7fb8b00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8ad59a640 1 -- 172.21.15.133:0/1280832380 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522371746 0 0) 0x7fb8a0001120 con 0x7fb8b0143ea0 2024-04-25T10:06:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8aed9d640 1 -- 172.21.15.133:0/1280832380 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b015aed0 con 0x7fb8b015b9b0 2024-04-25T10:06:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8add9b640 1 --2- 172.21.15.133:0/1280832380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8b00b2150 0x7fb8b006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8aed9d640 1 --2- 172.21.15.133:0/1280832380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b015b9b0 0x7fb8b006e8f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800bfa0 tx=0x7fb8a801eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2975ffe95e1123d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8ad59a640 1 -- 172.21.15.133:0/1280832380 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a800b2d0 con 0x7fb8b015b9b0 2024-04-25T10:06:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8ad59a640 1 -- 172.21.15.133:0/1280832380 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a80a78b0 con 0x7fb8b015b9b0 2024-04-25T10:06:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8ad59a640 1 -- 172.21.15.133:0/1280832380 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80a7c80 con 0x7fb8b015b9b0 2024-04-25T10:06:12.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8b4975640 1 -- 172.21.15.133:0/1280832380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b015b9b0 msgr2=0x7fb8b006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8b4975640 1 --2- 172.21.15.133:0/1280832380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b015b9b0 0x7fb8b006e8f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800bfa0 tx=0x7fb8a801eb70 comp rx=0 tx=0).stop 2024-04-25T10:06:12.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8b4975640 1 -- 172.21.15.133:0/1280832380 shutdown_connections 2024-04-25T10:06:12.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8b4975640 1 --2- 172.21.15.133:0/1280832380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8b00b2150 0x7fb8b006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8b4975640 1 --2- 172.21.15.133:0/1280832380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b015b9b0 0x7fb8b006e8f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8b4975640 1 -- 172.21.15.133:0/1280832380 >> 172.21.15.133:0/1280832380 conn(0x7fb8b0066f70 msgr2=0x7fb8b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8b4975640 1 -- 172.21.15.133:0/1280832380 shutdown_connections 2024-04-25T10:06:12.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.302+0000 7fb8b4975640 1 -- 172.21.15.133:0/1280832380 wait complete. 2024-04-25T10:06:12.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8b4975640 1 Processor -- start 2024-04-25T10:06:12.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8b4975640 1 -- start start 2024-04-25T10:06:12.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8b4975640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8b00b2150 0x7fb8b01340f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8b4975640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b0143ea0 0x7fb8b0157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8b4975640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b015b9b0 0x7fb8b0158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8b4975640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00b1270 con 0x7fb8b0143ea0 2024-04-25T10:06:12.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8b4975640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00b10f0 con 0x7fb8b015b9b0 2024-04-25T10:06:12.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8b4975640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00b13f0 con 0x7fb8b00b2150 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8add9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b0143ea0 0x7fb8b0157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8add9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b0143ea0 0x7fb8b0157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40462/0 (socket says 172.21.15.133:40462) 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8add9b640 1 -- 172.21.15.133:0/3557350889 learned_addr learned my addr 172.21.15.133:0/3557350889 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8ae59c640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8b00b2150 0x7fb8b01340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8aed9d640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b015b9b0 0x7fb8b0158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8add9b640 1 -- 172.21.15.133:0/3557350889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8b00b2150 msgr2=0x7fb8b01340f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8add9b640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8b00b2150 0x7fb8b01340f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8add9b640 1 -- 172.21.15.133:0/3557350889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b015b9b0 msgr2=0x7fb8b0158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8add9b640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b015b9b0 0x7fb8b0158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8add9b640 1 -- 172.21.15.133:0/3557350889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b0142660 con 0x7fb8b0143ea0 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8add9b640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b0143ea0 0x7fb8b0157ad0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb89800ebe0 tx=0x7fb89800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8a6ffd640 1 -- 172.21.15.133:0/3557350889 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898017070 con 0x7fb8b0143ea0 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8a6ffd640 1 -- 172.21.15.133:0/3557350889 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89800cc30 con 0x7fb8b0143ea0 2024-04-25T10:06:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8b4975640 1 -- 172.21.15.133:0/3557350889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b0142900 con 0x7fb8b0143ea0 2024-04-25T10:06:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8a6ffd640 1 -- 172.21.15.133:0/3557350889 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898013470 con 0x7fb8b0143ea0 2024-04-25T10:06:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.306+0000 7fb8b4975640 1 -- 172.21.15.133:0/3557350889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b0142ec0 con 0x7fb8b0143ea0 2024-04-25T10:06:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8a6ffd640 1 -- 172.21.15.133:0/3557350889 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb89801a070 con 0x7fb8b0143ea0 2024-04-25T10:06:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8a6ffd640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb890066f60 0x7fb890069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8a6ffd640 1 -- 172.21.15.133:0/3557350889 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb898084fe0 con 0x7fb8b0143ea0 2024-04-25T10:06:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8ae59c640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb890066f60 0x7fb890069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8ae59c640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb890066f60 0x7fb890069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0000f70 tx=0x7fb8a0007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 Processor -- start 2024-04-25T10:06:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 -- start start 2024-04-25T10:06:12.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28000 0x55bd40d283e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cdded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40cde410 0x55bd40cde7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd40b2bf40 con 0x55bd40d28920 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd40ab3fc0 con 0x55bd40cde410 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd40b27fe0 con 0x55bd40d28000 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8add9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cdded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8add9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cdded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40466/0 (socket says 172.21.15.133:40466) 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8add9b640 1 -- 172.21.15.133:0/3656254633 learned_addr learned my addr 172.21.15.133:0/3656254633 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8aed9d640 1 --2- 172.21.15.133:0/3656254633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40cde410 0x55bd40cde7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8add9b640 1 -- 172.21.15.133:0/3656254633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28000 msgr2=0x55bd40d283e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8add9b640 1 --2- 172.21.15.133:0/3656254633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28000 0x55bd40d283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8add9b640 1 -- 172.21.15.133:0/3656254633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40cde410 msgr2=0x55bd40cde7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8add9b640 1 --2- 172.21.15.133:0/3656254633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40cde410 0x55bd40cde7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8add9b640 1 -- 172.21.15.133:0/3656254633 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd40b73a50 con 0x55bd40d28920 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8add9b640 1 --2- 172.21.15.133:0/3656254633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cdded0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb89805d9d0 tx=0x7fb898055330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf040f89ab551fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb88ffff640 1 -- 172.21.15.133:0/3656254633 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89806d020 con 0x55bd40d28920 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb88ffff640 1 -- 172.21.15.133:0/3656254633 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb898053ba0 con 0x55bd40d28920 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb88ffff640 1 -- 172.21.15.133:0/3656254633 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898021040 con 0x55bd40d28920 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/3656254633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 msgr2=0x55bd40cdded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/3656254633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cdded0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb89805d9d0 tx=0x7fb898055330 comp rx=0 tx=0).stop 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/3656254633 shutdown_connections 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/3656254633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40cde410 0x55bd40cde7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/3656254633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cdded0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/3656254633 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28000 0x55bd40d283e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/3656254633 >> 172.21.15.133:0/3656254633 conn(0x55bd40cda3c0 msgr2=0x55bd40cdc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/3656254633 shutdown_connections 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.310+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/3656254633 wait complete. 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- start start 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 Processor -- start 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40cde410 0x55bd40ce9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40d28000 0x55bd40cea340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd40b30570 con 0x55bd40d28920 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd40ae65f0 con 0x55bd40d28000 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd40ae0d60 con 0x55bd40cde410 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 10 client.0 _ll_register_callbacks cb 0x55bd40be07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8aed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8aed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40472/0 (socket says 172.21.15.133:40472) 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8aed9d640 1 -- 172.21.15.133:0/4139884834 learned_addr learned my addr 172.21.15.133:0/4139884834 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8aed9d640 1 -- 172.21.15.133:0/4139884834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40cde410 msgr2=0x55bd40ce9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8ae59c640 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40cde410 0x55bd40ce9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8aed9d640 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40cde410 0x55bd40ce9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8aed9d640 1 -- 172.21.15.133:0/4139884834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40d28000 msgr2=0x55bd40cea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8add9b640 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40d28000 0x55bd40cea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8aed9d640 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40d28000 0x55bd40cea340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8aed9d640 1 -- 172.21.15.133:0/4139884834 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd40ab4210 con 0x55bd40d28920 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8aed9d640 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cea880 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800bfa0 tx=0x7fb8a801d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0a894a5550925 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb88d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb88d7fa640 1 -- 172.21.15.133:0/4139884834 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80039a0 con 0x55bd40d28920 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb88d7fa640 1 -- 172.21.15.133:0/4139884834 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8a8003b40 con 0x55bd40d28920 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/4139884834 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb898053ba0 con 0x55bd40d28920 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb88d7fa640 1 -- 172.21.15.133:0/4139884834 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8a8003d40 con 0x55bd40d28920 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 10 client.6531 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/4139884834 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55bd40bb0400 con 0x55bd40d28920 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb88d7fa640 1 -- 172.21.15.133:0/4139884834 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb898053ba0 con 0x55bd40d28920 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb88d7fa640 1 -- 172.21.15.133:0/4139884834 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb8a80a78b0 con 0x55bd40d28920 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 10 client.6531 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 10 client.6531 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 20 client.6531 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bd40ceb290 0x55bd40d3a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 4 client.6531 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/4139884834 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd40ac7c90 con 0x55bd40ceb290 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8ae59c640 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bd40ceb290 0x55bd40d3a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb88d7fa640 10 client.6531 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8ae59c640 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bd40ceb290 0x55bd40d3a2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2548f4461c1717c7 server_cookie=35f9936350913fed in_seq=0 out_seq=0 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb88d7fa640 1 -- 172.21.15.133:0/4139884834 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd40ac7c90 con 0x55bd40ceb290 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb88d7fa640 10 client.6531 handle_command_reply: tid=0 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 client.6531 shutdown 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/4139884834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 msgr2=0x55bd40cea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cea880 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800bfa0 tx=0x7fb8a801d3b0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/4139884834 shutdown_connections 2024-04-25T10:06:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bd40ceb290 0x55bd40d3a2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28920 0x55bd40cea880 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40d28000 0x55bd40cea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb88d7fa640 0 client.6531 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/4139884834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40cde410 0x55bd40ce9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/4139884834 >> 172.21.15.133:0/4139884834 conn(0x55bd40cda3c0 msgr2=0x55bd40cdd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/4139884834 shutdown_connections 2024-04-25T10:06:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/4139884834 wait complete. 2024-04-25T10:06:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.314+0000 7fb8b7ae2000 20 client.6531 trim_cache size 0 max 16384 2024-04-25T10:06:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 Processor -- start 2024-04-25T10:06:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 -- start start 2024-04-25T10:06:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40cde410 0x55bd40cdd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28000 0x55bd40cddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28920 0x55bd40d3a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b02034e0 con 0x55bd40d28000 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b02026c0 con 0x55bd40cde410 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0163be0 con 0x55bd40d28920 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8add9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28000 0x55bd40cddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8add9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28000 0x55bd40cddd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40478/0 (socket says 172.21.15.133:40478) 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8add9b640 1 -- 172.21.15.133:0/3913746779 learned_addr learned my addr 172.21.15.133:0/3913746779 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8add9b640 1 -- 172.21.15.133:0/3913746779 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28920 msgr2=0x55bd40d3a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8add9b640 1 --2- 172.21.15.133:0/3913746779 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28920 0x55bd40d3a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8add9b640 1 -- 172.21.15.133:0/3913746779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40cde410 msgr2=0x55bd40cdd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8add9b640 1 --2- 172.21.15.133:0/3913746779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40cde410 0x55bd40cdd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8add9b640 1 -- 172.21.15.133:0/3913746779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd40b86c10 con 0x55bd40d28000 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8add9b640 1 --2- 172.21.15.133:0/3913746779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28000 0x55bd40cddd00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb89805d9d0 tx=0x7fb89801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81dce94d15402802 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb88f7fe640 1 -- 172.21.15.133:0/3913746779 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89806d050 con 0x55bd40d28000 2024-04-25T10:06:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb88f7fe640 1 -- 172.21.15.133:0/3913746779 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb898053ba0 con 0x55bd40d28000 2024-04-25T10:06:12.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb88f7fe640 1 -- 172.21.15.133:0/3913746779 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898021040 con 0x55bd40d28000 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/3913746779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28000 msgr2=0x55bd40cddd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/3913746779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28000 0x55bd40cddd00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb89805d9d0 tx=0x7fb89801e000 comp rx=0 tx=0).stop 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/3913746779 shutdown_connections 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/3913746779 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28920 0x55bd40d3a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/3913746779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40d28000 0x55bd40cddd00 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/3913746779 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40cde410 0x55bd40cdd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/3913746779 >> 172.21.15.133:0/3913746779 conn(0x55bd40cda3c0 msgr2=0x55bd40ce9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/3913746779 shutdown_connections 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.330+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/3913746779 wait complete. 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- start start 2024-04-25T10:06:12.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 Processor -- start 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40cde410 0x55bd40d3c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28000 0x55bd40d44d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40d28920 0x55bd40d452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd40b69e80 con 0x55bd40cde410 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b02020c0 con 0x55bd40d28920 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd40b30570 con 0x55bd40d28000 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 10 client.0 _ll_register_callbacks cb 0x55bd40b7e2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8ae59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40cde410 0x55bd40d3c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8ae59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40cde410 0x55bd40d3c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40490/0 (socket says 172.21.15.133:40490) 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8ae59c640 1 -- 172.21.15.133:0/1865743751 learned_addr learned my addr 172.21.15.133:0/1865743751 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8add9b640 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28000 0x55bd40d44d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8ae59c640 1 -- 172.21.15.133:0/1865743751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28000 msgr2=0x55bd40d44d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8ae59c640 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28000 0x55bd40d44d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8ae59c640 1 -- 172.21.15.133:0/1865743751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40d28920 msgr2=0x55bd40d452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8ae59c640 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40d28920 0x55bd40d452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8ae59c640 1 -- 172.21.15.133:0/1865743751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd40ac0bb0 con 0x55bd40cde410 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8ae59c640 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40cde410 0x55bd40d3c2f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00017a0 tx=0x7fb8a0001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7420d2106076d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb88d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb88d7fa640 1 -- 172.21.15.133:0/1865743751 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0026070 con 0x55bd40cde410 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb88d7fa640 1 -- 172.21.15.133:0/1865743751 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8a0022730 con 0x55bd40cde410 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/1865743751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb898053ba0 con 0x55bd40cde410 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb88d7fa640 1 -- 172.21.15.133:0/1865743751 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8a0031050 con 0x55bd40cde410 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 10 client.6537 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/1865743751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55bd40ad3030 con 0x55bd40cde410 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb88d7fa640 1 -- 172.21.15.133:0/1865743751 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb898053ba0 con 0x55bd40cde410 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb88d7fa640 1 -- 172.21.15.133:0/1865743751 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb8a0034020 con 0x55bd40cde410 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 10 client.6537 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 10 client.6537 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:12.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 20 client.6537 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bd40d885e0 0x55bd40d889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 4 client.6537 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/1865743751 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55bd40b39480 con 0x55bd40d885e0 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8add9b640 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bd40d885e0 0x55bd40d889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8add9b640 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bd40d885e0 0x55bd40d889f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbaf636540931dac server_cookie=a535986aae7f44f0 in_seq=0 out_seq=0 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb88d7fa640 10 client.6537 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb88d7fa640 1 -- 172.21.15.133:0/1865743751 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bd40b39480 con 0x55bd40d885e0 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb88d7fa640 10 client.6537 handle_command_reply: tid=0 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 client.6537 shutdown 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/1865743751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40cde410 msgr2=0x55bd40d3c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40cde410 0x55bd40d3c2f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00017a0 tx=0x7fb8a0001090 comp rx=0 tx=0).stop 2024-04-25T10:06:12.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/1865743751 shutdown_connections 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bd40d885e0 0x55bd40d889f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bd40d28920 0x55bd40d452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb88d7fa640 0 client.6537 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bd40d28000 0x55bd40d44d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 --2- 172.21.15.133:0/1865743751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd40cde410 0x55bd40d3c2f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/1865743751 >> 172.21.15.133:0/1865743751 conn(0x55bd40cda3c0 msgr2=0x55bd40cea600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/1865743751 shutdown_connections 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 1 -- 172.21.15.133:0/1865743751 wait complete. 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.334+0000 7fb8b7ae2000 20 client.6537 trim_cache size 0 max 16384 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:12.345 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:06:12.346 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "fa1023ee-a770-43bd-abd8-03b60e6c61d1", 2024-04-25T10:06:12.346 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:06:12.346 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 -- 172.21.15.133:0/3557350889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb890066f60 msgr2=0x7fb890069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb890066f60 0x7fb890069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0000f70 tx=0x7fb8a0007b60 comp rx=0 tx=0).stop 2024-04-25T10:06:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 -- 172.21.15.133:0/3557350889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b0143ea0 msgr2=0x7fb8b0157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b0143ea0 0x7fb8b0157ad0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb89800ebe0 tx=0x7fb89800c520 comp rx=0 tx=0).stop 2024-04-25T10:06:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 -- 172.21.15.133:0/3557350889 shutdown_connections 2024-04-25T10:06:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8b015b9b0 0x7fb8b0158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb890066f60 0x7fb890069420 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b0143ea0 0x7fb8b0157ad0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 --2- 172.21.15.133:0/3557350889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8b00b2150 0x7fb8b01340f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 -- 172.21.15.133:0/3557350889 >> 172.21.15.133:0/3557350889 conn(0x7fb8b0066f70 msgr2=0x7fb8b015c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 -- 172.21.15.133:0/3557350889 shutdown_connections 2024-04-25T10:06:12.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.338+0000 7fb8b4975640 1 -- 172.21.15.133:0/3557350889 wait complete. 2024-04-25T10:06:12.356 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'fa1023ee-a770-43bd-abd8-03b60e6c61d1', 'mode': 'asynchronous'}' 2024-04-25T10:06:12.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:12.357 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T10:06:12.363 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T10:06:12.363 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:12.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.502+0000 7f97d4a74640 1 Processor -- start 2024-04-25T10:06:12.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97d4a74640 1 -- start start 2024-04-25T10:06:12.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97d4a74640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f97d015c9a0 0x7f97d00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97d4a74640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d00c61d0 con 0x7f97d00ce330 2024-04-25T10:06:12.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97d4a74640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d00c64b0 con 0x7f97d00b1f70 2024-04-25T10:06:12.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97d4a74640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d006d4f0 con 0x7f97d015c9a0 2024-04-25T10:06:12.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97ce59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f97d015c9a0 0x7f97d00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97ced9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f97d00ce330 0x7f97d015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34506/0 (socket says 172.21.15.133:34506) 2024-04-25T10:06:12.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97ced9d640 1 -- 172.21.15.133:0/113396019 learned_addr learned my addr 172.21.15.133:0/113396019 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:12.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97ce59c640 1 -- 172.21.15.133:0/113396019 >> v1:172.21.15.162:6789/0 conn(0x7f97d00b1f70 legacy=0x7f97d006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97ce59c640 1 -- 172.21.15.133:0/113396019 >> v1:172.21.15.133:6789/0 conn(0x7f97d00ce330 legacy=0x7f97d015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97cd59a640 1 -- 172.21.15.133:0/113396019 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967028836 0 0) 0x7f97d00c61d0 con 0x7f97d00ce330 2024-04-25T10:06:12.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97ce59c640 1 -- 172.21.15.133:0/113396019 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d006d7d0 con 0x7f97d015c9a0 2024-04-25T10:06:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97ce59c640 1 --2- 172.21.15.133:0/113396019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d015c9a0 0x7f97d00c5980 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f97c40097c0 tx=0x7f97c401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169d729a1b28475d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97cd59a640 1 -- 172.21.15.133:0/113396019 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40b0070 con 0x7f97d015c9a0 2024-04-25T10:06:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97cd59a640 1 -- 172.21.15.133:0/113396019 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c40ab600 con 0x7f97d015c9a0 2024-04-25T10:06:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.506+0000 7f97cd59a640 1 -- 172.21.15.133:0/113396019 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40ab9d0 con 0x7f97d015c9a0 2024-04-25T10:06:12.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 -- 172.21.15.133:0/113396019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d015c9a0 msgr2=0x7f97d00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 --2- 172.21.15.133:0/113396019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d015c9a0 0x7f97d00c5980 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f97c40097c0 tx=0x7f97c401c460 comp rx=0 tx=0).stop 2024-04-25T10:06:12.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 -- 172.21.15.133:0/113396019 shutdown_connections 2024-04-25T10:06:12.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 --2- 172.21.15.133:0/113396019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d015c9a0 0x7f97d00c5980 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 -- 172.21.15.133:0/113396019 >> 172.21.15.133:0/113396019 conn(0x7f97d0067000 msgr2=0x7f97d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 -- 172.21.15.133:0/113396019 shutdown_connections 2024-04-25T10:06:12.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 -- 172.21.15.133:0/113396019 wait complete. 2024-04-25T10:06:12.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 Processor -- start 2024-04-25T10:06:12.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 -- start start 2024-04-25T10:06:12.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97d00b1f70 0x7f97d0147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d00ce330 0x7f97d0151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97d015c9a0 0x7f97d014f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d015cf00 con 0x7f97d00ce330 2024-04-25T10:06:12.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d015cd80 con 0x7f97d00b1f70 2024-04-25T10:06:12.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d0169de0 con 0x7f97d015c9a0 2024-04-25T10:06:12.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97cdd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d00ce330 0x7f97d0151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97cdd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d00ce330 0x7f97d0151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40504/0 (socket says 172.21.15.133:40504) 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97cdd9b640 1 -- 172.21.15.133:0/2015407341 learned_addr learned my addr 172.21.15.133:0/2015407341 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97ced9d640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97d015c9a0 0x7f97d014f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97ce59c640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97d00b1f70 0x7f97d0147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97cdd9b640 1 -- 172.21.15.133:0/2015407341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97d015c9a0 msgr2=0x7f97d014f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97cdd9b640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97d015c9a0 0x7f97d014f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97cdd9b640 1 -- 172.21.15.133:0/2015407341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97d00b1f70 msgr2=0x7f97d0147350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97cdd9b640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97d00b1f70 0x7f97d0147350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97cdd9b640 1 -- 172.21.15.133:0/2015407341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d01493c0 con 0x7f97d00ce330 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97ced9d640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97d015c9a0 0x7f97d014f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97cdd9b640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d00ce330 0x7f97d0151120 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f97c000b410 tx=0x7f97c000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97beffd640 1 -- 172.21.15.133:0/2015407341 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0015030 con 0x7f97d00ce330 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97beffd640 1 -- 172.21.15.133:0/2015407341 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c0004020 con 0x7f97d00ce330 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 -- 172.21.15.133:0/2015407341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d01496a0 con 0x7f97d00ce330 2024-04-25T10:06:12.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97beffd640 1 -- 172.21.15.133:0/2015407341 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0004410 con 0x7f97d00ce330 2024-04-25T10:06:12.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.510+0000 7f97d4a74640 1 -- 172.21.15.133:0/2015407341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d014a6a0 con 0x7f97d00ce330 2024-04-25T10:06:12.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.514+0000 7f97d4a74640 1 -- 172.21.15.133:0/2015407341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d0068ef0 con 0x7f97d00ce330 2024-04-25T10:06:12.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.514+0000 7f97beffd640 1 -- 172.21.15.133:0/2015407341 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97c0018070 con 0x7f97d00ce330 2024-04-25T10:06:12.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.514+0000 7f97beffd640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97b0066f60 0x7f97b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.514+0000 7f97beffd640 1 -- 172.21.15.133:0/2015407341 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97c0084cd0 con 0x7f97d00ce330 2024-04-25T10:06:12.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.514+0000 7f97ce59c640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97b0066f60 0x7f97b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.514+0000 7f97ce59c640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97b0066f60 0x7f97b0069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f97c40ad000 tx=0x7f97c40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.518+0000 7f97beffd640 1 -- 172.21.15.133:0/2015407341 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97c0055be0 con 0x7f97d00ce330 2024-04-25T10:06:12.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.694+0000 7f97d4a74640 1 -- 172.21.15.133:0/2015407341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97d0155850 con 0x7f97d00ce330 2024-04-25T10:06:12.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97beffd640 1 -- 172.21.15.133:0/2015407341 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f97c00554e0 con 0x7f97d00ce330 2024-04-25T10:06:12.700 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:12.700 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:06:09.247920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 24 2024-04-25T10:06:12.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 -- 172.21.15.133:0/2015407341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97b0066f60 msgr2=0x7f97b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97b0066f60 0x7f97b0069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f97c40ad000 tx=0x7f97c40a6000 comp rx=0 tx=0).stop 2024-04-25T10:06:12.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 -- 172.21.15.133:0/2015407341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d00ce330 msgr2=0x7f97d0151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d00ce330 0x7f97d0151120 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f97c000b410 tx=0x7f97c000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 -- 172.21.15.133:0/2015407341 shutdown_connections 2024-04-25T10:06:12.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97d015c9a0 0x7f97d014f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97b0066f60 0x7f97b0069420 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d00ce330 0x7f97d0151120 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 --2- 172.21.15.133:0/2015407341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97d00b1f70 0x7f97d0147350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 -- 172.21.15.133:0/2015407341 >> 172.21.15.133:0/2015407341 conn(0x7f97d0067000 msgr2=0x7f97d00b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 -- 172.21.15.133:0/2015407341 shutdown_connections 2024-04-25T10:06:12.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:12.698+0000 7f97d4a74640 1 -- 172.21.15.133:0/2015407341 wait complete. 2024-04-25T10:06:12.719 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T10:06:12.843 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:12.844 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:06:12.844 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:12.858 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:12.858 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:06:12.858 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:12.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:12.863 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T10:06:12.926 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:06:12.926 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:06:12.926 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138842 s, 75.5 MB/s 2024-04-25T10:06:12.927 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T10:06:12.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.082+0000 7f5fe437e640 1 Processor -- start 2024-04-25T10:06:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe437e640 1 -- start start 2024-04-25T10:06:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe437e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5fdc152750 0x7f5fdc0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe437e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5fdc0ce870 0x7f5fdc0b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe437e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fdc06d350 con 0x7f5fdc152b30 2024-04-25T10:06:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe437e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc0cecb0 con 0x7f5fdc0ce870 2024-04-25T10:06:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe437e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc15bf90 con 0x7f5fdc152750 2024-04-25T10:06:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe291b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5fdc0ce870 0x7f5fdc0b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe291b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5fdc0ce870 0x7f5fdc0b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40514/0 (socket says 172.21.15.133:40514) 2024-04-25T10:06:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe291b640 1 -- 172.21.15.133:0/3123602238 learned_addr learned my addr 172.21.15.133:0/3123602238 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe211a640 1 --2- 172.21.15.133:0/3123602238 >> v2:172.21.15.162:3300/0 conn(0x7f5fdc152750 0x7f5fdc0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe291b640 1 -- 172.21.15.133:0/3123602238 >> v2:172.21.15.162:3300/0 conn(0x7f5fdc152750 msgr2=0x7f5fdc0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe291b640 1 --2- 172.21.15.133:0/3123602238 >> v2:172.21.15.162:3300/0 conn(0x7f5fdc152750 0x7f5fdc0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe291b640 1 -- 172.21.15.133:0/3123602238 >> v1:172.21.15.162:6789/0 conn(0x7f5fdc152b30 legacy=0x7f5fdc0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.086+0000 7f5fe291b640 1 -- 172.21.15.133:0/3123602238 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc06d5d0 con 0x7f5fdc0ce870 2024-04-25T10:06:13.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe291b640 1 --2- 172.21.15.133:0/3123602238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fdc0ce870 0x7f5fdc0b1ec0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc06f350 tx=0x7f5fd401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37edcd0cdae9351 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe1118640 1 -- 172.21.15.133:0/3123602238 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40a5780 con 0x7f5fdc0ce870 2024-04-25T10:06:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe1118640 1 -- 172.21.15.133:0/3123602238 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd40a5920 con 0x7f5fdc0ce870 2024-04-25T10:06:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe1118640 1 -- 172.21.15.133:0/3123602238 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40a5cf0 con 0x7f5fdc0ce870 2024-04-25T10:06:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 -- 172.21.15.133:0/3123602238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fdc0ce870 msgr2=0x7f5fdc0b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 --2- 172.21.15.133:0/3123602238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fdc0ce870 0x7f5fdc0b1ec0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc06f350 tx=0x7f5fd401c9d0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 -- 172.21.15.133:0/3123602238 shutdown_connections 2024-04-25T10:06:13.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 --2- 172.21.15.133:0/3123602238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fdc0ce870 0x7f5fdc0b1ec0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 --2- 172.21.15.133:0/3123602238 >> v2:172.21.15.162:3300/0 conn(0x7f5fdc152750 0x7f5fdc0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 -- 172.21.15.133:0/3123602238 >> 172.21.15.133:0/3123602238 conn(0x7f5fdc067000 msgr2=0x7f5fdc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 -- 172.21.15.133:0/3123602238 shutdown_connections 2024-04-25T10:06:13.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 -- 172.21.15.133:0/3123602238 wait complete. 2024-04-25T10:06:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 Processor -- start 2024-04-25T10:06:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 -- start start 2024-04-25T10:06:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fdc0ce870 0x7f5fdc0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc152750 0x7f5fdc1448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fdc152b30 0x7f5fdc164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc15fe70 con 0x7f5fdc152b30 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc15fcf0 con 0x7f5fdc152750 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc15fff0 con 0x7f5fdc0ce870 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe291b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fdc152b30 0x7f5fdc164990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe291b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fdc152b30 0x7f5fdc164990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40520/0 (socket says 172.21.15.133:40520) 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe291b640 1 -- 172.21.15.133:0/255630073 learned_addr learned my addr 172.21.15.133:0/255630073 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe1919640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc152750 0x7f5fdc1448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe211a640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fdc0ce870 0x7f5fdc0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe291b640 1 -- 172.21.15.133:0/255630073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fdc0ce870 msgr2=0x7f5fdc0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe291b640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fdc0ce870 0x7f5fdc0cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe291b640 1 -- 172.21.15.133:0/255630073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc152750 msgr2=0x7f5fdc1448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe291b640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc152750 0x7f5fdc1448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe291b640 1 -- 172.21.15.133:0/255630073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0cbdf0 con 0x7f5fdc152b30 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe291b640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fdc152b30 0x7f5fdc164990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40af040 tx=0x7f5fd4019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fd2ffd640 1 -- 172.21.15.133:0/255630073 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40bc020 con 0x7f5fdc152b30 2024-04-25T10:06:13.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fd2ffd640 1 -- 172.21.15.133:0/255630073 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd40b8690 con 0x7f5fdc152b30 2024-04-25T10:06:13.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fe437e640 1 -- 172.21.15.133:0/255630073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fdc151590 con 0x7f5fdc152b30 2024-04-25T10:06:13.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.090+0000 7f5fd2ffd640 1 -- 172.21.15.133:0/255630073 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40b8a80 con 0x7f5fdc152b30 2024-04-25T10:06:13.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.094+0000 7f5fe437e640 1 -- 172.21.15.133:0/255630073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fdc151db0 con 0x7f5fdc152b30 2024-04-25T10:06:13.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.094+0000 7f5fe437e640 1 -- 172.21.15.133:0/255630073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fdc068ef0 con 0x7f5fdc152b30 2024-04-25T10:06:13.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.094+0000 7f5fd2ffd640 1 -- 172.21.15.133:0/255630073 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fd40bf020 con 0x7f5fdc152b30 2024-04-25T10:06:13.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.094+0000 7f5fd2ffd640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5fc0066f60 0x7f5fc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.094+0000 7f5fe211a640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5fc0066f60 0x7f5fc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.094+0000 7f5fd2ffd640 1 -- 172.21.15.133:0/255630073 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5fd4120370 con 0x7f5fdc152b30 2024-04-25T10:06:13.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.094+0000 7f5fe211a640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5fc0066f60 0x7f5fc0069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc1456a0 tx=0x7f5fd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.098+0000 7f5fd2ffd640 1 -- 172.21.15.133:0/255630073 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5fd40c6030 con 0x7f5fdc152b30 2024-04-25T10:06:13.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.290+0000 7f5fe437e640 1 -- 172.21.15.133:0/255630073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fdc14ba00 con 0x7f5fdc152b30 2024-04-25T10:06:13.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.290+0000 7f5fd2ffd640 1 -- 172.21.15.133:0/255630073 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f5fd40165c0 con 0x7f5fdc152b30 2024-04-25T10:06:13.295 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:13.296 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:06:09.247920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:13.296 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 24 2024-04-25T10:06:13.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 -- 172.21.15.133:0/255630073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5fc0066f60 msgr2=0x7f5fc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5fc0066f60 0x7f5fc0069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc1456a0 tx=0x7f5fd8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:13.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 -- 172.21.15.133:0/255630073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fdc152b30 msgr2=0x7f5fdc164990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fdc152b30 0x7f5fdc164990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40af040 tx=0x7f5fd4019770 comp rx=0 tx=0).stop 2024-04-25T10:06:13.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 -- 172.21.15.133:0/255630073 shutdown_connections 2024-04-25T10:06:13.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5fc0066f60 0x7f5fc0069420 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fdc152b30 0x7f5fdc164990 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5fdc152750 0x7f5fdc1448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 --2- 172.21.15.133:0/255630073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5fdc0ce870 0x7f5fdc0cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 -- 172.21.15.133:0/255630073 >> 172.21.15.133:0/255630073 conn(0x7f5fdc067000 msgr2=0x7f5fdc0c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 -- 172.21.15.133:0/255630073 shutdown_connections 2024-04-25T10:06:13.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.294+0000 7f5fe437e640 1 -- 172.21.15.133:0/255630073 wait complete. 2024-04-25T10:06:13.314 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T10:06:13.434 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:13.434 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:06:13.435 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:13.447 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:13.448 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-04-25T10:06:13.448 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:06:13.448 DEBUG:teuthology.orchestra.run.smithi133:> import stat 2024-04-25T10:06:13.448 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:06:13.448 DEBUG:teuthology.orchestra.run.smithi133:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T10:06:13.448 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:06:13.513 INFO:teuthology.orchestra.run.smithi133.stdout:1099511630411 2024-04-25T10:06:13.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:13.521 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:13.521 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:06:13.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.678+0000 7efeb2dc5640 1 Processor -- start 2024-04-25T10:06:13.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.678+0000 7efeb2dc5640 1 -- start start 2024-04-25T10:06:13.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb2dc5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac145170 0x7efeac06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb2dc5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efeac06e4d0 0x7efeac0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb2dc5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeac0ce2e0 con 0x7efeac0cdd60 2024-04-25T10:06:13.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb2dc5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeac06db00 con 0x7efeac06e4d0 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb2dc5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac145550 con 0x7efeac145170 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb1362640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7efeac0cdd60 0x7efeac15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34514/0 (socket says 172.21.15.133:34514) 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb1362640 1 -- 172.21.15.133:0/453375359 learned_addr learned my addr 172.21.15.133:0/453375359 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeabfff640 1 --2- 172.21.15.133:0/453375359 >> v2:172.21.15.162:3300/0 conn(0x7efeac06e4d0 0x7efeac0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb0b61640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac145170 0x7efeac06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057220365 0 0) 0x7efeac0ce2e0 con 0x7efeac0cdd60 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe94003530 con 0x7efeac0cdd60 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1131243211 0 0) 0x7efe94003530 con 0x7efeac0cdd60 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac145170 msgr2=0x7efeac06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac145170 0x7efeac06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 >> v2:172.21.15.162:3300/0 conn(0x7efeac06e4d0 msgr2=0x7efeac0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 --2- 172.21.15.133:0/453375359 >> v2:172.21.15.162:3300/0 conn(0x7efeac06e4d0 0x7efeac0cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac06d4b0 con 0x7efeac0cdd60 2024-04-25T10:06:13.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7efea4001680 con 0x7efeac0cdd60 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 >> v1:172.21.15.133:6789/0 conn(0x7efeac0cdd60 legacy=0x7efeac15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe940039c0 0x7efe94003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac145170 0x7efe94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe94004d50 0x7efe940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac06db00 con 0x7efe94004d50 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac145550 con 0x7efe940039c0 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9400b9d0 con 0x7efeac145170 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb1362640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe94004d50 0x7efe940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb0b61640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe940039c0 0x7efe94003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeabfff640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac145170 0x7efe94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb1362640 1 -- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac145170 msgr2=0x7efe94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb1362640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac145170 0x7efe94004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb1362640 1 -- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe940039c0 msgr2=0x7efe94003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb1362640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe940039c0 0x7efe94003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb1362640 1 -- 172.21.15.133:0/453375359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94003530 con 0x7efe94004d50 2024-04-25T10:06:13.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb0b61640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe940039c0 0x7efe94003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeb1362640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe94004d50 0x7efe940051d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7efea4001060 tx=0x7efea401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279b210e54c263f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4007d80 con 0x7efe94004d50 2024-04-25T10:06:13.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea40a9020 con 0x7efe94004d50 2024-04-25T10:06:13.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.682+0000 7efeab7fe640 1 -- 172.21.15.133:0/453375359 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea40a5bc0 con 0x7efe94004d50 2024-04-25T10:06:13.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 -- 172.21.15.133:0/453375359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe94004d50 msgr2=0x7efe940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 --2- 172.21.15.133:0/453375359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe94004d50 0x7efe940051d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7efea4001060 tx=0x7efea401cca0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb1362640 1 -- 172.21.15.133:0/453375359 reap_dead start 2024-04-25T10:06:13.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 -- 172.21.15.133:0/453375359 shutdown_connections 2024-04-25T10:06:13.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 -- 172.21.15.133:0/453375359 >> 172.21.15.133:0/453375359 conn(0x7efeac066f70 msgr2=0x7efeac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 -- 172.21.15.133:0/453375359 shutdown_connections 2024-04-25T10:06:13.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 -- 172.21.15.133:0/453375359 wait complete. 2024-04-25T10:06:13.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 Processor -- start 2024-04-25T10:06:13.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 -- start start 2024-04-25T10:06:13.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac0ad310 0x7efeac1f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeac0ad6f0 0x7efeac139c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac138c10 0x7efeac139090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac15d5e0 con 0x7efeac138c10 2024-04-25T10:06:13.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac15cb70 con 0x7efeac0ad6f0 2024-04-25T10:06:13.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac15d760 con 0x7efeac0ad310 2024-04-25T10:06:13.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb1362640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac138c10 0x7efeac139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb1362640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac138c10 0x7efeac139090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40534/0 (socket says 172.21.15.133:40534) 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb1362640 1 -- 172.21.15.133:0/3434269528 learned_addr learned my addr 172.21.15.133:0/3434269528 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeabfff640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeac0ad6f0 0x7efeac139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb0b61640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac0ad310 0x7efeac1f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb1362640 1 -- 172.21.15.133:0/3434269528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac0ad310 msgr2=0x7efeac1f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb1362640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac0ad310 0x7efeac1f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb1362640 1 -- 172.21.15.133:0/3434269528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeac0ad6f0 msgr2=0x7efeac139c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb1362640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeac0ad6f0 0x7efeac139c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb1362640 1 -- 172.21.15.133:0/3434269528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac13e8d0 con 0x7efeac138c10 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeabfff640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeac0ad6f0 0x7efeac139c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb1362640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac138c10 0x7efeac139090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7efea40ab040 tx=0x7efe94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efea97fa640 1 -- 172.21.15.133:0/3434269528 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4001cd0 con 0x7efeac138c10 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efea97fa640 1 -- 172.21.15.133:0/3434269528 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea4004ab0 con 0x7efeac138c10 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 -- 172.21.15.133:0/3434269528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac13ebb0 con 0x7efeac138c10 2024-04-25T10:06:13.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efea97fa640 1 -- 172.21.15.133:0/3434269528 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4000e50 con 0x7efeac138c10 2024-04-25T10:06:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.686+0000 7efeb2dc5640 1 -- 172.21.15.133:0/3434269528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac0af790 con 0x7efeac138c10 2024-04-25T10:06:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efea97fa640 1 -- 172.21.15.133:0/3434269528 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efea40bd060 con 0x7efeac138c10 2024-04-25T10:06:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efea97fa640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe98066f60 0x7efe98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efea97fa640 1 -- 172.21.15.133:0/3434269528 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efea411e090 con 0x7efeac138c10 2024-04-25T10:06:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb0b61640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe98066f60 0x7efe98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb0b61640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe98066f60 0x7efe98069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efe9c007520 tx=0x7efe9c00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb5f32000 1 Processor -- start 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb5f32000 1 -- start start 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c327560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 0x555f8c2dd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2dd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb5f32000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8c12af20 con 0x555f8c327180 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb5f32000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8c0b3000 con 0x555f8c327aa0 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb5f32000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f8c126fc0 con 0x555f8c2dd600 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb0b61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c327560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb0b61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c327560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40536/0 (socket says 172.21.15.133:40536) 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb0b61640 1 -- 172.21.15.133:0/315925843 learned_addr learned my addr 172.21.15.133:0/315925843 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeabfff640 1 --2- 172.21.15.133:0/315925843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 0x555f8c2dd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb1362640 1 --2- 172.21.15.133:0/315925843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2dd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb0b61640 1 -- 172.21.15.133:0/315925843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 msgr2=0x555f8c2dd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb0b61640 1 --2- 172.21.15.133:0/315925843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2dd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb0b61640 1 -- 172.21.15.133:0/315925843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 msgr2=0x555f8c2dd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb0b61640 1 --2- 172.21.15.133:0/315925843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 0x555f8c2dd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb0b61640 1 -- 172.21.15.133:0/315925843 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f8c106ba0 con 0x555f8c327180 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeabfff640 1 --2- 172.21.15.133:0/315925843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 0x555f8c2dd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efeb0b61640 1 --2- 172.21.15.133:0/315925843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c327560 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efe9c018720 tx=0x7efe9c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eaccd8b4b222df1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efe927fc640 1 -- 172.21.15.133:0/315925843 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c028070 con 0x555f8c327180 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efe927fc640 1 -- 172.21.15.133:0/315925843 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c016a20 con 0x555f8c327180 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.690+0000 7efe927fc640 1 -- 172.21.15.133:0/315925843 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c016dd0 con 0x555f8c327180 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- 172.21.15.133:0/315925843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 msgr2=0x555f8c327560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 --2- 172.21.15.133:0/315925843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c327560 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efe9c018720 tx=0x7efe9c004a50 comp rx=0 tx=0).stop 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- 172.21.15.133:0/315925843 shutdown_connections 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 --2- 172.21.15.133:0/315925843 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2dd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 --2- 172.21.15.133:0/315925843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 0x555f8c2dd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 --2- 172.21.15.133:0/315925843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c327560 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- 172.21.15.133:0/315925843 >> 172.21.15.133:0/315925843 conn(0x555f8c2e3760 msgr2=0x555f8c2db9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- 172.21.15.133:0/315925843 shutdown_connections 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- 172.21.15.133:0/315925843 wait complete. 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- start start 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 Processor -- start 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2e2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 0x555f8c315450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8c1e89c0 con 0x555f8c327180 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8c0e55d0 con 0x555f8c327aa0 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f8c12a230 con 0x555f8c2dd600 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 10 client.0 _ll_register_callbacks cb 0x555f8c2c8a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2e2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2e2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40548/0 (socket says 172.21.15.133:40548) 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeabfff640 1 -- 172.21.15.133:0/3461154965 learned_addr learned my addr 172.21.15.133:0/3461154965 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb0b61640 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb1362640 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 0x555f8c315450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeabfff640 1 -- 172.21.15.133:0/3461154965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 msgr2=0x555f8c2e2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeabfff640 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2e2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeabfff640 1 -- 172.21.15.133:0/3461154965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 msgr2=0x555f8c315450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeabfff640 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 0x555f8c315450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeabfff640 1 -- 172.21.15.133:0/3461154965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f8c1715a0 con 0x555f8c327180 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb0b61640 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2e2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb1362640 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 0x555f8c315450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeabfff640 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2e2bb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efea00135b0 tx=0x7efea0013a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c1a7922ef319 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efe8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efe8b7fe640 1 -- 172.21.15.133:0/3461154965 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea001b070 con 0x555f8c327180 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- 172.21.15.133:0/3461154965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe9c016a20 con 0x555f8c327180 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efe8b7fe640 1 -- 172.21.15.133:0/3461154965 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efea000e6d0 con 0x555f8c327180 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efe8b7fe640 1 -- 172.21.15.133:0/3461154965 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7efea0026050 con 0x555f8c327180 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 10 client.6564 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- 172.21.15.133:0/3461154965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x555f8c1022b0 con 0x555f8c327180 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efe8b7fe640 1 -- 172.21.15.133:0/3461154965 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c016a20 con 0x555f8c327180 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efe8b7fe640 1 -- 172.21.15.133:0/3461154965 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efea0029020 con 0x555f8c327180 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 10 client.6564 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 10 client.6564 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 20 client.6564 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x555f8c314e60 0x555f8c315e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 4 client.6564 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 -- 172.21.15.133:0/3461154965 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f8c0c6c70 con 0x555f8c314e60 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb0b61640 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x555f8c314e60 0x555f8c315e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efe8b7fe640 10 client.6564 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb0b61640 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x555f8c314e60 0x555f8c315e60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d5ea4a74bc498e server_cookie=70ab37b32f006e8e in_seq=0 out_seq=0 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efe8b7fe640 1 -- 172.21.15.133:0/3461154965 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555f8c0c6c70 con 0x555f8c314e60 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efe8b7fe640 10 client.6564 handle_command_reply: tid=0 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.694+0000 7efeb5f32000 1 client.6564 shutdown 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efeb5f32000 1 -- 172.21.15.133:0/3461154965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 msgr2=0x555f8c2e2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efeb5f32000 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2e2bb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efea00135b0 tx=0x7efea0013a70 comp rx=0 tx=0).stop 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efeb5f32000 1 -- 172.21.15.133:0/3461154965 shutdown_connections 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efeb5f32000 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x555f8c314e60 0x555f8c315e60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efeb5f32000 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c327aa0 0x555f8c315450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efe8b7fe640 0 client.6564 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efeb5f32000 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2e2bb0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efeb5f32000 1 --2- 172.21.15.133:0/3461154965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2e2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efeb5f32000 1 -- 172.21.15.133:0/3461154965 >> 172.21.15.133:0/3461154965 conn(0x555f8c2e3760 msgr2=0x555f8c2dc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efeb5f32000 1 -- 172.21.15.133:0/3461154965 shutdown_connections 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efeb5f32000 1 -- 172.21.15.133:0/3461154965 wait complete. 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.698+0000 7efeb5f32000 20 client.6564 trim_cache size 0 max 16384 2024-04-25T10:06:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.706+0000 7efeb5f32000 1 Processor -- start 2024-04-25T10:06:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- start start 2024-04-25T10:06:13.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c2e3760 0x555f8c33d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2dc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2dc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac200b20 con 0x555f8c327180 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac1ffce0 con 0x555f8c2e3760 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac1ff0e0 con 0x555f8c2dd600 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb1362640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2dc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb1362640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2dc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40554/0 (socket says 172.21.15.133:40554) 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb1362640 1 -- 172.21.15.133:0/920196934 learned_addr learned my addr 172.21.15.133:0/920196934 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb1362640 1 -- 172.21.15.133:0/920196934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 msgr2=0x555f8c2dc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb1362640 1 --2- 172.21.15.133:0/920196934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2dc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb1362640 1 -- 172.21.15.133:0/920196934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c2e3760 msgr2=0x555f8c33d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb1362640 1 --2- 172.21.15.133:0/920196934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c2e3760 0x555f8c33d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb1362640 1 -- 172.21.15.133:0/920196934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f8c3161d0 con 0x555f8c327180 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb1362640 1 --2- 172.21.15.133:0/920196934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2dc660 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7efea401c710 tx=0x7efea40ee2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a536e96bdb54323b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efe91ffb640 1 -- 172.21.15.133:0/920196934 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea40ee8a0 con 0x555f8c327180 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efe91ffb640 1 -- 172.21.15.133:0/920196934 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea40a9040 con 0x555f8c327180 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efe91ffb640 1 -- 172.21.15.133:0/920196934 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea40f29f0 con 0x555f8c327180 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- 172.21.15.133:0/920196934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 msgr2=0x555f8c2dc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 --2- 172.21.15.133:0/920196934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2dc660 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7efea401c710 tx=0x7efea40ee2c0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- 172.21.15.133:0/920196934 shutdown_connections 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 --2- 172.21.15.133:0/920196934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c327180 0x555f8c2dc660 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 --2- 172.21.15.133:0/920196934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c2dd600 0x555f8c2dc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 --2- 172.21.15.133:0/920196934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c2e3760 0x555f8c33d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- 172.21.15.133:0/920196934 >> 172.21.15.133:0/920196934 conn(0x555f8c3155b0 msgr2=0x555f8c334770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- 172.21.15.133:0/920196934 shutdown_connections 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- 172.21.15.133:0/920196934 wait complete. 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- start start 2024-04-25T10:06:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 Processor -- start 2024-04-25T10:06:13.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c2dd600 0x555f8c387670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c2e3760 0x555f8c387bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c327180 0x555f8c33c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8c1e89c0 con 0x555f8c2e3760 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac1ff6e0 con 0x555f8c2dd600 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f8c0ecee0 con 0x555f8c327180 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb5f32000 10 client.0 _ll_register_callbacks cb 0x555f8c2a9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c2e3760 0x555f8c387bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c2e3760 0x555f8c387bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40570/0 (socket says 172.21.15.133:40570) 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeabfff640 1 -- 172.21.15.133:0/567622437 learned_addr learned my addr 172.21.15.133:0/567622437 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb0b61640 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c2dd600 0x555f8c387670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeabfff640 1 -- 172.21.15.133:0/567622437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c327180 msgr2=0x555f8c33c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeabfff640 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c327180 0x555f8c33c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeabfff640 1 -- 172.21.15.133:0/567622437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c2dd600 msgr2=0x555f8c387670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeabfff640 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c2dd600 0x555f8c387670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeabfff640 1 -- 172.21.15.133:0/567622437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f8c0b3250 con 0x555f8c2e3760 2024-04-25T10:06:13.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.710+0000 7efeb0b61640 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c2dd600 0x555f8c387670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeabfff640 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c2e3760 0x555f8c387bb0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7efea0013f70 tx=0x7efea00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec30663b0a66cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efe8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efe8b7fe640 1 -- 172.21.15.133:0/567622437 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea001b070 con 0x555f8c2e3760 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efe8b7fe640 1 -- 172.21.15.133:0/567622437 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efea002bc30 con 0x555f8c2e3760 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 -- 172.21.15.133:0/567622437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efea40a9040 con 0x555f8c2e3760 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efe8b7fe640 1 -- 172.21.15.133:0/567622437 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7efea000f040 con 0x555f8c2e3760 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 10 client.6570 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 -- 172.21.15.133:0/567622437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x555f8c0bfb90 con 0x555f8c2e3760 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efe8b7fe640 1 -- 172.21.15.133:0/567622437 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea40a9040 con 0x555f8c2e3760 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efe8b7fe640 1 -- 172.21.15.133:0/567622437 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efea0029040 con 0x555f8c2e3760 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 10 client.6570 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 10 client.6570 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 20 client.6570 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:13.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x555f8c327aa0 0x555f8c33d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 4 client.6570 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 -- 172.21.15.133:0/567622437 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555f8c138460 con 0x555f8c327aa0 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb0b61640 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x555f8c327aa0 0x555f8c33d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb0b61640 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x555f8c327aa0 0x555f8c33d050 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6093ed7f9d80e10 server_cookie=4270c18ee7a52970 in_seq=0 out_seq=0 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efe8b7fe640 10 client.6570 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efe8b7fe640 1 -- 172.21.15.133:0/567622437 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555f8c138460 con 0x555f8c327aa0 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efe8b7fe640 10 client.6570 handle_command_reply: tid=0 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 client.6570 shutdown 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 -- 172.21.15.133:0/567622437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c2e3760 msgr2=0x555f8c387bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c2e3760 0x555f8c387bb0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7efea0013f70 tx=0x7efea00046d0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 -- 172.21.15.133:0/567622437 shutdown_connections 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x555f8c327aa0 0x555f8c33d050 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555f8c327180 0x555f8c33c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555f8c2e3760 0x555f8c387bb0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efe8b7fe640 0 client.6570 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:13.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 --2- 172.21.15.133:0/567622437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555f8c2dd600 0x555f8c387670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 -- 172.21.15.133:0/567622437 >> 172.21.15.133:0/567622437 conn(0x555f8c3155b0 msgr2=0x555f8c2db110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 -- 172.21.15.133:0/567622437 shutdown_connections 2024-04-25T10:06:13.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 1 -- 172.21.15.133:0/567622437 wait complete. 2024-04-25T10:06:13.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb5f32000 20 client.6570 trim_cache size 0 max 16384 2024-04-25T10:06:13.724 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 -- 172.21.15.133:0/3434269528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe98066f60 msgr2=0x7efe98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe98066f60 0x7efe98069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efe9c007520 tx=0x7efe9c00d040 comp rx=0 tx=0).stop 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 -- 172.21.15.133:0/3434269528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac138c10 msgr2=0x7efeac139090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac138c10 0x7efeac139090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7efea40ab040 tx=0x7efe94005840 comp rx=0 tx=0).stop 2024-04-25T10:06:13.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 -- 172.21.15.133:0/3434269528 shutdown_connections 2024-04-25T10:06:13.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe98066f60 0x7efe98069420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac138c10 0x7efeac139090 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeac0ad6f0 0x7efeac139c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 --2- 172.21.15.133:0/3434269528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeac0ad310 0x7efeac1f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 -- 172.21.15.133:0/3434269528 >> 172.21.15.133:0/3434269528 conn(0x7efeac066f70 msgr2=0x7efeac0b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 -- 172.21.15.133:0/3434269528 shutdown_connections 2024-04-25T10:06:13.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.714+0000 7efeb2dc5640 1 -- 172.21.15.133:0/3434269528 wait complete. 2024-04-25T10:06:13.733 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4b.00000000 parent 2024-04-25T10:06:13.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.758+0000 7f0dd12a8080 1 Processor -- start 2024-04-25T10:06:13.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.758+0000 7f0dd12a8080 1 -- start start 2024-04-25T10:06:13.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.758+0000 7f0dd12a8080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564658d27330 0x564658db3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.758+0000 7f0dd12a8080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564658db2330 0x564658db4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.758+0000 7f0dd12a8080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2710 0x564658d07c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.758+0000 7f0dd12a8080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564658db3840 con 0x564658db2710 2024-04-25T10:06:13.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.758+0000 7f0dd12a8080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564658c839f0 con 0x564658d27330 2024-04-25T10:06:13.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.758+0000 7f0dd12a8080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564658db39c0 con 0x564658db2330 2024-04-25T10:06:13.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcf5c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2710 0x564658d07c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcf5c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2710 0x564658d07c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40580/0 (socket says 172.21.15.133:40580) 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcf5c7640 1 -- 172.21.15.133:0/524218717 learned_addr learned my addr 172.21.15.133:0/524218717 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcedc6640 1 --2- 172.21.15.133:0/524218717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564658d27330 0x564658db3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dce5c5640 1 --2- 172.21.15.133:0/524218717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564658db2330 0x564658db4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcf5c7640 1 -- 172.21.15.133:0/524218717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564658db2330 msgr2=0x564658db4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcf5c7640 1 --2- 172.21.15.133:0/524218717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564658db2330 0x564658db4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcf5c7640 1 -- 172.21.15.133:0/524218717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564658d27330 msgr2=0x564658db3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcf5c7640 1 --2- 172.21.15.133:0/524218717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564658d27330 0x564658db3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcf5c7640 1 -- 172.21.15.133:0/524218717 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564658d1e390 con 0x564658db2710 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcf5c7640 1 --2- 172.21.15.133:0/524218717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2710 0x564658d07c50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0dc801b5c0 tx=0x7f0dc80033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fe5a63abc7a939 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcd5c3640 1 -- 172.21.15.133:0/524218717 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80af070 con 0x564658db2710 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcd5c3640 1 -- 172.21.15.133:0/524218717 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc80b9070 con 0x564658db2710 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcd5c3640 1 -- 172.21.15.133:0/524218717 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80aad10 con 0x564658db2710 2024-04-25T10:06:13.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 -- 172.21.15.133:0/524218717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2710 msgr2=0x564658d07c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 --2- 172.21.15.133:0/524218717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2710 0x564658d07c50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0dc801b5c0 tx=0x7f0dc80033b0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 -- 172.21.15.133:0/524218717 shutdown_connections 2024-04-25T10:06:13.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 --2- 172.21.15.133:0/524218717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2710 0x564658d07c50 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 --2- 172.21.15.133:0/524218717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564658db2330 0x564658db4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 --2- 172.21.15.133:0/524218717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564658d27330 0x564658db3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 -- 172.21.15.133:0/524218717 >> 172.21.15.133:0/524218717 conn(0x564658cdca00 msgr2=0x564658cdbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 -- 172.21.15.133:0/524218717 shutdown_connections 2024-04-25T10:06:13.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 -- 172.21.15.133:0/524218717 wait complete. 2024-04-25T10:06:13.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 Processor -- start 2024-04-25T10:06:13.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 -- start start 2024-04-25T10:06:13.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564658d27330 0x564658db0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2330 0x564658cdfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564658db2710 0x564658db01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564658d1e630 con 0x564658db2330 2024-04-25T10:06:13.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564658d937e0 con 0x564658d27330 2024-04-25T10:06:13.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564658d098c0 con 0x564658db2710 2024-04-25T10:06:13.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dce5c5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2330 0x564658cdfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dce5c5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2330 0x564658cdfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40592/0 (socket says 172.21.15.133:40592) 2024-04-25T10:06:13.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dce5c5640 1 -- 172.21.15.133:0/2411784155 learned_addr learned my addr 172.21.15.133:0/2411784155 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcf5c7640 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564658db2710 0x564658db01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcedc6640 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564658d27330 0x564658db0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dce5c5640 1 -- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564658db2710 msgr2=0x564658db01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dce5c5640 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564658db2710 0x564658db01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dce5c5640 1 -- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564658d27330 msgr2=0x564658db0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dce5c5640 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564658d27330 0x564658db0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dce5c5640 1 -- 172.21.15.133:0/2411784155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564658d1fa60 con 0x564658db2330 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dcedc6640 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564658d27330 0x564658db0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dce5c5640 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2330 0x564658cdfdd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005fd0 tx=0x7f0db8007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dc7fff640 1 -- 172.21.15.133:0/2411784155 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800b0d0 con 0x564658db2330 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dc7fff640 1 -- 172.21.15.133:0/2411784155 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db800b270 con 0x564658db2330 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 -- 172.21.15.133:0/2411784155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564658dbc620 con 0x564658db2330 2024-04-25T10:06:13.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dc7fff640 1 -- 172.21.15.133:0/2411784155 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8006b70 con 0x564658db2330 2024-04-25T10:06:13.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.762+0000 7f0dd12a8080 1 -- 172.21.15.133:0/2411784155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564658dbcb60 con 0x564658db2330 2024-04-25T10:06:13.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.766+0000 7f0dc7fff640 1 -- 172.21.15.133:0/2411784155 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0db8006d10 con 0x564658db2330 2024-04-25T10:06:13.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.766+0000 7f0dc7fff640 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0db0065970 0x7f0db0067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.766+0000 7f0dc7fff640 1 -- 172.21.15.133:0/2411784155 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0db80839d0 con 0x564658db2330 2024-04-25T10:06:13.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.766+0000 7f0dcedc6640 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0db0065970 0x7f0db0067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.766+0000 7f0dd12a8080 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x564658db14a0 0x564658cdef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.766+0000 7f0dd12a8080 1 -- 172.21.15.133:0/2411784155 --> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] -- osd_op(unknown.0.0:1 8.24 8:256cebcc:::10000000a4b.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) v9 -- 0x564658ce0310 con 0x564658db14a0 2024-04-25T10:06:13.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.766+0000 7f0dcedc6640 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0db0065970 0x7f0db0067e30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0000f70 tx=0x7f0dc00014f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.766+0000 7f0dcf5c7640 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x564658db14a0 0x564658cdef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.766+0000 7f0dcf5c7640 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x564658db14a0 0x564658cdef20 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dcf5c7640 1 -- 172.21.15.133:0/2411784155 <== osd.6 v2:172.21.15.162:6824/2819373209 1 ==== osd_op_reply(1 10000000a4b.00000000 [rmxattr] v54'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0dc80b2070 con 0x564658db14a0 2024-04-25T10:06:13.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dd12a8080 1 -- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x564658db14a0 msgr2=0x564658cdef20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dd12a8080 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x564658db14a0 0x564658cdef20 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dd12a8080 1 -- 172.21.15.133:0/2411784155 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0db0065970 msgr2=0x7f0db0067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dd12a8080 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0db0065970 0x7f0db0067e30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0000f70 tx=0x7f0dc00014f0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dd12a8080 1 -- 172.21.15.133:0/2411784155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2330 msgr2=0x564658cdfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dd12a8080 1 --2- 172.21.15.133:0/2411784155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564658db2330 0x564658cdfdd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005fd0 tx=0x7f0db8007b10 comp rx=0 tx=0).stop 2024-04-25T10:06:13.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dcf5c7640 1 -- 172.21.15.133:0/2411784155 reap_dead start 2024-04-25T10:06:13.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dd12a8080 1 -- 172.21.15.133:0/2411784155 shutdown_connections 2024-04-25T10:06:13.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dd12a8080 1 -- 172.21.15.133:0/2411784155 >> 172.21.15.133:0/2411784155 conn(0x564658cdca00 msgr2=0x564658db5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dd12a8080 1 -- 172.21.15.133:0/2411784155 shutdown_connections 2024-04-25T10:06:13.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.770+0000 7f0dd12a8080 1 -- 172.21.15.133:0/2411784155 wait complete. 2024-04-25T10:06:13.776 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T10:06:13.776 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:13.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T10:06:13.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f97abe640 1 Processor -- start 2024-04-25T10:06:13.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f97abe640 1 -- start start 2024-04-25T10:06:13.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f97abe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8f90145170 0x7f8f9006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f97abe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8f9006e4d0 0x7f8f900cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f97abe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900cdd60 0x7f8f9015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f97abe640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9006db00 con 0x7f8f9006e4d0 2024-04-25T10:06:13.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f97abe640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f90145550 con 0x7f8f90145170 2024-04-25T10:06:13.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f97abe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900ce2e0 con 0x7f8f900cdd60 2024-04-25T10:06:13.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f95059640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8f9006e4d0 0x7f8f900cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f95059640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8f9006e4d0 0x7f8f900cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40598/0 (socket says 172.21.15.133:40598) 2024-04-25T10:06:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f95059640 1 -- 172.21.15.133:0/102516899 learned_addr learned my addr 172.21.15.133:0/102516899 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f9585a640 1 --2- 172.21.15.133:0/102516899 >> v2:172.21.15.162:3300/0 conn(0x7f8f90145170 0x7f8f9006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f9605b640 1 --2- 172.21.15.133:0/102516899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900cdd60 0x7f8f9015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f95059640 1 -- 172.21.15.133:0/102516899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900cdd60 msgr2=0x7f8f9015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f95059640 1 --2- 172.21.15.133:0/102516899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900cdd60 0x7f8f9015ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f95059640 1 -- 172.21.15.133:0/102516899 >> v2:172.21.15.162:3300/0 conn(0x7f8f90145170 msgr2=0x7f8f9006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f95059640 1 --2- 172.21.15.133:0/102516899 >> v2:172.21.15.162:3300/0 conn(0x7f8f90145170 0x7f8f9006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f95059640 1 -- 172.21.15.133:0/102516899 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9006d4b0 con 0x7f8f9006e4d0 2024-04-25T10:06:13.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.938+0000 7f8f95059640 1 --2- 172.21.15.133:0/102516899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f9006e4d0 0x7f8f900cd820 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8f9006b790 tx=0x7f8f8001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6917175f31d9ab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f94858640 1 -- 172.21.15.133:0/102516899 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f800b0070 con 0x7f8f9006e4d0 2024-04-25T10:06:13.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f94858640 1 -- 172.21.15.133:0/102516899 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f800ab600 con 0x7f8f9006e4d0 2024-04-25T10:06:13.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f94858640 1 -- 172.21.15.133:0/102516899 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f800ab9d0 con 0x7f8f9006e4d0 2024-04-25T10:06:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 -- 172.21.15.133:0/102516899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f9006e4d0 msgr2=0x7f8f900cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 --2- 172.21.15.133:0/102516899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f9006e4d0 0x7f8f900cd820 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8f9006b790 tx=0x7f8f8001c520 comp rx=0 tx=0).stop 2024-04-25T10:06:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 -- 172.21.15.133:0/102516899 shutdown_connections 2024-04-25T10:06:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 --2- 172.21.15.133:0/102516899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f900cdd60 0x7f8f9015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 --2- 172.21.15.133:0/102516899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f9006e4d0 0x7f8f900cd820 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 --2- 172.21.15.133:0/102516899 >> v2:172.21.15.162:3300/0 conn(0x7f8f90145170 0x7f8f9006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 -- 172.21.15.133:0/102516899 >> 172.21.15.133:0/102516899 conn(0x7f8f90066f70 msgr2=0x7f8f9006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 -- 172.21.15.133:0/102516899 shutdown_connections 2024-04-25T10:06:13.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 -- 172.21.15.133:0/102516899 wait complete. 2024-04-25T10:06:13.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 Processor -- start 2024-04-25T10:06:13.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 -- start start 2024-04-25T10:06:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f900cdd60 0x7f8f90138550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f90145170 0x7f8f90151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f9013edc0 0x7f8f90140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9015f690 con 0x7f8f900cdd60 2024-04-25T10:06:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9015f510 con 0x7f8f9013edc0 2024-04-25T10:06:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f97abe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9015f810 con 0x7f8f90145170 2024-04-25T10:06:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f9585a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f900cdd60 0x7f8f90138550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f9585a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f900cdd60 0x7f8f90138550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40614/0 (socket says 172.21.15.133:40614) 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f9585a640 1 -- 172.21.15.133:0/3744621705 learned_addr learned my addr 172.21.15.133:0/3744621705 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f95059640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f90145170 0x7f8f90151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f9585a640 1 -- 172.21.15.133:0/3744621705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f90145170 msgr2=0x7f8f90151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f9585a640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f90145170 0x7f8f90151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f9585a640 1 -- 172.21.15.133:0/3744621705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f9013edc0 msgr2=0x7f8f90140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f9605b640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f9013edc0 0x7f8f90140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f9585a640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f9013edc0 0x7f8f90140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f9585a640 1 -- 172.21.15.133:0/3744621705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f90139900 con 0x7f8f900cdd60 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f9585a640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f900cdd60 0x7f8f90138550 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00d620 tx=0x7f8f8c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f867fc640 1 -- 172.21.15.133:0/3744621705 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c00b620 con 0x7f8f900cdd60 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f867fc640 1 -- 172.21.15.133:0/3744621705 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c019070 con 0x7f8f900cdd60 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.942+0000 7f8f867fc640 1 -- 172.21.15.133:0/3744621705 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c00b9e0 con 0x7f8f900cdd60 2024-04-25T10:06:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f97abe640 1 -- 172.21.15.133:0/3744621705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f901370b0 con 0x7f8f900cdd60 2024-04-25T10:06:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f97abe640 1 -- 172.21.15.133:0/3744621705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f90139060 con 0x7f8f900cdd60 2024-04-25T10:06:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f867fc640 1 -- 172.21.15.133:0/3744621705 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f8c00bb80 con 0x7f8f900cdd60 2024-04-25T10:06:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f867fc640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f74066f60 0x7f8f74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f867fc640 1 -- 172.21.15.133:0/3744621705 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f8c0855d0 con 0x7f8f900cdd60 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f95059640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f74066f60 0x7f8f74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f95059640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f74066f60 0x7f8f74069420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8f800ad000 tx=0x7f8f80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f9ac2b000 1 Processor -- start 2024-04-25T10:06:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f9ac2b000 1 -- start start 2024-04-25T10:06:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 0x55bdb885d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb88103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb8810920 0x55bdb8810d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb8660ce0 con 0x55bdb885db10 2024-04-25T10:06:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb85e9530 con 0x55bdb8810920 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdb865cd80 con 0x55bdb885d1f0 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f95059640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb88103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f95059640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb88103e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40618/0 (socket says 172.21.15.133:40618) 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.946+0000 7f8f95059640 1 -- 172.21.15.133:0/3352185986 learned_addr learned my addr 172.21.15.133:0/3352185986 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9585a640 1 --2- 172.21.15.133:0/3352185986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 0x55bdb885d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9605b640 1 --2- 172.21.15.133:0/3352185986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb8810920 0x55bdb8810d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 -- 172.21.15.133:0/3352185986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 msgr2=0x55bdb885d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 --2- 172.21.15.133:0/3352185986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 0x55bdb885d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 -- 172.21.15.133:0/3352185986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb8810920 msgr2=0x55bdb8810d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 --2- 172.21.15.133:0/3352185986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb8810920 0x55bdb8810d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 -- 172.21.15.133:0/3352185986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdb865bcd0 con 0x55bdb885db10 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9585a640 1 --2- 172.21.15.133:0/3352185986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 0x55bdb885d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9605b640 1 --2- 172.21.15.133:0/3352185986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb8810920 0x55bdb8810d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 --2- 172.21.15.133:0/3352185986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb88103e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8f8001a150 tx=0x7f8f80004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820b47b15cf668b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f737fe640 1 -- 172.21.15.133:0/3352185986 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f800b0040 con 0x55bdb885db10 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f737fe640 1 -- 172.21.15.133:0/3352185986 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f800c5070 con 0x55bdb885db10 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f737fe640 1 -- 172.21.15.133:0/3352185986 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f800b6420 con 0x55bdb885db10 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3352185986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 msgr2=0x55bdb88103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/3352185986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb88103e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8f8001a150 tx=0x7f8f80004740 comp rx=0 tx=0).stop 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3352185986 shutdown_connections 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/3352185986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb8810920 0x55bdb8810d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/3352185986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb88103e0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/3352185986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 0x55bdb885d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3352185986 >> 172.21.15.133:0/3352185986 conn(0x55bdb883df90 msgr2=0x55bdb8840380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3352185986 shutdown_connections 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3352185986 wait complete. 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- start start 2024-04-25T10:06:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 Processor -- start 2024-04-25T10:06:13.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb8810920 0x55bdb8864760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885d1f0 0x55bdb8864ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb885db10 0x55bdb88169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb861bbc0 con 0x55bdb885d1f0 2024-04-25T10:06:13.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb8665310 con 0x55bdb885db10 2024-04-25T10:06:13.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdb865fff0 con 0x55bdb8810920 2024-04-25T10:06:13.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 10 client.0 _ll_register_callbacks cb 0x55bdb87fe4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885d1f0 0x55bdb8864ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885d1f0 0x55bdb8864ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40624/0 (socket says 172.21.15.133:40624) 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 -- 172.21.15.133:0/1497969927 learned_addr learned my addr 172.21.15.133:0/1497969927 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9605b640 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb885db10 0x55bdb88169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9585a640 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb8810920 0x55bdb8864760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 -- 172.21.15.133:0/1497969927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb8810920 msgr2=0x55bdb8864760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb8810920 0x55bdb8864760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 -- 172.21.15.133:0/1497969927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb885db10 msgr2=0x55bdb88169a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb885db10 0x55bdb88169a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 -- 172.21.15.133:0/1497969927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdb866e950 con 0x55bdb885d1f0 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9585a640 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb8810920 0x55bdb8864760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f95059640 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885d1f0 0x55bdb8864ca0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8f800c2040 tx=0x7f8f8001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1eac25e8f554e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:13.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f70ff9640 1 -- 172.21.15.133:0/1497969927 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f800cf070 con 0x55bdb885d1f0 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1497969927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f800c5070 con 0x55bdb885d1f0 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f70ff9640 1 -- 172.21.15.133:0/1497969927 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f800b68e0 con 0x55bdb885d1f0 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f70ff9640 1 -- 172.21.15.133:0/1497969927 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f800da050 con 0x55bdb885d1f0 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 10 client.6594 fetch_fsmap learned FSMap version 24 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1497969927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55bdb86dd790 con 0x55bdb885d1f0 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f70ff9640 1 -- 172.21.15.133:0/1497969927 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f800c5070 con 0x55bdb885d1f0 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f70ff9640 1 -- 172.21.15.133:0/1497969927 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f800dd020 con 0x55bdb885d1f0 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 10 client.6594 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 10 client.6594 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 20 client.6594 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bdb8817740 0x55bdb8817b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 4 client.6594 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.950+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1497969927 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdb85fcb80 con 0x55bdb8817740 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9585a640 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bdb8817740 0x55bdb8817b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f70ff9640 10 client.6594 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9585a640 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bdb8817740 0x55bdb8817b20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4518bf7ba47cbd server_cookie=a9aecc1b7efbcdb1 in_seq=0 out_seq=0 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f70ff9640 1 -- 172.21.15.133:0/1497969927 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bdb85fcb80 con 0x55bdb8817740 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f70ff9640 10 client.6594 handle_command_reply: tid=0 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 1 client.6594 shutdown 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1497969927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885d1f0 msgr2=0x55bdb8864ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885d1f0 0x55bdb8864ca0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8f800c2040 tx=0x7f8f8001ad60 comp rx=0 tx=0).stop 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1497969927 shutdown_connections 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bdb8817740 0x55bdb8817b20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb885db10 0x55bdb88169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f70ff9640 0 client.6594 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885d1f0 0x55bdb8864ca0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/1497969927 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb8810920 0x55bdb8864760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1497969927 >> 172.21.15.133:0/1497969927 conn(0x55bdb883df90 msgr2=0x55bdb880fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1497969927 shutdown_connections 2024-04-25T10:06:13.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1497969927 wait complete. 2024-04-25T10:06:13.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.954+0000 7f8f9ac2b000 20 client.6594 trim_cache size 0 max 16384 2024-04-25T10:06:13.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 Processor -- start 2024-04-25T10:06:13.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 -- start start 2024-04-25T10:06:13.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb8810920 0x55bdb885c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb885d1f0 0x55bdb885cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885db10 0x55bdb8818b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90201940 con 0x55bdb8810920 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90200b00 con 0x55bdb885d1f0 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f901fff60 con 0x55bdb885db10 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9585a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb8810920 0x55bdb885c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9585a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb8810920 0x55bdb885c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40634/0 (socket says 172.21.15.133:40634) 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9585a640 1 -- 172.21.15.133:0/1087563496 learned_addr learned my addr 172.21.15.133:0/1087563496 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9585a640 1 -- 172.21.15.133:0/1087563496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885db10 msgr2=0x55bdb8818b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9585a640 1 --2- 172.21.15.133:0/1087563496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885db10 0x55bdb8818b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9585a640 1 -- 172.21.15.133:0/1087563496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb885d1f0 msgr2=0x55bdb885cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f95059640 1 --2- 172.21.15.133:0/1087563496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb885d1f0 0x55bdb885cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9585a640 1 --2- 172.21.15.133:0/1087563496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb885d1f0 0x55bdb885cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9585a640 1 -- 172.21.15.133:0/1087563496 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdb86266f0 con 0x55bdb8810920 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9585a640 1 --2- 172.21.15.133:0/1087563496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb8810920 0x55bdb885c6d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c056600 tx=0x7f8f8c085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbc6c951a0b0491 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f72ffd640 1 -- 172.21.15.133:0/1087563496 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c05e460 con 0x55bdb8810920 2024-04-25T10:06:13.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f72ffd640 1 -- 172.21.15.133:0/1087563496 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c053df0 con 0x55bdb8810920 2024-04-25T10:06:13.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f72ffd640 1 -- 172.21.15.133:0/1087563496 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c05c0b0 con 0x55bdb8810920 2024-04-25T10:06:13.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1087563496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb8810920 msgr2=0x55bdb885c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/1087563496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb8810920 0x55bdb885c6d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c056600 tx=0x7f8f8c085040 comp rx=0 tx=0).stop 2024-04-25T10:06:13.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1087563496 shutdown_connections 2024-04-25T10:06:13.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/1087563496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885db10 0x55bdb8818b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/1087563496 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb885d1f0 0x55bdb885cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/1087563496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb8810920 0x55bdb885c6d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1087563496 >> 172.21.15.133:0/1087563496 conn(0x55bdb883df90 msgr2=0x55bdb886a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1087563496 shutdown_connections 2024-04-25T10:06:13.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/1087563496 wait complete. 2024-04-25T10:06:13.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 -- start start 2024-04-25T10:06:13.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.970+0000 7f8f9ac2b000 1 Processor -- start 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb8810920 0x55bdb885c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 0x55bdb885ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb8860440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdb865cd80 con 0x55bdb885db10 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90200500 con 0x55bdb8810920 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdb8665310 con 0x55bdb885d1f0 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 10 client.0 _ll_register_callbacks cb 0x7f8f800ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9605b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb8860440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9605b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb8860440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40650/0 (socket says 172.21.15.133:40650) 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9605b640 1 -- 172.21.15.133:0/3250000638 learned_addr learned my addr 172.21.15.133:0/3250000638 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9585a640 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb8810920 0x55bdb885c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f95059640 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 0x55bdb885ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9605b640 1 -- 172.21.15.133:0/3250000638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 msgr2=0x55bdb885ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9605b640 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 0x55bdb885ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9605b640 1 -- 172.21.15.133:0/3250000638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb8810920 msgr2=0x55bdb885c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9605b640 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb8810920 0x55bdb885c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9605b640 1 -- 172.21.15.133:0/3250000638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdb861cc80 con 0x55bdb885db10 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f95059640 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 0x55bdb885ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9605b640 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb8860440 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800e690 tx=0x7f8f8800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bcf2e8aeeccfdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f70ff9640 1 -- 172.21.15.133:0/3250000638 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8800cb40 con 0x55bdb885db10 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f70ff9640 1 -- 172.21.15.133:0/3250000638 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f8800cce0 con 0x55bdb885db10 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3250000638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f8c053df0 con 0x55bdb885db10 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f70ff9640 1 -- 172.21.15.133:0/3250000638 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f88023050 con 0x55bdb885db10 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 10 client.6600 fetch_fsmap learned FSMap version 25 2024-04-25T10:06:13.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3250000638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55bdb831d600 con 0x55bdb885db10 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f70ff9640 1 -- 172.21.15.133:0/3250000638 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c053df0 con 0x55bdb885db10 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f70ff9640 1 -- 172.21.15.133:0/3250000638 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f88026020 con 0x55bdb885db10 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 10 client.6600 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 10 client.6600 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 20 client.6600 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bdb88bd090 0x55bdb88bd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 4 client.6600 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3250000638 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x55bdb883dde0 con 0x55bdb88bd090 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9585a640 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bdb88bd090 0x55bdb88bd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f70ff9640 10 client.6600 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9585a640 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bdb88bd090 0x55bdb88bd4a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e178a1d367b17967 server_cookie=10b46d612823e95 in_seq=0 out_seq=0 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f70ff9640 1 -- 172.21.15.133:0/3250000638 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bdb883dde0 con 0x55bdb88bd090 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f70ff9640 10 client.6600 handle_command_reply: tid=0 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 client.6600 shutdown 2024-04-25T10:06:13.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3250000638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 msgr2=0x55bdb8860440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb8860440 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800e690 tx=0x7f8f8800eb50 comp rx=0 tx=0).stop 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3250000638 shutdown_connections 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55bdb88bd090 0x55bdb88bd4a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdb885db10 0x55bdb8860440 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bdb885d1f0 0x55bdb885ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f70ff9640 0 client.6600 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 --2- 172.21.15.133:0/3250000638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bdb8810920 0x55bdb885c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3250000638 >> 172.21.15.133:0/3250000638 conn(0x55bdb883df90 msgr2=0x55bdb886aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3250000638 shutdown_connections 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 1 -- 172.21.15.133:0/3250000638 wait complete. 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f9ac2b000 20 client.6600 trim_cache size 0 max 16384 2024-04-25T10:06:13.985 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:13.986 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:06:13.986 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "9492a7c9-5205-4f46-95ce-f911ed40c076", 2024-04-25T10:06:13.986 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:06:13.986 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:13.986 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f00d6c35640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4b [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v736 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6207=0-4194304@1} caps={6207=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=6207 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55607d5d1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a4b:[<0x10000000001/new_file_1 v736>,<0x10000000000/ceph-qa-suite v94>,<0x1/scrub_test_path v46>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T10:06:13.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 -- 172.21.15.133:0/3744621705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f74066f60 msgr2=0x7f8f74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f74066f60 0x7f8f74069420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8f800ad000 tx=0x7f8f80008040 comp rx=0 tx=0).stop 2024-04-25T10:06:13.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 -- 172.21.15.133:0/3744621705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f900cdd60 msgr2=0x7f8f90138550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f900cdd60 0x7f8f90138550 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00d620 tx=0x7f8f8c00dae0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 -- 172.21.15.133:0/3744621705 shutdown_connections 2024-04-25T10:06:13.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f9013edc0 0x7f8f90140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f90145170 0x7f8f90151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f74066f60 0x7f8f74069420 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 --2- 172.21.15.133:0/3744621705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f900cdd60 0x7f8f90138550 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 -- 172.21.15.133:0/3744621705 >> 172.21.15.133:0/3744621705 conn(0x7f8f90066f70 msgr2=0x7f8f900b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 -- 172.21.15.133:0/3744621705 shutdown_connections 2024-04-25T10:06:13.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:13.974+0000 7f8f97abe640 1 -- 172.21.15.133:0/3744621705 wait complete. 2024-04-25T10:06:13.994 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '9492a7c9-5205-4f46-95ce-f911ed40c076', 'mode': 'asynchronous'}' 2024-04-25T10:06:13.995 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:06:13.995 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:06:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.102+0000 7fbacefaa640 1 Processor -- start 2024-04-25T10:06:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.102+0000 7fbacefaa640 1 -- start start 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.102+0000 7fbacefaa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbac8145170 0x7fbac806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbacefaa640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbac815ced0 0x7fbac815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbacefaa640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac80ce2e0 con 0x7fbac806e4d0 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbacefaa640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac806db00 con 0x7fbac815ced0 2024-04-25T10:06:14.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbacefaa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac8145550 con 0x7fbac8145170 2024-04-25T10:06:14.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbacd547640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbac815ced0 0x7fbac815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbaccd46640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbac8145170 0x7fbac806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbacd547640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbac815ced0 0x7fbac815d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:46586/0 (socket says 172.21.15.133:46586) 2024-04-25T10:06:14.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbacd547640 1 -- 172.21.15.133:0/2053102086 learned_addr learned my addr 172.21.15.133:0/2053102086 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:14.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbaccd46640 1 -- 172.21.15.133:0/2053102086 >> v2:172.21.15.162:3300/0 conn(0x7fbac815ced0 msgr2=0x7fbac815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbaccd46640 1 --2- 172.21.15.133:0/2053102086 >> v2:172.21.15.162:3300/0 conn(0x7fbac815ced0 0x7fbac815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbaccd46640 1 -- 172.21.15.133:0/2053102086 >> v1:172.21.15.162:6789/0 conn(0x7fbac806e4d0 legacy=0x7fbac80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbaccd46640 1 -- 172.21.15.133:0/2053102086 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac806d4b0 con 0x7fbac8145170 2024-04-25T10:06:14.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbac77fe640 1 -- 172.21.15.133:0/2053102086 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935817339 0 0) 0x7fbac80ce2e0 con 0x7fbac806e4d0 2024-04-25T10:06:14.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbacd547640 1 --2- 172.21.15.133:0/2053102086 >> v2:172.21.15.162:3300/0 conn(0x7fbac815ced0 0x7fbac815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:14.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbaccd46640 1 --2- 172.21.15.133:0/2053102086 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbac8145170 0x7fbac806df90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbab80097c0 tx=0x7fbab801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12f59cb32a63cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbac77fe640 1 -- 172.21.15.133:0/2053102086 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab80b0070 con 0x7fbac8145170 2024-04-25T10:06:14.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbac77fe640 1 -- 172.21.15.133:0/2053102086 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab80ab600 con 0x7fbac8145170 2024-04-25T10:06:14.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.106+0000 7fbac77fe640 1 -- 172.21.15.133:0/2053102086 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab80ab9d0 con 0x7fbac8145170 2024-04-25T10:06:14.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 -- 172.21.15.133:0/2053102086 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbac8145170 msgr2=0x7fbac806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 --2- 172.21.15.133:0/2053102086 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbac8145170 0x7fbac806df90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbab80097c0 tx=0x7fbab801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 -- 172.21.15.133:0/2053102086 shutdown_connections 2024-04-25T10:06:14.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 --2- 172.21.15.133:0/2053102086 >> v2:172.21.15.162:3300/0 conn(0x7fbac815ced0 0x7fbac815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 --2- 172.21.15.133:0/2053102086 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbac8145170 0x7fbac806df90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 -- 172.21.15.133:0/2053102086 >> 172.21.15.133:0/2053102086 conn(0x7fbac8066f70 msgr2=0x7fbac806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 -- 172.21.15.133:0/2053102086 shutdown_connections 2024-04-25T10:06:14.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 -- 172.21.15.133:0/2053102086 wait complete. 2024-04-25T10:06:14.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 Processor -- start 2024-04-25T10:06:14.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 -- start start 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac806e4d0 0x7fbac8138ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbac8145170 0x7fbac8139300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbac815ced0 0x7fbac8161c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac8168610 con 0x7fbac806e4d0 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac8168490 con 0x7fbac8145170 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac8168790 con 0x7fbac815ced0 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbaccd46640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac806e4d0 0x7fbac8138ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbaccd46640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac806e4d0 0x7fbac8138ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40654/0 (socket says 172.21.15.133:40654) 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbaccd46640 1 -- 172.21.15.133:0/1364235369 learned_addr learned my addr 172.21.15.133:0/1364235369 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacd547640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbac815ced0 0x7fbac8161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbaccd46640 1 -- 172.21.15.133:0/1364235369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbac815ced0 msgr2=0x7fbac8161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbaccd46640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbac815ced0 0x7fbac8161c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbaccd46640 1 -- 172.21.15.133:0/1364235369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbac8145170 msgr2=0x7fbac8139300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbac7fff640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbac8145170 0x7fbac8139300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbaccd46640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbac8145170 0x7fbac8139300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbaccd46640 1 -- 172.21.15.133:0/1364235369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac8139c00 con 0x7fbac806e4d0 2024-04-25T10:06:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbaccd46640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac806e4d0 0x7fbac8138ac0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbab80097c0 tx=0x7fbab801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbac57fa640 1 -- 172.21.15.133:0/1364235369 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab80b0040 con 0x7fbac806e4d0 2024-04-25T10:06:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbac57fa640 1 -- 172.21.15.133:0/1364235369 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab8004030 con 0x7fbac806e4d0 2024-04-25T10:06:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 -- 172.21.15.133:0/1364235369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac8139ee0 con 0x7fbac806e4d0 2024-04-25T10:06:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbac57fa640 1 -- 172.21.15.133:0/1364235369 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8006e20 con 0x7fbac806e4d0 2024-04-25T10:06:14.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.110+0000 7fbacefaa640 1 -- 172.21.15.133:0/1364235369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac813a4a0 con 0x7fbac806e4d0 2024-04-25T10:06:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbac57fa640 1 -- 172.21.15.133:0/1364235369 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbab8003620 con 0x7fbac806e4d0 2024-04-25T10:06:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbac57fa640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbab4066f90 0x7fbab4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbac57fa640 1 -- 172.21.15.133:0/1364235369 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbab811fd50 con 0x7fbac806e4d0 2024-04-25T10:06:14.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbac7fff640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbab4066f90 0x7fbab4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbac7fff640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbab4066f90 0x7fbab4069450 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbac8144b90 tx=0x7fbabc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbad2117000 1 Processor -- start 2024-04-25T10:06:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbad2117000 1 -- start start 2024-04-25T10:06:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbad2117000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359ee30 0x55a75359f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbad2117000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359f750 0x55a753554f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbad2117000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a753555890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbad2117000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a75339dff0 con 0x55a75359ee30 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbad2117000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a753326150 con 0x55a75359f750 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbad2117000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a75339a090 con 0x55a7535554b0 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbaccd46640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359ee30 0x55a75359f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbaccd46640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359ee30 0x55a75359f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40660/0 (socket says 172.21.15.133:40660) 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbaccd46640 1 -- 172.21.15.133:0/1920972026 learned_addr learned my addr 172.21.15.133:0/1920972026 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbacd547640 1 --2- 172.21.15.133:0/1920972026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a753555890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbaccd46640 1 -- 172.21.15.133:0/1920972026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 msgr2=0x55a753555890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbac7fff640 1 --2- 172.21.15.133:0/1920972026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359f750 0x55a753554f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbaccd46640 1 --2- 172.21.15.133:0/1920972026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a753555890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbaccd46640 1 -- 172.21.15.133:0/1920972026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359f750 msgr2=0x55a753554f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbaccd46640 1 --2- 172.21.15.133:0/1920972026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359f750 0x55a753554f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbaccd46640 1 -- 172.21.15.133:0/1920972026 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7535340d0 con 0x55a75359ee30 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbaccd46640 1 --2- 172.21.15.133:0/1920972026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359ee30 0x55a75359f210 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbab80ae040 tx=0x7fbab8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a989e16110328422 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.114+0000 7fbaae7fc640 1 -- 172.21.15.133:0/1920972026 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab80f0600 con 0x55a75359ee30 2024-04-25T10:06:14.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaae7fc640 1 -- 172.21.15.133:0/1920972026 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab80f4750 con 0x55a75359ee30 2024-04-25T10:06:14.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaae7fc640 1 -- 172.21.15.133:0/1920972026 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab80f8d10 con 0x55a75359ee30 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- 172.21.15.133:0/1920972026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359ee30 msgr2=0x55a75359f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 --2- 172.21.15.133:0/1920972026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359ee30 0x55a75359f210 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbab80ae040 tx=0x7fbab8019240 comp rx=0 tx=0).stop 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- 172.21.15.133:0/1920972026 shutdown_connections 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 --2- 172.21.15.133:0/1920972026 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a753555890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 --2- 172.21.15.133:0/1920972026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359f750 0x55a753554f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 --2- 172.21.15.133:0/1920972026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359ee30 0x55a75359f210 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- 172.21.15.133:0/1920972026 >> 172.21.15.133:0/1920972026 conn(0x55a75354d020 msgr2=0x55a75354f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- 172.21.15.133:0/1920972026 shutdown_connections 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- 172.21.15.133:0/1920972026 wait complete. 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- start start 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 Processor -- start 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a75359e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359ee30 0x55a75355b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a75355b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7533a2620 con 0x55a75359f750 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a753358690 con 0x55a75359ee30 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a753326a40 con 0x55a7535554b0 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 10 client.0 _ll_register_callbacks cb 0x55a75353be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbacd547640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a75355b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbacd547640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a75355b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40676/0 (socket says 172.21.15.133:40676) 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbacd547640 1 -- 172.21.15.133:0/787181140 learned_addr learned my addr 172.21.15.133:0/787181140 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbac7fff640 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359ee30 0x55a75355b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaccd46640 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a75359e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbacd547640 1 -- 172.21.15.133:0/787181140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 msgr2=0x55a75359e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbacd547640 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a75359e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbacd547640 1 -- 172.21.15.133:0/787181140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359ee30 msgr2=0x55a75355b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbacd547640 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359ee30 0x55a75355b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbacd547640 1 -- 172.21.15.133:0/787181140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a75344b1c0 con 0x55a75359f750 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaccd46640 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a75359e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbacd547640 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a75355b8a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbac00079a0 tx=0x7fbac0007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3907fbcd94097c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaa77fe640 1 -- 172.21.15.133:0/787181140 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0019030 con 0x55a75359f750 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaa77fe640 1 -- 172.21.15.133:0/787181140 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbac0012040 con 0x55a75359f750 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- 172.21.15.133:0/787181140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbab80f4750 con 0x55a75359f750 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaa77fe640 1 -- 172.21.15.133:0/787181140 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbac0022050 con 0x55a75359f750 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 10 client.6609 fetch_fsmap learned FSMap version 25 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- 172.21.15.133:0/787181140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55a753447170 con 0x55a75359f750 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaa77fe640 1 -- 172.21.15.133:0/787181140 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab80f4750 con 0x55a75359f750 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaa77fe640 1 -- 172.21.15.133:0/787181140 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbac0025020 con 0x55a75359f750 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 10 client.6609 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 10 client.6609 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 20 client.6609 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55a75355c640 0x55a7535b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 4 client.6609 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbad2117000 1 -- 172.21.15.133:0/787181140 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a753339c60 con 0x55a75355c640 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaccd46640 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55a75355c640 0x55a7535b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaa77fe640 10 client.6609 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.118+0000 7fbaccd46640 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55a75355c640 0x55a7535b10f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d89d9bb3c43f8cb7 server_cookie=ca469e751b830ccb in_seq=0 out_seq=0 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbaa77fe640 1 -- 172.21.15.133:0/787181140 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a753339c60 con 0x55a75355c640 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbaa77fe640 10 client.6609 handle_command_reply: tid=0 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 1 client.6609 shutdown 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 1 -- 172.21.15.133:0/787181140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 msgr2=0x55a75355b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a75355b8a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbac00079a0 tx=0x7fbac0007e60 comp rx=0 tx=0).stop 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 1 -- 172.21.15.133:0/787181140 shutdown_connections 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55a75355c640 0x55a7535b10f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a75355b8a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359ee30 0x55a75355b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 1 --2- 172.21.15.133:0/787181140 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a75359e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbaa77fe640 0 client.6609 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 1 -- 172.21.15.133:0/787181140 >> 172.21.15.133:0/787181140 conn(0x55a75354d020 msgr2=0x55a753556120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 1 -- 172.21.15.133:0/787181140 shutdown_connections 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 1 -- 172.21.15.133:0/787181140 wait complete. 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.122+0000 7fbad2117000 20 client.6609 trim_cache size 0 max 16384 2024-04-25T10:06:14.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.134+0000 7fbad2117000 1 Processor -- start 2024-04-25T10:06:14.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.134+0000 7fbad2117000 1 -- start start 2024-04-25T10:06:14.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a7535b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359ee30 0x55a7535b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac81feef0 con 0x55a75359f750 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80b0cc0 con 0x55a75359ee30 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac8161350 con 0x55a7535554b0 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535ac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40678/0 (socket says 172.21.15.133:40678) 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 -- 172.21.15.133:0/247187532 learned_addr learned my addr 172.21.15.133:0/247187532 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 -- 172.21.15.133:0/247187532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 msgr2=0x55a7535b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 --2- 172.21.15.133:0/247187532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a7535b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 -- 172.21.15.133:0/247187532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359ee30 msgr2=0x55a7535b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 --2- 172.21.15.133:0/247187532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359ee30 0x55a7535b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 -- 172.21.15.133:0/247187532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7533e4de0 con 0x55a75359f750 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 --2- 172.21.15.133:0/247187532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535ac2a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbac00039c0 tx=0x7fbac0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fe4c2d687fb100 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbaadffb640 1 -- 172.21.15.133:0/247187532 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0019070 con 0x55a75359f750 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbaadffb640 1 -- 172.21.15.133:0/247187532 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac00094d0 con 0x55a75359f750 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbaadffb640 1 -- 172.21.15.133:0/247187532 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0003cd0 con 0x55a75359f750 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- 172.21.15.133:0/247187532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 msgr2=0x55a7535ac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 --2- 172.21.15.133:0/247187532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535ac2a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbac00039c0 tx=0x7fbac0007f90 comp rx=0 tx=0).stop 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- 172.21.15.133:0/247187532 shutdown_connections 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 --2- 172.21.15.133:0/247187532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535ac2a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 --2- 172.21.15.133:0/247187532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a75359ee30 0x55a7535b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 --2- 172.21.15.133:0/247187532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a7535554b0 0x55a7535b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- 172.21.15.133:0/247187532 >> 172.21.15.133:0/247187532 conn(0x55a75354d020 msgr2=0x55a7535abb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- 172.21.15.133:0/247187532 shutdown_connections 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- 172.21.15.133:0/247187532 wait complete. 2024-04-25T10:06:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- start start 2024-04-25T10:06:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 Processor -- start 2024-04-25T10:06:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a7535554b0 0x55a7535af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a75359ee30 0x55a7535af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a753338e00 con 0x55a75359f750 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80b06c0 con 0x55a7535554b0 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a75339d300 con 0x55a75359ee30 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 10 client.0 _ll_register_callbacks cb 0x55a75351c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40690/0 (socket says 172.21.15.133:40690) 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 -- 172.21.15.133:0/4158786553 learned_addr learned my addr 172.21.15.133:0/4158786553 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 -- 172.21.15.133:0/4158786553 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a75359ee30 msgr2=0x55a7535af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 --2- 172.21.15.133:0/4158786553 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a75359ee30 0x55a7535af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 -- 172.21.15.133:0/4158786553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a7535554b0 msgr2=0x55a7535af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 --2- 172.21.15.133:0/4158786553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a7535554b0 0x55a7535af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 -- 172.21.15.133:0/4158786553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a75333f8e0 con 0x55a75359f750 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbacd547640 1 --2- 172.21.15.133:0/4158786553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535afb60 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fbac000c010 tx=0x7fbac0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711acdcf691268be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbaa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbaa77fe640 1 -- 172.21.15.133:0/4158786553 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0032060 con 0x55a75359f750 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbaa77fe640 1 -- 172.21.15.133:0/4158786553 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbac0011bb0 con 0x55a75359f750 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- 172.21.15.133:0/4158786553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbac00094d0 con 0x55a75359f750 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbaa77fe640 1 -- 172.21.15.133:0/4158786553 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbac0011db0 con 0x55a75359f750 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 10 client.6615 fetch_fsmap learned FSMap version 25 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 -- 172.21.15.133:0/4158786553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55a7533f78f0 con 0x55a75359f750 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbaa77fe640 1 -- 172.21.15.133:0/4158786553 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac00094d0 con 0x55a75359f750 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbaa77fe640 1 -- 172.21.15.133:0/4158786553 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbac0025050 con 0x55a75359f750 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 10 client.6615 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 10 client.6615 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 20 client.6615 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.138+0000 7fbad2117000 1 --2- 172.21.15.133:0/4158786553 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55a7535b04c0 0x55a753606fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 4 client.6615 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:14.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 -- 172.21.15.133:0/4158786553 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7533ab530 con 0x55a7535b04c0 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbaccd46640 1 --2- 172.21.15.133:0/4158786553 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55a7535b04c0 0x55a753606fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbaccd46640 1 --2- 172.21.15.133:0/4158786553 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55a7535b04c0 0x55a753606fd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=502b308d4655ca3c server_cookie=8a63865e6e898c50 in_seq=0 out_seq=0 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbaa77fe640 10 client.6615 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbaa77fe640 1 -- 172.21.15.133:0/4158786553 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+696 (crc 0 0 0) 0x55a7533ab530 con 0x55a7535b04c0 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbaa77fe640 10 client.6615 handle_command_reply: tid=0 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 client.6615 shutdown 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 -- 172.21.15.133:0/4158786553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 msgr2=0x55a7535afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 --2- 172.21.15.133:0/4158786553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535afb60 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fbac000c010 tx=0x7fbac0013000 comp rx=0 tx=0).stop 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 -- 172.21.15.133:0/4158786553 shutdown_connections 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 --2- 172.21.15.133:0/4158786553 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x55a7535b04c0 0x55a753606fd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 --2- 172.21.15.133:0/4158786553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a75359f750 0x55a7535afb60 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 --2- 172.21.15.133:0/4158786553 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a75359ee30 0x55a7535af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbaa77fe640 0 client.6615 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 --2- 172.21.15.133:0/4158786553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a7535554b0 0x55a7535af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 -- 172.21.15.133:0/4158786553 >> 172.21.15.133:0/4158786553 conn(0x55a75354d020 msgr2=0x55a7535ac420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 -- 172.21.15.133:0/4158786553 shutdown_connections 2024-04-25T10:06:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 1 -- 172.21.15.133:0/4158786553 wait complete. 2024-04-25T10:06:14.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbad2117000 20 client.6615 trim_cache size 0 max 16384 2024-04-25T10:06:14.151 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:14.151 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:06:14.151 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-04-25T10:06:14.151 INFO:teuthology.orchestra.run.smithi133.stdout: "8b8a07f7-ce80-435b-8bda-d06916484062": { 2024-04-25T10:06:14.151 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "8b8a07f7-ce80-435b-8bda-d06916484062", 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "" 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: "9492a7c9-5205-4f46-95ce-f911ed40c076": { 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "9492a7c9-5205-4f46-95ce-f911ed40c076", 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "" 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: "fa1023ee-a770-43bd-abd8-03b60e6c61d1": { 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "fa1023ee-a770-43bd-abd8-03b60e6c61d1", 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "" 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:14.152 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 -- 172.21.15.133:0/1364235369 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbab4066f90 msgr2=0x7fbab4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbab4066f90 0x7fbab4069450 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbac8144b90 tx=0x7fbabc009040 comp rx=0 tx=0).stop 2024-04-25T10:06:14.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 -- 172.21.15.133:0/1364235369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac806e4d0 msgr2=0x7fbac8138ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac806e4d0 0x7fbac8138ac0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbab80097c0 tx=0x7fbab801ace0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 -- 172.21.15.133:0/1364235369 shutdown_connections 2024-04-25T10:06:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbac815ced0 0x7fbac8161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbac8145170 0x7fbac8139300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbab4066f90 0x7fbab4069450 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 --2- 172.21.15.133:0/1364235369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac806e4d0 0x7fbac8138ac0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 -- 172.21.15.133:0/1364235369 >> 172.21.15.133:0/1364235369 conn(0x7fbac8066f70 msgr2=0x7fbac80b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 -- 172.21.15.133:0/1364235369 shutdown_connections 2024-04-25T10:06:14.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:14.142+0000 7fbacefaa640 1 -- 172.21.15.133:0/1364235369 wait complete. 2024-04-25T10:06:17.978 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:06:17.974+0000 7fded4882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:06:19.161 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:06:19.162 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:06:19.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.290+0000 7fd48310d640 1 Processor -- start 2024-04-25T10:06:19.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.290+0000 7fd48310d640 1 -- start start 2024-04-25T10:06:19.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.290+0000 7fd48310d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd47c0d1110 0x7fd47c161920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.290+0000 7fd48310d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47c15e3a0 con 0x7fd47c0b2b60 2024-04-25T10:06:19.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.290+0000 7fd48310d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47c15e680 con 0x7fd47c1548b0 2024-04-25T10:06:19.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.290+0000 7fd48310d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47c15e960 con 0x7fd47c0d1110 2024-04-25T10:06:19.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.290+0000 7fd4816aa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd47c0d1110 0x7fd47c161920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.294+0000 7fd4816aa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd47c0d1110 0x7fd47c161920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41414/0 (socket says 172.21.15.133:41414) 2024-04-25T10:06:19.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.294+0000 7fd4816aa640 1 -- 172.21.15.133:0/3037123957 learned_addr learned my addr 172.21.15.133:0/3037123957 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.294+0000 7fd4816aa640 1 -- 172.21.15.133:0/3037123957 >> v1:172.21.15.162:6789/0 conn(0x7fd47c1548b0 legacy=0x7fd47c0d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.294+0000 7fd4816aa640 1 -- 172.21.15.133:0/3037123957 >> v1:172.21.15.133:6789/0 conn(0x7fd47c0b2b60 legacy=0x7fd47c0d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.294+0000 7fd4816aa640 1 -- 172.21.15.133:0/3037123957 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c15ec40 con 0x7fd47c0d1110 2024-04-25T10:06:19.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.294+0000 7fd4816aa640 1 --2- 172.21.15.133:0/3037123957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c0d1110 0x7fd47c161920 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd474009e50 tx=0x7fd47401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc718d7dd450cd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.294+0000 7fd47b7fe640 1 -- 172.21.15.133:0/3037123957 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4740a57d0 con 0x7fd47c0d1110 2024-04-25T10:06:19.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.294+0000 7fd47b7fe640 1 -- 172.21.15.133:0/3037123957 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4740a5970 con 0x7fd47c0d1110 2024-04-25T10:06:19.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.294+0000 7fd47b7fe640 1 -- 172.21.15.133:0/3037123957 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4740a5d40 con 0x7fd47c0d1110 2024-04-25T10:06:19.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 -- 172.21.15.133:0/3037123957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c0d1110 msgr2=0x7fd47c161920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 --2- 172.21.15.133:0/3037123957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c0d1110 0x7fd47c161920 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd474009e50 tx=0x7fd47401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 -- 172.21.15.133:0/3037123957 shutdown_connections 2024-04-25T10:06:19.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 --2- 172.21.15.133:0/3037123957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c0d1110 0x7fd47c161920 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 -- 172.21.15.133:0/3037123957 >> 172.21.15.133:0/3037123957 conn(0x7fd47c0727c0 msgr2=0x7fd47c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 -- 172.21.15.133:0/3037123957 shutdown_connections 2024-04-25T10:06:19.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 -- 172.21.15.133:0/3037123957 wait complete. 2024-04-25T10:06:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 Processor -- start 2024-04-25T10:06:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 -- start start 2024-04-25T10:06:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c0b2b60 0x7fd47c14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd47c0d1110 0x7fd47c0c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd47c1548b0 0x7fd47c14d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c162480 con 0x7fd47c0b2b60 2024-04-25T10:06:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c15e180 con 0x7fd47c0d1110 2024-04-25T10:06:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0ceab0 con 0x7fd47c1548b0 2024-04-25T10:06:19.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd480ea9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c0b2b60 0x7fd47c14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd480ea9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c0b2b60 0x7fd47c14c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41420/0 (socket says 172.21.15.133:41420) 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd480ea9640 1 -- 172.21.15.133:0/1591627711 learned_addr learned my addr 172.21.15.133:0/1591627711 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd4816aa640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd47c1548b0 0x7fd47c14d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd47bfff640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd47c0d1110 0x7fd47c0c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd480ea9640 1 -- 172.21.15.133:0/1591627711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd47c1548b0 msgr2=0x7fd47c14d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd480ea9640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd47c1548b0 0x7fd47c14d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd480ea9640 1 -- 172.21.15.133:0/1591627711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd47c0d1110 msgr2=0x7fd47c0c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd480ea9640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd47c0d1110 0x7fd47c0c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd480ea9640 1 -- 172.21.15.133:0/1591627711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c0c7660 con 0x7fd47c0b2b60 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd47bfff640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd47c0d1110 0x7fd47c0c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd480ea9640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c0b2b60 0x7fd47c14c9d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd46400b410 tx=0x7fd46400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd4797fa640 1 -- 172.21.15.133:0/1591627711 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464015030 con 0x7fd47c0b2b60 2024-04-25T10:06:19.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd4797fa640 1 -- 172.21.15.133:0/1591627711 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd464004020 con 0x7fd47c0b2b60 2024-04-25T10:06:19.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd4797fa640 1 -- 172.21.15.133:0/1591627711 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464004450 con 0x7fd47c0b2b60 2024-04-25T10:06:19.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 -- 172.21.15.133:0/1591627711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47c0c7940 con 0x7fd47c0b2b60 2024-04-25T10:06:19.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.298+0000 7fd48310d640 1 -- 172.21.15.133:0/1591627711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47c0c7f00 con 0x7fd47c0b2b60 2024-04-25T10:06:19.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd4797fa640 1 -- 172.21.15.133:0/1591627711 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd464018070 con 0x7fd47c0b2b60 2024-04-25T10:06:19.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd4797fa640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd45c066f90 0x7fd45c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd4797fa640 1 -- 172.21.15.133:0/1591627711 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd464084cd0 con 0x7fd47c0b2b60 2024-04-25T10:06:19.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd47bfff640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd45c066f90 0x7fd45c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd47bfff640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd45c066f90 0x7fd45c069450 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0b8840 tx=0x7fd46c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd48627a000 1 Processor -- start 2024-04-25T10:06:19.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd48627a000 1 -- start start 2024-04-25T10:06:19.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd48627a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f949d0 0x563d47f94db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd48627a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47f4ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd48627a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f4b0a0 0x563d47f4b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd48627a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d47d97ff0 con 0x563d47f952f0 2024-04-25T10:06:19.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd48627a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d47d20150 con 0x563d47f4b0a0 2024-04-25T10:06:19.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd48627a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d47d94090 con 0x563d47f949d0 2024-04-25T10:06:19.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd47bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47f4ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd47bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47f4ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41422/0 (socket says 172.21.15.133:41422) 2024-04-25T10:06:19.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd47bfff640 1 -- 172.21.15.133:0/2505039740 learned_addr learned my addr 172.21.15.133:0/2505039740 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd4816aa640 1 --2- 172.21.15.133:0/2505039740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f4b0a0 0x563d47f4b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd480ea9640 1 --2- 172.21.15.133:0/2505039740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f949d0 0x563d47f94db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd47bfff640 1 -- 172.21.15.133:0/2505039740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f949d0 msgr2=0x563d47f94db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd47bfff640 1 --2- 172.21.15.133:0/2505039740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f949d0 0x563d47f94db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd47bfff640 1 -- 172.21.15.133:0/2505039740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f4b0a0 msgr2=0x563d47f4b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd47bfff640 1 --2- 172.21.15.133:0/2505039740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f4b0a0 0x563d47f4b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd47bfff640 1 -- 172.21.15.133:0/2505039740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d47f2e0d0 con 0x563d47f952f0 2024-04-25T10:06:19.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd480ea9640 1 --2- 172.21.15.133:0/2505039740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f949d0 0x563d47f94db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.302+0000 7fd4816aa640 1 --2- 172.21.15.133:0/2505039740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f4b0a0 0x563d47f4b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd47bfff640 1 --2- 172.21.15.133:0/2505039740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47f4ab60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00d870 tx=0x7fd46c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca4f7b9d276880c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd4727fc640 1 -- 172.21.15.133:0/2505039740 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c024070 con 0x563d47f952f0 2024-04-25T10:06:19.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd4727fc640 1 -- 172.21.15.133:0/2505039740 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c014ba0 con 0x563d47f952f0 2024-04-25T10:06:19.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd4727fc640 1 -- 172.21.15.133:0/2505039740 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c01f400 con 0x563d47f952f0 2024-04-25T10:06:19.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- 172.21.15.133:0/2505039740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 msgr2=0x563d47f4ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 --2- 172.21.15.133:0/2505039740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47f4ab60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00d870 tx=0x7fd46c010000 comp rx=0 tx=0).stop 2024-04-25T10:06:19.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- 172.21.15.133:0/2505039740 shutdown_connections 2024-04-25T10:06:19.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 --2- 172.21.15.133:0/2505039740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f4b0a0 0x563d47f4b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 --2- 172.21.15.133:0/2505039740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47f4ab60 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 --2- 172.21.15.133:0/2505039740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f949d0 0x563d47f94db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- 172.21.15.133:0/2505039740 >> 172.21.15.133:0/2505039740 conn(0x563d47f47020 msgr2=0x563d47f49440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- 172.21.15.133:0/2505039740 shutdown_connections 2024-04-25T10:06:19.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- 172.21.15.133:0/2505039740 wait complete. 2024-04-25T10:06:19.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- start start 2024-04-25T10:06:19.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 Processor -- start 2024-04-25T10:06:19.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47f797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47f79d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f952f0 0x563d47f7a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d47d5c250 con 0x563d47f949d0 2024-04-25T10:06:19.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d47d52690 con 0x563d47f952f0 2024-04-25T10:06:19.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d47d20a40 con 0x563d47f4b0a0 2024-04-25T10:06:19.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 10 client.0 _ll_register_callbacks cb 0x563d47f35e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:19.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd47bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47f79d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd47bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47f79d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41428/0 (socket says 172.21.15.133:41428) 2024-04-25T10:06:19.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd47bfff640 1 -- 172.21.15.133:0/339577293 learned_addr learned my addr 172.21.15.133:0/339577293 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd480ea9640 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47f797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd4816aa640 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f952f0 0x563d47f7a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd47bfff640 1 -- 172.21.15.133:0/339577293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 msgr2=0x563d47f797f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd47bfff640 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47f797f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd47bfff640 1 -- 172.21.15.133:0/339577293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f952f0 msgr2=0x563d47f7a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd47bfff640 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f952f0 0x563d47f7a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd47bfff640 1 -- 172.21.15.133:0/339577293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d47e451c0 con 0x563d47f949d0 2024-04-25T10:06:19.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd480ea9640 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47f797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd47bfff640 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47f79d30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd46c017040 tx=0x7fd46c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea94139d14dac523 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd45b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:19.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd45b7fe640 1 -- 172.21.15.133:0/339577293 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c024040 con 0x563d47f949d0 2024-04-25T10:06:19.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- 172.21.15.133:0/339577293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd46c014ba0 con 0x563d47f949d0 2024-04-25T10:06:19.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd45b7fe640 1 -- 172.21.15.133:0/339577293 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd46c01fa10 con 0x563d47f949d0 2024-04-25T10:06:19.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd45b7fe640 1 -- 172.21.15.133:0/339577293 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd46c038050 con 0x563d47f949d0 2024-04-25T10:06:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 10 client.6630 fetch_fsmap learned FSMap version 26 2024-04-25T10:06:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- 172.21.15.133:0/339577293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x563d47e41170 con 0x563d47f949d0 2024-04-25T10:06:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd45b7fe640 1 -- 172.21.15.133:0/339577293 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c014ba0 con 0x563d47f949d0 2024-04-25T10:06:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd45b7fe640 1 -- 172.21.15.133:0/339577293 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd46c03b020 con 0x563d47f949d0 2024-04-25T10:06:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 10 client.6630 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:06:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 10 client.6630 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 20 client.6630 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:19.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563d47fa4c80 0x563d47fa7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 4 client.6630 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:19.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd48627a000 1 -- 172.21.15.133:0/339577293 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d47d33c60 con 0x563d47fa4c80 2024-04-25T10:06:19.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.306+0000 7fd480ea9640 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563d47fa4c80 0x563d47fa7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd480ea9640 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563d47fa4c80 0x563d47fa7070 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6750100fd588539 server_cookie=a545fb0195af09a7 in_seq=0 out_seq=0 2024-04-25T10:06:19.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd45b7fe640 10 client.6630 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:19.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd45b7fe640 1 -- 172.21.15.133:0/339577293 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d47d33c60 con 0x563d47fa4c80 2024-04-25T10:06:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd45b7fe640 10 client.6630 handle_command_reply: tid=0 2024-04-25T10:06:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 1 client.6630 shutdown 2024-04-25T10:06:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 1 -- 172.21.15.133:0/339577293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 msgr2=0x563d47f79d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47f79d30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd46c017040 tx=0x7fd46c000bd0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 1 -- 172.21.15.133:0/339577293 shutdown_connections 2024-04-25T10:06:19.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563d47fa4c80 0x563d47fa7070 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f952f0 0x563d47f7a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd45b7fe640 0 client.6630 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:19.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47f79d30 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 1 --2- 172.21.15.133:0/339577293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47f797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 1 -- 172.21.15.133:0/339577293 >> 172.21.15.133:0/339577293 conn(0x563d47f47020 msgr2=0x563d47f49ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 1 -- 172.21.15.133:0/339577293 shutdown_connections 2024-04-25T10:06:19.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 1 -- 172.21.15.133:0/339577293 wait complete. 2024-04-25T10:06:19.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.310+0000 7fd48627a000 20 client.6630 trim_cache size 0 max 16384 2024-04-25T10:06:19.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 Processor -- start 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- start start 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47ff0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f949d0 0x563d47ff1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47ff1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c2039f0 con 0x563d47f952f0 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c152b20 con 0x563d47f949d0 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c151f20 con 0x563d47f4b0a0 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd4816aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47ff1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd4816aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47ff1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41444/0 (socket says 172.21.15.133:41444) 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd4816aa640 1 -- 172.21.15.133:0/2334173649 learned_addr learned my addr 172.21.15.133:0/2334173649 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd480ea9640 1 --2- 172.21.15.133:0/2334173649 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47ff0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd47bfff640 1 --2- 172.21.15.133:0/2334173649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f949d0 0x563d47ff1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd4816aa640 1 -- 172.21.15.133:0/2334173649 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 msgr2=0x563d47ff0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd4816aa640 1 --2- 172.21.15.133:0/2334173649 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47ff0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd4816aa640 1 -- 172.21.15.133:0/2334173649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f949d0 msgr2=0x563d47ff1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd4816aa640 1 --2- 172.21.15.133:0/2334173649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f949d0 0x563d47ff1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd4816aa640 1 -- 172.21.15.133:0/2334173649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d47ddede0 con 0x563d47f952f0 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd4816aa640 1 --2- 172.21.15.133:0/2334173649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47ff1960 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd47401a620 tx=0x7fd474004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd8c60123fe1b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd471ffb640 1 -- 172.21.15.133:0/2334173649 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4740a57d0 con 0x563d47f952f0 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd471ffb640 1 -- 172.21.15.133:0/2334173649 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4740a5970 con 0x563d47f952f0 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd471ffb640 1 -- 172.21.15.133:0/2334173649 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4740a5d20 con 0x563d47f952f0 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- 172.21.15.133:0/2334173649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 msgr2=0x563d47ff1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 --2- 172.21.15.133:0/2334173649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47ff1960 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd47401a620 tx=0x7fd474004ac0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- 172.21.15.133:0/2334173649 shutdown_connections 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 --2- 172.21.15.133:0/2334173649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f952f0 0x563d47ff1960 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 --2- 172.21.15.133:0/2334173649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f949d0 0x563d47ff1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 --2- 172.21.15.133:0/2334173649 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47ff0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- 172.21.15.133:0/2334173649 >> 172.21.15.133:0/2334173649 conn(0x563d47f47020 msgr2=0x563d47f930d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- 172.21.15.133:0/2334173649 shutdown_connections 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- 172.21.15.133:0/2334173649 wait complete. 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- start start 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 Processor -- start 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47fa7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47fa78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f952f0 0x563d47fa7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d47d32e00 con 0x563d47f949d0 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c152520 con 0x563d47f952f0 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d47d97300 con 0x563d47f4b0a0 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd48627a000 10 client.0 _ll_register_callbacks cb 0x563d47f16040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd47bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47fa78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd47bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47fa78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41450/0 (socket says 172.21.15.133:41450) 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.326+0000 7fd47bfff640 1 -- 172.21.15.133:0/3774358710 learned_addr learned my addr 172.21.15.133:0/3774358710 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd480ea9640 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47fa7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd4816aa640 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f952f0 0x563d47fa7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd47bfff640 1 -- 172.21.15.133:0/3774358710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 msgr2=0x563d47fa7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd47bfff640 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47fa7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd47bfff640 1 -- 172.21.15.133:0/3774358710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f952f0 msgr2=0x563d47fa7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd47bfff640 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f952f0 0x563d47fa7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd47bfff640 1 -- 172.21.15.133:0/3774358710 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d47d398e0 con 0x563d47f949d0 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd480ea9640 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47fa7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd4816aa640 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f952f0 0x563d47fa7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd47bfff640 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47fa78c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd46c01f5d0 tx=0x7fd46c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b93ac1ee9c5f8ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd45b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd45b7fe640 1 -- 172.21.15.133:0/3774358710 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c041060 con 0x563d47f949d0 2024-04-25T10:06:19.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd45b7fe640 1 -- 172.21.15.133:0/3774358710 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd46c033700 con 0x563d47f949d0 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 -- 172.21.15.133:0/3774358710 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4740a5970 con 0x563d47f949d0 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd45b7fe640 1 -- 172.21.15.133:0/3774358710 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd46c047050 con 0x563d47f949d0 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 10 client.6636 fetch_fsmap learned FSMap version 26 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 -- 172.21.15.133:0/3774358710 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x563d47df18f0 con 0x563d47f949d0 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd45b7fe640 1 -- 172.21.15.133:0/3774358710 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4740a5970 con 0x563d47f949d0 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd45b7fe640 1 -- 172.21.15.133:0/3774358710 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd46c04a020 con 0x563d47f949d0 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 10 client.6636 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 10 client.6636 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 20 client.6636 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563d47fa87f0 0x563d47ffcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 4 client.6636 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 -- 172.21.15.133:0/3774358710 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563d47da5530 con 0x563d47fa87f0 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd480ea9640 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563d47fa87f0 0x563d47ffcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd45b7fe640 10 client.6636 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:19.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd480ea9640 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563d47fa87f0 0x563d47ffcf60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9c8e77da28b823f server_cookie=ce15465bf52ace79 in_seq=0 out_seq=0 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd45b7fe640 1 -- 172.21.15.133:0/3774358710 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563d47da5530 con 0x563d47fa87f0 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd45b7fe640 10 client.6636 handle_command_reply: tid=0 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 client.6636 shutdown 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 -- 172.21.15.133:0/3774358710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 msgr2=0x563d47fa78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47fa78c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd46c01f5d0 tx=0x7fd46c01d000 comp rx=0 tx=0).stop 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 -- 172.21.15.133:0/3774358710 shutdown_connections 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x563d47fa87f0 0x563d47ffcf60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d47f952f0 0x563d47fa7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d47f949d0 0x563d47fa78c0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd45b7fe640 0 client.6636 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 --2- 172.21.15.133:0/3774358710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563d47f4b0a0 0x563d47fa7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 -- 172.21.15.133:0/3774358710 >> 172.21.15.133:0/3774358710 conn(0x563d47f47020 msgr2=0x563d47f93960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 -- 172.21.15.133:0/3774358710 shutdown_connections 2024-04-25T10:06:19.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 1 -- 172.21.15.133:0/3774358710 wait complete. 2024-04-25T10:06:19.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48627a000 20 client.6636 trim_cache size 0 max 16384 2024-04-25T10:06:19.347 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:19.347 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-04-25T10:06:19.347 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:06:19.347 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:19.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 -- 172.21.15.133:0/1591627711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd45c066f90 msgr2=0x7fd45c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd45c066f90 0x7fd45c069450 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0b8840 tx=0x7fd46c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:19.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 -- 172.21.15.133:0/1591627711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c0b2b60 msgr2=0x7fd47c14c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c0b2b60 0x7fd47c14c9d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd46400b410 tx=0x7fd46400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 -- 172.21.15.133:0/1591627711 shutdown_connections 2024-04-25T10:06:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd47c1548b0 0x7fd47c14d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd47c0d1110 0x7fd47c0c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd45c066f90 0x7fd45c069450 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 --2- 172.21.15.133:0/1591627711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c0b2b60 0x7fd47c14c9d0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 -- 172.21.15.133:0/1591627711 >> 172.21.15.133:0/1591627711 conn(0x7fd47c0727c0 msgr2=0x7fd47c0d2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 -- 172.21.15.133:0/1591627711 shutdown_connections 2024-04-25T10:06:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.330+0000 7fd48310d640 1 -- 172.21.15.133:0/1591627711 wait complete. 2024-04-25T10:06:19.351 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:06:19.351 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:19.351 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:06:19.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.462+0000 7f83fd944640 1 Processor -- start 2024-04-25T10:06:19.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.462+0000 7f83fd944640 1 -- start start 2024-04-25T10:06:19.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.462+0000 7f83fd944640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f83f8142920 0x7f83f8143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83fd944640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f8140a40 0x7f83f80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83fd944640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f80cf000 con 0x7f83f815d9f0 2024-04-25T10:06:19.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83fd944640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f8140ea0 con 0x7f83f8142920 2024-04-25T10:06:19.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83fd944640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80cf280 con 0x7f83f8140a40 2024-04-25T10:06:19.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83f6ffd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f83f8142920 0x7f83f8143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83f6ffd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f83f8142920 0x7f83f8143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:38968/0 (socket says 172.21.15.133:38968) 2024-04-25T10:06:19.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83f6ffd640 1 -- 172.21.15.133:0/2307866213 learned_addr learned my addr 172.21.15.133:0/2307866213 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/2307866213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f8140a40 0x7f83f80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83f6ffd640 1 -- 172.21.15.133:0/2307866213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f8140a40 msgr2=0x7f83f80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/2307866213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f8140a40 0x7f83f80ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83f6ffd640 1 -- 172.21.15.133:0/2307866213 >> v1:172.21.15.162:6789/0 conn(0x7f83f815d9f0 legacy=0x7f83f806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83f67fc640 1 -- 172.21.15.133:0/2307866213 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588013730 0 0) 0x7f83f00014a0 con 0x7f83f815d9f0 2024-04-25T10:06:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83f6ffd640 1 -- 172.21.15.133:0/2307866213 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f806d440 con 0x7f83f8142920 2024-04-25T10:06:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/2307866213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f8140a40 0x7f83f80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/2307866213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f8142920 0x7f83f8143560 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009870 tx=0x7f83e801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf7a17b0f36dd179 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.466+0000 7f83f67fc640 1 -- 172.21.15.133:0/2307866213 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80b1070 con 0x7f83f8142920 2024-04-25T10:06:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83f67fc640 1 -- 172.21.15.133:0/2307866213 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e80ac620 con 0x7f83f8142920 2024-04-25T10:06:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83f67fc640 1 -- 172.21.15.133:0/2307866213 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80ac9f0 con 0x7f83f8142920 2024-04-25T10:06:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 -- 172.21.15.133:0/2307866213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f8142920 msgr2=0x7f83f8143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 --2- 172.21.15.133:0/2307866213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f8142920 0x7f83f8143560 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009870 tx=0x7f83e801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 -- 172.21.15.133:0/2307866213 shutdown_connections 2024-04-25T10:06:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 --2- 172.21.15.133:0/2307866213 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f8140a40 0x7f83f80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 --2- 172.21.15.133:0/2307866213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f8142920 0x7f83f8143560 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 -- 172.21.15.133:0/2307866213 >> 172.21.15.133:0/2307866213 conn(0x7f83f8066f70 msgr2=0x7f83f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 -- 172.21.15.133:0/2307866213 shutdown_connections 2024-04-25T10:06:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 -- 172.21.15.133:0/2307866213 wait complete. 2024-04-25T10:06:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 Processor -- start 2024-04-25T10:06:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 -- start start 2024-04-25T10:06:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f8140a40 0x7f83f80b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f8142920 0x7f83f81516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f815d9f0 0x7f83f81417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f806ebc0 con 0x7f83f8142920 2024-04-25T10:06:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f806ea40 con 0x7f83f815d9f0 2024-04-25T10:06:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83fd944640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f8142d00 con 0x7f83f8140a40 2024-04-25T10:06:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83ee7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f8142920 0x7f83f81516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83ee7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f8142920 0x7f83f81516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41464/0 (socket says 172.21.15.133:41464) 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83ee7fc640 1 -- 172.21.15.133:0/4156781405 learned_addr learned my addr 172.21.15.133:0/4156781405 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83f77fe640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f815d9f0 0x7f83f81417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f8140a40 0x7f83f80b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83ee7fc640 1 -- 172.21.15.133:0/4156781405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f8140a40 msgr2=0x7f83f80b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f8140a40 0x7f83f80b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83ee7fc640 1 -- 172.21.15.133:0/4156781405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f815d9f0 msgr2=0x7f83f81417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f815d9f0 0x7f83f81417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.470+0000 7f83ee7fc640 1 -- 172.21.15.133:0/4156781405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f80b1460 con 0x7f83f8142920 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f8142920 0x7f83f81516b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f83e000ec10 tx=0x7f83e000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f83f4ff9640 1 -- 172.21.15.133:0/4156781405 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0017070 con 0x7f83f8142920 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f83f4ff9640 1 -- 172.21.15.133:0/4156781405 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e000cc30 con 0x7f83f8142920 2024-04-25T10:06:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f83fd944640 1 -- 172.21.15.133:0/4156781405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f8132560 con 0x7f83f8142920 2024-04-25T10:06:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f83f4ff9640 1 -- 172.21.15.133:0/4156781405 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0013470 con 0x7f83f8142920 2024-04-25T10:06:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f83fd944640 1 -- 172.21.15.133:0/4156781405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f81f8940 con 0x7f83f8142920 2024-04-25T10:06:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f83f4ff9640 1 -- 172.21.15.133:0/4156781405 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83e001a070 con 0x7f83f8142920 2024-04-25T10:06:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f83f4ff9640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d8066f90 0x7f83d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f83f4ff9640 1 -- 172.21.15.133:0/4156781405 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83e0085100 con 0x7f83f8142920 2024-04-25T10:06:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d8066f90 0x7f83d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d8066f90 0x7f83d8069450 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f83e80ae000 tx=0x7f83e80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f8400ad2000 1 Processor -- start 2024-04-25T10:06:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.474+0000 7f8400ad2000 1 -- start start 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 0x559f0ec5c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec5caa0 0x559f0ec120c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec12600 0x559f0ec129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f0ea5ff20 con 0x559f0ec5caa0 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f0e9e8000 con 0x559f0ec12600 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f0ea5bfc0 con 0x559f0ec5c180 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ee7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec5caa0 0x559f0ec120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ee7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec5caa0 0x559f0ec120c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41468/0 (socket says 172.21.15.133:41468) 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ee7fc640 1 -- 172.21.15.133:0/2677195515 learned_addr learned my addr 172.21.15.133:0/2677195515 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/2677195515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 0x559f0ec5c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f77fe640 1 --2- 172.21.15.133:0/2677195515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec12600 0x559f0ec129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ee7fc640 1 -- 172.21.15.133:0/2677195515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 msgr2=0x559f0ec5c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/2677195515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 0x559f0ec5c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ee7fc640 1 -- 172.21.15.133:0/2677195515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec12600 msgr2=0x559f0ec129e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/2677195515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec12600 0x559f0ec129e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ee7fc640 1 -- 172.21.15.133:0/2677195515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f0ea3bba0 con 0x559f0ec5caa0 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/2677195515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 0x559f0ec5c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/2677195515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec5caa0 0x559f0ec120c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f83e00850d0 tx=0x7f83e0055f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519bacedc07f211b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ed7fa640 1 -- 172.21.15.133:0/2677195515 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e006d020 con 0x559f0ec5caa0 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ed7fa640 1 -- 172.21.15.133:0/2677195515 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e005df90 con 0x559f0ec5caa0 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ed7fa640 1 -- 172.21.15.133:0/2677195515 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0021040 con 0x559f0ec5caa0 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- 172.21.15.133:0/2677195515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec5caa0 msgr2=0x559f0ec120c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 --2- 172.21.15.133:0/2677195515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec5caa0 0x559f0ec120c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f83e00850d0 tx=0x7f83e0055f80 comp rx=0 tx=0).stop 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- 172.21.15.133:0/2677195515 shutdown_connections 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 --2- 172.21.15.133:0/2677195515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec12600 0x559f0ec129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 --2- 172.21.15.133:0/2677195515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec5caa0 0x559f0ec120c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 --2- 172.21.15.133:0/2677195515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 0x559f0ec5c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- 172.21.15.133:0/2677195515 >> 172.21.15.133:0/2677195515 conn(0x559f0ec18760 msgr2=0x559f0ec109a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- 172.21.15.133:0/2677195515 shutdown_connections 2024-04-25T10:06:19.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- 172.21.15.133:0/2677195515 wait complete. 2024-04-25T10:06:19.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- start start 2024-04-25T10:06:19.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 Processor -- start 2024-04-25T10:06:19.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec5c180 0x559f0ec17c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5caa0 0x559f0ec4a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f0eb1d9c0 con 0x559f0ec12600 2024-04-25T10:06:19.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f0ea1a5d0 con 0x559f0ec5c180 2024-04-25T10:06:19.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f0ea1c080 con 0x559f0ec5caa0 2024-04-25T10:06:19.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f8400ad2000 10 client.0 _ll_register_callbacks cb 0x559f0ebfda60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:19.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec176e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41472/0 (socket says 172.21.15.133:41472) 2024-04-25T10:06:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f6ffd640 1 -- 172.21.15.133:0/3009560159 learned_addr learned my addr 172.21.15.133:0/3009560159 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec5c180 0x559f0ec17c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f77fe640 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5caa0 0x559f0ec4a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f6ffd640 1 -- 172.21.15.133:0/3009560159 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5caa0 msgr2=0x559f0ec4a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5caa0 0x559f0ec4a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f6ffd640 1 -- 172.21.15.133:0/3009560159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec5c180 msgr2=0x559f0ec17c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec5c180 0x559f0ec17c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f6ffd640 1 -- 172.21.15.133:0/3009560159 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f0eaa65a0 con 0x559f0ec12600 2024-04-25T10:06:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec176e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f83e8097e90 tx=0x7f83e801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca03523a180d55d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.478+0000 7f83d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f83d67fc640 1 -- 172.21.15.133:0/3009560159 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80b1040 con 0x559f0ec12600 2024-04-25T10:06:19.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f83d67fc640 1 -- 172.21.15.133:0/3009560159 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83e80b7700 con 0x559f0ec12600 2024-04-25T10:06:19.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 -- 172.21.15.133:0/3009560159 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83e005df90 con 0x559f0ec12600 2024-04-25T10:06:19.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f83d67fc640 1 -- 172.21.15.133:0/3009560159 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83e80d0050 con 0x559f0ec12600 2024-04-25T10:06:19.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 10 client.6645 fetch_fsmap learned FSMap version 26 2024-04-25T10:06:19.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 -- 172.21.15.133:0/3009560159 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x559f0ea372b0 con 0x559f0ec12600 2024-04-25T10:06:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f83d67fc640 1 -- 172.21.15.133:0/3009560159 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e005df90 con 0x559f0ec12600 2024-04-25T10:06:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f83d67fc640 1 -- 172.21.15.133:0/3009560159 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f83e80d3020 con 0x559f0ec12600 2024-04-25T10:06:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 10 client.6645 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:06:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 10 client.6645 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 20 client.6645 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:19.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x559f0ec4ae60 0x559f0ec6e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 4 client.6645 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 -- 172.21.15.133:0/3009560159 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f0e9fbc70 con 0x559f0ec4ae60 2024-04-25T10:06:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x559f0ec4ae60 0x559f0ec6e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f83d67fc640 10 client.6645 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x559f0ec4ae60 0x559f0ec6e440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9cde1bb1009c36d server_cookie=466081b53a466c1b in_seq=0 out_seq=0 2024-04-25T10:06:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f83d67fc640 1 -- 172.21.15.133:0/3009560159 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559f0e9fbc70 con 0x559f0ec4ae60 2024-04-25T10:06:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f83d67fc640 10 client.6645 handle_command_reply: tid=0 2024-04-25T10:06:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 client.6645 shutdown 2024-04-25T10:06:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 -- 172.21.15.133:0/3009560159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 msgr2=0x559f0ec176e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec176e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f83e8097e90 tx=0x7f83e801ad10 comp rx=0 tx=0).stop 2024-04-25T10:06:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 -- 172.21.15.133:0/3009560159 shutdown_connections 2024-04-25T10:06:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x559f0ec4ae60 0x559f0ec6e440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5caa0 0x559f0ec4a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec5c180 0x559f0ec17c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f83d67fc640 0 client.6645 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 --2- 172.21.15.133:0/3009560159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec176e0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 -- 172.21.15.133:0/3009560159 >> 172.21.15.133:0/3009560159 conn(0x559f0ec18760 msgr2=0x559f0ec110b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 -- 172.21.15.133:0/3009560159 shutdown_connections 2024-04-25T10:06:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 1 -- 172.21.15.133:0/3009560159 wait complete. 2024-04-25T10:06:19.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.482+0000 7f8400ad2000 20 client.6645 trim_cache size 0 max 16384 2024-04-25T10:06:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f8400ad2000 1 Processor -- start 2024-04-25T10:06:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f8400ad2000 1 -- start start 2024-04-25T10:06:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec18760 0x559f0ec70680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec11120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 0x559f0ec11660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f8400ad2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f82016e0 con 0x559f0ec12600 2024-04-25T10:06:19.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f8400ad2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f82008d0 con 0x559f0ec18760 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f8400ad2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f814fb90 con 0x559f0ec5c180 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ee7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec11120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ee7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec11120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41482/0 (socket says 172.21.15.133:41482) 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ee7fc640 1 -- 172.21.15.133:0/156548328 learned_addr learned my addr 172.21.15.133:0/156548328 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ee7fc640 1 -- 172.21.15.133:0/156548328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 msgr2=0x559f0ec11660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/156548328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec18760 0x559f0ec70680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/156548328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 0x559f0ec11660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ee7fc640 1 -- 172.21.15.133:0/156548328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec18760 msgr2=0x559f0ec70680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/156548328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec18760 0x559f0ec70680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ee7fc640 1 -- 172.21.15.133:0/156548328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f0e9e8250 con 0x559f0ec12600 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/156548328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec11120 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f83e0056130 tx=0x7f83e0085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4bb561b7b5ac15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ecff9640 1 -- 172.21.15.133:0/156548328 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0021070 con 0x559f0ec12600 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ecff9640 1 -- 172.21.15.133:0/156548328 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e0053b80 con 0x559f0ec12600 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.494+0000 7f83ecff9640 1 -- 172.21.15.133:0/156548328 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e006d040 con 0x559f0ec12600 2024-04-25T10:06:19.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/156548328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 msgr2=0x559f0ec11120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- 172.21.15.133:0/156548328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec11120 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f83e0056130 tx=0x7f83e0085040 comp rx=0 tx=0).stop 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/156548328 shutdown_connections 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- 172.21.15.133:0/156548328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 0x559f0ec11660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- 172.21.15.133:0/156548328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec11120 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- 172.21.15.133:0/156548328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec18760 0x559f0ec70680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/156548328 >> 172.21.15.133:0/156548328 conn(0x559f0ec4a5b0 msgr2=0x559f0ec5b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/156548328 shutdown_connections 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/156548328 wait complete. 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- start start 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 Processor -- start 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec6efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec18760 0x559f0ec6f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 0x559f0ec6fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f0ea237b0 con 0x559f0ec12600 2024-04-25T10:06:19.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f8150190 con 0x559f0ec18760 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f0ea2a4b0 con 0x559f0ec5c180 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 10 client.0 _ll_register_callbacks cb 0x559f0ebde600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83f6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec6efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83f6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec6efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41496/0 (socket says 172.21.15.133:41496) 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83f6ffd640 1 -- 172.21.15.133:0/2805307493 learned_addr learned my addr 172.21.15.133:0/2805307493 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec18760 0x559f0ec6f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83f6ffd640 1 -- 172.21.15.133:0/2805307493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 msgr2=0x559f0ec6fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 0x559f0ec6fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83f6ffd640 1 -- 172.21.15.133:0/2805307493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec18760 msgr2=0x559f0ec6f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec18760 0x559f0ec6f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83f6ffd640 1 -- 172.21.15.133:0/2805307493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f0e9f4b90 con 0x559f0ec12600 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec18760 0x559f0ec6f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83f6ffd640 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec6efa0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009210 tx=0x7f83e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc05e4cf5dce2721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83d67fc640 1 -- 172.21.15.133:0/2805307493 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80b1040 con 0x559f0ec12600 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83d67fc640 1 -- 172.21.15.133:0/2805307493 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83e8004020 con 0x559f0ec12600 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/2805307493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83e0053b80 con 0x559f0ec12600 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83d67fc640 1 -- 172.21.15.133:0/2805307493 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83e80df050 con 0x559f0ec12600 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 10 client.6651 fetch_fsmap learned FSMap version 26 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/2805307493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x559f0ea07010 con 0x559f0ec12600 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83d67fc640 1 -- 172.21.15.133:0/2805307493 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e0053b80 con 0x559f0ec12600 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83d67fc640 1 -- 172.21.15.133:0/2805307493 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f83e80e2020 con 0x559f0ec12600 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 10 client.6651 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 10 client.6651 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 20 client.6651 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x559f0ec5caa0 0x559f0ecc4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 4 client.6651 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/2805307493 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559f0ea6d460 con 0x559f0ec5caa0 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x559f0ec5caa0 0x559f0ecc4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83d67fc640 10 client.6651 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83ee7fc640 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x559f0ec5caa0 0x559f0ecc4330 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bbed9a28e8b6b5c server_cookie=df3466a855b2a3f in_seq=0 out_seq=0 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83d67fc640 1 -- 172.21.15.133:0/2805307493 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+167 (crc 0 0 0) 0x559f0ea6d460 con 0x559f0ec5caa0 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83d67fc640 10 client.6651 handle_command_reply: tid=0 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 client.6651 shutdown 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/2805307493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 msgr2=0x559f0ec6efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec6efa0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009210 tx=0x7f83e8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/2805307493 shutdown_connections 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x559f0ec5caa0 0x559f0ecc4330 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f0ec5c180 0x559f0ec6fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f0ec18760 0x559f0ec6f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f83d67fc640 0 client.6651 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:19.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 --2- 172.21.15.133:0/2805307493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f0ec12600 0x559f0ec6efa0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/2805307493 >> 172.21.15.133:0/2805307493 conn(0x559f0ec4a5b0 msgr2=0x559f0ec69770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/2805307493 shutdown_connections 2024-04-25T10:06:19.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 1 -- 172.21.15.133:0/2805307493 wait complete. 2024-04-25T10:06:19.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.498+0000 7f8400ad2000 20 client.6651 trim_cache size 0 max 16384 2024-04-25T10:06:19.516 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:06:19.516 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:19.516 INFO:teuthology.orchestra.run.smithi133.stdout: "damage_type": "backtrace", 2024-04-25T10:06:19.516 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 405415875, 2024-04-25T10:06:19.516 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 1099511630411, 2024-04-25T10:06:19.516 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T10:06:19.517 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:19.517 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:06:19.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 -- 172.21.15.133:0/4156781405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d8066f90 msgr2=0x7f83d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d8066f90 0x7f83d8069450 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f83e80ae000 tx=0x7f83e80a7000 comp rx=0 tx=0).stop 2024-04-25T10:06:19.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 -- 172.21.15.133:0/4156781405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f8142920 msgr2=0x7f83f81516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f8142920 0x7f83f81516b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f83e000ec10 tx=0x7f83e000c520 comp rx=0 tx=0).stop 2024-04-25T10:06:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 -- 172.21.15.133:0/4156781405 shutdown_connections 2024-04-25T10:06:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f815d9f0 0x7f83f81417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83d8066f90 0x7f83d8069450 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f8142920 0x7f83f81516b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 --2- 172.21.15.133:0/4156781405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f8140a40 0x7f83f80b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 -- 172.21.15.133:0/4156781405 >> 172.21.15.133:0/4156781405 conn(0x7f83f8066f70 msgr2=0x7f83f815b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 -- 172.21.15.133:0/4156781405 shutdown_connections 2024-04-25T10:06:19.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.502+0000 7f83fd944640 1 -- 172.21.15.133:0/4156781405 wait complete. 2024-04-25T10:06:19.520 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:19.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.626+0000 7f2be3760640 1 Processor -- start 2024-04-25T10:06:19.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.626+0000 7f2be3760640 1 -- start start 2024-04-25T10:06:19.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2be3760640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2bdc0d0770 0x7f2bdc0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2be3760640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bdc0ceb20 con 0x7f2bdc15db00 2024-04-25T10:06:19.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2be3760640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bdc0b30a0 con 0x7f2bdc0b87e0 2024-04-25T10:06:19.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2be3760640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bdc0d1110 con 0x7f2bdc0d0770 2024-04-25T10:06:19.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2be14fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2bdc15db00 0x7f2bdc0b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54280/0 (socket says 172.21.15.133:54280) 2024-04-25T10:06:19.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2be14fc640 1 -- 172.21.15.133:0/821479634 learned_addr learned my addr 172.21.15.133:0/821479634 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:19.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2be0cfb640 1 --2- 172.21.15.133:0/821479634 >> v2:172.21.15.133:3300/0 conn(0x7f2bdc0d0770 0x7f2bdc0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2bdbfff640 1 -- 172.21.15.133:0/821479634 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793598579 0 0) 0x7f2bdc0ceb20 con 0x7f2bdc15db00 2024-04-25T10:06:19.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2bdbfff640 1 -- 172.21.15.133:0/821479634 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bc4003530 con 0x7f2bdc15db00 2024-04-25T10:06:19.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2be0cfb640 1 -- 172.21.15.133:0/821479634 >> v1:172.21.15.162:6789/0 conn(0x7f2bdc0b87e0 legacy=0x7f2bdc0b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2be0cfb640 1 -- 172.21.15.133:0/821479634 >> v1:172.21.15.133:6789/0 conn(0x7f2bdc15db00 legacy=0x7f2bdc0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2be0cfb640 1 -- 172.21.15.133:0/821479634 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc0b7740 con 0x7f2bdc0d0770 2024-04-25T10:06:19.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2be0cfb640 1 --2- 172.21.15.133:0/821479634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc0d0770 0x7f2bdc0d0bd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00097c0 tx=0x7f2bd001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbe71f18d5f39e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2bdbfff640 1 -- 172.21.15.133:0/821479634 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00a7070 con 0x7f2bdc0d0770 2024-04-25T10:06:19.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2bdbfff640 1 -- 172.21.15.133:0/821479634 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd00b0070 con 0x7f2bdc0d0770 2024-04-25T10:06:19.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.630+0000 7f2bdbfff640 1 -- 172.21.15.133:0/821479634 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00ab890 con 0x7f2bdc0d0770 2024-04-25T10:06:19.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 -- 172.21.15.133:0/821479634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc0d0770 msgr2=0x7f2bdc0d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 --2- 172.21.15.133:0/821479634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc0d0770 0x7f2bdc0d0bd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00097c0 tx=0x7f2bd001c470 comp rx=0 tx=0).stop 2024-04-25T10:06:19.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 -- 172.21.15.133:0/821479634 shutdown_connections 2024-04-25T10:06:19.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 --2- 172.21.15.133:0/821479634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc0d0770 0x7f2bdc0d0bd0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 -- 172.21.15.133:0/821479634 >> 172.21.15.133:0/821479634 conn(0x7f2bdc0727c0 msgr2=0x7f2bdc072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 -- 172.21.15.133:0/821479634 shutdown_connections 2024-04-25T10:06:19.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 -- 172.21.15.133:0/821479634 wait complete. 2024-04-25T10:06:19.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 Processor -- start 2024-04-25T10:06:19.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 -- start start 2024-04-25T10:06:19.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc0b87e0 0x7f2bdc15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc0d0770 0x7f2bdc15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc15db00 0x7f2bdc151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc0b82c0 con 0x7f2bdc0d0770 2024-04-25T10:06:19.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc0b7fe0 con 0x7f2bdc15db00 2024-04-25T10:06:19.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc0b7430 con 0x7f2bdc0b87e0 2024-04-25T10:06:19.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be0cfb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc0d0770 0x7f2bdc15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be0cfb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc0d0770 0x7f2bdc15c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41514/0 (socket says 172.21.15.133:41514) 2024-04-25T10:06:19.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be0cfb640 1 -- 172.21.15.133:0/1508304012 learned_addr learned my addr 172.21.15.133:0/1508304012 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be14fc640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc0b87e0 0x7f2bdc15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be1cfd640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc15db00 0x7f2bdc151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be0cfb640 1 -- 172.21.15.133:0/1508304012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc0b87e0 msgr2=0x7f2bdc15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be0cfb640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc0b87e0 0x7f2bdc15a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be0cfb640 1 -- 172.21.15.133:0/1508304012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc15db00 msgr2=0x7f2bdc151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be0cfb640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc15db00 0x7f2bdc151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be0cfb640 1 -- 172.21.15.133:0/1508304012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc1510d0 con 0x7f2bdc0d0770 2024-04-25T10:06:19.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be1cfd640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc15db00 0x7f2bdc151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be0cfb640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc0d0770 0x7f2bdc15c1b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00097c0 tx=0x7f2bd001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2bd9ffb640 1 -- 172.21.15.133:0/1508304012 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00bf610 con 0x7f2bdc0d0770 2024-04-25T10:06:19.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2bd9ffb640 1 -- 172.21.15.133:0/1508304012 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd00bf7d0 con 0x7f2bdc0d0770 2024-04-25T10:06:19.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 -- 172.21.15.133:0/1508304012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc1596e0 con 0x7f2bdc0d0770 2024-04-25T10:06:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2bd9ffb640 1 -- 172.21.15.133:0/1508304012 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00bfbc0 con 0x7f2bdc0d0770 2024-04-25T10:06:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.634+0000 7f2be3760640 1 -- 172.21.15.133:0/1508304012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bdc159ca0 con 0x7f2bdc0d0770 2024-04-25T10:06:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.638+0000 7f2be3760640 1 -- 172.21.15.133:0/1508304012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bdc068ef0 con 0x7f2bdc0d0770 2024-04-25T10:06:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.638+0000 7f2bd9ffb640 1 -- 172.21.15.133:0/1508304012 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bd00bd020 con 0x7f2bdc0d0770 2024-04-25T10:06:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.638+0000 7f2bd9ffb640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bc8066f60 0x7f2bc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.638+0000 7f2be14fc640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bc8066f60 0x7f2bc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.638+0000 7f2bd9ffb640 1 -- 172.21.15.133:0/1508304012 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2bd011fe60 con 0x7f2bdc0d0770 2024-04-25T10:06:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.638+0000 7f2be14fc640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bc8066f60 0x7f2bc8069420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0011f0 tx=0x7f2bcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.642+0000 7f2bd9ffb640 1 -- 172.21.15.133:0/1508304012 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bd0016090 con 0x7f2bdc0d0770 2024-04-25T10:06:19.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.826+0000 7f2be3760640 1 -- 172.21.15.133:0/1508304012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bdc0b2750 con 0x7f2bdc0d0770 2024-04-25T10:06:19.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.826+0000 7f2bd9ffb640 1 -- 172.21.15.133:0/1508304012 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f2bd00f0e00 con 0x7f2bdc0d0770 2024-04-25T10:06:19.831 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:19.831 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:06:17.247723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:19.832 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 26 2024-04-25T10:06:19.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 -- 172.21.15.133:0/1508304012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bc8066f60 msgr2=0x7f2bc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bc8066f60 0x7f2bc8069420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0011f0 tx=0x7f2bcc009040 comp rx=0 tx=0).stop 2024-04-25T10:06:19.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 -- 172.21.15.133:0/1508304012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc0d0770 msgr2=0x7f2bdc15c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc0d0770 0x7f2bdc15c1b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00097c0 tx=0x7f2bd001acb0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 -- 172.21.15.133:0/1508304012 shutdown_connections 2024-04-25T10:06:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2bdc15db00 0x7f2bdc151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2bc8066f60 0x7f2bc8069420 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc0d0770 0x7f2bdc15c1b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 --2- 172.21.15.133:0/1508304012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2bdc0b87e0 0x7f2bdc15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 -- 172.21.15.133:0/1508304012 >> 172.21.15.133:0/1508304012 conn(0x7f2bdc0727c0 msgr2=0x7f2bdc072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 -- 172.21.15.133:0/1508304012 shutdown_connections 2024-04-25T10:06:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.830+0000 7f2be3760640 1 -- 172.21.15.133:0/1508304012 wait complete. 2024-04-25T10:06:19.849 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 405415875 2024-04-25T10:06:19.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.954+0000 7f99d786a640 1 Processor -- start 2024-04-25T10:06:19.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.954+0000 7f99d786a640 1 -- start start 2024-04-25T10:06:19.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d786a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f99d0142920 0x7f99d0143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d786a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99d015d9f0 0x7f99d015de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d786a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d0140ea0 con 0x7f99d0140a40 2024-04-25T10:06:19.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d786a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d00cf280 con 0x7f99d0142920 2024-04-25T10:06:19.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d786a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d006d440 con 0x7f99d015d9f0 2024-04-25T10:06:19.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d5606640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f99d0142920 0x7f99d0143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d5e07640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99d015d9f0 0x7f99d015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d4e05640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f99d0140a40 0x7f99d00ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:56298/0 (socket says 172.21.15.133:56298) 2024-04-25T10:06:19.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d4e05640 1 -- 172.21.15.133:0/833534087 learned_addr learned my addr 172.21.15.133:0/833534087 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:19.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d5e07640 1 -- 172.21.15.133:0/833534087 >> v2:172.21.15.162:3300/0 conn(0x7f99d0142920 msgr2=0x7f99d0143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d5e07640 1 --2- 172.21.15.133:0/833534087 >> v2:172.21.15.162:3300/0 conn(0x7f99d0142920 0x7f99d0143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d5e07640 1 -- 172.21.15.133:0/833534087 >> v1:172.21.15.162:6789/0 conn(0x7f99d0140a40 legacy=0x7f99d00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d5e07640 1 -- 172.21.15.133:0/833534087 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d006d720 con 0x7f99d015d9f0 2024-04-25T10:06:19.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.958+0000 7f99d5606640 1 --2- 172.21.15.133:0/833534087 >> v2:172.21.15.162:3300/0 conn(0x7f99d0142920 0x7f99d0143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:19.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d5e07640 1 --2- 172.21.15.133:0/833534087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99d015d9f0 0x7f99d015de80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009d90 tx=0x7f99c801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3688b79139753fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99c7fff640 1 -- 172.21.15.133:0/833534087 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c80b2070 con 0x7f99d015d9f0 2024-04-25T10:06:19.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99c7fff640 1 -- 172.21.15.133:0/833534087 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c80a5720 con 0x7f99d015d9f0 2024-04-25T10:06:19.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99c7fff640 1 -- 172.21.15.133:0/833534087 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c80a5af0 con 0x7f99d015d9f0 2024-04-25T10:06:19.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 -- 172.21.15.133:0/833534087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99d015d9f0 msgr2=0x7f99d015de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 --2- 172.21.15.133:0/833534087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99d015d9f0 0x7f99d015de80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009d90 tx=0x7f99c801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 -- 172.21.15.133:0/833534087 shutdown_connections 2024-04-25T10:06:19.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 --2- 172.21.15.133:0/833534087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99d015d9f0 0x7f99d015de80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 --2- 172.21.15.133:0/833534087 >> v2:172.21.15.162:3300/0 conn(0x7f99d0142920 0x7f99d0143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 -- 172.21.15.133:0/833534087 >> 172.21.15.133:0/833534087 conn(0x7f99d0066f70 msgr2=0x7f99d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 -- 172.21.15.133:0/833534087 shutdown_connections 2024-04-25T10:06:19.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 -- 172.21.15.133:0/833534087 wait complete. 2024-04-25T10:06:19.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 Processor -- start 2024-04-25T10:06:19.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 -- start start 2024-04-25T10:06:19.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99d0140a40 0x7f99d0139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d0142920 0x7f99d013e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99d015d9f0 0x7f99d013ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d006e330 con 0x7f99d0142920 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d006e1b0 con 0x7f99d0140a40 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d786a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d006e4b0 con 0x7f99d015d9f0 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d4e05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d0142920 0x7f99d013e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d4e05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d0142920 0x7f99d013e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41516/0 (socket says 172.21.15.133:41516) 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d4e05640 1 -- 172.21.15.133:0/3318525106 learned_addr learned my addr 172.21.15.133:0/3318525106 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d5e07640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99d015d9f0 0x7f99d013ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.962+0000 7f99d5606640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99d0140a40 0x7f99d0139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99d4e05640 1 -- 172.21.15.133:0/3318525106 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99d015d9f0 msgr2=0x7f99d013ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99d4e05640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99d015d9f0 0x7f99d013ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99d4e05640 1 -- 172.21.15.133:0/3318525106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99d0140a40 msgr2=0x7f99d0139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99d4e05640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99d0140a40 0x7f99d0139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99d4e05640 1 -- 172.21.15.133:0/3318525106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d01f7b70 con 0x7f99d0142920 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99d4e05640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d0142920 0x7f99d013e6f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f99c000c450 tx=0x7f99c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99c5ffb640 1 -- 172.21.15.133:0/3318525106 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0017070 con 0x7f99d0142920 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99c5ffb640 1 -- 172.21.15.133:0/3318525106 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c0005cf0 con 0x7f99d0142920 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99d786a640 1 -- 172.21.15.133:0/3318525106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d01f7e50 con 0x7f99d0142920 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99c5ffb640 1 -- 172.21.15.133:0/3318525106 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0007290 con 0x7f99d0142920 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99d786a640 1 -- 172.21.15.133:0/3318525106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d01f8410 con 0x7f99d0142920 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99c5ffb640 1 -- 172.21.15.133:0/3318525106 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99c001a070 con 0x7f99d0142920 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99c5ffb640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99b4066f60 0x7f99b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99c5ffb640 1 -- 172.21.15.133:0/3318525106 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99c00843f0 con 0x7f99d0142920 2024-04-25T10:06:19.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99d5606640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99b4066f60 0x7f99b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99d5606640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99b4066f60 0x7f99b4069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f99d014ed50 tx=0x7f99cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99da9d7000 1 Processor -- start 2024-04-25T10:06:19.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99da9d7000 1 -- start start 2024-04-25T10:06:19.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29520 0x560142d29900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29e40 0x560142ce0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142ce0570 0x560142ce0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99da9d7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560142bc7a00 con 0x560142d29e40 2024-04-25T10:06:19.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99da9d7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560142af57d0 con 0x560142d29520 2024-04-25T10:06:19.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.966+0000 7f99da9d7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x560142b2dc40 con 0x560142ce0570 2024-04-25T10:06:19.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29e40 0x560142ce0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29e40 0x560142ce0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41524/0 (socket says 172.21.15.133:41524) 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 -- 172.21.15.133:0/1394576732 learned_addr learned my addr 172.21.15.133:0/1394576732 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d5606640 1 --2- 172.21.15.133:0/1394576732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29520 0x560142d29900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 -- 172.21.15.133:0/1394576732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142ce0570 msgr2=0x560142ce0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 --2- 172.21.15.133:0/1394576732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142ce0570 0x560142ce0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 -- 172.21.15.133:0/1394576732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29520 msgr2=0x560142d29900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 --2- 172.21.15.133:0/1394576732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29520 0x560142d29900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 -- 172.21.15.133:0/1394576732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560142b087b0 con 0x560142d29e40 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 --2- 172.21.15.133:0/1394576732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29e40 0x560142ce0030 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f99c005cde0 tx=0x7f99c0054cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29814aff28c0d895 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99baffd640 1 -- 172.21.15.133:0/1394576732 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0052f00 con 0x560142d29e40 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99baffd640 1 -- 172.21.15.133:0/1394576732 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c00840d0 con 0x560142d29e40 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99baffd640 1 -- 172.21.15.133:0/1394576732 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c005ce80 con 0x560142d29e40 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- 172.21.15.133:0/1394576732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29e40 msgr2=0x560142ce0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 --2- 172.21.15.133:0/1394576732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29e40 0x560142ce0030 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f99c005cde0 tx=0x7f99c0054cc0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- 172.21.15.133:0/1394576732 shutdown_connections 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 --2- 172.21.15.133:0/1394576732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142ce0570 0x560142ce0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 --2- 172.21.15.133:0/1394576732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29e40 0x560142ce0030 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 --2- 172.21.15.133:0/1394576732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29520 0x560142d29900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- 172.21.15.133:0/1394576732 >> 172.21.15.133:0/1394576732 conn(0x560142cdc520 msgr2=0x560142cde910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- 172.21.15.133:0/1394576732 shutdown_connections 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- 172.21.15.133:0/1394576732 wait complete. 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- start start 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 Processor -- start 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142ce0570 0x560142ce5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29520 0x560142cead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29e40 0x560142ceb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560142b29ce0 con 0x560142d29520 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560142ab6160 con 0x560142d29e40 2024-04-25T10:06:19.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x560142ae8a80 con 0x560142ce0570 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 10 client.0 _ll_register_callbacks cb 0x560142be25d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29520 0x560142cead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29520 0x560142cead20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41538/0 (socket says 172.21.15.133:41538) 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 -- 172.21.15.133:0/756995042 learned_addr learned my addr 172.21.15.133:0/756995042 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d5606640 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142ce0570 0x560142ce5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d5e07640 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29e40 0x560142ceb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 -- 172.21.15.133:0/756995042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142ce0570 msgr2=0x560142ce5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142ce0570 0x560142ce5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 -- 172.21.15.133:0/756995042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29e40 msgr2=0x560142ceb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29e40 0x560142ceb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 -- 172.21.15.133:0/756995042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560142b747a0 con 0x560142d29520 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99d4e05640 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29520 0x560142cead20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f99c000ce10 tx=0x7f99c0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f71962236de430 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99affff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99affff640 1 -- 172.21.15.133:0/756995042 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c005d080 con 0x560142d29520 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- 172.21.15.133:0/756995042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99c00840d0 con 0x560142d29520 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99affff640 1 -- 172.21.15.133:0/756995042 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99c0068bc0 con 0x560142d29520 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99affff640 1 -- 172.21.15.133:0/756995042 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99c0068dc0 con 0x560142d29520 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 10 client.6669 fetch_fsmap learned FSMap version 26 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- 172.21.15.133:0/756995042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x560142ab63b0 con 0x560142d29520 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99affff640 1 -- 172.21.15.133:0/756995042 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c00840d0 con 0x560142d29520 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99affff640 1 -- 172.21.15.133:0/756995042 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99c0082bb0 con 0x560142d29520 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 10 client.6669 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 10 client.6669 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 20 client.6669 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x560142d2fd40 0x560142d30120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 4 client.6669 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:19.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.970+0000 7f99da9d7000 1 -- 172.21.15.133:0/756995042 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560142ac9a40 con 0x560142d2fd40 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99d5606640 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x560142d2fd40 0x560142d30120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99affff640 10 client.6669 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99d5606640 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x560142d2fd40 0x560142d30120 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46a4146be1e238a1 server_cookie=478e0e26604ed8fa in_seq=0 out_seq=0 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99affff640 1 -- 172.21.15.133:0/756995042 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560142ac9a40 con 0x560142d2fd40 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99affff640 10 client.6669 handle_command_reply: tid=0 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 1 client.6669 shutdown 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 1 -- 172.21.15.133:0/756995042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29520 msgr2=0x560142cead20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29520 0x560142cead20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f99c000ce10 tx=0x7f99c0000bd0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 1 -- 172.21.15.133:0/756995042 shutdown_connections 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x560142d2fd40 0x560142d30120 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29e40 0x560142ceb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142d29520 0x560142cead20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99affff640 0 client.6669 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 1 --2- 172.21.15.133:0/756995042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142ce0570 0x560142ce5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 1 -- 172.21.15.133:0/756995042 >> 172.21.15.133:0/756995042 conn(0x560142cdc520 msgr2=0x560142cdf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 1 -- 172.21.15.133:0/756995042 shutdown_connections 2024-04-25T10:06:19.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 1 -- 172.21.15.133:0/756995042 wait complete. 2024-04-25T10:06:19.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.974+0000 7f99da9d7000 20 client.6669 trim_cache size 0 max 16384 2024-04-25T10:06:19.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 Processor -- start 2024-04-25T10:06:19.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 -- start start 2024-04-25T10:06:19.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142cde910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142d29520 0x560142cdee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29e40 0x560142cdf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d01ffbb0 con 0x560142ce0570 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560142af77a0 con 0x560142d29e40 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x560142b29ce0 con 0x560142d29520 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99d5606640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142cde910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99d5606640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142cde910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41550/0 (socket says 172.21.15.133:41550) 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99d5606640 1 -- 172.21.15.133:0/1928110671 learned_addr learned my addr 172.21.15.133:0/1928110671 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99d5606640 1 -- 172.21.15.133:0/1928110671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142d29520 msgr2=0x560142cdee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99d5606640 1 --2- 172.21.15.133:0/1928110671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142d29520 0x560142cdee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99d5606640 1 -- 172.21.15.133:0/1928110671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29e40 msgr2=0x560142cdf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99d5606640 1 --2- 172.21.15.133:0/1928110671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29e40 0x560142cdf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99d5606640 1 -- 172.21.15.133:0/1928110671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560142b28c30 con 0x560142ce0570 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99d5606640 1 --2- 172.21.15.133:0/1928110671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142cde910 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x560142b2fe10 tx=0x7f99cc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9de16f7a00f16a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99ba7fc640 1 -- 172.21.15.133:0/1928110671 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc02d070 con 0x560142ce0570 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99ba7fc640 1 -- 172.21.15.133:0/1928110671 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc019040 con 0x560142ce0570 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99ba7fc640 1 -- 172.21.15.133:0/1928110671 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00b040 con 0x560142ce0570 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 -- 172.21.15.133:0/1928110671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 msgr2=0x560142cde910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 --2- 172.21.15.133:0/1928110671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142cde910 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x560142b2fe10 tx=0x7f99cc013000 comp rx=0 tx=0).stop 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 -- 172.21.15.133:0/1928110671 shutdown_connections 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 --2- 172.21.15.133:0/1928110671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29e40 0x560142cdf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 --2- 172.21.15.133:0/1928110671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142d29520 0x560142cdee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 --2- 172.21.15.133:0/1928110671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142cde910 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 -- 172.21.15.133:0/1928110671 >> 172.21.15.133:0/1928110671 conn(0x560142cdc520 msgr2=0x560142ce5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 -- 172.21.15.133:0/1928110671 shutdown_connections 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 -- 172.21.15.133:0/1928110671 wait complete. 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 -- start start 2024-04-25T10:06:19.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.986+0000 7f99da9d7000 1 Processor -- start 2024-04-25T10:06:19.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142d88b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29520 0x560142d2d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142d29e40 0x560142d2dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560142b2cf50 con 0x560142ce0570 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560142ae8a80 con 0x560142d29520 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x560142af47d0 con 0x560142d29e40 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 10 client.0 _ll_register_callbacks cb 0x7f99c0050ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d5606640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142d88b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d5606640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142d88b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41562/0 (socket says 172.21.15.133:41562) 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d5606640 1 -- 172.21.15.133:0/2187769468 learned_addr learned my addr 172.21.15.133:0/2187769468 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d5e07640 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142d29e40 0x560142d2dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d5606640 1 -- 172.21.15.133:0/2187769468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142d29e40 msgr2=0x560142d2dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d5606640 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142d29e40 0x560142d2dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d4e05640 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29520 0x560142d2d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d5606640 1 -- 172.21.15.133:0/2187769468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29520 msgr2=0x560142d2d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d5606640 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29520 0x560142d2d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d5606640 1 -- 172.21.15.133:0/2187769468 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560142b87960 con 0x560142ce0570 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d5606640 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142d88b30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f99cc008010 tx=0x7f99cc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edff9d1c994cc600 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99affff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99affff640 1 -- 172.21.15.133:0/2187769468 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc038060 con 0x560142ce0570 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99affff640 1 -- 172.21.15.133:0/2187769468 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99cc027050 con 0x560142ce0570 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 -- 172.21.15.133:0/2187769468 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99cc019040 con 0x560142ce0570 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99affff640 1 -- 172.21.15.133:0/2187769468 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99cc03e050 con 0x560142ce0570 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 10 client.6675 fetch_fsmap learned FSMap version 26 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 -- 172.21.15.133:0/2187769468 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x560142ad5080 con 0x560142ce0570 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99affff640 1 -- 172.21.15.133:0/2187769468 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc019040 con 0x560142ce0570 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99affff640 1 -- 172.21.15.133:0/2187769468 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99cc041020 con 0x560142ce0570 2024-04-25T10:06:19.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 10 client.6675 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 10 client.6675 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 20 client.6675 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x560142d2e2a0 0x560142d2e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 4 client.6675 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 405415875}] 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 -- 172.21.15.133:0/2187769468 --> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] -- command(tid 0: {"prefix": "damage rm", "damage_id": 405415875}) v1 -- 0x560142b3b180 con 0x560142d2e2a0 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d4e05640 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x560142d2e2a0 0x560142d2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d4e05640 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x560142d2e2a0 0x560142d2e680 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6cd5a24cc807aee server_cookie=ed9618377bcb2c62 in_seq=0 out_seq=0 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99affff640 10 client.6675 ms_handle_connect on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99affff640 1 -- 172.21.15.133:0/2187769468 <== mds.0 v2:172.21.15.133:6835/1425600691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x560142b3b180 con 0x560142d2e2a0 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99affff640 10 client.6675 handle_command_reply: tid=0 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 client.6675 shutdown 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 -- 172.21.15.133:0/2187769468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 msgr2=0x560142d88b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142d88b30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f99cc008010 tx=0x7f99cc004740 comp rx=0 tx=0).stop 2024-04-25T10:06:20.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 -- 172.21.15.133:0/2187769468 shutdown_connections 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.133:6835/1425600691,v1:172.21.15.133:6837/1425600691] conn(0x560142d2e2a0 0x560142d2e680 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560142d29e40 0x560142d2dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560142d29520 0x560142d2d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99affff640 0 client.6675 ms_handle_reset on v2:172.21.15.133:6835/1425600691 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 --2- 172.21.15.133:0/2187769468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560142ce0570 0x560142d88b30 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 -- 172.21.15.133:0/2187769468 >> 172.21.15.133:0/2187769468 conn(0x560142cdc520 msgr2=0x560142d27ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 -- 172.21.15.133:0/2187769468 shutdown_connections 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 1 -- 172.21.15.133:0/2187769468 wait complete. 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99da9d7000 20 client.6675 trim_cache size 0 max 16384 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 -- 172.21.15.133:0/3318525106 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99b4066f60 msgr2=0x7f99b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99b4066f60 0x7f99b4069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f99d014ed50 tx=0x7f99cc008040 comp rx=0 tx=0).stop 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 -- 172.21.15.133:0/3318525106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d0142920 msgr2=0x7f99d013e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d0142920 0x7f99d013e6f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f99c000c450 tx=0x7f99c000c910 comp rx=0 tx=0).stop 2024-04-25T10:06:20.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 -- 172.21.15.133:0/3318525106 shutdown_connections 2024-04-25T10:06:20.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99d015d9f0 0x7f99d013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99b4066f60 0x7f99b4069420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d0142920 0x7f99d013e6f0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 --2- 172.21.15.133:0/3318525106 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99d0140a40 0x7f99d0139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 -- 172.21.15.133:0/3318525106 >> 172.21.15.133:0/3318525106 conn(0x7f99d0066f70 msgr2=0x7f99d0140270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 -- 172.21.15.133:0/3318525106 shutdown_connections 2024-04-25T10:06:20.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:19.990+0000 7f99d786a640 1 -- 172.21.15.133:0/3318525106 wait complete. 2024-04-25T10:06:20.011 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:06:20.011 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:20.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.114+0000 7f9d9ff00640 1 Processor -- start 2024-04-25T10:06:20.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.114+0000 7f9d9ff00640 1 -- start start 2024-04-25T10:06:20.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9ff00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9d9815c9a0 0x7f9d980c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9ff00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d980c61d0 con 0x7f9d980b1f70 2024-04-25T10:06:20.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9ff00640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d980c64b0 con 0x7f9d980ce330 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9ff00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9806d4f0 con 0x7f9d9815c9a0 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9dc9c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9d9815c9a0 0x7f9d980c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9d49b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9d980b1f70 0x7f9d9806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54288/0 (socket says 172.21.15.133:54288) 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9d49b640 1 -- 172.21.15.133:0/1081632549 learned_addr learned my addr 172.21.15.133:0/1081632549 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9dc9c640 1 -- 172.21.15.133:0/1081632549 >> v1:172.21.15.162:6789/0 conn(0x7f9d980ce330 legacy=0x7f9d9815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9dc9c640 1 -- 172.21.15.133:0/1081632549 >> v1:172.21.15.133:6789/0 conn(0x7f9d980b1f70 legacy=0x7f9d9806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9dc9c640 1 -- 172.21.15.133:0/1081632549 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9806d7d0 con 0x7f9d9815c9a0 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9cc9a640 1 -- 172.21.15.133:0/1081632549 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952950138 0 0) 0x7f9d980c61d0 con 0x7f9d980b1f70 2024-04-25T10:06:20.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9dc9c640 1 --2- 172.21.15.133:0/1081632549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9815c9a0 0x7f9d980c5980 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9d900097c0 tx=0x7f9d9001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df38a94e8520719 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9cc9a640 1 -- 172.21.15.133:0/1081632549 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d900b0070 con 0x7f9d9815c9a0 2024-04-25T10:06:20.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9cc9a640 1 -- 172.21.15.133:0/1081632549 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d900ab600 con 0x7f9d9815c9a0 2024-04-25T10:06:20.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.118+0000 7f9d9cc9a640 1 -- 172.21.15.133:0/1081632549 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d900ab9d0 con 0x7f9d9815c9a0 2024-04-25T10:06:20.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- 172.21.15.133:0/1081632549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9815c9a0 msgr2=0x7f9d980c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 --2- 172.21.15.133:0/1081632549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9815c9a0 0x7f9d980c5980 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9d900097c0 tx=0x7f9d9001c460 comp rx=0 tx=0).stop 2024-04-25T10:06:20.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- 172.21.15.133:0/1081632549 shutdown_connections 2024-04-25T10:06:20.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 --2- 172.21.15.133:0/1081632549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9815c9a0 0x7f9d980c5980 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- 172.21.15.133:0/1081632549 >> 172.21.15.133:0/1081632549 conn(0x7f9d98067000 msgr2=0x7f9d9806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- 172.21.15.133:0/1081632549 shutdown_connections 2024-04-25T10:06:20.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- 172.21.15.133:0/1081632549 wait complete. 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 Processor -- start 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- start start 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d980b1f70 0x7f9d981478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d980ce330 0x7f9d98150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d9815c9a0 0x7f9d98150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9815cf00 con 0x7f9d980b1f70 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9815cd80 con 0x7f9d9815c9a0 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d98169de0 con 0x7f9d980ce330 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9dc9c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d980b1f70 0x7f9d981478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9dc9c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d980b1f70 0x7f9d981478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41586/0 (socket says 172.21.15.133:41586) 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9dc9c640 1 -- 172.21.15.133:0/2421774964 learned_addr learned my addr 172.21.15.133:0/2421774964 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9d49b640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d980ce330 0x7f9d98150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9e49d640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d9815c9a0 0x7f9d98150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9dc9c640 1 -- 172.21.15.133:0/2421774964 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d980ce330 msgr2=0x7f9d98150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9dc9c640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d980ce330 0x7f9d98150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9dc9c640 1 -- 172.21.15.133:0/2421774964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d9815c9a0 msgr2=0x7f9d98150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9dc9c640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d9815c9a0 0x7f9d98150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9dc9c640 1 -- 172.21.15.133:0/2421774964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9814a0f0 con 0x7f9d980b1f70 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9dc9c640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d980b1f70 0x7f9d981478e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9d900ae040 tx=0x7f9d9001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d967fc640 1 -- 172.21.15.133:0/2421774964 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d900b0050 con 0x7f9d980b1f70 2024-04-25T10:06:20.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d967fc640 1 -- 172.21.15.133:0/2421774964 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d900bc070 con 0x7f9d980b1f70 2024-04-25T10:06:20.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- 172.21.15.133:0/2421774964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9814a390 con 0x7f9d980b1f70 2024-04-25T10:06:20.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d967fc640 1 -- 172.21.15.133:0/2421774964 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d900b5a50 con 0x7f9d980b1f70 2024-04-25T10:06:20.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- 172.21.15.133:0/2421774964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9814a950 con 0x7f9d980b1f70 2024-04-25T10:06:20.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.122+0000 7f9d9ff00640 1 -- 172.21.15.133:0/2421774964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d98068ef0 con 0x7f9d980b1f70 2024-04-25T10:06:20.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.126+0000 7f9d967fc640 1 -- 172.21.15.133:0/2421774964 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d900ab4a0 con 0x7f9d980b1f70 2024-04-25T10:06:20.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.126+0000 7f9d967fc640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d78066f90 0x7f9d78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.126+0000 7f9d967fc640 1 -- 172.21.15.133:0/2421774964 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9d9011fb60 con 0x7f9d980b1f70 2024-04-25T10:06:20.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.126+0000 7f9d9d49b640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d78066f90 0x7f9d78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.126+0000 7f9d9d49b640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d78066f90 0x7f9d78069450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9d9815e9f0 tx=0x7f9d80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.130+0000 7f9d967fc640 1 -- 172.21.15.133:0/2421774964 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d90016050 con 0x7f9d980b1f70 2024-04-25T10:06:20.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.306+0000 7f9d9ff00640 1 -- 172.21.15.133:0/2421774964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d980b4030 con 0x7f9d980b1f70 2024-04-25T10:06:20.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.306+0000 7f9d967fc640 1 -- 172.21.15.133:0/2421774964 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f9d900f0b00 con 0x7f9d980b1f70 2024-04-25T10:06:20.311 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:20.311 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:06:17.247723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:20.311 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 26 2024-04-25T10:06:20.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 -- 172.21.15.133:0/2421774964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d78066f90 msgr2=0x7f9d78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d78066f90 0x7f9d78069450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9d9815e9f0 tx=0x7f9d80009040 comp rx=0 tx=0).stop 2024-04-25T10:06:20.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 -- 172.21.15.133:0/2421774964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d980b1f70 msgr2=0x7f9d981478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d980b1f70 0x7f9d981478e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9d900ae040 tx=0x7f9d9001aca0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 -- 172.21.15.133:0/2421774964 shutdown_connections 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d9815c9a0 0x7f9d98150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d980ce330 0x7f9d98150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d78066f90 0x7f9d78069450 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 --2- 172.21.15.133:0/2421774964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d980b1f70 0x7f9d981478e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 -- 172.21.15.133:0/2421774964 >> 172.21.15.133:0/2421774964 conn(0x7f9d98067000 msgr2=0x7f9d980b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 -- 172.21.15.133:0/2421774964 shutdown_connections 2024-04-25T10:06:20.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.310+0000 7f9d9ff00640 1 -- 172.21.15.133:0/2421774964 wait complete. 2024-04-25T10:06:20.329 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T10:06:20.439 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:20.440 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:06:20.440 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:20.452 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:20.453 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:20.453 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:20.506 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:20.551 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:20.551 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:06:20.551 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:20.556 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:20.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.718+0000 7fa6cf539640 1 Processor -- start 2024-04-25T10:06:20.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.718+0000 7fa6cf539640 1 -- start start 2024-04-25T10:06:20.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6cf539640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa6c80b1f70 0x7fa6c806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6cf539640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c80c61d0 con 0x7fa6c815c9a0 2024-04-25T10:06:20.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6cf539640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c80c64b0 con 0x7fa6c80ce330 2024-04-25T10:06:20.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6cf539640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c806d4f0 con 0x7fa6c80b1f70 2024-04-25T10:06:20.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6ccad4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa6c80b1f70 0x7fa6c806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6ccad4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa6c80b1f70 0x7fa6c806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41592/0 (socket says 172.21.15.133:41592) 2024-04-25T10:06:20.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6ccad4640 1 -- 172.21.15.133:0/1653879583 learned_addr learned my addr 172.21.15.133:0/1653879583 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:20.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6cd2d5640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa6c815c9a0 0x7fa6c80c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54296/0 (socket says 172.21.15.133:54296) 2024-04-25T10:06:20.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6ccad4640 1 -- 172.21.15.133:0/1653879583 >> v1:172.21.15.162:6789/0 conn(0x7fa6c80ce330 legacy=0x7fa6c815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6ccad4640 1 -- 172.21.15.133:0/1653879583 >> v1:172.21.15.133:6789/0 conn(0x7fa6c815c9a0 legacy=0x7fa6c80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6ccad4640 1 -- 172.21.15.133:0/1653879583 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c806d7d0 con 0x7fa6c80b1f70 2024-04-25T10:06:20.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6ccad4640 1 --2- 172.21.15.133:0/1653879583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6c80b1f70 0x7fa6c806dcf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009870 tx=0x7fa6b801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1765c7e1d8b4c6fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6bffff640 1 -- 172.21.15.133:0/1653879583 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80a9400 con 0x7fa6c80b1f70 2024-04-25T10:06:20.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6bffff640 1 -- 172.21.15.133:0/1653879583 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b80a95a0 con 0x7fa6c80b1f70 2024-04-25T10:06:20.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6bffff640 1 -- 172.21.15.133:0/1653879583 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80a9970 con 0x7fa6c80b1f70 2024-04-25T10:06:20.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6cf539640 1 -- 172.21.15.133:0/1653879583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6c80b1f70 msgr2=0x7fa6c806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6cf539640 1 --2- 172.21.15.133:0/1653879583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6c80b1f70 0x7fa6c806dcf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009870 tx=0x7fa6b801c520 comp rx=0 tx=0).stop 2024-04-25T10:06:20.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6cf539640 1 -- 172.21.15.133:0/1653879583 shutdown_connections 2024-04-25T10:06:20.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6cf539640 1 --2- 172.21.15.133:0/1653879583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6c80b1f70 0x7fa6c806dcf0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.722+0000 7fa6cf539640 1 -- 172.21.15.133:0/1653879583 >> 172.21.15.133:0/1653879583 conn(0x7fa6c8067000 msgr2=0x7fa6c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 -- 172.21.15.133:0/1653879583 shutdown_connections 2024-04-25T10:06:20.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 -- 172.21.15.133:0/1653879583 wait complete. 2024-04-25T10:06:20.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 Processor -- start 2024-04-25T10:06:20.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 -- start start 2024-04-25T10:06:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c80b1f70 0x7fa6c81564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c80ce330 0x7fa6c8156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6c815c9a0 0x7fa6c8131970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c815cf00 con 0x7fa6c815c9a0 2024-04-25T10:06:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c815cd80 con 0x7fa6c80ce330 2024-04-25T10:06:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8169de0 con 0x7fa6c80b1f70 2024-04-25T10:06:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cdad6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6c815c9a0 0x7fa6c8131970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cdad6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6c815c9a0 0x7fa6c8131970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41596/0 (socket says 172.21.15.133:41596) 2024-04-25T10:06:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cdad6640 1 -- 172.21.15.133:0/3448918212 learned_addr learned my addr 172.21.15.133:0/3448918212 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6ccad4640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c80ce330 0x7fa6c8156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cd2d5640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c80b1f70 0x7fa6c81564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cdad6640 1 -- 172.21.15.133:0/3448918212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c80b1f70 msgr2=0x7fa6c81564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cdad6640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c80b1f70 0x7fa6c81564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cdad6640 1 -- 172.21.15.133:0/3448918212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c80ce330 msgr2=0x7fa6c8156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cdad6640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c80ce330 0x7fa6c8156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cdad6640 1 -- 172.21.15.133:0/3448918212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c8157f30 con 0x7fa6c815c9a0 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6ccad4640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c80ce330 0x7fa6c8156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cdad6640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6c815c9a0 0x7fa6c8131970 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000c780 tx=0x7fa6c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6bdffb640 1 -- 172.21.15.133:0/3448918212 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0017070 con 0x7fa6c815c9a0 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6bdffb640 1 -- 172.21.15.133:0/3448918212 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c0003db0 con 0x7fa6c815c9a0 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 -- 172.21.15.133:0/3448918212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c814ddf0 con 0x7fa6c815c9a0 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6bdffb640 1 -- 172.21.15.133:0/3448918212 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c00052b0 con 0x7fa6c815c9a0 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 -- 172.21.15.133:0/3448918212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c814e690 con 0x7fa6c815c9a0 2024-04-25T10:06:20.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.726+0000 7fa6cf539640 1 -- 172.21.15.133:0/3448918212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c8068ef0 con 0x7fa6c815c9a0 2024-04-25T10:06:20.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.730+0000 7fa6bdffb640 1 -- 172.21.15.133:0/3448918212 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6c001a070 con 0x7fa6c815c9a0 2024-04-25T10:06:20.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.730+0000 7fa6bdffb640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6ac066f60 0x7fa6ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.730+0000 7fa6cd2d5640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6ac066f60 0x7fa6ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.730+0000 7fa6bdffb640 1 -- 172.21.15.133:0/3448918212 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa6c0084840 con 0x7fa6c815c9a0 2024-04-25T10:06:20.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.730+0000 7fa6cd2d5640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6ac066f60 0x7fa6ac069420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4000f70 tx=0x7fa6c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.734+0000 7fa6bdffb640 1 -- 172.21.15.133:0/3448918212 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6c00556c0 con 0x7fa6c815c9a0 2024-04-25T10:06:20.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.890+0000 7fa6cf539640 1 -- 172.21.15.133:0/3448918212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6c814dfc0 con 0x7fa6c815c9a0 2024-04-25T10:06:20.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.894+0000 7fa6bdffb640 1 -- 172.21.15.133:0/3448918212 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16966 (secure 0 0 0) 0x7fa6c0054fc0 con 0x7fa6c815c9a0 2024-04-25T10:06:20.898 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:20.898 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":54,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:05:38.186046+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:34.805104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:35.512057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.894+0000 7fa6cf539640 1 -- 172.21.15.133:0/3448918212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6ac066f60 msgr2=0x7fa6ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.894+0000 7fa6cf539640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6ac066f60 0x7fa6ac069420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4000f70 tx=0x7fa6c4009040 comp rx=0 tx=0).stop 2024-04-25T10:06:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.894+0000 7fa6cf539640 1 -- 172.21.15.133:0/3448918212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6c815c9a0 msgr2=0x7fa6c8131970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.894+0000 7fa6cf539640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6c815c9a0 0x7fa6c8131970 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000c780 tx=0x7fa6c000cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.894+0000 7fa6cf539640 1 -- 172.21.15.133:0/3448918212 shutdown_connections 2024-04-25T10:06:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.894+0000 7fa6cf539640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6ac066f60 0x7fa6ac069420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.894+0000 7fa6cf539640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6c815c9a0 0x7fa6c8131970 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.894+0000 7fa6cf539640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6c80ce330 0x7fa6c8156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.894+0000 7fa6cf539640 1 --2- 172.21.15.133:0/3448918212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6c80b1f70 0x7fa6c81564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.894+0000 7fa6cf539640 1 -- 172.21.15.133:0/3448918212 >> 172.21.15.133:0/3448918212 conn(0x7fa6c8067000 msgr2=0x7fa6c815e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.898+0000 7fa6cf539640 1 -- 172.21.15.133:0/3448918212 shutdown_connections 2024-04-25T10:06:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:20.898+0000 7fa6cf539640 1 -- 172.21.15.133:0/3448918212 wait complete. 2024-04-25T10:06:20.916 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:34.805104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:35.512057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:20.917 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:20.924 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:20.961 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:21.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.034+0000 7ff18b6e7640 1 Processor -- start 2024-04-25T10:06:21.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.034+0000 7ff18b6e7640 1 -- start start 2024-04-25T10:06:21.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff18b6e7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1840ce600 0x7ff1840cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff18b6e7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff1840d1920 0x7ff1840d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff18b6e7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18415ebd0 con 0x7ff18415d9a0 2024-04-25T10:06:21.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff18b6e7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1840cefa0 con 0x7ff1840d1920 2024-04-25T10:06:21.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff18b6e7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18415eeb0 con 0x7ff1840ce600 2024-04-25T10:06:21.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff189c84640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff1840d1920 0x7ff1840d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff189483640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff18415d9a0 0x7ff1840d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54310/0 (socket says 172.21.15.133:54310) 2024-04-25T10:06:21.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff189483640 1 -- 172.21.15.133:0/3685011555 learned_addr learned my addr 172.21.15.133:0/3685011555 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:21.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff188c82640 1 --2- 172.21.15.133:0/3685011555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1840ce600 0x7ff1840cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff189c84640 1 -- 172.21.15.133:0/3685011555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1840ce600 msgr2=0x7ff1840cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:21.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff189c84640 1 --2- 172.21.15.133:0/3685011555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1840ce600 0x7ff1840cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff189c84640 1 -- 172.21.15.133:0/3685011555 >> v1:172.21.15.133:6789/0 conn(0x7ff18415d9a0 legacy=0x7ff1840d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff189c84640 1 -- 172.21.15.133:0/3685011555 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840d0330 con 0x7ff1840d1920 2024-04-25T10:06:21.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff17bfff640 1 -- 172.21.15.133:0/3685011555 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779196459 0 0) 0x7ff18415ebd0 con 0x7ff18415d9a0 2024-04-25T10:06:21.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff189c84640 1 --2- 172.21.15.133:0/3685011555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1840d1920 0x7ff1840d1db0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009d90 tx=0x7ff17c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc0de298b7285f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff17bfff640 1 -- 172.21.15.133:0/3685011555 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0b2070 con 0x7ff1840d1920 2024-04-25T10:06:21.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff17bfff640 1 -- 172.21.15.133:0/3685011555 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c0a5990 con 0x7ff1840d1920 2024-04-25T10:06:21.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff17bfff640 1 -- 172.21.15.133:0/3685011555 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0a5d60 con 0x7ff1840d1920 2024-04-25T10:06:21.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff18b6e7640 1 -- 172.21.15.133:0/3685011555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1840d1920 msgr2=0x7ff1840d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff18b6e7640 1 --2- 172.21.15.133:0/3685011555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1840d1920 0x7ff1840d1db0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009d90 tx=0x7ff17c01c8b0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.038+0000 7ff18b6e7640 1 -- 172.21.15.133:0/3685011555 shutdown_connections 2024-04-25T10:06:21.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 --2- 172.21.15.133:0/3685011555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1840d1920 0x7ff1840d1db0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 --2- 172.21.15.133:0/3685011555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1840ce600 0x7ff1840cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 -- 172.21.15.133:0/3685011555 >> 172.21.15.133:0/3685011555 conn(0x7ff18406bf80 msgr2=0x7ff18406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:21.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 -- 172.21.15.133:0/3685011555 shutdown_connections 2024-04-25T10:06:21.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 -- 172.21.15.133:0/3685011555 wait complete. 2024-04-25T10:06:21.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 Processor -- start 2024-04-25T10:06:21.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 -- start start 2024-04-25T10:06:21.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1840ce600 0x7ff1841594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1840d1920 0x7ff184154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18415d9a0 0x7ff1840c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840d2470 con 0x7ff18415d9a0 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840d22f0 con 0x7ff1840d1920 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18415e590 con 0x7ff1840ce600 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff189c84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18415d9a0 0x7ff1840c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff189c84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18415d9a0 0x7ff1840c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41616/0 (socket says 172.21.15.133:41616) 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff189c84640 1 -- 172.21.15.133:0/40712324 learned_addr learned my addr 172.21.15.133:0/40712324 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff188c82640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1840d1920 0x7ff184154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff189483640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1840ce600 0x7ff1841594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff189c84640 1 -- 172.21.15.133:0/40712324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1840ce600 msgr2=0x7ff1841594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff189c84640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1840ce600 0x7ff1841594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff189c84640 1 -- 172.21.15.133:0/40712324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1840d1920 msgr2=0x7ff184154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff189c84640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1840d1920 0x7ff184154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff189c84640 1 -- 172.21.15.133:0/40712324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840b4a40 con 0x7ff18415d9a0 2024-04-25T10:06:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff188c82640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1840d1920 0x7ff184154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff189c84640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18415d9a0 0x7ff1840c7320 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff17c01af60 tx=0x7ff17c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff179ffb640 1 -- 172.21.15.133:0/40712324 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0b2050 con 0x7ff18415d9a0 2024-04-25T10:06:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff179ffb640 1 -- 172.21.15.133:0/40712324 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c0be070 con 0x7ff18415d9a0 2024-04-25T10:06:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 -- 172.21.15.133:0/40712324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1840b4d20 con 0x7ff18415d9a0 2024-04-25T10:06:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff179ffb640 1 -- 172.21.15.133:0/40712324 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0ada50 con 0x7ff18415d9a0 2024-04-25T10:06:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 -- 172.21.15.133:0/40712324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff184158fc0 con 0x7ff18415d9a0 2024-04-25T10:06:21.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.042+0000 7ff18b6e7640 1 -- 172.21.15.133:0/40712324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff184068e80 con 0x7ff18415d9a0 2024-04-25T10:06:21.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.046+0000 7ff179ffb640 1 -- 172.21.15.133:0/40712324 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff17c0adbf0 con 0x7ff18415d9a0 2024-04-25T10:06:21.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.046+0000 7ff179ffb640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f60 0x7ff168069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.046+0000 7ff179ffb640 1 -- 172.21.15.133:0/40712324 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff17c120390 con 0x7ff18415d9a0 2024-04-25T10:06:21.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.046+0000 7ff189483640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f60 0x7ff168069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.046+0000 7ff189483640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f60 0x7ff168069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff180000f70 tx=0x7ff180009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.050+0000 7ff179ffb640 1 -- 172.21.15.133:0/40712324 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff17c016560 con 0x7ff18415d9a0 2024-04-25T10:06:21.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 -- 172.21.15.133:0/40712324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff184073c50 con 0x7ff18415d9a0 2024-04-25T10:06:21.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff179ffb640 1 -- 172.21.15.133:0/40712324 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff17c0ab340 con 0x7ff18415d9a0 2024-04-25T10:06:21.213 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:06:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 -- 172.21.15.133:0/40712324 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f60 msgr2=0x7ff168069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f60 0x7ff168069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff180000f70 tx=0x7ff180009040 comp rx=0 tx=0).stop 2024-04-25T10:06:21.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 -- 172.21.15.133:0/40712324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18415d9a0 msgr2=0x7ff1840c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18415d9a0 0x7ff1840c7320 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff17c01af60 tx=0x7ff17c019650 comp rx=0 tx=0).stop 2024-04-25T10:06:21.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 -- 172.21.15.133:0/40712324 shutdown_connections 2024-04-25T10:06:21.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff168066f60 0x7ff168069420 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff18415d9a0 0x7ff1840c7320 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1840d1920 0x7ff184154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 --2- 172.21.15.133:0/40712324 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1840ce600 0x7ff1841594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 -- 172.21.15.133:0/40712324 >> 172.21.15.133:0/40712324 conn(0x7ff18406bf80 msgr2=0x7ff184161620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:21.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 -- 172.21.15.133:0/40712324 shutdown_connections 2024-04-25T10:06:21.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.210+0000 7ff18b6e7640 1 -- 172.21.15.133:0/40712324 wait complete. 2024-04-25T10:06:21.232 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:21.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb26dc24640 1 Processor -- start 2024-04-25T10:06:21.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb26dc24640 1 -- start start 2024-04-25T10:06:21.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb26dc24640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb268145170 0x7fb26806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb26dc24640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb26815ced0 0x7fb26815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb26dc24640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2680ce2e0 con 0x7fb26806e4d0 2024-04-25T10:06:21.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb26dc24640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26806db00 con 0x7fb268145170 2024-04-25T10:06:21.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb26dc24640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb268145550 con 0x7fb26815ced0 2024-04-25T10:06:21.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb2677fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb268145170 0x7fb26806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb2677fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb268145170 0x7fb26806df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41620/0 (socket says 172.21.15.133:41620) 2024-04-25T10:06:21.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb2677fe640 1 -- 172.21.15.133:0/633944947 learned_addr learned my addr 172.21.15.133:0/633944947 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:21.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb267fff640 1 --2- 172.21.15.133:0/633944947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb26815ced0 0x7fb26815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb2677fe640 1 -- 172.21.15.133:0/633944947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb26815ced0 msgr2=0x7fb26815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:21.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb2677fe640 1 --2- 172.21.15.133:0/633944947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb26815ced0 0x7fb26815d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb2677fe640 1 -- 172.21.15.133:0/633944947 >> v1:172.21.15.162:6789/0 conn(0x7fb26806e4d0 legacy=0x7fb2680cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:21.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.394+0000 7fb2677fe640 1 -- 172.21.15.133:0/633944947 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26806d4b0 con 0x7fb268145170 2024-04-25T10:06:21.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb2677fe640 1 --2- 172.21.15.133:0/633944947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb268145170 0x7fb26806df90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb258009870 tx=0x7fb25801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59894e1124a4e800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb266ffd640 1 -- 172.21.15.133:0/633944947 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2580b0070 con 0x7fb268145170 2024-04-25T10:06:21.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb266ffd640 1 -- 172.21.15.133:0/633944947 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2580ab600 con 0x7fb268145170 2024-04-25T10:06:21.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb266ffd640 1 -- 172.21.15.133:0/633944947 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2580ab9d0 con 0x7fb268145170 2024-04-25T10:06:21.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb26dc24640 1 -- 172.21.15.133:0/633944947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb268145170 msgr2=0x7fb26806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb26dc24640 1 --2- 172.21.15.133:0/633944947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb268145170 0x7fb26806df90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb258009870 tx=0x7fb25801c520 comp rx=0 tx=0).stop 2024-04-25T10:06:21.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb26dc24640 1 -- 172.21.15.133:0/633944947 shutdown_connections 2024-04-25T10:06:21.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb26dc24640 1 --2- 172.21.15.133:0/633944947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb26815ced0 0x7fb26815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb26dc24640 1 --2- 172.21.15.133:0/633944947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb268145170 0x7fb26806df90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb26dc24640 1 -- 172.21.15.133:0/633944947 >> 172.21.15.133:0/633944947 conn(0x7fb268066f70 msgr2=0x7fb26806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:21.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb26dc24640 1 -- 172.21.15.133:0/633944947 shutdown_connections 2024-04-25T10:06:21.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb26dc24640 1 -- 172.21.15.133:0/633944947 wait complete. 2024-04-25T10:06:21.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb26dc24640 1 Processor -- start 2024-04-25T10:06:21.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.398+0000 7fb26dc24640 1 -- start start 2024-04-25T10:06:21.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb26dc24640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26806e4d0 0x7fb268152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb26dc24640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb268145170 0x7fb2681f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb26dc24640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26815ced0 0x7fb2681f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb26dc24640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb268168610 con 0x7fb26815ced0 2024-04-25T10:06:21.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb26dc24640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb268168490 con 0x7fb26806e4d0 2024-04-25T10:06:21.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb26dc24640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb268168790 con 0x7fb268145170 2024-04-25T10:06:21.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb267fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26815ced0 0x7fb2681f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb267fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26815ced0 0x7fb2681f72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41630/0 (socket says 172.21.15.133:41630) 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb267fff640 1 -- 172.21.15.133:0/758864368 learned_addr learned my addr 172.21.15.133:0/758864368 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb2677fe640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26806e4d0 0x7fb268152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb25ffff640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb268145170 0x7fb2681f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb267fff640 1 -- 172.21.15.133:0/758864368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb268145170 msgr2=0x7fb2681f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb267fff640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb268145170 0x7fb2681f6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb267fff640 1 -- 172.21.15.133:0/758864368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26806e4d0 msgr2=0x7fb268152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb267fff640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26806e4d0 0x7fb268152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb267fff640 1 -- 172.21.15.133:0/758864368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26813b190 con 0x7fb26815ced0 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb2677fe640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26806e4d0 0x7fb268152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb267fff640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26815ced0 0x7fb2681f72e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb26000cc60 tx=0x7fb26000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb264ff9640 1 -- 172.21.15.133:0/758864368 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2600039a0 con 0x7fb26815ced0 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb264ff9640 1 -- 172.21.15.133:0/758864368 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb260003b40 con 0x7fb26815ced0 2024-04-25T10:06:21.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb26dc24640 1 -- 172.21.15.133:0/758864368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26813b470 con 0x7fb26815ced0 2024-04-25T10:06:21.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb264ff9640 1 -- 172.21.15.133:0/758864368 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb260004020 con 0x7fb26815ced0 2024-04-25T10:06:21.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb26dc24640 1 -- 172.21.15.133:0/758864368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26813ba30 con 0x7fb26815ced0 2024-04-25T10:06:21.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb26dc24640 1 -- 172.21.15.133:0/758864368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb268068e80 con 0x7fb26815ced0 2024-04-25T10:06:21.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb264ff9640 1 -- 172.21.15.133:0/758864368 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2600041c0 con 0x7fb26815ced0 2024-04-25T10:06:21.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb264ff9640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb248066f60 0x7fb248069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb264ff9640 1 -- 172.21.15.133:0/758864368 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb260085690 con 0x7fb26815ced0 2024-04-25T10:06:21.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb2677fe640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb248066f60 0x7fb248069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.402+0000 7fb2677fe640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb248066f60 0x7fb248069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb2580ad000 tx=0x7fb258008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.406+0000 7fb264ff9640 1 -- 172.21.15.133:0/758864368 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb260014030 con 0x7fb26815ced0 2024-04-25T10:06:21.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 -- 172.21.15.133:0/758864368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2681659b0 con 0x7fb26815ced0 2024-04-25T10:06:21.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb264ff9640 1 -- 172.21.15.133:0/758864368 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb260056510 con 0x7fb26815ced0 2024-04-25T10:06:21.565 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:06:21.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 -- 172.21.15.133:0/758864368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb248066f60 msgr2=0x7fb248069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb248066f60 0x7fb248069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb2580ad000 tx=0x7fb258008040 comp rx=0 tx=0).stop 2024-04-25T10:06:21.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 -- 172.21.15.133:0/758864368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26815ced0 msgr2=0x7fb2681f72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26815ced0 0x7fb2681f72e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb26000cc60 tx=0x7fb26000a5c0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 -- 172.21.15.133:0/758864368 shutdown_connections 2024-04-25T10:06:21.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb248066f60 0x7fb248069420 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26815ced0 0x7fb2681f72e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb268145170 0x7fb2681f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 --2- 172.21.15.133:0/758864368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26806e4d0 0x7fb268152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 -- 172.21.15.133:0/758864368 >> 172.21.15.133:0/758864368 conn(0x7fb268066f70 msgr2=0x7fb2680b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:21.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 -- 172.21.15.133:0/758864368 shutdown_connections 2024-04-25T10:06:21.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.562+0000 7fb26dc24640 1 -- 172.21.15.133:0/758864368 wait complete. 2024-04-25T10:06:21.583 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:21.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0e5f1640 1 Processor -- start 2024-04-25T10:06:21.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0e5f1640 1 -- start start 2024-04-25T10:06:21.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0e5f1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c0815d5e0 0x7f5c080cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0e5f1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c080ce2e0 con 0x7f5c080cd820 2024-04-25T10:06:21.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0e5f1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0806d4b0 con 0x7f5c08145170 2024-04-25T10:06:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0e5f1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0806db00 con 0x7f5c0815d5e0 2024-04-25T10:06:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5bfffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5c080cd820 0x7f5c0815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54324/0 (socket says 172.21.15.133:54324) 2024-04-25T10:06:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5bfffff640 1 -- 172.21.15.133:0/2292861792 learned_addr learned my addr 172.21.15.133:0/2292861792 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0cb8e640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c0815d5e0 0x7f5c080cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531880341 0 0) 0x7f5c080ce2e0 con 0x7f5c080cd820 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bf4003530 con 0x7f5c080cd820 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 279811776 0 0) 0x7f5bf4003530 con 0x7f5c080cd820 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c0815d5e0 msgr2=0x7f5c080cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c0815d5e0 0x7f5c080cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 >> v1:172.21.15.162:6789/0 conn(0x7f5c08145170 legacy=0x7f5c0806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0806d790 con 0x7f5c080cd820 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5bf0001680 con 0x7f5c080cd820 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 >> v1:172.21.15.133:6789/0 conn(0x7f5c080cd820 legacy=0x7f5c0815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bf40039f0 0x7f5bf4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5bf4004810 0x7f5bf4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 0x7f5bf400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0806db00 con 0x7f5bf4005190 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf400bf10 con 0x7f5bf40039f0 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf400c090 con 0x7f5bf4004810 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0cb8e640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 0x7f5bf400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5bfffff640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5bf4004810 0x7f5bf4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c07fff640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bf40039f0 0x7f5bf4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0cb8e640 1 -- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5bf4004810 msgr2=0x7f5bf4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0cb8e640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5bf4004810 0x7f5bf4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0cb8e640 1 -- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bf40039f0 msgr2=0x7f5bf4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0cb8e640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bf40039f0 0x7f5bf4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c0cb8e640 1 -- 172.21.15.133:0/2292861792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf4003530 con 0x7f5bf4005190 2024-04-25T10:06:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.702+0000 7f5c07fff640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5bf40039f0 0x7f5bf4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0cb8e640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 0x7f5bf400b9d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5c0806f720 tx=0x7f5c0001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee89e1fb82ae644 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c000b2070 con 0x7f5bf4005190 2024-04-25T10:06:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c000a5c30 con 0x7f5bf4005190 2024-04-25T10:06:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c077fe640 1 -- 172.21.15.133:0/2292861792 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00009c10 con 0x7f5bf4005190 2024-04-25T10:06:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/2292861792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 msgr2=0x7f5bf400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 --2- 172.21.15.133:0/2292861792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 0x7f5bf400b9d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5c0806f720 tx=0x7f5c0001cbc0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0cb8e640 1 -- 172.21.15.133:0/2292861792 reap_dead start 2024-04-25T10:06:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/2292861792 shutdown_connections 2024-04-25T10:06:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/2292861792 >> 172.21.15.133:0/2292861792 conn(0x7f5c08066f70 msgr2=0x7f5c0806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/2292861792 shutdown_connections 2024-04-25T10:06:21.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/2292861792 wait complete. 2024-04-25T10:06:21.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 Processor -- start 2024-04-25T10:06:21.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 -- start start 2024-04-25T10:06:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 0x7f5c080b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c08135f80 0x7f5c0813ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0813ce80 0x7f5c0813d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0815cb70 con 0x7f5bf4005190 2024-04-25T10:06:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c08145550 con 0x7f5c0813ce80 2024-04-25T10:06:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0e5f1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080cdca0 con 0x7f5c08135f80 2024-04-25T10:06:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 0x7f5c080b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 0x7f5c080b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41658/0 (socket says 172.21.15.133:41658) 2024-04-25T10:06:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c07fff640 1 -- 172.21.15.133:0/3921359464 learned_addr learned my addr 172.21.15.133:0/3921359464 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5c0cb8e640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0813ce80 0x7f5c0813d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.706+0000 7f5bfffff640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c08135f80 0x7f5c0813ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c07fff640 1 -- 172.21.15.133:0/3921359464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c08135f80 msgr2=0x7f5c0813ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c07fff640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c08135f80 0x7f5c0813ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c07fff640 1 -- 172.21.15.133:0/3921359464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0813ce80 msgr2=0x7f5c0813d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c07fff640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0813ce80 0x7f5c0813d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c07fff640 1 -- 172.21.15.133:0/3921359464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0813d840 con 0x7f5bf4005190 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c0cb8e640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0813ce80 0x7f5c0813d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c07fff640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 0x7f5c080b47f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800d770 tx=0x7f5bf800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c057fa640 1 -- 172.21.15.133:0/3921359464 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8005270 con 0x7f5bf4005190 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c057fa640 1 -- 172.21.15.133:0/3921359464 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8017070 con 0x7f5bf4005190 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/3921359464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0813f5b0 con 0x7f5bf4005190 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c057fa640 1 -- 172.21.15.133:0/3921359464 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf800b650 con 0x7f5bf4005190 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/3921359464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0813fb70 con 0x7f5bf4005190 2024-04-25T10:06:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/3921359464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c08068e80 con 0x7f5bf4005190 2024-04-25T10:06:21.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c057fa640 1 -- 172.21.15.133:0/3921359464 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bf800b7f0 con 0x7f5bf4005190 2024-04-25T10:06:21.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c057fa640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5be8066f90 0x7f5be8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5c057fa640 1 -- 172.21.15.133:0/3921359464 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5bf8085080 con 0x7f5bf4005190 2024-04-25T10:06:21.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5bfffff640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5be8066f90 0x7f5be8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.710+0000 7f5bfffff640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5be8066f90 0x7f5be8069450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0000f70 tx=0x7f5bf0000ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.714+0000 7f5c057fa640 1 -- 172.21.15.133:0/3921359464 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bf8055f00 con 0x7f5bf4005190 2024-04-25T10:06:21.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.870+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/3921359464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c08153a30 con 0x7f5bf4005190 2024-04-25T10:06:21.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c057fa640 1 -- 172.21.15.133:0/3921359464 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5bf8055800 con 0x7f5bf4005190 2024-04-25T10:06:21.877 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:06:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/3921359464 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5be8066f90 msgr2=0x7f5be8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5be8066f90 0x7f5be8069450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0000f70 tx=0x7f5bf0000ee0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/3921359464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 msgr2=0x7f5c080b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 0x7f5c080b47f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800d770 tx=0x7f5bf800dc30 comp rx=0 tx=0).stop 2024-04-25T10:06:21.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/3921359464 shutdown_connections 2024-04-25T10:06:21.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c0813ce80 0x7f5c0813d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c08135f80 0x7f5c0813ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5be8066f90 0x7f5be8069450 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 --2- 172.21.15.133:0/3921359464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf4005190 0x7f5c080b47f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/3921359464 >> 172.21.15.133:0/3921359464 conn(0x7f5c08066f70 msgr2=0x7f5c08154570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:21.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/3921359464 shutdown_connections 2024-04-25T10:06:21.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:21.874+0000 7f5c0e5f1640 1 -- 172.21.15.133:0/3921359464 wait complete. 2024-04-25T10:06:21.896 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:22.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2ed00640 1 Processor -- start 2024-04-25T10:06:22.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2ed00640 1 -- start start 2024-04-25T10:06:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2ed00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f280cd820 0x7f6f280cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2ed00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6f2815ced0 0x7f6f2815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2ed00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f280ce2e0 con 0x7f6f28145170 2024-04-25T10:06:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2ed00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f2806db00 con 0x7f6f2815ced0 2024-04-25T10:06:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2ed00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28145550 con 0x7f6f280cd820 2024-04-25T10:06:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2d29d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6f2815ced0 0x7f6f2815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2d29d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6f2815ced0 0x7f6f2815d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41660/0 (socket says 172.21.15.133:41660) 2024-04-25T10:06:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2d29d640 1 -- 172.21.15.133:0/1966546310 learned_addr learned my addr 172.21.15.133:0/1966546310 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f27fff640 1 --2- 172.21.15.133:0/1966546310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f280cd820 0x7f6f280cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2d29d640 1 -- 172.21.15.133:0/1966546310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f280cd820 msgr2=0x7f6f280cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2d29d640 1 --2- 172.21.15.133:0/1966546310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f280cd820 0x7f6f280cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2d29d640 1 -- 172.21.15.133:0/1966546310 >> v1:172.21.15.133:6789/0 conn(0x7f6f28145170 legacy=0x7f6f2806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.006+0000 7f6f2d29d640 1 -- 172.21.15.133:0/1966546310 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2806d4b0 con 0x7f6f2815ced0 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2d29d640 1 --2- 172.21.15.133:0/1966546310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f2815ced0 0x7f6f2815d360 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009e50 tx=0x7f6f2001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d372a12ecbd52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f277fe640 1 -- 172.21.15.133:0/1966546310 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f200a5700 con 0x7f6f2815ced0 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f277fe640 1 -- 172.21.15.133:0/1966546310 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f200a58a0 con 0x7f6f2815ced0 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f277fe640 1 -- 172.21.15.133:0/1966546310 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f200a5c70 con 0x7f6f2815ced0 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 -- 172.21.15.133:0/1966546310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f2815ced0 msgr2=0x7f6f2815d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 --2- 172.21.15.133:0/1966546310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f2815ced0 0x7f6f2815d360 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009e50 tx=0x7f6f2001ca30 comp rx=0 tx=0).stop 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 -- 172.21.15.133:0/1966546310 shutdown_connections 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 --2- 172.21.15.133:0/1966546310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f2815ced0 0x7f6f2815d360 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 --2- 172.21.15.133:0/1966546310 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f280cd820 0x7f6f280cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 -- 172.21.15.133:0/1966546310 >> 172.21.15.133:0/1966546310 conn(0x7f6f28066f70 msgr2=0x7f6f2806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 -- 172.21.15.133:0/1966546310 shutdown_connections 2024-04-25T10:06:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 -- 172.21.15.133:0/1966546310 wait complete. 2024-04-25T10:06:22.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 Processor -- start 2024-04-25T10:06:22.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 -- start start 2024-04-25T10:06:22.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f280cd820 0x7f6f28137640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f28145170 0x7f6f2815d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f2815ced0 0x7f6f2815def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f280b1e50 con 0x7f6f28145170 2024-04-25T10:06:22.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f280b1cd0 con 0x7f6f280cd820 2024-04-25T10:06:22.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f280b1fd0 con 0x7f6f2815ced0 2024-04-25T10:06:22.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f28145170 0x7f6f2815d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f28145170 0x7f6f2815d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41676/0 (socket says 172.21.15.133:41676) 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f27fff640 1 -- 172.21.15.133:0/4010093187 learned_addr learned my addr 172.21.15.133:0/4010093187 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f27fff640 1 -- 172.21.15.133:0/4010093187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f2815ced0 msgr2=0x7f6f2815def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f27fff640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f2815ced0 0x7f6f2815def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f27fff640 1 -- 172.21.15.133:0/4010093187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f280cd820 msgr2=0x7f6f28137640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f27fff640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f280cd820 0x7f6f28137640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f27fff640 1 -- 172.21.15.133:0/4010093187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f28137b80 con 0x7f6f28145170 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f27fff640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f28145170 0x7f6f2815d9b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00d5f0 tx=0x7f6f1c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f257fa640 1 -- 172.21.15.133:0/4010093187 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0125f0 con 0x7f6f28145170 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f257fa640 1 -- 172.21.15.133:0/4010093187 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c019070 con 0x7f6f28145170 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 -- 172.21.15.133:0/4010093187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f28142fb0 con 0x7f6f28145170 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f257fa640 1 -- 172.21.15.133:0/4010093187 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0129d0 con 0x7f6f28145170 2024-04-25T10:06:22.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.010+0000 7f6f2ed00640 1 -- 172.21.15.133:0/4010093187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f28143570 con 0x7f6f28145170 2024-04-25T10:06:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.014+0000 7f6f2ed00640 1 -- 172.21.15.133:0/4010093187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f28068e80 con 0x7f6f28145170 2024-04-25T10:06:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.014+0000 7f6f257fa640 1 -- 172.21.15.133:0/4010093187 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f1c012b70 con 0x7f6f28145170 2024-04-25T10:06:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.014+0000 7f6f257fa640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f0c066f60 0x7f6f0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.014+0000 7f6f257fa640 1 -- 172.21.15.133:0/4010093187 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6f1c085600 con 0x7f6f28145170 2024-04-25T10:06:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.014+0000 7f6f2ca9c640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f0c066f60 0x7f6f0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.014+0000 7f6f2ca9c640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f0c066f60 0x7f6f0c069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6f14000f70 tx=0x7f6f140079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.018+0000 7f6f257fa640 1 -- 172.21.15.133:0/4010093187 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f1c056480 con 0x7f6f28145170 2024-04-25T10:06:22.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 -- 172.21.15.133:0/4010093187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6f2815fb30 con 0x7f6f28145170 2024-04-25T10:06:22.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f257fa640 1 -- 172.21.15.133:0/4010093187 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6f1c055d80 con 0x7f6f28145170 2024-04-25T10:06:22.190 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:06:22.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 -- 172.21.15.133:0/4010093187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f0c066f60 msgr2=0x7f6f0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f0c066f60 0x7f6f0c069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6f14000f70 tx=0x7f6f140079e0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 -- 172.21.15.133:0/4010093187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f28145170 msgr2=0x7f6f2815d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f28145170 0x7f6f2815d9b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00d5f0 tx=0x7f6f1c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 -- 172.21.15.133:0/4010093187 shutdown_connections 2024-04-25T10:06:22.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6f2815ced0 0x7f6f2815def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f0c066f60 0x7f6f0c069420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f28145170 0x7f6f2815d9b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 --2- 172.21.15.133:0/4010093187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f280cd820 0x7f6f28137640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 -- 172.21.15.133:0/4010093187 >> 172.21.15.133:0/4010093187 conn(0x7f6f28066f70 msgr2=0x7f6f2813d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:22.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 -- 172.21.15.133:0/4010093187 shutdown_connections 2024-04-25T10:06:22.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.186+0000 7f6f2ed00640 1 -- 172.21.15.133:0/4010093187 wait complete. 2024-04-25T10:06:22.207 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:06:22.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:22.207 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:22.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[53351]: fuse finished with error 0 and tester_r 0 2024-04-25T10:06:22.223 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:06:22.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:22.223 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:06:22.393 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:06:22.393 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:22.402 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:22.402 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:06:22.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:22.402 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:06:22.451 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:06:22.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:22.452 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:06:22.452 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:22.457 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:22.457 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:06:22.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:22.458 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:06:22.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:22.505 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:06:22.505 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:06:22.505 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:22.511 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:22.511 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:06:22.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:22.512 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:06:22.558 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:06:22.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:22.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:22.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.678+0000 7f3304fc7640 1 Processor -- start 2024-04-25T10:06:22.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.678+0000 7f3304fc7640 1 -- start start 2024-04-25T10:06:22.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f3304fc7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f330015d9b0 0x7f33000cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f3304fc7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f330015dd90 0x7f33000ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f3304fc7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33000ca9b0 con 0x7f33000b19f0 2024-04-25T10:06:22.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f3304fc7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33000cbe30 con 0x7f330015d9b0 2024-04-25T10:06:22.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f3304fc7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33000cac90 con 0x7f330015dd90 2024-04-25T10:06:22.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f32fe59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f330015d9b0 0x7f33000cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f32fe59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f330015d9b0 0x7f33000cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41690/0 (socket says 172.21.15.133:41690) 2024-04-25T10:06:22.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f32fe59c640 1 -- 172.21.15.133:0/1098727454 learned_addr learned my addr 172.21.15.133:0/1098727454 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:22.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f32fe59c640 1 -- 172.21.15.133:0/1098727454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f330015dd90 msgr2=0x7f33000ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:22.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f32fe59c640 1 --2- 172.21.15.133:0/1098727454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f330015dd90 0x7f33000ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f32fe59c640 1 -- 172.21.15.133:0/1098727454 >> v1:172.21.15.162:6789/0 conn(0x7f33000b19f0 legacy=0x7f33001616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:22.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.682+0000 7f32fe59c640 1 -- 172.21.15.133:0/1098727454 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33000caf70 con 0x7f330015d9b0 2024-04-25T10:06:22.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f32fe59c640 1 --2- 172.21.15.133:0/1098727454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f330015d9b0 0x7f33000cb5e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009e50 tx=0x7f32f801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0f584ac29ef799 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f32fd59a640 1 -- 172.21.15.133:0/1098727454 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f80a57a0 con 0x7f330015d9b0 2024-04-25T10:06:22.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f32fd59a640 1 -- 172.21.15.133:0/1098727454 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f80a5940 con 0x7f330015d9b0 2024-04-25T10:06:22.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f32fd59a640 1 -- 172.21.15.133:0/1098727454 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f80a5d10 con 0x7f330015d9b0 2024-04-25T10:06:22.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 -- 172.21.15.133:0/1098727454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f330015d9b0 msgr2=0x7f33000cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 --2- 172.21.15.133:0/1098727454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f330015d9b0 0x7f33000cb5e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009e50 tx=0x7f32f801c900 comp rx=0 tx=0).stop 2024-04-25T10:06:22.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 -- 172.21.15.133:0/1098727454 shutdown_connections 2024-04-25T10:06:22.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 --2- 172.21.15.133:0/1098727454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f330015dd90 0x7f33000ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 --2- 172.21.15.133:0/1098727454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f330015d9b0 0x7f33000cb5e0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 -- 172.21.15.133:0/1098727454 >> 172.21.15.133:0/1098727454 conn(0x7f3300067000 msgr2=0x7f330006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:22.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 -- 172.21.15.133:0/1098727454 shutdown_connections 2024-04-25T10:06:22.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 -- 172.21.15.133:0/1098727454 wait complete. 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 Processor -- start 2024-04-25T10:06:22.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 -- start start 2024-04-25T10:06:22.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33000b19f0 0x7f33001539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f330015d9b0 0x7f33000cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f330015dd90 0x7f3300151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330016abf0 con 0x7f33000b19f0 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330016aa70 con 0x7f330015dd90 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f3304fc7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330016ad70 con 0x7f330015d9b0 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f32fe59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33000b19f0 0x7f33001539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f32fe59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33000b19f0 0x7f33001539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41694/0 (socket says 172.21.15.133:41694) 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.686+0000 7f32fe59c640 1 -- 172.21.15.133:0/2327336811 learned_addr learned my addr 172.21.15.133:0/2327336811 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fe59c640 1 -- 172.21.15.133:0/2327336811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f330015d9b0 msgr2=0x7f33000cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fed9d640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f330015dd90 0x7f3300151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fdd9b640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f330015d9b0 0x7f33000cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fe59c640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f330015d9b0 0x7f33000cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fe59c640 1 -- 172.21.15.133:0/2327336811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f330015dd90 msgr2=0x7f3300151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fe59c640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f330015dd90 0x7f3300151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fe59c640 1 -- 172.21.15.133:0/2327336811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3300152940 con 0x7f33000b19f0 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fed9d640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f330015dd90 0x7f3300151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fdd9b640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f330015d9b0 0x7f33000cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fe59c640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33000b19f0 0x7f33001539e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f32f801b010 tx=0x7f32f8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32eeffd640 1 -- 172.21.15.133:0/2327336811 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f80bc020 con 0x7f33000b19f0 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32eeffd640 1 -- 172.21.15.133:0/2327336811 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f80b8680 con 0x7f33000b19f0 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f3304fc7640 1 -- 172.21.15.133:0/2327336811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3300152c20 con 0x7f33000b19f0 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32eeffd640 1 -- 172.21.15.133:0/2327336811 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f80b8a30 con 0x7f33000b19f0 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f3304fc7640 1 -- 172.21.15.133:0/2327336811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33001534c0 con 0x7f33000b19f0 2024-04-25T10:06:22.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f3304fc7640 1 -- 172.21.15.133:0/2327336811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3300068ef0 con 0x7f33000b19f0 2024-04-25T10:06:22.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32eeffd640 1 -- 172.21.15.133:0/2327336811 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32f80bf020 con 0x7f33000b19f0 2024-04-25T10:06:22.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32eeffd640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32e0066f60 0x7f32e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fdd9b640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32e0066f60 0x7f32e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32eeffd640 1 -- 172.21.15.133:0/2327336811 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32f8120360 con 0x7f33000b19f0 2024-04-25T10:06:22.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.690+0000 7f32fdd9b640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32e0066f60 0x7f32e0069420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3300151ea0 tx=0x7f32f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.694+0000 7f32eeffd640 1 -- 172.21.15.133:0/2327336811 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32f80c6030 con 0x7f33000b19f0 2024-04-25T10:06:22.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.886+0000 7f3304fc7640 1 -- 172.21.15.133:0/2327336811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3300152fb0 con 0x7f33000b19f0 2024-04-25T10:06:22.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f32eeffd640 1 -- 172.21.15.133:0/2327336811 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f32f80165e0 con 0x7f33000b19f0 2024-04-25T10:06:22.893 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:22.893 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:06:17.247723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:22.893 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 26 2024-04-25T10:06:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 -- 172.21.15.133:0/2327336811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32e0066f60 msgr2=0x7f32e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32e0066f60 0x7f32e0069420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3300151ea0 tx=0x7f32f4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 -- 172.21.15.133:0/2327336811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33000b19f0 msgr2=0x7f33001539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33000b19f0 0x7f33001539e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f32f801b010 tx=0x7f32f8019700 comp rx=0 tx=0).stop 2024-04-25T10:06:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 -- 172.21.15.133:0/2327336811 shutdown_connections 2024-04-25T10:06:22.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f330015dd90 0x7f3300151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f330015d9b0 0x7f33000cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32e0066f60 0x7f32e0069420 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 --2- 172.21.15.133:0/2327336811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33000b19f0 0x7f33001539e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 -- 172.21.15.133:0/2327336811 >> 172.21.15.133:0/2327336811 conn(0x7f3300067000 msgr2=0x7f33000b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 -- 172.21.15.133:0/2327336811 shutdown_connections 2024-04-25T10:06:22.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:22.890+0000 7f3304fc7640 1 -- 172.21.15.133:0/2327336811 wait complete. 2024-04-25T10:06:22.912 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:23.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.022+0000 7f55a14f6640 1 Processor -- start 2024-04-25T10:06:23.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f55a14f6640 1 -- start start 2024-04-25T10:06:23.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f55a14f6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f559c15c9a0 0x7f559c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f55a14f6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f559c0b1f70 0x7f559c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f55a14f6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f559c0ce330 0x7f559c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f55a14f6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559c0cecc0 con 0x7f559c0b1f70 2024-04-25T10:06:23.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f55a14f6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559c0c61d0 con 0x7f559c15c9a0 2024-04-25T10:06:23.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f55a14f6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c0c64b0 con 0x7f559c0ce330 2024-04-25T10:06:23.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559a7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f559c0b1f70 0x7f559c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559a7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f559c0b1f70 0x7f559c06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41708/0 (socket says 172.21.15.133:41708) 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559a7fc640 1 -- 172.21.15.133:0/4121829937 learned_addr learned my addr 172.21.15.133:0/4121829937 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559b7fe640 1 --2- 172.21.15.133:0/4121829937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f559c0ce330 0x7f559c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559affd640 1 --2- 172.21.15.133:0/4121829937 >> v2:172.21.15.162:3300/0 conn(0x7f559c15c9a0 0x7f559c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559a7fc640 1 -- 172.21.15.133:0/4121829937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f559c0ce330 msgr2=0x7f559c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559a7fc640 1 --2- 172.21.15.133:0/4121829937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f559c0ce330 0x7f559c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559a7fc640 1 -- 172.21.15.133:0/4121829937 >> v2:172.21.15.162:3300/0 conn(0x7f559c15c9a0 msgr2=0x7f559c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559a7fc640 1 --2- 172.21.15.133:0/4121829937 >> v2:172.21.15.162:3300/0 conn(0x7f559c15c9a0 0x7f559c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559a7fc640 1 -- 172.21.15.133:0/4121829937 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c06d4f0 con 0x7f559c0b1f70 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559b7fe640 1 --2- 172.21.15.133:0/4121829937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f559c0ce330 0x7f559c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559affd640 1 --2- 172.21.15.133:0/4121829937 >> v2:172.21.15.162:3300/0 conn(0x7f559c15c9a0 0x7f559c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.026+0000 7f559a7fc640 1 --2- 172.21.15.133:0/4121829937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c0b1f70 0x7f559c06dcf0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f559c06f5f0 tx=0x7f559001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b6e5ebd687a50c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f5599ffb640 1 -- 172.21.15.133:0/4121829937 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55900a7070 con 0x7f559c0b1f70 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f5599ffb640 1 -- 172.21.15.133:0/4121829937 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55900b0070 con 0x7f559c0b1f70 2024-04-25T10:06:23.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f5599ffb640 1 -- 172.21.15.133:0/4121829937 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55900ab890 con 0x7f559c0b1f70 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 -- 172.21.15.133:0/4121829937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c0b1f70 msgr2=0x7f559c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 --2- 172.21.15.133:0/4121829937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c0b1f70 0x7f559c06dcf0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f559c06f5f0 tx=0x7f559001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 -- 172.21.15.133:0/4121829937 shutdown_connections 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 --2- 172.21.15.133:0/4121829937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f559c0ce330 0x7f559c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 --2- 172.21.15.133:0/4121829937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c0b1f70 0x7f559c06dcf0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 --2- 172.21.15.133:0/4121829937 >> v2:172.21.15.162:3300/0 conn(0x7f559c15c9a0 0x7f559c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 -- 172.21.15.133:0/4121829937 >> 172.21.15.133:0/4121829937 conn(0x7f559c067000 msgr2=0x7f559c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 -- 172.21.15.133:0/4121829937 shutdown_connections 2024-04-25T10:06:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 -- 172.21.15.133:0/4121829937 wait complete. 2024-04-25T10:06:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 Processor -- start 2024-04-25T10:06:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 -- start start 2024-04-25T10:06:23.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f559c0b1f70 0x7f559c147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c0ce330 0x7f559c146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f559c15c9a0 0x7f559c153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c15cf00 con 0x7f559c0ce330 2024-04-25T10:06:23.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c15cd80 con 0x7f559c15c9a0 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c15d950 con 0x7f559c0b1f70 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f559a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c0ce330 0x7f559c146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f559a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c0ce330 0x7f559c146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41718/0 (socket says 172.21.15.133:41718) 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f559a7fc640 1 -- 172.21.15.133:0/2675632945 learned_addr learned my addr 172.21.15.133:0/2675632945 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f559affd640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f559c0b1f70 0x7f559c147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f559b7fe640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f559c15c9a0 0x7f559c153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f559a7fc640 1 -- 172.21.15.133:0/2675632945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f559c0b1f70 msgr2=0x7f559c147930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f559a7fc640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f559c0b1f70 0x7f559c147930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f559a7fc640 1 -- 172.21.15.133:0/2675632945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f559c15c9a0 msgr2=0x7f559c153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f559a7fc640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f559c15c9a0 0x7f559c153440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f559a7fc640 1 -- 172.21.15.133:0/2675632945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c0cb7e0 con 0x7f559c0ce330 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f559a7fc640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c0ce330 0x7f559c146520 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f559001aba0 tx=0x7f559001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55837fe640 1 -- 172.21.15.133:0/2675632945 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55900b4800 con 0x7f559c0ce330 2024-04-25T10:06:23.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55837fe640 1 -- 172.21.15.133:0/2675632945 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55900b49a0 con 0x7f559c0ce330 2024-04-25T10:06:23.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 -- 172.21.15.133:0/2675632945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559c0cb240 con 0x7f559c0ce330 2024-04-25T10:06:23.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55837fe640 1 -- 172.21.15.133:0/2675632945 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55900b4d90 con 0x7f559c0ce330 2024-04-25T10:06:23.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.030+0000 7f55a14f6640 1 -- 172.21.15.133:0/2675632945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c148450 con 0x7f559c0ce330 2024-04-25T10:06:23.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.034+0000 7f55a14f6640 1 -- 172.21.15.133:0/2675632945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559c06bc20 con 0x7f559c0ce330 2024-04-25T10:06:23.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.034+0000 7f55837fe640 1 -- 172.21.15.133:0/2675632945 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55900b3380 con 0x7f559c0ce330 2024-04-25T10:06:23.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.034+0000 7f55837fe640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5588066f60 0x7f5588069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.034+0000 7f55837fe640 1 -- 172.21.15.133:0/2675632945 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f559011f8a0 con 0x7f559c0ce330 2024-04-25T10:06:23.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.034+0000 7f559affd640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5588066f60 0x7f5588069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.034+0000 7f559affd640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5588066f60 0x7f5588069420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f558c006fd0 tx=0x7f558c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.038+0000 7f55837fe640 1 -- 172.21.15.133:0/2675632945 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55900c4060 con 0x7f559c0ce330 2024-04-25T10:06:23.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.198+0000 7f55a14f6640 1 -- 172.21.15.133:0/2675632945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f559c06e440 con 0x7f559c0ce330 2024-04-25T10:06:23.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.198+0000 7f55837fe640 1 -- 172.21.15.133:0/2675632945 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16966 (secure 0 0 0) 0x7f5590016110 con 0x7f559c0ce330 2024-04-25T10:06:23.203 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:23.203 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":54,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:05:38.186046+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:34.805104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:35.512057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:23.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 -- 172.21.15.133:0/2675632945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5588066f60 msgr2=0x7f5588069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5588066f60 0x7f5588069420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f558c006fd0 tx=0x7f558c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:23.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 -- 172.21.15.133:0/2675632945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c0ce330 msgr2=0x7f559c146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c0ce330 0x7f559c146520 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f559001aba0 tx=0x7f559001ad30 comp rx=0 tx=0).stop 2024-04-25T10:06:23.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 -- 172.21.15.133:0/2675632945 shutdown_connections 2024-04-25T10:06:23.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f559c15c9a0 0x7f559c153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5588066f60 0x7f5588069420 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c0ce330 0x7f559c146520 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 --2- 172.21.15.133:0/2675632945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f559c0b1f70 0x7f559c147930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 -- 172.21.15.133:0/2675632945 >> 172.21.15.133:0/2675632945 conn(0x7f559c067000 msgr2=0x7f559c15afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 -- 172.21.15.133:0/2675632945 shutdown_connections 2024-04-25T10:06:23.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.202+0000 7f55a14f6640 1 -- 172.21.15.133:0/2675632945 wait complete. 2024-04-25T10:06:23.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:34.805104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:35.512057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:23.223 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:23.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.338+0000 7fc0f34e0640 1 Processor -- start 2024-04-25T10:06:23.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.338+0000 7fc0f34e0640 1 -- start start 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f34e0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc0ec15d9a0 0x7fc0ec0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f34e0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ec15ebd0 con 0x7fc0ec0ce600 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f34e0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ec15eeb0 con 0x7fc0ec0d2030 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f34e0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ec0d0330 con 0x7fc0ec15d9a0 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f0a7b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc0ec0ce600 0x7fc0ec0d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54338/0 (socket says 172.21.15.133:54338) 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f0a7b640 1 -- 172.21.15.133:0/3443329799 learned_addr learned my addr 172.21.15.133:0/3443329799 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f127c640 1 --2- 172.21.15.133:0/3443329799 >> v2:172.21.15.162:3300/0 conn(0x7fc0ec15d9a0 0x7fc0ec0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807496255 0 0) 0x7fc0ec15ebd0 con 0x7fc0ec0ce600 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0d8003530 con 0x7fc0ec0ce600 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3829964332 0 0) 0x7fc0d8003530 con 0x7fc0ec0ce600 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 >> v2:172.21.15.162:3300/0 conn(0x7fc0ec15d9a0 msgr2=0x7fc0ec0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 --2- 172.21.15.133:0/3443329799 >> v2:172.21.15.162:3300/0 conn(0x7fc0ec15d9a0 0x7fc0ec0d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 >> v1:172.21.15.162:6789/0 conn(0x7fc0ec0d2030 legacy=0x7fc0ec15dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec0d0610 con 0x7fc0ec0ce600 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fc0dc001680 con 0x7fc0ec0ce600 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 >> v1:172.21.15.133:6789/0 conn(0x7fc0ec0ce600 legacy=0x7fc0ec0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0d80039c0 0x7fc0d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d8004810 0x7fc0d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0d80051d0 0x7fc0d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec0d0330 con 0x7fc0d80051d0 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d800c170 con 0x7fc0d80039c0 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d800c370 con 0x7fc0d8004810 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f1a7d640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0d80051d0 0x7fc0d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f127c640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0d80039c0 0x7fc0d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f0a7b640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d8004810 0x7fc0d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f1a7d640 1 -- 172.21.15.133:0/3443329799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d8004810 msgr2=0x7fc0d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f1a7d640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d8004810 0x7fc0d8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f1a7d640 1 -- 172.21.15.133:0/3443329799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0d80039c0 msgr2=0x7fc0d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f1a7d640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0d80039c0 0x7fc0d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f1a7d640 1 -- 172.21.15.133:0/3443329799 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d8003530 con 0x7fc0d80051d0 2024-04-25T10:06:23.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f0a7b640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d8004810 0x7fc0d8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.342+0000 7fc0f1a7d640 1 -- 172.21.15.133:0/3443329799 reap_dead start 2024-04-25T10:06:23.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f1a7d640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0d80051d0 0x7fc0d800b9d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4001e70 tx=0x7fc0e401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723528a6372cb106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4001a90 con 0x7fc0d80051d0 2024-04-25T10:06:23.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e4003bc0 con 0x7fc0d80051d0 2024-04-25T10:06:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0e3fff640 1 -- 172.21.15.133:0/3443329799 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4005020 con 0x7fc0d80051d0 2024-04-25T10:06:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3443329799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0d80051d0 msgr2=0x7fc0d800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0d80051d0 0x7fc0d800b9d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4001e70 tx=0x7fc0e401c780 comp rx=0 tx=0).stop 2024-04-25T10:06:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3443329799 shutdown_connections 2024-04-25T10:06:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 --2- 172.21.15.133:0/3443329799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0d80051d0 0x7fc0d800b9d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3443329799 >> 172.21.15.133:0/3443329799 conn(0x7fc0ec06bf80 msgr2=0x7fc0ec06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3443329799 shutdown_connections 2024-04-25T10:06:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3443329799 wait complete. 2024-04-25T10:06:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 Processor -- start 2024-04-25T10:06:23.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 -- start start 2024-04-25T10:06:23.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d80051d0 0x7fc0ec0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec1f2de0 0x7fc0ec15f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0ec15f630 0x7fc0ec0ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec0b96a0 con 0x7fc0ec15f630 2024-04-25T10:06:23.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec0cffd0 con 0x7fc0ec1f2de0 2024-04-25T10:06:23.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec0b9820 con 0x7fc0d80051d0 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f1a7d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0ec15f630 0x7fc0ec0ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f1a7d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0ec15f630 0x7fc0ec0ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41750/0 (socket says 172.21.15.133:41750) 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f1a7d640 1 -- 172.21.15.133:0/3620535799 learned_addr learned my addr 172.21.15.133:0/3620535799 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f127c640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d80051d0 0x7fc0ec0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f0a7b640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec1f2de0 0x7fc0ec15f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f1a7d640 1 -- 172.21.15.133:0/3620535799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d80051d0 msgr2=0x7fc0ec0c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f1a7d640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d80051d0 0x7fc0ec0c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f1a7d640 1 -- 172.21.15.133:0/3620535799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec1f2de0 msgr2=0x7fc0ec15f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f1a7d640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec1f2de0 0x7fc0ec15f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f1a7d640 1 -- 172.21.15.133:0/3620535799 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec14d020 con 0x7fc0ec15f630 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f1a7d640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0ec15f630 0x7fc0ec0ca900 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc0e401cc60 tx=0x7fc0d8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0e1ffb640 1 -- 172.21.15.133:0/3620535799 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4001a90 con 0x7fc0ec15f630 2024-04-25T10:06:23.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0e1ffb640 1 -- 172.21.15.133:0/3620535799 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e4001260 con 0x7fc0ec15f630 2024-04-25T10:06:23.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3620535799 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ec14d300 con 0x7fc0ec15f630 2024-04-25T10:06:23.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0e1ffb640 1 -- 172.21.15.133:0/3620535799 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4006b60 con 0x7fc0ec15f630 2024-04-25T10:06:23.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.346+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3620535799 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec14d8c0 con 0x7fc0ec15f630 2024-04-25T10:06:23.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.350+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3620535799 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec068e80 con 0x7fc0ec15f630 2024-04-25T10:06:23.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.350+0000 7fc0e1ffb640 1 -- 172.21.15.133:0/3620535799 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0e40af030 con 0x7fc0ec15f630 2024-04-25T10:06:23.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.350+0000 7fc0e1ffb640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0d0066f60 0x7fc0d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.350+0000 7fc0e1ffb640 1 -- 172.21.15.133:0/3620535799 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc0e411dd80 con 0x7fc0ec15f630 2024-04-25T10:06:23.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.350+0000 7fc0f127c640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0d0066f60 0x7fc0d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.350+0000 7fc0f127c640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0d0066f60 0x7fc0d0069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009aa0 tx=0x7fc0e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.354+0000 7fc0e1ffb640 1 -- 172.21.15.133:0/3620535799 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0e4016490 con 0x7fc0ec15f630 2024-04-25T10:06:23.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.510+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3620535799 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0ec161e60 con 0x7fc0ec15f630 2024-04-25T10:06:23.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0e1ffb640 1 -- 172.21.15.133:0/3620535799 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0e40eec90 con 0x7fc0ec15f630 2024-04-25T10:06:23.517 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:06:23.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3620535799 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0d0066f60 msgr2=0x7fc0d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0d0066f60 0x7fc0d0069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009aa0 tx=0x7fc0e8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:23.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3620535799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0ec15f630 msgr2=0x7fc0ec0ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0ec15f630 0x7fc0ec0ca900 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc0e401cc60 tx=0x7fc0d8005840 comp rx=0 tx=0).stop 2024-04-25T10:06:23.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3620535799 shutdown_connections 2024-04-25T10:06:23.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0d0066f60 0x7fc0d0069420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0ec15f630 0x7fc0ec0ca900 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0ec1f2de0 0x7fc0ec15f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 --2- 172.21.15.133:0/3620535799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0d80051d0 0x7fc0ec0c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3620535799 >> 172.21.15.133:0/3620535799 conn(0x7fc0ec06bf80 msgr2=0x7fc0ec0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3620535799 shutdown_connections 2024-04-25T10:06:23.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.514+0000 7fc0f34e0640 1 -- 172.21.15.133:0/3620535799 wait complete. 2024-04-25T10:06:23.536 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:23.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd91ba37640 1 Processor -- start 2024-04-25T10:06:23.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd91ba37640 1 -- start start 2024-04-25T10:06:23.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd91ba37640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914145170 0x7fd91406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd91ba37640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9140ce2e0 con 0x7fd91415ced0 2024-04-25T10:06:23.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd91ba37640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91406d4b0 con 0x7fd91406e4d0 2024-04-25T10:06:23.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd91ba37640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91406db00 con 0x7fd914145170 2024-04-25T10:06:23.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd919fd4640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd91415ced0 0x7fd9140b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54348/0 (socket says 172.21.15.133:54348) 2024-04-25T10:06:23.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd919fd4640 1 -- 172.21.15.133:0/470622740 learned_addr learned my addr 172.21.15.133:0/470622740 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:23.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd9197d3640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914145170 0x7fd91406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387214372 0 0) 0x7fd9140ce2e0 con 0x7fd91415ced0 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd900003530 con 0x7fd91415ced0 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 316723630 0 0) 0x7fd900003530 con 0x7fd91415ced0 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914145170 msgr2=0x7fd91406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914145170 0x7fd91406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 >> v1:172.21.15.162:6789/0 conn(0x7fd91406e4d0 legacy=0x7fd9140cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91406d790 con 0x7fd91415ced0 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fd90c001680 con 0x7fd91415ced0 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 >> v1:172.21.15.133:6789/0 conn(0x7fd91415ced0 legacy=0x7fd9140b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9000039f0 0x7fd900003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd900004810 0x7fd900004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914145170 0x7fd9000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91406db00 con 0x7fd9000039f0 2024-04-25T10:06:23.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90000ba80 con 0x7fd900004810 2024-04-25T10:06:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90000bc80 con 0x7fd914145170 2024-04-25T10:06:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd9197d3640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9000039f0 0x7fd900003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd919fd4640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914145170 0x7fd9000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd9197d3640 1 -- 172.21.15.133:0/470622740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914145170 msgr2=0x7fd9000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd9197d3640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914145170 0x7fd9000051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd918fd2640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd900004810 0x7fd900004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd9197d3640 1 -- 172.21.15.133:0/470622740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd900004810 msgr2=0x7fd900004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd9197d3640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd900004810 0x7fd900004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.658+0000 7fd9197d3640 1 -- 172.21.15.133:0/470622740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd900003530 con 0x7fd9000039f0 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd9197d3640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9000039f0 0x7fd900003e10 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd910009f90 tx=0x7fd91001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f75595024a271e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100b2070 con 0x7fd9000039f0 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9100a5a10 con 0x7fd9000039f0 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd90bfff640 1 -- 172.21.15.133:0/470622740 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910009c10 con 0x7fd9000039f0 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 -- 172.21.15.133:0/470622740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9000039f0 msgr2=0x7fd900003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 --2- 172.21.15.133:0/470622740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9000039f0 0x7fd900003e10 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd910009f90 tx=0x7fd91001cc20 comp rx=0 tx=0).stop 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd919fd4640 1 -- 172.21.15.133:0/470622740 reap_dead start 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 -- 172.21.15.133:0/470622740 shutdown_connections 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 -- 172.21.15.133:0/470622740 >> 172.21.15.133:0/470622740 conn(0x7fd914066f70 msgr2=0x7fd91406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 -- 172.21.15.133:0/470622740 shutdown_connections 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 -- 172.21.15.133:0/470622740 wait complete. 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 Processor -- start 2024-04-25T10:06:23.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 -- start start 2024-04-25T10:06:23.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9000039f0 0x7fd914136e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914139510 0x7fd914131ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd914137340 0x7fd9141321e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914145550 con 0x7fd914137340 2024-04-25T10:06:23.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140b1fe0 con 0x7fd9000039f0 2024-04-25T10:06:23.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd91ba37640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91406df70 con 0x7fd914139510 2024-04-25T10:06:23.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd919fd4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd914137340 0x7fd9141321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd919fd4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd914137340 0x7fd9141321e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41764/0 (socket says 172.21.15.133:41764) 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd919fd4640 1 -- 172.21.15.133:0/621828485 learned_addr learned my addr 172.21.15.133:0/621828485 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd9197d3640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9000039f0 0x7fd914136e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd918fd2640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914139510 0x7fd914131ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd919fd4640 1 -- 172.21.15.133:0/621828485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914139510 msgr2=0x7fd914131ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd919fd4640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914139510 0x7fd914131ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd919fd4640 1 -- 172.21.15.133:0/621828485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9000039f0 msgr2=0x7fd914136e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd919fd4640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9000039f0 0x7fd914136e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd919fd4640 1 -- 172.21.15.133:0/621828485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9141f75c0 con 0x7fd914137340 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd9197d3640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9000039f0 0x7fd914136e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.662+0000 7fd919fd4640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd914137340 0x7fd9141321e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd91406f700 tx=0x7fd90c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.666+0000 7fd909ffb640 1 -- 172.21.15.133:0/621828485 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0075a0 con 0x7fd914137340 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.666+0000 7fd909ffb640 1 -- 172.21.15.133:0/621828485 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c0047f0 con 0x7fd914137340 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.666+0000 7fd91ba37640 1 -- 172.21.15.133:0/621828485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9141399a0 con 0x7fd914137340 2024-04-25T10:06:23.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.666+0000 7fd909ffb640 1 -- 172.21.15.133:0/621828485 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c004be0 con 0x7fd914137340 2024-04-25T10:06:23.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.666+0000 7fd91ba37640 1 -- 172.21.15.133:0/621828485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd914139f60 con 0x7fd914137340 2024-04-25T10:06:23.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.666+0000 7fd91ba37640 1 -- 172.21.15.133:0/621828485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd914068e80 con 0x7fd914137340 2024-04-25T10:06:23.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.666+0000 7fd909ffb640 1 -- 172.21.15.133:0/621828485 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd90c01b020 con 0x7fd914137340 2024-04-25T10:06:23.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.666+0000 7fd909ffb640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8f8066f60 0x7fd8f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.666+0000 7fd909ffb640 1 -- 172.21.15.133:0/621828485 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd90c084d70 con 0x7fd914137340 2024-04-25T10:06:23.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.666+0000 7fd9197d3640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8f8066f60 0x7fd8f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.666+0000 7fd9197d3640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8f8066f60 0x7fd8f8069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd9100af000 tx=0x7fd9100a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.670+0000 7fd909ffb640 1 -- 172.21.15.133:0/621828485 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd90c055be0 con 0x7fd914137340 2024-04-25T10:06:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd91ba37640 1 -- 172.21.15.133:0/621828485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd91415e680 con 0x7fd914137340 2024-04-25T10:06:23.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd909ffb640 1 -- 172.21.15.133:0/621828485 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd90c0554e0 con 0x7fd914137340 2024-04-25T10:06:23.830 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:06:23.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd91ba37640 1 -- 172.21.15.133:0/621828485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8f8066f60 msgr2=0x7fd8f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd91ba37640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8f8066f60 0x7fd8f8069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd9100af000 tx=0x7fd9100a7040 comp rx=0 tx=0).stop 2024-04-25T10:06:23.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd91ba37640 1 -- 172.21.15.133:0/621828485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd914137340 msgr2=0x7fd9141321e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd91ba37640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd914137340 0x7fd9141321e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd91406f700 tx=0x7fd90c00d9d0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd91ba37640 1 -- 172.21.15.133:0/621828485 shutdown_connections 2024-04-25T10:06:23.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd91ba37640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8f8066f60 0x7fd8f8069420 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd91ba37640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd914137340 0x7fd9141321e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd91ba37640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd914139510 0x7fd914131ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd91ba37640 1 --2- 172.21.15.133:0/621828485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9000039f0 0x7fd914136e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.826+0000 7fd91ba37640 1 -- 172.21.15.133:0/621828485 >> 172.21.15.133:0/621828485 conn(0x7fd914066f70 msgr2=0x7fd91415f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.830+0000 7fd91ba37640 1 -- 172.21.15.133:0/621828485 shutdown_connections 2024-04-25T10:06:23.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.830+0000 7fd91ba37640 1 -- 172.21.15.133:0/621828485 wait complete. 2024-04-25T10:06:23.848 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:23.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.954+0000 7ff8cfb2b640 1 Processor -- start 2024-04-25T10:06:23.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cfb2b640 1 -- start start 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cfb2b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff8c80b2150 0x7ff8c806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cfb2b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c815ac50 con 0x7ff8c815b9b0 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cfb2b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c80cd180 con 0x7ff8c8143ea0 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cfb2b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c80ccf20 con 0x7ff8c80b2150 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8ce0c8640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff8c815b9b0 0x7ff8c80b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54358/0 (socket says 172.21.15.133:54358) 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8ce0c8640 1 -- 172.21.15.133:0/3225202942 learned_addr learned my addr 172.21.15.133:0/3225202942 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cd0c6640 1 --2- 172.21.15.133:0/3225202942 >> v2:172.21.15.162:3300/0 conn(0x7ff8c80b2150 0x7ff8c806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217839253 0 0) 0x7ff8c00014a0 con 0x7ff8c815b9b0 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b4003530 con 0x7ff8c815b9b0 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1696705546 0 0) 0x7ff8b4003530 con 0x7ff8c815b9b0 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 >> v2:172.21.15.162:3300/0 conn(0x7ff8c80b2150 msgr2=0x7ff8c806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 --2- 172.21.15.133:0/3225202942 >> v2:172.21.15.162:3300/0 conn(0x7ff8c80b2150 0x7ff8c806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 >> v1:172.21.15.162:6789/0 conn(0x7ff8c8143ea0 legacy=0x7ff8c80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c815aed0 con 0x7ff8c815b9b0 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff8c0001bd0 con 0x7ff8c815b9b0 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 >> v1:172.21.15.133:6789/0 conn(0x7ff8c815b9b0 legacy=0x7ff8c80b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 --2- 172.21.15.133:0/3225202942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8b40039c0 0x7ff8b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 --2- 172.21.15.133:0/3225202942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8b4004810 0x7ff8b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 --2- 172.21.15.133:0/3225202942 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8b40051d0 0x7ff8b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80ccf20 con 0x7ff8b4004810 2024-04-25T10:06:23.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b400c170 con 0x7ff8b40039c0 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b400c370 con 0x7ff8b40051d0 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cd0c6640 1 --2- 172.21.15.133:0/3225202942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8b4004810 0x7ff8b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cd8c7640 1 --2- 172.21.15.133:0/3225202942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8b40039c0 0x7ff8b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cd0c6640 1 -- 172.21.15.133:0/3225202942 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8b40051d0 msgr2=0x7ff8b400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cd0c6640 1 --2- 172.21.15.133:0/3225202942 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8b40051d0 0x7ff8b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cd0c6640 1 -- 172.21.15.133:0/3225202942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8b40039c0 msgr2=0x7ff8b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cd0c6640 1 --2- 172.21.15.133:0/3225202942 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8b40039c0 0x7ff8b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8ce0c8640 1 -- 172.21.15.133:0/3225202942 reap_dead start 2024-04-25T10:06:23.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.958+0000 7ff8cd0c6640 1 -- 172.21.15.133:0/3225202942 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b4003530 con 0x7ff8b4004810 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cd0c6640 1 --2- 172.21.15.133:0/3225202942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8b4004810 0x7ff8b4004c90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff8c806f700 tx=0x7ff8bc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d75d3724f441d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc009690 con 0x7ff8b4004810 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc0a64c0 con 0x7ff8b4004810 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cc8c5640 1 -- 172.21.15.133:0/3225202942 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0a6890 con 0x7ff8b4004810 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/3225202942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8b4004810 msgr2=0x7ff8b4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 --2- 172.21.15.133:0/3225202942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8b4004810 0x7ff8b4004c90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff8c806f700 tx=0x7ff8bc01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/3225202942 shutdown_connections 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 --2- 172.21.15.133:0/3225202942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8b4004810 0x7ff8b4004c90 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/3225202942 >> 172.21.15.133:0/3225202942 conn(0x7ff8c8066f70 msgr2=0x7ff8c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/3225202942 shutdown_connections 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/3225202942 wait complete. 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 Processor -- start 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 -- start start 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8b4004810 0x7ff8c8156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c8152730 0x7ff8c8152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c81f46b0 0x7ff8c81f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c806ed40 con 0x7ff8c8152730 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c806ebc0 con 0x7ff8c81f46b0 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c815e240 con 0x7ff8b4004810 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cd0c6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c8152730 0x7ff8c8152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cd0c6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c8152730 0x7ff8c8152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41784/0 (socket says 172.21.15.133:41784) 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cd0c6640 1 -- 172.21.15.133:0/1862862814 learned_addr learned my addr 172.21.15.133:0/1862862814 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8ce0c8640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c81f46b0 0x7ff8c81f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cd8c7640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8b4004810 0x7ff8c8156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cd0c6640 1 -- 172.21.15.133:0/1862862814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8b4004810 msgr2=0x7ff8c8156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cd0c6640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8b4004810 0x7ff8c8156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cd0c6640 1 -- 172.21.15.133:0/1862862814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c81f46b0 msgr2=0x7ff8c81f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cd0c6640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c81f46b0 0x7ff8c81f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cd0c6640 1 -- 172.21.15.133:0/1862862814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c81f5520 con 0x7ff8c8152730 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8ce0c8640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c81f46b0 0x7ff8c81f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cd0c6640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c8152730 0x7ff8c8152bb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0b0040 tx=0x7ff8bc019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8ba7fc640 1 -- 172.21.15.133:0/1862862814 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc00f080 con 0x7ff8c8152730 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8ba7fc640 1 -- 172.21.15.133:0/1862862814 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc0be070 con 0x7ff8c8152730 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/1862862814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c81f5800 con 0x7ff8c8152730 2024-04-25T10:06:23.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.962+0000 7ff8ba7fc640 1 -- 172.21.15.133:0/1862862814 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0a6870 con 0x7ff8c8152730 2024-04-25T10:06:23.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.966+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/1862862814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c81f5dc0 con 0x7ff8c8152730 2024-04-25T10:06:23.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.966+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/1862862814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c8068e80 con 0x7ff8c8152730 2024-04-25T10:06:23.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.966+0000 7ff8ba7fc640 1 -- 172.21.15.133:0/1862862814 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8bc0a6a10 con 0x7ff8c8152730 2024-04-25T10:06:23.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.966+0000 7ff8ba7fc640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8ac066f90 0x7ff8ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.966+0000 7ff8ba7fc640 1 -- 172.21.15.133:0/1862862814 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff8bc11f7a0 con 0x7ff8c8152730 2024-04-25T10:06:23.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.966+0000 7ff8cd8c7640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8ac066f90 0x7ff8ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.966+0000 7ff8cd8c7640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8ac066f90 0x7ff8ac069450 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4001620 tx=0x7ff8c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:23.970+0000 7ff8ba7fc640 1 -- 172.21.15.133:0/1862862814 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8bc016700 con 0x7ff8c8152730 2024-04-25T10:06:24.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/1862862814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff8c815eb90 con 0x7ff8c8152730 2024-04-25T10:06:24.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8ba7fc640 1 -- 172.21.15.133:0/1862862814 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff8bc0f06b0 con 0x7ff8c8152730 2024-04-25T10:06:24.125 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:06:24.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/1862862814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8ac066f90 msgr2=0x7ff8ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8ac066f90 0x7ff8ac069450 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4001620 tx=0x7ff8c4009040 comp rx=0 tx=0).stop 2024-04-25T10:06:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/1862862814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c8152730 msgr2=0x7ff8c8152bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c8152730 0x7ff8c8152bb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0b0040 tx=0x7ff8bc019790 comp rx=0 tx=0).stop 2024-04-25T10:06:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/1862862814 shutdown_connections 2024-04-25T10:06:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8c81f46b0 0x7ff8c81f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff8ac066f90 0x7ff8ac069450 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c8152730 0x7ff8c8152bb0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 --2- 172.21.15.133:0/1862862814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8b4004810 0x7ff8c8156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/1862862814 >> 172.21.15.133:0/1862862814 conn(0x7ff8c8066f70 msgr2=0x7ff8c80cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/1862862814 shutdown_connections 2024-04-25T10:06:24.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.122+0000 7ff8cfb2b640 1 -- 172.21.15.133:0/1862862814 wait complete. 2024-04-25T10:06:24.144 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:24.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.246+0000 7f937579a640 1 Processor -- start 2024-04-25T10:06:24.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.246+0000 7f937579a640 1 -- start start 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9370159dc0 0x7f93700cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f937015a1a0 0x7f937015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93700ce880 con 0x7f937015e050 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937015e4a0 con 0x7f937015a1a0 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937006e360 con 0x7f9370159dc0 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936e7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f937015a1a0 0x7f937015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936e7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f937015a1a0 0x7f937015db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41788/0 (socket says 172.21.15.133:41788) 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936e7fc640 1 -- 172.21.15.133:0/2416712699 learned_addr learned my addr 172.21.15.133:0/2416712699 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936effd640 1 --2- 172.21.15.133:0/2416712699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9370159dc0 0x7f93700cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936e7fc640 1 -- 172.21.15.133:0/2416712699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9370159dc0 msgr2=0x7f93700cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936e7fc640 1 --2- 172.21.15.133:0/2416712699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9370159dc0 0x7f93700cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936e7fc640 1 -- 172.21.15.133:0/2416712699 >> v1:172.21.15.162:6789/0 conn(0x7f937015e050 legacy=0x7f93700ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936e7fc640 1 -- 172.21.15.133:0/2416712699 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93700c49e0 con 0x7f937015a1a0 2024-04-25T10:06:24.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936e7fc640 1 --2- 172.21.15.133:0/2416712699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937015a1a0 0x7f937015db10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9358009870 tx=0x7f935801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e4b24087544744 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936dffb640 1 -- 172.21.15.133:0/2416712699 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580b0070 con 0x7f937015a1a0 2024-04-25T10:06:24.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936dffb640 1 -- 172.21.15.133:0/2416712699 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93580ab600 con 0x7f937015a1a0 2024-04-25T10:06:24.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f936dffb640 1 -- 172.21.15.133:0/2416712699 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580ab9d0 con 0x7f937015a1a0 2024-04-25T10:06:24.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 -- 172.21.15.133:0/2416712699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937015a1a0 msgr2=0x7f937015db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 --2- 172.21.15.133:0/2416712699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937015a1a0 0x7f937015db10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9358009870 tx=0x7f935801c520 comp rx=0 tx=0).stop 2024-04-25T10:06:24.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 -- 172.21.15.133:0/2416712699 shutdown_connections 2024-04-25T10:06:24.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 --2- 172.21.15.133:0/2416712699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937015a1a0 0x7f937015db10 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 --2- 172.21.15.133:0/2416712699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9370159dc0 0x7f93700cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 -- 172.21.15.133:0/2416712699 >> 172.21.15.133:0/2416712699 conn(0x7f9370067000 msgr2=0x7f937006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 -- 172.21.15.133:0/2416712699 shutdown_connections 2024-04-25T10:06:24.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.250+0000 7f937579a640 1 -- 172.21.15.133:0/2416712699 wait complete. 2024-04-25T10:06:24.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f937579a640 1 Processor -- start 2024-04-25T10:06:24.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f937579a640 1 -- start start 2024-04-25T10:06:24.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f937579a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9370159dc0 0x7f9370154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f937579a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937015e050 0x7f9370152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f937579a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9370152f80 0x7f93700ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f937579a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700c4e40 con 0x7f9370152f80 2024-04-25T10:06:24.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f937579a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700c4cc0 con 0x7f9370159dc0 2024-04-25T10:06:24.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f937579a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93700cf300 con 0x7f937015e050 2024-04-25T10:06:24.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9370152f80 0x7f93700ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9370152f80 0x7f93700ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41792/0 (socket says 172.21.15.133:41792) 2024-04-25T10:06:24.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936f7fe640 1 -- 172.21.15.133:0/674110985 learned_addr learned my addr 172.21.15.133:0/674110985 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:24.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936effd640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9370159dc0 0x7f9370154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936e7fc640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937015e050 0x7f9370152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936f7fe640 1 -- 172.21.15.133:0/674110985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937015e050 msgr2=0x7f9370152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936f7fe640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937015e050 0x7f9370152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936f7fe640 1 -- 172.21.15.133:0/674110985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9370159dc0 msgr2=0x7f9370154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936f7fe640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9370159dc0 0x7f9370154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936f7fe640 1 -- 172.21.15.133:0/674110985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93701fa070 con 0x7f9370152f80 2024-04-25T10:06:24.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936effd640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9370159dc0 0x7f9370154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:24.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f936f7fe640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9370152f80 0x7f93700ca610 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f936800b970 tx=0x7f936800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f93677fe640 1 -- 172.21.15.133:0/674110985 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368019070 con 0x7f9370152f80 2024-04-25T10:06:24.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f93677fe640 1 -- 172.21.15.133:0/674110985 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936800cb40 con 0x7f9370152f80 2024-04-25T10:06:24.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f937579a640 1 -- 172.21.15.133:0/674110985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93701fa350 con 0x7f9370152f80 2024-04-25T10:06:24.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f93677fe640 1 -- 172.21.15.133:0/674110985 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368014440 con 0x7f9370152f80 2024-04-25T10:06:24.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f937579a640 1 -- 172.21.15.133:0/674110985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93701fa910 con 0x7f9370152f80 2024-04-25T10:06:24.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.254+0000 7f937579a640 1 -- 172.21.15.133:0/674110985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9370068ef0 con 0x7f9370152f80 2024-04-25T10:06:24.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.258+0000 7f93677fe640 1 -- 172.21.15.133:0/674110985 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f936801c070 con 0x7f9370152f80 2024-04-25T10:06:24.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.258+0000 7f93677fe640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9350066f60 0x7f9350069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.258+0000 7f936effd640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9350066f60 0x7f9350069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.258+0000 7f93677fe640 1 -- 172.21.15.133:0/674110985 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9368084d90 con 0x7f9370152f80 2024-04-25T10:06:24.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.258+0000 7f936effd640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9350066f60 0x7f9350069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9360006fd0 tx=0x7f9360008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.262+0000 7f93677fe640 1 -- 172.21.15.133:0/674110985 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9368055ca0 con 0x7f9370152f80 2024-04-25T10:06:24.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.454+0000 7f937579a640 1 -- 172.21.15.133:0/674110985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9370149280 con 0x7f9370152f80 2024-04-25T10:06:24.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.454+0000 7f93677fe640 1 -- 172.21.15.133:0/674110985 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f93680555a0 con 0x7f9370152f80 2024-04-25T10:06:24.460 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:24.460 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:06:17.247723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:24.460 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 26 2024-04-25T10:06:24.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 -- 172.21.15.133:0/674110985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9350066f60 msgr2=0x7f9350069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9350066f60 0x7f9350069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9360006fd0 tx=0x7f9360008040 comp rx=0 tx=0).stop 2024-04-25T10:06:24.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 -- 172.21.15.133:0/674110985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9370152f80 msgr2=0x7f93700ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9370152f80 0x7f93700ca610 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f936800b970 tx=0x7f936800be30 comp rx=0 tx=0).stop 2024-04-25T10:06:24.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 -- 172.21.15.133:0/674110985 shutdown_connections 2024-04-25T10:06:24.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9350066f60 0x7f9350069420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9370152f80 0x7f93700ca610 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937015e050 0x7f9370152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 --2- 172.21.15.133:0/674110985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9370159dc0 0x7f9370154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 -- 172.21.15.133:0/674110985 >> 172.21.15.133:0/674110985 conn(0x7f9370067000 msgr2=0x7f93700ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 -- 172.21.15.133:0/674110985 shutdown_connections 2024-04-25T10:06:24.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.458+0000 7f937579a640 1 -- 172.21.15.133:0/674110985 wait complete. 2024-04-25T10:06:24.479 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:24.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.598+0000 7f536f173640 1 Processor -- start 2024-04-25T10:06:24.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.598+0000 7f536f173640 1 -- start start 2024-04-25T10:06:24.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.598+0000 7f536f173640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5368158110 0x7f53680b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.598+0000 7f536f173640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53681584f0 0x7f536815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.598+0000 7f536f173640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f536806e460 0x7f536806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.598+0000 7f536f173640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536806edf0 con 0x7f536806e460 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.598+0000 7f536f173640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53680cbe60 con 0x7f5368158110 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.598+0000 7f536f173640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53680b19b0 con 0x7f53681584f0 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.598+0000 7f536d710640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f536806e460 0x7f536806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.598+0000 7f536d710640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f536806e460 0x7f536806e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41794/0 (socket says 172.21.15.133:41794) 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.598+0000 7f536d710640 1 -- 172.21.15.133:0/1255264108 learned_addr learned my addr 172.21.15.133:0/1255264108 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f5367fff640 1 --2- 172.21.15.133:0/1255264108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53681584f0 0x7f536815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536cf0f640 1 --2- 172.21.15.133:0/1255264108 >> v2:172.21.15.162:3300/0 conn(0x7f5368158110 0x7f53680b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536d710640 1 -- 172.21.15.133:0/1255264108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53681584f0 msgr2=0x7f536815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536d710640 1 --2- 172.21.15.133:0/1255264108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53681584f0 0x7f536815ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536d710640 1 -- 172.21.15.133:0/1255264108 >> v2:172.21.15.162:3300/0 conn(0x7f5368158110 msgr2=0x7f53680b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536d710640 1 --2- 172.21.15.133:0/1255264108 >> v2:172.21.15.162:3300/0 conn(0x7f5368158110 0x7f53680b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536d710640 1 -- 172.21.15.133:0/1255264108 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536815b360 con 0x7f536806e460 2024-04-25T10:06:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536d710640 1 --2- 172.21.15.133:0/1255264108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f536806e460 0x7f536806e8b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f53680b4d40 tx=0x7f536001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b1836f3995ddf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f53677fe640 1 -- 172.21.15.133:0/1255264108 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53600a5840 con 0x7f536806e460 2024-04-25T10:06:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f53677fe640 1 -- 172.21.15.133:0/1255264108 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53600a59e0 con 0x7f536806e460 2024-04-25T10:06:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f53677fe640 1 -- 172.21.15.133:0/1255264108 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53600a5db0 con 0x7f536806e460 2024-04-25T10:06:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536f173640 1 -- 172.21.15.133:0/1255264108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f536806e460 msgr2=0x7f536806e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536f173640 1 --2- 172.21.15.133:0/1255264108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f536806e460 0x7f536806e8b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f53680b4d40 tx=0x7f536001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536f173640 1 -- 172.21.15.133:0/1255264108 shutdown_connections 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536f173640 1 --2- 172.21.15.133:0/1255264108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f536806e460 0x7f536806e8b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536f173640 1 --2- 172.21.15.133:0/1255264108 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53681584f0 0x7f536815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536f173640 1 --2- 172.21.15.133:0/1255264108 >> v2:172.21.15.162:3300/0 conn(0x7f5368158110 0x7f53680b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536f173640 1 -- 172.21.15.133:0/1255264108 >> 172.21.15.133:0/1255264108 conn(0x7f5368067000 msgr2=0x7f536806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536f173640 1 -- 172.21.15.133:0/1255264108 shutdown_connections 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.602+0000 7f536f173640 1 -- 172.21.15.133:0/1255264108 wait complete. 2024-04-25T10:06:24.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536f173640 1 Processor -- start 2024-04-25T10:06:24.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536f173640 1 -- start start 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536f173640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f536806e460 0x7f536814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536f173640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5368158110 0x7f536815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536f173640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53681584f0 0x7f536815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536f173640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53680cd7f0 con 0x7f536806e460 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536f173640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53680cd670 con 0x7f5368158110 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536f173640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536815eb10 con 0x7f53681584f0 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536cf0f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f536806e460 0x7f536814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536cf0f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f536806e460 0x7f536814fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41810/0 (socket says 172.21.15.133:41810) 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536cf0f640 1 -- 172.21.15.133:0/545440107 learned_addr learned my addr 172.21.15.133:0/545440107 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f5367fff640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5368158110 0x7f536815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536d710640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53681584f0 0x7f536815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536cf0f640 1 -- 172.21.15.133:0/545440107 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53681584f0 msgr2=0x7f536815d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536cf0f640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53681584f0 0x7f536815d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536cf0f640 1 -- 172.21.15.133:0/545440107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5368158110 msgr2=0x7f536815ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536cf0f640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5368158110 0x7f536815ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536cf0f640 1 -- 172.21.15.133:0/545440107 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536814ff60 con 0x7f536806e460 2024-04-25T10:06:24.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536cf0f640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f536806e460 0x7f536814fa20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f535800c820 tx=0x7f535800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f53657fa640 1 -- 172.21.15.133:0/545440107 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358015070 con 0x7f536806e460 2024-04-25T10:06:24.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f53657fa640 1 -- 172.21.15.133:0/545440107 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5358004220 con 0x7f536806e460 2024-04-25T10:06:24.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536f173640 1 -- 172.21.15.133:0/545440107 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53681515b0 con 0x7f536806e460 2024-04-25T10:06:24.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f53657fa640 1 -- 172.21.15.133:0/545440107 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535800ce10 con 0x7f536806e460 2024-04-25T10:06:24.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536f173640 1 -- 172.21.15.133:0/545440107 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5368151b70 con 0x7f536806e460 2024-04-25T10:06:24.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.606+0000 7f536f173640 1 -- 172.21.15.133:0/545440107 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5368068ef0 con 0x7f536806e460 2024-04-25T10:06:24.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.610+0000 7f53657fa640 1 -- 172.21.15.133:0/545440107 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5358007830 con 0x7f536806e460 2024-04-25T10:06:24.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.610+0000 7f53657fa640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5354066f60 0x7f5354069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.610+0000 7f53657fa640 1 -- 172.21.15.133:0/545440107 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5358085340 con 0x7f536806e460 2024-04-25T10:06:24.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.610+0000 7f5367fff640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5354066f60 0x7f5354069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.610+0000 7f5367fff640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5354066f60 0x7f5354069420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5368153df0 tx=0x7f535c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.614+0000 7f53657fa640 1 -- 172.21.15.133:0/545440107 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5358056210 con 0x7f536806e460 2024-04-25T10:06:24.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.770+0000 7f536f173640 1 -- 172.21.15.133:0/545440107 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5368153870 con 0x7f536806e460 2024-04-25T10:06:24.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f53657fa640 1 -- 172.21.15.133:0/545440107 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16966 (secure 0 0 0) 0x7f5358055b10 con 0x7f536806e460 2024-04-25T10:06:24.776 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:24.777 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":54,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:05:38.186046+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:34.805104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:35.512057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 -- 172.21.15.133:0/545440107 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5354066f60 msgr2=0x7f5354069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5354066f60 0x7f5354069420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5368153df0 tx=0x7f535c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 -- 172.21.15.133:0/545440107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f536806e460 msgr2=0x7f536814fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f536806e460 0x7f536814fa20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f535800c820 tx=0x7f535800cce0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 -- 172.21.15.133:0/545440107 shutdown_connections 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f53681584f0 0x7f536815d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5368158110 0x7f536815ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5354066f60 0x7f5354069420 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 --2- 172.21.15.133:0/545440107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f536806e460 0x7f536814fa20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 -- 172.21.15.133:0/545440107 >> 172.21.15.133:0/545440107 conn(0x7f5368067000 msgr2=0x7f53681573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 -- 172.21.15.133:0/545440107 shutdown_connections 2024-04-25T10:06:24.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.774+0000 7f536f173640 1 -- 172.21.15.133:0/545440107 wait complete. 2024-04-25T10:06:24.795 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:06:24.795 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:24.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e51f47640 1 Processor -- start 2024-04-25T10:06:24.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e51f47640 1 -- start start 2024-04-25T10:06:24.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e51f47640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3e4c15a540 0x7f3e4c0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e51f47640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e4c0d09b0 0x7f3e4c0b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e51f47640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4c15ef00 con 0x7f3e4c0cf730 2024-04-25T10:06:24.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e51f47640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4c0ce4e0 con 0x7f3e4c15a540 2024-04-25T10:06:24.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e51f47640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c15f1e0 con 0x7f3e4c0d09b0 2024-04-25T10:06:24.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3e4c15a540 0x7f3e4c0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3e4c15a540 0x7f3e4c0cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41820/0 (socket says 172.21.15.133:41820) 2024-04-25T10:06:24.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4b7fe640 1 -- 172.21.15.133:0/3310127244 learned_addr learned my addr 172.21.15.133:0/3310127244 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:24.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4bfff640 1 --2- 172.21.15.133:0/3310127244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e4c0d09b0 0x7f3e4c0b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4b7fe640 1 -- 172.21.15.133:0/3310127244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e4c0d09b0 msgr2=0x7f3e4c0b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4b7fe640 1 --2- 172.21.15.133:0/3310127244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e4c0d09b0 0x7f3e4c0b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4b7fe640 1 -- 172.21.15.133:0/3310127244 >> v1:172.21.15.162:6789/0 conn(0x7f3e4c0cf730 legacy=0x7f3e4c0d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4b7fe640 1 -- 172.21.15.133:0/3310127244 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c0ce710 con 0x7f3e4c15a540 2024-04-25T10:06:24.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4b7fe640 1 --2- 172.21.15.133:0/3310127244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e4c15a540 0x7f3e4c0cf1f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400d150 tx=0x7f3e4401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1439e7f17a6df6b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4a7fc640 1 -- 172.21.15.133:0/3310127244 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400b2d0 con 0x7f3e4c15a540 2024-04-25T10:06:24.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4a7fc640 1 -- 172.21.15.133:0/3310127244 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4400b470 con 0x7f3e4c15a540 2024-04-25T10:06:24.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.910+0000 7f3e4a7fc640 1 -- 172.21.15.133:0/3310127244 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440a7b90 con 0x7f3e4c15a540 2024-04-25T10:06:24.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 -- 172.21.15.133:0/3310127244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e4c15a540 msgr2=0x7f3e4c0cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 --2- 172.21.15.133:0/3310127244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e4c15a540 0x7f3e4c0cf1f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400d150 tx=0x7f3e4401ebf0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 -- 172.21.15.133:0/3310127244 shutdown_connections 2024-04-25T10:06:24.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 --2- 172.21.15.133:0/3310127244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e4c0d09b0 0x7f3e4c0b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 --2- 172.21.15.133:0/3310127244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e4c15a540 0x7f3e4c0cf1f0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 -- 172.21.15.133:0/3310127244 >> 172.21.15.133:0/3310127244 conn(0x7f3e4c0b2370 msgr2=0x7f3e4c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 -- 172.21.15.133:0/3310127244 shutdown_connections 2024-04-25T10:06:24.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 -- 172.21.15.133:0/3310127244 wait complete. 2024-04-25T10:06:24.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 Processor -- start 2024-04-25T10:06:24.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 -- start start 2024-04-25T10:06:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e4c0cf730 0x7f3e4c149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e4c0d09b0 0x7f3e4c150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e4c15a540 0x7f3e4c150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c15ea10 con 0x7f3e4c0d09b0 2024-04-25T10:06:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c15eb90 con 0x7f3e4c15a540 2024-04-25T10:06:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c06ac90 con 0x7f3e4c0cf730 2024-04-25T10:06:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e4c0d09b0 0x7f3e4c150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e4c0d09b0 0x7f3e4c150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41832/0 (socket says 172.21.15.133:41832) 2024-04-25T10:06:24.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4affd640 1 -- 172.21.15.133:0/901814524 learned_addr learned my addr 172.21.15.133:0/901814524 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4b7fe640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e4c0cf730 0x7f3e4c149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4bfff640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e4c15a540 0x7f3e4c150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4affd640 1 -- 172.21.15.133:0/901814524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e4c0cf730 msgr2=0x7f3e4c149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4affd640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e4c0cf730 0x7f3e4c149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4affd640 1 -- 172.21.15.133:0/901814524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e4c15a540 msgr2=0x7f3e4c150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4affd640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e4c15a540 0x7f3e4c150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4affd640 1 -- 172.21.15.133:0/901814524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c15b5f0 con 0x7f3e4c0d09b0 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4b7fe640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e4c0cf730 0x7f3e4c149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e4affd640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e4c0d09b0 0x7f3e4c150030 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000c450 tx=0x7f3e4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e33fff640 1 -- 172.21.15.133:0/901814524 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40012830 con 0x7f3e4c0d09b0 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e33fff640 1 -- 172.21.15.133:0/901814524 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e40017070 con 0x7f3e4c0d09b0 2024-04-25T10:06:24.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 -- 172.21.15.133:0/901814524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4c15b8d0 con 0x7f3e4c0d09b0 2024-04-25T10:06:24.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e33fff640 1 -- 172.21.15.133:0/901814524 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40012c30 con 0x7f3e4c0d09b0 2024-04-25T10:06:24.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.914+0000 7f3e51f47640 1 -- 172.21.15.133:0/901814524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c15be90 con 0x7f3e4c0d09b0 2024-04-25T10:06:24.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.918+0000 7f3e51f47640 1 -- 172.21.15.133:0/901814524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4c067c70 con 0x7f3e4c0d09b0 2024-04-25T10:06:24.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.918+0000 7f3e33fff640 1 -- 172.21.15.133:0/901814524 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e4001c460 con 0x7f3e4c0d09b0 2024-04-25T10:06:24.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.918+0000 7f3e33fff640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e38066f90 0x7f3e38069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.918+0000 7f3e33fff640 1 -- 172.21.15.133:0/901814524 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3e40084fe0 con 0x7f3e4c0d09b0 2024-04-25T10:06:24.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.918+0000 7f3e4b7fe640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e38066f90 0x7f3e38069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.918+0000 7f3e4b7fe640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e38066f90 0x7f3e38069450 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3e440b0000 tx=0x7f3e440a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:24.922+0000 7f3e33fff640 1 -- 172.21.15.133:0/901814524 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e40055dd0 con 0x7f3e4c0d09b0 2024-04-25T10:06:25.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.118+0000 7f3e51f47640 1 -- 172.21.15.133:0/901814524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e4c0b00c0 con 0x7f3e4c0d09b0 2024-04-25T10:06:25.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.118+0000 7f3e33fff640 1 -- 172.21.15.133:0/901814524 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f3e400556d0 con 0x7f3e4c0d09b0 2024-04-25T10:06:25.123 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:25.123 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:06:17.247723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:25.123 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 26 2024-04-25T10:06:25.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 -- 172.21.15.133:0/901814524 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e38066f90 msgr2=0x7f3e38069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e38066f90 0x7f3e38069450 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3e440b0000 tx=0x7f3e440a9070 comp rx=0 tx=0).stop 2024-04-25T10:06:25.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 -- 172.21.15.133:0/901814524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e4c0d09b0 msgr2=0x7f3e4c150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e4c0d09b0 0x7f3e4c150030 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000c450 tx=0x7f3e4000c910 comp rx=0 tx=0).stop 2024-04-25T10:06:25.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 -- 172.21.15.133:0/901814524 shutdown_connections 2024-04-25T10:06:25.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e4c15a540 0x7f3e4c150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e38066f90 0x7f3e38069450 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e4c0d09b0 0x7f3e4c150030 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 --2- 172.21.15.133:0/901814524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e4c0cf730 0x7f3e4c149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 -- 172.21.15.133:0/901814524 >> 172.21.15.133:0/901814524 conn(0x7f3e4c0b2370 msgr2=0x7f3e4c0b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 -- 172.21.15.133:0/901814524 shutdown_connections 2024-04-25T10:06:25.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.122+0000 7f3e51f47640 1 -- 172.21.15.133:0/901814524 wait complete. 2024-04-25T10:06:25.142 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0750e12640 1 Processor -- start 2024-04-25T10:06:25.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0750e12640 1 -- start start 2024-04-25T10:06:25.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0750e12640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f074c15c9a0 0x7f074c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0750e12640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f074c0b1f70 0x7f074c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0750e12640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074c0c61d0 con 0x7f074c0ce330 2024-04-25T10:06:25.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0750e12640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074c0c64b0 con 0x7f074c0b1f70 2024-04-25T10:06:25.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0750e12640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074c06d4f0 con 0x7f074c15c9a0 2024-04-25T10:06:25.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0749d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f074c0b1f70 0x7f074c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f074ad9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f074c0ce330 0x7f074c15d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54364/0 (socket says 172.21.15.133:54364) 2024-04-25T10:06:25.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f074ad9d640 1 -- 172.21.15.133:0/252285235 learned_addr learned my addr 172.21.15.133:0/252285235 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:25.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0749d9b640 1 -- 172.21.15.133:0/252285235 >> v2:172.21.15.162:3300/0 conn(0x7f074c15c9a0 msgr2=0x7f074c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f074a59c640 1 --2- 172.21.15.133:0/252285235 >> v2:172.21.15.162:3300/0 conn(0x7f074c15c9a0 0x7f074c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0749d9b640 1 --2- 172.21.15.133:0/252285235 >> v2:172.21.15.162:3300/0 conn(0x7f074c15c9a0 0x7f074c0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f074959a640 1 -- 172.21.15.133:0/252285235 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508458616 0 0) 0x7f074c0c61d0 con 0x7f074c0ce330 2024-04-25T10:06:25.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0749d9b640 1 -- 172.21.15.133:0/252285235 >> v1:172.21.15.133:6789/0 conn(0x7f074c0ce330 legacy=0x7f074c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.258+0000 7f0749d9b640 1 -- 172.21.15.133:0/252285235 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c06d7d0 con 0x7f074c0b1f70 2024-04-25T10:06:25.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f0749d9b640 1 --2- 172.21.15.133:0/252285235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0b1f70 0x7f074c06dcf0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f074c06f5f0 tx=0x7f074001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8d7b82a51d432 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f074959a640 1 -- 172.21.15.133:0/252285235 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07400ab640 con 0x7f074c0b1f70 2024-04-25T10:06:25.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f074959a640 1 -- 172.21.15.133:0/252285235 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07400b0070 con 0x7f074c0b1f70 2024-04-25T10:06:25.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f074959a640 1 -- 172.21.15.133:0/252285235 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07400aba00 con 0x7f074c0b1f70 2024-04-25T10:06:25.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f0750e12640 1 -- 172.21.15.133:0/252285235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0b1f70 msgr2=0x7f074c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f0750e12640 1 --2- 172.21.15.133:0/252285235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0b1f70 0x7f074c06dcf0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f074c06f5f0 tx=0x7f074001c420 comp rx=0 tx=0).stop 2024-04-25T10:06:25.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f0750e12640 1 -- 172.21.15.133:0/252285235 shutdown_connections 2024-04-25T10:06:25.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f0750e12640 1 --2- 172.21.15.133:0/252285235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0b1f70 0x7f074c06dcf0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f0750e12640 1 --2- 172.21.15.133:0/252285235 >> v2:172.21.15.162:3300/0 conn(0x7f074c15c9a0 0x7f074c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f0750e12640 1 -- 172.21.15.133:0/252285235 >> 172.21.15.133:0/252285235 conn(0x7f074c067000 msgr2=0x7f074c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f0750e12640 1 -- 172.21.15.133:0/252285235 shutdown_connections 2024-04-25T10:06:25.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f0750e12640 1 -- 172.21.15.133:0/252285235 wait complete. 2024-04-25T10:06:25.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f0750e12640 1 Processor -- start 2024-04-25T10:06:25.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.262+0000 7f0750e12640 1 -- start start 2024-04-25T10:06:25.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f0750e12640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0b1f70 0x7f074c0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f0750e12640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f074c0ce330 0x7f074c156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f0750e12640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f074c15c9a0 0x7f074c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f0750e12640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0ceb50 con 0x7f074c0b1f70 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f0750e12640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c15cd80 con 0x7f074c15c9a0 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f0750e12640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0cdfe0 con 0x7f074c0ce330 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f074a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0b1f70 0x7f074c0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f074a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0b1f70 0x7f074c0b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41846/0 (socket says 172.21.15.133:41846) 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f074a59c640 1 -- 172.21.15.133:0/3744336905 learned_addr learned my addr 172.21.15.133:0/3744336905 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f074a59c640 1 -- 172.21.15.133:0/3744336905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f074c0ce330 msgr2=0x7f074c156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f074a59c640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f074c0ce330 0x7f074c156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f074a59c640 1 -- 172.21.15.133:0/3744336905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f074c15c9a0 msgr2=0x7f074c1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f074a59c640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f074c15c9a0 0x7f074c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f074a59c640 1 -- 172.21.15.133:0/3744336905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c0b2370 con 0x7f074c0b1f70 2024-04-25T10:06:25.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f074a59c640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0b1f70 0x7f074c0b3f80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f073c00c7f0 tx=0x7f073c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f073affd640 1 -- 172.21.15.133:0/3744336905 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c015070 con 0x7f074c0b1f70 2024-04-25T10:06:25.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f073affd640 1 -- 172.21.15.133:0/3744336905 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073c004240 con 0x7f074c0b1f70 2024-04-25T10:06:25.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f0750e12640 1 -- 172.21.15.133:0/3744336905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074c154d40 con 0x7f074c0b1f70 2024-04-25T10:06:25.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f073affd640 1 -- 172.21.15.133:0/3744336905 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c003620 con 0x7f074c0b1f70 2024-04-25T10:06:25.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f0750e12640 1 -- 172.21.15.133:0/3744336905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c155300 con 0x7f074c0b1f70 2024-04-25T10:06:25.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f0750e12640 1 -- 172.21.15.133:0/3744336905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074c06bc20 con 0x7f074c0b1f70 2024-04-25T10:06:25.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f073affd640 1 -- 172.21.15.133:0/3744336905 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f073c0039d0 con 0x7f074c0b1f70 2024-04-25T10:06:25.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f073affd640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f072c066f90 0x7f072c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f073affd640 1 -- 172.21.15.133:0/3744336905 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f073c085170 con 0x7f074c0b1f70 2024-04-25T10:06:25.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f0749d9b640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f072c066f90 0x7f072c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.266+0000 7f0749d9b640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f072c066f90 0x7f072c069450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f07400af000 tx=0x7f0740004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.270+0000 7f073affd640 1 -- 172.21.15.133:0/3744336905 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f073c056080 con 0x7f074c0b1f70 2024-04-25T10:06:25.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.450+0000 7f0750e12640 1 -- 172.21.15.133:0/3744336905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f074c06dcf0 con 0x7f074c0b1f70 2024-04-25T10:06:25.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.450+0000 7f073affd640 1 -- 172.21.15.133:0/3744336905 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f073c055980 con 0x7f074c0b1f70 2024-04-25T10:06:25.455 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:25.455 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:36.528839+0000","modified":"2024-04-25T10:06:17.247723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.133:6837/1425600691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1425600691},{"type":"v1","addr":"172.21.15.133:6837","nonce":1425600691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:25.455 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 26 2024-04-25T10:06:25.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 -- 172.21.15.133:0/3744336905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f072c066f90 msgr2=0x7f072c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f072c066f90 0x7f072c069450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f07400af000 tx=0x7f0740004740 comp rx=0 tx=0).stop 2024-04-25T10:06:25.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 -- 172.21.15.133:0/3744336905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0b1f70 msgr2=0x7f074c0b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0b1f70 0x7f074c0b3f80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f073c00c7f0 tx=0x7f073c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 -- 172.21.15.133:0/3744336905 shutdown_connections 2024-04-25T10:06:25.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f074c15c9a0 0x7f074c1536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f074c0ce330 0x7f074c156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f072c066f90 0x7f072c069450 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 --2- 172.21.15.133:0/3744336905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0b1f70 0x7f074c0b3f80 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 -- 172.21.15.133:0/3744336905 >> 172.21.15.133:0/3744336905 conn(0x7f074c067000 msgr2=0x7f074c15b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 -- 172.21.15.133:0/3744336905 shutdown_connections 2024-04-25T10:06:25.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.454+0000 7f0750e12640 1 -- 172.21.15.133:0/3744336905 wait complete. 2024-04-25T10:06:25.473 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:25.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.590+0000 7f9e98ced640 1 Processor -- start 2024-04-25T10:06:25.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e98ced640 1 -- start start 2024-04-25T10:06:25.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e98ced640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9406ac90 0x7f9e9406b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e98ced640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9e940ccee0 0x7f9e940cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e98ced640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9415cdc0 con 0x7f9e94159280 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e98ced640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e9406b630 con 0x7f9e940ccee0 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e98ced640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9406fb80 con 0x7f9e9406ac90 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e9259c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9e94159280 0x7f9e94072ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54376/0 (socket says 172.21.15.133:54376) 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e9259c640 1 -- 172.21.15.133:0/1765454279 learned_addr learned my addr 172.21.15.133:0/1765454279 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e92d9d640 1 --2- 172.21.15.133:0/1765454279 >> v2:172.21.15.162:3300/0 conn(0x7f9e940ccee0 0x7f9e940cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e89d9b640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9406ac90 0x7f9e9406b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429832590 0 0) 0x7f9e9415cdc0 con 0x7f9e94159280 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e80003530 con 0x7f9e94159280 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2288525224 0 0) 0x7f9e80003530 con 0x7f9e94159280 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9406ac90 msgr2=0x7f9e9406b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9406ac90 0x7f9e9406b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 >> v2:172.21.15.162:3300/0 conn(0x7f9e940ccee0 msgr2=0x7f9e940cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 --2- 172.21.15.133:0/1765454279 >> v2:172.21.15.162:3300/0 conn(0x7f9e940ccee0 0x7f9e940cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e940732b0 con 0x7f9e94159280 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f9e84001680 con 0x7f9e94159280 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e89d9b640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9406ac90 0x7f9e9406b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 >> v1:172.21.15.133:6789/0 conn(0x7f9e94159280 legacy=0x7f9e94072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9406ac90 0x7f9e800039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e80003f00 0x7f9e80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e80004d50 0x7f9e800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9406fb80 con 0x7f9e80004d50 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8000b9d0 con 0x7f9e80003f00 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8000bb80 con 0x7f9e9406ac90 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e92d9d640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e80004d50 0x7f9e800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e89d9b640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e80003f00 0x7f9e80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e9259c640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9406ac90 0x7f9e800039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e92d9d640 1 -- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9406ac90 msgr2=0x7f9e800039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e92d9d640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9406ac90 0x7f9e800039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e92d9d640 1 -- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e80003f00 msgr2=0x7f9e80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e92d9d640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e80003f00 0x7f9e80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.594+0000 7f9e92d9d640 1 -- 172.21.15.133:0/1765454279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e80003530 con 0x7f9e80004d50 2024-04-25T10:06:25.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e92d9d640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e80004d50 0x7f9e800051d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009fc0 tx=0x7f9e8c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586f41d27edaeb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0b2070 con 0x7f9e80004d50 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8c0a5cd0 con 0x7f9e80004d50 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e91d9b640 1 -- 172.21.15.133:0/1765454279 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c009c10 con 0x7f9e80004d50 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 -- 172.21.15.133:0/1765454279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e80004d50 msgr2=0x7f9e800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 --2- 172.21.15.133:0/1765454279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e80004d50 0x7f9e800051d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009fc0 tx=0x7f9e8c01ccc0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e92d9d640 1 -- 172.21.15.133:0/1765454279 reap_dead start 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 -- 172.21.15.133:0/1765454279 shutdown_connections 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 -- 172.21.15.133:0/1765454279 >> 172.21.15.133:0/1765454279 conn(0x7f9e940b2370 msgr2=0x7f9e940b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 -- 172.21.15.133:0/1765454279 shutdown_connections 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 -- 172.21.15.133:0/1765454279 wait complete. 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 Processor -- start 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 -- start start 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9414cfb0 0x7f9e940ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9415cfe0 0x7f9e940b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e940b3cb0 0x7f9e9415d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940700b0 con 0x7f9e9414cfb0 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940709e0 con 0x7f9e940b3cb0 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e98ced640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e94070390 con 0x7f9e9415cfe0 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e9259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9414cfb0 0x7f9e940ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e9259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9414cfb0 0x7f9e940ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41862/0 (socket says 172.21.15.133:41862) 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e9259c640 1 -- 172.21.15.133:0/2588615231 learned_addr learned my addr 172.21.15.133:0/2588615231 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e89d9b640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9415cfe0 0x7f9e940b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e92d9d640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e940b3cb0 0x7f9e9415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e9259c640 1 -- 172.21.15.133:0/2588615231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9415cfe0 msgr2=0x7f9e940b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e9259c640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9415cfe0 0x7f9e940b3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e9259c640 1 -- 172.21.15.133:0/2588615231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e940b3cb0 msgr2=0x7f9e9415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e9259c640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e940b3cb0 0x7f9e9415d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.598+0000 7f9e9259c640 1 -- 172.21.15.133:0/2588615231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e940ad9c0 con 0x7f9e9414cfb0 2024-04-25T10:06:25.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e9259c640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9414cfb0 0x7f9e940ad350 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400eaf0 tx=0x7f9e84005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e8b7fe640 1 -- 172.21.15.133:0/2588615231 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400c990 con 0x7f9e9414cfb0 2024-04-25T10:06:25.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e8b7fe640 1 -- 172.21.15.133:0/2588615231 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e84018070 con 0x7f9e9414cfb0 2024-04-25T10:06:25.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e98ced640 1 -- 172.21.15.133:0/2588615231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e940adca0 con 0x7f9e9414cfb0 2024-04-25T10:06:25.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e8b7fe640 1 -- 172.21.15.133:0/2588615231 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400cd70 con 0x7f9e9414cfb0 2024-04-25T10:06:25.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e98ced640 1 -- 172.21.15.133:0/2588615231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e940ae260 con 0x7f9e9414cfb0 2024-04-25T10:06:25.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e98ced640 1 -- 172.21.15.133:0/2588615231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e94067c70 con 0x7f9e9414cfb0 2024-04-25T10:06:25.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e8b7fe640 1 -- 172.21.15.133:0/2588615231 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e8401d460 con 0x7f9e9414cfb0 2024-04-25T10:06:25.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e8b7fe640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e74066f60 0x7f9e74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e8b7fe640 1 -- 172.21.15.133:0/2588615231 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9e84085250 con 0x7f9e9414cfb0 2024-04-25T10:06:25.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e89d9b640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e74066f60 0x7f9e74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.602+0000 7f9e89d9b640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e74066f60 0x7f9e74069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9e940b1b90 tx=0x7f9e7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.606+0000 7f9e8b7fe640 1 -- 172.21.15.133:0/2588615231 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e84056160 con 0x7f9e9414cfb0 2024-04-25T10:06:25.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.754+0000 7f9e98ced640 1 -- 172.21.15.133:0/2588615231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e940b5a80 con 0x7f9e9414cfb0 2024-04-25T10:06:25.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.754+0000 7f9e8b7fe640 1 -- 172.21.15.133:0/2588615231 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16966 (secure 0 0 0) 0x7f9e84055a60 con 0x7f9e9414cfb0 2024-04-25T10:06:25.760 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:25.760 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":54,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:05:38.186046+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:34.805104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:35.512057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:25.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 -- 172.21.15.133:0/2588615231 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e74066f60 msgr2=0x7f9e74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e74066f60 0x7f9e74069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9e940b1b90 tx=0x7f9e7c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:25.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 -- 172.21.15.133:0/2588615231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9414cfb0 msgr2=0x7f9e940ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9414cfb0 0x7f9e940ad350 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400eaf0 tx=0x7f9e84005e40 comp rx=0 tx=0).stop 2024-04-25T10:06:25.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 -- 172.21.15.133:0/2588615231 shutdown_connections 2024-04-25T10:06:25.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e940b3cb0 0x7f9e9415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9e9415cfe0 0x7f9e940b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e74066f60 0x7f9e74069420 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 --2- 172.21.15.133:0/2588615231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9414cfb0 0x7f9e940ad350 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 -- 172.21.15.133:0/2588615231 >> 172.21.15.133:0/2588615231 conn(0x7f9e940b2370 msgr2=0x7f9e940d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 -- 172.21.15.133:0/2588615231 shutdown_connections 2024-04-25T10:06:25.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.758+0000 7f9e98ced640 1 -- 172.21.15.133:0/2588615231 wait complete. 2024-04-25T10:06:25.778 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:06:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f239a948640 1 Processor -- start 2024-04-25T10:06:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f239a948640 1 -- start start 2024-04-25T10:06:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f239a948640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2394145170 0x7f239406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f239a948640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f239406e4d0 0x7f23940cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f239a948640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23940ce2e0 con 0x7f23940cdd60 2024-04-25T10:06:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f239a948640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239406db00 con 0x7f2394145170 2024-04-25T10:06:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f239a948640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2394145550 con 0x7f239406e4d0 2024-04-25T10:06:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2398ee5640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f23940cdd60 0x7f239415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54386/0 (socket says 172.21.15.133:54386) 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2398ee5640 1 -- 172.21.15.133:0/3257994362 learned_addr learned my addr 172.21.15.133:0/3257994362 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2393fff640 1 --2- 172.21.15.133:0/3257994362 >> v2:172.21.15.162:3300/0 conn(0x7f2394145170 0x7f239406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f23937fe640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f239406e4d0 0x7f23940cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010748377 0 0) 0x7f23940ce2e0 con 0x7f23940cdd60 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f237c003530 con 0x7f23940cdd60 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 452218042 0 0) 0x7f237c003530 con 0x7f23940cdd60 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f239406e4d0 msgr2=0x7f23940cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f239406e4d0 0x7f23940cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 >> v2:172.21.15.162:3300/0 conn(0x7f2394145170 msgr2=0x7f239406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 --2- 172.21.15.133:0/3257994362 >> v2:172.21.15.162:3300/0 conn(0x7f2394145170 0x7f239406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239406d4b0 con 0x7f23940cdd60 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f238c001a00 con 0x7f23940cdd60 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 >> v1:172.21.15.133:6789/0 conn(0x7f23940cdd60 legacy=0x7f239415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 0x7f237c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f239406e4d0 0x7f237c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f237c004d50 0x7f237c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239406db00 con 0x7f237c0039c0 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2394145550 con 0x7f237c004d50 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237c00b9d0 con 0x7f239406e4d0 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2393fff640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 0x7f237c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2398ee5640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f237c004d50 0x7f237c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f23937fe640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f239406e4d0 0x7f237c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2393fff640 1 -- 172.21.15.133:0/3257994362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f239406e4d0 msgr2=0x7f237c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2393fff640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f239406e4d0 0x7f237c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2393fff640 1 -- 172.21.15.133:0/3257994362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f237c004d50 msgr2=0x7f237c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2393fff640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f237c004d50 0x7f237c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.882+0000 7f2393fff640 1 -- 172.21.15.133:0/3257994362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c003530 con 0x7f237c0039c0 2024-04-25T10:06:25.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f2393fff640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 0x7f237c003de0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2384009ae0 tx=0x7f238401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7819378c9eb8b78c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840096b0 con 0x7f237c0039c0 2024-04-25T10:06:25.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23840a6490 con 0x7f237c0039c0 2024-04-25T10:06:25.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f2392ffd640 1 -- 172.21.15.133:0/3257994362 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840a6880 con 0x7f237c0039c0 2024-04-25T10:06:25.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f239a948640 1 -- 172.21.15.133:0/3257994362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 msgr2=0x7f237c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f239a948640 1 --2- 172.21.15.133:0/3257994362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 0x7f237c003de0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2384009ae0 tx=0x7f238401cab0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f2398ee5640 1 -- 172.21.15.133:0/3257994362 reap_dead start 2024-04-25T10:06:25.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f239a948640 1 -- 172.21.15.133:0/3257994362 shutdown_connections 2024-04-25T10:06:25.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f239a948640 1 -- 172.21.15.133:0/3257994362 >> 172.21.15.133:0/3257994362 conn(0x7f2394066f70 msgr2=0x7f239406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f239a948640 1 -- 172.21.15.133:0/3257994362 shutdown_connections 2024-04-25T10:06:25.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f239a948640 1 -- 172.21.15.133:0/3257994362 wait complete. 2024-04-25T10:06:25.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f239a948640 1 Processor -- start 2024-04-25T10:06:25.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.886+0000 7f239a948640 1 -- start start 2024-04-25T10:06:25.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f239a948640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 0x7f23941367f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f239a948640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2394161020 0x7f2394137710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f239a948640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2394137c50 0x7f2394127530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f239a948640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2394166480 con 0x7f237c0039c0 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f239a948640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239415d7f0 con 0x7f2394137c50 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f239a948640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2394166600 con 0x7f2394161020 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2393fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 0x7f23941367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2393fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 0x7f23941367f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41878/0 (socket says 172.21.15.133:41878) 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2393fff640 1 -- 172.21.15.133:0/685851443 learned_addr learned my addr 172.21.15.133:0/685851443 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f23937fe640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2394161020 0x7f2394137710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2393fff640 1 -- 172.21.15.133:0/685851443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2394161020 msgr2=0x7f2394137710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2393fff640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2394161020 0x7f2394137710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2398ee5640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2394137c50 0x7f2394127530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2393fff640 1 -- 172.21.15.133:0/685851443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2394137c50 msgr2=0x7f2394127530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2393fff640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2394137c50 0x7f2394127530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2393fff640 1 -- 172.21.15.133:0/685851443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23941f4b00 con 0x7f237c0039c0 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2393fff640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 0x7f23941367f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f238401c8e0 tx=0x7f237c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2390ff9640 1 -- 172.21.15.133:0/685851443 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840096b0 con 0x7f237c0039c0 2024-04-25T10:06:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2390ff9640 1 -- 172.21.15.133:0/685851443 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23840a6cc0 con 0x7f237c0039c0 2024-04-25T10:06:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f239a948640 1 -- 172.21.15.133:0/685851443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23941361f0 con 0x7f237c0039c0 2024-04-25T10:06:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2390ff9640 1 -- 172.21.15.133:0/685851443 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840a76b0 con 0x7f237c0039c0 2024-04-25T10:06:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f239a948640 1 -- 172.21.15.133:0/685851443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2394128d20 con 0x7f237c0039c0 2024-04-25T10:06:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f239a948640 1 -- 172.21.15.133:0/685851443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2394068e80 con 0x7f237c0039c0 2024-04-25T10:06:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2390ff9640 1 -- 172.21.15.133:0/685851443 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23840c2060 con 0x7f237c0039c0 2024-04-25T10:06:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2390ff9640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2380066f90 0x7f2380069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f2390ff9640 1 -- 172.21.15.133:0/685851443 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f238411efb0 con 0x7f237c0039c0 2024-04-25T10:06:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f23937fe640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2380066f90 0x7f2380069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.890+0000 7f23937fe640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2380066f90 0x7f2380069450 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f239413a3c0 tx=0x7f2388008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:25.894+0000 7f2390ff9640 1 -- 172.21.15.133:0/685851443 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23840b6070 con 0x7f237c0039c0 2024-04-25T10:06:26.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.078+0000 7f239a948640 1 -- 172.21.15.133:0/685851443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2394138030 con 0x7f237c0039c0 2024-04-25T10:06:26.083 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:06:26.078+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:06:26.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.082+0000 7f2390ff9640 1 -- 172.21.15.133:0/685851443 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7f23840ac020 con 0x7f237c0039c0 2024-04-25T10:06:26.090 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:06:26.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 -- 172.21.15.133:0/685851443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2380066f90 msgr2=0x7f2380069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2380066f90 0x7f2380069450 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f239413a3c0 tx=0x7f2388008040 comp rx=0 tx=0).stop 2024-04-25T10:06:26.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 -- 172.21.15.133:0/685851443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 msgr2=0x7f23941367f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 0x7f23941367f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f238401c8e0 tx=0x7f237c005840 comp rx=0 tx=0).stop 2024-04-25T10:06:26.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 -- 172.21.15.133:0/685851443 shutdown_connections 2024-04-25T10:06:26.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2394137c50 0x7f2394127530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2394161020 0x7f2394137710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2380066f90 0x7f2380069450 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 --2- 172.21.15.133:0/685851443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f237c0039c0 0x7f23941367f0 secure :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f238401c8e0 tx=0x7f237c005840 comp rx=0 tx=0).stop 2024-04-25T10:06:26.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 -- 172.21.15.133:0/685851443 >> 172.21.15.133:0/685851443 conn(0x7f2394066f70 msgr2=0x7f239413fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 -- 172.21.15.133:0/685851443 shutdown_connections 2024-04-25T10:06:26.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.086+0000 7f239a948640 1 -- 172.21.15.133:0/685851443 wait complete. 2024-04-25T10:06:26.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:06:26.135 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:06:26.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.218+0000 7fd855ff6640 1 Processor -- start 2024-04-25T10:06:26.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.218+0000 7fd855ff6640 1 -- start start 2024-04-25T10:06:26.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.218+0000 7fd855ff6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8500cd820 0x7fd8500cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.218+0000 7fd855ff6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd85015ced0 0x7fd85015d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.218+0000 7fd855ff6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8500ce2e0 con 0x7fd850145170 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.218+0000 7fd855ff6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85006db00 con 0x7fd85015ced0 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.218+0000 7fd855ff6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd850145550 con 0x7fd8500cd820 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd84ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd85015ced0 0x7fd85015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd84ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd85015ced0 0x7fd85015d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45576/0 (socket says 172.21.15.133:45576) 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd84ffff640 1 -- 172.21.15.133:0/1053503297 learned_addr learned my addr 172.21.15.133:0/1053503297 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd847fff640 1 --2- 172.21.15.133:0/1053503297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8500cd820 0x7fd8500cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd84ffff640 1 -- 172.21.15.133:0/1053503297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8500cd820 msgr2=0x7fd8500cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd84ffff640 1 --2- 172.21.15.133:0/1053503297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8500cd820 0x7fd8500cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd84ffff640 1 -- 172.21.15.133:0/1053503297 >> v1:172.21.15.162:6789/0 conn(0x7fd850145170 legacy=0x7fd85006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd84ffff640 1 -- 172.21.15.133:0/1053503297 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85006d4b0 con 0x7fd85015ced0 2024-04-25T10:06:26.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd84ffff640 1 --2- 172.21.15.133:0/1053503297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd85015ced0 0x7fd85015d360 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd848009e50 tx=0x7fd84801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6e1d72cee9c181 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd84effd640 1 -- 172.21.15.133:0/1053503297 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480a5700 con 0x7fd85015ced0 2024-04-25T10:06:26.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd84effd640 1 -- 172.21.15.133:0/1053503297 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8480a58a0 con 0x7fd85015ced0 2024-04-25T10:06:26.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd84effd640 1 -- 172.21.15.133:0/1053503297 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480a5c70 con 0x7fd85015ced0 2024-04-25T10:06:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd855ff6640 1 -- 172.21.15.133:0/1053503297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd85015ced0 msgr2=0x7fd85015d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd855ff6640 1 --2- 172.21.15.133:0/1053503297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd85015ced0 0x7fd85015d360 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd848009e50 tx=0x7fd84801ca30 comp rx=0 tx=0).stop 2024-04-25T10:06:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd855ff6640 1 -- 172.21.15.133:0/1053503297 shutdown_connections 2024-04-25T10:06:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd855ff6640 1 --2- 172.21.15.133:0/1053503297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd85015ced0 0x7fd85015d360 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd855ff6640 1 --2- 172.21.15.133:0/1053503297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8500cd820 0x7fd8500cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd855ff6640 1 -- 172.21.15.133:0/1053503297 >> 172.21.15.133:0/1053503297 conn(0x7fd850066f70 msgr2=0x7fd85006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd855ff6640 1 -- 172.21.15.133:0/1053503297 shutdown_connections 2024-04-25T10:06:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.222+0000 7fd855ff6640 1 -- 172.21.15.133:0/1053503297 wait complete. 2024-04-25T10:06:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd855ff6640 1 Processor -- start 2024-04-25T10:06:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd855ff6640 1 -- start start 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd855ff6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8500cd820 0x7fd85013b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd855ff6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd850145170 0x7fd85013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd855ff6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd85015ced0 0x7fd85013dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd855ff6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd850168610 con 0x7fd8500cd820 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd855ff6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd850168490 con 0x7fd85015ced0 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd855ff6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd850168790 con 0x7fd850145170 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8500cd820 0x7fd85013b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8500cd820 0x7fd85013b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45586/0 (socket says 172.21.15.133:45586) 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84f7fe640 1 -- 172.21.15.133:0/1255720185 learned_addr learned my addr 172.21.15.133:0/1255720185 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84ffff640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd85015ced0 0x7fd85013dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd847fff640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd850145170 0x7fd85013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84f7fe640 1 -- 172.21.15.133:0/1255720185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd850145170 msgr2=0x7fd85013cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84f7fe640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd850145170 0x7fd85013cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84f7fe640 1 -- 172.21.15.133:0/1255720185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd85015ced0 msgr2=0x7fd85013dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84f7fe640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd85015ced0 0x7fd85013dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84f7fe640 1 -- 172.21.15.133:0/1255720185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd850160d80 con 0x7fd8500cd820 2024-04-25T10:06:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd847fff640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd850145170 0x7fd85013cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84f7fe640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8500cd820 0x7fd85013b520 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd84000c420 tx=0x7fd84000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84cff9640 1 -- 172.21.15.133:0/1255720185 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8400052d0 con 0x7fd8500cd820 2024-04-25T10:06:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84cff9640 1 -- 172.21.15.133:0/1255720185 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd840005470 con 0x7fd8500cd820 2024-04-25T10:06:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd855ff6640 1 -- 172.21.15.133:0/1255720185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd850161060 con 0x7fd8500cd820 2024-04-25T10:06:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd84cff9640 1 -- 172.21.15.133:0/1255720185 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840007260 con 0x7fd8500cd820 2024-04-25T10:06:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd855ff6640 1 -- 172.21.15.133:0/1255720185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd850161620 con 0x7fd8500cd820 2024-04-25T10:06:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.226+0000 7fd855ff6640 1 -- 172.21.15.133:0/1255720185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd850068e80 con 0x7fd8500cd820 2024-04-25T10:06:26.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.230+0000 7fd84cff9640 1 -- 172.21.15.133:0/1255720185 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd840007400 con 0x7fd8500cd820 2024-04-25T10:06:26.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.230+0000 7fd84cff9640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd830066f60 0x7fd830069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.230+0000 7fd84cff9640 1 -- 172.21.15.133:0/1255720185 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd8400847b0 con 0x7fd8500cd820 2024-04-25T10:06:26.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.230+0000 7fd847fff640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd830066f60 0x7fd830069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.230+0000 7fd847fff640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd830066f60 0x7fd830069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd850165cd0 tx=0x7fd838008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.234+0000 7fd84cff9640 1 -- 172.21.15.133:0/1255720185 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd840055560 con 0x7fd8500cd820 2024-04-25T10:06:26.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:26.426+0000 7fd855ff6640 1 -- 172.21.15.133:0/1255720185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd850073a60 con 0x7fd8500cd820 2024-04-25T10:06:27.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.086+0000 7fd84cff9640 1 -- 172.21.15.133:0/1255720185 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd840054e60 con 0x7fd8500cd820 2024-04-25T10:06:27.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 -- 172.21.15.133:0/1255720185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd830066f60 msgr2=0x7fd830069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd830066f60 0x7fd830069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd850165cd0 tx=0x7fd838008040 comp rx=0 tx=0).stop 2024-04-25T10:06:27.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 -- 172.21.15.133:0/1255720185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8500cd820 msgr2=0x7fd85013b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8500cd820 0x7fd85013b520 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd84000c420 tx=0x7fd84000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 -- 172.21.15.133:0/1255720185 shutdown_connections 2024-04-25T10:06:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd85015ced0 0x7fd85013dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd850145170 0x7fd85013cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd830066f60 0x7fd830069420 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 --2- 172.21.15.133:0/1255720185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8500cd820 0x7fd85013b520 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 -- 172.21.15.133:0/1255720185 >> 172.21.15.133:0/1255720185 conn(0x7fd850066f70 msgr2=0x7fd85015d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 -- 172.21.15.133:0/1255720185 shutdown_connections 2024-04-25T10:06:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.090+0000 7fd855ff6640 1 -- 172.21.15.133:0/1255720185 wait complete. 2024-04-25T10:06:27.115 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:06:27.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf9ac1640 1 Processor -- start 2024-04-25T10:06:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf9ac1640 1 -- start start 2024-04-25T10:06:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf9ac1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf40b4940 0x7f2cf40b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf9ac1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2cf415c420 0x7f2cf40cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf9ac1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf40b5260 con 0x7f2cf4071cb0 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf9ac1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf4072a90 con 0x7f2cf415c420 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf9ac1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40a85d0 con 0x7f2cf40b4940 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf3fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2cf415c420 0x7f2cf40cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf37fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf40b4940 0x7f2cf40b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf2ffd640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f2cf4071cb0 0x7f2cf415bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:54598/0 (socket says 172.21.15.133:54598) 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf2ffd640 1 -- 172.21.15.133:0/1562662253 learned_addr learned my addr 172.21.15.133:0/1562662253 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf37fe640 1 -- 172.21.15.133:0/1562662253 >> v2:172.21.15.162:3300/0 conn(0x7f2cf415c420 msgr2=0x7f2cf40cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf37fe640 1 --2- 172.21.15.133:0/1562662253 >> v2:172.21.15.162:3300/0 conn(0x7f2cf415c420 0x7f2cf40cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf37fe640 1 -- 172.21.15.133:0/1562662253 >> v1:172.21.15.162:6789/0 conn(0x7f2cf4071cb0 legacy=0x7f2cf415bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.282+0000 7f2cf37fe640 1 -- 172.21.15.133:0/1562662253 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf415c9b0 con 0x7f2cf40b4940 2024-04-25T10:06:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.286+0000 7f2cf37fe640 1 --2- 172.21.15.133:0/1562662253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf40b4940 0x7f2cf40b4d20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009d90 tx=0x7f2cec01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12e02a8caa928720 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.286+0000 7f2cf27fc640 1 -- 172.21.15.133:0/1562662253 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec0b2070 con 0x7f2cf40b4940 2024-04-25T10:06:27.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.286+0000 7f2cf27fc640 1 -- 172.21.15.133:0/1562662253 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cec0a5a00 con 0x7f2cf40b4940 2024-04-25T10:06:27.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.286+0000 7f2cf27fc640 1 -- 172.21.15.133:0/1562662253 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec0a5dd0 con 0x7f2cf40b4940 2024-04-25T10:06:27.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/1562662253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf40b4940 msgr2=0x7f2cf40b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 --2- 172.21.15.133:0/1562662253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf40b4940 0x7f2cf40b4d20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009d90 tx=0x7f2cec01c940 comp rx=0 tx=0).stop 2024-04-25T10:06:27.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/1562662253 shutdown_connections 2024-04-25T10:06:27.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 --2- 172.21.15.133:0/1562662253 >> v2:172.21.15.162:3300/0 conn(0x7f2cf415c420 0x7f2cf40cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 --2- 172.21.15.133:0/1562662253 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf40b4940 0x7f2cf40b4d20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/1562662253 >> 172.21.15.133:0/1562662253 conn(0x7f2cf406d370 msgr2=0x7f2cf406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:27.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/1562662253 shutdown_connections 2024-04-25T10:06:27.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/1562662253 wait complete. 2024-04-25T10:06:27.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 Processor -- start 2024-04-25T10:06:27.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 -- start start 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf4071cb0 0x7f2cf413c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf40b4940 0x7f2cf4135e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cf415c420 0x7f2cf4136380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40cc960 con 0x7f2cf40b4940 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf4071790 con 0x7f2cf415c420 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40ccae0 con 0x7f2cf4071cb0 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf40b4940 0x7f2cf4135e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf40b4940 0x7f2cf4135e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45602/0 (socket says 172.21.15.133:45602) 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf2ffd640 1 -- 172.21.15.133:0/454007834 learned_addr learned my addr 172.21.15.133:0/454007834 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf37fe640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf4071cb0 0x7f2cf413c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf3fff640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cf415c420 0x7f2cf4136380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf2ffd640 1 -- 172.21.15.133:0/454007834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf4071cb0 msgr2=0x7f2cf413c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf2ffd640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf4071cb0 0x7f2cf413c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf2ffd640 1 -- 172.21.15.133:0/454007834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cf415c420 msgr2=0x7f2cf4136380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf2ffd640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cf415c420 0x7f2cf4136380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf2ffd640 1 -- 172.21.15.133:0/454007834 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf4136ac0 con 0x7f2cf40b4940 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf2ffd640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf40b4940 0x7f2cf4135e40 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800c420 tx=0x7f2ce800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cdbfff640 1 -- 172.21.15.133:0/454007834 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8017070 con 0x7f2cf40b4940 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cdbfff640 1 -- 172.21.15.133:0/454007834 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce8005cf0 con 0x7f2cf40b4940 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/454007834 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf4136da0 con 0x7f2cf40b4940 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cdbfff640 1 -- 172.21.15.133:0/454007834 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80072f0 con 0x7f2cf40b4940 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.290+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/454007834 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf4137360 con 0x7f2cf40b4940 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.294+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/454007834 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf4069000 con 0x7f2cf40b4940 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.294+0000 7f2cdbfff640 1 -- 172.21.15.133:0/454007834 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ce801a070 con 0x7f2cf40b4940 2024-04-25T10:06:27.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.294+0000 7f2cdbfff640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2ce0066f60 0x7f2ce0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.294+0000 7f2cf37fe640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2ce0066f60 0x7f2ce0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.294+0000 7f2cdbfff640 1 -- 172.21.15.133:0/454007834 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2ce8084550 con 0x7f2cf40b4940 2024-04-25T10:06:27.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.294+0000 7f2cf37fe640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2ce0066f60 0x7f2ce0069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2cec0af000 tx=0x7f2cec0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.298+0000 7f2cdbfff640 1 -- 172.21.15.133:0/454007834 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ce8055410 con 0x7f2cf40b4940 2024-04-25T10:06:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:27.446+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/454007834 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2cf40af220 con 0x7f2cf40b4940 2024-04-25T10:06:28.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.090+0000 7f2cdbfff640 1 -- 172.21.15.133:0/454007834 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2ce8054d10 con 0x7f2cf40b4940 2024-04-25T10:06:28.094 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:06:28.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/454007834 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2ce0066f60 msgr2=0x7f2ce0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2ce0066f60 0x7f2ce0069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2cec0af000 tx=0x7f2cec0a7070 comp rx=0 tx=0).stop 2024-04-25T10:06:28.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/454007834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf40b4940 msgr2=0x7f2cf4135e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf40b4940 0x7f2cf4135e40 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800c420 tx=0x7f2ce800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/454007834 shutdown_connections 2024-04-25T10:06:28.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cf415c420 0x7f2cf4136380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2ce0066f60 0x7f2ce0069420 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf40b4940 0x7f2cf4135e40 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 --2- 172.21.15.133:0/454007834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf4071cb0 0x7f2cf413c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/454007834 >> 172.21.15.133:0/454007834 conn(0x7f2cf406d370 msgr2=0x7f2cf406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/454007834 shutdown_connections 2024-04-25T10:06:28.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.094+0000 7f2cf9ac1640 1 -- 172.21.15.133:0/454007834 wait complete. 2024-04-25T10:06:28.123 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:06:28.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9dd59d640 1 Processor -- start 2024-04-25T10:06:28.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9dd59d640 1 -- start start 2024-04-25T10:06:28.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9dd59d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d8071cb0 0x7fa9d815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9dd59d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa9d815c250 0x7fa9d80cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9dd59d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d815c9b0 con 0x7fa9d80b4940 2024-04-25T10:06:28.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9dd59d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d8072a90 con 0x7fa9d815c250 2024-04-25T10:06:28.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9dd59d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80a85d0 con 0x7fa9d8071cb0 2024-04-25T10:06:28.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9d77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa9d815c250 0x7fa9d80cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9d77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa9d815c250 0x7fa9d80cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45610/0 (socket says 172.21.15.133:45610) 2024-04-25T10:06:28.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9d77fe640 1 -- 172.21.15.133:0/2199536919 learned_addr learned my addr 172.21.15.133:0/2199536919 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:28.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9d67fc640 1 --2- 172.21.15.133:0/2199536919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d8071cb0 0x7fa9d815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9d77fe640 1 -- 172.21.15.133:0/2199536919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d8071cb0 msgr2=0x7fa9d815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9d77fe640 1 --2- 172.21.15.133:0/2199536919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d8071cb0 0x7fa9d815bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9d77fe640 1 -- 172.21.15.133:0/2199536919 >> v1:172.21.15.162:6789/0 conn(0x7fa9d80b4940 legacy=0x7fa9d80b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.242+0000 7fa9d77fe640 1 -- 172.21.15.133:0/2199536919 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d815cc90 con 0x7fa9d815c250 2024-04-25T10:06:28.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9d77fe640 1 --2- 172.21.15.133:0/2199536919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d815c250 0x7fa9d80cc010 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009e50 tx=0x7fa9d001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72576efa4b120171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9d5ffb640 1 -- 172.21.15.133:0/2199536919 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00a5840 con 0x7fa9d815c250 2024-04-25T10:06:28.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9d5ffb640 1 -- 172.21.15.133:0/2199536919 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d00a59e0 con 0x7fa9d815c250 2024-04-25T10:06:28.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9d5ffb640 1 -- 172.21.15.133:0/2199536919 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00a5db0 con 0x7fa9d815c250 2024-04-25T10:06:28.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9dd59d640 1 -- 172.21.15.133:0/2199536919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d815c250 msgr2=0x7fa9d80cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9dd59d640 1 --2- 172.21.15.133:0/2199536919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d815c250 0x7fa9d80cc010 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009e50 tx=0x7fa9d001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9dd59d640 1 -- 172.21.15.133:0/2199536919 shutdown_connections 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9dd59d640 1 --2- 172.21.15.133:0/2199536919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d815c250 0x7fa9d80cc010 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9dd59d640 1 --2- 172.21.15.133:0/2199536919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d8071cb0 0x7fa9d815bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9dd59d640 1 -- 172.21.15.133:0/2199536919 >> 172.21.15.133:0/2199536919 conn(0x7fa9d806d370 msgr2=0x7fa9d806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9dd59d640 1 -- 172.21.15.133:0/2199536919 shutdown_connections 2024-04-25T10:06:28.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.246+0000 7fa9dd59d640 1 -- 172.21.15.133:0/2199536919 wait complete. 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9dd59d640 1 Processor -- start 2024-04-25T10:06:28.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9dd59d640 1 -- start start 2024-04-25T10:06:28.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9dd59d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d8071cb0 0x7fa9d80ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9dd59d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d80b4940 0x7fa9d81f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9dd59d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d815c250 0x7fa9d81f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9dd59d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80cc960 con 0x7fa9d815c250 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9dd59d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d815c6a0 con 0x7fa9d8071cb0 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9dd59d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80ccae0 con 0x7fa9d80b4940 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d815c250 0x7fa9d81f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d815c250 0x7fa9d81f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45614/0 (socket says 172.21.15.133:45614) 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9d77fe640 1 -- 172.21.15.133:0/1776471000 learned_addr learned my addr 172.21.15.133:0/1776471000 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9d6ffd640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d8071cb0 0x7fa9d80ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9d67fc640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d80b4940 0x7fa9d81f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9d77fe640 1 -- 172.21.15.133:0/1776471000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d80b4940 msgr2=0x7fa9d81f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9d77fe640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d80b4940 0x7fa9d81f6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9d77fe640 1 -- 172.21.15.133:0/1776471000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d8071cb0 msgr2=0x7fa9d80ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9d77fe640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d8071cb0 0x7fa9d80ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9d77fe640 1 -- 172.21.15.133:0/1776471000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d813fe30 con 0x7fa9d815c250 2024-04-25T10:06:28.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9d77fe640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d815c250 0x7fa9d81f6a60 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00af040 tx=0x7fa9d0019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9bf7fe640 1 -- 172.21.15.133:0/1776471000 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00bc020 con 0x7fa9d815c250 2024-04-25T10:06:28.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9bf7fe640 1 -- 172.21.15.133:0/1776471000 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d00b8880 con 0x7fa9d815c250 2024-04-25T10:06:28.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9bf7fe640 1 -- 172.21.15.133:0/1776471000 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00b8cb0 con 0x7fa9d815c250 2024-04-25T10:06:28.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9dd59d640 1 -- 172.21.15.133:0/1776471000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d80addd0 con 0x7fa9d815c250 2024-04-25T10:06:28.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9dd59d640 1 -- 172.21.15.133:0/1776471000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d80ae390 con 0x7fa9d815c250 2024-04-25T10:06:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9dd59d640 1 -- 172.21.15.133:0/1776471000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d8069000 con 0x7fa9d815c250 2024-04-25T10:06:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.250+0000 7fa9bf7fe640 1 -- 172.21.15.133:0/1776471000 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9d00bf020 con 0x7fa9d815c250 2024-04-25T10:06:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.254+0000 7fa9bf7fe640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 0x7fa9c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.254+0000 7fa9bf7fe640 1 -- 172.21.15.133:0/1776471000 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fa9d0120270 con 0x7fa9d815c250 2024-04-25T10:06:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.254+0000 7fa9d6ffd640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 0x7fa9c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.254+0000 7fa9d6ffd640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 0x7fa9c4069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8004e30 tx=0x7fa9c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.258+0000 7fa9bf7fe640 1 -- 172.21.15.133:0/1776471000 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9d00c6030 con 0x7fa9d815c250 2024-04-25T10:06:28.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:28.414+0000 7fa9dd59d640 1 -- 172.21.15.133:0/1776471000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa9d806c2d0 con 0x7fa9d815c250 2024-04-25T10:06:29.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.102+0000 7fa9bf7fe640 1 -- 172.21.15.133:0/1776471000 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa9d00165e0 con 0x7fa9d815c250 2024-04-25T10:06:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:06:29.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.106+0000 7fa9dd59d640 1 -- 172.21.15.133:0/1776471000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 msgr2=0x7fa9c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.106+0000 7fa9dd59d640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 0x7fa9c4069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8004e30 tx=0x7fa9c8009040 comp rx=0 tx=0).stop 2024-04-25T10:06:29.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.106+0000 7fa9dd59d640 1 -- 172.21.15.133:0/1776471000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d815c250 msgr2=0x7fa9d81f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.106+0000 7fa9dd59d640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d815c250 0x7fa9d81f6a60 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00af040 tx=0x7fa9d0019760 comp rx=0 tx=0).stop 2024-04-25T10:06:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.110+0000 7fa9dd59d640 1 -- 172.21.15.133:0/1776471000 shutdown_connections 2024-04-25T10:06:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.110+0000 7fa9dd59d640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9c4066f60 0x7fa9c4069420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.110+0000 7fa9dd59d640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d815c250 0x7fa9d81f6a60 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.110+0000 7fa9dd59d640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9d80b4940 0x7fa9d81f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.110+0000 7fa9dd59d640 1 --2- 172.21.15.133:0/1776471000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d8071cb0 0x7fa9d80ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.110+0000 7fa9dd59d640 1 -- 172.21.15.133:0/1776471000 >> 172.21.15.133:0/1776471000 conn(0x7fa9d806d370 msgr2=0x7fa9d80b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.114+0000 7fa9dd59d640 1 -- 172.21.15.133:0/1776471000 shutdown_connections 2024-04-25T10:06:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.114+0000 7fa9dd59d640 1 -- 172.21.15.133:0/1776471000 wait complete. 2024-04-25T10:06:29.141 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:06:29.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.274+0000 7f6325f88640 1 Processor -- start 2024-04-25T10:06:29.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.274+0000 7f6325f88640 1 -- start start 2024-04-25T10:06:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.274+0000 7f6325f88640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63200b2150 0x7f632006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.274+0000 7f6325f88640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f632015b9b0 0x7f632006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.274+0000 7f6325f88640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632015ac50 con 0x7f6320143ea0 2024-04-25T10:06:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.274+0000 7f6325f88640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63200ccf20 con 0x7f632015b9b0 2024-04-25T10:06:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.274+0000 7f6325f88640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63200cd180 con 0x7f63200b2150 2024-04-25T10:06:29.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f631ffff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f632015b9b0 0x7f632006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f6317fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63200b2150 0x7f632006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f631f7fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f6320143ea0 0x7f63200cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:54618/0 (socket says 172.21.15.133:54618) 2024-04-25T10:06:29.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f631f7fe640 1 -- 172.21.15.133:0/3260177810 learned_addr learned my addr 172.21.15.133:0/3260177810 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:29.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f631effd640 1 -- 172.21.15.133:0/3260177810 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926407859 0 0) 0x7f6310001120 con 0x7f6320143ea0 2024-04-25T10:06:29.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f631effd640 1 -- 172.21.15.133:0/3260177810 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f630c003530 con 0x7f6320143ea0 2024-04-25T10:06:29.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f6317fff640 1 -- 172.21.15.133:0/3260177810 >> v2:172.21.15.162:3300/0 conn(0x7f632015b9b0 msgr2=0x7f632006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f6317fff640 1 --2- 172.21.15.133:0/3260177810 >> v2:172.21.15.162:3300/0 conn(0x7f632015b9b0 0x7f632006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f6317fff640 1 -- 172.21.15.133:0/3260177810 >> v1:172.21.15.162:6789/0 conn(0x7f6320143ea0 legacy=0x7f63200cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f6317fff640 1 -- 172.21.15.133:0/3260177810 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632015aed0 con 0x7f63200b2150 2024-04-25T10:06:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f6317fff640 1 --2- 172.21.15.133:0/3260177810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63200b2150 0x7f632006e3b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f63080097c0 tx=0x7f630801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e61605c9c75d2b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f631effd640 1 -- 172.21.15.133:0/3260177810 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63080b1070 con 0x7f63200b2150 2024-04-25T10:06:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f631effd640 1 -- 172.21.15.133:0/3260177810 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63080ac640 con 0x7f63200b2150 2024-04-25T10:06:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.278+0000 7f631effd640 1 -- 172.21.15.133:0/3260177810 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63080ac9f0 con 0x7f63200b2150 2024-04-25T10:06:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- 172.21.15.133:0/3260177810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63200b2150 msgr2=0x7f632006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 --2- 172.21.15.133:0/3260177810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63200b2150 0x7f632006e3b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f63080097c0 tx=0x7f630801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- 172.21.15.133:0/3260177810 shutdown_connections 2024-04-25T10:06:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 --2- 172.21.15.133:0/3260177810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63200b2150 0x7f632006e3b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 --2- 172.21.15.133:0/3260177810 >> v2:172.21.15.162:3300/0 conn(0x7f632015b9b0 0x7f632006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- 172.21.15.133:0/3260177810 >> 172.21.15.133:0/3260177810 conn(0x7f6320066f70 msgr2=0x7f632006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- 172.21.15.133:0/3260177810 shutdown_connections 2024-04-25T10:06:29.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- 172.21.15.133:0/3260177810 wait complete. 2024-04-25T10:06:29.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 Processor -- start 2024-04-25T10:06:29.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- start start 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63200b2150 0x7f632013c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6320143ea0 0x7f6320153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f632015b9b0 0x7f6320154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63200b1270 con 0x7f6320143ea0 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63200b10f0 con 0x7f63200b2150 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63200b13f0 con 0x7f632015b9b0 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6317fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6320143ea0 0x7f6320153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6317fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6320143ea0 0x7f6320153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45620/0 (socket says 172.21.15.133:45620) 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6317fff640 1 -- 172.21.15.133:0/582680718 learned_addr learned my addr 172.21.15.133:0/582680718 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f631ffff640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f632015b9b0 0x7f6320154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f631f7fe640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63200b2150 0x7f632013c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6317fff640 1 -- 172.21.15.133:0/582680718 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f632015b9b0 msgr2=0x7f6320154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6317fff640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f632015b9b0 0x7f6320154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6317fff640 1 -- 172.21.15.133:0/582680718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63200b2150 msgr2=0x7f632013c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6317fff640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63200b2150 0x7f632013c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6317fff640 1 -- 172.21.15.133:0/582680718 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632013b2c0 con 0x7f6320143ea0 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f631ffff640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f632015b9b0 0x7f6320154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6317fff640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6320143ea0 0x7f6320153fc0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6308000c10 tx=0x7f6308019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f631cff9640 1 -- 172.21.15.133:0/582680718 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63080bb070 con 0x7f6320143ea0 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f631cff9640 1 -- 172.21.15.133:0/582680718 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63080ac9f0 con 0x7f6320143ea0 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- 172.21.15.133:0/582680718 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632013b5a0 con 0x7f6320143ea0 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f631cff9640 1 -- 172.21.15.133:0/582680718 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63080b1040 con 0x7f6320143ea0 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- 172.21.15.133:0/582680718 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632013bb60 con 0x7f6320143ea0 2024-04-25T10:06:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.282+0000 7f6325f88640 1 -- 172.21.15.133:0/582680718 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6320068e80 con 0x7f6320143ea0 2024-04-25T10:06:29.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.286+0000 7f631cff9640 1 -- 172.21.15.133:0/582680718 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6308003620 con 0x7f6320143ea0 2024-04-25T10:06:29.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.286+0000 7f631cff9640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6300066f60 0x7f6300069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.286+0000 7f631cff9640 1 -- 172.21.15.133:0/582680718 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f630811dfe0 con 0x7f6320143ea0 2024-04-25T10:06:29.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.286+0000 7f631f7fe640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6300066f60 0x7f6300069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.286+0000 7f631f7fe640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6300066f60 0x7f6300069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6310005890 tx=0x7f6310009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.290+0000 7f631cff9640 1 -- 172.21.15.133:0/582680718 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6308016090 con 0x7f6320143ea0 2024-04-25T10:06:29.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.450+0000 7f6325f88640 1 -- 172.21.15.133:0/582680718 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f63200ccbc0 con 0x7f6320143ea0 2024-04-25T10:06:29.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.454+0000 7f631cff9640 1 -- 172.21.15.133:0/582680718 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f63080eea90 con 0x7f6320143ea0 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 -- 172.21.15.133:0/582680718 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6300066f60 msgr2=0x7f6300069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6300066f60 0x7f6300069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6310005890 tx=0x7f6310009040 comp rx=0 tx=0).stop 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 -- 172.21.15.133:0/582680718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6320143ea0 msgr2=0x7f6320153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6320143ea0 0x7f6320153fc0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6308000c10 tx=0x7f6308019240 comp rx=0 tx=0).stop 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 -- 172.21.15.133:0/582680718 shutdown_connections 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f632015b9b0 0x7f6320154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6300066f60 0x7f6300069420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6320143ea0 0x7f6320153fc0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 --2- 172.21.15.133:0/582680718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63200b2150 0x7f632013c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 -- 172.21.15.133:0/582680718 >> 172.21.15.133:0/582680718 conn(0x7f6320066f70 msgr2=0x7f632015bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 -- 172.21.15.133:0/582680718 shutdown_connections 2024-04-25T10:06:29.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.458+0000 7f6325f88640 1 -- 172.21.15.133:0/582680718 wait complete. 2024-04-25T10:06:29.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T10:06:29.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f35ce640 1 Processor -- start 2024-04-25T10:06:29.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f35ce640 1 -- start start 2024-04-25T10:06:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f35ce640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f97ec0ce330 0x7f97ec0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f35ce640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec0c61d0 con 0x7f97ec15c9a0 2024-04-25T10:06:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f35ce640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec0c64b0 con 0x7f97ec0b1f70 2024-04-25T10:06:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f35ce640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ec0cecc0 con 0x7f97ec0ce330 2024-04-25T10:06:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f136a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f97ec15c9a0 0x7f97ec0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50408/0 (socket says 172.21.15.133:50408) 2024-04-25T10:06:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f136a640 1 -- 172.21.15.133:0/761714478 learned_addr learned my addr 172.21.15.133:0/761714478 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f1b6b640 1 --2- 172.21.15.133:0/761714478 >> v2:172.21.15.133:3300/0 conn(0x7f97ec0ce330 0x7f97ec0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97e3fff640 1 -- 172.21.15.133:0/761714478 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127450485 0 0) 0x7f97ec0c61d0 con 0x7f97ec15c9a0 2024-04-25T10:06:29.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97e3fff640 1 -- 172.21.15.133:0/761714478 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97d8003530 con 0x7f97ec15c9a0 2024-04-25T10:06:29.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f1b6b640 1 -- 172.21.15.133:0/761714478 >> v1:172.21.15.162:6789/0 conn(0x7f97ec0b1f70 legacy=0x7f97ec06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f1b6b640 1 -- 172.21.15.133:0/761714478 >> v1:172.21.15.133:6789/0 conn(0x7f97ec15c9a0 legacy=0x7f97ec0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.594+0000 7f97f1b6b640 1 -- 172.21.15.133:0/761714478 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec06d4f0 con 0x7f97ec0ce330 2024-04-25T10:06:29.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f1b6b640 1 --2- 172.21.15.133:0/761714478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec0ce330 0x7f97ec0ce780 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009d90 tx=0x7f97e401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28793c653cf3d102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97e3fff640 1 -- 172.21.15.133:0/761714478 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40b2070 con 0x7f97ec0ce330 2024-04-25T10:06:29.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97e3fff640 1 -- 172.21.15.133:0/761714478 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e40a5900 con 0x7f97ec0ce330 2024-04-25T10:06:29.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97e3fff640 1 -- 172.21.15.133:0/761714478 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40a5cd0 con 0x7f97ec0ce330 2024-04-25T10:06:29.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 -- 172.21.15.133:0/761714478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec0ce330 msgr2=0x7f97ec0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 --2- 172.21.15.133:0/761714478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec0ce330 0x7f97ec0ce780 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009d90 tx=0x7f97e401c930 comp rx=0 tx=0).stop 2024-04-25T10:06:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 -- 172.21.15.133:0/761714478 shutdown_connections 2024-04-25T10:06:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 --2- 172.21.15.133:0/761714478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec0ce330 0x7f97ec0ce780 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 -- 172.21.15.133:0/761714478 >> 172.21.15.133:0/761714478 conn(0x7f97ec067000 msgr2=0x7f97ec06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 -- 172.21.15.133:0/761714478 shutdown_connections 2024-04-25T10:06:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 -- 172.21.15.133:0/761714478 wait complete. 2024-04-25T10:06:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 Processor -- start 2024-04-25T10:06:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 -- start start 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97ec0b1f70 0x7f97ec0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec0ce330 0x7f97ec150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97ec15c9a0 0x7f97ec151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec1695d0 con 0x7f97ec0ce330 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec15dc20 con 0x7f97ec15c9a0 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f35ce640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec169750 con 0x7f97ec0b1f70 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f0b69640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec0ce330 0x7f97ec150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f0b69640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec0ce330 0x7f97ec150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45640/0 (socket says 172.21.15.133:45640) 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f0b69640 1 -- 172.21.15.133:0/161873183 learned_addr learned my addr 172.21.15.133:0/161873183 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f1b6b640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97ec15c9a0 0x7f97ec151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f0b69640 1 -- 172.21.15.133:0/161873183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97ec0b1f70 msgr2=0x7f97ec0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f136a640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97ec0b1f70 0x7f97ec0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f0b69640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97ec0b1f70 0x7f97ec0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f0b69640 1 -- 172.21.15.133:0/161873183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97ec15c9a0 msgr2=0x7f97ec151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f0b69640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97ec15c9a0 0x7f97ec151110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f0b69640 1 -- 172.21.15.133:0/161873183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec153e20 con 0x7f97ec0ce330 2024-04-25T10:06:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.598+0000 7f97f0b69640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec0ce330 0x7f97ec150bd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00b410 tx=0x7f97dc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.602+0000 7f97e1ffb640 1 -- 172.21.15.133:0/161873183 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc015030 con 0x7f97ec0ce330 2024-04-25T10:06:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.602+0000 7f97e1ffb640 1 -- 172.21.15.133:0/161873183 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc004020 con 0x7f97ec0ce330 2024-04-25T10:06:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.602+0000 7f97f35ce640 1 -- 172.21.15.133:0/161873183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec154100 con 0x7f97ec0ce330 2024-04-25T10:06:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.602+0000 7f97e1ffb640 1 -- 172.21.15.133:0/161873183 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc004450 con 0x7f97ec0ce330 2024-04-25T10:06:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.602+0000 7f97f35ce640 1 -- 172.21.15.133:0/161873183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec1546c0 con 0x7f97ec0ce330 2024-04-25T10:06:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.602+0000 7f97f35ce640 1 -- 172.21.15.133:0/161873183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ec1548c0 con 0x7f97ec0ce330 2024-04-25T10:06:29.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.602+0000 7f97e1ffb640 1 -- 172.21.15.133:0/161873183 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97dc018070 con 0x7f97ec0ce330 2024-04-25T10:06:29.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.602+0000 7f97e1ffb640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97d0066f60 0x7f97d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.602+0000 7f97e1ffb640 1 -- 172.21.15.133:0/161873183 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f97dc084a70 con 0x7f97ec0ce330 2024-04-25T10:06:29.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.602+0000 7f97f136a640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97d0066f60 0x7f97d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.602+0000 7f97f136a640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97d0066f60 0x7f97d0069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f97e8007fd0 tx=0x7f97e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.606+0000 7f97e1ffb640 1 -- 172.21.15.133:0/161873183 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97ec1548c0 con 0x7f97ec0ce330 2024-04-25T10:06:29.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:29.822+0000 7f97f35ce640 1 -- 172.21.15.133:0/161873183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f97ec155cd0 con 0x7f97ec0ce330 2024-04-25T10:06:30.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.106+0000 7f97e1ffb640 1 -- 172.21.15.133:0/161873183 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v227) v1 ==== 143+0+0 (secure 0 0 0) 0x7f97dc055bf0 con 0x7f97ec0ce330 2024-04-25T10:06:30.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 -- 172.21.15.133:0/161873183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97d0066f60 msgr2=0x7f97d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97d0066f60 0x7f97d0069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f97e8007fd0 tx=0x7f97e8009040 comp rx=0 tx=0).stop 2024-04-25T10:06:30.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 -- 172.21.15.133:0/161873183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec0ce330 msgr2=0x7f97ec150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec0ce330 0x7f97ec150bd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00b410 tx=0x7f97dc00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 -- 172.21.15.133:0/161873183 shutdown_connections 2024-04-25T10:06:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f97ec15c9a0 0x7f97ec151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97d0066f60 0x7f97d0069420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec0ce330 0x7f97ec150bd0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 --2- 172.21.15.133:0/161873183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f97ec0b1f70 0x7f97ec0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 -- 172.21.15.133:0/161873183 >> 172.21.15.133:0/161873183 conn(0x7f97ec067000 msgr2=0x7f97ec15de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 -- 172.21.15.133:0/161873183 shutdown_connections 2024-04-25T10:06:30.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.110+0000 7f97f35ce640 1 -- 172.21.15.133:0/161873183 wait complete. 2024-04-25T10:06:30.133 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T10:06:30.133 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:06:30.134 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.250+0000 7f2f2e707640 1 Processor -- start 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.250+0000 7f2f2e707640 1 -- start start 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f2e707640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f2815dd90 0x7f2f280ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f2e707640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f280b19f0 0x7f2f280cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f2e707640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f280ca9b0 con 0x7f2f2815d9b0 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f2e707640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f280cbe30 con 0x7f2f2815dd90 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f2e707640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f280cac90 con 0x7f2f280b19f0 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f277fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f2815dd90 0x7f2f280ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f277fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f2815dd90 0x7f2f280ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45650/0 (socket says 172.21.15.133:45650) 2024-04-25T10:06:30.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f277fe640 1 -- 172.21.15.133:0/1709480051 learned_addr learned my addr 172.21.15.133:0/1709480051 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:30.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f2cca4640 1 --2- 172.21.15.133:0/1709480051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f280b19f0 0x7f2f280cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f277fe640 1 -- 172.21.15.133:0/1709480051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f280b19f0 msgr2=0x7f2f280cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f277fe640 1 --2- 172.21.15.133:0/1709480051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f280b19f0 0x7f2f280cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f277fe640 1 -- 172.21.15.133:0/1709480051 >> v1:172.21.15.162:6789/0 conn(0x7f2f2815d9b0 legacy=0x7f2f280cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f277fe640 1 -- 172.21.15.133:0/1709480051 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f280caf70 con 0x7f2f2815dd90 2024-04-25T10:06:30.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f2cca4640 1 --2- 172.21.15.133:0/1709480051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f280b19f0 0x7f2f280cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f277fe640 1 --2- 172.21.15.133:0/1709480051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f2815dd90 0x7f2f280ce770 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009870 tx=0x7f2f1801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2224bd3aab7697a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f26ffd640 1 -- 172.21.15.133:0/1709480051 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180a7070 con 0x7f2f2815dd90 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f26ffd640 1 -- 172.21.15.133:0/1709480051 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f180b0070 con 0x7f2f2815dd90 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.254+0000 7f2f26ffd640 1 -- 172.21.15.133:0/1709480051 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180ab890 con 0x7f2f2815dd90 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 -- 172.21.15.133:0/1709480051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f2815dd90 msgr2=0x7f2f280ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 --2- 172.21.15.133:0/1709480051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f2815dd90 0x7f2f280ce770 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009870 tx=0x7f2f1801c490 comp rx=0 tx=0).stop 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 -- 172.21.15.133:0/1709480051 shutdown_connections 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 --2- 172.21.15.133:0/1709480051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f280b19f0 0x7f2f280cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 --2- 172.21.15.133:0/1709480051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f2815dd90 0x7f2f280ce770 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 -- 172.21.15.133:0/1709480051 >> 172.21.15.133:0/1709480051 conn(0x7f2f28067000 msgr2=0x7f2f2806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 -- 172.21.15.133:0/1709480051 shutdown_connections 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 -- 172.21.15.133:0/1709480051 wait complete. 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 Processor -- start 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 -- start start 2024-04-25T10:06:30.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f280b19f0 0x7f2f2815f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f2815d9b0 0x7f2f28140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f2815dd90 0x7f2f28141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2806e290 con 0x7f2f2815d9b0 2024-04-25T10:06:30.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2806e110 con 0x7f2f280b19f0 2024-04-25T10:06:30.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2806e410 con 0x7f2f2815dd90 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f277fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f2815d9b0 0x7f2f28140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f277fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f2815d9b0 0x7f2f28140ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45652/0 (socket says 172.21.15.133:45652) 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f277fe640 1 -- 172.21.15.133:0/2599799949 learned_addr learned my addr 172.21.15.133:0/2599799949 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f27fff640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f280b19f0 0x7f2f2815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f277fe640 1 -- 172.21.15.133:0/2599799949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f2815dd90 msgr2=0x7f2f28141520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f277fe640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f2815dd90 0x7f2f28141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f277fe640 1 -- 172.21.15.133:0/2599799949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f280b19f0 msgr2=0x7f2f2815f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f277fe640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f280b19f0 0x7f2f2815f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f277fe640 1 -- 172.21.15.133:0/2599799949 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f28142db0 con 0x7f2f2815d9b0 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f27fff640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f280b19f0 0x7f2f2815f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f277fe640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f2815d9b0 0x7f2f28140ce0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2f180ad040 tx=0x7f2f1801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f24ff9640 1 -- 172.21.15.133:0/2599799949 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180aaa20 con 0x7f2f2815d9b0 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f24ff9640 1 -- 172.21.15.133:0/2599799949 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f180aabe0 con 0x7f2f2815d9b0 2024-04-25T10:06:30.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 -- 172.21.15.133:0/2599799949 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f281553b0 con 0x7f2f2815d9b0 2024-04-25T10:06:30.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f24ff9640 1 -- 172.21.15.133:0/2599799949 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180be3f0 con 0x7f2f2815d9b0 2024-04-25T10:06:30.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.258+0000 7f2f2e707640 1 -- 172.21.15.133:0/2599799949 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f28155970 con 0x7f2f2815d9b0 2024-04-25T10:06:30.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.262+0000 7f2f2e707640 1 -- 172.21.15.133:0/2599799949 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f28068ef0 con 0x7f2f2815d9b0 2024-04-25T10:06:30.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.262+0000 7f2f24ff9640 1 -- 172.21.15.133:0/2599799949 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f180b43f0 con 0x7f2f2815d9b0 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.262+0000 7f2f24ff9640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f14066f60 0x7f2f14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.262+0000 7f2f24ff9640 1 -- 172.21.15.133:0/2599799949 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2f1811fb70 con 0x7f2f2815d9b0 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.262+0000 7f2f27fff640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f14066f60 0x7f2f14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.262+0000 7f2f27fff640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f14066f60 0x7f2f14069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2f281497e0 tx=0x7f2f1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.266+0000 7f2f24ff9640 1 -- 172.21.15.133:0/2599799949 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f180c4060 con 0x7f2f2815d9b0 2024-04-25T10:06:30.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.422+0000 7f2f2e707640 1 -- 172.21.15.133:0/2599799949 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f280cc2d0 con 0x7f2f2815d9b0 2024-04-25T10:06:30.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.422+0000 7f2f24ff9640 1 -- 172.21.15.133:0/2599799949 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+13319 (secure 0 0 0) 0x7f2f28142db0 con 0x7f2f2815d9b0 2024-04-25T10:06:30.427 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:30.427 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":57,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:06:29.103265+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/1425600691":"2024-04-26T10:06:26.082163+0000","172.21.15.133:6835/1425600691":"2024-04-26T10:06:26.082163+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 -- 172.21.15.133:0/2599799949 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f14066f60 msgr2=0x7f2f14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f14066f60 0x7f2f14069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2f281497e0 tx=0x7f2f1c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 -- 172.21.15.133:0/2599799949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f2815d9b0 msgr2=0x7f2f28140ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f2815d9b0 0x7f2f28140ce0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2f180ad040 tx=0x7f2f1801acd0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 -- 172.21.15.133:0/2599799949 shutdown_connections 2024-04-25T10:06:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f2815dd90 0x7f2f28141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f14066f60 0x7f2f14069420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f2815d9b0 0x7f2f28140ce0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 --2- 172.21.15.133:0/2599799949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f280b19f0 0x7f2f2815f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 -- 172.21.15.133:0/2599799949 >> 172.21.15.133:0/2599799949 conn(0x7f2f28067000 msgr2=0x7f2f280b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 -- 172.21.15.133:0/2599799949 shutdown_connections 2024-04-25T10:06:30.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.426+0000 7f2f2e707640 1 -- 172.21.15.133:0/2599799949 wait complete. 2024-04-25T10:06:30.445 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:30.445 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:30.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9532efc640 1 Processor -- start 2024-04-25T10:06:30.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9532efc640 1 -- start start 2024-04-25T10:06:30.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9532efc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c145170 0x7f952c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9532efc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c0ce2e0 con 0x7f952c15ced0 2024-04-25T10:06:30.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9532efc640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c06d4b0 con 0x7f952c06e4d0 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9532efc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c06db00 con 0x7f952c145170 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9531499640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f952c15ced0 0x7f952c0b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50416/0 (socket says 172.21.15.133:50416) 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9531499640 1 -- 172.21.15.133:0/788679119 learned_addr learned my addr 172.21.15.133:0/788679119 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901103726 0 0) 0x7f952c0ce2e0 con 0x7f952c15ced0 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9530c98640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c145170 0x7f952c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9514003530 con 0x7f952c15ced0 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1069443261 0 0) 0x7f9514003530 con 0x7f952c15ced0 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c145170 msgr2=0x7f952c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c145170 0x7f952c06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 >> v1:172.21.15.162:6789/0 conn(0x7f952c06e4d0 legacy=0x7f952c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c06d790 con 0x7f952c15ced0 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f9524001a00 con 0x7f952c15ced0 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 >> v1:172.21.15.133:6789/0 conn(0x7f952c15ced0 legacy=0x7f952c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c145170 0x7f95140039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9514003f70 0x7f9514004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9514004d50 0x7f95140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c06db00 con 0x7f9514004d50 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951400bab0 con 0x7f9514003f70 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951400bcb0 con 0x7f952c145170 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9531499640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9514004d50 0x7f95140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f952bfff640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9514003f70 0x7f9514004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9530c98640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c145170 0x7f95140039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9531499640 1 -- 172.21.15.133:0/788679119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c145170 msgr2=0x7f95140039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9531499640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c145170 0x7f95140039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9531499640 1 -- 172.21.15.133:0/788679119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9514003f70 msgr2=0x7f9514004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9531499640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9514003f70 0x7f9514004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.566+0000 7f9531499640 1 -- 172.21.15.133:0/788679119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9514003530 con 0x7f9514004d50 2024-04-25T10:06:30.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9531499640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9514004d50 0x7f95140051d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f952400a850 tx=0x7f952401d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f5eae95e4edb3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952400a4a0 con 0x7f9514004d50 2024-04-25T10:06:30.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95240ad650 con 0x7f9514004d50 2024-04-25T10:06:30.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f952b7fe640 1 -- 172.21.15.133:0/788679119 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95240ada80 con 0x7f9514004d50 2024-04-25T10:06:30.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 -- 172.21.15.133:0/788679119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9514004d50 msgr2=0x7f95140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 --2- 172.21.15.133:0/788679119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9514004d50 0x7f95140051d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f952400a850 tx=0x7f952401d3a0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9531499640 1 -- 172.21.15.133:0/788679119 reap_dead start 2024-04-25T10:06:30.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 -- 172.21.15.133:0/788679119 shutdown_connections 2024-04-25T10:06:30.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 -- 172.21.15.133:0/788679119 >> 172.21.15.133:0/788679119 conn(0x7f952c066f70 msgr2=0x7f952c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 -- 172.21.15.133:0/788679119 shutdown_connections 2024-04-25T10:06:30.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 -- 172.21.15.133:0/788679119 wait complete. 2024-04-25T10:06:30.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 Processor -- start 2024-04-25T10:06:30.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 -- start start 2024-04-25T10:06:30.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c13c750 0x7f952c13a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f952c154430 0x7f952c1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c154df0 0x7f952c13efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0cdf30 con 0x7f952c13c750 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c145550 con 0x7f952c154430 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.570+0000 7f9532efc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c140270 con 0x7f952c154df0 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9530c98640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c13c750 0x7f952c13a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9530c98640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c13c750 0x7f952c13a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45670/0 (socket says 172.21.15.133:45670) 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9530c98640 1 -- 172.21.15.133:0/2142374285 learned_addr learned my addr 172.21.15.133:0/2142374285 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f952bfff640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f952c154430 0x7f952c1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9531499640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c154df0 0x7f952c13efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9530c98640 1 -- 172.21.15.133:0/2142374285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c154df0 msgr2=0x7f952c13efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9530c98640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c154df0 0x7f952c13efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9530c98640 1 -- 172.21.15.133:0/2142374285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f952c154430 msgr2=0x7f952c1548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9530c98640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f952c154430 0x7f952c1548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9530c98640 1 -- 172.21.15.133:0/2142374285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c1555e0 con 0x7f952c13c750 2024-04-25T10:06:30.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9531499640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c154df0 0x7f952c13efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:30.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9530c98640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c13c750 0x7f952c13a290 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f951c0115e0 tx=0x7f951c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f95297fa640 1 -- 172.21.15.133:0/2142374285 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c00f8f0 con 0x7f952c13c750 2024-04-25T10:06:30.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f95297fa640 1 -- 172.21.15.133:0/2142374285 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c00fa90 con 0x7f952c13c750 2024-04-25T10:06:30.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9532efc640 1 -- 172.21.15.133:0/2142374285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c1558c0 con 0x7f952c13c750 2024-04-25T10:06:30.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f95297fa640 1 -- 172.21.15.133:0/2142374285 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c00fe40 con 0x7f952c13c750 2024-04-25T10:06:30.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9532efc640 1 -- 172.21.15.133:0/2142374285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c155270 con 0x7f952c13c750 2024-04-25T10:06:30.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f9532efc640 1 -- 172.21.15.133:0/2142374285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c068e80 con 0x7f952c13c750 2024-04-25T10:06:30.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f95297fa640 1 -- 172.21.15.133:0/2142374285 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f951c015460 con 0x7f952c13c750 2024-04-25T10:06:30.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f95297fa640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9518066f90 0x7f9518069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f95297fa640 1 -- 172.21.15.133:0/2142374285 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f951c085850 con 0x7f952c13c750 2024-04-25T10:06:30.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f952bfff640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9518066f90 0x7f9518069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.574+0000 7f952bfff640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9518066f90 0x7f9518069450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f952c156170 tx=0x7f9520009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.578+0000 7f95297fa640 1 -- 172.21.15.133:0/2142374285 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f951c019030 con 0x7f952c13c750 2024-04-25T10:06:30.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 -- 172.21.15.133:0/2142374285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f952c1448d0 con 0x7f952c13c750 2024-04-25T10:06:30.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f95297fa640 1 -- 172.21.15.133:0/2142374285 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f951c056b40 con 0x7f952c13c750 2024-04-25T10:06:30.741 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 -- 172.21.15.133:0/2142374285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9518066f90 msgr2=0x7f9518069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9518066f90 0x7f9518069450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f952c156170 tx=0x7f9520009040 comp rx=0 tx=0).stop 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 -- 172.21.15.133:0/2142374285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c13c750 msgr2=0x7f952c13a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c13c750 0x7f952c13a290 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f951c0115e0 tx=0x7f951c011aa0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 -- 172.21.15.133:0/2142374285 shutdown_connections 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f952c154df0 0x7f952c13efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f952c154430 0x7f952c1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9518066f90 0x7f9518069450 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 --2- 172.21.15.133:0/2142374285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c13c750 0x7f952c13a290 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 -- 172.21.15.133:0/2142374285 >> 172.21.15.133:0/2142374285 conn(0x7f952c066f70 msgr2=0x7f952c0b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 -- 172.21.15.133:0/2142374285 shutdown_connections 2024-04-25T10:06:30.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.738+0000 7f9532efc640 1 -- 172.21.15.133:0/2142374285 wait complete. 2024-04-25T10:06:30.760 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T10:06:30.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.878+0000 7fa21efb4640 1 Processor -- start 2024-04-25T10:06:30.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.878+0000 7fa21efb4640 1 -- start start 2024-04-25T10:06:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa218158110 0x7fa2180b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa2181584f0 0x7fa21815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2180cbe60 con 0x7fa21806e460 2024-04-25T10:06:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2180b19b0 con 0x7fa2181584f0 2024-04-25T10:06:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21815b360 con 0x7fa218158110 2024-04-25T10:06:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa217fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa2181584f0 0x7fa21815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21d551640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa21806e460 0x7fa21815e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50418/0 (socket says 172.21.15.133:50418) 2024-04-25T10:06:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21d551640 1 -- 172.21.15.133:0/1359191066 learned_addr learned my addr 172.21.15.133:0/1359191066 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21cd50640 1 --2- 172.21.15.133:0/1359191066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa218158110 0x7fa2180b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa217fff640 1 -- 172.21.15.133:0/1359191066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa218158110 msgr2=0x7fa2180b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa217fff640 1 --2- 172.21.15.133:0/1359191066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa218158110 0x7fa2180b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa217fff640 1 -- 172.21.15.133:0/1359191066 >> v1:172.21.15.133:6789/0 conn(0x7fa21806e460 legacy=0x7fa21815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa217fff640 1 -- 172.21.15.133:0/1359191066 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21815b640 con 0x7fa2181584f0 2024-04-25T10:06:30.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa2177fe640 1 -- 172.21.15.133:0/1359191066 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152422106 0 0) 0x7fa2180cbe60 con 0x7fa21806e460 2024-04-25T10:06:30.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa217fff640 1 --2- 172.21.15.133:0/1359191066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2181584f0 0x7fa21815ad10 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa2080097c0 tx=0x7fa20801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a804f31b19f926b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa2177fe640 1 -- 172.21.15.133:0/1359191066 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2080b1070 con 0x7fa2181584f0 2024-04-25T10:06:30.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa2177fe640 1 -- 172.21.15.133:0/1359191066 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2080ac410 con 0x7fa2181584f0 2024-04-25T10:06:30.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa2177fe640 1 -- 172.21.15.133:0/1359191066 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2080ac7e0 con 0x7fa2181584f0 2024-04-25T10:06:30.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 -- 172.21.15.133:0/1359191066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2181584f0 msgr2=0x7fa21815ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 --2- 172.21.15.133:0/1359191066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2181584f0 0x7fa21815ad10 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa2080097c0 tx=0x7fa20801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 -- 172.21.15.133:0/1359191066 shutdown_connections 2024-04-25T10:06:30.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 --2- 172.21.15.133:0/1359191066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2181584f0 0x7fa21815ad10 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 --2- 172.21.15.133:0/1359191066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa218158110 0x7fa2180b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 -- 172.21.15.133:0/1359191066 >> 172.21.15.133:0/1359191066 conn(0x7fa218067000 msgr2=0x7fa21806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 -- 172.21.15.133:0/1359191066 shutdown_connections 2024-04-25T10:06:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.882+0000 7fa21efb4640 1 -- 172.21.15.133:0/1359191066 wait complete. 2024-04-25T10:06:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21efb4640 1 Processor -- start 2024-04-25T10:06:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21efb4640 1 -- start start 2024-04-25T10:06:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21efb4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa21806e460 0x7fa218156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21efb4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa218158110 0x7fa218157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21efb4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2181584f0 0x7fa2181f9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21efb4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2180cd7f0 con 0x7fa21806e460 2024-04-25T10:06:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21efb4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2180cd670 con 0x7fa2181584f0 2024-04-25T10:06:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21efb4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21806d350 con 0x7fa218158110 2024-04-25T10:06:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21cd50640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa21806e460 0x7fa218156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21cd50640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa21806e460 0x7fa218156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45690/0 (socket says 172.21.15.133:45690) 2024-04-25T10:06:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21cd50640 1 -- 172.21.15.133:0/2325133602 learned_addr learned my addr 172.21.15.133:0/2325133602 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21cd50640 1 -- 172.21.15.133:0/2325133602 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa218158110 msgr2=0x7fa218157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa217fff640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa218158110 0x7fa218157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21d551640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2181584f0 0x7fa2181f9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21cd50640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa218158110 0x7fa218157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21cd50640 1 -- 172.21.15.133:0/2325133602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2181584f0 msgr2=0x7fa2181f9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21cd50640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2181584f0 0x7fa2181f9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21cd50640 1 -- 172.21.15.133:0/2325133602 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa218152a40 con 0x7fa21806e460 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21cd50640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa21806e460 0x7fa218156e10 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa210007ef0 tx=0x7fa21000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa2157fa640 1 -- 172.21.15.133:0/2325133602 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210014620 con 0x7fa21806e460 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa2157fa640 1 -- 172.21.15.133:0/2325133602 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa210019070 con 0x7fa21806e460 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21efb4640 1 -- 172.21.15.133:0/2325133602 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa218152d20 con 0x7fa21806e460 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa2157fa640 1 -- 172.21.15.133:0/2325133602 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210014a00 con 0x7fa21806e460 2024-04-25T10:06:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21efb4640 1 -- 172.21.15.133:0/2325133602 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2181532e0 con 0x7fa21806e460 2024-04-25T10:06:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.886+0000 7fa21efb4640 1 -- 172.21.15.133:0/2325133602 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21806bc20 con 0x7fa21806e460 2024-04-25T10:06:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.890+0000 7fa2157fa640 1 -- 172.21.15.133:0/2325133602 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa210014ba0 con 0x7fa21806e460 2024-04-25T10:06:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.890+0000 7fa2157fa640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1f8066f90 0x7fa1f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.890+0000 7fa2157fa640 1 -- 172.21.15.133:0/2325133602 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa210085860 con 0x7fa21806e460 2024-04-25T10:06:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.890+0000 7fa217fff640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1f8066f90 0x7fa1f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.890+0000 7fa217fff640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1f8066f90 0x7fa1f8069450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa2080ae000 tx=0x7fa2080a6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:30.894+0000 7fa2157fa640 1 -- 172.21.15.133:0/2325133602 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa210056940 con 0x7fa21806e460 2024-04-25T10:06:31.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.098+0000 7fa21efb4640 1 -- 172.21.15.133:0/2325133602 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7fa2180cb490 con 0x7fa21806e460 2024-04-25T10:06:31.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.106+0000 7fa2157fa640 1 -- 172.21.15.133:0/2325133602 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v228) v1 ==== 146+0+0 (secure 0 0 0) 0x7fa210056240 con 0x7fa21806e460 2024-04-25T10:06:31.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 -- 172.21.15.133:0/2325133602 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1f8066f90 msgr2=0x7fa1f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1f8066f90 0x7fa1f8069450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa2080ae000 tx=0x7fa2080a6040 comp rx=0 tx=0).stop 2024-04-25T10:06:31.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 -- 172.21.15.133:0/2325133602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa21806e460 msgr2=0x7fa218156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa21806e460 0x7fa218156e10 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa210007ef0 tx=0x7fa21000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 -- 172.21.15.133:0/2325133602 shutdown_connections 2024-04-25T10:06:31.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2181584f0 0x7fa2181f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa218158110 0x7fa218157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1f8066f90 0x7fa1f8069450 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 --2- 172.21.15.133:0/2325133602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa21806e460 0x7fa218156e10 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 -- 172.21.15.133:0/2325133602 >> 172.21.15.133:0/2325133602 conn(0x7fa218067000 msgr2=0x7fa2181588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 -- 172.21.15.133:0/2325133602 shutdown_connections 2024-04-25T10:06:31.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.110+0000 7fa21efb4640 1 -- 172.21.15.133:0/2325133602 wait complete. 2024-04-25T10:06:31.129 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:06:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffbed2640 1 Processor -- start 2024-04-25T10:06:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffbed2640 1 -- start start 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffbed2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ff4145170 0x7f9ff406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffbed2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9ff406e4d0 0x7f9ff40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffbed2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ff40cdd60 0x7f9ff415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffbed2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff406db00 con 0x7f9ff40cdd60 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffbed2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff4145550 con 0x7f9ff406e4d0 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffbed2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40ce2e0 con 0x7f9ff4145170 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffa46f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ff40cdd60 0x7f9ff415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffa46f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ff40cdd60 0x7f9ff415ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45692/0 (socket says 172.21.15.133:45692) 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffa46f640 1 -- 172.21.15.133:0/579422746 learned_addr learned my addr 172.21.15.133:0/579422746 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ff9c6e640 1 --2- 172.21.15.133:0/579422746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ff4145170 0x7f9ff406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ff946d640 1 --2- 172.21.15.133:0/579422746 >> v2:172.21.15.162:3300/0 conn(0x7f9ff406e4d0 0x7f9ff40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffa46f640 1 -- 172.21.15.133:0/579422746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ff4145170 msgr2=0x7f9ff406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffa46f640 1 --2- 172.21.15.133:0/579422746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ff4145170 0x7f9ff406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffa46f640 1 -- 172.21.15.133:0/579422746 >> v2:172.21.15.162:3300/0 conn(0x7f9ff406e4d0 msgr2=0x7f9ff40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffa46f640 1 --2- 172.21.15.133:0/579422746 >> v2:172.21.15.162:3300/0 conn(0x7f9ff406e4d0 0x7f9ff40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.250+0000 7f9ffa46f640 1 -- 172.21.15.133:0/579422746 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff406d4b0 con 0x7f9ff40cdd60 2024-04-25T10:06:31.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffa46f640 1 --2- 172.21.15.133:0/579422746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff40cdd60 0x7f9ff415ced0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40b2300 tx=0x7f9fec01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48aaef907a2b4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff8c6c640 1 -- 172.21.15.133:0/579422746 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0b0070 con 0x7f9ff40cdd60 2024-04-25T10:06:31.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff8c6c640 1 -- 172.21.15.133:0/579422746 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec0ab600 con 0x7f9ff40cdd60 2024-04-25T10:06:31.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff8c6c640 1 -- 172.21.15.133:0/579422746 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0ab9d0 con 0x7f9ff40cdd60 2024-04-25T10:06:31.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 -- 172.21.15.133:0/579422746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff40cdd60 msgr2=0x7f9ff415ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 --2- 172.21.15.133:0/579422746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff40cdd60 0x7f9ff415ced0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40b2300 tx=0x7f9fec01c520 comp rx=0 tx=0).stop 2024-04-25T10:06:31.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 -- 172.21.15.133:0/579422746 shutdown_connections 2024-04-25T10:06:31.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 --2- 172.21.15.133:0/579422746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff40cdd60 0x7f9ff415ced0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 --2- 172.21.15.133:0/579422746 >> v2:172.21.15.162:3300/0 conn(0x7f9ff406e4d0 0x7f9ff40cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 --2- 172.21.15.133:0/579422746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ff4145170 0x7f9ff406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 -- 172.21.15.133:0/579422746 >> 172.21.15.133:0/579422746 conn(0x7f9ff4066f70 msgr2=0x7f9ff406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 -- 172.21.15.133:0/579422746 shutdown_connections 2024-04-25T10:06:31.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 -- 172.21.15.133:0/579422746 wait complete. 2024-04-25T10:06:31.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 Processor -- start 2024-04-25T10:06:31.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 -- start start 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ff406e4d0 0x7f9ff4164e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff40cdd60 0x7f9ff4160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff4145170 0x7f9ff41614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff415f690 con 0x7f9ff40cdd60 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff415f510 con 0x7f9ff4145170 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffbed2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff415f810 con 0x7f9ff406e4d0 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff946d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff40cdd60 0x7f9ff4160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff946d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff40cdd60 0x7f9ff4160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45702/0 (socket says 172.21.15.133:45702) 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff946d640 1 -- 172.21.15.133:0/214719141 learned_addr learned my addr 172.21.15.133:0/214719141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff9c6e640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ff406e4d0 0x7f9ff4164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffa46f640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff4145170 0x7f9ff41614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff946d640 1 -- 172.21.15.133:0/214719141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ff406e4d0 msgr2=0x7f9ff4164e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff946d640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ff406e4d0 0x7f9ff4164e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff946d640 1 -- 172.21.15.133:0/214719141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff4145170 msgr2=0x7f9ff41614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff946d640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff4145170 0x7f9ff41614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ff946d640 1 -- 172.21.15.133:0/214719141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff4161c30 con 0x7f9ff40cdd60 2024-04-25T10:06:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.254+0000 7f9ffa46f640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff4145170 0x7f9ff41614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9ff946d640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff40cdd60 0x7f9ff4160fb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400d800 tx=0x7f9fe400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9fea7fc640 1 -- 172.21.15.133:0/214719141 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40096a0 con 0x7f9ff40cdd60 2024-04-25T10:06:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9fea7fc640 1 -- 172.21.15.133:0/214719141 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe4019070 con 0x7f9ff40cdd60 2024-04-25T10:06:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9ffbed2640 1 -- 172.21.15.133:0/214719141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff4161f10 con 0x7f9ff40cdd60 2024-04-25T10:06:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9fea7fc640 1 -- 172.21.15.133:0/214719141 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4009a80 con 0x7f9ff40cdd60 2024-04-25T10:06:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9ffbed2640 1 -- 172.21.15.133:0/214719141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff413db90 con 0x7f9ff40cdd60 2024-04-25T10:06:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9ffbed2640 1 -- 172.21.15.133:0/214719141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff4068e80 con 0x7f9ff40cdd60 2024-04-25T10:06:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9fea7fc640 1 -- 172.21.15.133:0/214719141 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9fe4009c20 con 0x7f9ff40cdd60 2024-04-25T10:06:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9fea7fc640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fd8066f60 0x7f9fd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9fea7fc640 1 -- 172.21.15.133:0/214719141 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9fe40852e0 con 0x7f9ff40cdd60 2024-04-25T10:06:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9ff9c6e640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fd8066f60 0x7f9fd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.258+0000 7f9ff9c6e640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fd8066f60 0x7f9fd8069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0003fd0 tx=0x7f9ff00039a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.262+0000 7f9fea7fc640 1 -- 172.21.15.133:0/214719141 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9fe4056450 con 0x7f9ff40cdd60 2024-04-25T10:06:31.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.410+0000 7f9ffbed2640 1 -- 172.21.15.133:0/214719141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9ff415ce90 con 0x7f9ff40cdd60 2024-04-25T10:06:31.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.414+0000 7f9fea7fc640 1 -- 172.21.15.133:0/214719141 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9fe4055d50 con 0x7f9ff40cdd60 2024-04-25T10:06:31.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 -- 172.21.15.133:0/214719141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fd8066f60 msgr2=0x7f9fd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fd8066f60 0x7f9fd8069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0003fd0 tx=0x7f9ff00039a0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 -- 172.21.15.133:0/214719141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff40cdd60 msgr2=0x7f9ff4160fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff40cdd60 0x7f9ff4160fb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400d800 tx=0x7f9fe400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 -- 172.21.15.133:0/214719141 shutdown_connections 2024-04-25T10:06:31.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ff4145170 0x7f9ff41614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fd8066f60 0x7f9fd8069420 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff40cdd60 0x7f9ff4160fb0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 --2- 172.21.15.133:0/214719141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ff406e4d0 0x7f9ff4164e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 -- 172.21.15.133:0/214719141 >> 172.21.15.133:0/214719141 conn(0x7f9ff4066f70 msgr2=0x7f9ff40b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 -- 172.21.15.133:0/214719141 shutdown_connections 2024-04-25T10:06:31.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.418+0000 7f9ffbed2640 1 -- 172.21.15.133:0/214719141 wait complete. 2024-04-25T10:06:31.437 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:31.437 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:31.473 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:31.511 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:31.511 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:06:31.511 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:31.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:31.521 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:06:31.521 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:31.558 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:31.558 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:06:31.558 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:31.568 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:31.568 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:06:31.569 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:31.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:31.610 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:06:31.610 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:31.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.726+0000 7f2435020640 1 Processor -- start 2024-04-25T10:06:31.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.726+0000 7f2435020640 1 -- start start 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f2435020640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f243006eb70 0x7f24300b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f2435020640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f24301584f0 0x7f243015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f2435020640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24300cbe60 con 0x7f2430158110 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f2435020640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24300b19b0 con 0x7f243006eb70 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f2435020640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243015b360 con 0x7f24301584f0 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242e59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f243006eb70 0x7f24300b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242e59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f243006eb70 0x7f24300b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45716/0 (socket says 172.21.15.133:45716) 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242e59c640 1 -- 172.21.15.133:0/2607395988 learned_addr learned my addr 172.21.15.133:0/2607395988 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242e59c640 1 -- 172.21.15.133:0/2607395988 >> v2:172.21.15.162:3300/0 conn(0x7f24301584f0 msgr2=0x7f243015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242f59e640 1 --2- 172.21.15.133:0/2607395988 >> v2:172.21.15.162:3300/0 conn(0x7f24301584f0 0x7f243015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242e59c640 1 --2- 172.21.15.133:0/2607395988 >> v2:172.21.15.162:3300/0 conn(0x7f24301584f0 0x7f243015ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242e59c640 1 -- 172.21.15.133:0/2607395988 >> v1:172.21.15.162:6789/0 conn(0x7f2430158110 legacy=0x7f243006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242e59c640 1 -- 172.21.15.133:0/2607395988 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243015b640 con 0x7f243006eb70 2024-04-25T10:06:31.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242e59c640 1 --2- 172.21.15.133:0/2607395988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f243006eb70 0x7f24300b1f00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f24300b2470 tx=0x7f242401eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae003c62eafadd4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242dd9b640 1 -- 172.21.15.133:0/2607395988 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24240b4070 con 0x7f243006eb70 2024-04-25T10:06:31.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242dd9b640 1 -- 172.21.15.133:0/2607395988 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24240a7b70 con 0x7f243006eb70 2024-04-25T10:06:31.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.730+0000 7f242dd9b640 1 -- 172.21.15.133:0/2607395988 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24240af440 con 0x7f243006eb70 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 -- 172.21.15.133:0/2607395988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f243006eb70 msgr2=0x7f24300b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 --2- 172.21.15.133:0/2607395988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f243006eb70 0x7f24300b1f00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f24300b2470 tx=0x7f242401eb80 comp rx=0 tx=0).stop 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 -- 172.21.15.133:0/2607395988 shutdown_connections 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 --2- 172.21.15.133:0/2607395988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f243006eb70 0x7f24300b1f00 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 --2- 172.21.15.133:0/2607395988 >> v2:172.21.15.162:3300/0 conn(0x7f24301584f0 0x7f243015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 -- 172.21.15.133:0/2607395988 >> 172.21.15.133:0/2607395988 conn(0x7f2430067000 msgr2=0x7f243006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 -- 172.21.15.133:0/2607395988 shutdown_connections 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 -- 172.21.15.133:0/2607395988 wait complete. 2024-04-25T10:06:31.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 Processor -- start 2024-04-25T10:06:31.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 -- start start 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f243006eb70 0x7f24300ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2430158110 0x7f24301553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24301584f0 0x7f2430155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243006dce0 con 0x7f243006eb70 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243006db60 con 0x7f24301584f0 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243006de60 con 0x7f2430158110 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f242ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f243006eb70 0x7f24300ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f242ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f243006eb70 0x7f24300ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45722/0 (socket says 172.21.15.133:45722) 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f242ed9d640 1 -- 172.21.15.133:0/3249339986 learned_addr learned my addr 172.21.15.133:0/3249339986 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f242ed9d640 1 -- 172.21.15.133:0/3249339986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2430158110 msgr2=0x7f24301553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f242ed9d640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2430158110 0x7f24301553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f242f59e640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24301584f0 0x7f2430155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f242ed9d640 1 -- 172.21.15.133:0/3249339986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24301584f0 msgr2=0x7f2430155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f242ed9d640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24301584f0 0x7f2430155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f242ed9d640 1 -- 172.21.15.133:0/3249339986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430155e50 con 0x7f243006eb70 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f242ed9d640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f243006eb70 0x7f24300ce550 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f242000c480 tx=0x7f242000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f24177fe640 1 -- 172.21.15.133:0/3249339986 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420012830 con 0x7f243006eb70 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f24177fe640 1 -- 172.21.15.133:0/3249339986 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2420017070 con 0x7f243006eb70 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 -- 172.21.15.133:0/3249339986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243014dce0 con 0x7f243006eb70 2024-04-25T10:06:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f24177fe640 1 -- 172.21.15.133:0/3249339986 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420012c30 con 0x7f243006eb70 2024-04-25T10:06:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.734+0000 7f2435020640 1 -- 172.21.15.133:0/3249339986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243014e2a0 con 0x7f243006eb70 2024-04-25T10:06:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.738+0000 7f2435020640 1 -- 172.21.15.133:0/3249339986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243006bc20 con 0x7f243006eb70 2024-04-25T10:06:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.738+0000 7f24177fe640 1 -- 172.21.15.133:0/3249339986 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f242001c460 con 0x7f243006eb70 2024-04-25T10:06:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.738+0000 7f24177fe640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f241c066f60 0x7f241c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.738+0000 7f24177fe640 1 -- 172.21.15.133:0/3249339986 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2420084d00 con 0x7f243006eb70 2024-04-25T10:06:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.738+0000 7f242e59c640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f241c066f60 0x7f241c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.738+0000 7f242e59c640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f241c066f60 0x7f241c069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f24240b1000 tx=0x7f2424008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.742+0000 7f24177fe640 1 -- 172.21.15.133:0/3249339986 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2420055e80 con 0x7f243006eb70 2024-04-25T10:06:31.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.930+0000 7f2435020640 1 -- 172.21.15.133:0/3249339986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24300b3f10 con 0x7f243006eb70 2024-04-25T10:06:31.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.930+0000 7f24177fe640 1 -- 172.21.15.133:0/3249339986 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f2420055780 con 0x7f243006eb70 2024-04-25T10:06:31.935 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:31.935 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2901416068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2901416068},{"type":"v1","addr":"172.21.15.162:6833","nonce":2901416068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2377356527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2377356527},{"type":"v1","addr":"172.21.15.162:6835","nonce":2377356527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6009,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3663586284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3663586284},{"type":"v1","addr":"172.21.15.133:6836","nonce":3663586284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/302067200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":302067200},{"type":"v1","addr":"172.21.15.133:6837","nonce":302067200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[]} 2024-04-25T10:06:31.935 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 29 2024-04-25T10:06:31.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 -- 172.21.15.133:0/3249339986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f241c066f60 msgr2=0x7f241c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f241c066f60 0x7f241c069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f24240b1000 tx=0x7f2424008040 comp rx=0 tx=0).stop 2024-04-25T10:06:31.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 -- 172.21.15.133:0/3249339986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f243006eb70 msgr2=0x7f24300ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f243006eb70 0x7f24300ce550 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f242000c480 tx=0x7f242000c940 comp rx=0 tx=0).stop 2024-04-25T10:06:31.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 -- 172.21.15.133:0/3249339986 shutdown_connections 2024-04-25T10:06:31.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24301584f0 0x7f2430155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2430158110 0x7f24301553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f241c066f60 0x7f241c069420 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 --2- 172.21.15.133:0/3249339986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f243006eb70 0x7f24300ce550 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 -- 172.21.15.133:0/3249339986 >> 172.21.15.133:0/3249339986 conn(0x7f2430067000 msgr2=0x7f24300caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 -- 172.21.15.133:0/3249339986 shutdown_connections 2024-04-25T10:06:31.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:31.934+0000 7f2435020640 1 -- 172.21.15.133:0/3249339986 wait complete. 2024-04-25T10:06:31.953 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:06:31.953 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:06:31.953 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:06:31.953 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:31.953 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:06:31.954 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:06:31.954 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:06:31.954 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:31.954 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:06:31.954 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:06:31.955 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:06:31.955 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:31.955 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:06:31.955 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:06:31.955 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:06:31.955 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:31.964 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:06:31.964 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:06:31.965 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:06:31.965 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:06:31.968 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:06:31.969 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:06:31.973 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:06:31.973 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:06:32.010 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:06:32.010 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:32.010 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:06:32.011 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:32.019 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:06:32.019 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:32.020 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:06:32.020 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:32.021 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:06:32.068 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:06:32.070 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:06:32.092 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:06:32.092 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:06:32.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.126+0000 7fa2b065a640 1 Processor -- start 2024-04-25T10:06:32.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.126+0000 7fa2b065a640 1 -- start start 2024-04-25T10:06:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2b065a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa2a80ccd00 0x7fa2a80b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2b065a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a80cb750 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2b065a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a80cba30 con 0x7fa2a815aa50 2024-04-25T10:06:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2b065a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a80cc360 con 0x7fa2a80ccd00 2024-04-25T10:06:32.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2aebf7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa2a80cc7f0 0x7fa2a80caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50458/0 (socket says 172.21.15.133:50458) 2024-04-25T10:06:32.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2aebf7640 1 -- 172.21.15.133:0/4101311743 learned_addr learned my addr 172.21.15.133:0/4101311743 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:32.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2ae3f6640 1 --2- 172.21.15.133:0/4101311743 >> v2:172.21.15.162:3300/0 conn(0x7fa2a80ccd00 0x7fa2a80b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2ad3f4640 1 -- 172.21.15.133:0/4101311743 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450507278 0 0) 0x7fa2a80cb750 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2ad3f4640 1 -- 172.21.15.133:0/4101311743 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa294005740 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2ae3f6640 1 -- 172.21.15.133:0/4101311743 >> v1:172.21.15.162:6789/0 conn(0x7fa2a815aa50 legacy=0x7fa2a80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2ae3f6640 1 -- 172.21.15.133:0/4101311743 >> v1:172.21.15.133:6789/0 conn(0x7fa2a80cc7f0 legacy=0x7fa2a80caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2ae3f6640 1 -- 172.21.15.133:0/4101311743 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80cbd10 con 0x7fa2a80ccd00 2024-04-25T10:06:32.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2ae3f6640 1 --2- 172.21.15.133:0/4101311743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2a80ccd00 0x7fa2a80b29d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40097c0 tx=0x7fa2a401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1df97a4753b04fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2ad3f4640 1 -- 172.21.15.133:0/4101311743 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40b1070 con 0x7fa2a80ccd00 2024-04-25T10:06:32.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2ad3f4640 1 -- 172.21.15.133:0/4101311743 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a40ac410 con 0x7fa2a80ccd00 2024-04-25T10:06:32.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2ad3f4640 1 -- 172.21.15.133:0/4101311743 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40ac7c0 con 0x7fa2a80ccd00 2024-04-25T10:06:32.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2b065a640 1 -- 172.21.15.133:0/4101311743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2a80ccd00 msgr2=0x7fa2a80b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2b065a640 1 --2- 172.21.15.133:0/4101311743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2a80ccd00 0x7fa2a80b29d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40097c0 tx=0x7fa2a401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2b065a640 1 -- 172.21.15.133:0/4101311743 shutdown_connections 2024-04-25T10:06:32.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2b065a640 1 --2- 172.21.15.133:0/4101311743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2a80ccd00 0x7fa2a80b29d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.130+0000 7fa2b065a640 1 -- 172.21.15.133:0/4101311743 >> 172.21.15.133:0/4101311743 conn(0x7fa2a80999d0 msgr2=0x7fa2a80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 -- 172.21.15.133:0/4101311743 shutdown_connections 2024-04-25T10:06:32.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 -- 172.21.15.133:0/4101311743 wait complete. 2024-04-25T10:06:32.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 Processor -- start 2024-04-25T10:06:32.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 -- start start 2024-04-25T10:06:32.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a80cc7f0 0x7fa2a81512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2a80ccd00 0x7fa2a80b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2a815aa50 0x7fa2a80c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a806eea0 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a806ed20 con 0x7fa2a815aa50 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a806f020 con 0x7fa2a80ccd00 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2ae3f6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a80cc7f0 0x7fa2a81512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2ae3f6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a80cc7f0 0x7fa2a81512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45766/0 (socket says 172.21.15.133:45766) 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2ae3f6640 1 -- 172.21.15.133:0/3559901858 learned_addr learned my addr 172.21.15.133:0/3559901858 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2adbf5640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2a80ccd00 0x7fa2a80b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2aebf7640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2a815aa50 0x7fa2a80c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2ae3f6640 1 -- 172.21.15.133:0/3559901858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2a80ccd00 msgr2=0x7fa2a80b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2ae3f6640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2a80ccd00 0x7fa2a80b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2ae3f6640 1 -- 172.21.15.133:0/3559901858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2a815aa50 msgr2=0x7fa2a80c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2ae3f6640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2a815aa50 0x7fa2a80c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2ae3f6640 1 -- 172.21.15.133:0/3559901858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80c45a0 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2adbf5640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2a80ccd00 0x7fa2a80b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2aebf7640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2a815aa50 0x7fa2a80c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2ae3f6640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a80cc7f0 0x7fa2a81512b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40097c0 tx=0x7fa2a401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa29effd640 1 -- 172.21.15.133:0/3559901858 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40bd070 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa29effd640 1 -- 172.21.15.133:0/3559901858 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a40b2040 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 -- 172.21.15.133:0/3559901858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a80c4880 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa29effd640 1 -- 172.21.15.133:0/3559901858 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40b1050 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 -- 172.21.15.133:0/3559901858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a80c4e40 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.134+0000 7fa2b065a640 1 -- 172.21.15.133:0/3559901858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a8067c70 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.138+0000 7fa29effd640 1 -- 172.21.15.133:0/3559901858 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2a40c0070 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.138+0000 7fa29effd640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa28c066f60 0x7fa28c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.138+0000 7fa29effd640 1 -- 172.21.15.133:0/3559901858 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa2a411ead0 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.138+0000 7fa2adbf5640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa28c066f60 0x7fa28c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.138+0000 7fa2adbf5640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa28c066f60 0x7fa28c069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa2a81f5a80 tx=0x7fa298009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.142+0000 7fa29effd640 1 -- 172.21.15.133:0/3559901858 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa2a4016090 con 0x7fa2a80cc7f0 2024-04-25T10:06:32.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:32.302+0000 7fa2b065a640 1 -- 172.21.15.133:0/3559901858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa2a80b64a0 con 0x7fa2a80cc7f0 2024-04-25T10:06:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.126+0000 7fa29effd640 1 -- 172.21.15.133:0/3559901858 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa2a40efc50 con 0x7fa2a80cc7f0 2024-04-25T10:06:33.131 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:06:33.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 -- 172.21.15.133:0/3559901858 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa28c066f60 msgr2=0x7fa28c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa28c066f60 0x7fa28c069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa2a81f5a80 tx=0x7fa298009040 comp rx=0 tx=0).stop 2024-04-25T10:06:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 -- 172.21.15.133:0/3559901858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a80cc7f0 msgr2=0x7fa2a81512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a80cc7f0 0x7fa2a81512b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40097c0 tx=0x7fa2a401ace0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 -- 172.21.15.133:0/3559901858 shutdown_connections 2024-04-25T10:06:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa2a815aa50 0x7fa2a80c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2a80ccd00 0x7fa2a80b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa28c066f60 0x7fa28c069420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 --2- 172.21.15.133:0/3559901858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a80cc7f0 0x7fa2a81512b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 -- 172.21.15.133:0/3559901858 >> 172.21.15.133:0/3559901858 conn(0x7fa2a80999d0 msgr2=0x7fa2a80b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 -- 172.21.15.133:0/3559901858 shutdown_connections 2024-04-25T10:06:33.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.130+0000 7fa2b065a640 1 -- 172.21.15.133:0/3559901858 wait complete. 2024-04-25T10:06:33.154 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:06:33.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.270+0000 7f9a6598c640 1 Processor -- start 2024-04-25T10:06:33.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a6598c640 1 -- start start 2024-04-25T10:06:33.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a6598c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9a600b1f70 0x7f9a6006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a6598c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a600ce330 0x7f9a600ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a6598c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a600c61d0 con 0x7f9a6015c9a0 2024-04-25T10:06:33.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a6598c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a600cecc0 con 0x7f9a600b1f70 2024-04-25T10:06:33.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a6598c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a600c64b0 con 0x7f9a600ce330 2024-04-25T10:06:33.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a5e7fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9a600b1f70 0x7f9a6006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a5f7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a600ce330 0x7f9a600ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a5effd640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9a6015c9a0 0x7f9a600c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:54690/0 (socket says 172.21.15.133:54690) 2024-04-25T10:06:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a5effd640 1 -- 172.21.15.133:0/3794898607 learned_addr learned my addr 172.21.15.133:0/3794898607 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a5e7fc640 1 -- 172.21.15.133:0/3794898607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a600ce330 msgr2=0x7f9a600ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a5e7fc640 1 --2- 172.21.15.133:0/3794898607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a600ce330 0x7f9a600ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a5e7fc640 1 -- 172.21.15.133:0/3794898607 >> v1:172.21.15.162:6789/0 conn(0x7f9a6015c9a0 legacy=0x7f9a600c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a5e7fc640 1 -- 172.21.15.133:0/3794898607 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6006d4f0 con 0x7f9a600b1f70 2024-04-25T10:06:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.274+0000 7f9a5f7fe640 1 --2- 172.21.15.133:0/3794898607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a600ce330 0x7f9a600ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:33.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a5e7fc640 1 --2- 172.21.15.133:0/3794898607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a600b1f70 0x7f9a6006dcf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9a540097c0 tx=0x7f9a5401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f32e301cee16f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a5dffb640 1 -- 172.21.15.133:0/3794898607 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a540aa5f0 con 0x7f9a600b1f70 2024-04-25T10:06:33.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a5dffb640 1 -- 172.21.15.133:0/3794898607 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a540af070 con 0x7f9a600b1f70 2024-04-25T10:06:33.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a5dffb640 1 -- 172.21.15.133:0/3794898607 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a540aa9b0 con 0x7f9a600b1f70 2024-04-25T10:06:33.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a6598c640 1 -- 172.21.15.133:0/3794898607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a600b1f70 msgr2=0x7f9a6006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a6598c640 1 --2- 172.21.15.133:0/3794898607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a600b1f70 0x7f9a6006dcf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9a540097c0 tx=0x7f9a5401c470 comp rx=0 tx=0).stop 2024-04-25T10:06:33.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a6598c640 1 -- 172.21.15.133:0/3794898607 shutdown_connections 2024-04-25T10:06:33.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a6598c640 1 --2- 172.21.15.133:0/3794898607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a600ce330 0x7f9a600ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a6598c640 1 --2- 172.21.15.133:0/3794898607 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a600b1f70 0x7f9a6006dcf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a6598c640 1 -- 172.21.15.133:0/3794898607 >> 172.21.15.133:0/3794898607 conn(0x7f9a60067000 msgr2=0x7f9a6006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a6598c640 1 -- 172.21.15.133:0/3794898607 shutdown_connections 2024-04-25T10:06:33.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a6598c640 1 -- 172.21.15.133:0/3794898607 wait complete. 2024-04-25T10:06:33.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a6598c640 1 Processor -- start 2024-04-25T10:06:33.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.278+0000 7f9a6598c640 1 -- start start 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a6598c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a600b1f70 0x7f9a6015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a6598c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a600ce330 0x7f9a6014e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a6598c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a6015c9a0 0x7f9a60150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a6598c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6015d950 con 0x7f9a600b1f70 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a6598c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6015dad0 con 0x7f9a6015c9a0 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a6598c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6015dc50 con 0x7f9a600ce330 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a600b1f70 0x7f9a6015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a600b1f70 0x7f9a6015bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45776/0 (socket says 172.21.15.133:45776) 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5effd640 1 -- 172.21.15.133:0/179668264 learned_addr learned my addr 172.21.15.133:0/179668264 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5f7fe640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a6015c9a0 0x7f9a60150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5e7fc640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a600ce330 0x7f9a6014e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5effd640 1 -- 172.21.15.133:0/179668264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a600ce330 msgr2=0x7f9a6014e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5effd640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a600ce330 0x7f9a6014e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5effd640 1 -- 172.21.15.133:0/179668264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a6015c9a0 msgr2=0x7f9a60150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5effd640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a6015c9a0 0x7f9a60150080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5effd640 1 -- 172.21.15.133:0/179668264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6014ed00 con 0x7f9a600b1f70 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5effd640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a600b1f70 0x7f9a6015bdb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000c450 tx=0x7f9a5000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a47fff640 1 -- 172.21.15.133:0/179668264 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50017070 con 0x7f9a600b1f70 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a47fff640 1 -- 172.21.15.133:0/179668264 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a50005cf0 con 0x7f9a600b1f70 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a6598c640 1 -- 172.21.15.133:0/179668264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6014efe0 con 0x7f9a600b1f70 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a47fff640 1 -- 172.21.15.133:0/179668264 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50007290 con 0x7f9a600b1f70 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a6598c640 1 -- 172.21.15.133:0/179668264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6014f5a0 con 0x7f9a600b1f70 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a6598c640 1 -- 172.21.15.133:0/179668264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6006bc20 con 0x7f9a600b1f70 2024-04-25T10:06:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a47fff640 1 -- 172.21.15.133:0/179668264 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a5001a070 con 0x7f9a600b1f70 2024-04-25T10:06:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a47fff640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a4c066f90 0x7f9a4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a47fff640 1 -- 172.21.15.133:0/179668264 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9a50084160 con 0x7f9a600b1f70 2024-04-25T10:06:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5e7fc640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a4c066f90 0x7f9a4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.282+0000 7f9a5e7fc640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a4c066f90 0x7f9a4c069450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9a540ae000 tx=0x7f9a540a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.286+0000 7f9a47fff640 1 -- 172.21.15.133:0/179668264 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a50055330 con 0x7f9a600b1f70 2024-04-25T10:06:33.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.498+0000 7f9a6598c640 1 -- 172.21.15.133:0/179668264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9a60068ef0 con 0x7f9a600b1f70 2024-04-25T10:06:33.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.498+0000 7f9a47fff640 1 -- 172.21.15.133:0/179668264 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9a50054c30 con 0x7f9a600b1f70 2024-04-25T10:06:33.504 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:33.504 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:33.504 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:06:33.504 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.504 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.505 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:06:33.506 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.507 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:06:33.508 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:06:33.509 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:33.511 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.512 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:06:33.513 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:06:33.514 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:06:33.515 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:06:33.516 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:33.517 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:06:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:06:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:06:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:06:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:06:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:33.518 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:06:33.518 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 -- 172.21.15.133:0/179668264 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a4c066f90 msgr2=0x7f9a4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a4c066f90 0x7f9a4c069450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9a540ae000 tx=0x7f9a540a7000 comp rx=0 tx=0).stop 2024-04-25T10:06:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 -- 172.21.15.133:0/179668264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a600b1f70 msgr2=0x7f9a6015bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a600b1f70 0x7f9a6015bdb0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000c450 tx=0x7f9a5000c910 comp rx=0 tx=0).stop 2024-04-25T10:06:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 -- 172.21.15.133:0/179668264 shutdown_connections 2024-04-25T10:06:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a6015c9a0 0x7f9a60150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a600ce330 0x7f9a6014e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a4c066f90 0x7f9a4c069450 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 --2- 172.21.15.133:0/179668264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a600b1f70 0x7f9a6015bdb0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 -- 172.21.15.133:0/179668264 >> 172.21.15.133:0/179668264 conn(0x7f9a60067000 msgr2=0x7f9a601514e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 -- 172.21.15.133:0/179668264 shutdown_connections 2024-04-25T10:06:33.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.502+0000 7f9a6598c640 1 -- 172.21.15.133:0/179668264 wait complete. 2024-04-25T10:06:33.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:33.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b730b640 1 Processor -- start 2024-04-25T10:06:33.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b730b640 1 -- start start 2024-04-25T10:06:33.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b730b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f99b006eb70 0x7f99b00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b730b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b01584f0 0x7f99b015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b730b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b00cbe60 con 0x7f99b0158110 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b730b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b00b19b0 con 0x7f99b006eb70 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b730b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b015b360 con 0x7f99b01584f0 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b50a7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f99b0158110 0x7f99b006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50468/0 (socket says 172.21.15.133:50468) 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b50a7640 1 -- 172.21.15.133:0/3416378123 learned_addr learned my addr 172.21.15.133:0/3416378123 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b48a6640 1 --2- 172.21.15.133:0/3416378123 >> v2:172.21.15.162:3300/0 conn(0x7f99b006eb70 0x7f99b00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b58a8640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b01584f0 0x7f99b015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650059984 0 0) 0x7f99b00cbe60 con 0x7f99b0158110 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f999c005740 con 0x7f99b0158110 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1165079017 0 0) 0x7f999c005740 con 0x7f99b0158110 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b01584f0 msgr2=0x7f99b015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b01584f0 0x7f99b015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 >> v2:172.21.15.162:3300/0 conn(0x7f99b006eb70 msgr2=0x7f99b00b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 --2- 172.21.15.133:0/3416378123 >> v2:172.21.15.162:3300/0 conn(0x7f99b006eb70 0x7f99b00b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b015b640 con 0x7f99b0158110 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f99a4001680 con 0x7f99b0158110 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 >> v1:172.21.15.133:6789/0 conn(0x7f99b0158110 legacy=0x7f99b006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f999c005bd0 0x7f999c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b01584f0 0x7f999c006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f999c006f60 0x7f999c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00b19b0 con 0x7f999c006f60 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b015b360 con 0x7f999c005bd0 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999c00dbe0 con 0x7f99b01584f0 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.638+0000 7f99b58a8640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f999c006f60 0x7f999c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b50a7640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f999c005bd0 0x7f999c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b48a6640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b01584f0 0x7f999c006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b58a8640 1 -- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b01584f0 msgr2=0x7f999c006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b58a8640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b01584f0 0x7f999c006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b58a8640 1 -- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f999c005bd0 msgr2=0x7f999c005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b58a8640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f999c005bd0 0x7f999c005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b58a8640 1 -- 172.21.15.133:0/3416378123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c005740 con 0x7f999c006f60 2024-04-25T10:06:33.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b50a7640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f999c005bd0 0x7f999c005ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:33.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b58a8640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f999c006f60 0x7f999c0073e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009f90 tx=0x7f99a801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd4f71e90082171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a80b2070 con 0x7f999c006f60 2024-04-25T10:06:33.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a80a5c50 con 0x7f999c006f60 2024-04-25T10:06:33.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99a3fff640 1 -- 172.21.15.133:0/3416378123 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8009c10 con 0x7f999c006f60 2024-04-25T10:06:33.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b730b640 1 -- 172.21.15.133:0/3416378123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f999c006f60 msgr2=0x7f999c0073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b730b640 1 --2- 172.21.15.133:0/3416378123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f999c006f60 0x7f999c0073e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009f90 tx=0x7f99a801cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:33.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b58a8640 1 -- 172.21.15.133:0/3416378123 reap_dead start 2024-04-25T10:06:33.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b730b640 1 -- 172.21.15.133:0/3416378123 shutdown_connections 2024-04-25T10:06:33.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b730b640 1 -- 172.21.15.133:0/3416378123 >> 172.21.15.133:0/3416378123 conn(0x7f99b0067000 msgr2=0x7f99b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b730b640 1 -- 172.21.15.133:0/3416378123 shutdown_connections 2024-04-25T10:06:33.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.642+0000 7f99b730b640 1 -- 172.21.15.133:0/3416378123 wait complete. 2024-04-25T10:06:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b730b640 1 Processor -- start 2024-04-25T10:06:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b730b640 1 -- start start 2024-04-25T10:06:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b730b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b0146ee0 0x7f99b015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b730b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99b01472c0 0x7f99b00b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b730b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b00b3e20 0x7f99b0159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b730b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00cd670 con 0x7f99b01472c0 2024-04-25T10:06:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b730b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b015bc00 con 0x7f99b0146ee0 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b730b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00cd7f0 con 0x7f99b00b3e20 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b48a6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99b01472c0 0x7f99b00b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b48a6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99b01472c0 0x7f99b00b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45802/0 (socket says 172.21.15.133:45802) 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b48a6640 1 -- 172.21.15.133:0/2319799040 learned_addr learned my addr 172.21.15.133:0/2319799040 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b50a7640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b0146ee0 0x7f99b015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b58a8640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b00b3e20 0x7f99b0159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b48a6640 1 -- 172.21.15.133:0/2319799040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b00b3e20 msgr2=0x7f99b0159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b48a6640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b00b3e20 0x7f99b0159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b48a6640 1 -- 172.21.15.133:0/2319799040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b0146ee0 msgr2=0x7f99b015cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b48a6640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b0146ee0 0x7f99b015cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b48a6640 1 -- 172.21.15.133:0/2319799040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b015bf20 con 0x7f99b01472c0 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b50a7640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b0146ee0 0x7f99b015cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b48a6640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99b01472c0 0x7f99b00b38e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f99ac011600 tx=0x7f99ac011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99a1ffb640 1 -- 172.21.15.133:0/2319799040 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac004220 con 0x7f99b01472c0 2024-04-25T10:06:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99a1ffb640 1 -- 172.21.15.133:0/2319799040 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ac0043c0 con 0x7f99b01472c0 2024-04-25T10:06:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b730b640 1 -- 172.21.15.133:0/2319799040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b015c200 con 0x7f99b01472c0 2024-04-25T10:06:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99a1ffb640 1 -- 172.21.15.133:0/2319799040 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac00f960 con 0x7f99b01472c0 2024-04-25T10:06:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b730b640 1 -- 172.21.15.133:0/2319799040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b015c7c0 con 0x7f99b01472c0 2024-04-25T10:06:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.646+0000 7f99b730b640 1 -- 172.21.15.133:0/2319799040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b0068ef0 con 0x7f99b01472c0 2024-04-25T10:06:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.650+0000 7f99a1ffb640 1 -- 172.21.15.133:0/2319799040 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99ac00fb00 con 0x7f99b01472c0 2024-04-25T10:06:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.650+0000 7f99a1ffb640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9994066f60 0x7f9994069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.650+0000 7f99a1ffb640 1 -- 172.21.15.133:0/2319799040 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f99ac085550 con 0x7f99b01472c0 2024-04-25T10:06:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.650+0000 7f99b50a7640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9994066f60 0x7f9994069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.650+0000 7f99b50a7640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9994066f60 0x7f9994069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f99a4001ec0 tx=0x7f99a40047f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.654+0000 7f99a1ffb640 1 -- 172.21.15.133:0/2319799040 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99ac019030 con 0x7f99b01472c0 2024-04-25T10:06:33.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.802+0000 7f99b730b640 1 -- 172.21.15.133:0/2319799040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99b006da40 con 0x7f99b01472c0 2024-04-25T10:06:33.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.802+0000 7f99a1ffb640 1 -- 172.21.15.133:0/2319799040 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+13188 (secure 0 0 0) 0x7f99ac056720 con 0x7f99b01472c0 2024-04-25T10:06:33.808 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:33.808 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":62,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:06:33.125165+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 -- 172.21.15.133:0/2319799040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9994066f60 msgr2=0x7f9994069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9994066f60 0x7f9994069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f99a4001ec0 tx=0x7f99a40047f0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 -- 172.21.15.133:0/2319799040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99b01472c0 msgr2=0x7f99b00b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99b01472c0 0x7f99b00b38e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f99ac011600 tx=0x7f99ac011ac0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 -- 172.21.15.133:0/2319799040 shutdown_connections 2024-04-25T10:06:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99b00b3e20 0x7f99b0159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9994066f60 0x7f9994069420 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99b01472c0 0x7f99b00b38e0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 --2- 172.21.15.133:0/2319799040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99b0146ee0 0x7f99b015cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 -- 172.21.15.133:0/2319799040 >> 172.21.15.133:0/2319799040 conn(0x7f99b0067000 msgr2=0x7f99b00cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 -- 172.21.15.133:0/2319799040 shutdown_connections 2024-04-25T10:06:33.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.806+0000 7f99b730b640 1 -- 172.21.15.133:0/2319799040 wait complete. 2024-04-25T10:06:33.825 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:33.825 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.942+0000 7ff2b6ff5640 1 Processor -- start 2024-04-25T10:06:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.942+0000 7ff2b6ff5640 1 -- start start 2024-04-25T10:06:33.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b6ff5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff2b0142920 0x7ff2b0143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b6ff5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2b0140a40 0x7ff2b00ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b6ff5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b00cf000 con 0x7ff2b015d9f0 2024-04-25T10:06:33.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b6ff5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b0140ea0 con 0x7ff2b0142920 2024-04-25T10:06:33.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b6ff5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00cf280 con 0x7ff2b0140a40 2024-04-25T10:06:33.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b4d91640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff2b0142920 0x7ff2b0143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b5592640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff2b015d9f0 0x7ff2b006df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50472/0 (socket says 172.21.15.133:50472) 2024-04-25T10:06:33.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b5592640 1 -- 172.21.15.133:0/699943663 learned_addr learned my addr 172.21.15.133:0/699943663 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:33.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2affff640 1 --2- 172.21.15.133:0/699943663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2b0140a40 0x7ff2b00ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b4d91640 1 -- 172.21.15.133:0/699943663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2b0140a40 msgr2=0x7ff2b00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b4d91640 1 --2- 172.21.15.133:0/699943663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2b0140a40 0x7ff2b00ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b4d91640 1 -- 172.21.15.133:0/699943663 >> v1:172.21.15.133:6789/0 conn(0x7ff2b015d9f0 legacy=0x7ff2b006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2af7fe640 1 -- 172.21.15.133:0/699943663 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446674642 0 0) 0x7ff2a80014a0 con 0x7ff2b015d9f0 2024-04-25T10:06:33.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b4d91640 1 -- 172.21.15.133:0/699943663 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b006d440 con 0x7ff2b0142920 2024-04-25T10:06:33.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2b4d91640 1 --2- 172.21.15.133:0/699943663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b0142920 0x7ff2b0143560 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00097c0 tx=0x7ff2a001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1de2f7c1c89f813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2af7fe640 1 -- 172.21.15.133:0/699943663 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00b0070 con 0x7ff2b0142920 2024-04-25T10:06:33.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2af7fe640 1 -- 172.21.15.133:0/699943663 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00ab600 con 0x7ff2b0142920 2024-04-25T10:06:33.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.946+0000 7ff2af7fe640 1 -- 172.21.15.133:0/699943663 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00ab9d0 con 0x7ff2b0142920 2024-04-25T10:06:33.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/699943663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b0142920 msgr2=0x7ff2b0143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 --2- 172.21.15.133:0/699943663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b0142920 0x7ff2b0143560 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00097c0 tx=0x7ff2a001c410 comp rx=0 tx=0).stop 2024-04-25T10:06:33.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/699943663 shutdown_connections 2024-04-25T10:06:33.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 --2- 172.21.15.133:0/699943663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2b0140a40 0x7ff2b00ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 --2- 172.21.15.133:0/699943663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b0142920 0x7ff2b0143560 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/699943663 >> 172.21.15.133:0/699943663 conn(0x7ff2b0066f70 msgr2=0x7ff2b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/699943663 shutdown_connections 2024-04-25T10:06:33.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/699943663 wait complete. 2024-04-25T10:06:33.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 Processor -- start 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 -- start start 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2b0140a40 0x7ff2b013f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b0142920 0x7ff2b015bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b015d9f0 0x7ff2b0154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b006ebc0 con 0x7ff2b0142920 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b006ea40 con 0x7ff2b015d9f0 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0142d00 con 0x7ff2b0140a40 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2affff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b0142920 0x7ff2b015bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2affff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b0142920 0x7ff2b015bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45820/0 (socket says 172.21.15.133:45820) 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2affff640 1 -- 172.21.15.133:0/3624583499 learned_addr learned my addr 172.21.15.133:0/3624583499 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2affff640 1 -- 172.21.15.133:0/3624583499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2b0140a40 msgr2=0x7ff2b013f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2affff640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2b0140a40 0x7ff2b013f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b5592640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b015d9f0 0x7ff2b0154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2affff640 1 -- 172.21.15.133:0/3624583499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b015d9f0 msgr2=0x7ff2b0154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2affff640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b015d9f0 0x7ff2b0154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2affff640 1 -- 172.21.15.133:0/3624583499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00cb9f0 con 0x7ff2b0142920 2024-04-25T10:06:33.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2affff640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b0142920 0x7ff2b015bfc0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400d830 tx=0x7ff2a400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2ad7fa640 1 -- 172.21.15.133:0/3624583499 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400b6c0 con 0x7ff2b0142920 2024-04-25T10:06:33.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2ad7fa640 1 -- 172.21.15.133:0/3624583499 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a4019070 con 0x7ff2b0142920 2024-04-25T10:06:33.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/3624583499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b015ca40 con 0x7ff2b0142920 2024-04-25T10:06:33.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2ad7fa640 1 -- 172.21.15.133:0/3624583499 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400ba60 con 0x7ff2b0142920 2024-04-25T10:06:33.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.950+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/3624583499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b013fda0 con 0x7ff2b0142920 2024-04-25T10:06:33.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.954+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/3624583499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b0068e80 con 0x7ff2b0142920 2024-04-25T10:06:33.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.954+0000 7ff2ad7fa640 1 -- 172.21.15.133:0/3624583499 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2a401e460 con 0x7ff2b0142920 2024-04-25T10:06:33.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.954+0000 7ff2ad7fa640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff29c066f60 0x7ff29c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.954+0000 7ff2ad7fa640 1 -- 172.21.15.133:0/3624583499 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff2a4085550 con 0x7ff2b0142920 2024-04-25T10:06:33.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.954+0000 7ff2b4d91640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff29c066f60 0x7ff29c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.954+0000 7ff2b4d91640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff29c066f60 0x7ff29c069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00ad000 tx=0x7ff2a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:33.958+0000 7ff2ad7fa640 1 -- 172.21.15.133:0/3624583499 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2a4056720 con 0x7ff2b0142920 2024-04-25T10:06:34.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.110+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/3624583499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2b00cbd40 con 0x7ff2b0142920 2024-04-25T10:06:34.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.110+0000 7ff2ad7fa640 1 -- 172.21.15.133:0/3624583499 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2a4056020 con 0x7ff2b0142920 2024-04-25T10:06:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:06:34.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/3624583499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff29c066f60 msgr2=0x7ff29c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff29c066f60 0x7ff29c069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00ad000 tx=0x7ff2a0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:34.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/3624583499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b0142920 msgr2=0x7ff2b015bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b0142920 0x7ff2b015bfc0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400d830 tx=0x7ff2a400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/3624583499 shutdown_connections 2024-04-25T10:06:34.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2b015d9f0 0x7ff2b0154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff29c066f60 0x7ff29c069420 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b0142920 0x7ff2b015bfc0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 --2- 172.21.15.133:0/3624583499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2b0140a40 0x7ff2b013f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/3624583499 >> 172.21.15.133:0/3624583499 conn(0x7ff2b0066f70 msgr2=0x7ff2b00b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/3624583499 shutdown_connections 2024-04-25T10:06:34.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.114+0000 7ff2b6ff5640 1 -- 172.21.15.133:0/3624583499 wait complete. 2024-04-25T10:06:34.133 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:06:34.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.246+0000 7f96d8775640 1 Processor -- start 2024-04-25T10:06:34.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d8775640 1 -- start start 2024-04-25T10:06:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d8775640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96d015c9a0 0x7f96d00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d8775640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f96d00ce330 0x7f96d00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d8775640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d00c61d0 con 0x7f96d00b1f70 2024-04-25T10:06:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d8775640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d00cecc0 con 0x7f96d015c9a0 2024-04-25T10:06:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d8775640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d00c64b0 con 0x7f96d00ce330 2024-04-25T10:06:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d6511640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96d015c9a0 0x7f96d00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d5d10640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f96d00b1f70 0x7f96d006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50486/0 (socket says 172.21.15.133:50486) 2024-04-25T10:06:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d5d10640 1 -- 172.21.15.133:0/2984597933 learned_addr learned my addr 172.21.15.133:0/2984597933 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d6d12640 1 --2- 172.21.15.133:0/2984597933 >> v2:172.21.15.162:3300/0 conn(0x7f96d00ce330 0x7f96d00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d6511640 1 -- 172.21.15.133:0/2984597933 >> v2:172.21.15.162:3300/0 conn(0x7f96d00ce330 msgr2=0x7f96d00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d6511640 1 --2- 172.21.15.133:0/2984597933 >> v2:172.21.15.162:3300/0 conn(0x7f96d00ce330 0x7f96d00ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d6511640 1 -- 172.21.15.133:0/2984597933 >> v1:172.21.15.133:6789/0 conn(0x7f96d00b1f70 legacy=0x7f96d006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d550f640 1 -- 172.21.15.133:0/2984597933 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270094362 0 0) 0x7f96d00c61d0 con 0x7f96d00b1f70 2024-04-25T10:06:34.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d6511640 1 -- 172.21.15.133:0/2984597933 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d006d4f0 con 0x7f96d015c9a0 2024-04-25T10:06:34.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.250+0000 7f96d6d12640 1 --2- 172.21.15.133:0/2984597933 >> v2:172.21.15.162:3300/0 conn(0x7f96d00ce330 0x7f96d00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:34.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d6511640 1 --2- 172.21.15.133:0/2984597933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96d015c9a0 0x7f96d00c5980 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f96d00cc0f0 tx=0x7f96cc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8104928ca2eb69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d550f640 1 -- 172.21.15.133:0/2984597933 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0b0070 con 0x7f96d015c9a0 2024-04-25T10:06:34.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d550f640 1 -- 172.21.15.133:0/2984597933 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc0ab600 con 0x7f96d015c9a0 2024-04-25T10:06:34.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d550f640 1 -- 172.21.15.133:0/2984597933 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0ab9d0 con 0x7f96d015c9a0 2024-04-25T10:06:34.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 -- 172.21.15.133:0/2984597933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96d015c9a0 msgr2=0x7f96d00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 --2- 172.21.15.133:0/2984597933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96d015c9a0 0x7f96d00c5980 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f96d00cc0f0 tx=0x7f96cc01c480 comp rx=0 tx=0).stop 2024-04-25T10:06:34.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 -- 172.21.15.133:0/2984597933 shutdown_connections 2024-04-25T10:06:34.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 --2- 172.21.15.133:0/2984597933 >> v2:172.21.15.162:3300/0 conn(0x7f96d00ce330 0x7f96d00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 --2- 172.21.15.133:0/2984597933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96d015c9a0 0x7f96d00c5980 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 -- 172.21.15.133:0/2984597933 >> 172.21.15.133:0/2984597933 conn(0x7f96d0067000 msgr2=0x7f96d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 -- 172.21.15.133:0/2984597933 shutdown_connections 2024-04-25T10:06:34.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 -- 172.21.15.133:0/2984597933 wait complete. 2024-04-25T10:06:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 Processor -- start 2024-04-25T10:06:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 -- start start 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96d00b1f70 0x7f96d0149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d00ce330 0x7f96d015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96d015c9a0 0x7f96d015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d015dad0 con 0x7f96d00b1f70 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d015d950 con 0x7f96d00ce330 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d8775640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d015dc50 con 0x7f96d015c9a0 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d6511640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96d00b1f70 0x7f96d0149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d6511640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96d00b1f70 0x7f96d0149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45852/0 (socket says 172.21.15.133:45852) 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.254+0000 7f96d6511640 1 -- 172.21.15.133:0/2044788444 learned_addr learned my addr 172.21.15.133:0/2044788444 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96d6511640 1 -- 172.21.15.133:0/2044788444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96d015c9a0 msgr2=0x7f96d015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96d6511640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96d015c9a0 0x7f96d015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96d6511640 1 -- 172.21.15.133:0/2044788444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d00ce330 msgr2=0x7f96d015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96d6511640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d00ce330 0x7f96d015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96d6511640 1 -- 172.21.15.133:0/2044788444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d006e230 con 0x7f96d00b1f70 2024-04-25T10:06:34.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96d6511640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96d00b1f70 0x7f96d0149470 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0ae040 tx=0x7f96cc019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96c6ffd640 1 -- 172.21.15.133:0/2044788444 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0b0040 con 0x7f96d00b1f70 2024-04-25T10:06:34.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96c6ffd640 1 -- 172.21.15.133:0/2044788444 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc0bb070 con 0x7f96d00b1f70 2024-04-25T10:06:34.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96d8775640 1 -- 172.21.15.133:0/2044788444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d00c9390 con 0x7f96d00b1f70 2024-04-25T10:06:34.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96c6ffd640 1 -- 172.21.15.133:0/2044788444 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0b4850 con 0x7f96d00b1f70 2024-04-25T10:06:34.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96d8775640 1 -- 172.21.15.133:0/2044788444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d00c9950 con 0x7f96d00b1f70 2024-04-25T10:06:34.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96d8775640 1 -- 172.21.15.133:0/2044788444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d006bc20 con 0x7f96d00b1f70 2024-04-25T10:06:34.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96c6ffd640 1 -- 172.21.15.133:0/2044788444 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96cc0ab4c0 con 0x7f96d00b1f70 2024-04-25T10:06:34.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96c6ffd640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96b4066f60 0x7f96b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96c6ffd640 1 -- 172.21.15.133:0/2044788444 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f96cc11ec10 con 0x7f96d00b1f70 2024-04-25T10:06:34.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96d5d10640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96b4066f60 0x7f96b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.258+0000 7f96d5d10640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96b4066f60 0x7f96b4069420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f96d00cc9c0 tx=0x7f96c0007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.262+0000 7f96c6ffd640 1 -- 172.21.15.133:0/2044788444 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f96cc016070 con 0x7f96d00b1f70 2024-04-25T10:06:34.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.422+0000 7f96d8775640 1 -- 172.21.15.133:0/2044788444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f96d0068ef0 con 0x7f96d00b1f70 2024-04-25T10:06:34.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.422+0000 7f96c6ffd640 1 -- 172.21.15.133:0/2044788444 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+65 (secure 0 0 0) 0x7f96cc122020 con 0x7f96d00b1f70 2024-04-25T10:06:34.427 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:34.427 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:06:34.427 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:06:34.427 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:06:34.427 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:06:34.427 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:34.427 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 -- 172.21.15.133:0/2044788444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96b4066f60 msgr2=0x7f96b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96b4066f60 0x7f96b4069420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f96d00cc9c0 tx=0x7f96c0007970 comp rx=0 tx=0).stop 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 -- 172.21.15.133:0/2044788444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96d00b1f70 msgr2=0x7f96d0149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96d00b1f70 0x7f96d0149470 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0ae040 tx=0x7f96cc019220 comp rx=0 tx=0).stop 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 -- 172.21.15.133:0/2044788444 shutdown_connections 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96d015c9a0 0x7f96d015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96d00ce330 0x7f96d015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96b4066f60 0x7f96b4069420 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 --2- 172.21.15.133:0/2044788444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96d00b1f70 0x7f96d0149470 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 -- 172.21.15.133:0/2044788444 >> 172.21.15.133:0/2044788444 conn(0x7f96d0067000 msgr2=0x7f96d0153500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 -- 172.21.15.133:0/2044788444 shutdown_connections 2024-04-25T10:06:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.426+0000 7f96d8775640 1 -- 172.21.15.133:0/2044788444 wait complete. 2024-04-25T10:06:34.446 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:06:34.446 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:06:34.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.562+0000 7f3e2f356640 1 Processor -- start 2024-04-25T10:06:34.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.562+0000 7f3e2f356640 1 -- start start 2024-04-25T10:06:34.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3e2815b530 0x7f3e2815b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e280cc890 0x7f3e280ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e2815bf40 con 0x7f3e280ce690 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e281589c0 con 0x7f3e2815b530 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2815c200 con 0x7f3e280cc890 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2d0f2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3e2815b530 0x7f3e2815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2d0f2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3e2815b530 0x7f3e2815b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45856/0 (socket says 172.21.15.133:45856) 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2d0f2640 1 -- 172.21.15.133:0/190829704 learned_addr learned my addr 172.21.15.133:0/190829704 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2c8f1640 1 --2- 172.21.15.133:0/190829704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e280cc890 0x7f3e280ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2d0f2640 1 -- 172.21.15.133:0/190829704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e280cc890 msgr2=0x7f3e280ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2d0f2640 1 --2- 172.21.15.133:0/190829704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e280cc890 0x7f3e280ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2d0f2640 1 -- 172.21.15.133:0/190829704 >> v1:172.21.15.162:6789/0 conn(0x7f3e280ce690 legacy=0x7f3e280ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2d0f2640 1 -- 172.21.15.133:0/190829704 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2815c4e0 con 0x7f3e2815b530 2024-04-25T10:06:34.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2c8f1640 1 --2- 172.21.15.133:0/190829704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e280cc890 0x7f3e280ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:34.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2d0f2640 1 --2- 172.21.15.133:0/190829704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e2815b530 0x7f3e2815b910 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009870 tx=0x7f3e2401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b70d2f02a8e68d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e1ffff640 1 -- 172.21.15.133:0/190829704 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e240b2070 con 0x7f3e2815b530 2024-04-25T10:06:34.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e1ffff640 1 -- 172.21.15.133:0/190829704 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e240a9070 con 0x7f3e2815b530 2024-04-25T10:06:34.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e1ffff640 1 -- 172.21.15.133:0/190829704 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e240ad880 con 0x7f3e2815b530 2024-04-25T10:06:34.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 -- 172.21.15.133:0/190829704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e2815b530 msgr2=0x7f3e2815b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 --2- 172.21.15.133:0/190829704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e2815b530 0x7f3e2815b910 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009870 tx=0x7f3e2401c490 comp rx=0 tx=0).stop 2024-04-25T10:06:34.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 -- 172.21.15.133:0/190829704 shutdown_connections 2024-04-25T10:06:34.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 --2- 172.21.15.133:0/190829704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e280cc890 0x7f3e280ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 --2- 172.21.15.133:0/190829704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e2815b530 0x7f3e2815b910 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 -- 172.21.15.133:0/190829704 >> 172.21.15.133:0/190829704 conn(0x7f3e2806d370 msgr2=0x7f3e2806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 -- 172.21.15.133:0/190829704 shutdown_connections 2024-04-25T10:06:34.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.566+0000 7f3e2f356640 1 -- 172.21.15.133:0/190829704 wait complete. 2024-04-25T10:06:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2f356640 1 Processor -- start 2024-04-25T10:06:34.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2f356640 1 -- start start 2024-04-25T10:06:34.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2f356640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e280cc890 0x7f3e280b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2f356640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e280ce690 0x7f3e28159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2f356640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e2815b530 0x7f3e2815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2f356640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2815dbe0 con 0x7f3e2815b530 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2f356640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2815da60 con 0x7f3e280cc890 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2f356640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2815dd60 con 0x7f3e280ce690 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2d8f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e2815b530 0x7f3e2815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2d8f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e2815b530 0x7f3e2815a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45866/0 (socket says 172.21.15.133:45866) 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2d8f3640 1 -- 172.21.15.133:0/1186325368 learned_addr learned my addr 172.21.15.133:0/1186325368 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2c8f1640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e280ce690 0x7f3e28159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2d0f2640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e280cc890 0x7f3e280b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2d8f3640 1 -- 172.21.15.133:0/1186325368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e280ce690 msgr2=0x7f3e28159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2d8f3640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e280ce690 0x7f3e28159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2d8f3640 1 -- 172.21.15.133:0/1186325368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e280cc890 msgr2=0x7f3e280b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2d8f3640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e280cc890 0x7f3e280b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2d8f3640 1 -- 172.21.15.133:0/1186325368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e280b4c80 con 0x7f3e2815b530 2024-04-25T10:06:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2d0f2640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e280cc890 0x7f3e280b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2d8f3640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e2815b530 0x7f3e2815a3f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000b970 tx=0x7f3e2000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e1dffb640 1 -- 172.21.15.133:0/1186325368 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20019070 con 0x7f3e2815b530 2024-04-25T10:06:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e1dffb640 1 -- 172.21.15.133:0/1186325368 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2000cb40 con 0x7f3e2815b530 2024-04-25T10:06:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2f356640 1 -- 172.21.15.133:0/1186325368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e280b4f60 con 0x7f3e2815b530 2024-04-25T10:06:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e1dffb640 1 -- 172.21.15.133:0/1186325368 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20014440 con 0x7f3e2815b530 2024-04-25T10:06:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2f356640 1 -- 172.21.15.133:0/1186325368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e280b5520 con 0x7f3e2815b530 2024-04-25T10:06:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e2f356640 1 -- 172.21.15.133:0/1186325368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e28069000 con 0x7f3e2815b530 2024-04-25T10:06:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.570+0000 7f3e1dffb640 1 -- 172.21.15.133:0/1186325368 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e2001c070 con 0x7f3e2815b530 2024-04-25T10:06:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.574+0000 7f3e1dffb640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e0c066f60 0x7f3e0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.574+0000 7f3e1dffb640 1 -- 172.21.15.133:0/1186325368 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3e20084a40 con 0x7f3e2815b530 2024-04-25T10:06:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.574+0000 7f3e2d0f2640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e0c066f60 0x7f3e0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.574+0000 7f3e2d0f2640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e0c066f60 0x7f3e0c069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3e240af000 tx=0x7f3e24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.578+0000 7f3e1dffb640 1 -- 172.21.15.133:0/1186325368 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e20055ca0 con 0x7f3e2815b530 2024-04-25T10:06:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:34.726+0000 7f3e2f356640 1 -- 172.21.15.133:0/1186325368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3e280af720 con 0x7f3e2815b530 2024-04-25T10:06:35.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.138+0000 7f3e1dffb640 1 -- 172.21.15.133:0/1186325368 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3e200555a0 con 0x7f3e2815b530 2024-04-25T10:06:35.141 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:06:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 -- 172.21.15.133:0/1186325368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e0c066f60 msgr2=0x7f3e0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e0c066f60 0x7f3e0c069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3e240af000 tx=0x7f3e24008040 comp rx=0 tx=0).stop 2024-04-25T10:06:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 -- 172.21.15.133:0/1186325368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e2815b530 msgr2=0x7f3e2815a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e2815b530 0x7f3e2815a3f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000b970 tx=0x7f3e2000be30 comp rx=0 tx=0).stop 2024-04-25T10:06:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 -- 172.21.15.133:0/1186325368 shutdown_connections 2024-04-25T10:06:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e0c066f60 0x7f3e0c069420 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e2815b530 0x7f3e2815a3f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e280ce690 0x7f3e28159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 --2- 172.21.15.133:0/1186325368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e280cc890 0x7f3e280b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 -- 172.21.15.133:0/1186325368 >> 172.21.15.133:0/1186325368 conn(0x7f3e2806d370 msgr2=0x7f3e28071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 -- 172.21.15.133:0/1186325368 shutdown_connections 2024-04-25T10:06:35.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.142+0000 7f3e2f356640 1 -- 172.21.15.133:0/1186325368 wait complete. 2024-04-25T10:06:35.172 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:06:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.298+0000 7fb626a4c640 1 Processor -- start 2024-04-25T10:06:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.298+0000 7fb626a4c640 1 -- start start 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb626a4c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6200cc090 0x7fb6200b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb626a4c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6200ce250 0x7fb62006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb626a4c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62006b1d0 con 0x7fb6200b14d0 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb626a4c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62006b4a0 con 0x7fb6200ce250 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb626a4c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6200affa0 con 0x7fb6200cc090 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb61f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6200ce250 0x7fb62006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb624fe9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb6200b14d0 0x7fb6200b1900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50490/0 (socket says 172.21.15.133:50490) 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb624fe9640 1 -- 172.21.15.133:0/2224447887 learned_addr learned my addr 172.21.15.133:0/2224447887 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb61ffff640 1 --2- 172.21.15.133:0/2224447887 >> v2:172.21.15.162:3300/0 conn(0x7fb6200cc090 0x7fb6200b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb61f7fe640 1 -- 172.21.15.133:0/2224447887 >> v2:172.21.15.162:3300/0 conn(0x7fb6200cc090 msgr2=0x7fb6200b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb61f7fe640 1 --2- 172.21.15.133:0/2224447887 >> v2:172.21.15.162:3300/0 conn(0x7fb6200cc090 0x7fb6200b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb61effd640 1 -- 172.21.15.133:0/2224447887 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89814273 0 0) 0x7fb62006b1d0 con 0x7fb6200b14d0 2024-04-25T10:06:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb61f7fe640 1 -- 172.21.15.133:0/2224447887 >> v1:172.21.15.133:6789/0 conn(0x7fb6200b14d0 legacy=0x7fb6200b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb61f7fe640 1 -- 172.21.15.133:0/2224447887 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6200b0280 con 0x7fb6200ce250 2024-04-25T10:06:35.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb61ffff640 1 --2- 172.21.15.133:0/2224447887 >> v2:172.21.15.162:3300/0 conn(0x7fb6200cc090 0x7fb6200b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:35.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.302+0000 7fb61f7fe640 1 --2- 172.21.15.133:0/2224447887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6200ce250 0x7fb62006ac90 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb620070a20 tx=0x7fb61401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5750a1f3819c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61effd640 1 -- 172.21.15.133:0/2224447887 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6140ab5f0 con 0x7fb6200ce250 2024-04-25T10:06:35.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61effd640 1 -- 172.21.15.133:0/2224447887 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6140b0070 con 0x7fb6200ce250 2024-04-25T10:06:35.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61effd640 1 -- 172.21.15.133:0/2224447887 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6140ab9b0 con 0x7fb6200ce250 2024-04-25T10:06:35.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 -- 172.21.15.133:0/2224447887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6200ce250 msgr2=0x7fb62006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 --2- 172.21.15.133:0/2224447887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6200ce250 0x7fb62006ac90 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb620070a20 tx=0x7fb61401c450 comp rx=0 tx=0).stop 2024-04-25T10:06:35.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 -- 172.21.15.133:0/2224447887 shutdown_connections 2024-04-25T10:06:35.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 --2- 172.21.15.133:0/2224447887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6200ce250 0x7fb62006ac90 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 --2- 172.21.15.133:0/2224447887 >> v2:172.21.15.162:3300/0 conn(0x7fb6200cc090 0x7fb6200b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 -- 172.21.15.133:0/2224447887 >> 172.21.15.133:0/2224447887 conn(0x7fb6200999d0 msgr2=0x7fb6200d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 -- 172.21.15.133:0/2224447887 shutdown_connections 2024-04-25T10:06:35.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 -- 172.21.15.133:0/2224447887 wait complete. 2024-04-25T10:06:35.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 Processor -- start 2024-04-25T10:06:35.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 -- start start 2024-04-25T10:06:35.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6200b14d0 0x7fb620156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6200cc090 0x7fb620154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6200ce250 0x7fb62015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200cd2a0 con 0x7fb6200b14d0 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200cd120 con 0x7fb6200cc090 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb626a4c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200cd420 con 0x7fb6200ce250 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6200b14d0 0x7fb620156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6200b14d0 0x7fb620156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45882/0 (socket says 172.21.15.133:45882) 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61ffff640 1 -- 172.21.15.133:0/1780401241 learned_addr learned my addr 172.21.15.133:0/1780401241 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61f7fe640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6200cc090 0x7fb620154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb624fe9640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6200ce250 0x7fb62015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61ffff640 1 -- 172.21.15.133:0/1780401241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6200ce250 msgr2=0x7fb62015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61ffff640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6200ce250 0x7fb62015a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61ffff640 1 -- 172.21.15.133:0/1780401241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6200cc090 msgr2=0x7fb620154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61ffff640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6200cc090 0x7fb620154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61ffff640 1 -- 172.21.15.133:0/1780401241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6200c4240 con 0x7fb6200b14d0 2024-04-25T10:06:35.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61ffff640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6200b14d0 0x7fb620156ae0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb61000c7c0 tx=0x7fb61000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61cff9640 1 -- 172.21.15.133:0/1780401241 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610007a30 con 0x7fb6200b14d0 2024-04-25T10:06:35.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61cff9640 1 -- 172.21.15.133:0/1780401241 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb610007bd0 con 0x7fb6200b14d0 2024-04-25T10:06:35.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.306+0000 7fb61cff9640 1 -- 172.21.15.133:0/1780401241 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6100113f0 con 0x7fb6200b14d0 2024-04-25T10:06:35.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.310+0000 7fb626a4c640 1 -- 172.21.15.133:0/1780401241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6200c9e80 con 0x7fb6200b14d0 2024-04-25T10:06:35.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.310+0000 7fb626a4c640 1 -- 172.21.15.133:0/1780401241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6200ca440 con 0x7fb6200b14d0 2024-04-25T10:06:35.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.310+0000 7fb61cff9640 1 -- 172.21.15.133:0/1780401241 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb610011590 con 0x7fb6200b14d0 2024-04-25T10:06:35.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.310+0000 7fb626a4c640 1 -- 172.21.15.133:0/1780401241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb620067c70 con 0x7fb6200b14d0 2024-04-25T10:06:35.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.310+0000 7fb61cff9640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb60c066f90 0x7fb60c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.310+0000 7fb61cff9640 1 -- 172.21.15.133:0/1780401241 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fb610085170 con 0x7fb6200b14d0 2024-04-25T10:06:35.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.310+0000 7fb61f7fe640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb60c066f90 0x7fb60c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.310+0000 7fb61f7fe640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb60c066f90 0x7fb60c069450 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb6140af000 tx=0x7fb6140a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.314+0000 7fb61cff9640 1 -- 172.21.15.133:0/1780401241 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb610015020 con 0x7fb6200b14d0 2024-04-25T10:06:35.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:35.462+0000 7fb626a4c640 1 -- 172.21.15.133:0/1780401241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb6200b6520 con 0x7fb6200b14d0 2024-04-25T10:06:36.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.142+0000 7fb61cff9640 1 -- 172.21.15.133:0/1780401241 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb610056240 con 0x7fb6200b14d0 2024-04-25T10:06:36.146 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:06:36.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 -- 172.21.15.133:0/1780401241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb60c066f90 msgr2=0x7fb60c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb60c066f90 0x7fb60c069450 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb6140af000 tx=0x7fb6140a7000 comp rx=0 tx=0).stop 2024-04-25T10:06:36.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 -- 172.21.15.133:0/1780401241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6200b14d0 msgr2=0x7fb620156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6200b14d0 0x7fb620156ae0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb61000c7c0 tx=0x7fb61000cc80 comp rx=0 tx=0).stop 2024-04-25T10:06:36.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 -- 172.21.15.133:0/1780401241 shutdown_connections 2024-04-25T10:06:36.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6200ce250 0x7fb62015a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6200cc090 0x7fb620154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb60c066f90 0x7fb60c069450 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 --2- 172.21.15.133:0/1780401241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6200b14d0 0x7fb620156ae0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 -- 172.21.15.133:0/1780401241 >> 172.21.15.133:0/1780401241 conn(0x7fb6200999d0 msgr2=0x7fb6200cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 -- 172.21.15.133:0/1780401241 shutdown_connections 2024-04-25T10:06:36.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.146+0000 7fb626a4c640 1 -- 172.21.15.133:0/1780401241 wait complete. 2024-04-25T10:06:36.183 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.302+0000 7fdd9d78e640 1 Processor -- start 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.302+0000 7fdd9d78e640 1 -- start start 2024-04-25T10:06:36.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd9d78e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd9815d9a0 0x7fdd980d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd9d78e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd980d24c0 con 0x7fdd980ce600 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd9d78e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd9815ebd0 con 0x7fdd980d2030 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd9d78e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9815eeb0 con 0x7fdd9815d9a0 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd967fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdd980ce600 0x7fdd980d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57086/0 (socket says 172.21.15.133:57086) 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd967fc640 1 -- 172.21.15.133:0/1887517584 learned_addr learned my addr 172.21.15.133:0/1887517584 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361311665 0 0) 0x7fdd980d24c0 con 0x7fdd980ce600 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd96ffd640 1 --2- 172.21.15.133:0/1887517584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd9815d9a0 0x7fdd980d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd84003530 con 0x7fdd980ce600 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1041979773 0 0) 0x7fdd84003530 con 0x7fdd980ce600 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd9815d9a0 msgr2=0x7fdd980d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 --2- 172.21.15.133:0/1887517584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd9815d9a0 0x7fdd980d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 >> v1:172.21.15.162:6789/0 conn(0x7fdd980d2030 legacy=0x7fdd9815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd980d0330 con 0x7fdd980ce600 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fdd80001680 con 0x7fdd980ce600 2024-04-25T10:06:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 >> v1:172.21.15.133:6789/0 conn(0x7fdd980ce600 legacy=0x7fdd980d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 --2- 172.21.15.133:0/1887517584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd9815d9a0 0x7fdd840039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 --2- 172.21.15.133:0/1887517584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd84003f70 0x7fdd84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 --2- 172.21.15.133:0/1887517584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd84004d50 0x7fdd840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9815eeb0 con 0x7fdd84003f70 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8400bb60 con 0x7fdd84004d50 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8400bd60 con 0x7fdd9815d9a0 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd967fc640 1 --2- 172.21.15.133:0/1887517584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd84003f70 0x7fdd84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd977fe640 1 --2- 172.21.15.133:0/1887517584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd84004d50 0x7fdd840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd967fc640 1 -- 172.21.15.133:0/1887517584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd9815d9a0 msgr2=0x7fdd840039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd967fc640 1 --2- 172.21.15.133:0/1887517584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd9815d9a0 0x7fdd840039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd967fc640 1 -- 172.21.15.133:0/1887517584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd84004d50 msgr2=0x7fdd840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd967fc640 1 --2- 172.21.15.133:0/1887517584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd84004d50 0x7fdd840051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.306+0000 7fdd967fc640 1 -- 172.21.15.133:0/1887517584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84003530 con 0x7fdd84003f70 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd967fc640 1 --2- 172.21.15.133:0/1887517584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd84003f70 0x7fdd84004810 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000c1c0 tx=0x7fdd8001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5758f9eec79eec48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80007d00 con 0x7fdd84003f70 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd800a9020 con 0x7fdd84003f70 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd95ffb640 1 -- 172.21.15.133:0/1887517584 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd800a5b30 con 0x7fdd84003f70 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1887517584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd84003f70 msgr2=0x7fdd84004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 --2- 172.21.15.133:0/1887517584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd84003f70 0x7fdd84004810 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000c1c0 tx=0x7fdd8001cc30 comp rx=0 tx=0).stop 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd977fe640 1 -- 172.21.15.133:0/1887517584 reap_dead start 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1887517584 shutdown_connections 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1887517584 >> 172.21.15.133:0/1887517584 conn(0x7fdd9806bf80 msgr2=0x7fdd9806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1887517584 shutdown_connections 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1887517584 wait complete. 2024-04-25T10:06:36.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 Processor -- start 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 -- start start 2024-04-25T10:06:36.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd9815ccc0 0x7fdd981647b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd9815d0a0 0x7fdd980b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd980cc600 0x7fdd980b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd980b9820 con 0x7fdd9815ccc0 2024-04-25T10:06:36.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd980b96a0 con 0x7fdd9815d0a0 2024-04-25T10:06:36.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd9d78e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd980cee30 con 0x7fdd980cc600 2024-04-25T10:06:36.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd96ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd9815ccc0 0x7fdd981647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd96ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd9815ccc0 0x7fdd981647b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43668/0 (socket says 172.21.15.133:43668) 2024-04-25T10:06:36.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.310+0000 7fdd96ffd640 1 -- 172.21.15.133:0/1398135288 learned_addr learned my addr 172.21.15.133:0/1398135288 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd967fc640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd9815d0a0 0x7fdd980b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd977fe640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd980cc600 0x7fdd980b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd96ffd640 1 -- 172.21.15.133:0/1398135288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd980cc600 msgr2=0x7fdd980b6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd96ffd640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd980cc600 0x7fdd980b6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd96ffd640 1 -- 172.21.15.133:0/1398135288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd9815d0a0 msgr2=0x7fdd980b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd96ffd640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd9815d0a0 0x7fdd980b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd96ffd640 1 -- 172.21.15.133:0/1398135288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd98164cf0 con 0x7fdd9815ccc0 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd967fc640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd9815d0a0 0x7fdd980b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd96ffd640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd9815ccc0 0x7fdd981647b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800ef40 tx=0x7fdd8800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd8f7fe640 1 -- 172.21.15.133:0/1398135288 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88018020 con 0x7fdd9815ccc0 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd8f7fe640 1 -- 172.21.15.133:0/1398135288 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd88014400 con 0x7fdd9815ccc0 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1398135288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd98164fc0 con 0x7fdd9815ccc0 2024-04-25T10:06:36.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd8f7fe640 1 -- 172.21.15.133:0/1398135288 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd880147d0 con 0x7fdd9815ccc0 2024-04-25T10:06:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1398135288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd98165580 con 0x7fdd9815ccc0 2024-04-25T10:06:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1398135288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd98068e80 con 0x7fdd9815ccc0 2024-04-25T10:06:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd8f7fe640 1 -- 172.21.15.133:0/1398135288 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd8801b020 con 0x7fdd9815ccc0 2024-04-25T10:06:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd8f7fe640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd78066f60 0x7fdd78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd8f7fe640 1 -- 172.21.15.133:0/1398135288 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fdd88085fd0 con 0x7fdd9815ccc0 2024-04-25T10:06:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd967fc640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd78066f60 0x7fdd78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.314+0000 7fdd967fc640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd78066f60 0x7fdd78069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000c1c0 tx=0x7fdd9815eda0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.318+0000 7fdd8f7fe640 1 -- 172.21.15.133:0/1398135288 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd88022030 con 0x7fdd9815ccc0 2024-04-25T10:06:36.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:36.514+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1398135288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdd9815f8f0 con 0x7fdd9815ccc0 2024-04-25T10:06:37.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.210+0000 7fdd8f7fe640 1 -- 172.21.15.133:0/1398135288 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) v1 ==== 173+0+0 (secure 0 0 0) 0x7fdd88057020 con 0x7fdd9815ccc0 2024-04-25T10:06:37.215 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T10:06:37.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1398135288 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd78066f60 msgr2=0x7fdd78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd78066f60 0x7fdd78069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000c1c0 tx=0x7fdd9815eda0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1398135288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd9815ccc0 msgr2=0x7fdd981647b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd9815ccc0 0x7fdd981647b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800ef40 tx=0x7fdd8800b480 comp rx=0 tx=0).stop 2024-04-25T10:06:37.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1398135288 shutdown_connections 2024-04-25T10:06:37.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd980cc600 0x7fdd980b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd9815d0a0 0x7fdd980b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd78066f60 0x7fdd78069420 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 --2- 172.21.15.133:0/1398135288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd9815ccc0 0x7fdd981647b0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1398135288 >> 172.21.15.133:0/1398135288 conn(0x7fdd9806bf80 msgr2=0x7fdd9815fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1398135288 shutdown_connections 2024-04-25T10:06:37.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.218+0000 7fdd9d78e640 1 -- 172.21.15.133:0/1398135288 wait complete. 2024-04-25T10:06:37.255 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:37.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d47d8640 1 Processor -- start 2024-04-25T10:06:37.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d47d8640 1 -- start start 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d47d8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31cc0d0bd0 0x7f31cc0b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d47d8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31cc0ceb20 con 0x7f31cc0d0770 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d47d8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31cc0b30a0 con 0x7f31cc15db00 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d47d8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31cc0b7740 con 0x7f31cc0d0bd0 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1d73640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f31cc0d0770 0x7f31cc0b87e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57096/0 (socket says 172.21.15.133:57096) 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1d73640 1 -- 172.21.15.133:0/2426383841 learned_addr learned my addr 172.21.15.133:0/2426383841 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d2d75640 1 --2- 172.21.15.133:0/2426383841 >> v2:172.21.15.162:3300/0 conn(0x7f31cc0d0bd0 0x7f31cc0b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402377822 0 0) 0x7f31cc0ceb20 con 0x7f31cc0d0770 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b8003530 con 0x7f31cc0d0770 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 664561087 0 0) 0x7f31b8003530 con 0x7f31cc0d0770 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 >> v2:172.21.15.162:3300/0 conn(0x7f31cc0d0bd0 msgr2=0x7f31cc0b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 --2- 172.21.15.133:0/2426383841 >> v2:172.21.15.162:3300/0 conn(0x7f31cc0d0bd0 0x7f31cc0b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 >> v1:172.21.15.162:6789/0 conn(0x7f31cc15db00 legacy=0x7f31cc0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc0b7a20 con 0x7f31cc0d0770 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f31bc001680 con 0x7f31cc0d0770 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 >> v1:172.21.15.133:6789/0 conn(0x7f31cc0d0770 legacy=0x7f31cc0b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 --2- 172.21.15.133:0/2426383841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80039c0 0x7f31b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 --2- 172.21.15.133:0/2426383841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31b8004810 0x7f31b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 --2- 172.21.15.133:0/2426383841 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31b80051d0 0x7f31b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc0b7740 con 0x7f31b80039c0 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b800c070 con 0x7f31b8004810 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.366+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b800c270 con 0x7f31b80051d0 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d2574640 1 --2- 172.21.15.133:0/2426383841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80039c0 0x7f31b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d1d73640 1 --2- 172.21.15.133:0/2426383841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31b8004810 0x7f31b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d2574640 1 -- 172.21.15.133:0/2426383841 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31b80051d0 msgr2=0x7f31b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d2574640 1 --2- 172.21.15.133:0/2426383841 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31b80051d0 0x7f31b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d2574640 1 -- 172.21.15.133:0/2426383841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31b8004810 msgr2=0x7f31b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d2574640 1 --2- 172.21.15.133:0/2426383841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31b8004810 0x7f31b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d2574640 1 -- 172.21.15.133:0/2426383841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b8003530 con 0x7f31b80039c0 2024-04-25T10:06:37.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d1d73640 1 --2- 172.21.15.133:0/2426383841 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31b8004810 0x7f31b8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:37.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d2d75640 1 -- 172.21.15.133:0/2426383841 reap_dead start 2024-04-25T10:06:37.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d2574640 1 --2- 172.21.15.133:0/2426383841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80039c0 0x7f31b8003de0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f31c8001780 tx=0x7f31c801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e853c7440c526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80013a0 con 0x7f31b80039c0 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c80052f0 con 0x7f31b80039c0 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d1572640 1 -- 172.21.15.133:0/2426383841 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8003840 con 0x7f31b80039c0 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d47d8640 1 -- 172.21.15.133:0/2426383841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80039c0 msgr2=0x7f31b8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d47d8640 1 --2- 172.21.15.133:0/2426383841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80039c0 0x7f31b8003de0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f31c8001780 tx=0x7f31c801c210 comp rx=0 tx=0).stop 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d47d8640 1 -- 172.21.15.133:0/2426383841 shutdown_connections 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d47d8640 1 --2- 172.21.15.133:0/2426383841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80039c0 0x7f31b8003de0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d47d8640 1 -- 172.21.15.133:0/2426383841 >> 172.21.15.133:0/2426383841 conn(0x7f31cc0727c0 msgr2=0x7f31cc072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d47d8640 1 -- 172.21.15.133:0/2426383841 shutdown_connections 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.370+0000 7f31d47d8640 1 -- 172.21.15.133:0/2426383841 wait complete. 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d47d8640 1 Processor -- start 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d47d8640 1 -- start start 2024-04-25T10:06:37.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d47d8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31b80039c0 0x7f31cc15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d47d8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31cc15a9f0 0x7f31cc15dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d47d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc1ea460 0x7f31cc1ea910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d47d8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc0d0fe0 con 0x7f31cc1ea460 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d47d8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc0b7430 con 0x7f31b80039c0 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d47d8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc0d18a0 con 0x7f31cc15a9f0 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d2d75640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc1ea460 0x7f31cc1ea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d2d75640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc1ea460 0x7f31cc1ea910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43694/0 (socket says 172.21.15.133:43694) 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d2d75640 1 -- 172.21.15.133:0/792749522 learned_addr learned my addr 172.21.15.133:0/792749522 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d1d73640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31cc15a9f0 0x7f31cc15dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d2574640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31b80039c0 0x7f31cc15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d2d75640 1 -- 172.21.15.133:0/792749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31cc15a9f0 msgr2=0x7f31cc15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d2d75640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31cc15a9f0 0x7f31cc15dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d2d75640 1 -- 172.21.15.133:0/792749522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31b80039c0 msgr2=0x7f31cc15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d2d75640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31b80039c0 0x7f31cc15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d2d75640 1 -- 172.21.15.133:0/792749522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc1f7880 con 0x7f31cc1ea460 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d1d73640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31cc15a9f0 0x7f31cc15dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d2d75640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc1ea460 0x7f31cc1ea910 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0d00e0 tx=0x7f31c400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31c2ffd640 1 -- 172.21.15.133:0/792749522 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c400bba0 con 0x7f31cc1ea460 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31c2ffd640 1 -- 172.21.15.133:0/792749522 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c401a070 con 0x7f31cc1ea460 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31c2ffd640 1 -- 172.21.15.133:0/792749522 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4014400 con 0x7f31cc1ea460 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d47d8640 1 -- 172.21.15.133:0/792749522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31cc1f7b60 con 0x7f31cc1ea460 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d47d8640 1 -- 172.21.15.133:0/792749522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31cc1e9f40 con 0x7f31cc1ea460 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.374+0000 7f31d47d8640 1 -- 172.21.15.133:0/792749522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31cc1ea140 con 0x7f31cc1ea460 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.378+0000 7f31c2ffd640 1 -- 172.21.15.133:0/792749522 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f31c4013070 con 0x7f31cc1ea460 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.378+0000 7f31c2ffd640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f31b0066f60 0x7f31b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.378+0000 7f31c2ffd640 1 -- 172.21.15.133:0/792749522 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f31c4085260 con 0x7f31cc1ea460 2024-04-25T10:06:37.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.378+0000 7f31d2574640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f31b0066f60 0x7f31b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.378+0000 7f31d2574640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f31b0066f60 0x7f31b0069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f31c8001750 tx=0x7f31c80011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.382+0000 7f31c2ffd640 1 -- 172.21.15.133:0/792749522 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f31cc1ea140 con 0x7f31cc1ea460 2024-04-25T10:06:37.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.534+0000 7f31d47d8640 1 -- 172.21.15.133:0/792749522 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31cc15add0 con 0x7f31cc1ea460 2024-04-25T10:06:37.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31c2ffd640 1 -- 172.21.15.133:0/792749522 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16921 (secure 0 0 0) 0x7f31c4024020 con 0x7f31cc1ea460 2024-04-25T10:06:37.542 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:37.542 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:06:36.515996+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:34.764755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:35.504408+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 -- 172.21.15.133:0/792749522 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f31b0066f60 msgr2=0x7f31b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f31b0066f60 0x7f31b0069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f31c8001750 tx=0x7f31c80011a0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 -- 172.21.15.133:0/792749522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc1ea460 msgr2=0x7f31cc1ea910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc1ea460 0x7f31cc1ea910 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0d00e0 tx=0x7f31c400ef30 comp rx=0 tx=0).stop 2024-04-25T10:06:37.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 -- 172.21.15.133:0/792749522 shutdown_connections 2024-04-25T10:06:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f31b0066f60 0x7f31b0069420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31cc1ea460 0x7f31cc1ea910 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31cc15a9f0 0x7f31cc15dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 --2- 172.21.15.133:0/792749522 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31b80039c0 0x7f31cc15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 -- 172.21.15.133:0/792749522 >> 172.21.15.133:0/792749522 conn(0x7f31cc0727c0 msgr2=0x7f31cc0b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 -- 172.21.15.133:0/792749522 shutdown_connections 2024-04-25T10:06:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.538+0000 7f31d47d8640 1 -- 172.21.15.133:0/792749522 wait complete. 2024-04-25T10:06:37.560 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:37.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.662+0000 7fe8c38f9640 1 Processor -- start 2024-04-25T10:06:37.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.662+0000 7fe8c38f9640 1 -- start start 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c38f9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe8bc15db00 0x7fe8bc0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c38f9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc0d0bd0 0x7fe8bc161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c38f9640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc0ceb20 con 0x7fe8bc0d0770 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c38f9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc0b7740 con 0x7fe8bc15db00 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c38f9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0b7a20 con 0x7fe8bc0d0bd0 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c0e94640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe8bc0d0770 0x7fe8bc1614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57106/0 (socket says 172.21.15.133:57106) 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c0e94640 1 -- 172.21.15.133:0/1601461815 learned_addr learned my addr 172.21.15.133:0/1601461815 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c1e96640 1 --2- 172.21.15.133:0/1601461815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc0d0bd0 0x7fe8bc161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285326361 0 0) 0x7fe8bc0ceb20 con 0x7fe8bc0d0770 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8a4003530 con 0x7fe8bc0d0770 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c1695640 1 --2- 172.21.15.133:0/1601461815 >> v2:172.21.15.162:3300/0 conn(0x7fe8bc15db00 0x7fe8bc0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 457949485 0 0) 0x7fe8a4003530 con 0x7fe8bc0d0770 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc0d0bd0 msgr2=0x7fe8bc161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 --2- 172.21.15.133:0/1601461815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc0d0bd0 0x7fe8bc161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 >> v2:172.21.15.162:3300/0 conn(0x7fe8bc15db00 msgr2=0x7fe8bc0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 --2- 172.21.15.133:0/1601461815 >> v2:172.21.15.162:3300/0 conn(0x7fe8bc15db00 0x7fe8bc0b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0b97f0 con 0x7fe8bc0d0770 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe8b0001680 con 0x7fe8bc0d0770 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 >> v1:172.21.15.133:6789/0 conn(0x7fe8bc0d0770 legacy=0x7fe8bc1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 --2- 172.21.15.133:0/1601461815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 0x7fe8a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 --2- 172.21.15.133:0/1601461815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8a4004810 0x7fe8a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 --2- 172.21.15.133:0/1601461815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc0d0bd0 0x7fe8a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0b7740 con 0x7fe8a40039c0 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0b7a20 con 0x7fe8a4004810 2024-04-25T10:06:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a400ba00 con 0x7fe8bc0d0bd0 2024-04-25T10:06:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c1695640 1 --2- 172.21.15.133:0/1601461815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 0x7fe8a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c1695640 1 -- 172.21.15.133:0/1601461815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc0d0bd0 msgr2=0x7fe8a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c1695640 1 --2- 172.21.15.133:0/1601461815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc0d0bd0 0x7fe8a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c1695640 1 -- 172.21.15.133:0/1601461815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8a4004810 msgr2=0x7fe8a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c1695640 1 --2- 172.21.15.133:0/1601461815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8a4004810 0x7fe8a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.666+0000 7fe8c1695640 1 -- 172.21.15.133:0/1601461815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4003530 con 0x7fe8a40039c0 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1695640 1 --2- 172.21.15.133:0/1601461815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 0x7fe8a4003de0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400ba30 tx=0x7fe8b401c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b987fee239da35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40b3070 con 0x7fe8a40039c0 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b40ae660 con 0x7fe8a40039c0 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8bbfff640 1 -- 172.21.15.133:0/1601461815 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b400b600 con 0x7fe8a40039c0 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 -- 172.21.15.133:0/1601461815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 msgr2=0x7fe8a4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 --2- 172.21.15.133:0/1601461815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 0x7fe8a4003de0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400ba30 tx=0x7fe8b401c6b0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1e96640 1 -- 172.21.15.133:0/1601461815 reap_dead start 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 -- 172.21.15.133:0/1601461815 shutdown_connections 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 -- 172.21.15.133:0/1601461815 >> 172.21.15.133:0/1601461815 conn(0x7fe8bc0727c0 msgr2=0x7fe8bc072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 -- 172.21.15.133:0/1601461815 shutdown_connections 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 -- 172.21.15.133:0/1601461815 wait complete. 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 Processor -- start 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 -- start start 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 0x7fe8bc15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8bc0c97b0 0x7fe8bc15a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc15a910 0x7fe8bc15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0d0fe0 con 0x7fe8a40039c0 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0b7430 con 0x7fe8bc0c97b0 2024-04-25T10:06:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0d18a0 con 0x7fe8bc15a910 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1695640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 0x7fe8bc15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1695640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 0x7fe8bc15e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43726/0 (socket says 172.21.15.133:43726) 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1695640 1 -- 172.21.15.133:0/370410362 learned_addr learned my addr 172.21.15.133:0/370410362 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1e96640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc15a910 0x7fe8bc15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1695640 1 -- 172.21.15.133:0/370410362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc15a910 msgr2=0x7fe8bc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c0e94640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8bc0c97b0 0x7fe8bc15a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1695640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc15a910 0x7fe8bc15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1695640 1 -- 172.21.15.133:0/370410362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8bc0c97b0 msgr2=0x7fe8bc15a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1695640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8bc0c97b0 0x7fe8bc15a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1695640 1 -- 172.21.15.133:0/370410362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0cde50 con 0x7fe8a40039c0 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c1695640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 0x7fe8bc15e670 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe8b401cb90 tx=0x7fe8a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8b9ffb640 1 -- 172.21.15.133:0/370410362 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40b3040 con 0x7fe8a40039c0 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8b9ffb640 1 -- 172.21.15.133:0/370410362 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b40ae460 con 0x7fe8a40039c0 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 -- 172.21.15.133:0/370410362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc0cdb70 con 0x7fe8a40039c0 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8b9ffb640 1 -- 172.21.15.133:0/370410362 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4004240 con 0x7fe8a40039c0 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.670+0000 7fe8c38f9640 1 -- 172.21.15.133:0/370410362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc0cac70 con 0x7fe8a40039c0 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.674+0000 7fe8c38f9640 1 -- 172.21.15.133:0/370410362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc068ef0 con 0x7fe8a40039c0 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.674+0000 7fe8b9ffb640 1 -- 172.21.15.133:0/370410362 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8b40c0020 con 0x7fe8a40039c0 2024-04-25T10:06:37.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.674+0000 7fe8b9ffb640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8a8066f60 0x7fe8a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.674+0000 7fe8b9ffb640 1 -- 172.21.15.133:0/370410362 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8b411f430 con 0x7fe8a40039c0 2024-04-25T10:06:37.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.674+0000 7fe8c0e94640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8a8066f60 0x7fe8a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.674+0000 7fe8c0e94640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8a8066f60 0x7fe8a8069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0001ec0 tx=0x7fe8b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.678+0000 7fe8b9ffb640 1 -- 172.21.15.133:0/370410362 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8b40163f0 con 0x7fe8a40039c0 2024-04-25T10:06:37.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.826+0000 7fe8c38f9640 1 -- 172.21.15.133:0/370410362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8bc0b2750 con 0x7fe8a40039c0 2024-04-25T10:06:37.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.826+0000 7fe8b9ffb640 1 -- 172.21.15.133:0/370410362 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16921 (secure 0 0 0) 0x7fe8b40efc60 con 0x7fe8a40039c0 2024-04-25T10:06:37.832 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:37.832 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:06:36.515996+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:34.764755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:35.504408+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 -- 172.21.15.133:0/370410362 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8a8066f60 msgr2=0x7fe8a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8a8066f60 0x7fe8a8069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0001ec0 tx=0x7fe8b0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 -- 172.21.15.133:0/370410362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 msgr2=0x7fe8bc15e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 0x7fe8bc15e670 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe8b401cb90 tx=0x7fe8a4005840 comp rx=0 tx=0).stop 2024-04-25T10:06:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 -- 172.21.15.133:0/370410362 shutdown_connections 2024-04-25T10:06:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8bc15a910 0x7fe8bc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8bc0c97b0 0x7fe8bc15a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe8a8066f60 0x7fe8a8069420 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 --2- 172.21.15.133:0/370410362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a40039c0 0x7fe8bc15e670 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 -- 172.21.15.133:0/370410362 >> 172.21.15.133:0/370410362 conn(0x7fe8bc0727c0 msgr2=0x7fe8bc15cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 -- 172.21.15.133:0/370410362 shutdown_connections 2024-04-25T10:06:37.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.830+0000 7fe8c38f9640 1 -- 172.21.15.133:0/370410362 wait complete. 2024-04-25T10:06:37.850 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:06:37.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.966+0000 7fd8e7e5f640 1 Processor -- start 2024-04-25T10:06:37.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e7e5f640 1 -- start start 2024-04-25T10:06:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e7e5f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8e0145170 0x7fd8e006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e7e5f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e015ced0 0x7fd8e015d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e7e5f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e00ce2e0 con 0x7fd8e006e4d0 2024-04-25T10:06:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e7e5f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e006db00 con 0x7fd8e0145170 2024-04-25T10:06:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e7e5f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e0145550 con 0x7fd8e015ced0 2024-04-25T10:06:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e63fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e015ced0 0x7fd8e015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e63fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e015ced0 0x7fd8e015d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:44014/0 (socket says 172.21.15.133:44014) 2024-04-25T10:06:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e63fc640 1 -- 172.21.15.133:0/869860917 learned_addr learned my addr 172.21.15.133:0/869860917 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e5bfb640 1 --2- 172.21.15.133:0/869860917 >> v2:172.21.15.162:3300/0 conn(0x7fd8e0145170 0x7fd8e006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e63fc640 1 -- 172.21.15.133:0/869860917 >> v2:172.21.15.162:3300/0 conn(0x7fd8e0145170 msgr2=0x7fd8e006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e63fc640 1 --2- 172.21.15.133:0/869860917 >> v2:172.21.15.162:3300/0 conn(0x7fd8e0145170 0x7fd8e006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e63fc640 1 -- 172.21.15.133:0/869860917 >> v1:172.21.15.162:6789/0 conn(0x7fd8e006e4d0 legacy=0x7fd8e00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.970+0000 7fd8e63fc640 1 -- 172.21.15.133:0/869860917 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e006d4b0 con 0x7fd8e015ced0 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e63fc640 1 --2- 172.21.15.133:0/869860917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e015ced0 0x7fd8e015d360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009e50 tx=0x7fd8d801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fcf24cd31b62e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e4bf9640 1 -- 172.21.15.133:0/869860917 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80a5700 con 0x7fd8e015ced0 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e4bf9640 1 -- 172.21.15.133:0/869860917 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d80a58a0 con 0x7fd8e015ced0 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e4bf9640 1 -- 172.21.15.133:0/869860917 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80a5c70 con 0x7fd8e015ced0 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/869860917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e015ced0 msgr2=0x7fd8e015d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 --2- 172.21.15.133:0/869860917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e015ced0 0x7fd8e015d360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8009e50 tx=0x7fd8d801ca30 comp rx=0 tx=0).stop 2024-04-25T10:06:37.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/869860917 shutdown_connections 2024-04-25T10:06:37.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 --2- 172.21.15.133:0/869860917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e015ced0 0x7fd8e015d360 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 --2- 172.21.15.133:0/869860917 >> v2:172.21.15.162:3300/0 conn(0x7fd8e0145170 0x7fd8e006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/869860917 >> 172.21.15.133:0/869860917 conn(0x7fd8e0066f70 msgr2=0x7fd8e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/869860917 shutdown_connections 2024-04-25T10:06:37.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/869860917 wait complete. 2024-04-25T10:06:37.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 Processor -- start 2024-04-25T10:06:37.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 -- start start 2024-04-25T10:06:37.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8e006e4d0 0x7fd8e0134600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e0145170 0x7fd8e00af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e015ced0 0x7fd8e00af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e0168610 con 0x7fd8e006e4d0 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e0168490 con 0x7fd8e015ced0 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e0168790 con 0x7fd8e0145170 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e5bfb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8e006e4d0 0x7fd8e0134600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e5bfb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8e006e4d0 0x7fd8e0134600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43728/0 (socket says 172.21.15.133:43728) 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e5bfb640 1 -- 172.21.15.133:0/2628874122 learned_addr learned my addr 172.21.15.133:0/2628874122 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e53fa640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e0145170 0x7fd8e00af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e63fc640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e015ced0 0x7fd8e00af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e5bfb640 1 -- 172.21.15.133:0/2628874122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e0145170 msgr2=0x7fd8e00af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e5bfb640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e0145170 0x7fd8e00af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e5bfb640 1 -- 172.21.15.133:0/2628874122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e015ced0 msgr2=0x7fd8e00af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e5bfb640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e015ced0 0x7fd8e00af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e5bfb640 1 -- 172.21.15.133:0/2628874122 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e015ede0 con 0x7fd8e006e4d0 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e5bfb640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8e006e4d0 0x7fd8e0134600 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00e8a0 tx=0x7fd8dc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8d67fc640 1 -- 172.21.15.133:0/2628874122 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc017070 con 0x7fd8e006e4d0 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8d67fc640 1 -- 172.21.15.133:0/2628874122 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc00c940 con 0x7fd8e006e4d0 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/2628874122 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e0134b40 con 0x7fd8e006e4d0 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.974+0000 7fd8d67fc640 1 -- 172.21.15.133:0/2628874122 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc00ccf0 con 0x7fd8e006e4d0 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.978+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/2628874122 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e00b05e0 con 0x7fd8e006e4d0 2024-04-25T10:06:37.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.978+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/2628874122 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e0068e80 con 0x7fd8e006e4d0 2024-04-25T10:06:37.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.978+0000 7fd8d67fc640 1 -- 172.21.15.133:0/2628874122 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8dc01a070 con 0x7fd8e006e4d0 2024-04-25T10:06:37.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.978+0000 7fd8d67fc640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8c4066f60 0x7fd8c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.978+0000 7fd8d67fc640 1 -- 172.21.15.133:0/2628874122 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8dc084f60 con 0x7fd8e006e4d0 2024-04-25T10:06:37.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.978+0000 7fd8e53fa640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8c4066f60 0x7fd8c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.978+0000 7fd8e53fa640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8c4066f60 0x7fd8c4069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd8e015e7c0 tx=0x7fd8d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:37.982+0000 7fd8d67fc640 1 -- 172.21.15.133:0/2628874122 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8dc055820 con 0x7fd8e006e4d0 2024-04-25T10:06:38.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.170+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/2628874122 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd8e015ddc0 con 0x7fd8e006e4d0 2024-04-25T10:06:38.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.206+0000 7fd8d67fc640 1 -- 172.21.15.133:0/2628874122 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd8dc0598c0 con 0x7fd8e006e4d0 2024-04-25T10:06:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/2628874122 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8c4066f60 msgr2=0x7fd8c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8c4066f60 0x7fd8c4069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd8e015e7c0 tx=0x7fd8d0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/2628874122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8e006e4d0 msgr2=0x7fd8e0134600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8e006e4d0 0x7fd8e0134600 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00e8a0 tx=0x7fd8dc00eca0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/2628874122 shutdown_connections 2024-04-25T10:06:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8e015ced0 0x7fd8e00af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8e0145170 0x7fd8e00af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8c4066f60 0x7fd8c4069420 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 --2- 172.21.15.133:0/2628874122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8e006e4d0 0x7fd8e0134600 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/2628874122 >> 172.21.15.133:0/2628874122 conn(0x7fd8e0066f70 msgr2=0x7fd8e01530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:38.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/2628874122 shutdown_connections 2024-04-25T10:06:38.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.210+0000 7fd8e7e5f640 1 -- 172.21.15.133:0/2628874122 wait complete. 2024-04-25T10:06:38.231 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:06:38.231 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:06:38.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe037d74640 1 Processor -- start 2024-04-25T10:06:38.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe037d74640 1 -- start start 2024-04-25T10:06:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe037d74640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe030143fb0 0x7fe0300cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe037d74640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe0300ccd50 0x7fe03006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe037d74640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe03006e8f0 0x7fe03015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe037d74640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03006ed60 con 0x7fe0300ccd50 2024-04-25T10:06:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe037d74640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0300cd1f0 con 0x7fe030143fb0 2024-04-25T10:06:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe037d74640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03015aee0 con 0x7fe03006e8f0 2024-04-25T10:06:38.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe03530f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe0300ccd50 0x7fe03006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe03530f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe0300ccd50 0x7fe03006e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43742/0 (socket says 172.21.15.133:43742) 2024-04-25T10:06:38.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe03530f640 1 -- 172.21.15.133:0/200698259 learned_addr learned my addr 172.21.15.133:0/200698259 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:38.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe03530f640 1 -- 172.21.15.133:0/200698259 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe03006e8f0 msgr2=0x7fe03015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:38.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe03530f640 1 --2- 172.21.15.133:0/200698259 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe03006e8f0 0x7fe03015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe03530f640 1 -- 172.21.15.133:0/200698259 >> v2:172.21.15.162:3300/0 conn(0x7fe030143fb0 msgr2=0x7fe0300cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:38.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe03530f640 1 --2- 172.21.15.133:0/200698259 >> v2:172.21.15.162:3300/0 conn(0x7fe030143fb0 0x7fe0300cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.342+0000 7fe03530f640 1 -- 172.21.15.133:0/200698259 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03015b1c0 con 0x7fe0300ccd50 2024-04-25T10:06:38.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe03530f640 1 --2- 172.21.15.133:0/200698259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0300ccd50 0x7fe03006e3b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe03015b0f0 tx=0x7fe02001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914cd9415365b205 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe034b0e640 1 -- 172.21.15.133:0/200698259 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0200b0070 con 0x7fe0300ccd50 2024-04-25T10:06:38.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe034b0e640 1 -- 172.21.15.133:0/200698259 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0200ab600 con 0x7fe0300ccd50 2024-04-25T10:06:38.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe034b0e640 1 -- 172.21.15.133:0/200698259 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0200ab9d0 con 0x7fe0300ccd50 2024-04-25T10:06:38.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 -- 172.21.15.133:0/200698259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0300ccd50 msgr2=0x7fe03006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 --2- 172.21.15.133:0/200698259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0300ccd50 0x7fe03006e3b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe03015b0f0 tx=0x7fe02001c230 comp rx=0 tx=0).stop 2024-04-25T10:06:38.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 -- 172.21.15.133:0/200698259 shutdown_connections 2024-04-25T10:06:38.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 --2- 172.21.15.133:0/200698259 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe03006e8f0 0x7fe03015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 --2- 172.21.15.133:0/200698259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0300ccd50 0x7fe03006e3b0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 --2- 172.21.15.133:0/200698259 >> v2:172.21.15.162:3300/0 conn(0x7fe030143fb0 0x7fe0300cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 -- 172.21.15.133:0/200698259 >> 172.21.15.133:0/200698259 conn(0x7fe030066f70 msgr2=0x7fe03006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:38.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 -- 172.21.15.133:0/200698259 shutdown_connections 2024-04-25T10:06:38.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 -- 172.21.15.133:0/200698259 wait complete. 2024-04-25T10:06:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 Processor -- start 2024-04-25T10:06:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 -- start start 2024-04-25T10:06:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03006e8f0 0x7fe03012df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0300ccd50 0x7fe0300c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe030143fb0 0x7fe030130220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300b1f00 con 0x7fe0300ccd50 2024-04-25T10:06:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300b1d80 con 0x7fe03006e8f0 2024-04-25T10:06:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03015b9c0 con 0x7fe030143fb0 2024-04-25T10:06:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe03530f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0300ccd50 0x7fe0300c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe03530f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0300ccd50 0x7fe0300c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43744/0 (socket says 172.21.15.133:43744) 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe03530f640 1 -- 172.21.15.133:0/2457723273 learned_addr learned my addr 172.21.15.133:0/2457723273 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe03530f640 1 -- 172.21.15.133:0/2457723273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe030143fb0 msgr2=0x7fe030130220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe03530f640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe030143fb0 0x7fe030130220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe03530f640 1 -- 172.21.15.133:0/2457723273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03006e8f0 msgr2=0x7fe03012df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe03530f640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03006e8f0 0x7fe03012df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe03530f640 1 -- 172.21.15.133:0/2457723273 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03012e560 con 0x7fe0300ccd50 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe03530f640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0300ccd50 0x7fe0300c48d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe02001cad0 tx=0x7fe02001aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe0267fc640 1 -- 172.21.15.133:0/2457723273 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0200ba020 con 0x7fe0300ccd50 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe0267fc640 1 -- 172.21.15.133:0/2457723273 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0200b6620 con 0x7fe0300ccd50 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 -- 172.21.15.133:0/2457723273 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03012e840 con 0x7fe0300ccd50 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe0267fc640 1 -- 172.21.15.133:0/2457723273 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0200b0040 con 0x7fe0300ccd50 2024-04-25T10:06:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.346+0000 7fe037d74640 1 -- 172.21.15.133:0/2457723273 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03012ee00 con 0x7fe0300ccd50 2024-04-25T10:06:38.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.350+0000 7fe037d74640 1 -- 172.21.15.133:0/2457723273 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe030068e80 con 0x7fe0300ccd50 2024-04-25T10:06:38.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.350+0000 7fe0267fc640 1 -- 172.21.15.133:0/2457723273 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0200bd020 con 0x7fe0300ccd50 2024-04-25T10:06:38.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.350+0000 7fe0267fc640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe014066f60 0x7fe014069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.350+0000 7fe0267fc640 1 -- 172.21.15.133:0/2457723273 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe02011f550 con 0x7fe0300ccd50 2024-04-25T10:06:38.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.350+0000 7fe035b10640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe014066f60 0x7fe014069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.350+0000 7fe035b10640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe014066f60 0x7fe014069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe028009690 tx=0x7fe028006c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.354+0000 7fe0267fc640 1 -- 172.21.15.133:0/2457723273 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0200c4030 con 0x7fe0300ccd50 2024-04-25T10:06:38.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:38.530+0000 7fe037d74640 1 -- 172.21.15.133:0/2457723273 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe030073ac0 con 0x7fe0300ccd50 2024-04-25T10:06:39.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe0267fc640 1 -- 172.21.15.133:0/2457723273 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe020016110 con 0x7fe0300ccd50 2024-04-25T10:06:39.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 -- 172.21.15.133:0/2457723273 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe014066f60 msgr2=0x7fe014069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe014066f60 0x7fe014069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe028009690 tx=0x7fe028006c00 comp rx=0 tx=0).stop 2024-04-25T10:06:39.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 -- 172.21.15.133:0/2457723273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0300ccd50 msgr2=0x7fe0300c48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0300ccd50 0x7fe0300c48d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fe02001cad0 tx=0x7fe02001aa70 comp rx=0 tx=0).stop 2024-04-25T10:06:39.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 -- 172.21.15.133:0/2457723273 shutdown_connections 2024-04-25T10:06:39.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe030143fb0 0x7fe030130220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe014066f60 0x7fe014069420 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0300ccd50 0x7fe0300c48d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 --2- 172.21.15.133:0/2457723273 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe03006e8f0 0x7fe03012df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 -- 172.21.15.133:0/2457723273 >> 172.21.15.133:0/2457723273 conn(0x7fe030066f70 msgr2=0x7fe0301572a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 -- 172.21.15.133:0/2457723273 shutdown_connections 2024-04-25T10:06:39.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.218+0000 7fe037d74640 1 -- 172.21.15.133:0/2457723273 wait complete. 2024-04-25T10:06:39.243 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:06:39.243 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:39.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.350+0000 7f43e5446640 1 Processor -- start 2024-04-25T10:06:39.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43e5446640 1 -- start start 2024-04-25T10:06:39.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43e5446640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e015d9b0 0x7f43e00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43e5446640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f43e015dd90 0x7f43e00ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43e5446640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e00ca9b0 con 0x7f43e00b19f0 2024-04-25T10:06:39.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43e5446640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e00cbe30 con 0x7f43e015dd90 2024-04-25T10:06:39.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43e5446640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00cac90 con 0x7f43e015d9b0 2024-04-25T10:06:39.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43df7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f43e00b19f0 0x7f43e01616f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57114/0 (socket says 172.21.15.133:57114) 2024-04-25T10:06:39.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43df7fe640 1 -- 172.21.15.133:0/3200165803 learned_addr learned my addr 172.21.15.133:0/3200165803 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:39.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011102825 0 0) 0x7f43e00ca9b0 con 0x7f43e00b19f0 2024-04-25T10:06:39.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43d67fc640 1 --2- 172.21.15.133:0/3200165803 >> v2:172.21.15.162:3300/0 conn(0x7f43e015dd90 0x7f43e00ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43deffd640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e015d9b0 0x7f43e00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43cc003530 con 0x7f43e00b19f0 2024-04-25T10:06:39.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 458044874 0 0) 0x7f43cc003530 con 0x7f43e00b19f0 2024-04-25T10:06:39.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e015d9b0 msgr2=0x7f43e00cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e015d9b0 0x7f43e00cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 >> v2:172.21.15.162:3300/0 conn(0x7f43e015dd90 msgr2=0x7f43e00ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 --2- 172.21.15.133:0/3200165803 >> v2:172.21.15.162:3300/0 conn(0x7f43e015dd90 0x7f43e00ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e00caf70 con 0x7f43e00b19f0 2024-04-25T10:06:39.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f43d8001a00 con 0x7f43e00b19f0 2024-04-25T10:06:39.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 >> v1:172.21.15.133:6789/0 conn(0x7f43e00b19f0 legacy=0x7f43e01616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43cc0039c0 0x7f43cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e015d9b0 0x7f43cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43cc004d50 0x7f43cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00cbe30 con 0x7f43cc004d50 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00cac90 con 0x7f43cc0039c0 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc00ba00 con 0x7f43e015d9b0 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43df7fe640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43cc004d50 0x7f43cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43deffd640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43cc0039c0 0x7f43cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43d67fc640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e015d9b0 0x7f43cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43df7fe640 1 -- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e015d9b0 msgr2=0x7f43cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43df7fe640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e015d9b0 0x7f43cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43df7fe640 1 -- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43cc0039c0 msgr2=0x7f43cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43df7fe640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43cc0039c0 0x7f43cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43df7fe640 1 -- 172.21.15.133:0/3200165803 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc003530 con 0x7f43cc004d50 2024-04-25T10:06:39.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.354+0000 7f43deffd640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43cc0039c0 0x7f43cc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43df7fe640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43cc004d50 0x7f43cc0051d0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f43d80013e0 tx=0x7f43d801fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7290f6314be43c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d800a410 con 0x7f43cc004d50 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d80aa630 con 0x7f43cc004d50 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43de7fc640 1 -- 172.21.15.133:0/3200165803 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d80aaa20 con 0x7f43cc004d50 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 -- 172.21.15.133:0/3200165803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43cc004d50 msgr2=0x7f43cc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 --2- 172.21.15.133:0/3200165803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43cc004d50 0x7f43cc0051d0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f43d80013e0 tx=0x7f43d801fa00 comp rx=0 tx=0).stop 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43df7fe640 1 -- 172.21.15.133:0/3200165803 reap_dead start 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 -- 172.21.15.133:0/3200165803 shutdown_connections 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 -- 172.21.15.133:0/3200165803 >> 172.21.15.133:0/3200165803 conn(0x7f43e0067000 msgr2=0x7f43e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 -- 172.21.15.133:0/3200165803 shutdown_connections 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 -- 172.21.15.133:0/3200165803 wait complete. 2024-04-25T10:06:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 Processor -- start 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 -- start start 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e0142550 0x7f43e0159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43e0150ef0 0x7f43e0142930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43e013f050 0x7f43e013f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00cecb0 con 0x7f43e0150ef0 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e006e950 con 0x7f43e013f050 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43e5446640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00cee30 con 0x7f43e0142550 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43d67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43e0150ef0 0x7f43e0142930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43d67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43e0150ef0 0x7f43e0142930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43760/0 (socket says 172.21.15.133:43760) 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43d67fc640 1 -- 172.21.15.133:0/208926555 learned_addr learned my addr 172.21.15.133:0/208926555 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43df7fe640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43e013f050 0x7f43e013f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43deffd640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e0142550 0x7f43e0159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43d67fc640 1 -- 172.21.15.133:0/208926555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e0142550 msgr2=0x7f43e0159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43d67fc640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e0142550 0x7f43e0159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43d67fc640 1 -- 172.21.15.133:0/208926555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43e013f050 msgr2=0x7f43e013f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43d67fc640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43e013f050 0x7f43e013f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43d67fc640 1 -- 172.21.15.133:0/208926555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e013fa10 con 0x7f43e0150ef0 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43deffd640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e0142550 0x7f43e0159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.358+0000 7f43d67fc640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43e0150ef0 0x7f43e0142930 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f43c8011690 tx=0x7f43c8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.362+0000 7f43d7fff640 1 -- 172.21.15.133:0/208926555 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c800f960 con 0x7f43e0150ef0 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.362+0000 7f43d7fff640 1 -- 172.21.15.133:0/208926555 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c800fb00 con 0x7f43e0150ef0 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.362+0000 7f43e5446640 1 -- 172.21.15.133:0/208926555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e0151370 con 0x7f43e0150ef0 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.362+0000 7f43d7fff640 1 -- 172.21.15.133:0/208926555 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8015460 con 0x7f43e0150ef0 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.362+0000 7f43e5446640 1 -- 172.21.15.133:0/208926555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e0153500 con 0x7f43e0150ef0 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.362+0000 7f43e5446640 1 -- 172.21.15.133:0/208926555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e0068ef0 con 0x7f43e0150ef0 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.362+0000 7f43d7fff640 1 -- 172.21.15.133:0/208926555 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43c8015600 con 0x7f43e0150ef0 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.362+0000 7f43d7fff640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43c0066f90 0x7f43c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.362+0000 7f43d7fff640 1 -- 172.21.15.133:0/208926555 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f43c8085ad0 con 0x7f43e0150ef0 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.362+0000 7f43deffd640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43c0066f90 0x7f43c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.362+0000 7f43deffd640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43c0066f90 0x7f43c0069450 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009a20 tx=0x7f43d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.366+0000 7f43d7fff640 1 -- 172.21.15.133:0/208926555 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43c8019030 con 0x7f43e0150ef0 2024-04-25T10:06:39.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 -- 172.21.15.133:0/208926555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43e01589f0 con 0x7f43e0150ef0 2024-04-25T10:06:39.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43d7fff640 1 -- 172.21.15.133:0/208926555 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f43c8056950 con 0x7f43e0150ef0 2024-04-25T10:06:39.549 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:39.549 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3081100850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3081100850},{"type":"v1","addr":"172.21.15.162:6834","nonce":3081100850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/272966073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":272966073},{"type":"v1","addr":"172.21.15.162:6835","nonce":272966073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2840204486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2840204486},{"type":"v1","addr":"172.21.15.133:6835","nonce":2840204486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:36.516727+0000","modified":"2024-04-25T10:06:38.534667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/236214118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":236214118},{"type":"v1","addr":"172.21.15.133:6837","nonce":236214118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:39.549 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 33 2024-04-25T10:06:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 -- 172.21.15.133:0/208926555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43c0066f90 msgr2=0x7f43c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43c0066f90 0x7f43c0069450 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009a20 tx=0x7f43d0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 -- 172.21.15.133:0/208926555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43e0150ef0 msgr2=0x7f43e0142930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43e0150ef0 0x7f43e0142930 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f43c8011690 tx=0x7f43c8011b50 comp rx=0 tx=0).stop 2024-04-25T10:06:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 -- 172.21.15.133:0/208926555 shutdown_connections 2024-04-25T10:06:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43e013f050 0x7f43e013f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43c0066f90 0x7f43c0069450 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43e0150ef0 0x7f43e0142930 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 --2- 172.21.15.133:0/208926555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43e0142550 0x7f43e0159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 -- 172.21.15.133:0/208926555 >> 172.21.15.133:0/208926555 conn(0x7f43e0067000 msgr2=0x7f43e00cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 -- 172.21.15.133:0/208926555 shutdown_connections 2024-04-25T10:06:39.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.546+0000 7f43e5446640 1 -- 172.21.15.133:0/208926555 wait complete. 2024-04-25T10:06:39.568 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:39.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.674+0000 7f56f70a3640 1 Processor -- start 2024-04-25T10:06:39.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.674+0000 7f56f70a3640 1 -- start start 2024-04-25T10:06:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.674+0000 7f56f70a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f015b9b0 0x7f56f00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.674+0000 7f56f70a3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f015bd90 con 0x7f56f00cf7c0 2024-04-25T10:06:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.674+0000 7f56f70a3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f009ad10 con 0x7f56f00d0380 2024-04-25T10:06:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.674+0000 7f56f70a3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00c5610 con 0x7f56f015b9b0 2024-04-25T10:06:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f56f00cf7c0 0x7f56f00cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57118/0 (socket says 172.21.15.133:57118) 2024-04-25T10:06:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e7fff640 1 -- 172.21.15.133:0/3416790261 learned_addr learned my addr 172.21.15.133:0/3416790261 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f4e3f640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f015b9b0 0x7f56f00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054837617 0 0) 0x7f56f015bd90 con 0x7f56f00cf7c0 2024-04-25T10:06:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d8003530 con 0x7f56f00cf7c0 2024-04-25T10:06:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4163404334 0 0) 0x7f56d8003530 con 0x7f56f00cf7c0 2024-04-25T10:06:39.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f015b9b0 msgr2=0x7f56f00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f015b9b0 0x7f56f00cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 >> v1:172.21.15.162:6789/0 conn(0x7f56f00d0380 legacy=0x7f56f00c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f009a750 con 0x7f56f00cf7c0 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f56e00016a0 con 0x7f56f00cf7c0 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 >> v1:172.21.15.133:6789/0 conn(0x7f56f00cf7c0 legacy=0x7f56f00cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 0x7f56d8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f015b9b0 0x7f56d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56d8004d50 0x7f56d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00c5610 con 0x7f56d80039f0 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d800ba80 con 0x7f56d8004d50 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d800bc80 con 0x7f56f015b9b0 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f4e3f640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 0x7f56d8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e7fff640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f015b9b0 0x7f56d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f4e3f640 1 -- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f015b9b0 msgr2=0x7f56d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f5640640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56d8004d50 0x7f56d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f4e3f640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f015b9b0 0x7f56d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f4e3f640 1 -- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56d8004d50 msgr2=0x7f56d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f4e3f640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56d8004d50 0x7f56d80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f4e3f640 1 -- 172.21.15.133:0/3416790261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d8003530 con 0x7f56d80039f0 2024-04-25T10:06:39.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e7fff640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f015b9b0 0x7f56d8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f4e3f640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 0x7f56d8003e10 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009ab0 tx=0x7f56ec01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50da6b03a3e7130b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0096b0 con 0x7f56d80039f0 2024-04-25T10:06:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ec0a6510 con 0x7f56d80039f0 2024-04-25T10:06:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56e77fe640 1 -- 172.21.15.133:0/3416790261 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0a6900 con 0x7f56d80039f0 2024-04-25T10:06:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f70a3640 1 -- 172.21.15.133:0/3416790261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 msgr2=0x7f56d8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f70a3640 1 --2- 172.21.15.133:0/3416790261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 0x7f56d8003e10 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009ab0 tx=0x7f56ec01ca20 comp rx=0 tx=0).stop 2024-04-25T10:06:39.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.678+0000 7f56f5640640 1 -- 172.21.15.133:0/3416790261 reap_dead start 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 -- 172.21.15.133:0/3416790261 shutdown_connections 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 -- 172.21.15.133:0/3416790261 >> 172.21.15.133:0/3416790261 conn(0x7f56f00b31a0 msgr2=0x7f56f00b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 -- 172.21.15.133:0/3416790261 shutdown_connections 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 -- 172.21.15.133:0/3416790261 wait complete. 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 Processor -- start 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 -- start start 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 0x7f56f00aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f00cabb0 0x7f56f00b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56f014f790 0x7f56f014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00c6b40 con 0x7f56d80039f0 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00c69c0 con 0x7f56f014f790 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00c6cc0 con 0x7f56f00cabb0 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f4e3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 0x7f56f00aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f4e3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 0x7f56f00aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43788/0 (socket says 172.21.15.133:43788) 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f4e3f640 1 -- 172.21.15.133:0/3129489624 learned_addr learned my addr 172.21.15.133:0/3129489624 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:39.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56e7fff640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f00cabb0 0x7f56f00b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f4e3f640 1 -- 172.21.15.133:0/3129489624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f00cabb0 msgr2=0x7f56f00b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f4e3f640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f00cabb0 0x7f56f00b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f5640640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56f014f790 0x7f56f014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f4e3f640 1 -- 172.21.15.133:0/3129489624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56f014f790 msgr2=0x7f56f014fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f4e3f640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56f014f790 0x7f56f014fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f4e3f640 1 -- 172.21.15.133:0/3129489624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f0150110 con 0x7f56d80039f0 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f4e3f640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 0x7f56f00aed90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009ab0 tx=0x7f56d8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56e57fa640 1 -- 172.21.15.133:0/3129489624 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec00f000 con 0x7f56d80039f0 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56e57fa640 1 -- 172.21.15.133:0/3129489624 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ec004020 con 0x7f56d80039f0 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56e57fa640 1 -- 172.21.15.133:0/3129489624 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec004410 con 0x7f56d80039f0 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 -- 172.21.15.133:0/3129489624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f00af420 con 0x7f56d80039f0 2024-04-25T10:06:39.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 -- 172.21.15.133:0/3129489624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f00af9e0 con 0x7f56d80039f0 2024-04-25T10:06:39.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.682+0000 7f56f70a3640 1 -- 172.21.15.133:0/3129489624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f0067c70 con 0x7f56d80039f0 2024-04-25T10:06:39.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.686+0000 7f56e57fa640 1 -- 172.21.15.133:0/3129489624 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56ec0c2020 con 0x7f56d80039f0 2024-04-25T10:06:39.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.686+0000 7f56e57fa640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56dc066f90 0x7f56dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.686+0000 7f56e57fa640 1 -- 172.21.15.133:0/3129489624 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56ec11f850 con 0x7f56d80039f0 2024-04-25T10:06:39.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.686+0000 7f56e7fff640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56dc066f90 0x7f56dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.686+0000 7f56e7fff640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56dc066f90 0x7f56dc069450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f56e0001810 tx=0x7f56e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.690+0000 7f56e57fa640 1 -- 172.21.15.133:0/3129489624 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f56ec016760 con 0x7f56d80039f0 2024-04-25T10:06:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.854+0000 7f56f70a3640 1 -- 172.21.15.133:0/3129489624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56f00c5aa0 con 0x7f56d80039f0 2024-04-25T10:06:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.854+0000 7f56e57fa640 1 -- 172.21.15.133:0/3129489624 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16908 (secure 0 0 0) 0x7f56ec0efff0 con 0x7f56d80039f0 2024-04-25T10:06:39.858 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:39.858 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":66,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:06:38.208663+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:34.764755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:35.504408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:39.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.854+0000 7f56f70a3640 1 -- 172.21.15.133:0/3129489624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56dc066f90 msgr2=0x7f56dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.854+0000 7f56f70a3640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56dc066f90 0x7f56dc069450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f56e0001810 tx=0x7f56e0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:39.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.854+0000 7f56f70a3640 1 -- 172.21.15.133:0/3129489624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 msgr2=0x7f56f00aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.854+0000 7f56f70a3640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 0x7f56f00aed90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009ab0 tx=0x7f56d8005840 comp rx=0 tx=0).stop 2024-04-25T10:06:39.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.858+0000 7f56f70a3640 1 -- 172.21.15.133:0/3129489624 shutdown_connections 2024-04-25T10:06:39.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.858+0000 7f56f70a3640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56f014f790 0x7f56f014fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.858+0000 7f56f70a3640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56f00cabb0 0x7f56f00b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.858+0000 7f56f70a3640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56dc066f90 0x7f56dc069450 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.858+0000 7f56f70a3640 1 --2- 172.21.15.133:0/3129489624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d80039f0 0x7f56f00aed90 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.858+0000 7f56f70a3640 1 -- 172.21.15.133:0/3129489624 >> 172.21.15.133:0/3129489624 conn(0x7f56f00b31a0 msgr2=0x7f56f00708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.858+0000 7f56f70a3640 1 -- 172.21.15.133:0/3129489624 shutdown_connections 2024-04-25T10:06:39.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.858+0000 7f56f70a3640 1 -- 172.21.15.133:0/3129489624 wait complete. 2024-04-25T10:06:39.877 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:06:39.877 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:39.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.978+0000 7f057324b640 1 Processor -- start 2024-04-25T10:06:39.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.978+0000 7f057324b640 1 -- start start 2024-04-25T10:06:39.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f057324b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c15c9a0 0x7f056c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f057324b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f056c0ce330 0x7f056c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f057324b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056c0c61d0 con 0x7f056c0b1f70 2024-04-25T10:06:39.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f057324b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056c0cece0 con 0x7f056c0ce330 2024-04-25T10:06:39.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f057324b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0c64b0 con 0x7f056c15c9a0 2024-04-25T10:06:39.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f056c0b1f70 0x7f056c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57122/0 (socket says 172.21.15.133:57122) 2024-04-25T10:06:39.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056bfff640 1 -- 172.21.15.133:0/780728747 learned_addr learned my addr 172.21.15.133:0/780728747 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:39.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f05717e8640 1 --2- 172.21.15.133:0/780728747 >> v2:172.21.15.162:3300/0 conn(0x7f056c0ce330 0x7f056c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774468196 0 0) 0x7f056c0c61d0 con 0x7f056c0b1f70 2024-04-25T10:06:39.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f0570fe7640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c15c9a0 0x7f056c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0554003530 con 0x7f056c0b1f70 2024-04-25T10:06:39.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3276257649 0 0) 0x7f0554003530 con 0x7f056c0b1f70 2024-04-25T10:06:39.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c15c9a0 msgr2=0x7f056c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c15c9a0 0x7f056c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 >> v2:172.21.15.162:3300/0 conn(0x7f056c0ce330 msgr2=0x7f056c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 --2- 172.21.15.133:0/780728747 >> v2:172.21.15.162:3300/0 conn(0x7f056c0ce330 0x7f056c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c06d4f0 con 0x7f056c0b1f70 2024-04-25T10:06:39.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0560001680 con 0x7f056c0b1f70 2024-04-25T10:06:39.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 >> v1:172.21.15.133:6789/0 conn(0x7f056c0b1f70 legacy=0x7f056c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05540039c0 0x7f0554003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0554004810 0x7f0554004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c15c9a0 0x7f05540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0cece0 con 0x7f05540039c0 2024-04-25T10:06:39.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0c64b0 con 0x7f0554004810 2024-04-25T10:06:39.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055400b9d0 con 0x7f056c15c9a0 2024-04-25T10:06:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f0570fe7640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05540039c0 0x7f0554003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f056bfff640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0554004810 0x7f0554004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f05717e8640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c15c9a0 0x7f05540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f0570fe7640 1 -- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c15c9a0 msgr2=0x7f05540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f0570fe7640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c15c9a0 0x7f05540051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f0570fe7640 1 -- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0554004810 msgr2=0x7f0554004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f0570fe7640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0554004810 0x7f0554004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f0570fe7640 1 -- 172.21.15.133:0/780728747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0554003530 con 0x7f05540039c0 2024-04-25T10:06:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.982+0000 7f05717e8640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c15c9a0 0x7f05540051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:39.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f0570fe7640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05540039c0 0x7f0554003de0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f055c00ba30 tx=0x7f055c01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5e36137eeafd3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0b2070 con 0x7f05540039c0 2024-04-25T10:06:39.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c0ad410 con 0x7f05540039c0 2024-04-25T10:06:39.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f056b7fe640 1 -- 172.21.15.133:0/780728747 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c00b600 con 0x7f05540039c0 2024-04-25T10:06:39.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 -- 172.21.15.133:0/780728747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05540039c0 msgr2=0x7f0554003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 --2- 172.21.15.133:0/780728747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05540039c0 0x7f0554003de0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f055c00ba30 tx=0x7f055c01c6a0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05717e8640 1 -- 172.21.15.133:0/780728747 reap_dead start 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 -- 172.21.15.133:0/780728747 shutdown_connections 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 -- 172.21.15.133:0/780728747 >> 172.21.15.133:0/780728747 conn(0x7f056c067000 msgr2=0x7f056c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 -- 172.21.15.133:0/780728747 shutdown_connections 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 -- 172.21.15.133:0/780728747 wait complete. 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 Processor -- start 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 -- start start 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f056c0b05b0 0x7f056c14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c0c8460 0x7f056c15b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f056c15b870 0x7f056c14b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c15d950 con 0x7f056c15b870 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c15dad0 con 0x7f056c0b05b0 2024-04-25T10:06:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c15dc50 con 0x7f056c0c8460 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05717e8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f056c15b870 0x7f056c14b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05717e8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f056c15b870 0x7f056c14b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43804/0 (socket says 172.21.15.133:43804) 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05717e8640 1 -- 172.21.15.133:0/1911749317 learned_addr learned my addr 172.21.15.133:0/1911749317 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f056bfff640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c0c8460 0x7f056c15b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f0570fe7640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f056c0b05b0 0x7f056c14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05717e8640 1 -- 172.21.15.133:0/1911749317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c0c8460 msgr2=0x7f056c15b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05717e8640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c0c8460 0x7f056c15b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05717e8640 1 -- 172.21.15.133:0/1911749317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f056c0b05b0 msgr2=0x7f056c14c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05717e8640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f056c0b05b0 0x7f056c14c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05717e8640 1 -- 172.21.15.133:0/1911749317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c149e40 con 0x7f056c15b870 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05717e8640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f056c15b870 0x7f056c14b180 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f056c0cc0f0 tx=0x7f056400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05697fa640 1 -- 172.21.15.133:0/1911749317 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056401c070 con 0x7f056c15b870 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05697fa640 1 -- 172.21.15.133:0/1911749317 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0564017400 con 0x7f056c15b870 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 -- 172.21.15.133:0/1911749317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c0c8a30 con 0x7f056c15b870 2024-04-25T10:06:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f05697fa640 1 -- 172.21.15.133:0/1911749317 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05640177d0 con 0x7f056c15b870 2024-04-25T10:06:39.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.986+0000 7f057324b640 1 -- 172.21.15.133:0/1911749317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c14a0e0 con 0x7f056c15b870 2024-04-25T10:06:39.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.990+0000 7f057324b640 1 -- 172.21.15.133:0/1911749317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c068ef0 con 0x7f056c15b870 2024-04-25T10:06:39.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.990+0000 7f05697fa640 1 -- 172.21.15.133:0/1911749317 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f056401f070 con 0x7f056c15b870 2024-04-25T10:06:39.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.990+0000 7f05697fa640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0558066f60 0x7f0558069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.990+0000 7f05697fa640 1 -- 172.21.15.133:0/1911749317 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f05640851a0 con 0x7f056c15b870 2024-04-25T10:06:39.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.990+0000 7f0570fe7640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0558066f60 0x7f0558069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.990+0000 7f0570fe7640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0558066f60 0x7f0558069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f055c0af000 tx=0x7f055c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:39.994+0000 7f05697fa640 1 -- 172.21.15.133:0/1911749317 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0564012070 con 0x7f056c15b870 2024-04-25T10:06:40.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.138+0000 7f057324b640 1 -- 172.21.15.133:0/1911749317 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f056c15ea10 con 0x7f0558066f60 2024-04-25T10:06:40.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.154+0000 7f05697fa640 1 -- 172.21.15.133:0/1911749317 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252027 (secure 0 0 0) 0x7f056c15ea10 con 0x7f0558066f60 2024-04-25T10:06:40.162 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:40.164 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:06:40.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 -- 172.21.15.133:0/1911749317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0558066f60 msgr2=0x7f0558069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0558066f60 0x7f0558069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f055c0af000 tx=0x7f055c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 -- 172.21.15.133:0/1911749317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f056c15b870 msgr2=0x7f056c14b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f056c15b870 0x7f056c14b180 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f056c0cc0f0 tx=0x7f056400f4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 -- 172.21.15.133:0/1911749317 shutdown_connections 2024-04-25T10:06:40.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0558066f60 0x7f0558069420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f056c15b870 0x7f056c14b180 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f056c0c8460 0x7f056c15b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 --2- 172.21.15.133:0/1911749317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f056c0b05b0 0x7f056c14c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 -- 172.21.15.133:0/1911749317 >> 172.21.15.133:0/1911749317 conn(0x7f056c067000 msgr2=0x7f056c15a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 -- 172.21.15.133:0/1911749317 shutdown_connections 2024-04-25T10:06:40.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.158+0000 7f057324b640 1 -- 172.21.15.133:0/1911749317 wait complete. 2024-04-25T10:06:40.180 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-04-25T10:06:39.794094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":340192,"kb_used_data":7464,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754634528,"statfs":{"total":773094113280,"available":772745756672,"internally_reserved":0,"allocated":7643136,"data_stored":4084924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":340577841},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.652834"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328674+0000","last_change":"2024-04-25T10:06:37.259345+0000","last_active":"2024-04-25T10:06:38.328674+0000","last_peered":"2024-04-25T10:06:38.328674+0000","last_clean":"2024-04-25T10:06:38.328674+0000","last_became_active":"2024-04-25T10:06:37.258816+0000","last_became_peered":"2024-04-25T10:06:37.258816+0000","last_unstale":"2024-04-25T10:06:38.328674+0000","last_undegraded":"2024-04-25T10:06:38.328674+0000","last_fullsized":"2024-04-25T10:06:38.328674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:06.890930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327702+0000","last_change":"2024-04-25T10:06:37.229642+0000","last_active":"2024-04-25T10:06:38.327702+0000","last_peered":"2024-04-25T10:06:38.327702+0000","last_clean":"2024-04-25T10:06:38.327702+0000","last_became_active":"2024-04-25T10:06:37.229336+0000","last_became_peered":"2024-04-25T10:06:37.229336+0000","last_unstale":"2024-04-25T10:06:38.327702+0000","last_undegraded":"2024-04-25T10:06:38.327702+0000","last_fullsized":"2024-04-25T10:06:38.327702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:31.642221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326705+0000","last_change":"2024-04-25T10:06:37.244256+0000","last_active":"2024-04-25T10:06:38.326705+0000","last_peered":"2024-04-25T10:06:38.326705+0000","last_clean":"2024-04-25T10:06:38.326705+0000","last_became_active":"2024-04-25T10:06:37.243703+0000","last_became_peered":"2024-04-25T10:06:37.243703+0000","last_unstale":"2024-04-25T10:06:38.326705+0000","last_undegraded":"2024-04-25T10:06:38.326705+0000","last_fullsized":"2024-04-25T10:06:38.326705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:07.217705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220937+0000","last_change":"2024-04-25T10:06:37.215123+0000","last_active":"2024-04-25T10:06:38.220937+0000","last_peered":"2024-04-25T10:06:38.220937+0000","last_clean":"2024-04-25T10:06:38.220937+0000","last_became_active":"2024-04-25T10:06:37.212773+0000","last_became_peered":"2024-04-25T10:06:37.212773+0000","last_unstale":"2024-04-25T10:06:38.220937+0000","last_undegraded":"2024-04-25T10:06:38.220937+0000","last_fullsized":"2024-04-25T10:06:38.220937+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:22.292593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221408+0000","last_change":"2024-04-25T10:06:37.212173+0000","last_active":"2024-04-25T10:06:38.221408+0000","last_peered":"2024-04-25T10:06:38.221408+0000","last_clean":"2024-04-25T10:06:38.221408+0000","last_became_active":"2024-04-25T10:06:37.211383+0000","last_became_peered":"2024-04-25T10:06:37.211383+0000","last_unstale":"2024-04-25T10:06:38.221408+0000","last_undegraded":"2024-04-25T10:06:38.221408+0000","last_fullsized":"2024-04-25T10:06:38.221408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:40.032710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219108+0000","last_change":"2024-04-25T10:06:37.230922+0000","last_active":"2024-04-25T10:06:38.219108+0000","last_peered":"2024-04-25T10:06:38.219108+0000","last_clean":"2024-04-25T10:06:38.219108+0000","last_became_active":"2024-04-25T10:06:37.230596+0000","last_became_peered":"2024-04-25T10:06:37.230596+0000","last_unstale":"2024-04-25T10:06:38.219108+0000","last_undegraded":"2024-04-25T10:06:38.219108+0000","last_fullsized":"2024-04-25T10:06:38.219108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:40.225911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.329170+0000","last_change":"2024-04-25T10:06:37.235039+0000","last_active":"2024-04-25T10:06:38.329170+0000","last_peered":"2024-04-25T10:06:38.329170+0000","last_clean":"2024-04-25T10:06:38.329170+0000","last_became_active":"2024-04-25T10:06:37.234713+0000","last_became_peered":"2024-04-25T10:06:37.234713+0000","last_unstale":"2024-04-25T10:06:38.329170+0000","last_undegraded":"2024-04-25T10:06:38.329170+0000","last_fullsized":"2024-04-25T10:06:38.329170+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:07.241324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325297+0000","last_change":"2024-04-25T10:06:37.246081+0000","last_active":"2024-04-25T10:06:38.325297+0000","last_peered":"2024-04-25T10:06:38.325297+0000","last_clean":"2024-04-25T10:06:38.325297+0000","last_became_active":"2024-04-25T10:06:37.245546+0000","last_became_peered":"2024-04-25T10:06:37.245546+0000","last_unstale":"2024-04-25T10:06:38.325297+0000","last_undegraded":"2024-04-25T10:06:38.325297+0000","last_fullsized":"2024-04-25T10:06:38.325297+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:05.786554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327090+0000","last_change":"2024-04-25T10:06:37.253860+0000","last_active":"2024-04-25T10:06:38.327090+0000","last_peered":"2024-04-25T10:06:38.327090+0000","last_clean":"2024-04-25T10:06:38.327090+0000","last_became_active":"2024-04-25T10:06:37.253261+0000","last_became_peered":"2024-04-25T10:06:37.253261+0000","last_unstale":"2024-04-25T10:06:38.327090+0000","last_undegraded":"2024-04-25T10:06:38.327090+0000","last_fullsized":"2024-04-25T10:06:38.327090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:47.119623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327341+0000","last_change":"2024-04-25T10:06:37.232517+0000","last_active":"2024-04-25T10:06:38.327341+0000","last_peered":"2024-04-25T10:06:38.327341+0000","last_clean":"2024-04-25T10:06:38.327341+0000","last_became_active":"2024-04-25T10:06:37.231472+0000","last_became_peered":"2024-04-25T10:06:37.231472+0000","last_unstale":"2024-04-25T10:06:38.327341+0000","last_undegraded":"2024-04-25T10:06:38.327341+0000","last_fullsized":"2024-04-25T10:06:38.327341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:26.595418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221279+0000","last_change":"2024-04-25T10:06:37.225407+0000","last_active":"2024-04-25T10:06:38.221279+0000","last_peered":"2024-04-25T10:06:38.221279+0000","last_clean":"2024-04-25T10:06:38.221279+0000","last_became_active":"2024-04-25T10:06:37.224684+0000","last_became_peered":"2024-04-25T10:06:37.224684+0000","last_unstale":"2024-04-25T10:06:38.221279+0000","last_undegraded":"2024-04-25T10:06:38.221279+0000","last_fullsized":"2024-04-25T10:06:38.221279+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:41.228850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325457+0000","last_change":"2024-04-25T10:06:37.246701+0000","last_active":"2024-04-25T10:06:38.325457+0000","last_peered":"2024-04-25T10:06:38.325457+0000","last_clean":"2024-04-25T10:06:38.325457+0000","last_became_active":"2024-04-25T10:06:37.240383+0000","last_became_peered":"2024-04-25T10:06:37.240383+0000","last_unstale":"2024-04-25T10:06:38.325457+0000","last_undegraded":"2024-04-25T10:06:38.325457+0000","last_fullsized":"2024-04-25T10:06:38.325457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:51.322072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325415+0000","last_change":"2024-04-25T10:06:37.255444+0000","last_active":"2024-04-25T10:06:38.325415+0000","last_peered":"2024-04-25T10:06:38.325415+0000","last_clean":"2024-04-25T10:06:38.325415+0000","last_became_active":"2024-04-25T10:06:37.254910+0000","last_became_peered":"2024-04-25T10:06:37.254910+0000","last_unstale":"2024-04-25T10:06:38.325415+0000","last_undegraded":"2024-04-25T10:06:38.325415+0000","last_fullsized":"2024-04-25T10:06:38.325415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:40.225911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327767+0000","last_change":"2024-04-25T10:06:37.228259+0000","last_active":"2024-04-25T10:06:38.327767+0000","last_peered":"2024-04-25T10:06:38.327767+0000","last_clean":"2024-04-25T10:06:38.327767+0000","last_became_active":"2024-04-25T10:06:37.227701+0000","last_became_peered":"2024-04-25T10:06:37.227701+0000","last_unstale":"2024-04-25T10:06:38.327767+0000","last_undegraded":"2024-04-25T10:06:38.327767+0000","last_fullsized":"2024-04-25T10:06:38.327767+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:03.343765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326391+0000","last_change":"2024-04-25T10:06:37.236842+0000","last_active":"2024-04-25T10:06:38.326391+0000","last_peered":"2024-04-25T10:06:38.326391+0000","last_clean":"2024-04-25T10:06:38.326391+0000","last_became_active":"2024-04-25T10:06:37.235846+0000","last_became_peered":"2024-04-25T10:06:37.235846+0000","last_unstale":"2024-04-25T10:06:38.326391+0000","last_undegraded":"2024-04-25T10:06:38.326391+0000","last_fullsized":"2024-04-25T10:06:38.326391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:11.227591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328148+0000","last_change":"2024-04-25T10:06:37.215757+0000","last_active":"2024-04-25T10:06:38.328148+0000","last_peered":"2024-04-25T10:06:38.328148+0000","last_clean":"2024-04-25T10:06:38.328148+0000","last_became_active":"2024-04-25T10:06:37.215407+0000","last_became_peered":"2024-04-25T10:06:37.215407+0000","last_unstale":"2024-04-25T10:06:38.328148+0000","last_undegraded":"2024-04-25T10:06:38.328148+0000","last_fullsized":"2024-04-25T10:06:38.328148+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:25.898592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.329104+0000","last_change":"2024-04-25T10:06:37.250652+0000","last_active":"2024-04-25T10:06:38.329104+0000","last_peered":"2024-04-25T10:06:38.329104+0000","last_clean":"2024-04-25T10:06:38.329104+0000","last_became_active":"2024-04-25T10:06:37.250350+0000","last_became_peered":"2024-04-25T10:06:37.250350+0000","last_unstale":"2024-04-25T10:06:38.329104+0000","last_undegraded":"2024-04-25T10:06:38.329104+0000","last_fullsized":"2024-04-25T10:06:38.329104+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:56.369469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220966+0000","last_change":"2024-04-25T10:06:37.230330+0000","last_active":"2024-04-25T10:06:38.220966+0000","last_peered":"2024-04-25T10:06:38.220966+0000","last_clean":"2024-04-25T10:06:38.220966+0000","last_became_active":"2024-04-25T10:06:37.229121+0000","last_became_peered":"2024-04-25T10:06:37.229121+0000","last_unstale":"2024-04-25T10:06:38.220966+0000","last_undegraded":"2024-04-25T10:06:38.220966+0000","last_fullsized":"2024-04-25T10:06:38.220966+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:36.845800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220655+0000","last_change":"2024-04-25T10:06:37.206460+0000","last_active":"2024-04-25T10:06:38.220655+0000","last_peered":"2024-04-25T10:06:38.220655+0000","last_clean":"2024-04-25T10:06:38.220655+0000","last_became_active":"2024-04-25T10:06:37.205780+0000","last_became_peered":"2024-04-25T10:06:37.205780+0000","last_unstale":"2024-04-25T10:06:38.220655+0000","last_undegraded":"2024-04-25T10:06:38.220655+0000","last_fullsized":"2024-04-25T10:06:38.220655+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:54.228961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326339+0000","last_change":"2024-04-25T10:06:37.227423+0000","last_active":"2024-04-25T10:06:38.326339+0000","last_peered":"2024-04-25T10:06:38.326339+0000","last_clean":"2024-04-25T10:06:38.326339+0000","last_became_active":"2024-04-25T10:06:37.227089+0000","last_became_peered":"2024-04-25T10:06:37.227089+0000","last_unstale":"2024-04-25T10:06:38.326339+0000","last_undegraded":"2024-04-25T10:06:38.326339+0000","last_fullsized":"2024-04-25T10:06:38.326339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:26.595418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325850+0000","last_change":"2024-04-25T10:06:37.250442+0000","last_active":"2024-04-25T10:06:38.325850+0000","last_peered":"2024-04-25T10:06:38.325850+0000","last_clean":"2024-04-25T10:06:38.325850+0000","last_became_active":"2024-04-25T10:06:37.247920+0000","last_became_peered":"2024-04-25T10:06:37.247920+0000","last_unstale":"2024-04-25T10:06:38.325850+0000","last_undegraded":"2024-04-25T10:06:38.325850+0000","last_fullsized":"2024-04-25T10:06:38.325850+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:50.285859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326296+0000","last_change":"2024-04-25T10:06:37.235003+0000","last_active":"2024-04-25T10:06:38.326296+0000","last_peered":"2024-04-25T10:06:38.326296+0000","last_clean":"2024-04-25T10:06:38.326296+0000","last_became_active":"2024-04-25T10:06:37.234585+0000","last_became_peered":"2024-04-25T10:06:37.234585+0000","last_unstale":"2024-04-25T10:06:38.326296+0000","last_undegraded":"2024-04-25T10:06:38.326296+0000","last_fullsized":"2024-04-25T10:06:38.326296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:25.576789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326090+0000","last_change":"2024-04-25T10:06:37.234991+0000","last_active":"2024-04-25T10:06:38.326090+0000","last_peered":"2024-04-25T10:06:38.326090+0000","last_clean":"2024-04-25T10:06:38.326090+0000","last_became_active":"2024-04-25T10:06:37.234674+0000","last_became_peered":"2024-04-25T10:06:37.234674+0000","last_unstale":"2024-04-25T10:06:38.326090+0000","last_undegraded":"2024-04-25T10:06:38.326090+0000","last_fullsized":"2024-04-25T10:06:38.326090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:07.217705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325588+0000","last_change":"2024-04-25T10:06:37.237161+0000","last_active":"2024-04-25T10:06:38.325588+0000","last_peered":"2024-04-25T10:06:38.325588+0000","last_clean":"2024-04-25T10:06:38.325588+0000","last_became_active":"2024-04-25T10:06:37.236611+0000","last_became_peered":"2024-04-25T10:06:37.236611+0000","last_unstale":"2024-04-25T10:06:38.325588+0000","last_undegraded":"2024-04-25T10:06:38.325588+0000","last_fullsized":"2024-04-25T10:06:38.325588+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:50.285859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326757+0000","last_change":"2024-04-25T10:06:37.245158+0000","last_active":"2024-04-25T10:06:38.326757+0000","last_peered":"2024-04-25T10:06:38.326757+0000","last_clean":"2024-04-25T10:06:38.326757+0000","last_became_active":"2024-04-25T10:06:37.240168+0000","last_became_peered":"2024-04-25T10:06:37.240168+0000","last_unstale":"2024-04-25T10:06:38.326757+0000","last_undegraded":"2024-04-25T10:06:38.326757+0000","last_fullsized":"2024-04-25T10:06:38.326757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:07.217705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325914+0000","last_change":"2024-04-25T10:06:37.258424+0000","last_active":"2024-04-25T10:06:38.325914+0000","last_peered":"2024-04-25T10:06:38.325914+0000","last_clean":"2024-04-25T10:06:38.325914+0000","last_became_active":"2024-04-25T10:06:37.257227+0000","last_became_peered":"2024-04-25T10:06:37.257227+0000","last_unstale":"2024-04-25T10:06:38.325914+0000","last_undegraded":"2024-04-25T10:06:38.325914+0000","last_fullsized":"2024-04-25T10:06:38.325914+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:15.989745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220689+0000","last_change":"2024-04-25T10:06:37.213722+0000","last_active":"2024-04-25T10:06:38.220689+0000","last_peered":"2024-04-25T10:06:38.220689+0000","last_clean":"2024-04-25T10:06:38.220689+0000","last_became_active":"2024-04-25T10:06:37.213011+0000","last_became_peered":"2024-04-25T10:06:37.213011+0000","last_unstale":"2024-04-25T10:06:38.220689+0000","last_undegraded":"2024-04-25T10:06:38.220689+0000","last_fullsized":"2024-04-25T10:06:38.220689+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:52.879097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328249+0000","last_change":"2024-04-25T10:06:37.228946+0000","last_active":"2024-04-25T10:06:38.328249+0000","last_peered":"2024-04-25T10:06:38.328249+0000","last_clean":"2024-04-25T10:06:38.328249+0000","last_became_active":"2024-04-25T10:06:37.228472+0000","last_became_peered":"2024-04-25T10:06:37.228472+0000","last_unstale":"2024-04-25T10:06:38.328249+0000","last_undegraded":"2024-04-25T10:06:38.328249+0000","last_fullsized":"2024-04-25T10:06:38.328249+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:45.902250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325934+0000","last_change":"2024-04-25T10:06:37.249491+0000","last_active":"2024-04-25T10:06:38.325934+0000","last_peered":"2024-04-25T10:06:38.325934+0000","last_clean":"2024-04-25T10:06:38.325934+0000","last_became_active":"2024-04-25T10:06:37.248997+0000","last_became_peered":"2024-04-25T10:06:37.248997+0000","last_unstale":"2024-04-25T10:06:38.325934+0000","last_undegraded":"2024-04-25T10:06:38.325934+0000","last_fullsized":"2024-04-25T10:06:38.325934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:15.989745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325386+0000","last_change":"2024-04-25T10:06:37.247431+0000","last_active":"2024-04-25T10:06:38.325386+0000","last_peered":"2024-04-25T10:06:38.325386+0000","last_clean":"2024-04-25T10:06:38.325386+0000","last_became_active":"2024-04-25T10:06:37.246798+0000","last_became_peered":"2024-04-25T10:06:37.246798+0000","last_unstale":"2024-04-25T10:06:38.325386+0000","last_undegraded":"2024-04-25T10:06:38.325386+0000","last_fullsized":"2024-04-25T10:06:38.325386+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.392643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221058+0000","last_change":"2024-04-25T10:06:37.207508+0000","last_active":"2024-04-25T10:06:38.221058+0000","last_peered":"2024-04-25T10:06:38.221058+0000","last_clean":"2024-04-25T10:06:38.221058+0000","last_became_active":"2024-04-25T10:06:37.206721+0000","last_became_peered":"2024-04-25T10:06:37.206721+0000","last_unstale":"2024-04-25T10:06:38.221058+0000","last_undegraded":"2024-04-25T10:06:38.221058+0000","last_fullsized":"2024-04-25T10:06:38.221058+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:06.763393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219104+0000","last_change":"2024-04-25T10:06:37.220236+0000","last_active":"2024-04-25T10:06:38.219104+0000","last_peered":"2024-04-25T10:06:38.219104+0000","last_clean":"2024-04-25T10:06:38.219104+0000","last_became_active":"2024-04-25T10:06:37.219720+0000","last_became_peered":"2024-04-25T10:06:37.219720+0000","last_unstale":"2024-04-25T10:06:38.219104+0000","last_undegraded":"2024-04-25T10:06:38.219104+0000","last_fullsized":"2024-04-25T10:06:38.219104+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:51.322072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219655+0000","last_change":"2024-04-25T10:06:37.239433+0000","last_active":"2024-04-25T10:06:38.219655+0000","last_peered":"2024-04-25T10:06:38.219655+0000","last_clean":"2024-04-25T10:06:38.219655+0000","last_became_active":"2024-04-25T10:06:37.238802+0000","last_became_peered":"2024-04-25T10:06:37.238802+0000","last_unstale":"2024-04-25T10:06:38.219655+0000","last_undegraded":"2024-04-25T10:06:38.219655+0000","last_fullsized":"2024-04-25T10:06:38.219655+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:06.763393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328881+0000","last_change":"2024-04-25T10:06:36.176032+0000","last_active":"2024-04-25T10:06:38.328881+0000","last_peered":"2024-04-25T10:06:38.328881+0000","last_clean":"2024-04-25T10:06:38.328881+0000","last_became_active":"2024-04-25T10:06:36.175719+0000","last_became_peered":"2024-04-25T10:06:36.175719+0000","last_unstale":"2024-04-25T10:06:38.328881+0000","last_undegraded":"2024-04-25T10:06:38.328881+0000","last_fullsized":"2024-04-25T10:06:38.328881+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:20.671802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221112+0000","last_change":"2024-04-25T10:06:37.224964+0000","last_active":"2024-04-25T10:06:38.221112+0000","last_peered":"2024-04-25T10:06:38.221112+0000","last_clean":"2024-04-25T10:06:38.221112+0000","last_became_active":"2024-04-25T10:06:37.223533+0000","last_became_peered":"2024-04-25T10:06:37.223533+0000","last_unstale":"2024-04-25T10:06:38.221112+0000","last_undegraded":"2024-04-25T10:06:38.221112+0000","last_fullsized":"2024-04-25T10:06:38.221112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:08.891150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219604+0000","last_change":"2024-04-25T10:06:36.188564+0000","last_active":"2024-04-25T10:06:38.219604+0000","last_peered":"2024-04-25T10:06:38.219604+0000","last_clean":"2024-04-25T10:06:38.219604+0000","last_became_active":"2024-04-25T10:06:36.188249+0000","last_became_peered":"2024-04-25T10:06:36.188249+0000","last_unstale":"2024-04-25T10:06:38.219604+0000","last_undegraded":"2024-04-25T10:06:38.219604+0000","last_fullsized":"2024-04-25T10:06:38.219604+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:15.337553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219644+0000","last_change":"2024-04-25T10:06:37.218032+0000","last_active":"2024-04-25T10:06:38.219644+0000","last_peered":"2024-04-25T10:06:38.219644+0000","last_clean":"2024-04-25T10:06:38.219644+0000","last_became_active":"2024-04-25T10:06:37.217672+0000","last_became_peered":"2024-04-25T10:06:37.217672+0000","last_unstale":"2024-04-25T10:06:38.219644+0000","last_undegraded":"2024-04-25T10:06:38.219644+0000","last_fullsized":"2024-04-25T10:06:38.219644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:50.285859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219272+0000","last_change":"2024-04-25T10:06:36.191963+0000","last_active":"2024-04-25T10:06:38.219272+0000","last_peered":"2024-04-25T10:06:38.219272+0000","last_clean":"2024-04-25T10:06:38.219272+0000","last_became_active":"2024-04-25T10:06:36.191556+0000","last_became_peered":"2024-04-25T10:06:36.191556+0000","last_unstale":"2024-04-25T10:06:38.219272+0000","last_undegraded":"2024-04-25T10:06:38.219272+0000","last_fullsized":"2024-04-25T10:06:38.219272+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:32.143965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326595+0000","last_change":"2024-04-25T10:06:37.244406+0000","last_active":"2024-04-25T10:06:38.326595+0000","last_peered":"2024-04-25T10:06:38.326595+0000","last_clean":"2024-04-25T10:06:38.326595+0000","last_became_active":"2024-04-25T10:06:37.243384+0000","last_became_peered":"2024-04-25T10:06:37.243384+0000","last_unstale":"2024-04-25T10:06:38.326595+0000","last_undegraded":"2024-04-25T10:06:38.326595+0000","last_fullsized":"2024-04-25T10:06:38.326595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:52.001010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219269+0000","last_change":"2024-04-25T10:06:36.165068+0000","last_active":"2024-04-25T10:06:38.219269+0000","last_peered":"2024-04-25T10:06:38.219269+0000","last_clean":"2024-04-25T10:06:38.219269+0000","last_became_active":"2024-04-25T10:06:36.164413+0000","last_became_peered":"2024-04-25T10:06:36.164413+0000","last_unstale":"2024-04-25T10:06:38.219269+0000","last_undegraded":"2024-04-25T10:06:38.219269+0000","last_fullsized":"2024-04-25T10:06:38.219269+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:21.437190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328520+0000","last_change":"2024-04-25T10:06:37.223090+0000","last_active":"2024-04-25T10:06:38.328520+0000","last_peered":"2024-04-25T10:06:38.328520+0000","last_clean":"2024-04-25T10:06:38.328520+0000","last_became_active":"2024-04-25T10:06:37.222380+0000","last_became_peered":"2024-04-25T10:06:37.222380+0000","last_unstale":"2024-04-25T10:06:38.328520+0000","last_undegraded":"2024-04-25T10:06:38.328520+0000","last_fullsized":"2024-04-25T10:06:38.328520+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:16.101412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326443+0000","last_change":"2024-04-25T10:06:36.160916+0000","last_active":"2024-04-25T10:06:38.326443+0000","last_peered":"2024-04-25T10:06:38.326443+0000","last_clean":"2024-04-25T10:06:38.326443+0000","last_became_active":"2024-04-25T10:06:36.160599+0000","last_became_peered":"2024-04-25T10:06:36.160599+0000","last_unstale":"2024-04-25T10:06:38.326443+0000","last_undegraded":"2024-04-25T10:06:38.326443+0000","last_fullsized":"2024-04-25T10:06:38.326443+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:45.389453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327117+0000","last_change":"2024-04-25T10:06:37.247914+0000","last_active":"2024-04-25T10:06:38.327117+0000","last_peered":"2024-04-25T10:06:38.327117+0000","last_clean":"2024-04-25T10:06:38.327117+0000","last_became_active":"2024-04-25T10:06:37.247539+0000","last_became_peered":"2024-04-25T10:06:37.247539+0000","last_unstale":"2024-04-25T10:06:38.327117+0000","last_undegraded":"2024-04-25T10:06:38.327117+0000","last_fullsized":"2024-04-25T10:06:38.327117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:13.906449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326447+0000","last_change":"2024-04-25T10:06:36.167219+0000","last_active":"2024-04-25T10:06:38.326447+0000","last_peered":"2024-04-25T10:06:38.326447+0000","last_clean":"2024-04-25T10:06:38.326447+0000","last_became_active":"2024-04-25T10:06:36.164924+0000","last_became_peered":"2024-04-25T10:06:36.164924+0000","last_unstale":"2024-04-25T10:06:38.326447+0000","last_undegraded":"2024-04-25T10:06:38.326447+0000","last_fullsized":"2024-04-25T10:06:38.326447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:21.133255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219392+0000","last_change":"2024-04-25T10:06:37.221660+0000","last_active":"2024-04-25T10:06:38.219392+0000","last_peered":"2024-04-25T10:06:38.219392+0000","last_clean":"2024-04-25T10:06:38.219392+0000","last_became_active":"2024-04-25T10:06:37.220347+0000","last_became_peered":"2024-04-25T10:06:37.220347+0000","last_unstale":"2024-04-25T10:06:38.219392+0000","last_undegraded":"2024-04-25T10:06:38.219392+0000","last_fullsized":"2024-04-25T10:06:38.219392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:22.292593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326478+0000","last_change":"2024-04-25T10:06:36.169910+0000","last_active":"2024-04-25T10:06:38.326478+0000","last_peered":"2024-04-25T10:06:38.326478+0000","last_clean":"2024-04-25T10:06:38.326478+0000","last_became_active":"2024-04-25T10:06:36.169316+0000","last_became_peered":"2024-04-25T10:06:36.169316+0000","last_unstale":"2024-04-25T10:06:38.326478+0000","last_undegraded":"2024-04-25T10:06:38.326478+0000","last_fullsized":"2024-04-25T10:06:38.326478+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:16.043879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326336+0000","last_change":"2024-04-25T10:06:37.248060+0000","last_active":"2024-04-25T10:06:38.326336+0000","last_peered":"2024-04-25T10:06:38.326336+0000","last_clean":"2024-04-25T10:06:38.326336+0000","last_became_active":"2024-04-25T10:06:37.247760+0000","last_became_peered":"2024-04-25T10:06:37.247760+0000","last_unstale":"2024-04-25T10:06:38.326336+0000","last_undegraded":"2024-04-25T10:06:38.326336+0000","last_fullsized":"2024-04-25T10:06:38.326336+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:47.119623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219064+0000","last_change":"2024-04-25T10:06:36.174512+0000","last_active":"2024-04-25T10:06:38.219064+0000","last_peered":"2024-04-25T10:06:38.219064+0000","last_clean":"2024-04-25T10:06:38.219064+0000","last_became_active":"2024-04-25T10:06:36.173707+0000","last_became_peered":"2024-04-25T10:06:36.173707+0000","last_unstale":"2024-04-25T10:06:38.219064+0000","last_undegraded":"2024-04-25T10:06:38.219064+0000","last_fullsized":"2024-04-25T10:06:38.219064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:03.286057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328440+0000","last_change":"2024-04-25T10:06:37.221981+0000","last_active":"2024-04-25T10:06:38.328440+0000","last_peered":"2024-04-25T10:06:38.328440+0000","last_clean":"2024-04-25T10:06:38.328440+0000","last_became_active":"2024-04-25T10:06:37.221656+0000","last_became_peered":"2024-04-25T10:06:37.221656+0000","last_unstale":"2024-04-25T10:06:38.328440+0000","last_undegraded":"2024-04-25T10:06:38.328440+0000","last_fullsized":"2024-04-25T10:06:38.328440+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:28.091464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327453+0000","last_change":"2024-04-25T10:06:36.189290+0000","last_active":"2024-04-25T10:06:38.327453+0000","last_peered":"2024-04-25T10:06:38.327453+0000","last_clean":"2024-04-25T10:06:38.327453+0000","last_became_active":"2024-04-25T10:06:36.188473+0000","last_became_peered":"2024-04-25T10:06:36.188473+0000","last_unstale":"2024-04-25T10:06:38.327453+0000","last_undegraded":"2024-04-25T10:06:38.327453+0000","last_fullsized":"2024-04-25T10:06:38.327453+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:47.723565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326637+0000","last_change":"2024-04-25T10:06:37.257919+0000","last_active":"2024-04-25T10:06:38.326637+0000","last_peered":"2024-04-25T10:06:38.326637+0000","last_clean":"2024-04-25T10:06:38.326637+0000","last_became_active":"2024-04-25T10:06:37.257564+0000","last_became_peered":"2024-04-25T10:06:37.257564+0000","last_unstale":"2024-04-25T10:06:38.326637+0000","last_undegraded":"2024-04-25T10:06:38.326637+0000","last_fullsized":"2024-04-25T10:06:38.326637+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:03.343765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220854+0000","last_change":"2024-04-25T10:06:36.164318+0000","last_active":"2024-04-25T10:06:38.220854+0000","last_peered":"2024-04-25T10:06:38.220854+0000","last_clean":"2024-04-25T10:06:38.220854+0000","last_became_active":"2024-04-25T10:06:36.163584+0000","last_became_peered":"2024-04-25T10:06:36.163584+0000","last_unstale":"2024-04-25T10:06:38.220854+0000","last_undegraded":"2024-04-25T10:06:38.220854+0000","last_fullsized":"2024-04-25T10:06:38.220854+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:03.286057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219688+0000","last_change":"2024-04-25T10:06:37.258558+0000","last_active":"2024-04-25T10:06:38.219688+0000","last_peered":"2024-04-25T10:06:38.219688+0000","last_clean":"2024-04-25T10:06:38.219688+0000","last_became_active":"2024-04-25T10:06:37.258170+0000","last_became_peered":"2024-04-25T10:06:37.258170+0000","last_unstale":"2024-04-25T10:06:38.219688+0000","last_undegraded":"2024-04-25T10:06:38.219688+0000","last_fullsized":"2024-04-25T10:06:38.219688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:08.891150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327539+0000","last_change":"2024-04-25T10:06:36.187933+0000","last_active":"2024-04-25T10:06:38.327539+0000","last_peered":"2024-04-25T10:06:38.327539+0000","last_clean":"2024-04-25T10:06:38.327539+0000","last_became_active":"2024-04-25T10:06:36.185970+0000","last_became_peered":"2024-04-25T10:06:36.185970+0000","last_unstale":"2024-04-25T10:06:38.327539+0000","last_undegraded":"2024-04-25T10:06:38.327539+0000","last_fullsized":"2024-04-25T10:06:38.327539+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:15.337553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221296+0000","last_change":"2024-04-25T10:06:37.230247+0000","last_active":"2024-04-25T10:06:38.221296+0000","last_peered":"2024-04-25T10:06:38.221296+0000","last_clean":"2024-04-25T10:06:38.221296+0000","last_became_active":"2024-04-25T10:06:37.229396+0000","last_became_peered":"2024-04-25T10:06:37.229396+0000","last_unstale":"2024-04-25T10:06:38.221296+0000","last_undegraded":"2024-04-25T10:06:38.221296+0000","last_fullsized":"2024-04-25T10:06:38.221296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:21.415350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328572+0000","last_change":"2024-04-25T10:06:36.183609+0000","last_active":"2024-04-25T10:06:38.328572+0000","last_peered":"2024-04-25T10:06:38.328572+0000","last_clean":"2024-04-25T10:06:38.328572+0000","last_became_active":"2024-04-25T10:06:36.183246+0000","last_became_peered":"2024-04-25T10:06:36.183246+0000","last_unstale":"2024-04-25T10:06:38.328572+0000","last_undegraded":"2024-04-25T10:06:38.328572+0000","last_fullsized":"2024-04-25T10:06:38.328572+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:13.897357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.218835+0000","last_change":"2024-04-25T10:06:37.239448+0000","last_active":"2024-04-25T10:06:38.218835+0000","last_peered":"2024-04-25T10:06:38.218835+0000","last_clean":"2024-04-25T10:06:38.218835+0000","last_became_active":"2024-04-25T10:06:37.238870+0000","last_became_peered":"2024-04-25T10:06:37.238870+0000","last_unstale":"2024-04-25T10:06:38.218835+0000","last_undegraded":"2024-04-25T10:06:38.218835+0000","last_fullsized":"2024-04-25T10:06:38.218835+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:02.425868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326861+0000","last_change":"2024-04-25T10:06:36.186599+0000","last_active":"2024-04-25T10:06:38.326861+0000","last_peered":"2024-04-25T10:06:38.326861+0000","last_clean":"2024-04-25T10:06:38.326861+0000","last_became_active":"2024-04-25T10:06:36.185813+0000","last_became_peered":"2024-04-25T10:06:36.185813+0000","last_unstale":"2024-04-25T10:06:38.326861+0000","last_undegraded":"2024-04-25T10:06:38.326861+0000","last_fullsized":"2024-04-25T10:06:38.326861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:21.437190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326536+0000","last_change":"2024-04-25T10:06:37.238060+0000","last_active":"2024-04-25T10:06:38.326536+0000","last_peered":"2024-04-25T10:06:38.326536+0000","last_clean":"2024-04-25T10:06:38.326536+0000","last_became_active":"2024-04-25T10:06:37.237536+0000","last_became_peered":"2024-04-25T10:06:37.237536+0000","last_unstale":"2024-04-25T10:06:38.326536+0000","last_undegraded":"2024-04-25T10:06:38.326536+0000","last_fullsized":"2024-04-25T10:06:38.326536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:52.001010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325659+0000","last_change":"2024-04-25T10:06:36.188672+0000","last_active":"2024-04-25T10:06:38.325659+0000","last_peered":"2024-04-25T10:06:38.325659+0000","last_clean":"2024-04-25T10:06:38.325659+0000","last_became_active":"2024-04-25T10:06:36.181630+0000","last_became_peered":"2024-04-25T10:06:36.181630+0000","last_unstale":"2024-04-25T10:06:38.325659+0000","last_undegraded":"2024-04-25T10:06:38.325659+0000","last_fullsized":"2024-04-25T10:06:38.325659+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:27.088275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219754+0000","last_change":"2024-04-25T10:06:37.220268+0000","last_active":"2024-04-25T10:06:38.219754+0000","last_peered":"2024-04-25T10:06:38.219754+0000","last_clean":"2024-04-25T10:06:38.219754+0000","last_became_active":"2024-04-25T10:06:37.219258+0000","last_became_peered":"2024-04-25T10:06:37.219258+0000","last_unstale":"2024-04-25T10:06:38.219754+0000","last_undegraded":"2024-04-25T10:06:38.219754+0000","last_fullsized":"2024-04-25T10:06:38.219754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:41.228850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326849+0000","last_change":"2024-04-25T10:06:36.186685+0000","last_active":"2024-04-25T10:06:38.326849+0000","last_peered":"2024-04-25T10:06:38.326849+0000","last_clean":"2024-04-25T10:06:38.326849+0000","last_became_active":"2024-04-25T10:06:36.186019+0000","last_became_peered":"2024-04-25T10:06:36.186019+0000","last_unstale":"2024-04-25T10:06:38.326849+0000","last_undegraded":"2024-04-25T10:06:38.326849+0000","last_fullsized":"2024-04-25T10:06:38.326849+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:32.143965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328540+0000","last_change":"2024-04-25T10:06:37.239667+0000","last_active":"2024-04-25T10:06:38.328540+0000","last_peered":"2024-04-25T10:06:38.328540+0000","last_clean":"2024-04-25T10:06:38.328540+0000","last_became_active":"2024-04-25T10:06:37.239341+0000","last_became_peered":"2024-04-25T10:06:37.239341+0000","last_unstale":"2024-04-25T10:06:38.328540+0000","last_undegraded":"2024-04-25T10:06:38.328540+0000","last_fullsized":"2024-04-25T10:06:38.328540+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:58.223769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.218761+0000","last_change":"2024-04-25T10:06:36.175797+0000","last_active":"2024-04-25T10:06:38.218761+0000","last_peered":"2024-04-25T10:06:38.218761+0000","last_clean":"2024-04-25T10:06:38.218761+0000","last_became_active":"2024-04-25T10:06:36.169195+0000","last_became_peered":"2024-04-25T10:06:36.169195+0000","last_unstale":"2024-04-25T10:06:38.218761+0000","last_undegraded":"2024-04-25T10:06:38.218761+0000","last_fullsized":"2024-04-25T10:06:38.218761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:28.539746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326972+0000","last_change":"2024-04-25T10:06:37.249705+0000","last_active":"2024-04-25T10:06:38.326972+0000","last_peered":"2024-04-25T10:06:38.326972+0000","last_clean":"2024-04-25T10:06:38.326972+0000","last_became_active":"2024-04-25T10:06:37.249086+0000","last_became_peered":"2024-04-25T10:06:37.249086+0000","last_unstale":"2024-04-25T10:06:38.326972+0000","last_undegraded":"2024-04-25T10:06:38.326972+0000","last_fullsized":"2024-04-25T10:06:38.326972+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:25.576789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327051+0000","last_change":"2024-04-25T10:06:36.194242+0000","last_active":"2024-04-25T10:06:38.327051+0000","last_peered":"2024-04-25T10:06:38.327051+0000","last_clean":"2024-04-25T10:06:38.327051+0000","last_became_active":"2024-04-25T10:06:36.189515+0000","last_became_peered":"2024-04-25T10:06:36.189515+0000","last_unstale":"2024-04-25T10:06:38.327051+0000","last_undegraded":"2024-04-25T10:06:38.327051+0000","last_fullsized":"2024-04-25T10:06:38.327051+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:05.887740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221031+0000","last_change":"2024-04-25T10:06:37.211856+0000","last_active":"2024-04-25T10:06:38.221031+0000","last_peered":"2024-04-25T10:06:38.221031+0000","last_clean":"2024-04-25T10:06:38.221031+0000","last_became_active":"2024-04-25T10:06:37.210185+0000","last_became_peered":"2024-04-25T10:06:37.210185+0000","last_unstale":"2024-04-25T10:06:38.221031+0000","last_undegraded":"2024-04-25T10:06:38.221031+0000","last_fullsized":"2024-04-25T10:06:38.221031+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:21.674991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327414+0000","last_change":"2024-04-25T10:06:36.186486+0000","last_active":"2024-04-25T10:06:38.327414+0000","last_peered":"2024-04-25T10:06:38.327414+0000","last_clean":"2024-04-25T10:06:38.327414+0000","last_became_active":"2024-04-25T10:06:36.185838+0000","last_became_peered":"2024-04-25T10:06:36.185838+0000","last_unstale":"2024-04-25T10:06:38.327414+0000","last_undegraded":"2024-04-25T10:06:38.327414+0000","last_fullsized":"2024-04-25T10:06:38.327414+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:55.366280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220597+0000","last_change":"2024-04-25T10:06:37.215393+0000","last_active":"2024-04-25T10:06:38.220597+0000","last_peered":"2024-04-25T10:06:38.220597+0000","last_clean":"2024-04-25T10:06:38.220597+0000","last_became_active":"2024-04-25T10:06:37.214876+0000","last_became_peered":"2024-04-25T10:06:37.214876+0000","last_unstale":"2024-04-25T10:06:38.220597+0000","last_undegraded":"2024-04-25T10:06:38.220597+0000","last_fullsized":"2024-04-25T10:06:38.220597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:05.743949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327305+0000","last_change":"2024-04-25T10:06:36.188577+0000","last_active":"2024-04-25T10:06:38.327305+0000","last_peered":"2024-04-25T10:06:38.327305+0000","last_clean":"2024-04-25T10:06:38.327305+0000","last_became_active":"2024-04-25T10:06:36.186800+0000","last_became_peered":"2024-04-25T10:06:36.186800+0000","last_unstale":"2024-04-25T10:06:38.327305+0000","last_undegraded":"2024-04-25T10:06:38.327305+0000","last_fullsized":"2024-04-25T10:06:38.327305+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:48.883589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219201+0000","last_change":"2024-04-25T10:06:37.223539+0000","last_active":"2024-04-25T10:06:38.219201+0000","last_peered":"2024-04-25T10:06:38.219201+0000","last_clean":"2024-04-25T10:06:38.219201+0000","last_became_active":"2024-04-25T10:06:37.222327+0000","last_became_peered":"2024-04-25T10:06:37.222327+0000","last_unstale":"2024-04-25T10:06:38.219201+0000","last_undegraded":"2024-04-25T10:06:38.219201+0000","last_fullsized":"2024-04-25T10:06:38.219201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:52.879097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326502+0000","last_change":"2024-04-25T10:06:36.168120+0000","last_active":"2024-04-25T10:06:38.326502+0000","last_peered":"2024-04-25T10:06:38.326502+0000","last_clean":"2024-04-25T10:06:38.326502+0000","last_became_active":"2024-04-25T10:06:36.167819+0000","last_became_peered":"2024-04-25T10:06:36.167819+0000","last_unstale":"2024-04-25T10:06:38.326502+0000","last_undegraded":"2024-04-25T10:06:38.326502+0000","last_fullsized":"2024-04-25T10:06:38.326502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:39.222722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219017+0000","last_change":"2024-04-25T10:06:37.220420+0000","last_active":"2024-04-25T10:06:38.219017+0000","last_peered":"2024-04-25T10:06:38.219017+0000","last_clean":"2024-04-25T10:06:38.219017+0000","last_became_active":"2024-04-25T10:06:37.220000+0000","last_became_peered":"2024-04-25T10:06:37.220000+0000","last_unstale":"2024-04-25T10:06:38.219017+0000","last_undegraded":"2024-04-25T10:06:38.219017+0000","last_fullsized":"2024-04-25T10:06:38.219017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.392643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219056+0000","last_change":"2024-04-25T10:06:36.170236+0000","last_active":"2024-04-25T10:06:38.219056+0000","last_peered":"2024-04-25T10:06:38.219056+0000","last_clean":"2024-04-25T10:06:38.219056+0000","last_became_active":"2024-04-25T10:06:36.169376+0000","last_became_peered":"2024-04-25T10:06:36.169376+0000","last_unstale":"2024-04-25T10:06:38.219056+0000","last_undegraded":"2024-04-25T10:06:38.219056+0000","last_fullsized":"2024-04-25T10:06:38.219056+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:55.366280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327777+0000","last_change":"2024-04-25T10:06:37.235344+0000","last_active":"2024-04-25T10:06:38.327777+0000","last_peered":"2024-04-25T10:06:38.327777+0000","last_clean":"2024-04-25T10:06:38.327777+0000","last_became_active":"2024-04-25T10:06:37.218220+0000","last_became_peered":"2024-04-25T10:06:37.218220+0000","last_unstale":"2024-04-25T10:06:38.327777+0000","last_undegraded":"2024-04-25T10:06:38.327777+0000","last_fullsized":"2024-04-25T10:06:38.327777+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:25.576789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325931+0000","last_change":"2024-04-25T10:06:36.193835+0000","last_active":"2024-04-25T10:06:38.325931+0000","last_peered":"2024-04-25T10:06:38.325931+0000","last_clean":"2024-04-25T10:06:38.325931+0000","last_became_active":"2024-04-25T10:06:36.189802+0000","last_became_peered":"2024-04-25T10:06:36.189802+0000","last_unstale":"2024-04-25T10:06:38.325931+0000","last_undegraded":"2024-04-25T10:06:38.325931+0000","last_fullsized":"2024-04-25T10:06:38.325931+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:57.220580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220127+0000","last_change":"2024-04-25T10:06:37.221054+0000","last_active":"2024-04-25T10:06:38.220127+0000","last_peered":"2024-04-25T10:06:38.220127+0000","last_clean":"2024-04-25T10:06:38.220127+0000","last_became_active":"2024-04-25T10:06:37.219584+0000","last_became_peered":"2024-04-25T10:06:37.219584+0000","last_unstale":"2024-04-25T10:06:38.220127+0000","last_undegraded":"2024-04-25T10:06:38.220127+0000","last_fullsized":"2024-04-25T10:06:38.220127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:14.900546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219007+0000","last_change":"2024-04-25T10:06:36.175591+0000","last_active":"2024-04-25T10:06:38.219007+0000","last_peered":"2024-04-25T10:06:38.219007+0000","last_clean":"2024-04-25T10:06:38.219007+0000","last_became_active":"2024-04-25T10:06:36.175299+0000","last_became_peered":"2024-04-25T10:06:36.175299+0000","last_unstale":"2024-04-25T10:06:38.219007+0000","last_undegraded":"2024-04-25T10:06:38.219007+0000","last_fullsized":"2024-04-25T10:06:38.219007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.733077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219018+0000","last_change":"2024-04-25T10:06:37.211950+0000","last_active":"2024-04-25T10:06:38.219018+0000","last_peered":"2024-04-25T10:06:38.219018+0000","last_clean":"2024-04-25T10:06:38.219018+0000","last_became_active":"2024-04-25T10:06:37.211245+0000","last_became_peered":"2024-04-25T10:06:37.211245+0000","last_unstale":"2024-04-25T10:06:38.219018+0000","last_undegraded":"2024-04-25T10:06:38.219018+0000","last_fullsized":"2024-04-25T10:06:38.219018+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:05.786554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327563+0000","last_change":"2024-04-25T10:06:36.194979+0000","last_active":"2024-04-25T10:06:38.327563+0000","last_peered":"2024-04-25T10:06:38.327563+0000","last_clean":"2024-04-25T10:06:38.327563+0000","last_became_active":"2024-04-25T10:06:36.194166+0000","last_became_peered":"2024-04-25T10:06:36.194166+0000","last_unstale":"2024-04-25T10:06:38.327563+0000","last_undegraded":"2024-04-25T10:06:38.327563+0000","last_fullsized":"2024-04-25T10:06:38.327563+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:55.366280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328752+0000","last_change":"2024-04-25T10:06:37.215396+0000","last_active":"2024-04-25T10:06:38.328752+0000","last_peered":"2024-04-25T10:06:38.328752+0000","last_clean":"2024-04-25T10:06:38.328752+0000","last_became_active":"2024-04-25T10:06:37.214988+0000","last_became_peered":"2024-04-25T10:06:37.214988+0000","last_unstale":"2024-04-25T10:06:38.328752+0000","last_undegraded":"2024-04-25T10:06:38.328752+0000","last_fullsized":"2024-04-25T10:06:38.328752+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:56.006198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326830+0000","last_change":"2024-04-25T10:06:36.181677+0000","last_active":"2024-04-25T10:06:38.326830+0000","last_peered":"2024-04-25T10:06:38.326830+0000","last_clean":"2024-04-25T10:06:38.326830+0000","last_became_active":"2024-04-25T10:06:36.180832+0000","last_became_peered":"2024-04-25T10:06:36.180832+0000","last_unstale":"2024-04-25T10:06:38.326830+0000","last_undegraded":"2024-04-25T10:06:38.326830+0000","last_fullsized":"2024-04-25T10:06:38.326830+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.733077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326327+0000","last_change":"2024-04-25T10:06:37.258667+0000","last_active":"2024-04-25T10:06:38.326327+0000","last_peered":"2024-04-25T10:06:38.326327+0000","last_clean":"2024-04-25T10:06:38.326327+0000","last_became_active":"2024-04-25T10:06:37.256627+0000","last_became_peered":"2024-04-25T10:06:37.256627+0000","last_unstale":"2024-04-25T10:06:38.326327+0000","last_undegraded":"2024-04-25T10:06:38.326327+0000","last_fullsized":"2024-04-25T10:06:38.326327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:11.227591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219311+0000","last_change":"2024-04-25T10:06:36.191971+0000","last_active":"2024-04-25T10:06:38.219311+0000","last_peered":"2024-04-25T10:06:38.219311+0000","last_clean":"2024-04-25T10:06:38.219311+0000","last_became_active":"2024-04-25T10:06:36.191546+0000","last_became_peered":"2024-04-25T10:06:36.191546+0000","last_unstale":"2024-04-25T10:06:38.219311+0000","last_undegraded":"2024-04-25T10:06:38.219311+0000","last_fullsized":"2024-04-25T10:06:38.219311+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:47.723565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219639+0000","last_change":"2024-04-25T10:06:37.223021+0000","last_active":"2024-04-25T10:06:38.219639+0000","last_peered":"2024-04-25T10:06:38.219639+0000","last_clean":"2024-04-25T10:06:38.219639+0000","last_became_active":"2024-04-25T10:06:37.221312+0000","last_became_peered":"2024-04-25T10:06:37.221312+0000","last_unstale":"2024-04-25T10:06:38.219639+0000","last_undegraded":"2024-04-25T10:06:38.219639+0000","last_fullsized":"2024-04-25T10:06:38.219639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:36.845800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328700+0000","last_change":"2024-04-25T10:06:36.183655+0000","last_active":"2024-04-25T10:06:38.328700+0000","last_peered":"2024-04-25T10:06:38.328700+0000","last_clean":"2024-04-25T10:06:38.328700+0000","last_became_active":"2024-04-25T10:06:36.183250+0000","last_became_peered":"2024-04-25T10:06:36.183250+0000","last_unstale":"2024-04-25T10:06:38.328700+0000","last_undegraded":"2024-04-25T10:06:38.328700+0000","last_fullsized":"2024-04-25T10:06:38.328700+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:53.225771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.218979+0000","last_change":"2024-04-25T10:06:37.218508+0000","last_active":"2024-04-25T10:06:38.218979+0000","last_peered":"2024-04-25T10:06:38.218979+0000","last_clean":"2024-04-25T10:06:38.218979+0000","last_became_active":"2024-04-25T10:06:37.216279+0000","last_became_peered":"2024-04-25T10:06:37.216279+0000","last_unstale":"2024-04-25T10:06:38.218979+0000","last_undegraded":"2024-04-25T10:06:38.218979+0000","last_fullsized":"2024-04-25T10:06:38.218979+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:05.743949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327149+0000","last_change":"2024-04-25T10:06:36.187358+0000","last_active":"2024-04-25T10:06:38.327149+0000","last_peered":"2024-04-25T10:06:38.327149+0000","last_clean":"2024-04-25T10:06:38.327149+0000","last_became_active":"2024-04-25T10:06:36.186686+0000","last_became_peered":"2024-04-25T10:06:36.186686+0000","last_unstale":"2024-04-25T10:06:38.327149+0000","last_undegraded":"2024-04-25T10:06:38.327149+0000","last_fullsized":"2024-04-25T10:06:38.327149+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.733077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221375+0000","last_change":"2024-04-25T10:06:37.235221+0000","last_active":"2024-04-25T10:06:38.221375+0000","last_peered":"2024-04-25T10:06:38.221375+0000","last_clean":"2024-04-25T10:06:38.221375+0000","last_became_active":"2024-04-25T10:06:37.229996+0000","last_became_peered":"2024-04-25T10:06:37.229996+0000","last_unstale":"2024-04-25T10:06:38.221375+0000","last_undegraded":"2024-04-25T10:06:38.221375+0000","last_fullsized":"2024-04-25T10:06:38.221375+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:14.900546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326580+0000","last_change":"2024-04-25T10:06:36.157589+0000","last_active":"2024-04-25T10:06:38.326580+0000","last_peered":"2024-04-25T10:06:38.326580+0000","last_clean":"2024-04-25T10:06:38.326580+0000","last_became_active":"2024-04-25T10:06:36.157314+0000","last_became_peered":"2024-04-25T10:06:36.157314+0000","last_unstale":"2024-04-25T10:06:38.326580+0000","last_undegraded":"2024-04-25T10:06:38.326580+0000","last_fullsized":"2024-04-25T10:06:38.326580+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:04.783365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327332+0000","last_change":"2024-04-25T10:06:37.248310+0000","last_active":"2024-04-25T10:06:38.327332+0000","last_peered":"2024-04-25T10:06:38.327332+0000","last_clean":"2024-04-25T10:06:38.327332+0000","last_became_active":"2024-04-25T10:06:37.247412+0000","last_became_peered":"2024-04-25T10:06:37.247412+0000","last_unstale":"2024-04-25T10:06:38.327332+0000","last_undegraded":"2024-04-25T10:06:38.327332+0000","last_fullsized":"2024-04-25T10:06:38.327332+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:23.121736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326595+0000","last_change":"2024-04-25T10:06:36.169365+0000","last_active":"2024-04-25T10:06:38.326595+0000","last_peered":"2024-04-25T10:06:38.326595+0000","last_clean":"2024-04-25T10:06:38.326595+0000","last_became_active":"2024-04-25T10:06:36.168972+0000","last_became_peered":"2024-04-25T10:06:36.168972+0000","last_unstale":"2024-04-25T10:06:38.326595+0000","last_undegraded":"2024-04-25T10:06:38.326595+0000","last_fullsized":"2024-04-25T10:06:38.326595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:06.628360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327765+0000","last_change":"2024-04-25T10:06:37.259092+0000","last_active":"2024-04-25T10:06:38.327765+0000","last_peered":"2024-04-25T10:06:38.327765+0000","last_clean":"2024-04-25T10:06:38.327765+0000","last_became_active":"2024-04-25T10:06:37.258761+0000","last_became_peered":"2024-04-25T10:06:37.258761+0000","last_unstale":"2024-04-25T10:06:38.327765+0000","last_undegraded":"2024-04-25T10:06:38.327765+0000","last_fullsized":"2024-04-25T10:06:38.327765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:38.209278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326919+0000","last_change":"2024-04-25T10:06:36.170925+0000","last_active":"2024-04-25T10:06:38.326919+0000","last_peered":"2024-04-25T10:06:38.326919+0000","last_clean":"2024-04-25T10:06:38.326919+0000","last_became_active":"2024-04-25T10:06:36.170521+0000","last_became_peered":"2024-04-25T10:06:36.170521+0000","last_unstale":"2024-04-25T10:06:38.326919+0000","last_undegraded":"2024-04-25T10:06:38.326919+0000","last_fullsized":"2024-04-25T10:06:38.326919+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:20.412161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220036+0000","last_change":"2024-04-25T10:06:37.236349+0000","last_active":"2024-04-25T10:06:38.220036+0000","last_peered":"2024-04-25T10:06:38.220036+0000","last_clean":"2024-04-25T10:06:38.220036+0000","last_became_active":"2024-04-25T10:06:37.235659+0000","last_became_peered":"2024-04-25T10:06:37.235659+0000","last_unstale":"2024-04-25T10:06:38.220036+0000","last_undegraded":"2024-04-25T10:06:38.220036+0000","last_fullsized":"2024-04-25T10:06:38.220036+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:21.415350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326076+0000","last_change":"2024-04-25T10:06:36.161401+0000","last_active":"2024-04-25T10:06:38.326076+0000","last_peered":"2024-04-25T10:06:38.326076+0000","last_clean":"2024-04-25T10:06:38.326076+0000","last_became_active":"2024-04-25T10:06:36.161102+0000","last_became_peered":"2024-04-25T10:06:36.161102+0000","last_unstale":"2024-04-25T10:06:38.326076+0000","last_undegraded":"2024-04-25T10:06:38.326076+0000","last_fullsized":"2024-04-25T10:06:38.326076+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:19.507488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325952+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:06:38.325952+0000","last_peered":"2024-04-25T10:06:38.325952+0000","last_clean":"2024-04-25T10:06:38.325952+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:06:38.325952+0000","last_undegraded":"2024-04-25T10:06:38.325952+0000","last_fullsized":"2024-04-25T10:06:38.325952+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50604,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321236,"statfs":{"total":96636764160,"available":96584945664,"internally_reserved":0,"allocated":1409024,"data_stored":953565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40284,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331556,"statfs":{"total":96636764160,"available":96595513344,"internally_reserved":0,"allocated":802816,"data_stored":362455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40288,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331552,"statfs":{"total":96636764160,"available":96595509248,"internally_reserved":0,"allocated":802816,"data_stored":362443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45868,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325972,"statfs":{"total":96636764160,"available":96589795328,"internally_reserved":0,"allocated":811008,"data_stored":364033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":362421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44444,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327396,"statfs":{"total":96636764160,"available":96591253504,"internally_reserved":0,"allocated":798720,"data_stored":362421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40308,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331532,"statfs":{"total":96636764160,"available":96595488768,"internally_reserved":0,"allocated":827392,"data_stored":364797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50592,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321248,"statfs":{"total":96636764160,"available":96584957952,"internally_reserved":0,"allocated":1392640,"data_stored":952789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:40.186 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:40.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.338+0000 7fa40bbe0640 1 Processor -- start 2024-04-25T10:06:40.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.338+0000 7fa40bbe0640 1 -- start start 2024-04-25T10:06:40.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40bbe0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa40406eb70 0x7fa4040c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40bbe0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa40415a9d0 0x7fa4040cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40bbe0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4040cc5a0 con 0x7fa40415a5f0 2024-04-25T10:06:40.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40bbe0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4040cc880 con 0x7fa40406eb70 2024-04-25T10:06:40.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40bbe0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4040ccb60 con 0x7fa40415a9d0 2024-04-25T10:06:40.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40917b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa40406eb70 0x7fa4040c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40917b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa40406eb70 0x7fa4040c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43808/0 (socket says 172.21.15.133:43808) 2024-04-25T10:06:40.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40917b640 1 -- 172.21.15.133:0/1444044409 learned_addr learned my addr 172.21.15.133:0/1444044409 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:40.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40917b640 1 -- 172.21.15.133:0/1444044409 >> v2:172.21.15.162:3300/0 conn(0x7fa40415a9d0 msgr2=0x7fa4040cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40a17d640 1 --2- 172.21.15.133:0/1444044409 >> v2:172.21.15.162:3300/0 conn(0x7fa40415a9d0 0x7fa4040cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40917b640 1 --2- 172.21.15.133:0/1444044409 >> v2:172.21.15.162:3300/0 conn(0x7fa40415a9d0 0x7fa4040cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40917b640 1 -- 172.21.15.133:0/1444044409 >> v1:172.21.15.162:6789/0 conn(0x7fa40415a5f0 legacy=0x7fa40406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40917b640 1 -- 172.21.15.133:0/1444044409 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040cce40 con 0x7fa40406eb70 2024-04-25T10:06:40.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40a17d640 1 --2- 172.21.15.133:0/1444044409 >> v2:172.21.15.162:3300/0 conn(0x7fa40415a9d0 0x7fa4040cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:40.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40917b640 1 --2- 172.21.15.133:0/1444044409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa40406eb70 0x7fa4040c5260 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa4040cd490 tx=0x7fa3f401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a05f6ff3344852 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40897a640 1 -- 172.21.15.133:0/1444044409 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40b0070 con 0x7fa40406eb70 2024-04-25T10:06:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40897a640 1 -- 172.21.15.133:0/1444044409 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f40ab600 con 0x7fa40406eb70 2024-04-25T10:06:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.342+0000 7fa40897a640 1 -- 172.21.15.133:0/1444044409 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40ab9d0 con 0x7fa40406eb70 2024-04-25T10:06:40.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 -- 172.21.15.133:0/1444044409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa40406eb70 msgr2=0x7fa4040c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 --2- 172.21.15.133:0/1444044409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa40406eb70 0x7fa4040c5260 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa4040cd490 tx=0x7fa3f401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 -- 172.21.15.133:0/1444044409 shutdown_connections 2024-04-25T10:06:40.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 --2- 172.21.15.133:0/1444044409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa40406eb70 0x7fa4040c5260 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 --2- 172.21.15.133:0/1444044409 >> v2:172.21.15.162:3300/0 conn(0x7fa40415a9d0 0x7fa4040cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 -- 172.21.15.133:0/1444044409 >> 172.21.15.133:0/1444044409 conn(0x7fa404067000 msgr2=0x7fa40406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 -- 172.21.15.133:0/1444044409 shutdown_connections 2024-04-25T10:06:40.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 -- 172.21.15.133:0/1444044409 wait complete. 2024-04-25T10:06:40.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 Processor -- start 2024-04-25T10:06:40.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 -- start start 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa40406eb70 0x7fa4040b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa40415a5f0 0x7fa4040b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa40415a9d0 0x7fa4041f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040ce490 con 0x7fa40406eb70 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040ce310 con 0x7fa40415a9d0 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040ce610 con 0x7fa40415a5f0 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40997c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa40406eb70 0x7fa4040b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40997c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa40406eb70 0x7fa4040b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43810/0 (socket says 172.21.15.133:43810) 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40997c640 1 -- 172.21.15.133:0/2894607876 learned_addr learned my addr 172.21.15.133:0/2894607876 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40a17d640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa40415a9d0 0x7fa4041f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40917b640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa40415a5f0 0x7fa4040b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40997c640 1 -- 172.21.15.133:0/2894607876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa40415a5f0 msgr2=0x7fa4040b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40997c640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa40415a5f0 0x7fa4040b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40997c640 1 -- 172.21.15.133:0/2894607876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa40415a9d0 msgr2=0x7fa4041f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40997c640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa40415a9d0 0x7fa4041f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40997c640 1 -- 172.21.15.133:0/2894607876 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40415c640 con 0x7fa40406eb70 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40a17d640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa40415a9d0 0x7fa4041f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40997c640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa40406eb70 0x7fa4040b30a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa40000c450 tx=0x7fa40000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa3fa7fc640 1 -- 172.21.15.133:0/2894607876 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400012830 con 0x7fa40406eb70 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa3fa7fc640 1 -- 172.21.15.133:0/2894607876 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa400017070 con 0x7fa40406eb70 2024-04-25T10:06:40.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa40bbe0640 1 -- 172.21.15.133:0/2894607876 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40415c920 con 0x7fa40406eb70 2024-04-25T10:06:40.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.346+0000 7fa3fa7fc640 1 -- 172.21.15.133:0/2894607876 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400012c30 con 0x7fa40406eb70 2024-04-25T10:06:40.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.350+0000 7fa40bbe0640 1 -- 172.21.15.133:0/2894607876 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40415cee0 con 0x7fa40406eb70 2024-04-25T10:06:40.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.350+0000 7fa40bbe0640 1 -- 172.21.15.133:0/2894607876 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa404068ef0 con 0x7fa40406eb70 2024-04-25T10:06:40.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.350+0000 7fa3fa7fc640 1 -- 172.21.15.133:0/2894607876 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa40001c460 con 0x7fa40406eb70 2024-04-25T10:06:40.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.350+0000 7fa3fa7fc640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e8066f60 0x7fa3e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.350+0000 7fa3fa7fc640 1 -- 172.21.15.133:0/2894607876 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa400084f70 con 0x7fa40406eb70 2024-04-25T10:06:40.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.350+0000 7fa40917b640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e8066f60 0x7fa3e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.350+0000 7fa40917b640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e8066f60 0x7fa3e8069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40ad000 tx=0x7fa3f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.354+0000 7fa3fa7fc640 1 -- 172.21.15.133:0/2894607876 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa400055e80 con 0x7fa40406eb70 2024-04-25T10:06:40.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.510+0000 7fa40bbe0640 1 -- 172.21.15.133:0/2894607876 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4041649a0 con 0x7fa3e8066f60 2024-04-25T10:06:40.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.518+0000 7fa3fa7fc640 1 -- 172.21.15.133:0/2894607876 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252027 (secure 0 0 0) 0x7fa4041649a0 con 0x7fa3e8066f60 2024-04-25T10:06:40.524 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:40.527 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:06:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 -- 172.21.15.133:0/2894607876 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e8066f60 msgr2=0x7fa3e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e8066f60 0x7fa3e8069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40ad000 tx=0x7fa3f4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 -- 172.21.15.133:0/2894607876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa40406eb70 msgr2=0x7fa4040b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa40406eb70 0x7fa4040b30a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa40000c450 tx=0x7fa40000c910 comp rx=0 tx=0).stop 2024-04-25T10:06:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 -- 172.21.15.133:0/2894607876 shutdown_connections 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa40415a9d0 0x7fa4041f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa40415a5f0 0x7fa4040b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e8066f60 0x7fa3e8069420 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 --2- 172.21.15.133:0/2894607876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa40406eb70 0x7fa4040b30a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 -- 172.21.15.133:0/2894607876 >> 172.21.15.133:0/2894607876 conn(0x7fa404067000 msgr2=0x7fa4040ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 -- 172.21.15.133:0/2894607876 shutdown_connections 2024-04-25T10:06:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.522+0000 7fa40bbe0640 1 -- 172.21.15.133:0/2894607876 wait complete. 2024-04-25T10:06:40.543 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-04-25T10:06:39.794094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":340192,"kb_used_data":7464,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754634528,"statfs":{"total":773094113280,"available":772745756672,"internally_reserved":0,"allocated":7643136,"data_stored":4084924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":340577841},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.652834"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328674+0000","last_change":"2024-04-25T10:06:37.259345+0000","last_active":"2024-04-25T10:06:38.328674+0000","last_peered":"2024-04-25T10:06:38.328674+0000","last_clean":"2024-04-25T10:06:38.328674+0000","last_became_active":"2024-04-25T10:06:37.258816+0000","last_became_peered":"2024-04-25T10:06:37.258816+0000","last_unstale":"2024-04-25T10:06:38.328674+0000","last_undegraded":"2024-04-25T10:06:38.328674+0000","last_fullsized":"2024-04-25T10:06:38.328674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:06.890930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327702+0000","last_change":"2024-04-25T10:06:37.229642+0000","last_active":"2024-04-25T10:06:38.327702+0000","last_peered":"2024-04-25T10:06:38.327702+0000","last_clean":"2024-04-25T10:06:38.327702+0000","last_became_active":"2024-04-25T10:06:37.229336+0000","last_became_peered":"2024-04-25T10:06:37.229336+0000","last_unstale":"2024-04-25T10:06:38.327702+0000","last_undegraded":"2024-04-25T10:06:38.327702+0000","last_fullsized":"2024-04-25T10:06:38.327702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:31.642221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326705+0000","last_change":"2024-04-25T10:06:37.244256+0000","last_active":"2024-04-25T10:06:38.326705+0000","last_peered":"2024-04-25T10:06:38.326705+0000","last_clean":"2024-04-25T10:06:38.326705+0000","last_became_active":"2024-04-25T10:06:37.243703+0000","last_became_peered":"2024-04-25T10:06:37.243703+0000","last_unstale":"2024-04-25T10:06:38.326705+0000","last_undegraded":"2024-04-25T10:06:38.326705+0000","last_fullsized":"2024-04-25T10:06:38.326705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:07.217705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220937+0000","last_change":"2024-04-25T10:06:37.215123+0000","last_active":"2024-04-25T10:06:38.220937+0000","last_peered":"2024-04-25T10:06:38.220937+0000","last_clean":"2024-04-25T10:06:38.220937+0000","last_became_active":"2024-04-25T10:06:37.212773+0000","last_became_peered":"2024-04-25T10:06:37.212773+0000","last_unstale":"2024-04-25T10:06:38.220937+0000","last_undegraded":"2024-04-25T10:06:38.220937+0000","last_fullsized":"2024-04-25T10:06:38.220937+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:22.292593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221408+0000","last_change":"2024-04-25T10:06:37.212173+0000","last_active":"2024-04-25T10:06:38.221408+0000","last_peered":"2024-04-25T10:06:38.221408+0000","last_clean":"2024-04-25T10:06:38.221408+0000","last_became_active":"2024-04-25T10:06:37.211383+0000","last_became_peered":"2024-04-25T10:06:37.211383+0000","last_unstale":"2024-04-25T10:06:38.221408+0000","last_undegraded":"2024-04-25T10:06:38.221408+0000","last_fullsized":"2024-04-25T10:06:38.221408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:40.032710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219108+0000","last_change":"2024-04-25T10:06:37.230922+0000","last_active":"2024-04-25T10:06:38.219108+0000","last_peered":"2024-04-25T10:06:38.219108+0000","last_clean":"2024-04-25T10:06:38.219108+0000","last_became_active":"2024-04-25T10:06:37.230596+0000","last_became_peered":"2024-04-25T10:06:37.230596+0000","last_unstale":"2024-04-25T10:06:38.219108+0000","last_undegraded":"2024-04-25T10:06:38.219108+0000","last_fullsized":"2024-04-25T10:06:38.219108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:40.225911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.329170+0000","last_change":"2024-04-25T10:06:37.235039+0000","last_active":"2024-04-25T10:06:38.329170+0000","last_peered":"2024-04-25T10:06:38.329170+0000","last_clean":"2024-04-25T10:06:38.329170+0000","last_became_active":"2024-04-25T10:06:37.234713+0000","last_became_peered":"2024-04-25T10:06:37.234713+0000","last_unstale":"2024-04-25T10:06:38.329170+0000","last_undegraded":"2024-04-25T10:06:38.329170+0000","last_fullsized":"2024-04-25T10:06:38.329170+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:07.241324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325297+0000","last_change":"2024-04-25T10:06:37.246081+0000","last_active":"2024-04-25T10:06:38.325297+0000","last_peered":"2024-04-25T10:06:38.325297+0000","last_clean":"2024-04-25T10:06:38.325297+0000","last_became_active":"2024-04-25T10:06:37.245546+0000","last_became_peered":"2024-04-25T10:06:37.245546+0000","last_unstale":"2024-04-25T10:06:38.325297+0000","last_undegraded":"2024-04-25T10:06:38.325297+0000","last_fullsized":"2024-04-25T10:06:38.325297+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:05.786554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327090+0000","last_change":"2024-04-25T10:06:37.253860+0000","last_active":"2024-04-25T10:06:38.327090+0000","last_peered":"2024-04-25T10:06:38.327090+0000","last_clean":"2024-04-25T10:06:38.327090+0000","last_became_active":"2024-04-25T10:06:37.253261+0000","last_became_peered":"2024-04-25T10:06:37.253261+0000","last_unstale":"2024-04-25T10:06:38.327090+0000","last_undegraded":"2024-04-25T10:06:38.327090+0000","last_fullsized":"2024-04-25T10:06:38.327090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:47.119623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327341+0000","last_change":"2024-04-25T10:06:37.232517+0000","last_active":"2024-04-25T10:06:38.327341+0000","last_peered":"2024-04-25T10:06:38.327341+0000","last_clean":"2024-04-25T10:06:38.327341+0000","last_became_active":"2024-04-25T10:06:37.231472+0000","last_became_peered":"2024-04-25T10:06:37.231472+0000","last_unstale":"2024-04-25T10:06:38.327341+0000","last_undegraded":"2024-04-25T10:06:38.327341+0000","last_fullsized":"2024-04-25T10:06:38.327341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:26.595418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221279+0000","last_change":"2024-04-25T10:06:37.225407+0000","last_active":"2024-04-25T10:06:38.221279+0000","last_peered":"2024-04-25T10:06:38.221279+0000","last_clean":"2024-04-25T10:06:38.221279+0000","last_became_active":"2024-04-25T10:06:37.224684+0000","last_became_peered":"2024-04-25T10:06:37.224684+0000","last_unstale":"2024-04-25T10:06:38.221279+0000","last_undegraded":"2024-04-25T10:06:38.221279+0000","last_fullsized":"2024-04-25T10:06:38.221279+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:41.228850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325457+0000","last_change":"2024-04-25T10:06:37.246701+0000","last_active":"2024-04-25T10:06:38.325457+0000","last_peered":"2024-04-25T10:06:38.325457+0000","last_clean":"2024-04-25T10:06:38.325457+0000","last_became_active":"2024-04-25T10:06:37.240383+0000","last_became_peered":"2024-04-25T10:06:37.240383+0000","last_unstale":"2024-04-25T10:06:38.325457+0000","last_undegraded":"2024-04-25T10:06:38.325457+0000","last_fullsized":"2024-04-25T10:06:38.325457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:51.322072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325415+0000","last_change":"2024-04-25T10:06:37.255444+0000","last_active":"2024-04-25T10:06:38.325415+0000","last_peered":"2024-04-25T10:06:38.325415+0000","last_clean":"2024-04-25T10:06:38.325415+0000","last_became_active":"2024-04-25T10:06:37.254910+0000","last_became_peered":"2024-04-25T10:06:37.254910+0000","last_unstale":"2024-04-25T10:06:38.325415+0000","last_undegraded":"2024-04-25T10:06:38.325415+0000","last_fullsized":"2024-04-25T10:06:38.325415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:40.225911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327767+0000","last_change":"2024-04-25T10:06:37.228259+0000","last_active":"2024-04-25T10:06:38.327767+0000","last_peered":"2024-04-25T10:06:38.327767+0000","last_clean":"2024-04-25T10:06:38.327767+0000","last_became_active":"2024-04-25T10:06:37.227701+0000","last_became_peered":"2024-04-25T10:06:37.227701+0000","last_unstale":"2024-04-25T10:06:38.327767+0000","last_undegraded":"2024-04-25T10:06:38.327767+0000","last_fullsized":"2024-04-25T10:06:38.327767+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:03.343765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326391+0000","last_change":"2024-04-25T10:06:37.236842+0000","last_active":"2024-04-25T10:06:38.326391+0000","last_peered":"2024-04-25T10:06:38.326391+0000","last_clean":"2024-04-25T10:06:38.326391+0000","last_became_active":"2024-04-25T10:06:37.235846+0000","last_became_peered":"2024-04-25T10:06:37.235846+0000","last_unstale":"2024-04-25T10:06:38.326391+0000","last_undegraded":"2024-04-25T10:06:38.326391+0000","last_fullsized":"2024-04-25T10:06:38.326391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:11.227591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328148+0000","last_change":"2024-04-25T10:06:37.215757+0000","last_active":"2024-04-25T10:06:38.328148+0000","last_peered":"2024-04-25T10:06:38.328148+0000","last_clean":"2024-04-25T10:06:38.328148+0000","last_became_active":"2024-04-25T10:06:37.215407+0000","last_became_peered":"2024-04-25T10:06:37.215407+0000","last_unstale":"2024-04-25T10:06:38.328148+0000","last_undegraded":"2024-04-25T10:06:38.328148+0000","last_fullsized":"2024-04-25T10:06:38.328148+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:25.898592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.329104+0000","last_change":"2024-04-25T10:06:37.250652+0000","last_active":"2024-04-25T10:06:38.329104+0000","last_peered":"2024-04-25T10:06:38.329104+0000","last_clean":"2024-04-25T10:06:38.329104+0000","last_became_active":"2024-04-25T10:06:37.250350+0000","last_became_peered":"2024-04-25T10:06:37.250350+0000","last_unstale":"2024-04-25T10:06:38.329104+0000","last_undegraded":"2024-04-25T10:06:38.329104+0000","last_fullsized":"2024-04-25T10:06:38.329104+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:56.369469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220966+0000","last_change":"2024-04-25T10:06:37.230330+0000","last_active":"2024-04-25T10:06:38.220966+0000","last_peered":"2024-04-25T10:06:38.220966+0000","last_clean":"2024-04-25T10:06:38.220966+0000","last_became_active":"2024-04-25T10:06:37.229121+0000","last_became_peered":"2024-04-25T10:06:37.229121+0000","last_unstale":"2024-04-25T10:06:38.220966+0000","last_undegraded":"2024-04-25T10:06:38.220966+0000","last_fullsized":"2024-04-25T10:06:38.220966+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:36.845800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220655+0000","last_change":"2024-04-25T10:06:37.206460+0000","last_active":"2024-04-25T10:06:38.220655+0000","last_peered":"2024-04-25T10:06:38.220655+0000","last_clean":"2024-04-25T10:06:38.220655+0000","last_became_active":"2024-04-25T10:06:37.205780+0000","last_became_peered":"2024-04-25T10:06:37.205780+0000","last_unstale":"2024-04-25T10:06:38.220655+0000","last_undegraded":"2024-04-25T10:06:38.220655+0000","last_fullsized":"2024-04-25T10:06:38.220655+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:54.228961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326339+0000","last_change":"2024-04-25T10:06:37.227423+0000","last_active":"2024-04-25T10:06:38.326339+0000","last_peered":"2024-04-25T10:06:38.326339+0000","last_clean":"2024-04-25T10:06:38.326339+0000","last_became_active":"2024-04-25T10:06:37.227089+0000","last_became_peered":"2024-04-25T10:06:37.227089+0000","last_unstale":"2024-04-25T10:06:38.326339+0000","last_undegraded":"2024-04-25T10:06:38.326339+0000","last_fullsized":"2024-04-25T10:06:38.326339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:26.595418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325850+0000","last_change":"2024-04-25T10:06:37.250442+0000","last_active":"2024-04-25T10:06:38.325850+0000","last_peered":"2024-04-25T10:06:38.325850+0000","last_clean":"2024-04-25T10:06:38.325850+0000","last_became_active":"2024-04-25T10:06:37.247920+0000","last_became_peered":"2024-04-25T10:06:37.247920+0000","last_unstale":"2024-04-25T10:06:38.325850+0000","last_undegraded":"2024-04-25T10:06:38.325850+0000","last_fullsized":"2024-04-25T10:06:38.325850+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:50.285859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326296+0000","last_change":"2024-04-25T10:06:37.235003+0000","last_active":"2024-04-25T10:06:38.326296+0000","last_peered":"2024-04-25T10:06:38.326296+0000","last_clean":"2024-04-25T10:06:38.326296+0000","last_became_active":"2024-04-25T10:06:37.234585+0000","last_became_peered":"2024-04-25T10:06:37.234585+0000","last_unstale":"2024-04-25T10:06:38.326296+0000","last_undegraded":"2024-04-25T10:06:38.326296+0000","last_fullsized":"2024-04-25T10:06:38.326296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:25.576789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326090+0000","last_change":"2024-04-25T10:06:37.234991+0000","last_active":"2024-04-25T10:06:38.326090+0000","last_peered":"2024-04-25T10:06:38.326090+0000","last_clean":"2024-04-25T10:06:38.326090+0000","last_became_active":"2024-04-25T10:06:37.234674+0000","last_became_peered":"2024-04-25T10:06:37.234674+0000","last_unstale":"2024-04-25T10:06:38.326090+0000","last_undegraded":"2024-04-25T10:06:38.326090+0000","last_fullsized":"2024-04-25T10:06:38.326090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:07.217705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325588+0000","last_change":"2024-04-25T10:06:37.237161+0000","last_active":"2024-04-25T10:06:38.325588+0000","last_peered":"2024-04-25T10:06:38.325588+0000","last_clean":"2024-04-25T10:06:38.325588+0000","last_became_active":"2024-04-25T10:06:37.236611+0000","last_became_peered":"2024-04-25T10:06:37.236611+0000","last_unstale":"2024-04-25T10:06:38.325588+0000","last_undegraded":"2024-04-25T10:06:38.325588+0000","last_fullsized":"2024-04-25T10:06:38.325588+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:50.285859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326757+0000","last_change":"2024-04-25T10:06:37.245158+0000","last_active":"2024-04-25T10:06:38.326757+0000","last_peered":"2024-04-25T10:06:38.326757+0000","last_clean":"2024-04-25T10:06:38.326757+0000","last_became_active":"2024-04-25T10:06:37.240168+0000","last_became_peered":"2024-04-25T10:06:37.240168+0000","last_unstale":"2024-04-25T10:06:38.326757+0000","last_undegraded":"2024-04-25T10:06:38.326757+0000","last_fullsized":"2024-04-25T10:06:38.326757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:07.217705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325914+0000","last_change":"2024-04-25T10:06:37.258424+0000","last_active":"2024-04-25T10:06:38.325914+0000","last_peered":"2024-04-25T10:06:38.325914+0000","last_clean":"2024-04-25T10:06:38.325914+0000","last_became_active":"2024-04-25T10:06:37.257227+0000","last_became_peered":"2024-04-25T10:06:37.257227+0000","last_unstale":"2024-04-25T10:06:38.325914+0000","last_undegraded":"2024-04-25T10:06:38.325914+0000","last_fullsized":"2024-04-25T10:06:38.325914+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:15.989745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220689+0000","last_change":"2024-04-25T10:06:37.213722+0000","last_active":"2024-04-25T10:06:38.220689+0000","last_peered":"2024-04-25T10:06:38.220689+0000","last_clean":"2024-04-25T10:06:38.220689+0000","last_became_active":"2024-04-25T10:06:37.213011+0000","last_became_peered":"2024-04-25T10:06:37.213011+0000","last_unstale":"2024-04-25T10:06:38.220689+0000","last_undegraded":"2024-04-25T10:06:38.220689+0000","last_fullsized":"2024-04-25T10:06:38.220689+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:52.879097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328249+0000","last_change":"2024-04-25T10:06:37.228946+0000","last_active":"2024-04-25T10:06:38.328249+0000","last_peered":"2024-04-25T10:06:38.328249+0000","last_clean":"2024-04-25T10:06:38.328249+0000","last_became_active":"2024-04-25T10:06:37.228472+0000","last_became_peered":"2024-04-25T10:06:37.228472+0000","last_unstale":"2024-04-25T10:06:38.328249+0000","last_undegraded":"2024-04-25T10:06:38.328249+0000","last_fullsized":"2024-04-25T10:06:38.328249+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:45.902250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325934+0000","last_change":"2024-04-25T10:06:37.249491+0000","last_active":"2024-04-25T10:06:38.325934+0000","last_peered":"2024-04-25T10:06:38.325934+0000","last_clean":"2024-04-25T10:06:38.325934+0000","last_became_active":"2024-04-25T10:06:37.248997+0000","last_became_peered":"2024-04-25T10:06:37.248997+0000","last_unstale":"2024-04-25T10:06:38.325934+0000","last_undegraded":"2024-04-25T10:06:38.325934+0000","last_fullsized":"2024-04-25T10:06:38.325934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:15.989745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325386+0000","last_change":"2024-04-25T10:06:37.247431+0000","last_active":"2024-04-25T10:06:38.325386+0000","last_peered":"2024-04-25T10:06:38.325386+0000","last_clean":"2024-04-25T10:06:38.325386+0000","last_became_active":"2024-04-25T10:06:37.246798+0000","last_became_peered":"2024-04-25T10:06:37.246798+0000","last_unstale":"2024-04-25T10:06:38.325386+0000","last_undegraded":"2024-04-25T10:06:38.325386+0000","last_fullsized":"2024-04-25T10:06:38.325386+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.392643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221058+0000","last_change":"2024-04-25T10:06:37.207508+0000","last_active":"2024-04-25T10:06:38.221058+0000","last_peered":"2024-04-25T10:06:38.221058+0000","last_clean":"2024-04-25T10:06:38.221058+0000","last_became_active":"2024-04-25T10:06:37.206721+0000","last_became_peered":"2024-04-25T10:06:37.206721+0000","last_unstale":"2024-04-25T10:06:38.221058+0000","last_undegraded":"2024-04-25T10:06:38.221058+0000","last_fullsized":"2024-04-25T10:06:38.221058+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:06.763393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219104+0000","last_change":"2024-04-25T10:06:37.220236+0000","last_active":"2024-04-25T10:06:38.219104+0000","last_peered":"2024-04-25T10:06:38.219104+0000","last_clean":"2024-04-25T10:06:38.219104+0000","last_became_active":"2024-04-25T10:06:37.219720+0000","last_became_peered":"2024-04-25T10:06:37.219720+0000","last_unstale":"2024-04-25T10:06:38.219104+0000","last_undegraded":"2024-04-25T10:06:38.219104+0000","last_fullsized":"2024-04-25T10:06:38.219104+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:51.322072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219655+0000","last_change":"2024-04-25T10:06:37.239433+0000","last_active":"2024-04-25T10:06:38.219655+0000","last_peered":"2024-04-25T10:06:38.219655+0000","last_clean":"2024-04-25T10:06:38.219655+0000","last_became_active":"2024-04-25T10:06:37.238802+0000","last_became_peered":"2024-04-25T10:06:37.238802+0000","last_unstale":"2024-04-25T10:06:38.219655+0000","last_undegraded":"2024-04-25T10:06:38.219655+0000","last_fullsized":"2024-04-25T10:06:38.219655+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:06.763393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328881+0000","last_change":"2024-04-25T10:06:36.176032+0000","last_active":"2024-04-25T10:06:38.328881+0000","last_peered":"2024-04-25T10:06:38.328881+0000","last_clean":"2024-04-25T10:06:38.328881+0000","last_became_active":"2024-04-25T10:06:36.175719+0000","last_became_peered":"2024-04-25T10:06:36.175719+0000","last_unstale":"2024-04-25T10:06:38.328881+0000","last_undegraded":"2024-04-25T10:06:38.328881+0000","last_fullsized":"2024-04-25T10:06:38.328881+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:20.671802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221112+0000","last_change":"2024-04-25T10:06:37.224964+0000","last_active":"2024-04-25T10:06:38.221112+0000","last_peered":"2024-04-25T10:06:38.221112+0000","last_clean":"2024-04-25T10:06:38.221112+0000","last_became_active":"2024-04-25T10:06:37.223533+0000","last_became_peered":"2024-04-25T10:06:37.223533+0000","last_unstale":"2024-04-25T10:06:38.221112+0000","last_undegraded":"2024-04-25T10:06:38.221112+0000","last_fullsized":"2024-04-25T10:06:38.221112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:08.891150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219604+0000","last_change":"2024-04-25T10:06:36.188564+0000","last_active":"2024-04-25T10:06:38.219604+0000","last_peered":"2024-04-25T10:06:38.219604+0000","last_clean":"2024-04-25T10:06:38.219604+0000","last_became_active":"2024-04-25T10:06:36.188249+0000","last_became_peered":"2024-04-25T10:06:36.188249+0000","last_unstale":"2024-04-25T10:06:38.219604+0000","last_undegraded":"2024-04-25T10:06:38.219604+0000","last_fullsized":"2024-04-25T10:06:38.219604+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:15.337553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219644+0000","last_change":"2024-04-25T10:06:37.218032+0000","last_active":"2024-04-25T10:06:38.219644+0000","last_peered":"2024-04-25T10:06:38.219644+0000","last_clean":"2024-04-25T10:06:38.219644+0000","last_became_active":"2024-04-25T10:06:37.217672+0000","last_became_peered":"2024-04-25T10:06:37.217672+0000","last_unstale":"2024-04-25T10:06:38.219644+0000","last_undegraded":"2024-04-25T10:06:38.219644+0000","last_fullsized":"2024-04-25T10:06:38.219644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:50.285859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219272+0000","last_change":"2024-04-25T10:06:36.191963+0000","last_active":"2024-04-25T10:06:38.219272+0000","last_peered":"2024-04-25T10:06:38.219272+0000","last_clean":"2024-04-25T10:06:38.219272+0000","last_became_active":"2024-04-25T10:06:36.191556+0000","last_became_peered":"2024-04-25T10:06:36.191556+0000","last_unstale":"2024-04-25T10:06:38.219272+0000","last_undegraded":"2024-04-25T10:06:38.219272+0000","last_fullsized":"2024-04-25T10:06:38.219272+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:32.143965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326595+0000","last_change":"2024-04-25T10:06:37.244406+0000","last_active":"2024-04-25T10:06:38.326595+0000","last_peered":"2024-04-25T10:06:38.326595+0000","last_clean":"2024-04-25T10:06:38.326595+0000","last_became_active":"2024-04-25T10:06:37.243384+0000","last_became_peered":"2024-04-25T10:06:37.243384+0000","last_unstale":"2024-04-25T10:06:38.326595+0000","last_undegraded":"2024-04-25T10:06:38.326595+0000","last_fullsized":"2024-04-25T10:06:38.326595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:52.001010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219269+0000","last_change":"2024-04-25T10:06:36.165068+0000","last_active":"2024-04-25T10:06:38.219269+0000","last_peered":"2024-04-25T10:06:38.219269+0000","last_clean":"2024-04-25T10:06:38.219269+0000","last_became_active":"2024-04-25T10:06:36.164413+0000","last_became_peered":"2024-04-25T10:06:36.164413+0000","last_unstale":"2024-04-25T10:06:38.219269+0000","last_undegraded":"2024-04-25T10:06:38.219269+0000","last_fullsized":"2024-04-25T10:06:38.219269+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:21.437190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328520+0000","last_change":"2024-04-25T10:06:37.223090+0000","last_active":"2024-04-25T10:06:38.328520+0000","last_peered":"2024-04-25T10:06:38.328520+0000","last_clean":"2024-04-25T10:06:38.328520+0000","last_became_active":"2024-04-25T10:06:37.222380+0000","last_became_peered":"2024-04-25T10:06:37.222380+0000","last_unstale":"2024-04-25T10:06:38.328520+0000","last_undegraded":"2024-04-25T10:06:38.328520+0000","last_fullsized":"2024-04-25T10:06:38.328520+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:16.101412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326443+0000","last_change":"2024-04-25T10:06:36.160916+0000","last_active":"2024-04-25T10:06:38.326443+0000","last_peered":"2024-04-25T10:06:38.326443+0000","last_clean":"2024-04-25T10:06:38.326443+0000","last_became_active":"2024-04-25T10:06:36.160599+0000","last_became_peered":"2024-04-25T10:06:36.160599+0000","last_unstale":"2024-04-25T10:06:38.326443+0000","last_undegraded":"2024-04-25T10:06:38.326443+0000","last_fullsized":"2024-04-25T10:06:38.326443+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:45.389453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327117+0000","last_change":"2024-04-25T10:06:37.247914+0000","last_active":"2024-04-25T10:06:38.327117+0000","last_peered":"2024-04-25T10:06:38.327117+0000","last_clean":"2024-04-25T10:06:38.327117+0000","last_became_active":"2024-04-25T10:06:37.247539+0000","last_became_peered":"2024-04-25T10:06:37.247539+0000","last_unstale":"2024-04-25T10:06:38.327117+0000","last_undegraded":"2024-04-25T10:06:38.327117+0000","last_fullsized":"2024-04-25T10:06:38.327117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:13.906449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326447+0000","last_change":"2024-04-25T10:06:36.167219+0000","last_active":"2024-04-25T10:06:38.326447+0000","last_peered":"2024-04-25T10:06:38.326447+0000","last_clean":"2024-04-25T10:06:38.326447+0000","last_became_active":"2024-04-25T10:06:36.164924+0000","last_became_peered":"2024-04-25T10:06:36.164924+0000","last_unstale":"2024-04-25T10:06:38.326447+0000","last_undegraded":"2024-04-25T10:06:38.326447+0000","last_fullsized":"2024-04-25T10:06:38.326447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:21.133255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219392+0000","last_change":"2024-04-25T10:06:37.221660+0000","last_active":"2024-04-25T10:06:38.219392+0000","last_peered":"2024-04-25T10:06:38.219392+0000","last_clean":"2024-04-25T10:06:38.219392+0000","last_became_active":"2024-04-25T10:06:37.220347+0000","last_became_peered":"2024-04-25T10:06:37.220347+0000","last_unstale":"2024-04-25T10:06:38.219392+0000","last_undegraded":"2024-04-25T10:06:38.219392+0000","last_fullsized":"2024-04-25T10:06:38.219392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:22.292593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326478+0000","last_change":"2024-04-25T10:06:36.169910+0000","last_active":"2024-04-25T10:06:38.326478+0000","last_peered":"2024-04-25T10:06:38.326478+0000","last_clean":"2024-04-25T10:06:38.326478+0000","last_became_active":"2024-04-25T10:06:36.169316+0000","last_became_peered":"2024-04-25T10:06:36.169316+0000","last_unstale":"2024-04-25T10:06:38.326478+0000","last_undegraded":"2024-04-25T10:06:38.326478+0000","last_fullsized":"2024-04-25T10:06:38.326478+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:16.043879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326336+0000","last_change":"2024-04-25T10:06:37.248060+0000","last_active":"2024-04-25T10:06:38.326336+0000","last_peered":"2024-04-25T10:06:38.326336+0000","last_clean":"2024-04-25T10:06:38.326336+0000","last_became_active":"2024-04-25T10:06:37.247760+0000","last_became_peered":"2024-04-25T10:06:37.247760+0000","last_unstale":"2024-04-25T10:06:38.326336+0000","last_undegraded":"2024-04-25T10:06:38.326336+0000","last_fullsized":"2024-04-25T10:06:38.326336+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:47.119623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219064+0000","last_change":"2024-04-25T10:06:36.174512+0000","last_active":"2024-04-25T10:06:38.219064+0000","last_peered":"2024-04-25T10:06:38.219064+0000","last_clean":"2024-04-25T10:06:38.219064+0000","last_became_active":"2024-04-25T10:06:36.173707+0000","last_became_peered":"2024-04-25T10:06:36.173707+0000","last_unstale":"2024-04-25T10:06:38.219064+0000","last_undegraded":"2024-04-25T10:06:38.219064+0000","last_fullsized":"2024-04-25T10:06:38.219064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:03.286057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328440+0000","last_change":"2024-04-25T10:06:37.221981+0000","last_active":"2024-04-25T10:06:38.328440+0000","last_peered":"2024-04-25T10:06:38.328440+0000","last_clean":"2024-04-25T10:06:38.328440+0000","last_became_active":"2024-04-25T10:06:37.221656+0000","last_became_peered":"2024-04-25T10:06:37.221656+0000","last_unstale":"2024-04-25T10:06:38.328440+0000","last_undegraded":"2024-04-25T10:06:38.328440+0000","last_fullsized":"2024-04-25T10:06:38.328440+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:28.091464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327453+0000","last_change":"2024-04-25T10:06:36.189290+0000","last_active":"2024-04-25T10:06:38.327453+0000","last_peered":"2024-04-25T10:06:38.327453+0000","last_clean":"2024-04-25T10:06:38.327453+0000","last_became_active":"2024-04-25T10:06:36.188473+0000","last_became_peered":"2024-04-25T10:06:36.188473+0000","last_unstale":"2024-04-25T10:06:38.327453+0000","last_undegraded":"2024-04-25T10:06:38.327453+0000","last_fullsized":"2024-04-25T10:06:38.327453+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:47.723565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326637+0000","last_change":"2024-04-25T10:06:37.257919+0000","last_active":"2024-04-25T10:06:38.326637+0000","last_peered":"2024-04-25T10:06:38.326637+0000","last_clean":"2024-04-25T10:06:38.326637+0000","last_became_active":"2024-04-25T10:06:37.257564+0000","last_became_peered":"2024-04-25T10:06:37.257564+0000","last_unstale":"2024-04-25T10:06:38.326637+0000","last_undegraded":"2024-04-25T10:06:38.326637+0000","last_fullsized":"2024-04-25T10:06:38.326637+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:03.343765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220854+0000","last_change":"2024-04-25T10:06:36.164318+0000","last_active":"2024-04-25T10:06:38.220854+0000","last_peered":"2024-04-25T10:06:38.220854+0000","last_clean":"2024-04-25T10:06:38.220854+0000","last_became_active":"2024-04-25T10:06:36.163584+0000","last_became_peered":"2024-04-25T10:06:36.163584+0000","last_unstale":"2024-04-25T10:06:38.220854+0000","last_undegraded":"2024-04-25T10:06:38.220854+0000","last_fullsized":"2024-04-25T10:06:38.220854+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:03.286057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219688+0000","last_change":"2024-04-25T10:06:37.258558+0000","last_active":"2024-04-25T10:06:38.219688+0000","last_peered":"2024-04-25T10:06:38.219688+0000","last_clean":"2024-04-25T10:06:38.219688+0000","last_became_active":"2024-04-25T10:06:37.258170+0000","last_became_peered":"2024-04-25T10:06:37.258170+0000","last_unstale":"2024-04-25T10:06:38.219688+0000","last_undegraded":"2024-04-25T10:06:38.219688+0000","last_fullsized":"2024-04-25T10:06:38.219688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:08.891150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327539+0000","last_change":"2024-04-25T10:06:36.187933+0000","last_active":"2024-04-25T10:06:38.327539+0000","last_peered":"2024-04-25T10:06:38.327539+0000","last_clean":"2024-04-25T10:06:38.327539+0000","last_became_active":"2024-04-25T10:06:36.185970+0000","last_became_peered":"2024-04-25T10:06:36.185970+0000","last_unstale":"2024-04-25T10:06:38.327539+0000","last_undegraded":"2024-04-25T10:06:38.327539+0000","last_fullsized":"2024-04-25T10:06:38.327539+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:15.337553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221296+0000","last_change":"2024-04-25T10:06:37.230247+0000","last_active":"2024-04-25T10:06:38.221296+0000","last_peered":"2024-04-25T10:06:38.221296+0000","last_clean":"2024-04-25T10:06:38.221296+0000","last_became_active":"2024-04-25T10:06:37.229396+0000","last_became_peered":"2024-04-25T10:06:37.229396+0000","last_unstale":"2024-04-25T10:06:38.221296+0000","last_undegraded":"2024-04-25T10:06:38.221296+0000","last_fullsized":"2024-04-25T10:06:38.221296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:21.415350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328572+0000","last_change":"2024-04-25T10:06:36.183609+0000","last_active":"2024-04-25T10:06:38.328572+0000","last_peered":"2024-04-25T10:06:38.328572+0000","last_clean":"2024-04-25T10:06:38.328572+0000","last_became_active":"2024-04-25T10:06:36.183246+0000","last_became_peered":"2024-04-25T10:06:36.183246+0000","last_unstale":"2024-04-25T10:06:38.328572+0000","last_undegraded":"2024-04-25T10:06:38.328572+0000","last_fullsized":"2024-04-25T10:06:38.328572+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:13.897357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.218835+0000","last_change":"2024-04-25T10:06:37.239448+0000","last_active":"2024-04-25T10:06:38.218835+0000","last_peered":"2024-04-25T10:06:38.218835+0000","last_clean":"2024-04-25T10:06:38.218835+0000","last_became_active":"2024-04-25T10:06:37.238870+0000","last_became_peered":"2024-04-25T10:06:37.238870+0000","last_unstale":"2024-04-25T10:06:38.218835+0000","last_undegraded":"2024-04-25T10:06:38.218835+0000","last_fullsized":"2024-04-25T10:06:38.218835+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:02.425868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326861+0000","last_change":"2024-04-25T10:06:36.186599+0000","last_active":"2024-04-25T10:06:38.326861+0000","last_peered":"2024-04-25T10:06:38.326861+0000","last_clean":"2024-04-25T10:06:38.326861+0000","last_became_active":"2024-04-25T10:06:36.185813+0000","last_became_peered":"2024-04-25T10:06:36.185813+0000","last_unstale":"2024-04-25T10:06:38.326861+0000","last_undegraded":"2024-04-25T10:06:38.326861+0000","last_fullsized":"2024-04-25T10:06:38.326861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:21.437190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326536+0000","last_change":"2024-04-25T10:06:37.238060+0000","last_active":"2024-04-25T10:06:38.326536+0000","last_peered":"2024-04-25T10:06:38.326536+0000","last_clean":"2024-04-25T10:06:38.326536+0000","last_became_active":"2024-04-25T10:06:37.237536+0000","last_became_peered":"2024-04-25T10:06:37.237536+0000","last_unstale":"2024-04-25T10:06:38.326536+0000","last_undegraded":"2024-04-25T10:06:38.326536+0000","last_fullsized":"2024-04-25T10:06:38.326536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:52.001010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325659+0000","last_change":"2024-04-25T10:06:36.188672+0000","last_active":"2024-04-25T10:06:38.325659+0000","last_peered":"2024-04-25T10:06:38.325659+0000","last_clean":"2024-04-25T10:06:38.325659+0000","last_became_active":"2024-04-25T10:06:36.181630+0000","last_became_peered":"2024-04-25T10:06:36.181630+0000","last_unstale":"2024-04-25T10:06:38.325659+0000","last_undegraded":"2024-04-25T10:06:38.325659+0000","last_fullsized":"2024-04-25T10:06:38.325659+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:27.088275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219754+0000","last_change":"2024-04-25T10:06:37.220268+0000","last_active":"2024-04-25T10:06:38.219754+0000","last_peered":"2024-04-25T10:06:38.219754+0000","last_clean":"2024-04-25T10:06:38.219754+0000","last_became_active":"2024-04-25T10:06:37.219258+0000","last_became_peered":"2024-04-25T10:06:37.219258+0000","last_unstale":"2024-04-25T10:06:38.219754+0000","last_undegraded":"2024-04-25T10:06:38.219754+0000","last_fullsized":"2024-04-25T10:06:38.219754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:41.228850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326849+0000","last_change":"2024-04-25T10:06:36.186685+0000","last_active":"2024-04-25T10:06:38.326849+0000","last_peered":"2024-04-25T10:06:38.326849+0000","last_clean":"2024-04-25T10:06:38.326849+0000","last_became_active":"2024-04-25T10:06:36.186019+0000","last_became_peered":"2024-04-25T10:06:36.186019+0000","last_unstale":"2024-04-25T10:06:38.326849+0000","last_undegraded":"2024-04-25T10:06:38.326849+0000","last_fullsized":"2024-04-25T10:06:38.326849+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:32.143965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328540+0000","last_change":"2024-04-25T10:06:37.239667+0000","last_active":"2024-04-25T10:06:38.328540+0000","last_peered":"2024-04-25T10:06:38.328540+0000","last_clean":"2024-04-25T10:06:38.328540+0000","last_became_active":"2024-04-25T10:06:37.239341+0000","last_became_peered":"2024-04-25T10:06:37.239341+0000","last_unstale":"2024-04-25T10:06:38.328540+0000","last_undegraded":"2024-04-25T10:06:38.328540+0000","last_fullsized":"2024-04-25T10:06:38.328540+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:58.223769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.218761+0000","last_change":"2024-04-25T10:06:36.175797+0000","last_active":"2024-04-25T10:06:38.218761+0000","last_peered":"2024-04-25T10:06:38.218761+0000","last_clean":"2024-04-25T10:06:38.218761+0000","last_became_active":"2024-04-25T10:06:36.169195+0000","last_became_peered":"2024-04-25T10:06:36.169195+0000","last_unstale":"2024-04-25T10:06:38.218761+0000","last_undegraded":"2024-04-25T10:06:38.218761+0000","last_fullsized":"2024-04-25T10:06:38.218761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:28.539746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326972+0000","last_change":"2024-04-25T10:06:37.249705+0000","last_active":"2024-04-25T10:06:38.326972+0000","last_peered":"2024-04-25T10:06:38.326972+0000","last_clean":"2024-04-25T10:06:38.326972+0000","last_became_active":"2024-04-25T10:06:37.249086+0000","last_became_peered":"2024-04-25T10:06:37.249086+0000","last_unstale":"2024-04-25T10:06:38.326972+0000","last_undegraded":"2024-04-25T10:06:38.326972+0000","last_fullsized":"2024-04-25T10:06:38.326972+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:25.576789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327051+0000","last_change":"2024-04-25T10:06:36.194242+0000","last_active":"2024-04-25T10:06:38.327051+0000","last_peered":"2024-04-25T10:06:38.327051+0000","last_clean":"2024-04-25T10:06:38.327051+0000","last_became_active":"2024-04-25T10:06:36.189515+0000","last_became_peered":"2024-04-25T10:06:36.189515+0000","last_unstale":"2024-04-25T10:06:38.327051+0000","last_undegraded":"2024-04-25T10:06:38.327051+0000","last_fullsized":"2024-04-25T10:06:38.327051+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:05.887740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221031+0000","last_change":"2024-04-25T10:06:37.211856+0000","last_active":"2024-04-25T10:06:38.221031+0000","last_peered":"2024-04-25T10:06:38.221031+0000","last_clean":"2024-04-25T10:06:38.221031+0000","last_became_active":"2024-04-25T10:06:37.210185+0000","last_became_peered":"2024-04-25T10:06:37.210185+0000","last_unstale":"2024-04-25T10:06:38.221031+0000","last_undegraded":"2024-04-25T10:06:38.221031+0000","last_fullsized":"2024-04-25T10:06:38.221031+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:21.674991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327414+0000","last_change":"2024-04-25T10:06:36.186486+0000","last_active":"2024-04-25T10:06:38.327414+0000","last_peered":"2024-04-25T10:06:38.327414+0000","last_clean":"2024-04-25T10:06:38.327414+0000","last_became_active":"2024-04-25T10:06:36.185838+0000","last_became_peered":"2024-04-25T10:06:36.185838+0000","last_unstale":"2024-04-25T10:06:38.327414+0000","last_undegraded":"2024-04-25T10:06:38.327414+0000","last_fullsized":"2024-04-25T10:06:38.327414+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:55.366280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220597+0000","last_change":"2024-04-25T10:06:37.215393+0000","last_active":"2024-04-25T10:06:38.220597+0000","last_peered":"2024-04-25T10:06:38.220597+0000","last_clean":"2024-04-25T10:06:38.220597+0000","last_became_active":"2024-04-25T10:06:37.214876+0000","last_became_peered":"2024-04-25T10:06:37.214876+0000","last_unstale":"2024-04-25T10:06:38.220597+0000","last_undegraded":"2024-04-25T10:06:38.220597+0000","last_fullsized":"2024-04-25T10:06:38.220597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:05.743949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327305+0000","last_change":"2024-04-25T10:06:36.188577+0000","last_active":"2024-04-25T10:06:38.327305+0000","last_peered":"2024-04-25T10:06:38.327305+0000","last_clean":"2024-04-25T10:06:38.327305+0000","last_became_active":"2024-04-25T10:06:36.186800+0000","last_became_peered":"2024-04-25T10:06:36.186800+0000","last_unstale":"2024-04-25T10:06:38.327305+0000","last_undegraded":"2024-04-25T10:06:38.327305+0000","last_fullsized":"2024-04-25T10:06:38.327305+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:48.883589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219201+0000","last_change":"2024-04-25T10:06:37.223539+0000","last_active":"2024-04-25T10:06:38.219201+0000","last_peered":"2024-04-25T10:06:38.219201+0000","last_clean":"2024-04-25T10:06:38.219201+0000","last_became_active":"2024-04-25T10:06:37.222327+0000","last_became_peered":"2024-04-25T10:06:37.222327+0000","last_unstale":"2024-04-25T10:06:38.219201+0000","last_undegraded":"2024-04-25T10:06:38.219201+0000","last_fullsized":"2024-04-25T10:06:38.219201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:52.879097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326502+0000","last_change":"2024-04-25T10:06:36.168120+0000","last_active":"2024-04-25T10:06:38.326502+0000","last_peered":"2024-04-25T10:06:38.326502+0000","last_clean":"2024-04-25T10:06:38.326502+0000","last_became_active":"2024-04-25T10:06:36.167819+0000","last_became_peered":"2024-04-25T10:06:36.167819+0000","last_unstale":"2024-04-25T10:06:38.326502+0000","last_undegraded":"2024-04-25T10:06:38.326502+0000","last_fullsized":"2024-04-25T10:06:38.326502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:39.222722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219017+0000","last_change":"2024-04-25T10:06:37.220420+0000","last_active":"2024-04-25T10:06:38.219017+0000","last_peered":"2024-04-25T10:06:38.219017+0000","last_clean":"2024-04-25T10:06:38.219017+0000","last_became_active":"2024-04-25T10:06:37.220000+0000","last_became_peered":"2024-04-25T10:06:37.220000+0000","last_unstale":"2024-04-25T10:06:38.219017+0000","last_undegraded":"2024-04-25T10:06:38.219017+0000","last_fullsized":"2024-04-25T10:06:38.219017+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.392643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219056+0000","last_change":"2024-04-25T10:06:36.170236+0000","last_active":"2024-04-25T10:06:38.219056+0000","last_peered":"2024-04-25T10:06:38.219056+0000","last_clean":"2024-04-25T10:06:38.219056+0000","last_became_active":"2024-04-25T10:06:36.169376+0000","last_became_peered":"2024-04-25T10:06:36.169376+0000","last_unstale":"2024-04-25T10:06:38.219056+0000","last_undegraded":"2024-04-25T10:06:38.219056+0000","last_fullsized":"2024-04-25T10:06:38.219056+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:55.366280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327777+0000","last_change":"2024-04-25T10:06:37.235344+0000","last_active":"2024-04-25T10:06:38.327777+0000","last_peered":"2024-04-25T10:06:38.327777+0000","last_clean":"2024-04-25T10:06:38.327777+0000","last_became_active":"2024-04-25T10:06:37.218220+0000","last_became_peered":"2024-04-25T10:06:37.218220+0000","last_unstale":"2024-04-25T10:06:38.327777+0000","last_undegraded":"2024-04-25T10:06:38.327777+0000","last_fullsized":"2024-04-25T10:06:38.327777+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:25.576789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325931+0000","last_change":"2024-04-25T10:06:36.193835+0000","last_active":"2024-04-25T10:06:38.325931+0000","last_peered":"2024-04-25T10:06:38.325931+0000","last_clean":"2024-04-25T10:06:38.325931+0000","last_became_active":"2024-04-25T10:06:36.189802+0000","last_became_peered":"2024-04-25T10:06:36.189802+0000","last_unstale":"2024-04-25T10:06:38.325931+0000","last_undegraded":"2024-04-25T10:06:38.325931+0000","last_fullsized":"2024-04-25T10:06:38.325931+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:57.220580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220127+0000","last_change":"2024-04-25T10:06:37.221054+0000","last_active":"2024-04-25T10:06:38.220127+0000","last_peered":"2024-04-25T10:06:38.220127+0000","last_clean":"2024-04-25T10:06:38.220127+0000","last_became_active":"2024-04-25T10:06:37.219584+0000","last_became_peered":"2024-04-25T10:06:37.219584+0000","last_unstale":"2024-04-25T10:06:38.220127+0000","last_undegraded":"2024-04-25T10:06:38.220127+0000","last_fullsized":"2024-04-25T10:06:38.220127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:14.900546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219007+0000","last_change":"2024-04-25T10:06:36.175591+0000","last_active":"2024-04-25T10:06:38.219007+0000","last_peered":"2024-04-25T10:06:38.219007+0000","last_clean":"2024-04-25T10:06:38.219007+0000","last_became_active":"2024-04-25T10:06:36.175299+0000","last_became_peered":"2024-04-25T10:06:36.175299+0000","last_unstale":"2024-04-25T10:06:38.219007+0000","last_undegraded":"2024-04-25T10:06:38.219007+0000","last_fullsized":"2024-04-25T10:06:38.219007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.733077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219018+0000","last_change":"2024-04-25T10:06:37.211950+0000","last_active":"2024-04-25T10:06:38.219018+0000","last_peered":"2024-04-25T10:06:38.219018+0000","last_clean":"2024-04-25T10:06:38.219018+0000","last_became_active":"2024-04-25T10:06:37.211245+0000","last_became_peered":"2024-04-25T10:06:37.211245+0000","last_unstale":"2024-04-25T10:06:38.219018+0000","last_undegraded":"2024-04-25T10:06:38.219018+0000","last_fullsized":"2024-04-25T10:06:38.219018+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:05.786554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327563+0000","last_change":"2024-04-25T10:06:36.194979+0000","last_active":"2024-04-25T10:06:38.327563+0000","last_peered":"2024-04-25T10:06:38.327563+0000","last_clean":"2024-04-25T10:06:38.327563+0000","last_became_active":"2024-04-25T10:06:36.194166+0000","last_became_peered":"2024-04-25T10:06:36.194166+0000","last_unstale":"2024-04-25T10:06:38.327563+0000","last_undegraded":"2024-04-25T10:06:38.327563+0000","last_fullsized":"2024-04-25T10:06:38.327563+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:55.366280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328752+0000","last_change":"2024-04-25T10:06:37.215396+0000","last_active":"2024-04-25T10:06:38.328752+0000","last_peered":"2024-04-25T10:06:38.328752+0000","last_clean":"2024-04-25T10:06:38.328752+0000","last_became_active":"2024-04-25T10:06:37.214988+0000","last_became_peered":"2024-04-25T10:06:37.214988+0000","last_unstale":"2024-04-25T10:06:38.328752+0000","last_undegraded":"2024-04-25T10:06:38.328752+0000","last_fullsized":"2024-04-25T10:06:38.328752+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:56.006198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326830+0000","last_change":"2024-04-25T10:06:36.181677+0000","last_active":"2024-04-25T10:06:38.326830+0000","last_peered":"2024-04-25T10:06:38.326830+0000","last_clean":"2024-04-25T10:06:38.326830+0000","last_became_active":"2024-04-25T10:06:36.180832+0000","last_became_peered":"2024-04-25T10:06:36.180832+0000","last_unstale":"2024-04-25T10:06:38.326830+0000","last_undegraded":"2024-04-25T10:06:38.326830+0000","last_fullsized":"2024-04-25T10:06:38.326830+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.733077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326327+0000","last_change":"2024-04-25T10:06:37.258667+0000","last_active":"2024-04-25T10:06:38.326327+0000","last_peered":"2024-04-25T10:06:38.326327+0000","last_clean":"2024-04-25T10:06:38.326327+0000","last_became_active":"2024-04-25T10:06:37.256627+0000","last_became_peered":"2024-04-25T10:06:37.256627+0000","last_unstale":"2024-04-25T10:06:38.326327+0000","last_undegraded":"2024-04-25T10:06:38.326327+0000","last_fullsized":"2024-04-25T10:06:38.326327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:11.227591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219311+0000","last_change":"2024-04-25T10:06:36.191971+0000","last_active":"2024-04-25T10:06:38.219311+0000","last_peered":"2024-04-25T10:06:38.219311+0000","last_clean":"2024-04-25T10:06:38.219311+0000","last_became_active":"2024-04-25T10:06:36.191546+0000","last_became_peered":"2024-04-25T10:06:36.191546+0000","last_unstale":"2024-04-25T10:06:38.219311+0000","last_undegraded":"2024-04-25T10:06:38.219311+0000","last_fullsized":"2024-04-25T10:06:38.219311+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:47.723565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.219639+0000","last_change":"2024-04-25T10:06:37.223021+0000","last_active":"2024-04-25T10:06:38.219639+0000","last_peered":"2024-04-25T10:06:38.219639+0000","last_clean":"2024-04-25T10:06:38.219639+0000","last_became_active":"2024-04-25T10:06:37.221312+0000","last_became_peered":"2024-04-25T10:06:37.221312+0000","last_unstale":"2024-04-25T10:06:38.219639+0000","last_undegraded":"2024-04-25T10:06:38.219639+0000","last_fullsized":"2024-04-25T10:06:38.219639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:36.845800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.328700+0000","last_change":"2024-04-25T10:06:36.183655+0000","last_active":"2024-04-25T10:06:38.328700+0000","last_peered":"2024-04-25T10:06:38.328700+0000","last_clean":"2024-04-25T10:06:38.328700+0000","last_became_active":"2024-04-25T10:06:36.183250+0000","last_became_peered":"2024-04-25T10:06:36.183250+0000","last_unstale":"2024-04-25T10:06:38.328700+0000","last_undegraded":"2024-04-25T10:06:38.328700+0000","last_fullsized":"2024-04-25T10:06:38.328700+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:53.225771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.218979+0000","last_change":"2024-04-25T10:06:37.218508+0000","last_active":"2024-04-25T10:06:38.218979+0000","last_peered":"2024-04-25T10:06:38.218979+0000","last_clean":"2024-04-25T10:06:38.218979+0000","last_became_active":"2024-04-25T10:06:37.216279+0000","last_became_peered":"2024-04-25T10:06:37.216279+0000","last_unstale":"2024-04-25T10:06:38.218979+0000","last_undegraded":"2024-04-25T10:06:38.218979+0000","last_fullsized":"2024-04-25T10:06:38.218979+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:05.743949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327149+0000","last_change":"2024-04-25T10:06:36.187358+0000","last_active":"2024-04-25T10:06:38.327149+0000","last_peered":"2024-04-25T10:06:38.327149+0000","last_clean":"2024-04-25T10:06:38.327149+0000","last_became_active":"2024-04-25T10:06:36.186686+0000","last_became_peered":"2024-04-25T10:06:36.186686+0000","last_unstale":"2024-04-25T10:06:38.327149+0000","last_undegraded":"2024-04-25T10:06:38.327149+0000","last_fullsized":"2024-04-25T10:06:38.327149+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.733077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.221375+0000","last_change":"2024-04-25T10:06:37.235221+0000","last_active":"2024-04-25T10:06:38.221375+0000","last_peered":"2024-04-25T10:06:38.221375+0000","last_clean":"2024-04-25T10:06:38.221375+0000","last_became_active":"2024-04-25T10:06:37.229996+0000","last_became_peered":"2024-04-25T10:06:37.229996+0000","last_unstale":"2024-04-25T10:06:38.221375+0000","last_undegraded":"2024-04-25T10:06:38.221375+0000","last_fullsized":"2024-04-25T10:06:38.221375+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:14.900546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326580+0000","last_change":"2024-04-25T10:06:36.157589+0000","last_active":"2024-04-25T10:06:38.326580+0000","last_peered":"2024-04-25T10:06:38.326580+0000","last_clean":"2024-04-25T10:06:38.326580+0000","last_became_active":"2024-04-25T10:06:36.157314+0000","last_became_peered":"2024-04-25T10:06:36.157314+0000","last_unstale":"2024-04-25T10:06:38.326580+0000","last_undegraded":"2024-04-25T10:06:38.326580+0000","last_fullsized":"2024-04-25T10:06:38.326580+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:04.783365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327332+0000","last_change":"2024-04-25T10:06:37.248310+0000","last_active":"2024-04-25T10:06:38.327332+0000","last_peered":"2024-04-25T10:06:38.327332+0000","last_clean":"2024-04-25T10:06:38.327332+0000","last_became_active":"2024-04-25T10:06:37.247412+0000","last_became_peered":"2024-04-25T10:06:37.247412+0000","last_unstale":"2024-04-25T10:06:38.327332+0000","last_undegraded":"2024-04-25T10:06:38.327332+0000","last_fullsized":"2024-04-25T10:06:38.327332+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:23.121736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326595+0000","last_change":"2024-04-25T10:06:36.169365+0000","last_active":"2024-04-25T10:06:38.326595+0000","last_peered":"2024-04-25T10:06:38.326595+0000","last_clean":"2024-04-25T10:06:38.326595+0000","last_became_active":"2024-04-25T10:06:36.168972+0000","last_became_peered":"2024-04-25T10:06:36.168972+0000","last_unstale":"2024-04-25T10:06:38.326595+0000","last_undegraded":"2024-04-25T10:06:38.326595+0000","last_fullsized":"2024-04-25T10:06:38.326595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:06.628360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.327765+0000","last_change":"2024-04-25T10:06:37.259092+0000","last_active":"2024-04-25T10:06:38.327765+0000","last_peered":"2024-04-25T10:06:38.327765+0000","last_clean":"2024-04-25T10:06:38.327765+0000","last_became_active":"2024-04-25T10:06:37.258761+0000","last_became_peered":"2024-04-25T10:06:37.258761+0000","last_unstale":"2024-04-25T10:06:38.327765+0000","last_undegraded":"2024-04-25T10:06:38.327765+0000","last_fullsized":"2024-04-25T10:06:38.327765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:38.209278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326919+0000","last_change":"2024-04-25T10:06:36.170925+0000","last_active":"2024-04-25T10:06:38.326919+0000","last_peered":"2024-04-25T10:06:38.326919+0000","last_clean":"2024-04-25T10:06:38.326919+0000","last_became_active":"2024-04-25T10:06:36.170521+0000","last_became_peered":"2024-04-25T10:06:36.170521+0000","last_unstale":"2024-04-25T10:06:38.326919+0000","last_undegraded":"2024-04-25T10:06:38.326919+0000","last_fullsized":"2024-04-25T10:06:38.326919+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:20.412161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.220036+0000","last_change":"2024-04-25T10:06:37.236349+0000","last_active":"2024-04-25T10:06:38.220036+0000","last_peered":"2024-04-25T10:06:38.220036+0000","last_clean":"2024-04-25T10:06:38.220036+0000","last_became_active":"2024-04-25T10:06:37.235659+0000","last_became_peered":"2024-04-25T10:06:37.235659+0000","last_unstale":"2024-04-25T10:06:38.220036+0000","last_undegraded":"2024-04-25T10:06:38.220036+0000","last_fullsized":"2024-04-25T10:06:38.220036+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:36.139473+0000","last_clean_scrub_stamp":"2024-04-25T10:06:36.139473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:21.415350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.326076+0000","last_change":"2024-04-25T10:06:36.161401+0000","last_active":"2024-04-25T10:06:38.326076+0000","last_peered":"2024-04-25T10:06:38.326076+0000","last_clean":"2024-04-25T10:06:38.326076+0000","last_became_active":"2024-04-25T10:06:36.161102+0000","last_became_peered":"2024-04-25T10:06:36.161102+0000","last_unstale":"2024-04-25T10:06:38.326076+0000","last_undegraded":"2024-04-25T10:06:38.326076+0000","last_fullsized":"2024-04-25T10:06:38.326076+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:35.136284+0000","last_clean_scrub_stamp":"2024-04-25T10:06:35.136284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:19.507488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:06:38.325952+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:06:38.325952+0000","last_peered":"2024-04-25T10:06:38.325952+0000","last_clean":"2024-04-25T10:06:38.325952+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:06:38.325952+0000","last_undegraded":"2024-04-25T10:06:38.325952+0000","last_fullsized":"2024-04-25T10:06:38.325952+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50604,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321236,"statfs":{"total":96636764160,"available":96584945664,"internally_reserved":0,"allocated":1409024,"data_stored":953565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40284,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331556,"statfs":{"total":96636764160,"available":96595513344,"internally_reserved":0,"allocated":802816,"data_stored":362455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40288,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331552,"statfs":{"total":96636764160,"available":96595509248,"internally_reserved":0,"allocated":802816,"data_stored":362443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45868,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325972,"statfs":{"total":96636764160,"available":96589795328,"internally_reserved":0,"allocated":811008,"data_stored":364033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":362421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44444,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327396,"statfs":{"total":96636764160,"available":96591253504,"internally_reserved":0,"allocated":798720,"data_stored":362421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40308,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331532,"statfs":{"total":96636764160,"available":96595488768,"internally_reserved":0,"allocated":827392,"data_stored":364797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50592,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321248,"statfs":{"total":96636764160,"available":96584957952,"internally_reserved":0,"allocated":1392640,"data_stored":952789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:40.547 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:06:40.547 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:06:40.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.706+0000 7f7aeab5c640 1 Processor -- start 2024-04-25T10:06:40.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.706+0000 7f7aeab5c640 1 -- start start 2024-04-25T10:06:40.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.706+0000 7f7aeab5c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ae4145170 0x7f7ae406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.706+0000 7f7aeab5c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7ae406e4d0 0x7f7ae40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.706+0000 7f7aeab5c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae40ce2e0 con 0x7f7ae40cdd60 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7aeab5c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae406db00 con 0x7f7ae406e4d0 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7aeab5c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae4145550 con 0x7f7ae4145170 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7ae90f9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7ae40cdd60 0x7f7ae415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57130/0 (socket says 172.21.15.133:57130) 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7ae90f9640 1 -- 172.21.15.133:0/3961003441 learned_addr learned my addr 172.21.15.133:0/3961003441 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adbfff640 1 --2- 172.21.15.133:0/3961003441 >> v2:172.21.15.162:3300/0 conn(0x7f7ae406e4d0 0x7f7ae40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7ae88f8640 1 --2- 172.21.15.133:0/3961003441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ae4145170 0x7f7ae406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430416625 0 0) 0x7f7ae40ce2e0 con 0x7f7ae40cdd60 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7acc003530 con 0x7f7ae40cdd60 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4073782648 0 0) 0x7f7acc003530 con 0x7f7ae40cdd60 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ae4145170 msgr2=0x7f7ae406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 --2- 172.21.15.133:0/3961003441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ae4145170 0x7f7ae406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 >> v2:172.21.15.162:3300/0 conn(0x7f7ae406e4d0 msgr2=0x7f7ae40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 --2- 172.21.15.133:0/3961003441 >> v2:172.21.15.162:3300/0 conn(0x7f7ae406e4d0 0x7f7ae40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae406d4b0 con 0x7f7ae40cdd60 2024-04-25T10:06:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7adc001a00 con 0x7f7ae40cdd60 2024-04-25T10:06:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 >> v1:172.21.15.133:6789/0 conn(0x7f7ae40cdd60 legacy=0x7f7ae415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 --2- 172.21.15.133:0/3961003441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7acc0039c0 0x7f7acc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 --2- 172.21.15.133:0/3961003441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ae4145170 0x7f7acc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 --2- 172.21.15.133:0/3961003441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7acc004d50 0x7f7acc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae406db00 con 0x7f7acc004d50 2024-04-25T10:06:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae4145550 con 0x7f7acc0039c0 2024-04-25T10:06:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc00fde0 con 0x7f7ae4145170 2024-04-25T10:06:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7ae90f9640 1 --2- 172.21.15.133:0/3961003441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7acc004d50 0x7f7acc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7ae88f8640 1 --2- 172.21.15.133:0/3961003441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7acc0039c0 0x7f7acc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7ae90f9640 1 -- 172.21.15.133:0/3961003441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ae4145170 msgr2=0x7f7acc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7ae90f9640 1 --2- 172.21.15.133:0/3961003441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ae4145170 0x7f7acc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7ae90f9640 1 -- 172.21.15.133:0/3961003441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7acc0039c0 msgr2=0x7f7acc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7ae90f9640 1 --2- 172.21.15.133:0/3961003441 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7acc0039c0 0x7f7acc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7ae90f9640 1 -- 172.21.15.133:0/3961003441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc003530 con 0x7f7acc004d50 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7ae90f9640 1 --2- 172.21.15.133:0/3961003441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7acc004d50 0x7f7acc0051d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00a8a0 tx=0x7f7adc01d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894e30dab39c470b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00a4c0 con 0x7f7acc004d50 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc0a9020 con 0x7f7acc004d50 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.710+0000 7f7adb7fe640 1 -- 172.21.15.133:0/3961003441 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0072d0 con 0x7f7acc004d50 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- 172.21.15.133:0/3961003441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7acc004d50 msgr2=0x7f7acc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 --2- 172.21.15.133:0/3961003441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7acc004d50 0x7f7acc0051d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00a8a0 tx=0x7f7adc01d3c0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7ae90f9640 1 -- 172.21.15.133:0/3961003441 reap_dead start 2024-04-25T10:06:40.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- 172.21.15.133:0/3961003441 shutdown_connections 2024-04-25T10:06:40.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- 172.21.15.133:0/3961003441 >> 172.21.15.133:0/3961003441 conn(0x7f7ae4066f70 msgr2=0x7f7ae406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- 172.21.15.133:0/3961003441 shutdown_connections 2024-04-25T10:06:40.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- 172.21.15.133:0/3961003441 wait complete. 2024-04-25T10:06:40.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 Processor -- start 2024-04-25T10:06:40.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- start start 2024-04-25T10:06:40.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc004d50 0x7f7ae40b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae40b1a70 0x7f7ae41f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae40b2840 0x7f7ae41f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae415d5e0 con 0x7f7ae40b1a70 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae415cb70 con 0x7f7ae40b2840 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae415d760 con 0x7f7acc004d50 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae40b1a70 0x7f7ae41f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae40b1a70 0x7f7ae41f1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43838/0 (socket says 172.21.15.133:43838) 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7adbfff640 1 -- 172.21.15.133:0/2496997637 learned_addr learned my addr 172.21.15.133:0/2496997637 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7ae88f8640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc004d50 0x7f7ae40b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7ae90f9640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae40b2840 0x7f7ae41f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7adbfff640 1 -- 172.21.15.133:0/2496997637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc004d50 msgr2=0x7f7ae40b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7adbfff640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc004d50 0x7f7ae40b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7adbfff640 1 -- 172.21.15.133:0/2496997637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae40b2840 msgr2=0x7f7ae41f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7adbfff640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae40b2840 0x7f7ae41f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7adbfff640 1 -- 172.21.15.133:0/2496997637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae40b3bd0 con 0x7f7ae40b1a70 2024-04-25T10:06:40.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7ae90f9640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae40b2840 0x7f7ae41f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:40.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7adbfff640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae40b1a70 0x7f7ae41f1ca0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400e860 tx=0x7f7ad400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7ad97fa640 1 -- 172.21.15.133:0/2496997637 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4017070 con 0x7f7ae40b1a70 2024-04-25T10:06:40.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7ad97fa640 1 -- 172.21.15.133:0/2496997637 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad400c8c0 con 0x7f7ae40b1a70 2024-04-25T10:06:40.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- 172.21.15.133:0/2496997637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae40b3eb0 con 0x7f7ae40b1a70 2024-04-25T10:06:40.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7ad97fa640 1 -- 172.21.15.133:0/2496997637 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400ccb0 con 0x7f7ae40b1a70 2024-04-25T10:06:40.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- 172.21.15.133:0/2496997637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae40b4470 con 0x7f7ae40b1a70 2024-04-25T10:06:40.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.714+0000 7f7aeab5c640 1 -- 172.21.15.133:0/2496997637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae4068e80 con 0x7f7ae40b1a70 2024-04-25T10:06:40.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.718+0000 7f7ad97fa640 1 -- 172.21.15.133:0/2496997637 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7ad401a070 con 0x7f7ae40b1a70 2024-04-25T10:06:40.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.718+0000 7f7ad97fa640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ad0066f90 0x7f7ad0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.718+0000 7f7ad97fa640 1 -- 172.21.15.133:0/2496997637 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7ad4085960 con 0x7f7ae40b1a70 2024-04-25T10:06:40.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.718+0000 7f7ae88f8640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ad0066f90 0x7f7ad0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.718+0000 7f7ae88f8640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ad0066f90 0x7f7ad0069450 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009950 tx=0x7f7ae0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.722+0000 7f7ad97fa640 1 -- 172.21.15.133:0/2496997637 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ad40567c0 con 0x7f7ae40b1a70 2024-04-25T10:06:40.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.922+0000 7f7aeab5c640 1 -- 172.21.15.133:0/2496997637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7ae4141fe0 con 0x7f7ae40b1a70 2024-04-25T10:06:40.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7ad97fa640 1 -- 172.21.15.133:0/2496997637 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7ad40560c0 con 0x7f7ae40b1a70 2024-04-25T10:06:40.929 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:06:40.929 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:06:40.929 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:06:40.929 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:06:40.929 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:40.929 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:06:40.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 -- 172.21.15.133:0/2496997637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ad0066f90 msgr2=0x7f7ad0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ad0066f90 0x7f7ad0069450 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009950 tx=0x7f7ae0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 -- 172.21.15.133:0/2496997637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae40b1a70 msgr2=0x7f7ae41f1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae40b1a70 0x7f7ae41f1ca0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400e860 tx=0x7f7ad400ed20 comp rx=0 tx=0).stop 2024-04-25T10:06:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 -- 172.21.15.133:0/2496997637 shutdown_connections 2024-04-25T10:06:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ae40b2840 0x7f7ae41f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ad0066f90 0x7f7ad0069450 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae40b1a70 0x7f7ae41f1ca0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 --2- 172.21.15.133:0/2496997637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7acc004d50 0x7f7ae40b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 -- 172.21.15.133:0/2496997637 >> 172.21.15.133:0/2496997637 conn(0x7f7ae4066f70 msgr2=0x7f7ae415e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 -- 172.21.15.133:0/2496997637 shutdown_connections 2024-04-25T10:06:40.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:40.926+0000 7f7aeab5c640 1 -- 172.21.15.133:0/2496997637 wait complete. 2024-04-25T10:06:40.946 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:41.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.054+0000 7f64e9d82640 1 Processor -- start 2024-04-25T10:06:41.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.054+0000 7f64e9d82640 1 -- start start 2024-04-25T10:06:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.058+0000 7f64e9d82640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f64e415d9b0 0x7f64e40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.058+0000 7f64e9d82640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e40ca9b0 con 0x7f64e40cee80 2024-04-25T10:06:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.058+0000 7f64e9d82640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e40cac90 con 0x7f64e415dd90 2024-04-25T10:06:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.058+0000 7f64e9d82640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e40cbe30 con 0x7f64e415d9b0 2024-04-25T10:06:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.058+0000 7f64e37fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f64e415d9b0 0x7f64e40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.058+0000 7f64e3fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f64e40cee80 0x7f64e406d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57136/0 (socket says 172.21.15.133:57136) 2024-04-25T10:06:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.058+0000 7f64e3fff640 1 -- 172.21.15.133:0/3500737690 learned_addr learned my addr 172.21.15.133:0/3500737690 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:41.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.058+0000 7f64e37fe640 1 -- 172.21.15.133:0/3500737690 >> v1:172.21.15.162:6789/0 conn(0x7f64e415dd90 legacy=0x7f64e40ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.058+0000 7f64e37fe640 1 -- 172.21.15.133:0/3500737690 >> v1:172.21.15.133:6789/0 conn(0x7f64e40cee80 legacy=0x7f64e406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.058+0000 7f64e27fc640 1 -- 172.21.15.133:0/3500737690 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158964113 0 0) 0x7f64e40ca9b0 con 0x7f64e40cee80 2024-04-25T10:06:41.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.058+0000 7f64e37fe640 1 -- 172.21.15.133:0/3500737690 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e40caf70 con 0x7f64e415d9b0 2024-04-25T10:06:41.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e37fe640 1 --2- 172.21.15.133:0/3500737690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e415d9b0 0x7f64e40cb5e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f64d40097c0 tx=0x7f64d401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cd000ebe4c7a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e27fc640 1 -- 172.21.15.133:0/3500737690 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d40b1070 con 0x7f64e415d9b0 2024-04-25T10:06:41.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e27fc640 1 -- 172.21.15.133:0/3500737690 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d40ac5f0 con 0x7f64e415d9b0 2024-04-25T10:06:41.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e27fc640 1 -- 172.21.15.133:0/3500737690 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d40ac9c0 con 0x7f64e415d9b0 2024-04-25T10:06:41.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e9d82640 1 -- 172.21.15.133:0/3500737690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e415d9b0 msgr2=0x7f64e40cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e9d82640 1 --2- 172.21.15.133:0/3500737690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e415d9b0 0x7f64e40cb5e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f64d40097c0 tx=0x7f64d401c370 comp rx=0 tx=0).stop 2024-04-25T10:06:41.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e9d82640 1 -- 172.21.15.133:0/3500737690 shutdown_connections 2024-04-25T10:06:41.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e9d82640 1 --2- 172.21.15.133:0/3500737690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e415d9b0 0x7f64e40cb5e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e9d82640 1 -- 172.21.15.133:0/3500737690 >> 172.21.15.133:0/3500737690 conn(0x7f64e4067000 msgr2=0x7f64e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e9d82640 1 -- 172.21.15.133:0/3500737690 shutdown_connections 2024-04-25T10:06:41.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e9d82640 1 -- 172.21.15.133:0/3500737690 wait complete. 2024-04-25T10:06:41.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.062+0000 7f64e9d82640 1 Processor -- start 2024-04-25T10:06:41.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e9d82640 1 -- start start 2024-04-25T10:06:41.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e9d82640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e40cee80 0x7f64e41413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e9d82640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64e415dd90 0x7f64e414fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e9d82640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e4150ba0 0x7f64e4149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e9d82640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e406ea20 con 0x7f64e4150ba0 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e9d82640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e406e8a0 con 0x7f64e415dd90 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e9d82640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e416b870 con 0x7f64e40cee80 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e4150ba0 0x7f64e4149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e4150ba0 0x7f64e4149630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43856/0 (socket says 172.21.15.133:43856) 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e3fff640 1 -- 172.21.15.133:0/1372487763 learned_addr learned my addr 172.21.15.133:0/1372487763 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e3fff640 1 -- 172.21.15.133:0/1372487763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e40cee80 msgr2=0x7f64e41413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e3fff640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e40cee80 0x7f64e41413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e3fff640 1 -- 172.21.15.133:0/1372487763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64e415dd90 msgr2=0x7f64e414fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e3fff640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64e415dd90 0x7f64e414fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e3fff640 1 -- 172.21.15.133:0/1372487763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e415f920 con 0x7f64e4150ba0 2024-04-25T10:06:41.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e3fff640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e4150ba0 0x7f64e4149630 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00caf0 tx=0x7f64dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64cbfff640 1 -- 172.21.15.133:0/1372487763 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc00aca0 con 0x7f64e4150ba0 2024-04-25T10:06:41.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64cbfff640 1 -- 172.21.15.133:0/1372487763 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc00ae40 con 0x7f64e4150ba0 2024-04-25T10:06:41.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e9d82640 1 -- 172.21.15.133:0/1372487763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e415fc00 con 0x7f64e4150ba0 2024-04-25T10:06:41.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64cbfff640 1 -- 172.21.15.133:0/1372487763 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc012620 con 0x7f64e4150ba0 2024-04-25T10:06:41.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e9d82640 1 -- 172.21.15.133:0/1372487763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e41601c0 con 0x7f64e4150ba0 2024-04-25T10:06:41.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e9d82640 1 -- 172.21.15.133:0/1372487763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e4068ef0 con 0x7f64e4150ba0 2024-04-25T10:06:41.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64cbfff640 1 -- 172.21.15.133:0/1372487763 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64dc0127c0 con 0x7f64e4150ba0 2024-04-25T10:06:41.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64cbfff640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64d0066f90 0x7f64d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64cbfff640 1 -- 172.21.15.133:0/1372487763 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f64dc0857b0 con 0x7f64e4150ba0 2024-04-25T10:06:41.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e37fe640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64d0066f90 0x7f64d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.066+0000 7f64e37fe640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64d0066f90 0x7f64d0069450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f64d40ae000 tx=0x7f64d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.070+0000 7f64cbfff640 1 -- 172.21.15.133:0/1372487763 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64dc0566c0 con 0x7f64e4150ba0 2024-04-25T10:06:41.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.250+0000 7f64e9d82640 1 -- 172.21.15.133:0/1372487763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64e406d960 con 0x7f64e4150ba0 2024-04-25T10:06:41.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.250+0000 7f64cbfff640 1 -- 172.21.15.133:0/1372487763 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f64dc055fc0 con 0x7f64e4150ba0 2024-04-25T10:06:41.255 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:41.255 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3081100850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3081100850},{"type":"v1","addr":"172.21.15.162:6834","nonce":3081100850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/272966073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":272966073},{"type":"v1","addr":"172.21.15.162:6835","nonce":272966073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2840204486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2840204486},{"type":"v1","addr":"172.21.15.133:6835","nonce":2840204486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:36.516727+0000","modified":"2024-04-25T10:06:38.534667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/236214118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":236214118},{"type":"v1","addr":"172.21.15.133:6837","nonce":236214118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:41.255 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 33 2024-04-25T10:06:41.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 -- 172.21.15.133:0/1372487763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64d0066f90 msgr2=0x7f64d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64d0066f90 0x7f64d0069450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f64d40ae000 tx=0x7f64d4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:41.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 -- 172.21.15.133:0/1372487763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e4150ba0 msgr2=0x7f64e4149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e4150ba0 0x7f64e4149630 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00caf0 tx=0x7f64dc00a060 comp rx=0 tx=0).stop 2024-04-25T10:06:41.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 -- 172.21.15.133:0/1372487763 shutdown_connections 2024-04-25T10:06:41.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64d0066f90 0x7f64d0069450 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e4150ba0 0x7f64e4149630 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64e415dd90 0x7f64e414fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 --2- 172.21.15.133:0/1372487763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64e40cee80 0x7f64e41413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 -- 172.21.15.133:0/1372487763 >> 172.21.15.133:0/1372487763 conn(0x7f64e4067000 msgr2=0x7f64e40b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 -- 172.21.15.133:0/1372487763 shutdown_connections 2024-04-25T10:06:41.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.254+0000 7f64e9d82640 1 -- 172.21.15.133:0/1372487763 wait complete. 2024-04-25T10:06:41.675 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:36.516727+0000', 'modified': '2024-04-25T10:06:38.534667+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6876}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6876': {'gid': 6876, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/236214118', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 236214118}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 236214118}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:06:41.675 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:06:41.675 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:06:41.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f34458de640 1 Processor -- start 2024-04-25T10:06:41.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f34458de640 1 -- start start 2024-04-25T10:06:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f34458de640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f344006e8b0 0x7f344015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f34458de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400aebc0 0x7f34400af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f34458de640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34401471f0 con 0x7f34400b1e50 2024-04-25T10:06:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f34458de640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3440147490 con 0x7f344006e8b0 2024-04-25T10:06:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f34458de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34400b19c0 con 0x7f34400aebc0 2024-04-25T10:06:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f343effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f344006e8b0 0x7f344015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f343effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f344006e8b0 0x7f344015eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43864/0 (socket says 172.21.15.133:43864) 2024-04-25T10:06:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f343effd640 1 -- 172.21.15.133:0/4191241806 learned_addr learned my addr 172.21.15.133:0/4191241806 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f343f7fe640 1 --2- 172.21.15.133:0/4191241806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400aebc0 0x7f34400af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f343effd640 1 -- 172.21.15.133:0/4191241806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400aebc0 msgr2=0x7f34400af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f343effd640 1 --2- 172.21.15.133:0/4191241806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400aebc0 0x7f34400af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f343effd640 1 -- 172.21.15.133:0/4191241806 >> v1:172.21.15.162:6789/0 conn(0x7f34400b1e50 legacy=0x7f3440146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f343effd640 1 -- 172.21.15.133:0/4191241806 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34400b1090 con 0x7f344006e8b0 2024-04-25T10:06:41.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.802+0000 7f343f7fe640 1 --2- 172.21.15.133:0/4191241806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400aebc0 0x7f34400af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:41.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f343effd640 1 --2- 172.21.15.133:0/4191241806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f344006e8b0 0x7f344015eb40 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3440147610 tx=0x7f343001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4808964cb765e819 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f343dffb640 1 -- 172.21.15.133:0/4191241806 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300a55c0 con 0x7f344006e8b0 2024-04-25T10:06:41.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f343dffb640 1 -- 172.21.15.133:0/4191241806 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34300a5760 con 0x7f344006e8b0 2024-04-25T10:06:41.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f343dffb640 1 -- 172.21.15.133:0/4191241806 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300a5b30 con 0x7f344006e8b0 2024-04-25T10:06:41.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f34458de640 1 -- 172.21.15.133:0/4191241806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f344006e8b0 msgr2=0x7f344015eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f34458de640 1 --2- 172.21.15.133:0/4191241806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f344006e8b0 0x7f344015eb40 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3440147610 tx=0x7f343001c940 comp rx=0 tx=0).stop 2024-04-25T10:06:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f34458de640 1 -- 172.21.15.133:0/4191241806 shutdown_connections 2024-04-25T10:06:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f34458de640 1 --2- 172.21.15.133:0/4191241806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400aebc0 0x7f34400af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f34458de640 1 --2- 172.21.15.133:0/4191241806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f344006e8b0 0x7f344015eb40 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f34458de640 1 -- 172.21.15.133:0/4191241806 >> 172.21.15.133:0/4191241806 conn(0x7f3440066f70 msgr2=0x7f344006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f34458de640 1 -- 172.21.15.133:0/4191241806 shutdown_connections 2024-04-25T10:06:41.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f34458de640 1 -- 172.21.15.133:0/4191241806 wait complete. 2024-04-25T10:06:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f34458de640 1 Processor -- start 2024-04-25T10:06:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.806+0000 7f34458de640 1 -- start start 2024-04-25T10:06:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f34458de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f344006e8b0 0x7f34400cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f34458de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400aebc0 0x7f3440145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f34458de640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34400b1e50 0x7f3440146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f34458de640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344006dc30 con 0x7f344006e8b0 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f34458de640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344006dab0 con 0x7f34400b1e50 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f34458de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344006ddb0 con 0x7f34400aebc0 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f344006e8b0 0x7f34400cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f344006e8b0 0x7f34400cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43878/0 (socket says 172.21.15.133:43878) 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343effd640 1 -- 172.21.15.133:0/2523465881 learned_addr learned my addr 172.21.15.133:0/2523465881 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343f7fe640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34400b1e50 0x7f3440146090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343e7fc640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400aebc0 0x7f3440145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343effd640 1 -- 172.21.15.133:0/2523465881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400aebc0 msgr2=0x7f3440145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343effd640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400aebc0 0x7f3440145b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343effd640 1 -- 172.21.15.133:0/2523465881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34400b1e50 msgr2=0x7f3440146090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343effd640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34400b1e50 0x7f3440146090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343effd640 1 -- 172.21.15.133:0/2523465881 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3440142c70 con 0x7f344006e8b0 2024-04-25T10:06:41.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343f7fe640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34400b1e50 0x7f3440146090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:41.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343effd640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f344006e8b0 0x7f34400cee60 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f34300af040 tx=0x7f34300196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f3427fff640 1 -- 172.21.15.133:0/2523465881 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300bc020 con 0x7f344006e8b0 2024-04-25T10:06:41.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f3427fff640 1 -- 172.21.15.133:0/2523465881 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34300b8690 con 0x7f344006e8b0 2024-04-25T10:06:41.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f34458de640 1 -- 172.21.15.133:0/2523465881 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34401440f0 con 0x7f344006e8b0 2024-04-25T10:06:41.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f3427fff640 1 -- 172.21.15.133:0/2523465881 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300b8a80 con 0x7f344006e8b0 2024-04-25T10:06:41.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f34458de640 1 -- 172.21.15.133:0/2523465881 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34401446b0 con 0x7f344006e8b0 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f3427fff640 1 -- 172.21.15.133:0/2523465881 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34300bf020 con 0x7f344006e8b0 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f3427fff640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f342c066f60 0x7f342c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f3427fff640 1 -- 172.21.15.133:0/2523465881 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3430120290 con 0x7f344006e8b0 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343e7fc640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f342c066f60 0x7f342c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f343e7fc640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f342c066f60 0x7f342c069420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f344015f200 tx=0x7f3434009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f3448a4b000 1 Processor -- start 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.810+0000 7f3448a4b000 1 -- start start 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f133d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 0x55ef1f0ea310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1efd15f0 con 0x55ef1f134240 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1ef38da0 con 0x55ef1f0e9f30 2024-04-25T10:06:41.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef1eec1150 con 0x55ef1f133920 2024-04-25T10:06:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0e99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43892/0 (socket says 172.21.15.133:43892) 2024-04-25T10:06:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343e7fc640 1 -- 172.21.15.133:0/1587738996 learned_addr learned my addr 172.21.15.133:0/1587738996 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343f7fe640 1 --2- 172.21.15.133:0/1587738996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 0x55ef1f0ea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343e7fc640 1 -- 172.21.15.133:0/1587738996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 msgr2=0x55ef1f133d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343effd640 1 --2- 172.21.15.133:0/1587738996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f133d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343e7fc640 1 --2- 172.21.15.133:0/1587738996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f133d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343e7fc640 1 -- 172.21.15.133:0/1587738996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 msgr2=0x55ef1f0ea310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343e7fc640 1 --2- 172.21.15.133:0/1587738996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 0x55ef1f0ea310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343e7fc640 1 -- 172.21.15.133:0/1587738996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef1eedaca0 con 0x55ef1f134240 2024-04-25T10:06:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343f7fe640 1 --2- 172.21.15.133:0/1587738996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 0x55ef1f0ea310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343e7fc640 1 --2- 172.21.15.133:0/1587738996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0e99f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f343400d820 tx=0x7f3434010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444ce95119a13112 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f34257fa640 1 -- 172.21.15.133:0/1587738996 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434024070 con 0x55ef1f134240 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f34257fa640 1 -- 172.21.15.133:0/1587738996 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434014ba0 con 0x55ef1f134240 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f34257fa640 1 -- 172.21.15.133:0/1587738996 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343401f400 con 0x55ef1f134240 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- 172.21.15.133:0/1587738996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 msgr2=0x55ef1f0e99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 --2- 172.21.15.133:0/1587738996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0e99f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f343400d820 tx=0x7f3434010000 comp rx=0 tx=0).stop 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- 172.21.15.133:0/1587738996 shutdown_connections 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 --2- 172.21.15.133:0/1587738996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 0x55ef1f0ea310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 --2- 172.21.15.133:0/1587738996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0e99f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 --2- 172.21.15.133:0/1587738996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f133d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- 172.21.15.133:0/1587738996 >> 172.21.15.133:0/1587738996 conn(0x55ef1f0f02a0 msgr2=0x55ef1f0e82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- 172.21.15.133:0/1587738996 shutdown_connections 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- 172.21.15.133:0/1587738996 wait complete. 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- start start 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 Processor -- start 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 0x55ef1f13add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f13b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1eef3990 con 0x55ef1f134240 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1ef34e40 con 0x55ef1f0e9f30 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef1eec4900 con 0x55ef1f133920 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f3448a4b000 10 client.0 _ll_register_callbacks cb 0x55ef1efec1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0edaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43902/0 (socket says 172.21.15.133:43902) 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343f7fe640 1 -- 172.21.15.133:0/119666517 learned_addr learned my addr 172.21.15.133:0/119666517 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343effd640 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 0x55ef1f13add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343e7fc640 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f13b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343f7fe640 1 -- 172.21.15.133:0/119666517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 msgr2=0x55ef1f13b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343f7fe640 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f13b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343f7fe640 1 -- 172.21.15.133:0/119666517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 msgr2=0x55ef1f13add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343f7fe640 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 0x55ef1f13add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343f7fe640 1 -- 172.21.15.133:0/119666517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef1eef1630 con 0x55ef1f134240 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.814+0000 7f343effd640 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 0x55ef1f13add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f343f7fe640 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0edaa0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f343800e560 tx=0x7f343800ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbead0617abc946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f341e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f341e7fc640 1 -- 172.21.15.133:0/119666517 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3438012840 con 0x55ef1f134240 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f341e7fc640 1 -- 172.21.15.133:0/119666517 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f34380129e0 con 0x55ef1f134240 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 -- 172.21.15.133:0/119666517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3434014ba0 con 0x55ef1f134240 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f341e7fc640 1 -- 172.21.15.133:0/119666517 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3438012c20 con 0x55ef1f134240 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 10 client.7017 fetch_fsmap learned FSMap version 33 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 -- 172.21.15.133:0/119666517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55ef1eec13a0 con 0x55ef1f134240 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f341e7fc640 1 -- 172.21.15.133:0/119666517 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434014ba0 con 0x55ef1f134240 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f341e7fc640 1 -- 172.21.15.133:0/119666517 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3438025080 con 0x55ef1f134240 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 10 client.7017 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 10 client.7017 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 20 client.7017 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.133:6836/236214118,v1:172.21.15.133:6837/236214118] conn(0x55ef1f133540 0x55ef1f0ee4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 4 client.7017 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 -- 172.21.15.133:0/119666517 --> [v2:172.21.15.133:6836/236214118,v1:172.21.15.133:6837/236214118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef1eed4a50 con 0x55ef1f133540 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f343effd640 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.133:6836/236214118,v1:172.21.15.133:6837/236214118] conn(0x55ef1f133540 0x55ef1f0ee4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f341e7fc640 10 client.7017 ms_handle_connect on v2:172.21.15.133:6836/236214118 2024-04-25T10:06:41.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f343effd640 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.133:6836/236214118,v1:172.21.15.133:6837/236214118] conn(0x55ef1f133540 0x55ef1f0ee4b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ed0e5a2cdf8671 server_cookie=fc7e6c4e2dfb4f53 in_seq=0 out_seq=0 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f341e7fc640 1 -- 172.21.15.133:0/119666517 <== mds.0 v2:172.21.15.133:6836/236214118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef1eed4a50 con 0x55ef1f133540 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f341e7fc640 10 client.7017 handle_command_reply: tid=0 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 client.7017 shutdown 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 -- 172.21.15.133:0/119666517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 msgr2=0x55ef1f0edaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0edaa0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f343800e560 tx=0x7f343800ea20 comp rx=0 tx=0).stop 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 -- 172.21.15.133:0/119666517 shutdown_connections 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.133:6836/236214118,v1:172.21.15.133:6837/236214118] conn(0x55ef1f133540 0x55ef1f0ee4b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f134240 0x55ef1f0edaa0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f13b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f341e7fc640 0 client.7017 ms_handle_reset on v2:172.21.15.133:6836/236214118 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 --2- 172.21.15.133:0/119666517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f0e9f30 0x55ef1f13add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 -- 172.21.15.133:0/119666517 >> 172.21.15.133:0/119666517 conn(0x55ef1f0f02a0 msgr2=0x55ef1f0e89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 -- 172.21.15.133:0/119666517 shutdown_connections 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 1 -- 172.21.15.133:0/119666517 wait complete. 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.818+0000 7f3448a4b000 20 client.7017 trim_cache size 0 max 16384 2024-04-25T10:06:41.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 Processor -- start 2024-04-25T10:06:41.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 -- start start 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f0e9f30 0x55ef1f0ed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f133920 0x55ef1f0ed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f134240 0x55ef1f13b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3440203060 con 0x55ef1f133920 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3440202230 con 0x55ef1f134240 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344015cc90 con 0x55ef1f0e9f30 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f343e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f133920 0x55ef1f0ed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f343e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f133920 0x55ef1f0ed870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43918/0 (socket says 172.21.15.133:43918) 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f343e7fc640 1 -- 172.21.15.133:0/3170164668 learned_addr learned my addr 172.21.15.133:0/3170164668 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f343e7fc640 1 -- 172.21.15.133:0/3170164668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f0e9f30 msgr2=0x55ef1f0ed330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f343e7fc640 1 --2- 172.21.15.133:0/3170164668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f0e9f30 0x55ef1f0ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f343f7fe640 1 --2- 172.21.15.133:0/3170164668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f134240 0x55ef1f13b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f343e7fc640 1 -- 172.21.15.133:0/3170164668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f134240 msgr2=0x55ef1f13b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f343e7fc640 1 --2- 172.21.15.133:0/3170164668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f134240 0x55ef1f13b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f343e7fc640 1 -- 172.21.15.133:0/3170164668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef1f134070 con 0x55ef1f133920 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f343e7fc640 1 --2- 172.21.15.133:0/3170164668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f133920 0x55ef1f0ed870 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x55ef1eedc200 tx=0x7f3434012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6df9131241684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3424ff9640 1 -- 172.21.15.133:0/3170164668 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434024040 con 0x55ef1f133920 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3424ff9640 1 -- 172.21.15.133:0/3170164668 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434029480 con 0x55ef1f133920 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3424ff9640 1 -- 172.21.15.133:0/3170164668 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434029850 con 0x55ef1f133920 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 -- 172.21.15.133:0/3170164668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f133920 msgr2=0x55ef1f0ed870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 --2- 172.21.15.133:0/3170164668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f133920 0x55ef1f0ed870 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x55ef1eedc200 tx=0x7f3434012000 comp rx=0 tx=0).stop 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 -- 172.21.15.133:0/3170164668 shutdown_connections 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 --2- 172.21.15.133:0/3170164668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f134240 0x55ef1f13b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 --2- 172.21.15.133:0/3170164668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f133920 0x55ef1f0ed870 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 --2- 172.21.15.133:0/3170164668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f0e9f30 0x55ef1f0ed330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 -- 172.21.15.133:0/3170164668 >> 172.21.15.133:0/3170164668 conn(0x55ef1f0f02a0 msgr2=0x55ef1f0ef3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 -- 172.21.15.133:0/3170164668 shutdown_connections 2024-04-25T10:06:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.834+0000 7f3448a4b000 1 -- 172.21.15.133:0/3170164668 wait complete. 2024-04-25T10:06:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- start start 2024-04-25T10:06:41.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 Processor -- start 2024-04-25T10:06:41.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f0e9f30 0x55ef1f194560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f194aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f134240 0x55ef1f13e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1ef3d3d0 con 0x55ef1f0e9f30 2024-04-25T10:06:41.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1eff4d50 con 0x55ef1f134240 2024-04-25T10:06:41.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef1eed3980 con 0x55ef1f133920 2024-04-25T10:06:41.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 10 client.0 _ll_register_callbacks cb 0x7f3434003a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:41.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f0e9f30 0x55ef1f194560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f0e9f30 0x55ef1f194560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43920/0 (socket says 172.21.15.133:43920) 2024-04-25T10:06:41.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343effd640 1 -- 172.21.15.133:0/2380774628 learned_addr learned my addr 172.21.15.133:0/2380774628 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343f7fe640 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f134240 0x55ef1f13e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343e7fc640 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f194aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343effd640 1 -- 172.21.15.133:0/2380774628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 msgr2=0x55ef1f194aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343effd640 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f194aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343effd640 1 -- 172.21.15.133:0/2380774628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f134240 msgr2=0x55ef1f13e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343effd640 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f134240 0x55ef1f13e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343effd640 1 -- 172.21.15.133:0/2380774628 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef1efba600 con 0x55ef1f0e9f30 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343effd640 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f0e9f30 0x55ef1f194560 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f34300a54d0 tx=0x7f3430009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c560794b6d6e33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f341e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f341e7fc640 1 -- 172.21.15.133:0/2380774628 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300f1e60 con 0x55ef1f0e9f30 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f341e7fc640 1 -- 172.21.15.133:0/2380774628 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f34300b8c20 con 0x55ef1f0e9f30 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- 172.21.15.133:0/2380774628 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3434029480 con 0x55ef1f0e9f30 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f341e7fc640 1 -- 172.21.15.133:0/2380774628 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34300f0b30 con 0x55ef1f0e9f30 2024-04-25T10:06:41.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 10 client.7023 fetch_fsmap learned FSMap version 33 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- 172.21.15.133:0/2380774628 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55ef1eecd9b0 con 0x55ef1f0e9f30 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f341e7fc640 1 -- 172.21.15.133:0/2380774628 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434029480 con 0x55ef1f0e9f30 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f341e7fc640 1 -- 172.21.15.133:0/2380774628 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f34300ed1a0 con 0x55ef1f0e9f30 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 10 client.7023 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 10 client.7023 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 20 client.7023 populate_metadata read hostname 'smithi133' 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.133:6836/236214118,v1:172.21.15.133:6837/236214118] conn(0x55ef1f13eb90 0x55ef1f13ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 4 client.7023 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- 172.21.15.133:0/2380774628 --> [v2:172.21.15.133:6836/236214118,v1:172.21.15.133:6837/236214118] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ef1ef462e0 con 0x55ef1f13eb90 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343e7fc640 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.133:6836/236214118,v1:172.21.15.133:6837/236214118] conn(0x55ef1f13eb90 0x55ef1f13ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f341e7fc640 10 client.7023 ms_handle_connect on v2:172.21.15.133:6836/236214118 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f343e7fc640 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.133:6836/236214118,v1:172.21.15.133:6837/236214118] conn(0x55ef1f13eb90 0x55ef1f13ef70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d5ba6a4e2c6010e server_cookie=26c766283eeaaaa3 in_seq=0 out_seq=0 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f341e7fc640 1 -- 172.21.15.133:0/2380774628 <== mds.0 v2:172.21.15.133:6836/236214118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55ef1ef462e0 con 0x55ef1f13eb90 2024-04-25T10:06:41.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f341e7fc640 10 client.7023 handle_command_reply: tid=0 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 client.7023 shutdown 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- 172.21.15.133:0/2380774628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f0e9f30 msgr2=0x55ef1f194560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f0e9f30 0x55ef1f194560 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f34300a54d0 tx=0x7f3430009040 comp rx=0 tx=0).stop 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- 172.21.15.133:0/2380774628 shutdown_connections 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.133:6836/236214118,v1:172.21.15.133:6837/236214118] conn(0x55ef1f13eb90 0x55ef1f13ef70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef1f134240 0x55ef1f13e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef1f133920 0x55ef1f194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f341e7fc640 0 client.7023 ms_handle_reset on v2:172.21.15.133:6836/236214118 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 --2- 172.21.15.133:0/2380774628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef1f0e9f30 0x55ef1f194560 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- 172.21.15.133:0/2380774628 >> 172.21.15.133:0/2380774628 conn(0x55ef1f0f02a0 msgr2=0x55ef1f0ec320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- 172.21.15.133:0/2380774628 shutdown_connections 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 1 -- 172.21.15.133:0/2380774628 wait complete. 2024-04-25T10:06:41.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f3448a4b000 20 client.7023 trim_cache size 0 max 16384 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 6876, 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 4.6311026269999997, 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 33, 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 66, 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 66, 2024-04-25T10:06:41.852 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 9.771302725 2024-04-25T10:06:41.853 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:41.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f34458de640 1 -- 172.21.15.133:0/2523465881 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f342c066f60 msgr2=0x7f342c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f34458de640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f342c066f60 0x7f342c069420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f344015f200 tx=0x7f3434009040 comp rx=0 tx=0).stop 2024-04-25T10:06:41.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f34458de640 1 -- 172.21.15.133:0/2523465881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f344006e8b0 msgr2=0x7f34400cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f34458de640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f344006e8b0 0x7f34400cee60 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f34300af040 tx=0x7f34300196e0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f34458de640 1 -- 172.21.15.133:0/2523465881 shutdown_connections 2024-04-25T10:06:41.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f34458de640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34400b1e50 0x7f3440146090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f34458de640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34400aebc0 0x7f3440145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f34458de640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f342c066f60 0x7f342c069420 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f34458de640 1 --2- 172.21.15.133:0/2523465881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f344006e8b0 0x7f34400cee60 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.838+0000 7f34458de640 1 -- 172.21.15.133:0/2523465881 >> 172.21.15.133:0/2523465881 conn(0x7f3440066f70 msgr2=0x7f344015df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.842+0000 7f34458de640 1 -- 172.21.15.133:0/2523465881 shutdown_connections 2024-04-25T10:06:41.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.842+0000 7f34458de640 1 -- 172.21.15.133:0/2523465881 wait complete. 2024-04-25T10:06:41.859 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:06:41.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.966+0000 7fcda29f6640 1 Processor -- start 2024-04-25T10:06:41.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.966+0000 7fcda29f6640 1 -- start start 2024-04-25T10:06:41.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcda29f6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcd9c15a9d0 0x7fcd9c0c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcda29f6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c0cc5a0 con 0x7fcd9c15a5f0 2024-04-25T10:06:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcda29f6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c0cc880 con 0x7fcd9c06eb70 2024-04-25T10:06:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcda29f6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c0ccb60 con 0x7fcd9c15a9d0 2024-04-25T10:06:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcd9bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcd9c15a5f0 0x7fcd9c06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57152/0 (socket says 172.21.15.133:57152) 2024-04-25T10:06:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcd9bfff640 1 -- 172.21.15.133:0/1741322338 learned_addr learned my addr 172.21.15.133:0/1741322338 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcda0f93640 1 --2- 172.21.15.133:0/1741322338 >> v2:172.21.15.133:3300/0 conn(0x7fcd9c15a9d0 0x7fcd9c0c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcd9b7fe640 1 -- 172.21.15.133:0/1741322338 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186916308 0 0) 0x7fcd9c0cc5a0 con 0x7fcd9c15a5f0 2024-04-25T10:06:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcd9b7fe640 1 -- 172.21.15.133:0/1741322338 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd88003530 con 0x7fcd9c15a5f0 2024-04-25T10:06:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcda0f93640 1 -- 172.21.15.133:0/1741322338 >> v1:172.21.15.162:6789/0 conn(0x7fcd9c06eb70 legacy=0x7fcd9c0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcda0f93640 1 -- 172.21.15.133:0/1741322338 >> v1:172.21.15.133:6789/0 conn(0x7fcd9c15a5f0 legacy=0x7fcd9c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcda0f93640 1 -- 172.21.15.133:0/1741322338 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c0cce40 con 0x7fcd9c15a9d0 2024-04-25T10:06:41.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.970+0000 7fcda0f93640 1 --2- 172.21.15.133:0/1741322338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd9c15a9d0 0x7fcd9c0c5260 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009d90 tx=0x7fcd9401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211a8ddca8457c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9b7fe640 1 -- 172.21.15.133:0/1741322338 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd940b2070 con 0x7fcd9c15a9d0 2024-04-25T10:06:41.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9b7fe640 1 -- 172.21.15.133:0/1741322338 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd940a5940 con 0x7fcd9c15a9d0 2024-04-25T10:06:41.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9b7fe640 1 -- 172.21.15.133:0/1741322338 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd940a5cf0 con 0x7fcd9c15a9d0 2024-04-25T10:06:41.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 -- 172.21.15.133:0/1741322338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd9c15a9d0 msgr2=0x7fcd9c0c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 --2- 172.21.15.133:0/1741322338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd9c15a9d0 0x7fcd9c0c5260 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009d90 tx=0x7fcd9401c970 comp rx=0 tx=0).stop 2024-04-25T10:06:41.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 -- 172.21.15.133:0/1741322338 shutdown_connections 2024-04-25T10:06:41.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 --2- 172.21.15.133:0/1741322338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd9c15a9d0 0x7fcd9c0c5260 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 -- 172.21.15.133:0/1741322338 >> 172.21.15.133:0/1741322338 conn(0x7fcd9c067000 msgr2=0x7fcd9c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 -- 172.21.15.133:0/1741322338 shutdown_connections 2024-04-25T10:06:41.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 -- 172.21.15.133:0/1741322338 wait complete. 2024-04-25T10:06:41.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 Processor -- start 2024-04-25T10:06:41.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 -- start start 2024-04-25T10:06:41.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd9c06eb70 0x7fcd9c15c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd9c15a5f0 0x7fcd9c164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9c15a9d0 0x7fcd9c0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0ceb60 con 0x7fcd9c06eb70 2024-04-25T10:06:41.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0ce9e0 con 0x7fcd9c15a5f0 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c168500 con 0x7fcd9c15a9d0 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd9c06eb70 0x7fcd9c15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd9c06eb70 0x7fcd9c15c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43948/0 (socket says 172.21.15.133:43948) 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9bfff640 1 -- 172.21.15.133:0/1490021501 learned_addr learned my addr 172.21.15.133:0/1490021501 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda0f93640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9c15a9d0 0x7fcd9c0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd93fff640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd9c15a5f0 0x7fcd9c164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9bfff640 1 -- 172.21.15.133:0/1490021501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9c15a9d0 msgr2=0x7fcd9c0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9bfff640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9c15a9d0 0x7fcd9c0b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9bfff640 1 -- 172.21.15.133:0/1490021501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd9c15a5f0 msgr2=0x7fcd9c164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9bfff640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd9c15a5f0 0x7fcd9c164fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9bfff640 1 -- 172.21.15.133:0/1490021501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c15d0c0 con 0x7fcd9c06eb70 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd93fff640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd9c15a5f0 0x7fcd9c164fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda0f93640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9c15a9d0 0x7fcd9c0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:41.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd9bfff640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd9c06eb70 0x7fcd9c15c890 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400c500 tx=0x7fcd8400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd997fa640 1 -- 172.21.15.133:0/1490021501 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84017070 con 0x7fcd9c06eb70 2024-04-25T10:06:41.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd997fa640 1 -- 172.21.15.133:0/1490021501 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd84005cf0 con 0x7fcd9c06eb70 2024-04-25T10:06:41.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 -- 172.21.15.133:0/1490021501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c15d3a0 con 0x7fcd9c06eb70 2024-04-25T10:06:41.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcd997fa640 1 -- 172.21.15.133:0/1490021501 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840072f0 con 0x7fcd9c06eb70 2024-04-25T10:06:41.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.974+0000 7fcda29f6640 1 -- 172.21.15.133:0/1490021501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c15d960 con 0x7fcd9c06eb70 2024-04-25T10:06:41.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.978+0000 7fcda29f6640 1 -- 172.21.15.133:0/1490021501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c15ce30 con 0x7fcd9c06eb70 2024-04-25T10:06:41.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.978+0000 7fcd997fa640 1 -- 172.21.15.133:0/1490021501 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd8401a070 con 0x7fcd9c06eb70 2024-04-25T10:06:41.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.978+0000 7fcd997fa640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd7c066f60 0x7fcd7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.978+0000 7fcd997fa640 1 -- 172.21.15.133:0/1490021501 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd84084510 con 0x7fcd9c06eb70 2024-04-25T10:06:41.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.978+0000 7fcd93fff640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd7c066f60 0x7fcd7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.978+0000 7fcd93fff640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd7c066f60 0x7fcd7c069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0cb150 tx=0x7fcd8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:41.982+0000 7fcd997fa640 1 -- 172.21.15.133:0/1490021501 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd84054d40 con 0x7fcd9c06eb70 2024-04-25T10:06:42.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 -- 172.21.15.133:0/1490021501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcd9c140ed0 con 0x7fcd9c06eb70 2024-04-25T10:06:42.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcd997fa640 1 -- 172.21.15.133:0/1490021501 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcd84058de0 con 0x7fcd9c06eb70 2024-04-25T10:06:42.166 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:06:42.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 -- 172.21.15.133:0/1490021501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd7c066f60 msgr2=0x7fcd7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd7c066f60 0x7fcd7c069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c0cb150 tx=0x7fcd8c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 -- 172.21.15.133:0/1490021501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd9c06eb70 msgr2=0x7fcd9c15c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd9c06eb70 0x7fcd9c15c890 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400c500 tx=0x7fcd8400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 -- 172.21.15.133:0/1490021501 shutdown_connections 2024-04-25T10:06:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcd9c15a9d0 0x7fcd9c0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd9c15a5f0 0x7fcd9c164fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd7c066f60 0x7fcd7c069420 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 --2- 172.21.15.133:0/1490021501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd9c06eb70 0x7fcd9c15c890 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 -- 172.21.15.133:0/1490021501 >> 172.21.15.133:0/1490021501 conn(0x7fcd9c067000 msgr2=0x7fcd9c1597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 -- 172.21.15.133:0/1490021501 shutdown_connections 2024-04-25T10:06:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:42.162+0000 7fcda29f6640 1 -- 172.21.15.133:0/1490021501 wait complete. 2024-04-25T10:06:42.183 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:06:42.183 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:06:42.183 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:06:42.183 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:06:42.183 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:42.183 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:06:42.183 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:06:42.184 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:06:42.184 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:42.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:42.184 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42229sec preferred_lft 42229sec 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:06:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:06:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:42.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:42.193 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:06:42.241 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:06:42.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:42.243 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:06:42.291 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:06:42.291 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:06:42.291 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:06:42.291 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:06:42.291 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:06:42.291 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:06:42.291 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:06:42.291 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:06:42.291 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:06:42.291 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:06:42.292 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:06:42.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:42.292 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:06:42.292 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:06:42.292 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:06:42.292 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:06:42.292 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:06:42.292 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:06:42.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:42.362 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:06:42.408 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:06:42.408 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:06:42.409 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:42.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:42.409 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:06:42.430 INFO:tasks.ceph.mds.c.smithi133.stderr:2024-04-25T10:06:42.426+0000 7fdd93aaa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:42.441 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:06:42.441 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:42.441 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:42.488 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:42.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:42.489 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:42.537 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:06:42.594 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:06:42.642 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:06:42.643 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:06:42.643 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:06:42.643 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:06:42.643 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:06:42.643 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:06:42.643 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:06:42.643 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:06:42.643 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:06:42.643 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:06:42.643 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:06:42.643 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:06:42.644 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:06:42.645 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:06:42.645 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:06:42.645 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:06:42.645 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:42.645 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:42.645 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:42.645 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:42.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:42.645 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:06:42.688 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:06:42.689 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:06:42.734 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:06:42.748 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:06:42.797 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:06:42.798 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:06:42.799 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:06:42.800 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:06:42.800 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:42.800 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:42.800 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:42.800 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:42.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:42.801 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:06:42.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:06:42.806+0000 7f563b7b8580 -1 init, newargv = 0x564f34188d70 newargc=15 2024-04-25T10:06:42.810 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[56222]: starting ceph client 2024-04-25T10:06:42.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[56222]: starting fuse 2024-04-25T10:06:43.914 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:06:43.931 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:06:43.979 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:06:43.979 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:06:43.979 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:06:43.980 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:06:43.981 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:06:43.981 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:06:43.981 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:06:43.981 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:06:43.981 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:06:43.981 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:06:43.981 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:06:43.982 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:06:43.982 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:06:43.982 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:06:43.982 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:06:43.982 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:43.982 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:43.983 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:43.983 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:43.983 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:43.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:43.984 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:06:44.034 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:06:44.035 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:06:44.036 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:06:44.157 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.56222.asok 2024-04-25T10:06:44.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:44.169 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56222.asok 2024-04-25T10:06:44.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:44.170 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56222.asok status 2024-04-25T10:06:44.327 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:06:44.327 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:06:44.327 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:06:44.327 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:06:44.327 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:06:44.327 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "56222", 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 7038, 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 7038 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:06:44.328 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3895496381 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3895496381 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.7038 192.168.0.1:0/3895496381", 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/3895496381", 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 33, 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 66, 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:06:44.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:44.341 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:06:44.341 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:44.347 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:06:44.347 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:44.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:44.347 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:44.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:44.400 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T10:06:44.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:44.453 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T10:06:44.508 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-04-25T10:06:44.508 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:06:44.508 DEBUG:teuthology.orchestra.run.smithi133:> import stat 2024-04-25T10:06:44.508 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:06:44.508 DEBUG:teuthology.orchestra.run.smithi133:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T10:06:44.508 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:06:44.615 INFO:teuthology.orchestra.run.smithi133.stdout:1099511627776 2024-04-25T10:06:44.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:44.625 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:06:44.625 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:44.673 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:44.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.838+0000 7f1ad14c7640 1 Processor -- start 2024-04-25T10:06:44.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.838+0000 7f1ad14c7640 1 -- start start 2024-04-25T10:06:44.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1ad14c7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1acc159dc0 0x7f1acc0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1ad14c7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1acc15a1a0 0x7f1acc15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1ad14c7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1acc15e460 con 0x7f1acc15e050 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1ad14c7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1acc0ce880 con 0x7f1acc15a1a0 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1ad14c7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1acc06e360 con 0x7f1acc159dc0 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1aca7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1acc15a1a0 0x7f1acc15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1acb7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1acc15e050 0x7f1acc0ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57160/0 (socket says 172.21.15.133:57160) 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1acb7fe640 1 -- 172.21.15.133:0/380936688 learned_addr learned my addr 172.21.15.133:0/380936688 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1acaffd640 1 --2- 172.21.15.133:0/380936688 >> v2:172.21.15.162:3300/0 conn(0x7f1acc159dc0 0x7f1acc0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1aca7fc640 1 -- 172.21.15.133:0/380936688 >> v2:172.21.15.162:3300/0 conn(0x7f1acc159dc0 msgr2=0x7f1acc0cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1aca7fc640 1 --2- 172.21.15.133:0/380936688 >> v2:172.21.15.162:3300/0 conn(0x7f1acc159dc0 0x7f1acc0cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1aca7fc640 1 -- 172.21.15.133:0/380936688 >> v1:172.21.15.133:6789/0 conn(0x7f1acc15e050 legacy=0x7f1acc0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1ac9ffb640 1 -- 172.21.15.133:0/380936688 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692884579 0 0) 0x7f1ac40014a0 con 0x7f1acc15e050 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1aca7fc640 1 -- 172.21.15.133:0/380936688 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc0c49e0 con 0x7f1acc15a1a0 2024-04-25T10:06:44.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1acaffd640 1 --2- 172.21.15.133:0/380936688 >> v2:172.21.15.162:3300/0 conn(0x7f1acc159dc0 0x7f1acc0cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:44.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1aca7fc640 1 --2- 172.21.15.133:0/380936688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1acc15a1a0 0x7f1acc15db10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1acc06f040 tx=0x7f1ac001d850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe7d7d77c7a6c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1ac9ffb640 1 -- 172.21.15.133:0/380936688 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac000b040 con 0x7f1acc15a1a0 2024-04-25T10:06:44.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1ac9ffb640 1 -- 172.21.15.133:0/380936688 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac00a76b0 con 0x7f1acc15a1a0 2024-04-25T10:06:44.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.842+0000 7f1ac9ffb640 1 -- 172.21.15.133:0/380936688 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a7a80 con 0x7f1acc15a1a0 2024-04-25T10:06:44.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 -- 172.21.15.133:0/380936688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1acc15a1a0 msgr2=0x7f1acc15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 --2- 172.21.15.133:0/380936688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1acc15a1a0 0x7f1acc15db10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1acc06f040 tx=0x7f1ac001d850 comp rx=0 tx=0).stop 2024-04-25T10:06:44.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 -- 172.21.15.133:0/380936688 shutdown_connections 2024-04-25T10:06:44.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 --2- 172.21.15.133:0/380936688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1acc15a1a0 0x7f1acc15db10 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 --2- 172.21.15.133:0/380936688 >> v2:172.21.15.162:3300/0 conn(0x7f1acc159dc0 0x7f1acc0cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 -- 172.21.15.133:0/380936688 >> 172.21.15.133:0/380936688 conn(0x7f1acc067000 msgr2=0x7f1acc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 -- 172.21.15.133:0/380936688 shutdown_connections 2024-04-25T10:06:44.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 -- 172.21.15.133:0/380936688 wait complete. 2024-04-25T10:06:44.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 Processor -- start 2024-04-25T10:06:44.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 -- start start 2024-04-25T10:06:44.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1acc159dc0 0x7f1acc15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1acc15e050 0x7f1acc1f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1acc153280 0x7f1acc14e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0c4e40 con 0x7f1acc15e050 2024-04-25T10:06:44.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0c4cc0 con 0x7f1acc159dc0 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0cf300 con 0x7f1acc153280 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1aca7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1acc15e050 0x7f1acc1f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1aca7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1acc15e050 0x7f1acc1f6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43956/0 (socket says 172.21.15.133:43956) 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1aca7fc640 1 -- 172.21.15.133:0/2874443196 learned_addr learned my addr 172.21.15.133:0/2874443196 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1acaffd640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1acc159dc0 0x7f1acc15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1acb7fe640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1acc153280 0x7f1acc14e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1aca7fc640 1 -- 172.21.15.133:0/2874443196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1acc153280 msgr2=0x7f1acc14e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1aca7fc640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1acc153280 0x7f1acc14e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1aca7fc640 1 -- 172.21.15.133:0/2874443196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1acc159dc0 msgr2=0x7f1acc15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1aca7fc640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1acc159dc0 0x7f1acc15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1aca7fc640 1 -- 172.21.15.133:0/2874443196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc15c610 con 0x7f1acc15e050 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1acaffd640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1acc159dc0 0x7f1acc15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1aca7fc640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1acc15e050 0x7f1acc1f6740 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1ac001a460 tx=0x7f1ac001a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ab37fe640 1 -- 172.21.15.133:0/2874443196 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a7c70 con 0x7f1acc15e050 2024-04-25T10:06:44.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ab37fe640 1 -- 172.21.15.133:0/2874443196 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac00a7e10 con 0x7f1acc15e050 2024-04-25T10:06:44.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 -- 172.21.15.133:0/2874443196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1acc14eae0 con 0x7f1acc15e050 2024-04-25T10:06:44.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ab37fe640 1 -- 172.21.15.133:0/2874443196 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00b8680 con 0x7f1acc15e050 2024-04-25T10:06:44.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.846+0000 7f1ad14c7640 1 -- 172.21.15.133:0/2874443196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc1f6c80 con 0x7f1acc15e050 2024-04-25T10:06:44.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.850+0000 7f1ad14c7640 1 -- 172.21.15.133:0/2874443196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc068ef0 con 0x7f1acc15e050 2024-04-25T10:06:44.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.850+0000 7f1ab37fe640 1 -- 172.21.15.133:0/2874443196 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ac00c1020 con 0x7f1acc15e050 2024-04-25T10:06:44.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.850+0000 7f1ab37fe640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab8066f90 0x7f1ab8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.850+0000 7f1ab37fe640 1 -- 172.21.15.133:0/2874443196 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ac011f4d0 con 0x7f1acc15e050 2024-04-25T10:06:44.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.850+0000 7f1acaffd640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab8066f90 0x7f1ab8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.850+0000 7f1acaffd640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab8066f90 0x7f1ab8069450 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1abc008930 tx=0x7f1abc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:44.854+0000 7f1ab37fe640 1 -- 172.21.15.133:0/2874443196 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ac00c8020 con 0x7f1acc15e050 2024-04-25T10:06:45.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.014+0000 7f1ad14c7640 1 -- 172.21.15.133:0/2874443196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1acc0cdaf0 con 0x7f1acc15e050 2024-04-25T10:06:45.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ab37fe640 1 -- 172.21.15.133:0/2874443196 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16908 (secure 0 0 0) 0x7f1ac0017440 con 0x7f1acc15e050 2024-04-25T10:06:45.021 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:45.021 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":66,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:06:38.208663+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:34.764755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:35.504408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:45.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 -- 172.21.15.133:0/2874443196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab8066f90 msgr2=0x7f1ab8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab8066f90 0x7f1ab8069450 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1abc008930 tx=0x7f1abc008040 comp rx=0 tx=0).stop 2024-04-25T10:06:45.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 -- 172.21.15.133:0/2874443196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1acc15e050 msgr2=0x7f1acc1f6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1acc15e050 0x7f1acc1f6740 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1ac001a460 tx=0x7f1ac001a5f0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 -- 172.21.15.133:0/2874443196 shutdown_connections 2024-04-25T10:06:45.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1acc153280 0x7f1acc14e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab8066f90 0x7f1ab8069450 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1acc15e050 0x7f1acc1f6740 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 --2- 172.21.15.133:0/2874443196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1acc159dc0 0x7f1acc15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 -- 172.21.15.133:0/2874443196 >> 172.21.15.133:0/2874443196 conn(0x7f1acc067000 msgr2=0x7f1acc15a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 -- 172.21.15.133:0/2874443196 shutdown_connections 2024-04-25T10:06:45.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.018+0000 7f1ad14c7640 1 -- 172.21.15.133:0/2874443196 wait complete. 2024-04-25T10:06:45.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:34.764755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:35.504408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:45.041 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:45.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45cde24640 1 Processor -- start 2024-04-25T10:06:45.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45cde24640 1 -- start start 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45cde24640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f45c8143ea0 0x7f45c80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45cde24640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c80b2150 0x7f45c806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45cde24640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c815ac50 con 0x7f45c815b9b0 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45cde24640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c80cd180 con 0x7f45c8143ea0 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45cde24640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c815aed0 con 0x7f45c80b2150 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f45c815b9b0 0x7f45c815e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57168/0 (socket says 172.21.15.133:57168) 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c7fff640 1 -- 172.21.15.133:0/569681449 learned_addr learned my addr 172.21.15.133:0/569681449 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c77fe640 1 --2- 172.21.15.133:0/569681449 >> v2:172.21.15.162:3300/0 conn(0x7f45c8143ea0 0x7f45c80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c6ffd640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c80b2150 0x7f45c806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174713714 0 0) 0x7f45c00014a0 con 0x7f45c815b9b0 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45b0003530 con 0x7f45c815b9b0 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2251986055 0 0) 0x7f45b0003530 con 0x7f45c815b9b0 2024-04-25T10:06:45.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c80b2150 msgr2=0x7f45c806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c80b2150 0x7f45c806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 >> v2:172.21.15.162:3300/0 conn(0x7f45c8143ea0 msgr2=0x7f45c80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 --2- 172.21.15.133:0/569681449 >> v2:172.21.15.162:3300/0 conn(0x7f45c8143ea0 0x7f45c80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c815b1b0 con 0x7f45c815b9b0 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f45c0001bd0 con 0x7f45c815b9b0 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c77fe640 1 --2- 172.21.15.133:0/569681449 >> v2:172.21.15.162:3300/0 conn(0x7f45c8143ea0 0x7f45c80cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 >> v1:172.21.15.133:6789/0 conn(0x7f45c815b9b0 legacy=0x7f45c815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45b00039c0 0x7f45b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c80b2150 0x7f45b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b0004d50 0x7f45b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c80cd180 con 0x7f45b00039c0 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c815aed0 con 0x7f45b0004d50 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b000b9d0 con 0x7f45c80b2150 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c77fe640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45b00039c0 0x7f45b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c6ffd640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c80b2150 0x7f45b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c7fff640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b0004d50 0x7f45b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c77fe640 1 -- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c80b2150 msgr2=0x7f45b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c77fe640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c80b2150 0x7f45b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c77fe640 1 -- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b0004d50 msgr2=0x7f45b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c77fe640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b0004d50 0x7f45b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c77fe640 1 -- 172.21.15.133:0/569681449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b0003530 con 0x7f45b00039c0 2024-04-25T10:06:45.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.162+0000 7f45c7fff640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b0004d50 0x7f45b00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:45.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45c77fe640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45b00039c0 0x7f45b0003de0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009cb0 tx=0x7f45b801cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dd24793998cde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b80b2070 con 0x7f45b00039c0 2024-04-25T10:06:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b80a5840 con 0x7f45b00039c0 2024-04-25T10:06:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45c67fc640 1 -- 172.21.15.133:0/569681449 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8009880 con 0x7f45b00039c0 2024-04-25T10:06:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45cde24640 1 -- 172.21.15.133:0/569681449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45b00039c0 msgr2=0x7f45b0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45cde24640 1 --2- 172.21.15.133:0/569681449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45b00039c0 0x7f45b0003de0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009cb0 tx=0x7f45b801cc90 comp rx=0 tx=0).stop 2024-04-25T10:06:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45c7fff640 1 -- 172.21.15.133:0/569681449 reap_dead start 2024-04-25T10:06:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45cde24640 1 -- 172.21.15.133:0/569681449 shutdown_connections 2024-04-25T10:06:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45cde24640 1 -- 172.21.15.133:0/569681449 >> 172.21.15.133:0/569681449 conn(0x7f45c8066f70 msgr2=0x7f45c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45cde24640 1 -- 172.21.15.133:0/569681449 shutdown_connections 2024-04-25T10:06:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45cde24640 1 -- 172.21.15.133:0/569681449 wait complete. 2024-04-25T10:06:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45cde24640 1 Processor -- start 2024-04-25T10:06:45.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.166+0000 7f45cde24640 1 -- start start 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45cde24640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b00039c0 0x7f45c81f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45cde24640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c815d3e0 0x7f45c815d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45cde24640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c806d360 0x7f45c806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45cde24640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c806e740 con 0x7f45c815d3e0 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45cde24640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c806e5c0 con 0x7f45b00039c0 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45cde24640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c806e8c0 con 0x7f45c806d360 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c815d3e0 0x7f45c815d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c815d3e0 0x7f45c815d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43978/0 (socket says 172.21.15.133:43978) 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c6ffd640 1 -- 172.21.15.133:0/663503185 learned_addr learned my addr 172.21.15.133:0/663503185 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c7fff640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c806d360 0x7f45c806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c77fe640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b00039c0 0x7f45c81f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c6ffd640 1 -- 172.21.15.133:0/663503185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c806d360 msgr2=0x7f45c806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c6ffd640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c806d360 0x7f45c806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c6ffd640 1 -- 172.21.15.133:0/663503185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b00039c0 msgr2=0x7f45c81f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c6ffd640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b00039c0 0x7f45c81f6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c6ffd640 1 -- 172.21.15.133:0/663503185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c8132a30 con 0x7f45c815d3e0 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c7fff640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c806d360 0x7f45c806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c6ffd640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c815d3e0 0x7f45c815d840 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f45bc011640 tx=0x7f45bc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45affff640 1 -- 172.21.15.133:0/663503185 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00f920 con 0x7f45c815d3e0 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45affff640 1 -- 172.21.15.133:0/663503185 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc00fac0 con 0x7f45c815d3e0 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45affff640 1 -- 172.21.15.133:0/663503185 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00fe70 con 0x7f45c815d3e0 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45cde24640 1 -- 172.21.15.133:0/663503185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c81f6da0 con 0x7f45c815d3e0 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45cde24640 1 -- 172.21.15.133:0/663503185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c80c9a00 con 0x7f45c815d3e0 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45cde24640 1 -- 172.21.15.133:0/663503185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c8068e80 con 0x7f45c815d3e0 2024-04-25T10:06:45.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45affff640 1 -- 172.21.15.133:0/663503185 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45bc015460 con 0x7f45c815d3e0 2024-04-25T10:06:45.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45affff640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45b4066f60 0x7f45b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c77fe640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45b4066f60 0x7f45b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45affff640 1 -- 172.21.15.133:0/663503185 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f45bc085cb0 con 0x7f45c815d3e0 2024-04-25T10:06:45.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.170+0000 7f45c77fe640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45b4066f60 0x7f45b4069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f45b80af000 tx=0x7f45b80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.174+0000 7f45affff640 1 -- 172.21.15.133:0/663503185 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45bc019030 con 0x7f45c815d3e0 2024-04-25T10:06:45.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.338+0000 7f45cde24640 1 -- 172.21.15.133:0/663503185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45c80cc520 con 0x7f45c815d3e0 2024-04-25T10:06:45.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.338+0000 7f45affff640 1 -- 172.21.15.133:0/663503185 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45bc056a00 con 0x7f45c815d3e0 2024-04-25T10:06:45.342 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:06:45.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 -- 172.21.15.133:0/663503185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45b4066f60 msgr2=0x7f45b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45b4066f60 0x7f45b4069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f45b80af000 tx=0x7f45b80046d0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 -- 172.21.15.133:0/663503185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c815d3e0 msgr2=0x7f45c815d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c815d3e0 0x7f45c815d840 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f45bc011640 tx=0x7f45bc011b00 comp rx=0 tx=0).stop 2024-04-25T10:06:45.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 -- 172.21.15.133:0/663503185 shutdown_connections 2024-04-25T10:06:45.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45c806d360 0x7f45c806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45b4066f60 0x7f45b4069420 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c815d3e0 0x7f45c815d840 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 --2- 172.21.15.133:0/663503185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45b00039c0 0x7f45c81f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 -- 172.21.15.133:0/663503185 >> 172.21.15.133:0/663503185 conn(0x7f45c8066f70 msgr2=0x7f45c80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 -- 172.21.15.133:0/663503185 shutdown_connections 2024-04-25T10:06:45.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.342+0000 7f45cde24640 1 -- 172.21.15.133:0/663503185 wait complete. 2024-04-25T10:06:45.362 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:45.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd7e9c2640 1 Processor -- start 2024-04-25T10:06:45.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd7e9c2640 1 -- start start 2024-04-25T10:06:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd7e9c2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7806e8b0 0x7fdd7815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd7e9c2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdd780b1e50 0x7fdd78146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd7e9c2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd780b1090 con 0x7fdd780aebc0 2024-04-25T10:06:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd7e9c2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd78147490 con 0x7fdd780b1e50 2024-04-25T10:06:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd7e9c2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780b19c0 con 0x7fdd7806e8b0 2024-04-25T10:06:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd777fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdd780b1e50 0x7fdd78146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd777fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdd780b1e50 0x7fdd78146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43984/0 (socket says 172.21.15.133:43984) 2024-04-25T10:06:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd777fe640 1 -- 172.21.15.133:0/465697646 learned_addr learned my addr 172.21.15.133:0/465697646 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd77fff640 1 --2- 172.21.15.133:0/465697646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7806e8b0 0x7fdd7815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd777fe640 1 -- 172.21.15.133:0/465697646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7806e8b0 msgr2=0x7fdd7815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd777fe640 1 --2- 172.21.15.133:0/465697646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7806e8b0 0x7fdd7815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd777fe640 1 -- 172.21.15.133:0/465697646 >> v1:172.21.15.162:6789/0 conn(0x7fdd780aebc0 legacy=0x7fdd7806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd777fe640 1 -- 172.21.15.133:0/465697646 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd780b1370 con 0x7fdd780b1e50 2024-04-25T10:06:45.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.486+0000 7fdd77fff640 1 --2- 172.21.15.133:0/465697646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7806e8b0 0x7fdd7815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:45.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd777fe640 1 --2- 172.21.15.133:0/465697646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd780b1e50 0x7fdd78146ae0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fdd680098a0 tx=0x7fdd6801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f907564c3d4be931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd76ffd640 1 -- 172.21.15.133:0/465697646 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680b0070 con 0x7fdd780b1e50 2024-04-25T10:06:45.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd76ffd640 1 -- 172.21.15.133:0/465697646 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd680ab600 con 0x7fdd780b1e50 2024-04-25T10:06:45.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd76ffd640 1 -- 172.21.15.133:0/465697646 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680ab9d0 con 0x7fdd780b1e50 2024-04-25T10:06:45.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/465697646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd780b1e50 msgr2=0x7fdd78146ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd7e9c2640 1 --2- 172.21.15.133:0/465697646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd780b1e50 0x7fdd78146ae0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fdd680098a0 tx=0x7fdd6801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/465697646 shutdown_connections 2024-04-25T10:06:45.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd7e9c2640 1 --2- 172.21.15.133:0/465697646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd780b1e50 0x7fdd78146ae0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd7e9c2640 1 --2- 172.21.15.133:0/465697646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd7806e8b0 0x7fdd7815eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/465697646 >> 172.21.15.133:0/465697646 conn(0x7fdd78066f70 msgr2=0x7fdd7806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/465697646 shutdown_connections 2024-04-25T10:06:45.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/465697646 wait complete. 2024-04-25T10:06:45.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd7e9c2640 1 Processor -- start 2024-04-25T10:06:45.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.490+0000 7fdd7e9c2640 1 -- start start 2024-04-25T10:06:45.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7e9c2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd7806e8b0 0x7fdd7813e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7e9c2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd780aebc0 0x7fdd7815d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7e9c2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd780b1e50 0x7fdd780cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7e9c2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780af3e0 con 0x7fdd780b1e50 2024-04-25T10:06:45.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7e9c2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78147020 con 0x7fdd7806e8b0 2024-04-25T10:06:45.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7e9c2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7815d680 con 0x7fdd780aebc0 2024-04-25T10:06:45.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7cf5f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd780b1e50 0x7fdd780cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7cf5f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd780b1e50 0x7fdd780cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43996/0 (socket says 172.21.15.133:43996) 2024-04-25T10:06:45.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7cf5f640 1 -- 172.21.15.133:0/730333179 learned_addr learned my addr 172.21.15.133:0/730333179 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:45.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd77fff640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd7806e8b0 0x7fdd7813e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd777fe640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd780aebc0 0x7fdd7815d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7cf5f640 1 -- 172.21.15.133:0/730333179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd780aebc0 msgr2=0x7fdd7815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7cf5f640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd780aebc0 0x7fdd7815d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7cf5f640 1 -- 172.21.15.133:0/730333179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd7806e8b0 msgr2=0x7fdd7813e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7cf5f640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd7806e8b0 0x7fdd7813e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7cf5f640 1 -- 172.21.15.133:0/730333179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7813f200 con 0x7fdd780b1e50 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd777fe640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd780aebc0 0x7fdd7815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7cf5f640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd780b1e50 0x7fdd780cee60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c760 tx=0x7fdd7000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd74ff9640 1 -- 172.21.15.133:0/730333179 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70003b80 con 0x7fdd780b1e50 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd74ff9640 1 -- 172.21.15.133:0/730333179 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70003d20 con 0x7fdd780b1e50 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/730333179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7813d3e0 con 0x7fdd780b1e50 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd74ff9640 1 -- 172.21.15.133:0/730333179 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000aa30 con 0x7fdd780b1e50 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/730333179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7813ef20 con 0x7fdd780b1e50 2024-04-25T10:06:45.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/730333179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd78068e80 con 0x7fdd780b1e50 2024-04-25T10:06:45.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd74ff9640 1 -- 172.21.15.133:0/730333179 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd7000abd0 con 0x7fdd780b1e50 2024-04-25T10:06:45.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd74ff9640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd64066f90 0x7fdd64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd74ff9640 1 -- 172.21.15.133:0/730333179 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd70084fd0 con 0x7fdd780b1e50 2024-04-25T10:06:45.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.494+0000 7fdd77fff640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd64066f90 0x7fdd64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.498+0000 7fdd77fff640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd64066f90 0x7fdd64069450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00b930 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.498+0000 7fdd74ff9640 1 -- 172.21.15.133:0/730333179 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd70055ee0 con 0x7fdd780b1e50 2024-04-25T10:06:45.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.658+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/730333179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd78161a30 con 0x7fdd780b1e50 2024-04-25T10:06:45.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.658+0000 7fdd74ff9640 1 -- 172.21.15.133:0/730333179 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd700557e0 con 0x7fdd780b1e50 2024-04-25T10:06:45.663 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:06:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/730333179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd64066f90 msgr2=0x7fdd64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd64066f90 0x7fdd64069450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00b930 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/730333179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd780b1e50 msgr2=0x7fdd780cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd780b1e50 0x7fdd780cee60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c760 tx=0x7fdd7000cc20 comp rx=0 tx=0).stop 2024-04-25T10:06:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/730333179 shutdown_connections 2024-04-25T10:06:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd64066f90 0x7fdd64069450 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd780b1e50 0x7fdd780cee60 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd780aebc0 0x7fdd7815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 --2- 172.21.15.133:0/730333179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd7806e8b0 0x7fdd7813e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/730333179 >> 172.21.15.133:0/730333179 conn(0x7fdd78066f70 msgr2=0x7fdd78073870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/730333179 shutdown_connections 2024-04-25T10:06:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.662+0000 7fdd7e9c2640 1 -- 172.21.15.133:0/730333179 wait complete. 2024-04-25T10:06:45.682 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd8c10640 1 Processor -- start 2024-04-25T10:06:45.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd8c10640 1 -- start start 2024-04-25T10:06:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd8c10640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7effd406e8b0 0x7effd415eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd8c10640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7effd40b1e50 0x7effd4146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd8c10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7effd40aebc0 0x7effd40af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd8c10640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd4147490 con 0x7effd406e8b0 2024-04-25T10:06:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd8c10640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd40b19c0 con 0x7effd40b1e50 2024-04-25T10:06:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd8c10640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd40b1090 con 0x7effd40aebc0 2024-04-25T10:06:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7effd406e8b0 0x7effd415eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7effd406e8b0 0x7effd415eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43998/0 (socket says 172.21.15.133:43998) 2024-04-25T10:06:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd259c640 1 -- 172.21.15.133:0/506957589 learned_addr learned my addr 172.21.15.133:0/506957589 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd2d9d640 1 --2- 172.21.15.133:0/506957589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7effd40aebc0 0x7effd40af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd1d9b640 1 --2- 172.21.15.133:0/506957589 >> v2:172.21.15.162:3300/0 conn(0x7effd40b1e50 0x7effd4146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd259c640 1 -- 172.21.15.133:0/506957589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7effd40aebc0 msgr2=0x7effd40af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd259c640 1 --2- 172.21.15.133:0/506957589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7effd40aebc0 0x7effd40af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd259c640 1 -- 172.21.15.133:0/506957589 >> v2:172.21.15.162:3300/0 conn(0x7effd40b1e50 msgr2=0x7effd4146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd259c640 1 --2- 172.21.15.133:0/506957589 >> v2:172.21.15.162:3300/0 conn(0x7effd40b1e50 0x7effd4146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd259c640 1 -- 172.21.15.133:0/506957589 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd40b1370 con 0x7effd406e8b0 2024-04-25T10:06:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd1d9b640 1 --2- 172.21.15.133:0/506957589 >> v2:172.21.15.162:3300/0 conn(0x7effd40b1e50 0x7effd4146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.806+0000 7effd259c640 1 --2- 172.21.15.133:0/506957589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd406e8b0 0x7effd415eb40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7effd40b4e30 tx=0x7effc401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5ed72285834470 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd159a640 1 -- 172.21.15.133:0/506957589 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40a55e0 con 0x7effd406e8b0 2024-04-25T10:06:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd159a640 1 -- 172.21.15.133:0/506957589 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc40a5780 con 0x7effd406e8b0 2024-04-25T10:06:45.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd159a640 1 -- 172.21.15.133:0/506957589 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40a5b50 con 0x7effd406e8b0 2024-04-25T10:06:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 -- 172.21.15.133:0/506957589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd406e8b0 msgr2=0x7effd415eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 --2- 172.21.15.133:0/506957589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd406e8b0 0x7effd415eb40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7effd40b4e30 tx=0x7effc401c960 comp rx=0 tx=0).stop 2024-04-25T10:06:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 -- 172.21.15.133:0/506957589 shutdown_connections 2024-04-25T10:06:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 --2- 172.21.15.133:0/506957589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7effd40aebc0 0x7effd40af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 --2- 172.21.15.133:0/506957589 >> v2:172.21.15.162:3300/0 conn(0x7effd40b1e50 0x7effd4146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 --2- 172.21.15.133:0/506957589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd406e8b0 0x7effd415eb40 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 -- 172.21.15.133:0/506957589 >> 172.21.15.133:0/506957589 conn(0x7effd4066f70 msgr2=0x7effd406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 -- 172.21.15.133:0/506957589 shutdown_connections 2024-04-25T10:06:45.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 -- 172.21.15.133:0/506957589 wait complete. 2024-04-25T10:06:45.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 Processor -- start 2024-04-25T10:06:45.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 -- start start 2024-04-25T10:06:45.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd406e8b0 0x7effd4162820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40aebc0 0x7effd4141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7effd40b1e50 0x7effd415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd406dab0 con 0x7effd406e8b0 2024-04-25T10:06:45.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd4147230 con 0x7effd40aebc0 2024-04-25T10:06:45.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd406dc30 con 0x7effd40b1e50 2024-04-25T10:06:45.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd406e8b0 0x7effd4162820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd406e8b0 0x7effd4162820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44012/0 (socket says 172.21.15.133:44012) 2024-04-25T10:06:45.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd259c640 1 -- 172.21.15.133:0/397618087 learned_addr learned my addr 172.21.15.133:0/397618087 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:45.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd1d9b640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40aebc0 0x7effd4141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd2d9d640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7effd40b1e50 0x7effd415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd259c640 1 -- 172.21.15.133:0/397618087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7effd40b1e50 msgr2=0x7effd415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd259c640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7effd40b1e50 0x7effd415cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd259c640 1 -- 172.21.15.133:0/397618087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40aebc0 msgr2=0x7effd4141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd259c640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40aebc0 0x7effd4141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd259c640 1 -- 172.21.15.133:0/397618087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd40ad070 con 0x7effd406e8b0 2024-04-25T10:06:45.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd1d9b640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40aebc0 0x7effd4141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:45.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd259c640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd406e8b0 0x7effd4162820 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7effc40af040 tx=0x7effc4019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effc2ffd640 1 -- 172.21.15.133:0/397618087 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40bc020 con 0x7effd406e8b0 2024-04-25T10:06:45.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effc2ffd640 1 -- 172.21.15.133:0/397618087 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc40b8670 con 0x7effd406e8b0 2024-04-25T10:06:45.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effd8c10640 1 -- 172.21.15.133:0/397618087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd4140ee0 con 0x7effd406e8b0 2024-04-25T10:06:45.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.810+0000 7effc2ffd640 1 -- 172.21.15.133:0/397618087 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4009510 con 0x7effd406e8b0 2024-04-25T10:06:45.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.814+0000 7effd8c10640 1 -- 172.21.15.133:0/397618087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd41414a0 con 0x7effd406e8b0 2024-04-25T10:06:45.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.814+0000 7effd8c10640 1 -- 172.21.15.133:0/397618087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd4068e80 con 0x7effd406e8b0 2024-04-25T10:06:45.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.814+0000 7effc2ffd640 1 -- 172.21.15.133:0/397618087 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7effc40bf020 con 0x7effd406e8b0 2024-04-25T10:06:45.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.814+0000 7effc2ffd640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effb4066f60 0x7effb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.814+0000 7effc2ffd640 1 -- 172.21.15.133:0/397618087 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7effc4120250 con 0x7effd406e8b0 2024-04-25T10:06:45.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.814+0000 7effd1d9b640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effb4066f60 0x7effb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.814+0000 7effd1d9b640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effb4066f60 0x7effb4069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7effd40adf10 tx=0x7effc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.818+0000 7effc2ffd640 1 -- 172.21.15.133:0/397618087 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7effc40c6030 con 0x7effd406e8b0 2024-04-25T10:06:45.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.974+0000 7effd8c10640 1 -- 172.21.15.133:0/397618087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7effd40ad7f0 con 0x7effd406e8b0 2024-04-25T10:06:45.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.974+0000 7effc2ffd640 1 -- 172.21.15.133:0/397618087 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7effc40165e0 con 0x7effd406e8b0 2024-04-25T10:06:45.979 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:06:45.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 -- 172.21.15.133:0/397618087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effb4066f60 msgr2=0x7effb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effb4066f60 0x7effb4069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7effd40adf10 tx=0x7effc8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 -- 172.21.15.133:0/397618087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd406e8b0 msgr2=0x7effd4162820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd406e8b0 0x7effd4162820 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7effc40af040 tx=0x7effc4019700 comp rx=0 tx=0).stop 2024-04-25T10:06:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 -- 172.21.15.133:0/397618087 shutdown_connections 2024-04-25T10:06:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7effd40b1e50 0x7effd415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7effd40aebc0 0x7effd4141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effb4066f60 0x7effb4069420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 --2- 172.21.15.133:0/397618087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd406e8b0 0x7effd4162820 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 -- 172.21.15.133:0/397618087 >> 172.21.15.133:0/397618087 conn(0x7effd4066f70 msgr2=0x7effd40735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 -- 172.21.15.133:0/397618087 shutdown_connections 2024-04-25T10:06:45.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:45.978+0000 7effd8c10640 1 -- 172.21.15.133:0/397618087 wait complete. 2024-04-25T10:06:45.998 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:46.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67d57ec640 1 Processor -- start 2024-04-25T10:06:46.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67d57ec640 1 -- start start 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67d57ec640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f67d0143ea0 0x7f67d00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67d57ec640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67d00b2150 0x7f67d006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67d57ec640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d015ac50 con 0x7f67d015b9b0 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67d57ec640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d00cd180 con 0x7f67d00b2150 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67d57ec640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d015aed0 con 0x7f67d0143ea0 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67ce7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67d00b2150 0x7f67d006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67ce7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67d00b2150 0x7f67d006e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44026/0 (socket says 172.21.15.133:44026) 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67ce7fc640 1 -- 172.21.15.133:0/3059089291 learned_addr learned my addr 172.21.15.133:0/3059089291 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67cf7fe640 1 --1- 172.21.15.133:0/3059089291 >> v1:172.21.15.133:6789/0 conn(0x7f67d015b9b0 0x7f67d015e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57184/0 (socket says 172.21.15.133:57184) 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67ceffd640 1 --2- 172.21.15.133:0/3059089291 >> v2:172.21.15.162:3300/0 conn(0x7f67d0143ea0 0x7f67d00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67ce7fc640 1 -- 172.21.15.133:0/3059089291 >> v2:172.21.15.162:3300/0 conn(0x7f67d0143ea0 msgr2=0x7f67d00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67ce7fc640 1 --2- 172.21.15.133:0/3059089291 >> v2:172.21.15.162:3300/0 conn(0x7f67d0143ea0 0x7f67d00cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67ce7fc640 1 -- 172.21.15.133:0/3059089291 >> v1:172.21.15.133:6789/0 conn(0x7f67d015b9b0 legacy=0x7f67d015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67ce7fc640 1 -- 172.21.15.133:0/3059089291 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d015b1b0 con 0x7f67d00b2150 2024-04-25T10:06:46.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67cdffb640 1 -- 172.21.15.133:0/3059089291 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923847318 0 0) 0x7f67c80014a0 con 0x7f67d015b9b0 2024-04-25T10:06:46.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.106+0000 7f67ceffd640 1 --2- 172.21.15.133:0/3059089291 >> v2:172.21.15.162:3300/0 conn(0x7f67d0143ea0 0x7f67d00cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67ce7fc640 1 --2- 172.21.15.133:0/3059089291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d00b2150 0x7f67d006e3b0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f67d00b4b20 tx=0x7f67c401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4c8d59ae78ba68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67cdffb640 1 -- 172.21.15.133:0/3059089291 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40ab430 con 0x7f67d00b2150 2024-04-25T10:06:46.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67cdffb640 1 -- 172.21.15.133:0/3059089291 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c40b0070 con 0x7f67d00b2150 2024-04-25T10:06:46.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67cdffb640 1 -- 172.21.15.133:0/3059089291 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40ab7f0 con 0x7f67d00b2150 2024-04-25T10:06:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67d57ec640 1 -- 172.21.15.133:0/3059089291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d00b2150 msgr2=0x7f67d006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67d57ec640 1 --2- 172.21.15.133:0/3059089291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d00b2150 0x7f67d006e3b0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f67d00b4b20 tx=0x7f67c401c520 comp rx=0 tx=0).stop 2024-04-25T10:06:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67d57ec640 1 -- 172.21.15.133:0/3059089291 shutdown_connections 2024-04-25T10:06:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67d57ec640 1 --2- 172.21.15.133:0/3059089291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d00b2150 0x7f67d006e3b0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67d57ec640 1 --2- 172.21.15.133:0/3059089291 >> v2:172.21.15.162:3300/0 conn(0x7f67d0143ea0 0x7f67d00cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67d57ec640 1 -- 172.21.15.133:0/3059089291 >> 172.21.15.133:0/3059089291 conn(0x7f67d0066f70 msgr2=0x7f67d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67d57ec640 1 -- 172.21.15.133:0/3059089291 shutdown_connections 2024-04-25T10:06:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67d57ec640 1 -- 172.21.15.133:0/3059089291 wait complete. 2024-04-25T10:06:46.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67d57ec640 1 Processor -- start 2024-04-25T10:06:46.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.110+0000 7f67d57ec640 1 -- start start 2024-04-25T10:06:46.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67d57ec640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67d00b2150 0x7f67d0142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67d57ec640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d0143ea0 0x7f67d006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67d57ec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67d015b9b0 0x7f67d0142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67d57ec640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d01675e0 con 0x7f67d0143ea0 2024-04-25T10:06:46.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67d57ec640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d006ed00 con 0x7f67d00b2150 2024-04-25T10:06:46.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67d57ec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d0167760 con 0x7f67d015b9b0 2024-04-25T10:06:46.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ce7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d0143ea0 0x7f67d006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ce7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d0143ea0 0x7f67d006d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44042/0 (socket says 172.21.15.133:44042) 2024-04-25T10:06:46.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ce7fc640 1 -- 172.21.15.133:0/129264570 learned_addr learned my addr 172.21.15.133:0/129264570 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:46.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ceffd640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67d00b2150 0x7f67d0142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67cf7fe640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67d015b9b0 0x7f67d0142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ce7fc640 1 -- 172.21.15.133:0/129264570 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67d015b9b0 msgr2=0x7f67d0142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ce7fc640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67d015b9b0 0x7f67d0142210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ce7fc640 1 -- 172.21.15.133:0/129264570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67d00b2150 msgr2=0x7f67d0142e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ce7fc640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67d00b2150 0x7f67d0142e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ce7fc640 1 -- 172.21.15.133:0/129264570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d01414b0 con 0x7f67d0143ea0 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ce7fc640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d0143ea0 0x7f67d006d400 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f67c4019130 tx=0x7f67c401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67b77fe640 1 -- 172.21.15.133:0/129264570 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40b4800 con 0x7f67d0143ea0 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67b77fe640 1 -- 172.21.15.133:0/129264570 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c40b49a0 con 0x7f67d0143ea0 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67d57ec640 1 -- 172.21.15.133:0/129264570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d01417b0 con 0x7f67d0143ea0 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67b77fe640 1 -- 172.21.15.133:0/129264570 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40b4d90 con 0x7f67d0143ea0 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67d57ec640 1 -- 172.21.15.133:0/129264570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d0143390 con 0x7f67d0143ea0 2024-04-25T10:06:46.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67d57ec640 1 -- 172.21.15.133:0/129264570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d0068e80 con 0x7f67d0143ea0 2024-04-25T10:06:46.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67b77fe640 1 -- 172.21.15.133:0/129264570 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67c40b3380 con 0x7f67d0143ea0 2024-04-25T10:06:46.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67b77fe640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67bc066f60 0x7f67bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67b77fe640 1 -- 172.21.15.133:0/129264570 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67c40a8070 con 0x7f67d0143ea0 2024-04-25T10:06:46.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ceffd640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67bc066f60 0x7f67bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.114+0000 7f67ceffd640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67bc066f60 0x7f67bc069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f67c000a9a0 tx=0x7f67c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.118+0000 7f67b77fe640 1 -- 172.21.15.133:0/129264570 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67d01417b0 con 0x7f67d0143ea0 2024-04-25T10:06:46.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.290+0000 7f67d57ec640 1 -- 172.21.15.133:0/129264570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f67d00ceba0 con 0x7f67d0143ea0 2024-04-25T10:06:46.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.290+0000 7f67b77fe640 1 -- 172.21.15.133:0/129264570 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) v1 ==== 80+0+0 (secure 0 0 0) 0x7f67c4016110 con 0x7f67d0143ea0 2024-04-25T10:06:46.295 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:06:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 -- 172.21.15.133:0/129264570 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67bc066f60 msgr2=0x7f67bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67bc066f60 0x7f67bc069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f67c000a9a0 tx=0x7f67c0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 -- 172.21.15.133:0/129264570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d0143ea0 msgr2=0x7f67d006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d0143ea0 0x7f67d006d400 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f67c4019130 tx=0x7f67c401ad60 comp rx=0 tx=0).stop 2024-04-25T10:06:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 -- 172.21.15.133:0/129264570 shutdown_connections 2024-04-25T10:06:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67d015b9b0 0x7f67d0142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67bc066f60 0x7f67bc069420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d0143ea0 0x7f67d006d400 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 --2- 172.21.15.133:0/129264570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67d00b2150 0x7f67d0142e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 -- 172.21.15.133:0/129264570 >> 172.21.15.133:0/129264570 conn(0x7f67d0066f70 msgr2=0x7f67d00b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 -- 172.21.15.133:0/129264570 shutdown_connections 2024-04-25T10:06:46.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.294+0000 7f67d57ec640 1 -- 172.21.15.133:0/129264570 wait complete. 2024-04-25T10:06:46.314 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:06:46.314 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:46.320 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:46.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.482+0000 7f6e70b1d640 1 Processor -- start 2024-04-25T10:06:46.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.482+0000 7f6e70b1d640 1 -- start start 2024-04-25T10:06:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e70b1d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6e6c15c9a0 0x7f6e6c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e70b1d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c0c61d0 con 0x7f6e6c0b1f70 2024-04-25T10:06:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e70b1d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c0c64b0 con 0x7f6e6c0ce330 2024-04-25T10:06:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e70b1d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c06d4f0 con 0x7f6e6c15c9a0 2024-04-25T10:06:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e69d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6e6c0b1f70 0x7f6e6c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58004/0 (socket says 172.21.15.133:58004) 2024-04-25T10:06:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e69d9b640 1 -- 172.21.15.133:0/1750115473 learned_addr learned my addr 172.21.15.133:0/1750115473 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945978688 0 0) 0x7f6e6c0c61d0 con 0x7f6e6c0b1f70 2024-04-25T10:06:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6a59c640 1 --2- 172.21.15.133:0/1750115473 >> v2:172.21.15.162:3300/0 conn(0x7f6e6c15c9a0 0x7f6e6c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e54003530 con 0x7f6e6c0b1f70 2024-04-25T10:06:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3397298540 0 0) 0x7f6e54003530 con 0x7f6e6c0b1f70 2024-04-25T10:06:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 >> v2:172.21.15.162:3300/0 conn(0x7f6e6c15c9a0 msgr2=0x7f6e6c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 --2- 172.21.15.133:0/1750115473 >> v2:172.21.15.162:3300/0 conn(0x7f6e6c15c9a0 0x7f6e6c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 >> v1:172.21.15.162:6789/0 conn(0x7f6e6c0ce330 legacy=0x7f6e6c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c06d7d0 con 0x7f6e6c0b1f70 2024-04-25T10:06:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6e60001680 con 0x7f6e6c0b1f70 2024-04-25T10:06:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6a59c640 1 --2- 172.21.15.133:0/1750115473 >> v2:172.21.15.162:3300/0 conn(0x7f6e6c15c9a0 0x7f6e6c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 >> v1:172.21.15.133:6789/0 conn(0x7f6e6c0b1f70 legacy=0x7f6e6c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 --2- 172.21.15.133:0/1750115473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 0x7f6e54003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 --2- 172.21.15.133:0/1750115473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e54004810 0x7f6e54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 --2- 172.21.15.133:0/1750115473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e54005220 0x7f6e5400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c06d4f0 con 0x7f6e54003a10 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5400c170 con 0x7f6e54005220 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5400c370 con 0x7f6e54004810 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6a59c640 1 --2- 172.21.15.133:0/1750115473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 0x7f6e54003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6a59c640 1 -- 172.21.15.133:0/1750115473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e54004810 msgr2=0x7f6e54004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6a59c640 1 --2- 172.21.15.133:0/1750115473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e54004810 0x7f6e54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6ad9d640 1 --2- 172.21.15.133:0/1750115473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e54005220 0x7f6e5400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6a59c640 1 -- 172.21.15.133:0/1750115473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e54005220 msgr2=0x7f6e5400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6a59c640 1 --2- 172.21.15.133:0/1750115473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e54005220 0x7f6e5400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6a59c640 1 -- 172.21.15.133:0/1750115473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e54003530 con 0x7f6e54003a10 2024-04-25T10:06:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.486+0000 7f6e6ad9d640 1 -- 172.21.15.133:0/1750115473 reap_dead start 2024-04-25T10:06:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6a59c640 1 --2- 172.21.15.133:0/1750115473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 0x7f6e54003e30 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0098d0 tx=0x7f6e5c01c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e06262ed35c7df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c00f040 con 0x7f6e54003a10 2024-04-25T10:06:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c0af070 con 0x7f6e54003a10 2024-04-25T10:06:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6959a640 1 -- 172.21.15.133:0/1750115473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c0aa870 con 0x7f6e54003a10 2024-04-25T10:06:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 -- 172.21.15.133:0/1750115473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 msgr2=0x7f6e54003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 --2- 172.21.15.133:0/1750115473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 0x7f6e54003e30 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0098d0 tx=0x7f6e5c01c5d0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 -- 172.21.15.133:0/1750115473 shutdown_connections 2024-04-25T10:06:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 --2- 172.21.15.133:0/1750115473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 0x7f6e54003e30 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 -- 172.21.15.133:0/1750115473 >> 172.21.15.133:0/1750115473 conn(0x7f6e6c067000 msgr2=0x7f6e6c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 -- 172.21.15.133:0/1750115473 shutdown_connections 2024-04-25T10:06:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 -- 172.21.15.133:0/1750115473 wait complete. 2024-04-25T10:06:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 Processor -- start 2024-04-25T10:06:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 -- start start 2024-04-25T10:06:46.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 0x7f6e6c1f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e6c0c9c20 0x7f6e6c15b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15e360 0x7f6e6c15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15d140 con 0x7f6e54003a10 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15d2c0 con 0x7f6e6c0c9c20 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15d440 con 0x7f6e6c15e360 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 0x7f6e6c1f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 0x7f6e6c1f4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51868/0 (socket says 172.21.15.133:51868) 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6a59c640 1 -- 172.21.15.133:0/4096532711 learned_addr learned my addr 172.21.15.133:0/4096532711 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6ad9d640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15e360 0x7f6e6c15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e69d9b640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e6c0c9c20 0x7f6e6c15b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6a59c640 1 -- 172.21.15.133:0/4096532711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15e360 msgr2=0x7f6e6c15e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6a59c640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15e360 0x7f6e6c15e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6a59c640 1 -- 172.21.15.133:0/4096532711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e6c0c9c20 msgr2=0x7f6e6c15b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6a59c640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e6c0c9c20 0x7f6e6c15b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6a59c640 1 -- 172.21.15.133:0/4096532711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c0ca100 con 0x7f6e54003a10 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e6a59c640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 0x7f6e6c1f4800 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c01c400 tx=0x7f6e54005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e5affd640 1 -- 172.21.15.133:0/4096532711 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c0c1070 con 0x7f6e54003a10 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e5affd640 1 -- 172.21.15.133:0/4096532711 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c0aa6b0 con 0x7f6e54003a10 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 -- 172.21.15.133:0/4096532711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c0ca3e0 con 0x7f6e54003a10 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e5affd640 1 -- 172.21.15.133:0/4096532711 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c0aaa80 con 0x7f6e54003a10 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.490+0000 7f6e70b1d640 1 -- 172.21.15.133:0/4096532711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c0ca9a0 con 0x7f6e54003a10 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.494+0000 7f6e70b1d640 1 -- 172.21.15.133:0/4096532711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c068ef0 con 0x7f6e54003a10 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.494+0000 7f6e5affd640 1 -- 172.21.15.133:0/4096532711 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e5c0c4070 con 0x7f6e54003a10 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.494+0000 7f6e5affd640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 0x7f6e4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.494+0000 7f6e5affd640 1 -- 172.21.15.133:0/4096532711 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e5c11eed0 con 0x7f6e54003a10 2024-04-25T10:06:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.494+0000 7f6e69d9b640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 0x7f6e4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.494+0000 7f6e69d9b640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 0x7f6e4c069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6e60001ec0 tx=0x7f6e600017c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.498+0000 7f6e5affd640 1 -- 172.21.15.133:0/4096532711 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e5c0162e0 con 0x7f6e54003a10 2024-04-25T10:06:46.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.650+0000 7f6e70b1d640 1 -- 172.21.15.133:0/4096532711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e6c0b44e0 con 0x7f6e54003a10 2024-04-25T10:06:46.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.650+0000 7f6e5affd640 1 -- 172.21.15.133:0/4096532711 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16908 (secure 0 0 0) 0x7f6e5c0efcc0 con 0x7f6e54003a10 2024-04-25T10:06:46.655 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:46.656 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":66,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:06:38.208663+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:34.764755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:35.504408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 -- 172.21.15.133:0/4096532711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 msgr2=0x7f6e4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 0x7f6e4c069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6e60001ec0 tx=0x7f6e600017c0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 -- 172.21.15.133:0/4096532711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 msgr2=0x7f6e6c1f4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 0x7f6e6c1f4800 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c01c400 tx=0x7f6e54005840 comp rx=0 tx=0).stop 2024-04-25T10:06:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 -- 172.21.15.133:0/4096532711 shutdown_connections 2024-04-25T10:06:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15e360 0x7f6e6c15e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e6c0c9c20 0x7f6e6c15b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 0x7f6e4c069420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 --2- 172.21.15.133:0/4096532711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54003a10 0x7f6e6c1f4800 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 -- 172.21.15.133:0/4096532711 >> 172.21.15.133:0/4096532711 conn(0x7f6e6c067000 msgr2=0x7f6e6c0b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 -- 172.21.15.133:0/4096532711 shutdown_connections 2024-04-25T10:06:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.654+0000 7f6e70b1d640 1 -- 172.21.15.133:0/4096532711 wait complete. 2024-04-25T10:06:46.673 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:34.764755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:35.504408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:46.674 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:46.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe746f2640 1 Processor -- start 2024-04-25T10:06:46.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe746f2640 1 -- start start 2024-04-25T10:06:46.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe746f2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe6c0b33c0 0x7efe6c0b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe746f2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6c0b3840 con 0x7efe6c0b1640 2024-04-25T10:06:46.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe746f2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6c0cd930 con 0x7efe6c0ceb50 2024-04-25T10:06:46.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe746f2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0cb180 con 0x7efe6c0b33c0 2024-04-25T10:06:46.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe72c8f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7efe6c0b1640 0x7efe6c0cb8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58006/0 (socket says 172.21.15.133:58006) 2024-04-25T10:06:46.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe72c8f640 1 -- 172.21.15.133:0/2483103663 learned_addr learned my addr 172.21.15.133:0/2483103663 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:46.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe71c8d640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe6c0b33c0 0x7efe6c0b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947388244 0 0) 0x7efe640014a0 con 0x7efe6c0b1640 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe58003530 con 0x7efe6c0b1640 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2766200686 0 0) 0x7efe58003530 con 0x7efe6c0b1640 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe6c0b33c0 msgr2=0x7efe6c0b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe6c0b33c0 0x7efe6c0b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 >> v1:172.21.15.162:6789/0 conn(0x7efe6c0ceb50 legacy=0x7efe6c0cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c0ce4f0 con 0x7efe6c0b1640 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7efe64001bd0 con 0x7efe6c0b1640 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 >> v1:172.21.15.133:6789/0 conn(0x7efe6c0b1640 legacy=0x7efe6c0cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe580039f0 0x7efe58003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe6c0b33c0 0x7efe58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe58004d50 0x7efe580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5800ba80 con 0x7efe58004d50 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5800bc30 con 0x7efe580039f0 2024-04-25T10:06:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5800be30 con 0x7efe6c0b33c0 2024-04-25T10:06:46.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe72c8f640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe58004d50 0x7efe580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7248e640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe580039f0 0x7efe58003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe72c8f640 1 -- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe6c0b33c0 msgr2=0x7efe58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe71c8d640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe6c0b33c0 0x7efe58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe72c8f640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe6c0b33c0 0x7efe58004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe72c8f640 1 -- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe580039f0 msgr2=0x7efe58003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe72c8f640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe580039f0 0x7efe58003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe72c8f640 1 -- 172.21.15.133:0/2483103663 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe58003530 con 0x7efe58004d50 2024-04-25T10:06:46.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.782+0000 7efe7248e640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe580039f0 0x7efe58003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe72c8f640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe58004d50 0x7efe580051d0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7efe6400aa20 tx=0x7efe6401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86dc9fe5e19afeb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6400a670 con 0x7efe58004d50 2024-04-25T10:06:46.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe640ad650 con 0x7efe58004d50 2024-04-25T10:06:46.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe7148c640 1 -- 172.21.15.133:0/2483103663 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe640ada40 con 0x7efe58004d50 2024-04-25T10:06:46.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 -- 172.21.15.133:0/2483103663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe58004d50 msgr2=0x7efe580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 --2- 172.21.15.133:0/2483103663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe58004d50 0x7efe580051d0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7efe6400aa20 tx=0x7efe6401d360 comp rx=0 tx=0).stop 2024-04-25T10:06:46.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe72c8f640 1 -- 172.21.15.133:0/2483103663 reap_dead start 2024-04-25T10:06:46.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 -- 172.21.15.133:0/2483103663 shutdown_connections 2024-04-25T10:06:46.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 -- 172.21.15.133:0/2483103663 >> 172.21.15.133:0/2483103663 conn(0x7efe6c0d0570 msgr2=0x7efe6c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 -- 172.21.15.133:0/2483103663 shutdown_connections 2024-04-25T10:06:46.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 -- 172.21.15.133:0/2483103663 wait complete. 2024-04-25T10:06:46.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 Processor -- start 2024-04-25T10:06:46.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 -- start start 2024-04-25T10:06:46.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe58004d50 0x7efe6c0b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe6c0c7640 0x7efe6c148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c151ea0 0x7efe6c152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0cb440 con 0x7efe6c151ea0 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0d14c0 con 0x7efe6c0c7640 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.786+0000 7efe746f2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0b0d60 con 0x7efe58004d50 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe72c8f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c151ea0 0x7efe6c152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe72c8f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c151ea0 0x7efe6c152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51892/0 (socket says 172.21.15.133:51892) 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe72c8f640 1 -- 172.21.15.133:0/1602683588 learned_addr learned my addr 172.21.15.133:0/1602683588 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe7248e640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe58004d50 0x7efe6c0b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe71c8d640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe6c0c7640 0x7efe6c148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe72c8f640 1 -- 172.21.15.133:0/1602683588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe58004d50 msgr2=0x7efe6c0b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe72c8f640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe58004d50 0x7efe6c0b25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe72c8f640 1 -- 172.21.15.133:0/1602683588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe6c0c7640 msgr2=0x7efe6c148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe72c8f640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe6c0c7640 0x7efe6c148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe72c8f640 1 -- 172.21.15.133:0/1602683588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c169590 con 0x7efe6c151ea0 2024-04-25T10:06:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe72c8f640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c151ea0 0x7efe6c152320 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7efe6401cdd0 tx=0x7efe58005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe62ffd640 1 -- 172.21.15.133:0/1602683588 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6400a670 con 0x7efe6c151ea0 2024-04-25T10:06:46.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe62ffd640 1 -- 172.21.15.133:0/1602683588 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe640b3040 con 0x7efe6c151ea0 2024-04-25T10:06:46.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe746f2640 1 -- 172.21.15.133:0/1602683588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c169800 con 0x7efe6c151ea0 2024-04-25T10:06:46.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe62ffd640 1 -- 172.21.15.133:0/1602683588 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe64001b90 con 0x7efe6c151ea0 2024-04-25T10:06:46.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe746f2640 1 -- 172.21.15.133:0/1602683588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c1688b0 con 0x7efe6c151ea0 2024-04-25T10:06:46.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe746f2640 1 -- 172.21.15.133:0/1602683588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c067c70 con 0x7efe6c151ea0 2024-04-25T10:06:46.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe62ffd640 1 -- 172.21.15.133:0/1602683588 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe640ad650 con 0x7efe6c151ea0 2024-04-25T10:06:46.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe62ffd640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe50066f60 0x7efe50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe62ffd640 1 -- 172.21.15.133:0/1602683588 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe6411fd50 con 0x7efe6c151ea0 2024-04-25T10:06:46.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe7248e640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe50066f60 0x7efe50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.790+0000 7efe7248e640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe50066f60 0x7efe50069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efe68001760 tx=0x7efe68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.794+0000 7efe62ffd640 1 -- 172.21.15.133:0/1602683588 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe64123030 con 0x7efe6c151ea0 2024-04-25T10:06:46.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.942+0000 7efe746f2640 1 -- 172.21.15.133:0/1602683588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe6c0ce930 con 0x7efe6c151ea0 2024-04-25T10:06:46.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.942+0000 7efe62ffd640 1 -- 172.21.15.133:0/1602683588 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe640c0060 con 0x7efe6c151ea0 2024-04-25T10:06:46.948 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:06:46.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 -- 172.21.15.133:0/1602683588 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe50066f60 msgr2=0x7efe50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe50066f60 0x7efe50069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efe68001760 tx=0x7efe68009040 comp rx=0 tx=0).stop 2024-04-25T10:06:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 -- 172.21.15.133:0/1602683588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c151ea0 msgr2=0x7efe6c152320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c151ea0 0x7efe6c152320 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7efe6401cdd0 tx=0x7efe58005840 comp rx=0 tx=0).stop 2024-04-25T10:06:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 -- 172.21.15.133:0/1602683588 shutdown_connections 2024-04-25T10:06:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe50066f60 0x7efe50069420 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c151ea0 0x7efe6c152320 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe6c0c7640 0x7efe6c148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 --2- 172.21.15.133:0/1602683588 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe58004d50 0x7efe6c0b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 -- 172.21.15.133:0/1602683588 >> 172.21.15.133:0/1602683588 conn(0x7efe6c0d0570 msgr2=0x7efe6c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 -- 172.21.15.133:0/1602683588 shutdown_connections 2024-04-25T10:06:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:46.946+0000 7efe746f2640 1 -- 172.21.15.133:0/1602683588 wait complete. 2024-04-25T10:06:46.964 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a28b6d640 1 Processor -- start 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a28b6d640 1 -- start start 2024-04-25T10:06:47.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a28b6d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7a240d10c0 0x7f7a2406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a28b6d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7a240cabb0 0x7f7a240cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a28b6d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a240cc950 con 0x7f7a2406ac90 2024-04-25T10:06:47.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a28b6d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a2406b5b0 con 0x7f7a240d10c0 2024-04-25T10:06:47.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a28b6d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a240b1ac0 con 0x7f7a240cabb0 2024-04-25T10:06:47.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a2259c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7a2406ac90 0x7f7a240b1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58014/0 (socket says 172.21.15.133:58014) 2024-04-25T10:06:47.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a2259c640 1 -- 172.21.15.133:0/4272936336 learned_addr learned my addr 172.21.15.133:0/4272936336 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:47.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a21d9b640 1 --2- 172.21.15.133:0/4272936336 >> v2:172.21.15.133:3300/0 conn(0x7f7a240d10c0 0x7f7a2406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a22d9d640 1 --2- 172.21.15.133:0/4272936336 >> v2:172.21.15.162:3300/0 conn(0x7f7a240cabb0 0x7f7a240cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.070+0000 7f7a2159a640 1 -- 172.21.15.133:0/4272936336 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806692658 0 0) 0x7f7a240cc950 con 0x7f7a2406ac90 2024-04-25T10:06:47.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a2159a640 1 -- 172.21.15.133:0/4272936336 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a0c003530 con 0x7f7a2406ac90 2024-04-25T10:06:47.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a21d9b640 1 -- 172.21.15.133:0/4272936336 >> v2:172.21.15.162:3300/0 conn(0x7f7a240cabb0 msgr2=0x7f7a240cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a21d9b640 1 --2- 172.21.15.133:0/4272936336 >> v2:172.21.15.162:3300/0 conn(0x7f7a240cabb0 0x7f7a240cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a21d9b640 1 -- 172.21.15.133:0/4272936336 >> v1:172.21.15.133:6789/0 conn(0x7f7a2406ac90 legacy=0x7f7a240b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a21d9b640 1 -- 172.21.15.133:0/4272936336 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a240ccbc0 con 0x7f7a240d10c0 2024-04-25T10:06:47.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a22d9d640 1 --2- 172.21.15.133:0/4272936336 >> v2:172.21.15.162:3300/0 conn(0x7f7a240cabb0 0x7f7a240cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:47.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a21d9b640 1 --2- 172.21.15.133:0/4272936336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a240d10c0 0x7f7a2406b070 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7a240c75c0 tx=0x7f7a1801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f98a5d71efe4d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a2159a640 1 -- 172.21.15.133:0/4272936336 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a180b0070 con 0x7f7a240d10c0 2024-04-25T10:06:47.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a2159a640 1 -- 172.21.15.133:0/4272936336 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a180ab600 con 0x7f7a240d10c0 2024-04-25T10:06:47.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a2159a640 1 -- 172.21.15.133:0/4272936336 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a180ab9d0 con 0x7f7a240d10c0 2024-04-25T10:06:47.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a28b6d640 1 -- 172.21.15.133:0/4272936336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a240d10c0 msgr2=0x7f7a2406b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a28b6d640 1 --2- 172.21.15.133:0/4272936336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a240d10c0 0x7f7a2406b070 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7a240c75c0 tx=0x7f7a1801c3f0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a28b6d640 1 -- 172.21.15.133:0/4272936336 shutdown_connections 2024-04-25T10:06:47.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a28b6d640 1 --2- 172.21.15.133:0/4272936336 >> v2:172.21.15.162:3300/0 conn(0x7f7a240cabb0 0x7f7a240cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a28b6d640 1 --2- 172.21.15.133:0/4272936336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a240d10c0 0x7f7a2406b070 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a28b6d640 1 -- 172.21.15.133:0/4272936336 >> 172.21.15.133:0/4272936336 conn(0x7f7a240999d0 msgr2=0x7f7a240d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a28b6d640 1 -- 172.21.15.133:0/4272936336 shutdown_connections 2024-04-25T10:06:47.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.074+0000 7f7a28b6d640 1 -- 172.21.15.133:0/4272936336 wait complete. 2024-04-25T10:06:47.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a28b6d640 1 Processor -- start 2024-04-25T10:06:47.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a28b6d640 1 -- start start 2024-04-25T10:06:47.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a28b6d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a2406ac90 0x7f7a24150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a28b6d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a240cabb0 0x7f7a240aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a28b6d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a240d10c0 0x7f7a2406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a28b6d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2415df60 con 0x7f7a240cabb0 2024-04-25T10:06:47.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a28b6d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2415dde0 con 0x7f7a2406ac90 2024-04-25T10:06:47.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a28b6d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2416ae90 con 0x7f7a240d10c0 2024-04-25T10:06:47.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a21d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a240cabb0 0x7f7a240aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a21d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a240cabb0 0x7f7a240aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51918/0 (socket says 172.21.15.133:51918) 2024-04-25T10:06:47.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a21d9b640 1 -- 172.21.15.133:0/3849277690 learned_addr learned my addr 172.21.15.133:0/3849277690 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:47.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a22d9d640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a240d10c0 0x7f7a2406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a2259c640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a2406ac90 0x7f7a24150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a21d9b640 1 -- 172.21.15.133:0/3849277690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a240d10c0 msgr2=0x7f7a2406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a21d9b640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a240d10c0 0x7f7a2406e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a21d9b640 1 -- 172.21.15.133:0/3849277690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a2406ac90 msgr2=0x7f7a24150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a21d9b640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a2406ac90 0x7f7a24150af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a21d9b640 1 -- 172.21.15.133:0/3849277690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a24151030 con 0x7f7a240cabb0 2024-04-25T10:06:47.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a21d9b640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a240cabb0 0x7f7a240aed20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f7a180ae040 tx=0x7f7a18019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a12ffd640 1 -- 172.21.15.133:0/3849277690 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a180b0040 con 0x7f7a240cabb0 2024-04-25T10:06:47.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a12ffd640 1 -- 172.21.15.133:0/3849277690 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a180b9070 con 0x7f7a240cabb0 2024-04-25T10:06:47.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a28b6d640 1 -- 172.21.15.133:0/3849277690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a240af260 con 0x7f7a240cabb0 2024-04-25T10:06:47.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a12ffd640 1 -- 172.21.15.133:0/3849277690 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a180b4690 con 0x7f7a240cabb0 2024-04-25T10:06:47.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a28b6d640 1 -- 172.21.15.133:0/3849277690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a24151830 con 0x7f7a240cabb0 2024-04-25T10:06:47.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a28b6d640 1 -- 172.21.15.133:0/3849277690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a24067c70 con 0x7f7a240cabb0 2024-04-25T10:06:47.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a12ffd640 1 -- 172.21.15.133:0/3849277690 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a180b4830 con 0x7f7a240cabb0 2024-04-25T10:06:47.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.078+0000 7f7a12ffd640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a04066f60 0x7f7a04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.082+0000 7f7a2259c640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a04066f60 0x7f7a04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.082+0000 7f7a12ffd640 1 -- 172.21.15.133:0/3849277690 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7a1811ee30 con 0x7f7a240cabb0 2024-04-25T10:06:47.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.082+0000 7f7a2259c640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a04066f60 0x7f7a04069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7a14007fd0 tx=0x7f7a14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.086+0000 7f7a12ffd640 1 -- 172.21.15.133:0/3849277690 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a18016070 con 0x7f7a240cabb0 2024-04-25T10:06:47.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.230+0000 7f7a28b6d640 1 -- 172.21.15.133:0/3849277690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a2406b280 con 0x7f7a240cabb0 2024-04-25T10:06:47.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.230+0000 7f7a12ffd640 1 -- 172.21.15.133:0/3849277690 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a180efd40 con 0x7f7a240cabb0 2024-04-25T10:06:47.235 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:06:47.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 -- 172.21.15.133:0/3849277690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a04066f60 msgr2=0x7f7a04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a04066f60 0x7f7a04069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7a14007fd0 tx=0x7f7a14009040 comp rx=0 tx=0).stop 2024-04-25T10:06:47.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 -- 172.21.15.133:0/3849277690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a240cabb0 msgr2=0x7f7a240aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a240cabb0 0x7f7a240aed20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f7a180ae040 tx=0x7f7a18019190 comp rx=0 tx=0).stop 2024-04-25T10:06:47.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 -- 172.21.15.133:0/3849277690 shutdown_connections 2024-04-25T10:06:47.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a240d10c0 0x7f7a2406e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a04066f60 0x7f7a04069420 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a240cabb0 0x7f7a240aed20 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 --2- 172.21.15.133:0/3849277690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a2406ac90 0x7f7a24150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 -- 172.21.15.133:0/3849277690 >> 172.21.15.133:0/3849277690 conn(0x7f7a240999d0 msgr2=0x7f7a240d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 -- 172.21.15.133:0/3849277690 shutdown_connections 2024-04-25T10:06:47.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.234+0000 7f7a28b6d640 1 -- 172.21.15.133:0/3849277690 wait complete. 2024-04-25T10:06:47.252 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:47.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f06e8e640 1 Processor -- start 2024-04-25T10:06:47.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f06e8e640 1 -- start start 2024-04-25T10:06:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f06e8e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8f000d1060 0x7f8f000ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f06e8e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f000d0300 con 0x7f8f000b5ef0 2024-04-25T10:06:47.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f06e8e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f000cfaa0 con 0x7f8f0006b3a0 2024-04-25T10:06:47.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f06e8e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f000cfd40 con 0x7f8f000d1060 2024-04-25T10:06:47.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f04c2a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8f000b5ef0 0x7f8f0006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58018/0 (socket says 172.21.15.133:58018) 2024-04-25T10:06:47.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f04c2a640 1 -- 172.21.15.133:0/1297003668 learned_addr learned my addr 172.21.15.133:0/1297003668 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:47.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f0542b640 1 --2- 172.21.15.133:0/1297003668 >> v2:172.21.15.133:3300/0 conn(0x7f8f000d1060 0x7f8f000ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8eff7fe640 1 -- 172.21.15.133:0/1297003668 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966782548 0 0) 0x7f8f000d0300 con 0x7f8f000b5ef0 2024-04-25T10:06:47.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8eff7fe640 1 -- 172.21.15.133:0/1297003668 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ee8003530 con 0x7f8f000b5ef0 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f0542b640 1 -- 172.21.15.133:0/1297003668 >> v1:172.21.15.162:6789/0 conn(0x7f8f0006b3a0 legacy=0x7f8f000cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f0542b640 1 -- 172.21.15.133:0/1297003668 >> v1:172.21.15.133:6789/0 conn(0x7f8f000b5ef0 legacy=0x7f8f0006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.370+0000 7f8f0542b640 1 -- 172.21.15.133:0/1297003668 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f000d0020 con 0x7f8f000d1060 2024-04-25T10:06:47.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8f0542b640 1 --2- 172.21.15.133:0/1297003668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f000d1060 0x7f8f000ce140 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009d90 tx=0x7f8ef801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9125f1445fa78fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8eff7fe640 1 -- 172.21.15.133:0/1297003668 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80b2070 con 0x7f8f000d1060 2024-04-25T10:06:47.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8eff7fe640 1 -- 172.21.15.133:0/1297003668 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef80a59c0 con 0x7f8f000d1060 2024-04-25T10:06:47.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8eff7fe640 1 -- 172.21.15.133:0/1297003668 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80a5d90 con 0x7f8f000d1060 2024-04-25T10:06:47.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8f06e8e640 1 -- 172.21.15.133:0/1297003668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f000d1060 msgr2=0x7f8f000ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8f06e8e640 1 --2- 172.21.15.133:0/1297003668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f000d1060 0x7f8f000ce140 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009d90 tx=0x7f8ef801c900 comp rx=0 tx=0).stop 2024-04-25T10:06:47.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8f06e8e640 1 -- 172.21.15.133:0/1297003668 shutdown_connections 2024-04-25T10:06:47.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8f06e8e640 1 --2- 172.21.15.133:0/1297003668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f000d1060 0x7f8f000ce140 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8f06e8e640 1 -- 172.21.15.133:0/1297003668 >> 172.21.15.133:0/1297003668 conn(0x7f8f000d0570 msgr2=0x7f8f000d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8f06e8e640 1 -- 172.21.15.133:0/1297003668 shutdown_connections 2024-04-25T10:06:47.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8f06e8e640 1 -- 172.21.15.133:0/1297003668 wait complete. 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8f06e8e640 1 Processor -- start 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8f06e8e640 1 -- start start 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.374+0000 7f8f06e8e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f0006b3a0 0x7f8f001505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f06e8e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f000b5ef0 0x7f8f000ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f06e8e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f000d1060 0x7f8f00160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f06e8e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f000b6450 con 0x7f8f000d1060 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f06e8e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f000b62d0 con 0x7f8f000b5ef0 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f06e8e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0016b8b0 con 0x7f8f0006b3a0 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f0542b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f000d1060 0x7f8f00160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f0542b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f000d1060 0x7f8f00160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51924/0 (socket says 172.21.15.133:51924) 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f0542b640 1 -- 172.21.15.133:0/3835702009 learned_addr learned my addr 172.21.15.133:0/3835702009 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f04c2a640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f0006b3a0 0x7f8f001505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8efffff640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f000b5ef0 0x7f8f000ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f0542b640 1 -- 172.21.15.133:0/3835702009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f0006b3a0 msgr2=0x7f8f001505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f0542b640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f0006b3a0 0x7f8f001505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f0542b640 1 -- 172.21.15.133:0/3835702009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f000b5ef0 msgr2=0x7f8f000ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f0542b640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f000b5ef0 0x7f8f000ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f0542b640 1 -- 172.21.15.133:0/3835702009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0006ca90 con 0x7f8f000d1060 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8efffff640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f000b5ef0 0x7f8f000ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f0542b640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f000d1060 0x7f8f00160570 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80b0040 tx=0x7f8ef80196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8efd7fa640 1 -- 172.21.15.133:0/3835702009 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80b2050 con 0x7f8f000d1060 2024-04-25T10:06:47.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8efd7fa640 1 -- 172.21.15.133:0/3835702009 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef80be070 con 0x7f8f000d1060 2024-04-25T10:06:47.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8efd7fa640 1 -- 172.21.15.133:0/3835702009 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80ada70 con 0x7f8f000d1060 2024-04-25T10:06:47.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f06e8e640 1 -- 172.21.15.133:0/3835702009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0015f220 con 0x7f8f000d1060 2024-04-25T10:06:47.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f06e8e640 1 -- 172.21.15.133:0/3835702009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0015f7e0 con 0x7f8f000d1060 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f06e8e640 1 -- 172.21.15.133:0/3835702009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0015efb0 con 0x7f8f000d1060 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8efd7fa640 1 -- 172.21.15.133:0/3835702009 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f000cc3f0 con 0x7f8f000d1060 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8efd7fa640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8eec066f90 0x7f8eec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f04c2a640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8eec066f90 0x7f8eec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8efd7fa640 1 -- 172.21.15.133:0/3835702009 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ef8120040 con 0x7f8f000d1060 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.378+0000 7f8f04c2a640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8eec066f90 0x7f8eec069450 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0007fd0 tx=0x7f8ef0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.382+0000 7f8efd7fa640 1 -- 172.21.15.133:0/3835702009 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ef8016520 con 0x7f8f000d1060 2024-04-25T10:06:47.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.530+0000 7f8f06e8e640 1 -- 172.21.15.133:0/3835702009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f00067c70 con 0x7f8f000d1060 2024-04-25T10:06:47.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.530+0000 7f8efd7fa640 1 -- 172.21.15.133:0/3835702009 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8ef80f0f50 con 0x7f8f000d1060 2024-04-25T10:06:47.535 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:06:47.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 -- 172.21.15.133:0/3835702009 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8eec066f90 msgr2=0x7f8eec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8eec066f90 0x7f8eec069450 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0007fd0 tx=0x7f8ef0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:47.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 -- 172.21.15.133:0/3835702009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f000d1060 msgr2=0x7f8f00160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f000d1060 0x7f8f00160570 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80b0040 tx=0x7f8ef80196a0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 -- 172.21.15.133:0/3835702009 shutdown_connections 2024-04-25T10:06:47.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8eec066f90 0x7f8eec069450 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f000d1060 0x7f8f00160570 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f000b5ef0 0x7f8f000ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 --2- 172.21.15.133:0/3835702009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8f0006b3a0 0x7f8f001505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 -- 172.21.15.133:0/3835702009 >> 172.21.15.133:0/3835702009 conn(0x7f8f000d0570 msgr2=0x7f8f0006e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 -- 172.21.15.133:0/3835702009 shutdown_connections 2024-04-25T10:06:47.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.534+0000 7f8f06e8e640 1 -- 172.21.15.133:0/3835702009 wait complete. 2024-04-25T10:06:47.553 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:47.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee38a14640 1 Processor -- start 2024-04-25T10:06:47.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee38a14640 1 -- start start 2024-04-25T10:06:47.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee38a14640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee34143ea0 0x7fee340cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee38a14640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fee340b2150 0x7fee3406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee38a14640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3415ac50 con 0x7fee3415b9b0 2024-04-25T10:06:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee38a14640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee340cd180 con 0x7fee340b2150 2024-04-25T10:06:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee38a14640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3415aed0 con 0x7fee34143ea0 2024-04-25T10:06:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee31d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fee340b2150 0x7fee3406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee31d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fee340b2150 0x7fee3406e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51936/0 (socket says 172.21.15.133:51936) 2024-04-25T10:06:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee31d9b640 1 -- 172.21.15.133:0/1455924219 learned_addr learned my addr 172.21.15.133:0/1455924219 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee3259c640 1 --2- 172.21.15.133:0/1455924219 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee34143ea0 0x7fee340cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee31d9b640 1 -- 172.21.15.133:0/1455924219 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee34143ea0 msgr2=0x7fee340cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee31d9b640 1 --2- 172.21.15.133:0/1455924219 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee34143ea0 0x7fee340cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee31d9b640 1 -- 172.21.15.133:0/1455924219 >> v1:172.21.15.162:6789/0 conn(0x7fee3415b9b0 legacy=0x7fee3415e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee31d9b640 1 -- 172.21.15.133:0/1455924219 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3415b1b0 con 0x7fee340b2150 2024-04-25T10:06:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.670+0000 7fee3259c640 1 --2- 172.21.15.133:0/1455924219 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee34143ea0 0x7fee340cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:47.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee31d9b640 1 --2- 172.21.15.133:0/1455924219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee340b2150 0x7fee3406e3b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009870 tx=0x7fee1c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a1097ef26b21c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee3159a640 1 -- 172.21.15.133:0/1455924219 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0b0070 con 0x7fee340b2150 2024-04-25T10:06:47.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee3159a640 1 -- 172.21.15.133:0/1455924219 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c0ab600 con 0x7fee340b2150 2024-04-25T10:06:47.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee3159a640 1 -- 172.21.15.133:0/1455924219 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0ab9d0 con 0x7fee340b2150 2024-04-25T10:06:47.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 -- 172.21.15.133:0/1455924219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee340b2150 msgr2=0x7fee3406e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 --2- 172.21.15.133:0/1455924219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee340b2150 0x7fee3406e3b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009870 tx=0x7fee1c01c340 comp rx=0 tx=0).stop 2024-04-25T10:06:47.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 -- 172.21.15.133:0/1455924219 shutdown_connections 2024-04-25T10:06:47.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 --2- 172.21.15.133:0/1455924219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee340b2150 0x7fee3406e3b0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 --2- 172.21.15.133:0/1455924219 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee34143ea0 0x7fee340cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 -- 172.21.15.133:0/1455924219 >> 172.21.15.133:0/1455924219 conn(0x7fee34066f70 msgr2=0x7fee3406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 -- 172.21.15.133:0/1455924219 shutdown_connections 2024-04-25T10:06:47.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 -- 172.21.15.133:0/1455924219 wait complete. 2024-04-25T10:06:47.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 Processor -- start 2024-04-25T10:06:47.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 -- start start 2024-04-25T10:06:47.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee340b2150 0x7fee34135960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee34143ea0 0x7fee340cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3415b9b0 0x7fee340cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee341675e0 con 0x7fee3415b9b0 2024-04-25T10:06:47.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3406ed00 con 0x7fee34143ea0 2024-04-25T10:06:47.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee38a14640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee34167760 con 0x7fee340b2150 2024-04-25T10:06:47.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee32d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3415b9b0 0x7fee340cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee32d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3415b9b0 0x7fee340cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51948/0 (socket says 172.21.15.133:51948) 2024-04-25T10:06:47.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee32d9d640 1 -- 172.21.15.133:0/3673115475 learned_addr learned my addr 172.21.15.133:0/3673115475 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:47.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.674+0000 7fee31d9b640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee34143ea0 0x7fee340cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee3259c640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee340b2150 0x7fee34135960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee32d9d640 1 -- 172.21.15.133:0/3673115475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee340b2150 msgr2=0x7fee34135960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee32d9d640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee340b2150 0x7fee34135960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee32d9d640 1 -- 172.21.15.133:0/3673115475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee34143ea0 msgr2=0x7fee340cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee32d9d640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee34143ea0 0x7fee340cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee32d9d640 1 -- 172.21.15.133:0/3673115475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee341543c0 con 0x7fee3415b9b0 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee32d9d640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3415b9b0 0x7fee340cc480 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fee2400be60 tx=0x7fee2400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee2affd640 1 -- 172.21.15.133:0/3673115475 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24019070 con 0x7fee3415b9b0 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee2affd640 1 -- 172.21.15.133:0/3673115475 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee24014440 con 0x7fee3415b9b0 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee38a14640 1 -- 172.21.15.133:0/3673115475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee34156bc0 con 0x7fee3415b9b0 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee2affd640 1 -- 172.21.15.133:0/3673115475 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24014830 con 0x7fee3415b9b0 2024-04-25T10:06:47.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee38a14640 1 -- 172.21.15.133:0/3673115475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee34154940 con 0x7fee3415b9b0 2024-04-25T10:06:47.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee38a14640 1 -- 172.21.15.133:0/3673115475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee34068e80 con 0x7fee3415b9b0 2024-04-25T10:06:47.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee2affd640 1 -- 172.21.15.133:0/3673115475 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee2401c070 con 0x7fee3415b9b0 2024-04-25T10:06:47.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee2affd640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 0x7fee14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee2affd640 1 -- 172.21.15.133:0/3673115475 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fee24085150 con 0x7fee3415b9b0 2024-04-25T10:06:47.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee3259c640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 0x7fee14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.678+0000 7fee3259c640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 0x7fee14069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fee2c006fd0 tx=0x7fee2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.682+0000 7fee2affd640 1 -- 172.21.15.133:0/3673115475 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee24055fd0 con 0x7fee3415b9b0 2024-04-25T10:06:47.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.854+0000 7fee38a14640 1 -- 172.21.15.133:0/3673115475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fee34156990 con 0x7fee3415b9b0 2024-04-25T10:06:47.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.854+0000 7fee2affd640 1 -- 172.21.15.133:0/3673115475 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) v1 ==== 80+0+0 (secure 0 0 0) 0x7fee240558d0 con 0x7fee3415b9b0 2024-04-25T10:06:47.858 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:06:47.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.854+0000 7fee38a14640 1 -- 172.21.15.133:0/3673115475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 msgr2=0x7fee14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.854+0000 7fee38a14640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 0x7fee14069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fee2c006fd0 tx=0x7fee2c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:47.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.858+0000 7fee38a14640 1 -- 172.21.15.133:0/3673115475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3415b9b0 msgr2=0x7fee340cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.858+0000 7fee38a14640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3415b9b0 0x7fee340cc480 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fee2400be60 tx=0x7fee2400bf50 comp rx=0 tx=0).stop 2024-04-25T10:06:47.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.858+0000 7fee38a14640 1 -- 172.21.15.133:0/3673115475 shutdown_connections 2024-04-25T10:06:47.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.858+0000 7fee38a14640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 0x7fee14069420 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.858+0000 7fee38a14640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3415b9b0 0x7fee340cc480 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.858+0000 7fee38a14640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee34143ea0 0x7fee340cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.858+0000 7fee38a14640 1 --2- 172.21.15.133:0/3673115475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee340b2150 0x7fee34135960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.858+0000 7fee38a14640 1 -- 172.21.15.133:0/3673115475 >> 172.21.15.133:0/3673115475 conn(0x7fee34066f70 msgr2=0x7fee34142ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.858+0000 7fee38a14640 1 -- 172.21.15.133:0/3673115475 shutdown_connections 2024-04-25T10:06:47.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:47.858+0000 7fee38a14640 1 -- 172.21.15.133:0/3673115475 wait complete. 2024-04-25T10:06:47.878 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:06:47.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:47.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:47.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[56222]: fuse finished with error 0 and tester_r 0 2024-04-25T10:06:47.890 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:51.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:51.047 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:53.993 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:06:53.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:53.993 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:06:54.001 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:54.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f88479ac640 1 Processor -- start 2024-04-25T10:06:54.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f88479ac640 1 -- start start 2024-04-25T10:06:54.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f88479ac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8840158110 0x7f88400b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f88479ac640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f88401584f0 0x7f884015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f88479ac640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f884006e460 0x7f884006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f88479ac640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884006edf0 con 0x7f88401584f0 2024-04-25T10:06:54.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f88479ac640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88400cbe60 con 0x7f884006e460 2024-04-25T10:06:54.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f88479ac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88400b19b0 con 0x7f8840158110 2024-04-25T10:06:54.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f8844f47640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f88401584f0 0x7f884015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f8844f47640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f88401584f0 0x7f884015ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51952/0 (socket says 172.21.15.133:51952) 2024-04-25T10:06:54.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f8844f47640 1 -- 172.21.15.133:0/1822458735 learned_addr learned my addr 172.21.15.133:0/1822458735 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:54.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f8845f49640 1 --2- 172.21.15.133:0/1822458735 >> v2:172.21.15.162:3300/0 conn(0x7f884006e460 0x7f884006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.169+0000 7f8845748640 1 --2- 172.21.15.133:0/1822458735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8840158110 0x7f88400b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f8844f47640 1 -- 172.21.15.133:0/1822458735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8840158110 msgr2=0x7f88400b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f8844f47640 1 --2- 172.21.15.133:0/1822458735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8840158110 0x7f88400b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f8844f47640 1 -- 172.21.15.133:0/1822458735 >> v2:172.21.15.162:3300/0 conn(0x7f884006e460 msgr2=0x7f884006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f8844f47640 1 --2- 172.21.15.133:0/1822458735 >> v2:172.21.15.162:3300/0 conn(0x7f884006e460 0x7f884006e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f8844f47640 1 -- 172.21.15.133:0/1822458735 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884015b360 con 0x7f88401584f0 2024-04-25T10:06:54.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f8845f49640 1 --2- 172.21.15.133:0/1822458735 >> v2:172.21.15.162:3300/0 conn(0x7f884006e460 0x7f884006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:54.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f8844f47640 1 --2- 172.21.15.133:0/1822458735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88401584f0 0x7f884015ad10 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f884015f3a0 tx=0x7f883001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbf44c4d97c064b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f8837fff640 1 -- 172.21.15.133:0/1822458735 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88300aa610 con 0x7f88401584f0 2024-04-25T10:06:54.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f8837fff640 1 -- 172.21.15.133:0/1822458735 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88300af070 con 0x7f88401584f0 2024-04-25T10:06:54.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f8837fff640 1 -- 172.21.15.133:0/1822458735 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88300aa9d0 con 0x7f88401584f0 2024-04-25T10:06:54.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f88479ac640 1 -- 172.21.15.133:0/1822458735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88401584f0 msgr2=0x7f884015ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f88479ac640 1 --2- 172.21.15.133:0/1822458735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88401584f0 0x7f884015ad10 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f884015f3a0 tx=0x7f883001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f88479ac640 1 -- 172.21.15.133:0/1822458735 shutdown_connections 2024-04-25T10:06:54.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f88479ac640 1 --2- 172.21.15.133:0/1822458735 >> v2:172.21.15.162:3300/0 conn(0x7f884006e460 0x7f884006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f88479ac640 1 --2- 172.21.15.133:0/1822458735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88401584f0 0x7f884015ad10 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f88479ac640 1 --2- 172.21.15.133:0/1822458735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8840158110 0x7f88400b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f88479ac640 1 -- 172.21.15.133:0/1822458735 >> 172.21.15.133:0/1822458735 conn(0x7f8840067000 msgr2=0x7f884006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f88479ac640 1 -- 172.21.15.133:0/1822458735 shutdown_connections 2024-04-25T10:06:54.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.173+0000 7f88479ac640 1 -- 172.21.15.133:0/1822458735 wait complete. 2024-04-25T10:06:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f88479ac640 1 Processor -- start 2024-04-25T10:06:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f88479ac640 1 -- start start 2024-04-25T10:06:54.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f88479ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f884006e460 0x7f88400c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f88479ac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8840158110 0x7f8840150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f88479ac640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88401584f0 0x7f88400c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f88479ac640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88400cd7f0 con 0x7f884006e460 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f88479ac640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88400cd670 con 0x7f88401584f0 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f88479ac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884015eb10 con 0x7f8840158110 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8845748640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f884006e460 0x7f88400c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8845748640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f884006e460 0x7f88400c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51966/0 (socket says 172.21.15.133:51966) 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8845748640 1 -- 172.21.15.133:0/913755497 learned_addr learned my addr 172.21.15.133:0/913755497 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8844f47640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8840158110 0x7f8840150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8845f49640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88401584f0 0x7f88400c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8845748640 1 -- 172.21.15.133:0/913755497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8840158110 msgr2=0x7f8840150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8845748640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8840158110 0x7f8840150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8845748640 1 -- 172.21.15.133:0/913755497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88401584f0 msgr2=0x7f88400c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8845748640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88401584f0 0x7f88400c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8845748640 1 -- 172.21.15.133:0/913755497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400ca940 con 0x7f884006e460 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8845748640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f884006e460 0x7f88400c9ac0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f883c00d800 tx=0x7f883c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8835ffb640 1 -- 172.21.15.133:0/913755497 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00b6c0 con 0x7f884006e460 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8835ffb640 1 -- 172.21.15.133:0/913755497 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c019070 con 0x7f884006e460 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f88479ac640 1 -- 172.21.15.133:0/913755497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8840150f70 con 0x7f884006e460 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8835ffb640 1 -- 172.21.15.133:0/913755497 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00bae0 con 0x7f884006e460 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f88479ac640 1 -- 172.21.15.133:0/913755497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88400ca380 con 0x7f884006e460 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f88479ac640 1 -- 172.21.15.133:0/913755497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8840068ef0 con 0x7f884006e460 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.177+0000 7f8835ffb640 1 -- 172.21.15.133:0/913755497 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f883c00bc80 con 0x7f884006e460 2024-04-25T10:06:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.181+0000 7f8835ffb640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8824066f60 0x7f8824069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.181+0000 7f8835ffb640 1 -- 172.21.15.133:0/913755497 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f883c0856c0 con 0x7f884006e460 2024-04-25T10:06:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.181+0000 7f8844f47640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8824066f60 0x7f8824069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.181+0000 7f8844f47640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8824066f60 0x7f8824069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f88300ae000 tx=0x7f88300a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.185+0000 7f8835ffb640 1 -- 172.21.15.133:0/913755497 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f883c056530 con 0x7f884006e460 2024-04-25T10:06:54.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.377+0000 7f88479ac640 1 -- 172.21.15.133:0/913755497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8840154920 con 0x7f884006e460 2024-04-25T10:06:54.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.377+0000 7f8835ffb640 1 -- 172.21.15.133:0/913755497 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f883c055e30 con 0x7f884006e460 2024-04-25T10:06:54.383 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:54.383 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3081100850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3081100850},{"type":"v1","addr":"172.21.15.162:6834","nonce":3081100850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/272966073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":272966073},{"type":"v1","addr":"172.21.15.162:6835","nonce":272966073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6873,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2840204486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2840204486},{"type":"v1","addr":"172.21.15.133:6835","nonce":2840204486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:36.516727+0000","modified":"2024-04-25T10:06:38.534667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/236214118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":236214118},{"type":"v1","addr":"172.21.15.133:6837","nonce":236214118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:54.384 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 33 2024-04-25T10:06:54.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 -- 172.21.15.133:0/913755497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8824066f60 msgr2=0x7f8824069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8824066f60 0x7f8824069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f88300ae000 tx=0x7f88300a7000 comp rx=0 tx=0).stop 2024-04-25T10:06:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 -- 172.21.15.133:0/913755497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f884006e460 msgr2=0x7f88400c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f884006e460 0x7f88400c9ac0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f883c00d800 tx=0x7f883c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 -- 172.21.15.133:0/913755497 shutdown_connections 2024-04-25T10:06:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f88401584f0 0x7f88400c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8840158110 0x7f8840150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8824066f60 0x7f8824069420 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 --2- 172.21.15.133:0/913755497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f884006e460 0x7f88400c9ac0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 -- 172.21.15.133:0/913755497 >> 172.21.15.133:0/913755497 conn(0x7f8840067000 msgr2=0x7f884006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 -- 172.21.15.133:0/913755497 shutdown_connections 2024-04-25T10:06:54.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.381+0000 7f88479ac640 1 -- 172.21.15.133:0/913755497 wait complete. 2024-04-25T10:06:54.402 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:06:54.535 INFO:teuthology.orchestra.run.smithi133.stdout:{"message":"","return_code":0} 2024-04-25T10:06:54.535 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:06:54.536 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:06:54.536 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:54.536 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:06:54.536 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:54.537 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:06:54.537 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:54.537 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:06:54.537 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:54.549 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:06:54.549 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:54.550 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:06:54.550 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:06:54.551 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:54.551 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:54.551 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:06:54.552 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:54.552 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T10:06:54.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 Processor -- start 2024-04-25T10:06:54.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 -- start start 2024-04-25T10:06:54.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d588da8310 0x55d588e74400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d588dec0d0 0x55d588e7c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588e79110 0x55d588dcfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d588d4b9f0 con 0x55d588e79110 2024-04-25T10:06:54.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d588dec650 con 0x55d588dec0d0 2024-04-25T10:06:54.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d588dcf520 con 0x55d588da8310 2024-04-25T10:06:54.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7eab0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588e79110 0x55d588dcfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7eab0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588e79110 0x55d588dcfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51988/0 (socket says 172.21.15.133:51988) 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7eab0640 1 -- 172.21.15.133:0/1609378048 learned_addr learned my addr 172.21.15.133:0/1609378048 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7e2af640 1 --2- 172.21.15.133:0/1609378048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d588da8310 0x55d588e74400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7daae640 1 --2- 172.21.15.133:0/1609378048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d588dec0d0 0x55d588e7c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7eab0640 1 -- 172.21.15.133:0/1609378048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d588da8310 msgr2=0x55d588e74400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7eab0640 1 --2- 172.21.15.133:0/1609378048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d588da8310 0x55d588e74400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7eab0640 1 -- 172.21.15.133:0/1609378048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d588dec0d0 msgr2=0x55d588e7c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7eab0640 1 --2- 172.21.15.133:0/1609378048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d588dec0d0 0x55d588e7c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7eab0640 1 -- 172.21.15.133:0/1609378048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d588dd0e50 con 0x55d588e79110 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7daae640 1 --2- 172.21.15.133:0/1609378048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d588dec0d0 0x55d588e7c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7eab0640 1 --2- 172.21.15.133:0/1609378048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588e79110 0x55d588dcfe40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800a510 tx=0x7f9c7800a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1428dbacbf86d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7caac640 1 -- 172.21.15.133:0/1609378048 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78015070 con 0x55d588e79110 2024-04-25T10:06:54.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7caac640 1 -- 172.21.15.133:0/1609378048 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7800e710 con 0x55d588e79110 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c7caac640 1 -- 172.21.15.133:0/1609378048 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800eb00 con 0x55d588e79110 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 -- 172.21.15.133:0/1609378048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588e79110 msgr2=0x55d588dcfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 --2- 172.21.15.133:0/1609378048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588e79110 0x55d588dcfe40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800a510 tx=0x7f9c7800a9d0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 -- 172.21.15.133:0/1609378048 shutdown_connections 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 --2- 172.21.15.133:0/1609378048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588e79110 0x55d588dcfe40 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 --2- 172.21.15.133:0/1609378048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d588dec0d0 0x55d588e7c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 --2- 172.21.15.133:0/1609378048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d588da8310 0x55d588e74400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 -- 172.21.15.133:0/1609378048 >> 172.21.15.133:0/1609378048 conn(0x55d588da4a00 msgr2=0x55d588da6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 -- 172.21.15.133:0/1609378048 shutdown_connections 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 -- 172.21.15.133:0/1609378048 wait complete. 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 Processor -- start 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.621+0000 7f9c80791080 1 -- start start 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c80791080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d588da8310 0x55d588e72130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c80791080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588dec0d0 0x55d588dee360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c80791080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d588e79110 0x55d588dee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c80791080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d588dd0bb0 con 0x55d588dec0d0 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c80791080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d588dd0910 con 0x55d588e79110 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c80791080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d588dcf780 con 0x55d588da8310 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7daae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588dec0d0 0x55d588dee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7daae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588dec0d0 0x55d588dee360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51996/0 (socket says 172.21.15.133:51996) 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7daae640 1 -- 172.21.15.133:0/2118380446 learned_addr learned my addr 172.21.15.133:0/2118380446 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7e2af640 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d588da8310 0x55d588e72130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7eab0640 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d588e79110 0x55d588dee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7daae640 1 -- 172.21.15.133:0/2118380446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d588da8310 msgr2=0x55d588e72130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7daae640 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d588da8310 0x55d588e72130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7daae640 1 -- 172.21.15.133:0/2118380446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d588e79110 msgr2=0x55d588dee8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7daae640 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d588e79110 0x55d588dee8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7daae640 1 -- 172.21.15.133:0/2118380446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d588de8cc0 con 0x55d588dec0d0 2024-04-25T10:06:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7daae640 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588dec0d0 0x55d588dee360 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9c740066f0 tx=0x7f9c74006bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c6f7fe640 1 -- 172.21.15.133:0/2118380446 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7400c6c0 con 0x55d588dec0d0 2024-04-25T10:06:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c6f7fe640 1 -- 172.21.15.133:0/2118380446 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7400c860 con 0x55d588dec0d0 2024-04-25T10:06:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c80791080 1 -- 172.21.15.133:0/2118380446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d588de8f60 con 0x55d588dec0d0 2024-04-25T10:06:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c6f7fe640 1 -- 172.21.15.133:0/2118380446 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7400cc50 con 0x55d588dec0d0 2024-04-25T10:06:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c80791080 1 -- 172.21.15.133:0/2118380446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d588de94a0 con 0x55d588dec0d0 2024-04-25T10:06:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c6f7fe640 1 -- 172.21.15.133:0/2118380446 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c74019020 con 0x55d588dec0d0 2024-04-25T10:06:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c6f7fe640 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9c60065970 0x7f9c60067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7e2af640 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9c60065970 0x7f9c60067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c6f7fe640 1 -- 172.21.15.133:0/2118380446 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c74083b70 con 0x55d588dec0d0 2024-04-25T10:06:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c80791080 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x55d588e7d6b0 0x55d588e7da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c80791080 1 -- 172.21.15.133:0/2118380446 --> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] -- osd_op(unknown.0.0:1 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v9 -- 0x55d588e7dfd0 con 0x55d588e7d6b0 2024-04-25T10:06:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7e2af640 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9c60065970 0x7f9c60067e30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9c700ae000 tx=0x7f9c700a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7eab0640 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x55d588e7d6b0 0x55d588e7da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.625+0000 7f9c7eab0640 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x55d588e7d6b0 0x55d588e7da90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c7eab0640 1 -- 172.21.15.133:0/2118380446 <== osd.4 v2:172.21.15.162:6801/3188774481 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v66'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9c7800e070 con 0x55d588e7d6b0 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c80791080 1 -- 172.21.15.133:0/2118380446 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x55d588e7d6b0 msgr2=0x55d588e7da90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c80791080 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x55d588e7d6b0 0x55d588e7da90 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c80791080 1 -- 172.21.15.133:0/2118380446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9c60065970 msgr2=0x7f9c60067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c80791080 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9c60065970 0x7f9c60067e30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9c700ae000 tx=0x7f9c700a6000 comp rx=0 tx=0).stop 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c80791080 1 -- 172.21.15.133:0/2118380446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588dec0d0 msgr2=0x55d588dee360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c80791080 1 --2- 172.21.15.133:0/2118380446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d588dec0d0 0x55d588dee360 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9c740066f0 tx=0x7f9c74006bb0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c7eab0640 1 -- 172.21.15.133:0/2118380446 reap_dead start 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c80791080 1 -- 172.21.15.133:0/2118380446 shutdown_connections 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c80791080 1 -- 172.21.15.133:0/2118380446 >> 172.21.15.133:0/2118380446 conn(0x55d588da4a00 msgr2=0x55d588dcfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c80791080 1 -- 172.21.15.133:0/2118380446 shutdown_connections 2024-04-25T10:06:54.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.629+0000 7f9c80791080 1 -- 172.21.15.133:0/2118380446 wait complete. 2024-04-25T10:06:54.637 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T10:06:54.638 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T10:06:54.638 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T10:06:54.638 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T10:06:54.638 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T10:06:54.639 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T10:06:54.639 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T10:06:54.639 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T10:06:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6dfd530640 1 Processor -- start 2024-04-25T10:06:54.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6dfd530640 1 -- start start 2024-04-25T10:06:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29cda3a640 1 Processor -- start 2024-04-25T10:06:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe84ab3c640 1 Processor -- start 2024-04-25T10:06:54.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29cda3a640 1 -- start start 2024-04-25T10:06:54.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe84ab3c640 1 -- start start 2024-04-25T10:06:54.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c3577640 1 Processor -- start 2024-04-25T10:06:54.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c3577640 1 -- start start 2024-04-25T10:06:54.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29cda3a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c815ed70 0x7f29c80d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29cda3a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f29c80b1400 0x7f29c80ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29cda3a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c80cd380 con 0x7f29c80cea50 2024-04-25T10:06:54.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29cda3a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c80cf3f0 con 0x7f29c80b1400 2024-04-25T10:06:54.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29cda3a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80cd660 con 0x7f29c815ed70 2024-04-25T10:06:54.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe84ab3c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe84415c000 0x7fe84415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe84ab3c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8440caa90 0x7fe84406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe84ab3c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8440cde40 con 0x7fe84415c920 2024-04-25T10:06:54.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe84ab3c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8440ce120 con 0x7fe84415c000 2024-04-25T10:06:54.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe84ab3c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8440ce400 con 0x7fe8440caa90 2024-04-25T10:06:54.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29c77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f29c80b1400 0x7f29c80ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29c77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f29c80b1400 0x7f29c80ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52002/0 (socket says 172.21.15.133:52002) 2024-04-25T10:06:54.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29c77fe640 1 -- 172.21.15.133:0/448558842 learned_addr learned my addr 172.21.15.133:0/448558842 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:54.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6dfd530640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6df815a960 0x7f6df815ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6dfd530640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df80ccd10 0x7f6df80cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe8488d8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe84415c000 0x7fe84415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe8488d8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe84415c000 0x7fe84415c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52018/0 (socket says 172.21.15.133:52018) 2024-04-25T10:06:54.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe8488d8640 1 -- 172.21.15.133:0/3463856646 learned_addr learned my addr 172.21.15.133:0/3463856646 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:54.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6dfd530640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df80d14c0 con 0x7f6df80cc020 2024-04-25T10:06:54.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6dfd530640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df80cc480 con 0x7f6df815a960 2024-04-25T10:06:54.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6dfd530640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80cb260 con 0x7f6df80ccd10 2024-04-25T10:06:54.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df67fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6df80cc020 0x7f6df80d0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58022/0 (socket says 172.21.15.133:58022) 2024-04-25T10:06:54.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df67fc640 1 -- 172.21.15.133:0/1732383437 learned_addr learned my addr 172.21.15.133:0/1732383437 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:54.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29c6ffd640 1 --2- 172.21.15.133:0/448558842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c815ed70 0x7f29c80d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29c77fe640 1 -- 172.21.15.133:0/448558842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c815ed70 msgr2=0x7f29c80d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29c77fe640 1 --2- 172.21.15.133:0/448558842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c815ed70 0x7f29c80d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29c77fe640 1 -- 172.21.15.133:0/448558842 >> v1:172.21.15.162:6789/0 conn(0x7f29c80cea50 legacy=0x7f29c80cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c3577640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc3bc15baa0 0x7fc3bc0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe8490d9640 1 --2- 172.21.15.133:0/3463856646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8440caa90 0x7fe84406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df77fe640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df80ccd10 0x7f6df80cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f29c77fe640 1 -- 172.21.15.133:0/448558842 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c80cd940 con 0x7f29c80b1400 2024-04-25T10:06:54.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c3577640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3bc0d0390 0x7fc3bc0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c3577640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3bc0b1560 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c3577640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3bc0b1840 con 0x7fc3bc15baa0 2024-04-25T10:06:54.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c3577640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0b1b20 con 0x7fc3bc0d0390 2024-04-25T10:06:54.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df6ffd640 1 --2- 172.21.15.133:0/1732383437 >> v2:172.21.15.162:3300/0 conn(0x7f6df815a960 0x7f6df815ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe8488d8640 1 -- 172.21.15.133:0/3463856646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8440caa90 msgr2=0x7fe84406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe8488d8640 1 --2- 172.21.15.133:0/3463856646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8440caa90 0x7fe84406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe8488d8640 1 -- 172.21.15.133:0/3463856646 >> v1:172.21.15.162:6789/0 conn(0x7fe84415c920 legacy=0x7fe8440ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe8488d8640 1 -- 172.21.15.133:0/3463856646 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8440ce6e0 con 0x7fe84415c000 2024-04-25T10:06:54.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fe8490d9640 1 --2- 172.21.15.133:0/3463856646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8440caa90 0x7fe84406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:54.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884566008 0 0) 0x7f6df80d14c0 con 0x7f6df80cc020 2024-04-25T10:06:54.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6de0003530 con 0x7f6df80cc020 2024-04-25T10:06:54.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c1313640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc3bc15baa0 0x7fc3bc0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c1b14640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3bc0d0390 0x7fc3bc0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c1313640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc3bc15baa0 0x7fc3bc0cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:60020/0 (socket says 172.21.15.133:60020) 2024-04-25T10:06:54.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c1313640 1 -- 172.21.15.133:0/923014106 learned_addr learned my addr 172.21.15.133:0/923014106 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:54.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 217430231 0 0) 0x7f6de0003530 con 0x7f6df80cc020 2024-04-25T10:06:54.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df80ccd10 msgr2=0x7f6df80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df80ccd10 0x7f6df80cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 >> v2:172.21.15.162:3300/0 conn(0x7f6df815a960 msgr2=0x7f6df815ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 --2- 172.21.15.133:0/1732383437 >> v2:172.21.15.162:3300/0 conn(0x7f6df815a960 0x7f6df815ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df80cb540 con 0x7f6df80cc020 2024-04-25T10:06:54.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6dec001680 con 0x7f6df80cc020 2024-04-25T10:06:54.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 >> v1:172.21.15.133:6789/0 conn(0x7f6df80cc020 legacy=0x7f6df80d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de00039c0 0x7f6de0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df80ccd10 0x7f6de0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6de0004d50 0x7f6de00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80cc480 con 0x7f6de00039c0 2024-04-25T10:06:54.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80cb260 con 0x7f6de0004d50 2024-04-25T10:06:54.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de000ba00 con 0x7f6df80ccd10 2024-04-25T10:06:54.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df6ffd640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de00039c0 0x7f6de0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c1b14640 1 -- 172.21.15.133:0/923014106 >> v2:172.21.15.162:3300/0 conn(0x7fc3bc15baa0 msgr2=0x7fc3bc0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c1b14640 1 --2- 172.21.15.133:0/923014106 >> v2:172.21.15.162:3300/0 conn(0x7fc3bc15baa0 0x7fc3bc0cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c1b14640 1 -- 172.21.15.133:0/923014106 >> v1:172.21.15.133:6789/0 conn(0x7fc3bc0cf7b0 legacy=0x7fc3bc0cfc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:06:54.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7fc3c1b14640 1 -- 172.21.15.133:0/923014106 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc0b1e00 con 0x7fc3bc0d0390 2024-04-25T10:06:54.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df77fe640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6de0004d50 0x7f6de00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df6ffd640 1 -- 172.21.15.133:0/1732383437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df80ccd10 msgr2=0x7f6de0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df6ffd640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df80ccd10 0x7f6de0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df6ffd640 1 -- 172.21.15.133:0/1732383437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6de0004d50 msgr2=0x7f6de00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df6ffd640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6de0004d50 0x7f6de00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df6ffd640 1 -- 172.21.15.133:0/1732383437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de0003530 con 0x7f6de00039c0 2024-04-25T10:06:54.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.785+0000 7f6df77fe640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6de0004d50 0x7f6de00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:54.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe8488d8640 1 --2- 172.21.15.133:0/3463856646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe84415c000 0x7fe84415c3e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009e50 tx=0x7fe83c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc844e5cb285dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe8437fe640 1 -- 172.21.15.133:0/3463856646 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c0a57e0 con 0x7fe84415c000 2024-04-25T10:06:54.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe8437fe640 1 -- 172.21.15.133:0/3463856646 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c0a5980 con 0x7fe84415c000 2024-04-25T10:06:54.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe8437fe640 1 -- 172.21.15.133:0/3463856646 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c0a5d50 con 0x7fe84415c000 2024-04-25T10:06:54.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29c77fe640 1 --2- 172.21.15.133:0/448558842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c80b1400 0x7f29c80ceeb0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009e50 tx=0x7f29c001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaab1a8f8439321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29c5ffb640 1 -- 172.21.15.133:0/448558842 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c00a5650 con 0x7f29c80b1400 2024-04-25T10:06:54.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3c1b14640 1 --2- 172.21.15.133:0/923014106 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3bc0d0390 0x7fc3bc0cc3b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4009d90 tx=0x7fc3b401c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a7bf93ad86101f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29c5ffb640 1 -- 172.21.15.133:0/448558842 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c00a57f0 con 0x7f29c80b1400 2024-04-25T10:06:54.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29c5ffb640 1 -- 172.21.15.133:0/448558842 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c00a5bc0 con 0x7f29c80b1400 2024-04-25T10:06:54.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe84ab3c640 1 -- 172.21.15.133:0/3463856646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe84415c000 msgr2=0x7fe84415c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe84ab3c640 1 --2- 172.21.15.133:0/3463856646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe84415c000 0x7fe84415c3e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009e50 tx=0x7fe83c01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3b3fff640 1 -- 172.21.15.133:0/923014106 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40b2070 con 0x7fc3bc0d0390 2024-04-25T10:06:54.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3b3fff640 1 -- 172.21.15.133:0/923014106 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b40a57a0 con 0x7fc3bc0d0390 2024-04-25T10:06:54.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3b3fff640 1 -- 172.21.15.133:0/923014106 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40a5b70 con 0x7fc3bc0d0390 2024-04-25T10:06:54.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f6df6ffd640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de00039c0 0x7f6de0003de0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6de800de10 tx=0x7f6de8020b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64c274dca2360cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29cda3a640 1 -- 172.21.15.133:0/448558842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c80b1400 msgr2=0x7f29c80ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29cda3a640 1 --2- 172.21.15.133:0/448558842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c80b1400 0x7f29c80ceeb0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009e50 tx=0x7f29c001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe84ab3c640 1 -- 172.21.15.133:0/3463856646 shutdown_connections 2024-04-25T10:06:54.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe84ab3c640 1 --2- 172.21.15.133:0/3463856646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8440caa90 0x7fe84406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe84ab3c640 1 --2- 172.21.15.133:0/3463856646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe84415c000 0x7fe84415c3e0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe84ab3c640 1 -- 172.21.15.133:0/3463856646 >> 172.21.15.133:0/3463856646 conn(0x7fe8440cc7b0 msgr2=0x7fe8440ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe84ab3c640 1 -- 172.21.15.133:0/3463856646 shutdown_connections 2024-04-25T10:06:54.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fe84ab3c640 1 -- 172.21.15.133:0/3463856646 wait complete. 2024-04-25T10:06:54.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29cda3a640 1 -- 172.21.15.133:0/448558842 shutdown_connections 2024-04-25T10:06:54.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29cda3a640 1 --2- 172.21.15.133:0/448558842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c80b1400 0x7f29c80ceeb0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29cda3a640 1 --2- 172.21.15.133:0/448558842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c815ed70 0x7f29c80d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29cda3a640 1 -- 172.21.15.133:0/448558842 >> 172.21.15.133:0/448558842 conn(0x7f29c80999d0 msgr2=0x7f29c80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29cda3a640 1 -- 172.21.15.133:0/448558842 shutdown_connections 2024-04-25T10:06:54.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f29cda3a640 1 -- 172.21.15.133:0/448558842 wait complete. 2024-04-25T10:06:54.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3c3577640 1 -- 172.21.15.133:0/923014106 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3bc0d0390 msgr2=0x7fc3bc0cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3c3577640 1 --2- 172.21.15.133:0/923014106 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3bc0d0390 0x7fc3bc0cc3b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4009d90 tx=0x7fc3b401c810 comp rx=0 tx=0).stop 2024-04-25T10:06:54.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800da60 con 0x7f6de00039c0 2024-04-25T10:06:54.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de80a97e0 con 0x7f6de00039c0 2024-04-25T10:06:54.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7f6df5ffb640 1 -- 172.21.15.133:0/1732383437 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800d2d0 con 0x7f6de00039c0 2024-04-25T10:06:54.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3c3577640 1 -- 172.21.15.133:0/923014106 shutdown_connections 2024-04-25T10:06:54.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3c3577640 1 --2- 172.21.15.133:0/923014106 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3bc0d0390 0x7fc3bc0cc3b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3c3577640 1 --2- 172.21.15.133:0/923014106 >> v2:172.21.15.162:3300/0 conn(0x7fc3bc15baa0 0x7fc3bc0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3c3577640 1 -- 172.21.15.133:0/923014106 >> 172.21.15.133:0/923014106 conn(0x7fc3bc0d16a0 msgr2=0x7fc3bc0d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3c3577640 1 -- 172.21.15.133:0/923014106 shutdown_connections 2024-04-25T10:06:54.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.789+0000 7fc3c3577640 1 -- 172.21.15.133:0/923014106 wait complete. 2024-04-25T10:06:54.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe84ab3c640 1 Processor -- start 2024-04-25T10:06:54.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29cda3a640 1 Processor -- start 2024-04-25T10:06:54.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29cda3a640 1 -- start start 2024-04-25T10:06:54.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c3577640 1 Processor -- start 2024-04-25T10:06:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f6dfd530640 1 -- 172.21.15.133:0/1732383437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de00039c0 msgr2=0x7f6de0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f6dfd530640 1 --2- 172.21.15.133:0/1732383437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de00039c0 0x7f6de0003de0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6de800de10 tx=0x7f6de8020b00 comp rx=0 tx=0).stop 2024-04-25T10:06:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f6df77fe640 1 -- 172.21.15.133:0/1732383437 reap_dead start 2024-04-25T10:06:54.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c3577640 1 -- start start 2024-04-25T10:06:54.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29cda3a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c80b1400 0x7f29c80adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29cda3a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c80cea50 0x7f29c80ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe84ab3c640 1 -- start start 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe84ab3c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8440caa90 0x7fe8441559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe84ab3c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe84415c000 0x7fe84406ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe84ab3c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe84415c920 0x7fe84406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29cda3a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c815ed70 0x7f29c80c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe84ab3c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84406bb60 con 0x7fe84415c000 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe84ab3c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84406b9e0 con 0x7fe84415c920 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe84ab3c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84406bce0 con 0x7fe8440caa90 2024-04-25T10:06:54.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29cda3a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c809a0c0 con 0x7f29c815ed70 2024-04-25T10:06:54.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29cda3a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c8099f40 con 0x7f29c80b1400 2024-04-25T10:06:54.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29cda3a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c815f150 con 0x7f29c80cea50 2024-04-25T10:06:54.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe843fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe84415c000 0x7fe84406ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe843fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe84415c000 0x7fe84406ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52032/0 (socket says 172.21.15.133:52032) 2024-04-25T10:06:54.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe843fff640 1 -- 172.21.15.133:0/3755948692 learned_addr learned my addr 172.21.15.133:0/3755948692 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:54.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29c77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c815ed70 0x7f29c80c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29c77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c815ed70 0x7f29c80c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52042/0 (socket says 172.21.15.133:52042) 2024-04-25T10:06:54.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29c77fe640 1 -- 172.21.15.133:0/1764437764 learned_addr learned my addr 172.21.15.133:0/1764437764 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:54.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe8488d8640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8440caa90 0x7fe8441559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29c67fc640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c80cea50 0x7f29c80ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29c6ffd640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c80b1400 0x7f29c80adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe843fff640 1 -- 172.21.15.133:0/3755948692 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8440caa90 msgr2=0x7fe8441559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe843fff640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8440caa90 0x7fe8441559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe843fff640 1 -- 172.21.15.133:0/3755948692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe84415c920 msgr2=0x7fe84406d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe8490d9640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe84415c920 0x7fe84406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe843fff640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe84415c920 0x7fe84406d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe843fff640 1 -- 172.21.15.133:0/3755948692 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84406e680 con 0x7fe84415c000 2024-04-25T10:06:54.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe8490d9640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe84415c920 0x7fe84406d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:54.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c3577640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc0cf7b0 0x7fc3bc0c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c3577640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3bc0d0390 0x7fc3bc0c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c3577640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3bc15baa0 0x7fc3bc0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c3577640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0cd510 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c3577640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0cd390 con 0x7fc3bc15baa0 2024-04-25T10:06:54.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c3577640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0cd690 con 0x7fc3bc0d0390 2024-04-25T10:06:54.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f6dfd530640 1 -- 172.21.15.133:0/1732383437 shutdown_connections 2024-04-25T10:06:54.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f6dfd530640 1 -- 172.21.15.133:0/1732383437 >> 172.21.15.133:0/1732383437 conn(0x7f6df80999d0 msgr2=0x7f6df80d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29c77fe640 1 -- 172.21.15.133:0/1764437764 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c80cea50 msgr2=0x7f29c80ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29c77fe640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c80cea50 0x7f29c80ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29c77fe640 1 -- 172.21.15.133:0/1764437764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c80b1400 msgr2=0x7f29c80adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29c77fe640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c80b1400 0x7f29c80adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29c77fe640 1 -- 172.21.15.133:0/1764437764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c80ae610 con 0x7f29c815ed70 2024-04-25T10:06:54.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c1313640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc0cf7b0 0x7fc3bc0c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c1313640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc0cf7b0 0x7fc3bc0c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52050/0 (socket says 172.21.15.133:52050) 2024-04-25T10:06:54.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c1313640 1 -- 172.21.15.133:0/3099849512 learned_addr learned my addr 172.21.15.133:0/3099849512 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:54.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe843fff640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe84415c000 0x7fe84406ef20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe83800b440 tx=0x7fe83800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c0b12640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3bc0d0390 0x7fc3bc0c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f6dfd530640 1 -- 172.21.15.133:0/1732383437 shutdown_connections 2024-04-25T10:06:54.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c1b14640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3bc15baa0 0x7fc3bc0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe8417fa640 1 -- 172.21.15.133:0/3755948692 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838015020 con 0x7fe84415c000 2024-04-25T10:06:54.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f29c77fe640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c815ed70 0x7f29c80c6990 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f29c00af040 tx=0x7f29c0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe8417fa640 1 -- 172.21.15.133:0/3755948692 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838011890 con 0x7fe84415c000 2024-04-25T10:06:54.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe84ab3c640 1 -- 172.21.15.133:0/3755948692 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84406e3a0 con 0x7fe84415c000 2024-04-25T10:06:54.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe8417fa640 1 -- 172.21.15.133:0/3755948692 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838011c80 con 0x7fe84415c000 2024-04-25T10:06:54.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fe84ab3c640 1 -- 172.21.15.133:0/3755948692 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8441551d0 con 0x7fe84415c000 2024-04-25T10:06:54.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c1313640 1 -- 172.21.15.133:0/3099849512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3bc0d0390 msgr2=0x7fc3bc0c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c1313640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3bc0d0390 0x7fc3bc0c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c1313640 1 -- 172.21.15.133:0/3099849512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3bc15baa0 msgr2=0x7fc3bc0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c1313640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3bc15baa0 0x7fc3bc0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c1313640 1 -- 172.21.15.133:0/3099849512 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc0c8020 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7f6dfd530640 1 -- 172.21.15.133:0/1732383437 wait complete. 2024-04-25T10:06:54.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c1313640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc0cf7b0 0x7fc3bc0c7ae0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800e890 tx=0x7fc3b800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3b1ffb640 1 -- 172.21.15.133:0/3099849512 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8017070 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3b1ffb640 1 -- 172.21.15.133:0/3099849512 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b800c940 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3b1ffb640 1 -- 172.21.15.133:0/3099849512 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b800cd30 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c3577640 1 -- 172.21.15.133:0/3099849512 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3bc0ca330 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.793+0000 7fc3c3577640 1 -- 172.21.15.133:0/3099849512 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3bc0ca8f0 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f29affff640 1 -- 172.21.15.133:0/1764437764 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c0009510 con 0x7f29c815ed70 2024-04-25T10:06:54.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7fc3c3577640 1 -- 172.21.15.133:0/3099849512 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc39c0052a0 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7fc3b1ffb640 1 -- 172.21.15.133:0/3099849512 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3b801a070 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6dfd530640 1 Processor -- start 2024-04-25T10:06:54.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7fe82affd640 1 -- 172.21.15.133:0/3755948692 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c0052a0 con 0x7fe84415c000 2024-04-25T10:06:54.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7fe8417fa640 1 -- 172.21.15.133:0/3755948692 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe838018020 con 0x7fe84415c000 2024-04-25T10:06:54.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6dfd530640 1 -- start start 2024-04-25T10:06:54.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f29affff640 1 -- 172.21.15.133:0/1764437764 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c00be070 con 0x7f29c815ed70 2024-04-25T10:06:54.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f29affff640 1 -- 172.21.15.133:0/1764437764 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c00b88b0 con 0x7f29c815ed70 2024-04-25T10:06:54.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f29cda3a640 1 -- 172.21.15.133:0/1764437764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c80ae8f0 con 0x7f29c815ed70 2024-04-25T10:06:54.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f29cda3a640 1 -- 172.21.15.133:0/1764437764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c80aeeb0 con 0x7f29c815ed70 2024-04-25T10:06:54.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f29cda3a640 1 -- 172.21.15.133:0/1764437764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c8067c70 con 0x7f29c815ed70 2024-04-25T10:06:54.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f29affff640 1 -- 172.21.15.133:0/1764437764 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29c00b8a50 con 0x7f29c815ed70 2024-04-25T10:06:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6dfd530640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6de00039c0 0x7f6df814d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6dfd530640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6df80c4fe0 0x7f6df815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6dfd530640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df815b310 0x7f6df8152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6dfd530640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80b3d40 con 0x7f6df80c4fe0 2024-04-25T10:06:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6dfd530640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80b3bc0 con 0x7f6de00039c0 2024-04-25T10:06:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6dfd530640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80b3ec0 con 0x7f6df815b310 2024-04-25T10:06:54.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7fc3b1ffb640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc3a0066f60 0x7fc3a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7fc3b1ffb640 1 -- 172.21.15.133:0/3099849512 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc3b80850f0 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6df77fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df815b310 0x7f6df8152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6df77fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df815b310 0x7f6df8152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:40642/0 (socket says 172.21.15.133:40642) 2024-04-25T10:06:54.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6df6ffd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6de00039c0 0x7f6df814d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6df77fe640 1 -- 172.21.15.133:0/1601258761 learned_addr learned my addr 172.21.15.133:0/1601258761 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:54.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f29affff640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29b4066fe0 0x7f29b40694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f29affff640 1 -- 172.21.15.133:0/1764437764 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29c011fd80 con 0x7f29c815ed70 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6df77fe640 1 -- 172.21.15.133:0/1601258761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6de00039c0 msgr2=0x7f6df814d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6df77fe640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6de00039c0 0x7f6df814d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6df77fe640 1 -- 172.21.15.133:0/1601258761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6df80c4fe0 msgr2=0x7f6df815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6df77fe640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6df80c4fe0 0x7f6df815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6df77fe640 1 -- 172.21.15.133:0/1601258761 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df814d880 con 0x7f6df815b310 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6df77fe640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df815b310 0x7f6df8152ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6df80c3f50 tx=0x7f6df00075c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6ddf7fe640 1 -- 172.21.15.133:0/1601258761 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0007e60 con 0x7f6df815b310 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6dfd530640 1 -- 172.21.15.133:0/1601258761 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df81522f0 con 0x7f6df815b310 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6ddf7fe640 1 -- 172.21.15.133:0/1601258761 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df0019070 con 0x7f6df815b310 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6ddf7fe640 1 -- 172.21.15.133:0/1601258761 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0009540 con 0x7f6df815b310 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7f6dfd530640 1 -- 172.21.15.133:0/1601258761 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df80c88a0 con 0x7f6df815b310 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.801+0000 7f6dfd530640 1 -- 172.21.15.133:0/1601258761 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df8067c70 con 0x7f6df815b310 2024-04-25T10:06:54.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.801+0000 7f6ddf7fe640 1 -- 172.21.15.133:0/1601258761 <== mon.2 v2:172.21.15.162:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6df00096e0 con 0x7f6df815b310 2024-04-25T10:06:54.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.801+0000 7f6ddf7fe640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6de4066f60 0x7f6de4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.801+0000 7f6ddf7fe640 1 -- 172.21.15.133:0/1601258761 <== mon.2 v2:172.21.15.162:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6df0086260 con 0x7f6df815b310 2024-04-25T10:06:54.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.805+0000 7fc3c0b12640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc3a0066f60 0x7fc3a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7fe8417fa640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe830066fb0 0x7fe830069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.797+0000 7fe8417fa640 1 -- 172.21.15.133:0/3755948692 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe838084fc0 con 0x7fe84415c000 2024-04-25T10:06:54.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.805+0000 7fe8488d8640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe830066fb0 0x7fe830069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.805+0000 7f29c6ffd640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29b4066fe0 0x7f29b40694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.805+0000 7f6df6ffd640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6de4066f60 0x7f6de4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.805+0000 7fe8417fa640 1 -- 172.21.15.133:0/3755948692 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe83801f030 con 0x7fe84415c000 2024-04-25T10:06:54.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.805+0000 7fc3c0b12640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc3a0066f60 0x7fc3a0069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc1fdce0 tx=0x7fc3ac009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.805+0000 7f29c6ffd640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29b4066fe0 0x7f29b40694a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f29b80079c0 tx=0x7f29b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.805+0000 7f6df6ffd640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6de4066f60 0x7f6de4069420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6de80b2000 tx=0x7f6de80ab040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.805+0000 7fe8488d8640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe830066fb0 0x7fe830069470 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0ae000 tx=0x7fe83c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.805+0000 7f29affff640 1 -- 172.21.15.133:0/1764437764 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29c00165e0 con 0x7f29c815ed70 2024-04-25T10:06:54.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.813+0000 7fc3b1ffb640 1 -- 172.21.15.133:0/3099849512 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3b8055880 con 0x7fc3bc0cf7b0 2024-04-25T10:06:54.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:54.813+0000 7f6ddf7fe640 1 -- 172.21.15.133:0/1601258761 <== mon.2 v2:172.21.15.162:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6df814d880 con 0x7f6df815b310 2024-04-25T10:06:55.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.021+0000 7fe82affd640 1 -- 172.21.15.133:0/3755948692 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7fe81c005590 con 0x7fe84415c000 2024-04-25T10:06:55.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.025+0000 7fc3c3577640 1 -- 172.21.15.133:0/3099849512 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7fc39c005590 con 0x7fc3bc0cf7b0 2024-04-25T10:06:55.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.093+0000 7f6dfd530640 1 -- 172.21.15.133:0/1601258761 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f6df8099f10 con 0x7f6df815b310 2024-04-25T10:06:55.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.093+0000 7f29cda3a640 1 -- 172.21.15.133:0/1764437764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f29c8163e60 con 0x7f29c815ed70 2024-04-25T10:06:55.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.893+0000 7fe8417fa640 1 -- 172.21.15.133:0/3755948692 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6873 v34) v1 ==== 95+0+0 (secure 0 0 0) 0x7fe83800d040 con 0x7fe84415c000 2024-04-25T10:06:55.900 INFO:teuthology.orchestra.run.smithi133.stderr:failed mds gid 6873 2024-04-25T10:06:55.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 -- 172.21.15.133:0/3755948692 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe830066fb0 msgr2=0x7fe830069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe830066fb0 0x7fe830069470 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0ae000 tx=0x7fe83c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:06:55.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 -- 172.21.15.133:0/3755948692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe84415c000 msgr2=0x7fe84406ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe84415c000 0x7fe84406ef20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe83800b440 tx=0x7fe83800b900 comp rx=0 tx=0).stop 2024-04-25T10:06:55.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 -- 172.21.15.133:0/3755948692 shutdown_connections 2024-04-25T10:06:55.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe84415c920 0x7fe84406d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe830066fb0 0x7fe830069470 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe84415c000 0x7fe84406ef20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 --2- 172.21.15.133:0/3755948692 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe8440caa90 0x7fe8441559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 -- 172.21.15.133:0/3755948692 >> 172.21.15.133:0/3755948692 conn(0x7fe8440cc7b0 msgr2=0x7fe8440b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.904 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fdecf878640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:06:55.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 -- 172.21.15.133:0/3755948692 shutdown_connections 2024-04-25T10:06:55.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.897+0000 7fe82affd640 1 -- 172.21.15.133:0/3755948692 wait complete. 2024-04-25T10:06:55.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.901+0000 7fc3b1ffb640 1 -- 172.21.15.133:0/3099849512 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6876 v35) v1 ==== 95+0+0 (secure 0 0 0) 0x7fc3b8059920 con 0x7fc3bc0cf7b0 2024-04-25T10:06:55.906 INFO:teuthology.orchestra.run.smithi133.stderr:failed mds gid 6876 2024-04-25T10:06:55.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 -- 172.21.15.133:0/3099849512 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc3a0066f60 msgr2=0x7fc3a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc3a0066f60 0x7fc3a0069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc1fdce0 tx=0x7fc3ac009060 comp rx=0 tx=0).stop 2024-04-25T10:06:55.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 -- 172.21.15.133:0/3099849512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc0cf7b0 msgr2=0x7fc3bc0c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc0cf7b0 0x7fc3bc0c7ae0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800e890 tx=0x7fc3b800ed50 comp rx=0 tx=0).stop 2024-04-25T10:06:55.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 -- 172.21.15.133:0/3099849512 shutdown_connections 2024-04-25T10:06:55.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3bc15baa0 0x7fc3bc0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3bc0d0390 0x7fc3bc0c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc3a0066f60 0x7fc3a0069420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 --2- 172.21.15.133:0/3099849512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc0cf7b0 0x7fc3bc0c7ae0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 -- 172.21.15.133:0/3099849512 >> 172.21.15.133:0/3099849512 conn(0x7fc3bc0d16a0 msgr2=0x7fc3bc06fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 -- 172.21.15.133:0/3099849512 shutdown_connections 2024-04-25T10:06:55.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.905+0000 7fc3c3577640 1 -- 172.21.15.133:0/3099849512 wait complete. 2024-04-25T10:06:55.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6ddf7fe640 1 -- 172.21.15.133:0/1601258761 <== mon.2 v2:172.21.15.162:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5216 v36) v1 ==== 95+0+0 (secure 0 0 0) 0x7f6df00570e0 con 0x7f6df815b310 2024-04-25T10:06:55.930 INFO:teuthology.orchestra.run.smithi133.stderr:failed mds gid 5216 2024-04-25T10:06:55.931 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:06:55.931 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 -- 172.21.15.133:0/1601258761 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6de4066f60 msgr2=0x7f6de4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6de4066f60 0x7f6de4069420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6de80b2000 tx=0x7f6de80ab040 comp rx=0 tx=0).stop 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 -- 172.21.15.133:0/1601258761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df815b310 msgr2=0x7f6df8152ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df815b310 0x7f6df8152ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6df80c3f50 tx=0x7f6df00075c0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 -- 172.21.15.133:0/1601258761 shutdown_connections 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6df815b310 0x7f6df8152ad0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6de4066f60 0x7f6de4069420 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6df80c4fe0 0x7f6df815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 --2- 172.21.15.133:0/1601258761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6de00039c0 0x7f6df814d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 -- 172.21.15.133:0/1601258761 >> 172.21.15.133:0/1601258761 conn(0x7f6df80999d0 msgr2=0x7f6df806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 -- 172.21.15.133:0/1601258761 shutdown_connections 2024-04-25T10:06:55.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:55.925+0000 7f6dfd530640 1 -- 172.21.15.133:0/1601258761 wait complete. 2024-04-25T10:06:55.934 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:06:55.935 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:55.935 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:06:55.936 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:06:55.950 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:06:55.950 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:06:55.953 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:06:55.953 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:55.993 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:06:55.994 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:06:55.994 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:56.028 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:06:56.049 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:06:56.895 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:06:56.889+0000 7fdecf878640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:06:56.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.893+0000 7f29affff640 1 -- 172.21.15.133:0/1764437764 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5545 v37) v1 ==== 95+0+0 (secure 0 0 0) 0x7f29c0121020 con 0x7f29c815ed70 2024-04-25T10:06:56.898 INFO:teuthology.orchestra.run.smithi133.stderr:failed mds gid 5545 2024-04-25T10:06:56.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 -- 172.21.15.133:0/1764437764 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29b4066fe0 msgr2=0x7f29b40694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29b4066fe0 0x7f29b40694a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f29b80079c0 tx=0x7f29b8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:56.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 -- 172.21.15.133:0/1764437764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c815ed70 msgr2=0x7f29c80c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c815ed70 0x7f29c80c6990 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f29c00af040 tx=0x7f29c0019790 comp rx=0 tx=0).stop 2024-04-25T10:06:56.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 -- 172.21.15.133:0/1764437764 shutdown_connections 2024-04-25T10:06:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29b4066fe0 0x7f29b40694a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c815ed70 0x7f29c80c6990 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f29c80cea50 0x7f29c80ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 --2- 172.21.15.133:0/1764437764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f29c80b1400 0x7f29c80adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 -- 172.21.15.133:0/1764437764 >> 172.21.15.133:0/1764437764 conn(0x7f29c80999d0 msgr2=0x7f29c80b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 -- 172.21.15.133:0/1764437764 shutdown_connections 2024-04-25T10:06:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:56.897+0000 7f29cda3a640 1 -- 172.21.15.133:0/1764437764 wait complete. 2024-04-25T10:06:56.926 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:06:56.926 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:06:56.929 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:06:56.929 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:56.930 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:57.000 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:06:57.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e6abd640 1 Processor -- start 2024-04-25T10:06:57.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e6abd640 1 -- start start 2024-04-25T10:06:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e6abd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f49e015d9b0 0x7f49e00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e6abd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e00ca9b0 con 0x7f49e00cee80 2024-04-25T10:06:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e6abd640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e00cac90 con 0x7f49e015dd90 2024-04-25T10:06:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e6abd640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e00cbe30 con 0x7f49e015d9b0 2024-04-25T10:06:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e4859640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f49e015d9b0 0x7f49e00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e4859640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f49e015d9b0 0x7f49e00cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42118/0 (socket says 172.21.15.133:42118) 2024-04-25T10:06:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e4859640 1 -- 172.21.15.133:0/1953149161 learned_addr learned my addr 172.21.15.133:0/1953149161 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e505a640 1 --1- 172.21.15.133:0/1953149161 >> v1:172.21.15.133:6789/0 conn(0x7f49e00cee80 0x7f49e006d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53050/0 (socket says 172.21.15.133:53050) 2024-04-25T10:06:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e4859640 1 -- 172.21.15.133:0/1953149161 >> v1:172.21.15.162:6789/0 conn(0x7f49e015dd90 legacy=0x7f49e00ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e4859640 1 -- 172.21.15.133:0/1953149161 >> v1:172.21.15.133:6789/0 conn(0x7f49e00cee80 legacy=0x7f49e006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.085+0000 7f49e4859640 1 -- 172.21.15.133:0/1953149161 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e00caf70 con 0x7f49e015d9b0 2024-04-25T10:06:57.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e4859640 1 --2- 172.21.15.133:0/1953149161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49e015d9b0 0x7f49e00cb5e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009870 tx=0x7f49d001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdfdb2f85e06227 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49df7fe640 1 -- 172.21.15.133:0/1953149161 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d00ab650 con 0x7f49e015d9b0 2024-04-25T10:06:57.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49df7fe640 1 -- 172.21.15.133:0/1953149161 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d00b0070 con 0x7f49e015d9b0 2024-04-25T10:06:57.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49df7fe640 1 -- 172.21.15.133:0/1953149161 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d00aba10 con 0x7f49e015d9b0 2024-04-25T10:06:57.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 -- 172.21.15.133:0/1953149161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49e015d9b0 msgr2=0x7f49e00cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 --2- 172.21.15.133:0/1953149161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49e015d9b0 0x7f49e00cb5e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009870 tx=0x7f49d001c430 comp rx=0 tx=0).stop 2024-04-25T10:06:57.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 -- 172.21.15.133:0/1953149161 shutdown_connections 2024-04-25T10:06:57.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 --2- 172.21.15.133:0/1953149161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49e015d9b0 0x7f49e00cb5e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 -- 172.21.15.133:0/1953149161 >> 172.21.15.133:0/1953149161 conn(0x7f49e0067000 msgr2=0x7f49e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 -- 172.21.15.133:0/1953149161 shutdown_connections 2024-04-25T10:06:57.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 -- 172.21.15.133:0/1953149161 wait complete. 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 Processor -- start 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 -- start start 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e00cee80 0x7f49e015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e015d9b0 0x7f49e01577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49e015dd90 0x7f49e015fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e006ea20 con 0x7f49e015dd90 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e006e8a0 con 0x7f49e00cee80 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e6abd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e016b870 con 0x7f49e015d9b0 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e505a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49e015dd90 0x7f49e015fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e505a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49e015dd90 0x7f49e015fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42134/0 (socket says 172.21.15.133:42134) 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e505a640 1 -- 172.21.15.133:0/3638549526 learned_addr learned my addr 172.21.15.133:0/3638549526 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:57.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e4859640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e00cee80 0x7f49e015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49dffff640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e015d9b0 0x7f49e01577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e505a640 1 -- 172.21.15.133:0/3638549526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e015d9b0 msgr2=0x7f49e01577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e505a640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e015d9b0 0x7f49e01577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e505a640 1 -- 172.21.15.133:0/3638549526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e00cee80 msgr2=0x7f49e015c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e505a640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e00cee80 0x7f49e015c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.089+0000 7f49e505a640 1 -- 172.21.15.133:0/3638549526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0157cf0 con 0x7f49e015dd90 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49e505a640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49e015dd90 0x7f49e015fec0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c7a0 tx=0x7f49d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49dd7fa640 1 -- 172.21.15.133:0/3638549526 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8017070 con 0x7f49e015dd90 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49dd7fa640 1 -- 172.21.15.133:0/3638549526 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d800a5e0 con 0x7f49e015dd90 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49e6abd640 1 -- 172.21.15.133:0/3638549526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e0157f70 con 0x7f49e015dd90 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49dd7fa640 1 -- 172.21.15.133:0/3638549526 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d800a9d0 con 0x7f49e015dd90 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49e6abd640 1 -- 172.21.15.133:0/3638549526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e0158810 con 0x7f49e015dd90 2024-04-25T10:06:57.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49e6abd640 1 -- 172.21.15.133:0/3638549526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e0158360 con 0x7f49e015dd90 2024-04-25T10:06:57.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49dd7fa640 1 -- 172.21.15.133:0/3638549526 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49d801a070 con 0x7f49e015dd90 2024-04-25T10:06:57.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49dd7fa640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f49cc066f60 0x7f49cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49dd7fa640 1 -- 172.21.15.133:0/3638549526 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f49d8084860 con 0x7f49e015dd90 2024-04-25T10:06:57.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49e4859640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f49cc066f60 0x7f49cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.093+0000 7f49e4859640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f49cc066f60 0x7f49cc069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f49d00af000 tx=0x7f49d00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.097+0000 7f49dd7fa640 1 -- 172.21.15.133:0/3638549526 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49e0157cf0 con 0x7f49e015dd90 2024-04-25T10:06:57.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.277+0000 7f49e6abd640 1 -- 172.21.15.133:0/3638549526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49e00cce70 con 0x7f49e015dd90 2024-04-25T10:06:57.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.277+0000 7f49dd7fa640 1 -- 172.21.15.133:0/3638549526 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4152 (secure 0 0 0) 0x7f49d8055630 con 0x7f49e015dd90 2024-04-25T10:06:57.284 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:57.285 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/232744252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":232744252},{"type":"v1","addr":"172.21.15.162:6833","nonce":232744252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7149,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/912719603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":912719603},{"type":"v1","addr":"172.21.15.133:6835","nonce":912719603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:36.516727+0000","modified":"2024-04-25T10:06:56.900621+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.133:6837/2310040121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2310040121},{"type":"v1","addr":"172.21.15.133:6837","nonce":2310040121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:57.285 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 38 2024-04-25T10:06:57.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 -- 172.21.15.133:0/3638549526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f49cc066f60 msgr2=0x7f49cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f49cc066f60 0x7f49cc069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f49d00af000 tx=0x7f49d00a7000 comp rx=0 tx=0).stop 2024-04-25T10:06:57.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 -- 172.21.15.133:0/3638549526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49e015dd90 msgr2=0x7f49e015fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49e015dd90 0x7f49e015fec0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c7a0 tx=0x7f49d800cc60 comp rx=0 tx=0).stop 2024-04-25T10:06:57.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 -- 172.21.15.133:0/3638549526 shutdown_connections 2024-04-25T10:06:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f49cc066f60 0x7f49cc069420 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49e015dd90 0x7f49e015fec0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49e015d9b0 0x7f49e01577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 --2- 172.21.15.133:0/3638549526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49e00cee80 0x7f49e015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 -- 172.21.15.133:0/3638549526 >> 172.21.15.133:0/3638549526 conn(0x7f49e0067000 msgr2=0x7f49e00b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 -- 172.21.15.133:0/3638549526 shutdown_connections 2024-04-25T10:06:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:57.281+0000 7f49e6abd640 1 -- 172.21.15.133:0/3638549526 wait complete. 2024-04-25T10:06:57.705 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:36.516727+0000', 'modified': '2024-04-25T10:06:56.900621+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7167}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7167': {'gid': 7167, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.133:6837/2310040121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 2310040121}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 2310040121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:06:57.705 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7167:up:replay 2024-04-25T10:06:58.707 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:58.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5cb2e6640 1 Processor -- start 2024-04-25T10:06:58.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5cb2e6640 1 -- start start 2024-04-25T10:06:58.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5cb2e6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c40cc860 0x7fe5c40cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5cb2e6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe5c4162090 0x7fe5c40cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5cb2e6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c406e640 con 0x7fe5c4161cb0 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5cb2e6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c406ec00 con 0x7fe5c4162090 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5cb2e6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c406e920 con 0x7fe5c40cc860 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c9082640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe5c4161cb0 0x7fe5c406dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53066/0 (socket says 172.21.15.133:53066) 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c9082640 1 -- 172.21.15.133:0/483708624 learned_addr learned my addr 172.21.15.133:0/483708624 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c9883640 1 --2- 172.21.15.133:0/483708624 >> v2:172.21.15.162:3300/0 conn(0x7fe5c4162090 0x7fe5c40cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c8881640 1 --2- 172.21.15.133:0/483708624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c40cc860 0x7fe5c40cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833435880 0 0) 0x7fe5c406e640 con 0x7fe5c4161cb0 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5ac003530 con 0x7fe5c4161cb0 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1536333138 0 0) 0x7fe5ac003530 con 0x7fe5c4161cb0 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c40cc860 msgr2=0x7fe5c40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 --2- 172.21.15.133:0/483708624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c40cc860 0x7fe5c40cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 >> v2:172.21.15.162:3300/0 conn(0x7fe5c4162090 msgr2=0x7fe5c40cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 --2- 172.21.15.133:0/483708624 >> v2:172.21.15.162:3300/0 conn(0x7fe5c4162090 0x7fe5c40cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c41625b0 con 0x7fe5c4161cb0 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe5b4001a00 con 0x7fe5c4161cb0 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 >> v1:172.21.15.133:6789/0 conn(0x7fe5c4161cb0 legacy=0x7fe5c406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 --2- 172.21.15.133:0/483708624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 0x7fe5ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 --2- 172.21.15.133:0/483708624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5ac004810 0x7fe5ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 --2- 172.21.15.133:0/483708624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c40cc860 0x7fe5ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c406ec00 con 0x7fe5ac0039c0 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c406e920 con 0x7fe5ac004810 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac00b9d0 con 0x7fe5c40cc860 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c9082640 1 --2- 172.21.15.133:0/483708624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 0x7fe5ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c9883640 1 --2- 172.21.15.133:0/483708624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c40cc860 0x7fe5ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c9082640 1 -- 172.21.15.133:0/483708624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c40cc860 msgr2=0x7fe5ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c9082640 1 --2- 172.21.15.133:0/483708624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c40cc860 0x7fe5ac0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c9082640 1 -- 172.21.15.133:0/483708624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5ac004810 msgr2=0x7fe5ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c9082640 1 --2- 172.21.15.133:0/483708624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5ac004810 0x7fe5ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.833+0000 7fe5c9082640 1 -- 172.21.15.133:0/483708624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac003530 con 0x7fe5ac0039c0 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5c9082640 1 --2- 172.21.15.133:0/483708624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 0x7fe5ac003de0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400a880 tx=0x7fe5b401d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1039b70a395c2b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400a4a0 con 0x7fe5ac0039c0 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b40ad650 con 0x7fe5ac0039c0 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5c3fff640 1 -- 172.21.15.133:0/483708624 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40ada40 con 0x7fe5ac0039c0 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/483708624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 msgr2=0x7fe5ac003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 --2- 172.21.15.133:0/483708624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 0x7fe5ac003de0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400a880 tx=0x7fe5b401d3a0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5c9883640 1 -- 172.21.15.133:0/483708624 reap_dead start 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/483708624 shutdown_connections 2024-04-25T10:06:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/483708624 >> 172.21.15.133:0/483708624 conn(0x7fe5c4067000 msgr2=0x7fe5c406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/483708624 shutdown_connections 2024-04-25T10:06:58.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/483708624 wait complete. 2024-04-25T10:06:58.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 Processor -- start 2024-04-25T10:06:58.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 -- start start 2024-04-25T10:06:58.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 0x7fe5c4165990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c4168600 0x7fe5c414e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5c414ee80 0x7fe5c415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c4163950 con 0x7fe5ac0039c0 2024-04-25T10:06:58.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c41637d0 con 0x7fe5c414ee80 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.837+0000 7fe5cb2e6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c4163ad0 con 0x7fe5c4168600 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c9082640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 0x7fe5c4165990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c9082640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 0x7fe5c4165990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42158/0 (socket says 172.21.15.133:42158) 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c9082640 1 -- 172.21.15.133:0/375434992 learned_addr learned my addr 172.21.15.133:0/375434992 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c8881640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c4168600 0x7fe5c414e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c9082640 1 -- 172.21.15.133:0/375434992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c4168600 msgr2=0x7fe5c414e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c9082640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c4168600 0x7fe5c414e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c9082640 1 -- 172.21.15.133:0/375434992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5c414ee80 msgr2=0x7fe5c415dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c9883640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5c414ee80 0x7fe5c415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c9082640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5c414ee80 0x7fe5c415dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c9082640 1 -- 172.21.15.133:0/375434992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c41660e0 con 0x7fe5ac0039c0 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c8881640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c4168600 0x7fe5c414e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c9082640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 0x7fe5c4165990 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe5b401d1d0 tx=0x7fe5ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c1ffb640 1 -- 172.21.15.133:0/375434992 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40ad430 con 0x7fe5ac0039c0 2024-04-25T10:06:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c1ffb640 1 -- 172.21.15.133:0/375434992 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b40017c0 con 0x7fe5ac0039c0 2024-04-25T10:06:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/375434992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c4167b20 con 0x7fe5ac0039c0 2024-04-25T10:06:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c1ffb640 1 -- 172.21.15.133:0/375434992 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4004b70 con 0x7fe5ac0039c0 2024-04-25T10:06:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/375434992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c41680e0 con 0x7fe5ac0039c0 2024-04-25T10:06:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/375434992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c4068ef0 con 0x7fe5ac0039c0 2024-04-25T10:06:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c1ffb640 1 -- 172.21.15.133:0/375434992 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5b40b5070 con 0x7fe5ac0039c0 2024-04-25T10:06:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c1ffb640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe5b0066f60 0x7fe5b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c8881640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe5b0066f60 0x7fe5b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c1ffb640 1 -- 172.21.15.133:0/375434992 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fe5b411f3e0 con 0x7fe5ac0039c0 2024-04-25T10:06:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.841+0000 7fe5c8881640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe5b0066f60 0x7fe5b0069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4151f20 tx=0x7fe5b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:58.845+0000 7fe5c1ffb640 1 -- 172.21.15.133:0/375434992 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5b40bd060 con 0x7fe5ac0039c0 2024-04-25T10:06:59.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.033+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/375434992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5c40cbea0 con 0x7fe5ac0039c0 2024-04-25T10:06:59.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.033+0000 7fe5c1ffb640 1 -- 172.21.15.133:0/375434992 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe5b40170b0 con 0x7fe5ac0039c0 2024-04-25T10:06:59.040 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:06:59.040 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1579659657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1579659657},{"type":"v1","addr":"172.21.15.162:6835","nonce":1579659657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/232744252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":232744252},{"type":"v1","addr":"172.21.15.162:6833","nonce":232744252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7149,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/912719603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":912719603},{"type":"v1","addr":"172.21.15.133:6835","nonce":912719603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:36.516727+0000","modified":"2024-04-25T10:06:57.917377+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":38,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.133:6837/2310040121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2310040121},{"type":"v1","addr":"172.21.15.133:6837","nonce":2310040121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:59.040 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 40 2024-04-25T10:06:59.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/375434992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe5b0066f60 msgr2=0x7fe5b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe5b0066f60 0x7fe5b0069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4151f20 tx=0x7fe5b8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:59.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/375434992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 msgr2=0x7fe5c4165990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 0x7fe5c4165990 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe5b401d1d0 tx=0x7fe5ac005840 comp rx=0 tx=0).stop 2024-04-25T10:06:59.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/375434992 shutdown_connections 2024-04-25T10:06:59.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5c414ee80 0x7fe5c415dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5c4168600 0x7fe5c414e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe5b0066f60 0x7fe5b0069420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 --2- 172.21.15.133:0/375434992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5ac0039c0 0x7fe5c4165990 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/375434992 >> 172.21.15.133:0/375434992 conn(0x7fe5c4067000 msgr2=0x7fe5c41607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/375434992 shutdown_connections 2024-04-25T10:06:59.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:06:59.037+0000 7fe5cb2e6640 1 -- 172.21.15.133:0/375434992 wait complete. 2024-04-25T10:06:59.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:36.516727+0000', 'modified': '2024-04-25T10:06:57.917377+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7167}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7167': {'gid': 7167, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.133:6837/2310040121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 2310040121}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 2310040121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:06:59.461 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7167:up:rejoin 2024-04-25T10:07:00.462 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.585+0000 7fe4145a3640 1 Processor -- start 2024-04-25T10:07:00.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.585+0000 7fe4145a3640 1 -- start start 2024-04-25T10:07:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe4145a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe40c06eb70 0x7fe40c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe4145a3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40c0cbe60 con 0x7fe40c158110 2024-04-25T10:07:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe4145a3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40c0b19b0 con 0x7fe40c1584f0 2024-04-25T10:07:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe4145a3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c15b360 con 0x7fe40c06eb70 2024-04-25T10:07:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41233f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe40c158110 0x7fe40c06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53076/0 (socket says 172.21.15.133:53076) 2024-04-25T10:07:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41233f640 1 -- 172.21.15.133:0/1642269403 learned_addr learned my addr 172.21.15.133:0/1642269403 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592672954 0 0) 0x7fe40c0cbe60 con 0x7fe40c158110 2024-04-25T10:07:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f8005740 con 0x7fe40c158110 2024-04-25T10:07:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe411b3e640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe40c06eb70 0x7fe40c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2478620987 0 0) 0x7fe3f8005740 con 0x7fe40c158110 2024-04-25T10:07:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe40c06eb70 msgr2=0x7fe40c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe40c06eb70 0x7fe40c0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 >> v1:172.21.15.162:6789/0 conn(0x7fe40c1584f0 legacy=0x7fe40c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c15b640 con 0x7fe40c158110 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe408001680 con 0x7fe40c158110 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 >> v1:172.21.15.133:6789/0 conn(0x7fe40c158110 legacy=0x7fe40c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe40c06eb70 0x7fe3f8005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3f8006230 0x7fe3f8006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3f8006f60 0x7fe3f80073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c15b360 con 0x7fe3f8006230 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f800dda0 con 0x7fe3f8006f60 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f800dfa0 con 0x7fe40c06eb70 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe411b3e640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3f8006230 0x7fe3f8006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe41233f640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe40c06eb70 0x7fe3f8005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe412b40640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3f8006f60 0x7fe3f80073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe411b3e640 1 -- 172.21.15.133:0/1642269403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe40c06eb70 msgr2=0x7fe3f8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe411b3e640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe40c06eb70 0x7fe3f8005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe411b3e640 1 -- 172.21.15.133:0/1642269403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3f8006f60 msgr2=0x7fe3f80073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe411b3e640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3f8006f60 0x7fe3f80073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.589+0000 7fe411b3e640 1 -- 172.21.15.133:0/1642269403 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8005740 con 0x7fe3f8006230 2024-04-25T10:07:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe411b3e640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3f8006230 0x7fe3f8006a20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00ba30 tx=0x7fe3fc01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52114e389408e09c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0ae070 con 0x7fe3f8006230 2024-04-25T10:07:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0a9680 con 0x7fe3f8006230 2024-04-25T10:07:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe41133d640 1 -- 172.21.15.133:0/1642269403 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc00b600 con 0x7fe3f8006230 2024-04-25T10:07:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 -- 172.21.15.133:0/1642269403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3f8006230 msgr2=0x7fe3f8006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 --2- 172.21.15.133:0/1642269403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3f8006230 0x7fe3f8006a20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00ba30 tx=0x7fe3fc01c6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe412b40640 1 -- 172.21.15.133:0/1642269403 reap_dead start 2024-04-25T10:07:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 -- 172.21.15.133:0/1642269403 shutdown_connections 2024-04-25T10:07:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 -- 172.21.15.133:0/1642269403 >> 172.21.15.133:0/1642269403 conn(0x7fe40c067000 msgr2=0x7fe40c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 -- 172.21.15.133:0/1642269403 shutdown_connections 2024-04-25T10:07:00.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 -- 172.21.15.133:0/1642269403 wait complete. 2024-04-25T10:07:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 Processor -- start 2024-04-25T10:07:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 -- start start 2024-04-25T10:07:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3f8006230 0x7fe40c14b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe40c1543c0 0x7fe40c1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe40c13f620 0x7fe40c13fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0cd670 con 0x7fe40c1543c0 2024-04-25T10:07:00.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c15bc00 con 0x7fe40c13f620 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe4145a3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0cd7f0 con 0x7fe3f8006230 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe411b3e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe40c1543c0 0x7fe40c1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe411b3e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe40c1543c0 0x7fe40c1564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42188/0 (socket says 172.21.15.133:42188) 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe411b3e640 1 -- 172.21.15.133:0/4023631082 learned_addr learned my addr 172.21.15.133:0/4023631082 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe412b40640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe40c13f620 0x7fe40c13fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe41233f640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3f8006230 0x7fe40c14b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe411b3e640 1 -- 172.21.15.133:0/4023631082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3f8006230 msgr2=0x7fe40c14b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe411b3e640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3f8006230 0x7fe40c14b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe411b3e640 1 -- 172.21.15.133:0/4023631082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe40c13f620 msgr2=0x7fe40c13fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe411b3e640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe40c13f620 0x7fe40c13fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe411b3e640 1 -- 172.21.15.133:0/4023631082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c13ffc0 con 0x7fe40c1543c0 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.593+0000 7fe412b40640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe40c13f620 0x7fe40c13fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe411b3e640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe40c1543c0 0x7fe40c1564d0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc01cb80 tx=0x7fe3f8007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe402ffd640 1 -- 172.21.15.133:0/4023631082 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0ae040 con 0x7fe40c1543c0 2024-04-25T10:07:00.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe402ffd640 1 -- 172.21.15.133:0/4023631082 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0a9d00 con 0x7fe40c1543c0 2024-04-25T10:07:00.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe4145a3640 1 -- 172.21.15.133:0/4023631082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40c154ba0 con 0x7fe40c1543c0 2024-04-25T10:07:00.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe402ffd640 1 -- 172.21.15.133:0/4023631082 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0a9eb0 con 0x7fe40c1543c0 2024-04-25T10:07:00.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe4145a3640 1 -- 172.21.15.133:0/4023631082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c155160 con 0x7fe40c1543c0 2024-04-25T10:07:00.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe4145a3640 1 -- 172.21.15.133:0/4023631082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40c06bc20 con 0x7fe40c1543c0 2024-04-25T10:07:00.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe402ffd640 1 -- 172.21.15.133:0/4023631082 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3fc0b1070 con 0x7fe40c1543c0 2024-04-25T10:07:00.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe402ffd640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3f0066f60 0x7fe3f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe402ffd640 1 -- 172.21.15.133:0/4023631082 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fe3fc11f840 con 0x7fe40c1543c0 2024-04-25T10:07:00.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe41233f640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3f0066f60 0x7fe3f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.597+0000 7fe41233f640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3f0066f60 0x7fe3f0069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe408000c10 tx=0x7fe408009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.601+0000 7fe402ffd640 1 -- 172.21.15.133:0/4023631082 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3fc0163b0 con 0x7fe40c1543c0 2024-04-25T10:07:00.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 -- 172.21.15.133:0/4023631082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe40c068ef0 con 0x7fe40c1543c0 2024-04-25T10:07:00.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe402ffd640 1 -- 172.21.15.133:0/4023631082 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe3fc0f0610 con 0x7fe40c1543c0 2024-04-25T10:07:00.798 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:00.798 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":41,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1579659657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1579659657},{"type":"v1","addr":"172.21.15.162:6835","nonce":1579659657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/232744252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":232744252},{"type":"v1","addr":"172.21.15.162:6833","nonce":232744252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7149,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/912719603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":912719603},{"type":"v1","addr":"172.21.15.133:6835","nonce":912719603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:36.516727+0000","modified":"2024-04-25T10:06:58.925769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/2310040121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2310040121},{"type":"v1","addr":"172.21.15.133:6837","nonce":2310040121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:00.798 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 41 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 -- 172.21.15.133:0/4023631082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3f0066f60 msgr2=0x7fe3f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3f0066f60 0x7fe3f0069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe408000c10 tx=0x7fe408009040 comp rx=0 tx=0).stop 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 -- 172.21.15.133:0/4023631082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe40c1543c0 msgr2=0x7fe40c1564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe40c1543c0 0x7fe40c1564d0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc01cb80 tx=0x7fe3f8007a50 comp rx=0 tx=0).stop 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 -- 172.21.15.133:0/4023631082 shutdown_connections 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe40c13f620 0x7fe40c13fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3f0066f60 0x7fe3f0069420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe40c1543c0 0x7fe40c1564d0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 --2- 172.21.15.133:0/4023631082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3f8006230 0x7fe40c14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 -- 172.21.15.133:0/4023631082 >> 172.21.15.133:0/4023631082 conn(0x7fe40c067000 msgr2=0x7fe40c0ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 -- 172.21.15.133:0/4023631082 shutdown_connections 2024-04-25T10:07:00.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:00.793+0000 7fe4145a3640 1 -- 172.21.15.133:0/4023631082 wait complete. 2024-04-25T10:07:01.218 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:36.516727+0000', 'modified': '2024-04-25T10:06:58.925769+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7167}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7167': {'gid': 7167, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.133:6837/2310040121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 2310040121}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 2310040121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:01.219 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:07:01.219 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:07:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11d67ce640 1 Processor -- start 2024-04-25T10:07:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11d67ce640 1 -- start start 2024-04-25T10:07:01.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11d67ce640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f11d015d9a0 0x7f11d00d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11d67ce640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f11d00ce600 0x7f11d00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11d67ce640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d00d1920 0x7f11d00d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11d67ce640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d015ebd0 con 0x7f11d00ce600 2024-04-25T10:07:01.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11d67ce640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d015eeb0 con 0x7f11d015d9a0 2024-04-25T10:07:01.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11d67ce640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00d0330 con 0x7f11d00d1920 2024-04-25T10:07:01.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11cf7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f11d00ce600 0x7f11d00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11cf7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f11d00ce600 0x7f11d00cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42200/0 (socket says 172.21.15.133:42200) 2024-04-25T10:07:01.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11cf7fe640 1 -- 172.21.15.133:0/930001674 learned_addr learned my addr 172.21.15.133:0/930001674 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:01.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/930001674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d00d1920 0x7f11d00d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11cffff640 1 --2- 172.21.15.133:0/930001674 >> v2:172.21.15.162:3300/0 conn(0x7f11d015d9a0 0x7f11d00d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11cf7fe640 1 -- 172.21.15.133:0/930001674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d00d1920 msgr2=0x7f11d00d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/930001674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d00d1920 0x7f11d00d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11cf7fe640 1 -- 172.21.15.133:0/930001674 >> v2:172.21.15.162:3300/0 conn(0x7f11d015d9a0 msgr2=0x7f11d00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/930001674 >> v2:172.21.15.162:3300/0 conn(0x7f11d015d9a0 0x7f11d00d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11cf7fe640 1 -- 172.21.15.133:0/930001674 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d00d0610 con 0x7f11d00ce600 2024-04-25T10:07:01.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11cffff640 1 --2- 172.21.15.133:0/930001674 >> v2:172.21.15.162:3300/0 conn(0x7f11d015d9a0 0x7f11d00d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.349+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/930001674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d00d1920 0x7f11d00d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/930001674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d00ce600 0x7f11d00cea90 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f11d00d4750 tx=0x7f11c401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdd6e2abd64f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11ceffd640 1 -- 172.21.15.133:0/930001674 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c40b2070 con 0x7f11d00ce600 2024-04-25T10:07:01.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11ceffd640 1 -- 172.21.15.133:0/930001674 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c40a9070 con 0x7f11d00ce600 2024-04-25T10:07:01.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11ceffd640 1 -- 172.21.15.133:0/930001674 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c40ad880 con 0x7f11d00ce600 2024-04-25T10:07:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11d67ce640 1 -- 172.21.15.133:0/930001674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d00ce600 msgr2=0x7f11d00cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11d67ce640 1 --2- 172.21.15.133:0/930001674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d00ce600 0x7f11d00cea90 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f11d00d4750 tx=0x7f11c401c490 comp rx=0 tx=0).stop 2024-04-25T10:07:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11d67ce640 1 -- 172.21.15.133:0/930001674 shutdown_connections 2024-04-25T10:07:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11d67ce640 1 --2- 172.21.15.133:0/930001674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d00d1920 0x7f11d00d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11d67ce640 1 --2- 172.21.15.133:0/930001674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d00ce600 0x7f11d00cea90 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11d67ce640 1 --2- 172.21.15.133:0/930001674 >> v2:172.21.15.162:3300/0 conn(0x7f11d015d9a0 0x7f11d00d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11d67ce640 1 -- 172.21.15.133:0/930001674 >> 172.21.15.133:0/930001674 conn(0x7f11d006bf80 msgr2=0x7f11d006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11d67ce640 1 -- 172.21.15.133:0/930001674 shutdown_connections 2024-04-25T10:07:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11d67ce640 1 -- 172.21.15.133:0/930001674 wait complete. 2024-04-25T10:07:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11d67ce640 1 Processor -- start 2024-04-25T10:07:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.353+0000 7f11d67ce640 1 -- start start 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d67ce640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d00ce600 0x7f11d015f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d67ce640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11d00d1920 0x7f11d00cf470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d67ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d015d9a0 0x7f11d015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d67ce640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00d2470 con 0x7f11d015d9a0 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d67ce640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00d22f0 con 0x7f11d00d1920 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d67ce640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d015e590 con 0x7f11d00ce600 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d4d6b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d015d9a0 0x7f11d015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d4d6b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d015d9a0 0x7f11d015a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42216/0 (socket says 172.21.15.133:42216) 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d4d6b640 1 -- 172.21.15.133:0/4255961739 learned_addr learned my addr 172.21.15.133:0/4255961739 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11cffff640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d00ce600 0x7f11d015f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11d00d1920 0x7f11d00cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d4d6b640 1 -- 172.21.15.133:0/4255961739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d00ce600 msgr2=0x7f11d015f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d00ce600 0x7f11d015f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d4d6b640 1 -- 172.21.15.133:0/4255961739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11d00d1920 msgr2=0x7f11d00cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11d00d1920 0x7f11d00cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d4d6b640 1 -- 172.21.15.133:0/4255961739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d0155090 con 0x7f11d015d9a0 2024-04-25T10:07:01.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d015d9a0 0x7f11d015a750 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f11c800cef0 tx=0x7f11c800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11ccff9640 1 -- 172.21.15.133:0/4255961739 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c801a070 con 0x7f11d015d9a0 2024-04-25T10:07:01.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11ccff9640 1 -- 172.21.15.133:0/4255961739 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c800d840 con 0x7f11d015d9a0 2024-04-25T10:07:01.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d67ce640 1 -- 172.21.15.133:0/4255961739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d015fdc0 con 0x7f11d015d9a0 2024-04-25T10:07:01.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11ccff9640 1 -- 172.21.15.133:0/4255961739 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c800dc30 con 0x7f11d015d9a0 2024-04-25T10:07:01.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11d67ce640 1 -- 172.21.15.133:0/4255961739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d0160380 con 0x7f11d015d9a0 2024-04-25T10:07:01.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11ccff9640 1 -- 172.21.15.133:0/4255961739 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11c801d070 con 0x7f11d015d9a0 2024-04-25T10:07:01.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11ccff9640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11bc066f60 0x7f11bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11ccff9640 1 -- 172.21.15.133:0/4255961739 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f11c8085aa0 con 0x7f11d015d9a0 2024-04-25T10:07:01.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.357+0000 7f11cffff640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11bc066f60 0x7f11bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11cffff640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11bc066f60 0x7f11bc069420 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f11c0006fd0 tx=0x7f11c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 Processor -- start 2024-04-25T10:07:01.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 -- start start 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdaf920 0x55f87cdafd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cd659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cd65f30 0x55f87cd66310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87cc4d5f0 con 0x55f87cdb0240 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87cbb4da0 con 0x55f87cdaf920 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f87cb3d150 con 0x55f87cd65f30 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cd659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cd659f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42230/0 (socket says 172.21.15.133:42230) 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11cf7fe640 1 -- 172.21.15.133:0/1392757949 learned_addr learned my addr 172.21.15.133:0/1392757949 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/1392757949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cd65f30 0x55f87cd66310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11cffff640 1 --2- 172.21.15.133:0/1392757949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdaf920 0x55f87cdafd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11cf7fe640 1 -- 172.21.15.133:0/1392757949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cd65f30 msgr2=0x55f87cd66310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/1392757949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cd65f30 0x55f87cd66310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11cf7fe640 1 -- 172.21.15.133:0/1392757949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdaf920 msgr2=0x55f87cdafd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/1392757949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdaf920 0x55f87cdafd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11cf7fe640 1 -- 172.21.15.133:0/1392757949 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f87cb56ca0 con 0x55f87cdb0240 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/1392757949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cd659f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f11c401ab40 tx=0x7f11c40a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c892e2df24a48d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11b5ffb640 1 -- 172.21.15.133:0/1392757949 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c40b2040 con 0x55f87cdb0240 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11b5ffb640 1 -- 172.21.15.133:0/1392757949 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c40ad7f0 con 0x55f87cdb0240 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11b5ffb640 1 -- 172.21.15.133:0/1392757949 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c40adba0 con 0x55f87cdb0240 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 -- 172.21.15.133:0/1392757949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 msgr2=0x55f87cd659f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 --2- 172.21.15.133:0/1392757949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cd659f0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f11c401ab40 tx=0x7f11c40a7000 comp rx=0 tx=0).stop 2024-04-25T10:07:01.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 -- 172.21.15.133:0/1392757949 shutdown_connections 2024-04-25T10:07:01.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 --2- 172.21.15.133:0/1392757949 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cd65f30 0x55f87cd66310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 --2- 172.21.15.133:0/1392757949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cd659f0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 --2- 172.21.15.133:0/1392757949 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdaf920 0x55f87cdafd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 -- 172.21.15.133:0/1392757949 >> 172.21.15.133:0/1392757949 conn(0x55f87cd6c2a0 msgr2=0x55f87cd642d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 -- 172.21.15.133:0/1392757949 shutdown_connections 2024-04-25T10:07:01.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 -- 172.21.15.133:0/1392757949 wait complete. 2024-04-25T10:07:01.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 -- start start 2024-04-25T10:07:01.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.361+0000 7f11d993b000 1 Processor -- start 2024-04-25T10:07:01.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87cdb6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87cdb71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdb0240 0x55f87cd69aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87cb6f990 con 0x55f87cd65f30 2024-04-25T10:07:01.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87cbb0e40 con 0x55f87cdb0240 2024-04-25T10:07:01.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f87cb40900 con 0x55f87cdaf920 2024-04-25T10:07:01.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 10 client.0 _ll_register_callbacks cb 0x55f87cc681c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87cdb6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87cdb6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42232/0 (socket says 172.21.15.133:42232) 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cffff640 1 -- 172.21.15.133:0/888687896 learned_addr learned my addr 172.21.15.133:0/888687896 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdb0240 0x55f87cd69aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87cdb71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cffff640 1 -- 172.21.15.133:0/888687896 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 msgr2=0x55f87cdb71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cffff640 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87cdb71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cffff640 1 -- 172.21.15.133:0/888687896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdb0240 msgr2=0x55f87cd69aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cffff640 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdb0240 0x55f87cd69aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cffff640 1 -- 172.21.15.133:0/888687896 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f87cb6d630 con 0x55f87cd65f30 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdb0240 0x55f87cd69aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cffff640 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87cdb6c80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f11c0013720 tx=0x7f11c0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4872d25fb39912 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:07:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11aeffd640 1 -- 172.21.15.133:0/888687896 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0024020 con 0x55f87cd65f30 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11aeffd640 1 -- 172.21.15.133:0/888687896 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f11c0014850 con 0x55f87cd65f30 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 -- 172.21.15.133:0/888687896 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11c40ad7f0 con 0x55f87cd65f30 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11aeffd640 1 -- 172.21.15.133:0/888687896 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11c002e050 con 0x55f87cd65f30 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 10 client.7206 fetch_fsmap learned FSMap version 41 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 -- 172.21.15.133:0/888687896 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55f87cb3d3a0 con 0x55f87cd65f30 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11aeffd640 1 -- 172.21.15.133:0/888687896 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c40ad7f0 con 0x55f87cd65f30 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11aeffd640 1 -- 172.21.15.133:0/888687896 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f11c0031020 con 0x55f87cd65f30 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 10 client.7206 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 10 client.7206 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 20 client.7206 populate_metadata read hostname 'smithi133' 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55f87cd6b160 0x55f87cdc0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 4 client.7206 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 -- 172.21.15.133:0/888687896 --> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f87cb50a50 con 0x55f87cd6b160 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55f87cd6b160 0x55f87cdc0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55f87cd6b160 0x55f87cdc0ce0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58c126487e9758d server_cookie=ba4741e04f51b8a9 in_seq=0 out_seq=0 2024-04-25T10:07:01.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11aeffd640 10 client.7206 ms_handle_connect on v2:172.21.15.133:6836/2310040121 2024-04-25T10:07:01.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11aeffd640 1 -- 172.21.15.133:0/888687896 <== mds.0 v2:172.21.15.133:6836/2310040121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f87cb50a50 con 0x55f87cd6b160 2024-04-25T10:07:01.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11aeffd640 10 client.7206 handle_command_reply: tid=0 2024-04-25T10:07:01.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 client.7206 shutdown 2024-04-25T10:07:01.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 -- 172.21.15.133:0/888687896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 msgr2=0x55f87cdb6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87cdb6c80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f11c0013720 tx=0x7f11c0004740 comp rx=0 tx=0).stop 2024-04-25T10:07:01.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 -- 172.21.15.133:0/888687896 shutdown_connections 2024-04-25T10:07:01.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55f87cd6b160 0x55f87cdc0ce0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdb0240 0x55f87cd69aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11aeffd640 0 client.7206 ms_handle_reset on v2:172.21.15.133:6836/2310040121 2024-04-25T10:07:01.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87cdb71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 --2- 172.21.15.133:0/888687896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87cdb6c80 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 -- 172.21.15.133:0/888687896 >> 172.21.15.133:0/888687896 conn(0x55f87cd6c2a0 msgr2=0x55f87cd649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 -- 172.21.15.133:0/888687896 shutdown_connections 2024-04-25T10:07:01.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 1 -- 172.21.15.133:0/888687896 wait complete. 2024-04-25T10:07:01.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.365+0000 7f11d993b000 20 client.7206 trim_cache size 0 max 16384 2024-04-25T10:07:01.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 Processor -- start 2024-04-25T10:07:01.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 -- start start 2024-04-25T10:07:01.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87ce0af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87ce0b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdb0240 0x55f87ce0ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d0202090 con 0x55f87cd65f30 2024-04-25T10:07:01.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d01585b0 con 0x55f87cdb0240 2024-04-25T10:07:01.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d0201ac0 con 0x55f87cdaf920 2024-04-25T10:07:01.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87ce0af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87ce0af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42240/0 (socket says 172.21.15.133:42240) 2024-04-25T10:07:01.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11cffff640 1 -- 172.21.15.133:0/450712945 learned_addr learned my addr 172.21.15.133:0/450712945 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:01.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11cffff640 1 -- 172.21.15.133:0/450712945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 msgr2=0x55f87ce0b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11cffff640 1 --2- 172.21.15.133:0/450712945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87ce0b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11cffff640 1 -- 172.21.15.133:0/450712945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdb0240 msgr2=0x55f87ce0ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/450712945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdb0240 0x55f87ce0ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11cffff640 1 --2- 172.21.15.133:0/450712945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdb0240 0x55f87ce0ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11cffff640 1 -- 172.21.15.133:0/450712945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f87cdb0070 con 0x55f87cd65f30 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11cffff640 1 --2- 172.21.15.133:0/450712945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87ce0af80 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f11c000ed20 tx=0x7f11c001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2709086ee7ede86d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11b57fa640 1 -- 172.21.15.133:0/450712945 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0024070 con 0x55f87cd65f30 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11b57fa640 1 -- 172.21.15.133:0/450712945 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c001b040 con 0x55f87cd65f30 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11b57fa640 1 -- 172.21.15.133:0/450712945 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c001a070 con 0x55f87cd65f30 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 -- 172.21.15.133:0/450712945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 msgr2=0x55f87ce0af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 --2- 172.21.15.133:0/450712945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87ce0af80 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f11c000ed20 tx=0x7f11c001f000 comp rx=0 tx=0).stop 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 -- 172.21.15.133:0/450712945 shutdown_connections 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 --2- 172.21.15.133:0/450712945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cdb0240 0x55f87ce0ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 --2- 172.21.15.133:0/450712945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87ce0b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 --2- 172.21.15.133:0/450712945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cd65f30 0x55f87ce0af80 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 -- 172.21.15.133:0/450712945 >> 172.21.15.133:0/450712945 conn(0x55f87cd6c2a0 msgr2=0x55f87cd69220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 -- 172.21.15.133:0/450712945 shutdown_connections 2024-04-25T10:07:01.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.381+0000 7f11d993b000 1 -- 172.21.15.133:0/450712945 wait complete. 2024-04-25T10:07:01.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- start start 2024-04-25T10:07:01.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 Processor -- start 2024-04-25T10:07:01.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cd65f30 0x55f87cdbc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87cdf45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cdf4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87cbb93d0 con 0x55f87cdb0240 2024-04-25T10:07:01.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f87cc70d50 con 0x55f87cd65f30 2024-04-25T10:07:01.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f87cb4f980 con 0x55f87cdaf920 2024-04-25T10:07:01.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 10 client.0 _ll_register_callbacks cb 0x7f11c40b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:01.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d4d6b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cdf4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d4d6b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cdf4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42250/0 (socket says 172.21.15.133:42250) 2024-04-25T10:07:01.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d4d6b640 1 -- 172.21.15.133:0/3249114084 learned_addr learned my addr 172.21.15.133:0/3249114084 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:01.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11cffff640 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cd65f30 0x55f87cdbc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87cdf45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d4d6b640 1 -- 172.21.15.133:0/3249114084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 msgr2=0x55f87cdf45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87cdf45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d4d6b640 1 -- 172.21.15.133:0/3249114084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cd65f30 msgr2=0x55f87cdbc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cd65f30 0x55f87cdbc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d4d6b640 1 -- 172.21.15.133:0/3249114084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f87cc36600 con 0x55f87cdb0240 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11cf7fe640 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87cdf45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d4d6b640 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cdf4af0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f11c8055e70 tx=0x7f11c8056230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df507fe23eb95487 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11aeffd640 1 -- 172.21.15.133:0/3249114084 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c805e880 con 0x55f87cdb0240 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11aeffd640 1 -- 172.21.15.133:0/3249114084 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f11c8054210 con 0x55f87cdb0240 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- 172.21.15.133:0/3249114084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11c001b040 con 0x55f87cdb0240 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11aeffd640 1 -- 172.21.15.133:0/3249114084 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11c8054470 con 0x55f87cdb0240 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 10 client.7212 fetch_fsmap learned FSMap version 41 2024-04-25T10:07:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- 172.21.15.133:0/3249114084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55f87cb499b0 con 0x55f87cdb0240 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11aeffd640 1 -- 172.21.15.133:0/3249114084 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c001b040 con 0x55f87cdb0240 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11aeffd640 1 -- 172.21.15.133:0/3249114084 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f11c805e3f0 con 0x55f87cdb0240 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 10 client.7212 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 10 client.7212 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 20 client.7212 populate_metadata read hostname 'smithi133' 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55f87cd6be00 0x55f87cdf5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 4 client.7212 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- 172.21.15.133:0/3249114084 --> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f87cbc22e0 con 0x55f87cd6be00 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11cffff640 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55f87cd6be00 0x55f87cdf5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11aeffd640 10 client.7212 ms_handle_connect on v2:172.21.15.133:6836/2310040121 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11cffff640 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55f87cd6be00 0x55f87cdf5030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8de230e042671dd4 server_cookie=7a4482dde6c733d5 in_seq=0 out_seq=0 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11aeffd640 1 -- 172.21.15.133:0/3249114084 <== mds.0 v2:172.21.15.133:6836/2310040121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55f87cbc22e0 con 0x55f87cd6be00 2024-04-25T10:07:01.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11aeffd640 10 client.7212 handle_command_reply: tid=0 2024-04-25T10:07:01.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 client.7212 shutdown 2024-04-25T10:07:01.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- 172.21.15.133:0/3249114084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 msgr2=0x55f87cdf4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cdf4af0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f11c8055e70 tx=0x7f11c8056230 comp rx=0 tx=0).stop 2024-04-25T10:07:01.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- 172.21.15.133:0/3249114084 shutdown_connections 2024-04-25T10:07:01.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55f87cd6be00 0x55f87cdf5030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f87cdb0240 0x55f87cdf4af0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11aeffd640 0 client.7212 ms_handle_reset on v2:172.21.15.133:6836/2310040121 2024-04-25T10:07:01.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f87cdaf920 0x55f87cdf45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 --2- 172.21.15.133:0/3249114084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f87cd65f30 0x55f87cdbc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- 172.21.15.133:0/3249114084 >> 172.21.15.133:0/3249114084 conn(0x55f87cd6c2a0 msgr2=0x55f87cdb7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- 172.21.15.133:0/3249114084 shutdown_connections 2024-04-25T10:07:01.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 1 -- 172.21.15.133:0/3249114084 wait complete. 2024-04-25T10:07:01.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d993b000 20 client.7212 trim_cache size 0 max 16384 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 7167, 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 4.483220588, 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 41, 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 70, 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 70, 2024-04-25T10:07:01.400 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 5.3407039860000003 2024-04-25T10:07:01.401 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:07:01.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d67ce640 1 -- 172.21.15.133:0/4255961739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11bc066f60 msgr2=0x7f11bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d67ce640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11bc066f60 0x7f11bc069420 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f11c0006fd0 tx=0x7f11c0008040 comp rx=0 tx=0).stop 2024-04-25T10:07:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d67ce640 1 -- 172.21.15.133:0/4255961739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d015d9a0 msgr2=0x7f11d015a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d67ce640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d015d9a0 0x7f11d015a750 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f11c800cef0 tx=0x7f11c800a710 comp rx=0 tx=0).stop 2024-04-25T10:07:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d67ce640 1 -- 172.21.15.133:0/4255961739 shutdown_connections 2024-04-25T10:07:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d67ce640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11bc066f60 0x7f11bc069420 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d67ce640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11d015d9a0 0x7f11d015a750 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d67ce640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f11d00d1920 0x7f11d00cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d67ce640 1 --2- 172.21.15.133:0/4255961739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11d00ce600 0x7f11d015f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.385+0000 7f11d67ce640 1 -- 172.21.15.133:0/4255961739 >> 172.21.15.133:0/4255961739 conn(0x7f11d006bf80 msgr2=0x7f11d0168060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.389+0000 7f11d67ce640 1 -- 172.21.15.133:0/4255961739 shutdown_connections 2024-04-25T10:07:01.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.389+0000 7f11d67ce640 1 -- 172.21.15.133:0/4255961739 wait complete. 2024-04-25T10:07:01.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:01.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.513+0000 7fbe8bb9d640 1 Processor -- start 2024-04-25T10:07:01.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.513+0000 7fbe8bb9d640 1 -- start start 2024-04-25T10:07:01.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.513+0000 7fbe8bb9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbe8406bda0 0x7fbe840b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.513+0000 7fbe8bb9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe840cc250 0x7fbe840cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.513+0000 7fbe8bb9d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe840cfdc0 con 0x7fbe8406dca0 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.513+0000 7fbe8bb9d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840d0660 con 0x7fbe8406bda0 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.513+0000 7fbe8bb9d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840d00a0 con 0x7fbe840cc250 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe89138640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbe8406bda0 0x7fbe840b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe89138640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbe8406bda0 0x7fbe840b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42266/0 (socket says 172.21.15.133:42266) 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe89138640 1 -- 172.21.15.133:0/1149090276 learned_addr learned my addr 172.21.15.133:0/1149090276 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe8a13a640 1 --2- 172.21.15.133:0/1149090276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe840cc250 0x7fbe840cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe89138640 1 -- 172.21.15.133:0/1149090276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe840cc250 msgr2=0x7fbe840cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe89138640 1 --2- 172.21.15.133:0/1149090276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe840cc250 0x7fbe840cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe89138640 1 -- 172.21.15.133:0/1149090276 >> v1:172.21.15.162:6789/0 conn(0x7fbe8406dca0 legacy=0x7fbe840b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe89138640 1 -- 172.21.15.133:0/1149090276 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840d0380 con 0x7fbe8406bda0 2024-04-25T10:07:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe8a13a640 1 --2- 172.21.15.133:0/1149090276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe840cc250 0x7fbe840cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe89138640 1 --2- 172.21.15.133:0/1149090276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe8406bda0 0x7fbe840b1d30 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009870 tx=0x7fbe7401ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f342fb559918d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe88937640 1 -- 172.21.15.133:0/1149090276 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe740b0070 con 0x7fbe8406bda0 2024-04-25T10:07:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe88937640 1 -- 172.21.15.133:0/1149090276 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe740ab600 con 0x7fbe8406bda0 2024-04-25T10:07:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe88937640 1 -- 172.21.15.133:0/1149090276 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe740ab9d0 con 0x7fbe8406bda0 2024-04-25T10:07:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/1149090276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe8406bda0 msgr2=0x7fbe840b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe8bb9d640 1 --2- 172.21.15.133:0/1149090276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe8406bda0 0x7fbe840b1d30 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009870 tx=0x7fbe7401ce60 comp rx=0 tx=0).stop 2024-04-25T10:07:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/1149090276 shutdown_connections 2024-04-25T10:07:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe8bb9d640 1 --2- 172.21.15.133:0/1149090276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe840cc250 0x7fbe840cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe8bb9d640 1 --2- 172.21.15.133:0/1149090276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe8406bda0 0x7fbe840b1d30 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/1149090276 >> 172.21.15.133:0/1149090276 conn(0x7fbe840b31a0 msgr2=0x7fbe840b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/1149090276 shutdown_connections 2024-04-25T10:07:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.517+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/1149090276 wait complete. 2024-04-25T10:07:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8bb9d640 1 Processor -- start 2024-04-25T10:07:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8bb9d640 1 -- start start 2024-04-25T10:07:01.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8bb9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe8406bda0 0x7fbe8414e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8bb9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe8406dca0 0x7fbe840c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8bb9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840cc250 0x7fbe84159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8bb9d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8416a920 con 0x7fbe840cc250 2024-04-25T10:07:01.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8bb9d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8416a7a0 con 0x7fbe8406dca0 2024-04-25T10:07:01.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8bb9d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8416aaa0 con 0x7fbe8406bda0 2024-04-25T10:07:01.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8a13a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840cc250 0x7fbe84159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8a13a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840cc250 0x7fbe84159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42272/0 (socket says 172.21.15.133:42272) 2024-04-25T10:07:01.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8a13a640 1 -- 172.21.15.133:0/4067505459 learned_addr learned my addr 172.21.15.133:0/4067505459 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe89138640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe8406dca0 0x7fbe840c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8a13a640 1 -- 172.21.15.133:0/4067505459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe8406bda0 msgr2=0x7fbe8414e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe89939640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe8406bda0 0x7fbe8414e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8a13a640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe8406bda0 0x7fbe8414e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8a13a640 1 -- 172.21.15.133:0/4067505459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe8406dca0 msgr2=0x7fbe840c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8a13a640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe8406dca0 0x7fbe840c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8a13a640 1 -- 172.21.15.133:0/4067505459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840c8cc0 con 0x7fbe840cc250 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8a13a640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840cc250 0x7fbe84159600 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00cd50 tx=0x7fbe7c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe7a7fc640 1 -- 172.21.15.133:0/4067505459 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00f680 con 0x7fbe840cc250 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe7a7fc640 1 -- 172.21.15.133:0/4067505459 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c00f840 con 0x7fbe840cc250 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/4067505459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8415a0c0 con 0x7fbe840cc250 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe7a7fc640 1 -- 172.21.15.133:0/4067505459 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00fc30 con 0x7fbe840cc250 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/4067505459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8415f1c0 con 0x7fbe840cc250 2024-04-25T10:07:01.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/4067505459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe84067c70 con 0x7fbe840cc250 2024-04-25T10:07:01.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.521+0000 7fbe7a7fc640 1 -- 172.21.15.133:0/4067505459 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe7c0213f0 con 0x7fbe840cc250 2024-04-25T10:07:01.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.525+0000 7fbe7a7fc640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbe68066f90 0x7fbe68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.525+0000 7fbe7a7fc640 1 -- 172.21.15.133:0/4067505459 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fbe7c086020 con 0x7fbe840cc250 2024-04-25T10:07:01.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.525+0000 7fbe89939640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbe68066f90 0x7fbe68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.525+0000 7fbe89939640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbe68066f90 0x7fbe68069450 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000a750 tx=0x7fbe80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.529+0000 7fbe7a7fc640 1 -- 172.21.15.133:0/4067505459 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe7c014030 con 0x7fbe840cc250 2024-04-25T10:07:01.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.717+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/4067505459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbe8415afb0 con 0x7fbe840cc250 2024-04-25T10:07:01.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.717+0000 7fbe7a7fc640 1 -- 172.21.15.133:0/4067505459 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbe7c056df0 con 0x7fbe840cc250 2024-04-25T10:07:01.722 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.717+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/4067505459 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbe68066f90 msgr2=0x7fbe68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.717+0000 7fbe8bb9d640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbe68066f90 0x7fbe68069450 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000a750 tx=0x7fbe80008040 comp rx=0 tx=0).stop 2024-04-25T10:07:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.721+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/4067505459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840cc250 msgr2=0x7fbe84159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.721+0000 7fbe8bb9d640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840cc250 0x7fbe84159600 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00cd50 tx=0x7fbe7c00a650 comp rx=0 tx=0).stop 2024-04-25T10:07:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.721+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/4067505459 shutdown_connections 2024-04-25T10:07:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.721+0000 7fbe8bb9d640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbe68066f90 0x7fbe68069450 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.721+0000 7fbe8bb9d640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840cc250 0x7fbe84159600 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.721+0000 7fbe8bb9d640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbe8406dca0 0x7fbe840c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.721+0000 7fbe8bb9d640 1 --2- 172.21.15.133:0/4067505459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbe8406bda0 0x7fbe8414e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.721+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/4067505459 >> 172.21.15.133:0/4067505459 conn(0x7fbe840b31a0 msgr2=0x7fbe840cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.721+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/4067505459 shutdown_connections 2024-04-25T10:07:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:01.721+0000 7fbe8bb9d640 1 -- 172.21.15.133:0/4067505459 wait complete. 2024-04-25T10:07:01.740 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:07:01.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:01.740 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:07:01.740 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:07:01.740 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:01.740 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:07:01.740 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:07:01.740 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:01.740 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:01.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.740 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:07:01.746 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:01.746 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:01.747 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:01.747 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.747 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:01.747 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.747 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:01.747 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:01.747 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:07:01.747 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:01.747 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42210sec preferred_lft 42210sec 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:07:01.748 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:07:01.749 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:01.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.750 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:01.796 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:07:01.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.798 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:07:01.848 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:07:01.848 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:01.849 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:07:01.849 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:07:01.849 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:07:01.849 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:07:01.849 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:07:01.849 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:07:01.849 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:07:01.849 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:07:01.849 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:01.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.850 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:07:01.850 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:07:01.850 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:07:01.850 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:07:01.850 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:07:01.850 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:07:01.907 INFO:tasks.ceph.mds.c.smithi133.stderr:2024-04-25T10:07:01.901+0000 7f169d48c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:01.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.923 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:07:01.974 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:07:01.974 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:07:01.974 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:01.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.975 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:07:02.030 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:07:02.030 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:02.030 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:02.077 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:02.077 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:02.077 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:02.125 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:07:02.182 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:02.231 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:07:02.231 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:07:02.231 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:07:02.231 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:07:02.231 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:02.232 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:02.233 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:02.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:02.234 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:07:02.281 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:07:02.281 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:07:02.326 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:07:02.340 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:02.389 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:07:02.389 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:07:02.389 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:07:02.390 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:02.391 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:02.392 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:02.392 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:02.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:02.392 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:07:02.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:07:02.401+0000 7fc1653bf580 -1 init, newargv = 0x564828e3ebd0 newargc=15 2024-04-25T10:07:02.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[56951]: starting ceph client 2024-04-25T10:07:02.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[56951]: starting fuse 2024-04-25T10:07:03.513 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:07:03.529 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:03.579 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:07:03.579 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:07:03.580 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:03.581 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:07:03.582 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:03.582 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:03.582 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:03.582 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:03.582 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:03.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:03.583 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:07:03.630 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:07:03.630 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:07:03.630 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:07:03.630 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:07:03.630 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:07:03.630 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:07:03.631 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:07:03.632 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:07:03.632 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:07:03.632 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:07:03.632 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:07:03.632 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:07:03.632 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:03.632 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:07:03.632 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:07:03.753 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.56951.asok 2024-04-25T10:07:03.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:07:03.766 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56951.asok 2024-04-25T10:07:03.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:03.766 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56951.asok status 2024-04-25T10:07:03.917 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:07:03.917 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:07:03.917 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:07:03.917 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:07:03.917 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:07:03.917 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:07:03.917 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "56951", 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 7227, 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 7227 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:07:03.918 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3384112195 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3384112195 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.7227 192.168.0.1:0/3384112195", 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/3384112195", 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 41, 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 70, 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:07:03.919 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:07:03.920 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:07:03.920 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:07:03.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:03.931 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:07:03.931 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:03.937 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:07:03.937 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:03.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:03.937 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:03.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:03.994 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T10:07:04.043 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T10:07:04.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:04.044 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T10:07:04.045 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:07:04.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T10:07:04.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.217+0000 7ff25017b640 1 Processor -- start 2024-04-25T10:07:04.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.217+0000 7ff25017b640 1 -- start start 2024-04-25T10:07:04.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.217+0000 7ff25017b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff24806ac90 0x7ff24806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.217+0000 7ff25017b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2480afde0 0x7ff2480b0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.217+0000 7ff25017b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2480b1eb0 con 0x7ff24815b8f0 2024-04-25T10:07:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.217+0000 7ff25017b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24815cc30 con 0x7ff24806ac90 2024-04-25T10:07:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.217+0000 7ff25017b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24815c060 con 0x7ff2480afde0 2024-04-25T10:07:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24d716640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff24806ac90 0x7ff24806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24df17640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff24815b8f0 0x7ff2480b17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:46114/0 (socket says 172.21.15.133:46114) 2024-04-25T10:07:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24e718640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2480afde0 0x7ff2480b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24df17640 1 -- 172.21.15.133:0/4019864185 learned_addr learned my addr 172.21.15.133:0/4019864185 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24e718640 1 -- 172.21.15.133:0/4019864185 >> v2:172.21.15.162:3300/0 conn(0x7ff24806ac90 msgr2=0x7ff24806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24e718640 1 --2- 172.21.15.133:0/4019864185 >> v2:172.21.15.162:3300/0 conn(0x7ff24806ac90 0x7ff24806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24e718640 1 -- 172.21.15.133:0/4019864185 >> v1:172.21.15.162:6789/0 conn(0x7ff24815b8f0 legacy=0x7ff2480b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24e718640 1 -- 172.21.15.133:0/4019864185 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24815c300 con 0x7ff2480afde0 2024-04-25T10:07:04.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24cf15640 1 -- 172.21.15.133:0/4019864185 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803107667 0 0) 0x7ff2480b1eb0 con 0x7ff24815b8f0 2024-04-25T10:07:04.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24e718640 1 --2- 172.21.15.133:0/4019864185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2480afde0 0x7ff2480b0250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff2400097c0 tx=0x7ff24001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5adadbd241f5b4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24cf15640 1 -- 172.21.15.133:0/4019864185 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2400b0070 con 0x7ff2480afde0 2024-04-25T10:07:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24cf15640 1 -- 172.21.15.133:0/4019864185 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2400ab600 con 0x7ff2480afde0 2024-04-25T10:07:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff24cf15640 1 -- 172.21.15.133:0/4019864185 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2400ab9d0 con 0x7ff2480afde0 2024-04-25T10:07:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff25017b640 1 -- 172.21.15.133:0/4019864185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2480afde0 msgr2=0x7ff2480b0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff25017b640 1 --2- 172.21.15.133:0/4019864185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2480afde0 0x7ff2480b0250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff2400097c0 tx=0x7ff24001c410 comp rx=0 tx=0).stop 2024-04-25T10:07:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.221+0000 7ff25017b640 1 -- 172.21.15.133:0/4019864185 shutdown_connections 2024-04-25T10:07:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 --2- 172.21.15.133:0/4019864185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2480afde0 0x7ff2480b0250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 --2- 172.21.15.133:0/4019864185 >> v2:172.21.15.162:3300/0 conn(0x7ff24806ac90 0x7ff24806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 -- 172.21.15.133:0/4019864185 >> 172.21.15.133:0/4019864185 conn(0x7ff2480726f0 msgr2=0x7ff2480999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 -- 172.21.15.133:0/4019864185 shutdown_connections 2024-04-25T10:07:04.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 -- 172.21.15.133:0/4019864185 wait complete. 2024-04-25T10:07:04.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 Processor -- start 2024-04-25T10:07:04.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 -- start start 2024-04-25T10:07:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24806ac90 0x7ff24815a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2480afde0 0x7ff2480cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff24815b8f0 0x7ff248154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2480b1570 con 0x7ff24806ac90 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2480b13f0 con 0x7ff2480afde0 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2480cc1e0 con 0x7ff24815b8f0 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24df17640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24806ac90 0x7ff24815a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24df17640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24806ac90 0x7ff24815a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42284/0 (socket says 172.21.15.133:42284) 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24df17640 1 -- 172.21.15.133:0/2489059024 learned_addr learned my addr 172.21.15.133:0/2489059024 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24d716640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2480afde0 0x7ff2480cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24e718640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff24815b8f0 0x7ff248154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24df17640 1 -- 172.21.15.133:0/2489059024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff24815b8f0 msgr2=0x7ff248154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24df17640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff24815b8f0 0x7ff248154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24df17640 1 -- 172.21.15.133:0/2489059024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2480afde0 msgr2=0x7ff2480cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24df17640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2480afde0 0x7ff2480cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24df17640 1 -- 172.21.15.133:0/2489059024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2480c7840 con 0x7ff24806ac90 2024-04-25T10:07:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24e718640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff24815b8f0 0x7ff248154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff24df17640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24806ac90 0x7ff24815a9c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff24400c890 tx=0x7ff24400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff23e7fc640 1 -- 172.21.15.133:0/2489059024 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244018070 con 0x7ff24806ac90 2024-04-25T10:07:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff23e7fc640 1 -- 172.21.15.133:0/2489059024 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24400a640 con 0x7ff24806ac90 2024-04-25T10:07:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 -- 172.21.15.133:0/2489059024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2480b30f0 con 0x7ff24806ac90 2024-04-25T10:07:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff23e7fc640 1 -- 172.21.15.133:0/2489059024 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24400aa30 con 0x7ff24806ac90 2024-04-25T10:07:04.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.225+0000 7ff25017b640 1 -- 172.21.15.133:0/2489059024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24815e7a0 con 0x7ff24806ac90 2024-04-25T10:07:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff23e7fc640 1 -- 172.21.15.133:0/2489059024 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff24401b070 con 0x7ff24806ac90 2024-04-25T10:07:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff23e7fc640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff22c066f60 0x7ff22c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff23e7fc640 1 -- 172.21.15.133:0/2489059024 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff2440849d0 con 0x7ff24806ac90 2024-04-25T10:07:04.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24d716640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff22c066f60 0x7ff22c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24d716640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff22c066f60 0x7ff22c069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff2481f97d0 tx=0x7ff238008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff2532e8000 1 Processor -- start 2024-04-25T10:07:04.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff2532e8000 1 -- start start 2024-04-25T10:07:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf0e60 0x55d3e6cf1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6cf1780 0x55d3e6ca9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6caa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff2532e8000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6b8ec70 con 0x55d3e6ca9d80 2024-04-25T10:07:04.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff2532e8000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6af5040 con 0x55d3e6cf1780 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff2532e8000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6a7d040 con 0x55d3e6cf0e60 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24e718640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6caa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24e718640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6caa160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42294/0 (socket says 172.21.15.133:42294) 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24e718640 1 -- 172.21.15.133:0/366025824 learned_addr learned my addr 172.21.15.133:0/366025824 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24e718640 1 -- 172.21.15.133:0/366025824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf0e60 msgr2=0x55d3e6cf1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24d716640 1 --2- 172.21.15.133:0/366025824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6cf1780 0x55d3e6ca9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24e718640 1 --2- 172.21.15.133:0/366025824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf0e60 0x55d3e6cf1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24e718640 1 -- 172.21.15.133:0/366025824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6cf1780 msgr2=0x55d3e6ca9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24e718640 1 --2- 172.21.15.133:0/366025824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6cf1780 0x55d3e6ca9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24e718640 1 -- 172.21.15.133:0/366025824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e6aac240 con 0x55d3e6ca9d80 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff24e718640 1 --2- 172.21.15.133:0/366025824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6caa160 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff24001c450 tx=0x7ff240004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8907429e533273c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff22b7fe640 1 -- 172.21.15.133:0/366025824 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2400b0040 con 0x55d3e6ca9d80 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff22b7fe640 1 -- 172.21.15.133:0/366025824 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2400ab700 con 0x55d3e6ca9d80 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.229+0000 7ff22b7fe640 1 -- 172.21.15.133:0/366025824 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2400abab0 con 0x55d3e6ca9d80 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- 172.21.15.133:0/366025824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 msgr2=0x55d3e6caa160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 --2- 172.21.15.133:0/366025824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6caa160 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff24001c450 tx=0x7ff240004740 comp rx=0 tx=0).stop 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- 172.21.15.133:0/366025824 shutdown_connections 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 --2- 172.21.15.133:0/366025824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6caa160 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 --2- 172.21.15.133:0/366025824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6cf1780 0x55d3e6ca9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 --2- 172.21.15.133:0/366025824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf0e60 0x55d3e6cf1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- 172.21.15.133:0/366025824 >> 172.21.15.133:0/366025824 conn(0x55d3e6ca1100 msgr2=0x55d3e6ca8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- 172.21.15.133:0/366025824 shutdown_connections 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- 172.21.15.133:0/366025824 wait complete. 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- start start 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 Processor -- start 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6ca9d80 0x55d3e6cb2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf0e60 0x55d3e6cb2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf1780 0x55d3e6cb2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6aaf540 con 0x55d3e6cf1780 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6af10e0 con 0x55d3e6ca9d80 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6af4350 con 0x55d3e6cf0e60 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 10 client.0 _ll_register_callbacks cb 0x55d3e6ba93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24e718640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf1780 0x55d3e6cb2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24e718640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf1780 0x55d3e6cb2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42308/0 (socket says 172.21.15.133:42308) 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24e718640 1 -- 172.21.15.133:0/352212834 learned_addr learned my addr 172.21.15.133:0/352212834 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24df17640 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6ca9d80 0x55d3e6cb2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24d716640 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf0e60 0x55d3e6cb2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24e718640 1 -- 172.21.15.133:0/352212834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf0e60 msgr2=0x55d3e6cb2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24e718640 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf0e60 0x55d3e6cb2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24e718640 1 -- 172.21.15.133:0/352212834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6ca9d80 msgr2=0x55d3e6cb2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24e718640 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6ca9d80 0x55d3e6cb2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24e718640 1 -- 172.21.15.133:0/352212834 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3e6abd410 con 0x55d3e6cf1780 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24df17640 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6ca9d80 0x55d3e6cb2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24e718640 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf1780 0x55d3e6cb2c90 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff240008010 tx=0x7ff24001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f095c3062e22b11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff228ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff228ff9640 1 -- 172.21.15.133:0/352212834 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2400b0040 con 0x55d3e6cf1780 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff228ff9640 1 -- 172.21.15.133:0/352212834 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff2400b6440 con 0x55d3e6cf1780 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- 172.21.15.133:0/352212834 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2400ab700 con 0x55d3e6cf1780 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff228ff9640 1 -- 172.21.15.133:0/352212834 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2400d0050 con 0x55d3e6cf1780 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 10 client.7236 fetch_fsmap learned FSMap version 41 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- 172.21.15.133:0/352212834 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55d3e6a7d290 con 0x55d3e6cf1780 2024-04-25T10:07:04.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff228ff9640 1 -- 172.21.15.133:0/352212834 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2400ab700 con 0x55d3e6cf1780 2024-04-25T10:07:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff228ff9640 1 -- 172.21.15.133:0/352212834 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff2400d3020 con 0x55d3e6cf1780 2024-04-25T10:07:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 10 client.7236 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T10:07:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 10 client.7236 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:07:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 20 client.7236 populate_metadata read hostname 'smithi133' 2024-04-25T10:07:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55d3e6cb3530 0x55d3e6d025e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 4 client.7236 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff2532e8000 1 -- 172.21.15.133:0/352212834 --> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3e6a90d70 con 0x55d3e6cb3530 2024-04-25T10:07:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.233+0000 7ff24df17640 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55d3e6cb3530 0x55d3e6d025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff228ff9640 10 client.7236 ms_handle_connect on v2:172.21.15.133:6836/2310040121 2024-04-25T10:07:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff24df17640 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55d3e6cb3530 0x55d3e6d025e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87f2ea2b1a0e086c server_cookie=e017d0d176c10769 in_seq=0 out_seq=0 2024-04-25T10:07:04.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff228ff9640 1 -- 172.21.15.133:0/352212834 <== mds.0 v2:172.21.15.133:6836/2310040121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d3e6a90d70 con 0x55d3e6cb3530 2024-04-25T10:07:04.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff228ff9640 10 client.7236 handle_command_reply: tid=0 2024-04-25T10:07:04.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 1 client.7236 shutdown 2024-04-25T10:07:04.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 1 -- 172.21.15.133:0/352212834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf1780 msgr2=0x55d3e6cb2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf1780 0x55d3e6cb2c90 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff240008010 tx=0x7ff24001ac50 comp rx=0 tx=0).stop 2024-04-25T10:07:04.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 1 -- 172.21.15.133:0/352212834 shutdown_connections 2024-04-25T10:07:04.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55d3e6cb3530 0x55d3e6d025e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf1780 0x55d3e6cb2c90 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf0e60 0x55d3e6cb2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 1 --2- 172.21.15.133:0/352212834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6ca9d80 0x55d3e6cb2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff228ff9640 0 client.7236 ms_handle_reset on v2:172.21.15.133:6836/2310040121 2024-04-25T10:07:04.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 1 -- 172.21.15.133:0/352212834 >> 172.21.15.133:0/352212834 conn(0x55d3e6ca1100 msgr2=0x55d3e6ca8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 1 -- 172.21.15.133:0/352212834 shutdown_connections 2024-04-25T10:07:04.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 1 -- 172.21.15.133:0/352212834 wait complete. 2024-04-25T10:07:04.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.237+0000 7ff2532e8000 20 client.7236 trim_cache size 0 max 16384 2024-04-25T10:07:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 Processor -- start 2024-04-25T10:07:04.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- start start 2024-04-25T10:07:04.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6d015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6cf0e60 0x55d3e6d01b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf1780 0x55d3e6d02060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff248203190 con 0x55d3e6ca9d80 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff248202370 con 0x55d3e6cf0e60 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24815e0c0 con 0x55d3e6cf1780 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24df17640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6d015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24df17640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6d015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42310/0 (socket says 172.21.15.133:42310) 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24df17640 1 -- 172.21.15.133:0/39779369 learned_addr learned my addr 172.21.15.133:0/39779369 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24df17640 1 -- 172.21.15.133:0/39779369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf1780 msgr2=0x55d3e6d02060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24df17640 1 --2- 172.21.15.133:0/39779369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf1780 0x55d3e6d02060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24df17640 1 -- 172.21.15.133:0/39779369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6cf0e60 msgr2=0x55d3e6d01b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24df17640 1 --2- 172.21.15.133:0/39779369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6cf0e60 0x55d3e6d01b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24df17640 1 -- 172.21.15.133:0/39779369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e6b77890 con 0x55d3e6ca9d80 2024-04-25T10:07:04.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24df17640 1 --2- 172.21.15.133:0/39779369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6d015e0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff244054fa0 tx=0x7ff2440013f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820df84b3e1a77a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff22affd640 1 -- 172.21.15.133:0/39779369 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244053310 con 0x55d3e6ca9d80 2024-04-25T10:07:04.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff22affd640 1 -- 172.21.15.133:0/39779369 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2440730b0 con 0x55d3e6ca9d80 2024-04-25T10:07:04.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff22affd640 1 -- 172.21.15.133:0/39779369 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24405d320 con 0x55d3e6ca9d80 2024-04-25T10:07:04.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- 172.21.15.133:0/39779369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 msgr2=0x55d3e6d015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 --2- 172.21.15.133:0/39779369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6d015e0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff244054fa0 tx=0x7ff2440013f0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- 172.21.15.133:0/39779369 shutdown_connections 2024-04-25T10:07:04.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 --2- 172.21.15.133:0/39779369 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf1780 0x55d3e6d02060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 --2- 172.21.15.133:0/39779369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6cf0e60 0x55d3e6d01b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 --2- 172.21.15.133:0/39779369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6ca9d80 0x55d3e6d015e0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- 172.21.15.133:0/39779369 >> 172.21.15.133:0/39779369 conn(0x55d3e6ca1100 msgr2=0x55d3e6caecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- 172.21.15.133:0/39779369 shutdown_connections 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- 172.21.15.133:0/39779369 wait complete. 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- start start 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 Processor -- start 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6ca9d80 0x55d3e6d04870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf0e60 0x55d3e6d0d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf1780 0x55d3e6d0d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6af9670 con 0x55d3e6cf0e60 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff248201d70 con 0x55d3e6ca9d80 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3e6a7db50 con 0x55d3e6cf1780 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff2532e8000 10 client.0 _ll_register_callbacks cb 0x7ff2400ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24d716640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf0e60 0x55d3e6d0d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24d716640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf0e60 0x55d3e6d0d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42318/0 (socket says 172.21.15.133:42318) 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.253+0000 7ff24d716640 1 -- 172.21.15.133:0/326274732 learned_addr learned my addr 172.21.15.133:0/326274732 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:04.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff24df17640 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6ca9d80 0x55d3e6d04870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff24e718640 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf1780 0x55d3e6d0d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff24d716640 1 -- 172.21.15.133:0/326274732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf1780 msgr2=0x55d3e6d0d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff24d716640 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf1780 0x55d3e6d0d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff24d716640 1 -- 172.21.15.133:0/326274732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6ca9d80 msgr2=0x55d3e6d04870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff24d716640 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6ca9d80 0x55d3e6d04870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff24d716640 1 -- 172.21.15.133:0/326274732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3e6a89e50 con 0x55d3e6cf0e60 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff24d716640 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf0e60 0x55d3e6d0d0b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff2380090e0 tx=0x7ff238009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be95468d7a575d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff228ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff228ff9640 1 -- 172.21.15.133:0/326274732 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238024030 con 0x55d3e6cf0e60 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff228ff9640 1 -- 172.21.15.133:0/326274732 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff23800f050 con 0x55d3e6cf0e60 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 -- 172.21.15.133:0/326274732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2440730b0 con 0x55d3e6cf0e60 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff228ff9640 1 -- 172.21.15.133:0/326274732 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff238030050 con 0x55d3e6cf0e60 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 10 client.7242 fetch_fsmap learned FSMap version 41 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 -- 172.21.15.133:0/326274732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55d3e6a9bf80 con 0x55d3e6cf0e60 2024-04-25T10:07:04.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff228ff9640 1 -- 172.21.15.133:0/326274732 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2440730b0 con 0x55d3e6cf0e60 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff228ff9640 1 -- 172.21.15.133:0/326274732 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff238033020 con 0x55d3e6cf0e60 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 10 client.7242 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 10 client.7242 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 20 client.7242 populate_metadata read hostname 'smithi133' 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55d3e6d50b10 0x55d3e6d50ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 4 client.7242 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 -- 172.21.15.133:0/326274732 --> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x55d3e6b02580 con 0x55d3e6d50b10 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff24df17640 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55d3e6d50b10 0x55d3e6d50ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff228ff9640 10 client.7242 ms_handle_connect on v2:172.21.15.133:6836/2310040121 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff24df17640 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55d3e6d50b10 0x55d3e6d50ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b30ed258d7ef1d96 server_cookie=3cd393c1d9c5bdc5 in_seq=0 out_seq=0 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff228ff9640 1 -- 172.21.15.133:0/326274732 <== mds.0 v2:172.21.15.133:6836/2310040121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d3e6b02580 con 0x55d3e6d50b10 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff228ff9640 10 client.7242 handle_command_reply: tid=0 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 client.7242 shutdown 2024-04-25T10:07:04.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 -- 172.21.15.133:0/326274732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf0e60 msgr2=0x55d3e6d0d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf0e60 0x55d3e6d0d0b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff2380090e0 tx=0x7ff238009190 comp rx=0 tx=0).stop 2024-04-25T10:07:04.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 -- 172.21.15.133:0/326274732 shutdown_connections 2024-04-25T10:07:04.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.133:6836/2310040121,v1:172.21.15.133:6837/2310040121] conn(0x55d3e6d50b10 0x55d3e6d50ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d3e6cf1780 0x55d3e6d0d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d3e6cf0e60 0x55d3e6d0d0b0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 --2- 172.21.15.133:0/326274732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d3e6ca9d80 0x55d3e6d04870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff228ff9640 0 client.7242 ms_handle_reset on v2:172.21.15.133:6836/2310040121 2024-04-25T10:07:04.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 -- 172.21.15.133:0/326274732 >> 172.21.15.133:0/326274732 conn(0x55d3e6ca1100 msgr2=0x55d3e6d005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 -- 172.21.15.133:0/326274732 shutdown_connections 2024-04-25T10:07:04.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 1 -- 172.21.15.133:0/326274732 wait complete. 2024-04-25T10:07:04.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff2532e8000 20 client.7242 trim_cache size 0 max 16384 2024-04-25T10:07:04.268 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:07:04.268 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:07:04.268 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "88eb4598-9027-49cb-a3b2-55e9fdd25541", 2024-04-25T10:07:04.268 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:07:04.268 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 -- 172.21.15.133:0/2489059024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff22c066f60 msgr2=0x7ff22c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff22c066f60 0x7ff22c069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff2481f97d0 tx=0x7ff238008040 comp rx=0 tx=0).stop 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 -- 172.21.15.133:0/2489059024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24806ac90 msgr2=0x7ff24815a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24806ac90 0x7ff24815a9c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff24400c890 tx=0x7ff24400cd50 comp rx=0 tx=0).stop 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 -- 172.21.15.133:0/2489059024 shutdown_connections 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff24815b8f0 0x7ff248154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2480afde0 0x7ff2480cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff22c066f60 0x7ff22c069420 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 --2- 172.21.15.133:0/2489059024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24806ac90 0x7ff24815a9c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 -- 172.21.15.133:0/2489059024 >> 172.21.15.133:0/2489059024 conn(0x7ff2480726f0 msgr2=0x7ff2481658f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 -- 172.21.15.133:0/2489059024 shutdown_connections 2024-04-25T10:07:04.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:04.257+0000 7ff25017b640 1 -- 172.21.15.133:0/2489059024 wait complete. 2024-04-25T10:07:04.280 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': '88eb4598-9027-49cb-a3b2-55e9fdd25541', 'mode': 'asynchronous'}' 2024-04-25T10:07:14.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:14.281 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T10:07:14.295 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:07:14.296 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:14.378 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:14.436 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:07:14.436 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:07:14.436 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:14.442 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03ae3e4640 1 Processor -- start 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03ae3e4640 1 -- start start 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03ae3e4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03a81584f0 0x7f03a80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03ae3e4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a80cbe60 con 0x7f03a8158110 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03ae3e4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a80b19b0 con 0x7f03a806eb70 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03ae3e4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a815b360 con 0x7f03a81584f0 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03ac981640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03a81584f0 0x7f03a80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03a7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f03a8158110 0x7f03a806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37568/0 (socket says 172.21.15.133:37568) 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03a7fff640 1 -- 172.21.15.133:0/2872702620 learned_addr learned my addr 172.21.15.133:0/2872702620 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03a6ffd640 1 -- 172.21.15.133:0/2872702620 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253939307 0 0) 0x7f03a80cbe60 con 0x7f03a8158110 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03a6ffd640 1 -- 172.21.15.133:0/2872702620 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0390003530 con 0x7f03a8158110 2024-04-25T10:07:14.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03ac981640 1 -- 172.21.15.133:0/2872702620 >> v1:172.21.15.162:6789/0 conn(0x7f03a806eb70 legacy=0x7f03a80cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03ac981640 1 -- 172.21.15.133:0/2872702620 >> v1:172.21.15.133:6789/0 conn(0x7f03a8158110 legacy=0x7f03a806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03ac981640 1 -- 172.21.15.133:0/2872702620 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a815b640 con 0x7f03a81584f0 2024-04-25T10:07:14.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.609+0000 7f03ac981640 1 --2- 172.21.15.133:0/2872702620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a81584f0 0x7f03a80b1f00 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009d90 tx=0x7f03a001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b02af022e6835a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a6ffd640 1 -- 172.21.15.133:0/2872702620 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a00b2070 con 0x7f03a81584f0 2024-04-25T10:07:14.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a6ffd640 1 -- 172.21.15.133:0/2872702620 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a00a58d0 con 0x7f03a81584f0 2024-04-25T10:07:14.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a6ffd640 1 -- 172.21.15.133:0/2872702620 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a00a5ca0 con 0x7f03a81584f0 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 -- 172.21.15.133:0/2872702620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a81584f0 msgr2=0x7f03a80b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 --2- 172.21.15.133:0/2872702620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a81584f0 0x7f03a80b1f00 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009d90 tx=0x7f03a001c900 comp rx=0 tx=0).stop 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 -- 172.21.15.133:0/2872702620 shutdown_connections 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 --2- 172.21.15.133:0/2872702620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a81584f0 0x7f03a80b1f00 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 -- 172.21.15.133:0/2872702620 >> 172.21.15.133:0/2872702620 conn(0x7f03a8067000 msgr2=0x7f03a806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 -- 172.21.15.133:0/2872702620 shutdown_connections 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 -- 172.21.15.133:0/2872702620 wait complete. 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 Processor -- start 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 -- start start 2024-04-25T10:07:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03a806eb70 0x7f03a81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a8158110 0x7f03a8139c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a81584f0 0x7f03a813a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8168600 con 0x7f03a8158110 2024-04-25T10:07:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8168480 con 0x7f03a81584f0 2024-04-25T10:07:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8168780 con 0x7f03a806eb70 2024-04-25T10:07:14.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a8158110 0x7f03a8139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a8158110 0x7f03a8139c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51248/0 (socket says 172.21.15.133:51248) 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a77fe640 1 -- 172.21.15.133:0/3197173953 learned_addr learned my addr 172.21.15.133:0/3197173953 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a7fff640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03a806eb70 0x7f03a81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a77fe640 1 -- 172.21.15.133:0/3197173953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03a806eb70 msgr2=0x7f03a81555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ac981640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a81584f0 0x7f03a813a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a77fe640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03a806eb70 0x7f03a81555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a77fe640 1 -- 172.21.15.133:0/3197173953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a81584f0 msgr2=0x7f03a813a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a77fe640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a81584f0 0x7f03a813a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a77fe640 1 -- 172.21.15.133:0/3197173953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a8157820 con 0x7f03a8158110 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a77fe640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a8158110 0x7f03a8139c30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f039c00c440 tx=0x7f039c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a4ff9640 1 -- 172.21.15.133:0/3197173953 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c016020 con 0x7f03a8158110 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a4ff9640 1 -- 172.21.15.133:0/3197173953 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c012830 con 0x7f03a8158110 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 -- 172.21.15.133:0/3197173953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a8155dc0 con 0x7f03a8158110 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03a4ff9640 1 -- 172.21.15.133:0/3197173953 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c012c20 con 0x7f03a8158110 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.613+0000 7f03ae3e4640 1 -- 172.21.15.133:0/3197173953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a8155ae0 con 0x7f03a8158110 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.617+0000 7f03ae3e4640 1 -- 172.21.15.133:0/3197173953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a806bc20 con 0x7f03a8158110 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.617+0000 7f03a4ff9640 1 -- 172.21.15.133:0/3197173953 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f039c019020 con 0x7f03a8158110 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.617+0000 7f03a4ff9640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0394066f60 0x7f0394069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.617+0000 7f03a4ff9640 1 -- 172.21.15.133:0/3197173953 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f039c085220 con 0x7f03a8158110 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.617+0000 7f03a7fff640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0394066f60 0x7f0394069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.617+0000 7f03a7fff640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0394066f60 0x7f0394069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0398007fd0 tx=0x7f0398009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.621+0000 7f03a4ff9640 1 -- 172.21.15.133:0/3197173953 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f039c020020 con 0x7f03a8158110 2024-04-25T10:07:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.777+0000 7f03ae3e4640 1 -- 172.21.15.133:0/3197173953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03a8068ef0 con 0x7f03a8158110 2024-04-25T10:07:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.781+0000 7f03a4ff9640 1 -- 172.21.15.133:0/3197173953 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17167 (secure 0 0 0) 0x7f039c0182c0 con 0x7f03a8158110 2024-04-25T10:07:14.786 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:14.786 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":70,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:06:56.893382+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:34.764755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:35.504408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6835/272966073":"2024-04-26T10:06:56.893338+0000","172.21.15.162:6833/272966073":"2024-04-26T10:06:56.893338+0000","172.21.15.133:6837/236214118":"2024-04-26T10:06:55.901820+0000","172.21.15.133:6836/236214118":"2024-04-26T10:06:55.901820+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 -- 172.21.15.133:0/3197173953 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0394066f60 msgr2=0x7f0394069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0394066f60 0x7f0394069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0398007fd0 tx=0x7f0398009040 comp rx=0 tx=0).stop 2024-04-25T10:07:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 -- 172.21.15.133:0/3197173953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a8158110 msgr2=0x7f03a8139c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a8158110 0x7f03a8139c30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f039c00c440 tx=0x7f039c00c900 comp rx=0 tx=0).stop 2024-04-25T10:07:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 -- 172.21.15.133:0/3197173953 shutdown_connections 2024-04-25T10:07:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03a81584f0 0x7f03a813a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0394066f60 0x7f0394069420 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a8158110 0x7f03a8139c30 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 --2- 172.21.15.133:0/3197173953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03a806eb70 0x7f03a81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 -- 172.21.15.133:0/3197173953 >> 172.21.15.133:0/3197173953 conn(0x7f03a8067000 msgr2=0x7f03a8153f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 -- 172.21.15.133:0/3197173953 shutdown_connections 2024-04-25T10:07:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.785+0000 7f03ae3e4640 1 -- 172.21.15.133:0/3197173953 wait complete. 2024-04-25T10:07:14.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:34.764755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:35.504408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:14.806 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac810ef640 1 Processor -- start 2024-04-25T10:07:14.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac810ef640 1 -- start start 2024-04-25T10:07:14.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac810ef640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fac7c0af0e0 0x7fac7c0acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac810ef640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac7c0ad460 0x7fac7c151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac810ef640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7c15bd30 con 0x7fac7c15f580 2024-04-25T10:07:14.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac810ef640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac7c0ad8d0 con 0x7fac7c0af0e0 2024-04-25T10:07:14.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac810ef640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c15bff0 con 0x7fac7c0ad460 2024-04-25T10:07:14.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7ad9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fac7c15f580 0x7fac7c15f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37584/0 (socket says 172.21.15.133:37584) 2024-04-25T10:07:14.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7ad9d640 1 -- 172.21.15.133:0/2687928489 learned_addr learned my addr 172.21.15.133:0/2687928489 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:14.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7b59e640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac7c0ad460 0x7fac7c151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7a59c640 1 --2- 172.21.15.133:0/2687928489 >> v2:172.21.15.162:3300/0 conn(0x7fac7c0af0e0 0x7fac7c0acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820959105 0 0) 0x7fac7c15bd30 con 0x7fac7c15f580 2024-04-25T10:07:14.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac64003530 con 0x7fac7c15f580 2024-04-25T10:07:14.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3600468704 0 0) 0x7fac64003530 con 0x7fac7c15f580 2024-04-25T10:07:14.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac7c0ad460 msgr2=0x7fac7c151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac7c0ad460 0x7fac7c151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 >> v2:172.21.15.162:3300/0 conn(0x7fac7c0af0e0 msgr2=0x7fac7c0acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 --2- 172.21.15.133:0/2687928489 >> v2:172.21.15.162:3300/0 conn(0x7fac7c0af0e0 0x7fac7c0acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c15c2d0 con 0x7fac7c15f580 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fac6c0016a0 con 0x7fac7c15f580 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 >> v1:172.21.15.133:6789/0 conn(0x7fac7c15f580 legacy=0x7fac7c15f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac7c0ad460 0x7fac640039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac64003f40 0x7fac64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac64004d50 0x7fac640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c0ad8d0 con 0x7fac64003f40 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c15bff0 con 0x7fac64004d50 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6400ba20 con 0x7fac7c0ad460 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7a59c640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac64003f40 0x7fac64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7ad9d640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac7c0ad460 0x7fac640039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7b59e640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac64004d50 0x7fac640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7a59c640 1 -- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac7c0ad460 msgr2=0x7fac640039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7a59c640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac7c0ad460 0x7fac640039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7a59c640 1 -- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac64004d50 msgr2=0x7fac640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7a59c640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac64004d50 0x7fac640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7a59c640 1 -- 172.21.15.133:0/2687928489 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac64003530 con 0x7fac64003f40 2024-04-25T10:07:14.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.925+0000 7fac7b59e640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac64004d50 0x7fac640051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:14.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac7a59c640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac64003f40 0x7fac64004810 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fac70009b10 tx=0x7fac7001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b701afc3c370e391 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac700b2070 con 0x7fac64003f40 2024-04-25T10:07:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac700a5650 con 0x7fac64003f40 2024-04-25T10:07:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac79d9b640 1 -- 172.21.15.133:0/2687928489 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac700096b0 con 0x7fac64003f40 2024-04-25T10:07:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac810ef640 1 -- 172.21.15.133:0/2687928489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac64003f40 msgr2=0x7fac64004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac810ef640 1 --2- 172.21.15.133:0/2687928489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac64003f40 0x7fac64004810 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fac70009b10 tx=0x7fac7001ca80 comp rx=0 tx=0).stop 2024-04-25T10:07:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac7b59e640 1 -- 172.21.15.133:0/2687928489 reap_dead start 2024-04-25T10:07:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac810ef640 1 -- 172.21.15.133:0/2687928489 shutdown_connections 2024-04-25T10:07:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac810ef640 1 -- 172.21.15.133:0/2687928489 >> 172.21.15.133:0/2687928489 conn(0x7fac7c0cfd50 msgr2=0x7fac7c0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac810ef640 1 -- 172.21.15.133:0/2687928489 shutdown_connections 2024-04-25T10:07:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac810ef640 1 -- 172.21.15.133:0/2687928489 wait complete. 2024-04-25T10:07:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac810ef640 1 Processor -- start 2024-04-25T10:07:14.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.929+0000 7fac810ef640 1 -- start start 2024-04-25T10:07:14.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac810ef640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac64003f40 0x7fac7c140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac810ef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac7c09a3e0 0x7fac7c09c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac810ef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac7c09b940 0x7fac7c1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac810ef640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c1601f0 con 0x7fac7c09a3e0 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac810ef640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c160070 con 0x7fac7c09b940 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac810ef640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c1527b0 con 0x7fac64003f40 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac7c09a3e0 0x7fac7c09c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac7c09a3e0 0x7fac7c09c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51266/0 (socket says 172.21.15.133:51266) 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7a59c640 1 -- 172.21.15.133:0/3557081787 learned_addr learned my addr 172.21.15.133:0/3557081787 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7b59e640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac7c09b940 0x7fac7c1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7ad9d640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac64003f40 0x7fac7c140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7a59c640 1 -- 172.21.15.133:0/3557081787 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac64003f40 msgr2=0x7fac7c140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7a59c640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac64003f40 0x7fac7c140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7a59c640 1 -- 172.21.15.133:0/3557081787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac7c09b940 msgr2=0x7fac7c1504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7a59c640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac7c09b940 0x7fac7c1504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7a59c640 1 -- 172.21.15.133:0/3557081787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c09a8a0 con 0x7fac7c09a3e0 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7b59e640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac7c09b940 0x7fac7c1504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:14.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7a59c640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac7c09a3e0 0x7fac7c09c5f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fac7001c8b0 tx=0x7fac64005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac637fe640 1 -- 172.21.15.133:0/3557081787 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac700bc030 con 0x7fac7c09a3e0 2024-04-25T10:07:14.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac637fe640 1 -- 172.21.15.133:0/3557081787 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac700a5e20 con 0x7fac7c09a3e0 2024-04-25T10:07:14.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac810ef640 1 -- 172.21.15.133:0/3557081787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7c09ab80 con 0x7fac7c09a3e0 2024-04-25T10:07:14.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac637fe640 1 -- 172.21.15.133:0/3557081787 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac700b2040 con 0x7fac7c09a3e0 2024-04-25T10:07:14.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac810ef640 1 -- 172.21.15.133:0/3557081787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7c09b140 con 0x7fac7c09a3e0 2024-04-25T10:07:14.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac810ef640 1 -- 172.21.15.133:0/3557081787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7c067c70 con 0x7fac7c09a3e0 2024-04-25T10:07:14.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac637fe640 1 -- 172.21.15.133:0/3557081787 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac700bf070 con 0x7fac7c09a3e0 2024-04-25T10:07:14.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac637fe640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac68066f60 0x7fac68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac637fe640 1 -- 172.21.15.133:0/3557081787 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fac7011f010 con 0x7fac7c09a3e0 2024-04-25T10:07:14.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7ad9d640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac68066f60 0x7fac68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.933+0000 7fac7ad9d640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac68066f60 0x7fac68069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c10 tx=0x7fac6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:14.937+0000 7fac637fe640 1 -- 172.21.15.133:0/3557081787 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac700c7020 con 0x7fac7c09a3e0 2024-04-25T10:07:15.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.097+0000 7fac810ef640 1 -- 172.21.15.133:0/3557081787 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac7c14fb70 con 0x7fac7c09a3e0 2024-04-25T10:07:15.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac637fe640 1 -- 172.21.15.133:0/3557081787 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac70016800 con 0x7fac7c09a3e0 2024-04-25T10:07:15.104 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:07:15.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 -- 172.21.15.133:0/3557081787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac68066f60 msgr2=0x7fac68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac68066f60 0x7fac68069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c10 tx=0x7fac6c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:15.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 -- 172.21.15.133:0/3557081787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac7c09a3e0 msgr2=0x7fac7c09c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac7c09a3e0 0x7fac7c09c5f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fac7001c8b0 tx=0x7fac64005840 comp rx=0 tx=0).stop 2024-04-25T10:07:15.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 -- 172.21.15.133:0/3557081787 shutdown_connections 2024-04-25T10:07:15.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac7c09b940 0x7fac7c1504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac68066f60 0x7fac68069420 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac7c09a3e0 0x7fac7c09c5f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 --2- 172.21.15.133:0/3557081787 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac64003f40 0x7fac7c140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 -- 172.21.15.133:0/3557081787 >> 172.21.15.133:0/3557081787 conn(0x7fac7c0cfd50 msgr2=0x7fac7c15b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 -- 172.21.15.133:0/3557081787 shutdown_connections 2024-04-25T10:07:15.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.101+0000 7fac810ef640 1 -- 172.21.15.133:0/3557081787 wait complete. 2024-04-25T10:07:15.123 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:15.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff448df5640 1 Processor -- start 2024-04-25T10:07:15.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff448df5640 1 -- start start 2024-04-25T10:07:15.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff448df5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff444145170 0x7ff44406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff448df5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff44406e4d0 0x7ff4440cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff448df5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4440ce2e0 con 0x7ff4440cdd60 2024-04-25T10:07:15.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff448df5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44406db00 con 0x7ff444145170 2024-04-25T10:07:15.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff448df5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff444145550 con 0x7ff44406e4d0 2024-04-25T10:07:15.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff44259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff444145170 0x7ff44406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff44259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff444145170 0x7ff44406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51282/0 (socket says 172.21.15.133:51282) 2024-04-25T10:07:15.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff442d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff4440cdd60 0x7ff44415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37590/0 (socket says 172.21.15.133:37590) 2024-04-25T10:07:15.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff44259c640 1 -- 172.21.15.133:0/1589993082 learned_addr learned my addr 172.21.15.133:0/1589993082 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:15.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff441d9b640 1 --2- 172.21.15.133:0/1589993082 >> v2:172.21.15.162:3300/0 conn(0x7ff44406e4d0 0x7ff4440cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff44259c640 1 -- 172.21.15.133:0/1589993082 >> v2:172.21.15.162:3300/0 conn(0x7ff44406e4d0 msgr2=0x7ff4440cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff44259c640 1 --2- 172.21.15.133:0/1589993082 >> v2:172.21.15.162:3300/0 conn(0x7ff44406e4d0 0x7ff4440cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff44259c640 1 -- 172.21.15.133:0/1589993082 >> v1:172.21.15.133:6789/0 conn(0x7ff4440cdd60 legacy=0x7ff44415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff44259c640 1 -- 172.21.15.133:0/1589993082 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44406d4b0 con 0x7ff444145170 2024-04-25T10:07:15.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.245+0000 7ff44159a640 1 -- 172.21.15.133:0/1589993082 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886639984 0 0) 0x7ff4440ce2e0 con 0x7ff4440cdd60 2024-04-25T10:07:15.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff44259c640 1 --2- 172.21.15.133:0/1589993082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff444145170 0x7ff44406df90 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff4440b4dd0 tx=0x7ff43401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc91d3bba1843d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff44159a640 1 -- 172.21.15.133:0/1589993082 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4340b0070 con 0x7ff444145170 2024-04-25T10:07:15.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff44159a640 1 -- 172.21.15.133:0/1589993082 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4340ab600 con 0x7ff444145170 2024-04-25T10:07:15.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff44159a640 1 -- 172.21.15.133:0/1589993082 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4340ab9d0 con 0x7ff444145170 2024-04-25T10:07:15.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 -- 172.21.15.133:0/1589993082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff444145170 msgr2=0x7ff44406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 --2- 172.21.15.133:0/1589993082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff444145170 0x7ff44406df90 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff4440b4dd0 tx=0x7ff43401c490 comp rx=0 tx=0).stop 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 -- 172.21.15.133:0/1589993082 shutdown_connections 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 --2- 172.21.15.133:0/1589993082 >> v2:172.21.15.162:3300/0 conn(0x7ff44406e4d0 0x7ff4440cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 --2- 172.21.15.133:0/1589993082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff444145170 0x7ff44406df90 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 -- 172.21.15.133:0/1589993082 >> 172.21.15.133:0/1589993082 conn(0x7ff444066f70 msgr2=0x7ff44406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 -- 172.21.15.133:0/1589993082 shutdown_connections 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 -- 172.21.15.133:0/1589993082 wait complete. 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 Processor -- start 2024-04-25T10:07:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 -- start start 2024-04-25T10:07:15.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff44406e4d0 0x7ff4441329a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4440cdd60 0x7ff444151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff444145170 0x7ff4440b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff444168490 con 0x7ff4440cdd60 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44415d7f0 con 0x7ff444145170 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff448df5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff444168610 con 0x7ff44406e4d0 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff441d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4440cdd60 0x7ff444151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff441d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4440cdd60 0x7ff444151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51290/0 (socket says 172.21.15.133:51290) 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff441d9b640 1 -- 172.21.15.133:0/3223959743 learned_addr learned my addr 172.21.15.133:0/3223959743 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff442d9d640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff444145170 0x7ff4440b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff44259c640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff44406e4d0 0x7ff4441329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff441d9b640 1 -- 172.21.15.133:0/3223959743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff44406e4d0 msgr2=0x7ff4441329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff441d9b640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff44406e4d0 0x7ff4441329a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff441d9b640 1 -- 172.21.15.133:0/3223959743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff444145170 msgr2=0x7ff4440b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff441d9b640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff444145170 0x7ff4440b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff441d9b640 1 -- 172.21.15.133:0/3223959743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff444132f80 con 0x7ff4440cdd60 2024-04-25T10:07:15.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff44259c640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff44406e4d0 0x7ff4441329a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:15.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.249+0000 7ff441d9b640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4440cdd60 0x7ff444151f40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d800 tx=0x7ff43800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.253+0000 7ff432ffd640 1 -- 172.21.15.133:0/3223959743 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800b6c0 con 0x7ff4440cdd60 2024-04-25T10:07:15.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.253+0000 7ff432ffd640 1 -- 172.21.15.133:0/3223959743 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff438019070 con 0x7ff4440cdd60 2024-04-25T10:07:15.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.253+0000 7ff448df5640 1 -- 172.21.15.133:0/3223959743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff444133260 con 0x7ff4440cdd60 2024-04-25T10:07:15.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.253+0000 7ff432ffd640 1 -- 172.21.15.133:0/3223959743 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800ba80 con 0x7ff4440cdd60 2024-04-25T10:07:15.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.253+0000 7ff448df5640 1 -- 172.21.15.133:0/3223959743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff444133820 con 0x7ff4440cdd60 2024-04-25T10:07:15.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.253+0000 7ff448df5640 1 -- 172.21.15.133:0/3223959743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff444068e80 con 0x7ff4440cdd60 2024-04-25T10:07:15.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.253+0000 7ff432ffd640 1 -- 172.21.15.133:0/3223959743 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff43800bc20 con 0x7ff4440cdd60 2024-04-25T10:07:15.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.253+0000 7ff432ffd640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff424066f60 0x7ff424069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.253+0000 7ff432ffd640 1 -- 172.21.15.133:0/3223959743 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff438085630 con 0x7ff4440cdd60 2024-04-25T10:07:15.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.253+0000 7ff44259c640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff424066f60 0x7ff424069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.253+0000 7ff44259c640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff424066f60 0x7ff424069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff4340ad000 tx=0x7ff434008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.257+0000 7ff432ffd640 1 -- 172.21.15.133:0/3223959743 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff438056490 con 0x7ff4440cdd60 2024-04-25T10:07:15.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.413+0000 7ff448df5640 1 -- 172.21.15.133:0/3223959743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff44415de50 con 0x7ff4440cdd60 2024-04-25T10:07:15.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.413+0000 7ff432ffd640 1 -- 172.21.15.133:0/3223959743 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff438055d90 con 0x7ff4440cdd60 2024-04-25T10:07:15.419 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 -- 172.21.15.133:0/3223959743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff424066f60 msgr2=0x7ff424069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff424066f60 0x7ff424069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff4340ad000 tx=0x7ff434008040 comp rx=0 tx=0).stop 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 -- 172.21.15.133:0/3223959743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4440cdd60 msgr2=0x7ff444151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4440cdd60 0x7ff444151f40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d800 tx=0x7ff43800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 -- 172.21.15.133:0/3223959743 shutdown_connections 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff444145170 0x7ff4440b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff424066f60 0x7ff424069420 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4440cdd60 0x7ff444151f40 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 --2- 172.21.15.133:0/3223959743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff44406e4d0 0x7ff4441329a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 -- 172.21.15.133:0/3223959743 >> 172.21.15.133:0/3223959743 conn(0x7ff444066f70 msgr2=0x7ff44413dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 -- 172.21.15.133:0/3223959743 shutdown_connections 2024-04-25T10:07:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.417+0000 7ff448df5640 1 -- 172.21.15.133:0/3223959743 wait complete. 2024-04-25T10:07:15.438 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:15.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.545+0000 7fa5eeb03640 1 Processor -- start 2024-04-25T10:07:15.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.545+0000 7fa5eeb03640 1 -- start start 2024-04-25T10:07:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5eeb03640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa5e8142920 0x7fa5e8143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5eeb03640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e8140ea0 con 0x7fa5e815d9f0 2024-04-25T10:07:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5eeb03640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e80cf280 con 0x7fa5e8140a40 2024-04-25T10:07:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5eeb03640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e806d440 con 0x7fa5e8142920 2024-04-25T10:07:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5ed0a0640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa5e815d9f0 0x7fa5e806df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37592/0 (socket says 172.21.15.133:37592) 2024-04-25T10:07:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5ed0a0640 1 -- 172.21.15.133:0/2020297742 learned_addr learned my addr 172.21.15.133:0/2020297742 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5ec89f640 1 --2- 172.21.15.133:0/2020297742 >> v2:172.21.15.162:3300/0 conn(0x7fa5e8142920 0x7fa5e8143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645901854 0 0) 0x7fa5e8140ea0 con 0x7fa5e815d9f0 2024-04-25T10:07:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d0005740 con 0x7fa5e815d9f0 2024-04-25T10:07:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3734573054 0 0) 0x7fa5d0005740 con 0x7fa5e815d9f0 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 >> v2:172.21.15.162:3300/0 conn(0x7fa5e8142920 msgr2=0x7fa5e8143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 --2- 172.21.15.133:0/2020297742 >> v2:172.21.15.162:3300/0 conn(0x7fa5e8142920 0x7fa5e8143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 >> v1:172.21.15.162:6789/0 conn(0x7fa5e8140a40 legacy=0x7fa5e80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e806d720 con 0x7fa5e815d9f0 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa5e00016a0 con 0x7fa5e815d9f0 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 >> v1:172.21.15.133:6789/0 conn(0x7fa5e815d9f0 legacy=0x7fa5e806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 --2- 172.21.15.133:0/2020297742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5d0005bd0 0x7fa5d0005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 --2- 172.21.15.133:0/2020297742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d0006a20 0x7fa5d0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 --2- 172.21.15.133:0/2020297742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d00073e0 0x7fa5d000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e806d440 con 0x7fa5d0005bd0 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d000e380 con 0x7fa5d00073e0 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d000e580 con 0x7fa5d0006a20 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5ec89f640 1 --2- 172.21.15.133:0/2020297742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5d0005bd0 0x7fa5d0005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5ec89f640 1 -- 172.21.15.133:0/2020297742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d0006a20 msgr2=0x7fa5d0006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5ec89f640 1 --2- 172.21.15.133:0/2020297742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d0006a20 0x7fa5d0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5ec89f640 1 -- 172.21.15.133:0/2020297742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d00073e0 msgr2=0x7fa5d000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5ed0a0640 1 --2- 172.21.15.133:0/2020297742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d00073e0 0x7fa5d000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5ec89f640 1 --2- 172.21.15.133:0/2020297742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d00073e0 0x7fa5d000dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5ec89f640 1 -- 172.21.15.133:0/2020297742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d0005740 con 0x7fa5d0005bd0 2024-04-25T10:07:15.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.549+0000 7fa5ed0a0640 1 -- 172.21.15.133:0/2020297742 reap_dead start 2024-04-25T10:07:15.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ec89f640 1 --2- 172.21.15.133:0/2020297742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5d0005bd0 0x7fa5d0005ff0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009fc0 tx=0x7fa5d801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91f0ea5c77cad99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80b2070 con 0x7fa5d0005bd0 2024-04-25T10:07:15.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d80a5a70 con 0x7fa5d0005bd0 2024-04-25T10:07:15.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5e77fe640 1 -- 172.21.15.133:0/2020297742 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8009c10 con 0x7fa5d0005bd0 2024-04-25T10:07:15.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2020297742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5d0005bd0 msgr2=0x7fa5d0005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 --2- 172.21.15.133:0/2020297742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5d0005bd0 0x7fa5d0005ff0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009fc0 tx=0x7fa5d801cc80 comp rx=0 tx=0).stop 2024-04-25T10:07:15.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2020297742 shutdown_connections 2024-04-25T10:07:15.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 --2- 172.21.15.133:0/2020297742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5d0005bd0 0x7fa5d0005ff0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2020297742 >> 172.21.15.133:0/2020297742 conn(0x7fa5e8066f70 msgr2=0x7fa5e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2020297742 shutdown_connections 2024-04-25T10:07:15.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2020297742 wait complete. 2024-04-25T10:07:15.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 Processor -- start 2024-04-25T10:07:15.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 -- start start 2024-04-25T10:07:15.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d0005bd0 0x7fa5e81390b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5e813ff40 0x7fa5e8154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5e80b16f0 0x7fa5e81549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e815df50 con 0x7fa5e80b16f0 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e815ddd0 con 0x7fa5d0005bd0 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5eeb03640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e815e0d0 con 0x7fa5e813ff40 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ed0a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5e80b16f0 0x7fa5e81549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ed0a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5e80b16f0 0x7fa5e81549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51316/0 (socket says 172.21.15.133:51316) 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ed0a0640 1 -- 172.21.15.133:0/2018051119 learned_addr learned my addr 172.21.15.133:0/2018051119 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ec89f640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d0005bd0 0x7fa5e81390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5e7fff640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5e813ff40 0x7fa5e8154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ed0a0640 1 -- 172.21.15.133:0/2018051119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5e813ff40 msgr2=0x7fa5e8154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ed0a0640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5e813ff40 0x7fa5e8154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ed0a0640 1 -- 172.21.15.133:0/2018051119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d0005bd0 msgr2=0x7fa5e81390b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ed0a0640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d0005bd0 0x7fa5e81390b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ed0a0640 1 -- 172.21.15.133:0/2018051119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e80aca90 con 0x7fa5e80b16f0 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5e7fff640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5e813ff40 0x7fa5e8154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ec89f640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d0005bd0 0x7fa5e81390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.553+0000 7fa5ed0a0640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5e80b16f0 0x7fa5e81549a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80afa40 tx=0x7fa5e000d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.557+0000 7fa5e57fa640 1 -- 172.21.15.133:0/2018051119 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0007440 con 0x7fa5e80b16f0 2024-04-25T10:07:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.557+0000 7fa5e57fa640 1 -- 172.21.15.133:0/2018051119 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e0007600 con 0x7fa5e80b16f0 2024-04-25T10:07:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.557+0000 7fa5e57fa640 1 -- 172.21.15.133:0/2018051119 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0004a30 con 0x7fa5e80b16f0 2024-04-25T10:07:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.557+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2018051119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e80acd70 con 0x7fa5e80b16f0 2024-04-25T10:07:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.557+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2018051119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e80ad330 con 0x7fa5e80b16f0 2024-04-25T10:07:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.557+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2018051119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e8068e80 con 0x7fa5e80b16f0 2024-04-25T10:07:15.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.557+0000 7fa5e57fa640 1 -- 172.21.15.133:0/2018051119 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5e001b020 con 0x7fa5e80b16f0 2024-04-25T10:07:15.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.557+0000 7fa5e57fa640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5d4066f60 0x7fa5d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.557+0000 7fa5ec89f640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5d4066f60 0x7fa5d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.557+0000 7fa5e57fa640 1 -- 172.21.15.133:0/2018051119 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fa5e0084e00 con 0x7fa5e80b16f0 2024-04-25T10:07:15.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.557+0000 7fa5ec89f640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5d4066f60 0x7fa5d4069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa5d80af000 tx=0x7fa5d80a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.561+0000 7fa5e57fa640 1 -- 172.21.15.133:0/2018051119 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5e0055c00 con 0x7fa5e80b16f0 2024-04-25T10:07:15.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2018051119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5e80afa70 con 0x7fa5e80b16f0 2024-04-25T10:07:15.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5e57fa640 1 -- 172.21.15.133:0/2018051119 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa5e0055500 con 0x7fa5e80b16f0 2024-04-25T10:07:15.718 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:07:15.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2018051119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5d4066f60 msgr2=0x7fa5d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5eeb03640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5d4066f60 0x7fa5d4069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa5d80af000 tx=0x7fa5d80a7040 comp rx=0 tx=0).stop 2024-04-25T10:07:15.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2018051119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5e80b16f0 msgr2=0x7fa5e81549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5eeb03640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5e80b16f0 0x7fa5e81549a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80afa40 tx=0x7fa5e000d900 comp rx=0 tx=0).stop 2024-04-25T10:07:15.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2018051119 shutdown_connections 2024-04-25T10:07:15.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5eeb03640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5d4066f60 0x7fa5d4069420 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5eeb03640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5e80b16f0 0x7fa5e81549a0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5eeb03640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5e813ff40 0x7fa5e8154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5eeb03640 1 --2- 172.21.15.133:0/2018051119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d0005bd0 0x7fa5e81390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.713+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2018051119 >> 172.21.15.133:0/2018051119 conn(0x7fa5e8066f70 msgr2=0x7fa5e80b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.717+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2018051119 shutdown_connections 2024-04-25T10:07:15.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.717+0000 7fa5eeb03640 1 -- 172.21.15.133:0/2018051119 wait complete. 2024-04-25T10:07:15.736 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:15.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f7fcd640 1 Processor -- start 2024-04-25T10:07:15.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f7fcd640 1 -- start start 2024-04-25T10:07:15.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f7fcd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5f00ce600 0x7fc5f00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f7fcd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f015ebd0 con 0x7fc5f00d1920 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f7fcd640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f015eeb0 con 0x7fc5f015d9a0 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f7fcd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f00d0330 con 0x7fc5f00ce600 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f656a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc5f00d1920 0x7fc5f00d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37594/0 (socket says 172.21.15.133:37594) 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f656a640 1 -- 172.21.15.133:0/3829743242 learned_addr learned my addr 172.21.15.133:0/3829743242 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f5568640 1 --2- 172.21.15.133:0/3829743242 >> v2:172.21.15.162:3300/0 conn(0x7fc5f00ce600 0x7fc5f00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339859378 0 0) 0x7fc5f015ebd0 con 0x7fc5f00d1920 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5dc003530 con 0x7fc5f00d1920 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 701254756 0 0) 0x7fc5dc003530 con 0x7fc5f00d1920 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 >> v2:172.21.15.162:3300/0 conn(0x7fc5f00ce600 msgr2=0x7fc5f00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 --2- 172.21.15.133:0/3829743242 >> v2:172.21.15.162:3300/0 conn(0x7fc5f00ce600 0x7fc5f00cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 >> v1:172.21.15.162:6789/0 conn(0x7fc5f015d9a0 legacy=0x7fc5f00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00d0610 con 0x7fc5f00d1920 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fc5e8001a00 con 0x7fc5f00d1920 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 >> v1:172.21.15.133:6789/0 conn(0x7fc5f00d1920 legacy=0x7fc5f00d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 --2- 172.21.15.133:0/3829743242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5dc0039c0 0x7fc5dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 --2- 172.21.15.133:0/3829743242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5dc004810 0x7fc5dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 --2- 172.21.15.133:0/3829743242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5dc0051d0 0x7fc5dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00d0330 con 0x7fc5dc0039c0 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc00c170 con 0x7fc5dc004810 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc00c370 con 0x7fc5dc0051d0 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f5d69640 1 --2- 172.21.15.133:0/3829743242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5dc0039c0 0x7fc5dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f5568640 1 --2- 172.21.15.133:0/3829743242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5dc004810 0x7fc5dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f656a640 1 --2- 172.21.15.133:0/3829743242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5dc0051d0 0x7fc5dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f5d69640 1 -- 172.21.15.133:0/3829743242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5dc0051d0 msgr2=0x7fc5dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f5d69640 1 --2- 172.21.15.133:0/3829743242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5dc0051d0 0x7fc5dc00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f5d69640 1 -- 172.21.15.133:0/3829743242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5dc004810 msgr2=0x7fc5dc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f5d69640 1 --2- 172.21.15.133:0/3829743242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5dc004810 0x7fc5dc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f5d69640 1 -- 172.21.15.133:0/3829743242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc003530 con 0x7fc5dc0039c0 2024-04-25T10:07:15.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.849+0000 7fc5f656a640 1 -- 172.21.15.133:0/3829743242 reap_dead start 2024-04-25T10:07:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f5d69640 1 --2- 172.21.15.133:0/3829743242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5dc0039c0 0x7fc5dc003de0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001780 tx=0x7fc5ec01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c34da47130fc98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0013a0 con 0x7fc5dc0039c0 2024-04-25T10:07:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec0a9070 con 0x7fc5dc0039c0 2024-04-25T10:07:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f4d67640 1 -- 172.21.15.133:0/3829743242 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec003600 con 0x7fc5dc0039c0 2024-04-25T10:07:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/3829743242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5dc0039c0 msgr2=0x7fc5dc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 --2- 172.21.15.133:0/3829743242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5dc0039c0 0x7fc5dc003de0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001780 tx=0x7fc5ec01c420 comp rx=0 tx=0).stop 2024-04-25T10:07:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/3829743242 shutdown_connections 2024-04-25T10:07:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 --2- 172.21.15.133:0/3829743242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5dc0039c0 0x7fc5dc003de0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/3829743242 >> 172.21.15.133:0/3829743242 conn(0x7fc5f006bf80 msgr2=0x7fc5f006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/3829743242 shutdown_connections 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/3829743242 wait complete. 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 Processor -- start 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 -- start start 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5dc0039c0 0x7fc5f0161c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5f01f5d70 0x7fc5f01f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f00cbb80 0x7fc5f00cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00d0f40 con 0x7fc5f00cbb80 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00d2540 con 0x7fc5f01f5d70 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f7fcd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00d10c0 con 0x7fc5dc0039c0 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f656a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f00cbb80 0x7fc5f00cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f656a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f00cbb80 0x7fc5f00cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51338/0 (socket says 172.21.15.133:51338) 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.853+0000 7fc5f656a640 1 -- 172.21.15.133:0/490479351 learned_addr learned my addr 172.21.15.133:0/490479351 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f5568640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5f01f5d70 0x7fc5f01f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f5d69640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5dc0039c0 0x7fc5f0161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f656a640 1 -- 172.21.15.133:0/490479351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5dc0039c0 msgr2=0x7fc5f0161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f656a640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5dc0039c0 0x7fc5f0161c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f656a640 1 -- 172.21.15.133:0/490479351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5f01f5d70 msgr2=0x7fc5f01f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f656a640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5f01f5d70 0x7fc5f01f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f656a640 1 -- 172.21.15.133:0/490479351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00cc8d0 con 0x7fc5f00cbb80 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f656a640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f00cbb80 0x7fc5f00cc000 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fc5f008bdd0 tx=0x7fc5e800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5e67fc640 1 -- 172.21.15.133:0/490479351 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e800bb80 con 0x7fc5f00cbb80 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5e67fc640 1 -- 172.21.15.133:0/490479351 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e800bd20 con 0x7fc5f00cbb80 2024-04-25T10:07:15.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/490479351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f00c7400 con 0x7fc5f00cbb80 2024-04-25T10:07:15.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5e67fc640 1 -- 172.21.15.133:0/490479351 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80072b0 con 0x7fc5f00cbb80 2024-04-25T10:07:15.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/490479351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f00c79c0 con 0x7fc5f00cbb80 2024-04-25T10:07:15.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/490479351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f0068e80 con 0x7fc5f00cbb80 2024-04-25T10:07:15.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5e67fc640 1 -- 172.21.15.133:0/490479351 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5e801b060 con 0x7fc5f00cbb80 2024-04-25T10:07:15.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5e67fc640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5d4066f60 0x7fc5d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5e67fc640 1 -- 172.21.15.133:0/490479351 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fc5e8085240 con 0x7fc5f00cbb80 2024-04-25T10:07:15.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f5d69640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5d4066f60 0x7fc5d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.857+0000 7fc5f5d69640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5d4066f60 0x7fc5d4069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001750 tx=0x7fc5ec0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:15.861+0000 7fc5e67fc640 1 -- 172.21.15.133:0/490479351 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5e8056010 con 0x7fc5f00cbb80 2024-04-25T10:07:16.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.029+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/490479351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc5f015d0d0 con 0x7fc5f00cbb80 2024-04-25T10:07:16.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.029+0000 7fc5e67fc640 1 -- 172.21.15.133:0/490479351 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v70) v1 ==== 80+0+244 (secure 0 0 0) 0x7fc5e8055910 con 0x7fc5f00cbb80 2024-04-25T10:07:16.035 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.162:6835/272966073 2024-04-26T10:06:56.893338+0000 2024-04-25T10:07:16.035 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.162:6833/272966073 2024-04-26T10:06:56.893338+0000 2024-04-25T10:07:16.035 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6837/236214118 2024-04-26T10:06:55.901820+0000 2024-04-25T10:07:16.036 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6836/236214118 2024-04-26T10:06:55.901820+0000 2024-04-25T10:07:16.036 INFO:teuthology.orchestra.run.smithi133.stderr:listed 4 entries 2024-04-25T10:07:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/490479351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5d4066f60 msgr2=0x7fc5d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5d4066f60 0x7fc5d4069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001750 tx=0x7fc5ec0011a0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/490479351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f00cbb80 msgr2=0x7fc5f00cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f00cbb80 0x7fc5f00cc000 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fc5f008bdd0 tx=0x7fc5e800b4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/490479351 shutdown_connections 2024-04-25T10:07:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc5d4066f60 0x7fc5d4069420 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f00cbb80 0x7fc5f00cc000 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5f01f5d70 0x7fc5f01f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 --2- 172.21.15.133:0/490479351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5dc0039c0 0x7fc5f0161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/490479351 >> 172.21.15.133:0/490479351 conn(0x7fc5f006bf80 msgr2=0x7fc5f015dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/490479351 shutdown_connections 2024-04-25T10:07:16.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.033+0000 7fc5f7fcd640 1 -- 172.21.15.133:0/490479351 wait complete. 2024-04-25T10:07:16.053 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:07:16.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:16.054 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:16.064 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[56951]: fuse finished with error 0 and tester_r 0 2024-04-25T10:07:16.065 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:07:16.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:16.065 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:07:16.211 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:07:16.211 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:16.217 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:16.218 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:07:16.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:16.218 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:07:16.265 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:07:16.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:16.267 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:07:16.267 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:16.275 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:16.275 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:07:16.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:16.275 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:07:16.321 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:07:16.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:16.322 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:07:16.322 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:16.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:16.328 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:07:16.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:16.328 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:07:16.374 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:07:16.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:16.375 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:16.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9dd899640 1 Processor -- start 2024-04-25T10:07:16.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9dd899640 1 -- start start 2024-04-25T10:07:16.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9dd899640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb9d8158090 0x7fb9d80b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9dd899640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb9d815ec50 0x7fb9d806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9dd899640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d815e1b0 con 0x7fb9d80b3b40 2024-04-25T10:07:16.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9dd899640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d815df50 con 0x7fb9d8158090 2024-04-25T10:07:16.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9dd899640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d80b4cb0 con 0x7fb9d815ec50 2024-04-25T10:07:16.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9d6ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb9d8158090 0x7fb9d80b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9d6ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb9d8158090 0x7fb9d80b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36454/0 (socket says 172.21.15.133:36454) 2024-04-25T10:07:16.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9d6ffd640 1 -- 172.21.15.133:0/3283019019 learned_addr learned my addr 172.21.15.133:0/3283019019 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:16.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9d67fc640 1 --2- 172.21.15.133:0/3283019019 >> v2:172.21.15.162:3300/0 conn(0x7fb9d815ec50 0x7fb9d806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9d6ffd640 1 -- 172.21.15.133:0/3283019019 >> v2:172.21.15.162:3300/0 conn(0x7fb9d815ec50 msgr2=0x7fb9d806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9d6ffd640 1 --2- 172.21.15.133:0/3283019019 >> v2:172.21.15.162:3300/0 conn(0x7fb9d815ec50 0x7fb9d806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9d6ffd640 1 -- 172.21.15.133:0/3283019019 >> v1:172.21.15.162:6789/0 conn(0x7fb9d80b3b40 legacy=0x7fb9d806fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.501+0000 7fb9d6ffd640 1 -- 172.21.15.133:0/3283019019 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80b49d0 con 0x7fb9d8158090 2024-04-25T10:07:16.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9d6ffd640 1 --2- 172.21.15.133:0/3283019019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d8158090 0x7fb9d80b4180 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb9d806bd70 tx=0x7fb9c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2bacf69e24e08c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9d5ffb640 1 -- 172.21.15.133:0/3283019019 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80b0070 con 0x7fb9d8158090 2024-04-25T10:07:16.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9d5ffb640 1 -- 172.21.15.133:0/3283019019 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c80ab600 con 0x7fb9d8158090 2024-04-25T10:07:16.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9d5ffb640 1 -- 172.21.15.133:0/3283019019 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80ab9d0 con 0x7fb9d8158090 2024-04-25T10:07:16.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 -- 172.21.15.133:0/3283019019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d8158090 msgr2=0x7fb9d80b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 --2- 172.21.15.133:0/3283019019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d8158090 0x7fb9d80b4180 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb9d806bd70 tx=0x7fb9c801c520 comp rx=0 tx=0).stop 2024-04-25T10:07:16.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 -- 172.21.15.133:0/3283019019 shutdown_connections 2024-04-25T10:07:16.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 --2- 172.21.15.133:0/3283019019 >> v2:172.21.15.162:3300/0 conn(0x7fb9d815ec50 0x7fb9d806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 --2- 172.21.15.133:0/3283019019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d8158090 0x7fb9d80b4180 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 -- 172.21.15.133:0/3283019019 >> 172.21.15.133:0/3283019019 conn(0x7fb9d80b2370 msgr2=0x7fb9d80b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 -- 172.21.15.133:0/3283019019 shutdown_connections 2024-04-25T10:07:16.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 -- 172.21.15.133:0/3283019019 wait complete. 2024-04-25T10:07:16.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 Processor -- start 2024-04-25T10:07:16.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 -- start start 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9d80b3b40 0x7fb9d8168070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9d8158090 0x7fb9d81550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d815ec50 0x7fb9d80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d816a700 con 0x7fb9d815ec50 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d806b5e0 con 0x7fb9d8158090 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9dd899640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d816a880 con 0x7fb9d80b3b40 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d815ec50 0x7fb9d80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d815ec50 0x7fb9d80ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36458/0 (socket says 172.21.15.133:36458) 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.505+0000 7fb9d77fe640 1 -- 172.21.15.133:0/3439185673 learned_addr learned my addr 172.21.15.133:0/3439185673 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9d6ffd640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9d80b3b40 0x7fb9d8168070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9d67fc640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9d8158090 0x7fb9d81550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9d77fe640 1 -- 172.21.15.133:0/3439185673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9d80b3b40 msgr2=0x7fb9d8168070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9d77fe640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9d80b3b40 0x7fb9d8168070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9d77fe640 1 -- 172.21.15.133:0/3439185673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9d8158090 msgr2=0x7fb9d81550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9d77fe640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9d8158090 0x7fb9d81550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9d77fe640 1 -- 172.21.15.133:0/3439185673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80cdf50 con 0x7fb9d815ec50 2024-04-25T10:07:16.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9d77fe640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d815ec50 0x7fb9d80ce9d0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000c730 tx=0x7fb9d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9bffff640 1 -- 172.21.15.133:0/3439185673 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0003ba0 con 0x7fb9d815ec50 2024-04-25T10:07:16.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9bffff640 1 -- 172.21.15.133:0/3439185673 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0003d40 con 0x7fb9d815ec50 2024-04-25T10:07:16.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9dd899640 1 -- 172.21.15.133:0/3439185673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d80cffe0 con 0x7fb9d815ec50 2024-04-25T10:07:16.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9bffff640 1 -- 172.21.15.133:0/3439185673 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0005310 con 0x7fb9d815ec50 2024-04-25T10:07:16.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9dd899640 1 -- 172.21.15.133:0/3439185673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d80ce1d0 con 0x7fb9d815ec50 2024-04-25T10:07:16.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9dd899640 1 -- 172.21.15.133:0/3439185673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d8067c70 con 0x7fb9d815ec50 2024-04-25T10:07:16.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9bffff640 1 -- 172.21.15.133:0/3439185673 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9d00056c0 con 0x7fb9d815ec50 2024-04-25T10:07:16.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9bffff640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9c4066f60 0x7fb9c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9d6ffd640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9c4066f60 0x7fb9c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9bffff640 1 -- 172.21.15.133:0/3439185673 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fb9d00852a0 con 0x7fb9d815ec50 2024-04-25T10:07:16.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.509+0000 7fb9d6ffd640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9c4066f60 0x7fb9c4069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80ad000 tx=0x7fb9c80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.513+0000 7fb9bffff640 1 -- 172.21.15.133:0/3439185673 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9d80cdf50 con 0x7fb9d815ec50 2024-04-25T10:07:16.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.705+0000 7fb9dd899640 1 -- 172.21.15.133:0/3439185673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9d806d3e0 con 0x7fb9d815ec50 2024-04-25T10:07:16.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.705+0000 7fb9bffff640 1 -- 172.21.15.133:0/3439185673 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb9d0056100 con 0x7fb9d815ec50 2024-04-25T10:07:16.710 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:16.710 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":41,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1579659657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1579659657},{"type":"v1","addr":"172.21.15.162:6835","nonce":1579659657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/232744252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":232744252},{"type":"v1","addr":"172.21.15.162:6833","nonce":232744252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7149,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/912719603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":912719603},{"type":"v1","addr":"172.21.15.133:6835","nonce":912719603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:36.516727+0000","modified":"2024-04-25T10:06:58.925769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/2310040121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2310040121},{"type":"v1","addr":"172.21.15.133:6837","nonce":2310040121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:16.710 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 41 2024-04-25T10:07:16.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.705+0000 7fb9dd899640 1 -- 172.21.15.133:0/3439185673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9c4066f60 msgr2=0x7fb9c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.705+0000 7fb9dd899640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9c4066f60 0x7fb9c4069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80ad000 tx=0x7fb9c80a6000 comp rx=0 tx=0).stop 2024-04-25T10:07:16.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.705+0000 7fb9dd899640 1 -- 172.21.15.133:0/3439185673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d815ec50 msgr2=0x7fb9d80ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.705+0000 7fb9dd899640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d815ec50 0x7fb9d80ce9d0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000c730 tx=0x7fb9d000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.705+0000 7fb9dd899640 1 -- 172.21.15.133:0/3439185673 shutdown_connections 2024-04-25T10:07:16.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.705+0000 7fb9dd899640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb9c4066f60 0x7fb9c4069420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.705+0000 7fb9dd899640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d815ec50 0x7fb9d80ce9d0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.705+0000 7fb9dd899640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9d8158090 0x7fb9d81550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.705+0000 7fb9dd899640 1 --2- 172.21.15.133:0/3439185673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb9d80b3b40 0x7fb9d8168070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.709+0000 7fb9dd899640 1 -- 172.21.15.133:0/3439185673 >> 172.21.15.133:0/3439185673 conn(0x7fb9d80b2370 msgr2=0x7fb9d815a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.709+0000 7fb9dd899640 1 -- 172.21.15.133:0/3439185673 shutdown_connections 2024-04-25T10:07:16.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.709+0000 7fb9dd899640 1 -- 172.21.15.133:0/3439185673 wait complete. 2024-04-25T10:07:16.729 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:16.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff330f3b640 1 Processor -- start 2024-04-25T10:07:16.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff330f3b640 1 -- start start 2024-04-25T10:07:16.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff330f3b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff32c0b1f70 0x7ff32c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff330f3b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff32c0ce330 0x7ff32c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff330f3b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c0c61d0 con 0x7ff32c15c9a0 2024-04-25T10:07:16.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff330f3b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32c0cecc0 con 0x7ff32c0b1f70 2024-04-25T10:07:16.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff330f3b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0c64b0 con 0x7ff32c0ce330 2024-04-25T10:07:16.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff329d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff32c0b1f70 0x7ff32c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff329d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff32c0b1f70 0x7ff32c06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36474/0 (socket says 172.21.15.133:36474) 2024-04-25T10:07:16.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff329d9b640 1 -- 172.21.15.133:0/1681731056 learned_addr learned my addr 172.21.15.133:0/1681731056 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:16.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff32ad9d640 1 --2- 172.21.15.133:0/1681731056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff32c0ce330 0x7ff32c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff329d9b640 1 -- 172.21.15.133:0/1681731056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff32c0ce330 msgr2=0x7ff32c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff329d9b640 1 --2- 172.21.15.133:0/1681731056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff32c0ce330 0x7ff32c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff329d9b640 1 -- 172.21.15.133:0/1681731056 >> v1:172.21.15.162:6789/0 conn(0x7ff32c15c9a0 legacy=0x7ff32c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.841+0000 7ff329d9b640 1 -- 172.21.15.133:0/1681731056 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c06d4f0 con 0x7ff32c0b1f70 2024-04-25T10:07:16.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff329d9b640 1 --2- 172.21.15.133:0/1681731056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c0b1f70 0x7ff32c06dcf0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff320009e50 tx=0x7ff32001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6217a379f37e1a7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff32959a640 1 -- 172.21.15.133:0/1681731056 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3200a58f0 con 0x7ff32c0b1f70 2024-04-25T10:07:16.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff32959a640 1 -- 172.21.15.133:0/1681731056 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3200a5a90 con 0x7ff32c0b1f70 2024-04-25T10:07:16.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff32959a640 1 -- 172.21.15.133:0/1681731056 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3200a5e60 con 0x7ff32c0b1f70 2024-04-25T10:07:16.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 -- 172.21.15.133:0/1681731056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c0b1f70 msgr2=0x7ff32c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 --2- 172.21.15.133:0/1681731056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c0b1f70 0x7ff32c06dcf0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff320009e50 tx=0x7ff32001ca00 comp rx=0 tx=0).stop 2024-04-25T10:07:16.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 -- 172.21.15.133:0/1681731056 shutdown_connections 2024-04-25T10:07:16.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 --2- 172.21.15.133:0/1681731056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff32c0ce330 0x7ff32c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 --2- 172.21.15.133:0/1681731056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c0b1f70 0x7ff32c06dcf0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 -- 172.21.15.133:0/1681731056 >> 172.21.15.133:0/1681731056 conn(0x7ff32c067000 msgr2=0x7ff32c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 -- 172.21.15.133:0/1681731056 shutdown_connections 2024-04-25T10:07:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 -- 172.21.15.133:0/1681731056 wait complete. 2024-04-25T10:07:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 Processor -- start 2024-04-25T10:07:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 -- start start 2024-04-25T10:07:16.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff32c0b1f70 0x7ff32c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c0ce330 0x7ff32c138d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff32c15c9a0 0x7ff32c14a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c15dad0 con 0x7ff32c0ce330 2024-04-25T10:07:16.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c15d950 con 0x7ff32c15c9a0 2024-04-25T10:07:16.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff330f3b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c15dc50 con 0x7ff32c0b1f70 2024-04-25T10:07:16.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff329d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c0ce330 0x7ff32c138d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff329d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c0ce330 0x7ff32c138d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36488/0 (socket says 172.21.15.133:36488) 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff329d9b640 1 -- 172.21.15.133:0/1318180382 learned_addr learned my addr 172.21.15.133:0/1318180382 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff32a59c640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff32c0b1f70 0x7ff32c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff32ad9d640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff32c15c9a0 0x7ff32c14a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff329d9b640 1 -- 172.21.15.133:0/1318180382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff32c0b1f70 msgr2=0x7ff32c1562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff329d9b640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff32c0b1f70 0x7ff32c1562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff329d9b640 1 -- 172.21.15.133:0/1318180382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff32c15c9a0 msgr2=0x7ff32c14a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff329d9b640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff32c15c9a0 0x7ff32c14a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff329d9b640 1 -- 172.21.15.133:0/1318180382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c155290 con 0x7ff32c0ce330 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff329d9b640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c0ce330 0x7ff32c138d50 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff32001b0b0 tx=0x7ff3200197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.845+0000 7ff31affd640 1 -- 172.21.15.133:0/1318180382 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3200a58f0 con 0x7ff32c0ce330 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.849+0000 7ff31affd640 1 -- 172.21.15.133:0/1318180382 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3200be070 con 0x7ff32c0ce330 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.849+0000 7ff330f3b640 1 -- 172.21.15.133:0/1318180382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c155550 con 0x7ff32c0ce330 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.849+0000 7ff31affd640 1 -- 172.21.15.133:0/1318180382 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff320009510 con 0x7ff32c0ce330 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.849+0000 7ff330f3b640 1 -- 172.21.15.133:0/1318180382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c155b10 con 0x7ff32c0ce330 2024-04-25T10:07:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.849+0000 7ff330f3b640 1 -- 172.21.15.133:0/1318180382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c068ef0 con 0x7ff32c0ce330 2024-04-25T10:07:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.849+0000 7ff31affd640 1 -- 172.21.15.133:0/1318180382 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3200b86b0 con 0x7ff32c0ce330 2024-04-25T10:07:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.849+0000 7ff31affd640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff30c066f60 0x7ff30c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.849+0000 7ff31affd640 1 -- 172.21.15.133:0/1318180382 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff320120880 con 0x7ff32c0ce330 2024-04-25T10:07:16.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.849+0000 7ff32a59c640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff30c066f60 0x7ff30c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.849+0000 7ff32a59c640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff30c066f60 0x7ff30c069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff31c001810 tx=0x7ff31c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:16.853+0000 7ff31affd640 1 -- 172.21.15.133:0/1318180382 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff320016620 con 0x7ff32c0ce330 2024-04-25T10:07:17.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.017+0000 7ff330f3b640 1 -- 172.21.15.133:0/1318180382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff32c0c5a90 con 0x7ff32c0ce330 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.017+0000 7ff31affd640 1 -- 172.21.15.133:0/1318180382 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17167 (secure 0 0 0) 0x7ff3200f1790 con 0x7ff32c0ce330 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":70,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:06:56.893382+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:34.764755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:35.504408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6835/272966073":"2024-04-26T10:06:56.893338+0000","172.21.15.162:6833/272966073":"2024-04-26T10:06:56.893338+0000","172.21.15.133:6837/236214118":"2024-04-26T10:06:55.901820+0000","172.21.15.133:6836/236214118":"2024-04-26T10:06:55.901820+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 -- 172.21.15.133:0/1318180382 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff30c066f60 msgr2=0x7ff30c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff30c066f60 0x7ff30c069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff31c001810 tx=0x7ff31c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 -- 172.21.15.133:0/1318180382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c0ce330 msgr2=0x7ff32c138d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c0ce330 0x7ff32c138d50 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff32001b0b0 tx=0x7ff3200197a0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 -- 172.21.15.133:0/1318180382 shutdown_connections 2024-04-25T10:07:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff32c15c9a0 0x7ff32c14a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff30c066f60 0x7ff30c069420 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c0ce330 0x7ff32c138d50 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 --2- 172.21.15.133:0/1318180382 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff32c0b1f70 0x7ff32c1562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 -- 172.21.15.133:0/1318180382 >> 172.21.15.133:0/1318180382 conn(0x7ff32c067000 msgr2=0x7ff32c15b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 -- 172.21.15.133:0/1318180382 shutdown_connections 2024-04-25T10:07:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.021+0000 7ff330f3b640 1 -- 172.21.15.133:0/1318180382 wait complete. 2024-04-25T10:07:17.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:34.764755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:35.504408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:17.043 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:17.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8da71a6640 1 Processor -- start 2024-04-25T10:07:17.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8da71a6640 1 -- start start 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8da71a6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8da00b2150 0x7f8da006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8da71a6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8da015b9b0 0x7f8da006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8da71a6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da015ac50 con 0x7f8da0143ea0 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8da71a6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da00ccf20 con 0x7f8da00b2150 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8da71a6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da00cd180 con 0x7f8da015b9b0 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8da4f42640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8da0143ea0 0x7f8da00cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45516/0 (socket says 172.21.15.133:45516) 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8da4f42640 1 -- 172.21.15.133:0/2084626843 learned_addr learned my addr 172.21.15.133:0/2084626843 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8d9ffff640 1 --2- 172.21.15.133:0/2084626843 >> v2:172.21.15.133:3300/0 conn(0x7f8da00b2150 0x7f8da006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8da5743640 1 --2- 172.21.15.133:0/2084626843 >> v2:172.21.15.162:3300/0 conn(0x7f8da015b9b0 0x7f8da006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8d9ffff640 1 -- 172.21.15.133:0/2084626843 >> v2:172.21.15.162:3300/0 conn(0x7f8da015b9b0 msgr2=0x7f8da006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8d9ffff640 1 --2- 172.21.15.133:0/2084626843 >> v2:172.21.15.162:3300/0 conn(0x7f8da015b9b0 0x7f8da006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8d9ffff640 1 -- 172.21.15.133:0/2084626843 >> v1:172.21.15.133:6789/0 conn(0x7f8da0143ea0 legacy=0x7f8da00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8d9f7fe640 1 -- 172.21.15.133:0/2084626843 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561238943 0 0) 0x7f8d90001120 con 0x7f8da0143ea0 2024-04-25T10:07:17.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.165+0000 7f8d9ffff640 1 -- 172.21.15.133:0/2084626843 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da015aed0 con 0x7f8da00b2150 2024-04-25T10:07:17.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8d9ffff640 1 --2- 172.21.15.133:0/2084626843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8da00b2150 0x7f8da006e3b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f8da00cd7f0 tx=0x7f8d9401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a69c5fac32888 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8d9f7fe640 1 -- 172.21.15.133:0/2084626843 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d940b0070 con 0x7f8da00b2150 2024-04-25T10:07:17.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8d9f7fe640 1 -- 172.21.15.133:0/2084626843 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d940ab600 con 0x7f8da00b2150 2024-04-25T10:07:17.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8d9f7fe640 1 -- 172.21.15.133:0/2084626843 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d940ab9d0 con 0x7f8da00b2150 2024-04-25T10:07:17.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8da71a6640 1 -- 172.21.15.133:0/2084626843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8da00b2150 msgr2=0x7f8da006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8da71a6640 1 --2- 172.21.15.133:0/2084626843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8da00b2150 0x7f8da006e3b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f8da00cd7f0 tx=0x7f8d9401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8da71a6640 1 -- 172.21.15.133:0/2084626843 shutdown_connections 2024-04-25T10:07:17.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8da71a6640 1 --2- 172.21.15.133:0/2084626843 >> v2:172.21.15.162:3300/0 conn(0x7f8da015b9b0 0x7f8da006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8da71a6640 1 --2- 172.21.15.133:0/2084626843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8da00b2150 0x7f8da006e3b0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8da71a6640 1 -- 172.21.15.133:0/2084626843 >> 172.21.15.133:0/2084626843 conn(0x7f8da0066f70 msgr2=0x7f8da006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8da71a6640 1 -- 172.21.15.133:0/2084626843 shutdown_connections 2024-04-25T10:07:17.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.169+0000 7f8da71a6640 1 -- 172.21.15.133:0/2084626843 wait complete. 2024-04-25T10:07:17.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da71a6640 1 Processor -- start 2024-04-25T10:07:17.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da71a6640 1 -- start start 2024-04-25T10:07:17.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da71a6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da00b2150 0x7f8da0138230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da71a6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8da0143ea0 0x7f8da0138a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da71a6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8da015b9b0 0x7f8da0138f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da71a6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00b1270 con 0x7f8da015b9b0 2024-04-25T10:07:17.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da71a6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00b10f0 con 0x7f8da00b2150 2024-04-25T10:07:17.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da71a6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00b13f0 con 0x7f8da0143ea0 2024-04-25T10:07:17.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da5743640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8da015b9b0 0x7f8da0138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da5743640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8da015b9b0 0x7f8da0138f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36508/0 (socket says 172.21.15.133:36508) 2024-04-25T10:07:17.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da5743640 1 -- 172.21.15.133:0/1729533349 learned_addr learned my addr 172.21.15.133:0/1729533349 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:17.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da4f42640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da00b2150 0x7f8da0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8d9ffff640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8da0143ea0 0x7f8da0138a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da5743640 1 -- 172.21.15.133:0/1729533349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8da0143ea0 msgr2=0x7f8da0138a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da5743640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8da0143ea0 0x7f8da0138a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da5743640 1 -- 172.21.15.133:0/1729533349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da00b2150 msgr2=0x7f8da0138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da5743640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da00b2150 0x7f8da0138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da5743640 1 -- 172.21.15.133:0/1729533349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da013a630 con 0x7f8da015b9b0 2024-04-25T10:07:17.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da5743640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8da015b9b0 0x7f8da0138f60 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009fd0 tx=0x7f8d9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8d9d7fa640 1 -- 172.21.15.133:0/1729533349 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98014420 con 0x7f8da015b9b0 2024-04-25T10:07:17.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8d9d7fa640 1 -- 172.21.15.133:0/1729533349 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d98019070 con 0x7f8da015b9b0 2024-04-25T10:07:17.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da71a6640 1 -- 172.21.15.133:0/1729533349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da0139590 con 0x7f8da015b9b0 2024-04-25T10:07:17.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8d9d7fa640 1 -- 172.21.15.133:0/1729533349 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98014800 con 0x7f8da015b9b0 2024-04-25T10:07:17.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da71a6640 1 -- 172.21.15.133:0/1729533349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da0139b50 con 0x7f8da015b9b0 2024-04-25T10:07:17.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.173+0000 7f8da71a6640 1 -- 172.21.15.133:0/1729533349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da0068e80 con 0x7f8da015b9b0 2024-04-25T10:07:17.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.177+0000 7f8d9d7fa640 1 -- 172.21.15.133:0/1729533349 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d980149a0 con 0x7f8da015b9b0 2024-04-25T10:07:17.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.177+0000 7f8d9d7fa640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d8c066f60 0x7f8d8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.177+0000 7f8d9d7fa640 1 -- 172.21.15.133:0/1729533349 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f8d98085ac0 con 0x7f8da015b9b0 2024-04-25T10:07:17.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.177+0000 7f8da4f42640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d8c066f60 0x7f8d8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.177+0000 7f8da4f42640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d8c066f60 0x7f8d8c069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8d90007fd0 tx=0x7f8d90007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.181+0000 7f8d9d7fa640 1 -- 172.21.15.133:0/1729533349 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d98056890 con 0x7f8da015b9b0 2024-04-25T10:07:17.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.341+0000 7f8da71a6640 1 -- 172.21.15.133:0/1729533349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8da00b4880 con 0x7f8da015b9b0 2024-04-25T10:07:17.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8d9d7fa640 1 -- 172.21.15.133:0/1729533349 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d98056190 con 0x7f8da015b9b0 2024-04-25T10:07:17.349 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:07:17.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 -- 172.21.15.133:0/1729533349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d8c066f60 msgr2=0x7f8d8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d8c066f60 0x7f8d8c069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8d90007fd0 tx=0x7f8d90007980 comp rx=0 tx=0).stop 2024-04-25T10:07:17.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 -- 172.21.15.133:0/1729533349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8da015b9b0 msgr2=0x7f8da0138f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8da015b9b0 0x7f8da0138f60 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009fd0 tx=0x7f8d9800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 -- 172.21.15.133:0/1729533349 shutdown_connections 2024-04-25T10:07:17.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d8c066f60 0x7f8d8c069420 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8da015b9b0 0x7f8da0138f60 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8da0143ea0 0x7f8da0138a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 --2- 172.21.15.133:0/1729533349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8da00b2150 0x7f8da0138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 -- 172.21.15.133:0/1729533349 >> 172.21.15.133:0/1729533349 conn(0x7f8da0066f70 msgr2=0x7f8da00b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 -- 172.21.15.133:0/1729533349 shutdown_connections 2024-04-25T10:07:17.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.345+0000 7f8da71a6640 1 -- 172.21.15.133:0/1729533349 wait complete. 2024-04-25T10:07:17.368 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:17.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86c71c640 1 Processor -- start 2024-04-25T10:07:17.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86c71c640 1 -- start start 2024-04-25T10:07:17.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86c71c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff86406c7b0 0x7ff86406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86c71c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8640b1220 con 0x7ff86415d320 2024-04-25T10:07:17.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86c71c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8640b1500 con 0x7ff8640ce600 2024-04-25T10:07:17.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86c71c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8640b2ff0 con 0x7ff86406c7b0 2024-04-25T10:07:17.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86acb9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff86415d320 0x7ff86415d7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45522/0 (socket says 172.21.15.133:45522) 2024-04-25T10:07:17.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86acb9640 1 -- 172.21.15.133:0/2179649255 learned_addr learned my addr 172.21.15.133:0/2179649255 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff869cb7640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff86406c7b0 0x7ff86406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814096048 0 0) 0x7ff8640b1220 con 0x7ff86415d320 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff850003530 con 0x7ff86415d320 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 406671325 0 0) 0x7ff850003530 con 0x7ff86415d320 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff86406c7b0 msgr2=0x7ff86406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff86406c7b0 0x7ff86406cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 >> v1:172.21.15.162:6789/0 conn(0x7ff8640ce600 legacy=0x7ff8640d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8640b17e0 con 0x7ff86415d320 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff85c001a00 con 0x7ff86415d320 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 >> v1:172.21.15.133:6789/0 conn(0x7ff86415d320 legacy=0x7ff86415d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8500039f0 0x7ff850003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff850004810 0x7ff850004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff86406c7b0 0x7ff8500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8640b2ff0 con 0x7ff8500039f0 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86a4b8640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8500039f0 0x7ff850003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85000ba80 con 0x7ff850004810 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85000bc80 con 0x7ff86406c7b0 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff869cb7640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff850004810 0x7ff850004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86acb9640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff86406c7b0 0x7ff8500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86a4b8640 1 -- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff86406c7b0 msgr2=0x7ff8500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86a4b8640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff86406c7b0 0x7ff8500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86a4b8640 1 -- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff850004810 msgr2=0x7ff850004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86a4b8640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff850004810 0x7ff850004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86a4b8640 1 -- 172.21.15.133:0/2179649255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff850003530 con 0x7ff8500039f0 2024-04-25T10:07:17.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.485+0000 7ff86acb9640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff86406c7b0 0x7ff8500051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:17.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff86a4b8640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8500039f0 0x7ff850003e10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff860001750 tx=0x7ff86001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97401c1e86ee2301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8600013a0 con 0x7ff8500039f0 2024-04-25T10:07:17.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8600052f0 con 0x7ff8500039f0 2024-04-25T10:07:17.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff8694b6640 1 -- 172.21.15.133:0/2179649255 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff860003840 con 0x7ff8500039f0 2024-04-25T10:07:17.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff86c71c640 1 -- 172.21.15.133:0/2179649255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8500039f0 msgr2=0x7ff850003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff86c71c640 1 --2- 172.21.15.133:0/2179649255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8500039f0 0x7ff850003e10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff860001750 tx=0x7ff86001c3c0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff86acb9640 1 -- 172.21.15.133:0/2179649255 reap_dead start 2024-04-25T10:07:17.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff86c71c640 1 -- 172.21.15.133:0/2179649255 shutdown_connections 2024-04-25T10:07:17.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff86c71c640 1 -- 172.21.15.133:0/2179649255 >> 172.21.15.133:0/2179649255 conn(0x7ff86409a990 msgr2=0x7ff8640d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff86c71c640 1 -- 172.21.15.133:0/2179649255 shutdown_connections 2024-04-25T10:07:17.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff86c71c640 1 -- 172.21.15.133:0/2179649255 wait complete. 2024-04-25T10:07:17.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff86c71c640 1 Processor -- start 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.489+0000 7ff86c71c640 1 -- start start 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86c71c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8500039f0 0x7ff864163500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86c71c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8641597c0 0x7ff86415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86c71c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff864160080 0x7ff864160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86c71c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8640ae280 con 0x7ff8641597c0 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86c71c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8640ce9e0 con 0x7ff864160080 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86c71c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86415fbf0 con 0x7ff8500039f0 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff869cb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8641597c0 0x7ff86415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff869cb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8641597c0 0x7ff86415bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36522/0 (socket says 172.21.15.133:36522) 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff869cb7640 1 -- 172.21.15.133:0/2234211758 learned_addr learned my addr 172.21.15.133:0/2234211758 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86a4b8640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8500039f0 0x7ff864163500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86acb9640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff864160080 0x7ff864160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff869cb7640 1 -- 172.21.15.133:0/2234211758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8500039f0 msgr2=0x7ff864163500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff869cb7640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8500039f0 0x7ff864163500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff869cb7640 1 -- 172.21.15.133:0/2234211758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff864160080 msgr2=0x7ff864160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff869cb7640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff864160080 0x7ff864160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff869cb7640 1 -- 172.21.15.133:0/2234211758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff864163c50 con 0x7ff8641597c0 2024-04-25T10:07:17.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86acb9640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff864160080 0x7ff864160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:17.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff869cb7640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8641597c0 0x7ff86415bf50 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff8540119a0 tx=0x7ff854011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff85affd640 1 -- 172.21.15.133:0/2234211758 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85400faf0 con 0x7ff8641597c0 2024-04-25T10:07:17.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff85affd640 1 -- 172.21.15.133:0/2234211758 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85400fc90 con 0x7ff8641597c0 2024-04-25T10:07:17.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86c71c640 1 -- 172.21.15.133:0/2234211758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86415f2c0 con 0x7ff8641597c0 2024-04-25T10:07:17.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff85affd640 1 -- 172.21.15.133:0/2234211758 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854014460 con 0x7ff8641597c0 2024-04-25T10:07:17.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86c71c640 1 -- 172.21.15.133:0/2234211758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86415f880 con 0x7ff8641597c0 2024-04-25T10:07:17.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86c71c640 1 -- 172.21.15.133:0/2234211758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff864067c70 con 0x7ff8641597c0 2024-04-25T10:07:17.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff85affd640 1 -- 172.21.15.133:0/2234211758 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff854012040 con 0x7ff8641597c0 2024-04-25T10:07:17.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff85affd640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff848066f60 0x7ff848069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff85affd640 1 -- 172.21.15.133:0/2234211758 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff854085a30 con 0x7ff8641597c0 2024-04-25T10:07:17.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86a4b8640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff848066f60 0x7ff848069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.493+0000 7ff86a4b8640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff848066f60 0x7ff848069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff8500057f0 tx=0x7ff8600011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.497+0000 7ff85affd640 1 -- 172.21.15.133:0/2234211758 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff854019030 con 0x7ff8641597c0 2024-04-25T10:07:17.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.649+0000 7ff86c71c640 1 -- 172.21.15.133:0/2234211758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff86406c590 con 0x7ff8641597c0 2024-04-25T10:07:17.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.649+0000 7ff85affd640 1 -- 172.21.15.133:0/2234211758 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff854056760 con 0x7ff8641597c0 2024-04-25T10:07:17.656 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:07:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 -- 172.21.15.133:0/2234211758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff848066f60 msgr2=0x7ff848069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff848066f60 0x7ff848069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff8500057f0 tx=0x7ff8600011a0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 -- 172.21.15.133:0/2234211758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8641597c0 msgr2=0x7ff86415bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8641597c0 0x7ff86415bf50 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff8540119a0 tx=0x7ff854011e60 comp rx=0 tx=0).stop 2024-04-25T10:07:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 -- 172.21.15.133:0/2234211758 shutdown_connections 2024-04-25T10:07:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff864160080 0x7ff864160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff848066f60 0x7ff848069420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8641597c0 0x7ff86415bf50 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 --2- 172.21.15.133:0/2234211758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff8500039f0 0x7ff864163500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 -- 172.21.15.133:0/2234211758 >> 172.21.15.133:0/2234211758 conn(0x7ff86409a990 msgr2=0x7ff864168280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 -- 172.21.15.133:0/2234211758 shutdown_connections 2024-04-25T10:07:17.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.653+0000 7ff86c71c640 1 -- 172.21.15.133:0/2234211758 wait complete. 2024-04-25T10:07:17.674 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:17.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba9b7a4640 1 Processor -- start 2024-04-25T10:07:17.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba9b7a4640 1 -- start start 2024-04-25T10:07:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba9b7a4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fba940cc330 0x7fba940ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba9b7a4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba940cc790 0x7fba940ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba9b7a4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba940afe00 con 0x7fba9415bbb0 2024-04-25T10:07:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba9b7a4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba940b0730 con 0x7fba940cc790 2024-04-25T10:07:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba9b7a4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba940b00e0 con 0x7fba940cc330 2024-04-25T10:07:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba99d41640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba940cc790 0x7fba940ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba99d41640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba940cc790 0x7fba940ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36526/0 (socket says 172.21.15.133:36526) 2024-04-25T10:07:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba99d41640 1 -- 172.21.15.133:0/701358017 learned_addr learned my addr 172.21.15.133:0/701358017 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba98d3f640 1 --2- 172.21.15.133:0/701358017 >> v2:172.21.15.162:3300/0 conn(0x7fba940cc330 0x7fba940ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba99d41640 1 -- 172.21.15.133:0/701358017 >> v2:172.21.15.162:3300/0 conn(0x7fba940cc330 msgr2=0x7fba940ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba99d41640 1 --2- 172.21.15.133:0/701358017 >> v2:172.21.15.162:3300/0 conn(0x7fba940cc330 0x7fba940ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba99d41640 1 -- 172.21.15.133:0/701358017 >> v1:172.21.15.162:6789/0 conn(0x7fba9415bbb0 legacy=0x7fba940b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.793+0000 7fba99d41640 1 -- 172.21.15.133:0/701358017 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba940b03c0 con 0x7fba940cc790 2024-04-25T10:07:17.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba99d41640 1 --2- 172.21.15.133:0/701358017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba940cc790 0x7fba940ad700 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fba940ce030 tx=0x7fba8c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b718f80d34296568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba8bfff640 1 -- 172.21.15.133:0/701358017 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0a5590 con 0x7fba940cc790 2024-04-25T10:07:17.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba8bfff640 1 -- 172.21.15.133:0/701358017 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c0a5730 con 0x7fba940cc790 2024-04-25T10:07:17.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba8bfff640 1 -- 172.21.15.133:0/701358017 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0a5b00 con 0x7fba940cc790 2024-04-25T10:07:17.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 -- 172.21.15.133:0/701358017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba940cc790 msgr2=0x7fba940ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 --2- 172.21.15.133:0/701358017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba940cc790 0x7fba940ad700 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fba940ce030 tx=0x7fba8c01c8d0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 -- 172.21.15.133:0/701358017 shutdown_connections 2024-04-25T10:07:17.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 --2- 172.21.15.133:0/701358017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba940cc790 0x7fba940ad700 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 --2- 172.21.15.133:0/701358017 >> v2:172.21.15.162:3300/0 conn(0x7fba940cc330 0x7fba940ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 -- 172.21.15.133:0/701358017 >> 172.21.15.133:0/701358017 conn(0x7fba940d0570 msgr2=0x7fba940d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 -- 172.21.15.133:0/701358017 shutdown_connections 2024-04-25T10:07:17.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 -- 172.21.15.133:0/701358017 wait complete. 2024-04-25T10:07:17.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 Processor -- start 2024-04-25T10:07:17.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 -- start start 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba940cc330 0x7fba940c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba940cc790 0x7fba940b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba9415bbb0 0x7fba940c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba940cdb60 con 0x7fba940cc790 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba940cd9e0 con 0x7fba940cc330 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba9b7a4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba940cdce0 con 0x7fba9415bbb0 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba98d3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba940cc790 0x7fba940b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba98d3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba940cc790 0x7fba940b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36530/0 (socket says 172.21.15.133:36530) 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba98d3f640 1 -- 172.21.15.133:0/2405433669 learned_addr learned my addr 172.21.15.133:0/2405433669 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.797+0000 7fba99540640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba940cc330 0x7fba940c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba98d3f640 1 -- 172.21.15.133:0/2405433669 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba9415bbb0 msgr2=0x7fba940c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba98d3f640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba9415bbb0 0x7fba940c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba98d3f640 1 -- 172.21.15.133:0/2405433669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba940cc330 msgr2=0x7fba940c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba98d3f640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba940cc330 0x7fba940c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba98d3f640 1 -- 172.21.15.133:0/2405433669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba94066dd0 con 0x7fba940cc790 2024-04-25T10:07:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba99540640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba940cc330 0x7fba940c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba98d3f640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba940cc790 0x7fba940b33c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fba8400d5f0 tx=0x7fba8400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba89ffb640 1 -- 172.21.15.133:0/2405433669 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba840125f0 con 0x7fba940cc790 2024-04-25T10:07:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba89ffb640 1 -- 172.21.15.133:0/2405433669 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba84019070 con 0x7fba940cc790 2024-04-25T10:07:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba9b7a4640 1 -- 172.21.15.133:0/2405433669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba94155e80 con 0x7fba940cc790 2024-04-25T10:07:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba89ffb640 1 -- 172.21.15.133:0/2405433669 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba840129b0 con 0x7fba940cc790 2024-04-25T10:07:17.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba9b7a4640 1 -- 172.21.15.133:0/2405433669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba94156440 con 0x7fba940cc790 2024-04-25T10:07:17.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba9b7a4640 1 -- 172.21.15.133:0/2405433669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba94067c70 con 0x7fba940cc790 2024-04-25T10:07:17.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba89ffb640 1 -- 172.21.15.133:0/2405433669 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba84012b50 con 0x7fba940cc790 2024-04-25T10:07:17.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba89ffb640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba78066f60 0x7fba78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba99540640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba78066f60 0x7fba78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba89ffb640 1 -- 172.21.15.133:0/2405433669 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fba84085600 con 0x7fba940cc790 2024-04-25T10:07:17.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.801+0000 7fba99540640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba78066f60 0x7fba78069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fba90000f70 tx=0x7fba90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.805+0000 7fba89ffb640 1 -- 172.21.15.133:0/2405433669 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba84056460 con 0x7fba940cc790 2024-04-25T10:07:17.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.953+0000 7fba9b7a4640 1 -- 172.21.15.133:0/2405433669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba940c8110 con 0x7fba940cc790 2024-04-25T10:07:17.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba89ffb640 1 -- 172.21.15.133:0/2405433669 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba84055d60 con 0x7fba940cc790 2024-04-25T10:07:17.960 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 -- 172.21.15.133:0/2405433669 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba78066f60 msgr2=0x7fba78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba78066f60 0x7fba78069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fba90000f70 tx=0x7fba90009040 comp rx=0 tx=0).stop 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 -- 172.21.15.133:0/2405433669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba940cc790 msgr2=0x7fba940b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba940cc790 0x7fba940b33c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fba8400d5f0 tx=0x7fba8400dab0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 -- 172.21.15.133:0/2405433669 shutdown_connections 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba9415bbb0 0x7fba940c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba78066f60 0x7fba78069420 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba940cc790 0x7fba940b33c0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 --2- 172.21.15.133:0/2405433669 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba940cc330 0x7fba940c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 -- 172.21.15.133:0/2405433669 >> 172.21.15.133:0/2405433669 conn(0x7fba940d0570 msgr2=0x7fba940b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 -- 172.21.15.133:0/2405433669 shutdown_connections 2024-04-25T10:07:17.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:17.957+0000 7fba9b7a4640 1 -- 172.21.15.133:0/2405433669 wait complete. 2024-04-25T10:07:17.979 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:18.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.085+0000 7f4606c5b640 1 Processor -- start 2024-04-25T10:07:18.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.085+0000 7f4606c5b640 1 -- start start 2024-04-25T10:07:18.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.085+0000 7f4606c5b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f460006eb70 0x7f46000b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.085+0000 7f4606c5b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f46001584f0 0x7f460015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.085+0000 7f4606c5b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46000cbe60 con 0x7f4600158110 2024-04-25T10:07:18.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.085+0000 7f4606c5b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46000b19b0 con 0x7f46001584f0 2024-04-25T10:07:18.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.085+0000 7f4606c5b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460015b360 con 0x7f460006eb70 2024-04-25T10:07:18.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f46049f7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4600158110 0x7f460006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45526/0 (socket says 172.21.15.133:45526) 2024-04-25T10:07:18.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f46049f7640 1 -- 172.21.15.133:0/317112005 learned_addr learned my addr 172.21.15.133:0/317112005 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45fffff640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f460006eb70 0x7f46000b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f46051f8640 1 --2- 172.21.15.133:0/317112005 >> v2:172.21.15.162:3300/0 conn(0x7f46001584f0 0x7f460015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458467315 0 0) 0x7f46000cbe60 con 0x7f4600158110 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e8003530 con 0x7f4600158110 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3206243139 0 0) 0x7f45e8003530 con 0x7f4600158110 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f460006eb70 msgr2=0x7f46000b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f460006eb70 0x7f46000b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 >> v2:172.21.15.162:3300/0 conn(0x7f46001584f0 msgr2=0x7f460015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/317112005 >> v2:172.21.15.162:3300/0 conn(0x7f46001584f0 0x7f460015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460015b640 con 0x7f4600158110 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f45f0001680 con 0x7f4600158110 2024-04-25T10:07:18.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 >> v1:172.21.15.133:6789/0 conn(0x7f4600158110 legacy=0x7f460006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e80039c0 0x7f45e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f460006eb70 0x7f45e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e8004d50 0x7f45e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46000b19b0 con 0x7f45e8004d50 2024-04-25T10:07:18.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460015b360 con 0x7f45e80039c0 2024-04-25T10:07:18.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e800b9d0 con 0x7f460006eb70 2024-04-25T10:07:18.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f46051f8640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e8004d50 0x7f45e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f46049f7640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e80039c0 0x7f45e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45fffff640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f460006eb70 0x7f45e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f46051f8640 1 -- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f460006eb70 msgr2=0x7f45e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f46051f8640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f460006eb70 0x7f45e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f46051f8640 1 -- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e80039c0 msgr2=0x7f45e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f46051f8640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e80039c0 0x7f45e8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f46051f8640 1 -- 172.21.15.133:0/317112005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e8003530 con 0x7f45e8004d50 2024-04-25T10:07:18.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.089+0000 7f45fffff640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f460006eb70 0x7f45e8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46051f8640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e8004d50 0x7f45e80051d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009fc0 tx=0x7f45f801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ac99bda29424f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80b2070 con 0x7f45e8004d50 2024-04-25T10:07:18.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f80a5cd0 con 0x7f45e8004d50 2024-04-25T10:07:18.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f45ff7fe640 1 -- 172.21.15.133:0/317112005 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8009c10 con 0x7f45e8004d50 2024-04-25T10:07:18.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 -- 172.21.15.133:0/317112005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e8004d50 msgr2=0x7f45e80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 --2- 172.21.15.133:0/317112005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e8004d50 0x7f45e80051d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009fc0 tx=0x7f45f801ccc0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46051f8640 1 -- 172.21.15.133:0/317112005 reap_dead start 2024-04-25T10:07:18.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 -- 172.21.15.133:0/317112005 shutdown_connections 2024-04-25T10:07:18.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 -- 172.21.15.133:0/317112005 >> 172.21.15.133:0/317112005 conn(0x7f4600067000 msgr2=0x7f460006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 -- 172.21.15.133:0/317112005 shutdown_connections 2024-04-25T10:07:18.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 -- 172.21.15.133:0/317112005 wait complete. 2024-04-25T10:07:18.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 Processor -- start 2024-04-25T10:07:18.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 -- start start 2024-04-25T10:07:18.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e8004d50 0x7f46001f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46000b1430 0x7f460006aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46001f2550 0x7f46001f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46000cd670 con 0x7f46001f2550 2024-04-25T10:07:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460015bc00 con 0x7f45e8004d50 2024-04-25T10:07:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46000cd7f0 con 0x7f46000b1430 2024-04-25T10:07:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46051f8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46001f2550 0x7f46001f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46051f8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46001f2550 0x7f46001f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36552/0 (socket says 172.21.15.133:36552) 2024-04-25T10:07:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46051f8640 1 -- 172.21.15.133:0/4052449398 learned_addr learned my addr 172.21.15.133:0/4052449398 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46049f7640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e8004d50 0x7f46001f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f45fffff640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46000b1430 0x7f460006aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46051f8640 1 -- 172.21.15.133:0/4052449398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46000b1430 msgr2=0x7f460006aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46051f8640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46000b1430 0x7f460006aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46051f8640 1 -- 172.21.15.133:0/4052449398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e8004d50 msgr2=0x7f46001f71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46051f8640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e8004d50 0x7f46001f71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46051f8640 1 -- 172.21.15.133:0/4052449398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46000c95c0 con 0x7f46001f2550 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f45fffff640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46000b1430 0x7f460006aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f46051f8640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46001f2550 0x7f46001f2a00 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f45f80b0040 tx=0x7f45f8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f45fd7fa640 1 -- 172.21.15.133:0/4052449398 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80b2040 con 0x7f46001f2550 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f45fd7fa640 1 -- 172.21.15.133:0/4052449398 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f80a5ad0 con 0x7f46001f2550 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f4606c5b640 1 -- 172.21.15.133:0/4052449398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46001f7700 con 0x7f46001f2550 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.093+0000 7f45fd7fa640 1 -- 172.21.15.133:0/4052449398 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8009c10 con 0x7f46001f2550 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.097+0000 7f4606c5b640 1 -- 172.21.15.133:0/4052449398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46000c9b20 con 0x7f46001f2550 2024-04-25T10:07:18.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.097+0000 7f4606c5b640 1 -- 172.21.15.133:0/4052449398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4600068ed0 con 0x7f46001f2550 2024-04-25T10:07:18.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.097+0000 7f45fd7fa640 1 -- 172.21.15.133:0/4052449398 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45f80bf060 con 0x7f46001f2550 2024-04-25T10:07:18.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.097+0000 7f45fd7fa640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f60 0x7f45ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.097+0000 7f45fd7fa640 1 -- 172.21.15.133:0/4052449398 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f45f811eb40 con 0x7f46001f2550 2024-04-25T10:07:18.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.097+0000 7f46049f7640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f60 0x7f45ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.097+0000 7f46049f7640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f60 0x7f45ec069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f45f0000c10 tx=0x7f45f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.101+0000 7f45fd7fa640 1 -- 172.21.15.133:0/4052449398 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46001f7700 con 0x7f46001f2550 2024-04-25T10:07:18.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.281+0000 7f4606c5b640 1 -- 172.21.15.133:0/4052449398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46001554a0 con 0x7f46001f2550 2024-04-25T10:07:18.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.281+0000 7f45fd7fa640 1 -- 172.21.15.133:0/4052449398 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f46000c95c0 con 0x7f46001f2550 2024-04-25T10:07:18.288 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:18.288 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":41,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1579659657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1579659657},{"type":"v1","addr":"172.21.15.162:6835","nonce":1579659657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/232744252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":232744252},{"type":"v1","addr":"172.21.15.162:6833","nonce":232744252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7149,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/912719603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":912719603},{"type":"v1","addr":"172.21.15.133:6835","nonce":912719603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:36.516727+0000","modified":"2024-04-25T10:06:58.925769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/2310040121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2310040121},{"type":"v1","addr":"172.21.15.133:6837","nonce":2310040121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:18.288 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 41 2024-04-25T10:07:18.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 -- 172.21.15.133:0/4052449398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f60 msgr2=0x7f45ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f60 0x7f45ec069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f45f0000c10 tx=0x7f45f0009040 comp rx=0 tx=0).stop 2024-04-25T10:07:18.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 -- 172.21.15.133:0/4052449398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46001f2550 msgr2=0x7f46001f2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46001f2550 0x7f46001f2a00 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f45f80b0040 tx=0x7f45f8009750 comp rx=0 tx=0).stop 2024-04-25T10:07:18.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 -- 172.21.15.133:0/4052449398 shutdown_connections 2024-04-25T10:07:18.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f60 0x7f45ec069420 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46001f2550 0x7f46001f2a00 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46000b1430 0x7f460006aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 --2- 172.21.15.133:0/4052449398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e8004d50 0x7f46001f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 -- 172.21.15.133:0/4052449398 >> 172.21.15.133:0/4052449398 conn(0x7f4600067000 msgr2=0x7f46001573a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 -- 172.21.15.133:0/4052449398 shutdown_connections 2024-04-25T10:07:18.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.285+0000 7f4606c5b640 1 -- 172.21.15.133:0/4052449398 wait complete. 2024-04-25T10:07:18.306 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:18.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd625427640 1 Processor -- start 2024-04-25T10:07:18.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd625427640 1 -- start start 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd625427640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd62015db00 0x7fd6200b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd625427640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd6200d0bd0 0x7fd620161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd625427640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6200ceb20 con 0x7fd6200d0770 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd625427640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6200b7740 con 0x7fd6200d0bd0 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd625427640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200b7a20 con 0x7fd62015db00 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd61f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd6200d0bd0 0x7fd620161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd61e7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd6200d0770 0x7fd6201614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45542/0 (socket says 172.21.15.133:45542) 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd61e7fc640 1 -- 172.21.15.133:0/1098895686 learned_addr learned my addr 172.21.15.133:0/1098895686 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd61effd640 1 --2- 172.21.15.133:0/1098895686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd62015db00 0x7fd6200b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd61f7fe640 1 -- 172.21.15.133:0/1098895686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd62015db00 msgr2=0x7fd6200b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd61f7fe640 1 --2- 172.21.15.133:0/1098895686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd62015db00 0x7fd6200b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd61f7fe640 1 -- 172.21.15.133:0/1098895686 >> v1:172.21.15.133:6789/0 conn(0x7fd6200d0770 legacy=0x7fd6201614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd61f7fe640 1 -- 172.21.15.133:0/1098895686 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6200b97f0 con 0x7fd6200d0bd0 2024-04-25T10:07:18.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.417+0000 7fd61dffb640 1 -- 172.21.15.133:0/1098895686 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2081507325 0 0) 0x7fd6200ceb20 con 0x7fd6200d0770 2024-04-25T10:07:18.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd61f7fe640 1 --2- 172.21.15.133:0/1098895686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6200d0bd0 0x7fd620161bd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d90 tx=0x7fd61801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abb21e75c7296ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd61dffb640 1 -- 172.21.15.133:0/1098895686 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180b2070 con 0x7fd6200d0bd0 2024-04-25T10:07:18.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd61dffb640 1 -- 172.21.15.133:0/1098895686 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6180a5a00 con 0x7fd6200d0bd0 2024-04-25T10:07:18.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd61dffb640 1 -- 172.21.15.133:0/1098895686 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180a5dd0 con 0x7fd6200d0bd0 2024-04-25T10:07:18.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd625427640 1 -- 172.21.15.133:0/1098895686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6200d0bd0 msgr2=0x7fd620161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd625427640 1 --2- 172.21.15.133:0/1098895686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6200d0bd0 0x7fd620161bd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d90 tx=0x7fd61801c940 comp rx=0 tx=0).stop 2024-04-25T10:07:18.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd625427640 1 -- 172.21.15.133:0/1098895686 shutdown_connections 2024-04-25T10:07:18.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd625427640 1 --2- 172.21.15.133:0/1098895686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6200d0bd0 0x7fd620161bd0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd625427640 1 --2- 172.21.15.133:0/1098895686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd62015db00 0x7fd6200b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd625427640 1 -- 172.21.15.133:0/1098895686 >> 172.21.15.133:0/1098895686 conn(0x7fd6200727c0 msgr2=0x7fd620072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd625427640 1 -- 172.21.15.133:0/1098895686 shutdown_connections 2024-04-25T10:07:18.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd625427640 1 -- 172.21.15.133:0/1098895686 wait complete. 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd625427640 1 Processor -- start 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.421+0000 7fd625427640 1 -- start start 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd625427640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6200d0770 0x7fd62015c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd625427640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6200d0bd0 0x7fd6200b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd625427640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd62015db00 0x7fd6200b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd625427640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200b7430 con 0x7fd6200d0770 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd625427640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200d1060 con 0x7fd62015db00 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd625427640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200d18a0 con 0x7fd6200d0bd0 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6200d0770 0x7fd62015c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6200d0770 0x7fd62015c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36566/0 (socket says 172.21.15.133:36566) 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61effd640 1 -- 172.21.15.133:0/2840495612 learned_addr learned my addr 172.21.15.133:0/2840495612 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61e7fc640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6200d0bd0 0x7fd6200b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61effd640 1 -- 172.21.15.133:0/2840495612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6200d0bd0 msgr2=0x7fd6200b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61f7fe640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd62015db00 0x7fd6200b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61effd640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6200d0bd0 0x7fd6200b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61effd640 1 -- 172.21.15.133:0/2840495612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd62015db00 msgr2=0x7fd6200b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61effd640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd62015db00 0x7fd6200b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61effd640 1 -- 172.21.15.133:0/2840495612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd620158610 con 0x7fd6200d0770 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61e7fc640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6200d0bd0 0x7fd6200b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61effd640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6200d0770 0x7fd62015c6c0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d800 tx=0x7fd61000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd6077fe640 1 -- 172.21.15.133:0/2840495612 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61000b6c0 con 0x7fd6200d0770 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd6077fe640 1 -- 172.21.15.133:0/2840495612 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd610019070 con 0x7fd6200d0770 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd625427640 1 -- 172.21.15.133:0/2840495612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6201588f0 con 0x7fd6200d0770 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd6077fe640 1 -- 172.21.15.133:0/2840495612 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61000ba60 con 0x7fd6200d0770 2024-04-25T10:07:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd625427640 1 -- 172.21.15.133:0/2840495612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd620158eb0 con 0x7fd6200d0770 2024-04-25T10:07:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd625427640 1 -- 172.21.15.133:0/2840495612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd620068ef0 con 0x7fd6200d0770 2024-04-25T10:07:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd6077fe640 1 -- 172.21.15.133:0/2840495612 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd61000bc00 con 0x7fd6200d0770 2024-04-25T10:07:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd6077fe640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd60c066f60 0x7fd60c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd6077fe640 1 -- 172.21.15.133:0/2840495612 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd610085710 con 0x7fd6200d0770 2024-04-25T10:07:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.425+0000 7fd61e7fc640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd60c066f60 0x7fd60c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.429+0000 7fd61e7fc640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd60c066f60 0x7fd60c069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd614007fd0 tx=0x7fd614009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.429+0000 7fd6077fe640 1 -- 172.21.15.133:0/2840495612 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6100564e0 con 0x7fd6200d0770 2024-04-25T10:07:18.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.585+0000 7fd625427640 1 -- 172.21.15.133:0/2840495612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6200b2750 con 0x7fd6200d0770 2024-04-25T10:07:18.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd6077fe640 1 -- 172.21.15.133:0/2840495612 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17167 (secure 0 0 0) 0x7fd610083e20 con 0x7fd6200d0770 2024-04-25T10:07:18.592 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:18.593 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":70,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:06:56.893382+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:34.764755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:35.504408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6835/272966073":"2024-04-26T10:06:56.893338+0000","172.21.15.162:6833/272966073":"2024-04-26T10:06:56.893338+0000","172.21.15.133:6837/236214118":"2024-04-26T10:06:55.901820+0000","172.21.15.133:6836/236214118":"2024-04-26T10:06:55.901820+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 -- 172.21.15.133:0/2840495612 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd60c066f60 msgr2=0x7fd60c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd60c066f60 0x7fd60c069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd614007fd0 tx=0x7fd614009040 comp rx=0 tx=0).stop 2024-04-25T10:07:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 -- 172.21.15.133:0/2840495612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6200d0770 msgr2=0x7fd62015c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6200d0770 0x7fd62015c6c0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d800 tx=0x7fd61000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 -- 172.21.15.133:0/2840495612 shutdown_connections 2024-04-25T10:07:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd62015db00 0x7fd6200b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6200d0bd0 0x7fd6200b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd60c066f60 0x7fd60c069420 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 --2- 172.21.15.133:0/2840495612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6200d0770 0x7fd62015c6c0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 -- 172.21.15.133:0/2840495612 >> 172.21.15.133:0/2840495612 conn(0x7fd6200727c0 msgr2=0x7fd620167920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 -- 172.21.15.133:0/2840495612 shutdown_connections 2024-04-25T10:07:18.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.589+0000 7fd625427640 1 -- 172.21.15.133:0/2840495612 wait complete. 2024-04-25T10:07:18.610 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:07:18.611 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:18.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.713+0000 7f8750e87640 1 Processor -- start 2024-04-25T10:07:18.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.713+0000 7f8750e87640 1 -- start start 2024-04-25T10:07:18.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f874c15db00 0x7f874c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d0770 0x7f874c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874c0ceb20 con 0x7f874c1614c0 2024-04-25T10:07:18.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c0d1110 con 0x7f874c15db00 2024-04-25T10:07:18.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0b7740 con 0x7f874c0d0770 2024-04-25T10:07:18.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874a59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f874c15db00 0x7f874c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874a59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f874c15db00 0x7f874c0b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36576/0 (socket says 172.21.15.133:36576) 2024-04-25T10:07:18.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874a59c640 1 -- 172.21.15.133:0/1667536612 learned_addr learned my addr 172.21.15.133:0/1667536612 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:18.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8749d9b640 1 --2- 172.21.15.133:0/1667536612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d0770 0x7f874c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874a59c640 1 -- 172.21.15.133:0/1667536612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d0770 msgr2=0x7f874c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874a59c640 1 --2- 172.21.15.133:0/1667536612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d0770 0x7f874c0d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874a59c640 1 -- 172.21.15.133:0/1667536612 >> v1:172.21.15.133:6789/0 conn(0x7f874c1614c0 legacy=0x7f874c161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874a59c640 1 -- 172.21.15.133:0/1667536612 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c0b97f0 con 0x7f874c15db00 2024-04-25T10:07:18.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874959a640 1 -- 172.21.15.133:0/1667536612 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049719909 0 0) 0x7f874c0ceb20 con 0x7f874c1614c0 2024-04-25T10:07:18.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874a59c640 1 --2- 172.21.15.133:0/1667536612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15db00 0x7f874c0b2990 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f873c009870 tx=0x7f873c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f79de3715f21b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874959a640 1 -- 172.21.15.133:0/1667536612 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c0a95f0 con 0x7f874c15db00 2024-04-25T10:07:18.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874959a640 1 -- 172.21.15.133:0/1667536612 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873c0a9790 con 0x7f874c15db00 2024-04-25T10:07:18.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f874959a640 1 -- 172.21.15.133:0/1667536612 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c0a9b60 con 0x7f874c15db00 2024-04-25T10:07:18.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 -- 172.21.15.133:0/1667536612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15db00 msgr2=0x7f874c0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 --2- 172.21.15.133:0/1667536612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15db00 0x7f874c0b2990 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f873c009870 tx=0x7f873c01c530 comp rx=0 tx=0).stop 2024-04-25T10:07:18.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 -- 172.21.15.133:0/1667536612 shutdown_connections 2024-04-25T10:07:18.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 --2- 172.21.15.133:0/1667536612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d0770 0x7f874c0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 --2- 172.21.15.133:0/1667536612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15db00 0x7f874c0b2990 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 -- 172.21.15.133:0/1667536612 >> 172.21.15.133:0/1667536612 conn(0x7f874c0727c0 msgr2=0x7f874c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 -- 172.21.15.133:0/1667536612 shutdown_connections 2024-04-25T10:07:18.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.717+0000 7f8750e87640 1 -- 172.21.15.133:0/1667536612 wait complete. 2024-04-25T10:07:18.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8750e87640 1 Processor -- start 2024-04-25T10:07:18.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8750e87640 1 -- start start 2024-04-25T10:07:18.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8750e87640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d0770 0x7f874c158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8750e87640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15db00 0x7f874c160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8750e87640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f874c1614c0 0x7f874c156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8750e87640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0b7430 con 0x7f874c15db00 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8750e87640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0b82c0 con 0x7f874c1614c0 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8750e87640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0d18a0 con 0x7f874c0d0770 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8749d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15db00 0x7f874c160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8749d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15db00 0x7f874c160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36590/0 (socket says 172.21.15.133:36590) 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8749d9b640 1 -- 172.21.15.133:0/1440135977 learned_addr learned my addr 172.21.15.133:0/1440135977 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f874a59c640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d0770 0x7f874c158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f874ad9d640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f874c1614c0 0x7f874c156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8749d9b640 1 -- 172.21.15.133:0/1440135977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d0770 msgr2=0x7f874c158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8749d9b640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d0770 0x7f874c158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8749d9b640 1 -- 172.21.15.133:0/1440135977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f874c1614c0 msgr2=0x7f874c156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8749d9b640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f874c1614c0 0x7f874c156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8749d9b640 1 -- 172.21.15.133:0/1440135977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c160d70 con 0x7f874c15db00 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f874a59c640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d0770 0x7f874c158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8749d9b640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15db00 0x7f874c160830 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f874000c820 tx=0x7f874000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f873affd640 1 -- 172.21.15.133:0/1440135977 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740015070 con 0x7f874c15db00 2024-04-25T10:07:18.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f873affd640 1 -- 172.21.15.133:0/1440135977 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87400079d0 con 0x7f874c15db00 2024-04-25T10:07:18.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8750e87640 1 -- 172.21.15.133:0/1440135977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c157490 con 0x7f874c15db00 2024-04-25T10:07:18.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f873affd640 1 -- 172.21.15.133:0/1440135977 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740007dc0 con 0x7f874c15db00 2024-04-25T10:07:18.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8750e87640 1 -- 172.21.15.133:0/1440135977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c157a50 con 0x7f874c15db00 2024-04-25T10:07:18.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.721+0000 7f8750e87640 1 -- 172.21.15.133:0/1440135977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c068ef0 con 0x7f874c15db00 2024-04-25T10:07:18.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.725+0000 7f873affd640 1 -- 172.21.15.133:0/1440135977 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8740018070 con 0x7f874c15db00 2024-04-25T10:07:18.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.725+0000 7f873affd640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f872c066f60 0x7f872c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.725+0000 7f873affd640 1 -- 172.21.15.133:0/1440135977 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f8740085060 con 0x7f874c15db00 2024-04-25T10:07:18.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.725+0000 7f874a59c640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f872c066f60 0x7f872c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.725+0000 7f874a59c640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f872c066f60 0x7f872c069420 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f873c0ae000 tx=0x7f873c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.729+0000 7f873affd640 1 -- 172.21.15.133:0/1440135977 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8740055e30 con 0x7f874c15db00 2024-04-25T10:07:18.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.905+0000 7f8750e87640 1 -- 172.21.15.133:0/1440135977 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f874c0b93f0 con 0x7f874c15db00 2024-04-25T10:07:18.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.905+0000 7f873affd640 1 -- 172.21.15.133:0/1440135977 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f8740055730 con 0x7f874c15db00 2024-04-25T10:07:18.911 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:18.911 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":41,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1579659657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1579659657},{"type":"v1","addr":"172.21.15.162:6835","nonce":1579659657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/232744252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":232744252},{"type":"v1","addr":"172.21.15.162:6833","nonce":232744252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7149,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/912719603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":912719603},{"type":"v1","addr":"172.21.15.133:6835","nonce":912719603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:36.516727+0000","modified":"2024-04-25T10:06:58.925769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/2310040121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2310040121},{"type":"v1","addr":"172.21.15.133:6837","nonce":2310040121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:18.911 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 41 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 -- 172.21.15.133:0/1440135977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f872c066f60 msgr2=0x7f872c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f872c066f60 0x7f872c069420 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f873c0ae000 tx=0x7f873c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 -- 172.21.15.133:0/1440135977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15db00 msgr2=0x7f874c160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15db00 0x7f874c160830 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f874000c820 tx=0x7f874000cce0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 -- 172.21.15.133:0/1440135977 shutdown_connections 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f874c1614c0 0x7f874c156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f872c066f60 0x7f872c069420 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15db00 0x7f874c160830 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 --2- 172.21.15.133:0/1440135977 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d0770 0x7f874c158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 -- 172.21.15.133:0/1440135977 >> 172.21.15.133:0/1440135977 conn(0x7f874c0727c0 msgr2=0x7f874c0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 -- 172.21.15.133:0/1440135977 shutdown_connections 2024-04-25T10:07:18.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:18.909+0000 7f8750e87640 1 -- 172.21.15.133:0/1440135977 wait complete. 2024-04-25T10:07:18.929 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:19.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.041+0000 7f630a75a640 1 Processor -- start 2024-04-25T10:07:19.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.041+0000 7f630a75a640 1 -- start start 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63040b1f70 0x7f630406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f63040ce330 0x7f63040ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63040c61d0 con 0x7f630415c9a0 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63040cece0 con 0x7f63040ce330 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63040c64b0 con 0x7f63040b1f70 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f6303fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f630415c9a0 0x7f63040c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45560/0 (socket says 172.21.15.133:45560) 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f6308cf7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f63040ce330 0x7f63040ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f6303fff640 1 -- 172.21.15.133:0/3127808667 learned_addr learned my addr 172.21.15.133:0/3127808667 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f63037fe640 1 --2- 172.21.15.133:0/3127808667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63040b1f70 0x7f630406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f6308cf7640 1 -- 172.21.15.133:0/3127808667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63040b1f70 msgr2=0x7f630406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f6308cf7640 1 --2- 172.21.15.133:0/3127808667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63040b1f70 0x7f630406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f6308cf7640 1 -- 172.21.15.133:0/3127808667 >> v1:172.21.15.133:6789/0 conn(0x7f630415c9a0 legacy=0x7f63040c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f6308cf7640 1 -- 172.21.15.133:0/3127808667 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630406d4f0 con 0x7f63040ce330 2024-04-25T10:07:19.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f63037fe640 1 --2- 172.21.15.133:0/3127808667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63040b1f70 0x7f630406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:19.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f6308cf7640 1 --2- 172.21.15.133:0/3127808667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63040ce330 0x7f63040ce7a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009d90 tx=0x7f62fc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89f51d244efdd73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f6302ffd640 1 -- 172.21.15.133:0/3127808667 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0b2070 con 0x7f63040ce330 2024-04-25T10:07:19.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f6302ffd640 1 -- 172.21.15.133:0/3127808667 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc0a5910 con 0x7f63040ce330 2024-04-25T10:07:19.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f6302ffd640 1 -- 172.21.15.133:0/3127808667 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0a5ce0 con 0x7f63040ce330 2024-04-25T10:07:19.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 -- 172.21.15.133:0/3127808667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63040ce330 msgr2=0x7f63040ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 --2- 172.21.15.133:0/3127808667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63040ce330 0x7f63040ce7a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009d90 tx=0x7f62fc01c940 comp rx=0 tx=0).stop 2024-04-25T10:07:19.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 -- 172.21.15.133:0/3127808667 shutdown_connections 2024-04-25T10:07:19.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 --2- 172.21.15.133:0/3127808667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63040ce330 0x7f63040ce7a0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 --2- 172.21.15.133:0/3127808667 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63040b1f70 0x7f630406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 -- 172.21.15.133:0/3127808667 >> 172.21.15.133:0/3127808667 conn(0x7f6304067000 msgr2=0x7f630406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 -- 172.21.15.133:0/3127808667 shutdown_connections 2024-04-25T10:07:19.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.045+0000 7f630a75a640 1 -- 172.21.15.133:0/3127808667 wait complete. 2024-04-25T10:07:19.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f630a75a640 1 Processor -- start 2024-04-25T10:07:19.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f630a75a640 1 -- start start 2024-04-25T10:07:19.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f630a75a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63040b1f70 0x7f630414e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f630a75a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63040ce330 0x7f630415e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f630a75a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415c9a0 0x7f630414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f630a75a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630415dad0 con 0x7f63040b1f70 2024-04-25T10:07:19.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f630a75a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630415d950 con 0x7f63040ce330 2024-04-25T10:07:19.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f630a75a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630415dc50 con 0x7f630415c9a0 2024-04-25T10:07:19.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6303fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63040b1f70 0x7f630414e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6303fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63040b1f70 0x7f630414e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36620/0 (socket says 172.21.15.133:36620) 2024-04-25T10:07:19.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6303fff640 1 -- 172.21.15.133:0/2799706941 learned_addr learned my addr 172.21.15.133:0/2799706941 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f63037fe640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63040ce330 0x7f630415e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6303fff640 1 -- 172.21.15.133:0/2799706941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415c9a0 msgr2=0x7f630414faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6303fff640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415c9a0 0x7f630414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6303fff640 1 -- 172.21.15.133:0/2799706941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63040ce330 msgr2=0x7f630415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6303fff640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63040ce330 0x7f630415e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6303fff640 1 -- 172.21.15.133:0/2799706941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630406eb50 con 0x7f63040b1f70 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f63037fe640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63040ce330 0x7f630415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6303fff640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63040b1f70 0x7f630414e250 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f62f400c420 tx=0x7f62f400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6300ff9640 1 -- 172.21.15.133:0/2799706941 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4017070 con 0x7f63040b1f70 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6300ff9640 1 -- 172.21.15.133:0/2799706941 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f4005cf0 con 0x7f63040b1f70 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f630a75a640 1 -- 172.21.15.133:0/2799706941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630414e790 con 0x7f63040b1f70 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f6300ff9640 1 -- 172.21.15.133:0/2799706941 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f40072f0 con 0x7f63040b1f70 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f630a75a640 1 -- 172.21.15.133:0/2799706941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630414ece0 con 0x7f63040b1f70 2024-04-25T10:07:19.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.049+0000 7f630a75a640 1 -- 172.21.15.133:0/2799706941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6304068ef0 con 0x7f63040b1f70 2024-04-25T10:07:19.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.053+0000 7f6300ff9640 1 -- 172.21.15.133:0/2799706941 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62f401a070 con 0x7f63040b1f70 2024-04-25T10:07:19.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.053+0000 7f6300ff9640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62f0066f60 0x7f62f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.053+0000 7f6300ff9640 1 -- 172.21.15.133:0/2799706941 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f62f40846d0 con 0x7f63040b1f70 2024-04-25T10:07:19.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.053+0000 7f63037fe640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62f0066f60 0x7f62f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.053+0000 7f63037fe640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62f0066f60 0x7f62f0069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f63040b45c0 tx=0x7f62f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.057+0000 7f6300ff9640 1 -- 172.21.15.133:0/2799706941 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62f4055400 con 0x7f63040b1f70 2024-04-25T10:07:19.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.253+0000 7f630a75a640 1 -- 172.21.15.133:0/2799706941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f630406b630 con 0x7f63040b1f70 2024-04-25T10:07:19.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.253+0000 7f6300ff9640 1 -- 172.21.15.133:0/2799706941 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f62f4054d00 con 0x7f63040b1f70 2024-04-25T10:07:19.259 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:19.259 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":41,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1579659657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1579659657},{"type":"v1","addr":"172.21.15.162:6835","nonce":1579659657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/232744252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":232744252},{"type":"v1","addr":"172.21.15.162:6833","nonce":232744252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7149,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/912719603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":912719603},{"type":"v1","addr":"172.21.15.133:6835","nonce":912719603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:36.516727+0000","modified":"2024-04-25T10:06:58.925769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/2310040121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2310040121},{"type":"v1","addr":"172.21.15.133:6837","nonce":2310040121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:19.259 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 41 2024-04-25T10:07:19.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 -- 172.21.15.133:0/2799706941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62f0066f60 msgr2=0x7f62f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62f0066f60 0x7f62f0069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f63040b45c0 tx=0x7f62f8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:19.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 -- 172.21.15.133:0/2799706941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63040b1f70 msgr2=0x7f630414e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63040b1f70 0x7f630414e250 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f62f400c420 tx=0x7f62f400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 -- 172.21.15.133:0/2799706941 shutdown_connections 2024-04-25T10:07:19.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415c9a0 0x7f630414faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63040ce330 0x7f630415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62f0066f60 0x7f62f0069420 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 --2- 172.21.15.133:0/2799706941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63040b1f70 0x7f630414e250 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 -- 172.21.15.133:0/2799706941 >> 172.21.15.133:0/2799706941 conn(0x7f6304067000 msgr2=0x7f63040c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 -- 172.21.15.133:0/2799706941 shutdown_connections 2024-04-25T10:07:19.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.257+0000 7f630a75a640 1 -- 172.21.15.133:0/2799706941 wait complete. 2024-04-25T10:07:19.278 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:19.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7e5e7b640 1 Processor -- start 2024-04-25T10:07:19.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7e5e7b640 1 -- start start 2024-04-25T10:07:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7e5e7b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff7e0158110 0x7ff7e00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7e5e7b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff7e01584f0 0x7ff7e015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7e5e7b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e00cbe60 con 0x7ff7e006e460 2024-04-25T10:07:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7e5e7b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e00b19b0 con 0x7ff7e01584f0 2024-04-25T10:07:19.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7e5e7b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e015b360 con 0x7ff7e0158110 2024-04-25T10:07:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7d7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff7e01584f0 0x7ff7e015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7dffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff7e006e460 0x7ff7e015e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45576/0 (socket says 172.21.15.133:45576) 2024-04-25T10:07:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7dffff640 1 -- 172.21.15.133:0/3081466150 learned_addr learned my addr 172.21.15.133:0/3081466150 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7df7fe640 1 --2- 172.21.15.133:0/3081466150 >> v2:172.21.15.162:3300/0 conn(0x7ff7e0158110 0x7ff7e00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7d7fff640 1 -- 172.21.15.133:0/3081466150 >> v2:172.21.15.162:3300/0 conn(0x7ff7e0158110 msgr2=0x7ff7e00b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7d7fff640 1 --2- 172.21.15.133:0/3081466150 >> v2:172.21.15.162:3300/0 conn(0x7ff7e0158110 0x7ff7e00b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7d7fff640 1 -- 172.21.15.133:0/3081466150 >> v1:172.21.15.133:6789/0 conn(0x7ff7e006e460 legacy=0x7ff7e015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7deffd640 1 -- 172.21.15.133:0/3081466150 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341179782 0 0) 0x7ff7e00cbe60 con 0x7ff7e006e460 2024-04-25T10:07:19.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.389+0000 7ff7d7fff640 1 -- 172.21.15.133:0/3081466150 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e015b640 con 0x7ff7e01584f0 2024-04-25T10:07:19.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7d7fff640 1 --2- 172.21.15.133:0/3081466150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e01584f0 0x7ff7e015ad10 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00b4d40 tx=0x7ff7c801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d739ad0d3a3d5ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7deffd640 1 -- 172.21.15.133:0/3081466150 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80b1070 con 0x7ff7e01584f0 2024-04-25T10:07:19.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7deffd640 1 -- 172.21.15.133:0/3081466150 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c80ac5f0 con 0x7ff7e01584f0 2024-04-25T10:07:19.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7deffd640 1 -- 172.21.15.133:0/3081466150 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80ac9c0 con 0x7ff7e01584f0 2024-04-25T10:07:19.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/3081466150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e01584f0 msgr2=0x7ff7e015ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 --2- 172.21.15.133:0/3081466150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e01584f0 0x7ff7e015ad10 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00b4d40 tx=0x7ff7c801c480 comp rx=0 tx=0).stop 2024-04-25T10:07:19.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/3081466150 shutdown_connections 2024-04-25T10:07:19.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 --2- 172.21.15.133:0/3081466150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e01584f0 0x7ff7e015ad10 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 --2- 172.21.15.133:0/3081466150 >> v2:172.21.15.162:3300/0 conn(0x7ff7e0158110 0x7ff7e00b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/3081466150 >> 172.21.15.133:0/3081466150 conn(0x7ff7e0067000 msgr2=0x7ff7e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/3081466150 shutdown_connections 2024-04-25T10:07:19.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/3081466150 wait complete. 2024-04-25T10:07:19.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 Processor -- start 2024-04-25T10:07:19.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 -- start start 2024-04-25T10:07:19.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e006e460 0x7ff7e00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7e0158110 0x7ff7e0140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7e01584f0 0x7ff7e0151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e00cd7f0 con 0x7ff7e006e460 2024-04-25T10:07:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e00cd670 con 0x7ff7e01584f0 2024-04-25T10:07:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e006d350 con 0x7ff7e0158110 2024-04-25T10:07:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7df7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e006e460 0x7ff7e00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7df7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e006e460 0x7ff7e00b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36634/0 (socket says 172.21.15.133:36634) 2024-04-25T10:07:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7df7fe640 1 -- 172.21.15.133:0/1701193703 learned_addr learned my addr 172.21.15.133:0/1701193703 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7d7fff640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7e0158110 0x7ff7e0140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7dffff640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7e01584f0 0x7ff7e0151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7df7fe640 1 -- 172.21.15.133:0/1701193703 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7e0158110 msgr2=0x7ff7e0140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7df7fe640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7e0158110 0x7ff7e0140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7df7fe640 1 -- 172.21.15.133:0/1701193703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7e01584f0 msgr2=0x7ff7e0151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7df7fe640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7e01584f0 0x7ff7e0151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7df7fe640 1 -- 172.21.15.133:0/1701193703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e0164320 con 0x7ff7e006e460 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7dffff640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7e01584f0 0x7ff7e0151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7df7fe640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e006e460 0x7ff7e00b1f00 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000d800 tx=0x7ff7d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7dcff9640 1 -- 172.21.15.133:0/1701193703 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000b6c0 con 0x7ff7e006e460 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7dcff9640 1 -- 172.21.15.133:0/1701193703 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d0019070 con 0x7ff7e006e460 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.393+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/1701193703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e0164600 con 0x7ff7e006e460 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.397+0000 7ff7dcff9640 1 -- 172.21.15.133:0/1701193703 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000bae0 con 0x7ff7e006e460 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.397+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/1701193703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e0164bc0 con 0x7ff7e006e460 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.397+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/1701193703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e006bc20 con 0x7ff7e006e460 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.397+0000 7ff7dcff9640 1 -- 172.21.15.133:0/1701193703 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7d000bc80 con 0x7ff7e006e460 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.397+0000 7ff7dcff9640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7c0066f60 0x7ff7c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.397+0000 7ff7d7fff640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7c0066f60 0x7ff7c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.397+0000 7ff7dcff9640 1 -- 172.21.15.133:0/1701193703 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff7d0085710 con 0x7ff7e006e460 2024-04-25T10:07:19.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.397+0000 7ff7d7fff640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7c0066f60 0x7ff7c0069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80ae000 tx=0x7ff7c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.401+0000 7ff7dcff9640 1 -- 172.21.15.133:0/1701193703 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7d0056570 con 0x7ff7e006e460 2024-04-25T10:07:19.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.549+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/1701193703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7e014d790 con 0x7ff7e006e460 2024-04-25T10:07:19.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.549+0000 7ff7dcff9640 1 -- 172.21.15.133:0/1701193703 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+17167 (secure 0 0 0) 0x7ff7e014d790 con 0x7ff7e006e460 2024-04-25T10:07:19.554 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:19.554 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":70,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:06:56.893382+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:34.764755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:35.504408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6835/272966073":"2024-04-26T10:06:56.893338+0000","172.21.15.162:6833/272966073":"2024-04-26T10:06:56.893338+0000","172.21.15.133:6837/236214118":"2024-04-26T10:06:55.901820+0000","172.21.15.133:6836/236214118":"2024-04-26T10:06:55.901820+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:19.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.549+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/1701193703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7c0066f60 msgr2=0x7ff7c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.549+0000 7ff7e5e7b640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7c0066f60 0x7ff7c0069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80ae000 tx=0x7ff7c8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:19.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.549+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/1701193703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e006e460 msgr2=0x7ff7e00b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.549+0000 7ff7e5e7b640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e006e460 0x7ff7e00b1f00 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000d800 tx=0x7ff7d000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.549+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/1701193703 shutdown_connections 2024-04-25T10:07:19.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.549+0000 7ff7e5e7b640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7e01584f0 0x7ff7e0151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.549+0000 7ff7e5e7b640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7e0158110 0x7ff7e0140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.549+0000 7ff7e5e7b640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff7c0066f60 0x7ff7c0069420 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.549+0000 7ff7e5e7b640 1 --2- 172.21.15.133:0/1701193703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7e006e460 0x7ff7e00b1f00 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.553+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/1701193703 >> 172.21.15.133:0/1701193703 conn(0x7ff7e0067000 msgr2=0x7ff7e01501a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.553+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/1701193703 shutdown_connections 2024-04-25T10:07:19.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.553+0000 7ff7e5e7b640 1 -- 172.21.15.133:0/1701193703 wait complete. 2024-04-25T10:07:19.572 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:07:19.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fbc0c4640 1 Processor -- start 2024-04-25T10:07:19.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fbc0c4640 1 -- start start 2024-04-25T10:07:19.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fbc0c4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 0x7f6fb40b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fbc0c4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6fb40aebc0 0x7f6fb40af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fbc0c4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb41471f0 con 0x7f6fb406e8b0 2024-04-25T10:07:19.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fbc0c4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb4147490 con 0x7f6fb40aebc0 2024-04-25T10:07:19.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fbc0c4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40b19c0 con 0x7f6fb415eb40 2024-04-25T10:07:19.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fba661640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6fb40aebc0 0x7f6fb40af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fb965f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 0x7f6fb40b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fba661640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6fb40aebc0 0x7f6fb40af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:39292/0 (socket says 172.21.15.133:39292) 2024-04-25T10:07:19.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fba661640 1 -- 172.21.15.133:0/2987783636 learned_addr learned my addr 172.21.15.133:0/2987783636 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:19.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fb965f640 1 --2- 172.21.15.133:0/2987783636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 0x7f6fb40b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:34590/0 (socket says 172.21.15.133:34590) 2024-04-25T10:07:19.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fb965f640 1 -- 172.21.15.133:0/2987783636 >> v2:172.21.15.162:3300/0 conn(0x7f6fb40aebc0 msgr2=0x7f6fb40af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fb965f640 1 --2- 172.21.15.133:0/2987783636 >> v2:172.21.15.162:3300/0 conn(0x7f6fb40aebc0 0x7f6fb40af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fb965f640 1 -- 172.21.15.133:0/2987783636 >> v1:172.21.15.162:6789/0 conn(0x7f6fb406e8b0 legacy=0x7f6fb4146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fb965f640 1 -- 172.21.15.133:0/2987783636 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40b1090 con 0x7f6fb415eb40 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.677+0000 7f6fba661640 1 --2- 172.21.15.133:0/2987783636 >> v2:172.21.15.162:3300/0 conn(0x7f6fb40aebc0 0x7f6fb40af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fb965f640 1 --2- 172.21.15.133:0/2987783636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 0x7f6fb40b1e50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40097f0 tx=0x7f6fa401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f6a4539f626193c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fb8e5e640 1 -- 172.21.15.133:0/2987783636 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40b0070 con 0x7f6fb415eb40 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fb8e5e640 1 -- 172.21.15.133:0/2987783636 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa40ab600 con 0x7f6fb415eb40 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fb8e5e640 1 -- 172.21.15.133:0/2987783636 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40ab9d0 con 0x7f6fb415eb40 2024-04-25T10:07:19.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/2987783636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 msgr2=0x7f6fb40b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fbc0c4640 1 --2- 172.21.15.133:0/2987783636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 0x7f6fb40b1e50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40097f0 tx=0x7f6fa401c410 comp rx=0 tx=0).stop 2024-04-25T10:07:19.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/2987783636 shutdown_connections 2024-04-25T10:07:19.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fbc0c4640 1 --2- 172.21.15.133:0/2987783636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 0x7f6fb40b1e50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fbc0c4640 1 --2- 172.21.15.133:0/2987783636 >> v2:172.21.15.162:3300/0 conn(0x7f6fb40aebc0 0x7f6fb40af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/2987783636 >> 172.21.15.133:0/2987783636 conn(0x7f6fb4066f70 msgr2=0x7f6fb406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/2987783636 shutdown_connections 2024-04-25T10:07:19.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.681+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/2987783636 wait complete. 2024-04-25T10:07:19.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fbc0c4640 1 Processor -- start 2024-04-25T10:07:19.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fbc0c4640 1 -- start start 2024-04-25T10:07:19.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fbc0c4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fb406e8b0 0x7f6fb413ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fbc0c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb40aebc0 0x7f6fb40cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fbc0c4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 0x7f6fb40b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fbc0c4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb406de40 con 0x7f6fb40aebc0 2024-04-25T10:07:19.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fbc0c4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb406dcc0 con 0x7f6fb406e8b0 2024-04-25T10:07:19.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fbc0c4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40b2390 con 0x7f6fb415eb40 2024-04-25T10:07:19.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fb965f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb40aebc0 0x7f6fb40cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fb965f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb40aebc0 0x7f6fb40cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36644/0 (socket says 172.21.15.133:36644) 2024-04-25T10:07:19.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fb965f640 1 -- 172.21.15.133:0/597569247 learned_addr learned my addr 172.21.15.133:0/597569247 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:19.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fb9e60640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fb406e8b0 0x7f6fb413ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fba661640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 0x7f6fb40b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fb965f640 1 -- 172.21.15.133:0/597569247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 msgr2=0x7f6fb40b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fb965f640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 0x7f6fb40b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fb965f640 1 -- 172.21.15.133:0/597569247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fb406e8b0 msgr2=0x7f6fb413ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fb965f640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fb406e8b0 0x7f6fb413ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fb965f640 1 -- 172.21.15.133:0/597569247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4145030 con 0x7f6fb40aebc0 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fb9e60640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fb406e8b0 0x7f6fb413ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fb965f640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb40aebc0 0x7f6fb40cee00 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40ae040 tx=0x7f6fa40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6faa7fc640 1 -- 172.21.15.133:0/597569247 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40b0040 con 0x7f6fb40aebc0 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6faa7fc640 1 -- 172.21.15.133:0/597569247 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa40b9070 con 0x7f6fb40aebc0 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/597569247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb4143f90 con 0x7f6fb40aebc0 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6faa7fc640 1 -- 172.21.15.133:0/597569247 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4006e20 con 0x7f6fb40aebc0 2024-04-25T10:07:19.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/597569247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb4144550 con 0x7f6fb40aebc0 2024-04-25T10:07:19.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/597569247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb4068e80 con 0x7f6fb40aebc0 2024-04-25T10:07:19.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.685+0000 7f6faa7fc640 1 -- 172.21.15.133:0/597569247 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fa40b4650 con 0x7f6fb40aebc0 2024-04-25T10:07:19.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.689+0000 7f6faa7fc640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f98066f90 0x7f6f98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.689+0000 7f6faa7fc640 1 -- 172.21.15.133:0/597569247 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f6fa411ee20 con 0x7f6fb40aebc0 2024-04-25T10:07:19.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.689+0000 7f6fb9e60640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f98066f90 0x7f6f98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.689+0000 7f6fb9e60640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f98066f90 0x7f6f98069450 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0001710 tx=0x7f6fb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.693+0000 7f6faa7fc640 1 -- 172.21.15.133:0/597569247 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fa4016090 con 0x7f6fb40aebc0 2024-04-25T10:07:19.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.873+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/597569247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6fb406b960 con 0x7f6fb40aebc0 2024-04-25T10:07:19.876 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:07:19.873+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:07:19.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.873+0000 7f6faa7fc640 1 -- 172.21.15.133:0/597569247 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6fa40efbf0 con 0x7f6fb40aebc0 2024-04-25T10:07:19.879 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:07:19.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/597569247 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f98066f90 msgr2=0x7f6f98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f98066f90 0x7f6f98069450 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0001710 tx=0x7f6fb0009040 comp rx=0 tx=0).stop 2024-04-25T10:07:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/597569247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb40aebc0 msgr2=0x7f6fb40cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb40aebc0 0x7f6fb40cee00 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40ae040 tx=0x7f6fa40191b0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/597569247 shutdown_connections 2024-04-25T10:07:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6fb415eb40 0x7f6fb40b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6f98066f90 0x7f6f98069450 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fb40aebc0 0x7f6fb40cee00 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 --2- 172.21.15.133:0/597569247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6fb406e8b0 0x7f6fb413ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/597569247 >> 172.21.15.133:0/597569247 conn(0x7f6fb4066f70 msgr2=0x7f6fb4143600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/597569247 shutdown_connections 2024-04-25T10:07:19.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:19.877+0000 7f6fbc0c4640 1 -- 172.21.15.133:0/597569247 wait complete. 2024-04-25T10:07:19.889 INFO:tasks.ceph.mds.c.smithi133.stderr: -2289> 2024-04-25T10:07:01.901+0000 7f169d48c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:19.899 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:07:19.942 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:07:20.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.005+0000 7f18541c3640 1 Processor -- start 2024-04-25T10:07:20.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.005+0000 7f18541c3640 1 -- start start 2024-04-25T10:07:20.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.005+0000 7f18541c3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ce600 0x7f184c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.005+0000 7f18541c3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f184c0d1920 0x7f184c0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.005+0000 7f18541c3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f184c15ebd0 con 0x7f184c15d9a0 2024-04-25T10:07:20.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.005+0000 7f18541c3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184c0cefa0 con 0x7f184c0d1920 2024-04-25T10:07:20.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.005+0000 7f18541c3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184c15eeb0 con 0x7f184c0ce600 2024-04-25T10:07:20.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1851f5f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f184c15d9a0 0x7f184c0d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45600/0 (socket says 172.21.15.133:45600) 2024-04-25T10:07:20.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1851f5f640 1 -- 172.21.15.133:0/1907159415 learned_addr learned my addr 172.21.15.133:0/1907159415 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:20.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660725048 0 0) 0x7f184c15ebd0 con 0x7f184c15d9a0 2024-04-25T10:07:20.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1838003530 con 0x7f184c15d9a0 2024-04-25T10:07:20.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f185175e640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ce600 0x7f184c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1852760640 1 --2- 172.21.15.133:0/1907159415 >> v2:172.21.15.162:3300/0 conn(0x7f184c0d1920 0x7f184c0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1150406523 0 0) 0x7f1838003530 con 0x7f184c15d9a0 2024-04-25T10:07:20.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ce600 msgr2=0x7f184c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ce600 0x7f184c0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 >> v2:172.21.15.162:3300/0 conn(0x7f184c0d1920 msgr2=0x7f184c0d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 --2- 172.21.15.133:0/1907159415 >> v2:172.21.15.162:3300/0 conn(0x7f184c0d1920 0x7f184c0d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c0d0330 con 0x7f184c15d9a0 2024-04-25T10:07:20.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1848001680 con 0x7f184c15d9a0 2024-04-25T10:07:20.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 >> v1:172.21.15.133:6789/0 conn(0x7f184c15d9a0 legacy=0x7f184c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18380039c0 0x7f1838003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1838004810 0x7f1838004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ce600 0x7f18380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c15eeb0 con 0x7f18380039c0 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183800b9d0 con 0x7f1838004810 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183800bba0 con 0x7f184c0ce600 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1851f5f640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18380039c0 0x7f1838003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f185175e640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1838004810 0x7f1838004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1852760640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ce600 0x7f18380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1851f5f640 1 -- 172.21.15.133:0/1907159415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ce600 msgr2=0x7f18380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1851f5f640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ce600 0x7f18380051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1851f5f640 1 -- 172.21.15.133:0/1907159415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1838004810 msgr2=0x7f1838004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1851f5f640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1838004810 0x7f1838004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1851f5f640 1 -- 172.21.15.133:0/1907159415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1838003530 con 0x7f18380039c0 2024-04-25T10:07:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1851f5f640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18380039c0 0x7f1838003de0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f184800c1c0 tx=0x7f184801cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394eb04f54f86390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848007c70 con 0x7f18380039c0 2024-04-25T10:07:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18480a9020 con 0x7f18380039c0 2024-04-25T10:07:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1850f5d640 1 -- 172.21.15.133:0/1907159415 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848005b00 con 0x7f18380039c0 2024-04-25T10:07:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f18541c3640 1 -- 172.21.15.133:0/1907159415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18380039c0 msgr2=0x7f1838003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f18541c3640 1 --2- 172.21.15.133:0/1907159415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18380039c0 0x7f1838003de0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f184800c1c0 tx=0x7f184801cb90 comp rx=0 tx=0).stop 2024-04-25T10:07:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f1852760640 1 -- 172.21.15.133:0/1907159415 reap_dead start 2024-04-25T10:07:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f18541c3640 1 -- 172.21.15.133:0/1907159415 shutdown_connections 2024-04-25T10:07:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f18541c3640 1 -- 172.21.15.133:0/1907159415 >> 172.21.15.133:0/1907159415 conn(0x7f184c06bf80 msgr2=0x7f184c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.009+0000 7f18541c3640 1 -- 172.21.15.133:0/1907159415 shutdown_connections 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 -- 172.21.15.133:0/1907159415 wait complete. 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 Processor -- start 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 -- start start 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18380039c0 0x7f184c0cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f184c0b4bf0 0x7f184c0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ca900 0x7f184c0cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0d2470 con 0x7f184c0b4bf0 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0d22f0 con 0x7f18380039c0 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184c15e590 con 0x7f184c0ca900 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f185175e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f184c0b4bf0 0x7f184c0c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f185175e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f184c0b4bf0 0x7f184c0c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36676/0 (socket says 172.21.15.133:36676) 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f185175e640 1 -- 172.21.15.133:0/3228303137 learned_addr learned my addr 172.21.15.133:0/3228303137 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f1852760640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ca900 0x7f184c0cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f1851f5f640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18380039c0 0x7f184c0cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f185175e640 1 -- 172.21.15.133:0/3228303137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ca900 msgr2=0x7f184c0cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f185175e640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ca900 0x7f184c0cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f185175e640 1 -- 172.21.15.133:0/3228303137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18380039c0 msgr2=0x7f184c0cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f185175e640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18380039c0 0x7f184c0cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f185175e640 1 -- 172.21.15.133:0/3228303137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c0c9ae0 con 0x7f184c0b4bf0 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f1851f5f640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18380039c0 0x7f184c0cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f185175e640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f184c0b4bf0 0x7f184c0c95a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f183c011560 tx=0x7f183c011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18427fc640 1 -- 172.21.15.133:0/3228303137 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c015930 con 0x7f184c0b4bf0 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18427fc640 1 -- 172.21.15.133:0/3228303137 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c015ad0 con 0x7f184c0b4bf0 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18427fc640 1 -- 172.21.15.133:0/3228303137 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c01d400 con 0x7f184c0b4bf0 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 -- 172.21.15.133:0/3228303137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184c0ccf20 con 0x7f184c0b4bf0 2024-04-25T10:07:20.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 -- 172.21.15.133:0/3228303137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c0cd4e0 con 0x7f184c0b4bf0 2024-04-25T10:07:20.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.013+0000 7f18541c3640 1 -- 172.21.15.133:0/3228303137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184c068e80 con 0x7f184c0b4bf0 2024-04-25T10:07:20.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.017+0000 7f18427fc640 1 -- 172.21.15.133:0/3228303137 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f183c01d5a0 con 0x7f184c0b4bf0 2024-04-25T10:07:20.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.017+0000 7f18427fc640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1830066f60 0x7f1830069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.017+0000 7f18427fc640 1 -- 172.21.15.133:0/3228303137 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f183c085b30 con 0x7f184c0b4bf0 2024-04-25T10:07:20.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.017+0000 7f1851f5f640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1830066f60 0x7f1830069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.017+0000 7f1851f5f640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1830066f60 0x7f1830069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f18480aa000 tx=0x7f184c15eda0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.021+0000 7f18427fc640 1 -- 172.21.15.133:0/3228303137 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f183c019030 con 0x7f184c0b4bf0 2024-04-25T10:07:20.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.213+0000 7f18541c3640 1 -- 172.21.15.133:0/3228303137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f184c0d2800 con 0x7f184c0b4bf0 2024-04-25T10:07:20.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.877+0000 7f18427fc640 1 -- 172.21.15.133:0/3228303137 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f183c056820 con 0x7f184c0b4bf0 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 -- 172.21.15.133:0/3228303137 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1830066f60 msgr2=0x7f1830069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1830066f60 0x7f1830069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f18480aa000 tx=0x7f184c15eda0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 -- 172.21.15.133:0/3228303137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f184c0b4bf0 msgr2=0x7f184c0c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f184c0b4bf0 0x7f184c0c95a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f183c011560 tx=0x7f183c011a20 comp rx=0 tx=0).stop 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 -- 172.21.15.133:0/3228303137 shutdown_connections 2024-04-25T10:07:20.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f184c0ca900 0x7f184c0cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1830066f60 0x7f1830069420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f184c0b4bf0 0x7f184c0c95a0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 --2- 172.21.15.133:0/3228303137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18380039c0 0x7f184c0cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 -- 172.21.15.133:0/3228303137 >> 172.21.15.133:0/3228303137 conn(0x7f184c06bf80 msgr2=0x7f184c0d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 -- 172.21.15.133:0/3228303137 shutdown_connections 2024-04-25T10:07:20.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:20.881+0000 7f18541c3640 1 -- 172.21.15.133:0/3228303137 wait complete. 2024-04-25T10:07:20.909 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:07:21.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f040360a640 1 Processor -- start 2024-04-25T10:07:21.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f040360a640 1 -- start start 2024-04-25T10:07:21.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f040360a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f03fc071cb0 0x7f03fc15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f040360a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc15c9b0 con 0x7f03fc0b4940 2024-04-25T10:07:21.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f040360a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc15cc90 con 0x7f03fc15c250 2024-04-25T10:07:21.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f040360a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03fc072a90 con 0x7f03fc071cb0 2024-04-25T10:07:21.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f04013a6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f03fc0b4940 0x7f03fc0b4d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45604/0 (socket says 172.21.15.133:45604) 2024-04-25T10:07:21.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f04013a6640 1 -- 172.21.15.133:0/3886004151 learned_addr learned my addr 172.21.15.133:0/3886004151 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:21.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279677294 0 0) 0x7f03fc15c9b0 con 0x7f03fc0b4940 2024-04-25T10:07:21.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e8003530 con 0x7f03fc0b4940 2024-04-25T10:07:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f0400ba5640 1 --2- 172.21.15.133:0/3886004151 >> v2:172.21.15.162:3300/0 conn(0x7f03fc071cb0 0x7f03fc15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3970211711 0 0) 0x7f03e8003530 con 0x7f03fc0b4940 2024-04-25T10:07:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 >> v2:172.21.15.162:3300/0 conn(0x7f03fc071cb0 msgr2=0x7f03fc15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 --2- 172.21.15.133:0/3886004151 >> v2:172.21.15.162:3300/0 conn(0x7f03fc071cb0 0x7f03fc15bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 >> v1:172.21.15.162:6789/0 conn(0x7f03fc15c250 legacy=0x7f03fc0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc0711d0 con 0x7f03fc0b4940 2024-04-25T10:07:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f03f8001680 con 0x7f03fc0b4940 2024-04-25T10:07:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 >> v1:172.21.15.133:6789/0 conn(0x7f03fc0b4940 legacy=0x7f03fc0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03e8003a10 0x7f03e8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03e8004810 0x7f03e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.073+0000 7f03f3fff640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 0x7f03e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e800c140 con 0x7f03e8005220 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e800c2f0 con 0x7f03e8003a10 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e800c4f0 con 0x7f03e8004810 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f0401ba7640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 0x7f03e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f04013a6640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03e8003a10 0x7f03e8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f0400ba5640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03e8004810 0x7f03e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f0401ba7640 1 -- 172.21.15.133:0/3886004151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03e8004810 msgr2=0x7f03e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f0401ba7640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03e8004810 0x7f03e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f0401ba7640 1 -- 172.21.15.133:0/3886004151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03e8003a10 msgr2=0x7f03e8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f0401ba7640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03e8003a10 0x7f03e8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f0401ba7640 1 -- 172.21.15.133:0/3886004151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e8003530 con 0x7f03e8005220 2024-04-25T10:07:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f04013a6640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03e8003a10 0x7f03e8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f0401ba7640 1 -- 172.21.15.133:0/3886004151 reap_dead start 2024-04-25T10:07:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f0401ba7640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 0x7f03e800b9d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f03f4001e70 tx=0x7f03f401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5ab889e8a95c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4001a90 con 0x7f03e8005220 2024-04-25T10:07:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f4003ba0 con 0x7f03e8005220 2024-04-25T10:07:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f03f3fff640 1 -- 172.21.15.133:0/3886004151 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4005020 con 0x7f03e8005220 2024-04-25T10:07:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f040360a640 1 -- 172.21.15.133:0/3886004151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 msgr2=0x7f03e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f040360a640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 0x7f03e800b9d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f03f4001e70 tx=0x7f03f401c780 comp rx=0 tx=0).stop 2024-04-25T10:07:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f040360a640 1 -- 172.21.15.133:0/3886004151 shutdown_connections 2024-04-25T10:07:21.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f040360a640 1 --2- 172.21.15.133:0/3886004151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 0x7f03e800b9d0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f040360a640 1 -- 172.21.15.133:0/3886004151 >> 172.21.15.133:0/3886004151 conn(0x7f03fc06d370 msgr2=0x7f03fc06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f040360a640 1 -- 172.21.15.133:0/3886004151 shutdown_connections 2024-04-25T10:07:21.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.077+0000 7f040360a640 1 -- 172.21.15.133:0/3886004151 wait complete. 2024-04-25T10:07:21.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f040360a640 1 Processor -- start 2024-04-25T10:07:21.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f040360a640 1 -- start start 2024-04-25T10:07:21.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f040360a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 0x7f03fc159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f040360a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03fc0adf30 0x7f03fc0a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f040360a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03fc0ae3b0 0x7f03fc143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f040360a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc071790 con 0x7f03e8005220 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f040360a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1452d0 con 0x7f03fc0adf30 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f040360a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc145450 con 0x7f03fc0ae3b0 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f04013a6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 0x7f03fc159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f04013a6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 0x7f03fc159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36686/0 (socket says 172.21.15.133:36686) 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f04013a6640 1 -- 172.21.15.133:0/501462601 learned_addr learned my addr 172.21.15.133:0/501462601 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f04013a6640 1 -- 172.21.15.133:0/501462601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03fc0ae3b0 msgr2=0x7f03fc143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f04013a6640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03fc0ae3b0 0x7f03fc143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f04013a6640 1 -- 172.21.15.133:0/501462601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03fc0adf30 msgr2=0x7f03fc0a96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f04013a6640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03fc0adf30 0x7f03fc0a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f04013a6640 1 -- 172.21.15.133:0/501462601 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc13d2e0 con 0x7f03e8005220 2024-04-25T10:07:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f04013a6640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 0x7f03fc159060 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f03f8000c10 tx=0x7f03f800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f03f1ffb640 1 -- 172.21.15.133:0/501462601 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f800c690 con 0x7f03e8005220 2024-04-25T10:07:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f03f1ffb640 1 -- 172.21.15.133:0/501462601 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f8018070 con 0x7f03e8005220 2024-04-25T10:07:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f040360a640 1 -- 172.21.15.133:0/501462601 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc13d5c0 con 0x7f03e8005220 2024-04-25T10:07:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f03f1ffb640 1 -- 172.21.15.133:0/501462601 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f800ca70 con 0x7f03e8005220 2024-04-25T10:07:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f040360a640 1 -- 172.21.15.133:0/501462601 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc13db80 con 0x7f03e8005220 2024-04-25T10:07:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.081+0000 7f040360a640 1 -- 172.21.15.133:0/501462601 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc1f55f0 con 0x7f03e8005220 2024-04-25T10:07:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.085+0000 7f03f1ffb640 1 -- 172.21.15.133:0/501462601 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03f800cc10 con 0x7f03e8005220 2024-04-25T10:07:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.085+0000 7f03f1ffb640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f03e0066f60 0x7f03e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.085+0000 7f03f1ffb640 1 -- 172.21.15.133:0/501462601 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f03f80850f0 con 0x7f03e8005220 2024-04-25T10:07:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.085+0000 7f0400ba5640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f03e0066f60 0x7f03e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.085+0000 7f0400ba5640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f03e0066f60 0x7f03e0069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f03ec006b20 tx=0x7f03ec006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.089+0000 7f03f1ffb640 1 -- 172.21.15.133:0/501462601 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03f8055e70 con 0x7f03e8005220 2024-04-25T10:07:21.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:21.249+0000 7f040360a640 1 -- 172.21.15.133:0/501462601 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03fc069000 con 0x7f03e8005220 2024-04-25T10:07:21.364 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:21.367 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:22.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.005+0000 7f03f1ffb640 1 -- 172.21.15.133:0/501462601 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) v1 ==== 181+0+0 (secure 0 0 0) 0x7f03f8055770 con 0x7f03e8005220 2024-04-25T10:07:22.010 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:07:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 -- 172.21.15.133:0/501462601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f03e0066f60 msgr2=0x7f03e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f03e0066f60 0x7f03e0069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f03ec006b20 tx=0x7f03ec006a30 comp rx=0 tx=0).stop 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 -- 172.21.15.133:0/501462601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 msgr2=0x7f03fc159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 0x7f03fc159060 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f03f8000c10 tx=0x7f03f800eb50 comp rx=0 tx=0).stop 2024-04-25T10:07:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 -- 172.21.15.133:0/501462601 shutdown_connections 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03fc0ae3b0 0x7f03fc143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03fc0adf30 0x7f03fc0a96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f03e0066f60 0x7f03e0069420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 --2- 172.21.15.133:0/501462601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e8005220 0x7f03fc159060 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 -- 172.21.15.133:0/501462601 >> 172.21.15.133:0/501462601 conn(0x7f03fc06d370 msgr2=0x7f03fc142140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 -- 172.21.15.133:0/501462601 shutdown_connections 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.009+0000 7f040360a640 1 -- 172.21.15.133:0/501462601 wait complete. 2024-04-25T10:07:22.036 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:07:22.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f564322e640 1 Processor -- start 2024-04-25T10:07:22.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f564322e640 1 -- start start 2024-04-25T10:07:22.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f564322e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f563c15b910 0x7f563c0ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f564322e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f563c0cebd0 0x7f563c15cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f564322e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c15bf40 con 0x7f563c15b530 2024-04-25T10:07:22.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f564322e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c1589c0 con 0x7f563c15b910 2024-04-25T10:07:22.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f564322e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c15c200 con 0x7f563c0cebd0 2024-04-25T10:07:22.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f563bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f563c15b910 0x7f563c0ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f563bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f563c15b910 0x7f563c0ce690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36696/0 (socket says 172.21.15.133:36696) 2024-04-25T10:07:22.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f563bfff640 1 -- 172.21.15.133:0/2514517946 learned_addr learned my addr 172.21.15.133:0/2514517946 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:22.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f56417cb640 1 --2- 172.21.15.133:0/2514517946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f563c0cebd0 0x7f563c15cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f563bfff640 1 -- 172.21.15.133:0/2514517946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f563c0cebd0 msgr2=0x7f563c15cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f563bfff640 1 --2- 172.21.15.133:0/2514517946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f563c0cebd0 0x7f563c15cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f563bfff640 1 -- 172.21.15.133:0/2514517946 >> v1:172.21.15.162:6789/0 conn(0x7f563c15b530 legacy=0x7f563c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.157+0000 7f563bfff640 1 -- 172.21.15.133:0/2514517946 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c15c4e0 con 0x7f563c15b910 2024-04-25T10:07:22.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f563bfff640 1 --2- 172.21.15.133:0/2514517946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563c15b910 0x7f563c0ce690 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5630009870 tx=0x7f563001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba6c9ebd78a803c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f563b7fe640 1 -- 172.21.15.133:0/2514517946 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56300b0070 con 0x7f563c15b910 2024-04-25T10:07:22.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f563b7fe640 1 -- 172.21.15.133:0/2514517946 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56300ab600 con 0x7f563c15b910 2024-04-25T10:07:22.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f563b7fe640 1 -- 172.21.15.133:0/2514517946 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56300ab9d0 con 0x7f563c15b910 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f564322e640 1 -- 172.21.15.133:0/2514517946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563c15b910 msgr2=0x7f563c0ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f564322e640 1 --2- 172.21.15.133:0/2514517946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563c15b910 0x7f563c0ce690 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5630009870 tx=0x7f563001c490 comp rx=0 tx=0).stop 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f564322e640 1 -- 172.21.15.133:0/2514517946 shutdown_connections 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f564322e640 1 --2- 172.21.15.133:0/2514517946 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f563c0cebd0 0x7f563c15cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f564322e640 1 --2- 172.21.15.133:0/2514517946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563c15b910 0x7f563c0ce690 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f564322e640 1 -- 172.21.15.133:0/2514517946 >> 172.21.15.133:0/2514517946 conn(0x7f563c06d370 msgr2=0x7f563c06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f564322e640 1 -- 172.21.15.133:0/2514517946 shutdown_connections 2024-04-25T10:07:22.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.161+0000 7f564322e640 1 -- 172.21.15.133:0/2514517946 wait complete. 2024-04-25T10:07:22.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f564322e640 1 Processor -- start 2024-04-25T10:07:22.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f564322e640 1 -- start start 2024-04-25T10:07:22.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f564322e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f563c0cebd0 0x7f563c142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f564322e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f563c15b530 0x7f563c14d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f564322e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563c15b910 0x7f563c14d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f564322e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c15de90 con 0x7f563c15b910 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f564322e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c15dd10 con 0x7f563c15b530 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f564322e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c15e010 con 0x7f563c0cebd0 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56417cb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563c15b910 0x7f563c14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56417cb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563c15b910 0x7f563c14d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36708/0 (socket says 172.21.15.133:36708) 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56417cb640 1 -- 172.21.15.133:0/373766646 learned_addr learned my addr 172.21.15.133:0/373766646 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f5640fca640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f563c0cebd0 0x7f563c142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f563bfff640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f563c15b530 0x7f563c14d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56417cb640 1 -- 172.21.15.133:0/373766646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f563c0cebd0 msgr2=0x7f563c142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56417cb640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f563c0cebd0 0x7f563c142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56417cb640 1 -- 172.21.15.133:0/373766646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f563c15b530 msgr2=0x7f563c14d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56417cb640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f563c15b530 0x7f563c14d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56417cb640 1 -- 172.21.15.133:0/373766646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c155410 con 0x7f563c15b910 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56417cb640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563c15b910 0x7f563c14d550 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5634007e60 tx=0x7f56340106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56397fa640 1 -- 172.21.15.133:0/373766646 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634018070 con 0x7f563c15b910 2024-04-25T10:07:22.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56397fa640 1 -- 172.21.15.133:0/373766646 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5634010e90 con 0x7f563c15b910 2024-04-25T10:07:22.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56397fa640 1 -- 172.21.15.133:0/373766646 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634013ae0 con 0x7f563c15b910 2024-04-25T10:07:22.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f564322e640 1 -- 172.21.15.133:0/373766646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c144720 con 0x7f563c15b910 2024-04-25T10:07:22.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f564322e640 1 -- 172.21.15.133:0/373766646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c1556d0 con 0x7f563c15b910 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f564322e640 1 -- 172.21.15.133:0/373766646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c069000 con 0x7f563c15b910 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.165+0000 7f56397fa640 1 -- 172.21.15.133:0/373766646 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f563401b070 con 0x7f563c15b910 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.169+0000 7f56397fa640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5628066f60 0x7f5628069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.169+0000 7f56397fa640 1 -- 172.21.15.133:0/373766646 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5921+0+0 (secure 0 0 0) 0x7f5634087f70 con 0x7f563c15b910 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.169+0000 7f5640fca640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5628066f60 0x7f5628069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.169+0000 7f5640fca640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5628066f60 0x7f5628069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f562c00b810 tx=0x7f562c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.173+0000 7f56397fa640 1 -- 172.21.15.133:0/373766646 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5634058f30 con 0x7f563c15b910 2024-04-25T10:07:22.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:22.333+0000 7f564322e640 1 -- 172.21.15.133:0/373766646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f563c06c2d0 con 0x7f563c15b910 2024-04-25T10:07:23.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.013+0000 7f56397fa640 1 -- 172.21.15.133:0/373766646 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5634058830 con 0x7f563c15b910 2024-04-25T10:07:23.018 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:07:23.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 -- 172.21.15.133:0/373766646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5628066f60 msgr2=0x7f5628069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5628066f60 0x7f5628069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f562c00b810 tx=0x7f562c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:23.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 -- 172.21.15.133:0/373766646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563c15b910 msgr2=0x7f563c14d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563c15b910 0x7f563c14d550 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5634007e60 tx=0x7f56340106c0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 -- 172.21.15.133:0/373766646 shutdown_connections 2024-04-25T10:07:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5628066f60 0x7f5628069420 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563c15b910 0x7f563c14d550 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f563c15b530 0x7f563c14d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 --2- 172.21.15.133:0/373766646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f563c0cebd0 0x7f563c142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 -- 172.21.15.133:0/373766646 >> 172.21.15.133:0/373766646 conn(0x7f563c06d370 msgr2=0x7f563c0cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 -- 172.21.15.133:0/373766646 shutdown_connections 2024-04-25T10:07:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.017+0000 7f564322e640 1 -- 172.21.15.133:0/373766646 wait complete. 2024-04-25T10:07:23.046 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:07:23.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.157+0000 7f9fb11eb640 1 Processor -- start 2024-04-25T10:07:23.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.157+0000 7f9fb11eb640 1 -- start start 2024-04-25T10:07:23.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9fb11eb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fac143ea0 0x7f9fac0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9fb11eb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9fac0b2150 0x7f9fac06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9fb11eb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fac15ac50 con 0x7f9fac15b9b0 2024-04-25T10:07:23.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9fb11eb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fac0cd180 con 0x7f9fac0b2150 2024-04-25T10:07:23.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9fb11eb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac15aed0 con 0x7f9fac143ea0 2024-04-25T10:07:23.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9faad9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fac143ea0 0x7f9fac0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9fa3fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9fac0b2150 0x7f9fac06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9fa3fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9fac0b2150 0x7f9fac06e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:39412/0 (socket says 172.21.15.133:39412) 2024-04-25T10:07:23.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9fa3fff640 1 -- 172.21.15.133:0/3827722503 learned_addr learned my addr 172.21.15.133:0/3827722503 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:23.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9faad9d640 1 -- 172.21.15.133:0/3827722503 >> v2:172.21.15.162:3300/0 conn(0x7f9fac0b2150 msgr2=0x7f9fac06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9faad9d640 1 --2- 172.21.15.133:0/3827722503 >> v2:172.21.15.162:3300/0 conn(0x7f9fac0b2150 0x7f9fac06e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9faad9d640 1 -- 172.21.15.133:0/3827722503 >> v1:172.21.15.162:6789/0 conn(0x7f9fac15b9b0 legacy=0x7f9fac15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9faad9d640 1 -- 172.21.15.133:0/3827722503 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac15b1b0 con 0x7f9fac143ea0 2024-04-25T10:07:23.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.161+0000 7f9fa3fff640 1 --2- 172.21.15.133:0/3827722503 >> v2:172.21.15.162:3300/0 conn(0x7f9fac0b2150 0x7f9fac06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9faad9d640 1 --2- 172.21.15.133:0/3827722503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fac143ea0 0x7f9fac0cc810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0097c0 tx=0x7f9f9c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8657559f8611b3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9faa59c640 1 -- 172.21.15.133:0/3827722503 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0b1070 con 0x7f9fac143ea0 2024-04-25T10:07:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9faa59c640 1 -- 172.21.15.133:0/3827722503 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9c0ac410 con 0x7f9fac143ea0 2024-04-25T10:07:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9faa59c640 1 -- 172.21.15.133:0/3827722503 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0ac7e0 con 0x7f9fac143ea0 2024-04-25T10:07:23.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3827722503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fac143ea0 msgr2=0x7f9fac0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 --2- 172.21.15.133:0/3827722503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fac143ea0 0x7f9fac0cc810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0097c0 tx=0x7f9f9c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3827722503 shutdown_connections 2024-04-25T10:07:23.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 --2- 172.21.15.133:0/3827722503 >> v2:172.21.15.162:3300/0 conn(0x7f9fac0b2150 0x7f9fac06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 --2- 172.21.15.133:0/3827722503 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fac143ea0 0x7f9fac0cc810 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3827722503 >> 172.21.15.133:0/3827722503 conn(0x7f9fac066f70 msgr2=0x7f9fac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3827722503 shutdown_connections 2024-04-25T10:07:23.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3827722503 wait complete. 2024-04-25T10:07:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 Processor -- start 2024-04-25T10:07:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 -- start start 2024-04-25T10:07:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fac0b2150 0x7f9fac139470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fac143ea0 0x7f9fac154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fac15b9b0 0x7f9fac13d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac1675e0 con 0x7f9fac143ea0 2024-04-25T10:07:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac06ed00 con 0x7f9fac0b2150 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fb11eb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac167760 con 0x7f9fac15b9b0 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fa3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fac143ea0 0x7f9fac154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9faad9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fac0b2150 0x7f9fac139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fab59e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fac15b9b0 0x7f9fac13d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fa3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fac143ea0 0x7f9fac154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36718/0 (socket says 172.21.15.133:36718) 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fa3fff640 1 -- 172.21.15.133:0/3905014782 learned_addr learned my addr 172.21.15.133:0/3905014782 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fa3fff640 1 -- 172.21.15.133:0/3905014782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fac15b9b0 msgr2=0x7f9fac13d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fa3fff640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fac15b9b0 0x7f9fac13d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fa3fff640 1 -- 172.21.15.133:0/3905014782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fac0b2150 msgr2=0x7f9fac139470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fa3fff640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fac0b2150 0x7f9fac139470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fa3fff640 1 -- 172.21.15.133:0/3905014782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac13a170 con 0x7f9fac143ea0 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.165+0000 7f9fa3fff640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fac143ea0 0x7f9fac154950 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400e9d0 tx=0x7f9f9400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.169+0000 7f9fa37fe640 1 -- 172.21.15.133:0/3905014782 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94017070 con 0x7f9fac143ea0 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.169+0000 7f9fa37fe640 1 -- 172.21.15.133:0/3905014782 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9400ca60 con 0x7f9fac143ea0 2024-04-25T10:07:23.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.169+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3905014782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fac139bb0 con 0x7f9fac143ea0 2024-04-25T10:07:23.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.169+0000 7f9fa37fe640 1 -- 172.21.15.133:0/3905014782 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400ce10 con 0x7f9fac143ea0 2024-04-25T10:07:23.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.169+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3905014782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fac154150 con 0x7f9fac143ea0 2024-04-25T10:07:23.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.169+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3905014782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fac068e80 con 0x7f9fac143ea0 2024-04-25T10:07:23.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.169+0000 7f9fa37fe640 1 -- 172.21.15.133:0/3905014782 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f9401a070 con 0x7f9fac143ea0 2024-04-25T10:07:23.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.169+0000 7f9fa37fe640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 0x7f9f8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.169+0000 7f9fa37fe640 1 -- 172.21.15.133:0/3905014782 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f9f94084e40 con 0x7f9fac143ea0 2024-04-25T10:07:23.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.169+0000 7f9faad9d640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 0x7f9f8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.169+0000 7f9faad9d640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 0x7f9f8c069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0ae000 tx=0x7f9f9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.173+0000 7f9fa37fe640 1 -- 172.21.15.133:0/3905014782 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f940558c0 con 0x7f9fac143ea0 2024-04-25T10:07:23.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.337+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3905014782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9fac153f20 con 0x7f9fac143ea0 2024-04-25T10:07:23.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fa37fe640 1 -- 172.21.15.133:0/3905014782 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9f94059960 con 0x7f9fac143ea0 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3905014782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 msgr2=0x7f9f8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 0x7f9f8c069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0ae000 tx=0x7f9f9c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3905014782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fac143ea0 msgr2=0x7f9fac154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fac143ea0 0x7f9fac154950 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400e9d0 tx=0x7f9f9400ee90 comp rx=0 tx=0).stop 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3905014782 shutdown_connections 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9fac15b9b0 0x7f9fac13d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 0x7f9f8c069420 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fac143ea0 0x7f9fac154950 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 --2- 172.21.15.133:0/3905014782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fac0b2150 0x7f9fac139470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3905014782 >> 172.21.15.133:0/3905014782 conn(0x7f9fac066f70 msgr2=0x7f9fac159620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3905014782 shutdown_connections 2024-04-25T10:07:23.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.341+0000 7f9fb11eb640 1 -- 172.21.15.133:0/3905014782 wait complete. 2024-04-25T10:07:23.364 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T10:07:23.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff530d29640 1 Processor -- start 2024-04-25T10:07:23.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff530d29640 1 -- start start 2024-04-25T10:07:23.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff530d29640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff52c0ce330 0x7ff52c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff530d29640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c0c61d0 con 0x7ff52c0b1f70 2024-04-25T10:07:23.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff530d29640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c0c64b0 con 0x7ff52c15c9a0 2024-04-25T10:07:23.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff530d29640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c0cecc0 con 0x7ff52c0ce330 2024-04-25T10:07:23.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff529d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff52c0b1f70 0x7ff52c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45606/0 (socket says 172.21.15.133:45606) 2024-04-25T10:07:23.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff529d9b640 1 -- 172.21.15.133:0/838478389 learned_addr learned my addr 172.21.15.133:0/838478389 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895193872 0 0) 0x7ff52c0c61d0 con 0x7ff52c0b1f70 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff514003530 con 0x7ff52c0b1f70 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52ad9d640 1 --2- 172.21.15.133:0/838478389 >> v2:172.21.15.162:3300/0 conn(0x7ff52c0ce330 0x7ff52c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2722104985 0 0) 0x7ff514003530 con 0x7ff52c0b1f70 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 >> v2:172.21.15.162:3300/0 conn(0x7ff52c0ce330 msgr2=0x7ff52c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 --2- 172.21.15.133:0/838478389 >> v2:172.21.15.162:3300/0 conn(0x7ff52c0ce330 0x7ff52c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 >> v1:172.21.15.162:6789/0 conn(0x7ff52c15c9a0 legacy=0x7ff52c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c06d4f0 con 0x7ff52c0b1f70 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff520001680 con 0x7ff52c0b1f70 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52ad9d640 1 --2- 172.21.15.133:0/838478389 >> v2:172.21.15.162:3300/0 conn(0x7ff52c0ce330 0x7ff52c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 >> v1:172.21.15.133:6789/0 conn(0x7ff52c0b1f70 legacy=0x7ff52c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 --2- 172.21.15.133:0/838478389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5140039c0 0x7ff514003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 --2- 172.21.15.133:0/838478389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff514004810 0x7ff514004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 --2- 172.21.15.133:0/838478389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5140051d0 0x7ff51400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0cecc0 con 0x7ff5140051d0 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51400c120 con 0x7ff514004810 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51400c320 con 0x7ff5140039c0 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52ad9d640 1 --2- 172.21.15.133:0/838478389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5140051d0 0x7ff51400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52a59c640 1 --2- 172.21.15.133:0/838478389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5140039c0 0x7ff514003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff529d9b640 1 --2- 172.21.15.133:0/838478389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff514004810 0x7ff514004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52ad9d640 1 -- 172.21.15.133:0/838478389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5140039c0 msgr2=0x7ff514003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52ad9d640 1 --2- 172.21.15.133:0/838478389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5140039c0 0x7ff514003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52ad9d640 1 -- 172.21.15.133:0/838478389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff514004810 msgr2=0x7ff514004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52ad9d640 1 --2- 172.21.15.133:0/838478389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff514004810 0x7ff514004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.481+0000 7ff52ad9d640 1 -- 172.21.15.133:0/838478389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff514003530 con 0x7ff5140051d0 2024-04-25T10:07:23.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff52ad9d640 1 -- 172.21.15.133:0/838478389 reap_dead start 2024-04-25T10:07:23.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff52ad9d640 1 --2- 172.21.15.133:0/838478389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5140051d0 0x7ff51400b9d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff5240073e0 tx=0x7ff52401cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f258c21b7bb5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524009c80 con 0x7ff5140051d0 2024-04-25T10:07:23.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5240ad440 con 0x7ff5140051d0 2024-04-25T10:07:23.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff52959a640 1 -- 172.21.15.133:0/838478389 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5240ad830 con 0x7ff5140051d0 2024-04-25T10:07:23.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 -- 172.21.15.133:0/838478389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5140051d0 msgr2=0x7ff51400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 --2- 172.21.15.133:0/838478389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5140051d0 0x7ff51400b9d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff5240073e0 tx=0x7ff52401cf40 comp rx=0 tx=0).stop 2024-04-25T10:07:23.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 -- 172.21.15.133:0/838478389 shutdown_connections 2024-04-25T10:07:23.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 --2- 172.21.15.133:0/838478389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5140051d0 0x7ff51400b9d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 -- 172.21.15.133:0/838478389 >> 172.21.15.133:0/838478389 conn(0x7ff52c067000 msgr2=0x7ff52c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 -- 172.21.15.133:0/838478389 shutdown_connections 2024-04-25T10:07:23.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 -- 172.21.15.133:0/838478389 wait complete. 2024-04-25T10:07:23.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 Processor -- start 2024-04-25T10:07:23.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 -- start start 2024-04-25T10:07:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5140051d0 0x7ff52c1448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff52c1f12b0 0x7ff52c155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c1561f0 0x7ff52c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c15cf00 con 0x7ff52c1561f0 2024-04-25T10:07:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c15cd80 con 0x7ff5140051d0 2024-04-25T10:07:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.485+0000 7ff530d29640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0ce780 con 0x7ff52c1f12b0 2024-04-25T10:07:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c1561f0 0x7ff52c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c1561f0 0x7ff52c15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36728/0 (socket says 172.21.15.133:36728) 2024-04-25T10:07:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52ad9d640 1 -- 172.21.15.133:0/3485131349 learned_addr learned my addr 172.21.15.133:0/3485131349 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52a59c640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5140051d0 0x7ff52c1448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff529d9b640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff52c1f12b0 0x7ff52c155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52ad9d640 1 -- 172.21.15.133:0/3485131349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff52c1f12b0 msgr2=0x7ff52c155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52ad9d640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff52c1f12b0 0x7ff52c155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52ad9d640 1 -- 172.21.15.133:0/3485131349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5140051d0 msgr2=0x7ff52c1448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52ad9d640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5140051d0 0x7ff52c1448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52ad9d640 1 -- 172.21.15.133:0/3485131349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c151480 con 0x7ff52c1561f0 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff529d9b640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff52c1f12b0 0x7ff52c155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52ad9d640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c1561f0 0x7ff52c15ad90 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ff52401c9b0 tx=0x7ff514005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff51affd640 1 -- 172.21.15.133:0/3485131349 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5240bb070 con 0x7ff52c1561f0 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff51affd640 1 -- 172.21.15.133:0/3485131349 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5240ac5f0 con 0x7ff52c1561f0 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff51affd640 1 -- 172.21.15.133:0/3485131349 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5240a5e60 con 0x7ff52c1561f0 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff530d29640 1 -- 172.21.15.133:0/3485131349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c151760 con 0x7ff52c1561f0 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff530d29640 1 -- 172.21.15.133:0/3485131349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c151d20 con 0x7ff52c1561f0 2024-04-25T10:07:23.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff530d29640 1 -- 172.21.15.133:0/3485131349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c068ef0 con 0x7ff52c1561f0 2024-04-25T10:07:23.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff51affd640 1 -- 172.21.15.133:0/3485131349 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5240be070 con 0x7ff52c1561f0 2024-04-25T10:07:23.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff51affd640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff50c066f90 0x7ff50c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff51affd640 1 -- 172.21.15.133:0/3485131349 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ff52411ee50 con 0x7ff52c1561f0 2024-04-25T10:07:23.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52a59c640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff50c066f90 0x7ff50c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.489+0000 7ff52a59c640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff50c066f90 0x7ff50c069450 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff51c000c10 tx=0x7ff51c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.493+0000 7ff51affd640 1 -- 172.21.15.133:0/3485131349 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff524016c50 con 0x7ff52c1561f0 2024-04-25T10:07:23.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:23.713+0000 7ff530d29640 1 -- 172.21.15.133:0/3485131349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7ff52c144e10 con 0x7ff52c1561f0 2024-04-25T10:07:24.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.017+0000 7ff51affd640 1 -- 172.21.15.133:0/3485131349 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v271) v1 ==== 143+0+0 (secure 0 0 0) 0x7ff5240eff20 con 0x7ff52c1561f0 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 -- 172.21.15.133:0/3485131349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff50c066f90 msgr2=0x7ff50c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff50c066f90 0x7ff50c069450 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff51c000c10 tx=0x7ff51c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 -- 172.21.15.133:0/3485131349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c1561f0 msgr2=0x7ff52c15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c1561f0 0x7ff52c15ad90 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ff52401c9b0 tx=0x7ff514005840 comp rx=0 tx=0).stop 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 -- 172.21.15.133:0/3485131349 shutdown_connections 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff50c066f90 0x7ff50c069450 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c1561f0 0x7ff52c15ad90 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff52c1f12b0 0x7ff52c155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 --2- 172.21.15.133:0/3485131349 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5140051d0 0x7ff52c1448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 -- 172.21.15.133:0/3485131349 >> 172.21.15.133:0/3485131349 conn(0x7ff52c067000 msgr2=0x7ff52c0b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 -- 172.21.15.133:0/3485131349 shutdown_connections 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.021+0000 7ff530d29640 1 -- 172.21.15.133:0/3485131349 wait complete. 2024-04-25T10:07:24.047 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T10:07:24.047 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T10:07:24.047 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:07:24.048 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:24.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f23a558b640 1 Processor -- start 2024-04-25T10:07:24.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f23a558b640 1 -- start start 2024-04-25T10:07:24.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f23a558b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23a0159280 0x7f23a0072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f23a558b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f23a006b3a0 0x7f23a00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f23a558b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a015cdc0 con 0x7f23a00730e0 2024-04-25T10:07:24.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f23a558b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a006fb80 con 0x7f23a006b3a0 2024-04-25T10:07:24.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f23a558b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00709e0 con 0x7f23a0159280 2024-04-25T10:07:24.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f239f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f23a006b3a0 0x7f23a00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f239f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f23a006b3a0 0x7f23a00ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36744/0 (socket says 172.21.15.133:36744) 2024-04-25T10:07:24.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f239f7fe640 1 -- 172.21.15.133:0/364813020 learned_addr learned my addr 172.21.15.133:0/364813020 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:24.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f239effd640 1 --2- 172.21.15.133:0/364813020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23a0159280 0x7f23a0072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f239f7fe640 1 -- 172.21.15.133:0/364813020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23a0159280 msgr2=0x7f23a0072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f239f7fe640 1 --2- 172.21.15.133:0/364813020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23a0159280 0x7f23a0072ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f239f7fe640 1 -- 172.21.15.133:0/364813020 >> v1:172.21.15.162:6789/0 conn(0x7f23a00730e0 legacy=0x7f23a006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.169+0000 7f239f7fe640 1 -- 172.21.15.133:0/364813020 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a006fdd0 con 0x7f23a006b3a0 2024-04-25T10:07:24.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239f7fe640 1 --2- 172.21.15.133:0/364813020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23a006b3a0 0x7f23a00ccee0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2398009e50 tx=0x7f239801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c7b66fcc18cee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239dffb640 1 -- 172.21.15.133:0/364813020 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980a58d0 con 0x7f23a006b3a0 2024-04-25T10:07:24.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239dffb640 1 -- 172.21.15.133:0/364813020 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23980a5a70 con 0x7f23a006b3a0 2024-04-25T10:07:24.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239dffb640 1 -- 172.21.15.133:0/364813020 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980a5e40 con 0x7f23a006b3a0 2024-04-25T10:07:24.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 -- 172.21.15.133:0/364813020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23a006b3a0 msgr2=0x7f23a00ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 --2- 172.21.15.133:0/364813020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23a006b3a0 0x7f23a00ccee0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2398009e50 tx=0x7f239801ca00 comp rx=0 tx=0).stop 2024-04-25T10:07:24.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 -- 172.21.15.133:0/364813020 shutdown_connections 2024-04-25T10:07:24.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 --2- 172.21.15.133:0/364813020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23a006b3a0 0x7f23a00ccee0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 --2- 172.21.15.133:0/364813020 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23a0159280 0x7f23a0072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 -- 172.21.15.133:0/364813020 >> 172.21.15.133:0/364813020 conn(0x7f23a00b2370 msgr2=0x7f23a00b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 -- 172.21.15.133:0/364813020 shutdown_connections 2024-04-25T10:07:24.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 -- 172.21.15.133:0/364813020 wait complete. 2024-04-25T10:07:24.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 Processor -- start 2024-04-25T10:07:24.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 -- start start 2024-04-25T10:07:24.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a006b3a0 0x7f23a0157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23a0159280 0x7f23a00af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23a014c060 0x7f23a00cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a01597e0 con 0x7f23a014c060 2024-04-25T10:07:24.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0159660 con 0x7f23a006b3a0 2024-04-25T10:07:24.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f23a558b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00cdc30 con 0x7f23a0159280 2024-04-25T10:07:24.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23a014c060 0x7f23a00cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23a014c060 0x7f23a00cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36756/0 (socket says 172.21.15.133:36756) 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239f7fe640 1 -- 172.21.15.133:0/1492874615 learned_addr learned my addr 172.21.15.133:0/1492874615 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239effd640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a006b3a0 0x7f23a0157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239e7fc640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23a0159280 0x7f23a00af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239f7fe640 1 -- 172.21.15.133:0/1492874615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23a0159280 msgr2=0x7f23a00af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239f7fe640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23a0159280 0x7f23a00af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239f7fe640 1 -- 172.21.15.133:0/1492874615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a006b3a0 msgr2=0x7f23a0157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239f7fe640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a006b3a0 0x7f23a0157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.173+0000 7f239f7fe640 1 -- 172.21.15.133:0/1492874615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00b0fb0 con 0x7f23a014c060 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f239f7fe640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23a014c060 0x7f23a00cfa60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f23980af040 tx=0x7f23980197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f23877fe640 1 -- 172.21.15.133:0/1492874615 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980a58d0 con 0x7f23a014c060 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f23877fe640 1 -- 172.21.15.133:0/1492874615 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23980be070 con 0x7f23a014c060 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f23a558b640 1 -- 172.21.15.133:0/1492874615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a00afc70 con 0x7f23a014c060 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f23877fe640 1 -- 172.21.15.133:0/1492874615 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398009510 con 0x7f23a014c060 2024-04-25T10:07:24.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f23a558b640 1 -- 172.21.15.133:0/1492874615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a00b01f0 con 0x7f23a014c060 2024-04-25T10:07:24.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f23a558b640 1 -- 172.21.15.133:0/1492874615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a0067c70 con 0x7f23a014c060 2024-04-25T10:07:24.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f23877fe640 1 -- 172.21.15.133:0/1492874615 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23980039a0 con 0x7f23a014c060 2024-04-25T10:07:24.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f23877fe640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f238c066f90 0x7f238c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f23877fe640 1 -- 172.21.15.133:0/1492874615 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f23981202c0 con 0x7f23a014c060 2024-04-25T10:07:24.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f239effd640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f238c066f90 0x7f238c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.177+0000 7f239effd640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f238c066f90 0x7f238c069450 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f239400b930 tx=0x7f2394008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.181+0000 7f23877fe640 1 -- 172.21.15.133:0/1492874615 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2398016620 con 0x7f23a014c060 2024-04-25T10:07:24.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.341+0000 7f23a558b640 1 -- 172.21.15.133:0/1492874615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23a00b5a80 con 0x7f23a014c060 2024-04-25T10:07:24.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23877fe640 1 -- 172.21.15.133:0/1492874615 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+13580 (secure 0 0 0) 0x7f23980f1420 con 0x7f23a014c060 2024-04-25T10:07:24.350 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:24.350 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":73,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:07:23.008020+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/2310040121":"2024-04-26T10:07:19.875430+0000","172.21.15.133:6836/2310040121":"2024-04-26T10:07:19.875430+0000","172.21.15.162:6835/272966073":"2024-04-26T10:06:56.893338+0000","172.21.15.162:6833/272966073":"2024-04-26T10:06:56.893338+0000","172.21.15.133:6837/236214118":"2024-04-26T10:06:55.901820+0000","172.21.15.133:6836/236214118":"2024-04-26T10:06:55.901820+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:24.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 -- 172.21.15.133:0/1492874615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f238c066f90 msgr2=0x7f238c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f238c066f90 0x7f238c069450 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f239400b930 tx=0x7f2394008040 comp rx=0 tx=0).stop 2024-04-25T10:07:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 -- 172.21.15.133:0/1492874615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23a014c060 msgr2=0x7f23a00cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23a014c060 0x7f23a00cfa60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f23980af040 tx=0x7f23980197a0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 -- 172.21.15.133:0/1492874615 shutdown_connections 2024-04-25T10:07:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f238c066f90 0x7f238c069450 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23a014c060 0x7f23a00cfa60 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23a0159280 0x7f23a00af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 --2- 172.21.15.133:0/1492874615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23a006b3a0 0x7f23a0157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 -- 172.21.15.133:0/1492874615 >> 172.21.15.133:0/1492874615 conn(0x7f23a00b2370 msgr2=0x7f23a0151bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 -- 172.21.15.133:0/1492874615 shutdown_connections 2024-04-25T10:07:24.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.345+0000 7f23a558b640 1 -- 172.21.15.133:0/1492874615 wait complete. 2024-04-25T10:07:24.369 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:24.369 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:24.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4ac7fd640 1 Processor -- start 2024-04-25T10:07:24.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4ac7fd640 1 -- start start 2024-04-25T10:07:24.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4ac7fd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4a40cd820 0x7fb4a40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4ac7fd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a40ce2e0 con 0x7fb4a415ced0 2024-04-25T10:07:24.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4ac7fd640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a406d4b0 con 0x7fb4a4145170 2024-04-25T10:07:24.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4ac7fd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a406db00 con 0x7fb4a40cd820 2024-04-25T10:07:24.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4aad9a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb4a415ced0 0x7fb4a40b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45612/0 (socket says 172.21.15.133:45612) 2024-04-25T10:07:24.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4aad9a640 1 -- 172.21.15.133:0/4276674015 learned_addr learned my addr 172.21.15.133:0/4276674015 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:24.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9d98640 1 --2- 172.21.15.133:0/4276674015 >> v2:172.21.15.162:3300/0 conn(0x7fb4a40cd820 0x7fb4a40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477700162 0 0) 0x7fb4a40ce2e0 con 0x7fb4a415ced0 2024-04-25T10:07:24.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb490003530 con 0x7fb4a415ced0 2024-04-25T10:07:24.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2926248451 0 0) 0x7fb490003530 con 0x7fb4a415ced0 2024-04-25T10:07:24.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 >> v2:172.21.15.162:3300/0 conn(0x7fb4a40cd820 msgr2=0x7fb4a40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 --2- 172.21.15.133:0/4276674015 >> v2:172.21.15.162:3300/0 conn(0x7fb4a40cd820 0x7fb4a40cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 >> v1:172.21.15.162:6789/0 conn(0x7fb4a4145170 legacy=0x7fb4a406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a406d790 con 0x7fb4a415ced0 2024-04-25T10:07:24.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb49c001a00 con 0x7fb4a415ced0 2024-04-25T10:07:24.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 >> v1:172.21.15.133:6789/0 conn(0x7fb4a415ced0 legacy=0x7fb4a40b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4900039c0 0x7fb490003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb490004810 0x7fb490004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.489+0000 7fb4a9597640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4900051d0 0x7fb49000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a406db00 con 0x7fb490004810 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49000c070 con 0x7fb4900039c0 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49000c270 con 0x7fb4900051d0 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9d98640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb490004810 0x7fb490004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4aa599640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4900039c0 0x7fb490003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9d98640 1 -- 172.21.15.133:0/4276674015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4900051d0 msgr2=0x7fb49000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4aad9a640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4900051d0 0x7fb49000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9d98640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4900051d0 0x7fb49000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9d98640 1 -- 172.21.15.133:0/4276674015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4900039c0 msgr2=0x7fb490003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9d98640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4900039c0 0x7fb490003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9d98640 1 -- 172.21.15.133:0/4276674015 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb490003530 con 0x7fb490004810 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4aa599640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4900039c0 0x7fb490003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:24.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4aad9a640 1 -- 172.21.15.133:0/4276674015 reap_dead start 2024-04-25T10:07:24.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9d98640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb490004810 0x7fb490004c90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009ac0 tx=0x7fb4a001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea04b7d58f732665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00b2070 con 0x7fb490004810 2024-04-25T10:07:24.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a00a5640 con 0x7fb490004810 2024-04-25T10:07:24.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4a9597640 1 -- 172.21.15.133:0/4276674015 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0009690 con 0x7fb490004810 2024-04-25T10:07:24.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/4276674015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb490004810 msgr2=0x7fb490004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4ac7fd640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb490004810 0x7fb490004c90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009ac0 tx=0x7fb4a001ca50 comp rx=0 tx=0).stop 2024-04-25T10:07:24.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/4276674015 shutdown_connections 2024-04-25T10:07:24.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4ac7fd640 1 --2- 172.21.15.133:0/4276674015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb490004810 0x7fb490004c90 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/4276674015 >> 172.21.15.133:0/4276674015 conn(0x7fb4a4066f70 msgr2=0x7fb4a406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/4276674015 shutdown_connections 2024-04-25T10:07:24.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.493+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/4276674015 wait complete. 2024-04-25T10:07:24.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4ac7fd640 1 Processor -- start 2024-04-25T10:07:24.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4ac7fd640 1 -- start start 2024-04-25T10:07:24.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4ac7fd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb490004810 0x7fb4a413c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4ac7fd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4a415ea40 0x7fb4a415f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4ac7fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a415fb60 0x7fb4a413acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4ac7fd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40cdca0 con 0x7fb4a415fb60 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4ac7fd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a4145550 con 0x7fb4a415ea40 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4ac7fd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40cde20 con 0x7fb490004810 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4aad9a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a415fb60 0x7fb4a413acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4aad9a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a415fb60 0x7fb4a413acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36776/0 (socket says 172.21.15.133:36776) 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4aad9a640 1 -- 172.21.15.133:0/1584765863 learned_addr learned my addr 172.21.15.133:0/1584765863 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4aa599640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb490004810 0x7fb4a413c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4a9d98640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4a415ea40 0x7fb4a415f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4aad9a640 1 -- 172.21.15.133:0/1584765863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb490004810 msgr2=0x7fb4a413c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4aad9a640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb490004810 0x7fb4a413c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4aad9a640 1 -- 172.21.15.133:0/1584765863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4a415ea40 msgr2=0x7fb4a415f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4aad9a640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4a415ea40 0x7fb4a415f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4aad9a640 1 -- 172.21.15.133:0/1584765863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a40ae450 con 0x7fb4a415fb60 2024-04-25T10:07:24.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4aad9a640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a415fb60 0x7fb4a413acd0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0012f0 tx=0x7fb49c010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb49affd640 1 -- 172.21.15.133:0/1584765863 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c01a070 con 0x7fb4a415fb60 2024-04-25T10:07:24.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb49affd640 1 -- 172.21.15.133:0/1584765863 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49c011040 con 0x7fb4a415fb60 2024-04-25T10:07:24.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/1584765863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a415f200 con 0x7fb4a415fb60 2024-04-25T10:07:24.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb49affd640 1 -- 172.21.15.133:0/1584765863 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c00ea50 con 0x7fb4a415fb60 2024-04-25T10:07:24.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.497+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/1584765863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a40ad970 con 0x7fb4a415fb60 2024-04-25T10:07:24.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.501+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/1584765863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a4068e80 con 0x7fb4a415fb60 2024-04-25T10:07:24.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.501+0000 7fb49affd640 1 -- 172.21.15.133:0/1584765863 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb49c01d070 con 0x7fb4a415fb60 2024-04-25T10:07:24.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.501+0000 7fb49affd640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb488066f60 0x7fb488069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.501+0000 7fb49affd640 1 -- 172.21.15.133:0/1584765863 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fb49c0873f0 con 0x7fb4a415fb60 2024-04-25T10:07:24.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.501+0000 7fb4aa599640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb488066f60 0x7fb488069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.501+0000 7fb4aa599640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb488066f60 0x7fb488069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb494000c10 tx=0x7fb494009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.505+0000 7fb49affd640 1 -- 172.21.15.133:0/1584765863 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb49c058550 con 0x7fb4a415fb60 2024-04-25T10:07:24.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.665+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/1584765863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4a4155ce0 con 0x7fb4a415fb60 2024-04-25T10:07:24.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb49affd640 1 -- 172.21.15.133:0/1584765863 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb49c057e50 con 0x7fb4a415fb60 2024-04-25T10:07:24.672 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:07:24.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/1584765863 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb488066f60 msgr2=0x7fb488069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb488066f60 0x7fb488069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb494000c10 tx=0x7fb494009040 comp rx=0 tx=0).stop 2024-04-25T10:07:24.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/1584765863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a415fb60 msgr2=0x7fb4a413acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a415fb60 0x7fb4a413acd0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0012f0 tx=0x7fb49c010b40 comp rx=0 tx=0).stop 2024-04-25T10:07:24.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/1584765863 shutdown_connections 2024-04-25T10:07:24.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb488066f60 0x7fb488069420 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a415fb60 0x7fb4a413acd0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4a415ea40 0x7fb4a415f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 --2- 172.21.15.133:0/1584765863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb490004810 0x7fb4a413c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/1584765863 >> 172.21.15.133:0/1584765863 conn(0x7fb4a4066f70 msgr2=0x7fb4a41408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/1584765863 shutdown_connections 2024-04-25T10:07:24.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.669+0000 7fb4ac7fd640 1 -- 172.21.15.133:0/1584765863 wait complete. 2024-04-25T10:07:24.691 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T10:07:24.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.809+0000 7fba65550640 1 Processor -- start 2024-04-25T10:07:24.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.809+0000 7fba65550640 1 -- start start 2024-04-25T10:07:24.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba65550640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba6015c9a0 0x7fba600c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba65550640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba600c61d0 con 0x7fba600ce330 2024-04-25T10:07:24.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba65550640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba600c64b0 con 0x7fba600b1f70 2024-04-25T10:07:24.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba65550640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6006d4f0 con 0x7fba6015c9a0 2024-04-25T10:07:24.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba5effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba6015c9a0 0x7fba600c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba5f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fba600ce330 0x7fba6015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45624/0 (socket says 172.21.15.133:45624) 2024-04-25T10:07:24.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba5f7fe640 1 -- 172.21.15.133:0/239475140 learned_addr learned my addr 172.21.15.133:0/239475140 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:24.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba5effd640 1 -- 172.21.15.133:0/239475140 >> v1:172.21.15.162:6789/0 conn(0x7fba600b1f70 legacy=0x7fba6006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba5effd640 1 -- 172.21.15.133:0/239475140 >> v1:172.21.15.133:6789/0 conn(0x7fba600ce330 legacy=0x7fba6015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba5effd640 1 -- 172.21.15.133:0/239475140 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6006d7d0 con 0x7fba6015c9a0 2024-04-25T10:07:24.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba5e7fc640 1 -- 172.21.15.133:0/239475140 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539593640 0 0) 0x7fba600c61d0 con 0x7fba600ce330 2024-04-25T10:07:24.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba5effd640 1 --2- 172.21.15.133:0/239475140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba6015c9a0 0x7fba600c5980 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fba500097c0 tx=0x7fba5001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b361972ece8d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba5e7fc640 1 -- 172.21.15.133:0/239475140 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500b0070 con 0x7fba6015c9a0 2024-04-25T10:07:24.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba5e7fc640 1 -- 172.21.15.133:0/239475140 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba500ab600 con 0x7fba6015c9a0 2024-04-25T10:07:24.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.813+0000 7fba5e7fc640 1 -- 172.21.15.133:0/239475140 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500ab9d0 con 0x7fba6015c9a0 2024-04-25T10:07:24.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 -- 172.21.15.133:0/239475140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba6015c9a0 msgr2=0x7fba600c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 --2- 172.21.15.133:0/239475140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba6015c9a0 0x7fba600c5980 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fba500097c0 tx=0x7fba5001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 -- 172.21.15.133:0/239475140 shutdown_connections 2024-04-25T10:07:24.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 --2- 172.21.15.133:0/239475140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba6015c9a0 0x7fba600c5980 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 -- 172.21.15.133:0/239475140 >> 172.21.15.133:0/239475140 conn(0x7fba60067000 msgr2=0x7fba6006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 -- 172.21.15.133:0/239475140 shutdown_connections 2024-04-25T10:07:24.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 -- 172.21.15.133:0/239475140 wait complete. 2024-04-25T10:07:24.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 Processor -- start 2024-04-25T10:07:24.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 -- start start 2024-04-25T10:07:24.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba600b1f70 0x7fba6014d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba600ce330 0x7fba60150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba6015c9a0 0x7fba601f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6015cf00 con 0x7fba600b1f70 2024-04-25T10:07:24.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6015cd80 con 0x7fba600ce330 2024-04-25T10:07:24.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba65550640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba60169de0 con 0x7fba6015c9a0 2024-04-25T10:07:24.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba5effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba600b1f70 0x7fba6014d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba5effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba600b1f70 0x7fba6014d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36784/0 (socket says 172.21.15.133:36784) 2024-04-25T10:07:24.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba5effd640 1 -- 172.21.15.133:0/2086897391 learned_addr learned my addr 172.21.15.133:0/2086897391 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:24.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba567fc640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba600ce330 0x7fba60150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba5f7fe640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba6015c9a0 0x7fba601f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba5effd640 1 -- 172.21.15.133:0/2086897391 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba6015c9a0 msgr2=0x7fba601f82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba5effd640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba6015c9a0 0x7fba601f82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba5effd640 1 -- 172.21.15.133:0/2086897391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba600ce330 msgr2=0x7fba60150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba5effd640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba600ce330 0x7fba60150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba5effd640 1 -- 172.21.15.133:0/2086897391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba600c5050 con 0x7fba600b1f70 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba5f7fe640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba6015c9a0 0x7fba601f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba567fc640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba600ce330 0x7fba60150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba5effd640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba600b1f70 0x7fba6014d430 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fba500ae040 tx=0x7fba5001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba57fff640 1 -- 172.21.15.133:0/2086897391 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500b0050 con 0x7fba600b1f70 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.817+0000 7fba57fff640 1 -- 172.21.15.133:0/2086897391 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba500bc070 con 0x7fba600b1f70 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.821+0000 7fba65550640 1 -- 172.21.15.133:0/2086897391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6014f060 con 0x7fba600b1f70 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.821+0000 7fba57fff640 1 -- 172.21.15.133:0/2086897391 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500b5a50 con 0x7fba600b1f70 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.821+0000 7fba65550640 1 -- 172.21.15.133:0/2086897391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6014ed10 con 0x7fba600b1f70 2024-04-25T10:07:24.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.821+0000 7fba65550640 1 -- 172.21.15.133:0/2086897391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba60068ef0 con 0x7fba600b1f70 2024-04-25T10:07:24.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.821+0000 7fba57fff640 1 -- 172.21.15.133:0/2086897391 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba500ab4a0 con 0x7fba600b1f70 2024-04-25T10:07:24.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.821+0000 7fba57fff640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba40066f60 0x7fba40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.821+0000 7fba57fff640 1 -- 172.21.15.133:0/2086897391 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fba5011f910 con 0x7fba600b1f70 2024-04-25T10:07:24.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.821+0000 7fba567fc640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba40066f60 0x7fba40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.821+0000 7fba567fc640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba40066f60 0x7fba40069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fba601f8210 tx=0x7fba48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:24.825+0000 7fba57fff640 1 -- 172.21.15.133:0/2086897391 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba50016090 con 0x7fba600b1f70 2024-04-25T10:07:25.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:25.041+0000 7fba65550640 1 -- 172.21.15.133:0/2086897391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7fba60152720 con 0x7fba600b1f70 2024-04-25T10:07:26.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.025+0000 7fba57fff640 1 -- 172.21.15.133:0/2086897391 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v273) v1 ==== 161+0+0 (secure 0 0 0) 0x7fba500f09d0 con 0x7fba600b1f70 2024-04-25T10:07:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 -- 172.21.15.133:0/2086897391 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba40066f60 msgr2=0x7fba40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba40066f60 0x7fba40069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fba601f8210 tx=0x7fba48009040 comp rx=0 tx=0).stop 2024-04-25T10:07:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 -- 172.21.15.133:0/2086897391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba600b1f70 msgr2=0x7fba6014d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba600b1f70 0x7fba6014d430 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fba500ae040 tx=0x7fba5001ace0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 -- 172.21.15.133:0/2086897391 shutdown_connections 2024-04-25T10:07:26.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba6015c9a0 0x7fba601f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba600ce330 0x7fba60150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba40066f60 0x7fba40069420 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 --2- 172.21.15.133:0/2086897391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba600b1f70 0x7fba6014d430 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 -- 172.21.15.133:0/2086897391 >> 172.21.15.133:0/2086897391 conn(0x7fba60067000 msgr2=0x7fba6015de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:26.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 -- 172.21.15.133:0/2086897391 shutdown_connections 2024-04-25T10:07:26.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.029+0000 7fba65550640 1 -- 172.21.15.133:0/2086897391 wait complete. 2024-04-25T10:07:26.056 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:07:26.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41b6d13640 1 Processor -- start 2024-04-25T10:07:26.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41b6d13640 1 -- start start 2024-04-25T10:07:26.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41b6d13640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f41b015f020 0x7f41b00cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41b6d13640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b00cefa0 con 0x7f41b00d21c0 2024-04-25T10:07:26.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41b6d13640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b00ccdf0 con 0x7f41b006f8f0 2024-04-25T10:07:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41b6d13640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b00cd690 con 0x7f41b015f020 2024-04-25T10:07:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41affff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f41b00d21c0 0x7f41b00cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34044/0 (socket says 172.21.15.133:34044) 2024-04-25T10:07:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41affff640 1 -- 172.21.15.133:0/2583159616 learned_addr learned my addr 172.21.15.133:0/2583159616 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41b4aaf640 1 --2- 172.21.15.133:0/2583159616 >> v2:172.21.15.133:3300/0 conn(0x7f41b015f020 0x7f41b00cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41af7fe640 1 -- 172.21.15.133:0/2583159616 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078806888 0 0) 0x7f41a0001120 con 0x7f41b00d21c0 2024-04-25T10:07:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41af7fe640 1 -- 172.21.15.133:0/2583159616 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4198003530 con 0x7f41b00d21c0 2024-04-25T10:07:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41b4aaf640 1 -- 172.21.15.133:0/2583159616 >> v1:172.21.15.162:6789/0 conn(0x7f41b006f8f0 legacy=0x7f41b006fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41af7fe640 1 -- 172.21.15.133:0/2583159616 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3239101322 0 0) 0x7f4198003530 con 0x7f41b00d21c0 2024-04-25T10:07:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41b4aaf640 1 -- 172.21.15.133:0/2583159616 >> v1:172.21.15.133:6789/0 conn(0x7f41b00d21c0 legacy=0x7f41b00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.181+0000 7f41b4aaf640 1 -- 172.21.15.133:0/2583159616 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b00cd0d0 con 0x7f41b015f020 2024-04-25T10:07:26.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b4aaf640 1 --2- 172.21.15.133:0/2583159616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41b015f020 0x7f41b00cf270 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f41a40097c0 tx=0x7f41a401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cd41d72409d649 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:26.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41af7fe640 1 -- 172.21.15.133:0/2583159616 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a40b0070 con 0x7f41b015f020 2024-04-25T10:07:26.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41af7fe640 1 -- 172.21.15.133:0/2583159616 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a40ab400 con 0x7f41b015f020 2024-04-25T10:07:26.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41af7fe640 1 -- 172.21.15.133:0/2583159616 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a0001850 con 0x7f41b015f020 2024-04-25T10:07:26.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 -- 172.21.15.133:0/2583159616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41b015f020 msgr2=0x7f41b00cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 --2- 172.21.15.133:0/2583159616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41b015f020 0x7f41b00cf270 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f41a40097c0 tx=0x7f41a401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 -- 172.21.15.133:0/2583159616 shutdown_connections 2024-04-25T10:07:26.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 --2- 172.21.15.133:0/2583159616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41b015f020 0x7f41b00cf270 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 -- 172.21.15.133:0/2583159616 >> 172.21.15.133:0/2583159616 conn(0x7f41b00d16a0 msgr2=0x7f41b00d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:26.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 -- 172.21.15.133:0/2583159616 shutdown_connections 2024-04-25T10:07:26.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 -- 172.21.15.133:0/2583159616 wait complete. 2024-04-25T10:07:26.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 Processor -- start 2024-04-25T10:07:26.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 -- start start 2024-04-25T10:07:26.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41b006f8f0 0x7f41b0155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41b00d21c0 0x7f41b014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b015f020 0x7f41b014c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b0169570 con 0x7f41b006f8f0 2024-04-25T10:07:26.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00d00d0 con 0x7f41b015f020 2024-04-25T10:07:26.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b6d13640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b01696f0 con 0x7f41b00d21c0 2024-04-25T10:07:26.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b4aaf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41b006f8f0 0x7f41b0155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b4aaf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41b006f8f0 0x7f41b0155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40512/0 (socket says 172.21.15.133:40512) 2024-04-25T10:07:26.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.185+0000 7f41b4aaf640 1 -- 172.21.15.133:0/3521102359 learned_addr learned my addr 172.21.15.133:0/3521102359 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:26.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41b52b0640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b015f020 0x7f41b014c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41affff640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41b00d21c0 0x7f41b014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41b4aaf640 1 -- 172.21.15.133:0/3521102359 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41b00d21c0 msgr2=0x7f41b014efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41b4aaf640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41b00d21c0 0x7f41b014efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41b4aaf640 1 -- 172.21.15.133:0/3521102359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b015f020 msgr2=0x7f41b014c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41b4aaf640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b015f020 0x7f41b014c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41b4aaf640 1 -- 172.21.15.133:0/3521102359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b014f970 con 0x7f41b006f8f0 2024-04-25T10:07:26.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41b52b0640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b015f020 0x7f41b014c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:26.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41b4aaf640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41b006f8f0 0x7f41b0155410 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f41a40097c0 tx=0x7f41a401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:26.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41ad7fa640 1 -- 172.21.15.133:0/3521102359 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a40bb020 con 0x7f41b006f8f0 2024-04-25T10:07:26.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41ad7fa640 1 -- 172.21.15.133:0/3521102359 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a40b1040 con 0x7f41b006f8f0 2024-04-25T10:07:26.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41b6d13640 1 -- 172.21.15.133:0/3521102359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b014fc50 con 0x7f41b006f8f0 2024-04-25T10:07:26.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41ad7fa640 1 -- 172.21.15.133:0/3521102359 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a40b0050 con 0x7f41b006f8f0 2024-04-25T10:07:26.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41b6d13640 1 -- 172.21.15.133:0/3521102359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b0150210 con 0x7f41b006f8f0 2024-04-25T10:07:26.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41b6d13640 1 -- 172.21.15.133:0/3521102359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b0067610 con 0x7f41b006f8f0 2024-04-25T10:07:26.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41ad7fa640 1 -- 172.21.15.133:0/3521102359 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f41a40be070 con 0x7f41b006f8f0 2024-04-25T10:07:26.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41ad7fa640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f419c066f90 0x7f419c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41ad7fa640 1 -- 172.21.15.133:0/3521102359 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f41a411e920 con 0x7f41b006f8f0 2024-04-25T10:07:26.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.189+0000 7f41affff640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f419c066f90 0x7f419c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.193+0000 7f41affff640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f419c066f90 0x7f419c069450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f41a0001ec0 tx=0x7f41a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:26.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.193+0000 7f41ad7fa640 1 -- 172.21.15.133:0/3521102359 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f41a40c4020 con 0x7f41b006f8f0 2024-04-25T10:07:26.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.349+0000 7f41b6d13640 1 -- 172.21.15.133:0/3521102359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f41b01571c0 con 0x7f41b006f8f0 2024-04-25T10:07:26.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.353+0000 7f41ad7fa640 1 -- 172.21.15.133:0/3521102359 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f41a4015fb0 con 0x7f41b006f8f0 2024-04-25T10:07:26.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 -- 172.21.15.133:0/3521102359 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f419c066f90 msgr2=0x7f419c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f419c066f90 0x7f419c069450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f41a0001ec0 tx=0x7f41a0009040 comp rx=0 tx=0).stop 2024-04-25T10:07:26.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 -- 172.21.15.133:0/3521102359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41b006f8f0 msgr2=0x7f41b0155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41b006f8f0 0x7f41b0155410 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f41a40097c0 tx=0x7f41a401ac00 comp rx=0 tx=0).stop 2024-04-25T10:07:26.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 -- 172.21.15.133:0/3521102359 shutdown_connections 2024-04-25T10:07:26.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41b015f020 0x7f41b014c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f41b00d21c0 0x7f41b014efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f419c066f90 0x7f419c069450 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 --2- 172.21.15.133:0/3521102359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41b006f8f0 0x7f41b0155410 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 -- 172.21.15.133:0/3521102359 >> 172.21.15.133:0/3521102359 conn(0x7f41b00d16a0 msgr2=0x7f41b006fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:26.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 -- 172.21.15.133:0/3521102359 shutdown_connections 2024-04-25T10:07:26.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.357+0000 7f41b6d13640 1 -- 172.21.15.133:0/3521102359 wait complete. 2024-04-25T10:07:26.378 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:07:26.378 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:26.425 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:26.490 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:07:26.491 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:07:26.491 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:26.496 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:26.496 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:07:26.496 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:26.502 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:26.502 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:07:26.502 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:26.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:26.546 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:07:26.546 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:26.552 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:26.552 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:07:26.552 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:26.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.705+0000 7f603dd87640 1 Processor -- start 2024-04-25T10:07:26.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.705+0000 7f603dd87640 1 -- start start 2024-04-25T10:07:26.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f603dd87640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60380cee80 0x7f6038168850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f603dd87640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60380ca9b0 con 0x7f603815d9b0 2024-04-25T10:07:26.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f603dd87640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60380cac90 con 0x7f603815dd90 2024-04-25T10:07:26.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f603dd87640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60380cbe30 con 0x7f60380cee80 2024-04-25T10:07:26.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60377fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f603815d9b0 0x7f60380cb5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34060/0 (socket says 172.21.15.133:34060) 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60377fe640 1 -- 172.21.15.133:0/100308444 learned_addr learned my addr 172.21.15.133:0/100308444 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767143929 0 0) 0x7f60380ca9b0 con 0x7f603815d9b0 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6024003530 con 0x7f603815d9b0 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f6037fff640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60380cee80 0x7f6038168850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1664757608 0 0) 0x7f6024003530 con 0x7f603815d9b0 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60380cee80 msgr2=0x7f6038168850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60380cee80 0x7f6038168850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 >> v1:172.21.15.162:6789/0 conn(0x7f603815dd90 legacy=0x7f60380ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60380caf70 con 0x7f603815d9b0 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6028001a00 con 0x7f603815d9b0 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 >> v1:172.21.15.133:6789/0 conn(0x7f603815d9b0 legacy=0x7f60380cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60240039f0 0x7f6024003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60380cee80 0x7f6024004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6024004d50 0x7f60240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60380cbe30 con 0x7f60240039f0 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602400bbe0 con 0x7f6024004d50 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602400bde0 con 0x7f60380cee80 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60377fe640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60240039f0 0x7f6024003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f6036ffd640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60380cee80 0x7f6024004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60377fe640 1 -- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60380cee80 msgr2=0x7f6024004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60377fe640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60380cee80 0x7f6024004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f6037fff640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6024004d50 0x7f60240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60377fe640 1 -- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6024004d50 msgr2=0x7f60240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60377fe640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6024004d50 0x7f60240051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60377fe640 1 -- 172.21.15.133:0/100308444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6024003530 con 0x7f60240039f0 2024-04-25T10:07:26.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f6036ffd640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60380cee80 0x7f6024004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:26.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60377fe640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60240039f0 0x7f6024003e10 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f602800a840 tx=0x7f602801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede026c276a481dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:26.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800a460 con 0x7f60240039f0 2024-04-25T10:07:26.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60280ad430 con 0x7f60240039f0 2024-04-25T10:07:26.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.709+0000 7f60367fc640 1 -- 172.21.15.133:0/100308444 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60280ad840 con 0x7f60240039f0 2024-04-25T10:07:26.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 -- 172.21.15.133:0/100308444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60240039f0 msgr2=0x7f6024003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 --2- 172.21.15.133:0/100308444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60240039f0 0x7f6024003e10 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f602800a840 tx=0x7f602801d360 comp rx=0 tx=0).stop 2024-04-25T10:07:26.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f6037fff640 1 -- 172.21.15.133:0/100308444 reap_dead start 2024-04-25T10:07:26.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 -- 172.21.15.133:0/100308444 shutdown_connections 2024-04-25T10:07:26.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 -- 172.21.15.133:0/100308444 >> 172.21.15.133:0/100308444 conn(0x7f6038067000 msgr2=0x7f603806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:26.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 -- 172.21.15.133:0/100308444 shutdown_connections 2024-04-25T10:07:26.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 -- 172.21.15.133:0/100308444 wait complete. 2024-04-25T10:07:26.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 Processor -- start 2024-04-25T10:07:26.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 -- start start 2024-04-25T10:07:26.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60240039f0 0x7f603814a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6038153630 0x7f603814a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6038154480 0x7f6038154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60381693f0 con 0x7f6038153630 2024-04-25T10:07:26.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6038169270 con 0x7f6038154480 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6038169570 con 0x7f60240039f0 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f6036ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6038153630 0x7f603814a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f6036ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6038153630 0x7f603814a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40528/0 (socket says 172.21.15.133:40528) 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f6036ffd640 1 -- 172.21.15.133:0/1318276893 learned_addr learned my addr 172.21.15.133:0/1318276893 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f6036ffd640 1 -- 172.21.15.133:0/1318276893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60240039f0 msgr2=0x7f603814a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f6037fff640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6038154480 0x7f6038154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f60377fe640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60240039f0 0x7f603814a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f6036ffd640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60240039f0 0x7f603814a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f6036ffd640 1 -- 172.21.15.133:0/1318276893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6038154480 msgr2=0x7f6038154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f6036ffd640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6038154480 0x7f6038154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f6036ffd640 1 -- 172.21.15.133:0/1318276893 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6038160510 con 0x7f6038153630 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f6036ffd640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6038153630 0x7f603814a1c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6020000c10 tx=0x7f602000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f602ffff640 1 -- 172.21.15.133:0/1318276893 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020019070 con 0x7f6038153630 2024-04-25T10:07:26.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f602ffff640 1 -- 172.21.15.133:0/1318276893 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6020005270 con 0x7f6038153630 2024-04-25T10:07:26.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 -- 172.21.15.133:0/1318276893 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6038153bf0 con 0x7f6038153630 2024-04-25T10:07:26.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f602ffff640 1 -- 172.21.15.133:0/1318276893 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020007030 con 0x7f6038153630 2024-04-25T10:07:26.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.713+0000 7f603dd87640 1 -- 172.21.15.133:0/1318276893 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603815ff50 con 0x7f6038153630 2024-04-25T10:07:26.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.717+0000 7f603dd87640 1 -- 172.21.15.133:0/1318276893 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6038068ef0 con 0x7f6038153630 2024-04-25T10:07:26.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.717+0000 7f602ffff640 1 -- 172.21.15.133:0/1318276893 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f602001c070 con 0x7f6038153630 2024-04-25T10:07:26.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.717+0000 7f602ffff640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6018066f90 0x7f6018069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.717+0000 7f602ffff640 1 -- 172.21.15.133:0/1318276893 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f60200846d0 con 0x7f6038153630 2024-04-25T10:07:26.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.717+0000 7f60377fe640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6018066f90 0x7f6018069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.717+0000 7f60377fe640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6018066f90 0x7f6018069450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f60280ad000 tx=0x7f60380cb250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:26.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.721+0000 7f602ffff640 1 -- 172.21.15.133:0/1318276893 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6020055830 con 0x7f6038153630 2024-04-25T10:07:26.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.897+0000 7f603dd87640 1 -- 172.21.15.133:0/1318276893 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f603815cd90 con 0x7f6038153630 2024-04-25T10:07:26.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.897+0000 7f602ffff640 1 -- 172.21.15.133:0/1318276893 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f6020055130 con 0x7f6038153630 2024-04-25T10:07:26.903 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:26.903 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1579659657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1579659657},{"type":"v1","addr":"172.21.15.162:6835","nonce":1579659657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/232744252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":232744252},{"type":"v1","addr":"172.21.15.162:6833","nonce":232744252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7149,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/912719603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":912719603},{"type":"v1","addr":"172.21.15.133:6835","nonce":912719603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2382126739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2382126739},{"type":"v1","addr":"172.21.15.133:6837","nonce":2382126739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[]} 2024-04-25T10:07:26.904 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 43 2024-04-25T10:07:26.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 -- 172.21.15.133:0/1318276893 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6018066f90 msgr2=0x7f6018069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6018066f90 0x7f6018069450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f60280ad000 tx=0x7f60380cb250 comp rx=0 tx=0).stop 2024-04-25T10:07:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 -- 172.21.15.133:0/1318276893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6038153630 msgr2=0x7f603814a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6038153630 0x7f603814a1c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6020000c10 tx=0x7f602000dc00 comp rx=0 tx=0).stop 2024-04-25T10:07:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 -- 172.21.15.133:0/1318276893 shutdown_connections 2024-04-25T10:07:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6038154480 0x7f6038154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6018066f90 0x7f6018069450 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6038153630 0x7f603814a1c0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 --2- 172.21.15.133:0/1318276893 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60240039f0 0x7f603814a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 -- 172.21.15.133:0/1318276893 >> 172.21.15.133:0/1318276893 conn(0x7f6038067000 msgr2=0x7f60380b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 -- 172.21.15.133:0/1318276893 shutdown_connections 2024-04-25T10:07:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:26.901+0000 7f603dd87640 1 -- 172.21.15.133:0/1318276893 wait complete. 2024-04-25T10:07:26.922 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:07:26.922 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:07:26.923 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:07:26.923 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:26.923 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:07:26.923 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:07:26.923 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:07:26.923 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:26.923 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:07:26.924 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:07:26.924 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:07:26.924 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:26.924 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:07:26.924 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:07:26.924 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:07:26.924 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:26.933 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:07:26.933 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:07:26.935 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:07:26.935 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:07:26.937 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:07:26.937 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:07:26.938 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:07:26.939 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:07:26.976 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:07:26.977 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:26.978 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:07:26.978 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:26.982 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:07:26.982 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:26.983 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:07:26.983 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:26.984 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:07:27.034 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:07:27.035 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:07:27.056 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:07:27.057 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:07:27.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.089+0000 7f7f22ceb640 1 Processor -- start 2024-04-25T10:07:27.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.089+0000 7f7f22ceb640 1 -- start start 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f22ceb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7f1c0d2030 0x7f7f1c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f22ceb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1c15ebd0 con 0x7f7f1c0ce600 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f22ceb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1c15eeb0 con 0x7f7f1c15d9a0 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f22ceb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1c0d0330 con 0x7f7f1c0d2030 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f21288640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7f1c0d2030 0x7f7f1c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f1bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7f1c0ce600 0x7f7f1c0d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34078/0 (socket says 172.21.15.133:34078) 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f21288640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7f1c0d2030 0x7f7f1c0cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40574/0 (socket says 172.21.15.133:40574) 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f1bfff640 1 -- 172.21.15.133:0/2430189411 learned_addr learned my addr 172.21.15.133:0/2430189411 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f21288640 1 -- 172.21.15.133:0/2430189411 >> v1:172.21.15.162:6789/0 conn(0x7f7f1c15d9a0 legacy=0x7f7f1c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f21288640 1 -- 172.21.15.133:0/2430189411 >> v1:172.21.15.133:6789/0 conn(0x7f7f1c0ce600 legacy=0x7f7f1c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f21288640 1 -- 172.21.15.133:0/2430189411 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c0d0610 con 0x7f7f1c0d2030 2024-04-25T10:07:27.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f1b7fe640 1 -- 172.21.15.133:0/2430189411 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87566468 0 0) 0x7f7f1c15ebd0 con 0x7f7f1c0ce600 2024-04-25T10:07:27.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f21288640 1 --2- 172.21.15.133:0/2430189411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f1c0d2030 0x7f7f1c0cea90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009d90 tx=0x7f7f1401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30579a9ecdaa27f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f1b7fe640 1 -- 172.21.15.133:0/2430189411 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f140b2070 con 0x7f7f1c0d2030 2024-04-25T10:07:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f1b7fe640 1 -- 172.21.15.133:0/2430189411 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f140a5870 con 0x7f7f1c0d2030 2024-04-25T10:07:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f1b7fe640 1 -- 172.21.15.133:0/2430189411 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f140a5c40 con 0x7f7f1c0d2030 2024-04-25T10:07:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f22ceb640 1 -- 172.21.15.133:0/2430189411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f1c0d2030 msgr2=0x7f7f1c0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f22ceb640 1 --2- 172.21.15.133:0/2430189411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f1c0d2030 0x7f7f1c0cea90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009d90 tx=0x7f7f1401c8a0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f22ceb640 1 -- 172.21.15.133:0/2430189411 shutdown_connections 2024-04-25T10:07:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f22ceb640 1 --2- 172.21.15.133:0/2430189411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f1c0d2030 0x7f7f1c0cea90 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f22ceb640 1 -- 172.21.15.133:0/2430189411 >> 172.21.15.133:0/2430189411 conn(0x7f7f1c06bf80 msgr2=0x7f7f1c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f22ceb640 1 -- 172.21.15.133:0/2430189411 shutdown_connections 2024-04-25T10:07:27.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.093+0000 7f7f22ceb640 1 -- 172.21.15.133:0/2430189411 wait complete. 2024-04-25T10:07:27.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f22ceb640 1 Processor -- start 2024-04-25T10:07:27.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f22ceb640 1 -- start start 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f22ceb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f1c0ce600 0x7f7f1c15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f22ceb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f1c0d2030 0x7f7f1c153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f22ceb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f1c15d9a0 0x7f7f1c158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f22ceb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c15e710 con 0x7f7f1c0ce600 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f22ceb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c15e590 con 0x7f7f1c0d2030 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f22ceb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c15e890 con 0x7f7f1c15d9a0 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f20a87640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f1c0ce600 0x7f7f1c15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f20a87640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f1c0ce600 0x7f7f1c15b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40590/0 (socket says 172.21.15.133:40590) 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f20a87640 1 -- 172.21.15.133:0/3348151022 learned_addr learned my addr 172.21.15.133:0/3348151022 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f21288640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f1c15d9a0 0x7f7f1c158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f1bfff640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f1c0d2030 0x7f7f1c153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f20a87640 1 -- 172.21.15.133:0/3348151022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f1c15d9a0 msgr2=0x7f7f1c158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f20a87640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f1c15d9a0 0x7f7f1c158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f20a87640 1 -- 172.21.15.133:0/3348151022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f1c0d2030 msgr2=0x7f7f1c153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f20a87640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f1c0d2030 0x7f7f1c153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f20a87640 1 -- 172.21.15.133:0/3348151022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c155dd0 con 0x7f7f1c0ce600 2024-04-25T10:07:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f21288640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f1c15d9a0 0x7f7f1c158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f20a87640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f1c0ce600 0x7f7f1c15b8e0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000b410 tx=0x7f7f1000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f197fa640 1 -- 172.21.15.133:0/3348151022 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10015030 con 0x7f7f1c0ce600 2024-04-25T10:07:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f197fa640 1 -- 172.21.15.133:0/3348151022 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f10004020 con 0x7f7f1c0ce600 2024-04-25T10:07:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f22ceb640 1 -- 172.21.15.133:0/3348151022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c1560b0 con 0x7f7f1c0ce600 2024-04-25T10:07:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f197fa640 1 -- 172.21.15.133:0/3348151022 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10004450 con 0x7f7f1c0ce600 2024-04-25T10:07:27.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f22ceb640 1 -- 172.21.15.133:0/3348151022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c153c90 con 0x7f7f1c0ce600 2024-04-25T10:07:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.097+0000 7f7f22ceb640 1 -- 172.21.15.133:0/3348151022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1c068e80 con 0x7f7f1c0ce600 2024-04-25T10:07:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.101+0000 7f7f197fa640 1 -- 172.21.15.133:0/3348151022 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f10018070 con 0x7f7f1c0ce600 2024-04-25T10:07:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.101+0000 7f7f197fa640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f00066f60 0x7f7f00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.101+0000 7f7f197fa640 1 -- 172.21.15.133:0/3348151022 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f7f10084a00 con 0x7f7f1c0ce600 2024-04-25T10:07:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.101+0000 7f7f1bfff640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f00066f60 0x7f7f00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.101+0000 7f7f1bfff640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f00066f60 0x7f7f00069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7f08000f70 tx=0x7f7f08007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.105+0000 7f7f197fa640 1 -- 172.21.15.133:0/3348151022 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f10055480 con 0x7f7f1c0ce600 2024-04-25T10:07:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:27.273+0000 7f7f22ceb640 1 -- 172.21.15.133:0/3348151022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7f1c168a20 con 0x7f7f1c0ce600 2024-04-25T10:07:28.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.045+0000 7f7f197fa640 1 -- 172.21.15.133:0/3348151022 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7f10059520 con 0x7f7f1c0ce600 2024-04-25T10:07:28.050 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:07:28.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 -- 172.21.15.133:0/3348151022 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f00066f60 msgr2=0x7f7f00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f00066f60 0x7f7f00069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7f08000f70 tx=0x7f7f08007df0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 -- 172.21.15.133:0/3348151022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f1c0ce600 msgr2=0x7f7f1c15b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f1c0ce600 0x7f7f1c15b8e0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000b410 tx=0x7f7f1000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 -- 172.21.15.133:0/3348151022 shutdown_connections 2024-04-25T10:07:28.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f1c15d9a0 0x7f7f1c158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f1c0d2030 0x7f7f1c153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f00066f60 0x7f7f00069420 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 --2- 172.21.15.133:0/3348151022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f1c0ce600 0x7f7f1c15b8e0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 -- 172.21.15.133:0/3348151022 >> 172.21.15.133:0/3348151022 conn(0x7f7f1c06bf80 msgr2=0x7f7f1c168f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 -- 172.21.15.133:0/3348151022 shutdown_connections 2024-04-25T10:07:28.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.049+0000 7f7f22ceb640 1 -- 172.21.15.133:0/3348151022 wait complete. 2024-04-25T10:07:28.075 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc24c6640 1 Processor -- start 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc24c6640 1 -- start start 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc24c6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fefbc158110 0x7fefbc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc24c6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fefbc06eb70 0x7fefbc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc24c6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefbc0cbe60 con 0x7fefbc1584f0 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc24c6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefbc0b19b0 con 0x7fefbc06eb70 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc24c6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefbc15b360 con 0x7fefbc158110 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc0a63640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fefbc06eb70 0x7fefbc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc0a63640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fefbc06eb70 0x7fefbc15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40594/0 (socket says 172.21.15.133:40594) 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc0a63640 1 -- 172.21.15.133:0/181131015 learned_addr learned my addr 172.21.15.133:0/181131015 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefbbfff640 1 --2- 172.21.15.133:0/181131015 >> v2:172.21.15.162:3300/0 conn(0x7fefbc158110 0x7fefbc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc0a63640 1 -- 172.21.15.133:0/181131015 >> v2:172.21.15.162:3300/0 conn(0x7fefbc158110 msgr2=0x7fefbc0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc0a63640 1 --2- 172.21.15.133:0/181131015 >> v2:172.21.15.162:3300/0 conn(0x7fefbc158110 0x7fefbc0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc0a63640 1 -- 172.21.15.133:0/181131015 >> v1:172.21.15.162:6789/0 conn(0x7fefbc1584f0 legacy=0x7fefbc06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc0a63640 1 -- 172.21.15.133:0/181131015 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc15b640 con 0x7fefbc06eb70 2024-04-25T10:07:28.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefc0a63640 1 --2- 172.21.15.133:0/181131015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefbc06eb70 0x7fefbc15ad10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0b4d40 tx=0x7fefb401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b8b4b253dca3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefbb7fe640 1 -- 172.21.15.133:0/181131015 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40a5860 con 0x7fefbc06eb70 2024-04-25T10:07:28.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefbb7fe640 1 -- 172.21.15.133:0/181131015 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb40a5a00 con 0x7fefbc06eb70 2024-04-25T10:07:28.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.197+0000 7fefbb7fe640 1 -- 172.21.15.133:0/181131015 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40a5dd0 con 0x7fefbc06eb70 2024-04-25T10:07:28.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 -- 172.21.15.133:0/181131015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefbc06eb70 msgr2=0x7fefbc15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 --2- 172.21.15.133:0/181131015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefbc06eb70 0x7fefbc15ad10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0b4d40 tx=0x7fefb401c980 comp rx=0 tx=0).stop 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 -- 172.21.15.133:0/181131015 shutdown_connections 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 --2- 172.21.15.133:0/181131015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefbc06eb70 0x7fefbc15ad10 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 --2- 172.21.15.133:0/181131015 >> v2:172.21.15.162:3300/0 conn(0x7fefbc158110 0x7fefbc0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 -- 172.21.15.133:0/181131015 >> 172.21.15.133:0/181131015 conn(0x7fefbc067000 msgr2=0x7fefbc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 -- 172.21.15.133:0/181131015 shutdown_connections 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 -- 172.21.15.133:0/181131015 wait complete. 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 Processor -- start 2024-04-25T10:07:28.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 -- start start 2024-04-25T10:07:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefbc06eb70 0x7fefbc1448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefbc158110 0x7fefbc148a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefbc1584f0 0x7fefbc06d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc0ccca0 con 0x7fefbc06eb70 2024-04-25T10:07:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc0ccb20 con 0x7fefbc158110 2024-04-25T10:07:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc24c6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc0cce20 con 0x7fefbc1584f0 2024-04-25T10:07:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefbc06eb70 0x7fefbc1448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefbc06eb70 0x7fefbc1448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40602/0 (socket says 172.21.15.133:40602) 2024-04-25T10:07:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefbbfff640 1 -- 172.21.15.133:0/842962873 learned_addr learned my addr 172.21.15.133:0/842962873 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc0a63640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefbc1584f0 0x7fefbc06d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefb3fff640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefbc158110 0x7fefbc148a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefbbfff640 1 -- 172.21.15.133:0/842962873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefbc1584f0 msgr2=0x7fefbc06d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefbbfff640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefbc1584f0 0x7fefbc06d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefbbfff640 1 -- 172.21.15.133:0/842962873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefbc158110 msgr2=0x7fefbc148a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefbbfff640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefbc158110 0x7fefbc148a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefbbfff640 1 -- 172.21.15.133:0/842962873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc14f6c0 con 0x7fefbc06eb70 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefc0a63640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefbc1584f0 0x7fefbc06d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefb3fff640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefbc158110 0x7fefbc148a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefbbfff640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefbc06eb70 0x7fefbc1448d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fefac00d800 tx=0x7fefac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefb97fa640 1 -- 172.21.15.133:0/842962873 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac00b6c0 con 0x7fefbc06eb70 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefb97fa640 1 -- 172.21.15.133:0/842962873 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac019070 con 0x7fefbc06eb70 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.201+0000 7fefb97fa640 1 -- 172.21.15.133:0/842962873 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac00ba60 con 0x7fefbc06eb70 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.205+0000 7fefc24c6640 1 -- 172.21.15.133:0/842962873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc14f9a0 con 0x7fefbc06eb70 2024-04-25T10:07:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.205+0000 7fefc24c6640 1 -- 172.21.15.133:0/842962873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc14ff60 con 0x7fefbc06eb70 2024-04-25T10:07:28.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.205+0000 7fefc24c6640 1 -- 172.21.15.133:0/842962873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefbc06bc20 con 0x7fefbc06eb70 2024-04-25T10:07:28.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.205+0000 7fefb97fa640 1 -- 172.21.15.133:0/842962873 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fefac00bc00 con 0x7fefbc06eb70 2024-04-25T10:07:28.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.205+0000 7fefb97fa640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef9c066f60 0x7fef9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.205+0000 7fefb97fa640 1 -- 172.21.15.133:0/842962873 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fefac085d80 con 0x7fefbc06eb70 2024-04-25T10:07:28.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.205+0000 7fefb3fff640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef9c066f60 0x7fef9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.205+0000 7fefb3fff640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef9c066f60 0x7fef9c069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0ce2e0 tx=0x7fefa4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.209+0000 7fefb97fa640 1 -- 172.21.15.133:0/842962873 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fefac057030 con 0x7fefbc06eb70 2024-04-25T10:07:28.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.409+0000 7fefc24c6640 1 -- 172.21.15.133:0/842962873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fefbc068ef0 con 0x7fefbc06eb70 2024-04-25T10:07:28.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.409+0000 7fefb97fa640 1 -- 172.21.15.133:0/842962873 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fefac056930 con 0x7fefbc06eb70 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.415 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:07:28.416 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:07:28.417 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:07:28.418 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:07:28.419 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.419 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:28.419 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:07:28.419 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.419 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.419 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.419 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.419 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:07:28.419 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:07:28.419 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.420 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:28.420 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:07:28.420 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.420 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.420 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.420 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.420 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:07:28.420 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:07:28.420 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.420 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:28.420 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.421 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:28.422 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:07:28.423 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:07:28.424 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:07:28.425 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.425 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.425 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.425 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:07:28.425 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:07:28.425 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.425 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:07:28.425 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:07:28.425 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:07:28.425 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.426 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:07:28.427 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:07:28.428 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 -- 172.21.15.133:0/842962873 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef9c066f60 msgr2=0x7fef9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef9c066f60 0x7fef9c069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0ce2e0 tx=0x7fefa4008040 comp rx=0 tx=0).stop 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 -- 172.21.15.133:0/842962873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefbc06eb70 msgr2=0x7fefbc1448d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefbc06eb70 0x7fefbc1448d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fefac00d800 tx=0x7fefac00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 -- 172.21.15.133:0/842962873 shutdown_connections 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefbc1584f0 0x7fefbc06d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefbc158110 0x7fefbc148a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fef9c066f60 0x7fef9c069420 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 --2- 172.21.15.133:0/842962873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefbc06eb70 0x7fefbc1448d0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 -- 172.21.15.133:0/842962873 >> 172.21.15.133:0/842962873 conn(0x7fefbc067000 msgr2=0x7fefbc1521c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 -- 172.21.15.133:0/842962873 shutdown_connections 2024-04-25T10:07:28.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.413+0000 7fefc24c6640 1 -- 172.21.15.133:0/842962873 wait complete. 2024-04-25T10:07:28.433 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:28.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.537+0000 7f4bef6b3640 1 Processor -- start 2024-04-25T10:07:28.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.537+0000 7f4bef6b3640 1 -- start start 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.537+0000 7f4bef6b3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4be815b9b0 0x7f4be80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.537+0000 7f4bef6b3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4be80cf7c0 0x7f4be80cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.537+0000 7f4bef6b3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be815bd90 con 0x7f4be80d01b0 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.537+0000 7f4bef6b3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be80c5610 con 0x7f4be815b9b0 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.537+0000 7f4bef6b3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be809ad10 con 0x7f4be80cf7c0 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.537+0000 7f4bed44f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4be815b9b0 0x7f4be80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.537+0000 7f4bed44f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4be815b9b0 0x7f4be80cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40608/0 (socket says 172.21.15.133:40608) 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.537+0000 7f4bed44f640 1 -- 172.21.15.133:0/3460125290 learned_addr learned my addr 172.21.15.133:0/3460125290 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4becc4e640 1 --2- 172.21.15.133:0/3460125290 >> v2:172.21.15.162:3300/0 conn(0x7f4be80cf7c0 0x7f4be80cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bed44f640 1 -- 172.21.15.133:0/3460125290 >> v2:172.21.15.162:3300/0 conn(0x7f4be80cf7c0 msgr2=0x7f4be80cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bed44f640 1 --2- 172.21.15.133:0/3460125290 >> v2:172.21.15.162:3300/0 conn(0x7f4be80cf7c0 0x7f4be80cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bed44f640 1 -- 172.21.15.133:0/3460125290 >> v1:172.21.15.162:6789/0 conn(0x7f4be80d01b0 legacy=0x7f4be80cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bed44f640 1 -- 172.21.15.133:0/3460125290 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be809a750 con 0x7f4be815b9b0 2024-04-25T10:07:28.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bed44f640 1 --2- 172.21.15.133:0/3460125290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be815b9b0 0x7f4be80cf280 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009870 tx=0x7f4be401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e56464c4b0912 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bdffff640 1 -- 172.21.15.133:0/3460125290 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40b1070 con 0x7f4be815b9b0 2024-04-25T10:07:28.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bdffff640 1 -- 172.21.15.133:0/3460125290 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be40ac630 con 0x7f4be815b9b0 2024-04-25T10:07:28.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bdffff640 1 -- 172.21.15.133:0/3460125290 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40a8050 con 0x7f4be815b9b0 2024-04-25T10:07:28.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bef6b3640 1 -- 172.21.15.133:0/3460125290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be815b9b0 msgr2=0x7f4be80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bef6b3640 1 --2- 172.21.15.133:0/3460125290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be815b9b0 0x7f4be80cf280 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009870 tx=0x7f4be401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bef6b3640 1 -- 172.21.15.133:0/3460125290 shutdown_connections 2024-04-25T10:07:28.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bef6b3640 1 --2- 172.21.15.133:0/3460125290 >> v2:172.21.15.162:3300/0 conn(0x7f4be80cf7c0 0x7f4be80cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bef6b3640 1 --2- 172.21.15.133:0/3460125290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be815b9b0 0x7f4be80cf280 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bef6b3640 1 -- 172.21.15.133:0/3460125290 >> 172.21.15.133:0/3460125290 conn(0x7f4be80b31a0 msgr2=0x7f4be80b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bef6b3640 1 -- 172.21.15.133:0/3460125290 shutdown_connections 2024-04-25T10:07:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.541+0000 7f4bef6b3640 1 -- 172.21.15.133:0/3460125290 wait complete. 2024-04-25T10:07:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bef6b3640 1 Processor -- start 2024-04-25T10:07:28.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bef6b3640 1 -- start start 2024-04-25T10:07:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bef6b3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be80cf7c0 0x7f4be814e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bef6b3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4be80d01b0 0x7f4be814e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bef6b3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be815b9b0 0x7f4be80c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bef6b3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8168660 con 0x7f4be80cf7c0 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bef6b3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be81684e0 con 0x7f4be815b9b0 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bef6b3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be81687e0 con 0x7f4be80d01b0 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bed44f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be80cf7c0 0x7f4be814e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bed44f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be80cf7c0 0x7f4be814e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40610/0 (socket says 172.21.15.133:40610) 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bed44f640 1 -- 172.21.15.133:0/1209487485 learned_addr learned my addr 172.21.15.133:0/1209487485 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bedc50640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be815b9b0 0x7f4be80c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4becc4e640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4be80d01b0 0x7f4be814e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bed44f640 1 -- 172.21.15.133:0/1209487485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4be80d01b0 msgr2=0x7f4be814e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bed44f640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4be80d01b0 0x7f4be814e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bed44f640 1 -- 172.21.15.133:0/1209487485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be815b9b0 msgr2=0x7f4be80c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bed44f640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be815b9b0 0x7f4be80c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bed44f640 1 -- 172.21.15.133:0/1209487485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be814fb80 con 0x7f4be80cf7c0 2024-04-25T10:07:28.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4becc4e640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4be80d01b0 0x7f4be814e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bed44f640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be80cf7c0 0x7f4be814e270 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4be40af040 tx=0x7f4be4019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bddffb640 1 -- 172.21.15.133:0/1209487485 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40bc070 con 0x7f4be80cf7c0 2024-04-25T10:07:28.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bddffb640 1 -- 172.21.15.133:0/1209487485 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be40b7430 con 0x7f4be80cf7c0 2024-04-25T10:07:28.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bef6b3640 1 -- 172.21.15.133:0/1209487485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be814f2e0 con 0x7f4be80cf7c0 2024-04-25T10:07:28.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bddffb640 1 -- 172.21.15.133:0/1209487485 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40b1040 con 0x7f4be80cf7c0 2024-04-25T10:07:28.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bef6b3640 1 -- 172.21.15.133:0/1209487485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be814f8a0 con 0x7f4be80cf7c0 2024-04-25T10:07:28.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bef6b3640 1 -- 172.21.15.133:0/1209487485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be8067c70 con 0x7f4be80cf7c0 2024-04-25T10:07:28.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.545+0000 7f4bddffb640 1 -- 172.21.15.133:0/1209487485 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4be40bf070 con 0x7f4be80cf7c0 2024-04-25T10:07:28.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.549+0000 7f4bddffb640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4bcc066f90 0x7f4bcc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.549+0000 7f4bddffb640 1 -- 172.21.15.133:0/1209487485 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4be411e400 con 0x7f4be80cf7c0 2024-04-25T10:07:28.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.549+0000 7f4becc4e640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4bcc066f90 0x7f4bcc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.549+0000 7f4becc4e640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4bcc066f90 0x7f4bcc069450 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4be8159ee0 tx=0x7f4bd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.553+0000 7f4bddffb640 1 -- 172.21.15.133:0/1209487485 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4be40160d0 con 0x7f4be80cf7c0 2024-04-25T10:07:28.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.709+0000 7f4bef6b3640 1 -- 172.21.15.133:0/1209487485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4be80cc220 con 0x7f4be80cf7c0 2024-04-25T10:07:28.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bddffb640 1 -- 172.21.15.133:0/1209487485 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+13189 (secure 0 0 0) 0x7f4be40ef5d0 con 0x7f4be80cf7c0 2024-04-25T10:07:28.717 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:28.718 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":77,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:07:28.062310+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:28.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 -- 172.21.15.133:0/1209487485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4bcc066f90 msgr2=0x7f4bcc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4bcc066f90 0x7f4bcc069450 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4be8159ee0 tx=0x7f4bd8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:28.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 -- 172.21.15.133:0/1209487485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be80cf7c0 msgr2=0x7f4be814e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be80cf7c0 0x7f4be814e270 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4be40af040 tx=0x7f4be4019280 comp rx=0 tx=0).stop 2024-04-25T10:07:28.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 -- 172.21.15.133:0/1209487485 shutdown_connections 2024-04-25T10:07:28.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4be815b9b0 0x7f4be80c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4be80d01b0 0x7f4be814e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4bcc066f90 0x7f4bcc069450 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 --2- 172.21.15.133:0/1209487485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4be80cf7c0 0x7f4be814e270 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 -- 172.21.15.133:0/1209487485 >> 172.21.15.133:0/1209487485 conn(0x7f4be80b31a0 msgr2=0x7f4be80cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 -- 172.21.15.133:0/1209487485 shutdown_connections 2024-04-25T10:07:28.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.713+0000 7f4bef6b3640 1 -- 172.21.15.133:0/1209487485 wait complete. 2024-04-25T10:07:28.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:28.737 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:28.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe967a70640 1 Processor -- start 2024-04-25T10:07:28.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe967a70640 1 -- start start 2024-04-25T10:07:28.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe967a70640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe960142920 0x7fe960143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe967a70640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe96015d9f0 0x7fe96015de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe967a70640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe960140ea0 con 0x7fe960140a40 2024-04-25T10:07:28.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe967a70640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96015e370 con 0x7fe96015d9f0 2024-04-25T10:07:28.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe967a70640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9600cf280 con 0x7fe960142920 2024-04-25T10:07:28.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe96600d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe96015d9f0 0x7fe96015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe96500b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe960140a40 0x7fe9600ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34094/0 (socket says 172.21.15.133:34094) 2024-04-25T10:07:28.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe96500b640 1 -- 172.21.15.133:0/445227477 learned_addr learned my addr 172.21.15.133:0/445227477 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:28.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe96600d640 1 -- 172.21.15.133:0/445227477 >> v2:172.21.15.162:3300/0 conn(0x7fe960142920 msgr2=0x7fe960143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe96580c640 1 --2- 172.21.15.133:0/445227477 >> v2:172.21.15.162:3300/0 conn(0x7fe960142920 0x7fe960143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe96600d640 1 --2- 172.21.15.133:0/445227477 >> v2:172.21.15.162:3300/0 conn(0x7fe960142920 0x7fe960143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe96600d640 1 -- 172.21.15.133:0/445227477 >> v1:172.21.15.133:6789/0 conn(0x7fe960140a40 legacy=0x7fe9600ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe96480a640 1 -- 172.21.15.133:0/445227477 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887830923 0 0) 0x7fe960140ea0 con 0x7fe960140a40 2024-04-25T10:07:28.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe96600d640 1 -- 172.21.15.133:0/445227477 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96006d440 con 0x7fe96015d9f0 2024-04-25T10:07:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.861+0000 7fe96600d640 1 --2- 172.21.15.133:0/445227477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96015d9f0 0x7fe96015de30 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fe96006ee50 tx=0x7fe95c01d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ef2142352810a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96480a640 1 -- 172.21.15.133:0/445227477 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00b040 con 0x7fe96015d9f0 2024-04-25T10:07:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96480a640 1 -- 172.21.15.133:0/445227477 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c0a76f0 con 0x7fe96015d9f0 2024-04-25T10:07:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96480a640 1 -- 172.21.15.133:0/445227477 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0a7ac0 con 0x7fe96015d9f0 2024-04-25T10:07:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 -- 172.21.15.133:0/445227477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96015d9f0 msgr2=0x7fe96015de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 --2- 172.21.15.133:0/445227477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96015d9f0 0x7fe96015de30 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fe96006ee50 tx=0x7fe95c01d7c0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 -- 172.21.15.133:0/445227477 shutdown_connections 2024-04-25T10:07:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 --2- 172.21.15.133:0/445227477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe96015d9f0 0x7fe96015de30 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 --2- 172.21.15.133:0/445227477 >> v2:172.21.15.162:3300/0 conn(0x7fe960142920 0x7fe960143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 -- 172.21.15.133:0/445227477 >> 172.21.15.133:0/445227477 conn(0x7fe960066f70 msgr2=0x7fe96006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 -- 172.21.15.133:0/445227477 shutdown_connections 2024-04-25T10:07:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 -- 172.21.15.133:0/445227477 wait complete. 2024-04-25T10:07:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 Processor -- start 2024-04-25T10:07:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 -- start start 2024-04-25T10:07:28.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe960140a40 0x7fe960156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142920 0x7fe96015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe96015d9f0 0x7fe96015c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96006e0a0 con 0x7fe960142920 2024-04-25T10:07:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96006df20 con 0x7fe960140a40 2024-04-25T10:07:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96006e220 con 0x7fe96015d9f0 2024-04-25T10:07:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96500b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142920 0x7fe96015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96500b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142920 0x7fe96015bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40626/0 (socket says 172.21.15.133:40626) 2024-04-25T10:07:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96500b640 1 -- 172.21.15.133:0/3072276301 learned_addr learned my addr 172.21.15.133:0/3072276301 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96500b640 1 -- 172.21.15.133:0/3072276301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe96015d9f0 msgr2=0x7fe96015c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96600d640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe96015d9f0 0x7fe96015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96580c640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe960140a40 0x7fe960156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96500b640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe96015d9f0 0x7fe96015c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96500b640 1 -- 172.21.15.133:0/3072276301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe960140a40 msgr2=0x7fe960156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96500b640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe960140a40 0x7fe960156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96500b640 1 -- 172.21.15.133:0/3072276301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600b0740 con 0x7fe960142920 2024-04-25T10:07:28.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96580c640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe960140a40 0x7fe960156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe96500b640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142920 0x7fe96015bd20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c480 tx=0x7fe95000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe9567fc640 1 -- 172.21.15.133:0/3072276301 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950005cf0 con 0x7fe960142920 2024-04-25T10:07:28.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe9567fc640 1 -- 172.21.15.133:0/3072276301 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe950007070 con 0x7fe960142920 2024-04-25T10:07:28.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 -- 172.21.15.133:0/3072276301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9600b09e0 con 0x7fe960142920 2024-04-25T10:07:28.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe9567fc640 1 -- 172.21.15.133:0/3072276301 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9500074a0 con 0x7fe960142920 2024-04-25T10:07:28.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.865+0000 7fe967a70640 1 -- 172.21.15.133:0/3072276301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9600b0fa0 con 0x7fe960142920 2024-04-25T10:07:28.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.869+0000 7fe967a70640 1 -- 172.21.15.133:0/3072276301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe960068e80 con 0x7fe960142920 2024-04-25T10:07:28.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.869+0000 7fe9567fc640 1 -- 172.21.15.133:0/3072276301 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe95001b400 con 0x7fe960142920 2024-04-25T10:07:28.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.869+0000 7fe9567fc640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe944066f60 0x7fe944069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.869+0000 7fe9567fc640 1 -- 172.21.15.133:0/3072276301 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe950084b00 con 0x7fe960142920 2024-04-25T10:07:28.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.869+0000 7fe96580c640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe944066f60 0x7fe944069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.869+0000 7fe96580c640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe944066f60 0x7fe944069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe96015c940 tx=0x7fe9580074c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:28.873+0000 7fe9567fc640 1 -- 172.21.15.133:0/3072276301 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe950055d60 con 0x7fe960142920 2024-04-25T10:07:29.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 -- 172.21.15.133:0/3072276301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9600afa70 con 0x7fe960142920 2024-04-25T10:07:29.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe9567fc640 1 -- 172.21.15.133:0/3072276301 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe950055660 con 0x7fe960142920 2024-04-25T10:07:29.045 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:07:29.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 -- 172.21.15.133:0/3072276301 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe944066f60 msgr2=0x7fe944069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe944066f60 0x7fe944069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe96015c940 tx=0x7fe9580074c0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 -- 172.21.15.133:0/3072276301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142920 msgr2=0x7fe96015bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142920 0x7fe96015bd20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c480 tx=0x7fe95000c940 comp rx=0 tx=0).stop 2024-04-25T10:07:29.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 -- 172.21.15.133:0/3072276301 shutdown_connections 2024-04-25T10:07:29.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe96015d9f0 0x7fe96015c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe944066f60 0x7fe944069420 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe960142920 0x7fe96015bd20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 --2- 172.21.15.133:0/3072276301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe960140a40 0x7fe960156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 -- 172.21.15.133:0/3072276301 >> 172.21.15.133:0/3072276301 conn(0x7fe960066f70 msgr2=0x7fe96013fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:29.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 -- 172.21.15.133:0/3072276301 shutdown_connections 2024-04-25T10:07:29.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.041+0000 7fe967a70640 1 -- 172.21.15.133:0/3072276301 wait complete. 2024-04-25T10:07:29.066 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:07:29.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22ec5b640 1 Processor -- start 2024-04-25T10:07:29.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22ec5b640 1 -- start start 2024-04-25T10:07:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22ec5b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff228158110 0x7ff2280b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22ec5b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2281584f0 0x7ff22815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22ec5b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2280cbe60 con 0x7ff22806e460 2024-04-25T10:07:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22ec5b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2280b19b0 con 0x7ff228158110 2024-04-25T10:07:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22ec5b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22815b360 con 0x7ff2281584f0 2024-04-25T10:07:29.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22c9f7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff228158110 0x7ff2280b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22c9f7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff228158110 0x7ff2280b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:37070/0 (socket says 172.21.15.133:37070) 2024-04-25T10:07:29.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff227fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2281584f0 0x7ff22815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22c9f7640 1 -- 172.21.15.133:0/3009215892 learned_addr learned my addr 172.21.15.133:0/3009215892 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:29.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22c9f7640 1 -- 172.21.15.133:0/3009215892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2281584f0 msgr2=0x7ff22815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:29.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22c9f7640 1 --2- 172.21.15.133:0/3009215892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2281584f0 0x7ff22815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22c9f7640 1 -- 172.21.15.133:0/3009215892 >> v1:172.21.15.162:6789/0 conn(0x7ff22806e460 legacy=0x7ff22815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.177+0000 7ff22c9f7640 1 -- 172.21.15.133:0/3009215892 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22815b640 con 0x7ff228158110 2024-04-25T10:07:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22c9f7640 1 --2- 172.21.15.133:0/3009215892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff228158110 0x7ff2280b1f00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00bb10 tx=0x7ff21c01ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d61bdfd5a79ca1f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff2277fe640 1 -- 172.21.15.133:0/3009215892 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c0b4070 con 0x7ff228158110 2024-04-25T10:07:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff2277fe640 1 -- 172.21.15.133:0/3009215892 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c0a7c10 con 0x7ff228158110 2024-04-25T10:07:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff2277fe640 1 -- 172.21.15.133:0/3009215892 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c0af440 con 0x7ff228158110 2024-04-25T10:07:29.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 -- 172.21.15.133:0/3009215892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff228158110 msgr2=0x7ff2280b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 --2- 172.21.15.133:0/3009215892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff228158110 0x7ff2280b1f00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00bb10 tx=0x7ff21c01ec30 comp rx=0 tx=0).stop 2024-04-25T10:07:29.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 -- 172.21.15.133:0/3009215892 shutdown_connections 2024-04-25T10:07:29.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 --2- 172.21.15.133:0/3009215892 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2281584f0 0x7ff22815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 --2- 172.21.15.133:0/3009215892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff228158110 0x7ff2280b1f00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 -- 172.21.15.133:0/3009215892 >> 172.21.15.133:0/3009215892 conn(0x7ff228067000 msgr2=0x7ff22806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:29.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 -- 172.21.15.133:0/3009215892 shutdown_connections 2024-04-25T10:07:29.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 -- 172.21.15.133:0/3009215892 wait complete. 2024-04-25T10:07:29.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 Processor -- start 2024-04-25T10:07:29.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 -- start start 2024-04-25T10:07:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff22806e460 0x7ff22813cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff228158110 0x7ff22815caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2281584f0 0x7ff22815d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2280cd7f0 con 0x7ff2281584f0 2024-04-25T10:07:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2280cd670 con 0x7ff22806e460 2024-04-25T10:07:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22ec5b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22806d350 con 0x7ff228158110 2024-04-25T10:07:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22d1f8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2281584f0 0x7ff22815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22d1f8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2281584f0 0x7ff22815d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40632/0 (socket says 172.21.15.133:40632) 2024-04-25T10:07:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.181+0000 7ff22d1f8640 1 -- 172.21.15.133:0/633495831 learned_addr learned my addr 172.21.15.133:0/633495831 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22c9f7640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff22806e460 0x7ff22813cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff227fff640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff228158110 0x7ff22815caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22d1f8640 1 -- 172.21.15.133:0/633495831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff228158110 msgr2=0x7ff22815caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22d1f8640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff228158110 0x7ff22815caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22d1f8640 1 -- 172.21.15.133:0/633495831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff22806e460 msgr2=0x7ff22813cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22d1f8640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff22806e460 0x7ff22813cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22d1f8640 1 -- 172.21.15.133:0/633495831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2280b4430 con 0x7ff2281584f0 2024-04-25T10:07:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22d1f8640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2281584f0 0x7ff22815d030 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff22000c7a0 tx=0x7ff22000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff2257fa640 1 -- 172.21.15.133:0/633495831 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220003b80 con 0x7ff2281584f0 2024-04-25T10:07:29.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22ec5b640 1 -- 172.21.15.133:0/633495831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2280b33b0 con 0x7ff2281584f0 2024-04-25T10:07:29.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff2257fa640 1 -- 172.21.15.133:0/633495831 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff220003d20 con 0x7ff2281584f0 2024-04-25T10:07:29.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff2257fa640 1 -- 172.21.15.133:0/633495831 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22000aa90 con 0x7ff2281584f0 2024-04-25T10:07:29.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22ec5b640 1 -- 172.21.15.133:0/633495831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2280b3950 con 0x7ff2281584f0 2024-04-25T10:07:29.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22ec5b640 1 -- 172.21.15.133:0/633495831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22806bc20 con 0x7ff2281584f0 2024-04-25T10:07:29.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff2257fa640 1 -- 172.21.15.133:0/633495831 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff22000ac30 con 0x7ff2281584f0 2024-04-25T10:07:29.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff2257fa640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff214066f60 0x7ff214069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff2257fa640 1 -- 172.21.15.133:0/633495831 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff220084de0 con 0x7ff2281584f0 2024-04-25T10:07:29.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22c9f7640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff214066f60 0x7ff214069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.185+0000 7ff22c9f7640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff214066f60 0x7ff214069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0b1000 tx=0x7ff21c0a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.189+0000 7ff2257fa640 1 -- 172.21.15.133:0/633495831 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff220055ea0 con 0x7ff2281584f0 2024-04-25T10:07:29.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.349+0000 7ff22ec5b640 1 -- 172.21.15.133:0/633495831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff228153ca0 con 0x7ff2281584f0 2024-04-25T10:07:29.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.349+0000 7ff2257fa640 1 -- 172.21.15.133:0/633495831 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff2200557a0 con 0x7ff2281584f0 2024-04-25T10:07:29.355 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:29.355 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:07:29.355 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:07:29.355 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:07:29.355 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:07:29.355 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:29.355 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:07:29.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 -- 172.21.15.133:0/633495831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff214066f60 msgr2=0x7ff214069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff214066f60 0x7ff214069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0b1000 tx=0x7ff21c0a9070 comp rx=0 tx=0).stop 2024-04-25T10:07:29.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 -- 172.21.15.133:0/633495831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2281584f0 msgr2=0x7ff22815d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2281584f0 0x7ff22815d030 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff22000c7a0 tx=0x7ff22000cc60 comp rx=0 tx=0).stop 2024-04-25T10:07:29.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 -- 172.21.15.133:0/633495831 shutdown_connections 2024-04-25T10:07:29.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff214066f60 0x7ff214069420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2281584f0 0x7ff22815d030 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff228158110 0x7ff22815caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 --2- 172.21.15.133:0/633495831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff22806e460 0x7ff22813cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 -- 172.21.15.133:0/633495831 >> 172.21.15.133:0/633495831 conn(0x7ff228067000 msgr2=0x7ff2280cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:29.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 -- 172.21.15.133:0/633495831 shutdown_connections 2024-04-25T10:07:29.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.353+0000 7ff22ec5b640 1 -- 172.21.15.133:0/633495831 wait complete. 2024-04-25T10:07:29.373 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:07:29.373 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:07:29.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.489+0000 7f386021c640 1 Processor -- start 2024-04-25T10:07:29.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.489+0000 7f386021c640 1 -- start start 2024-04-25T10:07:29.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f386021c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815d8c0 0x7f3858161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f386021c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f38580cef10 0x7f38580d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f386021c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38580d1cf0 con 0x7f3858161c60 2024-04-25T10:07:29.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f386021c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38581626f0 con 0x7f38580cef10 2024-04-25T10:07:29.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f386021c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3858162130 con 0x7f385815d8c0 2024-04-25T10:07:29.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385d7b7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3858161c60 0x7f38580ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34110/0 (socket says 172.21.15.133:34110) 2024-04-25T10:07:29.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385d7b7640 1 -- 172.21.15.133:0/1301298047 learned_addr learned my addr 172.21.15.133:0/1301298047 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:29.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385dfb8640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815d8c0 0x7f3858161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385e7b9640 1 --2- 172.21.15.133:0/1301298047 >> v2:172.21.15.162:3300/0 conn(0x7f38580cef10 0x7f38580d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476143019 0 0) 0x7f38580d1cf0 con 0x7f3858161c60 2024-04-25T10:07:29.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3844003530 con 0x7f3858161c60 2024-04-25T10:07:29.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 249522004 0 0) 0x7f3844003530 con 0x7f3858161c60 2024-04-25T10:07:29.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815d8c0 msgr2=0x7f3858161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815d8c0 0x7f3858161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 >> v2:172.21.15.162:3300/0 conn(0x7f38580cef10 msgr2=0x7f38580d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 --2- 172.21.15.133:0/1301298047 >> v2:172.21.15.162:3300/0 conn(0x7f38580cef10 0x7f38580d24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3858162410 con 0x7f3858161c60 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f3848001680 con 0x7f3858161c60 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 >> v1:172.21.15.133:6789/0 conn(0x7f3858161c60 legacy=0x7f38580ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815d8c0 0x7f38440039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3844003f40 0x7f3844004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 0x7f38440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38581626f0 con 0x7f3844004d50 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3858162130 con 0x7f3844003f40 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384400ba20 con 0x7f385815d8c0 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385e7b9640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 0x7f38440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385d7b7640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3844003f40 0x7f3844004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385dfb8640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815d8c0 0x7f38440039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385e7b9640 1 -- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815d8c0 msgr2=0x7f38440039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:29.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385e7b9640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815d8c0 0x7f38440039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385e7b9640 1 -- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3844003f40 msgr2=0x7f3844004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:29.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385e7b9640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3844003f40 0x7f3844004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385e7b9640 1 -- 172.21.15.133:0/1301298047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3844003530 con 0x7f3844004d50 2024-04-25T10:07:29.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385d7b7640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3844003f40 0x7f3844004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:29.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.493+0000 7f385e7b9640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 0x7f38440051d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3850009fc0 tx=0x7f385001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d703835cf33ffcec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38500b2070 con 0x7f3844004d50 2024-04-25T10:07:29.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38500a5c90 con 0x7f3844004d50 2024-04-25T10:07:29.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385cfb6640 1 -- 172.21.15.133:0/1301298047 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850009c10 con 0x7f3844004d50 2024-04-25T10:07:29.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 -- 172.21.15.133:0/1301298047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 msgr2=0x7f38440051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 --2- 172.21.15.133:0/1301298047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 0x7f38440051d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3850009fc0 tx=0x7f385001cc80 comp rx=0 tx=0).stop 2024-04-25T10:07:29.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385e7b9640 1 -- 172.21.15.133:0/1301298047 reap_dead start 2024-04-25T10:07:29.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 -- 172.21.15.133:0/1301298047 shutdown_connections 2024-04-25T10:07:29.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 -- 172.21.15.133:0/1301298047 >> 172.21.15.133:0/1301298047 conn(0x7f38580723c0 msgr2=0x7f38580727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:29.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 -- 172.21.15.133:0/1301298047 shutdown_connections 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 -- 172.21.15.133:0/1301298047 wait complete. 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 Processor -- start 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 -- start start 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 0x7f3858143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815efd0 0x7f385814ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3858150510 0x7f3858160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38580b9990 con 0x7f3844004d50 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38580b9810 con 0x7f3858150510 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385815dca0 con 0x7f385815efd0 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385dfb8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 0x7f3858143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385dfb8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 0x7f3858143ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40646/0 (socket says 172.21.15.133:40646) 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385dfb8640 1 -- 172.21.15.133:0/1066715225 learned_addr learned my addr 172.21.15.133:0/1066715225 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385dfb8640 1 -- 172.21.15.133:0/1066715225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815efd0 msgr2=0x7f385814ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385dfb8640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815efd0 0x7f385814ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385dfb8640 1 -- 172.21.15.133:0/1066715225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3858150510 msgr2=0x7f3858160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385e7b9640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3858150510 0x7f3858160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385dfb8640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3858150510 0x7f3858160160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385dfb8640 1 -- 172.21.15.133:0/1066715225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385814ef30 con 0x7f3844004d50 2024-04-25T10:07:29.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f385dfb8640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 0x7f3858143ea0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f38540108c0 tx=0x7f3854010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f384e7fc640 1 -- 172.21.15.133:0/1066715225 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400f040 con 0x7f3844004d50 2024-04-25T10:07:29.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f384e7fc640 1 -- 172.21.15.133:0/1066715225 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385400eb00 con 0x7f3844004d50 2024-04-25T10:07:29.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 -- 172.21.15.133:0/1066715225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385814f210 con 0x7f3844004d50 2024-04-25T10:07:29.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f384e7fc640 1 -- 172.21.15.133:0/1066715225 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385401c400 con 0x7f3844004d50 2024-04-25T10:07:29.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.497+0000 7f386021c640 1 -- 172.21.15.133:0/1066715225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385814f7d0 con 0x7f3844004d50 2024-04-25T10:07:29.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.501+0000 7f386021c640 1 -- 172.21.15.133:0/1066715225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3858069000 con 0x7f3844004d50 2024-04-25T10:07:29.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.501+0000 7f384e7fc640 1 -- 172.21.15.133:0/1066715225 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f385401c5a0 con 0x7f3844004d50 2024-04-25T10:07:29.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.501+0000 7f384e7fc640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f383c066f90 0x7f383c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.501+0000 7f384e7fc640 1 -- 172.21.15.133:0/1066715225 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3854085520 con 0x7f3844004d50 2024-04-25T10:07:29.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.501+0000 7f385d7b7640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f383c066f90 0x7f383c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.501+0000 7f385d7b7640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f383c066f90 0x7f383c069450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3848004880 tx=0x7f3848000ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.505+0000 7f384e7fc640 1 -- 172.21.15.133:0/1066715225 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3854018030 con 0x7f3844004d50 2024-04-25T10:07:29.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:29.653+0000 7f386021c640 1 -- 172.21.15.133:0/1066715225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f38580d1700 con 0x7f3844004d50 2024-04-25T10:07:30.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.049+0000 7f384e7fc640 1 -- 172.21.15.133:0/1066715225 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3854056650 con 0x7f3844004d50 2024-04-25T10:07:30.054 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:07:30.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 -- 172.21.15.133:0/1066715225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f383c066f90 msgr2=0x7f383c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f383c066f90 0x7f383c069450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3848004880 tx=0x7f3848000ee0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 -- 172.21.15.133:0/1066715225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 msgr2=0x7f3858143ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 0x7f3858143ea0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f38540108c0 tx=0x7f3854010d80 comp rx=0 tx=0).stop 2024-04-25T10:07:30.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 -- 172.21.15.133:0/1066715225 shutdown_connections 2024-04-25T10:07:30.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3858150510 0x7f3858160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f385815efd0 0x7f385814ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f383c066f90 0x7f383c069450 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 --2- 172.21.15.133:0/1066715225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3844004d50 0x7f3858143ea0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 -- 172.21.15.133:0/1066715225 >> 172.21.15.133:0/1066715225 conn(0x7f38580723c0 msgr2=0x7f38580d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:30.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 -- 172.21.15.133:0/1066715225 shutdown_connections 2024-04-25T10:07:30.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.053+0000 7f386021c640 1 -- 172.21.15.133:0/1066715225 wait complete. 2024-04-25T10:07:30.081 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:07:30.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe25ac05640 1 Processor -- start 2024-04-25T10:07:30.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe25ac05640 1 -- start start 2024-04-25T10:07:30.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe25ac05640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe254143ea0 0x7fe2540cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe25ac05640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 0x7fe25406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe25ac05640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe25415b9b0 0x7fe25406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe25ac05640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25415ac50 con 0x7fe254143ea0 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe25ac05640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2540cd180 con 0x7fe25415b9b0 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe25ac05640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25415aed0 con 0x7fe2540b2150 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe2589a1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe254143ea0 0x7fe2540cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe2589a1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe254143ea0 0x7fe2540cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40658/0 (socket says 172.21.15.133:40658) 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe2589a1640 1 -- 172.21.15.133:0/623754617 learned_addr learned my addr 172.21.15.133:0/623754617 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe2591a2640 1 --2- 172.21.15.133:0/623754617 >> v2:172.21.15.162:3300/0 conn(0x7fe25415b9b0 0x7fe25406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe253fff640 1 --2- 172.21.15.133:0/623754617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 0x7fe25406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe2589a1640 1 -- 172.21.15.133:0/623754617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 msgr2=0x7fe25406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe2589a1640 1 --2- 172.21.15.133:0/623754617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 0x7fe25406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe2589a1640 1 -- 172.21.15.133:0/623754617 >> v2:172.21.15.162:3300/0 conn(0x7fe25415b9b0 msgr2=0x7fe25406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe2589a1640 1 --2- 172.21.15.133:0/623754617 >> v2:172.21.15.162:3300/0 conn(0x7fe25415b9b0 0x7fe25406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe2589a1640 1 -- 172.21.15.133:0/623754617 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25415b1b0 con 0x7fe254143ea0 2024-04-25T10:07:30.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.205+0000 7fe253fff640 1 --2- 172.21.15.133:0/623754617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 0x7fe25406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:30.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe2589a1640 1 --2- 172.21.15.133:0/623754617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe254143ea0 0x7fe2540cc810 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fe254151d40 tx=0x7fe24801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbb9b4aacb3ad7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:30.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe2537fe640 1 -- 172.21.15.133:0/623754617 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2480b4070 con 0x7fe254143ea0 2024-04-25T10:07:30.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe2537fe640 1 -- 172.21.15.133:0/623754617 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2480a7b90 con 0x7fe254143ea0 2024-04-25T10:07:30.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe2537fe640 1 -- 172.21.15.133:0/623754617 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2480af440 con 0x7fe254143ea0 2024-04-25T10:07:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe25ac05640 1 -- 172.21.15.133:0/623754617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe254143ea0 msgr2=0x7fe2540cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe25ac05640 1 --2- 172.21.15.133:0/623754617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe254143ea0 0x7fe2540cc810 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fe254151d40 tx=0x7fe24801ebf0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe25ac05640 1 -- 172.21.15.133:0/623754617 shutdown_connections 2024-04-25T10:07:30.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe25ac05640 1 --2- 172.21.15.133:0/623754617 >> v2:172.21.15.162:3300/0 conn(0x7fe25415b9b0 0x7fe25406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe25ac05640 1 --2- 172.21.15.133:0/623754617 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 0x7fe25406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe25ac05640 1 --2- 172.21.15.133:0/623754617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe254143ea0 0x7fe2540cc810 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe25ac05640 1 -- 172.21.15.133:0/623754617 >> 172.21.15.133:0/623754617 conn(0x7fe254066f70 msgr2=0x7fe25406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:30.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe25ac05640 1 -- 172.21.15.133:0/623754617 shutdown_connections 2024-04-25T10:07:30.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe25ac05640 1 -- 172.21.15.133:0/623754617 wait complete. 2024-04-25T10:07:30.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe25ac05640 1 Processor -- start 2024-04-25T10:07:30.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.209+0000 7fe25ac05640 1 -- start start 2024-04-25T10:07:30.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe25ac05640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 0x7fe254138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe25ac05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe254143ea0 0x7fe254156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe25ac05640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe25415b9b0 0x7fe2541577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe25ac05640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25415ede0 con 0x7fe254143ea0 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe25ac05640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25415ec60 con 0x7fe25415b9b0 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe25ac05640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25415ef60 con 0x7fe2540b2150 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe253fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe254143ea0 0x7fe254156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe253fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe254143ea0 0x7fe254156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40664/0 (socket says 172.21.15.133:40664) 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe253fff640 1 -- 172.21.15.133:0/1424714452 learned_addr learned my addr 172.21.15.133:0/1424714452 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe2589a1640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 0x7fe254138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe2591a2640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe25415b9b0 0x7fe2541577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe253fff640 1 -- 172.21.15.133:0/1424714452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 msgr2=0x7fe254138a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe253fff640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 0x7fe254138a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe253fff640 1 -- 172.21.15.133:0/1424714452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe25415b9b0 msgr2=0x7fe2541577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe253fff640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe25415b9b0 0x7fe2541577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe253fff640 1 -- 172.21.15.133:0/1424714452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2541393e0 con 0x7fe254143ea0 2024-04-25T10:07:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe2589a1640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 0x7fe254138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe253fff640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe254143ea0 0x7fe254156440 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe24000c7f0 tx=0x7fe24000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe2517fa640 1 -- 172.21.15.133:0/1424714452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240007a30 con 0x7fe254143ea0 2024-04-25T10:07:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe2517fa640 1 -- 172.21.15.133:0/1424714452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe240007bd0 con 0x7fe254143ea0 2024-04-25T10:07:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe25ac05640 1 -- 172.21.15.133:0/1424714452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe254156980 con 0x7fe254143ea0 2024-04-25T10:07:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe2517fa640 1 -- 172.21.15.133:0/1424714452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2400113f0 con 0x7fe254143ea0 2024-04-25T10:07:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe25ac05640 1 -- 172.21.15.133:0/1424714452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe254155960 con 0x7fe254143ea0 2024-04-25T10:07:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe25ac05640 1 -- 172.21.15.133:0/1424714452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe254068e80 con 0x7fe254143ea0 2024-04-25T10:07:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe2517fa640 1 -- 172.21.15.133:0/1424714452 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe240011590 con 0x7fe254143ea0 2024-04-25T10:07:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe2517fa640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe238066f60 0x7fe238069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe2517fa640 1 -- 172.21.15.133:0/1424714452 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fe2400851f0 con 0x7fe254143ea0 2024-04-25T10:07:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe2589a1640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe238066f60 0x7fe238069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.213+0000 7fe2589a1640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe238066f60 0x7fe238069420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe2480b1000 tx=0x7fe248008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:30.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.217+0000 7fe2517fa640 1 -- 172.21.15.133:0/1424714452 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2541393e0 con 0x7fe254143ea0 2024-04-25T10:07:30.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:30.377+0000 7fe25ac05640 1 -- 172.21.15.133:0/1424714452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe254158440 con 0x7fe254143ea0 2024-04-25T10:07:31.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.061+0000 7fe2517fa640 1 -- 172.21.15.133:0/1424714452 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe240015020 con 0x7fe254143ea0 2024-04-25T10:07:31.066 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:07:31.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 -- 172.21.15.133:0/1424714452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe238066f60 msgr2=0x7fe238069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe238066f60 0x7fe238069420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe2480b1000 tx=0x7fe248008040 comp rx=0 tx=0).stop 2024-04-25T10:07:31.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 -- 172.21.15.133:0/1424714452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe254143ea0 msgr2=0x7fe254156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe254143ea0 0x7fe254156440 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe24000c7f0 tx=0x7fe24000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 -- 172.21.15.133:0/1424714452 shutdown_connections 2024-04-25T10:07:31.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe25415b9b0 0x7fe2541577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe238066f60 0x7fe238069420 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe254143ea0 0x7fe254156440 secure :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe24000c7f0 tx=0x7fe24000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 --2- 172.21.15.133:0/1424714452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe2540b2150 0x7fe254138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 -- 172.21.15.133:0/1424714452 >> 172.21.15.133:0/1424714452 conn(0x7fe254066f70 msgr2=0x7fe254158dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:31.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 -- 172.21.15.133:0/1424714452 shutdown_connections 2024-04-25T10:07:31.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.065+0000 7fe25ac05640 1 -- 172.21.15.133:0/1424714452 wait complete. 2024-04-25T10:07:31.101 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:07:31.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.217+0000 7f5b83533640 1 Processor -- start 2024-04-25T10:07:31.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.217+0000 7f5b83533640 1 -- start start 2024-04-25T10:07:31.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b83533640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5b7c0cd820 0x7f5b7c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b83533640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5b7c15ced0 0x7f5b7c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b83533640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c0ce2e0 con 0x7f5b7c145170 2024-04-25T10:07:31.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b83533640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7c06db00 con 0x7f5b7c0cd820 2024-04-25T10:07:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b83533640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7c145550 con 0x7f5b7c15ced0 2024-04-25T10:07:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b80ace640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5b7c0cd820 0x7f5b7c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b80ace640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5b7c0cd820 0x7f5b7c0cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40672/0 (socket says 172.21.15.133:40672) 2024-04-25T10:07:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b80ace640 1 -- 172.21.15.133:0/3371297063 learned_addr learned my addr 172.21.15.133:0/3371297063 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b81ad0640 1 --2- 172.21.15.133:0/3371297063 >> v2:172.21.15.162:3300/0 conn(0x7f5b7c15ced0 0x7f5b7c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b80ace640 1 -- 172.21.15.133:0/3371297063 >> v2:172.21.15.162:3300/0 conn(0x7f5b7c15ced0 msgr2=0x7f5b7c15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b80ace640 1 --2- 172.21.15.133:0/3371297063 >> v2:172.21.15.162:3300/0 conn(0x7f5b7c15ced0 0x7f5b7c15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b80ace640 1 -- 172.21.15.133:0/3371297063 >> v1:172.21.15.162:6789/0 conn(0x7f5b7c145170 legacy=0x7f5b7c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b80ace640 1 -- 172.21.15.133:0/3371297063 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c06d4b0 con 0x7f5b7c0cd820 2024-04-25T10:07:31.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.221+0000 7f5b80ace640 1 --2- 172.21.15.133:0/3371297063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7c0cd820 0x7f5b7c0cdca0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0b27f0 tx=0x7f5b6c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3c2f5ddfa48377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b73fff640 1 -- 172.21.15.133:0/3371297063 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0b0070 con 0x7f5b7c0cd820 2024-04-25T10:07:31.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b73fff640 1 -- 172.21.15.133:0/3371297063 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6c0ab600 con 0x7f5b7c0cd820 2024-04-25T10:07:31.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b73fff640 1 -- 172.21.15.133:0/3371297063 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0ab9d0 con 0x7f5b7c0cd820 2024-04-25T10:07:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 -- 172.21.15.133:0/3371297063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7c0cd820 msgr2=0x7f5b7c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 --2- 172.21.15.133:0/3371297063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7c0cd820 0x7f5b7c0cdca0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0b27f0 tx=0x7f5b6c01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 -- 172.21.15.133:0/3371297063 shutdown_connections 2024-04-25T10:07:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 --2- 172.21.15.133:0/3371297063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7c0cd820 0x7f5b7c0cdca0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 --2- 172.21.15.133:0/3371297063 >> v2:172.21.15.162:3300/0 conn(0x7f5b7c15ced0 0x7f5b7c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 -- 172.21.15.133:0/3371297063 >> 172.21.15.133:0/3371297063 conn(0x7f5b7c066f70 msgr2=0x7f5b7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 -- 172.21.15.133:0/3371297063 shutdown_connections 2024-04-25T10:07:31.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 -- 172.21.15.133:0/3371297063 wait complete. 2024-04-25T10:07:31.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 Processor -- start 2024-04-25T10:07:31.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 -- start start 2024-04-25T10:07:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7c0cd820 0x7f5b7c1f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b7c145170 0x7f5b7c133eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b7c15ced0 0x7f5b7c13ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c168610 con 0x7f5b7c0cd820 2024-04-25T10:07:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c168490 con 0x7f5b7c15ced0 2024-04-25T10:07:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c168790 con 0x7f5b7c145170 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b812cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7c0cd820 0x7f5b7c1f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b812cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7c0cd820 0x7f5b7c1f6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40678/0 (socket says 172.21.15.133:40678) 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b812cf640 1 -- 172.21.15.133:0/4181646758 learned_addr learned my addr 172.21.15.133:0/4181646758 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b81ad0640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b7c15ced0 0x7f5b7c13ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b80ace640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b7c145170 0x7f5b7c133eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b812cf640 1 -- 172.21.15.133:0/4181646758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b7c145170 msgr2=0x7f5b7c133eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b812cf640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b7c145170 0x7f5b7c133eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b812cf640 1 -- 172.21.15.133:0/4181646758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b7c15ced0 msgr2=0x7f5b7c13ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b812cf640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b7c15ced0 0x7f5b7c13ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b812cf640 1 -- 172.21.15.133:0/4181646758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c140de0 con 0x7f5b7c0cd820 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b812cf640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7c0cd820 0x7f5b7c1f6ae0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c450 tx=0x7f5b7800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b71ffb640 1 -- 172.21.15.133:0/4181646758 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78012830 con 0x7f5b7c0cd820 2024-04-25T10:07:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b71ffb640 1 -- 172.21.15.133:0/4181646758 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b78017070 con 0x7f5b7c0cd820 2024-04-25T10:07:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 -- 172.21.15.133:0/4181646758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c15f7c0 con 0x7f5b7c0cd820 2024-04-25T10:07:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b71ffb640 1 -- 172.21.15.133:0/4181646758 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78012c30 con 0x7f5b7c0cd820 2024-04-25T10:07:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.225+0000 7f5b83533640 1 -- 172.21.15.133:0/4181646758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c144070 con 0x7f5b7c0cd820 2024-04-25T10:07:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.229+0000 7f5b83533640 1 -- 172.21.15.133:0/4181646758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7c068e80 con 0x7f5b7c0cd820 2024-04-25T10:07:31.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.229+0000 7f5b71ffb640 1 -- 172.21.15.133:0/4181646758 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b7801c460 con 0x7f5b7c0cd820 2024-04-25T10:07:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.229+0000 7f5b71ffb640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b60066f60 0x7f5b60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.229+0000 7f5b71ffb640 1 -- 172.21.15.133:0/4181646758 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f5b780859a0 con 0x7f5b7c0cd820 2024-04-25T10:07:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.229+0000 7f5b80ace640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b60066f60 0x7f5b60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.229+0000 7f5b80ace640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b60066f60 0x7f5b60069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0ad000 tx=0x7f5b6c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.233+0000 7f5b71ffb640 1 -- 172.21.15.133:0/4181646758 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b78056960 con 0x7f5b7c0cd820 2024-04-25T10:07:31.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:31.429+0000 7f5b83533640 1 -- 172.21.15.133:0/4181646758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5b7c13f350 con 0x7f5b7c0cd820 2024-04-25T10:07:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.061+0000 7f5b71ffb640 1 -- 172.21.15.133:0/4181646758 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v46) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5b78056260 con 0x7f5b7c0cd820 2024-04-25T10:07:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T10:07:32.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.065+0000 7f5b83533640 1 -- 172.21.15.133:0/4181646758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b60066f60 msgr2=0x7f5b60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.065+0000 7f5b83533640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b60066f60 0x7f5b60069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0ad000 tx=0x7f5b6c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:07:32.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.065+0000 7f5b83533640 1 -- 172.21.15.133:0/4181646758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7c0cd820 msgr2=0x7f5b7c1f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.065+0000 7f5b83533640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7c0cd820 0x7f5b7c1f6ae0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c450 tx=0x7f5b7800c910 comp rx=0 tx=0).stop 2024-04-25T10:07:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.065+0000 7f5b83533640 1 -- 172.21.15.133:0/4181646758 shutdown_connections 2024-04-25T10:07:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.065+0000 7f5b83533640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b7c15ced0 0x7f5b7c13ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.065+0000 7f5b83533640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b7c145170 0x7f5b7c133eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.065+0000 7f5b83533640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b60066f60 0x7f5b60069420 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.065+0000 7f5b83533640 1 --2- 172.21.15.133:0/4181646758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b7c0cd820 0x7f5b7c1f6ae0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.065+0000 7f5b83533640 1 -- 172.21.15.133:0/4181646758 >> 172.21.15.133:0/4181646758 conn(0x7f5b7c066f70 msgr2=0x7f5b7c1418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.069+0000 7f5b83533640 1 -- 172.21.15.133:0/4181646758 shutdown_connections 2024-04-25T10:07:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.069+0000 7f5b83533640 1 -- 172.21.15.133:0/4181646758 wait complete. 2024-04-25T10:07:32.110 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.229+0000 7f358877f640 1 Processor -- start 2024-04-25T10:07:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.229+0000 7f358877f640 1 -- start start 2024-04-25T10:07:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f358877f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f358015dd90 0x7f35800ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f358877f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 0x7f35800cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f358877f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35800ca9b0 con 0x7f358015d9b0 2024-04-25T10:07:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f358877f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35800cbe30 con 0x7f358015dd90 2024-04-25T10:07:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f358877f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35800cac90 con 0x7f35800b19f0 2024-04-25T10:07:32.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f3585d1a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f358015dd90 0x7f35800ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f3586d1c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 0x7f35800cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f3586d1c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 0x7f35800cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:51572/0 (socket says 172.21.15.133:51572) 2024-04-25T10:07:32.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f3585d1a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f358015dd90 0x7f35800ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:37160/0 (socket says 172.21.15.133:37160) 2024-04-25T10:07:32.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f3586d1c640 1 -- 172.21.15.133:0/1444282833 learned_addr learned my addr 172.21.15.133:0/1444282833 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:32.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f3586d1c640 1 -- 172.21.15.133:0/1444282833 >> v2:172.21.15.162:3300/0 conn(0x7f358015dd90 msgr2=0x7f35800ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f3586d1c640 1 --2- 172.21.15.133:0/1444282833 >> v2:172.21.15.162:3300/0 conn(0x7f358015dd90 0x7f35800ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f3586d1c640 1 -- 172.21.15.133:0/1444282833 >> v1:172.21.15.162:6789/0 conn(0x7f358015d9b0 legacy=0x7f35800cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f3586d1c640 1 -- 172.21.15.133:0/1444282833 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35800caf70 con 0x7f35800b19f0 2024-04-25T10:07:32.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.233+0000 7f3585d1a640 1 --2- 172.21.15.133:0/1444282833 >> v2:172.21.15.162:3300/0 conn(0x7f358015dd90 0x7f35800ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:32.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f3586d1c640 1 --2- 172.21.15.133:0/1444282833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 0x7f35800cecb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3578009e50 tx=0x7f357801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4ab59ec088f1c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f3585519640 1 -- 172.21.15.133:0/1444282833 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35780a5880 con 0x7f35800b19f0 2024-04-25T10:07:32.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f3585519640 1 -- 172.21.15.133:0/1444282833 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35780a5a20 con 0x7f35800b19f0 2024-04-25T10:07:32.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f3585519640 1 -- 172.21.15.133:0/1444282833 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35780a5df0 con 0x7f35800b19f0 2024-04-25T10:07:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f358877f640 1 -- 172.21.15.133:0/1444282833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 msgr2=0x7f35800cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f358877f640 1 --2- 172.21.15.133:0/1444282833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 0x7f35800cecb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3578009e50 tx=0x7f357801c9a0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f358877f640 1 -- 172.21.15.133:0/1444282833 shutdown_connections 2024-04-25T10:07:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f358877f640 1 --2- 172.21.15.133:0/1444282833 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 0x7f35800cecb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f358877f640 1 --2- 172.21.15.133:0/1444282833 >> v2:172.21.15.162:3300/0 conn(0x7f358015dd90 0x7f35800ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f358877f640 1 -- 172.21.15.133:0/1444282833 >> 172.21.15.133:0/1444282833 conn(0x7f3580067000 msgr2=0x7f358006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f358877f640 1 -- 172.21.15.133:0/1444282833 shutdown_connections 2024-04-25T10:07:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.237+0000 7f358877f640 1 -- 172.21.15.133:0/1444282833 wait complete. 2024-04-25T10:07:32.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358877f640 1 Processor -- start 2024-04-25T10:07:32.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358877f640 1 -- start start 2024-04-25T10:07:32.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358877f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 0x7f3580145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358877f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358015d9b0 0x7f3580153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358877f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358015dd90 0x7f358015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358877f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358006e270 con 0x7f358015d9b0 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358877f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358006e0f0 con 0x7f358015dd90 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358877f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358006e3f0 con 0x7f35800b19f0 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3585d1a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358015d9b0 0x7f3580153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3585d1a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358015d9b0 0x7f3580153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40680/0 (socket says 172.21.15.133:40680) 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3585d1a640 1 -- 172.21.15.133:0/1051336117 learned_addr learned my addr 172.21.15.133:0/1051336117 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358651b640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 0x7f3580145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3586d1c640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358015dd90 0x7f358015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3585d1a640 1 -- 172.21.15.133:0/1051336117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 msgr2=0x7f3580145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3585d1a640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 0x7f3580145f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3585d1a640 1 -- 172.21.15.133:0/1051336117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358015dd90 msgr2=0x7f358015f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3585d1a640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358015dd90 0x7f358015f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3585d1a640 1 -- 172.21.15.133:0/1051336117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358015e310 con 0x7f358015d9b0 2024-04-25T10:07:32.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3585d1a640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358015d9b0 0x7f3580153740 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f357000e980 tx=0x7f357000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3576ffd640 1 -- 172.21.15.133:0/1051336117 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570017070 con 0x7f358015d9b0 2024-04-25T10:07:32.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3576ffd640 1 -- 172.21.15.133:0/1051336117 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357000c9a0 con 0x7f358015d9b0 2024-04-25T10:07:32.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358877f640 1 -- 172.21.15.133:0/1051336117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358015e5f0 con 0x7f358015d9b0 2024-04-25T10:07:32.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3576ffd640 1 -- 172.21.15.133:0/1051336117 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357000cd70 con 0x7f358015d9b0 2024-04-25T10:07:32.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358877f640 1 -- 172.21.15.133:0/1051336117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358015ebb0 con 0x7f358015d9b0 2024-04-25T10:07:32.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f358877f640 1 -- 172.21.15.133:0/1051336117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3580068ef0 con 0x7f358015d9b0 2024-04-25T10:07:32.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.241+0000 7f3576ffd640 1 -- 172.21.15.133:0/1051336117 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f357001a070 con 0x7f358015d9b0 2024-04-25T10:07:32.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.245+0000 7f3576ffd640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3564066f60 0x7f3564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.245+0000 7f3576ffd640 1 -- 172.21.15.133:0/1051336117 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3570085320 con 0x7f358015d9b0 2024-04-25T10:07:32.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.245+0000 7f358651b640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3564066f60 0x7f3564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.245+0000 7f358651b640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3564066f60 0x7f3564069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f358015fe60 tx=0x7f357c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.249+0000 7f3576ffd640 1 -- 172.21.15.133:0/1051336117 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3570055b50 con 0x7f358015d9b0 2024-04-25T10:07:32.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.409+0000 7f358877f640 1 -- 172.21.15.133:0/1051336117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35800b15e0 con 0x7f358015d9b0 2024-04-25T10:07:32.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.413+0000 7f3576ffd640 1 -- 172.21.15.133:0/1051336117 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16982 (secure 0 0 0) 0x7f3570059bf0 con 0x7f358015d9b0 2024-04-25T10:07:32.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:32.419 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":80,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:07:31.434262+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:29.693214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:30.421009+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:32.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 -- 172.21.15.133:0/1051336117 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3564066f60 msgr2=0x7f3564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3564066f60 0x7f3564069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f358015fe60 tx=0x7f357c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:32.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 -- 172.21.15.133:0/1051336117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358015d9b0 msgr2=0x7f3580153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358015d9b0 0x7f3580153740 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f357000e980 tx=0x7f357000ee40 comp rx=0 tx=0).stop 2024-04-25T10:07:32.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 -- 172.21.15.133:0/1051336117 shutdown_connections 2024-04-25T10:07:32.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358015dd90 0x7f358015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3564066f60 0x7f3564069420 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358015d9b0 0x7f3580153740 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 --2- 172.21.15.133:0/1051336117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f35800b19f0 0x7f3580145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 -- 172.21.15.133:0/1051336117 >> 172.21.15.133:0/1051336117 conn(0x7f3580067000 msgr2=0x7f35800711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 -- 172.21.15.133:0/1051336117 shutdown_connections 2024-04-25T10:07:32.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.417+0000 7f358877f640 1 -- 172.21.15.133:0/1051336117 wait complete. 2024-04-25T10:07:32.438 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:32.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.549+0000 7facbfa89640 1 Processor -- start 2024-04-25T10:07:32.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.549+0000 7facbfa89640 1 -- start start 2024-04-25T10:07:32.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbfa89640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7facb815dd90 0x7facb80ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbfa89640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb80ca9b0 con 0x7facb80b19f0 2024-04-25T10:07:32.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbfa89640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb80cac90 con 0x7facb815d9b0 2024-04-25T10:07:32.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbfa89640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb80cbe30 con 0x7facb815dd90 2024-04-25T10:07:32.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbe026640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7facb80b19f0 0x7facb806d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34126/0 (socket says 172.21.15.133:34126) 2024-04-25T10:07:32.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbe026640 1 -- 172.21.15.133:0/481850722 learned_addr learned my addr 172.21.15.133:0/481850722 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:32.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbd024640 1 --2- 172.21.15.133:0/481850722 >> v2:172.21.15.162:3300/0 conn(0x7facb815dd90 0x7facb80ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684780620 0 0) 0x7facb80ca9b0 con 0x7facb80b19f0 2024-04-25T10:07:32.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faca4003530 con 0x7facb80b19f0 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1557915044 0 0) 0x7faca4003530 con 0x7facb80b19f0 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 >> v2:172.21.15.162:3300/0 conn(0x7facb815dd90 msgr2=0x7facb80ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 --2- 172.21.15.133:0/481850722 >> v2:172.21.15.162:3300/0 conn(0x7facb815dd90 0x7facb80ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 >> v1:172.21.15.162:6789/0 conn(0x7facb815d9b0 legacy=0x7facb80cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb80caf70 con 0x7facb80b19f0 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7facb0001a00 con 0x7facb80b19f0 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 >> v1:172.21.15.133:6789/0 conn(0x7facb80b19f0 legacy=0x7facb806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faca40039c0 0x7faca4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca4004810 0x7faca4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca40051d0 0x7faca400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb80cbe30 con 0x7faca4004810 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca400c170 con 0x7faca40039c0 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca400c370 con 0x7faca40051d0 2024-04-25T10:07:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbd024640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca4004810 0x7faca4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbd825640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faca40039c0 0x7faca4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbe026640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca40051d0 0x7faca400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbd024640 1 -- 172.21.15.133:0/481850722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca40051d0 msgr2=0x7faca400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbd024640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca40051d0 0x7faca400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbd024640 1 -- 172.21.15.133:0/481850722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faca40039c0 msgr2=0x7faca4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbd024640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faca40039c0 0x7faca4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbd024640 1 -- 172.21.15.133:0/481850722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca4003530 con 0x7faca4004810 2024-04-25T10:07:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbe026640 1 -- 172.21.15.133:0/481850722 reap_dead start 2024-04-25T10:07:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbd825640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faca40039c0 0x7faca4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:32.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbd024640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca4004810 0x7faca4004c90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7faca8009ae0 tx=0x7faca801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db59f815f1f869d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca80b2070 con 0x7faca4004810 2024-04-25T10:07:32.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca80a5570 con 0x7faca4004810 2024-04-25T10:07:32.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.553+0000 7facbc823640 1 -- 172.21.15.133:0/481850722 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca80096b0 con 0x7faca4004810 2024-04-25T10:07:32.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 -- 172.21.15.133:0/481850722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca4004810 msgr2=0x7faca4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca4004810 0x7faca4004c90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7faca8009ae0 tx=0x7faca801ca90 comp rx=0 tx=0).stop 2024-04-25T10:07:32.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 -- 172.21.15.133:0/481850722 shutdown_connections 2024-04-25T10:07:32.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 --2- 172.21.15.133:0/481850722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca4004810 0x7faca4004c90 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 -- 172.21.15.133:0/481850722 >> 172.21.15.133:0/481850722 conn(0x7facb8067000 msgr2=0x7facb806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 -- 172.21.15.133:0/481850722 shutdown_connections 2024-04-25T10:07:32.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 -- 172.21.15.133:0/481850722 wait complete. 2024-04-25T10:07:32.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 Processor -- start 2024-04-25T10:07:32.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 -- start start 2024-04-25T10:07:32.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca4004810 0x7facb8144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facb81ec470 0x7facb81ec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb815b480 0x7facb815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb8161a80 con 0x7facb81ec470 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb8161900 con 0x7facb815b480 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb8161c00 con 0x7faca4004810 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbd024640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facb81ec470 0x7facb81ec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbd024640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facb81ec470 0x7facb81ec8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40690/0 (socket says 172.21.15.133:40690) 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbd024640 1 -- 172.21.15.133:0/3297479923 learned_addr learned my addr 172.21.15.133:0/3297479923 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbd825640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca4004810 0x7facb8144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbe026640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb815b480 0x7facb815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbd024640 1 -- 172.21.15.133:0/3297479923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca4004810 msgr2=0x7facb8144500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbd024640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca4004810 0x7facb8144500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbd024640 1 -- 172.21.15.133:0/3297479923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb815b480 msgr2=0x7facb815b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbd024640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb815b480 0x7facb815b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbd024640 1 -- 172.21.15.133:0/3297479923 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb8144f00 con 0x7facb81ec470 2024-04-25T10:07:32.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbd024640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facb81ec470 0x7facb81ec8f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7faca801c8c0 tx=0x7faca8019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facae7fc640 1 -- 172.21.15.133:0/3297479923 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca80b2040 con 0x7facb81ec470 2024-04-25T10:07:32.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facae7fc640 1 -- 172.21.15.133:0/3297479923 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca800f000 con 0x7facb81ec470 2024-04-25T10:07:32.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facae7fc640 1 -- 172.21.15.133:0/3297479923 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca8004260 con 0x7facb81ec470 2024-04-25T10:07:32.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 -- 172.21.15.133:0/3297479923 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb81ece30 con 0x7facb81ec470 2024-04-25T10:07:32.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.557+0000 7facbfa89640 1 -- 172.21.15.133:0/3297479923 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb8145450 con 0x7facb81ec470 2024-04-25T10:07:32.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.561+0000 7facbfa89640 1 -- 172.21.15.133:0/3297479923 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb8068ef0 con 0x7facb81ec470 2024-04-25T10:07:32.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.561+0000 7facae7fc640 1 -- 172.21.15.133:0/3297479923 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faca80bf030 con 0x7facb81ec470 2024-04-25T10:07:32.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.561+0000 7facae7fc640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac9c066f90 0x7fac9c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.561+0000 7facae7fc640 1 -- 172.21.15.133:0/3297479923 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faca811ef00 con 0x7facb81ec470 2024-04-25T10:07:32.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.561+0000 7facbd825640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac9c066f90 0x7fac9c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.561+0000 7facbd825640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac9c066f90 0x7fac9c069450 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7facb4007fd0 tx=0x7facb4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.565+0000 7facae7fc640 1 -- 172.21.15.133:0/3297479923 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faca80c6060 con 0x7facb81ec470 2024-04-25T10:07:32.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.725+0000 7facbfa89640 1 -- 172.21.15.133:0/3297479923 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facb81e9ab0 con 0x7facb81ec470 2024-04-25T10:07:32.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.725+0000 7facae7fc640 1 -- 172.21.15.133:0/3297479923 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16982 (secure 0 0 0) 0x7faca80167a0 con 0x7facb81ec470 2024-04-25T10:07:32.732 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:32.732 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":80,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:07:31.434262+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:29.693214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:30.421009+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:32.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 -- 172.21.15.133:0/3297479923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac9c066f90 msgr2=0x7fac9c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac9c066f90 0x7fac9c069450 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7facb4007fd0 tx=0x7facb4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:32.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 -- 172.21.15.133:0/3297479923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facb81ec470 msgr2=0x7facb81ec8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facb81ec470 0x7facb81ec8f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7faca801c8c0 tx=0x7faca8019830 comp rx=0 tx=0).stop 2024-04-25T10:07:32.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 -- 172.21.15.133:0/3297479923 shutdown_connections 2024-04-25T10:07:32.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7facb815b480 0x7facb815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac9c066f90 0x7fac9c069450 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facb81ec470 0x7facb81ec8f0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 --2- 172.21.15.133:0/3297479923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faca4004810 0x7facb8144500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 -- 172.21.15.133:0/3297479923 >> 172.21.15.133:0/3297479923 conn(0x7facb8067000 msgr2=0x7facb80b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 -- 172.21.15.133:0/3297479923 shutdown_connections 2024-04-25T10:07:32.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.729+0000 7facbfa89640 1 -- 172.21.15.133:0/3297479923 wait complete. 2024-04-25T10:07:32.751 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:07:32.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.869+0000 7fab601f5640 1 Processor -- start 2024-04-25T10:07:32.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.869+0000 7fab601f5640 1 -- start start 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab601f5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b2150 0x7fab5806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab601f5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5815ac50 con 0x7fab5815b9b0 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab601f5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab580cd180 con 0x7fab58143ea0 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab601f5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab580ccf20 con 0x7fab580b2150 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5e792640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fab5815b9b0 0x7fab580b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34134/0 (socket says 172.21.15.133:34134) 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5e792640 1 -- 172.21.15.133:0/497320586 learned_addr learned my addr 172.21.15.133:0/497320586 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5d790640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b2150 0x7fab5806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752673221 0 0) 0x7fab500014a0 con 0x7fab5815b9b0 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab44003530 con 0x7fab5815b9b0 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4283026531 0 0) 0x7fab44003530 con 0x7fab5815b9b0 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b2150 msgr2=0x7fab5806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b2150 0x7fab5806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 >> v1:172.21.15.162:6789/0 conn(0x7fab58143ea0 legacy=0x7fab580cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5815aed0 con 0x7fab5815b9b0 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fab50001bd0 con 0x7fab5815b9b0 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 >> v1:172.21.15.133:6789/0 conn(0x7fab5815b9b0 legacy=0x7fab580b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b2150 0x7fab440039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab44003f70 0x7fab44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab44004d50 0x7fab440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab580ccf20 con 0x7fab44004d50 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4400bab0 con 0x7fab44003f70 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4400bcb0 con 0x7fab580b2150 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5e792640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab44004d50 0x7fab440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5d790640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab44003f70 0x7fab44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5df91640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b2150 0x7fab440039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5e792640 1 -- 172.21.15.133:0/497320586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b2150 msgr2=0x7fab440039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5e792640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b2150 0x7fab440039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5e792640 1 -- 172.21.15.133:0/497320586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab44003f70 msgr2=0x7fab44004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5e792640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab44003f70 0x7fab44004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.873+0000 7fab5e792640 1 -- 172.21.15.133:0/497320586 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab44003530 con 0x7fab44004d50 2024-04-25T10:07:32.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab5e792640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab44004d50 0x7fab440051d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fab5815bdc0 tx=0x7fab5001d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d252c7e32a9cad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000a650 con 0x7fab44004d50 2024-04-25T10:07:32.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab500ad650 con 0x7fab44004d50 2024-04-25T10:07:32.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab5cf8f640 1 -- 172.21.15.133:0/497320586 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab500ada80 con 0x7fab44004d50 2024-04-25T10:07:32.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab601f5640 1 -- 172.21.15.133:0/497320586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab44004d50 msgr2=0x7fab440051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab601f5640 1 --2- 172.21.15.133:0/497320586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab44004d50 0x7fab440051d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fab5815bdc0 tx=0x7fab5001d320 comp rx=0 tx=0).stop 2024-04-25T10:07:32.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab5e792640 1 -- 172.21.15.133:0/497320586 reap_dead start 2024-04-25T10:07:32.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab601f5640 1 -- 172.21.15.133:0/497320586 shutdown_connections 2024-04-25T10:07:32.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab601f5640 1 -- 172.21.15.133:0/497320586 >> 172.21.15.133:0/497320586 conn(0x7fab58066f70 msgr2=0x7fab5806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab601f5640 1 -- 172.21.15.133:0/497320586 shutdown_connections 2024-04-25T10:07:32.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab601f5640 1 -- 172.21.15.133:0/497320586 wait complete. 2024-04-25T10:07:32.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab601f5640 1 Processor -- start 2024-04-25T10:07:32.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.877+0000 7fab601f5640 1 -- start start 2024-04-25T10:07:32.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab601f5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5813b260 0x7fab580ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab601f5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab580ca6c0 0x7fab580b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab601f5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b3e70 0x7fab581f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab601f5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab58144280 con 0x7fab580ca6c0 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab601f5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5815b490 con 0x7fab5813b260 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab601f5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab58144400 con 0x7fab580b3e70 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5d790640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab580ca6c0 0x7fab580b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5d790640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab580ca6c0 0x7fab580b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40714/0 (socket says 172.21.15.133:40714) 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5d790640 1 -- 172.21.15.133:0/86496832 learned_addr learned my addr 172.21.15.133:0/86496832 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5e792640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b3e70 0x7fab581f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5df91640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5813b260 0x7fab580ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5d790640 1 -- 172.21.15.133:0/86496832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b3e70 msgr2=0x7fab581f6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5d790640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b3e70 0x7fab581f6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5d790640 1 -- 172.21.15.133:0/86496832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5813b260 msgr2=0x7fab580ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5d790640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5813b260 0x7fab580ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5d790640 1 -- 172.21.15.133:0/86496832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab581f3460 con 0x7fab580ca6c0 2024-04-25T10:07:32.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5df91640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5813b260 0x7fab580ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:32.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5d790640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab580ca6c0 0x7fab580b3930 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fab480115e0 tx=0x7fab48011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab4e7fc640 1 -- 172.21.15.133:0/86496832 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48015870 con 0x7fab580ca6c0 2024-04-25T10:07:32.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab4e7fc640 1 -- 172.21.15.133:0/86496832 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab48015a10 con 0x7fab580ca6c0 2024-04-25T10:07:32.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab601f5640 1 -- 172.21.15.133:0/86496832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab581f3740 con 0x7fab580ca6c0 2024-04-25T10:07:32.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab4e7fc640 1 -- 172.21.15.133:0/86496832 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48015dc0 con 0x7fab580ca6c0 2024-04-25T10:07:32.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab601f5640 1 -- 172.21.15.133:0/86496832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab581f4220 con 0x7fab580ca6c0 2024-04-25T10:07:32.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab601f5640 1 -- 172.21.15.133:0/86496832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab580b4280 con 0x7fab580ca6c0 2024-04-25T10:07:32.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab4e7fc640 1 -- 172.21.15.133:0/86496832 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab4800f480 con 0x7fab580ca6c0 2024-04-25T10:07:32.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab4e7fc640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab3c066f90 0x7fab3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab4e7fc640 1 -- 172.21.15.133:0/86496832 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab48085a70 con 0x7fab580ca6c0 2024-04-25T10:07:32.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5df91640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab3c066f90 0x7fab3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.881+0000 7fab5df91640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab3c066f90 0x7fab3c069450 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fab540016e0 tx=0x7fab54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:32.885+0000 7fab4e7fc640 1 -- 172.21.15.133:0/86496832 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab580b4280 con 0x7fab580ca6c0 2024-04-25T10:07:33.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:33.081+0000 7fab601f5640 1 -- 172.21.15.133:0/86496832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fab58143500 con 0x7fab580ca6c0 2024-04-25T10:07:34.057 INFO:tasks.ceph.mds.d.smithi162.stderr:2024-04-25T10:07:34.050+0000 7f919fb4c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:34.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.097+0000 7fab4e7fc640 1 -- 172.21.15.133:0/86496832 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) v1 ==== 118+0+0 (secure 0 0 0) 0x7fab48019030 con 0x7fab580ca6c0 2024-04-25T10:07:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 -- 172.21.15.133:0/86496832 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab3c066f90 msgr2=0x7fab3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab3c066f90 0x7fab3c069450 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fab540016e0 tx=0x7fab54009040 comp rx=0 tx=0).stop 2024-04-25T10:07:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 -- 172.21.15.133:0/86496832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab580ca6c0 msgr2=0x7fab580b3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab580ca6c0 0x7fab580b3930 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fab480115e0 tx=0x7fab48011aa0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 -- 172.21.15.133:0/86496832 shutdown_connections 2024-04-25T10:07:34.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab580b3e70 0x7fab581f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab3c066f90 0x7fab3c069450 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab580ca6c0 0x7fab580b3930 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 --2- 172.21.15.133:0/86496832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5813b260 0x7fab580ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 -- 172.21.15.133:0/86496832 >> 172.21.15.133:0/86496832 conn(0x7fab58066f70 msgr2=0x7fab58159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 -- 172.21.15.133:0/86496832 shutdown_connections 2024-04-25T10:07:34.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.101+0000 7fab601f5640 1 -- 172.21.15.133:0/86496832 wait complete. 2024-04-25T10:07:34.128 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:07:34.128 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:07:34.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.249+0000 7fd4fbbc0640 1 Processor -- start 2024-04-25T10:07:34.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.249+0000 7fd4fbbc0640 1 -- start start 2024-04-25T10:07:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4fbbc0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4f4145170 0x7fd4f406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4fbbc0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f406e4d0 0x7fd4f40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4fbbc0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f40ce2e0 con 0x7fd4f40cdd60 2024-04-25T10:07:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4fbbc0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f406db00 con 0x7fd4f4145170 2024-04-25T10:07:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4fbbc0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4145550 con 0x7fd4f406e4d0 2024-04-25T10:07:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4fa15d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd4f40cdd60 0x7fd4f415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34148/0 (socket says 172.21.15.133:34148) 2024-04-25T10:07:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4fa15d640 1 -- 172.21.15.133:0/1901323526 learned_addr learned my addr 172.21.15.133:0/1901323526 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f995c640 1 --2- 172.21.15.133:0/1901323526 >> v2:172.21.15.162:3300/0 conn(0x7fd4f4145170 0x7fd4f406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f915b640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f406e4d0 0x7fd4f40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029551123 0 0) 0x7fd4f40ce2e0 con 0x7fd4f40cdd60 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e0003530 con 0x7fd4f40cdd60 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1799990176 0 0) 0x7fd4e0003530 con 0x7fd4f40cdd60 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f406e4d0 msgr2=0x7fd4f40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f406e4d0 0x7fd4f40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 >> v2:172.21.15.162:3300/0 conn(0x7fd4f4145170 msgr2=0x7fd4f406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 --2- 172.21.15.133:0/1901323526 >> v2:172.21.15.162:3300/0 conn(0x7fd4f4145170 0x7fd4f406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f406d4b0 con 0x7fd4f40cdd60 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fd4ec001a00 con 0x7fd4f40cdd60 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 >> v1:172.21.15.133:6789/0 conn(0x7fd4f40cdd60 legacy=0x7fd4f415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4e00039c0 0x7fd4e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e0004810 0x7fd4e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f406e4d0 0x7fd4e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f406db00 con 0x7fd4e00039c0 2024-04-25T10:07:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4145550 con 0x7fd4e0004810 2024-04-25T10:07:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e000b9d0 con 0x7fd4f406e4d0 2024-04-25T10:07:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f995c640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4e00039c0 0x7fd4e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f915b640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e0004810 0x7fd4e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4fa15d640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f406e4d0 0x7fd4e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f995c640 1 -- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f406e4d0 msgr2=0x7fd4e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f995c640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f406e4d0 0x7fd4e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f995c640 1 -- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e0004810 msgr2=0x7fd4e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f995c640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4e0004810 0x7fd4e0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4f995c640 1 -- 172.21.15.133:0/1901323526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0003530 con 0x7fd4e00039c0 2024-04-25T10:07:34.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.253+0000 7fd4fa15d640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4f406e4d0 0x7fd4e00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4f995c640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4e00039c0 0x7fd4e0003de0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009ae0 tx=0x7fd4f001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb31cb58779dcf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00096b0 con 0x7fd4e00039c0 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f00a8410 con 0x7fd4e00039c0 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4f895a640 1 -- 172.21.15.133:0/1901323526 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00a8800 con 0x7fd4e00039c0 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1901323526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4e00039c0 msgr2=0x7fd4e0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 --2- 172.21.15.133:0/1901323526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4e00039c0 0x7fd4e0003de0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009ae0 tx=0x7fd4f001cab0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fa15d640 1 -- 172.21.15.133:0/1901323526 reap_dead start 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1901323526 shutdown_connections 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1901323526 >> 172.21.15.133:0/1901323526 conn(0x7fd4f4066f70 msgr2=0x7fd4f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1901323526 shutdown_connections 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1901323526 wait complete. 2024-04-25T10:07:34.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 Processor -- start 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 -- start start 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e00039c0 0x7fd4f4136890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4f4161460 0x7fd4f413da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f413df70 0x7fd4f41f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4166480 con 0x7fd4f413df70 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f415d7f0 con 0x7fd4f4161460 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fbbc0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4166600 con 0x7fd4e00039c0 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fa15d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f413df70 0x7fd4f41f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fa15d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f413df70 0x7fd4f41f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40732/0 (socket says 172.21.15.133:40732) 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fa15d640 1 -- 172.21.15.133:0/1893548278 learned_addr learned my addr 172.21.15.133:0/1893548278 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4f915b640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4f4161460 0x7fd4f413da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4f995c640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e00039c0 0x7fd4f4136890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fa15d640 1 -- 172.21.15.133:0/1893548278 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e00039c0 msgr2=0x7fd4f4136890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fa15d640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e00039c0 0x7fd4f4136890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fa15d640 1 -- 172.21.15.133:0/1893548278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4f4161460 msgr2=0x7fd4f413da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fa15d640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4f4161460 0x7fd4f413da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fa15d640 1 -- 172.21.15.133:0/1893548278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f415fec0 con 0x7fd4f413df70 2024-04-25T10:07:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.257+0000 7fd4fa15d640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f413df70 0x7fd4f41f4830 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0018a0 tx=0x7fd4ec00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.261+0000 7fd4ea7fc640 1 -- 172.21.15.133:0/1893548278 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec00bbb0 con 0x7fd4f413df70 2024-04-25T10:07:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.261+0000 7fd4ea7fc640 1 -- 172.21.15.133:0/1893548278 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec00bd70 con 0x7fd4f413df70 2024-04-25T10:07:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.261+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1893548278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f413e570 con 0x7fd4f413df70 2024-04-25T10:07:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.261+0000 7fd4ea7fc640 1 -- 172.21.15.133:0/1893548278 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0072b0 con 0x7fd4f413df70 2024-04-25T10:07:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.261+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1893548278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f415fbe0 con 0x7fd4f413df70 2024-04-25T10:07:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.261+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1893548278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f4068e80 con 0x7fd4f413df70 2024-04-25T10:07:34.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.261+0000 7fd4ea7fc640 1 -- 172.21.15.133:0/1893548278 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4ec01b020 con 0x7fd4f413df70 2024-04-25T10:07:34.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.261+0000 7fd4ea7fc640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4d8066f90 0x7fd4d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.261+0000 7fd4ea7fc640 1 -- 172.21.15.133:0/1893548278 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4ec085420 con 0x7fd4f413df70 2024-04-25T10:07:34.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.261+0000 7fd4f995c640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4d8066f90 0x7fd4d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.261+0000 7fd4f995c640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4d8066f90 0x7fd4d8069450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00af000 tx=0x7fd4f00b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.265+0000 7fd4ea7fc640 1 -- 172.21.15.133:0/1893548278 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4ec0562a0 con 0x7fd4f413df70 2024-04-25T10:07:34.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:34.461+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1893548278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd4f4161b30 con 0x7fd4f413df70 2024-04-25T10:07:35.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.105+0000 7fd4ea7fc640 1 -- 172.21.15.133:0/1893548278 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd4ec055ba0 con 0x7fd4f413df70 2024-04-25T10:07:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1893548278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4d8066f90 msgr2=0x7fd4d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4d8066f90 0x7fd4d8069450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00af000 tx=0x7fd4f00b2040 comp rx=0 tx=0).stop 2024-04-25T10:07:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1893548278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f413df70 msgr2=0x7fd4f41f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f413df70 0x7fd4f41f4830 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0018a0 tx=0x7fd4ec00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1893548278 shutdown_connections 2024-04-25T10:07:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd4d8066f90 0x7fd4d8069450 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f413df70 0x7fd4f41f4830 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4f4161460 0x7fd4f413da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 --2- 172.21.15.133:0/1893548278 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4e00039c0 0x7fd4f4136890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1893548278 >> 172.21.15.133:0/1893548278 conn(0x7fd4f4066f70 msgr2=0x7fd4f40b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1893548278 shutdown_connections 2024-04-25T10:07:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.109+0000 7fd4fbbc0640 1 -- 172.21.15.133:0/1893548278 wait complete. 2024-04-25T10:07:35.134 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:07:35.134 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:35.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.253+0000 7fb44b7ff640 1 Processor -- start 2024-04-25T10:07:35.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.253+0000 7fb44b7ff640 1 -- start start 2024-04-25T10:07:35.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb44b7ff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4440ced30 0x7fb444168950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb44b7ff640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb444161e70 con 0x7fb44415d910 2024-04-25T10:07:35.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb44b7ff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb444162150 con 0x7fb44415dcf0 2024-04-25T10:07:35.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb44b7ff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb444162430 con 0x7fb4440ced30 2024-04-25T10:07:35.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb44959b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb44415d910 0x7fb444161620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34160/0 (socket says 172.21.15.133:34160) 2024-04-25T10:07:35.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb44959b640 1 -- 172.21.15.133:0/2082863465 learned_addr learned my addr 172.21.15.133:0/2082863465 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:35.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303122935 0 0) 0x7fb444161e70 con 0x7fb44415d910 2024-04-25T10:07:35.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb430003530 con 0x7fb44415d910 2024-04-25T10:07:35.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb449d9c640 1 --2- 172.21.15.133:0/2082863465 >> v2:172.21.15.162:3300/0 conn(0x7fb4440ced30 0x7fb444168950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4223615874 0 0) 0x7fb430003530 con 0x7fb44415d910 2024-04-25T10:07:35.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 >> v2:172.21.15.162:3300/0 conn(0x7fb4440ced30 msgr2=0x7fb444168950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 --2- 172.21.15.133:0/2082863465 >> v2:172.21.15.162:3300/0 conn(0x7fb4440ced30 0x7fb444168950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 >> v1:172.21.15.162:6789/0 conn(0x7fb44415dcf0 legacy=0x7fb4440ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4440d1d20 con 0x7fb44415d910 2024-04-25T10:07:35.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb440001680 con 0x7fb44415d910 2024-04-25T10:07:35.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb449d9c640 1 --2- 172.21.15.133:0/2082863465 >> v2:172.21.15.162:3300/0 conn(0x7fb4440ced30 0x7fb444168950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:35.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 >> v1:172.21.15.133:6789/0 conn(0x7fb44415d910 legacy=0x7fb444161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4300039c0 0x7fb430003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb430004810 0x7fb430004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 0x7fb43000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb444162430 con 0x7fb4300051d0 2024-04-25T10:07:35.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43000c100 con 0x7fb430004810 2024-04-25T10:07:35.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43000c300 con 0x7fb4300039c0 2024-04-25T10:07:35.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb449d9c640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 0x7fb43000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb44959b640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4300039c0 0x7fb430003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb448d9a640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb430004810 0x7fb430004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb449d9c640 1 -- 172.21.15.133:0/2082863465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4300039c0 msgr2=0x7fb430003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb449d9c640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4300039c0 0x7fb430003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb449d9c640 1 -- 172.21.15.133:0/2082863465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb430004810 msgr2=0x7fb430004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb449d9c640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb430004810 0x7fb430004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb449d9c640 1 -- 172.21.15.133:0/2082863465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb430003530 con 0x7fb4300051d0 2024-04-25T10:07:35.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.257+0000 7fb448d9a640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb430004810 0x7fb430004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:35.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb449d9c640 1 -- 172.21.15.133:0/2082863465 reap_dead start 2024-04-25T10:07:35.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb449d9c640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 0x7fb43000b9d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0073b0 tx=0x7fb43c01cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542e3fdd800e2b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c009c80 con 0x7fb4300051d0 2024-04-25T10:07:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c0a5e20 con 0x7fb4300051d0 2024-04-25T10:07:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb43bfff640 1 -- 172.21.15.133:0/2082863465 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0ad680 con 0x7fb4300051d0 2024-04-25T10:07:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 -- 172.21.15.133:0/2082863465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 msgr2=0x7fb43000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 0x7fb43000b9d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0073b0 tx=0x7fb43c01cf20 comp rx=0 tx=0).stop 2024-04-25T10:07:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 -- 172.21.15.133:0/2082863465 shutdown_connections 2024-04-25T10:07:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 --2- 172.21.15.133:0/2082863465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 0x7fb43000b9d0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 -- 172.21.15.133:0/2082863465 >> 172.21.15.133:0/2082863465 conn(0x7fb4440d03d0 msgr2=0x7fb4440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 -- 172.21.15.133:0/2082863465 shutdown_connections 2024-04-25T10:07:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 -- 172.21.15.133:0/2082863465 wait complete. 2024-04-25T10:07:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 Processor -- start 2024-04-25T10:07:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 -- start start 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 0x7fb444160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb44415eaa0 0x7fb444152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4441e9500 0x7fb4441e9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440b92e0 con 0x7fb4300051d0 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440b6af0 con 0x7fb44415eaa0 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440b9460 con 0x7fb4441e9500 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44959b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 0x7fb444160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44959b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 0x7fb444160c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40738/0 (socket says 172.21.15.133:40738) 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44959b640 1 -- 172.21.15.133:0/484275788 learned_addr learned my addr 172.21.15.133:0/484275788 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb449d9c640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4441e9500 0x7fb4441e9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb448d9a640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb44415eaa0 0x7fb444152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44959b640 1 -- 172.21.15.133:0/484275788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4441e9500 msgr2=0x7fb4441e9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44959b640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4441e9500 0x7fb4441e9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44959b640 1 -- 172.21.15.133:0/484275788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb44415eaa0 msgr2=0x7fb444152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44959b640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb44415eaa0 0x7fb444152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44959b640 1 -- 172.21.15.133:0/484275788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44415f2f0 con 0x7fb4300051d0 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb449d9c640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4441e9500 0x7fb4441e9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb448d9a640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb44415eaa0 0x7fb444152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44959b640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 0x7fb444160c50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb4400017f0 tx=0x7fb44000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb439ffb640 1 -- 172.21.15.133:0/484275788 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44000ee60 con 0x7fb4300051d0 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb439ffb640 1 -- 172.21.15.133:0/484275788 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb440018070 con 0x7fb4300051d0 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 -- 172.21.15.133:0/484275788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44415f5d0 con 0x7fb4300051d0 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb439ffb640 1 -- 172.21.15.133:0/484275788 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44000caf0 con 0x7fb4300051d0 2024-04-25T10:07:35.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.261+0000 7fb44b7ff640 1 -- 172.21.15.133:0/484275788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44415fb90 con 0x7fb4300051d0 2024-04-25T10:07:35.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.265+0000 7fb44b7ff640 1 -- 172.21.15.133:0/484275788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb444068ef0 con 0x7fb4300051d0 2024-04-25T10:07:35.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.265+0000 7fb439ffb640 1 -- 172.21.15.133:0/484275788 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb44000cc90 con 0x7fb4300051d0 2024-04-25T10:07:35.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.265+0000 7fb439ffb640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb428066f90 0x7fb428069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.265+0000 7fb439ffb640 1 -- 172.21.15.133:0/484275788 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb440084cc0 con 0x7fb4300051d0 2024-04-25T10:07:35.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.265+0000 7fb448d9a640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb428066f90 0x7fb428069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.265+0000 7fb448d9a640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb428066f90 0x7fb428069450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb4440d44d0 tx=0x7fb434009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.269+0000 7fb439ffb640 1 -- 172.21.15.133:0/484275788 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb440055b40 con 0x7fb4300051d0 2024-04-25T10:07:35.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.457+0000 7fb44b7ff640 1 -- 172.21.15.133:0/484275788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4440d15f0 con 0x7fb4300051d0 2024-04-25T10:07:35.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.457+0000 7fb439ffb640 1 -- 172.21.15.133:0/484275788 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fb440055440 con 0x7fb4300051d0 2024-04-25T10:07:35.462 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:35.463 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3522294955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3522294955},{"type":"v1","addr":"172.21.15.162:6833","nonce":3522294955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3865231260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3865231260},{"type":"v1","addr":"172.21.15.133:6836","nonce":3865231260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7437,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3935311931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3935311931},{"type":"v1","addr":"172.21.15.133:6837","nonce":3935311931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:31.434780+0000","modified":"2024-04-25T10:07:34.465411+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/3192312240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3192312240},{"type":"v1","addr":"172.21.15.162:6835","nonce":3192312240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:35.463 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 49 2024-04-25T10:07:35.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 -- 172.21.15.133:0/484275788 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb428066f90 msgr2=0x7fb428069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb428066f90 0x7fb428069450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb4440d44d0 tx=0x7fb434009040 comp rx=0 tx=0).stop 2024-04-25T10:07:35.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 -- 172.21.15.133:0/484275788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 msgr2=0x7fb444160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 0x7fb444160c50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb4400017f0 tx=0x7fb44000ec10 comp rx=0 tx=0).stop 2024-04-25T10:07:35.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 -- 172.21.15.133:0/484275788 shutdown_connections 2024-04-25T10:07:35.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4441e9500 0x7fb4441e9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb44415eaa0 0x7fb444152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb428066f90 0x7fb428069450 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 --2- 172.21.15.133:0/484275788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4300051d0 0x7fb444160c50 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 -- 172.21.15.133:0/484275788 >> 172.21.15.133:0/484275788 conn(0x7fb4440d03d0 msgr2=0x7fb4440ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 -- 172.21.15.133:0/484275788 shutdown_connections 2024-04-25T10:07:35.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.461+0000 7fb44b7ff640 1 -- 172.21.15.133:0/484275788 wait complete. 2024-04-25T10:07:35.480 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:35.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad797de640 1 Processor -- start 2024-04-25T10:07:35.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad797de640 1 -- start start 2024-04-25T10:07:35.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad797de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad74161050 0x7fad740999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad797de640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fad740b3ea0 0x7fad74099f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad797de640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad740b06d0 con 0x7fad7409a450 2024-04-25T10:07:35.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad797de640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad740d0480 con 0x7fad740b3ea0 2024-04-25T10:07:35.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad797de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad740d0760 con 0x7fad74161050 2024-04-25T10:07:35.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad727fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fad740b3ea0 0x7fad74099f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad72ffd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad74161050 0x7fad740999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad727fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fad740b3ea0 0x7fad74099f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:37272/0 (socket says 172.21.15.133:37272) 2024-04-25T10:07:35.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad737fe640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fad7409a450 0x7fad7406bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:47672/0 (socket says 172.21.15.133:47672) 2024-04-25T10:07:35.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad727fc640 1 -- 172.21.15.133:0/498896696 learned_addr learned my addr 172.21.15.133:0/498896696 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:35.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad72ffd640 1 -- 172.21.15.133:0/498896696 >> v2:172.21.15.162:3300/0 conn(0x7fad740b3ea0 msgr2=0x7fad74099f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad72ffd640 1 --2- 172.21.15.133:0/498896696 >> v2:172.21.15.162:3300/0 conn(0x7fad740b3ea0 0x7fad74099f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad72ffd640 1 -- 172.21.15.133:0/498896696 >> v1:172.21.15.162:6789/0 conn(0x7fad7409a450 legacy=0x7fad7406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.589+0000 7fad72ffd640 1 -- 172.21.15.133:0/498896696 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad74161430 con 0x7fad74161050 2024-04-25T10:07:35.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad72ffd640 1 --2- 172.21.15.133:0/498896696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad74161050 0x7fad740999d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fad640097c0 tx=0x7fad6401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d09a50c443f34cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad71ffb640 1 -- 172.21.15.133:0/498896696 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad640b0070 con 0x7fad74161050 2024-04-25T10:07:35.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad71ffb640 1 -- 172.21.15.133:0/498896696 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad640ab600 con 0x7fad74161050 2024-04-25T10:07:35.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad71ffb640 1 -- 172.21.15.133:0/498896696 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad640ab9d0 con 0x7fad74161050 2024-04-25T10:07:35.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad797de640 1 -- 172.21.15.133:0/498896696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad74161050 msgr2=0x7fad740999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad797de640 1 --2- 172.21.15.133:0/498896696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad74161050 0x7fad740999d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fad640097c0 tx=0x7fad6401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad797de640 1 -- 172.21.15.133:0/498896696 shutdown_connections 2024-04-25T10:07:35.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad797de640 1 --2- 172.21.15.133:0/498896696 >> v2:172.21.15.162:3300/0 conn(0x7fad740b3ea0 0x7fad74099f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad797de640 1 --2- 172.21.15.133:0/498896696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad74161050 0x7fad740999d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad797de640 1 -- 172.21.15.133:0/498896696 >> 172.21.15.133:0/498896696 conn(0x7fad740b31a0 msgr2=0x7fad740b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad797de640 1 -- 172.21.15.133:0/498896696 shutdown_connections 2024-04-25T10:07:35.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad797de640 1 -- 172.21.15.133:0/498896696 wait complete. 2024-04-25T10:07:35.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad797de640 1 Processor -- start 2024-04-25T10:07:35.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.593+0000 7fad797de640 1 -- start start 2024-04-25T10:07:35.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad797de640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad7409a450 0x7fad74154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad797de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad740b3ea0 0x7fad741556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad797de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad74161050 0x7fad74155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad797de640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7406d090 con 0x7fad74161050 2024-04-25T10:07:35.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad797de640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7406cf10 con 0x7fad7409a450 2024-04-25T10:07:35.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad797de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7416a880 con 0x7fad740b3ea0 2024-04-25T10:07:35.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad737fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad74161050 0x7fad74155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad737fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad74161050 0x7fad74155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40752/0 (socket says 172.21.15.133:40752) 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad737fe640 1 -- 172.21.15.133:0/4225200327 learned_addr learned my addr 172.21.15.133:0/4225200327 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad727fc640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad740b3ea0 0x7fad741556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad72ffd640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad7409a450 0x7fad74154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad737fe640 1 -- 172.21.15.133:0/4225200327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad740b3ea0 msgr2=0x7fad741556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad737fe640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad740b3ea0 0x7fad741556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad737fe640 1 -- 172.21.15.133:0/4225200327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad7409a450 msgr2=0x7fad74154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad737fe640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad7409a450 0x7fad74154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad737fe640 1 -- 172.21.15.133:0/4225200327 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad740c7a80 con 0x7fad74161050 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad737fe640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad74161050 0x7fad74155be0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c730 tx=0x7fad6c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad5b7fe640 1 -- 172.21.15.133:0/4225200327 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c018070 con 0x7fad74161050 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad5b7fe640 1 -- 172.21.15.133:0/4225200327 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c004d70 con 0x7fad74161050 2024-04-25T10:07:35.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad797de640 1 -- 172.21.15.133:0/4225200327 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad740c7d60 con 0x7fad74161050 2024-04-25T10:07:35.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad5b7fe640 1 -- 172.21.15.133:0/4225200327 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00a840 con 0x7fad74161050 2024-04-25T10:07:35.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad797de640 1 -- 172.21.15.133:0/4225200327 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad74155340 con 0x7fad74161050 2024-04-25T10:07:35.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad797de640 1 -- 172.21.15.133:0/4225200327 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad74068ef0 con 0x7fad74161050 2024-04-25T10:07:35.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad5b7fe640 1 -- 172.21.15.133:0/4225200327 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad6c00a9e0 con 0x7fad74161050 2024-04-25T10:07:35.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad5b7fe640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad60066f60 0x7fad60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad5b7fe640 1 -- 172.21.15.133:0/4225200327 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fad6c0845c0 con 0x7fad74161050 2024-04-25T10:07:35.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad72ffd640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad60066f60 0x7fad60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.597+0000 7fad72ffd640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad60066f60 0x7fad60069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fad640ad000 tx=0x7fad64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.601+0000 7fad5b7fe640 1 -- 172.21.15.133:0/4225200327 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad6c054d70 con 0x7fad74161050 2024-04-25T10:07:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.757+0000 7fad797de640 1 -- 172.21.15.133:0/4225200327 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad740af380 con 0x7fad74161050 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.761+0000 7fad5b7fe640 1 -- 172.21.15.133:0/4225200327 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16969 (secure 0 0 0) 0x7fad6c05b0c0 con 0x7fad74161050 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:35.766 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":81,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:07:33.066396+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:29.693214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:30.421009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 -- 172.21.15.133:0/4225200327 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad60066f60 msgr2=0x7fad60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad60066f60 0x7fad60069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fad640ad000 tx=0x7fad64008040 comp rx=0 tx=0).stop 2024-04-25T10:07:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 -- 172.21.15.133:0/4225200327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad74161050 msgr2=0x7fad74155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad74161050 0x7fad74155be0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c730 tx=0x7fad6c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 -- 172.21.15.133:0/4225200327 shutdown_connections 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad60066f60 0x7fad60069420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad74161050 0x7fad74155be0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad740b3ea0 0x7fad741556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 --2- 172.21.15.133:0/4225200327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad7409a450 0x7fad74154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 -- 172.21.15.133:0/4225200327 >> 172.21.15.133:0/4225200327 conn(0x7fad740b31a0 msgr2=0x7fad740ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 -- 172.21.15.133:0/4225200327 shutdown_connections 2024-04-25T10:07:35.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.765+0000 7fad797de640 1 -- 172.21.15.133:0/4225200327 wait complete. 2024-04-25T10:07:35.784 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:07:35.784 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:35.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.889+0000 7f4f3d65a640 1 Processor -- start 2024-04-25T10:07:35.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.889+0000 7f4f3d65a640 1 -- start start 2024-04-25T10:07:35.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f3d65a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4f3815db00 0x7f4f380b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f3d65a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f380ceb20 con 0x7f4f380d0bd0 2024-04-25T10:07:35.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f3d65a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f380b97f0 con 0x7f4f380d0770 2024-04-25T10:07:35.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f3d65a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f381622e0 con 0x7f4f3815db00 2024-04-25T10:07:35.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f377fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4f380d0bd0 0x7f4f38161bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34164/0 (socket says 172.21.15.133:34164) 2024-04-25T10:07:35.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f377fe640 1 -- 172.21.15.133:0/3004984708 learned_addr learned my addr 172.21.15.133:0/3004984708 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:35.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f36ffd640 1 --2- 172.21.15.133:0/3004984708 >> v2:172.21.15.162:3300/0 conn(0x7f4f3815db00 0x7f4f380b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867442074 0 0) 0x7f4f380ceb20 con 0x7f4f380d0bd0 2024-04-25T10:07:35.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f20003530 con 0x7f4f380d0bd0 2024-04-25T10:07:35.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669916932 0 0) 0x7f4f20003530 con 0x7f4f380d0bd0 2024-04-25T10:07:35.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 >> v2:172.21.15.162:3300/0 conn(0x7f4f3815db00 msgr2=0x7f4f380b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 --2- 172.21.15.133:0/3004984708 >> v2:172.21.15.162:3300/0 conn(0x7f4f3815db00 0x7f4f380b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 >> v1:172.21.15.162:6789/0 conn(0x7f4f380d0770 legacy=0x7f4f381614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f380b7740 con 0x7f4f380d0bd0 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4f30001a00 con 0x7f4f380d0bd0 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 >> v1:172.21.15.133:6789/0 conn(0x7f4f380d0bd0 legacy=0x7f4f38161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 --2- 172.21.15.133:0/3004984708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f200039c0 0x7f4f20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 --2- 172.21.15.133:0/3004984708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f20004810 0x7f4f20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 --2- 172.21.15.133:0/3004984708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 0x7f4f2000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f381622e0 con 0x7f4f200051d0 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2000c070 con 0x7f4f20004810 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2000c270 con 0x7f4f200039c0 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f377fe640 1 --2- 172.21.15.133:0/3004984708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 0x7f4f2000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f367fc640 1 --2- 172.21.15.133:0/3004984708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f20004810 0x7f4f20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f36ffd640 1 --2- 172.21.15.133:0/3004984708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f200039c0 0x7f4f20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f377fe640 1 -- 172.21.15.133:0/3004984708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f200039c0 msgr2=0x7f4f20003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f377fe640 1 --2- 172.21.15.133:0/3004984708 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f200039c0 0x7f4f20003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f377fe640 1 -- 172.21.15.133:0/3004984708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f20004810 msgr2=0x7f4f20004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f377fe640 1 --2- 172.21.15.133:0/3004984708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f20004810 0x7f4f20004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f377fe640 1 -- 172.21.15.133:0/3004984708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f20003530 con 0x7f4f200051d0 2024-04-25T10:07:35.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f377fe640 1 -- 172.21.15.133:0/3004984708 reap_dead start 2024-04-25T10:07:35.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f377fe640 1 --2- 172.21.15.133:0/3004984708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 0x7f4f2000b9d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000a840 tx=0x7f4f3001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc332852146ead96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000a460 con 0x7f4f200051d0 2024-04-25T10:07:35.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f300ac640 con 0x7f4f200051d0 2024-04-25T10:07:35.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.893+0000 7f4f35ffb640 1 -- 172.21.15.133:0/3004984708 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f300aca30 con 0x7f4f200051d0 2024-04-25T10:07:35.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- 172.21.15.133:0/3004984708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 msgr2=0x7f4f2000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 --2- 172.21.15.133:0/3004984708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 0x7f4f2000b9d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000a840 tx=0x7f4f3001d360 comp rx=0 tx=0).stop 2024-04-25T10:07:35.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- 172.21.15.133:0/3004984708 shutdown_connections 2024-04-25T10:07:35.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 --2- 172.21.15.133:0/3004984708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 0x7f4f2000b9d0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- 172.21.15.133:0/3004984708 >> 172.21.15.133:0/3004984708 conn(0x7f4f380727c0 msgr2=0x7f4f38072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- 172.21.15.133:0/3004984708 shutdown_connections 2024-04-25T10:07:35.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- 172.21.15.133:0/3004984708 wait complete. 2024-04-25T10:07:35.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 Processor -- start 2024-04-25T10:07:35.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- start start 2024-04-25T10:07:35.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 0x7f4f3815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38151d40 0x7f4f380d2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f380ccfe0 0x7f4f381476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380b7fe0 con 0x7f4f200051d0 2024-04-25T10:07:35.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380b7d00 con 0x7f4f38151d40 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380b82c0 con 0x7f4f380ccfe0 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f36ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 0x7f4f3815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f36ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 0x7f4f3815dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40778/0 (socket says 172.21.15.133:40778) 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f36ffd640 1 -- 172.21.15.133:0/763356232 learned_addr learned my addr 172.21.15.133:0/763356232 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f367fc640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38151d40 0x7f4f380d2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f377fe640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f380ccfe0 0x7f4f381476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f36ffd640 1 -- 172.21.15.133:0/763356232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f380ccfe0 msgr2=0x7f4f381476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f36ffd640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f380ccfe0 0x7f4f381476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f36ffd640 1 -- 172.21.15.133:0/763356232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38151d40 msgr2=0x7f4f380d2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f36ffd640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38151d40 0x7f4f380d2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f36ffd640 1 -- 172.21.15.133:0/763356232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f380cbf40 con 0x7f4f200051d0 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f367fc640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38151d40 0x7f4f380d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f36ffd640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 0x7f4f3815dee0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f4f28011590 tx=0x7f4f28011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f1f7fe640 1 -- 172.21.15.133:0/763356232 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28015930 con 0x7f4f200051d0 2024-04-25T10:07:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f1f7fe640 1 -- 172.21.15.133:0/763356232 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f28015ad0 con 0x7f4f200051d0 2024-04-25T10:07:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- 172.21.15.133:0/763356232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f380cc220 con 0x7f4f200051d0 2024-04-25T10:07:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f1f7fe640 1 -- 172.21.15.133:0/763356232 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800f460 con 0x7f4f200051d0 2024-04-25T10:07:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- 172.21.15.133:0/763356232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f380cc7e0 con 0x7f4f200051d0 2024-04-25T10:07:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.897+0000 7f4f3d65a640 1 -- 172.21.15.133:0/763356232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f38068ef0 con 0x7f4f200051d0 2024-04-25T10:07:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.901+0000 7f4f1f7fe640 1 -- 172.21.15.133:0/763356232 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f2800f600 con 0x7f4f200051d0 2024-04-25T10:07:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.901+0000 7f4f1f7fe640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f24066f60 0x7f4f24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.901+0000 7f4f367fc640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f24066f60 0x7f4f24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.901+0000 7f4f1f7fe640 1 -- 172.21.15.133:0/763356232 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4f28085b80 con 0x7f4f200051d0 2024-04-25T10:07:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.901+0000 7f4f367fc640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f24066f60 0x7f4f24069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4f380d42a0 tx=0x7f4f2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:35.905+0000 7f4f1f7fe640 1 -- 172.21.15.133:0/763356232 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f28019030 con 0x7f4f200051d0 2024-04-25T10:07:36.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.045+0000 7f4f3d65a640 1 -- 172.21.15.133:0/763356232 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f380b2750 con 0x7f4f24066f60 2024-04-25T10:07:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.057+0000 7f4f1f7fe640 1 -- 172.21.15.133:0/763356232 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252075 (secure 0 0 0) 0x7f4f380b2750 con 0x7f4f24066f60 2024-04-25T10:07:36.066 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:36.068 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:07:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 -- 172.21.15.133:0/763356232 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f24066f60 msgr2=0x7f4f24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f24066f60 0x7f4f24069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4f380d42a0 tx=0x7f4f2c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 -- 172.21.15.133:0/763356232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 msgr2=0x7f4f3815dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 0x7f4f3815dee0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f4f28011590 tx=0x7f4f28011a50 comp rx=0 tx=0).stop 2024-04-25T10:07:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 -- 172.21.15.133:0/763356232 shutdown_connections 2024-04-25T10:07:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f380ccfe0 0x7f4f381476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38151d40 0x7f4f380d2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f24066f60 0x7f4f24069420 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 --2- 172.21.15.133:0/763356232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f200051d0 0x7f4f3815dee0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 -- 172.21.15.133:0/763356232 >> 172.21.15.133:0/763356232 conn(0x7f4f380727c0 msgr2=0x7f4f380b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 -- 172.21.15.133:0/763356232 shutdown_connections 2024-04-25T10:07:36.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.061+0000 7f4f3d65a640 1 -- 172.21.15.133:0/763356232 wait complete. 2024-04-25T10:07:36.084 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-04-25T10:07:35.811684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351424,"kb_used_data":8968,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754623296,"statfs":{"total":773094113280,"available":772734255104,"internally_reserved":0,"allocated":9183232,"data_stored":4837404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":350539313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.758453"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153668+0000","last_change":"2024-04-25T10:07:32.100331+0000","last_active":"2024-04-25T10:07:33.153668+0000","last_peered":"2024-04-25T10:07:33.153668+0000","last_clean":"2024-04-25T10:07:33.153668+0000","last_became_active":"2024-04-25T10:07:32.099806+0000","last_became_peered":"2024-04-25T10:07:32.099806+0000","last_unstale":"2024-04-25T10:07:33.153668+0000","last_undegraded":"2024-04-25T10:07:33.153668+0000","last_fullsized":"2024-04-25T10:07:33.153668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:57.478620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156537+0000","last_change":"2024-04-25T10:07:32.102192+0000","last_active":"2024-04-25T10:07:33.156537+0000","last_peered":"2024-04-25T10:07:33.156537+0000","last_clean":"2024-04-25T10:07:33.156537+0000","last_became_active":"2024-04-25T10:07:32.101908+0000","last_became_peered":"2024-04-25T10:07:32.101908+0000","last_unstale":"2024-04-25T10:07:33.156537+0000","last_undegraded":"2024-04-25T10:07:33.156537+0000","last_fullsized":"2024-04-25T10:07:33.156537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:57.419880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073761+0000","last_change":"2024-04-25T10:07:32.081936+0000","last_active":"2024-04-25T10:07:33.073761+0000","last_peered":"2024-04-25T10:07:33.073761+0000","last_clean":"2024-04-25T10:07:33.073761+0000","last_became_active":"2024-04-25T10:07:32.081665+0000","last_became_peered":"2024-04-25T10:07:32.081665+0000","last_unstale":"2024-04-25T10:07:33.073761+0000","last_undegraded":"2024-04-25T10:07:33.073761+0000","last_fullsized":"2024-04-25T10:07:33.073761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:25.396490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152980+0000","last_change":"2024-04-25T10:07:32.084830+0000","last_active":"2024-04-25T10:07:33.152980+0000","last_peered":"2024-04-25T10:07:33.152980+0000","last_clean":"2024-04-25T10:07:33.152980+0000","last_became_active":"2024-04-25T10:07:32.084507+0000","last_became_peered":"2024-04-25T10:07:32.084507+0000","last_unstale":"2024-04-25T10:07:33.152980+0000","last_undegraded":"2024-04-25T10:07:33.152980+0000","last_fullsized":"2024-04-25T10:07:33.152980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:01.944592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157492+0000","last_change":"2024-04-25T10:07:32.101042+0000","last_active":"2024-04-25T10:07:33.157492+0000","last_peered":"2024-04-25T10:07:33.157492+0000","last_clean":"2024-04-25T10:07:33.157492+0000","last_became_active":"2024-04-25T10:07:32.098677+0000","last_became_peered":"2024-04-25T10:07:32.098677+0000","last_unstale":"2024-04-25T10:07:33.157492+0000","last_undegraded":"2024-04-25T10:07:33.157492+0000","last_fullsized":"2024-04-25T10:07:33.157492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:22.280577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156379+0000","last_change":"2024-04-25T10:07:32.106242+0000","last_active":"2024-04-25T10:07:33.156379+0000","last_peered":"2024-04-25T10:07:33.156379+0000","last_clean":"2024-04-25T10:07:33.156379+0000","last_became_active":"2024-04-25T10:07:32.105864+0000","last_became_peered":"2024-04-25T10:07:32.105864+0000","last_unstale":"2024-04-25T10:07:33.156379+0000","last_undegraded":"2024-04-25T10:07:33.156379+0000","last_fullsized":"2024-04-25T10:07:33.156379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:57.419880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075568+0000","last_change":"2024-04-25T10:07:32.093653+0000","last_active":"2024-04-25T10:07:33.075568+0000","last_peered":"2024-04-25T10:07:33.075568+0000","last_clean":"2024-04-25T10:07:33.075568+0000","last_became_active":"2024-04-25T10:07:32.093232+0000","last_became_peered":"2024-04-25T10:07:32.093232+0000","last_unstale":"2024-04-25T10:07:33.075568+0000","last_undegraded":"2024-04-25T10:07:33.075568+0000","last_fullsized":"2024-04-25T10:07:33.075568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:42.032766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154781+0000","last_change":"2024-04-25T10:07:32.105048+0000","last_active":"2024-04-25T10:07:33.154781+0000","last_peered":"2024-04-25T10:07:33.154781+0000","last_clean":"2024-04-25T10:07:33.154781+0000","last_became_active":"2024-04-25T10:07:32.104726+0000","last_became_peered":"2024-04-25T10:07:32.104726+0000","last_unstale":"2024-04-25T10:07:33.154781+0000","last_undegraded":"2024-04-25T10:07:33.154781+0000","last_fullsized":"2024-04-25T10:07:33.154781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:09.813690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076829+0000","last_change":"2024-04-25T10:07:32.093193+0000","last_active":"2024-04-25T10:07:33.076829+0000","last_peered":"2024-04-25T10:07:33.076829+0000","last_clean":"2024-04-25T10:07:33.076829+0000","last_became_active":"2024-04-25T10:07:32.092796+0000","last_became_peered":"2024-04-25T10:07:32.092796+0000","last_unstale":"2024-04-25T10:07:33.076829+0000","last_undegraded":"2024-04-25T10:07:33.076829+0000","last_fullsized":"2024-04-25T10:07:33.076829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:20.489932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153440+0000","last_change":"2024-04-25T10:07:32.085311+0000","last_active":"2024-04-25T10:07:33.153440+0000","last_peered":"2024-04-25T10:07:33.153440+0000","last_clean":"2024-04-25T10:07:33.153440+0000","last_became_active":"2024-04-25T10:07:32.085034+0000","last_became_peered":"2024-04-25T10:07:32.085034+0000","last_unstale":"2024-04-25T10:07:33.153440+0000","last_undegraded":"2024-04-25T10:07:33.153440+0000","last_fullsized":"2024-04-25T10:07:33.153440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:47.903283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156633+0000","last_change":"2024-04-25T10:07:32.111336+0000","last_active":"2024-04-25T10:07:33.156633+0000","last_peered":"2024-04-25T10:07:33.156633+0000","last_clean":"2024-04-25T10:07:33.156633+0000","last_became_active":"2024-04-25T10:07:32.109120+0000","last_became_peered":"2024-04-25T10:07:32.109120+0000","last_unstale":"2024-04-25T10:07:33.156633+0000","last_undegraded":"2024-04-25T10:07:33.156633+0000","last_fullsized":"2024-04-25T10:07:33.156633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:47.903283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153097+0000","last_change":"2024-04-25T10:07:32.108045+0000","last_active":"2024-04-25T10:07:33.153097+0000","last_peered":"2024-04-25T10:07:33.153097+0000","last_clean":"2024-04-25T10:07:33.153097+0000","last_became_active":"2024-04-25T10:07:32.107726+0000","last_became_peered":"2024-04-25T10:07:32.107726+0000","last_unstale":"2024-04-25T10:07:33.153097+0000","last_undegraded":"2024-04-25T10:07:33.153097+0000","last_fullsized":"2024-04-25T10:07:33.153097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:12.882528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157640+0000","last_change":"2024-04-25T10:07:32.095757+0000","last_active":"2024-04-25T10:07:33.157640+0000","last_peered":"2024-04-25T10:07:33.157640+0000","last_clean":"2024-04-25T10:07:33.157640+0000","last_became_active":"2024-04-25T10:07:32.095377+0000","last_became_peered":"2024-04-25T10:07:32.095377+0000","last_unstale":"2024-04-25T10:07:33.157640+0000","last_undegraded":"2024-04-25T10:07:33.157640+0000","last_fullsized":"2024-04-25T10:07:33.157640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:45.401815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156642+0000","last_change":"2024-04-25T10:07:32.105857+0000","last_active":"2024-04-25T10:07:33.156642+0000","last_peered":"2024-04-25T10:07:33.156642+0000","last_clean":"2024-04-25T10:07:33.156642+0000","last_became_active":"2024-04-25T10:07:32.105568+0000","last_became_peered":"2024-04-25T10:07:32.105568+0000","last_unstale":"2024-04-25T10:07:33.156642+0000","last_undegraded":"2024-04-25T10:07:33.156642+0000","last_fullsized":"2024-04-25T10:07:33.156642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:47.903283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153891+0000","last_change":"2024-04-25T10:07:32.090273+0000","last_active":"2024-04-25T10:07:33.153891+0000","last_peered":"2024-04-25T10:07:33.153891+0000","last_clean":"2024-04-25T10:07:33.153891+0000","last_became_active":"2024-04-25T10:07:32.089861+0000","last_became_peered":"2024-04-25T10:07:32.089861+0000","last_unstale":"2024-04-25T10:07:33.153891+0000","last_undegraded":"2024-04-25T10:07:33.153891+0000","last_fullsized":"2024-04-25T10:07:33.153891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:45.678236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153299+0000","last_change":"2024-04-25T10:07:32.099935+0000","last_active":"2024-04-25T10:07:33.153299+0000","last_peered":"2024-04-25T10:07:33.153299+0000","last_clean":"2024-04-25T10:07:33.153299+0000","last_became_active":"2024-04-25T10:07:32.095663+0000","last_became_peered":"2024-04-25T10:07:32.095663+0000","last_unstale":"2024-04-25T10:07:33.153299+0000","last_undegraded":"2024-04-25T10:07:33.153299+0000","last_fullsized":"2024-04-25T10:07:33.153299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:55.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155773+0000","last_change":"2024-04-25T10:07:32.090286+0000","last_active":"2024-04-25T10:07:33.155773+0000","last_peered":"2024-04-25T10:07:33.155773+0000","last_clean":"2024-04-25T10:07:33.155773+0000","last_became_active":"2024-04-25T10:07:32.087175+0000","last_became_peered":"2024-04-25T10:07:32.087175+0000","last_unstale":"2024-04-25T10:07:33.155773+0000","last_undegraded":"2024-04-25T10:07:33.155773+0000","last_fullsized":"2024-04-25T10:07:33.155773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:34.945854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073432+0000","last_change":"2024-04-25T10:07:32.083722+0000","last_active":"2024-04-25T10:07:33.073432+0000","last_peered":"2024-04-25T10:07:33.073432+0000","last_clean":"2024-04-25T10:07:33.073432+0000","last_became_active":"2024-04-25T10:07:32.083300+0000","last_became_peered":"2024-04-25T10:07:32.083300+0000","last_unstale":"2024-04-25T10:07:33.073432+0000","last_undegraded":"2024-04-25T10:07:33.073432+0000","last_fullsized":"2024-04-25T10:07:33.073432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:41.196995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155339+0000","last_change":"2024-04-25T10:07:32.107028+0000","last_active":"2024-04-25T10:07:33.155339+0000","last_peered":"2024-04-25T10:07:33.155339+0000","last_clean":"2024-04-25T10:07:33.155339+0000","last_became_active":"2024-04-25T10:07:32.106752+0000","last_became_peered":"2024-04-25T10:07:32.106752+0000","last_unstale":"2024-04-25T10:07:33.155339+0000","last_undegraded":"2024-04-25T10:07:33.155339+0000","last_fullsized":"2024-04-25T10:07:33.155339+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:12.882528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154806+0000","last_change":"2024-04-25T10:07:32.095882+0000","last_active":"2024-04-25T10:07:33.154806+0000","last_peered":"2024-04-25T10:07:33.154806+0000","last_clean":"2024-04-25T10:07:33.154806+0000","last_became_active":"2024-04-25T10:07:32.094624+0000","last_became_peered":"2024-04-25T10:07:32.094624+0000","last_unstale":"2024-04-25T10:07:33.154806+0000","last_undegraded":"2024-04-25T10:07:33.154806+0000","last_fullsized":"2024-04-25T10:07:33.154806+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:49.142104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075489+0000","last_change":"2024-04-25T10:07:32.077540+0000","last_active":"2024-04-25T10:07:33.075489+0000","last_peered":"2024-04-25T10:07:33.075489+0000","last_clean":"2024-04-25T10:07:33.075489+0000","last_became_active":"2024-04-25T10:07:32.077241+0000","last_became_peered":"2024-04-25T10:07:32.077241+0000","last_unstale":"2024-04-25T10:07:33.075489+0000","last_undegraded":"2024-04-25T10:07:33.075489+0000","last_fullsized":"2024-04-25T10:07:33.075489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:06.140735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156195+0000","last_change":"2024-04-25T10:07:32.109140+0000","last_active":"2024-04-25T10:07:33.156195+0000","last_peered":"2024-04-25T10:07:33.156195+0000","last_clean":"2024-04-25T10:07:33.156195+0000","last_became_active":"2024-04-25T10:07:32.108924+0000","last_became_peered":"2024-04-25T10:07:32.108924+0000","last_unstale":"2024-04-25T10:07:33.156195+0000","last_undegraded":"2024-04-25T10:07:33.156195+0000","last_fullsized":"2024-04-25T10:07:33.156195+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:50.958157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.074734+0000","last_change":"2024-04-25T10:07:32.093599+0000","last_active":"2024-04-25T10:07:33.074734+0000","last_peered":"2024-04-25T10:07:33.074734+0000","last_clean":"2024-04-25T10:07:33.074734+0000","last_became_active":"2024-04-25T10:07:32.093293+0000","last_became_peered":"2024-04-25T10:07:32.093293+0000","last_unstale":"2024-04-25T10:07:33.074734+0000","last_undegraded":"2024-04-25T10:07:33.074734+0000","last_fullsized":"2024-04-25T10:07:33.074734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:35.139055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153825+0000","last_change":"2024-04-25T10:07:32.089675+0000","last_active":"2024-04-25T10:07:33.153825+0000","last_peered":"2024-04-25T10:07:33.153825+0000","last_clean":"2024-04-25T10:07:33.153825+0000","last_became_active":"2024-04-25T10:07:32.089296+0000","last_became_peered":"2024-04-25T10:07:32.089296+0000","last_unstale":"2024-04-25T10:07:33.153825+0000","last_undegraded":"2024-04-25T10:07:33.153825+0000","last_fullsized":"2024-04-25T10:07:33.153825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:48.477318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156497+0000","last_change":"2024-04-25T10:07:32.089589+0000","last_active":"2024-04-25T10:07:33.156497+0000","last_peered":"2024-04-25T10:07:33.156497+0000","last_clean":"2024-04-25T10:07:33.156497+0000","last_became_active":"2024-04-25T10:07:32.089270+0000","last_became_peered":"2024-04-25T10:07:32.089270+0000","last_unstale":"2024-04-25T10:07:33.156497+0000","last_undegraded":"2024-04-25T10:07:33.156497+0000","last_fullsized":"2024-04-25T10:07:33.156497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:12.882528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153880+0000","last_change":"2024-04-25T10:07:32.108202+0000","last_active":"2024-04-25T10:07:33.153880+0000","last_peered":"2024-04-25T10:07:33.153880+0000","last_clean":"2024-04-25T10:07:33.153880+0000","last_became_active":"2024-04-25T10:07:32.107866+0000","last_became_peered":"2024-04-25T10:07:32.107866+0000","last_unstale":"2024-04-25T10:07:33.153880+0000","last_undegraded":"2024-04-25T10:07:33.153880+0000","last_fullsized":"2024-04-25T10:07:33.153880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:49.849575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153623+0000","last_change":"2024-04-25T10:07:32.092178+0000","last_active":"2024-04-25T10:07:33.153623+0000","last_peered":"2024-04-25T10:07:33.153623+0000","last_clean":"2024-04-25T10:07:33.153623+0000","last_became_active":"2024-04-25T10:07:32.091959+0000","last_became_peered":"2024-04-25T10:07:32.091959+0000","last_unstale":"2024-04-25T10:07:33.153623+0000","last_undegraded":"2024-04-25T10:07:33.153623+0000","last_fullsized":"2024-04-25T10:07:33.153623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:57.568078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155775+0000","last_change":"2024-04-25T10:07:32.103842+0000","last_active":"2024-04-25T10:07:33.155775+0000","last_peered":"2024-04-25T10:07:33.155775+0000","last_clean":"2024-04-25T10:07:33.155775+0000","last_became_active":"2024-04-25T10:07:32.103556+0000","last_became_peered":"2024-04-25T10:07:32.103556+0000","last_unstale":"2024-04-25T10:07:33.155775+0000","last_undegraded":"2024-04-25T10:07:33.155775+0000","last_fullsized":"2024-04-25T10:07:33.155775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:06.816471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.158265+0000","last_change":"2024-04-25T10:07:32.082247+0000","last_active":"2024-04-25T10:07:33.158265+0000","last_peered":"2024-04-25T10:07:33.158265+0000","last_clean":"2024-04-25T10:07:33.158265+0000","last_became_active":"2024-04-25T10:07:32.081696+0000","last_became_peered":"2024-04-25T10:07:32.081696+0000","last_unstale":"2024-04-25T10:07:33.158265+0000","last_undegraded":"2024-04-25T10:07:33.158265+0000","last_fullsized":"2024-04-25T10:07:33.158265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:57.568078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153965+0000","last_change":"2024-04-25T10:07:32.087474+0000","last_active":"2024-04-25T10:07:33.153965+0000","last_peered":"2024-04-25T10:07:33.153965+0000","last_clean":"2024-04-25T10:07:33.153965+0000","last_became_active":"2024-04-25T10:07:32.087182+0000","last_became_peered":"2024-04-25T10:07:32.087182+0000","last_unstale":"2024-04-25T10:07:33.153965+0000","last_undegraded":"2024-04-25T10:07:33.153965+0000","last_fullsized":"2024-04-25T10:07:33.153965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:15.883375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073447+0000","last_change":"2024-04-25T10:07:32.076817+0000","last_active":"2024-04-25T10:07:33.073447+0000","last_peered":"2024-04-25T10:07:33.073447+0000","last_clean":"2024-04-25T10:07:33.073447+0000","last_became_active":"2024-04-25T10:07:32.076366+0000","last_became_peered":"2024-04-25T10:07:32.076366+0000","last_unstale":"2024-04-25T10:07:33.073447+0000","last_undegraded":"2024-04-25T10:07:33.073447+0000","last_fullsized":"2024-04-25T10:07:33.073447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:38.771120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153755+0000","last_change":"2024-04-25T10:07:32.100218+0000","last_active":"2024-04-25T10:07:33.153755+0000","last_peered":"2024-04-25T10:07:33.153755+0000","last_clean":"2024-04-25T10:07:33.153755+0000","last_became_active":"2024-04-25T10:07:32.099843+0000","last_became_peered":"2024-04-25T10:07:32.099843+0000","last_unstale":"2024-04-25T10:07:33.153755+0000","last_undegraded":"2024-04-25T10:07:33.153755+0000","last_fullsized":"2024-04-25T10:07:33.153755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:50.290103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073232+0000","last_change":"2024-04-25T10:07:32.076936+0000","last_active":"2024-04-25T10:07:33.073232+0000","last_peered":"2024-04-25T10:07:33.073232+0000","last_clean":"2024-04-25T10:07:33.073232+0000","last_became_active":"2024-04-25T10:07:32.076632+0000","last_became_peered":"2024-04-25T10:07:32.076632+0000","last_unstale":"2024-04-25T10:07:33.073232+0000","last_undegraded":"2024-04-25T10:07:33.073232+0000","last_fullsized":"2024-04-25T10:07:33.073232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:39.095604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156047+0000","last_change":"2024-04-25T10:07:31.092554+0000","last_active":"2024-04-25T10:07:33.156047+0000","last_peered":"2024-04-25T10:07:33.156047+0000","last_clean":"2024-04-25T10:07:33.156047+0000","last_became_active":"2024-04-25T10:07:31.092166+0000","last_became_peered":"2024-04-25T10:07:31.092166+0000","last_unstale":"2024-04-25T10:07:33.156047+0000","last_undegraded":"2024-04-25T10:07:33.156047+0000","last_fullsized":"2024-04-25T10:07:33.156047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:09.890364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154284+0000","last_change":"2024-04-25T10:07:32.097077+0000","last_active":"2024-04-25T10:07:33.154284+0000","last_peered":"2024-04-25T10:07:33.154284+0000","last_clean":"2024-04-25T10:07:33.154284+0000","last_became_active":"2024-04-25T10:07:32.096710+0000","last_became_peered":"2024-04-25T10:07:32.096710+0000","last_unstale":"2024-04-25T10:07:33.154284+0000","last_undegraded":"2024-04-25T10:07:33.154284+0000","last_fullsized":"2024-04-25T10:07:33.154284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:59.457468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152955+0000","last_change":"2024-04-25T10:07:31.106518+0000","last_active":"2024-04-25T10:07:33.152955+0000","last_peered":"2024-04-25T10:07:33.152955+0000","last_clean":"2024-04-25T10:07:33.152955+0000","last_became_active":"2024-04-25T10:07:31.105838+0000","last_became_peered":"2024-04-25T10:07:31.105838+0000","last_unstale":"2024-04-25T10:07:33.152955+0000","last_undegraded":"2024-04-25T10:07:33.152955+0000","last_fullsized":"2024-04-25T10:07:33.152955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:39.402806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075162+0000","last_change":"2024-04-25T10:07:32.093742+0000","last_active":"2024-04-25T10:07:33.075162+0000","last_peered":"2024-04-25T10:07:33.075162+0000","last_clean":"2024-04-25T10:07:33.075162+0000","last_became_active":"2024-04-25T10:07:32.093380+0000","last_became_peered":"2024-04-25T10:07:32.093380+0000","last_unstale":"2024-04-25T10:07:33.075162+0000","last_undegraded":"2024-04-25T10:07:33.075162+0000","last_fullsized":"2024-04-25T10:07:33.075162+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:45.199003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152878+0000","last_change":"2024-04-25T10:07:31.097506+0000","last_active":"2024-04-25T10:07:33.152878+0000","last_peered":"2024-04-25T10:07:33.152878+0000","last_clean":"2024-04-25T10:07:33.152878+0000","last_became_active":"2024-04-25T10:07:31.094832+0000","last_became_peered":"2024-04-25T10:07:31.094832+0000","last_unstale":"2024-04-25T10:07:33.152878+0000","last_undegraded":"2024-04-25T10:07:33.152878+0000","last_fullsized":"2024-04-25T10:07:33.152878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:10.481037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075857+0000","last_change":"2024-04-25T10:07:32.077794+0000","last_active":"2024-04-25T10:07:33.075857+0000","last_peered":"2024-04-25T10:07:33.075857+0000","last_clean":"2024-04-25T10:07:33.075857+0000","last_became_active":"2024-04-25T10:07:32.077393+0000","last_became_peered":"2024-04-25T10:07:32.077393+0000","last_unstale":"2024-04-25T10:07:33.075857+0000","last_undegraded":"2024-04-25T10:07:33.075857+0000","last_fullsized":"2024-04-25T10:07:33.075857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:46.914154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075904+0000","last_change":"2024-04-25T10:07:31.102307+0000","last_active":"2024-04-25T10:07:33.075904+0000","last_peered":"2024-04-25T10:07:33.075904+0000","last_clean":"2024-04-25T10:07:33.075904+0000","last_became_active":"2024-04-25T10:07:31.096754+0000","last_became_peered":"2024-04-25T10:07:31.096754+0000","last_unstale":"2024-04-25T10:07:33.075904+0000","last_undegraded":"2024-04-25T10:07:33.075904+0000","last_fullsized":"2024-04-25T10:07:33.075904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:14.416210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156399+0000","last_change":"2024-04-25T10:07:32.109189+0000","last_active":"2024-04-25T10:07:33.156399+0000","last_peered":"2024-04-25T10:07:33.156399+0000","last_clean":"2024-04-25T10:07:33.156399+0000","last_became_active":"2024-04-25T10:07:32.108915+0000","last_became_peered":"2024-04-25T10:07:32.108915+0000","last_unstale":"2024-04-25T10:07:33.156399+0000","last_undegraded":"2024-04-25T10:07:33.156399+0000","last_fullsized":"2024-04-25T10:07:33.156399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:01.944592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076114+0000","last_change":"2024-04-25T10:07:31.092118+0000","last_active":"2024-04-25T10:07:33.076114+0000","last_peered":"2024-04-25T10:07:33.076114+0000","last_clean":"2024-04-25T10:07:33.076114+0000","last_became_active":"2024-04-25T10:07:31.091348+0000","last_became_peered":"2024-04-25T10:07:31.091348+0000","last_unstale":"2024-04-25T10:07:33.076114+0000","last_undegraded":"2024-04-25T10:07:33.076114+0000","last_fullsized":"2024-04-25T10:07:33.076114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:27.052688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157080+0000","last_change":"2024-04-25T10:07:32.102446+0000","last_active":"2024-04-25T10:07:33.157080+0000","last_peered":"2024-04-25T10:07:33.157080+0000","last_clean":"2024-04-25T10:07:33.157080+0000","last_became_active":"2024-04-25T10:07:32.102078+0000","last_became_peered":"2024-04-25T10:07:32.102078+0000","last_unstale":"2024-04-25T10:07:33.157080+0000","last_undegraded":"2024-04-25T10:07:33.157080+0000","last_fullsized":"2024-04-25T10:07:33.157080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:55.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152444+0000","last_change":"2024-04-25T10:07:31.102989+0000","last_active":"2024-04-25T10:07:33.152444+0000","last_peered":"2024-04-25T10:07:33.152444+0000","last_clean":"2024-04-25T10:07:33.152444+0000","last_became_active":"2024-04-25T10:07:31.101610+0000","last_became_peered":"2024-04-25T10:07:31.101610+0000","last_unstale":"2024-04-25T10:07:33.152444+0000","last_undegraded":"2024-04-25T10:07:33.152444+0000","last_fullsized":"2024-04-25T10:07:33.152444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.461805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076563+0000","last_change":"2024-04-25T10:07:32.093028+0000","last_active":"2024-04-25T10:07:33.076563+0000","last_peered":"2024-04-25T10:07:33.076563+0000","last_clean":"2024-04-25T10:07:33.076563+0000","last_became_active":"2024-04-25T10:07:32.092695+0000","last_became_peered":"2024-04-25T10:07:32.092695+0000","last_unstale":"2024-04-25T10:07:33.076563+0000","last_undegraded":"2024-04-25T10:07:33.076563+0000","last_fullsized":"2024-04-25T10:07:33.076563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:02.130848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155698+0000","last_change":"2024-04-25T10:07:31.091316+0000","last_active":"2024-04-25T10:07:33.155698+0000","last_peered":"2024-04-25T10:07:33.155698+0000","last_clean":"2024-04-25T10:07:33.155698+0000","last_became_active":"2024-04-25T10:07:31.090936+0000","last_became_peered":"2024-04-25T10:07:31.090936+0000","last_unstale":"2024-04-25T10:07:33.155698+0000","last_undegraded":"2024-04-25T10:07:33.155698+0000","last_fullsized":"2024-04-25T10:07:33.155698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:37.197045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076031+0000","last_change":"2024-04-25T10:07:32.090395+0000","last_active":"2024-04-25T10:07:33.076031+0000","last_peered":"2024-04-25T10:07:33.076031+0000","last_clean":"2024-04-25T10:07:33.076031+0000","last_became_active":"2024-04-25T10:07:32.090046+0000","last_became_peered":"2024-04-25T10:07:32.090046+0000","last_unstale":"2024-04-25T10:07:33.076031+0000","last_undegraded":"2024-04-25T10:07:33.076031+0000","last_fullsized":"2024-04-25T10:07:33.076031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:29.015009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154892+0000","last_change":"2024-04-25T10:07:31.087810+0000","last_active":"2024-04-25T10:07:33.154892+0000","last_peered":"2024-04-25T10:07:33.154892+0000","last_clean":"2024-04-25T10:07:33.154892+0000","last_became_active":"2024-04-25T10:07:31.087517+0000","last_became_peered":"2024-04-25T10:07:31.087517+0000","last_unstale":"2024-04-25T10:07:33.154892+0000","last_undegraded":"2024-04-25T10:07:33.154892+0000","last_fullsized":"2024-04-25T10:07:33.154892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:51.033707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153879+0000","last_change":"2024-04-25T10:07:32.096917+0000","last_active":"2024-04-25T10:07:33.153879+0000","last_peered":"2024-04-25T10:07:33.153879+0000","last_clean":"2024-04-25T10:07:33.153879+0000","last_became_active":"2024-04-25T10:07:32.096534+0000","last_became_peered":"2024-04-25T10:07:32.096534+0000","last_unstale":"2024-04-25T10:07:33.153879+0000","last_undegraded":"2024-04-25T10:07:33.153879+0000","last_fullsized":"2024-04-25T10:07:33.153879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:11.488648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154153+0000","last_change":"2024-04-25T10:07:31.106890+0000","last_active":"2024-04-25T10:07:33.154153+0000","last_peered":"2024-04-25T10:07:33.154153+0000","last_clean":"2024-04-25T10:07:33.154153+0000","last_became_active":"2024-04-25T10:07:31.106095+0000","last_became_peered":"2024-04-25T10:07:31.106095+0000","last_unstale":"2024-04-25T10:07:33.154153+0000","last_undegraded":"2024-04-25T10:07:33.154153+0000","last_fullsized":"2024-04-25T10:07:33.154153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:55.233331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152689+0000","last_change":"2024-04-25T10:07:32.086663+0000","last_active":"2024-04-25T10:07:33.152689+0000","last_peered":"2024-04-25T10:07:33.152689+0000","last_clean":"2024-04-25T10:07:33.152689+0000","last_became_active":"2024-04-25T10:07:32.086249+0000","last_became_peered":"2024-04-25T10:07:32.086249+0000","last_unstale":"2024-04-25T10:07:33.152689+0000","last_undegraded":"2024-04-25T10:07:33.152689+0000","last_fullsized":"2024-04-25T10:07:33.152689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:59.457468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153927+0000","last_change":"2024-04-25T10:07:31.101866+0000","last_active":"2024-04-25T10:07:33.153927+0000","last_peered":"2024-04-25T10:07:33.153927+0000","last_clean":"2024-04-25T10:07:33.153927+0000","last_became_active":"2024-04-25T10:07:31.101115+0000","last_became_peered":"2024-04-25T10:07:31.101115+0000","last_unstale":"2024-04-25T10:07:33.153927+0000","last_undegraded":"2024-04-25T10:07:33.153927+0000","last_fullsized":"2024-04-25T10:07:33.153927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:59.552554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157007+0000","last_change":"2024-04-25T10:07:32.094098+0000","last_active":"2024-04-25T10:07:33.157007+0000","last_peered":"2024-04-25T10:07:33.157007+0000","last_clean":"2024-04-25T10:07:33.157007+0000","last_became_active":"2024-04-25T10:07:32.093723+0000","last_became_peered":"2024-04-25T10:07:32.093723+0000","last_unstale":"2024-04-25T10:07:33.157007+0000","last_undegraded":"2024-04-25T10:07:33.157007+0000","last_fullsized":"2024-04-25T10:07:33.157007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:49.849575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.158015+0000","last_change":"2024-04-25T10:07:31.091614+0000","last_active":"2024-04-25T10:07:33.158015+0000","last_peered":"2024-04-25T10:07:33.158015+0000","last_clean":"2024-04-25T10:07:33.158015+0000","last_became_active":"2024-04-25T10:07:31.091152+0000","last_became_peered":"2024-04-25T10:07:31.091152+0000","last_unstale":"2024-04-25T10:07:33.158015+0000","last_undegraded":"2024-04-25T10:07:33.158015+0000","last_fullsized":"2024-04-25T10:07:33.158015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:56.331401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.074279+0000","last_change":"2024-04-25T10:07:32.078952+0000","last_active":"2024-04-25T10:07:33.074279+0000","last_peered":"2024-04-25T10:07:33.074279+0000","last_clean":"2024-04-25T10:07:33.074279+0000","last_became_active":"2024-04-25T10:07:32.078578+0000","last_became_peered":"2024-04-25T10:07:32.078578+0000","last_unstale":"2024-04-25T10:07:33.074279+0000","last_undegraded":"2024-04-25T10:07:33.074279+0000","last_fullsized":"2024-04-25T10:07:33.074279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:37.983770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075928+0000","last_change":"2024-04-25T10:07:31.095833+0000","last_active":"2024-04-25T10:07:33.075928+0000","last_peered":"2024-04-25T10:07:33.075928+0000","last_clean":"2024-04-25T10:07:33.075928+0000","last_became_active":"2024-04-25T10:07:31.095046+0000","last_became_peered":"2024-04-25T10:07:31.095046+0000","last_unstale":"2024-04-25T10:07:33.075928+0000","last_undegraded":"2024-04-25T10:07:33.075928+0000","last_fullsized":"2024-04-25T10:07:33.075928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.952601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153371+0000","last_change":"2024-04-25T10:07:32.091498+0000","last_active":"2024-04-25T10:07:33.153371+0000","last_peered":"2024-04-25T10:07:33.153371+0000","last_clean":"2024-04-25T10:07:33.153371+0000","last_became_active":"2024-04-25T10:07:32.091204+0000","last_became_peered":"2024-04-25T10:07:32.091204+0000","last_unstale":"2024-04-25T10:07:33.153371+0000","last_undegraded":"2024-04-25T10:07:33.153371+0000","last_fullsized":"2024-04-25T10:07:33.153371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:57.419880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152470+0000","last_change":"2024-04-25T10:07:31.109685+0000","last_active":"2024-04-25T10:07:33.152470+0000","last_peered":"2024-04-25T10:07:33.152470+0000","last_clean":"2024-04-25T10:07:33.152470+0000","last_became_active":"2024-04-25T10:07:31.108946+0000","last_became_peered":"2024-04-25T10:07:31.108946+0000","last_unstale":"2024-04-25T10:07:33.152470+0000","last_undegraded":"2024-04-25T10:07:33.152470+0000","last_fullsized":"2024-04-25T10:07:33.152470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:55.466116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075379+0000","last_change":"2024-04-25T10:07:32.076111+0000","last_active":"2024-04-25T10:07:33.075379+0000","last_peered":"2024-04-25T10:07:33.075379+0000","last_clean":"2024-04-25T10:07:33.075379+0000","last_became_active":"2024-04-25T10:07:32.075778+0000","last_became_peered":"2024-04-25T10:07:32.075778+0000","last_unstale":"2024-04-25T10:07:33.075379+0000","last_undegraded":"2024-04-25T10:07:33.075379+0000","last_fullsized":"2024-04-25T10:07:33.075379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:02.130848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153961+0000","last_change":"2024-04-25T10:07:31.094378+0000","last_active":"2024-04-25T10:07:33.153961+0000","last_peered":"2024-04-25T10:07:33.153961+0000","last_clean":"2024-04-25T10:07:33.153961+0000","last_became_active":"2024-04-25T10:07:31.093533+0000","last_became_peered":"2024-04-25T10:07:31.093533+0000","last_unstale":"2024-04-25T10:07:33.153961+0000","last_undegraded":"2024-04-25T10:07:33.153961+0000","last_fullsized":"2024-04-25T10:07:33.153961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:20.145193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075825+0000","last_change":"2024-04-25T10:07:32.079413+0000","last_active":"2024-04-25T10:07:33.075825+0000","last_peered":"2024-04-25T10:07:33.075825+0000","last_clean":"2024-04-25T10:07:33.075825+0000","last_became_active":"2024-04-25T10:07:32.079023+0000","last_became_peered":"2024-04-25T10:07:32.079023+0000","last_unstale":"2024-04-25T10:07:33.075825+0000","last_undegraded":"2024-04-25T10:07:33.075825+0000","last_fullsized":"2024-04-25T10:07:33.075825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:58.256908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154825+0000","last_change":"2024-04-25T10:07:31.097619+0000","last_active":"2024-04-25T10:07:33.154825+0000","last_peered":"2024-04-25T10:07:33.154825+0000","last_clean":"2024-04-25T10:07:33.154825+0000","last_became_active":"2024-04-25T10:07:31.097231+0000","last_became_peered":"2024-04-25T10:07:31.097231+0000","last_unstale":"2024-04-25T10:07:33.154825+0000","last_undegraded":"2024-04-25T10:07:33.154825+0000","last_fullsized":"2024-04-25T10:07:33.154825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:56.626555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076153+0000","last_change":"2024-04-25T10:07:32.087885+0000","last_active":"2024-04-25T10:07:33.076153+0000","last_peered":"2024-04-25T10:07:33.076153+0000","last_clean":"2024-04-25T10:07:33.076153+0000","last_became_active":"2024-04-25T10:07:32.087502+0000","last_became_peered":"2024-04-25T10:07:32.087502+0000","last_unstale":"2024-04-25T10:07:33.076153+0000","last_undegraded":"2024-04-25T10:07:33.076153+0000","last_fullsized":"2024-04-25T10:07:33.076153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:08.019664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154122+0000","last_change":"2024-04-25T10:07:31.094407+0000","last_active":"2024-04-25T10:07:33.154122+0000","last_peered":"2024-04-25T10:07:33.154122+0000","last_clean":"2024-04-25T10:07:33.154122+0000","last_became_active":"2024-04-25T10:07:31.094124+0000","last_became_peered":"2024-04-25T10:07:31.094124+0000","last_unstale":"2024-04-25T10:07:33.154122+0000","last_undegraded":"2024-04-25T10:07:33.154122+0000","last_fullsized":"2024-04-25T10:07:33.154122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:48.360303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155124+0000","last_change":"2024-04-25T10:07:32.102602+0000","last_active":"2024-04-25T10:07:33.155124+0000","last_peered":"2024-04-25T10:07:33.155124+0000","last_clean":"2024-04-25T10:07:33.155124+0000","last_became_active":"2024-04-25T10:07:32.095878+0000","last_became_peered":"2024-04-25T10:07:32.095878+0000","last_unstale":"2024-04-25T10:07:33.155124+0000","last_undegraded":"2024-04-25T10:07:33.155124+0000","last_fullsized":"2024-04-25T10:07:33.155124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:01.944592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153957+0000","last_change":"2024-04-25T10:07:31.099468+0000","last_active":"2024-04-25T10:07:33.153957+0000","last_peered":"2024-04-25T10:07:33.153957+0000","last_clean":"2024-04-25T10:07:33.153957+0000","last_became_active":"2024-04-25T10:07:31.098663+0000","last_became_peered":"2024-04-25T10:07:31.098663+0000","last_unstale":"2024-04-25T10:07:33.153957+0000","last_undegraded":"2024-04-25T10:07:33.153957+0000","last_fullsized":"2024-04-25T10:07:33.153957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:47.469708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154871+0000","last_change":"2024-04-25T10:07:32.098705+0000","last_active":"2024-04-25T10:07:33.154871+0000","last_peered":"2024-04-25T10:07:33.154871+0000","last_clean":"2024-04-25T10:07:33.154871+0000","last_became_active":"2024-04-25T10:07:32.096026+0000","last_became_peered":"2024-04-25T10:07:32.096026+0000","last_unstale":"2024-04-25T10:07:33.154871+0000","last_undegraded":"2024-04-25T10:07:33.154871+0000","last_fullsized":"2024-04-25T10:07:33.154871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:16.588134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076739+0000","last_change":"2024-04-25T10:07:31.080057+0000","last_active":"2024-04-25T10:07:33.076739+0000","last_peered":"2024-04-25T10:07:33.076739+0000","last_clean":"2024-04-25T10:07:33.076739+0000","last_became_active":"2024-04-25T10:07:31.079109+0000","last_became_peered":"2024-04-25T10:07:31.079109+0000","last_unstale":"2024-04-25T10:07:33.076739+0000","last_undegraded":"2024-04-25T10:07:33.076739+0000","last_fullsized":"2024-04-25T10:07:33.076739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:42.632287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156569+0000","last_change":"2024-04-25T10:07:32.110754+0000","last_active":"2024-04-25T10:07:33.156569+0000","last_peered":"2024-04-25T10:07:33.156569+0000","last_clean":"2024-04-25T10:07:33.156569+0000","last_became_active":"2024-04-25T10:07:32.110503+0000","last_became_peered":"2024-04-25T10:07:32.110503+0000","last_unstale":"2024-04-25T10:07:33.156569+0000","last_undegraded":"2024-04-25T10:07:33.156569+0000","last_fullsized":"2024-04-25T10:07:33.156569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:32.767768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.158125+0000","last_change":"2024-04-25T10:07:31.090657+0000","last_active":"2024-04-25T10:07:33.158125+0000","last_peered":"2024-04-25T10:07:33.158125+0000","last_clean":"2024-04-25T10:07:33.158125+0000","last_became_active":"2024-04-25T10:07:31.090165+0000","last_became_peered":"2024-04-25T10:07:31.090165+0000","last_unstale":"2024-04-25T10:07:33.158125+0000","last_undegraded":"2024-04-25T10:07:33.158125+0000","last_fullsized":"2024-04-25T10:07:33.158125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:46.784630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157018+0000","last_change":"2024-04-25T10:07:32.093132+0000","last_active":"2024-04-25T10:07:33.157018+0000","last_peered":"2024-04-25T10:07:33.157018+0000","last_clean":"2024-04-25T10:07:33.157018+0000","last_became_active":"2024-04-25T10:07:32.092926+0000","last_became_peered":"2024-04-25T10:07:32.092926+0000","last_unstale":"2024-04-25T10:07:33.157018+0000","last_undegraded":"2024-04-25T10:07:33.157018+0000","last_fullsized":"2024-04-25T10:07:33.157018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:47.334242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073127+0000","last_change":"2024-04-25T10:07:31.088937+0000","last_active":"2024-04-25T10:07:33.073127+0000","last_peered":"2024-04-25T10:07:33.073127+0000","last_clean":"2024-04-25T10:07:33.073127+0000","last_became_active":"2024-04-25T10:07:31.088572+0000","last_became_peered":"2024-04-25T10:07:31.088572+0000","last_unstale":"2024-04-25T10:07:33.073127+0000","last_undegraded":"2024-04-25T10:07:33.073127+0000","last_fullsized":"2024-04-25T10:07:33.073127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:01.123238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156732+0000","last_change":"2024-04-25T10:07:32.097016+0000","last_active":"2024-04-25T10:07:33.156732+0000","last_peered":"2024-04-25T10:07:33.156732+0000","last_clean":"2024-04-25T10:07:33.156732+0000","last_became_active":"2024-04-25T10:07:32.096656+0000","last_became_peered":"2024-04-25T10:07:32.096656+0000","last_unstale":"2024-04-25T10:07:33.156732+0000","last_undegraded":"2024-04-25T10:07:33.156732+0000","last_fullsized":"2024-04-25T10:07:33.156732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:49.849575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157187+0000","last_change":"2024-04-25T10:07:31.087429+0000","last_active":"2024-04-25T10:07:33.157187+0000","last_peered":"2024-04-25T10:07:33.157187+0000","last_clean":"2024-04-25T10:07:33.157187+0000","last_became_active":"2024-04-25T10:07:31.086584+0000","last_became_peered":"2024-04-25T10:07:31.086584+0000","last_unstale":"2024-04-25T10:07:33.157187+0000","last_undegraded":"2024-04-25T10:07:33.157187+0000","last_fullsized":"2024-04-25T10:07:33.157187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.461805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157050+0000","last_change":"2024-04-25T10:07:32.095990+0000","last_active":"2024-04-25T10:07:33.157050+0000","last_peered":"2024-04-25T10:07:33.157050+0000","last_clean":"2024-04-25T10:07:33.157050+0000","last_became_active":"2024-04-25T10:07:32.090719+0000","last_became_peered":"2024-04-25T10:07:32.090719+0000","last_unstale":"2024-04-25T10:07:33.157050+0000","last_undegraded":"2024-04-25T10:07:33.157050+0000","last_fullsized":"2024-04-25T10:07:33.157050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:47.334242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.158512+0000","last_change":"2024-04-25T10:07:31.105499+0000","last_active":"2024-04-25T10:07:33.158512+0000","last_peered":"2024-04-25T10:07:33.158512+0000","last_clean":"2024-04-25T10:07:33.158512+0000","last_became_active":"2024-04-25T10:07:31.105190+0000","last_became_peered":"2024-04-25T10:07:31.105190+0000","last_unstale":"2024-04-25T10:07:33.158512+0000","last_undegraded":"2024-04-25T10:07:33.158512+0000","last_fullsized":"2024-04-25T10:07:33.158512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:59.552554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076289+0000","last_change":"2024-04-25T10:07:32.082247+0000","last_active":"2024-04-25T10:07:33.076289+0000","last_peered":"2024-04-25T10:07:33.076289+0000","last_clean":"2024-04-25T10:07:33.076289+0000","last_became_active":"2024-04-25T10:07:32.081900+0000","last_became_peered":"2024-04-25T10:07:32.081900+0000","last_unstale":"2024-04-25T10:07:33.076289+0000","last_undegraded":"2024-04-25T10:07:33.076289+0000","last_fullsized":"2024-04-25T10:07:33.076289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:20.966999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155554+0000","last_change":"2024-04-25T10:07:31.105897+0000","last_active":"2024-04-25T10:07:33.155554+0000","last_peered":"2024-04-25T10:07:33.155554+0000","last_clean":"2024-04-25T10:07:33.155554+0000","last_became_active":"2024-04-25T10:07:31.105600+0000","last_became_peered":"2024-04-25T10:07:31.105600+0000","last_unstale":"2024-04-25T10:07:33.155554+0000","last_undegraded":"2024-04-25T10:07:33.155554+0000","last_fullsized":"2024-04-25T10:07:33.155554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:20.145193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.074647+0000","last_change":"2024-04-25T10:07:32.074618+0000","last_active":"2024-04-25T10:07:33.074647+0000","last_peered":"2024-04-25T10:07:33.074647+0000","last_clean":"2024-04-25T10:07:33.074647+0000","last_became_active":"2024-04-25T10:07:32.074237+0000","last_became_peered":"2024-04-25T10:07:32.074237+0000","last_unstale":"2024-04-25T10:07:33.074647+0000","last_undegraded":"2024-04-25T10:07:33.074647+0000","last_fullsized":"2024-04-25T10:07:33.074647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:46.235215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153464+0000","last_change":"2024-04-25T10:07:31.086154+0000","last_active":"2024-04-25T10:07:33.153464+0000","last_peered":"2024-04-25T10:07:33.153464+0000","last_clean":"2024-04-25T10:07:33.153464+0000","last_became_active":"2024-04-25T10:07:31.085162+0000","last_became_peered":"2024-04-25T10:07:31.085162+0000","last_unstale":"2024-04-25T10:07:33.153464+0000","last_undegraded":"2024-04-25T10:07:33.153464+0000","last_fullsized":"2024-04-25T10:07:33.153464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:04.962615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076161+0000","last_change":"2024-04-25T10:07:32.087704+0000","last_active":"2024-04-25T10:07:33.076161+0000","last_peered":"2024-04-25T10:07:33.076161+0000","last_clean":"2024-04-25T10:07:33.076161+0000","last_became_active":"2024-04-25T10:07:32.087297+0000","last_became_peered":"2024-04-25T10:07:32.087297+0000","last_unstale":"2024-04-25T10:07:33.076161+0000","last_undegraded":"2024-04-25T10:07:33.076161+0000","last_fullsized":"2024-04-25T10:07:33.076161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:17.011308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:34.080205+0000","last_change":"2024-04-25T10:07:31.093532+0000","last_active":"2024-04-25T10:07:34.080205+0000","last_peered":"2024-04-25T10:07:34.080205+0000","last_clean":"2024-04-25T10:07:34.080205+0000","last_became_active":"2024-04-25T10:07:31.093066+0000","last_became_peered":"2024-04-25T10:07:31.093066+0000","last_unstale":"2024-04-25T10:07:34.080205+0000","last_undegraded":"2024-04-25T10:07:34.080205+0000","last_fullsized":"2024-04-25T10:07:34.080205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:36.976160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075377+0000","last_change":"2024-04-25T10:07:32.094310+0000","last_active":"2024-04-25T10:07:33.075377+0000","last_peered":"2024-04-25T10:07:33.075377+0000","last_clean":"2024-04-25T10:07:33.075377+0000","last_became_active":"2024-04-25T10:07:32.094002+0000","last_became_peered":"2024-04-25T10:07:32.094002+0000","last_unstale":"2024-04-25T10:07:33.075377+0000","last_undegraded":"2024-04-25T10:07:33.075377+0000","last_fullsized":"2024-04-25T10:07:33.075377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:10.902889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154036+0000","last_change":"2024-04-25T10:07:31.112884+0000","last_active":"2024-04-25T10:07:33.154036+0000","last_peered":"2024-04-25T10:07:33.154036+0000","last_clean":"2024-04-25T10:07:33.154036+0000","last_became_active":"2024-04-25T10:07:31.112418+0000","last_became_peered":"2024-04-25T10:07:31.112418+0000","last_unstale":"2024-04-25T10:07:33.154036+0000","last_undegraded":"2024-04-25T10:07:33.154036+0000","last_fullsized":"2024-04-25T10:07:33.154036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:58.449857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.158409+0000","last_change":"2024-04-25T10:07:32.100093+0000","last_active":"2024-04-25T10:07:33.158409+0000","last_peered":"2024-04-25T10:07:33.158409+0000","last_clean":"2024-04-25T10:07:33.158409+0000","last_became_active":"2024-04-25T10:07:32.099726+0000","last_became_peered":"2024-04-25T10:07:32.099726+0000","last_unstale":"2024-04-25T10:07:33.158409+0000","last_undegraded":"2024-04-25T10:07:33.158409+0000","last_fullsized":"2024-04-25T10:07:33.158409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:57.478620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153495+0000","last_change":"2024-04-25T10:07:31.107356+0000","last_active":"2024-04-25T10:07:33.153495+0000","last_peered":"2024-04-25T10:07:33.153495+0000","last_clean":"2024-04-25T10:07:33.153495+0000","last_became_active":"2024-04-25T10:07:31.106159+0000","last_became_peered":"2024-04-25T10:07:31.106159+0000","last_unstale":"2024-04-25T10:07:33.153495+0000","last_undegraded":"2024-04-25T10:07:33.153495+0000","last_fullsized":"2024-04-25T10:07:33.153495+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:31.608191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075793+0000","last_change":"2024-04-25T10:07:32.086994+0000","last_active":"2024-04-25T10:07:33.075793+0000","last_peered":"2024-04-25T10:07:33.075793+0000","last_clean":"2024-04-25T10:07:33.075793+0000","last_became_active":"2024-04-25T10:07:32.086674+0000","last_became_peered":"2024-04-25T10:07:32.086674+0000","last_unstale":"2024-04-25T10:07:33.075793+0000","last_undegraded":"2024-04-25T10:07:33.075793+0000","last_fullsized":"2024-04-25T10:07:33.075793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:21.508561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075392+0000","last_change":"2024-04-25T10:07:31.097263+0000","last_active":"2024-04-25T10:07:33.075392+0000","last_peered":"2024-04-25T10:07:33.075392+0000","last_clean":"2024-04-25T10:07:33.075392+0000","last_became_active":"2024-04-25T10:07:31.096456+0000","last_became_peered":"2024-04-25T10:07:31.096456+0000","last_unstale":"2024-04-25T10:07:33.075392+0000","last_undegraded":"2024-04-25T10:07:33.075392+0000","last_fullsized":"2024-04-25T10:07:33.075392+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:28.785406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075532+0000","last_change":"2024-04-25T10:07:32.104697+0000","last_active":"2024-04-25T10:07:33.075532+0000","last_peered":"2024-04-25T10:07:33.075532+0000","last_clean":"2024-04-25T10:07:33.075532+0000","last_became_active":"2024-04-25T10:07:32.104465+0000","last_became_peered":"2024-04-25T10:07:32.104465+0000","last_unstale":"2024-04-25T10:07:33.075532+0000","last_undegraded":"2024-04-25T10:07:33.075532+0000","last_fullsized":"2024-04-25T10:07:33.075532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:20.489932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156807+0000","last_change":"2024-04-25T10:07:31.098031+0000","last_active":"2024-04-25T10:07:33.156807+0000","last_peered":"2024-04-25T10:07:33.156807+0000","last_clean":"2024-04-25T10:07:33.156807+0000","last_became_active":"2024-04-25T10:07:31.097749+0000","last_became_peered":"2024-04-25T10:07:31.097749+0000","last_unstale":"2024-04-25T10:07:33.156807+0000","last_undegraded":"2024-04-25T10:07:33.156807+0000","last_fullsized":"2024-04-25T10:07:33.156807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:54.261617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073810+0000","last_change":"2024-04-25T10:07:32.092218+0000","last_active":"2024-04-25T10:07:33.073810+0000","last_peered":"2024-04-25T10:07:33.073810+0000","last_clean":"2024-04-25T10:07:33.073810+0000","last_became_active":"2024-04-25T10:07:32.091578+0000","last_became_peered":"2024-04-25T10:07:32.091578+0000","last_unstale":"2024-04-25T10:07:33.073810+0000","last_undegraded":"2024-04-25T10:07:33.073810+0000","last_fullsized":"2024-04-25T10:07:33.073810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:49.367913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076032+0000","last_change":"2024-04-25T10:07:31.096266+0000","last_active":"2024-04-25T10:07:33.076032+0000","last_peered":"2024-04-25T10:07:33.076032+0000","last_clean":"2024-04-25T10:07:33.076032+0000","last_became_active":"2024-04-25T10:07:31.095542+0000","last_became_peered":"2024-04-25T10:07:31.095542+0000","last_unstale":"2024-04-25T10:07:33.076032+0000","last_undegraded":"2024-04-25T10:07:33.076032+0000","last_fullsized":"2024-04-25T10:07:33.076032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:01.537083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073802+0000","last_change":"2024-04-25T10:07:32.099290+0000","last_active":"2024-04-25T10:07:33.073802+0000","last_peered":"2024-04-25T10:07:33.073802+0000","last_clean":"2024-04-25T10:07:33.073802+0000","last_became_active":"2024-04-25T10:07:32.098956+0000","last_became_peered":"2024-04-25T10:07:32.098956+0000","last_unstale":"2024-04-25T10:07:33.073802+0000","last_undegraded":"2024-04-25T10:07:33.073802+0000","last_fullsized":"2024-04-25T10:07:33.073802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:38.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073355+0000","last_change":"2024-04-25T10:07:31.083842+0000","last_active":"2024-04-25T10:07:33.073355+0000","last_peered":"2024-04-25T10:07:33.073355+0000","last_clean":"2024-04-25T10:07:33.073355+0000","last_became_active":"2024-04-25T10:07:31.083524+0000","last_became_peered":"2024-04-25T10:07:31.083524+0000","last_unstale":"2024-04-25T10:07:33.073355+0000","last_undegraded":"2024-04-25T10:07:33.073355+0000","last_fullsized":"2024-04-25T10:07:33.073355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:45.906543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154070+0000","last_change":"2024-04-25T10:07:32.102529+0000","last_active":"2024-04-25T10:07:33.154070+0000","last_peered":"2024-04-25T10:07:33.154070+0000","last_clean":"2024-04-25T10:07:33.154070+0000","last_became_active":"2024-04-25T10:07:32.102138+0000","last_became_peered":"2024-04-25T10:07:32.102138+0000","last_unstale":"2024-04-25T10:07:33.154070+0000","last_undegraded":"2024-04-25T10:07:33.154070+0000","last_fullsized":"2024-04-25T10:07:33.154070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:04.630658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157268+0000","last_change":"2024-04-25T10:07:31.105590+0000","last_active":"2024-04-25T10:07:33.157268+0000","last_peered":"2024-04-25T10:07:33.157268+0000","last_clean":"2024-04-25T10:07:33.157268+0000","last_became_active":"2024-04-25T10:07:31.105154+0000","last_became_peered":"2024-04-25T10:07:31.105154+0000","last_unstale":"2024-04-25T10:07:33.157268+0000","last_undegraded":"2024-04-25T10:07:33.157268+0000","last_fullsized":"2024-04-25T10:07:33.157268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:31.608191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":284,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153887+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:07:33.153887+0000","last_peered":"2024-04-25T10:07:33.153887+0000","last_clean":"2024-04-25T10:07:33.153887+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:07:33.153887+0000","last_undegraded":"2024-04-25T10:07:33.153887+0000","last_fullsized":"2024-04-25T10:07:33.153887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50780,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321060,"statfs":{"total":96636764160,"available":96584765440,"internally_reserved":0,"allocated":1589248,"data_stored":1046871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1003520,"data_stored":458093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40480,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331360,"statfs":{"total":96636764160,"available":96595312640,"internally_reserved":0,"allocated":995328,"data_stored":456515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50216,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321624,"statfs":{"total":96636764160,"available":96585342976,"internally_reserved":0,"allocated":1011712,"data_stored":458197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46044,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325796,"statfs":{"total":96636764160,"available":96589615104,"internally_reserved":0,"allocated":995328,"data_stored":457063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1003520,"data_stored":457223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40476,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331364,"statfs":{"total":96636764160,"available":96595316736,"internally_reserved":0,"allocated":999424,"data_stored":456593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54940,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316900,"statfs":{"total":96636764160,"available":96580505600,"internally_reserved":0,"allocated":1585152,"data_stored":1046849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:36.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:36.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7b1fc3640 1 Processor -- start 2024-04-25T10:07:36.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7b1fc3640 1 -- start start 2024-04-25T10:07:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7b1fc3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff7ac0d0770 0x7ff7ac0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7b1fc3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac0ceb20 con 0x7ff7ac15db00 2024-04-25T10:07:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7b1fc3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac0b30a0 con 0x7ff7ac0b87e0 2024-04-25T10:07:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7b1fc3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ac0d1110 con 0x7ff7ac0d0770 2024-04-25T10:07:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7ab7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff7ac15db00 0x7ff7ac0b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52468/0 (socket says 172.21.15.133:52468) 2024-04-25T10:07:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7ab7fe640 1 -- 172.21.15.133:0/67921397 learned_addr learned my addr 172.21.15.133:0/67921397 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7a3fff640 1 --2- 172.21.15.133:0/67921397 >> v2:172.21.15.162:3300/0 conn(0x7ff7ac0d0770 0x7ff7ac0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882442686 0 0) 0x7ff7ac0ceb20 con 0x7ff7ac15db00 2024-04-25T10:07:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff798003530 con 0x7ff7ac15db00 2024-04-25T10:07:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2011141207 0 0) 0x7ff798003530 con 0x7ff7ac15db00 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 >> v2:172.21.15.162:3300/0 conn(0x7ff7ac0d0770 msgr2=0x7ff7ac0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 --2- 172.21.15.133:0/67921397 >> v2:172.21.15.162:3300/0 conn(0x7ff7ac0d0770 0x7ff7ac0d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 >> v1:172.21.15.162:6789/0 conn(0x7ff7ac0b87e0 legacy=0x7ff7ac0b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac0b7740 con 0x7ff7ac15db00 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff79c001680 con 0x7ff7ac15db00 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 >> v1:172.21.15.133:6789/0 conn(0x7ff7ac15db00 legacy=0x7ff7ac0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7980039c0 0x7ff798003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff798004810 0x7ff798004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7980051d0 0x7ff79800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79800c040 con 0x7ff7980051d0 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79800c1f0 con 0x7ff798004810 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79800c3f0 con 0x7ff7980039c0 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7abfff640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7980051d0 0x7ff79800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7ab7fe640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7980039c0 0x7ff798003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7a3fff640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff798004810 0x7ff798004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7abfff640 1 -- 172.21.15.133:0/67921397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7980039c0 msgr2=0x7ff798003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7abfff640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7980039c0 0x7ff798003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7abfff640 1 -- 172.21.15.133:0/67921397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff798004810 msgr2=0x7ff798004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7abfff640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff798004810 0x7ff798004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7abfff640 1 -- 172.21.15.133:0/67921397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff798003530 con 0x7ff7980051d0 2024-04-25T10:07:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.237+0000 7ff7ab7fe640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7980039c0 0x7ff798003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:36.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7abfff640 1 -- 172.21.15.133:0/67921397 reap_dead start 2024-04-25T10:07:36.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7abfff640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7980051d0 0x7ff79800b9d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4001e70 tx=0x7ff7a401c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5af2b99281d314 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4001a90 con 0x7ff7980051d0 2024-04-25T10:07:36.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a4003bc0 con 0x7ff7980051d0 2024-04-25T10:07:36.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7aaffd640 1 -- 172.21.15.133:0/67921397 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40a6040 con 0x7ff7980051d0 2024-04-25T10:07:36.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/67921397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7980051d0 msgr2=0x7ff79800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7980051d0 0x7ff79800b9d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4001e70 tx=0x7ff7a401c5a0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/67921397 shutdown_connections 2024-04-25T10:07:36.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 --2- 172.21.15.133:0/67921397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7980051d0 0x7ff79800b9d0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/67921397 >> 172.21.15.133:0/67921397 conn(0x7ff7ac0727c0 msgr2=0x7ff7ac072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/67921397 shutdown_connections 2024-04-25T10:07:36.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/67921397 wait complete. 2024-04-25T10:07:36.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 Processor -- start 2024-04-25T10:07:36.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 -- start start 2024-04-25T10:07:36.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7980051d0 0x7ff7ac15df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7ac15f820 0x7ff7ac15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0d2730 0x7ff7ac0d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac0b7fe0 con 0x7ff7ac0d2730 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac0b7d00 con 0x7ff7980051d0 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7b1fc3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac0b82c0 con 0x7ff7ac15f820 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.241+0000 7ff7abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0d2730 0x7ff7ac0d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7abfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0d2730 0x7ff7ac0d2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45166/0 (socket says 172.21.15.133:45166) 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7abfff640 1 -- 172.21.15.133:0/3190403183 learned_addr learned my addr 172.21.15.133:0/3190403183 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7a3fff640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7ac15f820 0x7ff7ac15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7abfff640 1 -- 172.21.15.133:0/3190403183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7ac15f820 msgr2=0x7ff7ac15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7abfff640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7ac15f820 0x7ff7ac15a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7ab7fe640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7980051d0 0x7ff7ac15df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7abfff640 1 -- 172.21.15.133:0/3190403183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7980051d0 msgr2=0x7ff7ac15df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7abfff640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7980051d0 0x7ff7ac15df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7abfff640 1 -- 172.21.15.133:0/3190403183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac15fca0 con 0x7ff7ac0d2730 2024-04-25T10:07:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7abfff640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0d2730 0x7ff7ac0d2bb0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff7a401ce40 tx=0x7ff798005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7a8ff9640 1 -- 172.21.15.133:0/3190403183 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40b8070 con 0x7ff7ac0d2730 2024-04-25T10:07:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7a8ff9640 1 -- 172.21.15.133:0/3190403183 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a4001260 con 0x7ff7ac0d2730 2024-04-25T10:07:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/3190403183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac15e640 con 0x7ff7ac0d2730 2024-04-25T10:07:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7a8ff9640 1 -- 172.21.15.133:0/3190403183 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40af070 con 0x7ff7ac0d2730 2024-04-25T10:07:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/3190403183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac15ec00 con 0x7ff7ac0d2730 2024-04-25T10:07:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/3190403183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ac0d30f0 con 0x7ff7ac0d2730 2024-04-25T10:07:36.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7a8ff9640 1 -- 172.21.15.133:0/3190403183 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7a4007030 con 0x7ff7ac0d2730 2024-04-25T10:07:36.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7a8ff9640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff78c066f60 0x7ff78c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7ab7fe640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff78c066f60 0x7ff78c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7a8ff9640 1 -- 172.21.15.133:0/3190403183 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7a411e230 con 0x7ff7ac0d2730 2024-04-25T10:07:36.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.245+0000 7ff7ab7fe640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff78c066f60 0x7ff78c069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff79c001210 tx=0x7ff79c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.249+0000 7ff7a8ff9640 1 -- 172.21.15.133:0/3190403183 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7ac0b30a0 con 0x7ff7ac0d2730 2024-04-25T10:07:36.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.393+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/3190403183 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7ac068ef0 con 0x7ff78c066f60 2024-04-25T10:07:36.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.401+0000 7ff7a8ff9640 1 -- 172.21.15.133:0/3190403183 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252075 (secure 0 0 0) 0x7ff7ac068ef0 con 0x7ff78c066f60 2024-04-25T10:07:36.410 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:36.411 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.405+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/3190403183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff78c066f60 msgr2=0x7ff78c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.405+0000 7ff7b1fc3640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff78c066f60 0x7ff78c069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff79c001210 tx=0x7ff79c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.405+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/3190403183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0d2730 msgr2=0x7ff7ac0d2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.405+0000 7ff7b1fc3640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0d2730 0x7ff7ac0d2bb0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff7a401ce40 tx=0x7ff798005840 comp rx=0 tx=0).stop 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.405+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/3190403183 shutdown_connections 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.405+0000 7ff7b1fc3640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff78c066f60 0x7ff78c069420 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.405+0000 7ff7b1fc3640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0d2730 0x7ff7ac0d2bb0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.405+0000 7ff7b1fc3640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7ac15f820 0x7ff7ac15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.405+0000 7ff7b1fc3640 1 --2- 172.21.15.133:0/3190403183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7980051d0 0x7ff7ac15df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.405+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/3190403183 >> 172.21.15.133:0/3190403183 conn(0x7ff7ac0727c0 msgr2=0x7ff7ac0d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.409+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/3190403183 shutdown_connections 2024-04-25T10:07:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.409+0000 7ff7b1fc3640 1 -- 172.21.15.133:0/3190403183 wait complete. 2024-04-25T10:07:36.427 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-04-25T10:07:35.811684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351424,"kb_used_data":8968,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754623296,"statfs":{"total":773094113280,"available":772734255104,"internally_reserved":0,"allocated":9183232,"data_stored":4837404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":350539313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.758453"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153668+0000","last_change":"2024-04-25T10:07:32.100331+0000","last_active":"2024-04-25T10:07:33.153668+0000","last_peered":"2024-04-25T10:07:33.153668+0000","last_clean":"2024-04-25T10:07:33.153668+0000","last_became_active":"2024-04-25T10:07:32.099806+0000","last_became_peered":"2024-04-25T10:07:32.099806+0000","last_unstale":"2024-04-25T10:07:33.153668+0000","last_undegraded":"2024-04-25T10:07:33.153668+0000","last_fullsized":"2024-04-25T10:07:33.153668+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:57.478620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156537+0000","last_change":"2024-04-25T10:07:32.102192+0000","last_active":"2024-04-25T10:07:33.156537+0000","last_peered":"2024-04-25T10:07:33.156537+0000","last_clean":"2024-04-25T10:07:33.156537+0000","last_became_active":"2024-04-25T10:07:32.101908+0000","last_became_peered":"2024-04-25T10:07:32.101908+0000","last_unstale":"2024-04-25T10:07:33.156537+0000","last_undegraded":"2024-04-25T10:07:33.156537+0000","last_fullsized":"2024-04-25T10:07:33.156537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:57.419880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073761+0000","last_change":"2024-04-25T10:07:32.081936+0000","last_active":"2024-04-25T10:07:33.073761+0000","last_peered":"2024-04-25T10:07:33.073761+0000","last_clean":"2024-04-25T10:07:33.073761+0000","last_became_active":"2024-04-25T10:07:32.081665+0000","last_became_peered":"2024-04-25T10:07:32.081665+0000","last_unstale":"2024-04-25T10:07:33.073761+0000","last_undegraded":"2024-04-25T10:07:33.073761+0000","last_fullsized":"2024-04-25T10:07:33.073761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:25.396490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152980+0000","last_change":"2024-04-25T10:07:32.084830+0000","last_active":"2024-04-25T10:07:33.152980+0000","last_peered":"2024-04-25T10:07:33.152980+0000","last_clean":"2024-04-25T10:07:33.152980+0000","last_became_active":"2024-04-25T10:07:32.084507+0000","last_became_peered":"2024-04-25T10:07:32.084507+0000","last_unstale":"2024-04-25T10:07:33.152980+0000","last_undegraded":"2024-04-25T10:07:33.152980+0000","last_fullsized":"2024-04-25T10:07:33.152980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:01.944592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157492+0000","last_change":"2024-04-25T10:07:32.101042+0000","last_active":"2024-04-25T10:07:33.157492+0000","last_peered":"2024-04-25T10:07:33.157492+0000","last_clean":"2024-04-25T10:07:33.157492+0000","last_became_active":"2024-04-25T10:07:32.098677+0000","last_became_peered":"2024-04-25T10:07:32.098677+0000","last_unstale":"2024-04-25T10:07:33.157492+0000","last_undegraded":"2024-04-25T10:07:33.157492+0000","last_fullsized":"2024-04-25T10:07:33.157492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:22.280577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156379+0000","last_change":"2024-04-25T10:07:32.106242+0000","last_active":"2024-04-25T10:07:33.156379+0000","last_peered":"2024-04-25T10:07:33.156379+0000","last_clean":"2024-04-25T10:07:33.156379+0000","last_became_active":"2024-04-25T10:07:32.105864+0000","last_became_peered":"2024-04-25T10:07:32.105864+0000","last_unstale":"2024-04-25T10:07:33.156379+0000","last_undegraded":"2024-04-25T10:07:33.156379+0000","last_fullsized":"2024-04-25T10:07:33.156379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:57.419880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075568+0000","last_change":"2024-04-25T10:07:32.093653+0000","last_active":"2024-04-25T10:07:33.075568+0000","last_peered":"2024-04-25T10:07:33.075568+0000","last_clean":"2024-04-25T10:07:33.075568+0000","last_became_active":"2024-04-25T10:07:32.093232+0000","last_became_peered":"2024-04-25T10:07:32.093232+0000","last_unstale":"2024-04-25T10:07:33.075568+0000","last_undegraded":"2024-04-25T10:07:33.075568+0000","last_fullsized":"2024-04-25T10:07:33.075568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:42.032766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154781+0000","last_change":"2024-04-25T10:07:32.105048+0000","last_active":"2024-04-25T10:07:33.154781+0000","last_peered":"2024-04-25T10:07:33.154781+0000","last_clean":"2024-04-25T10:07:33.154781+0000","last_became_active":"2024-04-25T10:07:32.104726+0000","last_became_peered":"2024-04-25T10:07:32.104726+0000","last_unstale":"2024-04-25T10:07:33.154781+0000","last_undegraded":"2024-04-25T10:07:33.154781+0000","last_fullsized":"2024-04-25T10:07:33.154781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:09.813690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076829+0000","last_change":"2024-04-25T10:07:32.093193+0000","last_active":"2024-04-25T10:07:33.076829+0000","last_peered":"2024-04-25T10:07:33.076829+0000","last_clean":"2024-04-25T10:07:33.076829+0000","last_became_active":"2024-04-25T10:07:32.092796+0000","last_became_peered":"2024-04-25T10:07:32.092796+0000","last_unstale":"2024-04-25T10:07:33.076829+0000","last_undegraded":"2024-04-25T10:07:33.076829+0000","last_fullsized":"2024-04-25T10:07:33.076829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:20.489932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153440+0000","last_change":"2024-04-25T10:07:32.085311+0000","last_active":"2024-04-25T10:07:33.153440+0000","last_peered":"2024-04-25T10:07:33.153440+0000","last_clean":"2024-04-25T10:07:33.153440+0000","last_became_active":"2024-04-25T10:07:32.085034+0000","last_became_peered":"2024-04-25T10:07:32.085034+0000","last_unstale":"2024-04-25T10:07:33.153440+0000","last_undegraded":"2024-04-25T10:07:33.153440+0000","last_fullsized":"2024-04-25T10:07:33.153440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:47.903283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156633+0000","last_change":"2024-04-25T10:07:32.111336+0000","last_active":"2024-04-25T10:07:33.156633+0000","last_peered":"2024-04-25T10:07:33.156633+0000","last_clean":"2024-04-25T10:07:33.156633+0000","last_became_active":"2024-04-25T10:07:32.109120+0000","last_became_peered":"2024-04-25T10:07:32.109120+0000","last_unstale":"2024-04-25T10:07:33.156633+0000","last_undegraded":"2024-04-25T10:07:33.156633+0000","last_fullsized":"2024-04-25T10:07:33.156633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:47.903283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153097+0000","last_change":"2024-04-25T10:07:32.108045+0000","last_active":"2024-04-25T10:07:33.153097+0000","last_peered":"2024-04-25T10:07:33.153097+0000","last_clean":"2024-04-25T10:07:33.153097+0000","last_became_active":"2024-04-25T10:07:32.107726+0000","last_became_peered":"2024-04-25T10:07:32.107726+0000","last_unstale":"2024-04-25T10:07:33.153097+0000","last_undegraded":"2024-04-25T10:07:33.153097+0000","last_fullsized":"2024-04-25T10:07:33.153097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:12.882528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157640+0000","last_change":"2024-04-25T10:07:32.095757+0000","last_active":"2024-04-25T10:07:33.157640+0000","last_peered":"2024-04-25T10:07:33.157640+0000","last_clean":"2024-04-25T10:07:33.157640+0000","last_became_active":"2024-04-25T10:07:32.095377+0000","last_became_peered":"2024-04-25T10:07:32.095377+0000","last_unstale":"2024-04-25T10:07:33.157640+0000","last_undegraded":"2024-04-25T10:07:33.157640+0000","last_fullsized":"2024-04-25T10:07:33.157640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:45.401815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156642+0000","last_change":"2024-04-25T10:07:32.105857+0000","last_active":"2024-04-25T10:07:33.156642+0000","last_peered":"2024-04-25T10:07:33.156642+0000","last_clean":"2024-04-25T10:07:33.156642+0000","last_became_active":"2024-04-25T10:07:32.105568+0000","last_became_peered":"2024-04-25T10:07:32.105568+0000","last_unstale":"2024-04-25T10:07:33.156642+0000","last_undegraded":"2024-04-25T10:07:33.156642+0000","last_fullsized":"2024-04-25T10:07:33.156642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:47.903283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153891+0000","last_change":"2024-04-25T10:07:32.090273+0000","last_active":"2024-04-25T10:07:33.153891+0000","last_peered":"2024-04-25T10:07:33.153891+0000","last_clean":"2024-04-25T10:07:33.153891+0000","last_became_active":"2024-04-25T10:07:32.089861+0000","last_became_peered":"2024-04-25T10:07:32.089861+0000","last_unstale":"2024-04-25T10:07:33.153891+0000","last_undegraded":"2024-04-25T10:07:33.153891+0000","last_fullsized":"2024-04-25T10:07:33.153891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:45.678236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153299+0000","last_change":"2024-04-25T10:07:32.099935+0000","last_active":"2024-04-25T10:07:33.153299+0000","last_peered":"2024-04-25T10:07:33.153299+0000","last_clean":"2024-04-25T10:07:33.153299+0000","last_became_active":"2024-04-25T10:07:32.095663+0000","last_became_peered":"2024-04-25T10:07:32.095663+0000","last_unstale":"2024-04-25T10:07:33.153299+0000","last_undegraded":"2024-04-25T10:07:33.153299+0000","last_fullsized":"2024-04-25T10:07:33.153299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:55.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155773+0000","last_change":"2024-04-25T10:07:32.090286+0000","last_active":"2024-04-25T10:07:33.155773+0000","last_peered":"2024-04-25T10:07:33.155773+0000","last_clean":"2024-04-25T10:07:33.155773+0000","last_became_active":"2024-04-25T10:07:32.087175+0000","last_became_peered":"2024-04-25T10:07:32.087175+0000","last_unstale":"2024-04-25T10:07:33.155773+0000","last_undegraded":"2024-04-25T10:07:33.155773+0000","last_fullsized":"2024-04-25T10:07:33.155773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:34.945854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073432+0000","last_change":"2024-04-25T10:07:32.083722+0000","last_active":"2024-04-25T10:07:33.073432+0000","last_peered":"2024-04-25T10:07:33.073432+0000","last_clean":"2024-04-25T10:07:33.073432+0000","last_became_active":"2024-04-25T10:07:32.083300+0000","last_became_peered":"2024-04-25T10:07:32.083300+0000","last_unstale":"2024-04-25T10:07:33.073432+0000","last_undegraded":"2024-04-25T10:07:33.073432+0000","last_fullsized":"2024-04-25T10:07:33.073432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:41.196995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155339+0000","last_change":"2024-04-25T10:07:32.107028+0000","last_active":"2024-04-25T10:07:33.155339+0000","last_peered":"2024-04-25T10:07:33.155339+0000","last_clean":"2024-04-25T10:07:33.155339+0000","last_became_active":"2024-04-25T10:07:32.106752+0000","last_became_peered":"2024-04-25T10:07:32.106752+0000","last_unstale":"2024-04-25T10:07:33.155339+0000","last_undegraded":"2024-04-25T10:07:33.155339+0000","last_fullsized":"2024-04-25T10:07:33.155339+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:12.882528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154806+0000","last_change":"2024-04-25T10:07:32.095882+0000","last_active":"2024-04-25T10:07:33.154806+0000","last_peered":"2024-04-25T10:07:33.154806+0000","last_clean":"2024-04-25T10:07:33.154806+0000","last_became_active":"2024-04-25T10:07:32.094624+0000","last_became_peered":"2024-04-25T10:07:32.094624+0000","last_unstale":"2024-04-25T10:07:33.154806+0000","last_undegraded":"2024-04-25T10:07:33.154806+0000","last_fullsized":"2024-04-25T10:07:33.154806+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:49.142104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075489+0000","last_change":"2024-04-25T10:07:32.077540+0000","last_active":"2024-04-25T10:07:33.075489+0000","last_peered":"2024-04-25T10:07:33.075489+0000","last_clean":"2024-04-25T10:07:33.075489+0000","last_became_active":"2024-04-25T10:07:32.077241+0000","last_became_peered":"2024-04-25T10:07:32.077241+0000","last_unstale":"2024-04-25T10:07:33.075489+0000","last_undegraded":"2024-04-25T10:07:33.075489+0000","last_fullsized":"2024-04-25T10:07:33.075489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:06.140735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156195+0000","last_change":"2024-04-25T10:07:32.109140+0000","last_active":"2024-04-25T10:07:33.156195+0000","last_peered":"2024-04-25T10:07:33.156195+0000","last_clean":"2024-04-25T10:07:33.156195+0000","last_became_active":"2024-04-25T10:07:32.108924+0000","last_became_peered":"2024-04-25T10:07:32.108924+0000","last_unstale":"2024-04-25T10:07:33.156195+0000","last_undegraded":"2024-04-25T10:07:33.156195+0000","last_fullsized":"2024-04-25T10:07:33.156195+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:50.958157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.074734+0000","last_change":"2024-04-25T10:07:32.093599+0000","last_active":"2024-04-25T10:07:33.074734+0000","last_peered":"2024-04-25T10:07:33.074734+0000","last_clean":"2024-04-25T10:07:33.074734+0000","last_became_active":"2024-04-25T10:07:32.093293+0000","last_became_peered":"2024-04-25T10:07:32.093293+0000","last_unstale":"2024-04-25T10:07:33.074734+0000","last_undegraded":"2024-04-25T10:07:33.074734+0000","last_fullsized":"2024-04-25T10:07:33.074734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:35.139055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153825+0000","last_change":"2024-04-25T10:07:32.089675+0000","last_active":"2024-04-25T10:07:33.153825+0000","last_peered":"2024-04-25T10:07:33.153825+0000","last_clean":"2024-04-25T10:07:33.153825+0000","last_became_active":"2024-04-25T10:07:32.089296+0000","last_became_peered":"2024-04-25T10:07:32.089296+0000","last_unstale":"2024-04-25T10:07:33.153825+0000","last_undegraded":"2024-04-25T10:07:33.153825+0000","last_fullsized":"2024-04-25T10:07:33.153825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:48.477318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156497+0000","last_change":"2024-04-25T10:07:32.089589+0000","last_active":"2024-04-25T10:07:33.156497+0000","last_peered":"2024-04-25T10:07:33.156497+0000","last_clean":"2024-04-25T10:07:33.156497+0000","last_became_active":"2024-04-25T10:07:32.089270+0000","last_became_peered":"2024-04-25T10:07:32.089270+0000","last_unstale":"2024-04-25T10:07:33.156497+0000","last_undegraded":"2024-04-25T10:07:33.156497+0000","last_fullsized":"2024-04-25T10:07:33.156497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:12.882528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153880+0000","last_change":"2024-04-25T10:07:32.108202+0000","last_active":"2024-04-25T10:07:33.153880+0000","last_peered":"2024-04-25T10:07:33.153880+0000","last_clean":"2024-04-25T10:07:33.153880+0000","last_became_active":"2024-04-25T10:07:32.107866+0000","last_became_peered":"2024-04-25T10:07:32.107866+0000","last_unstale":"2024-04-25T10:07:33.153880+0000","last_undegraded":"2024-04-25T10:07:33.153880+0000","last_fullsized":"2024-04-25T10:07:33.153880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:49.849575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153623+0000","last_change":"2024-04-25T10:07:32.092178+0000","last_active":"2024-04-25T10:07:33.153623+0000","last_peered":"2024-04-25T10:07:33.153623+0000","last_clean":"2024-04-25T10:07:33.153623+0000","last_became_active":"2024-04-25T10:07:32.091959+0000","last_became_peered":"2024-04-25T10:07:32.091959+0000","last_unstale":"2024-04-25T10:07:33.153623+0000","last_undegraded":"2024-04-25T10:07:33.153623+0000","last_fullsized":"2024-04-25T10:07:33.153623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:57.568078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155775+0000","last_change":"2024-04-25T10:07:32.103842+0000","last_active":"2024-04-25T10:07:33.155775+0000","last_peered":"2024-04-25T10:07:33.155775+0000","last_clean":"2024-04-25T10:07:33.155775+0000","last_became_active":"2024-04-25T10:07:32.103556+0000","last_became_peered":"2024-04-25T10:07:32.103556+0000","last_unstale":"2024-04-25T10:07:33.155775+0000","last_undegraded":"2024-04-25T10:07:33.155775+0000","last_fullsized":"2024-04-25T10:07:33.155775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:06.816471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.158265+0000","last_change":"2024-04-25T10:07:32.082247+0000","last_active":"2024-04-25T10:07:33.158265+0000","last_peered":"2024-04-25T10:07:33.158265+0000","last_clean":"2024-04-25T10:07:33.158265+0000","last_became_active":"2024-04-25T10:07:32.081696+0000","last_became_peered":"2024-04-25T10:07:32.081696+0000","last_unstale":"2024-04-25T10:07:33.158265+0000","last_undegraded":"2024-04-25T10:07:33.158265+0000","last_fullsized":"2024-04-25T10:07:33.158265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:57.568078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153965+0000","last_change":"2024-04-25T10:07:32.087474+0000","last_active":"2024-04-25T10:07:33.153965+0000","last_peered":"2024-04-25T10:07:33.153965+0000","last_clean":"2024-04-25T10:07:33.153965+0000","last_became_active":"2024-04-25T10:07:32.087182+0000","last_became_peered":"2024-04-25T10:07:32.087182+0000","last_unstale":"2024-04-25T10:07:33.153965+0000","last_undegraded":"2024-04-25T10:07:33.153965+0000","last_fullsized":"2024-04-25T10:07:33.153965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:15.883375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073447+0000","last_change":"2024-04-25T10:07:32.076817+0000","last_active":"2024-04-25T10:07:33.073447+0000","last_peered":"2024-04-25T10:07:33.073447+0000","last_clean":"2024-04-25T10:07:33.073447+0000","last_became_active":"2024-04-25T10:07:32.076366+0000","last_became_peered":"2024-04-25T10:07:32.076366+0000","last_unstale":"2024-04-25T10:07:33.073447+0000","last_undegraded":"2024-04-25T10:07:33.073447+0000","last_fullsized":"2024-04-25T10:07:33.073447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:38.771120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153755+0000","last_change":"2024-04-25T10:07:32.100218+0000","last_active":"2024-04-25T10:07:33.153755+0000","last_peered":"2024-04-25T10:07:33.153755+0000","last_clean":"2024-04-25T10:07:33.153755+0000","last_became_active":"2024-04-25T10:07:32.099843+0000","last_became_peered":"2024-04-25T10:07:32.099843+0000","last_unstale":"2024-04-25T10:07:33.153755+0000","last_undegraded":"2024-04-25T10:07:33.153755+0000","last_fullsized":"2024-04-25T10:07:33.153755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:50.290103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073232+0000","last_change":"2024-04-25T10:07:32.076936+0000","last_active":"2024-04-25T10:07:33.073232+0000","last_peered":"2024-04-25T10:07:33.073232+0000","last_clean":"2024-04-25T10:07:33.073232+0000","last_became_active":"2024-04-25T10:07:32.076632+0000","last_became_peered":"2024-04-25T10:07:32.076632+0000","last_unstale":"2024-04-25T10:07:33.073232+0000","last_undegraded":"2024-04-25T10:07:33.073232+0000","last_fullsized":"2024-04-25T10:07:33.073232+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:39.095604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156047+0000","last_change":"2024-04-25T10:07:31.092554+0000","last_active":"2024-04-25T10:07:33.156047+0000","last_peered":"2024-04-25T10:07:33.156047+0000","last_clean":"2024-04-25T10:07:33.156047+0000","last_became_active":"2024-04-25T10:07:31.092166+0000","last_became_peered":"2024-04-25T10:07:31.092166+0000","last_unstale":"2024-04-25T10:07:33.156047+0000","last_undegraded":"2024-04-25T10:07:33.156047+0000","last_fullsized":"2024-04-25T10:07:33.156047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:09.890364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154284+0000","last_change":"2024-04-25T10:07:32.097077+0000","last_active":"2024-04-25T10:07:33.154284+0000","last_peered":"2024-04-25T10:07:33.154284+0000","last_clean":"2024-04-25T10:07:33.154284+0000","last_became_active":"2024-04-25T10:07:32.096710+0000","last_became_peered":"2024-04-25T10:07:32.096710+0000","last_unstale":"2024-04-25T10:07:33.154284+0000","last_undegraded":"2024-04-25T10:07:33.154284+0000","last_fullsized":"2024-04-25T10:07:33.154284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:59.457468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152955+0000","last_change":"2024-04-25T10:07:31.106518+0000","last_active":"2024-04-25T10:07:33.152955+0000","last_peered":"2024-04-25T10:07:33.152955+0000","last_clean":"2024-04-25T10:07:33.152955+0000","last_became_active":"2024-04-25T10:07:31.105838+0000","last_became_peered":"2024-04-25T10:07:31.105838+0000","last_unstale":"2024-04-25T10:07:33.152955+0000","last_undegraded":"2024-04-25T10:07:33.152955+0000","last_fullsized":"2024-04-25T10:07:33.152955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:39.402806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075162+0000","last_change":"2024-04-25T10:07:32.093742+0000","last_active":"2024-04-25T10:07:33.075162+0000","last_peered":"2024-04-25T10:07:33.075162+0000","last_clean":"2024-04-25T10:07:33.075162+0000","last_became_active":"2024-04-25T10:07:32.093380+0000","last_became_peered":"2024-04-25T10:07:32.093380+0000","last_unstale":"2024-04-25T10:07:33.075162+0000","last_undegraded":"2024-04-25T10:07:33.075162+0000","last_fullsized":"2024-04-25T10:07:33.075162+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:45.199003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152878+0000","last_change":"2024-04-25T10:07:31.097506+0000","last_active":"2024-04-25T10:07:33.152878+0000","last_peered":"2024-04-25T10:07:33.152878+0000","last_clean":"2024-04-25T10:07:33.152878+0000","last_became_active":"2024-04-25T10:07:31.094832+0000","last_became_peered":"2024-04-25T10:07:31.094832+0000","last_unstale":"2024-04-25T10:07:33.152878+0000","last_undegraded":"2024-04-25T10:07:33.152878+0000","last_fullsized":"2024-04-25T10:07:33.152878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:10.481037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075857+0000","last_change":"2024-04-25T10:07:32.077794+0000","last_active":"2024-04-25T10:07:33.075857+0000","last_peered":"2024-04-25T10:07:33.075857+0000","last_clean":"2024-04-25T10:07:33.075857+0000","last_became_active":"2024-04-25T10:07:32.077393+0000","last_became_peered":"2024-04-25T10:07:32.077393+0000","last_unstale":"2024-04-25T10:07:33.075857+0000","last_undegraded":"2024-04-25T10:07:33.075857+0000","last_fullsized":"2024-04-25T10:07:33.075857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:46.914154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075904+0000","last_change":"2024-04-25T10:07:31.102307+0000","last_active":"2024-04-25T10:07:33.075904+0000","last_peered":"2024-04-25T10:07:33.075904+0000","last_clean":"2024-04-25T10:07:33.075904+0000","last_became_active":"2024-04-25T10:07:31.096754+0000","last_became_peered":"2024-04-25T10:07:31.096754+0000","last_unstale":"2024-04-25T10:07:33.075904+0000","last_undegraded":"2024-04-25T10:07:33.075904+0000","last_fullsized":"2024-04-25T10:07:33.075904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:14.416210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156399+0000","last_change":"2024-04-25T10:07:32.109189+0000","last_active":"2024-04-25T10:07:33.156399+0000","last_peered":"2024-04-25T10:07:33.156399+0000","last_clean":"2024-04-25T10:07:33.156399+0000","last_became_active":"2024-04-25T10:07:32.108915+0000","last_became_peered":"2024-04-25T10:07:32.108915+0000","last_unstale":"2024-04-25T10:07:33.156399+0000","last_undegraded":"2024-04-25T10:07:33.156399+0000","last_fullsized":"2024-04-25T10:07:33.156399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:01.944592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076114+0000","last_change":"2024-04-25T10:07:31.092118+0000","last_active":"2024-04-25T10:07:33.076114+0000","last_peered":"2024-04-25T10:07:33.076114+0000","last_clean":"2024-04-25T10:07:33.076114+0000","last_became_active":"2024-04-25T10:07:31.091348+0000","last_became_peered":"2024-04-25T10:07:31.091348+0000","last_unstale":"2024-04-25T10:07:33.076114+0000","last_undegraded":"2024-04-25T10:07:33.076114+0000","last_fullsized":"2024-04-25T10:07:33.076114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:27.052688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157080+0000","last_change":"2024-04-25T10:07:32.102446+0000","last_active":"2024-04-25T10:07:33.157080+0000","last_peered":"2024-04-25T10:07:33.157080+0000","last_clean":"2024-04-25T10:07:33.157080+0000","last_became_active":"2024-04-25T10:07:32.102078+0000","last_became_peered":"2024-04-25T10:07:32.102078+0000","last_unstale":"2024-04-25T10:07:33.157080+0000","last_undegraded":"2024-04-25T10:07:33.157080+0000","last_fullsized":"2024-04-25T10:07:33.157080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:55.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152444+0000","last_change":"2024-04-25T10:07:31.102989+0000","last_active":"2024-04-25T10:07:33.152444+0000","last_peered":"2024-04-25T10:07:33.152444+0000","last_clean":"2024-04-25T10:07:33.152444+0000","last_became_active":"2024-04-25T10:07:31.101610+0000","last_became_peered":"2024-04-25T10:07:31.101610+0000","last_unstale":"2024-04-25T10:07:33.152444+0000","last_undegraded":"2024-04-25T10:07:33.152444+0000","last_fullsized":"2024-04-25T10:07:33.152444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.461805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076563+0000","last_change":"2024-04-25T10:07:32.093028+0000","last_active":"2024-04-25T10:07:33.076563+0000","last_peered":"2024-04-25T10:07:33.076563+0000","last_clean":"2024-04-25T10:07:33.076563+0000","last_became_active":"2024-04-25T10:07:32.092695+0000","last_became_peered":"2024-04-25T10:07:32.092695+0000","last_unstale":"2024-04-25T10:07:33.076563+0000","last_undegraded":"2024-04-25T10:07:33.076563+0000","last_fullsized":"2024-04-25T10:07:33.076563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:02.130848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155698+0000","last_change":"2024-04-25T10:07:31.091316+0000","last_active":"2024-04-25T10:07:33.155698+0000","last_peered":"2024-04-25T10:07:33.155698+0000","last_clean":"2024-04-25T10:07:33.155698+0000","last_became_active":"2024-04-25T10:07:31.090936+0000","last_became_peered":"2024-04-25T10:07:31.090936+0000","last_unstale":"2024-04-25T10:07:33.155698+0000","last_undegraded":"2024-04-25T10:07:33.155698+0000","last_fullsized":"2024-04-25T10:07:33.155698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:37.197045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076031+0000","last_change":"2024-04-25T10:07:32.090395+0000","last_active":"2024-04-25T10:07:33.076031+0000","last_peered":"2024-04-25T10:07:33.076031+0000","last_clean":"2024-04-25T10:07:33.076031+0000","last_became_active":"2024-04-25T10:07:32.090046+0000","last_became_peered":"2024-04-25T10:07:32.090046+0000","last_unstale":"2024-04-25T10:07:33.076031+0000","last_undegraded":"2024-04-25T10:07:33.076031+0000","last_fullsized":"2024-04-25T10:07:33.076031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:29.015009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154892+0000","last_change":"2024-04-25T10:07:31.087810+0000","last_active":"2024-04-25T10:07:33.154892+0000","last_peered":"2024-04-25T10:07:33.154892+0000","last_clean":"2024-04-25T10:07:33.154892+0000","last_became_active":"2024-04-25T10:07:31.087517+0000","last_became_peered":"2024-04-25T10:07:31.087517+0000","last_unstale":"2024-04-25T10:07:33.154892+0000","last_undegraded":"2024-04-25T10:07:33.154892+0000","last_fullsized":"2024-04-25T10:07:33.154892+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:51.033707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153879+0000","last_change":"2024-04-25T10:07:32.096917+0000","last_active":"2024-04-25T10:07:33.153879+0000","last_peered":"2024-04-25T10:07:33.153879+0000","last_clean":"2024-04-25T10:07:33.153879+0000","last_became_active":"2024-04-25T10:07:32.096534+0000","last_became_peered":"2024-04-25T10:07:32.096534+0000","last_unstale":"2024-04-25T10:07:33.153879+0000","last_undegraded":"2024-04-25T10:07:33.153879+0000","last_fullsized":"2024-04-25T10:07:33.153879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:11.488648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154153+0000","last_change":"2024-04-25T10:07:31.106890+0000","last_active":"2024-04-25T10:07:33.154153+0000","last_peered":"2024-04-25T10:07:33.154153+0000","last_clean":"2024-04-25T10:07:33.154153+0000","last_became_active":"2024-04-25T10:07:31.106095+0000","last_became_peered":"2024-04-25T10:07:31.106095+0000","last_unstale":"2024-04-25T10:07:33.154153+0000","last_undegraded":"2024-04-25T10:07:33.154153+0000","last_fullsized":"2024-04-25T10:07:33.154153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:55.233331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152689+0000","last_change":"2024-04-25T10:07:32.086663+0000","last_active":"2024-04-25T10:07:33.152689+0000","last_peered":"2024-04-25T10:07:33.152689+0000","last_clean":"2024-04-25T10:07:33.152689+0000","last_became_active":"2024-04-25T10:07:32.086249+0000","last_became_peered":"2024-04-25T10:07:32.086249+0000","last_unstale":"2024-04-25T10:07:33.152689+0000","last_undegraded":"2024-04-25T10:07:33.152689+0000","last_fullsized":"2024-04-25T10:07:33.152689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:59.457468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153927+0000","last_change":"2024-04-25T10:07:31.101866+0000","last_active":"2024-04-25T10:07:33.153927+0000","last_peered":"2024-04-25T10:07:33.153927+0000","last_clean":"2024-04-25T10:07:33.153927+0000","last_became_active":"2024-04-25T10:07:31.101115+0000","last_became_peered":"2024-04-25T10:07:31.101115+0000","last_unstale":"2024-04-25T10:07:33.153927+0000","last_undegraded":"2024-04-25T10:07:33.153927+0000","last_fullsized":"2024-04-25T10:07:33.153927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:59.552554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157007+0000","last_change":"2024-04-25T10:07:32.094098+0000","last_active":"2024-04-25T10:07:33.157007+0000","last_peered":"2024-04-25T10:07:33.157007+0000","last_clean":"2024-04-25T10:07:33.157007+0000","last_became_active":"2024-04-25T10:07:32.093723+0000","last_became_peered":"2024-04-25T10:07:32.093723+0000","last_unstale":"2024-04-25T10:07:33.157007+0000","last_undegraded":"2024-04-25T10:07:33.157007+0000","last_fullsized":"2024-04-25T10:07:33.157007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:49.849575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"80'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.158015+0000","last_change":"2024-04-25T10:07:31.091614+0000","last_active":"2024-04-25T10:07:33.158015+0000","last_peered":"2024-04-25T10:07:33.158015+0000","last_clean":"2024-04-25T10:07:33.158015+0000","last_became_active":"2024-04-25T10:07:31.091152+0000","last_became_peered":"2024-04-25T10:07:31.091152+0000","last_unstale":"2024-04-25T10:07:33.158015+0000","last_undegraded":"2024-04-25T10:07:33.158015+0000","last_fullsized":"2024-04-25T10:07:33.158015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:56.331401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.074279+0000","last_change":"2024-04-25T10:07:32.078952+0000","last_active":"2024-04-25T10:07:33.074279+0000","last_peered":"2024-04-25T10:07:33.074279+0000","last_clean":"2024-04-25T10:07:33.074279+0000","last_became_active":"2024-04-25T10:07:32.078578+0000","last_became_peered":"2024-04-25T10:07:32.078578+0000","last_unstale":"2024-04-25T10:07:33.074279+0000","last_undegraded":"2024-04-25T10:07:33.074279+0000","last_fullsized":"2024-04-25T10:07:33.074279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:37.983770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075928+0000","last_change":"2024-04-25T10:07:31.095833+0000","last_active":"2024-04-25T10:07:33.075928+0000","last_peered":"2024-04-25T10:07:33.075928+0000","last_clean":"2024-04-25T10:07:33.075928+0000","last_became_active":"2024-04-25T10:07:31.095046+0000","last_became_peered":"2024-04-25T10:07:31.095046+0000","last_unstale":"2024-04-25T10:07:33.075928+0000","last_undegraded":"2024-04-25T10:07:33.075928+0000","last_fullsized":"2024-04-25T10:07:33.075928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.952601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153371+0000","last_change":"2024-04-25T10:07:32.091498+0000","last_active":"2024-04-25T10:07:33.153371+0000","last_peered":"2024-04-25T10:07:33.153371+0000","last_clean":"2024-04-25T10:07:33.153371+0000","last_became_active":"2024-04-25T10:07:32.091204+0000","last_became_peered":"2024-04-25T10:07:32.091204+0000","last_unstale":"2024-04-25T10:07:33.153371+0000","last_undegraded":"2024-04-25T10:07:33.153371+0000","last_fullsized":"2024-04-25T10:07:33.153371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:57.419880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.152470+0000","last_change":"2024-04-25T10:07:31.109685+0000","last_active":"2024-04-25T10:07:33.152470+0000","last_peered":"2024-04-25T10:07:33.152470+0000","last_clean":"2024-04-25T10:07:33.152470+0000","last_became_active":"2024-04-25T10:07:31.108946+0000","last_became_peered":"2024-04-25T10:07:31.108946+0000","last_unstale":"2024-04-25T10:07:33.152470+0000","last_undegraded":"2024-04-25T10:07:33.152470+0000","last_fullsized":"2024-04-25T10:07:33.152470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:55.466116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075379+0000","last_change":"2024-04-25T10:07:32.076111+0000","last_active":"2024-04-25T10:07:33.075379+0000","last_peered":"2024-04-25T10:07:33.075379+0000","last_clean":"2024-04-25T10:07:33.075379+0000","last_became_active":"2024-04-25T10:07:32.075778+0000","last_became_peered":"2024-04-25T10:07:32.075778+0000","last_unstale":"2024-04-25T10:07:33.075379+0000","last_undegraded":"2024-04-25T10:07:33.075379+0000","last_fullsized":"2024-04-25T10:07:33.075379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:02.130848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153961+0000","last_change":"2024-04-25T10:07:31.094378+0000","last_active":"2024-04-25T10:07:33.153961+0000","last_peered":"2024-04-25T10:07:33.153961+0000","last_clean":"2024-04-25T10:07:33.153961+0000","last_became_active":"2024-04-25T10:07:31.093533+0000","last_became_peered":"2024-04-25T10:07:31.093533+0000","last_unstale":"2024-04-25T10:07:33.153961+0000","last_undegraded":"2024-04-25T10:07:33.153961+0000","last_fullsized":"2024-04-25T10:07:33.153961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:20.145193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075825+0000","last_change":"2024-04-25T10:07:32.079413+0000","last_active":"2024-04-25T10:07:33.075825+0000","last_peered":"2024-04-25T10:07:33.075825+0000","last_clean":"2024-04-25T10:07:33.075825+0000","last_became_active":"2024-04-25T10:07:32.079023+0000","last_became_peered":"2024-04-25T10:07:32.079023+0000","last_unstale":"2024-04-25T10:07:33.075825+0000","last_undegraded":"2024-04-25T10:07:33.075825+0000","last_fullsized":"2024-04-25T10:07:33.075825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:58.256908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154825+0000","last_change":"2024-04-25T10:07:31.097619+0000","last_active":"2024-04-25T10:07:33.154825+0000","last_peered":"2024-04-25T10:07:33.154825+0000","last_clean":"2024-04-25T10:07:33.154825+0000","last_became_active":"2024-04-25T10:07:31.097231+0000","last_became_peered":"2024-04-25T10:07:31.097231+0000","last_unstale":"2024-04-25T10:07:33.154825+0000","last_undegraded":"2024-04-25T10:07:33.154825+0000","last_fullsized":"2024-04-25T10:07:33.154825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:56.626555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076153+0000","last_change":"2024-04-25T10:07:32.087885+0000","last_active":"2024-04-25T10:07:33.076153+0000","last_peered":"2024-04-25T10:07:33.076153+0000","last_clean":"2024-04-25T10:07:33.076153+0000","last_became_active":"2024-04-25T10:07:32.087502+0000","last_became_peered":"2024-04-25T10:07:32.087502+0000","last_unstale":"2024-04-25T10:07:33.076153+0000","last_undegraded":"2024-04-25T10:07:33.076153+0000","last_fullsized":"2024-04-25T10:07:33.076153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:08.019664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154122+0000","last_change":"2024-04-25T10:07:31.094407+0000","last_active":"2024-04-25T10:07:33.154122+0000","last_peered":"2024-04-25T10:07:33.154122+0000","last_clean":"2024-04-25T10:07:33.154122+0000","last_became_active":"2024-04-25T10:07:31.094124+0000","last_became_peered":"2024-04-25T10:07:31.094124+0000","last_unstale":"2024-04-25T10:07:33.154122+0000","last_undegraded":"2024-04-25T10:07:33.154122+0000","last_fullsized":"2024-04-25T10:07:33.154122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:48.360303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155124+0000","last_change":"2024-04-25T10:07:32.102602+0000","last_active":"2024-04-25T10:07:33.155124+0000","last_peered":"2024-04-25T10:07:33.155124+0000","last_clean":"2024-04-25T10:07:33.155124+0000","last_became_active":"2024-04-25T10:07:32.095878+0000","last_became_peered":"2024-04-25T10:07:32.095878+0000","last_unstale":"2024-04-25T10:07:33.155124+0000","last_undegraded":"2024-04-25T10:07:33.155124+0000","last_fullsized":"2024-04-25T10:07:33.155124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:01.944592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153957+0000","last_change":"2024-04-25T10:07:31.099468+0000","last_active":"2024-04-25T10:07:33.153957+0000","last_peered":"2024-04-25T10:07:33.153957+0000","last_clean":"2024-04-25T10:07:33.153957+0000","last_became_active":"2024-04-25T10:07:31.098663+0000","last_became_peered":"2024-04-25T10:07:31.098663+0000","last_unstale":"2024-04-25T10:07:33.153957+0000","last_undegraded":"2024-04-25T10:07:33.153957+0000","last_fullsized":"2024-04-25T10:07:33.153957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:47.469708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154871+0000","last_change":"2024-04-25T10:07:32.098705+0000","last_active":"2024-04-25T10:07:33.154871+0000","last_peered":"2024-04-25T10:07:33.154871+0000","last_clean":"2024-04-25T10:07:33.154871+0000","last_became_active":"2024-04-25T10:07:32.096026+0000","last_became_peered":"2024-04-25T10:07:32.096026+0000","last_unstale":"2024-04-25T10:07:33.154871+0000","last_undegraded":"2024-04-25T10:07:33.154871+0000","last_fullsized":"2024-04-25T10:07:33.154871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:16.588134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076739+0000","last_change":"2024-04-25T10:07:31.080057+0000","last_active":"2024-04-25T10:07:33.076739+0000","last_peered":"2024-04-25T10:07:33.076739+0000","last_clean":"2024-04-25T10:07:33.076739+0000","last_became_active":"2024-04-25T10:07:31.079109+0000","last_became_peered":"2024-04-25T10:07:31.079109+0000","last_unstale":"2024-04-25T10:07:33.076739+0000","last_undegraded":"2024-04-25T10:07:33.076739+0000","last_fullsized":"2024-04-25T10:07:33.076739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:42.632287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156569+0000","last_change":"2024-04-25T10:07:32.110754+0000","last_active":"2024-04-25T10:07:33.156569+0000","last_peered":"2024-04-25T10:07:33.156569+0000","last_clean":"2024-04-25T10:07:33.156569+0000","last_became_active":"2024-04-25T10:07:32.110503+0000","last_became_peered":"2024-04-25T10:07:32.110503+0000","last_unstale":"2024-04-25T10:07:33.156569+0000","last_undegraded":"2024-04-25T10:07:33.156569+0000","last_fullsized":"2024-04-25T10:07:33.156569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:32.767768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.158125+0000","last_change":"2024-04-25T10:07:31.090657+0000","last_active":"2024-04-25T10:07:33.158125+0000","last_peered":"2024-04-25T10:07:33.158125+0000","last_clean":"2024-04-25T10:07:33.158125+0000","last_became_active":"2024-04-25T10:07:31.090165+0000","last_became_peered":"2024-04-25T10:07:31.090165+0000","last_unstale":"2024-04-25T10:07:33.158125+0000","last_undegraded":"2024-04-25T10:07:33.158125+0000","last_fullsized":"2024-04-25T10:07:33.158125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:46.784630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157018+0000","last_change":"2024-04-25T10:07:32.093132+0000","last_active":"2024-04-25T10:07:33.157018+0000","last_peered":"2024-04-25T10:07:33.157018+0000","last_clean":"2024-04-25T10:07:33.157018+0000","last_became_active":"2024-04-25T10:07:32.092926+0000","last_became_peered":"2024-04-25T10:07:32.092926+0000","last_unstale":"2024-04-25T10:07:33.157018+0000","last_undegraded":"2024-04-25T10:07:33.157018+0000","last_fullsized":"2024-04-25T10:07:33.157018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:47.334242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073127+0000","last_change":"2024-04-25T10:07:31.088937+0000","last_active":"2024-04-25T10:07:33.073127+0000","last_peered":"2024-04-25T10:07:33.073127+0000","last_clean":"2024-04-25T10:07:33.073127+0000","last_became_active":"2024-04-25T10:07:31.088572+0000","last_became_peered":"2024-04-25T10:07:31.088572+0000","last_unstale":"2024-04-25T10:07:33.073127+0000","last_undegraded":"2024-04-25T10:07:33.073127+0000","last_fullsized":"2024-04-25T10:07:33.073127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:01.123238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156732+0000","last_change":"2024-04-25T10:07:32.097016+0000","last_active":"2024-04-25T10:07:33.156732+0000","last_peered":"2024-04-25T10:07:33.156732+0000","last_clean":"2024-04-25T10:07:33.156732+0000","last_became_active":"2024-04-25T10:07:32.096656+0000","last_became_peered":"2024-04-25T10:07:32.096656+0000","last_unstale":"2024-04-25T10:07:33.156732+0000","last_undegraded":"2024-04-25T10:07:33.156732+0000","last_fullsized":"2024-04-25T10:07:33.156732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:49.849575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157187+0000","last_change":"2024-04-25T10:07:31.087429+0000","last_active":"2024-04-25T10:07:33.157187+0000","last_peered":"2024-04-25T10:07:33.157187+0000","last_clean":"2024-04-25T10:07:33.157187+0000","last_became_active":"2024-04-25T10:07:31.086584+0000","last_became_peered":"2024-04-25T10:07:31.086584+0000","last_unstale":"2024-04-25T10:07:33.157187+0000","last_undegraded":"2024-04-25T10:07:33.157187+0000","last_fullsized":"2024-04-25T10:07:33.157187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:17.461805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157050+0000","last_change":"2024-04-25T10:07:32.095990+0000","last_active":"2024-04-25T10:07:33.157050+0000","last_peered":"2024-04-25T10:07:33.157050+0000","last_clean":"2024-04-25T10:07:33.157050+0000","last_became_active":"2024-04-25T10:07:32.090719+0000","last_became_peered":"2024-04-25T10:07:32.090719+0000","last_unstale":"2024-04-25T10:07:33.157050+0000","last_undegraded":"2024-04-25T10:07:33.157050+0000","last_fullsized":"2024-04-25T10:07:33.157050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:47.334242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.158512+0000","last_change":"2024-04-25T10:07:31.105499+0000","last_active":"2024-04-25T10:07:33.158512+0000","last_peered":"2024-04-25T10:07:33.158512+0000","last_clean":"2024-04-25T10:07:33.158512+0000","last_became_active":"2024-04-25T10:07:31.105190+0000","last_became_peered":"2024-04-25T10:07:31.105190+0000","last_unstale":"2024-04-25T10:07:33.158512+0000","last_undegraded":"2024-04-25T10:07:33.158512+0000","last_fullsized":"2024-04-25T10:07:33.158512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:59.552554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076289+0000","last_change":"2024-04-25T10:07:32.082247+0000","last_active":"2024-04-25T10:07:33.076289+0000","last_peered":"2024-04-25T10:07:33.076289+0000","last_clean":"2024-04-25T10:07:33.076289+0000","last_became_active":"2024-04-25T10:07:32.081900+0000","last_became_peered":"2024-04-25T10:07:32.081900+0000","last_unstale":"2024-04-25T10:07:33.076289+0000","last_undegraded":"2024-04-25T10:07:33.076289+0000","last_fullsized":"2024-04-25T10:07:33.076289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:20.966999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.155554+0000","last_change":"2024-04-25T10:07:31.105897+0000","last_active":"2024-04-25T10:07:33.155554+0000","last_peered":"2024-04-25T10:07:33.155554+0000","last_clean":"2024-04-25T10:07:33.155554+0000","last_became_active":"2024-04-25T10:07:31.105600+0000","last_became_peered":"2024-04-25T10:07:31.105600+0000","last_unstale":"2024-04-25T10:07:33.155554+0000","last_undegraded":"2024-04-25T10:07:33.155554+0000","last_fullsized":"2024-04-25T10:07:33.155554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:20.145193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.074647+0000","last_change":"2024-04-25T10:07:32.074618+0000","last_active":"2024-04-25T10:07:33.074647+0000","last_peered":"2024-04-25T10:07:33.074647+0000","last_clean":"2024-04-25T10:07:33.074647+0000","last_became_active":"2024-04-25T10:07:32.074237+0000","last_became_peered":"2024-04-25T10:07:32.074237+0000","last_unstale":"2024-04-25T10:07:33.074647+0000","last_undegraded":"2024-04-25T10:07:33.074647+0000","last_fullsized":"2024-04-25T10:07:33.074647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:46.235215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153464+0000","last_change":"2024-04-25T10:07:31.086154+0000","last_active":"2024-04-25T10:07:33.153464+0000","last_peered":"2024-04-25T10:07:33.153464+0000","last_clean":"2024-04-25T10:07:33.153464+0000","last_became_active":"2024-04-25T10:07:31.085162+0000","last_became_peered":"2024-04-25T10:07:31.085162+0000","last_unstale":"2024-04-25T10:07:33.153464+0000","last_undegraded":"2024-04-25T10:07:33.153464+0000","last_fullsized":"2024-04-25T10:07:33.153464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:04.962615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076161+0000","last_change":"2024-04-25T10:07:32.087704+0000","last_active":"2024-04-25T10:07:33.076161+0000","last_peered":"2024-04-25T10:07:33.076161+0000","last_clean":"2024-04-25T10:07:33.076161+0000","last_became_active":"2024-04-25T10:07:32.087297+0000","last_became_peered":"2024-04-25T10:07:32.087297+0000","last_unstale":"2024-04-25T10:07:33.076161+0000","last_undegraded":"2024-04-25T10:07:33.076161+0000","last_fullsized":"2024-04-25T10:07:33.076161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:17.011308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:34.080205+0000","last_change":"2024-04-25T10:07:31.093532+0000","last_active":"2024-04-25T10:07:34.080205+0000","last_peered":"2024-04-25T10:07:34.080205+0000","last_clean":"2024-04-25T10:07:34.080205+0000","last_became_active":"2024-04-25T10:07:31.093066+0000","last_became_peered":"2024-04-25T10:07:31.093066+0000","last_unstale":"2024-04-25T10:07:34.080205+0000","last_undegraded":"2024-04-25T10:07:34.080205+0000","last_fullsized":"2024-04-25T10:07:34.080205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:36.976160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075377+0000","last_change":"2024-04-25T10:07:32.094310+0000","last_active":"2024-04-25T10:07:33.075377+0000","last_peered":"2024-04-25T10:07:33.075377+0000","last_clean":"2024-04-25T10:07:33.075377+0000","last_became_active":"2024-04-25T10:07:32.094002+0000","last_became_peered":"2024-04-25T10:07:32.094002+0000","last_unstale":"2024-04-25T10:07:33.075377+0000","last_undegraded":"2024-04-25T10:07:33.075377+0000","last_fullsized":"2024-04-25T10:07:33.075377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:10.902889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154036+0000","last_change":"2024-04-25T10:07:31.112884+0000","last_active":"2024-04-25T10:07:33.154036+0000","last_peered":"2024-04-25T10:07:33.154036+0000","last_clean":"2024-04-25T10:07:33.154036+0000","last_became_active":"2024-04-25T10:07:31.112418+0000","last_became_peered":"2024-04-25T10:07:31.112418+0000","last_unstale":"2024-04-25T10:07:33.154036+0000","last_undegraded":"2024-04-25T10:07:33.154036+0000","last_fullsized":"2024-04-25T10:07:33.154036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:58.449857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.158409+0000","last_change":"2024-04-25T10:07:32.100093+0000","last_active":"2024-04-25T10:07:33.158409+0000","last_peered":"2024-04-25T10:07:33.158409+0000","last_clean":"2024-04-25T10:07:33.158409+0000","last_became_active":"2024-04-25T10:07:32.099726+0000","last_became_peered":"2024-04-25T10:07:32.099726+0000","last_unstale":"2024-04-25T10:07:33.158409+0000","last_undegraded":"2024-04-25T10:07:33.158409+0000","last_fullsized":"2024-04-25T10:07:33.158409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:57.478620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153495+0000","last_change":"2024-04-25T10:07:31.107356+0000","last_active":"2024-04-25T10:07:33.153495+0000","last_peered":"2024-04-25T10:07:33.153495+0000","last_clean":"2024-04-25T10:07:33.153495+0000","last_became_active":"2024-04-25T10:07:31.106159+0000","last_became_peered":"2024-04-25T10:07:31.106159+0000","last_unstale":"2024-04-25T10:07:33.153495+0000","last_undegraded":"2024-04-25T10:07:33.153495+0000","last_fullsized":"2024-04-25T10:07:33.153495+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:31.608191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075793+0000","last_change":"2024-04-25T10:07:32.086994+0000","last_active":"2024-04-25T10:07:33.075793+0000","last_peered":"2024-04-25T10:07:33.075793+0000","last_clean":"2024-04-25T10:07:33.075793+0000","last_became_active":"2024-04-25T10:07:32.086674+0000","last_became_peered":"2024-04-25T10:07:32.086674+0000","last_unstale":"2024-04-25T10:07:33.075793+0000","last_undegraded":"2024-04-25T10:07:33.075793+0000","last_fullsized":"2024-04-25T10:07:33.075793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:21.508561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075392+0000","last_change":"2024-04-25T10:07:31.097263+0000","last_active":"2024-04-25T10:07:33.075392+0000","last_peered":"2024-04-25T10:07:33.075392+0000","last_clean":"2024-04-25T10:07:33.075392+0000","last_became_active":"2024-04-25T10:07:31.096456+0000","last_became_peered":"2024-04-25T10:07:31.096456+0000","last_unstale":"2024-04-25T10:07:33.075392+0000","last_undegraded":"2024-04-25T10:07:33.075392+0000","last_fullsized":"2024-04-25T10:07:33.075392+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:28.785406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.075532+0000","last_change":"2024-04-25T10:07:32.104697+0000","last_active":"2024-04-25T10:07:33.075532+0000","last_peered":"2024-04-25T10:07:33.075532+0000","last_clean":"2024-04-25T10:07:33.075532+0000","last_became_active":"2024-04-25T10:07:32.104465+0000","last_became_peered":"2024-04-25T10:07:32.104465+0000","last_unstale":"2024-04-25T10:07:33.075532+0000","last_undegraded":"2024-04-25T10:07:33.075532+0000","last_fullsized":"2024-04-25T10:07:33.075532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:20.489932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.156807+0000","last_change":"2024-04-25T10:07:31.098031+0000","last_active":"2024-04-25T10:07:33.156807+0000","last_peered":"2024-04-25T10:07:33.156807+0000","last_clean":"2024-04-25T10:07:33.156807+0000","last_became_active":"2024-04-25T10:07:31.097749+0000","last_became_peered":"2024-04-25T10:07:31.097749+0000","last_unstale":"2024-04-25T10:07:33.156807+0000","last_undegraded":"2024-04-25T10:07:33.156807+0000","last_fullsized":"2024-04-25T10:07:33.156807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:54.261617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073810+0000","last_change":"2024-04-25T10:07:32.092218+0000","last_active":"2024-04-25T10:07:33.073810+0000","last_peered":"2024-04-25T10:07:33.073810+0000","last_clean":"2024-04-25T10:07:33.073810+0000","last_became_active":"2024-04-25T10:07:32.091578+0000","last_became_peered":"2024-04-25T10:07:32.091578+0000","last_unstale":"2024-04-25T10:07:33.073810+0000","last_undegraded":"2024-04-25T10:07:33.073810+0000","last_fullsized":"2024-04-25T10:07:33.073810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:49.367913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.076032+0000","last_change":"2024-04-25T10:07:31.096266+0000","last_active":"2024-04-25T10:07:33.076032+0000","last_peered":"2024-04-25T10:07:33.076032+0000","last_clean":"2024-04-25T10:07:33.076032+0000","last_became_active":"2024-04-25T10:07:31.095542+0000","last_became_peered":"2024-04-25T10:07:31.095542+0000","last_unstale":"2024-04-25T10:07:33.076032+0000","last_undegraded":"2024-04-25T10:07:33.076032+0000","last_fullsized":"2024-04-25T10:07:33.076032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:01.537083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073802+0000","last_change":"2024-04-25T10:07:32.099290+0000","last_active":"2024-04-25T10:07:33.073802+0000","last_peered":"2024-04-25T10:07:33.073802+0000","last_clean":"2024-04-25T10:07:33.073802+0000","last_became_active":"2024-04-25T10:07:32.098956+0000","last_became_peered":"2024-04-25T10:07:32.098956+0000","last_unstale":"2024-04-25T10:07:33.073802+0000","last_undegraded":"2024-04-25T10:07:33.073802+0000","last_fullsized":"2024-04-25T10:07:33.073802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:38.204656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.073355+0000","last_change":"2024-04-25T10:07:31.083842+0000","last_active":"2024-04-25T10:07:33.073355+0000","last_peered":"2024-04-25T10:07:33.073355+0000","last_clean":"2024-04-25T10:07:33.073355+0000","last_became_active":"2024-04-25T10:07:31.083524+0000","last_became_peered":"2024-04-25T10:07:31.083524+0000","last_unstale":"2024-04-25T10:07:33.073355+0000","last_undegraded":"2024-04-25T10:07:33.073355+0000","last_fullsized":"2024-04-25T10:07:33.073355+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:45.906543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.154070+0000","last_change":"2024-04-25T10:07:32.102529+0000","last_active":"2024-04-25T10:07:33.154070+0000","last_peered":"2024-04-25T10:07:33.154070+0000","last_clean":"2024-04-25T10:07:33.154070+0000","last_became_active":"2024-04-25T10:07:32.102138+0000","last_became_peered":"2024-04-25T10:07:32.102138+0000","last_unstale":"2024-04-25T10:07:33.154070+0000","last_undegraded":"2024-04-25T10:07:33.154070+0000","last_fullsized":"2024-04-25T10:07:33.154070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:31.052617+0000","last_clean_scrub_stamp":"2024-04-25T10:07:31.052617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:04.630658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.157268+0000","last_change":"2024-04-25T10:07:31.105590+0000","last_active":"2024-04-25T10:07:33.157268+0000","last_peered":"2024-04-25T10:07:33.157268+0000","last_clean":"2024-04-25T10:07:33.157268+0000","last_became_active":"2024-04-25T10:07:31.105154+0000","last_became_peered":"2024-04-25T10:07:31.105154+0000","last_unstale":"2024-04-25T10:07:33.157268+0000","last_undegraded":"2024-04-25T10:07:33.157268+0000","last_fullsized":"2024-04-25T10:07:33.157268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:30.045007+0000","last_clean_scrub_stamp":"2024-04-25T10:07:30.045007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:31.608191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":284,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:07:33.153887+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:07:33.153887+0000","last_peered":"2024-04-25T10:07:33.153887+0000","last_clean":"2024-04-25T10:07:33.153887+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:07:33.153887+0000","last_undegraded":"2024-04-25T10:07:33.153887+0000","last_fullsized":"2024-04-25T10:07:33.153887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50780,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321060,"statfs":{"total":96636764160,"available":96584765440,"internally_reserved":0,"allocated":1589248,"data_stored":1046871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1003520,"data_stored":458093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40480,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331360,"statfs":{"total":96636764160,"available":96595312640,"internally_reserved":0,"allocated":995328,"data_stored":456515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50216,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321624,"statfs":{"total":96636764160,"available":96585342976,"internally_reserved":0,"allocated":1011712,"data_stored":458197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46044,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325796,"statfs":{"total":96636764160,"available":96589615104,"internally_reserved":0,"allocated":995328,"data_stored":457063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1003520,"data_stored":457223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40476,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331364,"statfs":{"total":96636764160,"available":96595316736,"internally_reserved":0,"allocated":999424,"data_stored":456593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54940,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316900,"statfs":{"total":96636764160,"available":96580505600,"internally_reserved":0,"allocated":1585152,"data_stored":1046849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:36.431 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:07:36.431 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:07:36.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa403acd640 1 Processor -- start 2024-04-25T10:07:36.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa403acd640 1 -- start start 2024-04-25T10:07:36.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa403acd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0ce990 0x7fa3fc0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa403acd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa3fc0ced70 0x7fa3fc0ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa403acd640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3fc0cb2e0 con 0x7fa3fc0ce1e0 2024-04-25T10:07:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa403acd640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3fc0cb5e0 con 0x7fa3fc0ced70 2024-04-25T10:07:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa403acd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0cd920 con 0x7fa3fc0ce990 2024-04-25T10:07:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa401068640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa3fc0ced70 0x7fa3fc0ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa401068640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa3fc0ced70 0x7fa3fc0ad330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45176/0 (socket says 172.21.15.133:45176) 2024-04-25T10:07:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa401068640 1 -- 172.21.15.133:0/2539071885 learned_addr learned my addr 172.21.15.133:0/2539071885 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa401869640 1 --2- 172.21.15.133:0/2539071885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0ce990 0x7fa3fc0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa401068640 1 -- 172.21.15.133:0/2539071885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0ce990 msgr2=0x7fa3fc0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa401068640 1 --2- 172.21.15.133:0/2539071885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0ce990 0x7fa3fc0d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa401068640 1 -- 172.21.15.133:0/2539071885 >> v1:172.21.15.162:6789/0 conn(0x7fa3fc0ce1e0 legacy=0x7fa3fc15da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.581+0000 7fa401068640 1 -- 172.21.15.133:0/2539071885 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc0cdbc0 con 0x7fa3fc0ced70 2024-04-25T10:07:36.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa401068640 1 --2- 172.21.15.133:0/2539071885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc0ced70 0x7fa3fc0ad330 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009870 tx=0x7fa3ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964a4a63758d545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa400867640 1 -- 172.21.15.133:0/2539071885 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0a7070 con 0x7fa3fc0ced70 2024-04-25T10:07:36.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa400867640 1 -- 172.21.15.133:0/2539071885 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec0b0070 con 0x7fa3fc0ced70 2024-04-25T10:07:36.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa400867640 1 -- 172.21.15.133:0/2539071885 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0ab890 con 0x7fa3fc0ced70 2024-04-25T10:07:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa403acd640 1 -- 172.21.15.133:0/2539071885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc0ced70 msgr2=0x7fa3fc0ad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa403acd640 1 --2- 172.21.15.133:0/2539071885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc0ced70 0x7fa3fc0ad330 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009870 tx=0x7fa3ec01c490 comp rx=0 tx=0).stop 2024-04-25T10:07:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa403acd640 1 -- 172.21.15.133:0/2539071885 shutdown_connections 2024-04-25T10:07:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa403acd640 1 --2- 172.21.15.133:0/2539071885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc0ced70 0x7fa3fc0ad330 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa403acd640 1 --2- 172.21.15.133:0/2539071885 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0ce990 0x7fa3fc0d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa403acd640 1 -- 172.21.15.133:0/2539071885 >> 172.21.15.133:0/2539071885 conn(0x7fa3fc0999d0 msgr2=0x7fa3fc0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa403acd640 1 -- 172.21.15.133:0/2539071885 shutdown_connections 2024-04-25T10:07:36.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa403acd640 1 -- 172.21.15.133:0/2539071885 wait complete. 2024-04-25T10:07:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa403acd640 1 Processor -- start 2024-04-25T10:07:36.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.585+0000 7fa403acd640 1 -- start start 2024-04-25T10:07:36.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa403acd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc0ce1e0 0x7fa3fc155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa403acd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0ce990 0x7fa3fc154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa403acd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc0ced70 0x7fa3fc0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa403acd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0cc3e0 con 0x7fa3fc0ced70 2024-04-25T10:07:36.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa403acd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0cc260 con 0x7fa3fc0ce1e0 2024-04-25T10:07:36.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa403acd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc167960 con 0x7fa3fc0ce990 2024-04-25T10:07:36.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa40206a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc0ced70 0x7fa3fc0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa40206a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc0ced70 0x7fa3fc0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45188/0 (socket says 172.21.15.133:45188) 2024-04-25T10:07:36.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa40206a640 1 -- 172.21.15.133:0/2546318891 learned_addr learned my addr 172.21.15.133:0/2546318891 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:36.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa401869640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc0ce1e0 0x7fa3fc155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa401068640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0ce990 0x7fa3fc154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa40206a640 1 -- 172.21.15.133:0/2546318891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0ce990 msgr2=0x7fa3fc154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa40206a640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0ce990 0x7fa3fc154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa40206a640 1 -- 172.21.15.133:0/2546318891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc0ce1e0 msgr2=0x7fa3fc155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa40206a640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc0ce1e0 0x7fa3fc155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa40206a640 1 -- 172.21.15.133:0/2546318891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc156680 con 0x7fa3fc0ced70 2024-04-25T10:07:36.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa401869640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc0ce1e0 0x7fa3fc155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:36.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa40206a640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc0ced70 0x7fa3fc0cc8d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800b970 tx=0x7fa3f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa3f27fc640 1 -- 172.21.15.133:0/2546318891 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8019070 con 0x7fa3fc0ced70 2024-04-25T10:07:36.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa3f27fc640 1 -- 172.21.15.133:0/2546318891 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f800cb40 con 0x7fa3fc0ced70 2024-04-25T10:07:36.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa403acd640 1 -- 172.21.15.133:0/2546318891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc155b00 con 0x7fa3fc0ced70 2024-04-25T10:07:36.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa3f27fc640 1 -- 172.21.15.133:0/2546318891 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8014440 con 0x7fa3fc0ced70 2024-04-25T10:07:36.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa403acd640 1 -- 172.21.15.133:0/2546318891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc1560c0 con 0x7fa3fc0ced70 2024-04-25T10:07:36.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa403acd640 1 -- 172.21.15.133:0/2546318891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3fc0c8e20 con 0x7fa3fc0ced70 2024-04-25T10:07:36.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa3f27fc640 1 -- 172.21.15.133:0/2546318891 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3f801c070 con 0x7fa3fc0ced70 2024-04-25T10:07:36.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa3f27fc640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e0066f90 0x7fa3e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa401869640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e0066f90 0x7fa3e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.589+0000 7fa3f27fc640 1 -- 172.21.15.133:0/2546318891 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3f8084e60 con 0x7fa3fc0ced70 2024-04-25T10:07:36.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.593+0000 7fa401869640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e0066f90 0x7fa3e0069450 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4006fd0 tx=0x7fa3f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.597+0000 7fa3f27fc640 1 -- 172.21.15.133:0/2546318891 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3f8055c40 con 0x7fa3fc0ced70 2024-04-25T10:07:36.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.813+0000 7fa403acd640 1 -- 172.21.15.133:0/2546318891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa3fc067c70 con 0x7fa3fc0ced70 2024-04-25T10:07:36.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.817+0000 7fa3f27fc640 1 -- 172.21.15.133:0/2546318891 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa3f8055540 con 0x7fa3fc0ced70 2024-04-25T10:07:36.821 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:07:36.822 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:07:36.822 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:07:36.822 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:07:36.822 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:07:36.822 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:07:36.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.817+0000 7fa403acd640 1 -- 172.21.15.133:0/2546318891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e0066f90 msgr2=0x7fa3e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.817+0000 7fa403acd640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e0066f90 0x7fa3e0069450 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4006fd0 tx=0x7fa3f4008040 comp rx=0 tx=0).stop 2024-04-25T10:07:36.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.817+0000 7fa403acd640 1 -- 172.21.15.133:0/2546318891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc0ced70 msgr2=0x7fa3fc0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.817+0000 7fa403acd640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc0ced70 0x7fa3fc0cc8d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800b970 tx=0x7fa3f800be30 comp rx=0 tx=0).stop 2024-04-25T10:07:36.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.817+0000 7fa403acd640 1 -- 172.21.15.133:0/2546318891 shutdown_connections 2024-04-25T10:07:36.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.817+0000 7fa403acd640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3e0066f90 0x7fa3e0069450 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.817+0000 7fa403acd640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc0ced70 0x7fa3fc0cc8d0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.817+0000 7fa403acd640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3fc0ce990 0x7fa3fc154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.817+0000 7fa403acd640 1 --2- 172.21.15.133:0/2546318891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3fc0ce1e0 0x7fa3fc155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.817+0000 7fa403acd640 1 -- 172.21.15.133:0/2546318891 >> 172.21.15.133:0/2546318891 conn(0x7fa3fc0999d0 msgr2=0x7fa3fc0aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.821+0000 7fa403acd640 1 -- 172.21.15.133:0/2546318891 shutdown_connections 2024-04-25T10:07:36.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.821+0000 7fa403acd640 1 -- 172.21.15.133:0/2546318891 wait complete. 2024-04-25T10:07:36.840 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:36.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a68b4c640 1 Processor -- start 2024-04-25T10:07:36.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a68b4c640 1 -- start start 2024-04-25T10:07:36.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a68b4c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6a6406eb70 0x7f6a640b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a68b4c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a641584f0 0x7f6a6415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a68b4c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a640cbe60 con 0x7f6a64158110 2024-04-25T10:07:36.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a68b4c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a640b19b0 con 0x7f6a6406eb70 2024-04-25T10:07:36.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a68b4c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6415b360 con 0x7f6a641584f0 2024-04-25T10:07:36.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a6259c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6a64158110 0x7f6a6406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52472/0 (socket says 172.21.15.133:52472) 2024-04-25T10:07:36.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a6259c640 1 -- 172.21.15.133:0/113582804 learned_addr learned my addr 172.21.15.133:0/113582804 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:36.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a5bfff640 1 --2- 172.21.15.133:0/113582804 >> v2:172.21.15.133:3300/0 conn(0x7f6a6406eb70 0x7f6a640b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a5bfff640 1 -- 172.21.15.133:0/113582804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a641584f0 msgr2=0x7f6a6415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:36.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a5bfff640 1 --2- 172.21.15.133:0/113582804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a641584f0 0x7f6a6415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a5bfff640 1 -- 172.21.15.133:0/113582804 >> v1:172.21.15.133:6789/0 conn(0x7f6a64158110 legacy=0x7f6a6406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a5bfff640 1 -- 172.21.15.133:0/113582804 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6415b640 con 0x7f6a6406eb70 2024-04-25T10:07:36.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a61d9b640 1 -- 172.21.15.133:0/113582804 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918275769 0 0) 0x7f6a640cbe60 con 0x7f6a64158110 2024-04-25T10:07:36.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.941+0000 7f6a5bfff640 1 --2- 172.21.15.133:0/113582804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6406eb70 0x7f6a640b1f00 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0097c0 tx=0x7f6a4c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a1395862f43870 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a61d9b640 1 -- 172.21.15.133:0/113582804 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0b2070 con 0x7f6a6406eb70 2024-04-25T10:07:36.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a61d9b640 1 -- 172.21.15.133:0/113582804 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c0ad640 con 0x7f6a6406eb70 2024-04-25T10:07:36.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a61d9b640 1 -- 172.21.15.133:0/113582804 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0ada10 con 0x7f6a6406eb70 2024-04-25T10:07:36.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 -- 172.21.15.133:0/113582804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6406eb70 msgr2=0x7f6a640b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 --2- 172.21.15.133:0/113582804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6406eb70 0x7f6a640b1f00 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0097c0 tx=0x7f6a4c01c470 comp rx=0 tx=0).stop 2024-04-25T10:07:36.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 -- 172.21.15.133:0/113582804 shutdown_connections 2024-04-25T10:07:36.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 --2- 172.21.15.133:0/113582804 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a641584f0 0x7f6a6415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 --2- 172.21.15.133:0/113582804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6406eb70 0x7f6a640b1f00 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 -- 172.21.15.133:0/113582804 >> 172.21.15.133:0/113582804 conn(0x7f6a64067000 msgr2=0x7f6a6406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:36.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 -- 172.21.15.133:0/113582804 shutdown_connections 2024-04-25T10:07:36.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 -- 172.21.15.133:0/113582804 wait complete. 2024-04-25T10:07:36.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 Processor -- start 2024-04-25T10:07:36.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 -- start start 2024-04-25T10:07:36.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6406eb70 0x7f6a6414cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64158110 0x7f6a6414c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a641584f0 0x7f6a64151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6406dce0 con 0x7f6a6406eb70 2024-04-25T10:07:36.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6406db60 con 0x7f6a641584f0 2024-04-25T10:07:36.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6406de60 con 0x7f6a64158110 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a6259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6406eb70 0x7f6a6414cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a6259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6406eb70 0x7f6a6414cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45204/0 (socket says 172.21.15.133:45204) 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a6259c640 1 -- 172.21.15.133:0/980605439 learned_addr learned my addr 172.21.15.133:0/980605439 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a5bfff640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64158110 0x7f6a6414c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a62d9d640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a641584f0 0x7f6a64151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a6259c640 1 -- 172.21.15.133:0/980605439 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64158110 msgr2=0x7f6a6414c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a6259c640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64158110 0x7f6a6414c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a6259c640 1 -- 172.21.15.133:0/980605439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a641584f0 msgr2=0x7f6a64151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a6259c640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a641584f0 0x7f6a64151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a6259c640 1 -- 172.21.15.133:0/980605439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6414ca50 con 0x7f6a6406eb70 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a6259c640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6406eb70 0x7f6a6414cc90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6a540011f0 tx=0x7f6a5400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a5affd640 1 -- 172.21.15.133:0/980605439 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54018070 con 0x7f6a6406eb70 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a5affd640 1 -- 172.21.15.133:0/980605439 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a540074e0 con 0x7f6a6406eb70 2024-04-25T10:07:36.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 -- 172.21.15.133:0/980605439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a641578b0 con 0x7f6a6406eb70 2024-04-25T10:07:36.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a5affd640 1 -- 172.21.15.133:0/980605439 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54004a10 con 0x7f6a6406eb70 2024-04-25T10:07:36.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.945+0000 7f6a68b4c640 1 -- 172.21.15.133:0/980605439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6415d930 con 0x7f6a6406eb70 2024-04-25T10:07:36.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.949+0000 7f6a68b4c640 1 -- 172.21.15.133:0/980605439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6414f550 con 0x7f6a6406eb70 2024-04-25T10:07:36.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.949+0000 7f6a5affd640 1 -- 172.21.15.133:0/980605439 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a5401b070 con 0x7f6a6406eb70 2024-04-25T10:07:36.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.949+0000 7f6a5affd640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a44066f60 0x7f6a44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:36.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.949+0000 7f6a5affd640 1 -- 172.21.15.133:0/980605439 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a54084470 con 0x7f6a6406eb70 2024-04-25T10:07:36.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.949+0000 7f6a5bfff640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a44066f60 0x7f6a44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:36.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.949+0000 7f6a5bfff640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a44066f60 0x7f6a44069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0af000 tx=0x7f6a4c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:36.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:36.953+0000 7f6a5affd640 1 -- 172.21.15.133:0/980605439 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a6414f550 con 0x7f6a6406eb70 2024-04-25T10:07:37.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.141+0000 7f6a68b4c640 1 -- 172.21.15.133:0/980605439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a64158930 con 0x7f6a6406eb70 2024-04-25T10:07:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.141+0000 7f6a5affd640 1 -- 172.21.15.133:0/980605439 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f6a54055380 con 0x7f6a6406eb70 2024-04-25T10:07:37.148 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:07:37.148 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3522294955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3522294955},{"type":"v1","addr":"172.21.15.162:6833","nonce":3522294955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3865231260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3865231260},{"type":"v1","addr":"172.21.15.133:6836","nonce":3865231260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7437,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3935311931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3935311931},{"type":"v1","addr":"172.21.15.133:6837","nonce":3935311931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:31.434780+0000","modified":"2024-04-25T10:07:34.465411+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/3192312240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3192312240},{"type":"v1","addr":"172.21.15.162:6835","nonce":3192312240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 49 2024-04-25T10:07:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 -- 172.21.15.133:0/980605439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a44066f60 msgr2=0x7f6a44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a44066f60 0x7f6a44069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0af000 tx=0x7f6a4c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:07:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 -- 172.21.15.133:0/980605439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6406eb70 msgr2=0x7f6a6414cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6406eb70 0x7f6a6414cc90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6a540011f0 tx=0x7f6a5400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 -- 172.21.15.133:0/980605439 shutdown_connections 2024-04-25T10:07:37.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a641584f0 0x7f6a64151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a64158110 0x7f6a6414c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a44066f60 0x7f6a44069420 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 --2- 172.21.15.133:0/980605439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6406eb70 0x7f6a6414cc90 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 -- 172.21.15.133:0/980605439 >> 172.21.15.133:0/980605439 conn(0x7f6a64067000 msgr2=0x7f6a640ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 -- 172.21.15.133:0/980605439 shutdown_connections 2024-04-25T10:07:37.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.145+0000 7f6a68b4c640 1 -- 172.21.15.133:0/980605439 wait complete. 2024-04-25T10:07:37.568 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:31.434780+0000', 'modified': '2024-04-25T10:07:34.465411+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7449': {'gid': 7449, 'name': 'd', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6835/3192312240', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6834', 'nonce': 3192312240}, {'type': 'v1', 'addr': '172.21.15.162:6835', 'nonce': 3192312240}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:37.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:07:37.569 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:07:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f4409b640 1 Processor -- start 2024-04-25T10:07:37.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f4409b640 1 -- start start 2024-04-25T10:07:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f4409b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f3c15d9a0 0x7f2f3c0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f4409b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2f3c0ce600 0x7f2f3c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f4409b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 0x7f2f3c0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f4409b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f3c15ebd0 con 0x7f2f3c15d9a0 2024-04-25T10:07:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f4409b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f3c15eeb0 con 0x7f2f3c0ce600 2024-04-25T10:07:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f4409b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0d0330 con 0x7f2f3c0d1920 2024-04-25T10:07:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f41e37640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f3c15d9a0 0x7f2f3c0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f41e37640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f3c15d9a0 0x7f2f3c0d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45220/0 (socket says 172.21.15.133:45220) 2024-04-25T10:07:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f41e37640 1 -- 172.21.15.133:0/3696999280 learned_addr learned my addr 172.21.15.133:0/3696999280 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f41636640 1 --2- 172.21.15.133:0/3696999280 >> v2:172.21.15.162:3300/0 conn(0x7f2f3c0ce600 0x7f2f3c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f42638640 1 --2- 172.21.15.133:0/3696999280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 0x7f2f3c0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f41e37640 1 -- 172.21.15.133:0/3696999280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 msgr2=0x7f2f3c0d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f41e37640 1 --2- 172.21.15.133:0/3696999280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 0x7f2f3c0d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f41e37640 1 -- 172.21.15.133:0/3696999280 >> v2:172.21.15.162:3300/0 conn(0x7f2f3c0ce600 msgr2=0x7f2f3c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f41e37640 1 --2- 172.21.15.133:0/3696999280 >> v2:172.21.15.162:3300/0 conn(0x7f2f3c0ce600 0x7f2f3c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f41e37640 1 -- 172.21.15.133:0/3696999280 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0d0610 con 0x7f2f3c15d9a0 2024-04-25T10:07:37.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f41636640 1 --2- 172.21.15.133:0/3696999280 >> v2:172.21.15.162:3300/0 conn(0x7f2f3c0ce600 0x7f2f3c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:37.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.693+0000 7f2f42638640 1 --2- 172.21.15.133:0/3696999280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 0x7f2f3c0d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f41e37640 1 --2- 172.21.15.133:0/3696999280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c15d9a0 0x7f2f3c0d0b30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0ce080 tx=0x7f2f3801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3605ec32a5bf28d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f40e35640 1 -- 172.21.15.133:0/3696999280 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380b2070 con 0x7f2f3c15d9a0 2024-04-25T10:07:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f40e35640 1 -- 172.21.15.133:0/3696999280 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f380a9070 con 0x7f2f3c15d9a0 2024-04-25T10:07:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f40e35640 1 -- 172.21.15.133:0/3696999280 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380ad880 con 0x7f2f3c15d9a0 2024-04-25T10:07:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f4409b640 1 -- 172.21.15.133:0/3696999280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c15d9a0 msgr2=0x7f2f3c0d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f4409b640 1 --2- 172.21.15.133:0/3696999280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c15d9a0 0x7f2f3c0d0b30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0ce080 tx=0x7f2f3801c490 comp rx=0 tx=0).stop 2024-04-25T10:07:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f4409b640 1 -- 172.21.15.133:0/3696999280 shutdown_connections 2024-04-25T10:07:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f4409b640 1 --2- 172.21.15.133:0/3696999280 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 0x7f2f3c0d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f4409b640 1 --2- 172.21.15.133:0/3696999280 >> v2:172.21.15.162:3300/0 conn(0x7f2f3c0ce600 0x7f2f3c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f4409b640 1 --2- 172.21.15.133:0/3696999280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c15d9a0 0x7f2f3c0d0b30 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f4409b640 1 -- 172.21.15.133:0/3696999280 >> 172.21.15.133:0/3696999280 conn(0x7f2f3c06bf80 msgr2=0x7f2f3c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f4409b640 1 -- 172.21.15.133:0/3696999280 shutdown_connections 2024-04-25T10:07:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f4409b640 1 -- 172.21.15.133:0/3696999280 wait complete. 2024-04-25T10:07:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f4409b640 1 Processor -- start 2024-04-25T10:07:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.697+0000 7f2f4409b640 1 -- start start 2024-04-25T10:07:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f4409b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0ce600 0x7f2f3c0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f4409b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 0x7f2f3c160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f4409b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f3c15d9a0 0x7f2f3c0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f4409b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0d2470 con 0x7f2f3c0ce600 2024-04-25T10:07:37.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f4409b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0d22f0 con 0x7f2f3c15d9a0 2024-04-25T10:07:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f4409b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c15e590 con 0x7f2f3c0d1920 2024-04-25T10:07:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41e37640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0ce600 0x7f2f3c0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41e37640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0ce600 0x7f2f3c0ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45234/0 (socket says 172.21.15.133:45234) 2024-04-25T10:07:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41e37640 1 -- 172.21.15.133:0/3881678319 learned_addr learned my addr 172.21.15.133:0/3881678319 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:37.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41e37640 1 -- 172.21.15.133:0/3881678319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 msgr2=0x7f2f3c160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41636640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 0x7f2f3c160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f42638640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f3c15d9a0 0x7f2f3c0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41e37640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 0x7f2f3c160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41e37640 1 -- 172.21.15.133:0/3881678319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f3c15d9a0 msgr2=0x7f2f3c0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41e37640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f3c15d9a0 0x7f2f3c0cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41e37640 1 -- 172.21.15.133:0/3881678319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c159f20 con 0x7f2f3c0ce600 2024-04-25T10:07:37.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41636640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 0x7f2f3c160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:37.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f42638640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f3c15d9a0 0x7f2f3c0cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:37.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41e37640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0ce600 0x7f2f3c0ce090 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2f380b0040 tx=0x7f2f38019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f327fc640 1 -- 172.21.15.133:0/3881678319 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380b2040 con 0x7f2f3c0ce600 2024-04-25T10:07:37.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f327fc640 1 -- 172.21.15.133:0/3881678319 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f380bb070 con 0x7f2f3c0ce600 2024-04-25T10:07:37.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f327fc640 1 -- 172.21.15.133:0/3881678319 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380b6680 con 0x7f2f3c0ce600 2024-04-25T10:07:37.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f4409b640 1 -- 172.21.15.133:0/3881678319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c15f660 con 0x7f2f3c0ce600 2024-04-25T10:07:37.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f4409b640 1 -- 172.21.15.133:0/3881678319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c15fc20 con 0x7f2f3c0ce600 2024-04-25T10:07:37.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f327fc640 1 -- 172.21.15.133:0/3881678319 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f380ab070 con 0x7f2f3c0ce600 2024-04-25T10:07:37.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f327fc640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f20066f60 0x7f2f20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f327fc640 1 -- 172.21.15.133:0/3881678319 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f3811ec00 con 0x7f2f3c0ce600 2024-04-25T10:07:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41636640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f20066f60 0x7f2f20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.701+0000 7f2f41636640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f20066f60 0x7f2f20069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c006fd0 tx=0x7f2f2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 Processor -- start 2024-04-25T10:07:37.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 -- start start 2024-04-25T10:07:37.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea984f920 0x562ea984fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9850240 0x562ea98059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9806310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea96ed5f0 con 0x562ea9805f30 2024-04-25T10:07:37.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea9654da0 con 0x562ea9850240 2024-04-25T10:07:37.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ea95dd150 con 0x562ea984f920 2024-04-25T10:07:37.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f42638640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9806310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f42638640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9806310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45244/0 (socket says 172.21.15.133:45244) 2024-04-25T10:07:37.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f42638640 1 -- 172.21.15.133:0/960956962 learned_addr learned my addr 172.21.15.133:0/960956962 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:37.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f41636640 1 --2- 172.21.15.133:0/960956962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9850240 0x562ea98059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f41e37640 1 --2- 172.21.15.133:0/960956962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea984f920 0x562ea984fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f42638640 1 -- 172.21.15.133:0/960956962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea984f920 msgr2=0x562ea984fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f42638640 1 --2- 172.21.15.133:0/960956962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea984f920 0x562ea984fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f42638640 1 -- 172.21.15.133:0/960956962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9850240 msgr2=0x562ea98059f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f42638640 1 --2- 172.21.15.133:0/960956962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9850240 0x562ea98059f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f42638640 1 -- 172.21.15.133:0/960956962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ea95f6ca0 con 0x562ea9805f30 2024-04-25T10:07:37.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f42638640 1 --2- 172.21.15.133:0/960956962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9806310 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400d4e0 tx=0x7f2f3400d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa49f24a905ee4e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f1f7fe640 1 -- 172.21.15.133:0/960956962 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3401a070 con 0x562ea9805f30 2024-04-25T10:07:37.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f1f7fe640 1 -- 172.21.15.133:0/960956962 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3400b760 con 0x562ea9805f30 2024-04-25T10:07:37.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f1f7fe640 1 -- 172.21.15.133:0/960956962 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400bb70 con 0x562ea9805f30 2024-04-25T10:07:37.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 -- 172.21.15.133:0/960956962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 msgr2=0x562ea9806310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 --2- 172.21.15.133:0/960956962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9806310 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400d4e0 tx=0x7f2f3400d9a0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 -- 172.21.15.133:0/960956962 shutdown_connections 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 --2- 172.21.15.133:0/960956962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9806310 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 --2- 172.21.15.133:0/960956962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9850240 0x562ea98059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 --2- 172.21.15.133:0/960956962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea984f920 0x562ea984fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 -- 172.21.15.133:0/960956962 >> 172.21.15.133:0/960956962 conn(0x562ea980c2a0 msgr2=0x562ea98042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 -- 172.21.15.133:0/960956962 shutdown_connections 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 -- 172.21.15.133:0/960956962 wait complete. 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 -- start start 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.705+0000 7f2f47208000 1 Processor -- start 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9856c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea984f920 0x562ea98571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea9850240 0x562ea9809aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea960f990 con 0x562ea9805f30 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea9650e40 con 0x562ea984f920 2024-04-25T10:07:37.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ea95e0900 con 0x562ea9850240 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 10 client.0 _ll_register_callbacks cb 0x562ea97081c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41e37640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9856c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41e37640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9856c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45260/0 (socket says 172.21.15.133:45260) 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41e37640 1 -- 172.21.15.133:0/261618797 learned_addr learned my addr 172.21.15.133:0/261618797 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41636640 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea984f920 0x562ea98571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f42638640 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea9850240 0x562ea9809aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41e37640 1 -- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea9850240 msgr2=0x562ea9809aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41e37640 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea9850240 0x562ea9809aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41e37640 1 -- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea984f920 msgr2=0x562ea98571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41e37640 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea984f920 0x562ea98571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41e37640 1 -- 172.21.15.133:0/261618797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ea960d630 con 0x562ea9805f30 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41e37640 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9856c80 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f2f38097e30 tx=0x7f2f380eeda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a62fcd507ffda5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f1cff9640 1 -- 172.21.15.133:0/261618797 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380f7800 con 0x562ea9805f30 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 1 -- 172.21.15.133:0/261618797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f3400b760 con 0x562ea9805f30 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f1cff9640 1 -- 172.21.15.133:0/261618797 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f38106c80 con 0x562ea9805f30 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f1cff9640 1 -- 172.21.15.133:0/261618797 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f380ef380 con 0x562ea9805f30 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 10 client.7569 fetch_fsmap learned FSMap version 49 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 1 -- 172.21.15.133:0/261618797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x562ea95dd3a0 con 0x562ea9805f30 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f1cff9640 1 -- 172.21.15.133:0/261618797 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3400b760 con 0x562ea9805f30 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f1cff9640 1 -- 172.21.15.133:0/261618797 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2f380f3020 con 0x562ea9805f30 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 10 client.7569 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 10 client.7569 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 20 client.7569 populate_metadata read hostname 'smithi133' 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562ea980a4b0 0x562ea980a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 4 client.7569 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:37.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f47208000 1 -- 172.21.15.133:0/261618797 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ea95f0a50 con 0x562ea980a4b0 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41636640 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562ea980a4b0 0x562ea980a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f1cff9640 10 client.7569 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.709+0000 7f2f41636640 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562ea980a4b0 0x562ea980a890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8d3610b77ed281 server_cookie=ed9cf099fe802062 in_seq=0 out_seq=0 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f1cff9640 1 -- 172.21.15.133:0/261618797 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ea95f0a50 con 0x562ea980a4b0 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f1cff9640 10 client.7569 handle_command_reply: tid=0 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 1 client.7569 shutdown 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 1 -- 172.21.15.133:0/261618797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 msgr2=0x562ea9856c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9856c80 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f2f38097e30 tx=0x7f2f380eeda0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 1 -- 172.21.15.133:0/261618797 shutdown_connections 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562ea980a4b0 0x562ea980a890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea9850240 0x562ea9809aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea984f920 0x562ea98571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 1 --2- 172.21.15.133:0/261618797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9805f30 0x562ea9856c80 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 1 -- 172.21.15.133:0/261618797 >> 172.21.15.133:0/261618797 conn(0x562ea980c2a0 msgr2=0x562ea98049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f1cff9640 0 client.7569 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:07:37.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 1 -- 172.21.15.133:0/261618797 shutdown_connections 2024-04-25T10:07:37.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 1 -- 172.21.15.133:0/261618797 wait complete. 2024-04-25T10:07:37.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.713+0000 7f2f47208000 20 client.7569 trim_cache size 0 max 16384 2024-04-25T10:07:37.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.725+0000 7f2f47208000 1 Processor -- start 2024-04-25T10:07:37.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.725+0000 7f2f47208000 1 -- start start 2024-04-25T10:07:37.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea9805f30 0x562ea9809330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea984f920 0x562ea9809870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9850240 0x562ea98577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c2020a0 con 0x562ea984f920 2024-04-25T10:07:37.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0b76e0 con 0x562ea9850240 2024-04-25T10:07:37.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c201ad0 con 0x562ea9805f30 2024-04-25T10:07:37.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41636640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea984f920 0x562ea9809870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41636640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea984f920 0x562ea9809870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45272/0 (socket says 172.21.15.133:45272) 2024-04-25T10:07:37.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41636640 1 -- 172.21.15.133:0/1763057202 learned_addr learned my addr 172.21.15.133:0/1763057202 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:37.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41636640 1 -- 172.21.15.133:0/1763057202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea9805f30 msgr2=0x562ea9809330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41e37640 1 --2- 172.21.15.133:0/1763057202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea9805f30 0x562ea9809330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41636640 1 --2- 172.21.15.133:0/1763057202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea9805f30 0x562ea9809330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f42638640 1 --2- 172.21.15.133:0/1763057202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9850240 0x562ea98577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41636640 1 -- 172.21.15.133:0/1763057202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9850240 msgr2=0x562ea98577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41636640 1 --2- 172.21.15.133:0/1763057202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9850240 0x562ea98577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41636640 1 -- 172.21.15.133:0/1763057202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ea96d6600 con 0x562ea984f920 2024-04-25T10:07:37.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41636640 1 --2- 172.21.15.133:0/1763057202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea984f920 0x562ea9809870 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0c76f0 tx=0x7f2f2c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d81769927579c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f1effd640 1 -- 172.21.15.133:0/1763057202 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c025210 con 0x562ea984f920 2024-04-25T10:07:37.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f1effd640 1 -- 172.21.15.133:0/1763057202 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c02b070 con 0x562ea984f920 2024-04-25T10:07:37.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f1effd640 1 -- 172.21.15.133:0/1763057202 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0255b0 con 0x562ea984f920 2024-04-25T10:07:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- 172.21.15.133:0/1763057202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea984f920 msgr2=0x562ea9809870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 --2- 172.21.15.133:0/1763057202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea984f920 0x562ea9809870 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0c76f0 tx=0x7f2f2c013000 comp rx=0 tx=0).stop 2024-04-25T10:07:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- 172.21.15.133:0/1763057202 shutdown_connections 2024-04-25T10:07:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 --2- 172.21.15.133:0/1763057202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9850240 0x562ea98577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 --2- 172.21.15.133:0/1763057202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea984f920 0x562ea9809870 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 --2- 172.21.15.133:0/1763057202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea9805f30 0x562ea9809330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- 172.21.15.133:0/1763057202 >> 172.21.15.133:0/1763057202 conn(0x562ea980c2a0 msgr2=0x562ea980b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- 172.21.15.133:0/1763057202 shutdown_connections 2024-04-25T10:07:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- 172.21.15.133:0/1763057202 wait complete. 2024-04-25T10:07:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- start start 2024-04-25T10:07:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 Processor -- start 2024-04-25T10:07:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9805f30 0x562ea98b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea984f920 0x562ea98b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9850240 0x562ea985a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea96593d0 con 0x562ea9850240 2024-04-25T10:07:37.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ea9710d50 con 0x562ea9805f30 2024-04-25T10:07:37.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ea95ef980 con 0x562ea984f920 2024-04-25T10:07:37.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 10 client.0 _ll_register_callbacks cb 0x7f2f340077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:37.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f42638640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9850240 0x562ea985a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f42638640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9850240 0x562ea985a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45282/0 (socket says 172.21.15.133:45282) 2024-04-25T10:07:37.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f42638640 1 -- 172.21.15.133:0/2252128761 learned_addr learned my addr 172.21.15.133:0/2252128761 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:37.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41636640 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea984f920 0x562ea98b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41e37640 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9805f30 0x562ea98b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f42638640 1 -- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea984f920 msgr2=0x562ea98b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f42638640 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea984f920 0x562ea98b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f42638640 1 -- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9805f30 msgr2=0x562ea98b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f42638640 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9805f30 0x562ea98b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f42638640 1 -- 172.21.15.133:0/2252128761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ea95e99b0 con 0x562ea9850240 2024-04-25T10:07:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f41e37640 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9805f30 0x562ea98b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f42638640 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9850240 0x562ea985a430 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400f010 tx=0x7f2f34004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3c71d56dbceba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:07:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f1cff9640 1 -- 172.21.15.133:0/2252128761 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34025070 con 0x562ea9850240 2024-04-25T10:07:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f1cff9640 1 -- 172.21.15.133:0/2252128761 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f3400b960 con 0x562ea9850240 2024-04-25T10:07:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- 172.21.15.133:0/2252128761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f2c02b070 con 0x562ea9850240 2024-04-25T10:07:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f1cff9640 1 -- 172.21.15.133:0/2252128761 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f34030050 con 0x562ea9850240 2024-04-25T10:07:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 10 client.7575 fetch_fsmap learned FSMap version 49 2024-04-25T10:07:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.729+0000 7f2f47208000 1 -- 172.21.15.133:0/2252128761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x562ea95fc0b0 con 0x562ea9850240 2024-04-25T10:07:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f1cff9640 1 -- 172.21.15.133:0/2252128761 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c02b070 con 0x562ea9850240 2024-04-25T10:07:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f1cff9640 1 -- 172.21.15.133:0/2252128761 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2f34033020 con 0x562ea9850240 2024-04-25T10:07:37.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 10 client.7575 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T10:07:37.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 10 client.7575 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:07:37.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 20 client.7575 populate_metadata read hostname 'smithi133' 2024-04-25T10:07:37.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562ea985a970 0x562ea985ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 4 client.7575 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:07:37.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 -- 172.21.15.133:0/2252128761 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562ea96622e0 con 0x562ea985a970 2024-04-25T10:07:37.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f41e37640 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562ea985a970 0x562ea985ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f1cff9640 10 client.7575 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:07:37.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f41e37640 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562ea985a970 0x562ea985ad50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a4f791665f032b4 server_cookie=4d360100f423b3a in_seq=0 out_seq=0 2024-04-25T10:07:37.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f1cff9640 1 -- 172.21.15.133:0/2252128761 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x562ea96622e0 con 0x562ea985a970 2024-04-25T10:07:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f1cff9640 10 client.7575 handle_command_reply: tid=0 2024-04-25T10:07:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 client.7575 shutdown 2024-04-25T10:07:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 -- 172.21.15.133:0/2252128761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9850240 msgr2=0x562ea985a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9850240 0x562ea985a430 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400f010 tx=0x7f2f34004ac0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 -- 172.21.15.133:0/2252128761 shutdown_connections 2024-04-25T10:07:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562ea985a970 0x562ea985ad50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562ea9850240 0x562ea985a430 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562ea984f920 0x562ea98b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 --2- 172.21.15.133:0/2252128761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562ea9805f30 0x562ea98b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f1cff9640 0 client.7575 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:07:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 -- 172.21.15.133:0/2252128761 >> 172.21.15.133:0/2252128761 conn(0x562ea980c2a0 msgr2=0x562ea9808320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 -- 172.21.15.133:0/2252128761 shutdown_connections 2024-04-25T10:07:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 1 -- 172.21.15.133:0/2252128761 wait complete. 2024-04-25T10:07:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f47208000 20 client.7575 trim_cache size 0 max 16384 2024-04-25T10:07:37.750 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:07:37.750 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:07:37.750 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:07:37.750 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 7449, 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 5.668215537, 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 49, 2024-04-25T10:07:37.751 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 81, 2024-04-25T10:07:37.752 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 81, 2024-04-25T10:07:37.752 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 10.678309131000001 2024-04-25T10:07:37.752 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:07:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 -- 172.21.15.133:0/3881678319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f20066f60 msgr2=0x7f2f20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f20066f60 0x7f2f20069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c006fd0 tx=0x7f2f2c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 -- 172.21.15.133:0/3881678319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0ce600 msgr2=0x7f2f3c0ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0ce600 0x7f2f3c0ce090 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2f380b0040 tx=0x7f2f38019230 comp rx=0 tx=0).stop 2024-04-25T10:07:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 -- 172.21.15.133:0/3881678319 shutdown_connections 2024-04-25T10:07:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f3c15d9a0 0x7f2f3c0cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f3c0d1920 0x7f2f3c160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f20066f60 0x7f2f20069420 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 --2- 172.21.15.133:0/3881678319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0ce600 0x7f2f3c0ce090 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 -- 172.21.15.133:0/3881678319 >> 172.21.15.133:0/3881678319 conn(0x7f2f3c06bf80 msgr2=0x7f2f3c0cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 -- 172.21.15.133:0/3881678319 shutdown_connections 2024-04-25T10:07:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.733+0000 7f2f4409b640 1 -- 172.21.15.133:0/3881678319 wait complete. 2024-04-25T10:07:37.757 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:37.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.861+0000 7f29521ad640 1 Processor -- start 2024-04-25T10:07:37.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.861+0000 7f29521ad640 1 -- start start 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f29521ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0b1f70 0x7f294c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f29521ad640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f294c0ce330 0x7f294c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f29521ad640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c0c61d0 con 0x7f294c15c9a0 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f29521ad640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c0cece0 con 0x7f294c0ce330 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f29521ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0c64b0 con 0x7f294c0b1f70 2024-04-25T10:07:37.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294b7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f294c15c9a0 0x7f294c0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52476/0 (socket says 172.21.15.133:52476) 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294b7fe640 1 -- 172.21.15.133:0/3218579862 learned_addr learned my addr 172.21.15.133:0/3218579862 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f2942ffd640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0b1f70 0x7f294c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112163385 0 0) 0x7f294c0c61d0 con 0x7f294c15c9a0 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294bfff640 1 --2- 172.21.15.133:0/3218579862 >> v2:172.21.15.162:3300/0 conn(0x7f294c0ce330 0x7f294c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2938003530 con 0x7f294c15c9a0 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2019362359 0 0) 0x7f2938003530 con 0x7f294c15c9a0 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0b1f70 msgr2=0x7f294c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0b1f70 0x7f294c06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 >> v2:172.21.15.162:3300/0 conn(0x7f294c0ce330 msgr2=0x7f294c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 --2- 172.21.15.133:0/3218579862 >> v2:172.21.15.162:3300/0 conn(0x7f294c0ce330 0x7f294c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c06d4f0 con 0x7f294c15c9a0 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f293c001680 con 0x7f294c15c9a0 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 >> v1:172.21.15.133:6789/0 conn(0x7f294c15c9a0 legacy=0x7f294c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0b1f70 0x7f29380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2938003f40 0x7f2938004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2938004d50 0x7f29380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0cece0 con 0x7f2938003f40 2024-04-25T10:07:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0c64b0 con 0x7f2938004d50 2024-04-25T10:07:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293800b9d0 con 0x7f294c0b1f70 2024-04-25T10:07:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f2942ffd640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2938003f40 0x7f2938004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294b7fe640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0b1f70 0x7f29380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f2942ffd640 1 -- 172.21.15.133:0/3218579862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0b1f70 msgr2=0x7f29380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f2942ffd640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f294c0b1f70 0x7f29380039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294bfff640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2938004d50 0x7f29380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f2942ffd640 1 -- 172.21.15.133:0/3218579862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2938004d50 msgr2=0x7f29380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f2942ffd640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2938004d50 0x7f29380051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f2942ffd640 1 -- 172.21.15.133:0/3218579862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938003530 con 0x7f2938003f40 2024-04-25T10:07:37.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f2942ffd640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2938003f40 0x7f2938004810 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2934009ae0 tx=0x7f293401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b2027512b9ff7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340b2070 con 0x7f2938003f40 2024-04-25T10:07:37.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29340a5660 con 0x7f2938003f40 2024-04-25T10:07:37.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.865+0000 7f294affd640 1 -- 172.21.15.133:0/3218579862 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340096b0 con 0x7f2938003f40 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- 172.21.15.133:0/3218579862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2938003f40 msgr2=0x7f2938004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 --2- 172.21.15.133:0/3218579862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2938003f40 0x7f2938004810 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2934009ae0 tx=0x7f293401ca10 comp rx=0 tx=0).stop 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f294bfff640 1 -- 172.21.15.133:0/3218579862 reap_dead start 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- 172.21.15.133:0/3218579862 shutdown_connections 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- 172.21.15.133:0/3218579862 >> 172.21.15.133:0/3218579862 conn(0x7f294c067000 msgr2=0x7f294c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- 172.21.15.133:0/3218579862 shutdown_connections 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- 172.21.15.133:0/3218579862 wait complete. 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 Processor -- start 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- start start 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2938003f40 0x7f294c14fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c1468a0 0x7f294c147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f294c147750 0x7f294c14dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c15cf00 con 0x7f294c1468a0 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c15cd80 con 0x7f294c147750 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c06dcf0 con 0x7f2938003f40 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2942ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c1468a0 0x7f294c147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2942ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c1468a0 0x7f294c147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45306/0 (socket says 172.21.15.133:45306) 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2942ffd640 1 -- 172.21.15.133:0/1175254297 learned_addr learned my addr 172.21.15.133:0/1175254297 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f294bfff640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f294c147750 0x7f294c14dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f294b7fe640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2938003f40 0x7f294c14fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2942ffd640 1 -- 172.21.15.133:0/1175254297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2938003f40 msgr2=0x7f294c14fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2942ffd640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2938003f40 0x7f294c14fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2942ffd640 1 -- 172.21.15.133:0/1175254297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f294c147750 msgr2=0x7f294c14dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2942ffd640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f294c147750 0x7f294c14dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2942ffd640 1 -- 172.21.15.133:0/1175254297 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c150140 con 0x7f294c1468a0 2024-04-25T10:07:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2942ffd640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c1468a0 0x7f294c147210 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f293401d2b0 tx=0x7f2938005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2948ff9640 1 -- 172.21.15.133:0/1175254297 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340b2040 con 0x7f294c1468a0 2024-04-25T10:07:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2948ff9640 1 -- 172.21.15.133:0/1175254297 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293400f000 con 0x7f294c1468a0 2024-04-25T10:07:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- 172.21.15.133:0/1175254297 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c150420 con 0x7f294c1468a0 2024-04-25T10:07:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f2948ff9640 1 -- 172.21.15.133:0/1175254297 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934004240 con 0x7f294c1468a0 2024-04-25T10:07:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- 172.21.15.133:0/1175254297 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c14e020 con 0x7f294c1468a0 2024-04-25T10:07:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.869+0000 7f29521ad640 1 -- 172.21.15.133:0/1175254297 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c068ef0 con 0x7f294c1468a0 2024-04-25T10:07:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.873+0000 7f2948ff9640 1 -- 172.21.15.133:0/1175254297 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2934003620 con 0x7f294c1468a0 2024-04-25T10:07:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.873+0000 7f2948ff9640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f292c066f60 0x7f292c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.873+0000 7f2948ff9640 1 -- 172.21.15.133:0/1175254297 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f293411e6f0 con 0x7f294c1468a0 2024-04-25T10:07:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.873+0000 7f294b7fe640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f292c066f60 0x7f292c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.873+0000 7f294b7fe640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f292c066f60 0x7f292c069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f293c001930 tx=0x7f293c001120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:37.877+0000 7f2948ff9640 1 -- 172.21.15.133:0/1175254297 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2934016780 con 0x7f294c1468a0 2024-04-25T10:07:38.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.053+0000 7f29521ad640 1 -- 172.21.15.133:0/1175254297 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f294c15d5b0 con 0x7f294c1468a0 2024-04-25T10:07:38.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f2948ff9640 1 -- 172.21.15.133:0/1175254297 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f29340ef570 con 0x7f294c1468a0 2024-04-25T10:07:38.060 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:38.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 -- 172.21.15.133:0/1175254297 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f292c066f60 msgr2=0x7f292c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f292c066f60 0x7f292c069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f293c001930 tx=0x7f293c001120 comp rx=0 tx=0).stop 2024-04-25T10:07:38.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 -- 172.21.15.133:0/1175254297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c1468a0 msgr2=0x7f294c147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c1468a0 0x7f294c147210 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f293401d2b0 tx=0x7f2938005840 comp rx=0 tx=0).stop 2024-04-25T10:07:38.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 -- 172.21.15.133:0/1175254297 shutdown_connections 2024-04-25T10:07:38.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f294c147750 0x7f294c14dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f292c066f60 0x7f292c069420 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c1468a0 0x7f294c147210 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 --2- 172.21.15.133:0/1175254297 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2938003f40 0x7f294c14fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 -- 172.21.15.133:0/1175254297 >> 172.21.15.133:0/1175254297 conn(0x7f294c067000 msgr2=0x7f294c1570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 -- 172.21.15.133:0/1175254297 shutdown_connections 2024-04-25T10:07:38.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:38.057+0000 7f29521ad640 1 -- 172.21.15.133:0/1175254297 wait complete. 2024-04-25T10:07:38.078 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:07:38.078 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:38.078 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:07:38.078 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:07:38.078 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:38.078 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:07:38.078 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:07:38.078 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:38.078 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:38.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:38.079 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:07:38.084 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:38.084 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:38.084 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:38.085 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42174sec preferred_lft 42174sec 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:38.086 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:38.087 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:38.087 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:07:38.087 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:38.087 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:38.087 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:38.087 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:07:38.087 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:38.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:38.087 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:38.138 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:07:38.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:38.140 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:07:38.193 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:07:38.193 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:38.193 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:07:38.193 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:07:38.193 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:07:38.193 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:07:38.193 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:07:38.193 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:07:38.194 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:07:38.194 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:07:38.194 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:38.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:38.194 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:07:38.194 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:07:38.194 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:07:38.194 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:07:38.194 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:07:38.194 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:07:38.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:38.268 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:07:38.317 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:07:38.317 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:07:38.317 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:38.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:38.317 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:07:38.373 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:07:38.373 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:38.374 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:38.420 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:38.421 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:38.421 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:38.472 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:07:38.530 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:38.577 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:38.578 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:07:38.580 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:38.580 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:38.580 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:38.580 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:38.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:38.581 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:07:38.629 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:07:38.629 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:07:38.674 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:07:38.688 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:38.741 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:07:38.741 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:07:38.741 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:07:38.741 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:07:38.741 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:07:38.742 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:07:38.742 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:07:38.742 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:07:38.742 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:07:38.742 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:07:38.742 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:07:38.742 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:07:38.742 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:38.742 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:07:38.742 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:07:38.742 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:07:38.743 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:07:38.743 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:07:38.743 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:38.743 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:07:38.743 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:38.743 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:07:38.743 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:07:38.743 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:07:38.743 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:38.744 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:38.744 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:38.744 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:38.744 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:07:38.744 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:38.744 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:38.744 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:38.744 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:38.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:38.745 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:07:38.753 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:07:38.749+0000 7fb0c009d580 -1 init, newargv = 0x55eb58796e30 newargc=15 2024-04-25T10:07:38.754 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[58401]: starting ceph client 2024-04-25T10:07:38.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[58401]: starting fuse 2024-04-25T10:07:39.860 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:07:39.877 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:39.930 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:07:39.930 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:07:39.930 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:07:39.930 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:07:39.930 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:07:39.930 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:07:39.930 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:07:39.930 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:07:39.930 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:07:39.930 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:39.931 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:39.932 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:39.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:39.933 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:07:39.984 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:07:39.985 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:07:39.985 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:07:39.985 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:07:39.985 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:07:39.985 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:07:39.985 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:07:39.985 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:07:39.986 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:07:39.987 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:07:39.987 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:07:39.987 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:07:39.987 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:39.987 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:07:39.987 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:07:40.109 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.58401.asok 2024-04-25T10:07:40.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:07:40.120 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58401.asok 2024-04-25T10:07:40.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:40.121 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58401.asok status 2024-04-25T10:07:40.277 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "58401", 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 7587, 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:07:40.278 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 7587 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 25559474 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 25559474 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.7587 192.168.0.1:0/25559474", 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/25559474", 2024-04-25T10:07:40.280 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:07:40.280 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 49, 2024-04-25T10:07:40.280 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 81, 2024-04-25T10:07:40.280 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:07:40.280 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:07:40.280 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:07:40.280 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:07:40.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:40.291 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:07:40.291 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:40.298 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:07:40.298 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:40.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:40.298 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:40.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T10:07:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e3e02640 1 Processor -- start 2024-04-25T10:07:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e3e02640 1 -- start start 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e3e02640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70dc160f10 0x7f70dc0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e3e02640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f70dc0b3aa0 0x7f70dc06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e3e02640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70dc0d0610 con 0x7f70dc0cd3e0 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e3e02640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc1612f0 con 0x7f70dc160f10 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e3e02640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc0d08b0 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e1b9e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70dc160f10 0x7f70dc0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e1b9e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70dc160f10 0x7f70dc0b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45314/0 (socket says 172.21.15.133:45314) 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e1b9e640 1 -- 172.21.15.133:0/1459355523 learned_addr learned my addr 172.21.15.133:0/1459355523 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e139d640 1 --2- 172.21.15.133:0/1459355523 >> v2:172.21.15.162:3300/0 conn(0x7f70dc0b3aa0 0x7f70dc06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e1b9e640 1 -- 172.21.15.133:0/1459355523 >> v2:172.21.15.162:3300/0 conn(0x7f70dc0b3aa0 msgr2=0x7f70dc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e1b9e640 1 --2- 172.21.15.133:0/1459355523 >> v2:172.21.15.162:3300/0 conn(0x7f70dc0b3aa0 0x7f70dc06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e1b9e640 1 -- 172.21.15.133:0/1459355523 >> v1:172.21.15.162:6789/0 conn(0x7f70dc0cd3e0 legacy=0x7f70dc06fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.525+0000 7f70e1b9e640 1 -- 172.21.15.133:0/1459355523 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0d0b90 con 0x7f70dc160f10 2024-04-25T10:07:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e1b9e640 1 --2- 172.21.15.133:0/1459355523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70dc160f10 0x7f70dc0b3560 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f70dc06bae0 tx=0x7f70d801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772e29069257f1df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e0b9c640 1 -- 172.21.15.133:0/1459355523 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80b1070 con 0x7f70dc160f10 2024-04-25T10:07:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e0b9c640 1 -- 172.21.15.133:0/1459355523 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d80ac640 con 0x7f70dc160f10 2024-04-25T10:07:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e0b9c640 1 -- 172.21.15.133:0/1459355523 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80aca10 con 0x7f70dc160f10 2024-04-25T10:07:40.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e3e02640 1 -- 172.21.15.133:0/1459355523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70dc160f10 msgr2=0x7f70dc0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e3e02640 1 --2- 172.21.15.133:0/1459355523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70dc160f10 0x7f70dc0b3560 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f70dc06bae0 tx=0x7f70d801c400 comp rx=0 tx=0).stop 2024-04-25T10:07:40.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e3e02640 1 -- 172.21.15.133:0/1459355523 shutdown_connections 2024-04-25T10:07:40.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e3e02640 1 --2- 172.21.15.133:0/1459355523 >> v2:172.21.15.162:3300/0 conn(0x7f70dc0b3aa0 0x7f70dc06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e3e02640 1 --2- 172.21.15.133:0/1459355523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70dc160f10 0x7f70dc0b3560 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e3e02640 1 -- 172.21.15.133:0/1459355523 >> 172.21.15.133:0/1459355523 conn(0x7f70dc06e480 msgr2=0x7f70dc06e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e3e02640 1 -- 172.21.15.133:0/1459355523 shutdown_connections 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e3e02640 1 -- 172.21.15.133:0/1459355523 wait complete. 2024-04-25T10:07:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e3e02640 1 Processor -- start 2024-04-25T10:07:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.529+0000 7f70e3e02640 1 -- start start 2024-04-25T10:07:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e3e02640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70dc0b3aa0 0x7f70dc1562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e3e02640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70dc0cd3e0 0x7f70dc0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e3e02640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70dc160f10 0x7f70dc15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e3e02640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc16a8b0 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e3e02640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc16a730 con 0x7f70dc0cd3e0 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e3e02640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc16aa30 con 0x7f70dc160f10 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e1b9e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70dc0b3aa0 0x7f70dc1562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e1b9e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70dc0b3aa0 0x7f70dc1562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45324/0 (socket says 172.21.15.133:45324) 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e1b9e640 1 -- 172.21.15.133:0/1856024961 learned_addr learned my addr 172.21.15.133:0/1856024961 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e1b9e640 1 -- 172.21.15.133:0/1856024961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70dc160f10 msgr2=0x7f70dc15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e139d640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70dc0cd3e0 0x7f70dc0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e1b9e640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70dc160f10 0x7f70dc15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e1b9e640 1 -- 172.21.15.133:0/1856024961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70dc0cd3e0 msgr2=0x7f70dc0cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e1b9e640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70dc0cd3e0 0x7f70dc0cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e1b9e640 1 -- 172.21.15.133:0/1856024961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0cb1f0 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e1b9e640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70dc0b3aa0 0x7f70dc1562e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f70d80af040 tx=0x7f70d801ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70d27fc640 1 -- 172.21.15.133:0/1856024961 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80bb020 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70d27fc640 1 -- 172.21.15.133:0/1856024961 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d80b7640 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e3e02640 1 -- 172.21.15.133:0/1856024961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc0cb4d0 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70d27fc640 1 -- 172.21.15.133:0/1856024961 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80b1040 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e3e02640 1 -- 172.21.15.133:0/1856024961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc0cba90 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e3e02640 1 -- 172.21.15.133:0/1856024961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc069000 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70d27fc640 1 -- 172.21.15.133:0/1856024961 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70d80be020 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70d27fc640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70c0066f60 0x7f70c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70d27fc640 1 -- 172.21.15.133:0/1856024961 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f70d811f8c0 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e139d640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70c0066f60 0x7f70c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.533+0000 7f70e139d640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70c0066f60 0x7f70c0069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0cd150 tx=0x7f70cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.537+0000 7f70d27fc640 1 -- 172.21.15.133:0/1856024961 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70d80c5030 con 0x7f70dc0b3aa0 2024-04-25T10:07:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:40.733+0000 7f70e3e02640 1 -- 172.21.15.133:0/1856024961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f70dc06d3e0 con 0x7f70dc0b3aa0 2024-04-25T10:07:41.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.133+0000 7f70d27fc640 1 -- 172.21.15.133:0/1856024961 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v50) v1 ==== 167+0+0 (secure 0 0 0) 0x7f70d80160b0 con 0x7f70dc0b3aa0 2024-04-25T10:07:41.139 INFO:teuthology.orchestra.run.smithi133.stderr:enabled new snapshots 2024-04-25T10:07:41.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 -- 172.21.15.133:0/1856024961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70c0066f60 msgr2=0x7f70c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70c0066f60 0x7f70c0069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0cd150 tx=0x7f70cc008040 comp rx=0 tx=0).stop 2024-04-25T10:07:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 -- 172.21.15.133:0/1856024961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70dc0b3aa0 msgr2=0x7f70dc1562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70dc0b3aa0 0x7f70dc1562e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f70d80af040 tx=0x7f70d801ac40 comp rx=0 tx=0).stop 2024-04-25T10:07:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 -- 172.21.15.133:0/1856024961 shutdown_connections 2024-04-25T10:07:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70dc160f10 0x7f70dc15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70dc0cd3e0 0x7f70dc0cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70c0066f60 0x7f70c0069420 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 --2- 172.21.15.133:0/1856024961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70dc0b3aa0 0x7f70dc1562e0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 -- 172.21.15.133:0/1856024961 >> 172.21.15.133:0/1856024961 conn(0x7f70dc06e480 msgr2=0x7f70dc0b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 -- 172.21.15.133:0/1856024961 shutdown_connections 2024-04-25T10:07:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:07:41.137+0000 7f70e3e02640 1 -- 172.21.15.133:0/1856024961 wait complete. 2024-04-25T10:07:41.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:41.161 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T10:07:41.182 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:07:41.182 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:41.230 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-04-25T10:07:41.230 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:07:41.230 DEBUG:teuthology.orchestra.run.smithi133:> import uuid 2024-04-25T10:07:41.230 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:41.230 DEBUG:teuthology.orchestra.run.smithi133:> n = 2000 2024-04-25T10:07:41.230 DEBUG:teuthology.orchestra.run.smithi133:> create_hard_links = False 2024-04-25T10:07:41.230 DEBUG:teuthology.orchestra.run.smithi133:> if 3 > 0: 2024-04-25T10:07:41.230 DEBUG:teuthology.orchestra.run.smithi133:> create_hard_links = True 2024-04-25T10:07:41.230 DEBUG:teuthology.orchestra.run.smithi133:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T10:07:41.230 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> dpath = os.path.dirname(path) 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> fnameprefix = os.path.basename(path) 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> try: 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> for i in range(n): 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> f.write(f"{i}") 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:07:41.231 DEBUG:teuthology.orchestra.run.smithi133:> f.flush() 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(f.fileno()) 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> os.unlink(fpath) 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(dirfd) 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> if create_hard_links: 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> for j in range(3): 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(dirfd) 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> finally: 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> os.close(dirfd) 2024-04-25T10:07:41.232 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:07:51.409 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:51.411 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:20.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:08:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:08:20.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:20.026 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:08:20.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:20.074 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T10:08:21.573 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:21.576 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:52.001 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:52.004 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:22.047 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:22.051 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:40.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:40.317 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:09:40.382 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:09:40.382 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T10:09:40.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.507+0000 7fd8c32bd640 1 Processor -- start 2024-04-25T10:09:40.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.507+0000 7fd8c32bd640 1 -- start start 2024-04-25T10:09:40.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.507+0000 7fd8c32bd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8bc06e970 0x7fd8bc162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.507+0000 7fd8c32bd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8bc0ce8e0 0x7fd8bc06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.507+0000 7fd8c32bd640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8bc0cd6f0 con 0x7fd8bc0ce450 2024-04-25T10:09:40.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.507+0000 7fd8c32bd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8bc163340 con 0x7fd8bc06e970 2024-04-25T10:09:40.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.507+0000 7fd8c32bd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0cf270 con 0x7fd8bc0ce8e0 2024-04-25T10:09:40.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8c1059640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8bc06e970 0x7fd8bc162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8c185a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8bc0ce8e0 0x7fd8bc06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8c185a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8bc0ce8e0 0x7fd8bc06d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:48764/0 (socket says 172.21.15.133:48764) 2024-04-25T10:09:40.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8c1059640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8bc06e970 0x7fd8bc162e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:49664/0 (socket says 172.21.15.133:49664) 2024-04-25T10:09:40.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8c185a640 1 -- 172.21.15.133:0/3180516057 learned_addr learned my addr 172.21.15.133:0/3180516057 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8c0858640 1 --1- 172.21.15.133:0/3180516057 >> v1:172.21.15.162:6789/0 conn(0x7fd8bc0ce450 0x7fd8bc0b14a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:37864/0 (socket says 172.21.15.133:37864) 2024-04-25T10:09:40.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8c1059640 1 -- 172.21.15.133:0/3180516057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8bc0ce8e0 msgr2=0x7fd8bc06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8c1059640 1 --2- 172.21.15.133:0/3180516057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8bc0ce8e0 0x7fd8bc06d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8c1059640 1 -- 172.21.15.133:0/3180516057 >> v1:172.21.15.162:6789/0 conn(0x7fd8bc0ce450 legacy=0x7fd8bc0b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8c1059640 1 -- 172.21.15.133:0/3180516057 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc0cd970 con 0x7fd8bc06e970 2024-04-25T10:09:40.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8c1059640 1 --2- 172.21.15.133:0/3180516057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8bc06e970 0x7fd8bc162e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80097c0 tx=0x7fd8b801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaace087cce20085 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8b3fff640 1 -- 172.21.15.133:0/3180516057 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80b0070 con 0x7fd8bc06e970 2024-04-25T10:09:40.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8b3fff640 1 -- 172.21.15.133:0/3180516057 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b80ab600 con 0x7fd8bc06e970 2024-04-25T10:09:40.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.511+0000 7fd8b3fff640 1 -- 172.21.15.133:0/3180516057 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80ab9d0 con 0x7fd8bc06e970 2024-04-25T10:09:40.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 -- 172.21.15.133:0/3180516057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8bc06e970 msgr2=0x7fd8bc162e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 --2- 172.21.15.133:0/3180516057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8bc06e970 0x7fd8bc162e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80097c0 tx=0x7fd8b801c410 comp rx=0 tx=0).stop 2024-04-25T10:09:40.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 -- 172.21.15.133:0/3180516057 shutdown_connections 2024-04-25T10:09:40.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 --2- 172.21.15.133:0/3180516057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8bc0ce8e0 0x7fd8bc06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 --2- 172.21.15.133:0/3180516057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8bc06e970 0x7fd8bc162e00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 -- 172.21.15.133:0/3180516057 >> 172.21.15.133:0/3180516057 conn(0x7fd8bc066f70 msgr2=0x7fd8bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 -- 172.21.15.133:0/3180516057 shutdown_connections 2024-04-25T10:09:40.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 -- 172.21.15.133:0/3180516057 wait complete. 2024-04-25T10:09:40.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 Processor -- start 2024-04-25T10:09:40.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 -- start start 2024-04-25T10:09:40.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8bc06e970 0x7fd8bc0a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8bc0ce450 0x7fd8bc15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8bc0ce8e0 0x7fd8bc15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc06dbf0 con 0x7fd8bc0ce450 2024-04-25T10:09:40.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0b21c0 con 0x7fd8bc0ce8e0 2024-04-25T10:09:40.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0ad070 con 0x7fd8bc06e970 2024-04-25T10:09:40.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c0858640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8bc0ce450 0x7fd8bc15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c0858640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8bc0ce450 0x7fd8bc15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47778/0 (socket says 172.21.15.133:47778) 2024-04-25T10:09:40.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c0858640 1 -- 172.21.15.133:0/4259121466 learned_addr learned my addr 172.21.15.133:0/4259121466 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c1059640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8bc06e970 0x7fd8bc0a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c185a640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8bc0ce8e0 0x7fd8bc15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c0858640 1 -- 172.21.15.133:0/4259121466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8bc06e970 msgr2=0x7fd8bc0a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c0858640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8bc06e970 0x7fd8bc0a9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c0858640 1 -- 172.21.15.133:0/4259121466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8bc0ce8e0 msgr2=0x7fd8bc15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c0858640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8bc0ce8e0 0x7fd8bc15a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c0858640 1 -- 172.21.15.133:0/4259121466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc06d610 con 0x7fd8bc0ce450 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c185a640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8bc0ce8e0 0x7fd8bc15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c0858640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8bc0ce450 0x7fd8bc15a080 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00c420 tx=0x7fd8ac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8b1ffb640 1 -- 172.21.15.133:0/4259121466 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0074e0 con 0x7fd8bc0ce450 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8b1ffb640 1 -- 172.21.15.133:0/4259121466 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac007680 con 0x7fd8bc0ce450 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 -- 172.21.15.133:0/4259121466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8bc0cc790 con 0x7fd8bc0ce450 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8b1ffb640 1 -- 172.21.15.133:0/4259121466 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac004a30 con 0x7fd8bc0ce450 2024-04-25T10:09:40.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.515+0000 7fd8c32bd640 1 -- 172.21.15.133:0/4259121466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc0ad630 con 0x7fd8bc0ce450 2024-04-25T10:09:40.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8b1ffb640 1 -- 172.21.15.133:0/4259121466 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8ac01a020 con 0x7fd8bc0ce450 2024-04-25T10:09:40.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8b1ffb640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8a0066f60 0x7fd8a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8b1ffb640 1 -- 172.21.15.133:0/4259121466 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8ac084a20 con 0x7fd8bc0ce450 2024-04-25T10:09:40.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c1059640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8a0066f60 0x7fd8a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c1059640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8a0066f60 0x7fd8a0069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80ad000 tx=0x7fd8b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c642a000 1 Processor -- start 2024-04-25T10:09:40.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c642a000 1 -- start start 2024-04-25T10:09:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dd2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030dd3360 0x55a030d8d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030d8dd90 0x55a030d8e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c642a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a030c701f0 con 0x55a030dd2a40 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c642a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a030bd6e90 con 0x55a030dd3360 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c642a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a030b5ef50 con 0x55a030d8dd90 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c1059640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dd2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c1059640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dd2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47792/0 (socket says 172.21.15.133:47792) 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c1059640 1 -- 172.21.15.133:0/2650796006 learned_addr learned my addr 172.21.15.133:0/2650796006 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c1059640 1 -- 172.21.15.133:0/2650796006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030d8dd90 msgr2=0x55a030d8e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c1059640 1 --2- 172.21.15.133:0/2650796006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030d8dd90 0x55a030d8e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c1059640 1 -- 172.21.15.133:0/2650796006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030dd3360 msgr2=0x55a030d8d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c1059640 1 --2- 172.21.15.133:0/2650796006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030dd3360 0x55a030d8d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.519+0000 7fd8c1059640 1 -- 172.21.15.133:0/2650796006 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a030bae8b0 con 0x55a030dd2a40 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c1059640 1 --2- 172.21.15.133:0/2650796006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dd2e20 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd8b801c450 tx=0x7fd8b8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f2c8f95709c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8a6ffd640 1 -- 172.21.15.133:0/2650796006 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80c7070 con 0x55a030dd2a40 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8a6ffd640 1 -- 172.21.15.133:0/2650796006 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b80c2470 con 0x55a030dd2a40 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8a6ffd640 1 -- 172.21.15.133:0/2650796006 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80b0040 con 0x55a030dd2a40 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- 172.21.15.133:0/2650796006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 msgr2=0x55a030dd2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 --2- 172.21.15.133:0/2650796006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dd2e20 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd8b801c450 tx=0x7fd8b8004740 comp rx=0 tx=0).stop 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- 172.21.15.133:0/2650796006 shutdown_connections 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 --2- 172.21.15.133:0/2650796006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030d8dd90 0x55a030d8e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 --2- 172.21.15.133:0/2650796006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030dd3360 0x55a030d8d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 --2- 172.21.15.133:0/2650796006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dd2e20 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- 172.21.15.133:0/2650796006 >> 172.21.15.133:0/2650796006 conn(0x55a030d89d40 msgr2=0x55a030d8c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- 172.21.15.133:0/2650796006 shutdown_connections 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- 172.21.15.133:0/2650796006 wait complete. 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- start start 2024-04-25T10:09:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 Processor -- start 2024-04-25T10:09:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030d8dd90 0x55a030dda1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dda700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030dd3360 0x55a030d93e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a030b91540 con 0x55a030dd2a40 2024-04-25T10:09:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a030bd2f30 con 0x55a030d8dd90 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a030bddaa0 con 0x55a030dd3360 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 10 client.0 _ll_register_callbacks cb 0x55a030c8b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c0858640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dda700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c0858640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dda700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47808/0 (socket says 172.21.15.133:47808) 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c0858640 1 -- 172.21.15.133:0/1927796022 learned_addr learned my addr 172.21.15.133:0/1927796022 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c1059640 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030d8dd90 0x55a030dda1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c185a640 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030dd3360 0x55a030d93e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c0858640 1 -- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030dd3360 msgr2=0x55a030d93e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c0858640 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030dd3360 0x55a030d93e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c0858640 1 -- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030d8dd90 msgr2=0x55a030dda1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c0858640 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030d8dd90 0x55a030dda1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c0858640 1 -- 172.21.15.133:0/1927796022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a030b5f1a0 con 0x55a030dd2a40 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c0858640 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dda700 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00cde0 tx=0x7fd8ac055950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65f8b93fc0ed339 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd89bfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd89bfff640 1 -- 172.21.15.133:0/1927796022 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0555f0 con 0x55a030dd2a40 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd89bfff640 1 -- 172.21.15.133:0/1927796022 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8ac004bd0 con 0x55a030dd2a40 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- 172.21.15.133:0/1927796022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8b80c2470 con 0x55a030dd2a40 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd89bfff640 1 -- 172.21.15.133:0/1927796022 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8ac05d8a0 con 0x55a030dd2a40 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 10 client.7602 fetch_fsmap learned FSMap version 50 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- 172.21.15.133:0/1927796022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55a030c59840 con 0x55a030dd2a40 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd89bfff640 1 -- 172.21.15.133:0/1927796022 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b80c2470 con 0x55a030dd2a40 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd89bfff640 1 -- 172.21.15.133:0/1927796022 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8ac073120 con 0x55a030dd2a40 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 10 client.7602 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 10 client.7602 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 20 client.7602 populate_metadata read hostname 'smithi133' 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55a030d94c00 0x55a030d94fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 4 client.7602 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.523+0000 7fd8c642a000 1 -- 172.21.15.133:0/1927796022 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a030b72be0 con 0x55a030d94c00 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c1059640 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55a030d94c00 0x55a030d94fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c1059640 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55a030d94c00 0x55a030d94fe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368f84acb6619c46 server_cookie=d3d949bded8ef4b7 in_seq=0 out_seq=0 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd89bfff640 10 client.7602 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd89bfff640 1 -- 172.21.15.133:0/1927796022 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a030b72be0 con 0x55a030d94c00 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd89bfff640 10 client.7602 handle_command_reply: tid=0 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 1 client.7602 shutdown 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 1 -- 172.21.15.133:0/1927796022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 msgr2=0x55a030dda700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dda700 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00cde0 tx=0x7fd8ac055950 comp rx=0 tx=0).stop 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 1 -- 172.21.15.133:0/1927796022 shutdown_connections 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55a030d94c00 0x55a030d94fe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030dd3360 0x55a030d93e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd2a40 0x55a030dda700 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd89bfff640 0 client.7602 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 1 --2- 172.21.15.133:0/1927796022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030d8dd90 0x55a030dda1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 1 -- 172.21.15.133:0/1927796022 >> 172.21.15.133:0/1927796022 conn(0x55a030d89d40 msgr2=0x55a030d8c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 1 -- 172.21.15.133:0/1927796022 shutdown_connections 2024-04-25T10:09:40.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 1 -- 172.21.15.133:0/1927796022 wait complete. 2024-04-25T10:09:40.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.527+0000 7fd8c642a000 20 client.7602 trim_cache size 0 max 16384 2024-04-25T10:09:40.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 Processor -- start 2024-04-25T10:09:40.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 -- start start 2024-04-25T10:09:40.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030d8dd90 0x55a030d8c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030dd2a40 0x55a030d8c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030d8cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc207d50 con 0x55a030dd3360 2024-04-25T10:09:40.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0b3760 con 0x55a030d8dd90 2024-04-25T10:09:40.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc1ffa00 con 0x55a030dd2a40 2024-04-25T10:09:40.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c185a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030d8cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c185a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030d8cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47818/0 (socket says 172.21.15.133:47818) 2024-04-25T10:09:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c185a640 1 -- 172.21.15.133:0/2282340037 learned_addr learned my addr 172.21.15.133:0/2282340037 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c185a640 1 -- 172.21.15.133:0/2282340037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030dd2a40 msgr2=0x55a030d8c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c185a640 1 --2- 172.21.15.133:0/2282340037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030dd2a40 0x55a030d8c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c185a640 1 -- 172.21.15.133:0/2282340037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030d8dd90 msgr2=0x55a030d8c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c185a640 1 --2- 172.21.15.133:0/2282340037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030d8dd90 0x55a030d8c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c185a640 1 -- 172.21.15.133:0/2282340037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a030c30a10 con 0x55a030dd3360 2024-04-25T10:09:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c185a640 1 --2- 172.21.15.133:0/2282340037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030d8cbb0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009f20 tx=0x7fd8b4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56e684e914dad32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8a67fc640 1 -- 172.21.15.133:0/2282340037 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b40153f0 con 0x55a030dd3360 2024-04-25T10:09:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8a67fc640 1 -- 172.21.15.133:0/2282340037 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4015590 con 0x55a030dd3360 2024-04-25T10:09:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8a67fc640 1 -- 172.21.15.133:0/2282340037 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4015940 con 0x55a030dd3360 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 -- 172.21.15.133:0/2282340037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 msgr2=0x55a030d8cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 --2- 172.21.15.133:0/2282340037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030d8cbb0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009f20 tx=0x7fd8b4009810 comp rx=0 tx=0).stop 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 -- 172.21.15.133:0/2282340037 shutdown_connections 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 --2- 172.21.15.133:0/2282340037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030d8cbb0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 --2- 172.21.15.133:0/2282340037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030dd2a40 0x55a030d8c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 --2- 172.21.15.133:0/2282340037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030d8dd90 0x55a030d8c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 -- 172.21.15.133:0/2282340037 >> 172.21.15.133:0/2282340037 conn(0x55a030d89d40 msgr2=0x55a030de08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 -- 172.21.15.133:0/2282340037 shutdown_connections 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 -- 172.21.15.133:0/2282340037 wait complete. 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 -- start start 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.543+0000 7fd8c642a000 1 Processor -- start 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030d8dd90 0x55a030dd7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030dd2a40 0x55a030dd7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030dd8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a030bd61a0 con 0x55a030dd3360 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0b3160 con 0x55a030dd2a40 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a030bd2f30 con 0x55a030d8dd90 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 10 client.0 _ll_register_callbacks cb 0x55a030b99a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c185a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030dd8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c185a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030dd8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47820/0 (socket says 172.21.15.133:47820) 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c185a640 1 -- 172.21.15.133:0/3708214209 learned_addr learned my addr 172.21.15.133:0/3708214209 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c0858640 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030dd2a40 0x55a030dd7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c1059640 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030d8dd90 0x55a030dd7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c185a640 1 -- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030d8dd90 msgr2=0x55a030dd7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c185a640 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030d8dd90 0x55a030dd7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c185a640 1 -- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030dd2a40 msgr2=0x55a030dd7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c185a640 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030dd2a40 0x55a030dd7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c185a640 1 -- 172.21.15.133:0/3708214209 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a030b6bb00 con 0x55a030dd3360 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c185a640 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030dd8520 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x55a030c3e7b0 tx=0x7fd8b4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaef675ceb1b3469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd89bfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd89bfff640 1 -- 172.21.15.133:0/3708214209 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4023070 con 0x55a030dd3360 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd89bfff640 1 -- 172.21.15.133:0/3708214209 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8b4003ba0 con 0x55a030dd3360 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 1 -- 172.21.15.133:0/3708214209 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8b4015590 con 0x55a030dd3360 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd89bfff640 1 -- 172.21.15.133:0/3708214209 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8b402f050 con 0x55a030dd3360 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 10 client.7608 fetch_fsmap learned FSMap version 50 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 1 -- 172.21.15.133:0/3708214209 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55a030b7df80 con 0x55a030dd3360 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd89bfff640 1 -- 172.21.15.133:0/3708214209 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4015590 con 0x55a030dd3360 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd89bfff640 1 -- 172.21.15.133:0/3708214209 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8b4032020 con 0x55a030dd3360 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 10 client.7608 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 10 client.7608 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 20 client.7608 populate_metadata read hostname 'smithi133' 2024-04-25T10:09:40.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55a030e38cd0 0x55a030e3b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 4 client.7608 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:09:40.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 1 -- 172.21.15.133:0/3708214209 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55a030be43d0 con 0x55a030e38cd0 2024-04-25T10:09:40.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c1059640 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55a030e38cd0 0x55a030e3b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd89bfff640 10 client.7608 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c1059640 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55a030e38cd0 0x55a030e3b0c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee684dd649ff0a30 server_cookie=8b368054a16eb044 in_seq=0 out_seq=0 2024-04-25T10:09:40.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd89bfff640 1 -- 172.21.15.133:0/3708214209 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17937 (crc 0 0 0) 0x55a030be43d0 con 0x55a030e38cd0 2024-04-25T10:09:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd89bfff640 10 client.7608 handle_command_reply: tid=0 2024-04-25T10:09:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.547+0000 7fd8c642a000 1 client.7608 shutdown 2024-04-25T10:09:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c642a000 1 -- 172.21.15.133:0/3708214209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 msgr2=0x55a030dd8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c642a000 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030dd8520 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x55a030c3e7b0 tx=0x7fd8b4004ac0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c642a000 1 -- 172.21.15.133:0/3708214209 shutdown_connections 2024-04-25T10:09:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c642a000 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55a030e38cd0 0x55a030e3b0c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c642a000 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a030dd3360 0x55a030dd8520 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c642a000 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a030dd2a40 0x55a030dd7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd89bfff640 0 client.7608 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c642a000 1 --2- 172.21.15.133:0/3708214209 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a030d8dd90 0x55a030dd7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c642a000 1 -- 172.21.15.133:0/3708214209 >> 172.21.15.133:0/3708214209 conn(0x55a030d89d40 msgr2=0x55a030dd11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c642a000 1 -- 172.21.15.133:0/3708214209 shutdown_connections 2024-04-25T10:09:40.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c642a000 1 -- 172.21.15.133:0/3708214209 wait complete. 2024-04-25T10:09:40.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c642a000 20 client.7608 trim_cache size 0 max 16384 2024-04-25T10:09:40.561 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:09:40.561 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:09:40.561 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2889, 2024-04-25T10:09:40.561 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2891, 2024-04-25T10:09:40.561 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 913453, 2024-04-25T10:09:40.561 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 13645777, 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 10, 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.423090689, 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.170268343, 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.090938907, 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.127082877, 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2888, 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.399222493, 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000138234 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2, 2024-04-25T10:09:40.562 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000413, 2024-04-25T10:09:40.563 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000206 2024-04-25T10:09:40.563 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.563 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:09:40.563 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:09:40.563 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.563 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:09:40.563 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 34588, 2024-04-25T10:09:40.563 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 60967, 2024-04-25T10:09:40.563 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 12232843, 2024-04-25T10:09:40.564 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 54125093, 2024-04-25T10:09:40.564 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 9, 2024-04-25T10:09:40.564 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-04-25T10:09:40.564 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 6.491198563, 2024-04-25T10:09:40.564 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 3.111403545, 2024-04-25T10:09:40.564 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 3.744610199, 2024-04-25T10:09:40.564 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.178354152, 2024-04-25T10:09:40.564 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:09:40.564 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 60966, 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3.047824069, 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000049992 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 87964, 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.032120969, 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000365 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 133856 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 4119, 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 4112, 2024-04-25T10:09:40.565 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1116258, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 45767773, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 6, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.749542172, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.347501222, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.145736049, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.212666212, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4111, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.600063715, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000145965 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.566 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 119056, 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 29712 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 10044, 2024-04-25T10:09:40.567 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 34.481136105, 2024-04-25T10:09:40.568 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003433008 2024-04-25T10:09:40.568 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:09:40.568 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.568 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-04-25T10:09:40.568 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T10:09:40.568 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T10:09:40.568 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2094, 2024-04-25T10:09:40.568 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.655251119, 2024-04-25T10:09:40.568 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000790473 2024-04-25T10:09:40.568 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:09:40.568 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.569 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-04-25T10:09:40.569 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 27418, 2024-04-25T10:09:40.569 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 27418, 2024-04-25T10:09:40.569 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-04-25T10:09:40.569 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 27418, 2024-04-25T10:09:40.569 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 475.466309478, 2024-04-25T10:09:40.569 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.017341392 2024-04-25T10:09:40.569 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.569 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-04-25T10:09:40.569 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 12, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 19848, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 19848, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 19848, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 145, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 145, 2024-04-25T10:09:40.570 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 2, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 45590, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 45096, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 161, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-04-25T10:09:40.571 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-04-25T10:09:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 6684, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 6530, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 6684, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 63, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 17921 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-04-25T10:09:40.573 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 2000, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 124, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 4107, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 1733, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 2211, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-04-25T10:09:40.574 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:09:40.575 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-04-25T10:09:40.575 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-04-25T10:09:40.575 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:09:40.575 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-04-25T10:09:40.575 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-04-25T10:09:40.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.575 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-04-25T10:09:40.575 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-04-25T10:09:40.575 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 22932, 2024-04-25T10:09:40.575 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-04-25T10:09:40.575 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 22932, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 29, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 29, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 3, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 83872883, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-04-25T10:09:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7106, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 118.368398759, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.016657528 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 2014, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 2014, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 0, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 13, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 13, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 0, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 19848, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 19848, 2024-04-25T10:09:40.577 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 0, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 145, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 2002, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 1857, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 100156, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215340 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 37037, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 27418, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 9, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2000, 2024-04-25T10:09:40.578 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3.184369499, 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001592184 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1198, 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.968099858, 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000808096 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-04-25T10:09:40.579 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6000, 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 21.269913974, 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003544985 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7996, 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 19.266075961, 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002409464 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3, 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.027118262, 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.009039420 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.006347167, 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.006347167 2024-04-25T10:09:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.006155719, 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.006155719 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6, 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.270474408, 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.045079068 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2211, 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 393.259500290, 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.177864993 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.008205747, 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.008205747 2024-04-25T10:09:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000558425, 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000558425 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8000, 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 37.199490168, 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004649936 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:09:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 1, 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 0, 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 1, 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 13370, 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 13370, 2024-04-25T10:09:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 118, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:09:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 21247694, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 5452, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 5424, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 76, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 21332093, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 823755, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-25T10:09:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 9360, 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 80024791, 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 37, 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 9360, 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 9360, 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 43.307613187, 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004626881 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 9360, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 9554 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 9360, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 136, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 9224, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 0, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 11, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 136, 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 7242, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 78, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 1733, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 30, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 0, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 22, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-04-25T10:09:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 81, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 1, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 8, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 8, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 247, 2024-04-25T10:09:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 0, 2024-04-25T10:09:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 55 2024-04-25T10:09:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-04-25T10:09:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2024-04-25T10:09:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:09:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-04-25T10:09:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-04-25T10:09:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-04-25T10:09:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 32, 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 1733, 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 1733, 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:09:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 41586, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 10512212, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 41586, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 41586, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 10512212, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 9360, 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 80213688, 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 9360, 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 80213688, 2024-04-25T10:09:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 9360, 2024-04-25T10:09:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 9360, 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 9360, 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 9360, 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 1733, 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 188897, 2024-04-25T10:09:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 1733, 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 136, 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 188897, 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55ef95d0a960": { 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 84, 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T10:09:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:09:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 22932, 2024-04-25T10:09:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 79678579, 2024-04-25T10:09:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:09:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 22932, 2024-04-25T10:09:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:09:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:09:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 7106, 2024-04-25T10:09:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 79678495, 2024-04-25T10:09:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:09:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:09:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:09:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:09:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:09:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:09:40.607 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:09:40.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 -- 172.21.15.133:0/4259121466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8a0066f60 msgr2=0x7fd8a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8a0066f60 0x7fd8a0069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80ad000 tx=0x7fd8b8008040 comp rx=0 tx=0).stop 2024-04-25T10:09:40.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 -- 172.21.15.133:0/4259121466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8bc0ce450 msgr2=0x7fd8bc15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8bc0ce450 0x7fd8bc15a080 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00c420 tx=0x7fd8ac00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 -- 172.21.15.133:0/4259121466 shutdown_connections 2024-04-25T10:09:40.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8bc0ce8e0 0x7fd8bc15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8a0066f60 0x7fd8a0069420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8bc0ce450 0x7fd8bc15a080 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 --2- 172.21.15.133:0/4259121466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8bc06e970 0x7fd8bc0a9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 -- 172.21.15.133:0/4259121466 >> 172.21.15.133:0/4259121466 conn(0x7fd8bc066f70 msgr2=0x7fd8bc15e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 -- 172.21.15.133:0/4259121466 shutdown_connections 2024-04-25T10:09:40.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.551+0000 7fd8c32bd640 1 -- 172.21.15.133:0/4259121466 wait complete. 2024-04-25T10:09:40.610 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T10:09:40.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:40.611 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T10:09:40.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.723+0000 7fcade2f4640 1 Processor -- start 2024-04-25T10:09:40.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.723+0000 7fcade2f4640 1 -- start start 2024-04-25T10:09:40.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.723+0000 7fcade2f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad815ba20 0x7fcad80cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.723+0000 7fcade2f4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcad80d0210 0x7fcad80b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.723+0000 7fcade2f4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad80d0630 con 0x7fcad815b640 2024-04-25T10:09:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcade2f4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad815f460 con 0x7fcad80d0210 2024-04-25T10:09:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcade2f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad81602c0 con 0x7fcad815ba20 2024-04-25T10:09:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcad7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcad815b640 0x7fcad80cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58228/0 (socket says 172.21.15.133:58228) 2024-04-25T10:09:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcad7fff640 1 -- 172.21.15.133:0/3293184784 learned_addr learned my addr 172.21.15.133:0/3293184784 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:09:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcadc891640 1 --2- 172.21.15.133:0/3293184784 >> v2:172.21.15.133:3300/0 conn(0x7fcad80d0210 0x7fcad80b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/3293184784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad815ba20 0x7fcad80cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcadc891640 1 -- 172.21.15.133:0/3293184784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad815ba20 msgr2=0x7fcad80cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcadc891640 1 --2- 172.21.15.133:0/3293184784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad815ba20 0x7fcad80cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcadc891640 1 -- 172.21.15.133:0/3293184784 >> v1:172.21.15.133:6789/0 conn(0x7fcad815b640 legacy=0x7fcad80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcadc891640 1 -- 172.21.15.133:0/3293184784 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad80cc300 con 0x7fcad80d0210 2024-04-25T10:09:40.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcad77fe640 1 -- 172.21.15.133:0/3293184784 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299666504 0 0) 0x7fcad80d0630 con 0x7fcad815b640 2024-04-25T10:09:40.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/3293184784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad815ba20 0x7fcad80cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcadc891640 1 --2- 172.21.15.133:0/3293184784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad80d0210 0x7fcad80b15d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009d90 tx=0x7fcad001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce579b2c79be0482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcad77fe640 1 -- 172.21.15.133:0/3293184784 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00b2070 con 0x7fcad80d0210 2024-04-25T10:09:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcad77fe640 1 -- 172.21.15.133:0/3293184784 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad00a5830 con 0x7fcad80d0210 2024-04-25T10:09:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcad77fe640 1 -- 172.21.15.133:0/3293184784 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00a5c00 con 0x7fcad80d0210 2024-04-25T10:09:40.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcade2f4640 1 -- 172.21.15.133:0/3293184784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad80d0210 msgr2=0x7fcad80b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcade2f4640 1 --2- 172.21.15.133:0/3293184784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad80d0210 0x7fcad80b15d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009d90 tx=0x7fcad001c970 comp rx=0 tx=0).stop 2024-04-25T10:09:40.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcade2f4640 1 -- 172.21.15.133:0/3293184784 shutdown_connections 2024-04-25T10:09:40.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcade2f4640 1 --2- 172.21.15.133:0/3293184784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad80d0210 0x7fcad80b15d0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcade2f4640 1 --2- 172.21.15.133:0/3293184784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad815ba20 0x7fcad80cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcade2f4640 1 -- 172.21.15.133:0/3293184784 >> 172.21.15.133:0/3293184784 conn(0x7fcad809aa00 msgr2=0x7fcad808af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcade2f4640 1 -- 172.21.15.133:0/3293184784 shutdown_connections 2024-04-25T10:09:40.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.727+0000 7fcade2f4640 1 -- 172.21.15.133:0/3293184784 wait complete. 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcade2f4640 1 Processor -- start 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcade2f4640 1 -- start start 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcade2f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad80d0210 0x7fcad81591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcade2f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad815b640 0x7fcad8155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcade2f4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcad815ba20 0x7fcad8155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcade2f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80cf9d0 con 0x7fcad815b640 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcade2f4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad815fd00 con 0x7fcad815ba20 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcade2f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80b2320 con 0x7fcad80d0210 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcacf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad815b640 0x7fcad8155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcacf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad815b640 0x7fcad8155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47834/0 (socket says 172.21.15.133:47834) 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcacf7fe640 1 -- 172.21.15.133:0/1015466955 learned_addr learned my addr 172.21.15.133:0/1015466955 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcacf7fe640 1 -- 172.21.15.133:0/1015466955 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad80d0210 msgr2=0x7fcad81591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcad7fff640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad80d0210 0x7fcad81591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcadc891640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcad815ba20 0x7fcad8155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad80d0210 0x7fcad81591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcacf7fe640 1 -- 172.21.15.133:0/1015466955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcad815ba20 msgr2=0x7fcad8155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcad815ba20 0x7fcad8155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcacf7fe640 1 -- 172.21.15.133:0/1015466955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad80cd1d0 con 0x7fcad815b640 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad815b640 0x7fcad8155710 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fcac000c7f0 tx=0x7fcac000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcad57fa640 1 -- 172.21.15.133:0/1015466955 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0015070 con 0x7fcad815b640 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcad57fa640 1 -- 172.21.15.133:0/1015466955 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac0007820 con 0x7fcad815b640 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcade2f4640 1 -- 172.21.15.133:0/1015466955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad80cd4b0 con 0x7fcad815b640 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcad57fa640 1 -- 172.21.15.133:0/1015466955 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0007c10 con 0x7fcad815b640 2024-04-25T10:09:40.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.731+0000 7fcade2f4640 1 -- 172.21.15.133:0/1015466955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad80cda70 con 0x7fcad815b640 2024-04-25T10:09:40.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcad57fa640 1 -- 172.21.15.133:0/1015466955 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcac0018070 con 0x7fcad815b640 2024-04-25T10:09:40.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcad57fa640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcab8066f60 0x7fcab8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcad57fa640 1 -- 172.21.15.133:0/1015466955 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcac0085330 con 0x7fcad815b640 2024-04-25T10:09:40.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcad7fff640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcab8066f60 0x7fcab8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcad7fff640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcab8066f60 0x7fcab8069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000f70 tx=0x7fcac8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 Processor -- start 2024-04-25T10:09:40.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 -- start start 2024-04-25T10:09:40.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7288c0 0x555a7e728ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 0x555a7e6de110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e6dea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7e52cdd0 con 0x555a7e6de650 2024-04-25T10:09:40.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7e4b5010 con 0x555a7e7288c0 2024-04-25T10:09:40.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a7e528e70 con 0x555a7e7291e0 2024-04-25T10:09:40.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcadc891640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e6dea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcadc891640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e6dea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47842/0 (socket says 172.21.15.133:47842) 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcadc891640 1 -- 172.21.15.133:0/4148416505 learned_addr learned my addr 172.21.15.133:0/4148416505 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/4148416505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 0x555a7e6de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcad7fff640 1 --2- 172.21.15.133:0/4148416505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7288c0 0x555a7e728ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcadc891640 1 -- 172.21.15.133:0/4148416505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 msgr2=0x555a7e6de110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcadc891640 1 --2- 172.21.15.133:0/4148416505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 0x555a7e6de110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcadc891640 1 -- 172.21.15.133:0/4148416505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7288c0 msgr2=0x555a7e728ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcadc891640 1 --2- 172.21.15.133:0/4148416505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7288c0 0x555a7e728ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcadc891640 1 -- 172.21.15.133:0/4148416505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a7e4e59c0 con 0x555a7e6de650 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/4148416505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 0x555a7e6de110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcadc891640 1 --2- 172.21.15.133:0/4148416505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e6dea30 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fcad001b020 tx=0x7fcad0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f09c63b5977ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcacdffb640 1 -- 172.21.15.133:0/4148416505 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00b2040 con 0x555a7e6de650 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcacdffb640 1 -- 172.21.15.133:0/4148416505 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad0003aa0 con 0x555a7e6de650 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcacdffb640 1 -- 172.21.15.133:0/4148416505 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0003e50 con 0x555a7e6de650 2024-04-25T10:09:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 -- 172.21.15.133:0/4148416505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 msgr2=0x555a7e6dea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 --2- 172.21.15.133:0/4148416505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e6dea30 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fcad001b020 tx=0x7fcad0004ac0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 -- 172.21.15.133:0/4148416505 shutdown_connections 2024-04-25T10:09:40.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 --2- 172.21.15.133:0/4148416505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e6dea30 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 --2- 172.21.15.133:0/4148416505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 0x555a7e6de110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 --2- 172.21.15.133:0/4148416505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7288c0 0x555a7e728ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 -- 172.21.15.133:0/4148416505 >> 172.21.15.133:0/4148416505 conn(0x555a7e5c6870 msgr2=0x555a7e6dc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 -- 172.21.15.133:0/4148416505 shutdown_connections 2024-04-25T10:09:40.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.735+0000 7fcae1461000 1 -- 172.21.15.133:0/4148416505 wait complete. 2024-04-25T10:09:40.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 -- start start 2024-04-25T10:09:40.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 Processor -- start 2024-04-25T10:09:40.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e6de650 0x555a7e6e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7288c0 0x555a7e6e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 0x555a7e6e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7e4e7420 con 0x555a7e7288c0 2024-04-25T10:09:40.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7e6e7c20 con 0x555a7e6de650 2024-04-25T10:09:40.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a7e58f000 con 0x555a7e7291e0 2024-04-25T10:09:40.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 10 client.0 _ll_register_callbacks cb 0x555a7e6c9f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:40.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcacf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7288c0 0x555a7e6e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcacf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7288c0 0x555a7e6e34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47852/0 (socket says 172.21.15.133:47852) 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcacf7fe640 1 -- 172.21.15.133:0/1225099521 learned_addr learned my addr 172.21.15.133:0/1225099521 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcadc891640 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 0x555a7e6e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcacf7fe640 1 -- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 msgr2=0x555a7e6e39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 0x555a7e6e39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcad7fff640 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e6de650 0x555a7e6e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcacf7fe640 1 -- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e6de650 msgr2=0x555a7e6e2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e6de650 0x555a7e6e2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcacf7fe640 1 -- 172.21.15.133:0/1225099521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a7e4b5260 con 0x555a7e7288c0 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcadc891640 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 0x555a7e6e39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7288c0 0x555a7e6e34b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fcac00836d0 tx=0x7fcac0055bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e2a6d9aee9663 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcab6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcab6ffd640 1 -- 172.21.15.133:0/1225099521 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac006d020 con 0x555a7e7288c0 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcab6ffd640 1 -- 172.21.15.133:0/1225099521 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcac0055100 con 0x555a7e7288c0 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 -- 172.21.15.133:0/1225099521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcad0003aa0 con 0x555a7e7288c0 2024-04-25T10:09:40.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcab6ffd640 1 -- 172.21.15.133:0/1225099521 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcac001f040 con 0x555a7e7288c0 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 10 client.7623 fetch_fsmap learned FSMap version 50 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 -- 172.21.15.133:0/1225099521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x555a7e4e7b30 con 0x555a7e7288c0 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcab6ffd640 1 -- 172.21.15.133:0/1225099521 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad0003aa0 con 0x555a7e7288c0 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcab6ffd640 1 -- 172.21.15.133:0/1225099521 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcac005ab20 con 0x555a7e7288c0 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 10 client.7623 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 10 client.7623 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 20 client.7623 populate_metadata read hostname 'smithi133' 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x555a7e6e4400 0x555a7e739960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 4 client.7623 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcae1461000 1 -- 172.21.15.133:0/1225099521 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a7e4c8c80 con 0x555a7e6e4400 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcad7fff640 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x555a7e6e4400 0x555a7e739960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcab6ffd640 10 client.7623 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.739+0000 7fcad7fff640 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x555a7e6e4400 0x555a7e739960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b86f402a3e0802b server_cookie=7868ec307f356bd in_seq=0 out_seq=0 2024-04-25T10:09:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcab6ffd640 1 -- 172.21.15.133:0/1225099521 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a7e4c8c80 con 0x555a7e6e4400 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcab6ffd640 10 client.7623 handle_command_reply: tid=0 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 1 client.7623 shutdown 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 1 -- 172.21.15.133:0/1225099521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7288c0 msgr2=0x555a7e6e34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7288c0 0x555a7e6e34b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fcac00836d0 tx=0x7fcac0055bf0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 1 -- 172.21.15.133:0/1225099521 shutdown_connections 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x555a7e6e4400 0x555a7e739960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7291e0 0x555a7e6e39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcab6ffd640 0 client.7623 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7288c0 0x555a7e6e34b0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 1 --2- 172.21.15.133:0/1225099521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e6de650 0x555a7e6e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 1 -- 172.21.15.133:0/1225099521 >> 172.21.15.133:0/1225099521 conn(0x555a7e5c6870 msgr2=0x555a7e6dc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 1 -- 172.21.15.133:0/1225099521 shutdown_connections 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 1 -- 172.21.15.133:0/1225099521 wait complete. 2024-04-25T10:09:40.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.743+0000 7fcae1461000 20 client.7623 trim_cache size 0 max 16384 2024-04-25T10:09:40.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.755+0000 7fcae1461000 1 Processor -- start 2024-04-25T10:09:40.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- start start 2024-04-25T10:09:40.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e6de650 0x555a7e784230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7288c0 0x555a7e784770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7291e0 0x555a7e784cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad8200a80 con 0x555a7e7291e0 2024-04-25T10:09:40.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad81ffce0 con 0x555a7e7288c0 2024-04-25T10:09:40.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80c9910 con 0x555a7e6de650 2024-04-25T10:09:40.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcadc891640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7291e0 0x555a7e784cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcadc891640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7291e0 0x555a7e784cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47854/0 (socket says 172.21.15.133:47854) 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcadc891640 1 -- 172.21.15.133:0/2044632166 learned_addr learned my addr 172.21.15.133:0/2044632166 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcadc891640 1 -- 172.21.15.133:0/2044632166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e6de650 msgr2=0x555a7e784230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcadc891640 1 --2- 172.21.15.133:0/2044632166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e6de650 0x555a7e784230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcadc891640 1 -- 172.21.15.133:0/2044632166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7288c0 msgr2=0x555a7e784770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcadc891640 1 --2- 172.21.15.133:0/2044632166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7288c0 0x555a7e784770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcadc891640 1 -- 172.21.15.133:0/2044632166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a7e5b26d0 con 0x555a7e7291e0 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcadc891640 1 --2- 172.21.15.133:0/2044632166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7291e0 0x555a7e784cb0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fcad0004b00 tx=0x7fcad00a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59d938f02129200 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcacd7fa640 1 -- 172.21.15.133:0/2044632166 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00b2070 con 0x555a7e7291e0 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcacd7fa640 1 -- 172.21.15.133:0/2044632166 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad00a5630 con 0x555a7e7291e0 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcacd7fa640 1 -- 172.21.15.133:0/2044632166 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00a5310 con 0x555a7e7291e0 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- 172.21.15.133:0/2044632166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7291e0 msgr2=0x555a7e784cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 --2- 172.21.15.133:0/2044632166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7291e0 0x555a7e784cb0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fcad0004b00 tx=0x7fcad00a7000 comp rx=0 tx=0).stop 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- 172.21.15.133:0/2044632166 shutdown_connections 2024-04-25T10:09:40.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 --2- 172.21.15.133:0/2044632166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e7291e0 0x555a7e784cb0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 --2- 172.21.15.133:0/2044632166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7288c0 0x555a7e784770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 --2- 172.21.15.133:0/2044632166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e6de650 0x555a7e784230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- 172.21.15.133:0/2044632166 >> 172.21.15.133:0/2044632166 conn(0x555a7e5c6870 msgr2=0x555a7e73bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- 172.21.15.133:0/2044632166 shutdown_connections 2024-04-25T10:09:40.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- 172.21.15.133:0/2044632166 wait complete. 2024-04-25T10:09:40.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- start start 2024-04-25T10:09:40.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 Processor -- start 2024-04-25T10:09:40.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e785840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7288c0 0x555a7e785d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7291e0 0x555a7e7862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7e52c0e0 con 0x555a7e6de650 2024-04-25T10:09:40.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80c9f10 con 0x555a7e7291e0 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a7e4d3ae0 con 0x555a7e7288c0 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcae1461000 10 client.0 _ll_register_callbacks cb 0x555a7e4c9200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcad7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e785840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcad7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e785840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47870/0 (socket says 172.21.15.133:47870) 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcad7fff640 1 -- 172.21.15.133:0/2157588509 learned_addr learned my addr 172.21.15.133:0/2157588509 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7288c0 0x555a7e785d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.759+0000 7fcadc891640 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7291e0 0x555a7e7862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcad7fff640 1 -- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7288c0 msgr2=0x555a7e785d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcad7fff640 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7288c0 0x555a7e785d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcad7fff640 1 -- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7291e0 msgr2=0x555a7e7862c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcad7fff640 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7291e0 0x555a7e7862c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcad7fff640 1 -- 172.21.15.133:0/2157588509 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a7e5af900 con 0x555a7e6de650 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcadc891640 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7291e0 0x555a7e7862c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcad7fff640 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e785840 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcac8007bc0 tx=0x7fcac8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d677d39d513727 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcab6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcab6ffd640 1 -- 172.21.15.133:0/2157588509 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac801d040 con 0x555a7e6de650 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcab6ffd640 1 -- 172.21.15.133:0/2157588509 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcac8004c00 con 0x555a7e6de650 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcae1461000 1 -- 172.21.15.133:0/2157588509 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcad00a5630 con 0x555a7e6de650 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcab6ffd640 1 -- 172.21.15.133:0/2157588509 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcac8033050 con 0x555a7e6de650 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcae1461000 10 client.7629 fetch_fsmap learned FSMap version 50 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcae1461000 1 -- 172.21.15.133:0/2157588509 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x555a7e4d3f20 con 0x555a7e6de650 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcab6ffd640 1 -- 172.21.15.133:0/2157588509 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad00a5630 con 0x555a7e6de650 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcab6ffd640 1 -- 172.21.15.133:0/2157588509 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcac8036020 con 0x555a7e6de650 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcae1461000 10 client.7629 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcae1461000 10 client.7629 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcae1461000 20 client.7629 populate_metadata read hostname 'smithi133' 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcae1461000 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x555a7e786800 0x555a7e786c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcae1461000 4 client.7629 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T10:09:40.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcae1461000 1 -- 172.21.15.133:0/2157588509 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x555a7e53a310 con 0x555a7e786800 2024-04-25T10:09:40.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x555a7e786800 0x555a7e786c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcab6ffd640 10 client.7629 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.763+0000 7fcacf7fe640 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x555a7e786800 0x555a7e786c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774880fa94f73a82 server_cookie=ac6a954e6139123d in_seq=0 out_seq=0 2024-04-25T10:09:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcab6ffd640 1 -- 172.21.15.133:0/2157588509 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555a7e53a310 con 0x555a7e786800 2024-04-25T10:09:40.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcab6ffd640 10 client.7629 handle_command_reply: tid=0 2024-04-25T10:09:40.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 1 client.7629 shutdown 2024-04-25T10:09:40.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 1 -- 172.21.15.133:0/2157588509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 msgr2=0x555a7e785840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e785840 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcac8007bc0 tx=0x7fcac8000ee0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 1 -- 172.21.15.133:0/2157588509 shutdown_connections 2024-04-25T10:09:40.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x555a7e786800 0x555a7e786c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555a7e7291e0 0x555a7e7862c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555a7e7288c0 0x555a7e785d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 1 --2- 172.21.15.133:0/2157588509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a7e6de650 0x555a7e785840 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 1 -- 172.21.15.133:0/2157588509 >> 172.21.15.133:0/2157588509 conn(0x555a7e5c6870 msgr2=0x555a7e727910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcab6ffd640 0 client.7629 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 1 -- 172.21.15.133:0/2157588509 shutdown_connections 2024-04-25T10:09:40.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 1 -- 172.21.15.133:0/2157588509 wait complete. 2024-04-25T10:09:40.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcae1461000 20 client.7629 trim_cache size 0 max 16384 2024-04-25T10:09:40.791 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:09:40.791 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:09:40.791 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "d9de1b73-a0d0-4c6a-845d-c91679388f6d", 2024-04-25T10:09:40.791 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:09:40.791 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:09:40.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 -- 172.21.15.133:0/1015466955 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcab8066f60 msgr2=0x7fcab8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcab8066f60 0x7fcab8069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000f70 tx=0x7fcac8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:40.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 -- 172.21.15.133:0/1015466955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad815b640 msgr2=0x7fcad8155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad815b640 0x7fcad8155710 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fcac000c7f0 tx=0x7fcac000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 -- 172.21.15.133:0/1015466955 shutdown_connections 2024-04-25T10:09:40.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcad815ba20 0x7fcad8155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcab8066f60 0x7fcab8069420 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcad815b640 0x7fcad8155710 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 --2- 172.21.15.133:0/1015466955 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcad80d0210 0x7fcad81591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 -- 172.21.15.133:0/1015466955 >> 172.21.15.133:0/1015466955 conn(0x7fcad809aa00 msgr2=0x7fcad8070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 -- 172.21.15.133:0/1015466955 shutdown_connections 2024-04-25T10:09:40.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.783+0000 7fcade2f4640 1 -- 172.21.15.133:0/1015466955 wait complete. 2024-04-25T10:09:40.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:40.806 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:09:40.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6ed81640 1 Processor -- start 2024-04-25T10:09:40.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6ed81640 1 -- start start 2024-04-25T10:09:40.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6ed81640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa6815b8b0 0x7faa680ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6ed81640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faa6815bc90 0x7faa680cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6ed81640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa680d01d0 0x7faa6815f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6ed81640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa680d0600 con 0x7faa6815b8b0 2024-04-25T10:09:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6ed81640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa680cc6a0 con 0x7faa6815bc90 2024-04-25T10:09:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6ed81640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa680cc980 con 0x7faa680d01d0 2024-04-25T10:09:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6cb1d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa6815b8b0 0x7faa680ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6cb1d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa6815b8b0 0x7faa680ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47884/0 (socket says 172.21.15.133:47884) 2024-04-25T10:09:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6cb1d640 1 -- 172.21.15.133:0/430498083 learned_addr learned my addr 172.21.15.133:0/430498083 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa67fff640 1 --2- 172.21.15.133:0/430498083 >> v2:172.21.15.162:3300/0 conn(0x7faa6815bc90 0x7faa680cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6d31e640 1 --2- 172.21.15.133:0/430498083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa680d01d0 0x7faa6815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6cb1d640 1 -- 172.21.15.133:0/430498083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa680d01d0 msgr2=0x7faa6815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/430498083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa680d01d0 0x7faa6815f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6cb1d640 1 -- 172.21.15.133:0/430498083 >> v2:172.21.15.162:3300/0 conn(0x7faa6815bc90 msgr2=0x7faa680cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/430498083 >> v2:172.21.15.162:3300/0 conn(0x7faa6815bc90 0x7faa680cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.927+0000 7faa6cb1d640 1 -- 172.21.15.133:0/430498083 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa680ccc60 con 0x7faa6815b8b0 2024-04-25T10:09:40.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/430498083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa6815b8b0 0x7faa680ce6d0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7faa6806ec50 tx=0x7faa5c01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edfdfe49c4a5f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa677fe640 1 -- 172.21.15.133:0/430498083 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0b0070 con 0x7faa6815b8b0 2024-04-25T10:09:40.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa677fe640 1 -- 172.21.15.133:0/430498083 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c0ab600 con 0x7faa6815b8b0 2024-04-25T10:09:40.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa677fe640 1 -- 172.21.15.133:0/430498083 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0ab9d0 con 0x7faa6815b8b0 2024-04-25T10:09:40.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6ed81640 1 -- 172.21.15.133:0/430498083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa6815b8b0 msgr2=0x7faa680ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6ed81640 1 --2- 172.21.15.133:0/430498083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa6815b8b0 0x7faa680ce6d0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7faa6806ec50 tx=0x7faa5c01c1d0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6ed81640 1 -- 172.21.15.133:0/430498083 shutdown_connections 2024-04-25T10:09:40.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6ed81640 1 --2- 172.21.15.133:0/430498083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa680d01d0 0x7faa6815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6ed81640 1 --2- 172.21.15.133:0/430498083 >> v2:172.21.15.162:3300/0 conn(0x7faa6815bc90 0x7faa680cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6ed81640 1 --2- 172.21.15.133:0/430498083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa6815b8b0 0x7faa680ce6d0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6ed81640 1 -- 172.21.15.133:0/430498083 >> 172.21.15.133:0/430498083 conn(0x7faa6809a990 msgr2=0x7faa680d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6ed81640 1 -- 172.21.15.133:0/430498083 shutdown_connections 2024-04-25T10:09:40.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6ed81640 1 -- 172.21.15.133:0/430498083 wait complete. 2024-04-25T10:09:40.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6ed81640 1 Processor -- start 2024-04-25T10:09:40.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.931+0000 7faa6ed81640 1 -- start start 2024-04-25T10:09:40.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa6ed81640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa680d01d0 0x7faa6814f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa6ed81640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa6815b8b0 0x7faa68154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa6ed81640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa6815bc90 0x7faa680b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa6ed81640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa680cc3b0 con 0x7faa6815b8b0 2024-04-25T10:09:40.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa6ed81640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa680cc230 con 0x7faa6815bc90 2024-04-25T10:09:40.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa6ed81640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa680b0aa0 con 0x7faa680d01d0 2024-04-25T10:09:40.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa67fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa6815b8b0 0x7faa68154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa67fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa6815b8b0 0x7faa68154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47896/0 (socket says 172.21.15.133:47896) 2024-04-25T10:09:40.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa67fff640 1 -- 172.21.15.133:0/854485067 learned_addr learned my addr 172.21.15.133:0/854485067 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa6d31e640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa6815bc90 0x7faa680b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa680d01d0 0x7faa6814f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa67fff640 1 -- 172.21.15.133:0/854485067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa680d01d0 msgr2=0x7faa6814f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa67fff640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa680d01d0 0x7faa6814f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa67fff640 1 -- 172.21.15.133:0/854485067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa6815bc90 msgr2=0x7faa680b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa67fff640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa6815bc90 0x7faa680b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa67fff640 1 -- 172.21.15.133:0/854485067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa680b4dd0 con 0x7faa6815b8b0 2024-04-25T10:09:40.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa67fff640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa6815b8b0 0x7faa68154430 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7faa5800e8a0 tx=0x7faa5800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa657fa640 1 -- 172.21.15.133:0/854485067 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58017070 con 0x7faa6815b8b0 2024-04-25T10:09:40.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa6ed81640 1 -- 172.21.15.133:0/854485067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa680b50b0 con 0x7faa6815b8b0 2024-04-25T10:09:40.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa6ed81640 1 -- 172.21.15.133:0/854485067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa680b5670 con 0x7faa6815b8b0 2024-04-25T10:09:40.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa657fa640 1 -- 172.21.15.133:0/854485067 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5800c940 con 0x7faa6815b8b0 2024-04-25T10:09:40.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa657fa640 1 -- 172.21.15.133:0/854485067 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5800ccf0 con 0x7faa6815b8b0 2024-04-25T10:09:40.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.935+0000 7faa657fa640 1 -- 172.21.15.133:0/854485067 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa5801a070 con 0x7faa6815b8b0 2024-04-25T10:09:40.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa657fa640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa54066f60 0x7faa54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa54066f60 0x7faa54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa657fa640 1 -- 172.21.15.133:0/854485067 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faa580853d0 con 0x7faa6815b8b0 2024-04-25T10:09:40.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa54066f60 0x7faa54069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0ad000 tx=0x7faa5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa71eee000 1 Processor -- start 2024-04-25T10:09:40.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa71eee000 1 -- start start 2024-04-25T10:09:40.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa71eee000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa71eee000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2b82b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa71eee000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2b830a0 0x55eba2b83480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa71eee000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba29cfff0 con 0x55eba2bcc9d0 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa71eee000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba2958150 con 0x55eba2b830a0 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa71eee000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eba29cc090 con 0x55eba2bcd2f0 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6cb1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6cb1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47904/0 (socket says 172.21.15.133:47904) 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6cb1d640 1 -- 172.21.15.133:0/1707711446 learned_addr learned my addr 172.21.15.133:0/1707711446 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6d31e640 1 --2- 172.21.15.133:0/1707711446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2b830a0 0x55eba2b83480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa67fff640 1 --2- 172.21.15.133:0/1707711446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2b82b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6cb1d640 1 -- 172.21.15.133:0/1707711446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 msgr2=0x55eba2b82b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/1707711446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2b82b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6cb1d640 1 -- 172.21.15.133:0/1707711446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2b830a0 msgr2=0x55eba2b83480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/1707711446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2b830a0 0x55eba2b83480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6cb1d640 1 -- 172.21.15.133:0/1707711446 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eba2b660d0 con 0x55eba2bcc9d0 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6d31e640 1 --2- 172.21.15.133:0/1707711446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2b830a0 0x55eba2b83480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa67fff640 1 --2- 172.21.15.133:0/1707711446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2b82b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/1707711446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bccdb0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0ae040 tx=0x7faa5c0a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4912d571c40710 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa4e7fc640 1 -- 172.21.15.133:0/1707711446 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0c7070 con 0x55eba2bcc9d0 2024-04-25T10:09:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa4e7fc640 1 -- 172.21.15.133:0/1707711446 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c0b1040 con 0x55eba2bcc9d0 2024-04-25T10:09:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa4e7fc640 1 -- 172.21.15.133:0/1707711446 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0b7630 con 0x55eba2bcc9d0 2024-04-25T10:09:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa71eee000 1 -- 172.21.15.133:0/1707711446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 msgr2=0x55eba2bccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.939+0000 7faa71eee000 1 --2- 172.21.15.133:0/1707711446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bccdb0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0ae040 tx=0x7faa5c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:09:40.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 -- 172.21.15.133:0/1707711446 shutdown_connections 2024-04-25T10:09:40.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 --2- 172.21.15.133:0/1707711446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2b830a0 0x55eba2b83480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 --2- 172.21.15.133:0/1707711446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2b82b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 --2- 172.21.15.133:0/1707711446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bccdb0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 -- 172.21.15.133:0/1707711446 >> 172.21.15.133:0/1707711446 conn(0x55eba2b7f020 msgr2=0x55eba2b81440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 -- 172.21.15.133:0/1707711446 shutdown_connections 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 -- 172.21.15.133:0/1707711446 wait complete. 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 -- start start 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 Processor -- start 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2bb17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcc9d0 0x55eba2bb1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2bcd2f0 0x55eba2bb2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba2994250 con 0x55eba2b830a0 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba298a690 con 0x55eba2bcd2f0 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eba2958a40 con 0x55eba2bcc9d0 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 10 client.0 _ll_register_callbacks cb 0x55eba2b6de90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa6cb1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2bb17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa6cb1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2bb17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47916/0 (socket says 172.21.15.133:47916) 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa6cb1d640 1 -- 172.21.15.133:0/3945666535 learned_addr learned my addr 172.21.15.133:0/3945666535 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa67fff640 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcc9d0 0x55eba2bb1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa6d31e640 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2bcd2f0 0x55eba2bb2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa6cb1d640 1 -- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcc9d0 msgr2=0x55eba2bb1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcc9d0 0x55eba2bb1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa6cb1d640 1 -- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2bcd2f0 msgr2=0x55eba2bb2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2bcd2f0 0x55eba2bb2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa6cb1d640 1 -- 172.21.15.133:0/3945666535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eba2a7d1c0 con 0x55eba2b830a0 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2bb17f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7faa5c01c6b0 tx=0x7faa5c01aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b95ea6e3069246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa477fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa477fe640 1 -- 172.21.15.133:0/3945666535 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0d1060 con 0x55eba2b830a0 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa477fe640 1 -- 172.21.15.133:0/3945666535 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faa5c0b7bd0 con 0x55eba2b830a0 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 -- 172.21.15.133:0/3945666535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa5c0b1040 con 0x55eba2b830a0 2024-04-25T10:09:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa477fe640 1 -- 172.21.15.133:0/3945666535 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa5c0d7050 con 0x55eba2b830a0 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 10 client.7641 fetch_fsmap learned FSMap version 50 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 -- 172.21.15.133:0/3945666535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55eba2a79170 con 0x55eba2b830a0 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa477fe640 1 -- 172.21.15.133:0/3945666535 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c0b1040 con 0x55eba2b830a0 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa477fe640 1 -- 172.21.15.133:0/3945666535 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faa5c0da020 con 0x55eba2b830a0 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 10 client.7641 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 10 client.7641 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 20 client.7641 populate_metadata read hostname 'smithi133' 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55eba2bdcc80 0x55eba2bdf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 4 client.7641 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa71eee000 1 -- 172.21.15.133:0/3945666535 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eba296bc60 con 0x55eba2bdcc80 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.943+0000 7faa67fff640 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55eba2bdcc80 0x55eba2bdf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa477fe640 10 client.7641 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa67fff640 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55eba2bdcc80 0x55eba2bdf070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d5bd6a71b6539b1 server_cookie=32ecda20ec16a813 in_seq=0 out_seq=0 2024-04-25T10:09:40.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa477fe640 1 -- 172.21.15.133:0/3945666535 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eba296bc60 con 0x55eba2bdcc80 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa477fe640 10 client.7641 handle_command_reply: tid=0 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 1 client.7641 shutdown 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 1 -- 172.21.15.133:0/3945666535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 msgr2=0x55eba2bb17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2bb17f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7faa5c01c6b0 tx=0x7faa5c01aa10 comp rx=0 tx=0).stop 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 1 -- 172.21.15.133:0/3945666535 shutdown_connections 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55eba2bdcc80 0x55eba2bdf070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2bcd2f0 0x55eba2bb2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcc9d0 0x55eba2bb1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa477fe640 0 client.7641 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 1 --2- 172.21.15.133:0/3945666535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2bb17f0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 1 -- 172.21.15.133:0/3945666535 >> 172.21.15.133:0/3945666535 conn(0x55eba2b7f020 msgr2=0x55eba2b81ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 1 -- 172.21.15.133:0/3945666535 shutdown_connections 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 1 -- 172.21.15.133:0/3945666535 wait complete. 2024-04-25T10:09:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.947+0000 7faa71eee000 20 client.7641 trim_cache size 0 max 16384 2024-04-25T10:09:40.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 Processor -- start 2024-04-25T10:09:40.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 -- start start 2024-04-25T10:09:40.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2b81410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2bcc9d0 0x55eba2b81950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2b81e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4800cc20 con 0x55eba2b830a0 2024-04-25T10:09:40.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4800bd40 con 0x55eba2bcc9d0 2024-04-25T10:09:40.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4800b100 con 0x55eba2bcd2f0 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa6cb1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2b81410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa6cb1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2b81410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47930/0 (socket says 172.21.15.133:47930) 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa6cb1d640 1 -- 172.21.15.133:0/2975492113 learned_addr learned my addr 172.21.15.133:0/2975492113 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa6d31e640 1 --2- 172.21.15.133:0/2975492113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2b81e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa67fff640 1 --2- 172.21.15.133:0/2975492113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2bcc9d0 0x55eba2b81950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa6cb1d640 1 -- 172.21.15.133:0/2975492113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 msgr2=0x55eba2b81e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/2975492113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2b81e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa6cb1d640 1 -- 172.21.15.133:0/2975492113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2bcc9d0 msgr2=0x55eba2b81950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/2975492113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2bcc9d0 0x55eba2b81950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa6cb1d640 1 -- 172.21.15.133:0/2975492113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eba2a16de0 con 0x55eba2b830a0 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa6d31e640 1 --2- 172.21.15.133:0/2975492113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2b81e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa67fff640 1 --2- 172.21.15.133:0/2975492113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2bcc9d0 0x55eba2b81950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:40.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/2975492113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2b81410 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7faa5c01ca70 tx=0x7faa5c0ca000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2e260240cdca0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa4dffb640 1 -- 172.21.15.133:0/2975492113 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0c7070 con 0x55eba2b830a0 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa4dffb640 1 -- 172.21.15.133:0/2975492113 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c0b1040 con 0x55eba2b830a0 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa4dffb640 1 -- 172.21.15.133:0/2975492113 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0b7b40 con 0x55eba2b830a0 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 -- 172.21.15.133:0/2975492113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 msgr2=0x55eba2b81410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 --2- 172.21.15.133:0/2975492113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2b81410 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7faa5c01ca70 tx=0x7faa5c0ca000 comp rx=0 tx=0).stop 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 -- 172.21.15.133:0/2975492113 shutdown_connections 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 --2- 172.21.15.133:0/2975492113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2b81e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 --2- 172.21.15.133:0/2975492113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2bcc9d0 0x55eba2b81950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 --2- 172.21.15.133:0/2975492113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2b830a0 0x55eba2b81410 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 -- 172.21.15.133:0/2975492113 >> 172.21.15.133:0/2975492113 conn(0x55eba2b7f020 msgr2=0x55eba2bcb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 -- 172.21.15.133:0/2975492113 shutdown_connections 2024-04-25T10:09:40.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 -- 172.21.15.133:0/2975492113 wait complete. 2024-04-25T10:09:40.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 -- start start 2024-04-25T10:09:40.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.967+0000 7faa71eee000 1 Processor -- start 2024-04-25T10:09:40.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2b830a0 0x55eba2bdf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bdf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2bdfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba296ae00 con 0x55eba2bcc9d0 2024-04-25T10:09:40.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4800b700 con 0x55eba2b830a0 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eba29cf300 con 0x55eba2bcd2f0 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 10 client.0 _ll_register_callbacks cb 0x55eba2b4e040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa67fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bdf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa6cb1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2b830a0 0x55eba2bdf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa67fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bdf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47934/0 (socket says 172.21.15.133:47934) 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa67fff640 1 -- 172.21.15.133:0/4280363315 learned_addr learned my addr 172.21.15.133:0/4280363315 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa6d31e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2bdfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa67fff640 1 -- 172.21.15.133:0/4280363315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 msgr2=0x55eba2bdfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa67fff640 1 --2- 172.21.15.133:0/4280363315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2bdfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa67fff640 1 -- 172.21.15.133:0/4280363315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2b830a0 msgr2=0x55eba2bdf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa67fff640 1 --2- 172.21.15.133:0/4280363315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2b830a0 0x55eba2bdf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa67fff640 1 -- 172.21.15.133:0/4280363315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eba29718e0 con 0x55eba2bcc9d0 2024-04-25T10:09:40.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa67fff640 1 --2- 172.21.15.133:0/4280363315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bdf8c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7faa5800c520 tx=0x7faa580835a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76fd9a122b6d01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa477fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa477fe640 1 -- 172.21.15.133:0/4280363315 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58080070 con 0x55eba2bcc9d0 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa477fe640 1 -- 172.21.15.133:0/4280363315 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faa58083750 con 0x55eba2bcc9d0 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 -- 172.21.15.133:0/4280363315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa5c0b1040 con 0x55eba2bcc9d0 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa477fe640 1 -- 172.21.15.133:0/4280363315 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa58021040 con 0x55eba2bcc9d0 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 10 client.7647 fetch_fsmap learned FSMap version 50 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 -- 172.21.15.133:0/4280363315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55eba2a298f0 con 0x55eba2bcc9d0 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa477fe640 1 -- 172.21.15.133:0/4280363315 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c0b1040 con 0x55eba2bcc9d0 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa477fe640 1 -- 172.21.15.133:0/4280363315 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faa58052210 con 0x55eba2bcc9d0 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 10 client.7647 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 10 client.7647 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 20 client.7647 populate_metadata read hostname 'smithi133' 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 --2- 172.21.15.133:0/4280363315 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55eba2be0920 0x55eba2c34f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 4 client.7647 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 -- 172.21.15.133:0/4280363315 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eba29dd530 con 0x55eba2be0920 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/4280363315 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55eba2be0920 0x55eba2c34f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa477fe640 10 client.7647 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa6cb1d640 1 --2- 172.21.15.133:0/4280363315 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55eba2be0920 0x55eba2c34f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b347bb6387c9100 server_cookie=621c75cd6c83a123 in_seq=0 out_seq=0 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa477fe640 1 -- 172.21.15.133:0/4280363315 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55eba29dd530 con 0x55eba2be0920 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa477fe640 10 client.7647 handle_command_reply: tid=0 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 client.7647 shutdown 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 -- 172.21.15.133:0/4280363315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 msgr2=0x55eba2bdf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.971+0000 7faa71eee000 1 --2- 172.21.15.133:0/4280363315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bdf8c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7faa5800c520 tx=0x7faa580835a0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa71eee000 1 -- 172.21.15.133:0/4280363315 shutdown_connections 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa71eee000 1 --2- 172.21.15.133:0/4280363315 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x55eba2be0920 0x55eba2c34f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa71eee000 1 --2- 172.21.15.133:0/4280363315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55eba2bcd2f0 0x55eba2bdfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa71eee000 1 --2- 172.21.15.133:0/4280363315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eba2bcc9d0 0x55eba2bdf8c0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa71eee000 1 --2- 172.21.15.133:0/4280363315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55eba2b830a0 0x55eba2bdf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa477fe640 0 client.7647 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:09:40.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa71eee000 1 -- 172.21.15.133:0/4280363315 >> 172.21.15.133:0/4280363315 conn(0x55eba2b7f020 msgr2=0x55eba2bcb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa71eee000 1 -- 172.21.15.133:0/4280363315 shutdown_connections 2024-04-25T10:09:40.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa71eee000 1 -- 172.21.15.133:0/4280363315 wait complete. 2024-04-25T10:09:40.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa71eee000 20 client.7647 trim_cache size 0 max 16384 2024-04-25T10:09:40.985 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:09:40.985 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1349 inodes in the stack)", 2024-04-25T10:09:40.985 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-04-25T10:09:40.986 INFO:teuthology.orchestra.run.smithi133.stdout: "d9de1b73-a0d0-4c6a-845d-c91679388f6d": { 2024-04-25T10:09:40.986 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "~mds0", 2024-04-25T10:09:40.986 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "d9de1b73-a0d0-4c6a-845d-c91679388f6d", 2024-04-25T10:09:40.986 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive" 2024-04-25T10:09:40.986 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:09:40.986 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:09:40.986 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:09:40.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 -- 172.21.15.133:0/854485067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa54066f60 msgr2=0x7faa54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa54066f60 0x7faa54069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0ad000 tx=0x7faa5c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:40.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 -- 172.21.15.133:0/854485067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa6815b8b0 msgr2=0x7faa68154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa6815b8b0 0x7faa68154430 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7faa5800e8a0 tx=0x7faa5800eca0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 -- 172.21.15.133:0/854485067 shutdown_connections 2024-04-25T10:09:40.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa6815bc90 0x7faa680b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa54066f60 0x7faa54069420 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa6815b8b0 0x7faa68154430 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 --2- 172.21.15.133:0/854485067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa680d01d0 0x7faa6814f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 -- 172.21.15.133:0/854485067 >> 172.21.15.133:0/854485067 conn(0x7faa6809a990 msgr2=0x7faa680b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 -- 172.21.15.133:0/854485067 shutdown_connections 2024-04-25T10:09:40.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:09:40.975+0000 7faa6ed81640 1 -- 172.21.15.133:0/854485067 wait complete. 2024-04-25T10:09:40.999 INFO:tasks.cephfs.filesystem:scrub status for tag:d9de1b73-a0d0-4c6a-845d-c91679388f6d - {'path': '~mds0', 'tag': 'd9de1b73-a0d0-4c6a-845d-c91679388f6d', 'options': 'recursive'} 2024-04-25T10:09:52.143 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:52.146 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:11.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:11.001 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:10:11.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.131+0000 7f0e72193640 1 Processor -- start 2024-04-25T10:10:11.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.131+0000 7f0e72193640 1 -- start start 2024-04-25T10:10:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.131+0000 7f0e72193640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e6c15d5e0 0x7f0e6c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.131+0000 7f0e72193640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6c0ce2e0 con 0x7f0e6c0cd820 2024-04-25T10:10:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.131+0000 7f0e72193640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6c06d4b0 con 0x7f0e6c145170 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.131+0000 7f0e72193640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c06db00 con 0x7f0e6c15d5e0 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e62ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0e6c0cd820 0x7f0e6c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41356/0 (socket says 172.21.15.133:41356) 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e62ffd640 1 -- 172.21.15.133:0/1328252490 learned_addr learned my addr 172.21.15.133:0/1328252490 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6bfff640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e6c15d5e0 0x7f0e6c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610332944 0 0) 0x7f0e6c0ce2e0 con 0x7f0e6c0cd820 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e58003530 con 0x7f0e6c0cd820 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2934872424 0 0) 0x7f0e58003530 con 0x7f0e6c0cd820 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e6c15d5e0 msgr2=0x7f0e6c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e6c15d5e0 0x7f0e6c0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 >> v1:172.21.15.162:6789/0 conn(0x7f0e6c145170 legacy=0x7f0e6c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c06d790 con 0x7f0e6c0cd820 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0e54001680 con 0x7f0e6c0cd820 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 >> v1:172.21.15.133:6789/0 conn(0x7f0e6c0cd820 legacy=0x7f0e6c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e580039f0 0x7f0e58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e580043a0 0x7f0e58004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e58005070 0x7f0e5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c06db00 con 0x7f0e580043a0 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5800bf10 con 0x7f0e580039f0 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5800c090 con 0x7f0e58005070 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e580043a0 0x7f0e58004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6bfff640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e58005070 0x7f0e5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e580039f0 0x7f0e58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e62ffd640 1 -- 172.21.15.133:0/1328252490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e58005070 msgr2=0x7f0e5800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e58005070 0x7f0e5800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e62ffd640 1 -- 172.21.15.133:0/1328252490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e580039f0 msgr2=0x7f0e58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e580039f0 0x7f0e58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.135+0000 7f0e62ffd640 1 -- 172.21.15.133:0/1328252490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e58003530 con 0x7f0e580043a0 2024-04-25T10:10:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e580043a0 0x7f0e58004870 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c2c0 tx=0x7f0e5401cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d662b4a37afb4e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54007d60 con 0x7f0e580043a0 2024-04-25T10:10:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e540a9070 con 0x7f0e580043a0 2024-04-25T10:10:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e6affd640 1 -- 172.21.15.133:0/1328252490 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540a5df0 con 0x7f0e580043a0 2024-04-25T10:10:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 -- 172.21.15.133:0/1328252490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e580043a0 msgr2=0x7f0e58004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 --2- 172.21.15.133:0/1328252490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e580043a0 0x7f0e58004870 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c2c0 tx=0x7f0e5401cd20 comp rx=0 tx=0).stop 2024-04-25T10:10:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e6bfff640 1 -- 172.21.15.133:0/1328252490 reap_dead start 2024-04-25T10:10:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 -- 172.21.15.133:0/1328252490 shutdown_connections 2024-04-25T10:10:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 -- 172.21.15.133:0/1328252490 >> 172.21.15.133:0/1328252490 conn(0x7f0e6c066f70 msgr2=0x7f0e6c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 -- 172.21.15.133:0/1328252490 shutdown_connections 2024-04-25T10:10:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 -- 172.21.15.133:0/1328252490 wait complete. 2024-04-25T10:10:11.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 Processor -- start 2024-04-25T10:10:11.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 -- start start 2024-04-25T10:10:11.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e580043a0 0x7f0e6c1f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c13bd90 0x7f0e6c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e6c13c210 0x7f0e6c1f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c15cb70 con 0x7f0e6c13bd90 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c145550 con 0x7f0e6c13c210 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e72193640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c0cdca0 con 0x7f0e580043a0 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e62ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c13bd90 0x7f0e6c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e62ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c13bd90 0x7f0e6c15d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34096/0 (socket says 172.21.15.133:34096) 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e62ffd640 1 -- 172.21.15.133:0/2328582319 learned_addr learned my addr 172.21.15.133:0/2328582319 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e580043a0 0x7f0e6c1f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e6bfff640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e6c13c210 0x7f0e6c1f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e62ffd640 1 -- 172.21.15.133:0/2328582319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e580043a0 msgr2=0x7f0e6c1f0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e580043a0 0x7f0e6c1f0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e62ffd640 1 -- 172.21.15.133:0/2328582319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e6c13c210 msgr2=0x7f0e6c1f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e6c13c210 0x7f0e6c1f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.139+0000 7f0e62ffd640 1 -- 172.21.15.133:0/2328582319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c15e200 con 0x7f0e6c13bd90 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c13bd90 0x7f0e6c15d700 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c2c0 tx=0x7f0e58005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e68ff9640 1 -- 172.21.15.133:0/2328582319 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540a5ba0 con 0x7f0e6c13bd90 2024-04-25T10:10:11.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e68ff9640 1 -- 172.21.15.133:0/2328582319 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e540a5d40 con 0x7f0e6c13bd90 2024-04-25T10:10:11.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e68ff9640 1 -- 172.21.15.133:0/2328582319 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540049f0 con 0x7f0e6c13bd90 2024-04-25T10:10:11.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e72193640 1 -- 172.21.15.133:0/2328582319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c13cc80 con 0x7f0e6c13bd90 2024-04-25T10:10:11.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e72193640 1 -- 172.21.15.133:0/2328582319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c13d240 con 0x7f0e6c13bd90 2024-04-25T10:10:11.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e68ff9640 1 -- 172.21.15.133:0/2328582319 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e540b7030 con 0x7f0e6c13bd90 2024-04-25T10:10:11.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e68ff9640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e4c066f60 0x7f0e4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e68ff9640 1 -- 172.21.15.133:0/2328582319 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f0e5411e650 con 0x7f0e6c13bd90 2024-04-25T10:10:11.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e4c066f60 0x7f0e4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e4c066f60 0x7f0e4c069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00afd0 tx=0x7f0e5c00abb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e75300000 1 Processor -- start 2024-04-25T10:10:11.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.143+0000 7f0e75300000 1 -- start start 2024-04-25T10:10:11.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd959d0 0x562d9bd95db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bd4bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd4c0a0 0x562d9bd4c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d9bb98ff0 con 0x562d9bd962f0 2024-04-25T10:10:11.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d9bb21150 con 0x562d9bd959d0 2024-04-25T10:10:11.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d9bb95090 con 0x562d9bd4c0a0 2024-04-25T10:10:11.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e62ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bd4bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e62ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bd4bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34100/0 (socket says 172.21.15.133:34100) 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e62ffd640 1 -- 172.21.15.133:0/1794210313 learned_addr learned my addr 172.21.15.133:0/1794210313 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6bfff640 1 --2- 172.21.15.133:0/1794210313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd4c0a0 0x562d9bd4c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/1794210313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd959d0 0x562d9bd95db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e62ffd640 1 -- 172.21.15.133:0/1794210313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd4c0a0 msgr2=0x562d9bd4c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1794210313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd4c0a0 0x562d9bd4c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e62ffd640 1 -- 172.21.15.133:0/1794210313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd959d0 msgr2=0x562d9bd95db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1794210313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd959d0 0x562d9bd95db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e62ffd640 1 -- 172.21.15.133:0/1794210313 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d9bd2f0d0 con 0x562d9bd962f0 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1794210313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bd4bb60 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0e540eb230 tx=0x7f0e540eef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf11f1ba69e52097 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e617fa640 1 -- 172.21.15.133:0/1794210313 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540be070 con 0x562d9bd962f0 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e617fa640 1 -- 172.21.15.133:0/1794210313 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e540a9040 con 0x562d9bd962f0 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e617fa640 1 -- 172.21.15.133:0/1794210313 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540ed0d0 con 0x562d9bd962f0 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- 172.21.15.133:0/1794210313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 msgr2=0x562d9bd4bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 --2- 172.21.15.133:0/1794210313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bd4bb60 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0e540eb230 tx=0x7f0e540eef20 comp rx=0 tx=0).stop 2024-04-25T10:10:11.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- 172.21.15.133:0/1794210313 shutdown_connections 2024-04-25T10:10:11.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 --2- 172.21.15.133:0/1794210313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd4c0a0 0x562d9bd4c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 --2- 172.21.15.133:0/1794210313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bd4bb60 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 --2- 172.21.15.133:0/1794210313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd959d0 0x562d9bd95db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- 172.21.15.133:0/1794210313 >> 172.21.15.133:0/1794210313 conn(0x562d9bd48020 msgr2=0x562d9bd4a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- 172.21.15.133:0/1794210313 shutdown_connections 2024-04-25T10:10:11.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- 172.21.15.133:0/1794210313 wait complete. 2024-04-25T10:10:11.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- start start 2024-04-25T10:10:11.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 Processor -- start 2024-04-25T10:10:11.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd4c0a0 0x562d9bd7a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd959d0 0x562d9bd7ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd962f0 0x562d9bd7b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d9bb5d250 con 0x562d9bd4c0a0 2024-04-25T10:10:11.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d9bb53690 con 0x562d9bd959d0 2024-04-25T10:10:11.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d9bb21a40 con 0x562d9bd962f0 2024-04-25T10:10:11.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd4c0a0 0x562d9bd7a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e75300000 10 client.0 _ll_register_callbacks cb 0x562d9bd36e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:11.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd4c0a0 0x562d9bd7a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34114/0 (socket says 172.21.15.133:34114) 2024-04-25T10:10:11.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6b7fe640 1 -- 172.21.15.133:0/1851109008 learned_addr learned my addr 172.21.15.133:0/1851109008 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6bfff640 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd962f0 0x562d9bd7b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd959d0 0x562d9bd7ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6b7fe640 1 -- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd962f0 msgr2=0x562d9bd7b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd962f0 0x562d9bd7b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6b7fe640 1 -- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd959d0 msgr2=0x562d9bd7ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd959d0 0x562d9bd7ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6b7fe640 1 -- 172.21.15.133:0/1851109008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d9bc461c0 con 0x562d9bd4c0a0 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd4c0a0 0x562d9bd7a7f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c015cb0 tx=0x7f0e5c001240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee488d51814d549d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.147+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1851109008 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c027020 con 0x562d9bd4c0a0 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1851109008 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f0e5c023480 con 0x562d9bd4c0a0 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 -- 172.21.15.133:0/1851109008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e540a9040 con 0x562d9bd4c0a0 2024-04-25T10:10:11.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1851109008 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e5c031050 con 0x562d9bd4c0a0 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 10 client.7659 fetch_fsmap learned FSMap version 50 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 -- 172.21.15.133:0/1851109008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x562d9bc42170 con 0x562d9bd4c0a0 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1851109008 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e540a9040 con 0x562d9bd4c0a0 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1851109008 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0e5c034020 con 0x562d9bd4c0a0 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 10 client.7659 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 10 client.7659 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 20 client.7659 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562d9bda5c80 0x562d9bda8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 4 client.7659 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 -- 172.21.15.133:0/1851109008 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d9bb34c60 con 0x562d9bda5c80 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562d9bda5c80 0x562d9bda8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e4a7fc640 10 client.7659 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562d9bda5c80 0x562d9bda8070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=142f0b756876c6d2 server_cookie=5abde869e8d4d051 in_seq=0 out_seq=0 2024-04-25T10:10:11.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1851109008 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d9bb34c60 con 0x562d9bda5c80 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e4a7fc640 10 client.7659 handle_command_reply: tid=0 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 client.7659 shutdown 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 -- 172.21.15.133:0/1851109008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd4c0a0 msgr2=0x562d9bd7a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd4c0a0 0x562d9bd7a7f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c015cb0 tx=0x7f0e5c001240 comp rx=0 tx=0).stop 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 -- 172.21.15.133:0/1851109008 shutdown_connections 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562d9bda5c80 0x562d9bda8070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd962f0 0x562d9bd7b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd959d0 0x562d9bd7ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e4a7fc640 0 client.7659 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 --2- 172.21.15.133:0/1851109008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd4c0a0 0x562d9bd7a7f0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 -- 172.21.15.133:0/1851109008 >> 172.21.15.133:0/1851109008 conn(0x562d9bd48020 msgr2=0x562d9bd4aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 -- 172.21.15.133:0/1851109008 shutdown_connections 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 1 -- 172.21.15.133:0/1851109008 wait complete. 2024-04-25T10:10:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.151+0000 7f0e75300000 20 client.7659 trim_cache size 0 max 16384 2024-04-25T10:10:11.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.167+0000 7f0e75300000 1 Processor -- start 2024-04-25T10:10:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.167+0000 7f0e75300000 1 -- start start 2024-04-25T10:10:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.167+0000 7f0e75300000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd4c0a0 0x562d9bdfab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.167+0000 7f0e75300000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 0x562d9bdfb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.167+0000 7f0e75300000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bdfb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.167+0000 7f0e75300000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1fe210 con 0x562d9bd962f0 2024-04-25T10:10:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.167+0000 7f0e75300000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1fd410 con 0x562d9bd4c0a0 2024-04-25T10:10:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.167+0000 7f0e75300000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1fc870 con 0x562d9bd959d0 2024-04-25T10:10:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.167+0000 7f0e6bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bdfb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.167+0000 7f0e6bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bdfb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34130/0 (socket says 172.21.15.133:34130) 2024-04-25T10:10:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.167+0000 7f0e6bfff640 1 -- 172.21.15.133:0/1527729066 learned_addr learned my addr 172.21.15.133:0/1527729066 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/1527729066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd4c0a0 0x562d9bdfab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1527729066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 0x562d9bdfb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 -- 172.21.15.133:0/1527729066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 msgr2=0x562d9bdfb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 --2- 172.21.15.133:0/1527729066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 0x562d9bdfb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 -- 172.21.15.133:0/1527729066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd4c0a0 msgr2=0x562d9bdfab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 --2- 172.21.15.133:0/1527729066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd4c0a0 0x562d9bdfab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 -- 172.21.15.133:0/1527729066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d9bbdfde0 con 0x562d9bd962f0 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1527729066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 0x562d9bdfb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 --2- 172.21.15.133:0/1527729066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bdfb590 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0b2810 tx=0x7f0e64009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e41ea251a17bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e60ff9640 1 -- 172.21.15.133:0/1527729066 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6401c070 con 0x562d9bd962f0 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e60ff9640 1 -- 172.21.15.133:0/1527729066 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e64018450 con 0x562d9bd962f0 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e60ff9640 1 -- 172.21.15.133:0/1527729066 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64018800 con 0x562d9bd962f0 2024-04-25T10:10:11.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 -- 172.21.15.133:0/1527729066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 msgr2=0x562d9bdfb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 --2- 172.21.15.133:0/1527729066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bdfb590 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0b2810 tx=0x7f0e64009750 comp rx=0 tx=0).stop 2024-04-25T10:10:11.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 -- 172.21.15.133:0/1527729066 shutdown_connections 2024-04-25T10:10:11.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 --2- 172.21.15.133:0/1527729066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bdfb590 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 --2- 172.21.15.133:0/1527729066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 0x562d9bdfb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 --2- 172.21.15.133:0/1527729066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd4c0a0 0x562d9bdfab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 -- 172.21.15.133:0/1527729066 >> 172.21.15.133:0/1527729066 conn(0x562d9bd48020 msgr2=0x562d9bd940d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 -- 172.21.15.133:0/1527729066 shutdown_connections 2024-04-25T10:10:11.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 -- 172.21.15.133:0/1527729066 wait complete. 2024-04-25T10:10:11.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 -- start start 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 Processor -- start 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd4c0a0 0x562d9bda8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 0x562d9bda8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bda91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d9bb33e00 con 0x562d9bd962f0 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1fce40 con 0x562d9bd4c0a0 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d9bb98300 con 0x562d9bd959d0 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 10 client.0 _ll_register_callbacks cb 0x562d9bd17040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bda91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bda91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34142/0 (socket says 172.21.15.133:34142) 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 -- 172.21.15.133:0/1257553464 learned_addr learned my addr 172.21.15.133:0/1257553464 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd4c0a0 0x562d9bda8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 0x562d9bda8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 -- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 msgr2=0x562d9bda8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 0x562d9bda8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 -- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd4c0a0 msgr2=0x562d9bda8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd4c0a0 0x562d9bda8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 -- 172.21.15.133:0/1257553464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d9bb3a8e0 con 0x562d9bd962f0 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e62ffd640 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 0x562d9bda8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e6bfff640 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bda91f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x562d9bc36af0 tx=0x7f0e64004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e47101ebdcaf715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1257553464 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6401c070 con 0x562d9bd962f0 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1257553464 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f0e64021860 con 0x562d9bd962f0 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 -- 172.21.15.133:0/1257553464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e64018450 con 0x562d9bd962f0 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1257553464 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e64032050 con 0x562d9bd962f0 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 10 client.7665 fetch_fsmap learned FSMap version 50 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.171+0000 7f0e75300000 1 -- 172.21.15.133:0/1257553464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x562d9bbf28f0 con 0x562d9bd962f0 2024-04-25T10:10:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1257553464 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e64018450 con 0x562d9bd962f0 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1257553464 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0e64035020 con 0x562d9bd962f0 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 10 client.7665 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 10 client.7665 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 20 client.7665 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562d9be04760 0x562d9be06b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 4 client.7665 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 -- 172.21.15.133:0/1257553464 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562d9bba6530 con 0x562d9be04760 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562d9be04760 0x562d9be06b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e6b7fe640 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562d9be04760 0x562d9be06b50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467b0fa6d8960885 server_cookie=2ce737e74c7250c7 in_seq=0 out_seq=0 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e4a7fc640 10 client.7665 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e4a7fc640 1 -- 172.21.15.133:0/1257553464 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562d9bba6530 con 0x562d9be04760 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e4a7fc640 10 client.7665 handle_command_reply: tid=0 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 client.7665 shutdown 2024-04-25T10:10:11.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 -- 172.21.15.133:0/1257553464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 msgr2=0x562d9bda91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bda91f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x562d9bc36af0 tx=0x7f0e64004a50 comp rx=0 tx=0).stop 2024-04-25T10:10:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 -- 172.21.15.133:0/1257553464 shutdown_connections 2024-04-25T10:10:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x562d9be04760 0x562d9be06b50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d9bd962f0 0x562d9bda91f0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562d9bd959d0 0x562d9bda8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 --2- 172.21.15.133:0/1257553464 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562d9bd4c0a0 0x562d9bda8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e4a7fc640 0 client.7665 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:10:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 -- 172.21.15.133:0/1257553464 >> 172.21.15.133:0/1257553464 conn(0x562d9bd48020 msgr2=0x562d9bd94960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 -- 172.21.15.133:0/1257553464 shutdown_connections 2024-04-25T10:10:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 1 -- 172.21.15.133:0/1257553464 wait complete. 2024-04-25T10:10:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e75300000 20 client.7665 trim_cache size 0 max 16384 2024-04-25T10:10:11.188 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:10:11.188 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-04-25T10:10:11.188 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:10:11.188 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:10:11.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 -- 172.21.15.133:0/2328582319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e4c066f60 msgr2=0x7f0e4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e4c066f60 0x7f0e4c069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00afd0 tx=0x7f0e5c00abb0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 -- 172.21.15.133:0/2328582319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c13bd90 msgr2=0x7f0e6c15d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c13bd90 0x7f0e6c15d700 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c2c0 tx=0x7f0e58005840 comp rx=0 tx=0).stop 2024-04-25T10:10:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 -- 172.21.15.133:0/2328582319 shutdown_connections 2024-04-25T10:10:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e6c13c210 0x7f0e6c1f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e4c066f60 0x7f0e4c069420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c13bd90 0x7f0e6c15d700 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 --2- 172.21.15.133:0/2328582319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e580043a0 0x7f0e6c1f0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 -- 172.21.15.133:0/2328582319 >> 172.21.15.133:0/2328582319 conn(0x7f0e6c066f70 msgr2=0x7f0e6c141cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 -- 172.21.15.133:0/2328582319 shutdown_connections 2024-04-25T10:10:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.175+0000 7f0e72193640 1 -- 172.21.15.133:0/2328582319 wait complete. 2024-04-25T10:10:11.198 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:10:11.199 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:10:11.199 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T10:10:11.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.303+0000 7f67ca7ad640 1 Processor -- start 2024-04-25T10:10:11.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.303+0000 7f67ca7ad640 1 -- start start 2024-04-25T10:10:11.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67ca7ad640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67c4145170 0x7f67c406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67ca7ad640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f67c415ced0 0x7f67c415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67ca7ad640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c40ce2e0 con 0x7f67c406e4d0 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67ca7ad640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c406db00 con 0x7f67c4145170 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67ca7ad640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c4145550 con 0x7f67c415ced0 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67c4145170 0x7f67c406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67bb7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f67c406e4d0 0x7f67c40cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41368/0 (socket says 172.21.15.133:41368) 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67c4145170 0x7f67c406df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34146/0 (socket says 172.21.15.133:34146) 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67bb7fe640 1 -- 172.21.15.133:0/2821045555 learned_addr learned my addr 172.21.15.133:0/2821045555 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c8d4a640 1 --2- 172.21.15.133:0/2821045555 >> v2:172.21.15.162:3300/0 conn(0x7f67c415ced0 0x7f67c415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c3fff640 1 -- 172.21.15.133:0/2821045555 >> v2:172.21.15.162:3300/0 conn(0x7f67c415ced0 msgr2=0x7f67c415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c3fff640 1 --2- 172.21.15.133:0/2821045555 >> v2:172.21.15.162:3300/0 conn(0x7f67c415ced0 0x7f67c415d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c3fff640 1 -- 172.21.15.133:0/2821045555 >> v1:172.21.15.133:6789/0 conn(0x7f67c406e4d0 legacy=0x7f67c40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c3fff640 1 -- 172.21.15.133:0/2821045555 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c406d4b0 con 0x7f67c4145170 2024-04-25T10:10:11.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c37fe640 1 -- 172.21.15.133:0/2821045555 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183654476 0 0) 0x7f67c40ce2e0 con 0x7f67c406e4d0 2024-04-25T10:10:11.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c3fff640 1 --2- 172.21.15.133:0/2821045555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c4145170 0x7f67c406df90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f67c40b27f0 tx=0x7f67b401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7d7de967b8464b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c37fe640 1 -- 172.21.15.133:0/2821045555 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40b0070 con 0x7f67c4145170 2024-04-25T10:10:11.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c37fe640 1 -- 172.21.15.133:0/2821045555 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b40ab600 con 0x7f67c4145170 2024-04-25T10:10:11.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.307+0000 7f67c37fe640 1 -- 172.21.15.133:0/2821045555 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40ab9d0 con 0x7f67c4145170 2024-04-25T10:10:11.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 -- 172.21.15.133:0/2821045555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c4145170 msgr2=0x7f67c406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 --2- 172.21.15.133:0/2821045555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c4145170 0x7f67c406df90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f67c40b27f0 tx=0x7f67b401c480 comp rx=0 tx=0).stop 2024-04-25T10:10:11.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 -- 172.21.15.133:0/2821045555 shutdown_connections 2024-04-25T10:10:11.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 --2- 172.21.15.133:0/2821045555 >> v2:172.21.15.162:3300/0 conn(0x7f67c415ced0 0x7f67c415d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 --2- 172.21.15.133:0/2821045555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c4145170 0x7f67c406df90 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 -- 172.21.15.133:0/2821045555 >> 172.21.15.133:0/2821045555 conn(0x7f67c4066f70 msgr2=0x7f67c406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 -- 172.21.15.133:0/2821045555 shutdown_connections 2024-04-25T10:10:11.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 -- 172.21.15.133:0/2821045555 wait complete. 2024-04-25T10:10:11.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 Processor -- start 2024-04-25T10:10:11.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 -- start start 2024-04-25T10:10:11.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c406e4d0 0x7f67c4139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c4145170 0x7f67c41525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67c415ced0 0x7f67c4151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c4168610 con 0x7f67c406e4d0 2024-04-25T10:10:11.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c4168490 con 0x7f67c4145170 2024-04-25T10:10:11.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c4168790 con 0x7f67c415ced0 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c406e4d0 0x7f67c4139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c406e4d0 0x7f67c4139e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34158/0 (socket says 172.21.15.133:34158) 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c3fff640 1 -- 172.21.15.133:0/1004590183 learned_addr learned my addr 172.21.15.133:0/1004590183 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c8d4a640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67c415ced0 0x7f67c4151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c4145170 0x7f67c41525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c3fff640 1 -- 172.21.15.133:0/1004590183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67c415ced0 msgr2=0x7f67c4151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c3fff640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67c415ced0 0x7f67c4151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c3fff640 1 -- 172.21.15.133:0/1004590183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c4145170 msgr2=0x7f67c41525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c3fff640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c4145170 0x7f67c41525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c3fff640 1 -- 172.21.15.133:0/1004590183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c4152b30 con 0x7f67c406e4d0 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c3fff640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c406e4d0 0x7f67c4139e80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f67b40ae040 tx=0x7f67b4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c17fa640 1 -- 172.21.15.133:0/1004590183 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40b8020 con 0x7f67c406e4d0 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c17fa640 1 -- 172.21.15.133:0/1004590183 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b40b4410 con 0x7f67c406e4d0 2024-04-25T10:10:11.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 -- 172.21.15.133:0/1004590183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c413a460 con 0x7f67c406e4d0 2024-04-25T10:10:11.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67c17fa640 1 -- 172.21.15.133:0/1004590183 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40b0040 con 0x7f67c406e4d0 2024-04-25T10:10:11.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.311+0000 7f67ca7ad640 1 -- 172.21.15.133:0/1004590183 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c413aa20 con 0x7f67c406e4d0 2024-04-25T10:10:11.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c17fa640 1 -- 172.21.15.133:0/1004590183 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67b40bb020 con 0x7f67c406e4d0 2024-04-25T10:10:11.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c17fa640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67a4066f60 0x7f67a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c17fa640 1 -- 172.21.15.133:0/1004590183 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f67b411ee50 con 0x7f67c406e4d0 2024-04-25T10:10:11.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67a4066f60 0x7f67a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67a4066f60 0x7f67a4069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f67c415f870 tx=0x7f67ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67cd91a000 1 Processor -- start 2024-04-25T10:10:11.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67cd91a000 1 -- start start 2024-04-25T10:10:11.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d02ff5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d02fb0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02fb0d90 0x558d02fb1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67cd91a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d02e931f0 con 0x558d02ff5a40 2024-04-25T10:10:11.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67cd91a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d02df9e90 con 0x558d02fb0d90 2024-04-25T10:10:11.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67cd91a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d02d81f50 con 0x558d02ff6360 2024-04-25T10:10:11.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d02ff5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d02ff5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34160/0 (socket says 172.21.15.133:34160) 2024-04-25T10:10:11.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c3fff640 1 -- 172.21.15.133:0/1124300346 learned_addr learned my addr 172.21.15.133:0/1124300346 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/1124300346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d02fb0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c3fff640 1 -- 172.21.15.133:0/1124300346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 msgr2=0x558d02fb0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c3fff640 1 --2- 172.21.15.133:0/1124300346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d02fb0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c3fff640 1 -- 172.21.15.133:0/1124300346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02fb0d90 msgr2=0x558d02fb1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c8d4a640 1 --2- 172.21.15.133:0/1124300346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02fb0d90 0x558d02fb1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c3fff640 1 --2- 172.21.15.133:0/1124300346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02fb0d90 0x558d02fb1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c3fff640 1 -- 172.21.15.133:0/1124300346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d02dd18b0 con 0x558d02ff5a40 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c8d4a640 1 --2- 172.21.15.133:0/1124300346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02fb0d90 0x558d02fb1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67c3fff640 1 --2- 172.21.15.133:0/1124300346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d02ff5e20 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f67b40f7820 tx=0x7f67b40ef540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a577df88ac0c2de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67b9ffb640 1 -- 172.21.15.133:0/1124300346 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40f7d50 con 0x558d02ff5a40 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67b9ffb640 1 -- 172.21.15.133:0/1124300346 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b4107120 con 0x558d02ff5a40 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.315+0000 7f67b9ffb640 1 -- 172.21.15.133:0/1124300346 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40ebcb0 con 0x558d02ff5a40 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- 172.21.15.133:0/1124300346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 msgr2=0x558d02ff5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1124300346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d02ff5e20 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f67b40f7820 tx=0x7f67b40ef540 comp rx=0 tx=0).stop 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- 172.21.15.133:0/1124300346 shutdown_connections 2024-04-25T10:10:11.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1124300346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02fb0d90 0x558d02fb1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1124300346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d02fb0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1124300346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d02ff5e20 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- 172.21.15.133:0/1124300346 >> 172.21.15.133:0/1124300346 conn(0x558d02facd40 msgr2=0x558d02faf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- 172.21.15.133:0/1124300346 shutdown_connections 2024-04-25T10:10:11.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- 172.21.15.133:0/1124300346 wait complete. 2024-04-25T10:10:11.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- start start 2024-04-25T10:10:11.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 Processor -- start 2024-04-25T10:10:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff5a40 0x558d02ffd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02ff6360 0x558d02fb6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d02db4540 con 0x558d02fb0d90 2024-04-25T10:10:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d02df5f30 con 0x558d02ff6360 2024-04-25T10:10:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d02da5260 con 0x558d02ff5a40 2024-04-25T10:10:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 10 client.0 _ll_register_callbacks cb 0x558d02eae180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34168/0 (socket says 172.21.15.133:34168) 2024-04-25T10:10:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67c3fff640 1 -- 172.21.15.133:0/1395188789 learned_addr learned my addr 172.21.15.133:0/1395188789 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff5a40 0x558d02ffd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67c8d4a640 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02ff6360 0x558d02fb6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67c3fff640 1 -- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff5a40 msgr2=0x558d02ffd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67c3fff640 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff5a40 0x558d02ffd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67c3fff640 1 -- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02ff6360 msgr2=0x558d02fb6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67c3fff640 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02ff6360 0x558d02fb6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67c3fff640 1 -- 172.21.15.133:0/1395188789 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d02d821a0 con 0x558d02fb0d90 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67c3fff640 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffcfb0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f67b4097d50 tx=0x7f67b4000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98a6015312e3659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67a2ffd640 1 -- 172.21.15.133:0/1395188789 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40f7ac0 con 0x558d02fb0d90 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67a2ffd640 1 -- 172.21.15.133:0/1395188789 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f67b4116bc0 con 0x558d02fb0d90 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- 172.21.15.133:0/1395188789 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67b4107120 con 0x558d02fb0d90 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67a2ffd640 1 -- 172.21.15.133:0/1395188789 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b40f7d50 con 0x558d02fb0d90 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 10 client.7680 fetch_fsmap learned FSMap version 50 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- 172.21.15.133:0/1395188789 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x558d02e7c840 con 0x558d02fb0d90 2024-04-25T10:10:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67a2ffd640 1 -- 172.21.15.133:0/1395188789 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b4107120 con 0x558d02fb0d90 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67a2ffd640 1 -- 172.21.15.133:0/1395188789 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67b40b3050 con 0x558d02fb0d90 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 10 client.7680 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 10 client.7680 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 20 client.7680 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x558d02fb7bc0 0x558d02fb7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 4 client.7680 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67cd91a000 1 -- 172.21.15.133:0/1395188789 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d02d95be0 con 0x558d02fb7bc0 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.319+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x558d02fb7bc0 0x558d02fb7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67a2ffd640 10 client.7680 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x558d02fb7bc0 0x558d02fb7fd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6cbcc1a6073c82 server_cookie=f2aff330b051e897 in_seq=0 out_seq=0 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67a2ffd640 1 -- 172.21.15.133:0/1395188789 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d02d95be0 con 0x558d02fb7bc0 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67a2ffd640 10 client.7680 handle_command_reply: tid=0 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 1 client.7680 shutdown 2024-04-25T10:10:11.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 1 -- 172.21.15.133:0/1395188789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 msgr2=0x558d02ffcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffcfb0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f67b4097d50 tx=0x7f67b4000bd0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 1 -- 172.21.15.133:0/1395188789 shutdown_connections 2024-04-25T10:10:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x558d02fb7bc0 0x558d02fb7fd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02ff6360 0x558d02fb6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff5a40 0x558d02ffd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67a2ffd640 0 client.7680 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:10:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1395188789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffcfb0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 1 -- 172.21.15.133:0/1395188789 >> 172.21.15.133:0/1395188789 conn(0x558d02facd40 msgr2=0x558d02faf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 1 -- 172.21.15.133:0/1395188789 shutdown_connections 2024-04-25T10:10:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 1 -- 172.21.15.133:0/1395188789 wait complete. 2024-04-25T10:10:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.323+0000 7f67cd91a000 20 client.7680 trim_cache size 0 max 16384 2024-04-25T10:10:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.335+0000 7f67cd91a000 1 Processor -- start 2024-04-25T10:10:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.335+0000 7f67cd91a000 1 -- start start 2024-04-25T10:10:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02fb0d90 0x558d030524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d030529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d03052f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c4200f10 con 0x558d02ff5a40 2024-04-25T10:10:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c4200110 con 0x558d02fb0d90 2024-04-25T10:10:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c4142ea0 con 0x558d02ff6360 2024-04-25T10:10:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67bb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d030529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67bb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d030529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34174/0 (socket says 172.21.15.133:34174) 2024-04-25T10:10:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67bb7fe640 1 -- 172.21.15.133:0/1363212593 learned_addr learned my addr 172.21.15.133:0/1363212593 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c3fff640 1 --2- 172.21.15.133:0/1363212593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02fb0d90 0x558d030524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67bb7fe640 1 -- 172.21.15.133:0/1363212593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 msgr2=0x558d03052f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c8d4a640 1 --2- 172.21.15.133:0/1363212593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d03052f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/1363212593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d03052f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67bb7fe640 1 -- 172.21.15.133:0/1363212593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02fb0d90 msgr2=0x558d030524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/1363212593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02fb0d90 0x558d030524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67bb7fe640 1 -- 172.21.15.133:0/1363212593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d02fb6d60 con 0x558d02ff5a40 2024-04-25T10:10:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/1363212593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d030529e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0071e0 tx=0x7f67ac016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bf2b30c65d9caf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67b97fa640 1 -- 172.21.15.133:0/1363212593 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac01d040 con 0x558d02ff5a40 2024-04-25T10:10:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67b97fa640 1 -- 172.21.15.133:0/1363212593 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac0047f0 con 0x558d02ff5a40 2024-04-25T10:10:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67b97fa640 1 -- 172.21.15.133:0/1363212593 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac004ba0 con 0x558d02ff5a40 2024-04-25T10:10:11.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- 172.21.15.133:0/1363212593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 msgr2=0x558d030529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1363212593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d030529e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0071e0 tx=0x7f67ac016040 comp rx=0 tx=0).stop 2024-04-25T10:10:11.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- 172.21.15.133:0/1363212593 shutdown_connections 2024-04-25T10:10:11.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1363212593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d03052f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1363212593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02ff5a40 0x558d030529e0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 --2- 172.21.15.133:0/1363212593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02fb0d90 0x558d030524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- 172.21.15.133:0/1363212593 >> 172.21.15.133:0/1363212593 conn(0x558d02facd40 msgr2=0x558d02ff41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- 172.21.15.133:0/1363212593 shutdown_connections 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- 172.21.15.133:0/1363212593 wait complete. 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- start start 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 Processor -- start 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02ff5a40 0x558d02ffb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d03013670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d02dc67c0 con 0x558d02fb0d90 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41434a0 con 0x558d02ff5a40 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d02df5f30 con 0x558d02ff6360 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67cd91a000 10 client.0 _ll_register_callbacks cb 0x558d02dbca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34188/0 (socket says 172.21.15.133:34188) 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c3fff640 1 -- 172.21.15.133:0/92086366 learned_addr learned my addr 172.21.15.133:0/92086366 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c8d4a640 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d03013670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c3fff640 1 -- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 msgr2=0x558d03013670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c3fff640 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d03013670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c3fff640 1 -- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02ff5a40 msgr2=0x558d02ffb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c3fff640 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02ff5a40 0x558d02ffb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c3fff640 1 -- 172.21.15.133:0/92086366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d02e53a10 con 0x558d02fb0d90 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c8d4a640 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d03013670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67c3fff640 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffaed0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f67b40f7f10 tx=0x7f67b40abf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670f03e5f9578471 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.339+0000 7f67a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67a2ffd640 1 -- 172.21.15.133:0/92086366 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4107120 con 0x558d02fb0d90 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67a2ffd640 1 -- 172.21.15.133:0/92086366 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f67b4116bc0 con 0x558d02fb0d90 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 -- 172.21.15.133:0/92086366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67ac0047f0 con 0x558d02fb0d90 2024-04-25T10:10:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67a2ffd640 1 -- 172.21.15.133:0/92086366 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b4116dc0 con 0x558d02fb0d90 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 10 client.7686 fetch_fsmap learned FSMap version 50 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 -- 172.21.15.133:0/92086366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x558d02d8eb00 con 0x558d02fb0d90 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67a2ffd640 1 -- 172.21.15.133:0/92086366 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac0047f0 con 0x558d02fb0d90 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67a2ffd640 1 -- 172.21.15.133:0/92086366 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67b40b3070 con 0x558d02fb0d90 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 10 client.7686 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 10 client.7686 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 20 client.7686 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x558d03014030 0x558d0305eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 4 client.7686 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 -- 172.21.15.133:0/92086366 --> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x558d02e073d0 con 0x558d03014030 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x558d03014030 0x558d0305eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67a2ffd640 10 client.7686 ms_handle_connect on v2:172.21.15.162:6834/3192312240 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67bb7fe640 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x558d03014030 0x558d0305eae0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e95092acac5c6cd2 server_cookie=8187a6715b644012 in_seq=0 out_seq=0 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67a2ffd640 1 -- 172.21.15.133:0/92086366 <== mds.0 v2:172.21.15.162:6834/3192312240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17957 (crc 0 0 0) 0x558d02e073d0 con 0x558d03014030 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67a2ffd640 10 client.7686 handle_command_reply: tid=0 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 client.7686 shutdown 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 -- 172.21.15.133:0/92086366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 msgr2=0x558d02ffaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffaed0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f67b40f7f10 tx=0x7f67b40abf90 comp rx=0 tx=0).stop 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 -- 172.21.15.133:0/92086366 shutdown_connections 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:6834/3192312240,v1:172.21.15.162:6835/3192312240] conn(0x558d03014030 0x558d0305eae0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558d02ff6360 0x558d03013670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558d02ff5a40 0x558d02ffb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67a2ffd640 0 client.7686 ms_handle_reset on v2:172.21.15.162:6834/3192312240 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 --2- 172.21.15.133:0/92086366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d02fb0d90 0x558d02ffaed0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 -- 172.21.15.133:0/92086366 >> 172.21.15.133:0/92086366 conn(0x558d02facd40 msgr2=0x558d02ff4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 -- 172.21.15.133:0/92086366 shutdown_connections 2024-04-25T10:10:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 1 -- 172.21.15.133:0/92086366 wait complete. 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67cd91a000 20 client.7686 trim_cache size 0 max 16384 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 4660, 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 4662, 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1564799, 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 15723380, 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 18, 2024-04-25T10:10:11.356 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.613148801, 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.251882835, 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.134369179, 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.180588423, 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4659, 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.783803373, 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000168234 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 10, 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000002007, 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000200 2024-04-25T10:10:11.357 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 35516, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 61868, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 12476715, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 54443386, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 9, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 6.585454529, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 3.145672293, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 3.766575807, 2024-04-25T10:10:11.358 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.206797863, 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 61867, 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3.143025159, 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000050802 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 87999, 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.032129155, 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000365 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 144656 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:10:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 5524, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 5517, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1474248, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 46231796, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 6, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.855400501, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.388818708, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.169109957, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.246260992, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 5516, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.787705209, 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000142803 2024-04-25T10:10:11.360 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 123600, 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 36464 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T10:10:11.361 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 11396, 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 35.060962611, 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003076602 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3612, 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.107921165, 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000583588 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.362 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 27418, 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 27418, 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 27418, 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 475.466309478, 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.017341392 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 12, 2024-04-25T10:10:11.363 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 17848, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 17848, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 17848, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 2, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 2, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 2, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 45591, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 45097, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-04-25T10:10:11.364 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 161, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 2011, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 0, 2024-04-25T10:10:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 0, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 2011, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 11, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 2000, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 6684, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 6530, 2024-04-25T10:10:11.366 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 6684, 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 65, 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 17921 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-04-25T10:10:11.367 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 4107, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 3733, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 2211, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-04-25T10:10:11.368 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 24942, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 24942, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 33, 2024-04-25T10:10:11.369 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 33, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 4, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 85081795, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7111, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 118.419232874, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.016652964 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 14, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 2015, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 2001, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 13, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 13, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 0, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 17848, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 19848, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 2000, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 2, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 2002, 2024-04-25T10:10:11.371 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 2000, 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 92880, 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215340 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 37037, 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 27418, 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 10, 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2000, 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3.184369499, 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001592184 2024-04-25T10:10:11.372 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1198, 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.968099858, 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000808096 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6000, 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 21.269913974, 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003544985 2024-04-25T10:10:11.373 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7996, 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 19.266075961, 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002409464 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.374 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3, 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.027118262, 2024-04-25T10:10:11.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.009039420 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.006347167, 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.006347167 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.376 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.006155719, 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.006155719 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6, 2024-04-25T10:10:11.377 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.270474408, 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.045079068 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2211, 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 393.259500290, 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.177864993 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-04-25T10:10:11.378 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.379 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.379 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.379 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.379 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-04-25T10:10:11.379 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.379 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.379 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.379 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.008205747, 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.008205747 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-04-25T10:10:11.381 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000558425, 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000558425 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.382 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.383 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8000, 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 37.199490168, 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004649936 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-04-25T10:10:11.384 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 1, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 0, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 1, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 13370, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 13370, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 148, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:10:11.385 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-25T10:10:11.386 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-25T10:10:11.387 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 251488, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 86, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-25T10:10:11.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 5528, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 78, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 13439508, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 661017, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-04-25T10:10:11.389 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 13406, 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 81452748, 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 37, 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 13406, 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 13406, 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 59.027472775, 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004403063 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 13406, 2024-04-25T10:10:11.390 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 13605 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 13406, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 2158, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 11248, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 0, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 11, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 146, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 7256, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 83, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 1733, 2024-04-25T10:10:11.391 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 2031, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 2012, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 22, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:10:11.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-04-25T10:10:11.393 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:10:11.393 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-04-25T10:10:11.394 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 83, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 3, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 8, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 8, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 253, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 0, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 58 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.395 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 32, 2024-04-25T10:10:11.396 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 3733, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 3733, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 45690, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 11395305, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 45690, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:10:11.397 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 45690, 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 11395305, 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T10:10:11.398 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 13406, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 81843645, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 13406, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 81843645, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-04-25T10:10:11.399 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 13406, 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 13406, 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 13406, 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 13406, 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:11.400 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 3733, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 390897, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 3733, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 145, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 390897, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.401 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55ef95d0a960": { 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 24942, 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 80887491, 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 24942, 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T10:10:11.402 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 7111, 2024-04-25T10:10:11.403 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 80887491, 2024-04-25T10:10:11.403 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T10:10:11.403 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T10:10:11.403 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T10:10:11.403 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T10:10:11.403 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:11.403 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:11.403 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:10:11.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 -- 172.21.15.133:0/1004590183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67a4066f60 msgr2=0x7f67a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67a4066f60 0x7f67a4069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f67c415f870 tx=0x7f67ac009040 comp rx=0 tx=0).stop 2024-04-25T10:10:11.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 -- 172.21.15.133:0/1004590183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c406e4d0 msgr2=0x7f67c4139e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c406e4d0 0x7f67c4139e80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f67b40ae040 tx=0x7f67b4019220 comp rx=0 tx=0).stop 2024-04-25T10:10:11.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 -- 172.21.15.133:0/1004590183 shutdown_connections 2024-04-25T10:10:11.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67c415ced0 0x7f67c4151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67c4145170 0x7f67c41525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67a4066f60 0x7f67a4069420 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 --2- 172.21.15.133:0/1004590183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c406e4d0 0x7f67c4139e80 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 -- 172.21.15.133:0/1004590183 >> 172.21.15.133:0/1004590183 conn(0x7f67c4066f70 msgr2=0x7f67c40b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 -- 172.21.15.133:0/1004590183 shutdown_connections 2024-04-25T10:10:11.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.343+0000 7f67ca7ad640 1 -- 172.21.15.133:0/1004590183 wait complete. 2024-04-25T10:10:11.407 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:10:11.407 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:11.441 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:11.493 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:10:11.493 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:10:11.494 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:11.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:11.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.663+0000 7f31ad7fd640 1 Processor -- start 2024-04-25T10:10:11.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.663+0000 7f31ad7fd640 1 -- start start 2024-04-25T10:10:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31ad7fd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31a815c9a0 0x7f31a80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31ad7fd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a80c61d0 con 0x7f31a80b1f70 2024-04-25T10:10:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31ad7fd640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a80c64b0 con 0x7f31a80ce330 2024-04-25T10:10:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31ad7fd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a806d4f0 con 0x7f31a815c9a0 2024-04-25T10:10:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f319ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f31a80b1f70 0x7f31a806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41370/0 (socket says 172.21.15.133:41370) 2024-04-25T10:10:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f319ffff640 1 -- 172.21.15.133:0/1474183630 learned_addr learned my addr 172.21.15.133:0/1474183630 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a6ffd640 1 --2- 172.21.15.133:0/1474183630 >> v2:172.21.15.162:3300/0 conn(0x7f31a815c9a0 0x7f31a80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020142901 0 0) 0x7f31a80c61d0 con 0x7f31a80b1f70 2024-04-25T10:10:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3194003530 con 0x7f31a80b1f70 2024-04-25T10:10:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 141932225 0 0) 0x7f3194003530 con 0x7f31a80b1f70 2024-04-25T10:10:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 >> v2:172.21.15.162:3300/0 conn(0x7f31a815c9a0 msgr2=0x7f31a80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 --2- 172.21.15.133:0/1474183630 >> v2:172.21.15.162:3300/0 conn(0x7f31a815c9a0 0x7f31a80c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 >> v1:172.21.15.162:6789/0 conn(0x7f31a80ce330 legacy=0x7f31a815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a806d7d0 con 0x7f31a80b1f70 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f3190001680 con 0x7f31a80b1f70 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 >> v1:172.21.15.133:6789/0 conn(0x7f31a80b1f70 legacy=0x7f31a806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940039c0 0x7f3194003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3194004810 0x7f3194004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31940051d0 0x7f319400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a806d4f0 con 0x7f3194004810 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319400c070 con 0x7f31940039c0 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319400c270 con 0x7f31940051d0 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f319ffff640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3194004810 0x7f3194004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a6ffd640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940039c0 0x7f3194003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f319ffff640 1 -- 172.21.15.133:0/1474183630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31940051d0 msgr2=0x7f319400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a77fe640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31940051d0 0x7f319400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f319ffff640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31940051d0 0x7f319400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f319ffff640 1 -- 172.21.15.133:0/1474183630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940039c0 msgr2=0x7f3194003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f319ffff640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940039c0 0x7f3194003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a77fe640 1 -- 172.21.15.133:0/1474183630 reap_dead start 2024-04-25T10:10:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f319ffff640 1 -- 172.21.15.133:0/1474183630 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3194003530 con 0x7f3194004810 2024-04-25T10:10:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.667+0000 7f31a6ffd640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31940039c0 0x7f3194003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f319ffff640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3194004810 0x7f3194004c90 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f319000c240 tx=0x7f319001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7482f588b184234 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190007d20 con 0x7f3194004810 2024-04-25T10:10:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3190005ae0 con 0x7f3194004810 2024-04-25T10:10:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31a67fc640 1 -- 172.21.15.133:0/1474183630 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190007070 con 0x7f3194004810 2024-04-25T10:10:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 -- 172.21.15.133:0/1474183630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3194004810 msgr2=0x7f3194004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3194004810 0x7f3194004c90 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f319000c240 tx=0x7f319001f3f0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 -- 172.21.15.133:0/1474183630 shutdown_connections 2024-04-25T10:10:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 --2- 172.21.15.133:0/1474183630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3194004810 0x7f3194004c90 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 -- 172.21.15.133:0/1474183630 >> 172.21.15.133:0/1474183630 conn(0x7f31a8067000 msgr2=0x7f31a806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 -- 172.21.15.133:0/1474183630 shutdown_connections 2024-04-25T10:10:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 -- 172.21.15.133:0/1474183630 wait complete. 2024-04-25T10:10:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 Processor -- start 2024-04-25T10:10:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 -- start start 2024-04-25T10:10:11.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3194004810 0x7f31a8142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31a80b0c20 0x7f31a80b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31a8152090 0x7f31a8152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80ce780 con 0x7f31a80b0c20 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80ce900 con 0x7f3194004810 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f31ad7fd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80cea80 con 0x7f31a8152090 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f319ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31a80b0c20 0x7f31a80b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f319ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31a80b0c20 0x7f31a80b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34212/0 (socket says 172.21.15.133:34212) 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.671+0000 7f319ffff640 1 -- 172.21.15.133:0/2453111688 learned_addr learned my addr 172.21.15.133:0/2453111688 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f31a6ffd640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3194004810 0x7f31a8142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319ffff640 1 -- 172.21.15.133:0/2453111688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31a8152090 msgr2=0x7f31a8152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f31a77fe640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31a8152090 0x7f31a8152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319ffff640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31a8152090 0x7f31a8152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319ffff640 1 -- 172.21.15.133:0/2453111688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3194004810 msgr2=0x7f31a8142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319ffff640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3194004810 0x7f31a8142020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319ffff640 1 -- 172.21.15.133:0/2453111688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a8142560 con 0x7f31a80b0c20 2024-04-25T10:10:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f31a6ffd640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3194004810 0x7f31a8142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319ffff640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31a80b0c20 0x7f31a80b10a0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f319001f220 tx=0x7f3194005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319f7fe640 1 -- 172.21.15.133:0/2453111688 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190007d20 con 0x7f31a80b0c20 2024-04-25T10:10:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319f7fe640 1 -- 172.21.15.133:0/2453111688 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3190007690 con 0x7f31a80b0c20 2024-04-25T10:10:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319f7fe640 1 -- 172.21.15.133:0/2453111688 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190001660 con 0x7f31a80b0c20 2024-04-25T10:10:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f31ad7fd640 1 -- 172.21.15.133:0/2453111688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a8152a50 con 0x7f31a80b0c20 2024-04-25T10:10:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f31ad7fd640 1 -- 172.21.15.133:0/2453111688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a81f3cd0 con 0x7f31a80b0c20 2024-04-25T10:10:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f31ad7fd640 1 -- 172.21.15.133:0/2453111688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a8068ef0 con 0x7f31a80b0c20 2024-04-25T10:10:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319f7fe640 1 -- 172.21.15.133:0/2453111688 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f31900047f0 con 0x7f31a80b0c20 2024-04-25T10:10:11.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319f7fe640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3188066f60 0x7f3188069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f319f7fe640 1 -- 172.21.15.133:0/2453111688 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f3190122410 con 0x7f31a80b0c20 2024-04-25T10:10:11.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f31a6ffd640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3188066f60 0x7f3188069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.675+0000 7f31a6ffd640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3188066f60 0x7f3188069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f31a804f450 tx=0x7f3198008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.679+0000 7f319f7fe640 1 -- 172.21.15.133:0/2453111688 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3190019100 con 0x7f31a80b0c20 2024-04-25T10:10:11.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.843+0000 7f31ad7fd640 1 -- 172.21.15.133:0/2453111688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31a815d610 con 0x7f31a80b0c20 2024-04-25T10:10:11.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.843+0000 7f319f7fe640 1 -- 172.21.15.133:0/2453111688 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17035 (secure 0 0 0) 0x7f31900f3300 con 0x7f31a80b0c20 2024-04-25T10:10:11.850 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:11.851 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":83,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:09:46.874553+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:29.693214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:30.421009+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":82,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 -- 172.21.15.133:0/2453111688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3188066f60 msgr2=0x7f3188069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3188066f60 0x7f3188069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f31a804f450 tx=0x7f3198008040 comp rx=0 tx=0).stop 2024-04-25T10:10:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 -- 172.21.15.133:0/2453111688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31a80b0c20 msgr2=0x7f31a80b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31a80b0c20 0x7f31a80b10a0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f319001f220 tx=0x7f3194005840 comp rx=0 tx=0).stop 2024-04-25T10:10:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 -- 172.21.15.133:0/2453111688 shutdown_connections 2024-04-25T10:10:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31a8152090 0x7f31a8152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3188066f60 0x7f3188069420 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31a80b0c20 0x7f31a80b10a0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 --2- 172.21.15.133:0/2453111688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3194004810 0x7f31a8142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 -- 172.21.15.133:0/2453111688 >> 172.21.15.133:0/2453111688 conn(0x7f31a8067000 msgr2=0x7f31a80b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 -- 172.21.15.133:0/2453111688 shutdown_connections 2024-04-25T10:10:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.847+0000 7f31ad7fd640 1 -- 172.21.15.133:0/2453111688 wait complete. 2024-04-25T10:10:11.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:29.693214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:30.421009+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 82, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:11.870 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38f35a640 1 Processor -- start 2024-04-25T10:10:11.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38f35a640 1 -- start start 2024-04-25T10:10:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38f35a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff38815eb40 0x7ff3880b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38f35a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3880aebc0 0x7ff3880af010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38f35a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3881471f0 con 0x7ff38806e8b0 2024-04-25T10:10:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38f35a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3880af550 con 0x7ff38815eb40 2024-04-25T10:10:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38f35a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff388147490 con 0x7ff3880aebc0 2024-04-25T10:10:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38c8f5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff38815eb40 0x7ff3880b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38c8f5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff38815eb40 0x7ff3880b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34218/0 (socket says 172.21.15.133:34218) 2024-04-25T10:10:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38c8f5640 1 -- 172.21.15.133:0/3413045069 learned_addr learned my addr 172.21.15.133:0/3413045069 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:11.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38d8f7640 1 --2- 172.21.15.133:0/3413045069 >> v2:172.21.15.162:3300/0 conn(0x7ff3880aebc0 0x7ff3880af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:11.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38c8f5640 1 -- 172.21.15.133:0/3413045069 >> v2:172.21.15.162:3300/0 conn(0x7ff3880aebc0 msgr2=0x7ff3880af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38c8f5640 1 --2- 172.21.15.133:0/3413045069 >> v2:172.21.15.162:3300/0 conn(0x7ff3880aebc0 0x7ff3880af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38c8f5640 1 -- 172.21.15.133:0/3413045069 >> v1:172.21.15.162:6789/0 conn(0x7ff38806e8b0 legacy=0x7ff388146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:11.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38c8f5640 1 -- 172.21.15.133:0/3413045069 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3880b1090 con 0x7ff38815eb40 2024-04-25T10:10:11.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.987+0000 7ff38d8f7640 1 --2- 172.21.15.133:0/3413045069 >> v2:172.21.15.162:3300/0 conn(0x7ff3880aebc0 0x7ff3880af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:11.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38c8f5640 1 --2- 172.21.15.133:0/3413045069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff38815eb40 0x7ff3880b1e50 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff3880b27f0 tx=0x7ff38401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cde8f28bbbc264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:11.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff37ffff640 1 -- 172.21.15.133:0/3413045069 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3840b0070 con 0x7ff38815eb40 2024-04-25T10:10:11.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff37ffff640 1 -- 172.21.15.133:0/3413045069 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3840ab600 con 0x7ff38815eb40 2024-04-25T10:10:11.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff37ffff640 1 -- 172.21.15.133:0/3413045069 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3840ab9d0 con 0x7ff38815eb40 2024-04-25T10:10:11.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 -- 172.21.15.133:0/3413045069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff38815eb40 msgr2=0x7ff3880b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:11.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 --2- 172.21.15.133:0/3413045069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff38815eb40 0x7ff3880b1e50 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff3880b27f0 tx=0x7ff38401c450 comp rx=0 tx=0).stop 2024-04-25T10:10:11.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 -- 172.21.15.133:0/3413045069 shutdown_connections 2024-04-25T10:10:11.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 --2- 172.21.15.133:0/3413045069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff38815eb40 0x7ff3880b1e50 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 --2- 172.21.15.133:0/3413045069 >> v2:172.21.15.162:3300/0 conn(0x7ff3880aebc0 0x7ff3880af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:11.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 -- 172.21.15.133:0/3413045069 >> 172.21.15.133:0/3413045069 conn(0x7ff388066f70 msgr2=0x7ff38806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:11.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 -- 172.21.15.133:0/3413045069 shutdown_connections 2024-04-25T10:10:11.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 -- 172.21.15.133:0/3413045069 wait complete. 2024-04-25T10:10:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 Processor -- start 2024-04-25T10:10:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 -- start start 2024-04-25T10:10:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff38806e8b0 0x7ff38815ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3880aebc0 0x7ff388145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff38815eb40 0x7ff3881408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38806de40 con 0x7ff38806e8b0 2024-04-25T10:10:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38806dcc0 con 0x7ff3880aebc0 2024-04-25T10:10:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38f35a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3880b2390 con 0x7ff38815eb40 2024-04-25T10:10:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38d0f6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff38806e8b0 0x7ff38815ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38d0f6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff38806e8b0 0x7ff38815ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34228/0 (socket says 172.21.15.133:34228) 2024-04-25T10:10:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38d0f6640 1 -- 172.21.15.133:0/1489998420 learned_addr learned my addr 172.21.15.133:0/1489998420 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38d8f7640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff38815eb40 0x7ff3881408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38c8f5640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3880aebc0 0x7ff388145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38d0f6640 1 -- 172.21.15.133:0/1489998420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff38815eb40 msgr2=0x7ff3881408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38d0f6640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff38815eb40 0x7ff3881408c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38d0f6640 1 -- 172.21.15.133:0/1489998420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3880aebc0 msgr2=0x7ff388145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38d0f6640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3880aebc0 0x7ff388145e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38d0f6640 1 -- 172.21.15.133:0/1489998420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3881463a0 con 0x7ff38806e8b0 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.991+0000 7ff38d0f6640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff38806e8b0 0x7ff38815ca70 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c450 tx=0x7ff37800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.995+0000 7ff37dffb640 1 -- 172.21.15.133:0/1489998420 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378012830 con 0x7ff38806e8b0 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.995+0000 7ff37dffb640 1 -- 172.21.15.133:0/1489998420 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff378017070 con 0x7ff38806e8b0 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.995+0000 7ff38f35a640 1 -- 172.21.15.133:0/1489998420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff388141430 con 0x7ff38806e8b0 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.995+0000 7ff37dffb640 1 -- 172.21.15.133:0/1489998420 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378012c30 con 0x7ff38806e8b0 2024-04-25T10:10:12.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.995+0000 7ff38f35a640 1 -- 172.21.15.133:0/1489998420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff388145380 con 0x7ff38806e8b0 2024-04-25T10:10:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.995+0000 7ff38f35a640 1 -- 172.21.15.133:0/1489998420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff388068e80 con 0x7ff38806e8b0 2024-04-25T10:10:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.995+0000 7ff37dffb640 1 -- 172.21.15.133:0/1489998420 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff37801c460 con 0x7ff38806e8b0 2024-04-25T10:10:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.995+0000 7ff37dffb640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff36c066f60 0x7ff36c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.995+0000 7ff37dffb640 1 -- 172.21.15.133:0/1489998420 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7ff378084f90 con 0x7ff38806e8b0 2024-04-25T10:10:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.995+0000 7ff38c8f5640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff36c066f60 0x7ff36c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.995+0000 7ff38c8f5640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff36c066f60 0x7ff36c069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff3840ad000 tx=0x7ff3840a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:11.999+0000 7ff37dffb640 1 -- 172.21.15.133:0/1489998420 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff378055df0 con 0x7ff38806e8b0 2024-04-25T10:10:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.163+0000 7ff38f35a640 1 -- 172.21.15.133:0/1489998420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff388161840 con 0x7ff38806e8b0 2024-04-25T10:10:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff37dffb640 1 -- 172.21.15.133:0/1489998420 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3780556f0 con 0x7ff38806e8b0 2024-04-25T10:10:12.172 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:10:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 -- 172.21.15.133:0/1489998420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff36c066f60 msgr2=0x7ff36c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff36c066f60 0x7ff36c069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff3840ad000 tx=0x7ff3840a6000 comp rx=0 tx=0).stop 2024-04-25T10:10:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 -- 172.21.15.133:0/1489998420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff38806e8b0 msgr2=0x7ff38815ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff38806e8b0 0x7ff38815ca70 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c450 tx=0x7ff37800c910 comp rx=0 tx=0).stop 2024-04-25T10:10:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 -- 172.21.15.133:0/1489998420 shutdown_connections 2024-04-25T10:10:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff38815eb40 0x7ff3881408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3880aebc0 0x7ff388145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff36c066f60 0x7ff36c069420 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 --2- 172.21.15.133:0/1489998420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff38806e8b0 0x7ff38815ca70 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 -- 172.21.15.133:0/1489998420 >> 172.21.15.133:0/1489998420 conn(0x7ff388066f70 msgr2=0x7ff388161d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 -- 172.21.15.133:0/1489998420 shutdown_connections 2024-04-25T10:10:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.167+0000 7ff38f35a640 1 -- 172.21.15.133:0/1489998420 wait complete. 2024-04-25T10:10:12.191 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f22753640 1 Processor -- start 2024-04-25T10:10:12.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f22753640 1 -- start start 2024-04-25T10:10:12.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f22753640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4f1c0cc440 0x7f4f1c15cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f22753640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4f1c15ba70 0x7f4f1c0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f22753640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f1c06ad20 0x7f4f1c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f22753640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c15beb0 con 0x7f4f1c15ba70 2024-04-25T10:10:12.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f22753640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c0ccd60 con 0x7f4f1c0cc440 2024-04-25T10:10:12.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f22753640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c06baa0 con 0x7f4f1c06ad20 2024-04-25T10:10:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f1b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4f1c15ba70 0x7f4f1c0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f1b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4f1c15ba70 0x7f4f1c0cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34232/0 (socket says 172.21.15.133:34232) 2024-04-25T10:10:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f1b7fe640 1 -- 172.21.15.133:0/3357083555 learned_addr learned my addr 172.21.15.133:0/3357083555 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f1b7fe640 1 -- 172.21.15.133:0/3357083555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f1c06ad20 msgr2=0x7f4f1c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f20cf0640 1 --2- 172.21.15.133:0/3357083555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f1c06ad20 0x7f4f1c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f1bfff640 1 --2- 172.21.15.133:0/3357083555 >> v2:172.21.15.162:3300/0 conn(0x7f4f1c0cc440 0x7f4f1c15cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f1b7fe640 1 --2- 172.21.15.133:0/3357083555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f1c06ad20 0x7f4f1c06b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f1b7fe640 1 -- 172.21.15.133:0/3357083555 >> v2:172.21.15.162:3300/0 conn(0x7f4f1c0cc440 msgr2=0x7f4f1c15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f1b7fe640 1 --2- 172.21.15.133:0/3357083555 >> v2:172.21.15.162:3300/0 conn(0x7f4f1c0cc440 0x7f4f1c15cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.299+0000 7f4f1b7fe640 1 -- 172.21.15.133:0/3357083555 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c15c100 con 0x7f4f1c15ba70 2024-04-25T10:10:12.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f1b7fe640 1 --2- 172.21.15.133:0/3357083555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c15ba70 0x7f4f1c0cc820 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0ccf80 tx=0x7f4f1001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884a2aac5948b10f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f1affd640 1 -- 172.21.15.133:0/3357083555 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f100aa610 con 0x7f4f1c15ba70 2024-04-25T10:10:12.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f1affd640 1 -- 172.21.15.133:0/3357083555 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f100af070 con 0x7f4f1c15ba70 2024-04-25T10:10:12.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f1affd640 1 -- 172.21.15.133:0/3357083555 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f100aa9d0 con 0x7f4f1c15ba70 2024-04-25T10:10:12.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 -- 172.21.15.133:0/3357083555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c15ba70 msgr2=0x7f4f1c0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 --2- 172.21.15.133:0/3357083555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c15ba70 0x7f4f1c0cc820 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0ccf80 tx=0x7f4f1001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 -- 172.21.15.133:0/3357083555 shutdown_connections 2024-04-25T10:10:12.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 --2- 172.21.15.133:0/3357083555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f1c06ad20 0x7f4f1c06b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 --2- 172.21.15.133:0/3357083555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c15ba70 0x7f4f1c0cc820 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 --2- 172.21.15.133:0/3357083555 >> v2:172.21.15.162:3300/0 conn(0x7f4f1c0cc440 0x7f4f1c15cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 -- 172.21.15.133:0/3357083555 >> 172.21.15.133:0/3357083555 conn(0x7f4f1c0999d0 msgr2=0x7f4f1c0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 -- 172.21.15.133:0/3357083555 shutdown_connections 2024-04-25T10:10:12.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 -- 172.21.15.133:0/3357083555 wait complete. 2024-04-25T10:10:12.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 Processor -- start 2024-04-25T10:10:12.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 -- start start 2024-04-25T10:10:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c06ad20 0x7f4f1c1516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f1c0cc440 0x7f4f1c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f1c15ba70 0x7f4f1c153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0b0860 con 0x7f4f1c06ad20 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0b06e0 con 0x7f4f1c15ba70 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f22753640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0b09e0 con 0x7f4f1c0cc440 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f1bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c06ad20 0x7f4f1c1516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f1bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c06ad20 0x7f4f1c1516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34236/0 (socket says 172.21.15.133:34236) 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.303+0000 7f4f1bfff640 1 -- 172.21.15.133:0/1980377854 learned_addr learned my addr 172.21.15.133:0/1980377854 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f1bfff640 1 -- 172.21.15.133:0/1980377854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f1c0cc440 msgr2=0x7f4f1c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f1bfff640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f1c0cc440 0x7f4f1c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f1bfff640 1 -- 172.21.15.133:0/1980377854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f1c15ba70 msgr2=0x7f4f1c153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f1bfff640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f1c15ba70 0x7f4f1c153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f1bfff640 1 -- 172.21.15.133:0/1980377854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c152790 con 0x7f4f1c06ad20 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f1bfff640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c06ad20 0x7f4f1c1516c0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00c7f0 tx=0x7f4f0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f18ff9640 1 -- 172.21.15.133:0/1980377854 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c015070 con 0x7f4f1c06ad20 2024-04-25T10:10:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f18ff9640 1 -- 172.21.15.133:0/1980377854 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c004240 con 0x7f4f1c06ad20 2024-04-25T10:10:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f18ff9640 1 -- 172.21.15.133:0/1980377854 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c003620 con 0x7f4f1c06ad20 2024-04-25T10:10:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f22753640 1 -- 172.21.15.133:0/1980377854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c152a70 con 0x7f4f1c06ad20 2024-04-25T10:10:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f22753640 1 -- 172.21.15.133:0/1980377854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c153030 con 0x7f4f1c06ad20 2024-04-25T10:10:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f22753640 1 -- 172.21.15.133:0/1980377854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c067c70 con 0x7f4f1c06ad20 2024-04-25T10:10:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f18ff9640 1 -- 172.21.15.133:0/1980377854 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f0c0039d0 con 0x7f4f1c06ad20 2024-04-25T10:10:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f18ff9640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f08066f60 0x7f4f08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f18ff9640 1 -- 172.21.15.133:0/1980377854 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f4f0c085130 con 0x7f4f1c06ad20 2024-04-25T10:10:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f1b7fe640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f08066f60 0x7f4f08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.307+0000 7f4f1b7fe640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f08066f60 0x7f4f08069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4f100ae000 tx=0x7f4f10004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.311+0000 7f4f18ff9640 1 -- 172.21.15.133:0/1980377854 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f0c056020 con 0x7f4f1c06ad20 2024-04-25T10:10:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.455+0000 7f4f22753640 1 -- 172.21.15.133:0/1980377854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4f1c0af2c0 con 0x7f4f1c06ad20 2024-04-25T10:10:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f18ff9640 1 -- 172.21.15.133:0/1980377854 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4f0c055920 con 0x7f4f1c06ad20 2024-04-25T10:10:12.464 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 -- 172.21.15.133:0/1980377854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f08066f60 msgr2=0x7f4f08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f08066f60 0x7f4f08069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4f100ae000 tx=0x7f4f10004740 comp rx=0 tx=0).stop 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 -- 172.21.15.133:0/1980377854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c06ad20 msgr2=0x7f4f1c1516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c06ad20 0x7f4f1c1516c0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00c7f0 tx=0x7f4f0c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 -- 172.21.15.133:0/1980377854 shutdown_connections 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f1c15ba70 0x7f4f1c153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f1c0cc440 0x7f4f1c0b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f08066f60 0x7f4f08069420 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 --2- 172.21.15.133:0/1980377854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c06ad20 0x7f4f1c1516c0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 -- 172.21.15.133:0/1980377854 >> 172.21.15.133:0/1980377854 conn(0x7f4f1c0999d0 msgr2=0x7f4f1c0b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 -- 172.21.15.133:0/1980377854 shutdown_connections 2024-04-25T10:10:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.459+0000 7f4f22753640 1 -- 172.21.15.133:0/1980377854 wait complete. 2024-04-25T10:10:12.483 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:12.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.599+0000 7f8ebb20b640 1 Processor -- start 2024-04-25T10:10:12.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8ebb20b640 1 -- start start 2024-04-25T10:10:12.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8ebb20b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb4143ea0 0x7f8eb40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8ebb20b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8eb406eac0 0x7f8eb40b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8ebb20b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb415ac50 con 0x7f8eb40b2150 2024-04-25T10:10:12.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8ebb20b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb40cd180 con 0x7f8eb406eac0 2024-04-25T10:10:12.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8ebb20b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb415aed0 con 0x7f8eb4143ea0 2024-04-25T10:10:12.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8eb8fa7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb4143ea0 0x7f8eb40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8eb97a8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8eb406eac0 0x7f8eb40b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8eb8fa7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb4143ea0 0x7f8eb40cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:48904/0 (socket says 172.21.15.133:48904) 2024-04-25T10:10:12.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8eb8fa7640 1 -- 172.21.15.133:0/501346158 learned_addr learned my addr 172.21.15.133:0/501346158 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:12.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8eb8fa7640 1 -- 172.21.15.133:0/501346158 >> v2:172.21.15.162:3300/0 conn(0x7f8eb406eac0 msgr2=0x7f8eb40b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:12.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8eb8fa7640 1 --2- 172.21.15.133:0/501346158 >> v2:172.21.15.162:3300/0 conn(0x7f8eb406eac0 0x7f8eb40b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8eb8fa7640 1 -- 172.21.15.133:0/501346158 >> v1:172.21.15.162:6789/0 conn(0x7f8eb40b2150 legacy=0x7f8eb406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.603+0000 7f8eb8fa7640 1 -- 172.21.15.133:0/501346158 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb415b1b0 con 0x7f8eb4143ea0 2024-04-25T10:10:12.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8eb8fa7640 1 --2- 172.21.15.133:0/501346158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb4143ea0 0x7f8eb40cc810 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009870 tx=0x7f8ea801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c87db217467a9dca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8eb37fe640 1 -- 172.21.15.133:0/501346158 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80a7070 con 0x7f8eb4143ea0 2024-04-25T10:10:12.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8eb37fe640 1 -- 172.21.15.133:0/501346158 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea80b0070 con 0x7f8eb4143ea0 2024-04-25T10:10:12.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8eb37fe640 1 -- 172.21.15.133:0/501346158 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80ab890 con 0x7f8eb4143ea0 2024-04-25T10:10:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 -- 172.21.15.133:0/501346158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb4143ea0 msgr2=0x7f8eb40cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 --2- 172.21.15.133:0/501346158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb4143ea0 0x7f8eb40cc810 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009870 tx=0x7f8ea801c560 comp rx=0 tx=0).stop 2024-04-25T10:10:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 -- 172.21.15.133:0/501346158 shutdown_connections 2024-04-25T10:10:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 --2- 172.21.15.133:0/501346158 >> v2:172.21.15.162:3300/0 conn(0x7f8eb406eac0 0x7f8eb40b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 --2- 172.21.15.133:0/501346158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb4143ea0 0x7f8eb40cc810 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 -- 172.21.15.133:0/501346158 >> 172.21.15.133:0/501346158 conn(0x7f8eb4066f70 msgr2=0x7f8eb406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 -- 172.21.15.133:0/501346158 shutdown_connections 2024-04-25T10:10:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 -- 172.21.15.133:0/501346158 wait complete. 2024-04-25T10:10:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 Processor -- start 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 -- start start 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8eb406eac0 0x7f8eb4142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb40b2150 0x7f8eb4143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb4143ea0 0x7f8eb413e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb4167550 con 0x7f8eb4143ea0 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb41673d0 con 0x7f8eb406eac0 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8ebb20b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb41676d0 con 0x7f8eb40b2150 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8eb97a8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb4143ea0 0x7f8eb413e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8eb97a8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb4143ea0 0x7f8eb413e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34246/0 (socket says 172.21.15.133:34246) 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8eb97a8640 1 -- 172.21.15.133:0/3202384986 learned_addr learned my addr 172.21.15.133:0/3202384986 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8eb8fa7640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8eb406eac0 0x7f8eb4142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.607+0000 7f8eb3fff640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb40b2150 0x7f8eb4143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb97a8640 1 -- 172.21.15.133:0/3202384986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb40b2150 msgr2=0x7f8eb4143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb97a8640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb40b2150 0x7f8eb4143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb97a8640 1 -- 172.21.15.133:0/3202384986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8eb406eac0 msgr2=0x7f8eb4142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb97a8640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8eb406eac0 0x7f8eb4142c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb97a8640 1 -- 172.21.15.133:0/3202384986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb415a210 con 0x7f8eb4143ea0 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb97a8640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb4143ea0 0x7f8eb413e170 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00efc0 tx=0x7f8eac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb17fa640 1 -- 172.21.15.133:0/3202384986 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac018020 con 0x7f8eb4143ea0 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb17fa640 1 -- 172.21.15.133:0/3202384986 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eac014480 con 0x7f8eb4143ea0 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8ebb20b640 1 -- 172.21.15.133:0/3202384986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb4138ef0 con 0x7f8eb4143ea0 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb17fa640 1 -- 172.21.15.133:0/3202384986 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac009e60 con 0x7f8eb4143ea0 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8ebb20b640 1 -- 172.21.15.133:0/3202384986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb41394b0 con 0x7f8eb4143ea0 2024-04-25T10:10:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8ebb20b640 1 -- 172.21.15.133:0/3202384986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb4068e80 con 0x7f8eb4143ea0 2024-04-25T10:10:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb17fa640 1 -- 172.21.15.133:0/3202384986 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8eac01b020 con 0x7f8eb4143ea0 2024-04-25T10:10:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb17fa640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e98066f60 0x7f8e98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb8fa7640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e98066f60 0x7f8e98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb17fa640 1 -- 172.21.15.133:0/3202384986 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f8eac085e20 con 0x7f8eb4143ea0 2024-04-25T10:10:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.611+0000 7f8eb8fa7640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e98066f60 0x7f8e98069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80af000 tx=0x7f8ea80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.615+0000 7f8eb17fa640 1 -- 172.21.15.133:0/3202384986 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8eac022030 con 0x7f8eb4143ea0 2024-04-25T10:10:12.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.759+0000 7f8ebb20b640 1 -- 172.21.15.133:0/3202384986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8eb415bf20 con 0x7f8eb4143ea0 2024-04-25T10:10:12.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8eb17fa640 1 -- 172.21.15.133:0/3202384986 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8eac010040 con 0x7f8eb4143ea0 2024-04-25T10:10:12.768 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:10:12.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 -- 172.21.15.133:0/3202384986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e98066f60 msgr2=0x7f8e98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e98066f60 0x7f8e98069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80af000 tx=0x7f8ea80a7000 comp rx=0 tx=0).stop 2024-04-25T10:10:12.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 -- 172.21.15.133:0/3202384986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb4143ea0 msgr2=0x7f8eb413e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb4143ea0 0x7f8eb413e170 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00efc0 tx=0x7f8eac00c490 comp rx=0 tx=0).stop 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 -- 172.21.15.133:0/3202384986 shutdown_connections 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e98066f60 0x7f8e98069420 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb4143ea0 0x7f8eb413e170 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb40b2150 0x7f8eb4143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 --2- 172.21.15.133:0/3202384986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8eb406eac0 0x7f8eb4142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 -- 172.21.15.133:0/3202384986 >> 172.21.15.133:0/3202384986 conn(0x7f8eb4066f70 msgr2=0x7f8eb4158850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 -- 172.21.15.133:0/3202384986 shutdown_connections 2024-04-25T10:10:12.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.763+0000 7f8ebb20b640 1 -- 172.21.15.133:0/3202384986 wait complete. 2024-04-25T10:10:12.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:10:12.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f06a59f4640 1 Processor -- start 2024-04-25T10:10:12.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f06a59f4640 1 -- start start 2024-04-25T10:10:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f06a59f4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f06a015d6d0 0x7f06a00b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f06a59f4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f06a00b6dc0 0x7f06a00b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f06a59f4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a00b72d0 con 0x7f06a00b9590 2024-04-25T10:10:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f06a59f4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a00b75b0 con 0x7f06a015d6d0 2024-04-25T10:10:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f06a59f4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a00b7890 con 0x7f06a00b6dc0 2024-04-25T10:10:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f069effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f06a015d6d0 0x7f06a00b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f069effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f06a015d6d0 0x7f06a00b6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34258/0 (socket says 172.21.15.133:34258) 2024-04-25T10:10:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f069effd640 1 -- 172.21.15.133:0/1774437568 learned_addr learned my addr 172.21.15.133:0/1774437568 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f069effd640 1 -- 172.21.15.133:0/1774437568 >> v2:172.21.15.162:3300/0 conn(0x7f06a00b6dc0 msgr2=0x7f06a00b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f069effd640 1 --2- 172.21.15.133:0/1774437568 >> v2:172.21.15.162:3300/0 conn(0x7f06a00b6dc0 0x7f06a00b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f069effd640 1 -- 172.21.15.133:0/1774437568 >> v1:172.21.15.162:6789/0 conn(0x7f06a00b9590 legacy=0x7f06a00d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.903+0000 7f069effd640 1 -- 172.21.15.133:0/1774437568 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a00b7b70 con 0x7f06a015d6d0 2024-04-25T10:10:12.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f069effd640 1 --2- 172.21.15.133:0/1774437568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06a015d6d0 0x7f06a00b6880 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f06a00b53c0 tx=0x7f069001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50f50afacba29ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f069dffb640 1 -- 172.21.15.133:0/1774437568 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900b5070 con 0x7f06a015d6d0 2024-04-25T10:10:12.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f069dffb640 1 -- 172.21.15.133:0/1774437568 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06900a89d0 con 0x7f06a015d6d0 2024-04-25T10:10:12.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f069dffb640 1 -- 172.21.15.133:0/1774437568 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900a8da0 con 0x7f06a015d6d0 2024-04-25T10:10:12.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 -- 172.21.15.133:0/1774437568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06a015d6d0 msgr2=0x7f06a00b6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 --2- 172.21.15.133:0/1774437568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06a015d6d0 0x7f06a00b6880 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f06a00b53c0 tx=0x7f069001ecd0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 -- 172.21.15.133:0/1774437568 shutdown_connections 2024-04-25T10:10:12.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 --2- 172.21.15.133:0/1774437568 >> v2:172.21.15.162:3300/0 conn(0x7f06a00b6dc0 0x7f06a00b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 --2- 172.21.15.133:0/1774437568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06a015d6d0 0x7f06a00b6880 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 -- 172.21.15.133:0/1774437568 >> 172.21.15.133:0/1774437568 conn(0x7f06a00727c0 msgr2=0x7f06a0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:12.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 -- 172.21.15.133:0/1774437568 shutdown_connections 2024-04-25T10:10:12.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 -- 172.21.15.133:0/1774437568 wait complete. 2024-04-25T10:10:12.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 Processor -- start 2024-04-25T10:10:12.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 -- start start 2024-04-25T10:10:12.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06a00b6dc0 0x7f06a01f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06a00b9590 0x7f06a01561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06a015d6d0 0x7f06a0155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a015dab0 con 0x7f06a00b9590 2024-04-25T10:10:12.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a00d1f60 con 0x7f06a015d6d0 2024-04-25T10:10:12.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.907+0000 7f06a59f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a015dc30 con 0x7f06a00b6dc0 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06a00b9590 0x7f06a01561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06a00b9590 0x7f06a01561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34274/0 (socket says 172.21.15.133:34274) 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069e7fc640 1 -- 172.21.15.133:0/4234736052 learned_addr learned my addr 172.21.15.133:0/4234736052 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069effd640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06a00b6dc0 0x7f06a01f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069f7fe640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06a015d6d0 0x7f06a0155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069e7fc640 1 -- 172.21.15.133:0/4234736052 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06a00b6dc0 msgr2=0x7f06a01f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069e7fc640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06a00b6dc0 0x7f06a01f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069e7fc640 1 -- 172.21.15.133:0/4234736052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06a015d6d0 msgr2=0x7f06a0155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069e7fc640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06a015d6d0 0x7f06a0155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069e7fc640 1 -- 172.21.15.133:0/4234736052 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a01f4130 con 0x7f06a00b9590 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069effd640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06a00b6dc0 0x7f06a01f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069e7fc640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06a00b9590 0x7f06a01561b0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f069400c4c0 tx=0x7f0694001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f0687fff640 1 -- 172.21.15.133:0/4234736052 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694017070 con 0x7f06a00b9590 2024-04-25T10:10:12.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f0687fff640 1 -- 172.21.15.133:0/4234736052 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694003600 con 0x7f06a00b9590 2024-04-25T10:10:12.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f06a59f4640 1 -- 172.21.15.133:0/4234736052 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a01f4410 con 0x7f06a00b9590 2024-04-25T10:10:12.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f0687fff640 1 -- 172.21.15.133:0/4234736052 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06940039f0 con 0x7f06a00b9590 2024-04-25T10:10:12.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f06a59f4640 1 -- 172.21.15.133:0/4234736052 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a01557c0 con 0x7f06a00b9590 2024-04-25T10:10:12.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f06a59f4640 1 -- 172.21.15.133:0/4234736052 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a0068e80 con 0x7f06a00b9590 2024-04-25T10:10:12.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f0687fff640 1 -- 172.21.15.133:0/4234736052 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f069401a070 con 0x7f06a00b9590 2024-04-25T10:10:12.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f0687fff640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f068c066f90 0x7f068c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:12.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f0687fff640 1 -- 172.21.15.133:0/4234736052 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f0694084450 con 0x7f06a00b9590 2024-04-25T10:10:12.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069effd640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f068c066f90 0x7f068c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:12.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.911+0000 7f069effd640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f068c066f90 0x7f068c069450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f06900b2000 tx=0x7f0690008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:12.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:12.915+0000 7f0687fff640 1 -- 172.21.15.133:0/4234736052 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0694055340 con 0x7f06a00b9590 2024-04-25T10:10:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 -- 172.21.15.133:0/4234736052 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f06a00d0d20 con 0x7f06a00b9590 2024-04-25T10:10:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f0687fff640 1 -- 172.21.15.133:0/4234736052 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0694054c40 con 0x7f06a00b9590 2024-04-25T10:10:13.097 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:10:13.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 -- 172.21.15.133:0/4234736052 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f068c066f90 msgr2=0x7f068c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f068c066f90 0x7f068c069450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f06900b2000 tx=0x7f0690008040 comp rx=0 tx=0).stop 2024-04-25T10:10:13.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 -- 172.21.15.133:0/4234736052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06a00b9590 msgr2=0x7f06a01561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06a00b9590 0x7f06a01561b0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f069400c4c0 tx=0x7f0694001e80 comp rx=0 tx=0).stop 2024-04-25T10:10:13.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 -- 172.21.15.133:0/4234736052 shutdown_connections 2024-04-25T10:10:13.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06a015d6d0 0x7f06a0155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f068c066f90 0x7f068c069450 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06a00b9590 0x7f06a01561b0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 --2- 172.21.15.133:0/4234736052 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06a00b6dc0 0x7f06a01f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 -- 172.21.15.133:0/4234736052 >> 172.21.15.133:0/4234736052 conn(0x7f06a00727c0 msgr2=0x7f06a00d28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:13.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 -- 172.21.15.133:0/4234736052 shutdown_connections 2024-04-25T10:10:13.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.091+0000 7f06a59f4640 1 -- 172.21.15.133:0/4234736052 wait complete. 2024-04-25T10:10:13.115 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:10:13.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:13.116 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:13.124 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[58401]: fuse finished with error 0 and tester_r 0 2024-04-25T10:10:13.129 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:10:13.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:13.129 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:10:13.298 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:10:13.298 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:13.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:13.304 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:10:13.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:13.305 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:10:13.352 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:10:13.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:10:13.353 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:10:13.353 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:13.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:10:13.359 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:10:13.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:13.359 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:10:13.404 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:10:13.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:10:13.405 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:10:13.405 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:13.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:13.411 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:10:13.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:13.411 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:10:13.459 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:10:13.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:10:13.461 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:13.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.583+0000 7f7c5cd9a640 1 Processor -- start 2024-04-25T10:10:13.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.583+0000 7f7c5cd9a640 1 -- start start 2024-04-25T10:10:13.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5cd9a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7c5806eb70 0x7f7c580b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5cd9a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c580cbe60 con 0x7f7c581584f0 2024-04-25T10:10:13.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5cd9a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c580b19b0 con 0x7f7c58158110 2024-04-25T10:10:13.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5cd9a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c5815b360 con 0x7f7c5806eb70 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c56d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7c581584f0 0x7f7c580cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41386/0 (socket says 172.21.15.133:41386) 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c56d9d640 1 -- 172.21.15.133:0/2993448743 learned_addr learned my addr 172.21.15.133:0/2993448743 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c55d9b640 1 --2- 172.21.15.133:0/2993448743 >> v2:172.21.15.162:3300/0 conn(0x7f7c5806eb70 0x7f7c580b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710554433 0 0) 0x7f7c580cbe60 con 0x7f7c581584f0 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c44003530 con 0x7f7c581584f0 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4144166534 0 0) 0x7f7c44003530 con 0x7f7c581584f0 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 >> v2:172.21.15.162:3300/0 conn(0x7f7c5806eb70 msgr2=0x7f7c580b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 --2- 172.21.15.133:0/2993448743 >> v2:172.21.15.162:3300/0 conn(0x7f7c5806eb70 0x7f7c580b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 >> v1:172.21.15.162:6789/0 conn(0x7f7c58158110 legacy=0x7f7c5806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5815b640 con 0x7f7c581584f0 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7c48001680 con 0x7f7c581584f0 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 >> v1:172.21.15.133:6789/0 conn(0x7f7c581584f0 legacy=0x7f7c580cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 --2- 172.21.15.133:0/2993448743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c440039c0 0x7f7c44003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 --2- 172.21.15.133:0/2993448743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c44004810 0x7f7c44004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 --2- 172.21.15.133:0/2993448743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c440051d0 0x7f7c4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5815b360 con 0x7f7c44004810 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4400c070 con 0x7f7c440039c0 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4400c270 con 0x7f7c440051d0 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c55d9b640 1 --2- 172.21.15.133:0/2993448743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c44004810 0x7f7c44004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c5659c640 1 --2- 172.21.15.133:0/2993448743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c440039c0 0x7f7c44003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c56d9d640 1 --2- 172.21.15.133:0/2993448743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c440051d0 0x7f7c4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c55d9b640 1 -- 172.21.15.133:0/2993448743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c440051d0 msgr2=0x7f7c4400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c55d9b640 1 --2- 172.21.15.133:0/2993448743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c440051d0 0x7f7c4400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c55d9b640 1 -- 172.21.15.133:0/2993448743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c440039c0 msgr2=0x7f7c44003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c55d9b640 1 --2- 172.21.15.133:0/2993448743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c440039c0 0x7f7c44003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c55d9b640 1 -- 172.21.15.133:0/2993448743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c44003530 con 0x7f7c44004810 2024-04-25T10:10:13.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.587+0000 7f7c56d9d640 1 -- 172.21.15.133:0/2993448743 reap_dead start 2024-04-25T10:10:13.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c55d9b640 1 --2- 172.21.15.133:0/2993448743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c44004810 0x7f7c44004c90 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009f70 tx=0x7f7c4001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724d496296e65b76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c400b2070 con 0x7f7c44004810 2024-04-25T10:10:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c400a5cf0 con 0x7f7c44004810 2024-04-25T10:10:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5559a640 1 -- 172.21.15.133:0/2993448743 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c40009bf0 con 0x7f7c44004810 2024-04-25T10:10:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/2993448743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c44004810 msgr2=0x7f7c44004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 --2- 172.21.15.133:0/2993448743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c44004810 0x7f7c44004c90 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009f70 tx=0x7f7c4001cc20 comp rx=0 tx=0).stop 2024-04-25T10:10:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/2993448743 shutdown_connections 2024-04-25T10:10:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 --2- 172.21.15.133:0/2993448743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c44004810 0x7f7c44004c90 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/2993448743 >> 172.21.15.133:0/2993448743 conn(0x7f7c58067000 msgr2=0x7f7c5806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/2993448743 shutdown_connections 2024-04-25T10:10:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/2993448743 wait complete. 2024-04-25T10:10:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 Processor -- start 2024-04-25T10:10:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 -- start start 2024-04-25T10:10:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44004810 0x7f7c58153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5813dea0 0x7f7c5813e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c581ef0c0 0x7f7c581ef520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580cd670 con 0x7f7c581ef0c0 2024-04-25T10:10:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5815bc00 con 0x7f7c44004810 2024-04-25T10:10:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5cd9a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580cd7f0 con 0x7f7c5813dea0 2024-04-25T10:10:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c56d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c581ef0c0 0x7f7c581ef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c56d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c581ef0c0 0x7f7c581ef520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34280/0 (socket says 172.21.15.133:34280) 2024-04-25T10:10:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c56d9d640 1 -- 172.21.15.133:0/3569152722 learned_addr learned my addr 172.21.15.133:0/3569152722 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c5659c640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44004810 0x7f7c58153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c55d9b640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5813dea0 0x7f7c5813e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c56d9d640 1 -- 172.21.15.133:0/3569152722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5813dea0 msgr2=0x7f7c5813e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c56d9d640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5813dea0 0x7f7c5813e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.591+0000 7f7c56d9d640 1 -- 172.21.15.133:0/3569152722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44004810 msgr2=0x7f7c58153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c56d9d640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44004810 0x7f7c58153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c56d9d640 1 -- 172.21.15.133:0/3569152722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5813e840 con 0x7f7c581ef0c0 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c56d9d640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c581ef0c0 0x7f7c581ef520 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f7c58158900 tx=0x7f7c48010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c4effd640 1 -- 172.21.15.133:0/3569152722 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4801b070 con 0x7f7c581ef0c0 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c4effd640 1 -- 172.21.15.133:0/3569152722 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c48011040 con 0x7f7c581ef0c0 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/3569152722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c581f33a0 con 0x7f7c581ef0c0 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c4effd640 1 -- 172.21.15.133:0/3569152722 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4800ed20 con 0x7f7c581ef0c0 2024-04-25T10:10:13.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/3569152722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c581f25e0 con 0x7f7c581ef0c0 2024-04-25T10:10:13.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/3569152722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5806d2d0 con 0x7f7c581ef0c0 2024-04-25T10:10:13.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c4effd640 1 -- 172.21.15.133:0/3569152722 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c4801e070 con 0x7f7c581ef0c0 2024-04-25T10:10:13.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c4effd640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 0x7f7c38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c4effd640 1 -- 172.21.15.133:0/3569152722 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f7c48087950 con 0x7f7c581ef0c0 2024-04-25T10:10:13.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c5659c640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 0x7f7c38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.595+0000 7f7c5659c640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 0x7f7c38069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005510 tx=0x7f7c50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.599+0000 7f7c4effd640 1 -- 172.21.15.133:0/3569152722 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c480587b0 con 0x7f7c581ef0c0 2024-04-25T10:10:13.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.791+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/3569152722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c5806bc20 con 0x7f7c581ef0c0 2024-04-25T10:10:13.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c4effd640 1 -- 172.21.15.133:0/3569152722 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f7c480580b0 con 0x7f7c581ef0c0 2024-04-25T10:10:13.800 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:13.801 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3522294955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3522294955},{"type":"v1","addr":"172.21.15.162:6833","nonce":3522294955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3865231260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3865231260},{"type":"v1","addr":"172.21.15.133:6836","nonce":3865231260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7437,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3935311931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3935311931},{"type":"v1","addr":"172.21.15.133:6837","nonce":3935311931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:07:31.434780+0000","modified":"2024-04-25T10:07:40.739361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/3192312240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3192312240},{"type":"v1","addr":"172.21.15.162:6835","nonce":3192312240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:13.801 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 50 2024-04-25T10:10:13.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/3569152722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 msgr2=0x7f7c38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 0x7f7c38069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005510 tx=0x7f7c50009040 comp rx=0 tx=0).stop 2024-04-25T10:10:13.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/3569152722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c581ef0c0 msgr2=0x7f7c581ef520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c581ef0c0 0x7f7c581ef520 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f7c58158900 tx=0x7f7c48010cf0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/3569152722 shutdown_connections 2024-04-25T10:10:13.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7c38066f60 0x7f7c38069420 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c581ef0c0 0x7f7c581ef520 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7c5813dea0 0x7f7c5813e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 --2- 172.21.15.133:0/3569152722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c44004810 0x7f7c58153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/3569152722 >> 172.21.15.133:0/3569152722 conn(0x7f7c58067000 msgr2=0x7f7c580ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:13.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/3569152722 shutdown_connections 2024-04-25T10:10:13.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.795+0000 7f7c5cd9a640 1 -- 172.21.15.133:0/3569152722 wait complete. 2024-04-25T10:10:13.820 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.935+0000 7f509e387640 1 Processor -- start 2024-04-25T10:10:13.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f509e387640 1 -- start start 2024-04-25T10:10:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f509e387640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f50980d0770 0x7f50980d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f509e387640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50980b87e0 0x7f50980b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f509e387640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50980ceb20 con 0x7f509815db00 2024-04-25T10:10:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f509e387640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50980d1110 con 0x7f50980d0770 2024-04-25T10:10:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f509e387640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50980b30a0 con 0x7f50980b87e0 2024-04-25T10:10:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f50977fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f50980d0770 0x7f50980d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f509c924640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50980b87e0 0x7f50980b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f50977fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f50980d0770 0x7f50980d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55852/0 (socket says 172.21.15.133:55852) 2024-04-25T10:10:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f50977fe640 1 -- 172.21.15.133:0/1709816537 learned_addr learned my addr 172.21.15.133:0/1709816537 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:13.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f509c924640 1 -- 172.21.15.133:0/1709816537 >> v2:172.21.15.162:3300/0 conn(0x7f50980d0770 msgr2=0x7f50980d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:13.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f509c924640 1 --2- 172.21.15.133:0/1709816537 >> v2:172.21.15.162:3300/0 conn(0x7f50980d0770 0x7f50980d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f509c924640 1 -- 172.21.15.133:0/1709816537 >> v1:172.21.15.162:6789/0 conn(0x7f509815db00 legacy=0x7f50980b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.939+0000 7f509c924640 1 -- 172.21.15.133:0/1709816537 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50980b7740 con 0x7f50980b87e0 2024-04-25T10:10:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f509c924640 1 --2- 172.21.15.133:0/1709816537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50980b87e0 0x7f50980b8c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5090009d90 tx=0x7f509001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1636cac5f59a47ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f5096ffd640 1 -- 172.21.15.133:0/1709816537 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50900b2070 con 0x7f50980b87e0 2024-04-25T10:10:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f5096ffd640 1 -- 172.21.15.133:0/1709816537 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50900a5a00 con 0x7f50980b87e0 2024-04-25T10:10:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f5096ffd640 1 -- 172.21.15.133:0/1709816537 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50900a5dd0 con 0x7f50980b87e0 2024-04-25T10:10:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f509e387640 1 -- 172.21.15.133:0/1709816537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50980b87e0 msgr2=0x7f50980b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f509e387640 1 --2- 172.21.15.133:0/1709816537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50980b87e0 0x7f50980b8c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5090009d90 tx=0x7f509001c940 comp rx=0 tx=0).stop 2024-04-25T10:10:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f509e387640 1 -- 172.21.15.133:0/1709816537 shutdown_connections 2024-04-25T10:10:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f509e387640 1 --2- 172.21.15.133:0/1709816537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50980b87e0 0x7f50980b8c70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f509e387640 1 --2- 172.21.15.133:0/1709816537 >> v2:172.21.15.162:3300/0 conn(0x7f50980d0770 0x7f50980d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f509e387640 1 -- 172.21.15.133:0/1709816537 >> 172.21.15.133:0/1709816537 conn(0x7f50980727c0 msgr2=0x7f5098072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f509e387640 1 -- 172.21.15.133:0/1709816537 shutdown_connections 2024-04-25T10:10:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f509e387640 1 -- 172.21.15.133:0/1709816537 wait complete. 2024-04-25T10:10:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f509e387640 1 Processor -- start 2024-04-25T10:10:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.943+0000 7f509e387640 1 -- start start 2024-04-25T10:10:13.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509e387640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50980b87e0 0x7f50981566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509e387640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50980d0770 0x7f5098157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509e387640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f509815db00 0x7f5098151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509e387640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50980b7430 con 0x7f509815db00 2024-04-25T10:10:13.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509e387640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50980b82c0 con 0x7f50980d0770 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509e387640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50980d18a0 con 0x7f50980b87e0 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509c924640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f509815db00 0x7f5098151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509c924640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f509815db00 0x7f5098151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34282/0 (socket says 172.21.15.133:34282) 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509c924640 1 -- 172.21.15.133:0/1719380676 learned_addr learned my addr 172.21.15.133:0/1719380676 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f50977fe640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50980d0770 0x7f5098157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f5097fff640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50980b87e0 0x7f50981566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509c924640 1 -- 172.21.15.133:0/1719380676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50980b87e0 msgr2=0x7f50981566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509c924640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50980b87e0 0x7f50981566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509c924640 1 -- 172.21.15.133:0/1719380676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50980d0770 msgr2=0x7f5098157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509c924640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50980d0770 0x7f5098157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509c924640 1 -- 172.21.15.133:0/1719380676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5098157eb0 con 0x7f509815db00 2024-04-25T10:10:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509c924640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f509815db00 0x7f5098151f40 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f50900b0040 tx=0x7f50900196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f5094ff9640 1 -- 172.21.15.133:0/1719380676 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50900b2050 con 0x7f509815db00 2024-04-25T10:10:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f5094ff9640 1 -- 172.21.15.133:0/1719380676 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50900be070 con 0x7f509815db00 2024-04-25T10:10:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f5094ff9640 1 -- 172.21.15.133:0/1719380676 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50900ada70 con 0x7f509815db00 2024-04-25T10:10:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509e387640 1 -- 172.21.15.133:0/1719380676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5098152480 con 0x7f509815db00 2024-04-25T10:10:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509e387640 1 -- 172.21.15.133:0/1719380676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50980c90f0 con 0x7f509815db00 2024-04-25T10:10:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f509e387640 1 -- 172.21.15.133:0/1719380676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5098068ef0 con 0x7f509815db00 2024-04-25T10:10:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f5094ff9640 1 -- 172.21.15.133:0/1719380676 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50900adc10 con 0x7f509815db00 2024-04-25T10:10:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f5094ff9640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5084066f60 0x7f5084069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f5094ff9640 1 -- 172.21.15.133:0/1719380676 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f5090120520 con 0x7f509815db00 2024-04-25T10:10:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.947+0000 7f5097fff640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5084066f60 0x7f5084069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.951+0000 7f5097fff640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5084066f60 0x7f5084069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5088000f70 tx=0x7f5088009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:13.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:13.951+0000 7f5094ff9640 1 -- 172.21.15.133:0/1719380676 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5098157eb0 con 0x7f509815db00 2024-04-25T10:10:14.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.111+0000 7f509e387640 1 -- 172.21.15.133:0/1719380676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f509806c620 con 0x7f509815db00 2024-04-25T10:10:14.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f5094ff9640 1 -- 172.21.15.133:0/1719380676 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17035 (secure 0 0 0) 0x7f5090016560 con 0x7f509815db00 2024-04-25T10:10:14.121 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:14.121 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":83,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:09:46.874553+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:29.693214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:30.421009+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":82,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 -- 172.21.15.133:0/1719380676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5084066f60 msgr2=0x7f5084069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5084066f60 0x7f5084069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5088000f70 tx=0x7f5088009040 comp rx=0 tx=0).stop 2024-04-25T10:10:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 -- 172.21.15.133:0/1719380676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f509815db00 msgr2=0x7f5098151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f509815db00 0x7f5098151f40 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f50900b0040 tx=0x7f50900196e0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 -- 172.21.15.133:0/1719380676 shutdown_connections 2024-04-25T10:10:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5084066f60 0x7f5084069420 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f509815db00 0x7f5098151f40 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50980d0770 0x7f5098157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 --2- 172.21.15.133:0/1719380676 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f50980b87e0 0x7f50981566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 -- 172.21.15.133:0/1719380676 >> 172.21.15.133:0/1719380676 conn(0x7f50980727c0 msgr2=0x7f50980d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 -- 172.21.15.133:0/1719380676 shutdown_connections 2024-04-25T10:10:14.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.115+0000 7f509e387640 1 -- 172.21.15.133:0/1719380676 wait complete. 2024-04-25T10:10:14.141 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:29.693214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:30.421009+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 82, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:14.141 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:14.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d9ce0640 1 Processor -- start 2024-04-25T10:10:14.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d9ce0640 1 -- start start 2024-04-25T10:10:14.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d9ce0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d415d9a0 0x7ff2d40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d9ce0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d40d24c0 con 0x7ff2d40ce600 2024-04-25T10:10:14.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d9ce0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d415ebd0 con 0x7ff2d40d2030 2024-04-25T10:10:14.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d9ce0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d415eeb0 con 0x7ff2d415d9a0 2024-04-25T10:10:14.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d2ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff2d40ce600 0x7ff2d40d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41394/0 (socket says 172.21.15.133:41394) 2024-04-25T10:10:14.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d2ffd640 1 -- 172.21.15.133:0/3684801773 learned_addr learned my addr 172.21.15.133:0/3684801773 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:14.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d37fe640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d415d9a0 0x7ff2d40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146815841 0 0) 0x7ff2d40d24c0 con 0x7ff2d40ce600 2024-04-25T10:10:14.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2bc003530 con 0x7ff2d40ce600 2024-04-25T10:10:14.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2835214518 0 0) 0x7ff2bc003530 con 0x7ff2d40ce600 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d415d9a0 msgr2=0x7ff2d40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d415d9a0 0x7ff2d40d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 >> v1:172.21.15.162:6789/0 conn(0x7ff2d40d2030 legacy=0x7ff2d415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d40d0330 con 0x7ff2d40ce600 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff2c8001680 con 0x7ff2d40ce600 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 >> v1:172.21.15.133:6789/0 conn(0x7ff2d40ce600 legacy=0x7ff2d40d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2bc0039f0 0x7ff2bc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2bc004810 0x7ff2bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d415d9a0 0x7ff2bc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d415eeb0 con 0x7ff2bc0039f0 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc00bb00 con 0x7ff2bc004810 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc00bd00 con 0x7ff2d415d9a0 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d37fe640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2bc0039f0 0x7ff2bc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d2ffd640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2bc004810 0x7ff2bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d3fff640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d415d9a0 0x7ff2bc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d37fe640 1 -- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d415d9a0 msgr2=0x7ff2bc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d37fe640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d415d9a0 0x7ff2bc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d37fe640 1 -- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2bc004810 msgr2=0x7ff2bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d37fe640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2bc004810 0x7ff2bc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d37fe640 1 -- 172.21.15.133:0/3684801773 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc003530 con 0x7ff2bc0039f0 2024-04-25T10:10:14.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.263+0000 7ff2d3fff640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d415d9a0 0x7ff2bc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:14.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.267+0000 7ff2d37fe640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2bc0039f0 0x7ff2bc003e10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009ae0 tx=0x7ff2c401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae110904888ca88a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.267+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40b2070 con 0x7ff2bc0039f0 2024-04-25T10:10:14.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.267+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c40a55b0 con 0x7ff2bc0039f0 2024-04-25T10:10:14.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.267+0000 7ff2d27fc640 1 -- 172.21.15.133:0/3684801773 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40096b0 con 0x7ff2bc0039f0 2024-04-25T10:10:14.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.267+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/3684801773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2bc0039f0 msgr2=0x7ff2bc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.267+0000 7ff2d9ce0640 1 --2- 172.21.15.133:0/3684801773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2bc0039f0 0x7ff2bc003e10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009ae0 tx=0x7ff2c401ca40 comp rx=0 tx=0).stop 2024-04-25T10:10:14.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.267+0000 7ff2d3fff640 1 -- 172.21.15.133:0/3684801773 reap_dead start 2024-04-25T10:10:14.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.267+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/3684801773 shutdown_connections 2024-04-25T10:10:14.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.267+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/3684801773 >> 172.21.15.133:0/3684801773 conn(0x7ff2d406bf80 msgr2=0x7ff2d406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.267+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/3684801773 shutdown_connections 2024-04-25T10:10:14.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.267+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/3684801773 wait complete. 2024-04-25T10:10:14.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d9ce0640 1 Processor -- start 2024-04-25T10:10:14.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d9ce0640 1 -- start start 2024-04-25T10:10:14.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d9ce0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2d414e920 0x7ff2d414ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d9ce0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d4159b90 0x7ff2d4159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d9ce0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2d4159540 0x7ff2d40b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d9ce0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40b9820 con 0x7ff2d4159540 2024-04-25T10:10:14.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d9ce0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40b96a0 con 0x7ff2d414e920 2024-04-25T10:10:14.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d9ce0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40cedf0 con 0x7ff2d4159b90 2024-04-25T10:10:14.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2d4159540 0x7ff2d40b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2d4159540 0x7ff2d40b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34300/0 (socket says 172.21.15.133:34300) 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d3fff640 1 -- 172.21.15.133:0/326918884 learned_addr learned my addr 172.21.15.133:0/326918884 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d37fe640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2d414e920 0x7ff2d414ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d2ffd640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d4159b90 0x7ff2d4159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d3fff640 1 -- 172.21.15.133:0/326918884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d4159b90 msgr2=0x7ff2d4159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d3fff640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d4159b90 0x7ff2d4159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d3fff640 1 -- 172.21.15.133:0/326918884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2d414e920 msgr2=0x7ff2d414ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d3fff640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2d414e920 0x7ff2d414ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d3fff640 1 -- 172.21.15.133:0/326918884 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d4157270 con 0x7ff2d4159540 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d2ffd640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d4159b90 0x7ff2d4159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d3fff640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2d4159540 0x7ff2d40b4080 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40d4750 tx=0x7ff2cc00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2bbfff640 1 -- 172.21.15.133:0/326918884 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc006d60 con 0x7ff2d4159540 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2bbfff640 1 -- 172.21.15.133:0/326918884 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc00de80 con 0x7ff2d4159540 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2bbfff640 1 -- 172.21.15.133:0/326918884 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc003bc0 con 0x7ff2d4159540 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/326918884 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d415a410 con 0x7ff2d4159540 2024-04-25T10:10:14.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/326918884 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d40c7320 con 0x7ff2d4159540 2024-04-25T10:10:14.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/326918884 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d4068e80 con 0x7ff2d4159540 2024-04-25T10:10:14.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2bbfff640 1 -- 172.21.15.133:0/326918884 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2cc01b020 con 0x7ff2d4159540 2024-04-25T10:10:14.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.271+0000 7ff2bbfff640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff2c0066f60 0x7ff2c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.275+0000 7ff2bbfff640 1 -- 172.21.15.133:0/326918884 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7ff2cc084d80 con 0x7ff2d4159540 2024-04-25T10:10:14.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.275+0000 7ff2d37fe640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff2c0066f60 0x7ff2c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.275+0000 7ff2d37fe640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff2c0066f60 0x7ff2c0069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40af000 tx=0x7ff2c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.279+0000 7ff2bbfff640 1 -- 172.21.15.133:0/326918884 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2cc055be0 con 0x7ff2d4159540 2024-04-25T10:10:14.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/326918884 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2d414f8d0 con 0x7ff2d4159540 2024-04-25T10:10:14.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2bbfff640 1 -- 172.21.15.133:0/326918884 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2cc0554e0 con 0x7ff2d4159540 2024-04-25T10:10:14.526 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:10:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/326918884 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff2c0066f60 msgr2=0x7ff2c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff2c0066f60 0x7ff2c0069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40af000 tx=0x7ff2c4008040 comp rx=0 tx=0).stop 2024-04-25T10:10:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/326918884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2d4159540 msgr2=0x7ff2d40b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2d4159540 0x7ff2d40b4080 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40d4750 tx=0x7ff2cc00db40 comp rx=0 tx=0).stop 2024-04-25T10:10:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/326918884 shutdown_connections 2024-04-25T10:10:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff2c0066f60 0x7ff2c0069420 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2d4159540 0x7ff2d40b4080 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2d4159b90 0x7ff2d4159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 --2- 172.21.15.133:0/326918884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2d414e920 0x7ff2d414ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/326918884 >> 172.21.15.133:0/326918884 conn(0x7ff2d406bf80 msgr2=0x7ff2d40b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/326918884 shutdown_connections 2024-04-25T10:10:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.439+0000 7ff2d9ce0640 1 -- 172.21.15.133:0/326918884 wait complete. 2024-04-25T10:10:14.530 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:14.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.655+0000 7f1d15467640 1 Processor -- start 2024-04-25T10:10:14.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.655+0000 7f1d15467640 1 -- start start 2024-04-25T10:10:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d15467640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1d100ce600 0x7f1d100cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d15467640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 0x7f1d100d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d15467640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1015ebd0 con 0x7f1d1015d9a0 2024-04-25T10:10:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d15467640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1015eeb0 con 0x7f1d100ce600 2024-04-25T10:10:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d15467640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100d0330 con 0x7f1d100d1920 2024-04-25T10:10:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d07fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1d100ce600 0x7f1d100cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d07fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1d100ce600 0x7f1d100cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34310/0 (socket says 172.21.15.133:34310) 2024-04-25T10:10:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d07fff640 1 -- 172.21.15.133:0/1083724794 learned_addr learned my addr 172.21.15.133:0/1083724794 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d0f7fe640 1 --2- 172.21.15.133:0/1083724794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 0x7f1d100d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d07fff640 1 -- 172.21.15.133:0/1083724794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 msgr2=0x7f1d100d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d07fff640 1 --2- 172.21.15.133:0/1083724794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 0x7f1d100d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d07fff640 1 -- 172.21.15.133:0/1083724794 >> v1:172.21.15.162:6789/0 conn(0x7f1d1015d9a0 legacy=0x7f1d100d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d07fff640 1 -- 172.21.15.133:0/1083724794 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d100d0610 con 0x7f1d100ce600 2024-04-25T10:10:14.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d0f7fe640 1 --2- 172.21.15.133:0/1083724794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 0x7f1d100d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:14.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d07fff640 1 --2- 172.21.15.133:0/1083724794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100ce600 0x7f1d100cea90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009870 tx=0x7f1cf801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3640e43ee3c6564 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d0e7fc640 1 -- 172.21.15.133:0/1083724794 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80b0070 con 0x7f1d100ce600 2024-04-25T10:10:14.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d0e7fc640 1 -- 172.21.15.133:0/1083724794 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf80ab600 con 0x7f1d100ce600 2024-04-25T10:10:14.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.659+0000 7f1d0e7fc640 1 -- 172.21.15.133:0/1083724794 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80ab9d0 con 0x7f1d100ce600 2024-04-25T10:10:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 -- 172.21.15.133:0/1083724794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100ce600 msgr2=0x7f1d100cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 --2- 172.21.15.133:0/1083724794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100ce600 0x7f1d100cea90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009870 tx=0x7f1cf801c490 comp rx=0 tx=0).stop 2024-04-25T10:10:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 -- 172.21.15.133:0/1083724794 shutdown_connections 2024-04-25T10:10:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 --2- 172.21.15.133:0/1083724794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 0x7f1d100d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 --2- 172.21.15.133:0/1083724794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100ce600 0x7f1d100cea90 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 -- 172.21.15.133:0/1083724794 >> 172.21.15.133:0/1083724794 conn(0x7f1d1006bf80 msgr2=0x7f1d1006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 -- 172.21.15.133:0/1083724794 shutdown_connections 2024-04-25T10:10:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 -- 172.21.15.133:0/1083724794 wait complete. 2024-04-25T10:10:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 Processor -- start 2024-04-25T10:10:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 -- start start 2024-04-25T10:10:14.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100ce600 0x7f1d100b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 0x7f1d10154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d1015d9a0 0x7f1d10157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1015e590 con 0x7f1d100ce600 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100d2540 con 0x7f1d1015d9a0 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1015e710 con 0x7f1d100d1920 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d0effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100ce600 0x7f1d100b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d0effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100ce600 0x7f1d100b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34322/0 (socket says 172.21.15.133:34322) 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d0effd640 1 -- 172.21.15.133:0/1781246350 learned_addr learned my addr 172.21.15.133:0/1781246350 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d07fff640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 0x7f1d10154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d0f7fe640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d1015d9a0 0x7f1d10157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d0effd640 1 -- 172.21.15.133:0/1781246350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 msgr2=0x7f1d10154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d0effd640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 0x7f1d10154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d0effd640 1 -- 172.21.15.133:0/1781246350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d1015d9a0 msgr2=0x7f1d10157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d0effd640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d1015d9a0 0x7f1d10157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d0effd640 1 -- 172.21.15.133:0/1781246350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d10159680 con 0x7f1d100ce600 2024-04-25T10:10:14.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d0f7fe640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d1015d9a0 0x7f1d10157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:14.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d07fff640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 0x7f1d10154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:14.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d0effd640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100ce600 0x7f1d100b4840 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000c450 tx=0x7f1d0000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d077fe640 1 -- 172.21.15.133:0/1781246350 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00012830 con 0x7f1d100ce600 2024-04-25T10:10:14.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d077fe640 1 -- 172.21.15.133:0/1781246350 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d00017070 con 0x7f1d100ce600 2024-04-25T10:10:14.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 -- 172.21.15.133:0/1781246350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d100b4d80 con 0x7f1d100ce600 2024-04-25T10:10:14.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d077fe640 1 -- 172.21.15.133:0/1781246350 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00012c30 con 0x7f1d100ce600 2024-04-25T10:10:14.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.663+0000 7f1d15467640 1 -- 172.21.15.133:0/1781246350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d10155d70 con 0x7f1d100ce600 2024-04-25T10:10:14.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.667+0000 7f1d15467640 1 -- 172.21.15.133:0/1781246350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d10155fb0 con 0x7f1d100ce600 2024-04-25T10:10:14.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.667+0000 7f1d077fe640 1 -- 172.21.15.133:0/1781246350 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d0001c460 con 0x7f1d100ce600 2024-04-25T10:10:14.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.667+0000 7f1d077fe640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf0066f60 0x7f1cf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.667+0000 7f1d077fe640 1 -- 172.21.15.133:0/1781246350 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f1d00084f90 con 0x7f1d100ce600 2024-04-25T10:10:14.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.667+0000 7f1d07fff640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf0066f60 0x7f1cf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.667+0000 7f1d07fff640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf0066f60 0x7f1cf0069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80ad000 tx=0x7f1cf8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.671+0000 7f1d077fe640 1 -- 172.21.15.133:0/1781246350 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d10159680 con 0x7f1d100ce600 2024-04-25T10:10:14.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.827+0000 7f1d15467640 1 -- 172.21.15.133:0/1781246350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d10068e80 con 0x7f1d100ce600 2024-04-25T10:10:14.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d077fe640 1 -- 172.21.15.133:0/1781246350 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d00055df0 con 0x7f1d100ce600 2024-04-25T10:10:14.835 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:10:14.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 -- 172.21.15.133:0/1781246350 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf0066f60 msgr2=0x7f1cf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf0066f60 0x7f1cf0069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80ad000 tx=0x7f1cf8008040 comp rx=0 tx=0).stop 2024-04-25T10:10:14.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 -- 172.21.15.133:0/1781246350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100ce600 msgr2=0x7f1d100b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100ce600 0x7f1d100b4840 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000c450 tx=0x7f1d0000c910 comp rx=0 tx=0).stop 2024-04-25T10:10:14.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 -- 172.21.15.133:0/1781246350 shutdown_connections 2024-04-25T10:10:14.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d1015d9a0 0x7f1d10157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d100d1920 0x7f1d10154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cf0066f60 0x7f1cf0069420 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 --2- 172.21.15.133:0/1781246350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100ce600 0x7f1d100b4840 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 -- 172.21.15.133:0/1781246350 >> 172.21.15.133:0/1781246350 conn(0x7f1d1006bf80 msgr2=0x7f1d1006c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 -- 172.21.15.133:0/1781246350 shutdown_connections 2024-04-25T10:10:14.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.831+0000 7f1d15467640 1 -- 172.21.15.133:0/1781246350 wait complete. 2024-04-25T10:10:14.855 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c1a9c8640 1 Processor -- start 2024-04-25T10:10:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c1a9c8640 1 -- start start 2024-04-25T10:10:14.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c1a9c8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1c14145170 0x7f1c1406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c1a9c8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c140ce2e0 con 0x7f1c1415ced0 2024-04-25T10:10:14.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c1a9c8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1406d4b0 con 0x7f1c1406e4d0 2024-04-25T10:10:14.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c1a9c8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1406db00 con 0x7f1c14145170 2024-04-25T10:10:14.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c18f65640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1c1415ced0 0x7f1c140b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41406/0 (socket says 172.21.15.133:41406) 2024-04-25T10:10:14.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c18f65640 1 -- 172.21.15.133:0/196550385 learned_addr learned my addr 172.21.15.133:0/196550385 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c13fff640 1 --2- 172.21.15.133:0/196550385 >> v2:172.21.15.162:3300/0 conn(0x7f1c14145170 0x7f1c1406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429567815 0 0) 0x7f1c140ce2e0 con 0x7f1c1415ced0 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bfc003530 con 0x7f1c1415ced0 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143641946 0 0) 0x7f1bfc003530 con 0x7f1c1415ced0 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 >> v2:172.21.15.162:3300/0 conn(0x7f1c14145170 msgr2=0x7f1c1406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 --2- 172.21.15.133:0/196550385 >> v2:172.21.15.162:3300/0 conn(0x7f1c14145170 0x7f1c1406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 >> v1:172.21.15.162:6789/0 conn(0x7f1c1406e4d0 legacy=0x7f1c140cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1406d790 con 0x7f1c1415ced0 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1c0c001a00 con 0x7f1c1415ced0 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 >> v1:172.21.15.133:6789/0 conn(0x7f1c1415ced0 legacy=0x7f1c140b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bfc0039c0 0x7f1bfc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bfc004810 0x7f1bfc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bfc0051d0 0x7f1bfc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1406db00 con 0x7f1bfc004810 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc00c070 con 0x7f1bfc0051d0 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc00c270 con 0x7f1bfc0039c0 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c137fe640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bfc004810 0x7f1bfc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c18f65640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bfc0051d0 0x7f1bfc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c13fff640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bfc0039c0 0x7f1bfc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c137fe640 1 -- 172.21.15.133:0/196550385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bfc0039c0 msgr2=0x7f1bfc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c137fe640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bfc0039c0 0x7f1bfc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c137fe640 1 -- 172.21.15.133:0/196550385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bfc0051d0 msgr2=0x7f1bfc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c137fe640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bfc0051d0 0x7f1bfc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c137fe640 1 -- 172.21.15.133:0/196550385 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc003530 con 0x7f1bfc004810 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c18f65640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bfc0051d0 0x7f1bfc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:14.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.975+0000 7f1c18f65640 1 -- 172.21.15.133:0/196550385 reap_dead start 2024-04-25T10:10:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c137fe640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bfc004810 0x7f1bfc004c90 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1c08001750 tx=0x7f1c0801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399b92c4c313fa56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c080013a0 con 0x7f1bfc004810 2024-04-25T10:10:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c080052f0 con 0x7f1bfc004810 2024-04-25T10:10:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c12ffd640 1 -- 172.21.15.133:0/196550385 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08003840 con 0x7f1bfc004810 2024-04-25T10:10:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/196550385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bfc004810 msgr2=0x7f1bfc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c1a9c8640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bfc004810 0x7f1bfc004c90 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1c08001750 tx=0x7f1c0801c3c0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/196550385 shutdown_connections 2024-04-25T10:10:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c1a9c8640 1 --2- 172.21.15.133:0/196550385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bfc004810 0x7f1bfc004c90 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/196550385 >> 172.21.15.133:0/196550385 conn(0x7f1c14066f70 msgr2=0x7f1c1406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/196550385 shutdown_connections 2024-04-25T10:10:14.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/196550385 wait complete. 2024-04-25T10:10:14.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c1a9c8640 1 Processor -- start 2024-04-25T10:10:14.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.979+0000 7f1c1a9c8640 1 -- start start 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c1a9c8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bfc004810 0x7f1c141e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c1a9c8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c14154890 0x7f1c14154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c1a9c8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c14151ae0 0x7f1c14151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c1a9c8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1406df90 con 0x7f1c14151ae0 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c1a9c8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c14145550 con 0x7f1bfc004810 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c1a9c8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1406e110 con 0x7f1c14154890 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c18f65640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c14151ae0 0x7f1c14151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c18f65640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c14151ae0 0x7f1c14151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34336/0 (socket says 172.21.15.133:34336) 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c18f65640 1 -- 172.21.15.133:0/970342417 learned_addr learned my addr 172.21.15.133:0/970342417 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c137fe640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c14154890 0x7f1c14154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c13fff640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bfc004810 0x7f1c141e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c18f65640 1 -- 172.21.15.133:0/970342417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c14154890 msgr2=0x7f1c14154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c18f65640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c14154890 0x7f1c14154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c18f65640 1 -- 172.21.15.133:0/970342417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bfc004810 msgr2=0x7f1c141e7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c18f65640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bfc004810 0x7f1c141e7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c18f65640 1 -- 172.21.15.133:0/970342417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c14155250 con 0x7f1c14151ae0 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c13fff640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bfc004810 0x7f1c141e7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c18f65640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c14151ae0 0x7f1c14151f60 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f1c1415d2b0 tx=0x7f1c0c0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c10ff9640 1 -- 172.21.15.133:0/970342417 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c01c070 con 0x7f1c14151ae0 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c10ff9640 1 -- 172.21.15.133:0/970342417 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c0137a0 con 0x7f1c14151ae0 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/970342417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c141524a0 con 0x7f1c14151ae0 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c10ff9640 1 -- 172.21.15.133:0/970342417 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c010040 con 0x7f1c14151ae0 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/970342417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c141554e0 con 0x7f1c14151ae0 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/970342417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c14068e80 con 0x7f1c14151ae0 2024-04-25T10:10:14.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c10ff9640 1 -- 172.21.15.133:0/970342417 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c0c01f070 con 0x7f1c14151ae0 2024-04-25T10:10:14.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c10ff9640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c00066f90 0x7f1c00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:14.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c10ff9640 1 -- 172.21.15.133:0/970342417 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f1c0c087900 con 0x7f1c14151ae0 2024-04-25T10:10:14.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c13fff640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c00066f90 0x7f1c00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:14.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.983+0000 7f1c13fff640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c00066f90 0x7f1c00069450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1c1415e640 tx=0x7f1c04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:14.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:14.987+0000 7f1c10ff9640 1 -- 172.21.15.133:0/970342417 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c0c058770 con 0x7f1c14151ae0 2024-04-25T10:10:15.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/970342417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c1415e390 con 0x7f1c14151ae0 2024-04-25T10:10:15.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c10ff9640 1 -- 172.21.15.133:0/970342417 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c0c058070 con 0x7f1c14151ae0 2024-04-25T10:10:15.148 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/970342417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c00066f90 msgr2=0x7f1c00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c00066f90 0x7f1c00069450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1c1415e640 tx=0x7f1c04008040 comp rx=0 tx=0).stop 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/970342417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c14151ae0 msgr2=0x7f1c14151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c14151ae0 0x7f1c14151f60 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f1c1415d2b0 tx=0x7f1c0c0119f0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/970342417 shutdown_connections 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c00066f90 0x7f1c00069450 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c14151ae0 0x7f1c14151f60 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c14154890 0x7f1c14154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 --2- 172.21.15.133:0/970342417 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bfc004810 0x7f1c141e7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/970342417 >> 172.21.15.133:0/970342417 conn(0x7f1c14066f70 msgr2=0x7f1c140b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/970342417 shutdown_connections 2024-04-25T10:10:15.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.143+0000 7f1c1a9c8640 1 -- 172.21.15.133:0/970342417 wait complete. 2024-04-25T10:10:15.167 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:15.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.267+0000 7f4962378640 1 Processor -- start 2024-04-25T10:10:15.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.267+0000 7f4962378640 1 -- start start 2024-04-25T10:10:15.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f4962378640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f495c0b1fa0 0x7f495c0d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f4962378640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495c0cf490 con 0x7f495c15b990 2024-04-25T10:10:15.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f4962378640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495c0cf770 con 0x7f495c15f7c0 2024-04-25T10:10:15.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f4962378640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495c0cfa50 con 0x7f495c0b1fa0 2024-04-25T10:10:15.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f495c15b990 0x7f495c0b1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41418/0 (socket says 172.21.15.133:41418) 2024-04-25T10:10:15.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495bfff640 1 -- 172.21.15.133:0/153317537 learned_addr learned my addr 172.21.15.133:0/153317537 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:15.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495b7fe640 1 --2- 172.21.15.133:0/153317537 >> v2:172.21.15.162:3300/0 conn(0x7f495c0b1fa0 0x7f495c0d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407298221 0 0) 0x7f495c0cf490 con 0x7f495c15b990 2024-04-25T10:10:15.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4944003530 con 0x7f495c15b990 2024-04-25T10:10:15.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3314682309 0 0) 0x7f4944003530 con 0x7f495c15b990 2024-04-25T10:10:15.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 >> v2:172.21.15.162:3300/0 conn(0x7f495c0b1fa0 msgr2=0x7f495c0d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:15.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 --2- 172.21.15.133:0/153317537 >> v2:172.21.15.162:3300/0 conn(0x7f495c0b1fa0 0x7f495c0d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 >> v1:172.21.15.162:6789/0 conn(0x7f495c15f7c0 legacy=0x7f495c15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c0cfd30 con 0x7f495c15b990 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f494c001680 con 0x7f495c15b990 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 >> v1:172.21.15.133:6789/0 conn(0x7f495c15b990 legacy=0x7f495c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 --2- 172.21.15.133:0/153317537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49440039c0 0x7f4944003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 --2- 172.21.15.133:0/153317537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4944004810 0x7f4944004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 --2- 172.21.15.133:0/153317537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49440051d0 0x7f494400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c0cfa50 con 0x7f4944004810 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494400c070 con 0x7f49440051d0 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494400c270 con 0x7f49440039c0 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495b7fe640 1 --2- 172.21.15.133:0/153317537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4944004810 0x7f4944004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f4960915640 1 --2- 172.21.15.133:0/153317537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49440051d0 0x7f494400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495b7fe640 1 -- 172.21.15.133:0/153317537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49440039c0 msgr2=0x7f4944003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495b7fe640 1 --2- 172.21.15.133:0/153317537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49440039c0 0x7f4944003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495b7fe640 1 -- 172.21.15.133:0/153317537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49440051d0 msgr2=0x7f494400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:15.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495b7fe640 1 --2- 172.21.15.133:0/153317537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49440051d0 0x7f494400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495b7fe640 1 -- 172.21.15.133:0/153317537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4944003530 con 0x7f4944004810 2024-04-25T10:10:15.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f4960915640 1 -- 172.21.15.133:0/153317537 reap_dead start 2024-04-25T10:10:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495b7fe640 1 --2- 172.21.15.133:0/153317537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4944004810 0x7f4944004c90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f4950009ae0 tx=0x7f495001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b3541237dd2ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500b2070 con 0x7f4944004810 2024-04-25T10:10:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49500a5660 con 0x7f4944004810 2024-04-25T10:10:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f495affd640 1 -- 172.21.15.133:0/153317537 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500096b0 con 0x7f4944004810 2024-04-25T10:10:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f4962378640 1 -- 172.21.15.133:0/153317537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4944004810 msgr2=0x7f4944004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f4962378640 1 --2- 172.21.15.133:0/153317537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4944004810 0x7f4944004c90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f4950009ae0 tx=0x7f495001ca70 comp rx=0 tx=0).stop 2024-04-25T10:10:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f4962378640 1 -- 172.21.15.133:0/153317537 shutdown_connections 2024-04-25T10:10:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f4962378640 1 --2- 172.21.15.133:0/153317537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4944004810 0x7f4944004c90 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.271+0000 7f4962378640 1 -- 172.21.15.133:0/153317537 >> 172.21.15.133:0/153317537 conn(0x7f495c09aa10 msgr2=0x7f495c08af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 -- 172.21.15.133:0/153317537 shutdown_connections 2024-04-25T10:10:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 -- 172.21.15.133:0/153317537 wait complete. 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 Processor -- start 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 -- start start 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4944004810 0x7f495c0c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495c0afd10 0x7f495c14c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f495c0c7630 0x7f495c0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c0d03d0 con 0x7f495c0c7630 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c0d0250 con 0x7f4944004810 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495c0d0550 con 0x7f495c0afd10 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4960915640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f495c0c7630 0x7f495c0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4960915640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f495c0c7630 0x7f495c0c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34350/0 (socket says 172.21.15.133:34350) 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4960915640 1 -- 172.21.15.133:0/782885392 learned_addr learned my addr 172.21.15.133:0/782885392 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f495b7fe640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495c0afd10 0x7f495c14c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f495bfff640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4944004810 0x7f495c0c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4960915640 1 -- 172.21.15.133:0/782885392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495c0afd10 msgr2=0x7f495c14c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4960915640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495c0afd10 0x7f495c14c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4960915640 1 -- 172.21.15.133:0/782885392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4944004810 msgr2=0x7f495c0c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4960915640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4944004810 0x7f495c0c5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4960915640 1 -- 172.21.15.133:0/782885392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c0c61a0 con 0x7f495c0c7630 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4960915640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f495c0c7630 0x7f495c0c7ab0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f495400d540 tx=0x7f495400da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4958ff9640 1 -- 172.21.15.133:0/782885392 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954018070 con 0x7f495c0c7630 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4958ff9640 1 -- 172.21.15.133:0/782885392 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4954003d90 con 0x7f495c0c7630 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 -- 172.21.15.133:0/782885392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495c0b02e0 con 0x7f495c0c7630 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4958ff9640 1 -- 172.21.15.133:0/782885392 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49540052b0 con 0x7f495c0c7630 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 -- 172.21.15.133:0/782885392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495c0b0880 con 0x7f495c0c7630 2024-04-25T10:10:15.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.275+0000 7f4962378640 1 -- 172.21.15.133:0/782885392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495c068ef0 con 0x7f495c0c7630 2024-04-25T10:10:15.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.279+0000 7f4958ff9640 1 -- 172.21.15.133:0/782885392 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f495401b070 con 0x7f495c0c7630 2024-04-25T10:10:15.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.279+0000 7f4958ff9640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4948066f60 0x7f4948069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.279+0000 7f4958ff9640 1 -- 172.21.15.133:0/782885392 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f49540848a0 con 0x7f495c0c7630 2024-04-25T10:10:15.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.279+0000 7f495bfff640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4948066f60 0x7f4948069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.279+0000 7f495bfff640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4948066f60 0x7f4948069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f494c001870 tx=0x7f494c00c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.283+0000 7f4958ff9640 1 -- 172.21.15.133:0/782885392 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4954055700 con 0x7f495c0c7630 2024-04-25T10:10:15.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.459+0000 7f4962378640 1 -- 172.21.15.133:0/782885392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f495c0b1360 con 0x7f495c0c7630 2024-04-25T10:10:15.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.459+0000 7f4958ff9640 1 -- 172.21.15.133:0/782885392 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f4954055000 con 0x7f495c0c7630 2024-04-25T10:10:15.466 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:15.467 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3522294955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3522294955},{"type":"v1","addr":"172.21.15.162:6833","nonce":3522294955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3865231260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3865231260},{"type":"v1","addr":"172.21.15.133:6836","nonce":3865231260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7437,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3935311931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3935311931},{"type":"v1","addr":"172.21.15.133:6837","nonce":3935311931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:07:31.434780+0000","modified":"2024-04-25T10:07:40.739361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/3192312240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3192312240},{"type":"v1","addr":"172.21.15.162:6835","nonce":3192312240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:15.467 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 50 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 -- 172.21.15.133:0/782885392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4948066f60 msgr2=0x7f4948069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4948066f60 0x7f4948069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f494c001870 tx=0x7f494c00c040 comp rx=0 tx=0).stop 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 -- 172.21.15.133:0/782885392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f495c0c7630 msgr2=0x7f495c0c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f495c0c7630 0x7f495c0c7ab0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f495400d540 tx=0x7f495400da00 comp rx=0 tx=0).stop 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 -- 172.21.15.133:0/782885392 shutdown_connections 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4948066f60 0x7f4948069420 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f495c0c7630 0x7f495c0c7ab0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495c0afd10 0x7f495c14c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 --2- 172.21.15.133:0/782885392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4944004810 0x7f495c0c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 -- 172.21.15.133:0/782885392 >> 172.21.15.133:0/782885392 conn(0x7f495c09aa10 msgr2=0x7f495c070620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 -- 172.21.15.133:0/782885392 shutdown_connections 2024-04-25T10:10:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.463+0000 7f4962378640 1 -- 172.21.15.133:0/782885392 wait complete. 2024-04-25T10:10:15.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:15.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c7b668640 1 Processor -- start 2024-04-25T10:10:15.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c7b668640 1 -- start start 2024-04-25T10:10:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c7b668640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c7415c9a0 0x7f4c740c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c7b668640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4c740ce330 0x7f4c740ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c7b668640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c740c61d0 con 0x7f4c740b1f70 2024-04-25T10:10:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c7b668640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c740cece0 con 0x7f4c740ce330 2024-04-25T10:10:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c7b668640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740c64b0 con 0x7f4c7415c9a0 2024-04-25T10:10:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c79c05640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4c740ce330 0x7f4c740ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c78c03640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4c740b1f70 0x7f4c7406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41420/0 (socket says 172.21.15.133:41420) 2024-04-25T10:10:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c78c03640 1 -- 172.21.15.133:0/288329210 learned_addr learned my addr 172.21.15.133:0/288329210 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c79404640 1 --2- 172.21.15.133:0/288329210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c7415c9a0 0x7f4c740c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c79c05640 1 -- 172.21.15.133:0/288329210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c7415c9a0 msgr2=0x7f4c740c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:15.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c79c05640 1 --2- 172.21.15.133:0/288329210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c7415c9a0 0x7f4c740c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c79c05640 1 -- 172.21.15.133:0/288329210 >> v1:172.21.15.133:6789/0 conn(0x7f4c740b1f70 legacy=0x7f4c7406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c6bfff640 1 -- 172.21.15.133:0/288329210 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340956351 0 0) 0x7f4c740c61d0 con 0x7f4c740b1f70 2024-04-25T10:10:15.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c79c05640 1 -- 172.21.15.133:0/288329210 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7406d4f0 con 0x7f4c740ce330 2024-04-25T10:10:15.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.587+0000 7f4c79404640 1 --2- 172.21.15.133:0/288329210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c7415c9a0 0x7f4c740c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:15.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c79c05640 1 --2- 172.21.15.133:0/288329210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c740ce330 0x7f4c740ce7a0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009d90 tx=0x7f4c6c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d514b0a52e921a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c6bfff640 1 -- 172.21.15.133:0/288329210 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c0b2070 con 0x7f4c740ce330 2024-04-25T10:10:15.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c6bfff640 1 -- 172.21.15.133:0/288329210 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c0a58e0 con 0x7f4c740ce330 2024-04-25T10:10:15.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c6bfff640 1 -- 172.21.15.133:0/288329210 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c0a5cb0 con 0x7f4c740ce330 2024-04-25T10:10:15.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 -- 172.21.15.133:0/288329210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c740ce330 msgr2=0x7f4c740ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 --2- 172.21.15.133:0/288329210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c740ce330 0x7f4c740ce7a0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009d90 tx=0x7f4c6c01c970 comp rx=0 tx=0).stop 2024-04-25T10:10:15.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 -- 172.21.15.133:0/288329210 shutdown_connections 2024-04-25T10:10:15.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 --2- 172.21.15.133:0/288329210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c740ce330 0x7f4c740ce7a0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 --2- 172.21.15.133:0/288329210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c7415c9a0 0x7f4c740c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 -- 172.21.15.133:0/288329210 >> 172.21.15.133:0/288329210 conn(0x7f4c74067000 msgr2=0x7f4c7406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:15.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 -- 172.21.15.133:0/288329210 shutdown_connections 2024-04-25T10:10:15.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 -- 172.21.15.133:0/288329210 wait complete. 2024-04-25T10:10:15.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 Processor -- start 2024-04-25T10:10:15.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 -- start start 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c740b1f70 0x7f4c7415a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c740ce330 0x7f4c741551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c7415c9a0 0x7f4c7414f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7415dad0 con 0x7f4c740ce330 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7415d950 con 0x7f4c740b1f70 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7415dc50 con 0x7f4c7415c9a0 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c78c03640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c740ce330 0x7f4c741551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c78c03640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c740ce330 0x7f4c741551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34362/0 (socket says 172.21.15.133:34362) 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c78c03640 1 -- 172.21.15.133:0/2940444894 learned_addr learned my addr 172.21.15.133:0/2940444894 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c78c03640 1 -- 172.21.15.133:0/2940444894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c7415c9a0 msgr2=0x7f4c7414f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c78c03640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c7415c9a0 0x7f4c7414f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c79404640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c740b1f70 0x7f4c7415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c78c03640 1 -- 172.21.15.133:0/2940444894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c740b1f70 msgr2=0x7f4c7415a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c78c03640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c740b1f70 0x7f4c7415a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c78c03640 1 -- 172.21.15.133:0/2940444894 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c74155c70 con 0x7f4c740ce330 2024-04-25T10:10:15.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c78c03640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c740ce330 0x7f4c741551f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400c480 tx=0x7f4c6400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c69ffb640 1 -- 172.21.15.133:0/2940444894 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64018070 con 0x7f4c740ce330 2024-04-25T10:10:15.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c69ffb640 1 -- 172.21.15.133:0/2940444894 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c64007500 con 0x7f4c740ce330 2024-04-25T10:10:15.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 -- 172.21.15.133:0/2940444894 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c74155f50 con 0x7f4c740ce330 2024-04-25T10:10:15.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c69ffb640 1 -- 172.21.15.133:0/2940444894 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64004a30 con 0x7f4c740ce330 2024-04-25T10:10:15.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.591+0000 7f4c7b668640 1 -- 172.21.15.133:0/2940444894 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c741567f0 con 0x7f4c740ce330 2024-04-25T10:10:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.595+0000 7f4c7b668640 1 -- 172.21.15.133:0/2940444894 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7406bc20 con 0x7f4c740ce330 2024-04-25T10:10:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.595+0000 7f4c69ffb640 1 -- 172.21.15.133:0/2940444894 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c6401b070 con 0x7f4c740ce330 2024-04-25T10:10:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.595+0000 7f4c69ffb640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4c58066f90 0x7f4c58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.595+0000 7f4c69ffb640 1 -- 172.21.15.133:0/2940444894 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f4c640844e0 con 0x7f4c740ce330 2024-04-25T10:10:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.595+0000 7f4c79404640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4c58066f90 0x7f4c58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.595+0000 7f4c79404640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4c58066f90 0x7f4c58069450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4c70006fd0 tx=0x7f4c70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.599+0000 7f4c69ffb640 1 -- 172.21.15.133:0/2940444894 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c640553d0 con 0x7f4c740ce330 2024-04-25T10:10:15.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.755+0000 7f4c7b668640 1 -- 172.21.15.133:0/2940444894 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c74068ef0 con 0x7f4c740ce330 2024-04-25T10:10:15.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c69ffb640 1 -- 172.21.15.133:0/2940444894 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17035 (secure 0 0 0) 0x7f4c64054cd0 con 0x7f4c740ce330 2024-04-25T10:10:15.764 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:15.765 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":83,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:09:46.874553+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:29.693214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:30.421009+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":82,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:15.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 -- 172.21.15.133:0/2940444894 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4c58066f90 msgr2=0x7f4c58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4c58066f90 0x7f4c58069450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4c70006fd0 tx=0x7f4c70008040 comp rx=0 tx=0).stop 2024-04-25T10:10:15.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 -- 172.21.15.133:0/2940444894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c740ce330 msgr2=0x7f4c741551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c740ce330 0x7f4c741551f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400c480 tx=0x7f4c6400c940 comp rx=0 tx=0).stop 2024-04-25T10:10:15.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 -- 172.21.15.133:0/2940444894 shutdown_connections 2024-04-25T10:10:15.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4c7415c9a0 0x7f4c7414f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4c58066f90 0x7f4c58069450 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c740ce330 0x7f4c741551f0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 --2- 172.21.15.133:0/2940444894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c740b1f70 0x7f4c7415a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 -- 172.21.15.133:0/2940444894 >> 172.21.15.133:0/2940444894 conn(0x7f4c74067000 msgr2=0x7f4c7415de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:15.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 -- 172.21.15.133:0/2940444894 shutdown_connections 2024-04-25T10:10:15.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.759+0000 7f4c7b668640 1 -- 172.21.15.133:0/2940444894 wait complete. 2024-04-25T10:10:15.782 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:10:15.783 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:15.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.883+0000 7f8997402640 1 Processor -- start 2024-04-25T10:10:15.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.883+0000 7f8997402640 1 -- start start 2024-04-25T10:10:15.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8997402640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015a5f0 0x7f89900c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8997402640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89900cc5a0 con 0x7f899015a9d0 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8997402640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89900cc880 con 0x7f899006eb70 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8997402640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89900ccb60 con 0x7f899015a5f0 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f899499d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f899015a9d0 0x7f899006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41424/0 (socket says 172.21.15.133:41424) 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f899499d640 1 -- 172.21.15.133:0/2648216076 learned_addr learned my addr 172.21.15.133:0/2648216076 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780492217 0 0) 0x7f89900cc5a0 con 0x7f899015a9d0 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f899519e640 1 --2- 172.21.15.133:0/2648216076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015a5f0 0x7f89900c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f897c003530 con 0x7f899015a9d0 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3780908550 0 0) 0x7f897c003530 con 0x7f899015a9d0 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015a5f0 msgr2=0x7f89900c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 --2- 172.21.15.133:0/2648216076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015a5f0 0x7f89900c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 >> v1:172.21.15.162:6789/0 conn(0x7f899006eb70 legacy=0x7f89900cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89900cce40 con 0x7f899015a9d0 2024-04-25T10:10:15.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f898c001680 con 0x7f899015a9d0 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 >> v1:172.21.15.133:6789/0 conn(0x7f899015a9d0 legacy=0x7f899006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 --2- 172.21.15.133:0/2648216076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015a5f0 0x7f897c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 --2- 172.21.15.133:0/2648216076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 0x7f897c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 --2- 172.21.15.133:0/2648216076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f897c004d50 0x7f897c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89900ccb60 con 0x7f897c003f70 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c00bab0 con 0x7f897c004d50 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c00bcb0 con 0x7f899015a5f0 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f899499d640 1 --2- 172.21.15.133:0/2648216076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 0x7f897c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f899519e640 1 --2- 172.21.15.133:0/2648216076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015a5f0 0x7f897c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f899499d640 1 -- 172.21.15.133:0/2648216076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015a5f0 msgr2=0x7f897c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f899499d640 1 --2- 172.21.15.133:0/2648216076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015a5f0 0x7f897c0039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f899499d640 1 -- 172.21.15.133:0/2648216076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f897c004d50 msgr2=0x7f897c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f899499d640 1 --2- 172.21.15.133:0/2648216076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f897c004d50 0x7f897c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f899499d640 1 -- 172.21.15.133:0/2648216076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c003530 con 0x7f897c003f70 2024-04-25T10:10:15.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f899499d640 1 --2- 172.21.15.133:0/2648216076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 0x7f897c004810 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f898c00c190 tx=0x7f898c01f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467426d04d70f6fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.887+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c007d00 con 0x7f897c003f70 2024-04-25T10:10:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c005ae0 con 0x7f897c003f70 2024-04-25T10:10:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8983fff640 1 -- 172.21.15.133:0/2648216076 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c007070 con 0x7f897c003f70 2024-04-25T10:10:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 -- 172.21.15.133:0/2648216076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 msgr2=0x7f897c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 --2- 172.21.15.133:0/2648216076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 0x7f897c004810 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f898c00c190 tx=0x7f898c01f390 comp rx=0 tx=0).stop 2024-04-25T10:10:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f899599f640 1 -- 172.21.15.133:0/2648216076 reap_dead start 2024-04-25T10:10:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 -- 172.21.15.133:0/2648216076 shutdown_connections 2024-04-25T10:10:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 -- 172.21.15.133:0/2648216076 >> 172.21.15.133:0/2648216076 conn(0x7f8990067000 msgr2=0x7f899006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 -- 172.21.15.133:0/2648216076 shutdown_connections 2024-04-25T10:10:15.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 -- 172.21.15.133:0/2648216076 wait complete. 2024-04-25T10:10:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 Processor -- start 2024-04-25T10:10:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 -- start start 2024-04-25T10:10:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 0x7f89901577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015ae00 0x7f899006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899015c3a0 0x7f899015c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89900c5630 con 0x7f897c003f70 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89900c54b0 con 0x7f899015c3a0 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89900cbe50 con 0x7f899015ae00 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f899519e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 0x7f89901577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f899519e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 0x7f89901577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34382/0 (socket says 172.21.15.133:34382) 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f899519e640 1 -- 172.21.15.133:0/3848276489 learned_addr learned my addr 172.21.15.133:0/3848276489 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f899499d640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015ae00 0x7f899006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f899519e640 1 -- 172.21.15.133:0/3848276489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015ae00 msgr2=0x7f899006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f899519e640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015ae00 0x7f899006d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f899519e640 1 -- 172.21.15.133:0/3848276489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899015c3a0 msgr2=0x7f899015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f899519e640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899015c3a0 0x7f899015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f899519e640 1 -- 172.21.15.133:0/3848276489 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8990157cf0 con 0x7f897c003f70 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f899519e640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 0x7f89901577b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8984010890 tx=0x7f8984010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8981ffb640 1 -- 172.21.15.133:0/3848276489 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898400f040 con 0x7f897c003f70 2024-04-25T10:10:15.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8981ffb640 1 -- 172.21.15.133:0/3848276489 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898400eb60 con 0x7f897c003f70 2024-04-25T10:10:15.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8981ffb640 1 -- 172.21.15.133:0/3848276489 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898401c400 con 0x7f897c003f70 2024-04-25T10:10:15.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 -- 172.21.15.133:0/3848276489 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899015b300 con 0x7f897c003f70 2024-04-25T10:10:15.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.891+0000 7f8997402640 1 -- 172.21.15.133:0/3848276489 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899015b8c0 con 0x7f897c003f70 2024-04-25T10:10:15.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.895+0000 7f8997402640 1 -- 172.21.15.133:0/3848276489 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899015bfe0 con 0x7f897c003f70 2024-04-25T10:10:15.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.895+0000 7f8981ffb640 1 -- 172.21.15.133:0/3848276489 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f898401c5a0 con 0x7f897c003f70 2024-04-25T10:10:15.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.895+0000 7f8981ffb640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8974066f60 0x7f8974069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:15.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.895+0000 7f8981ffb640 1 -- 172.21.15.133:0/3848276489 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f8984085800 con 0x7f897c003f70 2024-04-25T10:10:15.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.895+0000 7f899499d640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8974066f60 0x7f8974069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:15.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.895+0000 7f899499d640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8974066f60 0x7f8974069420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f898c0ae000 tx=0x7f89900cca50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:15.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:15.899+0000 7f8981ffb640 1 -- 172.21.15.133:0/3848276489 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8984018030 con 0x7f897c003f70 2024-04-25T10:10:16.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.083+0000 7f8997402640 1 -- 172.21.15.133:0/3848276489 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8990068ef0 con 0x7f897c003f70 2024-04-25T10:10:16.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.083+0000 7f8981ffb640 1 -- 172.21.15.133:0/3848276489 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f89840566f0 con 0x7f897c003f70 2024-04-25T10:10:16.090 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:16.090 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3522294955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3522294955},{"type":"v1","addr":"172.21.15.162:6833","nonce":3522294955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3865231260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3865231260},{"type":"v1","addr":"172.21.15.133:6836","nonce":3865231260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7437,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3935311931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3935311931},{"type":"v1","addr":"172.21.15.133:6837","nonce":3935311931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:07:31.434780+0000","modified":"2024-04-25T10:07:40.739361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/3192312240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3192312240},{"type":"v1","addr":"172.21.15.162:6835","nonce":3192312240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:16.090 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 50 2024-04-25T10:10:16.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 -- 172.21.15.133:0/3848276489 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8974066f60 msgr2=0x7f8974069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8974066f60 0x7f8974069420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f898c0ae000 tx=0x7f89900cca50 comp rx=0 tx=0).stop 2024-04-25T10:10:16.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 -- 172.21.15.133:0/3848276489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 msgr2=0x7f89901577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 0x7f89901577b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8984010890 tx=0x7f8984010d50 comp rx=0 tx=0).stop 2024-04-25T10:10:16.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 -- 172.21.15.133:0/3848276489 shutdown_connections 2024-04-25T10:10:16.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f899015c3a0 0x7f899015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f899015ae00 0x7f899006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8974066f60 0x7f8974069420 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 --2- 172.21.15.133:0/3848276489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f897c003f70 0x7f89901577b0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 -- 172.21.15.133:0/3848276489 >> 172.21.15.133:0/3848276489 conn(0x7f8990067000 msgr2=0x7f8990158950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:16.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 -- 172.21.15.133:0/3848276489 shutdown_connections 2024-04-25T10:10:16.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.087+0000 7f8997402640 1 -- 172.21.15.133:0/3848276489 wait complete. 2024-04-25T10:10:16.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.211+0000 7f6dbbf40640 1 Processor -- start 2024-04-25T10:10:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.211+0000 7f6dbbf40640 1 -- start start 2024-04-25T10:10:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6dbbf40640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 0x7f6db40b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6dbbf40640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6db40d0bd0 0x7f6db4161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6dbbf40640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db40ceb20 con 0x7f6db40d0770 2024-04-25T10:10:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6dbbf40640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db40b7740 con 0x7f6db40d0bd0 2024-04-25T10:10:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6dbbf40640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40b7a20 con 0x7f6db415db00 2024-04-25T10:10:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6dba4dd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6db40d0bd0 0x7f6db4161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6db94db640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6db40d0770 0x7f6db41614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50866/0 (socket says 172.21.15.133:50866) 2024-04-25T10:10:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6db94db640 1 -- 172.21.15.133:0/3389803171 learned_addr learned my addr 172.21.15.133:0/3389803171 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6db9cdc640 1 --2- 172.21.15.133:0/3389803171 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 0x7f6db40b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6dba4dd640 1 -- 172.21.15.133:0/3389803171 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 msgr2=0x7f6db40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6dba4dd640 1 --2- 172.21.15.133:0/3389803171 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 0x7f6db40b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6dba4dd640 1 -- 172.21.15.133:0/3389803171 >> v1:172.21.15.133:6789/0 conn(0x7f6db40d0770 legacy=0x7f6db41614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6dba4dd640 1 -- 172.21.15.133:0/3389803171 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db40b97f0 con 0x7f6db40d0bd0 2024-04-25T10:10:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6db8cda640 1 -- 172.21.15.133:0/3389803171 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329123452 0 0) 0x7f6db40ceb20 con 0x7f6db40d0770 2024-04-25T10:10:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6db9cdc640 1 --2- 172.21.15.133:0/3389803171 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 0x7f6db40b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:16.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6dba4dd640 1 --2- 172.21.15.133:0/3389803171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db40d0bd0 0x7f6db4161bd0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009d90 tx=0x7f6dac01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c540779d01db4534 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:16.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6db8cda640 1 -- 172.21.15.133:0/3389803171 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0b2070 con 0x7f6db40d0bd0 2024-04-25T10:10:16.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6db8cda640 1 -- 172.21.15.133:0/3389803171 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac0a5a00 con 0x7f6db40d0bd0 2024-04-25T10:10:16.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.215+0000 7f6db8cda640 1 -- 172.21.15.133:0/3389803171 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0a5dd0 con 0x7f6db40d0bd0 2024-04-25T10:10:16.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 -- 172.21.15.133:0/3389803171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db40d0bd0 msgr2=0x7f6db4161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 --2- 172.21.15.133:0/3389803171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db40d0bd0 0x7f6db4161bd0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009d90 tx=0x7f6dac01c940 comp rx=0 tx=0).stop 2024-04-25T10:10:16.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 -- 172.21.15.133:0/3389803171 shutdown_connections 2024-04-25T10:10:16.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 --2- 172.21.15.133:0/3389803171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db40d0bd0 0x7f6db4161bd0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 --2- 172.21.15.133:0/3389803171 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 0x7f6db40b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 -- 172.21.15.133:0/3389803171 >> 172.21.15.133:0/3389803171 conn(0x7f6db40727c0 msgr2=0x7f6db4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 -- 172.21.15.133:0/3389803171 shutdown_connections 2024-04-25T10:10:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 -- 172.21.15.133:0/3389803171 wait complete. 2024-04-25T10:10:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 Processor -- start 2024-04-25T10:10:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 -- start start 2024-04-25T10:10:16.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db40d0770 0x7f6db40d36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6db40d0bd0 0x7f6db415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 0x7f6db4156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40b7430 con 0x7f6db40d0770 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40d1060 con 0x7f6db40d0bd0 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40d18a0 con 0x7f6db415db00 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6db9cdc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db40d0770 0x7f6db40d36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6db9cdc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db40d0770 0x7f6db40d36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38864/0 (socket says 172.21.15.133:38864) 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6db9cdc640 1 -- 172.21.15.133:0/1862127060 learned_addr learned my addr 172.21.15.133:0/1862127060 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dba4dd640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 0x7f6db4156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6db94db640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6db40d0bd0 0x7f6db415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6db9cdc640 1 -- 172.21.15.133:0/1862127060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 msgr2=0x7f6db4156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6db9cdc640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 0x7f6db4156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6db9cdc640 1 -- 172.21.15.133:0/1862127060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6db40d0bd0 msgr2=0x7f6db415cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6db9cdc640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6db40d0bd0 0x7f6db415cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6db9cdc640 1 -- 172.21.15.133:0/1862127060 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db414edd0 con 0x7f6db40d0770 2024-04-25T10:10:16.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dba4dd640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 0x7f6db4156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:16.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6db9cdc640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db40d0770 0x7f6db40d36c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c7c0 tx=0x7f6da800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:16.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6da67fc640 1 -- 172.21.15.133:0/1862127060 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80079d0 con 0x7f6db40d0770 2024-04-25T10:10:16.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6da67fc640 1 -- 172.21.15.133:0/1862127060 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da8007b70 con 0x7f6db40d0770 2024-04-25T10:10:16.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 -- 172.21.15.133:0/1862127060 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db414f0b0 con 0x7f6db40d0770 2024-04-25T10:10:16.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6da67fc640 1 -- 172.21.15.133:0/1862127060 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80113f0 con 0x7f6db40d0770 2024-04-25T10:10:16.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.219+0000 7f6dbbf40640 1 -- 172.21.15.133:0/1862127060 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db40c89e0 con 0x7f6db40d0770 2024-04-25T10:10:16.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.223+0000 7f6dbbf40640 1 -- 172.21.15.133:0/1862127060 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db40c8530 con 0x7f6db40d0770 2024-04-25T10:10:16.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.223+0000 7f6da67fc640 1 -- 172.21.15.133:0/1862127060 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6da8011590 con 0x7f6db40d0770 2024-04-25T10:10:16.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.223+0000 7f6da67fc640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d98066f90 0x7f6d98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.223+0000 7f6db94db640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d98066f90 0x7f6d98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.223+0000 7f6da67fc640 1 -- 172.21.15.133:0/1862127060 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f6da80852a0 con 0x7f6db40d0770 2024-04-25T10:10:16.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.223+0000 7f6db94db640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d98066f90 0x7f6d98069450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6db41593c0 tx=0x7f6db0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:16.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.227+0000 7f6da67fc640 1 -- 172.21.15.133:0/1862127060 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6da8015020 con 0x7f6db40d0770 2024-04-25T10:10:16.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.411+0000 7f6dbbf40640 1 -- 172.21.15.133:0/1862127060 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6db40b2750 con 0x7f6db40d0770 2024-04-25T10:10:16.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.411+0000 7f6da67fc640 1 -- 172.21.15.133:0/1862127060 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f6da8056190 con 0x7f6db40d0770 2024-04-25T10:10:16.418 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:16.418 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3522294955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3522294955},{"type":"v1","addr":"172.21.15.162:6833","nonce":3522294955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3865231260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3865231260},{"type":"v1","addr":"172.21.15.133:6836","nonce":3865231260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7437,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3935311931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3935311931},{"type":"v1","addr":"172.21.15.133:6837","nonce":3935311931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:07:31.434780+0000","modified":"2024-04-25T10:07:40.739361+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/3192312240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3192312240},{"type":"v1","addr":"172.21.15.162:6835","nonce":3192312240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:10:16.418 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 50 2024-04-25T10:10:16.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 -- 172.21.15.133:0/1862127060 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d98066f90 msgr2=0x7f6d98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d98066f90 0x7f6d98069450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6db41593c0 tx=0x7f6db0009040 comp rx=0 tx=0).stop 2024-04-25T10:10:16.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 -- 172.21.15.133:0/1862127060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db40d0770 msgr2=0x7f6db40d36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db40d0770 0x7f6db40d36c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c7c0 tx=0x7f6da800cc80 comp rx=0 tx=0).stop 2024-04-25T10:10:16.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 -- 172.21.15.133:0/1862127060 shutdown_connections 2024-04-25T10:10:16.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6db415db00 0x7f6db4156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6db40d0bd0 0x7f6db415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6d98066f90 0x7f6d98069450 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 --2- 172.21.15.133:0/1862127060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db40d0770 0x7f6db40d36c0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 -- 172.21.15.133:0/1862127060 >> 172.21.15.133:0/1862127060 conn(0x7f6db40727c0 msgr2=0x7f6db40b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:16.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 -- 172.21.15.133:0/1862127060 shutdown_connections 2024-04-25T10:10:16.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.415+0000 7f6dbbf40640 1 -- 172.21.15.133:0/1862127060 wait complete. 2024-04-25T10:10:16.435 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:16.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f7a27b640 1 Processor -- start 2024-04-25T10:10:16.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f7a27b640 1 -- start start 2024-04-25T10:10:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f7a27b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9f7415c9a0 0x7f9f740c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f7a27b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9f740b1f70 0x7f9f7406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f7a27b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f740ce330 0x7f9f740ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f7a27b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f740cecc0 con 0x7f9f7415c9a0 2024-04-25T10:10:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f7a27b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f740c61d0 con 0x7f9f740b1f70 2024-04-25T10:10:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f7a27b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740c64b0 con 0x7f9f740ce330 2024-04-25T10:10:16.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f73fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9f7415c9a0 0x7f9f740c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f73fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9f7415c9a0 0x7f9f740c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38870/0 (socket says 172.21.15.133:38870) 2024-04-25T10:10:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f73fff640 1 -- 172.21.15.133:0/1708620618 learned_addr learned my addr 172.21.15.133:0/1708620618 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f78818640 1 --2- 172.21.15.133:0/1708620618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f740ce330 0x7f9f740ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f737fe640 1 --2- 172.21.15.133:0/1708620618 >> v2:172.21.15.162:3300/0 conn(0x7f9f740b1f70 0x7f9f7406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f73fff640 1 -- 172.21.15.133:0/1708620618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f740ce330 msgr2=0x7f9f740ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f73fff640 1 --2- 172.21.15.133:0/1708620618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f740ce330 0x7f9f740ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f73fff640 1 -- 172.21.15.133:0/1708620618 >> v2:172.21.15.162:3300/0 conn(0x7f9f740b1f70 msgr2=0x7f9f7406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f73fff640 1 --2- 172.21.15.133:0/1708620618 >> v2:172.21.15.162:3300/0 conn(0x7f9f740b1f70 0x7f9f7406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.551+0000 7f9f73fff640 1 -- 172.21.15.133:0/1708620618 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7406d4f0 con 0x7f9f7415c9a0 2024-04-25T10:10:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f73fff640 1 --2- 172.21.15.133:0/1708620618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7415c9a0 0x7f9f740c5980 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9f74072670 tx=0x7f9f6401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a0ad0358fbd9e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f72ffd640 1 -- 172.21.15.133:0/1708620618 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f640b1070 con 0x7f9f7415c9a0 2024-04-25T10:10:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f72ffd640 1 -- 172.21.15.133:0/1708620618 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f640ac620 con 0x7f9f7415c9a0 2024-04-25T10:10:16.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f72ffd640 1 -- 172.21.15.133:0/1708620618 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f640ac9f0 con 0x7f9f7415c9a0 2024-04-25T10:10:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 -- 172.21.15.133:0/1708620618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7415c9a0 msgr2=0x7f9f740c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 --2- 172.21.15.133:0/1708620618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7415c9a0 0x7f9f740c5980 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9f74072670 tx=0x7f9f6401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 -- 172.21.15.133:0/1708620618 shutdown_connections 2024-04-25T10:10:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 --2- 172.21.15.133:0/1708620618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f740ce330 0x7f9f740ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 --2- 172.21.15.133:0/1708620618 >> v2:172.21.15.162:3300/0 conn(0x7f9f740b1f70 0x7f9f7406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 --2- 172.21.15.133:0/1708620618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7415c9a0 0x7f9f740c5980 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 -- 172.21.15.133:0/1708620618 >> 172.21.15.133:0/1708620618 conn(0x7f9f74067000 msgr2=0x7f9f7406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 -- 172.21.15.133:0/1708620618 shutdown_connections 2024-04-25T10:10:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 -- 172.21.15.133:0/1708620618 wait complete. 2024-04-25T10:10:16.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 Processor -- start 2024-04-25T10:10:16.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 -- start start 2024-04-25T10:10:16.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f740b1f70 0x7f9f74163910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f740ce330 0x7f9f7414f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7415c9a0 0x7f9f74157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7415cf00 con 0x7f9f7415c9a0 2024-04-25T10:10:16.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7415cd80 con 0x7f9f740ce330 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f7a27b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7415d950 con 0x7f9f740b1f70 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f78818640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7415c9a0 0x7f9f74157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f78818640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7415c9a0 0x7f9f74157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38878/0 (socket says 172.21.15.133:38878) 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f78818640 1 -- 172.21.15.133:0/4283139568 learned_addr learned my addr 172.21.15.133:0/4283139568 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f737fe640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f740ce330 0x7f9f7414f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f73fff640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f740b1f70 0x7f9f74163910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f78818640 1 -- 172.21.15.133:0/4283139568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f740b1f70 msgr2=0x7f9f74163910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f78818640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f740b1f70 0x7f9f74163910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f78818640 1 -- 172.21.15.133:0/4283139568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f740ce330 msgr2=0x7f9f7414f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f78818640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f740ce330 0x7f9f7414f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f78818640 1 -- 172.21.15.133:0/4283139568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f74155e90 con 0x7f9f7415c9a0 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.555+0000 7f9f73fff640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f740b1f70 0x7f9f74163910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f78818640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7415c9a0 0x7f9f74157cd0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c007e40 tx=0x7f9f6c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f70ff9640 1 -- 172.21.15.133:0/4283139568 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c014620 con 0x7f9f7415c9a0 2024-04-25T10:10:16.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f70ff9640 1 -- 172.21.15.133:0/4283139568 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c019070 con 0x7f9f7415c9a0 2024-04-25T10:10:16.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f7a27b640 1 -- 172.21.15.133:0/4283139568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f740cdb00 con 0x7f9f7415c9a0 2024-04-25T10:10:16.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f70ff9640 1 -- 172.21.15.133:0/4283139568 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c014a20 con 0x7f9f7415c9a0 2024-04-25T10:10:16.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f7a27b640 1 -- 172.21.15.133:0/4283139568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f740ce060 con 0x7f9f7415c9a0 2024-04-25T10:10:16.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f7a27b640 1 -- 172.21.15.133:0/4283139568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7406bc20 con 0x7f9f7415c9a0 2024-04-25T10:10:16.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f70ff9640 1 -- 172.21.15.133:0/4283139568 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f6c014bc0 con 0x7f9f7415c9a0 2024-04-25T10:10:16.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f70ff9640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f60066f60 0x7f9f60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f70ff9640 1 -- 172.21.15.133:0/4283139568 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f9f6c0859e0 con 0x7f9f7415c9a0 2024-04-25T10:10:16.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f73fff640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f60066f60 0x7f9f60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.559+0000 7f9f73fff640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f60066f60 0x7f9f60069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9f640ae000 tx=0x7f9f64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:16.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.563+0000 7f9f70ff9640 1 -- 172.21.15.133:0/4283139568 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f740cdb00 con 0x7f9f7415c9a0 2024-04-25T10:10:16.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.711+0000 7f9f7a27b640 1 -- 172.21.15.133:0/4283139568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f74153fb0 con 0x7f9f7415c9a0 2024-04-25T10:10:16.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.711+0000 7f9f70ff9640 1 -- 172.21.15.133:0/4283139568 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17035 (secure 0 0 0) 0x7f9f6c0568d0 con 0x7f9f7415c9a0 2024-04-25T10:10:16.718 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:16.719 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":83,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:09:46.874553+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:29.693214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:30.421009+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":82,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 -- 172.21.15.133:0/4283139568 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f60066f60 msgr2=0x7f9f60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f60066f60 0x7f9f60069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9f640ae000 tx=0x7f9f64008040 comp rx=0 tx=0).stop 2024-04-25T10:10:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 -- 172.21.15.133:0/4283139568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7415c9a0 msgr2=0x7f9f74157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7415c9a0 0x7f9f74157cd0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c007e40 tx=0x7f9f6c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 -- 172.21.15.133:0/4283139568 shutdown_connections 2024-04-25T10:10:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f60066f60 0x7f9f60069420 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7415c9a0 0x7f9f74157cd0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f740ce330 0x7f9f7414f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 --2- 172.21.15.133:0/4283139568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f740b1f70 0x7f9f74163910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 -- 172.21.15.133:0/4283139568 >> 172.21.15.133:0/4283139568 conn(0x7f9f74067000 msgr2=0x7f9f740b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:16.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 -- 172.21.15.133:0/4283139568 shutdown_connections 2024-04-25T10:10:16.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.715+0000 7f9f7a27b640 1 -- 172.21.15.133:0/4283139568 wait complete. 2024-04-25T10:10:16.737 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:10:16.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91dd5dc640 1 Processor -- start 2024-04-25T10:10:16.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91dd5dc640 1 -- start start 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91dd5dc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91d8145170 0x7f91d806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91dd5dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91d806e4d0 0x7f91d80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91dd5dc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91d80cdd60 0x7f91d815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91dd5dc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d806db00 con 0x7f91d80cdd60 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91dd5dc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d8145550 con 0x7f91d8145170 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91dd5dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80ce2e0 con 0x7f91d806e4d0 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91d77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91d80cdd60 0x7f91d815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91d77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91d80cdd60 0x7f91d815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38884/0 (socket says 172.21.15.133:38884) 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91d77fe640 1 -- 172.21.15.133:0/508116187 learned_addr learned my addr 172.21.15.133:0/508116187 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91d67fc640 1 --2- 172.21.15.133:0/508116187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91d806e4d0 0x7f91d80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91d6ffd640 1 --2- 172.21.15.133:0/508116187 >> v2:172.21.15.162:3300/0 conn(0x7f91d8145170 0x7f91d806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91d77fe640 1 -- 172.21.15.133:0/508116187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91d806e4d0 msgr2=0x7f91d80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:16.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91d77fe640 1 --2- 172.21.15.133:0/508116187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91d806e4d0 0x7f91d80cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91d77fe640 1 -- 172.21.15.133:0/508116187 >> v2:172.21.15.162:3300/0 conn(0x7f91d8145170 msgr2=0x7f91d806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:16.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91d77fe640 1 --2- 172.21.15.133:0/508116187 >> v2:172.21.15.162:3300/0 conn(0x7f91d8145170 0x7f91d806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91d77fe640 1 -- 172.21.15.133:0/508116187 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d806d4b0 con 0x7f91d80cdd60 2024-04-25T10:10:16.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.839+0000 7f91d6ffd640 1 --2- 172.21.15.133:0/508116187 >> v2:172.21.15.162:3300/0 conn(0x7f91d8145170 0x7f91d806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:16.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91d77fe640 1 --2- 172.21.15.133:0/508116187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80cdd60 0x7f91d815ced0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f91d806b790 tx=0x7f91d001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac8f92ad599c707 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:16.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91d5ffb640 1 -- 172.21.15.133:0/508116187 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00a5670 con 0x7f91d80cdd60 2024-04-25T10:10:16.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91d5ffb640 1 -- 172.21.15.133:0/508116187 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d00a5810 con 0x7f91d80cdd60 2024-04-25T10:10:16.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91d5ffb640 1 -- 172.21.15.133:0/508116187 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00a5be0 con 0x7f91d80cdd60 2024-04-25T10:10:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91dd5dc640 1 -- 172.21.15.133:0/508116187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80cdd60 msgr2=0x7f91d815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91dd5dc640 1 --2- 172.21.15.133:0/508116187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80cdd60 0x7f91d815ced0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f91d806b790 tx=0x7f91d001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91dd5dc640 1 -- 172.21.15.133:0/508116187 shutdown_connections 2024-04-25T10:10:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91dd5dc640 1 --2- 172.21.15.133:0/508116187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80cdd60 0x7f91d815ced0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91dd5dc640 1 --2- 172.21.15.133:0/508116187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91d806e4d0 0x7f91d80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91dd5dc640 1 --2- 172.21.15.133:0/508116187 >> v2:172.21.15.162:3300/0 conn(0x7f91d8145170 0x7f91d806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91dd5dc640 1 -- 172.21.15.133:0/508116187 >> 172.21.15.133:0/508116187 conn(0x7f91d8066f70 msgr2=0x7f91d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91dd5dc640 1 -- 172.21.15.133:0/508116187 shutdown_connections 2024-04-25T10:10:16.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91dd5dc640 1 -- 172.21.15.133:0/508116187 wait complete. 2024-04-25T10:10:16.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91dd5dc640 1 Processor -- start 2024-04-25T10:10:16.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.843+0000 7f91dd5dc640 1 -- start start 2024-04-25T10:10:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91dd5dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91d806e4d0 0x7f91d80ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91dd5dc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91d80cdd60 0x7f91d80a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91dd5dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d8145170 0x7f91d80a9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91dd5dc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d815f690 con 0x7f91d8145170 2024-04-25T10:10:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91dd5dc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d815f510 con 0x7f91d80cdd60 2024-04-25T10:10:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91dd5dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d815f810 con 0x7f91d806e4d0 2024-04-25T10:10:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d8145170 0x7f91d80a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d8145170 0x7f91d80a9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38898/0 (socket says 172.21.15.133:38898) 2024-04-25T10:10:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d77fe640 1 -- 172.21.15.133:0/3510262886 learned_addr learned my addr 172.21.15.133:0/3510262886 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:16.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d6ffd640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91d806e4d0 0x7f91d80ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d67fc640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91d80cdd60 0x7f91d80a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d77fe640 1 -- 172.21.15.133:0/3510262886 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91d806e4d0 msgr2=0x7f91d80ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d77fe640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91d806e4d0 0x7f91d80ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d77fe640 1 -- 172.21.15.133:0/3510262886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91d80cdd60 msgr2=0x7f91d80a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d77fe640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91d80cdd60 0x7f91d80a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d77fe640 1 -- 172.21.15.133:0/3510262886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d80ac160 con 0x7f91d8145170 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d67fc640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91d80cdd60 0x7f91d80a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d77fe640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d8145170 0x7f91d80a9140 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f91d00af040 tx=0x7f91d0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91bf7fe640 1 -- 172.21.15.133:0/3510262886 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0009510 con 0x7f91d8145170 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91bf7fe640 1 -- 172.21.15.133:0/3510262886 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d00be070 con 0x7f91d8145170 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91dd5dc640 1 -- 172.21.15.133:0/3510262886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d80ac440 con 0x7f91d8145170 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91bf7fe640 1 -- 172.21.15.133:0/3510262886 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00b88b0 con 0x7f91d8145170 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91dd5dc640 1 -- 172.21.15.133:0/3510262886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d80aca00 con 0x7f91d8145170 2024-04-25T10:10:16.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91dd5dc640 1 -- 172.21.15.133:0/3510262886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d8068e80 con 0x7f91d8145170 2024-04-25T10:10:16.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91bf7fe640 1 -- 172.21.15.133:0/3510262886 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91d00b8a50 con 0x7f91d8145170 2024-04-25T10:10:16.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91bf7fe640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91c4066f60 0x7f91c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:16.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d6ffd640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91c4066f60 0x7f91c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:16.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91bf7fe640 1 -- 172.21.15.133:0/3510262886 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f91d0120510 con 0x7f91d8145170 2024-04-25T10:10:16.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.847+0000 7f91d6ffd640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91c4066f60 0x7f91c4069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006fd0 tx=0x7f91c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:16.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:16.851+0000 7f91bf7fe640 1 -- 172.21.15.133:0/3510262886 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91d00165e0 con 0x7f91d8145170 2024-04-25T10:10:17.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.042+0000 7f91dd5dc640 1 -- 172.21.15.133:0/3510262886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f91d8142100 con 0x7f91d8145170 2024-04-25T10:10:17.051 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:10:17.042+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:10:17.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.046+0000 7f91bf7fe640 1 -- 172.21.15.133:0/3510262886 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7f91d00f1400 con 0x7f91d8145170 2024-04-25T10:10:17.053 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:10:17.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 -- 172.21.15.133:0/3510262886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91c4066f60 msgr2=0x7f91c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:17.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91c4066f60 0x7f91c4069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006fd0 tx=0x7f91c8008040 comp rx=0 tx=0).stop 2024-04-25T10:10:17.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 -- 172.21.15.133:0/3510262886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d8145170 msgr2=0x7f91d80a9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:17.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d8145170 0x7f91d80a9140 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f91d00af040 tx=0x7f91d0019790 comp rx=0 tx=0).stop 2024-04-25T10:10:17.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 -- 172.21.15.133:0/3510262886 shutdown_connections 2024-04-25T10:10:17.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91c4066f60 0x7f91c4069420 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:17.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d8145170 0x7f91d80a9140 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:17.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91d80cdd60 0x7f91d80a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:17.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 --2- 172.21.15.133:0/3510262886 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91d806e4d0 0x7f91d80ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:17.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 -- 172.21.15.133:0/3510262886 >> 172.21.15.133:0/3510262886 conn(0x7f91d8066f70 msgr2=0x7f91d8140aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:17.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 -- 172.21.15.133:0/3510262886 shutdown_connections 2024-04-25T10:10:17.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.050+0000 7f91dd5dc640 1 -- 172.21.15.133:0/3510262886 wait complete. 2024-04-25T10:10:17.074 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:10:17.150 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:10:17.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f784bc75640 1 Processor -- start 2024-04-25T10:10:17.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f784bc75640 1 -- start start 2024-04-25T10:10:17.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f784bc75640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f784415f020 0x7f78440cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:17.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f784bc75640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78440d21c0 0x7f78440cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:17.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f784bc75640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78440ccdf0 con 0x7f78440cfcf0 2024-04-25T10:10:17.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f784bc75640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78440cd690 con 0x7f784415f020 2024-04-25T10:10:17.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f784bc75640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78440cd0d0 con 0x7f78440d21c0 2024-04-25T10:10:17.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f7849a11640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f784415f020 0x7f78440cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:17.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f7849a11640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f784415f020 0x7f78440cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38912/0 (socket says 172.21.15.133:38912) 2024-04-25T10:10:17.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f7849a11640 1 -- 172.21.15.133:0/254209590 learned_addr learned my addr 172.21.15.133:0/254209590 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:17.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f7849210640 1 --2- 172.21.15.133:0/254209590 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78440d21c0 0x7f78440cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:17.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f7849a11640 1 -- 172.21.15.133:0/254209590 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78440d21c0 msgr2=0x7f78440cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:17.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f7849a11640 1 --2- 172.21.15.133:0/254209590 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78440d21c0 0x7f78440cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:17.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f7849a11640 1 -- 172.21.15.133:0/254209590 >> v1:172.21.15.162:6789/0 conn(0x7f78440cfcf0 legacy=0x7f78440cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:17.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.186+0000 7f7849a11640 1 -- 172.21.15.133:0/254209590 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78440cd3b0 con 0x7f784415f020 2024-04-25T10:10:17.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f7849a11640 1 --2- 172.21.15.133:0/254209590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f784415f020 0x7f78440cf270 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7840009870 tx=0x7f784001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d4a90c49fa9f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:17.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f7848a0f640 1 -- 172.21.15.133:0/254209590 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400b2070 con 0x7f784415f020 2024-04-25T10:10:17.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f7848a0f640 1 -- 172.21.15.133:0/254209590 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78400a9070 con 0x7f784415f020 2024-04-25T10:10:17.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f7848a0f640 1 -- 172.21.15.133:0/254209590 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400ad880 con 0x7f784415f020 2024-04-25T10:10:17.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f784bc75640 1 -- 172.21.15.133:0/254209590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f784415f020 msgr2=0x7f78440cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:17.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f784bc75640 1 --2- 172.21.15.133:0/254209590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f784415f020 0x7f78440cf270 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7840009870 tx=0x7f784001c560 comp rx=0 tx=0).stop 2024-04-25T10:10:17.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f784bc75640 1 -- 172.21.15.133:0/254209590 shutdown_connections 2024-04-25T10:10:17.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f784bc75640 1 --2- 172.21.15.133:0/254209590 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78440d21c0 0x7f78440cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:17.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f784bc75640 1 --2- 172.21.15.133:0/254209590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f784415f020 0x7f78440cf270 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:17.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f784bc75640 1 -- 172.21.15.133:0/254209590 >> 172.21.15.133:0/254209590 conn(0x7f78440d16a0 msgr2=0x7f78440d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:17.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f784bc75640 1 -- 172.21.15.133:0/254209590 shutdown_connections 2024-04-25T10:10:17.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f784bc75640 1 -- 172.21.15.133:0/254209590 wait complete. 2024-04-25T10:10:17.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f784bc75640 1 Processor -- start 2024-04-25T10:10:17.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.190+0000 7f784bc75640 1 -- start start 2024-04-25T10:10:17.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784bc75640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78440cfcf0 0x7f7844149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:17.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784bc75640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78440d21c0 0x7f78440c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:17.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784bc75640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f784415f020 0x7f784414a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:17.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784bc75640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78441688f0 con 0x7f784415f020 2024-04-25T10:10:17.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784bc75640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7844168770 con 0x7f78440d21c0 2024-04-25T10:10:17.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784bc75640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7844168a70 con 0x7f78440cfcf0 2024-04-25T10:10:17.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784a212640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f784415f020 0x7f784414a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:17.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784a212640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f784415f020 0x7f784414a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38914/0 (socket says 172.21.15.133:38914) 2024-04-25T10:10:17.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784a212640 1 -- 172.21.15.133:0/2424821950 learned_addr learned my addr 172.21.15.133:0/2424821950 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f7849a11640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78440cfcf0 0x7f7844149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f7849210640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78440d21c0 0x7f78440c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784a212640 1 -- 172.21.15.133:0/2424821950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78440cfcf0 msgr2=0x7f7844149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784a212640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78440cfcf0 0x7f7844149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784a212640 1 -- 172.21.15.133:0/2424821950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78440d21c0 msgr2=0x7f78440c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784a212640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78440d21c0 0x7f78440c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784a212640 1 -- 172.21.15.133:0/2424821950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784414f5c0 con 0x7f784415f020 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784a212640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f784415f020 0x7f784414a490 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c730 tx=0x7f783c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f783a7fc640 1 -- 172.21.15.133:0/2424821950 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c003b80 con 0x7f784415f020 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f783a7fc640 1 -- 172.21.15.133:0/2424821950 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783c003d20 con 0x7f784415f020 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784bc75640 1 -- 172.21.15.133:0/2424821950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784414c6c0 con 0x7f784415f020 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f783a7fc640 1 -- 172.21.15.133:0/2424821950 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c005310 con 0x7f784415f020 2024-04-25T10:10:17.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784bc75640 1 -- 172.21.15.133:0/2424821950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784414bc30 con 0x7f784415f020 2024-04-25T10:10:17.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f784bc75640 1 -- 172.21.15.133:0/2424821950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7844067610 con 0x7f784415f020 2024-04-25T10:10:17.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f783a7fc640 1 -- 172.21.15.133:0/2424821950 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f783c0056c0 con 0x7f784415f020 2024-04-25T10:10:17.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f783a7fc640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7828066f90 0x7f7828069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:17.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f783a7fc640 1 -- 172.21.15.133:0/2424821950 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f783c085080 con 0x7f784415f020 2024-04-25T10:10:17.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f7849a11640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7828066f90 0x7f7828069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:17.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.194+0000 7f7849a11640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7828066f90 0x7f7828069450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f78400af000 tx=0x7f78400a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:17.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.198+0000 7f783a7fc640 1 -- 172.21.15.133:0/2424821950 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f783c0558f0 con 0x7f784415f020 2024-04-25T10:10:17.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:17.378+0000 7f784bc75640 1 -- 172.21.15.133:0/2424821950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7844067c70 con 0x7f784415f020 2024-04-25T10:10:18.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f783a7fc640 1 -- 172.21.15.133:0/2424821950 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7f783c059990 con 0x7f784415f020 2024-04-25T10:10:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 -- 172.21.15.133:0/2424821950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7828066f90 msgr2=0x7f7828069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7828066f90 0x7f7828069450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f78400af000 tx=0x7f78400a7000 comp rx=0 tx=0).stop 2024-04-25T10:10:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 -- 172.21.15.133:0/2424821950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f784415f020 msgr2=0x7f784414a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f784415f020 0x7f784414a490 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c730 tx=0x7f783c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:10:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 -- 172.21.15.133:0/2424821950 shutdown_connections 2024-04-25T10:10:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7828066f90 0x7f7828069450 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f784415f020 0x7f784414a490 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78440d21c0 0x7f78440c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 --2- 172.21.15.133:0/2424821950 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f78440cfcf0 0x7f7844149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 -- 172.21.15.133:0/2424821950 >> 172.21.15.133:0/2424821950 conn(0x7f78440d16a0 msgr2=0x7f78440cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 -- 172.21.15.133:0/2424821950 shutdown_connections 2024-04-25T10:10:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.050+0000 7f784bc75640 1 -- 172.21.15.133:0/2424821950 wait complete. 2024-04-25T10:10:18.081 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:10:18.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14cb404640 1 Processor -- start 2024-04-25T10:10:18.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14cb404640 1 -- start start 2024-04-25T10:10:18.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14cb404640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f14c4160f10 0x7f14c40b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:18.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14cb404640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c40d0610 con 0x7f14c406b3a0 2024-04-25T10:10:18.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14cb404640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c41612f0 con 0x7f14c40b3aa0 2024-04-25T10:10:18.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14cb404640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c40d08b0 con 0x7f14c4160f10 2024-04-25T10:10:18.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14c91a0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f14c4160f10 0x7f14c40b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14c91a0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f14c4160f10 0x7f14c40b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:44992/0 (socket says 172.21.15.133:44992) 2024-04-25T10:10:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14c91a0640 1 -- 172.21.15.133:0/3887775611 learned_addr learned my addr 172.21.15.133:0/3887775611 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14c899f640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f14c40b3aa0 0x7f14c406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:34756/0 (socket says 172.21.15.133:34756) 2024-04-25T10:10:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14c99a1640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f14c406b3a0 0x7f14c4161610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50878/0 (socket says 172.21.15.133:50878) 2024-04-25T10:10:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14bbfff640 1 -- 172.21.15.133:0/3887775611 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310279396 0 0) 0x7f14c40d0610 con 0x7f14c406b3a0 2024-04-25T10:10:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14bbfff640 1 -- 172.21.15.133:0/3887775611 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14b0003530 con 0x7f14c406b3a0 2024-04-25T10:10:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14c91a0640 1 -- 172.21.15.133:0/3887775611 >> v1:172.21.15.162:6789/0 conn(0x7f14c40b3aa0 legacy=0x7f14c406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14c91a0640 1 -- 172.21.15.133:0/3887775611 >> v1:172.21.15.133:6789/0 conn(0x7f14c406b3a0 legacy=0x7f14c4161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.198+0000 7f14bbfff640 1 -- 172.21.15.133:0/3887775611 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1826435054 0 0) 0x7f14b0003530 con 0x7f14c406b3a0 2024-04-25T10:10:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14c91a0640 1 -- 172.21.15.133:0/3887775611 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c40d0b90 con 0x7f14c4160f10 2024-04-25T10:10:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14c91a0640 1 --2- 172.21.15.133:0/3887775611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14c4160f10 0x7f14c40b3560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009e50 tx=0x7f14c001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2df3f78f2070f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14bbfff640 1 -- 172.21.15.133:0/3887775611 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4162530 con 0x7f14c4160f10 2024-04-25T10:10:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14bbfff640 1 -- 172.21.15.133:0/3887775611 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c00090d0 con 0x7f14c4160f10 2024-04-25T10:10:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14bbfff640 1 -- 172.21.15.133:0/3887775611 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc0016a0 con 0x7f14c4160f10 2024-04-25T10:10:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14cb404640 1 -- 172.21.15.133:0/3887775611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14c4160f10 msgr2=0x7f14c40b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14cb404640 1 --2- 172.21.15.133:0/3887775611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14c4160f10 0x7f14c40b3560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009e50 tx=0x7f14c001c890 comp rx=0 tx=0).stop 2024-04-25T10:10:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14cb404640 1 -- 172.21.15.133:0/3887775611 shutdown_connections 2024-04-25T10:10:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14cb404640 1 --2- 172.21.15.133:0/3887775611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14c4160f10 0x7f14c40b3560 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14cb404640 1 -- 172.21.15.133:0/3887775611 >> 172.21.15.133:0/3887775611 conn(0x7f14c406e480 msgr2=0x7f14c406e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14cb404640 1 -- 172.21.15.133:0/3887775611 shutdown_connections 2024-04-25T10:10:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.202+0000 7f14cb404640 1 -- 172.21.15.133:0/3887775611 wait complete. 2024-04-25T10:10:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14cb404640 1 Processor -- start 2024-04-25T10:10:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14cb404640 1 -- start start 2024-04-25T10:10:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14cb404640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14c406b3a0 0x7f14c41560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14cb404640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14c40b3aa0 0x7f14c4137720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14cb404640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14c4160f10 0x7f14c40b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14cb404640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c416b8c0 con 0x7f14c406b3a0 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14cb404640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c416b740 con 0x7f14c4160f10 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14cb404640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c416ba40 con 0x7f14c40b3aa0 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14c91a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14c406b3a0 0x7f14c41560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14c91a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14c406b3a0 0x7f14c41560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38918/0 (socket says 172.21.15.133:38918) 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14c91a0640 1 -- 172.21.15.133:0/2623167577 learned_addr learned my addr 172.21.15.133:0/2623167577 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14c899f640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14c40b3aa0 0x7f14c4137720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14c91a0640 1 -- 172.21.15.133:0/2623167577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14c40b3aa0 msgr2=0x7f14c4137720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14c91a0640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14c40b3aa0 0x7f14c4137720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14c91a0640 1 -- 172.21.15.133:0/2623167577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14c4160f10 msgr2=0x7f14c40b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:18.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14c91a0640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14c4160f10 0x7f14c40b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14c91a0640 1 -- 172.21.15.133:0/2623167577 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c4137c60 con 0x7f14c406b3a0 2024-04-25T10:10:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14c91a0640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14c406b3a0 0x7f14c41560d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f14c001af40 tx=0x7f14c0019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14b9ffb640 1 -- 172.21.15.133:0/2623167577 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c00bb020 con 0x7f14c406b3a0 2024-04-25T10:10:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14b9ffb640 1 -- 172.21.15.133:0/2623167577 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c00a5ae0 con 0x7f14c406b3a0 2024-04-25T10:10:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14b9ffb640 1 -- 172.21.15.133:0/2623167577 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0009510 con 0x7f14c406b3a0 2024-04-25T10:10:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14cb404640 1 -- 172.21.15.133:0/2623167577 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c4156720 con 0x7f14c406b3a0 2024-04-25T10:10:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14cb404640 1 -- 172.21.15.133:0/2623167577 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c41583e0 con 0x7f14c406b3a0 2024-04-25T10:10:18.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14cb404640 1 -- 172.21.15.133:0/2623167577 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c4069000 con 0x7f14c406b3a0 2024-04-25T10:10:18.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14b9ffb640 1 -- 172.21.15.133:0/2623167577 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14c0004040 con 0x7f14c406b3a0 2024-04-25T10:10:18.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14b9ffb640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f14a8066f60 0x7f14a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:18.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14b9ffb640 1 -- 172.21.15.133:0/2623167577 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f14c011fac0 con 0x7f14c406b3a0 2024-04-25T10:10:18.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.206+0000 7f14c899f640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f14a8066f60 0x7f14a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:18.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.210+0000 7f14c899f640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f14a8066f60 0x7f14a8069420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f14c40733e0 tx=0x7f14b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:18.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.214+0000 7f14b9ffb640 1 -- 172.21.15.133:0/2623167577 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14c00c5020 con 0x7f14c406b3a0 2024-04-25T10:10:18.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:18.358+0000 7f14cb404640 1 -- 172.21.15.133:0/2623167577 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f14c406e9e0 con 0x7f14c406b3a0 2024-04-25T10:10:19.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.202+0000 7f14b9ffb640 1 -- 172.21.15.133:0/2623167577 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) v1 ==== 181+0+0 (secure 0 0 0) 0x7f14c0016540 con 0x7f14c406b3a0 2024-04-25T10:10:19.211 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:10:19.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.206+0000 7f14cb404640 1 -- 172.21.15.133:0/2623167577 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f14a8066f60 msgr2=0x7f14a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:19.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.206+0000 7f14cb404640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f14a8066f60 0x7f14a8069420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f14c40733e0 tx=0x7f14b4009040 comp rx=0 tx=0).stop 2024-04-25T10:10:19.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.206+0000 7f14cb404640 1 -- 172.21.15.133:0/2623167577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14c406b3a0 msgr2=0x7f14c41560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:19.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.206+0000 7f14cb404640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14c406b3a0 0x7f14c41560d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f14c001af40 tx=0x7f14c0019630 comp rx=0 tx=0).stop 2024-04-25T10:10:19.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.206+0000 7f14cb404640 1 -- 172.21.15.133:0/2623167577 shutdown_connections 2024-04-25T10:10:19.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.206+0000 7f14cb404640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14c4160f10 0x7f14c40b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:19.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.206+0000 7f14cb404640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f14c40b3aa0 0x7f14c4137720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:19.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.206+0000 7f14cb404640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f14a8066f60 0x7f14a8069420 secure :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f14c40733e0 tx=0x7f14b4009040 comp rx=0 tx=0).stop 2024-04-25T10:10:19.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.206+0000 7f14cb404640 1 --2- 172.21.15.133:0/2623167577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14c406b3a0 0x7f14c41560d0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:19.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.206+0000 7f14cb404640 1 -- 172.21.15.133:0/2623167577 >> 172.21.15.133:0/2623167577 conn(0x7f14c406e480 msgr2=0x7f14c40b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:19.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.210+0000 7f14cb404640 1 -- 172.21.15.133:0/2623167577 shutdown_connections 2024-04-25T10:10:19.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.210+0000 7f14cb404640 1 -- 172.21.15.133:0/2623167577 wait complete. 2024-04-25T10:10:19.240 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:10:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f84000d5640 1 Processor -- start 2024-04-25T10:10:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f84000d5640 1 -- start start 2024-04-25T10:10:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f84000d5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f83f8071cb0 0x7f83f815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f84000d5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f83f815c250 0x7f83f80cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f84000d5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f815c9b0 con 0x7f83f80b4940 2024-04-25T10:10:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f84000d5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f8072a90 con 0x7f83f8071cb0 2024-04-25T10:10:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f84000d5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f80a85d0 con 0x7f83f815c250 2024-04-25T10:10:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f83fd670640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f83f8071cb0 0x7f83f815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f83fd670640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f83f8071cb0 0x7f83f815bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38930/0 (socket says 172.21.15.133:38930) 2024-04-25T10:10:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f83fd670640 1 -- 172.21.15.133:0/4199577977 learned_addr learned my addr 172.21.15.133:0/4199577977 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f83fe672640 1 --2- 172.21.15.133:0/4199577977 >> v2:172.21.15.162:3300/0 conn(0x7f83f815c250 0x7f83f80cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f83fd670640 1 -- 172.21.15.133:0/4199577977 >> v2:172.21.15.162:3300/0 conn(0x7f83f815c250 msgr2=0x7f83f80cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f83fd670640 1 --2- 172.21.15.133:0/4199577977 >> v2:172.21.15.162:3300/0 conn(0x7f83f815c250 0x7f83f80cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f83fd670640 1 -- 172.21.15.133:0/4199577977 >> v1:172.21.15.162:6789/0 conn(0x7f83f80b4940 legacy=0x7f83f80b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.358+0000 7f83fd670640 1 -- 172.21.15.133:0/4199577977 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f815cc90 con 0x7f83f8071cb0 2024-04-25T10:10:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f83fd670640 1 --2- 172.21.15.133:0/4199577977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f8071cb0 0x7f83f815bd10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f83f80cbf90 tx=0x7f83f401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b43d1a315e1a162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f83fce6f640 1 -- 172.21.15.133:0/4199577977 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f40aa610 con 0x7f83f8071cb0 2024-04-25T10:10:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f83fce6f640 1 -- 172.21.15.133:0/4199577977 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f40af070 con 0x7f83f8071cb0 2024-04-25T10:10:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f83fce6f640 1 -- 172.21.15.133:0/4199577977 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f40aa9d0 con 0x7f83f8071cb0 2024-04-25T10:10:19.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f84000d5640 1 -- 172.21.15.133:0/4199577977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f8071cb0 msgr2=0x7f83f815bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:19.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f84000d5640 1 --2- 172.21.15.133:0/4199577977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f8071cb0 0x7f83f815bd10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f83f80cbf90 tx=0x7f83f401c4b0 comp rx=0 tx=0).stop 2024-04-25T10:10:19.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f84000d5640 1 -- 172.21.15.133:0/4199577977 shutdown_connections 2024-04-25T10:10:19.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f84000d5640 1 --2- 172.21.15.133:0/4199577977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f8071cb0 0x7f83f815bd10 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:19.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f84000d5640 1 --2- 172.21.15.133:0/4199577977 >> v2:172.21.15.162:3300/0 conn(0x7f83f815c250 0x7f83f80cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f84000d5640 1 -- 172.21.15.133:0/4199577977 >> 172.21.15.133:0/4199577977 conn(0x7f83f806d370 msgr2=0x7f83f806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f84000d5640 1 -- 172.21.15.133:0/4199577977 shutdown_connections 2024-04-25T10:10:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f84000d5640 1 -- 172.21.15.133:0/4199577977 wait complete. 2024-04-25T10:10:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f84000d5640 1 Processor -- start 2024-04-25T10:10:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.362+0000 7f84000d5640 1 -- start start 2024-04-25T10:10:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f84000d5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f8071cb0 0x7f83f815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f84000d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80b4940 0x7f83f80b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f84000d5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f815c250 0x7f83f80c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f84000d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80cc960 con 0x7f83f80b4940 2024-04-25T10:10:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f84000d5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f815c630 con 0x7f83f8071cb0 2024-04-25T10:10:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f84000d5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80ccae0 con 0x7f83f815c250 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fd670640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80b4940 0x7f83f80b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fd670640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80b4940 0x7f83f80b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38934/0 (socket says 172.21.15.133:38934) 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fd670640 1 -- 172.21.15.133:0/2668971532 learned_addr learned my addr 172.21.15.133:0/2668971532 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fde71640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f8071cb0 0x7f83f815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fe672640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f815c250 0x7f83f80c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fd670640 1 -- 172.21.15.133:0/2668971532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f815c250 msgr2=0x7f83f80c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fd670640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f815c250 0x7f83f80c4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fd670640 1 -- 172.21.15.133:0/2668971532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f8071cb0 msgr2=0x7f83f815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fd670640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f8071cb0 0x7f83f815ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fd670640 1 -- 172.21.15.133:0/2668971532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f80b30e0 con 0x7f83f80b4940 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fe672640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f815c250 0x7f83f80c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fde71640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f8071cb0 0x7f83f815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:19.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fd670640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80b4940 0x7f83f80b4140 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f83f40ac040 tx=0x7f83f401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:19.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83ea7fc640 1 -- 172.21.15.133:0/2668971532 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f40b3800 con 0x7f83f80b4940 2024-04-25T10:10:19.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83ea7fc640 1 -- 172.21.15.133:0/2668971532 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f40b39c0 con 0x7f83f80b4940 2024-04-25T10:10:19.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f84000d5640 1 -- 172.21.15.133:0/2668971532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f80b3380 con 0x7f83f80b4940 2024-04-25T10:10:19.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83ea7fc640 1 -- 172.21.15.133:0/2668971532 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f40b3db0 con 0x7f83f80b4940 2024-04-25T10:10:19.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f84000d5640 1 -- 172.21.15.133:0/2668971532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f80b3940 con 0x7f83f80b4940 2024-04-25T10:10:19.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f84000d5640 1 -- 172.21.15.133:0/2668971532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f8069000 con 0x7f83f80b4940 2024-04-25T10:10:19.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83ea7fc640 1 -- 172.21.15.133:0/2668971532 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83f40c63f0 con 0x7f83f80b4940 2024-04-25T10:10:19.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83ea7fc640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83dc066f60 0x7f83dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:19.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83fde71640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83dc066f60 0x7f83dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:19.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.366+0000 7f83ea7fc640 1 -- 172.21.15.133:0/2668971532 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f83f411fc00 con 0x7f83f80b4940 2024-04-25T10:10:19.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.370+0000 7f83fde71640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83dc066f60 0x7f83dc069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f83ec003ec0 tx=0x7f83ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:19.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.370+0000 7f83ea7fc640 1 -- 172.21.15.133:0/2668971532 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83f4126080 con 0x7f83f80b4940 2024-04-25T10:10:19.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:19.530+0000 7f84000d5640 1 -- 172.21.15.133:0/2668971532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f83f80ad720 con 0x7f83f80b4940 2024-04-25T10:10:20.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.206+0000 7f83ea7fc640 1 -- 172.21.15.133:0/2668971532 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) v1 ==== 169+0+0 (secure 0 0 0) 0x7f83f40a7070 con 0x7f83f80b4940 2024-04-25T10:10:20.213 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:10:20.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.210+0000 7f84000d5640 1 -- 172.21.15.133:0/2668971532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83dc066f60 msgr2=0x7f83dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.210+0000 7f84000d5640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83dc066f60 0x7f83dc069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f83ec003ec0 tx=0x7f83ec009040 comp rx=0 tx=0).stop 2024-04-25T10:10:20.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.210+0000 7f84000d5640 1 -- 172.21.15.133:0/2668971532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80b4940 msgr2=0x7f83f80b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.210+0000 7f84000d5640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80b4940 0x7f83f80b4140 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f83f40ac040 tx=0x7f83f401acf0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.210+0000 7f84000d5640 1 -- 172.21.15.133:0/2668971532 shutdown_connections 2024-04-25T10:10:20.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.210+0000 7f84000d5640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83f815c250 0x7f83f80c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.210+0000 7f84000d5640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83dc066f60 0x7f83dc069420 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.210+0000 7f84000d5640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80b4940 0x7f83f80b4140 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.210+0000 7f84000d5640 1 --2- 172.21.15.133:0/2668971532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83f8071cb0 0x7f83f815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.210+0000 7f84000d5640 1 -- 172.21.15.133:0/2668971532 >> 172.21.15.133:0/2668971532 conn(0x7f83f806d370 msgr2=0x7f83f80add70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:20.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.214+0000 7f84000d5640 1 -- 172.21.15.133:0/2668971532 shutdown_connections 2024-04-25T10:10:20.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.214+0000 7f84000d5640 1 -- 172.21.15.133:0/2668971532 wait complete. 2024-04-25T10:10:20.245 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:10:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd2fca640 1 Processor -- start 2024-04-25T10:10:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd2fca640 1 -- start start 2024-04-25T10:10:20.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd2fca640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15d9a0 0x7f0bcc0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd2fca640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bcc0d24c0 con 0x7f0bcc0d2030 2024-04-25T10:10:20.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd2fca640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bcc15ebd0 con 0x7f0bcc0ce600 2024-04-25T10:10:20.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd2fca640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc15eeb0 con 0x7f0bcc15d9a0 2024-04-25T10:10:20.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd1567640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0bcc0d2030 0x7f0bcc15dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50894/0 (socket says 172.21.15.133:50894) 2024-04-25T10:10:20.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd1567640 1 -- 172.21.15.133:0/3781608723 learned_addr learned my addr 172.21.15.133:0/3781608723 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:20.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527091948 0 0) 0x7f0bcc0d24c0 con 0x7f0bcc0d2030 2024-04-25T10:10:20.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bb4003530 con 0x7f0bcc0d2030 2024-04-25T10:10:20.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd0d66640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15d9a0 0x7f0bcc0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4155937035 0 0) 0x7f0bb4003530 con 0x7f0bcc0d2030 2024-04-25T10:10:20.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15d9a0 msgr2=0x7f0bcc0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:20.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15d9a0 0x7f0bcc0d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 >> v1:172.21.15.162:6789/0 conn(0x7f0bcc0ce600 legacy=0x7f0bcc0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc0d0330 con 0x7f0bcc0d2030 2024-04-25T10:10:20.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0bc4001a00 con 0x7f0bcc0d2030 2024-04-25T10:10:20.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 >> v1:172.21.15.133:6789/0 conn(0x7f0bcc0d2030 legacy=0x7f0bcc15dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 0x7f0bb4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bb4004810 0x7f0bb4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15d9a0 0x7f0bb400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc15eeb0 con 0x7f0bb40039f0 2024-04-25T10:10:20.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb4005530 con 0x7f0bb4004810 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb400bf10 con 0x7f0bcc15d9a0 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd0d66640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 0x7f0bb4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd1567640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15d9a0 0x7f0bb400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcbfff640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bb4004810 0x7f0bb4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd0d66640 1 -- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15d9a0 msgr2=0x7f0bb400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd0d66640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15d9a0 0x7f0bb400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd0d66640 1 -- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bb4004810 msgr2=0x7f0bb4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd0d66640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bb4004810 0x7f0bb4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bd0d66640 1 -- 172.21.15.133:0/3781608723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb4003530 con 0x7f0bb40039f0 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.370+0000 7f0bcbfff640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bb4004810 0x7f0bb4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd0d66640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 0x7f0bb4003e10 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00ba10 tx=0x7f0bbc01c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ba50704d09915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:20.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00b5e0 con 0x7f0bb40039f0 2024-04-25T10:10:20.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bbc0b1070 con 0x7f0bb40039f0 2024-04-25T10:10:20.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bcb7fe640 1 -- 172.21.15.133:0/3781608723 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0ac8c0 con 0x7f0bb40039f0 2024-04-25T10:10:20.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3781608723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 msgr2=0x7f0bb4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 --2- 172.21.15.133:0/3781608723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 0x7f0bb4003e10 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00ba10 tx=0x7f0bbc01c650 comp rx=0 tx=0).stop 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd1567640 1 -- 172.21.15.133:0/3781608723 reap_dead start 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3781608723 shutdown_connections 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3781608723 >> 172.21.15.133:0/3781608723 conn(0x7f0bcc06bf80 msgr2=0x7f0bcc06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3781608723 shutdown_connections 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3781608723 wait complete. 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 Processor -- start 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 -- start start 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 0x7f0bcc159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15acd0 0x7f0bcc15b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bcc1576c0 0x7f0bcc157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0b9820 con 0x7f0bb40039f0 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0b96a0 con 0x7f0bcc1576c0 2024-04-25T10:10:20.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd2fca640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0cedf0 con 0x7f0bcc15acd0 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd0d66640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 0x7f0bcc159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd0d66640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 0x7f0bcc159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38954/0 (socket says 172.21.15.133:38954) 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.374+0000 7f0bd0d66640 1 -- 172.21.15.133:0/3249401715 learned_addr learned my addr 172.21.15.133:0/3249401715 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bd1567640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bcc1576c0 0x7f0bcc157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bd0d66640 1 -- 172.21.15.133:0/3249401715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15acd0 msgr2=0x7f0bcc15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bcbfff640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15acd0 0x7f0bcc15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bd0d66640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15acd0 0x7f0bcc15b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bd0d66640 1 -- 172.21.15.133:0/3249401715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bcc1576c0 msgr2=0x7f0bcc157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bd0d66640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bcc1576c0 0x7f0bcc157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bd0d66640 1 -- 172.21.15.133:0/3249401715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc158af0 con 0x7f0bb40039f0 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bd1567640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bcc1576c0 0x7f0bcc157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bd0d66640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 0x7f0bcc159310 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc01c480 tx=0x7f0bb4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bc97fa640 1 -- 172.21.15.133:0/3249401715 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0bb070 con 0x7f0bb40039f0 2024-04-25T10:10:20.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bc97fa640 1 -- 172.21.15.133:0/3249401715 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bbc0ac940 con 0x7f0bb40039f0 2024-04-25T10:10:20.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bc97fa640 1 -- 172.21.15.133:0/3249401715 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0accf0 con 0x7f0bb40039f0 2024-04-25T10:10:20.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3249401715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc158530 con 0x7f0bb40039f0 2024-04-25T10:10:20.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3249401715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc0b34f0 con 0x7f0bb40039f0 2024-04-25T10:10:20.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3249401715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc068e80 con 0x7f0bb40039f0 2024-04-25T10:10:20.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bc97fa640 1 -- 172.21.15.133:0/3249401715 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0bbc0aa050 con 0x7f0bb40039f0 2024-04-25T10:10:20.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bc97fa640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bb8066f60 0x7f0bb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bc97fa640 1 -- 172.21.15.133:0/3249401715 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0bbc11f510 con 0x7f0bb40039f0 2024-04-25T10:10:20.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bcbfff640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bb8066f60 0x7f0bb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.378+0000 7f0bcbfff640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bb8066f60 0x7f0bb8069420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0000c10 tx=0x7f0bc0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:20.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.382+0000 7f0bc97fa640 1 -- 172.21.15.133:0/3249401715 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0bbc016390 con 0x7f0bb40039f0 2024-04-25T10:10:20.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.542+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3249401715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0bcc162020 con 0x7f0bb40039f0 2024-04-25T10:10:20.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.550+0000 7f0bc97fa640 1 -- 172.21.15.133:0/3249401715 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0bbc0f0720 con 0x7f0bb40039f0 2024-04-25T10:10:20.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3249401715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bb8066f60 msgr2=0x7f0bb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bb8066f60 0x7f0bb8069420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0000c10 tx=0x7f0bc0009040 comp rx=0 tx=0).stop 2024-04-25T10:10:20.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3249401715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 msgr2=0x7f0bcc159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 0x7f0bcc159310 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc01c480 tx=0x7f0bb4005840 comp rx=0 tx=0).stop 2024-04-25T10:10:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3249401715 shutdown_connections 2024-04-25T10:10:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0bcc1576c0 0x7f0bcc157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0bcc15acd0 0x7f0bcc15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0bb8066f60 0x7f0bb8069420 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 --2- 172.21.15.133:0/3249401715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bb40039f0 0x7f0bcc159310 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3249401715 >> 172.21.15.133:0/3249401715 conn(0x7f0bcc06bf80 msgr2=0x7f0bcc0735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:20.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3249401715 shutdown_connections 2024-04-25T10:10:20.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.554+0000 7f0bd2fca640 1 -- 172.21.15.133:0/3249401715 wait complete. 2024-04-25T10:10:20.578 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T10:10:20.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.694+0000 7f1ca0705640 1 Processor -- start 2024-04-25T10:10:20.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.694+0000 7f1ca0705640 1 -- start start 2024-04-25T10:10:20.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1ca0705640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1c9815da40 0x7f1c980704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1ca0705640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c980b3d30 con 0x7f1c980b2a10 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1ca0705640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9815de20 con 0x7f1c98072e40 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1ca0705640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c98070300 con 0x7f1c9815da40 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9eca2640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1c980b2a10 0x7f1c98161550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50896/0 (socket says 172.21.15.133:50896) 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9eca2640 1 -- 172.21.15.133:0/307417272 learned_addr learned my addr 172.21.15.133:0/307417272 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9e4a1640 1 --2- 172.21.15.133:0/307417272 >> v2:172.21.15.162:3300/0 conn(0x7f1c9815da40 0x7f1c980704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277002938 0 0) 0x7f1c980b3d30 con 0x7f1c980b2a10 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c84003530 con 0x7f1c980b2a10 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3747665117 0 0) 0x7f1c84003530 con 0x7f1c980b2a10 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 >> v2:172.21.15.162:3300/0 conn(0x7f1c9815da40 msgr2=0x7f1c980704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 --2- 172.21.15.133:0/307417272 >> v2:172.21.15.162:3300/0 conn(0x7f1c9815da40 0x7f1c980704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 >> v1:172.21.15.162:6789/0 conn(0x7f1c98072e40 legacy=0x7f1c980b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9806fd40 con 0x7f1c980b2a10 2024-04-25T10:10:20.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1c90001a00 con 0x7f1c980b2a10 2024-04-25T10:10:20.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 >> v1:172.21.15.133:6789/0 conn(0x7f1c980b2a10 legacy=0x7f1c98161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 0x7f1c84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c84004810 0x7f1c84004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c840051d0 0x7f1c8400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8400c040 con 0x7f1c840039c0 2024-04-25T10:10:20.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8400c1f0 con 0x7f1c840051d0 2024-04-25T10:10:20.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8400c3f0 con 0x7f1c84004810 2024-04-25T10:10:20.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9e4a1640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 0x7f1c84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9eca2640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c840051d0 0x7f1c8400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9dca0640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c84004810 0x7f1c84004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9e4a1640 1 -- 172.21.15.133:0/307417272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c84004810 msgr2=0x7f1c84004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9e4a1640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c84004810 0x7f1c84004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9e4a1640 1 -- 172.21.15.133:0/307417272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c840051d0 msgr2=0x7f1c8400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9e4a1640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c840051d0 0x7f1c8400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9e4a1640 1 -- 172.21.15.133:0/307417272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c84003530 con 0x7f1c840039c0 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9dca0640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c84004810 0x7f1c84004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9eca2640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c840051d0 0x7f1c8400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9eca2640 1 -- 172.21.15.133:0/307417272 reap_dead start 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9e4a1640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 0x7f1c84003de0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009ae0 tx=0x7f1c9401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4142ffba266fb3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940096b0 con 0x7f1c840039c0 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c940a64e0 con 0x7f1c840039c0 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.698+0000 7f1c9d49f640 1 -- 172.21.15.133:0/307417272 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940a68d0 con 0x7f1c840039c0 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 -- 172.21.15.133:0/307417272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 msgr2=0x7f1c84003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 0x7f1c84003de0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009ae0 tx=0x7f1c9401ca70 comp rx=0 tx=0).stop 2024-04-25T10:10:20.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 -- 172.21.15.133:0/307417272 shutdown_connections 2024-04-25T10:10:20.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 --2- 172.21.15.133:0/307417272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 0x7f1c84003de0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 -- 172.21.15.133:0/307417272 >> 172.21.15.133:0/307417272 conn(0x7f1c9806e440 msgr2=0x7f1c9806e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:20.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 -- 172.21.15.133:0/307417272 shutdown_connections 2024-04-25T10:10:20.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 -- 172.21.15.133:0/307417272 wait complete. 2024-04-25T10:10:20.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 Processor -- start 2024-04-25T10:10:20.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 -- start start 2024-04-25T10:10:20.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 0x7f1c98150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c980cf580 0x7f1c98156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c981583c0 0x7f1c98158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980b2890 con 0x7f1c840039c0 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980b2710 con 0x7f1c980cf580 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9806b4a0 con 0x7f1c981583c0 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c9e4a1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 0x7f1c98150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c9e4a1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 0x7f1c98150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38970/0 (socket says 172.21.15.133:38970) 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c9e4a1640 1 -- 172.21.15.133:0/4168953526 learned_addr learned my addr 172.21.15.133:0/4168953526 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c9eca2640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c981583c0 0x7f1c98158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c9e4a1640 1 -- 172.21.15.133:0/4168953526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c981583c0 msgr2=0x7f1c98158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c9dca0640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c980cf580 0x7f1c98156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c9e4a1640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c981583c0 0x7f1c98158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c9e4a1640 1 -- 172.21.15.133:0/4168953526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c980cf580 msgr2=0x7f1c98156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c9e4a1640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c980cf580 0x7f1c98156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c9e4a1640 1 -- 172.21.15.133:0/4168953526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c981faa00 con 0x7f1c840039c0 2024-04-25T10:10:20.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c9e4a1640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 0x7f1c98150100 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1c9401c8a0 tx=0x7f1c94019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:20.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c8effd640 1 -- 172.21.15.133:0/4168953526 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940a6690 con 0x7f1c840039c0 2024-04-25T10:10:20.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c8effd640 1 -- 172.21.15.133:0/4168953526 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c940be070 con 0x7f1c840039c0 2024-04-25T10:10:20.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1c8effd640 1 -- 172.21.15.133:0/4168953526 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940a6a30 con 0x7f1c840039c0 2024-04-25T10:10:20.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 -- 172.21.15.133:0/4168953526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c981face0 con 0x7f1c840039c0 2024-04-25T10:10:20.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.702+0000 7f1ca0705640 1 -- 172.21.15.133:0/4168953526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c981fb2a0 con 0x7f1c840039c0 2024-04-25T10:10:20.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.706+0000 7f1ca0705640 1 -- 172.21.15.133:0/4168953526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c98067c70 con 0x7f1c840039c0 2024-04-25T10:10:20.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.706+0000 7f1c8effd640 1 -- 172.21.15.133:0/4168953526 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c940a6bd0 con 0x7f1c840039c0 2024-04-25T10:10:20.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.706+0000 7f1c8effd640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c7c066f60 0x7f1c7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:20.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.706+0000 7f1c8effd640 1 -- 172.21.15.133:0/4168953526 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1c9411f400 con 0x7f1c840039c0 2024-04-25T10:10:20.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.706+0000 7f1c9dca0640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c7c066f60 0x7f1c7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:20.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.706+0000 7f1c9dca0640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c7c066f60 0x7f1c7c069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1c88000c10 tx=0x7f1c88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:20.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.710+0000 7f1c8effd640 1 -- 172.21.15.133:0/4168953526 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c94016780 con 0x7f1c840039c0 2024-04-25T10:10:20.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:20.926+0000 7f1ca0705640 1 -- 172.21.15.133:0/4168953526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f1c980cfcf0 con 0x7f1c840039c0 2024-04-25T10:10:21.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.222+0000 7f1c8effd640 1 -- 172.21.15.133:0/4168953526 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v376) v1 ==== 158+0+0 (secure 0 0 0) 0x7f1c940ad3d0 con 0x7f1c840039c0 2024-04-25T10:10:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 -- 172.21.15.133:0/4168953526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c7c066f60 msgr2=0x7f1c7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c7c066f60 0x7f1c7c069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1c88000c10 tx=0x7f1c88009040 comp rx=0 tx=0).stop 2024-04-25T10:10:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 -- 172.21.15.133:0/4168953526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 msgr2=0x7f1c98150100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 0x7f1c98150100 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1c9401c8a0 tx=0x7f1c94019810 comp rx=0 tx=0).stop 2024-04-25T10:10:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 -- 172.21.15.133:0/4168953526 shutdown_connections 2024-04-25T10:10:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c981583c0 0x7f1c98158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c980cf580 0x7f1c98156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c7c066f60 0x7f1c7c069420 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 --2- 172.21.15.133:0/4168953526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c840039c0 0x7f1c98150100 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 -- 172.21.15.133:0/4168953526 >> 172.21.15.133:0/4168953526 conn(0x7f1c9806e440 msgr2=0x7f1c980d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 -- 172.21.15.133:0/4168953526 shutdown_connections 2024-04-25T10:10:21.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.226+0000 7f1ca0705640 1 -- 172.21.15.133:0/4168953526 wait complete. 2024-04-25T10:10:21.255 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T10:10:21.256 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:10:21.256 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.370+0000 7f40716c7640 1 Processor -- start 2024-04-25T10:10:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.370+0000 7f40716c7640 1 -- start start 2024-04-25T10:10:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f40716c7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f406c15fed0 0x7f406c0d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f40716c7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c0cf490 con 0x7f406c0b1fa0 2024-04-25T10:10:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f40716c7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c0cf770 con 0x7f406c15b990 2024-04-25T10:10:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f40716c7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c0cfa50 con 0x7f406c15fed0 2024-04-25T10:10:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f406b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f406c15fed0 0x7f406c0d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f406a7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f406c0b1fa0 0x7f406c15f7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50912/0 (socket says 172.21.15.133:50912) 2024-04-25T10:10:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f406a7fc640 1 -- 172.21.15.133:0/1477499910 learned_addr learned my addr 172.21.15.133:0/1477499910 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:21.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f406b7fe640 1 -- 172.21.15.133:0/1477499910 >> v1:172.21.15.162:6789/0 conn(0x7f406c15b990 legacy=0x7f406c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:21.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f406b7fe640 1 -- 172.21.15.133:0/1477499910 >> v1:172.21.15.133:6789/0 conn(0x7f406c0b1fa0 legacy=0x7f406c15f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f406b7fe640 1 -- 172.21.15.133:0/1477499910 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c0cfd30 con 0x7f406c15fed0 2024-04-25T10:10:21.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f4069ffb640 1 -- 172.21.15.133:0/1477499910 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996879737 0 0) 0x7f406c0cf490 con 0x7f406c0b1fa0 2024-04-25T10:10:21.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f406b7fe640 1 --2- 172.21.15.133:0/1477499910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f406c15fed0 0x7f406c0d0250 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4064009d90 tx=0x7f406401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966e88268ba1747a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f4069ffb640 1 -- 172.21.15.133:0/1477499910 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40640b2070 con 0x7f406c15fed0 2024-04-25T10:10:21.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f4069ffb640 1 -- 172.21.15.133:0/1477499910 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40640a57f0 con 0x7f406c15fed0 2024-04-25T10:10:21.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.374+0000 7f4069ffb640 1 -- 172.21.15.133:0/1477499910 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40640a5bc0 con 0x7f406c15fed0 2024-04-25T10:10:21.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 -- 172.21.15.133:0/1477499910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f406c15fed0 msgr2=0x7f406c0d0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 --2- 172.21.15.133:0/1477499910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f406c15fed0 0x7f406c0d0250 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4064009d90 tx=0x7f406401c930 comp rx=0 tx=0).stop 2024-04-25T10:10:21.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 -- 172.21.15.133:0/1477499910 shutdown_connections 2024-04-25T10:10:21.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 --2- 172.21.15.133:0/1477499910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f406c15fed0 0x7f406c0d0250 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 -- 172.21.15.133:0/1477499910 >> 172.21.15.133:0/1477499910 conn(0x7f406c09aa10 msgr2=0x7f406c08af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 -- 172.21.15.133:0/1477499910 shutdown_connections 2024-04-25T10:10:21.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 -- 172.21.15.133:0/1477499910 wait complete. 2024-04-25T10:10:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 Processor -- start 2024-04-25T10:10:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 -- start start 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f406c0b1fa0 0x7f406c1f8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f406c15b990 0x7f406c14e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c15fed0 0x7f406c14b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c169880 con 0x7f406c15b990 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c169700 con 0x7f406c15fed0 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c169a00 con 0x7f406c0b1fa0 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f406c15b990 0x7f406c14e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f406c15b990 0x7f406c14e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38996/0 (socket says 172.21.15.133:38996) 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406a7fc640 1 -- 172.21.15.133:0/535679791 learned_addr learned my addr 172.21.15.133:0/535679791 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406b7fe640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c15fed0 0x7f406c14b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406affd640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f406c0b1fa0 0x7f406c1f8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406a7fc640 1 -- 172.21.15.133:0/535679791 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f406c0b1fa0 msgr2=0x7f406c1f8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406a7fc640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f406c0b1fa0 0x7f406c1f8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406a7fc640 1 -- 172.21.15.133:0/535679791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c15fed0 msgr2=0x7f406c14b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406a7fc640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c15fed0 0x7f406c14b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406a7fc640 1 -- 172.21.15.133:0/535679791 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c14ece0 con 0x7f406c15b990 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406affd640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f406c0b1fa0 0x7f406c1f8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f406a7fc640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f406c15b990 0x7f406c14e540 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f40540011f0 tx=0x7f405400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40637fe640 1 -- 172.21.15.133:0/535679791 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054018070 con 0x7f406c15b990 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40637fe640 1 -- 172.21.15.133:0/535679791 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40540074e0 con 0x7f406c15b990 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 -- 172.21.15.133:0/535679791 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406c1507f0 con 0x7f406c15b990 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40637fe640 1 -- 172.21.15.133:0/535679791 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054004a10 con 0x7f406c15b990 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.378+0000 7f40716c7640 1 -- 172.21.15.133:0/535679791 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c14b5d0 con 0x7f406c15b990 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.382+0000 7f40716c7640 1 -- 172.21.15.133:0/535679791 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406c068ef0 con 0x7f406c15b990 2024-04-25T10:10:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.382+0000 7f40637fe640 1 -- 172.21.15.133:0/535679791 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f405401b070 con 0x7f406c15b990 2024-04-25T10:10:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.382+0000 7f40637fe640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f404c066f90 0x7f404c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.382+0000 7f406affd640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f404c066f90 0x7f404c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.382+0000 7f40637fe640 1 -- 172.21.15.133:0/535679791 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4054084280 con 0x7f406c15b990 2024-04-25T10:10:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.382+0000 7f406affd640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f404c066f90 0x7f404c069450 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f405c003e40 tx=0x7f405c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.386+0000 7f40637fe640 1 -- 172.21.15.133:0/535679791 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4054055400 con 0x7f406c15b990 2024-04-25T10:10:21.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.546+0000 7f40716c7640 1 -- 172.21.15.133:0/535679791 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f406c071170 con 0x7f406c15b990 2024-04-25T10:10:21.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.546+0000 7f40637fe640 1 -- 172.21.15.133:0/535679791 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13320 (secure 0 0 0) 0x7f4054054d00 con 0x7f406c15b990 2024-04-25T10:10:21.555 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:21.556 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":86,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:10:20.209172+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6835/3192312240":"2024-04-26T10:10:17.049873+0000","172.21.15.162:6834/3192312240":"2024-04-26T10:10:17.049873+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:21.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 -- 172.21.15.133:0/535679791 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f404c066f90 msgr2=0x7f404c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f404c066f90 0x7f404c069450 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f405c003e40 tx=0x7f405c009040 comp rx=0 tx=0).stop 2024-04-25T10:10:21.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 -- 172.21.15.133:0/535679791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f406c15b990 msgr2=0x7f406c14e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f406c15b990 0x7f406c14e540 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f40540011f0 tx=0x7f405400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 -- 172.21.15.133:0/535679791 shutdown_connections 2024-04-25T10:10:21.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f406c15fed0 0x7f406c14b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f404c066f90 0x7f404c069450 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f406c15b990 0x7f406c14e540 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 --2- 172.21.15.133:0/535679791 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f406c0b1fa0 0x7f406c1f8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 -- 172.21.15.133:0/535679791 >> 172.21.15.133:0/535679791 conn(0x7f406c09aa10 msgr2=0x7f406c0b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 -- 172.21.15.133:0/535679791 shutdown_connections 2024-04-25T10:10:21.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.550+0000 7f40716c7640 1 -- 172.21.15.133:0/535679791 wait complete. 2024-04-25T10:10:21.575 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:21.575 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:21.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f897e5640 1 Processor -- start 2024-04-25T10:10:21.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f897e5640 1 -- start start 2024-04-25T10:10:21.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f897e5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f840ce600 0x7f2f840cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f897e5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8415ebd0 con 0x7f2f840d1920 2024-04-25T10:10:21.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f897e5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8415eeb0 con 0x7f2f8415d9a0 2024-04-25T10:10:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f897e5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f840d0330 con 0x7f2f840ce600 2024-04-25T10:10:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f7bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f840ce600 0x7f2f840cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f7bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f840ce600 0x7f2f840cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39006/0 (socket says 172.21.15.133:39006) 2024-04-25T10:10:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f7bfff640 1 -- 172.21.15.133:0/3272665978 learned_addr learned my addr 172.21.15.133:0/3272665978 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f837fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2f840d1920 0x7f2f840d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50920/0 (socket says 172.21.15.133:50920) 2024-04-25T10:10:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f7bfff640 1 -- 172.21.15.133:0/3272665978 >> v1:172.21.15.162:6789/0 conn(0x7f2f8415d9a0 legacy=0x7f2f840d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f7bfff640 1 -- 172.21.15.133:0/3272665978 >> v1:172.21.15.133:6789/0 conn(0x7f2f840d1920 legacy=0x7f2f840d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f827fc640 1 -- 172.21.15.133:0/3272665978 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954984422 0 0) 0x7f2f8415ebd0 con 0x7f2f840d1920 2024-04-25T10:10:21.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.698+0000 7f2f7bfff640 1 -- 172.21.15.133:0/3272665978 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f840d0610 con 0x7f2f840ce600 2024-04-25T10:10:21.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f7bfff640 1 --2- 172.21.15.133:0/3272665978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f840ce600 0x7f2f840cea90 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009870 tx=0x7f2f6c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b190666a38df32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f827fc640 1 -- 172.21.15.133:0/3272665978 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0b1070 con 0x7f2f840ce600 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f827fc640 1 -- 172.21.15.133:0/3272665978 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c0a8050 con 0x7f2f840ce600 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f827fc640 1 -- 172.21.15.133:0/3272665978 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0ac860 con 0x7f2f840ce600 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 -- 172.21.15.133:0/3272665978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f840ce600 msgr2=0x7f2f840cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 --2- 172.21.15.133:0/3272665978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f840ce600 0x7f2f840cea90 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009870 tx=0x7f2f6c01c4d0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 -- 172.21.15.133:0/3272665978 shutdown_connections 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 --2- 172.21.15.133:0/3272665978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f840ce600 0x7f2f840cea90 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 -- 172.21.15.133:0/3272665978 >> 172.21.15.133:0/3272665978 conn(0x7f2f8406bf80 msgr2=0x7f2f8406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 -- 172.21.15.133:0/3272665978 shutdown_connections 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 -- 172.21.15.133:0/3272665978 wait complete. 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 Processor -- start 2024-04-25T10:10:21.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 -- start start 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f840ce600 0x7f2f8415a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f840d1920 0x7f2f84155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f8415d9a0 0x7f2f84155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8415e710 con 0x7f2f840ce600 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8415e590 con 0x7f2f8415d9a0 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8415e890 con 0x7f2f840d1920 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f82ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f840ce600 0x7f2f8415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f82ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f840ce600 0x7f2f8415a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39014/0 (socket says 172.21.15.133:39014) 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f82ffd640 1 -- 172.21.15.133:0/3551354635 learned_addr learned my addr 172.21.15.133:0/3551354635 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f837fe640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f8415d9a0 0x7f2f84155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f7bfff640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f840d1920 0x7f2f84155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f82ffd640 1 -- 172.21.15.133:0/3551354635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f840d1920 msgr2=0x7f2f84155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f82ffd640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f840d1920 0x7f2f84155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f82ffd640 1 -- 172.21.15.133:0/3551354635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f8415d9a0 msgr2=0x7f2f84155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f82ffd640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f8415d9a0 0x7f2f84155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f82ffd640 1 -- 172.21.15.133:0/3551354635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f84159ca0 con 0x7f2f840ce600 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f82ffd640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f840ce600 0x7f2f8415a140 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400c530 tx=0x7f2f7400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f7b7fe640 1 -- 172.21.15.133:0/3551354635 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74016020 con 0x7f2f840ce600 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f7b7fe640 1 -- 172.21.15.133:0/3551354635 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f74005220 con 0x7f2f840ce600 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 -- 172.21.15.133:0/3551354635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f84158920 con 0x7f2f840ce600 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f7b7fe640 1 -- 172.21.15.133:0/3551354635 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74007030 con 0x7f2f840ce600 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.702+0000 7f2f897e5640 1 -- 172.21.15.133:0/3551354635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f841591c0 con 0x7f2f840ce600 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.706+0000 7f2f897e5640 1 -- 172.21.15.133:0/3551354635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f84068e80 con 0x7f2f840ce600 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.706+0000 7f2f7b7fe640 1 -- 172.21.15.133:0/3551354635 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f740071d0 con 0x7f2f840ce600 2024-04-25T10:10:21.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.706+0000 7f2f7b7fe640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 0x7f2f64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.706+0000 7f2f7bfff640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 0x7f2f64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.706+0000 7f2f7b7fe640 1 -- 172.21.15.133:0/3551354635 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2f740848b0 con 0x7f2f840ce600 2024-04-25T10:10:21.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.706+0000 7f2f7bfff640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 0x7f2f64069420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0ae000 tx=0x7f2f6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:21.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.710+0000 7f2f7b7fe640 1 -- 172.21.15.133:0/3551354635 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f74020020 con 0x7f2f840ce600 2024-04-25T10:10:21.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 -- 172.21.15.133:0/3551354635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f840d20a0 con 0x7f2f840ce600 2024-04-25T10:10:21.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f7b7fe640 1 -- 172.21.15.133:0/3551354635 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f740074c0 con 0x7f2f840ce600 2024-04-25T10:10:21.877 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 -- 172.21.15.133:0/3551354635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 msgr2=0x7f2f64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 0x7f2f64069420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0ae000 tx=0x7f2f6c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 -- 172.21.15.133:0/3551354635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f840ce600 msgr2=0x7f2f8415a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f840ce600 0x7f2f8415a140 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400c530 tx=0x7f2f7400c9f0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 -- 172.21.15.133:0/3551354635 shutdown_connections 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f8415d9a0 0x7f2f84155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f840d1920 0x7f2f84155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 0x7f2f64069420 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 --2- 172.21.15.133:0/3551354635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f840ce600 0x7f2f8415a140 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 -- 172.21.15.133:0/3551354635 >> 172.21.15.133:0/3551354635 conn(0x7f2f8406bf80 msgr2=0x7f2f840b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 -- 172.21.15.133:0/3551354635 shutdown_connections 2024-04-25T10:10:21.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:21.870+0000 7f2f897e5640 1 -- 172.21.15.133:0/3551354635 wait complete. 2024-04-25T10:10:21.895 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T10:10:22.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.002+0000 7f057f74e640 1 Processor -- start 2024-04-25T10:10:22.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.002+0000 7f057f74e640 1 -- start start 2024-04-25T10:10:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057f74e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f05780ce330 0x7f05780ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057f74e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05780c61d0 con 0x7f057815c9a0 2024-04-25T10:10:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057f74e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05780c64b0 con 0x7f05780b1f70 2024-04-25T10:10:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057f74e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05780cecc0 con 0x7f05780ce330 2024-04-25T10:10:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057d4ea640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f057815c9a0 0x7f05780c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50930/0 (socket says 172.21.15.133:50930) 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057d4ea640 1 -- 172.21.15.133:0/471896871 learned_addr learned my addr 172.21.15.133:0/471896871 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057dceb640 1 --2- 172.21.15.133:0/471896871 >> v2:172.21.15.162:3300/0 conn(0x7f05780ce330 0x7f05780ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844386335 0 0) 0x7f05780c61d0 con 0x7f057815c9a0 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0564003530 con 0x7f057815c9a0 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 198545999 0 0) 0x7f0564003530 con 0x7f057815c9a0 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 >> v2:172.21.15.162:3300/0 conn(0x7f05780ce330 msgr2=0x7f05780ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 --2- 172.21.15.133:0/471896871 >> v2:172.21.15.162:3300/0 conn(0x7f05780ce330 0x7f05780ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 >> v1:172.21.15.162:6789/0 conn(0x7f05780b1f70 legacy=0x7f057806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057806d4f0 con 0x7f057815c9a0 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0574001680 con 0x7f057815c9a0 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 >> v1:172.21.15.133:6789/0 conn(0x7f057815c9a0 legacy=0x7f05780c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 --2- 172.21.15.133:0/471896871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05640039c0 0x7f0564003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 --2- 172.21.15.133:0/471896871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0564004810 0x7f0564004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 --2- 172.21.15.133:0/471896871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05640051d0 0x7f056400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05780cecc0 con 0x7f0564004810 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056400c070 con 0x7f05640051d0 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056400c270 con 0x7f05640039c0 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057cce9640 1 --2- 172.21.15.133:0/471896871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0564004810 0x7f0564004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057dceb640 1 --2- 172.21.15.133:0/471896871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05640051d0 0x7f056400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057cce9640 1 -- 172.21.15.133:0/471896871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05640039c0 msgr2=0x7f0564003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057cce9640 1 --2- 172.21.15.133:0/471896871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05640039c0 0x7f0564003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057cce9640 1 -- 172.21.15.133:0/471896871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05640051d0 msgr2=0x7f056400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057cce9640 1 --2- 172.21.15.133:0/471896871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05640051d0 0x7f056400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057cce9640 1 -- 172.21.15.133:0/471896871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0564003530 con 0x7f0564004810 2024-04-25T10:10:22.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057dceb640 1 --2- 172.21.15.133:0/471896871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05640051d0 0x7f056400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:22.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.006+0000 7f057dceb640 1 -- 172.21.15.133:0/471896871 reap_dead start 2024-04-25T10:10:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057cce9640 1 --2- 172.21.15.133:0/471896871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0564004810 0x7f0564004c90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f0568001780 tx=0x7f056801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772f75ef3d13683b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680013a0 con 0x7f0564004810 2024-04-25T10:10:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05680052f0 con 0x7f0564004810 2024-04-25T10:10:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f056ffff640 1 -- 172.21.15.133:0/471896871 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568003840 con 0x7f0564004810 2024-04-25T10:10:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 -- 172.21.15.133:0/471896871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0564004810 msgr2=0x7f0564004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 --2- 172.21.15.133:0/471896871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0564004810 0x7f0564004c90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f0568001780 tx=0x7f056801c400 comp rx=0 tx=0).stop 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 -- 172.21.15.133:0/471896871 shutdown_connections 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 --2- 172.21.15.133:0/471896871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0564004810 0x7f0564004c90 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 -- 172.21.15.133:0/471896871 >> 172.21.15.133:0/471896871 conn(0x7f0578067000 msgr2=0x7f057806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 -- 172.21.15.133:0/471896871 shutdown_connections 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 -- 172.21.15.133:0/471896871 wait complete. 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 Processor -- start 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 -- start start 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0564004810 0x7f05781f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05780b3600 0x7f05780c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05780b3a80 0x7f05780c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057815cf00 con 0x7f05780b3600 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057815cd80 con 0x7f0564004810 2024-04-25T10:10:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05780ce780 con 0x7f05780b3a80 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057cce9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05780b3600 0x7f05780c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057cce9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05780b3600 0x7f05780c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39030/0 (socket says 172.21.15.133:39030) 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057cce9640 1 -- 172.21.15.133:0/3696143845 learned_addr learned my addr 172.21.15.133:0/3696143845 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057d4ea640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0564004810 0x7f05781f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057dceb640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05780b3a80 0x7f05780c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057cce9640 1 -- 172.21.15.133:0/3696143845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05780b3a80 msgr2=0x7f05780c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057cce9640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05780b3a80 0x7f05780c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057cce9640 1 -- 172.21.15.133:0/3696143845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0564004810 msgr2=0x7f05781f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057cce9640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0564004810 0x7f05781f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057cce9640 1 -- 172.21.15.133:0/3696143845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05781f6cc0 con 0x7f05780b3600 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057dceb640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05780b3a80 0x7f05780c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057d4ea640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0564004810 0x7f05781f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057cce9640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05780b3600 0x7f05780c9370 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f056801c230 tx=0x7f0564005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f056dffb640 1 -- 172.21.15.133:0/3696143845 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568007a40 con 0x7f05780b3600 2024-04-25T10:10:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f056dffb640 1 -- 172.21.15.133:0/3696143845 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05680ba070 con 0x7f05780b3600 2024-04-25T10:10:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 -- 172.21.15.133:0/3696143845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05780c9e30 con 0x7f05780b3600 2024-04-25T10:10:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f056dffb640 1 -- 172.21.15.133:0/3696143845 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568007250 con 0x7f05780b3600 2024-04-25T10:10:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.010+0000 7f057f74e640 1 -- 172.21.15.133:0/3696143845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05781f6f60 con 0x7f05780b3600 2024-04-25T10:10:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.014+0000 7f057f74e640 1 -- 172.21.15.133:0/3696143845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05781f7390 con 0x7f05780b3600 2024-04-25T10:10:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.014+0000 7f056dffb640 1 -- 172.21.15.133:0/3696143845 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05680052f0 con 0x7f05780b3600 2024-04-25T10:10:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.014+0000 7f056dffb640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f055c066f60 0x7f055c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.014+0000 7f056dffb640 1 -- 172.21.15.133:0/3696143845 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f056811f200 con 0x7f05780b3600 2024-04-25T10:10:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.014+0000 7f057d4ea640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f055c066f60 0x7f055c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.014+0000 7f057d4ea640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f055c066f60 0x7f055c069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0574001590 tx=0x7f05740016a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.018+0000 7f056dffb640 1 -- 172.21.15.133:0/3696143845 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0568016110 con 0x7f05780b3600 2024-04-25T10:10:22.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.218+0000 7f057f74e640 1 -- 172.21.15.133:0/3696143845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f0578068ef0 con 0x7f05780b3600 2024-04-25T10:10:22.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.230+0000 7f056dffb640 1 -- 172.21.15.133:0/3696143845 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v377) v1 ==== 147+0+0 (secure 0 0 0) 0x7f05680f0410 con 0x7f05780b3600 2024-04-25T10:10:22.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 -- 172.21.15.133:0/3696143845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f055c066f60 msgr2=0x7f055c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f055c066f60 0x7f055c069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0574001590 tx=0x7f05740016a0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 -- 172.21.15.133:0/3696143845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05780b3600 msgr2=0x7f05780c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05780b3600 0x7f05780c9370 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f056801c230 tx=0x7f0564005840 comp rx=0 tx=0).stop 2024-04-25T10:10:22.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 -- 172.21.15.133:0/3696143845 shutdown_connections 2024-04-25T10:10:22.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05780b3a80 0x7f05780c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f055c066f60 0x7f055c069420 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05780b3600 0x7f05780c9370 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 --2- 172.21.15.133:0/3696143845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0564004810 0x7f05781f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 -- 172.21.15.133:0/3696143845 >> 172.21.15.133:0/3696143845 conn(0x7f0578067000 msgr2=0x7f05780c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 -- 172.21.15.133:0/3696143845 shutdown_connections 2024-04-25T10:10:22.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.234+0000 7f057f74e640 1 -- 172.21.15.133:0/3696143845 wait complete. 2024-04-25T10:10:22.257 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:10:22.273 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:22.305 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:22.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6bb184d640 1 Processor -- start 2024-04-25T10:10:22.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6bb184d640 1 -- start start 2024-04-25T10:10:22.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6bb184d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6bac15f020 0x7f6bac0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6bb184d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6bac0d21c0 0x7f6bac0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6bb184d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac0ccdf0 con 0x7f6bac0cfcf0 2024-04-25T10:10:22.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6bb184d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac0cd690 con 0x7f6bac15f020 2024-04-25T10:10:22.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6bb184d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac0cd0d0 con 0x7f6bac0d21c0 2024-04-25T10:10:22.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6baaffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6bac15f020 0x7f6bac0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6baaffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6bac15f020 0x7f6bac0cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39038/0 (socket says 172.21.15.133:39038) 2024-04-25T10:10:22.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6baaffd640 1 -- 172.21.15.133:0/625051131 learned_addr learned my addr 172.21.15.133:0/625051131 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:22.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6baa7fc640 1 --2- 172.21.15.133:0/625051131 >> v2:172.21.15.162:3300/0 conn(0x7f6bac0d21c0 0x7f6bac0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6baaffd640 1 -- 172.21.15.133:0/625051131 >> v2:172.21.15.162:3300/0 conn(0x7f6bac0d21c0 msgr2=0x7f6bac0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6baaffd640 1 --2- 172.21.15.133:0/625051131 >> v2:172.21.15.162:3300/0 conn(0x7f6bac0d21c0 0x7f6bac0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6baaffd640 1 -- 172.21.15.133:0/625051131 >> v1:172.21.15.162:6789/0 conn(0x7f6bac0cfcf0 legacy=0x7f6bac0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.374+0000 7f6baaffd640 1 -- 172.21.15.133:0/625051131 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac0cd3b0 con 0x7f6bac15f020 2024-04-25T10:10:22.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baaffd640 1 --2- 172.21.15.133:0/625051131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bac15f020 0x7f6bac0cf270 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6bac06f190 tx=0x7f6b9401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b012557b470da9ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6ba9ffb640 1 -- 172.21.15.133:0/625051131 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b940ab5f0 con 0x7f6bac15f020 2024-04-25T10:10:22.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6ba9ffb640 1 -- 172.21.15.133:0/625051131 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b940b0070 con 0x7f6bac15f020 2024-04-25T10:10:22.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6ba9ffb640 1 -- 172.21.15.133:0/625051131 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b940ab9b0 con 0x7f6bac15f020 2024-04-25T10:10:22.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 -- 172.21.15.133:0/625051131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bac15f020 msgr2=0x7f6bac0cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 --2- 172.21.15.133:0/625051131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bac15f020 0x7f6bac0cf270 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6bac06f190 tx=0x7f6b9401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 -- 172.21.15.133:0/625051131 shutdown_connections 2024-04-25T10:10:22.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 --2- 172.21.15.133:0/625051131 >> v2:172.21.15.162:3300/0 conn(0x7f6bac0d21c0 0x7f6bac0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 --2- 172.21.15.133:0/625051131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bac15f020 0x7f6bac0cf270 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 -- 172.21.15.133:0/625051131 >> 172.21.15.133:0/625051131 conn(0x7f6bac0d16a0 msgr2=0x7f6bac0d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 -- 172.21.15.133:0/625051131 shutdown_connections 2024-04-25T10:10:22.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 -- 172.21.15.133:0/625051131 wait complete. 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 Processor -- start 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 -- start start 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac0cfcf0 0x7f6bac14f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bac0d21c0 0x7f6bac0c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bac15f020 0x7f6bac0c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac1688f0 con 0x7f6bac0d21c0 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac168770 con 0x7f6bac0cfcf0 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bb184d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac168a70 con 0x7f6bac15f020 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baa7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bac0d21c0 0x7f6bac0c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baa7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bac0d21c0 0x7f6bac0c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39050/0 (socket says 172.21.15.133:39050) 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baa7fc640 1 -- 172.21.15.133:0/3685971606 learned_addr learned my addr 172.21.15.133:0/3685971606 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:22.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baaffd640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac0cfcf0 0x7f6bac14f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bab7fe640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bac15f020 0x7f6bac0c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baa7fc640 1 -- 172.21.15.133:0/3685971606 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bac15f020 msgr2=0x7f6bac0c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baa7fc640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bac15f020 0x7f6bac0c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baa7fc640 1 -- 172.21.15.133:0/3685971606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac0cfcf0 msgr2=0x7f6bac14f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baa7fc640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac0cfcf0 0x7f6bac14f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baa7fc640 1 -- 172.21.15.133:0/3685971606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac14f740 con 0x7f6bac0d21c0 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6bab7fe640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bac15f020 0x7f6bac0c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baaffd640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac0cfcf0 0x7f6bac14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6baa7fc640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bac0d21c0 0x7f6bac0c6300 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00d800 tx=0x7f6b9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6ba3fff640 1 -- 172.21.15.133:0/3685971606 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c00b660 con 0x7f6bac0d21c0 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6ba3fff640 1 -- 172.21.15.133:0/3685971606 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c019070 con 0x7f6bac0d21c0 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.378+0000 7f6ba3fff640 1 -- 172.21.15.133:0/3685971606 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c00ba80 con 0x7f6bac0d21c0 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.382+0000 7f6bb184d640 1 -- 172.21.15.133:0/3685971606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac14e160 con 0x7f6bac0d21c0 2024-04-25T10:10:22.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.382+0000 7f6bb184d640 1 -- 172.21.15.133:0/3685971606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac14e720 con 0x7f6bac0d21c0 2024-04-25T10:10:22.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.382+0000 7f6bb184d640 1 -- 172.21.15.133:0/3685971606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac067c70 con 0x7f6bac0d21c0 2024-04-25T10:10:22.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.382+0000 7f6ba3fff640 1 -- 172.21.15.133:0/3685971606 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b9c00bc20 con 0x7f6bac0d21c0 2024-04-25T10:10:22.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.382+0000 7f6ba3fff640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b8c066f60 0x7f6b8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.382+0000 7f6ba3fff640 1 -- 172.21.15.133:0/3685971606 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6b9c085330 con 0x7f6bac0d21c0 2024-04-25T10:10:22.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.382+0000 7f6baaffd640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b8c066f60 0x7f6b8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.382+0000 7f6baaffd640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b8c066f60 0x7f6b8c069420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6b940af000 tx=0x7f6b940a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.386+0000 7f6ba3fff640 1 -- 172.21.15.133:0/3685971606 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b9c0564b0 con 0x7f6bac0d21c0 2024-04-25T10:10:22.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.542+0000 7f6bb184d640 1 -- 172.21.15.133:0/3685971606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6bac14ee90 con 0x7f6bac0d21c0 2024-04-25T10:10:22.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6ba3fff640 1 -- 172.21.15.133:0/3685971606 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6b9c05ae20 con 0x7f6bac0d21c0 2024-04-25T10:10:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 -- 172.21.15.133:0/3685971606 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b8c066f60 msgr2=0x7f6b8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b8c066f60 0x7f6b8c069420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6b940af000 tx=0x7f6b940a7000 comp rx=0 tx=0).stop 2024-04-25T10:10:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 -- 172.21.15.133:0/3685971606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bac0d21c0 msgr2=0x7f6bac0c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bac0d21c0 0x7f6bac0c6300 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00d800 tx=0x7f6b9c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 -- 172.21.15.133:0/3685971606 shutdown_connections 2024-04-25T10:10:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6bac15f020 0x7f6bac0c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b8c066f60 0x7f6b8c069420 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bac0d21c0 0x7f6bac0c6300 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 --2- 172.21.15.133:0/3685971606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6bac0cfcf0 0x7f6bac14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 -- 172.21.15.133:0/3685971606 >> 172.21.15.133:0/3685971606 conn(0x7f6bac0d16a0 msgr2=0x7f6bac0cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 -- 172.21.15.133:0/3685971606 shutdown_connections 2024-04-25T10:10:22.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.550+0000 7f6bb184d640 1 -- 172.21.15.133:0/3685971606 wait complete. 2024-04-25T10:10:22.575 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:10:22.575 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:22.662 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:22.737 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:10:22.737 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:10:22.737 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:22.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:10:22.743 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:10:22.743 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:22.749 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:22.749 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:10:22.750 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:22.792 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:10:22.793 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:10:22.793 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:22.798 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:22.798 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:10:22.798 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:22.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.950+0000 7fbcd4e91640 1 Processor -- start 2024-04-25T10:10:22.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.950+0000 7fbcd4e91640 1 -- start start 2024-04-25T10:10:22.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcd4e91640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbcd00b1f70 0x7fbcd006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcd4e91640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd00c61d0 con 0x7fbcd00ce330 2024-04-25T10:10:22.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcd4e91640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd00c64b0 con 0x7fbcd015c9a0 2024-04-25T10:10:22.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcd4e91640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd006d4f0 con 0x7fbcd00b1f70 2024-04-25T10:10:22.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcc5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbcd00b1f70 0x7fbcd006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcced9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fbcd00ce330 0x7fbcd015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50936/0 (socket says 172.21.15.133:50936) 2024-04-25T10:10:22.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcced9d640 1 -- 172.21.15.133:0/1681915705 learned_addr learned my addr 172.21.15.133:0/1681915705 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:22.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcc5d9b640 1 -- 172.21.15.133:0/1681915705 >> v1:172.21.15.162:6789/0 conn(0x7fbcd015c9a0 legacy=0x7fbcd00c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcc5d9b640 1 -- 172.21.15.133:0/1681915705 >> v1:172.21.15.133:6789/0 conn(0x7fbcd00ce330 legacy=0x7fbcd015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbccdd9b640 1 -- 172.21.15.133:0/1681915705 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084207681 0 0) 0x7fbcd00c61d0 con 0x7fbcd00ce330 2024-04-25T10:10:22.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcc5d9b640 1 -- 172.21.15.133:0/1681915705 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd006d7d0 con 0x7fbcd00b1f70 2024-04-25T10:10:22.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcc5d9b640 1 --2- 172.21.15.133:0/1681915705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd00b1f70 0x7fbcd006dcf0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80097c0 tx=0x7fbcb801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3448a2ed485f5a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbccdd9b640 1 -- 172.21.15.133:0/1681915705 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80ab680 con 0x7fbcd00b1f70 2024-04-25T10:10:22.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbccdd9b640 1 -- 172.21.15.133:0/1681915705 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb80b0070 con 0x7fbcd00b1f70 2024-04-25T10:10:22.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbccdd9b640 1 -- 172.21.15.133:0/1681915705 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80aba40 con 0x7fbcd00b1f70 2024-04-25T10:10:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcd4e91640 1 -- 172.21.15.133:0/1681915705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd00b1f70 msgr2=0x7fbcd006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcd4e91640 1 --2- 172.21.15.133:0/1681915705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd00b1f70 0x7fbcd006dcf0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80097c0 tx=0x7fbcb801c460 comp rx=0 tx=0).stop 2024-04-25T10:10:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcd4e91640 1 -- 172.21.15.133:0/1681915705 shutdown_connections 2024-04-25T10:10:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.954+0000 7fbcd4e91640 1 --2- 172.21.15.133:0/1681915705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd00b1f70 0x7fbcd006dcf0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 -- 172.21.15.133:0/1681915705 >> 172.21.15.133:0/1681915705 conn(0x7fbcd0067000 msgr2=0x7fbcd006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 -- 172.21.15.133:0/1681915705 shutdown_connections 2024-04-25T10:10:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 -- 172.21.15.133:0/1681915705 wait complete. 2024-04-25T10:10:22.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 Processor -- start 2024-04-25T10:10:22.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 -- start start 2024-04-25T10:10:22.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd00b1f70 0x7fbcd00c4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd00ce330 0x7fbcd014a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd015c9a0 0x7fbcd0145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd015cf00 con 0x7fbcd00b1f70 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd015cd80 con 0x7fbcd015c9a0 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd0169de0 con 0x7fbcd00ce330 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcce59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd00b1f70 0x7fbcd00c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcce59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd00b1f70 0x7fbcd00c4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39068/0 (socket says 172.21.15.133:39068) 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcce59c640 1 -- 172.21.15.133:0/3586728945 learned_addr learned my addr 172.21.15.133:0/3586728945 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcced9d640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd015c9a0 0x7fbcd0145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcce59c640 1 -- 172.21.15.133:0/3586728945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd00ce330 msgr2=0x7fbcd014a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcce59c640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd00ce330 0x7fbcd014a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcce59c640 1 -- 172.21.15.133:0/3586728945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd015c9a0 msgr2=0x7fbcd0145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcce59c640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd015c9a0 0x7fbcd0145b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcce59c640 1 -- 172.21.15.133:0/3586728945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd014aa90 con 0x7fbcd00b1f70 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcced9d640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd015c9a0 0x7fbcd0145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:22.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcce59c640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd00b1f70 0x7fbcd00c4910 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000c480 tx=0x7fbcc000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcc77fe640 1 -- 172.21.15.133:0/3586728945 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0012830 con 0x7fbcd00b1f70 2024-04-25T10:10:22.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcc77fe640 1 -- 172.21.15.133:0/3586728945 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc0017070 con 0x7fbcd00b1f70 2024-04-25T10:10:22.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 -- 172.21.15.133:0/3586728945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd014ad70 con 0x7fbcd00b1f70 2024-04-25T10:10:22.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcc77fe640 1 -- 172.21.15.133:0/3586728945 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0012c30 con 0x7fbcd00b1f70 2024-04-25T10:10:22.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 -- 172.21.15.133:0/3586728945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd014b330 con 0x7fbcd00b1f70 2024-04-25T10:10:22.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.958+0000 7fbcd4e91640 1 -- 172.21.15.133:0/3586728945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd0068ef0 con 0x7fbcd00b1f70 2024-04-25T10:10:22.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.962+0000 7fbcc77fe640 1 -- 172.21.15.133:0/3586728945 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbcc001c460 con 0x7fbcd00b1f70 2024-04-25T10:10:22.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.962+0000 7fbcc77fe640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcb0066f60 0x7fbcb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.962+0000 7fbcc77fe640 1 -- 172.21.15.133:0/3586728945 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbcc0084d80 con 0x7fbcd00b1f70 2024-04-25T10:10:22.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.962+0000 7fbcc5d9b640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcb0066f60 0x7fbcb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.962+0000 7fbcc5d9b640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcb0066f60 0x7fbcb0069420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80af000 tx=0x7fbcb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:22.966+0000 7fbcc77fe640 1 -- 172.21.15.133:0/3586728945 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbcc0055ef0 con 0x7fbcd00b1f70 2024-04-25T10:10:23.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.158+0000 7fbcd4e91640 1 -- 172.21.15.133:0/3586728945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcd00c5e00 con 0x7fbcd00b1f70 2024-04-25T10:10:23.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.158+0000 7fbcc77fe640 1 -- 172.21.15.133:0/3586728945 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fbcc00557f0 con 0x7fbcd00b1f70 2024-04-25T10:10:23.166 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:23.166 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5465,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3522294955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3522294955},{"type":"v1","addr":"172.21.15.162:6833","nonce":3522294955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5600,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/580731464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":580731464},{"type":"v1","addr":"172.21.15.162:6835","nonce":580731464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3865231260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3865231260},{"type":"v1","addr":"172.21.15.133:6836","nonce":3865231260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7437,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3935311931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3935311931},{"type":"v1","addr":"172.21.15.133:6837","nonce":3935311931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[]} 2024-04-25T10:10:23.166 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 52 2024-04-25T10:10:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 -- 172.21.15.133:0/3586728945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcb0066f60 msgr2=0x7fbcb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcb0066f60 0x7fbcb0069420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80af000 tx=0x7fbcb8008040 comp rx=0 tx=0).stop 2024-04-25T10:10:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 -- 172.21.15.133:0/3586728945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd00b1f70 msgr2=0x7fbcd00c4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd00b1f70 0x7fbcd00c4910 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000c480 tx=0x7fbcc000c940 comp rx=0 tx=0).stop 2024-04-25T10:10:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 -- 172.21.15.133:0/3586728945 shutdown_connections 2024-04-25T10:10:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbcd015c9a0 0x7fbcd0145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbcd00ce330 0x7fbcd014a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbcb0066f60 0x7fbcb0069420 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 --2- 172.21.15.133:0/3586728945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcd00b1f70 0x7fbcd00c4910 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 -- 172.21.15.133:0/3586728945 >> 172.21.15.133:0/3586728945 conn(0x7fbcd0067000 msgr2=0x7fbcd015e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 -- 172.21.15.133:0/3586728945 shutdown_connections 2024-04-25T10:10:23.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.162+0000 7fbcd4e91640 1 -- 172.21.15.133:0/3586728945 wait complete. 2024-04-25T10:10:23.184 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:10:23.184 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:10:23.184 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:10:23.184 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:23.184 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:10:23.184 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:10:23.185 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:10:23.185 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:23.185 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:10:23.185 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:10:23.185 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:10:23.185 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:23.185 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:10:23.186 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:10:23.186 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:10:23.186 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:23.193 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:10:23.193 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:10:23.196 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:10:23.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:10:23.198 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:10:23.199 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:10:23.202 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:10:23.202 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:10:23.240 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:10:23.240 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:23.241 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:10:23.241 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:23.248 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:10:23.248 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:23.249 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:10:23.249 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:23.250 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:10:23.299 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:10:23.299 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:10:23.319 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:10:23.319 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:10:23.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81eac4d640 1 Processor -- start 2024-04-25T10:10:23.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81eac4d640 1 -- start start 2024-04-25T10:10:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81eac4d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f81e415dab0 0x7f81e40cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81eac4d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e415de90 con 0x7f81e40d1e00 2024-04-25T10:10:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81eac4d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e415e160 con 0x7f81e406d4d0 2024-04-25T10:10:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81eac4d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e40b1150 con 0x7f81e415dab0 2024-04-25T10:10:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81e89e9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f81e415dab0 0x7f81e40cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81e91ea640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f81e40d1e00 0x7f81e40cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50948/0 (socket says 172.21.15.133:50948) 2024-04-25T10:10:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81e91ea640 1 -- 172.21.15.133:0/3864752255 learned_addr learned my addr 172.21.15.133:0/3864752255 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81e89e9640 1 -- 172.21.15.133:0/3864752255 >> v1:172.21.15.162:6789/0 conn(0x7f81e406d4d0 legacy=0x7f81e40d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81e89e9640 1 -- 172.21.15.133:0/3864752255 >> v1:172.21.15.133:6789/0 conn(0x7f81e40d1e00 legacy=0x7f81e40cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.354+0000 7f81e89e9640 1 -- 172.21.15.133:0/3864752255 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e415e440 con 0x7f81e415dab0 2024-04-25T10:10:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81e89e9640 1 --2- 172.21.15.133:0/3864752255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e415dab0 0x7f81e40cdf50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0097c0 tx=0x7f81dc01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf959a7e4f853aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81e37fe640 1 -- 172.21.15.133:0/3864752255 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0b5070 con 0x7f81e415dab0 2024-04-25T10:10:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81e37fe640 1 -- 172.21.15.133:0/3864752255 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc0a8950 con 0x7f81e415dab0 2024-04-25T10:10:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81e37fe640 1 -- 172.21.15.133:0/3864752255 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0a8d20 con 0x7f81e415dab0 2024-04-25T10:10:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 -- 172.21.15.133:0/3864752255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e415dab0 msgr2=0x7f81e40cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 --2- 172.21.15.133:0/3864752255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e415dab0 0x7f81e40cdf50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0097c0 tx=0x7f81dc01ec50 comp rx=0 tx=0).stop 2024-04-25T10:10:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 -- 172.21.15.133:0/3864752255 shutdown_connections 2024-04-25T10:10:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 --2- 172.21.15.133:0/3864752255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e415dab0 0x7f81e40cdf50 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 -- 172.21.15.133:0/3864752255 >> 172.21.15.133:0/3864752255 conn(0x7f81e40b3370 msgr2=0x7f81e40b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 -- 172.21.15.133:0/3864752255 shutdown_connections 2024-04-25T10:10:23.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 -- 172.21.15.133:0/3864752255 wait complete. 2024-04-25T10:10:23.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 Processor -- start 2024-04-25T10:10:23.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 -- start start 2024-04-25T10:10:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81e406d4d0 0x7f81e4156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e40d1e00 0x7f81e40ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e415dab0 0x7f81e40cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e415ea00 con 0x7f81e415dab0 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e415ed70 con 0x7f81e40d1e00 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81eac4d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40cd9a0 con 0x7f81e406d4d0 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81e91ea640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e415dab0 0x7f81e40cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81e91ea640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e415dab0 0x7f81e40cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39124/0 (socket says 172.21.15.133:39124) 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.358+0000 7f81e91ea640 1 -- 172.21.15.133:0/1189746660 learned_addr learned my addr 172.21.15.133:0/1189746660 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e3fff640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e40d1e00 0x7f81e40ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e89e9640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81e406d4d0 0x7f81e4156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e91ea640 1 -- 172.21.15.133:0/1189746660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81e406d4d0 msgr2=0x7f81e4156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e91ea640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81e406d4d0 0x7f81e4156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e91ea640 1 -- 172.21.15.133:0/1189746660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e40d1e00 msgr2=0x7f81e40ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e91ea640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e40d1e00 0x7f81e40ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e91ea640 1 -- 172.21.15.133:0/1189746660 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e415f120 con 0x7f81e415dab0 2024-04-25T10:10:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e91ea640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e415dab0 0x7f81e40cec00 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f81d400cbc0 tx=0x7f81d400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e17fa640 1 -- 172.21.15.133:0/1189746660 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4017070 con 0x7f81e415dab0 2024-04-25T10:10:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e17fa640 1 -- 172.21.15.133:0/1189746660 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d400ad40 con 0x7f81e415dab0 2024-04-25T10:10:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81eac4d640 1 -- 172.21.15.133:0/1189746660 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e415f3c0 con 0x7f81e415dab0 2024-04-25T10:10:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e17fa640 1 -- 172.21.15.133:0/1189746660 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4012600 con 0x7f81e415dab0 2024-04-25T10:10:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81eac4d640 1 -- 172.21.15.133:0/1189746660 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e415f980 con 0x7f81e415dab0 2024-04-25T10:10:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81eac4d640 1 -- 172.21.15.133:0/1189746660 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e4068e80 con 0x7f81e415dab0 2024-04-25T10:10:23.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e17fa640 1 -- 172.21.15.133:0/1189746660 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81d401a070 con 0x7f81e415dab0 2024-04-25T10:10:23.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e17fa640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81d0066f60 0x7f81d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e17fa640 1 -- 172.21.15.133:0/1189746660 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f81d4084f20 con 0x7f81e415dab0 2024-04-25T10:10:23.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e89e9640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81d0066f60 0x7f81d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.362+0000 7f81e89e9640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81d0066f60 0x7f81d0069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0b2000 tx=0x7f81dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.366+0000 7f81e17fa640 1 -- 172.21.15.133:0/1189746660 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81d4055b30 con 0x7f81e415dab0 2024-04-25T10:10:23.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:23.522+0000 7f81eac4d640 1 -- 172.21.15.133:0/1189746660 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f81e40d2660 con 0x7f81e415dab0 2024-04-25T10:10:24.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.250+0000 7f81e17fa640 1 -- 172.21.15.133:0/1189746660 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) v1 ==== 97+0+0 (secure 0 0 0) 0x7f81d4059bd0 con 0x7f81e415dab0 2024-04-25T10:10:24.259 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 -- 172.21.15.133:0/1189746660 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81d0066f60 msgr2=0x7f81d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81d0066f60 0x7f81d0069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0b2000 tx=0x7f81dc008040 comp rx=0 tx=0).stop 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 -- 172.21.15.133:0/1189746660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e415dab0 msgr2=0x7f81e40cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e415dab0 0x7f81e40cec00 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f81d400cbc0 tx=0x7f81d400a430 comp rx=0 tx=0).stop 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 -- 172.21.15.133:0/1189746660 shutdown_connections 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81d0066f60 0x7f81d0069420 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e415dab0 0x7f81e40cec00 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e40d1e00 0x7f81e40ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 --2- 172.21.15.133:0/1189746660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81e406d4d0 0x7f81e4156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 -- 172.21.15.133:0/1189746660 >> 172.21.15.133:0/1189746660 conn(0x7f81e40b3370 msgr2=0x7f81e4159190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 -- 172.21.15.133:0/1189746660 shutdown_connections 2024-04-25T10:10:24.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.254+0000 7f81eac4d640 1 -- 172.21.15.133:0/1189746660 wait complete. 2024-04-25T10:10:24.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:10:24.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.402+0000 7f4d7733e640 1 Processor -- start 2024-04-25T10:10:24.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.402+0000 7f4d7733e640 1 -- start start 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d7733e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70152b30 0x7f4d700b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d7733e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7006d350 con 0x7f4d700cea40 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d7733e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7015bf90 con 0x7f4d70152750 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d7733e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7006d5d0 con 0x7f4d70152b30 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d758db640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4d700cea40 0x7f4d700b1cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50958/0 (socket says 172.21.15.133:50958) 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d758db640 1 -- 172.21.15.133:0/64883954 learned_addr learned my addr 172.21.15.133:0/64883954 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176108739 0 0) 0x7f4d680014a0 con 0x7f4d700cea40 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d5c003530 con 0x7f4d700cea40 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d748d9640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70152b30 0x7f4d700b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 390969401 0 0) 0x7f4d5c003530 con 0x7f4d700cea40 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70152b30 msgr2=0x7f4d700b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70152b30 0x7f4d700b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 >> v1:172.21.15.162:6789/0 conn(0x7f4d70152750 legacy=0x7f4d700ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7015c230 con 0x7f4d700cea40 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4d68001bd0 con 0x7f4d700cea40 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 >> v1:172.21.15.133:6789/0 conn(0x7f4d700cea40 legacy=0x7f4d700b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70152b30 0x7f4d5c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d5c004020 0x7f4d5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5c004d50 0x7f4d5c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7006d5d0 con 0x7f4d5c004020 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c00bb90 con 0x7f4d5c004d50 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c00bd90 con 0x7f4d70152b30 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d748d9640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d5c004020 0x7f4d5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d750da640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70152b30 0x7f4d5c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d758db640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5c004d50 0x7f4d5c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d748d9640 1 -- 172.21.15.133:0/64883954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70152b30 msgr2=0x7f4d5c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d748d9640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70152b30 0x7f4d5c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d748d9640 1 -- 172.21.15.133:0/64883954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5c004d50 msgr2=0x7f4d5c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d748d9640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5c004d50 0x7f4d5c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d748d9640 1 -- 172.21.15.133:0/64883954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c003530 con 0x7f4d5c004020 2024-04-25T10:10:24.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d748d9640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d5c004020 0x7f4d5c004810 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000ba30 tx=0x7f4d6001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871031c754298e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d600ae070 con 0x7f4d5c004020 2024-04-25T10:10:24.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d600a9680 con 0x7f4d5c004020 2024-04-25T10:10:24.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.406+0000 7f4d67fff640 1 -- 172.21.15.133:0/64883954 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000b600 con 0x7f4d5c004020 2024-04-25T10:10:24.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 -- 172.21.15.133:0/64883954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d5c004020 msgr2=0x7f4d5c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 --2- 172.21.15.133:0/64883954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d5c004020 0x7f4d5c004810 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000ba30 tx=0x7f4d6001c6a0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d758db640 1 -- 172.21.15.133:0/64883954 reap_dead start 2024-04-25T10:10:24.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 -- 172.21.15.133:0/64883954 shutdown_connections 2024-04-25T10:10:24.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 -- 172.21.15.133:0/64883954 >> 172.21.15.133:0/64883954 conn(0x7f4d70067000 msgr2=0x7f4d7006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:24.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 -- 172.21.15.133:0/64883954 shutdown_connections 2024-04-25T10:10:24.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 -- 172.21.15.133:0/64883954 wait complete. 2024-04-25T10:10:24.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 Processor -- start 2024-04-25T10:10:24.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 -- start start 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5c004020 0x7f4d7015e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70156490 0x7f4d700ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d7013bbc0 0x7f4d7013c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d700b17b0 con 0x7f4d7013bbc0 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7015c7f0 con 0x7f4d5c004020 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d700b1930 con 0x7f4d70156490 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d758db640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d7013bbc0 0x7f4d7013c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d758db640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d7013bbc0 0x7f4d7013c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39142/0 (socket says 172.21.15.133:39142) 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d758db640 1 -- 172.21.15.133:0/3030968083 learned_addr learned my addr 172.21.15.133:0/3030968083 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d750da640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5c004020 0x7f4d7015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d748d9640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70156490 0x7f4d700ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d758db640 1 -- 172.21.15.133:0/3030968083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70156490 msgr2=0x7f4d700ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d758db640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70156490 0x7f4d700ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d758db640 1 -- 172.21.15.133:0/3030968083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5c004020 msgr2=0x7f4d7015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d758db640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5c004020 0x7f4d7015e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d758db640 1 -- 172.21.15.133:0/3030968083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7015d900 con 0x7f4d7013bbc0 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d758db640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d7013bbc0 0x7f4d7013c040 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f4d68004c00 tx=0x7f4d68010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d65ffb640 1 -- 172.21.15.133:0/3030968083 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6801a070 con 0x7f4d7013bbc0 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d65ffb640 1 -- 172.21.15.133:0/3030968083 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d68011040 con 0x7f4d7013bbc0 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 -- 172.21.15.133:0/3030968083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d70156a50 con 0x7f4d7013bbc0 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d65ffb640 1 -- 172.21.15.133:0/3030968083 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800ed20 con 0x7f4d7013bbc0 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.410+0000 7f4d7733e640 1 -- 172.21.15.133:0/3030968083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7015de80 con 0x7f4d7013bbc0 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.414+0000 7f4d7733e640 1 -- 172.21.15.133:0/3030968083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d70068ef0 con 0x7f4d7013bbc0 2024-04-25T10:10:24.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.414+0000 7f4d65ffb640 1 -- 172.21.15.133:0/3030968083 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d6801d070 con 0x7f4d7013bbc0 2024-04-25T10:10:24.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.414+0000 7f4d65ffb640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d54066f60 0x7f4d54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.414+0000 7f4d65ffb640 1 -- 172.21.15.133:0/3030968083 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4d68087630 con 0x7f4d7013bbc0 2024-04-25T10:10:24.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.414+0000 7f4d750da640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d54066f60 0x7f4d54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.414+0000 7f4d750da640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d54066f60 0x7f4d54069420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c005510 tx=0x7f4d6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.418+0000 7f4d65ffb640 1 -- 172.21.15.133:0/3030968083 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d7006d350 con 0x7f4d7013bbc0 2024-04-25T10:10:24.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.638+0000 7f4d7733e640 1 -- 172.21.15.133:0/3030968083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4d7006e180 con 0x7f4d7013bbc0 2024-04-25T10:10:24.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.638+0000 7f4d65ffb640 1 -- 172.21.15.133:0/3030968083 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4d68058800 con 0x7f4d7013bbc0 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.647 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:10:24.648 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:10:24.649 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.650 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.651 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.652 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.652 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:10:24.652 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:10:24.652 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.652 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:24.652 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:10:24.652 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.652 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.653 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.653 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:10:24.653 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:10:24.653 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.653 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:24.653 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:10:24.653 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.653 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.653 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.653 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.654 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.655 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:10:24.656 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.657 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:10:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:10:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:10:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:10:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:10:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:10:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.658 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.659 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:10:24.660 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:10:24.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 -- 172.21.15.133:0/3030968083 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d54066f60 msgr2=0x7f4d54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d54066f60 0x7f4d54069420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c005510 tx=0x7f4d6c009040 comp rx=0 tx=0).stop 2024-04-25T10:10:24.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 -- 172.21.15.133:0/3030968083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d7013bbc0 msgr2=0x7f4d7013c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d7013bbc0 0x7f4d7013c040 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f4d68004c00 tx=0x7f4d68010d70 comp rx=0 tx=0).stop 2024-04-25T10:10:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 -- 172.21.15.133:0/3030968083 shutdown_connections 2024-04-25T10:10:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d54066f60 0x7f4d54069420 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d7013bbc0 0x7f4d7013c040 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d70156490 0x7f4d700ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 --2- 172.21.15.133:0/3030968083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d5c004020 0x7f4d7015e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 -- 172.21.15.133:0/3030968083 >> 172.21.15.133:0/3030968083 conn(0x7f4d70067000 msgr2=0x7f4d7006e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 -- 172.21.15.133:0/3030968083 shutdown_connections 2024-04-25T10:10:24.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.642+0000 7f4d7733e640 1 -- 172.21.15.133:0/3030968083 wait complete. 2024-04-25T10:10:24.666 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:24.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0319adc640 1 Processor -- start 2024-04-25T10:10:24.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0319adc640 1 -- start start 2024-04-25T10:10:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0319adc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03140d0770 0x7f03140d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0319adc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03140b87e0 0x7f03140b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0319adc640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03140ceb20 con 0x7f031415db00 2024-04-25T10:10:24.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0319adc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03140d1110 con 0x7f03140b87e0 2024-04-25T10:10:24.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0319adc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03140b30a0 con 0x7f03140d0770 2024-04-25T10:10:24.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0313fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03140b87e0 0x7f03140b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0313fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03140b87e0 0x7f03140b8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39158/0 (socket says 172.21.15.133:39158) 2024-04-25T10:10:24.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0313fff640 1 -- 172.21.15.133:0/1316655660 learned_addr learned my addr 172.21.15.133:0/1316655660 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:24.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f030bfff640 1 --2- 172.21.15.133:0/1316655660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03140d0770 0x7f03140d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0313fff640 1 -- 172.21.15.133:0/1316655660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03140d0770 msgr2=0x7f03140d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:24.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0313fff640 1 --2- 172.21.15.133:0/1316655660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03140d0770 0x7f03140d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0313fff640 1 -- 172.21.15.133:0/1316655660 >> v1:172.21.15.162:6789/0 conn(0x7f031415db00 legacy=0x7f03140b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:24.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0313fff640 1 -- 172.21.15.133:0/1316655660 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03140b7740 con 0x7f03140b87e0 2024-04-25T10:10:24.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f030bfff640 1 --2- 172.21.15.133:0/1316655660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03140d0770 0x7f03140d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:24.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.778+0000 7f0313fff640 1 --2- 172.21.15.133:0/1316655660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140b87e0 0x7f03140b8c70 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f030c009e50 tx=0x7f030c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfede09474b25e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0312ffd640 1 -- 172.21.15.133:0/1316655660 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c0a5840 con 0x7f03140b87e0 2024-04-25T10:10:24.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0312ffd640 1 -- 172.21.15.133:0/1316655660 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c0a59e0 con 0x7f03140b87e0 2024-04-25T10:10:24.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0312ffd640 1 -- 172.21.15.133:0/1316655660 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c0a5db0 con 0x7f03140b87e0 2024-04-25T10:10:24.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 -- 172.21.15.133:0/1316655660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140b87e0 msgr2=0x7f03140b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 --2- 172.21.15.133:0/1316655660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140b87e0 0x7f03140b8c70 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f030c009e50 tx=0x7f030c01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 -- 172.21.15.133:0/1316655660 shutdown_connections 2024-04-25T10:10:24.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 --2- 172.21.15.133:0/1316655660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140b87e0 0x7f03140b8c70 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 --2- 172.21.15.133:0/1316655660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03140d0770 0x7f03140d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 -- 172.21.15.133:0/1316655660 >> 172.21.15.133:0/1316655660 conn(0x7f03140727c0 msgr2=0x7f0314072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:24.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 -- 172.21.15.133:0/1316655660 shutdown_connections 2024-04-25T10:10:24.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 -- 172.21.15.133:0/1316655660 wait complete. 2024-04-25T10:10:24.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 Processor -- start 2024-04-25T10:10:24.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 -- start start 2024-04-25T10:10:24.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03140b87e0 0x7f0314160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03140d0770 0x7f031414e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031415db00 0x7f031415f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03140b7430 con 0x7f031415db00 2024-04-25T10:10:24.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03140b82c0 con 0x7f03140d0770 2024-04-25T10:10:24.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03140d18a0 con 0x7f03140b87e0 2024-04-25T10:10:24.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0313fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031415db00 0x7f031415f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0313fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031415db00 0x7f031415f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39172/0 (socket says 172.21.15.133:39172) 2024-04-25T10:10:24.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0313fff640 1 -- 172.21.15.133:0/3517923760 learned_addr learned my addr 172.21.15.133:0/3517923760 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:24.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0313fff640 1 -- 172.21.15.133:0/3517923760 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03140b87e0 msgr2=0x7f0314160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0313fff640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03140b87e0 0x7f0314160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0313fff640 1 -- 172.21.15.133:0/3517923760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03140d0770 msgr2=0x7f031414e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f030bfff640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03140d0770 0x7f031414e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0313fff640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03140d0770 0x7f031414e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0313fff640 1 -- 172.21.15.133:0/3517923760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0314160190 con 0x7f031415db00 2024-04-25T10:10:24.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0313fff640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031415db00 0x7f031415f140 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f030c0af040 tx=0x7f030c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0310ff9640 1 -- 172.21.15.133:0/3517923760 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c0bc020 con 0x7f031415db00 2024-04-25T10:10:24.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0310ff9640 1 -- 172.21.15.133:0/3517923760 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c0b88a0 con 0x7f031415db00 2024-04-25T10:10:24.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0310ff9640 1 -- 172.21.15.133:0/3517923760 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c0b8c90 con 0x7f031415db00 2024-04-25T10:10:24.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 -- 172.21.15.133:0/3517923760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03140c7460 con 0x7f031415db00 2024-04-25T10:10:24.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.782+0000 7f0319adc640 1 -- 172.21.15.133:0/3517923760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03140c7a20 con 0x7f031415db00 2024-04-25T10:10:24.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.786+0000 7f0319adc640 1 -- 172.21.15.133:0/3517923760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0314068ef0 con 0x7f031415db00 2024-04-25T10:10:24.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.786+0000 7f0310ff9640 1 -- 172.21.15.133:0/3517923760 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f030c0bf020 con 0x7f031415db00 2024-04-25T10:10:24.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.786+0000 7f0310ff9640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f4066f60 0x7f02f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.786+0000 7f03137fe640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f4066f60 0x7f02f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.786+0000 7f0310ff9640 1 -- 172.21.15.133:0/3517923760 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f030c11fd30 con 0x7f031415db00 2024-04-25T10:10:24.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.786+0000 7f03137fe640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f4066f60 0x7f02f4069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0304003cf0 tx=0x7f03040079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.790+0000 7f0310ff9640 1 -- 172.21.15.133:0/3517923760 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f030c0c6070 con 0x7f031415db00 2024-04-25T10:10:24.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.938+0000 7f0319adc640 1 -- 172.21.15.133:0/3517923760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03140c7bf0 con 0x7f031415db00 2024-04-25T10:10:24.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0310ff9640 1 -- 172.21.15.133:0/3517923760 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+13189 (secure 0 0 0) 0x7f030c0165e0 con 0x7f031415db00 2024-04-25T10:10:24.949 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:24.949 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:10:24.270299+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:24.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 -- 172.21.15.133:0/3517923760 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f4066f60 msgr2=0x7f02f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f4066f60 0x7f02f4069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0304003cf0 tx=0x7f03040079c0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 -- 172.21.15.133:0/3517923760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031415db00 msgr2=0x7f031415f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031415db00 0x7f031415f140 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f030c0af040 tx=0x7f030c019760 comp rx=0 tx=0).stop 2024-04-25T10:10:24.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 -- 172.21.15.133:0/3517923760 shutdown_connections 2024-04-25T10:10:24.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f4066f60 0x7f02f4069420 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031415db00 0x7f031415f140 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03140d0770 0x7f031414e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 --2- 172.21.15.133:0/3517923760 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03140b87e0 0x7f0314160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 -- 172.21.15.133:0/3517923760 >> 172.21.15.133:0/3517923760 conn(0x7f03140727c0 msgr2=0x7f03140d2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:24.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 -- 172.21.15.133:0/3517923760 shutdown_connections 2024-04-25T10:10:24.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:24.942+0000 7f0319adc640 1 -- 172.21.15.133:0/3517923760 wait complete. 2024-04-25T10:10:24.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:24.968 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:25.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f7575629640 1 Processor -- start 2024-04-25T10:10:25.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f7575629640 1 -- start start 2024-04-25T10:10:25.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f7575629640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75700cd820 0x7f75700cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f7575629640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f757015ced0 0x7f757015d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f7575629640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75700ce2e0 con 0x7f7570145170 2024-04-25T10:10:25.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f7575629640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757006db00 con 0x7f75700cd820 2024-04-25T10:10:25.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f7575629640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7570145550 con 0x7f757015ced0 2024-04-25T10:10:25.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f756effd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7570145170 0x7f757006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50972/0 (socket says 172.21.15.133:50972) 2024-04-25T10:10:25.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f756effd640 1 -- 172.21.15.133:0/3739294607 learned_addr learned my addr 172.21.15.133:0/3739294607 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:25.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f756e7fc640 1 --2- 172.21.15.133:0/3739294607 >> v2:172.21.15.133:3300/0 conn(0x7f75700cd820 0x7f75700cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f756dffb640 1 -- 172.21.15.133:0/3739294607 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604487256 0 0) 0x7f75700ce2e0 con 0x7f7570145170 2024-04-25T10:10:25.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f756dffb640 1 -- 172.21.15.133:0/3739294607 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7558003530 con 0x7f7570145170 2024-04-25T10:10:25.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f756f7fe640 1 --2- 172.21.15.133:0/3739294607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f757015ced0 0x7f757015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f756e7fc640 1 -- 172.21.15.133:0/3739294607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f757015ced0 msgr2=0x7f757015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f756e7fc640 1 --2- 172.21.15.133:0/3739294607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f757015ced0 0x7f757015d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f756e7fc640 1 -- 172.21.15.133:0/3739294607 >> v1:172.21.15.133:6789/0 conn(0x7f7570145170 legacy=0x7f757006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f756dffb640 1 -- 172.21.15.133:0/3739294607 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1492914733 0 0) 0x7f7558003530 con 0x7f7570145170 2024-04-25T10:10:25.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.082+0000 7f756e7fc640 1 -- 172.21.15.133:0/3739294607 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757006d4b0 con 0x7f75700cd820 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756e7fc640 1 --2- 172.21.15.133:0/3739294607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75700cd820 0x7f75700cdca0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f75640097c0 tx=0x7f756401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7888025a7086c18a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756dffb640 1 -- 172.21.15.133:0/3739294607 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75640b0070 con 0x7f75700cd820 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756dffb640 1 -- 172.21.15.133:0/3739294607 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75640ab400 con 0x7f75700cd820 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756dffb640 1 -- 172.21.15.133:0/3739294607 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560001680 con 0x7f75700cd820 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 -- 172.21.15.133:0/3739294607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75700cd820 msgr2=0x7f75700cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 --2- 172.21.15.133:0/3739294607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75700cd820 0x7f75700cdca0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f75640097c0 tx=0x7f756401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 -- 172.21.15.133:0/3739294607 shutdown_connections 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 --2- 172.21.15.133:0/3739294607 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f757015ced0 0x7f757015d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 --2- 172.21.15.133:0/3739294607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75700cd820 0x7f75700cdca0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 -- 172.21.15.133:0/3739294607 >> 172.21.15.133:0/3739294607 conn(0x7f7570066f70 msgr2=0x7f757006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 -- 172.21.15.133:0/3739294607 shutdown_connections 2024-04-25T10:10:25.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 -- 172.21.15.133:0/3739294607 wait complete. 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 Processor -- start 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 -- start start 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75700cd820 0x7f75701f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7570145170 0x7f75701f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f757015ced0 0x7f75701649b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7570168610 con 0x7f75700cd820 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7570168490 con 0x7f7570145170 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f7575629640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7570168790 con 0x7f757015ced0 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75700cd820 0x7f75701f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75700cd820 0x7f75701f7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39180/0 (socket says 172.21.15.133:39180) 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756effd640 1 -- 172.21.15.133:0/3802663575 learned_addr learned my addr 172.21.15.133:0/3802663575 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756f7fe640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f757015ced0 0x7f75701649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756e7fc640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7570145170 0x7f75701f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756effd640 1 -- 172.21.15.133:0/3802663575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f757015ced0 msgr2=0x7f75701649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756effd640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f757015ced0 0x7f75701649b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756effd640 1 -- 172.21.15.133:0/3802663575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7570145170 msgr2=0x7f75701f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756effd640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7570145170 0x7f75701f8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756effd640 1 -- 172.21.15.133:0/3802663575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75701f8690 con 0x7f75700cd820 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.086+0000 7f756effd640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75700cd820 0x7f75701f7b20 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f75600017f0 tx=0x7f756000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.090+0000 7f75577fe640 1 -- 172.21.15.133:0/3802663575 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560005ae0 con 0x7f75700cd820 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.090+0000 7f75577fe640 1 -- 172.21.15.133:0/3802663575 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7560005ca0 con 0x7f75700cd820 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.090+0000 7f7575629640 1 -- 172.21.15.133:0/3802663575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75701f8970 con 0x7f75700cd820 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.090+0000 7f75577fe640 1 -- 172.21.15.133:0/3802663575 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600072b0 con 0x7f75700cd820 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.090+0000 7f7575629640 1 -- 172.21.15.133:0/3802663575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75701f8f30 con 0x7f75700cd820 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.090+0000 7f7575629640 1 -- 172.21.15.133:0/3802663575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7570068e80 con 0x7f75700cd820 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.090+0000 7f75577fe640 1 -- 172.21.15.133:0/3802663575 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7560007450 con 0x7f75700cd820 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.090+0000 7f75577fe640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f755c066f60 0x7f755c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.090+0000 7f756e7fc640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f755c066f60 0x7f755c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.090+0000 7f75577fe640 1 -- 172.21.15.133:0/3802663575 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7560084c90 con 0x7f75700cd820 2024-04-25T10:10:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.090+0000 7f756e7fc640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f755c066f60 0x7f755c069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f75640ad000 tx=0x7f7564008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.094+0000 7f75577fe640 1 -- 172.21.15.133:0/3802663575 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7560010300 con 0x7f75700cd820 2024-04-25T10:10:25.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.250+0000 7f7575629640 1 -- 172.21.15.133:0/3802663575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75700b28c0 con 0x7f75700cd820 2024-04-25T10:10:25.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.250+0000 7f75577fe640 1 -- 172.21.15.133:0/3802663575 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7560055e70 con 0x7f75700cd820 2024-04-25T10:10:25.258 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:10:25.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 -- 172.21.15.133:0/3802663575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f755c066f60 msgr2=0x7f755c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f755c066f60 0x7f755c069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f75640ad000 tx=0x7f7564008040 comp rx=0 tx=0).stop 2024-04-25T10:10:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 -- 172.21.15.133:0/3802663575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75700cd820 msgr2=0x7f75701f7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75700cd820 0x7f75701f7b20 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f75600017f0 tx=0x7f756000cbe0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 -- 172.21.15.133:0/3802663575 shutdown_connections 2024-04-25T10:10:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f757015ced0 0x7f75701649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7570145170 0x7f75701f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f755c066f60 0x7f755c069420 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 --2- 172.21.15.133:0/3802663575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75700cd820 0x7f75701f7b20 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 -- 172.21.15.133:0/3802663575 >> 172.21.15.133:0/3802663575 conn(0x7f7570066f70 msgr2=0x7f757015f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 -- 172.21.15.133:0/3802663575 shutdown_connections 2024-04-25T10:10:25.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.254+0000 7f7575629640 1 -- 172.21.15.133:0/3802663575 wait complete. 2024-04-25T10:10:25.278 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:10:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.382+0000 7f91b5df1640 1 Processor -- start 2024-04-25T10:10:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.382+0000 7f91b5df1640 1 -- start start 2024-04-25T10:10:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91b5df1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91b015c9a0 0x7f91b00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91b5df1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b00c61d0 con 0x7f91b00ce330 2024-04-25T10:10:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91b5df1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b00c64b0 con 0x7f91b00b1f70 2024-04-25T10:10:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91b5df1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b006d4f0 con 0x7f91b015c9a0 2024-04-25T10:10:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91af7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91b015c9a0 0x7f91b00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91affff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f91b00ce330 0x7f91b015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50980/0 (socket says 172.21.15.133:50980) 2024-04-25T10:10:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91affff640 1 -- 172.21.15.133:0/2542967407 learned_addr learned my addr 172.21.15.133:0/2542967407 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91af7fe640 1 -- 172.21.15.133:0/2542967407 >> v1:172.21.15.162:6789/0 conn(0x7f91b00b1f70 legacy=0x7f91b006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91af7fe640 1 -- 172.21.15.133:0/2542967407 >> v1:172.21.15.133:6789/0 conn(0x7f91b00ce330 legacy=0x7f91b015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91af7fe640 1 -- 172.21.15.133:0/2542967407 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b006d7d0 con 0x7f91b015c9a0 2024-04-25T10:10:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91ae7fc640 1 -- 172.21.15.133:0/2542967407 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992677726 0 0) 0x7f91b00c61d0 con 0x7f91b00ce330 2024-04-25T10:10:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91af7fe640 1 --2- 172.21.15.133:0/2542967407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b015c9a0 0x7f91b00c5980 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f91a00097c0 tx=0x7f91a001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625731410b6a2ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91ae7fc640 1 -- 172.21.15.133:0/2542967407 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a00b0070 con 0x7f91b015c9a0 2024-04-25T10:10:25.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91ae7fc640 1 -- 172.21.15.133:0/2542967407 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a00ab600 con 0x7f91b015c9a0 2024-04-25T10:10:25.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91ae7fc640 1 -- 172.21.15.133:0/2542967407 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a00ab9d0 con 0x7f91b015c9a0 2024-04-25T10:10:25.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91b5df1640 1 -- 172.21.15.133:0/2542967407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b015c9a0 msgr2=0x7f91b00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.386+0000 7f91b5df1640 1 --2- 172.21.15.133:0/2542967407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b015c9a0 0x7f91b00c5980 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f91a00097c0 tx=0x7f91a001c460 comp rx=0 tx=0).stop 2024-04-25T10:10:25.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 -- 172.21.15.133:0/2542967407 shutdown_connections 2024-04-25T10:10:25.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 --2- 172.21.15.133:0/2542967407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b015c9a0 0x7f91b00c5980 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 -- 172.21.15.133:0/2542967407 >> 172.21.15.133:0/2542967407 conn(0x7f91b0067000 msgr2=0x7f91b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 -- 172.21.15.133:0/2542967407 shutdown_connections 2024-04-25T10:10:25.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 -- 172.21.15.133:0/2542967407 wait complete. 2024-04-25T10:10:25.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 Processor -- start 2024-04-25T10:10:25.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 -- start start 2024-04-25T10:10:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b00b1f70 0x7f91b015e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b00ce330 0x7f91b013d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91b015c9a0 0x7f91b013db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b015cf00 con 0x7f91b00b1f70 2024-04-25T10:10:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b015cd80 con 0x7f91b00ce330 2024-04-25T10:10:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b0169de0 con 0x7f91b015c9a0 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91af7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b00b1f70 0x7f91b015e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91af7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b00b1f70 0x7f91b015e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39196/0 (socket says 172.21.15.133:39196) 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91af7fe640 1 -- 172.21.15.133:0/3466409494 learned_addr learned my addr 172.21.15.133:0/3466409494 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91af7fe640 1 -- 172.21.15.133:0/3466409494 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91b015c9a0 msgr2=0x7f91b013db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91affff640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91b015c9a0 0x7f91b013db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91aeffd640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b00ce330 0x7f91b013d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91af7fe640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91b015c9a0 0x7f91b013db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91af7fe640 1 -- 172.21.15.133:0/3466409494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b00ce330 msgr2=0x7f91b013d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91af7fe640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b00ce330 0x7f91b013d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91af7fe640 1 -- 172.21.15.133:0/3466409494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b013e090 con 0x7f91b00b1f70 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91af7fe640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b00b1f70 0x7f91b015e670 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f91a00ae040 tx=0x7f91a001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f9197fff640 1 -- 172.21.15.133:0/3466409494 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a00b0050 con 0x7f91b00b1f70 2024-04-25T10:10:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f9197fff640 1 -- 172.21.15.133:0/3466409494 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a00bc070 con 0x7f91b00b1f70 2024-04-25T10:10:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 -- 172.21.15.133:0/3466409494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b013e340 con 0x7f91b00b1f70 2024-04-25T10:10:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f9197fff640 1 -- 172.21.15.133:0/3466409494 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a00b5a50 con 0x7f91b00b1f70 2024-04-25T10:10:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 -- 172.21.15.133:0/3466409494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b00ca2b0 con 0x7f91b00b1f70 2024-04-25T10:10:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.390+0000 7f91b5df1640 1 -- 172.21.15.133:0/3466409494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b0068ef0 con 0x7f91b00b1f70 2024-04-25T10:10:25.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.394+0000 7f9197fff640 1 -- 172.21.15.133:0/3466409494 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91a00ab4a0 con 0x7f91b00b1f70 2024-04-25T10:10:25.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.394+0000 7f9197fff640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f919c066f60 0x7f919c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.394+0000 7f9197fff640 1 -- 172.21.15.133:0/3466409494 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f91a011f7c0 con 0x7f91b00b1f70 2024-04-25T10:10:25.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.394+0000 7f91aeffd640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f919c066f60 0x7f919c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.394+0000 7f91aeffd640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f919c066f60 0x7f919c069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f91b00b17c0 tx=0x7f91a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.398+0000 7f9197fff640 1 -- 172.21.15.133:0/3466409494 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91a0016050 con 0x7f91b00b1f70 2024-04-25T10:10:25.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.558+0000 7f91b5df1640 1 -- 172.21.15.133:0/3466409494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f91b00c9cf0 con 0x7f91b00b1f70 2024-04-25T10:10:25.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.558+0000 7f9197fff640 1 -- 172.21.15.133:0/3466409494 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v90) v1 ==== 84+0+65 (secure 0 0 0) 0x7f91a00f0a20 con 0x7f91b00b1f70 2024-04-25T10:10:25.566 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:25.566 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:10:25.566 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:10:25.567 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:10:25.567 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:10:25.567 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:25.567 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:10:25.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 -- 172.21.15.133:0/3466409494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f919c066f60 msgr2=0x7f919c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f919c066f60 0x7f919c069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f91b00b17c0 tx=0x7f91a4008040 comp rx=0 tx=0).stop 2024-04-25T10:10:25.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 -- 172.21.15.133:0/3466409494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b00b1f70 msgr2=0x7f91b015e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b00b1f70 0x7f91b015e670 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f91a00ae040 tx=0x7f91a001aca0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 -- 172.21.15.133:0/3466409494 shutdown_connections 2024-04-25T10:10:25.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91b015c9a0 0x7f91b013db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b00ce330 0x7f91b013d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f919c066f60 0x7f919c069420 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 --2- 172.21.15.133:0/3466409494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b00b1f70 0x7f91b015e670 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 -- 172.21.15.133:0/3466409494 >> 172.21.15.133:0/3466409494 conn(0x7f91b0067000 msgr2=0x7f91b00b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 -- 172.21.15.133:0/3466409494 shutdown_connections 2024-04-25T10:10:25.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.562+0000 7f91b5df1640 1 -- 172.21.15.133:0/3466409494 wait complete. 2024-04-25T10:10:25.586 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:10:25.586 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:10:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f6021d6b640 1 Processor -- start 2024-04-25T10:10:25.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f6021d6b640 1 -- start start 2024-04-25T10:10:25.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f6021d6b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f601c0b4940 0x7f601c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f6021d6b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f601c15c420 0x7f601c0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f6021d6b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601c0b5260 con 0x7f601c071cb0 2024-04-25T10:10:25.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f6021d6b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601c072a90 con 0x7f601c15c420 2024-04-25T10:10:25.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f6021d6b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0a85d0 con 0x7f601c0b4940 2024-04-25T10:10:25.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f6012ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f601c071cb0 0x7f601c15bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50982/0 (socket says 172.21.15.133:50982) 2024-04-25T10:10:25.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f6012ffd640 1 -- 172.21.15.133:0/1457222128 learned_addr learned my addr 172.21.15.133:0/1457222128 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:25.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601b7fe640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f601c0b4940 0x7f601c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601bfff640 1 --2- 172.21.15.133:0/1457222128 >> v2:172.21.15.162:3300/0 conn(0x7f601c15c420 0x7f601c0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089907694 0 0) 0x7f601c0b5260 con 0x7f601c071cb0 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6008003530 con 0x7f601c071cb0 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2630255067 0 0) 0x7f6008003530 con 0x7f601c071cb0 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f601c0b4940 msgr2=0x7f601c0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f601c0b4940 0x7f601c0b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 >> v2:172.21.15.162:3300/0 conn(0x7f601c15c420 msgr2=0x7f601c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 --2- 172.21.15.133:0/1457222128 >> v2:172.21.15.162:3300/0 conn(0x7f601c15c420 0x7f601c0cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c15c9b0 con 0x7f601c071cb0 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6004001680 con 0x7f601c071cb0 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 >> v1:172.21.15.133:6789/0 conn(0x7f601c071cb0 legacy=0x7f601c15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60080039c0 0x7f6008003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6008004810 0x7f6008004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f601c0b4940 0x7f60080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600800b9d0 con 0x7f60080039c0 2024-04-25T10:10:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600800bb50 con 0x7f6008004810 2024-04-25T10:10:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600800bcd0 con 0x7f601c0b4940 2024-04-25T10:10:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601b7fe640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60080039c0 0x7f6008003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f601bfff640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f601c0b4940 0x7f60080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.702+0000 7f6012ffd640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6008004810 0x7f6008004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f601b7fe640 1 -- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f601c0b4940 msgr2=0x7f60080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f601b7fe640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f601c0b4940 0x7f60080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f601b7fe640 1 -- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6008004810 msgr2=0x7f6008004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f601b7fe640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6008004810 0x7f6008004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f601b7fe640 1 -- 172.21.15.133:0/1457222128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008003530 con 0x7f60080039c0 2024-04-25T10:10:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f601bfff640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f601c0b4940 0x7f60080051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:25.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f6012ffd640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6008004810 0x7f6008004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f601b7fe640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60080039c0 0x7f6008003de0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f600c009ac0 tx=0x7f600c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418c687ecc1039e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c0b2070 con 0x7f60080039c0 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c0a5510 con 0x7f60080039c0 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f601affd640 1 -- 172.21.15.133:0/1457222128 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c009690 con 0x7f60080039c0 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f6021d6b640 1 -- 172.21.15.133:0/1457222128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60080039c0 msgr2=0x7f6008003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f6021d6b640 1 --2- 172.21.15.133:0/1457222128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60080039c0 0x7f6008003de0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f600c009ac0 tx=0x7f600c01ca20 comp rx=0 tx=0).stop 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f601bfff640 1 -- 172.21.15.133:0/1457222128 reap_dead start 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f6021d6b640 1 -- 172.21.15.133:0/1457222128 shutdown_connections 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f6021d6b640 1 -- 172.21.15.133:0/1457222128 >> 172.21.15.133:0/1457222128 conn(0x7f601c06d370 msgr2=0x7f601c06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f6021d6b640 1 -- 172.21.15.133:0/1457222128 shutdown_connections 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.706+0000 7f6021d6b640 1 -- 172.21.15.133:0/1457222128 wait complete. 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6021d6b640 1 Processor -- start 2024-04-25T10:10:25.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6021d6b640 1 -- start start 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6021d6b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60080039c0 0x7f601c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6021d6b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f601c1495e0 0x7f601c147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6021d6b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f601c147fe0 0x7f601c149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6021d6b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c1452d0 con 0x7f601c147fe0 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6021d6b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c071790 con 0x7f601c1495e0 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6021d6b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c145450 con 0x7f60080039c0 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f601bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f601c147fe0 0x7f601c149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f601bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f601c147fe0 0x7f601c149ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39204/0 (socket says 172.21.15.133:39204) 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f601bfff640 1 -- 172.21.15.133:0/2058787077 learned_addr learned my addr 172.21.15.133:0/2058787077 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f601b7fe640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60080039c0 0x7f601c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6012ffd640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f601c1495e0 0x7f601c147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f601bfff640 1 -- 172.21.15.133:0/2058787077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60080039c0 msgr2=0x7f601c15a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f601bfff640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60080039c0 0x7f601c15a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f601bfff640 1 -- 172.21.15.133:0/2058787077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f601c1495e0 msgr2=0x7f601c147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f601bfff640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f601c1495e0 0x7f601c147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f601bfff640 1 -- 172.21.15.133:0/2058787077 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c148540 con 0x7f601c147fe0 2024-04-25T10:10:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f601bfff640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f601c147fe0 0x7f601c149ae0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f601c070c10 tx=0x7f601400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6018ff9640 1 -- 172.21.15.133:0/2058787077 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014016420 con 0x7f601c147fe0 2024-04-25T10:10:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6018ff9640 1 -- 172.21.15.133:0/2058787077 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601401b070 con 0x7f601c147fe0 2024-04-25T10:10:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6021d6b640 1 -- 172.21.15.133:0/2058787077 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601c148820 con 0x7f601c147fe0 2024-04-25T10:10:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6018ff9640 1 -- 172.21.15.133:0/2058787077 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60140167e0 con 0x7f601c147fe0 2024-04-25T10:10:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6021d6b640 1 -- 172.21.15.133:0/2058787077 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601c148de0 con 0x7f601c147fe0 2024-04-25T10:10:25.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6021d6b640 1 -- 172.21.15.133:0/2058787077 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601c069000 con 0x7f601c147fe0 2024-04-25T10:10:25.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6018ff9640 1 -- 172.21.15.133:0/2058787077 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6014016980 con 0x7f601c147fe0 2024-04-25T10:10:25.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.710+0000 7f6018ff9640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5ffc066f60 0x7f5ffc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.714+0000 7f6018ff9640 1 -- 172.21.15.133:0/2058787077 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6014085bc0 con 0x7f601c147fe0 2024-04-25T10:10:25.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.714+0000 7f601b7fe640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5ffc066f60 0x7f5ffc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.714+0000 7f601b7fe640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5ffc066f60 0x7f5ffc069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f600c0af000 tx=0x7f600c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.718+0000 7f6018ff9640 1 -- 172.21.15.133:0/2058787077 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6014056d00 con 0x7f601c147fe0 2024-04-25T10:10:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:25.874+0000 7f6021d6b640 1 -- 172.21.15.133:0/2058787077 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f601c1406b0 con 0x7f601c147fe0 2024-04-25T10:10:26.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.258+0000 7f6018ff9640 1 -- 172.21.15.133:0/2058787077 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v91) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6014056600 con 0x7f601c147fe0 2024-04-25T10:10:26.266 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:10:26.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 -- 172.21.15.133:0/2058787077 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5ffc066f60 msgr2=0x7f5ffc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:26.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5ffc066f60 0x7f5ffc069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f600c0af000 tx=0x7f600c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 -- 172.21.15.133:0/2058787077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f601c147fe0 msgr2=0x7f601c149ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:26.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f601c147fe0 0x7f601c149ae0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f601c070c10 tx=0x7f601400f520 comp rx=0 tx=0).stop 2024-04-25T10:10:26.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 -- 172.21.15.133:0/2058787077 shutdown_connections 2024-04-25T10:10:26.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5ffc066f60 0x7f5ffc069420 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f601c147fe0 0x7f601c149ae0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f601c1495e0 0x7f601c147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 --2- 172.21.15.133:0/2058787077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60080039c0 0x7f601c15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 -- 172.21.15.133:0/2058787077 >> 172.21.15.133:0/2058787077 conn(0x7f601c06d370 msgr2=0x7f601c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:26.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 -- 172.21.15.133:0/2058787077 shutdown_connections 2024-04-25T10:10:26.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.262+0000 7f6021d6b640 1 -- 172.21.15.133:0/2058787077 wait complete. 2024-04-25T10:10:26.297 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:10:26.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebffc9640 1 Processor -- start 2024-04-25T10:10:26.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebffc9640 1 -- start start 2024-04-25T10:10:26.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebffc9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efeb8143ea0 0x7efeb80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:26.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebffc9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb806eac0 0x7efeb80b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:26.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebffc9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb815ac50 con 0x7efeb80b2150 2024-04-25T10:10:26.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebffc9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb80cd180 con 0x7efeb8143ea0 2024-04-25T10:10:26.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebffc9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb815aed0 con 0x7efeb806eac0 2024-04-25T10:10:26.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebe566640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb806eac0 0x7efeb80b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:26.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebd564640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7efeb80b2150 0x7efeb806e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:44066/0 (socket says 172.21.15.133:44066) 2024-04-25T10:10:26.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebdd65640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efeb8143ea0 0x7efeb80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:26.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebd564640 1 -- 172.21.15.133:0/4236699674 learned_addr learned my addr 172.21.15.133:0/4236699674 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:26.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebe566640 1 -- 172.21.15.133:0/4236699674 >> v2:172.21.15.162:3300/0 conn(0x7efeb8143ea0 msgr2=0x7efeb80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:26.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebe566640 1 --2- 172.21.15.133:0/4236699674 >> v2:172.21.15.162:3300/0 conn(0x7efeb8143ea0 0x7efeb80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebe566640 1 -- 172.21.15.133:0/4236699674 >> v1:172.21.15.162:6789/0 conn(0x7efeb80b2150 legacy=0x7efeb806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:26.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.422+0000 7efebe566640 1 -- 172.21.15.133:0/4236699674 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb815b1b0 con 0x7efeb806eac0 2024-04-25T10:10:26.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebe566640 1 --2- 172.21.15.133:0/4236699674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb806eac0 0x7efeb80b06d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009d90 tx=0x7efeb001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f049ca4737f2f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:26.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebcd63640 1 -- 172.21.15.133:0/4236699674 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb00b2070 con 0x7efeb806eac0 2024-04-25T10:10:26.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebcd63640 1 -- 172.21.15.133:0/4236699674 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb00a5940 con 0x7efeb806eac0 2024-04-25T10:10:26.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebcd63640 1 -- 172.21.15.133:0/4236699674 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb00a5d10 con 0x7efeb806eac0 2024-04-25T10:10:26.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebffc9640 1 -- 172.21.15.133:0/4236699674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb806eac0 msgr2=0x7efeb80b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:26.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebffc9640 1 --2- 172.21.15.133:0/4236699674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb806eac0 0x7efeb80b06d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009d90 tx=0x7efeb001c970 comp rx=0 tx=0).stop 2024-04-25T10:10:26.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebffc9640 1 -- 172.21.15.133:0/4236699674 shutdown_connections 2024-04-25T10:10:26.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebffc9640 1 --2- 172.21.15.133:0/4236699674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb806eac0 0x7efeb80b06d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebffc9640 1 --2- 172.21.15.133:0/4236699674 >> v2:172.21.15.162:3300/0 conn(0x7efeb8143ea0 0x7efeb80cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebffc9640 1 -- 172.21.15.133:0/4236699674 >> 172.21.15.133:0/4236699674 conn(0x7efeb8066f70 msgr2=0x7efeb806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:26.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebffc9640 1 -- 172.21.15.133:0/4236699674 shutdown_connections 2024-04-25T10:10:26.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebffc9640 1 -- 172.21.15.133:0/4236699674 wait complete. 2024-04-25T10:10:26.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebffc9640 1 Processor -- start 2024-04-25T10:10:26.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.426+0000 7efebffc9640 1 -- start start 2024-04-25T10:10:26.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebffc9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeb806eac0 0x7efeb80b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:26.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebffc9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb80b2150 0x7efeb81309f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:26.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebffc9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb8143ea0 0x7efeb812e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:26.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebffc9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb8165540 con 0x7efeb806eac0 2024-04-25T10:10:26.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebffc9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb81653c0 con 0x7efeb80b2150 2024-04-25T10:10:26.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebffc9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb81656c0 con 0x7efeb8143ea0 2024-04-25T10:10:26.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebdd65640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeb806eac0 0x7efeb80b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:26.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebdd65640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeb806eac0 0x7efeb80b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42572/0 (socket says 172.21.15.133:42572) 2024-04-25T10:10:26.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebdd65640 1 -- 172.21.15.133:0/416485027 learned_addr learned my addr 172.21.15.133:0/416485027 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebd564640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb80b2150 0x7efeb81309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebdd65640 1 -- 172.21.15.133:0/416485027 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb8143ea0 msgr2=0x7efeb812e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebe566640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb8143ea0 0x7efeb812e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebdd65640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb8143ea0 0x7efeb812e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebdd65640 1 -- 172.21.15.133:0/416485027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb80b2150 msgr2=0x7efeb81309f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebdd65640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb80b2150 0x7efeb81309f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebdd65640 1 -- 172.21.15.133:0/416485027 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb81552d0 con 0x7efeb806eac0 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebdd65640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeb806eac0 0x7efeb80b3c10 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7efeb400e870 tx=0x7efeb400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efeae7fc640 1 -- 172.21.15.133:0/416485027 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4017070 con 0x7efeb806eac0 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efeae7fc640 1 -- 172.21.15.133:0/416485027 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb400c940 con 0x7efeb806eac0 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebffc9640 1 -- 172.21.15.133:0/416485027 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb8155540 con 0x7efeb806eac0 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efeae7fc640 1 -- 172.21.15.133:0/416485027 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb400ccf0 con 0x7efeb806eac0 2024-04-25T10:10:26.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebffc9640 1 -- 172.21.15.133:0/416485027 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb80b3410 con 0x7efeb806eac0 2024-04-25T10:10:26.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebffc9640 1 -- 172.21.15.133:0/416485027 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb8068e80 con 0x7efeb806eac0 2024-04-25T10:10:26.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efeae7fc640 1 -- 172.21.15.133:0/416485027 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efeb401a070 con 0x7efeb806eac0 2024-04-25T10:10:26.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efeae7fc640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe9c066f60 0x7efe9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:26.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efeae7fc640 1 -- 172.21.15.133:0/416485027 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5619+0+0 (secure 0 0 0) 0x7efeb4085260 con 0x7efeb806eac0 2024-04-25T10:10:26.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.430+0000 7efebd564640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe9c066f60 0x7efe9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:26.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.434+0000 7efebd564640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe9c066f60 0x7efe9c069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efeb80cbcb0 tx=0x7efea8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:26.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.438+0000 7efeae7fc640 1 -- 172.21.15.133:0/416485027 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efeb40562c0 con 0x7efeb806eac0 2024-04-25T10:10:26.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:26.594+0000 7efebffc9640 1 -- 172.21.15.133:0/416485027 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efeb80737e0 con 0x7efeb806eac0 2024-04-25T10:10:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.274+0000 7efeae7fc640 1 -- 172.21.15.133:0/416485027 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v92) v1 ==== 170+0+0 (secure 0 0 0) 0x7efeb4055bc0 con 0x7efeb806eac0 2024-04-25T10:10:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:10:27.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.278+0000 7efebffc9640 1 -- 172.21.15.133:0/416485027 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe9c066f60 msgr2=0x7efe9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.278+0000 7efebffc9640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe9c066f60 0x7efe9c069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efeb80cbcb0 tx=0x7efea8009040 comp rx=0 tx=0).stop 2024-04-25T10:10:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.278+0000 7efebffc9640 1 -- 172.21.15.133:0/416485027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeb806eac0 msgr2=0x7efeb80b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.278+0000 7efebffc9640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeb806eac0 0x7efeb80b3c10 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7efeb400e870 tx=0x7efeb400ec70 comp rx=0 tx=0).stop 2024-04-25T10:10:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.278+0000 7efebffc9640 1 -- 172.21.15.133:0/416485027 shutdown_connections 2024-04-25T10:10:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.278+0000 7efebffc9640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efeb8143ea0 0x7efeb812e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.278+0000 7efebffc9640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efeb80b2150 0x7efeb81309f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.278+0000 7efebffc9640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe9c066f60 0x7efe9c069420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.278+0000 7efebffc9640 1 --2- 172.21.15.133:0/416485027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeb806eac0 0x7efeb80b3c10 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:27.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.278+0000 7efebffc9640 1 -- 172.21.15.133:0/416485027 >> 172.21.15.133:0/416485027 conn(0x7efeb8066f70 msgr2=0x7efeb80cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:27.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.282+0000 7efebffc9640 1 -- 172.21.15.133:0/416485027 shutdown_connections 2024-04-25T10:10:27.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.282+0000 7efebffc9640 1 -- 172.21.15.133:0/416485027 wait complete. 2024-04-25T10:10:27.325 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:10:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.442+0000 7f04995e7640 1 Processor -- start 2024-04-25T10:10:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.442+0000 7f04995e7640 1 -- start start 2024-04-25T10:10:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04995e7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04940b2150 0x7f049406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04995e7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049415ac50 con 0x7f049415b9b0 2024-04-25T10:10:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04995e7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04940cd180 con 0x7f0494143ea0 2024-04-25T10:10:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04995e7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04940ccf20 con 0x7f04940b2150 2024-04-25T10:10:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04937fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f049415b9b0 0x7f04940b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40102/0 (socket says 172.21.15.133:40102) 2024-04-25T10:10:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04937fe640 1 -- 172.21.15.133:0/3438195431 learned_addr learned my addr 172.21.15.133:0/3438195431 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189405234 0 0) 0x7f048c0014a0 con 0x7f049415b9b0 2024-04-25T10:10:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04927fc640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04940b2150 0x7f049406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f047c005740 con 0x7f049415b9b0 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3760325729 0 0) 0x7f047c005740 con 0x7f049415b9b0 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04940b2150 msgr2=0x7f049406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04940b2150 0x7f049406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 >> v1:172.21.15.162:6789/0 conn(0x7f0494143ea0 legacy=0x7f04940cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049415aed0 con 0x7f049415b9b0 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f048c001bd0 con 0x7f049415b9b0 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 >> v1:172.21.15.133:6789/0 conn(0x7f049415b9b0 legacy=0x7f04940b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f047c005c00 0x7f047c006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f047c006a20 0x7f047c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04940b2150 0x7f047c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940ccf20 con 0x7f047c006a20 2024-04-25T10:10:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c00dc90 con 0x7f047c005c00 2024-04-25T10:10:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c00de90 con 0x7f04940b2150 2024-04-25T10:10:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04927fc640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f047c006a20 0x7f047c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04937fe640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04940b2150 0x7f047c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04927fc640 1 -- 172.21.15.133:0/3438195431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04940b2150 msgr2=0x7f047c0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04927fc640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04940b2150 0x7f047c0073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04927fc640 1 -- 172.21.15.133:0/3438195431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f047c005c00 msgr2=0x7f047c006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04927fc640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f047c005c00 0x7f047c006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04927fc640 1 -- 172.21.15.133:0/3438195431 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c005740 con 0x7f047c006a20 2024-04-25T10:10:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.446+0000 7f04937fe640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f04940b2150 0x7f047c0073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04927fc640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f047c006a20 0x7f047c006ea0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0488009ab0 tx=0x7f048801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5b7512e95c5619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04880b2070 con 0x7f047c006a20 2024-04-25T10:10:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04880a5510 con 0x7f047c006a20 2024-04-25T10:10:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f0491ffb640 1 -- 172.21.15.133:0/3438195431 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04880096b0 con 0x7f047c006a20 2024-04-25T10:10:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 -- 172.21.15.133:0/3438195431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f047c006a20 msgr2=0x7f047c006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 --2- 172.21.15.133:0/3438195431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f047c006a20 0x7f047c006ea0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0488009ab0 tx=0x7f048801ca30 comp rx=0 tx=0).stop 2024-04-25T10:10:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04937fe640 1 -- 172.21.15.133:0/3438195431 reap_dead start 2024-04-25T10:10:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 -- 172.21.15.133:0/3438195431 shutdown_connections 2024-04-25T10:10:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 -- 172.21.15.133:0/3438195431 >> 172.21.15.133:0/3438195431 conn(0x7f0494066f70 msgr2=0x7f049406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 -- 172.21.15.133:0/3438195431 shutdown_connections 2024-04-25T10:10:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 -- 172.21.15.133:0/3438195431 wait complete. 2024-04-25T10:10:27.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 Processor -- start 2024-04-25T10:10:27.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 -- start start 2024-04-25T10:10:27.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f047c006a20 0x7f04940cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:27.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0494151e20 0x7f04941f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:27.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0494153630 0x7f0494153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:27.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0494144280 con 0x7f0494153630 2024-04-25T10:10:27.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049415b490 con 0x7f0494151e20 2024-04-25T10:10:27.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04995e7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0494144400 con 0x7f047c006a20 2024-04-25T10:10:27.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04937fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0494153630 0x7f0494153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:27.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04937fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0494153630 0x7f0494153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42600/0 (socket says 172.21.15.133:42600) 2024-04-25T10:10:27.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04937fe640 1 -- 172.21.15.133:0/2359198895 learned_addr learned my addr 172.21.15.133:0/2359198895 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:27.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f0492ffd640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f047c006a20 0x7f04940cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:27.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04927fc640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0494151e20 0x7f04941f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:27.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04937fe640 1 -- 172.21.15.133:0/2359198895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f047c006a20 msgr2=0x7f04940cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:27.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04937fe640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f047c006a20 0x7f04940cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04937fe640 1 -- 172.21.15.133:0/2359198895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0494151e20 msgr2=0x7f04941f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04937fe640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0494151e20 0x7f04941f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.450+0000 7f04937fe640 1 -- 172.21.15.133:0/2359198895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0494140dd0 con 0x7f0494153630 2024-04-25T10:10:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f04937fe640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0494153630 0x7f0494153ae0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f048c000f90 tx=0x7f048c010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f047b7fe640 1 -- 172.21.15.133:0/2359198895 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c01a070 con 0x7f0494153630 2024-04-25T10:10:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f047b7fe640 1 -- 172.21.15.133:0/2359198895 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c011040 con 0x7f0494153630 2024-04-25T10:10:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f04995e7640 1 -- 172.21.15.133:0/2359198895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04940cb5b0 con 0x7f0494153630 2024-04-25T10:10:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f047b7fe640 1 -- 172.21.15.133:0/2359198895 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c00ea70 con 0x7f0494153630 2024-04-25T10:10:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f04995e7640 1 -- 172.21.15.133:0/2359198895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0494141350 con 0x7f0494153630 2024-04-25T10:10:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f04995e7640 1 -- 172.21.15.133:0/2359198895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0494068e80 con 0x7f0494153630 2024-04-25T10:10:27.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f047b7fe640 1 -- 172.21.15.133:0/2359198895 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f048c01d070 con 0x7f0494153630 2024-04-25T10:10:27.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f047b7fe640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0480066f90 0x7f0480069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:27.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f047b7fe640 1 -- 172.21.15.133:0/2359198895 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f048c0875d0 con 0x7f0494153630 2024-04-25T10:10:27.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f0492ffd640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0480066f90 0x7f0480069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:27.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.454+0000 7f0492ffd640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0480066f90 0x7f0480069450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f04840016a0 tx=0x7f0484009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:27.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.458+0000 7f047b7fe640 1 -- 172.21.15.133:0/2359198895 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f049415ac50 con 0x7f0494153630 2024-04-25T10:10:27.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:27.658+0000 7f04995e7640 1 -- 172.21.15.133:0/2359198895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f049406af30 con 0x7f0494153630 2024-04-25T10:10:28.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.310+0000 7f047b7fe640 1 -- 172.21.15.133:0/2359198895 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v55) v1 ==== 174+0+0 (secure 0 0 0) 0x7f048c058540 con 0x7f0494153630 2024-04-25T10:10:28.315 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T10:10:28.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.322+0000 7f04995e7640 1 -- 172.21.15.133:0/2359198895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0480066f90 msgr2=0x7f0480069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.322+0000 7f04995e7640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0480066f90 0x7f0480069450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f04840016a0 tx=0x7f0484009040 comp rx=0 tx=0).stop 2024-04-25T10:10:28.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.322+0000 7f04995e7640 1 -- 172.21.15.133:0/2359198895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0494153630 msgr2=0x7f0494153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.322+0000 7f04995e7640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0494153630 0x7f0494153ae0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f048c000f90 tx=0x7f048c010c30 comp rx=0 tx=0).stop 2024-04-25T10:10:28.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.326+0000 7f04995e7640 1 -- 172.21.15.133:0/2359198895 shutdown_connections 2024-04-25T10:10:28.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.326+0000 7f04995e7640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0480066f90 0x7f0480069450 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.326+0000 7f04995e7640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0494153630 0x7f0494153ae0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.326+0000 7f04995e7640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0494151e20 0x7f04941f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.326+0000 7f04995e7640 1 --2- 172.21.15.133:0/2359198895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f047c006a20 0x7f04940cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.326+0000 7f04995e7640 1 -- 172.21.15.133:0/2359198895 >> 172.21.15.133:0/2359198895 conn(0x7f0494066f70 msgr2=0x7f049415de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:28.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.326+0000 7f04995e7640 1 -- 172.21.15.133:0/2359198895 shutdown_connections 2024-04-25T10:10:28.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.326+0000 7f04995e7640 1 -- 172.21.15.133:0/2359198895 wait complete. 2024-04-25T10:10:28.355 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:28.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.462+0000 7fe1d10e1640 1 Processor -- start 2024-04-25T10:10:28.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.462+0000 7fe1d10e1640 1 -- start start 2024-04-25T10:10:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1cc161ef0 0x7fe1cc0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe1cc0999d0 0x7fe1cc0c4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1cc0c4bc0 0x7fe1cc165970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc0c5030 con 0x7fe1cc0c4bc0 2024-04-25T10:10:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc099e60 con 0x7fe1cc0999d0 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc09a140 con 0x7fe1cc161ef0 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1cb59e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1cc0c4bc0 0x7fe1cc165970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1cb59e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1cc0c4bc0 0x7fe1cc165970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42604/0 (socket says 172.21.15.133:42604) 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1cb59e640 1 -- 172.21.15.133:0/4147676262 learned_addr learned my addr 172.21.15.133:0/4147676262 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1ca59c640 1 --2- 172.21.15.133:0/4147676262 >> v2:172.21.15.162:3300/0 conn(0x7fe1cc0999d0 0x7fe1cc0c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1cad9d640 1 --2- 172.21.15.133:0/4147676262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1cc161ef0 0x7fe1cc0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1cb59e640 1 -- 172.21.15.133:0/4147676262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1cc161ef0 msgr2=0x7fe1cc0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1cb59e640 1 --2- 172.21.15.133:0/4147676262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1cc161ef0 0x7fe1cc0b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1cb59e640 1 -- 172.21.15.133:0/4147676262 >> v2:172.21.15.162:3300/0 conn(0x7fe1cc0999d0 msgr2=0x7fe1cc0c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1cb59e640 1 --2- 172.21.15.133:0/4147676262 >> v2:172.21.15.162:3300/0 conn(0x7fe1cc0999d0 0x7fe1cc0c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1cb59e640 1 -- 172.21.15.133:0/4147676262 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc09a420 con 0x7fe1cc0c4bc0 2024-04-25T10:10:28.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1ca59c640 1 --2- 172.21.15.133:0/4147676262 >> v2:172.21.15.162:3300/0 conn(0x7fe1cc0999d0 0x7fe1cc0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:28.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1cb59e640 1 --2- 172.21.15.133:0/4147676262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0c4bc0 0x7fe1cc165970 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc072aa0 tx=0x7fe1c401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f628520549a454b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:28.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1c9d9b640 1 -- 172.21.15.133:0/4147676262 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c40a55b0 con 0x7fe1cc0c4bc0 2024-04-25T10:10:28.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1c9d9b640 1 -- 172.21.15.133:0/4147676262 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c40a5750 con 0x7fe1cc0c4bc0 2024-04-25T10:10:28.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1c9d9b640 1 -- 172.21.15.133:0/4147676262 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c40a5b20 con 0x7fe1cc0c4bc0 2024-04-25T10:10:28.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 -- 172.21.15.133:0/4147676262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0c4bc0 msgr2=0x7fe1cc165970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 --2- 172.21.15.133:0/4147676262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0c4bc0 0x7fe1cc165970 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc072aa0 tx=0x7fe1c401c730 comp rx=0 tx=0).stop 2024-04-25T10:10:28.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 -- 172.21.15.133:0/4147676262 shutdown_connections 2024-04-25T10:10:28.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 --2- 172.21.15.133:0/4147676262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0c4bc0 0x7fe1cc165970 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 --2- 172.21.15.133:0/4147676262 >> v2:172.21.15.162:3300/0 conn(0x7fe1cc0999d0 0x7fe1cc0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 --2- 172.21.15.133:0/4147676262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1cc161ef0 0x7fe1cc0b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.466+0000 7fe1d10e1640 1 -- 172.21.15.133:0/4147676262 >> 172.21.15.133:0/4147676262 conn(0x7fe1cc0b31a0 msgr2=0x7fe1cc0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:28.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 -- 172.21.15.133:0/4147676262 shutdown_connections 2024-04-25T10:10:28.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 -- 172.21.15.133:0/4147676262 wait complete. 2024-04-25T10:10:28.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 Processor -- start 2024-04-25T10:10:28.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 -- start start 2024-04-25T10:10:28.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1cc0c4bc0 0x7fe1cc15ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1cc161ef0 0x7fe1cc0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cb170 0x7fe1cc0af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0b0b90 con 0x7fe1cc0cb170 2024-04-25T10:10:28.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0b0d10 con 0x7fe1cc161ef0 2024-04-25T10:10:28.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0b0e90 con 0x7fe1cc0c4bc0 2024-04-25T10:10:28.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1cb59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cb170 0x7fe1cc0af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1cb59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cb170 0x7fe1cc0af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42606/0 (socket says 172.21.15.133:42606) 2024-04-25T10:10:28.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1cb59e640 1 -- 172.21.15.133:0/2927426119 learned_addr learned my addr 172.21.15.133:0/2927426119 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:28.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1ca59c640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1cc161ef0 0x7fe1cc0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1cad9d640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1cc0c4bc0 0x7fe1cc15ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1cb59e640 1 -- 172.21.15.133:0/2927426119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1cc0c4bc0 msgr2=0x7fe1cc15ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1cb59e640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1cc0c4bc0 0x7fe1cc15ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1cb59e640 1 -- 172.21.15.133:0/2927426119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1cc161ef0 msgr2=0x7fe1cc0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1cb59e640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1cc161ef0 0x7fe1cc0c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1cb59e640 1 -- 172.21.15.133:0/2927426119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc0c72f0 con 0x7fe1cc0cb170 2024-04-25T10:10:28.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1cad9d640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1cc0c4bc0 0x7fe1cc15ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:28.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1cb59e640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cb170 0x7fe1cc0af630 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40af040 tx=0x7fe1c40194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:28.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1b37fe640 1 -- 172.21.15.133:0/2927426119 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4009510 con 0x7fe1cc0cb170 2024-04-25T10:10:28.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1b37fe640 1 -- 172.21.15.133:0/2927426119 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c40be070 con 0x7fe1cc0cb170 2024-04-25T10:10:28.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 -- 172.21.15.133:0/2927426119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc0b0450 con 0x7fe1cc0cb170 2024-04-25T10:10:28.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1b37fe640 1 -- 172.21.15.133:0/2927426119 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c40b88d0 con 0x7fe1cc0cb170 2024-04-25T10:10:28.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.470+0000 7fe1d10e1640 1 -- 172.21.15.133:0/2927426119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc0c7590 con 0x7fe1cc0cb170 2024-04-25T10:10:28.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.474+0000 7fe1d10e1640 1 -- 172.21.15.133:0/2927426119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc0b0970 con 0x7fe1cc0cb170 2024-04-25T10:10:28.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.474+0000 7fe1b37fe640 1 -- 172.21.15.133:0/2927426119 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1c40b8a70 con 0x7fe1cc0cb170 2024-04-25T10:10:28.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.474+0000 7fe1b37fe640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe1b8066f60 0x7fe1b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.474+0000 7fe1b37fe640 1 -- 172.21.15.133:0/2927426119 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1c411ffe0 con 0x7fe1cc0cb170 2024-04-25T10:10:28.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.474+0000 7fe1cad9d640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe1b8066f60 0x7fe1b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.474+0000 7fe1cad9d640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe1b8066f60 0x7fe1b8069420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc09f5a0 tx=0x7fe1bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:28.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.478+0000 7fe1b37fe640 1 -- 172.21.15.133:0/2927426119 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe1cc0b0970 con 0x7fe1cc0cb170 2024-04-25T10:10:28.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.638+0000 7fe1d10e1640 1 -- 172.21.15.133:0/2927426119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1cc067c70 con 0x7fe1cc0cb170 2024-04-25T10:10:28.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.638+0000 7fe1b37fe640 1 -- 172.21.15.133:0/2927426119 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16922 (secure 0 0 0) 0x7fe1c40165e0 con 0x7fe1cc0cb170 2024-04-25T10:10:28.642 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:28.642 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":93,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:10:27.660839+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:25.920690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:26.640190+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:28.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 -- 172.21.15.133:0/2927426119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe1b8066f60 msgr2=0x7fe1b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe1b8066f60 0x7fe1b8069420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc09f5a0 tx=0x7fe1bc008040 comp rx=0 tx=0).stop 2024-04-25T10:10:28.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 -- 172.21.15.133:0/2927426119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cb170 msgr2=0x7fe1cc0af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cb170 0x7fe1cc0af630 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40af040 tx=0x7fe1c40194d0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 -- 172.21.15.133:0/2927426119 shutdown_connections 2024-04-25T10:10:28.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe1b8066f60 0x7fe1b8069420 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cb170 0x7fe1cc0af630 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1cc161ef0 0x7fe1cc0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 --2- 172.21.15.133:0/2927426119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1cc0c4bc0 0x7fe1cc15ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 -- 172.21.15.133:0/2927426119 >> 172.21.15.133:0/2927426119 conn(0x7fe1cc0b31a0 msgr2=0x7fe1cc0b6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:28.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 -- 172.21.15.133:0/2927426119 shutdown_connections 2024-04-25T10:10:28.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.642+0000 7fe1d10e1640 1 -- 172.21.15.133:0/2927426119 wait complete. 2024-04-25T10:10:28.663 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.778+0000 7fb7e2cc9640 1 Processor -- start 2024-04-25T10:10:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.778+0000 7fb7e2cc9640 1 -- start start 2024-04-25T10:10:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7e2cc9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7dc15c9a0 0x7fb7dc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7e2cc9640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc0c61d0 con 0x7fb7dc0b1f70 2024-04-25T10:10:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7e2cc9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc0c64b0 con 0x7fb7dc0ce330 2024-04-25T10:10:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7e2cc9640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7dc06d4f0 con 0x7fb7dc15c9a0 2024-04-25T10:10:28.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7e0a65640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7dc15c9a0 0x7fb7dc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7dbfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb7dc0b1f70 0x7fb7dc06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40118/0 (socket says 172.21.15.133:40118) 2024-04-25T10:10:28.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7dbfff640 1 -- 172.21.15.133:0/1298130450 learned_addr learned my addr 172.21.15.133:0/1298130450 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:28.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7e0a65640 1 -- 172.21.15.133:0/1298130450 >> v1:172.21.15.162:6789/0 conn(0x7fb7dc0ce330 legacy=0x7fb7dc15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:28.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7db7fe640 1 -- 172.21.15.133:0/1298130450 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140101315 0 0) 0x7fb7dc0c61d0 con 0x7fb7dc0b1f70 2024-04-25T10:10:28.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7e0a65640 1 -- 172.21.15.133:0/1298130450 >> v1:172.21.15.133:6789/0 conn(0x7fb7dc0b1f70 legacy=0x7fb7dc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7e0a65640 1 -- 172.21.15.133:0/1298130450 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc06d7d0 con 0x7fb7dc15c9a0 2024-04-25T10:10:28.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7e0a65640 1 --2- 172.21.15.133:0/1298130450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc15c9a0 0x7fb7dc0c5980 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0097c0 tx=0x7fb7cc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d872fc186840d718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7db7fe640 1 -- 172.21.15.133:0/1298130450 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc0b1070 con 0x7fb7dc15c9a0 2024-04-25T10:10:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7db7fe640 1 -- 172.21.15.133:0/1298130450 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7cc0ac410 con 0x7fb7dc15c9a0 2024-04-25T10:10:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.782+0000 7fb7db7fe640 1 -- 172.21.15.133:0/1298130450 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc0ac7e0 con 0x7fb7dc15c9a0 2024-04-25T10:10:28.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/1298130450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc15c9a0 msgr2=0x7fb7dc0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 --2- 172.21.15.133:0/1298130450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc15c9a0 0x7fb7dc0c5980 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0097c0 tx=0x7fb7cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/1298130450 shutdown_connections 2024-04-25T10:10:28.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 --2- 172.21.15.133:0/1298130450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc15c9a0 0x7fb7dc0c5980 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/1298130450 >> 172.21.15.133:0/1298130450 conn(0x7fb7dc067000 msgr2=0x7fb7dc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:28.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/1298130450 shutdown_connections 2024-04-25T10:10:28.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/1298130450 wait complete. 2024-04-25T10:10:28.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 Processor -- start 2024-04-25T10:10:28.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 -- start start 2024-04-25T10:10:28.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc0b1f70 0x7fb7dc14efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0ce330 0x7fb7dc147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7dc1479c0 0x7fb7dc149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc15cf00 con 0x7fb7dc0ce330 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc15cd80 con 0x7fb7dc0b1f70 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc169de0 con 0x7fb7dc1479c0 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0ce330 0x7fb7dc147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0ce330 0x7fb7dc147390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42634/0 (socket says 172.21.15.133:42634) 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7dbfff640 1 -- 172.21.15.133:0/2888277381 learned_addr learned my addr 172.21.15.133:0/2888277381 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e0a65640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc0b1f70 0x7fb7dc14efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e1266640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7dc1479c0 0x7fb7dc149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7dbfff640 1 -- 172.21.15.133:0/2888277381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7dc1479c0 msgr2=0x7fb7dc149410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7dbfff640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7dc1479c0 0x7fb7dc149410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7dbfff640 1 -- 172.21.15.133:0/2888277381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc0b1f70 msgr2=0x7fb7dc14efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7dbfff640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc0b1f70 0x7fb7dc14efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7dbfff640 1 -- 172.21.15.133:0/2888277381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc148ba0 con 0x7fb7dc0ce330 2024-04-25T10:10:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e0a65640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc0b1f70 0x7fb7dc14efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7dbfff640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0ce330 0x7fb7dc147390 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000c410 tx=0x7fb7d000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7d97fa640 1 -- 172.21.15.133:0/2888277381 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00074e0 con 0x7fb7dc0ce330 2024-04-25T10:10:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7d97fa640 1 -- 172.21.15.133:0/2888277381 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d00047f0 con 0x7fb7dc0ce330 2024-04-25T10:10:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/2888277381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7dc14b280 con 0x7fb7dc0ce330 2024-04-25T10:10:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7d97fa640 1 -- 172.21.15.133:0/2888277381 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0007680 con 0x7fb7dc0ce330 2024-04-25T10:10:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.786+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/2888277381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc1481b0 con 0x7fb7dc0ce330 2024-04-25T10:10:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.790+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/2888277381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7dc068ef0 con 0x7fb7dc0ce330 2024-04-25T10:10:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.790+0000 7fb7d97fa640 1 -- 172.21.15.133:0/2888277381 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7d0011070 con 0x7fb7dc0ce330 2024-04-25T10:10:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.790+0000 7fb7d97fa640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7c8066f90 0x7fb7c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.790+0000 7fb7d97fa640 1 -- 172.21.15.133:0/2888277381 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb7d0084e10 con 0x7fb7dc0ce330 2024-04-25T10:10:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.790+0000 7fb7e0a65640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7c8066f90 0x7fb7c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.790+0000 7fb7e0a65640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7c8066f90 0x7fb7c8069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0ae000 tx=0x7fb7cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:28.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.794+0000 7fb7d97fa640 1 -- 172.21.15.133:0/2888277381 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7d0055c10 con 0x7fb7dc0ce330 2024-04-25T10:10:28.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.950+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/2888277381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7dc06c2d0 con 0x7fb7dc0ce330 2024-04-25T10:10:28.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.954+0000 7fb7d97fa640 1 -- 172.21.15.133:0/2888277381 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16922 (secure 0 0 0) 0x7fb7d0055510 con 0x7fb7dc0ce330 2024-04-25T10:10:28.958 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:28.958 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":93,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:10:27.660839+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:25.920690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:26.640190+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:28.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/2888277381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7c8066f90 msgr2=0x7fb7c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7c8066f90 0x7fb7c8069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0ae000 tx=0x7fb7cc008040 comp rx=0 tx=0).stop 2024-04-25T10:10:28.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/2888277381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0ce330 msgr2=0x7fb7dc147390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:28.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0ce330 0x7fb7dc147390 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000c410 tx=0x7fb7d000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/2888277381 shutdown_connections 2024-04-25T10:10:28.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7dc1479c0 0x7fb7dc149410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7c8066f90 0x7fb7c8069450 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0ce330 0x7fb7dc147390 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 --2- 172.21.15.133:0/2888277381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc0b1f70 0x7fb7dc14efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:28.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/2888277381 >> 172.21.15.133:0/2888277381 conn(0x7fb7dc067000 msgr2=0x7fb7dc0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:28.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/2888277381 shutdown_connections 2024-04-25T10:10:28.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:28.958+0000 7fb7e2cc9640 1 -- 172.21.15.133:0/2888277381 wait complete. 2024-04-25T10:10:28.978 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:10:29.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd61d41f640 1 Processor -- start 2024-04-25T10:10:29.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd61d41f640 1 -- start start 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd61d41f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd61806e8b0 0x7fd61815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd61d41f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6181471f0 con 0x7fd6180aebc0 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd61d41f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6180b1090 con 0x7fd6180b1e50 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd61d41f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd618147490 con 0x7fd61806e8b0 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd6177fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd6180aebc0 0x7fd61806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40134/0 (socket says 172.21.15.133:40134) 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd6177fe640 1 -- 172.21.15.133:0/3327438937 learned_addr learned my addr 172.21.15.133:0/3327438937 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd616ffd640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd61806e8b0 0x7fd61815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031165414 0 0) 0x7fd6181471f0 con 0x7fd6180aebc0 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd600003530 con 0x7fd6180aebc0 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1590367433 0 0) 0x7fd600003530 con 0x7fd6180aebc0 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd61806e8b0 msgr2=0x7fd61815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd61806e8b0 0x7fd61815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 >> v1:172.21.15.162:6789/0 conn(0x7fd6180b1e50 legacy=0x7fd618146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6180b1370 con 0x7fd6180aebc0 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fd610001a00 con 0x7fd6180aebc0 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 >> v1:172.21.15.133:6789/0 conn(0x7fd6180aebc0 legacy=0x7fd61806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6000039f0 0x7fd600003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd61806e8b0 0x7fd600004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd600004d50 0x7fd6000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618147490 con 0x7fd6000039f0 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60000bb00 con 0x7fd600004d50 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60000bd00 con 0x7fd61806e8b0 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd616ffd640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6000039f0 0x7fd600003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd6167fc640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd61806e8b0 0x7fd600004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd6177fe640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd600004d50 0x7fd6000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd616ffd640 1 -- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd61806e8b0 msgr2=0x7fd600004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd616ffd640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd61806e8b0 0x7fd600004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:29.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd616ffd640 1 -- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd600004d50 msgr2=0x7fd6000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:29.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd616ffd640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd600004d50 0x7fd6000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:29.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd616ffd640 1 -- 172.21.15.133:0/3327438937 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd600003530 con 0x7fd6000039f0 2024-04-25T10:10:29.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.102+0000 7fd6177fe640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd600004d50 0x7fd6000051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd616ffd640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6000039f0 0x7fd600003e10 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd608009ae0 tx=0x7fd60801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc997dd7cfc75b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080096b0 con 0x7fd6000039f0 2024-04-25T10:10:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6080b0400 con 0x7fd6000039f0 2024-04-25T10:10:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd615ffb640 1 -- 172.21.15.133:0/3327438937 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080b07f0 con 0x7fd6000039f0 2024-04-25T10:10:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd61d41f640 1 -- 172.21.15.133:0/3327438937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6000039f0 msgr2=0x7fd600003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd61d41f640 1 --2- 172.21.15.133:0/3327438937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6000039f0 0x7fd600003e10 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd608009ae0 tx=0x7fd60801c9b0 comp rx=0 tx=0).stop 2024-04-25T10:10:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd6177fe640 1 -- 172.21.15.133:0/3327438937 reap_dead start 2024-04-25T10:10:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd61d41f640 1 -- 172.21.15.133:0/3327438937 shutdown_connections 2024-04-25T10:10:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd61d41f640 1 -- 172.21.15.133:0/3327438937 >> 172.21.15.133:0/3327438937 conn(0x7fd618066f70 msgr2=0x7fd61806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd61d41f640 1 -- 172.21.15.133:0/3327438937 shutdown_connections 2024-04-25T10:10:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd61d41f640 1 -- 172.21.15.133:0/3327438937 wait complete. 2024-04-25T10:10:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd61d41f640 1 Processor -- start 2024-04-25T10:10:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.106+0000 7fd61d41f640 1 -- start start 2024-04-25T10:10:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd61d41f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000039f0 0x7fd61815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd61d41f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6181412e0 0x7fd61812eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd61d41f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61812f0d0 0x7fd6180adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd61d41f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180aeff0 con 0x7fd61812f0d0 2024-04-25T10:10:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd61d41f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180b19c0 con 0x7fd6181412e0 2024-04-25T10:10:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd61d41f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180af170 con 0x7fd6000039f0 2024-04-25T10:10:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd6177fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61812f0d0 0x7fd6180adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd6177fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61812f0d0 0x7fd6180adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42656/0 (socket says 172.21.15.133:42656) 2024-04-25T10:10:29.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd6177fe640 1 -- 172.21.15.133:0/1068822707 learned_addr learned my addr 172.21.15.133:0/1068822707 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:29.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd6167fc640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6181412e0 0x7fd61812eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:29.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd616ffd640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000039f0 0x7fd61815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:29.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd6177fe640 1 -- 172.21.15.133:0/1068822707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000039f0 msgr2=0x7fd61815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:29.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd6177fe640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000039f0 0x7fd61815d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:29.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd6177fe640 1 -- 172.21.15.133:0/1068822707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6181412e0 msgr2=0x7fd61812eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:29.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd6177fe640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6181412e0 0x7fd61812eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd6177fe640 1 -- 172.21.15.133:0/1068822707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61812f560 con 0x7fd61812f0d0 2024-04-25T10:10:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd6167fc640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6181412e0 0x7fd61812eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd6177fe640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61812f0d0 0x7fd6180adaa0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd61806b5e0 tx=0x7fd61000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd5ff7fe640 1 -- 172.21.15.133:0/1068822707 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610004d70 con 0x7fd61812f0d0 2024-04-25T10:10:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd5ff7fe640 1 -- 172.21.15.133:0/1068822707 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61000bb80 con 0x7fd61812f0d0 2024-04-25T10:10:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd61d41f640 1 -- 172.21.15.133:0/1068822707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61812f840 con 0x7fd61812f0d0 2024-04-25T10:10:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd5ff7fe640 1 -- 172.21.15.133:0/1068822707 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610007110 con 0x7fd61812f0d0 2024-04-25T10:10:29.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd61d41f640 1 -- 172.21.15.133:0/1068822707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6180adfe0 con 0x7fd61812f0d0 2024-04-25T10:10:29.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd61d41f640 1 -- 172.21.15.133:0/1068822707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd618068e80 con 0x7fd61812f0d0 2024-04-25T10:10:29.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd5ff7fe640 1 -- 172.21.15.133:0/1068822707 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd61001b020 con 0x7fd61812f0d0 2024-04-25T10:10:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd5ff7fe640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd604066f60 0x7fd604069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd5ff7fe640 1 -- 172.21.15.133:0/1068822707 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6100852a0 con 0x7fd61812f0d0 2024-04-25T10:10:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd616ffd640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd604066f60 0x7fd604069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.110+0000 7fd616ffd640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd604066f60 0x7fd604069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd6080af000 tx=0x7fd6080046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.114+0000 7fd5ff7fe640 1 -- 172.21.15.133:0/1068822707 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6100561b0 con 0x7fd61812f0d0 2024-04-25T10:10:29.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:29.310+0000 7fd61d41f640 1 -- 172.21.15.133:0/1068822707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd618161ca0 con 0x7fd61812f0d0 2024-04-25T10:10:30.263 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-04-25T10:10:30.258+0000 7f2bd7e97640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:10:30.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.330+0000 7fd5ff7fe640 1 -- 172.21.15.133:0/1068822707 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd610055ab0 con 0x7fd61812f0d0 2024-04-25T10:10:30.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 -- 172.21.15.133:0/1068822707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd604066f60 msgr2=0x7fd604069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:30.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd604066f60 0x7fd604069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd6080af000 tx=0x7fd6080046d0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 -- 172.21.15.133:0/1068822707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61812f0d0 msgr2=0x7fd6180adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:30.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61812f0d0 0x7fd6180adaa0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd61806b5e0 tx=0x7fd61000b4a0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 -- 172.21.15.133:0/1068822707 shutdown_connections 2024-04-25T10:10:30.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd604066f60 0x7fd604069420 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd61812f0d0 0x7fd6180adaa0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6181412e0 0x7fd61812eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 --2- 172.21.15.133:0/1068822707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6000039f0 0x7fd61815d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 -- 172.21.15.133:0/1068822707 >> 172.21.15.133:0/1068822707 conn(0x7fd618066f70 msgr2=0x7fd61815df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:30.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 -- 172.21.15.133:0/1068822707 shutdown_connections 2024-04-25T10:10:30.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.334+0000 7fd61d41f640 1 -- 172.21.15.133:0/1068822707 wait complete. 2024-04-25T10:10:30.360 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:10:30.360 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:10:30.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.482+0000 7f8fcedaa640 1 Processor -- start 2024-04-25T10:10:30.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fcedaa640 1 -- start start 2024-04-25T10:10:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fcedaa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fc80cd820 0x7f8fc80cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fcedaa640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8fc815ced0 0x7f8fc815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fcedaa640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc80ce2e0 con 0x7f8fc8145170 2024-04-25T10:10:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fcedaa640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc806db00 con 0x7f8fc80cd820 2024-04-25T10:10:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fcedaa640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc8145550 con 0x7f8fc815ced0 2024-04-25T10:10:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fc7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fc80cd820 0x7f8fc80cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fc7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fc80cd820 0x7f8fc80cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42668/0 (socket says 172.21.15.133:42668) 2024-04-25T10:10:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fc7fff640 1 -- 172.21.15.133:0/3918606670 learned_addr learned my addr 172.21.15.133:0/3918606670 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fcd347640 1 --2- 172.21.15.133:0/3918606670 >> v2:172.21.15.162:3300/0 conn(0x7f8fc815ced0 0x7f8fc815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fc7fff640 1 -- 172.21.15.133:0/3918606670 >> v2:172.21.15.162:3300/0 conn(0x7f8fc815ced0 msgr2=0x7f8fc815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:30.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fc7fff640 1 --2- 172.21.15.133:0/3918606670 >> v2:172.21.15.162:3300/0 conn(0x7f8fc815ced0 0x7f8fc815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fc7fff640 1 -- 172.21.15.133:0/3918606670 >> v1:172.21.15.162:6789/0 conn(0x7f8fc8145170 legacy=0x7f8fc806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:30.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fc7fff640 1 -- 172.21.15.133:0/3918606670 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc806d4b0 con 0x7f8fc80cd820 2024-04-25T10:10:30.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.486+0000 7f8fcd347640 1 --2- 172.21.15.133:0/3918606670 >> v2:172.21.15.162:3300/0 conn(0x7f8fc815ced0 0x7f8fc815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:30.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fc7fff640 1 --2- 172.21.15.133:0/3918606670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc80cd820 0x7f8fc80cdca0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80b27f0 tx=0x7f8fbc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22830713067f6b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fc77fe640 1 -- 172.21.15.133:0/3918606670 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0b0070 con 0x7f8fc80cd820 2024-04-25T10:10:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fc77fe640 1 -- 172.21.15.133:0/3918606670 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0ab600 con 0x7f8fc80cd820 2024-04-25T10:10:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fc77fe640 1 -- 172.21.15.133:0/3918606670 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0ab9d0 con 0x7f8fc80cd820 2024-04-25T10:10:30.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fcedaa640 1 -- 172.21.15.133:0/3918606670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc80cd820 msgr2=0x7f8fc80cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fcedaa640 1 --2- 172.21.15.133:0/3918606670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc80cd820 0x7f8fc80cdca0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80b27f0 tx=0x7f8fbc01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fcedaa640 1 -- 172.21.15.133:0/3918606670 shutdown_connections 2024-04-25T10:10:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fcedaa640 1 --2- 172.21.15.133:0/3918606670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc80cd820 0x7f8fc80cdca0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fcedaa640 1 --2- 172.21.15.133:0/3918606670 >> v2:172.21.15.162:3300/0 conn(0x7f8fc815ced0 0x7f8fc815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fcedaa640 1 -- 172.21.15.133:0/3918606670 >> 172.21.15.133:0/3918606670 conn(0x7f8fc8066f70 msgr2=0x7f8fc806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fcedaa640 1 -- 172.21.15.133:0/3918606670 shutdown_connections 2024-04-25T10:10:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fcedaa640 1 -- 172.21.15.133:0/3918606670 wait complete. 2024-04-25T10:10:30.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fcedaa640 1 Processor -- start 2024-04-25T10:10:30.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.490+0000 7f8fcedaa640 1 -- start start 2024-04-25T10:10:30.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fcedaa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fc80cd820 0x7f8fc8144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:30.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fcedaa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc8145170 0x7f8fc8165870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fcedaa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fc815ced0 0x7f8fc8165db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fcedaa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc8168610 con 0x7f8fc8145170 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fcedaa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc8168490 con 0x7f8fc815ced0 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fcedaa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc8168790 con 0x7f8fc80cd820 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc8145170 0x7f8fc8165870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc8145170 0x7f8fc8165870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42678/0 (socket says 172.21.15.133:42678) 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc7fff640 1 -- 172.21.15.133:0/2998131284 learned_addr learned my addr 172.21.15.133:0/2998131284 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc7fff640 1 -- 172.21.15.133:0/2998131284 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fc80cd820 msgr2=0x7f8fc8144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc7fff640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fc80cd820 0x7f8fc8144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc7fff640 1 -- 172.21.15.133:0/2998131284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fc815ced0 msgr2=0x7f8fc8165db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc7fff640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fc815ced0 0x7f8fc8165db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc7fff640 1 -- 172.21.15.133:0/2998131284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc8137490 con 0x7f8fc8145170 2024-04-25T10:10:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc7fff640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc8145170 0x7f8fc8165870 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc01ab90 tx=0x7f8fbc01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:30.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc57fa640 1 -- 172.21.15.133:0/2998131284 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0ba020 con 0x7f8fc8145170 2024-04-25T10:10:30.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc57fa640 1 -- 172.21.15.133:0/2998131284 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0b6640 con 0x7f8fc8145170 2024-04-25T10:10:30.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc57fa640 1 -- 172.21.15.133:0/2998131284 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0b0040 con 0x7f8fc8145170 2024-04-25T10:10:30.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fcedaa640 1 -- 172.21.15.133:0/2998131284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc8137770 con 0x7f8fc8145170 2024-04-25T10:10:30.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fcedaa640 1 -- 172.21.15.133:0/2998131284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc81370b0 con 0x7f8fc8145170 2024-04-25T10:10:30.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fcedaa640 1 -- 172.21.15.133:0/2998131284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc8068e80 con 0x7f8fc8145170 2024-04-25T10:10:30.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc57fa640 1 -- 172.21.15.133:0/2998131284 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fbc0bd020 con 0x7f8fc8145170 2024-04-25T10:10:30.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc57fa640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fb4066f60 0x7f8fb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:30.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fc57fa640 1 -- 172.21.15.133:0/2998131284 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8fbc11f8f0 con 0x7f8fc8145170 2024-04-25T10:10:30.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fccb46640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fb4066f60 0x7f8fb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:30.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.494+0000 7f8fccb46640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fb4066f60 0x7f8fb4069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8003cf0 tx=0x7f8fb80079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:30.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.498+0000 7f8fc57fa640 1 -- 172.21.15.133:0/2998131284 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fbc0c4030 con 0x7f8fc8145170 2024-04-25T10:10:30.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:30.678+0000 7f8fcedaa640 1 -- 172.21.15.133:0/2998131284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8fc81449a0 con 0x7f8fc8145170 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.330+0000 7f8fc57fa640 1 -- 172.21.15.133:0/2998131284 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8fbc0160d0 con 0x7f8fc8145170 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 -- 172.21.15.133:0/2998131284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fb4066f60 msgr2=0x7f8fb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fb4066f60 0x7f8fb4069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8003cf0 tx=0x7f8fb80079c0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 -- 172.21.15.133:0/2998131284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc8145170 msgr2=0x7f8fc8165870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc8145170 0x7f8fc8165870 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc01ab90 tx=0x7f8fbc01ad20 comp rx=0 tx=0).stop 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 -- 172.21.15.133:0/2998131284 shutdown_connections 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fc815ced0 0x7f8fc8165db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fb4066f60 0x7f8fb4069420 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc8145170 0x7f8fc8165870 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 --2- 172.21.15.133:0/2998131284 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fc80cd820 0x7f8fc8144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 -- 172.21.15.133:0/2998131284 >> 172.21.15.133:0/2998131284 conn(0x7f8fc8066f70 msgr2=0x7f8fc815c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 -- 172.21.15.133:0/2998131284 shutdown_connections 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.334+0000 7f8fcedaa640 1 -- 172.21.15.133:0/2998131284 wait complete. 2024-04-25T10:10:31.357 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:10:31.357 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:31.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.474+0000 7efc1a7d5640 1 Processor -- start 2024-04-25T10:10:31.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.474+0000 7efc1a7d5640 1 -- start start 2024-04-25T10:10:31.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.474+0000 7efc1a7d5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efc1415db00 0x7efc140b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.474+0000 7efc1a7d5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc140ceb20 con 0x7efc140d0770 2024-04-25T10:10:31.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.474+0000 7efc1a7d5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc140b97f0 con 0x7efc140d0bd0 2024-04-25T10:10:31.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.474+0000 7efc1a7d5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc141622e0 con 0x7efc1415db00 2024-04-25T10:10:31.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.474+0000 7efc13fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efc1415db00 0x7efc140b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.474+0000 7efc137fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7efc140d0770 0x7efc141614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40142/0 (socket says 172.21.15.133:40142) 2024-04-25T10:10:31.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.474+0000 7efc137fe640 1 -- 172.21.15.133:0/3533782242 learned_addr learned my addr 172.21.15.133:0/3533782242 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:31.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.474+0000 7efc13fff640 1 -- 172.21.15.133:0/3533782242 >> v1:172.21.15.162:6789/0 conn(0x7efc140d0bd0 legacy=0x7efc14161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc13fff640 1 -- 172.21.15.133:0/3533782242 >> v1:172.21.15.133:6789/0 conn(0x7efc140d0770 legacy=0x7efc141614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc13fff640 1 -- 172.21.15.133:0/3533782242 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc140b7740 con 0x7efc1415db00 2024-04-25T10:10:31.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc12ffd640 1 -- 172.21.15.133:0/3533782242 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094401924 0 0) 0x7efc140ceb20 con 0x7efc140d0770 2024-04-25T10:10:31.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc13fff640 1 --2- 172.21.15.133:0/3533782242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc1415db00 0x7efc140b2990 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7efc040097c0 tx=0x7efc0401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b727efe061613e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc12ffd640 1 -- 172.21.15.133:0/3533782242 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc040ab430 con 0x7efc1415db00 2024-04-25T10:10:31.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc12ffd640 1 -- 172.21.15.133:0/3533782242 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc040b0070 con 0x7efc1415db00 2024-04-25T10:10:31.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc12ffd640 1 -- 172.21.15.133:0/3533782242 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc040ab7f0 con 0x7efc1415db00 2024-04-25T10:10:31.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3533782242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc1415db00 msgr2=0x7efc140b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc1a7d5640 1 --2- 172.21.15.133:0/3533782242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc1415db00 0x7efc140b2990 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7efc040097c0 tx=0x7efc0401c430 comp rx=0 tx=0).stop 2024-04-25T10:10:31.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3533782242 shutdown_connections 2024-04-25T10:10:31.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc1a7d5640 1 --2- 172.21.15.133:0/3533782242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc1415db00 0x7efc140b2990 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3533782242 >> 172.21.15.133:0/3533782242 conn(0x7efc140727c0 msgr2=0x7efc14072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3533782242 shutdown_connections 2024-04-25T10:10:31.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.478+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3533782242 wait complete. 2024-04-25T10:10:31.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc1a7d5640 1 Processor -- start 2024-04-25T10:10:31.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc1a7d5640 1 -- start start 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc1a7d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc140d0770 0x7efc1414a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc1a7d5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc140d0bd0 0x7efc140d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc1a7d5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc1415db00 0x7efc1415e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc1a7d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140d1060 con 0x7efc140d0770 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc1a7d5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140b82c0 con 0x7efc140d0bd0 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc1a7d5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc140b7430 con 0x7efc1415db00 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc13fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc140d0770 0x7efc1414a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc13fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc140d0770 0x7efc1414a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42702/0 (socket says 172.21.15.133:42702) 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc13fff640 1 -- 172.21.15.133:0/3196860782 learned_addr learned my addr 172.21.15.133:0/3196860782 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc13fff640 1 -- 172.21.15.133:0/3196860782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc1415db00 msgr2=0x7efc1415e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc13fff640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc1415db00 0x7efc1415e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc13fff640 1 -- 172.21.15.133:0/3196860782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc140d0bd0 msgr2=0x7efc140d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc13fff640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc140d0bd0 0x7efc140d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc13fff640 1 -- 172.21.15.133:0/3196860782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1414ad10 con 0x7efc140d0770 2024-04-25T10:10:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc13fff640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc140d0770 0x7efc1414a370 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7efc040097c0 tx=0x7efc0401ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc10ff9640 1 -- 172.21.15.133:0/3196860782 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04004220 con 0x7efc140d0770 2024-04-25T10:10:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc10ff9640 1 -- 172.21.15.133:0/3196860782 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc040043c0 con 0x7efc140d0770 2024-04-25T10:10:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3196860782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1414b2a0 con 0x7efc140d0770 2024-04-25T10:10:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc10ff9640 1 -- 172.21.15.133:0/3196860782 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04003820 con 0x7efc140d0770 2024-04-25T10:10:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3196860782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1414b860 con 0x7efc140d0770 2024-04-25T10:10:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3196860782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1414baa0 con 0x7efc140d0770 2024-04-25T10:10:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.482+0000 7efc10ff9640 1 -- 172.21.15.133:0/3196860782 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc040039c0 con 0x7efc140d0770 2024-04-25T10:10:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.486+0000 7efc10ff9640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc00066f60 0x7efc00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.486+0000 7efc10ff9640 1 -- 172.21.15.133:0/3196860782 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efc040a8020 con 0x7efc140d0770 2024-04-25T10:10:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.486+0000 7efc137fe640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc00066f60 0x7efc00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.486+0000 7efc137fe640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc00066f60 0x7efc00069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7efc08000c10 tx=0x7efc08007910 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.490+0000 7efc10ff9640 1 -- 172.21.15.133:0/3196860782 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efc1414baa0 con 0x7efc140d0770 2024-04-25T10:10:31.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.682+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3196860782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc14068ef0 con 0x7efc140d0770 2024-04-25T10:10:31.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.682+0000 7efc10ff9640 1 -- 172.21.15.133:0/3196860782 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4952 (secure 0 0 0) 0x7efc040c4030 con 0x7efc140d0770 2024-04-25T10:10:31.686 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:31.686 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3089710291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3089710291},{"type":"v1","addr":"172.21.15.162:6834","nonce":3089710291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1390003376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1390003376},{"type":"v1","addr":"172.21.15.162:6835","nonce":1390003376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7866,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1134679682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1134679682},{"type":"v1","addr":"172.21.15.133:6836","nonce":1134679682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:27.661584+0000","modified":"2024-04-25T10:10:30.680424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3580592279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3580592279},{"type":"v1","addr":"172.21.15.133:6837","nonce":3580592279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:31.686 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 58 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3196860782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc00066f60 msgr2=0x7efc00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc00066f60 0x7efc00069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7efc08000c10 tx=0x7efc08007910 comp rx=0 tx=0).stop 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3196860782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc140d0770 msgr2=0x7efc1414a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc140d0770 0x7efc1414a370 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7efc040097c0 tx=0x7efc0401ac70 comp rx=0 tx=0).stop 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3196860782 shutdown_connections 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc1415db00 0x7efc1415e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc140d0bd0 0x7efc140d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc00066f60 0x7efc00069420 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 --2- 172.21.15.133:0/3196860782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc140d0770 0x7efc1414a370 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3196860782 >> 172.21.15.133:0/3196860782 conn(0x7efc140727c0 msgr2=0x7efc140b87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3196860782 shutdown_connections 2024-04-25T10:10:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.686+0000 7efc1a7d5640 1 -- 172.21.15.133:0/3196860782 wait complete. 2024-04-25T10:10:31.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:31.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f415efd4640 1 Processor -- start 2024-04-25T10:10:31.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f415efd4640 1 -- start start 2024-04-25T10:10:31.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f415efd4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f415815e1d0 0x7f41580cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f415efd4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f41580cbad0 0x7f41580cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f415efd4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415815ea00 con 0x7f4158157c10 2024-04-25T10:10:31.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f415efd4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41580c4770 con 0x7f415815e1d0 2024-04-25T10:10:31.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f415efd4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415815e720 con 0x7f41580cbad0 2024-04-25T10:10:31.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f4157fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f415815e1d0 0x7f41580cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f4157fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f415815e1d0 0x7f41580cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42706/0 (socket says 172.21.15.133:42706) 2024-04-25T10:10:31.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f4157fff640 1 -- 172.21.15.133:0/3880738472 learned_addr learned my addr 172.21.15.133:0/3880738472 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:31.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f415d571640 1 --2- 172.21.15.133:0/3880738472 >> v2:172.21.15.162:3300/0 conn(0x7f41580cbad0 0x7f41580cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f4157fff640 1 -- 172.21.15.133:0/3880738472 >> v2:172.21.15.162:3300/0 conn(0x7f41580cbad0 msgr2=0x7f41580cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f4157fff640 1 --2- 172.21.15.133:0/3880738472 >> v2:172.21.15.162:3300/0 conn(0x7f41580cbad0 0x7f41580cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f4157fff640 1 -- 172.21.15.133:0/3880738472 >> v1:172.21.15.162:6789/0 conn(0x7f4158157c10 legacy=0x7f415815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.818+0000 7f4157fff640 1 -- 172.21.15.133:0/3880738472 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580c4a10 con 0x7f415815e1d0 2024-04-25T10:10:31.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f4157fff640 1 --2- 172.21.15.133:0/3880738472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f415815e1d0 0x7f41580cf070 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f4158071160 tx=0x7f414c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14a21020e558b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f41577fe640 1 -- 172.21.15.133:0/3880738472 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c0b2070 con 0x7f415815e1d0 2024-04-25T10:10:31.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f41577fe640 1 -- 172.21.15.133:0/3880738472 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c0a9070 con 0x7f415815e1d0 2024-04-25T10:10:31.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f41577fe640 1 -- 172.21.15.133:0/3880738472 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c0ad880 con 0x7f415815e1d0 2024-04-25T10:10:31.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f415efd4640 1 -- 172.21.15.133:0/3880738472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f415815e1d0 msgr2=0x7f41580cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f415efd4640 1 --2- 172.21.15.133:0/3880738472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f415815e1d0 0x7f41580cf070 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f4158071160 tx=0x7f414c01c450 comp rx=0 tx=0).stop 2024-04-25T10:10:31.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f415efd4640 1 -- 172.21.15.133:0/3880738472 shutdown_connections 2024-04-25T10:10:31.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f415efd4640 1 --2- 172.21.15.133:0/3880738472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f415815e1d0 0x7f41580cf070 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f415efd4640 1 --2- 172.21.15.133:0/3880738472 >> v2:172.21.15.162:3300/0 conn(0x7f41580cbad0 0x7f41580cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f415efd4640 1 -- 172.21.15.133:0/3880738472 >> 172.21.15.133:0/3880738472 conn(0x7f4158067000 msgr2=0x7f415806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f415efd4640 1 -- 172.21.15.133:0/3880738472 shutdown_connections 2024-04-25T10:10:31.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f415efd4640 1 -- 172.21.15.133:0/3880738472 wait complete. 2024-04-25T10:10:31.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f415efd4640 1 Processor -- start 2024-04-25T10:10:31.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.822+0000 7f415efd4640 1 -- start start 2024-04-25T10:10:31.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415efd4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41580cbad0 0x7f41580caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415efd4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4158157c10 0x7f415814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415efd4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f415815e1d0 0x7f41580c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415efd4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415806ebe0 con 0x7f4158157c10 2024-04-25T10:10:31.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415efd4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415806ea60 con 0x7f41580cbad0 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415efd4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415806b3a0 con 0x7f415815e1d0 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f4157fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4158157c10 0x7f415814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f4157fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4158157c10 0x7f415814cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42710/0 (socket says 172.21.15.133:42710) 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f4157fff640 1 -- 172.21.15.133:0/860144032 learned_addr learned my addr 172.21.15.133:0/860144032 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415cd70640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41580cbad0 0x7f41580caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415d571640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f415815e1d0 0x7f41580c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f4157fff640 1 -- 172.21.15.133:0/860144032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f415815e1d0 msgr2=0x7f41580c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f4157fff640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f415815e1d0 0x7f41580c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f4157fff640 1 -- 172.21.15.133:0/860144032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41580cbad0 msgr2=0x7f41580caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f4157fff640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41580cbad0 0x7f41580caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f4157fff640 1 -- 172.21.15.133:0/860144032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580cb000 con 0x7f4158157c10 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415cd70640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41580cbad0 0x7f41580caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415d571640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f415815e1d0 0x7f41580c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f4157fff640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4158157c10 0x7f415814cea0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f414c01a080 tx=0x7f414c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f41557fa640 1 -- 172.21.15.133:0/860144032 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c0bc070 con 0x7f4158157c10 2024-04-25T10:10:31.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f41557fa640 1 -- 172.21.15.133:0/860144032 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c0b6620 con 0x7f4158157c10 2024-04-25T10:10:31.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415efd4640 1 -- 172.21.15.133:0/860144032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41581487d0 con 0x7f4158157c10 2024-04-25T10:10:31.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f41557fa640 1 -- 172.21.15.133:0/860144032 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c0b2040 con 0x7f4158157c10 2024-04-25T10:10:31.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415efd4640 1 -- 172.21.15.133:0/860144032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4158148d90 con 0x7f4158157c10 2024-04-25T10:10:31.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415efd4640 1 -- 172.21.15.133:0/860144032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4158068ef0 con 0x7f4158157c10 2024-04-25T10:10:31.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f41557fa640 1 -- 172.21.15.133:0/860144032 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f414c0bf070 con 0x7f4158157c10 2024-04-25T10:10:31.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f41557fa640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f413c066f60 0x7f413c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f41557fa640 1 -- 172.21.15.133:0/860144032 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f414c11e720 con 0x7f4158157c10 2024-04-25T10:10:31.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.826+0000 7f415cd70640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f413c066f60 0x7f413c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.830+0000 7f415cd70640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f413c066f60 0x7f413c069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4144003e40 tx=0x7f4144009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.830+0000 7f41557fa640 1 -- 172.21.15.133:0/860144032 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f414c0160d0 con 0x7f4158157c10 2024-04-25T10:10:31.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.982+0000 7f415efd4640 1 -- 172.21.15.133:0/860144032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41580cc970 con 0x7f4158157c10 2024-04-25T10:10:31.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f41557fa640 1 -- 172.21.15.133:0/860144032 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16909 (secure 0 0 0) 0x7f414c0ef5a0 con 0x7f4158157c10 2024-04-25T10:10:31.988 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:31.988 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":94,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:10:29.278587+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:25.920690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:26.640190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:31.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 -- 172.21.15.133:0/860144032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f413c066f60 msgr2=0x7f413c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f413c066f60 0x7f413c069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4144003e40 tx=0x7f4144009040 comp rx=0 tx=0).stop 2024-04-25T10:10:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 -- 172.21.15.133:0/860144032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4158157c10 msgr2=0x7f415814cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4158157c10 0x7f415814cea0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f414c01a080 tx=0x7f414c0191f0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 -- 172.21.15.133:0/860144032 shutdown_connections 2024-04-25T10:10:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f415815e1d0 0x7f41580c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f413c066f60 0x7f413c069420 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4158157c10 0x7f415814cea0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 --2- 172.21.15.133:0/860144032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41580cbad0 0x7f41580caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 -- 172.21.15.133:0/860144032 >> 172.21.15.133:0/860144032 conn(0x7f4158067000 msgr2=0x7f415815ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 -- 172.21.15.133:0/860144032 shutdown_connections 2024-04-25T10:10:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:31.986+0000 7f415efd4640 1 -- 172.21.15.133:0/860144032 wait complete. 2024-04-25T10:10:32.006 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:10:32.006 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:10:32.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.114+0000 7fa7efabb640 1 Processor -- start 2024-04-25T10:10:32.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.114+0000 7fa7efabb640 1 -- start start 2024-04-25T10:10:32.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7efabb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80d0770 0x7fa7e80d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7efabb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa7e80b87e0 0x7fa7e80b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7efabb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e80ceb20 con 0x7fa7e815db00 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7efabb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e80d1110 con 0x7fa7e80b87e0 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7efabb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80b30a0 con 0x7fa7e80d0770 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ed857640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa7e815db00 0x7fa7e80b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40156/0 (socket says 172.21.15.133:40156) 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ed857640 1 -- 172.21.15.133:0/2035578751 learned_addr learned my addr 172.21.15.133:0/2035578751 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218732842 0 0) 0x7fa7e80ceb20 con 0x7fa7e815db00 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ed056640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80d0770 0x7fa7e80d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7d4003530 con 0x7fa7e815db00 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ee058640 1 --2- 172.21.15.133:0/2035578751 >> v2:172.21.15.162:3300/0 conn(0x7fa7e80b87e0 0x7fa7e80b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3307776306 0 0) 0x7fa7d4003530 con 0x7fa7e815db00 2024-04-25T10:10:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80d0770 msgr2=0x7fa7e80d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80d0770 0x7fa7e80d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 >> v2:172.21.15.162:3300/0 conn(0x7fa7e80b87e0 msgr2=0x7fa7e80b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 --2- 172.21.15.133:0/2035578751 >> v2:172.21.15.162:3300/0 conn(0x7fa7e80b87e0 0x7fa7e80b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e80b7740 con 0x7fa7e815db00 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa7d8001680 con 0x7fa7e815db00 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 >> v1:172.21.15.133:6789/0 conn(0x7fa7e815db00 legacy=0x7fa7e80b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7d40039c0 0x7fa7d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d4004810 0x7fa7d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80d0770 0x7fa7d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80b30a0 con 0x7fa7d40039c0 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d400b9d0 con 0x7fa7d4004810 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d400bb50 con 0x7fa7e80d0770 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ed857640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7d40039c0 0x7fa7d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ee058640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80d0770 0x7fa7d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ed857640 1 -- 172.21.15.133:0/2035578751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80d0770 msgr2=0x7fa7d40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:32.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ed857640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80d0770 0x7fa7d40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ed857640 1 -- 172.21.15.133:0/2035578751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d4004810 msgr2=0x7fa7d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:32.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ed857640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d4004810 0x7fa7d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ed857640 1 -- 172.21.15.133:0/2035578751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d4003530 con 0x7fa7d40039c0 2024-04-25T10:10:32.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ee058640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80d0770 0x7fa7d40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:32.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ed857640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7d40039c0 0x7fa7d4003de0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c210 tx=0x7fa7d801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2494443872010ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8007d60 con 0x7fa7d40039c0 2024-04-25T10:10:32.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d80a9020 con 0x7fa7d40039c0 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.118+0000 7fa7ec855640 1 -- 172.21.15.133:0/2035578751 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8005b20 con 0x7fa7d40039c0 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 -- 172.21.15.133:0/2035578751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7d40039c0 msgr2=0x7fa7d4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 --2- 172.21.15.133:0/2035578751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7d40039c0 0x7fa7d4003de0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c210 tx=0x7fa7d801cc70 comp rx=0 tx=0).stop 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7ee058640 1 -- 172.21.15.133:0/2035578751 reap_dead start 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 -- 172.21.15.133:0/2035578751 shutdown_connections 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 -- 172.21.15.133:0/2035578751 >> 172.21.15.133:0/2035578751 conn(0x7fa7e80727c0 msgr2=0x7fa7e8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 -- 172.21.15.133:0/2035578751 shutdown_connections 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 -- 172.21.15.133:0/2035578751 wait complete. 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 Processor -- start 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 -- start start 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d40039c0 0x7fa7e814bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80ccf20 0x7fa7e80b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e8163860 0x7fa7e80b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80b7430 con 0x7fa7e8163860 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80b82c0 con 0x7fa7d40039c0 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80d18a0 con 0x7fa7e80ccf20 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7ee058640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e8163860 0x7fa7e80b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7ee058640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e8163860 0x7fa7e80b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42732/0 (socket says 172.21.15.133:42732) 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7ee058640 1 -- 172.21.15.133:0/1484039571 learned_addr learned my addr 172.21.15.133:0/1484039571 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7ed056640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80ccf20 0x7fa7e80b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7ee058640 1 -- 172.21.15.133:0/1484039571 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80ccf20 msgr2=0x7fa7e80b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7ee058640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80ccf20 0x7fa7e80b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7ee058640 1 -- 172.21.15.133:0/1484039571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d40039c0 msgr2=0x7fa7e814bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7ee058640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d40039c0 0x7fa7e814bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7ee058640 1 -- 172.21.15.133:0/1484039571 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e80cbe80 con 0x7fa7e8163860 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7ee058640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e8163860 0x7fa7e80b37c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80cfe70 tx=0x7fa7e000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7de7fc640 1 -- 172.21.15.133:0/1484039571 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e001c070 con 0x7fa7e8163860 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7de7fc640 1 -- 172.21.15.133:0/1484039571 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e0017400 con 0x7fa7e8163860 2024-04-25T10:10:32.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 -- 172.21.15.133:0/1484039571 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e80cc160 con 0x7fa7e8163860 2024-04-25T10:10:32.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7de7fc640 1 -- 172.21.15.133:0/1484039571 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00177d0 con 0x7fa7e8163860 2024-04-25T10:10:32.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.122+0000 7fa7efabb640 1 -- 172.21.15.133:0/1484039571 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e80cc720 con 0x7fa7e8163860 2024-04-25T10:10:32.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.126+0000 7fa7efabb640 1 -- 172.21.15.133:0/1484039571 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e8068ef0 con 0x7fa7e8163860 2024-04-25T10:10:32.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.126+0000 7fa7de7fc640 1 -- 172.21.15.133:0/1484039571 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7e001f070 con 0x7fa7e8163860 2024-04-25T10:10:32.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.126+0000 7fa7de7fc640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7cc066f90 0x7fa7cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.126+0000 7fa7de7fc640 1 -- 172.21.15.133:0/1484039571 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa7e0085140 con 0x7fa7e8163860 2024-04-25T10:10:32.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.126+0000 7fa7ed857640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7cc066f90 0x7fa7cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.126+0000 7fa7ed857640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7cc066f90 0x7fa7cc069450 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80aa000 tx=0x7fa7e80b7910 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.130+0000 7fa7de7fc640 1 -- 172.21.15.133:0/1484039571 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7e0012070 con 0x7fa7e8163860 2024-04-25T10:10:32.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.282+0000 7fa7efabb640 1 -- 172.21.15.133:0/1484039571 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7e80b2750 con 0x7fa7cc066f90 2024-04-25T10:10:32.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.294+0000 7fa7de7fc640 1 -- 172.21.15.133:0/1484039571 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252040 (secure 0 0 0) 0x7fa7e80b2750 con 0x7fa7cc066f90 2024-04-25T10:10:32.299 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:32.301 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:10:32.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 -- 172.21.15.133:0/1484039571 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7cc066f90 msgr2=0x7fa7cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7cc066f90 0x7fa7cc069450 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80aa000 tx=0x7fa7e80b7910 comp rx=0 tx=0).stop 2024-04-25T10:10:32.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 -- 172.21.15.133:0/1484039571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e8163860 msgr2=0x7fa7e80b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e8163860 0x7fa7e80b37c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80cfe70 tx=0x7fa7e000f4a0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 -- 172.21.15.133:0/1484039571 shutdown_connections 2024-04-25T10:10:32.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7cc066f90 0x7fa7cc069450 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e8163860 0x7fa7e80b37c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e80ccf20 0x7fa7e80b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 --2- 172.21.15.133:0/1484039571 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7d40039c0 0x7fa7e814bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 -- 172.21.15.133:0/1484039571 >> 172.21.15.133:0/1484039571 conn(0x7fa7e80727c0 msgr2=0x7fa7e815b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:32.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 -- 172.21.15.133:0/1484039571 shutdown_connections 2024-04-25T10:10:32.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.298+0000 7fa7efabb640 1 -- 172.21.15.133:0/1484039571 wait complete. 2024-04-25T10:10:32.317 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-04-25T10:10:31.866869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":492904,"kb_used_data":10208,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754481816,"statfs":{"total":773094113280,"available":772589379584,"internally_reserved":0,"allocated":10452992,"data_stored":5442412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.599974"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.319250+0000","last_change":"2024-04-25T10:10:28.318080+0000","last_active":"2024-04-25T10:10:28.319250+0000","last_peered":"2024-04-25T10:10:28.319250+0000","last_clean":"2024-04-25T10:10:28.319250+0000","last_became_active":"2024-04-25T10:10:28.317817+0000","last_became_peered":"2024-04-25T10:10:28.317817+0000","last_unstale":"2024-04-25T10:10:28.319250+0000","last_undegraded":"2024-04-25T10:10:28.319250+0000","last_fullsized":"2024-04-25T10:10:28.319250+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:36.803150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.339483+0000","last_change":"2024-04-25T10:10:28.324650+0000","last_active":"2024-04-25T10:10:28.339483+0000","last_peered":"2024-04-25T10:10:28.339483+0000","last_clean":"2024-04-25T10:10:28.339483+0000","last_became_active":"2024-04-25T10:10:28.324140+0000","last_became_peered":"2024-04-25T10:10:28.324140+0000","last_unstale":"2024-04-25T10:10:28.339483+0000","last_undegraded":"2024-04-25T10:10:28.339483+0000","last_fullsized":"2024-04-25T10:10:28.339483+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:21.813044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.319134+0000","last_change":"2024-04-25T10:10:28.316929+0000","last_active":"2024-04-25T10:10:28.319134+0000","last_peered":"2024-04-25T10:10:28.319134+0000","last_clean":"2024-04-25T10:10:28.319134+0000","last_became_active":"2024-04-25T10:10:28.316559+0000","last_became_peered":"2024-04-25T10:10:28.316559+0000","last_unstale":"2024-04-25T10:10:28.319134+0000","last_undegraded":"2024-04-25T10:10:28.319134+0000","last_fullsized":"2024-04-25T10:10:28.319134+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:24.133926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.331625+0000","last_change":"2024-04-25T10:10:28.330452+0000","last_active":"2024-04-25T10:10:28.331625+0000","last_peered":"2024-04-25T10:10:28.331625+0000","last_clean":"2024-04-25T10:10:28.331625+0000","last_became_active":"2024-04-25T10:10:28.326565+0000","last_became_peered":"2024-04-25T10:10:28.326565+0000","last_unstale":"2024-04-25T10:10:28.331625+0000","last_undegraded":"2024-04-25T10:10:28.331625+0000","last_fullsized":"2024-04-25T10:10:28.331625+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:08.050216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288276+0000","last_change":"2024-04-25T10:10:28.308300+0000","last_active":"2024-04-25T10:10:29.288276+0000","last_peered":"2024-04-25T10:10:29.288276+0000","last_clean":"2024-04-25T10:10:29.288276+0000","last_became_active":"2024-04-25T10:10:28.307143+0000","last_became_peered":"2024-04-25T10:10:28.307143+0000","last_unstale":"2024-04-25T10:10:29.288276+0000","last_undegraded":"2024-04-25T10:10:29.288276+0000","last_fullsized":"2024-04-25T10:10:29.288276+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:07.272079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284527+0000","last_change":"2024-04-25T10:10:28.328591+0000","last_active":"2024-04-25T10:10:29.284527+0000","last_peered":"2024-04-25T10:10:29.284527+0000","last_clean":"2024-04-25T10:10:29.284527+0000","last_became_active":"2024-04-25T10:10:28.328245+0000","last_became_peered":"2024-04-25T10:10:28.328245+0000","last_unstale":"2024-04-25T10:10:29.284527+0000","last_undegraded":"2024-04-25T10:10:29.284527+0000","last_fullsized":"2024-04-25T10:10:29.284527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:37.658716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.323608+0000","last_change":"2024-04-25T10:10:28.319783+0000","last_active":"2024-04-25T10:10:28.323608+0000","last_peered":"2024-04-25T10:10:28.323608+0000","last_clean":"2024-04-25T10:10:28.323608+0000","last_became_active":"2024-04-25T10:10:28.319199+0000","last_became_peered":"2024-04-25T10:10:28.319199+0000","last_unstale":"2024-04-25T10:10:28.323608+0000","last_undegraded":"2024-04-25T10:10:28.323608+0000","last_fullsized":"2024-04-25T10:10:28.323608+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:10.461603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.335048+0000","last_change":"2024-04-25T10:10:28.329253+0000","last_active":"2024-04-25T10:10:28.335048+0000","last_peered":"2024-04-25T10:10:28.335048+0000","last_clean":"2024-04-25T10:10:28.335048+0000","last_became_active":"2024-04-25T10:10:28.328915+0000","last_became_peered":"2024-04-25T10:10:28.328915+0000","last_unstale":"2024-04-25T10:10:28.335048+0000","last_undegraded":"2024-04-25T10:10:28.335048+0000","last_fullsized":"2024-04-25T10:10:28.335048+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:55.829641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.289482+0000","last_change":"2024-04-25T10:10:28.340916+0000","last_active":"2024-04-25T10:10:29.289482+0000","last_peered":"2024-04-25T10:10:29.289482+0000","last_clean":"2024-04-25T10:10:29.289482+0000","last_became_active":"2024-04-25T10:10:28.340409+0000","last_became_peered":"2024-04-25T10:10:28.340409+0000","last_unstale":"2024-04-25T10:10:29.289482+0000","last_undegraded":"2024-04-25T10:10:29.289482+0000","last_fullsized":"2024-04-25T10:10:29.289482+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.538305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.324131+0000","last_change":"2024-04-25T10:10:28.322769+0000","last_active":"2024-04-25T10:10:28.324131+0000","last_peered":"2024-04-25T10:10:28.324131+0000","last_clean":"2024-04-25T10:10:28.324131+0000","last_became_active":"2024-04-25T10:10:28.322410+0000","last_became_peered":"2024-04-25T10:10:28.322410+0000","last_unstale":"2024-04-25T10:10:28.324131+0000","last_undegraded":"2024-04-25T10:10:28.324131+0000","last_fullsized":"2024-04-25T10:10:28.324131+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:58.157544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287026+0000","last_change":"2024-04-25T10:10:28.318420+0000","last_active":"2024-04-25T10:10:29.287026+0000","last_peered":"2024-04-25T10:10:29.287026+0000","last_clean":"2024-04-25T10:10:29.287026+0000","last_became_active":"2024-04-25T10:10:28.317122+0000","last_became_peered":"2024-04-25T10:10:28.317122+0000","last_unstale":"2024-04-25T10:10:29.287026+0000","last_undegraded":"2024-04-25T10:10:29.287026+0000","last_fullsized":"2024-04-25T10:10:29.287026+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.887519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.319528+0000","last_change":"2024-04-25T10:10:28.318937+0000","last_active":"2024-04-25T10:10:28.319528+0000","last_peered":"2024-04-25T10:10:28.319528+0000","last_clean":"2024-04-25T10:10:28.319528+0000","last_became_active":"2024-04-25T10:10:28.318584+0000","last_became_peered":"2024-04-25T10:10:28.318584+0000","last_unstale":"2024-04-25T10:10:28.319528+0000","last_undegraded":"2024-04-25T10:10:28.319528+0000","last_fullsized":"2024-04-25T10:10:28.319528+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:55.670419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.331838+0000","last_change":"2024-04-25T10:10:28.330882+0000","last_active":"2024-04-25T10:10:28.331838+0000","last_peered":"2024-04-25T10:10:28.331838+0000","last_clean":"2024-04-25T10:10:28.331838+0000","last_became_active":"2024-04-25T10:10:28.330505+0000","last_became_peered":"2024-04-25T10:10:28.330505+0000","last_unstale":"2024-04-25T10:10:28.331838+0000","last_undegraded":"2024-04-25T10:10:28.331838+0000","last_fullsized":"2024-04-25T10:10:28.331838+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.887519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288027+0000","last_change":"2024-04-25T10:10:28.332568+0000","last_active":"2024-04-25T10:10:29.288027+0000","last_peered":"2024-04-25T10:10:29.288027+0000","last_clean":"2024-04-25T10:10:29.288027+0000","last_became_active":"2024-04-25T10:10:28.330421+0000","last_became_peered":"2024-04-25T10:10:28.330421+0000","last_unstale":"2024-04-25T10:10:29.288027+0000","last_undegraded":"2024-04-25T10:10:29.288027+0000","last_fullsized":"2024-04-25T10:10:29.288027+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:41.922164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.329551+0000","last_change":"2024-04-25T10:10:28.326379+0000","last_active":"2024-04-25T10:10:28.329551+0000","last_peered":"2024-04-25T10:10:28.329551+0000","last_clean":"2024-04-25T10:10:28.329551+0000","last_became_active":"2024-04-25T10:10:28.321718+0000","last_became_peered":"2024-04-25T10:10:28.321718+0000","last_unstale":"2024-04-25T10:10:28.329551+0000","last_undegraded":"2024-04-25T10:10:28.329551+0000","last_fullsized":"2024-04-25T10:10:28.329551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:22.622715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284349+0000","last_change":"2024-04-25T10:10:28.324293+0000","last_active":"2024-04-25T10:10:29.284349+0000","last_peered":"2024-04-25T10:10:29.284349+0000","last_clean":"2024-04-25T10:10:29.284349+0000","last_became_active":"2024-04-25T10:10:28.323975+0000","last_became_peered":"2024-04-25T10:10:28.323975+0000","last_unstale":"2024-04-25T10:10:29.284349+0000","last_undegraded":"2024-04-25T10:10:29.284349+0000","last_fullsized":"2024-04-25T10:10:29.284349+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:32.457132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284993+0000","last_change":"2024-04-25T10:10:28.314354+0000","last_active":"2024-04-25T10:10:29.284993+0000","last_peered":"2024-04-25T10:10:29.284993+0000","last_clean":"2024-04-25T10:10:29.284993+0000","last_became_active":"2024-04-25T10:10:28.313588+0000","last_became_peered":"2024-04-25T10:10:28.313588+0000","last_unstale":"2024-04-25T10:10:29.284993+0000","last_undegraded":"2024-04-25T10:10:29.284993+0000","last_fullsized":"2024-04-25T10:10:29.284993+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:13.894682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.325536+0000","last_change":"2024-04-25T10:10:28.325011+0000","last_active":"2024-04-25T10:10:28.325536+0000","last_peered":"2024-04-25T10:10:28.325536+0000","last_clean":"2024-04-25T10:10:28.325536+0000","last_became_active":"2024-04-25T10:10:28.324727+0000","last_became_peered":"2024-04-25T10:10:28.324727+0000","last_unstale":"2024-04-25T10:10:28.325536+0000","last_undegraded":"2024-04-25T10:10:28.325536+0000","last_fullsized":"2024-04-25T10:10:28.325536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:03.269418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287403+0000","last_change":"2024-04-25T10:10:28.334252+0000","last_active":"2024-04-25T10:10:29.287403+0000","last_peered":"2024-04-25T10:10:29.287403+0000","last_clean":"2024-04-25T10:10:29.287403+0000","last_became_active":"2024-04-25T10:10:28.324155+0000","last_became_peered":"2024-04-25T10:10:28.324155+0000","last_unstale":"2024-04-25T10:10:29.287403+0000","last_undegraded":"2024-04-25T10:10:29.287403+0000","last_fullsized":"2024-04-25T10:10:29.287403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:23.458722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.330758+0000","last_change":"2024-04-25T10:10:28.326913+0000","last_active":"2024-04-25T10:10:28.330758+0000","last_peered":"2024-04-25T10:10:28.330758+0000","last_clean":"2024-04-25T10:10:28.330758+0000","last_became_active":"2024-04-25T10:10:28.325914+0000","last_became_peered":"2024-04-25T10:10:28.325914+0000","last_unstale":"2024-04-25T10:10:28.330758+0000","last_undegraded":"2024-04-25T10:10:28.330758+0000","last_fullsized":"2024-04-25T10:10:28.330758+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:10.066431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287705+0000","last_change":"2024-04-25T10:10:28.302076+0000","last_active":"2024-04-25T10:10:29.287705+0000","last_peered":"2024-04-25T10:10:29.287705+0000","last_clean":"2024-04-25T10:10:29.287705+0000","last_became_active":"2024-04-25T10:10:28.301241+0000","last_became_peered":"2024-04-25T10:10:28.301241+0000","last_unstale":"2024-04-25T10:10:29.287705+0000","last_undegraded":"2024-04-25T10:10:29.287705+0000","last_fullsized":"2024-04-25T10:10:29.287705+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:30.381428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288393+0000","last_change":"2024-04-25T10:10:28.329051+0000","last_active":"2024-04-25T10:10:29.288393+0000","last_peered":"2024-04-25T10:10:29.288393+0000","last_clean":"2024-04-25T10:10:29.288393+0000","last_became_active":"2024-04-25T10:10:28.328161+0000","last_became_peered":"2024-04-25T10:10:28.328161+0000","last_unstale":"2024-04-25T10:10:29.288393+0000","last_undegraded":"2024-04-25T10:10:29.288393+0000","last_fullsized":"2024-04-25T10:10:29.288393+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:08.767292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286336+0000","last_change":"2024-04-25T10:10:28.337875+0000","last_active":"2024-04-25T10:10:29.286336+0000","last_peered":"2024-04-25T10:10:29.286336+0000","last_clean":"2024-04-25T10:10:29.286336+0000","last_became_active":"2024-04-25T10:10:28.332834+0000","last_became_peered":"2024-04-25T10:10:28.332834+0000","last_unstale":"2024-04-25T10:10:29.286336+0000","last_undegraded":"2024-04-25T10:10:29.286336+0000","last_fullsized":"2024-04-25T10:10:29.286336+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:10.066431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.319290+0000","last_change":"2024-04-25T10:10:28.318404+0000","last_active":"2024-04-25T10:10:28.319290+0000","last_peered":"2024-04-25T10:10:28.319290+0000","last_clean":"2024-04-25T10:10:28.319290+0000","last_became_active":"2024-04-25T10:10:28.317075+0000","last_became_peered":"2024-04-25T10:10:28.317075+0000","last_unstale":"2024-04-25T10:10:28.319290+0000","last_undegraded":"2024-04-25T10:10:28.319290+0000","last_fullsized":"2024-04-25T10:10:28.319290+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:03.501555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.339369+0000","last_change":"2024-04-25T10:10:28.327303+0000","last_active":"2024-04-25T10:10:28.339369+0000","last_peered":"2024-04-25T10:10:28.339369+0000","last_clean":"2024-04-25T10:10:28.339369+0000","last_became_active":"2024-04-25T10:10:28.326577+0000","last_became_peered":"2024-04-25T10:10:28.326577+0000","last_unstale":"2024-04-25T10:10:28.339369+0000","last_undegraded":"2024-04-25T10:10:28.339369+0000","last_fullsized":"2024-04-25T10:10:28.339369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:03.269418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288906+0000","last_change":"2024-04-25T10:10:28.323630+0000","last_active":"2024-04-25T10:10:29.288906+0000","last_peered":"2024-04-25T10:10:29.288906+0000","last_clean":"2024-04-25T10:10:29.288906+0000","last_became_active":"2024-04-25T10:10:28.323199+0000","last_became_peered":"2024-04-25T10:10:28.323199+0000","last_unstale":"2024-04-25T10:10:29.288906+0000","last_undegraded":"2024-04-25T10:10:29.288906+0000","last_fullsized":"2024-04-25T10:10:29.288906+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.452679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.308306+0000","last_change":"2024-04-25T10:10:28.303701+0000","last_active":"2024-04-25T10:10:28.308306+0000","last_peered":"2024-04-25T10:10:28.308306+0000","last_clean":"2024-04-25T10:10:28.308306+0000","last_became_active":"2024-04-25T10:10:28.302828+0000","last_became_peered":"2024-04-25T10:10:28.302828+0000","last_unstale":"2024-04-25T10:10:28.308306+0000","last_undegraded":"2024-04-25T10:10:28.308306+0000","last_fullsized":"2024-04-25T10:10:28.308306+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:07.701599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.331414+0000","last_change":"2024-04-25T10:10:28.330437+0000","last_active":"2024-04-25T10:10:28.331414+0000","last_peered":"2024-04-25T10:10:28.331414+0000","last_clean":"2024-04-25T10:10:28.331414+0000","last_became_active":"2024-04-25T10:10:28.326372+0000","last_became_peered":"2024-04-25T10:10:28.326372+0000","last_unstale":"2024-04-25T10:10:28.331414+0000","last_undegraded":"2024-04-25T10:10:28.331414+0000","last_fullsized":"2024-04-25T10:10:28.331414+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.388442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.310873+0000","last_change":"2024-04-25T10:10:28.309026+0000","last_active":"2024-04-25T10:10:28.310873+0000","last_peered":"2024-04-25T10:10:28.310873+0000","last_clean":"2024-04-25T10:10:28.310873+0000","last_became_active":"2024-04-25T10:10:28.306186+0000","last_became_peered":"2024-04-25T10:10:28.306186+0000","last_unstale":"2024-04-25T10:10:28.310873+0000","last_undegraded":"2024-04-25T10:10:28.310873+0000","last_fullsized":"2024-04-25T10:10:28.310873+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:19.434582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288954+0000","last_change":"2024-04-25T10:10:28.315961+0000","last_active":"2024-04-25T10:10:29.288954+0000","last_peered":"2024-04-25T10:10:29.288954+0000","last_clean":"2024-04-25T10:10:29.288954+0000","last_became_active":"2024-04-25T10:10:28.315120+0000","last_became_peered":"2024-04-25T10:10:28.315120+0000","last_unstale":"2024-04-25T10:10:29.288954+0000","last_undegraded":"2024-04-25T10:10:29.288954+0000","last_fullsized":"2024-04-25T10:10:29.288954+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:04.003802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.340065+0000","last_change":"2024-04-25T10:10:28.336815+0000","last_active":"2024-04-25T10:10:28.340065+0000","last_peered":"2024-04-25T10:10:28.340065+0000","last_clean":"2024-04-25T10:10:28.340065+0000","last_became_active":"2024-04-25T10:10:28.336323+0000","last_became_peered":"2024-04-25T10:10:28.336323+0000","last_unstale":"2024-04-25T10:10:28.340065+0000","last_undegraded":"2024-04-25T10:10:28.340065+0000","last_fullsized":"2024-04-25T10:10:28.340065+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:21.713065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284215+0000","last_change":"2024-04-25T10:10:28.322840+0000","last_active":"2024-04-25T10:10:29.284215+0000","last_peered":"2024-04-25T10:10:29.284215+0000","last_clean":"2024-04-25T10:10:29.284215+0000","last_became_active":"2024-04-25T10:10:28.318520+0000","last_became_peered":"2024-04-25T10:10:28.318520+0000","last_unstale":"2024-04-25T10:10:29.284215+0000","last_undegraded":"2024-04-25T10:10:29.284215+0000","last_fullsized":"2024-04-25T10:10:29.284215+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:40.790359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285214+0000","last_change":"2024-04-25T10:10:28.309982+0000","last_active":"2024-04-25T10:10:29.285214+0000","last_peered":"2024-04-25T10:10:29.285214+0000","last_clean":"2024-04-25T10:10:29.285214+0000","last_became_active":"2024-04-25T10:10:28.309228+0000","last_became_peered":"2024-04-25T10:10:28.309228+0000","last_unstale":"2024-04-25T10:10:29.285214+0000","last_undegraded":"2024-04-25T10:10:29.285214+0000","last_fullsized":"2024-04-25T10:10:29.285214+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:55.666895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"93'4","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288000+0000","last_change":"2024-04-25T10:10:27.302793+0000","last_active":"2024-04-25T10:10:29.288000+0000","last_peered":"2024-04-25T10:10:29.288000+0000","last_clean":"2024-04-25T10:10:29.288000+0000","last_became_active":"2024-04-25T10:10:27.292088+0000","last_became_peered":"2024-04-25T10:10:27.292088+0000","last_unstale":"2024-04-25T10:10:29.288000+0000","last_undegraded":"2024-04-25T10:10:29.288000+0000","last_fullsized":"2024-04-25T10:10:29.288000+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:26.154284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285146+0000","last_change":"2024-04-25T10:10:28.300731+0000","last_active":"2024-04-25T10:10:29.285146+0000","last_peered":"2024-04-25T10:10:29.285146+0000","last_clean":"2024-04-25T10:10:29.285146+0000","last_became_active":"2024-04-25T10:10:28.299979+0000","last_became_peered":"2024-04-25T10:10:28.299979+0000","last_unstale":"2024-04-25T10:10:29.285146+0000","last_undegraded":"2024-04-25T10:10:29.285146+0000","last_fullsized":"2024-04-25T10:10:29.285146+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:24.957535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.289019+0000","last_change":"2024-04-25T10:10:27.310672+0000","last_active":"2024-04-25T10:10:29.289019+0000","last_peered":"2024-04-25T10:10:29.289019+0000","last_clean":"2024-04-25T10:10:29.289019+0000","last_became_active":"2024-04-25T10:10:27.309845+0000","last_became_peered":"2024-04-25T10:10:27.309845+0000","last_unstale":"2024-04-25T10:10:29.289019+0000","last_undegraded":"2024-04-25T10:10:29.289019+0000","last_fullsized":"2024-04-25T10:10:29.289019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:42.806982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.325427+0000","last_change":"2024-04-25T10:10:28.322262+0000","last_active":"2024-04-25T10:10:28.325427+0000","last_peered":"2024-04-25T10:10:28.325427+0000","last_clean":"2024-04-25T10:10:28.325427+0000","last_became_active":"2024-04-25T10:10:28.321630+0000","last_became_peered":"2024-04-25T10:10:28.321630+0000","last_unstale":"2024-04-25T10:10:28.325427+0000","last_undegraded":"2024-04-25T10:10:28.325427+0000","last_fullsized":"2024-04-25T10:10:28.325427+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:22.550422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"93'2","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.333025+0000","last_change":"2024-04-25T10:10:27.300974+0000","last_active":"2024-04-25T10:10:28.333025+0000","last_peered":"2024-04-25T10:10:28.333025+0000","last_clean":"2024-04-25T10:10:28.333025+0000","last_became_active":"2024-04-25T10:10:27.300220+0000","last_became_peered":"2024-04-25T10:10:27.300220+0000","last_unstale":"2024-04-25T10:10:28.333025+0000","last_undegraded":"2024-04-25T10:10:28.333025+0000","last_fullsized":"2024-04-25T10:10:28.333025+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:22.801287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285356+0000","last_change":"2024-04-25T10:10:28.303317+0000","last_active":"2024-04-25T10:10:29.285356+0000","last_peered":"2024-04-25T10:10:29.285356+0000","last_clean":"2024-04-25T10:10:29.285356+0000","last_became_active":"2024-04-25T10:10:28.302504+0000","last_became_peered":"2024-04-25T10:10:28.302504+0000","last_unstale":"2024-04-25T10:10:29.285356+0000","last_undegraded":"2024-04-25T10:10:29.285356+0000","last_fullsized":"2024-04-25T10:10:29.285356+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:59.446821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287925+0000","last_change":"2024-04-25T10:10:27.309064+0000","last_active":"2024-04-25T10:10:29.287925+0000","last_peered":"2024-04-25T10:10:29.287925+0000","last_clean":"2024-04-25T10:10:29.287925+0000","last_became_active":"2024-04-25T10:10:27.308427+0000","last_became_peered":"2024-04-25T10:10:27.308427+0000","last_unstale":"2024-04-25T10:10:29.287925+0000","last_undegraded":"2024-04-25T10:10:29.287925+0000","last_fullsized":"2024-04-25T10:10:29.287925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:52.720532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.332021+0000","last_change":"2024-04-25T10:10:28.325994+0000","last_active":"2024-04-25T10:10:28.332021+0000","last_peered":"2024-04-25T10:10:28.332021+0000","last_clean":"2024-04-25T10:10:28.332021+0000","last_became_active":"2024-04-25T10:10:28.321835+0000","last_became_peered":"2024-04-25T10:10:28.321835+0000","last_unstale":"2024-04-25T10:10:28.332021+0000","last_undegraded":"2024-04-25T10:10:28.332021+0000","last_fullsized":"2024-04-25T10:10:28.332021+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:38.129582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288520+0000","last_change":"2024-04-25T10:10:27.310336+0000","last_active":"2024-04-25T10:10:29.288520+0000","last_peered":"2024-04-25T10:10:29.288520+0000","last_clean":"2024-04-25T10:10:29.288520+0000","last_became_active":"2024-04-25T10:10:27.305866+0000","last_became_peered":"2024-04-25T10:10:27.305866+0000","last_unstale":"2024-04-25T10:10:29.288520+0000","last_undegraded":"2024-04-25T10:10:29.288520+0000","last_fullsized":"2024-04-25T10:10:29.288520+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:46.384387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286679+0000","last_change":"2024-04-25T10:10:28.318935+0000","last_active":"2024-04-25T10:10:29.286679+0000","last_peered":"2024-04-25T10:10:29.286679+0000","last_clean":"2024-04-25T10:10:29.286679+0000","last_became_active":"2024-04-25T10:10:28.318261+0000","last_became_peered":"2024-04-25T10:10:28.318261+0000","last_unstale":"2024-04-25T10:10:29.286679+0000","last_undegraded":"2024-04-25T10:10:29.286679+0000","last_fullsized":"2024-04-25T10:10:29.286679+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:55.829641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"93'1","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.332969+0000","last_change":"2024-04-25T10:10:27.303155+0000","last_active":"2024-04-25T10:10:28.332969+0000","last_peered":"2024-04-25T10:10:28.332969+0000","last_clean":"2024-04-25T10:10:28.332969+0000","last_became_active":"2024-04-25T10:10:27.302591+0000","last_became_peered":"2024-04-25T10:10:27.302591+0000","last_unstale":"2024-04-25T10:10:28.332969+0000","last_undegraded":"2024-04-25T10:10:28.332969+0000","last_fullsized":"2024-04-25T10:10:28.332969+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:46.945830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284654+0000","last_change":"2024-04-25T10:10:28.328413+0000","last_active":"2024-04-25T10:10:29.284654+0000","last_peered":"2024-04-25T10:10:29.284654+0000","last_clean":"2024-04-25T10:10:29.284654+0000","last_became_active":"2024-04-25T10:10:28.327907+0000","last_became_peered":"2024-04-25T10:10:28.327907+0000","last_unstale":"2024-04-25T10:10:29.284654+0000","last_undegraded":"2024-04-25T10:10:29.284654+0000","last_fullsized":"2024-04-25T10:10:29.284654+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:10.451849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.296227+0000","last_change":"2024-04-25T10:10:27.301369+0000","last_active":"2024-04-25T10:10:28.296227+0000","last_peered":"2024-04-25T10:10:28.296227+0000","last_clean":"2024-04-25T10:10:28.296227+0000","last_became_active":"2024-04-25T10:10:27.300986+0000","last_became_peered":"2024-04-25T10:10:27.300986+0000","last_unstale":"2024-04-25T10:10:28.296227+0000","last_undegraded":"2024-04-25T10:10:28.296227+0000","last_fullsized":"2024-04-25T10:10:28.296227+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:32.526980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.332251+0000","last_change":"2024-04-25T10:10:28.331466+0000","last_active":"2024-04-25T10:10:28.332251+0000","last_peered":"2024-04-25T10:10:28.332251+0000","last_clean":"2024-04-25T10:10:28.332251+0000","last_became_active":"2024-04-25T10:10:28.331170+0000","last_became_peered":"2024-04-25T10:10:28.331170+0000","last_unstale":"2024-04-25T10:10:28.332251+0000","last_undegraded":"2024-04-25T10:10:28.332251+0000","last_fullsized":"2024-04-25T10:10:28.332251+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:37.342513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.295683+0000","last_change":"2024-04-25T10:10:27.308920+0000","last_active":"2024-04-25T10:10:28.295683+0000","last_peered":"2024-04-25T10:10:28.295683+0000","last_clean":"2024-04-25T10:10:28.295683+0000","last_became_active":"2024-04-25T10:10:27.307228+0000","last_became_peered":"2024-04-25T10:10:27.307228+0000","last_unstale":"2024-04-25T10:10:28.295683+0000","last_undegraded":"2024-04-25T10:10:28.295683+0000","last_fullsized":"2024-04-25T10:10:28.295683+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:39.250479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287269+0000","last_change":"2024-04-25T10:10:28.319163+0000","last_active":"2024-04-25T10:10:29.287269+0000","last_peered":"2024-04-25T10:10:29.287269+0000","last_clean":"2024-04-25T10:10:29.287269+0000","last_became_active":"2024-04-25T10:10:28.318584+0000","last_became_peered":"2024-04-25T10:10:28.318584+0000","last_unstale":"2024-04-25T10:10:29.287269+0000","last_undegraded":"2024-04-25T10:10:29.287269+0000","last_fullsized":"2024-04-25T10:10:29.287269+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:37.342513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.324642+0000","last_change":"2024-04-25T10:10:27.298886+0000","last_active":"2024-04-25T10:10:28.324642+0000","last_peered":"2024-04-25T10:10:28.324642+0000","last_clean":"2024-04-25T10:10:28.324642+0000","last_became_active":"2024-04-25T10:10:27.298189+0000","last_became_peered":"2024-04-25T10:10:27.298189+0000","last_unstale":"2024-04-25T10:10:28.324642+0000","last_undegraded":"2024-04-25T10:10:28.324642+0000","last_fullsized":"2024-04-25T10:10:28.324642+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:35.791825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.307481+0000","last_change":"2024-04-25T10:10:28.306127+0000","last_active":"2024-04-25T10:10:28.307481+0000","last_peered":"2024-04-25T10:10:28.307481+0000","last_clean":"2024-04-25T10:10:28.307481+0000","last_became_active":"2024-04-25T10:10:28.305041+0000","last_became_peered":"2024-04-25T10:10:28.305041+0000","last_unstale":"2024-04-25T10:10:28.307481+0000","last_undegraded":"2024-04-25T10:10:28.307481+0000","last_fullsized":"2024-04-25T10:10:28.307481+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:56.380015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.298225+0000","last_change":"2024-04-25T10:10:27.294212+0000","last_active":"2024-04-25T10:10:28.298225+0000","last_peered":"2024-04-25T10:10:28.298225+0000","last_clean":"2024-04-25T10:10:28.298225+0000","last_became_active":"2024-04-25T10:10:27.293226+0000","last_became_peered":"2024-04-25T10:10:27.293226+0000","last_unstale":"2024-04-25T10:10:28.298225+0000","last_undegraded":"2024-04-25T10:10:28.298225+0000","last_fullsized":"2024-04-25T10:10:28.298225+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:42.903578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.318868+0000","last_change":"2024-04-25T10:10:28.317659+0000","last_active":"2024-04-25T10:10:28.318868+0000","last_peered":"2024-04-25T10:10:28.318868+0000","last_clean":"2024-04-25T10:10:28.318868+0000","last_became_active":"2024-04-25T10:10:28.317388+0000","last_became_peered":"2024-04-25T10:10:28.317388+0000","last_unstale":"2024-04-25T10:10:28.318868+0000","last_undegraded":"2024-04-25T10:10:28.318868+0000","last_fullsized":"2024-04-25T10:10:28.318868+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:05.260306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"93'3","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287867+0000","last_change":"2024-04-25T10:10:27.322442+0000","last_active":"2024-04-25T10:10:29.287867+0000","last_peered":"2024-04-25T10:10:29.287867+0000","last_clean":"2024-04-25T10:10:29.287867+0000","last_became_active":"2024-04-25T10:10:27.318243+0000","last_became_peered":"2024-04-25T10:10:27.318243+0000","last_unstale":"2024-04-25T10:10:29.287867+0000","last_undegraded":"2024-04-25T10:10:29.287867+0000","last_fullsized":"2024-04-25T10:10:29.287867+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:57.976245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.341149+0000","last_change":"2024-04-25T10:10:28.340588+0000","last_active":"2024-04-25T10:10:28.341149+0000","last_peered":"2024-04-25T10:10:28.341149+0000","last_clean":"2024-04-25T10:10:28.341149+0000","last_became_active":"2024-04-25T10:10:28.340276+0000","last_became_peered":"2024-04-25T10:10:28.340276+0000","last_unstale":"2024-04-25T10:10:28.341149+0000","last_undegraded":"2024-04-25T10:10:28.341149+0000","last_fullsized":"2024-04-25T10:10:28.341149+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:23.458722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"93'6","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285806+0000","last_change":"2024-04-25T10:10:27.301510+0000","last_active":"2024-04-25T10:10:29.285806+0000","last_peered":"2024-04-25T10:10:29.285806+0000","last_clean":"2024-04-25T10:10:29.285806+0000","last_became_active":"2024-04-25T10:10:27.301023+0000","last_became_peered":"2024-04-25T10:10:27.301023+0000","last_unstale":"2024-04-25T10:10:29.285806+0000","last_undegraded":"2024-04-25T10:10:29.285806+0000","last_fullsized":"2024-04-25T10:10:29.285806+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:43.893353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.326155+0000","last_change":"2024-04-25T10:10:28.315268+0000","last_active":"2024-04-25T10:10:28.326155+0000","last_peered":"2024-04-25T10:10:28.326155+0000","last_clean":"2024-04-25T10:10:28.326155+0000","last_became_active":"2024-04-25T10:10:28.314749+0000","last_became_peered":"2024-04-25T10:10:28.314749+0000","last_unstale":"2024-04-25T10:10:28.326155+0000","last_undegraded":"2024-04-25T10:10:28.326155+0000","last_fullsized":"2024-04-25T10:10:28.326155+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:49.937031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"93'2","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.335097+0000","last_change":"2024-04-25T10:10:27.308575+0000","last_active":"2024-04-25T10:10:28.335097+0000","last_peered":"2024-04-25T10:10:28.335097+0000","last_clean":"2024-04-25T10:10:28.335097+0000","last_became_active":"2024-04-25T10:10:27.307340+0000","last_became_peered":"2024-04-25T10:10:27.307340+0000","last_unstale":"2024-04-25T10:10:28.335097+0000","last_undegraded":"2024-04-25T10:10:28.335097+0000","last_fullsized":"2024-04-25T10:10:28.335097+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:54.237235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286456+0000","last_change":"2024-04-25T10:10:28.311400+0000","last_active":"2024-04-25T10:10:29.286456+0000","last_peered":"2024-04-25T10:10:29.286456+0000","last_clean":"2024-04-25T10:10:29.286456+0000","last_became_active":"2024-04-25T10:10:28.310446+0000","last_became_peered":"2024-04-25T10:10:28.310446+0000","last_unstale":"2024-04-25T10:10:29.286456+0000","last_undegraded":"2024-04-25T10:10:29.286456+0000","last_fullsized":"2024-04-25T10:10:29.286456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:08.050216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284629+0000","last_change":"2024-04-25T10:10:27.317031+0000","last_active":"2024-04-25T10:10:29.284629+0000","last_peered":"2024-04-25T10:10:29.284629+0000","last_clean":"2024-04-25T10:10:29.284629+0000","last_became_active":"2024-04-25T10:10:27.316674+0000","last_became_peered":"2024-04-25T10:10:27.316674+0000","last_unstale":"2024-04-25T10:10:29.284629+0000","last_undegraded":"2024-04-25T10:10:29.284629+0000","last_fullsized":"2024-04-25T10:10:29.284629+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:43.373222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.332493+0000","last_change":"2024-04-25T10:10:28.331920+0000","last_active":"2024-04-25T10:10:28.332493+0000","last_peered":"2024-04-25T10:10:28.332493+0000","last_clean":"2024-04-25T10:10:28.332493+0000","last_became_active":"2024-04-25T10:10:28.331493+0000","last_became_peered":"2024-04-25T10:10:28.331493+0000","last_unstale":"2024-04-25T10:10:28.332493+0000","last_undegraded":"2024-04-25T10:10:28.332493+0000","last_fullsized":"2024-04-25T10:10:28.332493+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:09.095479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.305461+0000","last_change":"2024-04-25T10:10:27.304114+0000","last_active":"2024-04-25T10:10:28.305461+0000","last_peered":"2024-04-25T10:10:28.305461+0000","last_clean":"2024-04-25T10:10:28.305461+0000","last_became_active":"2024-04-25T10:10:27.303799+0000","last_became_peered":"2024-04-25T10:10:27.303799+0000","last_unstale":"2024-04-25T10:10:28.305461+0000","last_undegraded":"2024-04-25T10:10:28.305461+0000","last_fullsized":"2024-04-25T10:10:28.305461+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:47.197897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.308633+0000","last_change":"2024-04-25T10:10:28.306623+0000","last_active":"2024-04-25T10:10:28.308633+0000","last_peered":"2024-04-25T10:10:28.308633+0000","last_clean":"2024-04-25T10:10:28.308633+0000","last_became_active":"2024-04-25T10:10:28.305762+0000","last_became_peered":"2024-04-25T10:10:28.305762+0000","last_unstale":"2024-04-25T10:10:28.308633+0000","last_undegraded":"2024-04-25T10:10:28.308633+0000","last_fullsized":"2024-04-25T10:10:28.308633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:41.891188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"93'8","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.336083+0000","last_change":"2024-04-25T10:10:27.301949+0000","last_active":"2024-04-25T10:10:28.336083+0000","last_peered":"2024-04-25T10:10:28.336083+0000","last_clean":"2024-04-25T10:10:28.336083+0000","last_became_active":"2024-04-25T10:10:27.295986+0000","last_became_peered":"2024-04-25T10:10:27.295986+0000","last_unstale":"2024-04-25T10:10:28.336083+0000","last_undegraded":"2024-04-25T10:10:28.336083+0000","last_fullsized":"2024-04-25T10:10:28.336083+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:03.630684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285558+0000","last_change":"2024-04-25T10:10:28.310548+0000","last_active":"2024-04-25T10:10:29.285558+0000","last_peered":"2024-04-25T10:10:29.285558+0000","last_clean":"2024-04-25T10:10:29.285558+0000","last_became_active":"2024-04-25T10:10:28.309715+0000","last_became_peered":"2024-04-25T10:10:28.309715+0000","last_unstale":"2024-04-25T10:10:29.285558+0000","last_undegraded":"2024-04-25T10:10:29.285558+0000","last_fullsized":"2024-04-25T10:10:29.285558+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:14.815549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288462+0000","last_change":"2024-04-25T10:10:27.308552+0000","last_active":"2024-04-25T10:10:29.288462+0000","last_peered":"2024-04-25T10:10:29.288462+0000","last_clean":"2024-04-25T10:10:29.288462+0000","last_became_active":"2024-04-25T10:10:27.302875+0000","last_became_peered":"2024-04-25T10:10:27.302875+0000","last_unstale":"2024-04-25T10:10:29.288462+0000","last_undegraded":"2024-04-25T10:10:29.288462+0000","last_fullsized":"2024-04-25T10:10:29.288462+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:47.197897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286773+0000","last_change":"2024-04-25T10:10:28.332134+0000","last_active":"2024-04-25T10:10:29.286773+0000","last_peered":"2024-04-25T10:10:29.286773+0000","last_clean":"2024-04-25T10:10:29.286773+0000","last_became_active":"2024-04-25T10:10:28.331302+0000","last_became_peered":"2024-04-25T10:10:28.331302+0000","last_unstale":"2024-04-25T10:10:29.286773+0000","last_undegraded":"2024-04-25T10:10:29.286773+0000","last_fullsized":"2024-04-25T10:10:29.286773+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.388442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"93'2","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.341562+0000","last_change":"2024-04-25T10:10:27.310332+0000","last_active":"2024-04-25T10:10:28.341562+0000","last_peered":"2024-04-25T10:10:28.341562+0000","last_clean":"2024-04-25T10:10:28.341562+0000","last_became_active":"2024-04-25T10:10:27.309291+0000","last_became_peered":"2024-04-25T10:10:27.309291+0000","last_unstale":"2024-04-25T10:10:28.341562+0000","last_undegraded":"2024-04-25T10:10:28.341562+0000","last_fullsized":"2024-04-25T10:10:28.341562+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:31.783160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288867+0000","last_change":"2024-04-25T10:10:28.308072+0000","last_active":"2024-04-25T10:10:29.288867+0000","last_peered":"2024-04-25T10:10:29.288867+0000","last_clean":"2024-04-25T10:10:29.288867+0000","last_became_active":"2024-04-25T10:10:28.307187+0000","last_became_peered":"2024-04-25T10:10:28.307187+0000","last_unstale":"2024-04-25T10:10:29.288867+0000","last_undegraded":"2024-04-25T10:10:29.288867+0000","last_fullsized":"2024-04-25T10:10:29.288867+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:28.331746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288139+0000","last_change":"2024-04-25T10:10:27.315877+0000","last_active":"2024-04-25T10:10:29.288139+0000","last_peered":"2024-04-25T10:10:29.288139+0000","last_clean":"2024-04-25T10:10:29.288139+0000","last_became_active":"2024-04-25T10:10:27.312147+0000","last_became_peered":"2024-04-25T10:10:27.312147+0000","last_unstale":"2024-04-25T10:10:29.288139+0000","last_undegraded":"2024-04-25T10:10:29.288139+0000","last_fullsized":"2024-04-25T10:10:29.288139+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:19.142471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284382+0000","last_change":"2024-04-25T10:10:28.328730+0000","last_active":"2024-04-25T10:10:29.284382+0000","last_peered":"2024-04-25T10:10:29.284382+0000","last_clean":"2024-04-25T10:10:29.284382+0000","last_became_active":"2024-04-25T10:10:28.328242+0000","last_became_peered":"2024-04-25T10:10:28.328242+0000","last_unstale":"2024-04-25T10:10:29.284382+0000","last_undegraded":"2024-04-25T10:10:29.284382+0000","last_fullsized":"2024-04-25T10:10:29.284382+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:39.416124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"93'2","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.339304+0000","last_change":"2024-04-25T10:10:27.300887+0000","last_active":"2024-04-25T10:10:28.339304+0000","last_peered":"2024-04-25T10:10:28.339304+0000","last_clean":"2024-04-25T10:10:28.339304+0000","last_became_active":"2024-04-25T10:10:27.300023+0000","last_became_peered":"2024-04-25T10:10:27.300023+0000","last_unstale":"2024-04-25T10:10:28.339304+0000","last_undegraded":"2024-04-25T10:10:28.339304+0000","last_fullsized":"2024-04-25T10:10:28.339304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:42.903578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.340034+0000","last_change":"2024-04-25T10:10:28.334998+0000","last_active":"2024-04-25T10:10:28.340034+0000","last_peered":"2024-04-25T10:10:28.340034+0000","last_clean":"2024-04-25T10:10:28.340034+0000","last_became_active":"2024-04-25T10:10:28.325267+0000","last_became_peered":"2024-04-25T10:10:28.325267+0000","last_unstale":"2024-04-25T10:10:28.340034+0000","last_undegraded":"2024-04-25T10:10:28.340034+0000","last_fullsized":"2024-04-25T10:10:28.340034+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:09.037782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.289366+0000","last_change":"2024-04-25T10:10:27.305009+0000","last_active":"2024-04-25T10:10:28.289366+0000","last_peered":"2024-04-25T10:10:28.289366+0000","last_clean":"2024-04-25T10:10:28.289366+0000","last_became_active":"2024-04-25T10:10:27.304643+0000","last_became_peered":"2024-04-25T10:10:27.304643+0000","last_unstale":"2024-04-25T10:10:28.289366+0000","last_undegraded":"2024-04-25T10:10:28.289366+0000","last_fullsized":"2024-04-25T10:10:28.289366+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:51.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.340474+0000","last_change":"2024-04-25T10:10:28.336366+0000","last_active":"2024-04-25T10:10:28.340474+0000","last_peered":"2024-04-25T10:10:28.340474+0000","last_clean":"2024-04-25T10:10:28.340474+0000","last_became_active":"2024-04-25T10:10:28.335961+0000","last_became_peered":"2024-04-25T10:10:28.335961+0000","last_unstale":"2024-04-25T10:10:28.340474+0000","last_undegraded":"2024-04-25T10:10:28.340474+0000","last_fullsized":"2024-04-25T10:10:28.340474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:24.213072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.313566+0000","last_change":"2024-04-25T10:10:27.299089+0000","last_active":"2024-04-25T10:10:28.313566+0000","last_peered":"2024-04-25T10:10:28.313566+0000","last_clean":"2024-04-25T10:10:28.313566+0000","last_became_active":"2024-04-25T10:10:27.297943+0000","last_became_peered":"2024-04-25T10:10:27.297943+0000","last_unstale":"2024-04-25T10:10:28.313566+0000","last_undegraded":"2024-04-25T10:10:28.313566+0000","last_fullsized":"2024-04-25T10:10:28.313566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.248981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284918+0000","last_change":"2024-04-25T10:10:28.298794+0000","last_active":"2024-04-25T10:10:29.284918+0000","last_peered":"2024-04-25T10:10:29.284918+0000","last_clean":"2024-04-25T10:10:29.284918+0000","last_became_active":"2024-04-25T10:10:28.297973+0000","last_became_peered":"2024-04-25T10:10:28.297973+0000","last_unstale":"2024-04-25T10:10:29.284918+0000","last_undegraded":"2024-04-25T10:10:29.284918+0000","last_fullsized":"2024-04-25T10:10:29.284918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:48.953685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.285916+0000","last_change":"2024-04-25T10:10:27.306022+0000","last_active":"2024-04-25T10:10:28.285916+0000","last_peered":"2024-04-25T10:10:28.285916+0000","last_clean":"2024-04-25T10:10:28.285916+0000","last_became_active":"2024-04-25T10:10:27.305657+0000","last_became_peered":"2024-04-25T10:10:27.305657+0000","last_unstale":"2024-04-25T10:10:28.285916+0000","last_undegraded":"2024-04-25T10:10:28.285916+0000","last_fullsized":"2024-04-25T10:10:28.285916+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:35.612043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287651+0000","last_change":"2024-04-25T10:10:28.310030+0000","last_active":"2024-04-25T10:10:29.287651+0000","last_peered":"2024-04-25T10:10:29.287651+0000","last_clean":"2024-04-25T10:10:29.287651+0000","last_became_active":"2024-04-25T10:10:28.307706+0000","last_became_peered":"2024-04-25T10:10:28.307706+0000","last_unstale":"2024-04-25T10:10:29.287651+0000","last_undegraded":"2024-04-25T10:10:29.287651+0000","last_fullsized":"2024-04-25T10:10:29.287651+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.887519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"93'2","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.343645+0000","last_change":"2024-04-25T10:10:27.317000+0000","last_active":"2024-04-25T10:10:28.343645+0000","last_peered":"2024-04-25T10:10:28.343645+0000","last_clean":"2024-04-25T10:10:28.343645+0000","last_became_active":"2024-04-25T10:10:27.316258+0000","last_became_peered":"2024-04-25T10:10:27.316258+0000","last_unstale":"2024-04-25T10:10:28.343645+0000","last_undegraded":"2024-04-25T10:10:28.343645+0000","last_fullsized":"2024-04-25T10:10:28.343645+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:59.373353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285440+0000","last_change":"2024-04-25T10:10:28.314012+0000","last_active":"2024-04-25T10:10:29.285440+0000","last_peered":"2024-04-25T10:10:29.285440+0000","last_clean":"2024-04-25T10:10:29.285440+0000","last_became_active":"2024-04-25T10:10:28.312690+0000","last_became_peered":"2024-04-25T10:10:28.312690+0000","last_unstale":"2024-04-25T10:10:29.285440+0000","last_undegraded":"2024-04-25T10:10:29.285440+0000","last_fullsized":"2024-04-25T10:10:29.285440+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:21.905038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.289059+0000","last_change":"2024-04-25T10:10:27.317293+0000","last_active":"2024-04-25T10:10:29.289059+0000","last_peered":"2024-04-25T10:10:29.289059+0000","last_clean":"2024-04-25T10:10:29.289059+0000","last_became_active":"2024-04-25T10:10:27.315272+0000","last_became_peered":"2024-04-25T10:10:27.315272+0000","last_unstale":"2024-04-25T10:10:29.289059+0000","last_undegraded":"2024-04-25T10:10:29.289059+0000","last_fullsized":"2024-04-25T10:10:29.289059+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:52.720532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286139+0000","last_change":"2024-04-25T10:10:28.299773+0000","last_active":"2024-04-25T10:10:29.286139+0000","last_peered":"2024-04-25T10:10:29.286139+0000","last_clean":"2024-04-25T10:10:29.286139+0000","last_became_active":"2024-04-25T10:10:28.298976+0000","last_became_peered":"2024-04-25T10:10:28.298976+0000","last_unstale":"2024-04-25T10:10:29.286139+0000","last_undegraded":"2024-04-25T10:10:29.286139+0000","last_fullsized":"2024-04-25T10:10:29.286139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:16.565779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288376+0000","last_change":"2024-04-25T10:10:27.321974+0000","last_active":"2024-04-25T10:10:29.288376+0000","last_peered":"2024-04-25T10:10:29.288376+0000","last_clean":"2024-04-25T10:10:29.288376+0000","last_became_active":"2024-04-25T10:10:27.317317+0000","last_became_peered":"2024-04-25T10:10:27.317317+0000","last_unstale":"2024-04-25T10:10:29.288376+0000","last_undegraded":"2024-04-25T10:10:29.288376+0000","last_fullsized":"2024-04-25T10:10:29.288376+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:05.621873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.339122+0000","last_change":"2024-04-25T10:10:28.324947+0000","last_active":"2024-04-25T10:10:28.339122+0000","last_peered":"2024-04-25T10:10:28.339122+0000","last_clean":"2024-04-25T10:10:28.339122+0000","last_became_active":"2024-04-25T10:10:28.324225+0000","last_became_peered":"2024-04-25T10:10:28.324225+0000","last_unstale":"2024-04-25T10:10:28.339122+0000","last_undegraded":"2024-04-25T10:10:28.339122+0000","last_fullsized":"2024-04-25T10:10:28.339122+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:03.501555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288188+0000","last_change":"2024-04-25T10:10:27.308732+0000","last_active":"2024-04-25T10:10:29.288188+0000","last_peered":"2024-04-25T10:10:29.288188+0000","last_clean":"2024-04-25T10:10:29.288188+0000","last_became_active":"2024-04-25T10:10:27.307850+0000","last_became_peered":"2024-04-25T10:10:27.307850+0000","last_unstale":"2024-04-25T10:10:29.288188+0000","last_undegraded":"2024-04-25T10:10:29.288188+0000","last_fullsized":"2024-04-25T10:10:29.288188+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:31.790492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.301403+0000","last_change":"2024-04-25T10:10:28.299895+0000","last_active":"2024-04-25T10:10:28.301403+0000","last_peered":"2024-04-25T10:10:28.301403+0000","last_clean":"2024-04-25T10:10:28.301403+0000","last_became_active":"2024-04-25T10:10:28.299188+0000","last_became_peered":"2024-04-25T10:10:28.299188+0000","last_unstale":"2024-04-25T10:10:28.301403+0000","last_undegraded":"2024-04-25T10:10:28.301403+0000","last_fullsized":"2024-04-25T10:10:28.301403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:44.690270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285027+0000","last_change":"2024-04-25T10:10:27.319853+0000","last_active":"2024-04-25T10:10:29.285027+0000","last_peered":"2024-04-25T10:10:29.285027+0000","last_clean":"2024-04-25T10:10:29.285027+0000","last_became_active":"2024-04-25T10:10:27.319366+0000","last_became_peered":"2024-04-25T10:10:27.319366+0000","last_unstale":"2024-04-25T10:10:29.285027+0000","last_undegraded":"2024-04-25T10:10:29.285027+0000","last_fullsized":"2024-04-25T10:10:29.285027+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:31.783160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.324913+0000","last_change":"2024-04-25T10:10:28.323420+0000","last_active":"2024-04-25T10:10:28.324913+0000","last_peered":"2024-04-25T10:10:28.324913+0000","last_clean":"2024-04-25T10:10:28.324913+0000","last_became_active":"2024-04-25T10:10:28.323035+0000","last_became_peered":"2024-04-25T10:10:28.323035+0000","last_unstale":"2024-04-25T10:10:28.324913+0000","last_undegraded":"2024-04-25T10:10:28.324913+0000","last_fullsized":"2024-04-25T10:10:28.324913+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:21.813044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.312659+0000","last_change":"2024-04-25T10:10:27.298885+0000","last_active":"2024-04-25T10:10:28.312659+0000","last_peered":"2024-04-25T10:10:28.312659+0000","last_clean":"2024-04-25T10:10:28.312659+0000","last_became_active":"2024-04-25T10:10:27.298090+0000","last_became_peered":"2024-04-25T10:10:27.298090+0000","last_unstale":"2024-04-25T10:10:28.312659+0000","last_undegraded":"2024-04-25T10:10:28.312659+0000","last_fullsized":"2024-04-25T10:10:28.312659+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:23.122601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286951+0000","last_change":"2024-04-25T10:10:28.322727+0000","last_active":"2024-04-25T10:10:29.286951+0000","last_peered":"2024-04-25T10:10:29.286951+0000","last_clean":"2024-04-25T10:10:29.286951+0000","last_became_active":"2024-04-25T10:10:28.322072+0000","last_became_peered":"2024-04-25T10:10:28.322072+0000","last_unstale":"2024-04-25T10:10:29.286951+0000","last_undegraded":"2024-04-25T10:10:29.286951+0000","last_fullsized":"2024-04-25T10:10:29.286951+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:08.050216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287621+0000","last_change":"2024-04-25T10:10:27.301219+0000","last_active":"2024-04-25T10:10:29.287621+0000","last_peered":"2024-04-25T10:10:29.287621+0000","last_clean":"2024-04-25T10:10:29.287621+0000","last_became_active":"2024-04-25T10:10:27.300499+0000","last_became_peered":"2024-04-25T10:10:27.300499+0000","last_unstale":"2024-04-25T10:10:29.287621+0000","last_undegraded":"2024-04-25T10:10:29.287621+0000","last_fullsized":"2024-04-25T10:10:29.287621+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:03.630684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285058+0000","last_change":"2024-04-25T10:10:28.313177+0000","last_active":"2024-04-25T10:10:29.285058+0000","last_peered":"2024-04-25T10:10:29.285058+0000","last_clean":"2024-04-25T10:10:29.285058+0000","last_became_active":"2024-04-25T10:10:28.312379+0000","last_became_peered":"2024-04-25T10:10:28.312379+0000","last_unstale":"2024-04-25T10:10:29.285058+0000","last_undegraded":"2024-04-25T10:10:29.285058+0000","last_fullsized":"2024-04-25T10:10:29.285058+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:04.999305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.296036+0000","last_change":"2024-04-25T10:10:27.309578+0000","last_active":"2024-04-25T10:10:28.296036+0000","last_peered":"2024-04-25T10:10:28.296036+0000","last_clean":"2024-04-25T10:10:28.296036+0000","last_became_active":"2024-04-25T10:10:27.308199+0000","last_became_peered":"2024-04-25T10:10:27.308199+0000","last_unstale":"2024-04-25T10:10:28.296036+0000","last_undegraded":"2024-04-25T10:10:28.296036+0000","last_fullsized":"2024-04-25T10:10:28.296036+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:14.584043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.321516+0000","last_change":"2024-04-25T10:10:28.320107+0000","last_active":"2024-04-25T10:10:28.321516+0000","last_peered":"2024-04-25T10:10:28.321516+0000","last_clean":"2024-04-25T10:10:28.321516+0000","last_became_active":"2024-04-25T10:10:28.319437+0000","last_became_peered":"2024-04-25T10:10:28.319437+0000","last_unstale":"2024-04-25T10:10:28.321516+0000","last_undegraded":"2024-04-25T10:10:28.321516+0000","last_fullsized":"2024-04-25T10:10:28.321516+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:52.573149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.320438+0000","last_change":"2024-04-25T10:10:27.307639+0000","last_active":"2024-04-25T10:10:28.320438+0000","last_peered":"2024-04-25T10:10:28.320438+0000","last_clean":"2024-04-25T10:10:28.320438+0000","last_became_active":"2024-04-25T10:10:27.307078+0000","last_became_peered":"2024-04-25T10:10:27.307078+0000","last_unstale":"2024-04-25T10:10:28.320438+0000","last_undegraded":"2024-04-25T10:10:28.320438+0000","last_fullsized":"2024-04-25T10:10:28.320438+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.634506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.296170+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:10:28.296170+0000","last_peered":"2024-04-25T10:10:28.296170+0000","last_clean":"2024-04-25T10:10:28.296170+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:10:28.296170+0000","last_undegraded":"2024-04-25T10:10:28.296170+0000","last_fullsized":"2024-04-25T10:10:28.296170+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64828,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307012,"statfs":{"total":96636764160,"available":96570380288,"internally_reserved":0,"allocated":1748992,"data_stored":1120858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574959,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58696,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313144,"statfs":{"total":96636764160,"available":96576659456,"internally_reserved":0,"allocated":1167360,"data_stored":532650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54508,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317332,"statfs":{"total":96636764160,"available":96580947968,"internally_reserved":0,"allocated":1146880,"data_stored":530434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68420,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303420,"statfs":{"total":96636764160,"available":96566702080,"internally_reserved":0,"allocated":1167360,"data_stored":537286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574958,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68400,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303440,"statfs":{"total":96636764160,"available":96566722560,"internally_reserved":0,"allocated":1146880,"data_stored":530434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58676,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313164,"statfs":{"total":96636764160,"available":96576679936,"internally_reserved":0,"allocated":1146880,"data_stored":530434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54516,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317324,"statfs":{"total":96636764160,"available":96580939776,"internally_reserved":0,"allocated":1150976,"data_stored":530504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574959,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64860,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306980,"statfs":{"total":96636764160,"available":96570347520,"internally_reserved":0,"allocated":1777664,"data_stored":1129812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:10:32.322 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:10:32.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.470+0000 7fa713462640 1 Processor -- start 2024-04-25T10:10:32.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa713462640 1 -- start start 2024-04-25T10:10:32.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa713462640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa70c15d9b0 0x7fa70c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa713462640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa70c0cee80 0x7fa70c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa713462640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c0ca9b0 con 0x7fa70c15dd90 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa713462640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa70c0cbe30 con 0x7fa70c0cee80 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa713462640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0cac90 con 0x7fa70c15d9b0 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa7109fd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa70c15dd90 0x7fa70c0ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40162/0 (socket says 172.21.15.133:40162) 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa7109fd640 1 -- 172.21.15.133:0/2313859015 learned_addr learned my addr 172.21.15.133:0/2313859015 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa7119ff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa70c0cee80 0x7fa70c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa70bfff640 1 -- 172.21.15.133:0/2313859015 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857060038 0 0) 0x7fa70c0ca9b0 con 0x7fa70c15dd90 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa7111fe640 1 --2- 172.21.15.133:0/2313859015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa70c15d9b0 0x7fa70c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa70bfff640 1 -- 172.21.15.133:0/2313859015 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f4003530 con 0x7fa70c15dd90 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa7119ff640 1 -- 172.21.15.133:0/2313859015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa70c15d9b0 msgr2=0x7fa70c0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa7119ff640 1 --2- 172.21.15.133:0/2313859015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa70c15d9b0 0x7fa70c0cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa7119ff640 1 -- 172.21.15.133:0/2313859015 >> v1:172.21.15.133:6789/0 conn(0x7fa70c15dd90 legacy=0x7fa70c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa7119ff640 1 -- 172.21.15.133:0/2313859015 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c0caf70 con 0x7fa70c0cee80 2024-04-25T10:10:32.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa7111fe640 1 --2- 172.21.15.133:0/2313859015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa70c15d9b0 0x7fa70c0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa7119ff640 1 --2- 172.21.15.133:0/2313859015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70c0cee80 0x7fa70c06d6f0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0cbfb0 tx=0x7fa70401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fe935deb000c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa70bfff640 1 -- 172.21.15.133:0/2313859015 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040b1070 con 0x7fa70c0cee80 2024-04-25T10:10:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa70bfff640 1 -- 172.21.15.133:0/2313859015 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7040ac630 con 0x7fa70c0cee80 2024-04-25T10:10:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.474+0000 7fa70bfff640 1 -- 172.21.15.133:0/2313859015 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040a8050 con 0x7fa70c0cee80 2024-04-25T10:10:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 -- 172.21.15.133:0/2313859015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70c0cee80 msgr2=0x7fa70c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 --2- 172.21.15.133:0/2313859015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70c0cee80 0x7fa70c06d6f0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0cbfb0 tx=0x7fa70401c3b0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 -- 172.21.15.133:0/2313859015 shutdown_connections 2024-04-25T10:10:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 --2- 172.21.15.133:0/2313859015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70c0cee80 0x7fa70c06d6f0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 --2- 172.21.15.133:0/2313859015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa70c15d9b0 0x7fa70c0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 -- 172.21.15.133:0/2313859015 >> 172.21.15.133:0/2313859015 conn(0x7fa70c067000 msgr2=0x7fa70c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:32.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 -- 172.21.15.133:0/2313859015 shutdown_connections 2024-04-25T10:10:32.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 -- 172.21.15.133:0/2313859015 wait complete. 2024-04-25T10:10:32.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 Processor -- start 2024-04-25T10:10:32.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 -- start start 2024-04-25T10:10:32.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa70c0cee80 0x7fa70c156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70c15d9b0 0x7fa70c14d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa70c15dd90 0x7fa70c139d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c06e7d0 con 0x7fa70c15d9b0 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c06e650 con 0x7fa70c15dd90 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c06e950 con 0x7fa70c0cee80 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa7109fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70c15d9b0 0x7fa70c14d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa7109fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70c15d9b0 0x7fa70c14d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42762/0 (socket says 172.21.15.133:42762) 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa7109fd640 1 -- 172.21.15.133:0/20025917 learned_addr learned my addr 172.21.15.133:0/20025917 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa7109fd640 1 -- 172.21.15.133:0/20025917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa70c0cee80 msgr2=0x7fa70c156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa7109fd640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa70c0cee80 0x7fa70c156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa7109fd640 1 -- 172.21.15.133:0/20025917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa70c15dd90 msgr2=0x7fa70c139d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa7109fd640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa70c15dd90 0x7fa70c139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa7109fd640 1 -- 172.21.15.133:0/20025917 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c14db30 con 0x7fa70c15d9b0 2024-04-25T10:10:32.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa7109fd640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70c15d9b0 0x7fa70c14d390 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa70000caa0 tx=0x7fa70000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa709ffb640 1 -- 172.21.15.133:0/20025917 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700017070 con 0x7fa70c15d9b0 2024-04-25T10:10:32.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa709ffb640 1 -- 172.21.15.133:0/20025917 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70000ac00 con 0x7fa70c15d9b0 2024-04-25T10:10:32.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 -- 172.21.15.133:0/20025917 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70c14de10 con 0x7fa70c15d9b0 2024-04-25T10:10:32.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa709ffb640 1 -- 172.21.15.133:0/20025917 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700012400 con 0x7fa70c15d9b0 2024-04-25T10:10:32.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.478+0000 7fa713462640 1 -- 172.21.15.133:0/20025917 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70c13a2b0 con 0x7fa70c15d9b0 2024-04-25T10:10:32.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.482+0000 7fa713462640 1 -- 172.21.15.133:0/20025917 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70c068ef0 con 0x7fa70c15d9b0 2024-04-25T10:10:32.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.482+0000 7fa709ffb640 1 -- 172.21.15.133:0/20025917 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa70001a070 con 0x7fa70c15d9b0 2024-04-25T10:10:32.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.482+0000 7fa709ffb640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6f8066f60 0x7fa6f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.482+0000 7fa709ffb640 1 -- 172.21.15.133:0/20025917 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fa700085010 con 0x7fa70c15d9b0 2024-04-25T10:10:32.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.482+0000 7fa7111fe640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6f8066f60 0x7fa6f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.482+0000 7fa7111fe640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6f8066f60 0x7fa6f8069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00a8d0 tx=0x7fa6fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.486+0000 7fa709ffb640 1 -- 172.21.15.133:0/20025917 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa700055e20 con 0x7fa70c15d9b0 2024-04-25T10:10:32.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.626+0000 7fa713462640 1 -- 172.21.15.133:0/20025917 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa70c153c40 con 0x7fa6f8066f60 2024-04-25T10:10:32.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.646+0000 7fa709ffb640 1 -- 172.21.15.133:0/20025917 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252040 (secure 0 0 0) 0x7fa6fc00a4c0 con 0x7fa6f8066f60 2024-04-25T10:10:32.652 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:32.654 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:10:32.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 -- 172.21.15.133:0/20025917 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6f8066f60 msgr2=0x7fa6f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6f8066f60 0x7fa6f8069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00a8d0 tx=0x7fa6fc008040 comp rx=0 tx=0).stop 2024-04-25T10:10:32.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 -- 172.21.15.133:0/20025917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70c15d9b0 msgr2=0x7fa70c14d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70c15d9b0 0x7fa70c14d390 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa70000caa0 tx=0x7fa70000cf60 comp rx=0 tx=0).stop 2024-04-25T10:10:32.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 -- 172.21.15.133:0/20025917 shutdown_connections 2024-04-25T10:10:32.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa70c15dd90 0x7fa70c139d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa6f8066f60 0x7fa6f8069420 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa70c15d9b0 0x7fa70c14d390 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 --2- 172.21.15.133:0/20025917 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa70c0cee80 0x7fa70c156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 -- 172.21.15.133:0/20025917 >> 172.21.15.133:0/20025917 conn(0x7fa70c067000 msgr2=0x7fa70c0c9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 -- 172.21.15.133:0/20025917 shutdown_connections 2024-04-25T10:10:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.650+0000 7fa713462640 1 -- 172.21.15.133:0/20025917 wait complete. 2024-04-25T10:10:32.667 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-04-25T10:10:32.350843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":492904,"kb_used_data":10208,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754481816,"statfs":{"total":773094113280,"available":772589379584,"internally_reserved":0,"allocated":10452992,"data_stored":5442412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.073221"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.319250+0000","last_change":"2024-04-25T10:10:28.318080+0000","last_active":"2024-04-25T10:10:28.319250+0000","last_peered":"2024-04-25T10:10:28.319250+0000","last_clean":"2024-04-25T10:10:28.319250+0000","last_became_active":"2024-04-25T10:10:28.317817+0000","last_became_peered":"2024-04-25T10:10:28.317817+0000","last_unstale":"2024-04-25T10:10:28.319250+0000","last_undegraded":"2024-04-25T10:10:28.319250+0000","last_fullsized":"2024-04-25T10:10:28.319250+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:36.803150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.339483+0000","last_change":"2024-04-25T10:10:28.324650+0000","last_active":"2024-04-25T10:10:28.339483+0000","last_peered":"2024-04-25T10:10:28.339483+0000","last_clean":"2024-04-25T10:10:28.339483+0000","last_became_active":"2024-04-25T10:10:28.324140+0000","last_became_peered":"2024-04-25T10:10:28.324140+0000","last_unstale":"2024-04-25T10:10:28.339483+0000","last_undegraded":"2024-04-25T10:10:28.339483+0000","last_fullsized":"2024-04-25T10:10:28.339483+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:21.813044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.319134+0000","last_change":"2024-04-25T10:10:28.316929+0000","last_active":"2024-04-25T10:10:28.319134+0000","last_peered":"2024-04-25T10:10:28.319134+0000","last_clean":"2024-04-25T10:10:28.319134+0000","last_became_active":"2024-04-25T10:10:28.316559+0000","last_became_peered":"2024-04-25T10:10:28.316559+0000","last_unstale":"2024-04-25T10:10:28.319134+0000","last_undegraded":"2024-04-25T10:10:28.319134+0000","last_fullsized":"2024-04-25T10:10:28.319134+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:24.133926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.331625+0000","last_change":"2024-04-25T10:10:28.330452+0000","last_active":"2024-04-25T10:10:28.331625+0000","last_peered":"2024-04-25T10:10:28.331625+0000","last_clean":"2024-04-25T10:10:28.331625+0000","last_became_active":"2024-04-25T10:10:28.326565+0000","last_became_peered":"2024-04-25T10:10:28.326565+0000","last_unstale":"2024-04-25T10:10:28.331625+0000","last_undegraded":"2024-04-25T10:10:28.331625+0000","last_fullsized":"2024-04-25T10:10:28.331625+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:08.050216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288276+0000","last_change":"2024-04-25T10:10:28.308300+0000","last_active":"2024-04-25T10:10:29.288276+0000","last_peered":"2024-04-25T10:10:29.288276+0000","last_clean":"2024-04-25T10:10:29.288276+0000","last_became_active":"2024-04-25T10:10:28.307143+0000","last_became_peered":"2024-04-25T10:10:28.307143+0000","last_unstale":"2024-04-25T10:10:29.288276+0000","last_undegraded":"2024-04-25T10:10:29.288276+0000","last_fullsized":"2024-04-25T10:10:29.288276+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:07.272079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284527+0000","last_change":"2024-04-25T10:10:28.328591+0000","last_active":"2024-04-25T10:10:29.284527+0000","last_peered":"2024-04-25T10:10:29.284527+0000","last_clean":"2024-04-25T10:10:29.284527+0000","last_became_active":"2024-04-25T10:10:28.328245+0000","last_became_peered":"2024-04-25T10:10:28.328245+0000","last_unstale":"2024-04-25T10:10:29.284527+0000","last_undegraded":"2024-04-25T10:10:29.284527+0000","last_fullsized":"2024-04-25T10:10:29.284527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:37.658716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.323608+0000","last_change":"2024-04-25T10:10:28.319783+0000","last_active":"2024-04-25T10:10:28.323608+0000","last_peered":"2024-04-25T10:10:28.323608+0000","last_clean":"2024-04-25T10:10:28.323608+0000","last_became_active":"2024-04-25T10:10:28.319199+0000","last_became_peered":"2024-04-25T10:10:28.319199+0000","last_unstale":"2024-04-25T10:10:28.323608+0000","last_undegraded":"2024-04-25T10:10:28.323608+0000","last_fullsized":"2024-04-25T10:10:28.323608+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:10.461603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.335048+0000","last_change":"2024-04-25T10:10:28.329253+0000","last_active":"2024-04-25T10:10:28.335048+0000","last_peered":"2024-04-25T10:10:28.335048+0000","last_clean":"2024-04-25T10:10:28.335048+0000","last_became_active":"2024-04-25T10:10:28.328915+0000","last_became_peered":"2024-04-25T10:10:28.328915+0000","last_unstale":"2024-04-25T10:10:28.335048+0000","last_undegraded":"2024-04-25T10:10:28.335048+0000","last_fullsized":"2024-04-25T10:10:28.335048+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:55.829641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.289482+0000","last_change":"2024-04-25T10:10:28.340916+0000","last_active":"2024-04-25T10:10:29.289482+0000","last_peered":"2024-04-25T10:10:29.289482+0000","last_clean":"2024-04-25T10:10:29.289482+0000","last_became_active":"2024-04-25T10:10:28.340409+0000","last_became_peered":"2024-04-25T10:10:28.340409+0000","last_unstale":"2024-04-25T10:10:29.289482+0000","last_undegraded":"2024-04-25T10:10:29.289482+0000","last_fullsized":"2024-04-25T10:10:29.289482+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.538305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.324131+0000","last_change":"2024-04-25T10:10:28.322769+0000","last_active":"2024-04-25T10:10:28.324131+0000","last_peered":"2024-04-25T10:10:28.324131+0000","last_clean":"2024-04-25T10:10:28.324131+0000","last_became_active":"2024-04-25T10:10:28.322410+0000","last_became_peered":"2024-04-25T10:10:28.322410+0000","last_unstale":"2024-04-25T10:10:28.324131+0000","last_undegraded":"2024-04-25T10:10:28.324131+0000","last_fullsized":"2024-04-25T10:10:28.324131+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:58.157544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287026+0000","last_change":"2024-04-25T10:10:28.318420+0000","last_active":"2024-04-25T10:10:29.287026+0000","last_peered":"2024-04-25T10:10:29.287026+0000","last_clean":"2024-04-25T10:10:29.287026+0000","last_became_active":"2024-04-25T10:10:28.317122+0000","last_became_peered":"2024-04-25T10:10:28.317122+0000","last_unstale":"2024-04-25T10:10:29.287026+0000","last_undegraded":"2024-04-25T10:10:29.287026+0000","last_fullsized":"2024-04-25T10:10:29.287026+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.887519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.319528+0000","last_change":"2024-04-25T10:10:28.318937+0000","last_active":"2024-04-25T10:10:28.319528+0000","last_peered":"2024-04-25T10:10:28.319528+0000","last_clean":"2024-04-25T10:10:28.319528+0000","last_became_active":"2024-04-25T10:10:28.318584+0000","last_became_peered":"2024-04-25T10:10:28.318584+0000","last_unstale":"2024-04-25T10:10:28.319528+0000","last_undegraded":"2024-04-25T10:10:28.319528+0000","last_fullsized":"2024-04-25T10:10:28.319528+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:55.670419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.331838+0000","last_change":"2024-04-25T10:10:28.330882+0000","last_active":"2024-04-25T10:10:28.331838+0000","last_peered":"2024-04-25T10:10:28.331838+0000","last_clean":"2024-04-25T10:10:28.331838+0000","last_became_active":"2024-04-25T10:10:28.330505+0000","last_became_peered":"2024-04-25T10:10:28.330505+0000","last_unstale":"2024-04-25T10:10:28.331838+0000","last_undegraded":"2024-04-25T10:10:28.331838+0000","last_fullsized":"2024-04-25T10:10:28.331838+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.887519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288027+0000","last_change":"2024-04-25T10:10:28.332568+0000","last_active":"2024-04-25T10:10:29.288027+0000","last_peered":"2024-04-25T10:10:29.288027+0000","last_clean":"2024-04-25T10:10:29.288027+0000","last_became_active":"2024-04-25T10:10:28.330421+0000","last_became_peered":"2024-04-25T10:10:28.330421+0000","last_unstale":"2024-04-25T10:10:29.288027+0000","last_undegraded":"2024-04-25T10:10:29.288027+0000","last_fullsized":"2024-04-25T10:10:29.288027+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:41.922164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.329551+0000","last_change":"2024-04-25T10:10:28.326379+0000","last_active":"2024-04-25T10:10:28.329551+0000","last_peered":"2024-04-25T10:10:28.329551+0000","last_clean":"2024-04-25T10:10:28.329551+0000","last_became_active":"2024-04-25T10:10:28.321718+0000","last_became_peered":"2024-04-25T10:10:28.321718+0000","last_unstale":"2024-04-25T10:10:28.329551+0000","last_undegraded":"2024-04-25T10:10:28.329551+0000","last_fullsized":"2024-04-25T10:10:28.329551+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:22.622715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284349+0000","last_change":"2024-04-25T10:10:28.324293+0000","last_active":"2024-04-25T10:10:29.284349+0000","last_peered":"2024-04-25T10:10:29.284349+0000","last_clean":"2024-04-25T10:10:29.284349+0000","last_became_active":"2024-04-25T10:10:28.323975+0000","last_became_peered":"2024-04-25T10:10:28.323975+0000","last_unstale":"2024-04-25T10:10:29.284349+0000","last_undegraded":"2024-04-25T10:10:29.284349+0000","last_fullsized":"2024-04-25T10:10:29.284349+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:32.457132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284993+0000","last_change":"2024-04-25T10:10:28.314354+0000","last_active":"2024-04-25T10:10:29.284993+0000","last_peered":"2024-04-25T10:10:29.284993+0000","last_clean":"2024-04-25T10:10:29.284993+0000","last_became_active":"2024-04-25T10:10:28.313588+0000","last_became_peered":"2024-04-25T10:10:28.313588+0000","last_unstale":"2024-04-25T10:10:29.284993+0000","last_undegraded":"2024-04-25T10:10:29.284993+0000","last_fullsized":"2024-04-25T10:10:29.284993+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:13.894682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.325536+0000","last_change":"2024-04-25T10:10:28.325011+0000","last_active":"2024-04-25T10:10:28.325536+0000","last_peered":"2024-04-25T10:10:28.325536+0000","last_clean":"2024-04-25T10:10:28.325536+0000","last_became_active":"2024-04-25T10:10:28.324727+0000","last_became_peered":"2024-04-25T10:10:28.324727+0000","last_unstale":"2024-04-25T10:10:28.325536+0000","last_undegraded":"2024-04-25T10:10:28.325536+0000","last_fullsized":"2024-04-25T10:10:28.325536+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:03.269418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287403+0000","last_change":"2024-04-25T10:10:28.334252+0000","last_active":"2024-04-25T10:10:29.287403+0000","last_peered":"2024-04-25T10:10:29.287403+0000","last_clean":"2024-04-25T10:10:29.287403+0000","last_became_active":"2024-04-25T10:10:28.324155+0000","last_became_peered":"2024-04-25T10:10:28.324155+0000","last_unstale":"2024-04-25T10:10:29.287403+0000","last_undegraded":"2024-04-25T10:10:29.287403+0000","last_fullsized":"2024-04-25T10:10:29.287403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:23.458722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.330758+0000","last_change":"2024-04-25T10:10:28.326913+0000","last_active":"2024-04-25T10:10:28.330758+0000","last_peered":"2024-04-25T10:10:28.330758+0000","last_clean":"2024-04-25T10:10:28.330758+0000","last_became_active":"2024-04-25T10:10:28.325914+0000","last_became_peered":"2024-04-25T10:10:28.325914+0000","last_unstale":"2024-04-25T10:10:28.330758+0000","last_undegraded":"2024-04-25T10:10:28.330758+0000","last_fullsized":"2024-04-25T10:10:28.330758+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:10.066431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287705+0000","last_change":"2024-04-25T10:10:28.302076+0000","last_active":"2024-04-25T10:10:29.287705+0000","last_peered":"2024-04-25T10:10:29.287705+0000","last_clean":"2024-04-25T10:10:29.287705+0000","last_became_active":"2024-04-25T10:10:28.301241+0000","last_became_peered":"2024-04-25T10:10:28.301241+0000","last_unstale":"2024-04-25T10:10:29.287705+0000","last_undegraded":"2024-04-25T10:10:29.287705+0000","last_fullsized":"2024-04-25T10:10:29.287705+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:30.381428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288393+0000","last_change":"2024-04-25T10:10:28.329051+0000","last_active":"2024-04-25T10:10:29.288393+0000","last_peered":"2024-04-25T10:10:29.288393+0000","last_clean":"2024-04-25T10:10:29.288393+0000","last_became_active":"2024-04-25T10:10:28.328161+0000","last_became_peered":"2024-04-25T10:10:28.328161+0000","last_unstale":"2024-04-25T10:10:29.288393+0000","last_undegraded":"2024-04-25T10:10:29.288393+0000","last_fullsized":"2024-04-25T10:10:29.288393+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:08.767292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286336+0000","last_change":"2024-04-25T10:10:28.337875+0000","last_active":"2024-04-25T10:10:29.286336+0000","last_peered":"2024-04-25T10:10:29.286336+0000","last_clean":"2024-04-25T10:10:29.286336+0000","last_became_active":"2024-04-25T10:10:28.332834+0000","last_became_peered":"2024-04-25T10:10:28.332834+0000","last_unstale":"2024-04-25T10:10:29.286336+0000","last_undegraded":"2024-04-25T10:10:29.286336+0000","last_fullsized":"2024-04-25T10:10:29.286336+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:10.066431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.319290+0000","last_change":"2024-04-25T10:10:28.318404+0000","last_active":"2024-04-25T10:10:28.319290+0000","last_peered":"2024-04-25T10:10:28.319290+0000","last_clean":"2024-04-25T10:10:28.319290+0000","last_became_active":"2024-04-25T10:10:28.317075+0000","last_became_peered":"2024-04-25T10:10:28.317075+0000","last_unstale":"2024-04-25T10:10:28.319290+0000","last_undegraded":"2024-04-25T10:10:28.319290+0000","last_fullsized":"2024-04-25T10:10:28.319290+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:03.501555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.339369+0000","last_change":"2024-04-25T10:10:28.327303+0000","last_active":"2024-04-25T10:10:28.339369+0000","last_peered":"2024-04-25T10:10:28.339369+0000","last_clean":"2024-04-25T10:10:28.339369+0000","last_became_active":"2024-04-25T10:10:28.326577+0000","last_became_peered":"2024-04-25T10:10:28.326577+0000","last_unstale":"2024-04-25T10:10:28.339369+0000","last_undegraded":"2024-04-25T10:10:28.339369+0000","last_fullsized":"2024-04-25T10:10:28.339369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:03.269418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288906+0000","last_change":"2024-04-25T10:10:28.323630+0000","last_active":"2024-04-25T10:10:29.288906+0000","last_peered":"2024-04-25T10:10:29.288906+0000","last_clean":"2024-04-25T10:10:29.288906+0000","last_became_active":"2024-04-25T10:10:28.323199+0000","last_became_peered":"2024-04-25T10:10:28.323199+0000","last_unstale":"2024-04-25T10:10:29.288906+0000","last_undegraded":"2024-04-25T10:10:29.288906+0000","last_fullsized":"2024-04-25T10:10:29.288906+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:17.452679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.308306+0000","last_change":"2024-04-25T10:10:28.303701+0000","last_active":"2024-04-25T10:10:28.308306+0000","last_peered":"2024-04-25T10:10:28.308306+0000","last_clean":"2024-04-25T10:10:28.308306+0000","last_became_active":"2024-04-25T10:10:28.302828+0000","last_became_peered":"2024-04-25T10:10:28.302828+0000","last_unstale":"2024-04-25T10:10:28.308306+0000","last_undegraded":"2024-04-25T10:10:28.308306+0000","last_fullsized":"2024-04-25T10:10:28.308306+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:07.701599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.331414+0000","last_change":"2024-04-25T10:10:28.330437+0000","last_active":"2024-04-25T10:10:28.331414+0000","last_peered":"2024-04-25T10:10:28.331414+0000","last_clean":"2024-04-25T10:10:28.331414+0000","last_became_active":"2024-04-25T10:10:28.326372+0000","last_became_peered":"2024-04-25T10:10:28.326372+0000","last_unstale":"2024-04-25T10:10:28.331414+0000","last_undegraded":"2024-04-25T10:10:28.331414+0000","last_fullsized":"2024-04-25T10:10:28.331414+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.388442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.310873+0000","last_change":"2024-04-25T10:10:28.309026+0000","last_active":"2024-04-25T10:10:28.310873+0000","last_peered":"2024-04-25T10:10:28.310873+0000","last_clean":"2024-04-25T10:10:28.310873+0000","last_became_active":"2024-04-25T10:10:28.306186+0000","last_became_peered":"2024-04-25T10:10:28.306186+0000","last_unstale":"2024-04-25T10:10:28.310873+0000","last_undegraded":"2024-04-25T10:10:28.310873+0000","last_fullsized":"2024-04-25T10:10:28.310873+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:19.434582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288954+0000","last_change":"2024-04-25T10:10:28.315961+0000","last_active":"2024-04-25T10:10:29.288954+0000","last_peered":"2024-04-25T10:10:29.288954+0000","last_clean":"2024-04-25T10:10:29.288954+0000","last_became_active":"2024-04-25T10:10:28.315120+0000","last_became_peered":"2024-04-25T10:10:28.315120+0000","last_unstale":"2024-04-25T10:10:29.288954+0000","last_undegraded":"2024-04-25T10:10:29.288954+0000","last_fullsized":"2024-04-25T10:10:29.288954+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:04.003802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.340065+0000","last_change":"2024-04-25T10:10:28.336815+0000","last_active":"2024-04-25T10:10:28.340065+0000","last_peered":"2024-04-25T10:10:28.340065+0000","last_clean":"2024-04-25T10:10:28.340065+0000","last_became_active":"2024-04-25T10:10:28.336323+0000","last_became_peered":"2024-04-25T10:10:28.336323+0000","last_unstale":"2024-04-25T10:10:28.340065+0000","last_undegraded":"2024-04-25T10:10:28.340065+0000","last_fullsized":"2024-04-25T10:10:28.340065+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:21.713065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284215+0000","last_change":"2024-04-25T10:10:28.322840+0000","last_active":"2024-04-25T10:10:29.284215+0000","last_peered":"2024-04-25T10:10:29.284215+0000","last_clean":"2024-04-25T10:10:29.284215+0000","last_became_active":"2024-04-25T10:10:28.318520+0000","last_became_peered":"2024-04-25T10:10:28.318520+0000","last_unstale":"2024-04-25T10:10:29.284215+0000","last_undegraded":"2024-04-25T10:10:29.284215+0000","last_fullsized":"2024-04-25T10:10:29.284215+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:40.790359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285214+0000","last_change":"2024-04-25T10:10:28.309982+0000","last_active":"2024-04-25T10:10:29.285214+0000","last_peered":"2024-04-25T10:10:29.285214+0000","last_clean":"2024-04-25T10:10:29.285214+0000","last_became_active":"2024-04-25T10:10:28.309228+0000","last_became_peered":"2024-04-25T10:10:28.309228+0000","last_unstale":"2024-04-25T10:10:29.285214+0000","last_undegraded":"2024-04-25T10:10:29.285214+0000","last_fullsized":"2024-04-25T10:10:29.285214+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:55.666895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"93'4","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288000+0000","last_change":"2024-04-25T10:10:27.302793+0000","last_active":"2024-04-25T10:10:29.288000+0000","last_peered":"2024-04-25T10:10:29.288000+0000","last_clean":"2024-04-25T10:10:29.288000+0000","last_became_active":"2024-04-25T10:10:27.292088+0000","last_became_peered":"2024-04-25T10:10:27.292088+0000","last_unstale":"2024-04-25T10:10:29.288000+0000","last_undegraded":"2024-04-25T10:10:29.288000+0000","last_fullsized":"2024-04-25T10:10:29.288000+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:26.154284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285146+0000","last_change":"2024-04-25T10:10:28.300731+0000","last_active":"2024-04-25T10:10:29.285146+0000","last_peered":"2024-04-25T10:10:29.285146+0000","last_clean":"2024-04-25T10:10:29.285146+0000","last_became_active":"2024-04-25T10:10:28.299979+0000","last_became_peered":"2024-04-25T10:10:28.299979+0000","last_unstale":"2024-04-25T10:10:29.285146+0000","last_undegraded":"2024-04-25T10:10:29.285146+0000","last_fullsized":"2024-04-25T10:10:29.285146+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:24.957535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.289019+0000","last_change":"2024-04-25T10:10:27.310672+0000","last_active":"2024-04-25T10:10:29.289019+0000","last_peered":"2024-04-25T10:10:29.289019+0000","last_clean":"2024-04-25T10:10:29.289019+0000","last_became_active":"2024-04-25T10:10:27.309845+0000","last_became_peered":"2024-04-25T10:10:27.309845+0000","last_unstale":"2024-04-25T10:10:29.289019+0000","last_undegraded":"2024-04-25T10:10:29.289019+0000","last_fullsized":"2024-04-25T10:10:29.289019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:42.806982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.325427+0000","last_change":"2024-04-25T10:10:28.322262+0000","last_active":"2024-04-25T10:10:28.325427+0000","last_peered":"2024-04-25T10:10:28.325427+0000","last_clean":"2024-04-25T10:10:28.325427+0000","last_became_active":"2024-04-25T10:10:28.321630+0000","last_became_peered":"2024-04-25T10:10:28.321630+0000","last_unstale":"2024-04-25T10:10:28.325427+0000","last_undegraded":"2024-04-25T10:10:28.325427+0000","last_fullsized":"2024-04-25T10:10:28.325427+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:22.550422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"93'2","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.333025+0000","last_change":"2024-04-25T10:10:27.300974+0000","last_active":"2024-04-25T10:10:28.333025+0000","last_peered":"2024-04-25T10:10:28.333025+0000","last_clean":"2024-04-25T10:10:28.333025+0000","last_became_active":"2024-04-25T10:10:27.300220+0000","last_became_peered":"2024-04-25T10:10:27.300220+0000","last_unstale":"2024-04-25T10:10:28.333025+0000","last_undegraded":"2024-04-25T10:10:28.333025+0000","last_fullsized":"2024-04-25T10:10:28.333025+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:22.801287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285356+0000","last_change":"2024-04-25T10:10:28.303317+0000","last_active":"2024-04-25T10:10:29.285356+0000","last_peered":"2024-04-25T10:10:29.285356+0000","last_clean":"2024-04-25T10:10:29.285356+0000","last_became_active":"2024-04-25T10:10:28.302504+0000","last_became_peered":"2024-04-25T10:10:28.302504+0000","last_unstale":"2024-04-25T10:10:29.285356+0000","last_undegraded":"2024-04-25T10:10:29.285356+0000","last_fullsized":"2024-04-25T10:10:29.285356+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:59.446821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"93'2","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287925+0000","last_change":"2024-04-25T10:10:27.309064+0000","last_active":"2024-04-25T10:10:29.287925+0000","last_peered":"2024-04-25T10:10:29.287925+0000","last_clean":"2024-04-25T10:10:29.287925+0000","last_became_active":"2024-04-25T10:10:27.308427+0000","last_became_peered":"2024-04-25T10:10:27.308427+0000","last_unstale":"2024-04-25T10:10:29.287925+0000","last_undegraded":"2024-04-25T10:10:29.287925+0000","last_fullsized":"2024-04-25T10:10:29.287925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:52.720532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.332021+0000","last_change":"2024-04-25T10:10:28.325994+0000","last_active":"2024-04-25T10:10:28.332021+0000","last_peered":"2024-04-25T10:10:28.332021+0000","last_clean":"2024-04-25T10:10:28.332021+0000","last_became_active":"2024-04-25T10:10:28.321835+0000","last_became_peered":"2024-04-25T10:10:28.321835+0000","last_unstale":"2024-04-25T10:10:28.332021+0000","last_undegraded":"2024-04-25T10:10:28.332021+0000","last_fullsized":"2024-04-25T10:10:28.332021+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:38.129582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288520+0000","last_change":"2024-04-25T10:10:27.310336+0000","last_active":"2024-04-25T10:10:29.288520+0000","last_peered":"2024-04-25T10:10:29.288520+0000","last_clean":"2024-04-25T10:10:29.288520+0000","last_became_active":"2024-04-25T10:10:27.305866+0000","last_became_peered":"2024-04-25T10:10:27.305866+0000","last_unstale":"2024-04-25T10:10:29.288520+0000","last_undegraded":"2024-04-25T10:10:29.288520+0000","last_fullsized":"2024-04-25T10:10:29.288520+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:46.384387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286679+0000","last_change":"2024-04-25T10:10:28.318935+0000","last_active":"2024-04-25T10:10:29.286679+0000","last_peered":"2024-04-25T10:10:29.286679+0000","last_clean":"2024-04-25T10:10:29.286679+0000","last_became_active":"2024-04-25T10:10:28.318261+0000","last_became_peered":"2024-04-25T10:10:28.318261+0000","last_unstale":"2024-04-25T10:10:29.286679+0000","last_undegraded":"2024-04-25T10:10:29.286679+0000","last_fullsized":"2024-04-25T10:10:29.286679+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:55.829641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"93'1","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.332969+0000","last_change":"2024-04-25T10:10:27.303155+0000","last_active":"2024-04-25T10:10:28.332969+0000","last_peered":"2024-04-25T10:10:28.332969+0000","last_clean":"2024-04-25T10:10:28.332969+0000","last_became_active":"2024-04-25T10:10:27.302591+0000","last_became_peered":"2024-04-25T10:10:27.302591+0000","last_unstale":"2024-04-25T10:10:28.332969+0000","last_undegraded":"2024-04-25T10:10:28.332969+0000","last_fullsized":"2024-04-25T10:10:28.332969+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:46.945830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284654+0000","last_change":"2024-04-25T10:10:28.328413+0000","last_active":"2024-04-25T10:10:29.284654+0000","last_peered":"2024-04-25T10:10:29.284654+0000","last_clean":"2024-04-25T10:10:29.284654+0000","last_became_active":"2024-04-25T10:10:28.327907+0000","last_became_peered":"2024-04-25T10:10:28.327907+0000","last_unstale":"2024-04-25T10:10:29.284654+0000","last_undegraded":"2024-04-25T10:10:29.284654+0000","last_fullsized":"2024-04-25T10:10:29.284654+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:10.451849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.296227+0000","last_change":"2024-04-25T10:10:27.301369+0000","last_active":"2024-04-25T10:10:28.296227+0000","last_peered":"2024-04-25T10:10:28.296227+0000","last_clean":"2024-04-25T10:10:28.296227+0000","last_became_active":"2024-04-25T10:10:27.300986+0000","last_became_peered":"2024-04-25T10:10:27.300986+0000","last_unstale":"2024-04-25T10:10:28.296227+0000","last_undegraded":"2024-04-25T10:10:28.296227+0000","last_fullsized":"2024-04-25T10:10:28.296227+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:32.526980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.332251+0000","last_change":"2024-04-25T10:10:28.331466+0000","last_active":"2024-04-25T10:10:28.332251+0000","last_peered":"2024-04-25T10:10:28.332251+0000","last_clean":"2024-04-25T10:10:28.332251+0000","last_became_active":"2024-04-25T10:10:28.331170+0000","last_became_peered":"2024-04-25T10:10:28.331170+0000","last_unstale":"2024-04-25T10:10:28.332251+0000","last_undegraded":"2024-04-25T10:10:28.332251+0000","last_fullsized":"2024-04-25T10:10:28.332251+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:37.342513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.295683+0000","last_change":"2024-04-25T10:10:27.308920+0000","last_active":"2024-04-25T10:10:28.295683+0000","last_peered":"2024-04-25T10:10:28.295683+0000","last_clean":"2024-04-25T10:10:28.295683+0000","last_became_active":"2024-04-25T10:10:27.307228+0000","last_became_peered":"2024-04-25T10:10:27.307228+0000","last_unstale":"2024-04-25T10:10:28.295683+0000","last_undegraded":"2024-04-25T10:10:28.295683+0000","last_fullsized":"2024-04-25T10:10:28.295683+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:39.250479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287269+0000","last_change":"2024-04-25T10:10:28.319163+0000","last_active":"2024-04-25T10:10:29.287269+0000","last_peered":"2024-04-25T10:10:29.287269+0000","last_clean":"2024-04-25T10:10:29.287269+0000","last_became_active":"2024-04-25T10:10:28.318584+0000","last_became_peered":"2024-04-25T10:10:28.318584+0000","last_unstale":"2024-04-25T10:10:29.287269+0000","last_undegraded":"2024-04-25T10:10:29.287269+0000","last_fullsized":"2024-04-25T10:10:29.287269+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:37.342513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.324642+0000","last_change":"2024-04-25T10:10:27.298886+0000","last_active":"2024-04-25T10:10:28.324642+0000","last_peered":"2024-04-25T10:10:28.324642+0000","last_clean":"2024-04-25T10:10:28.324642+0000","last_became_active":"2024-04-25T10:10:27.298189+0000","last_became_peered":"2024-04-25T10:10:27.298189+0000","last_unstale":"2024-04-25T10:10:28.324642+0000","last_undegraded":"2024-04-25T10:10:28.324642+0000","last_fullsized":"2024-04-25T10:10:28.324642+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:35.791825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.307481+0000","last_change":"2024-04-25T10:10:28.306127+0000","last_active":"2024-04-25T10:10:28.307481+0000","last_peered":"2024-04-25T10:10:28.307481+0000","last_clean":"2024-04-25T10:10:28.307481+0000","last_became_active":"2024-04-25T10:10:28.305041+0000","last_became_peered":"2024-04-25T10:10:28.305041+0000","last_unstale":"2024-04-25T10:10:28.307481+0000","last_undegraded":"2024-04-25T10:10:28.307481+0000","last_fullsized":"2024-04-25T10:10:28.307481+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:56.380015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.298225+0000","last_change":"2024-04-25T10:10:27.294212+0000","last_active":"2024-04-25T10:10:28.298225+0000","last_peered":"2024-04-25T10:10:28.298225+0000","last_clean":"2024-04-25T10:10:28.298225+0000","last_became_active":"2024-04-25T10:10:27.293226+0000","last_became_peered":"2024-04-25T10:10:27.293226+0000","last_unstale":"2024-04-25T10:10:28.298225+0000","last_undegraded":"2024-04-25T10:10:28.298225+0000","last_fullsized":"2024-04-25T10:10:28.298225+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:42.903578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.318868+0000","last_change":"2024-04-25T10:10:28.317659+0000","last_active":"2024-04-25T10:10:28.318868+0000","last_peered":"2024-04-25T10:10:28.318868+0000","last_clean":"2024-04-25T10:10:28.318868+0000","last_became_active":"2024-04-25T10:10:28.317388+0000","last_became_peered":"2024-04-25T10:10:28.317388+0000","last_unstale":"2024-04-25T10:10:28.318868+0000","last_undegraded":"2024-04-25T10:10:28.318868+0000","last_fullsized":"2024-04-25T10:10:28.318868+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:05.260306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"93'3","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287867+0000","last_change":"2024-04-25T10:10:27.322442+0000","last_active":"2024-04-25T10:10:29.287867+0000","last_peered":"2024-04-25T10:10:29.287867+0000","last_clean":"2024-04-25T10:10:29.287867+0000","last_became_active":"2024-04-25T10:10:27.318243+0000","last_became_peered":"2024-04-25T10:10:27.318243+0000","last_unstale":"2024-04-25T10:10:29.287867+0000","last_undegraded":"2024-04-25T10:10:29.287867+0000","last_fullsized":"2024-04-25T10:10:29.287867+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:57.976245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.341149+0000","last_change":"2024-04-25T10:10:28.340588+0000","last_active":"2024-04-25T10:10:28.341149+0000","last_peered":"2024-04-25T10:10:28.341149+0000","last_clean":"2024-04-25T10:10:28.341149+0000","last_became_active":"2024-04-25T10:10:28.340276+0000","last_became_peered":"2024-04-25T10:10:28.340276+0000","last_unstale":"2024-04-25T10:10:28.341149+0000","last_undegraded":"2024-04-25T10:10:28.341149+0000","last_fullsized":"2024-04-25T10:10:28.341149+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:23.458722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"93'6","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285806+0000","last_change":"2024-04-25T10:10:27.301510+0000","last_active":"2024-04-25T10:10:29.285806+0000","last_peered":"2024-04-25T10:10:29.285806+0000","last_clean":"2024-04-25T10:10:29.285806+0000","last_became_active":"2024-04-25T10:10:27.301023+0000","last_became_peered":"2024-04-25T10:10:27.301023+0000","last_unstale":"2024-04-25T10:10:29.285806+0000","last_undegraded":"2024-04-25T10:10:29.285806+0000","last_fullsized":"2024-04-25T10:10:29.285806+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:43.893353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.326155+0000","last_change":"2024-04-25T10:10:28.315268+0000","last_active":"2024-04-25T10:10:28.326155+0000","last_peered":"2024-04-25T10:10:28.326155+0000","last_clean":"2024-04-25T10:10:28.326155+0000","last_became_active":"2024-04-25T10:10:28.314749+0000","last_became_peered":"2024-04-25T10:10:28.314749+0000","last_unstale":"2024-04-25T10:10:28.326155+0000","last_undegraded":"2024-04-25T10:10:28.326155+0000","last_fullsized":"2024-04-25T10:10:28.326155+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:49.937031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"93'2","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.335097+0000","last_change":"2024-04-25T10:10:27.308575+0000","last_active":"2024-04-25T10:10:28.335097+0000","last_peered":"2024-04-25T10:10:28.335097+0000","last_clean":"2024-04-25T10:10:28.335097+0000","last_became_active":"2024-04-25T10:10:27.307340+0000","last_became_peered":"2024-04-25T10:10:27.307340+0000","last_unstale":"2024-04-25T10:10:28.335097+0000","last_undegraded":"2024-04-25T10:10:28.335097+0000","last_fullsized":"2024-04-25T10:10:28.335097+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:54.237235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286456+0000","last_change":"2024-04-25T10:10:28.311400+0000","last_active":"2024-04-25T10:10:29.286456+0000","last_peered":"2024-04-25T10:10:29.286456+0000","last_clean":"2024-04-25T10:10:29.286456+0000","last_became_active":"2024-04-25T10:10:28.310446+0000","last_became_peered":"2024-04-25T10:10:28.310446+0000","last_unstale":"2024-04-25T10:10:29.286456+0000","last_undegraded":"2024-04-25T10:10:29.286456+0000","last_fullsized":"2024-04-25T10:10:29.286456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:08.050216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284629+0000","last_change":"2024-04-25T10:10:27.317031+0000","last_active":"2024-04-25T10:10:29.284629+0000","last_peered":"2024-04-25T10:10:29.284629+0000","last_clean":"2024-04-25T10:10:29.284629+0000","last_became_active":"2024-04-25T10:10:27.316674+0000","last_became_peered":"2024-04-25T10:10:27.316674+0000","last_unstale":"2024-04-25T10:10:29.284629+0000","last_undegraded":"2024-04-25T10:10:29.284629+0000","last_fullsized":"2024-04-25T10:10:29.284629+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:43.373222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.332493+0000","last_change":"2024-04-25T10:10:28.331920+0000","last_active":"2024-04-25T10:10:28.332493+0000","last_peered":"2024-04-25T10:10:28.332493+0000","last_clean":"2024-04-25T10:10:28.332493+0000","last_became_active":"2024-04-25T10:10:28.331493+0000","last_became_peered":"2024-04-25T10:10:28.331493+0000","last_unstale":"2024-04-25T10:10:28.332493+0000","last_undegraded":"2024-04-25T10:10:28.332493+0000","last_fullsized":"2024-04-25T10:10:28.332493+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:09.095479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.305461+0000","last_change":"2024-04-25T10:10:27.304114+0000","last_active":"2024-04-25T10:10:28.305461+0000","last_peered":"2024-04-25T10:10:28.305461+0000","last_clean":"2024-04-25T10:10:28.305461+0000","last_became_active":"2024-04-25T10:10:27.303799+0000","last_became_peered":"2024-04-25T10:10:27.303799+0000","last_unstale":"2024-04-25T10:10:28.305461+0000","last_undegraded":"2024-04-25T10:10:28.305461+0000","last_fullsized":"2024-04-25T10:10:28.305461+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:47.197897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.308633+0000","last_change":"2024-04-25T10:10:28.306623+0000","last_active":"2024-04-25T10:10:28.308633+0000","last_peered":"2024-04-25T10:10:28.308633+0000","last_clean":"2024-04-25T10:10:28.308633+0000","last_became_active":"2024-04-25T10:10:28.305762+0000","last_became_peered":"2024-04-25T10:10:28.305762+0000","last_unstale":"2024-04-25T10:10:28.308633+0000","last_undegraded":"2024-04-25T10:10:28.308633+0000","last_fullsized":"2024-04-25T10:10:28.308633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:41.891188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"93'8","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.336083+0000","last_change":"2024-04-25T10:10:27.301949+0000","last_active":"2024-04-25T10:10:28.336083+0000","last_peered":"2024-04-25T10:10:28.336083+0000","last_clean":"2024-04-25T10:10:28.336083+0000","last_became_active":"2024-04-25T10:10:27.295986+0000","last_became_peered":"2024-04-25T10:10:27.295986+0000","last_unstale":"2024-04-25T10:10:28.336083+0000","last_undegraded":"2024-04-25T10:10:28.336083+0000","last_fullsized":"2024-04-25T10:10:28.336083+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:03.630684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285558+0000","last_change":"2024-04-25T10:10:28.310548+0000","last_active":"2024-04-25T10:10:29.285558+0000","last_peered":"2024-04-25T10:10:29.285558+0000","last_clean":"2024-04-25T10:10:29.285558+0000","last_became_active":"2024-04-25T10:10:28.309715+0000","last_became_peered":"2024-04-25T10:10:28.309715+0000","last_unstale":"2024-04-25T10:10:29.285558+0000","last_undegraded":"2024-04-25T10:10:29.285558+0000","last_fullsized":"2024-04-25T10:10:29.285558+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:14.815549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288462+0000","last_change":"2024-04-25T10:10:27.308552+0000","last_active":"2024-04-25T10:10:29.288462+0000","last_peered":"2024-04-25T10:10:29.288462+0000","last_clean":"2024-04-25T10:10:29.288462+0000","last_became_active":"2024-04-25T10:10:27.302875+0000","last_became_peered":"2024-04-25T10:10:27.302875+0000","last_unstale":"2024-04-25T10:10:29.288462+0000","last_undegraded":"2024-04-25T10:10:29.288462+0000","last_fullsized":"2024-04-25T10:10:29.288462+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:47.197897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286773+0000","last_change":"2024-04-25T10:10:28.332134+0000","last_active":"2024-04-25T10:10:29.286773+0000","last_peered":"2024-04-25T10:10:29.286773+0000","last_clean":"2024-04-25T10:10:29.286773+0000","last_became_active":"2024-04-25T10:10:28.331302+0000","last_became_peered":"2024-04-25T10:10:28.331302+0000","last_unstale":"2024-04-25T10:10:29.286773+0000","last_undegraded":"2024-04-25T10:10:29.286773+0000","last_fullsized":"2024-04-25T10:10:29.286773+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:47.388442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"93'2","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.341562+0000","last_change":"2024-04-25T10:10:27.310332+0000","last_active":"2024-04-25T10:10:28.341562+0000","last_peered":"2024-04-25T10:10:28.341562+0000","last_clean":"2024-04-25T10:10:28.341562+0000","last_became_active":"2024-04-25T10:10:27.309291+0000","last_became_peered":"2024-04-25T10:10:27.309291+0000","last_unstale":"2024-04-25T10:10:28.341562+0000","last_undegraded":"2024-04-25T10:10:28.341562+0000","last_fullsized":"2024-04-25T10:10:28.341562+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:31.783160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288867+0000","last_change":"2024-04-25T10:10:28.308072+0000","last_active":"2024-04-25T10:10:29.288867+0000","last_peered":"2024-04-25T10:10:29.288867+0000","last_clean":"2024-04-25T10:10:29.288867+0000","last_became_active":"2024-04-25T10:10:28.307187+0000","last_became_peered":"2024-04-25T10:10:28.307187+0000","last_unstale":"2024-04-25T10:10:29.288867+0000","last_undegraded":"2024-04-25T10:10:29.288867+0000","last_fullsized":"2024-04-25T10:10:29.288867+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:28.331746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"93'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288139+0000","last_change":"2024-04-25T10:10:27.315877+0000","last_active":"2024-04-25T10:10:29.288139+0000","last_peered":"2024-04-25T10:10:29.288139+0000","last_clean":"2024-04-25T10:10:29.288139+0000","last_became_active":"2024-04-25T10:10:27.312147+0000","last_became_peered":"2024-04-25T10:10:27.312147+0000","last_unstale":"2024-04-25T10:10:29.288139+0000","last_undegraded":"2024-04-25T10:10:29.288139+0000","last_fullsized":"2024-04-25T10:10:29.288139+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:19.142471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284382+0000","last_change":"2024-04-25T10:10:28.328730+0000","last_active":"2024-04-25T10:10:29.284382+0000","last_peered":"2024-04-25T10:10:29.284382+0000","last_clean":"2024-04-25T10:10:29.284382+0000","last_became_active":"2024-04-25T10:10:28.328242+0000","last_became_peered":"2024-04-25T10:10:28.328242+0000","last_unstale":"2024-04-25T10:10:29.284382+0000","last_undegraded":"2024-04-25T10:10:29.284382+0000","last_fullsized":"2024-04-25T10:10:29.284382+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:39.416124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"93'2","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.339304+0000","last_change":"2024-04-25T10:10:27.300887+0000","last_active":"2024-04-25T10:10:28.339304+0000","last_peered":"2024-04-25T10:10:28.339304+0000","last_clean":"2024-04-25T10:10:28.339304+0000","last_became_active":"2024-04-25T10:10:27.300023+0000","last_became_peered":"2024-04-25T10:10:27.300023+0000","last_unstale":"2024-04-25T10:10:28.339304+0000","last_undegraded":"2024-04-25T10:10:28.339304+0000","last_fullsized":"2024-04-25T10:10:28.339304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:42.903578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.340034+0000","last_change":"2024-04-25T10:10:28.334998+0000","last_active":"2024-04-25T10:10:28.340034+0000","last_peered":"2024-04-25T10:10:28.340034+0000","last_clean":"2024-04-25T10:10:28.340034+0000","last_became_active":"2024-04-25T10:10:28.325267+0000","last_became_peered":"2024-04-25T10:10:28.325267+0000","last_unstale":"2024-04-25T10:10:28.340034+0000","last_undegraded":"2024-04-25T10:10:28.340034+0000","last_fullsized":"2024-04-25T10:10:28.340034+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:09.037782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.289366+0000","last_change":"2024-04-25T10:10:27.305009+0000","last_active":"2024-04-25T10:10:28.289366+0000","last_peered":"2024-04-25T10:10:28.289366+0000","last_clean":"2024-04-25T10:10:28.289366+0000","last_became_active":"2024-04-25T10:10:27.304643+0000","last_became_peered":"2024-04-25T10:10:27.304643+0000","last_unstale":"2024-04-25T10:10:28.289366+0000","last_undegraded":"2024-04-25T10:10:28.289366+0000","last_fullsized":"2024-04-25T10:10:28.289366+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:51.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.340474+0000","last_change":"2024-04-25T10:10:28.336366+0000","last_active":"2024-04-25T10:10:28.340474+0000","last_peered":"2024-04-25T10:10:28.340474+0000","last_clean":"2024-04-25T10:10:28.340474+0000","last_became_active":"2024-04-25T10:10:28.335961+0000","last_became_peered":"2024-04-25T10:10:28.335961+0000","last_unstale":"2024-04-25T10:10:28.340474+0000","last_undegraded":"2024-04-25T10:10:28.340474+0000","last_fullsized":"2024-04-25T10:10:28.340474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:24.213072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.313566+0000","last_change":"2024-04-25T10:10:27.299089+0000","last_active":"2024-04-25T10:10:28.313566+0000","last_peered":"2024-04-25T10:10:28.313566+0000","last_clean":"2024-04-25T10:10:28.313566+0000","last_became_active":"2024-04-25T10:10:27.297943+0000","last_became_peered":"2024-04-25T10:10:27.297943+0000","last_unstale":"2024-04-25T10:10:28.313566+0000","last_undegraded":"2024-04-25T10:10:28.313566+0000","last_fullsized":"2024-04-25T10:10:28.313566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.248981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.284918+0000","last_change":"2024-04-25T10:10:28.298794+0000","last_active":"2024-04-25T10:10:29.284918+0000","last_peered":"2024-04-25T10:10:29.284918+0000","last_clean":"2024-04-25T10:10:29.284918+0000","last_became_active":"2024-04-25T10:10:28.297973+0000","last_became_peered":"2024-04-25T10:10:28.297973+0000","last_unstale":"2024-04-25T10:10:29.284918+0000","last_undegraded":"2024-04-25T10:10:29.284918+0000","last_fullsized":"2024-04-25T10:10:29.284918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:48.953685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.285916+0000","last_change":"2024-04-25T10:10:27.306022+0000","last_active":"2024-04-25T10:10:28.285916+0000","last_peered":"2024-04-25T10:10:28.285916+0000","last_clean":"2024-04-25T10:10:28.285916+0000","last_became_active":"2024-04-25T10:10:27.305657+0000","last_became_peered":"2024-04-25T10:10:27.305657+0000","last_unstale":"2024-04-25T10:10:28.285916+0000","last_undegraded":"2024-04-25T10:10:28.285916+0000","last_fullsized":"2024-04-25T10:10:28.285916+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:35.612043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287651+0000","last_change":"2024-04-25T10:10:28.310030+0000","last_active":"2024-04-25T10:10:29.287651+0000","last_peered":"2024-04-25T10:10:29.287651+0000","last_clean":"2024-04-25T10:10:29.287651+0000","last_became_active":"2024-04-25T10:10:28.307706+0000","last_became_peered":"2024-04-25T10:10:28.307706+0000","last_unstale":"2024-04-25T10:10:29.287651+0000","last_undegraded":"2024-04-25T10:10:29.287651+0000","last_fullsized":"2024-04-25T10:10:29.287651+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.887519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"93'2","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.343645+0000","last_change":"2024-04-25T10:10:27.317000+0000","last_active":"2024-04-25T10:10:28.343645+0000","last_peered":"2024-04-25T10:10:28.343645+0000","last_clean":"2024-04-25T10:10:28.343645+0000","last_became_active":"2024-04-25T10:10:27.316258+0000","last_became_peered":"2024-04-25T10:10:27.316258+0000","last_unstale":"2024-04-25T10:10:28.343645+0000","last_undegraded":"2024-04-25T10:10:28.343645+0000","last_fullsized":"2024-04-25T10:10:28.343645+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:59.373353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285440+0000","last_change":"2024-04-25T10:10:28.314012+0000","last_active":"2024-04-25T10:10:29.285440+0000","last_peered":"2024-04-25T10:10:29.285440+0000","last_clean":"2024-04-25T10:10:29.285440+0000","last_became_active":"2024-04-25T10:10:28.312690+0000","last_became_peered":"2024-04-25T10:10:28.312690+0000","last_unstale":"2024-04-25T10:10:29.285440+0000","last_undegraded":"2024-04-25T10:10:29.285440+0000","last_fullsized":"2024-04-25T10:10:29.285440+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:21.905038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.289059+0000","last_change":"2024-04-25T10:10:27.317293+0000","last_active":"2024-04-25T10:10:29.289059+0000","last_peered":"2024-04-25T10:10:29.289059+0000","last_clean":"2024-04-25T10:10:29.289059+0000","last_became_active":"2024-04-25T10:10:27.315272+0000","last_became_peered":"2024-04-25T10:10:27.315272+0000","last_unstale":"2024-04-25T10:10:29.289059+0000","last_undegraded":"2024-04-25T10:10:29.289059+0000","last_fullsized":"2024-04-25T10:10:29.289059+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:52.720532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286139+0000","last_change":"2024-04-25T10:10:28.299773+0000","last_active":"2024-04-25T10:10:29.286139+0000","last_peered":"2024-04-25T10:10:29.286139+0000","last_clean":"2024-04-25T10:10:29.286139+0000","last_became_active":"2024-04-25T10:10:28.298976+0000","last_became_peered":"2024-04-25T10:10:28.298976+0000","last_unstale":"2024-04-25T10:10:29.286139+0000","last_undegraded":"2024-04-25T10:10:29.286139+0000","last_fullsized":"2024-04-25T10:10:29.286139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:16.565779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288376+0000","last_change":"2024-04-25T10:10:27.321974+0000","last_active":"2024-04-25T10:10:29.288376+0000","last_peered":"2024-04-25T10:10:29.288376+0000","last_clean":"2024-04-25T10:10:29.288376+0000","last_became_active":"2024-04-25T10:10:27.317317+0000","last_became_peered":"2024-04-25T10:10:27.317317+0000","last_unstale":"2024-04-25T10:10:29.288376+0000","last_undegraded":"2024-04-25T10:10:29.288376+0000","last_fullsized":"2024-04-25T10:10:29.288376+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:05.621873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.339122+0000","last_change":"2024-04-25T10:10:28.324947+0000","last_active":"2024-04-25T10:10:28.339122+0000","last_peered":"2024-04-25T10:10:28.339122+0000","last_clean":"2024-04-25T10:10:28.339122+0000","last_became_active":"2024-04-25T10:10:28.324225+0000","last_became_peered":"2024-04-25T10:10:28.324225+0000","last_unstale":"2024-04-25T10:10:28.339122+0000","last_undegraded":"2024-04-25T10:10:28.339122+0000","last_fullsized":"2024-04-25T10:10:28.339122+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:03.501555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.288188+0000","last_change":"2024-04-25T10:10:27.308732+0000","last_active":"2024-04-25T10:10:29.288188+0000","last_peered":"2024-04-25T10:10:29.288188+0000","last_clean":"2024-04-25T10:10:29.288188+0000","last_became_active":"2024-04-25T10:10:27.307850+0000","last_became_peered":"2024-04-25T10:10:27.307850+0000","last_unstale":"2024-04-25T10:10:29.288188+0000","last_undegraded":"2024-04-25T10:10:29.288188+0000","last_fullsized":"2024-04-25T10:10:29.288188+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:31.790492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.301403+0000","last_change":"2024-04-25T10:10:28.299895+0000","last_active":"2024-04-25T10:10:28.301403+0000","last_peered":"2024-04-25T10:10:28.301403+0000","last_clean":"2024-04-25T10:10:28.301403+0000","last_became_active":"2024-04-25T10:10:28.299188+0000","last_became_peered":"2024-04-25T10:10:28.299188+0000","last_unstale":"2024-04-25T10:10:28.301403+0000","last_undegraded":"2024-04-25T10:10:28.301403+0000","last_fullsized":"2024-04-25T10:10:28.301403+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:44.690270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285027+0000","last_change":"2024-04-25T10:10:27.319853+0000","last_active":"2024-04-25T10:10:29.285027+0000","last_peered":"2024-04-25T10:10:29.285027+0000","last_clean":"2024-04-25T10:10:29.285027+0000","last_became_active":"2024-04-25T10:10:27.319366+0000","last_became_peered":"2024-04-25T10:10:27.319366+0000","last_unstale":"2024-04-25T10:10:29.285027+0000","last_undegraded":"2024-04-25T10:10:29.285027+0000","last_fullsized":"2024-04-25T10:10:29.285027+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:31.783160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.324913+0000","last_change":"2024-04-25T10:10:28.323420+0000","last_active":"2024-04-25T10:10:28.324913+0000","last_peered":"2024-04-25T10:10:28.324913+0000","last_clean":"2024-04-25T10:10:28.324913+0000","last_became_active":"2024-04-25T10:10:28.323035+0000","last_became_peered":"2024-04-25T10:10:28.323035+0000","last_unstale":"2024-04-25T10:10:28.324913+0000","last_undegraded":"2024-04-25T10:10:28.324913+0000","last_fullsized":"2024-04-25T10:10:28.324913+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:21.813044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.312659+0000","last_change":"2024-04-25T10:10:27.298885+0000","last_active":"2024-04-25T10:10:28.312659+0000","last_peered":"2024-04-25T10:10:28.312659+0000","last_clean":"2024-04-25T10:10:28.312659+0000","last_became_active":"2024-04-25T10:10:27.298090+0000","last_became_peered":"2024-04-25T10:10:27.298090+0000","last_unstale":"2024-04-25T10:10:28.312659+0000","last_undegraded":"2024-04-25T10:10:28.312659+0000","last_fullsized":"2024-04-25T10:10:28.312659+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:23.122601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.286951+0000","last_change":"2024-04-25T10:10:28.322727+0000","last_active":"2024-04-25T10:10:29.286951+0000","last_peered":"2024-04-25T10:10:29.286951+0000","last_clean":"2024-04-25T10:10:29.286951+0000","last_became_active":"2024-04-25T10:10:28.322072+0000","last_became_peered":"2024-04-25T10:10:28.322072+0000","last_unstale":"2024-04-25T10:10:29.286951+0000","last_undegraded":"2024-04-25T10:10:29.286951+0000","last_fullsized":"2024-04-25T10:10:29.286951+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:08.050216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.287621+0000","last_change":"2024-04-25T10:10:27.301219+0000","last_active":"2024-04-25T10:10:29.287621+0000","last_peered":"2024-04-25T10:10:29.287621+0000","last_clean":"2024-04-25T10:10:29.287621+0000","last_became_active":"2024-04-25T10:10:27.300499+0000","last_became_peered":"2024-04-25T10:10:27.300499+0000","last_unstale":"2024-04-25T10:10:29.287621+0000","last_undegraded":"2024-04-25T10:10:29.287621+0000","last_fullsized":"2024-04-25T10:10:29.287621+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:03.630684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T10:10:29.285058+0000","last_change":"2024-04-25T10:10:28.313177+0000","last_active":"2024-04-25T10:10:29.285058+0000","last_peered":"2024-04-25T10:10:29.285058+0000","last_clean":"2024-04-25T10:10:29.285058+0000","last_became_active":"2024-04-25T10:10:28.312379+0000","last_became_peered":"2024-04-25T10:10:28.312379+0000","last_unstale":"2024-04-25T10:10:29.285058+0000","last_undegraded":"2024-04-25T10:10:29.285058+0000","last_fullsized":"2024-04-25T10:10:29.285058+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:04.999305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.296036+0000","last_change":"2024-04-25T10:10:27.309578+0000","last_active":"2024-04-25T10:10:28.296036+0000","last_peered":"2024-04-25T10:10:28.296036+0000","last_clean":"2024-04-25T10:10:28.296036+0000","last_became_active":"2024-04-25T10:10:27.308199+0000","last_became_peered":"2024-04-25T10:10:27.308199+0000","last_unstale":"2024-04-25T10:10:28.296036+0000","last_undegraded":"2024-04-25T10:10:28.296036+0000","last_fullsized":"2024-04-25T10:10:28.296036+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:14.584043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.321516+0000","last_change":"2024-04-25T10:10:28.320107+0000","last_active":"2024-04-25T10:10:28.321516+0000","last_peered":"2024-04-25T10:10:28.321516+0000","last_clean":"2024-04-25T10:10:28.321516+0000","last_became_active":"2024-04-25T10:10:28.319437+0000","last_became_peered":"2024-04-25T10:10:28.319437+0000","last_unstale":"2024-04-25T10:10:28.321516+0000","last_undegraded":"2024-04-25T10:10:28.321516+0000","last_fullsized":"2024-04-25T10:10:28.321516+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:27.265569+0000","last_clean_scrub_stamp":"2024-04-25T10:10:27.265569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:52.573149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.320438+0000","last_change":"2024-04-25T10:10:27.307639+0000","last_active":"2024-04-25T10:10:28.320438+0000","last_peered":"2024-04-25T10:10:28.320438+0000","last_clean":"2024-04-25T10:10:28.320438+0000","last_became_active":"2024-04-25T10:10:27.307078+0000","last_became_peered":"2024-04-25T10:10:27.307078+0000","last_unstale":"2024-04-25T10:10:28.320438+0000","last_undegraded":"2024-04-25T10:10:28.320438+0000","last_fullsized":"2024-04-25T10:10:28.320438+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:26.254244+0000","last_clean_scrub_stamp":"2024-04-25T10:10:26.254244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.634506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":308,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T10:10:28.296170+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:10:28.296170+0000","last_peered":"2024-04-25T10:10:28.296170+0000","last_clean":"2024-04-25T10:10:28.296170+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:10:28.296170+0000","last_undegraded":"2024-04-25T10:10:28.296170+0000","last_fullsized":"2024-04-25T10:10:28.296170+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64828,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307012,"statfs":{"total":96636764160,"available":96570380288,"internally_reserved":0,"allocated":1748992,"data_stored":1120858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574959,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58696,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313144,"statfs":{"total":96636764160,"available":96576659456,"internally_reserved":0,"allocated":1167360,"data_stored":532650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54508,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317332,"statfs":{"total":96636764160,"available":96580947968,"internally_reserved":0,"allocated":1146880,"data_stored":530434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68420,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303420,"statfs":{"total":96636764160,"available":96566702080,"internally_reserved":0,"allocated":1167360,"data_stored":537286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574958,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68400,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303440,"statfs":{"total":96636764160,"available":96566722560,"internally_reserved":0,"allocated":1146880,"data_stored":530434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58676,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313164,"statfs":{"total":96636764160,"available":96576679936,"internally_reserved":0,"allocated":1146880,"data_stored":530434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54516,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317324,"statfs":{"total":96636764160,"available":96580939776,"internally_reserved":0,"allocated":1150976,"data_stored":530504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574959,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64860,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306980,"statfs":{"total":96636764160,"available":96570347520,"internally_reserved":0,"allocated":1777664,"data_stored":1129812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:10:32.671 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:10:32.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:10:32.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f64279bd640 1 Processor -- start 2024-04-25T10:10:32.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f64279bd640 1 -- start start 2024-04-25T10:10:32.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f64279bd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f642006bfb0 0x7f6420165710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f64279bd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f64200d21c0 0x7f6420167c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f64279bd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64200cfbd0 con 0x7f64200ce2e0 2024-04-25T10:10:32.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f64279bd640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64200d0c70 con 0x7f642006bfb0 2024-04-25T10:10:32.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f64279bd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64200cfeb0 con 0x7f64200d21c0 2024-04-25T10:10:32.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f6425759640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f64200ce2e0 0x7f64200ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40178/0 (socket says 172.21.15.133:40178) 2024-04-25T10:10:32.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f6425759640 1 -- 172.21.15.133:0/1486646472 learned_addr learned my addr 172.21.15.133:0/1486646472 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:32.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f6424f58640 1 --2- 172.21.15.133:0/1486646472 >> v2:172.21.15.133:3300/0 conn(0x7f642006bfb0 0x7f6420165710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f6417fff640 1 -- 172.21.15.133:0/1486646472 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345865335 0 0) 0x7f64200cfbd0 con 0x7f64200ce2e0 2024-04-25T10:10:32.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f6417fff640 1 -- 172.21.15.133:0/1486646472 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f640c003530 con 0x7f64200ce2e0 2024-04-25T10:10:32.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f6425f5a640 1 --2- 172.21.15.133:0/1486646472 >> v2:172.21.15.162:3300/0 conn(0x7f64200d21c0 0x7f6420167c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f6424f58640 1 -- 172.21.15.133:0/1486646472 >> v2:172.21.15.162:3300/0 conn(0x7f64200d21c0 msgr2=0x7f6420167c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:32.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f6424f58640 1 --2- 172.21.15.133:0/1486646472 >> v2:172.21.15.162:3300/0 conn(0x7f64200d21c0 0x7f6420167c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f6424f58640 1 -- 172.21.15.133:0/1486646472 >> v1:172.21.15.133:6789/0 conn(0x7f64200ce2e0 legacy=0x7f64200ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.838+0000 7f6424f58640 1 -- 172.21.15.133:0/1486646472 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64200d0190 con 0x7f642006bfb0 2024-04-25T10:10:32.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f6424f58640 1 --2- 172.21.15.133:0/1486646472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f642006bfb0 0x7f6420165710 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f6420071c40 tx=0x7f641001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c753cc13835ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f6417fff640 1 -- 172.21.15.133:0/1486646472 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64100b0070 con 0x7f642006bfb0 2024-04-25T10:10:32.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f6417fff640 1 -- 172.21.15.133:0/1486646472 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64100ab600 con 0x7f642006bfb0 2024-04-25T10:10:32.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f6417fff640 1 -- 172.21.15.133:0/1486646472 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64100ab9b0 con 0x7f642006bfb0 2024-04-25T10:10:32.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 -- 172.21.15.133:0/1486646472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f642006bfb0 msgr2=0x7f6420165710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 --2- 172.21.15.133:0/1486646472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f642006bfb0 0x7f6420165710 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f6420071c40 tx=0x7f641001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 -- 172.21.15.133:0/1486646472 shutdown_connections 2024-04-25T10:10:32.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 --2- 172.21.15.133:0/1486646472 >> v2:172.21.15.162:3300/0 conn(0x7f64200d21c0 0x7f6420167c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 --2- 172.21.15.133:0/1486646472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f642006bfb0 0x7f6420165710 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 -- 172.21.15.133:0/1486646472 >> 172.21.15.133:0/1486646472 conn(0x7f64200d16a0 msgr2=0x7f64200d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:32.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 -- 172.21.15.133:0/1486646472 shutdown_connections 2024-04-25T10:10:32.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 -- 172.21.15.133:0/1486646472 wait complete. 2024-04-25T10:10:32.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 Processor -- start 2024-04-25T10:10:32.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 -- start start 2024-04-25T10:10:32.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f642006bfb0 0x7f642009d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64200ce2e0 0x7f64200700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64200d21c0 0x7f6420070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64200cf9b0 con 0x7f642006bfb0 2024-04-25T10:10:32.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64200cf830 con 0x7f64200d21c0 2024-04-25T10:10:32.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f64279bd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64200ccd00 con 0x7f64200ce2e0 2024-04-25T10:10:32.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f6425759640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f642006bfb0 0x7f642009d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f6425759640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f642006bfb0 0x7f642009d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42782/0 (socket says 172.21.15.133:42782) 2024-04-25T10:10:32.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.842+0000 7f6425759640 1 -- 172.21.15.133:0/3930868555 learned_addr learned my addr 172.21.15.133:0/3930868555 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6424f58640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64200ce2e0 0x7f64200700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6425759640 1 -- 172.21.15.133:0/3930868555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64200ce2e0 msgr2=0x7f64200700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6425759640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64200ce2e0 0x7f64200700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6425f5a640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64200d21c0 0x7f6420070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6425759640 1 -- 172.21.15.133:0/3930868555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64200d21c0 msgr2=0x7f6420070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6425759640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64200d21c0 0x7f6420070600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6425759640 1 -- 172.21.15.133:0/3930868555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64200af6e0 con 0x7f642006bfb0 2024-04-25T10:10:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6425759640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f642006bfb0 0x7f642009d3f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f641800c3c0 tx=0x7f641800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6415ffb640 1 -- 172.21.15.133:0/3930868555 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418017070 con 0x7f642006bfb0 2024-04-25T10:10:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6415ffb640 1 -- 172.21.15.133:0/3930868555 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6418005cf0 con 0x7f642006bfb0 2024-04-25T10:10:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f64279bd640 1 -- 172.21.15.133:0/3930868555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642009b5f0 con 0x7f642006bfb0 2024-04-25T10:10:32.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6415ffb640 1 -- 172.21.15.133:0/3930868555 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64180072f0 con 0x7f642006bfb0 2024-04-25T10:10:32.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f64279bd640 1 -- 172.21.15.133:0/3930868555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642009bbb0 con 0x7f642006bfb0 2024-04-25T10:10:32.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f64279bd640 1 -- 172.21.15.133:0/3930868555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6420067c70 con 0x7f642006bfb0 2024-04-25T10:10:32.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6415ffb640 1 -- 172.21.15.133:0/3930868555 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f641801a070 con 0x7f642006bfb0 2024-04-25T10:10:32.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6415ffb640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6404066f60 0x7f6404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6424f58640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6404066f60 0x7f6404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6415ffb640 1 -- 172.21.15.133:0/3930868555 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f64180845a0 con 0x7f642006bfb0 2024-04-25T10:10:32.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.846+0000 7f6424f58640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6404066f60 0x7f6404069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f64100ad000 tx=0x7f64100a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:32.850+0000 7f6415ffb640 1 -- 172.21.15.133:0/3930868555 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6418055440 con 0x7f642006bfb0 2024-04-25T10:10:33.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.050+0000 7f64279bd640 1 -- 172.21.15.133:0/3930868555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f64200ceb90 con 0x7f642006bfb0 2024-04-25T10:10:33.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f6415ffb640 1 -- 172.21.15.133:0/3930868555 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6418054d40 con 0x7f642006bfb0 2024-04-25T10:10:33.057 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:10:33.057 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:10:33.058 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:10:33.058 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:10:33.058 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:10:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:10:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 -- 172.21.15.133:0/3930868555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6404066f60 msgr2=0x7f6404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6404066f60 0x7f6404069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f64100ad000 tx=0x7f64100a6000 comp rx=0 tx=0).stop 2024-04-25T10:10:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 -- 172.21.15.133:0/3930868555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f642006bfb0 msgr2=0x7f642009d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f642006bfb0 0x7f642009d3f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f641800c3c0 tx=0x7f641800c880 comp rx=0 tx=0).stop 2024-04-25T10:10:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 -- 172.21.15.133:0/3930868555 shutdown_connections 2024-04-25T10:10:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64200d21c0 0x7f6420070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64200ce2e0 0x7f64200700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6404066f60 0x7f6404069420 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 --2- 172.21.15.133:0/3930868555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f642006bfb0 0x7f642009d3f0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 -- 172.21.15.133:0/3930868555 >> 172.21.15.133:0/3930868555 conn(0x7f64200d16a0 msgr2=0x7f64200b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 -- 172.21.15.133:0/3930868555 shutdown_connections 2024-04-25T10:10:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.054+0000 7f64279bd640 1 -- 172.21.15.133:0/3930868555 wait complete. 2024-04-25T10:10:33.074 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:33.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea716f640 1 Processor -- start 2024-04-25T10:10:33.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea716f640 1 -- start start 2024-04-25T10:10:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea716f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6ea0159280 0x7f6ea0072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea716f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea00730e0 0x7f6ea006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea716f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f6ea006b1d0 0x7f6ea015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea716f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea006b620 con 0x7f6ea0159280 2024-04-25T10:10:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea716f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea006fb80 con 0x7f6ea006b1d0 2024-04-25T10:10:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea716f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea00709e0 con 0x7f6ea00730e0 2024-04-25T10:10:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea4f0b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6ea0159280 0x7f6ea0072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea4f0b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6ea0159280 0x7f6ea0072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42792/0 (socket says 172.21.15.133:42792) 2024-04-25T10:10:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea4f0b640 1 -- 172.21.15.133:0/1870912176 learned_addr learned my addr 172.21.15.133:0/1870912176 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6e9ffff640 1 --2- 172.21.15.133:0/1870912176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea00730e0 0x7f6ea006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea4f0b640 1 -- 172.21.15.133:0/1870912176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea00730e0 msgr2=0x7f6ea006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea570c640 1 --2- 172.21.15.133:0/1870912176 >> v2:172.21.15.162:3300/0 conn(0x7f6ea006b1d0 0x7f6ea015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea4f0b640 1 --2- 172.21.15.133:0/1870912176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea00730e0 0x7f6ea006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea4f0b640 1 -- 172.21.15.133:0/1870912176 >> v2:172.21.15.162:3300/0 conn(0x7f6ea006b1d0 msgr2=0x7f6ea015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea4f0b640 1 --2- 172.21.15.133:0/1870912176 >> v2:172.21.15.162:3300/0 conn(0x7f6ea006b1d0 0x7f6ea015f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6ea4f0b640 1 -- 172.21.15.133:0/1870912176 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea015cdc0 con 0x7f6ea0159280 2024-04-25T10:10:33.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.190+0000 7f6e9ffff640 1 --2- 172.21.15.133:0/1870912176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea00730e0 0x7f6ea006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:33.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea4f0b640 1 --2- 172.21.15.133:0/1870912176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea0159280 0x7f6ea0072ba0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00afef0 tx=0x7f6e9801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21679f21ae659bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6e9f7fe640 1 -- 172.21.15.133:0/1870912176 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980b0070 con 0x7f6ea0159280 2024-04-25T10:10:33.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6e9f7fe640 1 -- 172.21.15.133:0/1870912176 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e980ab600 con 0x7f6ea0159280 2024-04-25T10:10:33.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6e9f7fe640 1 -- 172.21.15.133:0/1870912176 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980ab9d0 con 0x7f6ea0159280 2024-04-25T10:10:33.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea716f640 1 -- 172.21.15.133:0/1870912176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea0159280 msgr2=0x7f6ea0072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea716f640 1 --2- 172.21.15.133:0/1870912176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea0159280 0x7f6ea0072ba0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00afef0 tx=0x7f6e9801c230 comp rx=0 tx=0).stop 2024-04-25T10:10:33.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea716f640 1 -- 172.21.15.133:0/1870912176 shutdown_connections 2024-04-25T10:10:33.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea716f640 1 --2- 172.21.15.133:0/1870912176 >> v2:172.21.15.162:3300/0 conn(0x7f6ea006b1d0 0x7f6ea015f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea716f640 1 --2- 172.21.15.133:0/1870912176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea00730e0 0x7f6ea006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea716f640 1 --2- 172.21.15.133:0/1870912176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea0159280 0x7f6ea0072ba0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea716f640 1 -- 172.21.15.133:0/1870912176 >> 172.21.15.133:0/1870912176 conn(0x7f6ea00b2370 msgr2=0x7f6ea00b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea716f640 1 -- 172.21.15.133:0/1870912176 shutdown_connections 2024-04-25T10:10:33.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea716f640 1 -- 172.21.15.133:0/1870912176 wait complete. 2024-04-25T10:10:33.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea716f640 1 Processor -- start 2024-04-25T10:10:33.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.194+0000 7f6ea716f640 1 -- start start 2024-04-25T10:10:33.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea716f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ea006b1d0 0x7f6ea014bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea716f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea0159280 0x7f6ea00b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea716f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea014d630 0x7f6ea014a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea716f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea0159660 con 0x7f6ea0159280 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea716f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea0070670 con 0x7f6ea006b1d0 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea716f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea01597e0 con 0x7f6ea014d630 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea0159280 0x7f6ea00b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea0159280 0x7f6ea00b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42808/0 (socket says 172.21.15.133:42808) 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9ffff640 1 -- 172.21.15.133:0/2220240544 learned_addr learned my addr 172.21.15.133:0/2220240544 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea4f0b640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ea006b1d0 0x7f6ea014bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea570c640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea014d630 0x7f6ea014a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9ffff640 1 -- 172.21.15.133:0/2220240544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea014d630 msgr2=0x7f6ea014a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9ffff640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea014d630 0x7f6ea014a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9ffff640 1 -- 172.21.15.133:0/2220240544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ea006b1d0 msgr2=0x7f6ea014bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9ffff640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ea006b1d0 0x7f6ea014bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9ffff640 1 -- 172.21.15.133:0/2220240544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea014db10 con 0x7f6ea0159280 2024-04-25T10:10:33.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea4f0b640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ea006b1d0 0x7f6ea014bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:33.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9ffff640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea0159280 0x7f6ea00b14a0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400c7f0 tx=0x7f6e9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9d7fa640 1 -- 172.21.15.133:0/2220240544 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94007a30 con 0x7f6ea0159280 2024-04-25T10:10:33.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9d7fa640 1 -- 172.21.15.133:0/2220240544 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e94007bd0 con 0x7f6ea0159280 2024-04-25T10:10:33.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea716f640 1 -- 172.21.15.133:0/2220240544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea0150870 con 0x7f6ea0159280 2024-04-25T10:10:33.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9d7fa640 1 -- 172.21.15.133:0/2220240544 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e940113f0 con 0x7f6ea0159280 2024-04-25T10:10:33.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea716f640 1 -- 172.21.15.133:0/2220240544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea0150e30 con 0x7f6ea0159280 2024-04-25T10:10:33.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea716f640 1 -- 172.21.15.133:0/2220240544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea0151030 con 0x7f6ea0159280 2024-04-25T10:10:33.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9d7fa640 1 -- 172.21.15.133:0/2220240544 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e94011590 con 0x7f6ea0159280 2024-04-25T10:10:33.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9d7fa640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e8c066f60 0x7f6e8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6e9d7fa640 1 -- 172.21.15.133:0/2220240544 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6e940854a0 con 0x7f6ea0159280 2024-04-25T10:10:33.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea4f0b640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e8c066f60 0x7f6e8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.198+0000 7f6ea4f0b640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e8c066f60 0x7f6e8c069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6e980ad000 tx=0x7f6e98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.202+0000 7f6e9d7fa640 1 -- 172.21.15.133:0/2220240544 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ea0151030 con 0x7f6ea0159280 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.398+0000 7f6ea716f640 1 -- 172.21.15.133:0/2220240544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ea00b5a80 con 0x7f6ea0159280 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6e9d7fa640 1 -- 172.21.15.133:0/2220240544 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f6e94015020 con 0x7f6ea0159280 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3089710291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3089710291},{"type":"v1","addr":"172.21.15.162:6834","nonce":3089710291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1390003376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1390003376},{"type":"v1","addr":"172.21.15.162:6835","nonce":1390003376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7866,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1134679682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1134679682},{"type":"v1","addr":"172.21.15.133:6836","nonce":1134679682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:27.661584+0000","modified":"2024-04-25T10:10:30.680424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3580592279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3580592279},{"type":"v1","addr":"172.21.15.133:6837","nonce":3580592279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 58 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 -- 172.21.15.133:0/2220240544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e8c066f60 msgr2=0x7f6e8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e8c066f60 0x7f6e8c069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6e980ad000 tx=0x7f6e98008040 comp rx=0 tx=0).stop 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 -- 172.21.15.133:0/2220240544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea0159280 msgr2=0x7f6ea00b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea0159280 0x7f6ea00b14a0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400c7f0 tx=0x7f6e9400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 -- 172.21.15.133:0/2220240544 shutdown_connections 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea014d630 0x7f6ea014a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e8c066f60 0x7f6e8c069420 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea0159280 0x7f6ea00b14a0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 --2- 172.21.15.133:0/2220240544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ea006b1d0 0x7f6ea014bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 -- 172.21.15.133:0/2220240544 >> 172.21.15.133:0/2220240544 conn(0x7f6ea00b2370 msgr2=0x7f6ea015daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 -- 172.21.15.133:0/2220240544 shutdown_connections 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.402+0000 7f6ea716f640 1 -- 172.21.15.133:0/2220240544 wait complete. 2024-04-25T10:10:33.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:10:27.661584+0000', 'modified': '2024-04-25T10:10:30.680424+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7869}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7869': {'gid': 7869, 'name': 'a', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/3580592279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 3580592279}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3580592279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:10:33.827 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:10:33.827 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:10:33.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.958+0000 7fb7cd50c640 1 Processor -- start 2024-04-25T10:10:33.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.958+0000 7fb7cd50c640 1 -- start start 2024-04-25T10:10:33.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.958+0000 7fb7cd50c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c815b4a0 0x7fb7c815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.958+0000 7fb7cd50c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb7c80af360 0x7fb7c80ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.958+0000 7fb7cd50c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c80b1870 con 0x7fb7c80b0f60 2024-04-25T10:10:33.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.958+0000 7fb7cd50c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c80b1b50 con 0x7fb7c80af360 2024-04-25T10:10:33.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.958+0000 7fb7cd50c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80b1e30 con 0x7fb7c815b4a0 2024-04-25T10:10:33.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.958+0000 7fb7c67fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb7c80b0f60 0x7fb7c80aec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40190/0 (socket says 172.21.15.133:40190) 2024-04-25T10:10:33.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.958+0000 7fb7c67fc640 1 -- 172.21.15.133:0/736942423 learned_addr learned my addr 172.21.15.133:0/736942423 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:33.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c77fe640 1 --2- 172.21.15.133:0/736942423 >> v2:172.21.15.162:3300/0 conn(0x7fb7c80af360 0x7fb7c80ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c815b4a0 0x7fb7c815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660760154 0 0) 0x7fb7c80b1870 con 0x7fb7c80b0f60 2024-04-25T10:10:33.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7b4003530 con 0x7fb7c80b0f60 2024-04-25T10:10:33.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3675472183 0 0) 0x7fb7b4003530 con 0x7fb7c80b0f60 2024-04-25T10:10:33.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c815b4a0 msgr2=0x7fb7c815dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c815b4a0 0x7fb7c815dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 >> v2:172.21.15.162:3300/0 conn(0x7fb7c80af360 msgr2=0x7fb7c80ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 --2- 172.21.15.133:0/736942423 >> v2:172.21.15.162:3300/0 conn(0x7fb7c80af360 0x7fb7c80ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c80b2110 con 0x7fb7c80b0f60 2024-04-25T10:10:33.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb7b0001680 con 0x7fb7c80b0f60 2024-04-25T10:10:33.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 >> v1:172.21.15.133:6789/0 conn(0x7fb7c80b0f60 legacy=0x7fb7c80aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 0x7fb7b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7b4004810 0x7fb7b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c815b4a0 0x7fb7b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80b1b50 con 0x7fb7b40039c0 2024-04-25T10:10:33.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80b1e30 con 0x7fb7b4004810 2024-04-25T10:10:33.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b400b9d0 con 0x7fb7c815b4a0 2024-04-25T10:10:33.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 0x7fb7b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c77fe640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c815b4a0 0x7fb7b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c67fc640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7b4004810 0x7fb7b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c6ffd640 1 -- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c815b4a0 msgr2=0x7fb7b40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c815b4a0 0x7fb7b40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c6ffd640 1 -- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7b4004810 msgr2=0x7fb7b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7b4004810 0x7fb7b4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c6ffd640 1 -- 172.21.15.133:0/736942423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b4003530 con 0x7fb7b40039c0 2024-04-25T10:10:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c67fc640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7b4004810 0x7fb7b4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:33.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 0x7fb7b4003de0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009ae0 tx=0x7fb7c001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf54658dfe5fd67e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00096b0 con 0x7fb7b40039c0 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c00a9470 con 0x7fb7b40039c0 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c5ffb640 1 -- 172.21.15.133:0/736942423 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00a9860 con 0x7fb7b40039c0 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7cd50c640 1 -- 172.21.15.133:0/736942423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 msgr2=0x7fb7b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7cd50c640 1 --2- 172.21.15.133:0/736942423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 0x7fb7b4003de0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009ae0 tx=0x7fb7c001ca10 comp rx=0 tx=0).stop 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7c77fe640 1 -- 172.21.15.133:0/736942423 reap_dead start 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7cd50c640 1 -- 172.21.15.133:0/736942423 shutdown_connections 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7cd50c640 1 -- 172.21.15.133:0/736942423 >> 172.21.15.133:0/736942423 conn(0x7fb7c8066f70 msgr2=0x7fb7c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7cd50c640 1 -- 172.21.15.133:0/736942423 shutdown_connections 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.962+0000 7fb7cd50c640 1 -- 172.21.15.133:0/736942423 wait complete. 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7cd50c640 1 Processor -- start 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7cd50c640 1 -- start start 2024-04-25T10:10:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7cd50c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 0x7fb7c81eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7cd50c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c81422f0 0x7fb7c81e9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7cd50c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7c8141660 0x7fb7c815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7cd50c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80ae1f0 con 0x7fb7b40039c0 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7cd50c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80ae070 con 0x7fb7c8141660 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7cd50c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80ae370 con 0x7fb7c81422f0 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 0x7fb7c81eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 0x7fb7c81eaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42822/0 (socket says 172.21.15.133:42822) 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c6ffd640 1 -- 172.21.15.133:0/316183203 learned_addr learned my addr 172.21.15.133:0/316183203 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c67fc640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c81422f0 0x7fb7c81e9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c77fe640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7c8141660 0x7fb7c815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c6ffd640 1 -- 172.21.15.133:0/316183203 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c81422f0 msgr2=0x7fb7c81e9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c81422f0 0x7fb7c81e9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c6ffd640 1 -- 172.21.15.133:0/316183203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7c8141660 msgr2=0x7fb7c815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7c8141660 0x7fb7c815d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c6ffd640 1 -- 172.21.15.133:0/316183203 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c81e9b80 con 0x7fb7b40039c0 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c77fe640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7c8141660 0x7fb7c815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 0x7fb7c81eaea0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fb7c001d2b0 tx=0x7fb7c0019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7bf7fe640 1 -- 172.21.15.133:0/316183203 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00a9c20 con 0x7fb7b40039c0 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7bf7fe640 1 -- 172.21.15.133:0/316183203 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c00a9e00 con 0x7fb7b40039c0 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7cd50c640 1 -- 172.21.15.133:0/316183203 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c81e9e00 con 0x7fb7b40039c0 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7bf7fe640 1 -- 172.21.15.133:0/316183203 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0005be0 con 0x7fb7b40039c0 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.966+0000 7fb7cd50c640 1 -- 172.21.15.133:0/316183203 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c81ea3c0 con 0x7fb7b40039c0 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7bf7fe640 1 -- 172.21.15.133:0/316183203 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7c00c2020 con 0x7fb7b40039c0 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7bf7fe640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7a8066f60 0x7fb7a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7bf7fe640 1 -- 172.21.15.133:0/316183203 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fb7c011fd40 con 0x7fb7b40039c0 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c67fc640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7a8066f60 0x7fb7a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c67fc640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7a8066f60 0x7fb7a8069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00018b0 tx=0x7fb7b0000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 Processor -- start 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 -- start start 2024-04-25T10:10:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191aa4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191a5b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631919425f0 con 0x563191aa5240 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631918a9da0 con 0x563191aa4920 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563191832150 con 0x563191a5af30 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42824/0 (socket says 172.21.15.133:42824) 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c67fc640 1 -- 172.21.15.133:0/2699719043 learned_addr learned my addr 172.21.15.133:0/2699719043 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c67fc640 1 -- 172.21.15.133:0/2699719043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 msgr2=0x563191a5b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c77fe640 1 --2- 172.21.15.133:0/2699719043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191a5b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/2699719043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191aa4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c67fc640 1 --2- 172.21.15.133:0/2699719043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191a5b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c67fc640 1 -- 172.21.15.133:0/2699719043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 msgr2=0x563191aa4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c67fc640 1 --2- 172.21.15.133:0/2699719043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191aa4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c67fc640 1 -- 172.21.15.133:0/2699719043 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56319184bca0 con 0x563191aa5240 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7c67fc640 1 --2- 172.21.15.133:0/2699719043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5a9f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000fd90 tx=0x7fb7b0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a14989ed40e39d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7bcff9640 1 -- 172.21.15.133:0/2699719043 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00055d0 con 0x563191aa5240 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7bcff9640 1 -- 172.21.15.133:0/2699719043 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b0005770 con 0x563191aa5240 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7bcff9640 1 -- 172.21.15.133:0/2699719043 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0021650 con 0x563191aa5240 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 -- 172.21.15.133:0/2699719043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 msgr2=0x563191a5a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 --2- 172.21.15.133:0/2699719043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5a9f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000fd90 tx=0x7fb7b0012000 comp rx=0 tx=0).stop 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 -- 172.21.15.133:0/2699719043 shutdown_connections 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 --2- 172.21.15.133:0/2699719043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191a5b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 --2- 172.21.15.133:0/2699719043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5a9f0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 --2- 172.21.15.133:0/2699719043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191aa4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 -- 172.21.15.133:0/2699719043 >> 172.21.15.133:0/2699719043 conn(0x563191a612a0 msgr2=0x563191a592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 -- 172.21.15.133:0/2699719043 shutdown_connections 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.970+0000 7fb7d0679000 1 -- 172.21.15.133:0/2699719043 wait complete. 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 -- start start 2024-04-25T10:10:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 Processor -- start 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191aabd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191aac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563191864990 con 0x563191aa5240 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631918a5e40 con 0x563191aa4920 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563191835900 con 0x563191a5af30 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 10 client.0 _ll_register_callbacks cb 0x56319195d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42834/0 (socket says 172.21.15.133:42834) 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c77fe640 1 -- 172.21.15.133:0/3752478547 learned_addr learned my addr 172.21.15.133:0/3752478547 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c67fc640 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191aac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191aabd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c77fe640 1 -- 172.21.15.133:0/3752478547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 msgr2=0x563191aabd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c77fe640 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191aabd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c77fe640 1 -- 172.21.15.133:0/3752478547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 msgr2=0x563191aac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c77fe640 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191aac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c77fe640 1 -- 172.21.15.133:0/3752478547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563191862630 con 0x563191aa5240 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c67fc640 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191aac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191aabd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c77fe640 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5eaa0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fb7c806b6f0 tx=0x7fb7b8013e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc20701f9ac3e97a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3752478547 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b801b070 con 0x563191aa5240 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 -- 172.21.15.133:0/3752478547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7b0005770 con 0x563191aa5240 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3752478547 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fb7b8011c30 con 0x563191aa5240 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3752478547 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7b8025050 con 0x563191aa5240 2024-04-25T10:10:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 10 client.8016 fetch_fsmap learned FSMap version 58 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 -- 172.21.15.133:0/3752478547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x5631918323a0 con 0x563191aa5240 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3752478547 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b0005770 con 0x563191aa5240 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3752478547 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb7b8028020 con 0x563191aa5240 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 10 client.8016 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 10 client.8016 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 20 client.8016 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x563191a60160 0x563191ab5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 4 client.8016 mds_command: new command op to 7869 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 -- 172.21.15.133:0/3752478547 --> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563191845a50 con 0x563191a60160 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x563191a60160 0x563191ab5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x563191a60160 0x563191ab5dc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bbe633ef1d45882 server_cookie=291754add113f019 in_seq=0 out_seq=0 2024-04-25T10:10:33.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7a5ffb640 10 client.8016 ms_handle_connect on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3752478547 <== mds.0 v2:172.21.15.133:6835/3580592279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563191845a50 con 0x563191a60160 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7a5ffb640 10 client.8016 handle_command_reply: tid=0 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 client.8016 shutdown 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 -- 172.21.15.133:0/3752478547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 msgr2=0x563191a5eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5eaa0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fb7c806b6f0 tx=0x7fb7b8013e20 comp rx=0 tx=0).stop 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 -- 172.21.15.133:0/3752478547 shutdown_connections 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x563191a60160 0x563191ab5dc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191a5eaa0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7a5ffb640 0 client.8016 ms_handle_reset on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191aac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3752478547 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191aabd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.974+0000 7fb7d0679000 1 -- 172.21.15.133:0/3752478547 >> 172.21.15.133:0/3752478547 conn(0x563191a612a0 msgr2=0x563191a599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.978+0000 7fb7d0679000 1 -- 172.21.15.133:0/3752478547 shutdown_connections 2024-04-25T10:10:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.978+0000 7fb7d0679000 1 -- 172.21.15.133:0/3752478547 wait complete. 2024-04-25T10:10:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.978+0000 7fb7d0679000 20 client.8016 trim_cache size 0 max 16384 2024-04-25T10:10:33.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.990+0000 7fb7d0679000 1 Processor -- start 2024-04-25T10:10:33.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.990+0000 7fb7d0679000 1 -- start start 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191a5af30 0x563191b00060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191b005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191aa5240 0x563191b00ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c8203190 con 0x563191a5af30 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c8143650 con 0x563191aa4920 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c8202bc0 con 0x563191aa5240 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191a5af30 0x563191b00060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191a5af30 0x563191b00060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42844/0 (socket says 172.21.15.133:42844) 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c6ffd640 1 -- 172.21.15.133:0/529672302 learned_addr learned my addr 172.21.15.133:0/529672302 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c6ffd640 1 -- 172.21.15.133:0/529672302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191aa5240 msgr2=0x563191b00ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/529672302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191aa5240 0x563191b00ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c6ffd640 1 -- 172.21.15.133:0/529672302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 msgr2=0x563191b005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/529672302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191b005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c6ffd640 1 -- 172.21.15.133:0/529672302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563191aa5070 con 0x563191a5af30 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/529672302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191a5af30 0x563191b00060 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fb7c001cef0 tx=0x7fb7c00f0c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39aebf251854f4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7a7fff640 1 -- 172.21.15.133:0/529672302 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00f8b60 con 0x563191a5af30 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7a7fff640 1 -- 172.21.15.133:0/529672302 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c00ee4f0 con 0x563191a5af30 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7a7fff640 1 -- 172.21.15.133:0/529672302 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0005d80 con 0x563191a5af30 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- 172.21.15.133:0/529672302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191a5af30 msgr2=0x563191b00060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 --2- 172.21.15.133:0/529672302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191a5af30 0x563191b00060 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fb7c001cef0 tx=0x7fb7c00f0c10 comp rx=0 tx=0).stop 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- 172.21.15.133:0/529672302 shutdown_connections 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 --2- 172.21.15.133:0/529672302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191aa5240 0x563191b00ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 --2- 172.21.15.133:0/529672302 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191b005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 --2- 172.21.15.133:0/529672302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191a5af30 0x563191b00060 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- 172.21.15.133:0/529672302 >> 172.21.15.133:0/529672302 conn(0x563191a612a0 msgr2=0x563191a5e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- 172.21.15.133:0/529672302 shutdown_connections 2024-04-25T10:10:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- 172.21.15.133:0/529672302 wait complete. 2024-04-25T10:10:34.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- start start 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 Processor -- start 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191b05ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191b05ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191ae9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631918ae3d0 con 0x563191aa5240 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x563191965d50 con 0x563191aa4920 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x563191844980 con 0x563191a5af30 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 10 client.0 _ll_register_callbacks cb 0x7fb7b000f8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191ae9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191ae9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42854/0 (socket says 172.21.15.133:42854) 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c77fe640 1 -- 172.21.15.133:0/3460761974 learned_addr learned my addr 172.21.15.133:0/3460761974 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c77fe640 1 -- 172.21.15.133:0/3460761974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 msgr2=0x563191b05ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c77fe640 1 --2- 172.21.15.133:0/3460761974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191b05ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c77fe640 1 -- 172.21.15.133:0/3460761974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 msgr2=0x563191b05ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c77fe640 1 --2- 172.21.15.133:0/3460761974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191b05ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c77fe640 1 -- 172.21.15.133:0/3460761974 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56319192b600 con 0x563191aa5240 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7c77fe640 1 --2- 172.21.15.133:0/3460761974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191ae9810 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80166e0 tx=0x7fb7b8004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e463b8708c389e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3460761974 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b802e060 con 0x563191aa5240 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3460761974 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fb7b8016cd0 con 0x563191aa5240 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- 172.21.15.133:0/3460761974 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7c00ee4f0 con 0x563191aa5240 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3460761974 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7b8034050 con 0x563191aa5240 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 10 client.8022 fetch_fsmap learned FSMap version 58 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 1 -- 172.21.15.133:0/3460761974 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x56319183e9b0 con 0x563191aa5240 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3460761974 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c00ee4f0 con 0x563191aa5240 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3460761974 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb7b8037020 con 0x563191aa5240 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 10 client.8022 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:10:34.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.994+0000 7fb7d0679000 10 client.8022 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 20 client.8022 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3460761974 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x563191ae9d50 0x563191aea130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 4 client.8022 mds_command: new command op to 7869 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 -- 172.21.15.133:0/3460761974 --> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5631918b72e0 con 0x563191ae9d50 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/3460761974 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x563191ae9d50 0x563191aea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7c6ffd640 1 --2- 172.21.15.133:0/3460761974 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x563191ae9d50 0x563191aea130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ebcbdbb77c90c2 server_cookie=d975fc3ebc3296c3 in_seq=0 out_seq=0 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7a5ffb640 10 client.8022 ms_handle_connect on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7a5ffb640 1 -- 172.21.15.133:0/3460761974 <== mds.0 v2:172.21.15.133:6835/3580592279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5631918b72e0 con 0x563191ae9d50 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7a5ffb640 10 client.8022 handle_command_reply: tid=0 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 client.8022 shutdown 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 -- 172.21.15.133:0/3460761974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 msgr2=0x563191ae9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3460761974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191ae9810 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80166e0 tx=0x7fb7b8004a50 comp rx=0 tx=0).stop 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 -- 172.21.15.133:0/3460761974 shutdown_connections 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3460761974 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x563191ae9d50 0x563191aea130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3460761974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563191aa5240 0x563191ae9810 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3460761974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563191aa4920 0x563191b05ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7a5ffb640 0 client.8022 ms_handle_reset on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 --2- 172.21.15.133:0/3460761974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x563191a5af30 0x563191b05ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 -- 172.21.15.133:0/3460761974 >> 172.21.15.133:0/3460761974 conn(0x563191a612a0 msgr2=0x563191aac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 -- 172.21.15.133:0/3460761974 shutdown_connections 2024-04-25T10:10:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 1 -- 172.21.15.133:0/3460761974 wait complete. 2024-04-25T10:10:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7d0679000 20 client.8022 trim_cache size 0 max 16384 2024-04-25T10:10:34.006 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:10:34.006 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:10:34.006 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:10:34.007 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 7869, 2024-04-25T10:10:34.007 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:10:34.007 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:10:34.007 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:10:34.007 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 5.6854363729999999, 2024-04-25T10:10:34.007 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 58, 2024-04-25T10:10:34.007 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 96, 2024-04-25T10:10:34.007 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 94, 2024-04-25T10:10:34.007 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 10.698991561 2024-04-25T10:10:34.007 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:10:34.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 -- 172.21.15.133:0/316183203 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7a8066f60 msgr2=0x7fb7a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7a8066f60 0x7fb7a8069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00018b0 tx=0x7fb7b0000ff0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 -- 172.21.15.133:0/316183203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 msgr2=0x7fb7c81eaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 0x7fb7c81eaea0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fb7c001d2b0 tx=0x7fb7c0019810 comp rx=0 tx=0).stop 2024-04-25T10:10:34.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 -- 172.21.15.133:0/316183203 shutdown_connections 2024-04-25T10:10:34.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7c8141660 0x7fb7c815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7c81422f0 0x7fb7c81e9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7a8066f60 0x7fb7a8069420 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 --2- 172.21.15.133:0/316183203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7b40039c0 0x7fb7c81eaea0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 -- 172.21.15.133:0/316183203 >> 172.21.15.133:0/316183203 conn(0x7fb7c8066f70 msgr2=0x7fb7c806d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:34.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 -- 172.21.15.133:0/316183203 shutdown_connections 2024-04-25T10:10:34.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:33.998+0000 7fb7cd50c640 1 -- 172.21.15.133:0/316183203 wait complete. 2024-04-25T10:10:34.017 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:10:34.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.114+0000 7f288cc40640 1 Processor -- start 2024-04-25T10:10:34.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.114+0000 7f288cc40640 1 -- start start 2024-04-25T10:10:34.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f288cc40640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f28880b1f70 0x7f288806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f288cc40640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28880ce330 0x7f28880ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f288cc40640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28880c61d0 con 0x7f288815c9a0 2024-04-25T10:10:34.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f288cc40640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28880cecc0 con 0x7f28880b1f70 2024-04-25T10:10:34.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f288cc40640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28880c64b0 con 0x7f28880ce330 2024-04-25T10:10:34.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f288659c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f288815c9a0 0x7f28880c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40194/0 (socket says 172.21.15.133:40194) 2024-04-25T10:10:34.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f288659c640 1 -- 172.21.15.133:0/3682413242 learned_addr learned my addr 172.21.15.133:0/3682413242 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:34.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559678605 0 0) 0x7f28880c61d0 con 0x7f288815c9a0 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2886d9d640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28880ce330 0x7f28880ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2874003530 con 0x7f288815c9a0 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f287dd9b640 1 --2- 172.21.15.133:0/3682413242 >> v2:172.21.15.162:3300/0 conn(0x7f28880b1f70 0x7f288806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4103451519 0 0) 0x7f2874003530 con 0x7f288815c9a0 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28880ce330 msgr2=0x7f28880ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28880ce330 0x7f28880ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 >> v2:172.21.15.162:3300/0 conn(0x7f28880b1f70 msgr2=0x7f288806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 --2- 172.21.15.133:0/3682413242 >> v2:172.21.15.162:3300/0 conn(0x7f28880b1f70 0x7f288806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288806d4f0 con 0x7f288815c9a0 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2878001680 con 0x7f288815c9a0 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 >> v1:172.21.15.133:6789/0 conn(0x7f288815c9a0 legacy=0x7f28880c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28740039c0 0x7f2874003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28880ce330 0x7f2874004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2874004d50 0x7f28740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28880cecc0 con 0x7f2874004d50 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28880c64b0 con 0x7f28740039c0 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287400ba00 con 0x7f28880ce330 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f287dd9b640 1 --2- 172.21.15.133:0/3682413242 >> v2:172.21.15.162:3300/0 conn(0x7f28880b1f70 0x7f288806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2886d9d640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2874004d50 0x7f28740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f288659c640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28740039c0 0x7f2874003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2886d9d640 1 -- 172.21.15.133:0/3682413242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28880ce330 msgr2=0x7f2874004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2886d9d640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28880ce330 0x7f2874004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2886d9d640 1 -- 172.21.15.133:0/3682413242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28740039c0 msgr2=0x7f2874003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2886d9d640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28740039c0 0x7f2874003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f2886d9d640 1 -- 172.21.15.133:0/3682413242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2874003530 con 0x7f2874004d50 2024-04-25T10:10:34.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.118+0000 7f288659c640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28740039c0 0x7f2874003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:34.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f2886d9d640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2874004d50 0x7f28740051d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2880009fc0 tx=0x7f288001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1530f46a2fa176 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:34.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28800b2070 con 0x7f2874004d50 2024-04-25T10:10:34.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28800a5cd0 con 0x7f2874004d50 2024-04-25T10:10:34.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f2885d9b640 1 -- 172.21.15.133:0/3682413242 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880009c10 con 0x7f2874004d50 2024-04-25T10:10:34.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 -- 172.21.15.133:0/3682413242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2874004d50 msgr2=0x7f28740051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 --2- 172.21.15.133:0/3682413242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2874004d50 0x7f28740051d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2880009fc0 tx=0x7f288001ccc0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f2886d9d640 1 -- 172.21.15.133:0/3682413242 reap_dead start 2024-04-25T10:10:34.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 -- 172.21.15.133:0/3682413242 shutdown_connections 2024-04-25T10:10:34.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 -- 172.21.15.133:0/3682413242 >> 172.21.15.133:0/3682413242 conn(0x7f2888067000 msgr2=0x7f288806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 -- 172.21.15.133:0/3682413242 shutdown_connections 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 -- 172.21.15.133:0/3682413242 wait complete. 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 Processor -- start 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 -- start start 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2888153990 0x7f28880b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2888156820 0x7f2888141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2888153d70 0x7f2888141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288815cd80 con 0x7f2888156820 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288806e230 con 0x7f2888153990 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288815cf00 con 0x7f2888153d70 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2888156820 0x7f2888141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2888156820 0x7f2888141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42862/0 (socket says 172.21.15.133:42862) 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287dd9b640 1 -- 172.21.15.133:0/2382424802 learned_addr learned my addr 172.21.15.133:0/2382424802 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f2886d9d640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2888153d70 0x7f2888141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288659c640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2888153990 0x7f28880b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287dd9b640 1 -- 172.21.15.133:0/2382424802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2888153d70 msgr2=0x7f2888141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287dd9b640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2888153d70 0x7f2888141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287dd9b640 1 -- 172.21.15.133:0/2382424802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2888153990 msgr2=0x7f28880b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287dd9b640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2888153990 0x7f28880b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287dd9b640 1 -- 172.21.15.133:0/2382424802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28880b2dd0 con 0x7f2888156820 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287dd9b640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2888156820 0x7f2888141870 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f287000d5b0 tx=0x7f287000da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287f7fe640 1 -- 172.21.15.133:0/2382424802 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870004020 con 0x7f2888156820 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287f7fe640 1 -- 172.21.15.133:0/2382424802 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28700041c0 con 0x7f2888156820 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 -- 172.21.15.133:0/2382424802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28880b30b0 con 0x7f2888156820 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f287f7fe640 1 -- 172.21.15.133:0/2382424802 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700043b0 con 0x7f2888156820 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.122+0000 7f288cc40640 1 -- 172.21.15.133:0/2382424802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28880b3670 con 0x7f2888156820 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.126+0000 7f288cc40640 1 -- 172.21.15.133:0/2382424802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2888068ef0 con 0x7f2888156820 2024-04-25T10:10:34.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.126+0000 7f287f7fe640 1 -- 172.21.15.133:0/2382424802 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f287001b020 con 0x7f2888156820 2024-04-25T10:10:34.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.126+0000 7f287f7fe640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2868066f60 0x7f2868069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.126+0000 7f287f7fe640 1 -- 172.21.15.133:0/2382424802 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f28700850d0 con 0x7f2888156820 2024-04-25T10:10:34.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.126+0000 7f288659c640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2868066f60 0x7f2868069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.126+0000 7f288659c640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2868066f60 0x7f2868069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2878001ec0 tx=0x7f2878009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:34.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.130+0000 7f287f7fe640 1 -- 172.21.15.133:0/2382424802 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2870055e50 con 0x7f2888156820 2024-04-25T10:10:34.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 -- 172.21.15.133:0/2382424802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f288815e680 con 0x7f2888156820 2024-04-25T10:10:34.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f287f7fe640 1 -- 172.21.15.133:0/2382424802 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2870055750 con 0x7f2888156820 2024-04-25T10:10:34.313 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:10:34.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 -- 172.21.15.133:0/2382424802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2868066f60 msgr2=0x7f2868069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2868066f60 0x7f2868069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2878001ec0 tx=0x7f2878009040 comp rx=0 tx=0).stop 2024-04-25T10:10:34.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 -- 172.21.15.133:0/2382424802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2888156820 msgr2=0x7f2888141870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2888156820 0x7f2888141870 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f287000d5b0 tx=0x7f287000da70 comp rx=0 tx=0).stop 2024-04-25T10:10:34.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 -- 172.21.15.133:0/2382424802 shutdown_connections 2024-04-25T10:10:34.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2888153d70 0x7f2888141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2868066f60 0x7f2868069420 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2888156820 0x7f2888141870 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 --2- 172.21.15.133:0/2382424802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2888153990 0x7f28880b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 -- 172.21.15.133:0/2382424802 >> 172.21.15.133:0/2382424802 conn(0x7f2888067000 msgr2=0x7f288815d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:34.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 -- 172.21.15.133:0/2382424802 shutdown_connections 2024-04-25T10:10:34.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:34.310+0000 7f288cc40640 1 -- 172.21.15.133:0/2382424802 wait complete. 2024-04-25T10:10:34.329 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:10:34.329 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:10:34.329 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:10:34.329 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:10:34.329 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:34.329 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:10:34.329 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:10:34.330 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:10:34.330 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:34.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:34.330 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:10:34.336 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:10:34.336 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:10:34.337 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:10:34.337 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:34.337 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:10:34.337 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:34.337 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:10:34.337 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:34.337 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:10:34.337 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:10:34.337 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:34.337 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:10:34.337 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41997sec preferred_lft 41997sec 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:10:34.338 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:10:34.339 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:34.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:34.340 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:10:34.390 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:10:34.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:34.391 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:10:34.444 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:10:34.444 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:10:34.444 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:10:34.444 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:10:34.444 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:10:34.444 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:10:34.444 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:10:34.445 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:10:34.445 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:10:34.445 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:10:34.445 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:10:34.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:34.445 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:10:34.445 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:34.445 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:10:34.445 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:10:34.446 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:10:34.446 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:34.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:34.518 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:10:34.564 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:10:34.564 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:10:34.564 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:34.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:34.564 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:10:34.621 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:10:34.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:34.621 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:34.668 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:34.668 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:34.669 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:34.716 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:10:34.774 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:10:34.822 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:10:34.822 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:10:34.822 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:10:34.822 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:10:34.822 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:10:34.822 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:10:34.822 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:10:34.822 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:10:34.823 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:10:34.824 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:10:34.824 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:10:34.824 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:10:34.824 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:10:34.824 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:10:34.824 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:10:34.824 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:34.824 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:34.824 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:34.824 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:34.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:34.824 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:10:34.868 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:10:34.868 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:10:34.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:10:34.930 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:10:34.981 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:10:34.981 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:10:34.981 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:10:34.981 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:10:34.981 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:10:34.981 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:10:34.981 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:10:34.981 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:10:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:34.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:34.984 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:10:34.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:10:34.990+0000 7fee2ec70580 -1 init, newargv = 0x55aacb002da0 newargc=15 2024-04-25T10:10:34.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[72094]: starting ceph client 2024-04-25T10:10:34.999 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[72094]: starting fuse 2024-04-25T10:10:36.098 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:10:36.114 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:10:36.165 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:10:36.165 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:10:36.165 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:10:36.165 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:10:36.166 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:10:36.166 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:10:36.166 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:10:36.166 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:10:36.166 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:10:36.166 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:10:36.166 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:10:36.166 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:10:36.166 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:10:36.166 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:10:36.167 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:10:36.167 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:10:36.167 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:10:36.167 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:10:36.167 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:10:36.167 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:10:36.167 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:10:36.167 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:10:36.167 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:10:36.167 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:10:36.168 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:10:36.168 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:10:36.168 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:10:36.168 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:10:36.168 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:10:36.168 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:36.168 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:36.168 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:36.168 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:36.168 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:36.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:36.170 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:10:36.216 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:10:36.217 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:10:36.217 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:10:36.217 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:10:36.217 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:10:36.217 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:10:36.217 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:10:36.217 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:10:36.217 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:10:36.217 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:10:36.217 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:10:36.217 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:10:36.217 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:10:36.218 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:10:36.218 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:10:36.218 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:10:36.218 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:10:36.218 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:10:36.218 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:10:36.218 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:10:36.218 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:10:36.218 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:10:36.218 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:10:36.218 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:10:36.219 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:10:36.219 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:10:36.219 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:10:36.219 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:10:36.219 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:10:36.343 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.72094.asok 2024-04-25T10:10:36.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:10:36.355 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.72094.asok 2024-04-25T10:10:36.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:36.355 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.72094.asok status 2024-04-25T10:10:36.513 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:10:36.513 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:10:36.513 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:10:36.513 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "72094", 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 8037, 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:10:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:10:36.515 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 8037 2024-04-25T10:10:36.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:36.515 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:10:36.515 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:10:36.515 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:10:36.515 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3829247880 2024-04-25T10:10:36.515 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:10:36.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:36.515 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:10:36.515 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:10:36.515 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:10:36.516 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3829247880 2024-04-25T10:10:36.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:10:36.516 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.8037 192.168.0.1:0/3829247880", 2024-04-25T10:10:36.516 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/3829247880", 2024-04-25T10:10:36.516 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:10:36.516 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 58, 2024-04-25T10:10:36.516 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 96, 2024-04-25T10:10:36.516 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:10:36.516 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:10:36.517 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:10:36.517 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:10:36.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:36.527 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:10:36.527 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:36.534 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:10:36.534 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:36.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:36.535 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:36.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:36.589 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:36.589 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:36.589 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:10:36.589 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:36.589 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:36.589 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:36.589 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:36.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.680 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183999 s, 57.0 MB/s 2024-04-25T10:10:36.694 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.694 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.694 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116965 s, 89.6 MB/s 2024-04-25T10:10:36.714 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.714 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.714 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166239 s, 63.1 MB/s 2024-04-25T10:10:36.727 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.727 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.727 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101753 s, 103 MB/s 2024-04-25T10:10:36.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.741 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114104 s, 91.9 MB/s 2024-04-25T10:10:36.757 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135996 s, 77.1 MB/s 2024-04-25T10:10:36.773 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.773 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.773 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125166 s, 83.8 MB/s 2024-04-25T10:10:36.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.791 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144264 s, 72.7 MB/s 2024-04-25T10:10:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.809 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.810 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162657 s, 64.5 MB/s 2024-04-25T10:10:36.827 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.827 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.828 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144475 s, 72.6 MB/s 2024-04-25T10:10:36.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.842 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116263 s, 90.2 MB/s 2024-04-25T10:10:36.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.859 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128163 s, 81.8 MB/s 2024-04-25T10:10:36.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.873 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122456 s, 85.6 MB/s 2024-04-25T10:10:36.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.890 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135711 s, 77.3 MB/s 2024-04-25T10:10:36.905 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.905 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.905 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124791 s, 84.0 MB/s 2024-04-25T10:10:36.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.920 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011916 s, 88.0 MB/s 2024-04-25T10:10:36.938 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.939 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.939 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014709 s, 71.3 MB/s 2024-04-25T10:10:36.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.960 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170644 s, 61.4 MB/s 2024-04-25T10:10:36.979 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.979 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.979 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158273 s, 66.3 MB/s 2024-04-25T10:10:36.996 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:36.996 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:36.996 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136528 s, 76.8 MB/s 2024-04-25T10:10:37.014 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.014 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.014 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147402 s, 71.1 MB/s 2024-04-25T10:10:37.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.030 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131844 s, 79.5 MB/s 2024-04-25T10:10:37.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.050 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157647 s, 66.5 MB/s 2024-04-25T10:10:37.067 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.067 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.068 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148066 s, 70.8 MB/s 2024-04-25T10:10:37.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.084 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127173 s, 82.5 MB/s 2024-04-25T10:10:37.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.099 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119892 s, 87.5 MB/s 2024-04-25T10:10:37.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.117 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147731 s, 71.0 MB/s 2024-04-25T10:10:37.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124983 s, 83.9 MB/s 2024-04-25T10:10:37.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.149 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135428 s, 77.4 MB/s 2024-04-25T10:10:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129778 s, 80.8 MB/s 2024-04-25T10:10:37.182 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.182 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.183 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143672 s, 73.0 MB/s 2024-04-25T10:10:37.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.201 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156034 s, 67.2 MB/s 2024-04-25T10:10:37.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:37.203 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:37.203 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:37.203 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:10:37.203 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:37.203 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:37.203 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:37.203 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:37.226 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985791 s, 106 MB/s 2024-04-25T10:10:37.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.242 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.242 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012693 s, 82.6 MB/s 2024-04-25T10:10:37.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.257 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125514 s, 83.5 MB/s 2024-04-25T10:10:37.274 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.274 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.274 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139988 s, 74.9 MB/s 2024-04-25T10:10:37.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.290 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136207 s, 77.0 MB/s 2024-04-25T10:10:37.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.305 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011404 s, 91.9 MB/s 2024-04-25T10:10:37.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.318 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106528 s, 98.4 MB/s 2024-04-25T10:10:37.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.332 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112141 s, 93.5 MB/s 2024-04-25T10:10:37.347 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.347 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.347 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133635 s, 78.5 MB/s 2024-04-25T10:10:37.363 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133119 s, 78.8 MB/s 2024-04-25T10:10:37.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.381 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125805 s, 83.3 MB/s 2024-04-25T10:10:37.393 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.394 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.394 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119046 s, 88.1 MB/s 2024-04-25T10:10:37.408 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.408 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.408 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117015 s, 89.6 MB/s 2024-04-25T10:10:37.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.421 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010163 s, 103 MB/s 2024-04-25T10:10:37.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.435 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113025 s, 92.8 MB/s 2024-04-25T10:10:37.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.449 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104254 s, 101 MB/s 2024-04-25T10:10:37.461 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.461 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.461 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937588 s, 112 MB/s 2024-04-25T10:10:37.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.473 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939896 s, 112 MB/s 2024-04-25T10:10:37.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.488 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129501 s, 81.0 MB/s 2024-04-25T10:10:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112822 s, 92.9 MB/s 2024-04-25T10:10:37.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.517 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119823 s, 87.5 MB/s 2024-04-25T10:10:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992624 s, 106 MB/s 2024-04-25T10:10:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.546 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131024 s, 80.0 MB/s 2024-04-25T10:10:37.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.559 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112901 s, 92.9 MB/s 2024-04-25T10:10:37.574 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.574 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.575 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123238 s, 85.1 MB/s 2024-04-25T10:10:37.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.590 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130507 s, 80.3 MB/s 2024-04-25T10:10:37.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.605 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125729 s, 83.4 MB/s 2024-04-25T10:10:37.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.618 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103921 s, 101 MB/s 2024-04-25T10:10:37.632 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.632 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.632 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114729 s, 91.4 MB/s 2024-04-25T10:10:37.647 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.647 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.647 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121457 s, 86.3 MB/s 2024-04-25T10:10:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.664 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133834 s, 78.3 MB/s 2024-04-25T10:10:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112669 s, 93.1 MB/s 2024-04-25T10:10:37.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:37.678 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:37.678 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:37.679 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:10:37.679 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:37.679 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:37.679 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:37.679 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:37.743 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120319 s, 87.1 MB/s 2024-04-25T10:10:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115981 s, 90.4 MB/s 2024-04-25T10:10:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829489 s, 126 MB/s 2024-04-25T10:10:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128511 s, 81.6 MB/s 2024-04-25T10:10:37.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.797 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114527 s, 91.6 MB/s 2024-04-25T10:10:37.812 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.813 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.813 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127789 s, 82.1 MB/s 2024-04-25T10:10:37.827 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.827 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.827 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118467 s, 88.5 MB/s 2024-04-25T10:10:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115958 s, 90.4 MB/s 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117955 s, 88.9 MB/s 2024-04-25T10:10:37.868 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.868 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.868 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110462 s, 94.9 MB/s 2024-04-25T10:10:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101989 s, 103 MB/s 2024-04-25T10:10:37.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.894 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110272 s, 95.1 MB/s 2024-04-25T10:10:37.903 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.904 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765407 s, 137 MB/s 2024-04-25T10:10:37.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.916 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996584 s, 105 MB/s 2024-04-25T10:10:37.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.927 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903762 s, 116 MB/s 2024-04-25T10:10:37.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.941 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.941 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108962 s, 96.2 MB/s 2024-04-25T10:10:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.954 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111783 s, 93.8 MB/s 2024-04-25T10:10:37.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.969 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127017 s, 82.6 MB/s 2024-04-25T10:10:37.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.983 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117383 s, 89.3 MB/s 2024-04-25T10:10:37.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:37.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:37.998 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012669 s, 82.8 MB/s 2024-04-25T10:10:38.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012048 s, 87.0 MB/s 2024-04-25T10:10:38.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.027 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116914 s, 89.7 MB/s 2024-04-25T10:10:38.041 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.042 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.042 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121976 s, 86.0 MB/s 2024-04-25T10:10:38.057 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.058 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132022 s, 79.4 MB/s 2024-04-25T10:10:38.069 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.069 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.069 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990885 s, 106 MB/s 2024-04-25T10:10:38.080 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.081 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.081 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886587 s, 118 MB/s 2024-04-25T10:10:38.097 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.097 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.097 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136752 s, 76.7 MB/s 2024-04-25T10:10:38.111 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.111 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.111 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122998 s, 85.3 MB/s 2024-04-25T10:10:38.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.124 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103771 s, 101 MB/s 2024-04-25T10:10:38.136 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.136 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.136 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973819 s, 108 MB/s 2024-04-25T10:10:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010044 s, 104 MB/s 2024-04-25T10:10:38.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.163 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.163 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110384 s, 95.0 MB/s 2024-04-25T10:10:38.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:38.163 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:38.163 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:38.163 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:10:38.163 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:38.163 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:38.163 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:38.163 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:38.223 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.223 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.223 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850652 s, 123 MB/s 2024-04-25T10:10:38.236 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.236 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.237 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112731 s, 93.0 MB/s 2024-04-25T10:10:38.247 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.247 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.247 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838371 s, 125 MB/s 2024-04-25T10:10:38.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.261 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122688 s, 85.5 MB/s 2024-04-25T10:10:38.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.275 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116066 s, 90.3 MB/s 2024-04-25T10:10:38.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.290 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122479 s, 85.6 MB/s 2024-04-25T10:10:38.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.303 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108043 s, 97.1 MB/s 2024-04-25T10:10:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.319 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132184 s, 79.3 MB/s 2024-04-25T10:10:38.331 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.331 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.331 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981309 s, 107 MB/s 2024-04-25T10:10:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132071 s, 79.4 MB/s 2024-04-25T10:10:38.359 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.359 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.359 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102345 s, 102 MB/s 2024-04-25T10:10:38.372 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.372 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.372 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109238 s, 96.0 MB/s 2024-04-25T10:10:38.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.384 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875741 s, 120 MB/s 2024-04-25T10:10:38.395 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.395 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.395 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900012 s, 117 MB/s 2024-04-25T10:10:38.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.409 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117623 s, 89.1 MB/s 2024-04-25T10:10:38.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.422 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109364 s, 95.9 MB/s 2024-04-25T10:10:38.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.437 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.437 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121492 s, 86.3 MB/s 2024-04-25T10:10:38.452 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.452 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.452 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125986 s, 83.2 MB/s 2024-04-25T10:10:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092399 s, 113 MB/s 2024-04-25T10:10:38.476 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.476 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.476 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912296 s, 115 MB/s 2024-04-25T10:10:38.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.488 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992766 s, 106 MB/s 2024-04-25T10:10:38.502 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.502 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.503 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117556 s, 89.2 MB/s 2024-04-25T10:10:38.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.517 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118499 s, 88.5 MB/s 2024-04-25T10:10:38.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.532 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.532 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119824 s, 87.5 MB/s 2024-04-25T10:10:38.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.546 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119729 s, 87.6 MB/s 2024-04-25T10:10:38.557 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.557 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.558 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885288 s, 118 MB/s 2024-04-25T10:10:38.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.570 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979386 s, 107 MB/s 2024-04-25T10:10:38.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.583 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109447 s, 95.8 MB/s 2024-04-25T10:10:38.600 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.601 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.601 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144816 s, 72.4 MB/s 2024-04-25T10:10:38.614 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.614 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.614 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115039 s, 91.1 MB/s 2024-04-25T10:10:38.627 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.627 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.627 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100929 s, 104 MB/s 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116965 s, 89.6 MB/s 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:38.642 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:38.642 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:38.643 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:10:38.643 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:38.643 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:38.643 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:38.643 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:38.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.709 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129896 s, 80.7 MB/s 2024-04-25T10:10:38.725 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.725 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.725 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136272 s, 76.9 MB/s 2024-04-25T10:10:38.743 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.743 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.743 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143212 s, 73.2 MB/s 2024-04-25T10:10:38.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.761 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138154 s, 75.9 MB/s 2024-04-25T10:10:38.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.780 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153449 s, 68.3 MB/s 2024-04-25T10:10:38.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.797 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148924 s, 70.4 MB/s 2024-04-25T10:10:38.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.814 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140564 s, 74.6 MB/s 2024-04-25T10:10:38.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.832 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015034 s, 69.7 MB/s 2024-04-25T10:10:38.847 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.847 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.847 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116213 s, 90.2 MB/s 2024-04-25T10:10:38.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.861 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114876 s, 91.3 MB/s 2024-04-25T10:10:38.877 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.877 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.877 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126796 s, 82.7 MB/s 2024-04-25T10:10:38.893 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.893 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.893 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136611 s, 76.8 MB/s 2024-04-25T10:10:38.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.909 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126424 s, 82.9 MB/s 2024-04-25T10:10:38.923 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.923 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.923 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111004 s, 94.5 MB/s 2024-04-25T10:10:38.938 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.938 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.938 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012258 s, 85.5 MB/s 2024-04-25T10:10:38.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.953 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114998 s, 91.2 MB/s 2024-04-25T10:10:38.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.969 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114831 s, 91.3 MB/s 2024-04-25T10:10:38.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.984 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111988 s, 93.6 MB/s 2024-04-25T10:10:38.997 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:38.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:38.998 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108144 s, 97.0 MB/s 2024-04-25T10:10:39.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.017 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147741 s, 71.0 MB/s 2024-04-25T10:10:39.035 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.035 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.035 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154193 s, 68.0 MB/s 2024-04-25T10:10:39.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.053 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144781 s, 72.4 MB/s 2024-04-25T10:10:39.069 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.069 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.070 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013642 s, 76.9 MB/s 2024-04-25T10:10:39.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.086 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138624 s, 75.6 MB/s 2024-04-25T10:10:39.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.103 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131126 s, 80.0 MB/s 2024-04-25T10:10:39.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.118 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127521 s, 82.2 MB/s 2024-04-25T10:10:39.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.131 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102912 s, 102 MB/s 2024-04-25T10:10:39.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.147 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125412 s, 83.6 MB/s 2024-04-25T10:10:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.163 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133321 s, 78.7 MB/s 2024-04-25T10:10:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.180 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130473 s, 80.4 MB/s 2024-04-25T10:10:39.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.196 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138063 s, 75.9 MB/s 2024-04-25T10:10:39.212 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.212 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.212 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012731 s, 82.4 MB/s 2024-04-25T10:10:39.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:39.213 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:39.213 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:39.213 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:10:39.213 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:39.213 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:39.213 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:39.213 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:39.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.287 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120207 s, 87.2 MB/s 2024-04-25T10:10:39.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.303 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133691 s, 78.4 MB/s 2024-04-25T10:10:39.319 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.319 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.320 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135375 s, 77.5 MB/s 2024-04-25T10:10:39.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.335 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129024 s, 81.3 MB/s 2024-04-25T10:10:39.353 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.353 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.353 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135247 s, 77.5 MB/s 2024-04-25T10:10:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.367 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126851 s, 82.7 MB/s 2024-04-25T10:10:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.382 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117129 s, 89.5 MB/s 2024-04-25T10:10:39.399 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.399 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.400 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141008 s, 74.4 MB/s 2024-04-25T10:10:39.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.415 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125871 s, 83.3 MB/s 2024-04-25T10:10:39.432 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.432 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.432 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135131 s, 77.6 MB/s 2024-04-25T10:10:39.447 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.448 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.448 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129403 s, 81.0 MB/s 2024-04-25T10:10:39.461 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110757 s, 94.7 MB/s 2024-04-25T10:10:39.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.479 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141028 s, 74.4 MB/s 2024-04-25T10:10:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.495 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130709 s, 80.2 MB/s 2024-04-25T10:10:39.510 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.510 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.510 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124452 s, 84.3 MB/s 2024-04-25T10:10:39.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.525 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114301 s, 91.7 MB/s 2024-04-25T10:10:39.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.541 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131464 s, 79.8 MB/s 2024-04-25T10:10:39.558 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.558 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.558 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135777 s, 77.2 MB/s 2024-04-25T10:10:39.571 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.571 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.571 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105753 s, 99.2 MB/s 2024-04-25T10:10:39.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.584 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938271 s, 112 MB/s 2024-04-25T10:10:39.598 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.598 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.598 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119457 s, 87.8 MB/s 2024-04-25T10:10:39.610 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.610 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.610 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901211 s, 116 MB/s 2024-04-25T10:10:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.626 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130489 s, 80.4 MB/s 2024-04-25T10:10:39.641 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.641 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.641 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126202 s, 83.1 MB/s 2024-04-25T10:10:39.658 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.658 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.658 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139416 s, 75.2 MB/s 2024-04-25T10:10:39.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.675 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130952 s, 80.1 MB/s 2024-04-25T10:10:39.691 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.691 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.691 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012485 s, 84.0 MB/s 2024-04-25T10:10:39.711 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.711 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.711 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155221 s, 67.6 MB/s 2024-04-25T10:10:39.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.729 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141371 s, 74.2 MB/s 2024-04-25T10:10:39.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.747 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136919 s, 76.6 MB/s 2024-04-25T10:10:39.765 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.765 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.765 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135494 s, 77.4 MB/s 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120672 s, 86.9 MB/s 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:39.783 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:39.783 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:39.783 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:10:39.783 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:39.783 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:39.783 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:39.783 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:39.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.849 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126716 s, 82.8 MB/s 2024-04-25T10:10:39.862 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.862 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.862 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103268 s, 102 MB/s 2024-04-25T10:10:39.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.875 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985623 s, 106 MB/s 2024-04-25T10:10:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126792 s, 82.7 MB/s 2024-04-25T10:10:39.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.907 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132616 s, 79.1 MB/s 2024-04-25T10:10:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117395 s, 89.3 MB/s 2024-04-25T10:10:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134832 s, 77.8 MB/s 2024-04-25T10:10:39.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.952 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111312 s, 94.2 MB/s 2024-04-25T10:10:39.966 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.966 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.967 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122352 s, 85.7 MB/s 2024-04-25T10:10:39.980 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.980 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.980 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104662 s, 100 MB/s 2024-04-25T10:10:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123305 s, 85.0 MB/s 2024-04-25T10:10:40.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151902 s, 69.0 MB/s 2024-04-25T10:10:40.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.027 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107129 s, 97.9 MB/s 2024-04-25T10:10:40.041 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.041 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.041 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117579 s, 89.2 MB/s 2024-04-25T10:10:40.056 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.057 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.057 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125839 s, 83.3 MB/s 2024-04-25T10:10:40.072 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.072 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.072 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120587 s, 87.0 MB/s 2024-04-25T10:10:40.085 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.085 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.085 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109365 s, 95.9 MB/s 2024-04-25T10:10:40.097 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.097 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.097 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891667 s, 118 MB/s 2024-04-25T10:10:40.112 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.112 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.112 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123939 s, 84.6 MB/s 2024-04-25T10:10:40.123 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.123 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.123 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874363 s, 120 MB/s 2024-04-25T10:10:40.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.138 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012781 s, 82.0 MB/s 2024-04-25T10:10:40.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893949 s, 117 MB/s 2024-04-25T10:10:40.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.163 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104054 s, 101 MB/s 2024-04-25T10:10:40.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.174 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842792 s, 124 MB/s 2024-04-25T10:10:40.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.186 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911114 s, 115 MB/s 2024-04-25T10:10:40.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.199 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108605 s, 96.5 MB/s 2024-04-25T10:10:40.213 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.213 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.213 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111433 s, 94.1 MB/s 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129237 s, 81.1 MB/s 2024-04-25T10:10:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113607 s, 92.3 MB/s 2024-04-25T10:10:40.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.255 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107606 s, 97.4 MB/s 2024-04-25T10:10:40.267 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.267 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.267 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888574 s, 118 MB/s 2024-04-25T10:10:40.280 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.281 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.281 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114898 s, 91.3 MB/s 2024-04-25T10:10:40.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:40.282 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:40.282 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:40.282 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:10:40.282 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:40.282 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:40.282 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:40.282 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.353 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132946 s, 78.9 MB/s 2024-04-25T10:10:40.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.367 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113729 s, 92.2 MB/s 2024-04-25T10:10:40.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.379 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094133 s, 111 MB/s 2024-04-25T10:10:40.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.392 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963982 s, 109 MB/s 2024-04-25T10:10:40.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.407 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125259 s, 83.7 MB/s 2024-04-25T10:10:40.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.423 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.423 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127186 s, 82.4 MB/s 2024-04-25T10:10:40.437 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.437 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.437 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118522 s, 88.5 MB/s 2024-04-25T10:10:40.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.450 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940975 s, 111 MB/s 2024-04-25T10:10:40.460 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.461 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.461 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859426 s, 122 MB/s 2024-04-25T10:10:40.475 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.475 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.475 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115663 s, 90.7 MB/s 2024-04-25T10:10:40.490 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.490 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.490 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128478 s, 81.6 MB/s 2024-04-25T10:10:40.502 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.502 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.502 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918266 s, 114 MB/s 2024-04-25T10:10:40.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.516 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112922 s, 92.9 MB/s 2024-04-25T10:10:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131032 s, 80.0 MB/s 2024-04-25T10:10:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.545 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112888 s, 92.9 MB/s 2024-04-25T10:10:40.558 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.559 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011029 s, 95.1 MB/s 2024-04-25T10:10:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.573 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.573 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112297 s, 93.4 MB/s 2024-04-25T10:10:40.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.583 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828242 s, 127 MB/s 2024-04-25T10:10:40.597 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.597 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.597 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109705 s, 95.6 MB/s 2024-04-25T10:10:40.610 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.610 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.611 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108806 s, 96.4 MB/s 2024-04-25T10:10:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123542 s, 84.9 MB/s 2024-04-25T10:10:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103647 s, 101 MB/s 2024-04-25T10:10:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.654 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115306 s, 90.9 MB/s 2024-04-25T10:10:40.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.667 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110559 s, 94.8 MB/s 2024-04-25T10:10:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116638 s, 89.9 MB/s 2024-04-25T10:10:40.693 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.693 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.693 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891224 s, 118 MB/s 2024-04-25T10:10:40.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117379 s, 89.3 MB/s 2024-04-25T10:10:40.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.721 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106442 s, 98.5 MB/s 2024-04-25T10:10:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.733 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0097092 s, 108 MB/s 2024-04-25T10:10:40.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.748 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123906 s, 84.6 MB/s 2024-04-25T10:10:40.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.761 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102717 s, 102 MB/s 2024-04-25T10:10:40.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.775 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109676 s, 95.6 MB/s 2024-04-25T10:10:40.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:40.776 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:40.776 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:40.776 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:10:40.776 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:40.776 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:40.776 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:40.776 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:40.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.842 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109948 s, 95.4 MB/s 2024-04-25T10:10:40.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117929 s, 88.9 MB/s 2024-04-25T10:10:40.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.871 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111601 s, 94.0 MB/s 2024-04-25T10:10:40.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.886 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127052 s, 82.5 MB/s 2024-04-25T10:10:40.897 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.898 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924981 s, 113 MB/s 2024-04-25T10:10:40.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.912 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115163 s, 91.1 MB/s 2024-04-25T10:10:40.924 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.924 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.924 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944889 s, 111 MB/s 2024-04-25T10:10:40.939 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.939 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.939 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118522 s, 88.5 MB/s 2024-04-25T10:10:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.953 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114479 s, 91.6 MB/s 2024-04-25T10:10:40.965 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.965 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.965 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927024 s, 113 MB/s 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01195 s, 87.7 MB/s 2024-04-25T10:10:40.994 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:40.994 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:40.994 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116056 s, 90.4 MB/s 2024-04-25T10:10:41.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.007 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010667 s, 98.3 MB/s 2024-04-25T10:10:41.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.022 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125342 s, 83.7 MB/s 2024-04-25T10:10:41.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.037 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123818 s, 84.7 MB/s 2024-04-25T10:10:41.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.051 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109111 s, 96.1 MB/s 2024-04-25T10:10:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.062 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856932 s, 122 MB/s 2024-04-25T10:10:41.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.077 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129901 s, 80.7 MB/s 2024-04-25T10:10:41.090 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.090 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.090 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106081 s, 98.8 MB/s 2024-04-25T10:10:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109917 s, 95.4 MB/s 2024-04-25T10:10:41.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.118 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117093 s, 89.6 MB/s 2024-04-25T10:10:41.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.132 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112515 s, 93.2 MB/s 2024-04-25T10:10:41.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.147 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123901 s, 84.6 MB/s 2024-04-25T10:10:41.161 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.161 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.161 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115987 s, 90.4 MB/s 2024-04-25T10:10:41.177 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.177 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.177 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132052 s, 79.4 MB/s 2024-04-25T10:10:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115793 s, 90.6 MB/s 2024-04-25T10:10:41.206 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.206 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.206 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126019 s, 83.2 MB/s 2024-04-25T10:10:41.221 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.221 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.221 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119458 s, 87.8 MB/s 2024-04-25T10:10:41.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.232 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865053 s, 121 MB/s 2024-04-25T10:10:41.247 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.247 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.247 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123614 s, 84.8 MB/s 2024-04-25T10:10:41.260 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.261 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109094 s, 96.1 MB/s 2024-04-25T10:10:41.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.275 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101823 s, 103 MB/s 2024-04-25T10:10:41.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:41.276 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:41.276 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:41.276 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:10:41.276 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:41.276 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:41.276 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:41.276 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:41.344 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.344 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.344 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117325 s, 89.4 MB/s 2024-04-25T10:10:41.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.357 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103547 s, 101 MB/s 2024-04-25T10:10:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.371 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102306 s, 102 MB/s 2024-04-25T10:10:41.382 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.383 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911868 s, 115 MB/s 2024-04-25T10:10:41.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.398 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.398 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119702 s, 87.6 MB/s 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127112 s, 82.5 MB/s 2024-04-25T10:10:41.429 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.430 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.430 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140256 s, 74.8 MB/s 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114678 s, 91.4 MB/s 2024-04-25T10:10:41.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.462 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134419 s, 78.0 MB/s 2024-04-25T10:10:41.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.479 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133431 s, 78.6 MB/s 2024-04-25T10:10:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.493 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111824 s, 93.8 MB/s 2024-04-25T10:10:41.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997941 s, 105 MB/s 2024-04-25T10:10:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.520 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110383 s, 95.0 MB/s 2024-04-25T10:10:41.534 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.535 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.535 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012186 s, 86.0 MB/s 2024-04-25T10:10:41.548 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.548 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.548 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107279 s, 97.7 MB/s 2024-04-25T10:10:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.562 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116426 s, 90.1 MB/s 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126069 s, 83.2 MB/s 2024-04-25T10:10:41.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.592 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112645 s, 93.1 MB/s 2024-04-25T10:10:41.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.607 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127203 s, 82.4 MB/s 2024-04-25T10:10:41.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.622 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127555 s, 82.2 MB/s 2024-04-25T10:10:41.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.637 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119258 s, 87.9 MB/s 2024-04-25T10:10:41.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.651 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115848 s, 90.5 MB/s 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100021 s, 105 MB/s 2024-04-25T10:10:41.677 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108405 s, 96.7 MB/s 2024-04-25T10:10:41.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.692 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111167 s, 94.3 MB/s 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960563 s, 109 MB/s 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.720 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011115 s, 94.3 MB/s 2024-04-25T10:10:41.736 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.737 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.737 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127705 s, 82.1 MB/s 2024-04-25T10:10:41.755 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.756 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.756 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148794 s, 70.5 MB/s 2024-04-25T10:10:41.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.769 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991321 s, 106 MB/s 2024-04-25T10:10:41.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.785 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118605 s, 88.4 MB/s 2024-04-25T10:10:41.801 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.801 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.801 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133963 s, 78.3 MB/s 2024-04-25T10:10:41.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:41.802 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:41.802 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:41.802 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:10:41.802 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:41.802 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:41.803 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:41.803 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:41.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.871 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132728 s, 79.0 MB/s 2024-04-25T10:10:41.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.886 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105278 s, 99.6 MB/s 2024-04-25T10:10:41.902 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.902 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.903 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139309 s, 75.3 MB/s 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119902 s, 87.5 MB/s 2024-04-25T10:10:41.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.933 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124437 s, 84.3 MB/s 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156747 s, 66.9 MB/s 2024-04-25T10:10:41.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.969 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134938 s, 77.7 MB/s 2024-04-25T10:10:41.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:41.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:41.984 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122352 s, 85.7 MB/s 2024-04-25T10:10:42.001 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.001 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.001 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141845 s, 73.9 MB/s 2024-04-25T10:10:42.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.020 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969376 s, 108 MB/s 2024-04-25T10:10:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100752 s, 104 MB/s 2024-04-25T10:10:42.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.049 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013271 s, 79.0 MB/s 2024-04-25T10:10:42.061 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.062 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100856 s, 104 MB/s 2024-04-25T10:10:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.079 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.079 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131617 s, 79.7 MB/s 2024-04-25T10:10:42.093 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.093 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.093 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012123 s, 86.5 MB/s 2024-04-25T10:10:42.106 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.106 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.106 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100535 s, 104 MB/s 2024-04-25T10:10:42.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.118 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932591 s, 112 MB/s 2024-04-25T10:10:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.129 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844583 s, 124 MB/s 2024-04-25T10:10:42.144 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.145 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.145 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129801 s, 80.8 MB/s 2024-04-25T10:10:42.156 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.156 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.156 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904692 s, 116 MB/s 2024-04-25T10:10:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879746 s, 119 MB/s 2024-04-25T10:10:42.181 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.181 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.181 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107192 s, 97.8 MB/s 2024-04-25T10:10:42.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.194 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109196 s, 96.0 MB/s 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.205 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839194 s, 125 MB/s 2024-04-25T10:10:42.220 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.220 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.220 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120645 s, 86.9 MB/s 2024-04-25T10:10:42.234 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.234 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.234 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011707 s, 89.6 MB/s 2024-04-25T10:10:42.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.246 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922832 s, 114 MB/s 2024-04-25T10:10:42.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.259 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101787 s, 103 MB/s 2024-04-25T10:10:42.274 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.275 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123794 s, 84.7 MB/s 2024-04-25T10:10:42.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.290 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012423 s, 84.4 MB/s 2024-04-25T10:10:42.302 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.302 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.302 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967075 s, 108 MB/s 2024-04-25T10:10:42.316 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.316 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.316 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113274 s, 92.6 MB/s 2024-04-25T10:10:42.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:42.318 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:42.318 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:42.318 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:10:42.318 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:42.318 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:42.318 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:42.318 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:42.388 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.388 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.388 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122402 s, 85.7 MB/s 2024-04-25T10:10:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107431 s, 97.6 MB/s 2024-04-25T10:10:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107486 s, 97.6 MB/s 2024-04-25T10:10:42.428 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.428 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.428 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110234 s, 95.1 MB/s 2024-04-25T10:10:42.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.442 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116053 s, 90.4 MB/s 2024-04-25T10:10:42.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.457 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118261 s, 88.7 MB/s 2024-04-25T10:10:42.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.468 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841879 s, 125 MB/s 2024-04-25T10:10:42.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.480 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981403 s, 107 MB/s 2024-04-25T10:10:42.499 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.499 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.499 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159565 s, 65.7 MB/s 2024-04-25T10:10:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012041 s, 87.1 MB/s 2024-04-25T10:10:42.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.525 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852923 s, 123 MB/s 2024-04-25T10:10:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129423 s, 81.0 MB/s 2024-04-25T10:10:42.553 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.553 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.553 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010208 s, 103 MB/s 2024-04-25T10:10:42.565 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.566 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.566 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0096407 s, 109 MB/s 2024-04-25T10:10:42.576 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.576 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.576 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836413 s, 125 MB/s 2024-04-25T10:10:42.588 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.588 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.588 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949075 s, 110 MB/s 2024-04-25T10:10:42.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.602 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112902 s, 92.9 MB/s 2024-04-25T10:10:42.617 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.617 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.617 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117935 s, 88.9 MB/s 2024-04-25T10:10:42.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.630 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011109 s, 94.4 MB/s 2024-04-25T10:10:42.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.644 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.644 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105872 s, 99.0 MB/s 2024-04-25T10:10:42.657 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.657 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.657 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110701 s, 94.7 MB/s 2024-04-25T10:10:42.668 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.668 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.668 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008323 s, 126 MB/s 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.681 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099191 s, 106 MB/s 2024-04-25T10:10:42.694 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.694 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.694 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111011 s, 94.5 MB/s 2024-04-25T10:10:42.708 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.709 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111951 s, 93.7 MB/s 2024-04-25T10:10:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115848 s, 90.5 MB/s 2024-04-25T10:10:42.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.735 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.735 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933883 s, 112 MB/s 2024-04-25T10:10:42.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.748 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110712 s, 94.7 MB/s 2024-04-25T10:10:42.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.763 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128836 s, 81.4 MB/s 2024-04-25T10:10:42.780 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.780 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.780 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136653 s, 76.7 MB/s 2024-04-25T10:10:42.793 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.793 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.793 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103416 s, 101 MB/s 2024-04-25T10:10:42.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.808 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.808 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115885 s, 90.5 MB/s 2024-04-25T10:10:42.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:42.809 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:42.809 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:42.809 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:10:42.809 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:42.809 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:42.809 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:42.809 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:42.879 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.879 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.879 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010589 s, 99.0 MB/s 2024-04-25T10:10:42.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.894 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107488 s, 97.6 MB/s 2024-04-25T10:10:42.906 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.906 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.906 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010253 s, 102 MB/s 2024-04-25T10:10:42.918 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.918 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.918 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912596 s, 115 MB/s 2024-04-25T10:10:42.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.930 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00908915 s, 115 MB/s 2024-04-25T10:10:42.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.945 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119124 s, 88.0 MB/s 2024-04-25T10:10:42.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.958 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108229 s, 96.9 MB/s 2024-04-25T10:10:42.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.970 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901218 s, 116 MB/s 2024-04-25T10:10:42.982 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.982 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.982 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999819 s, 105 MB/s 2024-04-25T10:10:42.994 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:42.994 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:42.994 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895517 s, 117 MB/s 2024-04-25T10:10:43.012 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165012 s, 63.5 MB/s 2024-04-25T10:10:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101462 s, 103 MB/s 2024-04-25T10:10:43.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.055 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134259 s, 78.1 MB/s 2024-04-25T10:10:43.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.068 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01143 s, 91.7 MB/s 2024-04-25T10:10:43.081 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.082 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108376 s, 96.8 MB/s 2024-04-25T10:10:43.096 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.096 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.096 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121617 s, 86.2 MB/s 2024-04-25T10:10:43.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.107 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083366 s, 126 MB/s 2024-04-25T10:10:43.121 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124959 s, 83.9 MB/s 2024-04-25T10:10:43.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.133 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882743 s, 119 MB/s 2024-04-25T10:10:43.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.148 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128124 s, 81.8 MB/s 2024-04-25T10:10:43.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.162 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124225 s, 84.4 MB/s 2024-04-25T10:10:43.173 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.173 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.173 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833387 s, 126 MB/s 2024-04-25T10:10:43.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.186 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108246 s, 96.9 MB/s 2024-04-25T10:10:43.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.200 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.200 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119936 s, 87.4 MB/s 2024-04-25T10:10:43.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.211 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865833 s, 121 MB/s 2024-04-25T10:10:43.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.225 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120963 s, 86.7 MB/s 2024-04-25T10:10:43.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.241 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134077 s, 78.2 MB/s 2024-04-25T10:10:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012701 s, 82.6 MB/s 2024-04-25T10:10:43.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.270 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974806 s, 108 MB/s 2024-04-25T10:10:43.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.287 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139978 s, 74.9 MB/s 2024-04-25T10:10:43.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.300 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109767 s, 95.5 MB/s 2024-04-25T10:10:43.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.321 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184005 s, 57.0 MB/s 2024-04-25T10:10:43.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:43.322 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:43.322 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:43.322 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:10:43.322 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:43.322 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:43.323 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:43.323 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:43.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.427 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117987 s, 88.9 MB/s 2024-04-25T10:10:43.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.441 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011677 s, 89.8 MB/s 2024-04-25T10:10:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103278 s, 102 MB/s 2024-04-25T10:10:43.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.464 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826796 s, 127 MB/s 2024-04-25T10:10:43.477 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.477 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.478 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113434 s, 92.4 MB/s 2024-04-25T10:10:43.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.488 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789503 s, 133 MB/s 2024-04-25T10:10:43.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.501 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106161 s, 98.8 MB/s 2024-04-25T10:10:43.511 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.513 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850927 s, 123 MB/s 2024-04-25T10:10:43.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.525 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112555 s, 93.2 MB/s 2024-04-25T10:10:43.537 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.537 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.537 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904743 s, 116 MB/s 2024-04-25T10:10:43.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.551 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121907 s, 86.0 MB/s 2024-04-25T10:10:43.564 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.564 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.565 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107194 s, 97.8 MB/s 2024-04-25T10:10:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.578 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105504 s, 99.4 MB/s 2024-04-25T10:10:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108311 s, 96.8 MB/s 2024-04-25T10:10:43.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.602 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890819 s, 118 MB/s 2024-04-25T10:10:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109013 s, 96.2 MB/s 2024-04-25T10:10:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911003 s, 115 MB/s 2024-04-25T10:10:43.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.638 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856108 s, 122 MB/s 2024-04-25T10:10:43.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.651 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103103 s, 102 MB/s 2024-04-25T10:10:43.662 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.662 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.662 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867208 s, 121 MB/s 2024-04-25T10:10:43.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.676 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106285 s, 98.7 MB/s 2024-04-25T10:10:43.688 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.688 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.688 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106113 s, 98.8 MB/s 2024-04-25T10:10:43.701 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.701 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.701 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102066 s, 103 MB/s 2024-04-25T10:10:43.714 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.714 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.714 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108123 s, 97.0 MB/s 2024-04-25T10:10:43.727 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.727 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.727 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103408 s, 101 MB/s 2024-04-25T10:10:43.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.738 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854341 s, 123 MB/s 2024-04-25T10:10:43.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.749 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.749 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830433 s, 126 MB/s 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.763 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121097 s, 86.6 MB/s 2024-04-25T10:10:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011251 s, 93.2 MB/s 2024-04-25T10:10:43.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.789 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095063 s, 110 MB/s 2024-04-25T10:10:43.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.800 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861231 s, 122 MB/s 2024-04-25T10:10:43.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.815 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112732 s, 93.0 MB/s 2024-04-25T10:10:43.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:43.816 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:43.816 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:43.816 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:10:43.816 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:43.816 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:43.816 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:43.816 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:43.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.836 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.836 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858165 s, 122 MB/s 2024-04-25T10:10:43.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.849 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111076 s, 94.4 MB/s 2024-04-25T10:10:43.860 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.860 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.860 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837437 s, 125 MB/s 2024-04-25T10:10:43.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.873 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103642 s, 101 MB/s 2024-04-25T10:10:43.885 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.886 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010251 s, 102 MB/s 2024-04-25T10:10:43.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.899 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103007 s, 102 MB/s 2024-04-25T10:10:43.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.910 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836604 s, 125 MB/s 2024-04-25T10:10:43.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.922 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103717 s, 101 MB/s 2024-04-25T10:10:43.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.935 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106001 s, 98.9 MB/s 2024-04-25T10:10:43.949 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.949 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.949 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108333 s, 96.8 MB/s 2024-04-25T10:10:43.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.962 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104943 s, 99.9 MB/s 2024-04-25T10:10:43.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.973 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839106 s, 125 MB/s 2024-04-25T10:10:43.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.987 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.987 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110493 s, 94.9 MB/s 2024-04-25T10:10:43.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:43.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:43.999 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960433 s, 109 MB/s 2024-04-25T10:10:44.023 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.023 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.023 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215783 s, 48.6 MB/s 2024-04-25T10:10:44.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.043 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105213 s, 99.7 MB/s 2024-04-25T10:10:44.055 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.055 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.055 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101918 s, 103 MB/s 2024-04-25T10:10:44.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.065 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774028 s, 135 MB/s 2024-04-25T10:10:44.076 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.076 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.076 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081776 s, 128 MB/s 2024-04-25T10:10:44.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.086 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813101 s, 129 MB/s 2024-04-25T10:10:44.100 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.100 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.100 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111284 s, 94.2 MB/s 2024-04-25T10:10:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124774 s, 84.0 MB/s 2024-04-25T10:10:44.125 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.126 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.126 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085661 s, 122 MB/s 2024-04-25T10:10:44.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.138 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103091 s, 102 MB/s 2024-04-25T10:10:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850057 s, 123 MB/s 2024-04-25T10:10:44.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.162 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109742 s, 95.5 MB/s 2024-04-25T10:10:44.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.175 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101577 s, 103 MB/s 2024-04-25T10:10:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.188 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108082 s, 97.0 MB/s 2024-04-25T10:10:44.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.201 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106865 s, 98.1 MB/s 2024-04-25T10:10:44.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.214 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109348 s, 95.9 MB/s 2024-04-25T10:10:44.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.228 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117468 s, 89.3 MB/s 2024-04-25T10:10:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126671 s, 82.8 MB/s 2024-04-25T10:10:44.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:44.245 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:44.245 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:44.245 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:10:44.245 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:44.245 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:44.245 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:44.245 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:44.315 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.316 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.316 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143724 s, 73.0 MB/s 2024-04-25T10:10:44.333 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.333 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.333 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144594 s, 72.5 MB/s 2024-04-25T10:10:44.345 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.345 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.345 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00935796 s, 112 MB/s 2024-04-25T10:10:44.359 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.359 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.359 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115745 s, 90.6 MB/s 2024-04-25T10:10:44.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.370 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851318 s, 123 MB/s 2024-04-25T10:10:44.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.384 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114254 s, 91.8 MB/s 2024-04-25T10:10:44.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.397 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946831 s, 111 MB/s 2024-04-25T10:10:44.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.407 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817001 s, 128 MB/s 2024-04-25T10:10:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.420 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954523 s, 110 MB/s 2024-04-25T10:10:44.431 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.431 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.431 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834873 s, 126 MB/s 2024-04-25T10:10:44.446 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.446 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.446 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127658 s, 82.1 MB/s 2024-04-25T10:10:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126447 s, 82.9 MB/s 2024-04-25T10:10:44.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.474 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.474 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961961 s, 109 MB/s 2024-04-25T10:10:44.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.488 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118557 s, 88.4 MB/s 2024-04-25T10:10:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951335 s, 110 MB/s 2024-04-25T10:10:44.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.516 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.516 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129161 s, 81.2 MB/s 2024-04-25T10:10:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113384 s, 92.5 MB/s 2024-04-25T10:10:44.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.543 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111444 s, 94.1 MB/s 2024-04-25T10:10:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.554 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884331 s, 119 MB/s 2024-04-25T10:10:44.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.570 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133088 s, 78.8 MB/s 2024-04-25T10:10:44.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.582 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881739 s, 119 MB/s 2024-04-25T10:10:44.595 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.595 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.595 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969957 s, 108 MB/s 2024-04-25T10:10:44.608 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.608 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.608 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112391 s, 93.3 MB/s 2024-04-25T10:10:44.623 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.623 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.623 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122671 s, 85.5 MB/s 2024-04-25T10:10:44.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.637 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107124 s, 97.9 MB/s 2024-04-25T10:10:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.651 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116226 s, 90.2 MB/s 2024-04-25T10:10:44.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.666 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121551 s, 86.3 MB/s 2024-04-25T10:10:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126774 s, 82.7 MB/s 2024-04-25T10:10:44.697 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.698 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130538 s, 80.3 MB/s 2024-04-25T10:10:44.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.709 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913802 s, 115 MB/s 2024-04-25T10:10:44.722 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.722 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.722 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104403 s, 100 MB/s 2024-04-25T10:10:44.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.738 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112005 s, 93.6 MB/s 2024-04-25T10:10:44.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:44.738 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:44.738 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:44.739 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:10:44.739 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:44.739 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:44.739 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:44.739 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:44.802 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.802 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.802 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100061 s, 105 MB/s 2024-04-25T10:10:44.818 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.818 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.818 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123991 s, 84.6 MB/s 2024-04-25T10:10:44.831 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.831 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.831 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101364 s, 103 MB/s 2024-04-25T10:10:44.847 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.848 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.848 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136854 s, 76.6 MB/s 2024-04-25T10:10:44.862 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.862 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.862 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112054 s, 93.6 MB/s 2024-04-25T10:10:44.876 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.876 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.876 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106283 s, 98.7 MB/s 2024-04-25T10:10:44.891 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.892 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.892 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123268 s, 85.1 MB/s 2024-04-25T10:10:44.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.908 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129218 s, 81.1 MB/s 2024-04-25T10:10:44.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.922 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117221 s, 89.5 MB/s 2024-04-25T10:10:44.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.935 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102962 s, 102 MB/s 2024-04-25T10:10:44.948 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.948 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.948 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100985 s, 104 MB/s 2024-04-25T10:10:44.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.964 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131968 s, 79.5 MB/s 2024-04-25T10:10:44.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.975 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857324 s, 122 MB/s 2024-04-25T10:10:44.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:44.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:44.992 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013476 s, 77.8 MB/s 2024-04-25T10:10:45.006 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.006 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.006 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124959 s, 83.9 MB/s 2024-04-25T10:10:45.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.027 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171769 s, 61.0 MB/s 2024-04-25T10:10:45.051 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.051 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.051 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911512 s, 115 MB/s 2024-04-25T10:10:45.066 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.066 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.066 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127324 s, 82.4 MB/s 2024-04-25T10:10:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.077 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826511 s, 127 MB/s 2024-04-25T10:10:45.093 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.093 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.093 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132456 s, 79.2 MB/s 2024-04-25T10:10:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104992 s, 99.9 MB/s 2024-04-25T10:10:45.121 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.121 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.121 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127539 s, 82.2 MB/s 2024-04-25T10:10:45.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.134 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870938 s, 120 MB/s 2024-04-25T10:10:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112565 s, 93.2 MB/s 2024-04-25T10:10:45.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.236 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.236 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107856 s, 97.2 MB/s 2024-04-25T10:10:45.237 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.237 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.237 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132206 s, 79.3 MB/s 2024-04-25T10:10:45.237 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.237 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.237 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013516 s, 77.6 MB/s 2024-04-25T10:10:45.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.238 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116984 s, 89.6 MB/s 2024-04-25T10:10:45.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.238 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136286 s, 76.9 MB/s 2024-04-25T10:10:45.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.241 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932946 s, 112 MB/s 2024-04-25T10:10:45.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.253 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877526 s, 119 MB/s 2024-04-25T10:10:45.264 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.264 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.264 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886024 s, 118 MB/s 2024-04-25T10:10:45.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:45.266 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:45.267 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:45.267 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:10:45.267 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:45.267 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:45.267 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:45.267 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:45.331 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.332 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953443 s, 110 MB/s 2024-04-25T10:10:45.343 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.344 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.344 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090439 s, 116 MB/s 2024-04-25T10:10:45.358 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124274 s, 84.4 MB/s 2024-04-25T10:10:45.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.371 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103207 s, 102 MB/s 2024-04-25T10:10:45.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.384 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988169 s, 106 MB/s 2024-04-25T10:10:45.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.397 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116011 s, 90.4 MB/s 2024-04-25T10:10:45.408 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.408 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.408 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810448 s, 129 MB/s 2024-04-25T10:10:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851882 s, 123 MB/s 2024-04-25T10:10:45.431 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.431 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.431 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999298 s, 105 MB/s 2024-04-25T10:10:45.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.442 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836891 s, 125 MB/s 2024-04-25T10:10:45.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.455 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105083 s, 99.8 MB/s 2024-04-25T10:10:45.466 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.466 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.466 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871304 s, 120 MB/s 2024-04-25T10:10:45.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.480 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121587 s, 86.2 MB/s 2024-04-25T10:10:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.490 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074012 s, 142 MB/s 2024-04-25T10:10:45.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.501 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871518 s, 120 MB/s 2024-04-25T10:10:45.511 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.511 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.511 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762327 s, 138 MB/s 2024-04-25T10:10:45.522 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.522 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.522 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864379 s, 121 MB/s 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.532 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782078 s, 134 MB/s 2024-04-25T10:10:45.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.547 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127027 s, 82.5 MB/s 2024-04-25T10:10:45.557 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.557 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.558 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802948 s, 131 MB/s 2024-04-25T10:10:45.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.569 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912701 s, 115 MB/s 2024-04-25T10:10:45.580 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.580 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.580 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891571 s, 118 MB/s 2024-04-25T10:10:45.595 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.595 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.595 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012311 s, 85.2 MB/s 2024-04-25T10:10:45.609 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.609 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.609 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110091 s, 95.2 MB/s 2024-04-25T10:10:45.623 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.624 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.624 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124985 s, 83.9 MB/s 2024-04-25T10:10:45.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.639 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121815 s, 86.1 MB/s 2024-04-25T10:10:45.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.650 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859491 s, 122 MB/s 2024-04-25T10:10:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128194 s, 81.8 MB/s 2024-04-25T10:10:45.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.679 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114873 s, 91.3 MB/s 2024-04-25T10:10:45.694 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.695 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126545 s, 82.9 MB/s 2024-04-25T10:10:45.706 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.707 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966405 s, 109 MB/s 2024-04-25T10:10:45.718 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.718 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.718 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841143 s, 125 MB/s 2024-04-25T10:10:45.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:45.719 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:45.719 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:45.719 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:10:45.719 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:45.719 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:45.719 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:45.719 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:45.794 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.795 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.795 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119306 s, 87.9 MB/s 2024-04-25T10:10:45.810 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.811 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123618 s, 84.8 MB/s 2024-04-25T10:10:45.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.825 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122876 s, 85.3 MB/s 2024-04-25T10:10:45.836 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.836 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.837 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922417 s, 114 MB/s 2024-04-25T10:10:45.851 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.851 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.851 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120152 s, 87.3 MB/s 2024-04-25T10:10:45.864 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.864 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.864 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102778 s, 102 MB/s 2024-04-25T10:10:45.877 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.877 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.877 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010649 s, 98.5 MB/s 2024-04-25T10:10:45.892 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.892 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.893 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118396 s, 88.6 MB/s 2024-04-25T10:10:45.903 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.903 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.904 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842566 s, 124 MB/s 2024-04-25T10:10:45.918 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.918 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.918 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117788 s, 89.0 MB/s 2024-04-25T10:10:45.932 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.932 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.932 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01185 s, 88.5 MB/s 2024-04-25T10:10:45.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.945 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975466 s, 107 MB/s 2024-04-25T10:10:45.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.959 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110729 s, 94.7 MB/s 2024-04-25T10:10:45.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.970 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858592 s, 122 MB/s 2024-04-25T10:10:45.980 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.980 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.981 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082731 s, 127 MB/s 2024-04-25T10:10:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:45.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:45.994 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010677 s, 98.2 MB/s 2024-04-25T10:10:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.007 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106766 s, 98.2 MB/s 2024-04-25T10:10:46.024 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.024 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.024 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146198 s, 71.7 MB/s 2024-04-25T10:10:46.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786462 s, 133 MB/s 2024-04-25T10:10:46.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.073 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120778 s, 86.8 MB/s 2024-04-25T10:10:46.087 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.087 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.087 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118916 s, 88.2 MB/s 2024-04-25T10:10:46.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.101 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113336 s, 92.5 MB/s 2024-04-25T10:10:46.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.115 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011037 s, 95.0 MB/s 2024-04-25T10:10:46.126 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.126 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.126 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932967 s, 112 MB/s 2024-04-25T10:10:46.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.137 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844205 s, 124 MB/s 2024-04-25T10:10:46.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107784 s, 97.3 MB/s 2024-04-25T10:10:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826278 s, 127 MB/s 2024-04-25T10:10:46.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.174 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011046 s, 94.9 MB/s 2024-04-25T10:10:46.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.186 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925461 s, 113 MB/s 2024-04-25T10:10:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118645 s, 88.4 MB/s 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.213 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102614 s, 102 MB/s 2024-04-25T10:10:46.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.225 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103752 s, 101 MB/s 2024-04-25T10:10:46.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:46.227 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:46.227 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:46.227 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:10:46.227 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:46.227 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:46.227 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:46.227 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:46.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.301 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.301 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115911 s, 90.5 MB/s 2024-04-25T10:10:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.313 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102722 s, 102 MB/s 2024-04-25T10:10:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.327 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107669 s, 97.4 MB/s 2024-04-25T10:10:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130204 s, 80.5 MB/s 2024-04-25T10:10:46.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.357 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117894 s, 88.9 MB/s 2024-04-25T10:10:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126714 s, 82.8 MB/s 2024-04-25T10:10:46.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.386 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01097 s, 95.6 MB/s 2024-04-25T10:10:46.399 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.400 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.400 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112695 s, 93.0 MB/s 2024-04-25T10:10:46.412 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.412 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.412 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102814 s, 102 MB/s 2024-04-25T10:10:46.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.427 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011984 s, 87.5 MB/s 2024-04-25T10:10:46.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.442 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124259 s, 84.4 MB/s 2024-04-25T10:10:46.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.455 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106388 s, 98.6 MB/s 2024-04-25T10:10:46.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.468 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110548 s, 94.9 MB/s 2024-04-25T10:10:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974038 s, 108 MB/s 2024-04-25T10:10:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117505 s, 89.2 MB/s 2024-04-25T10:10:46.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919124 s, 114 MB/s 2024-04-25T10:10:46.520 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.520 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.520 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120574 s, 87.0 MB/s 2024-04-25T10:10:46.534 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.534 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.534 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111886 s, 93.7 MB/s 2024-04-25T10:10:46.548 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.548 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.548 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121024 s, 86.6 MB/s 2024-04-25T10:10:46.562 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.562 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.562 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113017 s, 92.8 MB/s 2024-04-25T10:10:46.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.575 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105552 s, 99.3 MB/s 2024-04-25T10:10:46.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.590 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120582 s, 87.0 MB/s 2024-04-25T10:10:46.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122429 s, 85.6 MB/s 2024-04-25T10:10:46.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.619 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119494 s, 87.8 MB/s 2024-04-25T10:10:46.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.634 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126575 s, 82.8 MB/s 2024-04-25T10:10:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.650 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129696 s, 80.8 MB/s 2024-04-25T10:10:46.664 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.664 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.664 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116835 s, 89.7 MB/s 2024-04-25T10:10:46.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.679 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121427 s, 86.4 MB/s 2024-04-25T10:10:46.694 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.695 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125449 s, 83.6 MB/s 2024-04-25T10:10:46.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.709 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124262 s, 84.4 MB/s 2024-04-25T10:10:46.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.724 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.724 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119221 s, 88.0 MB/s 2024-04-25T10:10:46.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.738 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117109 s, 89.5 MB/s 2024-04-25T10:10:46.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:46.739 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:46.739 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:46.739 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:10:46.739 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:46.739 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:46.739 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:46.739 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:46.808 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.808 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.808 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123257 s, 85.1 MB/s 2024-04-25T10:10:46.823 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.823 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.823 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126983 s, 82.6 MB/s 2024-04-25T10:10:46.837 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01047 s, 100 MB/s 2024-04-25T10:10:46.852 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.852 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.852 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105037 s, 99.8 MB/s 2024-04-25T10:10:46.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.868 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122389 s, 85.7 MB/s 2024-04-25T10:10:46.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.881 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989436 s, 106 MB/s 2024-04-25T10:10:46.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.894 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109543 s, 95.7 MB/s 2024-04-25T10:10:46.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.907 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991981 s, 106 MB/s 2024-04-25T10:10:46.921 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.921 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.921 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118389 s, 88.6 MB/s 2024-04-25T10:10:46.937 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.937 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.937 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126572 s, 82.8 MB/s 2024-04-25T10:10:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.950 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100151 s, 105 MB/s 2024-04-25T10:10:46.965 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.965 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.965 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123487 s, 84.9 MB/s 2024-04-25T10:10:46.977 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.978 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.978 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957158 s, 110 MB/s 2024-04-25T10:10:46.995 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:46.995 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:46.995 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143115 s, 73.3 MB/s 2024-04-25T10:10:47.011 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.011 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.011 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134393 s, 78.0 MB/s 2024-04-25T10:10:47.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.030 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155531 s, 67.4 MB/s 2024-04-25T10:10:47.066 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.066 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.066 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129986 s, 80.7 MB/s 2024-04-25T10:10:47.081 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.081 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.081 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126239 s, 83.1 MB/s 2024-04-25T10:10:47.097 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.097 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.097 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123038 s, 85.2 MB/s 2024-04-25T10:10:47.110 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.110 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.110 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100712 s, 104 MB/s 2024-04-25T10:10:47.125 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.125 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.125 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117958 s, 88.9 MB/s 2024-04-25T10:10:47.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.137 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877802 s, 119 MB/s 2024-04-25T10:10:47.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.152 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127361 s, 82.3 MB/s 2024-04-25T10:10:47.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.167 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.167 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116324 s, 90.1 MB/s 2024-04-25T10:10:47.179 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.179 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.179 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964889 s, 109 MB/s 2024-04-25T10:10:47.191 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.191 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.191 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957327 s, 110 MB/s 2024-04-25T10:10:47.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.206 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114354 s, 91.7 MB/s 2024-04-25T10:10:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111382 s, 94.1 MB/s 2024-04-25T10:10:47.235 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.235 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.235 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124468 s, 84.2 MB/s 2024-04-25T10:10:47.247 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.248 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102544 s, 102 MB/s 2024-04-25T10:10:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.264 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138764 s, 75.6 MB/s 2024-04-25T10:10:47.281 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.281 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.281 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140521 s, 74.6 MB/s 2024-04-25T10:10:47.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:47.283 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:47.283 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:47.283 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:10:47.283 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:47.283 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:47.283 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:47.283 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143232 s, 73.2 MB/s 2024-04-25T10:10:47.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.378 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.378 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125191 s, 83.8 MB/s 2024-04-25T10:10:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.392 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108739 s, 96.4 MB/s 2024-04-25T10:10:47.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.407 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121876 s, 86.0 MB/s 2024-04-25T10:10:47.425 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.425 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.425 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139899 s, 75.0 MB/s 2024-04-25T10:10:47.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.442 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133883 s, 78.3 MB/s 2024-04-25T10:10:47.458 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.458 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.458 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126371 s, 83.0 MB/s 2024-04-25T10:10:47.475 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.476 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.476 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139674 s, 75.1 MB/s 2024-04-25T10:10:47.493 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.493 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.493 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137409 s, 76.3 MB/s 2024-04-25T10:10:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135613 s, 77.3 MB/s 2024-04-25T10:10:47.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.527 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135191 s, 77.6 MB/s 2024-04-25T10:10:47.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.544 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119624 s, 87.7 MB/s 2024-04-25T10:10:47.561 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.561 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.561 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120921 s, 86.7 MB/s 2024-04-25T10:10:47.577 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.577 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.577 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121885 s, 86.0 MB/s 2024-04-25T10:10:47.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.590 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101357 s, 103 MB/s 2024-04-25T10:10:47.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.607 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129029 s, 81.3 MB/s 2024-04-25T10:10:47.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.621 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105722 s, 99.2 MB/s 2024-04-25T10:10:47.640 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.640 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.640 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159849 s, 65.6 MB/s 2024-04-25T10:10:47.657 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.657 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.657 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124721 s, 84.1 MB/s 2024-04-25T10:10:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130636 s, 80.3 MB/s 2024-04-25T10:10:47.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.693 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.693 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013128 s, 79.9 MB/s 2024-04-25T10:10:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.711 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134153 s, 78.2 MB/s 2024-04-25T10:10:47.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.729 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130266 s, 80.5 MB/s 2024-04-25T10:10:47.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.747 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013464 s, 77.9 MB/s 2024-04-25T10:10:47.762 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.762 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.763 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108071 s, 97.0 MB/s 2024-04-25T10:10:47.781 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.781 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.781 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133787 s, 78.4 MB/s 2024-04-25T10:10:47.798 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.798 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.798 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126228 s, 83.1 MB/s 2024-04-25T10:10:47.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.815 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120632 s, 86.9 MB/s 2024-04-25T10:10:47.833 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.833 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.833 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135624 s, 77.3 MB/s 2024-04-25T10:10:47.848 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.848 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.848 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113559 s, 92.3 MB/s 2024-04-25T10:10:47.863 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.864 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.864 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114787 s, 91.3 MB/s 2024-04-25T10:10:47.878 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.879 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.879 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109248 s, 96.0 MB/s 2024-04-25T10:10:47.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:47.880 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:47.880 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:47.880 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:10:47.880 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:47.880 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:47.880 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:47.880 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:47.950 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.950 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.950 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129727 s, 80.8 MB/s 2024-04-25T10:10:47.963 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.964 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104725 s, 100 MB/s 2024-04-25T10:10:47.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.975 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891153 s, 118 MB/s 2024-04-25T10:10:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:47.989 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106956 s, 98.0 MB/s 2024-04-25T10:10:48.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.000 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870735 s, 120 MB/s 2024-04-25T10:10:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115404 s, 90.9 MB/s 2024-04-25T10:10:48.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.025 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890393 s, 118 MB/s 2024-04-25T10:10:48.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.043 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149355 s, 70.2 MB/s 2024-04-25T10:10:48.066 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.066 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.066 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107721 s, 97.3 MB/s 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011028 s, 95.1 MB/s 2024-04-25T10:10:48.094 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.094 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.094 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122311 s, 85.7 MB/s 2024-04-25T10:10:48.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103503 s, 101 MB/s 2024-04-25T10:10:48.120 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.120 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.120 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995152 s, 105 MB/s 2024-04-25T10:10:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.134 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109775 s, 95.5 MB/s 2024-04-25T10:10:48.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.147 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101088 s, 104 MB/s 2024-04-25T10:10:48.158 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.158 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.158 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883557 s, 119 MB/s 2024-04-25T10:10:48.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.175 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144051 s, 72.8 MB/s 2024-04-25T10:10:48.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111556 s, 94.0 MB/s 2024-04-25T10:10:48.203 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.203 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.203 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118316 s, 88.6 MB/s 2024-04-25T10:10:48.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.215 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921321 s, 114 MB/s 2024-04-25T10:10:48.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.228 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987638 s, 106 MB/s 2024-04-25T10:10:48.239 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.239 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.239 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869239 s, 121 MB/s 2024-04-25T10:10:48.254 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.254 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.254 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126445 s, 82.9 MB/s 2024-04-25T10:10:48.268 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.268 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.268 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01102 s, 95.2 MB/s 2024-04-25T10:10:48.280 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.280 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.280 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980203 s, 107 MB/s 2024-04-25T10:10:48.291 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.291 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.291 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845106 s, 124 MB/s 2024-04-25T10:10:48.305 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.305 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.305 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114257 s, 91.8 MB/s 2024-04-25T10:10:48.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.320 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115632 s, 90.7 MB/s 2024-04-25T10:10:48.331 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.331 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.332 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920737 s, 114 MB/s 2024-04-25T10:10:48.346 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.346 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.346 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120948 s, 86.7 MB/s 2024-04-25T10:10:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.360 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107125 s, 97.9 MB/s 2024-04-25T10:10:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121038 s, 86.6 MB/s 2024-04-25T10:10:48.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:48.376 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:48.376 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:48.376 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:10:48.376 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:48.376 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:48.376 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:48.376 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:48.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.442 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127321 s, 82.4 MB/s 2024-04-25T10:10:48.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.456 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.456 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113896 s, 92.1 MB/s 2024-04-25T10:10:48.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.470 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123025 s, 85.2 MB/s 2024-04-25T10:10:48.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.481 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774667 s, 135 MB/s 2024-04-25T10:10:48.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.494 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011418 s, 91.8 MB/s 2024-04-25T10:10:48.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991362 s, 106 MB/s 2024-04-25T10:10:48.518 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.518 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.518 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010095 s, 104 MB/s 2024-04-25T10:10:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107409 s, 97.6 MB/s 2024-04-25T10:10:48.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.544 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101001 s, 104 MB/s 2024-04-25T10:10:48.556 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.556 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.557 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105546 s, 99.3 MB/s 2024-04-25T10:10:48.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.569 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100052 s, 105 MB/s 2024-04-25T10:10:48.580 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.580 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.580 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086187 s, 122 MB/s 2024-04-25T10:10:48.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.592 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102827 s, 102 MB/s 2024-04-25T10:10:48.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.606 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112208 s, 93.4 MB/s 2024-04-25T10:10:48.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.619 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114466 s, 91.6 MB/s 2024-04-25T10:10:48.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.634 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120148 s, 87.3 MB/s 2024-04-25T10:10:48.644 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.644 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.644 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085395 s, 123 MB/s 2024-04-25T10:10:48.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.655 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802297 s, 131 MB/s 2024-04-25T10:10:48.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.666 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849443 s, 123 MB/s 2024-04-25T10:10:48.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.680 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115303 s, 90.9 MB/s 2024-04-25T10:10:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835195 s, 126 MB/s 2024-04-25T10:10:48.705 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.705 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.705 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119616 s, 87.7 MB/s 2024-04-25T10:10:48.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.716 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.716 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838985 s, 125 MB/s 2024-04-25T10:10:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106878 s, 98.1 MB/s 2024-04-25T10:10:48.742 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.742 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.742 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105216 s, 99.7 MB/s 2024-04-25T10:10:48.755 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.755 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.755 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107248 s, 97.8 MB/s 2024-04-25T10:10:48.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.769 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011383 s, 92.1 MB/s 2024-04-25T10:10:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.779 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826953 s, 127 MB/s 2024-04-25T10:10:48.794 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.794 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.794 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120209 s, 87.2 MB/s 2024-04-25T10:10:48.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.804 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080936 s, 130 MB/s 2024-04-25T10:10:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.819 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127923 s, 82.0 MB/s 2024-04-25T10:10:48.834 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.834 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.834 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926325 s, 113 MB/s 2024-04-25T10:10:48.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:48.835 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:48.835 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:48.835 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:10:48.835 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:48.836 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:48.836 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:48.836 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:48.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.901 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.901 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101745 s, 103 MB/s 2024-04-25T10:10:48.915 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.915 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.916 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122866 s, 85.3 MB/s 2024-04-25T10:10:48.929 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.929 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.929 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117297 s, 89.4 MB/s 2024-04-25T10:10:48.941 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.941 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.941 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938433 s, 112 MB/s 2024-04-25T10:10:48.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.951 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795891 s, 132 MB/s 2024-04-25T10:10:48.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.962 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870153 s, 121 MB/s 2024-04-25T10:10:48.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.975 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109608 s, 95.7 MB/s 2024-04-25T10:10:48.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:48.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:48.987 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872668 s, 120 MB/s 2024-04-25T10:10:49.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.000 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011591 s, 90.5 MB/s 2024-04-25T10:10:49.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111206 s, 94.3 MB/s 2024-04-25T10:10:49.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.025 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101392 s, 103 MB/s 2024-04-25T10:10:49.045 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.045 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.045 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176925 s, 59.3 MB/s 2024-04-25T10:10:49.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.074 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118932 s, 88.2 MB/s 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.085 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.086 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983794 s, 107 MB/s 2024-04-25T10:10:49.100 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.100 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.100 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01264 s, 83.0 MB/s 2024-04-25T10:10:49.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.114 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109708 s, 95.6 MB/s 2024-04-25T10:10:49.128 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.128 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.128 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102597 s, 102 MB/s 2024-04-25T10:10:49.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.141 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010292 s, 102 MB/s 2024-04-25T10:10:49.155 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.155 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.155 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119619 s, 87.7 MB/s 2024-04-25T10:10:49.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.166 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861037 s, 122 MB/s 2024-04-25T10:10:49.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.180 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120511 s, 87.0 MB/s 2024-04-25T10:10:49.195 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.195 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.195 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118386 s, 88.6 MB/s 2024-04-25T10:10:49.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.208 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111293 s, 94.2 MB/s 2024-04-25T10:10:49.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.222 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011868 s, 88.4 MB/s 2024-04-25T10:10:49.235 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112788 s, 93.0 MB/s 2024-04-25T10:10:49.249 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.250 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.250 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117216 s, 89.5 MB/s 2024-04-25T10:10:49.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.261 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869804 s, 121 MB/s 2024-04-25T10:10:49.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.273 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940935 s, 111 MB/s 2024-04-25T10:10:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899181 s, 117 MB/s 2024-04-25T10:10:49.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.296 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907013 s, 116 MB/s 2024-04-25T10:10:49.309 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.309 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.309 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010474 s, 100 MB/s 2024-04-25T10:10:49.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.324 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124932 s, 83.9 MB/s 2024-04-25T10:10:49.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:49.325 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:49.325 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:49.325 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:10:49.325 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:49.325 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:49.325 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:49.325 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:49.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.391 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107807 s, 97.3 MB/s 2024-04-25T10:10:49.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.407 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129808 s, 80.8 MB/s 2024-04-25T10:10:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127403 s, 82.3 MB/s 2024-04-25T10:10:49.437 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.437 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.437 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113602 s, 92.3 MB/s 2024-04-25T10:10:49.448 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.448 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.449 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884924 s, 118 MB/s 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120982 s, 86.7 MB/s 2024-04-25T10:10:49.477 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.477 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.477 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011961 s, 87.7 MB/s 2024-04-25T10:10:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120958 s, 86.7 MB/s 2024-04-25T10:10:49.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116303 s, 90.2 MB/s 2024-04-25T10:10:49.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.517 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086641 s, 121 MB/s 2024-04-25T10:10:49.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.531 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116639 s, 89.9 MB/s 2024-04-25T10:10:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106174 s, 98.8 MB/s 2024-04-25T10:10:49.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.559 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119894 s, 87.5 MB/s 2024-04-25T10:10:49.572 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.572 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.572 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112718 s, 93.0 MB/s 2024-04-25T10:10:49.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.582 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076734 s, 137 MB/s 2024-04-25T10:10:49.595 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.596 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.596 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109836 s, 95.5 MB/s 2024-04-25T10:10:49.610 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.610 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.610 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115794 s, 90.6 MB/s 2024-04-25T10:10:49.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.622 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010336 s, 101 MB/s 2024-04-25T10:10:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.636 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110374 s, 95.0 MB/s 2024-04-25T10:10:49.645 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757126 s, 138 MB/s 2024-04-25T10:10:49.660 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.660 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.660 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123241 s, 85.1 MB/s 2024-04-25T10:10:49.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.671 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883139 s, 119 MB/s 2024-04-25T10:10:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129597 s, 80.9 MB/s 2024-04-25T10:10:49.700 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.700 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.700 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109272 s, 96.0 MB/s 2024-04-25T10:10:49.712 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.713 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.713 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104729 s, 100 MB/s 2024-04-25T10:10:49.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.726 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111743 s, 93.8 MB/s 2024-04-25T10:10:49.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.738 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103847 s, 101 MB/s 2024-04-25T10:10:49.753 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.753 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.753 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127528 s, 82.2 MB/s 2024-04-25T10:10:49.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.768 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123433 s, 85.0 MB/s 2024-04-25T10:10:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867572 s, 121 MB/s 2024-04-25T10:10:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.790 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833903 s, 126 MB/s 2024-04-25T10:10:49.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.804 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122827 s, 85.4 MB/s 2024-04-25T10:10:49.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:49.805 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:49.805 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:49.805 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:10:49.806 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:49.806 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:49.806 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:49.806 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:49.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.870 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115237 s, 91.0 MB/s 2024-04-25T10:10:49.884 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.885 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125658 s, 83.4 MB/s 2024-04-25T10:10:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.894 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770131 s, 136 MB/s 2024-04-25T10:10:49.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.909 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121141 s, 86.6 MB/s 2024-04-25T10:10:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118606 s, 88.4 MB/s 2024-04-25T10:10:49.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.936 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103952 s, 101 MB/s 2024-04-25T10:10:49.948 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.949 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.949 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105774 s, 99.1 MB/s 2024-04-25T10:10:49.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.959 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842196 s, 125 MB/s 2024-04-25T10:10:49.971 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.971 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.971 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099569 s, 105 MB/s 2024-04-25T10:10:49.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.985 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107939 s, 97.1 MB/s 2024-04-25T10:10:49.994 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:49.994 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:49.994 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759363 s, 138 MB/s 2024-04-25T10:10:50.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.007 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107051 s, 98.0 MB/s 2024-04-25T10:10:50.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.020 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010835 s, 96.8 MB/s 2024-04-25T10:10:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.035 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.035 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117813 s, 89.0 MB/s 2024-04-25T10:10:50.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.050 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125105 s, 83.8 MB/s 2024-04-25T10:10:50.081 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.082 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124276 s, 84.4 MB/s 2024-04-25T10:10:50.092 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.092 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.093 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877544 s, 119 MB/s 2024-04-25T10:10:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131413 s, 79.8 MB/s 2024-04-25T10:10:50.121 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.122 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.122 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106768 s, 98.2 MB/s 2024-04-25T10:10:50.135 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.135 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011423 s, 91.8 MB/s 2024-04-25T10:10:50.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011207 s, 93.6 MB/s 2024-04-25T10:10:50.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.165 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121102 s, 86.6 MB/s 2024-04-25T10:10:50.176 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.176 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.177 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861455 s, 122 MB/s 2024-04-25T10:10:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116643 s, 89.9 MB/s 2024-04-25T10:10:50.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.206 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.206 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122155 s, 85.8 MB/s 2024-04-25T10:10:50.218 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.219 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987817 s, 106 MB/s 2024-04-25T10:10:50.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.231 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00947757 s, 111 MB/s 2024-04-25T10:10:50.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.244 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.244 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102802 s, 102 MB/s 2024-04-25T10:10:50.256 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.256 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.256 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973209 s, 108 MB/s 2024-04-25T10:10:50.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.272 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01245 s, 84.2 MB/s 2024-04-25T10:10:50.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.288 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131515 s, 79.7 MB/s 2024-04-25T10:10:50.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.303 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127052 s, 82.5 MB/s 2024-04-25T10:10:50.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:50.304 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:50.304 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:50.304 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:10:50.304 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:50.304 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:50.304 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:50.305 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.369 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110537 s, 94.9 MB/s 2024-04-25T10:10:50.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.386 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132855 s, 78.9 MB/s 2024-04-25T10:10:50.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.401 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012046 s, 87.0 MB/s 2024-04-25T10:10:50.417 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.417 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.417 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125731 s, 83.4 MB/s 2024-04-25T10:10:50.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.435 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153556 s, 68.3 MB/s 2024-04-25T10:10:50.447 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.447 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.447 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927507 s, 113 MB/s 2024-04-25T10:10:50.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.462 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124746 s, 84.1 MB/s 2024-04-25T10:10:50.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.474 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.474 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881689 s, 119 MB/s 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.486 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960828 s, 109 MB/s 2024-04-25T10:10:50.496 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.497 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.497 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827761 s, 127 MB/s 2024-04-25T10:10:50.511 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.511 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.512 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119593 s, 87.7 MB/s 2024-04-25T10:10:50.522 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.522 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.522 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860814 s, 122 MB/s 2024-04-25T10:10:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011193 s, 93.7 MB/s 2024-04-25T10:10:50.548 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.549 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.549 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102592 s, 102 MB/s 2024-04-25T10:10:50.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.559 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082361 s, 127 MB/s 2024-04-25T10:10:50.573 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.573 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.574 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115564 s, 90.7 MB/s 2024-04-25T10:10:50.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.584 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837842 s, 125 MB/s 2024-04-25T10:10:50.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.600 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.600 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011915 s, 88.0 MB/s 2024-04-25T10:10:50.613 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.613 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.613 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113626 s, 92.3 MB/s 2024-04-25T10:10:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857496 s, 122 MB/s 2024-04-25T10:10:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.636 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909977 s, 115 MB/s 2024-04-25T10:10:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852422 s, 123 MB/s 2024-04-25T10:10:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110748 s, 94.7 MB/s 2024-04-25T10:10:50.672 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.672 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.672 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899864 s, 117 MB/s 2024-04-25T10:10:50.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.687 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.687 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118997 s, 88.1 MB/s 2024-04-25T10:10:50.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922462 s, 114 MB/s 2024-04-25T10:10:50.713 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.713 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.713 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115731 s, 90.6 MB/s 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882528 s, 119 MB/s 2024-04-25T10:10:50.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.738 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117473 s, 89.3 MB/s 2024-04-25T10:10:50.752 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.753 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.753 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119779 s, 87.5 MB/s 2024-04-25T10:10:50.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.766 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110149 s, 95.2 MB/s 2024-04-25T10:10:50.780 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.780 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.780 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111127 s, 94.4 MB/s 2024-04-25T10:10:50.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:50.781 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:50.781 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:50.781 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:10:50.781 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:50.781 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:50.781 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:50.781 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:50.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.849 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129483 s, 81.0 MB/s 2024-04-25T10:10:50.864 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.864 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.864 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123104 s, 85.2 MB/s 2024-04-25T10:10:50.879 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.879 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.879 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122642 s, 85.5 MB/s 2024-04-25T10:10:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.893 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.893 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104735 s, 100 MB/s 2024-04-25T10:10:50.905 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.905 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.905 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958729 s, 109 MB/s 2024-04-25T10:10:50.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.920 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115987 s, 90.4 MB/s 2024-04-25T10:10:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118086 s, 88.8 MB/s 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.948 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106615 s, 98.4 MB/s 2024-04-25T10:10:50.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.961 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.961 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105241 s, 99.6 MB/s 2024-04-25T10:10:50.974 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.974 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.974 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109484 s, 95.8 MB/s 2024-04-25T10:10:50.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:50.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:50.986 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965268 s, 109 MB/s 2024-04-25T10:10:50.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.000 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011241 s, 93.3 MB/s 2024-04-25T10:10:51.015 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.015 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.015 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121955 s, 86.0 MB/s 2024-04-25T10:10:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.026 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960962 s, 109 MB/s 2024-04-25T10:10:51.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.039 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101606 s, 103 MB/s 2024-04-25T10:10:51.055 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.055 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.055 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143129 s, 73.3 MB/s 2024-04-25T10:10:51.089 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.089 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.089 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914635 s, 115 MB/s 2024-04-25T10:10:51.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.103 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107316 s, 97.7 MB/s 2024-04-25T10:10:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.114 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856635 s, 122 MB/s 2024-04-25T10:10:51.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.129 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125114 s, 83.8 MB/s 2024-04-25T10:10:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109282 s, 96.0 MB/s 2024-04-25T10:10:51.154 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.155 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907066 s, 116 MB/s 2024-04-25T10:10:51.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.169 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117136 s, 89.5 MB/s 2024-04-25T10:10:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.186 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135961 s, 77.1 MB/s 2024-04-25T10:10:51.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.202 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128175 s, 81.8 MB/s 2024-04-25T10:10:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139272 s, 75.3 MB/s 2024-04-25T10:10:51.233 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.233 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.233 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112389 s, 93.3 MB/s 2024-04-25T10:10:51.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.248 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126382 s, 83.0 MB/s 2024-04-25T10:10:51.262 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.262 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.262 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103415 s, 101 MB/s 2024-04-25T10:10:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128497 s, 81.6 MB/s 2024-04-25T10:10:51.293 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.293 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.294 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121387 s, 86.4 MB/s 2024-04-25T10:10:51.309 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.309 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.309 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123491 s, 84.9 MB/s 2024-04-25T10:10:51.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:51.310 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:51.310 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:51.310 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:10:51.310 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:51.310 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:51.310 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:51.310 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:51.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.377 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012421 s, 84.4 MB/s 2024-04-25T10:10:51.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.392 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122536 s, 85.6 MB/s 2024-04-25T10:10:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.406 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117089 s, 89.6 MB/s 2024-04-25T10:10:51.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.423 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128325 s, 81.7 MB/s 2024-04-25T10:10:51.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.435 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103464 s, 101 MB/s 2024-04-25T10:10:51.448 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.448 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.448 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993791 s, 106 MB/s 2024-04-25T10:10:51.460 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.460 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.460 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911092 s, 115 MB/s 2024-04-25T10:10:51.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.474 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108027 s, 97.1 MB/s 2024-04-25T10:10:51.487 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.487 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.487 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110653 s, 94.8 MB/s 2024-04-25T10:10:51.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.502 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.502 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120153 s, 87.3 MB/s 2024-04-25T10:10:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145842 s, 71.9 MB/s 2024-04-25T10:10:51.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.536 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136165 s, 77.0 MB/s 2024-04-25T10:10:51.550 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.551 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011211 s, 93.5 MB/s 2024-04-25T10:10:51.564 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.564 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.564 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101644 s, 103 MB/s 2024-04-25T10:10:51.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.578 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110026 s, 95.3 MB/s 2024-04-25T10:10:51.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.591 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100249 s, 105 MB/s 2024-04-25T10:10:51.603 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102997 s, 102 MB/s 2024-04-25T10:10:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.617 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962358 s, 109 MB/s 2024-04-25T10:10:51.629 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.629 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.629 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109211 s, 96.0 MB/s 2024-04-25T10:10:51.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.643 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112415 s, 93.3 MB/s 2024-04-25T10:10:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090778 s, 116 MB/s 2024-04-25T10:10:51.668 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.669 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112167 s, 93.5 MB/s 2024-04-25T10:10:51.683 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.683 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.684 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121576 s, 86.2 MB/s 2024-04-25T10:10:51.694 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.695 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858841 s, 122 MB/s 2024-04-25T10:10:51.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.709 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119902 s, 87.5 MB/s 2024-04-25T10:10:51.722 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.722 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.722 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105837 s, 99.1 MB/s 2024-04-25T10:10:51.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.738 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125968 s, 83.2 MB/s 2024-04-25T10:10:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105589 s, 99.3 MB/s 2024-04-25T10:10:51.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.768 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131232 s, 79.9 MB/s 2024-04-25T10:10:51.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.779 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888978 s, 118 MB/s 2024-04-25T10:10:51.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.791 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872194 s, 120 MB/s 2024-04-25T10:10:51.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.805 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110361 s, 95.0 MB/s 2024-04-25T10:10:51.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:51.806 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:51.806 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:51.806 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:10:51.806 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:51.806 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:51.806 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:51.806 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:51.872 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.872 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.872 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129323 s, 81.1 MB/s 2024-04-25T10:10:51.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.886 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119365 s, 87.8 MB/s 2024-04-25T10:10:51.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.901 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.901 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114232 s, 91.8 MB/s 2024-04-25T10:10:51.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.913 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941291 s, 111 MB/s 2024-04-25T10:10:51.924 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.925 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.925 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930176 s, 113 MB/s 2024-04-25T10:10:51.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.940 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125718 s, 83.4 MB/s 2024-04-25T10:10:51.954 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.954 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.954 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117974 s, 88.9 MB/s 2024-04-25T10:10:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122354 s, 85.7 MB/s 2024-04-25T10:10:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887137 s, 118 MB/s 2024-04-25T10:10:51.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:51.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:51.993 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108354 s, 96.8 MB/s 2024-04-25T10:10:52.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.007 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106808 s, 98.2 MB/s 2024-04-25T10:10:52.018 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.018 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.018 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881042 s, 119 MB/s 2024-04-25T10:10:52.032 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.032 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.032 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112924 s, 92.9 MB/s 2024-04-25T10:10:52.042 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.043 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836666 s, 125 MB/s 2024-04-25T10:10:52.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.058 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130377 s, 80.4 MB/s 2024-04-25T10:10:52.090 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.090 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.090 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839854 s, 125 MB/s 2024-04-25T10:10:52.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.107 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136547 s, 76.8 MB/s 2024-04-25T10:10:52.119 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.119 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.119 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895797 s, 117 MB/s 2024-04-25T10:10:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115213 s, 91.0 MB/s 2024-04-25T10:10:52.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.146 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957236 s, 110 MB/s 2024-04-25T10:10:52.159 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.159 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.159 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109329 s, 95.9 MB/s 2024-04-25T10:10:52.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.174 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011639 s, 90.1 MB/s 2024-04-25T10:10:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854408 s, 123 MB/s 2024-04-25T10:10:52.198 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.199 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104325 s, 101 MB/s 2024-04-25T10:10:52.213 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.214 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126006 s, 83.2 MB/s 2024-04-25T10:10:52.229 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.229 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.229 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117778 s, 89.0 MB/s 2024-04-25T10:10:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.244 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.244 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117703 s, 89.1 MB/s 2024-04-25T10:10:52.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.259 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125124 s, 83.8 MB/s 2024-04-25T10:10:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109871 s, 95.4 MB/s 2024-04-25T10:10:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955381 s, 110 MB/s 2024-04-25T10:10:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899853 s, 117 MB/s 2024-04-25T10:10:52.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.311 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114088 s, 91.9 MB/s 2024-04-25T10:10:52.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:52.312 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:10:52.312 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:10:52.313 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:10:52.313 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:10:52.313 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:10:52.313 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:10:52.313 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:10:52.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.376 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946477 s, 111 MB/s 2024-04-25T10:10:52.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.392 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013396 s, 78.3 MB/s 2024-04-25T10:10:52.404 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.404 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.404 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934607 s, 112 MB/s 2024-04-25T10:10:52.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.416 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878282 s, 119 MB/s 2024-04-25T10:10:52.428 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.428 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.428 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010256 s, 102 MB/s 2024-04-25T10:10:52.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.442 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011371 s, 92.2 MB/s 2024-04-25T10:10:52.456 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.456 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.456 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116091 s, 90.3 MB/s 2024-04-25T10:10:52.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.473 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135275 s, 77.5 MB/s 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126367 s, 83.0 MB/s 2024-04-25T10:10:52.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135525 s, 77.4 MB/s 2024-04-25T10:10:52.513 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:52.516 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121986 s, 86.0 MB/s 2024-04-25T10:10:52.534 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.534 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.535 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010612 s, 98.8 MB/s 2024-04-25T10:10:52.548 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.548 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.549 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112092 s, 93.5 MB/s 2024-04-25T10:10:52.563 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.564 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.564 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116604 s, 89.9 MB/s 2024-04-25T10:10:52.576 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.576 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.577 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104664 s, 100 MB/s 2024-04-25T10:10:52.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.591 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112511 s, 93.2 MB/s 2024-04-25T10:10:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114678 s, 91.4 MB/s 2024-04-25T10:10:52.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.619 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114128 s, 91.9 MB/s 2024-04-25T10:10:52.633 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.634 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120117 s, 87.3 MB/s 2024-04-25T10:10:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124974 s, 83.9 MB/s 2024-04-25T10:10:52.659 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.659 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.659 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840004 s, 125 MB/s 2024-04-25T10:10:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852871 s, 123 MB/s 2024-04-25T10:10:52.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.686 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121526 s, 86.3 MB/s 2024-04-25T10:10:52.700 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.700 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.700 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117709 s, 89.1 MB/s 2024-04-25T10:10:52.714 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.715 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118236 s, 88.7 MB/s 2024-04-25T10:10:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.727 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896919 s, 117 MB/s 2024-04-25T10:10:52.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.739 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982887 s, 107 MB/s 2024-04-25T10:10:52.753 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119727 s, 87.6 MB/s 2024-04-25T10:10:52.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.769 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131451 s, 79.8 MB/s 2024-04-25T10:10:52.784 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.784 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.784 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124562 s, 84.2 MB/s 2024-04-25T10:10:52.798 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.798 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.798 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108513 s, 96.6 MB/s 2024-04-25T10:10:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:10:52.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:10:52.814 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130501 s, 80.4 MB/s 2024-04-25T10:10:52.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:52.815 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:10:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d634a2640 1 Processor -- start 2024-04-25T10:10:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d634a2640 1 -- start start 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d634a2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5d5c15ef20 0x7f5d5c09a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d634a2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5c0cd1e0 0x7f5d5c0c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d634a2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5c099b50 con 0x7f5d5c0ccd50 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d634a2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5c0c5ed0 con 0x7f5d5c15ef20 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d634a2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c09ae50 con 0x7f5d5c0cd1e0 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d6123e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5d5c15ef20 0x7f5d5c09a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d60a3d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5d5c0ccd50 0x7f5d5c0c5280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54594/0 (socket says 172.21.15.133:54594) 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d60a3d640 1 -- 172.21.15.133:0/380333027 learned_addr learned my addr 172.21.15.133:0/380333027 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d61a3f640 1 --2- 172.21.15.133:0/380333027 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5c0cd1e0 0x7f5d5c0c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d6123e640 1 -- 172.21.15.133:0/380333027 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5c0cd1e0 msgr2=0x7f5d5c0c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d6123e640 1 --2- 172.21.15.133:0/380333027 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5c0cd1e0 0x7f5d5c0c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d6123e640 1 -- 172.21.15.133:0/380333027 >> v1:172.21.15.133:6789/0 conn(0x7f5d5c0ccd50 legacy=0x7f5d5c0c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d5bfff640 1 -- 172.21.15.133:0/380333027 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925842233 0 0) 0x7f5d5c099b50 con 0x7f5d5c0ccd50 2024-04-25T10:10:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.986+0000 7f5d6123e640 1 -- 172.21.15.133:0/380333027 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c099e30 con 0x7f5d5c15ef20 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d6123e640 1 --2- 172.21.15.133:0/380333027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5c15ef20 0x7f5d5c09a910 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5d540097f0 tx=0x7f5d5401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f8a03fc1e18329 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d5bfff640 1 -- 172.21.15.133:0/380333027 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d540b1070 con 0x7f5d5c15ef20 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d5bfff640 1 -- 172.21.15.133:0/380333027 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d540ac410 con 0x7f5d5c15ef20 2024-04-25T10:10:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d5bfff640 1 -- 172.21.15.133:0/380333027 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d540ac7e0 con 0x7f5d5c15ef20 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d634a2640 1 -- 172.21.15.133:0/380333027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5c15ef20 msgr2=0x7f5d5c09a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d634a2640 1 --2- 172.21.15.133:0/380333027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5c15ef20 0x7f5d5c09a910 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5d540097f0 tx=0x7f5d5401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d634a2640 1 -- 172.21.15.133:0/380333027 shutdown_connections 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d634a2640 1 --2- 172.21.15.133:0/380333027 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5c0cd1e0 0x7f5d5c0c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d634a2640 1 --2- 172.21.15.133:0/380333027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5c15ef20 0x7f5d5c09a910 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d634a2640 1 -- 172.21.15.133:0/380333027 >> 172.21.15.133:0/380333027 conn(0x7f5d5c0b3220 msgr2=0x7f5d5c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d634a2640 1 -- 172.21.15.133:0/380333027 shutdown_connections 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d634a2640 1 -- 172.21.15.133:0/380333027 wait complete. 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d634a2640 1 Processor -- start 2024-04-25T10:10:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d634a2640 1 -- start start 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.990+0000 7f5d634a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5c0ccd50 0x7f5d5c154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d634a2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5c0cd1e0 0x7f5d5c09e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d634a2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5c15ef20 0x7f5d5c0ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d634a2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c15fc90 con 0x7f5d5c0ccd50 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d634a2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c15fb10 con 0x7f5d5c0cd1e0 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d634a2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c15fe10 con 0x7f5d5c15ef20 2024-04-25T10:10:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d6123e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5c0ccd50 0x7f5d5c154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d6123e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5c0ccd50 0x7f5d5c154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52682/0 (socket says 172.21.15.133:52682) 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d6123e640 1 -- 172.21.15.133:0/3547947829 learned_addr learned my addr 172.21.15.133:0/3547947829 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d60a3d640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5c0cd1e0 0x7f5d5c09e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d61a3f640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5c15ef20 0x7f5d5c0ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d6123e640 1 -- 172.21.15.133:0/3547947829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5c15ef20 msgr2=0x7f5d5c0ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d6123e640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5c15ef20 0x7f5d5c0ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d6123e640 1 -- 172.21.15.133:0/3547947829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5c0cd1e0 msgr2=0x7f5d5c09e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d6123e640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5c0cd1e0 0x7f5d5c09e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d6123e640 1 -- 172.21.15.133:0/3547947829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c09aa70 con 0x7f5d5c0ccd50 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d60a3d640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5c0cd1e0 0x7f5d5c09e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d6123e640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5c0ccd50 0x7f5d5c154ea0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5d540097f0 tx=0x7f5d5401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d59ffb640 1 -- 172.21.15.133:0/3547947829 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d540b1050 con 0x7f5d5c0ccd50 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d59ffb640 1 -- 172.21.15.133:0/3547947829 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d540b2040 con 0x7f5d5c0ccd50 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d59ffb640 1 -- 172.21.15.133:0/3547947829 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54006e20 con 0x7f5d5c0ccd50 2024-04-25T10:10:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d634a2640 1 -- 172.21.15.133:0/3547947829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c1fa290 con 0x7f5d5c0ccd50 2024-04-25T10:10:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d634a2640 1 -- 172.21.15.133:0/3547947829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c1fa7e0 con 0x7f5d5c0ccd50 2024-04-25T10:10:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d59ffb640 1 -- 172.21.15.133:0/3547947829 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d540bf060 con 0x7f5d5c0ccd50 2024-04-25T10:10:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d59ffb640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d48066f60 0x7f5d48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d59ffb640 1 -- 172.21.15.133:0/3547947829 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f5d5411e9c0 con 0x7f5d5c0ccd50 2024-04-25T10:10:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d60a3d640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d48066f60 0x7f5d48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d60a3d640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d48066f60 0x7f5d48069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c15e750 tx=0x7f5d50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d6660f000 1 Processor -- start 2024-04-25T10:10:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.994+0000 7f5d6660f000 1 -- start start 2024-04-25T10:10:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848db3f0 0x5648848db7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848dbd10 0x564884891fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884892900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648846dfd50 con 0x564884892520 2024-04-25T10:10:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564884667e40 con 0x5648848db3f0 2024-04-25T10:10:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648846dbdf0 con 0x5648848dbd10 2024-04-25T10:10:53.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884892900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884892900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52690/0 (socket says 172.21.15.133:52690) 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 -- 172.21.15.133:0/2385679516 learned_addr learned my addr 172.21.15.133:0/2385679516 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6123e640 1 --2- 172.21.15.133:0/2385679516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848db3f0 0x5648848db7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d60a3d640 1 --2- 172.21.15.133:0/2385679516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848dbd10 0x564884891fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 -- 172.21.15.133:0/2385679516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848dbd10 msgr2=0x564884891fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 --2- 172.21.15.133:0/2385679516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848dbd10 0x564884891fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 -- 172.21.15.133:0/2385679516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848db3f0 msgr2=0x5648848db7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 --2- 172.21.15.133:0/2385679516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848db3f0 0x5648848db7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 -- 172.21.15.133:0/2385679516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56488467cd90 con 0x564884892520 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6123e640 1 --2- 172.21.15.133:0/2385679516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848db3f0 0x5648848db7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 --2- 172.21.15.133:0/2385679516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884892900 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00b8c0 tx=0x7f5d4c00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418b1db02268eeb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d42ffd640 1 -- 172.21.15.133:0/2385679516 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c016020 con 0x564884892520 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d42ffd640 1 -- 172.21.15.133:0/2385679516 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c004440 con 0x564884892520 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d42ffd640 1 -- 172.21.15.133:0/2385679516 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00d980 con 0x564884892520 2024-04-25T10:10:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- 172.21.15.133:0/2385679516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 msgr2=0x564884892900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2385679516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884892900 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00b8c0 tx=0x7f5d4c00bd80 comp rx=0 tx=0).stop 2024-04-25T10:10:53.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- 172.21.15.133:0/2385679516 shutdown_connections 2024-04-25T10:10:53.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2385679516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884892900 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2385679516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848dbd10 0x564884891fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2385679516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848db3f0 0x5648848db7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- 172.21.15.133:0/2385679516 >> 172.21.15.133:0/2385679516 conn(0x56488488e4d0 msgr2=0x5648848908c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- 172.21.15.133:0/2385679516 shutdown_connections 2024-04-25T10:10:53.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- 172.21.15.133:0/2385679516 wait complete. 2024-04-25T10:10:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- start start 2024-04-25T10:10:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 Processor -- start 2024-04-25T10:10:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564884892520 0x564884898020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848db3f0 0x5648848c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564884685610 con 0x5648848dbd10 2024-04-25T10:10:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56488469a9a0 con 0x564884892520 2024-04-25T10:10:53.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648846e4380 con 0x5648848db3f0 2024-04-25T10:10:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6660f000 10 client.0 _ll_register_callbacks cb 0x5648847940d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52694/0 (socket says 172.21.15.133:52694) 2024-04-25T10:10:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d61a3f640 1 -- 172.21.15.133:0/2307374463 learned_addr learned my addr 172.21.15.133:0/2307374463 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d60a3d640 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848db3f0 0x5648848c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:52.998+0000 7f5d6123e640 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564884892520 0x564884898020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d61a3f640 1 -- 172.21.15.133:0/2307374463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848db3f0 msgr2=0x5648848c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d61a3f640 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848db3f0 0x5648848c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d61a3f640 1 -- 172.21.15.133:0/2307374463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564884892520 msgr2=0x564884898020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d61a3f640 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564884892520 0x564884898020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d61a3f640 1 -- 172.21.15.133:0/2307374463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648846dad40 con 0x5648848dbd10 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d61a3f640 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848c9600 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00bdc0 tx=0x7f5d4c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49a0892e6ec27e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d37fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d37fff640 1 -- 172.21.15.133:0/2307374463 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c02b060 con 0x5648848dbd10 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d37fff640 1 -- 172.21.15.133:0/2307374463 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f5d4c004240 con 0x5648848dbd10 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 -- 172.21.15.133:0/2307374463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d4c004440 con 0x5648848dbd10 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d37fff640 1 -- 172.21.15.133:0/2307374463 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d4c031050 con 0x5648848dbd10 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 10 client.8052 fetch_fsmap learned FSMap version 58 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 -- 172.21.15.133:0/2307374463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x564884762f00 con 0x5648848dbd10 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d37fff640 1 -- 172.21.15.133:0/2307374463 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c004440 con 0x5648848dbd10 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d37fff640 1 -- 172.21.15.133:0/2307374463 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d4c034020 con 0x5648848dbd10 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 10 client.8052 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 10 client.8052 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 20 client.8052 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5648848e2570 0x5648848e2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 4 client.8052 mds_command: new command op to 7869 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 -- 172.21.15.133:0/2307374463 --> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56488467ba20 con 0x5648848e2570 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6123e640 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5648848e2570 0x5648848e2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d37fff640 10 client.8052 ms_handle_connect on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6123e640 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5648848e2570 0x5648848e2950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18ab18f6af6d759 server_cookie=84c9b1641815c6a8 in_seq=0 out_seq=0 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d37fff640 1 -- 172.21.15.133:0/2307374463 <== mds.0 v2:172.21.15.133:6835/3580592279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56488467ba20 con 0x5648848e2570 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d37fff640 10 client.8052 handle_command_reply: tid=0 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 client.8052 shutdown 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 -- 172.21.15.133:0/2307374463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 msgr2=0x5648848c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848c9600 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00bdc0 tx=0x7f5d4c004740 comp rx=0 tx=0).stop 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 -- 172.21.15.133:0/2307374463 shutdown_connections 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5648848e2570 0x5648848e2950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848c9600 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848db3f0 0x5648848c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2307374463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564884892520 0x564884898020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d37fff640 0 client.8052 ms_handle_reset on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 -- 172.21.15.133:0/2307374463 >> 172.21.15.133:0/2307374463 conn(0x56488488e4d0 msgr2=0x564884891150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 -- 172.21.15.133:0/2307374463 shutdown_connections 2024-04-25T10:10:53.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 1 -- 172.21.15.133:0/2307374463 wait complete. 2024-04-25T10:10:53.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.002+0000 7f5d6660f000 20 client.8052 trim_cache size 0 max 16384 2024-04-25T10:10:53.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 Processor -- start 2024-04-25T10:10:53.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 -- start start 2024-04-25T10:10:53.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884891e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848db3f0 0x5648848908c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848dbd10 0x564884890e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c203180 con 0x564884892520 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0b7080 con 0x5648848dbd10 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c15d880 con 0x5648848db3f0 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6123e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884891e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6123e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884891e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52708/0 (socket says 172.21.15.133:52708) 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6123e640 1 -- 172.21.15.133:0/2160529271 learned_addr learned my addr 172.21.15.133:0/2160529271 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6123e640 1 -- 172.21.15.133:0/2160529271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848db3f0 msgr2=0x5648848908c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6123e640 1 --2- 172.21.15.133:0/2160529271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848db3f0 0x5648848908c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6123e640 1 -- 172.21.15.133:0/2160529271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848dbd10 msgr2=0x564884890e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6123e640 1 --2- 172.21.15.133:0/2160529271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848dbd10 0x564884890e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6123e640 1 -- 172.21.15.133:0/2160529271 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648848dbb40 con 0x564884892520 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6123e640 1 --2- 172.21.15.133:0/2160529271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884891e40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f5d54097e80 tx=0x7f5d5411cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936b77c31b5bca20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d427fc640 1 -- 172.21.15.133:0/2160529271 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54106e60 con 0x564884892520 2024-04-25T10:10:53.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d427fc640 1 -- 172.21.15.133:0/2160529271 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d540f77e0 con 0x564884892520 2024-04-25T10:10:53.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d427fc640 1 -- 172.21.15.133:0/2160529271 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54106040 con 0x564884892520 2024-04-25T10:10:53.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 -- 172.21.15.133:0/2160529271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 msgr2=0x564884891e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2160529271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884891e40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f5d54097e80 tx=0x7f5d5411cbc0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 -- 172.21.15.133:0/2160529271 shutdown_connections 2024-04-25T10:10:53.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2160529271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848dbd10 0x564884890e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2160529271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5648848db3f0 0x5648848908c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 --2- 172.21.15.133:0/2160529271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564884892520 0x564884891e40 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 -- 172.21.15.133:0/2160529271 >> 172.21.15.133:0/2160529271 conn(0x56488488e4d0 msgr2=0x564884897470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 -- 172.21.15.133:0/2160529271 shutdown_connections 2024-04-25T10:10:53.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.018+0000 7f5d6660f000 1 -- 172.21.15.133:0/2160529271 wait complete. 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 -- start start 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 Processor -- start 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564884892520 0x5648848e0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848db3f0 0x5648848e0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564884696dc0 con 0x5648848dbd10 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0b6a80 con 0x5648848db3f0 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648846df060 con 0x564884892520 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 10 client.0 _ll_register_callbacks cb 0x7f5d4c00b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d61a3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d61a3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848e0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52716/0 (socket says 172.21.15.133:52716) 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d61a3f640 1 -- 172.21.15.133:0/1415061354 learned_addr learned my addr 172.21.15.133:0/1415061354 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6123e640 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564884892520 0x5648848e0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d60a3d640 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848db3f0 0x5648848e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d61a3f640 1 -- 172.21.15.133:0/1415061354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564884892520 msgr2=0x5648848e0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d61a3f640 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564884892520 0x5648848e0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d61a3f640 1 -- 172.21.15.133:0/1415061354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848db3f0 msgr2=0x5648848e0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d61a3f640 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848db3f0 0x5648848e0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d61a3f640 1 -- 172.21.15.133:0/1415061354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564884739330 con 0x5648848dbd10 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d61a3f640 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848e0ea0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00de20 tx=0x7f5d4c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13860fe53f1ac2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d37fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d37fff640 1 -- 172.21.15.133:0/1415061354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c03a060 con 0x5648848dbd10 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d37fff640 1 -- 172.21.15.133:0/1415061354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f5d4c003950 con 0x5648848dbd10 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 -- 172.21.15.133:0/1415061354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d540f77e0 con 0x5648848dbd10 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d37fff640 1 -- 172.21.15.133:0/1415061354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d4c040050 con 0x5648848dbd10 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 10 client.8058 fetch_fsmap learned FSMap version 58 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 -- 172.21.15.133:0/1415061354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x5648846ac5f0 con 0x5648848dbd10 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d37fff640 1 -- 172.21.15.133:0/1415061354 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d540f77e0 con 0x5648848dbd10 2024-04-25T10:10:53.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d37fff640 1 -- 172.21.15.133:0/1415061354 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d4c043020 con 0x5648848dbd10 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 10 client.8058 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 10 client.8058 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 20 client.8058 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5648848e13e0 0x5648848e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 4 client.8058 mds_command: new command op to 7869 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6660f000 1 -- 172.21.15.133:0/1415061354 --> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5648846ed290 con 0x5648848e13e0 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6123e640 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5648848e13e0 0x5648848e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d6123e640 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5648848e13e0 0x5648848e17f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ba926716dbbcef server_cookie=177b3b74d90c688f in_seq=0 out_seq=0 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.022+0000 7f5d37fff640 10 client.8058 ms_handle_connect on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d37fff640 1 -- 172.21.15.133:0/1415061354 <== mds.0 v2:172.21.15.133:6835/3580592279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5648846ed290 con 0x5648848e13e0 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d37fff640 10 client.8058 handle_command_reply: tid=0 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 1 client.8058 shutdown 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 1 -- 172.21.15.133:0/1415061354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 msgr2=0x5648848e0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848e0ea0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00de20 tx=0x7f5d4c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 1 -- 172.21.15.133:0/1415061354 shutdown_connections 2024-04-25T10:10:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5648848e13e0 0x5648848e17f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5648848dbd10 0x5648848e0ea0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5648848db3f0 0x5648848e0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 1 --2- 172.21.15.133:0/1415061354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564884892520 0x5648848e0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 1 -- 172.21.15.133:0/1415061354 >> 172.21.15.133:0/1415061354 conn(0x56488488e4d0 msgr2=0x5648848da090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d37fff640 0 client.8058 ms_handle_reset on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 1 -- 172.21.15.133:0/1415061354 shutdown_connections 2024-04-25T10:10:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 1 -- 172.21.15.133:0/1415061354 wait complete. 2024-04-25T10:10:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d6660f000 20 client.8058 trim_cache size 0 max 16384 2024-04-25T10:10:53.033 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:10:53.033 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:10:53.033 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "58613dd7-11f2-4b71-ac0c-dc1f1486b1ea", 2024-04-25T10:10:53.033 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:10:53.033 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 -- 172.21.15.133:0/3547947829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d48066f60 msgr2=0x7f5d48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d48066f60 0x7f5d48069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c15e750 tx=0x7f5d50009040 comp rx=0 tx=0).stop 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 -- 172.21.15.133:0/3547947829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5c0ccd50 msgr2=0x7f5d5c154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5c0ccd50 0x7f5d5c154ea0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5d540097f0 tx=0x7f5d5401ace0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 -- 172.21.15.133:0/3547947829 shutdown_connections 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5c15ef20 0x7f5d5c0ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5c0cd1e0 0x7f5d5c09e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d48066f60 0x7f5d48069420 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 --2- 172.21.15.133:0/3547947829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5c0ccd50 0x7f5d5c154ea0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 -- 172.21.15.133:0/3547947829 >> 172.21.15.133:0/3547947829 conn(0x7f5d5c0b3220 msgr2=0x7f5d5c158010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 -- 172.21.15.133:0/3547947829 shutdown_connections 2024-04-25T10:10:53.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.026+0000 7f5d634a2640 1 -- 172.21.15.133:0/3547947829 wait complete. 2024-04-25T10:10:53.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:53.053 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.186+0000 7fcefc3bc640 1 Processor -- start 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.186+0000 7fcefc3bc640 1 -- start start 2024-04-25T10:10:53.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.186+0000 7fcefc3bc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef415d9a0 0x7fcef40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.186+0000 7fcefc3bc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcef40d2030 0x7fcef40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.186+0000 7fcefc3bc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef40d24c0 con 0x7fcef40ce600 2024-04-25T10:10:53.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.186+0000 7fcefc3bc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef40cefa0 con 0x7fcef40d2030 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.186+0000 7fcefc3bc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef415ebd0 con 0x7fcef415d9a0 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.186+0000 7fcefa959640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcef40d2030 0x7fcef40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.186+0000 7fcef9957640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcef40ce600 0x7fcef40d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54604/0 (socket says 172.21.15.133:54604) 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.186+0000 7fcef9957640 1 -- 172.21.15.133:0/828102482 learned_addr learned my addr 172.21.15.133:0/828102482 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.186+0000 7fcefa158640 1 --2- 172.21.15.133:0/828102482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef415d9a0 0x7fcef40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefa959640 1 -- 172.21.15.133:0/828102482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef415d9a0 msgr2=0x7fcef40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefa959640 1 --2- 172.21.15.133:0/828102482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef415d9a0 0x7fcef40d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefa959640 1 -- 172.21.15.133:0/828102482 >> v1:172.21.15.133:6789/0 conn(0x7fcef40ce600 legacy=0x7fcef40d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcef9156640 1 -- 172.21.15.133:0/828102482 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932555081 0 0) 0x7fcef40d24c0 con 0x7fcef40ce600 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefa959640 1 -- 172.21.15.133:0/828102482 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef415eeb0 con 0x7fcef40d2030 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefa158640 1 --2- 172.21.15.133:0/828102482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef415d9a0 0x7fcef40d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefa959640 1 --2- 172.21.15.133:0/828102482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef40d2030 0x7fcef40cea60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fceec009d90 tx=0x7fceec01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f936f4bf299d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcef9156640 1 -- 172.21.15.133:0/828102482 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0b2070 con 0x7fcef40d2030 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcef9156640 1 -- 172.21.15.133:0/828102482 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec0a5970 con 0x7fcef40d2030 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcef9156640 1 -- 172.21.15.133:0/828102482 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0a5d40 con 0x7fcef40d2030 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefc3bc640 1 -- 172.21.15.133:0/828102482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef40d2030 msgr2=0x7fcef40cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefc3bc640 1 --2- 172.21.15.133:0/828102482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef40d2030 0x7fcef40cea60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fceec009d90 tx=0x7fceec01c8b0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefc3bc640 1 -- 172.21.15.133:0/828102482 shutdown_connections 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefc3bc640 1 --2- 172.21.15.133:0/828102482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef40d2030 0x7fcef40cea60 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefc3bc640 1 --2- 172.21.15.133:0/828102482 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef415d9a0 0x7fcef40d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefc3bc640 1 -- 172.21.15.133:0/828102482 >> 172.21.15.133:0/828102482 conn(0x7fcef406bf80 msgr2=0x7fcef406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefc3bc640 1 -- 172.21.15.133:0/828102482 shutdown_connections 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.190+0000 7fcefc3bc640 1 -- 172.21.15.133:0/828102482 wait complete. 2024-04-25T10:10:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefc3bc640 1 Processor -- start 2024-04-25T10:10:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefc3bc640 1 -- start start 2024-04-25T10:10:53.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefc3bc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef40ce600 0x7fcef4157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefc3bc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef40d2030 0x7fcef41567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefc3bc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef415d9a0 0x7fcef4167980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefc3bc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40b96a0 con 0x7fcef415d9a0 2024-04-25T10:10:53.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefc3bc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40cffd0 con 0x7fcef40ce600 2024-04-25T10:10:53.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefc3bc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef40b9820 con 0x7fcef40d2030 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefa959640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef415d9a0 0x7fcef4167980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefa959640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef415d9a0 0x7fcef4167980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52726/0 (socket says 172.21.15.133:52726) 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefa959640 1 -- 172.21.15.133:0/404767968 learned_addr learned my addr 172.21.15.133:0/404767968 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcef9957640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef40d2030 0x7fcef41567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefa959640 1 -- 172.21.15.133:0/404767968 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef40d2030 msgr2=0x7fcef41567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefa959640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef40d2030 0x7fcef41567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefa158640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef40ce600 0x7fcef4157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefa959640 1 -- 172.21.15.133:0/404767968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef40ce600 msgr2=0x7fcef4157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefa959640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef40ce600 0x7fcef4157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefa959640 1 -- 172.21.15.133:0/404767968 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef40cae00 con 0x7fcef415d9a0 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefa959640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef415d9a0 0x7fcef4167980 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fceec0b0040 tx=0x7fceec019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fceeaffd640 1 -- 172.21.15.133:0/404767968 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0b2050 con 0x7fcef415d9a0 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fceeaffd640 1 -- 172.21.15.133:0/404767968 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec0be070 con 0x7fcef415d9a0 2024-04-25T10:10:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefc3bc640 1 -- 172.21.15.133:0/404767968 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef40cb0e0 con 0x7fcef415d9a0 2024-04-25T10:10:53.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fceeaffd640 1 -- 172.21.15.133:0/404767968 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0adb50 con 0x7fcef415d9a0 2024-04-25T10:10:53.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefc3bc640 1 -- 172.21.15.133:0/404767968 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef40cb6a0 con 0x7fcef415d9a0 2024-04-25T10:10:53.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fceeaffd640 1 -- 172.21.15.133:0/404767968 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fceec0adcf0 con 0x7fcef415d9a0 2024-04-25T10:10:53.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fceeaffd640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fced8066f60 0x7fced8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fceeaffd640 1 -- 172.21.15.133:0/404767968 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fceec1205c0 con 0x7fcef415d9a0 2024-04-25T10:10:53.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.194+0000 7fcefa158640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fced8066f60 0x7fced8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fcefa158640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fced8066f60 0x7fced8069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcef0006fd0 tx=0x7fcef0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 Processor -- start 2024-04-25T10:10:53.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 -- start start 2024-04-25T10:10:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecba40 0x56371cecbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 0x56371ce86850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ce87170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371cd691f0 con 0x56371ce86d90 2024-04-25T10:10:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371cccfe90 con 0x56371cecba40 2024-04-25T10:10:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56371cc57f50 con 0x56371cecc360 2024-04-25T10:10:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fcefa959640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ce87170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fcefa959640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ce87170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52732/0 (socket says 172.21.15.133:52732) 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fcefa959640 1 -- 172.21.15.133:0/4159714510 learned_addr learned my addr 172.21.15.133:0/4159714510 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fcefa959640 1 -- 172.21.15.133:0/4159714510 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 msgr2=0x56371ce86850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fcefa959640 1 --2- 172.21.15.133:0/4159714510 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 0x56371ce86850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fcefa959640 1 -- 172.21.15.133:0/4159714510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecba40 msgr2=0x56371cecbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fcefa959640 1 --2- 172.21.15.133:0/4159714510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecba40 0x56371cecbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fcefa959640 1 -- 172.21.15.133:0/4159714510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56371cca78b0 con 0x56371ce86d90 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fcefa959640 1 --2- 172.21.15.133:0/4159714510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ce87170 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fceec0ed1c0 tx=0x7fceec0f0c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc73566a1ea387b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fced7fff640 1 -- 172.21.15.133:0/4159714510 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0f0d60 con 0x56371ce86d90 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fced7fff640 1 -- 172.21.15.133:0/4159714510 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec0b3040 con 0x56371ce86d90 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fced7fff640 1 -- 172.21.15.133:0/4159714510 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0f4eb0 con 0x56371ce86d90 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 -- 172.21.15.133:0/4159714510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 msgr2=0x56371ce87170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 --2- 172.21.15.133:0/4159714510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ce87170 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fceec0ed1c0 tx=0x7fceec0f0c60 comp rx=0 tx=0).stop 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 -- 172.21.15.133:0/4159714510 shutdown_connections 2024-04-25T10:10:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 --2- 172.21.15.133:0/4159714510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ce87170 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 --2- 172.21.15.133:0/4159714510 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 0x56371ce86850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 --2- 172.21.15.133:0/4159714510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecba40 0x56371cecbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 -- 172.21.15.133:0/4159714510 >> 172.21.15.133:0/4159714510 conn(0x56371ce82d40 msgr2=0x56371ce85130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 -- 172.21.15.133:0/4159714510 shutdown_connections 2024-04-25T10:10:53.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 -- 172.21.15.133:0/4159714510 wait complete. 2024-04-25T10:10:53.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 -- start start 2024-04-25T10:10:53.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 Processor -- start 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ced31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecba40 0x56371ced3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.198+0000 7fceff529000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 0x56371ce8ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371cc8a540 con 0x56371ce86d90 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371cccbf30 con 0x56371cecba40 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56371cc7b260 con 0x56371cecc360 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 10 client.0 _ll_register_callbacks cb 0x56371cd84180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcefa158640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ced31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcefa158640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ced31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52738/0 (socket says 172.21.15.133:52738) 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcefa158640 1 -- 172.21.15.133:0/2697768226 learned_addr learned my addr 172.21.15.133:0/2697768226 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcef9957640 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecba40 0x56371ced3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcefa158640 1 -- 172.21.15.133:0/2697768226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 msgr2=0x56371ce8ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcefa959640 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 0x56371ce8ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcefa158640 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 0x56371ce8ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcefa158640 1 -- 172.21.15.133:0/2697768226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecba40 msgr2=0x56371ced3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcefa158640 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecba40 0x56371ced3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcefa158640 1 -- 172.21.15.133:0/2697768226 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56371cc581a0 con 0x56371ce86d90 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcef9957640 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecba40 0x56371ced3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcefa158640 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ced31c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fcef00126f0 tx=0x7fcef0012720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4c4472c91302f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fced57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fced57fa640 1 -- 172.21.15.133:0/2697768226 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0023060 con 0x56371ce86d90 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fced57fa640 1 -- 172.21.15.133:0/2697768226 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fcef001e600 con 0x56371ce86d90 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 -- 172.21.15.133:0/2697768226 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fceec0b3040 con 0x56371ce86d90 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fced57fa640 1 -- 172.21.15.133:0/2697768226 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcef0029050 con 0x56371ce86d90 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 10 client.8073 fetch_fsmap learned FSMap version 58 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 -- 172.21.15.133:0/2697768226 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x56371cd52840 con 0x56371ce86d90 2024-04-25T10:10:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fced57fa640 1 -- 172.21.15.133:0/2697768226 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec0b3040 con 0x56371ce86d90 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fced57fa640 1 -- 172.21.15.133:0/2697768226 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcef002c020 con 0x56371ce86d90 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 10 client.8073 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 10 client.8073 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 20 client.8073 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x56371ce8dbd0 0x56371ce8dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 4 client.8073 mds_command: new command op to 7869 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 -- 172.21.15.133:0/2697768226 --> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56371cc6bbe0 con 0x56371ce8dbd0 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcef9957640 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x56371ce8dbd0 0x56371ce8dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fced57fa640 10 client.8073 ms_handle_connect on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fcef9957640 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x56371ce8dbd0 0x56371ce8dfb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eae49db6e2cfca9 server_cookie=66347184e1414ee1 in_seq=0 out_seq=0 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fced57fa640 1 -- 172.21.15.133:0/2697768226 <== mds.0 v2:172.21.15.133:6835/3580592279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56371cc6bbe0 con 0x56371ce8dbd0 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fced57fa640 10 client.8073 handle_command_reply: tid=0 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 client.8073 shutdown 2024-04-25T10:10:53.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 -- 172.21.15.133:0/2697768226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 msgr2=0x56371ced31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ced31c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fcef00126f0 tx=0x7fcef0012720 comp rx=0 tx=0).stop 2024-04-25T10:10:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 -- 172.21.15.133:0/2697768226 shutdown_connections 2024-04-25T10:10:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 0x56371ce8ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecba40 0x56371ced3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x56371ce8dbd0 0x56371ce8dfb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 --2- 172.21.15.133:0/2697768226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371ce86d90 0x56371ced31c0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 -- 172.21.15.133:0/2697768226 >> 172.21.15.133:0/2697768226 conn(0x56371ce82d40 msgr2=0x56371ce859c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fced57fa640 0 client.8073 ms_handle_reset on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 -- 172.21.15.133:0/2697768226 shutdown_connections 2024-04-25T10:10:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 1 -- 172.21.15.133:0/2697768226 wait complete. 2024-04-25T10:10:53.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.202+0000 7fceff529000 20 client.8073 trim_cache size 0 max 16384 2024-04-25T10:10:53.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 Processor -- start 2024-04-25T10:10:53.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 -- start start 2024-04-25T10:10:53.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371ce86d90 0x56371cf27e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371cf283b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 0x56371cf288f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef4202ee0 con 0x56371cecba40 2024-04-25T10:10:53.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef4202180 con 0x56371ce86d90 2024-04-25T10:10:53.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef415cd30 con 0x56371cecc360 2024-04-25T10:10:53.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fcef9957640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371cf283b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fcef9957640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371cf283b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52740/0 (socket says 172.21.15.133:52740) 2024-04-25T10:10:53.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fcef9957640 1 -- 172.21.15.133:0/124983543 learned_addr learned my addr 172.21.15.133:0/124983543 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fcef9957640 1 -- 172.21.15.133:0/124983543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 msgr2=0x56371cf288f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fcef9957640 1 --2- 172.21.15.133:0/124983543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 0x56371cf288f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fcef9957640 1 -- 172.21.15.133:0/124983543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371ce86d90 msgr2=0x56371cf27e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fcef9957640 1 --2- 172.21.15.133:0/124983543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371ce86d90 0x56371cf27e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fcef9957640 1 -- 172.21.15.133:0/124983543 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56371ce82a50 con 0x56371cecba40 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fcef9957640 1 --2- 172.21.15.133:0/124983543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371cf283b0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fcee4000fc0 tx=0x7fcee4015840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16df5460e4e7b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fced77fe640 1 -- 172.21.15.133:0/124983543 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee40070e0 con 0x56371cecba40 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fced77fe640 1 -- 172.21.15.133:0/124983543 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee4007280 con 0x56371cecba40 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fced77fe640 1 -- 172.21.15.133:0/124983543 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4007630 con 0x56371cecba40 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 -- 172.21.15.133:0/124983543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 msgr2=0x56371cf283b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 --2- 172.21.15.133:0/124983543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371cf283b0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fcee4000fc0 tx=0x7fcee4015840 comp rx=0 tx=0).stop 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 -- 172.21.15.133:0/124983543 shutdown_connections 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 --2- 172.21.15.133:0/124983543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371cecc360 0x56371cf288f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 --2- 172.21.15.133:0/124983543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371cf283b0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 --2- 172.21.15.133:0/124983543 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371ce86d90 0x56371cf27e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 -- 172.21.15.133:0/124983543 >> 172.21.15.133:0/124983543 conn(0x56371ce82d40 msgr2=0x56371ce85130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 -- 172.21.15.133:0/124983543 shutdown_connections 2024-04-25T10:10:53.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 -- 172.21.15.133:0/124983543 wait complete. 2024-04-25T10:10:53.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 -- start start 2024-04-25T10:10:53.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.218+0000 7fceff529000 1 Processor -- start 2024-04-25T10:10:53.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371ce86d90 0x56371ced0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371ced11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecc360 0x56371ced1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371cc76c00 con 0x56371cecba40 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef4159a60 con 0x56371cecc360 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56371ce6d5b0 con 0x56371ce86d90 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 10 client.0 _ll_register_callbacks cb 0x56371cc92a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcef9957640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371ced11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcef9957640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371ced11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52748/0 (socket says 172.21.15.133:52748) 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcef9957640 1 -- 172.21.15.133:0/1024658707 learned_addr learned my addr 172.21.15.133:0/1024658707 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefa959640 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecc360 0x56371ced1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcef9957640 1 -- 172.21.15.133:0/1024658707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371ce86d90 msgr2=0x56371ced0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcef9957640 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371ce86d90 0x56371ced0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcef9957640 1 -- 172.21.15.133:0/1024658707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecc360 msgr2=0x56371ced1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcef9957640 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecc360 0x56371ced1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcef9957640 1 -- 172.21.15.133:0/1024658707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56371cd29a10 con 0x56371cecba40 2024-04-25T10:10:53.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcef9957640 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371ced11c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fcee4015880 tx=0x7fcee4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285e56cbf1b027ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fced57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fced57fa640 1 -- 172.21.15.133:0/1024658707 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4007630 con 0x56371cecba40 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fced57fa640 1 -- 172.21.15.133:0/1024658707 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fcee4026a80 con 0x56371cecba40 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 -- 172.21.15.133:0/1024658707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcee4007280 con 0x56371cecba40 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fced57fa640 1 -- 172.21.15.133:0/1024658707 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcee4030050 con 0x56371cecba40 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 10 client.8079 fetch_fsmap learned FSMap version 58 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 -- 172.21.15.133:0/1024658707 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x56371cc64b00 con 0x56371cecba40 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fced57fa640 1 -- 172.21.15.133:0/1024658707 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee4007280 con 0x56371cecba40 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fced57fa640 1 -- 172.21.15.133:0/1024658707 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcee4026c80 con 0x56371cecba40 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 10 client.8079 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 10 client.8079 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 20 client.8079 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x56371cf31cd0 0x56371cf340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 4 client.8079 mds_command: new command op to 7869 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 -- 172.21.15.133:0/1024658707 --> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x56371ccdd3d0 con 0x56371cf31cd0 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefa158640 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x56371cf31cd0 0x56371cf340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefa158640 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x56371cf31cd0 0x56371cf340c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218cd31cc2be89c server_cookie=3113b676c612590a in_seq=0 out_seq=0 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fced57fa640 10 client.8079 ms_handle_connect on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fced57fa640 1 -- 172.21.15.133:0/1024658707 <== mds.0 v2:172.21.15.133:6835/3580592279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x56371ccdd3d0 con 0x56371cf31cd0 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fced57fa640 10 client.8079 handle_command_reply: tid=0 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 client.8079 shutdown 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 -- 172.21.15.133:0/1024658707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 msgr2=0x56371ced11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371ced11c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fcee4015880 tx=0x7fcee4000ee0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 -- 172.21.15.133:0/1024658707 shutdown_connections 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56371cecc360 0x56371ced1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x56371cf31cd0 0x56371cf340c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56371cecba40 0x56371ced11c0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 --2- 172.21.15.133:0/1024658707 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56371ce86d90 0x56371ced0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fced57fa640 0 client.8079 ms_handle_reset on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 -- 172.21.15.133:0/1024658707 >> 172.21.15.133:0/1024658707 conn(0x56371ce82d40 msgr2=0x56371ce859c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 -- 172.21.15.133:0/1024658707 shutdown_connections 2024-04-25T10:10:53.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 1 -- 172.21.15.133:0/1024658707 wait complete. 2024-04-25T10:10:53.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fceff529000 20 client.8079 trim_cache size 0 max 16384 2024-04-25T10:10:53.231 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:10:53.231 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:10:53.231 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:10:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 -- 172.21.15.133:0/404767968 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fced8066f60 msgr2=0x7fced8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fced8066f60 0x7fced8069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcef0006fd0 tx=0x7fcef0008040 comp rx=0 tx=0).stop 2024-04-25T10:10:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 -- 172.21.15.133:0/404767968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef415d9a0 msgr2=0x7fcef4167980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef415d9a0 0x7fcef4167980 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fceec0b0040 tx=0x7fceec019650 comp rx=0 tx=0).stop 2024-04-25T10:10:53.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 -- 172.21.15.133:0/404767968 shutdown_connections 2024-04-25T10:10:53.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fced8066f60 0x7fced8069420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef415d9a0 0x7fcef4167980 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcef40d2030 0x7fcef41567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 --2- 172.21.15.133:0/404767968 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcef40ce600 0x7fcef4157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 -- 172.21.15.133:0/404767968 >> 172.21.15.133:0/404767968 conn(0x7fcef406bf80 msgr2=0x7fcef406c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 -- 172.21.15.133:0/404767968 shutdown_connections 2024-04-25T10:10:53.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.222+0000 7fcefc3bc640 1 -- 172.21.15.133:0/404767968 wait complete. 2024-04-25T10:10:53.240 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:10:53.241 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:10:53.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95b6647640 1 Processor -- start 2024-04-25T10:10:53.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95b6647640 1 -- start start 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95b6647640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f95b015d9a0 0x7f95b00d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95b6647640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b00ce600 0x7f95b00cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95b6647640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b00d24c0 con 0x7f95b00d1920 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95b6647640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b00cefa0 con 0x7f95b015d9a0 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95b6647640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b015ebd0 con 0x7f95b00ce600 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95affff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f95b015d9a0 0x7f95b00d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95affff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f95b015d9a0 0x7f95b00d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52758/0 (socket says 172.21.15.133:52758) 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95affff640 1 -- 172.21.15.133:0/743597869 learned_addr learned my addr 172.21.15.133:0/743597869 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95affff640 1 -- 172.21.15.133:0/743597869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b00ce600 msgr2=0x7f95b00cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95affff640 1 --2- 172.21.15.133:0/743597869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b00ce600 0x7f95b00cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95affff640 1 -- 172.21.15.133:0/743597869 >> v1:172.21.15.162:6789/0 conn(0x7f95b00d1920 legacy=0x7f95b00d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95affff640 1 -- 172.21.15.133:0/743597869 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b015eeb0 con 0x7f95b015d9a0 2024-04-25T10:10:53.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95affff640 1 --2- 172.21.15.133:0/743597869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95b015d9a0 0x7f95b00d0b30 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009870 tx=0x7f95a001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2929f92ee0a5c21c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95af7fe640 1 -- 172.21.15.133:0/743597869 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a00ab620 con 0x7f95b015d9a0 2024-04-25T10:10:53.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95af7fe640 1 -- 172.21.15.133:0/743597869 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a00b0070 con 0x7f95b015d9a0 2024-04-25T10:10:53.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.362+0000 7f95af7fe640 1 -- 172.21.15.133:0/743597869 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a00ab9e0 con 0x7f95b015d9a0 2024-04-25T10:10:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 -- 172.21.15.133:0/743597869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95b015d9a0 msgr2=0x7f95b00d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 --2- 172.21.15.133:0/743597869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95b015d9a0 0x7f95b00d0b30 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009870 tx=0x7f95a001c460 comp rx=0 tx=0).stop 2024-04-25T10:10:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 -- 172.21.15.133:0/743597869 shutdown_connections 2024-04-25T10:10:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 --2- 172.21.15.133:0/743597869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b00ce600 0x7f95b00cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 --2- 172.21.15.133:0/743597869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95b015d9a0 0x7f95b00d0b30 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 -- 172.21.15.133:0/743597869 >> 172.21.15.133:0/743597869 conn(0x7f95b006bf80 msgr2=0x7f95b006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 -- 172.21.15.133:0/743597869 shutdown_connections 2024-04-25T10:10:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 -- 172.21.15.133:0/743597869 wait complete. 2024-04-25T10:10:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 Processor -- start 2024-04-25T10:10:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 -- start start 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95b00ce600 0x7f95b00c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95b00d1920 0x7f95b00cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b015d9a0 0x7f95b014bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b96a0 con 0x7f95b00d1920 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00cffd0 con 0x7f95b00ce600 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b9820 con 0x7f95b015d9a0 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95a7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95b00d1920 0x7f95b00cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95a7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95b00d1920 0x7f95b00cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52762/0 (socket says 172.21.15.133:52762) 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95a7fff640 1 -- 172.21.15.133:0/3412001939 learned_addr learned my addr 172.21.15.133:0/3412001939 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b4be4640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b015d9a0 0x7f95b014bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95affff640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95b00ce600 0x7f95b00c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95a7fff640 1 -- 172.21.15.133:0/3412001939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b015d9a0 msgr2=0x7f95b014bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95a7fff640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b015d9a0 0x7f95b014bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95a7fff640 1 -- 172.21.15.133:0/3412001939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95b00ce600 msgr2=0x7f95b00c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95a7fff640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95b00ce600 0x7f95b00c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95a7fff640 1 -- 172.21.15.133:0/3412001939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b00cd830 con 0x7f95b00d1920 2024-04-25T10:10:53.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95a7fff640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95b00d1920 0x7f95b00cd2f0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f959800b550 tx=0x7f959800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95ad7fa640 1 -- 172.21.15.133:0/3412001939 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598014070 con 0x7f95b00d1920 2024-04-25T10:10:53.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95ad7fa640 1 -- 172.21.15.133:0/3412001939 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9598010890 con 0x7f95b00d1920 2024-04-25T10:10:53.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 -- 172.21.15.133:0/3412001939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b00c8e60 con 0x7f95b00d1920 2024-04-25T10:10:53.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95ad7fa640 1 -- 172.21.15.133:0/3412001939 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598010c80 con 0x7f95b00d1920 2024-04-25T10:10:53.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.366+0000 7f95b6647640 1 -- 172.21.15.133:0/3412001939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b00c9700 con 0x7f95b00d1920 2024-04-25T10:10:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.370+0000 7f95ad7fa640 1 -- 172.21.15.133:0/3412001939 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9598017070 con 0x7f95b00d1920 2024-04-25T10:10:53.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.370+0000 7f95ad7fa640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9590066f60 0x7f9590069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.370+0000 7f95ad7fa640 1 -- 172.21.15.133:0/3412001939 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f9598084fe0 con 0x7f95b00d1920 2024-04-25T10:10:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.370+0000 7f95affff640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9590066f60 0x7f9590069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.370+0000 7f95b97b4000 1 Processor -- start 2024-04-25T10:10:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.370+0000 7f95affff640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9590066f60 0x7f9590069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f95a00af000 tx=0x7f95a00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.370+0000 7f95b97b4000 1 -- start start 2024-04-25T10:10:53.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.370+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d286fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.370+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28702f0 0x5642d2825b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.370+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28260a0 0x5642d2826480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.370+0000 7f95b97b4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d2672ff0 con 0x5642d28702f0 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d25fb150 con 0x5642d286f9d0 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642d266f090 con 0x5642d28260a0 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28702f0 0x5642d2825b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28702f0 0x5642d2825b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52764/0 (socket says 172.21.15.133:52764) 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a7fff640 1 -- 172.21.15.133:0/3138220413 learned_addr learned my addr 172.21.15.133:0/3138220413 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95affff640 1 --2- 172.21.15.133:0/3138220413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d286fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b4be4640 1 --2- 172.21.15.133:0/3138220413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28260a0 0x5642d2826480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a7fff640 1 -- 172.21.15.133:0/3138220413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28260a0 msgr2=0x5642d2826480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a7fff640 1 --2- 172.21.15.133:0/3138220413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28260a0 0x5642d2826480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a7fff640 1 -- 172.21.15.133:0/3138220413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 msgr2=0x5642d286fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a7fff640 1 --2- 172.21.15.133:0/3138220413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d286fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a7fff640 1 -- 172.21.15.133:0/3138220413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642d28090d0 con 0x5642d28702f0 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a7fff640 1 --2- 172.21.15.133:0/3138220413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28702f0 0x5642d2825b60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f9598051b50 tx=0x7f9598059840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a572106b3167db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a5ffb640 1 -- 172.21.15.133:0/3138220413 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959801e040 con 0x5642d28702f0 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a5ffb640 1 -- 172.21.15.133:0/3138220413 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959806d1c0 con 0x5642d28702f0 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a5ffb640 1 -- 172.21.15.133:0/3138220413 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598051d60 con 0x5642d28702f0 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 -- 172.21.15.133:0/3138220413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28702f0 msgr2=0x5642d2825b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 --2- 172.21.15.133:0/3138220413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28702f0 0x5642d2825b60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f9598051b50 tx=0x7f9598059840 comp rx=0 tx=0).stop 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 -- 172.21.15.133:0/3138220413 shutdown_connections 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 --2- 172.21.15.133:0/3138220413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28260a0 0x5642d2826480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 --2- 172.21.15.133:0/3138220413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28702f0 0x5642d2825b60 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 --2- 172.21.15.133:0/3138220413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d286fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 -- 172.21.15.133:0/3138220413 >> 172.21.15.133:0/3138220413 conn(0x5642d2822020 msgr2=0x5642d2824440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 -- 172.21.15.133:0/3138220413 shutdown_connections 2024-04-25T10:10:53.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 -- 172.21.15.133:0/3138220413 wait complete. 2024-04-25T10:10:53.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 -- start start 2024-04-25T10:10:53.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 Processor -- start 2024-04-25T10:10:53.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d2854d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 0x5642d2855270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d2637250 con 0x5642d28260a0 2024-04-25T10:10:53.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d262d690 con 0x5642d286f9d0 2024-04-25T10:10:53.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642d25fba40 con 0x5642d28702f0 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95affff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b97b4000 10 client.0 _ll_register_callbacks cb 0x5642d2810e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95affff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52772/0 (socket says 172.21.15.133:52772) 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95affff640 1 -- 172.21.15.133:0/1514879493 learned_addr learned my addr 172.21.15.133:0/1514879493 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95a7fff640 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d2854d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95b4be4640 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 0x5642d2855270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95affff640 1 -- 172.21.15.133:0/1514879493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 msgr2=0x5642d2855270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95affff640 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 0x5642d2855270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95affff640 1 -- 172.21.15.133:0/1514879493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 msgr2=0x5642d2854d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95affff640 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d2854d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.374+0000 7f95affff640 1 -- 172.21.15.133:0/1514879493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642d27201c0 con 0x5642d28260a0 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95affff640 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28547f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009870 tx=0x7f95a001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c720f04f6a81823a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f958effd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f958effd640 1 -- 172.21.15.133:0/1514879493 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a00cc060 con 0x5642d28260a0 2024-04-25T10:10:53.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f958effd640 1 -- 172.21.15.133:0/1514879493 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f95a00abe00 con 0x5642d28260a0 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 -- 172.21.15.133:0/1514879493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f959806d1c0 con 0x5642d28260a0 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f958effd640 1 -- 172.21.15.133:0/1514879493 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95a00d2050 con 0x5642d28260a0 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 10 client.8091 fetch_fsmap learned FSMap version 58 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 -- 172.21.15.133:0/1514879493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x5642d271c170 con 0x5642d28260a0 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f958effd640 1 -- 172.21.15.133:0/1514879493 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a00b0040 con 0x5642d28260a0 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f958effd640 1 -- 172.21.15.133:0/1514879493 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f95a00d5020 con 0x5642d28260a0 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 10 client.8091 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 10 client.8091 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 20 client.8091 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5642d287fc80 0x5642d2882070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 4 client.8091 mds_command: new command op to 7869 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 -- 172.21.15.133:0/1514879493 --> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f959806d1c0 con 0x5642d287fc80 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95a7fff640 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5642d287fc80 0x5642d2882070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f958effd640 10 client.8091 ms_handle_connect on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95a7fff640 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5642d287fc80 0x5642d2882070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4fae3f09354aca server_cookie=2a8ff9c67a6f75d6 in_seq=0 out_seq=0 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f958effd640 1 -- 172.21.15.133:0/1514879493 <== mds.0 v2:172.21.15.133:6835/3580592279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f959806d1c0 con 0x5642d287fc80 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f958effd640 10 client.8091 handle_command_reply: tid=0 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 client.8091 shutdown 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 -- 172.21.15.133:0/1514879493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 msgr2=0x5642d28547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28547f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009870 tx=0x7f95a001aca0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 -- 172.21.15.133:0/1514879493 shutdown_connections 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 0x5642d2855270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d2854d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5642d287fc80 0x5642d2882070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 --2- 172.21.15.133:0/1514879493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28547f0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f958effd640 0 client.8091 ms_handle_reset on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 -- 172.21.15.133:0/1514879493 >> 172.21.15.133:0/1514879493 conn(0x5642d2822020 msgr2=0x5642d2824ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 -- 172.21.15.133:0/1514879493 shutdown_connections 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 1 -- 172.21.15.133:0/1514879493 wait complete. 2024-04-25T10:10:53.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.378+0000 7f95b97b4000 20 client.8091 trim_cache size 0 max 16384 2024-04-25T10:10:53.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95b97b4000 1 Processor -- start 2024-04-25T10:10:53.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95b97b4000 1 -- start start 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d28cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 0x5642d28ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95b97b4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b02031f0 con 0x5642d28260a0 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95b97b4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b02023e0 con 0x5642d286f9d0 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95b97b4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b01f72d0 con 0x5642d28702f0 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95affff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95affff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52786/0 (socket says 172.21.15.133:52786) 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95affff640 1 -- 172.21.15.133:0/178788835 learned_addr learned my addr 172.21.15.133:0/178788835 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95affff640 1 -- 172.21.15.133:0/178788835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 msgr2=0x5642d28ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95affff640 1 --2- 172.21.15.133:0/178788835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 0x5642d28ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95affff640 1 -- 172.21.15.133:0/178788835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 msgr2=0x5642d28cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95affff640 1 --2- 172.21.15.133:0/178788835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d28cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95affff640 1 -- 172.21.15.133:0/178788835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642d26b9de0 con 0x5642d28260a0 2024-04-25T10:10:53.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95affff640 1 --2- 172.21.15.133:0/178788835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cc0f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f95a001cd00 tx=0x7f95a00c3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9fdbb811a8731e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95a57fa640 1 -- 172.21.15.133:0/178788835 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a00db070 con 0x5642d28260a0 2024-04-25T10:10:53.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95a57fa640 1 -- 172.21.15.133:0/178788835 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a00b0040 con 0x5642d28260a0 2024-04-25T10:10:53.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.398+0000 7f95a57fa640 1 -- 172.21.15.133:0/178788835 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a00ba070 con 0x5642d28260a0 2024-04-25T10:10:53.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 -- 172.21.15.133:0/178788835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 msgr2=0x5642d28cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 --2- 172.21.15.133:0/178788835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cc0f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f95a001cd00 tx=0x7f95a00c3000 comp rx=0 tx=0).stop 2024-04-25T10:10:53.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 -- 172.21.15.133:0/178788835 shutdown_connections 2024-04-25T10:10:53.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 --2- 172.21.15.133:0/178788835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 0x5642d28ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 --2- 172.21.15.133:0/178788835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d28cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 --2- 172.21.15.133:0/178788835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cc0f0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 -- 172.21.15.133:0/178788835 >> 172.21.15.133:0/178788835 conn(0x5642d2822020 msgr2=0x5642d287cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 -- 172.21.15.133:0/178788835 shutdown_connections 2024-04-25T10:10:53.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 -- 172.21.15.133:0/178788835 wait complete. 2024-04-25T10:10:53.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 -- start start 2024-04-25T10:10:53.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 Processor -- start 2024-04-25T10:10:53.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d28d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 0x5642d2881fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d260de00 con 0x5642d28260a0 2024-04-25T10:10:53.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b01f78d0 con 0x5642d286f9d0 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642d2672300 con 0x5642d28702f0 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b97b4000 10 client.0 _ll_register_callbacks cb 0x5642d27f1040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95affff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95affff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52790/0 (socket says 172.21.15.133:52790) 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95affff640 1 -- 172.21.15.133:0/3787815392 learned_addr learned my addr 172.21.15.133:0/3787815392 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95b4be4640 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 0x5642d2881fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95a7fff640 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d28d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95affff640 1 -- 172.21.15.133:0/3787815392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 msgr2=0x5642d2881fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95affff640 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 0x5642d2881fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95affff640 1 -- 172.21.15.133:0/3787815392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 msgr2=0x5642d28d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95affff640 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d28d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95affff640 1 -- 172.21.15.133:0/3787815392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642d26148e0 con 0x5642d28260a0 2024-04-25T10:10:53.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95a7fff640 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d28d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f95affff640 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cff60 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f95a001c940 tx=0x7f95a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb855dd7f39575fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f958effd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.402+0000 7f958effd640 1 -- 172.21.15.133:0/3787815392 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a00db030 con 0x5642d28260a0 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f958effd640 1 -- 172.21.15.133:0/3787815392 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f95a00c4040 con 0x5642d28260a0 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 -- 172.21.15.133:0/3787815392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95a00b0040 con 0x5642d28260a0 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f958effd640 1 -- 172.21.15.133:0/3787815392 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95a00ba040 con 0x5642d28260a0 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 10 client.8097 fetch_fsmap learned FSMap version 58 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 -- 172.21.15.133:0/3787815392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x5642d26cc8f0 con 0x5642d28260a0 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f958effd640 1 -- 172.21.15.133:0/3787815392 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a00b0040 con 0x5642d28260a0 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f958effd640 1 -- 172.21.15.133:0/3787815392 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f95a00e9050 con 0x5642d28260a0 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 10 client.8097 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 10 client.8097 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 20 client.8097 populate_metadata read hostname 'smithi133' 2024-04-25T10:10:53.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5642d28829b0 0x5642d2882dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 4 client.8097 mds_command: new command op to 7869 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 -- 172.21.15.133:0/3787815392 --> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5642d260ec60 con 0x5642d28829b0 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95a7fff640 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5642d28829b0 0x5642d2882dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f958effd640 10 client.8097 ms_handle_connect on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95a7fff640 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5642d28829b0 0x5642d2882dc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2423a92fb1e2de99 server_cookie=ac86f89e703fb77a in_seq=0 out_seq=0 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f958effd640 1 -- 172.21.15.133:0/3787815392 <== mds.0 v2:172.21.15.133:6835/3580592279 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5642d260ec60 con 0x5642d28829b0 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f958effd640 10 client.8097 handle_command_reply: tid=0 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 client.8097 shutdown 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 -- 172.21.15.133:0/3787815392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 msgr2=0x5642d28cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cff60 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f95a001c940 tx=0x7f95a0008040 comp rx=0 tx=0).stop 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 -- 172.21.15.133:0/3787815392 shutdown_connections 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5642d28702f0 0x5642d2881fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5642d286f9d0 0x5642d28d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.133:6835/3580592279,v1:172.21.15.133:6837/3580592279] conn(0x5642d28829b0 0x5642d2882dc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 --2- 172.21.15.133:0/3787815392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642d28260a0 0x5642d28cff60 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f958effd640 0 client.8097 ms_handle_reset on v2:172.21.15.133:6835/3580592279 2024-04-25T10:10:53.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 -- 172.21.15.133:0/3787815392 >> 172.21.15.133:0/3787815392 conn(0x5642d2822020 msgr2=0x5642d286e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 -- 172.21.15.133:0/3787815392 shutdown_connections 2024-04-25T10:10:53.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 1 -- 172.21.15.133:0/3787815392 wait complete. 2024-04-25T10:10:53.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b97b4000 20 client.8097 trim_cache size 0 max 16384 2024-04-25T10:10:53.413 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:10:53.413 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-04-25T10:10:53.413 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:10:53.413 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:10:53.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 -- 172.21.15.133:0/3412001939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9590066f60 msgr2=0x7f9590069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9590066f60 0x7f9590069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f95a00af000 tx=0x7f95a00a7000 comp rx=0 tx=0).stop 2024-04-25T10:10:53.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 -- 172.21.15.133:0/3412001939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95b00d1920 msgr2=0x7f95b00cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95b00d1920 0x7f95b00cd2f0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f959800b550 tx=0x7f959800ba10 comp rx=0 tx=0).stop 2024-04-25T10:10:53.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 -- 172.21.15.133:0/3412001939 shutdown_connections 2024-04-25T10:10:53.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f95b015d9a0 0x7f95b014bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9590066f60 0x7f9590069420 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95b00d1920 0x7f95b00cd2f0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 --2- 172.21.15.133:0/3412001939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95b00ce600 0x7f95b00c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 -- 172.21.15.133:0/3412001939 >> 172.21.15.133:0/3412001939 conn(0x7f95b006bf80 msgr2=0x7f95b00b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 -- 172.21.15.133:0/3412001939 shutdown_connections 2024-04-25T10:10:53.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.406+0000 7f95b6647640 1 -- 172.21.15.133:0/3412001939 wait complete. 2024-04-25T10:10:53.429 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:10:53.430 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.530+0000 7fd2eb36c640 1 Processor -- start 2024-04-25T10:10:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.530+0000 7fd2eb36c640 1 -- start start 2024-04-25T10:10:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2eb36c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e415bb00 0x7fd2e40cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2eb36c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2e4066df0 0x7fd2e406dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2eb36c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd2e406f5b0 0x7fd2e406fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2eb36c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e4067250 con 0x7fd2e4066df0 2024-04-25T10:10:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2eb36c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e40b5d00 con 0x7fd2e406f5b0 2024-04-25T10:10:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2eb36c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e40cba80 con 0x7fd2e415bb00 2024-04-25T10:10:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2e8907640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2e4066df0 0x7fd2e406dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2e8907640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2e4066df0 0x7fd2e406dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52794/0 (socket says 172.21.15.133:52794) 2024-04-25T10:10:53.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2e8907640 1 -- 172.21.15.133:0/2004602182 learned_addr learned my addr 172.21.15.133:0/2004602182 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2e9108640 1 --2- 172.21.15.133:0/2004602182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e415bb00 0x7fd2e40cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2e9909640 1 --2- 172.21.15.133:0/2004602182 >> v2:172.21.15.162:3300/0 conn(0x7fd2e406f5b0 0x7fd2e406fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2e8907640 1 -- 172.21.15.133:0/2004602182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e415bb00 msgr2=0x7fd2e40cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2e8907640 1 --2- 172.21.15.133:0/2004602182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e415bb00 0x7fd2e40cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2e8907640 1 -- 172.21.15.133:0/2004602182 >> v2:172.21.15.162:3300/0 conn(0x7fd2e406f5b0 msgr2=0x7fd2e406fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2e8907640 1 --2- 172.21.15.133:0/2004602182 >> v2:172.21.15.162:3300/0 conn(0x7fd2e406f5b0 0x7fd2e406fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2e8907640 1 -- 172.21.15.133:0/2004602182 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e40cb7a0 con 0x7fd2e4066df0 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2e8907640 1 --2- 172.21.15.133:0/2004602182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e4066df0 0x7fd2e406dbc0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd2e406f040 tx=0x7fd2d401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5520c5dfe0bc30ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2dbfff640 1 -- 172.21.15.133:0/2004602182 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d40ab640 con 0x7fd2e4066df0 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2dbfff640 1 -- 172.21.15.133:0/2004602182 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d40b0070 con 0x7fd2e4066df0 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.534+0000 7fd2dbfff640 1 -- 172.21.15.133:0/2004602182 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d40aba00 con 0x7fd2e4066df0 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2004602182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e4066df0 msgr2=0x7fd2e406dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 --2- 172.21.15.133:0/2004602182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e4066df0 0x7fd2e406dbc0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd2e406f040 tx=0x7fd2d401c460 comp rx=0 tx=0).stop 2024-04-25T10:10:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2004602182 shutdown_connections 2024-04-25T10:10:53.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 --2- 172.21.15.133:0/2004602182 >> v2:172.21.15.162:3300/0 conn(0x7fd2e406f5b0 0x7fd2e406fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 --2- 172.21.15.133:0/2004602182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e4066df0 0x7fd2e406dbc0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 --2- 172.21.15.133:0/2004602182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e415bb00 0x7fd2e40cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2004602182 >> 172.21.15.133:0/2004602182 conn(0x7fd2e40cc8e0 msgr2=0x7fd2e40cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2004602182 shutdown_connections 2024-04-25T10:10:53.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2004602182 wait complete. 2024-04-25T10:10:53.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 Processor -- start 2024-04-25T10:10:53.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- start start 2024-04-25T10:10:53.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e4066df0 0x7fd2e40b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2e406f5b0 0x7fd2e40c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e415bb00 0x7fd2e41560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e406c030 con 0x7fd2e415bb00 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e406beb0 con 0x7fd2e406f5b0 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e406c1b0 con 0x7fd2e4066df0 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e9909640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e415bb00 0x7fd2e41560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e9909640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e415bb00 0x7fd2e41560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52796/0 (socket says 172.21.15.133:52796) 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e9909640 1 -- 172.21.15.133:0/2773084491 learned_addr learned my addr 172.21.15.133:0/2773084491 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e9108640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e4066df0 0x7fd2e40b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e8907640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2e406f5b0 0x7fd2e40c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e9909640 1 -- 172.21.15.133:0/2773084491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e4066df0 msgr2=0x7fd2e40b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e9909640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e4066df0 0x7fd2e40b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e9909640 1 -- 172.21.15.133:0/2773084491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2e406f5b0 msgr2=0x7fd2e40c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e9909640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2e406f5b0 0x7fd2e40c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e9909640 1 -- 172.21.15.133:0/2773084491 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e4155030 con 0x7fd2e415bb00 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e8907640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2e406f5b0 0x7fd2e40c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2e9909640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e415bb00 0x7fd2e41560e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000d800 tx=0x7fd2e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2d9ffb640 1 -- 172.21.15.133:0/2773084491 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0018020 con 0x7fd2e415bb00 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2d9ffb640 1 -- 172.21.15.133:0/2773084491 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e00096c0 con 0x7fd2e415bb00 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2773084491 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e4155310 con 0x7fd2e415bb00 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2d9ffb640 1 -- 172.21.15.133:0/2773084491 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0009ab0 con 0x7fd2e415bb00 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2773084491 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e41558d0 con 0x7fd2e415bb00 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.538+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2773084491 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e4067c70 con 0x7fd2e415bb00 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.542+0000 7fd2d9ffb640 1 -- 172.21.15.133:0/2773084491 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2e001b020 con 0x7fd2e415bb00 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.542+0000 7fd2d9ffb640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c8066f60 0x7fd2c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.542+0000 7fd2e9108640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c8066f60 0x7fd2c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.542+0000 7fd2d9ffb640 1 -- 172.21.15.133:0/2773084491 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fd2e0085b70 con 0x7fd2e415bb00 2024-04-25T10:10:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.542+0000 7fd2e9108640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c8066f60 0x7fd2c8069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009f60 tx=0x7fd2dc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.546+0000 7fd2d9ffb640 1 -- 172.21.15.133:0/2773084491 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2e0022020 con 0x7fd2e415bb00 2024-04-25T10:10:53.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.722+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2773084491 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2e409da10 con 0x7fd2e415bb00 2024-04-25T10:10:53.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.722+0000 7fd2d9ffb640 1 -- 172.21.15.133:0/2773084491 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fd2e0056a60 con 0x7fd2e415bb00 2024-04-25T10:10:53.726 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:53.726 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3089710291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3089710291},{"type":"v1","addr":"172.21.15.162:6834","nonce":3089710291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1390003376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1390003376},{"type":"v1","addr":"172.21.15.162:6835","nonce":1390003376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7866,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1134679682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1134679682},{"type":"v1","addr":"172.21.15.133:6836","nonce":1134679682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:27.661584+0000","modified":"2024-04-25T10:10:30.680424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3580592279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3580592279},{"type":"v1","addr":"172.21.15.133:6837","nonce":3580592279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:53.726 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 58 2024-04-25T10:10:53.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2773084491 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c8066f60 msgr2=0x7fd2c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c8066f60 0x7fd2c8069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009f60 tx=0x7fd2dc00b040 comp rx=0 tx=0).stop 2024-04-25T10:10:53.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2773084491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e415bb00 msgr2=0x7fd2e41560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e415bb00 0x7fd2e41560e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000d800 tx=0x7fd2e000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2773084491 shutdown_connections 2024-04-25T10:10:53.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c8066f60 0x7fd2c8069420 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e415bb00 0x7fd2e41560e0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2e406f5b0 0x7fd2e40c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 --2- 172.21.15.133:0/2773084491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e4066df0 0x7fd2e40b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2773084491 >> 172.21.15.133:0/2773084491 conn(0x7fd2e40cc8e0 msgr2=0x7fd2e40b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2773084491 shutdown_connections 2024-04-25T10:10:53.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.726+0000 7fd2eb36c640 1 -- 172.21.15.133:0/2773084491 wait complete. 2024-04-25T10:10:53.743 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:10:53.743 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:10:53.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.854+0000 7f0a3d467640 1 Processor -- start 2024-04-25T10:10:53.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.854+0000 7f0a3d467640 1 -- start start 2024-04-25T10:10:53.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.854+0000 7f0a3d467640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0a38159dc0 0x7f0a380cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.854+0000 7f0a3d467640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0a3815a1a0 0x7f0a3815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.854+0000 7f0a3d467640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a3815e050 0x7f0a380ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3815e480 con 0x7f0a38159dc0 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a380ce880 con 0x7f0a3815a1a0 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380c49e0 con 0x7f0a3815e050 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a36ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0a38159dc0 0x7f0a380cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a36ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0a38159dc0 0x7f0a380cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52808/0 (socket says 172.21.15.133:52808) 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a36ffd640 1 -- 172.21.15.133:0/4093218460 learned_addr learned my addr 172.21.15.133:0/4093218460 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a367fc640 1 --2- 172.21.15.133:0/4093218460 >> v2:172.21.15.162:3300/0 conn(0x7f0a3815a1a0 0x7f0a3815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a377fe640 1 --2- 172.21.15.133:0/4093218460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a3815e050 0x7f0a380ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a36ffd640 1 -- 172.21.15.133:0/4093218460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a3815e050 msgr2=0x7f0a380ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a36ffd640 1 --2- 172.21.15.133:0/4093218460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a3815e050 0x7f0a380ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a36ffd640 1 -- 172.21.15.133:0/4093218460 >> v2:172.21.15.162:3300/0 conn(0x7f0a3815a1a0 msgr2=0x7f0a3815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a36ffd640 1 --2- 172.21.15.133:0/4093218460 >> v2:172.21.15.162:3300/0 conn(0x7f0a3815a1a0 0x7f0a3815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a36ffd640 1 -- 172.21.15.133:0/4093218460 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a380ceb20 con 0x7f0a38159dc0 2024-04-25T10:10:53.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a36ffd640 1 --2- 172.21.15.133:0/4093218460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a38159dc0 0x7f0a380cba60 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f0a3806f6e0 tx=0x7f0a2c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbe1da136200076 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a35ffb640 1 -- 172.21.15.133:0/4093218460 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0b0070 con 0x7f0a38159dc0 2024-04-25T10:10:53.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a35ffb640 1 -- 172.21.15.133:0/4093218460 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c0ab600 con 0x7f0a38159dc0 2024-04-25T10:10:53.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a35ffb640 1 -- 172.21.15.133:0/4093218460 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c0ab9d0 con 0x7f0a38159dc0 2024-04-25T10:10:53.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 -- 172.21.15.133:0/4093218460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a38159dc0 msgr2=0x7f0a380cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 --2- 172.21.15.133:0/4093218460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a38159dc0 0x7f0a380cba60 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f0a3806f6e0 tx=0x7f0a2c01c260 comp rx=0 tx=0).stop 2024-04-25T10:10:53.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 -- 172.21.15.133:0/4093218460 shutdown_connections 2024-04-25T10:10:53.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 --2- 172.21.15.133:0/4093218460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a3815e050 0x7f0a380ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 --2- 172.21.15.133:0/4093218460 >> v2:172.21.15.162:3300/0 conn(0x7f0a3815a1a0 0x7f0a3815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 --2- 172.21.15.133:0/4093218460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a38159dc0 0x7f0a380cba60 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 -- 172.21.15.133:0/4093218460 >> 172.21.15.133:0/4093218460 conn(0x7f0a38067000 msgr2=0x7f0a3806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 -- 172.21.15.133:0/4093218460 shutdown_connections 2024-04-25T10:10:53.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.858+0000 7f0a3d467640 1 -- 172.21.15.133:0/4093218460 wait complete. 2024-04-25T10:10:53.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a3d467640 1 Processor -- start 2024-04-25T10:10:53.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a3d467640 1 -- start start 2024-04-25T10:10:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a3d467640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a38159dc0 0x7f0a38144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a3d467640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3815e050 0x7f0a38148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a3d467640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a38149100 0x7f0a381552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a3d467640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380c4e40 con 0x7f0a3815e050 2024-04-25T10:10:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a3d467640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380c4cc0 con 0x7f0a38159dc0 2024-04-25T10:10:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a3d467640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3806d350 con 0x7f0a38149100 2024-04-25T10:10:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a367fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3815e050 0x7f0a38148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a367fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3815e050 0x7f0a38148bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52814/0 (socket says 172.21.15.133:52814) 2024-04-25T10:10:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a367fc640 1 -- 172.21.15.133:0/2133998471 learned_addr learned my addr 172.21.15.133:0/2133998471 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a36ffd640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a38159dc0 0x7f0a38144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a377fe640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a38149100 0x7f0a381552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a367fc640 1 -- 172.21.15.133:0/2133998471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a38149100 msgr2=0x7f0a381552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a367fc640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a38149100 0x7f0a381552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a367fc640 1 -- 172.21.15.133:0/2133998471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a38159dc0 msgr2=0x7f0a38144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a367fc640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a38159dc0 0x7f0a38144e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a367fc640 1 -- 172.21.15.133:0/2133998471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a381495f0 con 0x7f0a3815e050 2024-04-25T10:10:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a367fc640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3815e050 0x7f0a38148bc0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800d830 tx=0x7f0a2800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a1f7fe640 1 -- 172.21.15.133:0/2133998471 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28018020 con 0x7f0a3815e050 2024-04-25T10:10:53.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a1f7fe640 1 -- 172.21.15.133:0/2133998471 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a280096a0 con 0x7f0a3815e050 2024-04-25T10:10:53.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a3d467640 1 -- 172.21.15.133:0/2133998471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a381498d0 con 0x7f0a3815e050 2024-04-25T10:10:53.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a1f7fe640 1 -- 172.21.15.133:0/2133998471 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28009a50 con 0x7f0a3815e050 2024-04-25T10:10:53.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a3d467640 1 -- 172.21.15.133:0/2133998471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a38149e90 con 0x7f0a3815e050 2024-04-25T10:10:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.862+0000 7f0a3d467640 1 -- 172.21.15.133:0/2133998471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a38068ef0 con 0x7f0a3815e050 2024-04-25T10:10:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.866+0000 7f0a1f7fe640 1 -- 172.21.15.133:0/2133998471 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a2801b020 con 0x7f0a3815e050 2024-04-25T10:10:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.866+0000 7f0a1f7fe640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a24066f60 0x7f0a24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.866+0000 7f0a1f7fe640 1 -- 172.21.15.133:0/2133998471 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f0a28085880 con 0x7f0a3815e050 2024-04-25T10:10:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.866+0000 7f0a36ffd640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a24066f60 0x7f0a24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.866+0000 7f0a36ffd640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a24066f60 0x7f0a24069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0ad000 tx=0x7f0a2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:53.870+0000 7f0a1f7fe640 1 -- 172.21.15.133:0/2133998471 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a28022020 con 0x7f0a3815e050 2024-04-25T10:10:54.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.090+0000 7f0a3d467640 1 -- 172.21.15.133:0/2133998471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0a3806dc90 con 0x7f0a3815e050 2024-04-25T10:10:54.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.090+0000 7f0a1f7fe640 1 -- 172.21.15.133:0/2133998471 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1024 (secure 0 0 0) 0x7f0a28056690 con 0x7f0a3815e050 2024-04-25T10:10:54.095 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:54.095 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":567,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":856,"data_bytes":875271829,"bytes_used":2192416768,"bytes_avail":770901696512,"bytes_total":773094113280,"write_bytes_sec":56088202,"read_op_per_sec":0,"write_op_per_sec":56},"fsmap":{"epoch":58,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7869}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T10:10:17.862579+0000","services":{}},"progress_events":{}} 2024-04-25T10:10:54.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 -- 172.21.15.133:0/2133998471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a24066f60 msgr2=0x7f0a24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a24066f60 0x7f0a24069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0ad000 tx=0x7f0a2c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:54.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 -- 172.21.15.133:0/2133998471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3815e050 msgr2=0x7f0a38148bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3815e050 0x7f0a38148bc0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800d830 tx=0x7f0a2800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 -- 172.21.15.133:0/2133998471 shutdown_connections 2024-04-25T10:10:54.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0a38149100 0x7f0a381552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a24066f60 0x7f0a24069420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3815e050 0x7f0a38148bc0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 --2- 172.21.15.133:0/2133998471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0a38159dc0 0x7f0a38144e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 -- 172.21.15.133:0/2133998471 >> 172.21.15.133:0/2133998471 conn(0x7f0a38067000 msgr2=0x7f0a3806e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:54.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 -- 172.21.15.133:0/2133998471 shutdown_connections 2024-04-25T10:10:54.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.094+0000 7f0a3d467640 1 -- 172.21.15.133:0/2133998471 wait complete. 2024-04-25T10:10:54.113 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:10:54.113 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:54.169 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:54.224 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:10:54.225 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:10:54.225 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:54.230 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:54.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f65979bc640 1 Processor -- start 2024-04-25T10:10:54.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f65979bc640 1 -- start start 2024-04-25T10:10:54.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f65979bc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f659015c9a0 0x7f65900c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f65979bc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65900b1f70 0x7f659006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f65979bc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f65900ce330 0x7f65900ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f65979bc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65900cece0 con 0x7f65900b1f70 2024-04-25T10:10:54.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f65979bc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65900c61d0 con 0x7f65900ce330 2024-04-25T10:10:54.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f65979bc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65900c64b0 con 0x7f659015c9a0 2024-04-25T10:10:54.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f6594f57640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65900b1f70 0x7f659006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f6594f57640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65900b1f70 0x7f659006dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52816/0 (socket says 172.21.15.133:52816) 2024-04-25T10:10:54.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f6594f57640 1 -- 172.21.15.133:0/2085771476 learned_addr learned my addr 172.21.15.133:0/2085771476 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:54.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f6595f59640 1 --2- 172.21.15.133:0/2085771476 >> v2:172.21.15.162:3300/0 conn(0x7f65900ce330 0x7f65900ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f6595758640 1 --2- 172.21.15.133:0/2085771476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f659015c9a0 0x7f65900c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f6594f57640 1 -- 172.21.15.133:0/2085771476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f659015c9a0 msgr2=0x7f65900c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:54.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f6594f57640 1 --2- 172.21.15.133:0/2085771476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f659015c9a0 0x7f65900c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f6594f57640 1 -- 172.21.15.133:0/2085771476 >> v2:172.21.15.162:3300/0 conn(0x7f65900ce330 msgr2=0x7f65900ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:54.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f6594f57640 1 --2- 172.21.15.133:0/2085771476 >> v2:172.21.15.162:3300/0 conn(0x7f65900ce330 0x7f65900ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.390+0000 7f6594f57640 1 -- 172.21.15.133:0/2085771476 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659006d4f0 con 0x7f65900b1f70 2024-04-25T10:10:54.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6594f57640 1 --2- 172.21.15.133:0/2085771476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65900b1f70 0x7f659006dcf0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f65900ceef0 tx=0x7f658001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76d5e2f1cd2a3a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6587fff640 1 -- 172.21.15.133:0/2085771476 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65800a7070 con 0x7f65900b1f70 2024-04-25T10:10:54.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6587fff640 1 -- 172.21.15.133:0/2085771476 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65800b0070 con 0x7f65900b1f70 2024-04-25T10:10:54.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6587fff640 1 -- 172.21.15.133:0/2085771476 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65800ab890 con 0x7f65900b1f70 2024-04-25T10:10:54.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 -- 172.21.15.133:0/2085771476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65900b1f70 msgr2=0x7f659006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 --2- 172.21.15.133:0/2085771476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65900b1f70 0x7f659006dcf0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f65900ceef0 tx=0x7f658001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 -- 172.21.15.133:0/2085771476 shutdown_connections 2024-04-25T10:10:54.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 --2- 172.21.15.133:0/2085771476 >> v2:172.21.15.162:3300/0 conn(0x7f65900ce330 0x7f65900ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 --2- 172.21.15.133:0/2085771476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65900b1f70 0x7f659006dcf0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 --2- 172.21.15.133:0/2085771476 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f659015c9a0 0x7f65900c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 -- 172.21.15.133:0/2085771476 >> 172.21.15.133:0/2085771476 conn(0x7f6590067000 msgr2=0x7f659006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:54.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 -- 172.21.15.133:0/2085771476 shutdown_connections 2024-04-25T10:10:54.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 -- 172.21.15.133:0/2085771476 wait complete. 2024-04-25T10:10:54.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 Processor -- start 2024-04-25T10:10:54.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 -- start start 2024-04-25T10:10:54.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65900b1f70 0x7f65901455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65900ce330 0x7f6590145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659015c9a0 0x7f65900b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659015cf00 con 0x7f659015c9a0 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659015cd80 con 0x7f65900b1f70 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f65979bc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659015d950 con 0x7f65900ce330 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6595f59640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659015c9a0 0x7f65900b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6595f59640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659015c9a0 0x7f65900b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52822/0 (socket says 172.21.15.133:52822) 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6595f59640 1 -- 172.21.15.133:0/3943921311 learned_addr learned my addr 172.21.15.133:0/3943921311 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6595758640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65900b1f70 0x7f65901455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6594f57640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65900ce330 0x7f6590145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6595f59640 1 -- 172.21.15.133:0/3943921311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65900ce330 msgr2=0x7f6590145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6595f59640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65900ce330 0x7f6590145d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6595f59640 1 -- 172.21.15.133:0/3943921311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65900b1f70 msgr2=0x7f65901455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6595f59640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65900b1f70 0x7f65901455a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6595f59640 1 -- 172.21.15.133:0/3943921311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6590146810 con 0x7f659015c9a0 2024-04-25T10:10:54.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.394+0000 7f6595758640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65900b1f70 0x7f65901455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:54.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f6595f59640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659015c9a0 0x7f65900b33c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6588007e60 tx=0x7f658800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f6585ffb640 1 -- 172.21.15.133:0/3943921311 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588014620 con 0x7f659015c9a0 2024-04-25T10:10:54.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f6585ffb640 1 -- 172.21.15.133:0/3943921311 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6588019070 con 0x7f659015c9a0 2024-04-25T10:10:54.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f65979bc640 1 -- 172.21.15.133:0/3943921311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6590146af0 con 0x7f659015c9a0 2024-04-25T10:10:54.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f6585ffb640 1 -- 172.21.15.133:0/3943921311 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588014a20 con 0x7f659015c9a0 2024-04-25T10:10:54.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f65979bc640 1 -- 172.21.15.133:0/3943921311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6590151b70 con 0x7f659015c9a0 2024-04-25T10:10:54.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f65979bc640 1 -- 172.21.15.133:0/3943921311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659006bc20 con 0x7f659015c9a0 2024-04-25T10:10:54.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f6585ffb640 1 -- 172.21.15.133:0/3943921311 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6588014bc0 con 0x7f659015c9a0 2024-04-25T10:10:54.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f6585ffb640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6574066f60 0x7f6574069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f6585ffb640 1 -- 172.21.15.133:0/3943921311 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6588085a60 con 0x7f659015c9a0 2024-04-25T10:10:54.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f6595758640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6574066f60 0x7f6574069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.398+0000 7f6595758640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6574066f60 0x7f6574069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f658c0097a0 tx=0x7f658c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.402+0000 7f6585ffb640 1 -- 172.21.15.133:0/3943921311 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6588056900 con 0x7f659015c9a0 2024-04-25T10:10:54.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.562+0000 7f65979bc640 1 -- 172.21.15.133:0/3943921311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6590068ef0 con 0x7f659015c9a0 2024-04-25T10:10:54.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.562+0000 7f6585ffb640 1 -- 172.21.15.133:0/3943921311 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17103 (secure 0 0 0) 0x7f6588085020 con 0x7f659015c9a0 2024-04-25T10:10:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:54.566 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":96,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:10:33.344314+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:25.920690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:26.640190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 -- 172.21.15.133:0/3943921311 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6574066f60 msgr2=0x7f6574069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6574066f60 0x7f6574069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f658c0097a0 tx=0x7f658c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 -- 172.21.15.133:0/3943921311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659015c9a0 msgr2=0x7f65900b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659015c9a0 0x7f65900b33c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6588007e60 tx=0x7f658800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 -- 172.21.15.133:0/3943921311 shutdown_connections 2024-04-25T10:10:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6574066f60 0x7f6574069420 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659015c9a0 0x7f65900b33c0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65900ce330 0x7f6590145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 --2- 172.21.15.133:0/3943921311 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65900b1f70 0x7f65901455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 -- 172.21.15.133:0/3943921311 >> 172.21.15.133:0/3943921311 conn(0x7f6590067000 msgr2=0x7f659015e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:54.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 -- 172.21.15.133:0/3943921311 shutdown_connections 2024-04-25T10:10:54.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.566+0000 7f65979bc640 1 -- 172.21.15.133:0/3943921311 wait complete. 2024-04-25T10:10:54.585 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:10:25.920690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:10:26.640190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:54.585 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:54.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.702+0000 7f0672743640 1 Processor -- start 2024-04-25T10:10:54.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.702+0000 7f0672743640 1 -- start start 2024-04-25T10:10:54.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f0672743640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f066c0ce620 0x7f066c15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f0672743640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066c0d2420 con 0x7f066c070490 2024-04-25T10:10:54.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f0672743640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066c0d0180 con 0x7f066c15bb50 2024-04-25T10:10:54.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f0672743640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066c0d21d0 con 0x7f066c0ce620 2024-04-25T10:10:54.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f06637fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f066c070490 0x7f066c0cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54616/0 (socket says 172.21.15.133:54616) 2024-04-25T10:10:54.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f06637fe640 1 -- 172.21.15.133:0/728414136 learned_addr learned my addr 172.21.15.133:0/728414136 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f0670ce0640 1 --2- 172.21.15.133:0/728414136 >> v2:172.21.15.162:3300/0 conn(0x7f066c0ce620 0x7f066c15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051725910 0 0) 0x7f066c0d2420 con 0x7f066c070490 2024-04-25T10:10:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0658003530 con 0x7f066c070490 2024-04-25T10:10:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4255077785 0 0) 0x7f0658003530 con 0x7f066c070490 2024-04-25T10:10:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 >> v2:172.21.15.162:3300/0 conn(0x7f066c0ce620 msgr2=0x7f066c15bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 --2- 172.21.15.133:0/728414136 >> v2:172.21.15.162:3300/0 conn(0x7f066c0ce620 0x7f066c15bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 >> v1:172.21.15.162:6789/0 conn(0x7f066c15bb50 legacy=0x7f066c0cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0cee70 con 0x7f066c070490 2024-04-25T10:10:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f065c001680 con 0x7f066c070490 2024-04-25T10:10:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 >> v1:172.21.15.133:6789/0 conn(0x7f066c070490 legacy=0x7f066c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 --2- 172.21.15.133:0/728414136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0658003a10 0x7f0658003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 --2- 172.21.15.133:0/728414136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0658004810 0x7f0658004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 --2- 172.21.15.133:0/728414136 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06580052f0 0x7f065800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0d21d0 con 0x7f0658004810 2024-04-25T10:10:54.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065800c1f0 con 0x7f06580052f0 2024-04-25T10:10:54.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065800c3f0 con 0x7f0658003a10 2024-04-25T10:10:54.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f06637fe640 1 --2- 172.21.15.133:0/728414136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0658004810 0x7f0658004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f0670ce0640 1 --2- 172.21.15.133:0/728414136 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06580052f0 0x7f065800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f066bfff640 1 --2- 172.21.15.133:0/728414136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0658003a10 0x7f0658003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f06637fe640 1 -- 172.21.15.133:0/728414136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0658003a10 msgr2=0x7f0658003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:54.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f06637fe640 1 --2- 172.21.15.133:0/728414136 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0658003a10 0x7f0658003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f06637fe640 1 -- 172.21.15.133:0/728414136 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06580052f0 msgr2=0x7f065800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:54.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f06637fe640 1 --2- 172.21.15.133:0/728414136 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06580052f0 0x7f065800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f0670ce0640 1 -- 172.21.15.133:0/728414136 reap_dead start 2024-04-25T10:10:54.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.706+0000 7f06637fe640 1 -- 172.21.15.133:0/728414136 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0658003530 con 0x7f0658004810 2024-04-25T10:10:54.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f06637fe640 1 --2- 172.21.15.133:0/728414136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0658004810 0x7f0658004c90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f065c00c190 tx=0x7f065c01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9870ae5fba0481a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c007c70 con 0x7f0658004810 2024-04-25T10:10:54.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c0a9070 con 0x7f0658004810 2024-04-25T10:10:54.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f066b7fe640 1 -- 172.21.15.133:0/728414136 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c005da0 con 0x7f0658004810 2024-04-25T10:10:54.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 -- 172.21.15.133:0/728414136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0658004810 msgr2=0x7f0658004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 --2- 172.21.15.133:0/728414136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0658004810 0x7f0658004c90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f065c00c190 tx=0x7f065c01cba0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 -- 172.21.15.133:0/728414136 shutdown_connections 2024-04-25T10:10:54.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 --2- 172.21.15.133:0/728414136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0658004810 0x7f0658004c90 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 -- 172.21.15.133:0/728414136 >> 172.21.15.133:0/728414136 conn(0x7f066c0d16b0 msgr2=0x7f066c0d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:54.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 -- 172.21.15.133:0/728414136 shutdown_connections 2024-04-25T10:10:54.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 -- 172.21.15.133:0/728414136 wait complete. 2024-04-25T10:10:54.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 Processor -- start 2024-04-25T10:10:54.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 -- start start 2024-04-25T10:10:54.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0658004810 0x7f066c0c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f066c0c9d10 0x7f066c0b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066c0b4810 0x7f066c1416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0ce2a0 con 0x7f066c0b4810 2024-04-25T10:10:54.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0ce120 con 0x7f0658004810 2024-04-25T10:10:54.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c0ce420 con 0x7f066c0c9d10 2024-04-25T10:10:54.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0670ce0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066c0b4810 0x7f066c1416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0670ce0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066c0b4810 0x7f066c1416d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52840/0 (socket says 172.21.15.133:52840) 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0670ce0640 1 -- 172.21.15.133:0/741359119 learned_addr learned my addr 172.21.15.133:0/741359119 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f066bfff640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0658004810 0x7f066c0c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f06637fe640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f066c0c9d10 0x7f066c0b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0670ce0640 1 -- 172.21.15.133:0/741359119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f066c0c9d10 msgr2=0x7f066c0b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0670ce0640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f066c0c9d10 0x7f066c0b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0670ce0640 1 -- 172.21.15.133:0/741359119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0658004810 msgr2=0x7f066c0c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0670ce0640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0658004810 0x7f066c0c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0670ce0640 1 -- 172.21.15.133:0/741359119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0ca290 con 0x7f066c0b4810 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f066bfff640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0658004810 0x7f066c0c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0670ce0640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066c0b4810 0x7f066c1416d0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f0664007cf0 tx=0x7f06640115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f06697fa640 1 -- 172.21.15.133:0/741359119 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664011de0 con 0x7f066c0b4810 2024-04-25T10:10:54.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f06697fa640 1 -- 172.21.15.133:0/741359119 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066401b070 con 0x7f066c0b4810 2024-04-25T10:10:54.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 -- 172.21.15.133:0/741359119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066c0ca570 con 0x7f066c0b4810 2024-04-25T10:10:54.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f06697fa640 1 -- 172.21.15.133:0/741359119 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06640039a0 con 0x7f066c0b4810 2024-04-25T10:10:54.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.710+0000 7f0672743640 1 -- 172.21.15.133:0/741359119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066c0cab30 con 0x7f066c0b4810 2024-04-25T10:10:54.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.714+0000 7f0672743640 1 -- 172.21.15.133:0/741359119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066c067c70 con 0x7f066c0b4810 2024-04-25T10:10:54.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.714+0000 7f06697fa640 1 -- 172.21.15.133:0/741359119 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0664003b40 con 0x7f066c0b4810 2024-04-25T10:10:54.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.714+0000 7f06697fa640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f064c066f60 0x7f064c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:54.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.714+0000 7f06697fa640 1 -- 172.21.15.133:0/741359119 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f06640869f0 con 0x7f066c0b4810 2024-04-25T10:10:54.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.714+0000 7f066bfff640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f064c066f60 0x7f064c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:54.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.714+0000 7f066bfff640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f064c066f60 0x7f064c069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f066c04f450 tx=0x7f0654009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:54.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.718+0000 7f06697fa640 1 -- 172.21.15.133:0/741359119 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0664057750 con 0x7f066c0b4810 2024-04-25T10:10:54.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 -- 172.21.15.133:0/741359119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f066c0cfa70 con 0x7f066c0b4810 2024-04-25T10:10:54.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f06697fa640 1 -- 172.21.15.133:0/741359119 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0664057050 con 0x7f066c0b4810 2024-04-25T10:10:54.880 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:10:54.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 -- 172.21.15.133:0/741359119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f064c066f60 msgr2=0x7f064c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f064c066f60 0x7f064c069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f066c04f450 tx=0x7f0654009040 comp rx=0 tx=0).stop 2024-04-25T10:10:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 -- 172.21.15.133:0/741359119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066c0b4810 msgr2=0x7f066c1416d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066c0b4810 0x7f066c1416d0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f0664007cf0 tx=0x7f06640115a0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 -- 172.21.15.133:0/741359119 shutdown_connections 2024-04-25T10:10:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f064c066f60 0x7f064c069420 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066c0b4810 0x7f066c1416d0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f066c0c9d10 0x7f066c0b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 --2- 172.21.15.133:0/741359119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0658004810 0x7f066c0c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 -- 172.21.15.133:0/741359119 >> 172.21.15.133:0/741359119 conn(0x7f066c0d16b0 msgr2=0x7f066c15ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 -- 172.21.15.133:0/741359119 shutdown_connections 2024-04-25T10:10:54.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:54.878+0000 7f0672743640 1 -- 172.21.15.133:0/741359119 wait complete. 2024-04-25T10:10:54.899 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:55.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.010+0000 7f71030b6640 1 Processor -- start 2024-04-25T10:10:55.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.010+0000 7f71030b6640 1 -- start start 2024-04-25T10:10:55.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70fc143ea0 0x7f70fc0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc06eac0 0x7f70fc0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70fc15ac50 con 0x7f70fc0b2150 2024-04-25T10:10:55.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70fc0cd180 con 0x7f70fc143ea0 2024-04-25T10:10:55.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc15aed0 con 0x7f70fc06eac0 2024-04-25T10:10:55.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f7100e52640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70fc143ea0 0x7f70fc0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f7100e52640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70fc143ea0 0x7f70fc0cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52850/0 (socket says 172.21.15.133:52850) 2024-04-25T10:10:55.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f7100e52640 1 -- 172.21.15.133:0/4026666118 learned_addr learned my addr 172.21.15.133:0/4026666118 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:55.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f7100e52640 1 -- 172.21.15.133:0/4026666118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc06eac0 msgr2=0x7f70fc0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f7100e52640 1 --2- 172.21.15.133:0/4026666118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc06eac0 0x7f70fc0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f7100e52640 1 -- 172.21.15.133:0/4026666118 >> v1:172.21.15.162:6789/0 conn(0x7f70fc0b2150 legacy=0x7f70fc06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f7100e52640 1 -- 172.21.15.133:0/4026666118 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc15b1b0 con 0x7f70fc143ea0 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f7100e52640 1 --2- 172.21.15.133:0/4026666118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc143ea0 0x7f70fc0cc810 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009870 tx=0x7f70ec01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8869c5990eb2c0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f70fb7fe640 1 -- 172.21.15.133:0/4026666118 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec0b0070 con 0x7f70fc143ea0 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f70fb7fe640 1 -- 172.21.15.133:0/4026666118 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec0ab600 con 0x7f70fc143ea0 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f70fb7fe640 1 -- 172.21.15.133:0/4026666118 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec0ab9d0 con 0x7f70fc143ea0 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 -- 172.21.15.133:0/4026666118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc143ea0 msgr2=0x7f70fc0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 --2- 172.21.15.133:0/4026666118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc143ea0 0x7f70fc0cc810 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009870 tx=0x7f70ec01c340 comp rx=0 tx=0).stop 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 -- 172.21.15.133:0/4026666118 shutdown_connections 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 --2- 172.21.15.133:0/4026666118 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc06eac0 0x7f70fc0b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 --2- 172.21.15.133:0/4026666118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc143ea0 0x7f70fc0cc810 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 -- 172.21.15.133:0/4026666118 >> 172.21.15.133:0/4026666118 conn(0x7f70fc066f70 msgr2=0x7f70fc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 -- 172.21.15.133:0/4026666118 shutdown_connections 2024-04-25T10:10:55.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.014+0000 7f71030b6640 1 -- 172.21.15.133:0/4026666118 wait complete. 2024-04-25T10:10:55.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f71030b6640 1 Processor -- start 2024-04-25T10:10:55.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f71030b6640 1 -- start start 2024-04-25T10:10:55.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f71030b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc06eac0 0x7f70fc13bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f71030b6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70fc0b2150 0x7f70fc139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f71030b6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc143ea0 0x7f70fc0b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f71030b6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc167550 con 0x7f70fc143ea0 2024-04-25T10:10:55.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f71030b6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc1673d0 con 0x7f70fc0b2150 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f71030b6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc1676d0 con 0x7f70fc06eac0 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f7101653640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc143ea0 0x7f70fc0b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f7101653640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc143ea0 0x7f70fc0b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52856/0 (socket says 172.21.15.133:52856) 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f7101653640 1 -- 172.21.15.133:0/1734316801 learned_addr learned my addr 172.21.15.133:0/1734316801 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f7100e52640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc06eac0 0x7f70fc13bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f70fbfff640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70fc0b2150 0x7f70fc139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f7101653640 1 -- 172.21.15.133:0/1734316801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc06eac0 msgr2=0x7f70fc13bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f7101653640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc06eac0 0x7f70fc13bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f7101653640 1 -- 172.21.15.133:0/1734316801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70fc0b2150 msgr2=0x7f70fc139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f7101653640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70fc0b2150 0x7f70fc139ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f7101653640 1 -- 172.21.15.133:0/1734316801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc0b3bf0 con 0x7f70fc143ea0 2024-04-25T10:10:55.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f7101653640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc143ea0 0x7f70fc0b32e0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f70f400b9e0 tx=0x7f70f400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f70f97fa640 1 -- 172.21.15.133:0/1734316801 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f4019070 con 0x7f70fc143ea0 2024-04-25T10:10:55.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f70f97fa640 1 -- 172.21.15.133:0/1734316801 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f400cb40 con 0x7f70fc143ea0 2024-04-25T10:10:55.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f71030b6640 1 -- 172.21.15.133:0/1734316801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc0b3ed0 con 0x7f70fc143ea0 2024-04-25T10:10:55.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f70f97fa640 1 -- 172.21.15.133:0/1734316801 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f4014440 con 0x7f70fc143ea0 2024-04-25T10:10:55.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f71030b6640 1 -- 172.21.15.133:0/1734316801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc0b4490 con 0x7f70fc143ea0 2024-04-25T10:10:55.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.018+0000 7f71030b6640 1 -- 172.21.15.133:0/1734316801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70fc068e80 con 0x7f70fc143ea0 2024-04-25T10:10:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.022+0000 7f70f97fa640 1 -- 172.21.15.133:0/1734316801 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70f401c070 con 0x7f70fc143ea0 2024-04-25T10:10:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.022+0000 7f70f97fa640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e8066f90 0x7f70e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.022+0000 7f70f97fa640 1 -- 172.21.15.133:0/1734316801 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f70f4085240 con 0x7f70fc143ea0 2024-04-25T10:10:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.022+0000 7f7100e52640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e8066f90 0x7f70e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.022+0000 7f7100e52640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e8066f90 0x7f70e8069450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0ad000 tx=0x7f70ec0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.026+0000 7f70f97fa640 1 -- 172.21.15.133:0/1734316801 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70f4023020 con 0x7f70fc143ea0 2024-04-25T10:10:55.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 -- 172.21.15.133:0/1734316801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70fc06e550 con 0x7f70fc143ea0 2024-04-25T10:10:55.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f70f97fa640 1 -- 172.21.15.133:0/1734316801 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f70f40561c0 con 0x7f70fc143ea0 2024-04-25T10:10:55.188 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:10:55.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 -- 172.21.15.133:0/1734316801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e8066f90 msgr2=0x7f70e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e8066f90 0x7f70e8069450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0ad000 tx=0x7f70ec0a6000 comp rx=0 tx=0).stop 2024-04-25T10:10:55.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 -- 172.21.15.133:0/1734316801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc143ea0 msgr2=0x7f70fc0b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc143ea0 0x7f70fc0b32e0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f70f400b9e0 tx=0x7f70f400bea0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 -- 172.21.15.133:0/1734316801 shutdown_connections 2024-04-25T10:10:55.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e8066f90 0x7f70e8069450 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc143ea0 0x7f70fc0b32e0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70fc0b2150 0x7f70fc139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 --2- 172.21.15.133:0/1734316801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc06eac0 0x7f70fc13bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 -- 172.21.15.133:0/1734316801 >> 172.21.15.133:0/1734316801 conn(0x7f70fc066f70 msgr2=0x7f70fc13d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 -- 172.21.15.133:0/1734316801 shutdown_connections 2024-04-25T10:10:55.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.186+0000 7f71030b6640 1 -- 172.21.15.133:0/1734316801 wait complete. 2024-04-25T10:10:55.206 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:55.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ace817640 1 Processor -- start 2024-04-25T10:10:55.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ace817640 1 -- start start 2024-04-25T10:10:55.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ace817640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80cd820 0x7f8ac80cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ace817640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac80ce2e0 con 0x7f8ac8145170 2024-04-25T10:10:55.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ace817640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac806d4b0 con 0x7f8ac815ced0 2024-04-25T10:10:55.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ace817640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac806db00 con 0x7f8ac80cd820 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8ac8145170 0x7f8ac806df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54632/0 (socket says 172.21.15.133:54632) 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac7fff640 1 -- 172.21.15.133:0/2722213951 learned_addr learned my addr 172.21.15.133:0/2722213951 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac77fe640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80cd820 0x7f8ac80cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815592637 0 0) 0x7f8ac80ce2e0 con 0x7f8ac8145170 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ab0003530 con 0x7f8ac8145170 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3683996087 0 0) 0x7f8ab0003530 con 0x7f8ac8145170 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80cd820 msgr2=0x7f8ac80cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80cd820 0x7f8ac80cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 >> v1:172.21.15.162:6789/0 conn(0x7f8ac815ced0 legacy=0x7f8ac80b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac806d790 con 0x7f8ac8145170 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f8ab8001680 con 0x7f8ac8145170 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 >> v1:172.21.15.133:6789/0 conn(0x7f8ac8145170 legacy=0x7f8ac806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80cd820 0x7f8ab00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ab0003f70 0x7f8ab0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ab0004d50 0x7f8ab00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac806db00 con 0x7f8ab0003f70 2024-04-25T10:10:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab000bab0 con 0x7f8ab0004d50 2024-04-25T10:10:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab000bcb0 con 0x7f8ac80cd820 2024-04-25T10:10:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac77fe640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ab0003f70 0x7f8ab0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac7fff640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80cd820 0x7f8ab00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8accdb4640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ab0004d50 0x7f8ab00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac77fe640 1 -- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80cd820 msgr2=0x7f8ab00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac77fe640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80cd820 0x7f8ab00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac77fe640 1 -- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ab0004d50 msgr2=0x7f8ab00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac77fe640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ab0004d50 0x7f8ab00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac77fe640 1 -- 172.21.15.133:0/2722213951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab0003530 con 0x7f8ab0003f70 2024-04-25T10:10:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8accdb4640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ab0004d50 0x7f8ab00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.330+0000 7f8ac7fff640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80cd820 0x7f8ab00039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ac77fe640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ab0003f70 0x7f8ab0004810 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009ae0 tx=0x7f8abc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76553106ccb882a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0b2070 con 0x7f8ab0003f70 2024-04-25T10:10:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc0a5550 con 0x7f8ab0003f70 2024-04-25T10:10:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ac6ffd640 1 -- 172.21.15.133:0/2722213951 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0096b0 con 0x7f8ab0003f70 2024-04-25T10:10:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 -- 172.21.15.133:0/2722213951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ab0003f70 msgr2=0x7f8ab0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 --2- 172.21.15.133:0/2722213951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ab0003f70 0x7f8ab0004810 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009ae0 tx=0x7f8abc01ca70 comp rx=0 tx=0).stop 2024-04-25T10:10:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8accdb4640 1 -- 172.21.15.133:0/2722213951 reap_dead start 2024-04-25T10:10:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 -- 172.21.15.133:0/2722213951 shutdown_connections 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 -- 172.21.15.133:0/2722213951 >> 172.21.15.133:0/2722213951 conn(0x7f8ac8066f70 msgr2=0x7f8ac806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 -- 172.21.15.133:0/2722213951 shutdown_connections 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 -- 172.21.15.133:0/2722213951 wait complete. 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 Processor -- start 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 -- start start 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ab0003f70 0x7f8ac81520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ac80adb80 0x7f8ac80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80b2ab0 0x7f8ac80b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac815cb70 con 0x7f8ac80adb80 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac8145550 con 0x7f8ab0003f70 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ace817640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80b1dd0 con 0x7f8ac80b2ab0 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ac77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ac80adb80 0x7f8ac80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ac77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ac80adb80 0x7f8ac80ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52870/0 (socket says 172.21.15.133:52870) 2024-04-25T10:10:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.334+0000 7f8ac77fe640 1 -- 172.21.15.133:0/467100995 learned_addr learned my addr 172.21.15.133:0/467100995 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac7fff640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ab0003f70 0x7f8ac81520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8accdb4640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80b2ab0 0x7f8ac80b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac77fe640 1 -- 172.21.15.133:0/467100995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80b2ab0 msgr2=0x7f8ac80b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac77fe640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80b2ab0 0x7f8ac80b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac77fe640 1 -- 172.21.15.133:0/467100995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ab0003f70 msgr2=0x7f8ac81520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac77fe640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ab0003f70 0x7f8ac81520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac77fe640 1 -- 172.21.15.133:0/467100995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac8152620 con 0x7f8ac80adb80 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8accdb4640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80b2ab0 0x7f8ac80b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac77fe640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ac80adb80 0x7f8ac80ae000 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009ae0 tx=0x7f8ab0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac4ff9640 1 -- 172.21.15.133:0/467100995 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0b2040 con 0x7f8ac80adb80 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac4ff9640 1 -- 172.21.15.133:0/467100995 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc00f000 con 0x7f8ac80adb80 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ace817640 1 -- 172.21.15.133:0/467100995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac80b37c0 con 0x7f8ac80adb80 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac4ff9640 1 -- 172.21.15.133:0/467100995 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc004020 con 0x7f8ac80adb80 2024-04-25T10:10:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ace817640 1 -- 172.21.15.133:0/467100995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac80b3d80 con 0x7f8ac80adb80 2024-04-25T10:10:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ace817640 1 -- 172.21.15.133:0/467100995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac8068e80 con 0x7f8ac80adb80 2024-04-25T10:10:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac4ff9640 1 -- 172.21.15.133:0/467100995 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8abc0c0030 con 0x7f8ac80adb80 2024-04-25T10:10:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac4ff9640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ab4066f60 0x7f8ab4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac4ff9640 1 -- 172.21.15.133:0/467100995 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f8abc11ed80 con 0x7f8ac80adb80 2024-04-25T10:10:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac7fff640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ab4066f60 0x7f8ab4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.338+0000 7f8ac7fff640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ab4066f60 0x7f8ab4069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8000c10 tx=0x7f8ab8001560 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.342+0000 7f8ac4ff9640 1 -- 172.21.15.133:0/467100995 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8abc0c7060 con 0x7f8ac80adb80 2024-04-25T10:10:55.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 -- 172.21.15.133:0/467100995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ac815c4b0 con 0x7f8ac80adb80 2024-04-25T10:10:55.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ac4ff9640 1 -- 172.21.15.133:0/467100995 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8abc016780 con 0x7f8ac80adb80 2024-04-25T10:10:55.505 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:10:55.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 -- 172.21.15.133:0/467100995 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ab4066f60 msgr2=0x7f8ab4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ab4066f60 0x7f8ab4069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8000c10 tx=0x7f8ab8001560 comp rx=0 tx=0).stop 2024-04-25T10:10:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 -- 172.21.15.133:0/467100995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ac80adb80 msgr2=0x7f8ac80ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ac80adb80 0x7f8ac80ae000 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009ae0 tx=0x7f8ab0005840 comp rx=0 tx=0).stop 2024-04-25T10:10:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 -- 172.21.15.133:0/467100995 shutdown_connections 2024-04-25T10:10:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8ac80b2ab0 0x7f8ac80b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ab4066f60 0x7f8ab4069420 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ac80adb80 0x7f8ac80ae000 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 --2- 172.21.15.133:0/467100995 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ab0003f70 0x7f8ac81520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 -- 172.21.15.133:0/467100995 >> 172.21.15.133:0/467100995 conn(0x7f8ac8066f70 msgr2=0x7f8ac8142e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 -- 172.21.15.133:0/467100995 shutdown_connections 2024-04-25T10:10:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.502+0000 7f8ace817640 1 -- 172.21.15.133:0/467100995 wait complete. 2024-04-25T10:10:55.523 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:10:55.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.634+0000 7f6eadc1c640 1 Processor -- start 2024-04-25T10:10:55.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.634+0000 7f6eadc1c640 1 -- start start 2024-04-25T10:10:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6eadc1c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80af2d0 0x7f6ea80b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6eadc1c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea81471f0 con 0x7f6ea815eb40 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6eadc1c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea80b1090 con 0x7f6ea806e8b0 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6eadc1c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea8147490 con 0x7f6ea80af2d0 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea6ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6ea815eb40 0x7f6ea80aebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54644/0 (socket says 172.21.15.133:54644) 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea6ffd640 1 -- 172.21.15.133:0/1669441903 learned_addr learned my addr 172.21.15.133:0/1669441903 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806881871 0 0) 0x7f6ea81471f0 con 0x7f6ea815eb40 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea7fff640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80af2d0 0x7f6ea80b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e90003530 con 0x7f6ea815eb40 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 924773766 0 0) 0x7f6e90003530 con 0x7f6ea815eb40 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80af2d0 msgr2=0x7f6ea80b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80af2d0 0x7f6ea80b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 >> v1:172.21.15.162:6789/0 conn(0x7f6ea806e8b0 legacy=0x7f6ea8146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea80b1370 con 0x7f6ea815eb40 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6e98001680 con 0x7f6ea815eb40 2024-04-25T10:10:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 >> v1:172.21.15.133:6789/0 conn(0x7f6ea815eb40 legacy=0x7f6ea80aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80af2d0 0x7f6e900039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e90003f70 0x7f6e90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e90004d50 0x7f6e900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea8147490 con 0x7f6e90003f70 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9000bb60 con 0x7f6e90004d50 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9000bd60 con 0x7f6ea80af2d0 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea6ffd640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e90003f70 0x7f6e90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea77fe640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80af2d0 0x7f6e900039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea7fff640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e90004d50 0x7f6e900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea6ffd640 1 -- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80af2d0 msgr2=0x7f6e900039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea6ffd640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80af2d0 0x7f6e900039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea6ffd640 1 -- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e90004d50 msgr2=0x7f6e900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea6ffd640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e90004d50 0x7f6e900051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea6ffd640 1 -- 172.21.15.133:0/1669441903 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e90003530 con 0x7f6e90003f70 2024-04-25T10:10:55.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea7fff640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e90004d50 0x7f6e900051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:55.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.638+0000 7f6ea6ffd640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e90003f70 0x7f6e90004810 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800c240 tx=0x7f6e9801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1e6d6008c7810f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98007d20 con 0x7f6e90003f70 2024-04-25T10:10:55.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e980a9070 con 0x7f6e90003f70 2024-04-25T10:10:55.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea67fc640 1 -- 172.21.15.133:0/1669441903 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98005d60 con 0x7f6e90003f70 2024-04-25T10:10:55.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 -- 172.21.15.133:0/1669441903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e90003f70 msgr2=0x7f6e90004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 --2- 172.21.15.133:0/1669441903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e90003f70 0x7f6e90004810 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800c240 tx=0x7f6e9801cc80 comp rx=0 tx=0).stop 2024-04-25T10:10:55.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea7fff640 1 -- 172.21.15.133:0/1669441903 reap_dead start 2024-04-25T10:10:55.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 -- 172.21.15.133:0/1669441903 shutdown_connections 2024-04-25T10:10:55.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 -- 172.21.15.133:0/1669441903 >> 172.21.15.133:0/1669441903 conn(0x7f6ea8066f70 msgr2=0x7f6ea806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 -- 172.21.15.133:0/1669441903 shutdown_connections 2024-04-25T10:10:55.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 -- 172.21.15.133:0/1669441903 wait complete. 2024-04-25T10:10:55.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 Processor -- start 2024-04-25T10:10:55.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 -- start start 2024-04-25T10:10:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e90003f70 0x7f6ea81386b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea8140210 0x7f6ea8136ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80acf70 0x7f6ea80ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea806de40 con 0x7f6ea8140210 2024-04-25T10:10:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea806dcc0 con 0x7f6e90003f70 2024-04-25T10:10:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80b19c0 con 0x7f6ea80acf70 2024-04-25T10:10:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea8140210 0x7f6ea8136ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea8140210 0x7f6ea8136ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52896/0 (socket says 172.21.15.133:52896) 2024-04-25T10:10:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea6ffd640 1 -- 172.21.15.133:0/3678568956 learned_addr learned my addr 172.21.15.133:0/3678568956 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea77fe640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e90003f70 0x7f6ea81386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea7fff640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80acf70 0x7f6ea80ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea6ffd640 1 -- 172.21.15.133:0/3678568956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80acf70 msgr2=0x7f6ea80ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea6ffd640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80acf70 0x7f6ea80ad3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea6ffd640 1 -- 172.21.15.133:0/3678568956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e90003f70 msgr2=0x7f6ea81386b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea6ffd640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e90003f70 0x7f6ea81386b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea6ffd640 1 -- 172.21.15.133:0/3678568956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea8136ff0 con 0x7f6ea8140210 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6ea6ffd640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea8140210 0x7f6ea8136ab0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6e9801c6f0 tx=0x7f6e90005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6e8ffff640 1 -- 172.21.15.133:0/3678568956 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98007d20 con 0x7f6ea8140210 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6e8ffff640 1 -- 172.21.15.133:0/3678568956 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e98005d40 con 0x7f6ea8140210 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6e8ffff640 1 -- 172.21.15.133:0/3678568956 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980a5ab0 con 0x7f6ea8140210 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 -- 172.21.15.133:0/3678568956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea8139f00 con 0x7f6ea8140210 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.642+0000 7f6eadc1c640 1 -- 172.21.15.133:0/3678568956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea813a1a0 con 0x7f6ea8140210 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.646+0000 7f6eadc1c640 1 -- 172.21.15.133:0/3678568956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea8068e80 con 0x7f6ea8140210 2024-04-25T10:10:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.646+0000 7f6e8ffff640 1 -- 172.21.15.133:0/3678568956 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e980a5c50 con 0x7f6ea8140210 2024-04-25T10:10:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.646+0000 7f6e8ffff640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e94066f90 0x7f6e94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.646+0000 7f6e8ffff640 1 -- 172.21.15.133:0/3678568956 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6e9811e340 con 0x7f6ea8140210 2024-04-25T10:10:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.646+0000 7f6ea77fe640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e94066f90 0x7f6e94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.646+0000 7f6ea77fe640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e94066f90 0x7f6e94069450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c000c10 tx=0x7f6e9c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.650+0000 7f6e8ffff640 1 -- 172.21.15.133:0/3678568956 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e980c0030 con 0x7f6ea8140210 2024-04-25T10:10:55.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.826+0000 7f6eadc1c640 1 -- 172.21.15.133:0/3678568956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6ea806b960 con 0x7f6ea8140210 2024-04-25T10:10:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.826+0000 7f6e8ffff640 1 -- 172.21.15.133:0/3678568956 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6e98016990 con 0x7f6ea8140210 2024-04-25T10:10:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:10:55.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 -- 172.21.15.133:0/3678568956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e94066f90 msgr2=0x7f6e94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e94066f90 0x7f6e94069450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c000c10 tx=0x7f6e9c009040 comp rx=0 tx=0).stop 2024-04-25T10:10:55.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 -- 172.21.15.133:0/3678568956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea8140210 msgr2=0x7f6ea8136ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea8140210 0x7f6ea8136ab0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6e9801c6f0 tx=0x7f6e90005840 comp rx=0 tx=0).stop 2024-04-25T10:10:55.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 -- 172.21.15.133:0/3678568956 shutdown_connections 2024-04-25T10:10:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ea80acf70 0x7f6ea80ad3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e94066f90 0x7f6e94069450 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ea8140210 0x7f6ea8136ab0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 --2- 172.21.15.133:0/3678568956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e90003f70 0x7f6ea81386b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 -- 172.21.15.133:0/3678568956 >> 172.21.15.133:0/3678568956 conn(0x7f6ea8066f70 msgr2=0x7f6ea8161310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 -- 172.21.15.133:0/3678568956 shutdown_connections 2024-04-25T10:10:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:55.830+0000 7f6eadc1c640 1 -- 172.21.15.133:0/3678568956 wait complete. 2024-04-25T10:10:55.848 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:10:55.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:55.849 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:55.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[72094]: fuse finished with error 0 and tester_r 0 2024-04-25T10:10:55.944 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:10:55.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:55.944 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:10:56.136 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:10:56.136 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:56.147 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:56.148 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:10:56.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:56.148 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:10:56.194 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:10:56.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:10:56.195 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:10:56.195 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:56.200 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:10:56.200 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:10:56.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:56.200 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:10:56.247 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:10:56.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:10:56.248 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:10:56.248 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:56.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:56.253 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:10:56.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:56.254 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:10:56.302 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:10:56.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:10:56.303 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:56.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6be94b640 1 Processor -- start 2024-04-25T10:10:56.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6be94b640 1 -- start start 2024-04-25T10:10:56.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6be94b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa6b80b1f70 0x7fa6b806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6be94b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa6b80ce330 0x7fa6b80ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6be94b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b80c61d0 con 0x7fa6b815c9a0 2024-04-25T10:10:56.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6be94b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b80cecc0 con 0x7fa6b80ce330 2024-04-25T10:10:56.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6be94b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b80c64b0 con 0x7fa6b80b1f70 2024-04-25T10:10:56.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6bcee8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa6b80ce330 0x7fa6b80ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6bcee8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa6b80ce330 0x7fa6b80ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45584/0 (socket says 172.21.15.133:45584) 2024-04-25T10:10:56.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6bcee8640 1 -- 172.21.15.133:0/669609766 learned_addr learned my addr 172.21.15.133:0/669609766 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:56.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6affff640 1 --2- 172.21.15.133:0/669609766 >> v2:172.21.15.162:3300/0 conn(0x7fa6b80b1f70 0x7fa6b806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6bcee8640 1 -- 172.21.15.133:0/669609766 >> v2:172.21.15.162:3300/0 conn(0x7fa6b80b1f70 msgr2=0x7fa6b806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6bcee8640 1 --2- 172.21.15.133:0/669609766 >> v2:172.21.15.162:3300/0 conn(0x7fa6b80b1f70 0x7fa6b806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6bcee8640 1 -- 172.21.15.133:0/669609766 >> v1:172.21.15.162:6789/0 conn(0x7fa6b815c9a0 legacy=0x7fa6b80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.426+0000 7fa6bcee8640 1 -- 172.21.15.133:0/669609766 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b806d4f0 con 0x7fa6b80ce330 2024-04-25T10:10:56.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6bcee8640 1 --2- 172.21.15.133:0/669609766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b80ce330 0x7fa6b80ce780 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8072670 tx=0x7fa6b001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e203b67b3bce5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b77fe640 1 -- 172.21.15.133:0/669609766 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00a5750 con 0x7fa6b80ce330 2024-04-25T10:10:56.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b77fe640 1 -- 172.21.15.133:0/669609766 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b00a58f0 con 0x7fa6b80ce330 2024-04-25T10:10:56.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b77fe640 1 -- 172.21.15.133:0/669609766 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00a5cc0 con 0x7fa6b80ce330 2024-04-25T10:10:56.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 -- 172.21.15.133:0/669609766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b80ce330 msgr2=0x7fa6b80ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 --2- 172.21.15.133:0/669609766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b80ce330 0x7fa6b80ce780 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8072670 tx=0x7fa6b001c980 comp rx=0 tx=0).stop 2024-04-25T10:10:56.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 -- 172.21.15.133:0/669609766 shutdown_connections 2024-04-25T10:10:56.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 --2- 172.21.15.133:0/669609766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b80ce330 0x7fa6b80ce780 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 --2- 172.21.15.133:0/669609766 >> v2:172.21.15.162:3300/0 conn(0x7fa6b80b1f70 0x7fa6b806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 -- 172.21.15.133:0/669609766 >> 172.21.15.133:0/669609766 conn(0x7fa6b8067000 msgr2=0x7fa6b806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:56.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 -- 172.21.15.133:0/669609766 shutdown_connections 2024-04-25T10:10:56.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 -- 172.21.15.133:0/669609766 wait complete. 2024-04-25T10:10:56.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 Processor -- start 2024-04-25T10:10:56.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 -- start start 2024-04-25T10:10:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b80b1f70 0x7fa6b8152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b80ce330 0x7fa6b80b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6b815c9a0 0x7fa6b80cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b815dad0 con 0x7fa6b80b1f70 2024-04-25T10:10:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b815d950 con 0x7fa6b80ce330 2024-04-25T10:10:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6be94b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b815dc50 con 0x7fa6b815c9a0 2024-04-25T10:10:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b80b1f70 0x7fa6b8152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b80b1f70 0x7fa6b8152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45600/0 (socket says 172.21.15.133:45600) 2024-04-25T10:10:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b7fff640 1 -- 172.21.15.133:0/3065141185 learned_addr learned my addr 172.21.15.133:0/3065141185 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6affff640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b80ce330 0x7fa6b80b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6bcee8640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6b815c9a0 0x7fa6b80cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b7fff640 1 -- 172.21.15.133:0/3065141185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6b815c9a0 msgr2=0x7fa6b80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b7fff640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6b815c9a0 0x7fa6b80cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b7fff640 1 -- 172.21.15.133:0/3065141185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b80ce330 msgr2=0x7fa6b80b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b7fff640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b80ce330 0x7fa6b80b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b7fff640 1 -- 172.21.15.133:0/3065141185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b80cc1c0 con 0x7fa6b80b1f70 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6affff640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b80ce330 0x7fa6b80b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.430+0000 7fa6b7fff640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b80b1f70 0x7fa6b8152660 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800b440 tx=0x7fa6a800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.434+0000 7fa6b57fa640 1 -- 172.21.15.133:0/3065141185 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8015020 con 0x7fa6b80b1f70 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.434+0000 7fa6b57fa640 1 -- 172.21.15.133:0/3065141185 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a8011890 con 0x7fa6b80b1f70 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.434+0000 7fa6be94b640 1 -- 172.21.15.133:0/3065141185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b80cc4a0 con 0x7fa6b80b1f70 2024-04-25T10:10:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.434+0000 7fa6b57fa640 1 -- 172.21.15.133:0/3065141185 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8011c80 con 0x7fa6b80b1f70 2024-04-25T10:10:56.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.434+0000 7fa6be94b640 1 -- 172.21.15.133:0/3065141185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b80cca60 con 0x7fa6b80b1f70 2024-04-25T10:10:56.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.434+0000 7fa6be94b640 1 -- 172.21.15.133:0/3065141185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b8068ef0 con 0x7fa6b80b1f70 2024-04-25T10:10:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.434+0000 7fa6b57fa640 1 -- 172.21.15.133:0/3065141185 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6a8018020 con 0x7fa6b80b1f70 2024-04-25T10:10:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.434+0000 7fa6b57fa640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa698066f60 0x7fa698069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.434+0000 7fa6b57fa640 1 -- 172.21.15.133:0/3065141185 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fa6a80855a0 con 0x7fa6b80b1f70 2024-04-25T10:10:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.434+0000 7fa6affff640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa698066f60 0x7fa698069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.434+0000 7fa6affff640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa698066f60 0x7fa698069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa6b815eb60 tx=0x7fa6a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.438+0000 7fa6b57fa640 1 -- 172.21.15.133:0/3065141185 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6a801f030 con 0x7fa6b80b1f70 2024-04-25T10:10:56.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.634+0000 7fa6be94b640 1 -- 172.21.15.133:0/3065141185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6b8150bc0 con 0x7fa6b80b1f70 2024-04-25T10:10:56.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.634+0000 7fa6b57fa640 1 -- 172.21.15.133:0/3065141185 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fa6b8150bc0 con 0x7fa6b80b1f70 2024-04-25T10:10:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:56.637 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3089710291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3089710291},{"type":"v1","addr":"172.21.15.162:6834","nonce":3089710291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1390003376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1390003376},{"type":"v1","addr":"172.21.15.162:6835","nonce":1390003376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7866,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1134679682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1134679682},{"type":"v1","addr":"172.21.15.133:6836","nonce":1134679682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:27.661584+0000","modified":"2024-04-25T10:10:30.680424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3580592279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3580592279},{"type":"v1","addr":"172.21.15.133:6837","nonce":3580592279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:56.637 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 58 2024-04-25T10:10:56.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.634+0000 7fa6be94b640 1 -- 172.21.15.133:0/3065141185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa698066f60 msgr2=0x7fa698069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.634+0000 7fa6be94b640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa698066f60 0x7fa698069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa6b815eb60 tx=0x7fa6a0008040 comp rx=0 tx=0).stop 2024-04-25T10:10:56.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.634+0000 7fa6be94b640 1 -- 172.21.15.133:0/3065141185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b80b1f70 msgr2=0x7fa6b8152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.634+0000 7fa6be94b640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b80b1f70 0x7fa6b8152660 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800b440 tx=0x7fa6a800b900 comp rx=0 tx=0).stop 2024-04-25T10:10:56.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.634+0000 7fa6be94b640 1 -- 172.21.15.133:0/3065141185 shutdown_connections 2024-04-25T10:10:56.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.634+0000 7fa6be94b640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa6b815c9a0 0x7fa6b80cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.634+0000 7fa6be94b640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6b80ce330 0x7fa6b80b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.634+0000 7fa6be94b640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa698066f60 0x7fa698069420 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.634+0000 7fa6be94b640 1 --2- 172.21.15.133:0/3065141185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6b80b1f70 0x7fa6b8152660 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.638+0000 7fa6be94b640 1 -- 172.21.15.133:0/3065141185 >> 172.21.15.133:0/3065141185 conn(0x7fa6b8067000 msgr2=0x7fa6b815b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:56.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.638+0000 7fa6be94b640 1 -- 172.21.15.133:0/3065141185 shutdown_connections 2024-04-25T10:10:56.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.638+0000 7fa6be94b640 1 -- 172.21.15.133:0/3065141185 wait complete. 2024-04-25T10:10:56.657 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:56.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.774+0000 7fc81fb80640 1 Processor -- start 2024-04-25T10:10:56.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81fb80640 1 -- start start 2024-04-25T10:10:56.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81fb80640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc818158110 0x7fc8180b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81fb80640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8181584f0 0x7fc81815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81fb80640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806e460 0x7fc81806e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81fb80640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81806ee10 con 0x7fc818158110 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81fb80640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8180cbe60 con 0x7fc8181584f0 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81fb80640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180b19b0 con 0x7fc81806e460 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81d91c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc818158110 0x7fc8180b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81d91c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc818158110 0x7fc8180b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45610/0 (socket says 172.21.15.133:45610) 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81d91c640 1 -- 172.21.15.133:0/595233899 learned_addr learned my addr 172.21.15.133:0/595233899 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81e11d640 1 --2- 172.21.15.133:0/595233899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806e460 0x7fc81806e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81d11b640 1 --2- 172.21.15.133:0/595233899 >> v2:172.21.15.162:3300/0 conn(0x7fc8181584f0 0x7fc81815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81d91c640 1 -- 172.21.15.133:0/595233899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806e460 msgr2=0x7fc81806e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81d91c640 1 --2- 172.21.15.133:0/595233899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806e460 0x7fc81806e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81d91c640 1 -- 172.21.15.133:0/595233899 >> v2:172.21.15.162:3300/0 conn(0x7fc8181584f0 msgr2=0x7fc81815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81d91c640 1 --2- 172.21.15.133:0/595233899 >> v2:172.21.15.162:3300/0 conn(0x7fc8181584f0 0x7fc81815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81d91c640 1 -- 172.21.15.133:0/595233899 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81815b360 con 0x7fc818158110 2024-04-25T10:10:56.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81d11b640 1 --2- 172.21.15.133:0/595233899 >> v2:172.21.15.162:3300/0 conn(0x7fc8181584f0 0x7fc81815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.778+0000 7fc81d91c640 1 --2- 172.21.15.133:0/595233899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818158110 0x7fc8180b1f00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc8180b4da0 tx=0x7fc81401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6096cd0916848aec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81c91a640 1 -- 172.21.15.133:0/595233899 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8140aa610 con 0x7fc818158110 2024-04-25T10:10:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81c91a640 1 -- 172.21.15.133:0/595233899 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8140af070 con 0x7fc818158110 2024-04-25T10:10:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81c91a640 1 -- 172.21.15.133:0/595233899 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8140aa9d0 con 0x7fc818158110 2024-04-25T10:10:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 -- 172.21.15.133:0/595233899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818158110 msgr2=0x7fc8180b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 --2- 172.21.15.133:0/595233899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818158110 0x7fc8180b1f00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc8180b4da0 tx=0x7fc81401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 -- 172.21.15.133:0/595233899 shutdown_connections 2024-04-25T10:10:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 --2- 172.21.15.133:0/595233899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc81806e460 0x7fc81806e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 --2- 172.21.15.133:0/595233899 >> v2:172.21.15.162:3300/0 conn(0x7fc8181584f0 0x7fc81815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 --2- 172.21.15.133:0/595233899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818158110 0x7fc8180b1f00 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 -- 172.21.15.133:0/595233899 >> 172.21.15.133:0/595233899 conn(0x7fc818067000 msgr2=0x7fc81806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:56.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 -- 172.21.15.133:0/595233899 shutdown_connections 2024-04-25T10:10:56.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 -- 172.21.15.133:0/595233899 wait complete. 2024-04-25T10:10:56.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 Processor -- start 2024-04-25T10:10:56.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 -- start start 2024-04-25T10:10:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc81806e460 0x7fc8181f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818158110 0x7fc8181633c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8181584f0 0x7fc81815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180cd7f0 con 0x7fc818158110 2024-04-25T10:10:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180cd670 con 0x7fc81806e460 2024-04-25T10:10:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81815eb10 con 0x7fc8181584f0 2024-04-25T10:10:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81d11b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818158110 0x7fc8181633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81d11b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818158110 0x7fc8181633c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45614/0 (socket says 172.21.15.133:45614) 2024-04-25T10:10:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81d11b640 1 -- 172.21.15.133:0/504342919 learned_addr learned my addr 172.21.15.133:0/504342919 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81d91c640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc81806e460 0x7fc8181f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81e11d640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8181584f0 0x7fc81815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81d11b640 1 -- 172.21.15.133:0/504342919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8181584f0 msgr2=0x7fc81815ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81d11b640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8181584f0 0x7fc81815ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81d11b640 1 -- 172.21.15.133:0/504342919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc81806e460 msgr2=0x7fc8181f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81d11b640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc81806e460 0x7fc8181f57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81d11b640 1 -- 172.21.15.133:0/504342919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818163ed0 con 0x7fc818158110 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81d91c640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc81806e460 0x7fc8181f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81d11b640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818158110 0x7fc8181633c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fc80800c7f0 tx=0x7fc80800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc80e7fc640 1 -- 172.21.15.133:0/504342919 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808015070 con 0x7fc818158110 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc80e7fc640 1 -- 172.21.15.133:0/504342919 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808007820 con 0x7fc818158110 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc80e7fc640 1 -- 172.21.15.133:0/504342919 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808007c10 con 0x7fc818158110 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 -- 172.21.15.133:0/504342919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8181641b0 con 0x7fc818158110 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.782+0000 7fc81fb80640 1 -- 172.21.15.133:0/504342919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8181562f0 con 0x7fc818158110 2024-04-25T10:10:56.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.786+0000 7fc81fb80640 1 -- 172.21.15.133:0/504342919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc818068ef0 con 0x7fc818158110 2024-04-25T10:10:56.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.786+0000 7fc80e7fc640 1 -- 172.21.15.133:0/504342919 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc808018070 con 0x7fc818158110 2024-04-25T10:10:56.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.786+0000 7fc80e7fc640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7fc066f60 0x7fc7fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.786+0000 7fc80e7fc640 1 -- 172.21.15.133:0/504342919 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc808085220 con 0x7fc818158110 2024-04-25T10:10:56.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.786+0000 7fc81d91c640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7fc066f60 0x7fc7fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.786+0000 7fc81d91c640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7fc066f60 0x7fc7fc069420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc8140ae000 tx=0x7fc814008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.790+0000 7fc80e7fc640 1 -- 172.21.15.133:0/504342919 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc8080560c0 con 0x7fc818158110 2024-04-25T10:10:56.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.950+0000 7fc81fb80640 1 -- 172.21.15.133:0/504342919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc81806c2d0 con 0x7fc818158110 2024-04-25T10:10:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc80e7fc640 1 -- 172.21.15.133:0/504342919 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17103 (secure 0 0 0) 0x7fc8080559c0 con 0x7fc818158110 2024-04-25T10:10:56.956 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:56.956 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":96,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:10:33.344314+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:25.920690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:26.640190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:56.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 -- 172.21.15.133:0/504342919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7fc066f60 msgr2=0x7fc7fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7fc066f60 0x7fc7fc069420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc8140ae000 tx=0x7fc814008040 comp rx=0 tx=0).stop 2024-04-25T10:10:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 -- 172.21.15.133:0/504342919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818158110 msgr2=0x7fc8181633c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818158110 0x7fc8181633c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fc80800c7f0 tx=0x7fc80800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 -- 172.21.15.133:0/504342919 shutdown_connections 2024-04-25T10:10:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8181584f0 0x7fc81815ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7fc066f60 0x7fc7fc069420 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818158110 0x7fc8181633c0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 --2- 172.21.15.133:0/504342919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc81806e460 0x7fc8181f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 -- 172.21.15.133:0/504342919 >> 172.21.15.133:0/504342919 conn(0x7fc818067000 msgr2=0x7fc8180ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 -- 172.21.15.133:0/504342919 shutdown_connections 2024-04-25T10:10:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:56.954+0000 7fc81fb80640 1 -- 172.21.15.133:0/504342919 wait complete. 2024-04-25T10:10:56.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:10:25.920690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:10:26.640190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:56.976 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:57.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.098+0000 7fe86325e640 1 Processor -- start 2024-04-25T10:10:57.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.098+0000 7fe86325e640 1 -- start start 2024-04-25T10:10:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.098+0000 7fe86325e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c143ea0 0x7fe85c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.098+0000 7fe86325e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe85c0b2150 0x7fe85c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.098+0000 7fe86325e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe85c15b9b0 0x7fe85c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.098+0000 7fe86325e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85c15ac50 con 0x7fe85c0b2150 2024-04-25T10:10:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.098+0000 7fe86325e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85c0cd180 con 0x7fe85c15b9b0 2024-04-25T10:10:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.098+0000 7fe86325e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c15aed0 con 0x7fe85c143ea0 2024-04-25T10:10:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe85c0b2150 0x7fe85c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe85c0b2150 0x7fe85c06e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45626/0 (socket says 172.21.15.133:45626) 2024-04-25T10:10:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85bfff640 1 -- 172.21.15.133:0/2000482555 learned_addr learned my addr 172.21.15.133:0/2000482555 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:57.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe860ffa640 1 --2- 172.21.15.133:0/2000482555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c143ea0 0x7fe85c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe8617fb640 1 --2- 172.21.15.133:0/2000482555 >> v2:172.21.15.162:3300/0 conn(0x7fe85c15b9b0 0x7fe85c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85bfff640 1 -- 172.21.15.133:0/2000482555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c143ea0 msgr2=0x7fe85c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:57.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85bfff640 1 --2- 172.21.15.133:0/2000482555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c143ea0 0x7fe85c0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85bfff640 1 -- 172.21.15.133:0/2000482555 >> v2:172.21.15.162:3300/0 conn(0x7fe85c15b9b0 msgr2=0x7fe85c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:57.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85bfff640 1 --2- 172.21.15.133:0/2000482555 >> v2:172.21.15.162:3300/0 conn(0x7fe85c15b9b0 0x7fe85c06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85bfff640 1 -- 172.21.15.133:0/2000482555 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c15b1b0 con 0x7fe85c0b2150 2024-04-25T10:10:57.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85bfff640 1 --2- 172.21.15.133:0/2000482555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c0b2150 0x7fe85c06e3b0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe85c151d40 tx=0x7fe85001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3defe193a7c6b39d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85b7fe640 1 -- 172.21.15.133:0/2000482555 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8500b0070 con 0x7fe85c0b2150 2024-04-25T10:10:57.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85b7fe640 1 -- 172.21.15.133:0/2000482555 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8500ab600 con 0x7fe85c0b2150 2024-04-25T10:10:57.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe85b7fe640 1 -- 172.21.15.133:0/2000482555 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8500ab9d0 con 0x7fe85c0b2150 2024-04-25T10:10:57.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe86325e640 1 -- 172.21.15.133:0/2000482555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c0b2150 msgr2=0x7fe85c06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe86325e640 1 --2- 172.21.15.133:0/2000482555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c0b2150 0x7fe85c06e3b0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe85c151d40 tx=0x7fe85001c520 comp rx=0 tx=0).stop 2024-04-25T10:10:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe86325e640 1 -- 172.21.15.133:0/2000482555 shutdown_connections 2024-04-25T10:10:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe86325e640 1 --2- 172.21.15.133:0/2000482555 >> v2:172.21.15.162:3300/0 conn(0x7fe85c15b9b0 0x7fe85c06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe86325e640 1 --2- 172.21.15.133:0/2000482555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c0b2150 0x7fe85c06e3b0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe86325e640 1 --2- 172.21.15.133:0/2000482555 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c143ea0 0x7fe85c0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe86325e640 1 -- 172.21.15.133:0/2000482555 >> 172.21.15.133:0/2000482555 conn(0x7fe85c066f70 msgr2=0x7fe85c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe86325e640 1 -- 172.21.15.133:0/2000482555 shutdown_connections 2024-04-25T10:10:57.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.102+0000 7fe86325e640 1 -- 172.21.15.133:0/2000482555 wait complete. 2024-04-25T10:10:57.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe86325e640 1 Processor -- start 2024-04-25T10:10:57.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe86325e640 1 -- start start 2024-04-25T10:10:57.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe86325e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0b2150 0x7fe85c155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe86325e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c143ea0 0x7fe85c141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe86325e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c15b9b0 0x7fe85c142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe86325e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c15ec60 con 0x7fe85c15b9b0 2024-04-25T10:10:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe86325e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0ccf60 con 0x7fe85c143ea0 2024-04-25T10:10:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe86325e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c15ede0 con 0x7fe85c0b2150 2024-04-25T10:10:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8617fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c15b9b0 0x7fe85c142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8617fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c15b9b0 0x7fe85c142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45642/0 (socket says 172.21.15.133:45642) 2024-04-25T10:10:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8617fb640 1 -- 172.21.15.133:0/2158452215 learned_addr learned my addr 172.21.15.133:0/2158452215 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe860ffa640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0b2150 0x7fe85c155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe85bfff640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c143ea0 0x7fe85c141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8617fb640 1 -- 172.21.15.133:0/2158452215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0b2150 msgr2=0x7fe85c155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8617fb640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0b2150 0x7fe85c155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8617fb640 1 -- 172.21.15.133:0/2158452215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c143ea0 msgr2=0x7fe85c141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8617fb640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c143ea0 0x7fe85c141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8617fb640 1 -- 172.21.15.133:0/2158452215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c157e30 con 0x7fe85c15b9b0 2024-04-25T10:10:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe85bfff640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c143ea0 0x7fe85c141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:57.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8617fb640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c15b9b0 0x7fe85c142350 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe854007d70 tx=0x7fe85400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8597fa640 1 -- 172.21.15.133:0/2158452215 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854014420 con 0x7fe85c15b9b0 2024-04-25T10:10:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8597fa640 1 -- 172.21.15.133:0/2158452215 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe854019070 con 0x7fe85c15b9b0 2024-04-25T10:10:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe86325e640 1 -- 172.21.15.133:0/2158452215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c1580d0 con 0x7fe85c15b9b0 2024-04-25T10:10:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8597fa640 1 -- 172.21.15.133:0/2158452215 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854014820 con 0x7fe85c15b9b0 2024-04-25T10:10:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe86325e640 1 -- 172.21.15.133:0/2158452215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c13f5d0 con 0x7fe85c15b9b0 2024-04-25T10:10:57.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe86325e640 1 -- 172.21.15.133:0/2158452215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85c158580 con 0x7fe85c15b9b0 2024-04-25T10:10:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.106+0000 7fe8597fa640 1 -- 172.21.15.133:0/2158452215 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8540149c0 con 0x7fe85c15b9b0 2024-04-25T10:10:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.110+0000 7fe8597fa640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 0x7fe848069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.110+0000 7fe8597fa640 1 -- 172.21.15.133:0/2158452215 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fe854085980 con 0x7fe85c15b9b0 2024-04-25T10:10:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.110+0000 7fe860ffa640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 0x7fe848069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.110+0000 7fe860ffa640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 0x7fe848069450 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe85c1f9c20 tx=0x7fe84c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.114+0000 7fe8597fa640 1 -- 172.21.15.133:0/2158452215 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe854056790 con 0x7fe85c15b9b0 2024-04-25T10:10:57.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 -- 172.21.15.133:0/2158452215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe85c152f80 con 0x7fe85c15b9b0 2024-04-25T10:10:57.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe8597fa640 1 -- 172.21.15.133:0/2158452215 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe854056090 con 0x7fe85c15b9b0 2024-04-25T10:10:57.276 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:10:57.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 -- 172.21.15.133:0/2158452215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 msgr2=0x7fe848069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 0x7fe848069450 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe85c1f9c20 tx=0x7fe84c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 -- 172.21.15.133:0/2158452215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c15b9b0 msgr2=0x7fe85c142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c15b9b0 0x7fe85c142350 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe854007d70 tx=0x7fe85400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 -- 172.21.15.133:0/2158452215 shutdown_connections 2024-04-25T10:10:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 0x7fe848069450 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c15b9b0 0x7fe85c142350 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c143ea0 0x7fe85c141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 --2- 172.21.15.133:0/2158452215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0b2150 0x7fe85c155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 -- 172.21.15.133:0/2158452215 >> 172.21.15.133:0/2158452215 conn(0x7fe85c066f70 msgr2=0x7fe85c158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 -- 172.21.15.133:0/2158452215 shutdown_connections 2024-04-25T10:10:57.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.274+0000 7fe86325e640 1 -- 172.21.15.133:0/2158452215 wait complete. 2024-04-25T10:10:57.295 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:57.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f1251c6b640 1 Processor -- start 2024-04-25T10:10:57.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f1251c6b640 1 -- start start 2024-04-25T10:10:57.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f1251c6b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f124c1645e0 0x7f124c1669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f1251c6b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c0b2f80 con 0x7f124c0b2770 2024-04-25T10:10:57.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f1251c6b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c0b2ca0 con 0x7f124c06ac90 2024-04-25T10:10:57.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f1251c6b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124c0cc350 con 0x7f124c1645e0 2024-04-25T10:10:57.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f124bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f124c1645e0 0x7f124c1669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f124affd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f124c0b2770 0x7f124c06b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:39950/0 (socket says 172.21.15.133:39950) 2024-04-25T10:10:57.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f124affd640 1 -- 172.21.15.133:0/1389619016 learned_addr learned my addr 172.21.15.133:0/1389619016 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:57.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f124bfff640 1 -- 172.21.15.133:0/1389619016 >> v1:172.21.15.162:6789/0 conn(0x7f124c06ac90 legacy=0x7f124c15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:57.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f124bfff640 1 -- 172.21.15.133:0/1389619016 >> v1:172.21.15.133:6789/0 conn(0x7f124c0b2770 legacy=0x7f124c06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f124a7fc640 1 -- 172.21.15.133:0/1389619016 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848764507 0 0) 0x7f124c0b2f80 con 0x7f124c0b2770 2024-04-25T10:10:57.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.418+0000 7f124bfff640 1 -- 172.21.15.133:0/1389619016 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0cc630 con 0x7f124c1645e0 2024-04-25T10:10:57.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f124bfff640 1 --2- 172.21.15.133:0/1389619016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124c1645e0 0x7f124c1669d0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1244009d90 tx=0x7f124401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbb0bb0bb0d5243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f124a7fc640 1 -- 172.21.15.133:0/1389619016 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440b2070 con 0x7f124c1645e0 2024-04-25T10:10:57.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f124a7fc640 1 -- 172.21.15.133:0/1389619016 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12440a5940 con 0x7f124c1645e0 2024-04-25T10:10:57.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f124a7fc640 1 -- 172.21.15.133:0/1389619016 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440a5d10 con 0x7f124c1645e0 2024-04-25T10:10:57.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 -- 172.21.15.133:0/1389619016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124c1645e0 msgr2=0x7f124c1669d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 --2- 172.21.15.133:0/1389619016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124c1645e0 0x7f124c1669d0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1244009d90 tx=0x7f124401c970 comp rx=0 tx=0).stop 2024-04-25T10:10:57.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 -- 172.21.15.133:0/1389619016 shutdown_connections 2024-04-25T10:10:57.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 --2- 172.21.15.133:0/1389619016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124c1645e0 0x7f124c1669d0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 -- 172.21.15.133:0/1389619016 >> 172.21.15.133:0/1389619016 conn(0x7f124c0999d0 msgr2=0x7f124c0b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 -- 172.21.15.133:0/1389619016 shutdown_connections 2024-04-25T10:10:57.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 -- 172.21.15.133:0/1389619016 wait complete. 2024-04-25T10:10:57.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 Processor -- start 2024-04-25T10:10:57.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 -- start start 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f124c06ac90 0x7f124c151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124c0b2770 0x7f124c1649c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124c1645e0 0x7f124c164f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c169730 con 0x7f124c0b2770 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c15f070 con 0x7f124c06ac90 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f1251c6b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c1698b0 con 0x7f124c1645e0 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f124affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124c0b2770 0x7f124c1649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f124affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124c0b2770 0x7f124c1649c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45656/0 (socket says 172.21.15.133:45656) 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.422+0000 7f124affd640 1 -- 172.21.15.133:0/3724918897 learned_addr learned my addr 172.21.15.133:0/3724918897 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f124bfff640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124c1645e0 0x7f124c164f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f124affd640 1 -- 172.21.15.133:0/3724918897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124c1645e0 msgr2=0x7f124c164f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f124b7fe640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f124c06ac90 0x7f124c151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f124affd640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124c1645e0 0x7f124c164f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f124affd640 1 -- 172.21.15.133:0/3724918897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f124c06ac90 msgr2=0x7f124c151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f124affd640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f124c06ac90 0x7f124c151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f124affd640 1 -- 172.21.15.133:0/3724918897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0c8630 con 0x7f124c0b2770 2024-04-25T10:10:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f124affd640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124c0b2770 0x7f124c1649c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f123400c4a0 tx=0x7f123400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f1243fff640 1 -- 172.21.15.133:0/3724918897 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234018070 con 0x7f124c0b2770 2024-04-25T10:10:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f1243fff640 1 -- 172.21.15.133:0/3724918897 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12340074e0 con 0x7f124c0b2770 2024-04-25T10:10:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f1251c6b640 1 -- 172.21.15.133:0/3724918897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c0c88d0 con 0x7f124c0b2770 2024-04-25T10:10:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f1243fff640 1 -- 172.21.15.133:0/3724918897 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234004a30 con 0x7f124c0b2770 2024-04-25T10:10:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f1251c6b640 1 -- 172.21.15.133:0/3724918897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c0c8e90 con 0x7f124c0b2770 2024-04-25T10:10:57.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f1251c6b640 1 -- 172.21.15.133:0/3724918897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124c068e80 con 0x7f124c0b2770 2024-04-25T10:10:57.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f1243fff640 1 -- 172.21.15.133:0/3724918897 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f123401b070 con 0x7f124c0b2770 2024-04-25T10:10:57.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f1243fff640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f122c066f60 0x7f122c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f1243fff640 1 -- 172.21.15.133:0/3724918897 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f1234084460 con 0x7f124c0b2770 2024-04-25T10:10:57.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f124b7fe640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f122c066f60 0x7f122c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.426+0000 7f124b7fe640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f122c066f60 0x7f122c069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f123c000f70 tx=0x7f123c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.430+0000 7f1243fff640 1 -- 172.21.15.133:0/3724918897 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1234055300 con 0x7f124c0b2770 2024-04-25T10:10:57.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 -- 172.21.15.133:0/3724918897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f124c06b7c0 con 0x7f124c0b2770 2024-04-25T10:10:57.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1243fff640 1 -- 172.21.15.133:0/3724918897 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1234054c00 con 0x7f124c0b2770 2024-04-25T10:10:57.593 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 -- 172.21.15.133:0/3724918897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f122c066f60 msgr2=0x7f122c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f122c066f60 0x7f122c069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f123c000f70 tx=0x7f123c009040 comp rx=0 tx=0).stop 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 -- 172.21.15.133:0/3724918897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124c0b2770 msgr2=0x7f124c1649c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124c0b2770 0x7f124c1649c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f123400c4a0 tx=0x7f123400c960 comp rx=0 tx=0).stop 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 -- 172.21.15.133:0/3724918897 shutdown_connections 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124c1645e0 0x7f124c164f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f122c066f60 0x7f122c069420 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124c0b2770 0x7f124c1649c0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 --2- 172.21.15.133:0/3724918897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f124c06ac90 0x7f124c151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 -- 172.21.15.133:0/3724918897 >> 172.21.15.133:0/3724918897 conn(0x7f124c0999d0 msgr2=0x7f124c0ad340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 -- 172.21.15.133:0/3724918897 shutdown_connections 2024-04-25T10:10:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.590+0000 7f1251c6b640 1 -- 172.21.15.133:0/3724918897 wait complete. 2024-04-25T10:10:57.611 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:57.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.722+0000 7ff5b6a5a640 1 Processor -- start 2024-04-25T10:10:57.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.722+0000 7ff5b6a5a640 1 -- start start 2024-04-25T10:10:57.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5b6a5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5b0145170 0x7ff5b006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5b6a5a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff5b006e4d0 0x7ff5b00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5b6a5a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b00ce2e0 con 0x7ff5b00cdd60 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5b6a5a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b006db00 con 0x7ff5b006e4d0 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5b6a5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0145550 con 0x7ff5b0145170 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5a7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff5b006e4d0 0x7ff5b00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5b4ff7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff5b00cdd60 0x7ff5b015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:39962/0 (socket says 172.21.15.133:39962) 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5a7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff5b006e4d0 0x7ff5b00cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45666/0 (socket says 172.21.15.133:45666) 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5b4ff7640 1 -- 172.21.15.133:0/1090181419 learned_addr learned my addr 172.21.15.133:0/1090181419 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5affff640 1 --2- 172.21.15.133:0/1090181419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5b0145170 0x7ff5b006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5a7fff640 1 -- 172.21.15.133:0/1090181419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5b0145170 msgr2=0x7ff5b006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5a7fff640 1 --2- 172.21.15.133:0/1090181419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5b0145170 0x7ff5b006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5a7fff640 1 -- 172.21.15.133:0/1090181419 >> v1:172.21.15.133:6789/0 conn(0x7ff5b00cdd60 legacy=0x7ff5b015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5a7fff640 1 -- 172.21.15.133:0/1090181419 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b006d4b0 con 0x7ff5b006e4d0 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5af7fe640 1 -- 172.21.15.133:0/1090181419 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110342344 0 0) 0x7ff5b00ce2e0 con 0x7ff5b00cdd60 2024-04-25T10:10:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5affff640 1 --2- 172.21.15.133:0/1090181419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5b0145170 0x7ff5b006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5a7fff640 1 --2- 172.21.15.133:0/1090181419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5b006e4d0 0x7ff5b00cd820 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff5980097c0 tx=0x7ff59801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d58cc126c19db32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5af7fe640 1 -- 172.21.15.133:0/1090181419 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5980b0070 con 0x7ff5b006e4d0 2024-04-25T10:10:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5af7fe640 1 -- 172.21.15.133:0/1090181419 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5980ab600 con 0x7ff5b006e4d0 2024-04-25T10:10:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.726+0000 7ff5af7fe640 1 -- 172.21.15.133:0/1090181419 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5980ab9d0 con 0x7ff5b006e4d0 2024-04-25T10:10:57.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/1090181419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5b006e4d0 msgr2=0x7ff5b00cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 --2- 172.21.15.133:0/1090181419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5b006e4d0 0x7ff5b00cd820 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff5980097c0 tx=0x7ff59801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/1090181419 shutdown_connections 2024-04-25T10:10:57.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 --2- 172.21.15.133:0/1090181419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5b006e4d0 0x7ff5b00cd820 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 --2- 172.21.15.133:0/1090181419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5b0145170 0x7ff5b006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/1090181419 >> 172.21.15.133:0/1090181419 conn(0x7ff5b0066f70 msgr2=0x7ff5b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/1090181419 shutdown_connections 2024-04-25T10:10:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/1090181419 wait complete. 2024-04-25T10:10:57.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 Processor -- start 2024-04-25T10:10:57.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 -- start start 2024-04-25T10:10:57.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5b006e4d0 0x7ff5b013cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5b00cdd60 0x7ff5b0163210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b0145170 0x7ff5b0163750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0168490 con 0x7ff5b00cdd60 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b015d7f0 con 0x7ff5b0145170 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0168610 con 0x7ff5b006e4d0 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5a7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5b00cdd60 0x7ff5b0163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5a7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5b00cdd60 0x7ff5b0163210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45678/0 (socket says 172.21.15.133:45678) 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5a7fff640 1 -- 172.21.15.133:0/3862207041 learned_addr learned my addr 172.21.15.133:0/3862207041 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5affff640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5b006e4d0 0x7ff5b013cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b4ff7640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b0145170 0x7ff5b0163750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5a7fff640 1 -- 172.21.15.133:0/3862207041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5b006e4d0 msgr2=0x7ff5b013cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5a7fff640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5b006e4d0 0x7ff5b013cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5a7fff640 1 -- 172.21.15.133:0/3862207041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b0145170 msgr2=0x7ff5b0163750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5a7fff640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b0145170 0x7ff5b0163750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5a7fff640 1 -- 172.21.15.133:0/3862207041 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b01641d0 con 0x7ff5b00cdd60 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5a7fff640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5b00cdd60 0x7ff5b0163210 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff59801a0d0 tx=0x7ff59801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5ad7fa640 1 -- 172.21.15.133:0/3862207041 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5980b0050 con 0x7ff5b00cdd60 2024-04-25T10:10:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5ad7fa640 1 -- 172.21.15.133:0/3862207041 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5980bc070 con 0x7ff5b00cdd60 2024-04-25T10:10:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/3862207041 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b0161830 con 0x7ff5b00cdd60 2024-04-25T10:10:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5ad7fa640 1 -- 172.21.15.133:0/3862207041 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5980b5a70 con 0x7ff5b00cdd60 2024-04-25T10:10:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.730+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/3862207041 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b0161df0 con 0x7ff5b00cdd60 2024-04-25T10:10:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.734+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/3862207041 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b0068e80 con 0x7ff5b00cdd60 2024-04-25T10:10:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.734+0000 7ff5ad7fa640 1 -- 172.21.15.133:0/3862207041 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5980ab480 con 0x7ff5b00cdd60 2024-04-25T10:10:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.734+0000 7ff5ad7fa640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff590066f60 0x7ff590069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.734+0000 7ff5affff640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff590066f60 0x7ff590069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.734+0000 7ff5ad7fa640 1 -- 172.21.15.133:0/3862207041 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff59811fba0 con 0x7ff5b00cdd60 2024-04-25T10:10:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.734+0000 7ff5affff640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff590066f60 0x7ff590069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0006fd0 tx=0x7ff5a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.738+0000 7ff5ad7fa640 1 -- 172.21.15.133:0/3862207041 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff598016090 con 0x7ff5b00cdd60 2024-04-25T10:10:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/3862207041 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5b006e150 con 0x7ff5b00cdd60 2024-04-25T10:10:57.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5ad7fa640 1 -- 172.21.15.133:0/3862207041 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5980f09b0 con 0x7ff5b00cdd60 2024-04-25T10:10:57.896 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:10:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/3862207041 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff590066f60 msgr2=0x7ff590069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff590066f60 0x7ff590069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0006fd0 tx=0x7ff5a0008040 comp rx=0 tx=0).stop 2024-04-25T10:10:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/3862207041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5b00cdd60 msgr2=0x7ff5b0163210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5b00cdd60 0x7ff5b0163210 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff59801a0d0 tx=0x7ff59801ace0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/3862207041 shutdown_connections 2024-04-25T10:10:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5b0145170 0x7ff5b0163750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff590066f60 0x7ff590069420 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5b00cdd60 0x7ff5b0163210 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 --2- 172.21.15.133:0/3862207041 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5b006e4d0 0x7ff5b013cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/3862207041 >> 172.21.15.133:0/3862207041 conn(0x7ff5b0066f70 msgr2=0x7ff5b00b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/3862207041 shutdown_connections 2024-04-25T10:10:57.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:57.894+0000 7ff5b6a5a640 1 -- 172.21.15.133:0/3862207041 wait complete. 2024-04-25T10:10:57.914 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f4605ceb640 1 Processor -- start 2024-04-25T10:10:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f4605ceb640 1 -- start start 2024-04-25T10:10:58.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f4605ceb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4600159dc0 0x7f46000cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f4605ceb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46000ce880 con 0x7f460015e220 2024-04-25T10:10:58.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f4605ceb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46000c49e0 con 0x7f460015a1a0 2024-04-25T10:10:58.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f4605ceb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46000ceb20 con 0x7f4600159dc0 2024-04-25T10:10:58.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f460015e220 0x7f460006d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:39974/0 (socket says 172.21.15.133:39974) 2024-04-25T10:10:58.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fffff640 1 -- 172.21.15.133:0/1869812287 learned_addr learned my addr 172.21.15.133:0/1869812287 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:58.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340231432 0 0) 0x7f46000ce880 con 0x7f460015e220 2024-04-25T10:10:58.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4600159dc0 0x7f46000cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e8003530 con 0x7f460015e220 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 644987363 0 0) 0x7f45e8003530 con 0x7f460015e220 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4600159dc0 msgr2=0x7f46000cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4600159dc0 0x7f46000cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 >> v1:172.21.15.162:6789/0 conn(0x7f460015a1a0 legacy=0x7f460015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46000cbfa0 con 0x7f460015e220 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f45f8001a20 con 0x7f460015e220 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 >> v1:172.21.15.133:6789/0 conn(0x7f460015e220 legacy=0x7f460006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e80039f0 0x7f45e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4600159dc0 0x7f45e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e8004d50 0x7f45e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46000ceb20 con 0x7f45e80039f0 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e800bbb0 con 0x7f45e8004d50 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e800bdb0 con 0x7f4600159dc0 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e80039f0 0x7f45e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45feffd640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4600159dc0 0x7f45e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fffff640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e8004d50 0x7f45e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45ff7fe640 1 -- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4600159dc0 msgr2=0x7f45e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4600159dc0 0x7f45e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45ff7fe640 1 -- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e8004d50 msgr2=0x7f45e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e8004d50 0x7f45e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45ff7fe640 1 -- 172.21.15.133:0/1869812287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e8003530 con 0x7f45e80039f0 2024-04-25T10:10:58.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.034+0000 7f45fffff640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f45e8004d50 0x7f45e80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:58.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e80039f0 0x7f45e8003e10 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f45f000ba30 tx=0x7f45f001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5871e0d6d6fa6478 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f00b2070 con 0x7f45e80039f0 2024-04-25T10:10:58.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f00ad410 con 0x7f45e80039f0 2024-04-25T10:10:58.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f45fe7fc640 1 -- 172.21.15.133:0/1869812287 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000b600 con 0x7f45e80039f0 2024-04-25T10:10:58.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 -- 172.21.15.133:0/1869812287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e80039f0 msgr2=0x7f45e8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 --2- 172.21.15.133:0/1869812287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45e80039f0 0x7f45e8003e10 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f45f000ba30 tx=0x7f45f001c6a0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f45fffff640 1 -- 172.21.15.133:0/1869812287 reap_dead start 2024-04-25T10:10:58.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 -- 172.21.15.133:0/1869812287 shutdown_connections 2024-04-25T10:10:58.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 -- 172.21.15.133:0/1869812287 >> 172.21.15.133:0/1869812287 conn(0x7f4600067000 msgr2=0x7f460006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 -- 172.21.15.133:0/1869812287 shutdown_connections 2024-04-25T10:10:58.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 -- 172.21.15.133:0/1869812287 wait complete. 2024-04-25T10:10:58.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 Processor -- start 2024-04-25T10:10:58.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 -- start start 2024-04-25T10:10:58.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e80039f0 0x7f46001579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46000b2560 0x7f46001486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46000ca610 0x7f46000caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460006dbe0 con 0x7f46000ca610 2024-04-25T10:10:58.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460006da60 con 0x7f46000b2560 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f4605ceb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460006dd60 con 0x7f45e80039f0 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f45fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46000ca610 0x7f46000caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f45fffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46000ca610 0x7f46000caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45698/0 (socket says 172.21.15.133:45698) 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.038+0000 7f45fffff640 1 -- 172.21.15.133:0/62781701 learned_addr learned my addr 172.21.15.133:0/62781701 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45feffd640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46000b2560 0x7f46001486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e80039f0 0x7f46001579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45fffff640 1 -- 172.21.15.133:0/62781701 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e80039f0 msgr2=0x7f46001579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45fffff640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e80039f0 0x7f46001579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45fffff640 1 -- 172.21.15.133:0/62781701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46000b2560 msgr2=0x7f46001486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45fffff640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46000b2560 0x7f46001486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45fffff640 1 -- 172.21.15.133:0/62781701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46000b3030 con 0x7f46000ca610 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45feffd640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46000b2560 0x7f46001486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45fffff640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46000ca610 0x7f46000caa90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f460006f530 tx=0x7f45f800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45e7fff640 1 -- 172.21.15.133:0/62781701 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800bb20 con 0x7f46000ca610 2024-04-25T10:10:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45e7fff640 1 -- 172.21.15.133:0/62781701 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f800bcc0 con 0x7f46000ca610 2024-04-25T10:10:58.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45e7fff640 1 -- 172.21.15.133:0/62781701 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80072b0 con 0x7f46000ca610 2024-04-25T10:10:58.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f4605ceb640 1 -- 172.21.15.133:0/62781701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46000b3310 con 0x7f46000ca610 2024-04-25T10:10:58.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f4605ceb640 1 -- 172.21.15.133:0/62781701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46000b38d0 con 0x7f46000ca610 2024-04-25T10:10:58.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f4605ceb640 1 -- 172.21.15.133:0/62781701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4600068ef0 con 0x7f46000ca610 2024-04-25T10:10:58.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45e7fff640 1 -- 172.21.15.133:0/62781701 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45f801b020 con 0x7f46000ca610 2024-04-25T10:10:58.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45e7fff640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f90 0x7f45ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45e7fff640 1 -- 172.21.15.133:0/62781701 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f45f8085bd0 con 0x7f46000ca610 2024-04-25T10:10:58.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f90 0x7f45ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.042+0000 7f45ff7fe640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f90 0x7f45ec069450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f45f000ba00 tx=0x7f45f00af040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.046+0000 7f45e7fff640 1 -- 172.21.15.133:0/62781701 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45f8022070 con 0x7f46000ca610 2024-04-25T10:10:58.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.226+0000 7f4605ceb640 1 -- 172.21.15.133:0/62781701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f460015c920 con 0x7f46000ca610 2024-04-25T10:10:58.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.226+0000 7f45e7fff640 1 -- 172.21.15.133:0/62781701 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f45f80569c0 con 0x7f46000ca610 2024-04-25T10:10:58.229 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:58.229 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3089710291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3089710291},{"type":"v1","addr":"172.21.15.162:6834","nonce":3089710291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1390003376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1390003376},{"type":"v1","addr":"172.21.15.162:6835","nonce":1390003376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7866,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1134679682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1134679682},{"type":"v1","addr":"172.21.15.133:6836","nonce":1134679682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:27.661584+0000","modified":"2024-04-25T10:10:30.680424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3580592279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3580592279},{"type":"v1","addr":"172.21.15.133:6837","nonce":3580592279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:58.230 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 58 2024-04-25T10:10:58.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 -- 172.21.15.133:0/62781701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f90 msgr2=0x7f45ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f90 0x7f45ec069450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f45f000ba00 tx=0x7f45f00af040 comp rx=0 tx=0).stop 2024-04-25T10:10:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 -- 172.21.15.133:0/62781701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46000ca610 msgr2=0x7f46000caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46000ca610 0x7f46000caa90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f460006f530 tx=0x7f45f800b4a0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 -- 172.21.15.133:0/62781701 shutdown_connections 2024-04-25T10:10:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f45ec066f90 0x7f45ec069450 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46000ca610 0x7f46000caa90 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46000b2560 0x7f46001486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 --2- 172.21.15.133:0/62781701 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f45e80039f0 0x7f46001579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 -- 172.21.15.133:0/62781701 >> 172.21.15.133:0/62781701 conn(0x7f4600067000 msgr2=0x7f460015cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 -- 172.21.15.133:0/62781701 shutdown_connections 2024-04-25T10:10:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.230+0000 7f4605ceb640 1 -- 172.21.15.133:0/62781701 wait complete. 2024-04-25T10:10:58.247 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.362+0000 7f98a2f4d640 1 Processor -- start 2024-04-25T10:10:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.362+0000 7f98a2f4d640 1 -- start start 2024-04-25T10:10:58.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a2f4d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f989c15c9a0 0x7f989c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a2f4d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f989c0b1f70 0x7f989c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a2f4d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f989c0ce330 0x7f989c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a2f4d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989c0cece0 con 0x7f989c0ce330 2024-04-25T10:10:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a2f4d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989c0c61d0 con 0x7f989c15c9a0 2024-04-25T10:10:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a2f4d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0c64b0 con 0x7f989c0b1f70 2024-04-25T10:10:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a14ea640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f989c0ce330 0x7f989c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a14ea640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f989c0ce330 0x7f989c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45708/0 (socket says 172.21.15.133:45708) 2024-04-25T10:10:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a14ea640 1 -- 172.21.15.133:0/3097411515 learned_addr learned my addr 172.21.15.133:0/3097411515 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a0ce9640 1 --2- 172.21.15.133:0/3097411515 >> v2:172.21.15.162:3300/0 conn(0x7f989c15c9a0 0x7f989c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f989bfff640 1 --2- 172.21.15.133:0/3097411515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f989c0b1f70 0x7f989c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a14ea640 1 -- 172.21.15.133:0/3097411515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f989c0b1f70 msgr2=0x7f989c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a14ea640 1 --2- 172.21.15.133:0/3097411515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f989c0b1f70 0x7f989c06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a14ea640 1 -- 172.21.15.133:0/3097411515 >> v2:172.21.15.162:3300/0 conn(0x7f989c15c9a0 msgr2=0x7f989c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a14ea640 1 --2- 172.21.15.133:0/3097411515 >> v2:172.21.15.162:3300/0 conn(0x7f989c15c9a0 0x7f989c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a14ea640 1 -- 172.21.15.133:0/3097411515 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c06d4f0 con 0x7f989c0ce330 2024-04-25T10:10:58.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a0ce9640 1 --2- 172.21.15.133:0/3097411515 >> v2:172.21.15.162:3300/0 conn(0x7f989c15c9a0 0x7f989c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:58.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f98a14ea640 1 --2- 172.21.15.133:0/3097411515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f989c0ce330 0x7f989c0ce7a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f989c06f5f0 tx=0x7f989401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a76323f8207454 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f989b7fe640 1 -- 172.21.15.133:0/3097411515 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940a5750 con 0x7f989c0ce330 2024-04-25T10:10:58.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f989b7fe640 1 -- 172.21.15.133:0/3097411515 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98940a58f0 con 0x7f989c0ce330 2024-04-25T10:10:58.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.366+0000 7f989b7fe640 1 -- 172.21.15.133:0/3097411515 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940a5cc0 con 0x7f989c0ce330 2024-04-25T10:10:58.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 -- 172.21.15.133:0/3097411515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f989c0ce330 msgr2=0x7f989c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 --2- 172.21.15.133:0/3097411515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f989c0ce330 0x7f989c0ce7a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f989c06f5f0 tx=0x7f989401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 -- 172.21.15.133:0/3097411515 shutdown_connections 2024-04-25T10:10:58.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 --2- 172.21.15.133:0/3097411515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f989c0ce330 0x7f989c0ce7a0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 --2- 172.21.15.133:0/3097411515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f989c0b1f70 0x7f989c06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 --2- 172.21.15.133:0/3097411515 >> v2:172.21.15.162:3300/0 conn(0x7f989c15c9a0 0x7f989c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 -- 172.21.15.133:0/3097411515 >> 172.21.15.133:0/3097411515 conn(0x7f989c067000 msgr2=0x7f989c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 -- 172.21.15.133:0/3097411515 shutdown_connections 2024-04-25T10:10:58.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 -- 172.21.15.133:0/3097411515 wait complete. 2024-04-25T10:10:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 Processor -- start 2024-04-25T10:10:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 -- start start 2024-04-25T10:10:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989c0b1f70 0x7f989c1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f989c0ce330 0x7f989c158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f989c15c9a0 0x7f989c13ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c15cf00 con 0x7f989c0ce330 2024-04-25T10:10:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c15cd80 con 0x7f989c0b1f70 2024-04-25T10:10:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c15d950 con 0x7f989c15c9a0 2024-04-25T10:10:58.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f989bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f989c0ce330 0x7f989c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f989bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f989c0ce330 0x7f989c158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45714/0 (socket says 172.21.15.133:45714) 2024-04-25T10:10:58.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f989bfff640 1 -- 172.21.15.133:0/1540472521 learned_addr learned my addr 172.21.15.133:0/1540472521 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:58.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a14ea640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f989c15c9a0 0x7f989c13ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a0ce9640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989c0b1f70 0x7f989c1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f989bfff640 1 -- 172.21.15.133:0/1540472521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f989c15c9a0 msgr2=0x7f989c13ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f989bfff640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f989c15c9a0 0x7f989c13ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f989bfff640 1 -- 172.21.15.133:0/1540472521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989c0b1f70 msgr2=0x7f989c1571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f989bfff640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989c0b1f70 0x7f989c1571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f989bfff640 1 -- 172.21.15.133:0/1540472521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c154f50 con 0x7f989c0ce330 2024-04-25T10:10:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a0ce9640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989c0b1f70 0x7f989c1571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a14ea640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f989c15c9a0 0x7f989c13ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:58.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f989bfff640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f989c0ce330 0x7f989c158500 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f989000c7f0 tx=0x7f989000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98997fa640 1 -- 172.21.15.133:0/1540472521 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890015070 con 0x7f989c0ce330 2024-04-25T10:10:58.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98997fa640 1 -- 172.21.15.133:0/1540472521 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9890004220 con 0x7f989c0ce330 2024-04-25T10:10:58.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 -- 172.21.15.133:0/1540472521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989c155230 con 0x7f989c0ce330 2024-04-25T10:10:58.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98997fa640 1 -- 172.21.15.133:0/1540472521 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890003620 con 0x7f989c0ce330 2024-04-25T10:10:58.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.370+0000 7f98a2f4d640 1 -- 172.21.15.133:0/1540472521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c1557f0 con 0x7f989c0ce330 2024-04-25T10:10:58.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.374+0000 7f98a2f4d640 1 -- 172.21.15.133:0/1540472521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989c06bc20 con 0x7f989c0ce330 2024-04-25T10:10:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.374+0000 7f98997fa640 1 -- 172.21.15.133:0/1540472521 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98900039d0 con 0x7f989c0ce330 2024-04-25T10:10:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.374+0000 7f98997fa640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9888066f60 0x7f9888069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.374+0000 7f98997fa640 1 -- 172.21.15.133:0/1540472521 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f98900856b0 con 0x7f989c0ce330 2024-04-25T10:10:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.374+0000 7f98a0ce9640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9888066f60 0x7f9888069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.374+0000 7f98a0ce9640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9888066f60 0x7f9888069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f988c008890 tx=0x7f988c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.378+0000 7f98997fa640 1 -- 172.21.15.133:0/1540472521 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98900565a0 con 0x7f989c0ce330 2024-04-25T10:10:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.526+0000 7f98a2f4d640 1 -- 172.21.15.133:0/1540472521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f989c068ef0 con 0x7f989c0ce330 2024-04-25T10:10:58.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.526+0000 7f98997fa640 1 -- 172.21.15.133:0/1540472521 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17103 (secure 0 0 0) 0x7f9890055ea0 con 0x7f989c0ce330 2024-04-25T10:10:58.529 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:58.529 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":96,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:10:33.344314+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:25.920690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:26.640190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.526+0000 7f98a2f4d640 1 -- 172.21.15.133:0/1540472521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9888066f60 msgr2=0x7f9888069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.526+0000 7f98a2f4d640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9888066f60 0x7f9888069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f988c008890 tx=0x7f988c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.526+0000 7f98a2f4d640 1 -- 172.21.15.133:0/1540472521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f989c0ce330 msgr2=0x7f989c158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.526+0000 7f98a2f4d640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f989c0ce330 0x7f989c158500 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f989000c7f0 tx=0x7f989000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.530+0000 7f98a2f4d640 1 -- 172.21.15.133:0/1540472521 shutdown_connections 2024-04-25T10:10:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.530+0000 7f98a2f4d640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f989c15c9a0 0x7f989c13ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.530+0000 7f98a2f4d640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9888066f60 0x7f9888069420 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.530+0000 7f98a2f4d640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f989c0ce330 0x7f989c158500 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.530+0000 7f98a2f4d640 1 --2- 172.21.15.133:0/1540472521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f989c0b1f70 0x7f989c1571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.530+0000 7f98a2f4d640 1 -- 172.21.15.133:0/1540472521 >> 172.21.15.133:0/1540472521 conn(0x7f989c067000 msgr2=0x7f989c15e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.530+0000 7f98a2f4d640 1 -- 172.21.15.133:0/1540472521 shutdown_connections 2024-04-25T10:10:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.530+0000 7f98a2f4d640 1 -- 172.21.15.133:0/1540472521 wait complete. 2024-04-25T10:10:58.548 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:10:58.548 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:58.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f217c899640 1 Processor -- start 2024-04-25T10:10:58.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f217c899640 1 -- start start 2024-04-25T10:10:58.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f217c899640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f21780ce330 0x7f21780ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f217c899640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21780c61d0 con 0x7f217815c9a0 2024-04-25T10:10:58.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f217c899640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21780c64b0 con 0x7f21780b1f70 2024-04-25T10:10:58.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f217c899640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21780cecc0 con 0x7f21780ce330 2024-04-25T10:10:58.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f217659c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f217815c9a0 0x7f21780c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:39980/0 (socket says 172.21.15.133:39980) 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f217659c640 1 -- 172.21.15.133:0/478460017 learned_addr learned my addr 172.21.15.133:0/478460017 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115704697 0 0) 0x7f21780c61d0 con 0x7f217815c9a0 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2158003530 con 0x7f217815c9a0 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2176d9d640 1 --2- 172.21.15.133:0/478460017 >> v2:172.21.15.162:3300/0 conn(0x7f21780ce330 0x7f21780ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1767130481 0 0) 0x7f2158003530 con 0x7f217815c9a0 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 >> v2:172.21.15.162:3300/0 conn(0x7f21780ce330 msgr2=0x7f21780ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 --2- 172.21.15.133:0/478460017 >> v2:172.21.15.162:3300/0 conn(0x7f21780ce330 0x7f21780ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 >> v1:172.21.15.162:6789/0 conn(0x7f21780b1f70 legacy=0x7f217806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217806d4f0 con 0x7f217815c9a0 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2160001680 con 0x7f217815c9a0 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 >> v1:172.21.15.133:6789/0 conn(0x7f217815c9a0 legacy=0x7f21780c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 --2- 172.21.15.133:0/478460017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21580039c0 0x7f2158003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 --2- 172.21.15.133:0/478460017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2158004810 0x7f2158004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 --2- 172.21.15.133:0/478460017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21580051d0 0x7f215800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21780cecc0 con 0x7f21580051d0 2024-04-25T10:10:58.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215800c120 con 0x7f21580039c0 2024-04-25T10:10:58.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215800c320 con 0x7f2158004810 2024-04-25T10:10:58.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2176d9d640 1 --2- 172.21.15.133:0/478460017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21580051d0 0x7f215800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f217659c640 1 --2- 172.21.15.133:0/478460017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21580039c0 0x7f2158003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f216dd9b640 1 --2- 172.21.15.133:0/478460017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2158004810 0x7f2158004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2176d9d640 1 -- 172.21.15.133:0/478460017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2158004810 msgr2=0x7f2158004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2176d9d640 1 --2- 172.21.15.133:0/478460017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2158004810 0x7f2158004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2176d9d640 1 -- 172.21.15.133:0/478460017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21580039c0 msgr2=0x7f2158003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2176d9d640 1 --2- 172.21.15.133:0/478460017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21580039c0 0x7f2158003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.654+0000 7f2176d9d640 1 -- 172.21.15.133:0/478460017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2158003530 con 0x7f21580051d0 2024-04-25T10:10:58.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f2176d9d640 1 -- 172.21.15.133:0/478460017 reap_dead start 2024-04-25T10:10:58.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f2176d9d640 1 --2- 172.21.15.133:0/478460017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21580051d0 0x7f215800b9d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2170007360 tx=0x7f217001ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a8eda8b2739fce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170009be0 con 0x7f21580051d0 2024-04-25T10:10:58.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21700a5dc0 con 0x7f21580051d0 2024-04-25T10:10:58.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f2175d9b640 1 -- 172.21.15.133:0/478460017 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170003be0 con 0x7f21580051d0 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 -- 172.21.15.133:0/478460017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21580051d0 msgr2=0x7f215800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 --2- 172.21.15.133:0/478460017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21580051d0 0x7f215800b9d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2170007360 tx=0x7f217001ce80 comp rx=0 tx=0).stop 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 -- 172.21.15.133:0/478460017 shutdown_connections 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 --2- 172.21.15.133:0/478460017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21580051d0 0x7f215800b9d0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 -- 172.21.15.133:0/478460017 >> 172.21.15.133:0/478460017 conn(0x7f2178067000 msgr2=0x7f217806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 -- 172.21.15.133:0/478460017 shutdown_connections 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 -- 172.21.15.133:0/478460017 wait complete. 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 Processor -- start 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 -- start start 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21580051d0 0x7f21780cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2178158570 0x7f21781f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21781e5100 0x7f21781e5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217815cf00 con 0x7f2178158570 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217815cd80 con 0x7f21781e5100 2024-04-25T10:10:58.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21780ce780 con 0x7f21580051d0 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2178158570 0x7f21781f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2178158570 0x7f21781f2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45734/0 (socket says 172.21.15.133:45734) 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216dd9b640 1 -- 172.21.15.133:0/1603467748 learned_addr learned my addr 172.21.15.133:0/1603467748 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f2176d9d640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21781e5100 0x7f21781e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217659c640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21580051d0 0x7f21780cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216dd9b640 1 -- 172.21.15.133:0/1603467748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21580051d0 msgr2=0x7f21780cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216dd9b640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21580051d0 0x7f21780cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216dd9b640 1 -- 172.21.15.133:0/1603467748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21781e5100 msgr2=0x7f21781e5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216dd9b640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21781e5100 0x7f21781e5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216dd9b640 1 -- 172.21.15.133:0/1603467748 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21780b11d0 con 0x7f2178158570 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216dd9b640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2178158570 0x7f21781f2cb0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f216800e4a0 tx=0x7f216800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216f7fe640 1 -- 172.21.15.133:0/1603467748 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21680052e0 con 0x7f2178158570 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216f7fe640 1 -- 172.21.15.133:0/1603467748 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2168018070 con 0x7f2178158570 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 -- 172.21.15.133:0/1603467748 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21781f6a00 con 0x7f2178158570 2024-04-25T10:10:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f216f7fe640 1 -- 172.21.15.133:0/1603467748 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168003820 con 0x7f2178158570 2024-04-25T10:10:58.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.658+0000 7f217c899640 1 -- 172.21.15.133:0/1603467748 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21781f6fc0 con 0x7f2178158570 2024-04-25T10:10:58.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.662+0000 7f217c899640 1 -- 172.21.15.133:0/1603467748 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2178068ef0 con 0x7f2178158570 2024-04-25T10:10:58.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.662+0000 7f216f7fe640 1 -- 172.21.15.133:0/1603467748 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21680039c0 con 0x7f2178158570 2024-04-25T10:10:58.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.662+0000 7f216f7fe640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f215c066f90 0x7f215c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.662+0000 7f216f7fe640 1 -- 172.21.15.133:0/1603467748 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f2168084af0 con 0x7f2178158570 2024-04-25T10:10:58.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.662+0000 7f217659c640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f215c066f90 0x7f215c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.662+0000 7f217659c640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f215c066f90 0x7f215c069450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2160001ec0 tx=0x7f2160000f20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.666+0000 7f216f7fe640 1 -- 172.21.15.133:0/1603467748 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2168005480 con 0x7f2178158570 2024-04-25T10:10:58.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.842+0000 7f217c899640 1 -- 172.21.15.133:0/1603467748 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f217815b290 con 0x7f2178158570 2024-04-25T10:10:58.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f216f7fe640 1 -- 172.21.15.133:0/1603467748 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f21680558f0 con 0x7f2178158570 2024-04-25T10:10:58.847 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:58.847 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3089710291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3089710291},{"type":"v1","addr":"172.21.15.162:6834","nonce":3089710291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1390003376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1390003376},{"type":"v1","addr":"172.21.15.162:6835","nonce":1390003376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7866,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1134679682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1134679682},{"type":"v1","addr":"172.21.15.133:6836","nonce":1134679682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:27.661584+0000","modified":"2024-04-25T10:10:30.680424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3580592279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3580592279},{"type":"v1","addr":"172.21.15.133:6837","nonce":3580592279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:58.848 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 58 2024-04-25T10:10:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 -- 172.21.15.133:0/1603467748 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f215c066f90 msgr2=0x7f215c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f215c066f90 0x7f215c069450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2160001ec0 tx=0x7f2160000f20 comp rx=0 tx=0).stop 2024-04-25T10:10:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 -- 172.21.15.133:0/1603467748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2178158570 msgr2=0x7f21781f2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2178158570 0x7f21781f2cb0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f216800e4a0 tx=0x7f216800e960 comp rx=0 tx=0).stop 2024-04-25T10:10:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 -- 172.21.15.133:0/1603467748 shutdown_connections 2024-04-25T10:10:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21781e5100 0x7f21781e5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f215c066f90 0x7f215c069450 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2178158570 0x7f21781f2cb0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 --2- 172.21.15.133:0/1603467748 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21580051d0 0x7f21780cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 -- 172.21.15.133:0/1603467748 >> 172.21.15.133:0/1603467748 conn(0x7f2178067000 msgr2=0x7f217814f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 -- 172.21.15.133:0/1603467748 shutdown_connections 2024-04-25T10:10:58.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.846+0000 7f217c899640 1 -- 172.21.15.133:0/1603467748 wait complete. 2024-04-25T10:10:58.865 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:58.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a8242e640 1 Processor -- start 2024-04-25T10:10:58.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a8242e640 1 -- start start 2024-04-25T10:10:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a8242e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c06c130 0x7f9a7c06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a8242e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7c06c5c0 con 0x7f9a7c06bca0 2024-04-25T10:10:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a8242e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7c0acd10 con 0x7f9a7c15a730 2024-04-25T10:10:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a8242e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0acff0 con 0x7f9a7c06c130 2024-04-25T10:10:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7b7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9a7c06bca0 0x7f9a7c06f140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:39992/0 (socket says 172.21.15.133:39992) 2024-04-25T10:10:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7b7fe640 1 -- 172.21.15.133:0/3348315826 learned_addr learned my addr 172.21.15.133:0/3348315826 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a809cb640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c06c130 0x7f9a7c06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940096633 0 0) 0x7f9a7c06c5c0 con 0x7f9a7c06bca0 2024-04-25T10:10:58.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a64003530 con 0x7f9a7c06bca0 2024-04-25T10:10:58.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 997268592 0 0) 0x7f9a64003530 con 0x7f9a7c06bca0 2024-04-25T10:10:58.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c06c130 msgr2=0x7f9a7c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c06c130 0x7f9a7c06f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 >> v1:172.21.15.162:6789/0 conn(0x7f9a7c15a730 legacy=0x7f9a7c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c0ad2d0 con 0x7f9a7c06bca0 2024-04-25T10:10:58.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f9a70001680 con 0x7f9a7c06bca0 2024-04-25T10:10:58.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 >> v1:172.21.15.133:6789/0 conn(0x7f9a7c06bca0 legacy=0x7f9a7c06f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a640039f0 0x7f9a64003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 0x7f9a64004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c06c130 0x7f9a640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0acff0 con 0x7f9a64004810 2024-04-25T10:10:58.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6400ba80 con 0x7f9a640039f0 2024-04-25T10:10:58.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6400de90 con 0x7f9a7c06c130 2024-04-25T10:10:58.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7b7fe640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 0x7f9a64004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a809cb640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c06c130 0x7f9a640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7bfff640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a640039f0 0x7f9a64003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7b7fe640 1 -- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c06c130 msgr2=0x7f9a640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7b7fe640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c06c130 0x7f9a640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7b7fe640 1 -- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a640039f0 msgr2=0x7f9a64003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7b7fe640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a640039f0 0x7f9a64003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a7b7fe640 1 -- 172.21.15.133:0/3348315826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a64003530 con 0x7f9a64004810 2024-04-25T10:10:58.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.982+0000 7f9a809cb640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c06c130 0x7f9a640051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a7b7fe640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 0x7f9a64004c90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c210 tx=0x7f9a7001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391e7d701d0191e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70007d20 con 0x7f9a64004810 2024-04-25T10:10:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a70005ae0 con 0x7f9a64004810 2024-04-25T10:10:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a7affd640 1 -- 172.21.15.133:0/3348315826 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70007070 con 0x7f9a64004810 2024-04-25T10:10:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a8242e640 1 -- 172.21.15.133:0/3348315826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 msgr2=0x7f9a64004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a8242e640 1 --2- 172.21.15.133:0/3348315826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 0x7f9a64004c90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c210 tx=0x7f9a7001f390 comp rx=0 tx=0).stop 2024-04-25T10:10:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a809cb640 1 -- 172.21.15.133:0/3348315826 reap_dead start 2024-04-25T10:10:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a8242e640 1 -- 172.21.15.133:0/3348315826 shutdown_connections 2024-04-25T10:10:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a8242e640 1 -- 172.21.15.133:0/3348315826 >> 172.21.15.133:0/3348315826 conn(0x7f9a7c071aa0 msgr2=0x7f9a7c071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a8242e640 1 -- 172.21.15.133:0/3348315826 shutdown_connections 2024-04-25T10:10:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a8242e640 1 -- 172.21.15.133:0/3348315826 wait complete. 2024-04-25T10:10:58.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a8242e640 1 Processor -- start 2024-04-25T10:10:58.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.986+0000 7f9a8242e640 1 -- start start 2024-04-25T10:10:58.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a8242e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 0x7f9a7c145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a8242e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a7c153bd0 0x7f9a7c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a8242e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c156df0 0x7f9a7c157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a8242e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0adb70 con 0x7f9a64004810 2024-04-25T10:10:58.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a8242e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0ad890 con 0x7f9a7c153bd0 2024-04-25T10:10:58.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a8242e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0cc6b0 con 0x7f9a7c156df0 2024-04-25T10:10:58.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 0x7f9a7c145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 0x7f9a7c145730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45750/0 (socket says 172.21.15.133:45750) 2024-04-25T10:10:58.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7bfff640 1 -- 172.21.15.133:0/2681440109 learned_addr learned my addr 172.21.15.133:0/2681440109 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:58.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7b7fe640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a7c153bd0 0x7f9a7c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a809cb640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c156df0 0x7f9a7c157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7bfff640 1 -- 172.21.15.133:0/2681440109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c156df0 msgr2=0x7f9a7c157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7bfff640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c156df0 0x7f9a7c157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7bfff640 1 -- 172.21.15.133:0/2681440109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a7c153bd0 msgr2=0x7f9a7c154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7bfff640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a7c153bd0 0x7f9a7c154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7bfff640 1 -- 172.21.15.133:0/2681440109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c157a00 con 0x7f9a64004810 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7bfff640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 0x7f9a7c145730 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00d540 tx=0x7f9a6c00da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a78ff9640 1 -- 172.21.15.133:0/2681440109 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c005480 con 0x7f9a64004810 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a78ff9640 1 -- 172.21.15.133:0/2681440109 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c017070 con 0x7f9a64004810 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a8242e640 1 -- 172.21.15.133:0/2681440109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7c157ce0 con 0x7f9a64004810 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a78ff9640 1 -- 172.21.15.133:0/2681440109 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c003820 con 0x7f9a64004810 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a8242e640 1 -- 172.21.15.133:0/2681440109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c1582a0 con 0x7f9a64004810 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a8242e640 1 -- 172.21.15.133:0/2681440109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7c067c70 con 0x7f9a64004810 2024-04-25T10:10:58.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a78ff9640 1 -- 172.21.15.133:0/2681440109 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a6c0039c0 con 0x7f9a64004810 2024-04-25T10:10:59.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a78ff9640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a68066f90 0x7f9a68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a78ff9640 1 -- 172.21.15.133:0/2681440109 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f9a6c084f30 con 0x7f9a64004810 2024-04-25T10:10:59.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7b7fe640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a68066f90 0x7f9a68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.990+0000 7f9a7b7fe640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a68066f90 0x7f9a68069450 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9a700af000 tx=0x7f9a700aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:58.994+0000 7f9a78ff9640 1 -- 172.21.15.133:0/2681440109 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a6c019310 con 0x7f9a64004810 2024-04-25T10:10:59.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.186+0000 7f9a8242e640 1 -- 172.21.15.133:0/2681440109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a7c0cbf20 con 0x7f9a64004810 2024-04-25T10:10:59.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a78ff9640 1 -- 172.21.15.133:0/2681440109 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f9a6c055dd0 con 0x7f9a64004810 2024-04-25T10:10:59.191 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:59.191 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3089710291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3089710291},{"type":"v1","addr":"172.21.15.162:6834","nonce":3089710291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1390003376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1390003376},{"type":"v1","addr":"172.21.15.162:6835","nonce":1390003376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7866,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1134679682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1134679682},{"type":"v1","addr":"172.21.15.133:6836","nonce":1134679682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:27.661584+0000","modified":"2024-04-25T10:10:30.680424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7869},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7869":{"gid":7869,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3580592279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3580592279},{"type":"v1","addr":"172.21.15.133:6837","nonce":3580592279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:59.192 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 58 2024-04-25T10:10:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 -- 172.21.15.133:0/2681440109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a68066f90 msgr2=0x7f9a68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a68066f90 0x7f9a68069450 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9a700af000 tx=0x7f9a700aa000 comp rx=0 tx=0).stop 2024-04-25T10:10:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 -- 172.21.15.133:0/2681440109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 msgr2=0x7f9a7c145730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 0x7f9a7c145730 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00d540 tx=0x7f9a6c00da00 comp rx=0 tx=0).stop 2024-04-25T10:10:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 -- 172.21.15.133:0/2681440109 shutdown_connections 2024-04-25T10:10:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a7c156df0 0x7f9a7c157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a7c153bd0 0x7f9a7c154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a68066f90 0x7f9a68069450 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 --2- 172.21.15.133:0/2681440109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a64004810 0x7f9a7c145730 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 -- 172.21.15.133:0/2681440109 >> 172.21.15.133:0/2681440109 conn(0x7f9a7c071aa0 msgr2=0x7f9a7c0cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 -- 172.21.15.133:0/2681440109 shutdown_connections 2024-04-25T10:10:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.190+0000 7f9a8242e640 1 -- 172.21.15.133:0/2681440109 wait complete. 2024-04-25T10:10:59.211 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:59.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.330+0000 7fe83dec0640 1 Processor -- start 2024-04-25T10:10:59.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.330+0000 7fe83dec0640 1 -- start start 2024-04-25T10:10:59.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe83dec0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe83815c9a0 0x7fe8380c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe83dec0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8380ce330 0x7fe8380ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe83dec0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8380c61d0 con 0x7fe8380b1f70 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe83dec0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8380cece0 con 0x7fe8380ce330 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe83dec0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8380c64b0 con 0x7fe83815c9a0 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe837fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8380ce330 0x7fe8380ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe837fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8380ce330 0x7fe8380ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45762/0 (socket says 172.21.15.133:45762) 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe837fff640 1 -- 172.21.15.133:0/3451701363 learned_addr learned my addr 172.21.15.133:0/3451701363 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe82effd640 1 --1- 172.21.15.133:0/3451701363 >> v1:172.21.15.133:6789/0 conn(0x7fe8380b1f70 0x7fe83806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:39996/0 (socket says 172.21.15.133:39996) 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe837fff640 1 -- 172.21.15.133:0/3451701363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe83815c9a0 msgr2=0x7fe8380c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe8377fe640 1 --2- 172.21.15.133:0/3451701363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe83815c9a0 0x7fe8380c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe837fff640 1 --2- 172.21.15.133:0/3451701363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe83815c9a0 0x7fe8380c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe837fff640 1 -- 172.21.15.133:0/3451701363 >> v1:172.21.15.133:6789/0 conn(0x7fe8380b1f70 legacy=0x7fe83806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe837fff640 1 -- 172.21.15.133:0/3451701363 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83806d4f0 con 0x7fe8380ce330 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe836ffd640 1 -- 172.21.15.133:0/3451701363 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135702917 0 0) 0x7fe8380c61d0 con 0x7fe8380b1f70 2024-04-25T10:10:59.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe837fff640 1 --2- 172.21.15.133:0/3451701363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8380ce330 0x7fe8380ce7a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe830009e50 tx=0x7fe83001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11bdcc4d40910bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe836ffd640 1 -- 172.21.15.133:0/3451701363 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300a57b0 con 0x7fe8380ce330 2024-04-25T10:10:59.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe836ffd640 1 -- 172.21.15.133:0/3451701363 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8300a5950 con 0x7fe8380ce330 2024-04-25T10:10:59.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.334+0000 7fe836ffd640 1 -- 172.21.15.133:0/3451701363 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300a5d20 con 0x7fe8380ce330 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- 172.21.15.133:0/3451701363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8380ce330 msgr2=0x7fe8380ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 --2- 172.21.15.133:0/3451701363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8380ce330 0x7fe8380ce7a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe830009e50 tx=0x7fe83001ca30 comp rx=0 tx=0).stop 2024-04-25T10:10:59.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- 172.21.15.133:0/3451701363 shutdown_connections 2024-04-25T10:10:59.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 --2- 172.21.15.133:0/3451701363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8380ce330 0x7fe8380ce7a0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 --2- 172.21.15.133:0/3451701363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe83815c9a0 0x7fe8380c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- 172.21.15.133:0/3451701363 >> 172.21.15.133:0/3451701363 conn(0x7fe838067000 msgr2=0x7fe83806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:59.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- 172.21.15.133:0/3451701363 shutdown_connections 2024-04-25T10:10:59.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- 172.21.15.133:0/3451701363 wait complete. 2024-04-25T10:10:59.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 Processor -- start 2024-04-25T10:10:59.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- start start 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8380b1f70 0x7fe83814be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8380ce330 0x7fe838146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe83815c9a0 0x7fe838147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83815dad0 con 0x7fe8380b1f70 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83815d950 con 0x7fe8380ce330 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83815dc50 con 0x7fe83815c9a0 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe8377fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8380b1f70 0x7fe83814be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe8377fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8380b1f70 0x7fe83814be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45768/0 (socket says 172.21.15.133:45768) 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe8377fe640 1 -- 172.21.15.133:0/4164029596 learned_addr learned my addr 172.21.15.133:0/4164029596 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe837fff640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe83815c9a0 0x7fe838147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe82effd640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8380ce330 0x7fe838146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe8377fe640 1 -- 172.21.15.133:0/4164029596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe83815c9a0 msgr2=0x7fe838147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe8377fe640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe83815c9a0 0x7fe838147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe8377fe640 1 -- 172.21.15.133:0/4164029596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8380ce330 msgr2=0x7fe838146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe8377fe640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8380ce330 0x7fe838146640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe8377fe640 1 -- 172.21.15.133:0/4164029596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8380b2960 con 0x7fe8380b1f70 2024-04-25T10:10:59.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe82effd640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8380ce330 0x7fe838146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:59.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe8377fe640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8380b1f70 0x7fe83814be00 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe82800c7f0 tx=0x7fe82800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe834ff9640 1 -- 172.21.15.133:0/4164029596 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828015070 con 0x7fe8380b1f70 2024-04-25T10:10:59.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe834ff9640 1 -- 172.21.15.133:0/4164029596 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe828004240 con 0x7fe8380b1f70 2024-04-25T10:10:59.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- 172.21.15.133:0/4164029596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8380b2c40 con 0x7fe8380b1f70 2024-04-25T10:10:59.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe834ff9640 1 -- 172.21.15.133:0/4164029596 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828003620 con 0x7fe8380b1f70 2024-04-25T10:10:59.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- 172.21.15.133:0/4164029596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8380b3200 con 0x7fe8380b1f70 2024-04-25T10:10:59.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.338+0000 7fe83dec0640 1 -- 172.21.15.133:0/4164029596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83806bc20 con 0x7fe8380b1f70 2024-04-25T10:10:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.342+0000 7fe834ff9640 1 -- 172.21.15.133:0/4164029596 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8280039d0 con 0x7fe8380b1f70 2024-04-25T10:10:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.342+0000 7fe834ff9640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe818066f60 0x7fe818069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.342+0000 7fe82effd640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe818066f60 0x7fe818069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.342+0000 7fe834ff9640 1 -- 172.21.15.133:0/4164029596 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fe828085300 con 0x7fe8380b1f70 2024-04-25T10:10:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.342+0000 7fe82effd640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe818066f60 0x7fe818069420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe820007fd0 tx=0x7fe820009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.346+0000 7fe834ff9640 1 -- 172.21.15.133:0/4164029596 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe828056110 con 0x7fe8380b1f70 2024-04-25T10:10:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.498+0000 7fe83dec0640 1 -- 172.21.15.133:0/4164029596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe838150de0 con 0x7fe8380b1f70 2024-04-25T10:10:59.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe834ff9640 1 -- 172.21.15.133:0/4164029596 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+17103 (secure 0 0 0) 0x7fe828055a10 con 0x7fe8380b1f70 2024-04-25T10:10:59.505 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:10:59.505 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":96,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:10:33.344314+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:25.920690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:26.640190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.17","mappings":[{"from":1,"to":7}]},{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 -- 172.21.15.133:0/4164029596 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe818066f60 msgr2=0x7fe818069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe818066f60 0x7fe818069420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe820007fd0 tx=0x7fe820009040 comp rx=0 tx=0).stop 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 -- 172.21.15.133:0/4164029596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8380b1f70 msgr2=0x7fe83814be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8380b1f70 0x7fe83814be00 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe82800c7f0 tx=0x7fe82800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 -- 172.21.15.133:0/4164029596 shutdown_connections 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe83815c9a0 0x7fe838147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8380ce330 0x7fe838146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe818066f60 0x7fe818069420 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 --2- 172.21.15.133:0/4164029596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8380b1f70 0x7fe83814be00 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 -- 172.21.15.133:0/4164029596 >> 172.21.15.133:0/4164029596 conn(0x7fe838067000 msgr2=0x7fe83815b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 -- 172.21.15.133:0/4164029596 shutdown_connections 2024-04-25T10:10:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.502+0000 7fe83dec0640 1 -- 172.21.15.133:0/4164029596 wait complete. 2024-04-25T10:10:59.523 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:10:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.638+0000 7f4907b1c640 1 Processor -- start 2024-04-25T10:10:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f4907b1c640 1 -- start start 2024-04-25T10:10:59.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f4907b1c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f490006c7b0 0x7f49000b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f4907b1c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f49000cd5c0 0x7f49000cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f4907b1c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000cc250 con 0x7f49000cbdd0 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f4907b1c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49000cd130 con 0x7f49000cd5c0 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f4907b1c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49000cc520 con 0x7f490006c7b0 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f49060b9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f49000cd5c0 0x7f49000cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f49050b7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f49000cbdd0 0x7f490006cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40010/0 (socket says 172.21.15.133:40010) 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f49050b7640 1 -- 172.21.15.133:0/2723198332 learned_addr learned my addr 172.21.15.133:0/2723198332 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f49058b8640 1 --2- 172.21.15.133:0/2723198332 >> v2:172.21.15.162:3300/0 conn(0x7f490006c7b0 0x7f49000b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f49060b9640 1 -- 172.21.15.133:0/2723198332 >> v2:172.21.15.162:3300/0 conn(0x7f490006c7b0 msgr2=0x7f49000b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f49060b9640 1 --2- 172.21.15.133:0/2723198332 >> v2:172.21.15.162:3300/0 conn(0x7f490006c7b0 0x7f49000b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f49048b6640 1 -- 172.21.15.133:0/2723198332 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107041355 0 0) 0x7f49000cc250 con 0x7f49000cbdd0 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f49060b9640 1 -- 172.21.15.133:0/2723198332 >> v1:172.21.15.133:6789/0 conn(0x7f49000cbdd0 legacy=0x7f490006cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f49060b9640 1 -- 172.21.15.133:0/2723198332 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49000cc800 con 0x7f49000cd5c0 2024-04-25T10:10:59.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f49060b9640 1 --2- 172.21.15.133:0/2723198332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49000cd5c0 0x7f49000cfaf0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009d90 tx=0x7f48f801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7340bdc171d29ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.642+0000 7f49048b6640 1 -- 172.21.15.133:0/2723198332 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f80b2070 con 0x7f49000cd5c0 2024-04-25T10:10:59.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49048b6640 1 -- 172.21.15.133:0/2723198332 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f80a58e0 con 0x7f49000cd5c0 2024-04-25T10:10:59.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49048b6640 1 -- 172.21.15.133:0/2723198332 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f80a5cb0 con 0x7f49000cd5c0 2024-04-25T10:10:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 -- 172.21.15.133:0/2723198332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49000cd5c0 msgr2=0x7f49000cfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 --2- 172.21.15.133:0/2723198332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49000cd5c0 0x7f49000cfaf0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009d90 tx=0x7f48f801c930 comp rx=0 tx=0).stop 2024-04-25T10:10:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 -- 172.21.15.133:0/2723198332 shutdown_connections 2024-04-25T10:10:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 --2- 172.21.15.133:0/2723198332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49000cd5c0 0x7f49000cfaf0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 --2- 172.21.15.133:0/2723198332 >> v2:172.21.15.162:3300/0 conn(0x7f490006c7b0 0x7f49000b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 -- 172.21.15.133:0/2723198332 >> 172.21.15.133:0/2723198332 conn(0x7f490009aa10 msgr2=0x7f490008af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 -- 172.21.15.133:0/2723198332 shutdown_connections 2024-04-25T10:10:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 -- 172.21.15.133:0/2723198332 wait complete. 2024-04-25T10:10:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 Processor -- start 2024-04-25T10:10:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 -- start start 2024-04-25T10:10:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f490006c7b0 0x7f49000b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49000cbdd0 0x7f49000b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49000cd5c0 0x7f49000b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49000cdbf0 con 0x7f490006c7b0 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49000cda70 con 0x7f49000cd5c0 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49000d0440 con 0x7f49000cbdd0 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49058b8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f490006c7b0 0x7f49000b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49058b8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f490006c7b0 0x7f49000b6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45782/0 (socket says 172.21.15.133:45782) 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49058b8640 1 -- 172.21.15.133:0/1590837473 learned_addr learned my addr 172.21.15.133:0/1590837473 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49060b9640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49000cd5c0 0x7f49000b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49058b8640 1 -- 172.21.15.133:0/1590837473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49000cbdd0 msgr2=0x7f49000b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49050b7640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49000cbdd0 0x7f49000b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49058b8640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49000cbdd0 0x7f49000b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49058b8640 1 -- 172.21.15.133:0/1590837473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49000cd5c0 msgr2=0x7f49000b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49058b8640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49000cd5c0 0x7f49000b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49058b8640 1 -- 172.21.15.133:0/1590837473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49001592c0 con 0x7f490006c7b0 2024-04-25T10:10:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49060b9640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49000cd5c0 0x7f49000b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f49058b8640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f490006c7b0 0x7f49000b6a60 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00d800 tx=0x7f48fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f48f27fc640 1 -- 172.21.15.133:0/1590837473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00b6c0 con 0x7f490006c7b0 2024-04-25T10:10:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f48f27fc640 1 -- 172.21.15.133:0/1590837473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc019070 con 0x7f490006c7b0 2024-04-25T10:10:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f48f27fc640 1 -- 172.21.15.133:0/1590837473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00bae0 con 0x7f490006c7b0 2024-04-25T10:10:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 -- 172.21.15.133:0/1590837473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4900159560 con 0x7f490006c7b0 2024-04-25T10:10:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.646+0000 7f4907b1c640 1 -- 172.21.15.133:0/1590837473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4900159b20 con 0x7f490006c7b0 2024-04-25T10:10:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.650+0000 7f4907b1c640 1 -- 172.21.15.133:0/1590837473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4900067c70 con 0x7f490006c7b0 2024-04-25T10:10:59.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.650+0000 7f48f27fc640 1 -- 172.21.15.133:0/1590837473 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48fc00bc80 con 0x7f490006c7b0 2024-04-25T10:10:59.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.650+0000 7f48f27fc640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48e4066f60 0x7f48e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.650+0000 7f48f27fc640 1 -- 172.21.15.133:0/1590837473 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f48fc085680 con 0x7f490006c7b0 2024-04-25T10:10:59.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.650+0000 7f49050b7640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48e4066f60 0x7f48e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.650+0000 7f49050b7640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48e4066f60 0x7f48e4069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f490009f650 tx=0x7f48f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.654+0000 7f48f27fc640 1 -- 172.21.15.133:0/1590837473 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48fc056520 con 0x7f490006c7b0 2024-04-25T10:10:59.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.846+0000 7f4907b1c640 1 -- 172.21.15.133:0/1590837473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f490006cb90 con 0x7f490006c7b0 2024-04-25T10:10:59.851 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:10:59.846+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:10:59.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.850+0000 7f48f27fc640 1 -- 172.21.15.133:0/1590837473 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7f48fc055e20 con 0x7f490006c7b0 2024-04-25T10:10:59.853 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:10:59.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.854+0000 7f4907b1c640 1 -- 172.21.15.133:0/1590837473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48e4066f60 msgr2=0x7f48e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.854+0000 7f4907b1c640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48e4066f60 0x7f48e4069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f490009f650 tx=0x7f48f4009040 comp rx=0 tx=0).stop 2024-04-25T10:10:59.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.854+0000 7f4907b1c640 1 -- 172.21.15.133:0/1590837473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f490006c7b0 msgr2=0x7f49000b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.854+0000 7f4907b1c640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f490006c7b0 0x7f49000b6a60 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00d800 tx=0x7f48fc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.854+0000 7f4907b1c640 1 -- 172.21.15.133:0/1590837473 shutdown_connections 2024-04-25T10:10:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.854+0000 7f4907b1c640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49000cd5c0 0x7f49000b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.854+0000 7f4907b1c640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f49000cbdd0 0x7f49000b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.854+0000 7f4907b1c640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48e4066f60 0x7f48e4069420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.854+0000 7f4907b1c640 1 --2- 172.21.15.133:0/1590837473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f490006c7b0 0x7f49000b6a60 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.854+0000 7f4907b1c640 1 -- 172.21.15.133:0/1590837473 >> 172.21.15.133:0/1590837473 conn(0x7f490009aa10 msgr2=0x7f49000ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.854+0000 7f4907b1c640 1 -- 172.21.15.133:0/1590837473 shutdown_connections 2024-04-25T10:10:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.858+0000 7f4907b1c640 1 -- 172.21.15.133:0/1590837473 wait complete. 2024-04-25T10:10:59.881 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:10:59.909 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:11:00.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.998+0000 7f56339ff640 1 Processor -- start 2024-04-25T10:11:00.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.998+0000 7f56339ff640 1 -- start start 2024-04-25T10:11:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.998+0000 7f56339ff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f562c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.998+0000 7f56339ff640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c15ac50 con 0x7f562c06eac0 2024-04-25T10:11:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.998+0000 7f56339ff640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c0cd180 con 0x7f562c0b2150 2024-04-25T10:11:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.998+0000 7f56339ff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15aed0 con 0x7f562c143ea0 2024-04-25T10:11:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.998+0000 7f5631f9c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f562c06eac0 0x7f562c0b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40034/0 (socket says 172.21.15.133:40034) 2024-04-25T10:11:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.998+0000 7f5631f9c640 1 -- 172.21.15.133:0/463546483 learned_addr learned my addr 172.21.15.133:0/463546483 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.998+0000 7f563179b640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f562c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.998+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200868420 0 0) 0x7f56240014a0 con 0x7f562c06eac0 2024-04-25T10:11:00.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:10:59.998+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5618003530 con 0x7f562c06eac0 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1547358860 0 0) 0x7f5618003530 con 0x7f562c06eac0 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 msgr2=0x7f562c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f562c0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 >> v1:172.21.15.162:6789/0 conn(0x7f562c0b2150 legacy=0x7f562c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c15b1b0 con 0x7f562c06eac0 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5624001bd0 con 0x7f562c06eac0 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 >> v1:172.21.15.133:6789/0 conn(0x7f562c06eac0 legacy=0x7f562c0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 0x7f5618003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f5618004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5618004d50 0x7f56180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15aed0 con 0x7f56180039f0 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561800ba80 con 0x7f5618004d50 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561800de90 con 0x7f562c143ea0 2024-04-25T10:11:00.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f563179b640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 0x7f5618003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5630f9a640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f5618004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f563179b640 1 -- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 msgr2=0x7f5618004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f563179b640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f5618004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5631f9c640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5618004d50 0x7f56180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f563179b640 1 -- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5618004d50 msgr2=0x7f56180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f563179b640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5618004d50 0x7f56180051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f563179b640 1 -- 172.21.15.133:0/463546483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5618003530 con 0x7f56180039f0 2024-04-25T10:11:00.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5630f9a640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c143ea0 0x7f5618004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f563179b640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 0x7f5618003e10 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f561c009ab0 tx=0x7f561c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5bf1e0ab5a6abb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0096b0 con 0x7f56180039f0 2024-04-25T10:11:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c0a6480 con 0x7f56180039f0 2024-04-25T10:11:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5623fff640 1 -- 172.21.15.133:0/463546483 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0a6870 con 0x7f56180039f0 2024-04-25T10:11:00.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f56339ff640 1 -- 172.21.15.133:0/463546483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 msgr2=0x7f5618003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f56339ff640 1 --2- 172.21.15.133:0/463546483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 0x7f5618003e10 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f561c009ab0 tx=0x7f561c01ca50 comp rx=0 tx=0).stop 2024-04-25T10:11:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f5631f9c640 1 -- 172.21.15.133:0/463546483 reap_dead start 2024-04-25T10:11:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f56339ff640 1 -- 172.21.15.133:0/463546483 shutdown_connections 2024-04-25T10:11:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f56339ff640 1 -- 172.21.15.133:0/463546483 >> 172.21.15.133:0/463546483 conn(0x7f562c066f70 msgr2=0x7f562c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f56339ff640 1 -- 172.21.15.133:0/463546483 shutdown_connections 2024-04-25T10:11:00.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.002+0000 7f56339ff640 1 -- 172.21.15.133:0/463546483 wait complete. 2024-04-25T10:11:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f56339ff640 1 Processor -- start 2024-04-25T10:11:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f56339ff640 1 -- start start 2024-04-25T10:11:00.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f56339ff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 0x7f562c137890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f56339ff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f562c157b80 0x7f562c1560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f56339ff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c137e10 0x7f562c156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f56339ff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0b1170 con 0x7f56180039f0 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f56339ff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0b0ff0 con 0x7f562c157b80 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f56339ff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0b12f0 con 0x7f562c137e10 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f563179b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 0x7f562c137890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f563179b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 0x7f562c137890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45820/0 (socket says 172.21.15.133:45820) 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f563179b640 1 -- 172.21.15.133:0/708567742 learned_addr learned my addr 172.21.15.133:0/708567742 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f5630f9a640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f562c157b80 0x7f562c1560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f5631f9c640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c137e10 0x7f562c156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f563179b640 1 -- 172.21.15.133:0/708567742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c137e10 msgr2=0x7f562c156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f563179b640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c137e10 0x7f562c156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f563179b640 1 -- 172.21.15.133:0/708567742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f562c157b80 msgr2=0x7f562c1560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f563179b640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f562c157b80 0x7f562c1560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f563179b640 1 -- 172.21.15.133:0/708567742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c1580d0 con 0x7f56180039f0 2024-04-25T10:11:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f5630f9a640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f562c157b80 0x7f562c1560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f563179b640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 0x7f562c137890 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f561c01c880 tx=0x7f5618005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f5621ffb640 1 -- 172.21.15.133:0/708567742 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0be070 con 0x7f56180039f0 2024-04-25T10:11:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f5621ffb640 1 -- 172.21.15.133:0/708567742 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c00f000 con 0x7f56180039f0 2024-04-25T10:11:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f56339ff640 1 -- 172.21.15.133:0/708567742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c155310 con 0x7f56180039f0 2024-04-25T10:11:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f5621ffb640 1 -- 172.21.15.133:0/708567742 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0a7460 con 0x7f56180039f0 2024-04-25T10:11:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f56339ff640 1 -- 172.21.15.133:0/708567742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c1558d0 con 0x7f56180039f0 2024-04-25T10:11:00.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.006+0000 7f56339ff640 1 -- 172.21.15.133:0/708567742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c068e80 con 0x7f56180039f0 2024-04-25T10:11:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.010+0000 7f5621ffb640 1 -- 172.21.15.133:0/708567742 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f561c0c1070 con 0x7f56180039f0 2024-04-25T10:11:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.010+0000 7f5621ffb640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5610066f90 0x7f5610069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.010+0000 7f5630f9a640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5610066f90 0x7f5610069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.010+0000 7f5621ffb640 1 -- 172.21.15.133:0/708567742 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f561c11ec30 con 0x7f56180039f0 2024-04-25T10:11:00.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.010+0000 7f5630f9a640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5610066f90 0x7f5610069450 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5628000c10 tx=0x7f5628009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.014+0000 7f5621ffb640 1 -- 172.21.15.133:0/708567742 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f561c0ac020 con 0x7f56180039f0 2024-04-25T10:11:00.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.202+0000 7f56339ff640 1 -- 172.21.15.133:0/708567742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f562c06e7e0 con 0x7f56180039f0 2024-04-25T10:11:00.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.862+0000 7f5621ffb640 1 -- 172.21.15.133:0/708567742 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7f561c016760 con 0x7f56180039f0 2024-04-25T10:11:00.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 -- 172.21.15.133:0/708567742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5610066f90 msgr2=0x7f5610069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5610066f90 0x7f5610069450 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5628000c10 tx=0x7f5628009040 comp rx=0 tx=0).stop 2024-04-25T10:11:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 -- 172.21.15.133:0/708567742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 msgr2=0x7f562c137890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 0x7f562c137890 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f561c01c880 tx=0x7f5618005840 comp rx=0 tx=0).stop 2024-04-25T10:11:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 -- 172.21.15.133:0/708567742 shutdown_connections 2024-04-25T10:11:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f562c137e10 0x7f562c156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f562c157b80 0x7f562c1560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5610066f90 0x7f5610069450 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 --2- 172.21.15.133:0/708567742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56180039f0 0x7f562c137890 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 -- 172.21.15.133:0/708567742 >> 172.21.15.133:0/708567742 conn(0x7f562c066f70 msgr2=0x7f562c15de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 -- 172.21.15.133:0/708567742 shutdown_connections 2024-04-25T10:11:00.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:00.866+0000 7f56339ff640 1 -- 172.21.15.133:0/708567742 wait complete. 2024-04-25T10:11:00.891 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:11:01.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb26bf1b640 1 Processor -- start 2024-04-25T10:11:01.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb26bf1b640 1 -- start start 2024-04-25T10:11:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb26bf1b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2640ced10 0x7fb2641685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb26bf1b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2640d22c0 con 0x7fb2640d3360 2024-04-25T10:11:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb26bf1b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2640d25a0 con 0x7fb26415d9d0 2024-04-25T10:11:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb26bf1b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2640d2880 con 0x7fb2640ced10 2024-04-25T10:11:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb26a4b8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2640ced10 0x7fb2641685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb2694b6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb2640d3360 0x7fb2640ce600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40050/0 (socket says 172.21.15.133:40050) 2024-04-25T10:11:01.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb2694b6640 1 -- 172.21.15.133:0/218406195 learned_addr learned my addr 172.21.15.133:0/218406195 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:01.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb26a4b8640 1 -- 172.21.15.133:0/218406195 >> v1:172.21.15.162:6789/0 conn(0x7fb26415d9d0 legacy=0x7fb2640b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb26a4b8640 1 -- 172.21.15.133:0/218406195 >> v1:172.21.15.133:6789/0 conn(0x7fb2640d3360 legacy=0x7fb2640ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb268cb5640 1 -- 172.21.15.133:0/218406195 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615419338 0 0) 0x7fb2640d22c0 con 0x7fb2640d3360 2024-04-25T10:11:01.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.062+0000 7fb26a4b8640 1 -- 172.21.15.133:0/218406195 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2640d2b60 con 0x7fb2640ced10 2024-04-25T10:11:01.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26a4b8640 1 --2- 172.21.15.133:0/218406195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2640ced10 0x7fb2641685d0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb2540097c0 tx=0x7fb25401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7217d3e72db23e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb268cb5640 1 -- 172.21.15.133:0/218406195 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2540b0070 con 0x7fb2640ced10 2024-04-25T10:11:01.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb268cb5640 1 -- 172.21.15.133:0/218406195 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2540ab600 con 0x7fb2640ced10 2024-04-25T10:11:01.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb268cb5640 1 -- 172.21.15.133:0/218406195 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2540ab9d0 con 0x7fb2640ced10 2024-04-25T10:11:01.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 -- 172.21.15.133:0/218406195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2640ced10 msgr2=0x7fb2641685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 --2- 172.21.15.133:0/218406195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2640ced10 0x7fb2641685d0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb2540097c0 tx=0x7fb25401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 -- 172.21.15.133:0/218406195 shutdown_connections 2024-04-25T10:11:01.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 --2- 172.21.15.133:0/218406195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2640ced10 0x7fb2641685d0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 -- 172.21.15.133:0/218406195 >> 172.21.15.133:0/218406195 conn(0x7fb26406c080 msgr2=0x7fb26406c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:01.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 -- 172.21.15.133:0/218406195 shutdown_connections 2024-04-25T10:11:01.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 -- 172.21.15.133:0/218406195 wait complete. 2024-04-25T10:11:01.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 Processor -- start 2024-04-25T10:11:01.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 -- start start 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2640ced10 0x7fb2640c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2640d3360 0x7fb26415fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26415d9d0 0x7fb26415c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26416bad0 con 0x7fb2640ced10 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26416b950 con 0x7fb26415d9d0 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb26bf1b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26416bc50 con 0x7fb2640d3360 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb269cb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2640ced10 0x7fb2640c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb269cb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2640ced10 0x7fb2640c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45832/0 (socket says 172.21.15.133:45832) 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.066+0000 7fb269cb7640 1 -- 172.21.15.133:0/3263095114 learned_addr learned my addr 172.21.15.133:0/3263095114 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb269cb7640 1 -- 172.21.15.133:0/3263095114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2640d3360 msgr2=0x7fb26415fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb269cb7640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2640d3360 0x7fb26415fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb269cb7640 1 -- 172.21.15.133:0/3263095114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26415d9d0 msgr2=0x7fb26415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb269cb7640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26415d9d0 0x7fb26415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb269cb7640 1 -- 172.21.15.133:0/3263095114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb264152ff0 con 0x7fb2640ced10 2024-04-25T10:11:01.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb269cb7640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2640ced10 0x7fb2640c93c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00c7a0 tx=0x7fb25c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb2627fc640 1 -- 172.21.15.133:0/3263095114 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c003b80 con 0x7fb2640ced10 2024-04-25T10:11:01.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb2627fc640 1 -- 172.21.15.133:0/3263095114 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c003d20 con 0x7fb2640ced10 2024-04-25T10:11:01.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb2627fc640 1 -- 172.21.15.133:0/3263095114 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c00aa90 con 0x7fb2640ced10 2024-04-25T10:11:01.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb26bf1b640 1 -- 172.21.15.133:0/3263095114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2641532d0 con 0x7fb2640ced10 2024-04-25T10:11:01.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb26bf1b640 1 -- 172.21.15.133:0/3263095114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb264153890 con 0x7fb2640ced10 2024-04-25T10:11:01.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb26bf1b640 1 -- 172.21.15.133:0/3263095114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb264069000 con 0x7fb2640ced10 2024-04-25T10:11:01.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb2627fc640 1 -- 172.21.15.133:0/3263095114 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb25c00ac30 con 0x7fb2640ced10 2024-04-25T10:11:01.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb2627fc640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 0x7fb250069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb2627fc640 1 -- 172.21.15.133:0/3263095114 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb25c085130 con 0x7fb2640ced10 2024-04-25T10:11:01.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb2694b6640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 0x7fb250069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.070+0000 7fb2694b6640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 0x7fb250069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb258000c10 tx=0x7fb258007910 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.074+0000 7fb2627fc640 1 -- 172.21.15.133:0/3263095114 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb25c055f80 con 0x7fb2640ced10 2024-04-25T10:11:01.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.222+0000 7fb26bf1b640 1 -- 172.21.15.133:0/3263095114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb2641547a0 con 0x7fb2640ced10 2024-04-25T10:11:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.994+0000 7fb2627fc640 1 -- 172.21.15.133:0/3263095114 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb25c055880 con 0x7fb2640ced10 2024-04-25T10:11:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:11:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 -- 172.21.15.133:0/3263095114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 msgr2=0x7fb250069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 0x7fb250069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb258000c10 tx=0x7fb258007910 comp rx=0 tx=0).stop 2024-04-25T10:11:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 -- 172.21.15.133:0/3263095114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2640ced10 msgr2=0x7fb2640c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2640ced10 0x7fb2640c93c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00c7a0 tx=0x7fb25c00cc60 comp rx=0 tx=0).stop 2024-04-25T10:11:02.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 -- 172.21.15.133:0/3263095114 shutdown_connections 2024-04-25T10:11:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26415d9d0 0x7fb26415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2640d3360 0x7fb26415fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 0x7fb250069420 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 --2- 172.21.15.133:0/3263095114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2640ced10 0x7fb2640c93c0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 -- 172.21.15.133:0/3263095114 >> 172.21.15.133:0/3263095114 conn(0x7fb26406c080 msgr2=0x7fb2640c9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 -- 172.21.15.133:0/3263095114 shutdown_connections 2024-04-25T10:11:02.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:01.998+0000 7fb26bf1b640 1 -- 172.21.15.133:0/3263095114 wait complete. 2024-04-25T10:11:02.023 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:11:02.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.154+0000 7fbbc529e640 1 Processor -- start 2024-04-25T10:11:02.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.154+0000 7fbbc529e640 1 -- start start 2024-04-25T10:11:02.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.154+0000 7fbbc529e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbbc015ea50 0x7fbbc00cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.154+0000 7fbbc529e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc00cb3b0 con 0x7fbbc006eff0 2024-04-25T10:11:02.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.154+0000 7fbbc529e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc015ef90 con 0x7fbbc00cf210 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.154+0000 7fbbc529e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc00cb630 con 0x7fbbc015ea50 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.154+0000 7fbbbe7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fbbc006eff0 0x7fbbc015e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40064/0 (socket says 172.21.15.133:40064) 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.154+0000 7fbbbe7fc640 1 -- 172.21.15.133:0/752075596 learned_addr learned my addr 172.21.15.133:0/752075596 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.154+0000 7fbbbf7fe640 1 --2- 172.21.15.133:0/752075596 >> v2:172.21.15.162:3300/0 conn(0x7fbbc015ea50 0x7fbbc00cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520883382 0 0) 0x7fbbb4001120 con 0x7fbbc006eff0 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbba8003530 con 0x7fbbc006eff0 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2890038259 0 0) 0x7fbba8003530 con 0x7fbbc006eff0 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 >> v2:172.21.15.162:3300/0 conn(0x7fbbc015ea50 msgr2=0x7fbbc00cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 --2- 172.21.15.133:0/752075596 >> v2:172.21.15.162:3300/0 conn(0x7fbbc015ea50 0x7fbbc00cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 >> v1:172.21.15.162:6789/0 conn(0x7fbbc00cf210 legacy=0x7fbbc006e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc00cb910 con 0x7fbbc006eff0 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fbbb4001850 con 0x7fbbc006eff0 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 >> v1:172.21.15.133:6789/0 conn(0x7fbbc006eff0 legacy=0x7fbbc015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 --2- 172.21.15.133:0/752075596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba80039c0 0x7fbba8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 --2- 172.21.15.133:0/752075596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba8004810 0x7fbba8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 --2- 172.21.15.133:0/752075596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba80051d0 0x7fbba800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00cb630 con 0x7fbba8004810 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba800c170 con 0x7fbba80039c0 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba800c370 con 0x7fbba80051d0 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbe7fc640 1 --2- 172.21.15.133:0/752075596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba8004810 0x7fbba8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbeffd640 1 --2- 172.21.15.133:0/752075596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba80039c0 0x7fbba8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbf7fe640 1 --2- 172.21.15.133:0/752075596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba80051d0 0x7fbba800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbe7fc640 1 -- 172.21.15.133:0/752075596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba80051d0 msgr2=0x7fbba800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbe7fc640 1 --2- 172.21.15.133:0/752075596 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbba80051d0 0x7fbba800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbe7fc640 1 -- 172.21.15.133:0/752075596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba80039c0 msgr2=0x7fbba8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbe7fc640 1 --2- 172.21.15.133:0/752075596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba80039c0 0x7fbba8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbe7fc640 1 -- 172.21.15.133:0/752075596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba8003530 con 0x7fbba8004810 2024-04-25T10:11:02.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbf7fe640 1 -- 172.21.15.133:0/752075596 reap_dead start 2024-04-25T10:11:02.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbe7fc640 1 --2- 172.21.15.133:0/752075596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba8004810 0x7fbba8004c90 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400c490 tx=0x7fbbb401cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00e58035b324f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb400c0e0 con 0x7fbba8004810 2024-04-25T10:11:02.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb40a9020 con 0x7fbba8004810 2024-04-25T10:11:02.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbbdffb640 1 -- 172.21.15.133:0/752075596 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4005ae0 con 0x7fbba8004810 2024-04-25T10:11:02.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbc529e640 1 -- 172.21.15.133:0/752075596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba8004810 msgr2=0x7fbba8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbc529e640 1 --2- 172.21.15.133:0/752075596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba8004810 0x7fbba8004c90 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400c490 tx=0x7fbbb401cb80 comp rx=0 tx=0).stop 2024-04-25T10:11:02.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbc529e640 1 -- 172.21.15.133:0/752075596 shutdown_connections 2024-04-25T10:11:02.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbc529e640 1 --2- 172.21.15.133:0/752075596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba8004810 0x7fbba8004c90 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbc529e640 1 -- 172.21.15.133:0/752075596 >> 172.21.15.133:0/752075596 conn(0x7fbbc00cc7c0 msgr2=0x7fbbc00ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbc529e640 1 -- 172.21.15.133:0/752075596 shutdown_connections 2024-04-25T10:11:02.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.158+0000 7fbbc529e640 1 -- 172.21.15.133:0/752075596 wait complete. 2024-04-25T10:11:02.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbc529e640 1 Processor -- start 2024-04-25T10:11:02.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbc529e640 1 -- start start 2024-04-25T10:11:02.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbc529e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8004810 0x7fbbc01534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbc529e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc00c8900 0x7fbbc00c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbc529e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc00c7350 0x7fbbc00af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbc529e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00cf770 con 0x7fbbc00c8900 2024-04-25T10:11:02.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbc529e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00cf5f0 con 0x7fbba8004810 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbc529e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00ce450 con 0x7fbbc00c7350 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbe7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc00c8900 0x7fbbc00c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbe7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc00c8900 0x7fbbc00c6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45860/0 (socket says 172.21.15.133:45860) 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbe7fc640 1 -- 172.21.15.133:0/2738033758 learned_addr learned my addr 172.21.15.133:0/2738033758 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbeffd640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8004810 0x7fbbc01534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbf7fe640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc00c7350 0x7fbbc00af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbe7fc640 1 -- 172.21.15.133:0/2738033758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc00c7350 msgr2=0x7fbbc00af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbe7fc640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc00c7350 0x7fbbc00af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbe7fc640 1 -- 172.21.15.133:0/2738033758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8004810 msgr2=0x7fbbc01534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbe7fc640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8004810 0x7fbbc01534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbe7fc640 1 -- 172.21.15.133:0/2738033758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc00ca550 con 0x7fbbc00c8900 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbeffd640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8004810 0x7fbbc01534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbbe7fc640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc00c8900 0x7fbbc00c6e10 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fbbb401c9b0 tx=0x7fbba8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbba77fe640 1 -- 172.21.15.133:0/2738033758 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4000ee0 con 0x7fbbc00c8900 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbba77fe640 1 -- 172.21.15.133:0/2738033758 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb4001650 con 0x7fbbc00c8900 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbc529e640 1 -- 172.21.15.133:0/2738033758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc01539f0 con 0x7fbbc00c8900 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbba77fe640 1 -- 172.21.15.133:0/2738033758 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb40a5930 con 0x7fbbc00c8900 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbc529e640 1 -- 172.21.15.133:0/2738033758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc00c7b40 con 0x7fbbc00c8900 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.162+0000 7fbbc529e640 1 -- 172.21.15.133:0/2738033758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc0069000 con 0x7fbbc00c8900 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.166+0000 7fbba77fe640 1 -- 172.21.15.133:0/2738033758 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbbb40a5ad0 con 0x7fbbc00c8900 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.166+0000 7fbba77fe640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbbac067130 0x7fbbac0695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.166+0000 7fbba77fe640 1 -- 172.21.15.133:0/2738033758 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5865+0+0 (secure 0 0 0) 0x7fbbb411e760 con 0x7fbbc00c8900 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.166+0000 7fbbbeffd640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbbac067130 0x7fbbac0695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.166+0000 7fbbbeffd640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbbac067130 0x7fbbac0695f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0000c10 tx=0x7fbbb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.170+0000 7fbba77fe640 1 -- 172.21.15.133:0/2738033758 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbbc01539f0 con 0x7fbbc00c8900 2024-04-25T10:11:02.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:02.326+0000 7fbbc529e640 1 -- 172.21.15.133:0/2738033758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbbc006e6c0 con 0x7fbbc00c8900 2024-04-25T10:11:03.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.002+0000 7fbba77fe640 1 -- 172.21.15.133:0/2738033758 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbbc00cb3b0 con 0x7fbbc00c8900 2024-04-25T10:11:03.007 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:11:03.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.010+0000 7fbbc529e640 1 -- 172.21.15.133:0/2738033758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbbac067130 msgr2=0x7fbbac0695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.010+0000 7fbbc529e640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbbac067130 0x7fbbac0695f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0000c10 tx=0x7fbbb0009040 comp rx=0 tx=0).stop 2024-04-25T10:11:03.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.010+0000 7fbbc529e640 1 -- 172.21.15.133:0/2738033758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc00c8900 msgr2=0x7fbbc00c6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.010+0000 7fbbc529e640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc00c8900 0x7fbbc00c6e10 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fbbb401c9b0 tx=0x7fbba8005840 comp rx=0 tx=0).stop 2024-04-25T10:11:03.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.010+0000 7fbbc529e640 1 -- 172.21.15.133:0/2738033758 shutdown_connections 2024-04-25T10:11:03.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.010+0000 7fbbc529e640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbbc00c7350 0x7fbbc00af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.010+0000 7fbbc529e640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbbac067130 0x7fbbac0695f0 secure :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0000c10 tx=0x7fbbb0009040 comp rx=0 tx=0).stop 2024-04-25T10:11:03.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.010+0000 7fbbc529e640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc00c8900 0x7fbbc00c6e10 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.010+0000 7fbbc529e640 1 --2- 172.21.15.133:0/2738033758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbba8004810 0x7fbbc01534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.010+0000 7fbbc529e640 1 -- 172.21.15.133:0/2738033758 >> 172.21.15.133:0/2738033758 conn(0x7fbbc00cc7c0 msgr2=0x7fbbc00ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.018+0000 7fbbc529e640 1 -- 172.21.15.133:0/2738033758 shutdown_connections 2024-04-25T10:11:03.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.018+0000 7fbbc529e640 1 -- 172.21.15.133:0/2738033758 wait complete. 2024-04-25T10:11:03.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:11:03.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.170+0000 7fc7d2ad6640 1 Processor -- start 2024-04-25T10:11:03.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.170+0000 7fc7d2ad6640 1 -- start start 2024-04-25T10:11:03.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.170+0000 7fc7d2ad6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7cc06c7b0 0x7fc7cc06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.170+0000 7fc7d2ad6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc7cc15d320 0x7fc7cc15d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.170+0000 7fc7d2ad6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7cc0b1220 con 0x7fc7cc0ce600 2024-04-25T10:11:03.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.170+0000 7fc7d2ad6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7cc0b2ff0 con 0x7fc7cc15d320 2024-04-25T10:11:03.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.170+0000 7fc7d2ad6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0b1500 con 0x7fc7cc06c7b0 2024-04-25T10:11:03.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d1073640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc7cc15d320 0x7fc7cc15d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d0872640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc7cc0ce600 0x7fc7cc0d0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40080/0 (socket says 172.21.15.133:40080) 2024-04-25T10:11:03.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d0872640 1 -- 172.21.15.133:0/3963025806 learned_addr learned my addr 172.21.15.133:0/3963025806 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:03.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7c3fff640 1 --2- 172.21.15.133:0/3963025806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7cc06c7b0 0x7fc7cc06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d1073640 1 -- 172.21.15.133:0/3963025806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7cc06c7b0 msgr2=0x7fc7cc06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d1073640 1 --2- 172.21.15.133:0/3963025806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7cc06c7b0 0x7fc7cc06cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d1073640 1 -- 172.21.15.133:0/3963025806 >> v1:172.21.15.133:6789/0 conn(0x7fc7cc0ce600 legacy=0x7fc7cc0d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d1073640 1 -- 172.21.15.133:0/3963025806 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc0b17e0 con 0x7fc7cc15d320 2024-04-25T10:11:03.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7c37fe640 1 -- 172.21.15.133:0/3963025806 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720759828 0 0) 0x7fc7cc0b1220 con 0x7fc7cc0ce600 2024-04-25T10:11:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d1073640 1 --2- 172.21.15.133:0/3963025806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7cc15d320 0x7fc7cc15d7b0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009d90 tx=0x7fc7c401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdbe82cbf6137cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7c37fe640 1 -- 172.21.15.133:0/3963025806 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40b2070 con 0x7fc7cc15d320 2024-04-25T10:11:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7c37fe640 1 -- 172.21.15.133:0/3963025806 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c40a5830 con 0x7fc7cc15d320 2024-04-25T10:11:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7c37fe640 1 -- 172.21.15.133:0/3963025806 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40a5c00 con 0x7fc7cc15d320 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3963025806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7cc15d320 msgr2=0x7fc7cc15d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d2ad6640 1 --2- 172.21.15.133:0/3963025806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7cc15d320 0x7fc7cc15d7b0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009d90 tx=0x7fc7c401c970 comp rx=0 tx=0).stop 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3963025806 shutdown_connections 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d2ad6640 1 --2- 172.21.15.133:0/3963025806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7cc15d320 0x7fc7cc15d7b0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d2ad6640 1 --2- 172.21.15.133:0/3963025806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7cc06c7b0 0x7fc7cc06cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3963025806 >> 172.21.15.133:0/3963025806 conn(0x7fc7cc09a990 msgr2=0x7fc7cc0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3963025806 shutdown_connections 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.174+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3963025806 wait complete. 2024-04-25T10:11:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d2ad6640 1 Processor -- start 2024-04-25T10:11:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d2ad6640 1 -- start start 2024-04-25T10:11:03.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d2ad6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7cc06c7b0 0x7fc7cc14f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d2ad6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7cc0ce600 0x7fc7cc0c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d2ad6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7cc15d320 0x7fc7cc0afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d2ad6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0ae280 con 0x7fc7cc15d320 2024-04-25T10:11:03.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d2ad6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0d21a0 con 0x7fc7cc0ce600 2024-04-25T10:11:03.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d2ad6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc15e4d0 con 0x7fc7cc06c7b0 2024-04-25T10:11:03.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d1073640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7cc15d320 0x7fc7cc0afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d1073640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7cc15d320 0x7fc7cc0afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45890/0 (socket says 172.21.15.133:45890) 2024-04-25T10:11:03.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d1073640 1 -- 172.21.15.133:0/3646862532 learned_addr learned my addr 172.21.15.133:0/3646862532 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d0872640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7cc06c7b0 0x7fc7cc14f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7c3fff640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7cc0ce600 0x7fc7cc0c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d1073640 1 -- 172.21.15.133:0/3646862532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7cc06c7b0 msgr2=0x7fc7cc14f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d1073640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7cc06c7b0 0x7fc7cc14f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d1073640 1 -- 172.21.15.133:0/3646862532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7cc0ce600 msgr2=0x7fc7cc0c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d1073640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7cc0ce600 0x7fc7cc0c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d1073640 1 -- 172.21.15.133:0/3646862532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc0c77b0 con 0x7fc7cc15d320 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d1073640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7cc15d320 0x7fc7cc0afcc0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fc7c401b020 tx=0x7fc7c4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7c17fa640 1 -- 172.21.15.133:0/3646862532 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40b2050 con 0x7fc7cc15d320 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7c17fa640 1 -- 172.21.15.133:0/3646862532 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c40be070 con 0x7fc7cc15d320 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3646862532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7cc0c7a90 con 0x7fc7cc15d320 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7c17fa640 1 -- 172.21.15.133:0/3646862532 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40ad850 con 0x7fc7cc15d320 2024-04-25T10:11:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3646862532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7cc14ff30 con 0x7fc7cc15d320 2024-04-25T10:11:03.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.178+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3646862532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7cc067c70 con 0x7fc7cc15d320 2024-04-25T10:11:03.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.182+0000 7fc7c17fa640 1 -- 172.21.15.133:0/3646862532 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7c40ad9f0 con 0x7fc7cc15d320 2024-04-25T10:11:03.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.182+0000 7fc7c17fa640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7b8066f60 0x7fc7b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.182+0000 7fc7c17fa640 1 -- 172.21.15.133:0/3646862532 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc7c411fec0 con 0x7fc7cc15d320 2024-04-25T10:11:03.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.182+0000 7fc7d0872640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7b8066f60 0x7fc7b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.182+0000 7fc7d0872640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7b8066f60 0x7fc7b8069420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8000f70 tx=0x7fc7c8007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.186+0000 7fc7c17fa640 1 -- 172.21.15.133:0/3646862532 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7c4016560 con 0x7fc7cc15d320 2024-04-25T10:11:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.346+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3646862532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc7cc06c590 con 0x7fc7cc15d320 2024-04-25T10:11:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7c17fa640 1 -- 172.21.15.133:0/3646862532 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc7c40f11f0 con 0x7fc7cc15d320 2024-04-25T10:11:03.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3646862532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7b8066f60 msgr2=0x7fc7b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7b8066f60 0x7fc7b8069420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8000f70 tx=0x7fc7c8007c10 comp rx=0 tx=0).stop 2024-04-25T10:11:03.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3646862532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7cc15d320 msgr2=0x7fc7cc0afcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7cc15d320 0x7fc7cc0afcc0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fc7c401b020 tx=0x7fc7c4019710 comp rx=0 tx=0).stop 2024-04-25T10:11:03.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3646862532 shutdown_connections 2024-04-25T10:11:03.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc7b8066f60 0x7fc7b8069420 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7cc15d320 0x7fc7cc0afcc0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7cc0ce600 0x7fc7cc0c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 --2- 172.21.15.133:0/3646862532 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7cc06c7b0 0x7fc7cc14f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3646862532 >> 172.21.15.133:0/3646862532 conn(0x7fc7cc09a990 msgr2=0x7fc7cc0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3646862532 shutdown_connections 2024-04-25T10:11:03.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.354+0000 7fc7d2ad6640 1 -- 172.21.15.133:0/3646862532 wait complete. 2024-04-25T10:11:03.375 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T10:11:03.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc024dbe640 1 Processor -- start 2024-04-25T10:11:03.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc024dbe640 1 -- start start 2024-04-25T10:11:03.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc024dbe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc02015da80 0x7fc020162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc024dbe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc02015e570 0x7fc0200ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc024dbe640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02015eb50 con 0x7fc0200ce6e0 2024-04-25T10:11:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc024dbe640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02015f110 con 0x7fc02015e570 2024-04-25T10:11:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc024dbe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc020161c00 con 0x7fc02015da80 2024-04-25T10:11:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc01ed9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc02015e570 0x7fc0200ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc01ed9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc02015e570 0x7fc0200ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45902/0 (socket says 172.21.15.133:45902) 2024-04-25T10:11:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc01ed9d640 1 -- 172.21.15.133:0/4052531379 learned_addr learned my addr 172.21.15.133:0/4052531379 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc01e59c640 1 --2- 172.21.15.133:0/4052531379 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc02015da80 0x7fc020162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc01ed9d640 1 -- 172.21.15.133:0/4052531379 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc02015da80 msgr2=0x7fc020162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc01ed9d640 1 --2- 172.21.15.133:0/4052531379 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc02015da80 0x7fc020162090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc01ed9d640 1 -- 172.21.15.133:0/4052531379 >> v1:172.21.15.162:6789/0 conn(0x7fc0200ce6e0 legacy=0x7fc02015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc01ed9d640 1 -- 172.21.15.133:0/4052531379 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02015ee30 con 0x7fc02015e570 2024-04-25T10:11:03.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.510+0000 7fc01ed9d640 1 --2- 172.21.15.133:0/4052531379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02015e570 0x7fc0200ceb60 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fc018009e50 tx=0x7fc01801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9433b3aa4be49259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01dd9b640 1 -- 172.21.15.133:0/4052531379 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0180a5720 con 0x7fc02015e570 2024-04-25T10:11:03.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01dd9b640 1 -- 172.21.15.133:0/4052531379 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0180a58c0 con 0x7fc02015e570 2024-04-25T10:11:03.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01dd9b640 1 -- 172.21.15.133:0/4052531379 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0180a5c90 con 0x7fc02015e570 2024-04-25T10:11:03.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 -- 172.21.15.133:0/4052531379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02015e570 msgr2=0x7fc0200ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 --2- 172.21.15.133:0/4052531379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02015e570 0x7fc0200ceb60 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fc018009e50 tx=0x7fc01801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 -- 172.21.15.133:0/4052531379 shutdown_connections 2024-04-25T10:11:03.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 --2- 172.21.15.133:0/4052531379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02015e570 0x7fc0200ceb60 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 --2- 172.21.15.133:0/4052531379 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc02015da80 0x7fc020162090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 -- 172.21.15.133:0/4052531379 >> 172.21.15.133:0/4052531379 conn(0x7fc02006c0d0 msgr2=0x7fc02006c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 -- 172.21.15.133:0/4052531379 shutdown_connections 2024-04-25T10:11:03.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 -- 172.21.15.133:0/4052531379 wait complete. 2024-04-25T10:11:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 Processor -- start 2024-04-25T10:11:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 -- start start 2024-04-25T10:11:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0200ce6e0 0x7fc020153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc02015da80 0x7fc0200caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02015e570 0x7fc0201552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200d25b0 con 0x7fc02015e570 2024-04-25T10:11:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200b9660 con 0x7fc02015da80 2024-04-25T10:11:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200d2730 con 0x7fc0200ce6e0 2024-04-25T10:11:03.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02015e570 0x7fc0201552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02015e570 0x7fc0201552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45906/0 (socket says 172.21.15.133:45906) 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01ed9d640 1 -- 172.21.15.133:0/526331398 learned_addr learned my addr 172.21.15.133:0/526331398 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc017fff640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc02015da80 0x7fc0200caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01e59c640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0200ce6e0 0x7fc020153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01ed9d640 1 -- 172.21.15.133:0/526331398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0200ce6e0 msgr2=0x7fc020153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01ed9d640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0200ce6e0 0x7fc020153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01ed9d640 1 -- 172.21.15.133:0/526331398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc02015da80 msgr2=0x7fc0200caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01ed9d640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc02015da80 0x7fc0200caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01ed9d640 1 -- 172.21.15.133:0/526331398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020154800 con 0x7fc02015e570 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01e59c640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0200ce6e0 0x7fc020153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc01ed9d640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02015e570 0x7fc0201552a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fc0180af040 tx=0x7fc018019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc016ffd640 1 -- 172.21.15.133:0/526331398 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0180bc020 con 0x7fc02015e570 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc016ffd640 1 -- 172.21.15.133:0/526331398 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0180b8670 con 0x7fc02015e570 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 -- 172.21.15.133:0/526331398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0200cb430 con 0x7fc02015e570 2024-04-25T10:11:03.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc016ffd640 1 -- 172.21.15.133:0/526331398 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0180b8a20 con 0x7fc02015e570 2024-04-25T10:11:03.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.514+0000 7fc024dbe640 1 -- 172.21.15.133:0/526331398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc020154d80 con 0x7fc02015e570 2024-04-25T10:11:03.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.518+0000 7fc024dbe640 1 -- 172.21.15.133:0/526331398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc020067c70 con 0x7fc02015e570 2024-04-25T10:11:03.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.518+0000 7fc016ffd640 1 -- 172.21.15.133:0/526331398 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0180bf020 con 0x7fc02015e570 2024-04-25T10:11:03.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.518+0000 7fc016ffd640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc000066f60 0x7fc000069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.518+0000 7fc016ffd640 1 -- 172.21.15.133:0/526331398 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc01811ffb0 con 0x7fc02015e570 2024-04-25T10:11:03.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.518+0000 7fc01e59c640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc000066f60 0x7fc000069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.518+0000 7fc01e59c640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc000066f60 0x7fc000069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc010006fd0 tx=0x7fc010008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.522+0000 7fc016ffd640 1 -- 172.21.15.133:0/526331398 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0180c6030 con 0x7fc02015e570 2024-04-25T10:11:03.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:03.750+0000 7fc024dbe640 1 -- 172.21.15.133:0/526331398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7fc0200d1670 con 0x7fc02015e570 2024-04-25T10:11:04.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.006+0000 7fc016ffd640 1 -- 172.21.15.133:0/526331398 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v409) v1 ==== 144+0+0 (secure 0 0 0) 0x7fc0180165e0 con 0x7fc02015e570 2024-04-25T10:11:04.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 -- 172.21.15.133:0/526331398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc000066f60 msgr2=0x7fc000069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc000066f60 0x7fc000069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc010006fd0 tx=0x7fc010008040 comp rx=0 tx=0).stop 2024-04-25T10:11:04.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 -- 172.21.15.133:0/526331398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02015e570 msgr2=0x7fc0201552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02015e570 0x7fc0201552a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fc0180af040 tx=0x7fc018019790 comp rx=0 tx=0).stop 2024-04-25T10:11:04.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 -- 172.21.15.133:0/526331398 shutdown_connections 2024-04-25T10:11:04.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc000066f60 0x7fc000069420 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc02015e570 0x7fc0201552a0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc02015da80 0x7fc0200caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 --2- 172.21.15.133:0/526331398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0200ce6e0 0x7fc020153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 -- 172.21.15.133:0/526331398 >> 172.21.15.133:0/526331398 conn(0x7fc02006c0d0 msgr2=0x7fc0200b83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 -- 172.21.15.133:0/526331398 shutdown_connections 2024-04-25T10:11:04.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.010+0000 7fc024dbe640 1 -- 172.21.15.133:0/526331398 wait complete. 2024-04-25T10:11:04.035 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:11:04.035 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:11:04.035 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:04.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff2978ad640 1 Processor -- start 2024-04-25T10:11:04.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff2978ad640 1 -- start start 2024-04-25T10:11:04.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff2978ad640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff290158110 0x7ff2900b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff2978ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2901584f0 0x7ff29015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff2978ad640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2900cbe60 con 0x7ff29006e460 2024-04-25T10:11:04.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff2978ad640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2900b19b0 con 0x7ff290158110 2024-04-25T10:11:04.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff2978ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29015b360 con 0x7ff2901584f0 2024-04-25T10:11:04.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff295649640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff290158110 0x7ff2900b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff294e48640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2901584f0 0x7ff29015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff295e4a640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7ff29006e460 0x7ff29015e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:55924/0 (socket says 172.21.15.133:55924) 2024-04-25T10:11:04.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff295e4a640 1 -- 172.21.15.133:0/343042215 learned_addr learned my addr 172.21.15.133:0/343042215 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:04.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff295649640 1 -- 172.21.15.133:0/343042215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2901584f0 msgr2=0x7ff29015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff295649640 1 --2- 172.21.15.133:0/343042215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2901584f0 0x7ff29015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff295649640 1 -- 172.21.15.133:0/343042215 >> v1:172.21.15.162:6789/0 conn(0x7ff29006e460 legacy=0x7ff29015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff295649640 1 -- 172.21.15.133:0/343042215 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29015b640 con 0x7ff290158110 2024-04-25T10:11:04.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.158+0000 7ff294e48640 1 --2- 172.21.15.133:0/343042215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2901584f0 0x7ff29015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:04.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff295649640 1 --2- 172.21.15.133:0/343042215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff290158110 0x7ff2900b1f00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0097c0 tx=0x7ff28c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b0df169f491640b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff287fff640 1 -- 172.21.15.133:0/343042215 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0b1070 con 0x7ff290158110 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff287fff640 1 -- 172.21.15.133:0/343042215 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c0ac410 con 0x7ff290158110 2024-04-25T10:11:04.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff287fff640 1 -- 172.21.15.133:0/343042215 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0ac7e0 con 0x7ff290158110 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff2978ad640 1 -- 172.21.15.133:0/343042215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff290158110 msgr2=0x7ff2900b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff2978ad640 1 --2- 172.21.15.133:0/343042215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff290158110 0x7ff2900b1f00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0097c0 tx=0x7ff28c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff2978ad640 1 -- 172.21.15.133:0/343042215 shutdown_connections 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff2978ad640 1 --2- 172.21.15.133:0/343042215 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2901584f0 0x7ff29015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff2978ad640 1 --2- 172.21.15.133:0/343042215 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff290158110 0x7ff2900b1f00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff2978ad640 1 -- 172.21.15.133:0/343042215 >> 172.21.15.133:0/343042215 conn(0x7ff290067000 msgr2=0x7ff29006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff2978ad640 1 -- 172.21.15.133:0/343042215 shutdown_connections 2024-04-25T10:11:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff2978ad640 1 -- 172.21.15.133:0/343042215 wait complete. 2024-04-25T10:11:04.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff2978ad640 1 Processor -- start 2024-04-25T10:11:04.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.162+0000 7ff2978ad640 1 -- start start 2024-04-25T10:11:04.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff2978ad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff29006e460 0x7ff29013eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff2978ad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff290158110 0x7ff29014e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff2978ad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2901584f0 0x7ff29014e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff2978ad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900cd7f0 con 0x7ff290158110 2024-04-25T10:11:04.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff2978ad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900cd670 con 0x7ff29006e460 2024-04-25T10:11:04.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff2978ad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff290167470 con 0x7ff2901584f0 2024-04-25T10:11:04.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff294e48640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff290158110 0x7ff29014e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff294e48640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff290158110 0x7ff29014e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45916/0 (socket says 172.21.15.133:45916) 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff294e48640 1 -- 172.21.15.133:0/1746115419 learned_addr learned my addr 172.21.15.133:0/1746115419 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff295649640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff29006e460 0x7ff29013eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff295e4a640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2901584f0 0x7ff29014e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff294e48640 1 -- 172.21.15.133:0/1746115419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2901584f0 msgr2=0x7ff29014e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff294e48640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2901584f0 0x7ff29014e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff294e48640 1 -- 172.21.15.133:0/1746115419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff29006e460 msgr2=0x7ff29013eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff294e48640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff29006e460 0x7ff29013eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff294e48640 1 -- 172.21.15.133:0/1746115419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29013f430 con 0x7ff290158110 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff294e48640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff290158110 0x7ff29014e130 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ff28000e9b0 tx=0x7ff28000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff285ffb640 1 -- 172.21.15.133:0/1746115419 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280017070 con 0x7ff290158110 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff2978ad640 1 -- 172.21.15.133:0/1746115419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff290152400 con 0x7ff290158110 2024-04-25T10:11:04.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff285ffb640 1 -- 172.21.15.133:0/1746115419 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28000ca00 con 0x7ff290158110 2024-04-25T10:11:04.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff285ffb640 1 -- 172.21.15.133:0/1746115419 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28000cdd0 con 0x7ff290158110 2024-04-25T10:11:04.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff2978ad640 1 -- 172.21.15.133:0/1746115419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2901529c0 con 0x7ff290158110 2024-04-25T10:11:04.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff2978ad640 1 -- 172.21.15.133:0/1746115419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29006bc20 con 0x7ff290158110 2024-04-25T10:11:04.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff285ffb640 1 -- 172.21.15.133:0/1746115419 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff28001a070 con 0x7ff290158110 2024-04-25T10:11:04.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff285ffb640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff274066f60 0x7ff274069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff285ffb640 1 -- 172.21.15.133:0/1746115419 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff280085580 con 0x7ff290158110 2024-04-25T10:11:04.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff295649640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff274066f60 0x7ff274069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.166+0000 7ff295649640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff274066f60 0x7ff274069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0ae000 tx=0x7ff28c0a6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.170+0000 7ff285ffb640 1 -- 172.21.15.133:0/1746115419 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff280056700 con 0x7ff290158110 2024-04-25T10:11:04.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 -- 172.21.15.133:0/1746115419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff290068ef0 con 0x7ff290158110 2024-04-25T10:11:04.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff285ffb640 1 -- 172.21.15.133:0/1746115419 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+13320 (secure 0 0 0) 0x7ff280056000 con 0x7ff290158110 2024-04-25T10:11:04.336 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:04.336 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":99,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:11:02.996991+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/3580592279":"2024-04-26T10:10:59.850003+0000","172.21.15.133:6835/3580592279":"2024-04-26T10:10:59.850003+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:04.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 -- 172.21.15.133:0/1746115419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff274066f60 msgr2=0x7ff274069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff274066f60 0x7ff274069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0ae000 tx=0x7ff28c0a6040 comp rx=0 tx=0).stop 2024-04-25T10:11:04.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 -- 172.21.15.133:0/1746115419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff290158110 msgr2=0x7ff29014e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff290158110 0x7ff29014e130 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ff28000e9b0 tx=0x7ff28000ee70 comp rx=0 tx=0).stop 2024-04-25T10:11:04.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 -- 172.21.15.133:0/1746115419 shutdown_connections 2024-04-25T10:11:04.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff2901584f0 0x7ff29014e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff274066f60 0x7ff274069420 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff290158110 0x7ff29014e130 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 --2- 172.21.15.133:0/1746115419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff29006e460 0x7ff29013eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 -- 172.21.15.133:0/1746115419 >> 172.21.15.133:0/1746115419 conn(0x7ff290067000 msgr2=0x7ff2901588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 -- 172.21.15.133:0/1746115419 shutdown_connections 2024-04-25T10:11:04.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.334+0000 7ff2978ad640 1 -- 172.21.15.133:0/1746115419 wait complete. 2024-04-25T10:11:04.356 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:04.356 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:04.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22ead61640 1 Processor -- start 2024-04-25T10:11:04.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22ead61640 1 -- start start 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22ead61640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22e406f3b0 0x7f22e40af140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22ead61640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f22e415bb00 0x7f22e415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22ead61640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e4072b40 con 0x7f22e406bca0 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22ead61640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e40af680 con 0x7f22e406f3b0 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22ead61640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e4160160 con 0x7f22e415bb00 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22dbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22e406f3b0 0x7f22e40af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22dbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22e406f3b0 0x7f22e40af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45918/0 (socket says 172.21.15.133:45918) 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22dbfff640 1 -- 172.21.15.133:0/3806738180 learned_addr learned my addr 172.21.15.133:0/3806738180 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22e92fe640 1 --2- 172.21.15.133:0/3806738180 >> v2:172.21.15.162:3300/0 conn(0x7f22e415bb00 0x7f22e415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22dbfff640 1 -- 172.21.15.133:0/3806738180 >> v2:172.21.15.162:3300/0 conn(0x7f22e415bb00 msgr2=0x7f22e415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22dbfff640 1 --2- 172.21.15.133:0/3806738180 >> v2:172.21.15.162:3300/0 conn(0x7f22e415bb00 0x7f22e415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22dbfff640 1 -- 172.21.15.133:0/3806738180 >> v1:172.21.15.162:6789/0 conn(0x7f22e406bca0 legacy=0x7f22e406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.478+0000 7f22dbfff640 1 -- 172.21.15.133:0/3806738180 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4072580 con 0x7f22e406f3b0 2024-04-25T10:11:04.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22dbfff640 1 --2- 172.21.15.133:0/3806738180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e406f3b0 0x7f22e40af140 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f22e415f320 tx=0x7f22d401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea14e3425a587af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22db7fe640 1 -- 172.21.15.133:0/3806738180 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40b0070 con 0x7f22e406f3b0 2024-04-25T10:11:04.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22db7fe640 1 -- 172.21.15.133:0/3806738180 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d40ab600 con 0x7f22e406f3b0 2024-04-25T10:11:04.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22db7fe640 1 -- 172.21.15.133:0/3806738180 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40ab9d0 con 0x7f22e406f3b0 2024-04-25T10:11:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22ead61640 1 -- 172.21.15.133:0/3806738180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e406f3b0 msgr2=0x7f22e40af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22ead61640 1 --2- 172.21.15.133:0/3806738180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e406f3b0 0x7f22e40af140 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f22e415f320 tx=0x7f22d401c400 comp rx=0 tx=0).stop 2024-04-25T10:11:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22ead61640 1 -- 172.21.15.133:0/3806738180 shutdown_connections 2024-04-25T10:11:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22ead61640 1 --2- 172.21.15.133:0/3806738180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e406f3b0 0x7f22e40af140 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22ead61640 1 --2- 172.21.15.133:0/3806738180 >> v2:172.21.15.162:3300/0 conn(0x7f22e415bb00 0x7f22e415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22ead61640 1 -- 172.21.15.133:0/3806738180 >> 172.21.15.133:0/3806738180 conn(0x7f22e4071850 msgr2=0x7f22e4071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22ead61640 1 -- 172.21.15.133:0/3806738180 shutdown_connections 2024-04-25T10:11:04.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22ead61640 1 -- 172.21.15.133:0/3806738180 wait complete. 2024-04-25T10:11:04.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22ead61640 1 Processor -- start 2024-04-25T10:11:04.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.482+0000 7f22ead61640 1 -- start start 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22ead61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e406bca0 0x7f22e41519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22ead61640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22e406f3b0 0x7f22e4158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22ead61640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22e415bb00 0x7f22e4158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22ead61640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40adc20 con 0x7f22e406bca0 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22ead61640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40adaa0 con 0x7f22e415bb00 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22ead61640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4168740 con 0x7f22e406f3b0 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22e8afd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e406bca0 0x7f22e41519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22e8afd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e406bca0 0x7f22e41519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45922/0 (socket says 172.21.15.133:45922) 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22e8afd640 1 -- 172.21.15.133:0/1453748853 learned_addr learned my addr 172.21.15.133:0/1453748853 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22e92fe640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22e415bb00 0x7f22e4158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22dbfff640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22e406f3b0 0x7f22e4158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22e8afd640 1 -- 172.21.15.133:0/1453748853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22e406f3b0 msgr2=0x7f22e4158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22e8afd640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22e406f3b0 0x7f22e4158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22e8afd640 1 -- 172.21.15.133:0/1453748853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22e415bb00 msgr2=0x7f22e4158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22e8afd640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22e415bb00 0x7f22e4158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22e8afd640 1 -- 172.21.15.133:0/1453748853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e414f1c0 con 0x7f22e406bca0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22e8afd640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e406bca0 0x7f22e41519c0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f22e000c370 tx=0x7f22e000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22d97fa640 1 -- 172.21.15.133:0/1453748853 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e00052d0 con 0x7f22e406bca0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22d97fa640 1 -- 172.21.15.133:0/1453748853 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e0005470 con 0x7f22e406bca0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22ead61640 1 -- 172.21.15.133:0/1453748853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e414f4a0 con 0x7f22e406bca0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22d97fa640 1 -- 172.21.15.133:0/1453748853 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0007260 con 0x7f22e406bca0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22ead61640 1 -- 172.21.15.133:0/1453748853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e414fd40 con 0x7f22e406bca0 2024-04-25T10:11:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22ead61640 1 -- 172.21.15.133:0/1453748853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e4067c70 con 0x7f22e406bca0 2024-04-25T10:11:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22d97fa640 1 -- 172.21.15.133:0/1453748853 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22e0007400 con 0x7f22e406bca0 2024-04-25T10:11:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22d97fa640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f60 0x7f22d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22dbfff640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f60 0x7f22d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22d97fa640 1 -- 172.21.15.133:0/1453748853 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f22e0084350 con 0x7f22e406bca0 2024-04-25T10:11:04.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.486+0000 7f22dbfff640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f60 0x7f22d0069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f22d40ad000 tx=0x7f22d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.490+0000 7f22d97fa640 1 -- 172.21.15.133:0/1453748853 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22e00554d0 con 0x7f22e406bca0 2024-04-25T10:11:04.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.650+0000 7f22ead61640 1 -- 172.21.15.133:0/1453748853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22e409de30 con 0x7f22e406bca0 2024-04-25T10:11:04.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.650+0000 7f22d97fa640 1 -- 172.21.15.133:0/1453748853 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22e0054dd0 con 0x7f22e406bca0 2024-04-25T10:11:04.654 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:11:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 -- 172.21.15.133:0/1453748853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f60 msgr2=0x7f22d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f60 0x7f22d0069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f22d40ad000 tx=0x7f22d4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 -- 172.21.15.133:0/1453748853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e406bca0 msgr2=0x7f22e41519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e406bca0 0x7f22e41519c0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f22e000c370 tx=0x7f22e000c830 comp rx=0 tx=0).stop 2024-04-25T10:11:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 -- 172.21.15.133:0/1453748853 shutdown_connections 2024-04-25T10:11:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22e415bb00 0x7f22e4158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22e406f3b0 0x7f22e4158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f60 0x7f22d0069420 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 --2- 172.21.15.133:0/1453748853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22e406bca0 0x7f22e41519c0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 -- 172.21.15.133:0/1453748853 >> 172.21.15.133:0/1453748853 conn(0x7f22e4071850 msgr2=0x7f22e40b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 -- 172.21.15.133:0/1453748853 shutdown_connections 2024-04-25T10:11:04.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.654+0000 7f22ead61640 1 -- 172.21.15.133:0/1453748853 wait complete. 2024-04-25T10:11:04.673 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T10:11:04.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3f09b640 1 Processor -- start 2024-04-25T10:11:04.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3f09b640 1 -- start start 2024-04-25T10:11:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3f09b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c38158110 0x7f5c380b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3f09b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c3806eb70 0x7f5c3815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3f09b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c380cbe60 con 0x7f5c381584f0 2024-04-25T10:11:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3f09b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c380b19b0 con 0x7f5c3806eb70 2024-04-25T10:11:04.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3f09b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3815b360 con 0x7f5c38158110 2024-04-25T10:11:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3d638640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c3806eb70 0x7f5c3815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3d638640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c3806eb70 0x7f5c3815ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45930/0 (socket says 172.21.15.133:45930) 2024-04-25T10:11:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3d638640 1 -- 172.21.15.133:0/3745111965 learned_addr learned my addr 172.21.15.133:0/3745111965 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3ce37640 1 --2- 172.21.15.133:0/3745111965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c38158110 0x7f5c380b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3d638640 1 -- 172.21.15.133:0/3745111965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c38158110 msgr2=0x7f5c380b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3d638640 1 --2- 172.21.15.133:0/3745111965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c38158110 0x7f5c380b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3d638640 1 -- 172.21.15.133:0/3745111965 >> v1:172.21.15.162:6789/0 conn(0x7f5c381584f0 legacy=0x7f5c3806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.786+0000 7f5c3d638640 1 -- 172.21.15.133:0/3745111965 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3815b640 con 0x7f5c3806eb70 2024-04-25T10:11:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c3d638640 1 --2- 172.21.15.133:0/3745111965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3806eb70 0x7f5c3815ad10 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009e50 tx=0x7f5c3001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d248202bb3ed74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c377fe640 1 -- 172.21.15.133:0/3745111965 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c300a58f0 con 0x7f5c3806eb70 2024-04-25T10:11:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c377fe640 1 -- 172.21.15.133:0/3745111965 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c300a5a90 con 0x7f5c3806eb70 2024-04-25T10:11:04.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c377fe640 1 -- 172.21.15.133:0/3745111965 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c300a5e60 con 0x7f5c3806eb70 2024-04-25T10:11:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c3f09b640 1 -- 172.21.15.133:0/3745111965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3806eb70 msgr2=0x7f5c3815ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c3f09b640 1 --2- 172.21.15.133:0/3745111965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3806eb70 0x7f5c3815ad10 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009e50 tx=0x7f5c3001ca00 comp rx=0 tx=0).stop 2024-04-25T10:11:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c3f09b640 1 -- 172.21.15.133:0/3745111965 shutdown_connections 2024-04-25T10:11:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c3f09b640 1 --2- 172.21.15.133:0/3745111965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3806eb70 0x7f5c3815ad10 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c3f09b640 1 --2- 172.21.15.133:0/3745111965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c38158110 0x7f5c380b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c3f09b640 1 -- 172.21.15.133:0/3745111965 >> 172.21.15.133:0/3745111965 conn(0x7f5c38067000 msgr2=0x7f5c3806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c3f09b640 1 -- 172.21.15.133:0/3745111965 shutdown_connections 2024-04-25T10:11:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c3f09b640 1 -- 172.21.15.133:0/3745111965 wait complete. 2024-04-25T10:11:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c3f09b640 1 Processor -- start 2024-04-25T10:11:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.790+0000 7f5c3f09b640 1 -- start start 2024-04-25T10:11:04.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3f09b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c3806eb70 0x7f5c380b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3f09b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c38158110 0x7f5c381505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3f09b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c381584f0 0x7f5c38150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3f09b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380ccca0 con 0x7f5c38158110 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3f09b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380ccb20 con 0x7f5c381584f0 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3f09b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380cce20 con 0x7f5c3806eb70 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c37fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c38158110 0x7f5c381505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c37fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c38158110 0x7f5c381505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45942/0 (socket says 172.21.15.133:45942) 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c37fff640 1 -- 172.21.15.133:0/841436696 learned_addr learned my addr 172.21.15.133:0/841436696 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3ce37640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c3806eb70 0x7f5c380b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3d638640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c381584f0 0x7f5c38150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c37fff640 1 -- 172.21.15.133:0/841436696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c3806eb70 msgr2=0x7f5c380b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c37fff640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c3806eb70 0x7f5c380b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c37fff640 1 -- 172.21.15.133:0/841436696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c381584f0 msgr2=0x7f5c38150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c37fff640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c381584f0 0x7f5c38150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c37fff640 1 -- 172.21.15.133:0/841436696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c38155bc0 con 0x7f5c38158110 2024-04-25T10:11:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c37fff640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c38158110 0x7f5c381505c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00c3a0 tx=0x7f5c2c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c357fa640 1 -- 172.21.15.133:0/841436696 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c0052d0 con 0x7f5c38158110 2024-04-25T10:11:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c357fa640 1 -- 172.21.15.133:0/841436696 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c005470 con 0x7f5c38158110 2024-04-25T10:11:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3f09b640 1 -- 172.21.15.133:0/841436696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c38155ea0 con 0x7f5c38158110 2024-04-25T10:11:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c357fa640 1 -- 172.21.15.133:0/841436696 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c007260 con 0x7f5c38158110 2024-04-25T10:11:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3f09b640 1 -- 172.21.15.133:0/841436696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c380b25b0 con 0x7f5c38158110 2024-04-25T10:11:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3f09b640 1 -- 172.21.15.133:0/841436696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3806bc20 con 0x7f5c38158110 2024-04-25T10:11:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c357fa640 1 -- 172.21.15.133:0/841436696 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c2c007400 con 0x7f5c38158110 2024-04-25T10:11:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c357fa640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c24066f90 0x7f5c24069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c357fa640 1 -- 172.21.15.133:0/841436696 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5c2c084300 con 0x7f5c38158110 2024-04-25T10:11:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3ce37640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c24066f90 0x7f5c24069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.794+0000 7f5c3ce37640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c24066f90 0x7f5c24069450 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009790 tx=0x7f5c28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:04.798+0000 7f5c357fa640 1 -- 172.21.15.133:0/841436696 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c2c055480 con 0x7f5c38158110 2024-04-25T10:11:05.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:05.038+0000 7f5c3f09b640 1 -- 172.21.15.133:0/841436696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f5c38068ef0 con 0x7f5c38158110 2024-04-25T10:11:06.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.018+0000 7f5c357fa640 1 -- 172.21.15.133:0/841436696 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v411) v1 ==== 158+0+0 (secure 0 0 0) 0x7f5c2c054d80 con 0x7f5c38158110 2024-04-25T10:11:06.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 -- 172.21.15.133:0/841436696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c24066f90 msgr2=0x7f5c24069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c24066f90 0x7f5c24069450 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009790 tx=0x7f5c28008040 comp rx=0 tx=0).stop 2024-04-25T10:11:06.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 -- 172.21.15.133:0/841436696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c38158110 msgr2=0x7f5c381505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c38158110 0x7f5c381505c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00c3a0 tx=0x7f5c2c00c860 comp rx=0 tx=0).stop 2024-04-25T10:11:06.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 -- 172.21.15.133:0/841436696 shutdown_connections 2024-04-25T10:11:06.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c381584f0 0x7f5c38150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c24066f90 0x7f5c24069450 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c38158110 0x7f5c381505c0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 --2- 172.21.15.133:0/841436696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c3806eb70 0x7f5c380b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 -- 172.21.15.133:0/841436696 >> 172.21.15.133:0/841436696 conn(0x7f5c38067000 msgr2=0x7f5c380ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 -- 172.21.15.133:0/841436696 shutdown_connections 2024-04-25T10:11:06.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.022+0000 7f5c3f09b640 1 -- 172.21.15.133:0/841436696 wait complete. 2024-04-25T10:11:06.048 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:11:06.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff41600b640 1 Processor -- start 2024-04-25T10:11:06.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff41600b640 1 -- start start 2024-04-25T10:11:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff41600b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4100cf680 0x7ff4100d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff41600b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff4100d0220 0x7ff4100d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff41600b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4100d1530 con 0x7ff41015db30 2024-04-25T10:11:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff41600b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4100d2140 con 0x7ff4100d0220 2024-04-25T10:11:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff41600b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100d42f0 con 0x7ff4100cf680 2024-04-25T10:11:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff40ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff4100d0220 0x7ff4100d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff40ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff4100d0220 0x7ff4100d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36798/0 (socket says 172.21.15.133:36798) 2024-04-25T10:11:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff40ffff640 1 -- 172.21.15.133:0/1533948025 learned_addr learned my addr 172.21.15.133:0/1533948025 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff40effd640 1 --2- 172.21.15.133:0/1533948025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4100cf680 0x7ff4100d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff40ffff640 1 -- 172.21.15.133:0/1533948025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4100cf680 msgr2=0x7ff4100d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff40ffff640 1 --2- 172.21.15.133:0/1533948025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4100cf680 0x7ff4100d25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff40ffff640 1 -- 172.21.15.133:0/1533948025 >> v1:172.21.15.162:6789/0 conn(0x7ff41015db30 legacy=0x7ff4100cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.174+0000 7ff40ffff640 1 -- 172.21.15.133:0/1533948025 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4100d1810 con 0x7ff4100d0220 2024-04-25T10:11:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff40ffff640 1 --2- 172.21.15.133:0/1533948025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4100d0220 0x7ff4100d06b0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff400009870 tx=0x7ff40001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18073732a355680b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff40e7fc640 1 -- 172.21.15.133:0/1533948025 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4000b1070 con 0x7ff4100d0220 2024-04-25T10:11:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff40e7fc640 1 -- 172.21.15.133:0/1533948025 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4000ac630 con 0x7ff4100d0220 2024-04-25T10:11:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff40e7fc640 1 -- 172.21.15.133:0/1533948025 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4000a8050 con 0x7ff4100d0220 2024-04-25T10:11:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff41600b640 1 -- 172.21.15.133:0/1533948025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4100d0220 msgr2=0x7ff4100d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff41600b640 1 --2- 172.21.15.133:0/1533948025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4100d0220 0x7ff4100d06b0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff400009870 tx=0x7ff40001c520 comp rx=0 tx=0).stop 2024-04-25T10:11:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff41600b640 1 -- 172.21.15.133:0/1533948025 shutdown_connections 2024-04-25T10:11:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff41600b640 1 --2- 172.21.15.133:0/1533948025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4100d0220 0x7ff4100d06b0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff41600b640 1 --2- 172.21.15.133:0/1533948025 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4100cf680 0x7ff4100d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff41600b640 1 -- 172.21.15.133:0/1533948025 >> 172.21.15.133:0/1533948025 conn(0x7ff41008af60 msgr2=0x7ff4100d38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff41600b640 1 -- 172.21.15.133:0/1533948025 shutdown_connections 2024-04-25T10:11:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff41600b640 1 -- 172.21.15.133:0/1533948025 wait complete. 2024-04-25T10:11:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff41600b640 1 Processor -- start 2024-04-25T10:11:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.178+0000 7ff41600b640 1 -- start start 2024-04-25T10:11:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff41600b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4100cf680 0x7ff41015e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff41600b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4100d0220 0x7ff4100cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff41600b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41015db30 0x7ff41015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff41600b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100b70a0 con 0x7ff4100d0220 2024-04-25T10:11:06.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff41600b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100b6f20 con 0x7ff41015db30 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff41600b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100b7220 con 0x7ff4100cf680 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4100d0220 0x7ff4100cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4100d0220 0x7ff4100cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36800/0 (socket says 172.21.15.133:36800) 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40effd640 1 -- 172.21.15.133:0/2896743575 learned_addr learned my addr 172.21.15.133:0/2896743575 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40ffff640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41015db30 0x7ff41015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40effd640 1 -- 172.21.15.133:0/2896743575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4100cf680 msgr2=0x7ff41015e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40f7fe640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4100cf680 0x7ff41015e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40effd640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4100cf680 0x7ff41015e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40effd640 1 -- 172.21.15.133:0/2896743575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41015db30 msgr2=0x7ff41015f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40effd640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41015db30 0x7ff41015f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40effd640 1 -- 172.21.15.133:0/2896743575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41015fd30 con 0x7ff4100d0220 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40ffff640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41015db30 0x7ff41015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:06.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40effd640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4100d0220 0x7ff4100cd3e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c7f0 tx=0x7ff3f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff407fff640 1 -- 172.21.15.133:0/2896743575 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8015070 con 0x7ff4100d0220 2024-04-25T10:11:06.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff407fff640 1 -- 172.21.15.133:0/2896743575 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f8004220 con 0x7ff4100d0220 2024-04-25T10:11:06.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff41600b640 1 -- 172.21.15.133:0/2896743575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41015e770 con 0x7ff4100d0220 2024-04-25T10:11:06.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff407fff640 1 -- 172.21.15.133:0/2896743575 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8003620 con 0x7ff4100d0220 2024-04-25T10:11:06.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff41600b640 1 -- 172.21.15.133:0/2896743575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41015f010 con 0x7ff4100d0220 2024-04-25T10:11:06.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff41600b640 1 -- 172.21.15.133:0/2896743575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41015eb60 con 0x7ff4100d0220 2024-04-25T10:11:06.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff407fff640 1 -- 172.21.15.133:0/2896743575 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3f80039d0 con 0x7ff4100d0220 2024-04-25T10:11:06.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff407fff640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3f0066f90 0x7ff3f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40f7fe640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3f0066f90 0x7ff3f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff407fff640 1 -- 172.21.15.133:0/2896743575 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff3f8084f60 con 0x7ff4100d0220 2024-04-25T10:11:06.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.182+0000 7ff40f7fe640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3f0066f90 0x7ff3f0069450 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff408007bf0 tx=0x7ff408007240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.190+0000 7ff407fff640 1 -- 172.21.15.133:0/2896743575 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3f8056030 con 0x7ff4100d0220 2024-04-25T10:11:06.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.350+0000 7ff41600b640 1 -- 172.21.15.133:0/2896743575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff4100d0ed0 con 0x7ff4100d0220 2024-04-25T10:11:06.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.354+0000 7ff407fff640 1 -- 172.21.15.133:0/2896743575 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff3f8055930 con 0x7ff4100d0220 2024-04-25T10:11:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 -- 172.21.15.133:0/2896743575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3f0066f90 msgr2=0x7ff3f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3f0066f90 0x7ff3f0069450 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff408007bf0 tx=0x7ff408007240 comp rx=0 tx=0).stop 2024-04-25T10:11:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 -- 172.21.15.133:0/2896743575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4100d0220 msgr2=0x7ff4100cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4100d0220 0x7ff4100cd3e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c7f0 tx=0x7ff3f800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 -- 172.21.15.133:0/2896743575 shutdown_connections 2024-04-25T10:11:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff41015db30 0x7ff41015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3f0066f90 0x7ff3f0069450 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4100d0220 0x7ff4100cd3e0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 --2- 172.21.15.133:0/2896743575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4100cf680 0x7ff41015e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 -- 172.21.15.133:0/2896743575 >> 172.21.15.133:0/2896743575 conn(0x7ff41008af60 msgr2=0x7ff4100b80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 -- 172.21.15.133:0/2896743575 shutdown_connections 2024-04-25T10:11:06.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.358+0000 7ff41600b640 1 -- 172.21.15.133:0/2896743575 wait complete. 2024-04-25T10:11:06.377 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:11:06.377 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:06.417 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:06.480 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:11:06.480 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:11:06.480 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:06.485 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:06.485 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:11:06.486 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:06.491 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:06.491 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:11:06.491 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:06.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:06.532 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:11:06.532 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:06.539 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:06.539 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:11:06.539 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:06.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.702+0000 7f313f4e0640 1 Processor -- start 2024-04-25T10:11:06.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.702+0000 7f313f4e0640 1 -- start start 2024-04-25T10:11:06.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313f4e0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f313815c9a0 0x7f31380c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313f4e0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31380ce330 0x7f31380ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313f4e0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31380c61d0 con 0x7f31380b1f70 2024-04-25T10:11:06.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313f4e0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31380cecc0 con 0x7f313815c9a0 2024-04-25T10:11:06.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313f4e0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31380c64b0 con 0x7f31380ce330 2024-04-25T10:11:06.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313d27c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f313815c9a0 0x7f31380c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313d27c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f313815c9a0 0x7f31380c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36812/0 (socket says 172.21.15.133:36812) 2024-04-25T10:11:06.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313ca7b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f31380b1f70 0x7f313806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34724/0 (socket says 172.21.15.133:34724) 2024-04-25T10:11:06.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313d27c640 1 -- 172.21.15.133:0/4280959801 learned_addr learned my addr 172.21.15.133:0/4280959801 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:06.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313d27c640 1 -- 172.21.15.133:0/4280959801 >> v2:172.21.15.162:3300/0 conn(0x7f31380ce330 msgr2=0x7f31380ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313d27c640 1 --2- 172.21.15.133:0/4280959801 >> v2:172.21.15.162:3300/0 conn(0x7f31380ce330 0x7f31380ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313d27c640 1 -- 172.21.15.133:0/4280959801 >> v1:172.21.15.133:6789/0 conn(0x7f31380b1f70 legacy=0x7f313806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313d27c640 1 -- 172.21.15.133:0/4280959801 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313806d4f0 con 0x7f313815c9a0 2024-04-25T10:11:06.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.706+0000 7f313d27c640 1 --2- 172.21.15.133:0/4280959801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313815c9a0 0x7f31380c5980 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f31380cc0f0 tx=0x7f313401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426ed79bec603030 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f312ffff640 1 -- 172.21.15.133:0/4280959801 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31340b0070 con 0x7f313815c9a0 2024-04-25T10:11:06.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f312ffff640 1 -- 172.21.15.133:0/4280959801 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31340ab600 con 0x7f313815c9a0 2024-04-25T10:11:06.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f312ffff640 1 -- 172.21.15.133:0/4280959801 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31340ab9d0 con 0x7f313815c9a0 2024-04-25T10:11:06.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 -- 172.21.15.133:0/4280959801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313815c9a0 msgr2=0x7f31380c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 --2- 172.21.15.133:0/4280959801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313815c9a0 0x7f31380c5980 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f31380cc0f0 tx=0x7f313401c520 comp rx=0 tx=0).stop 2024-04-25T10:11:06.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 -- 172.21.15.133:0/4280959801 shutdown_connections 2024-04-25T10:11:06.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 --2- 172.21.15.133:0/4280959801 >> v2:172.21.15.162:3300/0 conn(0x7f31380ce330 0x7f31380ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 --2- 172.21.15.133:0/4280959801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313815c9a0 0x7f31380c5980 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 -- 172.21.15.133:0/4280959801 >> 172.21.15.133:0/4280959801 conn(0x7f3138067000 msgr2=0x7f313806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 -- 172.21.15.133:0/4280959801 shutdown_connections 2024-04-25T10:11:06.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 -- 172.21.15.133:0/4280959801 wait complete. 2024-04-25T10:11:06.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 Processor -- start 2024-04-25T10:11:06.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 -- start start 2024-04-25T10:11:06.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31380b1f70 0x7f31381377b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31380ce330 0x7f31381f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313815c9a0 0x7f3138151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313815dad0 con 0x7f313815c9a0 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313815d950 con 0x7f31380b1f70 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313f4e0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313815dc50 con 0x7f31380ce330 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313da7d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313815c9a0 0x7f3138151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313da7d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313815c9a0 0x7f3138151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36814/0 (socket says 172.21.15.133:36814) 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313da7d640 1 -- 172.21.15.133:0/1013127736 learned_addr learned my addr 172.21.15.133:0/1013127736 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313d27c640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31380b1f70 0x7f31381377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313ca7b640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31380ce330 0x7f31381f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313da7d640 1 -- 172.21.15.133:0/1013127736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31380ce330 msgr2=0x7f31381f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313da7d640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31380ce330 0x7f31381f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313da7d640 1 -- 172.21.15.133:0/1013127736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31380b1f70 msgr2=0x7f31381377b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313da7d640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31380b1f70 0x7f31381377b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313da7d640 1 -- 172.21.15.133:0/1013127736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31380c8740 con 0x7f313815c9a0 2024-04-25T10:11:06.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f313da7d640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313815c9a0 0x7f3138151a50 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f313000b970 tx=0x7f313000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.710+0000 7f312dffb640 1 -- 172.21.15.133:0/1013127736 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130019070 con 0x7f313815c9a0 2024-04-25T10:11:06.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.714+0000 7f312dffb640 1 -- 172.21.15.133:0/1013127736 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313000cb60 con 0x7f313815c9a0 2024-04-25T10:11:06.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.714+0000 7f313f4e0640 1 -- 172.21.15.133:0/1013127736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31380c89e0 con 0x7f313815c9a0 2024-04-25T10:11:06.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.714+0000 7f312dffb640 1 -- 172.21.15.133:0/1013127736 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130014440 con 0x7f313815c9a0 2024-04-25T10:11:06.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.714+0000 7f313f4e0640 1 -- 172.21.15.133:0/1013127736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31380c8fa0 con 0x7f313815c9a0 2024-04-25T10:11:06.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.714+0000 7f313f4e0640 1 -- 172.21.15.133:0/1013127736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3138068ef0 con 0x7f313815c9a0 2024-04-25T10:11:06.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.714+0000 7f312dffb640 1 -- 172.21.15.133:0/1013127736 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f313001c070 con 0x7f313815c9a0 2024-04-25T10:11:06.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.714+0000 7f312dffb640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f311c066f60 0x7f311c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.714+0000 7f312dffb640 1 -- 172.21.15.133:0/1013127736 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3130084bd0 con 0x7f313815c9a0 2024-04-25T10:11:06.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.714+0000 7f313d27c640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f311c066f60 0x7f311c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.714+0000 7f313d27c640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f311c066f60 0x7f311c069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f31340ad000 tx=0x7f31340a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.718+0000 7f312dffb640 1 -- 172.21.15.133:0/1013127736 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3130055d50 con 0x7f313815c9a0 2024-04-25T10:11:06.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.906+0000 7f313f4e0640 1 -- 172.21.15.133:0/1013127736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31381535c0 con 0x7f313815c9a0 2024-04-25T10:11:06.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f312dffb640 1 -- 172.21.15.133:0/1013127736 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f3130055650 con 0x7f313815c9a0 2024-04-25T10:11:06.911 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:06.912 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3089710291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3089710291},{"type":"v1","addr":"172.21.15.162:6834","nonce":3089710291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6163,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1390003376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":1390003376},{"type":"v1","addr":"172.21.15.162:6835","nonce":1390003376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7866,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1134679682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1134679682},{"type":"v1","addr":"172.21.15.133:6836","nonce":1134679682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/879401999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":879401999},{"type":"v1","addr":"172.21.15.133:6837","nonce":879401999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[]} 2024-04-25T10:11:06.912 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 60 2024-04-25T10:11:06.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 -- 172.21.15.133:0/1013127736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f311c066f60 msgr2=0x7f311c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f311c066f60 0x7f311c069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f31340ad000 tx=0x7f31340a6000 comp rx=0 tx=0).stop 2024-04-25T10:11:06.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 -- 172.21.15.133:0/1013127736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313815c9a0 msgr2=0x7f3138151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313815c9a0 0x7f3138151a50 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f313000b970 tx=0x7f313000be30 comp rx=0 tx=0).stop 2024-04-25T10:11:06.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 -- 172.21.15.133:0/1013127736 shutdown_connections 2024-04-25T10:11:06.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f311c066f60 0x7f311c069420 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f313815c9a0 0x7f3138151a50 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31380ce330 0x7f31381f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 --2- 172.21.15.133:0/1013127736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31380b1f70 0x7f31381377b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 -- 172.21.15.133:0/1013127736 >> 172.21.15.133:0/1013127736 conn(0x7f3138067000 msgr2=0x7f3138149750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 -- 172.21.15.133:0/1013127736 shutdown_connections 2024-04-25T10:11:06.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:06.910+0000 7f313f4e0640 1 -- 172.21.15.133:0/1013127736 wait complete. 2024-04-25T10:11:06.929 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:11:06.929 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:11:06.929 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:11:06.929 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:06.930 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:11:06.930 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:11:06.930 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:11:06.930 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:06.930 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:11:06.930 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:11:06.931 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:11:06.931 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:06.931 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:11:06.931 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:11:06.931 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:11:06.931 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:06.939 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:11:06.939 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:11:06.941 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:11:06.941 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:11:06.946 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:11:06.946 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:11:06.947 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:11:06.947 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:11:06.984 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:11:06.984 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:06.985 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:11:06.985 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:06.990 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:11:06.991 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:06.992 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:11:06.992 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:06.992 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:11:07.044 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:11:07.047 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:11:07.062 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:11:07.062 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:11:07.062 INFO:tasks.ceph.mds.d.smithi162.stderr:2024-04-25T10:11:07.061+0000 7f416658d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T10:11:07.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d2d6a7640 1 Processor -- start 2024-04-25T10:11:07.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d2d6a7640 1 -- start start 2024-04-25T10:11:07.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d2d6a7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 0x7f8d280cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d2d6a7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8d280cde40 0x7f8d2806ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d2d6a7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2806dfe0 con 0x7f8d2815d880 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d2d6a7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d280ce2b0 con 0x7f8d280cde40 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d2d6a7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2806ebf0 con 0x7f8d28161520 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d277fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8d280cde40 0x7f8d2806ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d277fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8d280cde40 0x7f8d2806ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36844/0 (socket says 172.21.15.133:36844) 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d26ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8d2815d880 0x7f8d2806d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34758/0 (socket says 172.21.15.133:34758) 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d277fe640 1 -- 172.21.15.133:0/1703619194 learned_addr learned my addr 172.21.15.133:0/1703619194 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d1ffff640 1 --2- 172.21.15.133:0/1703619194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 0x7f8d280cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d277fe640 1 -- 172.21.15.133:0/1703619194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 msgr2=0x7f8d280cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d277fe640 1 --2- 172.21.15.133:0/1703619194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 0x7f8d280cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d277fe640 1 -- 172.21.15.133:0/1703619194 >> v1:172.21.15.133:6789/0 conn(0x7f8d2815d880 legacy=0x7f8d2806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d277fe640 1 -- 172.21.15.133:0/1703619194 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2806e2c0 con 0x7f8d280cde40 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d1ffff640 1 --2- 172.21.15.133:0/1703619194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 0x7f8d280cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d277fe640 1 --2- 172.21.15.133:0/1703619194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d280cde40 0x7f8d2806ae40 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009e80 tx=0x7f8d2001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7953bb8f6b306aad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d267fc640 1 -- 172.21.15.133:0/1703619194 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200a58a0 con 0x7f8d280cde40 2024-04-25T10:11:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d267fc640 1 -- 172.21.15.133:0/1703619194 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d200a5a40 con 0x7f8d280cde40 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.098+0000 7f8d267fc640 1 -- 172.21.15.133:0/1703619194 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200a5e10 con 0x7f8d280cde40 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/1703619194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d280cde40 msgr2=0x7f8d2806ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 --2- 172.21.15.133:0/1703619194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d280cde40 0x7f8d2806ae40 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009e80 tx=0x7f8d2001c9e0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/1703619194 shutdown_connections 2024-04-25T10:11:07.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 --2- 172.21.15.133:0/1703619194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d280cde40 0x7f8d2806ae40 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 --2- 172.21.15.133:0/1703619194 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 0x7f8d280cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/1703619194 >> 172.21.15.133:0/1703619194 conn(0x7f8d280cc7b0 msgr2=0x7f8d280ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:07.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/1703619194 shutdown_connections 2024-04-25T10:11:07.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/1703619194 wait complete. 2024-04-25T10:11:07.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 Processor -- start 2024-04-25T10:11:07.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- start start 2024-04-25T10:11:07.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d280cde40 0x7f8d281575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2815d880 0x7f8d2814c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 0x7f8d28158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2806bd10 con 0x7f8d2815d880 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2806bb90 con 0x7f8d280cde40 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2806be90 con 0x7f8d28161520 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2815d880 0x7f8d2814c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2815d880 0x7f8d2814c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36858/0 (socket says 172.21.15.133:36858) 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1ffff640 1 -- 172.21.15.133:0/958595655 learned_addr learned my addr 172.21.15.133:0/958595655 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d277fe640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 0x7f8d28158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1ffff640 1 -- 172.21.15.133:0/958595655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 msgr2=0x7f8d28158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1ffff640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 0x7f8d28158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1ffff640 1 -- 172.21.15.133:0/958595655 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d280cde40 msgr2=0x7f8d281575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1ffff640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d280cde40 0x7f8d281575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1ffff640 1 -- 172.21.15.133:0/958595655 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2814cdb0 con 0x7f8d2815d880 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d277fe640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 0x7f8d28158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:07.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1ffff640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2815d880 0x7f8d2814c870 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000c7c0 tx=0x7f8d1000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1f7fe640 1 -- 172.21.15.133:0/958595655 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10007a30 con 0x7f8d2815d880 2024-04-25T10:11:07.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1f7fe640 1 -- 172.21.15.133:0/958595655 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d10007bd0 con 0x7f8d2815d880 2024-04-25T10:11:07.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/958595655 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d28157c50 con 0x7f8d2815d880 2024-04-25T10:11:07.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d1f7fe640 1 -- 172.21.15.133:0/958595655 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d100113f0 con 0x7f8d2815d880 2024-04-25T10:11:07.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/958595655 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d28158210 con 0x7f8d2815d880 2024-04-25T10:11:07.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.102+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/958595655 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d28067c70 con 0x7f8d2815d880 2024-04-25T10:11:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.106+0000 7f8d1f7fe640 1 -- 172.21.15.133:0/958595655 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d10011590 con 0x7f8d2815d880 2024-04-25T10:11:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.106+0000 7f8d1f7fe640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d08066f60 0x7f8d08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.106+0000 7f8d1f7fe640 1 -- 172.21.15.133:0/958595655 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8d10085030 con 0x7f8d2815d880 2024-04-25T10:11:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.106+0000 7f8d26ffd640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d08066f60 0x7f8d08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.106+0000 7f8d26ffd640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d08066f60 0x7f8d08069420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8d28162640 tx=0x7f8d180079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.110+0000 7f8d1f7fe640 1 -- 172.21.15.133:0/958595655 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d10015020 con 0x7f8d2815d880 2024-04-25T10:11:07.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T10:11:07.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T10:11:07.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:11:07.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:11:07.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:07.278+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/958595655 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8d28066bc0 con 0x7f8d2815d880 2024-04-25T10:11:08.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.042+0000 7f8d1f7fe640 1 -- 172.21.15.133:0/958595655 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v101) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8d10056240 con 0x7f8d2815d880 2024-04-25T10:11:08.045 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:11:08.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/958595655 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d08066f60 msgr2=0x7f8d08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d08066f60 0x7f8d08069420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8d28162640 tx=0x7f8d180079b0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/958595655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2815d880 msgr2=0x7f8d2814c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2815d880 0x7f8d2814c870 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000c7c0 tx=0x7f8d1000cc80 comp rx=0 tx=0).stop 2024-04-25T10:11:08.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/958595655 shutdown_connections 2024-04-25T10:11:08.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8d28161520 0x7f8d28158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8d08066f60 0x7f8d08069420 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2815d880 0x7f8d2814c870 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 --2- 172.21.15.133:0/958595655 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8d280cde40 0x7f8d281575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/958595655 >> 172.21.15.133:0/958595655 conn(0x7f8d280cc7b0 msgr2=0x7f8d2806d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/958595655 shutdown_connections 2024-04-25T10:11:08.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.046+0000 7f8d2d6a7640 1 -- 172.21.15.133:0/958595655 wait complete. 2024-04-25T10:11:08.069 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:11:08.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f0586541640 1 Processor -- start 2024-04-25T10:11:08.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f0586541640 1 -- start start 2024-04-25T10:11:08.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f0586541640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f058006dca0 0x7f05800b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f0586541640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f058006bda0 0x7f05800b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f0586541640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f05800b20a0 0x7f05800999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f0586541640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05800d0660 con 0x7f058006dca0 2024-04-25T10:11:08.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f0586541640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05800cfdc0 con 0x7f05800b20a0 2024-04-25T10:11:08.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f0586541640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05800d00a0 con 0x7f058006bda0 2024-04-25T10:11:08.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f057ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f058006dca0 0x7f05800b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f057ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f058006dca0 0x7f05800b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36866/0 (socket says 172.21.15.133:36866) 2024-04-25T10:11:08.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f057ffff640 1 -- 172.21.15.133:0/2655102143 learned_addr learned my addr 172.21.15.133:0/2655102143 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:08.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f057ffff640 1 -- 172.21.15.133:0/2655102143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f058006bda0 msgr2=0x7f05800b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f057ffff640 1 --2- 172.21.15.133:0/2655102143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f058006bda0 0x7f05800b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f0584ade640 1 --2- 172.21.15.133:0/2655102143 >> v2:172.21.15.162:3300/0 conn(0x7f05800b20a0 0x7f05800999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f057ffff640 1 -- 172.21.15.133:0/2655102143 >> v2:172.21.15.162:3300/0 conn(0x7f05800b20a0 msgr2=0x7f05800999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f057ffff640 1 --2- 172.21.15.133:0/2655102143 >> v2:172.21.15.162:3300/0 conn(0x7f05800b20a0 0x7f05800999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.182+0000 7f057ffff640 1 -- 172.21.15.133:0/2655102143 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800d0380 con 0x7f058006dca0 2024-04-25T10:11:08.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f057ffff640 1 --2- 172.21.15.133:0/2655102143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f058006dca0 0x7f05800b1620 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f05800cbc80 tx=0x7f057001c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50d7c873ca3232b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f057effd640 1 -- 172.21.15.133:0/2655102143 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05700b1070 con 0x7f058006dca0 2024-04-25T10:11:08.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f057effd640 1 -- 172.21.15.133:0/2655102143 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05700ac440 con 0x7f058006dca0 2024-04-25T10:11:08.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f057effd640 1 -- 172.21.15.133:0/2655102143 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05700ac810 con 0x7f058006dca0 2024-04-25T10:11:08.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f0586541640 1 -- 172.21.15.133:0/2655102143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f058006dca0 msgr2=0x7f05800b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f0586541640 1 --2- 172.21.15.133:0/2655102143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f058006dca0 0x7f05800b1620 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f05800cbc80 tx=0x7f057001c140 comp rx=0 tx=0).stop 2024-04-25T10:11:08.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f0586541640 1 -- 172.21.15.133:0/2655102143 shutdown_connections 2024-04-25T10:11:08.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f0586541640 1 --2- 172.21.15.133:0/2655102143 >> v2:172.21.15.162:3300/0 conn(0x7f05800b20a0 0x7f05800999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f0586541640 1 --2- 172.21.15.133:0/2655102143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f058006bda0 0x7f05800b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f0586541640 1 --2- 172.21.15.133:0/2655102143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f058006dca0 0x7f05800b1620 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f0586541640 1 -- 172.21.15.133:0/2655102143 >> 172.21.15.133:0/2655102143 conn(0x7f05800b31a0 msgr2=0x7f05800b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f0586541640 1 -- 172.21.15.133:0/2655102143 shutdown_connections 2024-04-25T10:11:08.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f0586541640 1 -- 172.21.15.133:0/2655102143 wait complete. 2024-04-25T10:11:08.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f0586541640 1 Processor -- start 2024-04-25T10:11:08.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.186+0000 7f0586541640 1 -- start start 2024-04-25T10:11:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0586541640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f058006bda0 0x7f0580150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0586541640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f058006dca0 0x7f05800ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0586541640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05800b20a0 0x7f0580158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0586541640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800cc5e0 con 0x7f05800b20a0 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0586541640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800cc460 con 0x7f058006bda0 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0586541640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05800cc760 con 0x7f058006dca0 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0584ade640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05800b20a0 0x7f0580158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0584ade640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05800b20a0 0x7f0580158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36878/0 (socket says 172.21.15.133:36878) 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0584ade640 1 -- 172.21.15.133:0/2042982227 learned_addr learned my addr 172.21.15.133:0/2042982227 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f057ffff640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f058006bda0 0x7f0580150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f057f7fe640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f058006dca0 0x7f05800ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0584ade640 1 -- 172.21.15.133:0/2042982227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f058006dca0 msgr2=0x7f05800ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0584ade640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f058006dca0 0x7f05800ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0584ade640 1 -- 172.21.15.133:0/2042982227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f058006bda0 msgr2=0x7f0580150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0584ade640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f058006bda0 0x7f0580150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0584ade640 1 -- 172.21.15.133:0/2042982227 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800ce350 con 0x7f05800b20a0 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0584ade640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05800b20a0 0x7f0580158660 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f057800cc30 tx=0x7f057800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f057cff9640 1 -- 172.21.15.133:0/2042982227 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057800ce40 con 0x7f05800b20a0 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f057cff9640 1 -- 172.21.15.133:0/2042982227 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05780039a0 con 0x7f05800b20a0 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0586541640 1 -- 172.21.15.133:0/2042982227 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05800ce630 con 0x7f05800b20a0 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f057cff9640 1 -- 172.21.15.133:0/2042982227 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0578003d50 con 0x7f05800b20a0 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0586541640 1 -- 172.21.15.133:0/2042982227 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05800cebf0 con 0x7f05800b20a0 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f0586541640 1 -- 172.21.15.133:0/2042982227 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0580067c70 con 0x7f05800b20a0 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f057cff9640 1 -- 172.21.15.133:0/2042982227 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0578003f50 con 0x7f05800b20a0 2024-04-25T10:11:08.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f057cff9640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f056c066f60 0x7f056c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f057cff9640 1 -- 172.21.15.133:0/2042982227 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0578085360 con 0x7f05800b20a0 2024-04-25T10:11:08.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f057ffff640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f056c066f60 0x7f056c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.190+0000 7f057ffff640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f056c066f60 0x7f056c069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f05700ae000 tx=0x7f0570008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.194+0000 7f057cff9640 1 -- 172.21.15.133:0/2042982227 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0578014030 con 0x7f05800b20a0 2024-04-25T10:11:08.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 -- 172.21.15.133:0/2042982227 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f05800af5a0 con 0x7f05800b20a0 2024-04-25T10:11:08.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f057cff9640 1 -- 172.21.15.133:0/2042982227 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0578056530 con 0x7f05800b20a0 2024-04-25T10:11:08.401 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:08.401 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:11:08.401 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:11:08.401 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.401 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:11:08.401 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:11:08.401 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.401 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:11:08.401 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:11:08.401 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:11:08.401 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.402 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.403 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:11:08.404 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:11:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:11:08.406 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.406 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:08.406 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:11:08.406 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.406 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.406 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.406 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.406 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:11:08.406 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:11:08.406 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.406 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.407 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.408 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:08.409 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:11:08.409 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.409 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.409 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.409 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.409 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:11:08.409 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:11:08.409 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.409 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:08.409 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:11:08.410 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.410 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.410 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.410 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.410 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:11:08.410 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:11:08.410 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.410 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:11:08.410 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:11:08.410 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:11:08.411 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:11:08.412 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.412 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:11:08.412 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:11:08.412 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:11:08.412 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:11:08.412 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.412 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.412 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.412 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:11:08.412 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:11:08.412 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.413 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:11:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:11:08.415 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:11:08.415 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:11:08.415 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:11:08.415 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:11:08.415 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.415 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:08.415 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:11:08.415 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:11:08.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 -- 172.21.15.133:0/2042982227 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f056c066f60 msgr2=0x7f056c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f056c066f60 0x7f056c069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f05700ae000 tx=0x7f0570008040 comp rx=0 tx=0).stop 2024-04-25T10:11:08.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 -- 172.21.15.133:0/2042982227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05800b20a0 msgr2=0x7f0580158660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05800b20a0 0x7f0580158660 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f057800cc30 tx=0x7f057800a5c0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 -- 172.21.15.133:0/2042982227 shutdown_connections 2024-04-25T10:11:08.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f056c066f60 0x7f056c069420 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05800b20a0 0x7f0580158660 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f058006dca0 0x7f05800ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 --2- 172.21.15.133:0/2042982227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f058006bda0 0x7f0580150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 -- 172.21.15.133:0/2042982227 >> 172.21.15.133:0/2042982227 conn(0x7f05800b31a0 msgr2=0x7f05800afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 -- 172.21.15.133:0/2042982227 shutdown_connections 2024-04-25T10:11:08.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.398+0000 7f0586541640 1 -- 172.21.15.133:0/2042982227 wait complete. 2024-04-25T10:11:08.419 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:08.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.530+0000 7f5057adf640 1 Processor -- start 2024-04-25T10:11:08.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.530+0000 7f5057adf640 1 -- start start 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f5057adf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505006eb70 0x7f50500c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f5057adf640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f505015a9d0 0x7f50500cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f5057adf640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50500cc5a0 con 0x7f505015a5f0 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f5057adf640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50500cc880 con 0x7f505015a9d0 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f5057adf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50500ccb60 con 0x7f505006eb70 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f505587b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f505015a5f0 0x7f505006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34770/0 (socket says 172.21.15.133:34770) 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f505587b640 1 -- 172.21.15.133:0/1349483955 learned_addr learned my addr 172.21.15.133:0/1349483955 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f505607c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f505015a9d0 0x7f50500cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f505507a640 1 --2- 172.21.15.133:0/1349483955 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505006eb70 0x7f50500c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f505607c640 1 -- 172.21.15.133:0/1349483955 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505006eb70 msgr2=0x7f50500c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f505607c640 1 --2- 172.21.15.133:0/1349483955 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505006eb70 0x7f50500c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f505607c640 1 -- 172.21.15.133:0/1349483955 >> v1:172.21.15.133:6789/0 conn(0x7f505015a5f0 legacy=0x7f505006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f5054879640 1 -- 172.21.15.133:0/1349483955 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 53826275 0 0) 0x7f50500cc5a0 con 0x7f505015a5f0 2024-04-25T10:11:08.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f505607c640 1 -- 172.21.15.133:0/1349483955 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500cce40 con 0x7f505015a9d0 2024-04-25T10:11:08.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f505607c640 1 --2- 172.21.15.133:0/1349483955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505015a9d0 0x7f50500cbe50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5048009d90 tx=0x7f504801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0624cc6a9d916 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f5054879640 1 -- 172.21.15.133:0/1349483955 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50480b2070 con 0x7f505015a9d0 2024-04-25T10:11:08.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f5054879640 1 -- 172.21.15.133:0/1349483955 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50480a5730 con 0x7f505015a9d0 2024-04-25T10:11:08.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.534+0000 7f5054879640 1 -- 172.21.15.133:0/1349483955 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50480a5b00 con 0x7f505015a9d0 2024-04-25T10:11:08.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 -- 172.21.15.133:0/1349483955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505015a9d0 msgr2=0x7f50500cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 --2- 172.21.15.133:0/1349483955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505015a9d0 0x7f50500cbe50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5048009d90 tx=0x7f504801c970 comp rx=0 tx=0).stop 2024-04-25T10:11:08.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 -- 172.21.15.133:0/1349483955 shutdown_connections 2024-04-25T10:11:08.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 --2- 172.21.15.133:0/1349483955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505015a9d0 0x7f50500cbe50 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 --2- 172.21.15.133:0/1349483955 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505006eb70 0x7f50500c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 -- 172.21.15.133:0/1349483955 >> 172.21.15.133:0/1349483955 conn(0x7f5050067000 msgr2=0x7f505006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 -- 172.21.15.133:0/1349483955 shutdown_connections 2024-04-25T10:11:08.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 -- 172.21.15.133:0/1349483955 wait complete. 2024-04-25T10:11:08.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 Processor -- start 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 -- start start 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505006eb70 0x7f5050143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f505015a5f0 0x7f5050163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505015a9d0 0x7f50500c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505006def0 con 0x7f505006eb70 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505006dd70 con 0x7f505015a5f0 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505006e070 con 0x7f505015a9d0 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f505587b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505006eb70 0x7f5050143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f505587b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505006eb70 0x7f5050143fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36888/0 (socket says 172.21.15.133:36888) 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f505587b640 1 -- 172.21.15.133:0/174312944 learned_addr learned my addr 172.21.15.133:0/174312944 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f505507a640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f505015a5f0 0x7f5050163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f505607c640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505015a9d0 0x7f50500c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f505587b640 1 -- 172.21.15.133:0/174312944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505015a9d0 msgr2=0x7f50500c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f505587b640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505015a9d0 0x7f50500c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f505587b640 1 -- 172.21.15.133:0/174312944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f505015a5f0 msgr2=0x7f5050163ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f505587b640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f505015a5f0 0x7f5050163ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f505587b640 1 -- 172.21.15.133:0/174312944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50501645b0 con 0x7f505006eb70 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f505587b640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505006eb70 0x7f5050143fb0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f504c0011f0 tx=0x7f504c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f50467fc640 1 -- 172.21.15.133:0/174312944 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c018070 con 0x7f505006eb70 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f50467fc640 1 -- 172.21.15.133:0/174312944 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c0074e0 con 0x7f505006eb70 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 -- 172.21.15.133:0/174312944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5050144930 con 0x7f505006eb70 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f50467fc640 1 -- 172.21.15.133:0/174312944 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c004a10 con 0x7f505006eb70 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.538+0000 7f5057adf640 1 -- 172.21.15.133:0/174312944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5050155fb0 con 0x7f505006eb70 2024-04-25T10:11:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.542+0000 7f5057adf640 1 -- 172.21.15.133:0/174312944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5050144b00 con 0x7f505006eb70 2024-04-25T10:11:08.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.542+0000 7f50467fc640 1 -- 172.21.15.133:0/174312944 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f504c01b070 con 0x7f505006eb70 2024-04-25T10:11:08.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.542+0000 7f50467fc640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5034066f60 0x7f5034069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.542+0000 7f50467fc640 1 -- 172.21.15.133:0/174312944 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f504c084230 con 0x7f505006eb70 2024-04-25T10:11:08.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.542+0000 7f505507a640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5034066f60 0x7f5034069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.542+0000 7f505507a640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5034066f60 0x7f5034069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5050155600 tx=0x7f5040008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.546+0000 7f50467fc640 1 -- 172.21.15.133:0/174312944 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f504c055400 con 0x7f505006eb70 2024-04-25T10:11:08.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.702+0000 7f5057adf640 1 -- 172.21.15.133:0/174312944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5050068ef0 con 0x7f505006eb70 2024-04-25T10:11:08.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f50467fc640 1 -- 172.21.15.133:0/174312944 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+13190 (secure 0 0 0) 0x7f504c054d00 con 0x7f505006eb70 2024-04-25T10:11:08.708 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:08.708 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":103,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:11:08.057249+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 -- 172.21.15.133:0/174312944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5034066f60 msgr2=0x7f5034069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5034066f60 0x7f5034069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5050155600 tx=0x7f5040008040 comp rx=0 tx=0).stop 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 -- 172.21.15.133:0/174312944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505006eb70 msgr2=0x7f5050143fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505006eb70 0x7f5050143fb0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f504c0011f0 tx=0x7f504c00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 -- 172.21.15.133:0/174312944 shutdown_connections 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505015a9d0 0x7f50500c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f505015a5f0 0x7f5050163ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5034066f60 0x7f5034069420 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 --2- 172.21.15.133:0/174312944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505006eb70 0x7f5050143fb0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 -- 172.21.15.133:0/174312944 >> 172.21.15.133:0/174312944 conn(0x7f5050067000 msgr2=0x7f50500b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 -- 172.21.15.133:0/174312944 shutdown_connections 2024-04-25T10:11:08.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.706+0000 7f5057adf640 1 -- 172.21.15.133:0/174312944 wait complete. 2024-04-25T10:11:08.728 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:08.728 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:08.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feec544e640 1 Processor -- start 2024-04-25T10:11:08.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feec544e640 1 -- start start 2024-04-25T10:11:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feec544e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feec015eb40 0x7feec00b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feec544e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feec00aebc0 0x7feec00af010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feec544e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec01471f0 con 0x7feec006e8b0 2024-04-25T10:11:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feec544e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec00af550 con 0x7feec015eb40 2024-04-25T10:11:08.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feec544e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec0147490 con 0x7feec00aebc0 2024-04-25T10:11:08.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feebe7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feec015eb40 0x7feec00b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feebeffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7feec006e8b0 0x7feec0146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34786/0 (socket says 172.21.15.133:34786) 2024-04-25T10:11:08.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feebe7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feec015eb40 0x7feec00b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36892/0 (socket says 172.21.15.133:36892) 2024-04-25T10:11:08.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feebeffd640 1 -- 172.21.15.133:0/444814514 learned_addr learned my addr 172.21.15.133:0/444814514 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:08.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feebf7fe640 1 --2- 172.21.15.133:0/444814514 >> v2:172.21.15.162:3300/0 conn(0x7feec00aebc0 0x7feec00af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feebe7fc640 1 -- 172.21.15.133:0/444814514 >> v2:172.21.15.162:3300/0 conn(0x7feec00aebc0 msgr2=0x7feec00af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feebe7fc640 1 --2- 172.21.15.133:0/444814514 >> v2:172.21.15.162:3300/0 conn(0x7feec00aebc0 0x7feec00af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feebe7fc640 1 -- 172.21.15.133:0/444814514 >> v1:172.21.15.133:6789/0 conn(0x7feec006e8b0 legacy=0x7feec0146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feebe7fc640 1 -- 172.21.15.133:0/444814514 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec00b1090 con 0x7feec015eb40 2024-04-25T10:11:08.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.850+0000 7feebdffb640 1 -- 172.21.15.133:0/444814514 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223130083 0 0) 0x7feec01471f0 con 0x7feec006e8b0 2024-04-25T10:11:08.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feebe7fc640 1 --2- 172.21.15.133:0/444814514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec015eb40 0x7feec00b1e50 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7feec00b27f0 tx=0x7feeb401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9065b67bc9f6f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feebdffb640 1 -- 172.21.15.133:0/444814514 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb40b0070 con 0x7feec015eb40 2024-04-25T10:11:08.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feebdffb640 1 -- 172.21.15.133:0/444814514 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb40ab600 con 0x7feec015eb40 2024-04-25T10:11:08.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feebdffb640 1 -- 172.21.15.133:0/444814514 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb40ab9d0 con 0x7feec015eb40 2024-04-25T10:11:08.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 -- 172.21.15.133:0/444814514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec015eb40 msgr2=0x7feec00b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 --2- 172.21.15.133:0/444814514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec015eb40 0x7feec00b1e50 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7feec00b27f0 tx=0x7feeb401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 -- 172.21.15.133:0/444814514 shutdown_connections 2024-04-25T10:11:08.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 --2- 172.21.15.133:0/444814514 >> v2:172.21.15.162:3300/0 conn(0x7feec00aebc0 0x7feec00af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 --2- 172.21.15.133:0/444814514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec015eb40 0x7feec00b1e50 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 -- 172.21.15.133:0/444814514 >> 172.21.15.133:0/444814514 conn(0x7feec0066f70 msgr2=0x7feec006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 -- 172.21.15.133:0/444814514 shutdown_connections 2024-04-25T10:11:08.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 -- 172.21.15.133:0/444814514 wait complete. 2024-04-25T10:11:08.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 Processor -- start 2024-04-25T10:11:08.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 -- start start 2024-04-25T10:11:08.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feec006e8b0 0x7feec013e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feec00aebc0 0x7feec015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec015eb40 0x7feec015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec006de40 con 0x7feec015eb40 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec006dcc0 con 0x7feec006e8b0 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feec544e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec00b2390 con 0x7feec00aebc0 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feebf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec015eb40 0x7feec015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feebf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec015eb40 0x7feec015b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36902/0 (socket says 172.21.15.133:36902) 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.854+0000 7feebf7fe640 1 -- 172.21.15.133:0/3413473812 learned_addr learned my addr 172.21.15.133:0/3413473812 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feebeffd640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feec006e8b0 0x7feec013e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feebe7fc640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feec00aebc0 0x7feec015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feebf7fe640 1 -- 172.21.15.133:0/3413473812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feec00aebc0 msgr2=0x7feec015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feebf7fe640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feec00aebc0 0x7feec015bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feebf7fe640 1 -- 172.21.15.133:0/3413473812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feec006e8b0 msgr2=0x7feec013e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feebf7fe640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feec006e8b0 0x7feec013e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feebf7fe640 1 -- 172.21.15.133:0/3413473812 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec015c810 con 0x7feec015eb40 2024-04-25T10:11:08.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feebeffd640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feec006e8b0 0x7feec013e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feebf7fe640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec015eb40 0x7feec015b170 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7feeb8007e40 tx=0x7feeb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feea77fe640 1 -- 172.21.15.133:0/3413473812 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8014620 con 0x7feec015eb40 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feea77fe640 1 -- 172.21.15.133:0/3413473812 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb8019070 con 0x7feec015eb40 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feec544e640 1 -- 172.21.15.133:0/3413473812 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec015caf0 con 0x7feec015eb40 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feea77fe640 1 -- 172.21.15.133:0/3413473812 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8014a20 con 0x7feec015eb40 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feec544e640 1 -- 172.21.15.133:0/3413473812 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec00b2c70 con 0x7feec015eb40 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feec544e640 1 -- 172.21.15.133:0/3413473812 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec015d010 con 0x7feec015eb40 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feea77fe640 1 -- 172.21.15.133:0/3413473812 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feeb8014bc0 con 0x7feec015eb40 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feea77fe640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feeac066f60 0x7feeac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feea77fe640 1 -- 172.21.15.133:0/3413473812 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5249+0+0 (secure 0 0 0) 0x7feeb8085750 con 0x7feec015eb40 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feebeffd640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feeac066f60 0x7feeac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.858+0000 7feebeffd640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feeac066f60 0x7feeac069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feeb0000f70 tx=0x7feeb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:08.862+0000 7feea77fe640 1 -- 172.21.15.133:0/3413473812 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feec015d010 con 0x7feec015eb40 2024-04-25T10:11:09.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.022+0000 7feec544e640 1 -- 172.21.15.133:0/3413473812 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feec00b2e70 con 0x7feec015eb40 2024-04-25T10:11:09.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.022+0000 7feea77fe640 1 -- 172.21.15.133:0/3413473812 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7feec00b2e70 con 0x7feec015eb40 2024-04-25T10:11:09.026 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:11:09.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 -- 172.21.15.133:0/3413473812 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feeac066f60 msgr2=0x7feeac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feeac066f60 0x7feeac069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feeb0000f70 tx=0x7feeb0009040 comp rx=0 tx=0).stop 2024-04-25T10:11:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 -- 172.21.15.133:0/3413473812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec015eb40 msgr2=0x7feec015b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec015eb40 0x7feec015b170 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7feeb8007e40 tx=0x7feeb800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 -- 172.21.15.133:0/3413473812 shutdown_connections 2024-04-25T10:11:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feeac066f60 0x7feeac069420 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec015eb40 0x7feec015b170 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feec00aebc0 0x7feec015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 --2- 172.21.15.133:0/3413473812 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feec006e8b0 0x7feec013e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 -- 172.21.15.133:0/3413473812 >> 172.21.15.133:0/3413473812 conn(0x7feec0066f70 msgr2=0x7feec00cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 -- 172.21.15.133:0/3413473812 shutdown_connections 2024-04-25T10:11:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.026+0000 7feec544e640 1 -- 172.21.15.133:0/3413473812 wait complete. 2024-04-25T10:11:09.048 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:11:09.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.158+0000 7fa60db31640 1 Processor -- start 2024-04-25T10:11:09.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.158+0000 7fa60db31640 1 -- start start 2024-04-25T10:11:09.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa60db31640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa60815db40 0x7fa6080d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa60db31640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6080d23c0 con 0x7fa6080d1f20 2024-04-25T10:11:09.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa60db31640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6080d26a0 con 0x7fa6080b5a60 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa60db31640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6080d2fd0 con 0x7fa60815db40 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa606ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa6080d1f20 0x7fa6080b5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34796/0 (socket says 172.21.15.133:34796) 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa606ffd640 1 -- 172.21.15.133:0/494330428 learned_addr learned my addr 172.21.15.133:0/494330428 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112043625 0 0) 0x7fa6080d23c0 con 0x7fa6080d1f20 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5f4003530 con 0x7fa6080d1f20 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6077fe640 1 --2- 172.21.15.133:0/494330428 >> v2:172.21.15.162:3300/0 conn(0x7fa60815db40 0x7fa6080d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 62104921 0 0) 0x7fa5f4003530 con 0x7fa6080d1f20 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 >> v2:172.21.15.162:3300/0 conn(0x7fa60815db40 msgr2=0x7fa6080d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 --2- 172.21.15.133:0/494330428 >> v2:172.21.15.162:3300/0 conn(0x7fa60815db40 0x7fa6080d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 >> v1:172.21.15.162:6789/0 conn(0x7fa6080b5a60 legacy=0x7fa6080b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6080d2980 con 0x7fa6080d1f20 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa5f0001680 con 0x7fa6080d1f20 2024-04-25T10:11:09.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 >> v1:172.21.15.133:6789/0 conn(0x7fa6080d1f20 legacy=0x7fa6080b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 --2- 172.21.15.133:0/494330428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 0x7fa5f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 --2- 172.21.15.133:0/494330428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f4004810 0x7fa5f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 --2- 172.21.15.133:0/494330428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5f40051d0 0x7fa5f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6080d2fd0 con 0x7fa5f40039c0 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f400c120 con 0x7fa5f4004810 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f400c320 con 0x7fa5f40051d0 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6077fe640 1 --2- 172.21.15.133:0/494330428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 0x7fa5f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa606ffd640 1 --2- 172.21.15.133:0/494330428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f4004810 0x7fa5f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa607fff640 1 --2- 172.21.15.133:0/494330428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5f40051d0 0x7fa5f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6077fe640 1 -- 172.21.15.133:0/494330428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5f40051d0 msgr2=0x7fa5f400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6077fe640 1 --2- 172.21.15.133:0/494330428 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5f40051d0 0x7fa5f400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6077fe640 1 -- 172.21.15.133:0/494330428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f4004810 msgr2=0x7fa5f4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6077fe640 1 --2- 172.21.15.133:0/494330428 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5f4004810 0x7fa5f4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa6077fe640 1 -- 172.21.15.133:0/494330428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f4003530 con 0x7fa5f40039c0 2024-04-25T10:11:09.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.162+0000 7fa607fff640 1 -- 172.21.15.133:0/494330428 reap_dead start 2024-04-25T10:11:09.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6077fe640 1 --2- 172.21.15.133:0/494330428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 0x7fa5f4003de0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fa60000ba30 tx=0x7fa60001c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec4f2ff2d4a715a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6000ae020 con 0x7fa5f40039c0 2024-04-25T10:11:09.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6000a6620 con 0x7fa5f40039c0 2024-04-25T10:11:09.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6067fc640 1 -- 172.21.15.133:0/494330428 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000b600 con 0x7fa5f40039c0 2024-04-25T10:11:09.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 -- 172.21.15.133:0/494330428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 msgr2=0x7fa5f4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 --2- 172.21.15.133:0/494330428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 0x7fa5f4003de0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fa60000ba30 tx=0x7fa60001c690 comp rx=0 tx=0).stop 2024-04-25T10:11:09.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 -- 172.21.15.133:0/494330428 shutdown_connections 2024-04-25T10:11:09.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 --2- 172.21.15.133:0/494330428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 0x7fa5f4003de0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 -- 172.21.15.133:0/494330428 >> 172.21.15.133:0/494330428 conn(0x7fa6080d0350 msgr2=0x7fa6080d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 -- 172.21.15.133:0/494330428 shutdown_connections 2024-04-25T10:11:09.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 -- 172.21.15.133:0/494330428 wait complete. 2024-04-25T10:11:09.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 Processor -- start 2024-04-25T10:11:09.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 -- start start 2024-04-25T10:11:09.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 0x7fa60815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa60815a860 0x7fa60815fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa60815acc0 0x7fa608160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6080d35c0 con 0x7fa5f40039c0 2024-04-25T10:11:09.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6080d3440 con 0x7fa60815acc0 2024-04-25T10:11:09.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa60db31640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6080ce280 con 0x7fa60815a860 2024-04-25T10:11:09.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 0x7fa60815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 0x7fa60815e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36908/0 (socket says 172.21.15.133:36908) 2024-04-25T10:11:09.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6077fe640 1 -- 172.21.15.133:0/3883805852 learned_addr learned my addr 172.21.15.133:0/3883805852 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:09.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa607fff640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa60815acc0 0x7fa608160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa606ffd640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa60815a860 0x7fa60815fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6077fe640 1 -- 172.21.15.133:0/3883805852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa60815a860 msgr2=0x7fa60815fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6077fe640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa60815a860 0x7fa60815fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6077fe640 1 -- 172.21.15.133:0/3883805852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa60815acc0 msgr2=0x7fa608160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6077fe640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa60815acc0 0x7fa608160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.166+0000 7fa6077fe640 1 -- 172.21.15.133:0/3883805852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60815e6e0 con 0x7fa5f40039c0 2024-04-25T10:11:09.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa6077fe640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 0x7fa60815e1a0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa60001cb70 tx=0x7fa5f4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa5fffff640 1 -- 172.21.15.133:0/3883805852 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6000ae040 con 0x7fa5f40039c0 2024-04-25T10:11:09.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa5fffff640 1 -- 172.21.15.133:0/3883805852 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6000a6e20 con 0x7fa5f40039c0 2024-04-25T10:11:09.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa5fffff640 1 -- 172.21.15.133:0/3883805852 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6000ab040 con 0x7fa5f40039c0 2024-04-25T10:11:09.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa60db31640 1 -- 172.21.15.133:0/3883805852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60815b360 con 0x7fa5f40039c0 2024-04-25T10:11:09.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa60db31640 1 -- 172.21.15.133:0/3883805852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6080c73e0 con 0x7fa5f40039c0 2024-04-25T10:11:09.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa60db31640 1 -- 172.21.15.133:0/3883805852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60815b160 con 0x7fa5f40039c0 2024-04-25T10:11:09.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa5fffff640 1 -- 172.21.15.133:0/3883805852 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6000be020 con 0x7fa5f40039c0 2024-04-25T10:11:09.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa5fffff640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5e8066f90 0x7fa5e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa5fffff640 1 -- 172.21.15.133:0/3883805852 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa60011e4a0 con 0x7fa5f40039c0 2024-04-25T10:11:09.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa606ffd640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5e8066f90 0x7fa5e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.170+0000 7fa606ffd640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5e8066f90 0x7fa5e8069450 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0001060 tx=0x7fa5f0001160 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.174+0000 7fa5fffff640 1 -- 172.21.15.133:0/3883805852 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6000c5020 con 0x7fa5f40039c0 2024-04-25T10:11:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.334+0000 7fa60db31640 1 -- 172.21.15.133:0/3883805852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa608068ef0 con 0x7fa5f40039c0 2024-04-25T10:11:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.334+0000 7fa5fffff640 1 -- 172.21.15.133:0/3883805852 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa60815e6e0 con 0x7fa5f40039c0 2024-04-25T10:11:09.337 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:09.338 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:11:09.338 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:11:09.338 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:11:09.338 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:11:09.338 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:09.338 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:11:09.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 -- 172.21.15.133:0/3883805852 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5e8066f90 msgr2=0x7fa5e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5e8066f90 0x7fa5e8069450 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0001060 tx=0x7fa5f0001160 comp rx=0 tx=0).stop 2024-04-25T10:11:09.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 -- 172.21.15.133:0/3883805852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 msgr2=0x7fa60815e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 0x7fa60815e1a0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa60001cb70 tx=0x7fa5f4005840 comp rx=0 tx=0).stop 2024-04-25T10:11:09.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 -- 172.21.15.133:0/3883805852 shutdown_connections 2024-04-25T10:11:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa60815acc0 0x7fa608160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa60815a860 0x7fa60815fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5e8066f90 0x7fa5e8069450 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 --2- 172.21.15.133:0/3883805852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5f40039c0 0x7fa60815e1a0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 -- 172.21.15.133:0/3883805852 >> 172.21.15.133:0/3883805852 conn(0x7fa6080d0350 msgr2=0x7fa6080ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 -- 172.21.15.133:0/3883805852 shutdown_connections 2024-04-25T10:11:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.338+0000 7fa60db31640 1 -- 172.21.15.133:0/3883805852 wait complete. 2024-04-25T10:11:09.357 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:11:09.357 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:11:09.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0aba62d640 1 Processor -- start 2024-04-25T10:11:09.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0aba62d640 1 -- start start 2024-04-25T10:11:09.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0aba62d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab415d8c0 0x7f0ab4161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0aba62d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab40d1cf0 con 0x7f0ab40cef10 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0aba62d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab4162130 con 0x7f0ab4161c60 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0aba62d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41626f0 con 0x7f0ab415d8c0 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab8bca640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0ab40cef10 0x7f0ab40d24b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34808/0 (socket says 172.21.15.133:34808) 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab8bca640 1 -- 172.21.15.133:0/3358682810 learned_addr learned my addr 172.21.15.133:0/3358682810 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab3fff640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab415d8c0 0x7f0ab4161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579931739 0 0) 0x7f0ab40d1cf0 con 0x7f0ab40cef10 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa0003530 con 0x7f0ab40cef10 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1188006754 0 0) 0x7f0aa0003530 con 0x7f0ab40cef10 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab415d8c0 msgr2=0x7f0ab4161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab415d8c0 0x7f0ab4161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 >> v1:172.21.15.162:6789/0 conn(0x7f0ab4161c60 legacy=0x7f0ab40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab4162410 con 0x7f0ab40cef10 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0aac001a00 con 0x7f0ab40cef10 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 >> v1:172.21.15.133:6789/0 conn(0x7f0ab40cef10 legacy=0x7f0ab40d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0aa00039f0 0x7f0aa0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab415d8c0 0x7f0aa0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aa0004d50 0x7f0aa00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41626f0 con 0x7f0aa00039f0 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa000bb00 con 0x7f0aa0004d50 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa000bd00 con 0x7f0ab415d8c0 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab3fff640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0aa00039f0 0x7f0aa0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab8bca640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aa0004d50 0x7f0aa00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab3fff640 1 -- 172.21.15.133:0/3358682810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab415d8c0 msgr2=0x7f0aa0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab3fff640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab415d8c0 0x7f0aa0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab3fff640 1 -- 172.21.15.133:0/3358682810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aa0004d50 msgr2=0x7f0aa00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab3fff640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aa0004d50 0x7f0aa00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab3fff640 1 -- 172.21.15.133:0/3358682810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa0003530 con 0x7f0aa00039f0 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.474+0000 7f0ab8bca640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aa0004d50 0x7f0aa00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:09.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0ab3fff640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0aa00039f0 0x7f0aa0003e10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009ae0 tx=0x7f0aa401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663d831f07e27514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40b2070 con 0x7f0aa00039f0 2024-04-25T10:11:09.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab40b9a20 con 0x7f0aa00039f0 2024-04-25T10:11:09.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0ab37fe640 1 -- 172.21.15.133:0/3358682810 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40096b0 con 0x7f0aa00039f0 2024-04-25T10:11:09.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0aba62d640 1 -- 172.21.15.133:0/3358682810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0aa00039f0 msgr2=0x7f0aa0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0aba62d640 1 --2- 172.21.15.133:0/3358682810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0aa00039f0 0x7f0aa0003e10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009ae0 tx=0x7f0aa401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0ab8bca640 1 -- 172.21.15.133:0/3358682810 reap_dead start 2024-04-25T10:11:09.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0aba62d640 1 -- 172.21.15.133:0/3358682810 shutdown_connections 2024-04-25T10:11:09.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0aba62d640 1 -- 172.21.15.133:0/3358682810 >> 172.21.15.133:0/3358682810 conn(0x7f0ab40723c0 msgr2=0x7f0ab40727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0aba62d640 1 -- 172.21.15.133:0/3358682810 shutdown_connections 2024-04-25T10:11:09.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0aba62d640 1 -- 172.21.15.133:0/3358682810 wait complete. 2024-04-25T10:11:09.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0aba62d640 1 Processor -- start 2024-04-25T10:11:09.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.478+0000 7f0aba62d640 1 -- start start 2024-04-25T10:11:09.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0aba62d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aa00039f0 0x7f0ab4158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0aba62d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab4168820 0x7f0ab4156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0aba62d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ab40cd810 0x7f0ab40cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0aba62d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40d37e0 con 0x7f0ab40cd810 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0aba62d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40d3660 con 0x7f0aa00039f0 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0aba62d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab4169ac0 con 0x7f0ab4168820 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab8bca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ab40cd810 0x7f0ab40cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab8bca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ab40cd810 0x7f0ab40cdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36924/0 (socket says 172.21.15.133:36924) 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab8bca640 1 -- 172.21.15.133:0/1080540444 learned_addr learned my addr 172.21.15.133:0/1080540444 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0aabfff640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab4168820 0x7f0ab4156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab3fff640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aa00039f0 0x7f0ab4158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab8bca640 1 -- 172.21.15.133:0/1080540444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab4168820 msgr2=0x7f0ab4156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab8bca640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab4168820 0x7f0ab4156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab8bca640 1 -- 172.21.15.133:0/1080540444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aa00039f0 msgr2=0x7f0ab4158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab8bca640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aa00039f0 0x7f0ab4158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab8bca640 1 -- 172.21.15.133:0/1080540444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab41595d0 con 0x7f0ab40cd810 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab8bca640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ab40cd810 0x7f0ab40cdc90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40d1580 tx=0x7f0aac00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab17fa640 1 -- 172.21.15.133:0/1080540444 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac00bb20 con 0x7f0ab40cd810 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab17fa640 1 -- 172.21.15.133:0/1080540444 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac00bcc0 con 0x7f0ab40cd810 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0aba62d640 1 -- 172.21.15.133:0/1080540444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab41598b0 con 0x7f0ab40cd810 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab17fa640 1 -- 172.21.15.133:0/1080540444 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0072b0 con 0x7f0ab40cd810 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0aba62d640 1 -- 172.21.15.133:0/1080540444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab4159e70 con 0x7f0ab40cd810 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0aba62d640 1 -- 172.21.15.133:0/1080540444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab4069000 con 0x7f0ab40cd810 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab17fa640 1 -- 172.21.15.133:0/1080540444 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0aac01b020 con 0x7f0ab40cd810 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab17fa640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a94066f90 0x7f0a94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab17fa640 1 -- 172.21.15.133:0/1080540444 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0aac0850b0 con 0x7f0ab40cd810 2024-04-25T10:11:09.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab3fff640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a94066f90 0x7f0a94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.482+0000 7f0ab3fff640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a94066f90 0x7f0a94069450 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40af000 tx=0x7f0aa40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.486+0000 7f0ab17fa640 1 -- 172.21.15.133:0/1080540444 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0aac056280 con 0x7f0ab40cd810 2024-04-25T10:11:09.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:09.634+0000 7f0aba62d640 1 -- 172.21.15.133:0/1080540444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0ab40d1700 con 0x7f0ab40cd810 2024-04-25T10:11:10.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.046+0000 7f0ab17fa640 1 -- 172.21.15.133:0/1080540444 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0aac055b80 con 0x7f0ab40cd810 2024-04-25T10:11:10.049 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:11:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 -- 172.21.15.133:0/1080540444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a94066f90 msgr2=0x7f0a94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a94066f90 0x7f0a94069450 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0aa40af000 tx=0x7f0aa40046d0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 -- 172.21.15.133:0/1080540444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ab40cd810 msgr2=0x7f0ab40cdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ab40cd810 0x7f0ab40cdc90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40d1580 tx=0x7f0aac00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 -- 172.21.15.133:0/1080540444 shutdown_connections 2024-04-25T10:11:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0a94066f90 0x7f0a94069450 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ab40cd810 0x7f0ab40cdc90 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ab4168820 0x7f0ab4156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 --2- 172.21.15.133:0/1080540444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0aa00039f0 0x7f0ab4158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 -- 172.21.15.133:0/1080540444 >> 172.21.15.133:0/1080540444 conn(0x7f0ab40723c0 msgr2=0x7f0ab40ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 -- 172.21.15.133:0/1080540444 shutdown_connections 2024-04-25T10:11:10.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.050+0000 7f0aba62d640 1 -- 172.21.15.133:0/1080540444 wait complete. 2024-04-25T10:11:10.077 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:11:10.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f8752876640 1 Processor -- start 2024-04-25T10:11:10.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f8752876640 1 -- start start 2024-04-25T10:11:10.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f8752876640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d02b0 0x7f874c162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f8752876640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f874c15efd0 0x7f874c0d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f8752876640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f874c0ce850 0x7f874c0d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f8752876640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c15f450 con 0x7f874c15efd0 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f8752876640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c0d0bd0 con 0x7f874c0ce850 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f8752876640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0cf0c0 con 0x7f874c0d02b0 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f874b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f874c15efd0 0x7f874c0d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f874b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f874c15efd0 0x7f874c0d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36940/0 (socket says 172.21.15.133:36940) 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f874b7fe640 1 -- 172.21.15.133:0/2821644496 learned_addr learned my addr 172.21.15.133:0/2821644496 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f874bfff640 1 --2- 172.21.15.133:0/2821644496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d02b0 0x7f874c162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f8750e13640 1 --2- 172.21.15.133:0/2821644496 >> v2:172.21.15.162:3300/0 conn(0x7f874c0ce850 0x7f874c0d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f874b7fe640 1 -- 172.21.15.133:0/2821644496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d02b0 msgr2=0x7f874c162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f874b7fe640 1 --2- 172.21.15.133:0/2821644496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d02b0 0x7f874c162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f874b7fe640 1 -- 172.21.15.133:0/2821644496 >> v2:172.21.15.162:3300/0 conn(0x7f874c0ce850 msgr2=0x7f874c0d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f874b7fe640 1 --2- 172.21.15.133:0/2821644496 >> v2:172.21.15.162:3300/0 conn(0x7f874c0ce850 0x7f874c0d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.206+0000 7f874b7fe640 1 -- 172.21.15.133:0/2821644496 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c0cf3a0 con 0x7f874c15efd0 2024-04-25T10:11:10.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f874b7fe640 1 --2- 172.21.15.133:0/2821644496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15efd0 0x7f874c0d0690 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f874c0d1720 tx=0x7f874001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3936b9a78ad6f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f874affd640 1 -- 172.21.15.133:0/2821644496 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87400ab620 con 0x7f874c15efd0 2024-04-25T10:11:10.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f874affd640 1 -- 172.21.15.133:0/2821644496 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87400b0070 con 0x7f874c15efd0 2024-04-25T10:11:10.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f874affd640 1 -- 172.21.15.133:0/2821644496 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87400ab9e0 con 0x7f874c15efd0 2024-04-25T10:11:10.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f8752876640 1 -- 172.21.15.133:0/2821644496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15efd0 msgr2=0x7f874c0d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f8752876640 1 --2- 172.21.15.133:0/2821644496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15efd0 0x7f874c0d0690 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f874c0d1720 tx=0x7f874001c460 comp rx=0 tx=0).stop 2024-04-25T10:11:10.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f8752876640 1 -- 172.21.15.133:0/2821644496 shutdown_connections 2024-04-25T10:11:10.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f8752876640 1 --2- 172.21.15.133:0/2821644496 >> v2:172.21.15.162:3300/0 conn(0x7f874c0ce850 0x7f874c0d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f8752876640 1 --2- 172.21.15.133:0/2821644496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15efd0 0x7f874c0d0690 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f8752876640 1 --2- 172.21.15.133:0/2821644496 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0d02b0 0x7f874c162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f8752876640 1 -- 172.21.15.133:0/2821644496 >> 172.21.15.133:0/2821644496 conn(0x7f874c06bf30 msgr2=0x7f874c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:10.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f8752876640 1 -- 172.21.15.133:0/2821644496 shutdown_connections 2024-04-25T10:11:10.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f8752876640 1 -- 172.21.15.133:0/2821644496 wait complete. 2024-04-25T10:11:10.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f8752876640 1 Processor -- start 2024-04-25T10:11:10.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.210+0000 7f8752876640 1 -- start start 2024-04-25T10:11:10.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8752876640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0ce850 0x7f874c0b7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8752876640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f874c0d02b0 0x7f874c0cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8752876640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15efd0 0x7f874c0cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8752876640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0cede0 con 0x7f874c15efd0 2024-04-25T10:11:10.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8752876640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0cec60 con 0x7f874c0d02b0 2024-04-25T10:11:10.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8752876640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c15bc80 con 0x7f874c0ce850 2024-04-25T10:11:10.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8750e13640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15efd0 0x7f874c0cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8750e13640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15efd0 0x7f874c0cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36950/0 (socket says 172.21.15.133:36950) 2024-04-25T10:11:10.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8750e13640 1 -- 172.21.15.133:0/499790212 learned_addr learned my addr 172.21.15.133:0/499790212 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:10.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f874bfff640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0ce850 0x7f874c0b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f874b7fe640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f874c0d02b0 0x7f874c0cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8750e13640 1 -- 172.21.15.133:0/499790212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0ce850 msgr2=0x7f874c0b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8750e13640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0ce850 0x7f874c0b7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8750e13640 1 -- 172.21.15.133:0/499790212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f874c0d02b0 msgr2=0x7f874c0cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8750e13640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f874c0d02b0 0x7f874c0cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8750e13640 1 -- 172.21.15.133:0/499790212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c0b7f40 con 0x7f874c15efd0 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8750e13640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15efd0 0x7f874c0cb8d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f8744007e60 tx=0x7f874400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8748ff9640 1 -- 172.21.15.133:0/499790212 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744014620 con 0x7f874c15efd0 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8748ff9640 1 -- 172.21.15.133:0/499790212 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8744019070 con 0x7f874c15efd0 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8752876640 1 -- 172.21.15.133:0/499790212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c0b8220 con 0x7f874c15efd0 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8748ff9640 1 -- 172.21.15.133:0/499790212 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744014a20 con 0x7f874c15efd0 2024-04-25T10:11:10.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8752876640 1 -- 172.21.15.133:0/499790212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c0b87e0 con 0x7f874c15efd0 2024-04-25T10:11:10.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8752876640 1 -- 172.21.15.133:0/499790212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c067c70 con 0x7f874c15efd0 2024-04-25T10:11:10.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8748ff9640 1 -- 172.21.15.133:0/499790212 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8744014bc0 con 0x7f874c15efd0 2024-04-25T10:11:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8748ff9640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8738066f90 0x7f8738069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f8748ff9640 1 -- 172.21.15.133:0/499790212 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f8744085850 con 0x7f874c15efd0 2024-04-25T10:11:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f874bfff640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8738066f90 0x7f8738069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.214+0000 7f874bfff640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8738066f90 0x7f8738069450 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f873c00b9d0 tx=0x7f873c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.218+0000 7f8748ff9640 1 -- 172.21.15.133:0/499790212 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87440568b0 con 0x7f874c15efd0 2024-04-25T10:11:10.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:10.378+0000 7f8752876640 1 -- 172.21.15.133:0/499790212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f874c0d1a10 con 0x7f874c15efd0 2024-04-25T10:11:11.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.058+0000 7f8748ff9640 1 -- 172.21.15.133:0/499790212 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) v1 ==== 170+0+0 (secure 0 0 0) 0x7f87440561b0 con 0x7f874c15efd0 2024-04-25T10:11:11.061 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:11:11.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.062+0000 7f8752876640 1 -- 172.21.15.133:0/499790212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8738066f90 msgr2=0x7f8738069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.062+0000 7f8752876640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8738066f90 0x7f8738069450 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f873c00b9d0 tx=0x7f873c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:11.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.062+0000 7f8752876640 1 -- 172.21.15.133:0/499790212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15efd0 msgr2=0x7f874c0cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.062+0000 7f8752876640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15efd0 0x7f874c0cb8d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f8744007e60 tx=0x7f874400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.062+0000 7f8752876640 1 -- 172.21.15.133:0/499790212 shutdown_connections 2024-04-25T10:11:11.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.062+0000 7f8752876640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8738066f90 0x7f8738069450 secure :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f873c00b9d0 tx=0x7f873c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:11.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.062+0000 7f8752876640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f874c15efd0 0x7f874c0cb8d0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.062+0000 7f8752876640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f874c0d02b0 0x7f874c0cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.062+0000 7f8752876640 1 --2- 172.21.15.133:0/499790212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f874c0ce850 0x7f874c0b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.062+0000 7f8752876640 1 -- 172.21.15.133:0/499790212 >> 172.21.15.133:0/499790212 conn(0x7f874c06bf30 msgr2=0x7f874c15ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.066+0000 7f8752876640 1 -- 172.21.15.133:0/499790212 shutdown_connections 2024-04-25T10:11:11.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.066+0000 7f8752876640 1 -- 172.21.15.133:0/499790212 wait complete. 2024-04-25T10:11:11.103 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:11:11.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d9969640 1 Processor -- start 2024-04-25T10:11:11.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d9969640 1 -- start start 2024-04-25T10:11:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d9969640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86d406e8b0 0x7f86d415eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d9969640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86d40b1e50 0x7f86d4146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d9969640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d40b1090 con 0x7f86d40aebc0 2024-04-25T10:11:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d9969640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d4147490 con 0x7f86d406e8b0 2024-04-25T10:11:11.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d9969640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40b19c0 con 0x7f86d40b1e50 2024-04-25T10:11:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d37fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f86d40aebc0 0x7f86d406d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34822/0 (socket says 172.21.15.133:34822) 2024-04-25T10:11:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d37fe640 1 -- 172.21.15.133:0/3260375235 learned_addr learned my addr 172.21.15.133:0/3260375235 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d2ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86d406e8b0 0x7f86d415eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d27fc640 1 --2- 172.21.15.133:0/3260375235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86d40b1e50 0x7f86d4146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d1ffb640 1 -- 172.21.15.133:0/3260375235 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719372060 0 0) 0x7f86d40b1090 con 0x7f86d40aebc0 2024-04-25T10:11:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d1ffb640 1 -- 172.21.15.133:0/3260375235 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86bc003530 con 0x7f86d40aebc0 2024-04-25T10:11:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d2ffd640 1 -- 172.21.15.133:0/3260375235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86d40b1e50 msgr2=0x7f86d4146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d2ffd640 1 --2- 172.21.15.133:0/3260375235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86d40b1e50 0x7f86d4146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d1ffb640 1 -- 172.21.15.133:0/3260375235 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3072643674 0 0) 0x7f86bc003530 con 0x7f86d40aebc0 2024-04-25T10:11:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d2ffd640 1 -- 172.21.15.133:0/3260375235 >> v1:172.21.15.133:6789/0 conn(0x7f86d40aebc0 legacy=0x7f86d406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.222+0000 7f86d2ffd640 1 -- 172.21.15.133:0/3260375235 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d40b1370 con 0x7f86d406e8b0 2024-04-25T10:11:11.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d2ffd640 1 --2- 172.21.15.133:0/3260375235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86d406e8b0 0x7f86d415eb40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f86c40097f0 tx=0x7f86c401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7619cdb35bb5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d1ffb640 1 -- 172.21.15.133:0/3260375235 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c40b0070 con 0x7f86d406e8b0 2024-04-25T10:11:11.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d1ffb640 1 -- 172.21.15.133:0/3260375235 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c40ab400 con 0x7f86d406e8b0 2024-04-25T10:11:11.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d1ffb640 1 -- 172.21.15.133:0/3260375235 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc001a00 con 0x7f86d406e8b0 2024-04-25T10:11:11.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d9969640 1 -- 172.21.15.133:0/3260375235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86d406e8b0 msgr2=0x7f86d415eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d9969640 1 --2- 172.21.15.133:0/3260375235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86d406e8b0 0x7f86d415eb40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f86c40097f0 tx=0x7f86c401c330 comp rx=0 tx=0).stop 2024-04-25T10:11:11.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d9969640 1 -- 172.21.15.133:0/3260375235 shutdown_connections 2024-04-25T10:11:11.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d9969640 1 --2- 172.21.15.133:0/3260375235 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86d40b1e50 0x7f86d4146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d9969640 1 --2- 172.21.15.133:0/3260375235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86d406e8b0 0x7f86d415eb40 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d9969640 1 -- 172.21.15.133:0/3260375235 >> 172.21.15.133:0/3260375235 conn(0x7f86d4066f70 msgr2=0x7f86d406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d9969640 1 -- 172.21.15.133:0/3260375235 shutdown_connections 2024-04-25T10:11:11.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.226+0000 7f86d9969640 1 -- 172.21.15.133:0/3260375235 wait complete. 2024-04-25T10:11:11.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d9969640 1 Processor -- start 2024-04-25T10:11:11.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d9969640 1 -- start start 2024-04-25T10:11:11.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d9969640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86d406e8b0 0x7f86d41424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d9969640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86d40aebc0 0x7f86d415cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d9969640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86d40b1e50 0x7f86d40b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d9969640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40b23b0 con 0x7f86d40b1e50 2024-04-25T10:11:11.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d9969640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40b2230 con 0x7f86d40aebc0 2024-04-25T10:11:11.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d9969640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d4147020 con 0x7f86d406e8b0 2024-04-25T10:11:11.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86d40b1e50 0x7f86d40b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86d40b1e50 0x7f86d40b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36970/0 (socket says 172.21.15.133:36970) 2024-04-25T10:11:11.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d37fe640 1 -- 172.21.15.133:0/3499635815 learned_addr learned my addr 172.21.15.133:0/3499635815 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d27fc640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86d40aebc0 0x7f86d415cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d2ffd640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86d406e8b0 0x7f86d41424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d37fe640 1 -- 172.21.15.133:0/3499635815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86d406e8b0 msgr2=0x7f86d41424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d37fe640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86d406e8b0 0x7f86d41424a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d37fe640 1 -- 172.21.15.133:0/3499635815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86d40aebc0 msgr2=0x7f86d415cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d37fe640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86d40aebc0 0x7f86d415cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d37fe640 1 -- 172.21.15.133:0/3499635815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d4141f80 con 0x7f86d40b1e50 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d37fe640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86d40b1e50 0x7f86d40b3890 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00cc90 tx=0x7f86cc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86bbfff640 1 -- 172.21.15.133:0/3499635815 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc017070 con 0x7f86d40b1e50 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86bbfff640 1 -- 172.21.15.133:0/3499635815 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc012400 con 0x7f86d40b1e50 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d9969640 1 -- 172.21.15.133:0/3499635815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d41407f0 con 0x7f86d40b1e50 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86bbfff640 1 -- 172.21.15.133:0/3499635815 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0127b0 con 0x7f86d40b1e50 2024-04-25T10:11:11.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d9969640 1 -- 172.21.15.133:0/3499635815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d4140db0 con 0x7f86d40b1e50 2024-04-25T10:11:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.230+0000 7f86d9969640 1 -- 172.21.15.133:0/3499635815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d4140610 con 0x7f86d40b1e50 2024-04-25T10:11:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.234+0000 7f86bbfff640 1 -- 172.21.15.133:0/3499635815 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86cc01a070 con 0x7f86d40b1e50 2024-04-25T10:11:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.234+0000 7f86bbfff640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86c0066f60 0x7f86c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.234+0000 7f86d2ffd640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86c0066f60 0x7f86c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.234+0000 7f86bbfff640 1 -- 172.21.15.133:0/3499635815 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f86cc084f10 con 0x7f86d40b1e50 2024-04-25T10:11:11.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.234+0000 7f86d2ffd640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86c0066f60 0x7f86c0069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f86c40ad000 tx=0x7f86c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.238+0000 7f86bbfff640 1 -- 172.21.15.133:0/3499635815 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86d4140610 con 0x7f86d40b1e50 2024-04-25T10:11:11.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:11.430+0000 7f86d9969640 1 -- 172.21.15.133:0/3499635815 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f86d4140fb0 con 0x7f86d40b1e50 2024-04-25T10:11:12.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.062+0000 7f86bbfff640 1 -- 172.21.15.133:0/3499635815 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v63) v1 ==== 174+0+0 (secure 0 0 0) 0x7f86d4140fb0 con 0x7f86d40b1e50 2024-04-25T10:11:12.064 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T10:11:12.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.062+0000 7f86d9969640 1 -- 172.21.15.133:0/3499635815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86c0066f60 msgr2=0x7f86c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.062+0000 7f86d9969640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86c0066f60 0x7f86c0069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f86c40ad000 tx=0x7f86c4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:12.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.062+0000 7f86d9969640 1 -- 172.21.15.133:0/3499635815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86d40b1e50 msgr2=0x7f86d40b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.062+0000 7f86d9969640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86d40b1e50 0x7f86d40b3890 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00cc90 tx=0x7f86cc00a430 comp rx=0 tx=0).stop 2024-04-25T10:11:12.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.066+0000 7f86d9969640 1 -- 172.21.15.133:0/3499635815 shutdown_connections 2024-04-25T10:11:12.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.066+0000 7f86d9969640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f86c0066f60 0x7f86c0069420 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.066+0000 7f86d9969640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86d40b1e50 0x7f86d40b3890 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.066+0000 7f86d9969640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86d40aebc0 0x7f86d415cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.066+0000 7f86d9969640 1 --2- 172.21.15.133:0/3499635815 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86d406e8b0 0x7f86d41424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.066+0000 7f86d9969640 1 -- 172.21.15.133:0/3499635815 >> 172.21.15.133:0/3499635815 conn(0x7f86d4066f70 msgr2=0x7f86d40737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.066+0000 7f86d9969640 1 -- 172.21.15.133:0/3499635815 shutdown_connections 2024-04-25T10:11:12.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.066+0000 7f86d9969640 1 -- 172.21.15.133:0/3499635815 wait complete. 2024-04-25T10:11:12.110 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:12.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.222+0000 7f9fa52b6640 1 Processor -- start 2024-04-25T10:11:12.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.222+0000 7f9fa52b6640 1 -- start start 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9fa52b6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9fa015a9d0 0x7f9fa00c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9fa52b6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa00cc5a0 con 0x7f9fa006eb70 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9fa52b6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa00cc880 con 0x7f9fa015a5f0 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9fa52b6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa00ccb60 con 0x7f9fa015a9d0 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9e7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9fa006eb70 0x7f9fa00cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34836/0 (socket says 172.21.15.133:34836) 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9e7fc640 1 -- 172.21.15.133:0/2207360444 learned_addr learned my addr 172.21.15.133:0/2207360444 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9f7fe640 1 --2- 172.21.15.133:0/2207360444 >> v2:172.21.15.162:3300/0 conn(0x7f9fa015a9d0 0x7f9fa00c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627984909 0 0) 0x7f9fa00cc5a0 con 0x7f9fa006eb70 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f88003530 con 0x7f9fa006eb70 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2833565261 0 0) 0x7f9f88003530 con 0x7f9fa006eb70 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 >> v2:172.21.15.162:3300/0 conn(0x7f9fa015a9d0 msgr2=0x7f9fa00c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 --2- 172.21.15.133:0/2207360444 >> v2:172.21.15.162:3300/0 conn(0x7f9fa015a9d0 0x7f9fa00c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 >> v1:172.21.15.162:6789/0 conn(0x7f9fa015a5f0 legacy=0x7f9fa006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa00cce40 con 0x7f9fa006eb70 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f9f94001680 con 0x7f9fa006eb70 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 >> v1:172.21.15.133:6789/0 conn(0x7f9fa006eb70 legacy=0x7f9fa00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f880039c0 0x7f9f88003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f88004810 0x7f9f88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f880051d0 0x7f9f8800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00ccb60 con 0x7f9f880039c0 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8800c070 con 0x7f9f880051d0 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8800c270 con 0x7f9f88004810 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9effd640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f880039c0 0x7f9f88003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9f7fe640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f880051d0 0x7f9f8800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9e7fc640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f88004810 0x7f9f88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9effd640 1 -- 172.21.15.133:0/2207360444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f88004810 msgr2=0x7f9f88004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9effd640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f88004810 0x7f9f88004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9effd640 1 -- 172.21.15.133:0/2207360444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f880051d0 msgr2=0x7f9f8800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9effd640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f880051d0 0x7f9f8800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9effd640 1 -- 172.21.15.133:0/2207360444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f88003530 con 0x7f9f880039c0 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9f7fe640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9f880051d0 0x7f9f8800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9e7fc640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f88004810 0x7f9f88004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.226+0000 7f9f9f7fe640 1 -- 172.21.15.133:0/2207360444 reap_dead start 2024-04-25T10:11:12.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9effd640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f880039c0 0x7f9f88003de0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f9f90001780 tx=0x7f9f9001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f840057d85549f8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f900013a0 con 0x7f9f880039c0 2024-04-25T10:11:12.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f900050d0 con 0x7f9f880039c0 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9dffb640 1 -- 172.21.15.133:0/2207360444 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90003600 con 0x7f9f880039c0 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2207360444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f880039c0 msgr2=0x7f9f88003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f880039c0 0x7f9f88003de0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f9f90001780 tx=0x7f9f9001c400 comp rx=0 tx=0).stop 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2207360444 shutdown_connections 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 --2- 172.21.15.133:0/2207360444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f880039c0 0x7f9f88003de0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2207360444 >> 172.21.15.133:0/2207360444 conn(0x7f9fa0067000 msgr2=0x7f9fa006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2207360444 shutdown_connections 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2207360444 wait complete. 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 Processor -- start 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 -- start start 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f880039c0 0x7f9fa0158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa00c7ae0 0x7f9fa0144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fa0145100 0x7f9fa01f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa006e7f0 con 0x7f9fa0145100 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa006e670 con 0x7f9fa00c7ae0 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9fa52b6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa006e970 con 0x7f9f880039c0 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fa0145100 0x7f9fa01f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fa0145100 0x7f9fa01f80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36992/0 (socket says 172.21.15.133:36992) 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9f7fe640 1 -- 172.21.15.133:0/2100198109 learned_addr learned my addr 172.21.15.133:0/2100198109 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9effd640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f880039c0 0x7f9fa0158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9e7fc640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa00c7ae0 0x7f9fa0144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9f7fe640 1 -- 172.21.15.133:0/2100198109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f880039c0 msgr2=0x7f9fa0158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9f7fe640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f880039c0 0x7f9fa0158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9f7fe640 1 -- 172.21.15.133:0/2100198109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa00c7ae0 msgr2=0x7f9fa0144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9f7fe640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa00c7ae0 0x7f9fa0144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9f7fe640 1 -- 172.21.15.133:0/2100198109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa00c6d20 con 0x7f9fa0145100 2024-04-25T10:11:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9e7fc640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa00c7ae0 0x7f9fa0144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.230+0000 7f9f9f7fe640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fa0145100 0x7f9fa01f80f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f9fa015add0 tx=0x7f9f9800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.234+0000 7f9f877fe640 1 -- 172.21.15.133:0/2100198109 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98016420 con 0x7f9fa0145100 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.234+0000 7f9f877fe640 1 -- 172.21.15.133:0/2100198109 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9801b070 con 0x7f9fa0145100 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.234+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2100198109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa00c7000 con 0x7f9fa0145100 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.234+0000 7f9f877fe640 1 -- 172.21.15.133:0/2100198109 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f980167c0 con 0x7f9fa0145100 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.234+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2100198109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa0158dc0 con 0x7f9fa0145100 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.234+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2100198109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa0068ef0 con 0x7f9fa0145100 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.234+0000 7f9f877fe640 1 -- 172.21.15.133:0/2100198109 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f98016960 con 0x7f9fa0145100 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.234+0000 7f9f877fe640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 0x7f9f8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.234+0000 7f9f877fe640 1 -- 172.21.15.133:0/2100198109 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9f98085eb0 con 0x7f9fa0145100 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.234+0000 7f9f9effd640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 0x7f9f8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.234+0000 7f9f9effd640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 0x7f9f8c069420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9f90001750 tx=0x7f9f900011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.238+0000 7f9f877fe640 1 -- 172.21.15.133:0/2100198109 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f98056dc0 con 0x7f9fa0145100 2024-04-25T10:11:12.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.390+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2100198109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fa015ee70 con 0x7f9fa0145100 2024-04-25T10:11:12.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.390+0000 7f9f877fe640 1 -- 172.21.15.133:0/2100198109 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16921 (secure 0 0 0) 0x7f9f980566c0 con 0x7f9fa0145100 2024-04-25T10:11:12.394 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:12.395 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":106,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:11:11.432086+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:09.673185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:10.421023+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:12.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2100198109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 msgr2=0x7f9f8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 0x7f9f8c069420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9f90001750 tx=0x7f9f900011a0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2100198109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fa0145100 msgr2=0x7f9fa01f80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fa0145100 0x7f9fa01f80f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f9fa015add0 tx=0x7f9f9800f520 comp rx=0 tx=0).stop 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2100198109 shutdown_connections 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9f8c066f60 0x7f9f8c069420 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9fa0145100 0x7f9fa01f80f0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9fa00c7ae0 0x7f9fa0144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 --2- 172.21.15.133:0/2100198109 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9f880039c0 0x7f9fa0158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2100198109 >> 172.21.15.133:0/2100198109 conn(0x7f9fa0067000 msgr2=0x7f9fa00cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2100198109 shutdown_connections 2024-04-25T10:11:12.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.394+0000 7f9fa52b6640 1 -- 172.21.15.133:0/2100198109 wait complete. 2024-04-25T10:11:12.413 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:12.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.530+0000 7fa5ca886640 1 Processor -- start 2024-04-25T10:11:12.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.530+0000 7fa5ca886640 1 -- start start 2024-04-25T10:11:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5ca886640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5c415c9a0 0x7fa5c40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5ca886640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa5c40b1f70 0x7fa5c406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5ca886640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5c40ce330 0x7fa5c40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5ca886640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c40cecc0 con 0x7fa5c415c9a0 2024-04-25T10:11:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5ca886640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c40c61d0 con 0x7fa5c40b1f70 2024-04-25T10:11:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5ca886640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c40c64b0 con 0x7fa5c40ce330 2024-04-25T10:11:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5c415c9a0 0x7fa5c40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5c415c9a0 0x7fa5c40c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37000/0 (socket says 172.21.15.133:37000) 2024-04-25T10:11:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c3fff640 1 -- 172.21.15.133:0/1180646258 learned_addr learned my addr 172.21.15.133:0/1180646258 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c8e23640 1 --2- 172.21.15.133:0/1180646258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5c40ce330 0x7fa5c40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c37fe640 1 --2- 172.21.15.133:0/1180646258 >> v2:172.21.15.162:3300/0 conn(0x7fa5c40b1f70 0x7fa5c406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c3fff640 1 -- 172.21.15.133:0/1180646258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5c40ce330 msgr2=0x7fa5c40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c3fff640 1 --2- 172.21.15.133:0/1180646258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5c40ce330 0x7fa5c40ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c3fff640 1 -- 172.21.15.133:0/1180646258 >> v2:172.21.15.162:3300/0 conn(0x7fa5c40b1f70 msgr2=0x7fa5c406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c3fff640 1 --2- 172.21.15.133:0/1180646258 >> v2:172.21.15.162:3300/0 conn(0x7fa5c40b1f70 0x7fa5c406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c3fff640 1 -- 172.21.15.133:0/1180646258 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c406d4f0 con 0x7fa5c415c9a0 2024-04-25T10:11:12.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c3fff640 1 --2- 172.21.15.133:0/1180646258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9a0 0x7fa5c40c5980 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4072670 tx=0x7fa5b401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628365b33707ac09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c2ffd640 1 -- 172.21.15.133:0/1180646258 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40b1070 con 0x7fa5c415c9a0 2024-04-25T10:11:12.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c2ffd640 1 -- 172.21.15.133:0/1180646258 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b40ac620 con 0x7fa5c415c9a0 2024-04-25T10:11:12.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.534+0000 7fa5c2ffd640 1 -- 172.21.15.133:0/1180646258 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40ac9f0 con 0x7fa5c415c9a0 2024-04-25T10:11:12.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 -- 172.21.15.133:0/1180646258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9a0 msgr2=0x7fa5c40c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 --2- 172.21.15.133:0/1180646258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9a0 0x7fa5c40c5980 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4072670 tx=0x7fa5b401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 -- 172.21.15.133:0/1180646258 shutdown_connections 2024-04-25T10:11:12.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 --2- 172.21.15.133:0/1180646258 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5c40ce330 0x7fa5c40ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 --2- 172.21.15.133:0/1180646258 >> v2:172.21.15.162:3300/0 conn(0x7fa5c40b1f70 0x7fa5c406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 --2- 172.21.15.133:0/1180646258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9a0 0x7fa5c40c5980 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 -- 172.21.15.133:0/1180646258 >> 172.21.15.133:0/1180646258 conn(0x7fa5c4067000 msgr2=0x7fa5c406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 -- 172.21.15.133:0/1180646258 shutdown_connections 2024-04-25T10:11:12.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 -- 172.21.15.133:0/1180646258 wait complete. 2024-04-25T10:11:12.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 Processor -- start 2024-04-25T10:11:12.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 -- start start 2024-04-25T10:11:12.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5c40b1f70 0x7fa5c4150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5c40ce330 0x7fa5c41579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9a0 0x7fa5c40ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c415cf00 con 0x7fa5c415c9a0 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c415cd80 con 0x7fa5c40ce330 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c415d950 con 0x7fa5c40b1f70 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c8e23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9a0 0x7fa5c40ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c8e23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9a0 0x7fa5c40ca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37006/0 (socket says 172.21.15.133:37006) 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c8e23640 1 -- 172.21.15.133:0/803164101 learned_addr learned my addr 172.21.15.133:0/803164101 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c3fff640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5c40b1f70 0x7fa5c4150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c37fe640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5c40ce330 0x7fa5c41579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c8e23640 1 -- 172.21.15.133:0/803164101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5c40b1f70 msgr2=0x7fa5c4150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c8e23640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5c40b1f70 0x7fa5c4150270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c8e23640 1 -- 172.21.15.133:0/803164101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5c40ce330 msgr2=0x7fa5c41579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c8e23640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5c40ce330 0x7fa5c41579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c8e23640 1 -- 172.21.15.133:0/803164101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c4157ef0 con 0x7fa5c415c9a0 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c8e23640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9a0 0x7fa5c40ca110 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc007e40 tx=0x7fa5bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c0ff9640 1 -- 172.21.15.133:0/803164101 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc014620 con 0x7fa5c415c9a0 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c0ff9640 1 -- 172.21.15.133:0/803164101 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc019070 con 0x7fa5c415c9a0 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 -- 172.21.15.133:0/803164101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c40c8670 con 0x7fa5c415c9a0 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5c0ff9640 1 -- 172.21.15.133:0/803164101 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc014a20 con 0x7fa5c415c9a0 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.538+0000 7fa5ca886640 1 -- 172.21.15.133:0/803164101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c40cae70 con 0x7fa5c415c9a0 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.542+0000 7fa5ca886640 1 -- 172.21.15.133:0/803164101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c4068ef0 con 0x7fa5c415c9a0 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.542+0000 7fa5c0ff9640 1 -- 172.21.15.133:0/803164101 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5bc014bc0 con 0x7fa5c415c9a0 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.542+0000 7fa5c0ff9640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5b0066f60 0x7fa5b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.542+0000 7fa5c0ff9640 1 -- 172.21.15.133:0/803164101 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5bc085aa0 con 0x7fa5c415c9a0 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.542+0000 7fa5c3fff640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5b0066f60 0x7fa5b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.542+0000 7fa5c3fff640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5b0066f60 0x7fa5b0069420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40ae000 tx=0x7fa5b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.546+0000 7fa5c0ff9640 1 -- 172.21.15.133:0/803164101 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5bc056920 con 0x7fa5c415c9a0 2024-04-25T10:11:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.694+0000 7fa5ca886640 1 -- 172.21.15.133:0/803164101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5c41570f0 con 0x7fa5c415c9a0 2024-04-25T10:11:12.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.694+0000 7fa5c0ff9640 1 -- 172.21.15.133:0/803164101 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16921 (secure 0 0 0) 0x7fa5bc056220 con 0x7fa5c415c9a0 2024-04-25T10:11:12.699 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:12.699 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":106,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:11:11.432086+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:09.673185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:10.421023+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 -- 172.21.15.133:0/803164101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5b0066f60 msgr2=0x7fa5b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5b0066f60 0x7fa5b0069420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40ae000 tx=0x7fa5b4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 -- 172.21.15.133:0/803164101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9a0 msgr2=0x7fa5c40ca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9a0 0x7fa5c40ca110 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc007e40 tx=0x7fa5bc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 -- 172.21.15.133:0/803164101 shutdown_connections 2024-04-25T10:11:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5b0066f60 0x7fa5b0069420 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5c415c9a0 0x7fa5c40ca110 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5c40ce330 0x7fa5c41579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 --2- 172.21.15.133:0/803164101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5c40b1f70 0x7fa5c4150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 -- 172.21.15.133:0/803164101 >> 172.21.15.133:0/803164101 conn(0x7fa5c4067000 msgr2=0x7fa5c41562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 -- 172.21.15.133:0/803164101 shutdown_connections 2024-04-25T10:11:12.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.698+0000 7fa5ca886640 1 -- 172.21.15.133:0/803164101 wait complete. 2024-04-25T10:11:12.717 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:11:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.826+0000 7fe70b3b2640 1 Processor -- start 2024-04-25T10:11:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70b3b2640 1 -- start start 2024-04-25T10:11:12.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70b3b2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe70406e3b0 0x7fe70406e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70b3b2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe7040b06d0 0x7fe7040b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70b3b2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7040ccf20 con 0x7fe704143fb0 2024-04-25T10:11:12.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70b3b2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70406ed50 con 0x7fe7040b06d0 2024-04-25T10:11:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70b3b2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7040cd1f0 con 0x7fe70406e3b0 2024-04-25T10:11:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70894d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe70406e3b0 0x7fe70406e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70894d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe70406e3b0 0x7fe70406e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:58728/0 (socket says 172.21.15.133:58728) 2024-04-25T10:11:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70894d640 1 -- 172.21.15.133:0/3735933967 learned_addr learned my addr 172.21.15.133:0/3735933967 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70994f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe7040b06d0 0x7fe7040b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70894d640 1 -- 172.21.15.133:0/3735933967 >> v2:172.21.15.162:3300/0 conn(0x7fe7040b06d0 msgr2=0x7fe7040b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70894d640 1 --2- 172.21.15.133:0/3735933967 >> v2:172.21.15.162:3300/0 conn(0x7fe7040b06d0 0x7fe7040b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70894d640 1 -- 172.21.15.133:0/3735933967 >> v1:172.21.15.162:6789/0 conn(0x7fe704143fb0 legacy=0x7fe7040cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.830+0000 7fe70894d640 1 -- 172.21.15.133:0/3735933967 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70415aee0 con 0x7fe70406e3b0 2024-04-25T10:11:12.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70894d640 1 --2- 172.21.15.133:0/3735933967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe70406e3b0 0x7fe70406e810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009870 tx=0x7fe6f401e740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b49d19349c8c6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe6fbfff640 1 -- 172.21.15.133:0/3735933967 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40ae070 con 0x7fe70406e3b0 2024-04-25T10:11:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe6fbfff640 1 -- 172.21.15.133:0/3735933967 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f40a8610 con 0x7fe70406e3b0 2024-04-25T10:11:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe6fbfff640 1 -- 172.21.15.133:0/3735933967 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40a89e0 con 0x7fe70406e3b0 2024-04-25T10:11:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 -- 172.21.15.133:0/3735933967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe70406e3b0 msgr2=0x7fe70406e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 --2- 172.21.15.133:0/3735933967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe70406e3b0 0x7fe70406e810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009870 tx=0x7fe6f401e740 comp rx=0 tx=0).stop 2024-04-25T10:11:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 -- 172.21.15.133:0/3735933967 shutdown_connections 2024-04-25T10:11:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 --2- 172.21.15.133:0/3735933967 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe70406e3b0 0x7fe70406e810 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 --2- 172.21.15.133:0/3735933967 >> v2:172.21.15.162:3300/0 conn(0x7fe7040b06d0 0x7fe7040b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 -- 172.21.15.133:0/3735933967 >> 172.21.15.133:0/3735933967 conn(0x7fe704066f70 msgr2=0x7fe70406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 -- 172.21.15.133:0/3735933967 shutdown_connections 2024-04-25T10:11:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 -- 172.21.15.133:0/3735933967 wait complete. 2024-04-25T10:11:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 Processor -- start 2024-04-25T10:11:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 -- start start 2024-04-25T10:11:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70406e3b0 0x7fe70415c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7040b06d0 0x7fe704156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe704143fb0 0x7fe70413dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7040b1630 con 0x7fe7040b06d0 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7040b14b0 con 0x7fe70406e3b0 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70b3b2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe704144390 con 0x7fe704143fb0 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70894d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7040b06d0 0x7fe704156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70894d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7040b06d0 0x7fe704156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37012/0 (socket says 172.21.15.133:37012) 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70894d640 1 -- 172.21.15.133:0/1053852413 learned_addr learned my addr 172.21.15.133:0/1053852413 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70914e640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70406e3b0 0x7fe70415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70994f640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe704143fb0 0x7fe70413dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70894d640 1 -- 172.21.15.133:0/1053852413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe704143fb0 msgr2=0x7fe70413dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70894d640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe704143fb0 0x7fe70413dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70894d640 1 -- 172.21.15.133:0/1053852413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70406e3b0 msgr2=0x7fe70415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70894d640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70406e3b0 0x7fe70415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.834+0000 7fe70894d640 1 -- 172.21.15.133:0/1053852413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70413ed80 con 0x7fe7040b06d0 2024-04-25T10:11:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe70894d640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7040b06d0 0x7fe704156540 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe6f401c370 tx=0x7fe6f401b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe6f9ffb640 1 -- 172.21.15.133:0/1053852413 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40ae050 con 0x7fe7040b06d0 2024-04-25T10:11:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe6f9ffb640 1 -- 172.21.15.133:0/1053852413 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f40b7070 con 0x7fe7040b06d0 2024-04-25T10:11:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe6f9ffb640 1 -- 172.21.15.133:0/1053852413 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40b3880 con 0x7fe7040b06d0 2024-04-25T10:11:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe70b3b2640 1 -- 172.21.15.133:0/1053852413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70413f060 con 0x7fe7040b06d0 2024-04-25T10:11:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe70b3b2640 1 -- 172.21.15.133:0/1053852413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70413f620 con 0x7fe7040b06d0 2024-04-25T10:11:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe70b3b2640 1 -- 172.21.15.133:0/1053852413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe704068e80 con 0x7fe7040b06d0 2024-04-25T10:11:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe6f9ffb640 1 -- 172.21.15.133:0/1053852413 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6f40b1070 con 0x7fe7040b06d0 2024-04-25T10:11:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe6f9ffb640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe6e8066f60 0x7fe6e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe6f9ffb640 1 -- 172.21.15.133:0/1053852413 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe6f4121330 con 0x7fe7040b06d0 2024-04-25T10:11:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe70914e640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe6e8066f60 0x7fe6e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.838+0000 7fe70914e640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe6e8066f60 0x7fe6e8069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe7000015b0 tx=0x7fe700009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:12.842+0000 7fe6f9ffb640 1 -- 172.21.15.133:0/1053852413 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6f4018360 con 0x7fe7040b06d0 2024-04-25T10:11:13.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.030+0000 7fe70b3b2640 1 -- 172.21.15.133:0/1053852413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe7041648c0 con 0x7fe7040b06d0 2024-04-25T10:11:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.086+0000 7fe6f9ffb640 1 -- 172.21.15.133:0/1053852413 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe6f40f2240 con 0x7fe7040b06d0 2024-04-25T10:11:13.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 -- 172.21.15.133:0/1053852413 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe6e8066f60 msgr2=0x7fe6e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe6e8066f60 0x7fe6e8069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe7000015b0 tx=0x7fe700009040 comp rx=0 tx=0).stop 2024-04-25T10:11:13.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 -- 172.21.15.133:0/1053852413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7040b06d0 msgr2=0x7fe704156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7040b06d0 0x7fe704156540 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe6f401c370 tx=0x7fe6f401b4e0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 -- 172.21.15.133:0/1053852413 shutdown_connections 2024-04-25T10:11:13.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe704143fb0 0x7fe70413dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe6e8066f60 0x7fe6e8069420 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7040b06d0 0x7fe704156540 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 --2- 172.21.15.133:0/1053852413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe70406e3b0 0x7fe70415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 -- 172.21.15.133:0/1053852413 >> 172.21.15.133:0/1053852413 conn(0x7fe704066f70 msgr2=0x7fe7040b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 -- 172.21.15.133:0/1053852413 shutdown_connections 2024-04-25T10:11:13.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.090+0000 7fe70b3b2640 1 -- 172.21.15.133:0/1053852413 wait complete. 2024-04-25T10:11:13.110 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:11:13.110 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:11:13.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.230+0000 7f84312d6640 1 Processor -- start 2024-04-25T10:11:13.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.230+0000 7f84312d6640 1 -- start start 2024-04-25T10:11:13.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f84312d6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f842c143ea0 0x7f842c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f84312d6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f842c0b2150 0x7f842c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f84312d6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c15ac50 con 0x7f842c15b9b0 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f84312d6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c0cd180 con 0x7f842c0b2150 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f84312d6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c15aed0 con 0x7f842c143ea0 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842b7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f842c15b9b0 0x7f842c15e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34844/0 (socket says 172.21.15.133:34844) 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842b7fe640 1 -- 172.21.15.133:0/2656878492 learned_addr learned my addr 172.21.15.133:0/2656878492 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241495027 0 0) 0x7f84240014a0 con 0x7f842c15b9b0 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8414003530 con 0x7f842c15b9b0 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842affd640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f842c143ea0 0x7f842c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842a7fc640 1 --2- 172.21.15.133:0/2656878492 >> v2:172.21.15.162:3300/0 conn(0x7f842c0b2150 0x7f842c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2816719287 0 0) 0x7f8414003530 con 0x7f842c15b9b0 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f842c143ea0 msgr2=0x7f842c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f842c143ea0 0x7f842c0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 >> v2:172.21.15.162:3300/0 conn(0x7f842c0b2150 msgr2=0x7f842c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 --2- 172.21.15.133:0/2656878492 >> v2:172.21.15.162:3300/0 conn(0x7f842c0b2150 0x7f842c06e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c15b1b0 con 0x7f842c15b9b0 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f8424001bd0 con 0x7f842c15b9b0 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 >> v1:172.21.15.133:6789/0 conn(0x7f842c15b9b0 legacy=0x7f842c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84140039c0 0x7f8414003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f842c143ea0 0x7f8414004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8414004d50 0x7f84140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0cd180 con 0x7f84140039c0 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c15aed0 con 0x7f8414004d50 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841400ba80 con 0x7f842c143ea0 2024-04-25T10:11:13.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842affd640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84140039c0 0x7f8414003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842a7fc640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f842c143ea0 0x7f8414004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842b7fe640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8414004d50 0x7f84140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842affd640 1 -- 172.21.15.133:0/2656878492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f842c143ea0 msgr2=0x7f8414004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842affd640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f842c143ea0 0x7f8414004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842affd640 1 -- 172.21.15.133:0/2656878492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8414004d50 msgr2=0x7f84140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842affd640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8414004d50 0x7f84140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.234+0000 7f842affd640 1 -- 172.21.15.133:0/2656878492 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8414003530 con 0x7f84140039c0 2024-04-25T10:11:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842affd640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84140039c0 0x7f8414003de0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f841c00ba60 tx=0x7f841c01c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42efc8730ce41580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c0b3070 con 0x7f84140039c0 2024-04-25T10:11:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841c0ae660 con 0x7f84140039c0 2024-04-25T10:11:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f8429ffb640 1 -- 172.21.15.133:0/2656878492 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c00b600 con 0x7f84140039c0 2024-04-25T10:11:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 -- 172.21.15.133:0/2656878492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84140039c0 msgr2=0x7f8414003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 --2- 172.21.15.133:0/2656878492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84140039c0 0x7f8414003de0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f841c00ba60 tx=0x7f841c01c680 comp rx=0 tx=0).stop 2024-04-25T10:11:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842b7fe640 1 -- 172.21.15.133:0/2656878492 reap_dead start 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 -- 172.21.15.133:0/2656878492 shutdown_connections 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 -- 172.21.15.133:0/2656878492 >> 172.21.15.133:0/2656878492 conn(0x7f842c066f70 msgr2=0x7f842c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 -- 172.21.15.133:0/2656878492 shutdown_connections 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 -- 172.21.15.133:0/2656878492 wait complete. 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 Processor -- start 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 -- start start 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84140039c0 0x7f842c140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c156590 0x7f842c156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f842c1f72d0 0x7f842c1f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c15ece0 con 0x7f842c156590 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c15eb60 con 0x7f842c1f72d0 2024-04-25T10:11:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c15ee60 con 0x7f84140039c0 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c156590 0x7f842c156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c156590 0x7f842c156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37028/0 (socket says 172.21.15.133:37028) 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842a7fc640 1 -- 172.21.15.133:0/2057917952 learned_addr learned my addr 172.21.15.133:0/2057917952 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842b7fe640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f842c1f72d0 0x7f842c1f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842affd640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84140039c0 0x7f842c140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842a7fc640 1 -- 172.21.15.133:0/2057917952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84140039c0 msgr2=0x7f842c140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842a7fc640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84140039c0 0x7f842c140090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842a7fc640 1 -- 172.21.15.133:0/2057917952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f842c1f72d0 msgr2=0x7f842c1f7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842a7fc640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f842c1f72d0 0x7f842c1f7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842a7fc640 1 -- 172.21.15.133:0/2057917952 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c151ec0 con 0x7f842c156590 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f842a7fc640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c156590 0x7f842c156a10 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f842000f600 tx=0x7f842000fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84137fe640 1 -- 172.21.15.133:0/2057917952 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420019020 con 0x7f842c156590 2024-04-25T10:11:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84137fe640 1 -- 172.21.15.133:0/2057917952 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8420014870 con 0x7f842c156590 2024-04-25T10:11:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 -- 172.21.15.133:0/2057917952 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c1f7d90 con 0x7f842c156590 2024-04-25T10:11:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84137fe640 1 -- 172.21.15.133:0/2057917952 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420014c60 con 0x7f842c156590 2024-04-25T10:11:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.238+0000 7f84312d6640 1 -- 172.21.15.133:0/2057917952 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c152180 con 0x7f842c156590 2024-04-25T10:11:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.242+0000 7f84312d6640 1 -- 172.21.15.133:0/2057917952 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c068e80 con 0x7f842c156590 2024-04-25T10:11:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.242+0000 7f84137fe640 1 -- 172.21.15.133:0/2057917952 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f842001c020 con 0x7f842c156590 2024-04-25T10:11:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.242+0000 7f84137fe640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f90 0x7f8418069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.242+0000 7f84137fe640 1 -- 172.21.15.133:0/2057917952 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f84200858f0 con 0x7f842c156590 2024-04-25T10:11:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.242+0000 7f842affd640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f90 0x7f8418069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.242+0000 7f842affd640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f90 0x7f8418069450 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f841c0b0000 tx=0x7f841c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.246+0000 7f84137fe640 1 -- 172.21.15.133:0/2057917952 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8420023020 con 0x7f842c156590 2024-04-25T10:11:13.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:13.426+0000 7f84312d6640 1 -- 172.21.15.133:0/2057917952 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f842c158490 con 0x7f842c156590 2024-04-25T10:11:13.859 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T10:11:13.860 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T10:11:13.860 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:11:13.860 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:11:14.049 INFO:tasks.ceph.mds.b.smithi162.stderr:2024-04-25T10:11:14.044+0000 7fdb95dfc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:14.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.102+0000 7f84137fe640 1 -- 172.21.15.133:0/2057917952 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8420056770 con 0x7f842c156590 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.102+0000 7f84312d6640 1 -- 172.21.15.133:0/2057917952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f90 msgr2=0x7f8418069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.102+0000 7f84312d6640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f90 0x7f8418069450 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f841c0b0000 tx=0x7f841c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.102+0000 7f84312d6640 1 -- 172.21.15.133:0/2057917952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c156590 msgr2=0x7f842c156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.102+0000 7f84312d6640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c156590 0x7f842c156a10 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f842000f600 tx=0x7f842000fac0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.102+0000 7f84312d6640 1 -- 172.21.15.133:0/2057917952 shutdown_connections 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.102+0000 7f84312d6640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f842c1f72d0 0x7f842c1f7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.106+0000 7f84312d6640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f90 0x7f8418069450 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.106+0000 7f84312d6640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c156590 0x7f842c156a10 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.106+0000 7f84312d6640 1 --2- 172.21.15.133:0/2057917952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84140039c0 0x7f842c140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.106+0000 7f84312d6640 1 -- 172.21.15.133:0/2057917952 >> 172.21.15.133:0/2057917952 conn(0x7f842c066f70 msgr2=0x7f842c15d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.106+0000 7f84312d6640 1 -- 172.21.15.133:0/2057917952 shutdown_connections 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.106+0000 7f84312d6640 1 -- 172.21.15.133:0/2057917952 wait complete. 2024-04-25T10:11:14.126 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:11:14.127 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:14.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.246+0000 7f300f2ea640 1 Processor -- start 2024-04-25T10:11:14.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.246+0000 7f300f2ea640 1 -- start start 2024-04-25T10:11:14.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.246+0000 7f300f2ea640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3008159d50 0x7f30080cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.246+0000 7f300f2ea640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30080c4d30 con 0x7f300815e210 2024-04-25T10:11:14.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.246+0000 7f300f2ea640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30080cbb00 con 0x7f300815a130 2024-04-25T10:11:14.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.246+0000 7f300f2ea640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30080cbde0 con 0x7f3008159d50 2024-04-25T10:11:14.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.246+0000 7f300d887640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f300815e210 0x7f30080cd360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34860/0 (socket says 172.21.15.133:34860) 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.246+0000 7f300d887640 1 -- 172.21.15.133:0/2048455231 learned_addr learned my addr 172.21.15.133:0/2048455231 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300d086640 1 --2- 172.21.15.133:0/2048455231 >> v2:172.21.15.162:3300/0 conn(0x7f3008159d50 0x7f30080cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892382523 0 0) 0x7f30080c4d30 con 0x7f300815e210 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff4003530 con 0x7f300815e210 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1582552692 0 0) 0x7f2ff4003530 con 0x7f300815e210 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 >> v2:172.21.15.162:3300/0 conn(0x7f3008159d50 msgr2=0x7f30080cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 --2- 172.21.15.133:0/2048455231 >> v2:172.21.15.162:3300/0 conn(0x7f3008159d50 0x7f30080cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 >> v1:172.21.15.162:6789/0 conn(0x7f300815a130 legacy=0x7f300815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30080c46e0 con 0x7f300815e210 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f3000001a00 con 0x7f300815e210 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 >> v1:172.21.15.133:6789/0 conn(0x7f300815e210 legacy=0x7f30080cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff4003b90 0x7f2ff4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff4008840 0x7f2ff400acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff400b4d0 0x7f2ff400d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30080cbde0 con 0x7f2ff4003b90 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff400e270 con 0x7f2ff400b4d0 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff400e470 con 0x7f2ff4008840 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300d086640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff4003b90 0x7f2ff4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300d887640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff400b4d0 0x7f2ff400d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300c885640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff4008840 0x7f2ff400acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300d086640 1 -- 172.21.15.133:0/2048455231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff4008840 msgr2=0x7f2ff400acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300d086640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ff4008840 0x7f2ff400acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300d086640 1 -- 172.21.15.133:0/2048455231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff400b4d0 msgr2=0x7f2ff400d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300d086640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff400b4d0 0x7f2ff400d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300d086640 1 -- 172.21.15.133:0/2048455231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4003530 con 0x7f2ff4003b90 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300d887640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff400b4d0 0x7f2ff400d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:14.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300d887640 1 -- 172.21.15.133:0/2048455231 reap_dead start 2024-04-25T10:11:14.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300d086640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff4003b90 0x7f2ff4003fb0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f3004009ae0 tx=0x7f300401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cbb5050a843a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30040b2070 con 0x7f2ff4003b90 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30040a5550 con 0x7f2ff4003b90 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f2ffffff640 1 -- 172.21.15.133:0/2048455231 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30040096b0 con 0x7f2ff4003b90 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300f2ea640 1 -- 172.21.15.133:0/2048455231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff4003b90 msgr2=0x7f2ff4003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300f2ea640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff4003b90 0x7f2ff4003fb0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f3004009ae0 tx=0x7f300401ca70 comp rx=0 tx=0).stop 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300f2ea640 1 -- 172.21.15.133:0/2048455231 shutdown_connections 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300f2ea640 1 --2- 172.21.15.133:0/2048455231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff4003b90 0x7f2ff4003fb0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300f2ea640 1 -- 172.21.15.133:0/2048455231 >> 172.21.15.133:0/2048455231 conn(0x7f3008067000 msgr2=0x7f300806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300f2ea640 1 -- 172.21.15.133:0/2048455231 shutdown_connections 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.250+0000 7f300f2ea640 1 -- 172.21.15.133:0/2048455231 wait complete. 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300f2ea640 1 Processor -- start 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300f2ea640 1 -- start start 2024-04-25T10:11:14.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300f2ea640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff4003b90 0x7f3008144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300f2ea640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30080ca5e0 0x7f30080caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300f2ea640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f300815aeb0 0x7f300815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300f2ea640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300815de90 con 0x7f30080ca5e0 2024-04-25T10:11:14.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300f2ea640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300815dd10 con 0x7f2ff4003b90 2024-04-25T10:11:14.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300f2ea640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300815e010 con 0x7f300815aeb0 2024-04-25T10:11:14.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300c885640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30080ca5e0 0x7f30080caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300c885640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30080ca5e0 0x7f30080caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37040/0 (socket says 172.21.15.133:37040) 2024-04-25T10:11:14.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300c885640 1 -- 172.21.15.133:0/618956944 learned_addr learned my addr 172.21.15.133:0/618956944 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:14.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300c885640 1 -- 172.21.15.133:0/618956944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f300815aeb0 msgr2=0x7f300815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300d887640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f300815aeb0 0x7f300815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300c885640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f300815aeb0 0x7f300815b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300c885640 1 -- 172.21.15.133:0/618956944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff4003b90 msgr2=0x7f3008144040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300d086640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff4003b90 0x7f3008144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300c885640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff4003b90 0x7f3008144040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300c885640 1 -- 172.21.15.133:0/618956944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3008144580 con 0x7f30080ca5e0 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300c885640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30080ca5e0 0x7f30080caa60 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800d5a0 tx=0x7f2ff800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f2ffdffb640 1 -- 172.21.15.133:0/618956944 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8005480 con 0x7f30080ca5e0 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f2ffdffb640 1 -- 172.21.15.133:0/618956944 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff8017070 con 0x7f30080ca5e0 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300f2ea640 1 -- 172.21.15.133:0/618956944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300815bbe0 con 0x7f30080ca5e0 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f2ffdffb640 1 -- 172.21.15.133:0/618956944 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8003820 con 0x7f30080ca5e0 2024-04-25T10:11:14.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.254+0000 7f300f2ea640 1 -- 172.21.15.133:0/618956944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300815c1a0 con 0x7f30080ca5e0 2024-04-25T10:11:14.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.258+0000 7f2ffdffb640 1 -- 172.21.15.133:0/618956944 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ff80039c0 con 0x7f30080ca5e0 2024-04-25T10:11:14.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.258+0000 7f300f2ea640 1 -- 172.21.15.133:0/618956944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30080cafa0 con 0x7f30080ca5e0 2024-04-25T10:11:14.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.258+0000 7f2ffdffb640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fec066f60 0x7f2fec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.258+0000 7f300d086640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fec066f60 0x7f2fec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.258+0000 7f2ffdffb640 1 -- 172.21.15.133:0/618956944 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ff8084f70 con 0x7f30080ca5e0 2024-04-25T10:11:14.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.258+0000 7f300d086640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fec066f60 0x7f2fec069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f30040af000 tx=0x7f3004008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.262+0000 7f2ffdffb640 1 -- 172.21.15.133:0/618956944 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ff8019310 con 0x7f30080ca5e0 2024-04-25T10:11:14.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f300f2ea640 1 -- 172.21.15.133:0/618956944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30080c4d30 con 0x7f30080ca5e0 2024-04-25T10:11:14.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f2ffdffb640 1 -- 172.21.15.133:0/618956944 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f2ff8055df0 con 0x7f30080ca5e0 2024-04-25T10:11:14.441 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:14.441 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:14.441 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:14.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f300f2ea640 1 -- 172.21.15.133:0/618956944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fec066f60 msgr2=0x7f2fec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f300f2ea640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fec066f60 0x7f2fec069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f30040af000 tx=0x7f3004008040 comp rx=0 tx=0).stop 2024-04-25T10:11:14.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f300f2ea640 1 -- 172.21.15.133:0/618956944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30080ca5e0 msgr2=0x7f30080caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f300f2ea640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30080ca5e0 0x7f30080caa60 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800d5a0 tx=0x7f2ff800da60 comp rx=0 tx=0).stop 2024-04-25T10:11:14.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f300f2ea640 1 -- 172.21.15.133:0/618956944 shutdown_connections 2024-04-25T10:11:14.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f300f2ea640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f300815aeb0 0x7f300815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f300f2ea640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fec066f60 0x7f2fec069420 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f300f2ea640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30080ca5e0 0x7f30080caa60 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f300f2ea640 1 --2- 172.21.15.133:0/618956944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ff4003b90 0x7f3008144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.438+0000 7f300f2ea640 1 -- 172.21.15.133:0/618956944 >> 172.21.15.133:0/618956944 conn(0x7f3008067000 msgr2=0x7f30080b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.442+0000 7f300f2ea640 1 -- 172.21.15.133:0/618956944 shutdown_connections 2024-04-25T10:11:14.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.442+0000 7f300f2ea640 1 -- 172.21.15.133:0/618956944 wait complete. 2024-04-25T10:11:14.459 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f062cf53640 1 Processor -- start 2024-04-25T10:11:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f062cf53640 1 -- start start 2024-04-25T10:11:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f062cf53640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f06280b1f70 0x7f062806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f062cf53640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06280c61d0 con 0x7f062815c9a0 2024-04-25T10:11:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f062cf53640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06280c64b0 con 0x7f06280ce330 2024-04-25T10:11:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f062cf53640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062806d4f0 con 0x7f06280b1f70 2024-04-25T10:11:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f0625d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f06280b1f70 0x7f062806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f062659c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f062815c9a0 0x7f06280c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34870/0 (socket says 172.21.15.133:34870) 2024-04-25T10:11:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f062659c640 1 -- 172.21.15.133:0/1964410508 learned_addr learned my addr 172.21.15.133:0/1964410508 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f0625d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f06280b1f70 0x7f062806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37052/0 (socket says 172.21.15.133:37052) 2024-04-25T10:11:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f0625d9b640 1 -- 172.21.15.133:0/1964410508 >> v1:172.21.15.162:6789/0 conn(0x7f06280ce330 legacy=0x7f062815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f0625d9b640 1 -- 172.21.15.133:0/1964410508 >> v1:172.21.15.133:6789/0 conn(0x7f062815c9a0 legacy=0x7f06280c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f0625d9b640 1 -- 172.21.15.133:0/1964410508 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062806d7d0 con 0x7f06280b1f70 2024-04-25T10:11:14.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.578+0000 7f062559a640 1 -- 172.21.15.133:0/1964410508 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933729229 0 0) 0x7f06280c61d0 con 0x7f062815c9a0 2024-04-25T10:11:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f0625d9b640 1 --2- 172.21.15.133:0/1964410508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06280b1f70 0x7f062806dcf0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f061c009d90 tx=0x7f061c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b378b808a72db47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062559a640 1 -- 172.21.15.133:0/1964410508 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c0b2070 con 0x7f06280b1f70 2024-04-25T10:11:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062559a640 1 -- 172.21.15.133:0/1964410508 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c0a5a50 con 0x7f06280b1f70 2024-04-25T10:11:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062559a640 1 -- 172.21.15.133:0/1964410508 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c0a5e20 con 0x7f06280b1f70 2024-04-25T10:11:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 -- 172.21.15.133:0/1964410508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06280b1f70 msgr2=0x7f062806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 --2- 172.21.15.133:0/1964410508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06280b1f70 0x7f062806dcf0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f061c009d90 tx=0x7f061c01c970 comp rx=0 tx=0).stop 2024-04-25T10:11:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 -- 172.21.15.133:0/1964410508 shutdown_connections 2024-04-25T10:11:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 --2- 172.21.15.133:0/1964410508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06280b1f70 0x7f062806dcf0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 -- 172.21.15.133:0/1964410508 >> 172.21.15.133:0/1964410508 conn(0x7f0628067000 msgr2=0x7f062806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 -- 172.21.15.133:0/1964410508 shutdown_connections 2024-04-25T10:11:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 -- 172.21.15.133:0/1964410508 wait complete. 2024-04-25T10:11:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 Processor -- start 2024-04-25T10:11:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 -- start start 2024-04-25T10:11:14.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06280b1f70 0x7f0628148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06280ce330 0x7f0628149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f062815c9a0 0x7f062806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062815cf00 con 0x7f06280ce330 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062815cd80 con 0x7f062815c9a0 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f062cf53640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0628169de0 con 0x7f06280b1f70 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f0625d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06280ce330 0x7f0628149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f0625d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06280ce330 0x7f0628149250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37068/0 (socket says 172.21.15.133:37068) 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.582+0000 7f0625d9b640 1 -- 172.21.15.133:0/1634718851 learned_addr learned my addr 172.21.15.133:0/1634718851 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f062659c640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06280b1f70 0x7f0628148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0626d9d640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f062815c9a0 0x7f062806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0625d9b640 1 -- 172.21.15.133:0/1634718851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06280b1f70 msgr2=0x7f0628148ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0625d9b640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06280b1f70 0x7f0628148ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0625d9b640 1 -- 172.21.15.133:0/1634718851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f062815c9a0 msgr2=0x7f062806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0625d9b640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f062815c9a0 0x7f062806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0625d9b640 1 -- 172.21.15.133:0/1634718851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062814bcd0 con 0x7f06280ce330 2024-04-25T10:11:14.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0625d9b640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06280ce330 0x7f0628149250 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f061c0b0040 tx=0x7f061c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0616ffd640 1 -- 172.21.15.133:0/1634718851 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c0b2050 con 0x7f06280ce330 2024-04-25T10:11:14.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0616ffd640 1 -- 172.21.15.133:0/1634718851 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c0be070 con 0x7f06280ce330 2024-04-25T10:11:14.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f062cf53640 1 -- 172.21.15.133:0/1634718851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062814bfb0 con 0x7f06280ce330 2024-04-25T10:11:14.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0616ffd640 1 -- 172.21.15.133:0/1634718851 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c0b9690 con 0x7f06280ce330 2024-04-25T10:11:14.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f062cf53640 1 -- 172.21.15.133:0/1634718851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062814c570 con 0x7f06280ce330 2024-04-25T10:11:14.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f062cf53640 1 -- 172.21.15.133:0/1634718851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0628068ef0 con 0x7f06280ce330 2024-04-25T10:11:14.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0616ffd640 1 -- 172.21.15.133:0/1634718851 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f061c0b9830 con 0x7f06280ce330 2024-04-25T10:11:14.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0616ffd640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0608066f60 0x7f0608069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f0616ffd640 1 -- 172.21.15.133:0/1634718851 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f061c120a90 con 0x7f06280ce330 2024-04-25T10:11:14.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f062659c640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0608066f60 0x7f0608069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.586+0000 7f062659c640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0608066f60 0x7f0608069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0618007fd0 tx=0x7f0618007ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.590+0000 7f0616ffd640 1 -- 172.21.15.133:0/1634718851 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f061c016560 con 0x7f06280ce330 2024-04-25T10:11:14.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.750+0000 7f062cf53640 1 -- 172.21.15.133:0/1634718851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f062815e1f0 con 0x7f06280ce330 2024-04-25T10:11:14.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.750+0000 7f0616ffd640 1 -- 172.21.15.133:0/1634718851 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16908 (secure 0 0 0) 0x7f061c0f1940 con 0x7f06280ce330 2024-04-25T10:11:14.754 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:14.754 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":107,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:11:13.077557+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:09.673185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:10.421023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:14.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 -- 172.21.15.133:0/1634718851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0608066f60 msgr2=0x7f0608069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0608066f60 0x7f0608069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0618007fd0 tx=0x7f0618007ba0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 -- 172.21.15.133:0/1634718851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06280ce330 msgr2=0x7f0628149250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06280ce330 0x7f0628149250 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f061c0b0040 tx=0x7f061c019710 comp rx=0 tx=0).stop 2024-04-25T10:11:14.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 -- 172.21.15.133:0/1634718851 shutdown_connections 2024-04-25T10:11:14.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f062815c9a0 0x7f062806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0608066f60 0x7f0608069420 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06280ce330 0x7f0628149250 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 --2- 172.21.15.133:0/1634718851 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06280b1f70 0x7f0628148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 -- 172.21.15.133:0/1634718851 >> 172.21.15.133:0/1634718851 conn(0x7f0628067000 msgr2=0x7f062815afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 -- 172.21.15.133:0/1634718851 shutdown_connections 2024-04-25T10:11:14.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.754+0000 7f062cf53640 1 -- 172.21.15.133:0/1634718851 wait complete. 2024-04-25T10:11:14.774 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:11:14.774 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:14.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9defcd8640 1 Processor -- start 2024-04-25T10:11:14.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9defcd8640 1 -- start start 2024-04-25T10:11:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9defcd8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9de815c9a0 0x7f9de80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9defcd8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9de80b1f70 0x7f9de806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9defcd8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de80c61d0 con 0x7f9de80ce330 2024-04-25T10:11:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9defcd8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de80c64b0 con 0x7f9de815c9a0 2024-04-25T10:11:14.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9defcd8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de806d4f0 con 0x7f9de80b1f70 2024-04-25T10:11:14.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9deda74640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9de815c9a0 0x7f9de80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9deda74640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9de815c9a0 0x7f9de80c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37082/0 (socket says 172.21.15.133:37082) 2024-04-25T10:11:14.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9deda74640 1 -- 172.21.15.133:0/608625654 learned_addr learned my addr 172.21.15.133:0/608625654 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:14.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9ded273640 1 --2- 172.21.15.133:0/608625654 >> v2:172.21.15.162:3300/0 conn(0x7f9de80b1f70 0x7f9de806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9deda74640 1 -- 172.21.15.133:0/608625654 >> v2:172.21.15.162:3300/0 conn(0x7f9de80b1f70 msgr2=0x7f9de806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9deda74640 1 --2- 172.21.15.133:0/608625654 >> v2:172.21.15.162:3300/0 conn(0x7f9de80b1f70 0x7f9de806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9deda74640 1 -- 172.21.15.133:0/608625654 >> v1:172.21.15.162:6789/0 conn(0x7f9de80ce330 legacy=0x7f9de815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9deda74640 1 -- 172.21.15.133:0/608625654 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de806d7d0 con 0x7f9de815c9a0 2024-04-25T10:11:14.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.894+0000 7f9ded273640 1 --2- 172.21.15.133:0/608625654 >> v2:172.21.15.162:3300/0 conn(0x7f9de80b1f70 0x7f9de806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:14.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9deda74640 1 --2- 172.21.15.133:0/608625654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de815c9a0 0x7f9de80c5980 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f9de8072670 tx=0x7f9de001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baa292f324800c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9deca72640 1 -- 172.21.15.133:0/608625654 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00b2070 con 0x7f9de815c9a0 2024-04-25T10:11:14.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9deca72640 1 -- 172.21.15.133:0/608625654 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de00ad640 con 0x7f9de815c9a0 2024-04-25T10:11:14.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9deca72640 1 -- 172.21.15.133:0/608625654 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00ada10 con 0x7f9de815c9a0 2024-04-25T10:11:14.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 -- 172.21.15.133:0/608625654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de815c9a0 msgr2=0x7f9de80c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 --2- 172.21.15.133:0/608625654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de815c9a0 0x7f9de80c5980 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f9de8072670 tx=0x7f9de001c480 comp rx=0 tx=0).stop 2024-04-25T10:11:14.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 -- 172.21.15.133:0/608625654 shutdown_connections 2024-04-25T10:11:14.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 --2- 172.21.15.133:0/608625654 >> v2:172.21.15.162:3300/0 conn(0x7f9de80b1f70 0x7f9de806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 --2- 172.21.15.133:0/608625654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de815c9a0 0x7f9de80c5980 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 -- 172.21.15.133:0/608625654 >> 172.21.15.133:0/608625654 conn(0x7f9de8067000 msgr2=0x7f9de806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 -- 172.21.15.133:0/608625654 shutdown_connections 2024-04-25T10:11:14.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 -- 172.21.15.133:0/608625654 wait complete. 2024-04-25T10:11:14.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 Processor -- start 2024-04-25T10:11:14.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 -- start start 2024-04-25T10:11:14.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de80b1f70 0x7f9de8148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de80ce330 0x7f9de814a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de815c9a0 0x7f9de80b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de815cd80 con 0x7f9de815c9a0 2024-04-25T10:11:14.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de806e230 con 0x7f9de80b1f70 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9defcd8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de815cf00 con 0x7f9de80ce330 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9dee275640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de815c9a0 0x7f9de80b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9dee275640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de815c9a0 0x7f9de80b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37098/0 (socket says 172.21.15.133:37098) 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.898+0000 7f9dee275640 1 -- 172.21.15.133:0/1211102668 learned_addr learned my addr 172.21.15.133:0/1211102668 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9deda74640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de80b1f70 0x7f9de8148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9ded273640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de80ce330 0x7f9de814a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dee275640 1 -- 172.21.15.133:0/1211102668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de80ce330 msgr2=0x7f9de814a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dee275640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de80ce330 0x7f9de814a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dee275640 1 -- 172.21.15.133:0/1211102668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de80b1f70 msgr2=0x7f9de8148b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dee275640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de80b1f70 0x7f9de8148b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dee275640 1 -- 172.21.15.133:0/1211102668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de80ce000 con 0x7f9de815c9a0 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9ded273640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de80ce330 0x7f9de814a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dee275640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de815c9a0 0x7f9de80b2490 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00b440 tx=0x7f9ddc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dda7fc640 1 -- 172.21.15.133:0/1211102668 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc015020 con 0x7f9de815c9a0 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dda7fc640 1 -- 172.21.15.133:0/1211102668 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc011850 con 0x7f9de815c9a0 2024-04-25T10:11:14.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9defcd8640 1 -- 172.21.15.133:0/1211102668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de8163910 con 0x7f9de815c9a0 2024-04-25T10:11:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dda7fc640 1 -- 172.21.15.133:0/1211102668 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc011c40 con 0x7f9de815c9a0 2024-04-25T10:11:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9defcd8640 1 -- 172.21.15.133:0/1211102668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de8163ed0 con 0x7f9de815c9a0 2024-04-25T10:11:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9defcd8640 1 -- 172.21.15.133:0/1211102668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de8068ef0 con 0x7f9de815c9a0 2024-04-25T10:11:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dda7fc640 1 -- 172.21.15.133:0/1211102668 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ddc018020 con 0x7f9de815c9a0 2024-04-25T10:11:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dda7fc640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9dcc066f60 0x7f9dcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9dda7fc640 1 -- 172.21.15.133:0/1211102668 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ddc084e80 con 0x7f9de815c9a0 2024-04-25T10:11:14.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9deda74640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9dcc066f60 0x7f9dcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.902+0000 7f9deda74640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9dcc066f60 0x7f9dcc069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9de00af000 tx=0x7f9de0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:14.906+0000 7f9dda7fc640 1 -- 172.21.15.133:0/1211102668 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ddc01f030 con 0x7f9de815c9a0 2024-04-25T10:11:15.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.058+0000 7f9defcd8640 1 -- 172.21.15.133:0/1211102668 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9de8152e90 con 0x7f9dcc066f60 2024-04-25T10:11:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.066+0000 7f9dda7fc640 1 -- 172.21.15.133:0/1211102668 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237325 (secure 0 0 0) 0x7f9de8152e90 con 0x7f9dcc066f60 2024-04-25T10:11:15.072 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:15.073 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:11:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 -- 172.21.15.133:0/1211102668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9dcc066f60 msgr2=0x7f9dcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9dcc066f60 0x7f9dcc069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9de00af000 tx=0x7f9de0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 -- 172.21.15.133:0/1211102668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de815c9a0 msgr2=0x7f9de80b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de815c9a0 0x7f9de80b2490 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00b440 tx=0x7f9ddc00b900 comp rx=0 tx=0).stop 2024-04-25T10:11:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 -- 172.21.15.133:0/1211102668 shutdown_connections 2024-04-25T10:11:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9dcc066f60 0x7f9dcc069420 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de815c9a0 0x7f9de80b2490 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9de80ce330 0x7f9de814a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 --2- 172.21.15.133:0/1211102668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9de80b1f70 0x7f9de8148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 -- 172.21.15.133:0/1211102668 >> 172.21.15.133:0/1211102668 conn(0x7f9de8067000 msgr2=0x7f9de80b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 -- 172.21.15.133:0/1211102668 shutdown_connections 2024-04-25T10:11:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.070+0000 7f9defcd8640 1 -- 172.21.15.133:0/1211102668 wait complete. 2024-04-25T10:11:15.087 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-04-25T10:11:13.877881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":497960,"kb_used_data":11120,"kb_used_omap":12,"kb_used_meta":486707,"kb_avail":754476760,"statfs":{"total":773094113280,"available":772584202240,"internally_reserved":0,"allocated":11386880,"data_stored":5899254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":498388529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001113"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":326,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:11:09.367161+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:11:09.367161+0000","last_peered":"2024-04-25T10:11:09.367161+0000","last_clean":"2024-04-25T10:11:09.367161+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:11:09.367161+0000","last_undegraded":"2024-04-25T10:11:09.367161+0000","last_fullsized":"2024-04-25T10:11:09.367161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64920,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306920,"statfs":{"total":96636764160,"available":96570286080,"internally_reserved":0,"allocated":1847296,"data_stored":1170647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58800,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313040,"statfs":{"total":96636764160,"available":96576552960,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54640,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317200,"statfs":{"total":96636764160,"available":96580812800,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68532,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303308,"statfs":{"total":96636764160,"available":96566587392,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72684,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299156,"statfs":{"total":96636764160,"available":96562335744,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58800,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313040,"statfs":{"total":96636764160,"available":96576552960,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54636,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317204,"statfs":{"total":96636764160,"available":96580816896,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64948,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306892,"statfs":{"total":96636764160,"available":96570257408,"internally_reserved":0,"allocated":1871872,"data_stored":1181545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:15.092 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd60cc0c640 1 Processor -- start 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd60cc0c640 1 -- start start 2024-04-25T10:11:15.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd60cc0c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd60815a730 0x7fd6080669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd60cc0c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd608066ee0 0x7fd60806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd60cc0c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd608070150 con 0x7fd60806c1e0 2024-04-25T10:11:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd60cc0c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd60806c630 con 0x7fd608066ee0 2024-04-25T10:11:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd60cc0c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6080acd10 con 0x7fd60815a730 2024-04-25T10:11:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd606d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd60806c1e0 0x7fd6080b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34882/0 (socket says 172.21.15.133:34882) 2024-04-25T10:11:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd606d9d640 1 -- 172.21.15.133:0/1650403855 learned_addr learned my addr 172.21.15.133:0/1650403855 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:15.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd605d9b640 1 --2- 172.21.15.133:0/1650403855 >> v2:172.21.15.133:3300/0 conn(0x7fd608066ee0 0x7fd60806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd60659c640 1 --2- 172.21.15.133:0/1650403855 >> v2:172.21.15.162:3300/0 conn(0x7fd60815a730 0x7fd6080669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd60559a640 1 -- 172.21.15.133:0/1650403855 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513215752 0 0) 0x7fd5f8001120 con 0x7fd60806c1e0 2024-04-25T10:11:15.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd60559a640 1 -- 172.21.15.133:0/1650403855 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5f0003530 con 0x7fd60806c1e0 2024-04-25T10:11:15.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd605d9b640 1 -- 172.21.15.133:0/1650403855 >> v2:172.21.15.162:3300/0 conn(0x7fd60815a730 msgr2=0x7fd6080669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd605d9b640 1 --2- 172.21.15.133:0/1650403855 >> v2:172.21.15.162:3300/0 conn(0x7fd60815a730 0x7fd6080669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd605d9b640 1 -- 172.21.15.133:0/1650403855 >> v1:172.21.15.133:6789/0 conn(0x7fd60806c1e0 legacy=0x7fd6080b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd605d9b640 1 -- 172.21.15.133:0/1650403855 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080acff0 con 0x7fd608066ee0 2024-04-25T10:11:15.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.250+0000 7fd60659c640 1 --2- 172.21.15.133:0/1650403855 >> v2:172.21.15.162:3300/0 conn(0x7fd60815a730 0x7fd6080669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd605d9b640 1 --2- 172.21.15.133:0/1650403855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd608066ee0 0x7fd60806bca0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fd608071720 tx=0x7fd5fc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2840325dfc5555c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd60559a640 1 -- 172.21.15.133:0/1650403855 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0b0070 con 0x7fd608066ee0 2024-04-25T10:11:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd60559a640 1 -- 172.21.15.133:0/1650403855 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc0ab600 con 0x7fd608066ee0 2024-04-25T10:11:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd60559a640 1 -- 172.21.15.133:0/1650403855 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0ab9d0 con 0x7fd608066ee0 2024-04-25T10:11:15.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd60cc0c640 1 -- 172.21.15.133:0/1650403855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd608066ee0 msgr2=0x7fd60806bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd60cc0c640 1 --2- 172.21.15.133:0/1650403855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd608066ee0 0x7fd60806bca0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fd608071720 tx=0x7fd5fc01c450 comp rx=0 tx=0).stop 2024-04-25T10:11:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd60cc0c640 1 -- 172.21.15.133:0/1650403855 shutdown_connections 2024-04-25T10:11:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd60cc0c640 1 --2- 172.21.15.133:0/1650403855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd608066ee0 0x7fd60806bca0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd60cc0c640 1 --2- 172.21.15.133:0/1650403855 >> v2:172.21.15.162:3300/0 conn(0x7fd60815a730 0x7fd6080669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd60cc0c640 1 -- 172.21.15.133:0/1650403855 >> 172.21.15.133:0/1650403855 conn(0x7fd608071aa0 msgr2=0x7fd608071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd60cc0c640 1 -- 172.21.15.133:0/1650403855 shutdown_connections 2024-04-25T10:11:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.254+0000 7fd60cc0c640 1 -- 172.21.15.133:0/1650403855 wait complete. 2024-04-25T10:11:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60cc0c640 1 Processor -- start 2024-04-25T10:11:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60cc0c640 1 -- start start 2024-04-25T10:11:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60cc0c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd608066ee0 0x7fd6081583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60cc0c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd60806c1e0 0x7fd60814faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60cc0c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd60815a730 0x7fd60815ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60cc0c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080cc6b0 con 0x7fd60815a730 2024-04-25T10:11:15.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60cc0c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080adb70 con 0x7fd60806c1e0 2024-04-25T10:11:15.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60cc0c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080cc830 con 0x7fd608066ee0 2024-04-25T10:11:15.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd606d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd60815a730 0x7fd60815ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd606d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd60815a730 0x7fd60815ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37116/0 (socket says 172.21.15.133:37116) 2024-04-25T10:11:15.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd606d9d640 1 -- 172.21.15.133:0/2786448846 learned_addr learned my addr 172.21.15.133:0/2786448846 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:15.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60659c640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd608066ee0 0x7fd6081583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd605d9b640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd60806c1e0 0x7fd60814faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd606d9d640 1 -- 172.21.15.133:0/2786448846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd608066ee0 msgr2=0x7fd6081583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd606d9d640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd608066ee0 0x7fd6081583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd606d9d640 1 -- 172.21.15.133:0/2786448846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd60806c1e0 msgr2=0x7fd60814faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd606d9d640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd60806c1e0 0x7fd60814faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd606d9d640 1 -- 172.21.15.133:0/2786448846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd608157330 con 0x7fd60815a730 2024-04-25T10:11:15.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd605d9b640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd60806c1e0 0x7fd60814faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:15.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd606d9d640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd60815a730 0x7fd60815ca70 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800c730 tx=0x7fd5f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd5f6ffd640 1 -- 172.21.15.133:0/2786448846 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8017070 con 0x7fd60815a730 2024-04-25T10:11:15.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd5f6ffd640 1 -- 172.21.15.133:0/2786448846 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f80058c0 con 0x7fd60815a730 2024-04-25T10:11:15.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd5f6ffd640 1 -- 172.21.15.133:0/2786448846 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8005cd0 con 0x7fd60815a730 2024-04-25T10:11:15.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60cc0c640 1 -- 172.21.15.133:0/2786448846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd608157610 con 0x7fd60815a730 2024-04-25T10:11:15.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60cc0c640 1 -- 172.21.15.133:0/2786448846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd608157bd0 con 0x7fd60815a730 2024-04-25T10:11:15.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.258+0000 7fd60cc0c640 1 -- 172.21.15.133:0/2786448846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd608067c70 con 0x7fd60815a730 2024-04-25T10:11:15.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.262+0000 7fd5f6ffd640 1 -- 172.21.15.133:0/2786448846 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5f801a070 con 0x7fd60815a730 2024-04-25T10:11:15.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.262+0000 7fd5f6ffd640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd5e8066f60 0x7fd5e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.262+0000 7fd5f6ffd640 1 -- 172.21.15.133:0/2786448846 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd5f80844e0 con 0x7fd60815a730 2024-04-25T10:11:15.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.262+0000 7fd60659c640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd5e8066f60 0x7fd5e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.262+0000 7fd60659c640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd5e8066f60 0x7fd5e8069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd6080cecd0 tx=0x7fd6000039a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.266+0000 7fd5f6ffd640 1 -- 172.21.15.133:0/2786448846 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd608070150 con 0x7fd60815a730 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.422+0000 7fd60cc0c640 1 -- 172.21.15.133:0/2786448846 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6081555b0 con 0x7fd5e8066f60 2024-04-25T10:11:15.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.426+0000 7fd5f6ffd640 1 -- 172.21.15.133:0/2786448846 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237325 (secure 0 0 0) 0x7fd6081555b0 con 0x7fd5e8066f60 2024-04-25T10:11:15.432 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:15.434 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 -- 172.21.15.133:0/2786448846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd5e8066f60 msgr2=0x7fd5e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd5e8066f60 0x7fd5e8069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd6080cecd0 tx=0x7fd6000039a0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 -- 172.21.15.133:0/2786448846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd60815a730 msgr2=0x7fd60815ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd60815a730 0x7fd60815ca70 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800c730 tx=0x7fd5f800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 -- 172.21.15.133:0/2786448846 shutdown_connections 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd5e8066f60 0x7fd5e8069420 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd60815a730 0x7fd60815ca70 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd60806c1e0 0x7fd60814faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 --2- 172.21.15.133:0/2786448846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd608066ee0 0x7fd6081583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 -- 172.21.15.133:0/2786448846 >> 172.21.15.133:0/2786448846 conn(0x7fd608071aa0 msgr2=0x7fd608155db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 -- 172.21.15.133:0/2786448846 shutdown_connections 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.430+0000 7fd60cc0c640 1 -- 172.21.15.133:0/2786448846 wait complete. 2024-04-25T10:11:15.451 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-04-25T10:11:13.877881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":497960,"kb_used_data":11120,"kb_used_omap":12,"kb_used_meta":486707,"kb_avail":754476760,"statfs":{"total":773094113280,"available":772584202240,"internally_reserved":0,"allocated":11386880,"data_stored":5899254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":498388529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001113"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":326,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:11:09.367161+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:11:09.367161+0000","last_peered":"2024-04-25T10:11:09.367161+0000","last_clean":"2024-04-25T10:11:09.367161+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:11:09.367161+0000","last_undegraded":"2024-04-25T10:11:09.367161+0000","last_fullsized":"2024-04-25T10:11:09.367161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64920,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306920,"statfs":{"total":96636764160,"available":96570286080,"internally_reserved":0,"allocated":1847296,"data_stored":1170647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58800,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313040,"statfs":{"total":96636764160,"available":96576552960,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54640,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317200,"statfs":{"total":96636764160,"available":96580812800,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68532,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303308,"statfs":{"total":96636764160,"available":96566587392,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72684,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299156,"statfs":{"total":96636764160,"available":96562335744,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58800,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313040,"statfs":{"total":96636764160,"available":96576552960,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54636,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317204,"statfs":{"total":96636764160,"available":96580816896,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64948,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306892,"statfs":{"total":96636764160,"available":96570257408,"internally_reserved":0,"allocated":1871872,"data_stored":1181545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:15.455 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:15.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed0a467640 1 Processor -- start 2024-04-25T10:11:15.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed0a467640 1 -- start start 2024-04-25T10:11:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed0a467640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed0415db00 0x7fed040b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed0a467640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed040ceb20 con 0x7fed040d0770 2024-04-25T10:11:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed0a467640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed040b97f0 con 0x7fed040d0bd0 2024-04-25T10:11:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed0a467640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed041622e0 con 0x7fed0415db00 2024-04-25T10:11:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed03fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed0415db00 0x7fed040b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed03fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed0415db00 0x7fed040b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37132/0 (socket says 172.21.15.133:37132) 2024-04-25T10:11:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed03fff640 1 -- 172.21.15.133:0/2114128340 learned_addr learned my addr 172.21.15.133:0/2114128340 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fecfb7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fed040d0770 0x7fed041614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34892/0 (socket says 172.21.15.133:34892) 2024-04-25T10:11:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed03fff640 1 -- 172.21.15.133:0/2114128340 >> v1:172.21.15.162:6789/0 conn(0x7fed040d0bd0 legacy=0x7fed04161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed03fff640 1 -- 172.21.15.133:0/2114128340 >> v1:172.21.15.133:6789/0 conn(0x7fed040d0770 legacy=0x7fed041614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.614+0000 7fed03fff640 1 -- 172.21.15.133:0/2114128340 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed040b7740 con 0x7fed0415db00 2024-04-25T10:11:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed03fff640 1 --2- 172.21.15.133:0/2114128340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed0415db00 0x7fed040b2990 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009870 tx=0x7fecf401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d95c526315487f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed037fe640 1 -- 172.21.15.133:0/2114128340 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40b2070 con 0x7fed0415db00 2024-04-25T10:11:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed037fe640 1 -- 172.21.15.133:0/2114128340 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf40ad640 con 0x7fed0415db00 2024-04-25T10:11:15.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed037fe640 1 -- 172.21.15.133:0/2114128340 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40ada10 con 0x7fed0415db00 2024-04-25T10:11:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 -- 172.21.15.133:0/2114128340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed0415db00 msgr2=0x7fed040b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 --2- 172.21.15.133:0/2114128340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed0415db00 0x7fed040b2990 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009870 tx=0x7fecf401c3b0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 -- 172.21.15.133:0/2114128340 shutdown_connections 2024-04-25T10:11:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 --2- 172.21.15.133:0/2114128340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed0415db00 0x7fed040b2990 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 -- 172.21.15.133:0/2114128340 >> 172.21.15.133:0/2114128340 conn(0x7fed040727c0 msgr2=0x7fed04072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 -- 172.21.15.133:0/2114128340 shutdown_connections 2024-04-25T10:11:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 -- 172.21.15.133:0/2114128340 wait complete. 2024-04-25T10:11:15.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 Processor -- start 2024-04-25T10:11:15.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 -- start start 2024-04-25T10:11:15.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed040d0770 0x7fed04152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed040d0bd0 0x7fed041515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed0415db00 0x7fed0415b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed040b7fe0 con 0x7fed040d0770 2024-04-25T10:11:15.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed040b7d00 con 0x7fed040d0bd0 2024-04-25T10:11:15.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed0a467640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed040b82c0 con 0x7fed0415db00 2024-04-25T10:11:15.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed03fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed040d0770 0x7fed04152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed03fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed040d0770 0x7fed04152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37136/0 (socket says 172.21.15.133:37136) 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed03fff640 1 -- 172.21.15.133:0/2233322410 learned_addr learned my addr 172.21.15.133:0/2233322410 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed08a04640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed0415db00 0x7fed0415b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fecfb7fe640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed040d0bd0 0x7fed041515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed03fff640 1 -- 172.21.15.133:0/2233322410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed0415db00 msgr2=0x7fed0415b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed03fff640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed0415db00 0x7fed0415b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed03fff640 1 -- 172.21.15.133:0/2233322410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed040d0bd0 msgr2=0x7fed041515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed03fff640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed040d0bd0 0x7fed041515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed03fff640 1 -- 172.21.15.133:0/2233322410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04154fa0 con 0x7fed040d0770 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed08a04640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed0415db00 0x7fed0415b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed03fff640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed040d0770 0x7fed04152ca0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fecf4019fe0 tx=0x7fecf4019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed017fa640 1 -- 172.21.15.133:0/2233322410 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40b2040 con 0x7fed040d0770 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed017fa640 1 -- 172.21.15.133:0/2233322410 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf40bb070 con 0x7fed040d0770 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.618+0000 7fed017fa640 1 -- 172.21.15.133:0/2233322410 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40abb30 con 0x7fed040d0770 2024-04-25T10:11:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.622+0000 7fed0a467640 1 -- 172.21.15.133:0/2233322410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed04155260 con 0x7fed040d0770 2024-04-25T10:11:15.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.622+0000 7fed0a467640 1 -- 172.21.15.133:0/2233322410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed04155820 con 0x7fed040d0770 2024-04-25T10:11:15.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.622+0000 7fed0a467640 1 -- 172.21.15.133:0/2233322410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed04068ef0 con 0x7fed040d0770 2024-04-25T10:11:15.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.622+0000 7fed017fa640 1 -- 172.21.15.133:0/2233322410 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fecf40abcd0 con 0x7fed040d0770 2024-04-25T10:11:15.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.622+0000 7fed017fa640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fece4066f60 0x7fece4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.622+0000 7fed017fa640 1 -- 172.21.15.133:0/2233322410 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fecf411f610 con 0x7fed040d0770 2024-04-25T10:11:15.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.622+0000 7fecfb7fe640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fece4066f60 0x7fece4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.622+0000 7fecfb7fe640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fece4066f60 0x7fece4069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fecec001ec0 tx=0x7fecec00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.626+0000 7fed017fa640 1 -- 172.21.15.133:0/2233322410 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fecf4016070 con 0x7fed040d0770 2024-04-25T10:11:15.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.770+0000 7fed0a467640 1 -- 172.21.15.133:0/2233322410 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed040b3a40 con 0x7fece4066f60 2024-04-25T10:11:15.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.774+0000 7fed017fa640 1 -- 172.21.15.133:0/2233322410 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237325 (secure 0 0 0) 0x7fed040b3a40 con 0x7fece4066f60 2024-04-25T10:11:15.781 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:11:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 -- 172.21.15.133:0/2233322410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fece4066f60 msgr2=0x7fece4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fece4066f60 0x7fece4069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fecec001ec0 tx=0x7fecec00b040 comp rx=0 tx=0).stop 2024-04-25T10:11:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 -- 172.21.15.133:0/2233322410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed040d0770 msgr2=0x7fed04152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed040d0770 0x7fed04152ca0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fecf4019fe0 tx=0x7fecf4019150 comp rx=0 tx=0).stop 2024-04-25T10:11:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 -- 172.21.15.133:0/2233322410 shutdown_connections 2024-04-25T10:11:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed0415db00 0x7fed0415b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed040d0bd0 0x7fed041515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fece4066f60 0x7fece4069420 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 --2- 172.21.15.133:0/2233322410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed040d0770 0x7fed04152ca0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 -- 172.21.15.133:0/2233322410 >> 172.21.15.133:0/2233322410 conn(0x7fed040727c0 msgr2=0x7fed040c7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 -- 172.21.15.133:0/2233322410 shutdown_connections 2024-04-25T10:11:15.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.778+0000 7fed0a467640 1 -- 172.21.15.133:0/2233322410 wait complete. 2024-04-25T10:11:15.798 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-04-25T10:11:13.877881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":497960,"kb_used_data":11120,"kb_used_omap":12,"kb_used_meta":486707,"kb_avail":754476760,"statfs":{"total":773094113280,"available":772584202240,"internally_reserved":0,"allocated":11386880,"data_stored":5899254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":498388529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001113"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:11.048899+0000","last_change":"2024-04-25T10:11:11.048899+0000","last_active":"2024-04-25T10:11:11.048899+0000","last_peered":"2024-04-25T10:11:11.048899+0000","last_clean":"2024-04-25T10:11:11.048899+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:11.048899+0000","last_undegraded":"2024-04-25T10:11:11.048899+0000","last_fullsized":"2024-04-25T10:11:11.048899+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:10.039164+0000","last_change":"2024-04-25T10:11:10.039164+0000","last_active":"2024-04-25T10:11:10.039164+0000","last_peered":"2024-04-25T10:11:10.039164+0000","last_clean":"2024-04-25T10:11:10.039164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:10.039164+0000","last_undegraded":"2024-04-25T10:11:10.039164+0000","last_fullsized":"2024-04-25T10:11:10.039164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":326,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:11:09.367161+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:11:09.367161+0000","last_peered":"2024-04-25T10:11:09.367161+0000","last_clean":"2024-04-25T10:11:09.367161+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:11:09.367161+0000","last_undegraded":"2024-04-25T10:11:09.367161+0000","last_fullsized":"2024-04-25T10:11:09.367161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64920,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306920,"statfs":{"total":96636764160,"available":96570286080,"internally_reserved":0,"allocated":1847296,"data_stored":1170647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58800,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313040,"statfs":{"total":96636764160,"available":96576552960,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54640,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317200,"statfs":{"total":96636764160,"available":96580812800,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68532,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303308,"statfs":{"total":96636764160,"available":96566587392,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":72684,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299156,"statfs":{"total":96636764160,"available":96562335744,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58800,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313040,"statfs":{"total":96636764160,"available":96576552960,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54636,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317204,"statfs":{"total":96636764160,"available":96580816896,"internally_reserved":0,"allocated":1277952,"data_stored":591177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64948,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306892,"statfs":{"total":96636764160,"available":96570257408,"internally_reserved":0,"allocated":1871872,"data_stored":1181545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:15.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2024-04-25T10:11:15.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3e is not active+clean 2024-04-25T10:11:15.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2024-04-25T10:11:15.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2024-04-25T10:11:15.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2024-04-25T10:11:15.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3a is not active+clean 2024-04-25T10:11:15.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.39 is not active+clean 2024-04-25T10:11:15.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2024-04-25T10:11:15.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.37 is not active+clean 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.32 is not active+clean 2024-04-25T10:11:15.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.31 is not active+clean 2024-04-25T10:11:15.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.30 is not active+clean 2024-04-25T10:11:15.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2024-04-25T10:11:15.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2e is not active+clean 2024-04-25T10:11:15.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2024-04-25T10:11:15.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2024-04-25T10:11:15.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2024-04-25T10:11:15.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2024-04-25T10:11:15.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2024-04-25T10:11:15.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2024-04-25T10:11:15.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2024-04-25T10:11:15.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.24 is not active+clean 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.21 is not active+clean 2024-04-25T10:11:15.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2024-04-25T10:11:15.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2024-04-25T10:11:15.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2024-04-25T10:11:15.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2024-04-25T10:11:15.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2024-04-25T10:11:15.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2024-04-25T10:11:15.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2024-04-25T10:11:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2024-04-25T10:11:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2024-04-25T10:11:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2024-04-25T10:11:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2024-04-25T10:11:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2024-04-25T10:11:15.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2024-04-25T10:11:15.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2024-04-25T10:11:15.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2024-04-25T10:11:15.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2024-04-25T10:11:15.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2024-04-25T10:11:15.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2024-04-25T10:11:15.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2024-04-25T10:11:15.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2024-04-25T10:11:15.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2024-04-25T10:11:15.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2024-04-25T10:11:15.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2024-04-25T10:11:15.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2024-04-25T10:11:15.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2024-04-25T10:11:15.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2024-04-25T10:11:15.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2024-04-25T10:11:15.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2024-04-25T10:11:15.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2024-04-25T10:11:15.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2024-04-25T10:11:15.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2024-04-25T10:11:15.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2024-04-25T10:11:15.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2024-04-25T10:11:15.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2024-04-25T10:11:15.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2024-04-25T10:11:15.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2024-04-25T10:11:15.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2024-04-25T10:11:15.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2024-04-25T10:11:15.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2024-04-25T10:11:15.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2024-04-25T10:11:15.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2024-04-25T10:11:15.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2024-04-25T10:11:15.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2024-04-25T10:11:15.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2024-04-25T10:11:15.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2024-04-25T10:11:15.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2024-04-25T10:11:15.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2024-04-25T10:11:15.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2024-04-25T10:11:15.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2024-04-25T10:11:15.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2024-04-25T10:11:15.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2024-04-25T10:11:15.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2024-04-25T10:11:15.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2024-04-25T10:11:15.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2024-04-25T10:11:15.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2024-04-25T10:11:15.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2024-04-25T10:11:15.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2024-04-25T10:11:15.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2024-04-25T10:11:15.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2024-04-25T10:11:15.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:11.048899+0000', 'last_change': '2024-04-25T10:11:11.048899+0000', 'last_active': '2024-04-25T10:11:11.048899+0000', 'last_peered': '2024-04-25T10:11:11.048899+0000', 'last_clean': '2024-04-25T10:11:11.048899+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:11.048899+0000', 'last_undegraded': '2024-04-25T10:11:11.048899+0000', 'last_fullsized': '2024-04-25T10:11:11.048899+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:11.048899+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2024-04-25T10:11:15.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:11:10.039164+0000', 'last_change': '2024-04-25T10:11:10.039164+0000', 'last_active': '2024-04-25T10:11:10.039164+0000', 'last_peered': '2024-04-25T10:11:10.039164+0000', 'last_clean': '2024-04-25T10:11:10.039164+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:11:10.039164+0000', 'last_undegraded': '2024-04-25T10:11:10.039164+0000', 'last_fullsized': '2024-04-25T10:11:10.039164+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'last_clean_scrub_stamp': '2024-04-25T10:11:10.039164+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:11:15.822 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.958+0000 7fd2f9cf0640 1 Processor -- start 2024-04-25T10:11:15.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.958+0000 7fd2f9cf0640 1 -- start start 2024-04-25T10:11:15.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f9cf0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd2f415a1a0 0x7fd2f415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f9cf0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f415e460 con 0x7fd2f4159dc0 2024-04-25T10:11:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f9cf0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f40ce880 con 0x7fd2f415e050 2024-04-25T10:11:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f9cf0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f40c49e0 con 0x7fd2f415a1a0 2024-04-25T10:11:15.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f37fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd2f4159dc0 0x7fd2f40cba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34898/0 (socket says 172.21.15.133:34898) 2024-04-25T10:11:15.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f37fe640 1 -- 172.21.15.133:0/4126848582 learned_addr learned my addr 172.21.15.133:0/4126848582 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:15.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160432206 0 0) 0x7fd2e8001120 con 0x7fd2f4159dc0 2024-04-25T10:11:15.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2dc003530 con 0x7fd2f4159dc0 2024-04-25T10:11:15.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f2ffd640 1 --2- 172.21.15.133:0/4126848582 >> v2:172.21.15.162:3300/0 conn(0x7fd2f415a1a0 0x7fd2f415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1338534664 0 0) 0x7fd2dc003530 con 0x7fd2f4159dc0 2024-04-25T10:11:15.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 >> v2:172.21.15.162:3300/0 conn(0x7fd2f415a1a0 msgr2=0x7fd2f415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 --2- 172.21.15.133:0/4126848582 >> v2:172.21.15.162:3300/0 conn(0x7fd2f415a1a0 0x7fd2f415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 >> v1:172.21.15.162:6789/0 conn(0x7fd2f415e050 legacy=0x7fd2f406d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f40ceb20 con 0x7fd2f4159dc0 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fd2e8001850 con 0x7fd2f4159dc0 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 >> v1:172.21.15.133:6789/0 conn(0x7fd2f4159dc0 legacy=0x7fd2f40cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 --2- 172.21.15.133:0/4126848582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2dc0039c0 0x7fd2dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 --2- 172.21.15.133:0/4126848582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc004810 0x7fd2dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 --2- 172.21.15.133:0/4126848582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2dc0051d0 0x7fd2dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40c49e0 con 0x7fd2dc0051d0 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc00c120 con 0x7fd2dc004810 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc00c320 con 0x7fd2dc0039c0 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f3fff640 1 --2- 172.21.15.133:0/4126848582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2dc0051d0 0x7fd2dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f2ffd640 1 --2- 172.21.15.133:0/4126848582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc004810 0x7fd2dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f37fe640 1 --2- 172.21.15.133:0/4126848582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2dc0039c0 0x7fd2dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f3fff640 1 -- 172.21.15.133:0/4126848582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2dc0039c0 msgr2=0x7fd2dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f3fff640 1 --2- 172.21.15.133:0/4126848582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2dc0039c0 0x7fd2dc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f3fff640 1 -- 172.21.15.133:0/4126848582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc004810 msgr2=0x7fd2dc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f3fff640 1 --2- 172.21.15.133:0/4126848582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2dc004810 0x7fd2dc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.962+0000 7fd2f3fff640 1 -- 172.21.15.133:0/4126848582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc003530 con 0x7fd2dc0051d0 2024-04-25T10:11:15.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f3fff640 1 -- 172.21.15.133:0/4126848582 reap_dead start 2024-04-25T10:11:15.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f3fff640 1 --2- 172.21.15.133:0/4126848582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2dc0051d0 0x7fd2dc00b9d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec001e70 tx=0x7fd2ec01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc50f4625c78e0ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec001a90 con 0x7fd2dc0051d0 2024-04-25T10:11:15.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec003ba0 con 0x7fd2dc0051d0 2024-04-25T10:11:15.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f27fc640 1 -- 172.21.15.133:0/4126848582 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec005020 con 0x7fd2dc0051d0 2024-04-25T10:11:15.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/4126848582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2dc0051d0 msgr2=0x7fd2dc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 --2- 172.21.15.133:0/4126848582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2dc0051d0 0x7fd2dc00b9d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec001e70 tx=0x7fd2ec01c780 comp rx=0 tx=0).stop 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/4126848582 shutdown_connections 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 --2- 172.21.15.133:0/4126848582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2dc0051d0 0x7fd2dc00b9d0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/4126848582 >> 172.21.15.133:0/4126848582 conn(0x7fd2f4067000 msgr2=0x7fd2f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/4126848582 shutdown_connections 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/4126848582 wait complete. 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 Processor -- start 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 -- start start 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2dc0051d0 0x7fd2f4142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f4141cf0 0x7fd2f4155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2f4156370 0x7fd2f40ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40c4e40 con 0x7fd2f4156370 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40c4cc0 con 0x7fd2f4141cf0 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f9cf0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40c4640 con 0x7fd2dc0051d0 2024-04-25T10:11:15.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.966+0000 7fd2f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2f4156370 0x7fd2f40ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2f4156370 0x7fd2f40ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37146/0 (socket says 172.21.15.133:37146) 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f3fff640 1 -- 172.21.15.133:0/2545648674 learned_addr learned my addr 172.21.15.133:0/2545648674 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f37fe640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2dc0051d0 0x7fd2f4142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f3fff640 1 -- 172.21.15.133:0/2545648674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2dc0051d0 msgr2=0x7fd2f4142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f3fff640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2dc0051d0 0x7fd2f4142560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f3fff640 1 -- 172.21.15.133:0/2545648674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f4141cf0 msgr2=0x7fd2f4155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f3fff640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f4141cf0 0x7fd2f4155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f3fff640 1 -- 172.21.15.133:0/2545648674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4144590 con 0x7fd2f4156370 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f3fff640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2f4156370 0x7fd2f40ca610 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec01cc60 tx=0x7fd2dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2dbfff640 1 -- 172.21.15.133:0/2545648674 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec001a90 con 0x7fd2f4156370 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2dbfff640 1 -- 172.21.15.133:0/2545648674 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec005830 con 0x7fd2f4156370 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/2545648674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f4144840 con 0x7fd2f4156370 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2dbfff640 1 -- 172.21.15.133:0/2545648674 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec007dc0 con 0x7fd2f4156370 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/2545648674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f4144e00 con 0x7fd2f4156370 2024-04-25T10:11:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/2545648674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f4068ef0 con 0x7fd2f4156370 2024-04-25T10:11:15.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2dbfff640 1 -- 172.21.15.133:0/2545648674 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2ec0a6480 con 0x7fd2f4156370 2024-04-25T10:11:15.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2dbfff640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2e0066f90 0x7fd2e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2dbfff640 1 -- 172.21.15.133:0/2545648674 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2ec11ee80 con 0x7fd2f4156370 2024-04-25T10:11:15.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f37fe640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2e0066f90 0x7fd2e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.970+0000 7fd2f37fe640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2e0066f90 0x7fd2e0069450 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8001060 tx=0x7fd2e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:15.974+0000 7fd2dbfff640 1 -- 172.21.15.133:0/2545648674 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2ec0b7060 con 0x7fd2f4156370 2024-04-25T10:11:16.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.182+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/2545648674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd2f40cc970 con 0x7fd2f4156370 2024-04-25T10:11:16.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.182+0000 7fd2dbfff640 1 -- 172.21.15.133:0/2545648674 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1434 (secure 0 0 0) 0x7fd2ec016490 con 0x7fd2f4156370 2024-04-25T10:11:16.187 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:16.187 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":589,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":107,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":514658304,"bytes_avail":772579454976,"bytes_total":773094113280,"write_bytes_sec":2459,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:10:59.874227+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/2545648674 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2e0066f90 msgr2=0x7fd2e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2e0066f90 0x7fd2e0069450 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8001060 tx=0x7fd2e8009040 comp rx=0 tx=0).stop 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/2545648674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2f4156370 msgr2=0x7fd2f40ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2f4156370 0x7fd2f40ca610 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec01cc60 tx=0x7fd2dc005840 comp rx=0 tx=0).stop 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/2545648674 shutdown_connections 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2e0066f90 0x7fd2e0069450 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2f4156370 0x7fd2f40ca610 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2f4141cf0 0x7fd2f4155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 --2- 172.21.15.133:0/2545648674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2dc0051d0 0x7fd2f4142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/2545648674 >> 172.21.15.133:0/2545648674 conn(0x7fd2f4067000 msgr2=0x7fd2f414f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/2545648674 shutdown_connections 2024-04-25T10:11:16.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.186+0000 7fd2f9cf0640 1 -- 172.21.15.133:0/2545648674 wait complete. 2024-04-25T10:11:16.204 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:11:16.204 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:16.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb1e043640 1 Processor -- start 2024-04-25T10:11:16.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb1e043640 1 -- start start 2024-04-25T10:11:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb1e043640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcb180cfc70 0x7fcb180d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb1e043640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1815bd90 con 0x7fcb1815b9b0 2024-04-25T10:11:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb1e043640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1809ad10 con 0x7fcb180c5aa0 2024-04-25T10:11:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb1e043640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb180d0610 con 0x7fcb180cfc70 2024-04-25T10:11:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb16ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcb180cfc70 0x7fcb180d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb16ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcb180cfc70 0x7fcb180d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50328/0 (socket says 172.21.15.133:50328) 2024-04-25T10:11:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb177fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcb1815b9b0 0x7fcb180cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60718/0 (socket says 172.21.15.133:60718) 2024-04-25T10:11:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb16ffd640 1 -- 172.21.15.133:0/4088086802 learned_addr learned my addr 172.21.15.133:0/4088086802 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb16ffd640 1 -- 172.21.15.133:0/4088086802 >> v1:172.21.15.162:6789/0 conn(0x7fcb180c5aa0 legacy=0x7fcb180c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb16ffd640 1 -- 172.21.15.133:0/4088086802 >> v1:172.21.15.133:6789/0 conn(0x7fcb1815b9b0 legacy=0x7fcb180cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb16ffd640 1 -- 172.21.15.133:0/4088086802 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1809a750 con 0x7fcb180cfc70 2024-04-25T10:11:16.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.310+0000 7fcb167fc640 1 -- 172.21.15.133:0/4088086802 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268633176 0 0) 0x7fcb1815bd90 con 0x7fcb1815b9b0 2024-04-25T10:11:16.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb16ffd640 1 --2- 172.21.15.133:0/4088086802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb180cfc70 0x7fcb180d00d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009870 tx=0x7fcb0c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e2017a8419aab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb167fc640 1 -- 172.21.15.133:0/4088086802 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0ab610 con 0x7fcb180cfc70 2024-04-25T10:11:16.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb167fc640 1 -- 172.21.15.133:0/4088086802 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c0b0070 con 0x7fcb180cfc70 2024-04-25T10:11:16.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb167fc640 1 -- 172.21.15.133:0/4088086802 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0ab9d0 con 0x7fcb180cfc70 2024-04-25T10:11:16.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 -- 172.21.15.133:0/4088086802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb180cfc70 msgr2=0x7fcb180d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 --2- 172.21.15.133:0/4088086802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb180cfc70 0x7fcb180d00d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009870 tx=0x7fcb0c01c520 comp rx=0 tx=0).stop 2024-04-25T10:11:16.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 -- 172.21.15.133:0/4088086802 shutdown_connections 2024-04-25T10:11:16.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 --2- 172.21.15.133:0/4088086802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb180cfc70 0x7fcb180d00d0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 -- 172.21.15.133:0/4088086802 >> 172.21.15.133:0/4088086802 conn(0x7fcb180b31a0 msgr2=0x7fcb180b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 -- 172.21.15.133:0/4088086802 shutdown_connections 2024-04-25T10:11:16.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 -- 172.21.15.133:0/4088086802 wait complete. 2024-04-25T10:11:16.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 Processor -- start 2024-04-25T10:11:16.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 -- start start 2024-04-25T10:11:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb180c5aa0 0x7fcb180b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb1815b9b0 0x7fcb180b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb180b5da0 0x7fcb180c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb181694f0 con 0x7fcb180b5da0 2024-04-25T10:11:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1809a3e0 con 0x7fcb180c5aa0 2024-04-25T10:11:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb1e043640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18169670 con 0x7fcb1815b9b0 2024-04-25T10:11:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb17fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb180b5da0 0x7fcb180c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb17fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb180b5da0 0x7fcb180c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50342/0 (socket says 172.21.15.133:50342) 2024-04-25T10:11:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb17fff640 1 -- 172.21.15.133:0/1830500414 learned_addr learned my addr 172.21.15.133:0/1830500414 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb16ffd640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb1815b9b0 0x7fcb180b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb17fff640 1 -- 172.21.15.133:0/1830500414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb1815b9b0 msgr2=0x7fcb180b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb17fff640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb1815b9b0 0x7fcb180b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb177fe640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb180c5aa0 0x7fcb180b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb17fff640 1 -- 172.21.15.133:0/1830500414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb180c5aa0 msgr2=0x7fcb180b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb17fff640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb180c5aa0 0x7fcb180b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb17fff640 1 -- 172.21.15.133:0/1830500414 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180c97a0 con 0x7fcb180b5da0 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcb17fff640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb180b5da0 0x7fcb180c8780 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000b990 tx=0x7fcb1000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.314+0000 7fcafffff640 1 -- 172.21.15.133:0/1830500414 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10019070 con 0x7fcb180b5da0 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.318+0000 7fcafffff640 1 -- 172.21.15.133:0/1830500414 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1000cba0 con 0x7fcb180b5da0 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.318+0000 7fcb1e043640 1 -- 172.21.15.133:0/1830500414 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb180c9a80 con 0x7fcb180b5da0 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.318+0000 7fcafffff640 1 -- 172.21.15.133:0/1830500414 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10014440 con 0x7fcb180b5da0 2024-04-25T10:11:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.318+0000 7fcb1e043640 1 -- 172.21.15.133:0/1830500414 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb181f4710 con 0x7fcb180b5da0 2024-04-25T10:11:16.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.318+0000 7fcb1e043640 1 -- 172.21.15.133:0/1830500414 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb180c9fa0 con 0x7fcb180b5da0 2024-04-25T10:11:16.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.318+0000 7fcafffff640 1 -- 172.21.15.133:0/1830500414 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb1001c070 con 0x7fcb180b5da0 2024-04-25T10:11:16.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.318+0000 7fcafffff640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb04066f60 0x7fcb04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.318+0000 7fcb177fe640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb04066f60 0x7fcb04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.318+0000 7fcafffff640 1 -- 172.21.15.133:0/1830500414 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb10084f40 con 0x7fcb180b5da0 2024-04-25T10:11:16.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.318+0000 7fcb177fe640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb04066f60 0x7fcb04069420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcb08000f70 tx=0x7fcb08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.322+0000 7fcafffff640 1 -- 172.21.15.133:0/1830500414 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb10055dc0 con 0x7fcb180b5da0 2024-04-25T10:11:16.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.478+0000 7fcb1e043640 1 -- 172.21.15.133:0/1830500414 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb18067c70 con 0x7fcb04066f60 2024-04-25T10:11:16.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.490+0000 7fcafffff640 1 -- 172.21.15.133:0/1830500414 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252484 (secure 0 0 0) 0x7fcb18067c70 con 0x7fcb04066f60 2024-04-25T10:11:16.494 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:16.496 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:11:16.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 -- 172.21.15.133:0/1830500414 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb04066f60 msgr2=0x7fcb04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb04066f60 0x7fcb04069420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcb08000f70 tx=0x7fcb08009040 comp rx=0 tx=0).stop 2024-04-25T10:11:16.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 -- 172.21.15.133:0/1830500414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb180b5da0 msgr2=0x7fcb180c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb180b5da0 0x7fcb180c8780 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000b990 tx=0x7fcb1000be50 comp rx=0 tx=0).stop 2024-04-25T10:11:16.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 -- 172.21.15.133:0/1830500414 shutdown_connections 2024-04-25T10:11:16.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb04066f60 0x7fcb04069420 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb180b5da0 0x7fcb180c8780 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb1815b9b0 0x7fcb180b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 --2- 172.21.15.133:0/1830500414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb180c5aa0 0x7fcb180b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 -- 172.21.15.133:0/1830500414 >> 172.21.15.133:0/1830500414 conn(0x7fcb180b31a0 msgr2=0x7fcb180b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 -- 172.21.15.133:0/1830500414 shutdown_connections 2024-04-25T10:11:16.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:16.494+0000 7fcb1e043640 1 -- 172.21.15.133:0/1830500414 wait complete. 2024-04-25T10:11:16.514 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-04-25T10:11:15.878408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":502596,"kb_used_data":11580,"kb_used_omap":12,"kb_used_meta":490867,"kb_avail":754472124,"statfs":{"total":773094113280,"available":772579454976,"internally_reserved":0,"allocated":11857920,"data_stored":6122160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":502648369},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.828331"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086657+0000","last_change":"2024-04-25T10:11:12.093000+0000","last_active":"2024-04-25T10:11:13.086657+0000","last_peered":"2024-04-25T10:11:13.086657+0000","last_clean":"2024-04-25T10:11:13.086657+0000","last_became_active":"2024-04-25T10:11:12.092705+0000","last_became_peered":"2024-04-25T10:11:12.092705+0000","last_unstale":"2024-04-25T10:11:13.086657+0000","last_undegraded":"2024-04-25T10:11:13.086657+0000","last_fullsized":"2024-04-25T10:11:13.086657+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:51.418315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083695+0000","last_change":"2024-04-25T10:11:12.081569+0000","last_active":"2024-04-25T10:11:13.083695+0000","last_peered":"2024-04-25T10:11:13.083695+0000","last_clean":"2024-04-25T10:11:13.083695+0000","last_became_active":"2024-04-25T10:11:12.081160+0000","last_became_peered":"2024-04-25T10:11:12.081160+0000","last_unstale":"2024-04-25T10:11:13.083695+0000","last_undegraded":"2024-04-25T10:11:13.083695+0000","last_fullsized":"2024-04-25T10:11:13.083695+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:09.967725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086774+0000","last_change":"2024-04-25T10:11:12.098051+0000","last_active":"2024-04-25T10:11:13.086774+0000","last_peered":"2024-04-25T10:11:13.086774+0000","last_clean":"2024-04-25T10:11:13.086774+0000","last_became_active":"2024-04-25T10:11:12.097741+0000","last_became_peered":"2024-04-25T10:11:12.097741+0000","last_unstale":"2024-04-25T10:11:13.086774+0000","last_undegraded":"2024-04-25T10:11:13.086774+0000","last_fullsized":"2024-04-25T10:11:13.086774+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:08.116662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.489026+0000","last_change":"2024-04-25T10:11:12.078453+0000","last_active":"2024-04-25T10:11:14.489026+0000","last_peered":"2024-04-25T10:11:14.489026+0000","last_clean":"2024-04-25T10:11:14.489026+0000","last_became_active":"2024-04-25T10:11:12.078089+0000","last_became_peered":"2024-04-25T10:11:12.078089+0000","last_unstale":"2024-04-25T10:11:14.489026+0000","last_undegraded":"2024-04-25T10:11:14.489026+0000","last_fullsized":"2024-04-25T10:11:14.489026+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:08.740865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989481+0000","last_change":"2024-04-25T10:11:12.094445+0000","last_active":"2024-04-25T10:11:13.989481+0000","last_peered":"2024-04-25T10:11:13.989481+0000","last_clean":"2024-04-25T10:11:13.989481+0000","last_became_active":"2024-04-25T10:11:12.091729+0000","last_became_peered":"2024-04-25T10:11:12.091729+0000","last_unstale":"2024-04-25T10:11:13.989481+0000","last_undegraded":"2024-04-25T10:11:13.989481+0000","last_fullsized":"2024-04-25T10:11:13.989481+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:32.737015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083672+0000","last_change":"2024-04-25T10:11:12.097145+0000","last_active":"2024-04-25T10:11:13.083672+0000","last_peered":"2024-04-25T10:11:13.083672+0000","last_clean":"2024-04-25T10:11:13.083672+0000","last_became_active":"2024-04-25T10:11:12.096860+0000","last_became_peered":"2024-04-25T10:11:12.096860+0000","last_unstale":"2024-04-25T10:11:13.083672+0000","last_undegraded":"2024-04-25T10:11:13.083672+0000","last_fullsized":"2024-04-25T10:11:13.083672+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:47.835475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083031+0000","last_change":"2024-04-25T10:11:12.092527+0000","last_active":"2024-04-25T10:11:13.083031+0000","last_peered":"2024-04-25T10:11:13.083031+0000","last_clean":"2024-04-25T10:11:13.083031+0000","last_became_active":"2024-04-25T10:11:12.092129+0000","last_became_peered":"2024-04-25T10:11:12.092129+0000","last_unstale":"2024-04-25T10:11:13.083031+0000","last_undegraded":"2024-04-25T10:11:13.083031+0000","last_fullsized":"2024-04-25T10:11:13.083031+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.333751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488223+0000","last_change":"2024-04-25T10:11:12.092590+0000","last_active":"2024-04-25T10:11:14.488223+0000","last_peered":"2024-04-25T10:11:14.488223+0000","last_clean":"2024-04-25T10:11:14.488223+0000","last_became_active":"2024-04-25T10:11:12.092318+0000","last_became_peered":"2024-04-25T10:11:12.092318+0000","last_unstale":"2024-04-25T10:11:14.488223+0000","last_undegraded":"2024-04-25T10:11:14.488223+0000","last_fullsized":"2024-04-25T10:11:14.488223+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:57.429161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084231+0000","last_change":"2024-04-25T10:11:12.091826+0000","last_active":"2024-04-25T10:11:13.084231+0000","last_peered":"2024-04-25T10:11:13.084231+0000","last_clean":"2024-04-25T10:11:13.084231+0000","last_became_active":"2024-04-25T10:11:12.086628+0000","last_became_peered":"2024-04-25T10:11:12.086628+0000","last_unstale":"2024-04-25T10:11:13.084231+0000","last_undegraded":"2024-04-25T10:11:13.084231+0000","last_fullsized":"2024-04-25T10:11:13.084231+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:16.240462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989727+0000","last_change":"2024-04-25T10:11:12.103625+0000","last_active":"2024-04-25T10:11:13.989727+0000","last_peered":"2024-04-25T10:11:13.989727+0000","last_clean":"2024-04-25T10:11:13.989727+0000","last_became_active":"2024-04-25T10:11:12.103237+0000","last_became_peered":"2024-04-25T10:11:12.103237+0000","last_unstale":"2024-04-25T10:11:13.989727+0000","last_undegraded":"2024-04-25T10:11:13.989727+0000","last_fullsized":"2024-04-25T10:11:13.989727+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:08.740865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113692+0000","last_change":"2024-04-25T10:11:12.105007+0000","last_active":"2024-04-25T10:11:13.113692+0000","last_peered":"2024-04-25T10:11:13.113692+0000","last_clean":"2024-04-25T10:11:13.113692+0000","last_became_active":"2024-04-25T10:11:12.104721+0000","last_became_peered":"2024-04-25T10:11:12.104721+0000","last_unstale":"2024-04-25T10:11:13.113692+0000","last_undegraded":"2024-04-25T10:11:13.113692+0000","last_fullsized":"2024-04-25T10:11:13.113692+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:00.349109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085554+0000","last_change":"2024-04-25T10:11:12.077945+0000","last_active":"2024-04-25T10:11:13.085554+0000","last_peered":"2024-04-25T10:11:13.085554+0000","last_clean":"2024-04-25T10:11:13.085554+0000","last_became_active":"2024-04-25T10:11:12.077679+0000","last_became_peered":"2024-04-25T10:11:12.077679+0000","last_unstale":"2024-04-25T10:11:13.085554+0000","last_undegraded":"2024-04-25T10:11:13.085554+0000","last_fullsized":"2024-04-25T10:11:13.085554+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:08.740865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488635+0000","last_change":"2024-04-25T10:11:12.097231+0000","last_active":"2024-04-25T10:11:14.488635+0000","last_peered":"2024-04-25T10:11:14.488635+0000","last_clean":"2024-04-25T10:11:14.488635+0000","last_became_active":"2024-04-25T10:11:12.096930+0000","last_became_peered":"2024-04-25T10:11:12.096930+0000","last_unstale":"2024-04-25T10:11:14.488635+0000","last_undegraded":"2024-04-25T10:11:14.488635+0000","last_fullsized":"2024-04-25T10:11:14.488635+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:48.782635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556499+0000","last_change":"2024-04-25T10:11:12.088023+0000","last_active":"2024-04-25T10:11:13.556499+0000","last_peered":"2024-04-25T10:11:13.556499+0000","last_clean":"2024-04-25T10:11:13.556499+0000","last_became_active":"2024-04-25T10:11:12.087708+0000","last_became_peered":"2024-04-25T10:11:12.087708+0000","last_unstale":"2024-04-25T10:11:13.556499+0000","last_undegraded":"2024-04-25T10:11:13.556499+0000","last_fullsized":"2024-04-25T10:11:13.556499+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:49.410252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556300+0000","last_change":"2024-04-25T10:11:12.097091+0000","last_active":"2024-04-25T10:11:13.556300+0000","last_peered":"2024-04-25T10:11:13.556300+0000","last_clean":"2024-04-25T10:11:13.556300+0000","last_became_active":"2024-04-25T10:11:12.096769+0000","last_became_peered":"2024-04-25T10:11:12.096769+0000","last_unstale":"2024-04-25T10:11:13.556300+0000","last_undegraded":"2024-04-25T10:11:13.556300+0000","last_fullsized":"2024-04-25T10:11:13.556300+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:49.119547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556297+0000","last_change":"2024-04-25T10:11:12.084939+0000","last_active":"2024-04-25T10:11:13.556297+0000","last_peered":"2024-04-25T10:11:13.556297+0000","last_clean":"2024-04-25T10:11:13.556297+0000","last_became_active":"2024-04-25T10:11:12.084662+0000","last_became_peered":"2024-04-25T10:11:12.084662+0000","last_unstale":"2024-04-25T10:11:13.556297+0000","last_undegraded":"2024-04-25T10:11:13.556297+0000","last_fullsized":"2024-04-25T10:11:13.556297+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:52.193713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086513+0000","last_change":"2024-04-25T10:11:12.102661+0000","last_active":"2024-04-25T10:11:13.086513+0000","last_peered":"2024-04-25T10:11:13.086513+0000","last_clean":"2024-04-25T10:11:13.086513+0000","last_became_active":"2024-04-25T10:11:12.102253+0000","last_became_peered":"2024-04-25T10:11:12.102253+0000","last_unstale":"2024-04-25T10:11:13.086513+0000","last_undegraded":"2024-04-25T10:11:13.086513+0000","last_fullsized":"2024-04-25T10:11:13.086513+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:54.286436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083905+0000","last_change":"2024-04-25T10:11:12.085683+0000","last_active":"2024-04-25T10:11:13.083905+0000","last_peered":"2024-04-25T10:11:13.083905+0000","last_clean":"2024-04-25T10:11:13.083905+0000","last_became_active":"2024-04-25T10:11:12.085323+0000","last_became_peered":"2024-04-25T10:11:12.085323+0000","last_unstale":"2024-04-25T10:11:13.083905+0000","last_undegraded":"2024-04-25T10:11:13.083905+0000","last_fullsized":"2024-04-25T10:11:13.083905+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.550100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086997+0000","last_change":"2024-04-25T10:11:12.101583+0000","last_active":"2024-04-25T10:11:13.086997+0000","last_peered":"2024-04-25T10:11:13.086997+0000","last_clean":"2024-04-25T10:11:13.086997+0000","last_became_active":"2024-04-25T10:11:12.101252+0000","last_became_peered":"2024-04-25T10:11:12.101252+0000","last_unstale":"2024-04-25T10:11:13.086997+0000","last_undegraded":"2024-04-25T10:11:13.086997+0000","last_fullsized":"2024-04-25T10:11:13.086997+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:49.812198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085907+0000","last_change":"2024-04-25T10:11:12.077198+0000","last_active":"2024-04-25T10:11:13.085907+0000","last_peered":"2024-04-25T10:11:13.085907+0000","last_clean":"2024-04-25T10:11:13.085907+0000","last_became_active":"2024-04-25T10:11:12.076888+0000","last_became_peered":"2024-04-25T10:11:12.076888+0000","last_unstale":"2024-04-25T10:11:13.085907+0000","last_undegraded":"2024-04-25T10:11:13.085907+0000","last_fullsized":"2024-04-25T10:11:13.085907+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:39.450225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488499+0000","last_change":"2024-04-25T10:11:12.101652+0000","last_active":"2024-04-25T10:11:14.488499+0000","last_peered":"2024-04-25T10:11:14.488499+0000","last_clean":"2024-04-25T10:11:14.488499+0000","last_became_active":"2024-04-25T10:11:12.101287+0000","last_became_peered":"2024-04-25T10:11:12.101287+0000","last_unstale":"2024-04-25T10:11:14.488499+0000","last_undegraded":"2024-04-25T10:11:14.488499+0000","last_fullsized":"2024-04-25T10:11:14.488499+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.171772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989550+0000","last_change":"2024-04-25T10:11:12.099476+0000","last_active":"2024-04-25T10:11:13.989550+0000","last_peered":"2024-04-25T10:11:13.989550+0000","last_clean":"2024-04-25T10:11:13.989550+0000","last_became_active":"2024-04-25T10:11:12.096232+0000","last_became_peered":"2024-04-25T10:11:12.096232+0000","last_unstale":"2024-04-25T10:11:13.989550+0000","last_undegraded":"2024-04-25T10:11:13.989550+0000","last_fullsized":"2024-04-25T10:11:13.989550+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:57.678011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085816+0000","last_change":"2024-04-25T10:11:12.092952+0000","last_active":"2024-04-25T10:11:13.085816+0000","last_peered":"2024-04-25T10:11:13.085816+0000","last_clean":"2024-04-25T10:11:13.085816+0000","last_became_active":"2024-04-25T10:11:12.092638+0000","last_became_peered":"2024-04-25T10:11:12.092638+0000","last_unstale":"2024-04-25T10:11:13.085816+0000","last_undegraded":"2024-04-25T10:11:13.085816+0000","last_fullsized":"2024-04-25T10:11:13.085816+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:13.864982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989141+0000","last_change":"2024-04-25T10:11:12.099563+0000","last_active":"2024-04-25T10:11:13.989141+0000","last_peered":"2024-04-25T10:11:13.989141+0000","last_clean":"2024-04-25T10:11:13.989141+0000","last_became_active":"2024-04-25T10:11:12.099213+0000","last_became_peered":"2024-04-25T10:11:12.099213+0000","last_unstale":"2024-04-25T10:11:13.989141+0000","last_undegraded":"2024-04-25T10:11:13.989141+0000","last_fullsized":"2024-04-25T10:11:13.989141+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:21.442046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085248+0000","last_change":"2024-04-25T10:11:12.101960+0000","last_active":"2024-04-25T10:11:13.085248+0000","last_peered":"2024-04-25T10:11:13.085248+0000","last_clean":"2024-04-25T10:11:13.085248+0000","last_became_active":"2024-04-25T10:11:12.101627+0000","last_became_peered":"2024-04-25T10:11:12.101627+0000","last_unstale":"2024-04-25T10:11:13.085248+0000","last_undegraded":"2024-04-25T10:11:13.085248+0000","last_fullsized":"2024-04-25T10:11:13.085248+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:48.782635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085244+0000","last_change":"2024-04-25T10:11:12.073651+0000","last_active":"2024-04-25T10:11:13.085244+0000","last_peered":"2024-04-25T10:11:13.085244+0000","last_clean":"2024-04-25T10:11:13.085244+0000","last_became_active":"2024-04-25T10:11:12.073370+0000","last_became_peered":"2024-04-25T10:11:12.073370+0000","last_unstale":"2024-04-25T10:11:13.085244+0000","last_undegraded":"2024-04-25T10:11:13.085244+0000","last_fullsized":"2024-04-25T10:11:13.085244+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:57.678011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113862+0000","last_change":"2024-04-25T10:11:12.095432+0000","last_active":"2024-04-25T10:11:13.113862+0000","last_peered":"2024-04-25T10:11:13.113862+0000","last_clean":"2024-04-25T10:11:13.113862+0000","last_became_active":"2024-04-25T10:11:12.090819+0000","last_became_peered":"2024-04-25T10:11:12.090819+0000","last_unstale":"2024-04-25T10:11:13.113862+0000","last_undegraded":"2024-04-25T10:11:13.113862+0000","last_fullsized":"2024-04-25T10:11:13.113862+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.127816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084714+0000","last_change":"2024-04-25T10:11:12.093139+0000","last_active":"2024-04-25T10:11:13.084714+0000","last_peered":"2024-04-25T10:11:13.084714+0000","last_clean":"2024-04-25T10:11:13.084714+0000","last_became_active":"2024-04-25T10:11:12.092725+0000","last_became_peered":"2024-04-25T10:11:12.092725+0000","last_unstale":"2024-04-25T10:11:13.084714+0000","last_undegraded":"2024-04-25T10:11:13.084714+0000","last_fullsized":"2024-04-25T10:11:13.084714+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:23.199454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488737+0000","last_change":"2024-04-25T10:11:12.087843+0000","last_active":"2024-04-25T10:11:14.488737+0000","last_peered":"2024-04-25T10:11:14.488737+0000","last_clean":"2024-04-25T10:11:14.488737+0000","last_became_active":"2024-04-25T10:11:12.087566+0000","last_became_peered":"2024-04-25T10:11:12.087566+0000","last_unstale":"2024-04-25T10:11:14.488737+0000","last_undegraded":"2024-04-25T10:11:14.488737+0000","last_fullsized":"2024-04-25T10:11:14.488737+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:14.164758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084909+0000","last_change":"2024-04-25T10:11:12.097168+0000","last_active":"2024-04-25T10:11:13.084909+0000","last_peered":"2024-04-25T10:11:13.084909+0000","last_clean":"2024-04-25T10:11:13.084909+0000","last_became_active":"2024-04-25T10:11:12.096321+0000","last_became_peered":"2024-04-25T10:11:12.096321+0000","last_unstale":"2024-04-25T10:11:13.084909+0000","last_undegraded":"2024-04-25T10:11:13.084909+0000","last_fullsized":"2024-04-25T10:11:13.084909+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:54.235179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083450+0000","last_change":"2024-04-25T10:11:12.084636+0000","last_active":"2024-04-25T10:11:13.083450+0000","last_peered":"2024-04-25T10:11:13.083450+0000","last_clean":"2024-04-25T10:11:13.083450+0000","last_became_active":"2024-04-25T10:11:12.084242+0000","last_became_peered":"2024-04-25T10:11:12.084242+0000","last_unstale":"2024-04-25T10:11:13.083450+0000","last_undegraded":"2024-04-25T10:11:13.083450+0000","last_fullsized":"2024-04-25T10:11:13.083450+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:16.362990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488401+0000","last_change":"2024-04-25T10:11:12.079666+0000","last_active":"2024-04-25T10:11:14.488401+0000","last_peered":"2024-04-25T10:11:14.488401+0000","last_clean":"2024-04-25T10:11:14.488401+0000","last_became_active":"2024-04-25T10:11:12.079391+0000","last_became_peered":"2024-04-25T10:11:12.079391+0000","last_unstale":"2024-04-25T10:11:14.488401+0000","last_undegraded":"2024-04-25T10:11:14.488401+0000","last_fullsized":"2024-04-25T10:11:14.488401+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:39.612970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086228+0000","last_change":"2024-04-25T10:11:12.088291+0000","last_active":"2024-04-25T10:11:13.086228+0000","last_peered":"2024-04-25T10:11:13.086228+0000","last_clean":"2024-04-25T10:11:13.086228+0000","last_became_active":"2024-04-25T10:11:12.087889+0000","last_became_peered":"2024-04-25T10:11:12.087889+0000","last_unstale":"2024-04-25T10:11:13.086228+0000","last_undegraded":"2024-04-25T10:11:13.086228+0000","last_fullsized":"2024-04-25T10:11:13.086228+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:37.003254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"106'4","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.489474+0000","last_change":"2024-04-25T10:11:11.099778+0000","last_active":"2024-04-25T10:11:14.489474+0000","last_peered":"2024-04-25T10:11:14.489474+0000","last_clean":"2024-04-25T10:11:14.489474+0000","last_became_active":"2024-04-25T10:11:11.099546+0000","last_became_peered":"2024-04-25T10:11:11.099546+0000","last_unstale":"2024-04-25T10:11:14.489474+0000","last_undegraded":"2024-04-25T10:11:14.489474+0000","last_fullsized":"2024-04-25T10:11:14.489474+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.540366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488250+0000","last_change":"2024-04-25T10:11:12.096146+0000","last_active":"2024-04-25T10:11:14.488250+0000","last_peered":"2024-04-25T10:11:14.488250+0000","last_clean":"2024-04-25T10:11:14.488250+0000","last_became_active":"2024-04-25T10:11:12.095773+0000","last_became_peered":"2024-04-25T10:11:12.095773+0000","last_unstale":"2024-04-25T10:11:14.488250+0000","last_undegraded":"2024-04-25T10:11:14.488250+0000","last_fullsized":"2024-04-25T10:11:14.488250+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:21.912912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488727+0000","last_change":"2024-04-25T10:11:11.085487+0000","last_active":"2024-04-25T10:11:14.488727+0000","last_peered":"2024-04-25T10:11:14.488727+0000","last_clean":"2024-04-25T10:11:14.488727+0000","last_became_active":"2024-04-25T10:11:11.085141+0000","last_became_peered":"2024-04-25T10:11:11.085141+0000","last_unstale":"2024-04-25T10:11:14.488727+0000","last_undegraded":"2024-04-25T10:11:14.488727+0000","last_fullsized":"2024-04-25T10:11:14.488727+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:36.501389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085009+0000","last_change":"2024-04-25T10:11:12.089201+0000","last_active":"2024-04-25T10:11:13.085009+0000","last_peered":"2024-04-25T10:11:13.085009+0000","last_clean":"2024-04-25T10:11:13.085009+0000","last_became_active":"2024-04-25T10:11:12.088789+0000","last_became_peered":"2024-04-25T10:11:12.088789+0000","last_unstale":"2024-04-25T10:11:13.085009+0000","last_undegraded":"2024-04-25T10:11:13.085009+0000","last_fullsized":"2024-04-25T10:11:13.085009+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:32.737015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085280+0000","last_change":"2024-04-25T10:11:11.095732+0000","last_active":"2024-04-25T10:11:13.085280+0000","last_peered":"2024-04-25T10:11:13.085280+0000","last_clean":"2024-04-25T10:11:13.085280+0000","last_became_active":"2024-04-25T10:11:11.095005+0000","last_became_peered":"2024-04-25T10:11:11.095005+0000","last_unstale":"2024-04-25T10:11:13.085280+0000","last_undegraded":"2024-04-25T10:11:13.085280+0000","last_fullsized":"2024-04-25T10:11:13.085280+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:05.324017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113762+0000","last_change":"2024-04-25T10:11:12.093315+0000","last_active":"2024-04-25T10:11:13.113762+0000","last_peered":"2024-04-25T10:11:13.113762+0000","last_clean":"2024-04-25T10:11:13.113762+0000","last_became_active":"2024-04-25T10:11:12.092954+0000","last_became_peered":"2024-04-25T10:11:12.092954+0000","last_unstale":"2024-04-25T10:11:13.113762+0000","last_undegraded":"2024-04-25T10:11:13.113762+0000","last_fullsized":"2024-04-25T10:11:13.113762+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:23.327457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989903+0000","last_change":"2024-04-25T10:11:11.094878+0000","last_active":"2024-04-25T10:11:13.989903+0000","last_peered":"2024-04-25T10:11:13.989903+0000","last_clean":"2024-04-25T10:11:13.989903+0000","last_became_active":"2024-04-25T10:11:11.094453+0000","last_became_peered":"2024-04-25T10:11:11.094453+0000","last_unstale":"2024-04-25T10:11:13.989903+0000","last_undegraded":"2024-04-25T10:11:13.989903+0000","last_fullsized":"2024-04-25T10:11:13.989903+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:02.208178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.489181+0000","last_change":"2024-04-25T10:11:12.091705+0000","last_active":"2024-04-25T10:11:14.489181+0000","last_peered":"2024-04-25T10:11:14.489181+0000","last_clean":"2024-04-25T10:11:14.489181+0000","last_became_active":"2024-04-25T10:11:12.091409+0000","last_became_peered":"2024-04-25T10:11:12.091409+0000","last_unstale":"2024-04-25T10:11:14.489181+0000","last_undegraded":"2024-04-25T10:11:14.489181+0000","last_fullsized":"2024-04-25T10:11:14.489181+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:39.450225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556400+0000","last_change":"2024-04-25T10:11:11.100430+0000","last_active":"2024-04-25T10:11:13.556400+0000","last_peered":"2024-04-25T10:11:13.556400+0000","last_clean":"2024-04-25T10:11:13.556400+0000","last_became_active":"2024-04-25T10:11:11.099788+0000","last_became_peered":"2024-04-25T10:11:11.099788+0000","last_unstale":"2024-04-25T10:11:13.556400+0000","last_undegraded":"2024-04-25T10:11:13.556400+0000","last_fullsized":"2024-04-25T10:11:13.556400+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:23.869390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.087386+0000","last_change":"2024-04-25T10:11:12.087916+0000","last_active":"2024-04-25T10:11:13.087386+0000","last_peered":"2024-04-25T10:11:13.087386+0000","last_clean":"2024-04-25T10:11:13.087386+0000","last_became_active":"2024-04-25T10:11:12.087636+0000","last_became_peered":"2024-04-25T10:11:12.087636+0000","last_unstale":"2024-04-25T10:11:13.087386+0000","last_undegraded":"2024-04-25T10:11:13.087386+0000","last_fullsized":"2024-04-25T10:11:13.087386+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:14.115921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113283+0000","last_change":"2024-04-25T10:11:11.104519+0000","last_active":"2024-04-25T10:11:13.113283+0000","last_peered":"2024-04-25T10:11:13.113283+0000","last_clean":"2024-04-25T10:11:13.113283+0000","last_became_active":"2024-04-25T10:11:11.103889+0000","last_became_peered":"2024-04-25T10:11:11.103889+0000","last_unstale":"2024-04-25T10:11:13.113283+0000","last_undegraded":"2024-04-25T10:11:13.113283+0000","last_fullsized":"2024-04-25T10:11:13.113283+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:22.189720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084118+0000","last_change":"2024-04-25T10:11:12.093843+0000","last_active":"2024-04-25T10:11:13.084118+0000","last_peered":"2024-04-25T10:11:13.084118+0000","last_clean":"2024-04-25T10:11:13.084118+0000","last_became_active":"2024-04-25T10:11:12.093567+0000","last_became_peered":"2024-04-25T10:11:12.093567+0000","last_unstale":"2024-04-25T10:11:13.084118+0000","last_undegraded":"2024-04-25T10:11:13.084118+0000","last_fullsized":"2024-04-25T10:11:13.084118+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:24.573689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084628+0000","last_change":"2024-04-25T10:11:11.102540+0000","last_active":"2024-04-25T10:11:13.084628+0000","last_peered":"2024-04-25T10:11:13.084628+0000","last_clean":"2024-04-25T10:11:13.084628+0000","last_became_active":"2024-04-25T10:11:11.101923+0000","last_became_peered":"2024-04-25T10:11:11.101923+0000","last_unstale":"2024-04-25T10:11:13.084628+0000","last_undegraded":"2024-04-25T10:11:13.084628+0000","last_fullsized":"2024-04-25T10:11:13.084628+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:02.208178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.489393+0000","last_change":"2024-04-25T10:11:12.082040+0000","last_active":"2024-04-25T10:11:14.489393+0000","last_peered":"2024-04-25T10:11:14.489393+0000","last_clean":"2024-04-25T10:11:14.489393+0000","last_became_active":"2024-04-25T10:11:12.081618+0000","last_became_peered":"2024-04-25T10:11:12.081618+0000","last_unstale":"2024-04-25T10:11:14.489393+0000","last_undegraded":"2024-04-25T10:11:14.489393+0000","last_fullsized":"2024-04-25T10:11:14.489393+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:05.688368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.487997+0000","last_change":"2024-04-25T10:11:11.079085+0000","last_active":"2024-04-25T10:11:14.487997+0000","last_peered":"2024-04-25T10:11:14.487997+0000","last_clean":"2024-04-25T10:11:14.487997+0000","last_became_active":"2024-04-25T10:11:11.078454+0000","last_became_peered":"2024-04-25T10:11:11.078454+0000","last_unstale":"2024-04-25T10:11:14.487997+0000","last_undegraded":"2024-04-25T10:11:14.487997+0000","last_fullsized":"2024-04-25T10:11:14.487997+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:46.825741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085849+0000","last_change":"2024-04-25T10:11:12.102376+0000","last_active":"2024-04-25T10:11:13.085849+0000","last_peered":"2024-04-25T10:11:13.085849+0000","last_clean":"2024-04-25T10:11:13.085849+0000","last_became_active":"2024-04-25T10:11:12.102091+0000","last_became_peered":"2024-04-25T10:11:12.102091+0000","last_unstale":"2024-04-25T10:11:13.085849+0000","last_undegraded":"2024-04-25T10:11:13.085849+0000","last_fullsized":"2024-04-25T10:11:13.085849+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.617904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085389+0000","last_change":"2024-04-25T10:11:11.096132+0000","last_active":"2024-04-25T10:11:13.085389+0000","last_peered":"2024-04-25T10:11:13.085389+0000","last_clean":"2024-04-25T10:11:13.085389+0000","last_became_active":"2024-04-25T10:11:11.095503+0000","last_became_peered":"2024-04-25T10:11:11.095503+0000","last_unstale":"2024-04-25T10:11:13.085389+0000","last_undegraded":"2024-04-25T10:11:13.085389+0000","last_fullsized":"2024-04-25T10:11:13.085389+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:07.891794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989369+0000","last_change":"2024-04-25T10:11:12.091625+0000","last_active":"2024-04-25T10:11:13.989369+0000","last_peered":"2024-04-25T10:11:13.989369+0000","last_clean":"2024-04-25T10:11:13.989369+0000","last_became_active":"2024-04-25T10:11:12.090718+0000","last_became_peered":"2024-04-25T10:11:12.090718+0000","last_unstale":"2024-04-25T10:11:13.989369+0000","last_undegraded":"2024-04-25T10:11:13.989369+0000","last_fullsized":"2024-04-25T10:11:13.989369+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:54.235179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086045+0000","last_change":"2024-04-25T10:11:11.094363+0000","last_active":"2024-04-25T10:11:13.086045+0000","last_peered":"2024-04-25T10:11:13.086045+0000","last_clean":"2024-04-25T10:11:13.086045+0000","last_became_active":"2024-04-25T10:11:11.094037+0000","last_became_peered":"2024-04-25T10:11:11.094037+0000","last_unstale":"2024-04-25T10:11:13.086045+0000","last_undegraded":"2024-04-25T10:11:13.086045+0000","last_fullsized":"2024-04-25T10:11:13.086045+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:51.183979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488359+0000","last_change":"2024-04-25T10:11:12.096216+0000","last_active":"2024-04-25T10:11:14.488359+0000","last_peered":"2024-04-25T10:11:14.488359+0000","last_clean":"2024-04-25T10:11:14.488359+0000","last_became_active":"2024-04-25T10:11:12.095848+0000","last_became_peered":"2024-04-25T10:11:12.095848+0000","last_unstale":"2024-04-25T10:11:14.488359+0000","last_undegraded":"2024-04-25T10:11:14.488359+0000","last_fullsized":"2024-04-25T10:11:14.488359+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:53.849761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"106'3","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556694+0000","last_change":"2024-04-25T10:11:11.087036+0000","last_active":"2024-04-25T10:11:13.556694+0000","last_peered":"2024-04-25T10:11:13.556694+0000","last_clean":"2024-04-25T10:11:13.556694+0000","last_became_active":"2024-04-25T10:11:11.086301+0000","last_became_peered":"2024-04-25T10:11:11.086301+0000","last_unstale":"2024-04-25T10:11:13.556694+0000","last_undegraded":"2024-04-25T10:11:13.556694+0000","last_fullsized":"2024-04-25T10:11:13.556694+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:20.562287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989960+0000","last_change":"2024-04-25T10:11:12.091290+0000","last_active":"2024-04-25T10:11:13.989960+0000","last_peered":"2024-04-25T10:11:13.989960+0000","last_clean":"2024-04-25T10:11:13.989960+0000","last_became_active":"2024-04-25T10:11:12.090900+0000","last_became_peered":"2024-04-25T10:11:12.090900+0000","last_unstale":"2024-04-25T10:11:13.989960+0000","last_undegraded":"2024-04-25T10:11:13.989960+0000","last_fullsized":"2024-04-25T10:11:13.989960+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:39.450225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"106'6","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084966+0000","last_change":"2024-04-25T10:11:11.093719+0000","last_active":"2024-04-25T10:11:13.084966+0000","last_peered":"2024-04-25T10:11:13.084966+0000","last_clean":"2024-04-25T10:11:13.084966+0000","last_became_active":"2024-04-25T10:11:11.091809+0000","last_became_peered":"2024-04-25T10:11:11.091809+0000","last_unstale":"2024-04-25T10:11:13.084966+0000","last_undegraded":"2024-04-25T10:11:13.084966+0000","last_fullsized":"2024-04-25T10:11:13.084966+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:06.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113967+0000","last_change":"2024-04-25T10:11:12.099666+0000","last_active":"2024-04-25T10:11:13.113967+0000","last_peered":"2024-04-25T10:11:13.113967+0000","last_clean":"2024-04-25T10:11:13.113967+0000","last_became_active":"2024-04-25T10:11:12.096872+0000","last_became_peered":"2024-04-25T10:11:12.096872+0000","last_unstale":"2024-04-25T10:11:13.113967+0000","last_undegraded":"2024-04-25T10:11:13.113967+0000","last_fullsized":"2024-04-25T10:11:13.113967+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:36.323673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989828+0000","last_change":"2024-04-25T10:11:11.083667+0000","last_active":"2024-04-25T10:11:13.989828+0000","last_peered":"2024-04-25T10:11:13.989828+0000","last_clean":"2024-04-25T10:11:13.989828+0000","last_became_active":"2024-04-25T10:11:11.083377+0000","last_became_peered":"2024-04-25T10:11:11.083377+0000","last_unstale":"2024-04-25T10:11:13.989828+0000","last_undegraded":"2024-04-25T10:11:13.989828+0000","last_fullsized":"2024-04-25T10:11:13.989828+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:53.235198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085783+0000","last_change":"2024-04-25T10:11:12.079785+0000","last_active":"2024-04-25T10:11:13.085783+0000","last_peered":"2024-04-25T10:11:13.085783+0000","last_clean":"2024-04-25T10:11:13.085783+0000","last_became_active":"2024-04-25T10:11:12.079435+0000","last_became_peered":"2024-04-25T10:11:12.079435+0000","last_unstale":"2024-04-25T10:11:13.085783+0000","last_undegraded":"2024-04-25T10:11:13.085783+0000","last_fullsized":"2024-04-25T10:11:13.085783+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:21.442046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488397+0000","last_change":"2024-04-25T10:11:11.084530+0000","last_active":"2024-04-25T10:11:14.488397+0000","last_peered":"2024-04-25T10:11:14.488397+0000","last_clean":"2024-04-25T10:11:14.488397+0000","last_became_active":"2024-04-25T10:11:11.084051+0000","last_became_peered":"2024-04-25T10:11:11.084051+0000","last_unstale":"2024-04-25T10:11:14.488397+0000","last_undegraded":"2024-04-25T10:11:14.488397+0000","last_fullsized":"2024-04-25T10:11:14.488397+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:08.957991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488642+0000","last_change":"2024-04-25T10:11:12.086480+0000","last_active":"2024-04-25T10:11:14.488642+0000","last_peered":"2024-04-25T10:11:14.488642+0000","last_clean":"2024-04-25T10:11:14.488642+0000","last_became_active":"2024-04-25T10:11:12.086215+0000","last_became_peered":"2024-04-25T10:11:12.086215+0000","last_unstale":"2024-04-25T10:11:14.488642+0000","last_undegraded":"2024-04-25T10:11:14.488642+0000","last_fullsized":"2024-04-25T10:11:14.488642+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:21.125843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488474+0000","last_change":"2024-04-25T10:11:11.095999+0000","last_active":"2024-04-25T10:11:14.488474+0000","last_peered":"2024-04-25T10:11:14.488474+0000","last_clean":"2024-04-25T10:11:14.488474+0000","last_became_active":"2024-04-25T10:11:11.095678+0000","last_became_peered":"2024-04-25T10:11:11.095678+0000","last_unstale":"2024-04-25T10:11:14.488474+0000","last_undegraded":"2024-04-25T10:11:14.488474+0000","last_fullsized":"2024-04-25T10:11:14.488474+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:05.275695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086755+0000","last_change":"2024-04-25T10:11:12.102933+0000","last_active":"2024-04-25T10:11:13.086755+0000","last_peered":"2024-04-25T10:11:13.086755+0000","last_clean":"2024-04-25T10:11:13.086755+0000","last_became_active":"2024-04-25T10:11:12.101669+0000","last_became_peered":"2024-04-25T10:11:12.101669+0000","last_unstale":"2024-04-25T10:11:13.086755+0000","last_undegraded":"2024-04-25T10:11:13.086755+0000","last_fullsized":"2024-04-25T10:11:13.086755+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:50.105497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"106'8","reported_seq":22,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084384+0000","last_change":"2024-04-25T10:11:11.095809+0000","last_active":"2024-04-25T10:11:13.084384+0000","last_peered":"2024-04-25T10:11:13.084384+0000","last_clean":"2024-04-25T10:11:13.084384+0000","last_became_active":"2024-04-25T10:11:11.095117+0000","last_became_peered":"2024-04-25T10:11:11.095117+0000","last_unstale":"2024-04-25T10:11:13.084384+0000","last_undegraded":"2024-04-25T10:11:13.084384+0000","last_fullsized":"2024-04-25T10:11:13.084384+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.346745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488573+0000","last_change":"2024-04-25T10:11:12.084090+0000","last_active":"2024-04-25T10:11:14.488573+0000","last_peered":"2024-04-25T10:11:14.488573+0000","last_clean":"2024-04-25T10:11:14.488573+0000","last_became_active":"2024-04-25T10:11:12.083759+0000","last_became_peered":"2024-04-25T10:11:12.083759+0000","last_unstale":"2024-04-25T10:11:14.488573+0000","last_undegraded":"2024-04-25T10:11:14.488573+0000","last_fullsized":"2024-04-25T10:11:14.488573+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:51.833546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083606+0000","last_change":"2024-04-25T10:11:11.103097+0000","last_active":"2024-04-25T10:11:13.083606+0000","last_peered":"2024-04-25T10:11:13.083606+0000","last_clean":"2024-04-25T10:11:13.083606+0000","last_became_active":"2024-04-25T10:11:11.102480+0000","last_became_peered":"2024-04-25T10:11:11.102480+0000","last_unstale":"2024-04-25T10:11:13.083606+0000","last_undegraded":"2024-04-25T10:11:13.083606+0000","last_fullsized":"2024-04-25T10:11:13.083606+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:27.678274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083279+0000","last_change":"2024-04-25T10:11:12.082737+0000","last_active":"2024-04-25T10:11:13.083279+0000","last_peered":"2024-04-25T10:11:13.083279+0000","last_clean":"2024-04-25T10:11:13.083279+0000","last_became_active":"2024-04-25T10:11:12.082090+0000","last_became_peered":"2024-04-25T10:11:12.082090+0000","last_unstale":"2024-04-25T10:11:13.083279+0000","last_undegraded":"2024-04-25T10:11:13.083279+0000","last_fullsized":"2024-04-25T10:11:13.083279+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:08.901528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556838+0000","last_change":"2024-04-25T10:11:11.098061+0000","last_active":"2024-04-25T10:11:13.556838+0000","last_peered":"2024-04-25T10:11:13.556838+0000","last_clean":"2024-04-25T10:11:13.556838+0000","last_became_active":"2024-04-25T10:11:11.097380+0000","last_became_peered":"2024-04-25T10:11:11.097380+0000","last_unstale":"2024-04-25T10:11:13.556838+0000","last_undegraded":"2024-04-25T10:11:13.556838+0000","last_fullsized":"2024-04-25T10:11:13.556838+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:08.433048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989550+0000","last_change":"2024-04-25T10:11:12.095413+0000","last_active":"2024-04-25T10:11:13.989550+0000","last_peered":"2024-04-25T10:11:13.989550+0000","last_clean":"2024-04-25T10:11:13.989550+0000","last_became_active":"2024-04-25T10:11:12.095038+0000","last_became_peered":"2024-04-25T10:11:12.095038+0000","last_unstale":"2024-04-25T10:11:13.989550+0000","last_undegraded":"2024-04-25T10:11:13.989550+0000","last_fullsized":"2024-04-25T10:11:13.989550+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:48.782635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113611+0000","last_change":"2024-04-25T10:11:11.090017+0000","last_active":"2024-04-25T10:11:13.113611+0000","last_peered":"2024-04-25T10:11:13.113611+0000","last_clean":"2024-04-25T10:11:13.113611+0000","last_became_active":"2024-04-25T10:11:11.089277+0000","last_became_peered":"2024-04-25T10:11:11.089277+0000","last_unstale":"2024-04-25T10:11:13.113611+0000","last_undegraded":"2024-04-25T10:11:13.113611+0000","last_fullsized":"2024-04-25T10:11:13.113611+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:20.432311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086387+0000","last_change":"2024-04-25T10:11:12.089643+0000","last_active":"2024-04-25T10:11:13.086387+0000","last_peered":"2024-04-25T10:11:13.086387+0000","last_clean":"2024-04-25T10:11:13.086387+0000","last_became_active":"2024-04-25T10:11:12.089283+0000","last_became_peered":"2024-04-25T10:11:12.089283+0000","last_unstale":"2024-04-25T10:11:13.086387+0000","last_undegraded":"2024-04-25T10:11:13.086387+0000","last_fullsized":"2024-04-25T10:11:13.086387+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:26.143541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488946+0000","last_change":"2024-04-25T10:11:11.082632+0000","last_active":"2024-04-25T10:11:14.488946+0000","last_peered":"2024-04-25T10:11:14.488946+0000","last_clean":"2024-04-25T10:11:14.488946+0000","last_became_active":"2024-04-25T10:11:11.082337+0000","last_became_peered":"2024-04-25T10:11:11.082337+0000","last_unstale":"2024-04-25T10:11:14.488946+0000","last_undegraded":"2024-04-25T10:11:14.488946+0000","last_fullsized":"2024-04-25T10:11:14.488946+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:48.425028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.087066+0000","last_change":"2024-04-25T10:11:12.093345+0000","last_active":"2024-04-25T10:11:13.087066+0000","last_peered":"2024-04-25T10:11:13.087066+0000","last_clean":"2024-04-25T10:11:13.087066+0000","last_became_active":"2024-04-25T10:11:12.092955+0000","last_became_peered":"2024-04-25T10:11:12.092955+0000","last_unstale":"2024-04-25T10:11:13.087066+0000","last_undegraded":"2024-04-25T10:11:13.087066+0000","last_fullsized":"2024-04-25T10:11:13.087066+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:00.628620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488164+0000","last_change":"2024-04-25T10:11:11.083500+0000","last_active":"2024-04-25T10:11:14.488164+0000","last_peered":"2024-04-25T10:11:14.488164+0000","last_clean":"2024-04-25T10:11:14.488164+0000","last_became_active":"2024-04-25T10:11:11.083212+0000","last_became_peered":"2024-04-25T10:11:11.083212+0000","last_unstale":"2024-04-25T10:11:14.488164+0000","last_undegraded":"2024-04-25T10:11:14.488164+0000","last_fullsized":"2024-04-25T10:11:14.488164+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:14.618835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.489209+0000","last_change":"2024-04-25T10:11:12.080961+0000","last_active":"2024-04-25T10:11:14.489209+0000","last_peered":"2024-04-25T10:11:14.489209+0000","last_clean":"2024-04-25T10:11:14.489209+0000","last_became_active":"2024-04-25T10:11:12.080547+0000","last_became_peered":"2024-04-25T10:11:12.080547+0000","last_unstale":"2024-04-25T10:11:14.489209+0000","last_undegraded":"2024-04-25T10:11:14.489209+0000","last_fullsized":"2024-04-25T10:11:14.489209+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:43.230151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083749+0000","last_change":"2024-04-25T10:11:11.094930+0000","last_active":"2024-04-25T10:11:13.083749+0000","last_peered":"2024-04-25T10:11:13.083749+0000","last_clean":"2024-04-25T10:11:13.083749+0000","last_became_active":"2024-04-25T10:11:11.091689+0000","last_became_peered":"2024-04-25T10:11:11.091689+0000","last_unstale":"2024-04-25T10:11:13.083749+0000","last_undegraded":"2024-04-25T10:11:13.083749+0000","last_fullsized":"2024-04-25T10:11:13.083749+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:27.158143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488576+0000","last_change":"2024-04-25T10:11:12.100982+0000","last_active":"2024-04-25T10:11:14.488576+0000","last_peered":"2024-04-25T10:11:14.488576+0000","last_clean":"2024-04-25T10:11:14.488576+0000","last_became_active":"2024-04-25T10:11:12.100533+0000","last_became_peered":"2024-04-25T10:11:12.100533+0000","last_unstale":"2024-04-25T10:11:14.488576+0000","last_undegraded":"2024-04-25T10:11:14.488576+0000","last_fullsized":"2024-04-25T10:11:14.488576+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:06.670849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488683+0000","last_change":"2024-04-25T10:11:11.085963+0000","last_active":"2024-04-25T10:11:14.488683+0000","last_peered":"2024-04-25T10:11:14.488683+0000","last_clean":"2024-04-25T10:11:14.488683+0000","last_became_active":"2024-04-25T10:11:11.085624+0000","last_became_peered":"2024-04-25T10:11:11.085624+0000","last_unstale":"2024-04-25T10:11:14.488683+0000","last_undegraded":"2024-04-25T10:11:14.488683+0000","last_fullsized":"2024-04-25T10:11:14.488683+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:56.448766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086440+0000","last_change":"2024-04-25T10:11:12.092594+0000","last_active":"2024-04-25T10:11:13.086440+0000","last_peered":"2024-04-25T10:11:13.086440+0000","last_clean":"2024-04-25T10:11:13.086440+0000","last_became_active":"2024-04-25T10:11:12.092132+0000","last_became_peered":"2024-04-25T10:11:12.092132+0000","last_unstale":"2024-04-25T10:11:13.086440+0000","last_undegraded":"2024-04-25T10:11:13.086440+0000","last_fullsized":"2024-04-25T10:11:13.086440+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:08.696801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989668+0000","last_change":"2024-04-25T10:11:11.083271+0000","last_active":"2024-04-25T10:11:13.989668+0000","last_peered":"2024-04-25T10:11:13.989668+0000","last_clean":"2024-04-25T10:11:13.989668+0000","last_became_active":"2024-04-25T10:11:11.082765+0000","last_became_peered":"2024-04-25T10:11:11.082765+0000","last_unstale":"2024-04-25T10:11:13.989668+0000","last_undegraded":"2024-04-25T10:11:13.989668+0000","last_fullsized":"2024-04-25T10:11:13.989668+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:05.324017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084480+0000","last_change":"2024-04-25T10:11:12.072353+0000","last_active":"2024-04-25T10:11:13.084480+0000","last_peered":"2024-04-25T10:11:13.084480+0000","last_clean":"2024-04-25T10:11:13.084480+0000","last_became_active":"2024-04-25T10:11:12.072071+0000","last_became_peered":"2024-04-25T10:11:12.072071+0000","last_unstale":"2024-04-25T10:11:13.084480+0000","last_undegraded":"2024-04-25T10:11:13.084480+0000","last_fullsized":"2024-04-25T10:11:13.084480+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:21.442046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"107'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.080080+0000","last_change":"2024-04-25T10:11:11.084351+0000","last_active":"2024-04-25T10:11:14.080080+0000","last_peered":"2024-04-25T10:11:14.080080+0000","last_clean":"2024-04-25T10:11:14.080080+0000","last_became_active":"2024-04-25T10:11:11.083973+0000","last_became_peered":"2024-04-25T10:11:11.083973+0000","last_unstale":"2024-04-25T10:11:14.080080+0000","last_undegraded":"2024-04-25T10:11:14.080080+0000","last_fullsized":"2024-04-25T10:11:14.080080+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:48.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488260+0000","last_change":"2024-04-25T10:11:12.078950+0000","last_active":"2024-04-25T10:11:14.488260+0000","last_peered":"2024-04-25T10:11:14.488260+0000","last_clean":"2024-04-25T10:11:14.488260+0000","last_became_active":"2024-04-25T10:11:12.078619+0000","last_became_peered":"2024-04-25T10:11:12.078619+0000","last_unstale":"2024-04-25T10:11:14.488260+0000","last_undegraded":"2024-04-25T10:11:14.488260+0000","last_fullsized":"2024-04-25T10:11:14.488260+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:57.678011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085427+0000","last_change":"2024-04-25T10:11:11.094126+0000","last_active":"2024-04-25T10:11:13.085427+0000","last_peered":"2024-04-25T10:11:13.085427+0000","last_clean":"2024-04-25T10:11:13.085427+0000","last_became_active":"2024-04-25T10:11:11.093366+0000","last_became_peered":"2024-04-25T10:11:11.093366+0000","last_unstale":"2024-04-25T10:11:13.085427+0000","last_undegraded":"2024-04-25T10:11:13.085427+0000","last_fullsized":"2024-04-25T10:11:13.085427+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:04.486661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113689+0000","last_change":"2024-04-25T10:11:12.096360+0000","last_active":"2024-04-25T10:11:13.113689+0000","last_peered":"2024-04-25T10:11:13.113689+0000","last_clean":"2024-04-25T10:11:13.113689+0000","last_became_active":"2024-04-25T10:11:12.095951+0000","last_became_peered":"2024-04-25T10:11:12.095951+0000","last_unstale":"2024-04-25T10:11:13.113689+0000","last_undegraded":"2024-04-25T10:11:13.113689+0000","last_fullsized":"2024-04-25T10:11:13.113689+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:58.598879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989886+0000","last_change":"2024-04-25T10:11:11.090402+0000","last_active":"2024-04-25T10:11:13.989886+0000","last_peered":"2024-04-25T10:11:13.989886+0000","last_clean":"2024-04-25T10:11:13.989886+0000","last_became_active":"2024-04-25T10:11:11.089962+0000","last_became_peered":"2024-04-25T10:11:11.089962+0000","last_unstale":"2024-04-25T10:11:13.989886+0000","last_undegraded":"2024-04-25T10:11:13.989886+0000","last_fullsized":"2024-04-25T10:11:13.989886+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.346745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488352+0000","last_change":"2024-04-25T10:11:12.101904+0000","last_active":"2024-04-25T10:11:14.488352+0000","last_peered":"2024-04-25T10:11:14.488352+0000","last_clean":"2024-04-25T10:11:14.488352+0000","last_became_active":"2024-04-25T10:11:12.101594+0000","last_became_peered":"2024-04-25T10:11:12.101594+0000","last_unstale":"2024-04-25T10:11:14.488352+0000","last_undegraded":"2024-04-25T10:11:14.488352+0000","last_fullsized":"2024-04-25T10:11:14.488352+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:06.406045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085181+0000","last_change":"2024-04-25T10:11:11.102478+0000","last_active":"2024-04-25T10:11:13.085181+0000","last_peered":"2024-04-25T10:11:13.085181+0000","last_clean":"2024-04-25T10:11:13.085181+0000","last_became_active":"2024-04-25T10:11:11.101799+0000","last_became_peered":"2024-04-25T10:11:11.101799+0000","last_unstale":"2024-04-25T10:11:13.085181+0000","last_undegraded":"2024-04-25T10:11:13.085181+0000","last_fullsized":"2024-04-25T10:11:13.085181+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:53.235198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488710+0000","last_change":"2024-04-25T10:11:12.095515+0000","last_active":"2024-04-25T10:11:14.488710+0000","last_peered":"2024-04-25T10:11:14.488710+0000","last_clean":"2024-04-25T10:11:14.488710+0000","last_became_active":"2024-04-25T10:11:12.095230+0000","last_became_peered":"2024-04-25T10:11:12.095230+0000","last_unstale":"2024-04-25T10:11:14.488710+0000","last_undegraded":"2024-04-25T10:11:14.488710+0000","last_fullsized":"2024-04-25T10:11:14.488710+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:07.242052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488554+0000","last_change":"2024-04-25T10:11:11.096251+0000","last_active":"2024-04-25T10:11:14.488554+0000","last_peered":"2024-04-25T10:11:14.488554+0000","last_clean":"2024-04-25T10:11:14.488554+0000","last_became_active":"2024-04-25T10:11:11.095882+0000","last_became_peered":"2024-04-25T10:11:11.095882+0000","last_unstale":"2024-04-25T10:11:14.488554+0000","last_undegraded":"2024-04-25T10:11:14.488554+0000","last_fullsized":"2024-04-25T10:11:14.488554+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.852521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556729+0000","last_change":"2024-04-25T10:11:12.094559+0000","last_active":"2024-04-25T10:11:13.556729+0000","last_peered":"2024-04-25T10:11:13.556729+0000","last_clean":"2024-04-25T10:11:13.556729+0000","last_became_active":"2024-04-25T10:11:12.094184+0000","last_became_peered":"2024-04-25T10:11:12.094184+0000","last_unstale":"2024-04-25T10:11:13.556729+0000","last_undegraded":"2024-04-25T10:11:13.556729+0000","last_fullsized":"2024-04-25T10:11:13.556729+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.694914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083996+0000","last_change":"2024-04-25T10:11:11.096253+0000","last_active":"2024-04-25T10:11:13.083996+0000","last_peered":"2024-04-25T10:11:13.083996+0000","last_clean":"2024-04-25T10:11:13.083996+0000","last_became_active":"2024-04-25T10:11:11.095649+0000","last_became_peered":"2024-04-25T10:11:11.095649+0000","last_unstale":"2024-04-25T10:11:13.083996+0000","last_undegraded":"2024-04-25T10:11:13.083996+0000","last_fullsized":"2024-04-25T10:11:13.083996+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:29.187839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.114062+0000","last_change":"2024-04-25T10:11:12.095749+0000","last_active":"2024-04-25T10:11:13.114062+0000","last_peered":"2024-04-25T10:11:13.114062+0000","last_clean":"2024-04-25T10:11:13.114062+0000","last_became_active":"2024-04-25T10:11:12.095346+0000","last_became_peered":"2024-04-25T10:11:12.095346+0000","last_unstale":"2024-04-25T10:11:13.114062+0000","last_undegraded":"2024-04-25T10:11:13.114062+0000","last_fullsized":"2024-04-25T10:11:13.114062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:52.457595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488793+0000","last_change":"2024-04-25T10:11:11.096173+0000","last_active":"2024-04-25T10:11:14.488793+0000","last_peered":"2024-04-25T10:11:14.488793+0000","last_clean":"2024-04-25T10:11:14.488793+0000","last_became_active":"2024-04-25T10:11:11.095774+0000","last_became_peered":"2024-04-25T10:11:11.095774+0000","last_unstale":"2024-04-25T10:11:14.488793+0000","last_undegraded":"2024-04-25T10:11:14.488793+0000","last_fullsized":"2024-04-25T10:11:14.488793+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:20.219784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113941+0000","last_change":"2024-04-25T10:11:12.100193+0000","last_active":"2024-04-25T10:11:13.113941+0000","last_peered":"2024-04-25T10:11:13.113941+0000","last_clean":"2024-04-25T10:11:13.113941+0000","last_became_active":"2024-04-25T10:11:12.099799+0000","last_became_peered":"2024-04-25T10:11:12.099799+0000","last_unstale":"2024-04-25T10:11:13.113941+0000","last_undegraded":"2024-04-25T10:11:13.113941+0000","last_fullsized":"2024-04-25T10:11:13.113941+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:28.332101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488504+0000","last_change":"2024-04-25T10:11:11.085382+0000","last_active":"2024-04-25T10:11:14.488504+0000","last_peered":"2024-04-25T10:11:14.488504+0000","last_clean":"2024-04-25T10:11:14.488504+0000","last_became_active":"2024-04-25T10:11:11.085000+0000","last_became_peered":"2024-04-25T10:11:11.085000+0000","last_unstale":"2024-04-25T10:11:14.488504+0000","last_undegraded":"2024-04-25T10:11:14.488504+0000","last_fullsized":"2024-04-25T10:11:14.488504+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:15.245704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":334,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556018+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:11:13.556018+0000","last_peered":"2024-04-25T10:11:13.556018+0000","last_clean":"2024-04-25T10:11:13.556018+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:11:13.556018+0000","last_undegraded":"2024-04-25T10:11:13.556018+0000","last_fullsized":"2024-04-25T10:11:13.556018+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65000,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306840,"statfs":{"total":96636764160,"available":96570204160,"internally_reserved":0,"allocated":1929216,"data_stored":1208440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58872,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1347584,"data_stored":620128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574967,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54692,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317148,"statfs":{"total":96636764160,"available":96580759552,"internally_reserved":0,"allocated":1327104,"data_stored":617912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68588,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303252,"statfs":{"total":96636764160,"available":96566530048,"internally_reserved":0,"allocated":1335296,"data_stored":618004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72740,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299100,"statfs":{"total":96636764160,"available":96562278400,"internally_reserved":0,"allocated":1331200,"data_stored":618494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58840,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313000,"statfs":{"total":96636764160,"available":96576512000,"internally_reserved":0,"allocated":1318912,"data_stored":611254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58852,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312988,"statfs":{"total":96636764160,"available":96576499712,"internally_reserved":0,"allocated":1331200,"data_stored":617946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65012,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306828,"statfs":{"total":96636764160,"available":96570191872,"internally_reserved":0,"allocated":1937408,"data_stored":1209982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:19.519 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:19.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.646+0000 7f469a6ef640 1 Processor -- start 2024-04-25T10:11:19.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.646+0000 7f469a6ef640 1 -- start start 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f469a6ef640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f469415c9a0 0x7f46940c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f469a6ef640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f46940b1f70 0x7f469406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f469a6ef640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46940ce330 0x7f46940ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f469a6ef640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46940cecc0 con 0x7f46940b1f70 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f469a6ef640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46940c61d0 con 0x7f469415c9a0 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f469a6ef640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46940c64b0 con 0x7f46940ce330 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f46937fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f46940b1f70 0x7f469406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f46937fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f46940b1f70 0x7f469406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50350/0 (socket says 172.21.15.133:50350) 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f46937fe640 1 -- 172.21.15.133:0/3610139225 learned_addr learned my addr 172.21.15.133:0/3610139225 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f4693fff640 1 --2- 172.21.15.133:0/3610139225 >> v2:172.21.15.162:3300/0 conn(0x7f469415c9a0 0x7f46940c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f4698c8c640 1 --2- 172.21.15.133:0/3610139225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46940ce330 0x7f46940ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f46937fe640 1 -- 172.21.15.133:0/3610139225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46940ce330 msgr2=0x7f46940ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f46937fe640 1 --2- 172.21.15.133:0/3610139225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46940ce330 0x7f46940ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f46937fe640 1 -- 172.21.15.133:0/3610139225 >> v2:172.21.15.162:3300/0 conn(0x7f469415c9a0 msgr2=0x7f46940c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f46937fe640 1 --2- 172.21.15.133:0/3610139225 >> v2:172.21.15.162:3300/0 conn(0x7f469415c9a0 0x7f46940c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f46937fe640 1 -- 172.21.15.133:0/3610139225 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469406d4f0 con 0x7f46940b1f70 2024-04-25T10:11:19.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f46937fe640 1 --2- 172.21.15.133:0/3610139225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46940b1f70 0x7f469406dcf0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f469406f5f0 tx=0x7f467c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63658efa3386f077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f4692ffd640 1 -- 172.21.15.133:0/3610139225 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0a5800 con 0x7f46940b1f70 2024-04-25T10:11:19.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f4692ffd640 1 -- 172.21.15.133:0/3610139225 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c0a59a0 con 0x7f46940b1f70 2024-04-25T10:11:19.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.650+0000 7f4692ffd640 1 -- 172.21.15.133:0/3610139225 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0a5d70 con 0x7f46940b1f70 2024-04-25T10:11:19.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 -- 172.21.15.133:0/3610139225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46940b1f70 msgr2=0x7f469406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 --2- 172.21.15.133:0/3610139225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46940b1f70 0x7f469406dcf0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f469406f5f0 tx=0x7f467c01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 -- 172.21.15.133:0/3610139225 shutdown_connections 2024-04-25T10:11:19.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 --2- 172.21.15.133:0/3610139225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46940ce330 0x7f46940ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 --2- 172.21.15.133:0/3610139225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46940b1f70 0x7f469406dcf0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 --2- 172.21.15.133:0/3610139225 >> v2:172.21.15.162:3300/0 conn(0x7f469415c9a0 0x7f46940c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 -- 172.21.15.133:0/3610139225 >> 172.21.15.133:0/3610139225 conn(0x7f4694067000 msgr2=0x7f469406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 -- 172.21.15.133:0/3610139225 shutdown_connections 2024-04-25T10:11:19.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 -- 172.21.15.133:0/3610139225 wait complete. 2024-04-25T10:11:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 Processor -- start 2024-04-25T10:11:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 -- start start 2024-04-25T10:11:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46940b1f70 0x7f4694158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46940ce330 0x7f469406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f469415c9a0 0x7f469406b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469415cf00 con 0x7f46940ce330 2024-04-25T10:11:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469415cd80 con 0x7f46940b1f70 2024-04-25T10:11:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469415d950 con 0x7f469415c9a0 2024-04-25T10:11:19.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f46937fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46940ce330 0x7f469406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f46937fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46940ce330 0x7f469406ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50354/0 (socket says 172.21.15.133:50354) 2024-04-25T10:11:19.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f46937fe640 1 -- 172.21.15.133:0/2545557114 learned_addr learned my addr 172.21.15.133:0/2545557114 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:19.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f4698c8c640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f469415c9a0 0x7f469406b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f4693fff640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46940b1f70 0x7f4694158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f46937fe640 1 -- 172.21.15.133:0/2545557114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f469415c9a0 msgr2=0x7f469406b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f46937fe640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f469415c9a0 0x7f469406b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f46937fe640 1 -- 172.21.15.133:0/2545557114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46940b1f70 msgr2=0x7f4694158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f46937fe640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46940b1f70 0x7f4694158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f46937fe640 1 -- 172.21.15.133:0/2545557114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694152ef0 con 0x7f46940ce330 2024-04-25T10:11:19.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f4698c8c640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f469415c9a0 0x7f469406b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:19.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f4693fff640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46940b1f70 0x7f4694158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:19.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f46937fe640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46940ce330 0x7f469406ac90 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f467c0af040 tx=0x7f467c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f4690ff9640 1 -- 172.21.15.133:0/2545557114 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0b8670 con 0x7f46940ce330 2024-04-25T10:11:19.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f4690ff9640 1 -- 172.21.15.133:0/2545557114 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c0be070 con 0x7f46940ce330 2024-04-25T10:11:19.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 -- 172.21.15.133:0/2545557114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4694159330 con 0x7f46940ce330 2024-04-25T10:11:19.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f4690ff9640 1 -- 172.21.15.133:0/2545557114 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0b8a70 con 0x7f46940ce330 2024-04-25T10:11:19.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.654+0000 7f469a6ef640 1 -- 172.21.15.133:0/2545557114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4694153470 con 0x7f46940ce330 2024-04-25T10:11:19.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.658+0000 7f469a6ef640 1 -- 172.21.15.133:0/2545557114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4694068ef0 con 0x7f46940ce330 2024-04-25T10:11:19.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.658+0000 7f4690ff9640 1 -- 172.21.15.133:0/2545557114 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f467c0b8c10 con 0x7f46940ce330 2024-04-25T10:11:19.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.658+0000 7f4690ff9640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4674066f60 0x7f4674069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.658+0000 7f4693fff640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4674066f60 0x7f4674069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.658+0000 7f4690ff9640 1 -- 172.21.15.133:0/2545557114 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f467c120450 con 0x7f46940ce330 2024-04-25T10:11:19.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.658+0000 7f4693fff640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4674066f60 0x7f4674069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4694154de0 tx=0x7f4684008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.662+0000 7f4690ff9640 1 -- 172.21.15.133:0/2545557114 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f467c0165e0 con 0x7f46940ce330 2024-04-25T10:11:19.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.814+0000 7f469a6ef640 1 -- 172.21.15.133:0/2545557114 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f469415e180 con 0x7f4674066f60 2024-04-25T10:11:19.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.826+0000 7f4690ff9640 1 -- 172.21.15.133:0/2545557114 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252484 (secure 0 0 0) 0x7f469415e180 con 0x7f4674066f60 2024-04-25T10:11:19.832 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:19.833 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:11:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 -- 172.21.15.133:0/2545557114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4674066f60 msgr2=0x7f4674069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4674066f60 0x7f4674069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4694154de0 tx=0x7f4684008040 comp rx=0 tx=0).stop 2024-04-25T10:11:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 -- 172.21.15.133:0/2545557114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46940ce330 msgr2=0x7f469406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46940ce330 0x7f469406ac90 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f467c0af040 tx=0x7f467c019760 comp rx=0 tx=0).stop 2024-04-25T10:11:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 -- 172.21.15.133:0/2545557114 shutdown_connections 2024-04-25T10:11:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f469415c9a0 0x7f469406b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4674066f60 0x7f4674069420 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46940ce330 0x7f469406ac90 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 --2- 172.21.15.133:0/2545557114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46940b1f70 0x7f4694158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 -- 172.21.15.133:0/2545557114 >> 172.21.15.133:0/2545557114 conn(0x7f4694067000 msgr2=0x7f469415afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 -- 172.21.15.133:0/2545557114 shutdown_connections 2024-04-25T10:11:19.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:19.830+0000 7f469a6ef640 1 -- 172.21.15.133:0/2545557114 wait complete. 2024-04-25T10:11:19.848 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-04-25T10:11:17.879043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":502596,"kb_used_data":11580,"kb_used_omap":12,"kb_used_meta":490867,"kb_avail":754472124,"statfs":{"total":773094113280,"available":772579454976,"internally_reserved":0,"allocated":11857920,"data_stored":6122160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":502648369},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.816607"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086657+0000","last_change":"2024-04-25T10:11:12.093000+0000","last_active":"2024-04-25T10:11:13.086657+0000","last_peered":"2024-04-25T10:11:13.086657+0000","last_clean":"2024-04-25T10:11:13.086657+0000","last_became_active":"2024-04-25T10:11:12.092705+0000","last_became_peered":"2024-04-25T10:11:12.092705+0000","last_unstale":"2024-04-25T10:11:13.086657+0000","last_undegraded":"2024-04-25T10:11:13.086657+0000","last_fullsized":"2024-04-25T10:11:13.086657+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:51.418315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083695+0000","last_change":"2024-04-25T10:11:12.081569+0000","last_active":"2024-04-25T10:11:13.083695+0000","last_peered":"2024-04-25T10:11:13.083695+0000","last_clean":"2024-04-25T10:11:13.083695+0000","last_became_active":"2024-04-25T10:11:12.081160+0000","last_became_peered":"2024-04-25T10:11:12.081160+0000","last_unstale":"2024-04-25T10:11:13.083695+0000","last_undegraded":"2024-04-25T10:11:13.083695+0000","last_fullsized":"2024-04-25T10:11:13.083695+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:09.967725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086774+0000","last_change":"2024-04-25T10:11:12.098051+0000","last_active":"2024-04-25T10:11:13.086774+0000","last_peered":"2024-04-25T10:11:13.086774+0000","last_clean":"2024-04-25T10:11:13.086774+0000","last_became_active":"2024-04-25T10:11:12.097741+0000","last_became_peered":"2024-04-25T10:11:12.097741+0000","last_unstale":"2024-04-25T10:11:13.086774+0000","last_undegraded":"2024-04-25T10:11:13.086774+0000","last_fullsized":"2024-04-25T10:11:13.086774+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:08.116662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.489026+0000","last_change":"2024-04-25T10:11:12.078453+0000","last_active":"2024-04-25T10:11:14.489026+0000","last_peered":"2024-04-25T10:11:14.489026+0000","last_clean":"2024-04-25T10:11:14.489026+0000","last_became_active":"2024-04-25T10:11:12.078089+0000","last_became_peered":"2024-04-25T10:11:12.078089+0000","last_unstale":"2024-04-25T10:11:14.489026+0000","last_undegraded":"2024-04-25T10:11:14.489026+0000","last_fullsized":"2024-04-25T10:11:14.489026+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:08.740865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989481+0000","last_change":"2024-04-25T10:11:12.094445+0000","last_active":"2024-04-25T10:11:13.989481+0000","last_peered":"2024-04-25T10:11:13.989481+0000","last_clean":"2024-04-25T10:11:13.989481+0000","last_became_active":"2024-04-25T10:11:12.091729+0000","last_became_peered":"2024-04-25T10:11:12.091729+0000","last_unstale":"2024-04-25T10:11:13.989481+0000","last_undegraded":"2024-04-25T10:11:13.989481+0000","last_fullsized":"2024-04-25T10:11:13.989481+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:32.737015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083672+0000","last_change":"2024-04-25T10:11:12.097145+0000","last_active":"2024-04-25T10:11:13.083672+0000","last_peered":"2024-04-25T10:11:13.083672+0000","last_clean":"2024-04-25T10:11:13.083672+0000","last_became_active":"2024-04-25T10:11:12.096860+0000","last_became_peered":"2024-04-25T10:11:12.096860+0000","last_unstale":"2024-04-25T10:11:13.083672+0000","last_undegraded":"2024-04-25T10:11:13.083672+0000","last_fullsized":"2024-04-25T10:11:13.083672+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:47.835475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083031+0000","last_change":"2024-04-25T10:11:12.092527+0000","last_active":"2024-04-25T10:11:13.083031+0000","last_peered":"2024-04-25T10:11:13.083031+0000","last_clean":"2024-04-25T10:11:13.083031+0000","last_became_active":"2024-04-25T10:11:12.092129+0000","last_became_peered":"2024-04-25T10:11:12.092129+0000","last_unstale":"2024-04-25T10:11:13.083031+0000","last_undegraded":"2024-04-25T10:11:13.083031+0000","last_fullsized":"2024-04-25T10:11:13.083031+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.333751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488223+0000","last_change":"2024-04-25T10:11:12.092590+0000","last_active":"2024-04-25T10:11:14.488223+0000","last_peered":"2024-04-25T10:11:14.488223+0000","last_clean":"2024-04-25T10:11:14.488223+0000","last_became_active":"2024-04-25T10:11:12.092318+0000","last_became_peered":"2024-04-25T10:11:12.092318+0000","last_unstale":"2024-04-25T10:11:14.488223+0000","last_undegraded":"2024-04-25T10:11:14.488223+0000","last_fullsized":"2024-04-25T10:11:14.488223+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:57.429161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084231+0000","last_change":"2024-04-25T10:11:12.091826+0000","last_active":"2024-04-25T10:11:13.084231+0000","last_peered":"2024-04-25T10:11:13.084231+0000","last_clean":"2024-04-25T10:11:13.084231+0000","last_became_active":"2024-04-25T10:11:12.086628+0000","last_became_peered":"2024-04-25T10:11:12.086628+0000","last_unstale":"2024-04-25T10:11:13.084231+0000","last_undegraded":"2024-04-25T10:11:13.084231+0000","last_fullsized":"2024-04-25T10:11:13.084231+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:16.240462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989727+0000","last_change":"2024-04-25T10:11:12.103625+0000","last_active":"2024-04-25T10:11:13.989727+0000","last_peered":"2024-04-25T10:11:13.989727+0000","last_clean":"2024-04-25T10:11:13.989727+0000","last_became_active":"2024-04-25T10:11:12.103237+0000","last_became_peered":"2024-04-25T10:11:12.103237+0000","last_unstale":"2024-04-25T10:11:13.989727+0000","last_undegraded":"2024-04-25T10:11:13.989727+0000","last_fullsized":"2024-04-25T10:11:13.989727+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:08.740865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113692+0000","last_change":"2024-04-25T10:11:12.105007+0000","last_active":"2024-04-25T10:11:13.113692+0000","last_peered":"2024-04-25T10:11:13.113692+0000","last_clean":"2024-04-25T10:11:13.113692+0000","last_became_active":"2024-04-25T10:11:12.104721+0000","last_became_peered":"2024-04-25T10:11:12.104721+0000","last_unstale":"2024-04-25T10:11:13.113692+0000","last_undegraded":"2024-04-25T10:11:13.113692+0000","last_fullsized":"2024-04-25T10:11:13.113692+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:00.349109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085554+0000","last_change":"2024-04-25T10:11:12.077945+0000","last_active":"2024-04-25T10:11:13.085554+0000","last_peered":"2024-04-25T10:11:13.085554+0000","last_clean":"2024-04-25T10:11:13.085554+0000","last_became_active":"2024-04-25T10:11:12.077679+0000","last_became_peered":"2024-04-25T10:11:12.077679+0000","last_unstale":"2024-04-25T10:11:13.085554+0000","last_undegraded":"2024-04-25T10:11:13.085554+0000","last_fullsized":"2024-04-25T10:11:13.085554+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:08.740865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488635+0000","last_change":"2024-04-25T10:11:12.097231+0000","last_active":"2024-04-25T10:11:14.488635+0000","last_peered":"2024-04-25T10:11:14.488635+0000","last_clean":"2024-04-25T10:11:14.488635+0000","last_became_active":"2024-04-25T10:11:12.096930+0000","last_became_peered":"2024-04-25T10:11:12.096930+0000","last_unstale":"2024-04-25T10:11:14.488635+0000","last_undegraded":"2024-04-25T10:11:14.488635+0000","last_fullsized":"2024-04-25T10:11:14.488635+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:48.782635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556499+0000","last_change":"2024-04-25T10:11:12.088023+0000","last_active":"2024-04-25T10:11:13.556499+0000","last_peered":"2024-04-25T10:11:13.556499+0000","last_clean":"2024-04-25T10:11:13.556499+0000","last_became_active":"2024-04-25T10:11:12.087708+0000","last_became_peered":"2024-04-25T10:11:12.087708+0000","last_unstale":"2024-04-25T10:11:13.556499+0000","last_undegraded":"2024-04-25T10:11:13.556499+0000","last_fullsized":"2024-04-25T10:11:13.556499+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:49.410252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556300+0000","last_change":"2024-04-25T10:11:12.097091+0000","last_active":"2024-04-25T10:11:13.556300+0000","last_peered":"2024-04-25T10:11:13.556300+0000","last_clean":"2024-04-25T10:11:13.556300+0000","last_became_active":"2024-04-25T10:11:12.096769+0000","last_became_peered":"2024-04-25T10:11:12.096769+0000","last_unstale":"2024-04-25T10:11:13.556300+0000","last_undegraded":"2024-04-25T10:11:13.556300+0000","last_fullsized":"2024-04-25T10:11:13.556300+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:49.119547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556297+0000","last_change":"2024-04-25T10:11:12.084939+0000","last_active":"2024-04-25T10:11:13.556297+0000","last_peered":"2024-04-25T10:11:13.556297+0000","last_clean":"2024-04-25T10:11:13.556297+0000","last_became_active":"2024-04-25T10:11:12.084662+0000","last_became_peered":"2024-04-25T10:11:12.084662+0000","last_unstale":"2024-04-25T10:11:13.556297+0000","last_undegraded":"2024-04-25T10:11:13.556297+0000","last_fullsized":"2024-04-25T10:11:13.556297+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:52.193713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086513+0000","last_change":"2024-04-25T10:11:12.102661+0000","last_active":"2024-04-25T10:11:13.086513+0000","last_peered":"2024-04-25T10:11:13.086513+0000","last_clean":"2024-04-25T10:11:13.086513+0000","last_became_active":"2024-04-25T10:11:12.102253+0000","last_became_peered":"2024-04-25T10:11:12.102253+0000","last_unstale":"2024-04-25T10:11:13.086513+0000","last_undegraded":"2024-04-25T10:11:13.086513+0000","last_fullsized":"2024-04-25T10:11:13.086513+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:54.286436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083905+0000","last_change":"2024-04-25T10:11:12.085683+0000","last_active":"2024-04-25T10:11:13.083905+0000","last_peered":"2024-04-25T10:11:13.083905+0000","last_clean":"2024-04-25T10:11:13.083905+0000","last_became_active":"2024-04-25T10:11:12.085323+0000","last_became_peered":"2024-04-25T10:11:12.085323+0000","last_unstale":"2024-04-25T10:11:13.083905+0000","last_undegraded":"2024-04-25T10:11:13.083905+0000","last_fullsized":"2024-04-25T10:11:13.083905+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.550100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086997+0000","last_change":"2024-04-25T10:11:12.101583+0000","last_active":"2024-04-25T10:11:13.086997+0000","last_peered":"2024-04-25T10:11:13.086997+0000","last_clean":"2024-04-25T10:11:13.086997+0000","last_became_active":"2024-04-25T10:11:12.101252+0000","last_became_peered":"2024-04-25T10:11:12.101252+0000","last_unstale":"2024-04-25T10:11:13.086997+0000","last_undegraded":"2024-04-25T10:11:13.086997+0000","last_fullsized":"2024-04-25T10:11:13.086997+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:49.812198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085907+0000","last_change":"2024-04-25T10:11:12.077198+0000","last_active":"2024-04-25T10:11:13.085907+0000","last_peered":"2024-04-25T10:11:13.085907+0000","last_clean":"2024-04-25T10:11:13.085907+0000","last_became_active":"2024-04-25T10:11:12.076888+0000","last_became_peered":"2024-04-25T10:11:12.076888+0000","last_unstale":"2024-04-25T10:11:13.085907+0000","last_undegraded":"2024-04-25T10:11:13.085907+0000","last_fullsized":"2024-04-25T10:11:13.085907+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:39.450225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488499+0000","last_change":"2024-04-25T10:11:12.101652+0000","last_active":"2024-04-25T10:11:14.488499+0000","last_peered":"2024-04-25T10:11:14.488499+0000","last_clean":"2024-04-25T10:11:14.488499+0000","last_became_active":"2024-04-25T10:11:12.101287+0000","last_became_peered":"2024-04-25T10:11:12.101287+0000","last_unstale":"2024-04-25T10:11:14.488499+0000","last_undegraded":"2024-04-25T10:11:14.488499+0000","last_fullsized":"2024-04-25T10:11:14.488499+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:31.171772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989550+0000","last_change":"2024-04-25T10:11:12.099476+0000","last_active":"2024-04-25T10:11:13.989550+0000","last_peered":"2024-04-25T10:11:13.989550+0000","last_clean":"2024-04-25T10:11:13.989550+0000","last_became_active":"2024-04-25T10:11:12.096232+0000","last_became_peered":"2024-04-25T10:11:12.096232+0000","last_unstale":"2024-04-25T10:11:13.989550+0000","last_undegraded":"2024-04-25T10:11:13.989550+0000","last_fullsized":"2024-04-25T10:11:13.989550+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:57.678011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085816+0000","last_change":"2024-04-25T10:11:12.092952+0000","last_active":"2024-04-25T10:11:13.085816+0000","last_peered":"2024-04-25T10:11:13.085816+0000","last_clean":"2024-04-25T10:11:13.085816+0000","last_became_active":"2024-04-25T10:11:12.092638+0000","last_became_peered":"2024-04-25T10:11:12.092638+0000","last_unstale":"2024-04-25T10:11:13.085816+0000","last_undegraded":"2024-04-25T10:11:13.085816+0000","last_fullsized":"2024-04-25T10:11:13.085816+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:13.864982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989141+0000","last_change":"2024-04-25T10:11:12.099563+0000","last_active":"2024-04-25T10:11:13.989141+0000","last_peered":"2024-04-25T10:11:13.989141+0000","last_clean":"2024-04-25T10:11:13.989141+0000","last_became_active":"2024-04-25T10:11:12.099213+0000","last_became_peered":"2024-04-25T10:11:12.099213+0000","last_unstale":"2024-04-25T10:11:13.989141+0000","last_undegraded":"2024-04-25T10:11:13.989141+0000","last_fullsized":"2024-04-25T10:11:13.989141+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:21.442046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085248+0000","last_change":"2024-04-25T10:11:12.101960+0000","last_active":"2024-04-25T10:11:13.085248+0000","last_peered":"2024-04-25T10:11:13.085248+0000","last_clean":"2024-04-25T10:11:13.085248+0000","last_became_active":"2024-04-25T10:11:12.101627+0000","last_became_peered":"2024-04-25T10:11:12.101627+0000","last_unstale":"2024-04-25T10:11:13.085248+0000","last_undegraded":"2024-04-25T10:11:13.085248+0000","last_fullsized":"2024-04-25T10:11:13.085248+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:48.782635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085244+0000","last_change":"2024-04-25T10:11:12.073651+0000","last_active":"2024-04-25T10:11:13.085244+0000","last_peered":"2024-04-25T10:11:13.085244+0000","last_clean":"2024-04-25T10:11:13.085244+0000","last_became_active":"2024-04-25T10:11:12.073370+0000","last_became_peered":"2024-04-25T10:11:12.073370+0000","last_unstale":"2024-04-25T10:11:13.085244+0000","last_undegraded":"2024-04-25T10:11:13.085244+0000","last_fullsized":"2024-04-25T10:11:13.085244+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:57.678011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113862+0000","last_change":"2024-04-25T10:11:12.095432+0000","last_active":"2024-04-25T10:11:13.113862+0000","last_peered":"2024-04-25T10:11:13.113862+0000","last_clean":"2024-04-25T10:11:13.113862+0000","last_became_active":"2024-04-25T10:11:12.090819+0000","last_became_peered":"2024-04-25T10:11:12.090819+0000","last_unstale":"2024-04-25T10:11:13.113862+0000","last_undegraded":"2024-04-25T10:11:13.113862+0000","last_fullsized":"2024-04-25T10:11:13.113862+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.127816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084714+0000","last_change":"2024-04-25T10:11:12.093139+0000","last_active":"2024-04-25T10:11:13.084714+0000","last_peered":"2024-04-25T10:11:13.084714+0000","last_clean":"2024-04-25T10:11:13.084714+0000","last_became_active":"2024-04-25T10:11:12.092725+0000","last_became_peered":"2024-04-25T10:11:12.092725+0000","last_unstale":"2024-04-25T10:11:13.084714+0000","last_undegraded":"2024-04-25T10:11:13.084714+0000","last_fullsized":"2024-04-25T10:11:13.084714+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:23.199454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488737+0000","last_change":"2024-04-25T10:11:12.087843+0000","last_active":"2024-04-25T10:11:14.488737+0000","last_peered":"2024-04-25T10:11:14.488737+0000","last_clean":"2024-04-25T10:11:14.488737+0000","last_became_active":"2024-04-25T10:11:12.087566+0000","last_became_peered":"2024-04-25T10:11:12.087566+0000","last_unstale":"2024-04-25T10:11:14.488737+0000","last_undegraded":"2024-04-25T10:11:14.488737+0000","last_fullsized":"2024-04-25T10:11:14.488737+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:14.164758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084909+0000","last_change":"2024-04-25T10:11:12.097168+0000","last_active":"2024-04-25T10:11:13.084909+0000","last_peered":"2024-04-25T10:11:13.084909+0000","last_clean":"2024-04-25T10:11:13.084909+0000","last_became_active":"2024-04-25T10:11:12.096321+0000","last_became_peered":"2024-04-25T10:11:12.096321+0000","last_unstale":"2024-04-25T10:11:13.084909+0000","last_undegraded":"2024-04-25T10:11:13.084909+0000","last_fullsized":"2024-04-25T10:11:13.084909+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:54.235179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083450+0000","last_change":"2024-04-25T10:11:12.084636+0000","last_active":"2024-04-25T10:11:13.083450+0000","last_peered":"2024-04-25T10:11:13.083450+0000","last_clean":"2024-04-25T10:11:13.083450+0000","last_became_active":"2024-04-25T10:11:12.084242+0000","last_became_peered":"2024-04-25T10:11:12.084242+0000","last_unstale":"2024-04-25T10:11:13.083450+0000","last_undegraded":"2024-04-25T10:11:13.083450+0000","last_fullsized":"2024-04-25T10:11:13.083450+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:16.362990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488401+0000","last_change":"2024-04-25T10:11:12.079666+0000","last_active":"2024-04-25T10:11:14.488401+0000","last_peered":"2024-04-25T10:11:14.488401+0000","last_clean":"2024-04-25T10:11:14.488401+0000","last_became_active":"2024-04-25T10:11:12.079391+0000","last_became_peered":"2024-04-25T10:11:12.079391+0000","last_unstale":"2024-04-25T10:11:14.488401+0000","last_undegraded":"2024-04-25T10:11:14.488401+0000","last_fullsized":"2024-04-25T10:11:14.488401+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:39.612970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086228+0000","last_change":"2024-04-25T10:11:12.088291+0000","last_active":"2024-04-25T10:11:13.086228+0000","last_peered":"2024-04-25T10:11:13.086228+0000","last_clean":"2024-04-25T10:11:13.086228+0000","last_became_active":"2024-04-25T10:11:12.087889+0000","last_became_peered":"2024-04-25T10:11:12.087889+0000","last_unstale":"2024-04-25T10:11:13.086228+0000","last_undegraded":"2024-04-25T10:11:13.086228+0000","last_fullsized":"2024-04-25T10:11:13.086228+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:37.003254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"106'4","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.489474+0000","last_change":"2024-04-25T10:11:11.099778+0000","last_active":"2024-04-25T10:11:14.489474+0000","last_peered":"2024-04-25T10:11:14.489474+0000","last_clean":"2024-04-25T10:11:14.489474+0000","last_became_active":"2024-04-25T10:11:11.099546+0000","last_became_peered":"2024-04-25T10:11:11.099546+0000","last_unstale":"2024-04-25T10:11:14.489474+0000","last_undegraded":"2024-04-25T10:11:14.489474+0000","last_fullsized":"2024-04-25T10:11:14.489474+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.540366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488250+0000","last_change":"2024-04-25T10:11:12.096146+0000","last_active":"2024-04-25T10:11:14.488250+0000","last_peered":"2024-04-25T10:11:14.488250+0000","last_clean":"2024-04-25T10:11:14.488250+0000","last_became_active":"2024-04-25T10:11:12.095773+0000","last_became_peered":"2024-04-25T10:11:12.095773+0000","last_unstale":"2024-04-25T10:11:14.488250+0000","last_undegraded":"2024-04-25T10:11:14.488250+0000","last_fullsized":"2024-04-25T10:11:14.488250+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:21.912912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488727+0000","last_change":"2024-04-25T10:11:11.085487+0000","last_active":"2024-04-25T10:11:14.488727+0000","last_peered":"2024-04-25T10:11:14.488727+0000","last_clean":"2024-04-25T10:11:14.488727+0000","last_became_active":"2024-04-25T10:11:11.085141+0000","last_became_peered":"2024-04-25T10:11:11.085141+0000","last_unstale":"2024-04-25T10:11:14.488727+0000","last_undegraded":"2024-04-25T10:11:14.488727+0000","last_fullsized":"2024-04-25T10:11:14.488727+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:36.501389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085009+0000","last_change":"2024-04-25T10:11:12.089201+0000","last_active":"2024-04-25T10:11:13.085009+0000","last_peered":"2024-04-25T10:11:13.085009+0000","last_clean":"2024-04-25T10:11:13.085009+0000","last_became_active":"2024-04-25T10:11:12.088789+0000","last_became_peered":"2024-04-25T10:11:12.088789+0000","last_unstale":"2024-04-25T10:11:13.085009+0000","last_undegraded":"2024-04-25T10:11:13.085009+0000","last_fullsized":"2024-04-25T10:11:13.085009+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:32.737015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085280+0000","last_change":"2024-04-25T10:11:11.095732+0000","last_active":"2024-04-25T10:11:13.085280+0000","last_peered":"2024-04-25T10:11:13.085280+0000","last_clean":"2024-04-25T10:11:13.085280+0000","last_became_active":"2024-04-25T10:11:11.095005+0000","last_became_peered":"2024-04-25T10:11:11.095005+0000","last_unstale":"2024-04-25T10:11:13.085280+0000","last_undegraded":"2024-04-25T10:11:13.085280+0000","last_fullsized":"2024-04-25T10:11:13.085280+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:05.324017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113762+0000","last_change":"2024-04-25T10:11:12.093315+0000","last_active":"2024-04-25T10:11:13.113762+0000","last_peered":"2024-04-25T10:11:13.113762+0000","last_clean":"2024-04-25T10:11:13.113762+0000","last_became_active":"2024-04-25T10:11:12.092954+0000","last_became_peered":"2024-04-25T10:11:12.092954+0000","last_unstale":"2024-04-25T10:11:13.113762+0000","last_undegraded":"2024-04-25T10:11:13.113762+0000","last_fullsized":"2024-04-25T10:11:13.113762+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:23.327457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989903+0000","last_change":"2024-04-25T10:11:11.094878+0000","last_active":"2024-04-25T10:11:13.989903+0000","last_peered":"2024-04-25T10:11:13.989903+0000","last_clean":"2024-04-25T10:11:13.989903+0000","last_became_active":"2024-04-25T10:11:11.094453+0000","last_became_peered":"2024-04-25T10:11:11.094453+0000","last_unstale":"2024-04-25T10:11:13.989903+0000","last_undegraded":"2024-04-25T10:11:13.989903+0000","last_fullsized":"2024-04-25T10:11:13.989903+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:02.208178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.489181+0000","last_change":"2024-04-25T10:11:12.091705+0000","last_active":"2024-04-25T10:11:14.489181+0000","last_peered":"2024-04-25T10:11:14.489181+0000","last_clean":"2024-04-25T10:11:14.489181+0000","last_became_active":"2024-04-25T10:11:12.091409+0000","last_became_peered":"2024-04-25T10:11:12.091409+0000","last_unstale":"2024-04-25T10:11:14.489181+0000","last_undegraded":"2024-04-25T10:11:14.489181+0000","last_fullsized":"2024-04-25T10:11:14.489181+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:39.450225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556400+0000","last_change":"2024-04-25T10:11:11.100430+0000","last_active":"2024-04-25T10:11:13.556400+0000","last_peered":"2024-04-25T10:11:13.556400+0000","last_clean":"2024-04-25T10:11:13.556400+0000","last_became_active":"2024-04-25T10:11:11.099788+0000","last_became_peered":"2024-04-25T10:11:11.099788+0000","last_unstale":"2024-04-25T10:11:13.556400+0000","last_undegraded":"2024-04-25T10:11:13.556400+0000","last_fullsized":"2024-04-25T10:11:13.556400+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:23.869390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.087386+0000","last_change":"2024-04-25T10:11:12.087916+0000","last_active":"2024-04-25T10:11:13.087386+0000","last_peered":"2024-04-25T10:11:13.087386+0000","last_clean":"2024-04-25T10:11:13.087386+0000","last_became_active":"2024-04-25T10:11:12.087636+0000","last_became_peered":"2024-04-25T10:11:12.087636+0000","last_unstale":"2024-04-25T10:11:13.087386+0000","last_undegraded":"2024-04-25T10:11:13.087386+0000","last_fullsized":"2024-04-25T10:11:13.087386+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:14.115921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113283+0000","last_change":"2024-04-25T10:11:11.104519+0000","last_active":"2024-04-25T10:11:13.113283+0000","last_peered":"2024-04-25T10:11:13.113283+0000","last_clean":"2024-04-25T10:11:13.113283+0000","last_became_active":"2024-04-25T10:11:11.103889+0000","last_became_peered":"2024-04-25T10:11:11.103889+0000","last_unstale":"2024-04-25T10:11:13.113283+0000","last_undegraded":"2024-04-25T10:11:13.113283+0000","last_fullsized":"2024-04-25T10:11:13.113283+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:22.189720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084118+0000","last_change":"2024-04-25T10:11:12.093843+0000","last_active":"2024-04-25T10:11:13.084118+0000","last_peered":"2024-04-25T10:11:13.084118+0000","last_clean":"2024-04-25T10:11:13.084118+0000","last_became_active":"2024-04-25T10:11:12.093567+0000","last_became_peered":"2024-04-25T10:11:12.093567+0000","last_unstale":"2024-04-25T10:11:13.084118+0000","last_undegraded":"2024-04-25T10:11:13.084118+0000","last_fullsized":"2024-04-25T10:11:13.084118+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:24.573689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084628+0000","last_change":"2024-04-25T10:11:11.102540+0000","last_active":"2024-04-25T10:11:13.084628+0000","last_peered":"2024-04-25T10:11:13.084628+0000","last_clean":"2024-04-25T10:11:13.084628+0000","last_became_active":"2024-04-25T10:11:11.101923+0000","last_became_peered":"2024-04-25T10:11:11.101923+0000","last_unstale":"2024-04-25T10:11:13.084628+0000","last_undegraded":"2024-04-25T10:11:13.084628+0000","last_fullsized":"2024-04-25T10:11:13.084628+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:02.208178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.489393+0000","last_change":"2024-04-25T10:11:12.082040+0000","last_active":"2024-04-25T10:11:14.489393+0000","last_peered":"2024-04-25T10:11:14.489393+0000","last_clean":"2024-04-25T10:11:14.489393+0000","last_became_active":"2024-04-25T10:11:12.081618+0000","last_became_peered":"2024-04-25T10:11:12.081618+0000","last_unstale":"2024-04-25T10:11:14.489393+0000","last_undegraded":"2024-04-25T10:11:14.489393+0000","last_fullsized":"2024-04-25T10:11:14.489393+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:05.688368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.487997+0000","last_change":"2024-04-25T10:11:11.079085+0000","last_active":"2024-04-25T10:11:14.487997+0000","last_peered":"2024-04-25T10:11:14.487997+0000","last_clean":"2024-04-25T10:11:14.487997+0000","last_became_active":"2024-04-25T10:11:11.078454+0000","last_became_peered":"2024-04-25T10:11:11.078454+0000","last_unstale":"2024-04-25T10:11:14.487997+0000","last_undegraded":"2024-04-25T10:11:14.487997+0000","last_fullsized":"2024-04-25T10:11:14.487997+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:46.825741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085849+0000","last_change":"2024-04-25T10:11:12.102376+0000","last_active":"2024-04-25T10:11:13.085849+0000","last_peered":"2024-04-25T10:11:13.085849+0000","last_clean":"2024-04-25T10:11:13.085849+0000","last_became_active":"2024-04-25T10:11:12.102091+0000","last_became_peered":"2024-04-25T10:11:12.102091+0000","last_unstale":"2024-04-25T10:11:13.085849+0000","last_undegraded":"2024-04-25T10:11:13.085849+0000","last_fullsized":"2024-04-25T10:11:13.085849+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.617904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085389+0000","last_change":"2024-04-25T10:11:11.096132+0000","last_active":"2024-04-25T10:11:13.085389+0000","last_peered":"2024-04-25T10:11:13.085389+0000","last_clean":"2024-04-25T10:11:13.085389+0000","last_became_active":"2024-04-25T10:11:11.095503+0000","last_became_peered":"2024-04-25T10:11:11.095503+0000","last_unstale":"2024-04-25T10:11:13.085389+0000","last_undegraded":"2024-04-25T10:11:13.085389+0000","last_fullsized":"2024-04-25T10:11:13.085389+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:07.891794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989369+0000","last_change":"2024-04-25T10:11:12.091625+0000","last_active":"2024-04-25T10:11:13.989369+0000","last_peered":"2024-04-25T10:11:13.989369+0000","last_clean":"2024-04-25T10:11:13.989369+0000","last_became_active":"2024-04-25T10:11:12.090718+0000","last_became_peered":"2024-04-25T10:11:12.090718+0000","last_unstale":"2024-04-25T10:11:13.989369+0000","last_undegraded":"2024-04-25T10:11:13.989369+0000","last_fullsized":"2024-04-25T10:11:13.989369+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:54.235179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086045+0000","last_change":"2024-04-25T10:11:11.094363+0000","last_active":"2024-04-25T10:11:13.086045+0000","last_peered":"2024-04-25T10:11:13.086045+0000","last_clean":"2024-04-25T10:11:13.086045+0000","last_became_active":"2024-04-25T10:11:11.094037+0000","last_became_peered":"2024-04-25T10:11:11.094037+0000","last_unstale":"2024-04-25T10:11:13.086045+0000","last_undegraded":"2024-04-25T10:11:13.086045+0000","last_fullsized":"2024-04-25T10:11:13.086045+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:51.183979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488359+0000","last_change":"2024-04-25T10:11:12.096216+0000","last_active":"2024-04-25T10:11:14.488359+0000","last_peered":"2024-04-25T10:11:14.488359+0000","last_clean":"2024-04-25T10:11:14.488359+0000","last_became_active":"2024-04-25T10:11:12.095848+0000","last_became_peered":"2024-04-25T10:11:12.095848+0000","last_unstale":"2024-04-25T10:11:14.488359+0000","last_undegraded":"2024-04-25T10:11:14.488359+0000","last_fullsized":"2024-04-25T10:11:14.488359+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:53.849761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"106'3","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556694+0000","last_change":"2024-04-25T10:11:11.087036+0000","last_active":"2024-04-25T10:11:13.556694+0000","last_peered":"2024-04-25T10:11:13.556694+0000","last_clean":"2024-04-25T10:11:13.556694+0000","last_became_active":"2024-04-25T10:11:11.086301+0000","last_became_peered":"2024-04-25T10:11:11.086301+0000","last_unstale":"2024-04-25T10:11:13.556694+0000","last_undegraded":"2024-04-25T10:11:13.556694+0000","last_fullsized":"2024-04-25T10:11:13.556694+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:20.562287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989960+0000","last_change":"2024-04-25T10:11:12.091290+0000","last_active":"2024-04-25T10:11:13.989960+0000","last_peered":"2024-04-25T10:11:13.989960+0000","last_clean":"2024-04-25T10:11:13.989960+0000","last_became_active":"2024-04-25T10:11:12.090900+0000","last_became_peered":"2024-04-25T10:11:12.090900+0000","last_unstale":"2024-04-25T10:11:13.989960+0000","last_undegraded":"2024-04-25T10:11:13.989960+0000","last_fullsized":"2024-04-25T10:11:13.989960+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:39.450225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"106'6","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084966+0000","last_change":"2024-04-25T10:11:11.093719+0000","last_active":"2024-04-25T10:11:13.084966+0000","last_peered":"2024-04-25T10:11:13.084966+0000","last_clean":"2024-04-25T10:11:13.084966+0000","last_became_active":"2024-04-25T10:11:11.091809+0000","last_became_peered":"2024-04-25T10:11:11.091809+0000","last_unstale":"2024-04-25T10:11:13.084966+0000","last_undegraded":"2024-04-25T10:11:13.084966+0000","last_fullsized":"2024-04-25T10:11:13.084966+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:06.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113967+0000","last_change":"2024-04-25T10:11:12.099666+0000","last_active":"2024-04-25T10:11:13.113967+0000","last_peered":"2024-04-25T10:11:13.113967+0000","last_clean":"2024-04-25T10:11:13.113967+0000","last_became_active":"2024-04-25T10:11:12.096872+0000","last_became_peered":"2024-04-25T10:11:12.096872+0000","last_unstale":"2024-04-25T10:11:13.113967+0000","last_undegraded":"2024-04-25T10:11:13.113967+0000","last_fullsized":"2024-04-25T10:11:13.113967+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:36.323673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989828+0000","last_change":"2024-04-25T10:11:11.083667+0000","last_active":"2024-04-25T10:11:13.989828+0000","last_peered":"2024-04-25T10:11:13.989828+0000","last_clean":"2024-04-25T10:11:13.989828+0000","last_became_active":"2024-04-25T10:11:11.083377+0000","last_became_peered":"2024-04-25T10:11:11.083377+0000","last_unstale":"2024-04-25T10:11:13.989828+0000","last_undegraded":"2024-04-25T10:11:13.989828+0000","last_fullsized":"2024-04-25T10:11:13.989828+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:53.235198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085783+0000","last_change":"2024-04-25T10:11:12.079785+0000","last_active":"2024-04-25T10:11:13.085783+0000","last_peered":"2024-04-25T10:11:13.085783+0000","last_clean":"2024-04-25T10:11:13.085783+0000","last_became_active":"2024-04-25T10:11:12.079435+0000","last_became_peered":"2024-04-25T10:11:12.079435+0000","last_unstale":"2024-04-25T10:11:13.085783+0000","last_undegraded":"2024-04-25T10:11:13.085783+0000","last_fullsized":"2024-04-25T10:11:13.085783+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:21.442046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488397+0000","last_change":"2024-04-25T10:11:11.084530+0000","last_active":"2024-04-25T10:11:14.488397+0000","last_peered":"2024-04-25T10:11:14.488397+0000","last_clean":"2024-04-25T10:11:14.488397+0000","last_became_active":"2024-04-25T10:11:11.084051+0000","last_became_peered":"2024-04-25T10:11:11.084051+0000","last_unstale":"2024-04-25T10:11:14.488397+0000","last_undegraded":"2024-04-25T10:11:14.488397+0000","last_fullsized":"2024-04-25T10:11:14.488397+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:08.957991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488642+0000","last_change":"2024-04-25T10:11:12.086480+0000","last_active":"2024-04-25T10:11:14.488642+0000","last_peered":"2024-04-25T10:11:14.488642+0000","last_clean":"2024-04-25T10:11:14.488642+0000","last_became_active":"2024-04-25T10:11:12.086215+0000","last_became_peered":"2024-04-25T10:11:12.086215+0000","last_unstale":"2024-04-25T10:11:14.488642+0000","last_undegraded":"2024-04-25T10:11:14.488642+0000","last_fullsized":"2024-04-25T10:11:14.488642+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:21.125843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488474+0000","last_change":"2024-04-25T10:11:11.095999+0000","last_active":"2024-04-25T10:11:14.488474+0000","last_peered":"2024-04-25T10:11:14.488474+0000","last_clean":"2024-04-25T10:11:14.488474+0000","last_became_active":"2024-04-25T10:11:11.095678+0000","last_became_peered":"2024-04-25T10:11:11.095678+0000","last_unstale":"2024-04-25T10:11:14.488474+0000","last_undegraded":"2024-04-25T10:11:14.488474+0000","last_fullsized":"2024-04-25T10:11:14.488474+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:05.275695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086755+0000","last_change":"2024-04-25T10:11:12.102933+0000","last_active":"2024-04-25T10:11:13.086755+0000","last_peered":"2024-04-25T10:11:13.086755+0000","last_clean":"2024-04-25T10:11:13.086755+0000","last_became_active":"2024-04-25T10:11:12.101669+0000","last_became_peered":"2024-04-25T10:11:12.101669+0000","last_unstale":"2024-04-25T10:11:13.086755+0000","last_undegraded":"2024-04-25T10:11:13.086755+0000","last_fullsized":"2024-04-25T10:11:13.086755+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:50.105497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"106'8","reported_seq":22,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084384+0000","last_change":"2024-04-25T10:11:11.095809+0000","last_active":"2024-04-25T10:11:13.084384+0000","last_peered":"2024-04-25T10:11:13.084384+0000","last_clean":"2024-04-25T10:11:13.084384+0000","last_became_active":"2024-04-25T10:11:11.095117+0000","last_became_peered":"2024-04-25T10:11:11.095117+0000","last_unstale":"2024-04-25T10:11:13.084384+0000","last_undegraded":"2024-04-25T10:11:13.084384+0000","last_fullsized":"2024-04-25T10:11:13.084384+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.346745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488573+0000","last_change":"2024-04-25T10:11:12.084090+0000","last_active":"2024-04-25T10:11:14.488573+0000","last_peered":"2024-04-25T10:11:14.488573+0000","last_clean":"2024-04-25T10:11:14.488573+0000","last_became_active":"2024-04-25T10:11:12.083759+0000","last_became_peered":"2024-04-25T10:11:12.083759+0000","last_unstale":"2024-04-25T10:11:14.488573+0000","last_undegraded":"2024-04-25T10:11:14.488573+0000","last_fullsized":"2024-04-25T10:11:14.488573+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:51.833546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083606+0000","last_change":"2024-04-25T10:11:11.103097+0000","last_active":"2024-04-25T10:11:13.083606+0000","last_peered":"2024-04-25T10:11:13.083606+0000","last_clean":"2024-04-25T10:11:13.083606+0000","last_became_active":"2024-04-25T10:11:11.102480+0000","last_became_peered":"2024-04-25T10:11:11.102480+0000","last_unstale":"2024-04-25T10:11:13.083606+0000","last_undegraded":"2024-04-25T10:11:13.083606+0000","last_fullsized":"2024-04-25T10:11:13.083606+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:27.678274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083279+0000","last_change":"2024-04-25T10:11:12.082737+0000","last_active":"2024-04-25T10:11:13.083279+0000","last_peered":"2024-04-25T10:11:13.083279+0000","last_clean":"2024-04-25T10:11:13.083279+0000","last_became_active":"2024-04-25T10:11:12.082090+0000","last_became_peered":"2024-04-25T10:11:12.082090+0000","last_unstale":"2024-04-25T10:11:13.083279+0000","last_undegraded":"2024-04-25T10:11:13.083279+0000","last_fullsized":"2024-04-25T10:11:13.083279+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:08.901528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556838+0000","last_change":"2024-04-25T10:11:11.098061+0000","last_active":"2024-04-25T10:11:13.556838+0000","last_peered":"2024-04-25T10:11:13.556838+0000","last_clean":"2024-04-25T10:11:13.556838+0000","last_became_active":"2024-04-25T10:11:11.097380+0000","last_became_peered":"2024-04-25T10:11:11.097380+0000","last_unstale":"2024-04-25T10:11:13.556838+0000","last_undegraded":"2024-04-25T10:11:13.556838+0000","last_fullsized":"2024-04-25T10:11:13.556838+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:08.433048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989550+0000","last_change":"2024-04-25T10:11:12.095413+0000","last_active":"2024-04-25T10:11:13.989550+0000","last_peered":"2024-04-25T10:11:13.989550+0000","last_clean":"2024-04-25T10:11:13.989550+0000","last_became_active":"2024-04-25T10:11:12.095038+0000","last_became_peered":"2024-04-25T10:11:12.095038+0000","last_unstale":"2024-04-25T10:11:13.989550+0000","last_undegraded":"2024-04-25T10:11:13.989550+0000","last_fullsized":"2024-04-25T10:11:13.989550+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:48.782635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113611+0000","last_change":"2024-04-25T10:11:11.090017+0000","last_active":"2024-04-25T10:11:13.113611+0000","last_peered":"2024-04-25T10:11:13.113611+0000","last_clean":"2024-04-25T10:11:13.113611+0000","last_became_active":"2024-04-25T10:11:11.089277+0000","last_became_peered":"2024-04-25T10:11:11.089277+0000","last_unstale":"2024-04-25T10:11:13.113611+0000","last_undegraded":"2024-04-25T10:11:13.113611+0000","last_fullsized":"2024-04-25T10:11:13.113611+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:20.432311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086387+0000","last_change":"2024-04-25T10:11:12.089643+0000","last_active":"2024-04-25T10:11:13.086387+0000","last_peered":"2024-04-25T10:11:13.086387+0000","last_clean":"2024-04-25T10:11:13.086387+0000","last_became_active":"2024-04-25T10:11:12.089283+0000","last_became_peered":"2024-04-25T10:11:12.089283+0000","last_unstale":"2024-04-25T10:11:13.086387+0000","last_undegraded":"2024-04-25T10:11:13.086387+0000","last_fullsized":"2024-04-25T10:11:13.086387+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:26.143541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488946+0000","last_change":"2024-04-25T10:11:11.082632+0000","last_active":"2024-04-25T10:11:14.488946+0000","last_peered":"2024-04-25T10:11:14.488946+0000","last_clean":"2024-04-25T10:11:14.488946+0000","last_became_active":"2024-04-25T10:11:11.082337+0000","last_became_peered":"2024-04-25T10:11:11.082337+0000","last_unstale":"2024-04-25T10:11:14.488946+0000","last_undegraded":"2024-04-25T10:11:14.488946+0000","last_fullsized":"2024-04-25T10:11:14.488946+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:48.425028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.087066+0000","last_change":"2024-04-25T10:11:12.093345+0000","last_active":"2024-04-25T10:11:13.087066+0000","last_peered":"2024-04-25T10:11:13.087066+0000","last_clean":"2024-04-25T10:11:13.087066+0000","last_became_active":"2024-04-25T10:11:12.092955+0000","last_became_peered":"2024-04-25T10:11:12.092955+0000","last_unstale":"2024-04-25T10:11:13.087066+0000","last_undegraded":"2024-04-25T10:11:13.087066+0000","last_fullsized":"2024-04-25T10:11:13.087066+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:00.628620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488164+0000","last_change":"2024-04-25T10:11:11.083500+0000","last_active":"2024-04-25T10:11:14.488164+0000","last_peered":"2024-04-25T10:11:14.488164+0000","last_clean":"2024-04-25T10:11:14.488164+0000","last_became_active":"2024-04-25T10:11:11.083212+0000","last_became_peered":"2024-04-25T10:11:11.083212+0000","last_unstale":"2024-04-25T10:11:14.488164+0000","last_undegraded":"2024-04-25T10:11:14.488164+0000","last_fullsized":"2024-04-25T10:11:14.488164+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:14.618835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.489209+0000","last_change":"2024-04-25T10:11:12.080961+0000","last_active":"2024-04-25T10:11:14.489209+0000","last_peered":"2024-04-25T10:11:14.489209+0000","last_clean":"2024-04-25T10:11:14.489209+0000","last_became_active":"2024-04-25T10:11:12.080547+0000","last_became_peered":"2024-04-25T10:11:12.080547+0000","last_unstale":"2024-04-25T10:11:14.489209+0000","last_undegraded":"2024-04-25T10:11:14.489209+0000","last_fullsized":"2024-04-25T10:11:14.489209+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:43.230151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083749+0000","last_change":"2024-04-25T10:11:11.094930+0000","last_active":"2024-04-25T10:11:13.083749+0000","last_peered":"2024-04-25T10:11:13.083749+0000","last_clean":"2024-04-25T10:11:13.083749+0000","last_became_active":"2024-04-25T10:11:11.091689+0000","last_became_peered":"2024-04-25T10:11:11.091689+0000","last_unstale":"2024-04-25T10:11:13.083749+0000","last_undegraded":"2024-04-25T10:11:13.083749+0000","last_fullsized":"2024-04-25T10:11:13.083749+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:27.158143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488576+0000","last_change":"2024-04-25T10:11:12.100982+0000","last_active":"2024-04-25T10:11:14.488576+0000","last_peered":"2024-04-25T10:11:14.488576+0000","last_clean":"2024-04-25T10:11:14.488576+0000","last_became_active":"2024-04-25T10:11:12.100533+0000","last_became_peered":"2024-04-25T10:11:12.100533+0000","last_unstale":"2024-04-25T10:11:14.488576+0000","last_undegraded":"2024-04-25T10:11:14.488576+0000","last_fullsized":"2024-04-25T10:11:14.488576+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:06.670849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488683+0000","last_change":"2024-04-25T10:11:11.085963+0000","last_active":"2024-04-25T10:11:14.488683+0000","last_peered":"2024-04-25T10:11:14.488683+0000","last_clean":"2024-04-25T10:11:14.488683+0000","last_became_active":"2024-04-25T10:11:11.085624+0000","last_became_peered":"2024-04-25T10:11:11.085624+0000","last_unstale":"2024-04-25T10:11:14.488683+0000","last_undegraded":"2024-04-25T10:11:14.488683+0000","last_fullsized":"2024-04-25T10:11:14.488683+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:56.448766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.086440+0000","last_change":"2024-04-25T10:11:12.092594+0000","last_active":"2024-04-25T10:11:13.086440+0000","last_peered":"2024-04-25T10:11:13.086440+0000","last_clean":"2024-04-25T10:11:13.086440+0000","last_became_active":"2024-04-25T10:11:12.092132+0000","last_became_peered":"2024-04-25T10:11:12.092132+0000","last_unstale":"2024-04-25T10:11:13.086440+0000","last_undegraded":"2024-04-25T10:11:13.086440+0000","last_fullsized":"2024-04-25T10:11:13.086440+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:08.696801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989668+0000","last_change":"2024-04-25T10:11:11.083271+0000","last_active":"2024-04-25T10:11:13.989668+0000","last_peered":"2024-04-25T10:11:13.989668+0000","last_clean":"2024-04-25T10:11:13.989668+0000","last_became_active":"2024-04-25T10:11:11.082765+0000","last_became_peered":"2024-04-25T10:11:11.082765+0000","last_unstale":"2024-04-25T10:11:13.989668+0000","last_undegraded":"2024-04-25T10:11:13.989668+0000","last_fullsized":"2024-04-25T10:11:13.989668+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:05.324017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.084480+0000","last_change":"2024-04-25T10:11:12.072353+0000","last_active":"2024-04-25T10:11:13.084480+0000","last_peered":"2024-04-25T10:11:13.084480+0000","last_clean":"2024-04-25T10:11:13.084480+0000","last_became_active":"2024-04-25T10:11:12.072071+0000","last_became_peered":"2024-04-25T10:11:12.072071+0000","last_unstale":"2024-04-25T10:11:13.084480+0000","last_undegraded":"2024-04-25T10:11:13.084480+0000","last_fullsized":"2024-04-25T10:11:13.084480+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:21.442046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"107'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.080080+0000","last_change":"2024-04-25T10:11:11.084351+0000","last_active":"2024-04-25T10:11:14.080080+0000","last_peered":"2024-04-25T10:11:14.080080+0000","last_clean":"2024-04-25T10:11:14.080080+0000","last_became_active":"2024-04-25T10:11:11.083973+0000","last_became_peered":"2024-04-25T10:11:11.083973+0000","last_unstale":"2024-04-25T10:11:14.080080+0000","last_undegraded":"2024-04-25T10:11:14.080080+0000","last_fullsized":"2024-04-25T10:11:14.080080+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:48.109813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488260+0000","last_change":"2024-04-25T10:11:12.078950+0000","last_active":"2024-04-25T10:11:14.488260+0000","last_peered":"2024-04-25T10:11:14.488260+0000","last_clean":"2024-04-25T10:11:14.488260+0000","last_became_active":"2024-04-25T10:11:12.078619+0000","last_became_peered":"2024-04-25T10:11:12.078619+0000","last_unstale":"2024-04-25T10:11:14.488260+0000","last_undegraded":"2024-04-25T10:11:14.488260+0000","last_fullsized":"2024-04-25T10:11:14.488260+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:57.678011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085427+0000","last_change":"2024-04-25T10:11:11.094126+0000","last_active":"2024-04-25T10:11:13.085427+0000","last_peered":"2024-04-25T10:11:13.085427+0000","last_clean":"2024-04-25T10:11:13.085427+0000","last_became_active":"2024-04-25T10:11:11.093366+0000","last_became_peered":"2024-04-25T10:11:11.093366+0000","last_unstale":"2024-04-25T10:11:13.085427+0000","last_undegraded":"2024-04-25T10:11:13.085427+0000","last_fullsized":"2024-04-25T10:11:13.085427+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:04.486661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113689+0000","last_change":"2024-04-25T10:11:12.096360+0000","last_active":"2024-04-25T10:11:13.113689+0000","last_peered":"2024-04-25T10:11:13.113689+0000","last_clean":"2024-04-25T10:11:13.113689+0000","last_became_active":"2024-04-25T10:11:12.095951+0000","last_became_peered":"2024-04-25T10:11:12.095951+0000","last_unstale":"2024-04-25T10:11:13.113689+0000","last_undegraded":"2024-04-25T10:11:13.113689+0000","last_fullsized":"2024-04-25T10:11:13.113689+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:58.598879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.989886+0000","last_change":"2024-04-25T10:11:11.090402+0000","last_active":"2024-04-25T10:11:13.989886+0000","last_peered":"2024-04-25T10:11:13.989886+0000","last_clean":"2024-04-25T10:11:13.989886+0000","last_became_active":"2024-04-25T10:11:11.089962+0000","last_became_peered":"2024-04-25T10:11:11.089962+0000","last_unstale":"2024-04-25T10:11:13.989886+0000","last_undegraded":"2024-04-25T10:11:13.989886+0000","last_fullsized":"2024-04-25T10:11:13.989886+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:35.346745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488352+0000","last_change":"2024-04-25T10:11:12.101904+0000","last_active":"2024-04-25T10:11:14.488352+0000","last_peered":"2024-04-25T10:11:14.488352+0000","last_clean":"2024-04-25T10:11:14.488352+0000","last_became_active":"2024-04-25T10:11:12.101594+0000","last_became_peered":"2024-04-25T10:11:12.101594+0000","last_unstale":"2024-04-25T10:11:14.488352+0000","last_undegraded":"2024-04-25T10:11:14.488352+0000","last_fullsized":"2024-04-25T10:11:14.488352+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:06.406045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.085181+0000","last_change":"2024-04-25T10:11:11.102478+0000","last_active":"2024-04-25T10:11:13.085181+0000","last_peered":"2024-04-25T10:11:13.085181+0000","last_clean":"2024-04-25T10:11:13.085181+0000","last_became_active":"2024-04-25T10:11:11.101799+0000","last_became_peered":"2024-04-25T10:11:11.101799+0000","last_unstale":"2024-04-25T10:11:13.085181+0000","last_undegraded":"2024-04-25T10:11:13.085181+0000","last_fullsized":"2024-04-25T10:11:13.085181+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:53.235198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488710+0000","last_change":"2024-04-25T10:11:12.095515+0000","last_active":"2024-04-25T10:11:14.488710+0000","last_peered":"2024-04-25T10:11:14.488710+0000","last_clean":"2024-04-25T10:11:14.488710+0000","last_became_active":"2024-04-25T10:11:12.095230+0000","last_became_peered":"2024-04-25T10:11:12.095230+0000","last_unstale":"2024-04-25T10:11:14.488710+0000","last_undegraded":"2024-04-25T10:11:14.488710+0000","last_fullsized":"2024-04-25T10:11:14.488710+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:07.242052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488554+0000","last_change":"2024-04-25T10:11:11.096251+0000","last_active":"2024-04-25T10:11:14.488554+0000","last_peered":"2024-04-25T10:11:14.488554+0000","last_clean":"2024-04-25T10:11:14.488554+0000","last_became_active":"2024-04-25T10:11:11.095882+0000","last_became_peered":"2024-04-25T10:11:11.095882+0000","last_unstale":"2024-04-25T10:11:14.488554+0000","last_undegraded":"2024-04-25T10:11:14.488554+0000","last_fullsized":"2024-04-25T10:11:14.488554+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:53.852521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556729+0000","last_change":"2024-04-25T10:11:12.094559+0000","last_active":"2024-04-25T10:11:13.556729+0000","last_peered":"2024-04-25T10:11:13.556729+0000","last_clean":"2024-04-25T10:11:13.556729+0000","last_became_active":"2024-04-25T10:11:12.094184+0000","last_became_peered":"2024-04-25T10:11:12.094184+0000","last_unstale":"2024-04-25T10:11:13.556729+0000","last_undegraded":"2024-04-25T10:11:13.556729+0000","last_fullsized":"2024-04-25T10:11:13.556729+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.694914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.083996+0000","last_change":"2024-04-25T10:11:11.096253+0000","last_active":"2024-04-25T10:11:13.083996+0000","last_peered":"2024-04-25T10:11:13.083996+0000","last_clean":"2024-04-25T10:11:13.083996+0000","last_became_active":"2024-04-25T10:11:11.095649+0000","last_became_peered":"2024-04-25T10:11:11.095649+0000","last_unstale":"2024-04-25T10:11:13.083996+0000","last_undegraded":"2024-04-25T10:11:13.083996+0000","last_fullsized":"2024-04-25T10:11:13.083996+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:29.187839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.114062+0000","last_change":"2024-04-25T10:11:12.095749+0000","last_active":"2024-04-25T10:11:13.114062+0000","last_peered":"2024-04-25T10:11:13.114062+0000","last_clean":"2024-04-25T10:11:13.114062+0000","last_became_active":"2024-04-25T10:11:12.095346+0000","last_became_peered":"2024-04-25T10:11:12.095346+0000","last_unstale":"2024-04-25T10:11:13.114062+0000","last_undegraded":"2024-04-25T10:11:13.114062+0000","last_fullsized":"2024-04-25T10:11:13.114062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:52.457595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488793+0000","last_change":"2024-04-25T10:11:11.096173+0000","last_active":"2024-04-25T10:11:14.488793+0000","last_peered":"2024-04-25T10:11:14.488793+0000","last_clean":"2024-04-25T10:11:14.488793+0000","last_became_active":"2024-04-25T10:11:11.095774+0000","last_became_peered":"2024-04-25T10:11:11.095774+0000","last_unstale":"2024-04-25T10:11:14.488793+0000","last_undegraded":"2024-04-25T10:11:14.488793+0000","last_fullsized":"2024-04-25T10:11:14.488793+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:20.219784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.113941+0000","last_change":"2024-04-25T10:11:12.100193+0000","last_active":"2024-04-25T10:11:13.113941+0000","last_peered":"2024-04-25T10:11:13.113941+0000","last_clean":"2024-04-25T10:11:13.113941+0000","last_became_active":"2024-04-25T10:11:12.099799+0000","last_became_peered":"2024-04-25T10:11:12.099799+0000","last_unstale":"2024-04-25T10:11:13.113941+0000","last_undegraded":"2024-04-25T10:11:13.113941+0000","last_fullsized":"2024-04-25T10:11:13.113941+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:11.048899+0000","last_clean_scrub_stamp":"2024-04-25T10:11:11.048899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:28.332101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:14.488504+0000","last_change":"2024-04-25T10:11:11.085382+0000","last_active":"2024-04-25T10:11:14.488504+0000","last_peered":"2024-04-25T10:11:14.488504+0000","last_clean":"2024-04-25T10:11:14.488504+0000","last_became_active":"2024-04-25T10:11:11.085000+0000","last_became_peered":"2024-04-25T10:11:11.085000+0000","last_unstale":"2024-04-25T10:11:14.488504+0000","last_undegraded":"2024-04-25T10:11:14.488504+0000","last_fullsized":"2024-04-25T10:11:14.488504+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:10.039164+0000","last_clean_scrub_stamp":"2024-04-25T10:11:10.039164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:15.245704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":334,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-25T10:11:13.556018+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:11:13.556018+0000","last_peered":"2024-04-25T10:11:13.556018+0000","last_clean":"2024-04-25T10:11:13.556018+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:11:13.556018+0000","last_undegraded":"2024-04-25T10:11:13.556018+0000","last_fullsized":"2024-04-25T10:11:13.556018+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65000,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306840,"statfs":{"total":96636764160,"available":96570204160,"internally_reserved":0,"allocated":1929216,"data_stored":1208440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58872,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1347584,"data_stored":620128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574967,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54692,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317148,"statfs":{"total":96636764160,"available":96580759552,"internally_reserved":0,"allocated":1327104,"data_stored":617912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68588,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303252,"statfs":{"total":96636764160,"available":96566530048,"internally_reserved":0,"allocated":1335296,"data_stored":618004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72740,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299100,"statfs":{"total":96636764160,"available":96562278400,"internally_reserved":0,"allocated":1331200,"data_stored":618494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58840,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313000,"statfs":{"total":96636764160,"available":96576512000,"internally_reserved":0,"allocated":1318912,"data_stored":611254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58852,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312988,"statfs":{"total":96636764160,"available":96576499712,"internally_reserved":0,"allocated":1331200,"data_stored":617946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65012,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306828,"statfs":{"total":96636764160,"available":96570191872,"internally_reserved":0,"allocated":1937408,"data_stored":1209982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:19.852 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:11:19.853 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:11:20.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.014+0000 7f8ea170b640 1 Processor -- start 2024-04-25T10:11:20.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.014+0000 7f8ea170b640 1 -- start start 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.014+0000 7f8ea170b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8e9c0b5ef0 0x7f8e9c0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.014+0000 7f8ea170b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e9c0d1060 0x7f8e9c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.014+0000 7f8ea170b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9c0d0300 con 0x7f8e9c06b1d0 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.014+0000 7f8ea170b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e9c0cfaa0 con 0x7f8e9c0b5ef0 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.014+0000 7f8ea170b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c0cc4e0 con 0x7f8e9c0d1060 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9b7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8e9c06b1d0 0x7f8e9c0ccd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60722/0 (socket says 172.21.15.133:60722) 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9b7fe640 1 -- 172.21.15.133:0/726240716 learned_addr learned my addr 172.21.15.133:0/726240716 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9affd640 1 --2- 172.21.15.133:0/726240716 >> v2:172.21.15.162:3300/0 conn(0x7f8e9c0b5ef0 0x7f8e9c0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9a7fc640 1 --2- 172.21.15.133:0/726240716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e9c0d1060 0x7f8e9c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275389715 0 0) 0x7f8e9c0d0300 con 0x7f8e9c06b1d0 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e84003530 con 0x7f8e9c06b1d0 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1002504534 0 0) 0x7f8e84003530 con 0x7f8e9c06b1d0 2024-04-25T10:11:20.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e9c0d1060 msgr2=0x7f8e9c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 --2- 172.21.15.133:0/726240716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e9c0d1060 0x7f8e9c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 >> v2:172.21.15.162:3300/0 conn(0x7f8e9c0b5ef0 msgr2=0x7f8e9c0ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 --2- 172.21.15.133:0/726240716 >> v2:172.21.15.162:3300/0 conn(0x7f8e9c0b5ef0 0x7f8e9c0ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c0cfd40 con 0x7f8e9c06b1d0 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f8e94001a00 con 0x7f8e9c06b1d0 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 >> v1:172.21.15.133:6789/0 conn(0x7f8e9c06b1d0 legacy=0x7f8e9c0ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 --2- 172.21.15.133:0/726240716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e840039c0 0x7f8e84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 --2- 172.21.15.133:0/726240716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e84004810 0x7f8e84004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 --2- 172.21.15.133:0/726240716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e9c0d1060 0x7f8e840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c0cfaa0 con 0x7f8e84004810 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c0cc4e0 con 0x7f8e840039c0 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8400b9d0 con 0x7f8e9c0d1060 2024-04-25T10:11:20.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9a7fc640 1 --2- 172.21.15.133:0/726240716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e84004810 0x7f8e84004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9affd640 1 --2- 172.21.15.133:0/726240716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e840039c0 0x7f8e84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9a7fc640 1 -- 172.21.15.133:0/726240716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e9c0d1060 msgr2=0x7f8e840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9a7fc640 1 --2- 172.21.15.133:0/726240716 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e9c0d1060 0x7f8e840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9a7fc640 1 -- 172.21.15.133:0/726240716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e840039c0 msgr2=0x7f8e84003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9a7fc640 1 --2- 172.21.15.133:0/726240716 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e840039c0 0x7f8e84003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9a7fc640 1 -- 172.21.15.133:0/726240716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e84003530 con 0x7f8e84004810 2024-04-25T10:11:20.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e9a7fc640 1 --2- 172.21.15.133:0/726240716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e84004810 0x7f8e84004c90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009ac0 tx=0x7f8e9001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979ff865b05c668f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900b2070 con 0x7f8e84004810 2024-04-25T10:11:20.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e900a55a0 con 0x7f8e84004810 2024-04-25T10:11:20.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.018+0000 7f8e99ffb640 1 -- 172.21.15.133:0/726240716 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90009690 con 0x7f8e84004810 2024-04-25T10:11:20.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- 172.21.15.133:0/726240716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e84004810 msgr2=0x7f8e84004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 --2- 172.21.15.133:0/726240716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e84004810 0x7f8e84004c90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009ac0 tx=0x7f8e9001ca90 comp rx=0 tx=0).stop 2024-04-25T10:11:20.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9b7fe640 1 -- 172.21.15.133:0/726240716 reap_dead start 2024-04-25T10:11:20.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- 172.21.15.133:0/726240716 shutdown_connections 2024-04-25T10:11:20.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- 172.21.15.133:0/726240716 >> 172.21.15.133:0/726240716 conn(0x7f8e9c0d0570 msgr2=0x7f8e9c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- 172.21.15.133:0/726240716 shutdown_connections 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- 172.21.15.133:0/726240716 wait complete. 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 Processor -- start 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- start start 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e84004810 0x7f8e9c158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e9c1f4640 0x7f8e9c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e9c1f4aa0 0x7f8e9c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c0cafb0 con 0x7f8e9c1f4640 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c0cacd0 con 0x7f8e9c1f4aa0 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c0b33c0 con 0x7f8e84004810 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e9c1f4640 0x7f8e9c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e9c1f4640 0x7f8e9c0b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50372/0 (socket says 172.21.15.133:50372) 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9a7fc640 1 -- 172.21.15.133:0/1917000849 learned_addr learned my addr 172.21.15.133:0/1917000849 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9b7fe640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e9c1f4aa0 0x7f8e9c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9affd640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e84004810 0x7f8e9c158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9a7fc640 1 -- 172.21.15.133:0/1917000849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e84004810 msgr2=0x7f8e9c158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9a7fc640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e84004810 0x7f8e9c158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9a7fc640 1 -- 172.21.15.133:0/1917000849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e9c1f4aa0 msgr2=0x7f8e9c0b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9a7fc640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e9c1f4aa0 0x7f8e9c0b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9a7fc640 1 -- 172.21.15.133:0/1917000849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c0cede0 con 0x7f8e9c1f4640 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e9a7fc640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e9c1f4640 0x7f8e9c0b4680 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f8e9001c8c0 tx=0x7f8e84005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e837fe640 1 -- 172.21.15.133:0/1917000849 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900be070 con 0x7f8e9c1f4640 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e837fe640 1 -- 172.21.15.133:0/1917000849 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e900b7460 con 0x7f8e9c1f4640 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- 172.21.15.133:0/1917000849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9c0c48a0 con 0x7f8e9c1f4640 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8e837fe640 1 -- 172.21.15.133:0/1917000849 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900b2040 con 0x7f8e9c1f4640 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- 172.21.15.133:0/1917000849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9c0c4e60 con 0x7f8e9c1f4640 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.022+0000 7f8ea170b640 1 -- 172.21.15.133:0/1917000849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9c067c70 con 0x7f8e9c1f4640 2024-04-25T10:11:20.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.026+0000 7f8e837fe640 1 -- 172.21.15.133:0/1917000849 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e900c1070 con 0x7f8e9c1f4640 2024-04-25T10:11:20.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.026+0000 7f8e837fe640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e88066f60 0x7f8e88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.026+0000 7f8e837fe640 1 -- 172.21.15.133:0/1917000849 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e9011edb0 con 0x7f8e9c1f4640 2024-04-25T10:11:20.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.026+0000 7f8e9affd640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e88066f60 0x7f8e88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.026+0000 7f8e9affd640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e88066f60 0x7f8e88069420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c009ab0 tx=0x7f8e8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.030+0000 7f8e837fe640 1 -- 172.21.15.133:0/1917000849 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8e900167a0 con 0x7f8e9c1f4640 2024-04-25T10:11:20.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.234+0000 7f8ea170b640 1 -- 172.21.15.133:0/1917000849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8e9c15b020 con 0x7f8e9c1f4640 2024-04-25T10:11:20.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8e837fe640 1 -- 172.21.15.133:0/1917000849 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8e900ef5e0 con 0x7f8e9c1f4640 2024-04-25T10:11:20.239 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:11:20.239 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:11:20.239 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:11:20.239 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:11:20.240 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:20.240 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:11:20.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 -- 172.21.15.133:0/1917000849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e88066f60 msgr2=0x7f8e88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e88066f60 0x7f8e88069420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c009ab0 tx=0x7f8e8c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:20.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 -- 172.21.15.133:0/1917000849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e9c1f4640 msgr2=0x7f8e9c0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e9c1f4640 0x7f8e9c0b4680 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f8e9001c8c0 tx=0x7f8e84005840 comp rx=0 tx=0).stop 2024-04-25T10:11:20.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 -- 172.21.15.133:0/1917000849 shutdown_connections 2024-04-25T10:11:20.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e9c1f4aa0 0x7f8e9c0b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e88066f60 0x7f8e88069420 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e9c1f4640 0x7f8e9c0b4680 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 --2- 172.21.15.133:0/1917000849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e84004810 0x7f8e9c158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 -- 172.21.15.133:0/1917000849 >> 172.21.15.133:0/1917000849 conn(0x7f8e9c0d0570 msgr2=0x7f8e9c0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 -- 172.21.15.133:0/1917000849 shutdown_connections 2024-04-25T10:11:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.238+0000 7f8ea170b640 1 -- 172.21.15.133:0/1917000849 wait complete. 2024-04-25T10:11:20.257 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:20.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.370+0000 7f94bf282640 1 Processor -- start 2024-04-25T10:11:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.370+0000 7f94bf282640 1 -- start start 2024-04-25T10:11:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bf282640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b815c9a0 0x7f94b80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bf282640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f94b80ce330 0x7f94b80ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bf282640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b80c61d0 con 0x7f94b80b1f70 2024-04-25T10:11:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bf282640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b80cece0 con 0x7f94b80ce330 2024-04-25T10:11:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bf282640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b80c64b0 con 0x7f94b815c9a0 2024-04-25T10:11:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bd81f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f94b80ce330 0x7f94b80ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bd81f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f94b80ce330 0x7f94b80ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50382/0 (socket says 172.21.15.133:50382) 2024-04-25T10:11:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bd81f640 1 -- 172.21.15.133:0/3428740436 learned_addr learned my addr 172.21.15.133:0/3428740436 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bd01e640 1 --2- 172.21.15.133:0/3428740436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b815c9a0 0x7f94b80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bd81f640 1 -- 172.21.15.133:0/3428740436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b815c9a0 msgr2=0x7f94b80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bd81f640 1 --2- 172.21.15.133:0/3428740436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b815c9a0 0x7f94b80c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bd81f640 1 -- 172.21.15.133:0/3428740436 >> v1:172.21.15.162:6789/0 conn(0x7f94b80b1f70 legacy=0x7f94b806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94affff640 1 -- 172.21.15.133:0/3428740436 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080532058 0 0) 0x7f94b80c61d0 con 0x7f94b80b1f70 2024-04-25T10:11:20.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.374+0000 7f94bd81f640 1 -- 172.21.15.133:0/3428740436 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b806d4f0 con 0x7f94b80ce330 2024-04-25T10:11:20.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bd81f640 1 --2- 172.21.15.133:0/3428740436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b80ce330 0x7f94b80ce7a0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009870 tx=0x7f94b401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563be89195550d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94affff640 1 -- 172.21.15.133:0/3428740436 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b40ab610 con 0x7f94b80ce330 2024-04-25T10:11:20.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94affff640 1 -- 172.21.15.133:0/3428740436 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b40b0070 con 0x7f94b80ce330 2024-04-25T10:11:20.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94affff640 1 -- 172.21.15.133:0/3428740436 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b40ab9d0 con 0x7f94b80ce330 2024-04-25T10:11:20.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 -- 172.21.15.133:0/3428740436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b80ce330 msgr2=0x7f94b80ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 --2- 172.21.15.133:0/3428740436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b80ce330 0x7f94b80ce7a0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009870 tx=0x7f94b401c560 comp rx=0 tx=0).stop 2024-04-25T10:11:20.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 -- 172.21.15.133:0/3428740436 shutdown_connections 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 --2- 172.21.15.133:0/3428740436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b80ce330 0x7f94b80ce7a0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 --2- 172.21.15.133:0/3428740436 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b815c9a0 0x7f94b80c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 -- 172.21.15.133:0/3428740436 >> 172.21.15.133:0/3428740436 conn(0x7f94b8067000 msgr2=0x7f94b806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 -- 172.21.15.133:0/3428740436 shutdown_connections 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 -- 172.21.15.133:0/3428740436 wait complete. 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 Processor -- start 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 -- start start 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94b80b1f70 0x7f94b814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b80ce330 0x7f94b8155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b815c9a0 0x7f94b8149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b815dad0 con 0x7f94b815c9a0 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b815d950 con 0x7f94b80b1f70 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bf282640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b815dc50 con 0x7f94b80ce330 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bd81f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b815c9a0 0x7f94b8149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bd81f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b815c9a0 0x7f94b8149100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50390/0 (socket says 172.21.15.133:50390) 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.378+0000 7f94bd81f640 1 -- 172.21.15.133:0/3339895772 learned_addr learned my addr 172.21.15.133:0/3339895772 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bc81d640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b80ce330 0x7f94b8155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bd01e640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94b80b1f70 0x7f94b814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bd81f640 1 -- 172.21.15.133:0/3339895772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b80ce330 msgr2=0x7f94b8155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bd81f640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b80ce330 0x7f94b8155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bd81f640 1 -- 172.21.15.133:0/3339895772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94b80b1f70 msgr2=0x7f94b814ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bd81f640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94b80b1f70 0x7f94b814ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bd81f640 1 -- 172.21.15.133:0/3339895772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b814d4a0 con 0x7f94b815c9a0 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bd81f640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b815c9a0 0x7f94b8149100 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f94b401a190 tx=0x7f94b401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94adffb640 1 -- 172.21.15.133:0/3339895772 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b40ba070 con 0x7f94b815c9a0 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94adffb640 1 -- 172.21.15.133:0/3339895772 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b4004230 con 0x7f94b815c9a0 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94adffb640 1 -- 172.21.15.133:0/3339895772 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b40ab8e0 con 0x7f94b815c9a0 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bf282640 1 -- 172.21.15.133:0/3339895772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b814d780 con 0x7f94b815c9a0 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bf282640 1 -- 172.21.15.133:0/3339895772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b814da60 con 0x7f94b815c9a0 2024-04-25T10:11:20.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bf282640 1 -- 172.21.15.133:0/3339895772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b806bc00 con 0x7f94b815c9a0 2024-04-25T10:11:20.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94adffb640 1 -- 172.21.15.133:0/3339895772 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94b40aba80 con 0x7f94b815c9a0 2024-04-25T10:11:20.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94adffb640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f949c066f90 0x7f949c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bd01e640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f949c066f90 0x7f949c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94adffb640 1 -- 172.21.15.133:0/3339895772 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f94b411f220 con 0x7f94b815c9a0 2024-04-25T10:11:20.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.382+0000 7f94bd01e640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f949c066f90 0x7f949c069450 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f94b8151360 tx=0x7f94b000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.386+0000 7f94adffb640 1 -- 172.21.15.133:0/3339895772 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94b40a8050 con 0x7f94b815c9a0 2024-04-25T10:11:20.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.582+0000 7f94bf282640 1 -- 172.21.15.133:0/3339895772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94b80cbed0 con 0x7f94b815c9a0 2024-04-25T10:11:20.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.582+0000 7f94adffb640 1 -- 172.21.15.133:0/3339895772 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f94b4016150 con 0x7f94b815c9a0 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:20.587 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:20.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 -- 172.21.15.133:0/3339895772 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f949c066f90 msgr2=0x7f949c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f949c066f90 0x7f949c069450 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f94b8151360 tx=0x7f94b000b040 comp rx=0 tx=0).stop 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 -- 172.21.15.133:0/3339895772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b815c9a0 msgr2=0x7f94b8149100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b815c9a0 0x7f94b8149100 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f94b401a190 tx=0x7f94b401ada0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 -- 172.21.15.133:0/3339895772 shutdown_connections 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f949c066f90 0x7f949c069450 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b815c9a0 0x7f94b8149100 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94b80ce330 0x7f94b8155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 --2- 172.21.15.133:0/3339895772 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94b80b1f70 0x7f94b814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 -- 172.21.15.133:0/3339895772 >> 172.21.15.133:0/3339895772 conn(0x7f94b8067000 msgr2=0x7f94b815a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 -- 172.21.15.133:0/3339895772 shutdown_connections 2024-04-25T10:11:20.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:20.586+0000 7f94bf282640 1 -- 172.21.15.133:0/3339895772 wait complete. 2024-04-25T10:11:21.008 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:11.432440+0000', 'modified': '2024-04-25T10:11:13.428977+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8313}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8313': {'gid': 8313, 'name': 'b', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6834/488739264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6832', 'nonce': 488739264}, {'type': 'v1', 'addr': '172.21.15.162:6834', 'nonce': 488739264}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:21.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:11:21.083 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:11:21.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.214+0000 7f13e4028640 1 Processor -- start 2024-04-25T10:11:21.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.214+0000 7f13e4028640 1 -- start start 2024-04-25T10:11:21.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.214+0000 7f13e4028640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f13dc0cd820 0x7f13dc0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.214+0000 7f13e4028640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13dc0ce2e0 con 0x7f13dc145170 2024-04-25T10:11:21.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.214+0000 7f13e4028640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13dc06d4b0 con 0x7f13dc15ced0 2024-04-25T10:11:21.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.214+0000 7f13e4028640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13dc06db00 con 0x7f13dc0cd820 2024-04-25T10:11:21.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.214+0000 7f13e1dc4640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f13dc145170 0x7f13dc06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60734/0 (socket says 172.21.15.133:60734) 2024-04-25T10:11:21.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.214+0000 7f13e1dc4640 1 -- 172.21.15.133:0/2837156971 learned_addr learned my addr 172.21.15.133:0/2837156971 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:21.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.214+0000 7f13e15c3640 1 --2- 172.21.15.133:0/2837156971 >> v2:172.21.15.162:3300/0 conn(0x7f13dc0cd820 0x7f13dc0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.214+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 478956443 0 0) 0x7f13dc0ce2e0 con 0x7f13dc145170 2024-04-25T10:11:21.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.214+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13c8003530 con 0x7f13dc145170 2024-04-25T10:11:21.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2757104670 0 0) 0x7f13c8003530 con 0x7f13dc145170 2024-04-25T10:11:21.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 >> v2:172.21.15.162:3300/0 conn(0x7f13dc0cd820 msgr2=0x7f13dc0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 --2- 172.21.15.133:0/2837156971 >> v2:172.21.15.162:3300/0 conn(0x7f13dc0cd820 0x7f13dc0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 >> v1:172.21.15.162:6789/0 conn(0x7f13dc15ced0 legacy=0x7f13dc0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc06d790 con 0x7f13dc145170 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f13d8001680 con 0x7f13dc145170 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 >> v1:172.21.15.133:6789/0 conn(0x7f13dc145170 legacy=0x7f13dc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 --2- 172.21.15.133:0/2837156971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13c80039c0 0x7f13c8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 --2- 172.21.15.133:0/2837156971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 0x7f13c8004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 --2- 172.21.15.133:0/2837156971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13c8005090 0x7f13c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc06db00 con 0x7f13c80043c0 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c800bff0 con 0x7f13c8005090 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c800c1f0 con 0x7f13c80039c0 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e15c3640 1 --2- 172.21.15.133:0/2837156971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 0x7f13c8004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e25c5640 1 --2- 172.21.15.133:0/2837156971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13c8005090 0x7f13c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/2837156971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13c80039c0 0x7f13c8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e15c3640 1 -- 172.21.15.133:0/2837156971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13c80039c0 msgr2=0x7f13c8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e15c3640 1 --2- 172.21.15.133:0/2837156971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13c80039c0 0x7f13c8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e15c3640 1 -- 172.21.15.133:0/2837156971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13c8005090 msgr2=0x7f13c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e15c3640 1 --2- 172.21.15.133:0/2837156971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13c8005090 0x7f13c800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e15c3640 1 -- 172.21.15.133:0/2837156971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c8003530 con 0x7f13c80043c0 2024-04-25T10:11:21.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e25c5640 1 -- 172.21.15.133:0/2837156971 reap_dead start 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e15c3640 1 --2- 172.21.15.133:0/2837156971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 0x7f13c8004890 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009ae0 tx=0x7f13cc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c216397b79ef70a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0b2070 con 0x7f13c80043c0 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc0a5660 con 0x7f13c80043c0 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e0dc2640 1 -- 172.21.15.133:0/2837156971 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0096b0 con 0x7f13c80043c0 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e4028640 1 -- 172.21.15.133:0/2837156971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 msgr2=0x7f13c8004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e4028640 1 --2- 172.21.15.133:0/2837156971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 0x7f13c8004890 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009ae0 tx=0x7f13cc01ca70 comp rx=0 tx=0).stop 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e4028640 1 -- 172.21.15.133:0/2837156971 shutdown_connections 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e4028640 1 --2- 172.21.15.133:0/2837156971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 0x7f13c8004890 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e4028640 1 -- 172.21.15.133:0/2837156971 >> 172.21.15.133:0/2837156971 conn(0x7f13dc066f70 msgr2=0x7f13dc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e4028640 1 -- 172.21.15.133:0/2837156971 shutdown_connections 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.218+0000 7f13e4028640 1 -- 172.21.15.133:0/2837156971 wait complete. 2024-04-25T10:11:21.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e4028640 1 Processor -- start 2024-04-25T10:11:21.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e4028640 1 -- start start 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e4028640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 0x7f13dc136b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e4028640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13dc137060 0x7f13dc13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e4028640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13dc15fd90 0x7f13dc13fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e4028640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc0cdca0 con 0x7f13c80043c0 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e4028640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc145550 con 0x7f13dc137060 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e4028640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc0cde20 con 0x7f13dc15fd90 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e1dc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 0x7f13dc136b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e1dc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 0x7f13dc136b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50414/0 (socket says 172.21.15.133:50414) 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e1dc4640 1 -- 172.21.15.133:0/3509603162 learned_addr learned my addr 172.21.15.133:0/3509603162 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e25c5640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13dc15fd90 0x7f13dc13fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e15c3640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13dc137060 0x7f13dc13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e1dc4640 1 -- 172.21.15.133:0/3509603162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13dc15fd90 msgr2=0x7f13dc13fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13dc15fd90 0x7f13dc13fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e1dc4640 1 -- 172.21.15.133:0/3509603162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13dc137060 msgr2=0x7f13dc13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13dc137060 0x7f13dc13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e1dc4640 1 -- 172.21.15.133:0/3509603162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc15f9b0 con 0x7f13c80043c0 2024-04-25T10:11:21.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e15c3640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13dc137060 0x7f13dc13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 0x7f13dc136b20 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f13d8005fd0 tx=0x7f13d800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13d27fc640 1 -- 172.21.15.133:0/3509603162 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800c8d0 con 0x7f13c80043c0 2024-04-25T10:11:21.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13d27fc640 1 -- 172.21.15.133:0/3509603162 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d8018070 con 0x7f13c80043c0 2024-04-25T10:11:21.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e4028640 1 -- 172.21.15.133:0/3509603162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13dc15e5c0 con 0x7f13c80043c0 2024-04-25T10:11:21.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13d27fc640 1 -- 172.21.15.133:0/3509603162 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800cc90 con 0x7f13c80043c0 2024-04-25T10:11:21.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.222+0000 7f13e4028640 1 -- 172.21.15.133:0/3509603162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13dc15eb80 con 0x7f13c80043c0 2024-04-25T10:11:21.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13d27fc640 1 -- 172.21.15.133:0/3509603162 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13d8013420 con 0x7f13c80043c0 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13d27fc640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13c0066f90 0x7f13c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13d27fc640 1 -- 172.21.15.133:0/3509603162 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13d8084dc0 con 0x7f13c80043c0 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e15c3640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13c0066f90 0x7f13c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e15c3640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13c0066f90 0x7f13c0069450 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0af000 tx=0x7f13cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e7195000 1 Processor -- start 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e7195000 1 -- start start 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e7195000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca64d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e7195000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca65240 0x55fdcca1a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e7195000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca1af30 0x55fdcca1b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e7195000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcc9025f0 con 0x55fdcca64920 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e7195000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcc869da0 con 0x55fdcca1af30 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e7195000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdcc7f2150 con 0x55fdcca65240 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e1dc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca64d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e1dc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca64d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50426/0 (socket says 172.21.15.133:50426) 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e1dc4640 1 -- 172.21.15.133:0/2319067470 learned_addr learned my addr 172.21.15.133:0/2319067470 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e1dc4640 1 -- 172.21.15.133:0/2319067470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca65240 msgr2=0x55fdcca1a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/2319067470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca65240 0x55fdcca1a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e1dc4640 1 -- 172.21.15.133:0/2319067470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca1af30 msgr2=0x55fdcca1b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/2319067470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca1af30 0x55fdcca1b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e1dc4640 1 -- 172.21.15.133:0/2319067470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdcc80bca0 con 0x55fdcca64920 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/2319067470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca64d00 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f13d800a040 tx=0x7f13d800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e358cfdcaa0da693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13bf7fe640 1 -- 172.21.15.133:0/2319067470 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d806d020 con 0x55fdcca64920 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13bf7fe640 1 -- 172.21.15.133:0/2319067470 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d8059720 con 0x55fdcca64920 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.226+0000 7f13bf7fe640 1 -- 172.21.15.133:0/2319067470 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d805dce0 con 0x55fdcca64920 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- 172.21.15.133:0/2319067470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 msgr2=0x55fdcca64d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 --2- 172.21.15.133:0/2319067470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca64d00 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f13d800a040 tx=0x7f13d800cef0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- 172.21.15.133:0/2319067470 shutdown_connections 2024-04-25T10:11:21.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 --2- 172.21.15.133:0/2319067470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca1af30 0x55fdcca1b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 --2- 172.21.15.133:0/2319067470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca65240 0x55fdcca1a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 --2- 172.21.15.133:0/2319067470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca64d00 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- 172.21.15.133:0/2319067470 >> 172.21.15.133:0/2319067470 conn(0x55fdcca212a0 msgr2=0x55fdcca192d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- 172.21.15.133:0/2319067470 shutdown_connections 2024-04-25T10:11:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- 172.21.15.133:0/2319067470 wait complete. 2024-04-25T10:11:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- start start 2024-04-25T10:11:21.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 Processor -- start 2024-04-25T10:11:21.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdcca6bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca64920 0x55fdcca6c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca65240 0x55fdcca1eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcc824990 con 0x55fdcca1af30 2024-04-25T10:11:21.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcc865e40 con 0x55fdcca65240 2024-04-25T10:11:21.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdcc7f5900 con 0x55fdcca64920 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 10 client.0 _ll_register_callbacks cb 0x55fdcc91d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e1dc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdcca6bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e1dc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdcca6bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50428/0 (socket says 172.21.15.133:50428) 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e1dc4640 1 -- 172.21.15.133:0/1276796257 learned_addr learned my addr 172.21.15.133:0/1276796257 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e15c3640 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca64920 0x55fdcca6c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e25c5640 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca65240 0x55fdcca1eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e1dc4640 1 -- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca64920 msgr2=0x55fdcca6c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca64920 0x55fdcca6c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e1dc4640 1 -- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca65240 msgr2=0x55fdcca1eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca65240 0x55fdcca1eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e1dc4640 1 -- 172.21.15.133:0/1276796257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdcc822630 con 0x55fdcca1af30 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdcca6bcf0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f13d8055510 tx=0x7f13d801e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43df1a70c93b978 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:21.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13bcff9640 1 -- 172.21.15.133:0/1276796257 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8053990 con 0x55fdcca1af30 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13bcff9640 1 -- 172.21.15.133:0/1276796257 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13d805d850 con 0x55fdcca1af30 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- 172.21.15.133:0/1276796257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13d8059720 con 0x55fdcca1af30 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13bcff9640 1 -- 172.21.15.133:0/1276796257 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13d8073050 con 0x55fdcca1af30 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 10 client.8481 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- 172.21.15.133:0/1276796257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55fdcc7f23a0 con 0x55fdcca1af30 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13bcff9640 1 -- 172.21.15.133:0/1276796257 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d8059720 con 0x55fdcca1af30 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13bcff9640 1 -- 172.21.15.133:0/1276796257 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f13d807cb50 con 0x55fdcca1af30 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 10 client.8481 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 10 client.8481 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 20 client.8481 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fdcca20160 0x55fdcca75d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 4 client.8481 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e7195000 1 -- 172.21.15.133:0/1276796257 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdcc805a50 con 0x55fdcca20160 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e15c3640 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fdcca20160 0x55fdcca75d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13bcff9640 10 client.8481 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.230+0000 7f13e15c3640 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fdcca20160 0x55fdcca75d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0b72ff51281a266 server_cookie=802d413135733a2f in_seq=0 out_seq=0 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13bcff9640 1 -- 172.21.15.133:0/1276796257 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fdcc805a50 con 0x55fdcca20160 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13bcff9640 10 client.8481 handle_command_reply: tid=0 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 1 client.8481 shutdown 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 1 -- 172.21.15.133:0/1276796257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 msgr2=0x55fdcca6bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdcca6bcf0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f13d8055510 tx=0x7f13d801e070 comp rx=0 tx=0).stop 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 1 -- 172.21.15.133:0/1276796257 shutdown_connections 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fdcca20160 0x55fdcca75d50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca65240 0x55fdcca1eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca64920 0x55fdcca6c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13bcff9640 0 client.8481 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 1 --2- 172.21.15.133:0/1276796257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdcca6bcf0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 1 -- 172.21.15.133:0/1276796257 >> 172.21.15.133:0/1276796257 conn(0x55fdcca212a0 msgr2=0x55fdcca199e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 1 -- 172.21.15.133:0/1276796257 shutdown_connections 2024-04-25T10:11:21.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 1 -- 172.21.15.133:0/1276796257 wait complete. 2024-04-25T10:11:21.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.234+0000 7f13e7195000 20 client.8481 trim_cache size 0 max 16384 2024-04-25T10:11:21.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 Processor -- start 2024-04-25T10:11:21.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 -- start start 2024-04-25T10:11:21.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca1af30 0x55fdcca1e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca1e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca65240 0x55fdccaa4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc1fb730 con 0x55fdcca64920 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc1fab30 con 0x55fdcca65240 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc1fb130 con 0x55fdcca1af30 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e15c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca1e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e15c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca1e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50432/0 (socket says 172.21.15.133:50432) 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e15c3640 1 -- 172.21.15.133:0/1319125155 learned_addr learned my addr 172.21.15.133:0/1319125155 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e15c3640 1 -- 172.21.15.133:0/1319125155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca1af30 msgr2=0x55fdcca1e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e15c3640 1 --2- 172.21.15.133:0/1319125155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca1af30 0x55fdcca1e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e15c3640 1 -- 172.21.15.133:0/1319125155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca65240 msgr2=0x55fdccaa4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e15c3640 1 --2- 172.21.15.133:0/1319125155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca65240 0x55fdccaa4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e15c3640 1 -- 172.21.15.133:0/1319125155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdcca65070 con 0x55fdcca64920 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e15c3640 1 --2- 172.21.15.133:0/1319125155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca1e800 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0b0040 tx=0x7f13cc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac3f04c409c61c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13beffd640 1 -- 172.21.15.133:0/1319125155 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0b2040 con 0x55fdcca64920 2024-04-25T10:11:21.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13beffd640 1 -- 172.21.15.133:0/1319125155 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc0a5e10 con 0x55fdcca64920 2024-04-25T10:11:21.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13beffd640 1 -- 172.21.15.133:0/1319125155 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0c45c0 con 0x55fdcca64920 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 -- 172.21.15.133:0/1319125155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 msgr2=0x55fdcca1e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 --2- 172.21.15.133:0/1319125155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca1e800 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0b0040 tx=0x7f13cc0046d0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 -- 172.21.15.133:0/1319125155 shutdown_connections 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 --2- 172.21.15.133:0/1319125155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca65240 0x55fdccaa4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 --2- 172.21.15.133:0/1319125155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca64920 0x55fdcca1e800 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 --2- 172.21.15.133:0/1319125155 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca1af30 0x55fdcca1e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 -- 172.21.15.133:0/1319125155 >> 172.21.15.133:0/1319125155 conn(0x55fdcca212a0 msgr2=0x55fdcca1dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 -- 172.21.15.133:0/1319125155 shutdown_connections 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.250+0000 7f13e7195000 1 -- 172.21.15.133:0/1319125155 wait complete. 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- start start 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 Processor -- start 2024-04-25T10:11:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdccac5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca64920 0x55fdccac5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca65240 0x55fdcca7f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcc86e3d0 con 0x55fdcca1af30 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcc925d50 con 0x55fdcca64920 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdcc804980 con 0x55fdcca65240 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 10 client.0 _ll_register_callbacks cb 0x7f13d8051870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e1dc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdccac5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e1dc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdccac5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50442/0 (socket says 172.21.15.133:50442) 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e1dc4640 1 -- 172.21.15.133:0/2632205530 learned_addr learned my addr 172.21.15.133:0/2632205530 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e25c5640 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca65240 0x55fdcca7f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e15c3640 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca64920 0x55fdccac5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e1dc4640 1 -- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca65240 msgr2=0x55fdcca7f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca65240 0x55fdcca7f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e1dc4640 1 -- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca64920 msgr2=0x55fdccac5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca64920 0x55fdccac5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e1dc4640 1 -- 172.21.15.133:0/2632205530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdcc8eb600 con 0x55fdcca1af30 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e15c3640 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca64920 0x55fdccac5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e25c5640 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca65240 0x55fdcca7f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e1dc4640 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdccac5240 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f13d805d7b0 tx=0x7f13d80830c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf9ede3b0fde103 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13bcff9640 1 -- 172.21.15.133:0/2632205530 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8054f30 con 0x55fdcca1af30 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13bcff9640 1 -- 172.21.15.133:0/2632205530 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13d805da60 con 0x55fdcca1af30 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- 172.21.15.133:0/2632205530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13cc0a5e10 con 0x55fdcca1af30 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13bcff9640 1 -- 172.21.15.133:0/2632205530 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13d8053990 con 0x55fdcca1af30 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 10 client.8487 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- 172.21.15.133:0/2632205530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55fdcc7fe9b0 con 0x55fdcca1af30 2024-04-25T10:11:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13bcff9640 1 -- 172.21.15.133:0/2632205530 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc0a5e10 con 0x55fdcca1af30 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13bcff9640 1 -- 172.21.15.133:0/2632205530 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f13d8054a20 con 0x55fdcca1af30 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 10 client.8487 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 10 client.8487 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 20 client.8487 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fdcca7fd10 0x55fdcca800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 4 client.8487 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- 172.21.15.133:0/2632205530 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fdcc8772e0 con 0x55fdcca7fd10 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e15c3640 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fdcca7fd10 0x55fdcca800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13bcff9640 10 client.8487 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e15c3640 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fdcca7fd10 0x55fdcca800f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ac7e518480588c5 server_cookie=3b8cea30529343e7 in_seq=0 out_seq=0 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13bcff9640 1 -- 172.21.15.133:0/2632205530 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55fdcc8772e0 con 0x55fdcca7fd10 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13bcff9640 10 client.8487 handle_command_reply: tid=0 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 client.8487 shutdown 2024-04-25T10:11:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- 172.21.15.133:0/2632205530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 msgr2=0x55fdccac5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdccac5240 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f13d805d7b0 tx=0x7f13d80830c0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- 172.21.15.133:0/2632205530 shutdown_connections 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fdcca7fd10 0x55fdcca800f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fdcca65240 0x55fdcca7f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fdcca64920 0x55fdccac5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 --2- 172.21.15.133:0/2632205530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdcca1af30 0x55fdccac5240 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- 172.21.15.133:0/2632205530 >> 172.21.15.133:0/2632205530 conn(0x55fdcca212a0 msgr2=0x55fdcca1e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13bcff9640 0 client.8487 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- 172.21.15.133:0/2632205530 shutdown_connections 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.254+0000 7f13e7195000 1 -- 172.21.15.133:0/2632205530 wait complete. 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e7195000 20 client.8487 trim_cache size 0 max 16384 2024-04-25T10:11:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 8313, 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 9.1938347890000003, 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 65, 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 107, 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 107, 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 14.195258829 2024-04-25T10:11:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:11:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 -- 172.21.15.133:0/3509603162 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13c0066f90 msgr2=0x7f13c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13c0066f90 0x7f13c0069450 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0af000 tx=0x7f13cc008040 comp rx=0 tx=0).stop 2024-04-25T10:11:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 -- 172.21.15.133:0/3509603162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 msgr2=0x7f13dc136b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 0x7f13dc136b20 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f13d8005fd0 tx=0x7f13d800ec80 comp rx=0 tx=0).stop 2024-04-25T10:11:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 -- 172.21.15.133:0/3509603162 shutdown_connections 2024-04-25T10:11:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13dc15fd90 0x7f13dc13fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13dc137060 0x7f13dc13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13c0066f90 0x7f13c0069450 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 --2- 172.21.15.133:0/3509603162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c80043c0 0x7f13dc136b20 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 -- 172.21.15.133:0/3509603162 >> 172.21.15.133:0/3509603162 conn(0x7f13dc066f70 msgr2=0x7f13dc06de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 -- 172.21.15.133:0/3509603162 shutdown_connections 2024-04-25T10:11:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.258+0000 7f13e4028640 1 -- 172.21.15.133:0/3509603162 wait complete. 2024-04-25T10:11:21.275 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:21.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.382+0000 7f0e46996640 1 Processor -- start 2024-04-25T10:11:21.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.382+0000 7f0e46996640 1 -- start start 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e46996640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0e4015b8c0 0x7f0e4009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e46996640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e400999d0 con 0x7f0e4015f4a0 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e46996640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e40099cb0 con 0x7f0e400cd360 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e46996640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e400d0530 con 0x7f0e4015b8c0 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0e4015f4a0 0x7f0e400ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:60738/0 (socket says 172.21.15.133:60738) 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3f7fe640 1 -- 172.21.15.133:0/1816984965 learned_addr learned my addr 172.21.15.133:0/1816984965 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282697119 0 0) 0x7f0e400999d0 con 0x7f0e4015f4a0 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3ffff640 1 --2- 172.21.15.133:0/1816984965 >> v2:172.21.15.162:3300/0 conn(0x7f0e4015b8c0 0x7f0e4009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e2c003530 con 0x7f0e4015f4a0 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1157174470 0 0) 0x7f0e2c003530 con 0x7f0e4015f4a0 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 >> v2:172.21.15.162:3300/0 conn(0x7f0e4015b8c0 msgr2=0x7f0e4009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 --2- 172.21.15.133:0/1816984965 >> v2:172.21.15.162:3300/0 conn(0x7f0e4015b8c0 0x7f0e4009aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 >> v1:172.21.15.162:6789/0 conn(0x7f0e400cd360 legacy=0x7f0e400b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e40099f90 con 0x7f0e4015f4a0 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0e28001680 con 0x7f0e4015f4a0 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 >> v1:172.21.15.133:6789/0 conn(0x7f0e4015f4a0 legacy=0x7f0e400ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 --2- 172.21.15.133:0/1816984965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 0x7f0e2c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 --2- 172.21.15.133:0/1816984965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e2c004810 0x7f0e2c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 --2- 172.21.15.133:0/1816984965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e2c0051d0 0x7f0e2c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e400d0530 con 0x7f0e2c003a10 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c00c0d0 con 0x7f0e2c0051d0 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c00c2d0 con 0x7f0e2c004810 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3ffff640 1 --2- 172.21.15.133:0/1816984965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 0x7f0e2c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e44f33640 1 --2- 172.21.15.133:0/1816984965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e2c0051d0 0x7f0e2c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3ffff640 1 -- 172.21.15.133:0/1816984965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e2c004810 msgr2=0x7f0e2c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3ffff640 1 --2- 172.21.15.133:0/1816984965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e2c004810 0x7f0e2c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3ffff640 1 -- 172.21.15.133:0/1816984965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e2c0051d0 msgr2=0x7f0e2c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3ffff640 1 --2- 172.21.15.133:0/1816984965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e2c0051d0 0x7f0e2c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e3ffff640 1 -- 172.21.15.133:0/1816984965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c003530 con 0x7f0e2c003a10 2024-04-25T10:11:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e44f33640 1 --2- 172.21.15.133:0/1816984965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e2c0051d0 0x7f0e2c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.386+0000 7f0e44f33640 1 -- 172.21.15.133:0/1816984965 reap_dead start 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3ffff640 1 --2- 172.21.15.133:0/1816984965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 0x7f0e2c003e30 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009fc0 tx=0x7f0e3801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1b6b9605fd8bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380b2070 con 0x7f0e2c003a10 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e380a5b50 con 0x7f0e2c003a10 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3effd640 1 -- 172.21.15.133:0/1816984965 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38009c10 con 0x7f0e2c003a10 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 -- 172.21.15.133:0/1816984965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 msgr2=0x7f0e2c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 --2- 172.21.15.133:0/1816984965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 0x7f0e2c003e30 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009fc0 tx=0x7f0e3801cc50 comp rx=0 tx=0).stop 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 -- 172.21.15.133:0/1816984965 shutdown_connections 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 --2- 172.21.15.133:0/1816984965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 0x7f0e2c003e30 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 -- 172.21.15.133:0/1816984965 >> 172.21.15.133:0/1816984965 conn(0x7f0e400b3120 msgr2=0x7f0e400b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 -- 172.21.15.133:0/1816984965 shutdown_connections 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 -- 172.21.15.133:0/1816984965 wait complete. 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 Processor -- start 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 -- start start 2024-04-25T10:11:21.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 0x7f0e4015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e400c6f40 0x7f0e400c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e401f59a0 0x7f0e401f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e400cd1e0 con 0x7f0e2c003a10 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e400cd060 con 0x7f0e400c6f40 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e400b17a0 con 0x7f0e401f59a0 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 0x7f0e4015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 0x7f0e4015c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50470/0 (socket says 172.21.15.133:50470) 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3ffff640 1 -- 172.21.15.133:0/2891163582 learned_addr learned my addr 172.21.15.133:0/2891163582 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3f7fe640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e400c6f40 0x7f0e400c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e44f33640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e401f59a0 0x7f0e401f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3ffff640 1 -- 172.21.15.133:0/2891163582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e401f59a0 msgr2=0x7f0e401f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3ffff640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e401f59a0 0x7f0e401f5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3ffff640 1 -- 172.21.15.133:0/2891163582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e400c6f40 msgr2=0x7f0e400c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3ffff640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e400c6f40 0x7f0e400c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3ffff640 1 -- 172.21.15.133:0/2891163582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4015dba0 con 0x7f0e2c003a10 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3f7fe640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e400c6f40 0x7f0e400c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3ffff640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 0x7f0e4015c870 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f0e3801c6c0 tx=0x7f0e38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3cff9640 1 -- 172.21.15.133:0/2891163582 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380ba070 con 0x7f0e2c003a10 2024-04-25T10:11:21.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3cff9640 1 -- 172.21.15.133:0/2891163582 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e380ada40 con 0x7f0e2c003a10 2024-04-25T10:11:21.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 -- 172.21.15.133:0/2891163582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4015de80 con 0x7f0e2c003a10 2024-04-25T10:11:21.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e3cff9640 1 -- 172.21.15.133:0/2891163582 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380b2040 con 0x7f0e2c003a10 2024-04-25T10:11:21.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.390+0000 7f0e46996640 1 -- 172.21.15.133:0/2891163582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4015e440 con 0x7f0e2c003a10 2024-04-25T10:11:21.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.394+0000 7f0e46996640 1 -- 172.21.15.133:0/2891163582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e40067c70 con 0x7f0e2c003a10 2024-04-25T10:11:21.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.394+0000 7f0e3cff9640 1 -- 172.21.15.133:0/2891163582 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e380b6410 con 0x7f0e2c003a10 2024-04-25T10:11:21.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.394+0000 7f0e3cff9640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e20066f90 0x7f0e20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.394+0000 7f0e3cff9640 1 -- 172.21.15.133:0/2891163582 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e3811f7f0 con 0x7f0e2c003a10 2024-04-25T10:11:21.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.394+0000 7f0e3f7fe640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e20066f90 0x7f0e20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.394+0000 7f0e3f7fe640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e20066f90 0x7f0e20069450 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0e280017f0 tx=0x7f0e28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.398+0000 7f0e3cff9640 1 -- 172.21.15.133:0/2891163582 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e38016990 con 0x7f0e2c003a10 2024-04-25T10:11:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 -- 172.21.15.133:0/2891163582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0e400c8190 con 0x7f0e2c003a10 2024-04-25T10:11:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e3cff9640 1 -- 172.21.15.133:0/2891163582 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0e380f0010 con 0x7f0e2c003a10 2024-04-25T10:11:21.596 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 -- 172.21.15.133:0/2891163582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e20066f90 msgr2=0x7f0e20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e20066f90 0x7f0e20069450 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0e280017f0 tx=0x7f0e28009040 comp rx=0 tx=0).stop 2024-04-25T10:11:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 -- 172.21.15.133:0/2891163582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 msgr2=0x7f0e4015c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 0x7f0e4015c870 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f0e3801c6c0 tx=0x7f0e38009750 comp rx=0 tx=0).stop 2024-04-25T10:11:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 -- 172.21.15.133:0/2891163582 shutdown_connections 2024-04-25T10:11:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e401f59a0 0x7f0e401f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e400c6f40 0x7f0e400c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e20066f90 0x7f0e20069450 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 --2- 172.21.15.133:0/2891163582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c003a10 0x7f0e4015c870 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 -- 172.21.15.133:0/2891163582 >> 172.21.15.133:0/2891163582 conn(0x7f0e400b3120 msgr2=0x7f0e4015a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 -- 172.21.15.133:0/2891163582 shutdown_connections 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:21.594+0000 7f0e46996640 1 -- 172.21.15.133:0/2891163582 wait complete. 2024-04-25T10:11:21.615 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:11:21.615 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:21.615 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:11:21.615 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:21.615 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:21.615 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:11:21.615 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:11:21.615 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:21.615 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:21.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:21.615 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:11:21.621 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:21.621 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:21.621 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:21.621 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:21.621 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:21.621 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41950sec preferred_lft 41950sec 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:11:21.622 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:21.623 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:21.624 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:21.624 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:11:21.624 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:21.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:21.624 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:21.671 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:11:21.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:21.673 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:11:21.723 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:11:21.724 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:11:21.724 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:11:21.724 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:11:21.724 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:11:21.724 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:21.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:21.725 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:11:21.725 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:21.725 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:11:21.725 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:11:21.725 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:11:21.725 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:21.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:21.800 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:11:21.852 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:21.852 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:21.852 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:21.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:21.852 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:11:21.908 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:11:21.908 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:21.908 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:21.955 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:21.956 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:21.956 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:22.003 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:11:22.065 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:22.112 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:11:22.112 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:11:22.112 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:11:22.113 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:22.115 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:22.115 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:22.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:22.116 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:11:22.166 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:11:22.166 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:11:22.216 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:11:22.231 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:22.279 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:11:22.280 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:22.281 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:11:22.282 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:22.282 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:22.282 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:22.282 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:22.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:22.283 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:11:22.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:11:22.294+0000 7fbcdf310580 -1 init, newargv = 0x556f490e70c0 newargc=15 2024-04-25T10:11:22.297 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[74919]: starting ceph client 2024-04-25T10:11:22.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[74919]: starting fuse 2024-04-25T10:11:22.636 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:22.638 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:23.403 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:11:23.420 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:23.468 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:11:23.469 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:23.470 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:23.471 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:23.471 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:11:23.471 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:23.471 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:23.471 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:23.471 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:23.471 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:23.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:23.472 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:11:23.519 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:11:23.520 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:11:23.520 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:11:23.520 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:11:23.520 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:11:23.520 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:11:23.520 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:11:23.520 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:11:23.520 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:11:23.520 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:11:23.521 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:11:23.522 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:11:23.522 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:11:23.644 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.74919.asok 2024-04-25T10:11:23.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:23.657 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.74919.asok 2024-04-25T10:11:23.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:23.658 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.74919.asok status 2024-04-25T10:11:23.812 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:11:23.812 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:11:23.812 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:11:23.812 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:11:23.812 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "74919", 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 8499, 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 8499 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:23.813 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 980127760 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 980127760 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.8499 192.168.0.1:0/980127760", 2024-04-25T10:11:23.814 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/980127760", 2024-04-25T10:11:23.815 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:11:23.815 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 65, 2024-04-25T10:11:23.815 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 107, 2024-04-25T10:11:23.815 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:11:23.815 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:11:23.815 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:11:23.815 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:11:23.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:23.826 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:11:23.827 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:23.833 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:11:23.833 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:23.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:23.833 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:23.888 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:23.888 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T10:11:23.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:23.888 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:23.888 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:23.889 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:11:23.889 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:23.889 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:23.889 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:23.889 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:23.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:23.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:23.984 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174088 s, 60.2 MB/s 2024-04-25T10:11:24.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.004 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161651 s, 64.9 MB/s 2024-04-25T10:11:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.026 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170155 s, 61.6 MB/s 2024-04-25T10:11:24.042 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.043 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134436 s, 78.0 MB/s 2024-04-25T10:11:24.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.059 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126064 s, 83.2 MB/s 2024-04-25T10:11:24.076 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.077 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138897 s, 75.5 MB/s 2024-04-25T10:11:24.096 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.097 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.097 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160707 s, 65.2 MB/s 2024-04-25T10:11:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.114 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135212 s, 77.6 MB/s 2024-04-25T10:11:24.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.132 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136926 s, 76.6 MB/s 2024-04-25T10:11:24.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.152 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161614 s, 64.9 MB/s 2024-04-25T10:11:24.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.169 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133622 s, 78.5 MB/s 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120217 s, 87.2 MB/s 2024-04-25T10:11:24.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.201 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119908 s, 87.4 MB/s 2024-04-25T10:11:24.218 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.218 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.218 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136758 s, 76.7 MB/s 2024-04-25T10:11:24.235 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.235 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.235 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132994 s, 78.8 MB/s 2024-04-25T10:11:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.253 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013748 s, 76.3 MB/s 2024-04-25T10:11:24.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.270 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132107 s, 79.4 MB/s 2024-04-25T10:11:24.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.288 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135775 s, 77.2 MB/s 2024-04-25T10:11:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153657 s, 68.2 MB/s 2024-04-25T10:11:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155398 s, 67.5 MB/s 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.345 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138871 s, 75.5 MB/s 2024-04-25T10:11:24.363 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.363 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.363 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141493 s, 74.1 MB/s 2024-04-25T10:11:24.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.384 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171212 s, 61.2 MB/s 2024-04-25T10:11:24.402 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.402 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.402 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137335 s, 76.4 MB/s 2024-04-25T10:11:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.420 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138515 s, 75.7 MB/s 2024-04-25T10:11:24.437 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.437 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.437 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130038 s, 80.6 MB/s 2024-04-25T10:11:24.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.457 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147999 s, 70.8 MB/s 2024-04-25T10:11:24.474 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.474 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.474 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125614 s, 83.5 MB/s 2024-04-25T10:11:24.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.494 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155937 s, 67.2 MB/s 2024-04-25T10:11:24.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.512 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137528 s, 76.2 MB/s 2024-04-25T10:11:24.530 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.530 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.530 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144072 s, 72.8 MB/s 2024-04-25T10:11:24.549 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.549 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.549 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142464 s, 73.6 MB/s 2024-04-25T10:11:24.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:24.550 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:24.550 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:24.550 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:11:24.550 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:24.551 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:24.551 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:24.551 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:24.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.578 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011323 s, 92.6 MB/s 2024-04-25T10:11:24.594 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.594 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.594 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012733 s, 82.4 MB/s 2024-04-25T10:11:24.610 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.610 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.610 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124551 s, 84.2 MB/s 2024-04-25T10:11:24.626 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.626 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.626 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011997 s, 87.4 MB/s 2024-04-25T10:11:24.642 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.642 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.642 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123568 s, 84.9 MB/s 2024-04-25T10:11:24.659 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.659 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.659 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138101 s, 75.9 MB/s 2024-04-25T10:11:24.676 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.677 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.677 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143398 s, 73.1 MB/s 2024-04-25T10:11:24.693 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.693 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.693 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121958 s, 86.0 MB/s 2024-04-25T10:11:24.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.707 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105513 s, 99.4 MB/s 2024-04-25T10:11:24.722 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.723 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116428 s, 90.1 MB/s 2024-04-25T10:11:24.736 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.737 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.737 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106672 s, 98.3 MB/s 2024-04-25T10:11:24.753 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.753 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.753 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013128 s, 79.9 MB/s 2024-04-25T10:11:24.771 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.771 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.771 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140412 s, 74.7 MB/s 2024-04-25T10:11:24.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.786 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.786 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106774 s, 98.2 MB/s 2024-04-25T10:11:24.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.800 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104647 s, 100 MB/s 2024-04-25T10:11:24.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.816 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125515 s, 83.5 MB/s 2024-04-25T10:11:24.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.833 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.833 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131013 s, 80.0 MB/s 2024-04-25T10:11:24.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.849 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126781 s, 82.7 MB/s 2024-04-25T10:11:24.865 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.866 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.866 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135327 s, 77.5 MB/s 2024-04-25T10:11:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106083 s, 98.8 MB/s 2024-04-25T10:11:24.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.898 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014156 s, 74.1 MB/s 2024-04-25T10:11:24.915 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.915 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.915 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129326 s, 81.1 MB/s 2024-04-25T10:11:24.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.933 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143935 s, 72.9 MB/s 2024-04-25T10:11:24.950 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.950 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.950 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131603 s, 79.7 MB/s 2024-04-25T10:11:24.966 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.967 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.967 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129361 s, 81.1 MB/s 2024-04-25T10:11:24.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.981 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112818 s, 92.9 MB/s 2024-04-25T10:11:24.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:24.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:24.999 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131205 s, 79.9 MB/s 2024-04-25T10:11:25.015 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.015 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.015 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121645 s, 86.2 MB/s 2024-04-25T10:11:25.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.030 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108676 s, 96.5 MB/s 2024-04-25T10:11:25.047 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.048 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141706 s, 74.0 MB/s 2024-04-25T10:11:25.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.065 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136486 s, 76.8 MB/s 2024-04-25T10:11:25.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.082 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127097 s, 82.5 MB/s 2024-04-25T10:11:25.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:25.083 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:25.083 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:25.083 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:11:25.084 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:25.084 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:25.084 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:25.084 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:25.159 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.159 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.159 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159029 s, 65.9 MB/s 2024-04-25T10:11:25.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.178 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143435 s, 73.1 MB/s 2024-04-25T10:11:25.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.194 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127619 s, 82.2 MB/s 2024-04-25T10:11:25.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.208 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102984 s, 102 MB/s 2024-04-25T10:11:25.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.225 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01293 s, 81.1 MB/s 2024-04-25T10:11:25.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.243 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150223 s, 69.8 MB/s 2024-04-25T10:11:25.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.257 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104558 s, 100 MB/s 2024-04-25T10:11:25.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.272 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108312 s, 96.8 MB/s 2024-04-25T10:11:25.289 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.289 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.289 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134792 s, 77.8 MB/s 2024-04-25T10:11:25.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.306 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136285 s, 76.9 MB/s 2024-04-25T10:11:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143053 s, 73.3 MB/s 2024-04-25T10:11:25.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.341 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130654 s, 80.3 MB/s 2024-04-25T10:11:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014992 s, 69.9 MB/s 2024-04-25T10:11:25.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.378 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138756 s, 75.6 MB/s 2024-04-25T10:11:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112117 s, 93.5 MB/s 2024-04-25T10:11:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.411 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132321 s, 79.2 MB/s 2024-04-25T10:11:25.426 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.427 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135838 s, 77.2 MB/s 2024-04-25T10:11:25.443 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.443 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.443 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130995 s, 80.0 MB/s 2024-04-25T10:11:25.459 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.460 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.460 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128633 s, 81.5 MB/s 2024-04-25T10:11:25.475 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.475 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.476 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119068 s, 88.1 MB/s 2024-04-25T10:11:25.492 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.492 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.492 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131357 s, 79.8 MB/s 2024-04-25T10:11:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115401 s, 90.9 MB/s 2024-04-25T10:11:25.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.525 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141864 s, 73.9 MB/s 2024-04-25T10:11:25.542 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.542 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.542 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013734 s, 76.3 MB/s 2024-04-25T10:11:25.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143148 s, 73.3 MB/s 2024-04-25T10:11:25.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.578 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138602 s, 75.7 MB/s 2024-04-25T10:11:25.594 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.594 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.594 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132925 s, 78.9 MB/s 2024-04-25T10:11:25.609 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.609 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.609 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010896 s, 96.2 MB/s 2024-04-25T10:11:25.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.625 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129268 s, 81.1 MB/s 2024-04-25T10:11:25.641 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.642 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.642 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125566 s, 83.5 MB/s 2024-04-25T10:11:25.657 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.657 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.657 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118996 s, 88.1 MB/s 2024-04-25T10:11:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106597 s, 98.4 MB/s 2024-04-25T10:11:25.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:25.672 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:25.672 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:25.672 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:11:25.672 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:25.672 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:25.673 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:25.673 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:25.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.740 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.740 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100729 s, 104 MB/s 2024-04-25T10:11:25.756 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.756 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.756 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125833 s, 83.3 MB/s 2024-04-25T10:11:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.771 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121069 s, 86.6 MB/s 2024-04-25T10:11:25.788 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.788 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.789 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134918 s, 77.7 MB/s 2024-04-25T10:11:25.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126069 s, 83.2 MB/s 2024-04-25T10:11:25.823 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.823 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.823 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146124 s, 71.8 MB/s 2024-04-25T10:11:25.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.841 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146775 s, 71.4 MB/s 2024-04-25T10:11:25.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.858 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130708 s, 80.2 MB/s 2024-04-25T10:11:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129195 s, 81.2 MB/s 2024-04-25T10:11:25.891 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.891 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.891 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125757 s, 83.4 MB/s 2024-04-25T10:11:25.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.907 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125194 s, 83.8 MB/s 2024-04-25T10:11:25.924 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.924 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.924 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134294 s, 78.1 MB/s 2024-04-25T10:11:25.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.943 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014557 s, 72.0 MB/s 2024-04-25T10:11:25.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.960 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137961 s, 76.0 MB/s 2024-04-25T10:11:25.976 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.977 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.977 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130924 s, 80.1 MB/s 2024-04-25T10:11:25.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:25.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:25.991 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011224 s, 93.4 MB/s 2024-04-25T10:11:26.008 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.008 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.008 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137152 s, 76.5 MB/s 2024-04-25T10:11:26.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.026 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013501 s, 77.7 MB/s 2024-04-25T10:11:26.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.043 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134748 s, 77.8 MB/s 2024-04-25T10:11:26.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.059 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118567 s, 88.4 MB/s 2024-04-25T10:11:26.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.074 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.074 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114922 s, 91.2 MB/s 2024-04-25T10:11:26.089 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.089 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.089 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120622 s, 86.9 MB/s 2024-04-25T10:11:26.104 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.104 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.104 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109984 s, 95.3 MB/s 2024-04-25T10:11:26.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.117 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968572 s, 108 MB/s 2024-04-25T10:11:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139575 s, 75.1 MB/s 2024-04-25T10:11:26.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116528 s, 90.0 MB/s 2024-04-25T10:11:26.163 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.163 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.164 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966425 s, 109 MB/s 2024-04-25T10:11:26.177 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.177 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.177 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103417 s, 101 MB/s 2024-04-25T10:11:26.193 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.193 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.193 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120091 s, 87.3 MB/s 2024-04-25T10:11:26.209 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.209 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.209 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127853 s, 82.0 MB/s 2024-04-25T10:11:26.226 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.226 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.226 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139148 s, 75.4 MB/s 2024-04-25T10:11:26.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.242 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120424 s, 87.1 MB/s 2024-04-25T10:11:26.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:26.243 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:26.243 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:26.243 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:11:26.243 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:26.243 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:26.243 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:26.243 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:26.312 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.312 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.312 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121036 s, 86.6 MB/s 2024-04-25T10:11:26.327 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.327 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.327 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112268 s, 93.4 MB/s 2024-04-25T10:11:26.342 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.342 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.342 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116118 s, 90.3 MB/s 2024-04-25T10:11:26.355 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.355 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.356 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108587 s, 96.6 MB/s 2024-04-25T10:11:26.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.367 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873186 s, 120 MB/s 2024-04-25T10:11:26.382 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.382 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.382 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109636 s, 95.6 MB/s 2024-04-25T10:11:26.398 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.398 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.398 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128272 s, 81.7 MB/s 2024-04-25T10:11:26.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.414 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122818 s, 85.4 MB/s 2024-04-25T10:11:26.428 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.428 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.428 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115552 s, 90.7 MB/s 2024-04-25T10:11:26.444 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.444 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.444 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120248 s, 87.2 MB/s 2024-04-25T10:11:26.456 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.456 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.456 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912002 s, 115 MB/s 2024-04-25T10:11:26.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.473 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133785 s, 78.4 MB/s 2024-04-25T10:11:26.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.489 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.489 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116546 s, 90.0 MB/s 2024-04-25T10:11:26.505 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.505 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.505 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01313 s, 79.9 MB/s 2024-04-25T10:11:26.522 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.522 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.522 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131217 s, 79.9 MB/s 2024-04-25T10:11:26.537 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.537 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.537 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114424 s, 91.6 MB/s 2024-04-25T10:11:26.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.553 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.553 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012304 s, 85.2 MB/s 2024-04-25T10:11:26.568 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.569 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128797 s, 81.4 MB/s 2024-04-25T10:11:26.581 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.582 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951465 s, 110 MB/s 2024-04-25T10:11:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.597 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.597 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119521 s, 87.7 MB/s 2024-04-25T10:11:26.613 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.613 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.613 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132483 s, 79.1 MB/s 2024-04-25T10:11:26.626 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.626 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.626 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973115 s, 108 MB/s 2024-04-25T10:11:26.639 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.639 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.639 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997779 s, 105 MB/s 2024-04-25T10:11:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.656 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130412 s, 80.4 MB/s 2024-04-25T10:11:26.672 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.672 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.673 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013202 s, 79.4 MB/s 2024-04-25T10:11:26.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.686 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105134 s, 99.7 MB/s 2024-04-25T10:11:26.702 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.703 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126199 s, 83.1 MB/s 2024-04-25T10:11:26.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.719 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126888 s, 82.6 MB/s 2024-04-25T10:11:26.735 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.735 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.735 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130373 s, 80.4 MB/s 2024-04-25T10:11:26.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.751 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122393 s, 85.7 MB/s 2024-04-25T10:11:26.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.768 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134546 s, 77.9 MB/s 2024-04-25T10:11:26.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.785 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133042 s, 78.8 MB/s 2024-04-25T10:11:26.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:26.786 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:26.786 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:26.786 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:11:26.786 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:26.786 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:26.786 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:26.786 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133701 s, 78.4 MB/s 2024-04-25T10:11:26.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.875 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135235 s, 77.5 MB/s 2024-04-25T10:11:26.892 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.892 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.892 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133947 s, 78.3 MB/s 2024-04-25T10:11:26.910 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.910 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.910 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121613 s, 86.2 MB/s 2024-04-25T10:11:26.925 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.925 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.926 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136506 s, 76.8 MB/s 2024-04-25T10:11:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.943 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.943 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012909 s, 81.2 MB/s 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.958 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012759 s, 82.2 MB/s 2024-04-25T10:11:26.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.973 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111208 s, 94.3 MB/s 2024-04-25T10:11:26.989 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:26.989 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:26.989 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128428 s, 81.6 MB/s 2024-04-25T10:11:27.006 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.006 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.007 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013921 s, 75.3 MB/s 2024-04-25T10:11:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.021 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010339 s, 101 MB/s 2024-04-25T10:11:27.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.034 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101969 s, 103 MB/s 2024-04-25T10:11:27.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.050 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122938 s, 85.3 MB/s 2024-04-25T10:11:27.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.065 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123307 s, 85.0 MB/s 2024-04-25T10:11:27.083 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.084 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151602 s, 69.2 MB/s 2024-04-25T10:11:27.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.102 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140984 s, 74.4 MB/s 2024-04-25T10:11:27.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.117 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110549 s, 94.9 MB/s 2024-04-25T10:11:27.135 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.136 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.136 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142773 s, 73.4 MB/s 2024-04-25T10:11:27.154 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.154 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.155 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146732 s, 71.5 MB/s 2024-04-25T10:11:27.170 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.170 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.170 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121088 s, 86.6 MB/s 2024-04-25T10:11:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.185 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118582 s, 88.4 MB/s 2024-04-25T10:11:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.198 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967286 s, 108 MB/s 2024-04-25T10:11:27.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.215 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139105 s, 75.4 MB/s 2024-04-25T10:11:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.233 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133734 s, 78.4 MB/s 2024-04-25T10:11:27.247 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.247 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.247 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111466 s, 94.1 MB/s 2024-04-25T10:11:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963821 s, 109 MB/s 2024-04-25T10:11:27.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.275 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120736 s, 86.8 MB/s 2024-04-25T10:11:27.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.291 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123267 s, 85.1 MB/s 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918784 s, 114 MB/s 2024-04-25T10:11:27.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.318 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011647 s, 90.0 MB/s 2024-04-25T10:11:27.331 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.331 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.331 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966775 s, 108 MB/s 2024-04-25T10:11:27.346 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.347 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.347 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126199 s, 83.1 MB/s 2024-04-25T10:11:27.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:27.348 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:27.348 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:27.348 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:11:27.348 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:27.348 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:27.348 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:27.348 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108643 s, 96.5 MB/s 2024-04-25T10:11:27.429 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.430 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.430 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01091 s, 96.1 MB/s 2024-04-25T10:11:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.444 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110652 s, 94.8 MB/s 2024-04-25T10:11:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118007 s, 88.9 MB/s 2024-04-25T10:11:27.472 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.473 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104402 s, 100 MB/s 2024-04-25T10:11:27.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.488 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119523 s, 87.7 MB/s 2024-04-25T10:11:27.504 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.504 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.504 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013082 s, 80.2 MB/s 2024-04-25T10:11:27.520 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.520 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.521 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129331 s, 81.1 MB/s 2024-04-25T10:11:27.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.536 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119112 s, 88.0 MB/s 2024-04-25T10:11:27.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.546 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759879 s, 138 MB/s 2024-04-25T10:11:27.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110686 s, 94.7 MB/s 2024-04-25T10:11:27.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.576 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121411 s, 86.4 MB/s 2024-04-25T10:11:27.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.590 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117209 s, 89.5 MB/s 2024-04-25T10:11:27.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.603 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.603 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929913 s, 113 MB/s 2024-04-25T10:11:27.614 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.614 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.615 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813049 s, 129 MB/s 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010555 s, 99.3 MB/s 2024-04-25T10:11:27.644 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.645 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.645 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012718 s, 82.4 MB/s 2024-04-25T10:11:27.659 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.659 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.659 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111952 s, 93.7 MB/s 2024-04-25T10:11:27.672 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.672 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.672 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102173 s, 103 MB/s 2024-04-25T10:11:27.683 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.683 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.683 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081118 s, 129 MB/s 2024-04-25T10:11:27.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.698 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114085 s, 91.9 MB/s 2024-04-25T10:11:27.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.709 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787601 s, 133 MB/s 2024-04-25T10:11:27.722 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.722 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.722 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101887 s, 103 MB/s 2024-04-25T10:11:27.737 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.737 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.737 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117418 s, 89.3 MB/s 2024-04-25T10:11:27.750 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.751 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103657 s, 101 MB/s 2024-04-25T10:11:27.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.766 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119439 s, 87.8 MB/s 2024-04-25T10:11:27.780 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.780 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.780 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109608 s, 95.7 MB/s 2024-04-25T10:11:27.794 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.794 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.794 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108113 s, 97.0 MB/s 2024-04-25T10:11:27.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.808 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.808 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104004 s, 101 MB/s 2024-04-25T10:11:27.823 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129366 s, 81.1 MB/s 2024-04-25T10:11:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.837 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.838 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105574 s, 99.3 MB/s 2024-04-25T10:11:27.854 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.854 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.854 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129012 s, 81.3 MB/s 2024-04-25T10:11:27.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:27.855 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:27.855 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:27.855 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:11:27.855 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:27.855 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:27.855 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:27.856 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:27.926 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.929 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.929 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113141 s, 92.7 MB/s 2024-04-25T10:11:27.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.942 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124792 s, 84.0 MB/s 2024-04-25T10:11:27.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.958 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129932 s, 80.7 MB/s 2024-04-25T10:11:27.977 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.978 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.978 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143335 s, 73.2 MB/s 2024-04-25T10:11:27.996 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:27.996 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:27.996 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149697 s, 70.0 MB/s 2024-04-25T10:11:28.011 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.011 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.012 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108489 s, 96.7 MB/s 2024-04-25T10:11:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136651 s, 76.7 MB/s 2024-04-25T10:11:28.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.044 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972082 s, 108 MB/s 2024-04-25T10:11:28.060 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.060 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.060 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130164 s, 80.6 MB/s 2024-04-25T10:11:28.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.078 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140016 s, 74.9 MB/s 2024-04-25T10:11:28.096 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.096 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.096 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129673 s, 80.9 MB/s 2024-04-25T10:11:28.111 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.111 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.111 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114934 s, 91.2 MB/s 2024-04-25T10:11:28.127 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.127 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.127 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121227 s, 86.5 MB/s 2024-04-25T10:11:28.145 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.145 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.145 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146448 s, 71.6 MB/s 2024-04-25T10:11:28.163 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.163 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.163 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014729 s, 71.2 MB/s 2024-04-25T10:11:28.176 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.177 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.177 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100039 s, 105 MB/s 2024-04-25T10:11:28.192 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.192 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.192 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114727 s, 91.4 MB/s 2024-04-25T10:11:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.208 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128911 s, 81.3 MB/s 2024-04-25T10:11:28.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.222 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103049 s, 102 MB/s 2024-04-25T10:11:28.239 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.239 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.239 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133977 s, 78.3 MB/s 2024-04-25T10:11:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.255 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129051 s, 81.3 MB/s 2024-04-25T10:11:28.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.272 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135117 s, 77.6 MB/s 2024-04-25T10:11:28.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.288 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125339 s, 83.7 MB/s 2024-04-25T10:11:28.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.308 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015183 s, 69.1 MB/s 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107242 s, 97.8 MB/s 2024-04-25T10:11:28.336 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.336 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.336 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101364 s, 103 MB/s 2024-04-25T10:11:28.350 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.350 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.351 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106472 s, 98.5 MB/s 2024-04-25T10:11:28.366 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.367 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123235 s, 85.1 MB/s 2024-04-25T10:11:28.381 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.382 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.382 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115186 s, 91.0 MB/s 2024-04-25T10:11:28.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.396 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104965 s, 99.9 MB/s 2024-04-25T10:11:28.411 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.411 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.411 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114178 s, 91.8 MB/s 2024-04-25T10:11:28.428 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.428 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.428 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136814 s, 76.6 MB/s 2024-04-25T10:11:28.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:28.429 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:28.430 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:28.430 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:11:28.430 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:28.430 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:28.430 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:28.430 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:28.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.498 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127044 s, 82.5 MB/s 2024-04-25T10:11:28.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.512 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010841 s, 96.7 MB/s 2024-04-25T10:11:28.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.527 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110616 s, 94.8 MB/s 2024-04-25T10:11:28.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.543 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128774 s, 81.4 MB/s 2024-04-25T10:11:28.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.559 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120169 s, 87.3 MB/s 2024-04-25T10:11:28.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.576 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122693 s, 85.5 MB/s 2024-04-25T10:11:28.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.591 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122218 s, 85.8 MB/s 2024-04-25T10:11:28.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.607 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116184 s, 90.3 MB/s 2024-04-25T10:11:28.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.622 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109818 s, 95.5 MB/s 2024-04-25T10:11:28.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.638 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126557 s, 82.9 MB/s 2024-04-25T10:11:28.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.654 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121513 s, 86.3 MB/s 2024-04-25T10:11:28.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.671 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129986 s, 80.7 MB/s 2024-04-25T10:11:28.684 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.684 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.684 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978884 s, 107 MB/s 2024-04-25T10:11:28.700 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.700 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.700 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119466 s, 87.8 MB/s 2024-04-25T10:11:28.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.715 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109601 s, 95.7 MB/s 2024-04-25T10:11:28.731 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.731 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.731 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123296 s, 85.0 MB/s 2024-04-25T10:11:28.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.747 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122286 s, 85.7 MB/s 2024-04-25T10:11:28.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.763 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124713 s, 84.1 MB/s 2024-04-25T10:11:28.778 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.778 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.778 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105507 s, 99.4 MB/s 2024-04-25T10:11:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114974 s, 91.2 MB/s 2024-04-25T10:11:28.809 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.809 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.809 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122031 s, 85.9 MB/s 2024-04-25T10:11:28.822 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.822 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.822 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989304 s, 106 MB/s 2024-04-25T10:11:28.839 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.839 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.839 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129808 s, 80.8 MB/s 2024-04-25T10:11:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121642 s, 86.2 MB/s 2024-04-25T10:11:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114041 s, 91.9 MB/s 2024-04-25T10:11:28.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.888 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140418 s, 74.7 MB/s 2024-04-25T10:11:28.906 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.906 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.906 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138337 s, 75.8 MB/s 2024-04-25T10:11:28.924 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.924 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.924 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131908 s, 79.5 MB/s 2024-04-25T10:11:28.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.940 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118774 s, 88.3 MB/s 2024-04-25T10:11:28.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.954 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948944 s, 110 MB/s 2024-04-25T10:11:28.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.968 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113965 s, 92.0 MB/s 2024-04-25T10:11:28.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:28.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:28.984 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125032 s, 83.9 MB/s 2024-04-25T10:11:28.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:28.985 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:28.985 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:28.986 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:11:28.986 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:28.986 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:28.986 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:28.986 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:29.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.055 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.055 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121963 s, 86.0 MB/s 2024-04-25T10:11:29.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.071 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126691 s, 82.8 MB/s 2024-04-25T10:11:29.087 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.087 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.087 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124427 s, 84.3 MB/s 2024-04-25T10:11:29.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.102 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011744 s, 89.3 MB/s 2024-04-25T10:11:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122726 s, 85.4 MB/s 2024-04-25T10:11:29.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.133 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114681 s, 91.4 MB/s 2024-04-25T10:11:29.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.151 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.151 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01355 s, 77.4 MB/s 2024-04-25T10:11:29.168 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.168 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.168 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138362 s, 75.8 MB/s 2024-04-25T10:11:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128489 s, 81.6 MB/s 2024-04-25T10:11:29.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.201 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126607 s, 82.8 MB/s 2024-04-25T10:11:29.217 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.217 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.217 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121952 s, 86.0 MB/s 2024-04-25T10:11:29.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.232 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106856 s, 98.1 MB/s 2024-04-25T10:11:29.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.248 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126127 s, 83.1 MB/s 2024-04-25T10:11:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127331 s, 82.4 MB/s 2024-04-25T10:11:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120872 s, 86.8 MB/s 2024-04-25T10:11:29.298 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.298 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.298 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136683 s, 76.7 MB/s 2024-04-25T10:11:29.317 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.317 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.318 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159266 s, 65.8 MB/s 2024-04-25T10:11:29.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.339 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134624 s, 77.9 MB/s 2024-04-25T10:11:29.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.357 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012495 s, 83.9 MB/s 2024-04-25T10:11:29.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.370 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108199 s, 96.9 MB/s 2024-04-25T10:11:29.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.386 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129558 s, 80.9 MB/s 2024-04-25T10:11:29.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.402 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.402 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117512 s, 89.2 MB/s 2024-04-25T10:11:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975942 s, 107 MB/s 2024-04-25T10:11:29.431 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.432 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.432 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133211 s, 78.7 MB/s 2024-04-25T10:11:29.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.449 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135793 s, 77.2 MB/s 2024-04-25T10:11:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013619 s, 77.0 MB/s 2024-04-25T10:11:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121163 s, 86.5 MB/s 2024-04-25T10:11:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.500 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135241 s, 77.5 MB/s 2024-04-25T10:11:29.518 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.519 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.519 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144376 s, 72.6 MB/s 2024-04-25T10:11:29.535 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.535 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.535 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125177 s, 83.8 MB/s 2024-04-25T10:11:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.552 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128865 s, 81.4 MB/s 2024-04-25T10:11:29.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.569 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131574 s, 79.7 MB/s 2024-04-25T10:11:29.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:29.570 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:29.570 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:29.570 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:11:29.570 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:29.570 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:29.570 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:29.570 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:29.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.643 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126466 s, 82.9 MB/s 2024-04-25T10:11:29.661 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.661 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.661 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137115 s, 76.5 MB/s 2024-04-25T10:11:29.677 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.677 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.678 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124021 s, 84.5 MB/s 2024-04-25T10:11:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136583 s, 76.8 MB/s 2024-04-25T10:11:29.713 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.713 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.713 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137615 s, 76.2 MB/s 2024-04-25T10:11:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.728 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.729 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117658 s, 89.1 MB/s 2024-04-25T10:11:29.742 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.742 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.743 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105896 s, 99.0 MB/s 2024-04-25T10:11:29.758 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.758 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.758 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119371 s, 87.8 MB/s 2024-04-25T10:11:29.773 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.773 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.773 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117034 s, 89.6 MB/s 2024-04-25T10:11:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.790 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127996 s, 81.9 MB/s 2024-04-25T10:11:29.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.805 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118725 s, 88.3 MB/s 2024-04-25T10:11:29.820 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.821 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.821 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118747 s, 88.3 MB/s 2024-04-25T10:11:29.838 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.839 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.839 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135946 s, 77.1 MB/s 2024-04-25T10:11:29.854 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.854 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.854 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117847 s, 89.0 MB/s 2024-04-25T10:11:29.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.871 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128074 s, 81.9 MB/s 2024-04-25T10:11:29.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.888 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131428 s, 79.8 MB/s 2024-04-25T10:11:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.904 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129745 s, 80.8 MB/s 2024-04-25T10:11:29.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.922 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139002 s, 75.4 MB/s 2024-04-25T10:11:29.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.940 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143137 s, 73.3 MB/s 2024-04-25T10:11:29.956 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.956 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.956 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121087 s, 86.6 MB/s 2024-04-25T10:11:29.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.974 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.974 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137127 s, 76.5 MB/s 2024-04-25T10:11:29.990 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:29.990 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:29.990 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127549 s, 82.2 MB/s 2024-04-25T10:11:30.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.005 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115286 s, 91.0 MB/s 2024-04-25T10:11:30.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.020 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111624 s, 93.9 MB/s 2024-04-25T10:11:30.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.038 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138565 s, 75.7 MB/s 2024-04-25T10:11:30.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.054 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125989 s, 83.2 MB/s 2024-04-25T10:11:30.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.071 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124897 s, 84.0 MB/s 2024-04-25T10:11:30.085 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.085 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.085 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107784 s, 97.3 MB/s 2024-04-25T10:11:30.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.102 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132701 s, 79.0 MB/s 2024-04-25T10:11:30.119 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.119 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.119 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130162 s, 80.6 MB/s 2024-04-25T10:11:30.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.135 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119496 s, 87.7 MB/s 2024-04-25T10:11:30.151 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.151 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.152 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131463 s, 79.8 MB/s 2024-04-25T10:11:30.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:30.153 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:30.153 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:30.153 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:11:30.153 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:30.153 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:30.153 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:30.154 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:30.223 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.223 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.223 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133068 s, 78.8 MB/s 2024-04-25T10:11:30.237 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.237 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.237 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105124 s, 99.7 MB/s 2024-04-25T10:11:30.251 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.251 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.251 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106694 s, 98.3 MB/s 2024-04-25T10:11:30.268 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.268 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.269 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133685 s, 78.4 MB/s 2024-04-25T10:11:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132084 s, 79.4 MB/s 2024-04-25T10:11:30.301 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.301 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.302 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012556 s, 83.5 MB/s 2024-04-25T10:11:30.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.320 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136171 s, 77.0 MB/s 2024-04-25T10:11:30.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.377 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150617 s, 69.6 MB/s 2024-04-25T10:11:30.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.397 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147 s, 71.3 MB/s 2024-04-25T10:11:30.417 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.417 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.417 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153029 s, 68.5 MB/s 2024-04-25T10:11:30.434 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.434 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.435 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135083 s, 77.6 MB/s 2024-04-25T10:11:30.452 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.452 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.452 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140524 s, 74.6 MB/s 2024-04-25T10:11:30.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.467 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011029 s, 95.1 MB/s 2024-04-25T10:11:30.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.482 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106566 s, 98.4 MB/s 2024-04-25T10:11:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011942 s, 87.8 MB/s 2024-04-25T10:11:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122794 s, 85.4 MB/s 2024-04-25T10:11:30.530 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.530 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.531 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013592 s, 77.1 MB/s 2024-04-25T10:11:30.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.548 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133136 s, 78.8 MB/s 2024-04-25T10:11:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129011 s, 81.3 MB/s 2024-04-25T10:11:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129424 s, 81.0 MB/s 2024-04-25T10:11:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013306 s, 78.8 MB/s 2024-04-25T10:11:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128388 s, 81.7 MB/s 2024-04-25T10:11:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.633 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135656 s, 77.3 MB/s 2024-04-25T10:11:30.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.651 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130244 s, 80.5 MB/s 2024-04-25T10:11:30.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.669 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136693 s, 76.7 MB/s 2024-04-25T10:11:30.687 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.688 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.688 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137754 s, 76.1 MB/s 2024-04-25T10:11:30.705 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.706 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.706 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130396 s, 80.4 MB/s 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129145 s, 81.2 MB/s 2024-04-25T10:11:30.742 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.742 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.742 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132473 s, 79.2 MB/s 2024-04-25T10:11:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135671 s, 77.3 MB/s 2024-04-25T10:11:30.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.777 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117167 s, 89.5 MB/s 2024-04-25T10:11:30.795 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.795 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.795 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138788 s, 75.6 MB/s 2024-04-25T10:11:30.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:30.796 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:30.797 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:30.797 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:11:30.797 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:30.797 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:30.797 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:30.797 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:30.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.872 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.872 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129852 s, 80.8 MB/s 2024-04-25T10:11:30.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.888 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124063 s, 84.5 MB/s 2024-04-25T10:11:30.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.904 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010864 s, 96.5 MB/s 2024-04-25T10:11:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127834 s, 82.0 MB/s 2024-04-25T10:11:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131185 s, 79.9 MB/s 2024-04-25T10:11:30.956 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.956 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.956 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123055 s, 85.2 MB/s 2024-04-25T10:11:30.972 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.972 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.972 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112562 s, 93.2 MB/s 2024-04-25T10:11:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:30.986 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981284 s, 107 MB/s 2024-04-25T10:11:31.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.004 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131073 s, 80.0 MB/s 2024-04-25T10:11:31.021 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.021 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.022 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130515 s, 80.3 MB/s 2024-04-25T10:11:31.040 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.040 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.041 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143041 s, 73.3 MB/s 2024-04-25T10:11:31.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.058 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128036 s, 81.9 MB/s 2024-04-25T10:11:31.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.073 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103241 s, 102 MB/s 2024-04-25T10:11:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111938 s, 93.7 MB/s 2024-04-25T10:11:31.105 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.105 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.105 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128359 s, 81.7 MB/s 2024-04-25T10:11:31.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.118 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912903 s, 115 MB/s 2024-04-25T10:11:31.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.134 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128366 s, 81.7 MB/s 2024-04-25T10:11:31.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012313 s, 85.2 MB/s 2024-04-25T10:11:31.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.165 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121943 s, 86.0 MB/s 2024-04-25T10:11:31.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.180 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011269 s, 93.0 MB/s 2024-04-25T10:11:31.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.194 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104868 s, 100 MB/s 2024-04-25T10:11:31.209 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119985 s, 87.4 MB/s 2024-04-25T10:11:31.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.225 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118916 s, 88.2 MB/s 2024-04-25T10:11:31.242 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.383 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139626 s, 75.1 MB/s 2024-04-25T10:11:31.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.385 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126566 s, 82.8 MB/s 2024-04-25T10:11:31.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.385 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131765 s, 79.6 MB/s 2024-04-25T10:11:31.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.386 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924241 s, 113 MB/s 2024-04-25T10:11:31.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.386 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01291 s, 81.2 MB/s 2024-04-25T10:11:31.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.386 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014541 s, 72.1 MB/s 2024-04-25T10:11:31.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.396 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151919 s, 69.0 MB/s 2024-04-25T10:11:31.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.415 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146539 s, 71.6 MB/s 2024-04-25T10:11:31.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.434 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.434 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133468 s, 78.6 MB/s 2024-04-25T10:11:31.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:31.435 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:31.435 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:31.435 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:11:31.435 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:31.435 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:31.435 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:31.435 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:31.511 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.512 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134503 s, 78.0 MB/s 2024-04-25T10:11:31.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.528 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121313 s, 86.4 MB/s 2024-04-25T10:11:31.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.544 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124495 s, 84.2 MB/s 2024-04-25T10:11:31.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122368 s, 85.7 MB/s 2024-04-25T10:11:31.573 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.573 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.573 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962612 s, 109 MB/s 2024-04-25T10:11:31.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.591 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013218 s, 79.3 MB/s 2024-04-25T10:11:31.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.607 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127612 s, 82.2 MB/s 2024-04-25T10:11:31.623 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.623 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.623 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126698 s, 82.8 MB/s 2024-04-25T10:11:31.639 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.639 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.639 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124254 s, 84.4 MB/s 2024-04-25T10:11:31.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.653 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103524 s, 101 MB/s 2024-04-25T10:11:31.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.667 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010563 s, 99.3 MB/s 2024-04-25T10:11:31.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.681 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104602 s, 100 MB/s 2024-04-25T10:11:31.697 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.698 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125853 s, 83.3 MB/s 2024-04-25T10:11:31.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.715 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133598 s, 78.5 MB/s 2024-04-25T10:11:31.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.730 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111723 s, 93.9 MB/s 2024-04-25T10:11:31.746 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.746 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.746 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012581 s, 83.3 MB/s 2024-04-25T10:11:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132239 s, 79.3 MB/s 2024-04-25T10:11:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107864 s, 97.2 MB/s 2024-04-25T10:11:31.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.793 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.793 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118772 s, 88.3 MB/s 2024-04-25T10:11:31.811 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.811 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.812 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147652 s, 71.0 MB/s 2024-04-25T10:11:31.828 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.828 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.828 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127037 s, 82.5 MB/s 2024-04-25T10:11:31.845 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.845 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.845 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127994 s, 81.9 MB/s 2024-04-25T10:11:31.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.861 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118257 s, 88.7 MB/s 2024-04-25T10:11:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138228 s, 75.9 MB/s 2024-04-25T10:11:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01312 s, 79.9 MB/s 2024-04-25T10:11:31.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.912 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122731 s, 85.4 MB/s 2024-04-25T10:11:31.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.928 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.928 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117129 s, 89.5 MB/s 2024-04-25T10:11:31.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.945 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133861 s, 78.3 MB/s 2024-04-25T10:11:31.961 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.961 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.961 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129222 s, 81.1 MB/s 2024-04-25T10:11:31.976 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.976 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.976 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110975 s, 94.5 MB/s 2024-04-25T10:11:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:31.991 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113904 s, 92.1 MB/s 2024-04-25T10:11:32.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.009 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.009 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125051 s, 83.9 MB/s 2024-04-25T10:11:32.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:32.010 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:32.010 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:32.010 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:11:32.010 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:32.011 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:32.011 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:32.011 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138035 s, 76.0 MB/s 2024-04-25T10:11:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113584 s, 92.3 MB/s 2024-04-25T10:11:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.123 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150885 s, 69.5 MB/s 2024-04-25T10:11:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.140 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.141 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136938 s, 76.6 MB/s 2024-04-25T10:11:32.158 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.158 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.158 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140447 s, 74.7 MB/s 2024-04-25T10:11:32.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.174 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130743 s, 80.2 MB/s 2024-04-25T10:11:32.190 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.190 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.191 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117793 s, 89.0 MB/s 2024-04-25T10:11:32.207 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.207 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.207 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128804 s, 81.4 MB/s 2024-04-25T10:11:32.223 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.224 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124472 s, 84.2 MB/s 2024-04-25T10:11:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.243 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148129 s, 70.8 MB/s 2024-04-25T10:11:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.258 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115508 s, 90.8 MB/s 2024-04-25T10:11:32.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.275 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129954 s, 80.7 MB/s 2024-04-25T10:11:32.289 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.290 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111152 s, 94.3 MB/s 2024-04-25T10:11:32.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.303 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105592 s, 99.3 MB/s 2024-04-25T10:11:32.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.322 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152422 s, 68.8 MB/s 2024-04-25T10:11:32.355 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.355 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.355 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119905 s, 87.5 MB/s 2024-04-25T10:11:32.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.371 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129816 s, 80.8 MB/s 2024-04-25T10:11:32.387 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.387 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.387 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123208 s, 85.1 MB/s 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.402 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112928 s, 92.9 MB/s 2024-04-25T10:11:32.418 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.418 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.418 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120406 s, 87.1 MB/s 2024-04-25T10:11:32.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.433 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118932 s, 88.2 MB/s 2024-04-25T10:11:32.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.449 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116976 s, 89.6 MB/s 2024-04-25T10:11:32.463 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.463 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.463 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104308 s, 101 MB/s 2024-04-25T10:11:32.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.480 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130469 s, 80.4 MB/s 2024-04-25T10:11:32.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.498 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146475 s, 71.6 MB/s 2024-04-25T10:11:32.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118574 s, 88.4 MB/s 2024-04-25T10:11:32.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.530 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122791 s, 85.4 MB/s 2024-04-25T10:11:32.545 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.545 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.545 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118016 s, 88.9 MB/s 2024-04-25T10:11:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114665 s, 91.4 MB/s 2024-04-25T10:11:32.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.576 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.576 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117079 s, 89.6 MB/s 2024-04-25T10:11:32.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.591 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115459 s, 90.8 MB/s 2024-04-25T10:11:32.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.607 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122893 s, 85.3 MB/s 2024-04-25T10:11:32.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:32.608 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:32.608 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:32.608 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:11:32.608 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:32.608 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:32.608 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:32.608 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:32.674 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.675 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092453 s, 113 MB/s 2024-04-25T10:11:32.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.689 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112035 s, 93.6 MB/s 2024-04-25T10:11:32.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.705 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.705 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118232 s, 88.7 MB/s 2024-04-25T10:11:32.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.719 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107764 s, 97.3 MB/s 2024-04-25T10:11:32.732 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.732 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.732 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968678 s, 108 MB/s 2024-04-25T10:11:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119476 s, 87.8 MB/s 2024-04-25T10:11:32.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.764 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123099 s, 85.2 MB/s 2024-04-25T10:11:32.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.779 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115185 s, 91.0 MB/s 2024-04-25T10:11:32.794 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.795 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.795 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120494 s, 87.0 MB/s 2024-04-25T10:11:32.809 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.809 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.809 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011156 s, 94.0 MB/s 2024-04-25T10:11:32.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.825 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127898 s, 82.0 MB/s 2024-04-25T10:11:32.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.840 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112227 s, 93.4 MB/s 2024-04-25T10:11:32.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.856 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127049 s, 82.5 MB/s 2024-04-25T10:11:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124582 s, 84.2 MB/s 2024-04-25T10:11:32.884 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.885 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.885 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942235 s, 111 MB/s 2024-04-25T10:11:32.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.898 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102254 s, 103 MB/s 2024-04-25T10:11:32.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.912 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104798 s, 100 MB/s 2024-04-25T10:11:32.928 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.928 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.928 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112967 s, 92.8 MB/s 2024-04-25T10:11:32.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.940 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091726 s, 114 MB/s 2024-04-25T10:11:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131377 s, 79.8 MB/s 2024-04-25T10:11:32.972 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.972 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.972 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118365 s, 88.6 MB/s 2024-04-25T10:11:32.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.985 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971713 s, 108 MB/s 2024-04-25T10:11:32.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:32.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:32.999 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107121 s, 97.9 MB/s 2024-04-25T10:11:33.012 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:33.012 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:33.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100455 s, 104 MB/s 2024-04-25T10:11:33.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:33.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:33.027 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112932 s, 92.9 MB/s 2024-04-25T10:11:33.042 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:33.042 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:33.043 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120146 s, 87.3 MB/s 2024-04-25T10:11:33.055 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:33.056 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:33.056 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972185 s, 108 MB/s 2024-04-25T10:11:33.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126625 s, 82.8 MB/s 2024-04-25T10:11:33.088 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:33.088 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:33.088 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129105 s, 81.2 MB/s 2024-04-25T10:11:33.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:33.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:33.102 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112256 s, 93.4 MB/s 2024-04-25T10:11:33.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:33.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:33.116 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010718 s, 97.8 MB/s 2024-04-25T10:11:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113363 s, 92.5 MB/s 2024-04-25T10:11:33.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:33.132 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:33.132 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:33.132 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:11:33.133 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:33.133 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:33.133 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:33.133 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:34.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.200 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 1.00737 s, 1.0 MB/s 2024-04-25T10:11:34.223 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.224 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186899 s, 56.1 MB/s 2024-04-25T10:11:34.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.246 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173106 s, 60.6 MB/s 2024-04-25T10:11:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156835 s, 66.9 MB/s 2024-04-25T10:11:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179719 s, 58.3 MB/s 2024-04-25T10:11:34.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.306 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131781 s, 79.6 MB/s 2024-04-25T10:11:34.323 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.323 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.323 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132615 s, 79.1 MB/s 2024-04-25T10:11:34.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.413 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018032 s, 58.2 MB/s 2024-04-25T10:11:34.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.434 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149907 s, 69.9 MB/s 2024-04-25T10:11:34.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.454 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156801 s, 66.9 MB/s 2024-04-25T10:11:34.474 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.474 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.474 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015137 s, 69.3 MB/s 2024-04-25T10:11:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.493 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145819 s, 71.9 MB/s 2024-04-25T10:11:34.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.513 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151407 s, 69.3 MB/s 2024-04-25T10:11:34.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.531 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138744 s, 75.6 MB/s 2024-04-25T10:11:34.548 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.548 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.548 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133324 s, 78.6 MB/s 2024-04-25T10:11:34.566 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.566 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.566 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144154 s, 72.7 MB/s 2024-04-25T10:11:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.583 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013482 s, 77.8 MB/s 2024-04-25T10:11:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138448 s, 75.7 MB/s 2024-04-25T10:11:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108904 s, 96.3 MB/s 2024-04-25T10:11:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111311 s, 94.2 MB/s 2024-04-25T10:11:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.649 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134196 s, 78.1 MB/s 2024-04-25T10:11:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128934 s, 81.3 MB/s 2024-04-25T10:11:34.682 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.682 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.682 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127769 s, 82.1 MB/s 2024-04-25T10:11:34.699 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.699 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.699 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122998 s, 85.3 MB/s 2024-04-25T10:11:34.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.715 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012512 s, 83.8 MB/s 2024-04-25T10:11:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129344 s, 81.1 MB/s 2024-04-25T10:11:34.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.749 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013059 s, 80.3 MB/s 2024-04-25T10:11:34.765 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.765 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.765 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127241 s, 82.4 MB/s 2024-04-25T10:11:34.780 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.780 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.780 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113381 s, 92.5 MB/s 2024-04-25T10:11:34.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.797 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121089 s, 86.6 MB/s 2024-04-25T10:11:34.811 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.812 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.812 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110488 s, 94.9 MB/s 2024-04-25T10:11:34.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.829 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139323 s, 75.3 MB/s 2024-04-25T10:11:34.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:34.830 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:34.831 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:34.831 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:11:34.831 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:34.831 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:34.831 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:34.831 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:34.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108994 s, 96.2 MB/s 2024-04-25T10:11:34.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.870 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885427 s, 118 MB/s 2024-04-25T10:11:34.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.887 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.887 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130904 s, 80.1 MB/s 2024-04-25T10:11:34.902 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.902 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.902 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118306 s, 88.6 MB/s 2024-04-25T10:11:34.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.916 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105051 s, 99.8 MB/s 2024-04-25T10:11:34.931 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.931 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.931 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106405 s, 98.5 MB/s 2024-04-25T10:11:34.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.947 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129303 s, 81.1 MB/s 2024-04-25T10:11:34.963 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.964 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122043 s, 85.9 MB/s 2024-04-25T10:11:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126865 s, 82.7 MB/s 2024-04-25T10:11:34.996 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:34.996 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:34.997 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122883 s, 85.3 MB/s 2024-04-25T10:11:35.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123696 s, 84.8 MB/s 2024-04-25T10:11:35.029 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.029 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.029 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122033 s, 85.9 MB/s 2024-04-25T10:11:35.046 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.046 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.046 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134342 s, 78.1 MB/s 2024-04-25T10:11:35.061 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.061 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.061 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117023 s, 89.6 MB/s 2024-04-25T10:11:35.079 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.079 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.079 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137809 s, 76.1 MB/s 2024-04-25T10:11:35.097 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.098 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149587 s, 70.1 MB/s 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126191 s, 83.1 MB/s 2024-04-25T10:11:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013906 s, 75.4 MB/s 2024-04-25T10:11:35.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118308 s, 88.6 MB/s 2024-04-25T10:11:35.170 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.170 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.170 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147918 s, 70.9 MB/s 2024-04-25T10:11:35.187 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.187 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.187 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123304 s, 85.0 MB/s 2024-04-25T10:11:35.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137159 s, 76.4 MB/s 2024-04-25T10:11:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.221 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106411 s, 98.5 MB/s 2024-04-25T10:11:35.236 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.236 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.236 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104471 s, 100 MB/s 2024-04-25T10:11:35.251 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.251 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.251 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103856 s, 101 MB/s 2024-04-25T10:11:35.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.271 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143659 s, 73.0 MB/s 2024-04-25T10:11:35.285 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.286 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107101 s, 97.9 MB/s 2024-04-25T10:11:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139974 s, 74.9 MB/s 2024-04-25T10:11:35.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.321 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121827 s, 86.1 MB/s 2024-04-25T10:11:35.425 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.425 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.425 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169705 s, 61.8 MB/s 2024-04-25T10:11:35.446 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.446 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.446 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015371 s, 68.2 MB/s 2024-04-25T10:11:35.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.467 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160131 s, 65.5 MB/s 2024-04-25T10:11:35.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:35.468 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:35.468 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:35.468 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:11:35.469 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:35.469 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:35.469 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:35.469 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:35.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.544 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141886 s, 73.9 MB/s 2024-04-25T10:11:35.562 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.562 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.562 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136706 s, 76.7 MB/s 2024-04-25T10:11:35.581 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.581 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.581 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013986 s, 75.0 MB/s 2024-04-25T10:11:35.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.599 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130985 s, 80.1 MB/s 2024-04-25T10:11:35.617 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.618 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137781 s, 76.1 MB/s 2024-04-25T10:11:35.635 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.635 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.635 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123962 s, 84.6 MB/s 2024-04-25T10:11:35.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.651 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114096 s, 91.9 MB/s 2024-04-25T10:11:35.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.667 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106174 s, 98.8 MB/s 2024-04-25T10:11:35.684 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.685 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140847 s, 74.4 MB/s 2024-04-25T10:11:35.700 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.700 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.700 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121333 s, 86.4 MB/s 2024-04-25T10:11:35.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.715 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106546 s, 98.4 MB/s 2024-04-25T10:11:35.731 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.731 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.731 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129746 s, 80.8 MB/s 2024-04-25T10:11:35.749 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.749 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.749 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136204 s, 77.0 MB/s 2024-04-25T10:11:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.767 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.767 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138342 s, 75.8 MB/s 2024-04-25T10:11:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123473 s, 84.9 MB/s 2024-04-25T10:11:35.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.801 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.801 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138557 s, 75.7 MB/s 2024-04-25T10:11:35.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.815 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107099 s, 97.9 MB/s 2024-04-25T10:11:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138217 s, 75.9 MB/s 2024-04-25T10:11:35.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.849 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012997 s, 80.7 MB/s 2024-04-25T10:11:35.865 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.866 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.866 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012979 s, 80.8 MB/s 2024-04-25T10:11:35.879 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.879 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.879 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985953 s, 106 MB/s 2024-04-25T10:11:35.895 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.896 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.896 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128941 s, 81.3 MB/s 2024-04-25T10:11:35.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.912 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123877 s, 84.6 MB/s 2024-04-25T10:11:35.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.930 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142426 s, 73.6 MB/s 2024-04-25T10:11:35.946 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.946 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.946 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126965 s, 82.6 MB/s 2024-04-25T10:11:35.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.964 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144671 s, 72.5 MB/s 2024-04-25T10:11:35.979 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.979 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.979 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109 s, 96.2 MB/s 2024-04-25T10:11:35.996 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:35.996 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:35.997 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139897 s, 75.0 MB/s 2024-04-25T10:11:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127431 s, 82.3 MB/s 2024-04-25T10:11:36.031 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.031 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.031 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142285 s, 73.7 MB/s 2024-04-25T10:11:36.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.049 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014533 s, 72.2 MB/s 2024-04-25T10:11:36.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.068 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142773 s, 73.4 MB/s 2024-04-25T10:11:36.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:36.069 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:36.069 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:36.069 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:11:36.069 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:36.069 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:36.069 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:36.069 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:36.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.137 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116329 s, 90.1 MB/s 2024-04-25T10:11:36.157 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.157 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.157 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165174 s, 63.5 MB/s 2024-04-25T10:11:36.173 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.173 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.173 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121217 s, 86.5 MB/s 2024-04-25T10:11:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143376 s, 73.1 MB/s 2024-04-25T10:11:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.208 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129345 s, 81.1 MB/s 2024-04-25T10:11:36.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.225 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133317 s, 78.7 MB/s 2024-04-25T10:11:36.240 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.241 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119674 s, 87.6 MB/s 2024-04-25T10:11:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120084 s, 87.3 MB/s 2024-04-25T10:11:36.271 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.271 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.271 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107538 s, 97.5 MB/s 2024-04-25T10:11:36.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.286 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118795 s, 88.3 MB/s 2024-04-25T10:11:36.301 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.301 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.301 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120072 s, 87.3 MB/s 2024-04-25T10:11:36.317 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.317 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.317 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118455 s, 88.5 MB/s 2024-04-25T10:11:36.331 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.331 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.331 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111426 s, 94.1 MB/s 2024-04-25T10:11:36.389 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.389 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.389 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111298 s, 94.2 MB/s 2024-04-25T10:11:36.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.406 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126517 s, 82.9 MB/s 2024-04-25T10:11:36.424 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.424 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143319 s, 73.2 MB/s 2024-04-25T10:11:36.440 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.441 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124478 s, 84.2 MB/s 2024-04-25T10:11:36.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.457 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128251 s, 81.8 MB/s 2024-04-25T10:11:36.474 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.475 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.475 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138032 s, 76.0 MB/s 2024-04-25T10:11:36.492 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.492 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.492 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137374 s, 76.3 MB/s 2024-04-25T10:11:36.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106475 s, 98.5 MB/s 2024-04-25T10:11:36.521 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.521 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.521 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115759 s, 90.6 MB/s 2024-04-25T10:11:36.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.539 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135177 s, 77.6 MB/s 2024-04-25T10:11:36.555 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.555 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.555 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012806 s, 81.9 MB/s 2024-04-25T10:11:36.571 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.571 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.571 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127227 s, 82.4 MB/s 2024-04-25T10:11:36.586 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.586 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.586 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111772 s, 93.8 MB/s 2024-04-25T10:11:36.601 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.601 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.601 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115514 s, 90.8 MB/s 2024-04-25T10:11:36.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.618 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012864 s, 81.5 MB/s 2024-04-25T10:11:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.635 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.635 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122702 s, 85.5 MB/s 2024-04-25T10:11:36.652 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.652 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.652 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126321 s, 83.0 MB/s 2024-04-25T10:11:36.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.670 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131903 s, 79.5 MB/s 2024-04-25T10:11:36.688 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.688 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.688 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136627 s, 76.7 MB/s 2024-04-25T10:11:36.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:36.690 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:36.690 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:36.690 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:11:36.690 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:36.690 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:36.690 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:36.690 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943768 s, 111 MB/s 2024-04-25T10:11:36.773 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.773 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.773 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0096944 s, 108 MB/s 2024-04-25T10:11:36.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.789 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011807 s, 88.8 MB/s 2024-04-25T10:11:36.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.807 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132448 s, 79.2 MB/s 2024-04-25T10:11:36.824 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.824 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.824 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116859 s, 89.7 MB/s 2024-04-25T10:11:36.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.840 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119237 s, 87.9 MB/s 2024-04-25T10:11:36.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.858 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013161 s, 79.7 MB/s 2024-04-25T10:11:36.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.875 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122183 s, 85.8 MB/s 2024-04-25T10:11:36.893 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.893 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.893 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135414 s, 77.4 MB/s 2024-04-25T10:11:36.910 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.910 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.910 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011952 s, 87.7 MB/s 2024-04-25T10:11:36.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.927 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124168 s, 84.4 MB/s 2024-04-25T10:11:36.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.945 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121231 s, 86.5 MB/s 2024-04-25T10:11:36.961 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.961 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.961 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122579 s, 85.5 MB/s 2024-04-25T10:11:36.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.975 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948528 s, 111 MB/s 2024-04-25T10:11:36.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:36.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:36.993 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130051 s, 80.6 MB/s 2024-04-25T10:11:37.010 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.010 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.010 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129273 s, 81.1 MB/s 2024-04-25T10:11:37.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.026 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102706 s, 102 MB/s 2024-04-25T10:11:37.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.044 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132329 s, 79.2 MB/s 2024-04-25T10:11:37.057 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.057 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.057 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969593 s, 108 MB/s 2024-04-25T10:11:37.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.071 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886782 s, 118 MB/s 2024-04-25T10:11:37.088 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.089 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.089 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134059 s, 78.2 MB/s 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.105 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131701 s, 79.6 MB/s 2024-04-25T10:11:37.121 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.121 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.121 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012624 s, 83.1 MB/s 2024-04-25T10:11:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130331 s, 80.5 MB/s 2024-04-25T10:11:37.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.152 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994996 s, 105 MB/s 2024-04-25T10:11:37.167 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.167 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.167 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117899 s, 88.9 MB/s 2024-04-25T10:11:37.181 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.181 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.181 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986129 s, 106 MB/s 2024-04-25T10:11:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123958 s, 84.6 MB/s 2024-04-25T10:11:37.212 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.213 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.213 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126685 s, 82.8 MB/s 2024-04-25T10:11:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.228 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119465 s, 87.8 MB/s 2024-04-25T10:11:37.242 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.242 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.242 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112194 s, 93.5 MB/s 2024-04-25T10:11:37.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.253 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083119 s, 126 MB/s 2024-04-25T10:11:37.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:37.254 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:37.254 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:37.254 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:11:37.254 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:37.254 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:37.255 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:37.255 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:37.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.322 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110234 s, 95.1 MB/s 2024-04-25T10:11:37.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.339 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139797 s, 75.0 MB/s 2024-04-25T10:11:37.382 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.383 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077754 s, 135 MB/s 2024-04-25T10:11:37.399 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.399 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.399 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127291 s, 82.4 MB/s 2024-04-25T10:11:37.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.410 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080137 s, 131 MB/s 2024-04-25T10:11:37.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.427 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135876 s, 77.2 MB/s 2024-04-25T10:11:37.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.441 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105241 s, 99.6 MB/s 2024-04-25T10:11:37.459 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.459 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.459 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148235 s, 70.7 MB/s 2024-04-25T10:11:37.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.474 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115998 s, 90.4 MB/s 2024-04-25T10:11:37.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.488 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011862 s, 88.4 MB/s 2024-04-25T10:11:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104247 s, 101 MB/s 2024-04-25T10:11:37.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.517 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115652 s, 90.7 MB/s 2024-04-25T10:11:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095072 s, 110 MB/s 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125309 s, 83.7 MB/s 2024-04-25T10:11:37.558 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.558 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.558 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010336 s, 101 MB/s 2024-04-25T10:11:37.573 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.573 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.574 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119674 s, 87.6 MB/s 2024-04-25T10:11:37.589 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.589 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.589 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120987 s, 86.7 MB/s 2024-04-25T10:11:37.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121946 s, 86.0 MB/s 2024-04-25T10:11:37.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.619 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115162 s, 91.1 MB/s 2024-04-25T10:11:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.634 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111672 s, 93.9 MB/s 2024-04-25T10:11:37.647 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.647 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.647 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991138 s, 106 MB/s 2024-04-25T10:11:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123709 s, 84.8 MB/s 2024-04-25T10:11:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143609 s, 73.0 MB/s 2024-04-25T10:11:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.697 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124706 s, 84.1 MB/s 2024-04-25T10:11:37.713 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.714 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.714 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130976 s, 80.1 MB/s 2024-04-25T10:11:37.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.730 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122104 s, 85.9 MB/s 2024-04-25T10:11:37.743 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105474 s, 99.4 MB/s 2024-04-25T10:11:37.759 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.759 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117239 s, 89.4 MB/s 2024-04-25T10:11:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.775 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127087 s, 82.5 MB/s 2024-04-25T10:11:37.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.791 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118705 s, 88.3 MB/s 2024-04-25T10:11:37.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.804 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109383 s, 95.9 MB/s 2024-04-25T10:11:37.821 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.822 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.822 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134448 s, 78.0 MB/s 2024-04-25T10:11:37.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:37.823 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:37.823 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:37.823 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:11:37.823 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:37.823 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:37.823 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:37.823 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:37.895 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.895 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.895 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121965 s, 86.0 MB/s 2024-04-25T10:11:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116817 s, 89.8 MB/s 2024-04-25T10:11:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111597 s, 94.0 MB/s 2024-04-25T10:11:37.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.942 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130111 s, 80.6 MB/s 2024-04-25T10:11:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109169 s, 96.1 MB/s 2024-04-25T10:11:37.972 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.972 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.972 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106418 s, 98.5 MB/s 2024-04-25T10:11:37.987 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:37.988 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:37.988 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120784 s, 86.8 MB/s 2024-04-25T10:11:38.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.005 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128791 s, 81.4 MB/s 2024-04-25T10:11:38.021 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.022 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129718 s, 80.8 MB/s 2024-04-25T10:11:38.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091831 s, 114 MB/s 2024-04-25T10:11:38.051 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.051 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.051 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132933 s, 78.9 MB/s 2024-04-25T10:11:38.066 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.066 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.066 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106389 s, 98.6 MB/s 2024-04-25T10:11:38.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.084 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143103 s, 73.3 MB/s 2024-04-25T10:11:38.100 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.101 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131033 s, 80.0 MB/s 2024-04-25T10:11:38.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.114 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108927 s, 96.3 MB/s 2024-04-25T10:11:38.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.133 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141156 s, 74.3 MB/s 2024-04-25T10:11:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122036 s, 85.9 MB/s 2024-04-25T10:11:38.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.165 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126749 s, 82.7 MB/s 2024-04-25T10:11:38.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.186 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165451 s, 63.4 MB/s 2024-04-25T10:11:38.204 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.204 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.204 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141896 s, 73.9 MB/s 2024-04-25T10:11:38.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.222 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134792 s, 77.8 MB/s 2024-04-25T10:11:38.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.238 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128 s, 81.9 MB/s 2024-04-25T10:11:38.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.257 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146361 s, 71.6 MB/s 2024-04-25T10:11:38.271 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.272 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104246 s, 101 MB/s 2024-04-25T10:11:38.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.288 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129949 s, 80.7 MB/s 2024-04-25T10:11:38.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.304 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123942 s, 84.6 MB/s 2024-04-25T10:11:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121644 s, 86.2 MB/s 2024-04-25T10:11:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.336 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122948 s, 85.3 MB/s 2024-04-25T10:11:38.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.397 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152079 s, 68.9 MB/s 2024-04-25T10:11:38.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.413 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121883 s, 86.0 MB/s 2024-04-25T10:11:38.431 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.432 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.432 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138645 s, 75.6 MB/s 2024-04-25T10:11:38.451 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.451 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.451 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148029 s, 70.8 MB/s 2024-04-25T10:11:38.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:38.452 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:38.452 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:38.452 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:11:38.452 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:38.452 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:38.452 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:38.452 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:38.523 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.523 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.523 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126339 s, 83.0 MB/s 2024-04-25T10:11:38.538 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.538 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.538 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113918 s, 92.0 MB/s 2024-04-25T10:11:38.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.552 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107635 s, 97.4 MB/s 2024-04-25T10:11:38.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.569 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131159 s, 79.9 MB/s 2024-04-25T10:11:38.586 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.586 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.586 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135539 s, 77.4 MB/s 2024-04-25T10:11:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138699 s, 75.6 MB/s 2024-04-25T10:11:38.621 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.621 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.622 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138699 s, 75.6 MB/s 2024-04-25T10:11:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129145 s, 81.2 MB/s 2024-04-25T10:11:38.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.655 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134728 s, 77.8 MB/s 2024-04-25T10:11:38.673 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.673 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.673 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138414 s, 75.8 MB/s 2024-04-25T10:11:38.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.689 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115872 s, 90.5 MB/s 2024-04-25T10:11:38.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.703 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010642 s, 98.5 MB/s 2024-04-25T10:11:38.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.719 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012077 s, 86.8 MB/s 2024-04-25T10:11:38.733 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.733 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.733 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104769 s, 100 MB/s 2024-04-25T10:11:38.749 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.749 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.749 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123834 s, 84.7 MB/s 2024-04-25T10:11:38.765 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012766 s, 82.1 MB/s 2024-04-25T10:11:38.782 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.782 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.783 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132325 s, 79.2 MB/s 2024-04-25T10:11:38.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.800 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139115 s, 75.4 MB/s 2024-04-25T10:11:38.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.815 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011694 s, 89.7 MB/s 2024-04-25T10:11:38.833 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.833 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.833 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139441 s, 75.2 MB/s 2024-04-25T10:11:38.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.849 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126936 s, 82.6 MB/s 2024-04-25T10:11:38.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.867 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143926 s, 72.9 MB/s 2024-04-25T10:11:38.881 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.881 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.881 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110423 s, 95.0 MB/s 2024-04-25T10:11:38.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.898 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127292 s, 82.4 MB/s 2024-04-25T10:11:38.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.916 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01487 s, 70.5 MB/s 2024-04-25T10:11:38.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.933 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130765 s, 80.2 MB/s 2024-04-25T10:11:38.950 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.950 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.950 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129857 s, 80.7 MB/s 2024-04-25T10:11:38.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.968 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146973 s, 71.3 MB/s 2024-04-25T10:11:38.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:38.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:38.986 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014172 s, 74.0 MB/s 2024-04-25T10:11:39.003 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.003 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.003 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131666 s, 79.6 MB/s 2024-04-25T10:11:39.018 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.018 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.018 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108769 s, 96.4 MB/s 2024-04-25T10:11:39.033 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.033 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.033 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115177 s, 91.0 MB/s 2024-04-25T10:11:39.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:39.034 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:39.034 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:39.034 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:11:39.034 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:39.034 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:39.034 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:39.035 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:39.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.107 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131151 s, 80.0 MB/s 2024-04-25T10:11:39.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.124 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136414 s, 76.9 MB/s 2024-04-25T10:11:39.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.141 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133658 s, 78.5 MB/s 2024-04-25T10:11:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.157 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125141 s, 83.8 MB/s 2024-04-25T10:11:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.170 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937699 s, 112 MB/s 2024-04-25T10:11:39.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.187 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.187 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130049 s, 80.6 MB/s 2024-04-25T10:11:39.202 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.202 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.202 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123379 s, 85.0 MB/s 2024-04-25T10:11:39.217 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.217 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.218 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118391 s, 88.6 MB/s 2024-04-25T10:11:39.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.232 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109733 s, 95.6 MB/s 2024-04-25T10:11:39.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.246 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986222 s, 106 MB/s 2024-04-25T10:11:39.260 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.260 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.260 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107744 s, 97.3 MB/s 2024-04-25T10:11:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124207 s, 84.4 MB/s 2024-04-25T10:11:39.292 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.292 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.292 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125668 s, 83.4 MB/s 2024-04-25T10:11:39.310 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.310 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.310 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144738 s, 72.4 MB/s 2024-04-25T10:11:39.328 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.328 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.328 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014041 s, 74.7 MB/s 2024-04-25T10:11:39.347 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.347 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.347 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147917 s, 70.9 MB/s 2024-04-25T10:11:39.361 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.361 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.361 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107641 s, 97.4 MB/s 2024-04-25T10:11:39.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.377 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125678 s, 83.4 MB/s 2024-04-25T10:11:39.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.392 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116286 s, 90.2 MB/s 2024-04-25T10:11:39.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.407 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105388 s, 99.5 MB/s 2024-04-25T10:11:39.424 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.425 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.425 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147946 s, 70.9 MB/s 2024-04-25T10:11:39.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.441 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127932 s, 82.0 MB/s 2024-04-25T10:11:39.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.455 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106386 s, 98.6 MB/s 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.469 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.469 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994375 s, 105 MB/s 2024-04-25T10:11:39.483 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.483 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.483 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107831 s, 97.2 MB/s 2024-04-25T10:11:39.499 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.499 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.499 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127378 s, 82.3 MB/s 2024-04-25T10:11:39.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.515 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123589 s, 84.8 MB/s 2024-04-25T10:11:39.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.533 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141367 s, 74.2 MB/s 2024-04-25T10:11:39.549 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.549 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.549 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119103 s, 88.0 MB/s 2024-04-25T10:11:39.565 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.565 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.565 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125281 s, 83.7 MB/s 2024-04-25T10:11:39.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.584 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143762 s, 72.9 MB/s 2024-04-25T10:11:39.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.599 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114939 s, 91.2 MB/s 2024-04-25T10:11:39.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:39.600 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:39.600 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:39.600 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:11:39.600 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:39.601 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:39.601 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:39.601 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:39.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.670 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114604 s, 91.5 MB/s 2024-04-25T10:11:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102141 s, 103 MB/s 2024-04-25T10:11:39.702 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.702 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.702 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149515 s, 70.1 MB/s 2024-04-25T10:11:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.718 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122062 s, 85.9 MB/s 2024-04-25T10:11:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.734 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122673 s, 85.5 MB/s 2024-04-25T10:11:39.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.754 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156991 s, 66.8 MB/s 2024-04-25T10:11:39.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.769 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115386 s, 90.9 MB/s 2024-04-25T10:11:39.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.787 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136008 s, 77.1 MB/s 2024-04-25T10:11:39.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.804 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137268 s, 76.4 MB/s 2024-04-25T10:11:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013397 s, 78.3 MB/s 2024-04-25T10:11:39.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.841 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149268 s, 70.2 MB/s 2024-04-25T10:11:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135311 s, 77.5 MB/s 2024-04-25T10:11:39.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.874 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121695 s, 86.2 MB/s 2024-04-25T10:11:39.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.889 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109722 s, 95.6 MB/s 2024-04-25T10:11:39.905 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.905 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.905 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129628 s, 80.9 MB/s 2024-04-25T10:11:39.923 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.923 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.923 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138894 s, 75.5 MB/s 2024-04-25T10:11:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137873 s, 76.1 MB/s 2024-04-25T10:11:39.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.957 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134221 s, 78.1 MB/s 2024-04-25T10:11:39.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.975 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142768 s, 73.4 MB/s 2024-04-25T10:11:39.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:39.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:39.993 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138951 s, 75.5 MB/s 2024-04-25T10:11:40.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.007 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106328 s, 98.6 MB/s 2024-04-25T10:11:40.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.025 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136057 s, 77.1 MB/s 2024-04-25T10:11:40.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.040 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.040 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110881 s, 94.6 MB/s 2024-04-25T10:11:40.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.058 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146562 s, 71.5 MB/s 2024-04-25T10:11:40.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.073 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111292 s, 94.2 MB/s 2024-04-25T10:11:40.089 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.089 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.089 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124408 s, 84.3 MB/s 2024-04-25T10:11:40.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.107 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139488 s, 75.2 MB/s 2024-04-25T10:11:40.125 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.125 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.126 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139908 s, 74.9 MB/s 2024-04-25T10:11:40.143 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.143 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.143 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134888 s, 77.7 MB/s 2024-04-25T10:11:40.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.162 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145653 s, 72.0 MB/s 2024-04-25T10:11:40.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.179 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127255 s, 82.4 MB/s 2024-04-25T10:11:40.197 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.197 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.197 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135768 s, 77.2 MB/s 2024-04-25T10:11:40.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:40.198 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:40.198 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:40.198 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:11:40.198 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:40.198 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:40.198 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:40.198 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:40.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.272 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131755 s, 79.6 MB/s 2024-04-25T10:11:40.289 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.289 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.289 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134002 s, 78.3 MB/s 2024-04-25T10:11:40.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.306 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131318 s, 79.9 MB/s 2024-04-25T10:11:40.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.322 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126751 s, 82.7 MB/s 2024-04-25T10:11:40.340 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.340 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.340 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143563 s, 73.0 MB/s 2024-04-25T10:11:40.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.423 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.423 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01948 s, 53.8 MB/s 2024-04-25T10:11:40.443 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.443 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.443 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157733 s, 66.5 MB/s 2024-04-25T10:11:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.464 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152282 s, 68.9 MB/s 2024-04-25T10:11:40.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.482 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013058 s, 80.3 MB/s 2024-04-25T10:11:40.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.499 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.499 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133529 s, 78.5 MB/s 2024-04-25T10:11:40.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.515 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123688 s, 84.8 MB/s 2024-04-25T10:11:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.534 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144201 s, 72.7 MB/s 2024-04-25T10:11:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.551 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135109 s, 77.6 MB/s 2024-04-25T10:11:40.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.570 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143466 s, 73.1 MB/s 2024-04-25T10:11:40.588 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.588 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.588 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014274 s, 73.5 MB/s 2024-04-25T10:11:40.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.606 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134297 s, 78.1 MB/s 2024-04-25T10:11:40.624 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143716 s, 73.0 MB/s 2024-04-25T10:11:40.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.643 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138813 s, 75.5 MB/s 2024-04-25T10:11:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140879 s, 74.4 MB/s 2024-04-25T10:11:40.678 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.679 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124784 s, 84.0 MB/s 2024-04-25T10:11:40.697 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.697 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.697 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134864 s, 77.8 MB/s 2024-04-25T10:11:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.715 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145001 s, 72.3 MB/s 2024-04-25T10:11:40.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.734 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144551 s, 72.5 MB/s 2024-04-25T10:11:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.752 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141529 s, 74.1 MB/s 2024-04-25T10:11:40.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.768 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119376 s, 87.8 MB/s 2024-04-25T10:11:40.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.785 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137633 s, 76.2 MB/s 2024-04-25T10:11:40.802 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.805 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129004 s, 81.3 MB/s 2024-04-25T10:11:40.818 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.819 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.819 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122637 s, 85.5 MB/s 2024-04-25T10:11:40.837 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.837 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.837 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013782 s, 76.1 MB/s 2024-04-25T10:11:40.854 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.854 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.854 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136305 s, 76.9 MB/s 2024-04-25T10:11:40.872 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.872 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.872 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143099 s, 73.3 MB/s 2024-04-25T10:11:40.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.889 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130466 s, 80.4 MB/s 2024-04-25T10:11:40.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:40.890 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:40.890 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:40.891 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:11:40.891 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:40.891 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:40.891 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:40.891 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:40.965 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.965 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.965 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137507 s, 76.3 MB/s 2024-04-25T10:11:40.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.981 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123444 s, 84.9 MB/s 2024-04-25T10:11:40.997 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:40.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:40.998 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125748 s, 83.4 MB/s 2024-04-25T10:11:41.015 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.015 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.015 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139672 s, 75.1 MB/s 2024-04-25T10:11:41.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.030 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105135 s, 99.7 MB/s 2024-04-25T10:11:41.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.043 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010255 s, 102 MB/s 2024-04-25T10:11:41.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.058 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110702 s, 94.7 MB/s 2024-04-25T10:11:41.074 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.075 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.075 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132954 s, 78.9 MB/s 2024-04-25T10:11:41.089 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.089 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.090 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112663 s, 93.1 MB/s 2024-04-25T10:11:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104946 s, 99.9 MB/s 2024-04-25T10:11:41.115 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.116 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867696 s, 121 MB/s 2024-04-25T10:11:41.130 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.130 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.130 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113961 s, 92.0 MB/s 2024-04-25T10:11:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.143 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.143 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892991 s, 117 MB/s 2024-04-25T10:11:41.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.160 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136115 s, 77.0 MB/s 2024-04-25T10:11:41.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.174 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108742 s, 96.4 MB/s 2024-04-25T10:11:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142221 s, 73.7 MB/s 2024-04-25T10:11:41.207 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.208 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125064 s, 83.8 MB/s 2024-04-25T10:11:41.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.223 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114524 s, 91.6 MB/s 2024-04-25T10:11:41.235 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.235 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.235 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925908 s, 113 MB/s 2024-04-25T10:11:41.250 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.251 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.251 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116683 s, 89.9 MB/s 2024-04-25T10:11:41.265 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.266 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.266 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113052 s, 92.8 MB/s 2024-04-25T10:11:41.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.284 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147253 s, 71.2 MB/s 2024-04-25T10:11:41.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.300 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120684 s, 86.9 MB/s 2024-04-25T10:11:41.316 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.316 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.317 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127474 s, 82.3 MB/s 2024-04-25T10:11:41.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.332 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116878 s, 89.7 MB/s 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150065 s, 69.9 MB/s 2024-04-25T10:11:41.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.398 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106399 s, 98.6 MB/s 2024-04-25T10:11:41.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.415 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126838 s, 82.7 MB/s 2024-04-25T10:11:41.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.433 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141192 s, 74.3 MB/s 2024-04-25T10:11:41.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.449 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118599 s, 88.4 MB/s 2024-04-25T10:11:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01234 s, 85.0 MB/s 2024-04-25T10:11:41.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.479 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108534 s, 96.6 MB/s 2024-04-25T10:11:41.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:41.481 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:41.481 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:41.481 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:11:41.481 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:41.481 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:41.481 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:41.481 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:41.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.547 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985402 s, 106 MB/s 2024-04-25T10:11:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123577 s, 84.9 MB/s 2024-04-25T10:11:41.577 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.578 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117608 s, 89.2 MB/s 2024-04-25T10:11:41.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.592 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106704 s, 98.3 MB/s 2024-04-25T10:11:41.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.606 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111543 s, 94.0 MB/s 2024-04-25T10:11:41.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.620 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105666 s, 99.2 MB/s 2024-04-25T10:11:41.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.637 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133514 s, 78.5 MB/s 2024-04-25T10:11:41.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.650 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931505 s, 113 MB/s 2024-04-25T10:11:41.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.665 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117483 s, 89.3 MB/s 2024-04-25T10:11:41.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.679 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010754 s, 97.5 MB/s 2024-04-25T10:11:41.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.695 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125344 s, 83.7 MB/s 2024-04-25T10:11:41.712 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.712 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.712 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131917 s, 79.5 MB/s 2024-04-25T10:11:41.727 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.728 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.728 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124219 s, 84.4 MB/s 2024-04-25T10:11:41.740 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.740 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.741 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948282 s, 111 MB/s 2024-04-25T10:11:41.758 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.758 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.758 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135257 s, 77.5 MB/s 2024-04-25T10:11:41.772 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.772 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.772 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106678 s, 98.3 MB/s 2024-04-25T10:11:41.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.789 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130571 s, 80.3 MB/s 2024-04-25T10:11:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.807 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145524 s, 72.1 MB/s 2024-04-25T10:11:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.824 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130307 s, 80.5 MB/s 2024-04-25T10:11:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.843 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142895 s, 73.4 MB/s 2024-04-25T10:11:41.860 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.860 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.860 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014311 s, 73.3 MB/s 2024-04-25T10:11:41.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.875 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108551 s, 96.6 MB/s 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01314 s, 79.8 MB/s 2024-04-25T10:11:41.908 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.908 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.908 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129307 s, 81.1 MB/s 2024-04-25T10:11:41.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.922 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102662 s, 102 MB/s 2024-04-25T10:11:41.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.937 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106616 s, 98.4 MB/s 2024-04-25T10:11:41.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.954 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.954 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135044 s, 77.6 MB/s 2024-04-25T10:11:41.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.970 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121582 s, 86.2 MB/s 2024-04-25T10:11:41.987 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:41.987 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:41.987 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138246 s, 75.8 MB/s 2024-04-25T10:11:42.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.004 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134548 s, 77.9 MB/s 2024-04-25T10:11:42.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.020 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110643 s, 94.8 MB/s 2024-04-25T10:11:42.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.034 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104671 s, 100 MB/s 2024-04-25T10:11:42.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:42.035 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:42.037 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:42.037 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:11:42.037 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:42.037 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:42.037 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:42.037 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:42.283 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.284 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153221 s, 68.4 MB/s 2024-04-25T10:11:42.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.304 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159185 s, 65.9 MB/s 2024-04-25T10:11:42.323 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.323 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.323 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148452 s, 70.6 MB/s 2024-04-25T10:11:42.353 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.353 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.353 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254657 s, 41.2 MB/s 2024-04-25T10:11:42.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.409 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155097 s, 67.6 MB/s 2024-04-25T10:11:42.430 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.430 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.430 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156945 s, 66.8 MB/s 2024-04-25T10:11:42.447 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.448 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.448 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01352 s, 77.6 MB/s 2024-04-25T10:11:42.466 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.466 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.466 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138034 s, 76.0 MB/s 2024-04-25T10:11:42.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.484 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144699 s, 72.5 MB/s 2024-04-25T10:11:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143953 s, 72.8 MB/s 2024-04-25T10:11:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113636 s, 92.3 MB/s 2024-04-25T10:11:42.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.536 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136988 s, 76.5 MB/s 2024-04-25T10:11:42.555 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.555 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.555 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143451 s, 73.1 MB/s 2024-04-25T10:11:42.572 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.572 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.572 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012829 s, 81.7 MB/s 2024-04-25T10:11:42.589 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.589 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.589 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132768 s, 79.0 MB/s 2024-04-25T10:11:42.603 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.603 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010693 s, 98.1 MB/s 2024-04-25T10:11:42.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.620 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125131 s, 83.8 MB/s 2024-04-25T10:11:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139109 s, 75.4 MB/s 2024-04-25T10:11:42.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.655 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138925 s, 75.5 MB/s 2024-04-25T10:11:42.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.671 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121237 s, 86.5 MB/s 2024-04-25T10:11:42.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.689 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136797 s, 76.7 MB/s 2024-04-25T10:11:42.705 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.705 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.705 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117196 s, 89.5 MB/s 2024-04-25T10:11:42.721 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.721 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012741 s, 82.3 MB/s 2024-04-25T10:11:42.736 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.736 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.736 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01109 s, 94.6 MB/s 2024-04-25T10:11:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109961 s, 95.4 MB/s 2024-04-25T10:11:42.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.768 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131113 s, 80.0 MB/s 2024-04-25T10:11:42.783 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.783 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.783 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011041 s, 95.0 MB/s 2024-04-25T10:11:42.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.799 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120878 s, 86.7 MB/s 2024-04-25T10:11:42.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.815 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131311 s, 79.9 MB/s 2024-04-25T10:11:42.833 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.833 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.833 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138484 s, 75.7 MB/s 2024-04-25T10:11:42.848 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.848 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.848 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011973 s, 87.6 MB/s 2024-04-25T10:11:42.863 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.863 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.863 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108424 s, 96.7 MB/s 2024-04-25T10:11:42.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:42.865 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:42.865 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:42.865 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:11:42.865 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:42.865 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:42.865 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:42.865 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:42.893 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.893 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.893 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134762 s, 77.8 MB/s 2024-04-25T10:11:42.910 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.910 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.910 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128301 s, 81.7 MB/s 2024-04-25T10:11:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.927 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135495 s, 77.4 MB/s 2024-04-25T10:11:42.941 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.941 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.941 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107994 s, 97.1 MB/s 2024-04-25T10:11:42.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.958 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127577 s, 82.2 MB/s 2024-04-25T10:11:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139217 s, 75.3 MB/s 2024-04-25T10:11:42.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:42.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:42.991 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120688 s, 86.9 MB/s 2024-04-25T10:11:43.008 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.009 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.009 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133999 s, 78.3 MB/s 2024-04-25T10:11:43.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.025 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128025 s, 81.9 MB/s 2024-04-25T10:11:43.044 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.044 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.044 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140837 s, 74.5 MB/s 2024-04-25T10:11:43.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.062 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142212 s, 73.7 MB/s 2024-04-25T10:11:43.080 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.080 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.080 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141235 s, 74.2 MB/s 2024-04-25T10:11:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148944 s, 70.4 MB/s 2024-04-25T10:11:43.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.116 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127627 s, 82.2 MB/s 2024-04-25T10:11:43.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.133 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128702 s, 81.5 MB/s 2024-04-25T10:11:43.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129743 s, 80.8 MB/s 2024-04-25T10:11:43.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.166 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119933 s, 87.4 MB/s 2024-04-25T10:11:43.183 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.183 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.183 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125874 s, 83.3 MB/s 2024-04-25T10:11:43.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.201 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137932 s, 76.0 MB/s 2024-04-25T10:11:43.218 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.218 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.218 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131252 s, 79.9 MB/s 2024-04-25T10:11:43.234 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.234 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.235 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126668 s, 82.8 MB/s 2024-04-25T10:11:43.251 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.251 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.251 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132428 s, 79.2 MB/s 2024-04-25T10:11:43.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.270 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146924 s, 71.4 MB/s 2024-04-25T10:11:43.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.287 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132644 s, 79.1 MB/s 2024-04-25T10:11:43.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.303 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128572 s, 81.6 MB/s 2024-04-25T10:11:43.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.321 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140896 s, 74.4 MB/s 2024-04-25T10:11:43.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.341 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151628 s, 69.2 MB/s 2024-04-25T10:11:43.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.360 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151212 s, 69.3 MB/s 2024-04-25T10:11:43.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.439 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.439 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155157 s, 67.6 MB/s 2024-04-25T10:11:43.458 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.458 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.459 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151174 s, 69.4 MB/s 2024-04-25T10:11:43.476 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.476 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.476 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131798 s, 79.6 MB/s 2024-04-25T10:11:43.493 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.493 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.493 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127164 s, 82.5 MB/s 2024-04-25T10:11:43.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:43.495 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:43.495 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:11:43.495 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:11:43.495 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:43.495 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:43.495 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:11:43.495 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:11:43.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.571 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145633 s, 72.0 MB/s 2024-04-25T10:11:43.588 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.588 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.588 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140083 s, 74.9 MB/s 2024-04-25T10:11:43.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115941 s, 90.4 MB/s 2024-04-25T10:11:43.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.622 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141378 s, 74.2 MB/s 2024-04-25T10:11:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105572 s, 99.3 MB/s 2024-04-25T10:11:43.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.649 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929478 s, 113 MB/s 2024-04-25T10:11:43.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.666 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128402 s, 81.7 MB/s 2024-04-25T10:11:43.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.681 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118083 s, 88.8 MB/s 2024-04-25T10:11:43.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.698 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129512 s, 81.0 MB/s 2024-04-25T10:11:43.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.715 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139797 s, 75.0 MB/s 2024-04-25T10:11:43.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.730 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107471 s, 97.6 MB/s 2024-04-25T10:11:43.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.745 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112466 s, 93.2 MB/s 2024-04-25T10:11:43.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.761 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124516 s, 84.2 MB/s 2024-04-25T10:11:43.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.775 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107946 s, 97.1 MB/s 2024-04-25T10:11:43.793 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.793 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.793 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139783 s, 75.0 MB/s 2024-04-25T10:11:43.811 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.811 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.811 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139727 s, 75.0 MB/s 2024-04-25T10:11:43.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.826 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.826 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111439 s, 94.1 MB/s 2024-04-25T10:11:43.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.840 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108957 s, 96.2 MB/s 2024-04-25T10:11:43.859 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.859 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.859 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145042 s, 72.3 MB/s 2024-04-25T10:11:43.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.873 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110825 s, 94.6 MB/s 2024-04-25T10:11:43.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.890 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013141 s, 79.8 MB/s 2024-04-25T10:11:43.906 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.906 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.906 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122097 s, 85.9 MB/s 2024-04-25T10:11:43.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.922 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118569 s, 88.4 MB/s 2024-04-25T10:11:43.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.936 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101168 s, 104 MB/s 2024-04-25T10:11:43.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.952 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128219 s, 81.8 MB/s 2024-04-25T10:11:43.967 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.968 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120555 s, 87.0 MB/s 2024-04-25T10:11:43.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.983 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122616 s, 85.5 MB/s 2024-04-25T10:11:43.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:43.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:43.999 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120894 s, 86.7 MB/s 2024-04-25T10:11:44.016 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:44.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:44.017 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137052 s, 76.5 MB/s 2024-04-25T10:11:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:44.031 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108544 s, 96.6 MB/s 2024-04-25T10:11:44.046 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:44.046 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:44.046 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117467 s, 89.3 MB/s 2024-04-25T10:11:44.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:11:44.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:11:44.060 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0096344 s, 109 MB/s 2024-04-25T10:11:44.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:44.062 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:11:44.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc4519640 1 Processor -- start 2024-04-25T10:11:44.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc4519640 1 -- start start 2024-04-25T10:11:44.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc4519640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc15b530 0x7fdcbc15b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc4519640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcbc15bf40 con 0x7fdcbc0cc890 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc4519640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcbc15c200 con 0x7fdcbc0ceda0 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc4519640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc1589c0 con 0x7fdcbc15b530 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc1ab4640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdcbc0cc890 0x7fdcbc0ce690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42626/0 (socket says 172.21.15.133:42626) 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc1ab4640 1 -- 172.21.15.133:0/3324130568 learned_addr learned my addr 172.21.15.133:0/3324130568 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc22b5640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc15b530 0x7fdcbc15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420957648 0 0) 0x7fdcbc15bf40 con 0x7fdcbc0cc890 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdca8003530 con 0x7fdcbc0cc890 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3029442839 0 0) 0x7fdca8003530 con 0x7fdcbc0cc890 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc15b530 msgr2=0x7fdcbc15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc15b530 0x7fdcbc15b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 >> v1:172.21.15.162:6789/0 conn(0x7fdcbc0ceda0 legacy=0x7fdcbc15cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc15c4e0 con 0x7fdcbc0cc890 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fdcac001680 con 0x7fdcbc0cc890 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 >> v1:172.21.15.133:6789/0 conn(0x7fdcbc0cc890 legacy=0x7fdcbc0ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc15b530 0x7fdca80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca8003f70 0x7fdca8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca8004d50 0x7fdca80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca800bb60 con 0x7fdca8004d50 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca800bce0 con 0x7fdca8003f70 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca800e0f0 con 0x7fdcbc15b530 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca8004d50 0x7fdca80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc1ab4640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca8003f70 0x7fdca8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc22b5640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc15b530 0x7fdca80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/3324130568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc15b530 msgr2=0x7fdca80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc15b530 0x7fdca80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/3324130568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca8003f70 msgr2=0x7fdca8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca8003f70 0x7fdca8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.229+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/3324130568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca8003530 con 0x7fdca8004d50 2024-04-25T10:11:44.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.233+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca8004d50 0x7fdca80051d0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4001e70 tx=0x7fdcb401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9f91fb8dd6a375 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.233+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4001a90 con 0x7fdca8004d50 2024-04-25T10:11:44.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.233+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4003b80 con 0x7fdca8004d50 2024-04-25T10:11:44.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.233+0000 7fdcc12b3640 1 -- 172.21.15.133:0/3324130568 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4005080 con 0x7fdca8004d50 2024-04-25T10:11:44.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.233+0000 7fdcc4519640 1 -- 172.21.15.133:0/3324130568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca8004d50 msgr2=0x7fdca80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.233+0000 7fdcc4519640 1 --2- 172.21.15.133:0/3324130568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca8004d50 0x7fdca80051d0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4001e70 tx=0x7fdcb401c780 comp rx=0 tx=0).stop 2024-04-25T10:11:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.233+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/3324130568 reap_dead start 2024-04-25T10:11:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.233+0000 7fdcc4519640 1 -- 172.21.15.133:0/3324130568 shutdown_connections 2024-04-25T10:11:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.233+0000 7fdcc4519640 1 -- 172.21.15.133:0/3324130568 >> 172.21.15.133:0/3324130568 conn(0x7fdcbc06d370 msgr2=0x7fdcbc06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.233+0000 7fdcc4519640 1 -- 172.21.15.133:0/3324130568 shutdown_connections 2024-04-25T10:11:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.233+0000 7fdcc4519640 1 -- 172.21.15.133:0/3324130568 wait complete. 2024-04-25T10:11:44.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc4519640 1 Processor -- start 2024-04-25T10:11:44.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc4519640 1 -- start start 2024-04-25T10:11:44.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc4519640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcbc14e040 0x7fdcbc14f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc4519640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc150c50 0x7fdcbc1ef140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc4519640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcbc0b47c0 0x7fdcbc0b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc4519640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0cce90 con 0x7fdcbc14e040 2024-04-25T10:11:44.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc4519640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0ccd10 con 0x7fdcbc0b47c0 2024-04-25T10:11:44.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc4519640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0cd010 con 0x7fdcbc150c50 2024-04-25T10:11:44.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc22b5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcbc14e040 0x7fdcbc14f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc22b5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcbc14e040 0x7fdcbc14f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43204/0 (socket says 172.21.15.133:43204) 2024-04-25T10:11:44.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc22b5640 1 -- 172.21.15.133:0/2734872614 learned_addr learned my addr 172.21.15.133:0/2734872614 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcbc0b47c0 0x7fdcbc0b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc1ab4640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc150c50 0x7fdcbc1ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc22b5640 1 -- 172.21.15.133:0/2734872614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc150c50 msgr2=0x7fdcbc1ef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc22b5640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc150c50 0x7fdcbc1ef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc22b5640 1 -- 172.21.15.133:0/2734872614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcbc0b47c0 msgr2=0x7fdcbc0b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc22b5640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcbc0b47c0 0x7fdcbc0b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc22b5640 1 -- 172.21.15.133:0/2734872614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc1510d0 con 0x7fdcbc14e040 2024-04-25T10:11:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc1ab4640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc150c50 0x7fdcbc1ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc22b5640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcbc14e040 0x7fdcbc14f560 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8011610 tx=0x7fdcb8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcb2ffd640 1 -- 172.21.15.133:0/2734872614 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8015870 con 0x7fdcbc14e040 2024-04-25T10:11:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcb2ffd640 1 -- 172.21.15.133:0/2734872614 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb8015a10 con 0x7fdcbc14e040 2024-04-25T10:11:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcb2ffd640 1 -- 172.21.15.133:0/2734872614 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8015dc0 con 0x7fdcbc14e040 2024-04-25T10:11:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc4519640 1 -- 172.21.15.133:0/2734872614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcbc0b25c0 con 0x7fdcbc14e040 2024-04-25T10:11:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.237+0000 7fdcc4519640 1 -- 172.21.15.133:0/2734872614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcbc0b2b80 con 0x7fdcbc14e040 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 Processor -- start 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 -- start start 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94e3f0 0x5596cd94e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94ed10 0x5596cd904fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd905520 0x5596cd905900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cd752d50 con 0x5596cd94e3f0 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cd6dae40 con 0x5596cd905520 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596cd74edf0 con 0x5596cd94ed10 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcb2ffd640 1 -- 172.21.15.133:0/2734872614 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdcb800f480 con 0x7fdcbc14e040 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc2ab6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94e3f0 0x5596cd94e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc2ab6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94e3f0 0x5596cd94e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43206/0 (socket says 172.21.15.133:43206) 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/2103931240 learned_addr learned my addr 172.21.15.133:0/2103931240 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc1ab4640 1 --2- 172.21.15.133:0/2103931240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd905520 0x5596cd905900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/2103931240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94ed10 msgr2=0x5596cd904fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/2103931240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94ed10 0x5596cd904fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/2103931240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd905520 msgr2=0x5596cd905900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/2103931240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd905520 0x5596cd905900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/2103931240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596cd6efd90 con 0x5596cd94e3f0 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcb2ffd640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdca0066f60 0x7fdca0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/2103931240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94e3f0 0x5596cd94e7d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fdcb401c5b0 tx=0x7fdcb4001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c6c910065ba050 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcb2ffd640 1 -- 172.21.15.133:0/2734872614 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fdcb8085ba0 con 0x7fdcbc14e040 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc22b5640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdca0066f60 0x7fdca0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdc9ffff640 1 -- 172.21.15.133:0/2103931240 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4006b60 con 0x5596cd94e3f0 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdc9ffff640 1 -- 172.21.15.133:0/2103931240 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb40b6020 con 0x5596cd94e3f0 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdc9ffff640 1 -- 172.21.15.133:0/2103931240 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4003980 con 0x5596cd94e3f0 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc22b5640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdca0066f60 0x7fdca0069420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8000c10 tx=0x7fdcb8085f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 -- 172.21.15.133:0/2103931240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94e3f0 msgr2=0x5596cd94e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 --2- 172.21.15.133:0/2103931240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94e3f0 0x5596cd94e7d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fdcb401c5b0 tx=0x7fdcb4001520 comp rx=0 tx=0).stop 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 -- 172.21.15.133:0/2103931240 shutdown_connections 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 --2- 172.21.15.133:0/2103931240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd905520 0x5596cd905900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 --2- 172.21.15.133:0/2103931240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94ed10 0x5596cd904fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 --2- 172.21.15.133:0/2103931240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94e3f0 0x5596cd94e7d0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 -- 172.21.15.133:0/2103931240 >> 172.21.15.133:0/2103931240 conn(0x5596cd9014d0 msgr2=0x5596cd9038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 -- 172.21.15.133:0/2103931240 shutdown_connections 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 -- 172.21.15.133:0/2103931240 wait complete. 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 -- start start 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.241+0000 7fdcc7686000 1 Processor -- start 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd905520 0x5596cd955540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94e3f0 0x5596cd955a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd90fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cd6f8610 con 0x5596cd94ed10 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cd70d9a0 con 0x5596cd905520 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596cd757380 con 0x5596cd94e3f0 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 10 client.0 _ll_register_callbacks cb 0x5596cd8070d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc2ab6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd90fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc2ab6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd90fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43220/0 (socket says 172.21.15.133:43220) 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/4172053593 learned_addr learned my addr 172.21.15.133:0/4172053593 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc1ab4640 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94e3f0 0x5596cd955a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd905520 0x5596cd955540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94e3f0 msgr2=0x5596cd955a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94e3f0 0x5596cd955a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd905520 msgr2=0x5596cd955540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd905520 0x5596cd955540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/4172053593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596cd74dd40 con 0x5596cd94ed10 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd90fbf0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fdcb401c5b0 tx=0x7fdcb40a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79f2f09691934f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdc9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/4172053593 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4006b60 con 0x5596cd94ed10 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/4172053593 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fdcb4006d00 con 0x5596cd94ed10 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 1 -- 172.21.15.133:0/4172053593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcb40b6020 con 0x5596cd94ed10 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/4172053593 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcb40052c0 con 0x5596cd94ed10 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 10 client.8511 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 1 -- 172.21.15.133:0/4172053593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5596cd7d5f00 con 0x5596cd94ed10 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/4172053593 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb40b6020 con 0x5596cd94ed10 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/4172053593 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdcb4005460 con 0x5596cd94ed10 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 10 client.8511 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 10 client.8511 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 20 client.8511 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x5596cd910960 0x5596cd910d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 4 client.8511 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc7686000 1 -- 172.21.15.133:0/4172053593 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596cd6eea20 con 0x5596cd910960 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc1ab4640 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x5596cd910960 0x5596cd910d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdcc1ab4640 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x5596cd910960 0x5596cd910d70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50e58f9fdadbbdd4 server_cookie=b89715b82bcc7b5e in_seq=0 out_seq=0 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.245+0000 7fdc9d7fa640 10 client.8511 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/4172053593 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5596cd6eea20 con 0x5596cd910960 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdc9d7fa640 10 client.8511 handle_command_reply: tid=0 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 1 client.8511 shutdown 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 1 -- 172.21.15.133:0/4172053593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 msgr2=0x5596cd90fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd90fbf0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fdcb401c5b0 tx=0x7fdcb40a7000 comp rx=0 tx=0).stop 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 1 -- 172.21.15.133:0/4172053593 shutdown_connections 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x5596cd910960 0x5596cd910d70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd90fbf0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94e3f0 0x5596cd955a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdc9d7fa640 0 client.8511 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 1 --2- 172.21.15.133:0/4172053593 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd905520 0x5596cd955540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 1 -- 172.21.15.133:0/4172053593 >> 172.21.15.133:0/4172053593 conn(0x5596cd9014d0 msgr2=0x5596cd904150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 1 -- 172.21.15.133:0/4172053593 shutdown_connections 2024-04-25T10:11:44.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 1 -- 172.21.15.133:0/4172053593 wait complete. 2024-04-25T10:11:44.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.249+0000 7fdcc7686000 20 client.8511 trim_cache size 0 max 16384 2024-04-25T10:11:44.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.261+0000 7fdcc7686000 1 Processor -- start 2024-04-25T10:11:44.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.261+0000 7fdcc7686000 1 -- start start 2024-04-25T10:11:44.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd905520 0x5596cd9038c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd94e3f0 0x5596cd903e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd904340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc1fc700 con 0x5596cd94ed10 2024-04-25T10:11:44.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc1fb8e0 con 0x5596cd94e3f0 2024-04-25T10:11:44.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc1face0 con 0x5596cd905520 2024-04-25T10:11:44.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd904340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd904340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43230/0 (socket says 172.21.15.133:43230) 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/2334089365 learned_addr learned my addr 172.21.15.133:0/2334089365 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/2334089365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd905520 msgr2=0x5596cd9038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/2334089365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd905520 0x5596cd9038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/2334089365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd94e3f0 msgr2=0x5596cd903e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/2334089365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd94e3f0 0x5596cd903e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/2334089365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596cd7ac330 con 0x5596cd94ed10 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/2334089365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd904340 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4001640 tx=0x7fdcb4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b380c359473209b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdc9f7fe640 1 -- 172.21.15.133:0/2334089365 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4006b60 con 0x5596cd94ed10 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdc9f7fe640 1 -- 172.21.15.133:0/2334089365 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb40b6040 con 0x5596cd94ed10 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdc9f7fe640 1 -- 172.21.15.133:0/2334089365 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40a9470 con 0x5596cd94ed10 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- 172.21.15.133:0/2334089365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 msgr2=0x5596cd904340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 --2- 172.21.15.133:0/2334089365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd904340 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4001640 tx=0x7fdcb4009040 comp rx=0 tx=0).stop 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- 172.21.15.133:0/2334089365 shutdown_connections 2024-04-25T10:11:44.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 --2- 172.21.15.133:0/2334089365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd94ed10 0x5596cd904340 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 --2- 172.21.15.133:0/2334089365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd94e3f0 0x5596cd903e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 --2- 172.21.15.133:0/2334089365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd905520 0x5596cd9038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- 172.21.15.133:0/2334089365 >> 172.21.15.133:0/2334089365 conn(0x5596cd9014d0 msgr2=0x5596cd95bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- 172.21.15.133:0/2334089365 shutdown_connections 2024-04-25T10:11:44.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- 172.21.15.133:0/2334089365 wait complete. 2024-04-25T10:11:44.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- start start 2024-04-25T10:11:44.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 Processor -- start 2024-04-25T10:11:44.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd905520 0x5596cd952540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94e3f0 0x5596cd952f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd94ed10 0x5596cd9534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cd709dc0 con 0x5596cd905520 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc1fb2e0 con 0x5596cd94ed10 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596cd752060 con 0x5596cd94e3f0 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc7686000 10 client.0 _ll_register_callbacks cb 0x7fdca8003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd905520 0x5596cd952540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd905520 0x5596cd952540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43242/0 (socket says 172.21.15.133:43242) 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/289488613 learned_addr learned my addr 172.21.15.133:0/289488613 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc1ab4640 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94e3f0 0x5596cd952f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd94ed10 0x5596cd9534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94e3f0 msgr2=0x5596cd952f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94e3f0 0x5596cd952f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd94ed10 msgr2=0x5596cd9534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd94ed10 0x5596cd9534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc2ab6640 1 -- 172.21.15.133:0/289488613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596cd71f5f0 con 0x5596cd905520 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.265+0000 7fdcc1ab4640 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94e3f0 0x5596cd952f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc2ab6640 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd905520 0x5596cd952540 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40013e0 tx=0x7fdcb40bb000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5875079ef4bbf24b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdc9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/289488613 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40a9ab0 con 0x5596cd905520 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/289488613 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fdcb40a9c50 con 0x5596cd905520 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc7686000 1 -- 172.21.15.133:0/289488613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcb40b6040 con 0x5596cd905520 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/289488613 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcb40dc050 con 0x5596cd905520 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc7686000 10 client.8517 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc7686000 1 -- 172.21.15.133:0/289488613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5596cd6f9fe0 con 0x5596cd905520 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/289488613 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb40b6040 con 0x5596cd905520 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/289488613 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdcb40df020 con 0x5596cd905520 2024-04-25T10:11:44.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc7686000 10 client.8517 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:44.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc7686000 10 client.8517 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:11:44.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc7686000 20 client.8517 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:44.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc7686000 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x5596cd96c050 0x5596cd96c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc7686000 4 client.8517 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:11:44.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc7686000 1 -- 172.21.15.133:0/289488613 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5596cd760290 con 0x5596cd96c050 2024-04-25T10:11:44.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc1ab4640 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x5596cd96c050 0x5596cd96c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdc9d7fa640 10 client.8517 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.269+0000 7fdcc1ab4640 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x5596cd96c050 0x5596cd96c430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f905d97579fbce09 server_cookie=7fe5c4580be5d685 in_seq=0 out_seq=0 2024-04-25T10:11:44.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdc9d7fa640 1 -- 172.21.15.133:0/289488613 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5596cd760290 con 0x5596cd96c050 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdc9d7fa640 10 client.8517 handle_command_reply: tid=0 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 1 client.8517 shutdown 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 1 -- 172.21.15.133:0/289488613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd905520 msgr2=0x5596cd952540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd905520 0x5596cd952540 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40013e0 tx=0x7fdcb40bb000 comp rx=0 tx=0).stop 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 1 -- 172.21.15.133:0/289488613 shutdown_connections 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x5596cd96c050 0x5596cd96c430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5596cd94ed10 0x5596cd9534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdc9d7fa640 0 client.8517 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5596cd94e3f0 0x5596cd952f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 1 --2- 172.21.15.133:0/289488613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5596cd905520 0x5596cd952540 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 1 -- 172.21.15.133:0/289488613 >> 172.21.15.133:0/289488613 conn(0x5596cd9014d0 msgr2=0x5596cd94cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 1 -- 172.21.15.133:0/289488613 shutdown_connections 2024-04-25T10:11:44.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 1 -- 172.21.15.133:0/289488613 wait complete. 2024-04-25T10:11:44.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc7686000 20 client.8517 trim_cache size 0 max 16384 2024-04-25T10:11:44.279 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:11:44.279 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:11:44.279 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "3aafa117-8943-43e7-b495-998293c81538", 2024-04-25T10:11:44.279 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:11:44.279 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:11:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 -- 172.21.15.133:0/2734872614 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdca0066f60 msgr2=0x7fdca0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdca0066f60 0x7fdca0069420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8000c10 tx=0x7fdcb8085f90 comp rx=0 tx=0).stop 2024-04-25T10:11:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 -- 172.21.15.133:0/2734872614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcbc14e040 msgr2=0x7fdcbc14f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcbc14e040 0x7fdcbc14f560 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8011610 tx=0x7fdcb8011ad0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 -- 172.21.15.133:0/2734872614 shutdown_connections 2024-04-25T10:11:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcbc0b47c0 0x7fdcbc0b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcbc150c50 0x7fdcbc1ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdca0066f60 0x7fdca0069420 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 --2- 172.21.15.133:0/2734872614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcbc14e040 0x7fdcbc14f560 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 -- 172.21.15.133:0/2734872614 >> 172.21.15.133:0/2734872614 conn(0x7fdcbc06d370 msgr2=0x7fdcbc15b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 -- 172.21.15.133:0/2734872614 shutdown_connections 2024-04-25T10:11:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.273+0000 7fdcc4519640 1 -- 172.21.15.133:0/2734872614 wait complete. 2024-04-25T10:11:44.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:44.295 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T10:11:44.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.409+0000 7f024e32a640 1 Processor -- start 2024-04-25T10:11:44.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.409+0000 7f024e32a640 1 -- start start 2024-04-25T10:11:44.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f024e32a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f02480cb8d0 0x7f02480cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f024e32a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02480cd930 con 0x7f02480ceb50 2024-04-25T10:11:44.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f024e32a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02480ce4f0 con 0x7f02480b33c0 2024-04-25T10:11:44.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f024e32a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480cb180 con 0x7f02480cb8d0 2024-04-25T10:11:44.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f0247fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f02480ceb50 0x7f02480cef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42632/0 (socket says 172.21.15.133:42632) 2024-04-25T10:11:44.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f0247fff640 1 -- 172.21.15.133:0/3295603336 learned_addr learned my addr 172.21.15.133:0/3295603336 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:44.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f024c8c7640 1 --2- 172.21.15.133:0/3295603336 >> v2:172.21.15.133:3300/0 conn(0x7f02480cb8d0 0x7f02480cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f0246ffd640 1 -- 172.21.15.133:0/3295603336 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649397515 0 0) 0x7f02480cd930 con 0x7f02480ceb50 2024-04-25T10:11:44.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f0246ffd640 1 -- 172.21.15.133:0/3295603336 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0230005740 con 0x7f02480ceb50 2024-04-25T10:11:44.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f024c8c7640 1 -- 172.21.15.133:0/3295603336 >> v1:172.21.15.162:6789/0 conn(0x7f02480b33c0 legacy=0x7f02480b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f024c8c7640 1 -- 172.21.15.133:0/3295603336 >> v1:172.21.15.133:6789/0 conn(0x7f02480ceb50 legacy=0x7f02480cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f024c8c7640 1 -- 172.21.15.133:0/3295603336 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480d11e0 con 0x7f02480cb8d0 2024-04-25T10:11:44.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f024c8c7640 1 --2- 172.21.15.133:0/3295603336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02480cb8d0 0x7f02480cbd60 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f02400097f0 tx=0x7f024001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb4d8a46126a36d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f0246ffd640 1 -- 172.21.15.133:0/3295603336 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400b0070 con 0x7f02480cb8d0 2024-04-25T10:11:44.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f0246ffd640 1 -- 172.21.15.133:0/3295603336 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400ab600 con 0x7f02480cb8d0 2024-04-25T10:11:44.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.413+0000 7f0246ffd640 1 -- 172.21.15.133:0/3295603336 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400ab9b0 con 0x7f02480cb8d0 2024-04-25T10:11:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 -- 172.21.15.133:0/3295603336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02480cb8d0 msgr2=0x7f02480cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 --2- 172.21.15.133:0/3295603336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02480cb8d0 0x7f02480cbd60 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f02400097f0 tx=0x7f024001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 -- 172.21.15.133:0/3295603336 shutdown_connections 2024-04-25T10:11:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 --2- 172.21.15.133:0/3295603336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02480cb8d0 0x7f02480cbd60 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 -- 172.21.15.133:0/3295603336 >> 172.21.15.133:0/3295603336 conn(0x7f02480d0570 msgr2=0x7f02480d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 -- 172.21.15.133:0/3295603336 shutdown_connections 2024-04-25T10:11:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 -- 172.21.15.133:0/3295603336 wait complete. 2024-04-25T10:11:44.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 Processor -- start 2024-04-25T10:11:44.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 -- start start 2024-04-25T10:11:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02480b33c0 0x7f02480c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02480cb8d0 0x7f024814f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02480ceb50 0x7f02480af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02480cb440 con 0x7f02480ceb50 2024-04-25T10:11:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02480d14c0 con 0x7f02480b33c0 2024-04-25T10:11:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024e32a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024806b4a0 con 0x7f02480cb8d0 2024-04-25T10:11:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024c8c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02480ceb50 0x7f02480af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024c8c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02480ceb50 0x7f02480af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43270/0 (socket says 172.21.15.133:43270) 2024-04-25T10:11:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024c8c7640 1 -- 172.21.15.133:0/410655819 learned_addr learned my addr 172.21.15.133:0/410655819 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f02477fe640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02480cb8d0 0x7f024814f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f0247fff640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02480b33c0 0x7f02480c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024c8c7640 1 -- 172.21.15.133:0/410655819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02480cb8d0 msgr2=0x7f024814f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024c8c7640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02480cb8d0 0x7f024814f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024c8c7640 1 -- 172.21.15.133:0/410655819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02480b33c0 msgr2=0x7f02480c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024c8c7640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02480b33c0 0x7f02480c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024c8c7640 1 -- 172.21.15.133:0/410655819 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024814fda0 con 0x7f02480ceb50 2024-04-25T10:11:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f024c8c7640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02480ceb50 0x7f02480af8d0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f02400ae040 tx=0x7f024001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f0244ff9640 1 -- 172.21.15.133:0/410655819 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400bc070 con 0x7f02480ceb50 2024-04-25T10:11:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f0244ff9640 1 -- 172.21.15.133:0/410655819 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400b1040 con 0x7f02480ceb50 2024-04-25T10:11:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.417+0000 7f0244ff9640 1 -- 172.21.15.133:0/410655819 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400b0050 con 0x7f02480ceb50 2024-04-25T10:11:44.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.421+0000 7f024e32a640 1 -- 172.21.15.133:0/410655819 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0248150080 con 0x7f02480ceb50 2024-04-25T10:11:44.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.421+0000 7f024e32a640 1 -- 172.21.15.133:0/410655819 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0248150640 con 0x7f02480ceb50 2024-04-25T10:11:44.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.421+0000 7f0244ff9640 1 -- 172.21.15.133:0/410655819 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0240004230 con 0x7f02480ceb50 2024-04-25T10:11:44.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.421+0000 7f0244ff9640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0234066f60 0x7f0234069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.421+0000 7f0244ff9640 1 -- 172.21.15.133:0/410655819 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f024011ecb0 con 0x7f02480ceb50 2024-04-25T10:11:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.421+0000 7f0247fff640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0234066f60 0x7f0234069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.421+0000 7f0247fff640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0234066f60 0x7f0234069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0238001570 tx=0x7f0238009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.421+0000 7f0251497000 1 Processor -- start 2024-04-25T10:11:44.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.421+0000 7f0251497000 1 -- start start 2024-04-25T10:11:44.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c686e50 0x55fb7c687230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c687770 0x55fb7c641c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c6421a0 0x55fb7c642580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c5201f0 con 0x55fb7c686e50 2024-04-25T10:11:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c486e90 con 0x55fb7c687770 2024-04-25T10:11:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c40ef50 con 0x55fb7c6421a0 2024-04-25T10:11:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0247fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c686e50 0x55fb7c687230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0247fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c686e50 0x55fb7c687230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43284/0 (socket says 172.21.15.133:43284) 2024-04-25T10:11:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0247fff640 1 -- 172.21.15.133:0/796517856 learned_addr learned my addr 172.21.15.133:0/796517856 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f024c8c7640 1 --2- 172.21.15.133:0/796517856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c6421a0 0x55fb7c642580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f02477fe640 1 --2- 172.21.15.133:0/796517856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c687770 0x55fb7c641c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0247fff640 1 -- 172.21.15.133:0/796517856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c6421a0 msgr2=0x55fb7c642580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0247fff640 1 --2- 172.21.15.133:0/796517856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c6421a0 0x55fb7c642580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0247fff640 1 -- 172.21.15.133:0/796517856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c687770 msgr2=0x55fb7c641c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0247fff640 1 --2- 172.21.15.133:0/796517856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c687770 0x55fb7c641c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0247fff640 1 -- 172.21.15.133:0/796517856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb7c45e8b0 con 0x55fb7c686e50 2024-04-25T10:11:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0247fff640 1 --2- 172.21.15.133:0/796517856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c686e50 0x55fb7c687230 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f023800eed0 tx=0x7f0238011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1383eefc27a6879 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f022dffb640 1 -- 172.21.15.133:0/796517856 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238007ae0 con 0x55fb7c686e50 2024-04-25T10:11:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f022dffb640 1 -- 172.21.15.133:0/796517856 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0238013630 con 0x55fb7c686e50 2024-04-25T10:11:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f022dffb640 1 -- 172.21.15.133:0/796517856 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238013a00 con 0x55fb7c686e50 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- 172.21.15.133:0/796517856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c686e50 msgr2=0x55fb7c687230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 --2- 172.21.15.133:0/796517856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c686e50 0x55fb7c687230 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f023800eed0 tx=0x7f0238011000 comp rx=0 tx=0).stop 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- 172.21.15.133:0/796517856 shutdown_connections 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 --2- 172.21.15.133:0/796517856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c6421a0 0x55fb7c642580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 --2- 172.21.15.133:0/796517856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c687770 0x55fb7c641c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 --2- 172.21.15.133:0/796517856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c686e50 0x55fb7c687230 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- 172.21.15.133:0/796517856 >> 172.21.15.133:0/796517856 conn(0x55fb7c639d40 msgr2=0x55fb7c63c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- 172.21.15.133:0/796517856 shutdown_connections 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- 172.21.15.133:0/796517856 wait complete. 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- start start 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 Processor -- start 2024-04-25T10:11:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c6421a0 0x55fb7c68e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c686e50 0x55fb7c68e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c687770 0x55fb7c647dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c441540 con 0x55fb7c6421a0 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c482f30 con 0x55fb7c686e50 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.425+0000 7f0251497000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c48b4c0 con 0x55fb7c687770 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 10 client.0 _ll_register_callbacks cb 0x55fb7c53b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0247fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c6421a0 0x55fb7c68e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0247fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c6421a0 0x55fb7c68e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43294/0 (socket says 172.21.15.133:43294) 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0247fff640 1 -- 172.21.15.133:0/1515621890 learned_addr learned my addr 172.21.15.133:0/1515621890 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f024c8c7640 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c687770 0x55fb7c647dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f02477fe640 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c686e50 0x55fb7c68e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0247fff640 1 -- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c687770 msgr2=0x55fb7c647dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0247fff640 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c687770 0x55fb7c647dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0247fff640 1 -- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c686e50 msgr2=0x55fb7c68e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0247fff640 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c686e50 0x55fb7c68e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0247fff640 1 -- 172.21.15.133:0/1515621890 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb7c40f1a0 con 0x55fb7c6421a0 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f02477fe640 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c686e50 0x55fb7c68e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0247fff640 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c6421a0 0x55fb7c68e1b0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f0238007930 tx=0x7f0238013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b536b45962d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0226ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0226ffd640 1 -- 172.21.15.133:0/1515621890 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238013dd0 con 0x55fb7c6421a0 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0226ffd640 1 -- 172.21.15.133:0/1515621890 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0238010070 con 0x55fb7c6421a0 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 -- 172.21.15.133:0/1515621890 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0238013630 con 0x55fb7c6421a0 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0226ffd640 1 -- 172.21.15.133:0/1515621890 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0238004b70 con 0x55fb7c6421a0 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 10 client.8532 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 -- 172.21.15.133:0/1515621890 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55fb7c509840 con 0x55fb7c6421a0 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0226ffd640 1 -- 172.21.15.133:0/1515621890 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0238013630 con 0x55fb7c6421a0 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0226ffd640 1 -- 172.21.15.133:0/1515621890 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0238005030 con 0x55fb7c6421a0 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 10 client.8532 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:44.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 10 client.8532 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:11:44.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 20 client.8532 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:44.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fb7c648b40 0x55fb7c648f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 4 client.8532 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:44.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 -- 172.21.15.133:0/1515621890 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb7c422be0 con 0x55fb7c648b40 2024-04-25T10:11:44.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f02477fe640 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fb7c648b40 0x55fb7c648f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f02477fe640 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fb7c648b40 0x55fb7c648f20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b8b16e6a329337 server_cookie=a7616f973e473f87 in_seq=0 out_seq=0 2024-04-25T10:11:44.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0226ffd640 10 client.8532 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0226ffd640 1 -- 172.21.15.133:0/1515621890 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb7c422be0 con 0x55fb7c648b40 2024-04-25T10:11:44.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0226ffd640 10 client.8532 handle_command_reply: tid=0 2024-04-25T10:11:44.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 client.8532 shutdown 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 -- 172.21.15.133:0/1515621890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c6421a0 msgr2=0x55fb7c68e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c6421a0 0x55fb7c68e1b0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f0238007930 tx=0x7f0238013000 comp rx=0 tx=0).stop 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 -- 172.21.15.133:0/1515621890 shutdown_connections 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fb7c648b40 0x55fb7c648f20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c687770 0x55fb7c647dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c686e50 0x55fb7c68e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0226ffd640 0 client.8532 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 --2- 172.21.15.133:0/1515621890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c6421a0 0x55fb7c68e1b0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 -- 172.21.15.133:0/1515621890 >> 172.21.15.133:0/1515621890 conn(0x55fb7c639d40 msgr2=0x55fb7c63c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 -- 172.21.15.133:0/1515621890 shutdown_connections 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 1 -- 172.21.15.133:0/1515621890 wait complete. 2024-04-25T10:11:44.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.429+0000 7f0251497000 20 client.8532 trim_cache size 0 max 16384 2024-04-25T10:11:44.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 Processor -- start 2024-04-25T10:11:44.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 -- start start 2024-04-25T10:11:44.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c6421a0 0x55fb7c6e3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c686e50 0x55fb7c6e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6e3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02481ff300 con 0x55fb7c687770 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02481fe520 con 0x55fb7c6421a0 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024814eeb0 con 0x55fb7c686e50 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f024c8c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6e3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f024c8c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6e3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43308/0 (socket says 172.21.15.133:43308) 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f024c8c7640 1 -- 172.21.15.133:0/936884868 learned_addr learned my addr 172.21.15.133:0/936884868 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f02477fe640 1 --2- 172.21.15.133:0/936884868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c686e50 0x55fb7c6e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0247fff640 1 --2- 172.21.15.133:0/936884868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c6421a0 0x55fb7c6e3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f024c8c7640 1 -- 172.21.15.133:0/936884868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c686e50 msgr2=0x55fb7c6e37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f024c8c7640 1 --2- 172.21.15.133:0/936884868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c686e50 0x55fb7c6e37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f024c8c7640 1 -- 172.21.15.133:0/936884868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c6421a0 msgr2=0x55fb7c6e3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f024c8c7640 1 --2- 172.21.15.133:0/936884868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c6421a0 0x55fb7c6e3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f024c8c7640 1 -- 172.21.15.133:0/936884868 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb7c6875a0 con 0x55fb7c687770 2024-04-25T10:11:44.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f024c8c7640 1 --2- 172.21.15.133:0/936884868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6e3d00 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f02400eb7c0 tx=0x7f02400ef1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f346ee9da0415d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f022d7fa640 1 -- 172.21.15.133:0/936884868 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240006e20 con 0x55fb7c687770 2024-04-25T10:11:44.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f022d7fa640 1 -- 172.21.15.133:0/936884868 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400ab600 con 0x55fb7c687770 2024-04-25T10:11:44.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f022d7fa640 1 -- 172.21.15.133:0/936884868 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400efa60 con 0x55fb7c687770 2024-04-25T10:11:44.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 -- 172.21.15.133:0/936884868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 msgr2=0x55fb7c6e3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 --2- 172.21.15.133:0/936884868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6e3d00 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f02400eb7c0 tx=0x7f02400ef1f0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 -- 172.21.15.133:0/936884868 shutdown_connections 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 --2- 172.21.15.133:0/936884868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6e3d00 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 --2- 172.21.15.133:0/936884868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c686e50 0x55fb7c6e37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 --2- 172.21.15.133:0/936884868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c6421a0 0x55fb7c6e3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 -- 172.21.15.133:0/936884868 >> 172.21.15.133:0/936884868 conn(0x55fb7c639d40 msgr2=0x55fb7c63c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 -- 172.21.15.133:0/936884868 shutdown_connections 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 -- 172.21.15.133:0/936884868 wait complete. 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 -- start start 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.449+0000 7f0251497000 1 Processor -- start 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c6421a0 0x55fb7c68c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c686e50 0x55fb7c68c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6a4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c543d10 con 0x55fb7c687770 2024-04-25T10:11:44.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02480c6d00 con 0x55fb7c686e50 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb7c48b4c0 con 0x55fb7c6421a0 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 10 client.0 _ll_register_callbacks cb 0x55fb7c449a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f024c8c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f024c8c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6a4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43318/0 (socket says 172.21.15.133:43318) 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f024c8c7640 1 -- 172.21.15.133:0/966864064 learned_addr learned my addr 172.21.15.133:0/966864064 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f02477fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c686e50 0x55fb7c68c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0247fff640 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c6421a0 0x55fb7c68c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f024c8c7640 1 -- 172.21.15.133:0/966864064 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c6421a0 msgr2=0x55fb7c68c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f024c8c7640 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c6421a0 0x55fb7c68c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f024c8c7640 1 -- 172.21.15.133:0/966864064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c686e50 msgr2=0x55fb7c68c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f024c8c7640 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c686e50 0x55fb7c68c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f024c8c7640 1 -- 172.21.15.133:0/966864064 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb7c4e0a10 con 0x55fb7c687770 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f024c8c7640 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6a4870 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f024001c980 tx=0x7f0240019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d518a4b9356eba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0226ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0226ffd640 1 -- 172.21.15.133:0/966864064 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240006e20 con 0x55fb7c687770 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0226ffd640 1 -- 172.21.15.133:0/966864064 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f02400f30c0 con 0x55fb7c687770 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 1 -- 172.21.15.133:0/966864064 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02400ab600 con 0x55fb7c687770 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0226ffd640 1 -- 172.21.15.133:0/966864064 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02400efa60 con 0x55fb7c687770 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 10 client.8538 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 1 -- 172.21.15.133:0/966864064 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55fb7c41bb00 con 0x55fb7c687770 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0226ffd640 1 -- 172.21.15.133:0/966864064 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400ab600 con 0x55fb7c687770 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0226ffd640 1 -- 172.21.15.133:0/966864064 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f02400f75e0 con 0x55fb7c687770 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 10 client.8538 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 10 client.8538 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 20 client.8538 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fb7c6a4db0 0x55fb7c6a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 4 client.8538 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0251497000 1 -- 172.21.15.133:0/966864064 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55fb7c4943d0 con 0x55fb7c6a4db0 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0247fff640 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fb7c6a4db0 0x55fb7c6a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0226ffd640 10 client.8538 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.453+0000 7f0247fff640 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fb7c6a4db0 0x55fb7c6a51c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4585221338ece066 server_cookie=881a4040c679eb4b in_seq=0 out_seq=0 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0226ffd640 1 -- 172.21.15.133:0/966864064 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55fb7c4943d0 con 0x55fb7c6a4db0 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0226ffd640 10 client.8538 handle_command_reply: tid=0 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 1 client.8538 shutdown 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 1 -- 172.21.15.133:0/966864064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 msgr2=0x55fb7c6a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6a4870 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f024001c980 tx=0x7f0240019240 comp rx=0 tx=0).stop 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 1 -- 172.21.15.133:0/966864064 shutdown_connections 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55fb7c6a4db0 0x55fb7c6a51c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb7c687770 0x55fb7c6a4870 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fb7c686e50 0x55fb7c68c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 1 --2- 172.21.15.133:0/966864064 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55fb7c6421a0 0x55fb7c68c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0226ffd640 0 client.8538 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 1 -- 172.21.15.133:0/966864064 >> 172.21.15.133:0/966864064 conn(0x55fb7c639d40 msgr2=0x55fb7c63c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 1 -- 172.21.15.133:0/966864064 shutdown_connections 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 1 -- 172.21.15.133:0/966864064 wait complete. 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f0251497000 20 client.8538 trim_cache size 0 max 16384 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:11:44.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 -- 172.21.15.133:0/410655819 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0234066f60 msgr2=0x7f0234069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0234066f60 0x7f0234069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0238001570 tx=0x7f0238009040 comp rx=0 tx=0).stop 2024-04-25T10:11:44.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 -- 172.21.15.133:0/410655819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02480ceb50 msgr2=0x7f02480af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02480ceb50 0x7f02480af8d0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f02400ae040 tx=0x7f024001ace0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 -- 172.21.15.133:0/410655819 shutdown_connections 2024-04-25T10:11:44.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0234066f60 0x7f0234069420 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02480ceb50 0x7f02480af8d0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02480cb8d0 0x7f024814f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 --2- 172.21.15.133:0/410655819 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02480b33c0 0x7f02480c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 -- 172.21.15.133:0/410655819 >> 172.21.15.133:0/410655819 conn(0x7f02480d0570 msgr2=0x7f0248154d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 -- 172.21.15.133:0/410655819 shutdown_connections 2024-04-25T10:11:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.457+0000 7f024e32a640 1 -- 172.21.15.133:0/410655819 wait complete. 2024-04-25T10:11:44.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:44.475 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:11:44.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.585+0000 7fd8f84d1640 1 Processor -- start 2024-04-25T10:11:44.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.585+0000 7fd8f84d1640 1 -- start start 2024-04-25T10:11:44.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8f0145170 0x7fd8f006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8f006e4d0 0x7fd8f00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f00ce2e0 con 0x7fd8f00cdd60 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f006db00 con 0x7fd8f006e4d0 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f0145550 con 0x7fd8f0145170 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f5a6c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8f006e4d0 0x7fd8f00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f6a6e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd8f00cdd60 0x7fd8f015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42636/0 (socket says 172.21.15.133:42636) 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f5a6c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8f006e4d0 0x7fd8f00cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43328/0 (socket says 172.21.15.133:43328) 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/3046557753 learned_addr learned my addr 172.21.15.133:0/3046557753 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f626d640 1 --2- 172.21.15.133:0/3046557753 >> v2:172.21.15.162:3300/0 conn(0x7fd8f0145170 0x7fd8f006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f5a6c640 1 -- 172.21.15.133:0/3046557753 >> v2:172.21.15.162:3300/0 conn(0x7fd8f0145170 msgr2=0x7fd8f006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f5a6c640 1 --2- 172.21.15.133:0/3046557753 >> v2:172.21.15.162:3300/0 conn(0x7fd8f0145170 0x7fd8f006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f5a6c640 1 -- 172.21.15.133:0/3046557753 >> v1:172.21.15.133:6789/0 conn(0x7fd8f00cdd60 legacy=0x7fd8f015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f526b640 1 -- 172.21.15.133:0/3046557753 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961721168 0 0) 0x7fd8f00ce2e0 con 0x7fd8f00cdd60 2024-04-25T10:11:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f5a6c640 1 -- 172.21.15.133:0/3046557753 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f006d4b0 con 0x7fd8f006e4d0 2024-04-25T10:11:44.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f5a6c640 1 --2- 172.21.15.133:0/3046557753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f006e4d0 0x7fd8f00cd820 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd8f006b790 tx=0x7fd8e401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0e1d60b51e3401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f526b640 1 -- 172.21.15.133:0/3046557753 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40b2070 con 0x7fd8f006e4d0 2024-04-25T10:11:44.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f526b640 1 -- 172.21.15.133:0/3046557753 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e40a5760 con 0x7fd8f006e4d0 2024-04-25T10:11:44.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f526b640 1 -- 172.21.15.133:0/3046557753 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40a5b30 con 0x7fd8f006e4d0 2024-04-25T10:11:44.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 -- 172.21.15.133:0/3046557753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f006e4d0 msgr2=0x7fd8f00cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 --2- 172.21.15.133:0/3046557753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f006e4d0 0x7fd8f00cd820 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd8f006b790 tx=0x7fd8e401c8a0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 -- 172.21.15.133:0/3046557753 shutdown_connections 2024-04-25T10:11:44.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 --2- 172.21.15.133:0/3046557753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f006e4d0 0x7fd8f00cd820 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 --2- 172.21.15.133:0/3046557753 >> v2:172.21.15.162:3300/0 conn(0x7fd8f0145170 0x7fd8f006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 -- 172.21.15.133:0/3046557753 >> 172.21.15.133:0/3046557753 conn(0x7fd8f0066f70 msgr2=0x7fd8f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 -- 172.21.15.133:0/3046557753 shutdown_connections 2024-04-25T10:11:44.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.589+0000 7fd8f84d1640 1 -- 172.21.15.133:0/3046557753 wait complete. 2024-04-25T10:11:44.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f84d1640 1 Processor -- start 2024-04-25T10:11:44.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f84d1640 1 -- start start 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f84d1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8f006e4d0 0x7fd8f013a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f84d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f00cdd60 0x7fd8f0144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f84d1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f0145170 0x7fd8f00b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f84d1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f0168490 con 0x7fd8f0145170 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f84d1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f015d7f0 con 0x7fd8f006e4d0 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f84d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f0168610 con 0x7fd8f00cdd60 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f6a6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f0145170 0x7fd8f00b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f6a6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f0145170 0x7fd8f00b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43340/0 (socket says 172.21.15.133:43340) 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/2592574605 learned_addr learned my addr 172.21.15.133:0/2592574605 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f5a6c640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f00cdd60 0x7fd8f0144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f626d640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8f006e4d0 0x7fd8f013a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/2592574605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f00cdd60 msgr2=0x7fd8f0144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f6a6e640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f00cdd60 0x7fd8f0144020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/2592574605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8f006e4d0 msgr2=0x7fd8f013a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f6a6e640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8f006e4d0 0x7fd8f013a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/2592574605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f013a850 con 0x7fd8f0145170 2024-04-25T10:11:44.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f6a6e640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f0145170 0x7fd8f00b41f0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800c480 tx=0x7fd8e800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8eeffd640 1 -- 172.21.15.133:0/2592574605 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8018070 con 0x7fd8f0145170 2024-04-25T10:11:44.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8eeffd640 1 -- 172.21.15.133:0/2592574605 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e80074e0 con 0x7fd8f0145170 2024-04-25T10:11:44.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f84d1640 1 -- 172.21.15.133:0/2592574605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f013ab30 con 0x7fd8f0145170 2024-04-25T10:11:44.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8eeffd640 1 -- 172.21.15.133:0/2592574605 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8004a30 con 0x7fd8f0145170 2024-04-25T10:11:44.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.593+0000 7fd8f84d1640 1 -- 172.21.15.133:0/2592574605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f013b0f0 con 0x7fd8f0145170 2024-04-25T10:11:44.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8eeffd640 1 -- 172.21.15.133:0/2592574605 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8e801b070 con 0x7fd8f0145170 2024-04-25T10:11:44.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8eeffd640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8dc066f60 0x7fd8dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8eeffd640 1 -- 172.21.15.133:0/2592574605 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd8e8084470 con 0x7fd8f0145170 2024-04-25T10:11:44.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f626d640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8dc066f60 0x7fd8dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f626d640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8dc066f60 0x7fd8dc069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0073f40 tx=0x7fd8e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8fb63e000 1 Processor -- start 2024-04-25T10:11:44.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8fb63e000 1 -- start start 2024-04-25T10:11:44.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56de30 0x55f3cb56e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56e750 0x55f3cb523f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb524890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3cb36cff0 con 0x55f3cb5244b0 2024-04-25T10:11:44.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3cb2f5150 con 0x55f3cb56e750 2024-04-25T10:11:44.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3cb369090 con 0x55f3cb56de30 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f6a6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb524890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f6a6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb524890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43344/0 (socket says 172.21.15.133:43344) 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/1782448275 learned_addr learned my addr 172.21.15.133:0/1782448275 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f5a6c640 1 --2- 172.21.15.133:0/1782448275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56e750 0x55f3cb523f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/1782448275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56de30 msgr2=0x55f3cb56e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f6a6e640 1 --2- 172.21.15.133:0/1782448275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56de30 0x55f3cb56e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/1782448275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56e750 msgr2=0x55f3cb523f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f6a6e640 1 --2- 172.21.15.133:0/1782448275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56e750 0x55f3cb523f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/1782448275 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3cb5030d0 con 0x55f3cb5244b0 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f5a6c640 1 --2- 172.21.15.133:0/1782448275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56e750 0x55f3cb523f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8f6a6e640 1 --2- 172.21.15.133:0/1782448275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb524890 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800c450 tx=0x7fd8e8055410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bb3599a28c6447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8d7fff640 1 -- 172.21.15.133:0/1782448275 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e805b070 con 0x55f3cb5244b0 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8d7fff640 1 -- 172.21.15.133:0/1782448275 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e805d360 con 0x55f3cb5244b0 2024-04-25T10:11:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.597+0000 7fd8d7fff640 1 -- 172.21.15.133:0/1782448275 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80512c0 con 0x55f3cb5244b0 2024-04-25T10:11:44.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- 172.21.15.133:0/1782448275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 msgr2=0x55f3cb524890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/1782448275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb524890 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800c450 tx=0x7fd8e8055410 comp rx=0 tx=0).stop 2024-04-25T10:11:44.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- 172.21.15.133:0/1782448275 shutdown_connections 2024-04-25T10:11:44.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/1782448275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb524890 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/1782448275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56e750 0x55f3cb523f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/1782448275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56de30 0x55f3cb56e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- 172.21.15.133:0/1782448275 >> 172.21.15.133:0/1782448275 conn(0x55f3cb51c020 msgr2=0x55f3cb51e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- 172.21.15.133:0/1782448275 shutdown_connections 2024-04-25T10:11:44.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- 172.21.15.133:0/1782448275 wait complete. 2024-04-25T10:11:44.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- start start 2024-04-25T10:11:44.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 Processor -- start 2024-04-25T10:11:44.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb5244b0 0x55f3cb56d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56de30 0x55f3cb52a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56e750 0x55f3cb52a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3cb371620 con 0x55f3cb56de30 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3cb327690 con 0x55f3cb5244b0 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3cb2f5a40 con 0x55f3cb56e750 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 10 client.0 _ll_register_callbacks cb 0x55f3cb50ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f5a6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56de30 0x55f3cb52a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f5a6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56de30 0x55f3cb52a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43346/0 (socket says 172.21.15.133:43346) 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f5a6c640 1 -- 172.21.15.133:0/3374347433 learned_addr learned my addr 172.21.15.133:0/3374347433 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f626d640 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb5244b0 0x55f3cb56d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f6a6e640 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56e750 0x55f3cb52a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f5a6c640 1 -- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56e750 msgr2=0x55f3cb52a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f5a6c640 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56e750 0x55f3cb52a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f5a6c640 1 -- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb5244b0 msgr2=0x55f3cb56d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f5a6c640 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb5244b0 0x55f3cb56d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f5a6c640 1 -- 172.21.15.133:0/3374347433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3cb41a1c0 con 0x55f3cb56de30 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f6a6e640 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56e750 0x55f3cb52a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f626d640 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb5244b0 0x55f3cb56d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f5a6c640 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56de30 0x55f3cb52a360 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40b0040 tx=0x7fd8e401b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f702ff026f5fcdbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8d57fa640 1 -- 172.21.15.133:0/3374347433 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40be070 con 0x55f3cb56de30 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8d57fa640 1 -- 172.21.15.133:0/3374347433 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd8e40a5d90 con 0x55f3cb56de30 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- 172.21.15.133:0/3374347433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8e805d360 con 0x55f3cb56de30 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8d57fa640 1 -- 172.21.15.133:0/3374347433 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8e40c9050 con 0x55f3cb56de30 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 10 client.8553 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- 172.21.15.133:0/3374347433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55f3cb416170 con 0x55f3cb56de30 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8d57fa640 1 -- 172.21.15.133:0/3374347433 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e805d360 con 0x55f3cb56de30 2024-04-25T10:11:44.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8d57fa640 1 -- 172.21.15.133:0/3374347433 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8e40cc020 con 0x55f3cb56de30 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 10 client.8553 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 10 client.8553 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 20 client.8553 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55f3cb52b5e0 0x55f3cb5800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 4 client.8553 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8fb63e000 1 -- 172.21.15.133:0/3374347433 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3cb308c60 con 0x55f3cb52b5e0 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.601+0000 7fd8f626d640 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55f3cb52b5e0 0x55f3cb5800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8f626d640 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55f3cb52b5e0 0x55f3cb5800f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8255905775ce9e1 server_cookie=98e12724e81fa80e in_seq=0 out_seq=0 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8d57fa640 10 client.8553 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8d57fa640 1 -- 172.21.15.133:0/3374347433 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f3cb308c60 con 0x55f3cb52b5e0 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8d57fa640 10 client.8553 handle_command_reply: tid=0 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 1 client.8553 shutdown 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 1 -- 172.21.15.133:0/3374347433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56de30 msgr2=0x55f3cb52a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56de30 0x55f3cb52a360 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40b0040 tx=0x7fd8e401b0e0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 1 -- 172.21.15.133:0/3374347433 shutdown_connections 2024-04-25T10:11:44.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55f3cb52b5e0 0x55f3cb5800f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56e750 0x55f3cb52a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56de30 0x55f3cb52a360 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/3374347433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb5244b0 0x55f3cb56d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 1 -- 172.21.15.133:0/3374347433 >> 172.21.15.133:0/3374347433 conn(0x55f3cb51c020 msgr2=0x55f3cb525120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8d57fa640 0 client.8553 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 1 -- 172.21.15.133:0/3374347433 shutdown_connections 2024-04-25T10:11:44.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 1 -- 172.21.15.133:0/3374347433 wait complete. 2024-04-25T10:11:44.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.605+0000 7fd8fb63e000 20 client.8553 trim_cache size 0 max 16384 2024-04-25T10:11:44.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 Processor -- start 2024-04-25T10:11:44.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 -- start start 2024-04-25T10:11:44.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb582320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56de30 0x55f3cb54d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56e750 0x55f3cb54dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3cb56dba0 con 0x55f3cb5244b0 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f01ff000 con 0x55f3cb56e750 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00b0cc0 con 0x55f3cb56de30 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8f626d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb582320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8f626d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb582320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43350/0 (socket says 172.21.15.133:43350) 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8f626d640 1 -- 172.21.15.133:0/1667126695 learned_addr learned my addr 172.21.15.133:0/1667126695 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8f626d640 1 -- 172.21.15.133:0/1667126695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56de30 msgr2=0x55f3cb54d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8f626d640 1 --2- 172.21.15.133:0/1667126695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56de30 0x55f3cb54d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8f626d640 1 -- 172.21.15.133:0/1667126695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56e750 msgr2=0x55f3cb54dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8f626d640 1 --2- 172.21.15.133:0/1667126695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56e750 0x55f3cb54dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8f626d640 1 -- 172.21.15.133:0/1667126695 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3cb3b3de0 con 0x55f3cb5244b0 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8f626d640 1 --2- 172.21.15.133:0/1667126695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb582320 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000fab0 tx=0x7fd8e0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531264846cc2379c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8d77fe640 1 -- 172.21.15.133:0/1667126695 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0015990 con 0x55f3cb5244b0 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8d77fe640 1 -- 172.21.15.133:0/1667126695 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e0015b30 con 0x55f3cb5244b0 2024-04-25T10:11:44.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8d77fe640 1 -- 172.21.15.133:0/1667126695 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0024040 con 0x55f3cb5244b0 2024-04-25T10:11:44.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 -- 172.21.15.133:0/1667126695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 msgr2=0x55f3cb582320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/1667126695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb582320 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000fab0 tx=0x7fd8e0012000 comp rx=0 tx=0).stop 2024-04-25T10:11:44.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 -- 172.21.15.133:0/1667126695 shutdown_connections 2024-04-25T10:11:44.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/1667126695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56e750 0x55f3cb54dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/1667126695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb56de30 0x55f3cb54d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/1667126695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb5244b0 0x55f3cb582320 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 -- 172.21.15.133:0/1667126695 >> 172.21.15.133:0/1667126695 conn(0x55f3cb51c020 msgr2=0x55f3cb57b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 -- 172.21.15.133:0/1667126695 shutdown_connections 2024-04-25T10:11:44.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 -- 172.21.15.133:0/1667126695 wait complete. 2024-04-25T10:11:44.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 -- start start 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.621+0000 7fd8fb63e000 1 Processor -- start 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb5244b0 0x55f3cb54c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56de30 0x55f3cb54cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56e750 0x55f3cb581140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00b06c0 con 0x55f3cb56e750 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00b38c0 con 0x55f3cb56de30 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3cb307e00 con 0x55f3cb5244b0 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 10 client.0 _ll_register_callbacks cb 0x55f3cb4eb040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f6a6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56e750 0x55f3cb581140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f6a6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56e750 0x55f3cb581140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43366/0 (socket says 172.21.15.133:43366) 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/9030697 learned_addr learned my addr 172.21.15.133:0/9030697 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f5a6c640 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56de30 0x55f3cb54cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f626d640 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb5244b0 0x55f3cb54c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb5244b0 msgr2=0x55f3cb54c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f6a6e640 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb5244b0 0x55f3cb54c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56de30 msgr2=0x55f3cb54cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f6a6e640 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56de30 0x55f3cb54cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f6a6e640 1 -- 172.21.15.133:0/9030697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3cb30e8e0 con 0x55f3cb56e750 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f5a6c640 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56de30 0x55f3cb54cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f6a6e640 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56e750 0x55f3cb581140 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd8e805ce30 tx=0x7fd8e8082740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6896502526ae6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8d57fa640 1 -- 172.21.15.133:0/9030697 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e805d360 con 0x55f3cb56e750 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8d57fa640 1 -- 172.21.15.133:0/9030697 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd8e8054eb0 con 0x55f3cb56e750 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 -- 172.21.15.133:0/9030697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8e0015b30 con 0x55f3cb56e750 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8d57fa640 1 -- 172.21.15.133:0/9030697 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8e805b070 con 0x55f3cb56e750 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 10 client.8559 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 -- 172.21.15.133:0/9030697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55f3cb3c68f0 con 0x55f3cb56e750 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8d57fa640 1 -- 172.21.15.133:0/9030697 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e0015b30 con 0x55f3cb56e750 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8d57fa640 1 -- 172.21.15.133:0/9030697 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8e805ced0 con 0x55f3cb56e750 2024-04-25T10:11:44.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 10 client.8559 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 10 client.8559 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 20 client.8559 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55f3cb581680 0x55f3cb581a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 4 client.8559 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 -- 172.21.15.133:0/9030697 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f3cb37a530 con 0x55f3cb581680 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f626d640 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55f3cb581680 0x55f3cb581a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8d57fa640 10 client.8559 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f626d640 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55f3cb581680 0x55f3cb581a60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910c13dc9ba9f390 server_cookie=3ea125099089966f in_seq=0 out_seq=0 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8d57fa640 1 -- 172.21.15.133:0/9030697 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+278 (crc 0 0 0) 0x55f3cb37a530 con 0x55f3cb581680 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8d57fa640 10 client.8559 handle_command_reply: tid=0 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 client.8559 shutdown 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 -- 172.21.15.133:0/9030697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56e750 msgr2=0x55f3cb581140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56e750 0x55f3cb581140 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd8e805ce30 tx=0x7fd8e8082740 comp rx=0 tx=0).stop 2024-04-25T10:11:44.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 -- 172.21.15.133:0/9030697 shutdown_connections 2024-04-25T10:11:44.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55f3cb581680 0x55f3cb581a60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f3cb56e750 0x55f3cb581140 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55f3cb56de30 0x55f3cb54cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8d57fa640 0 client.8559 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:44.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 --2- 172.21.15.133:0/9030697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55f3cb5244b0 0x55f3cb54c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 -- 172.21.15.133:0/9030697 >> 172.21.15.133:0/9030697 conn(0x55f3cb51c020 msgr2=0x55f3cb51e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 -- 172.21.15.133:0/9030697 shutdown_connections 2024-04-25T10:11:44.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 1 -- 172.21.15.133:0/9030697 wait complete. 2024-04-25T10:11:44.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8fb63e000 20 client.8559 trim_cache size 0 max 16384 2024-04-25T10:11:44.634 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:11:44.635 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "PAUSED (366 inodes in the stack)", 2024-04-25T10:11:44.635 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-04-25T10:11:44.635 INFO:teuthology.orchestra.run.smithi133.stdout: "3aafa117-8943-43e7-b495-998293c81538": { 2024-04-25T10:11:44.635 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:11:44.635 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "3aafa117-8943-43e7-b495-998293c81538", 2024-04-25T10:11:44.635 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive" 2024-04-25T10:11:44.635 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:44.635 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:44.635 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:11:44.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 -- 172.21.15.133:0/2592574605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8dc066f60 msgr2=0x7fd8dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8dc066f60 0x7fd8dc069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0073f40 tx=0x7fd8e0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:44.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 -- 172.21.15.133:0/2592574605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f0145170 msgr2=0x7fd8f00b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f0145170 0x7fd8f00b41f0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800c480 tx=0x7fd8e800c940 comp rx=0 tx=0).stop 2024-04-25T10:11:44.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 -- 172.21.15.133:0/2592574605 shutdown_connections 2024-04-25T10:11:44.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd8dc066f60 0x7fd8dc069420 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f0145170 0x7fd8f00b41f0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8f00cdd60 0x7fd8f0144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 --2- 172.21.15.133:0/2592574605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8f006e4d0 0x7fd8f013a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 -- 172.21.15.133:0/2592574605 >> 172.21.15.133:0/2592574605 conn(0x7fd8f0066f70 msgr2=0x7fd8f015f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 -- 172.21.15.133:0/2592574605 shutdown_connections 2024-04-25T10:11:44.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.625+0000 7fd8f84d1640 1 -- 172.21.15.133:0/2592574605 wait complete. 2024-04-25T10:11:44.645 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:44.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff7ed3640 1 Processor -- start 2024-04-25T10:11:44.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff7ed3640 1 -- start start 2024-04-25T10:11:44.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff7ed3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbff00d0770 0x7fbff00d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff7ed3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff00b87e0 0x7fbff00b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff7ed3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff00ceb20 con 0x7fbff015db00 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff7ed3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff00d1110 con 0x7fbff00d0770 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff7ed3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00b30a0 con 0x7fbff00b87e0 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff5c6f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fbff015db00 0x7fbff00b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42646/0 (socket says 172.21.15.133:42646) 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff5c6f640 1 -- 172.21.15.133:0/3376619294 learned_addr learned my addr 172.21.15.133:0/3376619294 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff546e640 1 --2- 172.21.15.133:0/3376619294 >> v2:172.21.15.162:3300/0 conn(0x7fbff00d0770 0x7fbff00d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287136886 0 0) 0x7fbff00ceb20 con 0x7fbff015db00 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff6470640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff00b87e0 0x7fbff00b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfd8003530 con 0x7fbff015db00 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434446915 0 0) 0x7fbfd8003530 con 0x7fbff015db00 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff00b87e0 msgr2=0x7fbff00b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff00b87e0 0x7fbff00b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 >> v2:172.21.15.162:3300/0 conn(0x7fbff00d0770 msgr2=0x7fbff00d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 --2- 172.21.15.133:0/3376619294 >> v2:172.21.15.162:3300/0 conn(0x7fbff00d0770 0x7fbff00d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00b7740 con 0x7fbff015db00 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fbfe0001680 con 0x7fbff015db00 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff6470640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff00b87e0 0x7fbff00b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 >> v1:172.21.15.133:6789/0 conn(0x7fbff015db00 legacy=0x7fbff00b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 0x7fbfd8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfd8004810 0x7fbfd8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff00b87e0 0x7fbfd80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00b30a0 con 0x7fbfd80039c0 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd800b9d0 con 0x7fbfd8004810 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd800bb50 con 0x7fbff00b87e0 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff5c6f640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 0x7fbfd8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff546e640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfd8004810 0x7fbfd8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff5c6f640 1 -- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff00b87e0 msgr2=0x7fbfd80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff5c6f640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff00b87e0 0x7fbfd80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff5c6f640 1 -- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfd8004810 msgr2=0x7fbfd8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff5c6f640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfd8004810 0x7fbfd8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff5c6f640 1 -- 172.21.15.133:0/3376619294 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd8003530 con 0x7fbfd80039c0 2024-04-25T10:11:44.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.757+0000 7fbff546e640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfd8004810 0x7fbfd8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff5c6f640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 0x7fbfd8003de0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000c2f0 tx=0x7fbfe001cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddb9926c0d9fd06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0007d60 con 0x7fbfd80039c0 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe00a9020 con 0x7fbfd80039c0 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff4c6d640 1 -- 172.21.15.133:0/3376619294 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0005b00 con 0x7fbfd80039c0 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 -- 172.21.15.133:0/3376619294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 msgr2=0x7fbfd8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 --2- 172.21.15.133:0/3376619294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 0x7fbfd8003de0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000c2f0 tx=0x7fbfe001cd10 comp rx=0 tx=0).stop 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff6470640 1 -- 172.21.15.133:0/3376619294 reap_dead start 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 -- 172.21.15.133:0/3376619294 shutdown_connections 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 -- 172.21.15.133:0/3376619294 >> 172.21.15.133:0/3376619294 conn(0x7fbff00727c0 msgr2=0x7fbff0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 -- 172.21.15.133:0/3376619294 shutdown_connections 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 -- 172.21.15.133:0/3376619294 wait complete. 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 Processor -- start 2024-04-25T10:11:44.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 -- start start 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 0x7fbff0159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff014e780 0x7fbff014db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff014ecb0 0x7fbff014e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00b7430 con 0x7fbfd80039c0 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00b82c0 con 0x7fbff014e780 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff7ed3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00d18a0 con 0x7fbff014ecb0 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff5c6f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 0x7fbff0159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff5c6f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 0x7fbff0159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43370/0 (socket says 172.21.15.133:43370) 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff5c6f640 1 -- 172.21.15.133:0/1566238960 learned_addr learned my addr 172.21.15.133:0/1566238960 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff5c6f640 1 -- 172.21.15.133:0/1566238960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff014ecb0 msgr2=0x7fbff014e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff5c6f640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff014ecb0 0x7fbff014e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff5c6f640 1 -- 172.21.15.133:0/1566238960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff014e780 msgr2=0x7fbff014db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff546e640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff014e780 0x7fbff014db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff5c6f640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff014e780 0x7fbff014db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.761+0000 7fbff5c6f640 1 -- 172.21.15.133:0/1566238960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff015fc90 con 0x7fbfd80039c0 2024-04-25T10:11:44.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbff5c6f640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 0x7fbff0159790 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00ab040 tx=0x7fbfd8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbfee7fc640 1 -- 172.21.15.133:0/1566238960 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00a5b40 con 0x7fbfd80039c0 2024-04-25T10:11:44.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbfee7fc640 1 -- 172.21.15.133:0/1566238960 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe00a5ce0 con 0x7fbfd80039c0 2024-04-25T10:11:44.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbff7ed3640 1 -- 172.21.15.133:0/1566238960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff015ff70 con 0x7fbfd80039c0 2024-04-25T10:11:44.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbfee7fc640 1 -- 172.21.15.133:0/1566238960 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00049f0 con 0x7fbfd80039c0 2024-04-25T10:11:44.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbff7ed3640 1 -- 172.21.15.133:0/1566238960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff0160530 con 0x7fbfd80039c0 2024-04-25T10:11:44.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbff7ed3640 1 -- 172.21.15.133:0/1566238960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff0068ef0 con 0x7fbfd80039c0 2024-04-25T10:11:44.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbfee7fc640 1 -- 172.21.15.133:0/1566238960 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfe0004b90 con 0x7fbfd80039c0 2024-04-25T10:11:44.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbfee7fc640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfdc066f60 0x7fbfdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbfee7fc640 1 -- 172.21.15.133:0/1566238960 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbfe011f600 con 0x7fbfd80039c0 2024-04-25T10:11:44.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbff546e640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfdc066f60 0x7fbfdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.765+0000 7fbff546e640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfdc066f60 0x7fbfdc069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbff0160f50 tx=0x7fbfe4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.769+0000 7fbfee7fc640 1 -- 172.21.15.133:0/1566238960 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfe0016a50 con 0x7fbfd80039c0 2024-04-25T10:11:44.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.961+0000 7fbff7ed3640 1 -- 172.21.15.133:0/1566238960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbff0157fc0 con 0x7fbfd80039c0 2024-04-25T10:11:44.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbfee7fc640 1 -- 172.21.15.133:0/1566238960 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fbfe00f0450 con 0x7fbfd80039c0 2024-04-25T10:11:44.967 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:44.967 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:44.968 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:44.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 -- 172.21.15.133:0/1566238960 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfdc066f60 msgr2=0x7fbfdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfdc066f60 0x7fbfdc069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbff0160f50 tx=0x7fbfe4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:44.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 -- 172.21.15.133:0/1566238960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 msgr2=0x7fbff0159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 0x7fbff0159790 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00ab040 tx=0x7fbfd8005840 comp rx=0 tx=0).stop 2024-04-25T10:11:44.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 -- 172.21.15.133:0/1566238960 shutdown_connections 2024-04-25T10:11:44.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbff014ecb0 0x7fbff014e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbff014e780 0x7fbff014db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfdc066f60 0x7fbfdc069420 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 --2- 172.21.15.133:0/1566238960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfd80039c0 0x7fbff0159790 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 -- 172.21.15.133:0/1566238960 >> 172.21.15.133:0/1566238960 conn(0x7fbff00727c0 msgr2=0x7fbff00ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 -- 172.21.15.133:0/1566238960 shutdown_connections 2024-04-25T10:11:44.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:44.965+0000 7fbff7ed3640 1 -- 172.21.15.133:0/1566238960 wait complete. 2024-04-25T10:11:44.985 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:11:44.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f0888dd5640 1 Processor -- start 2024-04-25T10:11:45.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f0888dd5640 1 -- start start 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f0888dd5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0884158110 0x7f08840b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f0888dd5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08841584f0 0x7f088415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f0888dd5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08840cbe60 con 0x7f088406e460 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f0888dd5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08840b19b0 con 0x7f0884158110 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f0888dd5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088415b360 con 0x7f08841584f0 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f088259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0884158110 0x7f08840b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f088259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0884158110 0x7f08840b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43374/0 (socket says 172.21.15.133:43374) 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f088259c640 1 -- 172.21.15.133:0/1351320319 learned_addr learned my addr 172.21.15.133:0/1351320319 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f0881d9b640 1 --2- 172.21.15.133:0/1351320319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08841584f0 0x7f088415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f088259c640 1 -- 172.21.15.133:0/1351320319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08841584f0 msgr2=0x7f088415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f088259c640 1 --2- 172.21.15.133:0/1351320319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08841584f0 0x7f088415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f088259c640 1 -- 172.21.15.133:0/1351320319 >> v1:172.21.15.162:6789/0 conn(0x7f088406e460 legacy=0x7f088415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.101+0000 7f088259c640 1 -- 172.21.15.133:0/1351320319 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088415b640 con 0x7f0884158110 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f088259c640 1 --2- 172.21.15.133:0/1351320319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0884158110 0x7f08840b1f00 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f087400bb10 tx=0x7f087401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21570433aa4e36e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f088159a640 1 -- 172.21.15.133:0/1351320319 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08740b4070 con 0x7f0884158110 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f088159a640 1 -- 172.21.15.133:0/1351320319 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08740a7c10 con 0x7f0884158110 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f088159a640 1 -- 172.21.15.133:0/1351320319 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08740af440 con 0x7f0884158110 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 -- 172.21.15.133:0/1351320319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0884158110 msgr2=0x7f08840b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 --2- 172.21.15.133:0/1351320319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0884158110 0x7f08840b1f00 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f087400bb10 tx=0x7f087401ec30 comp rx=0 tx=0).stop 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 -- 172.21.15.133:0/1351320319 shutdown_connections 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 --2- 172.21.15.133:0/1351320319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08841584f0 0x7f088415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 --2- 172.21.15.133:0/1351320319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0884158110 0x7f08840b1f00 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 -- 172.21.15.133:0/1351320319 >> 172.21.15.133:0/1351320319 conn(0x7f0884067000 msgr2=0x7f088406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 -- 172.21.15.133:0/1351320319 shutdown_connections 2024-04-25T10:11:45.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 -- 172.21.15.133:0/1351320319 wait complete. 2024-04-25T10:11:45.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 Processor -- start 2024-04-25T10:11:45.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 -- start start 2024-04-25T10:11:45.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f088406e460 0x7f08840b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0884158110 0x7f08841540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08841584f0 0x7f0884154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08840cd7f0 con 0x7f0884158110 2024-04-25T10:11:45.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08840cd670 con 0x7f088406e460 2024-04-25T10:11:45.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0888dd5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088406d350 con 0x7f08841584f0 2024-04-25T10:11:45.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0881d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0884158110 0x7f08841540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0881d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0884158110 0x7f08841540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43390/0 (socket says 172.21.15.133:43390) 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0881d9b640 1 -- 172.21.15.133:0/3206317958 learned_addr learned my addr 172.21.15.133:0/3206317958 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f088259c640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f088406e460 0x7f08840b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0882d9d640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08841584f0 0x7f0884154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0881d9b640 1 -- 172.21.15.133:0/3206317958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08841584f0 msgr2=0x7f0884154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0881d9b640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08841584f0 0x7f0884154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0881d9b640 1 -- 172.21.15.133:0/3206317958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f088406e460 msgr2=0x7f08840b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0881d9b640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f088406e460 0x7f08840b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0881d9b640 1 -- 172.21.15.133:0/3206317958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08840c6e90 con 0x7f0884158110 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.105+0000 7f0882d9d640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08841584f0 0x7f0884154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f0881d9b640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0884158110 0x7f08841540b0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f087800d5f0 tx=0x7f087800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f0872ffd640 1 -- 172.21.15.133:0/3206317958 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08780125f0 con 0x7f0884158110 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f0872ffd640 1 -- 172.21.15.133:0/3206317958 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0878019070 con 0x7f0884158110 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f0888dd5640 1 -- 172.21.15.133:0/3206317958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08840c7170 con 0x7f0884158110 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f0872ffd640 1 -- 172.21.15.133:0/3206317958 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08780129b0 con 0x7f0884158110 2024-04-25T10:11:45.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f0888dd5640 1 -- 172.21.15.133:0/3206317958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08840c7730 con 0x7f0884158110 2024-04-25T10:11:45.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f0888dd5640 1 -- 172.21.15.133:0/3206317958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0884068ef0 con 0x7f0884158110 2024-04-25T10:11:45.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f0872ffd640 1 -- 172.21.15.133:0/3206317958 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0878012b50 con 0x7f0884158110 2024-04-25T10:11:45.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f0872ffd640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0864066f60 0x7f0864069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f0872ffd640 1 -- 172.21.15.133:0/3206317958 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0878085600 con 0x7f0884158110 2024-04-25T10:11:45.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f088259c640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0864066f60 0x7f0864069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.109+0000 7f088259c640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0864066f60 0x7f0864069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f08740b1000 tx=0x7f08740a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.113+0000 7f0872ffd640 1 -- 172.21.15.133:0/3206317958 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0878056450 con 0x7f0884158110 2024-04-25T10:11:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.333+0000 7f0888dd5640 1 -- 172.21.15.133:0/3206317958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f088406c2d0 con 0x7f0884158110 2024-04-25T10:11:45.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0872ffd640 1 -- 172.21.15.133:0/3206317958 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1619 (secure 0 0 0) 0x7f0878055d50 con 0x7f0884158110 2024-04-25T10:11:45.339 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:45.339 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":618,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":833,"data_bytes":851049305,"bytes_used":2192900096,"bytes_avail":770901213184,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2306233,"recovering_keys_per_sec":0,"num_objects_recovered":22,"num_bytes_recovered":23068672,"num_keys_recovered":0,"write_bytes_sec":50189343,"read_op_per_sec":0,"write_op_per_sec":51},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:10:59.874227+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:11:45.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 -- 172.21.15.133:0/3206317958 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0864066f60 msgr2=0x7f0864069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0864066f60 0x7f0864069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f08740b1000 tx=0x7f08740a9070 comp rx=0 tx=0).stop 2024-04-25T10:11:45.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 -- 172.21.15.133:0/3206317958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0884158110 msgr2=0x7f08841540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0884158110 0x7f08841540b0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f087800d5f0 tx=0x7f087800dab0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 -- 172.21.15.133:0/3206317958 shutdown_connections 2024-04-25T10:11:45.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08841584f0 0x7f0884154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0864066f60 0x7f0864069420 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0884158110 0x7f08841540b0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 --2- 172.21.15.133:0/3206317958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f088406e460 0x7f08840b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 -- 172.21.15.133:0/3206317958 >> 172.21.15.133:0/3206317958 conn(0x7f0884067000 msgr2=0x7f0884158a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 -- 172.21.15.133:0/3206317958 shutdown_connections 2024-04-25T10:11:45.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:45.337+0000 7f0888dd5640 1 -- 172.21.15.133:0/3206317958 wait complete. 2024-04-25T10:11:45.357 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:11:46.358 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:46.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f9940b48640 1 Processor -- start 2024-04-25T10:11:46.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f9940b48640 1 -- start start 2024-04-25T10:11:46.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f9940b48640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f993c06eb70 0x7f993c0c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f9940b48640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f993c15a9d0 0x7f993c0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f9940b48640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993c0cc5a0 con 0x7f993c15a5f0 2024-04-25T10:11:46.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f9940b48640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993c0cc880 con 0x7f993c15a9d0 2024-04-25T10:11:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f9940b48640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0ccb60 con 0x7f993c06eb70 2024-04-25T10:11:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f993ad9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f993c15a9d0 0x7f993c0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f993ad9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f993c15a9d0 0x7f993c0cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34152/0 (socket says 172.21.15.133:34152) 2024-04-25T10:11:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f993ad9d640 1 -- 172.21.15.133:0/1895447629 learned_addr learned my addr 172.21.15.133:0/1895447629 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f9939d9b640 1 --2- 172.21.15.133:0/1895447629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f993c06eb70 0x7f993c0c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f993ad9d640 1 -- 172.21.15.133:0/1895447629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f993c06eb70 msgr2=0x7f993c0c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f993ad9d640 1 --2- 172.21.15.133:0/1895447629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f993c06eb70 0x7f993c0c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f993ad9d640 1 -- 172.21.15.133:0/1895447629 >> v1:172.21.15.162:6789/0 conn(0x7f993c15a5f0 legacy=0x7f993c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.485+0000 7f993ad9d640 1 -- 172.21.15.133:0/1895447629 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c0cce40 con 0x7f993c15a9d0 2024-04-25T10:11:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f993ad9d640 1 --2- 172.21.15.133:0/1895447629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f993c15a9d0 0x7f993c0cbe50 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9934009e50 tx=0x7f993401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c8da8c69e786af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f993959a640 1 -- 172.21.15.133:0/1895447629 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340a5670 con 0x7f993c15a9d0 2024-04-25T10:11:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f993959a640 1 -- 172.21.15.133:0/1895447629 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99340a5810 con 0x7f993c15a9d0 2024-04-25T10:11:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f993959a640 1 -- 172.21.15.133:0/1895447629 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340a5be0 con 0x7f993c15a9d0 2024-04-25T10:11:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 -- 172.21.15.133:0/1895447629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f993c15a9d0 msgr2=0x7f993c0cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 --2- 172.21.15.133:0/1895447629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f993c15a9d0 0x7f993c0cbe50 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9934009e50 tx=0x7f993401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 -- 172.21.15.133:0/1895447629 shutdown_connections 2024-04-25T10:11:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 --2- 172.21.15.133:0/1895447629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f993c15a9d0 0x7f993c0cbe50 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 --2- 172.21.15.133:0/1895447629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f993c06eb70 0x7f993c0c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 -- 172.21.15.133:0/1895447629 >> 172.21.15.133:0/1895447629 conn(0x7f993c067000 msgr2=0x7f993c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:46.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 -- 172.21.15.133:0/1895447629 shutdown_connections 2024-04-25T10:11:46.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 -- 172.21.15.133:0/1895447629 wait complete. 2024-04-25T10:11:46.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 Processor -- start 2024-04-25T10:11:46.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 -- start start 2024-04-25T10:11:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f993c06eb70 0x7f993c14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f993c15a5f0 0x7f993c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f993c15a9d0 0x7f993c152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0ce450 con 0x7f993c15a9d0 2024-04-25T10:11:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0ce2d0 con 0x7f993c06eb70 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.489+0000 7f9940b48640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0ce5d0 con 0x7f993c15a5f0 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f993c15a9d0 0x7f993c152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f993c15a9d0 0x7f993c152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34158/0 (socket says 172.21.15.133:34158) 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993ad9d640 1 -- 172.21.15.133:0/316841549 learned_addr learned my addr 172.21.15.133:0/316841549 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993a59c640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f993c06eb70 0x7f993c14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f9939d9b640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f993c15a5f0 0x7f993c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993ad9d640 1 -- 172.21.15.133:0/316841549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f993c15a5f0 msgr2=0x7f993c158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993ad9d640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f993c15a5f0 0x7f993c158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993ad9d640 1 -- 172.21.15.133:0/316841549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f993c06eb70 msgr2=0x7f993c14d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993ad9d640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f993c06eb70 0x7f993c14d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993ad9d640 1 -- 172.21.15.133:0/316841549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c14da60 con 0x7f993c15a9d0 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993ad9d640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f993c15a9d0 0x7f993c152360 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9934009e50 tx=0x7f9934019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f992affd640 1 -- 172.21.15.133:0/316841549 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934009510 con 0x7f993c15a9d0 2024-04-25T10:11:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f992affd640 1 -- 172.21.15.133:0/316841549 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99340ad9b0 con 0x7f993c15a9d0 2024-04-25T10:11:46.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f992affd640 1 -- 172.21.15.133:0/316841549 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340adda0 con 0x7f993c15a9d0 2024-04-25T10:11:46.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f9940b48640 1 -- 172.21.15.133:0/316841549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993c157a90 con 0x7f993c15a9d0 2024-04-25T10:11:46.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f9940b48640 1 -- 172.21.15.133:0/316841549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c158050 con 0x7f993c15a9d0 2024-04-25T10:11:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f9940b48640 1 -- 172.21.15.133:0/316841549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c068ef0 con 0x7f993c15a9d0 2024-04-25T10:11:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f992affd640 1 -- 172.21.15.133:0/316841549 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99340c1020 con 0x7f993c15a9d0 2024-04-25T10:11:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f992affd640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f991c066f60 0x7f991c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f992affd640 1 -- 172.21.15.133:0/316841549 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f993411fb50 con 0x7f993c15a9d0 2024-04-25T10:11:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993a59c640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f991c066f60 0x7f991c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.493+0000 7f993a59c640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f991c066f60 0x7f991c069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f993c14e6f0 tx=0x7f9930009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.497+0000 7f992affd640 1 -- 172.21.15.133:0/316841549 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99340c8020 con 0x7f993c15a9d0 2024-04-25T10:11:46.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 -- 172.21.15.133:0/316841549 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f993c156550 con 0x7f993c15a9d0 2024-04-25T10:11:46.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f992affd640 1 -- 172.21.15.133:0/316841549 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f99340165e0 con 0x7f993c15a9d0 2024-04-25T10:11:46.688 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:46.688 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:46.688 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 -- 172.21.15.133:0/316841549 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f991c066f60 msgr2=0x7f991c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f991c066f60 0x7f991c069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f993c14e6f0 tx=0x7f9930009040 comp rx=0 tx=0).stop 2024-04-25T10:11:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 -- 172.21.15.133:0/316841549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f993c15a9d0 msgr2=0x7f993c152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f993c15a9d0 0x7f993c152360 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9934009e50 tx=0x7f9934019790 comp rx=0 tx=0).stop 2024-04-25T10:11:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 -- 172.21.15.133:0/316841549 shutdown_connections 2024-04-25T10:11:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f991c066f60 0x7f991c069420 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f993c15a9d0 0x7f993c152360 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f993c15a5f0 0x7f993c158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 --2- 172.21.15.133:0/316841549 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f993c06eb70 0x7f993c14d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 -- 172.21.15.133:0/316841549 >> 172.21.15.133:0/316841549 conn(0x7f993c067000 msgr2=0x7f993c0c5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 -- 172.21.15.133:0/316841549 shutdown_connections 2024-04-25T10:11:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.685+0000 7f9940b48640 1 -- 172.21.15.133:0/316841549 wait complete. 2024-04-25T10:11:46.705 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:11:46.705 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:46.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07ecc02640 1 Processor -- start 2024-04-25T10:11:46.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07ecc02640 1 -- start start 2024-04-25T10:11:46.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07ecc02640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f07e815db00 0x7f07e80b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07ecc02640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07e80d0770 0x7f07e80d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07ecc02640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e80ceb20 con 0x7f07e81614c0 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07ecc02640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e80d1110 con 0x7f07e80d0770 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07ecc02640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e80b7740 con 0x7f07e815db00 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07e5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07e80d0770 0x7f07e80d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07e6d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f07e81614c0 0x7f07e8161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51674/0 (socket says 172.21.15.133:51674) 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07e6d9d640 1 -- 172.21.15.133:0/1542082457 learned_addr learned my addr 172.21.15.133:0/1542082457 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07e659c640 1 --2- 172.21.15.133:0/1542082457 >> v2:172.21.15.162:3300/0 conn(0x7f07e815db00 0x7f07e80b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07e5d9b640 1 -- 172.21.15.133:0/1542082457 >> v2:172.21.15.162:3300/0 conn(0x7f07e815db00 msgr2=0x7f07e80b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07e5d9b640 1 --2- 172.21.15.133:0/1542082457 >> v2:172.21.15.162:3300/0 conn(0x7f07e815db00 0x7f07e80b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07e5d9b640 1 -- 172.21.15.133:0/1542082457 >> v1:172.21.15.133:6789/0 conn(0x7f07e81614c0 legacy=0x7f07e8161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07e559a640 1 -- 172.21.15.133:0/1542082457 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499818575 0 0) 0x7f07e80ceb20 con 0x7f07e81614c0 2024-04-25T10:11:46.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.821+0000 7f07e5d9b640 1 -- 172.21.15.133:0/1542082457 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80b97f0 con 0x7f07e80d0770 2024-04-25T10:11:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07e5d9b640 1 --2- 172.21.15.133:0/1542082457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e80d0770 0x7f07e80d0bd0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f07e80b53d0 tx=0x7f07dc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fd4d730dddaa48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07e559a640 1 -- 172.21.15.133:0/1542082457 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0ab650 con 0x7f07e80d0770 2024-04-25T10:11:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07e559a640 1 -- 172.21.15.133:0/1542082457 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc0b0070 con 0x7f07e80d0770 2024-04-25T10:11:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07e559a640 1 -- 172.21.15.133:0/1542082457 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0aba10 con 0x7f07e80d0770 2024-04-25T10:11:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07ecc02640 1 -- 172.21.15.133:0/1542082457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e80d0770 msgr2=0x7f07e80d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07ecc02640 1 --2- 172.21.15.133:0/1542082457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e80d0770 0x7f07e80d0bd0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f07e80b53d0 tx=0x7f07dc01c450 comp rx=0 tx=0).stop 2024-04-25T10:11:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07ecc02640 1 -- 172.21.15.133:0/1542082457 shutdown_connections 2024-04-25T10:11:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07ecc02640 1 --2- 172.21.15.133:0/1542082457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e80d0770 0x7f07e80d0bd0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07ecc02640 1 --2- 172.21.15.133:0/1542082457 >> v2:172.21.15.162:3300/0 conn(0x7f07e815db00 0x7f07e80b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07ecc02640 1 -- 172.21.15.133:0/1542082457 >> 172.21.15.133:0/1542082457 conn(0x7f07e80727c0 msgr2=0x7f07e8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07ecc02640 1 -- 172.21.15.133:0/1542082457 shutdown_connections 2024-04-25T10:11:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07ecc02640 1 -- 172.21.15.133:0/1542082457 wait complete. 2024-04-25T10:11:46.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07ecc02640 1 Processor -- start 2024-04-25T10:11:46.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.825+0000 7f07ecc02640 1 -- start start 2024-04-25T10:11:46.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07ecc02640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e80d0770 0x7f07e80b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07ecc02640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e815db00 0x7f07e815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07ecc02640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e81614c0 0x7f07e815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07ecc02640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80b7430 con 0x7f07e815db00 2024-04-25T10:11:46.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07ecc02640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80b82c0 con 0x7f07e81614c0 2024-04-25T10:11:46.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07ecc02640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80d18a0 con 0x7f07e80d0770 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e815db00 0x7f07e815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e815db00 0x7f07e815b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34182/0 (socket says 172.21.15.133:34182) 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e5d9b640 1 -- 172.21.15.133:0/561744847 learned_addr learned my addr 172.21.15.133:0/561744847 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e659c640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e80d0770 0x7f07e80b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e6d9d640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e81614c0 0x7f07e815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e5d9b640 1 -- 172.21.15.133:0/561744847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e80d0770 msgr2=0x7f07e80b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e5d9b640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e80d0770 0x7f07e80b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e5d9b640 1 -- 172.21.15.133:0/561744847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e81614c0 msgr2=0x7f07e815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e5d9b640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e81614c0 0x7f07e815bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e5d9b640 1 -- 172.21.15.133:0/561744847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80b3f00 con 0x7f07e815db00 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e6d9d640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e81614c0 0x7f07e815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e5d9b640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e815db00 0x7f07e815b5d0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0ad040 tx=0x7f07dc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07d6ffd640 1 -- 172.21.15.133:0/561744847 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0bf610 con 0x7f07e815db00 2024-04-25T10:11:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07d6ffd640 1 -- 172.21.15.133:0/561744847 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc0bf7d0 con 0x7f07e815db00 2024-04-25T10:11:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07ecc02640 1 -- 172.21.15.133:0/561744847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e80b41e0 con 0x7f07e815db00 2024-04-25T10:11:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07d6ffd640 1 -- 172.21.15.133:0/561744847 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0bfbc0 con 0x7f07e815db00 2024-04-25T10:11:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07ecc02640 1 -- 172.21.15.133:0/561744847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e80b47a0 con 0x7f07e815db00 2024-04-25T10:11:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07ecc02640 1 -- 172.21.15.133:0/561744847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e80b4f80 con 0x7f07e815db00 2024-04-25T10:11:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07d6ffd640 1 -- 172.21.15.133:0/561744847 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07dc0bd020 con 0x7f07e815db00 2024-04-25T10:11:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07d6ffd640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07c8066f60 0x7f07c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07d6ffd640 1 -- 172.21.15.133:0/561744847 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f07dc11fcd0 con 0x7f07e815db00 2024-04-25T10:11:46.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e659c640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07c8066f60 0x7f07c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.829+0000 7f07e659c640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07c8066f60 0x7f07c8069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f07e80ca340 tx=0x7f07d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:46.833+0000 7f07d6ffd640 1 -- 172.21.15.133:0/561744847 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07dc016070 con 0x7f07e815db00 2024-04-25T10:11:47.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.033+0000 7f07ecc02640 1 -- 172.21.15.133:0/561744847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f07e8068ef0 con 0x7f07e815db00 2024-04-25T10:11:47.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07d6ffd640 1 -- 172.21.15.133:0/561744847 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1621 (secure 0 0 0) 0x7f07dc0f0cd0 con 0x7f07e815db00 2024-04-25T10:11:47.039 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:47.039 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":619,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077446235,"bytes_used":2672754688,"bytes_avail":770421358592,"bytes_total":773094113280,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":1969687,"recovering_keys_per_sec":0,"num_objects_recovered":22,"num_bytes_recovered":23068672,"num_keys_recovered":0,"write_bytes_sec":62199025,"read_op_per_sec":0,"write_op_per_sec":63},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:10:59.874227+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:11:47.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 -- 172.21.15.133:0/561744847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07c8066f60 msgr2=0x7f07c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07c8066f60 0x7f07c8069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f07e80ca340 tx=0x7f07d8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:47.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 -- 172.21.15.133:0/561744847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e815db00 msgr2=0x7f07e815b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e815db00 0x7f07e815b5d0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0ad040 tx=0x7f07dc01ac90 comp rx=0 tx=0).stop 2024-04-25T10:11:47.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 -- 172.21.15.133:0/561744847 shutdown_connections 2024-04-25T10:11:47.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e81614c0 0x7f07e815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07c8066f60 0x7f07c8069420 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e815db00 0x7f07e815b5d0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 --2- 172.21.15.133:0/561744847 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e80d0770 0x7f07e80b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 -- 172.21.15.133:0/561744847 >> 172.21.15.133:0/561744847 conn(0x7f07e80727c0 msgr2=0x7f07e80d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:47.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 -- 172.21.15.133:0/561744847 shutdown_connections 2024-04-25T10:11:47.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:47.037+0000 7f07ecc02640 1 -- 172.21.15.133:0/561744847 wait complete. 2024-04-25T10:11:47.056 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:11:48.058 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.185+0000 7faf35a50640 1 Processor -- start 2024-04-25T10:11:48.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.185+0000 7faf35a50640 1 -- start start 2024-04-25T10:11:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf35a50640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faf3015dc80 0x7faf300ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf35a50640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 0x7faf300d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf35a50640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf300d1b30 con 0x7faf300d0e20 2024-04-25T10:11:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf35a50640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf300b95f0 con 0x7faf3015dc80 2024-04-25T10:11:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf35a50640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf300d12b0 con 0x7faf300ced20 2024-04-25T10:11:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf2effd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faf3015dc80 0x7faf300ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf267fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 0x7faf300d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf267fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 0x7faf300d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:40422/0 (socket says 172.21.15.133:40422) 2024-04-25T10:11:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf267fc640 1 -- 172.21.15.133:0/4107547968 learned_addr learned my addr 172.21.15.133:0/4107547968 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf267fc640 1 -- 172.21.15.133:0/4107547968 >> v2:172.21.15.162:3300/0 conn(0x7faf3015dc80 msgr2=0x7faf300ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf267fc640 1 --2- 172.21.15.133:0/4107547968 >> v2:172.21.15.162:3300/0 conn(0x7faf3015dc80 0x7faf300ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf267fc640 1 -- 172.21.15.133:0/4107547968 >> v1:172.21.15.162:6789/0 conn(0x7faf300d0e20 legacy=0x7faf300b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf267fc640 1 -- 172.21.15.133:0/4107547968 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3015e060 con 0x7faf300ced20 2024-04-25T10:11:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf2effd640 1 --2- 172.21.15.133:0/4107547968 >> v2:172.21.15.162:3300/0 conn(0x7faf3015dc80 0x7faf300ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf267fc640 1 --2- 172.21.15.133:0/4107547968 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 0x7faf300d08e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faf18009870 tx=0x7faf1801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f264c4b9a1f60c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf2e7fc640 1 -- 172.21.15.133:0/4107547968 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf180a95f0 con 0x7faf300ced20 2024-04-25T10:11:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf2e7fc640 1 -- 172.21.15.133:0/4107547968 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf180a9790 con 0x7faf300ced20 2024-04-25T10:11:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.189+0000 7faf2e7fc640 1 -- 172.21.15.133:0/4107547968 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf180a9b60 con 0x7faf300ced20 2024-04-25T10:11:48.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 -- 172.21.15.133:0/4107547968 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 msgr2=0x7faf300d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 --2- 172.21.15.133:0/4107547968 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 0x7faf300d08e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faf18009870 tx=0x7faf1801c530 comp rx=0 tx=0).stop 2024-04-25T10:11:48.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 -- 172.21.15.133:0/4107547968 shutdown_connections 2024-04-25T10:11:48.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 --2- 172.21.15.133:0/4107547968 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 0x7faf300d08e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 --2- 172.21.15.133:0/4107547968 >> v2:172.21.15.162:3300/0 conn(0x7faf3015dc80 0x7faf300ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 -- 172.21.15.133:0/4107547968 >> 172.21.15.133:0/4107547968 conn(0x7faf3006c550 msgr2=0x7faf3006c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 -- 172.21.15.133:0/4107547968 shutdown_connections 2024-04-25T10:11:48.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 -- 172.21.15.133:0/4107547968 wait complete. 2024-04-25T10:11:48.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 Processor -- start 2024-04-25T10:11:48.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 -- start start 2024-04-25T10:11:48.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 0x7faf30152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf300d0e20 0x7faf300b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3015dc80 0x7faf30149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf300b8e00 con 0x7faf3015dc80 2024-04-25T10:11:48.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf300d1da0 con 0x7faf300d0e20 2024-04-25T10:11:48.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3016a7d0 con 0x7faf300ced20 2024-04-25T10:11:48.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3015dc80 0x7faf30149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3015dc80 0x7faf30149ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34198/0 (socket says 172.21.15.133:34198) 2024-04-25T10:11:48.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2f7fe640 1 -- 172.21.15.133:0/3179425751 learned_addr learned my addr 172.21.15.133:0/3179425751 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2effd640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 0x7faf30152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf267fc640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf300d0e20 0x7faf300b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2f7fe640 1 -- 172.21.15.133:0/3179425751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 msgr2=0x7faf30152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2f7fe640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 0x7faf30152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2f7fe640 1 -- 172.21.15.133:0/3179425751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf300d0e20 msgr2=0x7faf300b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2f7fe640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf300d0e20 0x7faf300b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2f7fe640 1 -- 172.21.15.133:0/3179425751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3014e1e0 con 0x7faf3015dc80 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2f7fe640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3015dc80 0x7faf30149ba0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7faf2800c7a0 tx=0x7faf2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2cff9640 1 -- 172.21.15.133:0/3179425751 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28003b80 con 0x7faf3015dc80 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2cff9640 1 -- 172.21.15.133:0/3179425751 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf28003d20 con 0x7faf3015dc80 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 -- 172.21.15.133:0/3179425751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3014e4c0 con 0x7faf3015dc80 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf2cff9640 1 -- 172.21.15.133:0/3179425751 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800aa90 con 0x7faf3015dc80 2024-04-25T10:11:48.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.193+0000 7faf35a50640 1 -- 172.21.15.133:0/3179425751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3014ea80 con 0x7faf3015dc80 2024-04-25T10:11:48.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.197+0000 7faf35a50640 1 -- 172.21.15.133:0/3179425751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf30068ef0 con 0x7faf3015dc80 2024-04-25T10:11:48.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.197+0000 7faf2cff9640 1 -- 172.21.15.133:0/3179425751 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf2800ac30 con 0x7faf3015dc80 2024-04-25T10:11:48.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.197+0000 7faf2cff9640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faf10066f90 0x7faf10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.197+0000 7faf2cff9640 1 -- 172.21.15.133:0/3179425751 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7faf280850b0 con 0x7faf3015dc80 2024-04-25T10:11:48.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.197+0000 7faf2effd640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faf10066f90 0x7faf10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.197+0000 7faf2effd640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faf10066f90 0x7faf10069450 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7faf20009580 tx=0x7faf20009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.201+0000 7faf2cff9640 1 -- 172.21.15.133:0/3179425751 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf28055f00 con 0x7faf3015dc80 2024-04-25T10:11:48.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.385+0000 7faf35a50640 1 -- 172.21.15.133:0/3179425751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf300cbdc0 con 0x7faf3015dc80 2024-04-25T10:11:48.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.385+0000 7faf2cff9640 1 -- 172.21.15.133:0/3179425751 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7faf28055800 con 0x7faf3015dc80 2024-04-25T10:11:48.391 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:48.391 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:48.392 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 -- 172.21.15.133:0/3179425751 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faf10066f90 msgr2=0x7faf10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faf10066f90 0x7faf10069450 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7faf20009580 tx=0x7faf20009190 comp rx=0 tx=0).stop 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 -- 172.21.15.133:0/3179425751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3015dc80 msgr2=0x7faf30149ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3015dc80 0x7faf30149ba0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7faf2800c7a0 tx=0x7faf2800cc60 comp rx=0 tx=0).stop 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 -- 172.21.15.133:0/3179425751 shutdown_connections 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faf10066f90 0x7faf10069450 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf3015dc80 0x7faf30149ba0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faf300d0e20 0x7faf300b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 --2- 172.21.15.133:0/3179425751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faf300ced20 0x7faf30152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 -- 172.21.15.133:0/3179425751 >> 172.21.15.133:0/3179425751 conn(0x7faf3006c550 msgr2=0x7faf3015e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 -- 172.21.15.133:0/3179425751 shutdown_connections 2024-04-25T10:11:48.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.389+0000 7faf35a50640 1 -- 172.21.15.133:0/3179425751 wait complete. 2024-04-25T10:11:48.408 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:11:48.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:48.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.521+0000 7fe18044f640 1 Processor -- start 2024-04-25T10:11:48.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.521+0000 7fe18044f640 1 -- start start 2024-04-25T10:11:48.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.521+0000 7fe18044f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe1780b1f70 0x7fe17806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.521+0000 7fe18044f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1780c61d0 con 0x7fe1780ce330 2024-04-25T10:11:48.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.521+0000 7fe18044f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1780c64b0 con 0x7fe17815c9a0 2024-04-25T10:11:48.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.521+0000 7fe18044f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17806d4f0 con 0x7fe1780b1f70 2024-04-25T10:11:48.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.521+0000 7fe17e9ec640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe1780ce330 0x7fe17815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51682/0 (socket says 172.21.15.133:51682) 2024-04-25T10:11:48.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.521+0000 7fe17e9ec640 1 -- 172.21.15.133:0/3139194852 learned_addr learned my addr 172.21.15.133:0/3139194852 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:48.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d9ea640 1 --2- 172.21.15.133:0/3139194852 >> v2:172.21.15.162:3300/0 conn(0x7fe1780b1f70 0x7fe17806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498542936 0 0) 0x7fe1780c61d0 con 0x7fe1780ce330 2024-04-25T10:11:48.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe164003530 con 0x7fe1780ce330 2024-04-25T10:11:48.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 495681139 0 0) 0x7fe164003530 con 0x7fe1780ce330 2024-04-25T10:11:48.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 >> v2:172.21.15.162:3300/0 conn(0x7fe1780b1f70 msgr2=0x7fe17806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 --2- 172.21.15.133:0/3139194852 >> v2:172.21.15.162:3300/0 conn(0x7fe1780b1f70 0x7fe17806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 >> v1:172.21.15.162:6789/0 conn(0x7fe17815c9a0 legacy=0x7fe1780c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17806d7d0 con 0x7fe1780ce330 2024-04-25T10:11:48.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe170001a00 con 0x7fe1780ce330 2024-04-25T10:11:48.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 >> v1:172.21.15.133:6789/0 conn(0x7fe1780ce330 legacy=0x7fe17815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1640039c0 0x7fe164003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 0x7fe164004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1640051d0 0x7fe16400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17806d4f0 con 0x7fe164004810 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16400c070 con 0x7fe1640051d0 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16400c270 con 0x7fe1640039c0 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d9ea640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 0x7fe164004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17e1eb640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1640039c0 0x7fe164003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17e9ec640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1640051d0 0x7fe16400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d9ea640 1 -- 172.21.15.133:0/3139194852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1640039c0 msgr2=0x7fe164003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d9ea640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1640039c0 0x7fe164003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d9ea640 1 -- 172.21.15.133:0/3139194852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1640051d0 msgr2=0x7fe16400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d9ea640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1640051d0 0x7fe16400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d9ea640 1 -- 172.21.15.133:0/3139194852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe164003530 con 0x7fe164004810 2024-04-25T10:11:48.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17e9ec640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1640051d0 0x7fe16400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:48.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17e9ec640 1 -- 172.21.15.133:0/3139194852 reap_dead start 2024-04-25T10:11:48.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d9ea640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 0x7fe164004c90 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fe168009ae0 tx=0x7fe16801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211c424c0818b33b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1680b2070 con 0x7fe164004810 2024-04-25T10:11:48.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1680a5660 con 0x7fe164004810 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe17d1e9640 1 -- 172.21.15.133:0/3139194852 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1680096b0 con 0x7fe164004810 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe18044f640 1 -- 172.21.15.133:0/3139194852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 msgr2=0x7fe164004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe18044f640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 0x7fe164004c90 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fe168009ae0 tx=0x7fe16801ca70 comp rx=0 tx=0).stop 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe18044f640 1 -- 172.21.15.133:0/3139194852 shutdown_connections 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe18044f640 1 --2- 172.21.15.133:0/3139194852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 0x7fe164004c90 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe18044f640 1 -- 172.21.15.133:0/3139194852 >> 172.21.15.133:0/3139194852 conn(0x7fe178067000 msgr2=0x7fe17806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe18044f640 1 -- 172.21.15.133:0/3139194852 shutdown_connections 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.525+0000 7fe18044f640 1 -- 172.21.15.133:0/3139194852 wait complete. 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe18044f640 1 Processor -- start 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe18044f640 1 -- start start 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe18044f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 0x7fe17814e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe18044f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1780ca610 0x7fe1780b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe18044f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1780ac840 0x7fe1781516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe18044f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1780ce780 con 0x7fe164004810 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe18044f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1780ce900 con 0x7fe1780ac840 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe18044f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1780cea80 con 0x7fe1780ca610 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe17e1eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 0x7fe17814e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe17e1eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 0x7fe17814e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34202/0 (socket says 172.21.15.133:34202) 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe17e1eb640 1 -- 172.21.15.133:0/621078895 learned_addr learned my addr 172.21.15.133:0/621078895 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe17e9ec640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1780ac840 0x7fe1781516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe17d9ea640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1780ca610 0x7fe1780b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe17e1eb640 1 -- 172.21.15.133:0/621078895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1780ca610 msgr2=0x7fe1780b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe17e1eb640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1780ca610 0x7fe1780b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe17e1eb640 1 -- 172.21.15.133:0/621078895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1780ac840 msgr2=0x7fe1781516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe17e1eb640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1780ac840 0x7fe1781516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe17e1eb640 1 -- 172.21.15.133:0/621078895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1780c7ea0 con 0x7fe164004810 2024-04-25T10:11:48.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe17e1eb640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 0x7fe17814e630 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe17400e500 tx=0x7fe17400e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe16effd640 1 -- 172.21.15.133:0/621078895 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1740053a0 con 0x7fe164004810 2024-04-25T10:11:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe16effd640 1 -- 172.21.15.133:0/621078895 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe174018070 con 0x7fe164004810 2024-04-25T10:11:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe18044f640 1 -- 172.21.15.133:0/621078895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1780c8180 con 0x7fe164004810 2024-04-25T10:11:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe16effd640 1 -- 172.21.15.133:0/621078895 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174003840 con 0x7fe164004810 2024-04-25T10:11:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe18044f640 1 -- 172.21.15.133:0/621078895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1780b0d80 con 0x7fe164004810 2024-04-25T10:11:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.529+0000 7fe18044f640 1 -- 172.21.15.133:0/621078895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe178068ef0 con 0x7fe164004810 2024-04-25T10:11:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.533+0000 7fe16effd640 1 -- 172.21.15.133:0/621078895 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1740039e0 con 0x7fe164004810 2024-04-25T10:11:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.533+0000 7fe16effd640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe15c066f60 0x7fe15c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.533+0000 7fe16effd640 1 -- 172.21.15.133:0/621078895 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe174084ea0 con 0x7fe164004810 2024-04-25T10:11:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.533+0000 7fe17d9ea640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe15c066f60 0x7fe15c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.533+0000 7fe17d9ea640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe15c066f60 0x7fe15c069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe1680af000 tx=0x7fe168008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.537+0000 7fe16effd640 1 -- 172.21.15.133:0/621078895 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe174055d00 con 0x7fe164004810 2024-04-25T10:11:48.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 -- 172.21.15.133:0/621078895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe17806ddf0 con 0x7fe164004810 2024-04-25T10:11:48.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe16effd640 1 -- 172.21.15.133:0/621078895 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1446 (secure 0 0 0) 0x7fe174055600 con 0x7fe164004810 2024-04-25T10:11:48.760 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:48.760 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":621,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077446235,"bytes_used":2672754688,"bytes_avail":770421358592,"bytes_total":773094113280,"write_bytes_sec":42182095,"read_op_per_sec":0,"write_op_per_sec":43},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:10:59.874227+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:11:48.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 -- 172.21.15.133:0/621078895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe15c066f60 msgr2=0x7fe15c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe15c066f60 0x7fe15c069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe1680af000 tx=0x7fe168008040 comp rx=0 tx=0).stop 2024-04-25T10:11:48.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 -- 172.21.15.133:0/621078895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 msgr2=0x7fe17814e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 0x7fe17814e630 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe17400e500 tx=0x7fe17400e9c0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 -- 172.21.15.133:0/621078895 shutdown_connections 2024-04-25T10:11:48.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1780ac840 0x7fe1781516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe1780ca610 0x7fe1780b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe15c066f60 0x7fe15c069420 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 --2- 172.21.15.133:0/621078895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164004810 0x7fe17814e630 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 -- 172.21.15.133:0/621078895 >> 172.21.15.133:0/621078895 conn(0x7fe178067000 msgr2=0x7fe17806e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 -- 172.21.15.133:0/621078895 shutdown_connections 2024-04-25T10:11:48.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:48.757+0000 7fe18044f640 1 -- 172.21.15.133:0/621078895 wait complete. 2024-04-25T10:11:48.779 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:11:49.780 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:49.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.909+0000 7f7103d1d640 1 Processor -- start 2024-04-25T10:11:49.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.909+0000 7f7103d1d640 1 -- start start 2024-04-25T10:11:49.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7103d1d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70fc15c9a0 0x7f70fc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7103d1d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f70fc0b1f70 0x7f70fc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7103d1d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70fc0c61d0 con 0x7f70fc0ce330 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7103d1d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70fc0c64b0 con 0x7f70fc15c9a0 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7103d1d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70fc06d4f0 con 0x7f70fc0b1f70 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7101ab9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70fc15c9a0 0x7f70fc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7101ab9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70fc15c9a0 0x7f70fc0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34212/0 (socket says 172.21.15.133:34212) 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7101ab9640 1 -- 172.21.15.133:0/1719253930 learned_addr learned my addr 172.21.15.133:0/1719253930 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7101ab9640 1 -- 172.21.15.133:0/1719253930 >> v2:172.21.15.162:3300/0 conn(0x7f70fc0b1f70 msgr2=0x7f70fc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f71012b8640 1 --2- 172.21.15.133:0/1719253930 >> v2:172.21.15.162:3300/0 conn(0x7f70fc0b1f70 0x7f70fc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7101ab9640 1 --2- 172.21.15.133:0/1719253930 >> v2:172.21.15.162:3300/0 conn(0x7f70fc0b1f70 0x7f70fc06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7101ab9640 1 -- 172.21.15.133:0/1719253930 >> v1:172.21.15.162:6789/0 conn(0x7f70fc0ce330 legacy=0x7f70fc15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7101ab9640 1 -- 172.21.15.133:0/1719253930 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc06d7d0 con 0x7f70fc15c9a0 2024-04-25T10:11:49.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7101ab9640 1 --2- 172.21.15.133:0/1719253930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc15c9a0 0x7f70fc0c5980 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f70fc072670 tx=0x7f70f401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d4ddf2a71beaba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7100ab7640 1 -- 172.21.15.133:0/1719253930 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f40a5720 con 0x7f70fc15c9a0 2024-04-25T10:11:49.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7100ab7640 1 -- 172.21.15.133:0/1719253930 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f40a58c0 con 0x7f70fc15c9a0 2024-04-25T10:11:49.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.913+0000 7f7100ab7640 1 -- 172.21.15.133:0/1719253930 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f40a5c90 con 0x7f70fc15c9a0 2024-04-25T10:11:49.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 -- 172.21.15.133:0/1719253930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc15c9a0 msgr2=0x7f70fc0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 --2- 172.21.15.133:0/1719253930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc15c9a0 0x7f70fc0c5980 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f70fc072670 tx=0x7f70f401c950 comp rx=0 tx=0).stop 2024-04-25T10:11:49.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 -- 172.21.15.133:0/1719253930 shutdown_connections 2024-04-25T10:11:49.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 --2- 172.21.15.133:0/1719253930 >> v2:172.21.15.162:3300/0 conn(0x7f70fc0b1f70 0x7f70fc06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 --2- 172.21.15.133:0/1719253930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc15c9a0 0x7f70fc0c5980 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 -- 172.21.15.133:0/1719253930 >> 172.21.15.133:0/1719253930 conn(0x7f70fc067000 msgr2=0x7f70fc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 -- 172.21.15.133:0/1719253930 shutdown_connections 2024-04-25T10:11:49.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 -- 172.21.15.133:0/1719253930 wait complete. 2024-04-25T10:11:49.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 Processor -- start 2024-04-25T10:11:49.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 -- start start 2024-04-25T10:11:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70fc0b1f70 0x7f70fc148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc0ce330 0x7f70fc15b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc15c9a0 0x7f70fc15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc15cd80 con 0x7f70fc0ce330 2024-04-25T10:11:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc06e230 con 0x7f70fc0b1f70 2024-04-25T10:11:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc15cf00 con 0x7f70fc15c9a0 2024-04-25T10:11:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f71012b8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc0ce330 0x7f70fc15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f71012b8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc0ce330 0x7f70fc15b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34228/0 (socket says 172.21.15.133:34228) 2024-04-25T10:11:49.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f71012b8640 1 -- 172.21.15.133:0/432753312 learned_addr learned my addr 172.21.15.133:0/432753312 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7101ab9640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70fc0b1f70 0x7f70fc148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f71012b8640 1 -- 172.21.15.133:0/432753312 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc15c9a0 msgr2=0x7f70fc15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f71022ba640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc15c9a0 0x7f70fc15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f71012b8640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc15c9a0 0x7f70fc15e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f71012b8640 1 -- 172.21.15.133:0/432753312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70fc0b1f70 msgr2=0x7f70fc148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f71012b8640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70fc0b1f70 0x7f70fc148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f71012b8640 1 -- 172.21.15.133:0/432753312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc15bb20 con 0x7f70fc0ce330 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7101ab9640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70fc0b1f70 0x7f70fc148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f71012b8640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc0ce330 0x7f70fc15b490 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00c7f0 tx=0x7f70ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f70f27fc640 1 -- 172.21.15.133:0/432753312 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec015070 con 0x7f70fc0ce330 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f70f27fc640 1 -- 172.21.15.133:0/432753312 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec007820 con 0x7f70fc0ce330 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f70f27fc640 1 -- 172.21.15.133:0/432753312 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec007c10 con 0x7f70fc0ce330 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 -- 172.21.15.133:0/432753312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc0c9690 con 0x7f70fc0ce330 2024-04-25T10:11:49.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.917+0000 7f7103d1d640 1 -- 172.21.15.133:0/432753312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc0c9c50 con 0x7f70fc0ce330 2024-04-25T10:11:49.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.921+0000 7f7103d1d640 1 -- 172.21.15.133:0/432753312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70fc06bc20 con 0x7f70fc0ce330 2024-04-25T10:11:49.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.921+0000 7f70f27fc640 1 -- 172.21.15.133:0/432753312 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70ec018070 con 0x7f70fc0ce330 2024-04-25T10:11:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.921+0000 7f70f27fc640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e0066f90 0x7f70e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.921+0000 7f70f27fc640 1 -- 172.21.15.133:0/432753312 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f70ec085240 con 0x7f70fc0ce330 2024-04-25T10:11:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.921+0000 7f7101ab9640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e0066f90 0x7f70e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.921+0000 7f7101ab9640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e0066f90 0x7f70e0069450 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f70f40ae000 tx=0x7f70f40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:49.925+0000 7f70f27fc640 1 -- 172.21.15.133:0/432753312 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70ec056090 con 0x7f70fc0ce330 2024-04-25T10:11:50.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.117+0000 7f7103d1d640 1 -- 172.21.15.133:0/432753312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70fc068ef0 con 0x7f70fc0ce330 2024-04-25T10:11:50.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f70f27fc640 1 -- 172.21.15.133:0/432753312 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f70ec055990 con 0x7f70fc0ce330 2024-04-25T10:11:50.122 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:50.123 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:50.123 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 -- 172.21.15.133:0/432753312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e0066f90 msgr2=0x7f70e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e0066f90 0x7f70e0069450 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f70f40ae000 tx=0x7f70f40a7070 comp rx=0 tx=0).stop 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 -- 172.21.15.133:0/432753312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc0ce330 msgr2=0x7f70fc15b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc0ce330 0x7f70fc15b490 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00c7f0 tx=0x7f70ec00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 -- 172.21.15.133:0/432753312 shutdown_connections 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70fc15c9a0 0x7f70fc15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f70e0066f90 0x7f70e0069450 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70fc0ce330 0x7f70fc15b490 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 --2- 172.21.15.133:0/432753312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f70fc0b1f70 0x7f70fc148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 -- 172.21.15.133:0/432753312 >> 172.21.15.133:0/432753312 conn(0x7f70fc067000 msgr2=0x7f70fc153590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 -- 172.21.15.133:0/432753312 shutdown_connections 2024-04-25T10:11:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.121+0000 7f7103d1d640 1 -- 172.21.15.133:0/432753312 wait complete. 2024-04-25T10:11:50.140 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:11:50.141 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:50.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff974e75640 1 Processor -- start 2024-04-25T10:11:50.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff974e75640 1 -- start start 2024-04-25T10:11:50.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff974e75640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff97015c9a0 0x7ff9700c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff974e75640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9700c61d0 con 0x7ff9700ce330 2024-04-25T10:11:50.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff974e75640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9700c64b0 con 0x7ff9700b1f70 2024-04-25T10:11:50.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff974e75640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff97006d4f0 con 0x7ff97015c9a0 2024-04-25T10:11:50.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff96e59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff97015c9a0 0x7ff9700c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff96ed9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff9700ce330 0x7ff97015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51686/0 (socket says 172.21.15.133:51686) 2024-04-25T10:11:50.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff96ed9d640 1 -- 172.21.15.133:0/302944678 learned_addr learned my addr 172.21.15.133:0/302944678 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:50.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff96e59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff97015c9a0 0x7ff9700c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34238/0 (socket says 172.21.15.133:34238) 2024-04-25T10:11:50.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff96e59c640 1 -- 172.21.15.133:0/302944678 >> v1:172.21.15.162:6789/0 conn(0x7ff9700b1f70 legacy=0x7ff97006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff96e59c640 1 -- 172.21.15.133:0/302944678 >> v1:172.21.15.133:6789/0 conn(0x7ff9700ce330 legacy=0x7ff97015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.245+0000 7ff96e59c640 1 -- 172.21.15.133:0/302944678 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97006d7d0 con 0x7ff97015c9a0 2024-04-25T10:11:50.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.249+0000 7ff96e59c640 1 --2- 172.21.15.133:0/302944678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff97015c9a0 0x7ff9700c5980 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff9680097c0 tx=0x7ff96801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72b2a04e8fe73b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.249+0000 7ff96d59a640 1 -- 172.21.15.133:0/302944678 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9680b0070 con 0x7ff97015c9a0 2024-04-25T10:11:50.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.249+0000 7ff96d59a640 1 -- 172.21.15.133:0/302944678 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9680ab600 con 0x7ff97015c9a0 2024-04-25T10:11:50.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.249+0000 7ff96d59a640 1 -- 172.21.15.133:0/302944678 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9680ab9d0 con 0x7ff97015c9a0 2024-04-25T10:11:50.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.249+0000 7ff974e75640 1 -- 172.21.15.133:0/302944678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff97015c9a0 msgr2=0x7ff9700c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.249+0000 7ff974e75640 1 --2- 172.21.15.133:0/302944678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff97015c9a0 0x7ff9700c5980 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff9680097c0 tx=0x7ff96801c460 comp rx=0 tx=0).stop 2024-04-25T10:11:50.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.249+0000 7ff974e75640 1 -- 172.21.15.133:0/302944678 shutdown_connections 2024-04-25T10:11:50.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.249+0000 7ff974e75640 1 --2- 172.21.15.133:0/302944678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff97015c9a0 0x7ff9700c5980 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.249+0000 7ff974e75640 1 -- 172.21.15.133:0/302944678 >> 172.21.15.133:0/302944678 conn(0x7ff970067000 msgr2=0x7ff97006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.249+0000 7ff974e75640 1 -- 172.21.15.133:0/302944678 shutdown_connections 2024-04-25T10:11:50.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.249+0000 7ff974e75640 1 -- 172.21.15.133:0/302944678 wait complete. 2024-04-25T10:11:50.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff974e75640 1 Processor -- start 2024-04-25T10:11:50.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff974e75640 1 -- start start 2024-04-25T10:11:50.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff974e75640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9700b1f70 0x7ff97014c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff974e75640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9700ce330 0x7ff97014b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff974e75640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff97015c9a0 0x7ff970147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff974e75640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97015cf00 con 0x7ff9700ce330 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff974e75640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97015cd80 con 0x7ff9700b1f70 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff974e75640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff970169de0 con 0x7ff97015c9a0 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff96dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9700ce330 0x7ff97014b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff96dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9700ce330 0x7ff97014b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34248/0 (socket says 172.21.15.133:34248) 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff96dd9b640 1 -- 172.21.15.133:0/3551963562 learned_addr learned my addr 172.21.15.133:0/3551963562 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff96dd9b640 1 -- 172.21.15.133:0/3551963562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff97015c9a0 msgr2=0x7ff970147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff96dd9b640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff97015c9a0 0x7ff970147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff96dd9b640 1 -- 172.21.15.133:0/3551963562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9700b1f70 msgr2=0x7ff97014c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff96dd9b640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9700b1f70 0x7ff97014c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff96dd9b640 1 -- 172.21.15.133:0/3551963562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9700c8f50 con 0x7ff9700ce330 2024-04-25T10:11:50.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff96dd9b640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9700ce330 0x7ff97014b9b0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7ff96400ba80 tx=0x7ff96400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff95effd640 1 -- 172.21.15.133:0/3551963562 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964019070 con 0x7ff9700ce330 2024-04-25T10:11:50.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff95effd640 1 -- 172.21.15.133:0/3551963562 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96400cc90 con 0x7ff9700ce330 2024-04-25T10:11:50.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff974e75640 1 -- 172.21.15.133:0/3551963562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff97014c870 con 0x7ff9700ce330 2024-04-25T10:11:50.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff95effd640 1 -- 172.21.15.133:0/3551963562 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964014680 con 0x7ff9700ce330 2024-04-25T10:11:50.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff974e75640 1 -- 172.21.15.133:0/3551963562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9700c94d0 con 0x7ff9700ce330 2024-04-25T10:11:50.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff974e75640 1 -- 172.21.15.133:0/3551963562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff970068ef0 con 0x7ff9700ce330 2024-04-25T10:11:50.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.253+0000 7ff95effd640 1 -- 172.21.15.133:0/3551963562 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff964007370 con 0x7ff9700ce330 2024-04-25T10:11:50.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.257+0000 7ff95effd640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff950066f90 0x7ff950069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.257+0000 7ff95effd640 1 -- 172.21.15.133:0/3551963562 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff964084da0 con 0x7ff9700ce330 2024-04-25T10:11:50.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.257+0000 7ff96e59c640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff950066f90 0x7ff950069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.257+0000 7ff96e59c640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff950066f90 0x7ff950069450 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff9680ad000 tx=0x7ff968008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.261+0000 7ff95effd640 1 -- 172.21.15.133:0/3551963562 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff964055c80 con 0x7ff9700ce330 2024-04-25T10:11:50.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.469+0000 7ff974e75640 1 -- 172.21.15.133:0/3551963562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff9700b3ba0 con 0x7ff9700ce330 2024-04-25T10:11:50.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.469+0000 7ff95effd640 1 -- 172.21.15.133:0/3551963562 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7ff964055580 con 0x7ff9700ce330 2024-04-25T10:11:50.473 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:50.473 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":623,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078164813,"bytes_used":2674196480,"bytes_avail":770419916800,"bytes_total":773094113280,"write_bytes_sec":42242578,"read_op_per_sec":0,"write_op_per_sec":44},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:11:49.888630+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:11:50.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 -- 172.21.15.133:0/3551963562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff950066f90 msgr2=0x7ff950069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff950066f90 0x7ff950069450 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff9680ad000 tx=0x7ff968008040 comp rx=0 tx=0).stop 2024-04-25T10:11:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 -- 172.21.15.133:0/3551963562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9700ce330 msgr2=0x7ff97014b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9700ce330 0x7ff97014b9b0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7ff96400ba80 tx=0x7ff96400bf40 comp rx=0 tx=0).stop 2024-04-25T10:11:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 -- 172.21.15.133:0/3551963562 shutdown_connections 2024-04-25T10:11:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff97015c9a0 0x7ff970147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff950066f90 0x7ff950069450 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9700ce330 0x7ff97014b9b0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 --2- 172.21.15.133:0/3551963562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff9700b1f70 0x7ff97014c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 -- 172.21.15.133:0/3551963562 >> 172.21.15.133:0/3551963562 conn(0x7ff970067000 msgr2=0x7ff9700b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 -- 172.21.15.133:0/3551963562 shutdown_connections 2024-04-25T10:11:50.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.473+0000 7ff974e75640 1 -- 172.21.15.133:0/3551963562 wait complete. 2024-04-25T10:11:50.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:11:50.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:50.492 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T10:11:50.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45d0c5640 1 Processor -- start 2024-04-25T10:11:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45d0c5640 1 -- start start 2024-04-25T10:11:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45d0c5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4580cd820 0x7fc4580cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45d0c5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc45815ced0 0x7fc45815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45d0c5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4580ce2e0 con 0x7fc458145170 2024-04-25T10:11:50.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45d0c5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45806db00 con 0x7fc4580cd820 2024-04-25T10:11:50.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45d0c5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc458145550 con 0x7fc45815ced0 2024-04-25T10:11:50.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4580cd820 0x7fc4580cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc456d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc458145170 0x7fc45806df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51694/0 (socket says 172.21.15.133:51694) 2024-04-25T10:11:50.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc456d9d640 1 -- 172.21.15.133:0/3956512842 learned_addr learned my addr 172.21.15.133:0/3956512842 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:50.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45759e640 1 --2- 172.21.15.133:0/3956512842 >> v2:172.21.15.162:3300/0 conn(0x7fc45815ced0 0x7fc45815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45659c640 1 -- 172.21.15.133:0/3956512842 >> v2:172.21.15.162:3300/0 conn(0x7fc45815ced0 msgr2=0x7fc45815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45659c640 1 --2- 172.21.15.133:0/3956512842 >> v2:172.21.15.162:3300/0 conn(0x7fc45815ced0 0x7fc45815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45659c640 1 -- 172.21.15.133:0/3956512842 >> v1:172.21.15.133:6789/0 conn(0x7fc458145170 legacy=0x7fc45806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45659c640 1 -- 172.21.15.133:0/3956512842 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45806d4b0 con 0x7fc4580cd820 2024-04-25T10:11:50.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45759e640 1 --2- 172.21.15.133:0/3956512842 >> v2:172.21.15.162:3300/0 conn(0x7fc45815ced0 0x7fc45815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc45659c640 1 --2- 172.21.15.133:0/3956512842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4580cd820 0x7fc4580cdca0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fc4580b27f0 tx=0x7fc44c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732e8a5c9c4312a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.613+0000 7fc455d9b640 1 -- 172.21.15.133:0/3956512842 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0b0070 con 0x7fc4580cd820 2024-04-25T10:11:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc455d9b640 1 -- 172.21.15.133:0/3956512842 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44c0ab600 con 0x7fc4580cd820 2024-04-25T10:11:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc455d9b640 1 -- 172.21.15.133:0/3956512842 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0ab9d0 con 0x7fc4580cd820 2024-04-25T10:11:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 -- 172.21.15.133:0/3956512842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4580cd820 msgr2=0x7fc4580cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 --2- 172.21.15.133:0/3956512842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4580cd820 0x7fc4580cdca0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fc4580b27f0 tx=0x7fc44c01c480 comp rx=0 tx=0).stop 2024-04-25T10:11:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 -- 172.21.15.133:0/3956512842 shutdown_connections 2024-04-25T10:11:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 --2- 172.21.15.133:0/3956512842 >> v2:172.21.15.162:3300/0 conn(0x7fc45815ced0 0x7fc45815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 --2- 172.21.15.133:0/3956512842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4580cd820 0x7fc4580cdca0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 -- 172.21.15.133:0/3956512842 >> 172.21.15.133:0/3956512842 conn(0x7fc458066f70 msgr2=0x7fc45806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 -- 172.21.15.133:0/3956512842 shutdown_connections 2024-04-25T10:11:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 -- 172.21.15.133:0/3956512842 wait complete. 2024-04-25T10:11:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 Processor -- start 2024-04-25T10:11:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 -- start start 2024-04-25T10:11:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4580cd820 0x7fc45813e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc458145170 0x7fc458143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45815ced0 0x7fc4581315d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc458168610 con 0x7fc45815ced0 2024-04-25T10:11:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc458168490 con 0x7fc4580cd820 2024-04-25T10:11:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc458168790 con 0x7fc458145170 2024-04-25T10:11:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45815ced0 0x7fc4581315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45815ced0 0x7fc4581315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34276/0 (socket says 172.21.15.133:34276) 2024-04-25T10:11:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45759e640 1 -- 172.21.15.133:0/1908553989 learned_addr learned my addr 172.21.15.133:0/1908553989 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc456d9d640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4580cd820 0x7fc45813e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45659c640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc458145170 0x7fc458143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45759e640 1 -- 172.21.15.133:0/1908553989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc458145170 msgr2=0x7fc458143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45759e640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc458145170 0x7fc458143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45759e640 1 -- 172.21.15.133:0/1908553989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4580cd820 msgr2=0x7fc45813e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45759e640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4580cd820 0x7fc45813e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45759e640 1 -- 172.21.15.133:0/1908553989 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4581441d0 con 0x7fc45815ced0 2024-04-25T10:11:50.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45659c640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc458145170 0x7fc458143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45759e640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45815ced0 0x7fc4581315d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc45000cd50 tx=0x7fc45000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc43f7fe640 1 -- 172.21.15.133:0/1908553989 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45000f680 con 0x7fc45815ced0 2024-04-25T10:11:50.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc43f7fe640 1 -- 172.21.15.133:0/1908553989 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45000f840 con 0x7fc45815ced0 2024-04-25T10:11:50.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc43f7fe640 1 -- 172.21.15.133:0/1908553989 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45000fc30 con 0x7fc45815ced0 2024-04-25T10:11:50.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 -- 172.21.15.133:0/1908553989 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4581444a0 con 0x7fc45815ced0 2024-04-25T10:11:50.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.617+0000 7fc45d0c5640 1 -- 172.21.15.133:0/1908553989 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4580af6c0 con 0x7fc45815ced0 2024-04-25T10:11:50.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc43f7fe640 1 -- 172.21.15.133:0/1908553989 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc45000e3e0 con 0x7fc45815ced0 2024-04-25T10:11:50.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc43f7fe640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc444066f60 0x7fc444069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc43f7fe640 1 -- 172.21.15.133:0/1908553989 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc450086020 con 0x7fc45815ced0 2024-04-25T10:11:50.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc456d9d640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc444066f60 0x7fc444069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc456d9d640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc444066f60 0x7fc444069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc448000f70 tx=0x7fc448009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc460232000 1 Processor -- start 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc460232000 1 -- start start 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc460232000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad069d0 0x556edad06db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc460232000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edad072f0 0x556edacbcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc460232000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edacbd0a0 0x556edacbd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc460232000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edab09ff0 con 0x556edad069d0 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc460232000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edaa92150 con 0x556edacbd0a0 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc460232000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556edab06090 con 0x556edad072f0 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc456d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad069d0 0x556edad06db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc456d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad069d0 0x556edad06db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34280/0 (socket says 172.21.15.133:34280) 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc456d9d640 1 -- 172.21.15.133:0/3741210413 learned_addr learned my addr 172.21.15.133:0/3741210413 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc456d9d640 1 -- 172.21.15.133:0/3741210413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edad072f0 msgr2=0x556edacbcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc456d9d640 1 --2- 172.21.15.133:0/3741210413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edad072f0 0x556edacbcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc456d9d640 1 -- 172.21.15.133:0/3741210413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edacbd0a0 msgr2=0x556edacbd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc456d9d640 1 --2- 172.21.15.133:0/3741210413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edacbd0a0 0x556edacbd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc456d9d640 1 -- 172.21.15.133:0/3741210413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556edaca00d0 con 0x556edad069d0 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.621+0000 7fc456d9d640 1 --2- 172.21.15.133:0/3741210413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad069d0 0x556edad06db0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc448005c70 tx=0x7fc448010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b05fa4efed27ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc43cff9640 1 -- 172.21.15.133:0/3741210413 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448007760 con 0x556edad069d0 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc43cff9640 1 -- 172.21.15.133:0/3741210413 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44800da20 con 0x556edad069d0 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc43cff9640 1 -- 172.21.15.133:0/3741210413 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44800ddd0 con 0x556edad069d0 2024-04-25T10:11:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- 172.21.15.133:0/3741210413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad069d0 msgr2=0x556edad06db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 --2- 172.21.15.133:0/3741210413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad069d0 0x556edad06db0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc448005c70 tx=0x7fc448010000 comp rx=0 tx=0).stop 2024-04-25T10:11:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- 172.21.15.133:0/3741210413 shutdown_connections 2024-04-25T10:11:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 --2- 172.21.15.133:0/3741210413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edacbd0a0 0x556edacbd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 --2- 172.21.15.133:0/3741210413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edad072f0 0x556edacbcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 --2- 172.21.15.133:0/3741210413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad069d0 0x556edad06db0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- 172.21.15.133:0/3741210413 >> 172.21.15.133:0/3741210413 conn(0x556edacb9020 msgr2=0x556edacbb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- 172.21.15.133:0/3741210413 shutdown_connections 2024-04-25T10:11:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- 172.21.15.133:0/3741210413 wait complete. 2024-04-25T10:11:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- start start 2024-04-25T10:11:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 Processor -- start 2024-04-25T10:11:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edacbd0a0 0x556edaceb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edacebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edacec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edaace250 con 0x556edad072f0 2024-04-25T10:11:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edaac4690 con 0x556edad069d0 2024-04-25T10:11:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556edaa92a40 con 0x556edacbd0a0 2024-04-25T10:11:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 10 client.0 _ll_register_callbacks cb 0x556edaca7e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc45759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edacec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc45759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edacec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34284/0 (socket says 172.21.15.133:34284) 2024-04-25T10:11:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc45759e640 1 -- 172.21.15.133:0/959956249 learned_addr learned my addr 172.21.15.133:0/959956249 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc456d9d640 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edacbd0a0 0x556edaceb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc45659c640 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edacebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc45759e640 1 -- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edacbd0a0 msgr2=0x556edaceb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc45759e640 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edacbd0a0 0x556edaceb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc45759e640 1 -- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 msgr2=0x556edacebd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc45759e640 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edacebd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc45759e640 1 -- 172.21.15.133:0/959956249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556edabb71c0 con 0x556edad072f0 2024-04-25T10:11:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc456d9d640 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edacbd0a0 0x556edaceb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc45659c640 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edacebd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc45759e640 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edacec270 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fc45005e930 tx=0x7fc450056800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62e4eb25babe5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc435ffb640 1 -- 172.21.15.133:0/959956249 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45001d040 con 0x556edad072f0 2024-04-25T10:11:50.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc435ffb640 1 -- 172.21.15.133:0/959956249 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc45006e120 con 0x556edad072f0 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- 172.21.15.133:0/959956249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc44800da20 con 0x556edad072f0 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc435ffb640 1 -- 172.21.15.133:0/959956249 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc45005e9d0 con 0x556edad072f0 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 10 client.8625 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- 172.21.15.133:0/959956249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x556edabb3170 con 0x556edad072f0 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc435ffb640 1 -- 172.21.15.133:0/959956249 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44800da20 con 0x556edad072f0 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc435ffb640 1 -- 172.21.15.133:0/959956249 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc450055b10 con 0x556edad072f0 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 10 client.8625 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 10 client.8625 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 20 client.8625 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x556edad16c80 0x556edad19070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 4 client.8625 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc460232000 1 -- 172.21.15.133:0/959956249 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556edaaa5c60 con 0x556edad16c80 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.625+0000 7fc456d9d640 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x556edad16c80 0x556edad19070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc435ffb640 10 client.8625 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc456d9d640 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x556edad16c80 0x556edad19070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d607fe43f56f347 server_cookie=e5ab850a28ac509c in_seq=0 out_seq=0 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc435ffb640 1 -- 172.21.15.133:0/959956249 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556edaaa5c60 con 0x556edad16c80 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc435ffb640 10 client.8625 handle_command_reply: tid=0 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 1 client.8625 shutdown 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 1 -- 172.21.15.133:0/959956249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 msgr2=0x556edacec270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edacec270 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fc45005e930 tx=0x7fc450056800 comp rx=0 tx=0).stop 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 1 -- 172.21.15.133:0/959956249 shutdown_connections 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x556edad16c80 0x556edad19070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edacec270 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc435ffb640 0 client.8625 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edacebd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 1 --2- 172.21.15.133:0/959956249 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edacbd0a0 0x556edaceb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 1 -- 172.21.15.133:0/959956249 >> 172.21.15.133:0/959956249 conn(0x556edacb9020 msgr2=0x556edacbbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 1 -- 172.21.15.133:0/959956249 shutdown_connections 2024-04-25T10:11:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 1 -- 172.21.15.133:0/959956249 wait complete. 2024-04-25T10:11:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.629+0000 7fc460232000 20 client.8625 trim_cache size 0 max 16384 2024-04-25T10:11:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 Processor -- start 2024-04-25T10:11:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- start start 2024-04-25T10:11:50.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edacbd0a0 0x556edad1b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edad13e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edad14380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc458203510 con 0x556edad072f0 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc458202710 con 0x556edad069d0 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580b0c30 con 0x556edacbd0a0 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc45759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edad14380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc45759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edad14380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34294/0 (socket says 172.21.15.133:34294) 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc45759e640 1 -- 172.21.15.133:0/3478225608 learned_addr learned my addr 172.21.15.133:0/3478225608 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc45759e640 1 -- 172.21.15.133:0/3478225608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edacbd0a0 msgr2=0x556edad1b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc456d9d640 1 --2- 172.21.15.133:0/3478225608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edacbd0a0 0x556edad1b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc45759e640 1 --2- 172.21.15.133:0/3478225608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edacbd0a0 0x556edad1b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc45659c640 1 --2- 172.21.15.133:0/3478225608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edad13e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc45759e640 1 -- 172.21.15.133:0/3478225608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 msgr2=0x556edad13e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc45759e640 1 --2- 172.21.15.133:0/3478225608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edad13e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc45759e640 1 -- 172.21.15.133:0/3478225608 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556edab50de0 con 0x556edad072f0 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc45759e640 1 --2- 172.21.15.133:0/3478225608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edad14380 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fc45000cfd0 tx=0x7fc450019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716d7fe8521caf4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc437fff640 1 -- 172.21.15.133:0/3478225608 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45001d040 con 0x556edad072f0 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc437fff640 1 -- 172.21.15.133:0/3478225608 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45006e120 con 0x556edad072f0 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc437fff640 1 -- 172.21.15.133:0/3478225608 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450052410 con 0x556edad072f0 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- 172.21.15.133:0/3478225608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 msgr2=0x556edad14380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 --2- 172.21.15.133:0/3478225608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edad14380 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fc45000cfd0 tx=0x7fc450019000 comp rx=0 tx=0).stop 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- 172.21.15.133:0/3478225608 shutdown_connections 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 --2- 172.21.15.133:0/3478225608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edad072f0 0x556edad14380 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 --2- 172.21.15.133:0/3478225608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edad13e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 --2- 172.21.15.133:0/3478225608 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edacbd0a0 0x556edad1b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- 172.21.15.133:0/3478225608 >> 172.21.15.133:0/3478225608 conn(0x556edacb9020 msgr2=0x556edad13730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- 172.21.15.133:0/3478225608 shutdown_connections 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- 172.21.15.133:0/3478225608 wait complete. 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- start start 2024-04-25T10:11:50.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 Processor -- start 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edacbd0a0 0x556edad66720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edad66c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edad072f0 0x556edad18a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edab09300 con 0x556edacbd0a0 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556edaaa4e00 con 0x556edad069d0 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556edab0e620 con 0x556edad072f0 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.645+0000 7fc460232000 10 client.0 _ll_register_callbacks cb 0x556edac88040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc456d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edacbd0a0 0x556edad66720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc456d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edacbd0a0 0x556edad66720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34308/0 (socket says 172.21.15.133:34308) 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc456d9d640 1 -- 172.21.15.133:0/2384072330 learned_addr learned my addr 172.21.15.133:0/2384072330 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc456d9d640 1 -- 172.21.15.133:0/2384072330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edad072f0 msgr2=0x556edad18a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc456d9d640 1 --2- 172.21.15.133:0/2384072330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edad072f0 0x556edad18a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc456d9d640 1 -- 172.21.15.133:0/2384072330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 msgr2=0x556edad66c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc456d9d640 1 --2- 172.21.15.133:0/2384072330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edad66c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc456d9d640 1 -- 172.21.15.133:0/2384072330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556edaaab8e0 con 0x556edacbd0a0 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc456d9d640 1 --2- 172.21.15.133:0/2384072330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edacbd0a0 0x556edad66720 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fc448005fd0 tx=0x7fc448000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb932cddc8b8d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc435ffb640 1 -- 172.21.15.133:0/2384072330 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44800dbb0 con 0x556edacbd0a0 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc435ffb640 1 -- 172.21.15.133:0/2384072330 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc44800dd50 con 0x556edacbd0a0 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 -- 172.21.15.133:0/2384072330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc45006e120 con 0x556edacbd0a0 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc435ffb640 1 -- 172.21.15.133:0/2384072330 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc44802c330 con 0x556edacbd0a0 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 10 client.8631 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 -- 172.21.15.133:0/2384072330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x556edab638f0 con 0x556edacbd0a0 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc435ffb640 1 -- 172.21.15.133:0/2384072330 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc448030030 con 0x556edacbd0a0 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc435ffb640 1 -- 172.21.15.133:0/2384072330 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc44802c4d0 con 0x556edacbd0a0 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 10 client.8631 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 10 client.8631 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:11:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 20 client.8631 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 --2- 172.21.15.133:0/2384072330 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x556edad19380 0x556edad19790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 4 client.8631 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:11:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 -- 172.21.15.133:0/2384072330 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x556edab17530 con 0x556edad19380 2024-04-25T10:11:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc45659c640 1 --2- 172.21.15.133:0/2384072330 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x556edad19380 0x556edad19790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc435ffb640 10 client.8631 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:50.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc45659c640 1 --2- 172.21.15.133:0/2384072330 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x556edad19380 0x556edad19790 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=885c4e5e26cb1896 server_cookie=8768e94853b59d40 in_seq=0 out_seq=0 2024-04-25T10:11:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc435ffb640 1 -- 172.21.15.133:0/2384072330 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x556edab17530 con 0x556edad19380 2024-04-25T10:11:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc435ffb640 10 client.8631 handle_command_reply: tid=0 2024-04-25T10:11:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 client.8631 shutdown 2024-04-25T10:11:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 -- 172.21.15.133:0/2384072330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edacbd0a0 msgr2=0x556edad66720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 --2- 172.21.15.133:0/2384072330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edacbd0a0 0x556edad66720 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fc448005fd0 tx=0x7fc448000bd0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 -- 172.21.15.133:0/2384072330 shutdown_connections 2024-04-25T10:11:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 --2- 172.21.15.133:0/2384072330 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x556edad19380 0x556edad19790 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 --2- 172.21.15.133:0/2384072330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556edad072f0 0x556edad18a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 --2- 172.21.15.133:0/2384072330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556edad069d0 0x556edad66c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 --2- 172.21.15.133:0/2384072330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556edacbd0a0 0x556edad66720 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc435ffb640 0 client.8631 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 -- 172.21.15.133:0/2384072330 >> 172.21.15.133:0/2384072330 conn(0x556edacb9020 msgr2=0x556edad13fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 -- 172.21.15.133:0/2384072330 shutdown_connections 2024-04-25T10:11:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 1 -- 172.21.15.133:0/2384072330 wait complete. 2024-04-25T10:11:50.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.649+0000 7fc460232000 20 client.8631 trim_cache size 0 max 16384 2024-04-25T10:11:50.659 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:11:50.659 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:11:50.660 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:11:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 -- 172.21.15.133:0/1908553989 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc444066f60 msgr2=0x7fc444069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc444066f60 0x7fc444069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc448000f70 tx=0x7fc448009040 comp rx=0 tx=0).stop 2024-04-25T10:11:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 -- 172.21.15.133:0/1908553989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45815ced0 msgr2=0x7fc4581315d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45815ced0 0x7fc4581315d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc45000cd50 tx=0x7fc45000a650 comp rx=0 tx=0).stop 2024-04-25T10:11:50.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 -- 172.21.15.133:0/1908553989 shutdown_connections 2024-04-25T10:11:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc444066f60 0x7fc444069420 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45815ced0 0x7fc4581315d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc458145170 0x7fc458143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 --2- 172.21.15.133:0/1908553989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4580cd820 0x7fc45813e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 -- 172.21.15.133:0/1908553989 >> 172.21.15.133:0/1908553989 conn(0x7fc458066f70 msgr2=0x7fc45815f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 -- 172.21.15.133:0/1908553989 shutdown_connections 2024-04-25T10:11:50.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.653+0000 7fc45d0c5640 1 -- 172.21.15.133:0/1908553989 wait complete. 2024-04-25T10:11:50.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:50.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:11:50.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.781+0000 7f5abc7aa640 1 Processor -- start 2024-04-25T10:11:50.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.781+0000 7f5abc7aa640 1 -- start start 2024-04-25T10:11:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5abc7aa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5ab415d9a0 0x7f5ab40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5abc7aa640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab415ebd0 con 0x7f5ab40ce600 2024-04-25T10:11:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5abc7aa640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab415eeb0 con 0x7f5ab40d2030 2024-04-25T10:11:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5abc7aa640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab40d0330 con 0x7f5ab415d9a0 2024-04-25T10:11:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5aba546640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5ab415d9a0 0x7f5ab40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5ab9d45640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5ab40ce600 0x7f5ab40d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51698/0 (socket says 172.21.15.133:51698) 2024-04-25T10:11:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5ab9d45640 1 -- 172.21.15.133:0/557115849 learned_addr learned my addr 172.21.15.133:0/557115849 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5aba546640 1 -- 172.21.15.133:0/557115849 >> v1:172.21.15.162:6789/0 conn(0x7f5ab40d2030 legacy=0x7f5ab415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5aba546640 1 -- 172.21.15.133:0/557115849 >> v1:172.21.15.133:6789/0 conn(0x7f5ab40ce600 legacy=0x7f5ab40d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5aba546640 1 -- 172.21.15.133:0/557115849 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab40d0610 con 0x7f5ab415d9a0 2024-04-25T10:11:50.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5aba546640 1 --2- 172.21.15.133:0/557115849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab415d9a0 0x7f5ab40d0b30 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00097c0 tx=0x7f5ab001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36f3c00a9df5512 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5ab9544640 1 -- 172.21.15.133:0/557115849 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00b2070 con 0x7f5ab415d9a0 2024-04-25T10:11:50.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5ab9544640 1 -- 172.21.15.133:0/557115849 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab00ad680 con 0x7f5ab415d9a0 2024-04-25T10:11:50.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5ab9544640 1 -- 172.21.15.133:0/557115849 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00ada50 con 0x7f5ab415d9a0 2024-04-25T10:11:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5abc7aa640 1 -- 172.21.15.133:0/557115849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab415d9a0 msgr2=0x7f5ab40d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5abc7aa640 1 --2- 172.21.15.133:0/557115849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab415d9a0 0x7f5ab40d0b30 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00097c0 tx=0x7f5ab001c3a0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5abc7aa640 1 -- 172.21.15.133:0/557115849 shutdown_connections 2024-04-25T10:11:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5abc7aa640 1 --2- 172.21.15.133:0/557115849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab415d9a0 0x7f5ab40d0b30 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5abc7aa640 1 -- 172.21.15.133:0/557115849 >> 172.21.15.133:0/557115849 conn(0x7f5ab406bf80 msgr2=0x7f5ab406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5abc7aa640 1 -- 172.21.15.133:0/557115849 shutdown_connections 2024-04-25T10:11:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.785+0000 7f5abc7aa640 1 -- 172.21.15.133:0/557115849 wait complete. 2024-04-25T10:11:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abc7aa640 1 Processor -- start 2024-04-25T10:11:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abc7aa640 1 -- start start 2024-04-25T10:11:50.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abc7aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab40ce600 0x7f5ab41606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abc7aa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab40d2030 0x7f5ab4156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abc7aa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab415d9a0 0x7f5ab4155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abc7aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40b96a0 con 0x7f5ab40ce600 2024-04-25T10:11:50.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abc7aa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40cffd0 con 0x7f5ab40d2030 2024-04-25T10:11:50.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abc7aa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40b9820 con 0x7f5ab415d9a0 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aba546640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab40ce600 0x7f5ab41606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aba546640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab40ce600 0x7f5ab41606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34340/0 (socket says 172.21.15.133:34340) 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aba546640 1 -- 172.21.15.133:0/2206649525 learned_addr learned my addr 172.21.15.133:0/2206649525 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abad47640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab415d9a0 0x7f5ab4155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aba546640 1 -- 172.21.15.133:0/2206649525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab415d9a0 msgr2=0x7f5ab4155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aba546640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab415d9a0 0x7f5ab4155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5ab9d45640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab40d2030 0x7f5ab4156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aba546640 1 -- 172.21.15.133:0/2206649525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab40d2030 msgr2=0x7f5ab4156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aba546640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab40d2030 0x7f5ab4156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aba546640 1 -- 172.21.15.133:0/2206649525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab40ca1e0 con 0x7f5ab40ce600 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abad47640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab415d9a0 0x7f5ab4155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aba546640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab40ce600 0x7f5ab41606d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0018fb0 tx=0x7f5ab001abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aaaffd640 1 -- 172.21.15.133:0/2206649525 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00bc020 con 0x7f5ab40ce600 2024-04-25T10:11:50.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aaaffd640 1 -- 172.21.15.133:0/2206649525 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab00add10 con 0x7f5ab40ce600 2024-04-25T10:11:50.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abc7aa640 1 -- 172.21.15.133:0/2206649525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab40c9f00 con 0x7f5ab40ce600 2024-04-25T10:11:50.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5aaaffd640 1 -- 172.21.15.133:0/2206649525 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00b2040 con 0x7f5ab40ce600 2024-04-25T10:11:50.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.789+0000 7f5abc7aa640 1 -- 172.21.15.133:0/2206649525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab415fed0 con 0x7f5ab40ce600 2024-04-25T10:11:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5aaaffd640 1 -- 172.21.15.133:0/2206649525 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ab00bf020 con 0x7f5ab40ce600 2024-04-25T10:11:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5aaaffd640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a98066f60 0x7f5a98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5aaaffd640 1 -- 172.21.15.133:0/2206649525 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5ab011f410 con 0x7f5ab40ce600 2024-04-25T10:11:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5ab9d45640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a98066f60 0x7f5a98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5ab9d45640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a98066f60 0x7f5a98069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4073e00 tx=0x7f5aa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 Processor -- start 2024-04-25T10:11:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 -- start start 2024-04-25T10:11:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d539d0 0x55ef27d53db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d542f0 0x55ef27d09b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d0a0a0 0x55ef27d0a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef27b56ff0 con 0x55ef27d0a0a0 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef27adf150 con 0x55ef27d539d0 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef27b53090 con 0x55ef27d542f0 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abad47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d0a0a0 0x55ef27d0a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abad47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d0a0a0 0x55ef27d0a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34354/0 (socket says 172.21.15.133:34354) 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abad47640 1 -- 172.21.15.133:0/2105415849 learned_addr learned my addr 172.21.15.133:0/2105415849 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5ab9d45640 1 --2- 172.21.15.133:0/2105415849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d542f0 0x55ef27d09b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5aba546640 1 --2- 172.21.15.133:0/2105415849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d539d0 0x55ef27d53db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abad47640 1 -- 172.21.15.133:0/2105415849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d542f0 msgr2=0x55ef27d09b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abad47640 1 --2- 172.21.15.133:0/2105415849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d542f0 0x55ef27d09b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abad47640 1 -- 172.21.15.133:0/2105415849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d539d0 msgr2=0x55ef27d53db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abad47640 1 --2- 172.21.15.133:0/2105415849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d539d0 0x55ef27d53db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abad47640 1 -- 172.21.15.133:0/2105415849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef27ced0d0 con 0x55ef27d0a0a0 2024-04-25T10:11:50.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abad47640 1 --2- 172.21.15.133:0/2105415849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d0a0a0 0x55ef27d0a480 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5aac001e40 tx=0x7f5aac00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873aef933fa3cfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5a97fff640 1 -- 172.21.15.133:0/2105415849 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac018070 con 0x55ef27d0a0a0 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5a97fff640 1 -- 172.21.15.133:0/2105415849 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac00de50 con 0x55ef27d0a0a0 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5a97fff640 1 -- 172.21.15.133:0/2105415849 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac006d60 con 0x55ef27d0a0a0 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 -- 172.21.15.133:0/2105415849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d0a0a0 msgr2=0x55ef27d0a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 --2- 172.21.15.133:0/2105415849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d0a0a0 0x55ef27d0a480 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5aac001e40 tx=0x7f5aac00d520 comp rx=0 tx=0).stop 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 -- 172.21.15.133:0/2105415849 shutdown_connections 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 --2- 172.21.15.133:0/2105415849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d0a0a0 0x55ef27d0a480 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 --2- 172.21.15.133:0/2105415849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d542f0 0x55ef27d09b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 --2- 172.21.15.133:0/2105415849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d539d0 0x55ef27d53db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 -- 172.21.15.133:0/2105415849 >> 172.21.15.133:0/2105415849 conn(0x55ef27d06020 msgr2=0x55ef27d08440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 -- 172.21.15.133:0/2105415849 shutdown_connections 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 -- 172.21.15.133:0/2105415849 wait complete. 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.793+0000 7f5abf917000 1 -- start start 2024-04-25T10:11:50.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 1 Processor -- start 2024-04-25T10:11:50.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d0a0a0 0x55ef27d387f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d38d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d542f0 0x55ef27d39270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef27b1b250 con 0x55ef27d539d0 2024-04-25T10:11:50.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef27b11690 con 0x55ef27d542f0 2024-04-25T10:11:50.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef27adfa40 con 0x55ef27d0a0a0 2024-04-25T10:11:50.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 10 client.0 _ll_register_callbacks cb 0x55ef27cf4e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5ab9d45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d38d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5ab9d45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d38d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34360/0 (socket says 172.21.15.133:34360) 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5ab9d45640 1 -- 172.21.15.133:0/820599046 learned_addr learned my addr 172.21.15.133:0/820599046 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abad47640 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d542f0 0x55ef27d39270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5aba546640 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d0a0a0 0x55ef27d387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5ab9d45640 1 -- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d0a0a0 msgr2=0x55ef27d387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5ab9d45640 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d0a0a0 0x55ef27d387f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5ab9d45640 1 -- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d542f0 msgr2=0x55ef27d39270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5ab9d45640 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d542f0 0x55ef27d39270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5ab9d45640 1 -- 172.21.15.133:0/820599046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef27c041c0 con 0x55ef27d539d0 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5ab9d45640 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d38d30 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4007650 tx=0x7f5aa4000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d791fcb704881f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5a957fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5a957fa640 1 -- 172.21.15.133:0/820599046 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4026060 con 0x55ef27d539d0 2024-04-25T10:11:50.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5a957fa640 1 -- 172.21.15.133:0/820599046 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5aa4021c50 con 0x55ef27d539d0 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 1 -- 172.21.15.133:0/820599046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5aac00de50 con 0x55ef27d539d0 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5a957fa640 1 -- 172.21.15.133:0/820599046 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aa4005e60 con 0x55ef27d539d0 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 10 client.8646 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 1 -- 172.21.15.133:0/820599046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55ef27c00170 con 0x55ef27d539d0 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5a957fa640 1 -- 172.21.15.133:0/820599046 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac00de50 con 0x55ef27d539d0 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5a957fa640 1 -- 172.21.15.133:0/820599046 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5aa4012410 con 0x55ef27d539d0 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 10 client.8646 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 10 client.8646 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 20 client.8646 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55ef27d63c80 0x55ef27d66070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 4 client.8646 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5abf917000 1 -- 172.21.15.133:0/820599046 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef27af2c60 con 0x55ef27d63c80 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5aba546640 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55ef27d63c80 0x55ef27d66070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5a957fa640 10 client.8646 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:50.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.797+0000 7f5aba546640 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55ef27d63c80 0x55ef27d66070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d1fdc5178fe680 server_cookie=7ff5f8cd5b6cdf67 in_seq=0 out_seq=0 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5a957fa640 1 -- 172.21.15.133:0/820599046 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef27af2c60 con 0x55ef27d63c80 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5a957fa640 10 client.8646 handle_command_reply: tid=0 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 1 client.8646 shutdown 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 1 -- 172.21.15.133:0/820599046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 msgr2=0x55ef27d38d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d38d30 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4007650 tx=0x7f5aa4000bd0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 1 -- 172.21.15.133:0/820599046 shutdown_connections 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55ef27d63c80 0x55ef27d66070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d542f0 0x55ef27d39270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5a957fa640 0 client.8646 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d38d30 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 1 --2- 172.21.15.133:0/820599046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d0a0a0 0x55ef27d387f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 1 -- 172.21.15.133:0/820599046 >> 172.21.15.133:0/820599046 conn(0x55ef27d06020 msgr2=0x55ef27d08ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 1 -- 172.21.15.133:0/820599046 shutdown_connections 2024-04-25T10:11:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 1 -- 172.21.15.133:0/820599046 wait complete. 2024-04-25T10:11:50.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.801+0000 7f5abf917000 20 client.8646 trim_cache size 0 max 16384 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.813+0000 7f5abf917000 1 Processor -- start 2024-04-25T10:11:50.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.813+0000 7f5abf917000 1 -- start start 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.813+0000 7f5abf917000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d0a0a0 0x55ef27dafee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.813+0000 7f5abf917000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d539d0 0x55ef27db0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.813+0000 7f5abf917000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d542f0 0x55ef27db0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.813+0000 7f5abf917000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab4203920 con 0x55ef27d542f0 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.813+0000 7f5abf917000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab4202b20 con 0x55ef27d539d0 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.813+0000 7f5abf917000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab4159e10 con 0x55ef27d0a0a0 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.813+0000 7f5abad47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d542f0 0x55ef27db0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.813+0000 7f5abad47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d542f0 0x55ef27db0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34376/0 (socket says 172.21.15.133:34376) 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.813+0000 7f5abad47640 1 -- 172.21.15.133:0/2125393937 learned_addr learned my addr 172.21.15.133:0/2125393937 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abad47640 1 -- 172.21.15.133:0/2125393937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d0a0a0 msgr2=0x55ef27dafee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abad47640 1 --2- 172.21.15.133:0/2125393937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d0a0a0 0x55ef27dafee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abad47640 1 -- 172.21.15.133:0/2125393937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d539d0 msgr2=0x55ef27db0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abad47640 1 --2- 172.21.15.133:0/2125393937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d539d0 0x55ef27db0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abad47640 1 -- 172.21.15.133:0/2125393937 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef27b9dde0 con 0x55ef27d542f0 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abad47640 1 --2- 172.21.15.133:0/2125393937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d542f0 0x55ef27db0960 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5aac016040 tx=0x7f5aac00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73478dd6f07a55a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5a977fe640 1 -- 172.21.15.133:0/2125393937 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac018040 con 0x55ef27d542f0 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5a977fe640 1 -- 172.21.15.133:0/2125393937 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac019040 con 0x55ef27d542f0 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5a977fe640 1 -- 172.21.15.133:0/2125393937 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00de40 con 0x55ef27d542f0 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- 172.21.15.133:0/2125393937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d542f0 msgr2=0x55ef27db0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 --2- 172.21.15.133:0/2125393937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d542f0 0x55ef27db0960 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5aac016040 tx=0x7f5aac00b000 comp rx=0 tx=0).stop 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- 172.21.15.133:0/2125393937 shutdown_connections 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 --2- 172.21.15.133:0/2125393937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d542f0 0x55ef27db0960 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 --2- 172.21.15.133:0/2125393937 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d539d0 0x55ef27db0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 --2- 172.21.15.133:0/2125393937 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d0a0a0 0x55ef27dafee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- 172.21.15.133:0/2125393937 >> 172.21.15.133:0/2125393937 conn(0x55ef27d06020 msgr2=0x55ef27d520d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- 172.21.15.133:0/2125393937 shutdown_connections 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- 172.21.15.133:0/2125393937 wait complete. 2024-04-25T10:11:50.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- start start 2024-04-25T10:11:50.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 Processor -- start 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d0a0a0 0x55ef27d66380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d542f0 0x55ef27d66e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef27af1e00 con 0x55ef27d539d0 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab4202520 con 0x55ef27d0a0a0 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef27b56300 con 0x55ef27d542f0 2024-04-25T10:11:50.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 10 client.0 _ll_register_callbacks cb 0x55ef27cd5040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5ab9d45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5ab9d45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d668c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34386/0 (socket says 172.21.15.133:34386) 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5ab9d45640 1 -- 172.21.15.133:0/2628896487 learned_addr learned my addr 172.21.15.133:0/2628896487 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5aba546640 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d0a0a0 0x55ef27d66380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5ab9d45640 1 -- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d542f0 msgr2=0x55ef27d66e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abad47640 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d542f0 0x55ef27d66e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5ab9d45640 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d542f0 0x55ef27d66e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5ab9d45640 1 -- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d0a0a0 msgr2=0x55ef27d66380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5ab9d45640 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d0a0a0 0x55ef27d66380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5ab9d45640 1 -- 172.21.15.133:0/2628896487 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef27af88e0 con 0x55ef27d539d0 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5aba546640 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d0a0a0 0x55ef27d66380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5ab9d45640 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d668c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005e30 tx=0x7f5aa4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e10573e2d75fa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5a957fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:11:50.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5a957fa640 1 -- 172.21.15.133:0/2628896487 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4012410 con 0x55ef27d539d0 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5a957fa640 1 -- 172.21.15.133:0/2628896487 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5aa4012ce0 con 0x55ef27d539d0 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- 172.21.15.133:0/2628896487 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5aac019040 con 0x55ef27d539d0 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5a957fa640 1 -- 172.21.15.133:0/2628896487 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aa40072e0 con 0x55ef27d539d0 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 10 client.8652 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- 172.21.15.133:0/2628896487 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55ef27bb08f0 con 0x55ef27d539d0 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5a957fa640 1 -- 172.21.15.133:0/2628896487 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac019040 con 0x55ef27d539d0 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5a957fa640 1 -- 172.21.15.133:0/2628896487 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5aa403d050 con 0x55ef27d539d0 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 10 client.8652 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 10 client.8652 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 20 client.8652 populate_metadata read hostname 'smithi133' 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55ef27d67a10 0x55ef27dbbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 4 client.8652 mds_command: new command op to 8313 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5abf917000 1 -- 172.21.15.133:0/2628896487 --> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ef27b64530 con 0x55ef27d67a10 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.817+0000 7f5aba546640 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55ef27d67a10 0x55ef27dbbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5a957fa640 10 client.8652 ms_handle_connect on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5aba546640 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55ef27d67a10 0x55ef27dbbf60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa38bcbcdcff186f server_cookie=33c1696a6edab0ac in_seq=0 out_seq=0 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5a957fa640 1 -- 172.21.15.133:0/2628896487 <== mds.0 v2:172.21.15.162:6832/488739264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55ef27b64530 con 0x55ef27d67a10 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5a957fa640 10 client.8652 handle_command_reply: tid=0 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 1 client.8652 shutdown 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 1 -- 172.21.15.133:0/2628896487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 msgr2=0x55ef27d668c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d668c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005e30 tx=0x7f5aa4000ee0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 1 -- 172.21.15.133:0/2628896487 shutdown_connections 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:6832/488739264,v1:172.21.15.162:6834/488739264] conn(0x55ef27d67a10 0x55ef27dbbf60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55ef27d542f0 0x55ef27d66e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef27d539d0 0x55ef27d668c0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5a957fa640 0 client.8652 ms_handle_reset on v2:172.21.15.162:6832/488739264 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 1 --2- 172.21.15.133:0/2628896487 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ef27d0a0a0 0x55ef27d66380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 1 -- 172.21.15.133:0/2628896487 >> 172.21.15.133:0/2628896487 conn(0x55ef27d06020 msgr2=0x55ef27d52960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 1 -- 172.21.15.133:0/2628896487 shutdown_connections 2024-04-25T10:11:50.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 1 -- 172.21.15.133:0/2628896487 wait complete. 2024-04-25T10:11:50.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abf917000 20 client.8652 trim_cache size 0 max 16384 2024-04-25T10:11:50.829 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:11:50.829 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:11:50.829 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-04-25T10:11:50.829 INFO:teuthology.orchestra.run.smithi133.stdout: "3aafa117-8943-43e7-b495-998293c81538": { 2024-04-25T10:11:50.829 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:11:50.829 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "3aafa117-8943-43e7-b495-998293c81538", 2024-04-25T10:11:50.829 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive" 2024-04-25T10:11:50.829 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:50.830 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:11:50.830 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:11:50.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 -- 172.21.15.133:0/2206649525 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a98066f60 msgr2=0x7f5a98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a98066f60 0x7f5a98069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4073e00 tx=0x7f5aa4009040 comp rx=0 tx=0).stop 2024-04-25T10:11:50.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 -- 172.21.15.133:0/2206649525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab40ce600 msgr2=0x7f5ab41606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab40ce600 0x7f5ab41606d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0018fb0 tx=0x7f5ab001abe0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 -- 172.21.15.133:0/2206649525 shutdown_connections 2024-04-25T10:11:50.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5ab415d9a0 0x7f5ab4155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab40d2030 0x7f5ab4156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5a98066f60 0x7f5a98069420 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 --2- 172.21.15.133:0/2206649525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ab40ce600 0x7f5ab41606d0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 -- 172.21.15.133:0/2206649525 >> 172.21.15.133:0/2206649525 conn(0x7f5ab406bf80 msgr2=0x7f5ab40b8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 -- 172.21.15.133:0/2206649525 shutdown_connections 2024-04-25T10:11:50.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.821+0000 7f5abc7aa640 1 -- 172.21.15.133:0/2206649525 wait complete. 2024-04-25T10:11:50.839 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:50.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.941+0000 7fd4bfebb640 1 Processor -- start 2024-04-25T10:11:50.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.941+0000 7fd4bfebb640 1 -- start start 2024-04-25T10:11:50.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bfebb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4b80d0770 0x7fd4b80d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bfebb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b80b87e0 0x7fd4b80b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bfebb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b80ceb20 con 0x7fd4b815db00 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bfebb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b80d1110 con 0x7fd4b80d0770 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bfebb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80b30a0 con 0x7fd4b80b87e0 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bdc57640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd4b815db00 0x7fd4b80b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51706/0 (socket says 172.21.15.133:51706) 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bdc57640 1 -- 172.21.15.133:0/1938895034 learned_addr learned my addr 172.21.15.133:0/1938895034 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248159158 0 0) 0x7fd4b80ceb20 con 0x7fd4b815db00 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4be458640 1 --2- 172.21.15.133:0/1938895034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b80b87e0 0x7fd4b80b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4a4003530 con 0x7fd4b815db00 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bd456640 1 --2- 172.21.15.133:0/1938895034 >> v2:172.21.15.162:3300/0 conn(0x7fd4b80d0770 0x7fd4b80d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1659451222 0 0) 0x7fd4a4003530 con 0x7fd4b815db00 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b80b87e0 msgr2=0x7fd4b80b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 --2- 172.21.15.133:0/1938895034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b80b87e0 0x7fd4b80b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 >> v2:172.21.15.162:3300/0 conn(0x7fd4b80d0770 msgr2=0x7fd4b80d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 --2- 172.21.15.133:0/1938895034 >> v2:172.21.15.162:3300/0 conn(0x7fd4b80d0770 0x7fd4b80d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b80b7740 con 0x7fd4b815db00 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fd4b4001680 con 0x7fd4b815db00 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 >> v1:172.21.15.133:6789/0 conn(0x7fd4b815db00 legacy=0x7fd4b80b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 --2- 172.21.15.133:0/1938895034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 0x7fd4a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 --2- 172.21.15.133:0/1938895034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4a4004810 0x7fd4a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 --2- 172.21.15.133:0/1938895034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b80b87e0 0x7fd4a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80b30a0 con 0x7fd4a40039c0 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a400b9d0 con 0x7fd4a4004810 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a400bbd0 con 0x7fd4b80b87e0 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bdc57640 1 --2- 172.21.15.133:0/1938895034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 0x7fd4a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4be458640 1 --2- 172.21.15.133:0/1938895034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b80b87e0 0x7fd4a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bdc57640 1 -- 172.21.15.133:0/1938895034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b80b87e0 msgr2=0x7fd4a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bdc57640 1 --2- 172.21.15.133:0/1938895034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b80b87e0 0x7fd4a40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bdc57640 1 -- 172.21.15.133:0/1938895034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4a4004810 msgr2=0x7fd4a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bdc57640 1 --2- 172.21.15.133:0/1938895034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4a4004810 0x7fd4a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bdc57640 1 -- 172.21.15.133:0/1938895034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a4003530 con 0x7fd4a40039c0 2024-04-25T10:11:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bdc57640 1 --2- 172.21.15.133:0/1938895034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 0x7fd4a4003de0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400c160 tx=0x7fd4b401cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c85e30601e179b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4007c90 con 0x7fd4a40039c0 2024-04-25T10:11:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b40a9020 con 0x7fd4a40039c0 2024-04-25T10:11:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bcc55640 1 -- 172.21.15.133:0/1938895034 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4005b00 con 0x7fd4a40039c0 2024-04-25T10:11:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bfebb640 1 -- 172.21.15.133:0/1938895034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 msgr2=0x7fd4a4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bfebb640 1 --2- 172.21.15.133:0/1938895034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 0x7fd4a4003de0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400c160 tx=0x7fd4b401cb90 comp rx=0 tx=0).stop 2024-04-25T10:11:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4be458640 1 -- 172.21.15.133:0/1938895034 reap_dead start 2024-04-25T10:11:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bfebb640 1 -- 172.21.15.133:0/1938895034 shutdown_connections 2024-04-25T10:11:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.945+0000 7fd4bfebb640 1 -- 172.21.15.133:0/1938895034 >> 172.21.15.133:0/1938895034 conn(0x7fd4b80727c0 msgr2=0x7fd4b8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 -- 172.21.15.133:0/1938895034 shutdown_connections 2024-04-25T10:11:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 -- 172.21.15.133:0/1938895034 wait complete. 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 Processor -- start 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 -- start start 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 0x7fd4b81548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b814e780 0x7fd4b80ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b8159fd0 0x7fd4b80cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80b7430 con 0x7fd4a40039c0 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80b82c0 con 0x7fd4b814e780 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80d18a0 con 0x7fd4b8159fd0 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bdc57640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 0x7fd4b81548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bdc57640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 0x7fd4b81548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34400/0 (socket says 172.21.15.133:34400) 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bdc57640 1 -- 172.21.15.133:0/766027304 learned_addr learned my addr 172.21.15.133:0/766027304 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4be458640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b8159fd0 0x7fd4b80cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bd456640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b814e780 0x7fd4b80ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bdc57640 1 -- 172.21.15.133:0/766027304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b8159fd0 msgr2=0x7fd4b80cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bdc57640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b8159fd0 0x7fd4b80cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bdc57640 1 -- 172.21.15.133:0/766027304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b814e780 msgr2=0x7fd4b80ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bdc57640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b814e780 0x7fd4b80ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bdc57640 1 -- 172.21.15.133:0/766027304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b815a590 con 0x7fd4a40039c0 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bd456640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b814e780 0x7fd4b80ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bdc57640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 0x7fd4b81548b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40ab040 tx=0x7fd4a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4ae7fc640 1 -- 172.21.15.133:0/766027304 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b40049f0 con 0x7fd4a40039c0 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4ae7fc640 1 -- 172.21.15.133:0/766027304 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b4004b90 con 0x7fd4a40039c0 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 -- 172.21.15.133:0/766027304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b815a870 con 0x7fd4a40039c0 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4ae7fc640 1 -- 172.21.15.133:0/766027304 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b40070d0 con 0x7fd4a40039c0 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 -- 172.21.15.133:0/766027304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b815ae30 con 0x7fd4a40039c0 2024-04-25T10:11:50.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.949+0000 7fd4bfebb640 1 -- 172.21.15.133:0/766027304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b8068ef0 con 0x7fd4a40039c0 2024-04-25T10:11:50.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.953+0000 7fd4ae7fc640 1 -- 172.21.15.133:0/766027304 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4b4007270 con 0x7fd4a40039c0 2024-04-25T10:11:50.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.953+0000 7fd4ae7fc640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd49c066f60 0x7fd49c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.953+0000 7fd4bd456640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd49c066f60 0x7fd49c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.953+0000 7fd4ae7fc640 1 -- 172.21.15.133:0/766027304 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd4b411eb80 con 0x7fd4a40039c0 2024-04-25T10:11:50.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.953+0000 7fd4bd456640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd49c066f60 0x7fd49c069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8006aa0 tx=0x7fd4a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:50.957+0000 7fd4ae7fc640 1 -- 172.21.15.133:0/766027304 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4b40168d0 con 0x7fd4a40039c0 2024-04-25T10:11:51.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.137+0000 7fd4bfebb640 1 -- 172.21.15.133:0/766027304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4b80b2750 con 0x7fd4a40039c0 2024-04-25T10:11:51.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4ae7fc640 1 -- 172.21.15.133:0/766027304 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fd4b40c0390 con 0x7fd4a40039c0 2024-04-25T10:11:51.142 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:51.142 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 -- 172.21.15.133:0/766027304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd49c066f60 msgr2=0x7fd49c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd49c066f60 0x7fd49c069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8006aa0 tx=0x7fd4a8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 -- 172.21.15.133:0/766027304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 msgr2=0x7fd4b81548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 0x7fd4b81548b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40ab040 tx=0x7fd4a4005840 comp rx=0 tx=0).stop 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 -- 172.21.15.133:0/766027304 shutdown_connections 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b8159fd0 0x7fd4b80cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b814e780 0x7fd4b80ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd49c066f60 0x7fd49c069420 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 --2- 172.21.15.133:0/766027304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a40039c0 0x7fd4b81548b0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 -- 172.21.15.133:0/766027304 >> 172.21.15.133:0/766027304 conn(0x7fd4b80727c0 msgr2=0x7fd4b80b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 -- 172.21.15.133:0/766027304 shutdown_connections 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.141+0000 7fd4bfebb640 1 -- 172.21.15.133:0/766027304 wait complete. 2024-04-25T10:11:51.159 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:11:51.159 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:51.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dfc9b640 1 Processor -- start 2024-04-25T10:11:51.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dfc9b640 1 -- start start 2024-04-25T10:11:51.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dfc9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22d815a5f0 0x7f22d80c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dfc9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22d806eb70 0x7f22d80cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dfc9b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d80cc5a0 con 0x7f22d815a9d0 2024-04-25T10:11:51.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dfc9b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d80cc880 con 0x7f22d815a5f0 2024-04-25T10:11:51.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dfc9b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80ccb60 con 0x7f22d806eb70 2024-04-25T10:11:51.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dda37640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22d815a5f0 0x7f22d80c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dda37640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22d815a5f0 0x7f22d80c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34414/0 (socket says 172.21.15.133:34414) 2024-04-25T10:11:51.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dda37640 1 -- 172.21.15.133:0/498911062 learned_addr learned my addr 172.21.15.133:0/498911062 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:51.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dda37640 1 -- 172.21.15.133:0/498911062 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22d806eb70 msgr2=0x7f22d80cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:51.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22de238640 1 --2- 172.21.15.133:0/498911062 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22d806eb70 0x7f22d80cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dda37640 1 --2- 172.21.15.133:0/498911062 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22d806eb70 0x7f22d80cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dda37640 1 -- 172.21.15.133:0/498911062 >> v1:172.21.15.162:6789/0 conn(0x7f22d815a9d0 legacy=0x7f22d806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:51.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.265+0000 7f22dda37640 1 -- 172.21.15.133:0/498911062 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80cce40 con 0x7f22d815a5f0 2024-04-25T10:11:51.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dda37640 1 --2- 172.21.15.133:0/498911062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22d815a5f0 0x7f22d80c5260 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009870 tx=0x7f22d401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d247bd41fe98e343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dca35640 1 -- 172.21.15.133:0/498911062 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40b0070 con 0x7f22d815a5f0 2024-04-25T10:11:51.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dca35640 1 -- 172.21.15.133:0/498911062 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d40ab600 con 0x7f22d815a5f0 2024-04-25T10:11:51.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dca35640 1 -- 172.21.15.133:0/498911062 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40ab9d0 con 0x7f22d815a5f0 2024-04-25T10:11:51.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dfc9b640 1 -- 172.21.15.133:0/498911062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22d815a5f0 msgr2=0x7f22d80c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dfc9b640 1 --2- 172.21.15.133:0/498911062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22d815a5f0 0x7f22d80c5260 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009870 tx=0x7f22d401c520 comp rx=0 tx=0).stop 2024-04-25T10:11:51.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dfc9b640 1 -- 172.21.15.133:0/498911062 shutdown_connections 2024-04-25T10:11:51.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dfc9b640 1 --2- 172.21.15.133:0/498911062 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22d806eb70 0x7f22d80cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dfc9b640 1 --2- 172.21.15.133:0/498911062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22d815a5f0 0x7f22d80c5260 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dfc9b640 1 -- 172.21.15.133:0/498911062 >> 172.21.15.133:0/498911062 conn(0x7f22d8067000 msgr2=0x7f22d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dfc9b640 1 -- 172.21.15.133:0/498911062 shutdown_connections 2024-04-25T10:11:51.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.269+0000 7f22dfc9b640 1 -- 172.21.15.133:0/498911062 wait complete. 2024-04-25T10:11:51.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dfc9b640 1 Processor -- start 2024-04-25T10:11:51.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dfc9b640 1 -- start start 2024-04-25T10:11:51.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dfc9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22d806eb70 0x7f22d8151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dfc9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22d815a5f0 0x7f22d815c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dfc9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22d815a9d0 0x7f22d815c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dfc9b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80ce490 con 0x7f22d815a9d0 2024-04-25T10:11:51.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dfc9b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80ce310 con 0x7f22d806eb70 2024-04-25T10:11:51.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dfc9b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80ce610 con 0x7f22d815a5f0 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22de238640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22d815a9d0 0x7f22d815c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22de238640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22d815a9d0 0x7f22d815c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34430/0 (socket says 172.21.15.133:34430) 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22de238640 1 -- 172.21.15.133:0/414728440 learned_addr learned my addr 172.21.15.133:0/414728440 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dda37640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22d806eb70 0x7f22d8151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dd236640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22d815a5f0 0x7f22d815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22de238640 1 -- 172.21.15.133:0/414728440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22d815a5f0 msgr2=0x7f22d815c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22de238640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22d815a5f0 0x7f22d815c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22de238640 1 -- 172.21.15.133:0/414728440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22d806eb70 msgr2=0x7f22d8151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22de238640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22d806eb70 0x7f22d8151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22de238640 1 -- 172.21.15.133:0/414728440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d815d5f0 con 0x7f22d815a9d0 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dda37640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22d806eb70 0x7f22d8151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22de238640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22d815a9d0 0x7f22d815c6c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f22d000ce50 tx=0x7f22d000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22ce7fc640 1 -- 172.21.15.133:0/414728440 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0019020 con 0x7f22d815a9d0 2024-04-25T10:11:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22ce7fc640 1 -- 172.21.15.133:0/414728440 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d000c2e0 con 0x7f22d815a9d0 2024-04-25T10:11:51.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dfc9b640 1 -- 172.21.15.133:0/414728440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d80c75f0 con 0x7f22d815a9d0 2024-04-25T10:11:51.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22ce7fc640 1 -- 172.21.15.133:0/414728440 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000edb0 con 0x7f22d815a9d0 2024-04-25T10:11:51.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dfc9b640 1 -- 172.21.15.133:0/414728440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d80c6bd0 con 0x7f22d815a9d0 2024-04-25T10:11:51.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22dfc9b640 1 -- 172.21.15.133:0/414728440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d8068ef0 con 0x7f22d815a9d0 2024-04-25T10:11:51.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.273+0000 7f22ce7fc640 1 -- 172.21.15.133:0/414728440 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22d000a480 con 0x7f22d815a9d0 2024-04-25T10:11:51.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.277+0000 7f22ce7fc640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22bc066f60 0x7f22bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.277+0000 7f22ce7fc640 1 -- 172.21.15.133:0/414728440 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f22d0087190 con 0x7f22d815a9d0 2024-04-25T10:11:51.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.277+0000 7f22dda37640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22bc066f60 0x7f22bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.277+0000 7f22dda37640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22bc066f60 0x7f22bc069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f22d40ad000 tx=0x7f22d40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.281+0000 7f22ce7fc640 1 -- 172.21.15.133:0/414728440 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22d0023060 con 0x7f22d815a9d0 2024-04-25T10:11:51.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.493+0000 7f22dfc9b640 1 -- 172.21.15.133:0/414728440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f22d8152c20 con 0x7f22d815a9d0 2024-04-25T10:11:51.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22ce7fc640 1 -- 172.21.15.133:0/414728440 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f22d0058070 con 0x7f22d815a9d0 2024-04-25T10:11:51.499 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:51.499 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":624,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078164813,"bytes_used":2674196480,"bytes_avail":770419916800,"bytes_total":773094113280,"write_bytes_sec":42242578,"read_op_per_sec":0,"write_op_per_sec":44},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:11:49.888630+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:11:51.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 -- 172.21.15.133:0/414728440 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22bc066f60 msgr2=0x7f22bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22bc066f60 0x7f22bc069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f22d40ad000 tx=0x7f22d40a6000 comp rx=0 tx=0).stop 2024-04-25T10:11:51.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 -- 172.21.15.133:0/414728440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22d815a9d0 msgr2=0x7f22d815c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22d815a9d0 0x7f22d815c6c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f22d000ce50 tx=0x7f22d000e420 comp rx=0 tx=0).stop 2024-04-25T10:11:51.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 -- 172.21.15.133:0/414728440 shutdown_connections 2024-04-25T10:11:51.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22bc066f60 0x7f22bc069420 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22d815a9d0 0x7f22d815c6c0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22d815a5f0 0x7f22d815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 --2- 172.21.15.133:0/414728440 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22d806eb70 0x7f22d8151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 -- 172.21.15.133:0/414728440 >> 172.21.15.133:0/414728440 conn(0x7f22d8067000 msgr2=0x7f22d80b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 -- 172.21.15.133:0/414728440 shutdown_connections 2024-04-25T10:11:51.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:51.497+0000 7f22dfc9b640 1 -- 172.21.15.133:0/414728440 wait complete. 2024-04-25T10:11:51.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:11:52.518 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:52.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20eb9b0640 1 Processor -- start 2024-04-25T10:11:52.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20eb9b0640 1 -- start start 2024-04-25T10:11:52.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20eb9b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20e415db00 0x7f20e40b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20eb9b0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f20e40d0770 0x7f20e40d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20eb9b0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e40ceb20 con 0x7f20e41614c0 2024-04-25T10:11:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20eb9b0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e40d1110 con 0x7f20e40d0770 2024-04-25T10:11:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20eb9b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40b7740 con 0x7f20e415db00 2024-04-25T10:11:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20e8f4b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f20e40d0770 0x7f20e40d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20e8f4b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f20e40d0770 0x7f20e40d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34434/0 (socket says 172.21.15.133:34434) 2024-04-25T10:11:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20e8f4b640 1 -- 172.21.15.133:0/1912668560 learned_addr learned my addr 172.21.15.133:0/1912668560 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20e974c640 1 --2- 172.21.15.133:0/1912668560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20e415db00 0x7f20e40b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20e8f4b640 1 -- 172.21.15.133:0/1912668560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20e415db00 msgr2=0x7f20e40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20e8f4b640 1 --2- 172.21.15.133:0/1912668560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20e415db00 0x7f20e40b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20e8f4b640 1 -- 172.21.15.133:0/1912668560 >> v1:172.21.15.162:6789/0 conn(0x7f20e41614c0 legacy=0x7f20e4161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.641+0000 7f20e8f4b640 1 -- 172.21.15.133:0/1912668560 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e40b97f0 con 0x7f20e40d0770 2024-04-25T10:11:52.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20e8f4b640 1 --2- 172.21.15.133:0/1912668560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e40d0770 0x7f20e40d0bd0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009870 tx=0x7f20d401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8e53927feadf10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20dbfff640 1 -- 172.21.15.133:0/1912668560 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40aa610 con 0x7f20e40d0770 2024-04-25T10:11:52.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20dbfff640 1 -- 172.21.15.133:0/1912668560 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d40af070 con 0x7f20e40d0770 2024-04-25T10:11:52.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20dbfff640 1 -- 172.21.15.133:0/1912668560 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40aa9d0 con 0x7f20e40d0770 2024-04-25T10:11:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 -- 172.21.15.133:0/1912668560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e40d0770 msgr2=0x7f20e40d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 --2- 172.21.15.133:0/1912668560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e40d0770 0x7f20e40d0bd0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009870 tx=0x7f20d401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 -- 172.21.15.133:0/1912668560 shutdown_connections 2024-04-25T10:11:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 --2- 172.21.15.133:0/1912668560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e40d0770 0x7f20e40d0bd0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 --2- 172.21.15.133:0/1912668560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20e415db00 0x7f20e40b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 -- 172.21.15.133:0/1912668560 >> 172.21.15.133:0/1912668560 conn(0x7f20e40727c0 msgr2=0x7f20e4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 -- 172.21.15.133:0/1912668560 shutdown_connections 2024-04-25T10:11:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 -- 172.21.15.133:0/1912668560 wait complete. 2024-04-25T10:11:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 Processor -- start 2024-04-25T10:11:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 -- start start 2024-04-25T10:11:52.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e40d0770 0x7f20e40c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20e415db00 0x7f20e4157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20e41614c0 0x7f20e40c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40b7430 con 0x7f20e40d0770 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40b82c0 con 0x7f20e415db00 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20eb9b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40d18a0 con 0x7f20e41614c0 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20e974c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e40d0770 0x7f20e40c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20e974c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e40d0770 0x7f20e40c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34438/0 (socket says 172.21.15.133:34438) 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20e974c640 1 -- 172.21.15.133:0/3153879225 learned_addr learned my addr 172.21.15.133:0/3153879225 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20e9f4d640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20e41614c0 0x7f20e40c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20e974c640 1 -- 172.21.15.133:0/3153879225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20e41614c0 msgr2=0x7f20e40c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20e974c640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20e41614c0 0x7f20e40c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20e8f4b640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20e415db00 0x7f20e4157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20e974c640 1 -- 172.21.15.133:0/3153879225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20e415db00 msgr2=0x7f20e4157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20e974c640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20e415db00 0x7f20e4157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.645+0000 7f20e974c640 1 -- 172.21.15.133:0/3153879225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4156270 con 0x7f20e40d0770 2024-04-25T10:11:52.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20e974c640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e40d0770 0x7f20e40c7860 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f20e000d800 tx=0x7f20e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20d9ffb640 1 -- 172.21.15.133:0/3153879225 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000b660 con 0x7f20e40d0770 2024-04-25T10:11:52.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20d9ffb640 1 -- 172.21.15.133:0/3153879225 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e0019070 con 0x7f20e40d0770 2024-04-25T10:11:52.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20eb9b0640 1 -- 172.21.15.133:0/3153879225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e4156550 con 0x7f20e40d0770 2024-04-25T10:11:52.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20d9ffb640 1 -- 172.21.15.133:0/3153879225 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e000ba00 con 0x7f20e40d0770 2024-04-25T10:11:52.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20eb9b0640 1 -- 172.21.15.133:0/3153879225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e4156b10 con 0x7f20e40d0770 2024-04-25T10:11:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20eb9b0640 1 -- 172.21.15.133:0/3153879225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e4068ef0 con 0x7f20e40d0770 2024-04-25T10:11:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20d9ffb640 1 -- 172.21.15.133:0/3153879225 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20e000bba0 con 0x7f20e40d0770 2024-04-25T10:11:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20d9ffb640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20c8066f90 0x7f20c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20d9ffb640 1 -- 172.21.15.133:0/3153879225 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f20e0085570 con 0x7f20e40d0770 2024-04-25T10:11:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20e8f4b640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20c8066f90 0x7f20c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.649+0000 7f20e8f4b640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20c8066f90 0x7f20c8069450 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f20d40ae000 tx=0x7f20d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.653+0000 7f20d9ffb640 1 -- 172.21.15.133:0/3153879225 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20e0056450 con 0x7f20e40d0770 2024-04-25T10:11:52.736 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:52.738 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:52.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 -- 172.21.15.133:0/3153879225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20e40d44e0 con 0x7f20e40d0770 2024-04-25T10:11:52.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20d9ffb640 1 -- 172.21.15.133:0/3153879225 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f20e0055d50 con 0x7f20e40d0770 2024-04-25T10:11:52.839 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:52.839 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:52.839 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:52.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 -- 172.21.15.133:0/3153879225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20c8066f90 msgr2=0x7f20c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20c8066f90 0x7f20c8069450 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f20d40ae000 tx=0x7f20d4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:52.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 -- 172.21.15.133:0/3153879225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e40d0770 msgr2=0x7f20e40c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e40d0770 0x7f20e40c7860 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f20e000d800 tx=0x7f20e000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 -- 172.21.15.133:0/3153879225 shutdown_connections 2024-04-25T10:11:52.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20e41614c0 0x7f20e40c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20e415db00 0x7f20e4157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20c8066f90 0x7f20c8069450 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 --2- 172.21.15.133:0/3153879225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e40d0770 0x7f20e40c7860 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 -- 172.21.15.133:0/3153879225 >> 172.21.15.133:0/3153879225 conn(0x7f20e40727c0 msgr2=0x7f20e4167cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 -- 172.21.15.133:0/3153879225 shutdown_connections 2024-04-25T10:11:52.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.837+0000 7f20eb9b0640 1 -- 172.21.15.133:0/3153879225 wait complete. 2024-04-25T10:11:52.856 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:11:52.856 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:52.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.969+0000 7fe0681c9640 1 Processor -- start 2024-04-25T10:11:52.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.969+0000 7fe0681c9640 1 -- start start 2024-04-25T10:11:52.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.969+0000 7fe0681c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe060158110 0x7fe0600b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.969+0000 7fe0681c9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe0601584f0 0x7fe06015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.969+0000 7fe0681c9640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0600cbe60 con 0x7fe06006e460 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.969+0000 7fe0681c9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0600b19b0 con 0x7fe0601584f0 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.969+0000 7fe0681c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06015b360 con 0x7fe060158110 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe066766640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe06006e460 0x7fe06015e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51720/0 (socket says 172.21.15.133:51720) 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe066766640 1 -- 172.21.15.133:0/2843738498 learned_addr learned my addr 172.21.15.133:0/2843738498 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe065f65640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe060158110 0x7fe0600b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe065764640 1 --2- 172.21.15.133:0/2843738498 >> v2:172.21.15.162:3300/0 conn(0x7fe0601584f0 0x7fe06015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944027918 0 0) 0x7fe0600cbe60 con 0x7fe06006e460 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe04c003530 con 0x7fe06006e460 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4260283981 0 0) 0x7fe04c003530 con 0x7fe06006e460 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe060158110 msgr2=0x7fe0600b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe060158110 0x7fe0600b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 >> v2:172.21.15.162:3300/0 conn(0x7fe0601584f0 msgr2=0x7fe06015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 --2- 172.21.15.133:0/2843738498 >> v2:172.21.15.162:3300/0 conn(0x7fe0601584f0 0x7fe06015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06015b640 con 0x7fe06006e460 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe058001a00 con 0x7fe06006e460 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 >> v1:172.21.15.133:6789/0 conn(0x7fe06006e460 legacy=0x7fe06015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe04c0039c0 0x7fe04c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe060158110 0x7fe04c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 0x7fe04c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600b19b0 con 0x7fe04c004d50 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06015b360 con 0x7fe04c0039c0 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c00b9d0 con 0x7fe060158110 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe066766640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 0x7fe04c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe065764640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe060158110 0x7fe04c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe065f65640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe04c0039c0 0x7fe04c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe066766640 1 -- 172.21.15.133:0/2843738498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe060158110 msgr2=0x7fe04c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe066766640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe060158110 0x7fe04c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe066766640 1 -- 172.21.15.133:0/2843738498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe04c0039c0 msgr2=0x7fe04c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe066766640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe04c0039c0 0x7fe04c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe066766640 1 -- 172.21.15.133:0/2843738498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c003530 con 0x7fe04c004d50 2024-04-25T10:11:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe066766640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 0x7fe04c0051d0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fe05800a920 tx=0x7fe05801d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d965a181d9104ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05800a4c0 con 0x7fe04c004d50 2024-04-25T10:11:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0580072f0 con 0x7fe04c004d50 2024-04-25T10:11:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.973+0000 7fe064f63640 1 -- 172.21.15.133:0/2843738498 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0580076e0 con 0x7fe04c004d50 2024-04-25T10:11:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 -- 172.21.15.133:0/2843738498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 msgr2=0x7fe04c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 --2- 172.21.15.133:0/2843738498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 0x7fe04c0051d0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fe05800a920 tx=0x7fe05801d440 comp rx=0 tx=0).stop 2024-04-25T10:11:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe066766640 1 -- 172.21.15.133:0/2843738498 reap_dead start 2024-04-25T10:11:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 -- 172.21.15.133:0/2843738498 shutdown_connections 2024-04-25T10:11:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 -- 172.21.15.133:0/2843738498 >> 172.21.15.133:0/2843738498 conn(0x7fe060067000 msgr2=0x7fe06006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 -- 172.21.15.133:0/2843738498 shutdown_connections 2024-04-25T10:11:52.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 -- 172.21.15.133:0/2843738498 wait complete. 2024-04-25T10:11:52.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 Processor -- start 2024-04-25T10:11:52.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 -- start start 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 0x7fe0600ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0601561f0 0x7fe0600a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0600a8c00 0x7fe06006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600cd670 con 0x7fe04c004d50 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06015bc00 con 0x7fe0601561f0 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600cd7f0 con 0x7fe0600a8c00 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe065f65640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 0x7fe0600ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe065f65640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 0x7fe0600ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34460/0 (socket says 172.21.15.133:34460) 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe065f65640 1 -- 172.21.15.133:0/340762696 learned_addr learned my addr 172.21.15.133:0/340762696 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe066766640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0600a8c00 0x7fe06006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe065764640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0601561f0 0x7fe0600a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe065f65640 1 -- 172.21.15.133:0/340762696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0600a8c00 msgr2=0x7fe06006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe065f65640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0600a8c00 0x7fe06006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe065f65640 1 -- 172.21.15.133:0/340762696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0601561f0 msgr2=0x7fe0600a86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe065f65640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0601561f0 0x7fe0600a86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe065f65640 1 -- 172.21.15.133:0/340762696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe060156670 con 0x7fe04c004d50 2024-04-25T10:11:52.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe065764640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0601561f0 0x7fe0600a86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:52.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe065f65640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 0x7fe0600ca850 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe05c011610 tx=0x7fe05c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0567fc640 1 -- 172.21.15.133:0/340762696 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00f8f0 con 0x7fe04c004d50 2024-04-25T10:11:52.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0567fc640 1 -- 172.21.15.133:0/340762696 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c00fa90 con 0x7fe04c004d50 2024-04-25T10:11:52.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 -- 172.21.15.133:0/340762696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0600c97b0 con 0x7fe04c004d50 2024-04-25T10:11:52.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0567fc640 1 -- 172.21.15.133:0/340762696 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00fe40 con 0x7fe04c004d50 2024-04-25T10:11:52.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.977+0000 7fe0681c9640 1 -- 172.21.15.133:0/340762696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0600c9d70 con 0x7fe04c004d50 2024-04-25T10:11:52.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.981+0000 7fe0681c9640 1 -- 172.21.15.133:0/340762696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06006bc20 con 0x7fe04c004d50 2024-04-25T10:11:52.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.981+0000 7fe0567fc640 1 -- 172.21.15.133:0/340762696 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe05c015460 con 0x7fe04c004d50 2024-04-25T10:11:52.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.981+0000 7fe0567fc640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe044066f60 0x7fe044069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.981+0000 7fe0567fc640 1 -- 172.21.15.133:0/340762696 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe05c085b20 con 0x7fe04c004d50 2024-04-25T10:11:52.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.981+0000 7fe065764640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe044066f60 0x7fe044069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.981+0000 7fe065764640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe044066f60 0x7fe044069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe0600b1680 tx=0x7fe050008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:52.985+0000 7fe0567fc640 1 -- 172.21.15.133:0/340762696 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe05c019030 con 0x7fe04c004d50 2024-04-25T10:11:53.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.201+0000 7fe0681c9640 1 -- 172.21.15.133:0/340762696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe0600cd0a0 con 0x7fe04c004d50 2024-04-25T10:11:53.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0567fc640 1 -- 172.21.15.133:0/340762696 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7fe05c056a00 con 0x7fe04c004d50 2024-04-25T10:11:53.207 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:53.207 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":626,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078164813,"bytes_used":2674196480,"bytes_avail":770419916800,"bytes_total":773094113280,"write_bytes_sec":18923789,"read_op_per_sec":0,"write_op_per_sec":20},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:11:49.888630+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 -- 172.21.15.133:0/340762696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe044066f60 msgr2=0x7fe044069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe044066f60 0x7fe044069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe0600b1680 tx=0x7fe050008040 comp rx=0 tx=0).stop 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 -- 172.21.15.133:0/340762696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 msgr2=0x7fe0600ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 0x7fe0600ca850 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe05c011610 tx=0x7fe05c011ad0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 -- 172.21.15.133:0/340762696 shutdown_connections 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0600a8c00 0x7fe06006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0601561f0 0x7fe0600a86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe044066f60 0x7fe044069420 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 --2- 172.21.15.133:0/340762696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe04c004d50 0x7fe0600ca850 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 -- 172.21.15.133:0/340762696 >> 172.21.15.133:0/340762696 conn(0x7fe060067000 msgr2=0x7fe0600cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 -- 172.21.15.133:0/340762696 shutdown_connections 2024-04-25T10:11:53.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:53.205+0000 7fe0681c9640 1 -- 172.21.15.133:0/340762696 wait complete. 2024-04-25T10:11:53.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:11:54.226 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:54.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f038029f640 1 Processor -- start 2024-04-25T10:11:54.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f038029f640 1 -- start start 2024-04-25T10:11:54.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f038029f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f037815a5f0 0x7f03780c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f038029f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f037806eb70 0x7f03780cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f038029f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03780cc5a0 con 0x7f037815a9d0 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f038029f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03780cc880 con 0x7f037806eb70 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f038029f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03780ccb60 con 0x7f037815a5f0 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f037e83c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f037806eb70 0x7f03780cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f037e83c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f037806eb70 0x7f03780cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34476/0 (socket says 172.21.15.133:34476) 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f037e83c640 1 -- 172.21.15.133:0/962759181 learned_addr learned my addr 172.21.15.133:0/962759181 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f037e03b640 1 --2- 172.21.15.133:0/962759181 >> v2:172.21.15.162:3300/0 conn(0x7f037815a5f0 0x7f03780c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f037e83c640 1 -- 172.21.15.133:0/962759181 >> v2:172.21.15.162:3300/0 conn(0x7f037815a5f0 msgr2=0x7f03780c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f037e83c640 1 --2- 172.21.15.133:0/962759181 >> v2:172.21.15.162:3300/0 conn(0x7f037815a5f0 0x7f03780c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f037e83c640 1 -- 172.21.15.133:0/962759181 >> v1:172.21.15.162:6789/0 conn(0x7f037815a9d0 legacy=0x7f037806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f037e83c640 1 -- 172.21.15.133:0/962759181 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03780cce40 con 0x7f037806eb70 2024-04-25T10:11:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.357+0000 7f037e03b640 1 --2- 172.21.15.133:0/962759181 >> v2:172.21.15.162:3300/0 conn(0x7f037815a5f0 0x7f03780c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f037e83c640 1 --2- 172.21.15.133:0/962759181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f037806eb70 0x7f03780cbe50 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f037806f350 tx=0x7f037001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a222a0cf964f5dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f037d039640 1 -- 172.21.15.133:0/962759181 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03700a56c0 con 0x7f037806eb70 2024-04-25T10:11:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f037d039640 1 -- 172.21.15.133:0/962759181 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03700a5860 con 0x7f037806eb70 2024-04-25T10:11:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f037d039640 1 -- 172.21.15.133:0/962759181 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03700a5c30 con 0x7f037806eb70 2024-04-25T10:11:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 -- 172.21.15.133:0/962759181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f037806eb70 msgr2=0x7f03780cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 --2- 172.21.15.133:0/962759181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f037806eb70 0x7f03780cbe50 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f037806f350 tx=0x7f037001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 -- 172.21.15.133:0/962759181 shutdown_connections 2024-04-25T10:11:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 --2- 172.21.15.133:0/962759181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f037806eb70 0x7f03780cbe50 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 --2- 172.21.15.133:0/962759181 >> v2:172.21.15.162:3300/0 conn(0x7f037815a5f0 0x7f03780c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 -- 172.21.15.133:0/962759181 >> 172.21.15.133:0/962759181 conn(0x7f0378067000 msgr2=0x7f037806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 -- 172.21.15.133:0/962759181 shutdown_connections 2024-04-25T10:11:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 -- 172.21.15.133:0/962759181 wait complete. 2024-04-25T10:11:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 Processor -- start 2024-04-25T10:11:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 -- start start 2024-04-25T10:11:54.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f037806eb70 0x7f0378151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f037815a5f0 0x7f0378151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f037815a9d0 0x7f0378163b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03780ce470 con 0x7f037815a9d0 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03780ce2f0 con 0x7f037806eb70 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.361+0000 7f038029f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03780ce5f0 con 0x7f037815a5f0 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e83c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f037815a9d0 0x7f0378163b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e83c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f037815a9d0 0x7f0378163b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34484/0 (socket says 172.21.15.133:34484) 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e83c640 1 -- 172.21.15.133:0/1335948257 learned_addr learned my addr 172.21.15.133:0/1335948257 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e03b640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f037806eb70 0x7f0378151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037d83a640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f037815a5f0 0x7f0378151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e83c640 1 -- 172.21.15.133:0/1335948257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f037815a5f0 msgr2=0x7f0378151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e83c640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f037815a5f0 0x7f0378151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e83c640 1 -- 172.21.15.133:0/1335948257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f037806eb70 msgr2=0x7f0378151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e83c640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f037806eb70 0x7f0378151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e83c640 1 -- 172.21.15.133:0/1335948257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0378152290 con 0x7f037815a9d0 2024-04-25T10:11:54.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037d83a640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f037815a5f0 0x7f0378151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e03b640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f037806eb70 0x7f0378151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e83c640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f037815a9d0 0x7f0378163b20 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f03700af040 tx=0x7f0370019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f036effd640 1 -- 172.21.15.133:0/1335948257 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03700bc020 con 0x7f037815a9d0 2024-04-25T10:11:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f036effd640 1 -- 172.21.15.133:0/1335948257 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0370007bb0 con 0x7f037815a9d0 2024-04-25T10:11:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f036effd640 1 -- 172.21.15.133:0/1335948257 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370009510 con 0x7f037815a9d0 2024-04-25T10:11:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f038029f640 1 -- 172.21.15.133:0/1335948257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03781516a0 con 0x7f037815a9d0 2024-04-25T10:11:54.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f038029f640 1 -- 172.21.15.133:0/1335948257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03781527f0 con 0x7f037815a9d0 2024-04-25T10:11:54.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f038029f640 1 -- 172.21.15.133:0/1335948257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0378068ef0 con 0x7f037815a9d0 2024-04-25T10:11:54.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f036effd640 1 -- 172.21.15.133:0/1335948257 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03700bf020 con 0x7f037815a9d0 2024-04-25T10:11:54.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f036effd640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f035c066f60 0x7f035c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f037e03b640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f035c066f60 0x7f035c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.365+0000 7f036effd640 1 -- 172.21.15.133:0/1335948257 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f03701205e0 con 0x7f037815a9d0 2024-04-25T10:11:54.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.369+0000 7f037e03b640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f035c066f60 0x7f035c069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0374009970 tx=0x7f0374008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.369+0000 7f036effd640 1 -- 172.21.15.133:0/1335948257 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03700c6030 con 0x7f037815a9d0 2024-04-25T10:11:54.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.565+0000 7f038029f640 1 -- 172.21.15.133:0/1335948257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03780b3d90 con 0x7f037815a9d0 2024-04-25T10:11:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.565+0000 7f036effd640 1 -- 172.21.15.133:0/1335948257 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0378152290 con 0x7f037815a9d0 2024-04-25T10:11:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:54.568 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:54.568 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:54.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.565+0000 7f038029f640 1 -- 172.21.15.133:0/1335948257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f035c066f60 msgr2=0x7f035c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.565+0000 7f038029f640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f035c066f60 0x7f035c069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0374009970 tx=0x7f0374008040 comp rx=0 tx=0).stop 2024-04-25T10:11:54.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.565+0000 7f038029f640 1 -- 172.21.15.133:0/1335948257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f037815a9d0 msgr2=0x7f0378163b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.565+0000 7f038029f640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f037815a9d0 0x7f0378163b20 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f03700af040 tx=0x7f0370019790 comp rx=0 tx=0).stop 2024-04-25T10:11:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.569+0000 7f038029f640 1 -- 172.21.15.133:0/1335948257 shutdown_connections 2024-04-25T10:11:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.569+0000 7f038029f640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f035c066f60 0x7f035c069420 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.569+0000 7f038029f640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f037815a9d0 0x7f0378163b20 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.569+0000 7f038029f640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f037815a5f0 0x7f0378151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.569+0000 7f038029f640 1 --2- 172.21.15.133:0/1335948257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f037806eb70 0x7f0378151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.569+0000 7f038029f640 1 -- 172.21.15.133:0/1335948257 >> 172.21.15.133:0/1335948257 conn(0x7f0378067000 msgr2=0x7f037806d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.569+0000 7f038029f640 1 -- 172.21.15.133:0/1335948257 shutdown_connections 2024-04-25T10:11:54.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.569+0000 7f038029f640 1 -- 172.21.15.133:0/1335948257 wait complete. 2024-04-25T10:11:54.587 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:11:54.587 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:54.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70f4b3640 1 Processor -- start 2024-04-25T10:11:54.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70f4b3640 1 -- start start 2024-04-25T10:11:54.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70f4b3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff708158090 0x7ff7080b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70f4b3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70815e1b0 con 0x7ff70815ec50 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70f4b3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7080b4cb0 con 0x7ff70806b3a0 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70f4b3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70815df50 con 0x7ff708158090 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70ca4e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff70815ec50 0x7ff70806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51726/0 (socket says 172.21.15.133:51726) 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70d24f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff708158090 0x7ff7080b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70ca4e640 1 -- 172.21.15.133:0/3076756315 learned_addr learned my addr 172.21.15.133:0/3076756315 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff6fffff640 1 -- 172.21.15.133:0/3076756315 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950727854 0 0) 0x7ff70815e1b0 con 0x7ff70815ec50 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff6fffff640 1 -- 172.21.15.133:0/3076756315 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f4003530 con 0x7ff70815ec50 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70d24f640 1 -- 172.21.15.133:0/3076756315 >> v1:172.21.15.162:6789/0 conn(0x7ff70806b3a0 legacy=0x7ff708161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70d24f640 1 -- 172.21.15.133:0/3076756315 >> v1:172.21.15.133:6789/0 conn(0x7ff70815ec50 legacy=0x7ff70806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.697+0000 7ff70d24f640 1 -- 172.21.15.133:0/3076756315 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7080b49d0 con 0x7ff708158090 2024-04-25T10:11:54.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.701+0000 7ff70d24f640 1 --2- 172.21.15.133:0/3076756315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff708158090 0x7ff7080b4180 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ff7040097c0 tx=0x7ff70401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef78108fa964c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.701+0000 7ff6fffff640 1 -- 172.21.15.133:0/3076756315 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7040b0070 con 0x7ff708158090 2024-04-25T10:11:54.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.701+0000 7ff6fffff640 1 -- 172.21.15.133:0/3076756315 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7040ab600 con 0x7ff708158090 2024-04-25T10:11:54.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.701+0000 7ff6fffff640 1 -- 172.21.15.133:0/3076756315 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7040ab9b0 con 0x7ff708158090 2024-04-25T10:11:54.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.701+0000 7ff70f4b3640 1 -- 172.21.15.133:0/3076756315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff708158090 msgr2=0x7ff7080b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.701+0000 7ff70f4b3640 1 --2- 172.21.15.133:0/3076756315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff708158090 0x7ff7080b4180 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ff7040097c0 tx=0x7ff70401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.701+0000 7ff70f4b3640 1 -- 172.21.15.133:0/3076756315 shutdown_connections 2024-04-25T10:11:54.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.701+0000 7ff70f4b3640 1 --2- 172.21.15.133:0/3076756315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff708158090 0x7ff7080b4180 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.701+0000 7ff70f4b3640 1 -- 172.21.15.133:0/3076756315 >> 172.21.15.133:0/3076756315 conn(0x7ff7080b2370 msgr2=0x7ff7080b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.701+0000 7ff70f4b3640 1 -- 172.21.15.133:0/3076756315 shutdown_connections 2024-04-25T10:11:54.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.701+0000 7ff70f4b3640 1 -- 172.21.15.133:0/3076756315 wait complete. 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70f4b3640 1 Processor -- start 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70f4b3640 1 -- start start 2024-04-25T10:11:54.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70f4b3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70806b3a0 0x7ff708158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70f4b3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff708158090 0x7ff70815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70f4b3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70815ec50 0x7ff708154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70f4b3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70816b890 con 0x7ff70815ec50 2024-04-25T10:11:54.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70f4b3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70816b710 con 0x7ff70806b3a0 2024-04-25T10:11:54.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70f4b3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70816ba10 con 0x7ff708158090 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70da50640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70815ec50 0x7ff708154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70da50640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70815ec50 0x7ff708154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34502/0 (socket says 172.21.15.133:34502) 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70da50640 1 -- 172.21.15.133:0/809812460 learned_addr learned my addr 172.21.15.133:0/809812460 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70d24f640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70806b3a0 0x7ff708158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70ca4e640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff708158090 0x7ff70815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70da50640 1 -- 172.21.15.133:0/809812460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff708158090 msgr2=0x7ff70815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70da50640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff708158090 0x7ff70815af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70da50640 1 -- 172.21.15.133:0/809812460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70806b3a0 msgr2=0x7ff708158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70da50640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70806b3a0 0x7ff708158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70da50640 1 -- 172.21.15.133:0/809812460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7081594c0 con 0x7ff70815ec50 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70da50640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70815ec50 0x7ff708154010 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7ff70000c7a0 tx=0x7ff70000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff6fdffb640 1 -- 172.21.15.133:0/809812460 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700003b80 con 0x7ff70815ec50 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff6fdffb640 1 -- 172.21.15.133:0/809812460 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff700003d20 con 0x7ff70815ec50 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff6fdffb640 1 -- 172.21.15.133:0/809812460 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70000aa30 con 0x7ff70815ec50 2024-04-25T10:11:54.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70f4b3640 1 -- 172.21.15.133:0/809812460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7080ce610 con 0x7ff70815ec50 2024-04-25T10:11:54.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70f4b3640 1 -- 172.21.15.133:0/809812460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff708159a30 con 0x7ff70815ec50 2024-04-25T10:11:54.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70f4b3640 1 -- 172.21.15.133:0/809812460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff708067c70 con 0x7ff70815ec50 2024-04-25T10:11:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff6fdffb640 1 -- 172.21.15.133:0/809812460 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff70000abd0 con 0x7ff70815ec50 2024-04-25T10:11:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff6fdffb640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6ec066f60 0x7ff6ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff6fdffb640 1 -- 172.21.15.133:0/809812460 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff7000850a0 con 0x7ff70815ec50 2024-04-25T10:11:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.705+0000 7ff70d24f640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6ec066f60 0x7ff6ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:54.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.709+0000 7ff70d24f640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6ec066f60 0x7ff6ec069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff7040ad000 tx=0x7ff7040a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:54.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.709+0000 7ff6fdffb640 1 -- 172.21.15.133:0/809812460 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff700055f80 con 0x7ff70815ec50 2024-04-25T10:11:54.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.913+0000 7ff70f4b3640 1 -- 172.21.15.133:0/809812460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff7080ccec0 con 0x7ff70815ec50 2024-04-25T10:11:54.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.913+0000 7ff6fdffb640 1 -- 172.21.15.133:0/809812460 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7ff700055880 con 0x7ff70815ec50 2024-04-25T10:11:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:54.917 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":627,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078164813,"bytes_used":2674196480,"bytes_avail":770419916800,"bytes_total":773094113280,"write_bytes_sec":18924072,"read_op_per_sec":0,"write_op_per_sec":20},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:11:49.888630+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 -- 172.21.15.133:0/809812460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6ec066f60 msgr2=0x7ff6ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6ec066f60 0x7ff6ec069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff7040ad000 tx=0x7ff7040a6000 comp rx=0 tx=0).stop 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 -- 172.21.15.133:0/809812460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70815ec50 msgr2=0x7ff708154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70815ec50 0x7ff708154010 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7ff70000c7a0 tx=0x7ff70000cc60 comp rx=0 tx=0).stop 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 -- 172.21.15.133:0/809812460 shutdown_connections 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6ec066f60 0x7ff6ec069420 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70815ec50 0x7ff708154010 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff708158090 0x7ff70815af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 --2- 172.21.15.133:0/809812460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70806b3a0 0x7ff708158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 -- 172.21.15.133:0/809812460 >> 172.21.15.133:0/809812460 conn(0x7ff7080b2370 msgr2=0x7ff70815f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 -- 172.21.15.133:0/809812460 shutdown_connections 2024-04-25T10:11:54.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:54.917+0000 7ff70f4b3640 1 -- 172.21.15.133:0/809812460 wait complete. 2024-04-25T10:11:54.934 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:11:55.935 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:56.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa5a7a3640 1 Processor -- start 2024-04-25T10:11:56.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa5a7a3640 1 -- start start 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa5a7a3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7faa5415c9a0 0x7faa540c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa5a7a3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa540c61d0 con 0x7faa540ce330 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa5a7a3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa540c64b0 con 0x7faa540b1f70 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa5a7a3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5406d4f0 con 0x7faa5415c9a0 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa58d40640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7faa540ce330 0x7faa5415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:51738/0 (socket says 172.21.15.133:51738) 2024-04-25T10:11:56.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa58d40640 1 -- 172.21.15.133:0/2934099000 learned_addr learned my addr 172.21.15.133:0/2934099000 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:56.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa53fff640 1 --2- 172.21.15.133:0/2934099000 >> v2:172.21.15.162:3300/0 conn(0x7faa5415c9a0 0x7faa540c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255698666 0 0) 0x7faa540c61d0 con 0x7faa540ce330 2024-04-25T10:11:56.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa3c003530 con 0x7faa540ce330 2024-04-25T10:11:56.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 353827201 0 0) 0x7faa3c003530 con 0x7faa540ce330 2024-04-25T10:11:56.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 >> v2:172.21.15.162:3300/0 conn(0x7faa5415c9a0 msgr2=0x7faa540c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 --2- 172.21.15.133:0/2934099000 >> v2:172.21.15.162:3300/0 conn(0x7faa5415c9a0 0x7faa540c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 >> v1:172.21.15.162:6789/0 conn(0x7faa540b1f70 legacy=0x7faa5406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5406d7d0 con 0x7faa540ce330 2024-04-25T10:11:56.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7faa4c001a00 con 0x7faa540ce330 2024-04-25T10:11:56.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 >> v1:172.21.15.133:6789/0 conn(0x7faa540ce330 legacy=0x7faa5415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 --2- 172.21.15.133:0/2934099000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa3c003a10 0x7faa3c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 --2- 172.21.15.133:0/2934099000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa3c004810 0x7faa3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 --2- 172.21.15.133:0/2934099000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa3c0051d0 0x7faa3c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5406d4f0 con 0x7faa3c0051d0 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c00c0d0 con 0x7faa3c003a10 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c00c2d0 con 0x7faa3c004810 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa58d40640 1 --2- 172.21.15.133:0/2934099000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa3c0051d0 0x7faa3c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa53fff640 1 --2- 172.21.15.133:0/2934099000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa3c003a10 0x7faa3c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa537fe640 1 --2- 172.21.15.133:0/2934099000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa3c004810 0x7faa3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa58d40640 1 -- 172.21.15.133:0/2934099000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa3c004810 msgr2=0x7faa3c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa58d40640 1 --2- 172.21.15.133:0/2934099000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa3c004810 0x7faa3c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa58d40640 1 -- 172.21.15.133:0/2934099000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa3c003a10 msgr2=0x7faa3c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa58d40640 1 --2- 172.21.15.133:0/2934099000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa3c003a10 0x7faa3c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.065+0000 7faa58d40640 1 -- 172.21.15.133:0/2934099000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c003530 con 0x7faa3c0051d0 2024-04-25T10:11:56.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa58d40640 1 -- 172.21.15.133:0/2934099000 reap_dead start 2024-04-25T10:11:56.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa58d40640 1 --2- 172.21.15.133:0/2934099000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa3c0051d0 0x7faa3c00b9d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00a790 tx=0x7faa4c01d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca72c4353f973678 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00a3b0 con 0x7faa3c0051d0 2024-04-25T10:11:56.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c0ad650 con 0x7faa3c0051d0 2024-04-25T10:11:56.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa52ffd640 1 -- 172.21.15.133:0/2934099000 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c0ada40 con 0x7faa3c0051d0 2024-04-25T10:11:56.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2934099000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa3c0051d0 msgr2=0x7faa3c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa5a7a3640 1 --2- 172.21.15.133:0/2934099000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa3c0051d0 0x7faa3c00b9d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00a790 tx=0x7faa4c01d2c0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2934099000 shutdown_connections 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa5a7a3640 1 --2- 172.21.15.133:0/2934099000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa3c0051d0 0x7faa3c00b9d0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2934099000 >> 172.21.15.133:0/2934099000 conn(0x7faa54067000 msgr2=0x7faa5406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2934099000 shutdown_connections 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2934099000 wait complete. 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa5a7a3640 1 Processor -- start 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.069+0000 7faa5a7a3640 1 -- start start 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa5a7a3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa3c0051d0 0x7faa54152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa5a7a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa5415b1b0 0x7faa540b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa5a7a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa5413fa70 0x7faa5413fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa5a7a3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa540ce780 con 0x7faa5413fa70 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa5a7a3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa540ce900 con 0x7faa3c0051d0 2024-04-25T10:11:56.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa5a7a3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa540cea80 con 0x7faa5415b1b0 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa58d40640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa5413fa70 0x7faa5413fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa58d40640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa5413fa70 0x7faa5413fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34518/0 (socket says 172.21.15.133:34518) 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa58d40640 1 -- 172.21.15.133:0/2146479699 learned_addr learned my addr 172.21.15.133:0/2146479699 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa53fff640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa3c0051d0 0x7faa54152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa537fe640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa5415b1b0 0x7faa540b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa58d40640 1 -- 172.21.15.133:0/2146479699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa5415b1b0 msgr2=0x7faa540b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa58d40640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa5415b1b0 0x7faa540b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa58d40640 1 -- 172.21.15.133:0/2146479699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa3c0051d0 msgr2=0x7faa54152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa58d40640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa3c0051d0 0x7faa54152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa58d40640 1 -- 172.21.15.133:0/2146479699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5414c2e0 con 0x7faa5413fa70 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa58d40640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa5413fa70 0x7faa5413fef0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7faa4c01d0f0 tx=0x7faa3c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa50ff9640 1 -- 172.21.15.133:0/2146479699 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c001940 con 0x7faa5413fa70 2024-04-25T10:11:56.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa50ff9640 1 -- 172.21.15.133:0/2146479699 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c0b4070 con 0x7faa5413fa70 2024-04-25T10:11:56.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2146479699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5415b630 con 0x7faa5413fa70 2024-04-25T10:11:56.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa50ff9640 1 -- 172.21.15.133:0/2146479699 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c0ada00 con 0x7faa5413fa70 2024-04-25T10:11:56.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2146479699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa54152b30 con 0x7faa5413fa70 2024-04-25T10:11:56.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2146479699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa54068ef0 con 0x7faa5413fa70 2024-04-25T10:11:56.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa50ff9640 1 -- 172.21.15.133:0/2146479699 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa4c0b6020 con 0x7faa5413fa70 2024-04-25T10:11:56.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa50ff9640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa40066f60 0x7faa40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa53fff640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa40066f60 0x7faa40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa50ff9640 1 -- 172.21.15.133:0/2146479699 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7faa4c11f1b0 con 0x7faa5413fa70 2024-04-25T10:11:56.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.073+0000 7faa53fff640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa40066f60 0x7faa40069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7faa480099b0 tx=0x7faa48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.077+0000 7faa50ff9640 1 -- 172.21.15.133:0/2146479699 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa4c016fd0 con 0x7faa5413fa70 2024-04-25T10:11:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.261+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2146479699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa5406b580 con 0x7faa5413fa70 2024-04-25T10:11:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.261+0000 7faa50ff9640 1 -- 172.21.15.133:0/2146479699 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7faa4c0f0090 con 0x7faa5413fa70 2024-04-25T10:11:56.264 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:56.265 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:56.265 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2146479699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa40066f60 msgr2=0x7faa40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa40066f60 0x7faa40069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7faa480099b0 tx=0x7faa48008040 comp rx=0 tx=0).stop 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2146479699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa5413fa70 msgr2=0x7faa5413fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa5413fa70 0x7faa5413fef0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7faa4c01d0f0 tx=0x7faa3c005840 comp rx=0 tx=0).stop 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2146479699 shutdown_connections 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faa40066f60 0x7faa40069420 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa5413fa70 0x7faa5413fef0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faa5415b1b0 0x7faa540b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 --2- 172.21.15.133:0/2146479699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa3c0051d0 0x7faa54152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2146479699 >> 172.21.15.133:0/2146479699 conn(0x7faa54067000 msgr2=0x7faa5414f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2146479699 shutdown_connections 2024-04-25T10:11:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.265+0000 7faa5a7a3640 1 -- 172.21.15.133:0/2146479699 wait complete. 2024-04-25T10:11:56.282 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:11:56.282 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:56.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e240cc640 1 Processor -- start 2024-04-25T10:11:56.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e240cc640 1 -- start start 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e240cc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1e1c15d9b0 0x7f1e1c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e240cc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c15dd90 0x7f1e1c0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e240cc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c0ca9b0 con 0x7f1e1c0b19f0 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e240cc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c0cbe30 con 0x7f1e1c15d9b0 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e240cc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0cac90 con 0x7f1e1c15dd90 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e22669640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1e1c0b19f0 0x7f1e1c1616f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48186/0 (socket says 172.21.15.133:48186) 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e22669640 1 -- 172.21.15.133:0/2085951490 learned_addr learned my addr 172.21.15.133:0/2085951490 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e21667640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c15dd90 0x7f1e1c0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060329234 0 0) 0x7f1e1c0ca9b0 con 0x7f1e1c0b19f0 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e08003530 con 0x7f1e1c0b19f0 2024-04-25T10:11:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.389+0000 7f1e21e68640 1 --2- 172.21.15.133:0/2085951490 >> v2:172.21.15.162:3300/0 conn(0x7f1e1c15d9b0 0x7f1e1c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 531075619 0 0) 0x7f1e08003530 con 0x7f1e1c0b19f0 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c15dd90 msgr2=0x7f1e1c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c15dd90 0x7f1e1c0ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 >> v2:172.21.15.162:3300/0 conn(0x7f1e1c15d9b0 msgr2=0x7f1e1c0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 --2- 172.21.15.133:0/2085951490 >> v2:172.21.15.162:3300/0 conn(0x7f1e1c15d9b0 0x7f1e1c0cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c0caf70 con 0x7f1e1c0b19f0 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1e10001680 con 0x7f1e1c0b19f0 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 >> v1:172.21.15.133:6789/0 conn(0x7f1e1c0b19f0 legacy=0x7f1e1c1616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c15dd90 0x7f1e080039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e08003f40 0x7f1e08004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e08004d50 0x7f1e080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0cbe30 con 0x7f1e08004d50 2024-04-25T10:11:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0cac90 con 0x7f1e08003f40 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0800bad0 con 0x7f1e1c15dd90 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e22669640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e08004d50 0x7f1e080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e21e68640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c15dd90 0x7f1e080039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e21667640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e08003f40 0x7f1e08004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e22669640 1 -- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c15dd90 msgr2=0x7f1e080039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e22669640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c15dd90 0x7f1e080039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e22669640 1 -- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e08003f40 msgr2=0x7f1e08004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e22669640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e08003f40 0x7f1e08004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e22669640 1 -- 172.21.15.133:0/2085951490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e08003530 con 0x7f1e08004d50 2024-04-25T10:11:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e21e68640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c15dd90 0x7f1e080039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e22669640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e08004d50 0x7f1e080051d0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0b1e20 tx=0x7f1e1001f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127585c0f96ac160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10007d80 con 0x7f1e08004d50 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e10005b00 con 0x7f1e08004d50 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e20e66640 1 -- 172.21.15.133:0/2085951490 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10007070 con 0x7f1e08004d50 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e240cc640 1 -- 172.21.15.133:0/2085951490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e08004d50 msgr2=0x7f1e080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e240cc640 1 --2- 172.21.15.133:0/2085951490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e08004d50 0x7f1e080051d0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0b1e20 tx=0x7f1e1001f3d0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e22669640 1 -- 172.21.15.133:0/2085951490 reap_dead start 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e240cc640 1 -- 172.21.15.133:0/2085951490 shutdown_connections 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e240cc640 1 -- 172.21.15.133:0/2085951490 >> 172.21.15.133:0/2085951490 conn(0x7f1e1c067000 msgr2=0x7f1e1c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e240cc640 1 -- 172.21.15.133:0/2085951490 shutdown_connections 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.393+0000 7f1e240cc640 1 -- 172.21.15.133:0/2085951490 wait complete. 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e240cc640 1 Processor -- start 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e240cc640 1 -- start start 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e240cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e1c14cbb0 0x7f1e1c1f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e240cc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c14cf90 0x7f1e1c151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e240cc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e1c14ada0 0x7f1e1c14a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e240cc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c06e880 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e240cc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c06e700 con 0x7f1e1c14ada0 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e240cc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c06ea00 con 0x7f1e1c14cf90 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e21e68640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e1c14cbb0 0x7f1e1c1f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e21e68640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e1c14cbb0 0x7f1e1c1f38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33602/0 (socket says 172.21.15.133:33602) 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e21e68640 1 -- 172.21.15.133:0/2519962520 learned_addr learned my addr 172.21.15.133:0/2519962520 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e21667640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c14cf90 0x7f1e1c151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e21e68640 1 -- 172.21.15.133:0/2519962520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c14cf90 msgr2=0x7f1e1c151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e21e68640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c14cf90 0x7f1e1c151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e21e68640 1 -- 172.21.15.133:0/2519962520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e1c14ada0 msgr2=0x7f1e1c14a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e22669640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e1c14ada0 0x7f1e1c14a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e21e68640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e1c14ada0 0x7f1e1c14a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e21e68640 1 -- 172.21.15.133:0/2519962520 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c153960 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e21e68640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e1c14cbb0 0x7f1e1c1f38a0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400e810 tx=0x7f1e1400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e0e7fc640 1 -- 172.21.15.133:0/2519962520 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14017070 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e0e7fc640 1 -- 172.21.15.133:0/2519962520 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1400c8d0 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e240cc640 1 -- 172.21.15.133:0/2519962520 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c153c40 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e0e7fc640 1 -- 172.21.15.133:0/2519962520 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400f040 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e240cc640 1 -- 172.21.15.133:0/2519962520 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c154200 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e240cc640 1 -- 172.21.15.133:0/2519962520 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1c068ef0 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e0e7fc640 1 -- 172.21.15.133:0/2519962520 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e1401a070 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e0e7fc640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e00066f60 0x7f1e00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.397+0000 7f1e0e7fc640 1 -- 172.21.15.133:0/2519962520 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1e14084e30 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.401+0000 7f1e21667640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e00066f60 0x7f1e00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.401+0000 7f1e21667640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e00066f60 0x7f1e00069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c14dd20 tx=0x7f1e1800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.405+0000 7f1e0e7fc640 1 -- 172.21.15.133:0/2519962520 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e14055d10 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.617+0000 7f1e240cc640 1 -- 172.21.15.133:0/2519962520 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1e1c0b2e30 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.617+0000 7f1e0e7fc640 1 -- 172.21.15.133:0/2519962520 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1637 (secure 0 0 0) 0x7f1e14055610 con 0x7f1e1c14cbb0 2024-04-25T10:11:56.621 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:56.621 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":629,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078432287,"bytes_used":2674728960,"bytes_avail":770419384320,"bytes_total":773094113280,"write_bytes_sec":18946250,"read_op_per_sec":0,"write_op_per_sec":21},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:11:55.890677+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 -- 172.21.15.133:0/2519962520 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e00066f60 msgr2=0x7f1e00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e00066f60 0x7f1e00069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c14dd20 tx=0x7f1e1800b040 comp rx=0 tx=0).stop 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 -- 172.21.15.133:0/2519962520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e1c14cbb0 msgr2=0x7f1e1c1f38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e1c14cbb0 0x7f1e1c1f38a0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400e810 tx=0x7f1e1400ecd0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 -- 172.21.15.133:0/2519962520 shutdown_connections 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e1c14ada0 0x7f1e1c14a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e1c14cf90 0x7f1e1c151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e00066f60 0x7f1e00069420 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 --2- 172.21.15.133:0/2519962520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e1c14cbb0 0x7f1e1c1f38a0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 -- 172.21.15.133:0/2519962520 >> 172.21.15.133:0/2519962520 conn(0x7f1e1c067000 msgr2=0x7f1e1c14e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 -- 172.21.15.133:0/2519962520 shutdown_connections 2024-04-25T10:11:56.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:56.621+0000 7f1e240cc640 1 -- 172.21.15.133:0/2519962520 wait complete. 2024-04-25T10:11:56.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:11:57.640 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:57.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f097ac08640 1 Processor -- start 2024-04-25T10:11:57.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f097ac08640 1 -- start start 2024-04-25T10:11:57.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f097ac08640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f097415c9a0 0x7f09740c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f097ac08640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f09740b1f70 0x7f097406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f097ac08640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09740ce330 0x7f09740ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f097ac08640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09740cecc0 con 0x7f09740b1f70 2024-04-25T10:11:57.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f097ac08640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09740c61d0 con 0x7f097415c9a0 2024-04-25T10:11:57.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f097ac08640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09740c64b0 con 0x7f09740ce330 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f0973fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f09740b1f70 0x7f097406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f0973fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f09740b1f70 0x7f097406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33618/0 (socket says 172.21.15.133:33618) 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f0973fff640 1 -- 172.21.15.133:0/1210581092 learned_addr learned my addr 172.21.15.133:0/1210581092 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f09791a5640 1 --2- 172.21.15.133:0/1210581092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09740ce330 0x7f09740ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f09789a4640 1 --2- 172.21.15.133:0/1210581092 >> v2:172.21.15.162:3300/0 conn(0x7f097415c9a0 0x7f09740c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f0973fff640 1 -- 172.21.15.133:0/1210581092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09740ce330 msgr2=0x7f09740ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f0973fff640 1 --2- 172.21.15.133:0/1210581092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09740ce330 0x7f09740ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f0973fff640 1 -- 172.21.15.133:0/1210581092 >> v2:172.21.15.162:3300/0 conn(0x7f097415c9a0 msgr2=0x7f09740c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f0973fff640 1 --2- 172.21.15.133:0/1210581092 >> v2:172.21.15.162:3300/0 conn(0x7f097415c9a0 0x7f09740c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f0973fff640 1 -- 172.21.15.133:0/1210581092 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097406d4f0 con 0x7f09740b1f70 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f09791a5640 1 --2- 172.21.15.133:0/1210581092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09740ce330 0x7f09740ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:57.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.769+0000 7f09789a4640 1 --2- 172.21.15.133:0/1210581092 >> v2:172.21.15.162:3300/0 conn(0x7f097415c9a0 0x7f09740c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:57.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f0973fff640 1 --2- 172.21.15.133:0/1210581092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09740b1f70 0x7f097406dcf0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f097406f5f0 tx=0x7f096401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0d0b9da8b848a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f09737fe640 1 -- 172.21.15.133:0/1210581092 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09640a7070 con 0x7f09740b1f70 2024-04-25T10:11:57.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f09737fe640 1 -- 172.21.15.133:0/1210581092 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09640b0070 con 0x7f09740b1f70 2024-04-25T10:11:57.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f09737fe640 1 -- 172.21.15.133:0/1210581092 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09640ab890 con 0x7f09740b1f70 2024-04-25T10:11:57.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 -- 172.21.15.133:0/1210581092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09740b1f70 msgr2=0x7f097406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 --2- 172.21.15.133:0/1210581092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09740b1f70 0x7f097406dcf0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f097406f5f0 tx=0x7f096401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 -- 172.21.15.133:0/1210581092 shutdown_connections 2024-04-25T10:11:57.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 --2- 172.21.15.133:0/1210581092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09740ce330 0x7f09740ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 --2- 172.21.15.133:0/1210581092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09740b1f70 0x7f097406dcf0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 --2- 172.21.15.133:0/1210581092 >> v2:172.21.15.162:3300/0 conn(0x7f097415c9a0 0x7f09740c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 -- 172.21.15.133:0/1210581092 >> 172.21.15.133:0/1210581092 conn(0x7f0974067000 msgr2=0x7f097406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 -- 172.21.15.133:0/1210581092 shutdown_connections 2024-04-25T10:11:57.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 -- 172.21.15.133:0/1210581092 wait complete. 2024-04-25T10:11:57.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 Processor -- start 2024-04-25T10:11:57.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 -- start start 2024-04-25T10:11:57.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09740b1f70 0x7f09740cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09740ce330 0x7f0974153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097415c9a0 0x7f0974153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097415cf00 con 0x7f097415c9a0 2024-04-25T10:11:57.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097415cd80 con 0x7f09740b1f70 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f097ac08640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097415d950 con 0x7f09740ce330 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f09791a5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097415c9a0 0x7f0974153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f09791a5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097415c9a0 0x7f0974153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33626/0 (socket says 172.21.15.133:33626) 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f09791a5640 1 -- 172.21.15.133:0/3189873609 learned_addr learned my addr 172.21.15.133:0/3189873609 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f09789a4640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09740b1f70 0x7f09740cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.773+0000 7f0973fff640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09740ce330 0x7f0974153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09791a5640 1 -- 172.21.15.133:0/3189873609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09740ce330 msgr2=0x7f0974153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09791a5640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09740ce330 0x7f0974153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09791a5640 1 -- 172.21.15.133:0/3189873609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09740b1f70 msgr2=0x7f09740cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09791a5640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09740b1f70 0x7f09740cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09791a5640 1 -- 172.21.15.133:0/3189873609 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09740cdbd0 con 0x7f097415c9a0 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09791a5640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097415c9a0 0x7f0974153980 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f096c00cef0 tx=0x7f096c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09717fa640 1 -- 172.21.15.133:0/3189873609 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c01a070 con 0x7f097415c9a0 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09717fa640 1 -- 172.21.15.133:0/3189873609 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c00d840 con 0x7f097415c9a0 2024-04-25T10:11:57.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09717fa640 1 -- 172.21.15.133:0/3189873609 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c00dc30 con 0x7f097415c9a0 2024-04-25T10:11:57.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f097ac08640 1 -- 172.21.15.133:0/3189873609 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09740ccb30 con 0x7f097415c9a0 2024-04-25T10:11:57.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f097ac08640 1 -- 172.21.15.133:0/3189873609 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09740cd0f0 con 0x7f097415c9a0 2024-04-25T10:11:57.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f097ac08640 1 -- 172.21.15.133:0/3189873609 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0974068ef0 con 0x7f097415c9a0 2024-04-25T10:11:57.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09717fa640 1 -- 172.21.15.133:0/3189873609 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f096c01d070 con 0x7f097415c9a0 2024-04-25T10:11:57.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09717fa640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0960066f60 0x7f0960069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09717fa640 1 -- 172.21.15.133:0/3189873609 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f096c085a20 con 0x7f097415c9a0 2024-04-25T10:11:57.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09789a4640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0960066f60 0x7f0960069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.777+0000 7f09789a4640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0960066f60 0x7f0960069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0968006fd0 tx=0x7f0968008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.781+0000 7f09717fa640 1 -- 172.21.15.133:0/3189873609 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f096c056870 con 0x7f097415c9a0 2024-04-25T10:11:57.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 -- 172.21.15.133:0/3189873609 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f097415e2c0 con 0x7f097415c9a0 2024-04-25T10:11:57.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f09717fa640 1 -- 172.21.15.133:0/3189873609 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f096c085070 con 0x7f097415c9a0 2024-04-25T10:11:57.979 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:57.979 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:57.979 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:57.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 -- 172.21.15.133:0/3189873609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0960066f60 msgr2=0x7f0960069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0960066f60 0x7f0960069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0968006fd0 tx=0x7f0968008040 comp rx=0 tx=0).stop 2024-04-25T10:11:57.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 -- 172.21.15.133:0/3189873609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097415c9a0 msgr2=0x7f0974153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097415c9a0 0x7f0974153980 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f096c00cef0 tx=0x7f096c00a710 comp rx=0 tx=0).stop 2024-04-25T10:11:57.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 -- 172.21.15.133:0/3189873609 shutdown_connections 2024-04-25T10:11:57.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0960066f60 0x7f0960069420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097415c9a0 0x7f0974153980 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09740ce330 0x7f0974153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 --2- 172.21.15.133:0/3189873609 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09740b1f70 0x7f09740cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 -- 172.21.15.133:0/3189873609 >> 172.21.15.133:0/3189873609 conn(0x7f0974067000 msgr2=0x7f097415afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 -- 172.21.15.133:0/3189873609 shutdown_connections 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:57.977+0000 7f097ac08640 1 -- 172.21.15.133:0/3189873609 wait complete. 2024-04-25T10:11:57.998 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:11:57.998 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:58.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f088d858640 1 Processor -- start 2024-04-25T10:11:58.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f088d858640 1 -- start start 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f088d858640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f088815c9a0 0x7f08880c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f088d858640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08880b1f70 0x7f088806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f088d858640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08880c61d0 con 0x7f08880ce330 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f088d858640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880c64b0 con 0x7f088815c9a0 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f088d858640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088806d4f0 con 0x7f08880b1f70 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f0886ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f088815c9a0 0x7f08880c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f0886ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f088815c9a0 0x7f08880c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33634/0 (socket says 172.21.15.133:33634) 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f0886ffd640 1 -- 172.21.15.133:0/1361189462 learned_addr learned my addr 172.21.15.133:0/1361189462 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f08867fc640 1 --2- 172.21.15.133:0/1361189462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08880b1f70 0x7f088806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f0886ffd640 1 -- 172.21.15.133:0/1361189462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08880b1f70 msgr2=0x7f088806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f0886ffd640 1 --2- 172.21.15.133:0/1361189462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08880b1f70 0x7f088806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f0886ffd640 1 -- 172.21.15.133:0/1361189462 >> v1:172.21.15.162:6789/0 conn(0x7f08880ce330 legacy=0x7f088815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:58.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f0886ffd640 1 -- 172.21.15.133:0/1361189462 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088806d7d0 con 0x7f088815c9a0 2024-04-25T10:11:58.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.105+0000 7f08867fc640 1 --2- 172.21.15.133:0/1361189462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08880b1f70 0x7f088806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:58.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f0886ffd640 1 --2- 172.21.15.133:0/1361189462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f088815c9a0 0x7f08880c5980 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f0878009870 tx=0x7f087801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5e4c246a3c8407 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f0885ffb640 1 -- 172.21.15.133:0/1361189462 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08780b2070 con 0x7f088815c9a0 2024-04-25T10:11:58.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f0885ffb640 1 -- 172.21.15.133:0/1361189462 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08780ad640 con 0x7f088815c9a0 2024-04-25T10:11:58.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f0885ffb640 1 -- 172.21.15.133:0/1361189462 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08780ada10 con 0x7f088815c9a0 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f088d858640 1 -- 172.21.15.133:0/1361189462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f088815c9a0 msgr2=0x7f08880c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f088d858640 1 --2- 172.21.15.133:0/1361189462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f088815c9a0 0x7f08880c5980 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f0878009870 tx=0x7f087801c4c0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f088d858640 1 -- 172.21.15.133:0/1361189462 shutdown_connections 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f088d858640 1 --2- 172.21.15.133:0/1361189462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08880b1f70 0x7f088806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f088d858640 1 --2- 172.21.15.133:0/1361189462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f088815c9a0 0x7f08880c5980 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f088d858640 1 -- 172.21.15.133:0/1361189462 >> 172.21.15.133:0/1361189462 conn(0x7f0888067000 msgr2=0x7f088806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f088d858640 1 -- 172.21.15.133:0/1361189462 shutdown_connections 2024-04-25T10:11:58.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f088d858640 1 -- 172.21.15.133:0/1361189462 wait complete. 2024-04-25T10:11:58.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f088d858640 1 Processor -- start 2024-04-25T10:11:58.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.109+0000 7f088d858640 1 -- start start 2024-04-25T10:11:58.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f088d858640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08880b1f70 0x7f0888157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f088d858640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08880ce330 0x7f08880c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f088d858640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f088815c9a0 0x7f08881f8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f088d858640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088815cd80 con 0x7f08880b1f70 2024-04-25T10:11:58.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f088d858640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088806e230 con 0x7f08880ce330 2024-04-25T10:11:58.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f088d858640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088815cf00 con 0x7f088815c9a0 2024-04-25T10:11:58.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f0886ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08880b1f70 0x7f0888157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f0886ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08880b1f70 0x7f0888157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33644/0 (socket says 172.21.15.133:33644) 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f0886ffd640 1 -- 172.21.15.133:0/1186968229 learned_addr learned my addr 172.21.15.133:0/1186968229 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f0886ffd640 1 -- 172.21.15.133:0/1186968229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f088815c9a0 msgr2=0x7f08881f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f0886ffd640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f088815c9a0 0x7f08881f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f08867fc640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08880ce330 0x7f08880c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f0886ffd640 1 -- 172.21.15.133:0/1186968229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08880ce330 msgr2=0x7f08880c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f0886ffd640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08880ce330 0x7f08880c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f0886ffd640 1 -- 172.21.15.133:0/1186968229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880cb460 con 0x7f08880b1f70 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f0886ffd640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08880b1f70 0x7f0888157d50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f08780b0040 tx=0x7f08780192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f086ffff640 1 -- 172.21.15.133:0/1186968229 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08780bd070 con 0x7f08880b1f70 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f086ffff640 1 -- 172.21.15.133:0/1186968229 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08780b8650 con 0x7f08880b1f70 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f088d858640 1 -- 172.21.15.133:0/1186968229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08880cb740 con 0x7f08880b1f70 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f086ffff640 1 -- 172.21.15.133:0/1186968229 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08780b2040 con 0x7f08880b1f70 2024-04-25T10:11:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f088d858640 1 -- 172.21.15.133:0/1186968229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08880cbd00 con 0x7f08880b1f70 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f088d858640 1 -- 172.21.15.133:0/1186968229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088806bc20 con 0x7f08880b1f70 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f086ffff640 1 -- 172.21.15.133:0/1186968229 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08780c0070 con 0x7f08880b1f70 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f086ffff640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0874066f60 0x7f0874069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:58.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f08867fc640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0874066f60 0x7f0874069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:58.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f086ffff640 1 -- 172.21.15.133:0/1186968229 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f087811ede0 con 0x7f08880b1f70 2024-04-25T10:11:58.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.113+0000 7f08867fc640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0874066f60 0x7f0874069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f087c006fd0 tx=0x7f087c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:58.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.117+0000 7f086ffff640 1 -- 172.21.15.133:0/1186968229 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0878016110 con 0x7f08880b1f70 2024-04-25T10:11:58.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.325+0000 7f088d858640 1 -- 172.21.15.133:0/1186968229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0888150490 con 0x7f08880b1f70 2024-04-25T10:11:58.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.325+0000 7f086ffff640 1 -- 172.21.15.133:0/1186968229 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1633 (secure 0 0 0) 0x7f08780efd80 con 0x7f08880b1f70 2024-04-25T10:11:58.329 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:58.329 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":631,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078432287,"bytes_used":2674728960,"bytes_avail":770419384320,"bytes_total":773094113280,"write_bytes_sec":82997,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:11:55.890677+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:11:58.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 -- 172.21.15.133:0/1186968229 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0874066f60 msgr2=0x7f0874069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0874066f60 0x7f0874069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f087c006fd0 tx=0x7f087c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:58.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 -- 172.21.15.133:0/1186968229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08880b1f70 msgr2=0x7f0888157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:58.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08880b1f70 0x7f0888157d50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f08780b0040 tx=0x7f08780192c0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 -- 172.21.15.133:0/1186968229 shutdown_connections 2024-04-25T10:11:58.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f088815c9a0 0x7f08881f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08880ce330 0x7f08880c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0874066f60 0x7f0874069420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 --2- 172.21.15.133:0/1186968229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08880b1f70 0x7f0888157d50 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:58.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 -- 172.21.15.133:0/1186968229 >> 172.21.15.133:0/1186968229 conn(0x7f0888067000 msgr2=0x7f0888153700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:58.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 -- 172.21.15.133:0/1186968229 shutdown_connections 2024-04-25T10:11:58.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:58.329+0000 7f088d858640 1 -- 172.21.15.133:0/1186968229 wait complete. 2024-04-25T10:11:58.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:11:59.346 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:59.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fccab466640 1 Processor -- start 2024-04-25T10:11:59.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fccab466640 1 -- start start 2024-04-25T10:11:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fccab466640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcca4070270 0x7fcca40cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fccab466640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca40cf800 0x7fcca40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fccab466640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca40cc660 con 0x7fcca40cda30 2024-04-25T10:11:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fccab466640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca40af7a0 con 0x7fcca4070270 2024-04-25T10:11:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fccab466640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca40af4c0 con 0x7fcca40cf800 2024-04-25T10:11:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fcca9a03640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcca40cda30 0x7fcca40cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48196/0 (socket says 172.21.15.133:48196) 2024-04-25T10:11:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fcca9a03640 1 -- 172.21.15.133:0/4079012867 learned_addr learned my addr 172.21.15.133:0/4079012867 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:11:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fcca9202640 1 --2- 172.21.15.133:0/4079012867 >> v2:172.21.15.133:3300/0 conn(0x7fcca4070270 0x7fcca40cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fcca8a01640 1 --2- 172.21.15.133:0/4079012867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca40cf800 0x7fcca40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fcc97fff640 1 -- 172.21.15.133:0/4079012867 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693079962 0 0) 0x7fcca40cc660 con 0x7fcca40cda30 2024-04-25T10:11:59.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fcc97fff640 1 -- 172.21.15.133:0/4079012867 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc90003530 con 0x7fcca40cda30 2024-04-25T10:11:59.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fcca9202640 1 -- 172.21.15.133:0/4079012867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca40cf800 msgr2=0x7fcca40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:59.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fcca9202640 1 --2- 172.21.15.133:0/4079012867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca40cf800 0x7fcca40ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fcca9202640 1 -- 172.21.15.133:0/4079012867 >> v1:172.21.15.133:6789/0 conn(0x7fcca40cda30 legacy=0x7fcca40cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fcca9202640 1 -- 172.21.15.133:0/4079012867 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca40ce690 con 0x7fcca4070270 2024-04-25T10:11:59.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.477+0000 7fcca8a01640 1 --2- 172.21.15.133:0/4079012867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca40cf800 0x7fcca40ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:59.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fcca9202640 1 --2- 172.21.15.133:0/4079012867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca4070270 0x7fcca40cc120 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fcca00097c0 tx=0x7fcca001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf5c31017dd9bab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fcc97fff640 1 -- 172.21.15.133:0/4079012867 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca00b0070 con 0x7fcca4070270 2024-04-25T10:11:59.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fcc97fff640 1 -- 172.21.15.133:0/4079012867 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca00ab600 con 0x7fcca4070270 2024-04-25T10:11:59.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fcc97fff640 1 -- 172.21.15.133:0/4079012867 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca00ab9d0 con 0x7fcca4070270 2024-04-25T10:11:59.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fccab466640 1 -- 172.21.15.133:0/4079012867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca4070270 msgr2=0x7fcca40cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fccab466640 1 --2- 172.21.15.133:0/4079012867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca4070270 0x7fcca40cc120 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fcca00097c0 tx=0x7fcca001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fccab466640 1 -- 172.21.15.133:0/4079012867 shutdown_connections 2024-04-25T10:11:59.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fccab466640 1 --2- 172.21.15.133:0/4079012867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca40cf800 0x7fcca40ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fccab466640 1 --2- 172.21.15.133:0/4079012867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca4070270 0x7fcca40cc120 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fccab466640 1 -- 172.21.15.133:0/4079012867 >> 172.21.15.133:0/4079012867 conn(0x7fcca40cc8e0 msgr2=0x7fcca40cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fccab466640 1 -- 172.21.15.133:0/4079012867 shutdown_connections 2024-04-25T10:11:59.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fccab466640 1 -- 172.21.15.133:0/4079012867 wait complete. 2024-04-25T10:11:59.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fccab466640 1 Processor -- start 2024-04-25T10:11:59.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.481+0000 7fccab466640 1 -- start start 2024-04-25T10:11:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fccab466640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca4070270 0x7fcca4159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fccab466640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca40cda30 0x7fcca4162f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fccab466640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcca40cf800 0x7fcca409e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fccab466640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca406e3e0 con 0x7fcca40cda30 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fccab466640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca406e260 con 0x7fcca40cf800 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fccab466640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca406e560 con 0x7fcca4070270 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca8a01640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca40cda30 0x7fcca4162f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca8a01640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca40cda30 0x7fcca4162f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33664/0 (socket says 172.21.15.133:33664) 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca8a01640 1 -- 172.21.15.133:0/835828735 learned_addr learned my addr 172.21.15.133:0/835828735 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca9202640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca4070270 0x7fcca4159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca9a03640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcca40cf800 0x7fcca409e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca8a01640 1 -- 172.21.15.133:0/835828735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca4070270 msgr2=0x7fcca4159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca8a01640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca4070270 0x7fcca4159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca8a01640 1 -- 172.21.15.133:0/835828735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcca40cf800 msgr2=0x7fcca409e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca8a01640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcca40cf800 0x7fcca409e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca8a01640 1 -- 172.21.15.133:0/835828735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca40a1030 con 0x7fcca40cda30 2024-04-25T10:11:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca9202640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca4070270 0x7fcca4159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca8a01640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca40cda30 0x7fcca4162f30 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c007f20 tx=0x7fcc9c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcc95ffb640 1 -- 172.21.15.133:0/835828735 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c014620 con 0x7fcca40cda30 2024-04-25T10:11:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcc95ffb640 1 -- 172.21.15.133:0/835828735 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c019070 con 0x7fcca40cda30 2024-04-25T10:11:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fccab466640 1 -- 172.21.15.133:0/835828735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca415a6d0 con 0x7fcca40cda30 2024-04-25T10:11:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcc95ffb640 1 -- 172.21.15.133:0/835828735 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c014a00 con 0x7fcca40cda30 2024-04-25T10:11:59.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fccab466640 1 -- 172.21.15.133:0/835828735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca415ac90 con 0x7fcca40cda30 2024-04-25T10:11:59.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fccab466640 1 -- 172.21.15.133:0/835828735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca40a1200 con 0x7fcca40cda30 2024-04-25T10:11:59.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcc95ffb640 1 -- 172.21.15.133:0/835828735 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc9c014ba0 con 0x7fcca40cda30 2024-04-25T10:11:59.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcc95ffb640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc88066f60 0x7fcc88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcc95ffb640 1 -- 172.21.15.133:0/835828735 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcc9c085b50 con 0x7fcca40cda30 2024-04-25T10:11:59.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca9202640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc88066f60 0x7fcc88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.485+0000 7fcca9202640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc88066f60 0x7fcc88069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcca00ad000 tx=0x7fcca0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.489+0000 7fcc95ffb640 1 -- 172.21.15.133:0/835828735 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc9c0569a0 con 0x7fcca40cda30 2024-04-25T10:11:59.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.673+0000 7fccab466640 1 -- 172.21.15.133:0/835828735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcca4067c70 con 0x7fcca40cda30 2024-04-25T10:11:59.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fcc95ffb640 1 -- 172.21.15.133:0/835828735 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fcc9c0562a0 con 0x7fcca40cda30 2024-04-25T10:11:59.679 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:11:59.679 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:59.679 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:11:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 -- 172.21.15.133:0/835828735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc88066f60 msgr2=0x7fcc88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc88066f60 0x7fcc88069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcca00ad000 tx=0x7fcca0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 -- 172.21.15.133:0/835828735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca40cda30 msgr2=0x7fcca4162f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca40cda30 0x7fcca4162f30 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c007f20 tx=0x7fcc9c00e570 comp rx=0 tx=0).stop 2024-04-25T10:11:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 -- 172.21.15.133:0/835828735 shutdown_connections 2024-04-25T10:11:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcca40cf800 0x7fcca409e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc88066f60 0x7fcc88069420 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca40cda30 0x7fcca4162f30 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 --2- 172.21.15.133:0/835828735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcca4070270 0x7fcca4159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 -- 172.21.15.133:0/835828735 >> 172.21.15.133:0/835828735 conn(0x7fcca40cc8e0 msgr2=0x7fcca406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 -- 172.21.15.133:0/835828735 shutdown_connections 2024-04-25T10:11:59.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.677+0000 7fccab466640 1 -- 172.21.15.133:0/835828735 wait complete. 2024-04-25T10:11:59.696 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:11:59.696 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:59.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.809+0000 7fface727640 1 Processor -- start 2024-04-25T10:11:59.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.809+0000 7fface727640 1 -- start start 2024-04-25T10:11:59.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.809+0000 7fface727640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffac8152750 0x7ffac80ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.809+0000 7fface727640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffac80ce870 0x7ffac80b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.809+0000 7fface727640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac806d350 con 0x7ffac8152b30 2024-04-25T10:11:59.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.809+0000 7fface727640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac815bf90 con 0x7ffac80ce870 2024-04-25T10:11:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.809+0000 7fface727640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac806d5d0 con 0x7ffac8152750 2024-04-25T10:11:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.809+0000 7ffacccc4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffac80ce870 0x7ffac80b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7ffacccc4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffac80ce870 0x7ffac80b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33680/0 (socket says 172.21.15.133:33680) 2024-04-25T10:11:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7ffacccc4640 1 -- 172.21.15.133:0/3146502869 learned_addr learned my addr 172.21.15.133:0/3146502869 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7ffac7fff640 1 --2- 172.21.15.133:0/3146502869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffac8152750 0x7ffac80ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7ffacccc4640 1 -- 172.21.15.133:0/3146502869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffac8152750 msgr2=0x7ffac80ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7ffacccc4640 1 --2- 172.21.15.133:0/3146502869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffac8152750 0x7ffac80ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7ffacccc4640 1 -- 172.21.15.133:0/3146502869 >> v1:172.21.15.162:6789/0 conn(0x7ffac8152b30 legacy=0x7ffac80b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7ffacccc4640 1 -- 172.21.15.133:0/3146502869 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac815c230 con 0x7ffac80ce870 2024-04-25T10:11:59.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7ffacccc4640 1 --2- 172.21.15.133:0/3146502869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac80ce870 0x7ffac80b1ec0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009e50 tx=0x7ffac001c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef2a5419cb8005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7ffac6ffd640 1 -- 172.21.15.133:0/3146502869 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00ac020 con 0x7ffac80ce870 2024-04-25T10:11:59.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7ffac6ffd640 1 -- 172.21.15.133:0/3146502869 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac00a6480 con 0x7ffac80ce870 2024-04-25T10:11:59.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7ffac6ffd640 1 -- 172.21.15.133:0/3146502869 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00a6850 con 0x7ffac80ce870 2024-04-25T10:11:59.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7fface727640 1 -- 172.21.15.133:0/3146502869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac80ce870 msgr2=0x7ffac80b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7fface727640 1 --2- 172.21.15.133:0/3146502869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac80ce870 0x7ffac80b1ec0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009e50 tx=0x7ffac001c7f0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7fface727640 1 -- 172.21.15.133:0/3146502869 shutdown_connections 2024-04-25T10:11:59.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7fface727640 1 --2- 172.21.15.133:0/3146502869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac80ce870 0x7ffac80b1ec0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7fface727640 1 --2- 172.21.15.133:0/3146502869 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffac8152750 0x7ffac80ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7fface727640 1 -- 172.21.15.133:0/3146502869 >> 172.21.15.133:0/3146502869 conn(0x7ffac8067000 msgr2=0x7ffac806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7fface727640 1 -- 172.21.15.133:0/3146502869 shutdown_connections 2024-04-25T10:11:59.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.813+0000 7fface727640 1 -- 172.21.15.133:0/3146502869 wait complete. 2024-04-25T10:11:59.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7fface727640 1 Processor -- start 2024-04-25T10:11:59.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7fface727640 1 -- start start 2024-04-25T10:11:59.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7fface727640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac80ce870 0x7ffac8149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7fface727640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffac8152750 0x7ffac81499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7fface727640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffac8152b30 0x7ffac8149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7fface727640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8160080 con 0x7ffac80ce870 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7fface727640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac815ff00 con 0x7ffac8152b30 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7fface727640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8160200 con 0x7ffac8152750 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac80ce870 0x7ffac8149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac80ce870 0x7ffac8149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33696/0 (socket says 172.21.15.133:33696) 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac7fff640 1 -- 172.21.15.133:0/3879821957 learned_addr learned my addr 172.21.15.133:0/3879821957 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac77fe640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffac8152750 0x7ffac81499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffacccc4640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffac8152b30 0x7ffac8149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac7fff640 1 -- 172.21.15.133:0/3879821957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffac8152750 msgr2=0x7ffac81499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac7fff640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffac8152750 0x7ffac81499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac7fff640 1 -- 172.21.15.133:0/3879821957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffac8152b30 msgr2=0x7ffac8149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac7fff640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffac8152b30 0x7ffac8149f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac7fff640 1 -- 172.21.15.133:0/3879821957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac814b220 con 0x7ffac80ce870 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac7fff640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac80ce870 0x7ffac8149470 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ffab800d5f0 tx=0x7ffab800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac4ff9640 1 -- 172.21.15.133:0/3879821957 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab80125f0 con 0x7ffac80ce870 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac4ff9640 1 -- 172.21.15.133:0/3879821957 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab8019070 con 0x7ffac80ce870 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7fface727640 1 -- 172.21.15.133:0/3879821957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac80c9210 con 0x7ffac80ce870 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac4ff9640 1 -- 172.21.15.133:0/3879821957 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab80129b0 con 0x7ffac80ce870 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7fface727640 1 -- 172.21.15.133:0/3879821957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac80c9ab0 con 0x7ffac80ce870 2024-04-25T10:11:59.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7fface727640 1 -- 172.21.15.133:0/3879821957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac8068ef0 con 0x7ffac80ce870 2024-04-25T10:11:59.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.817+0000 7ffac4ff9640 1 -- 172.21.15.133:0/3879821957 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffab8012b50 con 0x7ffac80ce870 2024-04-25T10:11:59.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.821+0000 7ffac4ff9640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffaa8066f60 0x7ffaa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.821+0000 7ffac4ff9640 1 -- 172.21.15.133:0/3879821957 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ffab80856a0 con 0x7ffac80ce870 2024-04-25T10:11:59.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.821+0000 7ffac77fe640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffaa8066f60 0x7ffaa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.821+0000 7ffac77fe640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffaa8066f60 0x7ffaa8069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffac815ee20 tx=0x7ffab0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:11:59.825+0000 7ffac4ff9640 1 -- 172.21.15.133:0/3879821957 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffab8056460 con 0x7ffac80ce870 2024-04-25T10:12:00.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.037+0000 7fface727640 1 -- 172.21.15.133:0/3879821957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ffac80c5120 con 0x7ffac80ce870 2024-04-25T10:12:00.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.037+0000 7ffac4ff9640 1 -- 172.21.15.133:0/3879821957 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1633 (secure 0 0 0) 0x7ffab8055d60 con 0x7ffac80ce870 2024-04-25T10:12:00.041 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:00.042 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":632,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078432287,"bytes_used":2674728960,"bytes_avail":770419384320,"bytes_total":773094113280,"write_bytes_sec":82997,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:11:55.890677+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 -- 172.21.15.133:0/3879821957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffaa8066f60 msgr2=0x7ffaa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffaa8066f60 0x7ffaa8069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffac815ee20 tx=0x7ffab0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 -- 172.21.15.133:0/3879821957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac80ce870 msgr2=0x7ffac8149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac80ce870 0x7ffac8149470 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ffab800d5f0 tx=0x7ffab800dab0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 -- 172.21.15.133:0/3879821957 shutdown_connections 2024-04-25T10:12:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffac8152b30 0x7ffac8149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffac8152750 0x7ffac81499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffaa8066f60 0x7ffaa8069420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 --2- 172.21.15.133:0/3879821957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac80ce870 0x7ffac8149470 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 -- 172.21.15.133:0/3879821957 >> 172.21.15.133:0/3879821957 conn(0x7ffac8067000 msgr2=0x7ffac80b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 -- 172.21.15.133:0/3879821957 shutdown_connections 2024-04-25T10:12:00.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:00.041+0000 7fface727640 1 -- 172.21.15.133:0/3879821957 wait complete. 2024-04-25T10:12:00.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:12:01.059 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe107a69640 1 Processor -- start 2024-04-25T10:12:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe107a69640 1 -- start start 2024-04-25T10:12:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe107a69640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10015c9a0 0x7fe1000c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe107a69640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1000c61d0 con 0x7fe1000b1f70 2024-04-25T10:12:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe107a69640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1000c64b0 con 0x7fe1000ce330 2024-04-25T10:12:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe107a69640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10006d4f0 con 0x7fe10015c9a0 2024-04-25T10:12:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe105004640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe1000b1f70 0x7fe10006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48206/0 (socket says 172.21.15.133:48206) 2024-04-25T10:12:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe105004640 1 -- 172.21.15.133:0/2501001331 learned_addr learned my addr 172.21.15.133:0/2501001331 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe105805640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10015c9a0 0x7fe1000c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481217488 0 0) 0x7fe1000c61d0 con 0x7fe1000b1f70 2024-04-25T10:12:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0ec003530 con 0x7fe1000b1f70 2024-04-25T10:12:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4039515146 0 0) 0x7fe0ec003530 con 0x7fe1000b1f70 2024-04-25T10:12:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10015c9a0 msgr2=0x7fe1000c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10015c9a0 0x7fe1000c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 >> v1:172.21.15.162:6789/0 conn(0x7fe1000ce330 legacy=0x7fe10015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10006d7d0 con 0x7fe1000b1f70 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe0f0001680 con 0x7fe1000b1f70 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 >> v1:172.21.15.133:6789/0 conn(0x7fe1000b1f70 legacy=0x7fe10006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 0x7fe0ec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10015c9a0 0x7fe0ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0ec004d50 0x7fe0ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10006d4f0 con 0x7fe0ec0039f0 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec00ba80 con 0x7fe0ec004d50 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec00bc80 con 0x7fe10015c9a0 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe105805640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 0x7fe0ec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe105004640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10015c9a0 0x7fe0ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe105805640 1 -- 172.21.15.133:0/2501001331 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10015c9a0 msgr2=0x7fe0ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe105805640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe10015c9a0 0x7fe0ec004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe105805640 1 -- 172.21.15.133:0/2501001331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0ec004d50 msgr2=0x7fe0ec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe106006640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0ec004d50 0x7fe0ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe105805640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0ec004d50 0x7fe0ec0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.185+0000 7fe105805640 1 -- 172.21.15.133:0/2501001331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec003530 con 0x7fe0ec0039f0 2024-04-25T10:12:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe105805640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 0x7fe0ec003e10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009ab0 tx=0x7fe0fc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255a3878f6fae797 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0b2070 con 0x7fe0ec0039f0 2024-04-25T10:12:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc0a5530 con 0x7fe0ec0039f0 2024-04-25T10:12:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe104803640 1 -- 172.21.15.133:0/2501001331 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0096b0 con 0x7fe0ec0039f0 2024-04-25T10:12:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe107a69640 1 -- 172.21.15.133:0/2501001331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 msgr2=0x7fe0ec003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe107a69640 1 --2- 172.21.15.133:0/2501001331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 0x7fe0ec003e10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009ab0 tx=0x7fe0fc01ca50 comp rx=0 tx=0).stop 2024-04-25T10:12:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe106006640 1 -- 172.21.15.133:0/2501001331 reap_dead start 2024-04-25T10:12:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe107a69640 1 -- 172.21.15.133:0/2501001331 shutdown_connections 2024-04-25T10:12:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe107a69640 1 -- 172.21.15.133:0/2501001331 >> 172.21.15.133:0/2501001331 conn(0x7fe100067000 msgr2=0x7fe10006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe107a69640 1 -- 172.21.15.133:0/2501001331 shutdown_connections 2024-04-25T10:12:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe107a69640 1 -- 172.21.15.133:0/2501001331 wait complete. 2024-04-25T10:12:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe107a69640 1 Processor -- start 2024-04-25T10:12:01.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.189+0000 7fe107a69640 1 -- start start 2024-04-25T10:12:01.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe107a69640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 0x7fe1000c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe107a69640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe100158840 0x7fe1001506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe107a69640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1000c9660 0x7fe1000ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe107a69640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000c5980 con 0x7fe0ec0039f0 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe107a69640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000c5b00 con 0x7fe1000c9660 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe107a69640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000c5c80 con 0x7fe100158840 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe105805640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 0x7fe1000c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe105805640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 0x7fe1000c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33726/0 (socket says 172.21.15.133:33726) 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe105805640 1 -- 172.21.15.133:0/1047588410 learned_addr learned my addr 172.21.15.133:0/1047588410 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe105805640 1 -- 172.21.15.133:0/1047588410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe100158840 msgr2=0x7fe1001506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe106006640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1000c9660 0x7fe1000ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe105805640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe100158840 0x7fe1001506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe105805640 1 -- 172.21.15.133:0/1047588410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1000c9660 msgr2=0x7fe1000ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe105805640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1000c9660 0x7fe1000ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe105805640 1 -- 172.21.15.133:0/1047588410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1000c85c0 con 0x7fe0ec0039f0 2024-04-25T10:12:01.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe105805640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 0x7fe1000c7ce0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009ab0 tx=0x7fe0ec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe0f67fc640 1 -- 172.21.15.133:0/1047588410 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0b2040 con 0x7fe0ec0039f0 2024-04-25T10:12:01.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe107a69640 1 -- 172.21.15.133:0/1047588410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1000c88a0 con 0x7fe0ec0039f0 2024-04-25T10:12:01.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe0f67fc640 1 -- 172.21.15.133:0/1047588410 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc00f000 con 0x7fe0ec0039f0 2024-04-25T10:12:01.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe0f67fc640 1 -- 172.21.15.133:0/1047588410 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc004020 con 0x7fe0ec0039f0 2024-04-25T10:12:01.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe107a69640 1 -- 172.21.15.133:0/1047588410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1000c8e60 con 0x7fe0ec0039f0 2024-04-25T10:12:01.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe107a69640 1 -- 172.21.15.133:0/1047588410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe100068ef0 con 0x7fe0ec0039f0 2024-04-25T10:12:01.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe0f67fc640 1 -- 172.21.15.133:0/1047588410 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0fc0c0020 con 0x7fe0ec0039f0 2024-04-25T10:12:01.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe0f67fc640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe0e4066f90 0x7fe0e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe0f67fc640 1 -- 172.21.15.133:0/1047588410 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe0fc11f6b0 con 0x7fe0ec0039f0 2024-04-25T10:12:01.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe105004640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe0e4066f90 0x7fe0e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.193+0000 7fe105004640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe0e4066f90 0x7fe0e4069450 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0001210 tx=0x7fe0f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.197+0000 7fe0f67fc640 1 -- 172.21.15.133:0/1047588410 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0fc0c7020 con 0x7fe0ec0039f0 2024-04-25T10:12:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.377+0000 7fe107a69640 1 -- 172.21.15.133:0/1047588410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1000b44a0 con 0x7fe0ec0039f0 2024-04-25T10:12:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.377+0000 7fe0f67fc640 1 -- 172.21.15.133:0/1047588410 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe0fc016760 con 0x7fe0ec0039f0 2024-04-25T10:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:12:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 -- 172.21.15.133:0/1047588410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe0e4066f90 msgr2=0x7fe0e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe0e4066f90 0x7fe0e4069450 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0001210 tx=0x7fe0f0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 -- 172.21.15.133:0/1047588410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 msgr2=0x7fe1000c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 0x7fe1000c7ce0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009ab0 tx=0x7fe0ec005840 comp rx=0 tx=0).stop 2024-04-25T10:12:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 -- 172.21.15.133:0/1047588410 shutdown_connections 2024-04-25T10:12:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe1000c9660 0x7fe1000ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe100158840 0x7fe1001506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe0e4066f90 0x7fe0e4069450 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 --2- 172.21.15.133:0/1047588410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0ec0039f0 0x7fe1000c7ce0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 -- 172.21.15.133:0/1047588410 >> 172.21.15.133:0/1047588410 conn(0x7fe100067000 msgr2=0x7fe10015f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 -- 172.21.15.133:0/1047588410 shutdown_connections 2024-04-25T10:12:01.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.381+0000 7fe107a69640 1 -- 172.21.15.133:0/1047588410 wait complete. 2024-04-25T10:12:01.399 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:12:01.399 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:01.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b8cf1640 1 Processor -- start 2024-04-25T10:12:01.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b8cf1640 1 -- start start 2024-04-25T10:12:01.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b8cf1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe0b40d0bd0 0x7fe0b40b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b8cf1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b40ceb20 con 0x7fe0b415db00 2024-04-25T10:12:01.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b8cf1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b40b30a0 con 0x7fe0b40d0770 2024-04-25T10:12:01.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b8cf1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b40b7740 con 0x7fe0b40d0bd0 2024-04-25T10:12:01.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b259c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe0b415db00 0x7fe0b40b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48210/0 (socket says 172.21.15.133:48210) 2024-04-25T10:12:01.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b259c640 1 -- 172.21.15.133:0/2384619803 learned_addr learned my addr 172.21.15.133:0/2384619803 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:01.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b2d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe0b40d0bd0 0x7fe0b40b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b2d9d640 1 -- 172.21.15.133:0/2384619803 >> v1:172.21.15.162:6789/0 conn(0x7fe0b40d0770 legacy=0x7fe0b40b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:01.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b2d9d640 1 -- 172.21.15.133:0/2384619803 >> v1:172.21.15.133:6789/0 conn(0x7fe0b415db00 legacy=0x7fe0b40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.513+0000 7fe0b2d9d640 1 -- 172.21.15.133:0/2384619803 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b40b7a20 con 0x7fe0b40d0bd0 2024-04-25T10:12:01.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b2d9d640 1 --2- 172.21.15.133:0/2384619803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0b40d0bd0 0x7fe0b40b8ef0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009d90 tx=0x7fe0ac01c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0625f50f3391498 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b159a640 1 -- 172.21.15.133:0/2384619803 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0a6700 con 0x7fe0b40d0bd0 2024-04-25T10:12:01.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b159a640 1 -- 172.21.15.133:0/2384619803 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac0a68a0 con 0x7fe0b40d0bd0 2024-04-25T10:12:01.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b159a640 1 -- 172.21.15.133:0/2384619803 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0a6c70 con 0x7fe0b40d0bd0 2024-04-25T10:12:01.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/2384619803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0b40d0bd0 msgr2=0x7fe0b40b8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b8cf1640 1 --2- 172.21.15.133:0/2384619803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0b40d0bd0 0x7fe0b40b8ef0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009d90 tx=0x7fe0ac01c720 comp rx=0 tx=0).stop 2024-04-25T10:12:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/2384619803 shutdown_connections 2024-04-25T10:12:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b8cf1640 1 --2- 172.21.15.133:0/2384619803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0b40d0bd0 0x7fe0b40b8ef0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/2384619803 >> 172.21.15.133:0/2384619803 conn(0x7fe0b40727c0 msgr2=0x7fe0b4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/2384619803 shutdown_connections 2024-04-25T10:12:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/2384619803 wait complete. 2024-04-25T10:12:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b8cf1640 1 Processor -- start 2024-04-25T10:12:01.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.517+0000 7fe0b8cf1640 1 -- start start 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b8cf1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0b40d0770 0x7fe0b4152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b8cf1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0b40d0bd0 0x7fe0b4159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b8cf1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0b415db00 0x7fe0b415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b8cf1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b40b7430 con 0x7fe0b40d0770 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b8cf1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b40d1060 con 0x7fe0b415db00 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b8cf1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b40d18a0 con 0x7fe0b40d0bd0 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0b40d0770 0x7fe0b4152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0b40d0770 0x7fe0b4152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33750/0 (socket says 172.21.15.133:33750) 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b259c640 1 -- 172.21.15.133:0/3717621986 learned_addr learned my addr 172.21.15.133:0/3717621986 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b2d9d640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0b415db00 0x7fe0b415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b1d9b640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0b40d0bd0 0x7fe0b4159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b259c640 1 -- 172.21.15.133:0/3717621986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0b40d0bd0 msgr2=0x7fe0b4159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b259c640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0b40d0bd0 0x7fe0b4159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b259c640 1 -- 172.21.15.133:0/3717621986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0b415db00 msgr2=0x7fe0b415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b259c640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0b415db00 0x7fe0b415a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b259c640 1 -- 172.21.15.133:0/3717621986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b415f5d0 con 0x7fe0b40d0770 2024-04-25T10:12:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b259c640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0b40d0770 0x7fe0b4152040 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400c3a0 tx=0x7fe0a400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0a2ffd640 1 -- 172.21.15.133:0/3717621986 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4017070 con 0x7fe0b40d0770 2024-04-25T10:12:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0a2ffd640 1 -- 172.21.15.133:0/3717621986 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a4005cf0 con 0x7fe0b40d0770 2024-04-25T10:12:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/3717621986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b415f8b0 con 0x7fe0b40d0770 2024-04-25T10:12:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0a2ffd640 1 -- 172.21.15.133:0/3717621986 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a40072f0 con 0x7fe0b40d0770 2024-04-25T10:12:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/3717621986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b415fe70 con 0x7fe0b40d0770 2024-04-25T10:12:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/3717621986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b4068ef0 con 0x7fe0b40d0770 2024-04-25T10:12:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0a2ffd640 1 -- 172.21.15.133:0/3717621986 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0a401a070 con 0x7fe0b40d0770 2024-04-25T10:12:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0a2ffd640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe094066f60 0x7fe094069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0a2ffd640 1 -- 172.21.15.133:0/3717621986 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe0a4084500 con 0x7fe0b40d0770 2024-04-25T10:12:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b1d9b640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe094066f60 0x7fe094069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.521+0000 7fe0b1d9b640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe094066f60 0x7fe094069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40b92e0 tx=0x7fe0a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:01.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.525+0000 7fe0a2ffd640 1 -- 172.21.15.133:0/3717621986 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0a40553e0 con 0x7fe0b40d0770 2024-04-25T10:12:01.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.733+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/3717621986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe0b40b2750 con 0x7fe0b40d0770 2024-04-25T10:12:01.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.733+0000 7fe0a2ffd640 1 -- 172.21.15.133:0/3717621986 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1571 (secure 0 0 0) 0x7fe0a4054ce0 con 0x7fe0b40d0770 2024-04-25T10:12:01.737 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:01.737 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":634,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078432287,"bytes_used":2674728960,"bytes_avail":770419384320,"bytes_total":773094113280,"write_bytes_sec":83001,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":65,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8313}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:11:59.892084+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/3717621986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe094066f60 msgr2=0x7fe094069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe094066f60 0x7fe094069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40b92e0 tx=0x7fe0a8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/3717621986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0b40d0770 msgr2=0x7fe0b4152040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0b40d0770 0x7fe0b4152040 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400c3a0 tx=0x7fe0a400c860 comp rx=0 tx=0).stop 2024-04-25T10:12:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/3717621986 shutdown_connections 2024-04-25T10:12:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0b415db00 0x7fe0b415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe0b40d0bd0 0x7fe0b4159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe094066f60 0x7fe094069420 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 --2- 172.21.15.133:0/3717621986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0b40d0770 0x7fe0b4152040 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:01.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/3717621986 >> 172.21.15.133:0/3717621986 conn(0x7fe0b40727c0 msgr2=0x7fe0b40b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:01.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/3717621986 shutdown_connections 2024-04-25T10:12:01.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:01.737+0000 7fe0b8cf1640 1 -- 172.21.15.133:0/3717621986 wait complete. 2024-04-25T10:12:01.754 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:12:01.754 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:12:01.755 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:01.795 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:01.847 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:12:01.847 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:12:01.847 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:01.853 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:02.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.009+0000 7fa35fbb3640 1 Processor -- start 2024-04-25T10:12:02.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.009+0000 7fa35fbb3640 1 -- start start 2024-04-25T10:12:02.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35fbb3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa358158110 0x7fa3580b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35fbb3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa35806eb70 0x7fa35815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35fbb3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3580cbe60 con 0x7fa3581584f0 2024-04-25T10:12:02.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35fbb3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3580b19b0 con 0x7fa35806eb70 2024-04-25T10:12:02.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35fbb3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35815b360 con 0x7fa358158110 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35d94f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa358158110 0x7fa3580b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35e150640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa35806eb70 0x7fa35815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35d14e640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa3581584f0 0x7fa35806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:56560/0 (socket says 172.21.15.133:56560) 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35d14e640 1 -- 172.21.15.133:0/273425112 learned_addr learned my addr 172.21.15.133:0/273425112 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:02.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35c94d640 1 -- 172.21.15.133:0/273425112 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243461182 0 0) 0x7fa3580cbe60 con 0x7fa3581584f0 2024-04-25T10:12:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35d94f640 1 -- 172.21.15.133:0/273425112 >> v2:172.21.15.162:3300/0 conn(0x7fa35806eb70 msgr2=0x7fa35815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35d94f640 1 --2- 172.21.15.133:0/273425112 >> v2:172.21.15.162:3300/0 conn(0x7fa35806eb70 0x7fa35815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35d94f640 1 -- 172.21.15.133:0/273425112 >> v1:172.21.15.162:6789/0 conn(0x7fa3581584f0 legacy=0x7fa35806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.013+0000 7fa35d94f640 1 -- 172.21.15.133:0/273425112 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35815b640 con 0x7fa358158110 2024-04-25T10:12:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35d94f640 1 --2- 172.21.15.133:0/273425112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa358158110 0x7fa3580b1f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa3540097c0 tx=0x7fa35401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e356fde73b38f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35c94d640 1 -- 172.21.15.133:0/273425112 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3540aa5f0 con 0x7fa358158110 2024-04-25T10:12:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35c94d640 1 -- 172.21.15.133:0/273425112 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3540af070 con 0x7fa358158110 2024-04-25T10:12:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35c94d640 1 -- 172.21.15.133:0/273425112 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3540aa9b0 con 0x7fa358158110 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 -- 172.21.15.133:0/273425112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa358158110 msgr2=0x7fa3580b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 --2- 172.21.15.133:0/273425112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa358158110 0x7fa3580b1f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa3540097c0 tx=0x7fa35401c470 comp rx=0 tx=0).stop 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 -- 172.21.15.133:0/273425112 shutdown_connections 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 --2- 172.21.15.133:0/273425112 >> v2:172.21.15.162:3300/0 conn(0x7fa35806eb70 0x7fa35815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 --2- 172.21.15.133:0/273425112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa358158110 0x7fa3580b1f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 -- 172.21.15.133:0/273425112 >> 172.21.15.133:0/273425112 conn(0x7fa358067000 msgr2=0x7fa35806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 -- 172.21.15.133:0/273425112 shutdown_connections 2024-04-25T10:12:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 -- 172.21.15.133:0/273425112 wait complete. 2024-04-25T10:12:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 Processor -- start 2024-04-25T10:12:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 -- start start 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35806eb70 0x7fa358149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa358158110 0x7fa35814b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3581584f0 0x7fa358152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580ccca0 con 0x7fa35806eb70 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580ccb20 con 0x7fa358158110 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35fbb3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580cce20 con 0x7fa3581584f0 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35d94f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35806eb70 0x7fa358149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35d94f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35806eb70 0x7fa358149e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33762/0 (socket says 172.21.15.133:33762) 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.017+0000 7fa35d94f640 1 -- 172.21.15.133:0/2776524189 learned_addr learned my addr 172.21.15.133:0/2776524189 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35e150640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3581584f0 0x7fa358152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35d14e640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa358158110 0x7fa35814b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35d94f640 1 -- 172.21.15.133:0/2776524189 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3581584f0 msgr2=0x7fa358152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35d94f640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3581584f0 0x7fa358152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35d94f640 1 -- 172.21.15.133:0/2776524189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa358158110 msgr2=0x7fa35814b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35d94f640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa358158110 0x7fa35814b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35d94f640 1 -- 172.21.15.133:0/2776524189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35814a3c0 con 0x7fa35806eb70 2024-04-25T10:12:02.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35d94f640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35806eb70 0x7fa358149e80 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa3540ac040 tx=0x7fa35401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa34e7fc640 1 -- 172.21.15.133:0/2776524189 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354004020 con 0x7fa35806eb70 2024-04-25T10:12:02.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa34e7fc640 1 -- 172.21.15.133:0/2776524189 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3540041c0 con 0x7fa35806eb70 2024-04-25T10:12:02.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35fbb3640 1 -- 172.21.15.133:0/2776524189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35814a660 con 0x7fa35806eb70 2024-04-25T10:12:02.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa34e7fc640 1 -- 172.21.15.133:0/2776524189 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354003620 con 0x7fa35806eb70 2024-04-25T10:12:02.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35fbb3640 1 -- 172.21.15.133:0/2776524189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35814ac20 con 0x7fa35806eb70 2024-04-25T10:12:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35fbb3640 1 -- 172.21.15.133:0/2776524189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa358068ef0 con 0x7fa35806eb70 2024-04-25T10:12:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa34e7fc640 1 -- 172.21.15.133:0/2776524189 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3540037c0 con 0x7fa35806eb70 2024-04-25T10:12:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa34e7fc640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa33c066f60 0x7fa33c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35d14e640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa33c066f60 0x7fa33c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa34e7fc640 1 -- 172.21.15.133:0/2776524189 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa35411f880 con 0x7fa35806eb70 2024-04-25T10:12:02.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.021+0000 7fa35d14e640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa33c066f60 0x7fa33c069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa358151010 tx=0x7fa348007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.025+0000 7fa34e7fc640 1 -- 172.21.15.133:0/2776524189 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa354016090 con 0x7fa35806eb70 2024-04-25T10:12:02.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.173+0000 7fa35fbb3640 1 -- 172.21.15.133:0/2776524189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa35815ec10 con 0x7fa35806eb70 2024-04-25T10:12:02.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa34e7fc640 1 -- 172.21.15.133:0/2776524189 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17005 (secure 0 0 0) 0x7fa3540f06d0 con 0x7fa35806eb70 2024-04-25T10:12:02.178 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:02.178 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":109,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:11:34.158734+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:09.673185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:10.421023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":5,"to":4}]},{"pgid":"16.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:02.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 -- 172.21.15.133:0/2776524189 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa33c066f60 msgr2=0x7fa33c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa33c066f60 0x7fa33c069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa358151010 tx=0x7fa348007c10 comp rx=0 tx=0).stop 2024-04-25T10:12:02.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 -- 172.21.15.133:0/2776524189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35806eb70 msgr2=0x7fa358149e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35806eb70 0x7fa358149e80 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa3540ac040 tx=0x7fa35401acb0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 -- 172.21.15.133:0/2776524189 shutdown_connections 2024-04-25T10:12:02.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3581584f0 0x7fa358152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa358158110 0x7fa35814b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa33c066f60 0x7fa33c069420 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 --2- 172.21.15.133:0/2776524189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35806eb70 0x7fa358149e80 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 -- 172.21.15.133:0/2776524189 >> 172.21.15.133:0/2776524189 conn(0x7fa358067000 msgr2=0x7fa358157100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 -- 172.21.15.133:0/2776524189 shutdown_connections 2024-04-25T10:12:02.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.177+0000 7fa35fbb3640 1 -- 172.21.15.133:0/2776524189 wait complete. 2024-04-25T10:12:02.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:09.673185+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:10.421023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:02.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:02.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.301+0000 7fae88d54640 1 Processor -- start 2024-04-25T10:12:02.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.301+0000 7fae88d54640 1 -- start start 2024-04-25T10:12:02.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae88d54640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fae84145170 0x7fae8406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae88d54640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae8415ced0 0x7fae8415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae88d54640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae840ce2e0 con 0x7fae8406e4d0 2024-04-25T10:12:02.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae88d54640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8406db00 con 0x7fae8415ced0 2024-04-25T10:12:02.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae88d54640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae84145550 con 0x7fae84145170 2024-04-25T10:12:02.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae82d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae8415ced0 0x7fae8415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae82d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae8415ced0 0x7fae8415d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33764/0 (socket says 172.21.15.133:33764) 2024-04-25T10:12:02.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae82d9d640 1 -- 172.21.15.133:0/147482766 learned_addr learned my addr 172.21.15.133:0/147482766 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:02.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae8259c640 1 --2- 172.21.15.133:0/147482766 >> v2:172.21.15.162:3300/0 conn(0x7fae84145170 0x7fae8406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae82d9d640 1 -- 172.21.15.133:0/147482766 >> v2:172.21.15.162:3300/0 conn(0x7fae84145170 msgr2=0x7fae8406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae82d9d640 1 --2- 172.21.15.133:0/147482766 >> v2:172.21.15.162:3300/0 conn(0x7fae84145170 0x7fae8406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae82d9d640 1 -- 172.21.15.133:0/147482766 >> v1:172.21.15.162:6789/0 conn(0x7fae8406e4d0 legacy=0x7fae840cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae82d9d640 1 -- 172.21.15.133:0/147482766 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8406d4b0 con 0x7fae8415ced0 2024-04-25T10:12:02.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae8259c640 1 --2- 172.21.15.133:0/147482766 >> v2:172.21.15.162:3300/0 conn(0x7fae84145170 0x7fae8406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:02.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae82d9d640 1 --2- 172.21.15.133:0/147482766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae8415ced0 0x7fae8415d320 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fae8406b790 tx=0x7fae7c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89e6083092e4d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.305+0000 7fae8159a640 1 -- 172.21.15.133:0/147482766 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0a5670 con 0x7fae8415ced0 2024-04-25T10:12:02.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae8159a640 1 -- 172.21.15.133:0/147482766 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c0a5810 con 0x7fae8415ced0 2024-04-25T10:12:02.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae8159a640 1 -- 172.21.15.133:0/147482766 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0a5be0 con 0x7fae8415ced0 2024-04-25T10:12:02.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 -- 172.21.15.133:0/147482766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae8415ced0 msgr2=0x7fae8415d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 --2- 172.21.15.133:0/147482766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae8415ced0 0x7fae8415d320 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fae8406b790 tx=0x7fae7c01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 -- 172.21.15.133:0/147482766 shutdown_connections 2024-04-25T10:12:02.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 --2- 172.21.15.133:0/147482766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae8415ced0 0x7fae8415d320 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 --2- 172.21.15.133:0/147482766 >> v2:172.21.15.162:3300/0 conn(0x7fae84145170 0x7fae8406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 -- 172.21.15.133:0/147482766 >> 172.21.15.133:0/147482766 conn(0x7fae84066f70 msgr2=0x7fae8406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 -- 172.21.15.133:0/147482766 shutdown_connections 2024-04-25T10:12:02.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 -- 172.21.15.133:0/147482766 wait complete. 2024-04-25T10:12:02.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 Processor -- start 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 -- start start 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae8406e4d0 0x7fae84154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae84145170 0x7fae8415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae8415ced0 0x7fae8415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae84168610 con 0x7fae8406e4d0 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae84168490 con 0x7fae84145170 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae84168790 con 0x7fae8415ced0 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae8259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae8406e4d0 0x7fae84154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae8259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae8406e4d0 0x7fae84154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33776/0 (socket says 172.21.15.133:33776) 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae8259c640 1 -- 172.21.15.133:0/833251067 learned_addr learned my addr 172.21.15.133:0/833251067 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae82d9d640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae8415ced0 0x7fae8415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae8259c640 1 -- 172.21.15.133:0/833251067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae8415ced0 msgr2=0x7fae8415ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae8259c640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae8415ced0 0x7fae8415ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae8259c640 1 -- 172.21.15.133:0/833251067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae84145170 msgr2=0x7fae8415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae8259c640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae84145170 0x7fae8415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae8259c640 1 -- 172.21.15.133:0/833251067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae84155680 con 0x7fae8406e4d0 2024-04-25T10:12:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae8259c640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae8406e4d0 0x7fae84154cf0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c820 tx=0x7fae7400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae72ffd640 1 -- 172.21.15.133:0/833251067 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74015070 con 0x7fae8406e4d0 2024-04-25T10:12:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae72ffd640 1 -- 172.21.15.133:0/833251067 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae74007a30 con 0x7fae8406e4d0 2024-04-25T10:12:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 -- 172.21.15.133:0/833251067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae84155960 con 0x7fae8406e4d0 2024-04-25T10:12:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae72ffd640 1 -- 172.21.15.133:0/833251067 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74007e40 con 0x7fae8406e4d0 2024-04-25T10:12:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.309+0000 7fae88d54640 1 -- 172.21.15.133:0/833251067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae84155f20 con 0x7fae8406e4d0 2024-04-25T10:12:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.313+0000 7fae88d54640 1 -- 172.21.15.133:0/833251067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae84068e80 con 0x7fae8406e4d0 2024-04-25T10:12:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.313+0000 7fae72ffd640 1 -- 172.21.15.133:0/833251067 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae74018070 con 0x7fae8406e4d0 2024-04-25T10:12:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.313+0000 7fae72ffd640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae64066f60 0x7fae64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.313+0000 7fae72ffd640 1 -- 172.21.15.133:0/833251067 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fae74085110 con 0x7fae8406e4d0 2024-04-25T10:12:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.313+0000 7fae81d9b640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae64066f60 0x7fae64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.313+0000 7fae81d9b640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae64066f60 0x7fae64069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fae84160250 tx=0x7fae78006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.317+0000 7fae72ffd640 1 -- 172.21.15.133:0/833251067 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae74055f60 con 0x7fae8406e4d0 2024-04-25T10:12:02.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.469+0000 7fae88d54640 1 -- 172.21.15.133:0/833251067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae840af4b0 con 0x7fae8406e4d0 2024-04-25T10:12:02.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.469+0000 7fae72ffd640 1 -- 172.21.15.133:0/833251067 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae74055860 con 0x7fae8406e4d0 2024-04-25T10:12:02.473 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 -- 172.21.15.133:0/833251067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae64066f60 msgr2=0x7fae64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae64066f60 0x7fae64069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fae84160250 tx=0x7fae78006bd0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 -- 172.21.15.133:0/833251067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae8406e4d0 msgr2=0x7fae84154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae8406e4d0 0x7fae84154cf0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c820 tx=0x7fae7400cce0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 -- 172.21.15.133:0/833251067 shutdown_connections 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae8415ced0 0x7fae8415ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae84145170 0x7fae8415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fae64066f60 0x7fae64069420 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 --2- 172.21.15.133:0/833251067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae8406e4d0 0x7fae84154cf0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 -- 172.21.15.133:0/833251067 >> 172.21.15.133:0/833251067 conn(0x7fae84066f70 msgr2=0x7fae8415d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 -- 172.21.15.133:0/833251067 shutdown_connections 2024-04-25T10:12:02.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.473+0000 7fae88d54640 1 -- 172.21.15.133:0/833251067 wait complete. 2024-04-25T10:12:02.490 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:02.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.597+0000 7f1230169640 1 Processor -- start 2024-04-25T10:12:02.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.597+0000 7f1230169640 1 -- start start 2024-04-25T10:12:02.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f1230169640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1228145170 0x7f122806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f1230169640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 0x7f12280cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f1230169640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12280ce2e0 con 0x7f12280cdd60 2024-04-25T10:12:02.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f1230169640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122806db00 con 0x7f1228145170 2024-04-25T10:12:02.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f1230169640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1228145550 con 0x7f122806e4d0 2024-04-25T10:12:02.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f122d704640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 0x7f12280cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f122df05640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1228145170 0x7f122806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f122d704640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 0x7f12280cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:37330/0 (socket says 172.21.15.133:37330) 2024-04-25T10:12:02.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f122d704640 1 -- 172.21.15.133:0/824134649 learned_addr learned my addr 172.21.15.133:0/824134649 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:02.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f122df05640 1 -- 172.21.15.133:0/824134649 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 msgr2=0x7f12280cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f122df05640 1 --2- 172.21.15.133:0/824134649 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 0x7f12280cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f122df05640 1 -- 172.21.15.133:0/824134649 >> v1:172.21.15.162:6789/0 conn(0x7f12280cdd60 legacy=0x7f122815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f122df05640 1 -- 172.21.15.133:0/824134649 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122806d4b0 con 0x7f1228145170 2024-04-25T10:12:02.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.601+0000 7f122d704640 1 --2- 172.21.15.133:0/824134649 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 0x7f12280cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:02.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122df05640 1 --2- 172.21.15.133:0/824134649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1228145170 0x7f122806df90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f12240097c0 tx=0x7f122401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cef80f20c37cd58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122cf03640 1 -- 172.21.15.133:0/824134649 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240b0070 con 0x7f1228145170 2024-04-25T10:12:02.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122cf03640 1 -- 172.21.15.133:0/824134649 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12240ab600 con 0x7f1228145170 2024-04-25T10:12:02.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122cf03640 1 -- 172.21.15.133:0/824134649 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240ab9d0 con 0x7f1228145170 2024-04-25T10:12:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 -- 172.21.15.133:0/824134649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1228145170 msgr2=0x7f122806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 --2- 172.21.15.133:0/824134649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1228145170 0x7f122806df90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f12240097c0 tx=0x7f122401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 -- 172.21.15.133:0/824134649 shutdown_connections 2024-04-25T10:12:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 --2- 172.21.15.133:0/824134649 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 0x7f12280cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 --2- 172.21.15.133:0/824134649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1228145170 0x7f122806df90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 -- 172.21.15.133:0/824134649 >> 172.21.15.133:0/824134649 conn(0x7f1228066f70 msgr2=0x7f122806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 -- 172.21.15.133:0/824134649 shutdown_connections 2024-04-25T10:12:02.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 -- 172.21.15.133:0/824134649 wait complete. 2024-04-25T10:12:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 Processor -- start 2024-04-25T10:12:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 -- start start 2024-04-25T10:12:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 0x7f1228139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12280cdd60 0x7f1228138d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1228145170 0x7f12281433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1228168490 con 0x7f1228145170 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122815d7f0 con 0x7f12280cdd60 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f1230169640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1228168610 con 0x7f122806e4d0 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122e706640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1228145170 0x7f12281433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122e706640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1228145170 0x7f12281433e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33788/0 (socket says 172.21.15.133:33788) 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122e706640 1 -- 172.21.15.133:0/931325859 learned_addr learned my addr 172.21.15.133:0/931325859 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122df05640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 0x7f1228139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122d704640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12280cdd60 0x7f1228138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122e706640 1 -- 172.21.15.133:0/931325859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 msgr2=0x7f1228139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122e706640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 0x7f1228139ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122e706640 1 -- 172.21.15.133:0/931325859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12280cdd60 msgr2=0x7f1228138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122e706640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12280cdd60 0x7f1228138d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122e706640 1 -- 172.21.15.133:0/931325859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1228141110 con 0x7f1228145170 2024-04-25T10:12:02.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.605+0000 7f122e706640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1228145170 0x7f12281433e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f122000c730 tx=0x7f122000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.609+0000 7f121e7fc640 1 -- 172.21.15.133:0/931325859 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220003b80 con 0x7f1228145170 2024-04-25T10:12:02.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.609+0000 7f121e7fc640 1 -- 172.21.15.133:0/931325859 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1220003d20 con 0x7f1228145170 2024-04-25T10:12:02.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.609+0000 7f1230169640 1 -- 172.21.15.133:0/931325859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12281413f0 con 0x7f1228145170 2024-04-25T10:12:02.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.609+0000 7f121e7fc640 1 -- 172.21.15.133:0/931325859 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122000aa30 con 0x7f1228145170 2024-04-25T10:12:02.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.609+0000 7f1230169640 1 -- 172.21.15.133:0/931325859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12281419b0 con 0x7f1228145170 2024-04-25T10:12:02.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.609+0000 7f1230169640 1 -- 172.21.15.133:0/931325859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1228068e80 con 0x7f1228145170 2024-04-25T10:12:02.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.609+0000 7f121e7fc640 1 -- 172.21.15.133:0/931325859 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f122000abd0 con 0x7f1228145170 2024-04-25T10:12:02.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.609+0000 7f121e7fc640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f120c066f90 0x7f120c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.609+0000 7f121e7fc640 1 -- 172.21.15.133:0/931325859 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f12200850e0 con 0x7f1228145170 2024-04-25T10:12:02.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.609+0000 7f122df05640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f120c066f90 0x7f120c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.609+0000 7f122df05640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f120c066f90 0x7f120c069450 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f12240ad000 tx=0x7f12240a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.613+0000 7f121e7fc640 1 -- 172.21.15.133:0/931325859 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1220055ea0 con 0x7f1228145170 2024-04-25T10:12:02.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.757+0000 7f1230169640 1 -- 172.21.15.133:0/931325859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1228141db0 con 0x7f1228145170 2024-04-25T10:12:02.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f121e7fc640 1 -- 172.21.15.133:0/931325859 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12200557a0 con 0x7f1228145170 2024-04-25T10:12:02.763 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:12:02.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 -- 172.21.15.133:0/931325859 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f120c066f90 msgr2=0x7f120c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f120c066f90 0x7f120c069450 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f12240ad000 tx=0x7f12240a6000 comp rx=0 tx=0).stop 2024-04-25T10:12:02.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 -- 172.21.15.133:0/931325859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1228145170 msgr2=0x7f12281433e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1228145170 0x7f12281433e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f122000c730 tx=0x7f122000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 -- 172.21.15.133:0/931325859 shutdown_connections 2024-04-25T10:12:02.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f120c066f90 0x7f120c069450 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1228145170 0x7f12281433e0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12280cdd60 0x7f1228138d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 --2- 172.21.15.133:0/931325859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f122806e4d0 0x7f1228139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 -- 172.21.15.133:0/931325859 >> 172.21.15.133:0/931325859 conn(0x7f1228066f70 msgr2=0x7f122813dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 -- 172.21.15.133:0/931325859 shutdown_connections 2024-04-25T10:12:02.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.761+0000 7f1230169640 1 -- 172.21.15.133:0/931325859 wait complete. 2024-04-25T10:12:02.780 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:02.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9392ffc640 1 Processor -- start 2024-04-25T10:12:02.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9392ffc640 1 -- start start 2024-04-25T10:12:02.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9392ffc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f938c15d9a0 0x7f938c0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9392ffc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938c0ce600 0x7f938c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9392ffc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f938c0d1920 0x7f938c0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9392ffc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938c0cefa0 con 0x7f938c0d1920 2024-04-25T10:12:02.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9392ffc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938c15ebd0 con 0x7f938c15d9a0 2024-04-25T10:12:02.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9392ffc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c15eeb0 con 0x7f938c0ce600 2024-04-25T10:12:02.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9391599640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f938c0d1920 0x7f938c0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9391599640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f938c0d1920 0x7f938c0d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33792/0 (socket says 172.21.15.133:33792) 2024-04-25T10:12:02.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9391599640 1 -- 172.21.15.133:0/3445255485 learned_addr learned my addr 172.21.15.133:0/3445255485 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:02.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f938bfff640 1 --2- 172.21.15.133:0/3445255485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938c0ce600 0x7f938c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9390d98640 1 --2- 172.21.15.133:0/3445255485 >> v2:172.21.15.162:3300/0 conn(0x7f938c15d9a0 0x7f938c0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9391599640 1 -- 172.21.15.133:0/3445255485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938c0ce600 msgr2=0x7f938c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9391599640 1 --2- 172.21.15.133:0/3445255485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938c0ce600 0x7f938c0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9391599640 1 -- 172.21.15.133:0/3445255485 >> v2:172.21.15.162:3300/0 conn(0x7f938c15d9a0 msgr2=0x7f938c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:02.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9391599640 1 --2- 172.21.15.133:0/3445255485 >> v2:172.21.15.162:3300/0 conn(0x7f938c15d9a0 0x7f938c0d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9391599640 1 -- 172.21.15.133:0/3445255485 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c0d0330 con 0x7f938c0d1920 2024-04-25T10:12:02.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9390d98640 1 --2- 172.21.15.133:0/3445255485 >> v2:172.21.15.162:3300/0 conn(0x7f938c15d9a0 0x7f938c0d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:02.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.889+0000 7f9391599640 1 --2- 172.21.15.133:0/3445255485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f938c0d1920 0x7f938c0d1db0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f938c0d4750 tx=0x7f938401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfe01dfd262eb96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938b7fe640 1 -- 172.21.15.133:0/3445255485 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93840a5790 con 0x7f938c0d1920 2024-04-25T10:12:02.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938b7fe640 1 -- 172.21.15.133:0/3445255485 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93840a5930 con 0x7f938c0d1920 2024-04-25T10:12:02.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938b7fe640 1 -- 172.21.15.133:0/3445255485 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93840a5d00 con 0x7f938c0d1920 2024-04-25T10:12:02.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 -- 172.21.15.133:0/3445255485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f938c0d1920 msgr2=0x7f938c0d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 --2- 172.21.15.133:0/3445255485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f938c0d1920 0x7f938c0d1db0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f938c0d4750 tx=0x7f938401c930 comp rx=0 tx=0).stop 2024-04-25T10:12:02.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 -- 172.21.15.133:0/3445255485 shutdown_connections 2024-04-25T10:12:02.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 --2- 172.21.15.133:0/3445255485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f938c0d1920 0x7f938c0d1db0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 --2- 172.21.15.133:0/3445255485 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938c0ce600 0x7f938c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 --2- 172.21.15.133:0/3445255485 >> v2:172.21.15.162:3300/0 conn(0x7f938c15d9a0 0x7f938c0d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 -- 172.21.15.133:0/3445255485 >> 172.21.15.133:0/3445255485 conn(0x7f938c06bf80 msgr2=0x7f938c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:02.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 -- 172.21.15.133:0/3445255485 shutdown_connections 2024-04-25T10:12:02.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 -- 172.21.15.133:0/3445255485 wait complete. 2024-04-25T10:12:02.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 Processor -- start 2024-04-25T10:12:02.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 -- start start 2024-04-25T10:12:02.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938c0ce600 0x7f938c154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f938c0d1920 0x7f938c152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f938c15d9a0 0x7f938c157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0b9820 con 0x7f938c0d1920 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0b96a0 con 0x7f938c15d9a0 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0d22f0 con 0x7f938c0ce600 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f938c0d1920 0x7f938c152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f938c0d1920 0x7f938c152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33796/0 (socket says 172.21.15.133:33796) 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938bfff640 1 -- 172.21.15.133:0/973992576 learned_addr learned my addr 172.21.15.133:0/973992576 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9390d98640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938c0ce600 0x7f938c154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9391599640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f938c15d9a0 0x7f938c157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938bfff640 1 -- 172.21.15.133:0/973992576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938c0ce600 msgr2=0x7f938c154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938bfff640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938c0ce600 0x7f938c154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938bfff640 1 -- 172.21.15.133:0/973992576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f938c15d9a0 msgr2=0x7f938c157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938bfff640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f938c15d9a0 0x7f938c157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938bfff640 1 -- 172.21.15.133:0/973992576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c153780 con 0x7f938c0d1920 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9390d98640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938c0ce600 0x7f938c154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:02.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f938bfff640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f938c0d1920 0x7f938c152cb0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f937800d830 tx=0x7f937800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f93897fa640 1 -- 172.21.15.133:0/973992576 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378018030 con 0x7f938c0d1920 2024-04-25T10:12:02.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f93897fa640 1 -- 172.21.15.133:0/973992576 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937800be10 con 0x7f938c0d1920 2024-04-25T10:12:02.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 -- 172.21.15.133:0/973992576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c153a60 con 0x7f938c0d1920 2024-04-25T10:12:02.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f93897fa640 1 -- 172.21.15.133:0/973992576 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93780098b0 con 0x7f938c0d1920 2024-04-25T10:12:02.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.893+0000 7f9392ffc640 1 -- 172.21.15.133:0/973992576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938c154020 con 0x7f938c0d1920 2024-04-25T10:12:02.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.897+0000 7f9392ffc640 1 -- 172.21.15.133:0/973992576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938c068e80 con 0x7f938c0d1920 2024-04-25T10:12:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.897+0000 7f93897fa640 1 -- 172.21.15.133:0/973992576 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f937801b070 con 0x7f938c0d1920 2024-04-25T10:12:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.897+0000 7f93897fa640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9370066f60 0x7f9370069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.897+0000 7f93897fa640 1 -- 172.21.15.133:0/973992576 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f9378085430 con 0x7f938c0d1920 2024-04-25T10:12:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.897+0000 7f9390d98640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9370066f60 0x7f9370069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.897+0000 7f9390d98640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9370066f60 0x7f9370069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9380008930 tx=0x7f9380008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:02.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:02.901+0000 7f93897fa640 1 -- 172.21.15.133:0/973992576 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93780561e0 con 0x7f938c0d1920 2024-04-25T10:12:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.045+0000 7f9392ffc640 1 -- 172.21.15.133:0/973992576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f938c0d0d60 con 0x7f938c0d1920 2024-04-25T10:12:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.045+0000 7f93897fa640 1 -- 172.21.15.133:0/973992576 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9378055ae0 con 0x7f938c0d1920 2024-04-25T10:12:03.049 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:12:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 -- 172.21.15.133:0/973992576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9370066f60 msgr2=0x7f9370069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9370066f60 0x7f9370069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9380008930 tx=0x7f9380008040 comp rx=0 tx=0).stop 2024-04-25T10:12:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 -- 172.21.15.133:0/973992576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f938c0d1920 msgr2=0x7f938c152cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f938c0d1920 0x7f938c152cb0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f937800d830 tx=0x7f937800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 -- 172.21.15.133:0/973992576 shutdown_connections 2024-04-25T10:12:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f938c15d9a0 0x7f938c157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9370066f60 0x7f9370069420 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f938c0d1920 0x7f938c152cb0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 --2- 172.21.15.133:0/973992576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f938c0ce600 0x7f938c154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 -- 172.21.15.133:0/973992576 >> 172.21.15.133:0/973992576 conn(0x7f938c06bf80 msgr2=0x7f938c0b77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 -- 172.21.15.133:0/973992576 shutdown_connections 2024-04-25T10:12:03.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.049+0000 7f9392ffc640 1 -- 172.21.15.133:0/973992576 wait complete. 2024-04-25T10:12:03.067 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:12:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f563b4d6640 1 Processor -- start 2024-04-25T10:12:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f563b4d6640 1 -- start start 2024-04-25T10:12:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f563b4d6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5634145170 0x7f563406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f563b4d6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f563406e4d0 0x7f56340cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f563b4d6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56340ce2e0 con 0x7f56340cdd60 2024-04-25T10:12:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f563b4d6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563406db00 con 0x7f563406e4d0 2024-04-25T10:12:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f563b4d6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5634145550 con 0x7f5634145170 2024-04-25T10:12:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5639a73640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f56340cdd60 0x7f563415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48212/0 (socket says 172.21.15.133:48212) 2024-04-25T10:12:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5639a73640 1 -- 172.21.15.133:0/3062622459 learned_addr learned my addr 172.21.15.133:0/3062622459 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:03.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5639272640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5634145170 0x7f563406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402919690 0 0) 0x7f56340ce2e0 con 0x7f56340cdd60 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5620003530 con 0x7f56340cdd60 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5638a71640 1 --2- 172.21.15.133:0/3062622459 >> v2:172.21.15.162:3300/0 conn(0x7f563406e4d0 0x7f56340cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 919420626 0 0) 0x7f5620003530 con 0x7f56340cdd60 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5634145170 msgr2=0x7f563406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5634145170 0x7f563406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 >> v2:172.21.15.162:3300/0 conn(0x7f563406e4d0 msgr2=0x7f56340cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 --2- 172.21.15.133:0/3062622459 >> v2:172.21.15.162:3300/0 conn(0x7f563406e4d0 0x7f56340cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563406d4b0 con 0x7f56340cdd60 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f562c001a00 con 0x7f56340cdd60 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5638a71640 1 --2- 172.21.15.133:0/3062622459 >> v2:172.21.15.162:3300/0 conn(0x7f563406e4d0 0x7f56340cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 >> v1:172.21.15.133:6789/0 conn(0x7f56340cdd60 legacy=0x7f563415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56200039c0 0x7f5620003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5634145170 0x7f5620004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5620004d50 0x7f56200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563406db00 con 0x7f5620004d50 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5634145550 con 0x7f56200039c0 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562000b9d0 con 0x7f5634145170 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5639a73640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5620004d50 0x7f56200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5639272640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56200039c0 0x7f5620003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5638a71640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5634145170 0x7f5620004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5639a73640 1 -- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5634145170 msgr2=0x7f5620004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5639a73640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5634145170 0x7f5620004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5639a73640 1 -- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56200039c0 msgr2=0x7f5620003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5639a73640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56200039c0 0x7f5620003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5639a73640 1 -- 172.21.15.133:0/3062622459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5620003530 con 0x7f5620004d50 2024-04-25T10:12:03.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.173+0000 7f5638a71640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5634145170 0x7f5620004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:03.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f5639a73640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5620004d50 0x7f56200051d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f562c00a8d0 tx=0x7f562c01d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163e52e05270c60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c00a4c0 con 0x7f5620004d50 2024-04-25T10:12:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c0a9020 con 0x7f5620004d50 2024-04-25T10:12:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f562bfff640 1 -- 172.21.15.133:0/3062622459 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0072f0 con 0x7f5620004d50 2024-04-25T10:12:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 -- 172.21.15.133:0/3062622459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5620004d50 msgr2=0x7f56200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 --2- 172.21.15.133:0/3062622459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5620004d50 0x7f56200051d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f562c00a8d0 tx=0x7f562c01d3c0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f5639a73640 1 -- 172.21.15.133:0/3062622459 reap_dead start 2024-04-25T10:12:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 -- 172.21.15.133:0/3062622459 shutdown_connections 2024-04-25T10:12:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 -- 172.21.15.133:0/3062622459 >> 172.21.15.133:0/3062622459 conn(0x7f5634066f70 msgr2=0x7f563406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 -- 172.21.15.133:0/3062622459 shutdown_connections 2024-04-25T10:12:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 -- 172.21.15.133:0/3062622459 wait complete. 2024-04-25T10:12:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 Processor -- start 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 -- start start 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56341430c0 0x7f56340ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56340cce50 0x7f56340b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56340cd2b0 0x7f56340b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563415d5e0 con 0x7f56340cce50 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563415cb70 con 0x7f56340cd2b0 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f563b4d6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563415d760 con 0x7f56341430c0 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f5638a71640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56340cce50 0x7f56340b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f5638a71640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56340cce50 0x7f56340b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33816/0 (socket says 172.21.15.133:33816) 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f5638a71640 1 -- 172.21.15.133:0/3447146518 learned_addr learned my addr 172.21.15.133:0/3447146518 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f5639272640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56341430c0 0x7f56340ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f5639a73640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56340cd2b0 0x7f56340b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.177+0000 7f5638a71640 1 -- 172.21.15.133:0/3447146518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56341430c0 msgr2=0x7f56340ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5638a71640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56341430c0 0x7f56340ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5638a71640 1 -- 172.21.15.133:0/3447146518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56340cd2b0 msgr2=0x7f56340b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5638a71640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56340cd2b0 0x7f56340b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5638a71640 1 -- 172.21.15.133:0/3447146518 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340ac660 con 0x7f56340cce50 2024-04-25T10:12:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5639272640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56341430c0 0x7f56340ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5638a71640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56340cce50 0x7f56340b0620 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f562400f650 tx=0x7f562400fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5629ffb640 1 -- 172.21.15.133:0/3447146518 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624018070 con 0x7f56340cce50 2024-04-25T10:12:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5629ffb640 1 -- 172.21.15.133:0/3447146518 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562400d940 con 0x7f56340cce50 2024-04-25T10:12:03.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f563b4d6640 1 -- 172.21.15.133:0/3447146518 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56340ac940 con 0x7f56340cce50 2024-04-25T10:12:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5629ffb640 1 -- 172.21.15.133:0/3447146518 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562400dcf0 con 0x7f56340cce50 2024-04-25T10:12:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f563b4d6640 1 -- 172.21.15.133:0/3447146518 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56340acf00 con 0x7f56340cce50 2024-04-25T10:12:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f563b4d6640 1 -- 172.21.15.133:0/3447146518 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5634152c40 con 0x7f56340cce50 2024-04-25T10:12:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5629ffb640 1 -- 172.21.15.133:0/3447146518 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f562401b070 con 0x7f56340cce50 2024-04-25T10:12:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5629ffb640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5618066f90 0x7f5618069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5629ffb640 1 -- 172.21.15.133:0/3447146518 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5624085430 con 0x7f56340cce50 2024-04-25T10:12:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5639272640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5618066f90 0x7f5618069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.181+0000 7f5639272640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5618066f90 0x7f5618069450 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5630009a70 tx=0x7f5630008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.185+0000 7f5629ffb640 1 -- 172.21.15.133:0/3447146518 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5624056310 con 0x7f56340cce50 2024-04-25T10:12:03.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.349+0000 7f563b4d6640 1 -- 172.21.15.133:0/3447146518 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5634068e80 con 0x7f56340cce50 2024-04-25T10:12:03.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.349+0000 7f5629ffb640 1 -- 172.21.15.133:0/3447146518 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5624055c10 con 0x7f56340cce50 2024-04-25T10:12:03.352 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:12:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 -- 172.21.15.133:0/3447146518 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5618066f90 msgr2=0x7f5618069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5618066f90 0x7f5618069450 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5630009a70 tx=0x7f5630008040 comp rx=0 tx=0).stop 2024-04-25T10:12:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 -- 172.21.15.133:0/3447146518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56340cce50 msgr2=0x7f56340b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56340cce50 0x7f56340b0620 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f562400f650 tx=0x7f562400fb10 comp rx=0 tx=0).stop 2024-04-25T10:12:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 -- 172.21.15.133:0/3447146518 shutdown_connections 2024-04-25T10:12:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56340cd2b0 0x7f56340b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5618066f90 0x7f5618069450 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56340cce50 0x7f56340b0620 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 --2- 172.21.15.133:0/3447146518 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56341430c0 0x7f56340ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 -- 172.21.15.133:0/3447146518 >> 172.21.15.133:0/3447146518 conn(0x7f5634066f70 msgr2=0x7f56340b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 -- 172.21.15.133:0/3447146518 shutdown_connections 2024-04-25T10:12:03.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.353+0000 7f563b4d6640 1 -- 172.21.15.133:0/3447146518 wait complete. 2024-04-25T10:12:03.369 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:12:03.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:03.370 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:03.380 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[74919]: fuse finished with error 0 and tester_r 0 2024-04-25T10:12:03.488 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:12:03.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:03.488 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:12:03.646 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:12:03.646 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:03.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:03.653 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:12:03.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:03.653 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:12:03.701 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:12:03.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:03.703 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:12:03.703 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:03.709 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:03.709 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:12:03.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:03.709 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:12:03.754 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:12:03.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:03.756 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:12:03.756 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:03.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:03.762 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:12:03.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:03.762 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:12:03.810 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:12:03.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:03.811 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0009ec2640 1 Processor -- start 2024-04-25T10:12:03.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0009ec2640 1 -- start start 2024-04-25T10:12:03.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0009ec2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000415b9b0 0x7f00040cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0009ec2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f00040d0380 0x7f00040c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0009ec2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000415bd90 con 0x7f00040cf7c0 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0009ec2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00040c5610 con 0x7f00040d0380 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0009ec2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000409ad10 con 0x7f000415b9b0 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0002ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f00040cf7c0 0x7f00040cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48226/0 (socket says 172.21.15.133:48226) 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0002ffd640 1 -- 172.21.15.133:0/3246487210 learned_addr learned my addr 172.21.15.133:0/3246487210 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00037fe640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000415b9b0 0x7f00040cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081789307 0 0) 0x7f000415bd90 con 0x7f00040cf7c0 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff0003530 con 0x7f00040cf7c0 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0003fff640 1 --2- 172.21.15.133:0/3246487210 >> v2:172.21.15.162:3300/0 conn(0x7f00040d0380 0x7f00040c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1652968999 0 0) 0x7efff0003530 con 0x7f00040cf7c0 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000415b9b0 msgr2=0x7f00040cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000415b9b0 0x7f00040cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 >> v2:172.21.15.162:3300/0 conn(0x7f00040d0380 msgr2=0x7f00040c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 --2- 172.21.15.133:0/3246487210 >> v2:172.21.15.162:3300/0 conn(0x7f00040d0380 0x7f00040c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000409a750 con 0x7f00040cf7c0 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7effec0016a0 con 0x7f00040cf7c0 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 >> v1:172.21.15.133:6789/0 conn(0x7f00040cf7c0 legacy=0x7f00040cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efff00039c0 0x7efff0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efff0004810 0x7efff0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000415b9b0 0x7efff00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00040c5610 con 0x7efff00039c0 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000409ad10 con 0x7efff0004810 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff000b9d0 con 0x7f000415b9b0 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00037fe640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efff00039c0 0x7efff0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0002ffd640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efff0004810 0x7efff0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0003fff640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000415b9b0 0x7efff00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00037fe640 1 -- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000415b9b0 msgr2=0x7efff00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00037fe640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000415b9b0 0x7efff00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00037fe640 1 -- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efff0004810 msgr2=0x7efff0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00037fe640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efff0004810 0x7efff0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f00037fe640 1 -- 172.21.15.133:0/3246487210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff0003530 con 0x7efff00039c0 2024-04-25T10:12:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.937+0000 7f0003fff640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f000415b9b0 0x7efff00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f00037fe640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efff00039c0 0x7efff0003de0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7efff4009ae0 tx=0x7efff401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4476c0738ed008b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40096b0 con 0x7efff00039c0 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff40a6450 con 0x7efff00039c0 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f00027fc640 1 -- 172.21.15.133:0/3246487210 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40a6840 con 0x7efff00039c0 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f0009ec2640 1 -- 172.21.15.133:0/3246487210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efff00039c0 msgr2=0x7efff0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f0009ec2640 1 --2- 172.21.15.133:0/3246487210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efff00039c0 0x7efff0003de0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7efff4009ae0 tx=0x7efff401ca40 comp rx=0 tx=0).stop 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f0003fff640 1 -- 172.21.15.133:0/3246487210 reap_dead start 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f0009ec2640 1 -- 172.21.15.133:0/3246487210 shutdown_connections 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f0009ec2640 1 -- 172.21.15.133:0/3246487210 >> 172.21.15.133:0/3246487210 conn(0x7f00040b31a0 msgr2=0x7f00040b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f0009ec2640 1 -- 172.21.15.133:0/3246487210 shutdown_connections 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f0009ec2640 1 -- 172.21.15.133:0/3246487210 wait complete. 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f0009ec2640 1 Processor -- start 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.941+0000 7f0009ec2640 1 -- start start 2024-04-25T10:12:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0009ec2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efff00039c0 0x7f0004159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0009ec2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00041fa600 0x7f00041f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0009ec2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00041f6ba0 0x7f00040cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0009ec2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00040c6970 con 0x7f00041f6ba0 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0009ec2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00040c67f0 con 0x7f00041fa600 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0009ec2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00040c6af0 con 0x7efff00039c0 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0003fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00041f6ba0 0x7f00040cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0003fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00041f6ba0 0x7f00040cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33838/0 (socket says 172.21.15.133:33838) 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0003fff640 1 -- 172.21.15.133:0/53401997 learned_addr learned my addr 172.21.15.133:0/53401997 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0002ffd640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00041fa600 0x7f00041f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f00037fe640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efff00039c0 0x7f0004159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0003fff640 1 -- 172.21.15.133:0/53401997 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efff00039c0 msgr2=0x7f0004159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0003fff640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efff00039c0 0x7f0004159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0003fff640 1 -- 172.21.15.133:0/53401997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00041fa600 msgr2=0x7f00041f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:03.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0003fff640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00041fa600 0x7f00041f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:03.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0003fff640 1 -- 172.21.15.133:0/53401997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0004159a50 con 0x7f00041f6ba0 2024-04-25T10:12:03.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0003fff640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00041f6ba0 0x7f00040cc460 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7efffc009fd0 tx=0x7efffc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7efffbfff640 1 -- 172.21.15.133:0/53401997 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc01c070 con 0x7f00041f6ba0 2024-04-25T10:12:03.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7efffbfff640 1 -- 172.21.15.133:0/53401997 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc017400 con 0x7f00041f6ba0 2024-04-25T10:12:03.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0009ec2640 1 -- 172.21.15.133:0/53401997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00040b17f0 con 0x7f00041f6ba0 2024-04-25T10:12:03.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7efffbfff640 1 -- 172.21.15.133:0/53401997 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc0177d0 con 0x7f00041f6ba0 2024-04-25T10:12:03.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0009ec2640 1 -- 172.21.15.133:0/53401997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00041f70e0 con 0x7f00041f6ba0 2024-04-25T10:12:03.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f0009ec2640 1 -- 172.21.15.133:0/53401997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0004067c70 con 0x7f00041f6ba0 2024-04-25T10:12:03.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7efffbfff640 1 -- 172.21.15.133:0/53401997 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efffc01f070 con 0x7f00041f6ba0 2024-04-25T10:12:03.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7efffbfff640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effe4066f60 0x7effe4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:03.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7efffbfff640 1 -- 172.21.15.133:0/53401997 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7efffc0852d0 con 0x7f00041f6ba0 2024-04-25T10:12:03.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f00037fe640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effe4066f60 0x7effe4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:03.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.945+0000 7f00037fe640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effe4066f60 0x7effe4069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efff40ad000 tx=0x7efff40b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:03.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:03.949+0000 7efffbfff640 1 -- 172.21.15.133:0/53401997 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efffc012070 con 0x7f00041f6ba0 2024-04-25T10:12:04.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 -- 172.21.15.133:0/53401997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00040cdbd0 con 0x7f00041f6ba0 2024-04-25T10:12:04.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7efffbfff640 1 -- 172.21.15.133:0/53401997 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7efffc056120 con 0x7f00041f6ba0 2024-04-25T10:12:04.136 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:04.136 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:04.136 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 -- 172.21.15.133:0/53401997 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effe4066f60 msgr2=0x7effe4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effe4066f60 0x7effe4069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efff40ad000 tx=0x7efff40b2040 comp rx=0 tx=0).stop 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 -- 172.21.15.133:0/53401997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00041f6ba0 msgr2=0x7f00040cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00041f6ba0 0x7f00040cc460 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7efffc009fd0 tx=0x7efffc00f4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 -- 172.21.15.133:0/53401997 shutdown_connections 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7effe4066f60 0x7effe4069420 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00041f6ba0 0x7f00040cc460 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f00041fa600 0x7f00041f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 --2- 172.21.15.133:0/53401997 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efff00039c0 0x7f0004159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 -- 172.21.15.133:0/53401997 >> 172.21.15.133:0/53401997 conn(0x7f00040b31a0 msgr2=0x7f00040735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 -- 172.21.15.133:0/53401997 shutdown_connections 2024-04-25T10:12:04.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.133+0000 7f0009ec2640 1 -- 172.21.15.133:0/53401997 wait complete. 2024-04-25T10:12:04.153 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:04.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.265+0000 7f6680538640 1 Processor -- start 2024-04-25T10:12:04.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.265+0000 7f6680538640 1 -- start start 2024-04-25T10:12:04.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f6680538640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f66780b1fa0 0x7f66780d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f6680538640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f667815f7c0 0x7f667815fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f6680538640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66780cf490 con 0x7f667815b990 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f6680538640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66780cf770 con 0x7f66780b1fa0 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f6680538640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66780cfa50 con 0x7f667815f7c0 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667e2d4640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f667815b990 0x7f66780b1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48238/0 (socket says 172.21.15.133:48238) 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667e2d4640 1 -- 172.21.15.133:0/3483600704 learned_addr learned my addr 172.21.15.133:0/3483600704 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667dad3640 1 --2- 172.21.15.133:0/3483600704 >> v2:172.21.15.133:3300/0 conn(0x7f66780b1fa0 0x7f66780d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667ead5640 1 --2- 172.21.15.133:0/3483600704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f667815f7c0 0x7f667815fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667dad3640 1 -- 172.21.15.133:0/3483600704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f667815f7c0 msgr2=0x7f667815fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667dad3640 1 --2- 172.21.15.133:0/3483600704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f667815f7c0 0x7f667815fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667dad3640 1 -- 172.21.15.133:0/3483600704 >> v1:172.21.15.133:6789/0 conn(0x7f667815b990 legacy=0x7f66780b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667d2d2640 1 -- 172.21.15.133:0/3483600704 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271222619 0 0) 0x7f66780cf490 con 0x7f667815b990 2024-04-25T10:12:04.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667dad3640 1 -- 172.21.15.133:0/3483600704 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66780cfd30 con 0x7f66780b1fa0 2024-04-25T10:12:04.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667dad3640 1 --2- 172.21.15.133:0/3483600704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66780b1fa0 0x7f66780d0250 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f66740097c0 tx=0x7f667401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63dfedfa75bb900 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:04.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667d2d2640 1 -- 172.21.15.133:0/3483600704 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66740b0070 con 0x7f66780b1fa0 2024-04-25T10:12:04.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667d2d2640 1 -- 172.21.15.133:0/3483600704 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66740ab600 con 0x7f66780b1fa0 2024-04-25T10:12:04.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.269+0000 7f667d2d2640 1 -- 172.21.15.133:0/3483600704 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66740ab9d0 con 0x7f66780b1fa0 2024-04-25T10:12:04.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 -- 172.21.15.133:0/3483600704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66780b1fa0 msgr2=0x7f66780d0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 --2- 172.21.15.133:0/3483600704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66780b1fa0 0x7f66780d0250 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f66740097c0 tx=0x7f667401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 -- 172.21.15.133:0/3483600704 shutdown_connections 2024-04-25T10:12:04.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 --2- 172.21.15.133:0/3483600704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f667815f7c0 0x7f667815fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 --2- 172.21.15.133:0/3483600704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66780b1fa0 0x7f66780d0250 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 -- 172.21.15.133:0/3483600704 >> 172.21.15.133:0/3483600704 conn(0x7f667809aa10 msgr2=0x7f667808af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:04.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 -- 172.21.15.133:0/3483600704 shutdown_connections 2024-04-25T10:12:04.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 -- 172.21.15.133:0/3483600704 wait complete. 2024-04-25T10:12:04.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 Processor -- start 2024-04-25T10:12:04.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 -- start start 2024-04-25T10:12:04.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66780b1fa0 0x7f66780b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667815b990 0x7f66780aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f667815f7c0 0x7f66780ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66780d1f70 con 0x7f667815b990 2024-04-25T10:12:04.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66780d1df0 con 0x7f667815f7c0 2024-04-25T10:12:04.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66780d20f0 con 0x7f66780b1fa0 2024-04-25T10:12:04.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f667dad3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667815b990 0x7f66780aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f667dad3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667815b990 0x7f66780aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33852/0 (socket says 172.21.15.133:33852) 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f667dad3640 1 -- 172.21.15.133:0/2314877012 learned_addr learned my addr 172.21.15.133:0/2314877012 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f667dad3640 1 -- 172.21.15.133:0/2314877012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66780b1fa0 msgr2=0x7f66780b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f667dad3640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66780b1fa0 0x7f66780b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f667dad3640 1 -- 172.21.15.133:0/2314877012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f667815f7c0 msgr2=0x7f66780ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f667dad3640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f667815f7c0 0x7f66780ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f667dad3640 1 -- 172.21.15.133:0/2314877012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667806cf90 con 0x7f667815b990 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f667dad3640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667815b990 0x7f66780aeaf0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f667401a0d0 tx=0x7f6674019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f666effd640 1 -- 172.21.15.133:0/2314877012 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66740b0040 con 0x7f667815b990 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f666effd640 1 -- 172.21.15.133:0/2314877012 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66740ab400 con 0x7f667815b990 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 -- 172.21.15.133:0/2314877012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66780c6250 con 0x7f667815b990 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f666effd640 1 -- 172.21.15.133:0/2314877012 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66740b4630 con 0x7f667815b990 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.273+0000 7f6680538640 1 -- 172.21.15.133:0/2314877012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66780c6810 con 0x7f667815b990 2024-04-25T10:12:04.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.277+0000 7f6680538640 1 -- 172.21.15.133:0/2314877012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6678068ef0 con 0x7f667815b990 2024-04-25T10:12:04.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.277+0000 7f666effd640 1 -- 172.21.15.133:0/2314877012 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66740bc020 con 0x7f667815b990 2024-04-25T10:12:04.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.277+0000 7f666effd640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f665c066f60 0x7f665c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.277+0000 7f666effd640 1 -- 172.21.15.133:0/2314877012 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f667411e600 con 0x7f667815b990 2024-04-25T10:12:04.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.277+0000 7f667e2d4640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f665c066f60 0x7f665c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.277+0000 7f667e2d4640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f665c066f60 0x7f665c069420 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f66680011f0 tx=0x7f6668007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:04.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.281+0000 7f666effd640 1 -- 172.21.15.133:0/2314877012 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f66740c3020 con 0x7f667815b990 2024-04-25T10:12:04.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.425+0000 7f6680538640 1 -- 172.21.15.133:0/2314877012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66780c69e0 con 0x7f667815b990 2024-04-25T10:12:04.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.429+0000 7f666effd640 1 -- 172.21.15.133:0/2314877012 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17005 (secure 0 0 0) 0x7f6674016090 con 0x7f667815b990 2024-04-25T10:12:04.433 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:04.434 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":109,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:11:34.158734+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:09.673185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:10.421023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":5,"to":4}]},{"pgid":"16.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 -- 172.21.15.133:0/2314877012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f665c066f60 msgr2=0x7f665c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f665c066f60 0x7f665c069420 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f66680011f0 tx=0x7f6668007930 comp rx=0 tx=0).stop 2024-04-25T10:12:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 -- 172.21.15.133:0/2314877012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667815b990 msgr2=0x7f66780aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667815b990 0x7f66780aeaf0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f667401a0d0 tx=0x7f6674019240 comp rx=0 tx=0).stop 2024-04-25T10:12:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 -- 172.21.15.133:0/2314877012 shutdown_connections 2024-04-25T10:12:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f667815f7c0 0x7f66780ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f665c066f60 0x7f665c069420 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667815b990 0x7f66780aeaf0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 --2- 172.21.15.133:0/2314877012 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66780b1fa0 0x7f66780b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 -- 172.21.15.133:0/2314877012 >> 172.21.15.133:0/2314877012 conn(0x7f667809aa10 msgr2=0x7f66780b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 -- 172.21.15.133:0/2314877012 shutdown_connections 2024-04-25T10:12:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.433+0000 7f6680538640 1 -- 172.21.15.133:0/2314877012 wait complete. 2024-04-25T10:12:04.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:09.673185+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:10.421023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:04.452 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:04.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.569+0000 7f02098e1640 1 Processor -- start 2024-04-25T10:12:04.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.569+0000 7f02098e1640 1 -- start start 2024-04-25T10:12:04.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f02098e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0204145170 0x7f020406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f02098e1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f020415ced0 0x7f020415d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f02098e1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02040ce2e0 con 0x7f020406e4d0 2024-04-25T10:12:04.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f02098e1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020406db00 con 0x7f020415ced0 2024-04-25T10:12:04.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f02098e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0204145550 con 0x7f0204145170 2024-04-25T10:12:04.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f02027fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f020406e4d0 0x7f02040cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48244/0 (socket says 172.21.15.133:48244) 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f02027fc640 1 -- 172.21.15.133:0/785377947 learned_addr learned my addr 172.21.15.133:0/785377947 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0202ffd640 1 --2- 172.21.15.133:0/785377947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0204145170 0x7f020406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570314715 0 0) 0x7f02040ce2e0 con 0x7f020406e4d0 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ec003530 con 0x7f020406e4d0 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f02037fe640 1 --2- 172.21.15.133:0/785377947 >> v2:172.21.15.162:3300/0 conn(0x7f020415ced0 0x7f020415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2051656174 0 0) 0x7f01ec003530 con 0x7f020406e4d0 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0204145170 msgr2=0x7f020406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 --2- 172.21.15.133:0/785377947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0204145170 0x7f020406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 >> v2:172.21.15.162:3300/0 conn(0x7f020415ced0 msgr2=0x7f020415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 --2- 172.21.15.133:0/785377947 >> v2:172.21.15.162:3300/0 conn(0x7f020415ced0 0x7f020415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020406d4b0 con 0x7f020406e4d0 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f01f8001680 con 0x7f020406e4d0 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f02037fe640 1 --2- 172.21.15.133:0/785377947 >> v2:172.21.15.162:3300/0 conn(0x7f020415ced0 0x7f020415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:04.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 >> v1:172.21.15.133:6789/0 conn(0x7f020406e4d0 legacy=0x7f02040cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 --2- 172.21.15.133:0/785377947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 0x7f01ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 --2- 172.21.15.133:0/785377947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0204145170 0x7f01ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 --2- 172.21.15.133:0/785377947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec004d50 0x7f01ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020406db00 con 0x7f01ec0039c0 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0204145550 con 0x7f01ec004d50 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec00b9d0 con 0x7f0204145170 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0202ffd640 1 --2- 172.21.15.133:0/785377947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 0x7f01ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0202ffd640 1 -- 172.21.15.133:0/785377947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0204145170 msgr2=0x7f01ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0202ffd640 1 --2- 172.21.15.133:0/785377947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0204145170 0x7f01ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0202ffd640 1 -- 172.21.15.133:0/785377947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec004d50 msgr2=0x7f01ec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0202ffd640 1 --2- 172.21.15.133:0/785377947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01ec004d50 0x7f01ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0202ffd640 1 -- 172.21.15.133:0/785377947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec003530 con 0x7f01ec0039c0 2024-04-25T10:12:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0202ffd640 1 --2- 172.21.15.133:0/785377947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 0x7f01ec003de0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009b10 tx=0x7f01f401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2418b8f315cc84d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40096b0 con 0x7f01ec0039c0 2024-04-25T10:12:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f40a6490 con 0x7f01ec0039c0 2024-04-25T10:12:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.573+0000 7f0201ffb640 1 -- 172.21.15.133:0/785377947 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40a6880 con 0x7f01ec0039c0 2024-04-25T10:12:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 -- 172.21.15.133:0/785377947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 msgr2=0x7f01ec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 --2- 172.21.15.133:0/785377947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 0x7f01ec003de0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009b10 tx=0x7f01f401cab0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02037fe640 1 -- 172.21.15.133:0/785377947 reap_dead start 2024-04-25T10:12:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 -- 172.21.15.133:0/785377947 shutdown_connections 2024-04-25T10:12:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 -- 172.21.15.133:0/785377947 >> 172.21.15.133:0/785377947 conn(0x7f0204066f70 msgr2=0x7f020406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:04.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 -- 172.21.15.133:0/785377947 shutdown_connections 2024-04-25T10:12:04.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 -- 172.21.15.133:0/785377947 wait complete. 2024-04-25T10:12:04.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 Processor -- start 2024-04-25T10:12:04.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 -- start start 2024-04-25T10:12:04.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 0x7f020413c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020413cd50 0x7f02040aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02041541e0 0x7f0204154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0204166600 con 0x7f01ec0039c0 2024-04-25T10:12:04.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0204166480 con 0x7f02041541e0 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0204166780 con 0x7f020413cd50 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f0202ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 0x7f020413c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f0202ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 0x7f020413c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33878/0 (socket says 172.21.15.133:33878) 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f0202ffd640 1 -- 172.21.15.133:0/2001891576 learned_addr learned my addr 172.21.15.133:0/2001891576 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02027fc640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020413cd50 0x7f02040aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f0202ffd640 1 -- 172.21.15.133:0/2001891576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020413cd50 msgr2=0x7f02040aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f0202ffd640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020413cd50 0x7f02040aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02037fe640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02041541e0 0x7f0204154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f0202ffd640 1 -- 172.21.15.133:0/2001891576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02041541e0 msgr2=0x7f0204154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f0202ffd640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02041541e0 0x7f0204154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f0202ffd640 1 -- 172.21.15.133:0/2001891576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02041550f0 con 0x7f01ec0039c0 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f0202ffd640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 0x7f020413c7a0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009b10 tx=0x7f01ec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f01ebfff640 1 -- 172.21.15.133:0/2001891576 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40be070 con 0x7f01ec0039c0 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f01ebfff640 1 -- 172.21.15.133:0/2001891576 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f40a6640 con 0x7f01ec0039c0 2024-04-25T10:12:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 -- 172.21.15.133:0/2001891576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02041553d0 con 0x7f01ec0039c0 2024-04-25T10:12:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f01ebfff640 1 -- 172.21.15.133:0/2001891576 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40096b0 con 0x7f01ec0039c0 2024-04-25T10:12:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.577+0000 7f02098e1640 1 -- 172.21.15.133:0/2001891576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02040b04c0 con 0x7f01ec0039c0 2024-04-25T10:12:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.581+0000 7f02098e1640 1 -- 172.21.15.133:0/2001891576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0204068e80 con 0x7f01ec0039c0 2024-04-25T10:12:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.581+0000 7f01ebfff640 1 -- 172.21.15.133:0/2001891576 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01f40c1070 con 0x7f01ec0039c0 2024-04-25T10:12:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.581+0000 7f01ebfff640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f0066f60 0x7f01f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.581+0000 7f01ebfff640 1 -- 172.21.15.133:0/2001891576 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f01f411ebf0 con 0x7f01ec0039c0 2024-04-25T10:12:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.581+0000 7f02027fc640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f0066f60 0x7f01f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.581+0000 7f02027fc640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f0066f60 0x7f01f0069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f01f8001ec0 tx=0x7f01f80016b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:04.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.585+0000 7f01ebfff640 1 -- 172.21.15.133:0/2001891576 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01f40ac020 con 0x7f01ec0039c0 2024-04-25T10:12:04.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.729+0000 7f02098e1640 1 -- 172.21.15.133:0/2001891576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0204143ec0 con 0x7f01ec0039c0 2024-04-25T10:12:04.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f01ebfff640 1 -- 172.21.15.133:0/2001891576 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01f4016800 con 0x7f01ec0039c0 2024-04-25T10:12:04.734 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:12:04.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 -- 172.21.15.133:0/2001891576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f0066f60 msgr2=0x7f01f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f0066f60 0x7f01f0069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f01f8001ec0 tx=0x7f01f80016b0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 -- 172.21.15.133:0/2001891576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 msgr2=0x7f020413c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 0x7f020413c7a0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009b10 tx=0x7f01ec005840 comp rx=0 tx=0).stop 2024-04-25T10:12:04.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 -- 172.21.15.133:0/2001891576 shutdown_connections 2024-04-25T10:12:04.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02041541e0 0x7f0204154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f020413cd50 0x7f02040aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01f0066f60 0x7f01f0069420 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 --2- 172.21.15.133:0/2001891576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ec0039c0 0x7f020413c7a0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 -- 172.21.15.133:0/2001891576 >> 172.21.15.133:0/2001891576 conn(0x7f0204066f70 msgr2=0x7f0204143040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:04.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 -- 172.21.15.133:0/2001891576 shutdown_connections 2024-04-25T10:12:04.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.733+0000 7f02098e1640 1 -- 172.21.15.133:0/2001891576 wait complete. 2024-04-25T10:12:04.752 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:04.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc4d856640 1 Processor -- start 2024-04-25T10:12:04.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc4d856640 1 -- start start 2024-04-25T10:12:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc4d856640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdc480b2150 0x7fdc4806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc4d856640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc4815b9b0 0x7fdc4806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc4d856640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4815ac50 con 0x7fdc48143ea0 2024-04-25T10:12:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc4d856640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc480ccf20 con 0x7fdc4815b9b0 2024-04-25T10:12:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc4d856640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc480cd180 con 0x7fdc480b2150 2024-04-25T10:12:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc477fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc4815b9b0 0x7fdc4806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc477fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc4815b9b0 0x7fdc4806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33894/0 (socket says 172.21.15.133:33894) 2024-04-25T10:12:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc477fe640 1 -- 172.21.15.133:0/212067345 learned_addr learned my addr 172.21.15.133:0/212067345 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc467fc640 1 --2- 172.21.15.133:0/212067345 >> v2:172.21.15.162:3300/0 conn(0x7fdc480b2150 0x7fdc4806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc477fe640 1 -- 172.21.15.133:0/212067345 >> v2:172.21.15.162:3300/0 conn(0x7fdc480b2150 msgr2=0x7fdc4806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:04.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc477fe640 1 --2- 172.21.15.133:0/212067345 >> v2:172.21.15.162:3300/0 conn(0x7fdc480b2150 0x7fdc4806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc477fe640 1 -- 172.21.15.133:0/212067345 >> v1:172.21.15.162:6789/0 conn(0x7fdc48143ea0 legacy=0x7fdc480cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.869+0000 7fdc477fe640 1 -- 172.21.15.133:0/212067345 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4815aed0 con 0x7fdc4815b9b0 2024-04-25T10:12:04.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc477fe640 1 --2- 172.21.15.133:0/212067345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc4815b9b0 0x7fdc4806e8f0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fdc480ce370 tx=0x7fdc3801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdbde80f06caf4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:04.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc45ffb640 1 -- 172.21.15.133:0/212067345 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380a5690 con 0x7fdc4815b9b0 2024-04-25T10:12:04.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc45ffb640 1 -- 172.21.15.133:0/212067345 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc380a5830 con 0x7fdc4815b9b0 2024-04-25T10:12:04.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc45ffb640 1 -- 172.21.15.133:0/212067345 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380a5c00 con 0x7fdc4815b9b0 2024-04-25T10:12:04.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 -- 172.21.15.133:0/212067345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc4815b9b0 msgr2=0x7fdc4806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 --2- 172.21.15.133:0/212067345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc4815b9b0 0x7fdc4806e8f0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fdc480ce370 tx=0x7fdc3801c960 comp rx=0 tx=0).stop 2024-04-25T10:12:04.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 -- 172.21.15.133:0/212067345 shutdown_connections 2024-04-25T10:12:04.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 --2- 172.21.15.133:0/212067345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc4815b9b0 0x7fdc4806e8f0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 --2- 172.21.15.133:0/212067345 >> v2:172.21.15.162:3300/0 conn(0x7fdc480b2150 0x7fdc4806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 -- 172.21.15.133:0/212067345 >> 172.21.15.133:0/212067345 conn(0x7fdc48066f70 msgr2=0x7fdc4806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:04.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 -- 172.21.15.133:0/212067345 shutdown_connections 2024-04-25T10:12:04.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 -- 172.21.15.133:0/212067345 wait complete. 2024-04-25T10:12:04.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 Processor -- start 2024-04-25T10:12:04.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 -- start start 2024-04-25T10:12:04.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc480b2150 0x7fdc48137220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc48143ea0 0x7fdc48165f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc4815b9b0 0x7fdc48165380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480b1270 con 0x7fdc480b2150 2024-04-25T10:12:04.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480b10f0 con 0x7fdc4815b9b0 2024-04-25T10:12:04.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc4d856640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480b13f0 con 0x7fdc48143ea0 2024-04-25T10:12:04.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc46ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc480b2150 0x7fdc48137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc46ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc480b2150 0x7fdc48137220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33908/0 (socket says 172.21.15.133:33908) 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc46ffd640 1 -- 172.21.15.133:0/3796200600 learned_addr learned my addr 172.21.15.133:0/3796200600 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc467fc640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc48143ea0 0x7fdc48165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc477fe640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc4815b9b0 0x7fdc48165380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc46ffd640 1 -- 172.21.15.133:0/3796200600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc48143ea0 msgr2=0x7fdc48165f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc46ffd640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc48143ea0 0x7fdc48165f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc46ffd640 1 -- 172.21.15.133:0/3796200600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc4815b9b0 msgr2=0x7fdc48165380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc46ffd640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc4815b9b0 0x7fdc48165380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc46ffd640 1 -- 172.21.15.133:0/3796200600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc481668c0 con 0x7fdc480b2150 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc477fe640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc4815b9b0 0x7fdc48165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc46ffd640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc480b2150 0x7fdc48137220 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c450 tx=0x7fdc4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.873+0000 7fdc2ffff640 1 -- 172.21.15.133:0/3796200600 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40016020 con 0x7fdc480b2150 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.877+0000 7fdc2ffff640 1 -- 172.21.15.133:0/3796200600 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc400052d0 con 0x7fdc480b2150 2024-04-25T10:12:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.877+0000 7fdc4d856640 1 -- 172.21.15.133:0/3796200600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc480b2ab0 con 0x7fdc480b2150 2024-04-25T10:12:04.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.877+0000 7fdc2ffff640 1 -- 172.21.15.133:0/3796200600 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40007260 con 0x7fdc480b2150 2024-04-25T10:12:04.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.877+0000 7fdc4d856640 1 -- 172.21.15.133:0/3796200600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc48166440 con 0x7fdc480b2150 2024-04-25T10:12:04.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.877+0000 7fdc4d856640 1 -- 172.21.15.133:0/3796200600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc48068e80 con 0x7fdc480b2150 2024-04-25T10:12:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.877+0000 7fdc2ffff640 1 -- 172.21.15.133:0/3796200600 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc40007400 con 0x7fdc480b2150 2024-04-25T10:12:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.877+0000 7fdc2ffff640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc34066f60 0x7fdc34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.877+0000 7fdc467fc640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc34066f60 0x7fdc34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.877+0000 7fdc2ffff640 1 -- 172.21.15.133:0/3796200600 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fdc40084af0 con 0x7fdc480b2150 2024-04-25T10:12:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.877+0000 7fdc467fc640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc34066f60 0x7fdc34069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdc48165110 tx=0x7fdc3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:04.881+0000 7fdc2ffff640 1 -- 172.21.15.133:0/3796200600 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc40020020 con 0x7fdc480b2150 2024-04-25T10:12:05.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 -- 172.21.15.133:0/3796200600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdc48164e60 con 0x7fdc480b2150 2024-04-25T10:12:05.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc2ffff640 1 -- 172.21.15.133:0/3796200600 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdc4815ac50 con 0x7fdc480b2150 2024-04-25T10:12:05.032 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:12:05.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 -- 172.21.15.133:0/3796200600 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc34066f60 msgr2=0x7fdc34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc34066f60 0x7fdc34069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdc48165110 tx=0x7fdc3c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:05.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 -- 172.21.15.133:0/3796200600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc480b2150 msgr2=0x7fdc48137220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc480b2150 0x7fdc48137220 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c450 tx=0x7fdc4000c910 comp rx=0 tx=0).stop 2024-04-25T10:12:05.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 -- 172.21.15.133:0/3796200600 shutdown_connections 2024-04-25T10:12:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc4815b9b0 0x7fdc48165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc48143ea0 0x7fdc48165f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc34066f60 0x7fdc34069420 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 --2- 172.21.15.133:0/3796200600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc480b2150 0x7fdc48137220 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 -- 172.21.15.133:0/3796200600 >> 172.21.15.133:0/3796200600 conn(0x7fdc48066f70 msgr2=0x7fdc4815c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 -- 172.21.15.133:0/3796200600 shutdown_connections 2024-04-25T10:12:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.029+0000 7fdc4d856640 1 -- 172.21.15.133:0/3796200600 wait complete. 2024-04-25T10:12:05.049 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:05.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfce3be640 1 Processor -- start 2024-04-25T10:12:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfce3be640 1 -- start start 2024-04-25T10:12:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfce3be640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbfc809a7c0 0x7fbfc815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfce3be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 0x7fbfc80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfce3be640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc80cc5a0 con 0x7fbfc815bab0 2024-04-25T10:12:05.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfce3be640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc80cc880 con 0x7fbfc809a7c0 2024-04-25T10:12:05.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfce3be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80ccb60 con 0x7fbfc809a0e0 2024-04-25T10:12:05.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfcc95b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 0x7fbfc80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfcc95b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 0x7fbfc80cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:37488/0 (socket says 172.21.15.133:37488) 2024-04-25T10:12:05.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfcc95b640 1 -- 172.21.15.133:0/1682583149 learned_addr learned my addr 172.21.15.133:0/1682583149 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:05.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfbffff640 1 --2- 172.21.15.133:0/1682583149 >> v2:172.21.15.162:3300/0 conn(0x7fbfc809a7c0 0x7fbfc815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfcc95b640 1 -- 172.21.15.133:0/1682583149 >> v2:172.21.15.162:3300/0 conn(0x7fbfc809a7c0 msgr2=0x7fbfc815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfcc95b640 1 --2- 172.21.15.133:0/1682583149 >> v2:172.21.15.162:3300/0 conn(0x7fbfc809a7c0 0x7fbfc815c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfcc95b640 1 -- 172.21.15.133:0/1682583149 >> v1:172.21.15.162:6789/0 conn(0x7fbfc815bab0 legacy=0x7fbfc80999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.173+0000 7fbfcc95b640 1 -- 172.21.15.133:0/1682583149 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80cce40 con 0x7fbfc809a0e0 2024-04-25T10:12:05.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfcc95b640 1 --2- 172.21.15.133:0/1682583149 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 0x7fbfc80cf280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009e50 tx=0x7fbfc001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=397463b82aacfb11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfc77fe640 1 -- 172.21.15.133:0/1682583149 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00a5660 con 0x7fbfc809a0e0 2024-04-25T10:12:05.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfc77fe640 1 -- 172.21.15.133:0/1682583149 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc00a5800 con 0x7fbfc809a0e0 2024-04-25T10:12:05.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfc77fe640 1 -- 172.21.15.133:0/1682583149 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00a5bd0 con 0x7fbfc809a0e0 2024-04-25T10:12:05.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfce3be640 1 -- 172.21.15.133:0/1682583149 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 msgr2=0x7fbfc80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfce3be640 1 --2- 172.21.15.133:0/1682583149 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 0x7fbfc80cf280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009e50 tx=0x7fbfc001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfce3be640 1 -- 172.21.15.133:0/1682583149 shutdown_connections 2024-04-25T10:12:05.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfce3be640 1 --2- 172.21.15.133:0/1682583149 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 0x7fbfc80cf280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfce3be640 1 --2- 172.21.15.133:0/1682583149 >> v2:172.21.15.162:3300/0 conn(0x7fbfc809a7c0 0x7fbfc815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfce3be640 1 -- 172.21.15.133:0/1682583149 >> 172.21.15.133:0/1682583149 conn(0x7fbfc80b31a0 msgr2=0x7fbfc80b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfce3be640 1 -- 172.21.15.133:0/1682583149 shutdown_connections 2024-04-25T10:12:05.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfce3be640 1 -- 172.21.15.133:0/1682583149 wait complete. 2024-04-25T10:12:05.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfce3be640 1 Processor -- start 2024-04-25T10:12:05.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.177+0000 7fbfce3be640 1 -- start start 2024-04-25T10:12:05.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfce3be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 0x7fbfc815a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfce3be640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfc809a7c0 0x7fbfc8155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfce3be640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc815bab0 0x7fbfc8155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfce3be640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80d0360 con 0x7fbfc815bab0 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfce3be640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80d01e0 con 0x7fbfc809a7c0 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfce3be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc81687d0 con 0x7fbfc809a0e0 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfcc95b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc815bab0 0x7fbfc8155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfcc95b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc815bab0 0x7fbfc8155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33918/0 (socket says 172.21.15.133:33918) 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfcc95b640 1 -- 172.21.15.133:0/2428463308 learned_addr learned my addr 172.21.15.133:0/2428463308 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfc7fff640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 0x7fbfc815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfbffff640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfc809a7c0 0x7fbfc8155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfcc95b640 1 -- 172.21.15.133:0/2428463308 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 msgr2=0x7fbfc815a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfcc95b640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 0x7fbfc815a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfcc95b640 1 -- 172.21.15.133:0/2428463308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfc809a7c0 msgr2=0x7fbfc8155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfcc95b640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfc809a7c0 0x7fbfc8155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfcc95b640 1 -- 172.21.15.133:0/2428463308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80c9ba0 con 0x7fbfc815bab0 2024-04-25T10:12:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfcc95b640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc815bab0 0x7fbfc8155690 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009e50 tx=0x7fbfc0019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfc57fa640 1 -- 172.21.15.133:0/2428463308 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00bc020 con 0x7fbfc815bab0 2024-04-25T10:12:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfc57fa640 1 -- 172.21.15.133:0/2428463308 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc00b8670 con 0x7fbfc815bab0 2024-04-25T10:12:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfce3be640 1 -- 172.21.15.133:0/2428463308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc80c9e80 con 0x7fbfc815bab0 2024-04-25T10:12:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfc57fa640 1 -- 172.21.15.133:0/2428463308 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0009510 con 0x7fbfc815bab0 2024-04-25T10:12:05.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfce3be640 1 -- 172.21.15.133:0/2428463308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc80ca440 con 0x7fbfc815bab0 2024-04-25T10:12:05.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfce3be640 1 -- 172.21.15.133:0/2428463308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc8068e80 con 0x7fbfc815bab0 2024-04-25T10:12:05.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfc57fa640 1 -- 172.21.15.133:0/2428463308 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfc00bf020 con 0x7fbfc815bab0 2024-04-25T10:12:05.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfc57fa640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfa8066f60 0x7fbfa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfc57fa640 1 -- 172.21.15.133:0/2428463308 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbfc01200e0 con 0x7fbfc815bab0 2024-04-25T10:12:05.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfc7fff640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfa8066f60 0x7fbfa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.181+0000 7fbfc7fff640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfa8066f60 0x7fbfa8069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8001650 tx=0x7fbfb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.185+0000 7fbfc57fa640 1 -- 172.21.15.133:0/2428463308 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfc00c6030 con 0x7fbfc815bab0 2024-04-25T10:12:05.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.345+0000 7fbfce3be640 1 -- 172.21.15.133:0/2428463308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbfc80b7630 con 0x7fbfc815bab0 2024-04-25T10:12:05.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.345+0000 7fbfc57fa640 1 -- 172.21.15.133:0/2428463308 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbfc0016620 con 0x7fbfc815bab0 2024-04-25T10:12:05.349 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 -- 172.21.15.133:0/2428463308 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfa8066f60 msgr2=0x7fbfa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfa8066f60 0x7fbfa8069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8001650 tx=0x7fbfb8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 -- 172.21.15.133:0/2428463308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc815bab0 msgr2=0x7fbfc8155690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc815bab0 0x7fbfc8155690 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009e50 tx=0x7fbfc0019740 comp rx=0 tx=0).stop 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 -- 172.21.15.133:0/2428463308 shutdown_connections 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbfa8066f60 0x7fbfa8069420 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfc815bab0 0x7fbfc8155690 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbfc809a7c0 0x7fbfc8155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 --2- 172.21.15.133:0/2428463308 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbfc809a0e0 0x7fbfc815a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 -- 172.21.15.133:0/2428463308 >> 172.21.15.133:0/2428463308 conn(0x7fbfc80b31a0 msgr2=0x7fbfc80b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 -- 172.21.15.133:0/2428463308 shutdown_connections 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.349+0000 7fbfce3be640 1 -- 172.21.15.133:0/2428463308 wait complete. 2024-04-25T10:12:05.368 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:05.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.481+0000 7fce8a167640 1 Processor -- start 2024-04-25T10:12:05.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.481+0000 7fce8a167640 1 -- start start 2024-04-25T10:12:05.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce8a167640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fce840ce330 0x7fce840ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce8a167640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce840c61d0 con 0x7fce8415c9a0 2024-04-25T10:12:05.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce8a167640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce840c64b0 con 0x7fce840b1f70 2024-04-25T10:12:05.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce8a167640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce840cecc0 con 0x7fce840ce330 2024-04-25T10:12:05.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce83fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fce840ce330 0x7fce840ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce83fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fce840ce330 0x7fce840ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33924/0 (socket says 172.21.15.133:33924) 2024-04-25T10:12:05.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce83fff640 1 -- 172.21.15.133:0/1235391907 learned_addr learned my addr 172.21.15.133:0/1235391907 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:05.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce837fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fce8415c9a0 0x7fce840c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48254/0 (socket says 172.21.15.133:48254) 2024-04-25T10:12:05.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce83fff640 1 -- 172.21.15.133:0/1235391907 >> v1:172.21.15.162:6789/0 conn(0x7fce840b1f70 legacy=0x7fce8406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce83fff640 1 -- 172.21.15.133:0/1235391907 >> v1:172.21.15.133:6789/0 conn(0x7fce8415c9a0 legacy=0x7fce840c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce83fff640 1 -- 172.21.15.133:0/1235391907 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8406d4f0 con 0x7fce840ce330 2024-04-25T10:12:05.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce82ffd640 1 -- 172.21.15.133:0/1235391907 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908422092 0 0) 0x7fce840c61d0 con 0x7fce8415c9a0 2024-04-25T10:12:05.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce83fff640 1 --2- 172.21.15.133:0/1235391907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce840ce330 0x7fce840ce780 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009e50 tx=0x7fce7c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1548f35a7ba08fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce82ffd640 1 -- 172.21.15.133:0/1235391907 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0a57d0 con 0x7fce840ce330 2024-04-25T10:12:05.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce82ffd640 1 -- 172.21.15.133:0/1235391907 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c0a5970 con 0x7fce840ce330 2024-04-25T10:12:05.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.485+0000 7fce82ffd640 1 -- 172.21.15.133:0/1235391907 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0a5d40 con 0x7fce840ce330 2024-04-25T10:12:05.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 -- 172.21.15.133:0/1235391907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce840ce330 msgr2=0x7fce840ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 --2- 172.21.15.133:0/1235391907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce840ce330 0x7fce840ce780 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009e50 tx=0x7fce7c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 -- 172.21.15.133:0/1235391907 shutdown_connections 2024-04-25T10:12:05.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 --2- 172.21.15.133:0/1235391907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce840ce330 0x7fce840ce780 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 -- 172.21.15.133:0/1235391907 >> 172.21.15.133:0/1235391907 conn(0x7fce84067000 msgr2=0x7fce8406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 -- 172.21.15.133:0/1235391907 shutdown_connections 2024-04-25T10:12:05.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 -- 172.21.15.133:0/1235391907 wait complete. 2024-04-25T10:12:05.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 Processor -- start 2024-04-25T10:12:05.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 -- start start 2024-04-25T10:12:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce840b1f70 0x7fce841488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce840ce330 0x7fce8414b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce8415c9a0 0x7fce8414e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce841695d0 con 0x7fce8415c9a0 2024-04-25T10:12:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8415dc20 con 0x7fce840ce330 2024-04-25T10:12:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce84169750 con 0x7fce840b1f70 2024-04-25T10:12:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce83fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce8415c9a0 0x7fce8414e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce83fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce8415c9a0 0x7fce8414e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33930/0 (socket says 172.21.15.133:33930) 2024-04-25T10:12:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce83fff640 1 -- 172.21.15.133:0/746029412 learned_addr learned my addr 172.21.15.133:0/746029412 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:05.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce837fe640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce840b1f70 0x7fce841488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce7affd640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce840ce330 0x7fce8414b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce83fff640 1 -- 172.21.15.133:0/746029412 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce840b1f70 msgr2=0x7fce841488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce83fff640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce840b1f70 0x7fce841488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce83fff640 1 -- 172.21.15.133:0/746029412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce840ce330 msgr2=0x7fce8414b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce83fff640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce840ce330 0x7fce8414b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce83fff640 1 -- 172.21.15.133:0/746029412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce840c7e80 con 0x7fce8415c9a0 2024-04-25T10:12:05.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce837fe640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce840b1f70 0x7fce841488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:05.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce7affd640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce840ce330 0x7fce8414b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:05.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce83fff640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce8415c9a0 0x7fce8414e0e0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0af040 tx=0x7fce7c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce80ff9640 1 -- 172.21.15.133:0/746029412 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0bc020 con 0x7fce8415c9a0 2024-04-25T10:12:05.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce80ff9640 1 -- 172.21.15.133:0/746029412 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c0b8690 con 0x7fce8415c9a0 2024-04-25T10:12:05.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 -- 172.21.15.133:0/746029412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce840c8160 con 0x7fce8415c9a0 2024-04-25T10:12:05.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce80ff9640 1 -- 172.21.15.133:0/746029412 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0b8a80 con 0x7fce8415c9a0 2024-04-25T10:12:05.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.489+0000 7fce8a167640 1 -- 172.21.15.133:0/746029412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce840c8720 con 0x7fce8415c9a0 2024-04-25T10:12:05.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.493+0000 7fce8a167640 1 -- 172.21.15.133:0/746029412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce84068ef0 con 0x7fce8415c9a0 2024-04-25T10:12:05.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.493+0000 7fce80ff9640 1 -- 172.21.15.133:0/746029412 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce7c0bf020 con 0x7fce8415c9a0 2024-04-25T10:12:05.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.493+0000 7fce80ff9640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce64066f60 0x7fce64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.493+0000 7fce80ff9640 1 -- 172.21.15.133:0/746029412 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fce7c120470 con 0x7fce8415c9a0 2024-04-25T10:12:05.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.493+0000 7fce837fe640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce64066f60 0x7fce64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.493+0000 7fce837fe640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce64066f60 0x7fce64069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fce74000f70 tx=0x7fce74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.497+0000 7fce80ff9640 1 -- 172.21.15.133:0/746029412 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce7c0c6030 con 0x7fce8415c9a0 2024-04-25T10:12:05.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.685+0000 7fce8a167640 1 -- 172.21.15.133:0/746029412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce840cbb50 con 0x7fce8415c9a0 2024-04-25T10:12:05.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.685+0000 7fce80ff9640 1 -- 172.21.15.133:0/746029412 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fce7c0165e0 con 0x7fce8415c9a0 2024-04-25T10:12:05.690 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:05.690 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:05.690 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:12:05.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 -- 172.21.15.133:0/746029412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce64066f60 msgr2=0x7fce64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce64066f60 0x7fce64069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fce74000f70 tx=0x7fce74009040 comp rx=0 tx=0).stop 2024-04-25T10:12:05.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 -- 172.21.15.133:0/746029412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce8415c9a0 msgr2=0x7fce8414e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce8415c9a0 0x7fce8414e0e0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0af040 tx=0x7fce7c019790 comp rx=0 tx=0).stop 2024-04-25T10:12:05.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 -- 172.21.15.133:0/746029412 shutdown_connections 2024-04-25T10:12:05.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce64066f60 0x7fce64069420 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce8415c9a0 0x7fce8414e0e0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce840ce330 0x7fce8414b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 --2- 172.21.15.133:0/746029412 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce840b1f70 0x7fce841488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 -- 172.21.15.133:0/746029412 >> 172.21.15.133:0/746029412 conn(0x7fce84067000 msgr2=0x7fce8406dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 -- 172.21.15.133:0/746029412 shutdown_connections 2024-04-25T10:12:05.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.689+0000 7fce8a167640 1 -- 172.21.15.133:0/746029412 wait complete. 2024-04-25T10:12:05.707 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:05.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.809+0000 7fcda4a3f640 1 Processor -- start 2024-04-25T10:12:05.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.809+0000 7fcda4a3f640 1 -- start start 2024-04-25T10:12:05.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcda4a3f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcda015c9a0 0x7fcda00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcda4a3f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcda00b1f70 0x7fcda006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcda4a3f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda00ce330 0x7fcda00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcda4a3f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda00cecc0 con 0x7fcda015c9a0 2024-04-25T10:12:05.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcda4a3f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda00c61d0 con 0x7fcda00b1f70 2024-04-25T10:12:05.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcda4a3f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00c64b0 con 0x7fcda00ce330 2024-04-25T10:12:05.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcd9e59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcda015c9a0 0x7fcda00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcd9e59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcda015c9a0 0x7fcda00c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33942/0 (socket says 172.21.15.133:33942) 2024-04-25T10:12:05.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcd9e59c640 1 -- 172.21.15.133:0/2660627446 learned_addr learned my addr 172.21.15.133:0/2660627446 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:05.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcd9ed9d640 1 --2- 172.21.15.133:0/2660627446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda00ce330 0x7fcda00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcd9dd9b640 1 --2- 172.21.15.133:0/2660627446 >> v2:172.21.15.162:3300/0 conn(0x7fcda00b1f70 0x7fcda006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcd9e59c640 1 -- 172.21.15.133:0/2660627446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda00ce330 msgr2=0x7fcda00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcd9e59c640 1 --2- 172.21.15.133:0/2660627446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda00ce330 0x7fcda00ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcd9e59c640 1 -- 172.21.15.133:0/2660627446 >> v2:172.21.15.162:3300/0 conn(0x7fcda00b1f70 msgr2=0x7fcda006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcd9e59c640 1 --2- 172.21.15.133:0/2660627446 >> v2:172.21.15.162:3300/0 conn(0x7fcda00b1f70 0x7fcda006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcd9e59c640 1 -- 172.21.15.133:0/2660627446 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda006d4f0 con 0x7fcda015c9a0 2024-04-25T10:12:05.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.813+0000 7fcd9ed9d640 1 --2- 172.21.15.133:0/2660627446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda00ce330 0x7fcda00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:05.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcd9e59c640 1 --2- 172.21.15.133:0/2660627446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda015c9a0 0x7fcda00c5980 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fcda0072670 tx=0x7fcd9001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622484670e50a7b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcd9d59a640 1 -- 172.21.15.133:0/2660627446 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd900b1070 con 0x7fcda015c9a0 2024-04-25T10:12:05.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcd9d59a640 1 -- 172.21.15.133:0/2660627446 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd900ac620 con 0x7fcda015c9a0 2024-04-25T10:12:05.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcd9d59a640 1 -- 172.21.15.133:0/2660627446 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd900ac9f0 con 0x7fcda015c9a0 2024-04-25T10:12:05.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2660627446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda015c9a0 msgr2=0x7fcda00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcda4a3f640 1 --2- 172.21.15.133:0/2660627446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda015c9a0 0x7fcda00c5980 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fcda0072670 tx=0x7fcd9001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2660627446 shutdown_connections 2024-04-25T10:12:05.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcda4a3f640 1 --2- 172.21.15.133:0/2660627446 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda00ce330 0x7fcda00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcda4a3f640 1 --2- 172.21.15.133:0/2660627446 >> v2:172.21.15.162:3300/0 conn(0x7fcda00b1f70 0x7fcda006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcda4a3f640 1 --2- 172.21.15.133:0/2660627446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda015c9a0 0x7fcda00c5980 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2660627446 >> 172.21.15.133:0/2660627446 conn(0x7fcda0067000 msgr2=0x7fcda006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2660627446 shutdown_connections 2024-04-25T10:12:05.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.817+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2660627446 wait complete. 2024-04-25T10:12:05.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcda4a3f640 1 Processor -- start 2024-04-25T10:12:05.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcda4a3f640 1 -- start start 2024-04-25T10:12:05.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcda4a3f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda00b1f70 0x7fcda00c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcda4a3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda00ce330 0x7fcda00ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcda4a3f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda015c9a0 0x7fcda00b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcda4a3f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda015cf00 con 0x7fcda00ce330 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcda4a3f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda015cd80 con 0x7fcda015c9a0 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcda4a3f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda015d950 con 0x7fcda00b1f70 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda00ce330 0x7fcda00ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda00ce330 0x7fcda00ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33958/0 (socket says 172.21.15.133:33958) 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9dd9b640 1 -- 172.21.15.133:0/2129900101 learned_addr learned my addr 172.21.15.133:0/2129900101 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9e59c640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda00b1f70 0x7fcda00c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9ed9d640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda015c9a0 0x7fcda00b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9dd9b640 1 -- 172.21.15.133:0/2129900101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda00b1f70 msgr2=0x7fcda00c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9dd9b640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda00b1f70 0x7fcda00c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9dd9b640 1 -- 172.21.15.133:0/2129900101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda015c9a0 msgr2=0x7fcda00b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9dd9b640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda015c9a0 0x7fcda00b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9dd9b640 1 -- 172.21.15.133:0/2129900101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda0155fc0 con 0x7fcda00ce330 2024-04-25T10:12:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9ed9d640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda015c9a0 0x7fcda00b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9dd9b640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda00ce330 0x7fcda00ca4b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800d800 tx=0x7fcd8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd96ffd640 1 -- 172.21.15.133:0/2129900101 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88018020 con 0x7fcda00ce330 2024-04-25T10:12:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd96ffd640 1 -- 172.21.15.133:0/2129900101 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd880096a0 con 0x7fcda00ce330 2024-04-25T10:12:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd96ffd640 1 -- 172.21.15.133:0/2129900101 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88009a50 con 0x7fcda00ce330 2024-04-25T10:12:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2129900101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda00c80e0 con 0x7fcda00ce330 2024-04-25T10:12:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2129900101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda00c86a0 con 0x7fcda00ce330 2024-04-25T10:12:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2129900101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda006bc20 con 0x7fcda00ce330 2024-04-25T10:12:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd96ffd640 1 -- 172.21.15.133:0/2129900101 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd8801b020 con 0x7fcda00ce330 2024-04-25T10:12:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd96ffd640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd80066f90 0x7fcd80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd96ffd640 1 -- 172.21.15.133:0/2129900101 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcd88085870 con 0x7fcda00ce330 2024-04-25T10:12:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.821+0000 7fcd9e59c640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd80066f90 0x7fcd80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.825+0000 7fcd9e59c640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd80066f90 0x7fcd80069450 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcd900ae000 tx=0x7fcd90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.825+0000 7fcd96ffd640 1 -- 172.21.15.133:0/2129900101 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd88022020 con 0x7fcda00ce330 2024-04-25T10:12:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2129900101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcda0068ef0 con 0x7fcda00ce330 2024-04-25T10:12:05.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcd96ffd640 1 -- 172.21.15.133:0/2129900101 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17005 (secure 0 0 0) 0x7fcd880565b0 con 0x7fcda00ce330 2024-04-25T10:12:05.980 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:05.980 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":109,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:11:34.158734+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:09.673185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:10.421023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":5,"to":4}]},{"pgid":"16.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2129900101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd80066f90 msgr2=0x7fcd80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd80066f90 0x7fcd80069450 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcd900ae000 tx=0x7fcd90008040 comp rx=0 tx=0).stop 2024-04-25T10:12:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2129900101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda00ce330 msgr2=0x7fcda00ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda00ce330 0x7fcda00ca4b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800d800 tx=0x7fcd8800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2129900101 shutdown_connections 2024-04-25T10:12:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcda015c9a0 0x7fcda00b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcd80066f90 0x7fcd80069450 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda00ce330 0x7fcda00ca4b0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 --2- 172.21.15.133:0/2129900101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcda00b1f70 0x7fcda00c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2129900101 >> 172.21.15.133:0/2129900101 conn(0x7fcda0067000 msgr2=0x7fcda01567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2129900101 shutdown_connections 2024-04-25T10:12:05.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:05.977+0000 7fcda4a3f640 1 -- 172.21.15.133:0/2129900101 wait complete. 2024-04-25T10:12:05.998 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:12:05.998 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:06.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0eb0a9640 1 Processor -- start 2024-04-25T10:12:06.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0eb0a9640 1 -- start start 2024-04-25T10:12:06.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0eb0a9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e415bb00 0x7ff0e40cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0eb0a9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0e4066df0 0x7ff0e406dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0eb0a9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e40cba80 con 0x7ff0e406f5b0 2024-04-25T10:12:06.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0eb0a9640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e4067250 con 0x7ff0e4066df0 2024-04-25T10:12:06.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0eb0a9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40b5d00 con 0x7ff0e415bb00 2024-04-25T10:12:06.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0e3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0e4066df0 0x7ff0e406dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0e3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0e4066df0 0x7ff0e406dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33960/0 (socket says 172.21.15.133:33960) 2024-04-25T10:12:06.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0e3fff640 1 -- 172.21.15.133:0/704739624 learned_addr learned my addr 172.21.15.133:0/704739624 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:06.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0e8e45640 1 --2- 172.21.15.133:0/704739624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e415bb00 0x7ff0e40cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0e3fff640 1 -- 172.21.15.133:0/704739624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e415bb00 msgr2=0x7ff0e40cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0e3fff640 1 --2- 172.21.15.133:0/704739624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e415bb00 0x7ff0e40cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0e3fff640 1 -- 172.21.15.133:0/704739624 >> v1:172.21.15.162:6789/0 conn(0x7ff0e406f5b0 legacy=0x7ff0e406fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0e3fff640 1 -- 172.21.15.133:0/704739624 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40cb7a0 con 0x7ff0e4066df0 2024-04-25T10:12:06.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.113+0000 7ff0e3fff640 1 --2- 172.21.15.133:0/704739624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e4066df0 0x7ff0e406dbc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009870 tx=0x7ff0d001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e9026f119be666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e37fe640 1 -- 172.21.15.133:0/704739624 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00b2070 con 0x7ff0e4066df0 2024-04-25T10:12:06.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e37fe640 1 -- 172.21.15.133:0/704739624 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d00ad640 con 0x7ff0e4066df0 2024-04-25T10:12:06.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e37fe640 1 -- 172.21.15.133:0/704739624 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00ada10 con 0x7ff0e4066df0 2024-04-25T10:12:06.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/704739624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e4066df0 msgr2=0x7ff0e406dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 --2- 172.21.15.133:0/704739624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e4066df0 0x7ff0e406dbc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009870 tx=0x7ff0d001c3d0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/704739624 shutdown_connections 2024-04-25T10:12:06.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 --2- 172.21.15.133:0/704739624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e4066df0 0x7ff0e406dbc0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 --2- 172.21.15.133:0/704739624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e415bb00 0x7ff0e40cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/704739624 >> 172.21.15.133:0/704739624 conn(0x7ff0e40cc8e0 msgr2=0x7ff0e40cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/704739624 shutdown_connections 2024-04-25T10:12:06.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/704739624 wait complete. 2024-04-25T10:12:06.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 Processor -- start 2024-04-25T10:12:06.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 -- start start 2024-04-25T10:12:06.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e4066df0 0x7ff0e4154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0e406f5b0 0x7ff0e40cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e415bb00 0x7ff0e41649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e406c030 con 0x7ff0e4066df0 2024-04-25T10:12:06.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e406beb0 con 0x7ff0e406f5b0 2024-04-25T10:12:06.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e406c1b0 con 0x7ff0e415bb00 2024-04-25T10:12:06.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e8e45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e4066df0 0x7ff0e4154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e8e45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e4066df0 0x7ff0e4154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33976/0 (socket says 172.21.15.133:33976) 2024-04-25T10:12:06.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e8e45640 1 -- 172.21.15.133:0/311204584 learned_addr learned my addr 172.21.15.133:0/311204584 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:06.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e9646640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e415bb00 0x7ff0e41649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e8e45640 1 -- 172.21.15.133:0/311204584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e415bb00 msgr2=0x7ff0e41649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e3fff640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0e406f5b0 0x7ff0e40cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e8e45640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e415bb00 0x7ff0e41649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e8e45640 1 -- 172.21.15.133:0/311204584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0e406f5b0 msgr2=0x7ff0e40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e8e45640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0e406f5b0 0x7ff0e40cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e8e45640 1 -- 172.21.15.133:0/311204584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40ce920 con 0x7ff0e4066df0 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e9646640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e415bb00 0x7ff0e41649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e8e45640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e4066df0 0x7ff0e4154f70 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800d5f0 tx=0x7ff0d800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e17fa640 1 -- 172.21.15.133:0/311204584 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80125f0 con 0x7ff0e4066df0 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e17fa640 1 -- 172.21.15.133:0/311204584 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d8019070 con 0x7ff0e4066df0 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/311204584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e40cec00 con 0x7ff0e4066df0 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0e17fa640 1 -- 172.21.15.133:0/311204584 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80129b0 con 0x7ff0e4066df0 2024-04-25T10:12:06.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.117+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/311204584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e40c73b0 con 0x7ff0e4066df0 2024-04-25T10:12:06.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.121+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/311204584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e4067c70 con 0x7ff0e4066df0 2024-04-25T10:12:06.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.121+0000 7ff0e17fa640 1 -- 172.21.15.133:0/311204584 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0d8012b50 con 0x7ff0e4066df0 2024-04-25T10:12:06.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.121+0000 7ff0e17fa640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0c8066f60 0x7ff0c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.121+0000 7ff0e17fa640 1 -- 172.21.15.133:0/311204584 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff0d8086060 con 0x7ff0e4066df0 2024-04-25T10:12:06.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.121+0000 7ff0e3fff640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0c8066f60 0x7ff0c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.121+0000 7ff0e3fff640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0c8066f60 0x7ff0c8069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00af000 tx=0x7ff0d00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.125+0000 7ff0e17fa640 1 -- 172.21.15.133:0/311204584 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0d8056f90 con 0x7ff0e4066df0 2024-04-25T10:12:06.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.313+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/311204584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0e4072830 con 0x7ff0e4066df0 2024-04-25T10:12:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0e17fa640 1 -- 172.21.15.133:0/311204584 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff0d8056890 con 0x7ff0e4066df0 2024-04-25T10:12:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:06.319 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:12:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/311204584 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0c8066f60 msgr2=0x7ff0c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0c8066f60 0x7ff0c8069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00af000 tx=0x7ff0d00a7070 comp rx=0 tx=0).stop 2024-04-25T10:12:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/311204584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e4066df0 msgr2=0x7ff0e4154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e4066df0 0x7ff0e4154f70 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800d5f0 tx=0x7ff0d800dab0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/311204584 shutdown_connections 2024-04-25T10:12:06.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e415bb00 0x7ff0e41649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0e406f5b0 0x7ff0e40cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0c8066f60 0x7ff0c8069420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 --2- 172.21.15.133:0/311204584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e4066df0 0x7ff0e4154f70 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/311204584 >> 172.21.15.133:0/311204584 conn(0x7ff0e40cc8e0 msgr2=0x7ff0e415c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/311204584 shutdown_connections 2024-04-25T10:12:06.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.317+0000 7ff0eb0a9640 1 -- 172.21.15.133:0/311204584 wait complete. 2024-04-25T10:12:06.337 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:06.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.445+0000 7fedbff84640 1 Processor -- start 2024-04-25T10:12:06.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.445+0000 7fedbff84640 1 -- start start 2024-04-25T10:12:06.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbff84640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fedb8158110 0x7fedb80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbff84640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fedb806eb70 0x7fedb815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbff84640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb80cbe60 con 0x7fedb81584f0 2024-04-25T10:12:06.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbff84640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb80b19b0 con 0x7fedb806eb70 2024-04-25T10:12:06.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbff84640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb815b360 con 0x7fedb8158110 2024-04-25T10:12:06.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbe521640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fedb806eb70 0x7fedb815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbe521640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fedb806eb70 0x7fedb815ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32804/0 (socket says 172.21.15.133:32804) 2024-04-25T10:12:06.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbe521640 1 -- 172.21.15.133:0/2697767758 learned_addr learned my addr 172.21.15.133:0/2697767758 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:06.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbdd20640 1 --2- 172.21.15.133:0/2697767758 >> v2:172.21.15.162:3300/0 conn(0x7fedb8158110 0x7fedb80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbe521640 1 -- 172.21.15.133:0/2697767758 >> v2:172.21.15.162:3300/0 conn(0x7fedb8158110 msgr2=0x7fedb80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbe521640 1 --2- 172.21.15.133:0/2697767758 >> v2:172.21.15.162:3300/0 conn(0x7fedb8158110 0x7fedb80b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbe521640 1 -- 172.21.15.133:0/2697767758 >> v1:172.21.15.162:6789/0 conn(0x7fedb81584f0 legacy=0x7fedb806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbe521640 1 -- 172.21.15.133:0/2697767758 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb815b640 con 0x7fedb806eb70 2024-04-25T10:12:06.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbe521640 1 --2- 172.21.15.133:0/2697767758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedb806eb70 0x7fedb815ad10 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fedb80b4d40 tx=0x7fedb001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388fb29261eb532a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbcd1e640 1 -- 172.21.15.133:0/2697767758 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00a5860 con 0x7fedb806eb70 2024-04-25T10:12:06.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbcd1e640 1 -- 172.21.15.133:0/2697767758 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb00a5a00 con 0x7fedb806eb70 2024-04-25T10:12:06.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbcd1e640 1 -- 172.21.15.133:0/2697767758 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00a5dd0 con 0x7fedb806eb70 2024-04-25T10:12:06.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbff84640 1 -- 172.21.15.133:0/2697767758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedb806eb70 msgr2=0x7fedb815ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.449+0000 7fedbff84640 1 --2- 172.21.15.133:0/2697767758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedb806eb70 0x7fedb815ad10 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fedb80b4d40 tx=0x7fedb001c980 comp rx=0 tx=0).stop 2024-04-25T10:12:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 -- 172.21.15.133:0/2697767758 shutdown_connections 2024-04-25T10:12:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 --2- 172.21.15.133:0/2697767758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedb806eb70 0x7fedb815ad10 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 --2- 172.21.15.133:0/2697767758 >> v2:172.21.15.162:3300/0 conn(0x7fedb8158110 0x7fedb80b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 -- 172.21.15.133:0/2697767758 >> 172.21.15.133:0/2697767758 conn(0x7fedb8067000 msgr2=0x7fedb806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 -- 172.21.15.133:0/2697767758 shutdown_connections 2024-04-25T10:12:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 -- 172.21.15.133:0/2697767758 wait complete. 2024-04-25T10:12:06.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 Processor -- start 2024-04-25T10:12:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 -- start start 2024-04-25T10:12:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedb806eb70 0x7fedb814b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fedb8158110 0x7fedb8152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedb81584f0 0x7fedb814d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80ccca0 con 0x7fedb81584f0 2024-04-25T10:12:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80ccb20 con 0x7fedb806eb70 2024-04-25T10:12:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80cce20 con 0x7fedb8158110 2024-04-25T10:12:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbe521640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedb81584f0 0x7fedb814d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbe521640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedb81584f0 0x7fedb814d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32806/0 (socket says 172.21.15.133:32806) 2024-04-25T10:12:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbe521640 1 -- 172.21.15.133:0/3177742763 learned_addr learned my addr 172.21.15.133:0/3177742763 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbd51f640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fedb8158110 0x7fedb8152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbdd20640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedb806eb70 0x7fedb814b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbe521640 1 -- 172.21.15.133:0/3177742763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fedb8158110 msgr2=0x7fedb8152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbe521640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fedb8158110 0x7fedb8152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbe521640 1 -- 172.21.15.133:0/3177742763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedb806eb70 msgr2=0x7fedb814b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbe521640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedb806eb70 0x7fedb814b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbe521640 1 -- 172.21.15.133:0/3177742763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8151aa0 con 0x7fedb81584f0 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbd51f640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fedb8158110 0x7fedb8152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbe521640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedb81584f0 0x7fedb814d620 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fedb00af040 tx=0x7fedb0019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedb67fc640 1 -- 172.21.15.133:0/3177742763 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00b8880 con 0x7fedb81584f0 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedb67fc640 1 -- 172.21.15.133:0/3177742763 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb00be070 con 0x7fedb81584f0 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 -- 172.21.15.133:0/3177742763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb8151d80 con 0x7fedb81584f0 2024-04-25T10:12:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedb67fc640 1 -- 172.21.15.133:0/3177742763 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00b8c80 con 0x7fedb81584f0 2024-04-25T10:12:06.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 -- 172.21.15.133:0/3177742763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb8152340 con 0x7fedb81584f0 2024-04-25T10:12:06.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.453+0000 7fedbff84640 1 -- 172.21.15.133:0/3177742763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb806bc20 con 0x7fedb81584f0 2024-04-25T10:12:06.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.457+0000 7fedb67fc640 1 -- 172.21.15.133:0/3177742763 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fedb00c3460 con 0x7fedb81584f0 2024-04-25T10:12:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.457+0000 7fedb67fc640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feda4066f60 0x7feda4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.457+0000 7fedb67fc640 1 -- 172.21.15.133:0/3177742763 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fedb0120580 con 0x7fedb81584f0 2024-04-25T10:12:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.457+0000 7fedbdd20640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feda4066f60 0x7feda4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.457+0000 7fedbdd20640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feda4066f60 0x7feda4069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feda800b800 tx=0x7feda8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.461+0000 7fedb67fc640 1 -- 172.21.15.133:0/3177742763 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fedb00165a0 con 0x7fedb81584f0 2024-04-25T10:12:06.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.641+0000 7fedbff84640 1 -- 172.21.15.133:0/3177742763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedb8068ef0 con 0x7fedb81584f0 2024-04-25T10:12:06.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedb67fc640 1 -- 172.21.15.133:0/3177742763 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fedb00ac3a0 con 0x7fedb81584f0 2024-04-25T10:12:06.646 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:06.646 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:11.432440+0000","modified":"2024-04-25T10:11:13.428977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8313},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8313":{"gid":8313,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6834/488739264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":488739264},{"type":"v1","addr":"172.21.15.162:6834","nonce":488739264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:06.647 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 65 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 -- 172.21.15.133:0/3177742763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feda4066f60 msgr2=0x7feda4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feda4066f60 0x7feda4069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feda800b800 tx=0x7feda8008040 comp rx=0 tx=0).stop 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 -- 172.21.15.133:0/3177742763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedb81584f0 msgr2=0x7fedb814d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedb81584f0 0x7fedb814d620 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fedb00af040 tx=0x7fedb0019720 comp rx=0 tx=0).stop 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 -- 172.21.15.133:0/3177742763 shutdown_connections 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feda4066f60 0x7feda4069420 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedb81584f0 0x7fedb814d620 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fedb8158110 0x7fedb8152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 --2- 172.21.15.133:0/3177742763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedb806eb70 0x7fedb814b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 -- 172.21.15.133:0/3177742763 >> 172.21.15.133:0/3177742763 conn(0x7fedb8067000 msgr2=0x7fedb80ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 -- 172.21.15.133:0/3177742763 shutdown_connections 2024-04-25T10:12:06.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.645+0000 7fedbff84640 1 -- 172.21.15.133:0/3177742763 wait complete. 2024-04-25T10:12:06.664 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:06.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.777+0000 7f664711f640 1 Processor -- start 2024-04-25T10:12:06.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.777+0000 7f664711f640 1 -- start start 2024-04-25T10:12:06.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f664711f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f664015b9b0 0x7f66400cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f664711f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400cf7c0 0x7f66400cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f664711f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664015bd90 con 0x7f66400d01b0 2024-04-25T10:12:06.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f664711f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66400c5610 con 0x7f664015b9b0 2024-04-25T10:12:06.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f664711f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664009ad10 con 0x7f66400cf7c0 2024-04-25T10:12:06.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f66456bc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f66400d01b0 0x7f66400cc1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43250/0 (socket says 172.21.15.133:43250) 2024-04-25T10:12:06.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f66456bc640 1 -- 172.21.15.133:0/279740462 learned_addr learned my addr 172.21.15.133:0/279740462 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:06.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663ffff640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400cf7c0 0x7f66400cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679092566 0 0) 0x7f664015bd90 con 0x7f66400d01b0 2024-04-25T10:12:06.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f6644ebb640 1 --2- 172.21.15.133:0/279740462 >> v2:172.21.15.162:3300/0 conn(0x7f664015b9b0 0x7f66400cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6628003530 con 0x7f66400d01b0 2024-04-25T10:12:06.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3688073509 0 0) 0x7f6628003530 con 0x7f66400d01b0 2024-04-25T10:12:06.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400cf7c0 msgr2=0x7f66400cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400cf7c0 0x7f66400cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 >> v2:172.21.15.162:3300/0 conn(0x7f664015b9b0 msgr2=0x7f66400cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 --2- 172.21.15.133:0/279740462 >> v2:172.21.15.162:3300/0 conn(0x7f664015b9b0 0x7f66400cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664009a750 con 0x7f66400d01b0 2024-04-25T10:12:06.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f6638001a00 con 0x7f66400d01b0 2024-04-25T10:12:06.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 >> v1:172.21.15.133:6789/0 conn(0x7f66400d01b0 legacy=0x7f66400cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400cf7c0 0x7f66280039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6628003f40 0x7f6628004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6628004d50 0x7f66280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66400c5610 con 0x7f6628003f40 2024-04-25T10:12:06.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664009ad10 con 0x7f6628004d50 2024-04-25T10:12:06.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662800b9d0 con 0x7f66400cf7c0 2024-04-25T10:12:06.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663ffff640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6628003f40 0x7f6628004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f6644ebb640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400cf7c0 0x7f66280039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663ffff640 1 -- 172.21.15.133:0/279740462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400cf7c0 msgr2=0x7f66280039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f66456bc640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6628004d50 0x7f66280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663ffff640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400cf7c0 0x7f66280039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663ffff640 1 -- 172.21.15.133:0/279740462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6628004d50 msgr2=0x7f66280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:06.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663ffff640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6628004d50 0x7f66280051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663ffff640 1 -- 172.21.15.133:0/279740462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628003530 con 0x7f6628003f40 2024-04-25T10:12:06.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663ffff640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6628003f40 0x7f6628004810 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f6634009ae0 tx=0x7f663401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a5a765928d024c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66340b2070 con 0x7f6628003f40 2024-04-25T10:12:06.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66340a5660 con 0x7f6628003f40 2024-04-25T10:12:06.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.781+0000 7f663f7fe640 1 -- 172.21.15.133:0/279740462 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66340096b0 con 0x7f6628003f40 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 -- 172.21.15.133:0/279740462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6628003f40 msgr2=0x7f6628004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 --2- 172.21.15.133:0/279740462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6628003f40 0x7f6628004810 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f6634009ae0 tx=0x7f663401ca10 comp rx=0 tx=0).stop 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f66456bc640 1 -- 172.21.15.133:0/279740462 reap_dead start 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 -- 172.21.15.133:0/279740462 shutdown_connections 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 -- 172.21.15.133:0/279740462 >> 172.21.15.133:0/279740462 conn(0x7f66400b31a0 msgr2=0x7f66400b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 -- 172.21.15.133:0/279740462 shutdown_connections 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 -- 172.21.15.133:0/279740462 wait complete. 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 Processor -- start 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 -- start start 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6628003f40 0x7f66401f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66400cb000 0x7f6640158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400c6e40 0x7f66400c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664009a3e0 con 0x7f66400cb000 2024-04-25T10:12:06.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66400c52a0 con 0x7f6628003f40 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66400c6ab0 con 0x7f66400c6e40 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66400cb000 0x7f6640158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66400cb000 0x7f6640158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32826/0 (socket says 172.21.15.133:32826) 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663ffff640 1 -- 172.21.15.133:0/2453437444 learned_addr learned my addr 172.21.15.133:0/2453437444 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f6644ebb640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6628003f40 0x7f66401f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f66456bc640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400c6e40 0x7f66400c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663ffff640 1 -- 172.21.15.133:0/2453437444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400c6e40 msgr2=0x7f66400c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663ffff640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400c6e40 0x7f66400c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663ffff640 1 -- 172.21.15.133:0/2453437444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6628003f40 msgr2=0x7f66401f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663ffff640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6628003f40 0x7f66401f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663ffff640 1 -- 172.21.15.133:0/2453437444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66400c90d0 con 0x7f66400cb000 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663ffff640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66400cb000 0x7f6640158a10 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f663401c840 tx=0x7f6628005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663d7fa640 1 -- 172.21.15.133:0/2453437444 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66340b2040 con 0x7f66400cb000 2024-04-25T10:12:06.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663d7fa640 1 -- 172.21.15.133:0/2453437444 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663400f000 con 0x7f66400cb000 2024-04-25T10:12:06.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 -- 172.21.15.133:0/2453437444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66401f1e20 con 0x7f66400cb000 2024-04-25T10:12:06.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f663d7fa640 1 -- 172.21.15.133:0/2453437444 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634004020 con 0x7f66400cb000 2024-04-25T10:12:06.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.785+0000 7f664711f640 1 -- 172.21.15.133:0/2453437444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66401f23e0 con 0x7f66400cb000 2024-04-25T10:12:06.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.789+0000 7f664711f640 1 -- 172.21.15.133:0/2453437444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6640067c70 con 0x7f66400cb000 2024-04-25T10:12:06.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.789+0000 7f663d7fa640 1 -- 172.21.15.133:0/2453437444 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66340c0030 con 0x7f66400cb000 2024-04-25T10:12:06.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.789+0000 7f663d7fa640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f662c066f60 0x7f662c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:06.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.789+0000 7f663d7fa640 1 -- 172.21.15.133:0/2453437444 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f663411eef0 con 0x7f66400cb000 2024-04-25T10:12:06.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.789+0000 7f6644ebb640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f662c066f60 0x7f662c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:06.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.789+0000 7f6644ebb640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f662c066f60 0x7f662c069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6630009ab0 tx=0x7f6630008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:06.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.793+0000 7f663d7fa640 1 -- 172.21.15.133:0/2453437444 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f66340c7020 con 0x7f66400cb000 2024-04-25T10:12:06.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.937+0000 7f664711f640 1 -- 172.21.15.133:0/2453437444 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66400c7800 con 0x7f66400cb000 2024-04-25T10:12:06.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f663d7fa640 1 -- 172.21.15.133:0/2453437444 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17005 (secure 0 0 0) 0x7f6634016780 con 0x7f66400cb000 2024-04-25T10:12:06.943 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:06.943 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":109,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:11:34.158734+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:09.673185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:10.421023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":5,"to":4}]},{"pgid":"16.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 -- 172.21.15.133:0/2453437444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f662c066f60 msgr2=0x7f662c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f662c066f60 0x7f662c069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6630009ab0 tx=0x7f6630008040 comp rx=0 tx=0).stop 2024-04-25T10:12:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 -- 172.21.15.133:0/2453437444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66400cb000 msgr2=0x7f6640158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66400cb000 0x7f6640158a10 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f663401c840 tx=0x7f6628005840 comp rx=0 tx=0).stop 2024-04-25T10:12:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 -- 172.21.15.133:0/2453437444 shutdown_connections 2024-04-25T10:12:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66400c6e40 0x7f66400c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f662c066f60 0x7f662c069420 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66400cb000 0x7f6640158a10 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 --2- 172.21.15.133:0/2453437444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6628003f40 0x7f66401f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:06.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 -- 172.21.15.133:0/2453437444 >> 172.21.15.133:0/2453437444 conn(0x7f66400b31a0 msgr2=0x7f664006f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:06.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 -- 172.21.15.133:0/2453437444 shutdown_connections 2024-04-25T10:12:06.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:06.941+0000 7f664711f640 1 -- 172.21.15.133:0/2453437444 wait complete. 2024-04-25T10:12:06.961 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:12:07.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1acdd8c640 1 Processor -- start 2024-04-25T10:12:07.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1acdd8c640 1 -- start start 2024-04-25T10:12:07.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1acdd8c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1ac80cfc80 0x7f1ac80d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1acdd8c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1ac80cc3b0 0x7f1ac80cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1acdd8c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac80b1560 con 0x7f1ac815baa0 2024-04-25T10:12:07.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1acdd8c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac80d0620 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1acdd8c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac80b1840 con 0x7f1ac80cfc80 2024-04-25T10:12:07.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1ac7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1ac80cc3b0 0x7f1ac80cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1ac7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1ac80cc3b0 0x7f1ac80cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32838/0 (socket says 172.21.15.133:32838) 2024-04-25T10:12:07.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1ac7fff640 1 -- 172.21.15.133:0/1170575141 learned_addr learned my addr 172.21.15.133:0/1170575141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:07.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1ac6ffd640 1 --2- 172.21.15.133:0/1170575141 >> v2:172.21.15.162:3300/0 conn(0x7f1ac80cfc80 0x7f1ac80d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1ac7fff640 1 -- 172.21.15.133:0/1170575141 >> v2:172.21.15.162:3300/0 conn(0x7f1ac80cfc80 msgr2=0x7f1ac80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:07.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1ac7fff640 1 --2- 172.21.15.133:0/1170575141 >> v2:172.21.15.162:3300/0 conn(0x7f1ac80cfc80 0x7f1ac80d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1ac7fff640 1 -- 172.21.15.133:0/1170575141 >> v1:172.21.15.162:6789/0 conn(0x7f1ac815baa0 legacy=0x7f1ac80cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:07.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.077+0000 7f1ac7fff640 1 -- 172.21.15.133:0/1170575141 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac80b1b20 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1ac7fff640 1 --2- 172.21.15.133:0/1170575141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac80cc3b0 0x7f1ac80cc7c0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80ced10 tx=0x7f1ac001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3ef06b2938e13a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:07.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1ac67fc640 1 -- 172.21.15.133:0/1170575141 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a5640 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1ac67fc640 1 -- 172.21.15.133:0/1170575141 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac00a57e0 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1ac67fc640 1 -- 172.21.15.133:0/1170575141 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a5bb0 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1170575141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac80cc3b0 msgr2=0x7f1ac80cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:07.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1acdd8c640 1 --2- 172.21.15.133:0/1170575141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac80cc3b0 0x7f1ac80cc7c0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80ced10 tx=0x7f1ac001c890 comp rx=0 tx=0).stop 2024-04-25T10:12:07.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1170575141 shutdown_connections 2024-04-25T10:12:07.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1acdd8c640 1 --2- 172.21.15.133:0/1170575141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac80cc3b0 0x7f1ac80cc7c0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1acdd8c640 1 --2- 172.21.15.133:0/1170575141 >> v2:172.21.15.162:3300/0 conn(0x7f1ac80cfc80 0x7f1ac80d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1170575141 >> 172.21.15.133:0/1170575141 conn(0x7f1ac80d16a0 msgr2=0x7f1ac80d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:07.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1170575141 shutdown_connections 2024-04-25T10:12:07.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1170575141 wait complete. 2024-04-25T10:12:07.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1acdd8c640 1 Processor -- start 2024-04-25T10:12:07.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.081+0000 7f1acdd8c640 1 -- start start 2024-04-25T10:12:07.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1acdd8c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac80cc3b0 0x7f1ac814d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1acdd8c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ac80cfc80 0x7f1ac815c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1acdd8c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac815baa0 0x7f1ac8159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1acdd8c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac81685e0 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1acdd8c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8168460 con 0x7f1ac815baa0 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1acdd8c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8168760 con 0x7f1ac80cfc80 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac80cc3b0 0x7f1ac814d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac80cc3b0 0x7f1ac814d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32852/0 (socket says 172.21.15.133:32852) 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac77fe640 1 -- 172.21.15.133:0/1841095796 learned_addr learned my addr 172.21.15.133:0/1841095796 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac6ffd640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ac80cfc80 0x7f1ac815c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac7fff640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac815baa0 0x7f1ac8159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac77fe640 1 -- 172.21.15.133:0/1841095796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ac80cfc80 msgr2=0x7f1ac815c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac77fe640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ac80cfc80 0x7f1ac815c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac77fe640 1 -- 172.21.15.133:0/1841095796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac815baa0 msgr2=0x7f1ac8159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac77fe640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac815baa0 0x7f1ac8159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac77fe640 1 -- 172.21.15.133:0/1841095796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac814e410 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac6ffd640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ac80cfc80 0x7f1ac815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:07.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac77fe640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac80cc3b0 0x7f1ac814d940 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800b410 tx=0x7f1ab800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:07.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1aaffff640 1 -- 172.21.15.133:0/1841095796 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8015030 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1aaffff640 1 -- 172.21.15.133:0/1841095796 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab8004020 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1841095796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac814f2c0 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1aaffff640 1 -- 172.21.15.133:0/1841095796 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8004450 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1841095796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac814f880 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1841095796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8067c70 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1aaffff640 1 -- 172.21.15.133:0/1841095796 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ab8018070 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1aaffff640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab4066f60 0x7f1ab4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1aaffff640 1 -- 172.21.15.133:0/1841095796 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1ab8084d80 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac6ffd640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab4066f60 0x7f1ab4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.085+0000 7f1ac6ffd640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab4066f60 0x7f1ab4069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1ac815efa0 tx=0x7f1abc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:07.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.089+0000 7f1aaffff640 1 -- 172.21.15.133:0/1841095796 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ab8055bd0 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.269+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1841095796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1ac80ce140 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.275 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:12:07.273+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:12:07.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.273+0000 7f1aaffff640 1 -- 172.21.15.133:0/1841095796 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1ab80554d0 con 0x7f1ac80cc3b0 2024-04-25T10:12:07.278 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:12:07.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1841095796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab4066f60 msgr2=0x7f1ab4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:07.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab4066f60 0x7f1ab4069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1ac815efa0 tx=0x7f1abc008040 comp rx=0 tx=0).stop 2024-04-25T10:12:07.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1841095796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac80cc3b0 msgr2=0x7f1ac814d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:07.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac80cc3b0 0x7f1ac814d940 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800b410 tx=0x7f1ab800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1841095796 shutdown_connections 2024-04-25T10:12:07.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ac815baa0 0x7f1ac8159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ac80cfc80 0x7f1ac815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ab4066f60 0x7f1ab4069420 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 --2- 172.21.15.133:0/1841095796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ac80cc3b0 0x7f1ac814d940 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1841095796 >> 172.21.15.133:0/1841095796 conn(0x7f1ac80d16a0 msgr2=0x7f1ac80b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:07.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1841095796 shutdown_connections 2024-04-25T10:12:07.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.277+0000 7f1acdd8c640 1 -- 172.21.15.133:0/1841095796 wait complete. 2024-04-25T10:12:07.298 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:12:07.372 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:12:07.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7dfaa46640 1 Processor -- start 2024-04-25T10:12:07.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7dfaa46640 1 -- start start 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7dfaa46640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df415a8e0 0x7f7df406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7dfaa46640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7df40b2cc0 0x7f7df40cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7dfaa46640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df406ad30 con 0x7f7df406dea0 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7dfaa46640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df406b5f0 con 0x7f7df40b2cc0 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7dfaa46640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df415acc0 con 0x7f7df415a8e0 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7df8fe3640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7df406dea0 0x7f7df406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43262/0 (socket says 172.21.15.133:43262) 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7df8fe3640 1 -- 172.21.15.133:0/2772098796 learned_addr learned my addr 172.21.15.133:0/2772098796 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7df3fff640 1 --2- 172.21.15.133:0/2772098796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df415a8e0 0x7f7df406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7df37fe640 1 --2- 172.21.15.133:0/2772098796 >> v2:172.21.15.162:3300/0 conn(0x7f7df40b2cc0 0x7f7df40cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7df2ffd640 1 -- 172.21.15.133:0/2772098796 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623584327 0 0) 0x7f7df406ad30 con 0x7f7df406dea0 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7df2ffd640 1 -- 172.21.15.133:0/2772098796 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ddc003530 con 0x7f7df406dea0 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7df3fff640 1 -- 172.21.15.133:0/2772098796 >> v2:172.21.15.162:3300/0 conn(0x7f7df40b2cc0 msgr2=0x7f7df40cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:07.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7df3fff640 1 --2- 172.21.15.133:0/2772098796 >> v2:172.21.15.162:3300/0 conn(0x7f7df40b2cc0 0x7f7df40cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7df3fff640 1 -- 172.21.15.133:0/2772098796 >> v1:172.21.15.133:6789/0 conn(0x7f7df406dea0 legacy=0x7f7df406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:07.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.413+0000 7f7df3fff640 1 -- 172.21.15.133:0/2772098796 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df406afa0 con 0x7f7df415a8e0 2024-04-25T10:12:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7df3fff640 1 --2- 172.21.15.133:0/2772098796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df415a8e0 0x7f7df406eec0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7de40097c0 tx=0x7f7de401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8689b8735d347fc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7df2ffd640 1 -- 172.21.15.133:0/2772098796 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de40ab650 con 0x7f7df415a8e0 2024-04-25T10:12:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7df2ffd640 1 -- 172.21.15.133:0/2772098796 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de40b0070 con 0x7f7df415a8e0 2024-04-25T10:12:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7df2ffd640 1 -- 172.21.15.133:0/2772098796 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de40ab9f0 con 0x7f7df415a8e0 2024-04-25T10:12:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2772098796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df415a8e0 msgr2=0x7f7df406eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 --2- 172.21.15.133:0/2772098796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df415a8e0 0x7f7df406eec0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7de40097c0 tx=0x7f7de401c470 comp rx=0 tx=0).stop 2024-04-25T10:12:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2772098796 shutdown_connections 2024-04-25T10:12:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 --2- 172.21.15.133:0/2772098796 >> v2:172.21.15.162:3300/0 conn(0x7f7df40b2cc0 0x7f7df40cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 --2- 172.21.15.133:0/2772098796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df415a8e0 0x7f7df406eec0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2772098796 >> 172.21.15.133:0/2772098796 conn(0x7f7df40cfd30 msgr2=0x7f7df40d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2772098796 shutdown_connections 2024-04-25T10:12:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2772098796 wait complete. 2024-04-25T10:12:07.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 Processor -- start 2024-04-25T10:12:07.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 -- start start 2024-04-25T10:12:07.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df406dea0 0x7f7df4145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df40b2cc0 0x7f7df40ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df415a8e0 0x7f7df40ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40b3220 con 0x7f7df406dea0 2024-04-25T10:12:07.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40b30a0 con 0x7f7df40b2cc0 2024-04-25T10:12:07.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7dfaa46640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df406c290 con 0x7f7df415a8e0 2024-04-25T10:12:07.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df406dea0 0x7f7df4145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df406dea0 0x7f7df4145820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32856/0 (socket says 172.21.15.133:32856) 2024-04-25T10:12:07.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.417+0000 7f7df3fff640 1 -- 172.21.15.133:0/2903377424 learned_addr learned my addr 172.21.15.133:0/2903377424 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df8fe3640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df415a8e0 0x7f7df40ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df37fe640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df40b2cc0 0x7f7df40ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df3fff640 1 -- 172.21.15.133:0/2903377424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df415a8e0 msgr2=0x7f7df40ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df3fff640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df415a8e0 0x7f7df40ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df3fff640 1 -- 172.21.15.133:0/2903377424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df40b2cc0 msgr2=0x7f7df40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df3fff640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df40b2cc0 0x7f7df40ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df3fff640 1 -- 172.21.15.133:0/2903377424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df40adc60 con 0x7f7df406dea0 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df3fff640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df406dea0 0x7f7df4145820 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f7de40ad040 tx=0x7f7de4019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df0ff9640 1 -- 172.21.15.133:0/2903377424 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4003820 con 0x7f7df406dea0 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df0ff9640 1 -- 172.21.15.133:0/2903377424 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de40039c0 con 0x7f7df406dea0 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2903377424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df40adee0 con 0x7f7df406dea0 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df0ff9640 1 -- 172.21.15.133:0/2903377424 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4003d90 con 0x7f7df406dea0 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2903377424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df40ae4a0 con 0x7f7df406dea0 2024-04-25T10:12:07.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2903377424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df4068e80 con 0x7f7df406dea0 2024-04-25T10:12:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df0ff9640 1 -- 172.21.15.133:0/2903377424 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7de40c3060 con 0x7f7df406dea0 2024-04-25T10:12:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df0ff9640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7de0066f60 0x7f7de0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df0ff9640 1 -- 172.21.15.133:0/2903377424 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f7de411ef20 con 0x7f7df406dea0 2024-04-25T10:12:07.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df37fe640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7de0066f60 0x7f7de0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:07.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.421+0000 7f7df37fe640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7de0066f60 0x7f7de0069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7df4152ff0 tx=0x7f7de8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:07.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.425+0000 7f7df0ff9640 1 -- 172.21.15.133:0/2903377424 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7df40adc60 con 0x7f7df406dea0 2024-04-25T10:12:07.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:07.605+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2903377424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7df40737e0 con 0x7f7df406dea0 2024-04-25T10:12:08.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.277+0000 7f7df0ff9640 1 -- 172.21.15.133:0/2903377424 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7de4016090 con 0x7f7df406dea0 2024-04-25T10:12:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2903377424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7de0066f60 msgr2=0x7f7de0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7de0066f60 0x7f7de0069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7df4152ff0 tx=0x7f7de8008040 comp rx=0 tx=0).stop 2024-04-25T10:12:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2903377424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df406dea0 msgr2=0x7f7df4145820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df406dea0 0x7f7df4145820 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f7de40ad040 tx=0x7f7de4019210 comp rx=0 tx=0).stop 2024-04-25T10:12:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2903377424 shutdown_connections 2024-04-25T10:12:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df415a8e0 0x7f7df40ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df40b2cc0 0x7f7df40ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7de0066f60 0x7f7de0069420 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 --2- 172.21.15.133:0/2903377424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df406dea0 0x7f7df4145820 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:08.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2903377424 >> 172.21.15.133:0/2903377424 conn(0x7f7df40cfd30 msgr2=0x7f7df4156580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:08.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2903377424 shutdown_connections 2024-04-25T10:12:08.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.281+0000 7f7dfaa46640 1 -- 172.21.15.133:0/2903377424 wait complete. 2024-04-25T10:12:08.303 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:12:08.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad24949640 1 Processor -- start 2024-04-25T10:12:08.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad24949640 1 -- start start 2024-04-25T10:12:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad24949640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad2015c420 0x7fad200cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad24949640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2015c9b0 con 0x7fad20071cb0 2024-04-25T10:12:08.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad24949640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2015cc90 con 0x7fad200b4940 2024-04-25T10:12:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad24949640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad20072a90 con 0x7fad2015c420 2024-04-25T10:12:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad1ed9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad2015c420 0x7fad200cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad1ed9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad2015c420 0x7fad200cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32866/0 (socket says 172.21.15.133:32866) 2024-04-25T10:12:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad17fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fad20071cb0 0x7fad2015bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43264/0 (socket says 172.21.15.133:43264) 2024-04-25T10:12:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad1ed9d640 1 -- 172.21.15.133:0/3394812633 learned_addr learned my addr 172.21.15.133:0/3394812633 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad1ed9d640 1 -- 172.21.15.133:0/3394812633 >> v1:172.21.15.162:6789/0 conn(0x7fad200b4940 legacy=0x7fad200b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad1ed9d640 1 -- 172.21.15.133:0/3394812633 >> v1:172.21.15.133:6789/0 conn(0x7fad20071cb0 legacy=0x7fad2015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad1ed9d640 1 -- 172.21.15.133:0/3394812633 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200711d0 con 0x7fad2015c420 2024-04-25T10:12:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.429+0000 7fad1dd9b640 1 -- 172.21.15.133:0/3394812633 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366942306 0 0) 0x7fad2015c9b0 con 0x7fad20071cb0 2024-04-25T10:12:08.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad1ed9d640 1 --2- 172.21.15.133:0/3394812633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad2015c420 0x7fad200cc010 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fad18009e50 tx=0x7fad1801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314595f7d9dbf787 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:08.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad1dd9b640 1 -- 172.21.15.133:0/3394812633 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad180a5700 con 0x7fad2015c420 2024-04-25T10:12:08.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad1dd9b640 1 -- 172.21.15.133:0/3394812633 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad180a58a0 con 0x7fad2015c420 2024-04-25T10:12:08.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad1dd9b640 1 -- 172.21.15.133:0/3394812633 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad180a5c70 con 0x7fad2015c420 2024-04-25T10:12:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad24949640 1 -- 172.21.15.133:0/3394812633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad2015c420 msgr2=0x7fad200cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad24949640 1 --2- 172.21.15.133:0/3394812633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad2015c420 0x7fad200cc010 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fad18009e50 tx=0x7fad1801ca30 comp rx=0 tx=0).stop 2024-04-25T10:12:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad24949640 1 -- 172.21.15.133:0/3394812633 shutdown_connections 2024-04-25T10:12:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad24949640 1 --2- 172.21.15.133:0/3394812633 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad2015c420 0x7fad200cc010 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad24949640 1 -- 172.21.15.133:0/3394812633 >> 172.21.15.133:0/3394812633 conn(0x7fad2006d370 msgr2=0x7fad2006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad24949640 1 -- 172.21.15.133:0/3394812633 shutdown_connections 2024-04-25T10:12:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad24949640 1 -- 172.21.15.133:0/3394812633 wait complete. 2024-04-25T10:12:08.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad24949640 1 Processor -- start 2024-04-25T10:12:08.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.433+0000 7fad24949640 1 -- start start 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad24949640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad20071cb0 0x7fad200c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad24949640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad200b4940 0x7fad201428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad24949640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad2015c420 0x7fad20142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad24949640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad20145450 con 0x7fad200b4940 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad24949640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad201452d0 con 0x7fad20071cb0 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad24949640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad20072120 con 0x7fad2015c420 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad17fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad200b4940 0x7fad201428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad17fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad200b4940 0x7fad201428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32868/0 (socket says 172.21.15.133:32868) 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad17fff640 1 -- 172.21.15.133:0/2705934493 learned_addr learned my addr 172.21.15.133:0/2705934493 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad1e59c640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad20071cb0 0x7fad200c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad1ed9d640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad2015c420 0x7fad20142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad17fff640 1 -- 172.21.15.133:0/2705934493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad2015c420 msgr2=0x7fad20142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad17fff640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad2015c420 0x7fad20142e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad17fff640 1 -- 172.21.15.133:0/2705934493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad20071cb0 msgr2=0x7fad200c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad17fff640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad20071cb0 0x7fad200c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad17fff640 1 -- 172.21.15.133:0/2705934493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200b2960 con 0x7fad200b4940 2024-04-25T10:12:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad1e59c640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad20071cb0 0x7fad200c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad17fff640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad200b4940 0x7fad201428d0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c4a0 tx=0x7fad0800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad16ffd640 1 -- 172.21.15.133:0/2705934493 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08018070 con 0x7fad200b4940 2024-04-25T10:12:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad16ffd640 1 -- 172.21.15.133:0/2705934493 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad080074e0 con 0x7fad200b4940 2024-04-25T10:12:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad24949640 1 -- 172.21.15.133:0/2705934493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad20143420 con 0x7fad200b4940 2024-04-25T10:12:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad16ffd640 1 -- 172.21.15.133:0/2705934493 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08004a30 con 0x7fad200b4940 2024-04-25T10:12:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad24949640 1 -- 172.21.15.133:0/2705934493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad201439e0 con 0x7fad200b4940 2024-04-25T10:12:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad24949640 1 -- 172.21.15.133:0/2705934493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad20069000 con 0x7fad200b4940 2024-04-25T10:12:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad16ffd640 1 -- 172.21.15.133:0/2705934493 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad0801b070 con 0x7fad200b4940 2024-04-25T10:12:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad16ffd640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad00066f60 0x7fad00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad1e59c640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad00066f60 0x7fad00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.437+0000 7fad16ffd640 1 -- 172.21.15.133:0/2705934493 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fad08084580 con 0x7fad200b4940 2024-04-25T10:12:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.441+0000 7fad1e59c640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad00066f60 0x7fad00069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fad10000f70 tx=0x7fad10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:08.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.441+0000 7fad16ffd640 1 -- 172.21.15.133:0/2705934493 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad08055370 con 0x7fad200b4940 2024-04-25T10:12:08.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:08.605+0000 7fad24949640 1 -- 172.21.15.133:0/2705934493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad20141c00 con 0x7fad200b4940 2024-04-25T10:12:09.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.485+0000 7fad16ffd640 1 -- 172.21.15.133:0/2705934493 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) v1 ==== 181+0+0 (secure 0 0 0) 0x7fad08054c70 con 0x7fad200b4940 2024-04-25T10:12:09.488 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:12:09.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.489+0000 7fad24949640 1 -- 172.21.15.133:0/2705934493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad00066f60 msgr2=0x7fad00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.489+0000 7fad24949640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad00066f60 0x7fad00069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fad10000f70 tx=0x7fad10009040 comp rx=0 tx=0).stop 2024-04-25T10:12:09.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.489+0000 7fad24949640 1 -- 172.21.15.133:0/2705934493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad200b4940 msgr2=0x7fad201428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.489+0000 7fad24949640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad200b4940 0x7fad201428d0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c4a0 tx=0x7fad0800c960 comp rx=0 tx=0).stop 2024-04-25T10:12:09.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.489+0000 7fad24949640 1 -- 172.21.15.133:0/2705934493 shutdown_connections 2024-04-25T10:12:09.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.489+0000 7fad24949640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fad2015c420 0x7fad20142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.489+0000 7fad24949640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fad00066f60 0x7fad00069420 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.489+0000 7fad24949640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad200b4940 0x7fad201428d0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.489+0000 7fad24949640 1 --2- 172.21.15.133:0/2705934493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad20071cb0 0x7fad200c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.489+0000 7fad24949640 1 -- 172.21.15.133:0/2705934493 >> 172.21.15.133:0/2705934493 conn(0x7fad2006d370 msgr2=0x7fad2013f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:09.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.493+0000 7fad24949640 1 -- 172.21.15.133:0/2705934493 shutdown_connections 2024-04-25T10:12:09.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.493+0000 7fad24949640 1 -- 172.21.15.133:0/2705934493 wait complete. 2024-04-25T10:12:09.515 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:12:09.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08b50c7640 1 Processor -- start 2024-04-25T10:12:09.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08b50c7640 1 -- start start 2024-04-25T10:12:09.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08b50c7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b00b4940 0x7f08b00b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08b50c7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f08b0071cb0 0x7f08b015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08b50c7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b00b5260 con 0x7f08b015c250 2024-04-25T10:12:09.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08b50c7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b0072a90 con 0x7f08b0071cb0 2024-04-25T10:12:09.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08b50c7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00a85d0 con 0x7f08b00b4940 2024-04-25T10:12:09.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08af59e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f08b015c250 0x7f08b00cc010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43280/0 (socket says 172.21.15.133:43280) 2024-04-25T10:12:09.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08af59e640 1 -- 172.21.15.133:0/2141548767 learned_addr learned my addr 172.21.15.133:0/2141548767 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:09.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08ae59c640 1 --2- 172.21.15.133:0/2141548767 >> v2:172.21.15.162:3300/0 conn(0x7f08b0071cb0 0x7f08b015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08aed9d640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b00b4940 0x7f08b00b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256388235 0 0) 0x7f08b00b5260 con 0x7f08b015c250 2024-04-25T10:12:09.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.641+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0898003530 con 0x7f08b015c250 2024-04-25T10:12:09.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 391358255 0 0) 0x7f0898003530 con 0x7f08b015c250 2024-04-25T10:12:09.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b00b4940 msgr2=0x7f08b00b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:09.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b00b4940 0x7f08b00b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 >> v2:172.21.15.162:3300/0 conn(0x7f08b0071cb0 msgr2=0x7f08b015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:09.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 --2- 172.21.15.133:0/2141548767 >> v2:172.21.15.162:3300/0 conn(0x7f08b0071cb0 0x7f08b015bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b015c9b0 con 0x7f08b015c250 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f08a8001a00 con 0x7f08b015c250 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 >> v1:172.21.15.133:6789/0 conn(0x7f08b015c250 legacy=0x7f08b00cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08980039c0 0x7f0898003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0898004810 0x7f0898004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b00b4940 0x7f08980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089800b9d0 con 0x7f0898004810 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089800bb50 con 0x7f08980039c0 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089800bcd0 con 0x7f08b00b4940 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08ae59c640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0898004810 0x7f0898004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08aed9d640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08980039c0 0x7f0898003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08af59e640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b00b4940 0x7f08980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08ae59c640 1 -- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b00b4940 msgr2=0x7f08980051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08ae59c640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b00b4940 0x7f08980051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08ae59c640 1 -- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08980039c0 msgr2=0x7f0898003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:09.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08ae59c640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08980039c0 0x7f0898003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08ae59c640 1 -- 172.21.15.133:0/2141548767 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0898003530 con 0x7f0898004810 2024-04-25T10:12:09.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08aed9d640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08980039c0 0x7f0898003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:09.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08ae59c640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0898004810 0x7f0898004c90 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009ae0 tx=0x7f08a401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f28a2286cba4ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:09.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40096b0 con 0x7f0898004810 2024-04-25T10:12:09.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a40a64c0 con 0x7f0898004810 2024-04-25T10:12:09.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.645+0000 7f08add9b640 1 -- 172.21.15.133:0/2141548767 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40a68b0 con 0x7f0898004810 2024-04-25T10:12:09.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 -- 172.21.15.133:0/2141548767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0898004810 msgr2=0x7f0898004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 --2- 172.21.15.133:0/2141548767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0898004810 0x7f0898004c90 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009ae0 tx=0x7f08a401cab0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08af59e640 1 -- 172.21.15.133:0/2141548767 reap_dead start 2024-04-25T10:12:09.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 -- 172.21.15.133:0/2141548767 shutdown_connections 2024-04-25T10:12:09.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 -- 172.21.15.133:0/2141548767 >> 172.21.15.133:0/2141548767 conn(0x7f08b006d370 msgr2=0x7f08b006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:09.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 -- 172.21.15.133:0/2141548767 shutdown_connections 2024-04-25T10:12:09.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 -- 172.21.15.133:0/2141548767 wait complete. 2024-04-25T10:12:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 Processor -- start 2024-04-25T10:12:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 -- start start 2024-04-25T10:12:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0898004810 0x7f08b00a9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b013ae80 0x7f08b013dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b013e1c0 0x7f08b01f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00714b0 con 0x7f08b013ae80 2024-04-25T10:12:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00711d0 con 0x7f0898004810 2024-04-25T10:12:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b0071790 con 0x7f08b013e1c0 2024-04-25T10:12:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08ae59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b013ae80 0x7f08b013dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08ae59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b013ae80 0x7f08b013dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32892/0 (socket says 172.21.15.133:32892) 2024-04-25T10:12:09.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08ae59c640 1 -- 172.21.15.133:0/1136787478 learned_addr learned my addr 172.21.15.133:0/1136787478 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08af59e640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b013e1c0 0x7f08b01f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08aed9d640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0898004810 0x7f08b00a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08ae59c640 1 -- 172.21.15.133:0/1136787478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b013e1c0 msgr2=0x7f08b01f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08ae59c640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b013e1c0 0x7f08b01f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08ae59c640 1 -- 172.21.15.133:0/1136787478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0898004810 msgr2=0x7f08b00a9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08ae59c640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0898004810 0x7f08b00a9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08ae59c640 1 -- 172.21.15.133:0/1136787478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b00b1060 con 0x7f08b013ae80 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08aed9d640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0898004810 0x7f08b00a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08ae59c640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b013ae80 0x7f08b013dc80 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f08a401c8e0 tx=0x7f0898005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08977fe640 1 -- 172.21.15.133:0/1136787478 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40a6940 con 0x7f08b013ae80 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08977fe640 1 -- 172.21.15.133:0/1136787478 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a40bf070 con 0x7f08b013ae80 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 -- 172.21.15.133:0/1136787478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b0140f10 con 0x7f08b013ae80 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08977fe640 1 -- 172.21.15.133:0/1136787478 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40a6d20 con 0x7f08b013ae80 2024-04-25T10:12:09.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.649+0000 7f08b50c7640 1 -- 172.21.15.133:0/1136787478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b0141490 con 0x7f08b013ae80 2024-04-25T10:12:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.653+0000 7f08b50c7640 1 -- 172.21.15.133:0/1136787478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b0069000 con 0x7f08b013ae80 2024-04-25T10:12:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.653+0000 7f08977fe640 1 -- 172.21.15.133:0/1136787478 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08a4006040 con 0x7f08b013ae80 2024-04-25T10:12:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.653+0000 7f08977fe640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f089c066f90 0x7f089c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.653+0000 7f08977fe640 1 -- 172.21.15.133:0/1136787478 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5807+0+0 (secure 0 0 0) 0x7f08a411fe90 con 0x7f08b013ae80 2024-04-25T10:12:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.653+0000 7f08aed9d640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f089c066f90 0x7f089c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:09.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.653+0000 7f08aed9d640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f089c066f90 0x7f089c069450 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f08b01447e0 tx=0x7f08a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:09.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.657+0000 7f08977fe640 1 -- 172.21.15.133:0/1136787478 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08a40167c0 con 0x7f08b013ae80 2024-04-25T10:12:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:09.817+0000 7f08b50c7640 1 -- 172.21.15.133:0/1136787478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f08b013e5a0 con 0x7f08b013ae80 2024-04-25T10:12:10.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.493+0000 7f08977fe640 1 -- 172.21.15.133:0/1136787478 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) v1 ==== 169+0+0 (secure 0 0 0) 0x7f08a40f0e40 con 0x7f08b013ae80 2024-04-25T10:12:10.497 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:12:10.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.497+0000 7f08b50c7640 1 -- 172.21.15.133:0/1136787478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f089c066f90 msgr2=0x7f089c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.497+0000 7f08b50c7640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f089c066f90 0x7f089c069450 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f08b01447e0 tx=0x7f08a0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:10.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.497+0000 7f08b50c7640 1 -- 172.21.15.133:0/1136787478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b013ae80 msgr2=0x7f08b013dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.497+0000 7f08b50c7640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b013ae80 0x7f08b013dc80 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f08a401c8e0 tx=0x7f0898005840 comp rx=0 tx=0).stop 2024-04-25T10:12:10.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.501+0000 7f08b50c7640 1 -- 172.21.15.133:0/1136787478 shutdown_connections 2024-04-25T10:12:10.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.501+0000 7f08b50c7640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b013e1c0 0x7f08b01f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.501+0000 7f08b50c7640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f089c066f90 0x7f089c069450 secure :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f08b01447e0 tx=0x7f08a0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:10.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.501+0000 7f08b50c7640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b013ae80 0x7f08b013dc80 secure :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f08a401c8e0 tx=0x7f0898005840 comp rx=0 tx=0).stop 2024-04-25T10:12:10.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.501+0000 7f08b50c7640 1 --2- 172.21.15.133:0/1136787478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0898004810 0x7f08b00a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.501+0000 7f08b50c7640 1 -- 172.21.15.133:0/1136787478 >> 172.21.15.133:0/1136787478 conn(0x7f08b006d370 msgr2=0x7f08b015ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:10.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.501+0000 7f08b50c7640 1 -- 172.21.15.133:0/1136787478 shutdown_connections 2024-04-25T10:12:10.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.501+0000 7f08b50c7640 1 -- 172.21.15.133:0/1136787478 wait complete. 2024-04-25T10:12:10.528 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:12:10.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951c3d4640 1 Processor -- start 2024-04-25T10:12:10.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951c3d4640 1 -- start start 2024-04-25T10:12:10.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951c3d4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9514145170 0x7f951406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951c3d4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f951406e4d0 0x7f95140cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951c3d4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95140ce2e0 con 0x7f95140cdd60 2024-04-25T10:12:10.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951c3d4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951406db00 con 0x7f951406e4d0 2024-04-25T10:12:10.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951c3d4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9514145550 con 0x7f9514145170 2024-04-25T10:12:10.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951996f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f951406e4d0 0x7f95140cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951a971640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f95140cdd60 0x7f951415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43282/0 (socket says 172.21.15.133:43282) 2024-04-25T10:12:10.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951a971640 1 -- 172.21.15.133:0/493492952 learned_addr learned my addr 172.21.15.133:0/493492952 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:10.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951996f640 1 -- 172.21.15.133:0/493492952 >> v2:172.21.15.162:3300/0 conn(0x7f9514145170 msgr2=0x7f951406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:10.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951a170640 1 --2- 172.21.15.133:0/493492952 >> v2:172.21.15.162:3300/0 conn(0x7f9514145170 0x7f951406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951916e640 1 -- 172.21.15.133:0/493492952 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055805581 0 0) 0x7f95140ce2e0 con 0x7f95140cdd60 2024-04-25T10:12:10.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951996f640 1 --2- 172.21.15.133:0/493492952 >> v2:172.21.15.162:3300/0 conn(0x7f9514145170 0x7f951406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951996f640 1 -- 172.21.15.133:0/493492952 >> v1:172.21.15.133:6789/0 conn(0x7f95140cdd60 legacy=0x7f951415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951996f640 1 -- 172.21.15.133:0/493492952 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951406d4b0 con 0x7f951406e4d0 2024-04-25T10:12:10.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.653+0000 7f951a170640 1 --2- 172.21.15.133:0/493492952 >> v2:172.21.15.162:3300/0 conn(0x7f9514145170 0x7f951406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:10.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951996f640 1 --2- 172.21.15.133:0/493492952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f951406e4d0 0x7f95140cd820 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f951406b790 tx=0x7f950401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad2fbae17ca2ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:10.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951916e640 1 -- 172.21.15.133:0/493492952 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040b0070 con 0x7f951406e4d0 2024-04-25T10:12:10.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951916e640 1 -- 172.21.15.133:0/493492952 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95040ab600 con 0x7f951406e4d0 2024-04-25T10:12:10.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951916e640 1 -- 172.21.15.133:0/493492952 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040ab9d0 con 0x7f951406e4d0 2024-04-25T10:12:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951c3d4640 1 -- 172.21.15.133:0/493492952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f951406e4d0 msgr2=0x7f95140cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951c3d4640 1 --2- 172.21.15.133:0/493492952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f951406e4d0 0x7f95140cd820 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f951406b790 tx=0x7f950401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951c3d4640 1 -- 172.21.15.133:0/493492952 shutdown_connections 2024-04-25T10:12:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951c3d4640 1 --2- 172.21.15.133:0/493492952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f951406e4d0 0x7f95140cd820 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951c3d4640 1 --2- 172.21.15.133:0/493492952 >> v2:172.21.15.162:3300/0 conn(0x7f9514145170 0x7f951406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951c3d4640 1 -- 172.21.15.133:0/493492952 >> 172.21.15.133:0/493492952 conn(0x7f9514066f70 msgr2=0x7f951406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:10.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951c3d4640 1 -- 172.21.15.133:0/493492952 shutdown_connections 2024-04-25T10:12:10.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.657+0000 7f951c3d4640 1 -- 172.21.15.133:0/493492952 wait complete. 2024-04-25T10:12:10.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951c3d4640 1 Processor -- start 2024-04-25T10:12:10.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951c3d4640 1 -- start start 2024-04-25T10:12:10.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951c3d4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f951406e4d0 0x7f951415ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951c3d4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95140cdd60 0x7f95141559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951c3d4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9514145170 0x7f951413ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951c3d4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9514168490 con 0x7f951406e4d0 2024-04-25T10:12:10.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951c3d4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951415d7f0 con 0x7f95140cdd60 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951c3d4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9514168610 con 0x7f9514145170 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951a170640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f951406e4d0 0x7f951415ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951a170640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f951406e4d0 0x7f951415ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32906/0 (socket says 172.21.15.133:32906) 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951a170640 1 -- 172.21.15.133:0/518258132 learned_addr learned my addr 172.21.15.133:0/518258132 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951a170640 1 -- 172.21.15.133:0/518258132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9514145170 msgr2=0x7f951413ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951a170640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9514145170 0x7f951413ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951a170640 1 -- 172.21.15.133:0/518258132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95140cdd60 msgr2=0x7f95141559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951996f640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95140cdd60 0x7f95141559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951a170640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95140cdd60 0x7f95141559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951a170640 1 -- 172.21.15.133:0/518258132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95140b3680 con 0x7f951406e4d0 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951a170640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f951406e4d0 0x7f951415ee30 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f951000c7c0 tx=0x7f951000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f950affd640 1 -- 172.21.15.133:0/518258132 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510007a30 con 0x7f951406e4d0 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f950affd640 1 -- 172.21.15.133:0/518258132 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9510007bd0 con 0x7f951406e4d0 2024-04-25T10:12:10.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951c3d4640 1 -- 172.21.15.133:0/518258132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951413b170 con 0x7f951406e4d0 2024-04-25T10:12:10.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f950affd640 1 -- 172.21.15.133:0/518258132 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100113f0 con 0x7f951406e4d0 2024-04-25T10:12:10.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951c3d4640 1 -- 172.21.15.133:0/518258132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95140b28c0 con 0x7f951406e4d0 2024-04-25T10:12:10.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951c3d4640 1 -- 172.21.15.133:0/518258132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9514068e80 con 0x7f951406e4d0 2024-04-25T10:12:10.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f950affd640 1 -- 172.21.15.133:0/518258132 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9510011590 con 0x7f951406e4d0 2024-04-25T10:12:10.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f950affd640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94f8066f60 0x7f94f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f950affd640 1 -- 172.21.15.133:0/518258132 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f95100850a0 con 0x7f951406e4d0 2024-04-25T10:12:10.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.661+0000 7f951996f640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94f8066f60 0x7f94f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.665+0000 7f951996f640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94f8066f60 0x7f94f8069420 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f95040ad000 tx=0x7f9504008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:10.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.665+0000 7f950affd640 1 -- 172.21.15.133:0/518258132 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9510015020 con 0x7f951406e4d0 2024-04-25T10:12:10.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.825+0000 7f951c3d4640 1 -- 172.21.15.133:0/518258132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f951413a370 con 0x7f951406e4d0 2024-04-25T10:12:10.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.829+0000 7f950affd640 1 -- 172.21.15.133:0/518258132 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9510056220 con 0x7f951406e4d0 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 -- 172.21.15.133:0/518258132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94f8066f60 msgr2=0x7f94f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94f8066f60 0x7f94f8069420 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f95040ad000 tx=0x7f9504008040 comp rx=0 tx=0).stop 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 -- 172.21.15.133:0/518258132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f951406e4d0 msgr2=0x7f951415ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f951406e4d0 0x7f951415ee30 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f951000c7c0 tx=0x7f951000cc80 comp rx=0 tx=0).stop 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 -- 172.21.15.133:0/518258132 shutdown_connections 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9514145170 0x7f951413ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f95140cdd60 0x7f95141559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94f8066f60 0x7f94f8069420 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 --2- 172.21.15.133:0/518258132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f951406e4d0 0x7f951415ee30 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 -- 172.21.15.133:0/518258132 >> 172.21.15.133:0/518258132 conn(0x7f9514066f70 msgr2=0x7f951406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 -- 172.21.15.133:0/518258132 shutdown_connections 2024-04-25T10:12:10.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.833+0000 7f951c3d4640 1 -- 172.21.15.133:0/518258132 wait complete. 2024-04-25T10:12:10.853 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T10:12:10.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.993+0000 7fed83ad1640 1 Processor -- start 2024-04-25T10:12:10.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.993+0000 7fed83ad1640 1 -- start start 2024-04-25T10:12:10.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed83ad1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed7c0b2e20 0x7fed7c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed83ad1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed7c15c500 0x7fed7c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed83ad1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c072ae0 con 0x7fed7c145280 2024-04-25T10:12:10.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed83ad1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c072dc0 con 0x7fed7c0b2e20 2024-04-25T10:12:10.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed83ad1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0730a0 con 0x7fed7c15c500 2024-04-25T10:12:10.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8106c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed7c0b2e20 0x7fed7c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8106c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed7c0b2e20 0x7fed7c15bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32912/0 (socket says 172.21.15.133:32912) 2024-04-25T10:12:11.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8106c640 1 -- 172.21.15.133:0/3495425089 learned_addr learned my addr 172.21.15.133:0/3495425089 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:11.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8206e640 1 --2- 172.21.15.133:0/3495425089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed7c15c500 0x7fed7c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8106c640 1 -- 172.21.15.133:0/3495425089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed7c15c500 msgr2=0x7fed7c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8106c640 1 --2- 172.21.15.133:0/3495425089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed7c15c500 0x7fed7c06ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8106c640 1 -- 172.21.15.133:0/3495425089 >> v1:172.21.15.162:6789/0 conn(0x7fed7c145280 legacy=0x7fed7c072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8106c640 1 -- 172.21.15.133:0/3495425089 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0cd010 con 0x7fed7c0b2e20 2024-04-25T10:12:11.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8106c640 1 --2- 172.21.15.133:0/3495425089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed7c0b2e20 0x7fed7c15bfc0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009870 tx=0x7fed6c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cdaf838e48724f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8086b640 1 -- 172.21.15.133:0/3495425089 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0b0070 con 0x7fed7c0b2e20 2024-04-25T10:12:11.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8086b640 1 -- 172.21.15.133:0/3495425089 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c0ab600 con 0x7fed7c0b2e20 2024-04-25T10:12:11.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:10.997+0000 7fed8086b640 1 -- 172.21.15.133:0/3495425089 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0ab9d0 con 0x7fed7c0b2e20 2024-04-25T10:12:11.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 -- 172.21.15.133:0/3495425089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed7c0b2e20 msgr2=0x7fed7c15bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 --2- 172.21.15.133:0/3495425089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed7c0b2e20 0x7fed7c15bfc0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009870 tx=0x7fed6c01c490 comp rx=0 tx=0).stop 2024-04-25T10:12:11.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 -- 172.21.15.133:0/3495425089 shutdown_connections 2024-04-25T10:12:11.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 --2- 172.21.15.133:0/3495425089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed7c15c500 0x7fed7c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 --2- 172.21.15.133:0/3495425089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed7c0b2e20 0x7fed7c15bfc0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 -- 172.21.15.133:0/3495425089 >> 172.21.15.133:0/3495425089 conn(0x7fed7c0b2300 msgr2=0x7fed7c0b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:11.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 -- 172.21.15.133:0/3495425089 shutdown_connections 2024-04-25T10:12:11.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 -- 172.21.15.133:0/3495425089 wait complete. 2024-04-25T10:12:11.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 Processor -- start 2024-04-25T10:12:11.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 -- start start 2024-04-25T10:12:11.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed7c0b2e20 0x7fed7c140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed7c145280 0x7fed7c13c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed7c15c500 0x7fed7c0b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c06b5e0 con 0x7fed7c145280 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0ce270 con 0x7fed7c15c500 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c168670 con 0x7fed7c0b2e20 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed8106c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed7c145280 0x7fed7c13c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed8106c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed7c145280 0x7fed7c13c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32920/0 (socket says 172.21.15.133:32920) 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed8106c640 1 -- 172.21.15.133:0/3253832506 learned_addr learned my addr 172.21.15.133:0/3253832506 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed8206e640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed7c15c500 0x7fed7c0b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed8186d640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed7c0b2e20 0x7fed7c140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed8106c640 1 -- 172.21.15.133:0/3253832506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed7c0b2e20 msgr2=0x7fed7c140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed8106c640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed7c0b2e20 0x7fed7c140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed8106c640 1 -- 172.21.15.133:0/3253832506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed7c15c500 msgr2=0x7fed7c0b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed8106c640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed7c15c500 0x7fed7c0b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed8106c640 1 -- 172.21.15.133:0/3253832506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c15ae70 con 0x7fed7c145280 2024-04-25T10:12:11.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed8106c640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed7c145280 0x7fed7c13c0d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0ae040 tx=0x7fed6c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed727fc640 1 -- 172.21.15.133:0/3253832506 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0ba020 con 0x7fed7c145280 2024-04-25T10:12:11.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed727fc640 1 -- 172.21.15.133:0/3253832506 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c0aa7d0 con 0x7fed7c145280 2024-04-25T10:12:11.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 -- 172.21.15.133:0/3253832506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c13d7e0 con 0x7fed7c145280 2024-04-25T10:12:11.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed727fc640 1 -- 172.21.15.133:0/3253832506 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0b0040 con 0x7fed7c145280 2024-04-25T10:12:11.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.001+0000 7fed83ad1640 1 -- 172.21.15.133:0/3253832506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c15b3f0 con 0x7fed7c145280 2024-04-25T10:12:11.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.005+0000 7fed83ad1640 1 -- 172.21.15.133:0/3253832506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c13d610 con 0x7fed7c145280 2024-04-25T10:12:11.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.005+0000 7fed727fc640 1 -- 172.21.15.133:0/3253832506 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed6c0bd020 con 0x7fed7c145280 2024-04-25T10:12:11.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.005+0000 7fed727fc640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed60066f60 0x7fed60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.005+0000 7fed727fc640 1 -- 172.21.15.133:0/3253832506 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fed6c11f230 con 0x7fed7c145280 2024-04-25T10:12:11.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.005+0000 7fed8186d640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed60066f60 0x7fed60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.005+0000 7fed8186d640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed60066f60 0x7fed60069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fed7800b810 tx=0x7fed78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.009+0000 7fed727fc640 1 -- 172.21.15.133:0/3253832506 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed6c0c4030 con 0x7fed7c145280 2024-04-25T10:12:11.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.225+0000 7fed83ad1640 1 -- 172.21.15.133:0/3253832506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7fed7c067c70 con 0x7fed7c145280 2024-04-25T10:12:11.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.497+0000 7fed727fc640 1 -- 172.21.15.133:0/3253832506 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v462) v1 ==== 155+0+0 (secure 0 0 0) 0x7fed6c016110 con 0x7fed7c145280 2024-04-25T10:12:11.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.497+0000 7fed83ad1640 1 -- 172.21.15.133:0/3253832506 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed60066f60 msgr2=0x7fed60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.497+0000 7fed83ad1640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed60066f60 0x7fed60069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fed7800b810 tx=0x7fed78008040 comp rx=0 tx=0).stop 2024-04-25T10:12:11.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.497+0000 7fed83ad1640 1 -- 172.21.15.133:0/3253832506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed7c145280 msgr2=0x7fed7c13c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.497+0000 7fed83ad1640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed7c145280 0x7fed7c13c0d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0ae040 tx=0x7fed6c01acd0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.501+0000 7fed83ad1640 1 -- 172.21.15.133:0/3253832506 shutdown_connections 2024-04-25T10:12:11.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.501+0000 7fed83ad1640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fed7c15c500 0x7fed7c0b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.501+0000 7fed83ad1640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fed60066f60 0x7fed60069420 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.501+0000 7fed83ad1640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed7c145280 0x7fed7c13c0d0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.501+0000 7fed83ad1640 1 --2- 172.21.15.133:0/3253832506 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fed7c0b2e20 0x7fed7c140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.501+0000 7fed83ad1640 1 -- 172.21.15.133:0/3253832506 >> 172.21.15.133:0/3253832506 conn(0x7fed7c0b2300 msgr2=0x7fed7c0b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:11.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.501+0000 7fed83ad1640 1 -- 172.21.15.133:0/3253832506 shutdown_connections 2024-04-25T10:12:11.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.501+0000 7fed83ad1640 1 -- 172.21.15.133:0/3253832506 wait complete. 2024-04-25T10:12:11.522 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:12:11.522 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:12:11.522 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:11.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.645+0000 7f0a011f3640 1 Processor -- start 2024-04-25T10:12:11.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.645+0000 7f0a011f3640 1 -- start start 2024-04-25T10:12:11.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f0a011f3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f09fc06eb70 0x7f09fc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f0a011f3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09fc0cbe60 con 0x7f09fc158110 2024-04-25T10:12:11.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f0a011f3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09fc0b19b0 con 0x7f09fc1584f0 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f0a011f3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc15b360 con 0x7f09fc06eb70 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fad9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f09fc158110 0x7f09fc06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43284/0 (socket says 172.21.15.133:43284) 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fad9d640 1 -- 172.21.15.133:0/470976381 learned_addr learned my addr 172.21.15.133:0/470976381 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09f259c640 1 --2- 172.21.15.133:0/470976381 >> v2:172.21.15.162:3300/0 conn(0x7f09fc06eb70 0x7f09fc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329055477 0 0) 0x7f09fc0cbe60 con 0x7f09fc158110 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09e8003530 con 0x7f09fc158110 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2978339704 0 0) 0x7f09e8003530 con 0x7f09fc158110 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 >> v2:172.21.15.162:3300/0 conn(0x7f09fc06eb70 msgr2=0x7f09fc0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 --2- 172.21.15.133:0/470976381 >> v2:172.21.15.162:3300/0 conn(0x7f09fc06eb70 0x7f09fc0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 >> v1:172.21.15.162:6789/0 conn(0x7f09fc1584f0 legacy=0x7f09fc0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc15b640 con 0x7f09fc158110 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f09ec001680 con 0x7f09fc158110 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 >> v1:172.21.15.133:6789/0 conn(0x7f09fc158110 legacy=0x7f09fc06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09e80039c0 0x7f09e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09e8004810 0x7f09e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e80051d0 0x7f09e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc15b360 con 0x7f09e80039c0 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e800c070 con 0x7f09e80051d0 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e800c270 con 0x7f09e8004810 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fad9d640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09e80039c0 0x7f09e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fb59e640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e80051d0 0x7f09e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09f259c640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09e8004810 0x7f09e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fad9d640 1 -- 172.21.15.133:0/470976381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09e8004810 msgr2=0x7f09e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fad9d640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09e8004810 0x7f09e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fad9d640 1 -- 172.21.15.133:0/470976381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e80051d0 msgr2=0x7f09e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fad9d640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e80051d0 0x7f09e800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fad9d640 1 -- 172.21.15.133:0/470976381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e8003530 con 0x7f09e80039c0 2024-04-25T10:12:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fb59e640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e80051d0 0x7f09e800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:11.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09f259c640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09e8004810 0x7f09e8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:11.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.649+0000 7f09fb59e640 1 -- 172.21.15.133:0/470976381 reap_dead start 2024-04-25T10:12:11.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f09fad9d640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09e80039c0 0x7f09e8003de0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00c240 tx=0x7f09ec01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40843f23d28ab389 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec007d20 con 0x7f09e80039c0 2024-04-25T10:12:11.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec0a9070 con 0x7f09e80039c0 2024-04-25T10:12:11.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f09fa59c640 1 -- 172.21.15.133:0/470976381 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec005da0 con 0x7f09e80039c0 2024-04-25T10:12:11.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f0a011f3640 1 -- 172.21.15.133:0/470976381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09e80039c0 msgr2=0x7f09e8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f0a011f3640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09e80039c0 0x7f09e8003de0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00c240 tx=0x7f09ec01cc50 comp rx=0 tx=0).stop 2024-04-25T10:12:11.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f0a011f3640 1 -- 172.21.15.133:0/470976381 shutdown_connections 2024-04-25T10:12:11.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f0a011f3640 1 --2- 172.21.15.133:0/470976381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09e80039c0 0x7f09e8003de0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f0a011f3640 1 -- 172.21.15.133:0/470976381 >> 172.21.15.133:0/470976381 conn(0x7f09fc067000 msgr2=0x7f09fc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:11.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f0a011f3640 1 -- 172.21.15.133:0/470976381 shutdown_connections 2024-04-25T10:12:11.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f0a011f3640 1 -- 172.21.15.133:0/470976381 wait complete. 2024-04-25T10:12:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f0a011f3640 1 Processor -- start 2024-04-25T10:12:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f0a011f3640 1 -- start start 2024-04-25T10:12:11.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.653+0000 7f0a011f3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e80039c0 0x7f09fc1409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f0a011f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc143240 0x7f09fc1436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f0a011f3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09fc141450 0x7f09fc1418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f0a011f3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc06e460 con 0x7f09fc143240 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f0a011f3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc15b120 con 0x7f09e80039c0 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f0a011f3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc06e5e0 con 0x7f09fc141450 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc143240 0x7f09fc1436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc143240 0x7f09fc1436c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32944/0 (socket says 172.21.15.133:32944) 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f259c640 1 -- 172.21.15.133:0/2864161470 learned_addr learned my addr 172.21.15.133:0/2864161470 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09fb59e640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09fc141450 0x7f09fc1418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09fad9d640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e80039c0 0x7f09fc1409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f259c640 1 -- 172.21.15.133:0/2864161470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09fc141450 msgr2=0x7f09fc1418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f259c640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09fc141450 0x7f09fc1418d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f259c640 1 -- 172.21.15.133:0/2864161470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e80039c0 msgr2=0x7f09fc1409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f259c640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e80039c0 0x7f09fc1409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f259c640 1 -- 172.21.15.133:0/2864161470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc140ee0 con 0x7f09fc143240 2024-04-25T10:12:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f259c640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc143240 0x7f09fc1436c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f09e4010790 tx=0x7f09e4010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f3fff640 1 -- 172.21.15.133:0/2864161470 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e400f040 con 0x7f09fc143240 2024-04-25T10:12:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f3fff640 1 -- 172.21.15.133:0/2864161470 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e4004020 con 0x7f09fc143240 2024-04-25T10:12:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f0a011f3640 1 -- 172.21.15.133:0/2864161470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc142180 con 0x7f09fc143240 2024-04-25T10:12:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f3fff640 1 -- 172.21.15.133:0/2864161470 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4004410 con 0x7f09fc143240 2024-04-25T10:12:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f0a011f3640 1 -- 172.21.15.133:0/2864161470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc142740 con 0x7f09fc143240 2024-04-25T10:12:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f0a011f3640 1 -- 172.21.15.133:0/2864161470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc06bc20 con 0x7f09fc143240 2024-04-25T10:12:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f3fff640 1 -- 172.21.15.133:0/2864161470 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09e400e900 con 0x7f09fc143240 2024-04-25T10:12:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f3fff640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09dc066f90 0x7f09dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09f3fff640 1 -- 172.21.15.133:0/2864161470 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f09e4084fa0 con 0x7f09fc143240 2024-04-25T10:12:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09fad9d640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09dc066f90 0x7f09dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.657+0000 7f09fad9d640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09dc066f90 0x7f09dc069450 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0ab000 tx=0x7f09ec0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.661+0000 7f09f3fff640 1 -- 172.21.15.133:0/2864161470 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09e4018030 con 0x7f09fc143240 2024-04-25T10:12:11.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.821+0000 7f0a011f3640 1 -- 172.21.15.133:0/2864161470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09fc159420 con 0x7f09fc143240 2024-04-25T10:12:11.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.821+0000 7f09f3fff640 1 -- 172.21.15.133:0/2864161470 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+13327 (secure 0 0 0) 0x7f09e4056110 con 0x7f09fc143240 2024-04-25T10:12:11.825 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:11.825 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":112,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:12:10.488969+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6834/488739264":"2024-04-26T10:12:07.275462+0000","172.21.15.162:6832/488739264":"2024-04-26T10:12:07.275462+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:11.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 -- 172.21.15.133:0/2864161470 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09dc066f90 msgr2=0x7f09dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09dc066f90 0x7f09dc069450 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0ab000 tx=0x7f09ec0a7070 comp rx=0 tx=0).stop 2024-04-25T10:12:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 -- 172.21.15.133:0/2864161470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc143240 msgr2=0x7f09fc1436c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc143240 0x7f09fc1436c0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f09e4010790 tx=0x7f09e4010c50 comp rx=0 tx=0).stop 2024-04-25T10:12:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 -- 172.21.15.133:0/2864161470 shutdown_connections 2024-04-25T10:12:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f09fc141450 0x7f09fc1418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f09dc066f90 0x7f09dc069450 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc143240 0x7f09fc1436c0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 --2- 172.21.15.133:0/2864161470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09e80039c0 0x7f09fc1409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 -- 172.21.15.133:0/2864161470 >> 172.21.15.133:0/2864161470 conn(0x7f09fc067000 msgr2=0x7f09fc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 -- 172.21.15.133:0/2864161470 shutdown_connections 2024-04-25T10:12:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.825+0000 7f0a011f3640 1 -- 172.21.15.133:0/2864161470 wait complete. 2024-04-25T10:12:11.844 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:11.845 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:11.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5fcac640 1 Processor -- start 2024-04-25T10:12:11.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5fcac640 1 -- start start 2024-04-25T10:12:11.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5fcac640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9d58145170 0x7f9d5806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5fcac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d5815ced0 0x7f9d5815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5fcac640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d580ce2e0 con 0x7f9d5806e4d0 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5fcac640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5806db00 con 0x7f9d58145170 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5fcac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58145550 con 0x7f9d5815ced0 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5d247640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9d5806e4d0 0x7f9d580cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43286/0 (socket says 172.21.15.133:43286) 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5d247640 1 -- 172.21.15.133:0/326831131 learned_addr learned my addr 172.21.15.133:0/326831131 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5da48640 1 --2- 172.21.15.133:0/326831131 >> v2:172.21.15.162:3300/0 conn(0x7f9d58145170 0x7f9d5806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5e249640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d5815ced0 0x7f9d5815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047133174 0 0) 0x7f9d580ce2e0 con 0x7f9d5806e4d0 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d44003530 con 0x7f9d5806e4d0 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2544454412 0 0) 0x7f9d44003530 con 0x7f9d5806e4d0 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d5815ced0 msgr2=0x7f9d5815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d5815ced0 0x7f9d5815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 >> v2:172.21.15.162:3300/0 conn(0x7f9d58145170 msgr2=0x7f9d5806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 --2- 172.21.15.133:0/326831131 >> v2:172.21.15.162:3300/0 conn(0x7f9d58145170 0x7f9d5806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5806d4b0 con 0x7f9d5806e4d0 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f9d48001680 con 0x7f9d5806e4d0 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 >> v1:172.21.15.133:6789/0 conn(0x7f9d5806e4d0 legacy=0x7f9d580cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d440039c0 0x7f9d44003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 0x7f9d44004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d5815ced0 0x7f9d44004f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5806db00 con 0x7f9d44004370 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58145550 con 0x7f9d440039c0 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4400b9d0 con 0x7f9d5815ced0 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5d247640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 0x7f9d44004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5e249640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d5815ced0 0x7f9d44004f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5da48640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d440039c0 0x7f9d44003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5d247640 1 -- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d5815ced0 msgr2=0x7f9d44004f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5d247640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d5815ced0 0x7f9d44004f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5d247640 1 -- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d440039c0 msgr2=0x7f9d44003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5d247640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d440039c0 0x7f9d44003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5d247640 1 -- 172.21.15.133:0/326831131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d44003530 con 0x7f9d44004370 2024-04-25T10:12:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.953+0000 7f9d5da48640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d440039c0 0x7f9d44003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5d247640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 0x7f9d44004850 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f9d5806e930 tx=0x7f9d4801fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f780d2a0f709158d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48004cd0 con 0x7f9d44004370 2024-04-25T10:12:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d48005ae0 con 0x7f9d44004370 2024-04-25T10:12:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5ca46640 1 -- 172.21.15.133:0/326831131 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48008110 con 0x7f9d44004370 2024-04-25T10:12:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 -- 172.21.15.133:0/326831131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 msgr2=0x7f9d44004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 --2- 172.21.15.133:0/326831131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 0x7f9d44004850 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f9d5806e930 tx=0x7f9d4801fa90 comp rx=0 tx=0).stop 2024-04-25T10:12:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5e249640 1 -- 172.21.15.133:0/326831131 reap_dead start 2024-04-25T10:12:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 -- 172.21.15.133:0/326831131 shutdown_connections 2024-04-25T10:12:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 -- 172.21.15.133:0/326831131 >> 172.21.15.133:0/326831131 conn(0x7f9d58066f70 msgr2=0x7f9d5806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 -- 172.21.15.133:0/326831131 shutdown_connections 2024-04-25T10:12:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 -- 172.21.15.133:0/326831131 wait complete. 2024-04-25T10:12:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 Processor -- start 2024-04-25T10:12:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 -- start start 2024-04-25T10:12:11.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 0x7f9d5813f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d581f5170 0x7f9d58138df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d580a9ff0 0x7f9d5813dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d580cdf30 con 0x7f9d44004370 2024-04-25T10:12:11.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5815cb70 con 0x7f9d580a9ff0 2024-04-25T10:12:11.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5fcac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5813fd40 con 0x7f9d581f5170 2024-04-25T10:12:11.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5da48640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 0x7f9d5813f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5da48640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 0x7f9d5813f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32964/0 (socket says 172.21.15.133:32964) 2024-04-25T10:12:11.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.957+0000 7f9d5da48640 1 -- 172.21.15.133:0/1093048095 learned_addr learned my addr 172.21.15.133:0/1093048095 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5d247640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d581f5170 0x7f9d58138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5da48640 1 -- 172.21.15.133:0/1093048095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d581f5170 msgr2=0x7f9d58138df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5da48640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d581f5170 0x7f9d58138df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5e249640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d580a9ff0 0x7f9d5813dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5da48640 1 -- 172.21.15.133:0/1093048095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d580a9ff0 msgr2=0x7f9d5813dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5da48640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d580a9ff0 0x7f9d5813dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5da48640 1 -- 172.21.15.133:0/1093048095 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d581f55f0 con 0x7f9d44004370 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5da48640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 0x7f9d5813f610 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f9d500108d0 tx=0x7f9d50010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d4e7fc640 1 -- 172.21.15.133:0/1093048095 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5000f040 con 0x7f9d44004370 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d4e7fc640 1 -- 172.21.15.133:0/1093048095 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d50004220 con 0x7f9d44004370 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5fcac640 1 -- 172.21.15.133:0/1093048095 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5813d220 con 0x7f9d44004370 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d4e7fc640 1 -- 172.21.15.133:0/1093048095 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5000e960 con 0x7f9d44004370 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5fcac640 1 -- 172.21.15.133:0/1093048095 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5813d7e0 con 0x7f9d44004370 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5fcac640 1 -- 172.21.15.133:0/1093048095 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d58068e80 con 0x7f9d44004370 2024-04-25T10:12:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d4e7fc640 1 -- 172.21.15.133:0/1093048095 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d5000eb00 con 0x7f9d44004370 2024-04-25T10:12:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d4e7fc640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d3c066f90 0x7f9d3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d4e7fc640 1 -- 172.21.15.133:0/1093048095 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9d50085310 con 0x7f9d44004370 2024-04-25T10:12:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5d247640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d3c066f90 0x7f9d3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.961+0000 7f9d5d247640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d3c066f90 0x7f9d3c069450 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9d480ae000 tx=0x7f9d48004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:11.965+0000 7f9d4e7fc640 1 -- 172.21.15.133:0/1093048095 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d50018030 con 0x7f9d44004370 2024-04-25T10:12:12.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 -- 172.21.15.133:0/1093048095 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d581600e0 con 0x7f9d44004370 2024-04-25T10:12:12.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d4e7fc640 1 -- 172.21.15.133:0/1093048095 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d50056490 con 0x7f9d44004370 2024-04-25T10:12:12.115 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 -- 172.21.15.133:0/1093048095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d3c066f90 msgr2=0x7f9d3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d3c066f90 0x7f9d3c069450 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9d480ae000 tx=0x7f9d48004ac0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 -- 172.21.15.133:0/1093048095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 msgr2=0x7f9d5813f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 0x7f9d5813f610 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f9d500108d0 tx=0x7f9d50010d90 comp rx=0 tx=0).stop 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 -- 172.21.15.133:0/1093048095 shutdown_connections 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d580a9ff0 0x7f9d5813dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d581f5170 0x7f9d58138df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d3c066f90 0x7f9d3c069450 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 --2- 172.21.15.133:0/1093048095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d44004370 0x7f9d5813f610 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 -- 172.21.15.133:0/1093048095 >> 172.21.15.133:0/1093048095 conn(0x7f9d58066f70 msgr2=0x7f9d580b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 -- 172.21.15.133:0/1093048095 shutdown_connections 2024-04-25T10:12:12.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.113+0000 7f9d5fcac640 1 -- 172.21.15.133:0/1093048095 wait complete. 2024-04-25T10:12:12.133 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:12:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bccf20640 1 Processor -- start 2024-04-25T10:12:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bccf20640 1 -- start start 2024-04-25T10:12:12.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bccf20640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8bc815c9a0 0x7f8bc80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bccf20640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8bc80b1f70 0x7f8bc806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bccf20640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc80ce330 0x7f8bc80ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bccf20640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc80cecc0 con 0x7f8bc815c9a0 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bccf20640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc80c61d0 con 0x7f8bc80b1f70 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bccf20640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc80c64b0 con 0x7f8bc80ce330 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bc659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8bc815c9a0 0x7f8bc80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bc659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8bc815c9a0 0x7f8bc80c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32966/0 (socket says 172.21.15.133:32966) 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bc659c640 1 -- 172.21.15.133:0/3052412517 learned_addr learned my addr 172.21.15.133:0/3052412517 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bc6d9d640 1 --2- 172.21.15.133:0/3052412517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc80ce330 0x7f8bc80ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bc659c640 1 -- 172.21.15.133:0/3052412517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc80ce330 msgr2=0x7f8bc80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bc5d9b640 1 --2- 172.21.15.133:0/3052412517 >> v2:172.21.15.162:3300/0 conn(0x7f8bc80b1f70 0x7f8bc806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bc659c640 1 --2- 172.21.15.133:0/3052412517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc80ce330 0x7f8bc80ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bc659c640 1 -- 172.21.15.133:0/3052412517 >> v2:172.21.15.162:3300/0 conn(0x7f8bc80b1f70 msgr2=0x7f8bc806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bc659c640 1 --2- 172.21.15.133:0/3052412517 >> v2:172.21.15.162:3300/0 conn(0x7f8bc80b1f70 0x7f8bc806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.245+0000 7f8bc659c640 1 -- 172.21.15.133:0/3052412517 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc806d4f0 con 0x7f8bc815c9a0 2024-04-25T10:12:12.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bc659c640 1 --2- 172.21.15.133:0/3052412517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc815c9a0 0x7f8bc80c5980 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8072670 tx=0x7f8bb801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97be028598397248 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:12.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bc559a640 1 -- 172.21.15.133:0/3052412517 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80b1070 con 0x7f8bc815c9a0 2024-04-25T10:12:12.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bc559a640 1 -- 172.21.15.133:0/3052412517 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb80ac620 con 0x7f8bc815c9a0 2024-04-25T10:12:12.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bc559a640 1 -- 172.21.15.133:0/3052412517 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80ac9f0 con 0x7f8bc815c9a0 2024-04-25T10:12:12.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bccf20640 1 -- 172.21.15.133:0/3052412517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc815c9a0 msgr2=0x7f8bc80c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bccf20640 1 --2- 172.21.15.133:0/3052412517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc815c9a0 0x7f8bc80c5980 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8072670 tx=0x7f8bb801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bccf20640 1 -- 172.21.15.133:0/3052412517 shutdown_connections 2024-04-25T10:12:12.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bccf20640 1 --2- 172.21.15.133:0/3052412517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc80ce330 0x7f8bc80ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bccf20640 1 --2- 172.21.15.133:0/3052412517 >> v2:172.21.15.162:3300/0 conn(0x7f8bc80b1f70 0x7f8bc806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bccf20640 1 --2- 172.21.15.133:0/3052412517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc815c9a0 0x7f8bc80c5980 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bccf20640 1 -- 172.21.15.133:0/3052412517 >> 172.21.15.133:0/3052412517 conn(0x7f8bc8067000 msgr2=0x7f8bc806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bccf20640 1 -- 172.21.15.133:0/3052412517 shutdown_connections 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bccf20640 1 -- 172.21.15.133:0/3052412517 wait complete. 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bccf20640 1 Processor -- start 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.249+0000 7f8bccf20640 1 -- start start 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bccf20640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc80b1f70 0x7f8bc8144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bccf20640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc80ce330 0x7f8bc80b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bccf20640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc815c9a0 0x7f8bc80b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bccf20640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc815cf00 con 0x7f8bc80ce330 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bccf20640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc815cd80 con 0x7f8bc815c9a0 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bccf20640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc815d950 con 0x7f8bc80b1f70 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc80ce330 0x7f8bc80b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc80ce330 0x7f8bc80b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32968/0 (socket says 172.21.15.133:32968) 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc5d9b640 1 -- 172.21.15.133:0/194776957 learned_addr learned my addr 172.21.15.133:0/194776957 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc5d9b640 1 -- 172.21.15.133:0/194776957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc80b1f70 msgr2=0x7f8bc8144e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc6d9d640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc815c9a0 0x7f8bc80b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc5d9b640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc80b1f70 0x7f8bc8144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc5d9b640 1 -- 172.21.15.133:0/194776957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc815c9a0 msgr2=0x7f8bc80b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc5d9b640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc815c9a0 0x7f8bc80b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc5d9b640 1 -- 172.21.15.133:0/194776957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc8166080 con 0x7f8bc80ce330 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc5d9b640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc80ce330 0x7f8bc80b3310 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000c7f0 tx=0x7f8bb000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bbeffd640 1 -- 172.21.15.133:0/194776957 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0015070 con 0x7f8bc80ce330 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bbeffd640 1 -- 172.21.15.133:0/194776957 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb0004220 con 0x7f8bc80ce330 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bbeffd640 1 -- 172.21.15.133:0/194776957 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0003620 con 0x7f8bc80ce330 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bccf20640 1 -- 172.21.15.133:0/194776957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc8153dc0 con 0x7f8bc80ce330 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bccf20640 1 -- 172.21.15.133:0/194776957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc8166360 con 0x7f8bc80ce330 2024-04-25T10:12:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bccf20640 1 -- 172.21.15.133:0/194776957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc806bc20 con 0x7f8bc80ce330 2024-04-25T10:12:12.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bbeffd640 1 -- 172.21.15.133:0/194776957 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8bb00039d0 con 0x7f8bc80ce330 2024-04-25T10:12:12.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bbeffd640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ba8066f60 0x7f8ba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc659c640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ba8066f60 0x7f8ba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bbeffd640 1 -- 172.21.15.133:0/194776957 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8bb0085080 con 0x7f8bc80ce330 2024-04-25T10:12:12.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.253+0000 7f8bc659c640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ba8066f60 0x7f8ba8069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80ae000 tx=0x7f8bb8004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:12.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.261+0000 7f8bbeffd640 1 -- 172.21.15.133:0/194776957 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8bb00560d0 con 0x7f8bc80ce330 2024-04-25T10:12:12.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.457+0000 7f8bccf20640 1 -- 172.21.15.133:0/194776957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f8bc81562a0 con 0x7f8bc80ce330 2024-04-25T10:12:12.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.505+0000 7f8bbeffd640 1 -- 172.21.15.133:0/194776957 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v463) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8bb00559d0 con 0x7f8bc80ce330 2024-04-25T10:12:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 -- 172.21.15.133:0/194776957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ba8066f60 msgr2=0x7f8ba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ba8066f60 0x7f8ba8069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80ae000 tx=0x7f8bb8004740 comp rx=0 tx=0).stop 2024-04-25T10:12:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 -- 172.21.15.133:0/194776957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc80ce330 msgr2=0x7f8bc80b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc80ce330 0x7f8bc80b3310 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000c7f0 tx=0x7f8bb000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 -- 172.21.15.133:0/194776957 shutdown_connections 2024-04-25T10:12:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bc815c9a0 0x7f8bc80b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ba8066f60 0x7f8ba8069420 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bc80ce330 0x7f8bc80b3310 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 --2- 172.21.15.133:0/194776957 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8bc80b1f70 0x7f8bc8144e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 -- 172.21.15.133:0/194776957 >> 172.21.15.133:0/194776957 conn(0x7f8bc8067000 msgr2=0x7f8bc81567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:12.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 -- 172.21.15.133:0/194776957 shutdown_connections 2024-04-25T10:12:12.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.509+0000 7f8bccf20640 1 -- 172.21.15.133:0/194776957 wait complete. 2024-04-25T10:12:12.529 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:12:12.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2caf85b640 1 Processor -- start 2024-04-25T10:12:12.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2caf85b640 1 -- start start 2024-04-25T10:12:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2caf85b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ca815d9a0 0x7f2ca80d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2caf85b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2ca80ce600 0x7f2ca80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2caf85b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2ca80d1920 0x7f2ca80d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2caf85b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca80cefa0 con 0x7f2ca80ce600 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2caf85b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca815ebd0 con 0x7f2ca80d1920 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2caf85b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca815eeb0 con 0x7f2ca815d9a0 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2cacdf6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2ca80ce600 0x7f2ca80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2cacdf6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2ca80ce600 0x7f2ca80cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32982/0 (socket says 172.21.15.133:32982) 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2cacdf6640 1 -- 172.21.15.133:0/1206423046 learned_addr learned my addr 172.21.15.133:0/1206423046 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2caddf8640 1 --2- 172.21.15.133:0/1206423046 >> v2:172.21.15.162:3300/0 conn(0x7f2ca80d1920 0x7f2ca80d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2cad5f7640 1 --2- 172.21.15.133:0/1206423046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ca815d9a0 0x7f2ca80d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2cacdf6640 1 -- 172.21.15.133:0/1206423046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ca815d9a0 msgr2=0x7f2ca80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2cacdf6640 1 --2- 172.21.15.133:0/1206423046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ca815d9a0 0x7f2ca80d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2cacdf6640 1 -- 172.21.15.133:0/1206423046 >> v2:172.21.15.162:3300/0 conn(0x7f2ca80d1920 msgr2=0x7f2ca80d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2cacdf6640 1 --2- 172.21.15.133:0/1206423046 >> v2:172.21.15.162:3300/0 conn(0x7f2ca80d1920 0x7f2ca80d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.649+0000 7f2cacdf6640 1 -- 172.21.15.133:0/1206423046 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80d0330 con 0x7f2ca80ce600 2024-04-25T10:12:12.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cacdf6640 1 --2- 172.21.15.133:0/1206423046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ca80ce600 0x7f2ca80cea60 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f2ca81615d0 tx=0x7f2c9801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb2c10b3875c580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2c9ffff640 1 -- 172.21.15.133:0/1206423046 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980ab600 con 0x7f2ca80ce600 2024-04-25T10:12:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2c9ffff640 1 -- 172.21.15.133:0/1206423046 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c980b0070 con 0x7f2ca80ce600 2024-04-25T10:12:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2c9ffff640 1 -- 172.21.15.133:0/1206423046 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980ab9c0 con 0x7f2ca80ce600 2024-04-25T10:12:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 -- 172.21.15.133:0/1206423046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ca80ce600 msgr2=0x7f2ca80cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 --2- 172.21.15.133:0/1206423046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ca80ce600 0x7f2ca80cea60 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f2ca81615d0 tx=0x7f2c9801c460 comp rx=0 tx=0).stop 2024-04-25T10:12:12.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 -- 172.21.15.133:0/1206423046 shutdown_connections 2024-04-25T10:12:12.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 --2- 172.21.15.133:0/1206423046 >> v2:172.21.15.162:3300/0 conn(0x7f2ca80d1920 0x7f2ca80d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 --2- 172.21.15.133:0/1206423046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ca80ce600 0x7f2ca80cea60 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 --2- 172.21.15.133:0/1206423046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ca815d9a0 0x7f2ca80d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 -- 172.21.15.133:0/1206423046 >> 172.21.15.133:0/1206423046 conn(0x7f2ca806bf80 msgr2=0x7f2ca806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:12.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 -- 172.21.15.133:0/1206423046 shutdown_connections 2024-04-25T10:12:12.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 -- 172.21.15.133:0/1206423046 wait complete. 2024-04-25T10:12:12.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 Processor -- start 2024-04-25T10:12:12.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 -- start start 2024-04-25T10:12:12.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca80ce600 0x7f2ca80c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ca80d1920 0x7f2ca81557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ca815d9a0 0x7f2ca80b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80b9820 con 0x7f2ca80d1920 2024-04-25T10:12:12.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80b96a0 con 0x7f2ca80ce600 2024-04-25T10:12:12.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caf85b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80d22f0 con 0x7f2ca815d9a0 2024-04-25T10:12:12.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cacdf6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ca80d1920 0x7f2ca81557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cacdf6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ca80d1920 0x7f2ca81557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32994/0 (socket says 172.21.15.133:32994) 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cacdf6640 1 -- 172.21.15.133:0/1043058802 learned_addr learned my addr 172.21.15.133:0/1043058802 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2caddf8640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ca815d9a0 0x7f2ca80b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cad5f7640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca80ce600 0x7f2ca80c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cacdf6640 1 -- 172.21.15.133:0/1043058802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ca815d9a0 msgr2=0x7f2ca80b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cacdf6640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ca815d9a0 0x7f2ca80b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cacdf6640 1 -- 172.21.15.133:0/1043058802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca80ce600 msgr2=0x7f2ca80c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cacdf6640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca80ce600 0x7f2ca80c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cacdf6640 1 -- 172.21.15.133:0/1043058802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80c9c40 con 0x7f2ca80d1920 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cad5f7640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca80ce600 0x7f2ca80c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.653+0000 7f2cacdf6640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ca80d1920 0x7f2ca81557b0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2c9801ab10 tx=0x7f2c9801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:12.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.657+0000 7f2c9dffb640 1 -- 172.21.15.133:0/1043058802 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980b4800 con 0x7f2ca80d1920 2024-04-25T10:12:12.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.657+0000 7f2c9dffb640 1 -- 172.21.15.133:0/1043058802 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c980b49a0 con 0x7f2ca80d1920 2024-04-25T10:12:12.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.657+0000 7f2caf85b640 1 -- 172.21.15.133:0/1043058802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca80c9f20 con 0x7f2ca80d1920 2024-04-25T10:12:12.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.657+0000 7f2c9dffb640 1 -- 172.21.15.133:0/1043058802 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980b4d90 con 0x7f2ca80d1920 2024-04-25T10:12:12.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.657+0000 7f2caf85b640 1 -- 172.21.15.133:0/1043058802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca80c8480 con 0x7f2ca80d1920 2024-04-25T10:12:12.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.657+0000 7f2caf85b640 1 -- 172.21.15.133:0/1043058802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca8068e80 con 0x7f2ca80d1920 2024-04-25T10:12:12.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.657+0000 7f2c9dffb640 1 -- 172.21.15.133:0/1043058802 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c980aa070 con 0x7f2ca80d1920 2024-04-25T10:12:12.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.657+0000 7f2c9dffb640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c8c066f60 0x7f2c8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:12.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.657+0000 7f2c9dffb640 1 -- 172.21.15.133:0/1043058802 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2c9811f3d0 con 0x7f2ca80d1920 2024-04-25T10:12:12.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.657+0000 7f2cad5f7640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c8c066f60 0x7f2c8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:12.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.657+0000 7f2cad5f7640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c8c066f60 0x7f2c8c069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4006fd0 tx=0x7f2ca4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:12.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.661+0000 7f2c9dffb640 1 -- 172.21.15.133:0/1043058802 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c98016110 con 0x7f2ca80d1920 2024-04-25T10:12:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.809+0000 7f2caf85b640 1 -- 172.21.15.133:0/1043058802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2ca806af40 con 0x7f2ca80d1920 2024-04-25T10:12:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.813+0000 7f2c9dffb640 1 -- 172.21.15.133:0/1043058802 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2c980f0540 con 0x7f2ca80d1920 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 -- 172.21.15.133:0/1043058802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c8c066f60 msgr2=0x7f2c8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c8c066f60 0x7f2c8c069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4006fd0 tx=0x7f2ca4008040 comp rx=0 tx=0).stop 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 -- 172.21.15.133:0/1043058802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ca80d1920 msgr2=0x7f2ca81557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ca80d1920 0x7f2ca81557b0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2c9801ab10 tx=0x7f2c9801aca0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 -- 172.21.15.133:0/1043058802 shutdown_connections 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2ca815d9a0 0x7f2ca80b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2c8c066f60 0x7f2c8c069420 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ca80d1920 0x7f2ca81557b0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 --2- 172.21.15.133:0/1043058802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2ca80ce600 0x7f2ca80c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 -- 172.21.15.133:0/1043058802 >> 172.21.15.133:0/1043058802 conn(0x7f2ca806bf80 msgr2=0x7f2ca80b7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 -- 172.21.15.133:0/1043058802 shutdown_connections 2024-04-25T10:12:12.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:12.817+0000 7f2caf85b640 1 -- 172.21.15.133:0/1043058802 wait complete. 2024-04-25T10:12:12.834 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:12:12.834 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:12.896 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:12.943 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:12:12.943 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:12:12.943 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:12.949 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:12.949 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:12:12.949 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:12.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:12.955 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:12:12.955 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:12.995 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:12.995 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:12:12.995 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:13.002 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:13.002 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:12:13.002 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:13.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3e018c640 1 Processor -- start 2024-04-25T10:12:13.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3e018c640 1 -- start start 2024-04-25T10:12:13.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3e018c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb3d815da80 0x7fb3d8162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3e018c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d815eb50 con 0x7fb3d815e570 2024-04-25T10:12:13.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3e018c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d815ee30 con 0x7fb3d80ce6e0 2024-04-25T10:12:13.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3e018c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d815f110 con 0x7fb3d815da80 2024-04-25T10:12:13.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3de729640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb3d815e570 0x7fb3d80d1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43300/0 (socket says 172.21.15.133:43300) 2024-04-25T10:12:13.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3de729640 1 -- 172.21.15.133:0/2555851105 learned_addr learned my addr 172.21.15.133:0/2555851105 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:13.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3ddf28640 1 --2- 172.21.15.133:0/2555851105 >> v2:172.21.15.162:3300/0 conn(0x7fb3d815da80 0x7fb3d8162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812225436 0 0) 0x7fb3d815eb50 con 0x7fb3d815e570 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3c4003530 con 0x7fb3d815e570 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1192297138 0 0) 0x7fb3c4003530 con 0x7fb3d815e570 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 >> v2:172.21.15.162:3300/0 conn(0x7fb3d815da80 msgr2=0x7fb3d8162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 --2- 172.21.15.133:0/2555851105 >> v2:172.21.15.162:3300/0 conn(0x7fb3d815da80 0x7fb3d8162090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 >> v1:172.21.15.162:6789/0 conn(0x7fb3d80ce6e0 legacy=0x7fb3d815de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d8161890 con 0x7fb3d815e570 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb3d0001a00 con 0x7fb3d815e570 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 >> v1:172.21.15.133:6789/0 conn(0x7fb3d815e570 legacy=0x7fb3d80d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 --2- 172.21.15.133:0/2555851105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c40039c0 0x7fb3c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 --2- 172.21.15.133:0/2555851105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c4004810 0x7fb3c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 --2- 172.21.15.133:0/2555851105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c40051d0 0x7fb3c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c400c040 con 0x7fb3c4004810 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c400c1f0 con 0x7fb3c40039c0 2024-04-25T10:12:13.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c400c3f0 con 0x7fb3c40051d0 2024-04-25T10:12:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dd727640 1 --2- 172.21.15.133:0/2555851105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c4004810 0x7fb3c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3ddf28640 1 --2- 172.21.15.133:0/2555851105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c40039c0 0x7fb3c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3de729640 1 --2- 172.21.15.133:0/2555851105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c40051d0 0x7fb3c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dd727640 1 -- 172.21.15.133:0/2555851105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c40051d0 msgr2=0x7fb3c400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dd727640 1 --2- 172.21.15.133:0/2555851105 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c40051d0 0x7fb3c400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dd727640 1 -- 172.21.15.133:0/2555851105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c40039c0 msgr2=0x7fb3c4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dd727640 1 --2- 172.21.15.133:0/2555851105 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3c40039c0 0x7fb3c4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3dd727640 1 -- 172.21.15.133:0/2555851105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c4003530 con 0x7fb3c4004810 2024-04-25T10:12:13.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.157+0000 7fb3de729640 1 -- 172.21.15.133:0/2555851105 reap_dead start 2024-04-25T10:12:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3dd727640 1 --2- 172.21.15.133:0/2555851105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c4004810 0x7fb3c4004c90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4001780 tx=0x7fb3d401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be056187aeb726 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40013a0 con 0x7fb3c4004810 2024-04-25T10:12:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d40052f0 con 0x7fb3c4004810 2024-04-25T10:12:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3dcf26640 1 -- 172.21.15.133:0/2555851105 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4003840 con 0x7fb3c4004810 2024-04-25T10:12:13.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 -- 172.21.15.133:0/2555851105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c4004810 msgr2=0x7fb3c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 --2- 172.21.15.133:0/2555851105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c4004810 0x7fb3c4004c90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4001780 tx=0x7fb3d401c400 comp rx=0 tx=0).stop 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 -- 172.21.15.133:0/2555851105 shutdown_connections 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 --2- 172.21.15.133:0/2555851105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c4004810 0x7fb3c4004c90 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 -- 172.21.15.133:0/2555851105 >> 172.21.15.133:0/2555851105 conn(0x7fb3d806c0d0 msgr2=0x7fb3d806c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 -- 172.21.15.133:0/2555851105 shutdown_connections 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 -- 172.21.15.133:0/2555851105 wait complete. 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 Processor -- start 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 -- start start 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c4004810 0x7fb3d8154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d8158c10 0x7fb3d8159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3d80b74a0 0x7fb3d80b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80b9660 con 0x7fb3d80b74a0 2024-04-25T10:12:13.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80ceb90 con 0x7fb3d8158c10 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3e018c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80c8bb0 con 0x7fb3c4004810 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3de729640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3d80b74a0 0x7fb3d80b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3de729640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3d80b74a0 0x7fb3d80b7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33022/0 (socket says 172.21.15.133:33022) 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.161+0000 7fb3de729640 1 -- 172.21.15.133:0/4271234857 learned_addr learned my addr 172.21.15.133:0/4271234857 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3ddf28640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c4004810 0x7fb3d8154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3dd727640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d8158c10 0x7fb3d8159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3de729640 1 -- 172.21.15.133:0/4271234857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c4004810 msgr2=0x7fb3d8154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3de729640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c4004810 0x7fb3d8154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3de729640 1 -- 172.21.15.133:0/4271234857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d8158c10 msgr2=0x7fb3d8159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3de729640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d8158c10 0x7fb3d8159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3de729640 1 -- 172.21.15.133:0/4271234857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d8157350 con 0x7fb3d80b74a0 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3dd727640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d8158c10 0x7fb3d8159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3de729640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3d80b74a0 0x7fb3d80b7950 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0004c00 tx=0x7fb3d0005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3ce7fc640 1 -- 172.21.15.133:0/4271234857 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d001a070 con 0x7fb3d80b74a0 2024-04-25T10:12:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3ce7fc640 1 -- 172.21.15.133:0/4271234857 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d0011040 con 0x7fb3d80b74a0 2024-04-25T10:12:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3ce7fc640 1 -- 172.21.15.133:0/4271234857 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000ee60 con 0x7fb3d80b74a0 2024-04-25T10:12:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3e018c640 1 -- 172.21.15.133:0/4271234857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d8157630 con 0x7fb3d80b74a0 2024-04-25T10:12:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3e018c640 1 -- 172.21.15.133:0/4271234857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d8157bf0 con 0x7fb3d80b74a0 2024-04-25T10:12:13.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3e018c640 1 -- 172.21.15.133:0/4271234857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d8067c70 con 0x7fb3d80b74a0 2024-04-25T10:12:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3ce7fc640 1 -- 172.21.15.133:0/4271234857 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3d001d070 con 0x7fb3d80b74a0 2024-04-25T10:12:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3ce7fc640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3bc066f90 0x7fb3bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3ce7fc640 1 -- 172.21.15.133:0/4271234857 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb3d0087530 con 0x7fb3d80b74a0 2024-04-25T10:12:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3ddf28640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3bc066f90 0x7fb3bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.165+0000 7fb3ddf28640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3bc066f90 0x7fb3bc069450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb3d804f450 tx=0x7fb3c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.169+0000 7fb3ce7fc640 1 -- 172.21.15.133:0/4271234857 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3d00586b0 con 0x7fb3d80b74a0 2024-04-25T10:12:13.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.353+0000 7fb3e018c640 1 -- 172.21.15.133:0/4271234857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3d80c89a0 con 0x7fb3d80b74a0 2024-04-25T10:12:13.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3ce7fc640 1 -- 172.21.15.133:0/4271234857 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fb3d0057fb0 con 0x7fb3d80b74a0 2024-04-25T10:12:13.358 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:13.358 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/3148817103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3148817103},{"type":"v1","addr":"172.21.15.162:6834","nonce":3148817103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/571094507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":571094507},{"type":"v1","addr":"172.21.15.162:6835","nonce":571094507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1193861341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1193861341},{"type":"v1","addr":"172.21.15.133:6835","nonce":1193861341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2360591204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2360591204},{"type":"v1","addr":"172.21.15.133:6837","nonce":2360591204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[]} 2024-04-25T10:12:13.358 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 67 2024-04-25T10:12:13.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 -- 172.21.15.133:0/4271234857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3bc066f90 msgr2=0x7fb3bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3bc066f90 0x7fb3bc069450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb3d804f450 tx=0x7fb3c8008040 comp rx=0 tx=0).stop 2024-04-25T10:12:13.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 -- 172.21.15.133:0/4271234857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3d80b74a0 msgr2=0x7fb3d80b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3d80b74a0 0x7fb3d80b7950 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0004c00 tx=0x7fb3d0005a60 comp rx=0 tx=0).stop 2024-04-25T10:12:13.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 -- 172.21.15.133:0/4271234857 shutdown_connections 2024-04-25T10:12:13.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3bc066f90 0x7fb3bc069450 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3d80b74a0 0x7fb3d80b7950 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3d8158c10 0x7fb3d8159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 --2- 172.21.15.133:0/4271234857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3c4004810 0x7fb3d8154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 -- 172.21.15.133:0/4271234857 >> 172.21.15.133:0/4271234857 conn(0x7fb3d806c0d0 msgr2=0x7fb3d8155de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:13.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 -- 172.21.15.133:0/4271234857 shutdown_connections 2024-04-25T10:12:13.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.357+0000 7fb3e018c640 1 -- 172.21.15.133:0/4271234857 wait complete. 2024-04-25T10:12:13.376 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:12:13.376 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:12:13.376 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:12:13.377 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:13.377 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:12:13.377 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:12:13.377 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:12:13.377 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:13.377 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:12:13.377 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:12:13.378 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:12:13.378 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:13.378 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:12:13.378 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:12:13.378 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:12:13.379 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:13.386 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:12:13.386 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:12:13.388 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:12:13.388 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:12:13.401 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:12:13.402 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:12:13.402 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:12:13.403 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:12:13.431 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:12:13.431 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:13.432 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:12:13.433 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:13.446 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:12:13.446 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:13.447 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:12:13.447 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:13.448 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:12:13.489 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:12:13.489 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:12:13.521 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:12:13.521 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:12:13.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f6828d46640 1 Processor -- start 2024-04-25T10:12:13.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f6828d46640 1 -- start start 2024-04-25T10:12:13.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f6828d46640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68240caa90 0x7f68241666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f6828d46640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68240cde40 con 0x7f682415c000 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f6828d46640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68240ce120 con 0x7f682415caf0 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f6828d46640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68240ce400 con 0x7f68240caa90 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f682259c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f682415c000 0x7f682415c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43308/0 (socket says 172.21.15.133:43308) 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f682259c640 1 -- 172.21.15.133:0/653366570 learned_addr learned my addr 172.21.15.133:0/653366570 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f6822d9d640 1 --2- 172.21.15.133:0/653366570 >> v2:172.21.15.162:3300/0 conn(0x7f68240caa90 0x7f68241666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f682159a640 1 -- 172.21.15.133:0/653366570 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851364178 0 0) 0x7f68240cde40 con 0x7f682415c000 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f682159a640 1 -- 172.21.15.133:0/653366570 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f680c003530 con 0x7f682415c000 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f682159a640 1 -- 172.21.15.133:0/653366570 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521810287 0 0) 0x7f68240ce120 con 0x7f682415caf0 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f682159a640 1 -- 172.21.15.133:0/653366570 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68240cde40 con 0x7f682415caf0 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f6822d9d640 1 -- 172.21.15.133:0/653366570 >> v1:172.21.15.162:6789/0 conn(0x7f682415caf0 legacy=0x7f68240ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f682159a640 1 -- 172.21.15.133:0/653366570 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1469473845 0 0) 0x7f680c003530 con 0x7f682415c000 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f6822d9d640 1 -- 172.21.15.133:0/653366570 >> v1:172.21.15.133:6789/0 conn(0x7f682415c000 legacy=0x7f682415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.569+0000 7f6822d9d640 1 -- 172.21.15.133:0/653366570 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68240ce6e0 con 0x7f68240caa90 2024-04-25T10:12:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6822d9d640 1 --2- 172.21.15.133:0/653366570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68240caa90 0x7f68241666b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f681c009d90 tx=0x7f681c01c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=370e04789657b01a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f682159a640 1 -- 172.21.15.133:0/653366570 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c0b2070 con 0x7f68240caa90 2024-04-25T10:12:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f682159a640 1 -- 172.21.15.133:0/653366570 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c0a5640 con 0x7f68240caa90 2024-04-25T10:12:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f682159a640 1 -- 172.21.15.133:0/653366570 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814001680 con 0x7f68240caa90 2024-04-25T10:12:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 -- 172.21.15.133:0/653366570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68240caa90 msgr2=0x7f68241666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 --2- 172.21.15.133:0/653366570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68240caa90 0x7f68241666b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f681c009d90 tx=0x7f681c01c860 comp rx=0 tx=0).stop 2024-04-25T10:12:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 -- 172.21.15.133:0/653366570 shutdown_connections 2024-04-25T10:12:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 --2- 172.21.15.133:0/653366570 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68240caa90 0x7f68241666b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 -- 172.21.15.133:0/653366570 >> 172.21.15.133:0/653366570 conn(0x7f68240cc7b0 msgr2=0x7f68240ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 -- 172.21.15.133:0/653366570 shutdown_connections 2024-04-25T10:12:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 -- 172.21.15.133:0/653366570 wait complete. 2024-04-25T10:12:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 Processor -- start 2024-04-25T10:12:13.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 -- start start 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68240caa90 0x7f6824146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f682415c000 0x7f6824152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f682415caf0 0x7f68240b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6824169590 con 0x7f682415c000 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6824169410 con 0x7f68240caa90 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6828d46640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6824169710 con 0x7f682415caf0 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6821d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f682415c000 0x7f6824152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6821d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f682415c000 0x7f6824152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33046/0 (socket says 172.21.15.133:33046) 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6821d9b640 1 -- 172.21.15.133:0/1891306907 learned_addr learned my addr 172.21.15.133:0/1891306907 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6822d9d640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f682415caf0 0x7f68240b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f682259c640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68240caa90 0x7f6824146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6821d9b640 1 -- 172.21.15.133:0/1891306907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f682415caf0 msgr2=0x7f68240b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6821d9b640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f682415caf0 0x7f68240b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6821d9b640 1 -- 172.21.15.133:0/1891306907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68240caa90 msgr2=0x7f6824146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6821d9b640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68240caa90 0x7f6824146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.573+0000 7f6821d9b640 1 -- 172.21.15.133:0/1891306907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68241519a0 con 0x7f682415c000 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f6821d9b640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f682415c000 0x7f6824152760 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f681800c430 tx=0x7f681800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f6812ffd640 1 -- 172.21.15.133:0/1891306907 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818017070 con 0x7f682415c000 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f6812ffd640 1 -- 172.21.15.133:0/1891306907 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68180052f0 con 0x7f682415c000 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f6828d46640 1 -- 172.21.15.133:0/1891306907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6824151c80 con 0x7f682415c000 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f6812ffd640 1 -- 172.21.15.133:0/1891306907 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818003840 con 0x7f682415c000 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f6828d46640 1 -- 172.21.15.133:0/1891306907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6824152240 con 0x7f682415c000 2024-04-25T10:12:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f6828d46640 1 -- 172.21.15.133:0/1891306907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6824067c70 con 0x7f682415c000 2024-04-25T10:12:13.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f6812ffd640 1 -- 172.21.15.133:0/1891306907 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f681801a070 con 0x7f682415c000 2024-04-25T10:12:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f6812ffd640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6804066f60 0x7f6804069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f6812ffd640 1 -- 172.21.15.133:0/1891306907 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6818084240 con 0x7f682415c000 2024-04-25T10:12:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f682259c640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6804066f60 0x7f6804069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.577+0000 7f682259c640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6804066f60 0x7f6804069420 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f68140011f0 tx=0x7f68140016a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.581+0000 7f6812ffd640 1 -- 172.21.15.133:0/1891306907 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6818054d60 con 0x7f682415c000 2024-04-25T10:12:13.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:13.737+0000 7f6828d46640 1 -- 172.21.15.133:0/1891306907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f68240cc550 con 0x7f682415c000 2024-04-25T10:12:14.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.533+0000 7f6812ffd640 1 -- 172.21.15.133:0/1891306907 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v114) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6818058e00 con 0x7f682415c000 2024-04-25T10:12:14.536 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:12:14.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 -- 172.21.15.133:0/1891306907 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6804066f60 msgr2=0x7f6804069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:14.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6804066f60 0x7f6804069420 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f68140011f0 tx=0x7f68140016a0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 -- 172.21.15.133:0/1891306907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f682415c000 msgr2=0x7f6824152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:14.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f682415c000 0x7f6824152760 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f681800c430 tx=0x7f681800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 -- 172.21.15.133:0/1891306907 shutdown_connections 2024-04-25T10:12:14.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f682415caf0 0x7f68240b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6804066f60 0x7f6804069420 secure :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f68140011f0 tx=0x7f68140016a0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f682415c000 0x7f6824152760 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 --2- 172.21.15.133:0/1891306907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68240caa90 0x7f6824146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 -- 172.21.15.133:0/1891306907 >> 172.21.15.133:0/1891306907 conn(0x7f68240cc7b0 msgr2=0x7f68240b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:14.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 -- 172.21.15.133:0/1891306907 shutdown_connections 2024-04-25T10:12:14.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.537+0000 7f6828d46640 1 -- 172.21.15.133:0/1891306907 wait complete. 2024-04-25T10:12:14.560 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:12:14.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.721+0000 7f794838e640 1 Processor -- start 2024-04-25T10:12:14.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.721+0000 7f794838e640 1 -- start start 2024-04-25T10:12:14.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794838e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79400b1f70 0x7f794006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:14.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794838e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79400ce330 0x7f79400ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:14.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794838e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79400c61d0 con 0x7f794015c9a0 2024-04-25T10:12:14.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794838e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79400cece0 con 0x7f79400ce330 2024-04-25T10:12:14.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794838e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79400c64b0 con 0x7f79400b1f70 2024-04-25T10:12:14.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794692b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79400ce330 0x7f79400ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:14.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794612a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f794015c9a0 0x7f79400c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43314/0 (socket says 172.21.15.133:43314) 2024-04-25T10:12:14.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794612a640 1 -- 172.21.15.133:0/4156159277 learned_addr learned my addr 172.21.15.133:0/4156159277 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:14.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f7945929640 1 --2- 172.21.15.133:0/4156159277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79400b1f70 0x7f794006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:14.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794692b640 1 -- 172.21.15.133:0/4156159277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79400b1f70 msgr2=0x7f794006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:14.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794692b640 1 --2- 172.21.15.133:0/4156159277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79400b1f70 0x7f794006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f7945128640 1 -- 172.21.15.133:0/4156159277 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287508782 0 0) 0x7f79400c61d0 con 0x7f794015c9a0 2024-04-25T10:12:14.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794692b640 1 -- 172.21.15.133:0/4156159277 >> v1:172.21.15.133:6789/0 conn(0x7f794015c9a0 legacy=0x7f79400c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:14.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794692b640 1 -- 172.21.15.133:0/4156159277 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794006d4f0 con 0x7f79400ce330 2024-04-25T10:12:14.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f7945929640 1 --2- 172.21.15.133:0/4156159277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79400b1f70 0x7f794006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:14.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f794692b640 1 --2- 172.21.15.133:0/4156159277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79400ce330 0x7f79400ce7a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f7938009d90 tx=0x7f793801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5fdb0abecc646d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:14.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f7945128640 1 -- 172.21.15.133:0/4156159277 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380b2070 con 0x7f79400ce330 2024-04-25T10:12:14.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f7945128640 1 -- 172.21.15.133:0/4156159277 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79380a5910 con 0x7f79400ce330 2024-04-25T10:12:14.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.725+0000 7f7945128640 1 -- 172.21.15.133:0/4156159277 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380a5ce0 con 0x7f79400ce330 2024-04-25T10:12:14.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 -- 172.21.15.133:0/4156159277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79400ce330 msgr2=0x7f79400ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:14.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 --2- 172.21.15.133:0/4156159277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79400ce330 0x7f79400ce7a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f7938009d90 tx=0x7f793801c940 comp rx=0 tx=0).stop 2024-04-25T10:12:14.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 -- 172.21.15.133:0/4156159277 shutdown_connections 2024-04-25T10:12:14.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 --2- 172.21.15.133:0/4156159277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79400ce330 0x7f79400ce7a0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 --2- 172.21.15.133:0/4156159277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f79400b1f70 0x7f794006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 -- 172.21.15.133:0/4156159277 >> 172.21.15.133:0/4156159277 conn(0x7f7940067000 msgr2=0x7f794006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:14.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 -- 172.21.15.133:0/4156159277 shutdown_connections 2024-04-25T10:12:14.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 -- 172.21.15.133:0/4156159277 wait complete. 2024-04-25T10:12:14.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 Processor -- start 2024-04-25T10:12:14.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 -- start start 2024-04-25T10:12:14.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79400b1f70 0x7f79400b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:14.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79400ce330 0x7f7940158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:14.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f794015c9a0 0x7f79400b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:14.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794015dad0 con 0x7f79400ce330 2024-04-25T10:12:14.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794015d950 con 0x7f79400b1f70 2024-04-25T10:12:14.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794015dc50 con 0x7f794015c9a0 2024-04-25T10:12:14.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7945929640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79400ce330 0x7f7940158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:14.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7945929640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79400ce330 0x7f7940158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33064/0 (socket says 172.21.15.133:33064) 2024-04-25T10:12:14.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7945929640 1 -- 172.21.15.133:0/552722615 learned_addr learned my addr 172.21.15.133:0/552722615 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:14.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794612a640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79400b1f70 0x7f79400b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794692b640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f794015c9a0 0x7f79400b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7945929640 1 -- 172.21.15.133:0/552722615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f794015c9a0 msgr2=0x7f79400b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7945929640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f794015c9a0 0x7f79400b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7945929640 1 -- 172.21.15.133:0/552722615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79400b1f70 msgr2=0x7f79400b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7945929640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79400b1f70 0x7f79400b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7945929640 1 -- 172.21.15.133:0/552722615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794014ffc0 con 0x7f79400ce330 2024-04-25T10:12:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7945929640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79400ce330 0x7f7940158440 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f793000c7f0 tx=0x7f793000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7936ffd640 1 -- 172.21.15.133:0/552722615 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930015070 con 0x7f79400ce330 2024-04-25T10:12:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7936ffd640 1 -- 172.21.15.133:0/552722615 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7930007820 con 0x7f79400ce330 2024-04-25T10:12:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 -- 172.21.15.133:0/552722615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794014e960 con 0x7f79400ce330 2024-04-25T10:12:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f7936ffd640 1 -- 172.21.15.133:0/552722615 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930007c10 con 0x7f79400ce330 2024-04-25T10:12:14.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.729+0000 7f794838e640 1 -- 172.21.15.133:0/552722615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7940150260 con 0x7f79400ce330 2024-04-25T10:12:14.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.733+0000 7f794838e640 1 -- 172.21.15.133:0/552722615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794006bc20 con 0x7f79400ce330 2024-04-25T10:12:14.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.733+0000 7f7936ffd640 1 -- 172.21.15.133:0/552722615 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7930018070 con 0x7f79400ce330 2024-04-25T10:12:14.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.733+0000 7f7936ffd640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7924066f60 0x7f7924069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:14.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.733+0000 7f7936ffd640 1 -- 172.21.15.133:0/552722615 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7930084d80 con 0x7f79400ce330 2024-04-25T10:12:14.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.733+0000 7f794612a640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7924066f60 0x7f7924069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:14.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.733+0000 7f794612a640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7924066f60 0x7f7924069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f793c000f70 tx=0x7f793c007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:14.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.737+0000 7f7936ffd640 1 -- 172.21.15.133:0/552722615 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7930055f50 con 0x7f79400ce330 2024-04-25T10:12:14.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.937+0000 7f794838e640 1 -- 172.21.15.133:0/552722615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f79401504a0 con 0x7f79400ce330 2024-04-25T10:12:14.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.937+0000 7f7936ffd640 1 -- 172.21.15.133:0/552722615 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7930055850 con 0x7f79400ce330 2024-04-25T10:12:14.942 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:14.942 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:12:14.942 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:12:14.942 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.942 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:12:14.942 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:12:14.942 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.942 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:12:14.942 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:12:14.942 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:12:14.942 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.943 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:12:14.944 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:12:14.945 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:12:14.945 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.945 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.945 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.945 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.945 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:12:14.945 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:12:14.945 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.945 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:14.946 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:12:14.946 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.946 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.946 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.946 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.946 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:12:14.946 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.947 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:12:14.948 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:12:14.950 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.951 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.952 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:12:14.953 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.954 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:12:14.955 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.956 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:14.956 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:12:14.956 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:12:14.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 -- 172.21.15.133:0/552722615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7924066f60 msgr2=0x7f7924069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:14.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7924066f60 0x7f7924069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f793c000f70 tx=0x7f793c007b80 comp rx=0 tx=0).stop 2024-04-25T10:12:14.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 -- 172.21.15.133:0/552722615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79400ce330 msgr2=0x7f7940158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:14.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79400ce330 0x7f7940158440 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f793000c7f0 tx=0x7f793000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 -- 172.21.15.133:0/552722615 shutdown_connections 2024-04-25T10:12:14.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f794015c9a0 0x7f79400b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7924066f60 0x7f7924069420 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79400ce330 0x7f7940158440 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 --2- 172.21.15.133:0/552722615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f79400b1f70 0x7f79400b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:14.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 -- 172.21.15.133:0/552722615 >> 172.21.15.133:0/552722615 conn(0x7f7940067000 msgr2=0x7f79400c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:14.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 -- 172.21.15.133:0/552722615 shutdown_connections 2024-04-25T10:12:14.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:14.941+0000 7f794838e640 1 -- 172.21.15.133:0/552722615 wait complete. 2024-04-25T10:12:14.960 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:15.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.061+0000 7efe04cca640 1 Processor -- start 2024-04-25T10:12:15.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.061+0000 7efe04cca640 1 -- start start 2024-04-25T10:12:15.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efe04cca640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efe000b1f70 0x7efe0006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efe04cca640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efe000ce330 0x7efe000ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efe04cca640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe000c61d0 con 0x7efe0015c9a0 2024-04-25T10:12:15.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efe04cca640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe000cecc0 con 0x7efe000ce330 2024-04-25T10:12:15.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efe04cca640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe000c64b0 con 0x7efe000b1f70 2024-04-25T10:12:15.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfed9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efe000ce330 0x7efe000ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfed9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efe000ce330 0x7efe000ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33074/0 (socket says 172.21.15.133:33074) 2024-04-25T10:12:15.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfed9d640 1 -- 172.21.15.133:0/1068075525 learned_addr learned my addr 172.21.15.133:0/1068075525 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:15.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfdd9b640 1 --2- 172.21.15.133:0/1068075525 >> v2:172.21.15.162:3300/0 conn(0x7efe000b1f70 0x7efe0006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfed9d640 1 -- 172.21.15.133:0/1068075525 >> v2:172.21.15.162:3300/0 conn(0x7efe000b1f70 msgr2=0x7efe0006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:15.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfed9d640 1 --2- 172.21.15.133:0/1068075525 >> v2:172.21.15.162:3300/0 conn(0x7efe000b1f70 0x7efe0006dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfed9d640 1 -- 172.21.15.133:0/1068075525 >> v1:172.21.15.162:6789/0 conn(0x7efe0015c9a0 legacy=0x7efe000c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:15.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfed9d640 1 -- 172.21.15.133:0/1068075525 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0006d4f0 con 0x7efe000ce330 2024-04-25T10:12:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfed9d640 1 --2- 172.21.15.133:0/1068075525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe000ce330 0x7efe000ce780 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7efe00072670 tx=0x7efdf801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d499c959a6806aef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfd59a640 1 -- 172.21.15.133:0/1068075525 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf80a5750 con 0x7efe000ce330 2024-04-25T10:12:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfd59a640 1 -- 172.21.15.133:0/1068075525 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf80a58f0 con 0x7efe000ce330 2024-04-25T10:12:15.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.065+0000 7efdfd59a640 1 -- 172.21.15.133:0/1068075525 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf80a5cc0 con 0x7efe000ce330 2024-04-25T10:12:15.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- 172.21.15.133:0/1068075525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe000ce330 msgr2=0x7efe000ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 --2- 172.21.15.133:0/1068075525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe000ce330 0x7efe000ce780 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7efe00072670 tx=0x7efdf801c980 comp rx=0 tx=0).stop 2024-04-25T10:12:15.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- 172.21.15.133:0/1068075525 shutdown_connections 2024-04-25T10:12:15.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 --2- 172.21.15.133:0/1068075525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe000ce330 0x7efe000ce780 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 --2- 172.21.15.133:0/1068075525 >> v2:172.21.15.162:3300/0 conn(0x7efe000b1f70 0x7efe0006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- 172.21.15.133:0/1068075525 >> 172.21.15.133:0/1068075525 conn(0x7efe00067000 msgr2=0x7efe0006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:15.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- 172.21.15.133:0/1068075525 shutdown_connections 2024-04-25T10:12:15.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- 172.21.15.133:0/1068075525 wait complete. 2024-04-25T10:12:15.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 Processor -- start 2024-04-25T10:12:15.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- start start 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe000b1f70 0x7efe00157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe000ce330 0x7efe000b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe0015c9a0 0x7efe00144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0015dad0 con 0x7efe0015c9a0 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0015d950 con 0x7efe000b1f70 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0015dc50 con 0x7efe000ce330 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdfed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe0015c9a0 0x7efe00144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdfed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe0015c9a0 0x7efe00144b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33086/0 (socket says 172.21.15.133:33086) 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdfed9d640 1 -- 172.21.15.133:0/3728606738 learned_addr learned my addr 172.21.15.133:0/3728606738 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdfe59c640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe000b1f70 0x7efe00157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdfed9d640 1 -- 172.21.15.133:0/3728606738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe000ce330 msgr2=0x7efe000b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdfed9d640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe000ce330 0x7efe000b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdfed9d640 1 -- 172.21.15.133:0/3728606738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe000b1f70 msgr2=0x7efe00157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdfed9d640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe000b1f70 0x7efe00157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdfed9d640 1 -- 172.21.15.133:0/3728606738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000cb550 con 0x7efe0015c9a0 2024-04-25T10:12:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdfed9d640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe0015c9a0 0x7efe00144b30 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7efdf801a5b0 tx=0x7efdf8019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdf6ffd640 1 -- 172.21.15.133:0/3728606738 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf80b8470 con 0x7efe0015c9a0 2024-04-25T10:12:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdf6ffd640 1 -- 172.21.15.133:0/3728606738 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf80b8610 con 0x7efe0015c9a0 2024-04-25T10:12:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- 172.21.15.133:0/3728606738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe000cb830 con 0x7efe0015c9a0 2024-04-25T10:12:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efdf6ffd640 1 -- 172.21.15.133:0/3728606738 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf80b8a00 con 0x7efe0015c9a0 2024-04-25T10:12:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- 172.21.15.133:0/3728606738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe00158680 con 0x7efe0015c9a0 2024-04-25T10:12:15.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.069+0000 7efe04cca640 1 -- 172.21.15.133:0/3728606738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0006bc20 con 0x7efe0015c9a0 2024-04-25T10:12:15.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.073+0000 7efdf6ffd640 1 -- 172.21.15.133:0/3728606738 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efdf80c0020 con 0x7efe0015c9a0 2024-04-25T10:12:15.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.073+0000 7efdf6ffd640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 0x7efde0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.073+0000 7efdf6ffd640 1 -- 172.21.15.133:0/3728606738 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efdf81205e0 con 0x7efe0015c9a0 2024-04-25T10:12:15.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.073+0000 7efdfe59c640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 0x7efde0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.073+0000 7efdfe59c640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 0x7efde0069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efdf000a750 tx=0x7efdf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.077+0000 7efdf6ffd640 1 -- 172.21.15.133:0/3728606738 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efdf80c7020 con 0x7efe0015c9a0 2024-04-25T10:12:15.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.233+0000 7efe04cca640 1 -- 172.21.15.133:0/3728606738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe00150bc0 con 0x7efe0015c9a0 2024-04-25T10:12:15.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.233+0000 7efdf6ffd640 1 -- 172.21.15.133:0/3728606738 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+13198 (secure 0 0 0) 0x7efdf80165a0 con 0x7efe0015c9a0 2024-04-25T10:12:15.237 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:15.237 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":116,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:12:14.547346+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 -- 172.21.15.133:0/3728606738 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 msgr2=0x7efde0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 0x7efde0069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efdf000a750 tx=0x7efdf0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 -- 172.21.15.133:0/3728606738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe0015c9a0 msgr2=0x7efe00144b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe0015c9a0 0x7efe00144b30 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7efdf801a5b0 tx=0x7efdf8019720 comp rx=0 tx=0).stop 2024-04-25T10:12:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 -- 172.21.15.133:0/3728606738 shutdown_connections 2024-04-25T10:12:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 0x7efde0069420 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe0015c9a0 0x7efe00144b30 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe000ce330 0x7efe000b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 --2- 172.21.15.133:0/3728606738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe000b1f70 0x7efe00157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 -- 172.21.15.133:0/3728606738 >> 172.21.15.133:0/3728606738 conn(0x7efe00067000 msgr2=0x7efe0015b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 -- 172.21.15.133:0/3728606738 shutdown_connections 2024-04-25T10:12:15.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.237+0000 7efe04cca640 1 -- 172.21.15.133:0/3728606738 wait complete. 2024-04-25T10:12:15.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:15.257 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:15.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.373+0000 7f32857c4640 1 Processor -- start 2024-04-25T10:12:15.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.373+0000 7f32857c4640 1 -- start start 2024-04-25T10:12:15.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f32857c4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3280143ea0 0x7f32800cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f32857c4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f328006eac0 0x7f32800b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f32857c4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328015ac50 con 0x7f32800b2150 2024-04-25T10:12:15.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f32857c4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32800cd180 con 0x7f328006eac0 2024-04-25T10:12:15.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f32857c4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328015aed0 con 0x7f3280143ea0 2024-04-25T10:12:15.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f327effd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3280143ea0 0x7f32800cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f327f7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f328006eac0 0x7f32800b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f327effd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3280143ea0 0x7f32800cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:59824/0 (socket says 172.21.15.133:59824) 2024-04-25T10:12:15.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f327effd640 1 -- 172.21.15.133:0/122358404 learned_addr learned my addr 172.21.15.133:0/122358404 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:15.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f32767fc640 1 --1- 172.21.15.133:0/122358404 >> v1:172.21.15.162:6789/0 conn(0x7f32800b2150 0x7f328006e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:50140/0 (socket says 172.21.15.133:50140) 2024-04-25T10:12:15.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f327f7fe640 1 -- 172.21.15.133:0/122358404 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3280143ea0 msgr2=0x7f32800cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:15.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f327f7fe640 1 --2- 172.21.15.133:0/122358404 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3280143ea0 0x7f32800cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f327f7fe640 1 -- 172.21.15.133:0/122358404 >> v1:172.21.15.162:6789/0 conn(0x7f32800b2150 legacy=0x7f328006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f327f7fe640 1 -- 172.21.15.133:0/122358404 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328015b1b0 con 0x7f328006eac0 2024-04-25T10:12:15.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f327f7fe640 1 --2- 172.21.15.133:0/122358404 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328006eac0 0x7f32800b06d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3278009d90 tx=0x7f327801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=120489ef07e58038 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.377+0000 7f327e7fc640 1 -- 172.21.15.133:0/122358404 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32780b2070 con 0x7f328006eac0 2024-04-25T10:12:15.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f327e7fc640 1 -- 172.21.15.133:0/122358404 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32780a5940 con 0x7f328006eac0 2024-04-25T10:12:15.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f327e7fc640 1 -- 172.21.15.133:0/122358404 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32780a5d10 con 0x7f328006eac0 2024-04-25T10:12:15.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 -- 172.21.15.133:0/122358404 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328006eac0 msgr2=0x7f32800b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 --2- 172.21.15.133:0/122358404 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328006eac0 0x7f32800b06d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3278009d90 tx=0x7f327801c970 comp rx=0 tx=0).stop 2024-04-25T10:12:15.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 -- 172.21.15.133:0/122358404 shutdown_connections 2024-04-25T10:12:15.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 --2- 172.21.15.133:0/122358404 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328006eac0 0x7f32800b06d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 --2- 172.21.15.133:0/122358404 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3280143ea0 0x7f32800cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 -- 172.21.15.133:0/122358404 >> 172.21.15.133:0/122358404 conn(0x7f3280066f70 msgr2=0x7f328006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:15.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 -- 172.21.15.133:0/122358404 shutdown_connections 2024-04-25T10:12:15.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 -- 172.21.15.133:0/122358404 wait complete. 2024-04-25T10:12:15.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 Processor -- start 2024-04-25T10:12:15.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 -- start start 2024-04-25T10:12:15.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328006eac0 0x7f32801521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32800b2150 0x7f32801527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3280143ea0 0x7f32801401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328006d420 con 0x7f32800b2150 2024-04-25T10:12:15.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328006d2a0 con 0x7f328006eac0 2024-04-25T10:12:15.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328006d5a0 con 0x7f3280143ea0 2024-04-25T10:12:15.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32767fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32800b2150 0x7f32801527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32767fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32800b2150 0x7f32801527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33100/0 (socket says 172.21.15.133:33100) 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32767fc640 1 -- 172.21.15.133:0/3793485636 learned_addr learned my addr 172.21.15.133:0/3793485636 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f327f7fe640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3280143ea0 0x7f32801401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f327effd640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328006eac0 0x7f32801521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32767fc640 1 -- 172.21.15.133:0/3793485636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3280143ea0 msgr2=0x7f32801401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32767fc640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3280143ea0 0x7f32801401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32767fc640 1 -- 172.21.15.133:0/3793485636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328006eac0 msgr2=0x7f32801521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32767fc640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328006eac0 0x7f32801521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32767fc640 1 -- 172.21.15.133:0/3793485636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3280153010 con 0x7f32800b2150 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32767fc640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32800b2150 0x7f32801527b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f326800c450 tx=0x7f326800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f3277fff640 1 -- 172.21.15.133:0/3793485636 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3268018070 con 0x7f32800b2150 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f3277fff640 1 -- 172.21.15.133:0/3793485636 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3268007480 con 0x7f32800b2150 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 -- 172.21.15.133:0/3793485636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32801535a0 con 0x7f32800b2150 2024-04-25T10:12:15.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f3277fff640 1 -- 172.21.15.133:0/3793485636 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32680047f0 con 0x7f32800b2150 2024-04-25T10:12:15.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.381+0000 7f32857c4640 1 -- 172.21.15.133:0/3793485636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3280152cf0 con 0x7f32800b2150 2024-04-25T10:12:15.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.385+0000 7f32857c4640 1 -- 172.21.15.133:0/3793485636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3280068e80 con 0x7f32800b2150 2024-04-25T10:12:15.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.385+0000 7f3277fff640 1 -- 172.21.15.133:0/3793485636 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f326801b070 con 0x7f32800b2150 2024-04-25T10:12:15.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.385+0000 7f3277fff640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3260066f60 0x7f3260069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.385+0000 7f3277fff640 1 -- 172.21.15.133:0/3793485636 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3268084070 con 0x7f32800b2150 2024-04-25T10:12:15.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.385+0000 7f327effd640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3260066f60 0x7f3260069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.385+0000 7f327effd640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3260066f60 0x7f3260069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f327000bbc0 tx=0x7f327000b450 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.389+0000 7f3277fff640 1 -- 172.21.15.133:0/3793485636 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f328015ac50 con 0x7f32800b2150 2024-04-25T10:12:15.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.537+0000 7f32857c4640 1 -- 172.21.15.133:0/3793485636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3280159ba0 con 0x7f32800b2150 2024-04-25T10:12:15.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.537+0000 7f3277fff640 1 -- 172.21.15.133:0/3793485636 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3268055240 con 0x7f32800b2150 2024-04-25T10:12:15.542 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 -- 172.21.15.133:0/3793485636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3260066f60 msgr2=0x7f3260069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3260066f60 0x7f3260069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f327000bbc0 tx=0x7f327000b450 comp rx=0 tx=0).stop 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 -- 172.21.15.133:0/3793485636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32800b2150 msgr2=0x7f32801527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32800b2150 0x7f32801527b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f326800c450 tx=0x7f326800c910 comp rx=0 tx=0).stop 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 -- 172.21.15.133:0/3793485636 shutdown_connections 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3280143ea0 0x7f32801401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3260066f60 0x7f3260069420 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32800b2150 0x7f32801527b0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 --2- 172.21.15.133:0/3793485636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f328006eac0 0x7f32801521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 -- 172.21.15.133:0/3793485636 >> 172.21.15.133:0/3793485636 conn(0x7f3280066f70 msgr2=0x7f328006d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 -- 172.21.15.133:0/3793485636 shutdown_connections 2024-04-25T10:12:15.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.541+0000 7f32857c4640 1 -- 172.21.15.133:0/3793485636 wait complete. 2024-04-25T10:12:15.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:12:15.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.669+0000 7f06c24d2640 1 Processor -- start 2024-04-25T10:12:15.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.669+0000 7f06c24d2640 1 -- start start 2024-04-25T10:12:15.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06c24d2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f06bc15c9a0 0x7f06bc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06c24d2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f06bc0b1f70 0x7f06bc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06c24d2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06bc0ce330 0x7f06bc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06c24d2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc0cecc0 con 0x7f06bc15c9a0 2024-04-25T10:12:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06c24d2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc0c61d0 con 0x7f06bc0b1f70 2024-04-25T10:12:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06c24d2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0c64b0 con 0x7f06bc0ce330 2024-04-25T10:12:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06bbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f06bc15c9a0 0x7f06bc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06bbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f06bc15c9a0 0x7f06bc0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33116/0 (socket says 172.21.15.133:33116) 2024-04-25T10:12:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06bbfff640 1 -- 172.21.15.133:0/3730518143 learned_addr learned my addr 172.21.15.133:0/3730518143 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06bbfff640 1 -- 172.21.15.133:0/3730518143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06bc0ce330 msgr2=0x7f06bc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06bbfff640 1 --2- 172.21.15.133:0/3730518143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06bc0ce330 0x7f06bc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06bbfff640 1 -- 172.21.15.133:0/3730518143 >> v2:172.21.15.162:3300/0 conn(0x7f06bc0b1f70 msgr2=0x7f06bc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:15.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06bbfff640 1 --2- 172.21.15.133:0/3730518143 >> v2:172.21.15.162:3300/0 conn(0x7f06bc0b1f70 0x7f06bc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06bbfff640 1 -- 172.21.15.133:0/3730518143 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc06d4f0 con 0x7f06bc15c9a0 2024-04-25T10:12:15.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06bbfff640 1 --2- 172.21.15.133:0/3730518143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06bc15c9a0 0x7f06bc0c5980 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f06bc072670 tx=0x7f06ac01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad4eb6f96077016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06baffd640 1 -- 172.21.15.133:0/3730518143 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0a7070 con 0x7f06bc15c9a0 2024-04-25T10:12:15.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06baffd640 1 -- 172.21.15.133:0/3730518143 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ac0b0070 con 0x7f06bc15c9a0 2024-04-25T10:12:15.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.673+0000 7f06baffd640 1 -- 172.21.15.133:0/3730518143 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0ab890 con 0x7f06bc15c9a0 2024-04-25T10:12:15.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 -- 172.21.15.133:0/3730518143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06bc15c9a0 msgr2=0x7f06bc0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 --2- 172.21.15.133:0/3730518143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06bc15c9a0 0x7f06bc0c5980 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f06bc072670 tx=0x7f06ac01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 -- 172.21.15.133:0/3730518143 shutdown_connections 2024-04-25T10:12:15.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 --2- 172.21.15.133:0/3730518143 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06bc0ce330 0x7f06bc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 --2- 172.21.15.133:0/3730518143 >> v2:172.21.15.162:3300/0 conn(0x7f06bc0b1f70 0x7f06bc06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 --2- 172.21.15.133:0/3730518143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06bc15c9a0 0x7f06bc0c5980 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 -- 172.21.15.133:0/3730518143 >> 172.21.15.133:0/3730518143 conn(0x7f06bc067000 msgr2=0x7f06bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:15.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 -- 172.21.15.133:0/3730518143 shutdown_connections 2024-04-25T10:12:15.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 -- 172.21.15.133:0/3730518143 wait complete. 2024-04-25T10:12:15.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 Processor -- start 2024-04-25T10:12:15.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 -- start start 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06bc0b1f70 0x7f06bc13dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06bc0ce330 0x7f06bc15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06bc15c9a0 0x7f06bc15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc15cf00 con 0x7f06bc0b1f70 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc15cd80 con 0x7f06bc0ce330 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc15d950 con 0x7f06bc15c9a0 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06bbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06bc0b1f70 0x7f06bc13dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06bbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06bc0b1f70 0x7f06bc13dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33132/0 (socket says 172.21.15.133:33132) 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06bbfff640 1 -- 172.21.15.133:0/2354825715 learned_addr learned my addr 172.21.15.133:0/2354825715 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06bb7fe640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06bc0ce330 0x7f06bc15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06bbfff640 1 -- 172.21.15.133:0/2354825715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06bc15c9a0 msgr2=0x7f06bc15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06bbfff640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06bc15c9a0 0x7f06bc15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06bbfff640 1 -- 172.21.15.133:0/2354825715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06bc0ce330 msgr2=0x7f06bc15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06bbfff640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06bc0ce330 0x7f06bc15b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06bbfff640 1 -- 172.21.15.133:0/2354825715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc143000 con 0x7f06bc0b1f70 2024-04-25T10:12:15.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06bbfff640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06bc0b1f70 0x7f06bc13dad0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0ad040 tx=0x7f06ac01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06b8ff9640 1 -- 172.21.15.133:0/2354825715 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0b4800 con 0x7f06bc0b1f70 2024-04-25T10:12:15.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06b8ff9640 1 -- 172.21.15.133:0/2354825715 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ac0b49c0 con 0x7f06bc0b1f70 2024-04-25T10:12:15.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 -- 172.21.15.133:0/2354825715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc151f70 con 0x7f06bc0b1f70 2024-04-25T10:12:15.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06b8ff9640 1 -- 172.21.15.133:0/2354825715 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0b4db0 con 0x7f06bc0b1f70 2024-04-25T10:12:15.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.677+0000 7f06c24d2640 1 -- 172.21.15.133:0/2354825715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc152530 con 0x7f06bc0b1f70 2024-04-25T10:12:15.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.681+0000 7f06c24d2640 1 -- 172.21.15.133:0/2354825715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06bc068ef0 con 0x7f06bc0b1f70 2024-04-25T10:12:15.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.681+0000 7f06b8ff9640 1 -- 172.21.15.133:0/2354825715 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06ac0b3380 con 0x7f06bc0b1f70 2024-04-25T10:12:15.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.681+0000 7f06b8ff9640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06a8066f60 0x7f06a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.681+0000 7f06b8ff9640 1 -- 172.21.15.133:0/2354825715 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f06ac11f4c0 con 0x7f06bc0b1f70 2024-04-25T10:12:15.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.681+0000 7f06bb7fe640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06a8066f60 0x7f06a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.681+0000 7f06bb7fe640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06a8066f60 0x7f06a8069420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0b1c40 tx=0x7f06b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.685+0000 7f06b8ff9640 1 -- 172.21.15.133:0/2354825715 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06ac0c4060 con 0x7f06bc0b1f70 2024-04-25T10:12:15.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.845+0000 7f06c24d2640 1 -- 172.21.15.133:0/2354825715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f06bc156b10 con 0x7f06bc0b1f70 2024-04-25T10:12:15.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.845+0000 7f06b8ff9640 1 -- 172.21.15.133:0/2354825715 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) v1 ==== 84+0+65 (secure 0 0 0) 0x7f06bc143000 con 0x7f06bc0b1f70 2024-04-25T10:12:15.848 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:15.848 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:12:15.848 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:12:15.849 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:12:15.849 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:12:15.849 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:15.849 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:12:15.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 -- 172.21.15.133:0/2354825715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06a8066f60 msgr2=0x7f06a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06a8066f60 0x7f06a8069420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0b1c40 tx=0x7f06b0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 -- 172.21.15.133:0/2354825715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06bc0b1f70 msgr2=0x7f06bc13dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06bc0b1f70 0x7f06bc13dad0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0ad040 tx=0x7f06ac01ad30 comp rx=0 tx=0).stop 2024-04-25T10:12:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 -- 172.21.15.133:0/2354825715 shutdown_connections 2024-04-25T10:12:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f06bc15c9a0 0x7f06bc15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06bc0ce330 0x7f06bc15b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f06a8066f60 0x7f06a8069420 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 --2- 172.21.15.133:0/2354825715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06bc0b1f70 0x7f06bc13dad0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 -- 172.21.15.133:0/2354825715 >> 172.21.15.133:0/2354825715 conn(0x7f06bc067000 msgr2=0x7f06bc0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 -- 172.21.15.133:0/2354825715 shutdown_connections 2024-04-25T10:12:15.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.849+0000 7f06c24d2640 1 -- 172.21.15.133:0/2354825715 wait complete. 2024-04-25T10:12:15.866 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:12:15.866 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:12:15.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f459094a640 1 Processor -- start 2024-04-25T10:12:15.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f459094a640 1 -- start start 2024-04-25T10:12:15.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f459094a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f458c0b4940 0x7f458c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f459094a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f458c15c420 0x7f458c0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f459094a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c0b5260 con 0x7f458c071cb0 2024-04-25T10:12:15.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f459094a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c072a90 con 0x7f458c0b4940 2024-04-25T10:12:15.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f459094a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c0a85d0 con 0x7f458c15c420 2024-04-25T10:12:15.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f458a59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f458c0b4940 0x7f458c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f458a59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f458c0b4940 0x7f458c0b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33146/0 (socket says 172.21.15.133:33146) 2024-04-25T10:12:15.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f458a59c640 1 -- 172.21.15.133:0/3180629073 learned_addr learned my addr 172.21.15.133:0/3180629073 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:15.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f458ad9d640 1 --2- 172.21.15.133:0/3180629073 >> v2:172.21.15.162:3300/0 conn(0x7f458c15c420 0x7f458c0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f458a59c640 1 -- 172.21.15.133:0/3180629073 >> v2:172.21.15.162:3300/0 conn(0x7f458c15c420 msgr2=0x7f458c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:15.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f458a59c640 1 --2- 172.21.15.133:0/3180629073 >> v2:172.21.15.162:3300/0 conn(0x7f458c15c420 0x7f458c0cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f458a59c640 1 -- 172.21.15.133:0/3180629073 >> v1:172.21.15.162:6789/0 conn(0x7f458c071cb0 legacy=0x7f458c15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f458a59c640 1 -- 172.21.15.133:0/3180629073 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c15c9b0 con 0x7f458c0b4940 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f458a59c640 1 --2- 172.21.15.133:0/3180629073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f458c0b4940 0x7f458c0b4d20 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f458c0cbfc0 tx=0x7f457c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e70cc9ee551da05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f4589d9b640 1 -- 172.21.15.133:0/3180629073 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c0aa650 con 0x7f458c0b4940 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f4589d9b640 1 -- 172.21.15.133:0/3180629073 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c0af070 con 0x7f458c0b4940 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.985+0000 7f4589d9b640 1 -- 172.21.15.133:0/3180629073 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c0aaa10 con 0x7f458c0b4940 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- 172.21.15.133:0/3180629073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f458c0b4940 msgr2=0x7f458c0b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 --2- 172.21.15.133:0/3180629073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f458c0b4940 0x7f458c0b4d20 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f458c0cbfc0 tx=0x7f457c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- 172.21.15.133:0/3180629073 shutdown_connections 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 --2- 172.21.15.133:0/3180629073 >> v2:172.21.15.162:3300/0 conn(0x7f458c15c420 0x7f458c0cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 --2- 172.21.15.133:0/3180629073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f458c0b4940 0x7f458c0b4d20 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- 172.21.15.133:0/3180629073 >> 172.21.15.133:0/3180629073 conn(0x7f458c06d370 msgr2=0x7f458c06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- 172.21.15.133:0/3180629073 shutdown_connections 2024-04-25T10:12:15.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- 172.21.15.133:0/3180629073 wait complete. 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 Processor -- start 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- start start 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f458c071cb0 0x7f458c138050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f458c0b4940 0x7f458c1429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f458c15c420 0x7f458c15a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c1452d0 con 0x7f458c15c420 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c071790 con 0x7f458c0b4940 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c145450 con 0x7f458c071cb0 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f458ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f458c15c420 0x7f458c15a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f458ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f458c15c420 0x7f458c15a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33154/0 (socket says 172.21.15.133:33154) 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f458ad9d640 1 -- 172.21.15.133:0/424810070 learned_addr learned my addr 172.21.15.133:0/424810070 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f458a59c640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f458c071cb0 0x7f458c138050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f4583fff640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f458c0b4940 0x7f458c1429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f458ad9d640 1 -- 172.21.15.133:0/424810070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f458c071cb0 msgr2=0x7f458c138050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f458ad9d640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f458c071cb0 0x7f458c138050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f458ad9d640 1 -- 172.21.15.133:0/424810070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f458c0b4940 msgr2=0x7f458c1429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f458ad9d640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f458c0b4940 0x7f458c1429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f458ad9d640 1 -- 172.21.15.133:0/424810070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c142f10 con 0x7f458c15c420 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f4583fff640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f458c0b4940 0x7f458c1429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f458a59c640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f458c071cb0 0x7f458c138050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f458ad9d640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f458c15c420 0x7f458c15a610 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4584007e90 tx=0x7f458400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f4582ffd640 1 -- 172.21.15.133:0/424810070 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4584014420 con 0x7f458c15c420 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f4582ffd640 1 -- 172.21.15.133:0/424810070 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4584019070 con 0x7f458c15c420 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f4582ffd640 1 -- 172.21.15.133:0/424810070 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4584014800 con 0x7f458c15c420 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- 172.21.15.133:0/424810070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c13e9a0 con 0x7f458c15c420 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- 172.21.15.133:0/424810070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c141ef0 con 0x7f458c15c420 2024-04-25T10:12:15.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.989+0000 7f459094a640 1 -- 172.21.15.133:0/424810070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c069000 con 0x7f458c15c420 2024-04-25T10:12:15.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.993+0000 7f4582ffd640 1 -- 172.21.15.133:0/424810070 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45840149a0 con 0x7f458c15c420 2024-04-25T10:12:15.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.993+0000 7f4582ffd640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f456c066f90 0x7f456c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.993+0000 7f4582ffd640 1 -- 172.21.15.133:0/424810070 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4584085780 con 0x7f458c15c420 2024-04-25T10:12:15.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.993+0000 7f458a59c640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f456c066f90 0x7f456c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.993+0000 7f458a59c640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f456c066f90 0x7f456c069450 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f457c0ae000 tx=0x7f457c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:15.997+0000 7f4582ffd640 1 -- 172.21.15.133:0/424810070 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45840568b0 con 0x7f458c15c420 2024-04-25T10:12:16.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.157+0000 7f459094a640 1 -- 172.21.15.133:0/424810070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f458c0b1080 con 0x7f458c15c420 2024-04-25T10:12:16.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.537+0000 7f4582ffd640 1 -- 172.21.15.133:0/424810070 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v117) v1 ==== 138+0+0 (secure 0 0 0) 0x7f45840561b0 con 0x7f458c15c420 2024-04-25T10:12:16.540 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:12:16.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 -- 172.21.15.133:0/424810070 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f456c066f90 msgr2=0x7f456c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f456c066f90 0x7f456c069450 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f457c0ae000 tx=0x7f457c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:16.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 -- 172.21.15.133:0/424810070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f458c15c420 msgr2=0x7f458c15a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f458c15c420 0x7f458c15a610 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4584007e90 tx=0x7f458400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 -- 172.21.15.133:0/424810070 shutdown_connections 2024-04-25T10:12:16.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f456c066f90 0x7f456c069450 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f458c15c420 0x7f458c15a610 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f458c0b4940 0x7f458c1429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 --2- 172.21.15.133:0/424810070 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f458c071cb0 0x7f458c138050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 -- 172.21.15.133:0/424810070 >> 172.21.15.133:0/424810070 conn(0x7f458c06d370 msgr2=0x7f458c06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:16.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 -- 172.21.15.133:0/424810070 shutdown_connections 2024-04-25T10:12:16.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.541+0000 7f459094a640 1 -- 172.21.15.133:0/424810070 wait complete. 2024-04-25T10:12:16.569 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:12:16.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f4595d05640 1 Processor -- start 2024-04-25T10:12:16.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f4595d05640 1 -- start start 2024-04-25T10:12:16.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f4595d05640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4590143560 0x7f4590140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f4595d05640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f459015d9f0 0x7f459015de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f4595d05640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45900cf280 con 0x7f4590142920 2024-04-25T10:12:16.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f4595d05640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f459006d440 con 0x7f459015d9f0 2024-04-25T10:12:16.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f4595d05640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459006d720 con 0x7f4590143560 2024-04-25T10:12:16.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4590142920 0x7f45900ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46642/0 (socket says 172.21.15.133:46642) 2024-04-25T10:12:16.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458f7fe640 1 -- 172.21.15.133:0/1490114759 learned_addr learned my addr 172.21.15.133:0/1490114759 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:16.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458effd640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4590143560 0x7f4590140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458ffff640 1 --2- 172.21.15.133:0/1490114759 >> v2:172.21.15.162:3300/0 conn(0x7f459015d9f0 0x7f459015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614378063 0 0) 0x7f45900cf280 con 0x7f4590142920 2024-04-25T10:12:16.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f457c005740 con 0x7f4590142920 2024-04-25T10:12:16.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1374045714 0 0) 0x7f457c005740 con 0x7f4590142920 2024-04-25T10:12:16.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4590143560 msgr2=0x7f4590140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:16.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4590143560 0x7f4590140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 >> v2:172.21.15.162:3300/0 conn(0x7f459015d9f0 msgr2=0x7f459015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:16.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 --2- 172.21.15.133:0/1490114759 >> v2:172.21.15.162:3300/0 conn(0x7f459015d9f0 0x7f459015de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459006da00 con 0x7f4590142920 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4588001680 con 0x7f4590142920 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 >> v1:172.21.15.133:6789/0 conn(0x7f4590142920 legacy=0x7f45900ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f457c005bd0 0x7f457c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4590143560 0x7f457c006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f457c006f60 0x7f457c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459006d440 con 0x7f457c005bd0 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459006d720 con 0x7f457c006f60 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457c00dbe0 con 0x7f4590143560 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458f7fe640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f457c005bd0 0x7f457c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458ffff640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f457c006f60 0x7f457c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.693+0000 7f458f7fe640 1 -- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4590143560 msgr2=0x7f457c006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:16.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f458effd640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4590143560 0x7f457c006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f458f7fe640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4590143560 0x7f457c006a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f458f7fe640 1 -- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f457c006f60 msgr2=0x7f457c0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:16.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f458f7fe640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f457c006f60 0x7f457c0073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f458f7fe640 1 -- 172.21.15.133:0/1490114759 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c005740 con 0x7f457c005bd0 2024-04-25T10:12:16.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f458ffff640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f457c006f60 0x7f457c0073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:16.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f458f7fe640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f457c005bd0 0x7f457c005ff0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f458800c210 tx=0x7f458801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ccf001420f3075 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588007d60 con 0x7f457c005bd0 2024-04-25T10:12:16.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45880a9020 con 0x7f457c005bd0 2024-04-25T10:12:16.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f458e7fc640 1 -- 172.21.15.133:0/1490114759 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588005b20 con 0x7f457c005bd0 2024-04-25T10:12:16.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f4595d05640 1 -- 172.21.15.133:0/1490114759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f457c005bd0 msgr2=0x7f457c005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f4595d05640 1 --2- 172.21.15.133:0/1490114759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f457c005bd0 0x7f457c005ff0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f458800c210 tx=0x7f458801cca0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f458ffff640 1 -- 172.21.15.133:0/1490114759 reap_dead start 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f4595d05640 1 -- 172.21.15.133:0/1490114759 shutdown_connections 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f4595d05640 1 -- 172.21.15.133:0/1490114759 >> 172.21.15.133:0/1490114759 conn(0x7f4590066f70 msgr2=0x7f459006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f4595d05640 1 -- 172.21.15.133:0/1490114759 shutdown_connections 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.697+0000 7f4595d05640 1 -- 172.21.15.133:0/1490114759 wait complete. 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4595d05640 1 Processor -- start 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4595d05640 1 -- start start 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4595d05640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f457c005bd0 0x7f4590160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4595d05640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4590152a90 0x7f4590155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4595d05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4590156520 0x7f45901349c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4595d05640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459006e2b0 con 0x7f4590156520 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4595d05640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459006e130 con 0x7f4590152a90 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4595d05640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459006e430 con 0x7f457c005bd0 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f458ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4590156520 0x7f45901349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f458ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4590156520 0x7f45901349c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43734/0 (socket says 172.21.15.133:43734) 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f458ffff640 1 -- 172.21.15.133:0/2775691415 learned_addr learned my addr 172.21.15.133:0/2775691415 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f458f7fe640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f457c005bd0 0x7f4590160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f458effd640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4590152a90 0x7f4590155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f458ffff640 1 -- 172.21.15.133:0/2775691415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f457c005bd0 msgr2=0x7f4590160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f458ffff640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f457c005bd0 0x7f4590160480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f458ffff640 1 -- 172.21.15.133:0/2775691415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4590152a90 msgr2=0x7f4590155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f458ffff640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4590152a90 0x7f4590155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f458ffff640 1 -- 172.21.15.133:0/2775691415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4590151760 con 0x7f4590156520 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f458ffff640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4590156520 0x7f45901349c0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f459006f600 tx=0x7f458000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4587fff640 1 -- 172.21.15.133:0/2775691415 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580016620 con 0x7f4590156520 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4587fff640 1 -- 172.21.15.133:0/2775691415 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458001b070 con 0x7f4590156520 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4595d05640 1 -- 172.21.15.133:0/2775691415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45901519f0 con 0x7f4590156520 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4587fff640 1 -- 172.21.15.133:0/2775691415 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45800169e0 con 0x7f4590156520 2024-04-25T10:12:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4595d05640 1 -- 172.21.15.133:0/2775691415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4590151fb0 con 0x7f4590156520 2024-04-25T10:12:16.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4595d05640 1 -- 172.21.15.133:0/2775691415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4590068e80 con 0x7f4590156520 2024-04-25T10:12:16.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.701+0000 7f4587fff640 1 -- 172.21.15.133:0/2775691415 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4580016b80 con 0x7f4590156520 2024-04-25T10:12:16.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.705+0000 7f4587fff640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4570066f60 0x7f4570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.705+0000 7f4587fff640 1 -- 172.21.15.133:0/2775691415 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f4580086650 con 0x7f4590156520 2024-04-25T10:12:16.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.705+0000 7f458f7fe640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4570066f60 0x7f4570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.705+0000 7f458f7fe640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4570066f60 0x7f4570069420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f45880aa000 tx=0x7f4588000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.709+0000 7f4587fff640 1 -- 172.21.15.133:0/2775691415 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4580013330 con 0x7f4590156520 2024-04-25T10:12:16.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:16.865+0000 7f4595d05640 1 -- 172.21.15.133:0/2775691415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f45901598a0 con 0x7f4590156520 2024-04-25T10:12:17.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.541+0000 7f4587fff640 1 -- 172.21.15.133:0/2775691415 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v118) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4580057790 con 0x7f4590156520 2024-04-25T10:12:17.545 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:12:17.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 -- 172.21.15.133:0/2775691415 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4570066f60 msgr2=0x7f4570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4570066f60 0x7f4570069420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f45880aa000 tx=0x7f4588000ff0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 -- 172.21.15.133:0/2775691415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4590156520 msgr2=0x7f45901349c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4590156520 0x7f45901349c0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f459006f600 tx=0x7f458000f520 comp rx=0 tx=0).stop 2024-04-25T10:12:17.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 -- 172.21.15.133:0/2775691415 shutdown_connections 2024-04-25T10:12:17.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4570066f60 0x7f4570069420 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4590156520 0x7f45901349c0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4590152a90 0x7f4590155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 --2- 172.21.15.133:0/2775691415 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f457c005bd0 0x7f4590160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 -- 172.21.15.133:0/2775691415 >> 172.21.15.133:0/2775691415 conn(0x7f4590066f70 msgr2=0x7f459015a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:17.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 -- 172.21.15.133:0/2775691415 shutdown_connections 2024-04-25T10:12:17.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.545+0000 7f4595d05640 1 -- 172.21.15.133:0/2775691415 wait complete. 2024-04-25T10:12:17.584 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:12:17.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1cc4c8640 1 Processor -- start 2024-04-25T10:12:17.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1cc4c8640 1 -- start start 2024-04-25T10:12:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1cc4c8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4143ea0 0x7ff1c40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1cc4c8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff1c406eac0 0x7ff1c40b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1cc4c8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c415ac50 con 0x7ff1c40b2150 2024-04-25T10:12:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1cc4c8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c40cd180 con 0x7ff1c406eac0 2024-04-25T10:12:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1cc4c8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c415aed0 con 0x7ff1c4143ea0 2024-04-25T10:12:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9a63640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff1c40b2150 0x7ff1c406e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46644/0 (socket says 172.21.15.133:46644) 2024-04-25T10:12:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9a63640 1 -- 172.21.15.133:0/1663681768 learned_addr learned my addr 172.21.15.133:0/1663681768 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1ca264640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4143ea0 0x7ff1c40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1caa65640 1 --2- 172.21.15.133:0/1663681768 >> v2:172.21.15.162:3300/0 conn(0x7ff1c406eac0 0x7ff1c40b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118248172 0 0) 0x7ff1b4001120 con 0x7ff1c40b2150 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b0003530 con 0x7ff1c40b2150 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3205185873 0 0) 0x7ff1b0003530 con 0x7ff1c40b2150 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4143ea0 msgr2=0x7ff1c40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4143ea0 0x7ff1c40cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 >> v2:172.21.15.162:3300/0 conn(0x7ff1c406eac0 msgr2=0x7ff1c40b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 --2- 172.21.15.133:0/1663681768 >> v2:172.21.15.162:3300/0 conn(0x7ff1c406eac0 0x7ff1c40b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c415b1b0 con 0x7ff1c40b2150 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff1b4001850 con 0x7ff1c40b2150 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 >> v1:172.21.15.133:6789/0 conn(0x7ff1c40b2150 legacy=0x7ff1c406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4143ea0 0x7ff1b00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 0x7ff1b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1b0004d50 0x7ff1b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40cd180 con 0x7ff1b0003f40 2024-04-25T10:12:17.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c415aed0 con 0x7ff1b0004d50 2024-04-25T10:12:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b000b9d0 con 0x7ff1c4143ea0 2024-04-25T10:12:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9a63640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 0x7ff1b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1ca264640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4143ea0 0x7ff1b00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1caa65640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1b0004d50 0x7ff1b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9a63640 1 -- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4143ea0 msgr2=0x7ff1b00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9a63640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4143ea0 0x7ff1b00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9a63640 1 -- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1b0004d50 msgr2=0x7ff1b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9a63640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1b0004d50 0x7ff1b00051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1c9a63640 1 -- 172.21.15.133:0/1663681768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b0003530 con 0x7ff1b0003f40 2024-04-25T10:12:17.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.713+0000 7ff1ca264640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4143ea0 0x7ff1b00039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:17.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1c9a63640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 0x7ff1b0004810 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c4c0 tx=0x7ff1b401f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1428a98ca38325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b400c0e0 con 0x7ff1b0003f40 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b4005b00 con 0x7ff1b0003f40 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1c9262640 1 -- 172.21.15.133:0/1663681768 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4007070 con 0x7ff1b0003f40 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1cc4c8640 1 -- 172.21.15.133:0/1663681768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 msgr2=0x7ff1b0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1cc4c8640 1 --2- 172.21.15.133:0/1663681768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 0x7ff1b0004810 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c4c0 tx=0x7ff1b401f3e0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1caa65640 1 -- 172.21.15.133:0/1663681768 reap_dead start 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1cc4c8640 1 -- 172.21.15.133:0/1663681768 shutdown_connections 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1cc4c8640 1 -- 172.21.15.133:0/1663681768 >> 172.21.15.133:0/1663681768 conn(0x7ff1c4066f70 msgr2=0x7ff1c406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1cc4c8640 1 -- 172.21.15.133:0/1663681768 shutdown_connections 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1cc4c8640 1 -- 172.21.15.133:0/1663681768 wait complete. 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1cc4c8640 1 Processor -- start 2024-04-25T10:12:17.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.717+0000 7ff1cc4c8640 1 -- start start 2024-04-25T10:12:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1cc4c8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 0x7ff1c41f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1cc4c8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4154600 0x7ff1c41579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1cc4c8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1c4154a80 0x7ff1c4157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1cc4c8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c415bb30 con 0x7ff1b0003f40 2024-04-25T10:12:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1cc4c8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c415b9b0 con 0x7ff1c4154a80 2024-04-25T10:12:17.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1cc4c8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c4144280 con 0x7ff1c4154600 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1ca264640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 0x7ff1c41f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1ca264640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 0x7ff1c41f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43750/0 (socket says 172.21.15.133:43750) 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1ca264640 1 -- 172.21.15.133:0/2303877495 learned_addr learned my addr 172.21.15.133:0/2303877495 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1c9a63640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4154600 0x7ff1c41579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1caa65640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1c4154a80 0x7ff1c4157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1ca264640 1 -- 172.21.15.133:0/2303877495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4154600 msgr2=0x7ff1c41579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1ca264640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4154600 0x7ff1c41579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1ca264640 1 -- 172.21.15.133:0/2303877495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1c4154a80 msgr2=0x7ff1c4157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1ca264640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1c4154a80 0x7ff1c4157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1ca264640 1 -- 172.21.15.133:0/2303877495 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c41522a0 con 0x7ff1b0003f40 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1c9a63640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4154600 0x7ff1c41579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1ca264640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 0x7ff1c41f3de0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00108d0 tx=0x7ff1c0010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1baffd640 1 -- 172.21.15.133:0/2303877495 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c000f040 con 0x7ff1b0003f40 2024-04-25T10:12:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1baffd640 1 -- 172.21.15.133:0/2303877495 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c0004220 con 0x7ff1b0003f40 2024-04-25T10:12:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1cc4c8640 1 -- 172.21.15.133:0/2303877495 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c4152580 con 0x7ff1b0003f40 2024-04-25T10:12:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1baffd640 1 -- 172.21.15.133:0/2303877495 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c000e960 con 0x7ff1b0003f40 2024-04-25T10:12:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1cc4c8640 1 -- 172.21.15.133:0/2303877495 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c4152b40 con 0x7ff1b0003f40 2024-04-25T10:12:17.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1cc4c8640 1 -- 172.21.15.133:0/2303877495 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c4068e80 con 0x7ff1b0003f40 2024-04-25T10:12:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1baffd640 1 -- 172.21.15.133:0/2303877495 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1c000eb00 con 0x7ff1b0003f40 2024-04-25T10:12:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1baffd640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1a8066f60 0x7ff1a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1baffd640 1 -- 172.21.15.133:0/2303877495 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6001+0+0 (secure 0 0 0) 0x7ff1c0085540 con 0x7ff1b0003f40 2024-04-25T10:12:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1c9a63640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1a8066f60 0x7ff1a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.721+0000 7ff1c9a63640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1a8066f60 0x7ff1a8069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c490 tx=0x7ff1b4001c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.729+0000 7ff1baffd640 1 -- 172.21.15.133:0/2303877495 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1c0018030 con 0x7ff1b0003f40 2024-04-25T10:12:17.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:17.921+0000 7ff1cc4c8640 1 -- 172.21.15.133:0/2303877495 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff1c406d1a0 con 0x7ff1b0003f40 2024-04-25T10:12:18.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.601+0000 7ff1baffd640 1 -- 172.21.15.133:0/2303877495 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v70) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff1c0056420 con 0x7ff1b0003f40 2024-04-25T10:12:18.626 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T10:12:18.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 -- 172.21.15.133:0/2303877495 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1a8066f60 msgr2=0x7ff1a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1a8066f60 0x7ff1a8069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c490 tx=0x7ff1b4001c20 comp rx=0 tx=0).stop 2024-04-25T10:12:18.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 -- 172.21.15.133:0/2303877495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 msgr2=0x7ff1c41f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 0x7ff1c41f3de0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00108d0 tx=0x7ff1c0010d90 comp rx=0 tx=0).stop 2024-04-25T10:12:18.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 -- 172.21.15.133:0/2303877495 shutdown_connections 2024-04-25T10:12:18.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1c4154a80 0x7ff1c4157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4154600 0x7ff1c41579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1a8066f60 0x7ff1a8069420 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 --2- 172.21.15.133:0/2303877495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1b0003f40 0x7ff1c41f3de0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 -- 172.21.15.133:0/2303877495 >> 172.21.15.133:0/2303877495 conn(0x7ff1c4066f70 msgr2=0x7ff1c4159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:18.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 -- 172.21.15.133:0/2303877495 shutdown_connections 2024-04-25T10:12:18.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.633+0000 7ff1b8ff9640 1 -- 172.21.15.133:0/2303877495 wait complete. 2024-04-25T10:12:18.657 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:18.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.765+0000 7f56a5a98640 1 Processor -- start 2024-04-25T10:12:18.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.765+0000 7f56a5a98640 1 -- start start 2024-04-25T10:12:18.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f56a5a98640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56a015c9a0 0x7f56a00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f56a5a98640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f56a00b1f70 0x7f56a006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f56a5a98640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56a00ce330 0x7f56a00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f56a5a98640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a00cece0 con 0x7f56a00ce330 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f56a5a98640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a00c61d0 con 0x7f56a00b1f70 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f56a5a98640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00c64b0 con 0x7f56a015c9a0 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56a00ce330 0x7f56a00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56a00ce330 0x7f56a00ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43754/0 (socket says 172.21.15.133:43754) 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569ffff640 1 -- 172.21.15.133:0/2242592630 learned_addr learned my addr 172.21.15.133:0/2242592630 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569f7fe640 1 --2- 172.21.15.133:0/2242592630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56a015c9a0 0x7f56a00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569effd640 1 --2- 172.21.15.133:0/2242592630 >> v2:172.21.15.162:3300/0 conn(0x7f56a00b1f70 0x7f56a006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569ffff640 1 -- 172.21.15.133:0/2242592630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56a015c9a0 msgr2=0x7f56a00c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569ffff640 1 --2- 172.21.15.133:0/2242592630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56a015c9a0 0x7f56a00c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569ffff640 1 -- 172.21.15.133:0/2242592630 >> v2:172.21.15.162:3300/0 conn(0x7f56a00b1f70 msgr2=0x7f56a006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569ffff640 1 --2- 172.21.15.133:0/2242592630 >> v2:172.21.15.162:3300/0 conn(0x7f56a00b1f70 0x7f56a006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569ffff640 1 -- 172.21.15.133:0/2242592630 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a006d4f0 con 0x7f56a00ce330 2024-04-25T10:12:18.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569ffff640 1 --2- 172.21.15.133:0/2242592630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56a00ce330 0x7f56a00ce7a0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f56a006f500 tx=0x7f569801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83c5eb285ec7773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:18.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569e7fc640 1 -- 172.21.15.133:0/2242592630 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56980b1070 con 0x7f56a00ce330 2024-04-25T10:12:18.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569e7fc640 1 -- 172.21.15.133:0/2242592630 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56980ac620 con 0x7f56a00ce330 2024-04-25T10:12:18.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.769+0000 7f569e7fc640 1 -- 172.21.15.133:0/2242592630 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56980ac9f0 con 0x7f56a00ce330 2024-04-25T10:12:18.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- 172.21.15.133:0/2242592630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56a00ce330 msgr2=0x7f56a00ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 --2- 172.21.15.133:0/2242592630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56a00ce330 0x7f56a00ce7a0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f56a006f500 tx=0x7f569801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- 172.21.15.133:0/2242592630 shutdown_connections 2024-04-25T10:12:18.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 --2- 172.21.15.133:0/2242592630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56a00ce330 0x7f56a00ce7a0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 --2- 172.21.15.133:0/2242592630 >> v2:172.21.15.162:3300/0 conn(0x7f56a00b1f70 0x7f56a006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 --2- 172.21.15.133:0/2242592630 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56a015c9a0 0x7f56a00c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- 172.21.15.133:0/2242592630 >> 172.21.15.133:0/2242592630 conn(0x7f56a0067000 msgr2=0x7f56a006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- 172.21.15.133:0/2242592630 shutdown_connections 2024-04-25T10:12:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- 172.21.15.133:0/2242592630 wait complete. 2024-04-25T10:12:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 Processor -- start 2024-04-25T10:12:18.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- start start 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56a00b1f70 0x7f56a0148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56a00ce330 0x7f56a01481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56a015c9a0 0x7f56a01497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a015cf00 con 0x7f56a00ce330 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a015cd80 con 0x7f56a00b1f70 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a015d950 con 0x7f56a015c9a0 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56a00ce330 0x7f56a01481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56a00ce330 0x7f56a01481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43756/0 (socket says 172.21.15.133:43756) 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569effd640 1 -- 172.21.15.133:0/975214601 learned_addr learned my addr 172.21.15.133:0/975214601 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569f7fe640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56a00b1f70 0x7f56a0148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569ffff640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56a015c9a0 0x7f56a01497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569effd640 1 -- 172.21.15.133:0/975214601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56a015c9a0 msgr2=0x7f56a01497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569effd640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56a015c9a0 0x7f56a01497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569effd640 1 -- 172.21.15.133:0/975214601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56a00b1f70 msgr2=0x7f56a0148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569effd640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56a00b1f70 0x7f56a0148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569effd640 1 -- 172.21.15.133:0/975214601 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a0150360 con 0x7f56a00ce330 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569f7fe640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56a00b1f70 0x7f56a0148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f569effd640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56a00ce330 0x7f56a01481a0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f5688007e90 tx=0x7f568800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f5697fff640 1 -- 172.21.15.133:0/975214601 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5688014620 con 0x7f56a00ce330 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f5697fff640 1 -- 172.21.15.133:0/975214601 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5688019070 con 0x7f56a00ce330 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- 172.21.15.133:0/975214601 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a0150600 con 0x7f56a00ce330 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f5697fff640 1 -- 172.21.15.133:0/975214601 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5688014a00 con 0x7f56a00ce330 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- 172.21.15.133:0/975214601 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a0149d00 con 0x7f56a00ce330 2024-04-25T10:12:18.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.773+0000 7f56a5a98640 1 -- 172.21.15.133:0/975214601 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a006bc20 con 0x7f56a00ce330 2024-04-25T10:12:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.777+0000 7f5697fff640 1 -- 172.21.15.133:0/975214601 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5688014ba0 con 0x7f56a00ce330 2024-04-25T10:12:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.777+0000 7f5697fff640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5680066f90 0x7f5680069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.777+0000 7f5697fff640 1 -- 172.21.15.133:0/975214601 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5688085990 con 0x7f56a00ce330 2024-04-25T10:12:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.777+0000 7f569f7fe640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5680066f90 0x7f5680069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.777+0000 7f569f7fe640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5680066f90 0x7f5680069450 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f569000b9f0 tx=0x7f5690008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:18.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.781+0000 7f5697fff640 1 -- 172.21.15.133:0/975214601 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f56880568a0 con 0x7f56a00ce330 2024-04-25T10:12:18.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.937+0000 7f56a5a98640 1 -- 172.21.15.133:0/975214601 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56a015e570 con 0x7f56a00ce330 2024-04-25T10:12:18.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.941+0000 7f5697fff640 1 -- 172.21.15.133:0/975214601 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f56880561a0 con 0x7f56a00ce330 2024-04-25T10:12:18.944 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:18.944 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":119,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:12:17.924662+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:16.197174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:16.904937+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:18.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.941+0000 7f56a5a98640 1 -- 172.21.15.133:0/975214601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5680066f90 msgr2=0x7f5680069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.941+0000 7f56a5a98640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5680066f90 0x7f5680069450 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f569000b9f0 tx=0x7f5690008040 comp rx=0 tx=0).stop 2024-04-25T10:12:18.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.941+0000 7f56a5a98640 1 -- 172.21.15.133:0/975214601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56a00ce330 msgr2=0x7f56a01481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.941+0000 7f56a5a98640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56a00ce330 0x7f56a01481a0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f5688007e90 tx=0x7f568800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.941+0000 7f56a5a98640 1 -- 172.21.15.133:0/975214601 shutdown_connections 2024-04-25T10:12:18.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.941+0000 7f56a5a98640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56a015c9a0 0x7f56a01497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.941+0000 7f56a5a98640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5680066f90 0x7f5680069450 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.941+0000 7f56a5a98640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56a00ce330 0x7f56a01481a0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.945+0000 7f56a5a98640 1 --2- 172.21.15.133:0/975214601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56a00b1f70 0x7f56a0148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.945+0000 7f56a5a98640 1 -- 172.21.15.133:0/975214601 >> 172.21.15.133:0/975214601 conn(0x7f56a0067000 msgr2=0x7f56a015afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:18.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.945+0000 7f56a5a98640 1 -- 172.21.15.133:0/975214601 shutdown_connections 2024-04-25T10:12:18.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:18.945+0000 7f56a5a98640 1 -- 172.21.15.133:0/975214601 wait complete. 2024-04-25T10:12:18.963 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:19.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b36a3b640 1 Processor -- start 2024-04-25T10:12:19.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b36a3b640 1 -- start start 2024-04-25T10:12:19.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b36a3b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7b30161fd0 0x7f7b300d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b36a3b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b30161660 con 0x7f7b300b06f0 2024-04-25T10:12:19.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b36a3b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b300cf930 con 0x7f7b30161280 2024-04-25T10:12:19.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b36a3b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b300cc4a0 con 0x7f7b30161fd0 2024-04-25T10:12:19.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b27fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7b300b06f0 0x7f7b301618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46648/0 (socket says 172.21.15.133:46648) 2024-04-25T10:12:19.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b27fff640 1 -- 172.21.15.133:0/1183274508 learned_addr learned my addr 172.21.15.133:0/1183274508 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:19.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b34fd8640 1 --2- 172.21.15.133:0/1183274508 >> v2:172.21.15.133:3300/0 conn(0x7f7b30161fd0 0x7f7b300d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b2f7fe640 1 -- 172.21.15.133:0/1183274508 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359327837 0 0) 0x7f7b30161660 con 0x7f7b300b06f0 2024-04-25T10:12:19.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b2f7fe640 1 -- 172.21.15.133:0/1183274508 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b1c003530 con 0x7f7b300b06f0 2024-04-25T10:12:19.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b34fd8640 1 -- 172.21.15.133:0/1183274508 >> v1:172.21.15.162:6789/0 conn(0x7f7b30161280 legacy=0x7f7b300cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b34fd8640 1 -- 172.21.15.133:0/1183274508 >> v1:172.21.15.133:6789/0 conn(0x7f7b300b06f0 legacy=0x7f7b301618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b2f7fe640 1 -- 172.21.15.133:0/1183274508 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1124687374 0 0) 0x7f7b1c003530 con 0x7f7b300b06f0 2024-04-25T10:12:19.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.069+0000 7f7b34fd8640 1 -- 172.21.15.133:0/1183274508 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3006c930 con 0x7f7b30161fd0 2024-04-25T10:12:19.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b34fd8640 1 --2- 172.21.15.133:0/1183274508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b30161fd0 0x7f7b300d00f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009d90 tx=0x7f7b2801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217e6a475eedb32d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b2f7fe640 1 -- 172.21.15.133:0/1183274508 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18001680 con 0x7f7b30161fd0 2024-04-25T10:12:19.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b2f7fe640 1 -- 172.21.15.133:0/1183274508 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b280a64e0 con 0x7f7b30161fd0 2024-04-25T10:12:19.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b2f7fe640 1 -- 172.21.15.133:0/1183274508 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b280a68b0 con 0x7f7b30161fd0 2024-04-25T10:12:19.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 -- 172.21.15.133:0/1183274508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b30161fd0 msgr2=0x7f7b300d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 --2- 172.21.15.133:0/1183274508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b30161fd0 0x7f7b300d00f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009d90 tx=0x7f7b2801c890 comp rx=0 tx=0).stop 2024-04-25T10:12:19.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 -- 172.21.15.133:0/1183274508 shutdown_connections 2024-04-25T10:12:19.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 --2- 172.21.15.133:0/1183274508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b30161fd0 0x7f7b300d00f0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 -- 172.21.15.133:0/1183274508 >> 172.21.15.133:0/1183274508 conn(0x7f7b300b3230 msgr2=0x7f7b300b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:19.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 -- 172.21.15.133:0/1183274508 shutdown_connections 2024-04-25T10:12:19.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 -- 172.21.15.133:0/1183274508 wait complete. 2024-04-25T10:12:19.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 Processor -- start 2024-04-25T10:12:19.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 -- start start 2024-04-25T10:12:19.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b300b06f0 0x7f7b3015cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b30161280 0x7f7b30151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b30161fd0 0x7f7b3015f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300d0bc0 con 0x7f7b30161fd0 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300d0a40 con 0x7f7b300b06f0 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3016c420 con 0x7f7b30161280 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b34fd8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b30161fd0 0x7f7b3015f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b34fd8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b30161fd0 0x7f7b3015f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43768/0 (socket says 172.21.15.133:43768) 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b34fd8640 1 -- 172.21.15.133:0/4290743346 learned_addr learned my addr 172.21.15.133:0/4290743346 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b2ffff640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b300b06f0 0x7f7b3015cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b27fff640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b30161280 0x7f7b30151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b34fd8640 1 -- 172.21.15.133:0/4290743346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b30161280 msgr2=0x7f7b30151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b34fd8640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b30161280 0x7f7b30151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b34fd8640 1 -- 172.21.15.133:0/4290743346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b300b06f0 msgr2=0x7f7b3015cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b34fd8640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b300b06f0 0x7f7b3015cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b34fd8640 1 -- 172.21.15.133:0/4290743346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3015d130 con 0x7f7b30161fd0 2024-04-25T10:12:19.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b34fd8640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b30161fd0 0x7f7b3015f6c0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009d90 tx=0x7f7b28019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b2d7fa640 1 -- 172.21.15.133:0/4290743346 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28009450 con 0x7f7b30161fd0 2024-04-25T10:12:19.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b2d7fa640 1 -- 172.21.15.133:0/4290743346 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b280c1070 con 0x7f7b30161fd0 2024-04-25T10:12:19.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b2d7fa640 1 -- 172.21.15.133:0/4290743346 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b280b8a60 con 0x7f7b30161fd0 2024-04-25T10:12:19.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 -- 172.21.15.133:0/4290743346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3015bb50 con 0x7f7b30161fd0 2024-04-25T10:12:19.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.073+0000 7f7b36a3b640 1 -- 172.21.15.133:0/4290743346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3015c110 con 0x7f7b30161fd0 2024-04-25T10:12:19.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.077+0000 7f7b36a3b640 1 -- 172.21.15.133:0/4290743346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b300b3ec0 con 0x7f7b30161fd0 2024-04-25T10:12:19.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.077+0000 7f7b2d7fa640 1 -- 172.21.15.133:0/4290743346 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b280b8c00 con 0x7f7b30161fd0 2024-04-25T10:12:19.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.077+0000 7f7b2d7fa640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7b10066f90 0x7f7b10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.077+0000 7f7b2d7fa640 1 -- 172.21.15.133:0/4290743346 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b2811fb40 con 0x7f7b30161fd0 2024-04-25T10:12:19.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.077+0000 7f7b2ffff640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7b10066f90 0x7f7b10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.077+0000 7f7b2ffff640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7b10066f90 0x7f7b10069450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000f70 tx=0x7f7b20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.081+0000 7f7b2d7fa640 1 -- 172.21.15.133:0/4290743346 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b280ca020 con 0x7f7b30161fd0 2024-04-25T10:12:19.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.229+0000 7f7b36a3b640 1 -- 172.21.15.133:0/4290743346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b300afee0 con 0x7f7b30161fd0 2024-04-25T10:12:19.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.229+0000 7f7b2d7fa640 1 -- 172.21.15.133:0/4290743346 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f7b3015d130 con 0x7f7b30161fd0 2024-04-25T10:12:19.232 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:19.232 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":119,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:12:17.924662+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:16.197174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:16.904937+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:19.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.229+0000 7f7b36a3b640 1 -- 172.21.15.133:0/4290743346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7b10066f90 msgr2=0x7f7b10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.229+0000 7f7b36a3b640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7b10066f90 0x7f7b10069450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000f70 tx=0x7f7b20009040 comp rx=0 tx=0).stop 2024-04-25T10:12:19.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.229+0000 7f7b36a3b640 1 -- 172.21.15.133:0/4290743346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b30161fd0 msgr2=0x7f7b3015f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.229+0000 7f7b36a3b640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b30161fd0 0x7f7b3015f6c0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009d90 tx=0x7f7b28019630 comp rx=0 tx=0).stop 2024-04-25T10:12:19.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.233+0000 7f7b36a3b640 1 -- 172.21.15.133:0/4290743346 shutdown_connections 2024-04-25T10:12:19.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.233+0000 7f7b36a3b640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7b10066f90 0x7f7b10069450 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.233+0000 7f7b36a3b640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b30161fd0 0x7f7b3015f6c0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.233+0000 7f7b36a3b640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b30161280 0x7f7b30151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.233+0000 7f7b36a3b640 1 --2- 172.21.15.133:0/4290743346 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b300b06f0 0x7f7b3015cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.233+0000 7f7b36a3b640 1 -- 172.21.15.133:0/4290743346 >> 172.21.15.133:0/4290743346 conn(0x7f7b300b3230 msgr2=0x7f7b300ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:19.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.233+0000 7f7b36a3b640 1 -- 172.21.15.133:0/4290743346 shutdown_connections 2024-04-25T10:12:19.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.233+0000 7f7b36a3b640 1 -- 172.21.15.133:0/4290743346 wait complete. 2024-04-25T10:12:19.250 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:12:19.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafdd513640 1 Processor -- start 2024-04-25T10:12:19.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafdd513640 1 -- start start 2024-04-25T10:12:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafdd513640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fafd80b2150 0x7fafd806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafdd513640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd815ac50 con 0x7fafd8143ea0 2024-04-25T10:12:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafdd513640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd80cd180 con 0x7fafd815b9b0 2024-04-25T10:12:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafdd513640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd80ccf20 con 0x7fafd80b2150 2024-04-25T10:12:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd6ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fafd8143ea0 0x7fafd80cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46656/0 (socket says 172.21.15.133:46656) 2024-04-25T10:12:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd6ffd640 1 -- 172.21.15.133:0/994922614 learned_addr learned my addr 172.21.15.133:0/994922614 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767711529 0 0) 0x7fafc8001120 con 0x7fafd8143ea0 2024-04-25T10:12:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafc0003530 con 0x7fafd8143ea0 2024-04-25T10:12:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd67fc640 1 --2- 172.21.15.133:0/994922614 >> v2:172.21.15.162:3300/0 conn(0x7fafd80b2150 0x7fafd806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1078263472 0 0) 0x7fafc0003530 con 0x7fafd8143ea0 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 >> v2:172.21.15.162:3300/0 conn(0x7fafd80b2150 msgr2=0x7fafd806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 --2- 172.21.15.133:0/994922614 >> v2:172.21.15.162:3300/0 conn(0x7fafd80b2150 0x7fafd806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 >> v1:172.21.15.162:6789/0 conn(0x7fafd815b9b0 legacy=0x7fafd80b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd815aed0 con 0x7fafd8143ea0 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fafc8001850 con 0x7fafd8143ea0 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 >> v1:172.21.15.133:6789/0 conn(0x7fafd8143ea0 legacy=0x7fafd80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafc00039c0 0x7fafc0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafc0004810 0x7fafc0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafc00052f0 0x7fafc000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd80ccf20 con 0x7fafc00039c0 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc000c1c0 con 0x7fafc0004810 2024-04-25T10:12:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc000c3c0 con 0x7fafc00052f0 2024-04-25T10:12:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd6ffd640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafc00039c0 0x7fafc0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd67fc640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafc0004810 0x7fafc0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd77fe640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafc00052f0 0x7fafc000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd6ffd640 1 -- 172.21.15.133:0/994922614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafc00052f0 msgr2=0x7fafc000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd6ffd640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafc00052f0 0x7fafc000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd6ffd640 1 -- 172.21.15.133:0/994922614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafc0004810 msgr2=0x7fafc0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd6ffd640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafc0004810 0x7fafc0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd6ffd640 1 -- 172.21.15.133:0/994922614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0003530 con 0x7fafc00039c0 2024-04-25T10:12:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd67fc640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafc0004810 0x7fafc0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.361+0000 7fafd77fe640 1 -- 172.21.15.133:0/994922614 reap_dead start 2024-04-25T10:12:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafd6ffd640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafc00039c0 0x7fafc0003de0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fafc800c490 tx=0x7fafc801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d50d4b1764e64d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc800c0e0 con 0x7fafc00039c0 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc80a9070 con 0x7fafc00039c0 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafd5ffb640 1 -- 172.21.15.133:0/994922614 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc8005d80 con 0x7fafc00039c0 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 -- 172.21.15.133:0/994922614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafc00039c0 msgr2=0x7fafc0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafc00039c0 0x7fafc0003de0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fafc800c490 tx=0x7fafc801cc40 comp rx=0 tx=0).stop 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 -- 172.21.15.133:0/994922614 shutdown_connections 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 --2- 172.21.15.133:0/994922614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafc00039c0 0x7fafc0003de0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 -- 172.21.15.133:0/994922614 >> 172.21.15.133:0/994922614 conn(0x7fafd8066f70 msgr2=0x7fafd806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 -- 172.21.15.133:0/994922614 shutdown_connections 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 -- 172.21.15.133:0/994922614 wait complete. 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 Processor -- start 2024-04-25T10:12:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 -- start start 2024-04-25T10:12:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafc00039c0 0x7fafd813a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd8153cf0 0x7fafd81f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd813f250 0x7fafd813f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd806ed40 con 0x7fafd8153cf0 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd806ebc0 con 0x7fafc00039c0 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafdd513640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd815e240 con 0x7fafd813f250 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafd67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd8153cf0 0x7fafd81f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafd67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd8153cf0 0x7fafd81f8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43786/0 (socket says 172.21.15.133:43786) 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafd67fc640 1 -- 172.21.15.133:0/2145199011 learned_addr learned my addr 172.21.15.133:0/2145199011 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.365+0000 7fafd6ffd640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafc00039c0 0x7fafd813a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafd67fc640 1 -- 172.21.15.133:0/2145199011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd813f250 msgr2=0x7fafd813f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafd67fc640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd813f250 0x7fafd813f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafd67fc640 1 -- 172.21.15.133:0/2145199011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafc00039c0 msgr2=0x7fafd813a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafd67fc640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafc00039c0 0x7fafd813a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafd67fc640 1 -- 172.21.15.133:0/2145199011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd813ea90 con 0x7fafd8153cf0 2024-04-25T10:12:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafd6ffd640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafc00039c0 0x7fafd813a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafd67fc640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd8153cf0 0x7fafd81f8490 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00e5c0 tx=0x7fafcc00ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafbf7fe640 1 -- 172.21.15.133:0/2145199011 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc00f040 con 0x7fafd8153cf0 2024-04-25T10:12:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafbf7fe640 1 -- 172.21.15.133:0/2145199011 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc004230 con 0x7fafd8153cf0 2024-04-25T10:12:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafdd513640 1 -- 172.21.15.133:0/2145199011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd81f8d30 con 0x7fafd8153cf0 2024-04-25T10:12:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafbf7fe640 1 -- 172.21.15.133:0/2145199011 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc003620 con 0x7fafd8153cf0 2024-04-25T10:12:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafdd513640 1 -- 172.21.15.133:0/2145199011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd813ed30 con 0x7fafd8153cf0 2024-04-25T10:12:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafdd513640 1 -- 172.21.15.133:0/2145199011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd8068e80 con 0x7fafd8153cf0 2024-04-25T10:12:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafbf7fe640 1 -- 172.21.15.133:0/2145199011 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafcc0037c0 con 0x7fafd8153cf0 2024-04-25T10:12:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafbf7fe640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafc4066f60 0x7fafc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafbf7fe640 1 -- 172.21.15.133:0/2145199011 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fafcc084b90 con 0x7fafd8153cf0 2024-04-25T10:12:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafd6ffd640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafc4066f60 0x7fafc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.369+0000 7fafd6ffd640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafc4066f60 0x7fafc4069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ab000 tx=0x7fafc80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.373+0000 7fafbf7fe640 1 -- 172.21.15.133:0/2145199011 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fafcc017020 con 0x7fafd8153cf0 2024-04-25T10:12:19.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.553+0000 7fafdd513640 1 -- 172.21.15.133:0/2145199011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fafd815ee50 con 0x7fafd8153cf0 2024-04-25T10:12:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.605+0000 7fafbf7fe640 1 -- 172.21.15.133:0/2145199011 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v71) v1 ==== 118+0+0 (secure 0 0 0) 0x7fafcc055a20 con 0x7fafd8153cf0 2024-04-25T10:12:19.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 -- 172.21.15.133:0/2145199011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafc4066f60 msgr2=0x7fafc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafc4066f60 0x7fafc4069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ab000 tx=0x7fafc80a7070 comp rx=0 tx=0).stop 2024-04-25T10:12:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 -- 172.21.15.133:0/2145199011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd8153cf0 msgr2=0x7fafd81f8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd8153cf0 0x7fafd81f8490 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00e5c0 tx=0x7fafcc00ea80 comp rx=0 tx=0).stop 2024-04-25T10:12:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 -- 172.21.15.133:0/2145199011 shutdown_connections 2024-04-25T10:12:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd813f250 0x7fafd813f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafc4066f60 0x7fafc4069420 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd8153cf0 0x7fafd81f8490 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 --2- 172.21.15.133:0/2145199011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafc00039c0 0x7fafd813a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 -- 172.21.15.133:0/2145199011 >> 172.21.15.133:0/2145199011 conn(0x7fafd8066f70 msgr2=0x7fafd8159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 -- 172.21.15.133:0/2145199011 shutdown_connections 2024-04-25T10:12:19.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.609+0000 7fafdd513640 1 -- 172.21.15.133:0/2145199011 wait complete. 2024-04-25T10:12:19.629 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:12:19.629 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:12:19.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9ebb25c640 1 Processor -- start 2024-04-25T10:12:19.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9ebb25c640 1 -- start start 2024-04-25T10:12:19.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9ebb25c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9eb415bbb0 0x7f9eb40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9ebb25c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9eb40cc330 0x7f9eb40ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9ebb25c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb40afe00 con 0x7f9eb40cc790 2024-04-25T10:12:19.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9ebb25c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb40b0730 con 0x7f9eb415bbb0 2024-04-25T10:12:19.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9ebb25c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb40b00e0 con 0x7f9eb40cc330 2024-04-25T10:12:19.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9eb8ff8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9eb415bbb0 0x7f9eb40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9eb8ff8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9eb415bbb0 0x7f9eb40b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43802/0 (socket says 172.21.15.133:43802) 2024-04-25T10:12:19.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9eb8ff8640 1 -- 172.21.15.133:0/3575684535 learned_addr learned my addr 172.21.15.133:0/3575684535 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:19.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9eb3fff640 1 --2- 172.21.15.133:0/3575684535 >> v2:172.21.15.162:3300/0 conn(0x7f9eb40cc330 0x7f9eb40ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9eb8ff8640 1 -- 172.21.15.133:0/3575684535 >> v2:172.21.15.162:3300/0 conn(0x7f9eb40cc330 msgr2=0x7f9eb40ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:19.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9eb8ff8640 1 --2- 172.21.15.133:0/3575684535 >> v2:172.21.15.162:3300/0 conn(0x7f9eb40cc330 0x7f9eb40ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9eb8ff8640 1 -- 172.21.15.133:0/3575684535 >> v1:172.21.15.162:6789/0 conn(0x7f9eb40cc790 legacy=0x7f9eb406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:19.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9eb8ff8640 1 -- 172.21.15.133:0/3575684535 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb40b03c0 con 0x7f9eb415bbb0 2024-04-25T10:12:19.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.745+0000 7f9eb3fff640 1 --2- 172.21.15.133:0/3575684535 >> v2:172.21.15.162:3300/0 conn(0x7f9eb40cc330 0x7f9eb40ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:19.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb8ff8640 1 --2- 172.21.15.133:0/3575684535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb415bbb0 0x7f9eb40b0bc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40ce030 tx=0x7f9e9c01ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484d5d257d086e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb37fe640 1 -- 172.21.15.133:0/3575684535 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0b0070 con 0x7f9eb415bbb0 2024-04-25T10:12:19.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb37fe640 1 -- 172.21.15.133:0/3575684535 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c0ab600 con 0x7f9eb415bbb0 2024-04-25T10:12:19.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb37fe640 1 -- 172.21.15.133:0/3575684535 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0ab9d0 con 0x7f9eb415bbb0 2024-04-25T10:12:19.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 -- 172.21.15.133:0/3575684535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb415bbb0 msgr2=0x7f9eb40b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 --2- 172.21.15.133:0/3575684535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb415bbb0 0x7f9eb40b0bc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40ce030 tx=0x7f9e9c01ce20 comp rx=0 tx=0).stop 2024-04-25T10:12:19.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 -- 172.21.15.133:0/3575684535 shutdown_connections 2024-04-25T10:12:19.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 --2- 172.21.15.133:0/3575684535 >> v2:172.21.15.162:3300/0 conn(0x7f9eb40cc330 0x7f9eb40ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 --2- 172.21.15.133:0/3575684535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb415bbb0 0x7f9eb40b0bc0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 -- 172.21.15.133:0/3575684535 >> 172.21.15.133:0/3575684535 conn(0x7f9eb40d0570 msgr2=0x7f9eb40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:19.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 -- 172.21.15.133:0/3575684535 shutdown_connections 2024-04-25T10:12:19.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 -- 172.21.15.133:0/3575684535 wait complete. 2024-04-25T10:12:19.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 Processor -- start 2024-04-25T10:12:19.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 -- start start 2024-04-25T10:12:19.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9eb40cc330 0x7f9eb40c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb40cc790 0x7f9eb40c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb415bbb0 0x7f9eb40cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb41687c0 con 0x7f9eb415bbb0 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb4168640 con 0x7f9eb40cc330 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9ebb25c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb4168940 con 0x7f9eb40cc790 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb97f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb415bbb0 0x7f9eb40cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb97f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb415bbb0 0x7f9eb40cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43808/0 (socket says 172.21.15.133:43808) 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb97f9640 1 -- 172.21.15.133:0/941237697 learned_addr learned my addr 172.21.15.133:0/941237697 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb8ff8640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9eb40cc330 0x7f9eb40c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb97f9640 1 -- 172.21.15.133:0/941237697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb40cc790 msgr2=0x7f9eb40c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb97f9640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb40cc790 0x7f9eb40c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb97f9640 1 -- 172.21.15.133:0/941237697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9eb40cc330 msgr2=0x7f9eb40c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb97f9640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9eb40cc330 0x7f9eb40c40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb97f9640 1 -- 172.21.15.133:0/941237697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb40c9d70 con 0x7f9eb415bbb0 2024-04-25T10:12:19.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.749+0000 7f9eb97f9640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb415bbb0 0x7f9eb40cfa50 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00c780 tx=0x7f9eac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.753+0000 7f9eb17fa640 1 -- 172.21.15.133:0/941237697 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac004b70 con 0x7f9eb415bbb0 2024-04-25T10:12:19.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.753+0000 7f9eb17fa640 1 -- 172.21.15.133:0/941237697 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac004d10 con 0x7f9eb415bbb0 2024-04-25T10:12:19.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.753+0000 7f9ebb25c640 1 -- 172.21.15.133:0/941237697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb40ca050 con 0x7f9eb415bbb0 2024-04-25T10:12:19.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.753+0000 7f9eb17fa640 1 -- 172.21.15.133:0/941237697 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac00aa90 con 0x7f9eb415bbb0 2024-04-25T10:12:19.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.753+0000 7f9ebb25c640 1 -- 172.21.15.133:0/941237697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb40b33c0 con 0x7f9eb415bbb0 2024-04-25T10:12:19.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.753+0000 7f9ebb25c640 1 -- 172.21.15.133:0/941237697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb4067c70 con 0x7f9eb415bbb0 2024-04-25T10:12:19.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.753+0000 7f9eb17fa640 1 -- 172.21.15.133:0/941237697 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9eac00ac30 con 0x7f9eb415bbb0 2024-04-25T10:12:19.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.753+0000 7f9eb17fa640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e94066f60 0x7f9e94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.753+0000 7f9eb17fa640 1 -- 172.21.15.133:0/941237697 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9eac0853e0 con 0x7f9eb415bbb0 2024-04-25T10:12:19.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.753+0000 7f9eb8ff8640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e94066f60 0x7f9e94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.753+0000 7f9eb8ff8640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e94066f60 0x7f9e94069420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0ad000 tx=0x7f9e9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.757+0000 7f9eb17fa640 1 -- 172.21.15.133:0/941237697 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9eac020070 con 0x7f9eb415bbb0 2024-04-25T10:12:19.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:19.953+0000 7f9ebb25c640 1 -- 172.21.15.133:0/941237697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9eb40ade30 con 0x7f9eb415bbb0 2024-04-25T10:12:20.538 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-04-25T10:12:20.537+0000 7f013b236640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:12:20.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.609+0000 7f9eb17fa640 1 -- 172.21.15.133:0/941237697 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v72) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9eac0562f0 con 0x7f9eb415bbb0 2024-04-25T10:12:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 -- 172.21.15.133:0/941237697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e94066f60 msgr2=0x7f9e94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e94066f60 0x7f9e94069420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0ad000 tx=0x7f9e9c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 -- 172.21.15.133:0/941237697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb415bbb0 msgr2=0x7f9eb40cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb415bbb0 0x7f9eb40cfa50 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00c780 tx=0x7f9eac00cc40 comp rx=0 tx=0).stop 2024-04-25T10:12:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 -- 172.21.15.133:0/941237697 shutdown_connections 2024-04-25T10:12:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9e94066f60 0x7f9e94069420 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9eb415bbb0 0x7f9eb40cfa50 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9eb40cc790 0x7f9eb40c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 --2- 172.21.15.133:0/941237697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9eb40cc330 0x7f9eb40c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 -- 172.21.15.133:0/941237697 >> 172.21.15.133:0/941237697 conn(0x7f9eb40d0570 msgr2=0x7f9eb40d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 -- 172.21.15.133:0/941237697 shutdown_connections 2024-04-25T10:12:20.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.613+0000 7f9ebb25c640 1 -- 172.21.15.133:0/941237697 wait complete. 2024-04-25T10:12:20.638 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:12:20.638 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:20.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.757+0000 7f24ddd0f640 1 Processor -- start 2024-04-25T10:12:20.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.757+0000 7f24ddd0f640 1 -- start start 2024-04-25T10:12:20.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24ddd0f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d806eb70 0x7f24d80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24ddd0f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f24d81584f0 0x7f24d815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24ddd0f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d80cbe60 con 0x7f24d8158110 2024-04-25T10:12:20.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24ddd0f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d80b19b0 con 0x7f24d81584f0 2024-04-25T10:12:20.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24ddd0f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d815b360 con 0x7f24d806eb70 2024-04-25T10:12:20.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d77fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f24d8158110 0x7f24d806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46664/0 (socket says 172.21.15.133:46664) 2024-04-25T10:12:20.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d77fe640 1 -- 172.21.15.133:0/2409964141 learned_addr learned my addr 172.21.15.133:0/2409964141 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:20.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d7fff640 1 --2- 172.21.15.133:0/2409964141 >> v2:172.21.15.162:3300/0 conn(0x7f24d81584f0 0x7f24d815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d6ffd640 1 --2- 172.21.15.133:0/2409964141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d806eb70 0x7f24d80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931022372 0 0) 0x7f24d80cbe60 con 0x7f24d8158110 2024-04-25T10:12:20.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24c0005740 con 0x7f24d8158110 2024-04-25T10:12:20.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2459690870 0 0) 0x7f24c0005740 con 0x7f24d8158110 2024-04-25T10:12:20.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d806eb70 msgr2=0x7f24d80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 --2- 172.21.15.133:0/2409964141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d806eb70 0x7f24d80b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 >> v2:172.21.15.162:3300/0 conn(0x7f24d81584f0 msgr2=0x7f24d815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 --2- 172.21.15.133:0/2409964141 >> v2:172.21.15.162:3300/0 conn(0x7f24d81584f0 0x7f24d815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d815b640 con 0x7f24d8158110 2024-04-25T10:12:20.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f24c8001680 con 0x7f24d8158110 2024-04-25T10:12:20.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 >> v1:172.21.15.133:6789/0 conn(0x7f24d8158110 legacy=0x7f24d806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 --2- 172.21.15.133:0/2409964141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d806eb70 0x7f24c0005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 --2- 172.21.15.133:0/2409964141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 0x7f24c0006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 --2- 172.21.15.133:0/2409964141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c0006f60 0x7f24c00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80b19b0 con 0x7f24c0006150 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d815b360 con 0x7f24c0006f60 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c000dbe0 con 0x7f24d806eb70 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d6ffd640 1 --2- 172.21.15.133:0/2409964141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 0x7f24c0006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d6ffd640 1 -- 172.21.15.133:0/2409964141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d806eb70 msgr2=0x7f24c0005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d6ffd640 1 --2- 172.21.15.133:0/2409964141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d806eb70 0x7f24c0005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d6ffd640 1 -- 172.21.15.133:0/2409964141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c0006f60 msgr2=0x7f24c00073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d6ffd640 1 --2- 172.21.15.133:0/2409964141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c0006f60 0x7f24c00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.761+0000 7f24d6ffd640 1 -- 172.21.15.133:0/2409964141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c0005740 con 0x7f24c0006150 2024-04-25T10:12:20.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d6ffd640 1 --2- 172.21.15.133:0/2409964141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 0x7f24c0006a20 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009ae0 tx=0x7f24cc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7819506baba2063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc0b2070 con 0x7f24c0006150 2024-04-25T10:12:20.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc0a56a0 con 0x7f24c0006150 2024-04-25T10:12:20.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d67fc640 1 -- 172.21.15.133:0/2409964141 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc0096b0 con 0x7f24c0006150 2024-04-25T10:12:20.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 -- 172.21.15.133:0/2409964141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 msgr2=0x7f24c0006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 --2- 172.21.15.133:0/2409964141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 0x7f24c0006a20 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009ae0 tx=0x7f24cc01ca50 comp rx=0 tx=0).stop 2024-04-25T10:12:20.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d7fff640 1 -- 172.21.15.133:0/2409964141 reap_dead start 2024-04-25T10:12:20.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 -- 172.21.15.133:0/2409964141 shutdown_connections 2024-04-25T10:12:20.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 -- 172.21.15.133:0/2409964141 >> 172.21.15.133:0/2409964141 conn(0x7f24d8067000 msgr2=0x7f24d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 -- 172.21.15.133:0/2409964141 shutdown_connections 2024-04-25T10:12:20.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 -- 172.21.15.133:0/2409964141 wait complete. 2024-04-25T10:12:20.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 Processor -- start 2024-04-25T10:12:20.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 -- start start 2024-04-25T10:12:20.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 0x7f24d8159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d814b230 0x7f24d815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d814ac70 0x7f24d814a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80cd670 con 0x7f24c0006150 2024-04-25T10:12:20.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d815bc00 con 0x7f24d814b230 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80cd7f0 con 0x7f24d814ac70 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 0x7f24d8159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 0x7f24d8159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43816/0 (socket says 172.21.15.133:43816) 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d77fe640 1 -- 172.21.15.133:0/4215263187 learned_addr learned my addr 172.21.15.133:0/4215263187 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d77fe640 1 -- 172.21.15.133:0/4215263187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d814ac70 msgr2=0x7f24d814a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d77fe640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d814ac70 0x7f24d814a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d77fe640 1 -- 172.21.15.133:0/4215263187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d814b230 msgr2=0x7f24d815cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d77fe640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d814b230 0x7f24d815cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d77fe640 1 -- 172.21.15.133:0/4215263187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d8158d10 con 0x7f24c0006150 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24d77fe640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 0x7f24d8159db0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f24c800f700 tx=0x7f24c800fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24bffff640 1 -- 172.21.15.133:0/4215263187 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c801b070 con 0x7f24c0006150 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24bffff640 1 -- 172.21.15.133:0/4215263187 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c8007b30 con 0x7f24c0006150 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 -- 172.21.15.133:0/4215263187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d8158ff0 con 0x7f24c0006150 2024-04-25T10:12:20.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24bffff640 1 -- 172.21.15.133:0/4215263187 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800a0d0 con 0x7f24c0006150 2024-04-25T10:12:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.765+0000 7f24ddd0f640 1 -- 172.21.15.133:0/4215263187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d81595b0 con 0x7f24c0006150 2024-04-25T10:12:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.769+0000 7f24ddd0f640 1 -- 172.21.15.133:0/4215263187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d806bc20 con 0x7f24c0006150 2024-04-25T10:12:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.769+0000 7f24bffff640 1 -- 172.21.15.133:0/4215263187 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24c801e070 con 0x7f24c0006150 2024-04-25T10:12:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.769+0000 7f24bffff640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f24c4066f60 0x7f24c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.769+0000 7f24bffff640 1 -- 172.21.15.133:0/4215263187 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f24c8087500 con 0x7f24c0006150 2024-04-25T10:12:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.769+0000 7f24d6ffd640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f24c4066f60 0x7f24c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.769+0000 7f24d6ffd640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f24c4066f60 0x7f24c4069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0af000 tx=0x7f24cc004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.773+0000 7f24bffff640 1 -- 172.21.15.133:0/4215263187 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24c8058380 con 0x7f24c0006150 2024-04-25T10:12:20.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.949+0000 7f24ddd0f640 1 -- 172.21.15.133:0/4215263187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24d80c5400 con 0x7f24c0006150 2024-04-25T10:12:20.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.949+0000 7f24bffff640 1 -- 172.21.15.133:0/4215263187 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f24c8057c80 con 0x7f24c0006150 2024-04-25T10:12:20.952 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:20.952 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:20.953 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:20.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 -- 172.21.15.133:0/4215263187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f24c4066f60 msgr2=0x7f24c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f24c4066f60 0x7f24c4069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0af000 tx=0x7f24cc004740 comp rx=0 tx=0).stop 2024-04-25T10:12:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 -- 172.21.15.133:0/4215263187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 msgr2=0x7f24d8159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 0x7f24d8159db0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f24c800f700 tx=0x7f24c800fbc0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 -- 172.21.15.133:0/4215263187 shutdown_connections 2024-04-25T10:12:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24d814ac70 0x7f24d814a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24d814b230 0x7f24d815cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f24c4066f60 0x7f24c4069420 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 --2- 172.21.15.133:0/4215263187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c0006150 0x7f24d8159db0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 -- 172.21.15.133:0/4215263187 >> 172.21.15.133:0/4215263187 conn(0x7f24d8067000 msgr2=0x7f24d80caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 -- 172.21.15.133:0/4215263187 shutdown_connections 2024-04-25T10:12:20.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:20.953+0000 7f24ddd0f640 1 -- 172.21.15.133:0/4215263187 wait complete. 2024-04-25T10:12:20.970 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd79a4ee640 1 Processor -- start 2024-04-25T10:12:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd79a4ee640 1 -- start start 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd79a4ee640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd7940d0770 0x7fd7940d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd79a4ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7940b87e0 0x7fd7940b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd79a4ee640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7940ceb20 con 0x7fd79415db00 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd79a4ee640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7940d1110 con 0x7fd7940d0770 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd79a4ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b30a0 con 0x7fd7940b87e0 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd793fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd79415db00 0x7fd7940b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46674/0 (socket says 172.21.15.133:46674) 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd793fff640 1 -- 172.21.15.133:0/4007614825 learned_addr learned my addr 172.21.15.133:0/4007614825 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938098434 0 0) 0x7fd7940ceb20 con 0x7fd79415db00 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd77c003530 con 0x7fd79415db00 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd7937fe640 1 --2- 172.21.15.133:0/4007614825 >> v2:172.21.15.162:3300/0 conn(0x7fd7940d0770 0x7fd7940d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd798a8b640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7940b87e0 0x7fd7940b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 436022891 0 0) 0x7fd77c003530 con 0x7fd79415db00 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7940b87e0 msgr2=0x7fd7940b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7940b87e0 0x7fd7940b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 >> v2:172.21.15.162:3300/0 conn(0x7fd7940d0770 msgr2=0x7fd7940d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 --2- 172.21.15.133:0/4007614825 >> v2:172.21.15.162:3300/0 conn(0x7fd7940d0770 0x7fd7940d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940b7740 con 0x7fd79415db00 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fd784001680 con 0x7fd79415db00 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 >> v1:172.21.15.133:6789/0 conn(0x7fd79415db00 legacy=0x7fd7940b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c0039c0 0x7fd77c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7940b87e0 0x7fd77c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd77c004d50 0x7fd77c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b30a0 con 0x7fd77c004d50 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c00ba80 con 0x7fd77c0039c0 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c00bc30 con 0x7fd7940b87e0 2024-04-25T10:12:21.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd798a8b640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd77c004d50 0x7fd77c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd793fff640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c0039c0 0x7fd77c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd7937fe640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7940b87e0 0x7fd77c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd798a8b640 1 -- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7940b87e0 msgr2=0x7fd77c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd798a8b640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7940b87e0 0x7fd77c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.077+0000 7fd798a8b640 1 -- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c0039c0 msgr2=0x7fd77c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd798a8b640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c0039c0 0x7fd77c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd798a8b640 1 -- 172.21.15.133:0/4007614825 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c003530 con 0x7fd77c004d50 2024-04-25T10:12:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd793fff640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd77c0039c0 0x7fd77c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd7937fe640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7940b87e0 0x7fd77c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd798a8b640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd77c004d50 0x7fd77c0051d0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fd78c007360 tx=0x7fd78c01cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38b283bdbd3bed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0b2070 con 0x7fd77c004d50 2024-04-25T10:12:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c0a5d40 con 0x7fd77c004d50 2024-04-25T10:12:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd792ffd640 1 -- 172.21.15.133:0/4007614825 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c009be0 con 0x7fd77c004d50 2024-04-25T10:12:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4007614825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd77c004d50 msgr2=0x7fd77c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd79a4ee640 1 --2- 172.21.15.133:0/4007614825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd77c004d50 0x7fd77c0051d0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fd78c007360 tx=0x7fd78c01cde0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd798a8b640 1 -- 172.21.15.133:0/4007614825 reap_dead start 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4007614825 shutdown_connections 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4007614825 >> 172.21.15.133:0/4007614825 conn(0x7fd7940727c0 msgr2=0x7fd794072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4007614825 shutdown_connections 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4007614825 wait complete. 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.081+0000 7fd79a4ee640 1 Processor -- start 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd79a4ee640 1 -- start start 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd79a4ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd794160980 0x7fd79414aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd79a4ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd794155710 0x7fd794146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd79a4ee640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd794146f90 0x7fd7941f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd79a4ee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b7fe0 con 0x7fd794160980 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd79a4ee640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b7d00 con 0x7fd794146f90 2024-04-25T10:12:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd79a4ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b82c0 con 0x7fd794155710 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd793fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd794160980 0x7fd79414aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd793fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd794160980 0x7fd79414aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43838/0 (socket says 172.21.15.133:43838) 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd793fff640 1 -- 172.21.15.133:0/4157089073 learned_addr learned my addr 172.21.15.133:0/4157089073 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd793fff640 1 -- 172.21.15.133:0/4157089073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd794155710 msgr2=0x7fd794146a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd7937fe640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd794155710 0x7fd794146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd793fff640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd794155710 0x7fd794146a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd793fff640 1 -- 172.21.15.133:0/4157089073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd794146f90 msgr2=0x7fd7941f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd793fff640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd794146f90 0x7fd7941f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd793fff640 1 -- 172.21.15.133:0/4157089073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd794157fa0 con 0x7fd794160980 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd793fff640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd794160980 0x7fd79414aad0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fd78400dae0 tx=0x7fd784001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd790ff9640 1 -- 172.21.15.133:0/4157089073 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784019070 con 0x7fd794160980 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd790ff9640 1 -- 172.21.15.133:0/4157089073 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd784004cc0 con 0x7fd794160980 2024-04-25T10:12:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4157089073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd794158280 con 0x7fd794160980 2024-04-25T10:12:21.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd790ff9640 1 -- 172.21.15.133:0/4157089073 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784007310 con 0x7fd794160980 2024-04-25T10:12:21.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4157089073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd794158840 con 0x7fd794160980 2024-04-25T10:12:21.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4157089073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7941473b0 con 0x7fd794160980 2024-04-25T10:12:21.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd790ff9640 1 -- 172.21.15.133:0/4157089073 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd78401c070 con 0x7fd794160980 2024-04-25T10:12:21.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd790ff9640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd780066f90 0x7fd780069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd790ff9640 1 -- 172.21.15.133:0/4157089073 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd784084b50 con 0x7fd794160980 2024-04-25T10:12:21.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd7937fe640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd780066f90 0x7fd780069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.085+0000 7fd7937fe640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd780066f90 0x7fd780069450 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd7940b51f0 tx=0x7fd788008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.089+0000 7fd790ff9640 1 -- 172.21.15.133:0/4157089073 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd784055a60 con 0x7fd794160980 2024-04-25T10:12:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.237+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4157089073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd794068ef0 con 0x7fd794160980 2024-04-25T10:12:21.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd790ff9640 1 -- 172.21.15.133:0/4157089073 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fd784055360 con 0x7fd794160980 2024-04-25T10:12:21.243 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:21.243 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":120,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:12:19.584218+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:16.197174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:16.904937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4157089073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd780066f90 msgr2=0x7fd780069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd780066f90 0x7fd780069450 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd7940b51f0 tx=0x7fd788008040 comp rx=0 tx=0).stop 2024-04-25T10:12:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4157089073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd794160980 msgr2=0x7fd79414aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd794160980 0x7fd79414aad0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fd78400dae0 tx=0x7fd784001710 comp rx=0 tx=0).stop 2024-04-25T10:12:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4157089073 shutdown_connections 2024-04-25T10:12:21.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd794146f90 0x7fd7941f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd794155710 0x7fd794146a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd780066f90 0x7fd780069450 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 --2- 172.21.15.133:0/4157089073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd794160980 0x7fd79414aad0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4157089073 >> 172.21.15.133:0/4157089073 conn(0x7fd7940727c0 msgr2=0x7fd79406c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:21.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4157089073 shutdown_connections 2024-04-25T10:12:21.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.241+0000 7fd79a4ee640 1 -- 172.21.15.133:0/4157089073 wait complete. 2024-04-25T10:12:21.261 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:12:21.261 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:21.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.365+0000 7f9a54b69640 1 Processor -- start 2024-04-25T10:12:21.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.365+0000 7f9a54b69640 1 -- start start 2024-04-25T10:12:21.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a54b69640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a500d0bd0 0x7f9a500b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a54b69640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a500ceb20 con 0x7f9a500d0770 2024-04-25T10:12:21.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a54b69640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a500b30a0 con 0x7f9a5015db00 2024-04-25T10:12:21.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a54b69640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500b7740 con 0x7f9a500d0bd0 2024-04-25T10:12:21.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4dd9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9a500d0770 0x7f9a500b87e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46688/0 (socket says 172.21.15.133:46688) 2024-04-25T10:12:21.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4dd9b640 1 -- 172.21.15.133:0/85995206 learned_addr learned my addr 172.21.15.133:0/85995206 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:21.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049158893 0 0) 0x7f9a500ceb20 con 0x7f9a500d0770 2024-04-25T10:12:21.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a38003530 con 0x7f9a500d0770 2024-04-25T10:12:21.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4ed9d640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a500d0bd0 0x7f9a500b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2359663383 0 0) 0x7f9a38003530 con 0x7f9a500d0770 2024-04-25T10:12:21.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a500d0bd0 msgr2=0x7f9a500b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a500d0bd0 0x7f9a500b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 >> v1:172.21.15.162:6789/0 conn(0x7f9a5015db00 legacy=0x7f9a500b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500b7a20 con 0x7f9a500d0770 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f9a44001680 con 0x7f9a500d0770 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 >> v1:172.21.15.133:6789/0 conn(0x7f9a500d0770 legacy=0x7f9a500b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a500d0bd0 0x7f9a380039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a38004020 0x7f9a38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 0x7f9a380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500b7740 con 0x7f9a38004d50 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3800bb90 con 0x7f9a38004020 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3800bd90 con 0x7f9a500d0bd0 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4ed9d640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 0x7f9a380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4e59c640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a500d0bd0 0x7f9a380039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4dd9b640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a38004020 0x7f9a38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4ed9d640 1 -- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a500d0bd0 msgr2=0x7f9a380039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4ed9d640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a500d0bd0 0x7f9a380039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4ed9d640 1 -- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a38004020 msgr2=0x7f9a38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4ed9d640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a38004020 0x7f9a38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4ed9d640 1 -- 172.21.15.133:0/85995206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a38003530 con 0x7f9a38004d50 2024-04-25T10:12:21.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4dd9b640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a38004020 0x7f9a38004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:21.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4ed9d640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 0x7f9a380051d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f9a48007360 tx=0x7f9a4801ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bbff8baa1dec4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a480b2070 con 0x7f9a38004d50 2024-04-25T10:12:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a480a5d40 con 0x7f9a38004d50 2024-04-25T10:12:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.369+0000 7f9a4d59a640 1 -- 172.21.15.133:0/85995206 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a48009be0 con 0x7f9a38004d50 2024-04-25T10:12:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- 172.21.15.133:0/85995206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 msgr2=0x7f9a380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 --2- 172.21.15.133:0/85995206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 0x7f9a380051d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f9a48007360 tx=0x7f9a4801ce40 comp rx=0 tx=0).stop 2024-04-25T10:12:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4ed9d640 1 -- 172.21.15.133:0/85995206 reap_dead start 2024-04-25T10:12:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- 172.21.15.133:0/85995206 shutdown_connections 2024-04-25T10:12:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- 172.21.15.133:0/85995206 >> 172.21.15.133:0/85995206 conn(0x7f9a500727c0 msgr2=0x7f9a50072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- 172.21.15.133:0/85995206 shutdown_connections 2024-04-25T10:12:21.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- 172.21.15.133:0/85995206 wait complete. 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 Processor -- start 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- start start 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 0x7f9a500cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a5015aeb0 0x7f9a50160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a5015b330 0x7f9a501f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500b82c0 con 0x7f9a38004d50 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500b7fe0 con 0x7f9a5015aeb0 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500d1060 con 0x7f9a5015b330 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 0x7f9a500cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 0x7f9a500cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43848/0 (socket says 172.21.15.133:43848) 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4e59c640 1 -- 172.21.15.133:0/1628616695 learned_addr learned my addr 172.21.15.133:0/1628616695 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4ed9d640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a5015b330 0x7f9a501f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4dd9b640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a5015aeb0 0x7f9a50160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4e59c640 1 -- 172.21.15.133:0/1628616695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a5015b330 msgr2=0x7f9a501f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4e59c640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a5015b330 0x7f9a501f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4e59c640 1 -- 172.21.15.133:0/1628616695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a5015aeb0 msgr2=0x7f9a50160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4e59c640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a5015aeb0 0x7f9a50160580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4e59c640 1 -- 172.21.15.133:0/1628616695 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a500cc0e0 con 0x7f9a38004d50 2024-04-25T10:12:21.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a4e59c640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 0x7f9a500cd180 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000d770 tx=0x7f9a4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a3effd640 1 -- 172.21.15.133:0/1628616695 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40005270 con 0x7f9a38004d50 2024-04-25T10:12:21.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a3effd640 1 -- 172.21.15.133:0/1628616695 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a40017070 con 0x7f9a38004d50 2024-04-25T10:12:21.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- 172.21.15.133:0/1628616695 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a500cc3c0 con 0x7f9a38004d50 2024-04-25T10:12:21.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a3effd640 1 -- 172.21.15.133:0/1628616695 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000b6b0 con 0x7f9a38004d50 2024-04-25T10:12:21.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- 172.21.15.133:0/1628616695 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a500cc980 con 0x7f9a38004d50 2024-04-25T10:12:21.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.373+0000 7f9a54b69640 1 -- 172.21.15.133:0/1628616695 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a50068ef0 con 0x7f9a38004d50 2024-04-25T10:12:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.377+0000 7f9a3effd640 1 -- 172.21.15.133:0/1628616695 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a4000b850 con 0x7f9a38004d50 2024-04-25T10:12:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.377+0000 7f9a3effd640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a30066f60 0x7f9a30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.377+0000 7f9a3effd640 1 -- 172.21.15.133:0/1628616695 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9a40085030 con 0x7f9a38004d50 2024-04-25T10:12:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.377+0000 7f9a4dd9b640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a30066f60 0x7f9a30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.377+0000 7f9a4dd9b640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a30066f60 0x7f9a30069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9a500c6980 tx=0x7f9a44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.381+0000 7f9a3effd640 1 -- 172.21.15.133:0/1628616695 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a500b30a0 con 0x7f9a38004d50 2024-04-25T10:12:21.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.525+0000 7f9a54b69640 1 -- 172.21.15.133:0/1628616695 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a500b2750 con 0x7f9a30066f60 2024-04-25T10:12:21.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.541+0000 7f9a3effd640 1 -- 172.21.15.133:0/1628616695 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252483 (secure 0 0 0) 0x7f9a500b2750 con 0x7f9a30066f60 2024-04-25T10:12:21.548 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:21.549 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:12:21.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 -- 172.21.15.133:0/1628616695 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a30066f60 msgr2=0x7f9a30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a30066f60 0x7f9a30069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9a500c6980 tx=0x7f9a44009040 comp rx=0 tx=0).stop 2024-04-25T10:12:21.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 -- 172.21.15.133:0/1628616695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 msgr2=0x7f9a500cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 0x7f9a500cd180 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000d770 tx=0x7f9a4000dc30 comp rx=0 tx=0).stop 2024-04-25T10:12:21.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 -- 172.21.15.133:0/1628616695 shutdown_connections 2024-04-25T10:12:21.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9a5015b330 0x7f9a501f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a5015aeb0 0x7f9a50160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9a30066f60 0x7f9a30069420 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 --2- 172.21.15.133:0/1628616695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a38004d50 0x7f9a500cd180 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 -- 172.21.15.133:0/1628616695 >> 172.21.15.133:0/1628616695 conn(0x7f9a500727c0 msgr2=0x7f9a500d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:21.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 -- 172.21.15.133:0/1628616695 shutdown_connections 2024-04-25T10:12:21.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.545+0000 7f9a54b69640 1 -- 172.21.15.133:0/1628616695 wait complete. 2024-04-25T10:12:21.564 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-04-25T10:12:19.897150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":491332,"kb_used_data":12820,"kb_used_omap":12,"kb_used_meta":478387,"kb_avail":754483388,"statfs":{"total":773094113280,"available":772590989312,"internally_reserved":0,"allocated":13127680,"data_stored":6725640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":489868849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000962"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703776+0000","last_change":"2024-04-25T10:12:18.640529+0000","last_active":"2024-04-25T10:12:19.703776+0000","last_peered":"2024-04-25T10:12:19.703776+0000","last_clean":"2024-04-25T10:12:19.703776+0000","last_became_active":"2024-04-25T10:12:18.624696+0000","last_became_peered":"2024-04-25T10:12:18.624696+0000","last_unstale":"2024-04-25T10:12:19.703776+0000","last_undegraded":"2024-04-25T10:12:19.703776+0000","last_fullsized":"2024-04-25T10:12:19.703776+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:56.597024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701567+0000","last_change":"2024-04-25T10:12:18.615785+0000","last_active":"2024-04-25T10:12:19.701567+0000","last_peered":"2024-04-25T10:12:19.701567+0000","last_clean":"2024-04-25T10:12:19.701567+0000","last_became_active":"2024-04-25T10:12:18.615161+0000","last_became_peered":"2024-04-25T10:12:18.615161+0000","last_unstale":"2024-04-25T10:12:19.701567+0000","last_undegraded":"2024-04-25T10:12:19.701567+0000","last_fullsized":"2024-04-25T10:12:19.701567+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:55.697502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701422+0000","last_change":"2024-04-25T10:12:18.616412+0000","last_active":"2024-04-25T10:12:19.701422+0000","last_peered":"2024-04-25T10:12:19.701422+0000","last_clean":"2024-04-25T10:12:19.701422+0000","last_became_active":"2024-04-25T10:12:18.614475+0000","last_became_peered":"2024-04-25T10:12:18.614475+0000","last_unstale":"2024-04-25T10:12:19.701422+0000","last_undegraded":"2024-04-25T10:12:19.701422+0000","last_fullsized":"2024-04-25T10:12:19.701422+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:59.962187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703509+0000","last_change":"2024-04-25T10:12:18.642820+0000","last_active":"2024-04-25T10:12:19.703509+0000","last_peered":"2024-04-25T10:12:19.703509+0000","last_clean":"2024-04-25T10:12:19.703509+0000","last_became_active":"2024-04-25T10:12:18.642288+0000","last_became_peered":"2024-04-25T10:12:18.642288+0000","last_unstale":"2024-04-25T10:12:19.703509+0000","last_undegraded":"2024-04-25T10:12:19.703509+0000","last_fullsized":"2024-04-25T10:12:19.703509+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:13.070946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701448+0000","last_change":"2024-04-25T10:12:18.638296+0000","last_active":"2024-04-25T10:12:19.701448+0000","last_peered":"2024-04-25T10:12:19.701448+0000","last_clean":"2024-04-25T10:12:19.701448+0000","last_became_active":"2024-04-25T10:12:18.637508+0000","last_became_peered":"2024-04-25T10:12:18.637508+0000","last_unstale":"2024-04-25T10:12:19.701448+0000","last_undegraded":"2024-04-25T10:12:19.701448+0000","last_fullsized":"2024-04-25T10:12:19.701448+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:43.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617724+0000","last_change":"2024-04-25T10:12:18.633248+0000","last_active":"2024-04-25T10:12:19.617724+0000","last_peered":"2024-04-25T10:12:19.617724+0000","last_clean":"2024-04-25T10:12:19.617724+0000","last_became_active":"2024-04-25T10:12:18.632227+0000","last_became_peered":"2024-04-25T10:12:18.632227+0000","last_unstale":"2024-04-25T10:12:19.617724+0000","last_undegraded":"2024-04-25T10:12:19.617724+0000","last_fullsized":"2024-04-25T10:12:19.617724+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:17.862576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617387+0000","last_change":"2024-04-25T10:12:18.632723+0000","last_active":"2024-04-25T10:12:19.617387+0000","last_peered":"2024-04-25T10:12:19.617387+0000","last_clean":"2024-04-25T10:12:19.617387+0000","last_became_active":"2024-04-25T10:12:18.632428+0000","last_became_peered":"2024-04-25T10:12:18.632428+0000","last_unstale":"2024-04-25T10:12:19.617387+0000","last_undegraded":"2024-04-25T10:12:19.617387+0000","last_fullsized":"2024-04-25T10:12:19.617387+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:53.239093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.699954+0000","last_change":"2024-04-25T10:12:18.619723+0000","last_active":"2024-04-25T10:12:19.699954+0000","last_peered":"2024-04-25T10:12:19.699954+0000","last_clean":"2024-04-25T10:12:19.699954+0000","last_became_active":"2024-04-25T10:12:18.613711+0000","last_became_peered":"2024-04-25T10:12:18.613711+0000","last_unstale":"2024-04-25T10:12:19.699954+0000","last_undegraded":"2024-04-25T10:12:19.699954+0000","last_fullsized":"2024-04-25T10:12:19.699954+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:37.893622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703528+0000","last_change":"2024-04-25T10:12:18.632189+0000","last_active":"2024-04-25T10:12:19.703528+0000","last_peered":"2024-04-25T10:12:19.703528+0000","last_clean":"2024-04-25T10:12:19.703528+0000","last_became_active":"2024-04-25T10:12:18.631412+0000","last_became_peered":"2024-04-25T10:12:18.631412+0000","last_unstale":"2024-04-25T10:12:19.703528+0000","last_undegraded":"2024-04-25T10:12:19.703528+0000","last_fullsized":"2024-04-25T10:12:19.703528+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:26.110237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599588+0000","last_change":"2024-04-25T10:12:18.617400+0000","last_active":"2024-04-25T10:12:19.599588+0000","last_peered":"2024-04-25T10:12:19.599588+0000","last_clean":"2024-04-25T10:12:19.599588+0000","last_became_active":"2024-04-25T10:12:18.617064+0000","last_became_peered":"2024-04-25T10:12:18.617064+0000","last_unstale":"2024-04-25T10:12:19.599588+0000","last_undegraded":"2024-04-25T10:12:19.599588+0000","last_fullsized":"2024-04-25T10:12:19.599588+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:14.608188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702909+0000","last_change":"2024-04-25T10:12:18.652345+0000","last_active":"2024-04-25T10:12:19.702909+0000","last_peered":"2024-04-25T10:12:19.702909+0000","last_clean":"2024-04-25T10:12:19.702909+0000","last_became_active":"2024-04-25T10:12:18.651859+0000","last_became_peered":"2024-04-25T10:12:18.651859+0000","last_unstale":"2024-04-25T10:12:19.702909+0000","last_undegraded":"2024-04-25T10:12:19.702909+0000","last_fullsized":"2024-04-25T10:12:19.702909+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:00.777963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617767+0000","last_change":"2024-04-25T10:12:18.644659+0000","last_active":"2024-04-25T10:12:19.617767+0000","last_peered":"2024-04-25T10:12:19.617767+0000","last_clean":"2024-04-25T10:12:19.617767+0000","last_became_active":"2024-04-25T10:12:18.644341+0000","last_became_peered":"2024-04-25T10:12:18.644341+0000","last_unstale":"2024-04-25T10:12:19.617767+0000","last_undegraded":"2024-04-25T10:12:19.617767+0000","last_fullsized":"2024-04-25T10:12:19.617767+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:50.342384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599772+0000","last_change":"2024-04-25T10:12:18.657079+0000","last_active":"2024-04-25T10:12:19.599772+0000","last_peered":"2024-04-25T10:12:19.599772+0000","last_clean":"2024-04-25T10:12:19.599772+0000","last_became_active":"2024-04-25T10:12:18.656801+0000","last_became_peered":"2024-04-25T10:12:18.656801+0000","last_unstale":"2024-04-25T10:12:19.599772+0000","last_undegraded":"2024-04-25T10:12:19.599772+0000","last_fullsized":"2024-04-25T10:12:19.599772+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:07.120146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701607+0000","last_change":"2024-04-25T10:12:18.624368+0000","last_active":"2024-04-25T10:12:19.701607+0000","last_peered":"2024-04-25T10:12:19.701607+0000","last_clean":"2024-04-25T10:12:19.701607+0000","last_became_active":"2024-04-25T10:12:18.623999+0000","last_became_peered":"2024-04-25T10:12:18.623999+0000","last_unstale":"2024-04-25T10:12:19.701607+0000","last_undegraded":"2024-04-25T10:12:19.701607+0000","last_fullsized":"2024-04-25T10:12:19.701607+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:00.741659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701865+0000","last_change":"2024-04-25T10:12:18.653210+0000","last_active":"2024-04-25T10:12:19.701865+0000","last_peered":"2024-04-25T10:12:19.701865+0000","last_clean":"2024-04-25T10:12:19.701865+0000","last_became_active":"2024-04-25T10:12:18.652493+0000","last_became_peered":"2024-04-25T10:12:18.652493+0000","last_unstale":"2024-04-25T10:12:19.701865+0000","last_undegraded":"2024-04-25T10:12:19.701865+0000","last_fullsized":"2024-04-25T10:12:19.701865+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:32.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617332+0000","last_change":"2024-04-25T10:12:18.625906+0000","last_active":"2024-04-25T10:12:19.617332+0000","last_peered":"2024-04-25T10:12:19.617332+0000","last_clean":"2024-04-25T10:12:19.617332+0000","last_became_active":"2024-04-25T10:12:18.622387+0000","last_became_peered":"2024-04-25T10:12:18.622387+0000","last_unstale":"2024-04-25T10:12:19.617332+0000","last_undegraded":"2024-04-25T10:12:19.617332+0000","last_fullsized":"2024-04-25T10:12:19.617332+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.200844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599876+0000","last_change":"2024-04-25T10:12:18.658941+0000","last_active":"2024-04-25T10:12:19.599876+0000","last_peered":"2024-04-25T10:12:19.599876+0000","last_clean":"2024-04-25T10:12:19.599876+0000","last_became_active":"2024-04-25T10:12:18.658603+0000","last_became_peered":"2024-04-25T10:12:18.658603+0000","last_unstale":"2024-04-25T10:12:19.599876+0000","last_undegraded":"2024-04-25T10:12:19.599876+0000","last_fullsized":"2024-04-25T10:12:19.599876+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:20.607447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701953+0000","last_change":"2024-04-25T10:12:18.615498+0000","last_active":"2024-04-25T10:12:19.701953+0000","last_peered":"2024-04-25T10:12:19.701953+0000","last_clean":"2024-04-25T10:12:19.701953+0000","last_became_active":"2024-04-25T10:12:18.614717+0000","last_became_peered":"2024-04-25T10:12:18.614717+0000","last_unstale":"2024-04-25T10:12:19.701953+0000","last_undegraded":"2024-04-25T10:12:19.701953+0000","last_fullsized":"2024-04-25T10:12:19.701953+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:39.250378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605804+0000","last_change":"2024-04-25T10:12:18.611963+0000","last_active":"2024-04-25T10:12:19.605804+0000","last_peered":"2024-04-25T10:12:19.605804+0000","last_clean":"2024-04-25T10:12:19.605804+0000","last_became_active":"2024-04-25T10:12:18.610934+0000","last_became_peered":"2024-04-25T10:12:18.610934+0000","last_unstale":"2024-04-25T10:12:19.605804+0000","last_undegraded":"2024-04-25T10:12:19.605804+0000","last_fullsized":"2024-04-25T10:12:19.605804+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:34.559450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700772+0000","last_change":"2024-04-25T10:12:18.617560+0000","last_active":"2024-04-25T10:12:19.700772+0000","last_peered":"2024-04-25T10:12:19.700772+0000","last_clean":"2024-04-25T10:12:19.700772+0000","last_became_active":"2024-04-25T10:12:18.617212+0000","last_became_peered":"2024-04-25T10:12:18.617212+0000","last_unstale":"2024-04-25T10:12:19.700772+0000","last_undegraded":"2024-04-25T10:12:19.700772+0000","last_fullsized":"2024-04-25T10:12:19.700772+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:12.825278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702924+0000","last_change":"2024-04-25T10:12:18.611176+0000","last_active":"2024-04-25T10:12:19.702924+0000","last_peered":"2024-04-25T10:12:19.702924+0000","last_clean":"2024-04-25T10:12:19.702924+0000","last_became_active":"2024-04-25T10:12:18.610455+0000","last_became_peered":"2024-04-25T10:12:18.610455+0000","last_unstale":"2024-04-25T10:12:19.702924+0000","last_undegraded":"2024-04-25T10:12:19.702924+0000","last_fullsized":"2024-04-25T10:12:19.702924+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:03.751919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702286+0000","last_change":"2024-04-25T10:12:18.642193+0000","last_active":"2024-04-25T10:12:19.702286+0000","last_peered":"2024-04-25T10:12:19.702286+0000","last_clean":"2024-04-25T10:12:19.702286+0000","last_became_active":"2024-04-25T10:12:18.641645+0000","last_became_peered":"2024-04-25T10:12:18.641645+0000","last_unstale":"2024-04-25T10:12:19.702286+0000","last_undegraded":"2024-04-25T10:12:19.702286+0000","last_fullsized":"2024-04-25T10:12:19.702286+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:15.188327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702119+0000","last_change":"2024-04-25T10:12:18.628091+0000","last_active":"2024-04-25T10:12:19.702119+0000","last_peered":"2024-04-25T10:12:19.702119+0000","last_clean":"2024-04-25T10:12:19.702119+0000","last_became_active":"2024-04-25T10:12:18.627202+0000","last_became_peered":"2024-04-25T10:12:18.627202+0000","last_unstale":"2024-04-25T10:12:19.702119+0000","last_undegraded":"2024-04-25T10:12:19.702119+0000","last_fullsized":"2024-04-25T10:12:19.702119+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:59.182914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.604935+0000","last_change":"2024-04-25T10:12:18.650350+0000","last_active":"2024-04-25T10:12:19.604935+0000","last_peered":"2024-04-25T10:12:19.604935+0000","last_clean":"2024-04-25T10:12:19.604935+0000","last_became_active":"2024-04-25T10:12:18.649666+0000","last_became_peered":"2024-04-25T10:12:18.649666+0000","last_unstale":"2024-04-25T10:12:19.604935+0000","last_undegraded":"2024-04-25T10:12:19.604935+0000","last_fullsized":"2024-04-25T10:12:19.604935+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:56.303725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.607374+0000","last_change":"2024-04-25T10:12:18.657681+0000","last_active":"2024-04-25T10:12:19.607374+0000","last_peered":"2024-04-25T10:12:19.607374+0000","last_clean":"2024-04-25T10:12:19.607374+0000","last_became_active":"2024-04-25T10:12:18.654829+0000","last_became_peered":"2024-04-25T10:12:18.654829+0000","last_unstale":"2024-04-25T10:12:19.607374+0000","last_undegraded":"2024-04-25T10:12:19.607374+0000","last_fullsized":"2024-04-25T10:12:19.607374+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:24.095664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700795+0000","last_change":"2024-04-25T10:12:18.610837+0000","last_active":"2024-04-25T10:12:19.700795+0000","last_peered":"2024-04-25T10:12:19.700795+0000","last_clean":"2024-04-25T10:12:19.700795+0000","last_became_active":"2024-04-25T10:12:18.610479+0000","last_became_peered":"2024-04-25T10:12:18.610479+0000","last_unstale":"2024-04-25T10:12:19.700795+0000","last_undegraded":"2024-04-25T10:12:19.700795+0000","last_fullsized":"2024-04-25T10:12:19.700795+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:15.393054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606197+0000","last_change":"2024-04-25T10:12:18.599480+0000","last_active":"2024-04-25T10:12:19.606197+0000","last_peered":"2024-04-25T10:12:19.606197+0000","last_clean":"2024-04-25T10:12:19.606197+0000","last_became_active":"2024-04-25T10:12:18.598701+0000","last_became_peered":"2024-04-25T10:12:18.598701+0000","last_unstale":"2024-04-25T10:12:19.606197+0000","last_undegraded":"2024-04-25T10:12:19.606197+0000","last_fullsized":"2024-04-25T10:12:19.606197+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:54.194570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617758+0000","last_change":"2024-04-25T10:12:18.650490+0000","last_active":"2024-04-25T10:12:19.617758+0000","last_peered":"2024-04-25T10:12:19.617758+0000","last_clean":"2024-04-25T10:12:19.617758+0000","last_became_active":"2024-04-25T10:12:18.650194+0000","last_became_peered":"2024-04-25T10:12:18.650194+0000","last_unstale":"2024-04-25T10:12:19.617758+0000","last_undegraded":"2024-04-25T10:12:19.617758+0000","last_fullsized":"2024-04-25T10:12:19.617758+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:46.630033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700483+0000","last_change":"2024-04-25T10:12:18.614318+0000","last_active":"2024-04-25T10:12:19.700483+0000","last_peered":"2024-04-25T10:12:19.700483+0000","last_clean":"2024-04-25T10:12:19.700483+0000","last_became_active":"2024-04-25T10:12:18.613947+0000","last_became_peered":"2024-04-25T10:12:18.613947+0000","last_unstale":"2024-04-25T10:12:19.700483+0000","last_undegraded":"2024-04-25T10:12:19.700483+0000","last_fullsized":"2024-04-25T10:12:19.700483+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:42.848005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700894+0000","last_change":"2024-04-25T10:12:18.642558+0000","last_active":"2024-04-25T10:12:19.700894+0000","last_peered":"2024-04-25T10:12:19.700894+0000","last_clean":"2024-04-25T10:12:19.700894+0000","last_became_active":"2024-04-25T10:12:18.642284+0000","last_became_peered":"2024-04-25T10:12:18.642284+0000","last_unstale":"2024-04-25T10:12:19.700894+0000","last_undegraded":"2024-04-25T10:12:19.700894+0000","last_fullsized":"2024-04-25T10:12:19.700894+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:17.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606256+0000","last_change":"2024-04-25T10:12:18.650875+0000","last_active":"2024-04-25T10:12:19.606256+0000","last_peered":"2024-04-25T10:12:19.606256+0000","last_clean":"2024-04-25T10:12:19.606256+0000","last_became_active":"2024-04-25T10:12:18.650161+0000","last_became_peered":"2024-04-25T10:12:18.650161+0000","last_unstale":"2024-04-25T10:12:19.606256+0000","last_undegraded":"2024-04-25T10:12:19.606256+0000","last_fullsized":"2024-04-25T10:12:19.606256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:42.950870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605342+0000","last_change":"2024-04-25T10:12:18.649608+0000","last_active":"2024-04-25T10:12:19.605342+0000","last_peered":"2024-04-25T10:12:19.605342+0000","last_clean":"2024-04-25T10:12:19.605342+0000","last_became_active":"2024-04-25T10:12:18.647174+0000","last_became_peered":"2024-04-25T10:12:18.647174+0000","last_unstale":"2024-04-25T10:12:19.605342+0000","last_undegraded":"2024-04-25T10:12:19.605342+0000","last_fullsized":"2024-04-25T10:12:19.605342+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:07.120146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599226+0000","last_change":"2024-04-25T10:12:18.627941+0000","last_active":"2024-04-25T10:12:19.599226+0000","last_peered":"2024-04-25T10:12:19.599226+0000","last_clean":"2024-04-25T10:12:19.599226+0000","last_became_active":"2024-04-25T10:12:18.627628+0000","last_became_peered":"2024-04-25T10:12:18.627628+0000","last_unstale":"2024-04-25T10:12:19.599226+0000","last_undegraded":"2024-04-25T10:12:19.599226+0000","last_fullsized":"2024-04-25T10:12:19.599226+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:00.777963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"119'4","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605624+0000","last_change":"2024-04-25T10:12:17.579133+0000","last_active":"2024-04-25T10:12:19.605624+0000","last_peered":"2024-04-25T10:12:19.605624+0000","last_clean":"2024-04-25T10:12:19.605624+0000","last_became_active":"2024-04-25T10:12:17.578600+0000","last_became_peered":"2024-04-25T10:12:17.578600+0000","last_unstale":"2024-04-25T10:12:19.605624+0000","last_undegraded":"2024-04-25T10:12:19.605624+0000","last_fullsized":"2024-04-25T10:12:19.605624+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:54.890814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617834+0000","last_change":"2024-04-25T10:12:18.632595+0000","last_active":"2024-04-25T10:12:19.617834+0000","last_peered":"2024-04-25T10:12:19.617834+0000","last_clean":"2024-04-25T10:12:19.617834+0000","last_became_active":"2024-04-25T10:12:18.632290+0000","last_became_peered":"2024-04-25T10:12:18.632290+0000","last_unstale":"2024-04-25T10:12:19.617834+0000","last_undegraded":"2024-04-25T10:12:19.617834+0000","last_fullsized":"2024-04-25T10:12:19.617834+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:59.505045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599414+0000","last_change":"2024-04-25T10:12:17.576602+0000","last_active":"2024-04-25T10:12:19.599414+0000","last_peered":"2024-04-25T10:12:19.599414+0000","last_clean":"2024-04-25T10:12:19.599414+0000","last_became_active":"2024-04-25T10:12:17.574129+0000","last_became_peered":"2024-04-25T10:12:17.574129+0000","last_unstale":"2024-04-25T10:12:19.599414+0000","last_undegraded":"2024-04-25T10:12:19.599414+0000","last_fullsized":"2024-04-25T10:12:19.599414+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:18.518046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606869+0000","last_change":"2024-04-25T10:12:18.649335+0000","last_active":"2024-04-25T10:12:19.606869+0000","last_peered":"2024-04-25T10:12:19.606869+0000","last_clean":"2024-04-25T10:12:19.606869+0000","last_became_active":"2024-04-25T10:12:18.647062+0000","last_became_peered":"2024-04-25T10:12:18.647062+0000","last_unstale":"2024-04-25T10:12:19.606869+0000","last_undegraded":"2024-04-25T10:12:19.606869+0000","last_fullsized":"2024-04-25T10:12:19.606869+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:17.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702852+0000","last_change":"2024-04-25T10:12:17.581450+0000","last_active":"2024-04-25T10:12:19.702852+0000","last_peered":"2024-04-25T10:12:19.702852+0000","last_clean":"2024-04-25T10:12:19.702852+0000","last_became_active":"2024-04-25T10:12:17.580879+0000","last_became_peered":"2024-04-25T10:12:17.580879+0000","last_unstale":"2024-04-25T10:12:19.702852+0000","last_undegraded":"2024-04-25T10:12:19.702852+0000","last_fullsized":"2024-04-25T10:12:19.702852+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:59.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703883+0000","last_change":"2024-04-25T10:12:18.633464+0000","last_active":"2024-04-25T10:12:19.703883+0000","last_peered":"2024-04-25T10:12:19.703883+0000","last_clean":"2024-04-25T10:12:19.703883+0000","last_became_active":"2024-04-25T10:12:18.632206+0000","last_became_peered":"2024-04-25T10:12:18.632206+0000","last_unstale":"2024-04-25T10:12:19.703883+0000","last_undegraded":"2024-04-25T10:12:19.703883+0000","last_fullsized":"2024-04-25T10:12:19.703883+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:01.399821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599267+0000","last_change":"2024-04-25T10:12:17.560626+0000","last_active":"2024-04-25T10:12:19.599267+0000","last_peered":"2024-04-25T10:12:19.599267+0000","last_clean":"2024-04-25T10:12:19.599267+0000","last_became_active":"2024-04-25T10:12:17.560209+0000","last_became_peered":"2024-04-25T10:12:17.560209+0000","last_unstale":"2024-04-25T10:12:19.599267+0000","last_undegraded":"2024-04-25T10:12:19.599267+0000","last_fullsized":"2024-04-25T10:12:19.599267+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:43.268617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599050+0000","last_change":"2024-04-25T10:12:18.610961+0000","last_active":"2024-04-25T10:12:19.599050+0000","last_peered":"2024-04-25T10:12:19.599050+0000","last_clean":"2024-04-25T10:12:19.599050+0000","last_became_active":"2024-04-25T10:12:18.609985+0000","last_became_peered":"2024-04-25T10:12:18.609985+0000","last_unstale":"2024-04-25T10:12:19.599050+0000","last_undegraded":"2024-04-25T10:12:19.599050+0000","last_fullsized":"2024-04-25T10:12:19.599050+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:32.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701764+0000","last_change":"2024-04-25T10:12:17.567188+0000","last_active":"2024-04-25T10:12:19.701764+0000","last_peered":"2024-04-25T10:12:19.701764+0000","last_clean":"2024-04-25T10:12:19.701764+0000","last_became_active":"2024-04-25T10:12:17.566439+0000","last_became_peered":"2024-04-25T10:12:17.566439+0000","last_unstale":"2024-04-25T10:12:19.701764+0000","last_undegraded":"2024-04-25T10:12:19.701764+0000","last_fullsized":"2024-04-25T10:12:19.701764+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:05.326032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701403+0000","last_change":"2024-04-25T10:12:18.636742+0000","last_active":"2024-04-25T10:12:19.701403+0000","last_peered":"2024-04-25T10:12:19.701403+0000","last_clean":"2024-04-25T10:12:19.701403+0000","last_became_active":"2024-04-25T10:12:18.624588+0000","last_became_peered":"2024-04-25T10:12:18.624588+0000","last_unstale":"2024-04-25T10:12:19.701403+0000","last_undegraded":"2024-04-25T10:12:19.701403+0000","last_fullsized":"2024-04-25T10:12:19.701403+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:22.109430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617557+0000","last_change":"2024-04-25T10:12:17.576005+0000","last_active":"2024-04-25T10:12:19.617557+0000","last_peered":"2024-04-25T10:12:19.617557+0000","last_clean":"2024-04-25T10:12:19.617557+0000","last_became_active":"2024-04-25T10:12:17.575599+0000","last_became_peered":"2024-04-25T10:12:17.575599+0000","last_unstale":"2024-04-25T10:12:19.617557+0000","last_undegraded":"2024-04-25T10:12:19.617557+0000","last_fullsized":"2024-04-25T10:12:19.617557+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:44.231969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701343+0000","last_change":"2024-04-25T10:12:18.623793+0000","last_active":"2024-04-25T10:12:19.701343+0000","last_peered":"2024-04-25T10:12:19.701343+0000","last_clean":"2024-04-25T10:12:19.701343+0000","last_became_active":"2024-04-25T10:12:18.623136+0000","last_became_peered":"2024-04-25T10:12:18.623136+0000","last_unstale":"2024-04-25T10:12:19.701343+0000","last_undegraded":"2024-04-25T10:12:19.701343+0000","last_fullsized":"2024-04-25T10:12:19.701343+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:24.095664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606334+0000","last_change":"2024-04-25T10:12:17.571160+0000","last_active":"2024-04-25T10:12:19.606334+0000","last_peered":"2024-04-25T10:12:19.606334+0000","last_clean":"2024-04-25T10:12:19.606334+0000","last_became_active":"2024-04-25T10:12:17.570821+0000","last_became_peered":"2024-04-25T10:12:17.570821+0000","last_unstale":"2024-04-25T10:12:19.606334+0000","last_undegraded":"2024-04-25T10:12:19.606334+0000","last_fullsized":"2024-04-25T10:12:19.606334+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:33.548486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599275+0000","last_change":"2024-04-25T10:12:18.614695+0000","last_active":"2024-04-25T10:12:19.599275+0000","last_peered":"2024-04-25T10:12:19.599275+0000","last_clean":"2024-04-25T10:12:19.599275+0000","last_became_active":"2024-04-25T10:12:18.614372+0000","last_became_peered":"2024-04-25T10:12:18.614372+0000","last_unstale":"2024-04-25T10:12:19.599275+0000","last_undegraded":"2024-04-25T10:12:19.599275+0000","last_fullsized":"2024-04-25T10:12:19.599275+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.909841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617538+0000","last_change":"2024-04-25T10:12:17.581954+0000","last_active":"2024-04-25T10:12:19.617538+0000","last_peered":"2024-04-25T10:12:19.617538+0000","last_clean":"2024-04-25T10:12:19.617538+0000","last_became_active":"2024-04-25T10:12:17.581569+0000","last_became_peered":"2024-04-25T10:12:17.581569+0000","last_unstale":"2024-04-25T10:12:19.617538+0000","last_undegraded":"2024-04-25T10:12:19.617538+0000","last_fullsized":"2024-04-25T10:12:19.617538+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:54.435602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.604873+0000","last_change":"2024-04-25T10:12:18.604234+0000","last_active":"2024-04-25T10:12:19.604873+0000","last_peered":"2024-04-25T10:12:19.604873+0000","last_clean":"2024-04-25T10:12:19.604873+0000","last_became_active":"2024-04-25T10:12:18.603464+0000","last_became_peered":"2024-04-25T10:12:18.603464+0000","last_unstale":"2024-04-25T10:12:19.604873+0000","last_undegraded":"2024-04-25T10:12:19.604873+0000","last_fullsized":"2024-04-25T10:12:19.604873+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:22.013337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605901+0000","last_change":"2024-04-25T10:12:17.583252+0000","last_active":"2024-04-25T10:12:19.605901+0000","last_peered":"2024-04-25T10:12:19.605901+0000","last_clean":"2024-04-25T10:12:19.605901+0000","last_became_active":"2024-04-25T10:12:17.582630+0000","last_became_peered":"2024-04-25T10:12:17.582630+0000","last_unstale":"2024-04-25T10:12:19.605901+0000","last_undegraded":"2024-04-25T10:12:19.605901+0000","last_fullsized":"2024-04-25T10:12:19.605901+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:08.124899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606834+0000","last_change":"2024-04-25T10:12:18.653889+0000","last_active":"2024-04-25T10:12:19.606834+0000","last_peered":"2024-04-25T10:12:19.606834+0000","last_clean":"2024-04-25T10:12:19.606834+0000","last_became_active":"2024-04-25T10:12:18.653303+0000","last_became_peered":"2024-04-25T10:12:18.653303+0000","last_unstale":"2024-04-25T10:12:19.606834+0000","last_undegraded":"2024-04-25T10:12:19.606834+0000","last_fullsized":"2024-04-25T10:12:19.606834+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:21.891519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606776+0000","last_change":"2024-04-25T10:12:17.575106+0000","last_active":"2024-04-25T10:12:19.606776+0000","last_peered":"2024-04-25T10:12:19.606776+0000","last_clean":"2024-04-25T10:12:19.606776+0000","last_became_active":"2024-04-25T10:12:17.574190+0000","last_became_peered":"2024-04-25T10:12:17.574190+0000","last_unstale":"2024-04-25T10:12:19.606776+0000","last_undegraded":"2024-04-25T10:12:19.606776+0000","last_fullsized":"2024-04-25T10:12:19.606776+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.706095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702089+0000","last_change":"2024-04-25T10:12:18.613630+0000","last_active":"2024-04-25T10:12:19.702089+0000","last_peered":"2024-04-25T10:12:19.702089+0000","last_clean":"2024-04-25T10:12:19.702089+0000","last_became_active":"2024-04-25T10:12:18.613310+0000","last_became_peered":"2024-04-25T10:12:18.613310+0000","last_unstale":"2024-04-25T10:12:19.702089+0000","last_undegraded":"2024-04-25T10:12:19.702089+0000","last_fullsized":"2024-04-25T10:12:19.702089+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.538210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"119'3","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606883+0000","last_change":"2024-04-25T10:12:17.583737+0000","last_active":"2024-04-25T10:12:19.606883+0000","last_peered":"2024-04-25T10:12:19.606883+0000","last_clean":"2024-04-25T10:12:19.606883+0000","last_became_active":"2024-04-25T10:12:17.583046+0000","last_became_peered":"2024-04-25T10:12:17.583046+0000","last_unstale":"2024-04-25T10:12:19.606883+0000","last_undegraded":"2024-04-25T10:12:19.606883+0000","last_fullsized":"2024-04-25T10:12:19.606883+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:21.002373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.607461+0000","last_change":"2024-04-25T10:12:18.657615+0000","last_active":"2024-04-25T10:12:19.607461+0000","last_peered":"2024-04-25T10:12:19.607461+0000","last_clean":"2024-04-25T10:12:19.607461+0000","last_became_active":"2024-04-25T10:12:18.654737+0000","last_became_peered":"2024-04-25T10:12:18.654737+0000","last_unstale":"2024-04-25T10:12:19.607461+0000","last_undegraded":"2024-04-25T10:12:19.607461+0000","last_fullsized":"2024-04-25T10:12:19.607461+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:59.962187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"119'6","reported_seq":20,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702215+0000","last_change":"2024-04-25T10:12:17.572523+0000","last_active":"2024-04-25T10:12:19.702215+0000","last_peered":"2024-04-25T10:12:19.702215+0000","last_clean":"2024-04-25T10:12:19.702215+0000","last_became_active":"2024-04-25T10:12:17.571923+0000","last_became_peered":"2024-04-25T10:12:17.571923+0000","last_unstale":"2024-04-25T10:12:19.702215+0000","last_undegraded":"2024-04-25T10:12:19.702215+0000","last_fullsized":"2024-04-25T10:12:19.702215+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:19.345544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605299+0000","last_change":"2024-04-25T10:12:18.617255+0000","last_active":"2024-04-25T10:12:19.605299+0000","last_peered":"2024-04-25T10:12:19.605299+0000","last_clean":"2024-04-25T10:12:19.605299+0000","last_became_active":"2024-04-25T10:12:18.616411+0000","last_became_peered":"2024-04-25T10:12:18.616411+0000","last_unstale":"2024-04-25T10:12:19.605299+0000","last_undegraded":"2024-04-25T10:12:19.605299+0000","last_fullsized":"2024-04-25T10:12:19.605299+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:25.717059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701406+0000","last_change":"2024-04-25T10:12:17.578832+0000","last_active":"2024-04-25T10:12:19.701406+0000","last_peered":"2024-04-25T10:12:19.701406+0000","last_clean":"2024-04-25T10:12:19.701406+0000","last_became_active":"2024-04-25T10:12:17.578516+0000","last_became_peered":"2024-04-25T10:12:17.578516+0000","last_unstale":"2024-04-25T10:12:19.701406+0000","last_undegraded":"2024-04-25T10:12:19.701406+0000","last_fullsized":"2024-04-25T10:12:19.701406+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:53.183606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599052+0000","last_change":"2024-04-25T10:12:18.610534+0000","last_active":"2024-04-25T10:12:19.599052+0000","last_peered":"2024-04-25T10:12:19.599052+0000","last_clean":"2024-04-25T10:12:19.599052+0000","last_became_active":"2024-04-25T10:12:18.609616+0000","last_became_peered":"2024-04-25T10:12:18.609616+0000","last_unstale":"2024-04-25T10:12:19.599052+0000","last_undegraded":"2024-04-25T10:12:19.599052+0000","last_fullsized":"2024-04-25T10:12:19.599052+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:15.188327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701833+0000","last_change":"2024-04-25T10:12:17.573191+0000","last_active":"2024-04-25T10:12:19.701833+0000","last_peered":"2024-04-25T10:12:19.701833+0000","last_clean":"2024-04-25T10:12:19.701833+0000","last_became_active":"2024-04-25T10:12:17.572696+0000","last_became_peered":"2024-04-25T10:12:17.572696+0000","last_unstale":"2024-04-25T10:12:19.701833+0000","last_undegraded":"2024-04-25T10:12:19.701833+0000","last_fullsized":"2024-04-25T10:12:19.701833+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.706095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700416+0000","last_change":"2024-04-25T10:12:18.642739+0000","last_active":"2024-04-25T10:12:19.700416+0000","last_peered":"2024-04-25T10:12:19.700416+0000","last_clean":"2024-04-25T10:12:19.700416+0000","last_became_active":"2024-04-25T10:12:18.642415+0000","last_became_peered":"2024-04-25T10:12:18.642415+0000","last_unstale":"2024-04-25T10:12:19.700416+0000","last_undegraded":"2024-04-25T10:12:19.700416+0000","last_fullsized":"2024-04-25T10:12:19.700416+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:00.736459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700417+0000","last_change":"2024-04-25T10:12:17.571971+0000","last_active":"2024-04-25T10:12:19.700417+0000","last_peered":"2024-04-25T10:12:19.700417+0000","last_clean":"2024-04-25T10:12:19.700417+0000","last_became_active":"2024-04-25T10:12:17.571593+0000","last_became_peered":"2024-04-25T10:12:17.571593+0000","last_unstale":"2024-04-25T10:12:19.700417+0000","last_undegraded":"2024-04-25T10:12:19.700417+0000","last_fullsized":"2024-04-25T10:12:19.700417+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:21.002373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605992+0000","last_change":"2024-04-25T10:12:18.649874+0000","last_active":"2024-04-25T10:12:19.605992+0000","last_peered":"2024-04-25T10:12:19.605992+0000","last_clean":"2024-04-25T10:12:19.605992+0000","last_became_active":"2024-04-25T10:12:18.648045+0000","last_became_peered":"2024-04-25T10:12:18.648045+0000","last_unstale":"2024-04-25T10:12:19.605992+0000","last_undegraded":"2024-04-25T10:12:19.605992+0000","last_fullsized":"2024-04-25T10:12:19.605992+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:37.893622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599962+0000","last_change":"2024-04-25T10:12:17.577393+0000","last_active":"2024-04-25T10:12:19.599962+0000","last_peered":"2024-04-25T10:12:19.599962+0000","last_clean":"2024-04-25T10:12:19.599962+0000","last_became_active":"2024-04-25T10:12:17.576862+0000","last_became_peered":"2024-04-25T10:12:17.576862+0000","last_unstale":"2024-04-25T10:12:19.599962+0000","last_undegraded":"2024-04-25T10:12:19.599962+0000","last_fullsized":"2024-04-25T10:12:19.599962+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:05.326032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703239+0000","last_change":"2024-04-25T10:12:18.641574+0000","last_active":"2024-04-25T10:12:19.703239+0000","last_peered":"2024-04-25T10:12:19.703239+0000","last_clean":"2024-04-25T10:12:19.703239+0000","last_became_active":"2024-04-25T10:12:18.640847+0000","last_became_peered":"2024-04-25T10:12:18.640847+0000","last_unstale":"2024-04-25T10:12:19.703239+0000","last_undegraded":"2024-04-25T10:12:19.703239+0000","last_fullsized":"2024-04-25T10:12:19.703239+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.909841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701024+0000","last_change":"2024-04-25T10:12:17.570500+0000","last_active":"2024-04-25T10:12:19.701024+0000","last_peered":"2024-04-25T10:12:19.701024+0000","last_clean":"2024-04-25T10:12:19.701024+0000","last_became_active":"2024-04-25T10:12:17.569801+0000","last_became_peered":"2024-04-25T10:12:17.569801+0000","last_unstale":"2024-04-25T10:12:19.701024+0000","last_undegraded":"2024-04-25T10:12:19.701024+0000","last_fullsized":"2024-04-25T10:12:19.701024+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:29.525697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.618336+0000","last_change":"2024-04-25T10:12:18.627706+0000","last_active":"2024-04-25T10:12:19.618336+0000","last_peered":"2024-04-25T10:12:19.618336+0000","last_clean":"2024-04-25T10:12:19.618336+0000","last_became_active":"2024-04-25T10:12:18.627424+0000","last_became_peered":"2024-04-25T10:12:18.627424+0000","last_unstale":"2024-04-25T10:12:19.618336+0000","last_undegraded":"2024-04-25T10:12:19.618336+0000","last_fullsized":"2024-04-25T10:12:19.618336+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:43.817554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703092+0000","last_change":"2024-04-25T10:12:17.583961+0000","last_active":"2024-04-25T10:12:19.703092+0000","last_peered":"2024-04-25T10:12:19.703092+0000","last_clean":"2024-04-25T10:12:19.703092+0000","last_became_active":"2024-04-25T10:12:17.581672+0000","last_became_peered":"2024-04-25T10:12:17.581672+0000","last_unstale":"2024-04-25T10:12:19.703092+0000","last_undegraded":"2024-04-25T10:12:19.703092+0000","last_fullsized":"2024-04-25T10:12:19.703092+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:50.546661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.607596+0000","last_change":"2024-04-25T10:12:18.656628+0000","last_active":"2024-04-25T10:12:19.607596+0000","last_peered":"2024-04-25T10:12:19.607596+0000","last_clean":"2024-04-25T10:12:19.607596+0000","last_became_active":"2024-04-25T10:12:18.653910+0000","last_became_peered":"2024-04-25T10:12:18.653910+0000","last_unstale":"2024-04-25T10:12:19.607596+0000","last_undegraded":"2024-04-25T10:12:19.607596+0000","last_fullsized":"2024-04-25T10:12:19.607596+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:55.697502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703512+0000","last_change":"2024-04-25T10:12:17.587776+0000","last_active":"2024-04-25T10:12:19.703512+0000","last_peered":"2024-04-25T10:12:19.703512+0000","last_clean":"2024-04-25T10:12:19.703512+0000","last_became_active":"2024-04-25T10:12:17.587168+0000","last_became_peered":"2024-04-25T10:12:17.587168+0000","last_unstale":"2024-04-25T10:12:19.703512+0000","last_undegraded":"2024-04-25T10:12:19.703512+0000","last_fullsized":"2024-04-25T10:12:19.703512+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:54.686538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617536+0000","last_change":"2024-04-25T10:12:18.616514+0000","last_active":"2024-04-25T10:12:19.617536+0000","last_peered":"2024-04-25T10:12:19.617536+0000","last_clean":"2024-04-25T10:12:19.617536+0000","last_became_active":"2024-04-25T10:12:18.616254+0000","last_became_peered":"2024-04-25T10:12:18.616254+0000","last_unstale":"2024-04-25T10:12:19.617536+0000","last_undegraded":"2024-04-25T10:12:19.617536+0000","last_fullsized":"2024-04-25T10:12:19.617536+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:59.683317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617573+0000","last_change":"2024-04-25T10:12:17.575139+0000","last_active":"2024-04-25T10:12:19.617573+0000","last_peered":"2024-04-25T10:12:19.617573+0000","last_clean":"2024-04-25T10:12:19.617573+0000","last_became_active":"2024-04-25T10:12:17.574722+0000","last_became_peered":"2024-04-25T10:12:17.574722+0000","last_unstale":"2024-04-25T10:12:19.617573+0000","last_undegraded":"2024-04-25T10:12:19.617573+0000","last_fullsized":"2024-04-25T10:12:19.617573+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:05.468669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700648+0000","last_change":"2024-04-25T10:12:18.654910+0000","last_active":"2024-04-25T10:12:19.700648+0000","last_peered":"2024-04-25T10:12:19.700648+0000","last_clean":"2024-04-25T10:12:19.700648+0000","last_became_active":"2024-04-25T10:12:18.654589+0000","last_became_peered":"2024-04-25T10:12:18.654589+0000","last_unstale":"2024-04-25T10:12:19.700648+0000","last_undegraded":"2024-04-25T10:12:19.700648+0000","last_fullsized":"2024-04-25T10:12:19.700648+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:21.891519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702465+0000","last_change":"2024-04-25T10:12:17.581343+0000","last_active":"2024-04-25T10:12:19.702465+0000","last_peered":"2024-04-25T10:12:19.702465+0000","last_clean":"2024-04-25T10:12:19.702465+0000","last_became_active":"2024-04-25T10:12:17.580750+0000","last_became_peered":"2024-04-25T10:12:17.580750+0000","last_unstale":"2024-04-25T10:12:19.702465+0000","last_undegraded":"2024-04-25T10:12:19.702465+0000","last_fullsized":"2024-04-25T10:12:19.702465+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:58.951223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701874+0000","last_change":"2024-04-25T10:12:18.623442+0000","last_active":"2024-04-25T10:12:19.701874+0000","last_peered":"2024-04-25T10:12:19.701874+0000","last_clean":"2024-04-25T10:12:19.701874+0000","last_became_active":"2024-04-25T10:12:18.620792+0000","last_became_peered":"2024-04-25T10:12:18.620792+0000","last_unstale":"2024-04-25T10:12:19.701874+0000","last_undegraded":"2024-04-25T10:12:19.701874+0000","last_fullsized":"2024-04-25T10:12:19.701874+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:51.557626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700513+0000","last_change":"2024-04-25T10:12:17.568160+0000","last_active":"2024-04-25T10:12:19.700513+0000","last_peered":"2024-04-25T10:12:19.700513+0000","last_clean":"2024-04-25T10:12:19.700513+0000","last_became_active":"2024-04-25T10:12:17.567792+0000","last_became_peered":"2024-04-25T10:12:17.567792+0000","last_unstale":"2024-04-25T10:12:19.700513+0000","last_undegraded":"2024-04-25T10:12:19.700513+0000","last_fullsized":"2024-04-25T10:12:19.700513+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:33.548486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605804+0000","last_change":"2024-04-25T10:12:18.657016+0000","last_active":"2024-04-25T10:12:19.605804+0000","last_peered":"2024-04-25T10:12:19.605804+0000","last_clean":"2024-04-25T10:12:19.605804+0000","last_became_active":"2024-04-25T10:12:18.655162+0000","last_became_peered":"2024-04-25T10:12:18.655162+0000","last_unstale":"2024-04-25T10:12:19.605804+0000","last_undegraded":"2024-04-25T10:12:19.605804+0000","last_fullsized":"2024-04-25T10:12:19.605804+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:20.607447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701460+0000","last_change":"2024-04-25T10:12:17.582627+0000","last_active":"2024-04-25T10:12:19.701460+0000","last_peered":"2024-04-25T10:12:19.701460+0000","last_clean":"2024-04-25T10:12:19.701460+0000","last_became_active":"2024-04-25T10:12:17.582110+0000","last_became_peered":"2024-04-25T10:12:17.582110+0000","last_unstale":"2024-04-25T10:12:19.701460+0000","last_undegraded":"2024-04-25T10:12:19.701460+0000","last_fullsized":"2024-04-25T10:12:19.701460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:16.325184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617908+0000","last_change":"2024-04-25T10:12:18.616363+0000","last_active":"2024-04-25T10:12:19.617908+0000","last_peered":"2024-04-25T10:12:19.617908+0000","last_clean":"2024-04-25T10:12:19.617908+0000","last_became_active":"2024-04-25T10:12:18.615989+0000","last_became_peered":"2024-04-25T10:12:18.615989+0000","last_unstale":"2024-04-25T10:12:19.617908+0000","last_undegraded":"2024-04-25T10:12:19.617908+0000","last_fullsized":"2024-04-25T10:12:19.617908+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:16.206199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.618186+0000","last_change":"2024-04-25T10:12:17.587972+0000","last_active":"2024-04-25T10:12:19.618186+0000","last_peered":"2024-04-25T10:12:19.618186+0000","last_clean":"2024-04-25T10:12:19.618186+0000","last_became_active":"2024-04-25T10:12:17.587669+0000","last_became_peered":"2024-04-25T10:12:17.587669+0000","last_unstale":"2024-04-25T10:12:19.618186+0000","last_undegraded":"2024-04-25T10:12:19.618186+0000","last_fullsized":"2024-04-25T10:12:19.618186+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:52.365283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599664+0000","last_change":"2024-04-25T10:12:18.610351+0000","last_active":"2024-04-25T10:12:19.599664+0000","last_peered":"2024-04-25T10:12:19.599664+0000","last_clean":"2024-04-25T10:12:19.599664+0000","last_became_active":"2024-04-25T10:12:18.609937+0000","last_became_peered":"2024-04-25T10:12:18.609937+0000","last_unstale":"2024-04-25T10:12:19.599664+0000","last_undegraded":"2024-04-25T10:12:19.599664+0000","last_fullsized":"2024-04-25T10:12:19.599664+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:56.303725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"120'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617189+0000","last_change":"2024-04-25T10:12:17.568368+0000","last_active":"2024-04-25T10:12:19.617189+0000","last_peered":"2024-04-25T10:12:19.617189+0000","last_clean":"2024-04-25T10:12:19.617189+0000","last_became_active":"2024-04-25T10:12:17.567843+0000","last_became_peered":"2024-04-25T10:12:17.567843+0000","last_unstale":"2024-04-25T10:12:19.617189+0000","last_undegraded":"2024-04-25T10:12:19.617189+0000","last_fullsized":"2024-04-25T10:12:19.617189+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:53.175476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.704136+0000","last_change":"2024-04-25T10:12:18.645617+0000","last_active":"2024-04-25T10:12:19.704136+0000","last_peered":"2024-04-25T10:12:19.704136+0000","last_clean":"2024-04-25T10:12:19.704136+0000","last_became_active":"2024-04-25T10:12:18.644810+0000","last_became_peered":"2024-04-25T10:12:18.644810+0000","last_unstale":"2024-04-25T10:12:19.704136+0000","last_undegraded":"2024-04-25T10:12:19.704136+0000","last_fullsized":"2024-04-25T10:12:19.704136+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:56.308484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606435+0000","last_change":"2024-04-25T10:12:17.566765+0000","last_active":"2024-04-25T10:12:19.606435+0000","last_peered":"2024-04-25T10:12:19.606435+0000","last_clean":"2024-04-25T10:12:19.606435+0000","last_became_active":"2024-04-25T10:12:17.566464+0000","last_became_peered":"2024-04-25T10:12:17.566464+0000","last_unstale":"2024-04-25T10:12:19.606435+0000","last_undegraded":"2024-04-25T10:12:19.606435+0000","last_fullsized":"2024-04-25T10:12:19.606435+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:00.082357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701151+0000","last_change":"2024-04-25T10:12:18.658436+0000","last_active":"2024-04-25T10:12:19.701151+0000","last_peered":"2024-04-25T10:12:19.701151+0000","last_clean":"2024-04-25T10:12:19.701151+0000","last_became_active":"2024-04-25T10:12:18.658218+0000","last_became_peered":"2024-04-25T10:12:18.658218+0000","last_unstale":"2024-04-25T10:12:19.701151+0000","last_undegraded":"2024-04-25T10:12:19.701151+0000","last_fullsized":"2024-04-25T10:12:19.701151+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:22.854516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.618014+0000","last_change":"2024-04-25T10:12:17.581866+0000","last_active":"2024-04-25T10:12:19.618014+0000","last_peered":"2024-04-25T10:12:19.618014+0000","last_clean":"2024-04-25T10:12:19.618014+0000","last_became_active":"2024-04-25T10:12:17.581381+0000","last_became_peered":"2024-04-25T10:12:17.581381+0000","last_unstale":"2024-04-25T10:12:19.618014+0000","last_undegraded":"2024-04-25T10:12:19.618014+0000","last_fullsized":"2024-04-25T10:12:19.618014+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:29.025177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617469+0000","last_change":"2024-04-25T10:12:18.644477+0000","last_active":"2024-04-25T10:12:19.617469+0000","last_peered":"2024-04-25T10:12:19.617469+0000","last_clean":"2024-04-25T10:12:19.617469+0000","last_became_active":"2024-04-25T10:12:18.644157+0000","last_became_peered":"2024-04-25T10:12:18.644157+0000","last_unstale":"2024-04-25T10:12:19.617469+0000","last_undegraded":"2024-04-25T10:12:19.617469+0000","last_fullsized":"2024-04-25T10:12:19.617469+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:53.708012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702028+0000","last_change":"2024-04-25T10:12:17.565323+0000","last_active":"2024-04-25T10:12:19.702028+0000","last_peered":"2024-04-25T10:12:19.702028+0000","last_clean":"2024-04-25T10:12:19.702028+0000","last_became_active":"2024-04-25T10:12:17.564693+0000","last_became_peered":"2024-04-25T10:12:17.564693+0000","last_unstale":"2024-04-25T10:12:19.702028+0000","last_undegraded":"2024-04-25T10:12:19.702028+0000","last_fullsized":"2024-04-25T10:12:19.702028+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:18.518046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700321+0000","last_change":"2024-04-25T10:12:18.627212+0000","last_active":"2024-04-25T10:12:19.700321+0000","last_peered":"2024-04-25T10:12:19.700321+0000","last_clean":"2024-04-25T10:12:19.700321+0000","last_became_active":"2024-04-25T10:12:18.626914+0000","last_became_peered":"2024-04-25T10:12:18.626914+0000","last_unstale":"2024-04-25T10:12:19.700321+0000","last_undegraded":"2024-04-25T10:12:19.700321+0000","last_fullsized":"2024-04-25T10:12:19.700321+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:42.950870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702510+0000","last_change":"2024-04-25T10:12:17.573885+0000","last_active":"2024-04-25T10:12:19.702510+0000","last_peered":"2024-04-25T10:12:19.702510+0000","last_clean":"2024-04-25T10:12:19.702510+0000","last_became_active":"2024-04-25T10:12:17.573084+0000","last_became_peered":"2024-04-25T10:12:17.573084+0000","last_unstale":"2024-04-25T10:12:19.702510+0000","last_undegraded":"2024-04-25T10:12:19.702510+0000","last_fullsized":"2024-04-25T10:12:19.702510+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:05.147159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702507+0000","last_change":"2024-04-25T10:12:18.623638+0000","last_active":"2024-04-25T10:12:19.702507+0000","last_peered":"2024-04-25T10:12:19.702507+0000","last_clean":"2024-04-25T10:12:19.702507+0000","last_became_active":"2024-04-25T10:12:18.622902+0000","last_became_peered":"2024-04-25T10:12:18.622902+0000","last_unstale":"2024-04-25T10:12:19.702507+0000","last_undegraded":"2024-04-25T10:12:19.702507+0000","last_fullsized":"2024-04-25T10:12:19.702507+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:47.116591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617411+0000","last_change":"2024-04-25T10:12:17.582346+0000","last_active":"2024-04-25T10:12:19.617411+0000","last_peered":"2024-04-25T10:12:19.617411+0000","last_clean":"2024-04-25T10:12:19.617411+0000","last_became_active":"2024-04-25T10:12:17.581724+0000","last_became_peered":"2024-04-25T10:12:17.581724+0000","last_unstale":"2024-04-25T10:12:19.617411+0000","last_undegraded":"2024-04-25T10:12:19.617411+0000","last_fullsized":"2024-04-25T10:12:19.617411+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:39.445476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599502+0000","last_change":"2024-04-25T10:12:18.602683+0000","last_active":"2024-04-25T10:12:19.599502+0000","last_peered":"2024-04-25T10:12:19.599502+0000","last_clean":"2024-04-25T10:12:19.599502+0000","last_became_active":"2024-04-25T10:12:18.601897+0000","last_became_peered":"2024-04-25T10:12:18.601897+0000","last_unstale":"2024-04-25T10:12:19.599502+0000","last_undegraded":"2024-04-25T10:12:19.599502+0000","last_fullsized":"2024-04-25T10:12:19.599502+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:56.597024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701616+0000","last_change":"2024-04-25T10:12:17.582230+0000","last_active":"2024-04-25T10:12:19.701616+0000","last_peered":"2024-04-25T10:12:19.701616+0000","last_clean":"2024-04-25T10:12:19.701616+0000","last_became_active":"2024-04-25T10:12:17.581529+0000","last_became_peered":"2024-04-25T10:12:17.581529+0000","last_unstale":"2024-04-25T10:12:19.701616+0000","last_undegraded":"2024-04-25T10:12:19.701616+0000","last_fullsized":"2024-04-25T10:12:19.701616+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.084700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702906+0000","last_change":"2024-04-25T10:12:18.616992+0000","last_active":"2024-04-25T10:12:19.702906+0000","last_peered":"2024-04-25T10:12:19.702906+0000","last_clean":"2024-04-25T10:12:19.702906+0000","last_became_active":"2024-04-25T10:12:18.616154+0000","last_became_peered":"2024-04-25T10:12:18.616154+0000","last_unstale":"2024-04-25T10:12:19.702906+0000","last_undegraded":"2024-04-25T10:12:19.702906+0000","last_fullsized":"2024-04-25T10:12:19.702906+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:11.015595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"119'8","reported_seq":22,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700068+0000","last_change":"2024-04-25T10:12:17.569428+0000","last_active":"2024-04-25T10:12:19.700068+0000","last_peered":"2024-04-25T10:12:19.700068+0000","last_clean":"2024-04-25T10:12:19.700068+0000","last_became_active":"2024-04-25T10:12:17.569061+0000","last_became_peered":"2024-04-25T10:12:17.569061+0000","last_unstale":"2024-04-25T10:12:19.700068+0000","last_undegraded":"2024-04-25T10:12:19.700068+0000","last_fullsized":"2024-04-25T10:12:19.700068+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:05.147159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":358,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701791+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:12:19.701791+0000","last_peered":"2024-04-25T10:12:19.701791+0000","last_clean":"2024-04-25T10:12:19.701791+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:12:19.701791+0000","last_undegraded":"2024-04-25T10:12:19.701791+0000","last_fullsized":"2024-04-25T10:12:19.701791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65160,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306680,"statfs":{"total":96636764160,"available":96570040320,"internally_reserved":0,"allocated":2093056,"data_stored":1282818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46524,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325316,"statfs":{"total":96636764160,"available":96589123584,"internally_reserved":0,"allocated":1486848,"data_stored":692256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59012,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312828,"statfs":{"total":96636764160,"available":96576335872,"internally_reserved":0,"allocated":1490944,"data_stored":691834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72908,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298932,"statfs":{"total":96636764160,"available":96562106368,"internally_reserved":0,"allocated":1507328,"data_stored":699978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72900,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298940,"statfs":{"total":96636764160,"available":96562114560,"internally_reserved":0,"allocated":1499136,"data_stored":693320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46520,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325320,"statfs":{"total":96636764160,"available":96589127680,"internally_reserved":0,"allocated":1482752,"data_stored":691674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59000,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312840,"statfs":{"total":96636764160,"available":96576348160,"internally_reserved":0,"allocated":1486848,"data_stored":691696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69308,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302532,"statfs":{"total":96636764160,"available":96565792768,"internally_reserved":0,"allocated":2080768,"data_stored":1282064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:21.569 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:21.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.721+0000 7ff5ef0fd640 1 Processor -- start 2024-04-25T10:12:21.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.721+0000 7ff5ef0fd640 1 -- start start 2024-04-25T10:12:21.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ef0fd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff5e8159dc0 0x7ff5e80cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ef0fd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff5e815e220 0x7ff5e806d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ef0fd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e80ce880 con 0x7ff5e815a1a0 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ef0fd640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e80c49e0 con 0x7ff5e815e220 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ef0fd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e80ceb20 con 0x7ff5e8159dc0 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5e7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff5e815a1a0 0x7ff5e815db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46702/0 (socket says 172.21.15.133:46702) 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5e7fff640 1 -- 172.21.15.133:0/472112911 learned_addr learned my addr 172.21.15.133:0/472112911 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ed69a640 1 --2- 172.21.15.133:0/472112911 >> v2:172.21.15.133:3300/0 conn(0x7ff5e815e220 0x7ff5e806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ece99640 1 --2- 172.21.15.133:0/472112911 >> v2:172.21.15.162:3300/0 conn(0x7ff5e8159dc0 0x7ff5e80cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5e77fe640 1 -- 172.21.15.133:0/472112911 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892387964 0 0) 0x7ff5e80ce880 con 0x7ff5e815a1a0 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5e77fe640 1 -- 172.21.15.133:0/472112911 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5d8003530 con 0x7ff5e815a1a0 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ed69a640 1 -- 172.21.15.133:0/472112911 >> v2:172.21.15.162:3300/0 conn(0x7ff5e8159dc0 msgr2=0x7ff5e80cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ed69a640 1 --2- 172.21.15.133:0/472112911 >> v2:172.21.15.162:3300/0 conn(0x7ff5e8159dc0 0x7ff5e80cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ed69a640 1 -- 172.21.15.133:0/472112911 >> v1:172.21.15.133:6789/0 conn(0x7ff5e815a1a0 legacy=0x7ff5e815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ed69a640 1 -- 172.21.15.133:0/472112911 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80cbfa0 con 0x7ff5e815e220 2024-04-25T10:12:21.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5ed69a640 1 --2- 172.21.15.133:0/472112911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e815e220 0x7ff5e806d350 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7ff5e806f040 tx=0x7ff5e001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8775ce1c7333a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5e77fe640 1 -- 172.21.15.133:0/472112911 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00b2070 con 0x7ff5e815e220 2024-04-25T10:12:21.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5e77fe640 1 -- 172.21.15.133:0/472112911 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e00a57f0 con 0x7ff5e815e220 2024-04-25T10:12:21.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.725+0000 7ff5e77fe640 1 -- 172.21.15.133:0/472112911 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00a5bc0 con 0x7ff5e815e220 2024-04-25T10:12:21.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/472112911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e815e220 msgr2=0x7ff5e806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 --2- 172.21.15.133:0/472112911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e815e220 0x7ff5e806d350 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7ff5e806f040 tx=0x7ff5e001c930 comp rx=0 tx=0).stop 2024-04-25T10:12:21.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/472112911 shutdown_connections 2024-04-25T10:12:21.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 --2- 172.21.15.133:0/472112911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e815e220 0x7ff5e806d350 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 --2- 172.21.15.133:0/472112911 >> v2:172.21.15.162:3300/0 conn(0x7ff5e8159dc0 0x7ff5e80cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/472112911 >> 172.21.15.133:0/472112911 conn(0x7ff5e8067000 msgr2=0x7ff5e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:21.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/472112911 shutdown_connections 2024-04-25T10:12:21.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/472112911 wait complete. 2024-04-25T10:12:21.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 Processor -- start 2024-04-25T10:12:21.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 -- start start 2024-04-25T10:12:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8159dc0 0x7ff5e814a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5e815e220 0x7ff5e8150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5e814da10 0x7ff5e814de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e806e220 con 0x7ff5e8159dc0 2024-04-25T10:12:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e806e0a0 con 0x7ff5e815e220 2024-04-25T10:12:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ef0fd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e806e3a0 con 0x7ff5e814da10 2024-04-25T10:12:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ece99640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8159dc0 0x7ff5e814a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ece99640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8159dc0 0x7ff5e814a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43864/0 (socket says 172.21.15.133:43864) 2024-04-25T10:12:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ece99640 1 -- 172.21.15.133:0/3543596145 learned_addr learned my addr 172.21.15.133:0/3543596145 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5e7fff640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5e815e220 0x7ff5e8150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ed69a640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5e814da10 0x7ff5e814de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ece99640 1 -- 172.21.15.133:0/3543596145 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5e814da10 msgr2=0x7ff5e814de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ece99640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5e814da10 0x7ff5e814de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ece99640 1 -- 172.21.15.133:0/3543596145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5e815e220 msgr2=0x7ff5e8150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ece99640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5e815e220 0x7ff5e8150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ece99640 1 -- 172.21.15.133:0/3543596145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e814ff40 con 0x7ff5e8159dc0 2024-04-25T10:12:21.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5e7fff640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5e815e220 0x7ff5e8150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:21.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5ece99640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8159dc0 0x7ff5e814a930 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00d800 tx=0x7ff5dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.729+0000 7ff5e57fa640 1 -- 172.21.15.133:0/3543596145 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00b660 con 0x7ff5e8159dc0 2024-04-25T10:12:21.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.733+0000 7ff5e57fa640 1 -- 172.21.15.133:0/3543596145 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5dc019070 con 0x7ff5e8159dc0 2024-04-25T10:12:21.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.733+0000 7ff5e57fa640 1 -- 172.21.15.133:0/3543596145 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00ba80 con 0x7ff5e8159dc0 2024-04-25T10:12:21.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.733+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/3543596145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e8150220 con 0x7ff5e8159dc0 2024-04-25T10:12:21.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.733+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/3543596145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e81507e0 con 0x7ff5e8159dc0 2024-04-25T10:12:21.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.733+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/3543596145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e8068ef0 con 0x7ff5e8159dc0 2024-04-25T10:12:21.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.733+0000 7ff5e57fa640 1 -- 172.21.15.133:0/3543596145 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5dc00bc20 con 0x7ff5e8159dc0 2024-04-25T10:12:21.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.733+0000 7ff5e57fa640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5cc066f60 0x7ff5cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.733+0000 7ff5e57fa640 1 -- 172.21.15.133:0/3543596145 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5dc0855f0 con 0x7ff5e8159dc0 2024-04-25T10:12:21.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.733+0000 7ff5e7fff640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5cc066f60 0x7ff5cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.733+0000 7ff5e7fff640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5cc066f60 0x7ff5cc069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8153d30 tx=0x7ff5d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.737+0000 7ff5e57fa640 1 -- 172.21.15.133:0/3543596145 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5dc056500 con 0x7ff5e8159dc0 2024-04-25T10:12:21.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.893+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/3543596145 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5e8157f00 con 0x7ff5cc066f60 2024-04-25T10:12:21.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.909+0000 7ff5e57fa640 1 -- 172.21.15.133:0/3543596145 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252483 (secure 0 0 0) 0x7ff5e8157f00 con 0x7ff5cc066f60 2024-04-25T10:12:21.912 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:21.914 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:12:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/3543596145 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5cc066f60 msgr2=0x7ff5cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5cc066f60 0x7ff5cc069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8153d30 tx=0x7ff5d4009040 comp rx=0 tx=0).stop 2024-04-25T10:12:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/3543596145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8159dc0 msgr2=0x7ff5e814a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8159dc0 0x7ff5e814a930 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00d800 tx=0x7ff5dc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/3543596145 shutdown_connections 2024-04-25T10:12:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff5e814da10 0x7ff5e814de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff5e815e220 0x7ff5e8150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff5cc066f60 0x7ff5cc069420 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 --2- 172.21.15.133:0/3543596145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5e8159dc0 0x7ff5e814a930 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/3543596145 >> 172.21.15.133:0/3543596145 conn(0x7ff5e8067000 msgr2=0x7ff5e8158540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:21.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/3543596145 shutdown_connections 2024-04-25T10:12:21.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:21.913+0000 7ff5ef0fd640 1 -- 172.21.15.133:0/3543596145 wait complete. 2024-04-25T10:12:21.931 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-04-25T10:12:19.897150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":491332,"kb_used_data":12820,"kb_used_omap":12,"kb_used_meta":478387,"kb_avail":754483388,"statfs":{"total":773094113280,"available":772590989312,"internally_reserved":0,"allocated":13127680,"data_stored":6725640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":489868849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000962"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703776+0000","last_change":"2024-04-25T10:12:18.640529+0000","last_active":"2024-04-25T10:12:19.703776+0000","last_peered":"2024-04-25T10:12:19.703776+0000","last_clean":"2024-04-25T10:12:19.703776+0000","last_became_active":"2024-04-25T10:12:18.624696+0000","last_became_peered":"2024-04-25T10:12:18.624696+0000","last_unstale":"2024-04-25T10:12:19.703776+0000","last_undegraded":"2024-04-25T10:12:19.703776+0000","last_fullsized":"2024-04-25T10:12:19.703776+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:56.597024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701567+0000","last_change":"2024-04-25T10:12:18.615785+0000","last_active":"2024-04-25T10:12:19.701567+0000","last_peered":"2024-04-25T10:12:19.701567+0000","last_clean":"2024-04-25T10:12:19.701567+0000","last_became_active":"2024-04-25T10:12:18.615161+0000","last_became_peered":"2024-04-25T10:12:18.615161+0000","last_unstale":"2024-04-25T10:12:19.701567+0000","last_undegraded":"2024-04-25T10:12:19.701567+0000","last_fullsized":"2024-04-25T10:12:19.701567+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:55.697502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701422+0000","last_change":"2024-04-25T10:12:18.616412+0000","last_active":"2024-04-25T10:12:19.701422+0000","last_peered":"2024-04-25T10:12:19.701422+0000","last_clean":"2024-04-25T10:12:19.701422+0000","last_became_active":"2024-04-25T10:12:18.614475+0000","last_became_peered":"2024-04-25T10:12:18.614475+0000","last_unstale":"2024-04-25T10:12:19.701422+0000","last_undegraded":"2024-04-25T10:12:19.701422+0000","last_fullsized":"2024-04-25T10:12:19.701422+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:59.962187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703509+0000","last_change":"2024-04-25T10:12:18.642820+0000","last_active":"2024-04-25T10:12:19.703509+0000","last_peered":"2024-04-25T10:12:19.703509+0000","last_clean":"2024-04-25T10:12:19.703509+0000","last_became_active":"2024-04-25T10:12:18.642288+0000","last_became_peered":"2024-04-25T10:12:18.642288+0000","last_unstale":"2024-04-25T10:12:19.703509+0000","last_undegraded":"2024-04-25T10:12:19.703509+0000","last_fullsized":"2024-04-25T10:12:19.703509+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:13.070946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701448+0000","last_change":"2024-04-25T10:12:18.638296+0000","last_active":"2024-04-25T10:12:19.701448+0000","last_peered":"2024-04-25T10:12:19.701448+0000","last_clean":"2024-04-25T10:12:19.701448+0000","last_became_active":"2024-04-25T10:12:18.637508+0000","last_became_peered":"2024-04-25T10:12:18.637508+0000","last_unstale":"2024-04-25T10:12:19.701448+0000","last_undegraded":"2024-04-25T10:12:19.701448+0000","last_fullsized":"2024-04-25T10:12:19.701448+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:43.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617724+0000","last_change":"2024-04-25T10:12:18.633248+0000","last_active":"2024-04-25T10:12:19.617724+0000","last_peered":"2024-04-25T10:12:19.617724+0000","last_clean":"2024-04-25T10:12:19.617724+0000","last_became_active":"2024-04-25T10:12:18.632227+0000","last_became_peered":"2024-04-25T10:12:18.632227+0000","last_unstale":"2024-04-25T10:12:19.617724+0000","last_undegraded":"2024-04-25T10:12:19.617724+0000","last_fullsized":"2024-04-25T10:12:19.617724+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:17.862576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617387+0000","last_change":"2024-04-25T10:12:18.632723+0000","last_active":"2024-04-25T10:12:19.617387+0000","last_peered":"2024-04-25T10:12:19.617387+0000","last_clean":"2024-04-25T10:12:19.617387+0000","last_became_active":"2024-04-25T10:12:18.632428+0000","last_became_peered":"2024-04-25T10:12:18.632428+0000","last_unstale":"2024-04-25T10:12:19.617387+0000","last_undegraded":"2024-04-25T10:12:19.617387+0000","last_fullsized":"2024-04-25T10:12:19.617387+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:53.239093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.699954+0000","last_change":"2024-04-25T10:12:18.619723+0000","last_active":"2024-04-25T10:12:19.699954+0000","last_peered":"2024-04-25T10:12:19.699954+0000","last_clean":"2024-04-25T10:12:19.699954+0000","last_became_active":"2024-04-25T10:12:18.613711+0000","last_became_peered":"2024-04-25T10:12:18.613711+0000","last_unstale":"2024-04-25T10:12:19.699954+0000","last_undegraded":"2024-04-25T10:12:19.699954+0000","last_fullsized":"2024-04-25T10:12:19.699954+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:37.893622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703528+0000","last_change":"2024-04-25T10:12:18.632189+0000","last_active":"2024-04-25T10:12:19.703528+0000","last_peered":"2024-04-25T10:12:19.703528+0000","last_clean":"2024-04-25T10:12:19.703528+0000","last_became_active":"2024-04-25T10:12:18.631412+0000","last_became_peered":"2024-04-25T10:12:18.631412+0000","last_unstale":"2024-04-25T10:12:19.703528+0000","last_undegraded":"2024-04-25T10:12:19.703528+0000","last_fullsized":"2024-04-25T10:12:19.703528+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:26.110237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599588+0000","last_change":"2024-04-25T10:12:18.617400+0000","last_active":"2024-04-25T10:12:19.599588+0000","last_peered":"2024-04-25T10:12:19.599588+0000","last_clean":"2024-04-25T10:12:19.599588+0000","last_became_active":"2024-04-25T10:12:18.617064+0000","last_became_peered":"2024-04-25T10:12:18.617064+0000","last_unstale":"2024-04-25T10:12:19.599588+0000","last_undegraded":"2024-04-25T10:12:19.599588+0000","last_fullsized":"2024-04-25T10:12:19.599588+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:14.608188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702909+0000","last_change":"2024-04-25T10:12:18.652345+0000","last_active":"2024-04-25T10:12:19.702909+0000","last_peered":"2024-04-25T10:12:19.702909+0000","last_clean":"2024-04-25T10:12:19.702909+0000","last_became_active":"2024-04-25T10:12:18.651859+0000","last_became_peered":"2024-04-25T10:12:18.651859+0000","last_unstale":"2024-04-25T10:12:19.702909+0000","last_undegraded":"2024-04-25T10:12:19.702909+0000","last_fullsized":"2024-04-25T10:12:19.702909+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:00.777963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617767+0000","last_change":"2024-04-25T10:12:18.644659+0000","last_active":"2024-04-25T10:12:19.617767+0000","last_peered":"2024-04-25T10:12:19.617767+0000","last_clean":"2024-04-25T10:12:19.617767+0000","last_became_active":"2024-04-25T10:12:18.644341+0000","last_became_peered":"2024-04-25T10:12:18.644341+0000","last_unstale":"2024-04-25T10:12:19.617767+0000","last_undegraded":"2024-04-25T10:12:19.617767+0000","last_fullsized":"2024-04-25T10:12:19.617767+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:50.342384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599772+0000","last_change":"2024-04-25T10:12:18.657079+0000","last_active":"2024-04-25T10:12:19.599772+0000","last_peered":"2024-04-25T10:12:19.599772+0000","last_clean":"2024-04-25T10:12:19.599772+0000","last_became_active":"2024-04-25T10:12:18.656801+0000","last_became_peered":"2024-04-25T10:12:18.656801+0000","last_unstale":"2024-04-25T10:12:19.599772+0000","last_undegraded":"2024-04-25T10:12:19.599772+0000","last_fullsized":"2024-04-25T10:12:19.599772+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:07.120146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701607+0000","last_change":"2024-04-25T10:12:18.624368+0000","last_active":"2024-04-25T10:12:19.701607+0000","last_peered":"2024-04-25T10:12:19.701607+0000","last_clean":"2024-04-25T10:12:19.701607+0000","last_became_active":"2024-04-25T10:12:18.623999+0000","last_became_peered":"2024-04-25T10:12:18.623999+0000","last_unstale":"2024-04-25T10:12:19.701607+0000","last_undegraded":"2024-04-25T10:12:19.701607+0000","last_fullsized":"2024-04-25T10:12:19.701607+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:00.741659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701865+0000","last_change":"2024-04-25T10:12:18.653210+0000","last_active":"2024-04-25T10:12:19.701865+0000","last_peered":"2024-04-25T10:12:19.701865+0000","last_clean":"2024-04-25T10:12:19.701865+0000","last_became_active":"2024-04-25T10:12:18.652493+0000","last_became_peered":"2024-04-25T10:12:18.652493+0000","last_unstale":"2024-04-25T10:12:19.701865+0000","last_undegraded":"2024-04-25T10:12:19.701865+0000","last_fullsized":"2024-04-25T10:12:19.701865+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:32.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617332+0000","last_change":"2024-04-25T10:12:18.625906+0000","last_active":"2024-04-25T10:12:19.617332+0000","last_peered":"2024-04-25T10:12:19.617332+0000","last_clean":"2024-04-25T10:12:19.617332+0000","last_became_active":"2024-04-25T10:12:18.622387+0000","last_became_peered":"2024-04-25T10:12:18.622387+0000","last_unstale":"2024-04-25T10:12:19.617332+0000","last_undegraded":"2024-04-25T10:12:19.617332+0000","last_fullsized":"2024-04-25T10:12:19.617332+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.200844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599876+0000","last_change":"2024-04-25T10:12:18.658941+0000","last_active":"2024-04-25T10:12:19.599876+0000","last_peered":"2024-04-25T10:12:19.599876+0000","last_clean":"2024-04-25T10:12:19.599876+0000","last_became_active":"2024-04-25T10:12:18.658603+0000","last_became_peered":"2024-04-25T10:12:18.658603+0000","last_unstale":"2024-04-25T10:12:19.599876+0000","last_undegraded":"2024-04-25T10:12:19.599876+0000","last_fullsized":"2024-04-25T10:12:19.599876+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:20.607447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701953+0000","last_change":"2024-04-25T10:12:18.615498+0000","last_active":"2024-04-25T10:12:19.701953+0000","last_peered":"2024-04-25T10:12:19.701953+0000","last_clean":"2024-04-25T10:12:19.701953+0000","last_became_active":"2024-04-25T10:12:18.614717+0000","last_became_peered":"2024-04-25T10:12:18.614717+0000","last_unstale":"2024-04-25T10:12:19.701953+0000","last_undegraded":"2024-04-25T10:12:19.701953+0000","last_fullsized":"2024-04-25T10:12:19.701953+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:39.250378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605804+0000","last_change":"2024-04-25T10:12:18.611963+0000","last_active":"2024-04-25T10:12:19.605804+0000","last_peered":"2024-04-25T10:12:19.605804+0000","last_clean":"2024-04-25T10:12:19.605804+0000","last_became_active":"2024-04-25T10:12:18.610934+0000","last_became_peered":"2024-04-25T10:12:18.610934+0000","last_unstale":"2024-04-25T10:12:19.605804+0000","last_undegraded":"2024-04-25T10:12:19.605804+0000","last_fullsized":"2024-04-25T10:12:19.605804+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:34.559450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700772+0000","last_change":"2024-04-25T10:12:18.617560+0000","last_active":"2024-04-25T10:12:19.700772+0000","last_peered":"2024-04-25T10:12:19.700772+0000","last_clean":"2024-04-25T10:12:19.700772+0000","last_became_active":"2024-04-25T10:12:18.617212+0000","last_became_peered":"2024-04-25T10:12:18.617212+0000","last_unstale":"2024-04-25T10:12:19.700772+0000","last_undegraded":"2024-04-25T10:12:19.700772+0000","last_fullsized":"2024-04-25T10:12:19.700772+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:12.825278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702924+0000","last_change":"2024-04-25T10:12:18.611176+0000","last_active":"2024-04-25T10:12:19.702924+0000","last_peered":"2024-04-25T10:12:19.702924+0000","last_clean":"2024-04-25T10:12:19.702924+0000","last_became_active":"2024-04-25T10:12:18.610455+0000","last_became_peered":"2024-04-25T10:12:18.610455+0000","last_unstale":"2024-04-25T10:12:19.702924+0000","last_undegraded":"2024-04-25T10:12:19.702924+0000","last_fullsized":"2024-04-25T10:12:19.702924+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:03.751919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702286+0000","last_change":"2024-04-25T10:12:18.642193+0000","last_active":"2024-04-25T10:12:19.702286+0000","last_peered":"2024-04-25T10:12:19.702286+0000","last_clean":"2024-04-25T10:12:19.702286+0000","last_became_active":"2024-04-25T10:12:18.641645+0000","last_became_peered":"2024-04-25T10:12:18.641645+0000","last_unstale":"2024-04-25T10:12:19.702286+0000","last_undegraded":"2024-04-25T10:12:19.702286+0000","last_fullsized":"2024-04-25T10:12:19.702286+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:15.188327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702119+0000","last_change":"2024-04-25T10:12:18.628091+0000","last_active":"2024-04-25T10:12:19.702119+0000","last_peered":"2024-04-25T10:12:19.702119+0000","last_clean":"2024-04-25T10:12:19.702119+0000","last_became_active":"2024-04-25T10:12:18.627202+0000","last_became_peered":"2024-04-25T10:12:18.627202+0000","last_unstale":"2024-04-25T10:12:19.702119+0000","last_undegraded":"2024-04-25T10:12:19.702119+0000","last_fullsized":"2024-04-25T10:12:19.702119+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:59.182914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.604935+0000","last_change":"2024-04-25T10:12:18.650350+0000","last_active":"2024-04-25T10:12:19.604935+0000","last_peered":"2024-04-25T10:12:19.604935+0000","last_clean":"2024-04-25T10:12:19.604935+0000","last_became_active":"2024-04-25T10:12:18.649666+0000","last_became_peered":"2024-04-25T10:12:18.649666+0000","last_unstale":"2024-04-25T10:12:19.604935+0000","last_undegraded":"2024-04-25T10:12:19.604935+0000","last_fullsized":"2024-04-25T10:12:19.604935+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:56.303725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.607374+0000","last_change":"2024-04-25T10:12:18.657681+0000","last_active":"2024-04-25T10:12:19.607374+0000","last_peered":"2024-04-25T10:12:19.607374+0000","last_clean":"2024-04-25T10:12:19.607374+0000","last_became_active":"2024-04-25T10:12:18.654829+0000","last_became_peered":"2024-04-25T10:12:18.654829+0000","last_unstale":"2024-04-25T10:12:19.607374+0000","last_undegraded":"2024-04-25T10:12:19.607374+0000","last_fullsized":"2024-04-25T10:12:19.607374+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:24.095664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700795+0000","last_change":"2024-04-25T10:12:18.610837+0000","last_active":"2024-04-25T10:12:19.700795+0000","last_peered":"2024-04-25T10:12:19.700795+0000","last_clean":"2024-04-25T10:12:19.700795+0000","last_became_active":"2024-04-25T10:12:18.610479+0000","last_became_peered":"2024-04-25T10:12:18.610479+0000","last_unstale":"2024-04-25T10:12:19.700795+0000","last_undegraded":"2024-04-25T10:12:19.700795+0000","last_fullsized":"2024-04-25T10:12:19.700795+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:15.393054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606197+0000","last_change":"2024-04-25T10:12:18.599480+0000","last_active":"2024-04-25T10:12:19.606197+0000","last_peered":"2024-04-25T10:12:19.606197+0000","last_clean":"2024-04-25T10:12:19.606197+0000","last_became_active":"2024-04-25T10:12:18.598701+0000","last_became_peered":"2024-04-25T10:12:18.598701+0000","last_unstale":"2024-04-25T10:12:19.606197+0000","last_undegraded":"2024-04-25T10:12:19.606197+0000","last_fullsized":"2024-04-25T10:12:19.606197+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:54.194570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617758+0000","last_change":"2024-04-25T10:12:18.650490+0000","last_active":"2024-04-25T10:12:19.617758+0000","last_peered":"2024-04-25T10:12:19.617758+0000","last_clean":"2024-04-25T10:12:19.617758+0000","last_became_active":"2024-04-25T10:12:18.650194+0000","last_became_peered":"2024-04-25T10:12:18.650194+0000","last_unstale":"2024-04-25T10:12:19.617758+0000","last_undegraded":"2024-04-25T10:12:19.617758+0000","last_fullsized":"2024-04-25T10:12:19.617758+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:46.630033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700483+0000","last_change":"2024-04-25T10:12:18.614318+0000","last_active":"2024-04-25T10:12:19.700483+0000","last_peered":"2024-04-25T10:12:19.700483+0000","last_clean":"2024-04-25T10:12:19.700483+0000","last_became_active":"2024-04-25T10:12:18.613947+0000","last_became_peered":"2024-04-25T10:12:18.613947+0000","last_unstale":"2024-04-25T10:12:19.700483+0000","last_undegraded":"2024-04-25T10:12:19.700483+0000","last_fullsized":"2024-04-25T10:12:19.700483+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:42.848005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700894+0000","last_change":"2024-04-25T10:12:18.642558+0000","last_active":"2024-04-25T10:12:19.700894+0000","last_peered":"2024-04-25T10:12:19.700894+0000","last_clean":"2024-04-25T10:12:19.700894+0000","last_became_active":"2024-04-25T10:12:18.642284+0000","last_became_peered":"2024-04-25T10:12:18.642284+0000","last_unstale":"2024-04-25T10:12:19.700894+0000","last_undegraded":"2024-04-25T10:12:19.700894+0000","last_fullsized":"2024-04-25T10:12:19.700894+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:17.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606256+0000","last_change":"2024-04-25T10:12:18.650875+0000","last_active":"2024-04-25T10:12:19.606256+0000","last_peered":"2024-04-25T10:12:19.606256+0000","last_clean":"2024-04-25T10:12:19.606256+0000","last_became_active":"2024-04-25T10:12:18.650161+0000","last_became_peered":"2024-04-25T10:12:18.650161+0000","last_unstale":"2024-04-25T10:12:19.606256+0000","last_undegraded":"2024-04-25T10:12:19.606256+0000","last_fullsized":"2024-04-25T10:12:19.606256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:42.950870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605342+0000","last_change":"2024-04-25T10:12:18.649608+0000","last_active":"2024-04-25T10:12:19.605342+0000","last_peered":"2024-04-25T10:12:19.605342+0000","last_clean":"2024-04-25T10:12:19.605342+0000","last_became_active":"2024-04-25T10:12:18.647174+0000","last_became_peered":"2024-04-25T10:12:18.647174+0000","last_unstale":"2024-04-25T10:12:19.605342+0000","last_undegraded":"2024-04-25T10:12:19.605342+0000","last_fullsized":"2024-04-25T10:12:19.605342+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:07.120146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599226+0000","last_change":"2024-04-25T10:12:18.627941+0000","last_active":"2024-04-25T10:12:19.599226+0000","last_peered":"2024-04-25T10:12:19.599226+0000","last_clean":"2024-04-25T10:12:19.599226+0000","last_became_active":"2024-04-25T10:12:18.627628+0000","last_became_peered":"2024-04-25T10:12:18.627628+0000","last_unstale":"2024-04-25T10:12:19.599226+0000","last_undegraded":"2024-04-25T10:12:19.599226+0000","last_fullsized":"2024-04-25T10:12:19.599226+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:00.777963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"119'4","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605624+0000","last_change":"2024-04-25T10:12:17.579133+0000","last_active":"2024-04-25T10:12:19.605624+0000","last_peered":"2024-04-25T10:12:19.605624+0000","last_clean":"2024-04-25T10:12:19.605624+0000","last_became_active":"2024-04-25T10:12:17.578600+0000","last_became_peered":"2024-04-25T10:12:17.578600+0000","last_unstale":"2024-04-25T10:12:19.605624+0000","last_undegraded":"2024-04-25T10:12:19.605624+0000","last_fullsized":"2024-04-25T10:12:19.605624+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:54.890814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617834+0000","last_change":"2024-04-25T10:12:18.632595+0000","last_active":"2024-04-25T10:12:19.617834+0000","last_peered":"2024-04-25T10:12:19.617834+0000","last_clean":"2024-04-25T10:12:19.617834+0000","last_became_active":"2024-04-25T10:12:18.632290+0000","last_became_peered":"2024-04-25T10:12:18.632290+0000","last_unstale":"2024-04-25T10:12:19.617834+0000","last_undegraded":"2024-04-25T10:12:19.617834+0000","last_fullsized":"2024-04-25T10:12:19.617834+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:59.505045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599414+0000","last_change":"2024-04-25T10:12:17.576602+0000","last_active":"2024-04-25T10:12:19.599414+0000","last_peered":"2024-04-25T10:12:19.599414+0000","last_clean":"2024-04-25T10:12:19.599414+0000","last_became_active":"2024-04-25T10:12:17.574129+0000","last_became_peered":"2024-04-25T10:12:17.574129+0000","last_unstale":"2024-04-25T10:12:19.599414+0000","last_undegraded":"2024-04-25T10:12:19.599414+0000","last_fullsized":"2024-04-25T10:12:19.599414+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:18.518046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606869+0000","last_change":"2024-04-25T10:12:18.649335+0000","last_active":"2024-04-25T10:12:19.606869+0000","last_peered":"2024-04-25T10:12:19.606869+0000","last_clean":"2024-04-25T10:12:19.606869+0000","last_became_active":"2024-04-25T10:12:18.647062+0000","last_became_peered":"2024-04-25T10:12:18.647062+0000","last_unstale":"2024-04-25T10:12:19.606869+0000","last_undegraded":"2024-04-25T10:12:19.606869+0000","last_fullsized":"2024-04-25T10:12:19.606869+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:17.336149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702852+0000","last_change":"2024-04-25T10:12:17.581450+0000","last_active":"2024-04-25T10:12:19.702852+0000","last_peered":"2024-04-25T10:12:19.702852+0000","last_clean":"2024-04-25T10:12:19.702852+0000","last_became_active":"2024-04-25T10:12:17.580879+0000","last_became_peered":"2024-04-25T10:12:17.580879+0000","last_unstale":"2024-04-25T10:12:19.702852+0000","last_undegraded":"2024-04-25T10:12:19.702852+0000","last_fullsized":"2024-04-25T10:12:19.702852+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:59.730695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703883+0000","last_change":"2024-04-25T10:12:18.633464+0000","last_active":"2024-04-25T10:12:19.703883+0000","last_peered":"2024-04-25T10:12:19.703883+0000","last_clean":"2024-04-25T10:12:19.703883+0000","last_became_active":"2024-04-25T10:12:18.632206+0000","last_became_peered":"2024-04-25T10:12:18.632206+0000","last_unstale":"2024-04-25T10:12:19.703883+0000","last_undegraded":"2024-04-25T10:12:19.703883+0000","last_fullsized":"2024-04-25T10:12:19.703883+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:01.399821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599267+0000","last_change":"2024-04-25T10:12:17.560626+0000","last_active":"2024-04-25T10:12:19.599267+0000","last_peered":"2024-04-25T10:12:19.599267+0000","last_clean":"2024-04-25T10:12:19.599267+0000","last_became_active":"2024-04-25T10:12:17.560209+0000","last_became_peered":"2024-04-25T10:12:17.560209+0000","last_unstale":"2024-04-25T10:12:19.599267+0000","last_undegraded":"2024-04-25T10:12:19.599267+0000","last_fullsized":"2024-04-25T10:12:19.599267+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:43.268617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599050+0000","last_change":"2024-04-25T10:12:18.610961+0000","last_active":"2024-04-25T10:12:19.599050+0000","last_peered":"2024-04-25T10:12:19.599050+0000","last_clean":"2024-04-25T10:12:19.599050+0000","last_became_active":"2024-04-25T10:12:18.609985+0000","last_became_peered":"2024-04-25T10:12:18.609985+0000","last_unstale":"2024-04-25T10:12:19.599050+0000","last_undegraded":"2024-04-25T10:12:19.599050+0000","last_fullsized":"2024-04-25T10:12:19.599050+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:32.635067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701764+0000","last_change":"2024-04-25T10:12:17.567188+0000","last_active":"2024-04-25T10:12:19.701764+0000","last_peered":"2024-04-25T10:12:19.701764+0000","last_clean":"2024-04-25T10:12:19.701764+0000","last_became_active":"2024-04-25T10:12:17.566439+0000","last_became_peered":"2024-04-25T10:12:17.566439+0000","last_unstale":"2024-04-25T10:12:19.701764+0000","last_undegraded":"2024-04-25T10:12:19.701764+0000","last_fullsized":"2024-04-25T10:12:19.701764+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:05.326032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701403+0000","last_change":"2024-04-25T10:12:18.636742+0000","last_active":"2024-04-25T10:12:19.701403+0000","last_peered":"2024-04-25T10:12:19.701403+0000","last_clean":"2024-04-25T10:12:19.701403+0000","last_became_active":"2024-04-25T10:12:18.624588+0000","last_became_peered":"2024-04-25T10:12:18.624588+0000","last_unstale":"2024-04-25T10:12:19.701403+0000","last_undegraded":"2024-04-25T10:12:19.701403+0000","last_fullsized":"2024-04-25T10:12:19.701403+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:22.109430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617557+0000","last_change":"2024-04-25T10:12:17.576005+0000","last_active":"2024-04-25T10:12:19.617557+0000","last_peered":"2024-04-25T10:12:19.617557+0000","last_clean":"2024-04-25T10:12:19.617557+0000","last_became_active":"2024-04-25T10:12:17.575599+0000","last_became_peered":"2024-04-25T10:12:17.575599+0000","last_unstale":"2024-04-25T10:12:19.617557+0000","last_undegraded":"2024-04-25T10:12:19.617557+0000","last_fullsized":"2024-04-25T10:12:19.617557+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:44.231969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701343+0000","last_change":"2024-04-25T10:12:18.623793+0000","last_active":"2024-04-25T10:12:19.701343+0000","last_peered":"2024-04-25T10:12:19.701343+0000","last_clean":"2024-04-25T10:12:19.701343+0000","last_became_active":"2024-04-25T10:12:18.623136+0000","last_became_peered":"2024-04-25T10:12:18.623136+0000","last_unstale":"2024-04-25T10:12:19.701343+0000","last_undegraded":"2024-04-25T10:12:19.701343+0000","last_fullsized":"2024-04-25T10:12:19.701343+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:24.095664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606334+0000","last_change":"2024-04-25T10:12:17.571160+0000","last_active":"2024-04-25T10:12:19.606334+0000","last_peered":"2024-04-25T10:12:19.606334+0000","last_clean":"2024-04-25T10:12:19.606334+0000","last_became_active":"2024-04-25T10:12:17.570821+0000","last_became_peered":"2024-04-25T10:12:17.570821+0000","last_unstale":"2024-04-25T10:12:19.606334+0000","last_undegraded":"2024-04-25T10:12:19.606334+0000","last_fullsized":"2024-04-25T10:12:19.606334+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:33.548486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599275+0000","last_change":"2024-04-25T10:12:18.614695+0000","last_active":"2024-04-25T10:12:19.599275+0000","last_peered":"2024-04-25T10:12:19.599275+0000","last_clean":"2024-04-25T10:12:19.599275+0000","last_became_active":"2024-04-25T10:12:18.614372+0000","last_became_peered":"2024-04-25T10:12:18.614372+0000","last_unstale":"2024-04-25T10:12:19.599275+0000","last_undegraded":"2024-04-25T10:12:19.599275+0000","last_fullsized":"2024-04-25T10:12:19.599275+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.909841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617538+0000","last_change":"2024-04-25T10:12:17.581954+0000","last_active":"2024-04-25T10:12:19.617538+0000","last_peered":"2024-04-25T10:12:19.617538+0000","last_clean":"2024-04-25T10:12:19.617538+0000","last_became_active":"2024-04-25T10:12:17.581569+0000","last_became_peered":"2024-04-25T10:12:17.581569+0000","last_unstale":"2024-04-25T10:12:19.617538+0000","last_undegraded":"2024-04-25T10:12:19.617538+0000","last_fullsized":"2024-04-25T10:12:19.617538+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:54.435602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.604873+0000","last_change":"2024-04-25T10:12:18.604234+0000","last_active":"2024-04-25T10:12:19.604873+0000","last_peered":"2024-04-25T10:12:19.604873+0000","last_clean":"2024-04-25T10:12:19.604873+0000","last_became_active":"2024-04-25T10:12:18.603464+0000","last_became_peered":"2024-04-25T10:12:18.603464+0000","last_unstale":"2024-04-25T10:12:19.604873+0000","last_undegraded":"2024-04-25T10:12:19.604873+0000","last_fullsized":"2024-04-25T10:12:19.604873+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:22.013337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605901+0000","last_change":"2024-04-25T10:12:17.583252+0000","last_active":"2024-04-25T10:12:19.605901+0000","last_peered":"2024-04-25T10:12:19.605901+0000","last_clean":"2024-04-25T10:12:19.605901+0000","last_became_active":"2024-04-25T10:12:17.582630+0000","last_became_peered":"2024-04-25T10:12:17.582630+0000","last_unstale":"2024-04-25T10:12:19.605901+0000","last_undegraded":"2024-04-25T10:12:19.605901+0000","last_fullsized":"2024-04-25T10:12:19.605901+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:08.124899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606834+0000","last_change":"2024-04-25T10:12:18.653889+0000","last_active":"2024-04-25T10:12:19.606834+0000","last_peered":"2024-04-25T10:12:19.606834+0000","last_clean":"2024-04-25T10:12:19.606834+0000","last_became_active":"2024-04-25T10:12:18.653303+0000","last_became_peered":"2024-04-25T10:12:18.653303+0000","last_unstale":"2024-04-25T10:12:19.606834+0000","last_undegraded":"2024-04-25T10:12:19.606834+0000","last_fullsized":"2024-04-25T10:12:19.606834+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:21.891519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606776+0000","last_change":"2024-04-25T10:12:17.575106+0000","last_active":"2024-04-25T10:12:19.606776+0000","last_peered":"2024-04-25T10:12:19.606776+0000","last_clean":"2024-04-25T10:12:19.606776+0000","last_became_active":"2024-04-25T10:12:17.574190+0000","last_became_peered":"2024-04-25T10:12:17.574190+0000","last_unstale":"2024-04-25T10:12:19.606776+0000","last_undegraded":"2024-04-25T10:12:19.606776+0000","last_fullsized":"2024-04-25T10:12:19.606776+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.706095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702089+0000","last_change":"2024-04-25T10:12:18.613630+0000","last_active":"2024-04-25T10:12:19.702089+0000","last_peered":"2024-04-25T10:12:19.702089+0000","last_clean":"2024-04-25T10:12:19.702089+0000","last_became_active":"2024-04-25T10:12:18.613310+0000","last_became_peered":"2024-04-25T10:12:18.613310+0000","last_unstale":"2024-04-25T10:12:19.702089+0000","last_undegraded":"2024-04-25T10:12:19.702089+0000","last_fullsized":"2024-04-25T10:12:19.702089+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:49.538210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"119'3","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606883+0000","last_change":"2024-04-25T10:12:17.583737+0000","last_active":"2024-04-25T10:12:19.606883+0000","last_peered":"2024-04-25T10:12:19.606883+0000","last_clean":"2024-04-25T10:12:19.606883+0000","last_became_active":"2024-04-25T10:12:17.583046+0000","last_became_peered":"2024-04-25T10:12:17.583046+0000","last_unstale":"2024-04-25T10:12:19.606883+0000","last_undegraded":"2024-04-25T10:12:19.606883+0000","last_fullsized":"2024-04-25T10:12:19.606883+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:21.002373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.607461+0000","last_change":"2024-04-25T10:12:18.657615+0000","last_active":"2024-04-25T10:12:19.607461+0000","last_peered":"2024-04-25T10:12:19.607461+0000","last_clean":"2024-04-25T10:12:19.607461+0000","last_became_active":"2024-04-25T10:12:18.654737+0000","last_became_peered":"2024-04-25T10:12:18.654737+0000","last_unstale":"2024-04-25T10:12:19.607461+0000","last_undegraded":"2024-04-25T10:12:19.607461+0000","last_fullsized":"2024-04-25T10:12:19.607461+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:59.962187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"119'6","reported_seq":20,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702215+0000","last_change":"2024-04-25T10:12:17.572523+0000","last_active":"2024-04-25T10:12:19.702215+0000","last_peered":"2024-04-25T10:12:19.702215+0000","last_clean":"2024-04-25T10:12:19.702215+0000","last_became_active":"2024-04-25T10:12:17.571923+0000","last_became_peered":"2024-04-25T10:12:17.571923+0000","last_unstale":"2024-04-25T10:12:19.702215+0000","last_undegraded":"2024-04-25T10:12:19.702215+0000","last_fullsized":"2024-04-25T10:12:19.702215+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:19.345544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605299+0000","last_change":"2024-04-25T10:12:18.617255+0000","last_active":"2024-04-25T10:12:19.605299+0000","last_peered":"2024-04-25T10:12:19.605299+0000","last_clean":"2024-04-25T10:12:19.605299+0000","last_became_active":"2024-04-25T10:12:18.616411+0000","last_became_peered":"2024-04-25T10:12:18.616411+0000","last_unstale":"2024-04-25T10:12:19.605299+0000","last_undegraded":"2024-04-25T10:12:19.605299+0000","last_fullsized":"2024-04-25T10:12:19.605299+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:25.717059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701406+0000","last_change":"2024-04-25T10:12:17.578832+0000","last_active":"2024-04-25T10:12:19.701406+0000","last_peered":"2024-04-25T10:12:19.701406+0000","last_clean":"2024-04-25T10:12:19.701406+0000","last_became_active":"2024-04-25T10:12:17.578516+0000","last_became_peered":"2024-04-25T10:12:17.578516+0000","last_unstale":"2024-04-25T10:12:19.701406+0000","last_undegraded":"2024-04-25T10:12:19.701406+0000","last_fullsized":"2024-04-25T10:12:19.701406+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:53.183606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599052+0000","last_change":"2024-04-25T10:12:18.610534+0000","last_active":"2024-04-25T10:12:19.599052+0000","last_peered":"2024-04-25T10:12:19.599052+0000","last_clean":"2024-04-25T10:12:19.599052+0000","last_became_active":"2024-04-25T10:12:18.609616+0000","last_became_peered":"2024-04-25T10:12:18.609616+0000","last_unstale":"2024-04-25T10:12:19.599052+0000","last_undegraded":"2024-04-25T10:12:19.599052+0000","last_fullsized":"2024-04-25T10:12:19.599052+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:15.188327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701833+0000","last_change":"2024-04-25T10:12:17.573191+0000","last_active":"2024-04-25T10:12:19.701833+0000","last_peered":"2024-04-25T10:12:19.701833+0000","last_clean":"2024-04-25T10:12:19.701833+0000","last_became_active":"2024-04-25T10:12:17.572696+0000","last_became_peered":"2024-04-25T10:12:17.572696+0000","last_unstale":"2024-04-25T10:12:19.701833+0000","last_undegraded":"2024-04-25T10:12:19.701833+0000","last_fullsized":"2024-04-25T10:12:19.701833+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.706095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700416+0000","last_change":"2024-04-25T10:12:18.642739+0000","last_active":"2024-04-25T10:12:19.700416+0000","last_peered":"2024-04-25T10:12:19.700416+0000","last_clean":"2024-04-25T10:12:19.700416+0000","last_became_active":"2024-04-25T10:12:18.642415+0000","last_became_peered":"2024-04-25T10:12:18.642415+0000","last_unstale":"2024-04-25T10:12:19.700416+0000","last_undegraded":"2024-04-25T10:12:19.700416+0000","last_fullsized":"2024-04-25T10:12:19.700416+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:00.736459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700417+0000","last_change":"2024-04-25T10:12:17.571971+0000","last_active":"2024-04-25T10:12:19.700417+0000","last_peered":"2024-04-25T10:12:19.700417+0000","last_clean":"2024-04-25T10:12:19.700417+0000","last_became_active":"2024-04-25T10:12:17.571593+0000","last_became_peered":"2024-04-25T10:12:17.571593+0000","last_unstale":"2024-04-25T10:12:19.700417+0000","last_undegraded":"2024-04-25T10:12:19.700417+0000","last_fullsized":"2024-04-25T10:12:19.700417+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:21.002373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605992+0000","last_change":"2024-04-25T10:12:18.649874+0000","last_active":"2024-04-25T10:12:19.605992+0000","last_peered":"2024-04-25T10:12:19.605992+0000","last_clean":"2024-04-25T10:12:19.605992+0000","last_became_active":"2024-04-25T10:12:18.648045+0000","last_became_peered":"2024-04-25T10:12:18.648045+0000","last_unstale":"2024-04-25T10:12:19.605992+0000","last_undegraded":"2024-04-25T10:12:19.605992+0000","last_fullsized":"2024-04-25T10:12:19.605992+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:37.893622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599962+0000","last_change":"2024-04-25T10:12:17.577393+0000","last_active":"2024-04-25T10:12:19.599962+0000","last_peered":"2024-04-25T10:12:19.599962+0000","last_clean":"2024-04-25T10:12:19.599962+0000","last_became_active":"2024-04-25T10:12:17.576862+0000","last_became_peered":"2024-04-25T10:12:17.576862+0000","last_unstale":"2024-04-25T10:12:19.599962+0000","last_undegraded":"2024-04-25T10:12:19.599962+0000","last_fullsized":"2024-04-25T10:12:19.599962+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:05.326032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703239+0000","last_change":"2024-04-25T10:12:18.641574+0000","last_active":"2024-04-25T10:12:19.703239+0000","last_peered":"2024-04-25T10:12:19.703239+0000","last_clean":"2024-04-25T10:12:19.703239+0000","last_became_active":"2024-04-25T10:12:18.640847+0000","last_became_peered":"2024-04-25T10:12:18.640847+0000","last_unstale":"2024-04-25T10:12:19.703239+0000","last_undegraded":"2024-04-25T10:12:19.703239+0000","last_fullsized":"2024-04-25T10:12:19.703239+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.909841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701024+0000","last_change":"2024-04-25T10:12:17.570500+0000","last_active":"2024-04-25T10:12:19.701024+0000","last_peered":"2024-04-25T10:12:19.701024+0000","last_clean":"2024-04-25T10:12:19.701024+0000","last_became_active":"2024-04-25T10:12:17.569801+0000","last_became_peered":"2024-04-25T10:12:17.569801+0000","last_unstale":"2024-04-25T10:12:19.701024+0000","last_undegraded":"2024-04-25T10:12:19.701024+0000","last_fullsized":"2024-04-25T10:12:19.701024+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:29.525697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.618336+0000","last_change":"2024-04-25T10:12:18.627706+0000","last_active":"2024-04-25T10:12:19.618336+0000","last_peered":"2024-04-25T10:12:19.618336+0000","last_clean":"2024-04-25T10:12:19.618336+0000","last_became_active":"2024-04-25T10:12:18.627424+0000","last_became_peered":"2024-04-25T10:12:18.627424+0000","last_unstale":"2024-04-25T10:12:19.618336+0000","last_undegraded":"2024-04-25T10:12:19.618336+0000","last_fullsized":"2024-04-25T10:12:19.618336+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:43.817554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703092+0000","last_change":"2024-04-25T10:12:17.583961+0000","last_active":"2024-04-25T10:12:19.703092+0000","last_peered":"2024-04-25T10:12:19.703092+0000","last_clean":"2024-04-25T10:12:19.703092+0000","last_became_active":"2024-04-25T10:12:17.581672+0000","last_became_peered":"2024-04-25T10:12:17.581672+0000","last_unstale":"2024-04-25T10:12:19.703092+0000","last_undegraded":"2024-04-25T10:12:19.703092+0000","last_fullsized":"2024-04-25T10:12:19.703092+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:50.546661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.607596+0000","last_change":"2024-04-25T10:12:18.656628+0000","last_active":"2024-04-25T10:12:19.607596+0000","last_peered":"2024-04-25T10:12:19.607596+0000","last_clean":"2024-04-25T10:12:19.607596+0000","last_became_active":"2024-04-25T10:12:18.653910+0000","last_became_peered":"2024-04-25T10:12:18.653910+0000","last_unstale":"2024-04-25T10:12:19.607596+0000","last_undegraded":"2024-04-25T10:12:19.607596+0000","last_fullsized":"2024-04-25T10:12:19.607596+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:55.697502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.703512+0000","last_change":"2024-04-25T10:12:17.587776+0000","last_active":"2024-04-25T10:12:19.703512+0000","last_peered":"2024-04-25T10:12:19.703512+0000","last_clean":"2024-04-25T10:12:19.703512+0000","last_became_active":"2024-04-25T10:12:17.587168+0000","last_became_peered":"2024-04-25T10:12:17.587168+0000","last_unstale":"2024-04-25T10:12:19.703512+0000","last_undegraded":"2024-04-25T10:12:19.703512+0000","last_fullsized":"2024-04-25T10:12:19.703512+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:54.686538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617536+0000","last_change":"2024-04-25T10:12:18.616514+0000","last_active":"2024-04-25T10:12:19.617536+0000","last_peered":"2024-04-25T10:12:19.617536+0000","last_clean":"2024-04-25T10:12:19.617536+0000","last_became_active":"2024-04-25T10:12:18.616254+0000","last_became_peered":"2024-04-25T10:12:18.616254+0000","last_unstale":"2024-04-25T10:12:19.617536+0000","last_undegraded":"2024-04-25T10:12:19.617536+0000","last_fullsized":"2024-04-25T10:12:19.617536+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:59.683317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617573+0000","last_change":"2024-04-25T10:12:17.575139+0000","last_active":"2024-04-25T10:12:19.617573+0000","last_peered":"2024-04-25T10:12:19.617573+0000","last_clean":"2024-04-25T10:12:19.617573+0000","last_became_active":"2024-04-25T10:12:17.574722+0000","last_became_peered":"2024-04-25T10:12:17.574722+0000","last_unstale":"2024-04-25T10:12:19.617573+0000","last_undegraded":"2024-04-25T10:12:19.617573+0000","last_fullsized":"2024-04-25T10:12:19.617573+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:05.468669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700648+0000","last_change":"2024-04-25T10:12:18.654910+0000","last_active":"2024-04-25T10:12:19.700648+0000","last_peered":"2024-04-25T10:12:19.700648+0000","last_clean":"2024-04-25T10:12:19.700648+0000","last_became_active":"2024-04-25T10:12:18.654589+0000","last_became_peered":"2024-04-25T10:12:18.654589+0000","last_unstale":"2024-04-25T10:12:19.700648+0000","last_undegraded":"2024-04-25T10:12:19.700648+0000","last_fullsized":"2024-04-25T10:12:19.700648+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:21.891519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702465+0000","last_change":"2024-04-25T10:12:17.581343+0000","last_active":"2024-04-25T10:12:19.702465+0000","last_peered":"2024-04-25T10:12:19.702465+0000","last_clean":"2024-04-25T10:12:19.702465+0000","last_became_active":"2024-04-25T10:12:17.580750+0000","last_became_peered":"2024-04-25T10:12:17.580750+0000","last_unstale":"2024-04-25T10:12:19.702465+0000","last_undegraded":"2024-04-25T10:12:19.702465+0000","last_fullsized":"2024-04-25T10:12:19.702465+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:58.951223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701874+0000","last_change":"2024-04-25T10:12:18.623442+0000","last_active":"2024-04-25T10:12:19.701874+0000","last_peered":"2024-04-25T10:12:19.701874+0000","last_clean":"2024-04-25T10:12:19.701874+0000","last_became_active":"2024-04-25T10:12:18.620792+0000","last_became_peered":"2024-04-25T10:12:18.620792+0000","last_unstale":"2024-04-25T10:12:19.701874+0000","last_undegraded":"2024-04-25T10:12:19.701874+0000","last_fullsized":"2024-04-25T10:12:19.701874+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:51.557626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700513+0000","last_change":"2024-04-25T10:12:17.568160+0000","last_active":"2024-04-25T10:12:19.700513+0000","last_peered":"2024-04-25T10:12:19.700513+0000","last_clean":"2024-04-25T10:12:19.700513+0000","last_became_active":"2024-04-25T10:12:17.567792+0000","last_became_peered":"2024-04-25T10:12:17.567792+0000","last_unstale":"2024-04-25T10:12:19.700513+0000","last_undegraded":"2024-04-25T10:12:19.700513+0000","last_fullsized":"2024-04-25T10:12:19.700513+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:33.548486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.605804+0000","last_change":"2024-04-25T10:12:18.657016+0000","last_active":"2024-04-25T10:12:19.605804+0000","last_peered":"2024-04-25T10:12:19.605804+0000","last_clean":"2024-04-25T10:12:19.605804+0000","last_became_active":"2024-04-25T10:12:18.655162+0000","last_became_peered":"2024-04-25T10:12:18.655162+0000","last_unstale":"2024-04-25T10:12:19.605804+0000","last_undegraded":"2024-04-25T10:12:19.605804+0000","last_fullsized":"2024-04-25T10:12:19.605804+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:20.607447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701460+0000","last_change":"2024-04-25T10:12:17.582627+0000","last_active":"2024-04-25T10:12:19.701460+0000","last_peered":"2024-04-25T10:12:19.701460+0000","last_clean":"2024-04-25T10:12:19.701460+0000","last_became_active":"2024-04-25T10:12:17.582110+0000","last_became_peered":"2024-04-25T10:12:17.582110+0000","last_unstale":"2024-04-25T10:12:19.701460+0000","last_undegraded":"2024-04-25T10:12:19.701460+0000","last_fullsized":"2024-04-25T10:12:19.701460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:16.325184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617908+0000","last_change":"2024-04-25T10:12:18.616363+0000","last_active":"2024-04-25T10:12:19.617908+0000","last_peered":"2024-04-25T10:12:19.617908+0000","last_clean":"2024-04-25T10:12:19.617908+0000","last_became_active":"2024-04-25T10:12:18.615989+0000","last_became_peered":"2024-04-25T10:12:18.615989+0000","last_unstale":"2024-04-25T10:12:19.617908+0000","last_undegraded":"2024-04-25T10:12:19.617908+0000","last_fullsized":"2024-04-25T10:12:19.617908+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:16.206199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.618186+0000","last_change":"2024-04-25T10:12:17.587972+0000","last_active":"2024-04-25T10:12:19.618186+0000","last_peered":"2024-04-25T10:12:19.618186+0000","last_clean":"2024-04-25T10:12:19.618186+0000","last_became_active":"2024-04-25T10:12:17.587669+0000","last_became_peered":"2024-04-25T10:12:17.587669+0000","last_unstale":"2024-04-25T10:12:19.618186+0000","last_undegraded":"2024-04-25T10:12:19.618186+0000","last_fullsized":"2024-04-25T10:12:19.618186+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:52.365283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599664+0000","last_change":"2024-04-25T10:12:18.610351+0000","last_active":"2024-04-25T10:12:19.599664+0000","last_peered":"2024-04-25T10:12:19.599664+0000","last_clean":"2024-04-25T10:12:19.599664+0000","last_became_active":"2024-04-25T10:12:18.609937+0000","last_became_peered":"2024-04-25T10:12:18.609937+0000","last_unstale":"2024-04-25T10:12:19.599664+0000","last_undegraded":"2024-04-25T10:12:19.599664+0000","last_fullsized":"2024-04-25T10:12:19.599664+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:56.303725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"120'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617189+0000","last_change":"2024-04-25T10:12:17.568368+0000","last_active":"2024-04-25T10:12:19.617189+0000","last_peered":"2024-04-25T10:12:19.617189+0000","last_clean":"2024-04-25T10:12:19.617189+0000","last_became_active":"2024-04-25T10:12:17.567843+0000","last_became_peered":"2024-04-25T10:12:17.567843+0000","last_unstale":"2024-04-25T10:12:19.617189+0000","last_undegraded":"2024-04-25T10:12:19.617189+0000","last_fullsized":"2024-04-25T10:12:19.617189+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:53.175476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.704136+0000","last_change":"2024-04-25T10:12:18.645617+0000","last_active":"2024-04-25T10:12:19.704136+0000","last_peered":"2024-04-25T10:12:19.704136+0000","last_clean":"2024-04-25T10:12:19.704136+0000","last_became_active":"2024-04-25T10:12:18.644810+0000","last_became_peered":"2024-04-25T10:12:18.644810+0000","last_unstale":"2024-04-25T10:12:19.704136+0000","last_undegraded":"2024-04-25T10:12:19.704136+0000","last_fullsized":"2024-04-25T10:12:19.704136+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:56.308484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.606435+0000","last_change":"2024-04-25T10:12:17.566765+0000","last_active":"2024-04-25T10:12:19.606435+0000","last_peered":"2024-04-25T10:12:19.606435+0000","last_clean":"2024-04-25T10:12:19.606435+0000","last_became_active":"2024-04-25T10:12:17.566464+0000","last_became_peered":"2024-04-25T10:12:17.566464+0000","last_unstale":"2024-04-25T10:12:19.606435+0000","last_undegraded":"2024-04-25T10:12:19.606435+0000","last_fullsized":"2024-04-25T10:12:19.606435+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:00.082357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701151+0000","last_change":"2024-04-25T10:12:18.658436+0000","last_active":"2024-04-25T10:12:19.701151+0000","last_peered":"2024-04-25T10:12:19.701151+0000","last_clean":"2024-04-25T10:12:19.701151+0000","last_became_active":"2024-04-25T10:12:18.658218+0000","last_became_peered":"2024-04-25T10:12:18.658218+0000","last_unstale":"2024-04-25T10:12:19.701151+0000","last_undegraded":"2024-04-25T10:12:19.701151+0000","last_fullsized":"2024-04-25T10:12:19.701151+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:22.854516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.618014+0000","last_change":"2024-04-25T10:12:17.581866+0000","last_active":"2024-04-25T10:12:19.618014+0000","last_peered":"2024-04-25T10:12:19.618014+0000","last_clean":"2024-04-25T10:12:19.618014+0000","last_became_active":"2024-04-25T10:12:17.581381+0000","last_became_peered":"2024-04-25T10:12:17.581381+0000","last_unstale":"2024-04-25T10:12:19.618014+0000","last_undegraded":"2024-04-25T10:12:19.618014+0000","last_fullsized":"2024-04-25T10:12:19.618014+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:29.025177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617469+0000","last_change":"2024-04-25T10:12:18.644477+0000","last_active":"2024-04-25T10:12:19.617469+0000","last_peered":"2024-04-25T10:12:19.617469+0000","last_clean":"2024-04-25T10:12:19.617469+0000","last_became_active":"2024-04-25T10:12:18.644157+0000","last_became_peered":"2024-04-25T10:12:18.644157+0000","last_unstale":"2024-04-25T10:12:19.617469+0000","last_undegraded":"2024-04-25T10:12:19.617469+0000","last_fullsized":"2024-04-25T10:12:19.617469+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:53.708012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702028+0000","last_change":"2024-04-25T10:12:17.565323+0000","last_active":"2024-04-25T10:12:19.702028+0000","last_peered":"2024-04-25T10:12:19.702028+0000","last_clean":"2024-04-25T10:12:19.702028+0000","last_became_active":"2024-04-25T10:12:17.564693+0000","last_became_peered":"2024-04-25T10:12:17.564693+0000","last_unstale":"2024-04-25T10:12:19.702028+0000","last_undegraded":"2024-04-25T10:12:19.702028+0000","last_fullsized":"2024-04-25T10:12:19.702028+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:18.518046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700321+0000","last_change":"2024-04-25T10:12:18.627212+0000","last_active":"2024-04-25T10:12:19.700321+0000","last_peered":"2024-04-25T10:12:19.700321+0000","last_clean":"2024-04-25T10:12:19.700321+0000","last_became_active":"2024-04-25T10:12:18.626914+0000","last_became_peered":"2024-04-25T10:12:18.626914+0000","last_unstale":"2024-04-25T10:12:19.700321+0000","last_undegraded":"2024-04-25T10:12:19.700321+0000","last_fullsized":"2024-04-25T10:12:19.700321+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:42.950870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702510+0000","last_change":"2024-04-25T10:12:17.573885+0000","last_active":"2024-04-25T10:12:19.702510+0000","last_peered":"2024-04-25T10:12:19.702510+0000","last_clean":"2024-04-25T10:12:19.702510+0000","last_became_active":"2024-04-25T10:12:17.573084+0000","last_became_peered":"2024-04-25T10:12:17.573084+0000","last_unstale":"2024-04-25T10:12:19.702510+0000","last_undegraded":"2024-04-25T10:12:19.702510+0000","last_fullsized":"2024-04-25T10:12:19.702510+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:05.147159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702507+0000","last_change":"2024-04-25T10:12:18.623638+0000","last_active":"2024-04-25T10:12:19.702507+0000","last_peered":"2024-04-25T10:12:19.702507+0000","last_clean":"2024-04-25T10:12:19.702507+0000","last_became_active":"2024-04-25T10:12:18.622902+0000","last_became_peered":"2024-04-25T10:12:18.622902+0000","last_unstale":"2024-04-25T10:12:19.702507+0000","last_undegraded":"2024-04-25T10:12:19.702507+0000","last_fullsized":"2024-04-25T10:12:19.702507+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:47.116591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.617411+0000","last_change":"2024-04-25T10:12:17.582346+0000","last_active":"2024-04-25T10:12:19.617411+0000","last_peered":"2024-04-25T10:12:19.617411+0000","last_clean":"2024-04-25T10:12:19.617411+0000","last_became_active":"2024-04-25T10:12:17.581724+0000","last_became_peered":"2024-04-25T10:12:17.581724+0000","last_unstale":"2024-04-25T10:12:19.617411+0000","last_undegraded":"2024-04-25T10:12:19.617411+0000","last_fullsized":"2024-04-25T10:12:19.617411+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:39.445476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.599502+0000","last_change":"2024-04-25T10:12:18.602683+0000","last_active":"2024-04-25T10:12:19.599502+0000","last_peered":"2024-04-25T10:12:19.599502+0000","last_clean":"2024-04-25T10:12:19.599502+0000","last_became_active":"2024-04-25T10:12:18.601897+0000","last_became_peered":"2024-04-25T10:12:18.601897+0000","last_unstale":"2024-04-25T10:12:19.599502+0000","last_undegraded":"2024-04-25T10:12:19.599502+0000","last_fullsized":"2024-04-25T10:12:19.599502+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:56.597024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701616+0000","last_change":"2024-04-25T10:12:17.582230+0000","last_active":"2024-04-25T10:12:19.701616+0000","last_peered":"2024-04-25T10:12:19.701616+0000","last_clean":"2024-04-25T10:12:19.701616+0000","last_became_active":"2024-04-25T10:12:17.581529+0000","last_became_peered":"2024-04-25T10:12:17.581529+0000","last_unstale":"2024-04-25T10:12:19.701616+0000","last_undegraded":"2024-04-25T10:12:19.701616+0000","last_fullsized":"2024-04-25T10:12:19.701616+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.084700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.702906+0000","last_change":"2024-04-25T10:12:18.616992+0000","last_active":"2024-04-25T10:12:19.702906+0000","last_peered":"2024-04-25T10:12:19.702906+0000","last_clean":"2024-04-25T10:12:19.702906+0000","last_became_active":"2024-04-25T10:12:18.616154+0000","last_became_peered":"2024-04-25T10:12:18.616154+0000","last_unstale":"2024-04-25T10:12:19.702906+0000","last_undegraded":"2024-04-25T10:12:19.702906+0000","last_fullsized":"2024-04-25T10:12:19.702906+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:17.540425+0000","last_clean_scrub_stamp":"2024-04-25T10:12:17.540425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:11.015595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"119'8","reported_seq":22,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.700068+0000","last_change":"2024-04-25T10:12:17.569428+0000","last_active":"2024-04-25T10:12:19.700068+0000","last_peered":"2024-04-25T10:12:19.700068+0000","last_clean":"2024-04-25T10:12:19.700068+0000","last_became_active":"2024-04-25T10:12:17.569061+0000","last_became_peered":"2024-04-25T10:12:17.569061+0000","last_unstale":"2024-04-25T10:12:19.700068+0000","last_undegraded":"2024-04-25T10:12:19.700068+0000","last_fullsized":"2024-04-25T10:12:19.700068+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:16.529461+0000","last_clean_scrub_stamp":"2024-04-25T10:12:16.529461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:05.147159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":358,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:19.701791+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:12:19.701791+0000","last_peered":"2024-04-25T10:12:19.701791+0000","last_clean":"2024-04-25T10:12:19.701791+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:12:19.701791+0000","last_undegraded":"2024-04-25T10:12:19.701791+0000","last_fullsized":"2024-04-25T10:12:19.701791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65160,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306680,"statfs":{"total":96636764160,"available":96570040320,"internally_reserved":0,"allocated":2093056,"data_stored":1282818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46524,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325316,"statfs":{"total":96636764160,"available":96589123584,"internally_reserved":0,"allocated":1486848,"data_stored":692256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59012,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312828,"statfs":{"total":96636764160,"available":96576335872,"internally_reserved":0,"allocated":1490944,"data_stored":691834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72908,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298932,"statfs":{"total":96636764160,"available":96562106368,"internally_reserved":0,"allocated":1507328,"data_stored":699978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72900,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298940,"statfs":{"total":96636764160,"available":96562114560,"internally_reserved":0,"allocated":1499136,"data_stored":693320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46520,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325320,"statfs":{"total":96636764160,"available":96589127680,"internally_reserved":0,"allocated":1482752,"data_stored":691674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59000,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312840,"statfs":{"total":96636764160,"available":96576348160,"internally_reserved":0,"allocated":1486848,"data_stored":691696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69308,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302532,"statfs":{"total":96636764160,"available":96565792768,"internally_reserved":0,"allocated":2080768,"data_stored":1282064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:21.935 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:12:21.935 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:12:22.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.105+0000 7f0e855b6640 1 Processor -- start 2024-04-25T10:12:22.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.105+0000 7f0e855b6640 1 -- start start 2024-04-25T10:12:22.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.105+0000 7f0e855b6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e8015d9a0 0x7f0e800d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.105+0000 7f0e855b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e800ce600 0x7f0e800cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.105+0000 7f0e855b6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e800d24c0 con 0x7f0e800d1920 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.105+0000 7f0e855b6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e800cefa0 con 0x7f0e8015d9a0 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.105+0000 7f0e855b6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8015ebd0 con 0x7f0e800ce600 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.105+0000 7f0e7f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0e800d1920 0x7f0e800d1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46710/0 (socket says 172.21.15.133:46710) 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7f7fe640 1 -- 172.21.15.133:0/1890984237 learned_addr learned my addr 172.21.15.133:0/1890984237 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7effd640 1 --2- 172.21.15.133:0/1890984237 >> v2:172.21.15.133:3300/0 conn(0x7f0e8015d9a0 0x7f0e800d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7dffb640 1 -- 172.21.15.133:0/1890984237 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156820322 0 0) 0x7f0e800d24c0 con 0x7f0e800d1920 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7dffb640 1 -- 172.21.15.133:0/1890984237 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e68003530 con 0x7f0e800d1920 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7e7fc640 1 --2- 172.21.15.133:0/1890984237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e800ce600 0x7f0e800cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7effd640 1 -- 172.21.15.133:0/1890984237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e800ce600 msgr2=0x7f0e800cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7effd640 1 --2- 172.21.15.133:0/1890984237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e800ce600 0x7f0e800cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7effd640 1 -- 172.21.15.133:0/1890984237 >> v1:172.21.15.133:6789/0 conn(0x7f0e800d1920 legacy=0x7f0e800d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7effd640 1 -- 172.21.15.133:0/1890984237 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8015eeb0 con 0x7f0e8015d9a0 2024-04-25T10:12:22.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7e7fc640 1 --2- 172.21.15.133:0/1890984237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e800ce600 0x7f0e800cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:22.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7effd640 1 --2- 172.21.15.133:0/1890984237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015d9a0 0x7f0e800d0b30 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0e700097c0 tx=0x7f0e7001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467c1436de370a26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7dffb640 1 -- 172.21.15.133:0/1890984237 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e700b1070 con 0x7f0e8015d9a0 2024-04-25T10:12:22.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7dffb640 1 -- 172.21.15.133:0/1890984237 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e700a8050 con 0x7f0e8015d9a0 2024-04-25T10:12:22.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e7dffb640 1 -- 172.21.15.133:0/1890984237 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e700ac860 con 0x7f0e8015d9a0 2024-04-25T10:12:22.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e855b6640 1 -- 172.21.15.133:0/1890984237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015d9a0 msgr2=0x7f0e800d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e855b6640 1 --2- 172.21.15.133:0/1890984237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015d9a0 0x7f0e800d0b30 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0e700097c0 tx=0x7f0e7001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e855b6640 1 -- 172.21.15.133:0/1890984237 shutdown_connections 2024-04-25T10:12:22.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e855b6640 1 --2- 172.21.15.133:0/1890984237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e800ce600 0x7f0e800cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e855b6640 1 --2- 172.21.15.133:0/1890984237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015d9a0 0x7f0e800d0b30 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e855b6640 1 -- 172.21.15.133:0/1890984237 >> 172.21.15.133:0/1890984237 conn(0x7f0e8006bf80 msgr2=0x7f0e8006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:22.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e855b6640 1 -- 172.21.15.133:0/1890984237 shutdown_connections 2024-04-25T10:12:22.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.109+0000 7f0e855b6640 1 -- 172.21.15.133:0/1890984237 wait complete. 2024-04-25T10:12:22.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e855b6640 1 Processor -- start 2024-04-25T10:12:22.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e855b6640 1 -- start start 2024-04-25T10:12:22.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e855b6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e800ce600 0x7f0e801567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e855b6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e800d1920 0x7f0e800b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e855b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e8015d9a0 0x7f0e8015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e855b6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800b96a0 con 0x7f0e800ce600 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e855b6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800cffd0 con 0x7f0e800d1920 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e855b6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800b9820 con 0x7f0e8015d9a0 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e800ce600 0x7f0e801567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e800ce600 0x7f0e801567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43874/0 (socket says 172.21.15.133:43874) 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e7effd640 1 -- 172.21.15.133:0/4017757638 learned_addr learned my addr 172.21.15.133:0/4017757638 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e7effd640 1 -- 172.21.15.133:0/4017757638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e8015d9a0 msgr2=0x7f0e8015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e7effd640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e8015d9a0 0x7f0e8015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e7effd640 1 -- 172.21.15.133:0/4017757638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e800d1920 msgr2=0x7f0e800b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e7effd640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e800d1920 0x7f0e800b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e7effd640 1 -- 172.21.15.133:0/4017757638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800b3010 con 0x7f0e800ce600 2024-04-25T10:12:22.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e7effd640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e800ce600 0x7f0e801567f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f0e700097c0 tx=0x7f0e7001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e677fe640 1 -- 172.21.15.133:0/4017757638 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e700b1040 con 0x7f0e800ce600 2024-04-25T10:12:22.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e677fe640 1 -- 172.21.15.133:0/4017757638 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e700bc070 con 0x7f0e800ce600 2024-04-25T10:12:22.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e855b6640 1 -- 172.21.15.133:0/4017757638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e800b32f0 con 0x7f0e800ce600 2024-04-25T10:12:22.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e677fe640 1 -- 172.21.15.133:0/4017757638 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e700b7690 con 0x7f0e800ce600 2024-04-25T10:12:22.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e855b6640 1 -- 172.21.15.133:0/4017757638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e800b38b0 con 0x7f0e800ce600 2024-04-25T10:12:22.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e855b6640 1 -- 172.21.15.133:0/4017757638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e80068e80 con 0x7f0e800ce600 2024-04-25T10:12:22.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.113+0000 7f0e677fe640 1 -- 172.21.15.133:0/4017757638 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e700b7830 con 0x7f0e800ce600 2024-04-25T10:12:22.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.117+0000 7f0e677fe640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e6c066f90 0x7f0e6c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.117+0000 7f0e677fe640 1 -- 172.21.15.133:0/4017757638 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e7011f470 con 0x7f0e800ce600 2024-04-25T10:12:22.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.117+0000 7f0e7e7fc640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e6c066f90 0x7f0e6c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.117+0000 7f0e7e7fc640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e6c066f90 0x7f0e6c069450 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0e800b80b0 tx=0x7f0e74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.121+0000 7f0e677fe640 1 -- 172.21.15.133:0/4017757638 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e700c6020 con 0x7f0e800ce600 2024-04-25T10:12:22.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.337+0000 7f0e855b6640 1 -- 172.21.15.133:0/4017757638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0e800b5f00 con 0x7f0e800ce600 2024-04-25T10:12:22.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e677fe640 1 -- 172.21.15.133:0/4017757638 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0e70016090 con 0x7f0e800ce600 2024-04-25T10:12:22.343 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:12:22.343 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:12:22.343 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:12:22.343 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:12:22.343 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:12:22.343 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:12:22.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 -- 172.21.15.133:0/4017757638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e6c066f90 msgr2=0x7f0e6c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e6c066f90 0x7f0e6c069450 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0e800b80b0 tx=0x7f0e74008040 comp rx=0 tx=0).stop 2024-04-25T10:12:22.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 -- 172.21.15.133:0/4017757638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e800ce600 msgr2=0x7f0e801567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e800ce600 0x7f0e801567f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f0e700097c0 tx=0x7f0e7001ac20 comp rx=0 tx=0).stop 2024-04-25T10:12:22.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 -- 172.21.15.133:0/4017757638 shutdown_connections 2024-04-25T10:12:22.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e8015d9a0 0x7f0e8015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e800d1920 0x7f0e800b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e6c066f90 0x7f0e6c069450 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 --2- 172.21.15.133:0/4017757638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e800ce600 0x7f0e801567f0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 -- 172.21.15.133:0/4017757638 >> 172.21.15.133:0/4017757638 conn(0x7f0e8006bf80 msgr2=0x7f0e80167bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:22.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 -- 172.21.15.133:0/4017757638 shutdown_connections 2024-04-25T10:12:22.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.341+0000 7f0e855b6640 1 -- 172.21.15.133:0/4017757638 wait complete. 2024-04-25T10:12:22.365 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.477+0000 7f76aba19640 1 Processor -- start 2024-04-25T10:12:22.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.477+0000 7f76aba19640 1 -- start start 2024-04-25T10:12:22.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76aba19640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f76a415ef50 0x7f76a409a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76aba19640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40cc2c0 con 0x7f76a40d02a0 2024-04-25T10:12:22.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76aba19640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40cc560 con 0x7f76a40cf310 2024-04-25T10:12:22.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76aba19640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a40cc840 con 0x7f76a415ef50 2024-04-25T10:12:22.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a9fb6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f76a40d02a0 0x7f76a40c5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46714/0 (socket says 172.21.15.133:46714) 2024-04-25T10:12:22.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a9fb6640 1 -- 172.21.15.133:0/1141815113 learned_addr learned my addr 172.21.15.133:0/1141815113 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a97b5640 1 --2- 172.21.15.133:0/1141815113 >> v2:172.21.15.162:3300/0 conn(0x7f76a415ef50 0x7f76a409a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923801964 0 0) 0x7f76a40cc2c0 con 0x7f76a40d02a0 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7690003530 con 0x7f76a40d02a0 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 556574055 0 0) 0x7f7690003530 con 0x7f76a40d02a0 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 >> v2:172.21.15.162:3300/0 conn(0x7f76a415ef50 msgr2=0x7f76a409a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 --2- 172.21.15.133:0/1141815113 >> v2:172.21.15.162:3300/0 conn(0x7f76a415ef50 0x7f76a409a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 >> v1:172.21.15.162:6789/0 conn(0x7f76a40cf310 legacy=0x7f76a40cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a40ccb20 con 0x7f76a40d02a0 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f769c001a00 con 0x7f76a40d02a0 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 >> v1:172.21.15.133:6789/0 conn(0x7f76a40d02a0 legacy=0x7f76a40c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 --2- 172.21.15.133:0/1141815113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76900039c0 0x7f7690003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 --2- 172.21.15.133:0/1141815113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7690004810 0x7f7690004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 --2- 172.21.15.133:0/1141815113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76900052f0 0x7f769000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40cc840 con 0x7f76900039c0 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769000c1c0 con 0x7f76900052f0 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769000c3c0 con 0x7f7690004810 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a97b5640 1 --2- 172.21.15.133:0/1141815113 >> v2:172.21.15.162:3300/0 conn(0x7f76a415ef50 0x7f76a409a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a97b5640 1 --2- 172.21.15.133:0/1141815113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76900039c0 0x7f7690003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a9fb6640 1 --2- 172.21.15.133:0/1141815113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76900052f0 0x7f769000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a8fb4640 1 --2- 172.21.15.133:0/1141815113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7690004810 0x7f7690004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a97b5640 1 -- 172.21.15.133:0/1141815113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7690004810 msgr2=0x7f7690004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a97b5640 1 --2- 172.21.15.133:0/1141815113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7690004810 0x7f7690004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a97b5640 1 -- 172.21.15.133:0/1141815113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76900052f0 msgr2=0x7f769000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a97b5640 1 --2- 172.21.15.133:0/1141815113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76900052f0 0x7f769000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a9fb6640 1 -- 172.21.15.133:0/1141815113 reap_dead start 2024-04-25T10:12:22.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.481+0000 7f76a97b5640 1 -- 172.21.15.133:0/1141815113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690003530 con 0x7f76900039c0 2024-04-25T10:12:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a97b5640 1 --2- 172.21.15.133:0/1141815113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76900039c0 0x7f7690003de0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f7694009880 tx=0x7f769401c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d6c38840272e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769400f040 con 0x7f76900039c0 2024-04-25T10:12:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7694004020 con 0x7f76900039c0 2024-04-25T10:12:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f769bfff640 1 -- 172.21.15.133:0/1141815113 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76940043f0 con 0x7f76900039c0 2024-04-25T10:12:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 -- 172.21.15.133:0/1141815113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76900039c0 msgr2=0x7f7690003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 --2- 172.21.15.133:0/1141815113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76900039c0 0x7f7690003de0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f7694009880 tx=0x7f769401c570 comp rx=0 tx=0).stop 2024-04-25T10:12:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 -- 172.21.15.133:0/1141815113 shutdown_connections 2024-04-25T10:12:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 --2- 172.21.15.133:0/1141815113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76900039c0 0x7f7690003de0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 -- 172.21.15.133:0/1141815113 >> 172.21.15.133:0/1141815113 conn(0x7f76a40b3230 msgr2=0x7f76a40b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 -- 172.21.15.133:0/1141815113 shutdown_connections 2024-04-25T10:12:22.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 -- 172.21.15.133:0/1141815113 wait complete. 2024-04-25T10:12:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 Processor -- start 2024-04-25T10:12:22.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 -- start start 2024-04-25T10:12:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76900039c0 0x7f76a4157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a40b5f70 0x7f76a40b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a40b48a0 0x7f76a40b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40cd3d0 con 0x7f76a40b48a0 2024-04-25T10:12:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40cd250 con 0x7f76900039c0 2024-04-25T10:12:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40c5aa0 con 0x7f76a40b5f70 2024-04-25T10:12:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a9fb6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a40b48a0 0x7f76a40b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a9fb6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a40b48a0 0x7f76a40b67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43882/0 (socket says 172.21.15.133:43882) 2024-04-25T10:12:22.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a9fb6640 1 -- 172.21.15.133:0/1791672739 learned_addr learned my addr 172.21.15.133:0/1791672739 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a8fb4640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a40b5f70 0x7f76a40b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a97b5640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76900039c0 0x7f76a4157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a9fb6640 1 -- 172.21.15.133:0/1791672739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a40b5f70 msgr2=0x7f76a40b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a9fb6640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a40b5f70 0x7f76a40b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a9fb6640 1 -- 172.21.15.133:0/1791672739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76900039c0 msgr2=0x7f76a4157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a9fb6640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76900039c0 0x7f76a4157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a9fb6640 1 -- 172.21.15.133:0/1791672739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a4158180 con 0x7f76a40b48a0 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a97b5640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76900039c0 0x7f76a4157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76a9fb6640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a40b48a0 0x7f76a40b67c0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f769c007730 tx=0x7f769c007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f7699ffb640 1 -- 172.21.15.133:0/1791672739 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00eb10 con 0x7f76a40b48a0 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f7699ffb640 1 -- 172.21.15.133:0/1791672739 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c00ecd0 con 0x7f76a40b48a0 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 -- 172.21.15.133:0/1791672739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a40b4ed0 con 0x7f76a40b48a0 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f7699ffb640 1 -- 172.21.15.133:0/1791672739 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c014670 con 0x7f76a40b48a0 2024-04-25T10:12:22.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.485+0000 7f76aba19640 1 -- 172.21.15.133:0/1791672739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a40b5490 con 0x7f76a40b48a0 2024-04-25T10:12:22.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.489+0000 7f76aba19640 1 -- 172.21.15.133:0/1791672739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a4068ef0 con 0x7f76a40b48a0 2024-04-25T10:12:22.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.489+0000 7f7699ffb640 1 -- 172.21.15.133:0/1791672739 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f769c014810 con 0x7f76a40b48a0 2024-04-25T10:12:22.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.489+0000 7f7699ffb640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f90 0x7f7688069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.489+0000 7f76a97b5640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f90 0x7f7688069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.489+0000 7f7699ffb640 1 -- 172.21.15.133:0/1791672739 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f769c084e70 con 0x7f76a40b48a0 2024-04-25T10:12:22.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.489+0000 7f76a97b5640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f90 0x7f7688069450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f76940ad000 tx=0x7f7694004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.493+0000 7f7699ffb640 1 -- 172.21.15.133:0/1791672739 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f769c055ce0 con 0x7f76a40b48a0 2024-04-25T10:12:22.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.685+0000 7f76aba19640 1 -- 172.21.15.133:0/1791672739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76a415c3b0 con 0x7f76a40b48a0 2024-04-25T10:12:22.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.685+0000 7f7699ffb640 1 -- 172.21.15.133:0/1791672739 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f769c0555e0 con 0x7f76a40b48a0 2024-04-25T10:12:22.690 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:22.690 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:22.690 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:22.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 -- 172.21.15.133:0/1791672739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f90 msgr2=0x7f7688069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f90 0x7f7688069450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f76940ad000 tx=0x7f7694004740 comp rx=0 tx=0).stop 2024-04-25T10:12:22.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 -- 172.21.15.133:0/1791672739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a40b48a0 msgr2=0x7f76a40b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a40b48a0 0x7f76a40b67c0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f769c007730 tx=0x7f769c007bf0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 -- 172.21.15.133:0/1791672739 shutdown_connections 2024-04-25T10:12:22.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f90 0x7f7688069450 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a40b48a0 0x7f76a40b67c0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a40b5f70 0x7f76a40b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 --2- 172.21.15.133:0/1791672739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76900039c0 0x7f76a4157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 -- 172.21.15.133:0/1791672739 >> 172.21.15.133:0/1791672739 conn(0x7f76a40b3230 msgr2=0x7f76a415e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:22.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 -- 172.21.15.133:0/1791672739 shutdown_connections 2024-04-25T10:12:22.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:22.689+0000 7f76aba19640 1 -- 172.21.15.133:0/1791672739 wait complete. 2024-04-25T10:12:22.856 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:22.859 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:23.111 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:17.925393+0000', 'modified': '2024-04-25T10:12:19.956686+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8919}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8919': {'gid': 8919, 'name': 'a', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/4218771669', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 4218771669}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 4218771669}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:12:23.111 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:12:23.112 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:12:23.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.237+0000 7fe86285a640 1 Processor -- start 2024-04-25T10:12:23.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.237+0000 7fe86285a640 1 -- start start 2024-04-25T10:12:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.237+0000 7fe86285a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0ce600 0x7fe85c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.237+0000 7fe86285a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe85c0d1920 0x7fe85c0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.237+0000 7fe86285a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85c15ebd0 con 0x7fe85c15d9a0 2024-04-25T10:12:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.237+0000 7fe86285a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85c0cefa0 con 0x7fe85c0d1920 2024-04-25T10:12:23.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.237+0000 7fe86285a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c15eeb0 con 0x7fe85c0ce600 2024-04-25T10:12:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.237+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0ce600 0x7fe85c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.237+0000 7fe860df7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe85c0d1920 0x7fe85c0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.237+0000 7fe85bfff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe85c15d9a0 0x7fe85c0d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:44170/0 (socket says 172.21.15.133:44170) 2024-04-25T10:12:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.237+0000 7fe85bfff640 1 -- 172.21.15.133:0/1025045752 learned_addr learned my addr 172.21.15.133:0/1025045752 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe860df7640 1 -- 172.21.15.133:0/1025045752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0ce600 msgr2=0x7fe85c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe860df7640 1 --2- 172.21.15.133:0/1025045752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0ce600 0x7fe85c0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe860df7640 1 -- 172.21.15.133:0/1025045752 >> v1:172.21.15.162:6789/0 conn(0x7fe85c15d9a0 legacy=0x7fe85c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe860df7640 1 -- 172.21.15.133:0/1025045752 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c0d0330 con 0x7fe85c0d1920 2024-04-25T10:12:23.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe860df7640 1 --2- 172.21.15.133:0/1025045752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c0d1920 0x7fe85c0d1db0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe854009d90 tx=0x7fe85401c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e256d110da179a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe85affd640 1 -- 172.21.15.133:0/1025045752 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8540b2070 con 0x7fe85c0d1920 2024-04-25T10:12:23.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe85affd640 1 -- 172.21.15.133:0/1025045752 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8540a5990 con 0x7fe85c0d1920 2024-04-25T10:12:23.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe85affd640 1 -- 172.21.15.133:0/1025045752 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8540a5d60 con 0x7fe85c0d1920 2024-04-25T10:12:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe86285a640 1 -- 172.21.15.133:0/1025045752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c0d1920 msgr2=0x7fe85c0d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe86285a640 1 --2- 172.21.15.133:0/1025045752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c0d1920 0x7fe85c0d1db0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe854009d90 tx=0x7fe85401c8b0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe86285a640 1 -- 172.21.15.133:0/1025045752 shutdown_connections 2024-04-25T10:12:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe86285a640 1 --2- 172.21.15.133:0/1025045752 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0ce600 0x7fe85c0cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe86285a640 1 --2- 172.21.15.133:0/1025045752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c0d1920 0x7fe85c0d1db0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe86285a640 1 -- 172.21.15.133:0/1025045752 >> 172.21.15.133:0/1025045752 conn(0x7fe85c06bf80 msgr2=0x7fe85c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe86285a640 1 -- 172.21.15.133:0/1025045752 shutdown_connections 2024-04-25T10:12:23.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.241+0000 7fe86285a640 1 -- 172.21.15.133:0/1025045752 wait complete. 2024-04-25T10:12:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe86285a640 1 Processor -- start 2024-04-25T10:12:23.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe86285a640 1 -- start start 2024-04-25T10:12:23.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe86285a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0ce600 0x7fe85c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe86285a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c0d1920 0x7fe85c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe86285a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c15d9a0 0x7fe85c156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe86285a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0d2470 con 0x7fe85c0d1920 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe86285a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0d22f0 con 0x7fe85c15d9a0 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe86285a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c15e590 con 0x7fe85c0ce600 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c0d1920 0x7fe85c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c0d1920 0x7fe85c0b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43890/0 (socket says 172.21.15.133:43890) 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1989822947 learned_addr learned my addr 172.21.15.133:0/1989822947 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe860df7640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c15d9a0 0x7fe85c156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe85bfff640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0ce600 0x7fe85c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1989822947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0ce600 msgr2=0x7fe85c0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0ce600 0x7fe85c0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1989822947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c15d9a0 msgr2=0x7fe85c156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c15d9a0 0x7fe85c156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1989822947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c0b4e60 con 0x7fe85c0d1920 2024-04-25T10:12:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe860df7640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c15d9a0 0x7fe85c156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c0d1920 0x7fe85c0b3c90 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe85000e960 tx=0x7fe85000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe858ff9640 1 -- 172.21.15.133:0/1989822947 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850017070 con 0x7fe85c0d1920 2024-04-25T10:12:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe858ff9640 1 -- 172.21.15.133:0/1989822947 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85000ca00 con 0x7fe85c0d1920 2024-04-25T10:12:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe86285a640 1 -- 172.21.15.133:0/1989822947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c0b31b0 con 0x7fe85c0d1920 2024-04-25T10:12:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe858ff9640 1 -- 172.21.15.133:0/1989822947 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85000cdb0 con 0x7fe85c0d1920 2024-04-25T10:12:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.245+0000 7fe86285a640 1 -- 172.21.15.133:0/1989822947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c156890 con 0x7fe85c0d1920 2024-04-25T10:12:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe858ff9640 1 -- 172.21.15.133:0/1989822947 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe85001a070 con 0x7fe85c0d1920 2024-04-25T10:12:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe858ff9640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 0x7fe848069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe858ff9640 1 -- 172.21.15.133:0/1989822947 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe850085640 con 0x7fe85c0d1920 2024-04-25T10:12:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85bfff640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 0x7fe848069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85bfff640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 0x7fe848069450 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0cdf30 tx=0x7fe84c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe8659c7000 1 Processor -- start 2024-04-25T10:12:23.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe8659c7000 1 -- start start 2024-04-25T10:12:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d85920 0x55bc93d85d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d3bf30 0x55bc93d3c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe8659c7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc93c235f0 con 0x55bc93d86240 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe8659c7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc93b8ada0 con 0x55bc93d85920 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe8659c7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc93b13150 con 0x55bc93d3bf30 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43894/0 (socket says 172.21.15.133:43894) 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1108237836 learned_addr learned my addr 172.21.15.133:0/1108237836 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85bfff640 1 --2- 172.21.15.133:0/1108237836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d85920 0x55bc93d85d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1108237836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d3bf30 msgr2=0x55bc93d3c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/1108237836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d3bf30 0x55bc93d3c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1108237836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d85920 msgr2=0x55bc93d85d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/1108237836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d85920 0x55bc93d85d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1108237836 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc93b2cca0 con 0x55bc93d86240 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.249+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/1108237836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3b9f0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe8500097c0 tx=0x7fe85000efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9202e0f32df3c08e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe841ffb640 1 -- 172.21.15.133:0/1108237836 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850021070 con 0x55bc93d86240 2024-04-25T10:12:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe841ffb640 1 -- 172.21.15.133:0/1108237836 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85005c0c0 con 0x55bc93d86240 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe841ffb640 1 -- 172.21.15.133:0/1108237836 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850052390 con 0x55bc93d86240 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- 172.21.15.133:0/1108237836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 msgr2=0x55bc93d3b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 --2- 172.21.15.133:0/1108237836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3b9f0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe8500097c0 tx=0x7fe85000efd0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- 172.21.15.133:0/1108237836 shutdown_connections 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 --2- 172.21.15.133:0/1108237836 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d3bf30 0x55bc93d3c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 --2- 172.21.15.133:0/1108237836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3b9f0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 --2- 172.21.15.133:0/1108237836 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d85920 0x55bc93d85d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- 172.21.15.133:0/1108237836 >> 172.21.15.133:0/1108237836 conn(0x55bc93d422a0 msgr2=0x55bc93d3a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- 172.21.15.133:0/1108237836 shutdown_connections 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- 172.21.15.133:0/1108237836 wait complete. 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- start start 2024-04-25T10:12:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 Processor -- start 2024-04-25T10:12:23.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d3bf30 0x55bc93d8cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d85920 0x55bc93d8d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc93b45990 con 0x55bc93d86240 2024-04-25T10:12:23.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc93b86e40 con 0x55bc93d3bf30 2024-04-25T10:12:23.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc93b16900 con 0x55bc93d85920 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 10 client.0 _ll_register_callbacks cb 0x55bc93c3e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe860df7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe860df7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43910/0 (socket says 172.21.15.133:43910) 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe860df7640 1 -- 172.21.15.133:0/728122584 learned_addr learned my addr 172.21.15.133:0/728122584 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe85bfff640 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d3bf30 0x55bc93d8cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d85920 0x55bc93d8d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe860df7640 1 -- 172.21.15.133:0/728122584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d85920 msgr2=0x55bc93d8d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe860df7640 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d85920 0x55bc93d8d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe860df7640 1 -- 172.21.15.133:0/728122584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d3bf30 msgr2=0x55bc93d8cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe860df7640 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d3bf30 0x55bc93d8cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe860df7640 1 -- 172.21.15.133:0/728122584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc93b43630 con 0x55bc93d86240 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe860df7640 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3faa0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fe854009d90 tx=0x7fe854019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f331da02d04143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe83affd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:23.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe83affd640 1 -- 172.21.15.133:0/728122584 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8540bd070 con 0x55bc93d86240 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- 172.21.15.133:0/728122584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe85005c0c0 con 0x55bc93d86240 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe83affd640 1 -- 172.21.15.133:0/728122584 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8540b9460 con 0x55bc93d86240 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe83affd640 1 -- 172.21.15.133:0/728122584 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8540c8050 con 0x55bc93d86240 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 10 client.9057 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- 172.21.15.133:0/728122584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55bc93b133a0 con 0x55bc93d86240 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe83affd640 1 -- 172.21.15.133:0/728122584 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85005c0c0 con 0x55bc93d86240 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe83affd640 1 -- 172.21.15.133:0/728122584 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe8540cb020 con 0x55bc93d86240 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 10 client.9057 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 10 client.9057 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 20 client.9057 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55bc93d41160 0x55bc93d96ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 4 client.9057 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe8659c7000 1 -- 172.21.15.133:0/728122584 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc93b26a50 con 0x55bc93d41160 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.253+0000 7fe85bfff640 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55bc93d41160 0x55bc93d96ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe83affd640 10 client.9057 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe85bfff640 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55bc93d41160 0x55bc93d96ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=812c29156fcbef1c server_cookie=94eb66b2019b29c0 in_seq=0 out_seq=0 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe83affd640 1 -- 172.21.15.133:0/728122584 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc93b26a50 con 0x55bc93d41160 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe83affd640 10 client.9057 handle_command_reply: tid=0 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 1 client.9057 shutdown 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 1 -- 172.21.15.133:0/728122584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 msgr2=0x55bc93d3faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3faa0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fe854009d90 tx=0x7fe854019650 comp rx=0 tx=0).stop 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 1 -- 172.21.15.133:0/728122584 shutdown_connections 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55bc93d41160 0x55bc93d96ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d86240 0x55bc93d3faa0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe83affd640 0 client.9057 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d85920 0x55bc93d8d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 1 --2- 172.21.15.133:0/728122584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d3bf30 0x55bc93d8cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 1 -- 172.21.15.133:0/728122584 >> 172.21.15.133:0/728122584 conn(0x55bc93d422a0 msgr2=0x55bc93d3a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 1 -- 172.21.15.133:0/728122584 shutdown_connections 2024-04-25T10:12:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 1 -- 172.21.15.133:0/728122584 wait complete. 2024-04-25T10:12:23.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.257+0000 7fe8659c7000 20 client.9057 trim_cache size 0 max 16384 2024-04-25T10:12:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 Processor -- start 2024-04-25T10:12:23.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 -- start start 2024-04-25T10:12:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d3bf30 0x55bc93de13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d85920 0x55bc93de18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d86240 0x55bc93de1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c202210 con 0x55bc93d85920 2024-04-25T10:12:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c159400 con 0x55bc93d86240 2024-04-25T10:12:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c201c40 con 0x55bc93d3bf30 2024-04-25T10:12:23.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d85920 0x55bc93de18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe85b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d85920 0x55bc93de18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43916/0 (socket says 172.21.15.133:43916) 2024-04-25T10:12:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1842392710 learned_addr learned my addr 172.21.15.133:0/1842392710 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1842392710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d3bf30 msgr2=0x55bc93de13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/1842392710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d3bf30 0x55bc93de13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1842392710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d86240 msgr2=0x55bc93de1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/1842392710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d86240 0x55bc93de1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe85b7fe640 1 -- 172.21.15.133:0/1842392710 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc93d86070 con 0x55bc93d85920 2024-04-25T10:12:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/1842392710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d85920 0x55bc93de18e0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fe850055ee0 tx=0x7fe85005c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21911c7cf5869737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8417fa640 1 -- 172.21.15.133:0/1842392710 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850072070 con 0x55bc93d85920 2024-04-25T10:12:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8417fa640 1 -- 172.21.15.133:0/1842392710 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85005c0c0 con 0x55bc93d85920 2024-04-25T10:12:23.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8417fa640 1 -- 172.21.15.133:0/1842392710 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850055af0 con 0x55bc93d85920 2024-04-25T10:12:23.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 -- 172.21.15.133:0/1842392710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d85920 msgr2=0x55bc93de18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 --2- 172.21.15.133:0/1842392710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d85920 0x55bc93de18e0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fe850055ee0 tx=0x7fe85005c050 comp rx=0 tx=0).stop 2024-04-25T10:12:23.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 -- 172.21.15.133:0/1842392710 shutdown_connections 2024-04-25T10:12:23.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 --2- 172.21.15.133:0/1842392710 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d86240 0x55bc93de1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 --2- 172.21.15.133:0/1842392710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d85920 0x55bc93de18e0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 --2- 172.21.15.133:0/1842392710 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d3bf30 0x55bc93de13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 -- 172.21.15.133:0/1842392710 >> 172.21.15.133:0/1842392710 conn(0x55bc93d422a0 msgr2=0x55bc93d3f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 -- 172.21.15.133:0/1842392710 shutdown_connections 2024-04-25T10:12:23.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 -- 172.21.15.133:0/1842392710 wait complete. 2024-04-25T10:12:23.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 -- start start 2024-04-25T10:12:23.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.273+0000 7fe8659c7000 1 Processor -- start 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d3bf30 0x55bc93d92120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d85920 0x55bc93de6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d86240 0x55bc93de7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc93b8f3d0 con 0x55bc93d3bf30 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc93c46d50 con 0x55bc93d85920 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc93b25980 con 0x55bc93d86240 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 10 client.0 _ll_register_callbacks cb 0x7fe850009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe85bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d3bf30 0x55bc93d92120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe85bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d3bf30 0x55bc93d92120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43928/0 (socket says 172.21.15.133:43928) 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe85bfff640 1 -- 172.21.15.133:0/3636285095 learned_addr learned my addr 172.21.15.133:0/3636285095 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe85bfff640 1 -- 172.21.15.133:0/3636285095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d86240 msgr2=0x55bc93de7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe85bfff640 1 --2- 172.21.15.133:0/3636285095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d86240 0x55bc93de7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe85bfff640 1 -- 172.21.15.133:0/3636285095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d85920 msgr2=0x55bc93de6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe85bfff640 1 --2- 172.21.15.133:0/3636285095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d85920 0x55bc93de6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe85bfff640 1 -- 172.21.15.133:0/3636285095 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc93c0c600 con 0x55bc93d3bf30 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe85bfff640 1 --2- 172.21.15.133:0/3636285095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d3bf30 0x55bc93d92120 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x55bc93b94b00 tx=0x7fe84c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465a8bfa2bceb307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe83affd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe83affd640 1 -- 172.21.15.133:0/3636285095 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0253d0 con 0x55bc93d3bf30 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe83affd640 1 -- 172.21.15.133:0/3636285095 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe84c010040 con 0x55bc93d3bf30 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 -- 172.21.15.133:0/3636285095 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe85005c0c0 con 0x55bc93d3bf30 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe83affd640 1 -- 172.21.15.133:0/3636285095 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe84c025570 con 0x55bc93d3bf30 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 10 client.9063 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 -- 172.21.15.133:0/3636285095 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55bc93b1f9b0 con 0x55bc93d3bf30 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe83affd640 1 -- 172.21.15.133:0/3636285095 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85005c0c0 con 0x55bc93d3bf30 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe83affd640 1 -- 172.21.15.133:0/3636285095 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe84c025710 con 0x55bc93d3bf30 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 10 client.9063 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 10 client.9063 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 20 client.9063 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:23.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 --2- 172.21.15.133:0/3636285095 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55bc93de7850 0x55bc93da0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 4 client.9063 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 -- 172.21.15.133:0/3636285095 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bc93b982e0 con 0x55bc93de7850 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/3636285095 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55bc93de7850 0x55bc93da0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe83affd640 10 client.9063 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe85b7fe640 1 --2- 172.21.15.133:0/3636285095 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55bc93de7850 0x55bc93da0760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa38f882fba6b1ab server_cookie=50d0c085aedd5095 in_seq=0 out_seq=0 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe83affd640 1 -- 172.21.15.133:0/3636285095 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55bc93b982e0 con 0x55bc93de7850 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe83affd640 10 client.9063 handle_command_reply: tid=0 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 client.9063 shutdown 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 -- 172.21.15.133:0/3636285095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d3bf30 msgr2=0x55bc93d92120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 --2- 172.21.15.133:0/3636285095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d3bf30 0x55bc93d92120 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x55bc93b94b00 tx=0x7fe84c000bd0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 -- 172.21.15.133:0/3636285095 shutdown_connections 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 --2- 172.21.15.133:0/3636285095 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55bc93de7850 0x55bc93da0760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 --2- 172.21.15.133:0/3636285095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55bc93d86240 0x55bc93de7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe83affd640 0 client.9063 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:23.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 --2- 172.21.15.133:0/3636285095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc93d85920 0x55bc93de6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 --2- 172.21.15.133:0/3636285095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc93d3bf30 0x55bc93d92120 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 -- 172.21.15.133:0/3636285095 >> 172.21.15.133:0/3636285095 conn(0x55bc93d422a0 msgr2=0x55bc93d8d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 -- 172.21.15.133:0/3636285095 shutdown_connections 2024-04-25T10:12:23.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 1 -- 172.21.15.133:0/3636285095 wait complete. 2024-04-25T10:12:23.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe8659c7000 20 client.9063 trim_cache size 0 max 16384 2024-04-25T10:12:23.286 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:12:23.286 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:12:23.286 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:12:23.286 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 8919, 2024-04-25T10:12:23.287 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:12:23.287 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:12:23.287 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:12:23.287 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 4.6722761149999998, 2024-04-25T10:12:23.287 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 72, 2024-04-25T10:12:23.287 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 120, 2024-04-25T10:12:23.287 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 120, 2024-04-25T10:12:23.287 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 9.7897995249999994 2024-04-25T10:12:23.287 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:12:23.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 -- 172.21.15.133:0/1989822947 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 msgr2=0x7fe848069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 0x7fe848069450 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0cdf30 tx=0x7fe84c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:23.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 -- 172.21.15.133:0/1989822947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c0d1920 msgr2=0x7fe85c0b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c0d1920 0x7fe85c0b3c90 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe85000e960 tx=0x7fe85000ee20 comp rx=0 tx=0).stop 2024-04-25T10:12:23.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 -- 172.21.15.133:0/1989822947 shutdown_connections 2024-04-25T10:12:23.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe85c15d9a0 0x7fe85c156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe848066f90 0x7fe848069450 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe85c0d1920 0x7fe85c0b3c90 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 --2- 172.21.15.133:0/1989822947 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe85c0ce600 0x7fe85c0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 -- 172.21.15.133:0/1989822947 >> 172.21.15.133:0/1989822947 conn(0x7fe85c06bf80 msgr2=0x7fe85c0b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 -- 172.21.15.133:0/1989822947 shutdown_connections 2024-04-25T10:12:23.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.277+0000 7fe86285a640 1 -- 172.21.15.133:0/1989822947 wait complete. 2024-04-25T10:12:23.298 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:12:23.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f3727640 1 Processor -- start 2024-04-25T10:12:23.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f3727640 1 -- start start 2024-04-25T10:12:23.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f3727640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0b1f70 0x7fa9ec06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f3727640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ec0c61d0 con 0x7fa9ec0ce330 2024-04-25T10:12:23.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f3727640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ec0c64b0 con 0x7fa9ec15c9a0 2024-04-25T10:12:23.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f3727640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec06d4f0 con 0x7fa9ec0b1f70 2024-04-25T10:12:23.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f1cc4640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa9ec0ce330 0x7fa9ec15d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46716/0 (socket says 172.21.15.133:46716) 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f1cc4640 1 -- 172.21.15.133:0/2709062380 learned_addr learned my addr 172.21.15.133:0/2709062380 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f0cc2640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0b1f70 0x7fa9ec06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161486625 0 0) 0x7fa9ec0c61d0 con 0x7fa9ec0ce330 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d8003530 con 0x7fa9ec0ce330 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3640684533 0 0) 0x7fa9d8003530 con 0x7fa9ec0ce330 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0b1f70 msgr2=0x7fa9ec06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0b1f70 0x7fa9ec06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 >> v1:172.21.15.162:6789/0 conn(0x7fa9ec15c9a0 legacy=0x7fa9ec0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec06d7d0 con 0x7fa9ec0ce330 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa9e4001a00 con 0x7fa9ec0ce330 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 >> v1:172.21.15.133:6789/0 conn(0x7fa9ec0ce330 legacy=0x7fa9ec15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d80039f0 0x7fa9d8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0b1f70 0x7fa9d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d8004d50 0x7fa9d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec06d4f0 con 0x7fa9d8004d50 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d800ba80 con 0x7fa9d80039f0 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d800bc80 con 0x7fa9ec0b1f70 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f1cc4640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d8004d50 0x7fa9d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f14c3640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d80039f0 0x7fa9d8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f0cc2640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0b1f70 0x7fa9d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f1cc4640 1 -- 172.21.15.133:0/2709062380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0b1f70 msgr2=0x7fa9d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f1cc4640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec0b1f70 0x7fa9d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f1cc4640 1 -- 172.21.15.133:0/2709062380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d80039f0 msgr2=0x7fa9d8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f1cc4640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d80039f0 0x7fa9d8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.409+0000 7fa9f1cc4640 1 -- 172.21.15.133:0/2709062380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d8003530 con 0x7fa9d8004d50 2024-04-25T10:12:23.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9f1cc4640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d8004d50 0x7fa9d80051d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400a830 tx=0x7fa9e401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf77706c1761af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e400a480 con 0x7fa9d8004d50 2024-04-25T10:12:23.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e40ad630 con 0x7fa9d8004d50 2024-04-25T10:12:23.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9e3fff640 1 -- 172.21.15.133:0/2709062380 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40ada60 con 0x7fa9d8004d50 2024-04-25T10:12:23.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9f3727640 1 -- 172.21.15.133:0/2709062380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d8004d50 msgr2=0x7fa9d80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9f3727640 1 --2- 172.21.15.133:0/2709062380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9d8004d50 0x7fa9d80051d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400a830 tx=0x7fa9e401d360 comp rx=0 tx=0).stop 2024-04-25T10:12:23.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9f1cc4640 1 -- 172.21.15.133:0/2709062380 reap_dead start 2024-04-25T10:12:23.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9f3727640 1 -- 172.21.15.133:0/2709062380 shutdown_connections 2024-04-25T10:12:23.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9f3727640 1 -- 172.21.15.133:0/2709062380 >> 172.21.15.133:0/2709062380 conn(0x7fa9ec067000 msgr2=0x7fa9ec06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9f3727640 1 -- 172.21.15.133:0/2709062380 shutdown_connections 2024-04-25T10:12:23.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9f3727640 1 -- 172.21.15.133:0/2709062380 wait complete. 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9f3727640 1 Processor -- start 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.413+0000 7fa9f3727640 1 -- start start 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f3727640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d8004d50 0x7fa9ec0a8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f3727640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec156540 0x7fa9ec0a8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f3727640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec158470 0x7fa9ec1f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f3727640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec15d9d0 con 0x7fa9ec156540 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f3727640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec15d850 con 0x7fa9d8004d50 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f3727640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec15db50 con 0x7fa9ec158470 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f0cc2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec156540 0x7fa9ec0a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f0cc2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec156540 0x7fa9ec0a8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43946/0 (socket says 172.21.15.133:43946) 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f0cc2640 1 -- 172.21.15.133:0/2811907746 learned_addr learned my addr 172.21.15.133:0/2811907746 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f14c3640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d8004d50 0x7fa9ec0a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f1cc4640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec158470 0x7fa9ec1f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f0cc2640 1 -- 172.21.15.133:0/2811907746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec158470 msgr2=0x7fa9ec1f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f0cc2640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec158470 0x7fa9ec1f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f0cc2640 1 -- 172.21.15.133:0/2811907746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d8004d50 msgr2=0x7fa9ec0a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f0cc2640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d8004d50 0x7fa9ec0a8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f0cc2640 1 -- 172.21.15.133:0/2811907746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec154390 con 0x7fa9ec156540 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f0cc2640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec156540 0x7fa9ec0a8970 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8011610 tx=0x7fa9e8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9e1ffb640 1 -- 172.21.15.133:0/2811907746 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800f8f0 con 0x7fa9ec156540 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9e1ffb640 1 -- 172.21.15.133:0/2811907746 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e800fa90 con 0x7fa9ec156540 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f3727640 1 -- 172.21.15.133:0/2811907746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ec154670 con 0x7fa9ec156540 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9e1ffb640 1 -- 172.21.15.133:0/2811907746 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800fe40 con 0x7fa9ec156540 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f3727640 1 -- 172.21.15.133:0/2811907746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ec154c30 con 0x7fa9ec156540 2024-04-25T10:12:23.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f3727640 1 -- 172.21.15.133:0/2811907746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ec068ef0 con 0x7fa9ec156540 2024-04-25T10:12:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9e1ffb640 1 -- 172.21.15.133:0/2811907746 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9e8015460 con 0x7fa9ec156540 2024-04-25T10:12:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9e1ffb640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9d0066f60 0x7fa9d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f14c3640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9d0066f60 0x7fa9d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9e1ffb640 1 -- 172.21.15.133:0/2811907746 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa9e8085ab0 con 0x7fa9ec156540 2024-04-25T10:12:23.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.417+0000 7fa9f14c3640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9d0066f60 0x7fa9d0069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc005510 tx=0x7fa9dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.421+0000 7fa9e1ffb640 1 -- 172.21.15.133:0/2811907746 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9e8019030 con 0x7fa9ec156540 2024-04-25T10:12:23.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.613+0000 7fa9f3727640 1 -- 172.21.15.133:0/2811907746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa9ec15e820 con 0x7fa9ec156540 2024-04-25T10:12:23.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.613+0000 7fa9e1ffb640 1 -- 172.21.15.133:0/2811907746 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa9e80569c0 con 0x7fa9ec156540 2024-04-25T10:12:23.617 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:12:23.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 -- 172.21.15.133:0/2811907746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9d0066f60 msgr2=0x7fa9d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9d0066f60 0x7fa9d0069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc005510 tx=0x7fa9dc009040 comp rx=0 tx=0).stop 2024-04-25T10:12:23.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 -- 172.21.15.133:0/2811907746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec156540 msgr2=0x7fa9ec0a8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec156540 0x7fa9ec0a8970 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8011610 tx=0x7fa9e8011ad0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 -- 172.21.15.133:0/2811907746 shutdown_connections 2024-04-25T10:12:23.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9ec158470 0x7fa9ec1f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa9d0066f60 0x7fa9d0069420 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9ec156540 0x7fa9ec0a8970 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 --2- 172.21.15.133:0/2811907746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9d8004d50 0x7fa9ec0a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 -- 172.21.15.133:0/2811907746 >> 172.21.15.133:0/2811907746 conn(0x7fa9ec067000 msgr2=0x7fa9ec0c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 -- 172.21.15.133:0/2811907746 shutdown_connections 2024-04-25T10:12:23.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:23.617+0000 7fa9f3727640 1 -- 172.21.15.133:0/2811907746 wait complete. 2024-04-25T10:12:23.634 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:12:23.634 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:12:23.634 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:12:23.634 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:12:23.634 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:23.635 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:12:23.635 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:12:23.635 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:12:23.635 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:23.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:23.635 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:12:23.641 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:23.641 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:23.641 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:23.641 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:23.641 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:23.641 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:23.641 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:23.641 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:23.641 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:12:23.641 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41888sec preferred_lft 41888sec 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:12:23.642 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:12:23.643 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:23.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:23.644 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:12:23.692 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:12:23.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:23.694 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:12:23.746 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:12:23.746 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:12:23.747 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:12:23.747 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:12:23.747 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:12:23.747 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:12:23.747 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:12:23.747 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:23.747 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:23.747 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:12:23.747 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:12:23.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:23.748 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:12:23.748 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:23.748 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:12:23.748 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:12:23.748 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:12:23.748 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:23.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:23.820 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:12:23.867 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:12:23.867 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:12:23.867 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:23.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:23.867 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:12:23.922 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:12:23.922 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:23.922 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:23.970 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:23.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:23.971 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:24.018 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:12:24.075 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:12:24.125 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:12:24.125 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:12:24.125 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:12:24.125 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:12:24.125 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:12:24.125 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:12:24.125 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:12:24.125 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:12:24.125 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:12:24.126 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:12:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:12:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:12:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:12:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:12:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:12:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:12:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:12:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:24.127 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:24.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:24.128 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:12:24.175 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:12:24.175 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:12:24.219 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:12:24.236 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:12:24.283 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:12:24.283 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:12:24.283 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:12:24.283 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:12:24.283 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:12:24.283 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:12:24.283 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:12:24.284 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:24.285 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:24.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:24.286 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:12:24.299 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:12:24.297+0000 7fd72971d580 -1 init, newargv = 0x558b9aab62f0 newargc=15 2024-04-25T10:12:24.299 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[78255]: starting ceph client 2024-04-25T10:12:24.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[78255]: starting fuse 2024-04-25T10:12:25.404 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:12:25.421 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:12:25.469 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:12:25.469 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:12:25.469 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:12:25.470 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:12:25.470 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:12:25.470 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:12:25.470 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:12:25.470 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:12:25.470 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:12:25.470 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:12:25.470 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:12:25.470 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:12:25.470 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:12:25.471 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:12:25.472 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:12:25.472 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:25.472 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:25.472 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:25.472 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:25.472 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:25.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:25.473 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:12:25.519 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:12:25.519 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:12:25.519 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:12:25.520 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:12:25.520 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:12:25.520 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:12:25.520 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:12:25.520 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:12:25.520 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:12:25.520 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:12:25.520 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:12:25.521 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:12:25.521 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:12:25.521 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:12:25.521 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:12:25.521 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:12:25.521 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:12:25.521 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:12:25.521 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:12:25.522 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:12:25.522 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:12:25.522 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:12:25.522 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:12:25.522 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:12:25.522 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:12:25.522 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:12:25.522 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:12:25.523 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:12:25.523 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:12:25.648 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.78255.asok 2024-04-25T10:12:25.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:12:25.659 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.78255.asok 2024-04-25T10:12:25.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:25.659 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.78255.asok status 2024-04-25T10:12:25.810 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:12:25.810 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:12:25.810 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:12:25.810 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:12:25.810 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:12:25.810 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:12:25.810 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:12:25.810 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "78255", 2024-04-25T10:12:25.810 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:12:25.810 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:25.810 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 9075, 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 9075 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2519477436 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:12:25.811 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:12:25.812 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2519477436 2024-04-25T10:12:25.812 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:12:25.812 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.9075 192.168.0.1:0/2519477436", 2024-04-25T10:12:25.812 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/2519477436", 2024-04-25T10:12:25.812 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:12:25.812 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 72, 2024-04-25T10:12:25.812 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 120, 2024-04-25T10:12:25.812 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:12:25.812 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:12:25.812 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:12:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:12:25.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:25.823 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:12:25.824 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:25.830 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:12:25.830 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:25.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:25.830 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:25.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:25.887 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:25.887 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:25.887 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:12:25.887 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:25.887 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:25.887 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:25.888 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:25.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:25.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:25.975 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175888 s, 59.6 MB/s 2024-04-25T10:12:25.989 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:25.990 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:25.990 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110556 s, 94.8 MB/s 2024-04-25T10:12:26.010 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.010 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.010 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016967 s, 61.8 MB/s 2024-04-25T10:12:26.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.022 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941055 s, 111 MB/s 2024-04-25T10:12:26.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.038 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125812 s, 83.3 MB/s 2024-04-25T10:12:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.054 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137711 s, 76.1 MB/s 2024-04-25T10:12:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.072 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146583 s, 71.5 MB/s 2024-04-25T10:12:26.089 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.090 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.090 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146503 s, 71.6 MB/s 2024-04-25T10:12:26.105 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.105 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.105 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127305 s, 82.4 MB/s 2024-04-25T10:12:26.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.118 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998151 s, 105 MB/s 2024-04-25T10:12:26.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.134 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124337 s, 84.3 MB/s 2024-04-25T10:12:26.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.152 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142689 s, 73.5 MB/s 2024-04-25T10:12:26.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.169 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142624 s, 73.5 MB/s 2024-04-25T10:12:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.187 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145864 s, 71.9 MB/s 2024-04-25T10:12:26.204 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.204 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.204 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137843 s, 76.1 MB/s 2024-04-25T10:12:26.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.219 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124233 s, 84.4 MB/s 2024-04-25T10:12:26.236 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.237 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.237 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143132 s, 73.3 MB/s 2024-04-25T10:12:26.252 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.252 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.252 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124595 s, 84.2 MB/s 2024-04-25T10:12:26.267 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.267 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.267 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125946 s, 83.3 MB/s 2024-04-25T10:12:26.280 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.280 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.280 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981734 s, 107 MB/s 2024-04-25T10:12:26.295 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.295 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.295 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126504 s, 82.9 MB/s 2024-04-25T10:12:26.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.311 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128455 s, 81.6 MB/s 2024-04-25T10:12:26.326 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.326 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.326 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125233 s, 83.7 MB/s 2024-04-25T10:12:26.342 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.343 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.343 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013653 s, 76.8 MB/s 2024-04-25T10:12:26.359 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.359 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.360 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136858 s, 76.6 MB/s 2024-04-25T10:12:26.375 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.375 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.375 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013271 s, 79.0 MB/s 2024-04-25T10:12:26.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.392 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143768 s, 72.9 MB/s 2024-04-25T10:12:26.408 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.408 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.408 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134796 s, 77.8 MB/s 2024-04-25T10:12:26.424 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.424 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.424 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129694 s, 80.8 MB/s 2024-04-25T10:12:26.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.438 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123469 s, 84.9 MB/s 2024-04-25T10:12:26.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.455 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122625 s, 85.5 MB/s 2024-04-25T10:12:26.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.471 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.471 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130749 s, 80.2 MB/s 2024-04-25T10:12:26.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:26.472 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:26.472 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:26.472 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:12:26.472 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:26.472 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:26.472 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:26.472 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:26.496 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.496 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.496 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116423 s, 90.1 MB/s 2024-04-25T10:12:26.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.513 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139875 s, 75.0 MB/s 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.527 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110911 s, 94.5 MB/s 2024-04-25T10:12:26.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.543 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139191 s, 75.3 MB/s 2024-04-25T10:12:26.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136056 s, 77.1 MB/s 2024-04-25T10:12:26.576 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.576 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.576 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128865 s, 81.4 MB/s 2024-04-25T10:12:26.589 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.589 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.589 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109506 s, 95.8 MB/s 2024-04-25T10:12:26.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121036 s, 86.6 MB/s 2024-04-25T10:12:26.621 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.621 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.621 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135621 s, 77.3 MB/s 2024-04-25T10:12:26.635 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.635 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.635 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114462 s, 91.6 MB/s 2024-04-25T10:12:26.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.653 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144905 s, 72.4 MB/s 2024-04-25T10:12:26.668 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.669 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125203 s, 83.7 MB/s 2024-04-25T10:12:26.682 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.682 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.682 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105843 s, 99.1 MB/s 2024-04-25T10:12:26.696 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.696 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.696 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118642 s, 88.4 MB/s 2024-04-25T10:12:26.710 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.710 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.711 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116815 s, 89.8 MB/s 2024-04-25T10:12:26.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.723 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990155 s, 106 MB/s 2024-04-25T10:12:26.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.734 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938413 s, 112 MB/s 2024-04-25T10:12:26.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.751 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137763 s, 76.1 MB/s 2024-04-25T10:12:26.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.764 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113134 s, 92.7 MB/s 2024-04-25T10:12:26.778 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.778 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.778 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112997 s, 92.8 MB/s 2024-04-25T10:12:26.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.792 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114658 s, 91.5 MB/s 2024-04-25T10:12:26.802 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.802 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.802 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894594 s, 117 MB/s 2024-04-25T10:12:26.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.816 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118335 s, 88.6 MB/s 2024-04-25T10:12:26.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.830 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119759 s, 87.6 MB/s 2024-04-25T10:12:26.843 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.844 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.844 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109042 s, 96.2 MB/s 2024-04-25T10:12:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120425 s, 87.1 MB/s 2024-04-25T10:12:26.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.873 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126748 s, 82.7 MB/s 2024-04-25T10:12:26.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.890 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013016 s, 80.6 MB/s 2024-04-25T10:12:26.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.907 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134553 s, 77.9 MB/s 2024-04-25T10:12:26.921 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.922 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119924 s, 87.4 MB/s 2024-04-25T10:12:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117422 s, 89.3 MB/s 2024-04-25T10:12:26.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:26.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:26.952 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012792 s, 82.0 MB/s 2024-04-25T10:12:26.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:26.953 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:26.953 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:26.953 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:12:26.953 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:26.953 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:26.953 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:26.953 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:27.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.020 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114737 s, 91.4 MB/s 2024-04-25T10:12:27.035 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.035 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.035 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126825 s, 82.7 MB/s 2024-04-25T10:12:27.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.049 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101239 s, 104 MB/s 2024-04-25T10:12:27.064 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.064 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.065 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122819 s, 85.4 MB/s 2024-04-25T10:12:27.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.078 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109131 s, 96.1 MB/s 2024-04-25T10:12:27.093 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.093 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.093 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012468 s, 84.1 MB/s 2024-04-25T10:12:27.109 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.109 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.109 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128321 s, 81.7 MB/s 2024-04-25T10:12:27.122 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.122 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.122 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106345 s, 98.6 MB/s 2024-04-25T10:12:27.136 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.136 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.136 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011693 s, 89.7 MB/s 2024-04-25T10:12:27.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113133 s, 92.7 MB/s 2024-04-25T10:12:27.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.166 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012592 s, 83.3 MB/s 2024-04-25T10:12:27.182 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.182 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.182 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132823 s, 78.9 MB/s 2024-04-25T10:12:27.195 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.196 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111027 s, 94.4 MB/s 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124788 s, 84.0 MB/s 2024-04-25T10:12:27.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.224 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102527 s, 102 MB/s 2024-04-25T10:12:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117886 s, 88.9 MB/s 2024-04-25T10:12:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.254 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126696 s, 82.8 MB/s 2024-04-25T10:12:27.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.270 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136075 s, 77.1 MB/s 2024-04-25T10:12:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.286 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134915 s, 77.7 MB/s 2024-04-25T10:12:27.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.304 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137383 s, 76.3 MB/s 2024-04-25T10:12:27.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.322 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148117 s, 70.8 MB/s 2024-04-25T10:12:27.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.339 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137949 s, 76.0 MB/s 2024-04-25T10:12:27.354 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.355 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.355 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012808 s, 81.9 MB/s 2024-04-25T10:12:27.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.371 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138641 s, 75.6 MB/s 2024-04-25T10:12:27.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.384 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992308 s, 106 MB/s 2024-04-25T10:12:27.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014599 s, 71.8 MB/s 2024-04-25T10:12:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135703 s, 77.3 MB/s 2024-04-25T10:12:27.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.435 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014111 s, 74.3 MB/s 2024-04-25T10:12:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117363 s, 89.3 MB/s 2024-04-25T10:12:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.464 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115552 s, 90.7 MB/s 2024-04-25T10:12:27.481 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.481 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.481 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138724 s, 75.6 MB/s 2024-04-25T10:12:27.496 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.497 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.497 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127169 s, 82.5 MB/s 2024-04-25T10:12:27.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:27.498 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:27.498 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:27.498 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:12:27.498 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:27.498 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:27.499 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:27.499 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:27.567 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.567 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.567 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128363 s, 81.7 MB/s 2024-04-25T10:12:27.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.584 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135889 s, 77.2 MB/s 2024-04-25T10:12:27.600 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.600 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.600 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135671 s, 77.3 MB/s 2024-04-25T10:12:27.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.616 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134561 s, 77.9 MB/s 2024-04-25T10:12:27.632 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.632 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.633 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013451 s, 78.0 MB/s 2024-04-25T10:12:27.647 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.647 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.647 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110955 s, 94.5 MB/s 2024-04-25T10:12:27.662 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.662 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.662 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121542 s, 86.3 MB/s 2024-04-25T10:12:27.676 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.676 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.676 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107448 s, 97.6 MB/s 2024-04-25T10:12:27.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.692 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137146 s, 76.5 MB/s 2024-04-25T10:12:27.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.707 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117255 s, 89.4 MB/s 2024-04-25T10:12:27.721 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.722 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.722 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120265 s, 87.2 MB/s 2024-04-25T10:12:27.736 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.736 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.736 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118735 s, 88.3 MB/s 2024-04-25T10:12:27.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.749 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.749 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963993 s, 109 MB/s 2024-04-25T10:12:27.759 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.760 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910779 s, 115 MB/s 2024-04-25T10:12:27.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.775 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127942 s, 82.0 MB/s 2024-04-25T10:12:27.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.790 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126084 s, 83.2 MB/s 2024-04-25T10:12:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110076 s, 95.3 MB/s 2024-04-25T10:12:27.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.817 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100085 s, 105 MB/s 2024-04-25T10:12:27.831 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.831 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.831 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123508 s, 84.9 MB/s 2024-04-25T10:12:27.846 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.846 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.846 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118746 s, 88.3 MB/s 2024-04-25T10:12:27.860 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.860 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.860 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122682 s, 85.5 MB/s 2024-04-25T10:12:27.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.873 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946478 s, 111 MB/s 2024-04-25T10:12:27.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.889 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135509 s, 77.4 MB/s 2024-04-25T10:12:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933379 s, 112 MB/s 2024-04-25T10:12:27.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.916 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123277 s, 85.1 MB/s 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938581 s, 112 MB/s 2024-04-25T10:12:27.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.942 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120875 s, 86.7 MB/s 2024-04-25T10:12:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.957 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118965 s, 88.1 MB/s 2024-04-25T10:12:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988075 s, 106 MB/s 2024-04-25T10:12:27.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:27.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:27.986 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139215 s, 75.3 MB/s 2024-04-25T10:12:28.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.000 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118399 s, 88.6 MB/s 2024-04-25T10:12:28.015 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.015 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.015 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128912 s, 81.3 MB/s 2024-04-25T10:12:28.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:28.016 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:28.017 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:28.017 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:12:28.017 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:28.017 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:28.017 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:28.017 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:28.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.084 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132528 s, 79.1 MB/s 2024-04-25T10:12:28.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.100 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.100 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013027 s, 80.5 MB/s 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.116 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131678 s, 79.6 MB/s 2024-04-25T10:12:28.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.131 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124495 s, 84.2 MB/s 2024-04-25T10:12:28.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.148 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140625 s, 74.6 MB/s 2024-04-25T10:12:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150144 s, 69.8 MB/s 2024-04-25T10:12:28.184 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.184 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.184 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142497 s, 73.6 MB/s 2024-04-25T10:12:28.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.201 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136055 s, 77.1 MB/s 2024-04-25T10:12:28.218 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148664 s, 70.5 MB/s 2024-04-25T10:12:28.234 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.234 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.234 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128356 s, 81.7 MB/s 2024-04-25T10:12:28.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.253 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157639 s, 66.5 MB/s 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117348 s, 89.4 MB/s 2024-04-25T10:12:28.283 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.283 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.283 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012226 s, 85.8 MB/s 2024-04-25T10:12:28.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.300 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139524 s, 75.2 MB/s 2024-04-25T10:12:28.315 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.316 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.316 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121557 s, 86.3 MB/s 2024-04-25T10:12:28.331 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.331 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.331 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128635 s, 81.5 MB/s 2024-04-25T10:12:28.346 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.346 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.346 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116257 s, 90.2 MB/s 2024-04-25T10:12:28.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.360 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112802 s, 93.0 MB/s 2024-04-25T10:12:28.373 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.373 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.373 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101551 s, 103 MB/s 2024-04-25T10:12:28.390 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.391 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145822 s, 71.9 MB/s 2024-04-25T10:12:28.405 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.405 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.405 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118793 s, 88.3 MB/s 2024-04-25T10:12:28.420 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.421 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125519 s, 83.5 MB/s 2024-04-25T10:12:28.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.436 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123445 s, 84.9 MB/s 2024-04-25T10:12:28.453 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.453 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.454 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146482 s, 71.6 MB/s 2024-04-25T10:12:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115219 s, 91.0 MB/s 2024-04-25T10:12:28.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.484 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130279 s, 80.5 MB/s 2024-04-25T10:12:28.497 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.497 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.497 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103338 s, 101 MB/s 2024-04-25T10:12:28.511 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.511 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.511 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111626 s, 93.9 MB/s 2024-04-25T10:12:28.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.527 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126788 s, 82.7 MB/s 2024-04-25T10:12:28.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.543 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130632 s, 80.3 MB/s 2024-04-25T10:12:28.556 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.556 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.556 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111312 s, 94.2 MB/s 2024-04-25T10:12:28.572 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.572 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.572 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131969 s, 79.5 MB/s 2024-04-25T10:12:28.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:28.573 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:28.573 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:28.573 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:12:28.573 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:28.573 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:28.573 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:28.573 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:28.635 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.636 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110093 s, 95.2 MB/s 2024-04-25T10:12:28.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.650 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116079 s, 90.3 MB/s 2024-04-25T10:12:28.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.665 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125797 s, 83.4 MB/s 2024-04-25T10:12:28.678 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.678 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.679 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111677 s, 93.9 MB/s 2024-04-25T10:12:28.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.692 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110969 s, 94.5 MB/s 2024-04-25T10:12:28.708 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.708 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.708 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135284 s, 77.5 MB/s 2024-04-25T10:12:28.724 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.724 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.724 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126615 s, 82.8 MB/s 2024-04-25T10:12:28.735 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.735 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.735 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889892 s, 118 MB/s 2024-04-25T10:12:28.750 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.750 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.750 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120576 s, 87.0 MB/s 2024-04-25T10:12:28.765 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.765 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.765 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125488 s, 83.6 MB/s 2024-04-25T10:12:28.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125563 s, 83.5 MB/s 2024-04-25T10:12:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125876 s, 83.3 MB/s 2024-04-25T10:12:28.811 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.811 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.811 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014092 s, 74.4 MB/s 2024-04-25T10:12:28.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.825 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112489 s, 93.2 MB/s 2024-04-25T10:12:28.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.841 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133227 s, 78.7 MB/s 2024-04-25T10:12:28.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.853 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933199 s, 112 MB/s 2024-04-25T10:12:28.868 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.868 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.868 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122854 s, 85.4 MB/s 2024-04-25T10:12:28.884 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.884 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.884 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132956 s, 78.9 MB/s 2024-04-25T10:12:28.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.900 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138946 s, 75.5 MB/s 2024-04-25T10:12:28.915 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.915 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.915 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122318 s, 85.7 MB/s 2024-04-25T10:12:28.929 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.929 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.929 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112961 s, 92.8 MB/s 2024-04-25T10:12:28.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.944 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125387 s, 83.6 MB/s 2024-04-25T10:12:28.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.960 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128263 s, 81.8 MB/s 2024-04-25T10:12:28.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.975 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129865 s, 80.7 MB/s 2024-04-25T10:12:28.990 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:28.990 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:28.990 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122646 s, 85.5 MB/s 2024-04-25T10:12:29.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.005 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123787 s, 84.7 MB/s 2024-04-25T10:12:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138074 s, 75.9 MB/s 2024-04-25T10:12:29.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.037 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126731 s, 82.7 MB/s 2024-04-25T10:12:29.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.054 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141665 s, 74.0 MB/s 2024-04-25T10:12:29.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.070 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129917 s, 80.7 MB/s 2024-04-25T10:12:29.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.086 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134058 s, 78.2 MB/s 2024-04-25T10:12:29.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.102 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013018 s, 80.5 MB/s 2024-04-25T10:12:29.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:29.103 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:29.103 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:29.103 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:12:29.103 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:29.103 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:29.103 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:29.103 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:29.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.174 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116058 s, 90.3 MB/s 2024-04-25T10:12:29.187 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.187 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.187 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105298 s, 99.6 MB/s 2024-04-25T10:12:29.200 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.200 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109793 s, 95.5 MB/s 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123485 s, 84.9 MB/s 2024-04-25T10:12:29.229 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.229 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.229 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113231 s, 92.6 MB/s 2024-04-25T10:12:29.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.245 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131013 s, 80.0 MB/s 2024-04-25T10:12:29.258 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.258 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.258 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107858 s, 97.2 MB/s 2024-04-25T10:12:29.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.273 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124731 s, 84.1 MB/s 2024-04-25T10:12:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.289 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137258 s, 76.4 MB/s 2024-04-25T10:12:29.305 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.305 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.305 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128862 s, 81.4 MB/s 2024-04-25T10:12:29.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.323 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141703 s, 74.0 MB/s 2024-04-25T10:12:29.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.340 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013702 s, 76.5 MB/s 2024-04-25T10:12:29.356 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.356 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.356 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133649 s, 78.5 MB/s 2024-04-25T10:12:29.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.372 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122559 s, 85.6 MB/s 2024-04-25T10:12:29.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.387 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.387 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121972 s, 86.0 MB/s 2024-04-25T10:12:29.404 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.404 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.404 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144094 s, 72.8 MB/s 2024-04-25T10:12:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139281 s, 75.3 MB/s 2024-04-25T10:12:29.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.438 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126977 s, 82.6 MB/s 2024-04-25T10:12:29.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.454 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133554 s, 78.5 MB/s 2024-04-25T10:12:29.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.470 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128414 s, 81.7 MB/s 2024-04-25T10:12:29.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.486 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133141 s, 78.8 MB/s 2024-04-25T10:12:29.503 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.503 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.503 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137319 s, 76.4 MB/s 2024-04-25T10:12:29.521 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.521 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.521 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142333 s, 73.7 MB/s 2024-04-25T10:12:29.537 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.538 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.538 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120811 s, 86.8 MB/s 2024-04-25T10:12:29.553 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.553 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.553 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012984 s, 80.8 MB/s 2024-04-25T10:12:29.571 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.571 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.571 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143364 s, 73.1 MB/s 2024-04-25T10:12:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.587 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012806 s, 81.9 MB/s 2024-04-25T10:12:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111075 s, 94.4 MB/s 2024-04-25T10:12:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122138 s, 85.9 MB/s 2024-04-25T10:12:29.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.634 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128235 s, 81.8 MB/s 2024-04-25T10:12:29.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.650 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125563 s, 83.5 MB/s 2024-04-25T10:12:29.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.667 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138203 s, 75.9 MB/s 2024-04-25T10:12:29.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:29.668 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:29.668 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:29.668 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:12:29.669 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:29.669 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:29.669 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:29.669 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126002 s, 83.2 MB/s 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126385 s, 83.0 MB/s 2024-04-25T10:12:29.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.769 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139431 s, 75.2 MB/s 2024-04-25T10:12:29.783 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.784 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.784 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113341 s, 92.5 MB/s 2024-04-25T10:12:29.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.800 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012671 s, 82.8 MB/s 2024-04-25T10:12:29.817 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.817 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.817 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133741 s, 78.4 MB/s 2024-04-25T10:12:29.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.835 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132679 s, 79.0 MB/s 2024-04-25T10:12:29.850 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.850 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.850 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101396 s, 103 MB/s 2024-04-25T10:12:29.864 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.864 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.864 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102966 s, 102 MB/s 2024-04-25T10:12:29.881 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.881 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.881 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136996 s, 76.5 MB/s 2024-04-25T10:12:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.896 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122323 s, 85.7 MB/s 2024-04-25T10:12:29.910 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.911 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114063 s, 91.9 MB/s 2024-04-25T10:12:29.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.927 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130614 s, 80.3 MB/s 2024-04-25T10:12:29.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.944 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131122 s, 80.0 MB/s 2024-04-25T10:12:29.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.960 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120473 s, 87.0 MB/s 2024-04-25T10:12:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.977 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.977 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136214 s, 77.0 MB/s 2024-04-25T10:12:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:29.992 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012299 s, 85.3 MB/s 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117744 s, 89.1 MB/s 2024-04-25T10:12:30.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.023 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011983 s, 87.5 MB/s 2024-04-25T10:12:30.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.039 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124547 s, 84.2 MB/s 2024-04-25T10:12:30.052 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.053 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104627 s, 100 MB/s 2024-04-25T10:12:30.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.068 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121732 s, 86.1 MB/s 2024-04-25T10:12:30.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.084 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131683 s, 79.6 MB/s 2024-04-25T10:12:30.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.100 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.100 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120625 s, 86.9 MB/s 2024-04-25T10:12:30.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.116 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125073 s, 83.8 MB/s 2024-04-25T10:12:30.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.132 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124557 s, 84.2 MB/s 2024-04-25T10:12:30.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137175 s, 76.4 MB/s 2024-04-25T10:12:30.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.165 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135515 s, 77.4 MB/s 2024-04-25T10:12:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123213 s, 85.1 MB/s 2024-04-25T10:12:30.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.196 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120375 s, 87.1 MB/s 2024-04-25T10:12:30.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.209 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.209 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975009 s, 108 MB/s 2024-04-25T10:12:30.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.225 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124099 s, 84.5 MB/s 2024-04-25T10:12:30.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:30.226 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:30.226 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:30.226 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:12:30.226 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:30.226 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:30.226 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:30.226 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:30.295 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.295 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.295 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115131 s, 91.1 MB/s 2024-04-25T10:12:30.312 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.312 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.312 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134719 s, 77.8 MB/s 2024-04-25T10:12:30.326 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.326 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.326 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117626 s, 89.1 MB/s 2024-04-25T10:12:30.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.341 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117994 s, 88.9 MB/s 2024-04-25T10:12:30.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.358 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.358 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128704 s, 81.5 MB/s 2024-04-25T10:12:30.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.371 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111367 s, 94.2 MB/s 2024-04-25T10:12:30.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.383 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981207 s, 107 MB/s 2024-04-25T10:12:30.399 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.399 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.399 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134529 s, 77.9 MB/s 2024-04-25T10:12:30.414 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.414 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.414 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117225 s, 89.5 MB/s 2024-04-25T10:12:30.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.427 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01002 s, 105 MB/s 2024-04-25T10:12:30.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.441 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113077 s, 92.7 MB/s 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135291 s, 77.5 MB/s 2024-04-25T10:12:30.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.469 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898631 s, 117 MB/s 2024-04-25T10:12:30.483 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.484 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122974 s, 85.3 MB/s 2024-04-25T10:12:30.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.499 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124547 s, 84.2 MB/s 2024-04-25T10:12:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117555 s, 89.2 MB/s 2024-04-25T10:12:30.526 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.526 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.526 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995597 s, 105 MB/s 2024-04-25T10:12:30.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.541 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127353 s, 82.3 MB/s 2024-04-25T10:12:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121428 s, 86.4 MB/s 2024-04-25T10:12:30.571 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.571 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.571 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125849 s, 83.3 MB/s 2024-04-25T10:12:30.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.583 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917773 s, 114 MB/s 2024-04-25T10:12:30.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.599 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136059 s, 77.1 MB/s 2024-04-25T10:12:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.615 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012444 s, 84.3 MB/s 2024-04-25T10:12:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.630 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120991 s, 86.7 MB/s 2024-04-25T10:12:30.645 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.645 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.645 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011777 s, 89.0 MB/s 2024-04-25T10:12:30.660 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.661 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.661 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127515 s, 82.2 MB/s 2024-04-25T10:12:30.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.675 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120211 s, 87.2 MB/s 2024-04-25T10:12:30.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.692 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134002 s, 78.3 MB/s 2024-04-25T10:12:30.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.707 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125245 s, 83.7 MB/s 2024-04-25T10:12:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.723 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129477 s, 81.0 MB/s 2024-04-25T10:12:30.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.739 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138262 s, 75.8 MB/s 2024-04-25T10:12:30.756 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.756 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.756 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135108 s, 77.6 MB/s 2024-04-25T10:12:30.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:30.757 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:30.757 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:30.757 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:12:30.757 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:30.757 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:30.757 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:30.758 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:30.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.826 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012405 s, 84.5 MB/s 2024-04-25T10:12:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118868 s, 88.2 MB/s 2024-04-25T10:12:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134268 s, 78.1 MB/s 2024-04-25T10:12:30.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.871 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010723 s, 97.8 MB/s 2024-04-25T10:12:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.885 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112412 s, 93.3 MB/s 2024-04-25T10:12:30.902 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.902 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.902 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135183 s, 77.6 MB/s 2024-04-25T10:12:30.915 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.915 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.915 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990348 s, 106 MB/s 2024-04-25T10:12:30.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.930 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127195 s, 82.4 MB/s 2024-04-25T10:12:30.943 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.943 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.943 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938395 s, 112 MB/s 2024-04-25T10:12:30.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.959 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133345 s, 78.6 MB/s 2024-04-25T10:12:30.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.975 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128139 s, 81.8 MB/s 2024-04-25T10:12:30.989 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:30.989 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:30.989 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118662 s, 88.4 MB/s 2024-04-25T10:12:31.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.006 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.006 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130105 s, 80.6 MB/s 2024-04-25T10:12:31.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.020 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116603 s, 89.9 MB/s 2024-04-25T10:12:31.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.034 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114697 s, 91.4 MB/s 2024-04-25T10:12:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117079 s, 89.6 MB/s 2024-04-25T10:12:31.063 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.063 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.063 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125735 s, 83.4 MB/s 2024-04-25T10:12:31.080 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.080 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.080 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134843 s, 77.8 MB/s 2024-04-25T10:12:31.094 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.094 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.094 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117599 s, 89.2 MB/s 2024-04-25T10:12:31.108 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.108 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.109 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121213 s, 86.5 MB/s 2024-04-25T10:12:31.123 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.123 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.123 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119014 s, 88.1 MB/s 2024-04-25T10:12:31.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.137 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116715 s, 89.8 MB/s 2024-04-25T10:12:31.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.152 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124954 s, 83.9 MB/s 2024-04-25T10:12:31.167 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.167 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.168 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128274 s, 81.7 MB/s 2024-04-25T10:12:31.181 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.181 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.181 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105034 s, 99.8 MB/s 2024-04-25T10:12:31.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.196 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130228 s, 80.5 MB/s 2024-04-25T10:12:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115983 s, 90.4 MB/s 2024-04-25T10:12:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.226 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133805 s, 78.4 MB/s 2024-04-25T10:12:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997108 s, 105 MB/s 2024-04-25T10:12:31.252 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.253 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116421 s, 90.1 MB/s 2024-04-25T10:12:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119327 s, 87.9 MB/s 2024-04-25T10:12:31.282 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.282 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.282 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123021 s, 85.2 MB/s 2024-04-25T10:12:31.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:31.283 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:31.283 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:31.283 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:12:31.283 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:31.283 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:31.284 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:31.284 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:31.353 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.353 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.353 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011954 s, 87.7 MB/s 2024-04-25T10:12:31.366 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.366 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.366 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010546 s, 99.4 MB/s 2024-04-25T10:12:31.381 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.381 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.381 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113472 s, 92.4 MB/s 2024-04-25T10:12:31.395 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.395 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.395 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119398 s, 87.8 MB/s 2024-04-25T10:12:31.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.409 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120776 s, 86.8 MB/s 2024-04-25T10:12:31.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.421 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972078 s, 108 MB/s 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.437 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125216 s, 83.7 MB/s 2024-04-25T10:12:31.452 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.453 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.453 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013647 s, 76.8 MB/s 2024-04-25T10:12:31.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.467 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120082 s, 87.3 MB/s 2024-04-25T10:12:31.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.479 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977853 s, 107 MB/s 2024-04-25T10:12:31.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.491 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916429 s, 114 MB/s 2024-04-25T10:12:31.505 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.505 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.505 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117236 s, 89.4 MB/s 2024-04-25T10:12:31.519 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.519 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.519 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111264 s, 94.2 MB/s 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125919 s, 83.3 MB/s 2024-04-25T10:12:31.549 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.549 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.549 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012874 s, 81.4 MB/s 2024-04-25T10:12:31.563 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.563 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.563 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115074 s, 91.1 MB/s 2024-04-25T10:12:31.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.575 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948806 s, 111 MB/s 2024-04-25T10:12:31.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.590 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121268 s, 86.5 MB/s 2024-04-25T10:12:31.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115413 s, 90.9 MB/s 2024-04-25T10:12:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.619 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119646 s, 87.6 MB/s 2024-04-25T10:12:31.631 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.631 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.631 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102511 s, 102 MB/s 2024-04-25T10:12:31.646 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.646 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.646 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120097 s, 87.3 MB/s 2024-04-25T10:12:31.661 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.661 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.661 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124905 s, 84.0 MB/s 2024-04-25T10:12:31.674 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.674 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.675 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114395 s, 91.7 MB/s 2024-04-25T10:12:31.688 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.689 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111892 s, 93.7 MB/s 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126507 s, 82.9 MB/s 2024-04-25T10:12:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128365 s, 81.7 MB/s 2024-04-25T10:12:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117169 s, 89.5 MB/s 2024-04-25T10:12:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106006 s, 98.9 MB/s 2024-04-25T10:12:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012167 s, 86.2 MB/s 2024-04-25T10:12:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117765 s, 89.0 MB/s 2024-04-25T10:12:31.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.790 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118843 s, 88.2 MB/s 2024-04-25T10:12:31.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:31.794 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:31.794 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:31.794 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:12:31.794 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:31.794 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:31.794 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:31.794 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:31.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.861 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128372 s, 81.7 MB/s 2024-04-25T10:12:31.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.874 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104544 s, 100 MB/s 2024-04-25T10:12:31.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.889 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123559 s, 84.9 MB/s 2024-04-25T10:12:31.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.904 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127533 s, 82.2 MB/s 2024-04-25T10:12:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110761 s, 94.7 MB/s 2024-04-25T10:12:31.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.931 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105675 s, 99.2 MB/s 2024-04-25T10:12:31.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.947 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01371 s, 76.5 MB/s 2024-04-25T10:12:31.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.962 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121302 s, 86.4 MB/s 2024-04-25T10:12:31.978 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.978 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.978 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133097 s, 78.8 MB/s 2024-04-25T10:12:31.995 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:31.995 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:31.997 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143452 s, 73.1 MB/s 2024-04-25T10:12:32.008 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.008 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.009 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103397 s, 101 MB/s 2024-04-25T10:12:32.024 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.025 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129177 s, 81.2 MB/s 2024-04-25T10:12:32.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.038 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106409 s, 98.5 MB/s 2024-04-25T10:12:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121593 s, 86.2 MB/s 2024-04-25T10:12:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122485 s, 85.6 MB/s 2024-04-25T10:12:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128992 s, 81.3 MB/s 2024-04-25T10:12:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130476 s, 80.4 MB/s 2024-04-25T10:12:32.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.117 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137045 s, 76.5 MB/s 2024-04-25T10:12:32.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.132 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120595 s, 87.0 MB/s 2024-04-25T10:12:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.149 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140929 s, 74.4 MB/s 2024-04-25T10:12:32.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.165 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129969 s, 80.7 MB/s 2024-04-25T10:12:32.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.181 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.181 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126896 s, 82.6 MB/s 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136057 s, 77.1 MB/s 2024-04-25T10:12:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.213 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122358 s, 85.7 MB/s 2024-04-25T10:12:32.229 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.229 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.229 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132937 s, 78.9 MB/s 2024-04-25T10:12:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.246 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135979 s, 77.1 MB/s 2024-04-25T10:12:32.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.261 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126996 s, 82.6 MB/s 2024-04-25T10:12:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.278 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135135 s, 77.6 MB/s 2024-04-25T10:12:32.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.296 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139199 s, 75.3 MB/s 2024-04-25T10:12:32.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.312 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126119 s, 83.1 MB/s 2024-04-25T10:12:32.363 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.363 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.363 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127938 s, 82.0 MB/s 2024-04-25T10:12:32.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.379 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012899 s, 81.3 MB/s 2024-04-25T10:12:32.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:32.380 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:32.380 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:32.380 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:12:32.380 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:32.380 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:32.380 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:32.380 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:32.448 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.448 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.449 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127887 s, 82.0 MB/s 2024-04-25T10:12:32.463 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.464 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121035 s, 86.6 MB/s 2024-04-25T10:12:32.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.479 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127355 s, 82.3 MB/s 2024-04-25T10:12:32.492 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.492 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.492 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966386 s, 109 MB/s 2024-04-25T10:12:32.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.507 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126242 s, 83.1 MB/s 2024-04-25T10:12:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128493 s, 81.6 MB/s 2024-04-25T10:12:32.538 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.538 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.538 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121496 s, 86.3 MB/s 2024-04-25T10:12:32.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.551 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996606 s, 105 MB/s 2024-04-25T10:12:32.563 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.563 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.563 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971585 s, 108 MB/s 2024-04-25T10:12:32.577 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.577 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.577 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115249 s, 91.0 MB/s 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.592 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113776 s, 92.2 MB/s 2024-04-25T10:12:32.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.606 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112402 s, 93.3 MB/s 2024-04-25T10:12:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119816 s, 87.5 MB/s 2024-04-25T10:12:32.635 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.635 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.636 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126496 s, 82.9 MB/s 2024-04-25T10:12:32.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.649 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112328 s, 93.3 MB/s 2024-04-25T10:12:32.663 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.663 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.663 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112564 s, 93.2 MB/s 2024-04-25T10:12:32.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.764 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961638 s, 109 MB/s 2024-04-25T10:12:32.765 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.765 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132079 s, 79.4 MB/s 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122325 s, 85.7 MB/s 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123136 s, 85.2 MB/s 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126295 s, 83.0 MB/s 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967688 s, 108 MB/s 2024-04-25T10:12:32.767 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.767 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.767 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121115 s, 86.6 MB/s 2024-04-25T10:12:32.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.777 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109418 s, 95.8 MB/s 2024-04-25T10:12:32.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.792 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125617 s, 83.5 MB/s 2024-04-25T10:12:32.803 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.803 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.803 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889954 s, 118 MB/s 2024-04-25T10:12:32.817 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.818 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.818 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114727 s, 91.4 MB/s 2024-04-25T10:12:32.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.833 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121935 s, 86.0 MB/s 2024-04-25T10:12:32.844 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.844 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.844 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905161 s, 116 MB/s 2024-04-25T10:12:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122825 s, 85.4 MB/s 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103708 s, 101 MB/s 2024-04-25T10:12:32.887 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.887 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.887 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119156 s, 88.0 MB/s 2024-04-25T10:12:32.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:32.888 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:32.888 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:32.888 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:12:32.888 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:32.888 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:32.888 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:32.888 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:32.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.959 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137781 s, 76.1 MB/s 2024-04-25T10:12:32.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.973 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114353 s, 91.7 MB/s 2024-04-25T10:12:32.990 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:32.990 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:32.990 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133454 s, 78.6 MB/s 2024-04-25T10:12:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124792 s, 84.0 MB/s 2024-04-25T10:12:33.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.019 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109909 s, 95.4 MB/s 2024-04-25T10:12:33.031 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.031 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.031 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952377 s, 110 MB/s 2024-04-25T10:12:33.045 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.045 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.045 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117467 s, 89.3 MB/s 2024-04-25T10:12:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114297 s, 91.7 MB/s 2024-04-25T10:12:33.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.071 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094058 s, 111 MB/s 2024-04-25T10:12:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113636 s, 92.3 MB/s 2024-04-25T10:12:33.096 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.096 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.097 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933761 s, 112 MB/s 2024-04-25T10:12:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110413 s, 95.0 MB/s 2024-04-25T10:12:33.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.124 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115402 s, 90.9 MB/s 2024-04-25T10:12:33.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.138 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112726 s, 93.0 MB/s 2024-04-25T10:12:33.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.152 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116029 s, 90.4 MB/s 2024-04-25T10:12:33.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.166 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110866 s, 94.6 MB/s 2024-04-25T10:12:33.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.180 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114855 s, 91.3 MB/s 2024-04-25T10:12:33.193 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.194 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110611 s, 94.8 MB/s 2024-04-25T10:12:33.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.211 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142666 s, 73.5 MB/s 2024-04-25T10:12:33.226 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.226 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.226 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011856 s, 88.4 MB/s 2024-04-25T10:12:33.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.243 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126966 s, 82.6 MB/s 2024-04-25T10:12:33.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.260 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.260 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130473 s, 80.4 MB/s 2024-04-25T10:12:33.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.275 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116204 s, 90.2 MB/s 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134725 s, 77.8 MB/s 2024-04-25T10:12:33.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.307 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128698 s, 81.5 MB/s 2024-04-25T10:12:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160227 s, 65.4 MB/s 2024-04-25T10:12:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868051 s, 121 MB/s 2024-04-25T10:12:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011355 s, 92.3 MB/s 2024-04-25T10:12:33.372 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.372 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.372 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110659 s, 94.8 MB/s 2024-04-25T10:12:33.387 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.387 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.387 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122241 s, 85.8 MB/s 2024-04-25T10:12:33.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.401 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011493 s, 91.2 MB/s 2024-04-25T10:12:33.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.415 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119524 s, 87.7 MB/s 2024-04-25T10:12:33.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:33.416 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:33.416 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:33.417 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:12:33.417 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:33.417 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:33.417 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:33.417 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:33.481 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.482 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124784 s, 84.0 MB/s 2024-04-25T10:12:33.496 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.496 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.496 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114737 s, 91.4 MB/s 2024-04-25T10:12:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.513 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138937 s, 75.5 MB/s 2024-04-25T10:12:33.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.527 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116587 s, 89.9 MB/s 2024-04-25T10:12:33.542 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.542 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.542 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120249 s, 87.2 MB/s 2024-04-25T10:12:33.556 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.556 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.556 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119047 s, 88.1 MB/s 2024-04-25T10:12:33.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.569 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.569 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104 s, 101 MB/s 2024-04-25T10:12:33.586 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.587 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.587 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147651 s, 71.0 MB/s 2024-04-25T10:12:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119061 s, 88.1 MB/s 2024-04-25T10:12:33.614 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.615 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111322 s, 94.2 MB/s 2024-04-25T10:12:33.628 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.628 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.628 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110929 s, 94.5 MB/s 2024-04-25T10:12:33.642 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.642 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.642 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116694 s, 89.9 MB/s 2024-04-25T10:12:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902505 s, 116 MB/s 2024-04-25T10:12:33.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.665 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010069 s, 104 MB/s 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.677 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996573 s, 105 MB/s 2024-04-25T10:12:33.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.689 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101269 s, 104 MB/s 2024-04-25T10:12:33.702 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.702 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.702 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100603 s, 104 MB/s 2024-04-25T10:12:33.711 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.711 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.711 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748144 s, 140 MB/s 2024-04-25T10:12:33.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.724 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.724 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102968 s, 102 MB/s 2024-04-25T10:12:33.737 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.737 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.737 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114311 s, 91.7 MB/s 2024-04-25T10:12:33.749 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.749 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.749 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010246 s, 102 MB/s 2024-04-25T10:12:33.762 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.762 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.762 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105372 s, 99.5 MB/s 2024-04-25T10:12:33.774 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.775 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106532 s, 98.4 MB/s 2024-04-25T10:12:33.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.787 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100195 s, 105 MB/s 2024-04-25T10:12:33.796 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.797 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764403 s, 137 MB/s 2024-04-25T10:12:33.806 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.806 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.806 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743471 s, 141 MB/s 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.819 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103955 s, 101 MB/s 2024-04-25T10:12:33.833 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.833 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.833 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122457 s, 85.6 MB/s 2024-04-25T10:12:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.845 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.846 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101041 s, 104 MB/s 2024-04-25T10:12:33.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.856 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733952 s, 143 MB/s 2024-04-25T10:12:33.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.867 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967085 s, 108 MB/s 2024-04-25T10:12:33.876 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.876 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.876 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735141 s, 143 MB/s 2024-04-25T10:12:33.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:33.878 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:33.878 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:33.878 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:12:33.878 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:33.878 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:33.878 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:33.878 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:33.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.944 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969573 s, 108 MB/s 2024-04-25T10:12:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.956 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985209 s, 106 MB/s 2024-04-25T10:12:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.968 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992481 s, 106 MB/s 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106346 s, 98.6 MB/s 2024-04-25T10:12:33.995 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:33.995 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:33.995 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115244 s, 91.0 MB/s 2024-04-25T10:12:34.009 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.009 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.009 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011406 s, 91.9 MB/s 2024-04-25T10:12:34.023 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.023 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.023 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121686 s, 86.2 MB/s 2024-04-25T10:12:34.036 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.036 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.036 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112188 s, 93.5 MB/s 2024-04-25T10:12:34.051 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.051 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.051 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123377 s, 85.0 MB/s 2024-04-25T10:12:34.064 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.065 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115861 s, 90.5 MB/s 2024-04-25T10:12:34.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.078 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116539 s, 90.0 MB/s 2024-04-25T10:12:34.091 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.091 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110355 s, 95.0 MB/s 2024-04-25T10:12:34.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.103 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879545 s, 119 MB/s 2024-04-25T10:12:34.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.117 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117048 s, 89.6 MB/s 2024-04-25T10:12:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.129 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105981 s, 98.9 MB/s 2024-04-25T10:12:34.143 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.143 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.143 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011526 s, 91.0 MB/s 2024-04-25T10:12:34.155 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.155 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.155 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102929 s, 102 MB/s 2024-04-25T10:12:34.168 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.168 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.168 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108142 s, 97.0 MB/s 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100345 s, 104 MB/s 2024-04-25T10:12:34.193 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.193 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.193 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104905 s, 100 MB/s 2024-04-25T10:12:34.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.205 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100004 s, 105 MB/s 2024-04-25T10:12:34.218 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.218 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.218 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109137 s, 96.1 MB/s 2024-04-25T10:12:34.230 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.230 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.230 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989425 s, 106 MB/s 2024-04-25T10:12:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865455 s, 121 MB/s 2024-04-25T10:12:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.254 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107901 s, 97.2 MB/s 2024-04-25T10:12:34.266 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.267 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103804 s, 101 MB/s 2024-04-25T10:12:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127257 s, 82.4 MB/s 2024-04-25T10:12:34.297 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.297 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.297 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115905 s, 90.5 MB/s 2024-04-25T10:12:34.313 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.313 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.313 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126253 s, 83.1 MB/s 2024-04-25T10:12:34.333 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.333 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.333 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189876 s, 55.2 MB/s 2024-04-25T10:12:34.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.357 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782264 s, 134 MB/s 2024-04-25T10:12:34.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.372 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.372 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117352 s, 89.4 MB/s 2024-04-25T10:12:34.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:34.373 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:34.373 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:34.373 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:12:34.373 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:34.373 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:34.373 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:34.373 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:34.437 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.437 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.438 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130296 s, 80.5 MB/s 2024-04-25T10:12:34.452 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.452 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.452 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119314 s, 87.9 MB/s 2024-04-25T10:12:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901375 s, 116 MB/s 2024-04-25T10:12:34.478 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.478 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.478 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113083 s, 92.7 MB/s 2024-04-25T10:12:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.494 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136892 s, 76.6 MB/s 2024-04-25T10:12:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.508 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109741 s, 95.5 MB/s 2024-04-25T10:12:34.521 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.521 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.521 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010345 s, 101 MB/s 2024-04-25T10:12:34.535 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.536 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116191 s, 90.2 MB/s 2024-04-25T10:12:34.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.552 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135589 s, 77.3 MB/s 2024-04-25T10:12:34.564 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.564 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.564 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943466 s, 111 MB/s 2024-04-25T10:12:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106157 s, 98.8 MB/s 2024-04-25T10:12:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126624 s, 82.8 MB/s 2024-04-25T10:12:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110818 s, 94.6 MB/s 2024-04-25T10:12:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.622 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123733 s, 84.7 MB/s 2024-04-25T10:12:34.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.637 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124286 s, 84.4 MB/s 2024-04-25T10:12:34.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.653 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115647 s, 90.7 MB/s 2024-04-25T10:12:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.666 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011735 s, 89.4 MB/s 2024-04-25T10:12:34.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.679 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114325 s, 91.7 MB/s 2024-04-25T10:12:34.691 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.691 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.691 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873731 s, 120 MB/s 2024-04-25T10:12:34.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.704 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01105 s, 94.9 MB/s 2024-04-25T10:12:34.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.720 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129118 s, 81.2 MB/s 2024-04-25T10:12:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108023 s, 97.1 MB/s 2024-04-25T10:12:34.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.745 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938355 s, 112 MB/s 2024-04-25T10:12:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119999 s, 87.4 MB/s 2024-04-25T10:12:34.774 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.774 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.774 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117699 s, 89.1 MB/s 2024-04-25T10:12:34.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.790 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130755 s, 80.2 MB/s 2024-04-25T10:12:34.803 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.803 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.804 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010805 s, 97.0 MB/s 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.816 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962614 s, 109 MB/s 2024-04-25T10:12:34.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.830 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114716 s, 91.4 MB/s 2024-04-25T10:12:34.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.842 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933868 s, 112 MB/s 2024-04-25T10:12:34.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.857 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117763 s, 89.0 MB/s 2024-04-25T10:12:34.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.871 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122311 s, 85.7 MB/s 2024-04-25T10:12:34.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:34.872 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:34.872 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:34.873 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:12:34.873 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:34.873 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:34.873 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:34.873 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:34.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012109 s, 86.6 MB/s 2024-04-25T10:12:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011667 s, 89.9 MB/s 2024-04-25T10:12:34.963 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.963 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.963 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902909 s, 116 MB/s 2024-04-25T10:12:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114423 s, 91.6 MB/s 2024-04-25T10:12:34.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:34.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:34.991 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116259 s, 90.2 MB/s 2024-04-25T10:12:35.006 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.006 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.006 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011904 s, 88.1 MB/s 2024-04-25T10:12:35.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.019 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966608 s, 108 MB/s 2024-04-25T10:12:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122953 s, 85.3 MB/s 2024-04-25T10:12:35.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.048 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012122 s, 86.5 MB/s 2024-04-25T10:12:35.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.062 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111053 s, 94.4 MB/s 2024-04-25T10:12:35.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.078 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127989 s, 81.9 MB/s 2024-04-25T10:12:35.092 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.092 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.092 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116413 s, 90.1 MB/s 2024-04-25T10:12:35.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.107 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122971 s, 85.3 MB/s 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114791 s, 91.3 MB/s 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.136 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118897 s, 88.2 MB/s 2024-04-25T10:12:35.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.152 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133768 s, 78.4 MB/s 2024-04-25T10:12:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114901 s, 91.3 MB/s 2024-04-25T10:12:35.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.179 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967917 s, 108 MB/s 2024-04-25T10:12:35.191 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.192 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.192 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102552 s, 102 MB/s 2024-04-25T10:12:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127369 s, 82.3 MB/s 2024-04-25T10:12:35.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.222 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123981 s, 84.6 MB/s 2024-04-25T10:12:35.234 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.234 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.234 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910124 s, 115 MB/s 2024-04-25T10:12:35.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.248 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115109 s, 91.1 MB/s 2024-04-25T10:12:35.263 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.263 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.263 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122465 s, 85.6 MB/s 2024-04-25T10:12:35.277 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.277 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.277 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112511 s, 93.2 MB/s 2024-04-25T10:12:35.292 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.292 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013142 s, 79.8 MB/s 2024-04-25T10:12:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.306 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108872 s, 96.3 MB/s 2024-04-25T10:12:35.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.321 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126791 s, 82.7 MB/s 2024-04-25T10:12:35.336 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.336 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.337 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131111 s, 80.0 MB/s 2024-04-25T10:12:35.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.367 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131894 s, 79.5 MB/s 2024-04-25T10:12:35.382 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.382 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.382 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013101 s, 80.0 MB/s 2024-04-25T10:12:35.398 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.398 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.398 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133977 s, 78.3 MB/s 2024-04-25T10:12:35.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:35.399 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:35.399 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:35.399 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:12:35.400 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:35.400 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:35.400 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:35.400 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:35.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.469 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125762 s, 83.4 MB/s 2024-04-25T10:12:35.485 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.485 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.485 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132059 s, 79.4 MB/s 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.501 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134657 s, 77.9 MB/s 2024-04-25T10:12:35.514 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.514 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.514 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983216 s, 107 MB/s 2024-04-25T10:12:35.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.529 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123356 s, 85.0 MB/s 2024-04-25T10:12:35.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.545 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125941 s, 83.3 MB/s 2024-04-25T10:12:35.562 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.562 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.562 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144188 s, 72.7 MB/s 2024-04-25T10:12:35.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.579 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.579 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135948 s, 77.1 MB/s 2024-04-25T10:12:35.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.591 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964291 s, 109 MB/s 2024-04-25T10:12:35.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.605 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112419 s, 93.3 MB/s 2024-04-25T10:12:35.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.620 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115646 s, 90.7 MB/s 2024-04-25T10:12:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105059 s, 99.8 MB/s 2024-04-25T10:12:35.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.649 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126758 s, 82.7 MB/s 2024-04-25T10:12:35.664 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.664 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.664 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120846 s, 86.8 MB/s 2024-04-25T10:12:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993819 s, 106 MB/s 2024-04-25T10:12:35.693 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.693 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.694 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136233 s, 77.0 MB/s 2024-04-25T10:12:35.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.709 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123917 s, 84.6 MB/s 2024-04-25T10:12:35.724 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.724 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.724 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121904 s, 86.0 MB/s 2024-04-25T10:12:35.737 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.737 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.737 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106025 s, 98.9 MB/s 2024-04-25T10:12:35.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.754 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131996 s, 79.4 MB/s 2024-04-25T10:12:35.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.769 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123975 s, 84.6 MB/s 2024-04-25T10:12:35.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.785 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131996 s, 79.4 MB/s 2024-04-25T10:12:35.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.801 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.801 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124952 s, 83.9 MB/s 2024-04-25T10:12:35.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.814 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105917 s, 99.0 MB/s 2024-04-25T10:12:35.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.831 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.831 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136027 s, 77.1 MB/s 2024-04-25T10:12:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124079 s, 84.5 MB/s 2024-04-25T10:12:35.860 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.861 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123993 s, 84.6 MB/s 2024-04-25T10:12:35.876 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.876 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.876 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125209 s, 83.7 MB/s 2024-04-25T10:12:35.892 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.892 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.892 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129035 s, 81.3 MB/s 2024-04-25T10:12:35.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.907 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123528 s, 84.9 MB/s 2024-04-25T10:12:35.923 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.923 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.923 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132879 s, 78.9 MB/s 2024-04-25T10:12:35.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:35.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:35.936 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978747 s, 107 MB/s 2024-04-25T10:12:35.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:35.937 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:35.937 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:35.937 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:12:35.937 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:35.937 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:35.937 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:35.938 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:36.009 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.009 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.009 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132697 s, 79.0 MB/s 2024-04-25T10:12:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.026 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129132 s, 81.2 MB/s 2024-04-25T10:12:36.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.044 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136107 s, 77.0 MB/s 2024-04-25T10:12:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130981 s, 80.1 MB/s 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.072 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932095 s, 112 MB/s 2024-04-25T10:12:36.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.086 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114703 s, 91.4 MB/s 2024-04-25T10:12:36.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.102 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128793 s, 81.4 MB/s 2024-04-25T10:12:36.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.117 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120109 s, 87.3 MB/s 2024-04-25T10:12:36.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.130 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.130 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992426 s, 106 MB/s 2024-04-25T10:12:36.144 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.144 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.144 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119995 s, 87.4 MB/s 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140442 s, 74.7 MB/s 2024-04-25T10:12:36.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.179 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132582 s, 79.1 MB/s 2024-04-25T10:12:36.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.195 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129329 s, 81.1 MB/s 2024-04-25T10:12:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140707 s, 74.5 MB/s 2024-04-25T10:12:36.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.227 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125037 s, 83.9 MB/s 2024-04-25T10:12:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.244 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133411 s, 78.6 MB/s 2024-04-25T10:12:36.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.259 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128806 s, 81.4 MB/s 2024-04-25T10:12:36.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.274 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.274 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116904 s, 89.7 MB/s 2024-04-25T10:12:36.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.287 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101939 s, 103 MB/s 2024-04-25T10:12:36.302 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.302 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.302 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123992 s, 84.6 MB/s 2024-04-25T10:12:36.315 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.315 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.315 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101609 s, 103 MB/s 2024-04-25T10:12:36.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.333 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145008 s, 72.3 MB/s 2024-04-25T10:12:36.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.386 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113912 s, 92.1 MB/s 2024-04-25T10:12:36.400 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119678 s, 87.6 MB/s 2024-04-25T10:12:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146677 s, 71.5 MB/s 2024-04-25T10:12:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104062 s, 101 MB/s 2024-04-25T10:12:36.448 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.449 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128039 s, 81.9 MB/s 2024-04-25T10:12:36.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.466 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.466 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136649 s, 76.7 MB/s 2024-04-25T10:12:36.483 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.483 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.483 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137303 s, 76.4 MB/s 2024-04-25T10:12:36.499 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.499 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.499 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012867 s, 81.5 MB/s 2024-04-25T10:12:36.514 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.514 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.514 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012293 s, 85.3 MB/s 2024-04-25T10:12:36.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.530 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130515 s, 80.3 MB/s 2024-04-25T10:12:36.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:36.531 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:36.531 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:36.531 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:12:36.531 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:36.531 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:36.531 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:36.531 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:36.600 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.600 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.600 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146089 s, 71.8 MB/s 2024-04-25T10:12:36.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.616 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123157 s, 85.1 MB/s 2024-04-25T10:12:36.631 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.631 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.631 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01199 s, 87.5 MB/s 2024-04-25T10:12:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124651 s, 84.1 MB/s 2024-04-25T10:12:36.663 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.664 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.664 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131759 s, 79.6 MB/s 2024-04-25T10:12:36.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.679 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123817 s, 84.7 MB/s 2024-04-25T10:12:36.694 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.695 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012391 s, 84.6 MB/s 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012001 s, 87.4 MB/s 2024-04-25T10:12:36.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.726 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124842 s, 84.0 MB/s 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125457 s, 83.6 MB/s 2024-04-25T10:12:36.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.754 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100233 s, 105 MB/s 2024-04-25T10:12:36.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.767 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961201 s, 109 MB/s 2024-04-25T10:12:36.782 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.783 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.783 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131599 s, 79.7 MB/s 2024-04-25T10:12:36.798 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.798 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.798 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118611 s, 88.4 MB/s 2024-04-25T10:12:36.812 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.812 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.812 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113942 s, 92.0 MB/s 2024-04-25T10:12:36.827 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.827 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.827 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119214 s, 88.0 MB/s 2024-04-25T10:12:36.843 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.843 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.843 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132329 s, 79.2 MB/s 2024-04-25T10:12:36.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.857 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116201 s, 90.2 MB/s 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.870 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998772 s, 105 MB/s 2024-04-25T10:12:36.885 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.885 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.885 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125023 s, 83.9 MB/s 2024-04-25T10:12:36.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.900 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120246 s, 87.2 MB/s 2024-04-25T10:12:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.918 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013271 s, 79.0 MB/s 2024-04-25T10:12:36.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.933 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127574 s, 82.2 MB/s 2024-04-25T10:12:36.948 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.948 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.948 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124048 s, 84.5 MB/s 2024-04-25T10:12:36.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.964 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131178 s, 79.9 MB/s 2024-04-25T10:12:36.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.981 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131963 s, 79.5 MB/s 2024-04-25T10:12:36.994 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:36.994 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:36.994 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010523 s, 99.6 MB/s 2024-04-25T10:12:37.009 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.009 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.009 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123397 s, 85.0 MB/s 2024-04-25T10:12:37.024 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.024 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.024 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119096 s, 88.0 MB/s 2024-04-25T10:12:37.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.039 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127202 s, 82.4 MB/s 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.054 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114777 s, 91.4 MB/s 2024-04-25T10:12:37.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.068 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119563 s, 87.7 MB/s 2024-04-25T10:12:37.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:37.070 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:37.070 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:37.070 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:12:37.070 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:37.070 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:37.070 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:37.070 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131155 s, 79.9 MB/s 2024-04-25T10:12:37.156 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.156 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.157 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011428 s, 91.8 MB/s 2024-04-25T10:12:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.172 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.172 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126824 s, 82.7 MB/s 2024-04-25T10:12:37.184 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.184 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.184 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095519 s, 110 MB/s 2024-04-25T10:12:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119125 s, 88.0 MB/s 2024-04-25T10:12:37.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.214 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122698 s, 85.5 MB/s 2024-04-25T10:12:37.230 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.230 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.230 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013032 s, 80.5 MB/s 2024-04-25T10:12:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122163 s, 85.8 MB/s 2024-04-25T10:12:37.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.260 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111843 s, 93.8 MB/s 2024-04-25T10:12:37.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.275 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122958 s, 85.3 MB/s 2024-04-25T10:12:37.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.290 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122746 s, 85.4 MB/s 2024-04-25T10:12:37.305 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.305 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.305 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119986 s, 87.4 MB/s 2024-04-25T10:12:37.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.321 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124884 s, 84.0 MB/s 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.338 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145874 s, 71.9 MB/s 2024-04-25T10:12:37.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.376 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123505 s, 84.9 MB/s 2024-04-25T10:12:37.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.391 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123303 s, 85.0 MB/s 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.406 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120731 s, 86.9 MB/s 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.418 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.419 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0096029 s, 109 MB/s 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.433 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011945 s, 87.8 MB/s 2024-04-25T10:12:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01393 s, 75.3 MB/s 2024-04-25T10:12:37.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.467 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130364 s, 80.4 MB/s 2024-04-25T10:12:37.481 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.482 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011956 s, 87.7 MB/s 2024-04-25T10:12:37.497 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.497 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.497 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012764 s, 82.2 MB/s 2024-04-25T10:12:37.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.512 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119803 s, 87.5 MB/s 2024-04-25T10:12:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144181 s, 72.7 MB/s 2024-04-25T10:12:37.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012482 s, 84.0 MB/s 2024-04-25T10:12:37.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118411 s, 88.6 MB/s 2024-04-25T10:12:37.576 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.576 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.576 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131306 s, 79.9 MB/s 2024-04-25T10:12:37.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.591 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126516 s, 82.9 MB/s 2024-04-25T10:12:37.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.605 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109528 s, 95.7 MB/s 2024-04-25T10:12:37.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.620 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118486 s, 88.5 MB/s 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122128 s, 85.9 MB/s 2024-04-25T10:12:37.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:37.637 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:37.637 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:37.637 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:12:37.637 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:37.637 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:37.637 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:37.637 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130829 s, 80.1 MB/s 2024-04-25T10:12:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.720 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129737 s, 80.8 MB/s 2024-04-25T10:12:37.732 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.732 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.732 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934278 s, 112 MB/s 2024-04-25T10:12:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129742 s, 80.8 MB/s 2024-04-25T10:12:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984035 s, 107 MB/s 2024-04-25T10:12:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130759 s, 80.2 MB/s 2024-04-25T10:12:37.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.793 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.793 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129146 s, 81.2 MB/s 2024-04-25T10:12:37.808 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.808 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.808 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127237 s, 82.4 MB/s 2024-04-25T10:12:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.824 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125898 s, 83.3 MB/s 2024-04-25T10:12:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139825 s, 75.0 MB/s 2024-04-25T10:12:37.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.858 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137959 s, 76.0 MB/s 2024-04-25T10:12:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.874 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132909 s, 78.9 MB/s 2024-04-25T10:12:37.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.889 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121284 s, 86.5 MB/s 2024-04-25T10:12:37.903 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.903 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.903 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110408 s, 95.0 MB/s 2024-04-25T10:12:37.914 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.914 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.915 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902275 s, 116 MB/s 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.929 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119661 s, 87.6 MB/s 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121212 s, 86.5 MB/s 2024-04-25T10:12:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.958 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112347 s, 93.3 MB/s 2024-04-25T10:12:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.973 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125458 s, 83.6 MB/s 2024-04-25T10:12:37.987 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:37.987 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:37.988 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118552 s, 88.4 MB/s 2024-04-25T10:12:38.003 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.004 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133411 s, 78.6 MB/s 2024-04-25T10:12:38.016 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.016 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.016 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0097671 s, 107 MB/s 2024-04-25T10:12:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011211 s, 93.5 MB/s 2024-04-25T10:12:38.045 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.045 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.045 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123806 s, 84.7 MB/s 2024-04-25T10:12:38.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.058 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970726 s, 108 MB/s 2024-04-25T10:12:38.072 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.072 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.072 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116564 s, 90.0 MB/s 2024-04-25T10:12:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.086 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112775 s, 93.0 MB/s 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.100 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118198 s, 88.7 MB/s 2024-04-25T10:12:38.115 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.116 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012593 s, 83.3 MB/s 2024-04-25T10:12:38.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.131 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127973 s, 81.9 MB/s 2024-04-25T10:12:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123729 s, 84.7 MB/s 2024-04-25T10:12:38.159 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.160 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010822 s, 96.9 MB/s 2024-04-25T10:12:38.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:38.161 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:38.161 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:38.161 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:12:38.161 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:38.161 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:38.161 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:38.161 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:38.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.224 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100096 s, 105 MB/s 2024-04-25T10:12:38.239 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.239 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.239 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127705 s, 82.1 MB/s 2024-04-25T10:12:38.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.255 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124976 s, 83.9 MB/s 2024-04-25T10:12:38.271 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.271 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.271 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133469 s, 78.6 MB/s 2024-04-25T10:12:38.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.287 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127334 s, 82.3 MB/s 2024-04-25T10:12:38.301 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.301 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.301 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115463 s, 90.8 MB/s 2024-04-25T10:12:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126738 s, 82.7 MB/s 2024-04-25T10:12:38.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.332 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122394 s, 85.7 MB/s 2024-04-25T10:12:38.351 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.351 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.351 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164103 s, 63.9 MB/s 2024-04-25T10:12:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132693 s, 79.0 MB/s 2024-04-25T10:12:38.389 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.389 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.390 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011678 s, 89.8 MB/s 2024-04-25T10:12:38.403 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.403 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.403 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104413 s, 100 MB/s 2024-04-25T10:12:38.417 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.418 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.418 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119229 s, 87.9 MB/s 2024-04-25T10:12:38.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.433 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128503 s, 81.6 MB/s 2024-04-25T10:12:38.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.449 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134946 s, 77.7 MB/s 2024-04-25T10:12:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123916 s, 84.6 MB/s 2024-04-25T10:12:38.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.480 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121739 s, 86.1 MB/s 2024-04-25T10:12:38.495 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.495 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.495 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121001 s, 86.7 MB/s 2024-04-25T10:12:38.509 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.510 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.510 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118973 s, 88.1 MB/s 2024-04-25T10:12:38.526 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.526 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.526 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013584 s, 77.2 MB/s 2024-04-25T10:12:38.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.540 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.540 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101679 s, 103 MB/s 2024-04-25T10:12:38.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.552 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948322 s, 111 MB/s 2024-04-25T10:12:38.564 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.564 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.564 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094209 s, 111 MB/s 2024-04-25T10:12:38.579 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.579 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.580 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121881 s, 86.0 MB/s 2024-04-25T10:12:38.593 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.593 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.593 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110664 s, 94.8 MB/s 2024-04-25T10:12:38.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.606 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107549 s, 97.5 MB/s 2024-04-25T10:12:38.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.623 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.623 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133491 s, 78.6 MB/s 2024-04-25T10:12:38.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121706 s, 86.2 MB/s 2024-04-25T10:12:38.652 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.652 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.652 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117529 s, 89.2 MB/s 2024-04-25T10:12:38.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.667 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124404 s, 84.3 MB/s 2024-04-25T10:12:38.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.680 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103129 s, 102 MB/s 2024-04-25T10:12:38.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.692 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100573 s, 104 MB/s 2024-04-25T10:12:38.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:38.693 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:38.693 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:38.693 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:12:38.693 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:38.694 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:38.694 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:38.694 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:38.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.761 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109454 s, 95.8 MB/s 2024-04-25T10:12:38.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.777 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123005 s, 85.2 MB/s 2024-04-25T10:12:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.791 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114935 s, 91.2 MB/s 2024-04-25T10:12:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122096 s, 85.9 MB/s 2024-04-25T10:12:38.817 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.817 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.817 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843243 s, 124 MB/s 2024-04-25T10:12:38.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.832 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123602 s, 84.8 MB/s 2024-04-25T10:12:38.846 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.846 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.846 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109693 s, 95.6 MB/s 2024-04-25T10:12:38.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.858 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088126 s, 119 MB/s 2024-04-25T10:12:38.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.875 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143277 s, 73.2 MB/s 2024-04-25T10:12:38.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.888 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108167 s, 96.9 MB/s 2024-04-25T10:12:38.903 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.903 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.903 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116916 s, 89.7 MB/s 2024-04-25T10:12:38.917 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.917 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.917 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011405 s, 91.9 MB/s 2024-04-25T10:12:38.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.933 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123868 s, 84.7 MB/s 2024-04-25T10:12:38.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.951 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134857 s, 77.8 MB/s 2024-04-25T10:12:38.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.968 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124243 s, 84.4 MB/s 2024-04-25T10:12:38.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:38.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:38.985 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125021 s, 83.9 MB/s 2024-04-25T10:12:39.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.000 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106355 s, 98.6 MB/s 2024-04-25T10:12:39.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.017 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131074 s, 80.0 MB/s 2024-04-25T10:12:39.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.034 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01347 s, 77.8 MB/s 2024-04-25T10:12:39.051 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.051 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.051 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130037 s, 80.6 MB/s 2024-04-25T10:12:39.067 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.067 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.067 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136885 s, 76.6 MB/s 2024-04-25T10:12:39.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114708 s, 91.4 MB/s 2024-04-25T10:12:39.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.099 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133557 s, 78.5 MB/s 2024-04-25T10:12:39.115 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.115 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.115 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129682 s, 80.9 MB/s 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.132 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134782 s, 77.8 MB/s 2024-04-25T10:12:39.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.148 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126929 s, 82.6 MB/s 2024-04-25T10:12:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.162 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112509 s, 93.2 MB/s 2024-04-25T10:12:39.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.178 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127326 s, 82.4 MB/s 2024-04-25T10:12:39.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.194 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125768 s, 83.4 MB/s 2024-04-25T10:12:39.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.211 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013471 s, 77.8 MB/s 2024-04-25T10:12:39.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.228 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131452 s, 79.8 MB/s 2024-04-25T10:12:39.244 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.244 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.244 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132768 s, 79.0 MB/s 2024-04-25T10:12:39.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.245 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:39.245 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:39.245 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:12:39.245 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:39.246 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:39.246 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:39.246 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:39.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.323 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.323 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122513 s, 85.6 MB/s 2024-04-25T10:12:39.338 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.338 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.338 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122288 s, 85.7 MB/s 2024-04-25T10:12:39.365 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.365 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.365 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231489 s, 45.3 MB/s 2024-04-25T10:12:39.381 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.381 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.381 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116682 s, 89.9 MB/s 2024-04-25T10:12:39.398 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.398 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.399 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144675 s, 72.5 MB/s 2024-04-25T10:12:39.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.415 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130993 s, 80.0 MB/s 2024-04-25T10:12:39.431 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.431 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.431 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131682 s, 79.6 MB/s 2024-04-25T10:12:39.445 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.446 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.446 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119477 s, 87.8 MB/s 2024-04-25T10:12:39.463 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.463 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.463 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143978 s, 72.8 MB/s 2024-04-25T10:12:39.478 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.479 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129803 s, 80.8 MB/s 2024-04-25T10:12:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130651 s, 80.3 MB/s 2024-04-25T10:12:39.510 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.511 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.511 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012899 s, 81.3 MB/s 2024-04-25T10:12:39.528 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.529 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145973 s, 71.8 MB/s 2024-04-25T10:12:39.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.544 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121432 s, 86.4 MB/s 2024-04-25T10:12:39.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.559 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110136 s, 95.2 MB/s 2024-04-25T10:12:39.573 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.573 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.573 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127051 s, 82.5 MB/s 2024-04-25T10:12:39.588 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.588 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.589 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125621 s, 83.5 MB/s 2024-04-25T10:12:39.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.606 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136079 s, 77.1 MB/s 2024-04-25T10:12:39.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.622 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125823 s, 83.3 MB/s 2024-04-25T10:12:39.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.636 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104185 s, 101 MB/s 2024-04-25T10:12:39.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.653 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133345 s, 78.6 MB/s 2024-04-25T10:12:39.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.670 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136923 s, 76.6 MB/s 2024-04-25T10:12:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.684 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999987 s, 105 MB/s 2024-04-25T10:12:39.702 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.702 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.702 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115306 s, 90.9 MB/s 2024-04-25T10:12:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.715 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011991 s, 87.4 MB/s 2024-04-25T10:12:39.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.731 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.731 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122414 s, 85.7 MB/s 2024-04-25T10:12:39.746 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.747 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119765 s, 87.6 MB/s 2024-04-25T10:12:39.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.764 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131042 s, 80.0 MB/s 2024-04-25T10:12:39.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.779 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117265 s, 89.4 MB/s 2024-04-25T10:12:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.793 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104843 s, 100 MB/s 2024-04-25T10:12:39.809 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.809 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.810 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126916 s, 82.6 MB/s 2024-04-25T10:12:39.826 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136738 s, 76.7 MB/s 2024-04-25T10:12:39.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:39.828 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:39.828 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:39.828 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:12:39.828 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:39.828 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:39.828 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:39.828 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.891 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944594 s, 111 MB/s 2024-04-25T10:12:39.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.907 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123002 s, 85.2 MB/s 2024-04-25T10:12:39.918 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.918 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.918 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870512 s, 120 MB/s 2024-04-25T10:12:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112245 s, 93.4 MB/s 2024-04-25T10:12:39.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.947 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122808 s, 85.4 MB/s 2024-04-25T10:12:39.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.962 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124161 s, 84.5 MB/s 2024-04-25T10:12:39.979 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.979 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.979 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138339 s, 75.8 MB/s 2024-04-25T10:12:39.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:39.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:39.993 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118411 s, 88.6 MB/s 2024-04-25T10:12:40.008 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.008 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.008 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121821 s, 86.1 MB/s 2024-04-25T10:12:40.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.022 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113519 s, 92.4 MB/s 2024-04-25T10:12:40.036 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.037 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117125 s, 89.5 MB/s 2024-04-25T10:12:40.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.050 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107032 s, 98.0 MB/s 2024-04-25T10:12:40.063 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.063 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.063 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107905 s, 97.2 MB/s 2024-04-25T10:12:40.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.077 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115586 s, 90.7 MB/s 2024-04-25T10:12:40.090 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.090 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.090 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956592 s, 110 MB/s 2024-04-25T10:12:40.104 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.104 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.105 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117879 s, 89.0 MB/s 2024-04-25T10:12:40.120 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.121 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.121 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134788 s, 77.8 MB/s 2024-04-25T10:12:40.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.133 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957014 s, 110 MB/s 2024-04-25T10:12:40.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.147 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108302 s, 96.8 MB/s 2024-04-25T10:12:40.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.160 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010501 s, 99.9 MB/s 2024-04-25T10:12:40.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.175 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121831 s, 86.1 MB/s 2024-04-25T10:12:40.189 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.189 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.189 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120257 s, 87.2 MB/s 2024-04-25T10:12:40.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.205 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131501 s, 79.7 MB/s 2024-04-25T10:12:40.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.219 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111238 s, 94.3 MB/s 2024-04-25T10:12:40.234 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.234 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.234 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124117 s, 84.5 MB/s 2024-04-25T10:12:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122369 s, 85.7 MB/s 2024-04-25T10:12:40.263 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.263 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.263 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117062 s, 89.6 MB/s 2024-04-25T10:12:40.277 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.277 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.277 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116148 s, 90.3 MB/s 2024-04-25T10:12:40.291 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.291 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.291 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111653 s, 93.9 MB/s 2024-04-25T10:12:40.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.304 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106272 s, 98.7 MB/s 2024-04-25T10:12:40.317 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.318 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110496 s, 94.9 MB/s 2024-04-25T10:12:40.333 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.333 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.333 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129335 s, 81.1 MB/s 2024-04-25T10:12:40.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:40.334 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:40.334 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:40.334 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:12:40.334 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:40.334 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:40.334 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:40.335 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:40.400 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.400 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.400 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116064 s, 90.3 MB/s 2024-04-25T10:12:40.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.416 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131109 s, 80.0 MB/s 2024-04-25T10:12:40.426 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.426 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.426 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769554 s, 136 MB/s 2024-04-25T10:12:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104596 s, 100 MB/s 2024-04-25T10:12:40.452 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114408 s, 91.7 MB/s 2024-04-25T10:12:40.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.466 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107424 s, 97.6 MB/s 2024-04-25T10:12:40.481 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.481 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.481 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133785 s, 78.4 MB/s 2024-04-25T10:12:40.495 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.496 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.496 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119548 s, 87.7 MB/s 2024-04-25T10:12:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887659 s, 118 MB/s 2024-04-25T10:12:40.519 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.520 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.520 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106843 s, 98.1 MB/s 2024-04-25T10:12:40.535 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.535 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.535 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126596 s, 82.8 MB/s 2024-04-25T10:12:40.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.547 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100002 s, 105 MB/s 2024-04-25T10:12:40.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996316 s, 105 MB/s 2024-04-25T10:12:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.572 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965218 s, 109 MB/s 2024-04-25T10:12:40.585 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.585 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.585 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102375 s, 102 MB/s 2024-04-25T10:12:40.600 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.600 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.600 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126836 s, 82.7 MB/s 2024-04-25T10:12:40.613 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.613 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.613 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919882 s, 114 MB/s 2024-04-25T10:12:40.627 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.627 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.627 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113794 s, 92.1 MB/s 2024-04-25T10:12:40.641 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126063 s, 83.2 MB/s 2024-04-25T10:12:40.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.655 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112841 s, 92.9 MB/s 2024-04-25T10:12:40.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.665 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788457 s, 133 MB/s 2024-04-25T10:12:40.678 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.678 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.678 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104267 s, 101 MB/s 2024-04-25T10:12:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900711 s, 116 MB/s 2024-04-25T10:12:40.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.704 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122529 s, 85.6 MB/s 2024-04-25T10:12:40.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.719 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130595 s, 80.3 MB/s 2024-04-25T10:12:40.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.730 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871481 s, 120 MB/s 2024-04-25T10:12:40.744 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.744 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.744 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111771 s, 93.8 MB/s 2024-04-25T10:12:40.758 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.758 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.758 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114842 s, 91.3 MB/s 2024-04-25T10:12:40.770 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.771 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.771 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100212 s, 105 MB/s 2024-04-25T10:12:40.784 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.784 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.784 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115195 s, 91.0 MB/s 2024-04-25T10:12:40.798 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.798 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.798 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105927 s, 99.0 MB/s 2024-04-25T10:12:40.812 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.812 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.812 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115372 s, 90.9 MB/s 2024-04-25T10:12:40.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:40.813 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:40.814 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:40.814 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:12:40.814 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:40.814 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:40.814 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:40.814 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.880 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114298 s, 91.7 MB/s 2024-04-25T10:12:40.893 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.893 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.893 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103607 s, 101 MB/s 2024-04-25T10:12:40.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.907 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116566 s, 90.0 MB/s 2024-04-25T10:12:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109025 s, 96.2 MB/s 2024-04-25T10:12:40.934 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120284 s, 87.2 MB/s 2024-04-25T10:12:40.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.947 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099334 s, 106 MB/s 2024-04-25T10:12:40.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.960 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113596 s, 92.3 MB/s 2024-04-25T10:12:40.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.973 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103862 s, 101 MB/s 2024-04-25T10:12:40.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:40.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:40.986 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109742 s, 95.5 MB/s 2024-04-25T10:12:41.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.000 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111757 s, 93.8 MB/s 2024-04-25T10:12:41.012 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.012 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.012 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985423 s, 106 MB/s 2024-04-25T10:12:41.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.027 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118838 s, 88.2 MB/s 2024-04-25T10:12:41.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.038 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924377 s, 113 MB/s 2024-04-25T10:12:41.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.050 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961049 s, 109 MB/s 2024-04-25T10:12:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.062 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876783 s, 120 MB/s 2024-04-25T10:12:41.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.073 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895781 s, 117 MB/s 2024-04-25T10:12:41.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.084 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868187 s, 121 MB/s 2024-04-25T10:12:41.098 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.098 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.098 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115615 s, 90.7 MB/s 2024-04-25T10:12:41.111 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.111 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.111 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107021 s, 98.0 MB/s 2024-04-25T10:12:41.123 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.124 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102156 s, 103 MB/s 2024-04-25T10:12:41.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.138 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119434 s, 87.8 MB/s 2024-04-25T10:12:41.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991019 s, 106 MB/s 2024-04-25T10:12:41.163 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.164 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.164 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112289 s, 93.4 MB/s 2024-04-25T10:12:41.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.176 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099019 s, 106 MB/s 2024-04-25T10:12:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110067 s, 95.3 MB/s 2024-04-25T10:12:41.203 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.203 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.203 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112301 s, 93.4 MB/s 2024-04-25T10:12:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106003 s, 98.9 MB/s 2024-04-25T10:12:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109722 s, 95.6 MB/s 2024-04-25T10:12:41.244 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.244 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.244 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119667 s, 87.6 MB/s 2024-04-25T10:12:41.260 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.260 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.260 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125664 s, 83.4 MB/s 2024-04-25T10:12:41.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.277 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122785 s, 85.4 MB/s 2024-04-25T10:12:41.291 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.292 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.292 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011873 s, 88.3 MB/s 2024-04-25T10:12:41.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.293 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:41.293 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:41.293 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:12:41.293 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:41.293 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:41.293 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:41.293 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:41.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.360 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116864 s, 89.7 MB/s 2024-04-25T10:12:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123499 s, 84.9 MB/s 2024-04-25T10:12:41.432 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.432 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.432 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155839 s, 67.3 MB/s 2024-04-25T10:12:41.447 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.447 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.447 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113165 s, 92.7 MB/s 2024-04-25T10:12:41.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.462 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118853 s, 88.2 MB/s 2024-04-25T10:12:41.478 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.478 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.478 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011751 s, 89.2 MB/s 2024-04-25T10:12:41.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.492 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985991 s, 106 MB/s 2024-04-25T10:12:41.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.507 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127804 s, 82.0 MB/s 2024-04-25T10:12:41.523 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.523 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.523 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130812 s, 80.2 MB/s 2024-04-25T10:12:41.538 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.538 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.538 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121362 s, 86.4 MB/s 2024-04-25T10:12:41.554 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.554 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.554 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129512 s, 81.0 MB/s 2024-04-25T10:12:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.568 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107994 s, 97.1 MB/s 2024-04-25T10:12:41.580 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.580 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.580 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100259 s, 105 MB/s 2024-04-25T10:12:41.594 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.594 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.594 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010707 s, 97.9 MB/s 2024-04-25T10:12:41.610 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.610 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.610 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012884 s, 81.4 MB/s 2024-04-25T10:12:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141879 s, 73.9 MB/s 2024-04-25T10:12:41.642 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.642 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.642 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119546 s, 87.7 MB/s 2024-04-25T10:12:41.659 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.659 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.659 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139994 s, 74.9 MB/s 2024-04-25T10:12:41.674 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.674 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.674 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122485 s, 85.6 MB/s 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122993 s, 85.3 MB/s 2024-04-25T10:12:41.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.704 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103401 s, 101 MB/s 2024-04-25T10:12:41.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.719 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122528 s, 85.6 MB/s 2024-04-25T10:12:41.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.734 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125137 s, 83.8 MB/s 2024-04-25T10:12:41.749 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.749 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.749 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113178 s, 92.6 MB/s 2024-04-25T10:12:41.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.764 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122569 s, 85.5 MB/s 2024-04-25T10:12:41.781 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.781 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.781 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133236 s, 78.7 MB/s 2024-04-25T10:12:41.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.797 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127063 s, 82.5 MB/s 2024-04-25T10:12:41.812 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120957 s, 86.7 MB/s 2024-04-25T10:12:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129018 s, 81.3 MB/s 2024-04-25T10:12:41.845 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.845 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.845 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129742 s, 80.8 MB/s 2024-04-25T10:12:41.860 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.860 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.860 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119725 s, 87.6 MB/s 2024-04-25T10:12:41.876 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.876 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.876 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122053 s, 85.9 MB/s 2024-04-25T10:12:41.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:41.879 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:41.879 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:41.879 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:12:41.879 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:41.879 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:41.879 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:41.879 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:41.949 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.950 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.950 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130587 s, 80.3 MB/s 2024-04-25T10:12:41.965 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.965 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.965 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126255 s, 83.1 MB/s 2024-04-25T10:12:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.981 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116847 s, 89.7 MB/s 2024-04-25T10:12:41.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:41.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:41.998 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138817 s, 75.5 MB/s 2024-04-25T10:12:42.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.018 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125819 s, 83.3 MB/s 2024-04-25T10:12:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140808 s, 74.5 MB/s 2024-04-25T10:12:42.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.048 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123643 s, 84.8 MB/s 2024-04-25T10:12:42.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.065 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01287 s, 81.5 MB/s 2024-04-25T10:12:42.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.082 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129558 s, 80.9 MB/s 2024-04-25T10:12:42.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.101 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135211 s, 77.6 MB/s 2024-04-25T10:12:42.120 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.120 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.121 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137871 s, 76.1 MB/s 2024-04-25T10:12:42.139 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.139 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.140 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142449 s, 73.6 MB/s 2024-04-25T10:12:42.158 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.158 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.158 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136049 s, 77.1 MB/s 2024-04-25T10:12:42.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.175 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136831 s, 76.6 MB/s 2024-04-25T10:12:42.192 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.192 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.192 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132942 s, 78.9 MB/s 2024-04-25T10:12:42.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.208 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.208 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117601 s, 89.2 MB/s 2024-04-25T10:12:42.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.223 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110749 s, 94.7 MB/s 2024-04-25T10:12:42.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.238 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116986 s, 89.6 MB/s 2024-04-25T10:12:42.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.256 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012816 s, 81.8 MB/s 2024-04-25T10:12:42.271 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.271 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.271 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109777 s, 95.5 MB/s 2024-04-25T10:12:42.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.288 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120242 s, 87.2 MB/s 2024-04-25T10:12:42.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.304 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121586 s, 86.2 MB/s 2024-04-25T10:12:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138121 s, 75.9 MB/s 2024-04-25T10:12:42.343 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.343 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.343 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153406 s, 68.4 MB/s 2024-04-25T10:12:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134859 s, 77.8 MB/s 2024-04-25T10:12:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.383 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139077 s, 75.4 MB/s 2024-04-25T10:12:42.456 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.456 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.457 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158447 s, 66.2 MB/s 2024-04-25T10:12:42.472 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.472 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.472 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116425 s, 90.1 MB/s 2024-04-25T10:12:42.490 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.490 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.491 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144222 s, 72.7 MB/s 2024-04-25T10:12:42.505 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114704 s, 91.4 MB/s 2024-04-25T10:12:42.523 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.523 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.523 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142816 s, 73.4 MB/s 2024-04-25T10:12:42.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.539 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013303 s, 78.8 MB/s 2024-04-25T10:12:42.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:42.540 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:42.541 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:12:42.541 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:12:42.541 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:42.541 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:42.541 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:12:42.541 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:12:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122094 s, 85.9 MB/s 2024-04-25T10:12:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.623 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.623 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122689 s, 85.5 MB/s 2024-04-25T10:12:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116056 s, 90.4 MB/s 2024-04-25T10:12:42.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.650 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968075 s, 108 MB/s 2024-04-25T10:12:42.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.666 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128014 s, 81.9 MB/s 2024-04-25T10:12:42.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.681 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124615 s, 84.1 MB/s 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.696 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118491 s, 88.5 MB/s 2024-04-25T10:12:42.711 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.711 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.711 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124768 s, 84.0 MB/s 2024-04-25T10:12:42.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.726 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124801 s, 84.0 MB/s 2024-04-25T10:12:42.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.741 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123703 s, 84.8 MB/s 2024-04-25T10:12:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129609 s, 80.9 MB/s 2024-04-25T10:12:42.770 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.770 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.770 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974786 s, 108 MB/s 2024-04-25T10:12:42.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.785 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011986 s, 87.5 MB/s 2024-04-25T10:12:42.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.800 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116948 s, 89.7 MB/s 2024-04-25T10:12:42.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.814 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117492 s, 89.2 MB/s 2024-04-25T10:12:42.828 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.829 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114395 s, 91.7 MB/s 2024-04-25T10:12:42.843 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.843 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.843 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114179 s, 91.8 MB/s 2024-04-25T10:12:42.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.856 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982375 s, 107 MB/s 2024-04-25T10:12:42.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.871 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127869 s, 82.0 MB/s 2024-04-25T10:12:42.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.886 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122718 s, 85.4 MB/s 2024-04-25T10:12:42.901 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.901 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.901 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126759 s, 82.7 MB/s 2024-04-25T10:12:42.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.916 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115613 s, 90.7 MB/s 2024-04-25T10:12:42.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.930 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123141 s, 85.2 MB/s 2024-04-25T10:12:42.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.945 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121181 s, 86.5 MB/s 2024-04-25T10:12:42.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.960 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115844 s, 90.5 MB/s 2024-04-25T10:12:42.972 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.972 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.972 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973595 s, 108 MB/s 2024-04-25T10:12:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:42.988 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133054 s, 78.8 MB/s 2024-04-25T10:12:43.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:43.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:43.004 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135621 s, 77.3 MB/s 2024-04-25T10:12:43.016 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:43.016 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:43.016 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937927 s, 112 MB/s 2024-04-25T10:12:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:43.029 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990547 s, 106 MB/s 2024-04-25T10:12:43.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:43.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:43.043 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115771 s, 90.6 MB/s 2024-04-25T10:12:43.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:12:43.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:12:43.059 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121935 s, 86.0 MB/s 2024-04-25T10:12:43.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.061 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3db512640 1 Processor -- start 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3db512640 1 -- start start 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3db512640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 0x7fa3d415b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3db512640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa3d40cc890 0x7fa3d40ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3db512640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d415bf40 con 0x7fa3d40ce690 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3db512640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d41589c0 con 0x7fa3d40cc890 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3db512640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d415c200 con 0x7fa3d415b530 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3d8aad640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa3d40cc890 0x7fa3d40ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3d9aaf640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa3d40ce690 0x7fa3d40ceae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45328/0 (socket says 172.21.15.133:45328) 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3d9aaf640 1 -- 172.21.15.133:0/1815045486 learned_addr learned my addr 172.21.15.133:0/1815045486 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/1815045486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 0x7fa3d415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3d8aad640 1 -- 172.21.15.133:0/1815045486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 msgr2=0x7fa3d415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3d8aad640 1 --2- 172.21.15.133:0/1815045486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 0x7fa3d415b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3d8aad640 1 -- 172.21.15.133:0/1815045486 >> v1:172.21.15.133:6789/0 conn(0x7fa3d40ce690 legacy=0x7fa3d40ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3d8aad640 1 -- 172.21.15.133:0/1815045486 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d415c4e0 con 0x7fa3d40cc890 2024-04-25T10:12:43.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3cbfff640 1 -- 172.21.15.133:0/1815045486 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036017662 0 0) 0x7fa3d415bf40 con 0x7fa3d40ce690 2024-04-25T10:12:43.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.233+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/1815045486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 0x7fa3d415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3d8aad640 1 --2- 172.21.15.133:0/1815045486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3d40cc890 0x7fa3d40ccd10 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40097c0 tx=0x7fa3c401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db743693e788f460 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3cbfff640 1 -- 172.21.15.133:0/1815045486 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40b0070 con 0x7fa3d40cc890 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3cbfff640 1 -- 172.21.15.133:0/1815045486 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c40ab600 con 0x7fa3d40cc890 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3cbfff640 1 -- 172.21.15.133:0/1815045486 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40ab9d0 con 0x7fa3d40cc890 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3db512640 1 -- 172.21.15.133:0/1815045486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3d40cc890 msgr2=0x7fa3d40ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3db512640 1 --2- 172.21.15.133:0/1815045486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3d40cc890 0x7fa3d40ccd10 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40097c0 tx=0x7fa3c401c410 comp rx=0 tx=0).stop 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3db512640 1 -- 172.21.15.133:0/1815045486 shutdown_connections 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3db512640 1 --2- 172.21.15.133:0/1815045486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3d40cc890 0x7fa3d40ccd10 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3db512640 1 --2- 172.21.15.133:0/1815045486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 0x7fa3d415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3db512640 1 -- 172.21.15.133:0/1815045486 >> 172.21.15.133:0/1815045486 conn(0x7fa3d406d370 msgr2=0x7fa3d406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3db512640 1 -- 172.21.15.133:0/1815045486 shutdown_connections 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3db512640 1 -- 172.21.15.133:0/1815045486 wait complete. 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3db512640 1 Processor -- start 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.237+0000 7fa3db512640 1 -- start start 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3db512640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3d40cc890 0x7fa3d4155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3db512640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3d40ce690 0x7fa3d40acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3db512640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 0x7fa3d414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3db512640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d415dba0 con 0x7fa3d40cc890 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3db512640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d415da20 con 0x7fa3d40ce690 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3db512640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d415dd20 con 0x7fa3d415b530 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d92ae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3d40cc890 0x7fa3d4155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d92ae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3d40cc890 0x7fa3d4155440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56866/0 (socket says 172.21.15.133:56866) 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d92ae640 1 -- 172.21.15.133:0/4157554397 learned_addr learned my addr 172.21.15.133:0/4157554397 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d9aaf640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 0x7fa3d414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d92ae640 1 -- 172.21.15.133:0/4157554397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 msgr2=0x7fa3d414e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d8aad640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3d40ce690 0x7fa3d40acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 0x7fa3d414e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d92ae640 1 -- 172.21.15.133:0/4157554397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3d40ce690 msgr2=0x7fa3d40acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3d40ce690 0x7fa3d40acaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d92ae640 1 -- 172.21.15.133:0/4157554397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d414d360 con 0x7fa3d40cc890 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d9aaf640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 0x7fa3d414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3d40cc890 0x7fa3d4155440 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000c7f0 tx=0x7fa3d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3c9ffb640 1 -- 172.21.15.133:0/4157554397 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0015070 con 0x7fa3d40cc890 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3c9ffb640 1 -- 172.21.15.133:0/4157554397 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0007820 con 0x7fa3d40cc890 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3db512640 1 -- 172.21.15.133:0/4157554397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d4168a60 con 0x7fa3d40cc890 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3c9ffb640 1 -- 172.21.15.133:0/4157554397 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0007c10 con 0x7fa3d40cc890 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3db512640 1 -- 172.21.15.133:0/4157554397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d4169020 con 0x7fa3d40cc890 2024-04-25T10:12:43.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3c9ffb640 1 -- 172.21.15.133:0/4157554397 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3d0018070 con 0x7fa3d40cc890 2024-04-25T10:12:43.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3c9ffb640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3b8066f60 0x7fa3b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3c9ffb640 1 -- 172.21.15.133:0/4157554397 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3d0085280 con 0x7fa3d40cc890 2024-04-25T10:12:43.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.241+0000 7fa3d8aad640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3b8066f60 0x7fa3b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3d8aad640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3b8066f60 0x7fa3b8069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40ad000 tx=0x7fa3c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 Processor -- start 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 -- start start 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065907d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306590d10 0x558306546fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306547520 0x558306547900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558306394d50 con 0x5583065903f0 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830631ce40 con 0x558306590d10 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x558306390df0 con 0x558306547520 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3d92ae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065907d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3d92ae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065907d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56876/0 (socket says 172.21.15.133:56876) 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3d92ae640 1 -- 172.21.15.133:0/3693689199 learned_addr learned my addr 172.21.15.133:0/3693689199 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3d8aad640 1 --2- 172.21.15.133:0/3693689199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306590d10 0x558306546fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3d92ae640 1 -- 172.21.15.133:0/3693689199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306547520 msgr2=0x558306547900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/3693689199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306547520 0x558306547900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3d92ae640 1 -- 172.21.15.133:0/3693689199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306590d10 msgr2=0x558306546fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/3693689199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306590d10 0x558306546fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3d92ae640 1 -- 172.21.15.133:0/3693689199 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558306331d90 con 0x5583065903f0 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/3693689199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065907d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa3d005dbe0 tx=0x7fa3d0055b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9b89889a5a0066 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3beffd640 1 -- 172.21.15.133:0/3693689199 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d001f040 con 0x5583065903f0 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3beffd640 1 -- 172.21.15.133:0/3693689199 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0053d00 con 0x5583065903f0 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3beffd640 1 -- 172.21.15.133:0/3693689199 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0054e50 con 0x5583065903f0 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 -- 172.21.15.133:0/3693689199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 msgr2=0x5583065907d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 --2- 172.21.15.133:0/3693689199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065907d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa3d005dbe0 tx=0x7fa3d0055b40 comp rx=0 tx=0).stop 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 -- 172.21.15.133:0/3693689199 shutdown_connections 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 --2- 172.21.15.133:0/3693689199 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306547520 0x558306547900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 --2- 172.21.15.133:0/3693689199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306590d10 0x558306546fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 --2- 172.21.15.133:0/3693689199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065907d0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 -- 172.21.15.133:0/3693689199 >> 172.21.15.133:0/3693689199 conn(0x5583065434d0 msgr2=0x5583065458c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 -- 172.21.15.133:0/3693689199 shutdown_connections 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 -- 172.21.15.133:0/3693689199 wait complete. 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 -- start start 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.245+0000 7fa3de67f000 1 Processor -- start 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306547520 0x55830654d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5583065903f0 0x55830657e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306590d10 0x55830657e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830633a610 con 0x558306590d10 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830634f9a0 con 0x5583065903f0 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x558306399380 con 0x558306547520 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 10 client.0 _ll_register_callbacks cb 0x5583064490d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3d9aaf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306590d10 0x55830657e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3d9aaf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306590d10 0x55830657e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56892/0 (socket says 172.21.15.133:56892) 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3d9aaf640 1 -- 172.21.15.133:0/4267158810 learned_addr learned my addr 172.21.15.133:0/4267158810 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3d9aaf640 1 -- 172.21.15.133:0/4267158810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306547520 msgr2=0x55830654d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3d9aaf640 1 --2- 172.21.15.133:0/4267158810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306547520 0x55830654d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3d9aaf640 1 -- 172.21.15.133:0/4267158810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5583065903f0 msgr2=0x55830657e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3d9aaf640 1 --2- 172.21.15.133:0/4267158810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5583065903f0 0x55830657e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3d9aaf640 1 -- 172.21.15.133:0/4267158810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55830638fd40 con 0x558306590d10 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3d9aaf640 1 --2- 172.21.15.133:0/4267158810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306590d10 0x55830657e600 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00d480 tx=0x7fa3cc00d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd5e48c87c94e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3b3fff640 1 -- 172.21.15.133:0/4267158810 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0070d0 con 0x558306590d10 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3b3fff640 1 -- 172.21.15.133:0/4267158810 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3cc013c10 con 0x558306590d10 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 -- 172.21.15.133:0/4267158810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3d0053d00 con 0x558306590d10 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3b3fff640 1 -- 172.21.15.133:0/4267158810 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3cc013e10 con 0x558306590d10 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 10 client.9090 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 -- 172.21.15.133:0/4267158810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x558306417f00 con 0x558306590d10 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3b3fff640 1 -- 172.21.15.133:0/4267158810 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0053d00 con 0x558306590d10 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3b3fff640 1 -- 172.21.15.133:0/4267158810 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa3cc013420 con 0x558306590d10 2024-04-25T10:12:43.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 10 client.9090 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 10 client.9090 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 20 client.9090 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 --2- 172.21.15.133:0/4267158810 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x558306597360 0x558306597770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 4 client.9090 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 -- 172.21.15.133:0/4267158810 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558306330a20 con 0x558306597360 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/4267158810 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x558306597360 0x558306597770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/4267158810 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x558306597360 0x558306597770 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc47bacea156bb28 server_cookie=6decaef463f3692d in_seq=0 out_seq=0 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3b3fff640 10 client.9090 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3b3fff640 1 -- 172.21.15.133:0/4267158810 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558306330a20 con 0x558306597360 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3b3fff640 10 client.9090 handle_command_reply: tid=0 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 client.9090 shutdown 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 -- 172.21.15.133:0/4267158810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306590d10 msgr2=0x55830657e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 --2- 172.21.15.133:0/4267158810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306590d10 0x55830657e600 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00d480 tx=0x7fa3cc00d940 comp rx=0 tx=0).stop 2024-04-25T10:12:43.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 -- 172.21.15.133:0/4267158810 shutdown_connections 2024-04-25T10:12:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 --2- 172.21.15.133:0/4267158810 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x558306597360 0x558306597770 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 --2- 172.21.15.133:0/4267158810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306590d10 0x55830657e600 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 --2- 172.21.15.133:0/4267158810 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5583065903f0 0x55830657e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 --2- 172.21.15.133:0/4267158810 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306547520 0x55830654d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3b3fff640 0 client.9090 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 -- 172.21.15.133:0/4267158810 >> 172.21.15.133:0/4267158810 conn(0x5583065434d0 msgr2=0x558306546150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 -- 172.21.15.133:0/4267158810 shutdown_connections 2024-04-25T10:12:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 1 -- 172.21.15.133:0/4267158810 wait complete. 2024-04-25T10:12:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.249+0000 7fa3de67f000 20 client.9090 trim_cache size 0 max 16384 2024-04-25T10:12:43.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3de67f000 1 Processor -- start 2024-04-25T10:12:43.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3de67f000 1 -- start start 2024-04-25T10:12:43.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306547520 0x5583065f5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5583065903f0 0x5583065f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306590d10 0x5583065f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3de67f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d42030b0 con 0x558306547520 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3de67f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d42022b0 con 0x558306590d10 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3de67f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d41fa330 con 0x5583065903f0 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3d92ae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306547520 0x5583065f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3d92ae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306547520 0x5583065f5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56902/0 (socket says 172.21.15.133:56902) 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3d92ae640 1 -- 172.21.15.133:0/329305325 learned_addr learned my addr 172.21.15.133:0/329305325 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3d92ae640 1 -- 172.21.15.133:0/329305325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5583065903f0 msgr2=0x5583065f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/329305325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5583065903f0 0x5583065f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3d92ae640 1 -- 172.21.15.133:0/329305325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306590d10 msgr2=0x5583065f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/329305325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306590d10 0x5583065f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3d92ae640 1 -- 172.21.15.133:0/329305325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558306590b40 con 0x558306547520 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/329305325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306547520 0x5583065f5660 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0055700 tx=0x7fa3d0085050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a66f5be31c11ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3be7fc640 1 -- 172.21.15.133:0/329305325 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d001f040 con 0x558306547520 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3be7fc640 1 -- 172.21.15.133:0/329305325 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0080070 con 0x558306547520 2024-04-25T10:12:43.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.265+0000 7fa3be7fc640 1 -- 172.21.15.133:0/329305325 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0055090 con 0x558306547520 2024-04-25T10:12:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- 172.21.15.133:0/329305325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306547520 msgr2=0x5583065f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 --2- 172.21.15.133:0/329305325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306547520 0x5583065f5660 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0055700 tx=0x7fa3d0085050 comp rx=0 tx=0).stop 2024-04-25T10:12:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- 172.21.15.133:0/329305325 shutdown_connections 2024-04-25T10:12:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 --2- 172.21.15.133:0/329305325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306590d10 0x5583065f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 --2- 172.21.15.133:0/329305325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5583065903f0 0x5583065f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 --2- 172.21.15.133:0/329305325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558306547520 0x5583065f5660 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- 172.21.15.133:0/329305325 >> 172.21.15.133:0/329305325 conn(0x5583065434d0 msgr2=0x55830654c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- 172.21.15.133:0/329305325 shutdown_connections 2024-04-25T10:12:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- 172.21.15.133:0/329305325 wait complete. 2024-04-25T10:12:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- start start 2024-04-25T10:12:43.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 Processor -- start 2024-04-25T10:12:43.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306547520 0x5583065f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065fa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306590d10 0x558306595420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830634bdc0 con 0x5583065903f0 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d4201e80 con 0x558306547520 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x558306394060 con 0x558306590d10 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 10 client.0 _ll_register_callbacks cb 0x7fa3d0051d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d8aad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065fa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d8aad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065fa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56918/0 (socket says 172.21.15.133:56918) 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d8aad640 1 -- 172.21.15.133:0/429136114 learned_addr learned my addr 172.21.15.133:0/429136114 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306547520 0x5583065f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d8aad640 1 -- 172.21.15.133:0/429136114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306590d10 msgr2=0x558306595420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d9aaf640 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306590d10 0x558306595420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d8aad640 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306590d10 0x558306595420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d8aad640 1 -- 172.21.15.133:0/429136114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306547520 msgr2=0x5583065f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d8aad640 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306547520 0x5583065f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d8aad640 1 -- 172.21.15.133:0/429136114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583063ee330 con 0x5583065903f0 2024-04-25T10:12:43.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306547520 0x5583065f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d8aad640 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065fa170 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4097df0 tx=0x7fa3c40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65115086133788a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3b3fff640 1 -- 172.21.15.133:0/429136114 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40b7840 con 0x5583065903f0 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3b3fff640 1 -- 172.21.15.133:0/429136114 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3c40b79e0 con 0x5583065903f0 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- 172.21.15.133:0/429136114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3d0080070 con 0x5583065903f0 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3b3fff640 1 -- 172.21.15.133:0/429136114 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c40ce050 con 0x5583065903f0 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 10 client.9096 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- 172.21.15.133:0/429136114 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5583063615f0 con 0x5583065903f0 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3b3fff640 1 -- 172.21.15.133:0/429136114 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0080070 con 0x5583065903f0 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3b3fff640 1 -- 172.21.15.133:0/429136114 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa3c40d1020 con 0x5583065903f0 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 10 client.9096 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 10 client.9096 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 20 client.9096 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:43.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x558306596150 0x558306596560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 4 client.9096 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:12:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3de67f000 1 -- 172.21.15.133:0/429136114 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5583063a2290 con 0x558306596150 2024-04-25T10:12:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x558306596150 0x558306596560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3b3fff640 10 client.9096 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.269+0000 7fa3d92ae640 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x558306596150 0x558306596560 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b784f7d01765c26 server_cookie=b8ce836618f1f83d in_seq=0 out_seq=0 2024-04-25T10:12:43.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3b3fff640 1 -- 172.21.15.133:0/429136114 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5583063a2290 con 0x558306596150 2024-04-25T10:12:43.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3b3fff640 10 client.9096 handle_command_reply: tid=0 2024-04-25T10:12:43.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 1 client.9096 shutdown 2024-04-25T10:12:43.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 1 -- 172.21.15.133:0/429136114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 msgr2=0x5583065fa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065fa170 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4097df0 tx=0x7fa3c40191b0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 1 -- 172.21.15.133:0/429136114 shutdown_connections 2024-04-25T10:12:43.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x558306596150 0x558306596560 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558306590d10 0x558306595420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583065903f0 0x5583065fa170 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 1 --2- 172.21.15.133:0/429136114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558306547520 0x5583065f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3b3fff640 0 client.9096 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 1 -- 172.21.15.133:0/429136114 >> 172.21.15.133:0/429136114 conn(0x5583065434d0 msgr2=0x55830658f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 1 -- 172.21.15.133:0/429136114 shutdown_connections 2024-04-25T10:12:43.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 1 -- 172.21.15.133:0/429136114 wait complete. 2024-04-25T10:12:43.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3de67f000 20 client.9096 trim_cache size 0 max 16384 2024-04-25T10:12:43.280 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:12:43.280 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:12:43.280 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "7562f27d-7d3d-4d02-b12f-6139760d3f73", 2024-04-25T10:12:43.281 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:12:43.281 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:12:43.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 -- 172.21.15.133:0/4157554397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3b8066f60 msgr2=0x7fa3b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3b8066f60 0x7fa3b8069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40ad000 tx=0x7fa3c4008040 comp rx=0 tx=0).stop 2024-04-25T10:12:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 -- 172.21.15.133:0/4157554397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3d40cc890 msgr2=0x7fa3d4155440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3d40cc890 0x7fa3d4155440 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000c7f0 tx=0x7fa3d000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 -- 172.21.15.133:0/4157554397 shutdown_connections 2024-04-25T10:12:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa3d415b530 0x7fa3d414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa3d40ce690 0x7fa3d40acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa3b8066f60 0x7fa3b8069420 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 --2- 172.21.15.133:0/4157554397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3d40cc890 0x7fa3d4155440 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 -- 172.21.15.133:0/4157554397 >> 172.21.15.133:0/4157554397 conn(0x7fa3d406d370 msgr2=0x7fa3d40ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 -- 172.21.15.133:0/4157554397 shutdown_connections 2024-04-25T10:12:43.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.273+0000 7fa3db512640 1 -- 172.21.15.133:0/4157554397 wait complete. 2024-04-25T10:12:43.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.297 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T10:12:43.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.425+0000 7f7e5d49d640 1 Processor -- start 2024-04-25T10:12:43.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e5d49d640 1 -- start start 2024-04-25T10:12:43.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e5d49d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e5815dc00 0x7f7e5806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e5d49d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7e580cc7d0 0x7f7e580b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e5d49d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5806b4e0 con 0x7f7e580b0af0 2024-04-25T10:12:43.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e5d49d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e580b0f80 con 0x7f7e580cc7d0 2024-04-25T10:12:43.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e5d49d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5806baa0 con 0x7f7e5815dc00 2024-04-25T10:12:43.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e4ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7e580cc7d0 0x7f7e580b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e4ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7e580cc7d0 0x7f7e580b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56930/0 (socket says 172.21.15.133:56930) 2024-04-25T10:12:43.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e4ffff640 1 -- 172.21.15.133:0/3348122441 learned_addr learned my addr 172.21.15.133:0/3348122441 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e577fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7e580b0af0 0x7f7e580cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45336/0 (socket says 172.21.15.133:45336) 2024-04-25T10:12:43.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/3348122441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e5815dc00 0x7f7e5806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e4ffff640 1 -- 172.21.15.133:0/3348122441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e5815dc00 msgr2=0x7f7e5806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/3348122441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e5815dc00 0x7f7e5806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e4ffff640 1 -- 172.21.15.133:0/3348122441 >> v1:172.21.15.133:6789/0 conn(0x7f7e580b0af0 legacy=0x7f7e580cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e4ffff640 1 -- 172.21.15.133:0/3348122441 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5806b7c0 con 0x7f7e580cc7d0 2024-04-25T10:12:43.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/3348122441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e5815dc00 0x7f7e5806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:43.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/3348122441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e580cc7d0 0x7f7e580b05b0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009870 tx=0x7f7e4001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5c43541d223252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e567fc640 1 -- 172.21.15.133:0/3348122441 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e400b0070 con 0x7f7e580cc7d0 2024-04-25T10:12:43.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e567fc640 1 -- 172.21.15.133:0/3348122441 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e400ab600 con 0x7f7e580cc7d0 2024-04-25T10:12:43.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.429+0000 7f7e567fc640 1 -- 172.21.15.133:0/3348122441 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e400ab9d0 con 0x7f7e580cc7d0 2024-04-25T10:12:43.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 -- 172.21.15.133:0/3348122441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e580cc7d0 msgr2=0x7f7e580b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 --2- 172.21.15.133:0/3348122441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e580cc7d0 0x7f7e580b05b0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009870 tx=0x7f7e4001c4d0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 -- 172.21.15.133:0/3348122441 shutdown_connections 2024-04-25T10:12:43.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 --2- 172.21.15.133:0/3348122441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e580cc7d0 0x7f7e580b05b0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 --2- 172.21.15.133:0/3348122441 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e5815dc00 0x7f7e5806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 -- 172.21.15.133:0/3348122441 >> 172.21.15.133:0/3348122441 conn(0x7f7e580d0610 msgr2=0x7f7e580d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 -- 172.21.15.133:0/3348122441 shutdown_connections 2024-04-25T10:12:43.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 -- 172.21.15.133:0/3348122441 wait complete. 2024-04-25T10:12:43.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 Processor -- start 2024-04-25T10:12:43.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 -- start start 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e580b0af0 0x7f7e580ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e580cc7d0 0x7f7e580c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e5815dc00 0x7f7e580c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580cb5f0 con 0x7f7e580b0af0 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580cb470 con 0x7f7e5815dc00 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580cb770 con 0x7f7e580cc7d0 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e56ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e580b0af0 0x7f7e580ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e56ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e580b0af0 0x7f7e580ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56938/0 (socket says 172.21.15.133:56938) 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e56ffd640 1 -- 172.21.15.133:0/597407597 learned_addr learned my addr 172.21.15.133:0/597407597 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e56ffd640 1 -- 172.21.15.133:0/597407597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e580cc7d0 msgr2=0x7f7e580c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e580cc7d0 0x7f7e580c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e56ffd640 1 -- 172.21.15.133:0/597407597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e5815dc00 msgr2=0x7f7e580c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e5815dc00 0x7f7e580c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e56ffd640 1 -- 172.21.15.133:0/597407597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e581548c0 con 0x7f7e580b0af0 2024-04-25T10:12:43.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e580b0af0 0x7f7e580ca3f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800c7f0 tx=0x7f7e4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e4f7fe640 1 -- 172.21.15.133:0/597407597 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48015070 con 0x7f7e580b0af0 2024-04-25T10:12:43.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e4f7fe640 1 -- 172.21.15.133:0/597407597 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e48007820 con 0x7f7e580b0af0 2024-04-25T10:12:43.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 -- 172.21.15.133:0/597407597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e58154ba0 con 0x7f7e580b0af0 2024-04-25T10:12:43.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e4f7fe640 1 -- 172.21.15.133:0/597407597 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48007c10 con 0x7f7e580b0af0 2024-04-25T10:12:43.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.433+0000 7f7e5d49d640 1 -- 172.21.15.133:0/597407597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e58155160 con 0x7f7e580b0af0 2024-04-25T10:12:43.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4f7fe640 1 -- 172.21.15.133:0/597407597 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e48018070 con 0x7f7e580b0af0 2024-04-25T10:12:43.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4f7fe640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e38066f60 0x7f7e38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e38066f60 0x7f7e38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4f7fe640 1 -- 172.21.15.133:0/597407597 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e480853d0 con 0x7f7e580b0af0 2024-04-25T10:12:43.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e38066f60 0x7f7e38069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7e400ad000 tx=0x7f7e40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e6060a000 1 Processor -- start 2024-04-25T10:12:43.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e6060a000 1 -- start start 2024-04-25T10:12:43.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae967a40 0x5571ae967e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae922850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 0x5571ae923170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e6060a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ae8051f0 con 0x5571ae968360 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e6060a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ae76be90 con 0x5571ae967a40 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e6060a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571ae6f3f50 con 0x5571ae922d90 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae922850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae922850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56952/0 (socket says 172.21.15.133:56952) 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4ffff640 1 -- 172.21.15.133:0/881016035 learned_addr learned my addr 172.21.15.133:0/881016035 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4ffff640 1 -- 172.21.15.133:0/881016035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 msgr2=0x5571ae923170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/881016035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 0x5571ae923170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4ffff640 1 -- 172.21.15.133:0/881016035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae967a40 msgr2=0x5571ae967e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/881016035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae967a40 0x5571ae967e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/881016035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae967a40 0x5571ae967e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4ffff640 1 -- 172.21.15.133:0/881016035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571ae7438b0 con 0x5571ae968360 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/881016035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae922850 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f7e400ae040 tx=0x7f7e400a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ab11cf5992c7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4cff9640 1 -- 172.21.15.133:0/881016035 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e400c6070 con 0x5571ae968360 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4cff9640 1 -- 172.21.15.133:0/881016035 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e40004030 con 0x5571ae968360 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.437+0000 7f7e4cff9640 1 -- 172.21.15.133:0/881016035 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e400b0040 con 0x5571ae968360 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- 172.21.15.133:0/881016035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 msgr2=0x5571ae922850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- 172.21.15.133:0/881016035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae922850 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f7e400ae040 tx=0x7f7e400a8000 comp rx=0 tx=0).stop 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- 172.21.15.133:0/881016035 shutdown_connections 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- 172.21.15.133:0/881016035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 0x5571ae923170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- 172.21.15.133:0/881016035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae922850 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- 172.21.15.133:0/881016035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae967a40 0x5571ae967e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- 172.21.15.133:0/881016035 >> 172.21.15.133:0/881016035 conn(0x5571ae91ed40 msgr2=0x5571ae921130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- 172.21.15.133:0/881016035 shutdown_connections 2024-04-25T10:12:43.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- 172.21.15.133:0/881016035 wait complete. 2024-04-25T10:12:43.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- start start 2024-04-25T10:12:43.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 Processor -- start 2024-04-25T10:12:43.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 0x5571ae96f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae967a40 0x5571ae96f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae928e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ae726540 con 0x5571ae968360 2024-04-25T10:12:43.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ae767f30 con 0x5571ae967a40 2024-04-25T10:12:43.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571ae7387c0 con 0x5571ae922d90 2024-04-25T10:12:43.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 10 client.0 _ll_register_callbacks cb 0x5571ae820180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e577fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae928e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e577fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae928e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56956/0 (socket says 172.21.15.133:56956) 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e577fe640 1 -- 172.21.15.133:0/3068995624 learned_addr learned my addr 172.21.15.133:0/3068995624 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae967a40 0x5571ae96f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 0x5571ae96f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e577fe640 1 -- 172.21.15.133:0/3068995624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 msgr2=0x5571ae96f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e577fe640 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 0x5571ae96f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e577fe640 1 -- 172.21.15.133:0/3068995624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae967a40 msgr2=0x5571ae96f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e577fe640 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae967a40 0x5571ae96f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e577fe640 1 -- 172.21.15.133:0/3068995624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571ae6f41a0 con 0x5571ae968360 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae967a40 0x5571ae96f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e577fe640 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae928e90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c3f0 tx=0x7f7e5000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5115df4321932c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:43.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e35ffb640 1 -- 172.21.15.133:0/3068995624 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50017060 con 0x5571ae968360 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e35ffb640 1 -- 172.21.15.133:0/3068995624 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e500070d0 con 0x5571ae968360 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- 172.21.15.133:0/3068995624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e40004030 con 0x5571ae968360 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e35ffb640 1 -- 172.21.15.133:0/3068995624 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e5001d050 con 0x5571ae968360 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 10 client.9111 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- 172.21.15.133:0/3068995624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5571ae7ee840 con 0x5571ae968360 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e35ffb640 1 -- 172.21.15.133:0/3068995624 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e40004030 con 0x5571ae968360 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e35ffb640 1 -- 172.21.15.133:0/3068995624 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7e50020020 con 0x5571ae968360 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 10 client.9111 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 10 client.9111 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 20 client.9111 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5571ae929bd0 0x5571ae929fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 4 client.9111 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- 172.21.15.133:0/3068995624 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571ae707be0 con 0x5571ae929bd0 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5571ae929bd0 0x5571ae929fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5571ae929bd0 0x5571ae929fe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b466b46ab5d0b5 server_cookie=63c2b035f674402f in_seq=0 out_seq=0 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e35ffb640 10 client.9111 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e35ffb640 1 -- 172.21.15.133:0/3068995624 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5571ae707be0 con 0x5571ae929bd0 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e35ffb640 10 client.9111 handle_command_reply: tid=0 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 client.9111 shutdown 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- 172.21.15.133:0/3068995624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 msgr2=0x5571ae928e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae928e90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c3f0 tx=0x7f7e5000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- 172.21.15.133:0/3068995624 shutdown_connections 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5571ae929bd0 0x5571ae929fe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae968360 0x5571ae928e90 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae967a40 0x5571ae96f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e35ffb640 0 client.9111 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 --2- 172.21.15.133:0/3068995624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 0x5571ae96f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.441+0000 7f7e6060a000 1 -- 172.21.15.133:0/3068995624 >> 172.21.15.133:0/3068995624 conn(0x5571ae91ed40 msgr2=0x5571ae9219c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.445+0000 7f7e6060a000 1 -- 172.21.15.133:0/3068995624 shutdown_connections 2024-04-25T10:12:43.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.445+0000 7f7e6060a000 1 -- 172.21.15.133:0/3068995624 wait complete. 2024-04-25T10:12:43.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.445+0000 7f7e6060a000 20 client.9111 trim_cache size 0 max 16384 2024-04-25T10:12:43.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 Processor -- start 2024-04-25T10:12:43.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 -- start start 2024-04-25T10:12:43.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 0x5571ae9c4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae967a40 0x5571ae9c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae968360 0x5571ae9c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e58207920 con 0x5571ae967a40 2024-04-25T10:12:43.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e58206b50 con 0x5571ae968360 2024-04-25T10:12:43.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580c9090 con 0x5571ae922d90 2024-04-25T10:12:43.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e4ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae967a40 0x5571ae9c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e4ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae967a40 0x5571ae9c45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56970/0 (socket says 172.21.15.133:56970) 2024-04-25T10:12:43.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e4ffff640 1 -- 172.21.15.133:0/4243445962 learned_addr learned my addr 172.21.15.133:0/4243445962 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e4ffff640 1 -- 172.21.15.133:0/4243445962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 msgr2=0x5571ae9c4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/4243445962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 0x5571ae9c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e4ffff640 1 -- 172.21.15.133:0/4243445962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae968360 msgr2=0x5571ae9c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/4243445962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae968360 0x5571ae9c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e4ffff640 1 -- 172.21.15.133:0/4243445962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571ae7c5a10 con 0x5571ae967a40 2024-04-25T10:12:43.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/4243445962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae967a40 0x5571ae9c45c0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f7e4001c9b0 tx=0x7f7e40004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9bca7ae7a93e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e37fff640 1 -- 172.21.15.133:0/4243445962 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e400cf070 con 0x5571ae967a40 2024-04-25T10:12:43.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e37fff640 1 -- 172.21.15.133:0/4243445962 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e40004230 con 0x5571ae967a40 2024-04-25T10:12:43.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e37fff640 1 -- 172.21.15.133:0/4243445962 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e400c6040 con 0x5571ae967a40 2024-04-25T10:12:43.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 -- 172.21.15.133:0/4243445962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae967a40 msgr2=0x5571ae9c45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 --2- 172.21.15.133:0/4243445962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae967a40 0x5571ae9c45c0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f7e4001c9b0 tx=0x7f7e40004740 comp rx=0 tx=0).stop 2024-04-25T10:12:43.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 -- 172.21.15.133:0/4243445962 shutdown_connections 2024-04-25T10:12:43.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 --2- 172.21.15.133:0/4243445962 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae968360 0x5571ae9c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 --2- 172.21.15.133:0/4243445962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae967a40 0x5571ae9c45c0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 --2- 172.21.15.133:0/4243445962 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae922d90 0x5571ae9c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 -- 172.21.15.133:0/4243445962 >> 172.21.15.133:0/4243445962 conn(0x5571ae91ed40 msgr2=0x5571ae9661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 -- 172.21.15.133:0/4243445962 shutdown_connections 2024-04-25T10:12:43.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.457+0000 7f7e6060a000 1 -- 172.21.15.133:0/4243445962 wait complete. 2024-04-25T10:12:43.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- start start 2024-04-25T10:12:43.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 Processor -- start 2024-04-25T10:12:43.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae922d90 0x5571ae96ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae967a40 0x5571ae96d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae968360 0x5571ae96d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ae767f30 con 0x5571ae922d90 2024-04-25T10:12:43.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580c9690 con 0x5571ae968360 2024-04-25T10:12:43.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5571ae72f720 con 0x5571ae967a40 2024-04-25T10:12:43.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 10 client.0 _ll_register_callbacks cb 0x5571ae72ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:43.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e56ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae922d90 0x5571ae96ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e56ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae922d90 0x5571ae96ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56982/0 (socket says 172.21.15.133:56982) 2024-04-25T10:12:43.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e56ffd640 1 -- 172.21.15.133:0/573289784 learned_addr learned my addr 172.21.15.133:0/573289784 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae967a40 0x5571ae96d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e56ffd640 1 -- 172.21.15.133:0/573289784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae967a40 msgr2=0x5571ae96d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae967a40 0x5571ae96d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e56ffd640 1 -- 172.21.15.133:0/573289784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae968360 msgr2=0x5571ae96d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae968360 0x5571ae96d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e56ffd640 1 -- 172.21.15.133:0/573289784 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571ae700b00 con 0x5571ae922d90 2024-04-25T10:12:43.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e56ffd640 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae922d90 0x5571ae96ce60 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f7e48051f10 tx=0x7f7e48051e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae7f47e1929d5bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:43.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e35ffb640 1 -- 172.21.15.133:0/573289784 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4806d030 con 0x5571ae922d90 2024-04-25T10:12:43.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e35ffb640 1 -- 172.21.15.133:0/573289784 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e48052120 con 0x5571ae922d90 2024-04-25T10:12:43.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- 172.21.15.133:0/573289784 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e40004230 con 0x5571ae922d90 2024-04-25T10:12:43.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e35ffb640 1 -- 172.21.15.133:0/573289784 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e4801f040 con 0x5571ae922d90 2024-04-25T10:12:43.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 10 client.9117 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:43.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- 172.21.15.133:0/573289784 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5571ae712f80 con 0x5571ae922d90 2024-04-25T10:12:43.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e35ffb640 1 -- 172.21.15.133:0/573289784 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e40004230 con 0x5571ae922d90 2024-04-25T10:12:43.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e35ffb640 1 -- 172.21.15.133:0/573289784 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7e48083b30 con 0x5571ae922d90 2024-04-25T10:12:43.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 10 client.9117 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:43.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 10 client.9117 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:43.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 20 client.9117 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:43.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5571ae9cdcd0 0x5571ae9d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 4 client.9117 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:12:43.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- 172.21.15.133:0/573289784 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x5571ae7793d0 con 0x5571ae9cdcd0 2024-04-25T10:12:43.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5571ae9cdcd0 0x5571ae9d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e35ffb640 10 client.9117 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e4ffff640 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5571ae9cdcd0 0x5571ae9d00c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa824a5e95bbade9 server_cookie=d23201175906bf64 in_seq=0 out_seq=0 2024-04-25T10:12:43.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e35ffb640 1 -- 172.21.15.133:0/573289784 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5571ae7793d0 con 0x5571ae9cdcd0 2024-04-25T10:12:43.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e35ffb640 10 client.9117 handle_command_reply: tid=0 2024-04-25T10:12:43.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 client.9117 shutdown 2024-04-25T10:12:43.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- 172.21.15.133:0/573289784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae922d90 msgr2=0x5571ae96ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae922d90 0x5571ae96ce60 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f7e48051f10 tx=0x7f7e48051e50 comp rx=0 tx=0).stop 2024-04-25T10:12:43.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- 172.21.15.133:0/573289784 shutdown_connections 2024-04-25T10:12:43.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5571ae9cdcd0 0x5571ae9d00c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5571ae968360 0x5571ae96d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5571ae967a40 0x5571ae96d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e35ffb640 0 client.9117 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 --2- 172.21.15.133:0/573289784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5571ae922d90 0x5571ae96ce60 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- 172.21.15.133:0/573289784 >> 172.21.15.133:0/573289784 conn(0x5571ae91ed40 msgr2=0x5571ae966a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- 172.21.15.133:0/573289784 shutdown_connections 2024-04-25T10:12:43.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 1 -- 172.21.15.133:0/573289784 wait complete. 2024-04-25T10:12:43.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e6060a000 20 client.9117 trim_cache size 0 max 16384 2024-04-25T10:12:43.482 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:12:43.482 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:12:43.482 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:12:43.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 -- 172.21.15.133:0/597407597 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e38066f60 msgr2=0x7f7e38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e38066f60 0x7f7e38069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7e400ad000 tx=0x7f7e40008040 comp rx=0 tx=0).stop 2024-04-25T10:12:43.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 -- 172.21.15.133:0/597407597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e580b0af0 msgr2=0x7f7e580ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e580b0af0 0x7f7e580ca3f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800c7f0 tx=0x7f7e4800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 -- 172.21.15.133:0/597407597 shutdown_connections 2024-04-25T10:12:43.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7e5815dc00 0x7f7e580c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7e580cc7d0 0x7f7e580c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7e38066f60 0x7f7e38069420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 --2- 172.21.15.133:0/597407597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e580b0af0 0x7f7e580ca3f0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 -- 172.21.15.133:0/597407597 >> 172.21.15.133:0/597407597 conn(0x7f7e580d0610 msgr2=0x7f7e580b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 -- 172.21.15.133:0/597407597 shutdown_connections 2024-04-25T10:12:43.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.461+0000 7f7e5d49d640 1 -- 172.21.15.133:0/597407597 wait complete. 2024-04-25T10:12:43.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:43.486 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:12:43.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.609+0000 7f2cf4fa2640 1 Processor -- start 2024-04-25T10:12:43.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.609+0000 7f2cf4fa2640 1 -- start start 2024-04-25T10:12:43.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cf4fa2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf00cc0b0 0x7f2cf00cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cf4fa2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2cf006b3a0 0x7f2cf00cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cf4fa2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf01617c0 con 0x7f2cf00cc490 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cf4fa2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf01623d0 con 0x7f2cf006b3a0 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cf4fa2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf00d03d0 con 0x7f2cf00cc0b0 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cedd9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2cf00cc490 0x7f2cf006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45346/0 (socket says 172.21.15.133:45346) 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cedd9b640 1 -- 172.21.15.133:0/3378835111 learned_addr learned my addr 172.21.15.133:0/3378835111 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391152952 0 0) 0x7f2cf01617c0 con 0x7f2cf00cc490 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cee59c640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf00cc0b0 0x7f2cf00cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ceed9d640 1 --2- 172.21.15.133:0/3378835111 >> v2:172.21.15.162:3300/0 conn(0x7f2cf006b3a0 0x7f2cf00cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd8003530 con 0x7f2cf00cc490 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 107712649 0 0) 0x7f2cd8003530 con 0x7f2cf00cc490 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf00cc0b0 msgr2=0x7f2cf00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf00cc0b0 0x7f2cf00cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 >> v2:172.21.15.162:3300/0 conn(0x7f2cf006b3a0 msgr2=0x7f2cf00cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 --2- 172.21.15.133:0/3378835111 >> v2:172.21.15.162:3300/0 conn(0x7f2cf006b3a0 0x7f2cf00cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0161aa0 con 0x7f2cf00cc490 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2ce4001680 con 0x7f2cf00cc490 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 >> v1:172.21.15.133:6789/0 conn(0x7f2cf00cc490 legacy=0x7f2cf006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cd80039c0 0x7f2cd8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cd8004810 0x7f2cd8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf00cc0b0 0x7f2cd80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf00d03d0 con 0x7f2cd8004810 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd800b9d0 con 0x7f2cd80039c0 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd800bb50 con 0x7f2cf00cc0b0 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cedd9b640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cd8004810 0x7f2cd8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cee59c640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cd80039c0 0x7f2cd8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ceed9d640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf00cc0b0 0x7f2cd80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cedd9b640 1 -- 172.21.15.133:0/3378835111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf00cc0b0 msgr2=0x7f2cd80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cedd9b640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf00cc0b0 0x7f2cd80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cedd9b640 1 -- 172.21.15.133:0/3378835111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cd80039c0 msgr2=0x7f2cd8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cedd9b640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cd80039c0 0x7f2cd8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cedd9b640 1 -- 172.21.15.133:0/3378835111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd8003530 con 0x7f2cd8004810 2024-04-25T10:12:43.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2cedd9b640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cd8004810 0x7f2cd8004c90 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400c240 tx=0x7f2ce401f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e0bda11e0299d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4007d40 con 0x7f2cd8004810 2024-04-25T10:12:43.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce4005b00 con 0x7f2cd8004810 2024-04-25T10:12:43.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.613+0000 7f2ced59a640 1 -- 172.21.15.133:0/3378835111 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4007070 con 0x7f2cd8004810 2024-04-25T10:12:43.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/3378835111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cd8004810 msgr2=0x7f2cd8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 --2- 172.21.15.133:0/3378835111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cd8004810 0x7f2cd8004c90 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400c240 tx=0x7f2ce401f410 comp rx=0 tx=0).stop 2024-04-25T10:12:43.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2ceed9d640 1 -- 172.21.15.133:0/3378835111 reap_dead start 2024-04-25T10:12:43.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/3378835111 shutdown_connections 2024-04-25T10:12:43.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/3378835111 >> 172.21.15.133:0/3378835111 conn(0x7f2cf00999d0 msgr2=0x7f2cf00d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/3378835111 shutdown_connections 2024-04-25T10:12:43.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/3378835111 wait complete. 2024-04-25T10:12:43.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 Processor -- start 2024-04-25T10:12:43.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 -- start start 2024-04-25T10:12:43.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cd8004810 0x7f2cf0154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf015edc0 0x7f2cf00b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf00b6150 0x7f2cf00c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf00cf950 con 0x7f2cf00b6150 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf0162060 con 0x7f2cd8004810 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf00cfad0 con 0x7f2cf015edc0 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2ceed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf00b6150 0x7f2cf00c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2ceed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf00b6150 0x7f2cf00c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56996/0 (socket says 172.21.15.133:56996) 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2ceed9d640 1 -- 172.21.15.133:0/1347849201 learned_addr learned my addr 172.21.15.133:0/1347849201 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cedd9b640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf015edc0 0x7f2cf00b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cee59c640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cd8004810 0x7f2cf0154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2ceed9d640 1 -- 172.21.15.133:0/1347849201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf015edc0 msgr2=0x7f2cf00b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2ceed9d640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf015edc0 0x7f2cf00b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2ceed9d640 1 -- 172.21.15.133:0/1347849201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cd8004810 msgr2=0x7f2cf0154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2ceed9d640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cd8004810 0x7f2cf0154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2ceed9d640 1 -- 172.21.15.133:0/1347849201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf00c65a0 con 0x7f2cf00b6150 2024-04-25T10:12:43.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cedd9b640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf015edc0 0x7f2cf00b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:43.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2ceed9d640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf00b6150 0x7f2cf00c58d0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00d16c0 tx=0x7f2ce80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cdeffd640 1 -- 172.21.15.133:0/1347849201 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80039a0 con 0x7f2cf00b6150 2024-04-25T10:12:43.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cdeffd640 1 -- 172.21.15.133:0/1347849201 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce801b070 con 0x7f2cf00b6150 2024-04-25T10:12:43.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cdeffd640 1 -- 172.21.15.133:0/1347849201 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8003b80 con 0x7f2cf00b6150 2024-04-25T10:12:43.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/1347849201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf00c6880 con 0x7f2cf00b6150 2024-04-25T10:12:43.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.617+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/1347849201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf00c6e40 con 0x7f2cf00b6150 2024-04-25T10:12:43.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cdeffd640 1 -- 172.21.15.133:0/1347849201 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ce8004030 con 0x7f2cf00b6150 2024-04-25T10:12:43.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cdeffd640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cd0066f60 0x7f2cd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cdeffd640 1 -- 172.21.15.133:0/1347849201 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ce8085ad0 con 0x7f2cf00b6150 2024-04-25T10:12:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cee59c640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cd0066f60 0x7f2cd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cee59c640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cd0066f60 0x7f2cd0069420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2cf01f6f40 tx=0x7f2ce0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cf810f000 1 Processor -- start 2024-04-25T10:12:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cf810f000 1 -- start start 2024-04-25T10:12:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86979d0 0x55c0c8697db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86982f0 0x55c0c864db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c864e0a0 0x55c0c864e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cf810f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c849aff0 con 0x55c0c86979d0 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cf810f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c8423150 con 0x55c0c86982f0 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cf810f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0c8497090 con 0x55c0c864e0a0 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cee59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86979d0 0x55c0c8697db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cee59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86979d0 0x55c0c8697db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57006/0 (socket says 172.21.15.133:57006) 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cee59c640 1 -- 172.21.15.133:0/2749081618 learned_addr learned my addr 172.21.15.133:0/2749081618 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cee59c640 1 -- 172.21.15.133:0/2749081618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c864e0a0 msgr2=0x55c0c864e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cee59c640 1 --2- 172.21.15.133:0/2749081618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c864e0a0 0x55c0c864e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cee59c640 1 -- 172.21.15.133:0/2749081618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86982f0 msgr2=0x55c0c864db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cee59c640 1 --2- 172.21.15.133:0/2749081618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86982f0 0x55c0c864db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.621+0000 7f2cee59c640 1 -- 172.21.15.133:0/2749081618 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0c86310d0 con 0x55c0c86979d0 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cee59c640 1 --2- 172.21.15.133:0/2749081618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86979d0 0x55c0c8697db0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0011a20 tx=0x7f2ce0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63caf271b2bbd7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ccffff640 1 -- 172.21.15.133:0/2749081618 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce00159c0 con 0x55c0c86979d0 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ccffff640 1 -- 172.21.15.133:0/2749081618 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce0026070 con 0x55c0c86979d0 2024-04-25T10:12:43.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ccffff640 1 -- 172.21.15.133:0/2749081618 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0015d60 con 0x55c0c86979d0 2024-04-25T10:12:43.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- 172.21.15.133:0/2749081618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86979d0 msgr2=0x55c0c8697db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 --2- 172.21.15.133:0/2749081618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86979d0 0x55c0c8697db0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0011a20 tx=0x7f2ce0014000 comp rx=0 tx=0).stop 2024-04-25T10:12:43.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- 172.21.15.133:0/2749081618 shutdown_connections 2024-04-25T10:12:43.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 --2- 172.21.15.133:0/2749081618 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c864e0a0 0x55c0c864e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 --2- 172.21.15.133:0/2749081618 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86982f0 0x55c0c864db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 --2- 172.21.15.133:0/2749081618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86979d0 0x55c0c8697db0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- 172.21.15.133:0/2749081618 >> 172.21.15.133:0/2749081618 conn(0x55c0c864a020 msgr2=0x55c0c864c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- 172.21.15.133:0/2749081618 shutdown_connections 2024-04-25T10:12:43.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- 172.21.15.133:0/2749081618 wait complete. 2024-04-25T10:12:43.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- start start 2024-04-25T10:12:43.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 Processor -- start 2024-04-25T10:12:43.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c864e0a0 0x55c0c867c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86979d0 0x55c0c867cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86982f0 0x55c0c867d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c845f250 con 0x55c0c86982f0 2024-04-25T10:12:43.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c8455690 con 0x55c0c86979d0 2024-04-25T10:12:43.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0c8423a40 con 0x55c0c864e0a0 2024-04-25T10:12:43.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 10 client.0 _ll_register_callbacks cb 0x55c0c8638e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ceed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86982f0 0x55c0c867d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ceed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86982f0 0x55c0c867d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57022/0 (socket says 172.21.15.133:57022) 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ceed9d640 1 -- 172.21.15.133:0/902149294 learned_addr learned my addr 172.21.15.133:0/902149294 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cedd9b640 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86979d0 0x55c0c867cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cee59c640 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c864e0a0 0x55c0c867c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ceed9d640 1 -- 172.21.15.133:0/902149294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c864e0a0 msgr2=0x55c0c867c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ceed9d640 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c864e0a0 0x55c0c867c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ceed9d640 1 -- 172.21.15.133:0/902149294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86979d0 msgr2=0x55c0c867cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ceed9d640 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86979d0 0x55c0c867cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ceed9d640 1 -- 172.21.15.133:0/902149294 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0c85481c0 con 0x55c0c86982f0 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ceed9d640 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86982f0 0x55c0c867d270 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80070b0 tx=0x7f2ce8056aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb54fe54a4b9d0eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:43.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/902149294 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8073060 con 0x55c0c86982f0 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- 172.21.15.133:0/902149294 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ce0026070 con 0x55c0c86982f0 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/902149294 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ce805a150 con 0x55c0c86982f0 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/902149294 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce805e9f0 con 0x55c0c86982f0 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 10 client.9129 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- 172.21.15.133:0/902149294 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55c0c8544170 con 0x55c0c86982f0 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/902149294 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce0026070 con 0x55c0c86982f0 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/902149294 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2ce8074250 con 0x55c0c86982f0 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 10 client.9129 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 10 client.9129 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 20 client.9129 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55c0c86a7c80 0x55c0c86aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 4 client.9129 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cf810f000 1 -- 172.21.15.133:0/902149294 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0c8436c60 con 0x55c0c86a7c80 2024-04-25T10:12:43.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.625+0000 7f2cee59c640 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55c0c86a7c80 0x55c0c86aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2ccd7fa640 10 client.9129 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cee59c640 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55c0c86a7c80 0x55c0c86aa070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce40dd0d4fe13275 server_cookie=1f54b753684d1a6d in_seq=0 out_seq=0 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/902149294 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0c8436c60 con 0x55c0c86a7c80 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2ccd7fa640 10 client.9129 handle_command_reply: tid=0 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 1 client.9129 shutdown 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 1 -- 172.21.15.133:0/902149294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86982f0 msgr2=0x55c0c867d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86982f0 0x55c0c867d270 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80070b0 tx=0x7f2ce8056aa0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 1 -- 172.21.15.133:0/902149294 shutdown_connections 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55c0c86a7c80 0x55c0c86aa070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c86982f0 0x55c0c867d270 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2ccd7fa640 0 client.9129 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86979d0 0x55c0c867cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 1 --2- 172.21.15.133:0/902149294 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c864e0a0 0x55c0c867c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 1 -- 172.21.15.133:0/902149294 >> 172.21.15.133:0/902149294 conn(0x55c0c864a020 msgr2=0x55c0c864cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 1 -- 172.21.15.133:0/902149294 shutdown_connections 2024-04-25T10:12:43.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 1 -- 172.21.15.133:0/902149294 wait complete. 2024-04-25T10:12:43.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.629+0000 7f2cf810f000 20 client.9129 trim_cache size 0 max 16384 2024-04-25T10:12:43.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.641+0000 7f2cf810f000 1 Processor -- start 2024-04-25T10:12:43.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.641+0000 7f2cf810f000 1 -- start start 2024-04-25T10:12:43.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86fcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86979d0 0x55c0c86fd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c86982f0 0x55c0c86fd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf02007c0 con 0x55c0c864e0a0 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf01ff970 con 0x55c0c86979d0 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf00b4840 con 0x55c0c86982f0 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86fcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86fcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57026/0 (socket says 172.21.15.133:57026) 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 -- 172.21.15.133:0/1490516422 learned_addr learned my addr 172.21.15.133:0/1490516422 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 -- 172.21.15.133:0/1490516422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c86982f0 msgr2=0x55c0c86fd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 --2- 172.21.15.133:0/1490516422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c86982f0 0x55c0c86fd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 -- 172.21.15.133:0/1490516422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86979d0 msgr2=0x55c0c86fd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 --2- 172.21.15.133:0/1490516422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86979d0 0x55c0c86fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 -- 172.21.15.133:0/1490516422 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0c84e1de0 con 0x55c0c864e0a0 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 --2- 172.21.15.133:0/1490516422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86fcf30 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0023040 tx=0x7f2ce0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b36314bbc5cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2ccf7fe640 1 -- 172.21.15.133:0/1490516422 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0033b00 con 0x55c0c864e0a0 2024-04-25T10:12:43.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2ccf7fe640 1 -- 172.21.15.133:0/1490516422 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce0026070 con 0x55c0c864e0a0 2024-04-25T10:12:43.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2ccf7fe640 1 -- 172.21.15.133:0/1490516422 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0033ce0 con 0x55c0c864e0a0 2024-04-25T10:12:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- 172.21.15.133:0/1490516422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 msgr2=0x55c0c86fcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 --2- 172.21.15.133:0/1490516422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86fcf30 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0023040 tx=0x7f2ce0020000 comp rx=0 tx=0).stop 2024-04-25T10:12:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- 172.21.15.133:0/1490516422 shutdown_connections 2024-04-25T10:12:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 --2- 172.21.15.133:0/1490516422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c86982f0 0x55c0c86fd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 --2- 172.21.15.133:0/1490516422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86979d0 0x55c0c86fd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 --2- 172.21.15.133:0/1490516422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86fcf30 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- 172.21.15.133:0/1490516422 >> 172.21.15.133:0/1490516422 conn(0x55c0c864a020 msgr2=0x55c0c86ac2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- 172.21.15.133:0/1490516422 shutdown_connections 2024-04-25T10:12:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- 172.21.15.133:0/1490516422 wait complete. 2024-04-25T10:12:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- start start 2024-04-25T10:12:43.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 Processor -- start 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c86979d0 0x55c0c86aa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86982f0 0x55c0c86aa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0c8435e00 con 0x55c0c864e0a0 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf01ff370 con 0x55c0c86982f0 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0c849a300 con 0x55c0c86979d0 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cf810f000 10 client.0 _ll_register_callbacks cb 0x55c0c8619040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86a9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57028/0 (socket says 172.21.15.133:57028) 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 -- 172.21.15.133:0/2954410117 learned_addr learned my addr 172.21.15.133:0/2954410117 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 -- 172.21.15.133:0/2954410117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c86979d0 msgr2=0x55c0c86aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 --2- 172.21.15.133:0/2954410117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c86979d0 0x55c0c86aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 -- 172.21.15.133:0/2954410117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86982f0 msgr2=0x55c0c86aa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 --2- 172.21.15.133:0/2954410117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86982f0 0x55c0c86aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 -- 172.21.15.133:0/2954410117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0c843c8e0 con 0x55c0c864e0a0 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2cee59c640 1 --2- 172.21.15.133:0/2954410117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86a9e10 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0015810 tx=0x7f2ce00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74715f2d2c7b49b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.645+0000 7f2ccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/2954410117 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce00158c0 con 0x55c0c864e0a0 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/2954410117 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ce0033050 con 0x55c0c864e0a0 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 -- 172.21.15.133:0/2954410117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ce0026070 con 0x55c0c864e0a0 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/2954410117 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce004a050 con 0x55c0c864e0a0 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 10 client.9135 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 -- 172.21.15.133:0/2954410117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55c0c84f48f0 con 0x55c0c864e0a0 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/2954410117 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce0026070 con 0x55c0c864e0a0 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/2954410117 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2ce004d020 con 0x55c0c864e0a0 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 10 client.9135 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 10 client.9135 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:43.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 20 client.9135 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 --2- 172.21.15.133:0/2954410117 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55c0c8706760 0x55c0c8708b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 4 client.9135 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 -- 172.21.15.133:0/2954410117 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c0c84a8530 con 0x55c0c8706760 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cedd9b640 1 --2- 172.21.15.133:0/2954410117 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55c0c8706760 0x55c0c8708b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2ccd7fa640 10 client.9135 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cedd9b640 1 --2- 172.21.15.133:0/2954410117 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55c0c8706760 0x55c0c8708b50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5683d5b6f879cb3d server_cookie=6fd7516282e4a8f0 in_seq=0 out_seq=0 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2ccd7fa640 1 -- 172.21.15.133:0/2954410117 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x55c0c84a8530 con 0x55c0c8706760 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2ccd7fa640 10 client.9135 handle_command_reply: tid=0 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 client.9135 shutdown 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 -- 172.21.15.133:0/2954410117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 msgr2=0x55c0c86a9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 --2- 172.21.15.133:0/2954410117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86a9e10 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0015810 tx=0x7f2ce00046d0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 -- 172.21.15.133:0/2954410117 shutdown_connections 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 --2- 172.21.15.133:0/2954410117 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55c0c8706760 0x55c0c8708b50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 --2- 172.21.15.133:0/2954410117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c0c86982f0 0x55c0c86aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 --2- 172.21.15.133:0/2954410117 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c0c86979d0 0x55c0c86aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2ccd7fa640 0 client.9135 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:43.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 --2- 172.21.15.133:0/2954410117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0c864e0a0 0x55c0c86a9e10 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 -- 172.21.15.133:0/2954410117 >> 172.21.15.133:0/2954410117 conn(0x55c0c864a020 msgr2=0x55c0c86a4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 -- 172.21.15.133:0/2954410117 shutdown_connections 2024-04-25T10:12:43.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 1 -- 172.21.15.133:0/2954410117 wait complete. 2024-04-25T10:12:43.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf810f000 20 client.9135 trim_cache size 0 max 16384 2024-04-25T10:12:43.659 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:12:43.659 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:12:43.659 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:12:43.660 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:12:43.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/1347849201 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cd0066f60 msgr2=0x7f2cd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cd0066f60 0x7f2cd0069420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2cf01f6f40 tx=0x7f2ce0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:43.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/1347849201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf00b6150 msgr2=0x7f2cf00c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf00b6150 0x7f2cf00c58d0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00d16c0 tx=0x7f2ce80115a0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/1347849201 shutdown_connections 2024-04-25T10:12:43.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cd0066f60 0x7f2cd0069420 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cf00b6150 0x7f2cf00c58d0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cf015edc0 0x7f2cf00b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 --2- 172.21.15.133:0/1347849201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cd8004810 0x7f2cf0154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/1347849201 >> 172.21.15.133:0/1347849201 conn(0x7f2cf00999d0 msgr2=0x7f2cf00af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/1347849201 shutdown_connections 2024-04-25T10:12:43.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.649+0000 7f2cf4fa2640 1 -- 172.21.15.133:0/1347849201 wait complete. 2024-04-25T10:12:43.666 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:43.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.765+0000 7fcae96c1640 1 Processor -- start 2024-04-25T10:12:43.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.765+0000 7fcae96c1640 1 -- start start 2024-04-25T10:12:43.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.765+0000 7fcae96c1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae406dca0 0x7fcae40b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.765+0000 7fcae96c1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcae406bda0 0x7fcae40b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.765+0000 7fcae96c1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae40cfdc0 con 0x7fcae40b20a0 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.765+0000 7fcae96c1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae40d0660 con 0x7fcae406bda0 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.765+0000 7fcae96c1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae40d00a0 con 0x7fcae406dca0 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.765+0000 7fcae27fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcae406bda0 0x7fcae40b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.765+0000 7fcae37fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcae40b20a0 0x7fcae40999d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45352/0 (socket says 172.21.15.133:45352) 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.765+0000 7fcae27fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcae406bda0 0x7fcae40b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57030/0 (socket says 172.21.15.133:57030) 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.765+0000 7fcae37fe640 1 -- 172.21.15.133:0/1521138394 learned_addr learned my addr 172.21.15.133:0/1521138394 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae2ffd640 1 --2- 172.21.15.133:0/1521138394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae406dca0 0x7fcae40b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae27fc640 1 -- 172.21.15.133:0/1521138394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae406dca0 msgr2=0x7fcae40b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae27fc640 1 --2- 172.21.15.133:0/1521138394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae406dca0 0x7fcae40b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae27fc640 1 -- 172.21.15.133:0/1521138394 >> v1:172.21.15.133:6789/0 conn(0x7fcae40b20a0 legacy=0x7fcae40999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae27fc640 1 -- 172.21.15.133:0/1521138394 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae40d0380 con 0x7fcae406bda0 2024-04-25T10:12:43.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae1ffb640 1 -- 172.21.15.133:0/1521138394 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714617328 0 0) 0x7fcae40cfdc0 con 0x7fcae40b20a0 2024-04-25T10:12:43.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae27fc640 1 --2- 172.21.15.133:0/1521138394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae406bda0 0x7fcae40b1b60 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fcad80097c0 tx=0x7fcad801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa643d09ac6b141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae1ffb640 1 -- 172.21.15.133:0/1521138394 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad80b0070 con 0x7fcae406bda0 2024-04-25T10:12:43.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae1ffb640 1 -- 172.21.15.133:0/1521138394 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad80ab600 con 0x7fcae406bda0 2024-04-25T10:12:43.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae1ffb640 1 -- 172.21.15.133:0/1521138394 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad80ab9d0 con 0x7fcae406bda0 2024-04-25T10:12:43.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae96c1640 1 -- 172.21.15.133:0/1521138394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae406bda0 msgr2=0x7fcae40b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae96c1640 1 --2- 172.21.15.133:0/1521138394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae406bda0 0x7fcae40b1b60 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fcad80097c0 tx=0x7fcad801c380 comp rx=0 tx=0).stop 2024-04-25T10:12:43.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae96c1640 1 -- 172.21.15.133:0/1521138394 shutdown_connections 2024-04-25T10:12:43.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae96c1640 1 --2- 172.21.15.133:0/1521138394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae406bda0 0x7fcae40b1b60 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae96c1640 1 --2- 172.21.15.133:0/1521138394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae406dca0 0x7fcae40b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae96c1640 1 -- 172.21.15.133:0/1521138394 >> 172.21.15.133:0/1521138394 conn(0x7fcae40b31a0 msgr2=0x7fcae40b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae96c1640 1 -- 172.21.15.133:0/1521138394 shutdown_connections 2024-04-25T10:12:43.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae96c1640 1 -- 172.21.15.133:0/1521138394 wait complete. 2024-04-25T10:12:43.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.769+0000 7fcae96c1640 1 Processor -- start 2024-04-25T10:12:43.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae96c1640 1 -- start start 2024-04-25T10:12:43.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae96c1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae406bda0 0x7fcae415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae96c1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae406dca0 0x7fcae41569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae96c1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40b20a0 0x7fcae415c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae96c1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae416ab30 con 0x7fcae40b20a0 2024-04-25T10:12:43.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae96c1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae416a9b0 con 0x7fcae406bda0 2024-04-25T10:12:43.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae96c1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae416acb0 con 0x7fcae406dca0 2024-04-25T10:12:43.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40b20a0 0x7fcae415c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40b20a0 0x7fcae415c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57046/0 (socket says 172.21.15.133:57046) 2024-04-25T10:12:43.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae37fe640 1 -- 172.21.15.133:0/3079969334 learned_addr learned my addr 172.21.15.133:0/3079969334 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:43.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae27fc640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae406dca0 0x7fcae41569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae37fe640 1 -- 172.21.15.133:0/3079969334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae406dca0 msgr2=0x7fcae41569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae2ffd640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae406bda0 0x7fcae415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae37fe640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae406dca0 0x7fcae41569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae37fe640 1 -- 172.21.15.133:0/3079969334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae406bda0 msgr2=0x7fcae415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae37fe640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae406bda0 0x7fcae415dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae37fe640 1 -- 172.21.15.133:0/3079969334 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae4154110 con 0x7fcae40b20a0 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae37fe640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40b20a0 0x7fcae415c0c0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00c700 tx=0x7fcadc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcacb7fe640 1 -- 172.21.15.133:0/3079969334 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc018070 con 0x7fcae40b20a0 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcacb7fe640 1 -- 172.21.15.133:0/3079969334 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcadc00ce10 con 0x7fcae40b20a0 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae96c1640 1 -- 172.21.15.133:0/3079969334 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae4154450 con 0x7fcae40b20a0 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcacb7fe640 1 -- 172.21.15.133:0/3079969334 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc00a860 con 0x7fcae40b20a0 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae96c1640 1 -- 172.21.15.133:0/3079969334 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae4154c90 con 0x7fcae40b20a0 2024-04-25T10:12:43.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae96c1640 1 -- 172.21.15.133:0/3079969334 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae4067c70 con 0x7fcae40b20a0 2024-04-25T10:12:43.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcacb7fe640 1 -- 172.21.15.133:0/3079969334 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcadc01b070 con 0x7fcae40b20a0 2024-04-25T10:12:43.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcacb7fe640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcad0066f60 0x7fcad0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcacb7fe640 1 -- 172.21.15.133:0/3079969334 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcadc084760 con 0x7fcae40b20a0 2024-04-25T10:12:43.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.773+0000 7fcae2ffd640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcad0066f60 0x7fcad0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.777+0000 7fcae2ffd640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcad0066f60 0x7fcad0069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcad400b7c0 tx=0x7fcad4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.781+0000 7fcacb7fe640 1 -- 172.21.15.133:0/3079969334 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcae4154450 con 0x7fcae40b20a0 2024-04-25T10:12:43.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 -- 172.21.15.133:0/3079969334 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcae406cbb0 con 0x7fcae40b20a0 2024-04-25T10:12:43.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcacb7fe640 1 -- 172.21.15.133:0/3079969334 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fcadc0555e0 con 0x7fcae40b20a0 2024-04-25T10:12:43.971 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:43.971 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:43.972 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:43.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 -- 172.21.15.133:0/3079969334 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcad0066f60 msgr2=0x7fcad0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcad0066f60 0x7fcad0069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcad400b7c0 tx=0x7fcad4008040 comp rx=0 tx=0).stop 2024-04-25T10:12:43.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 -- 172.21.15.133:0/3079969334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40b20a0 msgr2=0x7fcae415c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40b20a0 0x7fcae415c0c0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00c700 tx=0x7fcadc00cbc0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 -- 172.21.15.133:0/3079969334 shutdown_connections 2024-04-25T10:12:43.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcad0066f60 0x7fcad0069420 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcae40b20a0 0x7fcae415c0c0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcae406dca0 0x7fcae41569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 --2- 172.21.15.133:0/3079969334 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcae406bda0 0x7fcae415dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 -- 172.21.15.133:0/3079969334 >> 172.21.15.133:0/3079969334 conn(0x7fcae40b31a0 msgr2=0x7fcae40ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 -- 172.21.15.133:0/3079969334 shutdown_connections 2024-04-25T10:12:43.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:43.969+0000 7fcae96c1640 1 -- 172.21.15.133:0/3079969334 wait complete. 2024-04-25T10:12:43.989 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:12:43.989 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:44.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.101+0000 7fe27f81c640 1 Processor -- start 2024-04-25T10:12:44.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.101+0000 7fe27f81c640 1 -- start start 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27f81c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe278158110 0x7fe2780b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27f81c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27806eb70 0x7fe27815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27f81c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2780cbe60 con 0x7fe2781584f0 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27f81c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2780b19b0 con 0x7fe278158110 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27f81c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27815b360 con 0x7fe27806eb70 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27cdb7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe2781584f0 0x7fe27806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45358/0 (socket says 172.21.15.133:45358) 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27cdb7640 1 -- 172.21.15.133:0/1617932313 learned_addr learned my addr 172.21.15.133:0/1617932313 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27d5b8640 1 --2- 172.21.15.133:0/1617932313 >> v2:172.21.15.162:3300/0 conn(0x7fe278158110 0x7fe2780b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27ddb9640 1 --2- 172.21.15.133:0/1617932313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27806eb70 0x7fe27815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008952567 0 0) 0x7fe2780cbe60 con 0x7fe2781584f0 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe264003530 con 0x7fe2781584f0 2024-04-25T10:12:44.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2293400532 0 0) 0x7fe264003530 con 0x7fe2781584f0 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27806eb70 msgr2=0x7fe27815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 --2- 172.21.15.133:0/1617932313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27806eb70 0x7fe27815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 >> v2:172.21.15.162:3300/0 conn(0x7fe278158110 msgr2=0x7fe2780b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 --2- 172.21.15.133:0/1617932313 >> v2:172.21.15.162:3300/0 conn(0x7fe278158110 0x7fe2780b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27815b640 con 0x7fe2781584f0 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe268001680 con 0x7fe2781584f0 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 >> v1:172.21.15.133:6789/0 conn(0x7fe2781584f0 legacy=0x7fe27806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 --2- 172.21.15.133:0/1617932313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2640039c0 0x7fe264003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 --2- 172.21.15.133:0/1617932313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27806eb70 0x7fe264004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 --2- 172.21.15.133:0/1617932313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe264004d50 0x7fe2640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780b19b0 con 0x7fe2640039c0 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27815b360 con 0x7fe264004d50 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26400dbe0 con 0x7fe27806eb70 2024-04-25T10:12:44.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27d5b8640 1 --2- 172.21.15.133:0/1617932313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2640039c0 0x7fe264003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27d5b8640 1 -- 172.21.15.133:0/1617932313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27806eb70 msgr2=0x7fe264004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27d5b8640 1 --2- 172.21.15.133:0/1617932313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27806eb70 0x7fe264004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27d5b8640 1 -- 172.21.15.133:0/1617932313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe264004d50 msgr2=0x7fe2640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27d5b8640 1 --2- 172.21.15.133:0/1617932313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe264004d50 0x7fe2640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27d5b8640 1 -- 172.21.15.133:0/1617932313 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe264003530 con 0x7fe2640039c0 2024-04-25T10:12:44.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.105+0000 7fe27d5b8640 1 --2- 172.21.15.133:0/1617932313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2640039c0 0x7fe264003de0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fe274009b10 tx=0x7fe27401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1b780bee61a59c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2740096b0 con 0x7fe2640039c0 2024-04-25T10:12:44.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2740a6470 con 0x7fe2640039c0 2024-04-25T10:12:44.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe26ffff640 1 -- 172.21.15.133:0/1617932313 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2740a6860 con 0x7fe2640039c0 2024-04-25T10:12:44.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 -- 172.21.15.133:0/1617932313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2640039c0 msgr2=0x7fe264003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 --2- 172.21.15.133:0/1617932313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2640039c0 0x7fe264003de0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fe274009b10 tx=0x7fe27401ca80 comp rx=0 tx=0).stop 2024-04-25T10:12:44.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27ddb9640 1 -- 172.21.15.133:0/1617932313 reap_dead start 2024-04-25T10:12:44.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 -- 172.21.15.133:0/1617932313 shutdown_connections 2024-04-25T10:12:44.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 -- 172.21.15.133:0/1617932313 >> 172.21.15.133:0/1617932313 conn(0x7fe278067000 msgr2=0x7fe27806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:44.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 -- 172.21.15.133:0/1617932313 shutdown_connections 2024-04-25T10:12:44.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 -- 172.21.15.133:0/1617932313 wait complete. 2024-04-25T10:12:44.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 Processor -- start 2024-04-25T10:12:44.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 -- start start 2024-04-25T10:12:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2640039c0 0x7fe278147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27815c820 0x7fe278153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe27815cca0 0x7fe278153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27806dce0 con 0x7fe27815cca0 2024-04-25T10:12:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27806db60 con 0x7fe2640039c0 2024-04-25T10:12:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27806de60 con 0x7fe27815c820 2024-04-25T10:12:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27ddb9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe27815cca0 0x7fe278153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27ddb9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe27815cca0 0x7fe278153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57056/0 (socket says 172.21.15.133:57056) 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27ddb9640 1 -- 172.21.15.133:0/162448196 learned_addr learned my addr 172.21.15.133:0/162448196 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27d5b8640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2640039c0 0x7fe278147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27cdb7640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27815c820 0x7fe278153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27ddb9640 1 -- 172.21.15.133:0/162448196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27815c820 msgr2=0x7fe278153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27ddb9640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27815c820 0x7fe278153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27ddb9640 1 -- 172.21.15.133:0/162448196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2640039c0 msgr2=0x7fe278147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27ddb9640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2640039c0 0x7fe278147000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27ddb9640 1 -- 172.21.15.133:0/162448196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27815d190 con 0x7fe27815cca0 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27ddb9640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe27815cca0 0x7fe278153970 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe2780cb7d0 tx=0x7fe27000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe26dffb640 1 -- 172.21.15.133:0/162448196 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270018020 con 0x7fe27815cca0 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe26dffb640 1 -- 172.21.15.133:0/162448196 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27000bb10 con 0x7fe27815cca0 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 -- 172.21.15.133:0/162448196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27815d470 con 0x7fe27815cca0 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe26dffb640 1 -- 172.21.15.133:0/162448196 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270014400 con 0x7fe27815cca0 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.109+0000 7fe27f81c640 1 -- 172.21.15.133:0/162448196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27814fda0 con 0x7fe27815cca0 2024-04-25T10:12:44.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.113+0000 7fe27f81c640 1 -- 172.21.15.133:0/162448196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27814ffa0 con 0x7fe27815cca0 2024-04-25T10:12:44.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.113+0000 7fe26dffb640 1 -- 172.21.15.133:0/162448196 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe27001b020 con 0x7fe27815cca0 2024-04-25T10:12:44.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.113+0000 7fe26dffb640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe25c066f90 0x7fe25c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.113+0000 7fe27d5b8640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe25c066f90 0x7fe25c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.113+0000 7fe26dffb640 1 -- 172.21.15.133:0/162448196 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe270085890 con 0x7fe27815cca0 2024-04-25T10:12:44.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.113+0000 7fe27d5b8640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe25c066f90 0x7fe25c069450 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe2740ad000 tx=0x7fe2740046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.117+0000 7fe26dffb640 1 -- 172.21.15.133:0/162448196 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe27814ffa0 con 0x7fe27815cca0 2024-04-25T10:12:44.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.313+0000 7fe27f81c640 1 -- 172.21.15.133:0/162448196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe278157920 con 0x7fe27815cca0 2024-04-25T10:12:44.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe26dffb640 1 -- 172.21.15.133:0/162448196 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1186 (secure 0 0 0) 0x7fe270022030 con 0x7fe27815cca0 2024-04-25T10:12:44.318 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:44.318 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":677,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":120,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":856,"data_bytes":875413189,"bytes_used":2214068224,"bytes_avail":770880045056,"bytes_total":773094113280,"write_bytes_sec":52682291,"read_op_per_sec":0,"write_op_per_sec":53},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8919}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T10:12:09.894947+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:44.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 -- 172.21.15.133:0/162448196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe25c066f90 msgr2=0x7fe25c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe25c066f90 0x7fe25c069450 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe2740ad000 tx=0x7fe2740046d0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 -- 172.21.15.133:0/162448196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe27815cca0 msgr2=0x7fe278153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe27815cca0 0x7fe278153970 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe2780cb7d0 tx=0x7fe27000b480 comp rx=0 tx=0).stop 2024-04-25T10:12:44.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 -- 172.21.15.133:0/162448196 shutdown_connections 2024-04-25T10:12:44.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe25c066f90 0x7fe25c069450 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe27815cca0 0x7fe278153970 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe27815c820 0x7fe278153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 --2- 172.21.15.133:0/162448196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2640039c0 0x7fe278147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 -- 172.21.15.133:0/162448196 >> 172.21.15.133:0/162448196 conn(0x7fe278067000 msgr2=0x7fe2780ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:44.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 -- 172.21.15.133:0/162448196 shutdown_connections 2024-04-25T10:12:44.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:44.317+0000 7fe27f81c640 1 -- 172.21.15.133:0/162448196 wait complete. 2024-04-25T10:12:44.334 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:12:45.336 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:45.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7bcaf0640 1 Processor -- start 2024-04-25T10:12:45.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7bcaf0640 1 -- start start 2024-04-25T10:12:45.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7bcaf0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd7b806ff80 0x7fd7b815ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7bcaf0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7b806bca0 0x7fd7b806c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7bcaf0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b81614b0 0x7fd7b8161940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7bcaf0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b806c650 con 0x7fd7b806bca0 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7bcaf0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b80b3080 con 0x7fd7b806ff80 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7bcaf0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b8162090 con 0x7fd7b81614b0 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7b5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7b806bca0 0x7fd7b806c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7b5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7b806bca0 0x7fd7b806c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57066/0 (socket says 172.21.15.133:57066) 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7b5d9b640 1 -- 172.21.15.133:0/1454662045 learned_addr learned my addr 172.21.15.133:0/1454662045 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7b659c640 1 --2- 172.21.15.133:0/1454662045 >> v2:172.21.15.162:3300/0 conn(0x7fd7b806ff80 0x7fd7b815ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7b6d9d640 1 --2- 172.21.15.133:0/1454662045 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b81614b0 0x7fd7b8161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7b5d9b640 1 -- 172.21.15.133:0/1454662045 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b81614b0 msgr2=0x7fd7b8161940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7b5d9b640 1 --2- 172.21.15.133:0/1454662045 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b81614b0 0x7fd7b8161940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7b5d9b640 1 -- 172.21.15.133:0/1454662045 >> v2:172.21.15.162:3300/0 conn(0x7fd7b806ff80 msgr2=0x7fd7b815ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7b5d9b640 1 --2- 172.21.15.133:0/1454662045 >> v2:172.21.15.162:3300/0 conn(0x7fd7b806ff80 0x7fd7b815ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.469+0000 7fd7b5d9b640 1 -- 172.21.15.133:0/1454662045 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b8162370 con 0x7fd7b806bca0 2024-04-25T10:12:45.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7b5d9b640 1 --2- 172.21.15.133:0/1454662045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b806bca0 0x7fd7b806c110 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8072e60 tx=0x7fd7ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4557a12de2fc8a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:45.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7b559a640 1 -- 172.21.15.133:0/1454662045 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0b1070 con 0x7fd7b806bca0 2024-04-25T10:12:45.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7b559a640 1 -- 172.21.15.133:0/1454662045 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac0ac630 con 0x7fd7b806bca0 2024-04-25T10:12:45.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7b559a640 1 -- 172.21.15.133:0/1454662045 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0a8050 con 0x7fd7b806bca0 2024-04-25T10:12:45.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/1454662045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b806bca0 msgr2=0x7fd7b806c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7bcaf0640 1 --2- 172.21.15.133:0/1454662045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b806bca0 0x7fd7b806c110 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8072e60 tx=0x7fd7ac01c520 comp rx=0 tx=0).stop 2024-04-25T10:12:45.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/1454662045 shutdown_connections 2024-04-25T10:12:45.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7bcaf0640 1 --2- 172.21.15.133:0/1454662045 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b81614b0 0x7fd7b8161940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7bcaf0640 1 --2- 172.21.15.133:0/1454662045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b806bca0 0x7fd7b806c110 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7bcaf0640 1 --2- 172.21.15.133:0/1454662045 >> v2:172.21.15.162:3300/0 conn(0x7fd7b806ff80 0x7fd7b815ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/1454662045 >> 172.21.15.133:0/1454662045 conn(0x7fd7b8071aa0 msgr2=0x7fd7b8071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:45.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/1454662045 shutdown_connections 2024-04-25T10:12:45.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/1454662045 wait complete. 2024-04-25T10:12:45.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7bcaf0640 1 Processor -- start 2024-04-25T10:12:45.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.473+0000 7fd7bcaf0640 1 -- start start 2024-04-25T10:12:45.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7bcaf0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7b806bca0 0x7fd7b80cee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7bcaf0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b806ff80 0x7fd7b813f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7bcaf0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b81614b0 0x7fd7b813f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7bcaf0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80704e0 con 0x7fd7b81614b0 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7bcaf0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b8070360 con 0x7fd7b806bca0 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7bcaf0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80ada50 con 0x7fd7b806ff80 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b81614b0 0x7fd7b813f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b81614b0 0x7fd7b813f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57068/0 (socket says 172.21.15.133:57068) 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b6d9d640 1 -- 172.21.15.133:0/3572293936 learned_addr learned my addr 172.21.15.133:0/3572293936 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b5d9b640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b806ff80 0x7fd7b813f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b659c640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7b806bca0 0x7fd7b80cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b6d9d640 1 -- 172.21.15.133:0/3572293936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b806ff80 msgr2=0x7fd7b813f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b6d9d640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b806ff80 0x7fd7b813f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b6d9d640 1 -- 172.21.15.133:0/3572293936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7b806bca0 msgr2=0x7fd7b80cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b6d9d640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7b806bca0 0x7fd7b80cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b6d9d640 1 -- 172.21.15.133:0/3572293936 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b8153df0 con 0x7fd7b81614b0 2024-04-25T10:12:45.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b6d9d640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b81614b0 0x7fd7b813f7f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0007e60 tx=0x7fd7b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:45.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7a6ffd640 1 -- 172.21.15.133:0/3572293936 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0014420 con 0x7fd7b81614b0 2024-04-25T10:12:45.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7a6ffd640 1 -- 172.21.15.133:0/3572293936 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b0019070 con 0x7fd7b81614b0 2024-04-25T10:12:45.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/3572293936 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b8155c30 con 0x7fd7b81614b0 2024-04-25T10:12:45.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7a6ffd640 1 -- 172.21.15.133:0/3572293936 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0014820 con 0x7fd7b81614b0 2024-04-25T10:12:45.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/3572293936 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b81561f0 con 0x7fd7b81614b0 2024-04-25T10:12:45.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/3572293936 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b8067c70 con 0x7fd7b81614b0 2024-04-25T10:12:45.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7a6ffd640 1 -- 172.21.15.133:0/3572293936 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7b00149c0 con 0x7fd7b81614b0 2024-04-25T10:12:45.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7a6ffd640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd798066f60 0x7fd798069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b659c640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd798066f60 0x7fd798069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7a6ffd640 1 -- 172.21.15.133:0/3572293936 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7b0085a60 con 0x7fd7b81614b0 2024-04-25T10:12:45.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.477+0000 7fd7b659c640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd798066f60 0x7fd798069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800ba30 tx=0x7fd7a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:45.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.481+0000 7fd7a6ffd640 1 -- 172.21.15.133:0/3572293936 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7b00568e0 con 0x7fd7b81614b0 2024-04-25T10:12:45.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.661+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/3572293936 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b8066d50 con 0x7fd7b81614b0 2024-04-25T10:12:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.661+0000 7fd7a6ffd640 1 -- 172.21.15.133:0/3572293936 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fd7b00561e0 con 0x7fd7b81614b0 2024-04-25T10:12:45.665 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:45.665 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:45.665 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:45.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/3572293936 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd798066f60 msgr2=0x7fd798069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd798066f60 0x7fd798069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800ba30 tx=0x7fd7a8008040 comp rx=0 tx=0).stop 2024-04-25T10:12:45.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/3572293936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b81614b0 msgr2=0x7fd7b813f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b81614b0 0x7fd7b813f7f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0007e60 tx=0x7fd7b000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/3572293936 shutdown_connections 2024-04-25T10:12:45.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd798066f60 0x7fd798069420 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b81614b0 0x7fd7b813f7f0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7b806ff80 0x7fd7b813f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 --2- 172.21.15.133:0/3572293936 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7b806bca0 0x7fd7b80cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/3572293936 >> 172.21.15.133:0/3572293936 conn(0x7fd7b8071aa0 msgr2=0x7fd7b806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:45.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/3572293936 shutdown_connections 2024-04-25T10:12:45.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.665+0000 7fd7bcaf0640 1 -- 172.21.15.133:0/3572293936 wait complete. 2024-04-25T10:12:45.682 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:12:45.682 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:45.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe5c994640 1 Processor -- start 2024-04-25T10:12:45.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe5c994640 1 -- start start 2024-04-25T10:12:45.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe5c994640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efe580b1f70 0x7efe5806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe5c994640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efe580ce330 0x7efe580ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe5c994640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe580c61d0 con 0x7efe5815c9a0 2024-04-25T10:12:45.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe5c994640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe580cecc0 con 0x7efe580b1f70 2024-04-25T10:12:45.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe5c994640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe580c64b0 con 0x7efe580ce330 2024-04-25T10:12:45.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe55d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efe580b1f70 0x7efe5806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe55d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efe580b1f70 0x7efe5806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57082/0 (socket says 172.21.15.133:57082) 2024-04-25T10:12:45.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe55d9b640 1 -- 172.21.15.133:0/2495342512 learned_addr learned my addr 172.21.15.133:0/2495342512 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:45.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe56d9d640 1 --2- 172.21.15.133:0/2495342512 >> v2:172.21.15.162:3300/0 conn(0x7efe580ce330 0x7efe580ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe55d9b640 1 -- 172.21.15.133:0/2495342512 >> v2:172.21.15.162:3300/0 conn(0x7efe580ce330 msgr2=0x7efe580ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:45.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe55d9b640 1 --2- 172.21.15.133:0/2495342512 >> v2:172.21.15.162:3300/0 conn(0x7efe580ce330 0x7efe580ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.785+0000 7efe55d9b640 1 -- 172.21.15.133:0/2495342512 >> v1:172.21.15.162:6789/0 conn(0x7efe5815c9a0 legacy=0x7efe580c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe55d9b640 1 -- 172.21.15.133:0/2495342512 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5806d4f0 con 0x7efe580b1f70 2024-04-25T10:12:45.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe55d9b640 1 --2- 172.21.15.133:0/2495342512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe580b1f70 0x7efe5806dcf0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7efe580cc0f0 tx=0x7efe4001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35561ebbfc9fd05a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:45.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5559a640 1 -- 172.21.15.133:0/2495342512 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe400aa670 con 0x7efe580b1f70 2024-04-25T10:12:45.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5559a640 1 -- 172.21.15.133:0/2495342512 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe400af070 con 0x7efe580b1f70 2024-04-25T10:12:45.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5559a640 1 -- 172.21.15.133:0/2495342512 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe400aaa30 con 0x7efe580b1f70 2024-04-25T10:12:45.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5c994640 1 -- 172.21.15.133:0/2495342512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe580b1f70 msgr2=0x7efe5806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5c994640 1 --2- 172.21.15.133:0/2495342512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe580b1f70 0x7efe5806dcf0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7efe580cc0f0 tx=0x7efe4001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5c994640 1 -- 172.21.15.133:0/2495342512 shutdown_connections 2024-04-25T10:12:45.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5c994640 1 --2- 172.21.15.133:0/2495342512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe580b1f70 0x7efe5806dcf0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5c994640 1 --2- 172.21.15.133:0/2495342512 >> v2:172.21.15.162:3300/0 conn(0x7efe580ce330 0x7efe580ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5c994640 1 -- 172.21.15.133:0/2495342512 >> 172.21.15.133:0/2495342512 conn(0x7efe58067000 msgr2=0x7efe5806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:45.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5c994640 1 -- 172.21.15.133:0/2495342512 shutdown_connections 2024-04-25T10:12:45.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5c994640 1 -- 172.21.15.133:0/2495342512 wait complete. 2024-04-25T10:12:45.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.789+0000 7efe5c994640 1 Processor -- start 2024-04-25T10:12:45.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5c994640 1 -- start start 2024-04-25T10:12:45.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5c994640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe580b1f70 0x7efe580cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5c994640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe580ce330 0x7efe58163c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5c994640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe5815c9a0 0x7efe58165900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5c994640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5815dad0 con 0x7efe580ce330 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5c994640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5815d950 con 0x7efe5815c9a0 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5c994640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5815dc50 con 0x7efe580b1f70 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe55d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe580ce330 0x7efe58163c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe55d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe580ce330 0x7efe58163c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57094/0 (socket says 172.21.15.133:57094) 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe55d9b640 1 -- 172.21.15.133:0/1453928933 learned_addr learned my addr 172.21.15.133:0/1453928933 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5659c640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe580b1f70 0x7efe580cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe56d9d640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe5815c9a0 0x7efe58165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe55d9b640 1 -- 172.21.15.133:0/1453928933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe580b1f70 msgr2=0x7efe580cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe55d9b640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe580b1f70 0x7efe580cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe55d9b640 1 -- 172.21.15.133:0/1453928933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe5815c9a0 msgr2=0x7efe58165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe55d9b640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe5815c9a0 0x7efe58165900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe55d9b640 1 -- 172.21.15.133:0/1453928933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe580ccae0 con 0x7efe580ce330 2024-04-25T10:12:45.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5659c640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe580b1f70 0x7efe580cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:45.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe55d9b640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe580ce330 0x7efe58163c40 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7efe40019100 tx=0x7efe4001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:45.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe4effd640 1 -- 172.21.15.133:0/1453928933 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe400bd5f0 con 0x7efe580ce330 2024-04-25T10:12:45.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe4effd640 1 -- 172.21.15.133:0/1453928933 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe400bd790 con 0x7efe580ce330 2024-04-25T10:12:45.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5c994640 1 -- 172.21.15.133:0/1453928933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe580ccdc0 con 0x7efe580ce330 2024-04-25T10:12:45.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe4effd640 1 -- 172.21.15.133:0/1453928933 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe400bdb80 con 0x7efe580ce330 2024-04-25T10:12:45.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5c994640 1 -- 172.21.15.133:0/1453928933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe580cd380 con 0x7efe580ce330 2024-04-25T10:12:45.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5c994640 1 -- 172.21.15.133:0/1453928933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe58068ef0 con 0x7efe580ce330 2024-04-25T10:12:45.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe4effd640 1 -- 172.21.15.133:0/1453928933 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe400b33f0 con 0x7efe580ce330 2024-04-25T10:12:45.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe4effd640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe38066f60 0x7efe38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe4effd640 1 -- 172.21.15.133:0/1453928933 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe401200b0 con 0x7efe580ce330 2024-04-25T10:12:45.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5659c640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe38066f60 0x7efe38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.793+0000 7efe5659c640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe38066f60 0x7efe38069420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efe48007fd0 tx=0x7efe48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:45.797+0000 7efe4effd640 1 -- 172.21.15.133:0/1453928933 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe400c3060 con 0x7efe580ce330 2024-04-25T10:12:46.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe5c994640 1 -- 172.21.15.133:0/1453928933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efe580cbbb0 con 0x7efe580ce330 2024-04-25T10:12:46.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe4effd640 1 -- 172.21.15.133:0/1453928933 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7efe40016110 con 0x7efe580ce330 2024-04-25T10:12:46.019 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:46.019 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":678,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":120,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077879776,"bytes_used":2661990400,"bytes_avail":770432122880,"bytes_total":773094113280,"write_bytes_sec":69552336,"read_op_per_sec":0,"write_op_per_sec":70},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8919}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:12:45.917935+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:46.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe5c994640 1 -- 172.21.15.133:0/1453928933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe38066f60 msgr2=0x7efe38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe5c994640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe38066f60 0x7efe38069420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efe48007fd0 tx=0x7efe48009040 comp rx=0 tx=0).stop 2024-04-25T10:12:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe5c994640 1 -- 172.21.15.133:0/1453928933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe580ce330 msgr2=0x7efe58163c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe5c994640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe580ce330 0x7efe58163c40 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7efe40019100 tx=0x7efe4001ad30 comp rx=0 tx=0).stop 2024-04-25T10:12:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe5c994640 1 -- 172.21.15.133:0/1453928933 shutdown_connections 2024-04-25T10:12:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe5c994640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efe5815c9a0 0x7efe58165900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe5c994640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efe38066f60 0x7efe38069420 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe5c994640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe580ce330 0x7efe58163c40 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe5c994640 1 --2- 172.21.15.133:0/1453928933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efe580b1f70 0x7efe580cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.017+0000 7efe5c994640 1 -- 172.21.15.133:0/1453928933 >> 172.21.15.133:0/1453928933 conn(0x7efe58067000 msgr2=0x7efe580c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.021+0000 7efe5c994640 1 -- 172.21.15.133:0/1453928933 shutdown_connections 2024-04-25T10:12:46.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.021+0000 7efe5c994640 1 -- 172.21.15.133:0/1453928933 wait complete. 2024-04-25T10:12:46.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T10:12:46.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:46.038 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T10:12:46.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.153+0000 7f74990db640 1 Processor -- start 2024-04-25T10:12:46.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.153+0000 7f74990db640 1 -- start start 2024-04-25T10:12:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f74990db640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940d2030 0x7f74940cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f74990db640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74940d24c0 con 0x7f749415d9a0 2024-04-25T10:12:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f74990db640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749415ebd0 con 0x7f74940ce600 2024-04-25T10:12:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f74990db640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749415eeb0 con 0x7f74940d2030 2024-04-25T10:12:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7492d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f749415d9a0 0x7f74940d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45372/0 (socket says 172.21.15.133:45372) 2024-04-25T10:12:46.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7492d9d640 1 -- 172.21.15.133:0/2675561969 learned_addr learned my addr 172.21.15.133:0/2675561969 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:46.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f749359e640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940d2030 0x7f74940cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755878188 0 0) 0x7f74940d24c0 con 0x7f749415d9a0 2024-04-25T10:12:46.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f747c003530 con 0x7f749415d9a0 2024-04-25T10:12:46.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1560573573 0 0) 0x7f747c003530 con 0x7f749415d9a0 2024-04-25T10:12:46.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940d2030 msgr2=0x7f74940cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940d2030 0x7f74940cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 >> v1:172.21.15.162:6789/0 conn(0x7f74940ce600 legacy=0x7f74940d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940d0330 con 0x7f749415d9a0 2024-04-25T10:12:46.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7484001680 con 0x7f749415d9a0 2024-04-25T10:12:46.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 >> v1:172.21.15.133:6789/0 conn(0x7f749415d9a0 legacy=0x7f74940d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940d2030 0x7f747c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 0x7f747c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f747c004d50 0x7f747c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749415eeb0 con 0x7f747c003f70 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c00bb60 con 0x7f747c004d50 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c00bd60 con 0x7f74940d2030 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f749259c640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 0x7f747c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f749359e640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f747c004d50 0x7f747c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7492d9d640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940d2030 0x7f747c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f749259c640 1 -- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940d2030 msgr2=0x7f747c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f749259c640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940d2030 0x7f747c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f749259c640 1 -- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f747c004d50 msgr2=0x7f747c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f749259c640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f747c004d50 0x7f747c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f749259c640 1 -- 172.21.15.133:0/2675561969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c003530 con 0x7f747c003f70 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f749359e640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f747c004d50 0x7f747c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:46.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.157+0000 7f7492d9d640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940d2030 0x7f747c0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:46.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f749259c640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 0x7f747c004810 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f7488001780 tx=0x7f748801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e7439b34753b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74880013a0 con 0x7f747c003f70 2024-04-25T10:12:46.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74880052f0 con 0x7f747c003f70 2024-04-25T10:12:46.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7491d9b640 1 -- 172.21.15.133:0/2675561969 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488003840 con 0x7f747c003f70 2024-04-25T10:12:46.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 -- 172.21.15.133:0/2675561969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 msgr2=0x7f747c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 --2- 172.21.15.133:0/2675561969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 0x7f747c004810 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f7488001780 tx=0x7f748801c3f0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f749359e640 1 -- 172.21.15.133:0/2675561969 reap_dead start 2024-04-25T10:12:46.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 -- 172.21.15.133:0/2675561969 shutdown_connections 2024-04-25T10:12:46.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 -- 172.21.15.133:0/2675561969 >> 172.21.15.133:0/2675561969 conn(0x7f749406bf80 msgr2=0x7f749406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 -- 172.21.15.133:0/2675561969 shutdown_connections 2024-04-25T10:12:46.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 -- 172.21.15.133:0/2675561969 wait complete. 2024-04-25T10:12:46.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 Processor -- start 2024-04-25T10:12:46.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 -- start start 2024-04-25T10:12:46.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 0x7f749415c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74940c8ff0 0x7f74940cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940cc5b0 0x7f749415b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74940cee20 con 0x7f747c003f70 2024-04-25T10:12:46.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749415e590 con 0x7f74940c8ff0 2024-04-25T10:12:46.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f74990db640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74940cca60 con 0x7f74940cc5b0 2024-04-25T10:12:46.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7492d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 0x7f749415c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7492d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 0x7f749415c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57120/0 (socket says 172.21.15.133:57120) 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7492d9d640 1 -- 172.21.15.133:0/1324141031 learned_addr learned my addr 172.21.15.133:0/1324141031 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f749259c640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74940c8ff0 0x7f74940cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f749359e640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940cc5b0 0x7f749415b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7492d9d640 1 -- 172.21.15.133:0/1324141031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940cc5b0 msgr2=0x7f749415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7492d9d640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940cc5b0 0x7f749415b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7492d9d640 1 -- 172.21.15.133:0/1324141031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74940c8ff0 msgr2=0x7f74940cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7492d9d640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74940c8ff0 0x7f74940cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7492d9d640 1 -- 172.21.15.133:0/1324141031 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940ccf80 con 0x7f747c003f70 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f7492d9d640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 0x7f749415c8d0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f7484007fd0 tx=0x7f748400db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.161+0000 7f747b7fe640 1 -- 172.21.15.133:0/1324141031 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484019070 con 0x7f747c003f70 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f747b7fe640 1 -- 172.21.15.133:0/1324141031 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74840072d0 con 0x7f747c003f70 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f74990db640 1 -- 172.21.15.133:0/1324141031 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74940ccca0 con 0x7f747c003f70 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f747b7fe640 1 -- 172.21.15.133:0/1324141031 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484007680 con 0x7f747c003f70 2024-04-25T10:12:46.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f74990db640 1 -- 172.21.15.133:0/1324141031 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74940c87f0 con 0x7f747c003f70 2024-04-25T10:12:46.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f747b7fe640 1 -- 172.21.15.133:0/1324141031 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f748401c070 con 0x7f747c003f70 2024-04-25T10:12:46.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f747b7fe640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7480066f60 0x7f7480069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f749259c640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7480066f60 0x7f7480069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f747b7fe640 1 -- 172.21.15.133:0/1324141031 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7484084d30 con 0x7f747c003f70 2024-04-25T10:12:46.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f749259c640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7480066f60 0x7f7480069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f74880ac000 tx=0x7f74880011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f749c248000 1 Processor -- start 2024-04-25T10:12:46.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f749c248000 1 -- start start 2024-04-25T10:12:46.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f749c248000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67aa40 0x5627ea67ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f749c248000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67b360 0x5627ea635850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f749c248000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea636170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f749c248000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ea5181f0 con 0x5627ea67b360 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.165+0000 7f749c248000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ea47ee90 con 0x5627ea67aa40 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627ea406f50 con 0x5627ea635d90 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67b360 0x5627ea635850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67b360 0x5627ea635850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56362/0 (socket says 172.21.15.133:56362) 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 -- 172.21.15.133:0/3282666477 learned_addr learned my addr 172.21.15.133:0/3282666477 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749359e640 1 --2- 172.21.15.133:0/3282666477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea636170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f7492d9d640 1 --2- 172.21.15.133:0/3282666477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67aa40 0x5627ea67ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 -- 172.21.15.133:0/3282666477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 msgr2=0x5627ea636170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 --2- 172.21.15.133:0/3282666477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea636170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 -- 172.21.15.133:0/3282666477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67aa40 msgr2=0x5627ea67ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 --2- 172.21.15.133:0/3282666477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67aa40 0x5627ea67ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 -- 172.21.15.133:0/3282666477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627ea4568b0 con 0x5627ea67b360 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 --2- 172.21.15.133:0/3282666477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67b360 0x5627ea635850 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f74880ad040 tx=0x7f7488001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d58f1741f46a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f7478ff9640 1 -- 172.21.15.133:0/3282666477 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74880bb030 con 0x5627ea67b360 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f7478ff9640 1 -- 172.21.15.133:0/3282666477 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74880c7070 con 0x5627ea67b360 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f7478ff9640 1 -- 172.21.15.133:0/3282666477 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488007590 con 0x5627ea67b360 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- 172.21.15.133:0/3282666477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67b360 msgr2=0x5627ea635850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 --2- 172.21.15.133:0/3282666477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67b360 0x5627ea635850 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f74880ad040 tx=0x7f7488001130 comp rx=0 tx=0).stop 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- 172.21.15.133:0/3282666477 shutdown_connections 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 --2- 172.21.15.133:0/3282666477 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea636170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 --2- 172.21.15.133:0/3282666477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67b360 0x5627ea635850 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 --2- 172.21.15.133:0/3282666477 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67aa40 0x5627ea67ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- 172.21.15.133:0/3282666477 >> 172.21.15.133:0/3282666477 conn(0x5627ea631d40 msgr2=0x5627ea634130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- 172.21.15.133:0/3282666477 shutdown_connections 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- 172.21.15.133:0/3282666477 wait complete. 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- start start 2024-04-25T10:12:46.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 Processor -- start 2024-04-25T10:12:46.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea682700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 0x5627ea63be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ea439540 con 0x5627ea67aa40 2024-04-25T10:12:46.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ea47af30 con 0x5627ea67b360 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627ea45d010 con 0x5627ea635d90 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 10 client.0 _ll_register_callbacks cb 0x5627ea533180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea682700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea682700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56370/0 (socket says 172.21.15.133:56370) 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 -- 172.21.15.133:0/3127523063 learned_addr learned my addr 172.21.15.133:0/3127523063 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749359e640 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 0x5627ea63be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f7492d9d640 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 -- 172.21.15.133:0/3127523063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 msgr2=0x5627ea6821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6821c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 -- 172.21.15.133:0/3127523063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 msgr2=0x5627ea63be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 0x5627ea63be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 -- 172.21.15.133:0/3127523063 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627ea4071a0 con 0x5627ea67aa40 2024-04-25T10:12:46.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f7492d9d640 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749259c640 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea682700 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f748801c8d0 tx=0x7f7488019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bde25921b79d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f7471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f7471ffb640 1 -- 172.21.15.133:0/3127523063 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488006d30 con 0x5627ea67aa40 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f7471ffb640 1 -- 172.21.15.133:0/3127523063 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7488003600 con 0x5627ea67aa40 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- 172.21.15.133:0/3127523063 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74880c7070 con 0x5627ea67aa40 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f7471ffb640 1 -- 172.21.15.133:0/3127523063 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74880db050 con 0x5627ea67aa40 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 10 client.9174 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.169+0000 7f749c248000 1 -- 172.21.15.133:0/3127523063 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5627ea501840 con 0x5627ea67aa40 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f7471ffb640 1 -- 172.21.15.133:0/3127523063 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74880c7070 con 0x5627ea67aa40 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f7471ffb640 1 -- 172.21.15.133:0/3127523063 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f74880c1070 con 0x5627ea67aa40 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 10 client.9174 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 10 client.9174 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 20 client.9174 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:46.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5627ea63cc00 0x5627ea63cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 4 client.9174 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 -- 172.21.15.133:0/3127523063 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627ea41abe0 con 0x5627ea63cc00 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f7492d9d640 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5627ea63cc00 0x5627ea63cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f7471ffb640 10 client.9174 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f7492d9d640 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5627ea63cc00 0x5627ea63cfe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df902fe56833bea6 server_cookie=5a715c49a6ffeec3 in_seq=0 out_seq=0 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f7471ffb640 1 -- 172.21.15.133:0/3127523063 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5627ea41abe0 con 0x5627ea63cc00 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f7471ffb640 10 client.9174 handle_command_reply: tid=0 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 client.9174 shutdown 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 -- 172.21.15.133:0/3127523063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 msgr2=0x5627ea682700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea682700 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f748801c8d0 tx=0x7f7488019190 comp rx=0 tx=0).stop 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 -- 172.21.15.133:0/3127523063 shutdown_connections 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5627ea63cc00 0x5627ea63cfe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 0x5627ea63be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea682700 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f7471ffb640 0 client.9174 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:46.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 --2- 172.21.15.133:0/3127523063 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 -- 172.21.15.133:0/3127523063 >> 172.21.15.133:0/3127523063 conn(0x5627ea631d40 msgr2=0x5627ea6349c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 -- 172.21.15.133:0/3127523063 shutdown_connections 2024-04-25T10:12:46.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 1 -- 172.21.15.133:0/3127523063 wait complete. 2024-04-25T10:12:46.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.173+0000 7f749c248000 20 client.9174 trim_cache size 0 max 16384 2024-04-25T10:12:46.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 Processor -- start 2024-04-25T10:12:46.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- start start 2024-04-25T10:12:46.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 0x5627ea6e0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7494203a50 con 0x5627ea67aa40 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7494202bf0 con 0x5627ea67b360 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7494202020 con 0x5627ea635d90 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56374/0 (socket says 172.21.15.133:56374) 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 -- 172.21.15.133:0/3437749912 learned_addr learned my addr 172.21.15.133:0/3437749912 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 -- 172.21.15.133:0/3437749912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 msgr2=0x5627ea6dfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 --2- 172.21.15.133:0/3437749912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 -- 172.21.15.133:0/3437749912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 msgr2=0x5627ea6e0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 --2- 172.21.15.133:0/3437749912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 0x5627ea6e0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 -- 172.21.15.133:0/3437749912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627ea4d8a10 con 0x5627ea67aa40 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 --2- 172.21.15.133:0/3437749912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e0400 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7488097e00 tx=0x7f74880cc000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e8f9e13738b3f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f7473fff640 1 -- 172.21.15.133:0/3437749912 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74880bb070 con 0x5627ea67aa40 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f7473fff640 1 -- 172.21.15.133:0/3437749912 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74880bc040 con 0x5627ea67aa40 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f7473fff640 1 -- 172.21.15.133:0/3437749912 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488007630 con 0x5627ea67aa40 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- 172.21.15.133:0/3437749912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 msgr2=0x5627ea6e0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 --2- 172.21.15.133:0/3437749912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e0400 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7488097e00 tx=0x7f74880cc000 comp rx=0 tx=0).stop 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- 172.21.15.133:0/3437749912 shutdown_connections 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 --2- 172.21.15.133:0/3437749912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 0x5627ea6e0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 --2- 172.21.15.133:0/3437749912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e0400 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 --2- 172.21.15.133:0/3437749912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6dfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- 172.21.15.133:0/3437749912 >> 172.21.15.133:0/3437749912 conn(0x5627ea631d40 msgr2=0x5627ea6791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- 172.21.15.133:0/3437749912 shutdown_connections 2024-04-25T10:12:46.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- 172.21.15.133:0/3437749912 wait complete. 2024-04-25T10:12:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- start start 2024-04-25T10:12:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 Processor -- start 2024-04-25T10:12:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 0x5627ea67fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627ea442720 con 0x5627ea67aa40 2024-04-25T10:12:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74942025f0 con 0x5627ea67b360 2024-04-25T10:12:46.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627ea4222d0 con 0x5627ea635d90 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749c248000 10 client.0 _ll_register_callbacks cb 0x5627ea441a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56390/0 (socket says 172.21.15.133:56390) 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 -- 172.21.15.133:0/3421950499 learned_addr learned my addr 172.21.15.133:0/3421950499 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749359e640 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 0x5627ea67fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f7492d9d640 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 -- 172.21.15.133:0/3421950499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 msgr2=0x5627ea6e3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6e3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 -- 172.21.15.133:0/3421950499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 msgr2=0x5627ea67fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 0x5627ea67fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f749259c640 1 -- 172.21.15.133:0/3421950499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627ea413b00 con 0x5627ea67aa40 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.189+0000 7f7492d9d640 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6e3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:46.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749259c640 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e4200 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7488000bf0 tx=0x7f74880a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b4ffad1d29bf99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7471ffb640 1 -- 172.21.15.133:0/3421950499 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74880ec060 con 0x5627ea67aa40 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7471ffb640 1 -- 172.21.15.133:0/3421950499 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7488003600 con 0x5627ea67aa40 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 -- 172.21.15.133:0/3421950499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74880bc040 con 0x5627ea67aa40 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7471ffb640 1 -- 172.21.15.133:0/3421950499 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74880f2050 con 0x5627ea67aa40 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 10 client.9180 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 -- 172.21.15.133:0/3421950499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5627ea425f80 con 0x5627ea67aa40 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7471ffb640 1 -- 172.21.15.133:0/3421950499 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74880bc040 con 0x5627ea67aa40 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7471ffb640 1 -- 172.21.15.133:0/3421950499 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f74880f5020 con 0x5627ea67aa40 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 10 client.9180 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 10 client.9180 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 20 client.9180 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5627ea680830 0x5627ea680c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 4 client.9180 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 -- 172.21.15.133:0/3421950499 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x5627ea48c3d0 con 0x5627ea680830 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7492d9d640 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5627ea680830 0x5627ea680c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7471ffb640 10 client.9180 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7492d9d640 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5627ea680830 0x5627ea680c10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4676e5bdb5efb027 server_cookie=12c519d8493396fd in_seq=0 out_seq=0 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7471ffb640 1 -- 172.21.15.133:0/3421950499 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5627ea48c3d0 con 0x5627ea680830 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7471ffb640 10 client.9180 handle_command_reply: tid=0 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 client.9180 shutdown 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 -- 172.21.15.133:0/3421950499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 msgr2=0x5627ea6e4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e4200 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7488000bf0 tx=0x7f74880a6000 comp rx=0 tx=0).stop 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 -- 172.21.15.133:0/3421950499 shutdown_connections 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5627ea680830 0x5627ea680c10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5627ea67b360 0x5627ea67fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627ea67aa40 0x5627ea6e4200 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 --2- 172.21.15.133:0/3421950499 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5627ea635d90 0x5627ea6e3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f7471ffb640 0 client.9180 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:46.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 -- 172.21.15.133:0/3421950499 >> 172.21.15.133:0/3421950499 conn(0x5627ea631d40 msgr2=0x5627ea679a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 -- 172.21.15.133:0/3421950499 shutdown_connections 2024-04-25T10:12:46.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 1 -- 172.21.15.133:0/3421950499 wait complete. 2024-04-25T10:12:46.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f749c248000 20 client.9180 trim_cache size 0 max 16384 2024-04-25T10:12:46.201 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:12:46.201 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:12:46.201 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:12:46.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 -- 172.21.15.133:0/1324141031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7480066f60 msgr2=0x7f7480069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7480066f60 0x7f7480069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f74880ac000 tx=0x7f74880011a0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 -- 172.21.15.133:0/1324141031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 msgr2=0x7f749415c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 0x7f749415c8d0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f7484007fd0 tx=0x7f748400db80 comp rx=0 tx=0).stop 2024-04-25T10:12:46.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 -- 172.21.15.133:0/1324141031 shutdown_connections 2024-04-25T10:12:46.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f74940cc5b0 0x7f749415b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f74940c8ff0 0x7f74940cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7480066f60 0x7f7480069420 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 --2- 172.21.15.133:0/1324141031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f747c003f70 0x7f749415c8d0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 -- 172.21.15.133:0/1324141031 >> 172.21.15.133:0/1324141031 conn(0x7f749406bf80 msgr2=0x7f74940cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 -- 172.21.15.133:0/1324141031 shutdown_connections 2024-04-25T10:12:46.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.193+0000 7f74990db640 1 -- 172.21.15.133:0/1324141031 wait complete. 2024-04-25T10:12:46.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:46.212 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:12:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.325+0000 7f240c2e5640 1 Processor -- start 2024-04-25T10:12:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.325+0000 7f240c2e5640 1 -- start start 2024-04-25T10:12:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.325+0000 7f240c2e5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f240415b8b0 0x7f24040ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.325+0000 7f240c2e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415bc90 0x7f24040cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.325+0000 7f240c2e5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24040cc6a0 con 0x7f24040d01d0 2024-04-25T10:12:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.325+0000 7f240c2e5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24040d0640 con 0x7f240415b8b0 2024-04-25T10:12:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.325+0000 7f240c2e5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24040cc980 con 0x7f240415bc90 2024-04-25T10:12:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.329+0000 7f2409880640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415bc90 0x7f24040cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.329+0000 7f240a081640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f240415b8b0 0x7f24040ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.329+0000 7f2409880640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415bc90 0x7f24040cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:44854/0 (socket says 172.21.15.133:44854) 2024-04-25T10:12:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.329+0000 7f240a882640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f24040d01d0 0x7f240415f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:36926/0 (socket says 172.21.15.133:36926) 2024-04-25T10:12:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.329+0000 7f2409880640 1 -- 172.21.15.133:0/1533328542 learned_addr learned my addr 172.21.15.133:0/1533328542 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.329+0000 7f2409880640 1 -- 172.21.15.133:0/1533328542 >> v2:172.21.15.162:3300/0 conn(0x7f240415b8b0 msgr2=0x7f24040ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.329+0000 7f2409880640 1 --2- 172.21.15.133:0/1533328542 >> v2:172.21.15.162:3300/0 conn(0x7f240415b8b0 0x7f24040ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.329+0000 7f240907f640 1 -- 172.21.15.133:0/1533328542 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169486448 0 0) 0x7f24040cc6a0 con 0x7f24040d01d0 2024-04-25T10:12:46.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.329+0000 7f2409880640 1 -- 172.21.15.133:0/1533328542 >> v1:172.21.15.162:6789/0 conn(0x7f24040d01d0 legacy=0x7f240415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.329+0000 7f2409880640 1 -- 172.21.15.133:0/1533328542 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24040ccc60 con 0x7f240415bc90 2024-04-25T10:12:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f2409880640 1 --2- 172.21.15.133:0/1533328542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415bc90 0x7f24040cfc90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009870 tx=0x7f23f401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e15f2ffae02c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f240907f640 1 -- 172.21.15.133:0/1533328542 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f40b0070 con 0x7f240415bc90 2024-04-25T10:12:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f240907f640 1 -- 172.21.15.133:0/1533328542 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f40ab600 con 0x7f240415bc90 2024-04-25T10:12:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f240907f640 1 -- 172.21.15.133:0/1533328542 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f40ab9d0 con 0x7f240415bc90 2024-04-25T10:12:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f240c2e5640 1 -- 172.21.15.133:0/1533328542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415bc90 msgr2=0x7f24040cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f240c2e5640 1 --2- 172.21.15.133:0/1533328542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415bc90 0x7f24040cfc90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009870 tx=0x7f23f401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f240c2e5640 1 -- 172.21.15.133:0/1533328542 shutdown_connections 2024-04-25T10:12:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f240c2e5640 1 --2- 172.21.15.133:0/1533328542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415bc90 0x7f24040cfc90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f240c2e5640 1 --2- 172.21.15.133:0/1533328542 >> v2:172.21.15.162:3300/0 conn(0x7f240415b8b0 0x7f24040ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f240c2e5640 1 -- 172.21.15.133:0/1533328542 >> 172.21.15.133:0/1533328542 conn(0x7f240409a990 msgr2=0x7f24040d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f240c2e5640 1 -- 172.21.15.133:0/1533328542 shutdown_connections 2024-04-25T10:12:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.333+0000 7f240c2e5640 1 -- 172.21.15.133:0/1533328542 wait complete. 2024-04-25T10:12:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240c2e5640 1 Processor -- start 2024-04-25T10:12:46.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240c2e5640 1 -- start start 2024-04-25T10:12:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240c2e5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24040d01d0 0x7f24040b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240c2e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415b8b0 0x7f24040cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240c2e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240415bc90 0x7f24040c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240c2e5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24040cc3b0 con 0x7f240415bc90 2024-04-25T10:12:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240c2e5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24040cc230 con 0x7f24040d01d0 2024-04-25T10:12:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240c2e5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24040b1ab0 con 0x7f240415b8b0 2024-04-25T10:12:46.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240a882640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240415bc90 0x7f24040c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240a882640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240415bc90 0x7f24040c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56404/0 (socket says 172.21.15.133:56404) 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240a882640 1 -- 172.21.15.133:0/805436783 learned_addr learned my addr 172.21.15.133:0/805436783 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f2409880640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415b8b0 0x7f24040cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240a081640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24040d01d0 0x7f24040b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240a882640 1 -- 172.21.15.133:0/805436783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415b8b0 msgr2=0x7f24040cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240a882640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415b8b0 0x7f24040cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240a882640 1 -- 172.21.15.133:0/805436783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24040d01d0 msgr2=0x7f24040b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240a882640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24040d01d0 0x7f24040b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240a882640 1 -- 172.21.15.133:0/805436783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2404070d00 con 0x7f240415bc90 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240a882640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240415bc90 0x7f24040c8f30 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00c700 tx=0x7f23fc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f23faffd640 1 -- 172.21.15.133:0/805436783 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc018070 con 0x7f240415bc90 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f23faffd640 1 -- 172.21.15.133:0/805436783 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23fc00ce10 con 0x7f240415bc90 2024-04-25T10:12:46.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f23faffd640 1 -- 172.21.15.133:0/805436783 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc00a880 con 0x7f240415bc90 2024-04-25T10:12:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240c2e5640 1 -- 172.21.15.133:0/805436783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24040b63a0 con 0x7f240415bc90 2024-04-25T10:12:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.337+0000 7f240c2e5640 1 -- 172.21.15.133:0/805436783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24040b6960 con 0x7f240415bc90 2024-04-25T10:12:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f23faffd640 1 -- 172.21.15.133:0/805436783 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23fc00aa20 con 0x7f240415bc90 2024-04-25T10:12:46.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f23faffd640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f23e8066f60 0x7f23e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f23faffd640 1 -- 172.21.15.133:0/805436783 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f23fc084610 con 0x7f240415bc90 2024-04-25T10:12:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a081640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f23e8066f60 0x7f23e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a081640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f23e8066f60 0x7f23e8069420 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f24040b6fa0 tx=0x7f2400009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240f452000 1 Processor -- start 2024-04-25T10:12:46.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240f452000 1 -- start start 2024-04-25T10:12:46.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240f452000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7b19d0 0x55693b7b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240f452000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 0x55693b767b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240f452000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b768480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240f452000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693b5b4ff0 con 0x55693b7680a0 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240f452000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693b53d150 con 0x55693b7b19d0 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240f452000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55693b5b1090 con 0x55693b7b22f0 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a882640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b768480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a882640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b768480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56420/0 (socket says 172.21.15.133:56420) 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a882640 1 -- 172.21.15.133:0/2368915272 learned_addr learned my addr 172.21.15.133:0/2368915272 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a081640 1 --2- 172.21.15.133:0/2368915272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7b19d0 0x55693b7b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f2409880640 1 --2- 172.21.15.133:0/2368915272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 0x55693b767b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a882640 1 -- 172.21.15.133:0/2368915272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 msgr2=0x55693b767b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a882640 1 --2- 172.21.15.133:0/2368915272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 0x55693b767b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a882640 1 -- 172.21.15.133:0/2368915272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7b19d0 msgr2=0x55693b7b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a882640 1 --2- 172.21.15.133:0/2368915272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7b19d0 0x55693b7b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a882640 1 -- 172.21.15.133:0/2368915272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55693b74b0d0 con 0x55693b7680a0 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a081640 1 --2- 172.21.15.133:0/2368915272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7b19d0 0x55693b7b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f240a882640 1 --2- 172.21.15.133:0/2368915272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b768480 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00c6d0 tx=0x7f23fc054f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dede2b7a9c7bf7a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f23e7fff640 1 -- 172.21.15.133:0/2368915272 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc0531d0 con 0x55693b7680a0 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f23e7fff640 1 -- 172.21.15.133:0/2368915272 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23fc00ad10 con 0x55693b7680a0 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.341+0000 7f23e7fff640 1 -- 172.21.15.133:0/2368915272 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc05b070 con 0x55693b7680a0 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- 172.21.15.133:0/2368915272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 msgr2=0x55693b768480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 --2- 172.21.15.133:0/2368915272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b768480 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00c6d0 tx=0x7f23fc054f90 comp rx=0 tx=0).stop 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- 172.21.15.133:0/2368915272 shutdown_connections 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 --2- 172.21.15.133:0/2368915272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b768480 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 --2- 172.21.15.133:0/2368915272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 0x55693b767b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 --2- 172.21.15.133:0/2368915272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7b19d0 0x55693b7b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- 172.21.15.133:0/2368915272 >> 172.21.15.133:0/2368915272 conn(0x55693b764020 msgr2=0x55693b766440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- 172.21.15.133:0/2368915272 shutdown_connections 2024-04-25T10:12:46.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- 172.21.15.133:0/2368915272 wait complete. 2024-04-25T10:12:46.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- start start 2024-04-25T10:12:46.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 Processor -- start 2024-04-25T10:12:46.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7680a0 0x55693b7967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b19d0 0x55693b796d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 0x55693b797270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693b579250 con 0x55693b7b19d0 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693b56f690 con 0x55693b7680a0 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55693b53da40 con 0x55693b7b22f0 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 10 client.0 _ll_register_callbacks cb 0x55693b752e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f2409880640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b19d0 0x55693b796d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f2409880640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b19d0 0x55693b796d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56434/0 (socket says 172.21.15.133:56434) 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f2409880640 1 -- 172.21.15.133:0/572299711 learned_addr learned my addr 172.21.15.133:0/572299711 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240a081640 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7680a0 0x55693b7967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240a882640 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 0x55693b797270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f2409880640 1 -- 172.21.15.133:0/572299711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 msgr2=0x55693b797270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f2409880640 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 0x55693b797270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f2409880640 1 -- 172.21.15.133:0/572299711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7680a0 msgr2=0x55693b7967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f2409880640 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7680a0 0x55693b7967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f2409880640 1 -- 172.21.15.133:0/572299711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55693b6621c0 con 0x55693b7b19d0 2024-04-25T10:12:46.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f2409880640 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b19d0 0x55693b796d30 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f23f401c510 tx=0x7f23f401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4439d1874f25ccd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f23e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f23e57fa640 1 -- 172.21.15.133:0/572299711 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f40b0040 con 0x55693b7b19d0 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f23e57fa640 1 -- 172.21.15.133:0/572299711 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f23f40abc30 con 0x55693b7b19d0 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- 172.21.15.133:0/572299711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23fc00ad10 con 0x55693b7b19d0 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f23e57fa640 1 -- 172.21.15.133:0/572299711 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23f40c5050 con 0x55693b7b19d0 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 10 client.9189 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- 172.21.15.133:0/572299711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55693b65e170 con 0x55693b7b19d0 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f23e57fa640 1 -- 172.21.15.133:0/572299711 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23fc00ad10 con 0x55693b7b19d0 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f23e57fa640 1 -- 172.21.15.133:0/572299711 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f23f40b6450 con 0x55693b7b19d0 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 10 client.9189 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 10 client.9189 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 20 client.9189 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55693b7c1c80 0x55693b7c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 4 client.9189 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 -- 172.21.15.133:0/572299711 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55693b550c60 con 0x55693b7c1c80 2024-04-25T10:12:46.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240a081640 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55693b7c1c80 0x55693b7c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f23e57fa640 10 client.9189 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240a081640 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55693b7c1c80 0x55693b7c4070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a90075a69658f02c server_cookie=3ebe77759a30adc5 in_seq=0 out_seq=0 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f23e57fa640 1 -- 172.21.15.133:0/572299711 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55693b550c60 con 0x55693b7c1c80 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f23e57fa640 10 client.9189 handle_command_reply: tid=0 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.345+0000 7f240f452000 1 client.9189 shutdown 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f240f452000 1 -- 172.21.15.133:0/572299711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b19d0 msgr2=0x55693b796d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f240f452000 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b19d0 0x55693b796d30 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f23f401c510 tx=0x7f23f401ad10 comp rx=0 tx=0).stop 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f240f452000 1 -- 172.21.15.133:0/572299711 shutdown_connections 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f240f452000 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55693b7c1c80 0x55693b7c4070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f240f452000 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 0x55693b797270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f23e57fa640 0 client.9189 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f240f452000 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b19d0 0x55693b796d30 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f240f452000 1 --2- 172.21.15.133:0/572299711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7680a0 0x55693b7967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f240f452000 1 -- 172.21.15.133:0/572299711 >> 172.21.15.133:0/572299711 conn(0x55693b764020 msgr2=0x55693b766ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f240f452000 1 -- 172.21.15.133:0/572299711 shutdown_connections 2024-04-25T10:12:46.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f240f452000 1 -- 172.21.15.133:0/572299711 wait complete. 2024-04-25T10:12:46.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.349+0000 7f240f452000 20 client.9189 trim_cache size 0 max 16384 2024-04-25T10:12:46.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240f452000 1 Processor -- start 2024-04-25T10:12:46.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240f452000 1 -- start start 2024-04-25T10:12:46.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240f452000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7680a0 0x55693b812130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240f452000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b19d0 0x55693b812670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240f452000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b22f0 0x55693b812bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240f452000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2404200e50 con 0x55693b7b22f0 2024-04-25T10:12:46.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240f452000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2404200020 con 0x55693b7680a0 2024-04-25T10:12:46.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240f452000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2404159b80 con 0x55693b7b19d0 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240a882640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b22f0 0x55693b812bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240a882640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b22f0 0x55693b812bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56438/0 (socket says 172.21.15.133:56438) 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240a882640 1 -- 172.21.15.133:0/4174289232 learned_addr learned my addr 172.21.15.133:0/4174289232 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240a882640 1 -- 172.21.15.133:0/4174289232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b19d0 msgr2=0x55693b812670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240a882640 1 --2- 172.21.15.133:0/4174289232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b19d0 0x55693b812670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240a882640 1 -- 172.21.15.133:0/4174289232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7680a0 msgr2=0x55693b812130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240a882640 1 --2- 172.21.15.133:0/4174289232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7680a0 0x55693b812130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240a882640 1 -- 172.21.15.133:0/4174289232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55693b5fbde0 con 0x55693b7b22f0 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f240a882640 1 --2- 172.21.15.133:0/4174289232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b22f0 0x55693b812bb0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f23fc054bd0 tx=0x7f23fc05b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46984dcc57d836a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f23e77fe640 1 -- 172.21.15.133:0/4174289232 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc00ad10 con 0x55693b7b22f0 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f23e77fe640 1 -- 172.21.15.133:0/4174289232 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23fc051540 con 0x55693b7b22f0 2024-04-25T10:12:46.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.361+0000 7f23e77fe640 1 -- 172.21.15.133:0/4174289232 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc054320 con 0x55693b7b22f0 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/4174289232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b22f0 msgr2=0x55693b812bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- 172.21.15.133:0/4174289232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b22f0 0x55693b812bb0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f23fc054bd0 tx=0x7f23fc05b000 comp rx=0 tx=0).stop 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/4174289232 shutdown_connections 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- 172.21.15.133:0/4174289232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7b22f0 0x55693b812bb0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- 172.21.15.133:0/4174289232 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b19d0 0x55693b812670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- 172.21.15.133:0/4174289232 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7680a0 0x55693b812130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/4174289232 >> 172.21.15.133:0/4174289232 conn(0x55693b764020 msgr2=0x55693b7b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/4174289232 shutdown_connections 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/4174289232 wait complete. 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- start start 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 Processor -- start 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b816380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7b19d0 0x55693b8168c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 0x55693b7d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55693b54fe00 con 0x55693b7680a0 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240415a180 con 0x55693b7b19d0 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55693b5b4300 con 0x55693b7b22f0 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 10 client.0 _ll_register_callbacks cb 0x55693b733040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240a081640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b816380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240a081640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b816380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56450/0 (socket says 172.21.15.133:56450) 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240a081640 1 -- 172.21.15.133:0/2128011674 learned_addr learned my addr 172.21.15.133:0/2128011674 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f2409880640 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7b19d0 0x55693b8168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240a081640 1 -- 172.21.15.133:0/2128011674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 msgr2=0x55693b7d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240a081640 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 0x55693b7d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240a081640 1 -- 172.21.15.133:0/2128011674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7b19d0 msgr2=0x55693b8168c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240a081640 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7b19d0 0x55693b8168c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240a081640 1 -- 172.21.15.133:0/2128011674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55693b5568e0 con 0x55693b7680a0 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240a081640 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b816380 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2400006a70 tx=0x7f2400011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798f44d073a25ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f23e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f23e57fa640 1 -- 172.21.15.133:0/2128011674 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240002c060 con 0x55693b7680a0 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f23e57fa640 1 -- 172.21.15.133:0/2128011674 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2400014d60 con 0x55693b7680a0 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/2128011674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23fc051540 con 0x55693b7680a0 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f23e57fa640 1 -- 172.21.15.133:0/2128011674 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2400032050 con 0x55693b7680a0 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 10 client.9195 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/2128011674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55693b60e8f0 con 0x55693b7680a0 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f23e57fa640 1 -- 172.21.15.133:0/2128011674 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23fc051540 con 0x55693b7680a0 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f23e57fa640 1 -- 172.21.15.133:0/2128011674 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2400035020 con 0x55693b7680a0 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 10 client.9195 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 10 client.9195 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 20 client.9195 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55693b7d2ba0 0x55693b7d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 4 client.9195 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/2128011674 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55693b5c2530 con 0x55693b7d2ba0 2024-04-25T10:12:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f2409880640 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55693b7d2ba0 0x55693b7d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f2409880640 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55693b7d2ba0 0x55693b7d2fb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db59cd1f5befdb18 server_cookie=5d311a7e90c20be7 in_seq=0 out_seq=0 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f23e57fa640 10 client.9195 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f23e57fa640 1 -- 172.21.15.133:0/2128011674 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x55693b5c2530 con 0x55693b7d2ba0 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f23e57fa640 10 client.9195 handle_command_reply: tid=0 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 client.9195 shutdown 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/2128011674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 msgr2=0x55693b816380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b816380 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2400006a70 tx=0x7f2400011f20 comp rx=0 tx=0).stop 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/2128011674 shutdown_connections 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x55693b7d2ba0 0x55693b7d2fb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55693b7b22f0 0x55693b7d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f23e57fa640 0 client.9195 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55693b7b19d0 0x55693b8168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 --2- 172.21.15.133:0/2128011674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55693b7680a0 0x55693b816380 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/2128011674 >> 172.21.15.133:0/2128011674 conn(0x55693b764020 msgr2=0x55693b7b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/2128011674 shutdown_connections 2024-04-25T10:12:46.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 1 -- 172.21.15.133:0/2128011674 wait complete. 2024-04-25T10:12:46.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.365+0000 7f240f452000 20 client.9195 trim_cache size 0 max 16384 2024-04-25T10:12:46.375 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:12:46.375 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:12:46.375 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:12:46.375 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:12:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 -- 172.21.15.133:0/805436783 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f23e8066f60 msgr2=0x7f23e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f23e8066f60 0x7f23e8069420 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f24040b6fa0 tx=0x7f2400009190 comp rx=0 tx=0).stop 2024-04-25T10:12:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 -- 172.21.15.133:0/805436783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240415bc90 msgr2=0x7f24040c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240415bc90 0x7f24040c8f30 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00c700 tx=0x7f23fc00cbc0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 -- 172.21.15.133:0/805436783 shutdown_connections 2024-04-25T10:12:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f23e8066f60 0x7f23e8069420 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f240415bc90 0x7f24040c8f30 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f240415b8b0 0x7f24040cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 --2- 172.21.15.133:0/805436783 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24040d01d0 0x7f24040b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 -- 172.21.15.133:0/805436783 >> 172.21.15.133:0/805436783 conn(0x7f240409a990 msgr2=0x7f24040b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 -- 172.21.15.133:0/805436783 shutdown_connections 2024-04-25T10:12:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.369+0000 7f240c2e5640 1 -- 172.21.15.133:0/805436783 wait complete. 2024-04-25T10:12:46.385 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.485+0000 7fda5ba5b640 1 Processor -- start 2024-04-25T10:12:46.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.485+0000 7fda5ba5b640 1 -- start start 2024-04-25T10:12:46.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda5ba5b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fda5415db00 0x7fda540b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda5ba5b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 0x7fda540d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda5ba5b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda540ceb20 con 0x7fda541614c0 2024-04-25T10:12:46.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda5ba5b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda540d1110 con 0x7fda5415db00 2024-04-25T10:12:46.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda5ba5b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda540b7740 con 0x7fda540d0770 2024-04-25T10:12:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda597f7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fda5415db00 0x7fda540b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda58ff6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 0x7fda540d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda58ff6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 0x7fda540d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:44912/0 (socket says 172.21.15.133:44912) 2024-04-25T10:12:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda59ff8640 1 --1- 172.21.15.133:0/2319222778 >> v1:172.21.15.162:6789/0 conn(0x7fda541614c0 0x7fda54161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:36940/0 (socket says 172.21.15.133:36940) 2024-04-25T10:12:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda58ff6640 1 -- 172.21.15.133:0/2319222778 learned_addr learned my addr 172.21.15.133:0/2319222778 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda58ff6640 1 -- 172.21.15.133:0/2319222778 >> v2:172.21.15.162:3300/0 conn(0x7fda5415db00 msgr2=0x7fda540b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda58ff6640 1 --2- 172.21.15.133:0/2319222778 >> v2:172.21.15.162:3300/0 conn(0x7fda5415db00 0x7fda540b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda58ff6640 1 -- 172.21.15.133:0/2319222778 >> v1:172.21.15.162:6789/0 conn(0x7fda541614c0 legacy=0x7fda54161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.489+0000 7fda58ff6640 1 -- 172.21.15.133:0/2319222778 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda540b97f0 con 0x7fda540d0770 2024-04-25T10:12:46.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda58ff6640 1 --2- 172.21.15.133:0/2319222778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 0x7fda540d0bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda50009870 tx=0x7fda5001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f06d5b977581599 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda4bfff640 1 -- 172.21.15.133:0/2319222778 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda500a95f0 con 0x7fda540d0770 2024-04-25T10:12:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda4bfff640 1 -- 172.21.15.133:0/2319222778 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda500a9790 con 0x7fda540d0770 2024-04-25T10:12:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda4bfff640 1 -- 172.21.15.133:0/2319222778 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda500a9b60 con 0x7fda540d0770 2024-04-25T10:12:46.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 -- 172.21.15.133:0/2319222778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 msgr2=0x7fda540d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 --2- 172.21.15.133:0/2319222778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 0x7fda540d0bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda50009870 tx=0x7fda5001c530 comp rx=0 tx=0).stop 2024-04-25T10:12:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 -- 172.21.15.133:0/2319222778 shutdown_connections 2024-04-25T10:12:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 --2- 172.21.15.133:0/2319222778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 0x7fda540d0bd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 --2- 172.21.15.133:0/2319222778 >> v2:172.21.15.162:3300/0 conn(0x7fda5415db00 0x7fda540b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 -- 172.21.15.133:0/2319222778 >> 172.21.15.133:0/2319222778 conn(0x7fda540727c0 msgr2=0x7fda54072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 -- 172.21.15.133:0/2319222778 shutdown_connections 2024-04-25T10:12:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 -- 172.21.15.133:0/2319222778 wait complete. 2024-04-25T10:12:46.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 Processor -- start 2024-04-25T10:12:46.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 -- start start 2024-04-25T10:12:46.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 0x7fda540c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda5415db00 0x7fda540c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda541614c0 0x7fda541563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda540b7430 con 0x7fda541614c0 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda540b82c0 con 0x7fda5415db00 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda540d18a0 con 0x7fda540d0770 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda59ff8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda541614c0 0x7fda541563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda59ff8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda541614c0 0x7fda541563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56456/0 (socket says 172.21.15.133:56456) 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda59ff8640 1 -- 172.21.15.133:0/265241645 learned_addr learned my addr 172.21.15.133:0/265241645 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda597f7640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 0x7fda540c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda58ff6640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda5415db00 0x7fda540c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda59ff8640 1 -- 172.21.15.133:0/265241645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 msgr2=0x7fda540c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda59ff8640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 0x7fda540c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda59ff8640 1 -- 172.21.15.133:0/265241645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda5415db00 msgr2=0x7fda540c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda59ff8640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda5415db00 0x7fda540c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda59ff8640 1 -- 172.21.15.133:0/265241645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5415e080 con 0x7fda541614c0 2024-04-25T10:12:46.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda59ff8640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda541614c0 0x7fda541563b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00c7a0 tx=0x7fda4c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda49ffb640 1 -- 172.21.15.133:0/265241645 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c017070 con 0x7fda541614c0 2024-04-25T10:12:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda49ffb640 1 -- 172.21.15.133:0/265241645 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c00a640 con 0x7fda541614c0 2024-04-25T10:12:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 -- 172.21.15.133:0/265241645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5415e360 con 0x7fda541614c0 2024-04-25T10:12:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda49ffb640 1 -- 172.21.15.133:0/265241645 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00a9f0 con 0x7fda541614c0 2024-04-25T10:12:46.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.493+0000 7fda5ba5b640 1 -- 172.21.15.133:0/265241645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda540ce620 con 0x7fda541614c0 2024-04-25T10:12:46.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.497+0000 7fda5ba5b640 1 -- 172.21.15.133:0/265241645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda54068ef0 con 0x7fda541614c0 2024-04-25T10:12:46.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.497+0000 7fda49ffb640 1 -- 172.21.15.133:0/265241645 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda4c01a070 con 0x7fda541614c0 2024-04-25T10:12:46.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.497+0000 7fda49ffb640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda38066f60 0x7fda38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.497+0000 7fda49ffb640 1 -- 172.21.15.133:0/265241645 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda4c084770 con 0x7fda541614c0 2024-04-25T10:12:46.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.497+0000 7fda597f7640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda38066f60 0x7fda38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.497+0000 7fda597f7640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda38066f60 0x7fda38069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fda54168420 tx=0x7fda44009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.501+0000 7fda49ffb640 1 -- 172.21.15.133:0/265241645 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda4c054fa0 con 0x7fda541614c0 2024-04-25T10:12:46.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda5ba5b640 1 -- 172.21.15.133:0/265241645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda540b2750 con 0x7fda541614c0 2024-04-25T10:12:46.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda49ffb640 1 -- 172.21.15.133:0/265241645 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fda4c059040 con 0x7fda541614c0 2024-04-25T10:12:46.683 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:46.684 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:46.684 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda5ba5b640 1 -- 172.21.15.133:0/265241645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda38066f60 msgr2=0x7fda38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda5ba5b640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda38066f60 0x7fda38069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fda54168420 tx=0x7fda44009190 comp rx=0 tx=0).stop 2024-04-25T10:12:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda5ba5b640 1 -- 172.21.15.133:0/265241645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda541614c0 msgr2=0x7fda541563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda5ba5b640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda541614c0 0x7fda541563b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00c7a0 tx=0x7fda4c00cc60 comp rx=0 tx=0).stop 2024-04-25T10:12:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda5ba5b640 1 -- 172.21.15.133:0/265241645 shutdown_connections 2024-04-25T10:12:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda5ba5b640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fda38066f60 0x7fda38069420 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda5ba5b640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda541614c0 0x7fda541563b0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda5ba5b640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda5415db00 0x7fda540c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda5ba5b640 1 --2- 172.21.15.133:0/265241645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fda540d0770 0x7fda540c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.681+0000 7fda5ba5b640 1 -- 172.21.15.133:0/265241645 >> 172.21.15.133:0/265241645 conn(0x7fda540727c0 msgr2=0x7fda54072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.685+0000 7fda5ba5b640 1 -- 172.21.15.133:0/265241645 shutdown_connections 2024-04-25T10:12:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.685+0000 7fda5ba5b640 1 -- 172.21.15.133:0/265241645 wait complete. 2024-04-25T10:12:46.701 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:12:46.701 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:46.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff4258c3640 1 Processor -- start 2024-04-25T10:12:46.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff4258c3640 1 -- start start 2024-04-25T10:12:46.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff4258c3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff42015dd90 0x7ff4200ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff4258c3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff4200b19f0 0x7ff4200cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff4258c3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4200ca9b0 con 0x7ff42015d9b0 2024-04-25T10:12:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff4258c3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4200cbe30 con 0x7ff4200b19f0 2024-04-25T10:12:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff4258c3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200cac90 con 0x7ff42015dd90 2024-04-25T10:12:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff41f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff4200b19f0 0x7ff4200cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff41f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff4200b19f0 0x7ff4200cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56472/0 (socket says 172.21.15.133:56472) 2024-04-25T10:12:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff41f7fe640 1 -- 172.21.15.133:0/592886133 learned_addr learned my addr 172.21.15.133:0/592886133 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff41e7fc640 1 --2- 172.21.15.133:0/592886133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff42015dd90 0x7ff4200ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff41f7fe640 1 -- 172.21.15.133:0/592886133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff42015dd90 msgr2=0x7ff4200ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff41f7fe640 1 --2- 172.21.15.133:0/592886133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff42015dd90 0x7ff4200ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.817+0000 7ff41f7fe640 1 -- 172.21.15.133:0/592886133 >> v1:172.21.15.162:6789/0 conn(0x7ff42015d9b0 legacy=0x7ff4200cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff41f7fe640 1 -- 172.21.15.133:0/592886133 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200caf70 con 0x7ff4200b19f0 2024-04-25T10:12:46.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff41f7fe640 1 --2- 172.21.15.133:0/592886133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4200b19f0 0x7ff4200cecb0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7ff418009e50 tx=0x7ff41801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f022e06742ef26d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff41dffb640 1 -- 172.21.15.133:0/592886133 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4180a5880 con 0x7ff4200b19f0 2024-04-25T10:12:46.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff41dffb640 1 -- 172.21.15.133:0/592886133 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4180a5a20 con 0x7ff4200b19f0 2024-04-25T10:12:46.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff41dffb640 1 -- 172.21.15.133:0/592886133 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4180a5df0 con 0x7ff4200b19f0 2024-04-25T10:12:46.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff4258c3640 1 -- 172.21.15.133:0/592886133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4200b19f0 msgr2=0x7ff4200cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff4258c3640 1 --2- 172.21.15.133:0/592886133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4200b19f0 0x7ff4200cecb0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7ff418009e50 tx=0x7ff41801c9a0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff4258c3640 1 -- 172.21.15.133:0/592886133 shutdown_connections 2024-04-25T10:12:46.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff4258c3640 1 --2- 172.21.15.133:0/592886133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4200b19f0 0x7ff4200cecb0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff4258c3640 1 --2- 172.21.15.133:0/592886133 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff42015dd90 0x7ff4200ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff4258c3640 1 -- 172.21.15.133:0/592886133 >> 172.21.15.133:0/592886133 conn(0x7ff420067000 msgr2=0x7ff42006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff4258c3640 1 -- 172.21.15.133:0/592886133 shutdown_connections 2024-04-25T10:12:46.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.821+0000 7ff4258c3640 1 -- 172.21.15.133:0/592886133 wait complete. 2024-04-25T10:12:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff4258c3640 1 Processor -- start 2024-04-25T10:12:46.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff4258c3640 1 -- start start 2024-04-25T10:12:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff4258c3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4200b19f0 0x7ff42014a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff4258c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff42015d9b0 0x7ff420148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff4258c3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff42015dd90 0x7ff420148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff4258c3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42006e270 con 0x7ff42015d9b0 2024-04-25T10:12:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff4258c3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42006e0f0 con 0x7ff42015dd90 2024-04-25T10:12:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff4258c3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42006e3f0 con 0x7ff4200b19f0 2024-04-25T10:12:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff41e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff42015d9b0 0x7ff420148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff41e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff42015d9b0 0x7ff420148690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56486/0 (socket says 172.21.15.133:56486) 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff41e7fc640 1 -- 172.21.15.133:0/4282896511 learned_addr learned my addr 172.21.15.133:0/4282896511 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff41f7fe640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff42015dd90 0x7ff420148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff41effd640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4200b19f0 0x7ff42014a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff41e7fc640 1 -- 172.21.15.133:0/4282896511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4200b19f0 msgr2=0x7ff42014a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff41e7fc640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4200b19f0 0x7ff42014a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff41e7fc640 1 -- 172.21.15.133:0/4282896511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff42015dd90 msgr2=0x7ff420148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff41e7fc640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff42015dd90 0x7ff420148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff41e7fc640 1 -- 172.21.15.133:0/4282896511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff420144760 con 0x7ff42015d9b0 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff41e7fc640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff42015d9b0 0x7ff420148690 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7ff41400d5f0 tx=0x7ff41400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff407fff640 1 -- 172.21.15.133:0/4282896511 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4140125f0 con 0x7ff42015d9b0 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff407fff640 1 -- 172.21.15.133:0/4282896511 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff414019070 con 0x7ff42015d9b0 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff4258c3640 1 -- 172.21.15.133:0/4282896511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff420144a40 con 0x7ff42015d9b0 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff407fff640 1 -- 172.21.15.133:0/4282896511 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4140129b0 con 0x7ff42015d9b0 2024-04-25T10:12:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff4258c3640 1 -- 172.21.15.133:0/4282896511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff420145000 con 0x7ff42015d9b0 2024-04-25T10:12:46.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff4258c3640 1 -- 172.21.15.133:0/4282896511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff420068ef0 con 0x7ff42015d9b0 2024-04-25T10:12:46.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.825+0000 7ff407fff640 1 -- 172.21.15.133:0/4282896511 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff414012b50 con 0x7ff42015d9b0 2024-04-25T10:12:46.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.829+0000 7ff407fff640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff40c066f60 0x7ff40c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.829+0000 7ff407fff640 1 -- 172.21.15.133:0/4282896511 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4140855b0 con 0x7ff42015d9b0 2024-04-25T10:12:46.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.829+0000 7ff41effd640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff40c066f60 0x7ff40c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.829+0000 7ff41effd640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff40c066f60 0x7ff40c069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff410001650 tx=0x7ff410009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:46.833+0000 7ff407fff640 1 -- 172.21.15.133:0/4282896511 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff414011340 con 0x7ff42015d9b0 2024-04-25T10:12:47.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.041+0000 7ff4258c3640 1 -- 172.21.15.133:0/4282896511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff4200cc330 con 0x7ff42015d9b0 2024-04-25T10:12:47.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.041+0000 7ff407fff640 1 -- 172.21.15.133:0/4282896511 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7ff414056430 con 0x7ff42015d9b0 2024-04-25T10:12:47.043 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:47.043 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":679,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":120,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077879776,"bytes_used":2661990400,"bytes_avail":770432122880,"bytes_total":773094113280,"write_bytes_sec":69552336,"read_op_per_sec":0,"write_op_per_sec":70},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8919}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:12:45.917935+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:47.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.041+0000 7ff4258c3640 1 -- 172.21.15.133:0/4282896511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff40c066f60 msgr2=0x7ff40c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.041+0000 7ff4258c3640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff40c066f60 0x7ff40c069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff410001650 tx=0x7ff410009040 comp rx=0 tx=0).stop 2024-04-25T10:12:47.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.041+0000 7ff4258c3640 1 -- 172.21.15.133:0/4282896511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff42015d9b0 msgr2=0x7ff420148690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.041+0000 7ff4258c3640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff42015d9b0 0x7ff420148690 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7ff41400d5f0 tx=0x7ff41400dab0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.045+0000 7ff4258c3640 1 -- 172.21.15.133:0/4282896511 shutdown_connections 2024-04-25T10:12:47.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.045+0000 7ff4258c3640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff42015dd90 0x7ff420148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.045+0000 7ff4258c3640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff40c066f60 0x7ff40c069420 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.045+0000 7ff4258c3640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff42015d9b0 0x7ff420148690 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.045+0000 7ff4258c3640 1 --2- 172.21.15.133:0/4282896511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff4200b19f0 0x7ff42014a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.045+0000 7ff4258c3640 1 -- 172.21.15.133:0/4282896511 >> 172.21.15.133:0/4282896511 conn(0x7ff420067000 msgr2=0x7ff420152df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.045+0000 7ff4258c3640 1 -- 172.21.15.133:0/4282896511 shutdown_connections 2024-04-25T10:12:47.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.045+0000 7ff4258c3640 1 -- 172.21.15.133:0/4282896511 wait complete. 2024-04-25T10:12:47.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T10:12:47.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:47.061 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T10:12:47.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f1838bb3640 1 Processor -- start 2024-04-25T10:12:47.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f1838bb3640 1 -- start start 2024-04-25T10:12:47.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f1838bb3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f183415d9a0 0x7f18340d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f1838bb3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f18340ce600 0x7f18340cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f1838bb3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183415ebd0 con 0x7f18340d1920 2024-04-25T10:12:47.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f1838bb3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183415eeb0 con 0x7f183415d9a0 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f1838bb3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18340d0330 con 0x7f18340ce600 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f183259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f183415d9a0 0x7f18340d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f183259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f183415d9a0 0x7f18340d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56494/0 (socket says 172.21.15.133:56494) 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f183259c640 1 -- 172.21.15.133:0/1684219743 learned_addr learned my addr 172.21.15.133:0/1684219743 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f183259c640 1 -- 172.21.15.133:0/1684219743 >> v2:172.21.15.162:3300/0 conn(0x7f18340ce600 msgr2=0x7f18340cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f183259c640 1 --2- 172.21.15.133:0/1684219743 >> v2:172.21.15.162:3300/0 conn(0x7f18340ce600 0x7f18340cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f183259c640 1 -- 172.21.15.133:0/1684219743 >> v1:172.21.15.133:6789/0 conn(0x7f18340d1920 legacy=0x7f18340d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.173+0000 7f183259c640 1 -- 172.21.15.133:0/1684219743 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18340d0610 con 0x7f183415d9a0 2024-04-25T10:12:47.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f183259c640 1 --2- 172.21.15.133:0/1684219743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f183415d9a0 0x7f18340d0b30 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f18340ce080 tx=0x7f182401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee2a13a26044e37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f183159a640 1 -- 172.21.15.133:0/1684219743 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240b2070 con 0x7f183415d9a0 2024-04-25T10:12:47.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f183159a640 1 -- 172.21.15.133:0/1684219743 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18240a9070 con 0x7f183415d9a0 2024-04-25T10:12:47.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f183159a640 1 -- 172.21.15.133:0/1684219743 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240ad880 con 0x7f183415d9a0 2024-04-25T10:12:47.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f1838bb3640 1 -- 172.21.15.133:0/1684219743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f183415d9a0 msgr2=0x7f18340d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f1838bb3640 1 --2- 172.21.15.133:0/1684219743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f183415d9a0 0x7f18340d0b30 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f18340ce080 tx=0x7f182401c490 comp rx=0 tx=0).stop 2024-04-25T10:12:47.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f1838bb3640 1 -- 172.21.15.133:0/1684219743 shutdown_connections 2024-04-25T10:12:47.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f1838bb3640 1 --2- 172.21.15.133:0/1684219743 >> v2:172.21.15.162:3300/0 conn(0x7f18340ce600 0x7f18340cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f1838bb3640 1 --2- 172.21.15.133:0/1684219743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f183415d9a0 0x7f18340d0b30 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f1838bb3640 1 -- 172.21.15.133:0/1684219743 >> 172.21.15.133:0/1684219743 conn(0x7f183406bf80 msgr2=0x7f183406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f1838bb3640 1 -- 172.21.15.133:0/1684219743 shutdown_connections 2024-04-25T10:12:47.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f1838bb3640 1 -- 172.21.15.133:0/1684219743 wait complete. 2024-04-25T10:12:47.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.177+0000 7f1838bb3640 1 Processor -- start 2024-04-25T10:12:47.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1838bb3640 1 -- start start 2024-04-25T10:12:47.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1838bb3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18340ce600 0x7f183415c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1838bb3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18340d1920 0x7f1834160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1838bb3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415d9a0 0x7f1834160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1838bb3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340b9820 con 0x7f18340ce600 2024-04-25T10:12:47.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1838bb3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340b96a0 con 0x7f18340d1920 2024-04-25T10:12:47.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1838bb3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183415e590 con 0x7f183415d9a0 2024-04-25T10:12:47.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f183259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18340ce600 0x7f183415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f183259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18340ce600 0x7f183415c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56500/0 (socket says 172.21.15.133:56500) 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f183259c640 1 -- 172.21.15.133:0/2026737017 learned_addr learned my addr 172.21.15.133:0/2026737017 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f183259c640 1 -- 172.21.15.133:0/2026737017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415d9a0 msgr2=0x7f1834160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f183259c640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415d9a0 0x7f1834160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f183259c640 1 -- 172.21.15.133:0/2026737017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18340d1920 msgr2=0x7f1834160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f183259c640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18340d1920 0x7f1834160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f183259c640 1 -- 172.21.15.133:0/2026737017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18340c8370 con 0x7f18340ce600 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f183259c640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18340ce600 0x7f183415c400 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f18240b0040 tx=0x7f182401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1822ffd640 1 -- 172.21.15.133:0/2026737017 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240bc020 con 0x7f18340ce600 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1822ffd640 1 -- 172.21.15.133:0/2026737017 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18240b8420 con 0x7f18340ce600 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1838bb3640 1 -- 172.21.15.133:0/2026737017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18340c75b0 con 0x7f18340ce600 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1822ffd640 1 -- 172.21.15.133:0/2026737017 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240b2040 con 0x7f18340ce600 2024-04-25T10:12:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1838bb3640 1 -- 172.21.15.133:0/2026737017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18340c7b70 con 0x7f18340ce600 2024-04-25T10:12:47.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1822ffd640 1 -- 172.21.15.133:0/2026737017 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18240bf020 con 0x7f18340ce600 2024-04-25T10:12:47.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1822ffd640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1814066f60 0x7f1814069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1822ffd640 1 -- 172.21.15.133:0/2026737017 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f182411f4a0 con 0x7f18340ce600 2024-04-25T10:12:47.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1831d9b640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1814066f60 0x7f1814069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.181+0000 7f1831d9b640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1814066f60 0x7f1814069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f18340b8190 tx=0x7f1828006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 Processor -- start 2024-04-25T10:12:47.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- start start 2024-04-25T10:12:47.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5590073649d0 0x559007364db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073652f0 0x55900731ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55900731b0a0 0x55900731b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559007167ff0 con 0x55900731b0a0 2024-04-25T10:12:47.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590070f0150 con 0x5590073652f0 2024-04-25T10:12:47.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x559007164090 con 0x5590073649d0 2024-04-25T10:12:47.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1832d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55900731b0a0 0x55900731b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1832d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55900731b0a0 0x55900731b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56504/0 (socket says 172.21.15.133:56504) 2024-04-25T10:12:47.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1832d9d640 1 -- 172.21.15.133:0/3852819068 learned_addr learned my addr 172.21.15.133:0/3852819068 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183259c640 1 --2- 172.21.15.133:0/3852819068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5590073649d0 0x559007364db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1831d9b640 1 --2- 172.21.15.133:0/3852819068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073652f0 0x55900731ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1832d9d640 1 -- 172.21.15.133:0/3852819068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5590073649d0 msgr2=0x559007364db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1832d9d640 1 --2- 172.21.15.133:0/3852819068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5590073649d0 0x559007364db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1832d9d640 1 -- 172.21.15.133:0/3852819068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073652f0 msgr2=0x55900731ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1832d9d640 1 --2- 172.21.15.133:0/3852819068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073652f0 0x55900731ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1832d9d640 1 -- 172.21.15.133:0/3852819068 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590072fe0d0 con 0x55900731b0a0 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1832d9d640 1 --2- 172.21.15.133:0/3852819068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55900731b0a0 0x55900731b480 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f182c001cc0 tx=0x7f182c002180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13961df17af3108 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1813fff640 1 -- 172.21.15.133:0/3852819068 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c017070 con 0x55900731b0a0 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1813fff640 1 -- 172.21.15.133:0/3852819068 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c00cc30 con 0x55900731b0a0 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1813fff640 1 -- 172.21.15.133:0/3852819068 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c003be0 con 0x55900731b0a0 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- 172.21.15.133:0/3852819068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55900731b0a0 msgr2=0x55900731b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 --2- 172.21.15.133:0/3852819068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55900731b0a0 0x55900731b480 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f182c001cc0 tx=0x7f182c002180 comp rx=0 tx=0).stop 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- 172.21.15.133:0/3852819068 shutdown_connections 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 --2- 172.21.15.133:0/3852819068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55900731b0a0 0x55900731b480 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 --2- 172.21.15.133:0/3852819068 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073652f0 0x55900731ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 --2- 172.21.15.133:0/3852819068 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5590073649d0 0x559007364db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- 172.21.15.133:0/3852819068 >> 172.21.15.133:0/3852819068 conn(0x559007317020 msgr2=0x559007319440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- 172.21.15.133:0/3852819068 shutdown_connections 2024-04-25T10:12:47.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- 172.21.15.133:0/3852819068 wait complete. 2024-04-25T10:12:47.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- start start 2024-04-25T10:12:47.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 Processor -- start 2024-04-25T10:12:47.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55900731b0a0 0x5590073497f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x559007349d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073652f0 0x55900734a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900712c250 con 0x5590073649d0 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559007122690 con 0x5590073652f0 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590070f0a40 con 0x55900731b0a0 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f183bd20000 10 client.0 _ll_register_callbacks cb 0x559007305e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1831d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x559007349d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1831d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x559007349d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56508/0 (socket says 172.21.15.133:56508) 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.185+0000 7f1831d9b640 1 -- 172.21.15.133:0/2031670679 learned_addr learned my addr 172.21.15.133:0/2031670679 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183259c640 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55900731b0a0 0x5590073497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f1831d9b640 1 -- 172.21.15.133:0/2031670679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55900731b0a0 msgr2=0x5590073497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f1832d9d640 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073652f0 0x55900734a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f1831d9b640 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55900731b0a0 0x5590073497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f1831d9b640 1 -- 172.21.15.133:0/2031670679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073652f0 msgr2=0x55900734a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f1831d9b640 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073652f0 0x55900734a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f1831d9b640 1 -- 172.21.15.133:0/2031670679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590072151c0 con 0x5590073649d0 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183259c640 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55900731b0a0 0x5590073497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f1831d9b640 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x559007349d30 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f182800bee0 tx=0x7f18280047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3219a4e39acacaf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f18117fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f18117fa640 1 -- 172.21.15.133:0/2031670679 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828023060 con 0x5590073649d0 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f18117fa640 1 -- 172.21.15.133:0/2031670679 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f18280127c0 con 0x5590073649d0 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 -- 172.21.15.133:0/2031670679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f182c00cc30 con 0x5590073649d0 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f18117fa640 1 -- 172.21.15.133:0/2031670679 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1828029050 con 0x5590073649d0 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 10 client.9219 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 -- 172.21.15.133:0/2031670679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x559007211170 con 0x5590073649d0 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f18117fa640 1 -- 172.21.15.133:0/2031670679 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c00cc30 con 0x5590073649d0 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f18117fa640 1 -- 172.21.15.133:0/2031670679 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f182802c020 con 0x5590073649d0 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 10 client.9219 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:47.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 10 client.9219 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 20 client.9219 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x559007374c80 0x559007377070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 4 client.9219 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 -- 172.21.15.133:0/2031670679 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559007103c60 con 0x559007374c80 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183259c640 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x559007374c80 0x559007377070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f18117fa640 10 client.9219 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183259c640 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x559007374c80 0x559007377070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e715b368e734cd8 server_cookie=950fb1419e4025e0 in_seq=0 out_seq=0 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f18117fa640 1 -- 172.21.15.133:0/2031670679 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559007103c60 con 0x559007374c80 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f18117fa640 10 client.9219 handle_command_reply: tid=0 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 client.9219 shutdown 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 -- 172.21.15.133:0/2031670679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 msgr2=0x559007349d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x559007349d30 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f182800bee0 tx=0x7f18280047e0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 -- 172.21.15.133:0/2031670679 shutdown_connections 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x559007374c80 0x559007377070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073652f0 0x55900734a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x559007349d30 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f18117fa640 0 client.9219 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:47.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 --2- 172.21.15.133:0/2031670679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55900731b0a0 0x5590073497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 -- 172.21.15.133:0/2031670679 >> 172.21.15.133:0/2031670679 conn(0x559007317020 msgr2=0x559007319ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 -- 172.21.15.133:0/2031670679 shutdown_connections 2024-04-25T10:12:47.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 1 -- 172.21.15.133:0/2031670679 wait complete. 2024-04-25T10:12:47.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.189+0000 7f183bd20000 20 client.9219 trim_cache size 0 max 16384 2024-04-25T10:12:47.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 Processor -- start 2024-04-25T10:12:47.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- start start 2024-04-25T10:12:47.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55900731b0a0 0x55900731a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073649d0 0x559007371e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073652f0 0x559007372380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834203030 con 0x5590073652f0 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18342021f0 con 0x5590073649d0 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18341f7170 con 0x55900731b0a0 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1832d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073652f0 0x559007372380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1832d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073652f0 0x559007372380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56522/0 (socket says 172.21.15.133:56522) 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1832d9d640 1 -- 172.21.15.133:0/1430046077 learned_addr learned my addr 172.21.15.133:0/1430046077 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1832d9d640 1 -- 172.21.15.133:0/1430046077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55900731b0a0 msgr2=0x55900731a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1832d9d640 1 --2- 172.21.15.133:0/1430046077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55900731b0a0 0x55900731a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1832d9d640 1 -- 172.21.15.133:0/1430046077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073649d0 msgr2=0x559007371e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1832d9d640 1 --2- 172.21.15.133:0/1430046077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073649d0 0x559007371e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1832d9d640 1 -- 172.21.15.133:0/1430046077 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590071aede0 con 0x5590073652f0 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1832d9d640 1 --2- 172.21.15.133:0/1430046077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073652f0 0x559007372380 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f182c005040 tx=0x7f182c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d388da752d5c6e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f18137fe640 1 -- 172.21.15.133:0/1430046077 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c017030 con 0x5590073652f0 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f18137fe640 1 -- 172.21.15.133:0/1430046077 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c023070 con 0x5590073652f0 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f18137fe640 1 -- 172.21.15.133:0/1430046077 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00ca30 con 0x5590073652f0 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- 172.21.15.133:0/1430046077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073652f0 msgr2=0x559007372380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 --2- 172.21.15.133:0/1430046077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073652f0 0x559007372380 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f182c005040 tx=0x7f182c011000 comp rx=0 tx=0).stop 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- 172.21.15.133:0/1430046077 shutdown_connections 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 --2- 172.21.15.133:0/1430046077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073652f0 0x559007372380 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 --2- 172.21.15.133:0/1430046077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5590073649d0 0x559007371e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 --2- 172.21.15.133:0/1430046077 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55900731b0a0 0x55900731a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- 172.21.15.133:0/1430046077 >> 172.21.15.133:0/1430046077 conn(0x559007317020 msgr2=0x559007371730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- 172.21.15.133:0/1430046077 shutdown_connections 2024-04-25T10:12:47.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- 172.21.15.133:0/1430046077 wait complete. 2024-04-25T10:12:47.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- start start 2024-04-25T10:12:47.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 Processor -- start 2024-04-25T10:12:47.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55900731b0a0 0x5590073c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x5590073c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5590073652f0 0x559007376ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559007102e00 con 0x5590073649d0 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834201d90 con 0x55900731b0a0 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x559007167300 con 0x5590073652f0 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183bd20000 10 client.0 _ll_register_callbacks cb 0x5590072e6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1831d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x5590073c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1831d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x5590073c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56534/0 (socket says 172.21.15.133:56534) 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1831d9b640 1 -- 172.21.15.133:0/1022397724 learned_addr learned my addr 172.21.15.133:0/1022397724 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1832d9d640 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5590073652f0 0x559007376ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f183259c640 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55900731b0a0 0x5590073c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1831d9b640 1 -- 172.21.15.133:0/1022397724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5590073652f0 msgr2=0x559007376ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.205+0000 7f1831d9b640 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5590073652f0 0x559007376ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1831d9b640 1 -- 172.21.15.133:0/1022397724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55900731b0a0 msgr2=0x5590073c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1831d9b640 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55900731b0a0 0x5590073c4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1831d9b640 1 -- 172.21.15.133:0/1022397724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590071098e0 con 0x5590073649d0 2024-04-25T10:12:47.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1831d9b640 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x5590073c4e70 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f18280094a0 tx=0x7f1828010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c6b266a6bcb73c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f18117fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:47.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f18117fa640 1 -- 172.21.15.133:0/1022397724 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828013c80 con 0x5590073649d0 2024-04-25T10:12:47.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f18117fa640 1 -- 172.21.15.133:0/1022397724 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1828035410 con 0x5590073649d0 2024-04-25T10:12:47.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 -- 172.21.15.133:0/1022397724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f182c023070 con 0x5590073649d0 2024-04-25T10:12:47.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f18117fa640 1 -- 172.21.15.133:0/1022397724 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f182803c050 con 0x5590073649d0 2024-04-25T10:12:47.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 10 client.9225 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:47.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 -- 172.21.15.133:0/1022397724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5590071c18f0 con 0x5590073649d0 2024-04-25T10:12:47.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f18117fa640 1 -- 172.21.15.133:0/1022397724 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c023070 con 0x5590073649d0 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f18117fa640 1 -- 172.21.15.133:0/1022397724 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f182803f020 con 0x5590073649d0 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 10 client.9225 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 10 client.9225 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 20 client.9225 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x559007377620 0x559007377a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 4 client.9225 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 -- 172.21.15.133:0/1022397724 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x559007175530 con 0x559007377620 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183259c640 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x559007377620 0x559007377a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f18117fa640 10 client.9225 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183259c640 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x559007377620 0x559007377a30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd05eaf2aaf31253 server_cookie=fb4f93e55db304c2 in_seq=0 out_seq=0 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f18117fa640 1 -- 172.21.15.133:0/1022397724 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x559007175530 con 0x559007377620 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f18117fa640 10 client.9225 handle_command_reply: tid=0 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 client.9225 shutdown 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 -- 172.21.15.133:0/1022397724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 msgr2=0x5590073c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x5590073c4e70 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f18280094a0 tx=0x7f1828010000 comp rx=0 tx=0).stop 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 -- 172.21.15.133:0/1022397724 shutdown_connections 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x559007377620 0x559007377a30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5590073652f0 0x559007376ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f18117fa640 0 client.9225 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590073649d0 0x5590073c4e70 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 --2- 172.21.15.133:0/1022397724 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55900731b0a0 0x5590073c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 -- 172.21.15.133:0/1022397724 >> 172.21.15.133:0/1022397724 conn(0x559007317020 msgr2=0x559007371fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 -- 172.21.15.133:0/1022397724 shutdown_connections 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 1 -- 172.21.15.133:0/1022397724 wait complete. 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f183bd20000 20 client.9225 trim_cache size 0 max 16384 2024-04-25T10:12:47.217 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:12:47.218 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:12:47.218 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:12:47.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 -- 172.21.15.133:0/2026737017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1814066f60 msgr2=0x7f1814069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1814066f60 0x7f1814069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f18340b8190 tx=0x7f1828006ca0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 -- 172.21.15.133:0/2026737017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18340ce600 msgr2=0x7f183415c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18340ce600 0x7f183415c400 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f18240b0040 tx=0x7f182401acd0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 -- 172.21.15.133:0/2026737017 shutdown_connections 2024-04-25T10:12:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f183415d9a0 0x7f1834160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18340d1920 0x7f1834160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1814066f60 0x7f1814069420 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 --2- 172.21.15.133:0/2026737017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18340ce600 0x7f183415c400 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 -- 172.21.15.133:0/2026737017 >> 172.21.15.133:0/2026737017 conn(0x7f183406bf80 msgr2=0x7f18340b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 -- 172.21.15.133:0/2026737017 shutdown_connections 2024-04-25T10:12:47.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.209+0000 7f1838bb3640 1 -- 172.21.15.133:0/2026737017 wait complete. 2024-04-25T10:12:47.226 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:12:47.226 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.337+0000 7f03c085e640 1 Processor -- start 2024-04-25T10:12:47.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.337+0000 7f03c085e640 1 -- start start 2024-04-25T10:12:47.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 0x7f03bc0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03bc15ced0 0x7f03bc15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc0ce2e0 con 0x7f03bc145170 2024-04-25T10:12:47.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03bc06db00 con 0x7f03bc15ced0 2024-04-25T10:12:47.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc145550 con 0x7f03bc0cd820 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03bad9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03bc15ced0 0x7f03bc15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03ba59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f03bc145170 0x7f03bc06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:56568/0 (socket says 172.21.15.133:56568) 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03bad9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03bc15ced0 0x7f03bc15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56542/0 (socket says 172.21.15.133:56542) 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03ba59c640 1 -- 172.21.15.133:0/422421005 learned_addr learned my addr 172.21.15.133:0/422421005 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03bad9d640 1 -- 172.21.15.133:0/422421005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 msgr2=0x7f03bc0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03b1d9b640 1 --2- 172.21.15.133:0/422421005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 0x7f03bc0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03bad9d640 1 --2- 172.21.15.133:0/422421005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 0x7f03bc0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03bad9d640 1 -- 172.21.15.133:0/422421005 >> v1:172.21.15.133:6789/0 conn(0x7f03bc145170 legacy=0x7f03bc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03bad9d640 1 -- 172.21.15.133:0/422421005 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc06d4b0 con 0x7f03bc15ced0 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03b9d9b640 1 -- 172.21.15.133:0/422421005 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37757004 0 0) 0x7f03bc0ce2e0 con 0x7f03bc145170 2024-04-25T10:12:47.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03b1d9b640 1 --2- 172.21.15.133:0/422421005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 0x7f03bc0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:47.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03bad9d640 1 --2- 172.21.15.133:0/422421005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03bc15ced0 0x7f03bc15d360 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009d90 tx=0x7f03b401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a6715a8e1ebd94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03b9d9b640 1 -- 172.21.15.133:0/422421005 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40b2070 con 0x7f03bc15ced0 2024-04-25T10:12:47.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03b9d9b640 1 -- 172.21.15.133:0/422421005 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b40a5830 con 0x7f03bc15ced0 2024-04-25T10:12:47.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03b9d9b640 1 -- 172.21.15.133:0/422421005 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40a5c00 con 0x7f03bc15ced0 2024-04-25T10:12:47.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 -- 172.21.15.133:0/422421005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03bc15ced0 msgr2=0x7f03bc15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 --2- 172.21.15.133:0/422421005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03bc15ced0 0x7f03bc15d360 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009d90 tx=0x7f03b401c970 comp rx=0 tx=0).stop 2024-04-25T10:12:47.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 -- 172.21.15.133:0/422421005 shutdown_connections 2024-04-25T10:12:47.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 --2- 172.21.15.133:0/422421005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03bc15ced0 0x7f03bc15d360 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 --2- 172.21.15.133:0/422421005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 0x7f03bc0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 -- 172.21.15.133:0/422421005 >> 172.21.15.133:0/422421005 conn(0x7f03bc066f70 msgr2=0x7f03bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 -- 172.21.15.133:0/422421005 shutdown_connections 2024-04-25T10:12:47.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.341+0000 7f03c085e640 1 -- 172.21.15.133:0/422421005 wait complete. 2024-04-25T10:12:47.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03c085e640 1 Processor -- start 2024-04-25T10:12:47.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03c085e640 1 -- start start 2024-04-25T10:12:47.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03c085e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 0x7f03bc164a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03c085e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03bc145170 0x7f03bc13dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03c085e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03bc15ced0 0x7f03bc13e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03c085e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc168610 con 0x7f03bc145170 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03c085e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc168490 con 0x7f03bc15ced0 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03c085e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc168790 con 0x7f03bc0cd820 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03bc145170 0x7f03bc13dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03bc145170 0x7f03bc13dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56546/0 (socket says 172.21.15.133:56546) 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b1d9b640 1 -- 172.21.15.133:0/3517064923 learned_addr learned my addr 172.21.15.133:0/3517064923 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03bad9d640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03bc15ced0 0x7f03bc13e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03ba59c640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 0x7f03bc164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b1d9b640 1 -- 172.21.15.133:0/3517064923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 msgr2=0x7f03bc164a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b1d9b640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 0x7f03bc164a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b1d9b640 1 -- 172.21.15.133:0/3517064923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03bc15ced0 msgr2=0x7f03bc13e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b1d9b640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03bc15ced0 0x7f03bc13e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b1d9b640 1 -- 172.21.15.133:0/3517064923 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc0abf30 con 0x7f03bc145170 2024-04-25T10:12:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03bad9d640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03bc15ced0 0x7f03bc13e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03ba59c640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 0x7f03bc164a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b1d9b640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03bc145170 0x7f03bc13dca0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f03a400c7c0 tx=0x7f03a400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b37fe640 1 -- 172.21.15.133:0/3517064923 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4007a30 con 0x7f03bc145170 2024-04-25T10:12:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b37fe640 1 -- 172.21.15.133:0/3517064923 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a4007bd0 con 0x7f03bc145170 2024-04-25T10:12:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03c085e640 1 -- 172.21.15.133:0/3517064923 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc0a8a60 con 0x7f03bc145170 2024-04-25T10:12:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03b37fe640 1 -- 172.21.15.133:0/3517064923 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a40113f0 con 0x7f03bc145170 2024-04-25T10:12:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.345+0000 7f03c085e640 1 -- 172.21.15.133:0/3517064923 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc0a9020 con 0x7f03bc145170 2024-04-25T10:12:47.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03b37fe640 1 -- 172.21.15.133:0/3517064923 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03a4011590 con 0x7f03bc145170 2024-04-25T10:12:47.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03b37fe640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f039c066f90 0x7f039c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03b37fe640 1 -- 172.21.15.133:0/3517064923 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03a40852f0 con 0x7f03bc145170 2024-04-25T10:12:47.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03ba59c640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f039c066f90 0x7f039c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03ba59c640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f039c066f90 0x7f039c069450 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f03ac000f70 tx=0x7f03ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 Processor -- start 2024-04-25T10:12:47.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 -- start start 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a120e9d0 0x5634a120edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120f2f0 0x5634a11c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a11c50a0 0x5634a11c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a1011ff0 con 0x5634a120e9d0 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a0f9a150 con 0x5634a11c50a0 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634a100e090 con 0x5634a120f2f0 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03ba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a120e9d0 0x5634a120edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03ba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a120e9d0 0x5634a120edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56554/0 (socket says 172.21.15.133:56554) 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03ba59c640 1 -- 172.21.15.133:0/3057154615 learned_addr learned my addr 172.21.15.133:0/3057154615 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03ba59c640 1 -- 172.21.15.133:0/3057154615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120f2f0 msgr2=0x5634a11c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03ba59c640 1 --2- 172.21.15.133:0/3057154615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120f2f0 0x5634a11c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03ba59c640 1 -- 172.21.15.133:0/3057154615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a11c50a0 msgr2=0x5634a11c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03ba59c640 1 --2- 172.21.15.133:0/3057154615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a11c50a0 0x5634a11c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03ba59c640 1 -- 172.21.15.133:0/3057154615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634a11a80d0 con 0x5634a120e9d0 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03ba59c640 1 --2- 172.21.15.133:0/3057154615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a120e9d0 0x5634a120edb0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f03ac007360 tx=0x7f03ac010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe86e6da9b2982e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f039bfff640 1 -- 172.21.15.133:0/3057154615 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00db80 con 0x5634a120e9d0 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f039bfff640 1 -- 172.21.15.133:0/3057154615 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac00dd20 con 0x5634a120e9d0 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f039bfff640 1 -- 172.21.15.133:0/3057154615 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac020650 con 0x5634a120e9d0 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 -- 172.21.15.133:0/3057154615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a120e9d0 msgr2=0x5634a120edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 --2- 172.21.15.133:0/3057154615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a120e9d0 0x5634a120edb0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f03ac007360 tx=0x7f03ac010000 comp rx=0 tx=0).stop 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 -- 172.21.15.133:0/3057154615 shutdown_connections 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 --2- 172.21.15.133:0/3057154615 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a11c50a0 0x5634a11c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 --2- 172.21.15.133:0/3057154615 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120f2f0 0x5634a11c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 --2- 172.21.15.133:0/3057154615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a120e9d0 0x5634a120edb0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 -- 172.21.15.133:0/3057154615 >> 172.21.15.133:0/3057154615 conn(0x5634a11c1020 msgr2=0x5634a11c3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 -- 172.21.15.133:0/3057154615 shutdown_connections 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 -- 172.21.15.133:0/3057154615 wait complete. 2024-04-25T10:12:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 -- start start 2024-04-25T10:12:47.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.349+0000 7f03c39cb000 1 Processor -- start 2024-04-25T10:12:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a11f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120e9d0 0x5634a11f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120f2f0 0x5634a11f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a0fd6250 con 0x5634a11c50a0 2024-04-25T10:12:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a0fcc690 con 0x5634a120f2f0 2024-04-25T10:12:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634a0f9aa40 con 0x5634a120e9d0 2024-04-25T10:12:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 10 client.0 _ll_register_callbacks cb 0x5634a11afe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03ba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a11f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03ba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a11f37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56560/0 (socket says 172.21.15.133:56560) 2024-04-25T10:12:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03ba59c640 1 -- 172.21.15.133:0/3682879007 learned_addr learned my addr 172.21.15.133:0/3682879007 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03ba59c640 1 -- 172.21.15.133:0/3682879007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120e9d0 msgr2=0x5634a11f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03ba59c640 1 --2- 172.21.15.133:0/3682879007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120e9d0 0x5634a11f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03ba59c640 1 -- 172.21.15.133:0/3682879007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120f2f0 msgr2=0x5634a11f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03ba59c640 1 --2- 172.21.15.133:0/3682879007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120f2f0 0x5634a11f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03ba59c640 1 -- 172.21.15.133:0/3682879007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634a10bf1c0 con 0x5634a11c50a0 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03ba59c640 1 --2- 172.21.15.133:0/3682879007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a11f37f0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009010 tx=0x7f03ac007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf33ece4773ecc94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03997fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03997fa640 1 -- 172.21.15.133:0/3682879007 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac020b90 con 0x5634a11c50a0 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03997fa640 1 -- 172.21.15.133:0/3682879007 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03ac020d30 con 0x5634a11c50a0 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 -- 172.21.15.133:0/3682879007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03ac00dd20 con 0x5634a11c50a0 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03997fa640 1 -- 172.21.15.133:0/3682879007 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03ac034050 con 0x5634a11c50a0 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 10 client.9240 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 -- 172.21.15.133:0/3682879007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5634a10bb170 con 0x5634a11c50a0 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03997fa640 1 -- 172.21.15.133:0/3682879007 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac00dd20 con 0x5634a11c50a0 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03997fa640 1 -- 172.21.15.133:0/3682879007 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f03ac037020 con 0x5634a11c50a0 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 10 client.9240 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 10 client.9240 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:47.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 20 client.9240 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 --2- 172.21.15.133:0/3682879007 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5634a121ec80 0x5634a1221070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 4 client.9240 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 -- 172.21.15.133:0/3682879007 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634a0fadc60 con 0x5634a121ec80 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03b1d9b640 1 --2- 172.21.15.133:0/3682879007 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5634a121ec80 0x5634a1221070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03997fa640 10 client.9240 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03b1d9b640 1 --2- 172.21.15.133:0/3682879007 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5634a121ec80 0x5634a1221070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27dbce3680102b70 server_cookie=58b7944495f14ae6 in_seq=0 out_seq=0 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03997fa640 1 -- 172.21.15.133:0/3682879007 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634a0fadc60 con 0x5634a121ec80 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03997fa640 10 client.9240 handle_command_reply: tid=0 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 client.9240 shutdown 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 -- 172.21.15.133:0/3682879007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 msgr2=0x5634a11f37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 --2- 172.21.15.133:0/3682879007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a11f37f0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009010 tx=0x7f03ac007000 comp rx=0 tx=0).stop 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 -- 172.21.15.133:0/3682879007 shutdown_connections 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 --2- 172.21.15.133:0/3682879007 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5634a121ec80 0x5634a1221070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 --2- 172.21.15.133:0/3682879007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120f2f0 0x5634a11f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 --2- 172.21.15.133:0/3682879007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120e9d0 0x5634a11f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03997fa640 0 client.9240 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:47.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 --2- 172.21.15.133:0/3682879007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a11f37f0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 -- 172.21.15.133:0/3682879007 >> 172.21.15.133:0/3682879007 conn(0x5634a11c1020 msgr2=0x5634a11c3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 -- 172.21.15.133:0/3682879007 shutdown_connections 2024-04-25T10:12:47.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 1 -- 172.21.15.133:0/3682879007 wait complete. 2024-04-25T10:12:47.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.353+0000 7f03c39cb000 20 client.9240 trim_cache size 0 max 16384 2024-04-25T10:12:47.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03c39cb000 1 Processor -- start 2024-04-25T10:12:47.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03c39cb000 1 -- start start 2024-04-25T10:12:47.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a120de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120e9d0 0x5634a11c4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120f2f0 0x5634a11c4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03c39cb000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc203210 con 0x5634a11c50a0 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03c39cb000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc202410 con 0x5634a120f2f0 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03c39cb000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0aa8a0 con 0x5634a120e9d0 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03ba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a120de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03ba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a120de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56570/0 (socket says 172.21.15.133:56570) 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03ba59c640 1 -- 172.21.15.133:0/1432667408 learned_addr learned my addr 172.21.15.133:0/1432667408 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03ba59c640 1 -- 172.21.15.133:0/1432667408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120e9d0 msgr2=0x5634a11c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03ba59c640 1 --2- 172.21.15.133:0/1432667408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120e9d0 0x5634a11c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03ba59c640 1 -- 172.21.15.133:0/1432667408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120f2f0 msgr2=0x5634a11c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03ba59c640 1 --2- 172.21.15.133:0/1432667408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120f2f0 0x5634a11c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03ba59c640 1 -- 172.21.15.133:0/1432667408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634a1058de0 con 0x5634a11c50a0 2024-04-25T10:12:47.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03ba59c640 1 --2- 172.21.15.133:0/1432667408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a120de10 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00d520 tx=0x7f03ac011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2761d194732973 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f039b7fe640 1 -- 172.21.15.133:0/1432667408 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac007b50 con 0x5634a11c50a0 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f039b7fe640 1 -- 172.21.15.133:0/1432667408 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac020990 con 0x5634a11c50a0 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f039b7fe640 1 -- 172.21.15.133:0/1432667408 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac020d60 con 0x5634a11c50a0 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03c39cb000 1 -- 172.21.15.133:0/1432667408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 msgr2=0x5634a120de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.369+0000 7f03c39cb000 1 --2- 172.21.15.133:0/1432667408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a120de10 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00d520 tx=0x7f03ac011000 comp rx=0 tx=0).stop 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1432667408 shutdown_connections 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- 172.21.15.133:0/1432667408 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120f2f0 0x5634a11c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- 172.21.15.133:0/1432667408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120e9d0 0x5634a11c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- 172.21.15.133:0/1432667408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a120de10 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1432667408 >> 172.21.15.133:0/1432667408 conn(0x5634a11c1020 msgr2=0x5634a120d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1432667408 shutdown_connections 2024-04-25T10:12:47.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1432667408 wait complete. 2024-04-25T10:12:47.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- start start 2024-04-25T10:12:47.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 Processor -- start 2024-04-25T10:12:47.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a126f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120e9d0 0x5634a126fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120f2f0 0x5634a1221380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a0face00 con 0x5634a11c50a0 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc201e40 con 0x5634a120e9d0 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634a1011300 con 0x5634a120f2f0 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 10 client.0 _ll_register_callbacks cb 0x5634a1190040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03ba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a126f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03ba59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a126f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56576/0 (socket says 172.21.15.133:56576) 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03ba59c640 1 -- 172.21.15.133:0/1807512535 learned_addr learned my addr 172.21.15.133:0/1807512535 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03ba59c640 1 -- 172.21.15.133:0/1807512535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120f2f0 msgr2=0x5634a1221380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03ba59c640 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120f2f0 0x5634a1221380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03ba59c640 1 -- 172.21.15.133:0/1807512535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120e9d0 msgr2=0x5634a126fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03b1d9b640 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120e9d0 0x5634a126fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03ba59c640 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120e9d0 0x5634a126fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03ba59c640 1 -- 172.21.15.133:0/1807512535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634a0fb38e0 con 0x5634a11c50a0 2024-04-25T10:12:47.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03ba59c640 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a126f520 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00d520 tx=0x7f03ac000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72ebdc55ea1b4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03997fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03997fa640 1 -- 172.21.15.133:0/1807512535 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac020d40 con 0x5634a11c50a0 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03997fa640 1 -- 172.21.15.133:0/1807512535 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03ac02c520 con 0x5634a11c50a0 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1807512535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03ac020990 con 0x5634a11c50a0 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03997fa640 1 -- 172.21.15.133:0/1807512535 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03ac049050 con 0x5634a11c50a0 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 10 client.9246 fetch_fsmap learned FSMap version 72 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1807512535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x5634a106b8f0 con 0x5634a11c50a0 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03997fa640 1 -- 172.21.15.133:0/1807512535 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac020990 con 0x5634a11c50a0 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03997fa640 1 -- 172.21.15.133:0/1807512535 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f03ac04c020 con 0x5634a11c50a0 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 10 client.9246 fetch_fsmap finished waiting for FSMap version 72 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 10 client.9246 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 20 client.9246 populate_metadata read hostname 'smithi133' 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5634a1221d70 0x5634a1222180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 4 client.9246 mds_command: new command op to 8919 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:47.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1807512535 --> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5634a101f530 con 0x5634a1221d70 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03b1d9b640 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5634a1221d70 0x5634a1222180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03997fa640 10 client.9246 ms_handle_connect on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03b1d9b640 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5634a1221d70 0x5634a1222180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa9a3c255d94339 server_cookie=3caa8b394e8b6ca6 in_seq=0 out_seq=0 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03997fa640 1 -- 172.21.15.133:0/1807512535 <== mds.0 v2:172.21.15.133:6835/4218771669 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5634a101f530 con 0x5634a1221d70 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03997fa640 10 client.9246 handle_command_reply: tid=0 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 client.9246 shutdown 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1807512535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 msgr2=0x5634a126f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a126f520 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00d520 tx=0x7f03ac000ee0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1807512535 shutdown_connections 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.133:6835/4218771669,v1:172.21.15.133:6837/4218771669] conn(0x5634a1221d70 0x5634a1222180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5634a120f2f0 0x5634a1221380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03997fa640 0 client.9246 ms_handle_reset on v2:172.21.15.133:6835/4218771669 2024-04-25T10:12:47.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5634a120e9d0 0x5634a126fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 --2- 172.21.15.133:0/1807512535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5634a11c50a0 0x5634a126f520 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1807512535 >> 172.21.15.133:0/1807512535 conn(0x5634a11c1020 msgr2=0x5634a120d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1807512535 shutdown_connections 2024-04-25T10:12:47.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 1 -- 172.21.15.133:0/1807512535 wait complete. 2024-04-25T10:12:47.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.373+0000 7f03c39cb000 20 client.9246 trim_cache size 0 max 16384 2024-04-25T10:12:47.383 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:12:47.383 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-04-25T10:12:47.383 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:12:47.383 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:12:47.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 -- 172.21.15.133:0/3517064923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f039c066f90 msgr2=0x7f039c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f039c066f90 0x7f039c069450 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f03ac000f70 tx=0x7f03ac009040 comp rx=0 tx=0).stop 2024-04-25T10:12:47.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 -- 172.21.15.133:0/3517064923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03bc145170 msgr2=0x7f03bc13dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03bc145170 0x7f03bc13dca0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f03a400c7c0 tx=0x7f03a400cc80 comp rx=0 tx=0).stop 2024-04-25T10:12:47.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 -- 172.21.15.133:0/3517064923 shutdown_connections 2024-04-25T10:12:47.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f03bc15ced0 0x7f03bc13e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f039c066f90 0x7f039c069450 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03bc145170 0x7f03bc13dca0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 --2- 172.21.15.133:0/3517064923 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f03bc0cd820 0x7f03bc164a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 -- 172.21.15.133:0/3517064923 >> 172.21.15.133:0/3517064923 conn(0x7f03bc066f70 msgr2=0x7f03bc0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 -- 172.21.15.133:0/3517064923 shutdown_connections 2024-04-25T10:12:47.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.377+0000 7f03c085e640 1 -- 172.21.15.133:0/3517064923 wait complete. 2024-04-25T10:12:47.392 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:12:47.392 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.501+0000 7f4dd31e6640 1 Processor -- start 2024-04-25T10:12:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.501+0000 7f4dd31e6640 1 -- start start 2024-04-25T10:12:47.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dd31e6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4dcc0b1f70 0x7f4dcc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dd31e6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dcc0c61d0 con 0x7f4dcc0ce330 2024-04-25T10:12:47.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dd31e6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dcc0c64b0 con 0x7f4dcc15c9a0 2024-04-25T10:12:47.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dd31e6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dcc06d4f0 con 0x7f4dcc0b1f70 2024-04-25T10:12:47.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dd1783640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4dcc0ce330 0x7f4dcc15d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:56578/0 (socket says 172.21.15.133:56578) 2024-04-25T10:12:47.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dd1783640 1 -- 172.21.15.133:0/3003150814 learned_addr learned my addr 172.21.15.133:0/3003150814 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:47.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcbfff640 1 --2- 172.21.15.133:0/3003150814 >> v2:172.21.15.162:3300/0 conn(0x7f4dcc0b1f70 0x7f4dcc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343224134 0 0) 0x7f4dcc0c61d0 con 0x7f4dcc0ce330 2024-04-25T10:12:47.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4db8003530 con 0x7f4dcc0ce330 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2063210046 0 0) 0x7f4db8003530 con 0x7f4dcc0ce330 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 >> v2:172.21.15.162:3300/0 conn(0x7f4dcc0b1f70 msgr2=0x7f4dcc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 --2- 172.21.15.133:0/3003150814 >> v2:172.21.15.162:3300/0 conn(0x7f4dcc0b1f70 0x7f4dcc06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 >> v1:172.21.15.162:6789/0 conn(0x7f4dcc15c9a0 legacy=0x7f4dcc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc06d7d0 con 0x7f4dcc0ce330 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4dc4001a00 con 0x7f4dcc0ce330 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 >> v1:172.21.15.133:6789/0 conn(0x7f4dcc0ce330 legacy=0x7f4dcc15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4db80039c0 0x7f4db8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4db8004810 0x7f4db8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db80051d0 0x7f4db800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc06d4f0 con 0x7f4db8004810 2024-04-25T10:12:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db800c070 con 0x7f4db80051d0 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db800c270 con 0x7f4db80039c0 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcbfff640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4db8004810 0x7f4db8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dd1783640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db80051d0 0x7f4db800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dd0f82640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4db80039c0 0x7f4db8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcbfff640 1 -- 172.21.15.133:0/3003150814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4db80039c0 msgr2=0x7f4db8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcbfff640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4db80039c0 0x7f4db8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcbfff640 1 -- 172.21.15.133:0/3003150814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db80051d0 msgr2=0x7f4db800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcbfff640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db80051d0 0x7f4db800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dcbfff640 1 -- 172.21.15.133:0/3003150814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8003530 con 0x7f4db8004810 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dd1783640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4db80051d0 0x7f4db800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:47.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.505+0000 7f4dd1783640 1 -- 172.21.15.133:0/3003150814 reap_dead start 2024-04-25T10:12:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dcbfff640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4db8004810 0x7f4db8004c90 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009ae0 tx=0x7f4dbc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd80505c1039b1e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc0b2070 con 0x7f4db8004810 2024-04-25T10:12:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc0a5660 con 0x7f4db8004810 2024-04-25T10:12:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dcb7fe640 1 -- 172.21.15.133:0/3003150814 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc0096b0 con 0x7f4db8004810 2024-04-25T10:12:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 -- 172.21.15.133:0/3003150814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4db8004810 msgr2=0x7f4db8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4db8004810 0x7f4db8004c90 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009ae0 tx=0x7f4dbc01ca70 comp rx=0 tx=0).stop 2024-04-25T10:12:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 -- 172.21.15.133:0/3003150814 shutdown_connections 2024-04-25T10:12:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 --2- 172.21.15.133:0/3003150814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4db8004810 0x7f4db8004c90 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 -- 172.21.15.133:0/3003150814 >> 172.21.15.133:0/3003150814 conn(0x7f4dcc067000 msgr2=0x7f4dcc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 -- 172.21.15.133:0/3003150814 shutdown_connections 2024-04-25T10:12:47.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 -- 172.21.15.133:0/3003150814 wait complete. 2024-04-25T10:12:47.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 Processor -- start 2024-04-25T10:12:47.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 -- start start 2024-04-25T10:12:47.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4db8004810 0x7f4dcc141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dcc0c7f30 0x7f4dcc0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dcc0cb010 0x7f4dcc06e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc0ce780 con 0x7f4dcc0cb010 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc0ce900 con 0x7f4dcc0c7f30 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd31e6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc0cea80 con 0x7f4db8004810 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd1783640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dcc0cb010 0x7f4dcc06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd1783640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dcc0cb010 0x7f4dcc06e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56602/0 (socket says 172.21.15.133:56602) 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd1783640 1 -- 172.21.15.133:0/1839734397 learned_addr learned my addr 172.21.15.133:0/1839734397 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dcbfff640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dcc0c7f30 0x7f4dcc0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd0f82640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4db8004810 0x7f4dcc141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd1783640 1 -- 172.21.15.133:0/1839734397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4db8004810 msgr2=0x7f4dcc141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd1783640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4db8004810 0x7f4dcc141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd1783640 1 -- 172.21.15.133:0/1839734397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dcc0c7f30 msgr2=0x7f4dcc0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd1783640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dcc0c7f30 0x7f4dcc0caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.509+0000 7f4dd1783640 1 -- 172.21.15.133:0/1839734397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc0c8390 con 0x7f4dcc0cb010 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dd1783640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dcc0cb010 0x7f4dcc06e390 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc06f4d0 tx=0x7f4dc40118b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dc97fa640 1 -- 172.21.15.133:0/1839734397 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc401c070 con 0x7f4dcc0cb010 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dc97fa640 1 -- 172.21.15.133:0/1839734397 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc40137a0 con 0x7f4dcc0cb010 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dd31e6640 1 -- 172.21.15.133:0/1839734397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dcc151f70 con 0x7f4dcc0cb010 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dc97fa640 1 -- 172.21.15.133:0/1839734397 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4010040 con 0x7f4dcc0cb010 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dd31e6640 1 -- 172.21.15.133:0/1839734397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dcc152530 con 0x7f4dcc0cb010 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dd31e6640 1 -- 172.21.15.133:0/1839734397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dcc068ef0 con 0x7f4dcc0cb010 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dc97fa640 1 -- 172.21.15.133:0/1839734397 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4dc401f070 con 0x7f4dcc0cb010 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dc97fa640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4dac066f60 0x7f4dac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dc97fa640 1 -- 172.21.15.133:0/1839734397 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4dc4087800 con 0x7f4dcc0cb010 2024-04-25T10:12:47.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dd0f82640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4dac066f60 0x7f4dac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.513+0000 7f4dd0f82640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4dac066f60 0x7f4dac069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4db4000c10 tx=0x7f4db4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.517+0000 7f4dc97fa640 1 -- 172.21.15.133:0/1839734397 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4dc4058740 con 0x7f4dcc0cb010 2024-04-25T10:12:47.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.697+0000 7f4dd31e6640 1 -- 172.21.15.133:0/1839734397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dcc0b1c00 con 0x7f4dcc0cb010 2024-04-25T10:12:47.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dc97fa640 1 -- 172.21.15.133:0/1839734397 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f4dc4058040 con 0x7f4dcc0cb010 2024-04-25T10:12:47.703 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:47.703 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:47.703 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 -- 172.21.15.133:0/1839734397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4dac066f60 msgr2=0x7f4dac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4dac066f60 0x7f4dac069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4db4000c10 tx=0x7f4db4009040 comp rx=0 tx=0).stop 2024-04-25T10:12:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 -- 172.21.15.133:0/1839734397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dcc0cb010 msgr2=0x7f4dcc06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dcc0cb010 0x7f4dcc06e390 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc06f4d0 tx=0x7f4dc40118b0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 -- 172.21.15.133:0/1839734397 shutdown_connections 2024-04-25T10:12:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4dac066f60 0x7f4dac069420 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dcc0cb010 0x7f4dcc06e390 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4dcc0c7f30 0x7f4dcc0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 --2- 172.21.15.133:0/1839734397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4db8004810 0x7f4dcc141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 -- 172.21.15.133:0/1839734397 >> 172.21.15.133:0/1839734397 conn(0x7f4dcc067000 msgr2=0x7f4dcc0c4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 -- 172.21.15.133:0/1839734397 shutdown_connections 2024-04-25T10:12:47.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.701+0000 7f4dd31e6640 1 -- 172.21.15.133:0/1839734397 wait complete. 2024-04-25T10:12:47.720 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:12:47.720 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:47.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.829+0000 7fa9ab851640 1 Processor -- start 2024-04-25T10:12:47.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.829+0000 7fa9ab851640 1 -- start start 2024-04-25T10:12:47.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9ab851640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a4158110 0x7fa9a40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9ab851640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa9a41584f0 0x7fa9a415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9ab851640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a40cbe60 con 0x7fa9a406e460 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9ab851640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a40b19b0 con 0x7fa9a41584f0 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9ab851640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a415b360 con 0x7fa9a4158110 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a9dee640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa9a406e460 0x7fa9a415e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:56588/0 (socket says 172.21.15.133:56588) 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a9dee640 1 -- 172.21.15.133:0/1939399729 learned_addr learned my addr 172.21.15.133:0/1939399729 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a8dec640 1 --2- 172.21.15.133:0/1939399729 >> v2:172.21.15.162:3300/0 conn(0x7fa9a41584f0 0x7fa9a415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a95ed640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a4158110 0x7fa9a40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138928517 0 0) 0x7fa9a40cbe60 con 0x7fa9a406e460 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa990003530 con 0x7fa9a406e460 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2509891314 0 0) 0x7fa990003530 con 0x7fa9a406e460 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a4158110 msgr2=0x7fa9a40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a4158110 0x7fa9a40b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 >> v2:172.21.15.162:3300/0 conn(0x7fa9a41584f0 msgr2=0x7fa9a415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 --2- 172.21.15.133:0/1939399729 >> v2:172.21.15.162:3300/0 conn(0x7fa9a41584f0 0x7fa9a415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a415b640 con 0x7fa9a406e460 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa99c001a00 con 0x7fa9a406e460 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 >> v1:172.21.15.133:6789/0 conn(0x7fa9a406e460 legacy=0x7fa9a415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9900039c0 0x7fa990003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a4158110 0x7fa990004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa990004d50 0x7fa9900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40b19b0 con 0x7fa990004d50 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a415b360 con 0x7fa9900039c0 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99000b9d0 con 0x7fa9a4158110 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a9dee640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa990004d50 0x7fa9900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a95ed640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9900039c0 0x7fa990003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a8dec640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a4158110 0x7fa990004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a9dee640 1 -- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a4158110 msgr2=0x7fa990004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a9dee640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a4158110 0x7fa990004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a9dee640 1 -- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9900039c0 msgr2=0x7fa990003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a9dee640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9900039c0 0x7fa990003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a9dee640 1 -- 172.21.15.133:0/1939399729 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa990003530 con 0x7fa990004d50 2024-04-25T10:12:47.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a95ed640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9900039c0 0x7fa990003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:47.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa9a9dee640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa990004d50 0x7fa9900051d0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00a870 tx=0x7fa99c01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5762b728c83755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c00a4c0 con 0x7fa990004d50 2024-04-25T10:12:47.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c0072d0 con 0x7fa990004d50 2024-04-25T10:12:47.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.833+0000 7fa99bfff640 1 -- 172.21.15.133:0/1939399729 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c0076c0 con 0x7fa990004d50 2024-04-25T10:12:47.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- 172.21.15.133:0/1939399729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa990004d50 msgr2=0x7fa9900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 --2- 172.21.15.133:0/1939399729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa990004d50 0x7fa9900051d0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00a870 tx=0x7fa99c01d3a0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9a9dee640 1 -- 172.21.15.133:0/1939399729 reap_dead start 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- 172.21.15.133:0/1939399729 shutdown_connections 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- 172.21.15.133:0/1939399729 >> 172.21.15.133:0/1939399729 conn(0x7fa9a4067000 msgr2=0x7fa9a406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- 172.21.15.133:0/1939399729 shutdown_connections 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- 172.21.15.133:0/1939399729 wait complete. 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 Processor -- start 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- start start 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a415d030 0x7fa9a40c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a40ccd60 0x7fa9a40b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9a415d410 0x7fa9a40b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40cd670 con 0x7fa9a415d410 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a415bc00 con 0x7fa9a40ccd60 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40cd7f0 con 0x7fa9a415d030 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9a9dee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9a415d410 0x7fa9a40b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9a9dee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9a415d410 0x7fa9a40b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56630/0 (socket says 172.21.15.133:56630) 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9a9dee640 1 -- 172.21.15.133:0/3928153822 learned_addr learned my addr 172.21.15.133:0/3928153822 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9a9dee640 1 -- 172.21.15.133:0/3928153822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a415d030 msgr2=0x7fa9a40c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9a9dee640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a415d030 0x7fa9a40c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9a9dee640 1 -- 172.21.15.133:0/3928153822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a40ccd60 msgr2=0x7fa9a40b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9a9dee640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a40ccd60 0x7fa9a40b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9a9dee640 1 -- 172.21.15.133:0/3928153822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a40b17e0 con 0x7fa9a415d410 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9a9dee640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9a415d410 0x7fa9a40b2fc0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fa99c01ce10 tx=0x7fa990005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa999ffb640 1 -- 172.21.15.133:0/3928153822 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c00a4c0 con 0x7fa9a415d410 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa999ffb640 1 -- 172.21.15.133:0/3928153822 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c0bb070 con 0x7fa9a415d410 2024-04-25T10:12:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- 172.21.15.133:0/3928153822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a40b1cc0 con 0x7fa9a415d410 2024-04-25T10:12:47.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa999ffb640 1 -- 172.21.15.133:0/3928153822 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c002040 con 0x7fa9a415d410 2024-04-25T10:12:47.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- 172.21.15.133:0/3928153822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a40b2280 con 0x7fa9a415d410 2024-04-25T10:12:47.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.837+0000 7fa9ab851640 1 -- 172.21.15.133:0/3928153822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a406bc20 con 0x7fa9a415d410 2024-04-25T10:12:47.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.841+0000 7fa999ffb640 1 -- 172.21.15.133:0/3928153822 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa99c0072d0 con 0x7fa9a415d410 2024-04-25T10:12:47.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.841+0000 7fa999ffb640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa988066f90 0x7fa988069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.841+0000 7fa999ffb640 1 -- 172.21.15.133:0/3928153822 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa99c11f630 con 0x7fa9a415d410 2024-04-25T10:12:47.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.841+0000 7fa9a95ed640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa988066f90 0x7fa988069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.841+0000 7fa9a95ed640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa988066f90 0x7fa988069450 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40cc600 tx=0x7fa9a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:47.845+0000 7fa999ffb640 1 -- 172.21.15.133:0/3928153822 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa99c0170b0 con 0x7fa9a415d410 2024-04-25T10:12:48.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.045+0000 7fa9ab851640 1 -- 172.21.15.133:0/3928153822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa9a4068ef0 con 0x7fa9a415d410 2024-04-25T10:12:48.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.045+0000 7fa999ffb640 1 -- 172.21.15.133:0/3928153822 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fa99c0f04a0 con 0x7fa9a415d410 2024-04-25T10:12:48.048 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:48.048 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":680,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":120,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077879776,"bytes_used":2661990400,"bytes_avail":770432122880,"bytes_total":773094113280,"write_bytes_sec":42303056,"read_op_per_sec":0,"write_op_per_sec":42},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8919}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:12:45.917935+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:48.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 -- 172.21.15.133:0/3928153822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa988066f90 msgr2=0x7fa988069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa988066f90 0x7fa988069450 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40cc600 tx=0x7fa9a0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:48.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 -- 172.21.15.133:0/3928153822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9a415d410 msgr2=0x7fa9a40b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9a415d410 0x7fa9a40b2fc0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fa99c01ce10 tx=0x7fa990005840 comp rx=0 tx=0).stop 2024-04-25T10:12:48.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 -- 172.21.15.133:0/3928153822 shutdown_connections 2024-04-25T10:12:48.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa988066f90 0x7fa988069450 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9a415d410 0x7fa9a40b2fc0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9a40ccd60 0x7fa9a40b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 --2- 172.21.15.133:0/3928153822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa9a415d030 0x7fa9a40c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 -- 172.21.15.133:0/3928153822 >> 172.21.15.133:0/3928153822 conn(0x7fa9a4067000 msgr2=0x7fa9a40ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 -- 172.21.15.133:0/3928153822 shutdown_connections 2024-04-25T10:12:48.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:48.049+0000 7fa9ab851640 1 -- 172.21.15.133:0/3928153822 wait complete. 2024-04-25T10:12:48.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T10:12:49.065 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:49.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5064795640 1 Processor -- start 2024-04-25T10:12:49.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5064795640 1 -- start start 2024-04-25T10:12:49.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5064795640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f505c158110 0x7f505c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5064795640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f505c1584f0 0x7f505c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5064795640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505c06e460 0x7f505c06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5064795640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c06ee10 con 0x7f505c1584f0 2024-04-25T10:12:49.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5064795640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c0cbe60 con 0x7f505c158110 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5064795640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0b19b0 con 0x7f505c06e460 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5061d30640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f505c1584f0 0x7f505c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5061d30640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f505c1584f0 0x7f505c15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56642/0 (socket says 172.21.15.133:56642) 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5061d30640 1 -- 172.21.15.133:0/2642248042 learned_addr learned my addr 172.21.15.133:0/2642248042 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5062531640 1 --2- 172.21.15.133:0/2642248042 >> v2:172.21.15.162:3300/0 conn(0x7f505c158110 0x7f505c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5062d32640 1 --2- 172.21.15.133:0/2642248042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505c06e460 0x7f505c06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5061d30640 1 -- 172.21.15.133:0/2642248042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505c06e460 msgr2=0x7f505c06e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5061d30640 1 --2- 172.21.15.133:0/2642248042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505c06e460 0x7f505c06e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5061d30640 1 -- 172.21.15.133:0/2642248042 >> v2:172.21.15.162:3300/0 conn(0x7f505c158110 msgr2=0x7f505c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5061d30640 1 --2- 172.21.15.133:0/2642248042 >> v2:172.21.15.162:3300/0 conn(0x7f505c158110 0x7f505c0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.193+0000 7f5061d30640 1 -- 172.21.15.133:0/2642248042 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c15b360 con 0x7f505c1584f0 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5061d30640 1 --2- 172.21.15.133:0/2642248042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505c1584f0 0x7f505c15ad10 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f505c0b4d40 tx=0x7f505801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3f49b54d0703d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f506152f640 1 -- 172.21.15.133:0/2642248042 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580aa610 con 0x7f505c1584f0 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f506152f640 1 -- 172.21.15.133:0/2642248042 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50580af070 con 0x7f505c1584f0 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f506152f640 1 -- 172.21.15.133:0/2642248042 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580aa9d0 con 0x7f505c1584f0 2024-04-25T10:12:49.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5064795640 1 -- 172.21.15.133:0/2642248042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505c1584f0 msgr2=0x7f505c15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5064795640 1 --2- 172.21.15.133:0/2642248042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505c1584f0 0x7f505c15ad10 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f505c0b4d40 tx=0x7f505801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5064795640 1 -- 172.21.15.133:0/2642248042 shutdown_connections 2024-04-25T10:12:49.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5064795640 1 --2- 172.21.15.133:0/2642248042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505c06e460 0x7f505c06e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5064795640 1 --2- 172.21.15.133:0/2642248042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505c1584f0 0x7f505c15ad10 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5064795640 1 --2- 172.21.15.133:0/2642248042 >> v2:172.21.15.162:3300/0 conn(0x7f505c158110 0x7f505c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5064795640 1 -- 172.21.15.133:0/2642248042 >> 172.21.15.133:0/2642248042 conn(0x7f505c067000 msgr2=0x7f505c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5064795640 1 -- 172.21.15.133:0/2642248042 shutdown_connections 2024-04-25T10:12:49.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5064795640 1 -- 172.21.15.133:0/2642248042 wait complete. 2024-04-25T10:12:49.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5064795640 1 Processor -- start 2024-04-25T10:12:49.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.197+0000 7f5064795640 1 -- start start 2024-04-25T10:12:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5064795640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f505c06e460 0x7f505c15c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5064795640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505c158110 0x7f505c1446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5064795640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505c1584f0 0x7f505c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5064795640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0cd7f0 con 0x7f505c158110 2024-04-25T10:12:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5064795640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0cd670 con 0x7f505c06e460 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5064795640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c15eb10 con 0x7f505c1584f0 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5061d30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505c158110 0x7f505c1446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5061d30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505c158110 0x7f505c1446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56652/0 (socket says 172.21.15.133:56652) 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5061d30640 1 -- 172.21.15.133:0/1204681964 learned_addr learned my addr 172.21.15.133:0/1204681964 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5062531640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f505c06e460 0x7f505c15c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5061d30640 1 -- 172.21.15.133:0/1204681964 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505c1584f0 msgr2=0x7f505c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5061d30640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505c1584f0 0x7f505c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5061d30640 1 -- 172.21.15.133:0/1204681964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f505c06e460 msgr2=0x7f505c15c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5061d30640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f505c06e460 0x7f505c15c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5061d30640 1 -- 172.21.15.133:0/1204681964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c145050 con 0x7f505c158110 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5062531640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f505c06e460 0x7f505c15c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5061d30640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505c158110 0x7f505c1446e0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f50580ac040 tx=0x7f505801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5052ffd640 1 -- 172.21.15.133:0/1204681964 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580bd5f0 con 0x7f505c158110 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5052ffd640 1 -- 172.21.15.133:0/1204681964 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50580bd7b0 con 0x7f505c158110 2024-04-25T10:12:49.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5064795640 1 -- 172.21.15.133:0/1204681964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c145330 con 0x7f505c158110 2024-04-25T10:12:49.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5052ffd640 1 -- 172.21.15.133:0/1204681964 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580bdba0 con 0x7f505c158110 2024-04-25T10:12:49.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5064795640 1 -- 172.21.15.133:0/1204681964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c156b50 con 0x7f505c158110 2024-04-25T10:12:49.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5064795640 1 -- 172.21.15.133:0/1204681964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c06bc20 con 0x7f505c158110 2024-04-25T10:12:49.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5052ffd640 1 -- 172.21.15.133:0/1204681964 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50580b33f0 con 0x7f505c158110 2024-04-25T10:12:49.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5052ffd640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5040066f60 0x7f5040069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5052ffd640 1 -- 172.21.15.133:0/1204681964 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f505811ff00 con 0x7f505c158110 2024-04-25T10:12:49.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.201+0000 7f5062531640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5040066f60 0x7f5040069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.205+0000 7f5062531640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5040066f60 0x7f5040069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f504c009840 tx=0x7f504c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.205+0000 7f5052ffd640 1 -- 172.21.15.133:0/1204681964 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50580c3060 con 0x7f505c158110 2024-04-25T10:12:49.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.397+0000 7f5064795640 1 -- 172.21.15.133:0/1204681964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f505c068ef0 con 0x7f505c158110 2024-04-25T10:12:49.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5052ffd640 1 -- 172.21.15.133:0/1204681964 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f50580a7070 con 0x7f505c158110 2024-04-25T10:12:49.402 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:49.402 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:49.402 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 -- 172.21.15.133:0/1204681964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5040066f60 msgr2=0x7f5040069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5040066f60 0x7f5040069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f504c009840 tx=0x7f504c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 -- 172.21.15.133:0/1204681964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505c158110 msgr2=0x7f505c1446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505c158110 0x7f505c1446e0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f50580ac040 tx=0x7f505801ad30 comp rx=0 tx=0).stop 2024-04-25T10:12:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 -- 172.21.15.133:0/1204681964 shutdown_connections 2024-04-25T10:12:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f505c1584f0 0x7f505c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5040066f60 0x7f5040069420 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f505c158110 0x7f505c1446e0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 --2- 172.21.15.133:0/1204681964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f505c06e460 0x7f505c15c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 -- 172.21.15.133:0/1204681964 >> 172.21.15.133:0/1204681964 conn(0x7f505c067000 msgr2=0x7f505c159190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 -- 172.21.15.133:0/1204681964 shutdown_connections 2024-04-25T10:12:49.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.401+0000 7f5064795640 1 -- 172.21.15.133:0/1204681964 wait complete. 2024-04-25T10:12:49.420 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:12:49.421 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:49.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.537+0000 7f99eafa7640 1 Processor -- start 2024-04-25T10:12:49.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.537+0000 7f99eafa7640 1 -- start start 2024-04-25T10:12:49.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.537+0000 7f99eafa7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e40ce330 0x7f99e40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.537+0000 7f99eafa7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e40c61d0 con 0x7f99e40b1f70 2024-04-25T10:12:49.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.537+0000 7f99eafa7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e40c64b0 con 0x7f99e415c9a0 2024-04-25T10:12:49.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.537+0000 7f99eafa7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e40cecc0 con 0x7f99e40ce330 2024-04-25T10:12:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.537+0000 7f99e3fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f99e40b1f70 0x7f99e406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:56596/0 (socket says 172.21.15.133:56596) 2024-04-25T10:12:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.537+0000 7f99e3fff640 1 -- 172.21.15.133:0/3432669475 learned_addr learned my addr 172.21.15.133:0/3432669475 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.537+0000 7f99e9544640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e40ce330 0x7f99e40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.537+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292042675 0 0) 0x7f99e40c61d0 con 0x7f99e40b1f70 2024-04-25T10:12:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.537+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99cc003530 con 0x7f99e40b1f70 2024-04-25T10:12:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464178184 0 0) 0x7f99cc003530 con 0x7f99e40b1f70 2024-04-25T10:12:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e40ce330 msgr2=0x7f99e40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e40ce330 0x7f99e40ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 >> v1:172.21.15.162:6789/0 conn(0x7f99e415c9a0 legacy=0x7f99e40c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e406d4f0 con 0x7f99e40b1f70 2024-04-25T10:12:49.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f99d8001680 con 0x7f99e40b1f70 2024-04-25T10:12:49.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 >> v1:172.21.15.133:6789/0 conn(0x7f99e40b1f70 legacy=0x7f99e406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e40ce330 0x7f99cc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99cc003f70 0x7f99cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc004d50 0x7f99cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e40cecc0 con 0x7f99cc004d50 2024-04-25T10:12:49.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc00bab0 con 0x7f99cc003f70 2024-04-25T10:12:49.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc00bcb0 con 0x7f99e40ce330 2024-04-25T10:12:49.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e9544640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc004d50 0x7f99cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e3fff640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99cc003f70 0x7f99cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e8d43640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e40ce330 0x7f99cc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e9544640 1 -- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e40ce330 msgr2=0x7f99cc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e9544640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e40ce330 0x7f99cc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e9544640 1 -- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99cc003f70 msgr2=0x7f99cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e9544640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99cc003f70 0x7f99cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e9544640 1 -- 172.21.15.133:0/3432669475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc003530 con 0x7f99cc004d50 2024-04-25T10:12:49.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e8d43640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e40ce330 0x7f99cc0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:49.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e9544640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc004d50 0x7f99cc0051d0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009fc0 tx=0x7f99dc01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482a67da8f69f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc0b2070 con 0x7f99cc004d50 2024-04-25T10:12:49.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc0a5c90 con 0x7f99cc004d50 2024-04-25T10:12:49.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e37fe640 1 -- 172.21.15.133:0/3432669475 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc009c10 con 0x7f99cc004d50 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99eafa7640 1 -- 172.21.15.133:0/3432669475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc004d50 msgr2=0x7f99cc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99eafa7640 1 --2- 172.21.15.133:0/3432669475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99cc004d50 0x7f99cc0051d0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009fc0 tx=0x7f99dc01cc80 comp rx=0 tx=0).stop 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99e9544640 1 -- 172.21.15.133:0/3432669475 reap_dead start 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99eafa7640 1 -- 172.21.15.133:0/3432669475 shutdown_connections 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99eafa7640 1 -- 172.21.15.133:0/3432669475 >> 172.21.15.133:0/3432669475 conn(0x7f99e4067000 msgr2=0x7f99e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99eafa7640 1 -- 172.21.15.133:0/3432669475 shutdown_connections 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.541+0000 7f99eafa7640 1 -- 172.21.15.133:0/3432669475 wait complete. 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99eafa7640 1 Processor -- start 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99eafa7640 1 -- start start 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99eafa7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e4148110 0x7f99e41e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99eafa7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99e41e8490 0x7f99e415de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99eafa7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e415e380 0x7f99e40c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99eafa7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e415cf00 con 0x7f99e41e8490 2024-04-25T10:12:49.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99eafa7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e415cd80 con 0x7f99e415e380 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99eafa7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e40ce780 con 0x7f99e4148110 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99e41e8490 0x7f99e415de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99e41e8490 0x7f99e415de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56674/0 (socket says 172.21.15.133:56674) 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e3fff640 1 -- 172.21.15.133:0/3439384007 learned_addr learned my addr 172.21.15.133:0/3439384007 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e8d43640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e4148110 0x7f99e41e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e9544640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e415e380 0x7f99e40c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e3fff640 1 -- 172.21.15.133:0/3439384007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e4148110 msgr2=0x7f99e41e7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e3fff640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e4148110 0x7f99e41e7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e3fff640 1 -- 172.21.15.133:0/3439384007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e415e380 msgr2=0x7f99e40c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e3fff640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e415e380 0x7f99e40c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e3fff640 1 -- 172.21.15.133:0/3439384007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e41e8930 con 0x7f99e41e8490 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e3fff640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99e41e8490 0x7f99e415de40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f99d8001060 tx=0x7f99d800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e17fa640 1 -- 172.21.15.133:0/3439384007 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d800c8d0 con 0x7f99e41e8490 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e17fa640 1 -- 172.21.15.133:0/3439384007 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d8018070 con 0x7f99e41e8490 2024-04-25T10:12:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99eafa7640 1 -- 172.21.15.133:0/3439384007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e41e8c10 con 0x7f99e41e8490 2024-04-25T10:12:49.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99e17fa640 1 -- 172.21.15.133:0/3439384007 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d800cc90 con 0x7f99e41e8490 2024-04-25T10:12:49.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99eafa7640 1 -- 172.21.15.133:0/3439384007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e41e4460 con 0x7f99e41e8490 2024-04-25T10:12:49.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.545+0000 7f99eafa7640 1 -- 172.21.15.133:0/3439384007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e41e4660 con 0x7f99e41e8490 2024-04-25T10:12:49.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.549+0000 7f99e17fa640 1 -- 172.21.15.133:0/3439384007 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99d801d460 con 0x7f99e41e8490 2024-04-25T10:12:49.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.549+0000 7f99e17fa640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99d0066f90 0x7f99d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.549+0000 7f99e8d43640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99d0066f90 0x7f99d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.549+0000 7f99e17fa640 1 -- 172.21.15.133:0/3439384007 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99d8085c40 con 0x7f99e41e8490 2024-04-25T10:12:49.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.549+0000 7f99e8d43640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99d0066f90 0x7f99d0069450 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f99e41f4d60 tx=0x7f99d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.553+0000 7f99e17fa640 1 -- 172.21.15.133:0/3439384007 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99e41e4660 con 0x7f99e41e8490 2024-04-25T10:12:49.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.773+0000 7f99eafa7640 1 -- 172.21.15.133:0/3439384007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f99e4068ef0 con 0x7f99e41e8490 2024-04-25T10:12:49.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.773+0000 7f99e17fa640 1 -- 172.21.15.133:0/3439384007 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f99d8056ba0 con 0x7f99e41e8490 2024-04-25T10:12:49.777 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:49.777 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":682,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":120,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077879776,"bytes_used":2661990400,"bytes_avail":770432122880,"bytes_total":773094113280,"write_bytes_sec":42303056,"read_op_per_sec":0,"write_op_per_sec":42},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8919}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:12:45.917935+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 -- 172.21.15.133:0/3439384007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99d0066f90 msgr2=0x7f99d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99d0066f90 0x7f99d0069450 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f99e41f4d60 tx=0x7f99d4009040 comp rx=0 tx=0).stop 2024-04-25T10:12:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 -- 172.21.15.133:0/3439384007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99e41e8490 msgr2=0x7f99e415de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99e41e8490 0x7f99e415de40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f99d8001060 tx=0x7f99d800ed30 comp rx=0 tx=0).stop 2024-04-25T10:12:49.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 -- 172.21.15.133:0/3439384007 shutdown_connections 2024-04-25T10:12:49.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99e415e380 0x7f99e40c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f99d0066f90 0x7f99d0069450 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99e41e8490 0x7f99e415de40 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 --2- 172.21.15.133:0/3439384007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f99e4148110 0x7f99e41e7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 -- 172.21.15.133:0/3439384007 >> 172.21.15.133:0/3439384007 conn(0x7f99e4067000 msgr2=0x7f99e40b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 -- 172.21.15.133:0/3439384007 shutdown_connections 2024-04-25T10:12:49.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:49.777+0000 7f99eafa7640 1 -- 172.21.15.133:0/3439384007 wait complete. 2024-04-25T10:12:49.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T10:12:50.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.925+0000 7f1cc29fd640 1 Processor -- start 2024-04-25T10:12:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.925+0000 7f1cc29fd640 1 -- start start 2024-04-25T10:12:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc29fd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1cbc15c9a0 0x7f1cbc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc29fd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cbc0b1f70 0x7f1cbc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc29fd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cbc0c61d0 con 0x7f1cbc0ce330 2024-04-25T10:12:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc29fd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cbc0c64b0 con 0x7f1cbc15c9a0 2024-04-25T10:12:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc29fd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc06d4f0 con 0x7f1cbc0b1f70 2024-04-25T10:12:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc0f9a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1cbc0ce330 0x7f1cbc15d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:56602/0 (socket says 172.21.15.133:56602) 2024-04-25T10:12:50.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc0f9a640 1 -- 172.21.15.133:0/1689096751 learned_addr learned my addr 172.21.15.133:0/1689096751 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbbfff640 1 --2- 172.21.15.133:0/1689096751 >> v2:172.21.15.162:3300/0 conn(0x7f1cbc15c9a0 0x7f1cbc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cb37fe640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cbc0b1f70 0x7f1cbc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076414383 0 0) 0x7f1cbc0c61d0 con 0x7f1cbc0ce330 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ca8003530 con 0x7f1cbc0ce330 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2142157765 0 0) 0x7f1ca8003530 con 0x7f1cbc0ce330 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cbc0b1f70 msgr2=0x7f1cbc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cbc0b1f70 0x7f1cbc06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 >> v2:172.21.15.162:3300/0 conn(0x7f1cbc15c9a0 msgr2=0x7f1cbc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 --2- 172.21.15.133:0/1689096751 >> v2:172.21.15.162:3300/0 conn(0x7f1cbc15c9a0 0x7f1cbc0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc06d7d0 con 0x7f1cbc0ce330 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1cb4001a00 con 0x7f1cbc0ce330 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 >> v1:172.21.15.133:6789/0 conn(0x7f1cbc0ce330 legacy=0x7f1cbc15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cbc0b1f70 0x7f1ca80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ca8003f40 0x7f1ca8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca8004d50 0x7f1ca80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0c64b0 con 0x7f1ca8004d50 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc06d4f0 con 0x7f1ca8003f40 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca800b9d0 con 0x7f1cbc0b1f70 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc0f9a640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca8004d50 0x7f1ca80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cb37fe640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ca8003f40 0x7f1ca8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cbbfff640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cbc0b1f70 0x7f1ca80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc0f9a640 1 -- 172.21.15.133:0/1689096751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cbc0b1f70 msgr2=0x7f1ca80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc0f9a640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cbc0b1f70 0x7f1ca80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc0f9a640 1 -- 172.21.15.133:0/1689096751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ca8003f40 msgr2=0x7f1ca8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc0f9a640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ca8003f40 0x7f1ca8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.929+0000 7f1cc0f9a640 1 -- 172.21.15.133:0/1689096751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8003530 con 0x7f1ca8004d50 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc0f9a640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca8004d50 0x7f1ca80051d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400a870 tx=0x7f1cb401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ce8e913eabc0c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400a4c0 con 0x7f1ca8004d50 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb40072d0 con 0x7f1ca8004d50 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cbb7fe640 1 -- 172.21.15.133:0/1689096751 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40076c0 con 0x7f1ca8004d50 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 -- 172.21.15.133:0/1689096751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca8004d50 msgr2=0x7f1ca80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 --2- 172.21.15.133:0/1689096751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca8004d50 0x7f1ca80051d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400a870 tx=0x7f1cb401d340 comp rx=0 tx=0).stop 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc0f9a640 1 -- 172.21.15.133:0/1689096751 reap_dead start 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 -- 172.21.15.133:0/1689096751 shutdown_connections 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 -- 172.21.15.133:0/1689096751 >> 172.21.15.133:0/1689096751 conn(0x7f1cbc067000 msgr2=0x7f1cbc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 -- 172.21.15.133:0/1689096751 shutdown_connections 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 -- 172.21.15.133:0/1689096751 wait complete. 2024-04-25T10:12:50.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 Processor -- start 2024-04-25T10:12:50.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 -- start start 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ca8004d50 0x7f1cbc15b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cbc0a85c0 0x7f1cbc1f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cbc15b540 0x7f1cbc1f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc15cf00 con 0x7f1cbc15b540 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc15cd80 con 0x7f1cbc0a85c0 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc15d850 con 0x7f1ca8004d50 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc0f9a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cbc15b540 0x7f1cbc1f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc0f9a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cbc15b540 0x7f1cbc1f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56690/0 (socket says 172.21.15.133:56690) 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc0f9a640 1 -- 172.21.15.133:0/958379592 learned_addr learned my addr 172.21.15.133:0/958379592 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cb37fe640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cbc0a85c0 0x7f1cbc1f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cbbfff640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ca8004d50 0x7f1cbc15b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc0f9a640 1 -- 172.21.15.133:0/958379592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ca8004d50 msgr2=0x7f1cbc15b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc0f9a640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ca8004d50 0x7f1cbc15b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc0f9a640 1 -- 172.21.15.133:0/958379592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cbc0a85c0 msgr2=0x7f1cbc1f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc0f9a640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cbc0a85c0 0x7f1cbc1f0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc0f9a640 1 -- 172.21.15.133:0/958379592 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc15b9c0 con 0x7f1cbc15b540 2024-04-25T10:12:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc0f9a640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cbc15b540 0x7f1cbc1f14a0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f1cb401d170 tx=0x7f1ca8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cb97fa640 1 -- 172.21.15.133:0/958379592 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40b2040 con 0x7f1cbc15b540 2024-04-25T10:12:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cb97fa640 1 -- 172.21.15.133:0/958379592 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb40017c0 con 0x7f1cbc15b540 2024-04-25T10:12:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 -- 172.21.15.133:0/958379592 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc15bca0 con 0x7f1cbc15b540 2024-04-25T10:12:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cb97fa640 1 -- 172.21.15.133:0/958379592 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4002040 con 0x7f1cbc15b540 2024-04-25T10:12:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.933+0000 7f1cc29fd640 1 -- 172.21.15.133:0/958379592 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc1f19e0 con 0x7f1cbc15b540 2024-04-25T10:12:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.937+0000 7f1cc29fd640 1 -- 172.21.15.133:0/958379592 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc068ef0 con 0x7f1cbc15b540 2024-04-25T10:12:50.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.937+0000 7f1cb97fa640 1 -- 172.21.15.133:0/958379592 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cb40072d0 con 0x7f1cbc15b540 2024-04-25T10:12:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.937+0000 7f1cb97fa640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c9c066f60 0x7f1c9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.937+0000 7f1cb97fa640 1 -- 172.21.15.133:0/958379592 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1cb411eb10 con 0x7f1cbc15b540 2024-04-25T10:12:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.937+0000 7f1cbbfff640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c9c066f60 0x7f1c9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.937+0000 7f1cbbfff640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c9c066f60 0x7f1c9c069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc06ebd0 tx=0x7f1cac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:50.941+0000 7f1cb97fa640 1 -- 172.21.15.133:0/958379592 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cb40bb060 con 0x7f1cbc15b540 2024-04-25T10:12:51.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.137+0000 7f1cc29fd640 1 -- 172.21.15.133:0/958379592 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cbc15eb60 con 0x7f1cbc15b540 2024-04-25T10:12:51.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.137+0000 7f1cb97fa640 1 -- 172.21.15.133:0/958379592 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f1cb40170b0 con 0x7f1cbc15b540 2024-04-25T10:12:51.140 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:51.140 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:51.141 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:51.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 -- 172.21.15.133:0/958379592 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c9c066f60 msgr2=0x7f1c9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c9c066f60 0x7f1c9c069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc06ebd0 tx=0x7f1cac008040 comp rx=0 tx=0).stop 2024-04-25T10:12:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 -- 172.21.15.133:0/958379592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cbc15b540 msgr2=0x7f1cbc1f14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cbc15b540 0x7f1cbc1f14a0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f1cb401d170 tx=0x7f1ca8005840 comp rx=0 tx=0).stop 2024-04-25T10:12:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 -- 172.21.15.133:0/958379592 shutdown_connections 2024-04-25T10:12:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c9c066f60 0x7f1c9c069420 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cbc15b540 0x7f1cbc1f14a0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cbc0a85c0 0x7f1cbc1f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 --2- 172.21.15.133:0/958379592 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ca8004d50 0x7f1cbc15b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 -- 172.21.15.133:0/958379592 >> 172.21.15.133:0/958379592 conn(0x7f1cbc067000 msgr2=0x7f1cbc0b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 -- 172.21.15.133:0/958379592 shutdown_connections 2024-04-25T10:12:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.141+0000 7f1cc29fd640 1 -- 172.21.15.133:0/958379592 wait complete. 2024-04-25T10:12:51.159 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:12:51.159 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:51.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b2da79640 1 Processor -- start 2024-04-25T10:12:51.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b2da79640 1 -- start start 2024-04-25T10:12:51.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b2da79640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3b2815a1a0 0x7f3b2815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b2da79640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3b2815e050 0x7f3b2806d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b2da79640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2815e460 con 0x7f3b28159dc0 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b2da79640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b280ce880 con 0x7f3b2815e050 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b2da79640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b280c49e0 con 0x7f3b2815a1a0 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b27fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3b2815e050 0x7f3b2806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b277fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3b28159dc0 0x7f3b280cba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:56610/0 (socket says 172.21.15.133:56610) 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b277fe640 1 -- 172.21.15.133:0/3189477595 learned_addr learned my addr 172.21.15.133:0/3189477595 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b26ffd640 1 --2- 172.21.15.133:0/3189477595 >> v2:172.21.15.162:3300/0 conn(0x7f3b2815a1a0 0x7f3b2815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b27fff640 1 -- 172.21.15.133:0/3189477595 >> v2:172.21.15.162:3300/0 conn(0x7f3b2815a1a0 msgr2=0x7f3b2815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b27fff640 1 --2- 172.21.15.133:0/3189477595 >> v2:172.21.15.162:3300/0 conn(0x7f3b2815a1a0 0x7f3b2815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b27fff640 1 -- 172.21.15.133:0/3189477595 >> v1:172.21.15.133:6789/0 conn(0x7f3b28159dc0 legacy=0x7f3b280cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b267fc640 1 -- 172.21.15.133:0/3189477595 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390439305 0 0) 0x7f3b18001120 con 0x7f3b28159dc0 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.273+0000 7f3b27fff640 1 -- 172.21.15.133:0/3189477595 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280ceb20 con 0x7f3b2815e050 2024-04-25T10:12:51.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b26ffd640 1 --2- 172.21.15.133:0/3189477595 >> v2:172.21.15.162:3300/0 conn(0x7f3b2815a1a0 0x7f3b2815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:51.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b27fff640 1 --2- 172.21.15.133:0/3189477595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b2815e050 0x7f3b2806d350 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f3b2806f6e0 tx=0x7f3b2001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a94b5b147f7bd52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b267fc640 1 -- 172.21.15.133:0/3189477595 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200ad070 con 0x7f3b2815e050 2024-04-25T10:12:51.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b267fc640 1 -- 172.21.15.133:0/3189477595 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b200a7980 con 0x7f3b2815e050 2024-04-25T10:12:51.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b267fc640 1 -- 172.21.15.133:0/3189477595 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200a7d50 con 0x7f3b2815e050 2024-04-25T10:12:51.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b2da79640 1 -- 172.21.15.133:0/3189477595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b2815e050 msgr2=0x7f3b2806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b2da79640 1 --2- 172.21.15.133:0/3189477595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b2815e050 0x7f3b2806d350 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f3b2806f6e0 tx=0x7f3b2001eb30 comp rx=0 tx=0).stop 2024-04-25T10:12:51.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b2da79640 1 -- 172.21.15.133:0/3189477595 shutdown_connections 2024-04-25T10:12:51.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b2da79640 1 --2- 172.21.15.133:0/3189477595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b2815e050 0x7f3b2806d350 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b2da79640 1 --2- 172.21.15.133:0/3189477595 >> v2:172.21.15.162:3300/0 conn(0x7f3b2815a1a0 0x7f3b2815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b2da79640 1 -- 172.21.15.133:0/3189477595 >> 172.21.15.133:0/3189477595 conn(0x7f3b28067000 msgr2=0x7f3b2806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b2da79640 1 -- 172.21.15.133:0/3189477595 shutdown_connections 2024-04-25T10:12:51.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.277+0000 7f3b2da79640 1 -- 172.21.15.133:0/3189477595 wait complete. 2024-04-25T10:12:51.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b2da79640 1 Processor -- start 2024-04-25T10:12:51.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b2da79640 1 -- start start 2024-04-25T10:12:51.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b2da79640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b28159dc0 0x7f3b2814f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b2da79640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2815e050 0x7f3b2814bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b2da79640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28154a00 0x7f3b28155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b2da79640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2806da10 con 0x7f3b28154a00 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b2da79640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2806d890 con 0x7f3b2815e050 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b2da79640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2806db90 con 0x7f3b28159dc0 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28154a00 0x7f3b28155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28154a00 0x7f3b28155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56706/0 (socket says 172.21.15.133:56706) 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b27fff640 1 -- 172.21.15.133:0/1056897304 learned_addr learned my addr 172.21.15.133:0/1056897304 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b26ffd640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2815e050 0x7f3b2814bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b27fff640 1 -- 172.21.15.133:0/1056897304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b28159dc0 msgr2=0x7f3b2814f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b27fff640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b28159dc0 0x7f3b2814f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b27fff640 1 -- 172.21.15.133:0/1056897304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2815e050 msgr2=0x7f3b2814bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b27fff640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2815e050 0x7f3b2814bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b27fff640 1 -- 172.21.15.133:0/1056897304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b281543c0 con 0x7f3b28154a00 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b26ffd640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2815e050 0x7f3b2814bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:51.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b27fff640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28154a00 0x7f3b28155700 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f3b200b3040 tx=0x7f3b2001b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b0ffff640 1 -- 172.21.15.133:0/1056897304 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200bf020 con 0x7f3b28154a00 2024-04-25T10:12:51.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b0ffff640 1 -- 172.21.15.133:0/1056897304 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2000b280 con 0x7f3b28154a00 2024-04-25T10:12:51.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b0ffff640 1 -- 172.21.15.133:0/1056897304 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200ad040 con 0x7f3b28154a00 2024-04-25T10:12:51.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b2da79640 1 -- 172.21.15.133:0/1056897304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b281546a0 con 0x7f3b28154a00 2024-04-25T10:12:51.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b2da79640 1 -- 172.21.15.133:0/1056897304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2815ba20 con 0x7f3b28154a00 2024-04-25T10:12:51.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b2da79640 1 -- 172.21.15.133:0/1056897304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b28068ef0 con 0x7f3b28154a00 2024-04-25T10:12:51.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b0ffff640 1 -- 172.21.15.133:0/1056897304 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b200b1890 con 0x7f3b28154a00 2024-04-25T10:12:51.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.281+0000 7f3b0ffff640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b14066f60 0x7f3b14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.285+0000 7f3b0ffff640 1 -- 172.21.15.133:0/1056897304 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b2011fbd0 con 0x7f3b28154a00 2024-04-25T10:12:51.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.285+0000 7f3b277fe640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b14066f60 0x7f3b14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.285+0000 7f3b277fe640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b14066f60 0x7f3b14069420 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001680 tx=0x7f3b18001320 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.289+0000 7f3b0ffff640 1 -- 172.21.15.133:0/1056897304 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b200c9060 con 0x7f3b28154a00 2024-04-25T10:12:51.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.505+0000 7f3b2da79640 1 -- 172.21.15.133:0/1056897304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3b280cdaf0 con 0x7f3b28154a00 2024-04-25T10:12:51.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.505+0000 7f3b0ffff640 1 -- 172.21.15.133:0/1056897304 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f3b20018720 con 0x7f3b28154a00 2024-04-25T10:12:51.508 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:51.508 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":684,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":120,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078426026,"bytes_used":2663088128,"bytes_avail":770431025152,"bytes_total":773094113280,"write_bytes_sec":42348691,"read_op_per_sec":0,"write_op_per_sec":43},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8919}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:12:45.917935+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:51.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 -- 172.21.15.133:0/1056897304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b14066f60 msgr2=0x7f3b14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b14066f60 0x7f3b14069420 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001680 tx=0x7f3b18001320 comp rx=0 tx=0).stop 2024-04-25T10:12:51.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 -- 172.21.15.133:0/1056897304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28154a00 msgr2=0x7f3b28155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28154a00 0x7f3b28155700 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f3b200b3040 tx=0x7f3b2001b8d0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 -- 172.21.15.133:0/1056897304 shutdown_connections 2024-04-25T10:12:51.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b14066f60 0x7f3b14069420 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28154a00 0x7f3b28155700 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3b2815e050 0x7f3b2814bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 --2- 172.21.15.133:0/1056897304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3b28159dc0 0x7f3b2814f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 -- 172.21.15.133:0/1056897304 >> 172.21.15.133:0/1056897304 conn(0x7f3b28067000 msgr2=0x7f3b280b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 -- 172.21.15.133:0/1056897304 shutdown_connections 2024-04-25T10:12:51.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:51.509+0000 7f3b2da79640 1 -- 172.21.15.133:0/1056897304 wait complete. 2024-04-25T10:12:51.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T10:12:52.528 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:52.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.657+0000 7f2d2923b640 1 Processor -- start 2024-04-25T10:12:52.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.657+0000 7f2d2923b640 1 -- start start 2024-04-25T10:12:52.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.657+0000 7f2d2923b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d2415a1a0 0x7f2d2415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.657+0000 7f2d2923b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d240ce880 con 0x7f2d24159dc0 2024-04-25T10:12:52.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.657+0000 7f2d2923b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d240c49e0 con 0x7f2d2415e050 2024-04-25T10:12:52.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.657+0000 7f2d2923b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2415e4a0 con 0x7f2d2415a1a0 2024-04-25T10:12:52.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.657+0000 7f2d22d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2d24159dc0 0x7f2d240cba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:56618/0 (socket says 172.21.15.133:56618) 2024-04-25T10:12:52.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.657+0000 7f2d22d9d640 1 -- 172.21.15.133:0/3914099298 learned_addr learned my addr 172.21.15.133:0/3914099298 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.657+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025295897 0 0) 0x7f2d240ce880 con 0x7f2d24159dc0 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.657+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d0c003530 con 0x7f2d24159dc0 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.657+0000 7f2d2259c640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d2415a1a0 0x7f2d2415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3903007024 0 0) 0x7f2d0c003530 con 0x7f2d24159dc0 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d2415a1a0 msgr2=0x7f2d2415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d2415a1a0 0x7f2d2415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 >> v1:172.21.15.162:6789/0 conn(0x7f2d2415e050 legacy=0x7f2d2406d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d240ceb20 con 0x7f2d24159dc0 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2d14001a00 con 0x7f2d24159dc0 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 >> v1:172.21.15.133:6789/0 conn(0x7f2d24159dc0 legacy=0x7f2d240cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d0c0039f0 0x7f2d0c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c004810 0x7f2d0c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d0c0051d0 0x7f2d0c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2415e4a0 con 0x7f2d0c004810 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c00bf10 con 0x7f2d0c0039f0 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c00c0e0 con 0x7f2d0c0051d0 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2259c640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c004810 0x7f2d0c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d22d9d640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d0c0039f0 0x7f2d0c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2359e640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d0c0051d0 0x7f2d0c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2259c640 1 -- 172.21.15.133:0/3914099298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d0c0051d0 msgr2=0x7f2d0c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2259c640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d0c0051d0 0x7f2d0c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2259c640 1 -- 172.21.15.133:0/3914099298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d0c0039f0 msgr2=0x7f2d0c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2259c640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d0c0039f0 0x7f2d0c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2259c640 1 -- 172.21.15.133:0/3914099298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c003530 con 0x7f2d0c004810 2024-04-25T10:12:52.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2259c640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c004810 0x7f2d0c004c90 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800ca80 tx=0x7f2d1801db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4a44221855c995 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d180ac070 con 0x7f2d0c004810 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d180a6a40 con 0x7f2d0c004810 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d21d9b640 1 -- 172.21.15.133:0/3914099298 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800b040 con 0x7f2d0c004810 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2923b640 1 -- 172.21.15.133:0/3914099298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c004810 msgr2=0x7f2d0c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2923b640 1 --2- 172.21.15.133:0/3914099298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d0c004810 0x7f2d0c004c90 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800ca80 tx=0x7f2d1801db50 comp rx=0 tx=0).stop 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2359e640 1 -- 172.21.15.133:0/3914099298 reap_dead start 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2923b640 1 -- 172.21.15.133:0/3914099298 shutdown_connections 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2923b640 1 -- 172.21.15.133:0/3914099298 >> 172.21.15.133:0/3914099298 conn(0x7f2d24067000 msgr2=0x7f2d2406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2923b640 1 -- 172.21.15.133:0/3914099298 shutdown_connections 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.661+0000 7f2d2923b640 1 -- 172.21.15.133:0/3914099298 wait complete. 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2923b640 1 Processor -- start 2024-04-25T10:12:52.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2923b640 1 -- start start 2024-04-25T10:12:52.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2923b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d0c004810 0x7f2d2413ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2923b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d2413d700 0x7f2d24150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2923b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2413db80 0x7f2d241514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2923b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240c47c0 con 0x7f2d2413db80 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2923b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240c4640 con 0x7f2d0c004810 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2923b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240c4cc0 con 0x7f2d2413d700 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2413db80 0x7f2d241514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2413db80 0x7f2d241514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56730/0 (socket says 172.21.15.133:56730) 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2359e640 1 -- 172.21.15.133:0/4210919774 learned_addr learned my addr 172.21.15.133:0/4210919774 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2259c640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d2413d700 0x7f2d24150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d22d9d640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d0c004810 0x7f2d2413ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2359e640 1 -- 172.21.15.133:0/4210919774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d2413d700 msgr2=0x7f2d24150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2359e640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d2413d700 0x7f2d24150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2359e640 1 -- 172.21.15.133:0/4210919774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d0c004810 msgr2=0x7f2d2413ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2359e640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d0c004810 0x7f2d2413ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2359e640 1 -- 172.21.15.133:0/4210919774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d241501e0 con 0x7f2d2413db80 2024-04-25T10:12:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2359e640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2413db80 0x7f2d241514e0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f2d2415e430 tx=0x7f2d1c00e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d0b7fe640 1 -- 172.21.15.133:0/4210919774 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c019070 con 0x7f2d2413db80 2024-04-25T10:12:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d0b7fe640 1 -- 172.21.15.133:0/4210919774 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c0052e0 con 0x7f2d2413db80 2024-04-25T10:12:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2923b640 1 -- 172.21.15.133:0/4210919774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d241504c0 con 0x7f2d2413db80 2024-04-25T10:12:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d0b7fe640 1 -- 172.21.15.133:0/4210919774 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c003820 con 0x7f2d2413db80 2024-04-25T10:12:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2923b640 1 -- 172.21.15.133:0/4210919774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d24150a80 con 0x7f2d2413db80 2024-04-25T10:12:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.665+0000 7f2d2923b640 1 -- 172.21.15.133:0/4210919774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d24150c80 con 0x7f2d2413db80 2024-04-25T10:12:52.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.669+0000 7f2d0b7fe640 1 -- 172.21.15.133:0/4210919774 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d1c01c070 con 0x7f2d2413db80 2024-04-25T10:12:52.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.669+0000 7f2d0b7fe640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d10066f60 0x7f2d10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.669+0000 7f2d0b7fe640 1 -- 172.21.15.133:0/4210919774 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d1c084660 con 0x7f2d2413db80 2024-04-25T10:12:52.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.669+0000 7f2d22d9d640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d10066f60 0x7f2d10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.669+0000 7f2d22d9d640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d10066f60 0x7f2d10069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2d14005600 tx=0x7f2d14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.673+0000 7f2d0b7fe640 1 -- 172.21.15.133:0/4210919774 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d24150c80 con 0x7f2d2413db80 2024-04-25T10:12:52.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.857+0000 7f2d2923b640 1 -- 172.21.15.133:0/4210919774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d24068ef0 con 0x7f2d2413db80 2024-04-25T10:12:52.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.857+0000 7f2d0b7fe640 1 -- 172.21.15.133:0/4210919774 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f2d1c082e20 con 0x7f2d2413db80 2024-04-25T10:12:52.861 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:52.861 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:52.861 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:52.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 -- 172.21.15.133:0/4210919774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d10066f60 msgr2=0x7f2d10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d10066f60 0x7f2d10069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2d14005600 tx=0x7f2d14009040 comp rx=0 tx=0).stop 2024-04-25T10:12:52.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 -- 172.21.15.133:0/4210919774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2413db80 msgr2=0x7f2d241514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2413db80 0x7f2d241514e0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f2d2415e430 tx=0x7f2d1c00e8a0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 -- 172.21.15.133:0/4210919774 shutdown_connections 2024-04-25T10:12:52.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d10066f60 0x7f2d10069420 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2413db80 0x7f2d241514e0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2d2413d700 0x7f2d24150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 --2- 172.21.15.133:0/4210919774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2d0c004810 0x7f2d2413ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 -- 172.21.15.133:0/4210919774 >> 172.21.15.133:0/4210919774 conn(0x7f2d24067000 msgr2=0x7f2d2406d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 -- 172.21.15.133:0/4210919774 shutdown_connections 2024-04-25T10:12:52.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.861+0000 7f2d2923b640 1 -- 172.21.15.133:0/4210919774 wait complete. 2024-04-25T10:12:52.878 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:12:52.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:52.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.981+0000 7f68bf5d0640 1 Processor -- start 2024-04-25T10:12:52.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.981+0000 7f68bf5d0640 1 -- start start 2024-04-25T10:12:52.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.981+0000 7f68bf5d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b815da90 0x7f68b80729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.981+0000 7f68bf5d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68b8072ee0 0x7f68b80b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.981+0000 7f68bf5d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b80b4300 con 0x7f68b80b3bb0 2024-04-25T10:12:52.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.981+0000 7f68bf5d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b80b45e0 con 0x7f68b8072ee0 2024-04-25T10:12:52.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.981+0000 7f68bf5d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b80b48c0 con 0x7f68b815da90 2024-04-25T10:12:52.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bcb6b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68b8072ee0 0x7f68b80b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bd36c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b815da90 0x7f68b80729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bdb6d640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f68b80b3bb0 0x7f68b806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:36968/0 (socket says 172.21.15.133:36968) 2024-04-25T10:12:52.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bdb6d640 1 -- 172.21.15.133:0/1469786371 learned_addr learned my addr 172.21.15.133:0/1469786371 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:52.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bcb6b640 1 -- 172.21.15.133:0/1469786371 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b815da90 msgr2=0x7f68b80729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bcb6b640 1 --2- 172.21.15.133:0/1469786371 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b815da90 0x7f68b80729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bcb6b640 1 -- 172.21.15.133:0/1469786371 >> v1:172.21.15.162:6789/0 conn(0x7f68b80b3bb0 legacy=0x7f68b806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bcb6b640 1 -- 172.21.15.133:0/1469786371 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b8072480 con 0x7f68b8072ee0 2024-04-25T10:12:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bcb6b640 1 --2- 172.21.15.133:0/1469786371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b8072ee0 0x7f68b80b3670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f68a80097c0 tx=0x7f68a801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e3a122d14286ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68affff640 1 -- 172.21.15.133:0/1469786371 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a80b0070 con 0x7f68b8072ee0 2024-04-25T10:12:52.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68affff640 1 -- 172.21.15.133:0/1469786371 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a80ab600 con 0x7f68b8072ee0 2024-04-25T10:12:52.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68affff640 1 -- 172.21.15.133:0/1469786371 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a80ab9d0 con 0x7f68b8072ee0 2024-04-25T10:12:52.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1469786371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b8072ee0 msgr2=0x7f68b80b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bf5d0640 1 --2- 172.21.15.133:0/1469786371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b8072ee0 0x7f68b80b3670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f68a80097c0 tx=0x7f68a801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1469786371 shutdown_connections 2024-04-25T10:12:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bf5d0640 1 --2- 172.21.15.133:0/1469786371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b8072ee0 0x7f68b80b3670 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bf5d0640 1 --2- 172.21.15.133:0/1469786371 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b815da90 0x7f68b80729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1469786371 >> 172.21.15.133:0/1469786371 conn(0x7f68b80b2370 msgr2=0x7f68b80b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1469786371 shutdown_connections 2024-04-25T10:12:52.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.985+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1469786371 wait complete. 2024-04-25T10:12:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bf5d0640 1 Processor -- start 2024-04-25T10:12:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bf5d0640 1 -- start start 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bf5d0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b8072ee0 0x7f68b813dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bf5d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b80b3bb0 0x7f68b815eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bf5d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b815da90 0x7f68b8155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bf5d0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b806b520 con 0x7f68b8072ee0 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bf5d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b806b3a0 con 0x7f68b80b3bb0 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bf5d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b816aa00 con 0x7f68b815da90 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bd36c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b8072ee0 0x7f68b813dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bd36c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b8072ee0 0x7f68b813dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56732/0 (socket says 172.21.15.133:56732) 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bd36c640 1 -- 172.21.15.133:0/1801414796 learned_addr learned my addr 172.21.15.133:0/1801414796 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bcb6b640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b80b3bb0 0x7f68b815eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bdb6d640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b815da90 0x7f68b8155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bd36c640 1 -- 172.21.15.133:0/1801414796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b815da90 msgr2=0x7f68b8155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bd36c640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b815da90 0x7f68b8155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bd36c640 1 -- 172.21.15.133:0/1801414796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b80b3bb0 msgr2=0x7f68b815eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bd36c640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b80b3bb0 0x7f68b815eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bd36c640 1 -- 172.21.15.133:0/1801414796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b815f5b0 con 0x7f68b8072ee0 2024-04-25T10:12:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bd36c640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b8072ee0 0x7f68b813dc70 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f68b400e930 tx=0x7f68b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68adffb640 1 -- 172.21.15.133:0/1801414796 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b4017070 con 0x7f68b8072ee0 2024-04-25T10:12:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68adffb640 1 -- 172.21.15.133:0/1801414796 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b400ca00 con 0x7f68b8072ee0 2024-04-25T10:12:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1801414796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b815f890 con 0x7f68b8072ee0 2024-04-25T10:12:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68adffb640 1 -- 172.21.15.133:0/1801414796 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b400cdb0 con 0x7f68b8072ee0 2024-04-25T10:12:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1801414796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b813e820 con 0x7f68b8072ee0 2024-04-25T10:12:52.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.989+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1801414796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b8068ef0 con 0x7f68b8072ee0 2024-04-25T10:12:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.993+0000 7f68adffb640 1 -- 172.21.15.133:0/1801414796 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68b401a070 con 0x7f68b8072ee0 2024-04-25T10:12:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.993+0000 7f68adffb640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f689c066f60 0x7f689c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.993+0000 7f68adffb640 1 -- 172.21.15.133:0/1801414796 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68b4085530 con 0x7f68b8072ee0 2024-04-25T10:12:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.993+0000 7f68bcb6b640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f689c066f60 0x7f689c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.993+0000 7f68bcb6b640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f689c066f60 0x7f689c069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f68a80ad000 tx=0x7f68a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:52.997+0000 7f68adffb640 1 -- 172.21.15.133:0/1801414796 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68b4056500 con 0x7f68b8072ee0 2024-04-25T10:12:53.159 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:53.163 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:53.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.197+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1801414796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f68b8154a40 con 0x7f68b8072ee0 2024-04-25T10:12:53.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.197+0000 7f68adffb640 1 -- 172.21.15.133:0/1801414796 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f68b4055e00 con 0x7f68b8072ee0 2024-04-25T10:12:53.200 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:53.200 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":686,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":120,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078426026,"bytes_used":2663088128,"bytes_avail":770431025152,"bytes_total":773094113280,"write_bytes_sec":16914154,"read_op_per_sec":0,"write_op_per_sec":18},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8919}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:12:51.919939+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1801414796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f689c066f60 msgr2=0x7f689c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f689c066f60 0x7f689c069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f68a80ad000 tx=0x7f68a8008040 comp rx=0 tx=0).stop 2024-04-25T10:12:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1801414796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b8072ee0 msgr2=0x7f68b813dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b8072ee0 0x7f68b813dc70 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f68b400e930 tx=0x7f68b400edf0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1801414796 shutdown_connections 2024-04-25T10:12:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68b815da90 0x7f68b8155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68b80b3bb0 0x7f68b815eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f689c066f60 0x7f689c069420 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 --2- 172.21.15.133:0/1801414796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b8072ee0 0x7f68b813dc70 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1801414796 >> 172.21.15.133:0/1801414796 conn(0x7f68b80b2370 msgr2=0x7f68b80b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1801414796 shutdown_connections 2024-04-25T10:12:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.201+0000 7f68bf5d0640 1 -- 172.21.15.133:0/1801414796 wait complete. 2024-04-25T10:12:53.217 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:12:53.217 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:12:53.218 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:53.331 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:53.370 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:12:53.370 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:12:53.370 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:53.377 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:53.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.537+0000 7fc8d60b8640 1 Processor -- start 2024-04-25T10:12:53.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8d60b8640 1 -- start start 2024-04-25T10:12:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8d60b8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc8d015c9a0 0x7fc8d00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8d60b8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc8d00ce330 0x7fc8d00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8d60b8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00c61d0 con 0x7fc8d00b1f70 2024-04-25T10:12:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8d60b8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d00cecc0 con 0x7fc8d015c9a0 2024-04-25T10:12:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8d60b8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d00c64b0 con 0x7fc8d00ce330 2024-04-25T10:12:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8cf7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc8d015c9a0 0x7fc8d00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8ceffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc8d00b1f70 0x7fc8d006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:56628/0 (socket says 172.21.15.133:56628) 2024-04-25T10:12:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8ceffd640 1 -- 172.21.15.133:0/2614734409 learned_addr learned my addr 172.21.15.133:0/2614734409 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:53.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8cffff640 1 --2- 172.21.15.133:0/2614734409 >> v2:172.21.15.162:3300/0 conn(0x7fc8d00ce330 0x7fc8d00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8cf7fe640 1 -- 172.21.15.133:0/2614734409 >> v2:172.21.15.162:3300/0 conn(0x7fc8d00ce330 msgr2=0x7fc8d00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8cf7fe640 1 --2- 172.21.15.133:0/2614734409 >> v2:172.21.15.162:3300/0 conn(0x7fc8d00ce330 0x7fc8d00ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8cf7fe640 1 -- 172.21.15.133:0/2614734409 >> v1:172.21.15.133:6789/0 conn(0x7fc8d00b1f70 legacy=0x7fc8d006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8ce7fc640 1 -- 172.21.15.133:0/2614734409 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512277876 0 0) 0x7fc8d00c61d0 con 0x7fc8d00b1f70 2024-04-25T10:12:53.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8cf7fe640 1 -- 172.21.15.133:0/2614734409 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d006d4f0 con 0x7fc8d015c9a0 2024-04-25T10:12:53.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.541+0000 7fc8cffff640 1 --2- 172.21.15.133:0/2614734409 >> v2:172.21.15.162:3300/0 conn(0x7fc8d00ce330 0x7fc8d00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8cf7fe640 1 --2- 172.21.15.133:0/2614734409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8d015c9a0 0x7fc8d00c5980 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00cc0f0 tx=0x7fc8c001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5014ed11db60233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8ce7fc640 1 -- 172.21.15.133:0/2614734409 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00b0070 con 0x7fc8d015c9a0 2024-04-25T10:12:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8ce7fc640 1 -- 172.21.15.133:0/2614734409 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c00ab600 con 0x7fc8d015c9a0 2024-04-25T10:12:53.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8ce7fc640 1 -- 172.21.15.133:0/2614734409 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00ab9d0 con 0x7fc8d015c9a0 2024-04-25T10:12:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 -- 172.21.15.133:0/2614734409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8d015c9a0 msgr2=0x7fc8d00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 --2- 172.21.15.133:0/2614734409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8d015c9a0 0x7fc8d00c5980 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00cc0f0 tx=0x7fc8c001c480 comp rx=0 tx=0).stop 2024-04-25T10:12:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 -- 172.21.15.133:0/2614734409 shutdown_connections 2024-04-25T10:12:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 --2- 172.21.15.133:0/2614734409 >> v2:172.21.15.162:3300/0 conn(0x7fc8d00ce330 0x7fc8d00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 --2- 172.21.15.133:0/2614734409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8d015c9a0 0x7fc8d00c5980 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 -- 172.21.15.133:0/2614734409 >> 172.21.15.133:0/2614734409 conn(0x7fc8d0067000 msgr2=0x7fc8d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 -- 172.21.15.133:0/2614734409 shutdown_connections 2024-04-25T10:12:53.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 -- 172.21.15.133:0/2614734409 wait complete. 2024-04-25T10:12:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 Processor -- start 2024-04-25T10:12:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 -- start start 2024-04-25T10:12:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8d00b1f70 0x7fc8d00a8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8d00ce330 0x7fc8d0164960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8d015c9a0 0x7fc8d0164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d015dad0 con 0x7fc8d015c9a0 2024-04-25T10:12:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d015d950 con 0x7fc8d00ce330 2024-04-25T10:12:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8d60b8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d015dc50 con 0x7fc8d00b1f70 2024-04-25T10:12:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8d015c9a0 0x7fc8d0164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8d015c9a0 0x7fc8d0164ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56754/0 (socket says 172.21.15.133:56754) 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8cffff640 1 -- 172.21.15.133:0/3172262058 learned_addr learned my addr 172.21.15.133:0/3172262058 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8cf7fe640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8d00b1f70 0x7fc8d00a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8ceffd640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8d00ce330 0x7fc8d0164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8cffff640 1 -- 172.21.15.133:0/3172262058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8d00b1f70 msgr2=0x7fc8d00a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8cffff640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8d00b1f70 0x7fc8d00a8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8cffff640 1 -- 172.21.15.133:0/3172262058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8d00ce330 msgr2=0x7fc8d0164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8cffff640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8d00ce330 0x7fc8d0164960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.545+0000 7fc8cffff640 1 -- 172.21.15.133:0/3172262058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00a8970 con 0x7fc8d015c9a0 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8cffff640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8d015c9a0 0x7fc8d0164ea0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800cd80 tx=0x7fc8c800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8b7fff640 1 -- 172.21.15.133:0/3172262058 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c800f680 con 0x7fc8d015c9a0 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8b7fff640 1 -- 172.21.15.133:0/3172262058 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c800f840 con 0x7fc8d015c9a0 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8d60b8640 1 -- 172.21.15.133:0/3172262058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d00c8890 con 0x7fc8d015c9a0 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8b7fff640 1 -- 172.21.15.133:0/3172262058 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c800fc30 con 0x7fc8d015c9a0 2024-04-25T10:12:53.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8d60b8640 1 -- 172.21.15.133:0/3172262058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d00c8e50 con 0x7fc8d015c9a0 2024-04-25T10:12:53.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8d60b8640 1 -- 172.21.15.133:0/3172262058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d006bc20 con 0x7fc8d015c9a0 2024-04-25T10:12:53.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8b7fff640 1 -- 172.21.15.133:0/3172262058 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8c80213f0 con 0x7fc8d015c9a0 2024-04-25T10:12:53.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8b7fff640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc8bc066f60 0x7fc8bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8b7fff640 1 -- 172.21.15.133:0/3172262058 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8c8085f30 con 0x7fc8d015c9a0 2024-04-25T10:12:53.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8cf7fe640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc8bc066f60 0x7fc8bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.549+0000 7fc8cf7fe640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc8bc066f60 0x7fc8bc069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00ad000 tx=0x7fc8c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.553+0000 7fc8b7fff640 1 -- 172.21.15.133:0/3172262058 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc8c8014030 con 0x7fc8d015c9a0 2024-04-25T10:12:53.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.713+0000 7fc8d60b8640 1 -- 172.21.15.133:0/3172262058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8d0068ef0 con 0x7fc8d015c9a0 2024-04-25T10:12:53.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.713+0000 7fc8b7fff640 1 -- 172.21.15.133:0/3172262058 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fc8c8056db0 con 0x7fc8d015c9a0 2024-04-25T10:12:53.718 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:53.718 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":120,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:12:19.584218+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:16.197174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:16.904937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:53.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 -- 172.21.15.133:0/3172262058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc8bc066f60 msgr2=0x7fc8bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc8bc066f60 0x7fc8bc069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00ad000 tx=0x7fc8c0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:53.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 -- 172.21.15.133:0/3172262058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8d015c9a0 msgr2=0x7fc8d0164ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8d015c9a0 0x7fc8d0164ea0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800cd80 tx=0x7fc8c800a650 comp rx=0 tx=0).stop 2024-04-25T10:12:53.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 -- 172.21.15.133:0/3172262058 shutdown_connections 2024-04-25T10:12:53.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc8bc066f60 0x7fc8bc069420 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8d015c9a0 0x7fc8d0164ea0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8d00ce330 0x7fc8d0164960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 --2- 172.21.15.133:0/3172262058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc8d00b1f70 0x7fc8d00a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 -- 172.21.15.133:0/3172262058 >> 172.21.15.133:0/3172262058 conn(0x7fc8d0067000 msgr2=0x7fc8d00b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 -- 172.21.15.133:0/3172262058 shutdown_connections 2024-04-25T10:12:53.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.717+0000 7fc8d60b8640 1 -- 172.21.15.133:0/3172262058 wait complete. 2024-04-25T10:12:53.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:16.197174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:16.904937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:53.737 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:53.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fde1fd640 1 Processor -- start 2024-04-25T10:12:53.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fde1fd640 1 -- start start 2024-04-25T10:12:53.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fde1fd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2fd8143ea0 0x7f2fd80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fde1fd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd80b2150 0x7f2fd806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fde1fd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2fd815b9b0 0x7f2fd806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fde1fd640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd815ac50 con 0x7f2fd815b9b0 2024-04-25T10:12:53.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fde1fd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd80cd180 con 0x7f2fd8143ea0 2024-04-25T10:12:53.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fde1fd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd815aed0 con 0x7f2fd80b2150 2024-04-25T10:12:53.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fd7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2fd815b9b0 0x7f2fd806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fd7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2fd815b9b0 0x7f2fd806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56764/0 (socket says 172.21.15.133:56764) 2024-04-25T10:12:53.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fd7fff640 1 -- 172.21.15.133:0/1337370455 learned_addr learned my addr 172.21.15.133:0/1337370455 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fd77fe640 1 --2- 172.21.15.133:0/1337370455 >> v2:172.21.15.162:3300/0 conn(0x7f2fd8143ea0 0x7f2fd80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fd6ffd640 1 --2- 172.21.15.133:0/1337370455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd80b2150 0x7f2fd806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fd7fff640 1 -- 172.21.15.133:0/1337370455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd80b2150 msgr2=0x7f2fd806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fd7fff640 1 --2- 172.21.15.133:0/1337370455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd80b2150 0x7f2fd806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fd7fff640 1 -- 172.21.15.133:0/1337370455 >> v2:172.21.15.162:3300/0 conn(0x7f2fd8143ea0 msgr2=0x7f2fd80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fd7fff640 1 --2- 172.21.15.133:0/1337370455 >> v2:172.21.15.162:3300/0 conn(0x7f2fd8143ea0 0x7f2fd80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.857+0000 7f2fd7fff640 1 -- 172.21.15.133:0/1337370455 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd815b1b0 con 0x7f2fd815b9b0 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd7fff640 1 --2- 172.21.15.133:0/1337370455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd815b9b0 0x7f2fd806e8f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80b4b20 tx=0x7f2fd001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9e980201a157a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd67fc640 1 -- 172.21.15.133:0/1337370455 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00a5780 con 0x7f2fd815b9b0 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd67fc640 1 -- 172.21.15.133:0/1337370455 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd00a5920 con 0x7f2fd815b9b0 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd67fc640 1 -- 172.21.15.133:0/1337370455 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00a5cf0 con 0x7f2fd815b9b0 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 -- 172.21.15.133:0/1337370455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd815b9b0 msgr2=0x7f2fd806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 --2- 172.21.15.133:0/1337370455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd815b9b0 0x7f2fd806e8f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80b4b20 tx=0x7f2fd001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 -- 172.21.15.133:0/1337370455 shutdown_connections 2024-04-25T10:12:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 --2- 172.21.15.133:0/1337370455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd815b9b0 0x7f2fd806e8f0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 --2- 172.21.15.133:0/1337370455 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd80b2150 0x7f2fd806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 --2- 172.21.15.133:0/1337370455 >> v2:172.21.15.162:3300/0 conn(0x7f2fd8143ea0 0x7f2fd80cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 -- 172.21.15.133:0/1337370455 >> 172.21.15.133:0/1337370455 conn(0x7f2fd8066f70 msgr2=0x7f2fd806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 -- 172.21.15.133:0/1337370455 shutdown_connections 2024-04-25T10:12:53.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 -- 172.21.15.133:0/1337370455 wait complete. 2024-04-25T10:12:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 Processor -- start 2024-04-25T10:12:53.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 -- start start 2024-04-25T10:12:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd80b2150 0x7f2fd813f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd8143ea0 0x7f2fd81431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd815b9b0 0x7f2fd8135430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd815ede0 con 0x7f2fd80b2150 2024-04-25T10:12:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd815ec60 con 0x7f2fd8143ea0 2024-04-25T10:12:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fde1fd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd815ef60 con 0x7f2fd815b9b0 2024-04-25T10:12:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd80b2150 0x7f2fd813f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd80b2150 0x7f2fd813f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56768/0 (socket says 172.21.15.133:56768) 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd77fe640 1 -- 172.21.15.133:0/533143846 learned_addr learned my addr 172.21.15.133:0/533143846 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd6ffd640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd8143ea0 0x7f2fd81431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd77fe640 1 -- 172.21.15.133:0/533143846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd815b9b0 msgr2=0x7f2fd8135430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd7fff640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd815b9b0 0x7f2fd8135430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd77fe640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd815b9b0 0x7f2fd8135430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd77fe640 1 -- 172.21.15.133:0/533143846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd8143ea0 msgr2=0x7f2fd81431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd77fe640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd8143ea0 0x7f2fd81431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd77fe640 1 -- 172.21.15.133:0/533143846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd81651f0 con 0x7f2fd80b2150 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.861+0000 7f2fd77fe640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd80b2150 0x7f2fd813f300 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800d800 tx=0x7f2fc800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.865+0000 7f2fbffff640 1 -- 172.21.15.133:0/533143846 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800b6c0 con 0x7f2fd80b2150 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.865+0000 7f2fbffff640 1 -- 172.21.15.133:0/533143846 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc8019070 con 0x7f2fd80b2150 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.865+0000 7f2fde1fd640 1 -- 172.21.15.133:0/533143846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd813f920 con 0x7f2fd80b2150 2024-04-25T10:12:53.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.865+0000 7f2fbffff640 1 -- 172.21.15.133:0/533143846 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800ba60 con 0x7f2fd80b2150 2024-04-25T10:12:53.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.865+0000 7f2fde1fd640 1 -- 172.21.15.133:0/533143846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd815dba0 con 0x7f2fd80b2150 2024-04-25T10:12:53.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.865+0000 7f2fde1fd640 1 -- 172.21.15.133:0/533143846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd8068e80 con 0x7f2fd80b2150 2024-04-25T10:12:53.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.865+0000 7f2fbffff640 1 -- 172.21.15.133:0/533143846 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2fc800bc00 con 0x7f2fd80b2150 2024-04-25T10:12:53.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.865+0000 7f2fbffff640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fc4066f60 0x7f2fc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.865+0000 7f2fbffff640 1 -- 172.21.15.133:0/533143846 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2fc8085540 con 0x7f2fd80b2150 2024-04-25T10:12:53.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.865+0000 7f2fd6ffd640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fc4066f60 0x7f2fc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.865+0000 7f2fd6ffd640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fc4066f60 0x7f2fc4069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2fd806de80 tx=0x7f2fcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:53.869+0000 7f2fbffff640 1 -- 172.21.15.133:0/533143846 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2fc8056450 con 0x7f2fd80b2150 2024-04-25T10:12:54.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.013+0000 7f2fde1fd640 1 -- 172.21.15.133:0/533143846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2fd815a720 con 0x7f2fd80b2150 2024-04-25T10:12:54.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fbffff640 1 -- 172.21.15.133:0/533143846 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2fc8055d50 con 0x7f2fd80b2150 2024-04-25T10:12:54.018 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:12:54.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 -- 172.21.15.133:0/533143846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fc4066f60 msgr2=0x7f2fc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fc4066f60 0x7f2fc4069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2fd806de80 tx=0x7f2fcc008040 comp rx=0 tx=0).stop 2024-04-25T10:12:54.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 -- 172.21.15.133:0/533143846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd80b2150 msgr2=0x7f2fd813f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd80b2150 0x7f2fd813f300 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800d800 tx=0x7f2fc800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 -- 172.21.15.133:0/533143846 shutdown_connections 2024-04-25T10:12:54.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2fd815b9b0 0x7f2fd8135430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2fd8143ea0 0x7f2fd81431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2fc4066f60 0x7f2fc4069420 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 --2- 172.21.15.133:0/533143846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd80b2150 0x7f2fd813f300 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 -- 172.21.15.133:0/533143846 >> 172.21.15.133:0/533143846 conn(0x7f2fd8066f70 msgr2=0x7f2fd815f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 -- 172.21.15.133:0/533143846 shutdown_connections 2024-04-25T10:12:54.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.017+0000 7f2fde1fd640 1 -- 172.21.15.133:0/533143846 wait complete. 2024-04-25T10:12:54.035 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:54.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.145+0000 7f1cd8c98640 1 Processor -- start 2024-04-25T10:12:54.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.145+0000 7f1cd8c98640 1 -- start start 2024-04-25T10:12:54.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd8c98640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1cd40ce600 0x7f1cd40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd8c98640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd40d1920 0x7f1cd40d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd8c98640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd415ebd0 con 0x7f1cd415d9a0 2024-04-25T10:12:54.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd8c98640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd415eeb0 con 0x7f1cd40ce600 2024-04-25T10:12:54.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd8c98640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40d0330 con 0x7f1cd40d1920 2024-04-25T10:12:54.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd1d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1cd40ce600 0x7f1cd40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd1d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1cd40ce600 0x7f1cd40cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56774/0 (socket says 172.21.15.133:56774) 2024-04-25T10:12:54.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd1d9b640 1 -- 172.21.15.133:0/4067866335 learned_addr learned my addr 172.21.15.133:0/4067866335 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:54.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd2d9d640 1 --2- 172.21.15.133:0/4067866335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd40d1920 0x7f1cd40d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd1d9b640 1 -- 172.21.15.133:0/4067866335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd40d1920 msgr2=0x7f1cd40d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd1d9b640 1 --2- 172.21.15.133:0/4067866335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd40d1920 0x7f1cd40d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd1d9b640 1 -- 172.21.15.133:0/4067866335 >> v1:172.21.15.162:6789/0 conn(0x7f1cd415d9a0 legacy=0x7f1cd40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd1d9b640 1 -- 172.21.15.133:0/4067866335 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd40d0610 con 0x7f1cd40ce600 2024-04-25T10:12:54.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd2d9d640 1 --2- 172.21.15.133:0/4067866335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd40d1920 0x7f1cd40d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:54.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd1d9b640 1 --2- 172.21.15.133:0/4067866335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cd40ce600 0x7f1cd40cea90 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009870 tx=0x7f1cc801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee87e9f6801a08e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd159a640 1 -- 172.21.15.133:0/4067866335 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80b0070 con 0x7f1cd40ce600 2024-04-25T10:12:54.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd159a640 1 -- 172.21.15.133:0/4067866335 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc80ab600 con 0x7f1cd40ce600 2024-04-25T10:12:54.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.149+0000 7f1cd159a640 1 -- 172.21.15.133:0/4067866335 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80ab9d0 con 0x7f1cd40ce600 2024-04-25T10:12:54.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- 172.21.15.133:0/4067866335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cd40ce600 msgr2=0x7f1cd40cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 --2- 172.21.15.133:0/4067866335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cd40ce600 0x7f1cd40cea90 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009870 tx=0x7f1cc801c490 comp rx=0 tx=0).stop 2024-04-25T10:12:54.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- 172.21.15.133:0/4067866335 shutdown_connections 2024-04-25T10:12:54.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 --2- 172.21.15.133:0/4067866335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd40d1920 0x7f1cd40d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 --2- 172.21.15.133:0/4067866335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cd40ce600 0x7f1cd40cea90 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- 172.21.15.133:0/4067866335 >> 172.21.15.133:0/4067866335 conn(0x7f1cd406bf80 msgr2=0x7f1cd406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- 172.21.15.133:0/4067866335 shutdown_connections 2024-04-25T10:12:54.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- 172.21.15.133:0/4067866335 wait complete. 2024-04-25T10:12:54.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 Processor -- start 2024-04-25T10:12:54.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- start start 2024-04-25T10:12:54.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd40ce600 0x7f1cd40b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cd40d1920 0x7f1cd4154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd415d9a0 0x7f1cd4157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd415e590 con 0x7f1cd40d1920 2024-04-25T10:12:54.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40d2540 con 0x7f1cd415d9a0 2024-04-25T10:12:54.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd415e710 con 0x7f1cd40ce600 2024-04-25T10:12:54.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cd40d1920 0x7f1cd4154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cd40d1920 0x7f1cd4154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56776/0 (socket says 172.21.15.133:56776) 2024-04-25T10:12:54.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd1d9b640 1 -- 172.21.15.133:0/269572473 learned_addr learned my addr 172.21.15.133:0/269572473 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:54.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd259c640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd40ce600 0x7f1cd40b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd2d9d640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd415d9a0 0x7f1cd4157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd1d9b640 1 -- 172.21.15.133:0/269572473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd40ce600 msgr2=0x7f1cd40b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd1d9b640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd40ce600 0x7f1cd40b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd1d9b640 1 -- 172.21.15.133:0/269572473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd415d9a0 msgr2=0x7f1cd4157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd1d9b640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd415d9a0 0x7f1cd4157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd1d9b640 1 -- 172.21.15.133:0/269572473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd4159680 con 0x7f1cd40d1920 2024-04-25T10:12:54.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd2d9d640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd415d9a0 0x7f1cd4157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:54.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd1d9b640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cd40d1920 0x7f1cd4154d40 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80ae040 tx=0x7f1cc8019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cc2ffd640 1 -- 172.21.15.133:0/269572473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80b0040 con 0x7f1cd40d1920 2024-04-25T10:12:54.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cc2ffd640 1 -- 172.21.15.133:0/269572473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc80b9070 con 0x7f1cd40d1920 2024-04-25T10:12:54.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- 172.21.15.133:0/269572473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd40b4d80 con 0x7f1cd40d1920 2024-04-25T10:12:54.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cc2ffd640 1 -- 172.21.15.133:0/269572473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80b4880 con 0x7f1cd40d1920 2024-04-25T10:12:54.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- 172.21.15.133:0/269572473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd4155d70 con 0x7f1cd40d1920 2024-04-25T10:12:54.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.153+0000 7f1cd8c98640 1 -- 172.21.15.133:0/269572473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd4068e80 con 0x7f1cd40d1920 2024-04-25T10:12:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.157+0000 7f1cc2ffd640 1 -- 172.21.15.133:0/269572473 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cc80ab4c0 con 0x7f1cd40d1920 2024-04-25T10:12:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.157+0000 7f1cc2ffd640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cb4066f60 0x7f1cb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.157+0000 7f1cc2ffd640 1 -- 172.21.15.133:0/269572473 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1cc811eff0 con 0x7f1cd40d1920 2024-04-25T10:12:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.157+0000 7f1cd259c640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cb4066f60 0x7f1cb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.157+0000 7f1cd259c640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cb4066f60 0x7f1cb4069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4004e30 tx=0x7f1cc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.161+0000 7f1cc2ffd640 1 -- 172.21.15.133:0/269572473 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cd4159680 con 0x7f1cd40d1920 2024-04-25T10:12:54.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.305+0000 7f1cd8c98640 1 -- 172.21.15.133:0/269572473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1cd4155280 con 0x7f1cd40d1920 2024-04-25T10:12:54.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.305+0000 7f1cc2ffd640 1 -- 172.21.15.133:0/269572473 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1cc8016110 con 0x7f1cd40d1920 2024-04-25T10:12:54.308 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:12:54.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 -- 172.21.15.133:0/269572473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cb4066f60 msgr2=0x7f1cb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cb4066f60 0x7f1cb4069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4004e30 tx=0x7f1cc4009040 comp rx=0 tx=0).stop 2024-04-25T10:12:54.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 -- 172.21.15.133:0/269572473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cd40d1920 msgr2=0x7f1cd4154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cd40d1920 0x7f1cd4154d40 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80ae040 tx=0x7f1cc8019230 comp rx=0 tx=0).stop 2024-04-25T10:12:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 -- 172.21.15.133:0/269572473 shutdown_connections 2024-04-25T10:12:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1cd415d9a0 0x7f1cd4157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1cb4066f60 0x7f1cb4069420 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cd40d1920 0x7f1cd4154d40 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 --2- 172.21.15.133:0/269572473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1cd40ce600 0x7f1cd40b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 -- 172.21.15.133:0/269572473 >> 172.21.15.133:0/269572473 conn(0x7f1cd406bf80 msgr2=0x7f1cd406c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 -- 172.21.15.133:0/269572473 shutdown_connections 2024-04-25T10:12:54.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.309+0000 7f1cd8c98640 1 -- 172.21.15.133:0/269572473 wait complete. 2024-04-25T10:12:54.326 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:54.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.445+0000 7f2dbc86b640 1 Processor -- start 2024-04-25T10:12:54.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.445+0000 7f2dbc86b640 1 -- start start 2024-04-25T10:12:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2dbc86b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8145170 0x7f2db806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2dbc86b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2db806e4d0 0x7f2db80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2dbc86b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db80ce2e0 con 0x7f2db80cdd60 2024-04-25T10:12:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2dbc86b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db806db00 con 0x7f2db806e4d0 2024-04-25T10:12:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2dbc86b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db8145550 con 0x7f2db8145170 2024-04-25T10:12:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db6d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2db80cdd60 0x7f2db815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:56636/0 (socket says 172.21.15.133:56636) 2024-04-25T10:12:54.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db6d9d640 1 -- 172.21.15.133:0/3757846158 learned_addr learned my addr 172.21.15.133:0/3757846158 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:54.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901763130 0 0) 0x7f2db80ce2e0 con 0x7f2db80cdd60 2024-04-25T10:12:54.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db5d9b640 1 --2- 172.21.15.133:0/3757846158 >> v2:172.21.15.162:3300/0 conn(0x7f2db806e4d0 0x7f2db80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db659c640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8145170 0x7f2db806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2da0003530 con 0x7f2db80cdd60 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 809421043 0 0) 0x7f2da0003530 con 0x7f2db80cdd60 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8145170 msgr2=0x7f2db806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8145170 0x7f2db806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 >> v2:172.21.15.162:3300/0 conn(0x7f2db806e4d0 msgr2=0x7f2db80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 --2- 172.21.15.133:0/3757846158 >> v2:172.21.15.162:3300/0 conn(0x7f2db806e4d0 0x7f2db80cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db806d4b0 con 0x7f2db80cdd60 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2db0001a00 con 0x7f2db80cdd60 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db5d9b640 1 --2- 172.21.15.133:0/3757846158 >> v2:172.21.15.162:3300/0 conn(0x7f2db806e4d0 0x7f2db80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 >> v1:172.21.15.133:6789/0 conn(0x7f2db80cdd60 legacy=0x7f2db815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da00039c0 0x7f2da0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 0x7f2da0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8145170 0x7f2da00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db806db00 con 0x7f2da0004810 2024-04-25T10:12:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db8145550 con 0x7f2da00039c0 2024-04-25T10:12:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da000b9d0 con 0x7f2db8145170 2024-04-25T10:12:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db5d9b640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 0x7f2da0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db659c640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da00039c0 0x7f2da0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db6d9d640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8145170 0x7f2da00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db5d9b640 1 -- 172.21.15.133:0/3757846158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8145170 msgr2=0x7f2da00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db5d9b640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8145170 0x7f2da00051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db5d9b640 1 -- 172.21.15.133:0/3757846158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da00039c0 msgr2=0x7f2da0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db5d9b640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da00039c0 0x7f2da0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.449+0000 7f2db5d9b640 1 -- 172.21.15.133:0/3757846158 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da0003530 con 0x7f2da0004810 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db5d9b640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 0x7f2da0004c90 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0098a0 tx=0x7f2dac01c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdc1a7cb9306e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00f040 con 0x7f2da0004810 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db00020c0 con 0x7f2da0004810 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db559a640 1 -- 172.21.15.133:0/3757846158 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac0a9840 con 0x7f2da0004810 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3757846158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 msgr2=0x7f2da0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 --2- 172.21.15.133:0/3757846158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 0x7f2da0004c90 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0098a0 tx=0x7f2dac01c610 comp rx=0 tx=0).stop 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db6d9d640 1 -- 172.21.15.133:0/3757846158 reap_dead start 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3757846158 shutdown_connections 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3757846158 >> 172.21.15.133:0/3757846158 conn(0x7f2db8066f70 msgr2=0x7f2db806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3757846158 shutdown_connections 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3757846158 wait complete. 2024-04-25T10:12:54.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 Processor -- start 2024-04-25T10:12:54.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 -- start start 2024-04-25T10:12:54.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 0x7f2db80a8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2db813d8a0 0x7f2db8141810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8141d50 0x7f2db81421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db815d5e0 con 0x7f2da0004810 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db815cb70 con 0x7f2db813d8a0 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db815d760 con 0x7f2db8141d50 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 0x7f2db80a8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 0x7f2db80a8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56798/0 (socket says 172.21.15.133:56798) 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db659c640 1 -- 172.21.15.133:0/3047074252 learned_addr learned my addr 172.21.15.133:0/3047074252 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db6d9d640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8141d50 0x7f2db81421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db5d9b640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2db813d8a0 0x7f2db8141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db659c640 1 -- 172.21.15.133:0/3047074252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8141d50 msgr2=0x7f2db81421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db659c640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8141d50 0x7f2db81421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db659c640 1 -- 172.21.15.133:0/3047074252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2db813d8a0 msgr2=0x7f2db8141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db659c640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2db813d8a0 0x7f2db8141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db659c640 1 -- 172.21.15.133:0/3047074252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db812f7e0 con 0x7f2da0004810 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db5d9b640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2db813d8a0 0x7f2db8141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2db659c640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 0x7f2db80a8880 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f2da8011610 tx=0x7f2da8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2da6ffd640 1 -- 172.21.15.133:0/3047074252 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da800f8f0 con 0x7f2da0004810 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2da6ffd640 1 -- 172.21.15.133:0/3047074252 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da800fa90 con 0x7f2da0004810 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3047074252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db813de60 con 0x7f2da0004810 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2da6ffd640 1 -- 172.21.15.133:0/3047074252 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da800fe60 con 0x7f2da0004810 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.453+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3047074252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db813e420 con 0x7f2da0004810 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.457+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3047074252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db80aef90 con 0x7f2da0004810 2024-04-25T10:12:54.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.457+0000 7f2da6ffd640 1 -- 172.21.15.133:0/3047074252 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2da8015460 con 0x7f2da0004810 2024-04-25T10:12:54.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.457+0000 7f2da6ffd640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d98066f90 0x7f2d98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.457+0000 7f2db5d9b640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d98066f90 0x7f2d98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.457+0000 7f2da6ffd640 1 -- 172.21.15.133:0/3047074252 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2da80859d0 con 0x7f2da0004810 2024-04-25T10:12:54.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.457+0000 7f2db5d9b640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d98066f90 0x7f2d98069450 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0ae000 tx=0x7f2dac0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.461+0000 7f2da6ffd640 1 -- 172.21.15.133:0/3047074252 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2da8019030 con 0x7f2da0004810 2024-04-25T10:12:54.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.617+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3047074252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2db8068e80 con 0x7f2da0004810 2024-04-25T10:12:54.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.617+0000 7f2da6ffd640 1 -- 172.21.15.133:0/3047074252 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2da80568e0 con 0x7f2da0004810 2024-04-25T10:12:54.620 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:12:54.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3047074252 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d98066f90 msgr2=0x7f2d98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d98066f90 0x7f2d98069450 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0ae000 tx=0x7f2dac0046d0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3047074252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 msgr2=0x7f2db80a8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 0x7f2db80a8880 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f2da8011610 tx=0x7f2da8011ad0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3047074252 shutdown_connections 2024-04-25T10:12:54.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2db8141d50 0x7f2db81421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2db813d8a0 0x7f2db8141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2d98066f90 0x7f2d98069450 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 --2- 172.21.15.133:0/3047074252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da0004810 0x7f2db80a8880 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3047074252 >> 172.21.15.133:0/3047074252 conn(0x7f2db8066f70 msgr2=0x7f2db8144400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3047074252 shutdown_connections 2024-04-25T10:12:54.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.621+0000 7f2dbc86b640 1 -- 172.21.15.133:0/3047074252 wait complete. 2024-04-25T10:12:54.640 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:12:54.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f6797d1d640 1 Processor -- start 2024-04-25T10:12:54.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f6797d1d640 1 -- start start 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f6797d1d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f679015d9a0 0x7f67900d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f6797d1d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67900ce600 0x7f67900cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f6797d1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67900d1920 0x7f67900d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f6797d1d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679015ebd0 con 0x7f67900ce600 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f6797d1d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679015eeb0 con 0x7f679015d9a0 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f6797d1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67900d0330 con 0x7f67900d1920 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f67952b8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67900ce600 0x7f67900cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f67952b8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67900ce600 0x7f67900cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56804/0 (socket says 172.21.15.133:56804) 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f67952b8640 1 -- 172.21.15.133:0/4273245842 learned_addr learned my addr 172.21.15.133:0/4273245842 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f6795ab9640 1 --2- 172.21.15.133:0/4273245842 >> v2:172.21.15.162:3300/0 conn(0x7f679015d9a0 0x7f67900d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f67962ba640 1 --2- 172.21.15.133:0/4273245842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67900d1920 0x7f67900d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f67952b8640 1 -- 172.21.15.133:0/4273245842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67900d1920 msgr2=0x7f67900d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f67952b8640 1 --2- 172.21.15.133:0/4273245842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67900d1920 0x7f67900d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f67952b8640 1 -- 172.21.15.133:0/4273245842 >> v2:172.21.15.162:3300/0 conn(0x7f679015d9a0 msgr2=0x7f67900d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f67952b8640 1 --2- 172.21.15.133:0/4273245842 >> v2:172.21.15.162:3300/0 conn(0x7f679015d9a0 0x7f67900d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f67952b8640 1 -- 172.21.15.133:0/4273245842 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67900d0610 con 0x7f67900ce600 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.757+0000 7f6795ab9640 1 --2- 172.21.15.133:0/4273245842 >> v2:172.21.15.162:3300/0 conn(0x7f679015d9a0 0x7f67900d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f67952b8640 1 --2- 172.21.15.133:0/4273245842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ce600 0x7f67900cea90 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f67900d4750 tx=0x7f678001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15c072e1c4a0489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6794ab7640 1 -- 172.21.15.133:0/4273245842 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67800b2070 con 0x7f67900ce600 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6794ab7640 1 -- 172.21.15.133:0/4273245842 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67800a9070 con 0x7f67900ce600 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6794ab7640 1 -- 172.21.15.133:0/4273245842 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67800ad880 con 0x7f67900ce600 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 -- 172.21.15.133:0/4273245842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ce600 msgr2=0x7f67900cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 --2- 172.21.15.133:0/4273245842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ce600 0x7f67900cea90 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f67900d4750 tx=0x7f678001c490 comp rx=0 tx=0).stop 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 -- 172.21.15.133:0/4273245842 shutdown_connections 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 --2- 172.21.15.133:0/4273245842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67900d1920 0x7f67900d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 --2- 172.21.15.133:0/4273245842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ce600 0x7f67900cea90 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 --2- 172.21.15.133:0/4273245842 >> v2:172.21.15.162:3300/0 conn(0x7f679015d9a0 0x7f67900d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 -- 172.21.15.133:0/4273245842 >> 172.21.15.133:0/4273245842 conn(0x7f679006bf80 msgr2=0x7f679006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 -- 172.21.15.133:0/4273245842 shutdown_connections 2024-04-25T10:12:54.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 -- 172.21.15.133:0/4273245842 wait complete. 2024-04-25T10:12:54.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 Processor -- start 2024-04-25T10:12:54.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 -- start start 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ce600 0x7f67901530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67900d1920 0x7f679015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f679015d9a0 0x7f67901542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67900d2470 con 0x7f67900ce600 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67900d22f0 con 0x7f67900d1920 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6797d1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679015e590 con 0x7f679015d9a0 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6795ab9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ce600 0x7f67901530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6795ab9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ce600 0x7f67901530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56810/0 (socket says 172.21.15.133:56810) 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6795ab9640 1 -- 172.21.15.133:0/4142316038 learned_addr learned my addr 172.21.15.133:0/4142316038 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f67952b8640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67900d1920 0x7f679015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f67962ba640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f679015d9a0 0x7f67901542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6795ab9640 1 -- 172.21.15.133:0/4142316038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f679015d9a0 msgr2=0x7f67901542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6795ab9640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f679015d9a0 0x7f67901542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6795ab9640 1 -- 172.21.15.133:0/4142316038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67900d1920 msgr2=0x7f679015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6795ab9640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67900d1920 0x7f679015a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f6795ab9640 1 -- 172.21.15.133:0/4142316038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6790155870 con 0x7f67900ce600 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.761+0000 7f67962ba640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f679015d9a0 0x7f67901542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f6795ab9640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ce600 0x7f67901530d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c7f0 tx=0x7f678c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f67867fc640 1 -- 172.21.15.133:0/4142316038 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c015070 con 0x7f67900ce600 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f67867fc640 1 -- 172.21.15.133:0/4142316038 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c007820 con 0x7f67900ce600 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f6797d1d640 1 -- 172.21.15.133:0/4142316038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6790153810 con 0x7f67900ce600 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f67867fc640 1 -- 172.21.15.133:0/4142316038 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c007c10 con 0x7f67900ce600 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f6797d1d640 1 -- 172.21.15.133:0/4142316038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6790155b50 con 0x7f67900ce600 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f6797d1d640 1 -- 172.21.15.133:0/4142316038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6790068e80 con 0x7f67900ce600 2024-04-25T10:12:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f67867fc640 1 -- 172.21.15.133:0/4142316038 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f678c018070 con 0x7f67900ce600 2024-04-25T10:12:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f67867fc640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6774066f60 0x7f6774069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f67952b8640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6774066f60 0x7f6774069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f67867fc640 1 -- 172.21.15.133:0/4142316038 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f678c085240 con 0x7f67900ce600 2024-04-25T10:12:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.765+0000 7f67952b8640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6774066f60 0x7f6774069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f67800af000 tx=0x7f6780008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.769+0000 7f67867fc640 1 -- 172.21.15.133:0/4142316038 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f678c056150 con 0x7f67900ce600 2024-04-25T10:12:54.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.933+0000 7f6797d1d640 1 -- 172.21.15.133:0/4142316038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f67900d0980 con 0x7f67900ce600 2024-04-25T10:12:54.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f67867fc640 1 -- 172.21.15.133:0/4142316038 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v120) v1 ==== 80+0+0 (secure 0 0 0) 0x7f678c055a50 con 0x7f67900ce600 2024-04-25T10:12:54.937 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 -- 172.21.15.133:0/4142316038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6774066f60 msgr2=0x7f6774069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6774066f60 0x7f6774069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f67800af000 tx=0x7f6780008040 comp rx=0 tx=0).stop 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 -- 172.21.15.133:0/4142316038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ce600 msgr2=0x7f67901530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ce600 0x7f67901530d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c7f0 tx=0x7f678c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 -- 172.21.15.133:0/4142316038 shutdown_connections 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f679015d9a0 0x7f67901542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67900d1920 0x7f679015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6774066f60 0x7f6774069420 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 --2- 172.21.15.133:0/4142316038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ce600 0x7f67901530d0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 -- 172.21.15.133:0/4142316038 >> 172.21.15.133:0/4142316038 conn(0x7f679006bf80 msgr2=0x7f6790168060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 -- 172.21.15.133:0/4142316038 shutdown_connections 2024-04-25T10:12:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:54.937+0000 7f6797d1d640 1 -- 172.21.15.133:0/4142316038 wait complete. 2024-04-25T10:12:54.955 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:12:54.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:54.956 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:54.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[78255]: fuse finished with error 0 and tester_r 0 2024-04-25T10:12:55.031 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:12:55.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.031 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:12:55.122 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:12:55.122 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:55.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:55.128 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:12:55.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.128 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:12:55.176 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:12:55.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:55.177 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:12:55.178 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:55.183 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:55.183 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:12:55.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.183 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:12:55.229 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:12:55.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:55.230 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:12:55.230 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:55.236 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:55.236 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:12:55.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.236 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:12:55.284 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:12:55.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:55.286 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:55.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.401+0000 7f8ebd6b0640 1 Processor -- start 2024-04-25T10:12:55.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8ebd6b0640 1 -- start start 2024-04-25T10:12:55.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8ebd6b0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8eb80cdf30 0x7f8eb806d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8ebd6b0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8eb80ce310 0x7f8eb80cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8ebd6b0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb806de10 con 0x7f8eb806d890 2024-04-25T10:12:55.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8ebd6b0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb806ea20 con 0x7f8eb80ce310 2024-04-25T10:12:55.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8ebd6b0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb806e0f0 con 0x7f8eb80cdf30 2024-04-25T10:12:55.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb67fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8eb80ce310 0x7f8eb80cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb67fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8eb80ce310 0x7f8eb80cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56818/0 (socket says 172.21.15.133:56818) 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb67fc640 1 -- 172.21.15.133:0/3181352453 learned_addr learned my addr 172.21.15.133:0/3181352453 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb6ffd640 1 --2- 172.21.15.133:0/3181352453 >> v2:172.21.15.162:3300/0 conn(0x7f8eb80cdf30 0x7f8eb806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb67fc640 1 -- 172.21.15.133:0/3181352453 >> v2:172.21.15.162:3300/0 conn(0x7f8eb80cdf30 msgr2=0x7f8eb806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb67fc640 1 --2- 172.21.15.133:0/3181352453 >> v2:172.21.15.162:3300/0 conn(0x7f8eb80cdf30 0x7f8eb806d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb67fc640 1 -- 172.21.15.133:0/3181352453 >> v1:172.21.15.162:6789/0 conn(0x7f8eb806d890 legacy=0x7f8eb815d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb67fc640 1 -- 172.21.15.133:0/3181352453 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb806e3d0 con 0x7f8eb80ce310 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb67fc640 1 --2- 172.21.15.133:0/3181352453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb80ce310 0x7f8eb80cb640 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8072670 tx=0x7f8eac01cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bd5e3297f2d737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb5ffb640 1 -- 172.21.15.133:0/3181352453 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0aa430 con 0x7f8eb80ce310 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb5ffb640 1 -- 172.21.15.133:0/3181352453 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eac0af070 con 0x7f8eb80ce310 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.405+0000 7f8eb5ffb640 1 -- 172.21.15.133:0/3181352453 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0aa7f0 con 0x7f8eb80ce310 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/3181352453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb80ce310 msgr2=0x7f8eb80cb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 --2- 172.21.15.133:0/3181352453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb80ce310 0x7f8eb80cb640 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8072670 tx=0x7f8eac01cf10 comp rx=0 tx=0).stop 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/3181352453 shutdown_connections 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 --2- 172.21.15.133:0/3181352453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb80ce310 0x7f8eb80cb640 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 --2- 172.21.15.133:0/3181352453 >> v2:172.21.15.162:3300/0 conn(0x7f8eb80cdf30 0x7f8eb806d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/3181352453 >> 172.21.15.133:0/3181352453 conn(0x7f8eb8067000 msgr2=0x7f8eb806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/3181352453 shutdown_connections 2024-04-25T10:12:55.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/3181352453 wait complete. 2024-04-25T10:12:55.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 Processor -- start 2024-04-25T10:12:55.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 -- start start 2024-04-25T10:12:55.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb806d890 0x7f8eb80ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb80cdf30 0x7f8eb80c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8eb80ce310 0x7f8eb80c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80cbf10 con 0x7f8eb806d890 2024-04-25T10:12:55.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80cbd90 con 0x7f8eb80ce310 2024-04-25T10:12:55.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb81647b0 con 0x7f8eb80cdf30 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb806d890 0x7f8eb80ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb806d890 0x7f8eb80ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56834/0 (socket says 172.21.15.133:56834) 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb6ffd640 1 -- 172.21.15.133:0/4242970380 learned_addr learned my addr 172.21.15.133:0/4242970380 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb67fc640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb80cdf30 0x7f8eb80c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb77fe640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8eb80ce310 0x7f8eb80c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb6ffd640 1 -- 172.21.15.133:0/4242970380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb80cdf30 msgr2=0x7f8eb80c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb6ffd640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb80cdf30 0x7f8eb80c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb6ffd640 1 -- 172.21.15.133:0/4242970380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8eb80ce310 msgr2=0x7f8eb80c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb6ffd640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8eb80ce310 0x7f8eb80c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb6ffd640 1 -- 172.21.15.133:0/4242970380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb80c90c0 con 0x7f8eb806d890 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb67fc640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb80cdf30 0x7f8eb80c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8eb6ffd640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb806d890 0x7f8eb80ca730 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c7f0 tx=0x7f8ea800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8e9f7fe640 1 -- 172.21.15.133:0/4242970380 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8015070 con 0x7f8eb806d890 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8e9f7fe640 1 -- 172.21.15.133:0/4242970380 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea8004240 con 0x7f8eb806d890 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/4242970380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb80c93a0 con 0x7f8eb806d890 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8e9f7fe640 1 -- 172.21.15.133:0/4242970380 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8003620 con 0x7f8eb806d890 2024-04-25T10:12:55.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.409+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/4242970380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb80cac70 con 0x7f8eb806d890 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.413+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/4242970380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb80c9850 con 0x7f8eb806d890 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.413+0000 7f8e9f7fe640 1 -- 172.21.15.133:0/4242970380 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ea80039d0 con 0x7f8eb806d890 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.413+0000 7f8e9f7fe640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ea4066f60 0x7f8ea4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.413+0000 7f8eb67fc640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ea4066f60 0x7f8ea4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.413+0000 7f8e9f7fe640 1 -- 172.21.15.133:0/4242970380 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ea8085290 con 0x7f8eb806d890 2024-04-25T10:12:55.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.413+0000 7f8eb67fc640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ea4066f60 0x7f8ea4069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0ae000 tx=0x7f8eac004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.417+0000 7f8e9f7fe640 1 -- 172.21.15.133:0/4242970380 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ea8056110 con 0x7f8eb806d890 2024-04-25T10:12:55.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/4242970380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8eb8068ef0 con 0x7f8eb806d890 2024-04-25T10:12:55.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8e9f7fe640 1 -- 172.21.15.133:0/4242970380 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f8ea8055a10 con 0x7f8eb806d890 2024-04-25T10:12:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:55.599 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:55.600 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:55.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/4242970380 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ea4066f60 msgr2=0x7f8ea4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8ebd6b0640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ea4066f60 0x7f8ea4069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0ae000 tx=0x7f8eac004740 comp rx=0 tx=0).stop 2024-04-25T10:12:55.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/4242970380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb806d890 msgr2=0x7f8eb80ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8ebd6b0640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb806d890 0x7f8eb80ca730 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c7f0 tx=0x7f8ea800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/4242970380 shutdown_connections 2024-04-25T10:12:55.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8ebd6b0640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8eb80ce310 0x7f8eb80c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8ebd6b0640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8eb80cdf30 0x7f8eb80c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8ebd6b0640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8ea4066f60 0x7f8ea4069420 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8ebd6b0640 1 --2- 172.21.15.133:0/4242970380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb806d890 0x7f8eb80ca730 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.597+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/4242970380 >> 172.21.15.133:0/4242970380 conn(0x7f8eb8067000 msgr2=0x7f8eb80b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.601+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/4242970380 shutdown_connections 2024-04-25T10:12:55.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.601+0000 7f8ebd6b0640 1 -- 172.21.15.133:0/4242970380 wait complete. 2024-04-25T10:12:55.617 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc478361640 1 Processor -- start 2024-04-25T10:12:55.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc478361640 1 -- start start 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc478361640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc470158110 0x7fc4700b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc478361640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc4701584f0 0x7fc47015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc478361640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4700cbe60 con 0x7fc47006e460 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc478361640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4700b19b0 con 0x7fc470158110 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc478361640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47015b360 con 0x7fc4701584f0 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc4760fd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc470158110 0x7fc4700b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc4760fd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc470158110 0x7fc4700b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56852/0 (socket says 172.21.15.133:56852) 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc4760fd640 1 -- 172.21.15.133:0/1739491703 learned_addr learned my addr 172.21.15.133:0/1739491703 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc4760fd640 1 -- 172.21.15.133:0/1739491703 >> v2:172.21.15.162:3300/0 conn(0x7fc4701584f0 msgr2=0x7fc47015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc4760fd640 1 --2- 172.21.15.133:0/1739491703 >> v2:172.21.15.162:3300/0 conn(0x7fc4701584f0 0x7fc47015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc4760fd640 1 -- 172.21.15.133:0/1739491703 >> v1:172.21.15.162:6789/0 conn(0x7fc47006e460 legacy=0x7fc47015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:55.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.733+0000 7fc4760fd640 1 -- 172.21.15.133:0/1739491703 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47015b640 con 0x7fc470158110 2024-04-25T10:12:55.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4760fd640 1 --2- 172.21.15.133:0/1739491703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc470158110 0x7fc4700b1f00 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc4700b4da0 tx=0x7fc46c01ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783e47b4a0b41ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4750fb640 1 -- 172.21.15.133:0/1739491703 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0b4070 con 0x7fc470158110 2024-04-25T10:12:55.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4750fb640 1 -- 172.21.15.133:0/1739491703 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c0a7bb0 con 0x7fc470158110 2024-04-25T10:12:55.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4750fb640 1 -- 172.21.15.133:0/1739491703 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0af440 con 0x7fc470158110 2024-04-25T10:12:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 -- 172.21.15.133:0/1739491703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc470158110 msgr2=0x7fc4700b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 --2- 172.21.15.133:0/1739491703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc470158110 0x7fc4700b1f00 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc4700b4da0 tx=0x7fc46c01ebd0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 -- 172.21.15.133:0/1739491703 shutdown_connections 2024-04-25T10:12:55.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 --2- 172.21.15.133:0/1739491703 >> v2:172.21.15.162:3300/0 conn(0x7fc4701584f0 0x7fc47015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 --2- 172.21.15.133:0/1739491703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc470158110 0x7fc4700b1f00 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 -- 172.21.15.133:0/1739491703 >> 172.21.15.133:0/1739491703 conn(0x7fc470067000 msgr2=0x7fc47006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 -- 172.21.15.133:0/1739491703 shutdown_connections 2024-04-25T10:12:55.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 -- 172.21.15.133:0/1739491703 wait complete. 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 Processor -- start 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 -- start start 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47006e460 0x7fc470157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc470158110 0x7fc470150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4701584f0 0x7fc470152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700cd7f0 con 0x7fc47006e460 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700cd670 con 0x7fc4701584f0 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc478361640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47006d350 con 0x7fc470158110 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4760fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47006e460 0x7fc470157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4760fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47006e460 0x7fc470157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56856/0 (socket says 172.21.15.133:56856) 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4760fd640 1 -- 172.21.15.133:0/932192763 learned_addr learned my addr 172.21.15.133:0/932192763 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4760fd640 1 -- 172.21.15.133:0/932192763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc470158110 msgr2=0x7fc470150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4760fd640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc470158110 0x7fc470150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4760fd640 1 -- 172.21.15.133:0/932192763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4701584f0 msgr2=0x7fc470152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4760fd640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4701584f0 0x7fc470152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.737+0000 7fc4760fd640 1 -- 172.21.15.133:0/932192763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4700b4010 con 0x7fc47006e460 2024-04-25T10:12:55.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc4760fd640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47006e460 0x7fc470157350 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00b2c0 tx=0x7fc46c01b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc466ffd640 1 -- 172.21.15.133:0/932192763 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0b4040 con 0x7fc47006e460 2024-04-25T10:12:55.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc466ffd640 1 -- 172.21.15.133:0/932192763 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c0c1070 con 0x7fc47006e460 2024-04-25T10:12:55.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc478361640 1 -- 172.21.15.133:0/932192763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4700b42f0 con 0x7fc47006e460 2024-04-25T10:12:55.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc466ffd640 1 -- 172.21.15.133:0/932192763 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0b9460 con 0x7fc47006e460 2024-04-25T10:12:55.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc478361640 1 -- 172.21.15.133:0/932192763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc470155cf0 con 0x7fc47006e460 2024-04-25T10:12:55.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc478361640 1 -- 172.21.15.133:0/932192763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47006bc20 con 0x7fc47006e460 2024-04-25T10:12:55.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc466ffd640 1 -- 172.21.15.133:0/932192763 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc46c0b7070 con 0x7fc47006e460 2024-04-25T10:12:55.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc466ffd640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc454066f60 0x7fc454069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc466ffd640 1 -- 172.21.15.133:0/932192763 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc46c122340 con 0x7fc47006e460 2024-04-25T10:12:55.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc4758fc640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc454066f60 0x7fc454069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.741+0000 7fc4758fc640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc454066f60 0x7fc454069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc4701f6050 tx=0x7fc460006c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.745+0000 7fc466ffd640 1 -- 172.21.15.133:0/932192763 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc46c0187c0 con 0x7fc47006e460 2024-04-25T10:12:55.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.893+0000 7fc478361640 1 -- 172.21.15.133:0/932192763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc470068ef0 con 0x7fc47006e460 2024-04-25T10:12:55.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc466ffd640 1 -- 172.21.15.133:0/932192763 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fc46c0ad340 con 0x7fc47006e460 2024-04-25T10:12:55.898 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:55.899 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":120,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:12:19.584218+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:16.197174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:16.904937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:55.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 -- 172.21.15.133:0/932192763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc454066f60 msgr2=0x7fc454069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc454066f60 0x7fc454069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc4701f6050 tx=0x7fc460006c00 comp rx=0 tx=0).stop 2024-04-25T10:12:55.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 -- 172.21.15.133:0/932192763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47006e460 msgr2=0x7fc470157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47006e460 0x7fc470157350 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00b2c0 tx=0x7fc46c01b970 comp rx=0 tx=0).stop 2024-04-25T10:12:55.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 -- 172.21.15.133:0/932192763 shutdown_connections 2024-04-25T10:12:55.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4701584f0 0x7fc470152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc470158110 0x7fc470150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc454066f60 0x7fc454069420 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 --2- 172.21.15.133:0/932192763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47006e460 0x7fc470157350 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 -- 172.21.15.133:0/932192763 >> 172.21.15.133:0/932192763 conn(0x7fc470067000 msgr2=0x7fc4700cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 -- 172.21.15.133:0/932192763 shutdown_connections 2024-04-25T10:12:55.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:55.897+0000 7fc478361640 1 -- 172.21.15.133:0/932192763 wait complete. 2024-04-25T10:12:55.916 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:16.197174+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:16.904937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:55.916 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab62e8b640 1 Processor -- start 2024-04-25T10:12:56.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab62e8b640 1 -- start start 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab62e8b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab5c0ce600 0x7fab5c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab62e8b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab5c0d1920 0x7fab5c0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab62e8b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5c15ebd0 con 0x7fab5c15d9a0 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab62e8b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5c15eeb0 con 0x7fab5c0ce600 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab62e8b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0d0330 con 0x7fab5c0d1920 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab5bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab5c0ce600 0x7fab5c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab5bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab5c0ce600 0x7fab5c0cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56858/0 (socket says 172.21.15.133:56858) 2024-04-25T10:12:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab5bfff640 1 -- 172.21.15.133:0/238119688 learned_addr learned my addr 172.21.15.133:0/238119688 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab61428640 1 --2- 172.21.15.133:0/238119688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab5c0d1920 0x7fab5c0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab5bfff640 1 -- 172.21.15.133:0/238119688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab5c0d1920 msgr2=0x7fab5c0d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab5bfff640 1 --2- 172.21.15.133:0/238119688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab5c0d1920 0x7fab5c0d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab5bfff640 1 -- 172.21.15.133:0/238119688 >> v1:172.21.15.162:6789/0 conn(0x7fab5c15d9a0 legacy=0x7fab5c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.037+0000 7fab5bfff640 1 -- 172.21.15.133:0/238119688 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0d0610 con 0x7fab5c0ce600 2024-04-25T10:12:56.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab5bfff640 1 --2- 172.21.15.133:0/238119688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5c0ce600 0x7fab5c0cea90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fab50009870 tx=0x7fab5001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e81ad96ad51da30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab5b7fe640 1 -- 172.21.15.133:0/238119688 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab500b0070 con 0x7fab5c0ce600 2024-04-25T10:12:56.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab5b7fe640 1 -- 172.21.15.133:0/238119688 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab500ab600 con 0x7fab5c0ce600 2024-04-25T10:12:56.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab5b7fe640 1 -- 172.21.15.133:0/238119688 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab500ab9d0 con 0x7fab5c0ce600 2024-04-25T10:12:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab62e8b640 1 -- 172.21.15.133:0/238119688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5c0ce600 msgr2=0x7fab5c0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab62e8b640 1 --2- 172.21.15.133:0/238119688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5c0ce600 0x7fab5c0cea90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fab50009870 tx=0x7fab5001c4d0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab62e8b640 1 -- 172.21.15.133:0/238119688 shutdown_connections 2024-04-25T10:12:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab62e8b640 1 --2- 172.21.15.133:0/238119688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab5c0d1920 0x7fab5c0d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab62e8b640 1 --2- 172.21.15.133:0/238119688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5c0ce600 0x7fab5c0cea90 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab62e8b640 1 -- 172.21.15.133:0/238119688 >> 172.21.15.133:0/238119688 conn(0x7fab5c06bf80 msgr2=0x7fab5c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab62e8b640 1 -- 172.21.15.133:0/238119688 shutdown_connections 2024-04-25T10:12:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab62e8b640 1 -- 172.21.15.133:0/238119688 wait complete. 2024-04-25T10:12:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab62e8b640 1 Processor -- start 2024-04-25T10:12:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.041+0000 7fab62e8b640 1 -- start start 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab62e8b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5c0ce600 0x7fab5c158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab62e8b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab5c0d1920 0x7fab5c14dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab62e8b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5c15d9a0 0x7fab5c14e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab62e8b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c15e590 con 0x7fab5c15d9a0 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab62e8b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0d2540 con 0x7fab5c0ce600 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab62e8b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c15e710 con 0x7fab5c0d1920 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab61428640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5c15d9a0 0x7fab5c14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab61428640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5c15d9a0 0x7fab5c14e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56872/0 (socket says 172.21.15.133:56872) 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab61428640 1 -- 172.21.15.133:0/2658887467 learned_addr learned my addr 172.21.15.133:0/2658887467 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab5bfff640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab5c0d1920 0x7fab5c14dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab60c27640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5c0ce600 0x7fab5c158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab61428640 1 -- 172.21.15.133:0/2658887467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab5c0d1920 msgr2=0x7fab5c14dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab61428640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab5c0d1920 0x7fab5c14dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab61428640 1 -- 172.21.15.133:0/2658887467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5c0ce600 msgr2=0x7fab5c158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab61428640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5c0ce600 0x7fab5c158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab61428640 1 -- 172.21.15.133:0/2658887467 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c14dad0 con 0x7fab5c15d9a0 2024-04-25T10:12:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab61428640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5c15d9a0 0x7fab5c14e510 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fab54007e60 tx=0x7fab5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab597fa640 1 -- 172.21.15.133:0/2658887467 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54014420 con 0x7fab5c15d9a0 2024-04-25T10:12:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab597fa640 1 -- 172.21.15.133:0/2658887467 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab54019070 con 0x7fab5c15d9a0 2024-04-25T10:12:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab62e8b640 1 -- 172.21.15.133:0/2658887467 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c157450 con 0x7fab5c15d9a0 2024-04-25T10:12:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab597fa640 1 -- 172.21.15.133:0/2658887467 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54014820 con 0x7fab5c15d9a0 2024-04-25T10:12:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab62e8b640 1 -- 172.21.15.133:0/2658887467 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c157a10 con 0x7fab5c15d9a0 2024-04-25T10:12:56.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab62e8b640 1 -- 172.21.15.133:0/2658887467 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c14dca0 con 0x7fab5c15d9a0 2024-04-25T10:12:56.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab597fa640 1 -- 172.21.15.133:0/2658887467 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab540149c0 con 0x7fab5c15d9a0 2024-04-25T10:12:56.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab597fa640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 0x7fab48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab597fa640 1 -- 172.21.15.133:0/2658887467 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab54085940 con 0x7fab5c15d9a0 2024-04-25T10:12:56.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab60c27640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 0x7fab48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.045+0000 7fab60c27640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 0x7fab48069450 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fab4c007fd0 tx=0x7fab4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.049+0000 7fab597fa640 1 -- 172.21.15.133:0/2658887467 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab54056850 con 0x7fab5c15d9a0 2024-04-25T10:12:56.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.212+0000 7fab62e8b640 1 -- 172.21.15.133:0/2658887467 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab5c068e80 con 0x7fab5c15d9a0 2024-04-25T10:12:56.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.212+0000 7fab597fa640 1 -- 172.21.15.133:0/2658887467 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab54056150 con 0x7fab5c15d9a0 2024-04-25T10:12:56.216 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 -- 172.21.15.133:0/2658887467 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 msgr2=0x7fab48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 0x7fab48069450 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fab4c007fd0 tx=0x7fab4c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 -- 172.21.15.133:0/2658887467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5c15d9a0 msgr2=0x7fab5c14e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5c15d9a0 0x7fab5c14e510 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fab54007e60 tx=0x7fab5400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 -- 172.21.15.133:0/2658887467 shutdown_connections 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab48066f90 0x7fab48069450 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5c15d9a0 0x7fab5c14e510 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fab5c0d1920 0x7fab5c14dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 --2- 172.21.15.133:0/2658887467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab5c0ce600 0x7fab5c158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 -- 172.21.15.133:0/2658887467 >> 172.21.15.133:0/2658887467 conn(0x7fab5c06bf80 msgr2=0x7fab5c0b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 -- 172.21.15.133:0/2658887467 shutdown_connections 2024-04-25T10:12:56.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.216+0000 7fab62e8b640 1 -- 172.21.15.133:0/2658887467 wait complete. 2024-04-25T10:12:56.235 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:56.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.348+0000 7f13cce91640 1 Processor -- start 2024-04-25T10:12:56.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.348+0000 7f13cce91640 1 -- start start 2024-04-25T10:12:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13cce91640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f13c8143ea0 0x7f13c80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13cce91640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c815ac50 con 0x7f13c80b2150 2024-04-25T10:12:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13cce91640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80cd180 con 0x7f13c806eac0 2024-04-25T10:12:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13cce91640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c815aed0 con 0x7f13c8143ea0 2024-04-25T10:12:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13c659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f13c8143ea0 0x7f13c80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13c5d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f13c80b2150 0x7f13c806e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35282/0 (socket says 172.21.15.133:35282) 2024-04-25T10:12:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13c5d9b640 1 -- 172.21.15.133:0/2545058725 learned_addr learned my addr 172.21.15.133:0/2545058725 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13c659c640 1 -- 172.21.15.133:0/2545058725 >> v1:172.21.15.162:6789/0 conn(0x7f13c806eac0 legacy=0x7f13c80b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13c659c640 1 -- 172.21.15.133:0/2545058725 >> v1:172.21.15.133:6789/0 conn(0x7f13c80b2150 legacy=0x7f13c806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13c659c640 1 -- 172.21.15.133:0/2545058725 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c815b1b0 con 0x7f13c8143ea0 2024-04-25T10:12:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13c559a640 1 -- 172.21.15.133:0/2545058725 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842219556 0 0) 0x7f13bc001120 con 0x7f13c80b2150 2024-04-25T10:12:56.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13c659c640 1 --2- 172.21.15.133:0/2545058725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c8143ea0 0x7f13c80cc810 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f13b80097c0 tx=0x7f13b801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820113f2a1872c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13c559a640 1 -- 172.21.15.133:0/2545058725 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b80b0070 con 0x7f13c8143ea0 2024-04-25T10:12:56.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13c559a640 1 -- 172.21.15.133:0/2545058725 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b80ab600 con 0x7f13c8143ea0 2024-04-25T10:12:56.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.352+0000 7f13c559a640 1 -- 172.21.15.133:0/2545058725 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b80ab9d0 con 0x7f13c8143ea0 2024-04-25T10:12:56.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 -- 172.21.15.133:0/2545058725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c8143ea0 msgr2=0x7f13c80cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 --2- 172.21.15.133:0/2545058725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c8143ea0 0x7f13c80cc810 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f13b80097c0 tx=0x7f13b801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 -- 172.21.15.133:0/2545058725 shutdown_connections 2024-04-25T10:12:56.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 --2- 172.21.15.133:0/2545058725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c8143ea0 0x7f13c80cc810 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 -- 172.21.15.133:0/2545058725 >> 172.21.15.133:0/2545058725 conn(0x7f13c8066f70 msgr2=0x7f13c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 -- 172.21.15.133:0/2545058725 shutdown_connections 2024-04-25T10:12:56.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 -- 172.21.15.133:0/2545058725 wait complete. 2024-04-25T10:12:56.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 Processor -- start 2024-04-25T10:12:56.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 -- start start 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13c806eac0 0x7f13c81552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13c80b2150 0x7f13c8158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c8143ea0 0x7f13c8156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8168770 con 0x7f13c8143ea0 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c81685f0 con 0x7f13c80b2150 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c81688f0 con 0x7f13c806eac0 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13c6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c8143ea0 0x7f13c8156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13c6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c8143ea0 0x7f13c8156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52480/0 (socket says 172.21.15.133:52480) 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13c6d9d640 1 -- 172.21.15.133:0/1282578008 learned_addr learned my addr 172.21.15.133:0/1282578008 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13c6d9d640 1 -- 172.21.15.133:0/1282578008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13c806eac0 msgr2=0x7f13c81552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13c6d9d640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13c806eac0 0x7f13c81552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13c6d9d640 1 -- 172.21.15.133:0/1282578008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13c80b2150 msgr2=0x7f13c8158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13c6d9d640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13c80b2150 0x7f13c8158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13c6d9d640 1 -- 172.21.15.133:0/1282578008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c8154f70 con 0x7f13c8143ea0 2024-04-25T10:12:56.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13c6d9d640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c8143ea0 0x7f13c8156b80 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f13c000c780 tx=0x7f13c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13b6ffd640 1 -- 172.21.15.133:0/1282578008 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0017070 con 0x7f13c8143ea0 2024-04-25T10:12:56.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13b6ffd640 1 -- 172.21.15.133:0/1282578008 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c0003db0 con 0x7f13c8143ea0 2024-04-25T10:12:56.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 -- 172.21.15.133:0/1282578008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c8155ae0 con 0x7f13c8143ea0 2024-04-25T10:12:56.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13b6ffd640 1 -- 172.21.15.133:0/1282578008 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00052b0 con 0x7f13c8143ea0 2024-04-25T10:12:56.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.356+0000 7f13cce91640 1 -- 172.21.15.133:0/1282578008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c81560a0 con 0x7f13c8143ea0 2024-04-25T10:12:56.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.360+0000 7f13cce91640 1 -- 172.21.15.133:0/1282578008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c8068e80 con 0x7f13c8143ea0 2024-04-25T10:12:56.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.360+0000 7f13b6ffd640 1 -- 172.21.15.133:0/1282578008 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13c001a070 con 0x7f13c8143ea0 2024-04-25T10:12:56.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.360+0000 7f13b6ffd640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13a8066f60 0x7f13a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.360+0000 7f13b6ffd640 1 -- 172.21.15.133:0/1282578008 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13c00847b0 con 0x7f13c8143ea0 2024-04-25T10:12:56.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.360+0000 7f13c659c640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13a8066f60 0x7f13a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.360+0000 7f13c659c640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13a8066f60 0x7f13a8069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f13b80ad000 tx=0x7f13b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.364+0000 7f13b6ffd640 1 -- 172.21.15.133:0/1282578008 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13c00556c0 con 0x7f13c8143ea0 2024-04-25T10:12:56.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.508+0000 7f13cce91640 1 -- 172.21.15.133:0/1282578008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13c806c2d0 con 0x7f13c8143ea0 2024-04-25T10:12:56.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13b6ffd640 1 -- 172.21.15.133:0/1282578008 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13c0054fc0 con 0x7f13c8143ea0 2024-04-25T10:12:56.514 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:12:56.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 -- 172.21.15.133:0/1282578008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13a8066f60 msgr2=0x7f13a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13a8066f60 0x7f13a8069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f13b80ad000 tx=0x7f13b8008040 comp rx=0 tx=0).stop 2024-04-25T10:12:56.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 -- 172.21.15.133:0/1282578008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c8143ea0 msgr2=0x7f13c8156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c8143ea0 0x7f13c8156b80 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f13c000c780 tx=0x7f13c000cc40 comp rx=0 tx=0).stop 2024-04-25T10:12:56.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 -- 172.21.15.133:0/1282578008 shutdown_connections 2024-04-25T10:12:56.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f13a8066f60 0x7f13a8069420 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13c8143ea0 0x7f13c8156b80 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f13c80b2150 0x7f13c8158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 --2- 172.21.15.133:0/1282578008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f13c806eac0 0x7f13c81552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 -- 172.21.15.133:0/1282578008 >> 172.21.15.133:0/1282578008 conn(0x7f13c8066f70 msgr2=0x7f13c806db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 -- 172.21.15.133:0/1282578008 shutdown_connections 2024-04-25T10:12:56.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.512+0000 7f13cce91640 1 -- 172.21.15.133:0/1282578008 wait complete. 2024-04-25T10:12:56.532 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:56.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.640+0000 7f073267f640 1 Processor -- start 2024-04-25T10:12:56.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.640+0000 7f073267f640 1 -- start start 2024-04-25T10:12:56.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073267f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f072c0b5110 0x7f072c0cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073267f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072c0d27b0 con 0x7f072c15f680 2024-04-25T10:12:56.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073267f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072c0d1ca0 con 0x7f072c0b54f0 2024-04-25T10:12:56.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073267f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072c0d1fa0 con 0x7f072c0b5110 2024-04-25T10:12:56.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073167d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f072c0b5110 0x7f072c0cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f0731e7e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f072c15f680 0x7f072c15fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35296/0 (socket says 172.21.15.133:35296) 2024-04-25T10:12:56.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f0731e7e640 1 -- 172.21.15.133:0/197822399 learned_addr learned my addr 172.21.15.133:0/197822399 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:56.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073167d640 1 -- 172.21.15.133:0/197822399 >> v1:172.21.15.162:6789/0 conn(0x7f072c0b54f0 legacy=0x7f072c0d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:56.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073167d640 1 -- 172.21.15.133:0/197822399 >> v1:172.21.15.133:6789/0 conn(0x7f072c15f680 legacy=0x7f072c15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073167d640 1 -- 172.21.15.133:0/197822399 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0b28e0 con 0x7f072c0b5110 2024-04-25T10:12:56.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073167d640 1 --2- 172.21.15.133:0/197822399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f072c0b5110 0x7f072c0cf700 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f07280097a0 tx=0x7f072801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca38558f48175c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f0723fff640 1 -- 172.21.15.133:0/197822399 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07280b0070 con 0x7f072c0b5110 2024-04-25T10:12:56.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f0723fff640 1 -- 172.21.15.133:0/197822399 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07280ab600 con 0x7f072c0b5110 2024-04-25T10:12:56.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f0723fff640 1 -- 172.21.15.133:0/197822399 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07280ab9d0 con 0x7f072c0b5110 2024-04-25T10:12:56.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073267f640 1 -- 172.21.15.133:0/197822399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f072c0b5110 msgr2=0x7f072c0cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073267f640 1 --2- 172.21.15.133:0/197822399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f072c0b5110 0x7f072c0cf700 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f07280097a0 tx=0x7f072801c3b0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073267f640 1 -- 172.21.15.133:0/197822399 shutdown_connections 2024-04-25T10:12:56.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073267f640 1 --2- 172.21.15.133:0/197822399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f072c0b5110 0x7f072c0cf700 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073267f640 1 -- 172.21.15.133:0/197822399 >> 172.21.15.133:0/197822399 conn(0x7f072c08af60 msgr2=0x7f072c0d3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073267f640 1 -- 172.21.15.133:0/197822399 shutdown_connections 2024-04-25T10:12:56.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.644+0000 7f073267f640 1 -- 172.21.15.133:0/197822399 wait complete. 2024-04-25T10:12:56.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f073267f640 1 Processor -- start 2024-04-25T10:12:56.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f073267f640 1 -- start start 2024-04-25T10:12:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f073267f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f072c0b5110 0x7f072c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f073267f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f072c0b54f0 0x7f072c0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f073267f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f072c15f680 0x7f072c06be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f073267f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c160220 con 0x7f072c0b54f0 2024-04-25T10:12:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f073267f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0b5950 con 0x7f072c15f680 2024-04-25T10:12:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f073267f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1603a0 con 0x7f072c0b5110 2024-04-25T10:12:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0730e7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f072c0b54f0 0x7f072c0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0730e7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f072c0b54f0 0x7f072c0c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52496/0 (socket says 172.21.15.133:52496) 2024-04-25T10:12:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0730e7c640 1 -- 172.21.15.133:0/3344311181 learned_addr learned my addr 172.21.15.133:0/3344311181 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0730e7c640 1 -- 172.21.15.133:0/3344311181 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f072c0b5110 msgr2=0x7f072c1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0730e7c640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f072c0b5110 0x7f072c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0730e7c640 1 -- 172.21.15.133:0/3344311181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f072c15f680 msgr2=0x7f072c06be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0731e7e640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f072c15f680 0x7f072c06be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0730e7c640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f072c15f680 0x7f072c06be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0730e7c640 1 -- 172.21.15.133:0/3344311181 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0c8ea0 con 0x7f072c0b54f0 2024-04-25T10:12:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0730e7c640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f072c0b54f0 0x7f072c0c7e60 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f071c00b410 tx=0x7f071c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0721ffb640 1 -- 172.21.15.133:0/3344311181 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c015030 con 0x7f072c0b54f0 2024-04-25T10:12:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0721ffb640 1 -- 172.21.15.133:0/3344311181 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c004020 con 0x7f072c0b54f0 2024-04-25T10:12:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f073267f640 1 -- 172.21.15.133:0/3344311181 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c0c9180 con 0x7f072c0b54f0 2024-04-25T10:12:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0721ffb640 1 -- 172.21.15.133:0/3344311181 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c004410 con 0x7f072c0b54f0 2024-04-25T10:12:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f073267f640 1 -- 172.21.15.133:0/3344311181 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c0c9a20 con 0x7f072c0b54f0 2024-04-25T10:12:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f073267f640 1 -- 172.21.15.133:0/3344311181 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c0c9c20 con 0x7f072c0b54f0 2024-04-25T10:12:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.648+0000 7f0721ffb640 1 -- 172.21.15.133:0/3344311181 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f071c018070 con 0x7f072c0b54f0 2024-04-25T10:12:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.652+0000 7f0721ffb640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0710066f60 0x7f0710069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.652+0000 7f0721ffb640 1 -- 172.21.15.133:0/3344311181 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f071c084d80 con 0x7f072c0b54f0 2024-04-25T10:12:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.652+0000 7f073167d640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0710066f60 0x7f0710069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.652+0000 7f073167d640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0710066f60 0x7f0710069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f07280ad000 tx=0x7f0728008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.656+0000 7f0721ffb640 1 -- 172.21.15.133:0/3344311181 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f072c0c9c20 con 0x7f072c0b54f0 2024-04-25T10:12:56.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.800+0000 7f073267f640 1 -- 172.21.15.133:0/3344311181 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f072c0c9350 con 0x7f072c0b54f0 2024-04-25T10:12:56.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.800+0000 7f0721ffb640 1 -- 172.21.15.133:0/3344311181 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) v1 ==== 100+0+11 (secure 0 0 0) 0x7f071c088030 con 0x7f072c0b54f0 2024-04-25T10:12:56.805 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:12:56.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 -- 172.21.15.133:0/3344311181 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0710066f60 msgr2=0x7f0710069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0710066f60 0x7f0710069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f07280ad000 tx=0x7f0728008040 comp rx=0 tx=0).stop 2024-04-25T10:12:56.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 -- 172.21.15.133:0/3344311181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f072c0b54f0 msgr2=0x7f072c0c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f072c0b54f0 0x7f072c0c7e60 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f071c00b410 tx=0x7f071c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 -- 172.21.15.133:0/3344311181 shutdown_connections 2024-04-25T10:12:56.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f072c15f680 0x7f072c06be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0710066f60 0x7f0710069420 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f072c0b54f0 0x7f072c0c7e60 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 --2- 172.21.15.133:0/3344311181 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f072c0b5110 0x7f072c1536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 -- 172.21.15.133:0/3344311181 >> 172.21.15.133:0/3344311181 conn(0x7f072c08af60 msgr2=0x7f072c15faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 -- 172.21.15.133:0/3344311181 shutdown_connections 2024-04-25T10:12:56.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.804+0000 7f073267f640 1 -- 172.21.15.133:0/3344311181 wait complete. 2024-04-25T10:12:56.823 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:56.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.936+0000 7ff0c8aa0640 1 Processor -- start 2024-04-25T10:12:56.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.936+0000 7ff0c8aa0640 1 -- start start 2024-04-25T10:12:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c406de90 0x7ff0c40b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0c40b38d0 0x7ff0c40acae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c406f4d0 con 0x7ff0c41471d0 2024-04-25T10:12:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c40b3d10 con 0x7ff0c40b38d0 2024-04-25T10:12:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c406e430 con 0x7ff0c406de90 2024-04-25T10:12:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c2d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0c40b38d0 0x7ff0c40acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c259c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff0c41471d0 0x7ff0c40b2b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35310/0 (socket says 172.21.15.133:35310) 2024-04-25T10:12:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c259c640 1 -- 172.21.15.133:0/2742288466 learned_addr learned my addr 172.21.15.133:0/2742288466 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:56.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c1d9b640 1 --2- 172.21.15.133:0/2742288466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c406de90 0x7ff0c40b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c2d9d640 1 -- 172.21.15.133:0/2742288466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c406de90 msgr2=0x7ff0c40b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:56.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c2d9d640 1 --2- 172.21.15.133:0/2742288466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c406de90 0x7ff0c40b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c2d9d640 1 -- 172.21.15.133:0/2742288466 >> v1:172.21.15.133:6789/0 conn(0x7ff0c41471d0 legacy=0x7ff0c40b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c2d9d640 1 -- 172.21.15.133:0/2742288466 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c406e710 con 0x7ff0c40b38d0 2024-04-25T10:12:56.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c2d9d640 1 --2- 172.21.15.133:0/2742288466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c40b38d0 0x7ff0c40acae0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009d90 tx=0x7ff0bc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b35f5dd1a81703 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c159a640 1 -- 172.21.15.133:0/2742288466 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0b2070 con 0x7ff0c40b38d0 2024-04-25T10:12:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c159a640 1 -- 172.21.15.133:0/2742288466 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0a5940 con 0x7ff0c40b38d0 2024-04-25T10:12:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c159a640 1 -- 172.21.15.133:0/2742288466 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0a5d10 con 0x7ff0c40b38d0 2024-04-25T10:12:56.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/2742288466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c40b38d0 msgr2=0x7ff0c40acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 --2- 172.21.15.133:0/2742288466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c40b38d0 0x7ff0c40acae0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009d90 tx=0x7ff0bc01c970 comp rx=0 tx=0).stop 2024-04-25T10:12:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/2742288466 shutdown_connections 2024-04-25T10:12:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 --2- 172.21.15.133:0/2742288466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c40b38d0 0x7ff0c40acae0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 --2- 172.21.15.133:0/2742288466 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c406de90 0x7ff0c40b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/2742288466 >> 172.21.15.133:0/2742288466 conn(0x7ff0c406ac90 msgr2=0x7ff0c406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/2742288466 shutdown_connections 2024-04-25T10:12:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.940+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/2742288466 wait complete. 2024-04-25T10:12:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c8aa0640 1 Processor -- start 2024-04-25T10:12:56.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c8aa0640 1 -- start start 2024-04-25T10:12:56.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c8aa0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0c406de90 0x7ff0c415d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c8aa0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c40b38d0 0x7ff0c40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c8aa0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c41471d0 0x7ff0c40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c8aa0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4147730 con 0x7ff0c40b38d0 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c8aa0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c41475b0 con 0x7ff0c406de90 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c8aa0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40ad830 con 0x7ff0c41471d0 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c40b38d0 0x7ff0c40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c40b38d0 0x7ff0c40ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52522/0 (socket says 172.21.15.133:52522) 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c1d9b640 1 -- 172.21.15.133:0/746465750 learned_addr learned my addr 172.21.15.133:0/746465750 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c259c640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0c406de90 0x7ff0c415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c1d9b640 1 -- 172.21.15.133:0/746465750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c41471d0 msgr2=0x7ff0c40cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c1d9b640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c41471d0 0x7ff0c40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c1d9b640 1 -- 172.21.15.133:0/746465750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0c406de90 msgr2=0x7ff0c415d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c1d9b640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0c406de90 0x7ff0c415d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c1d9b640 1 -- 172.21.15.133:0/746465750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c413ac20 con 0x7ff0c40b38d0 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c259c640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0c406de90 0x7ff0c415d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c1d9b640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c40b38d0 0x7ff0c40ce880 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800d800 tx=0x7ff0b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0b2ffd640 1 -- 172.21.15.133:0/746465750 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b800b6c0 con 0x7ff0c40b38d0 2024-04-25T10:12:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0b2ffd640 1 -- 172.21.15.133:0/746465750 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b8019070 con 0x7ff0c40b38d0 2024-04-25T10:12:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/746465750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c413cec0 con 0x7ff0c40b38d0 2024-04-25T10:12:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0b2ffd640 1 -- 172.21.15.133:0/746465750 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b800ba60 con 0x7ff0c40b38d0 2024-04-25T10:12:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/746465750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c413d480 con 0x7ff0c40b38d0 2024-04-25T10:12:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.944+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/746465750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c4068ef0 con 0x7ff0c40b38d0 2024-04-25T10:12:56.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.948+0000 7ff0b2ffd640 1 -- 172.21.15.133:0/746465750 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0b800bc00 con 0x7ff0c40b38d0 2024-04-25T10:12:56.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.948+0000 7ff0b2ffd640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a4066f60 0x7ff0a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:56.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.948+0000 7ff0b2ffd640 1 -- 172.21.15.133:0/746465750 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff0b8085620 con 0x7ff0c40b38d0 2024-04-25T10:12:56.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.948+0000 7ff0c259c640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a4066f60 0x7ff0a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:56.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.948+0000 7ff0c259c640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a4066f60 0x7ff0a4069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4007fd0 tx=0x7ff0b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:56.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:56.952+0000 7ff0b2ffd640 1 -- 172.21.15.133:0/746465750 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0b80564a0 con 0x7ff0c40b38d0 2024-04-25T10:12:57.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.132+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/746465750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0c4160f40 con 0x7ff0c40b38d0 2024-04-25T10:12:57.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.132+0000 7ff0b2ffd640 1 -- 172.21.15.133:0/746465750 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7ff0b8055da0 con 0x7ff0c40b38d0 2024-04-25T10:12:57.136 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:57.136 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:57.137 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/746465750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a4066f60 msgr2=0x7ff0a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a4066f60 0x7ff0a4069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4007fd0 tx=0x7ff0b4009040 comp rx=0 tx=0).stop 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/746465750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c40b38d0 msgr2=0x7ff0c40ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c40b38d0 0x7ff0c40ce880 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800d800 tx=0x7ff0b800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/746465750 shutdown_connections 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c41471d0 0x7ff0c40cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0a4066f60 0x7ff0a4069420 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c40b38d0 0x7ff0c40ce880 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 --2- 172.21.15.133:0/746465750 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0c406de90 0x7ff0c415d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/746465750 >> 172.21.15.133:0/746465750 conn(0x7ff0c406ac90 msgr2=0x7ff0c413e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/746465750 shutdown_connections 2024-04-25T10:12:57.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.136+0000 7ff0c8aa0640 1 -- 172.21.15.133:0/746465750 wait complete. 2024-04-25T10:12:57.154 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:57.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa239023640 1 Processor -- start 2024-04-25T10:12:57.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa239023640 1 -- start start 2024-04-25T10:12:57.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa239023640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa23415b9b0 0x7fa2340cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa239023640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa2340cf7c0 0x7fa2340cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa239023640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23415bd90 con 0x7fa2340d01b0 2024-04-25T10:12:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa239023640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340c5610 con 0x7fa2340cf7c0 2024-04-25T10:12:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa239023640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23409ad10 con 0x7fa23415b9b0 2024-04-25T10:12:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa23259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa2340cf7c0 0x7fa2340cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa23359e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa2340d01b0 0x7fa2340cc1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35326/0 (socket says 172.21.15.133:35326) 2024-04-25T10:12:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa23259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa2340cf7c0 0x7fa2340cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52532/0 (socket says 172.21.15.133:52532) 2024-04-25T10:12:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa23359e640 1 -- 172.21.15.133:0/3281664516 learned_addr learned my addr 172.21.15.133:0/3281664516 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa232d9d640 1 --2- 172.21.15.133:0/3281664516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa23415b9b0 0x7fa2340cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa23259c640 1 -- 172.21.15.133:0/3281664516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa23415b9b0 msgr2=0x7fa2340cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa23259c640 1 --2- 172.21.15.133:0/3281664516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa23415b9b0 0x7fa2340cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa23259c640 1 -- 172.21.15.133:0/3281664516 >> v1:172.21.15.133:6789/0 conn(0x7fa2340d01b0 legacy=0x7fa2340cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.260+0000 7fa23259c640 1 -- 172.21.15.133:0/3281664516 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23409a750 con 0x7fa2340cf7c0 2024-04-25T10:12:57.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa23259c640 1 --2- 172.21.15.133:0/3281664516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340cf7c0 0x7fa2340cfc70 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fa2240097c0 tx=0x7fa22401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5247afe577bca3d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa231d9b640 1 -- 172.21.15.133:0/3281664516 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240b2070 con 0x7fa2340cf7c0 2024-04-25T10:12:57.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa231d9b640 1 -- 172.21.15.133:0/3281664516 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2240ad640 con 0x7fa2340cf7c0 2024-04-25T10:12:57.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa231d9b640 1 -- 172.21.15.133:0/3281664516 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240ada10 con 0x7fa2340cf7c0 2024-04-25T10:12:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 -- 172.21.15.133:0/3281664516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340cf7c0 msgr2=0x7fa2340cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 --2- 172.21.15.133:0/3281664516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340cf7c0 0x7fa2340cfc70 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fa2240097c0 tx=0x7fa22401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 -- 172.21.15.133:0/3281664516 shutdown_connections 2024-04-25T10:12:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 --2- 172.21.15.133:0/3281664516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340cf7c0 0x7fa2340cfc70 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 --2- 172.21.15.133:0/3281664516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa23415b9b0 0x7fa2340cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 -- 172.21.15.133:0/3281664516 >> 172.21.15.133:0/3281664516 conn(0x7fa2340b31a0 msgr2=0x7fa2340b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 -- 172.21.15.133:0/3281664516 shutdown_connections 2024-04-25T10:12:57.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 -- 172.21.15.133:0/3281664516 wait complete. 2024-04-25T10:12:57.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 Processor -- start 2024-04-25T10:12:57.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 -- start start 2024-04-25T10:12:57.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340cf7c0 0x7fa234155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2340d01b0 0x7fa2341524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa23415b9b0 0x7fa23406f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa234168660 con 0x7fa2340cf7c0 2024-04-25T10:12:57.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341684e0 con 0x7fa23415b9b0 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa239023640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341687e0 con 0x7fa2340d01b0 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa232d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340cf7c0 0x7fa234155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa232d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340cf7c0 0x7fa234155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52534/0 (socket says 172.21.15.133:52534) 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.264+0000 7fa232d9d640 1 -- 172.21.15.133:0/1366936120 learned_addr learned my addr 172.21.15.133:0/1366936120 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa232d9d640 1 -- 172.21.15.133:0/1366936120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2340d01b0 msgr2=0x7fa2341524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa232d9d640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2340d01b0 0x7fa2341524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa232d9d640 1 -- 172.21.15.133:0/1366936120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa23415b9b0 msgr2=0x7fa23406f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa232d9d640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa23415b9b0 0x7fa23406f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa232d9d640 1 -- 172.21.15.133:0/1366936120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23406d6d0 con 0x7fa2340cf7c0 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa232d9d640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340cf7c0 0x7fa234155340 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fa22800d5f0 tx=0x7fa22800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa21b7fe640 1 -- 172.21.15.133:0/1366936120 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2280125f0 con 0x7fa2340cf7c0 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa21b7fe640 1 -- 172.21.15.133:0/1366936120 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa228019070 con 0x7fa2340cf7c0 2024-04-25T10:12:57.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa239023640 1 -- 172.21.15.133:0/1366936120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa234153870 con 0x7fa2340cf7c0 2024-04-25T10:12:57.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa21b7fe640 1 -- 172.21.15.133:0/1366936120 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2280129b0 con 0x7fa2340cf7c0 2024-04-25T10:12:57.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa239023640 1 -- 172.21.15.133:0/1366936120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa234153e30 con 0x7fa2340cf7c0 2024-04-25T10:12:57.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa239023640 1 -- 172.21.15.133:0/1366936120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa234067c70 con 0x7fa2340cf7c0 2024-04-25T10:12:57.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa21b7fe640 1 -- 172.21.15.133:0/1366936120 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa228012b50 con 0x7fa2340cf7c0 2024-04-25T10:12:57.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa21b7fe640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa220066f60 0x7fa220069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa21b7fe640 1 -- 172.21.15.133:0/1366936120 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa228085fe0 con 0x7fa2340cf7c0 2024-04-25T10:12:57.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa23259c640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa220066f60 0x7fa220069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.268+0000 7fa23259c640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa220066f60 0x7fa220069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa2240af000 tx=0x7fa224004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.272+0000 7fa21b7fe640 1 -- 172.21.15.133:0/1366936120 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa228013390 con 0x7fa2340cf7c0 2024-04-25T10:12:57.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.420+0000 7fa239023640 1 -- 172.21.15.133:0/1366936120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2340b3c00 con 0x7fa2340cf7c0 2024-04-25T10:12:57.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa21b7fe640 1 -- 172.21.15.133:0/1366936120 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fa228056f30 con 0x7fa2340cf7c0 2024-04-25T10:12:57.426 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:57.426 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":120,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:12:19.584218+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:16.197174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:16.904937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 -- 172.21.15.133:0/1366936120 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa220066f60 msgr2=0x7fa220069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa220066f60 0x7fa220069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa2240af000 tx=0x7fa224004740 comp rx=0 tx=0).stop 2024-04-25T10:12:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 -- 172.21.15.133:0/1366936120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340cf7c0 msgr2=0x7fa234155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340cf7c0 0x7fa234155340 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fa22800d5f0 tx=0x7fa22800dab0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 -- 172.21.15.133:0/1366936120 shutdown_connections 2024-04-25T10:12:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa23415b9b0 0x7fa23406f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa2340d01b0 0x7fa2341524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa220066f60 0x7fa220069420 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 --2- 172.21.15.133:0/1366936120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340cf7c0 0x7fa234155340 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 -- 172.21.15.133:0/1366936120 >> 172.21.15.133:0/1366936120 conn(0x7fa2340b31a0 msgr2=0x7fa2340b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 -- 172.21.15.133:0/1366936120 shutdown_connections 2024-04-25T10:12:57.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.424+0000 7fa239023640 1 -- 172.21.15.133:0/1366936120 wait complete. 2024-04-25T10:12:57.444 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:12:57.444 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b8def2640 1 Processor -- start 2024-04-25T10:12:57.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b8def2640 1 -- start start 2024-04-25T10:12:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b8def2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6b880b4b70 0x7f6b8815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b8def2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b880b3f40 0x7f6b8815e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b8def2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8815b490 con 0x7f6b8815eeb0 2024-04-25T10:12:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b8def2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b88155150 con 0x7f6b880b4b70 2024-04-25T10:12:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b8def2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8815e200 con 0x7f6b880b3f40 2024-04-25T10:12:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b877fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6b880b4b70 0x7f6b8815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b877fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6b880b4b70 0x7f6b8815ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52548/0 (socket says 172.21.15.133:52548) 2024-04-25T10:12:57.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b877fe640 1 -- 172.21.15.133:0/3214407408 learned_addr learned my addr 172.21.15.133:0/3214407408 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b86ffd640 1 --2- 172.21.15.133:0/3214407408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b880b3f40 0x7f6b8815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b877fe640 1 -- 172.21.15.133:0/3214407408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b880b3f40 msgr2=0x7f6b8815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b877fe640 1 --2- 172.21.15.133:0/3214407408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b880b3f40 0x7f6b8815e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b877fe640 1 -- 172.21.15.133:0/3214407408 >> v1:172.21.15.162:6789/0 conn(0x7f6b8815eeb0 legacy=0x7f6b880ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.560+0000 7f6b877fe640 1 -- 172.21.15.133:0/3214407408 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8815b770 con 0x7f6b880b4b70 2024-04-25T10:12:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b877fe640 1 --2- 172.21.15.133:0/3214407408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b880b4b70 0x7f6b8815ac10 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009870 tx=0x7f6b8001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfabf29da4f16ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b867fc640 1 -- 172.21.15.133:0/3214407408 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b800b0070 con 0x7f6b880b4b70 2024-04-25T10:12:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b867fc640 1 -- 172.21.15.133:0/3214407408 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b800ab600 con 0x7f6b880b4b70 2024-04-25T10:12:57.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b867fc640 1 -- 172.21.15.133:0/3214407408 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b800ab9d0 con 0x7f6b880b4b70 2024-04-25T10:12:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b8def2640 1 -- 172.21.15.133:0/3214407408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b880b4b70 msgr2=0x7f6b8815ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b8def2640 1 --2- 172.21.15.133:0/3214407408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b880b4b70 0x7f6b8815ac10 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009870 tx=0x7f6b8001c400 comp rx=0 tx=0).stop 2024-04-25T10:12:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b8def2640 1 -- 172.21.15.133:0/3214407408 shutdown_connections 2024-04-25T10:12:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b8def2640 1 --2- 172.21.15.133:0/3214407408 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b880b3f40 0x7f6b8815e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b8def2640 1 --2- 172.21.15.133:0/3214407408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b880b4b70 0x7f6b8815ac10 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b8def2640 1 -- 172.21.15.133:0/3214407408 >> 172.21.15.133:0/3214407408 conn(0x7f6b88071aa0 msgr2=0x7f6b88071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b8def2640 1 -- 172.21.15.133:0/3214407408 shutdown_connections 2024-04-25T10:12:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b8def2640 1 -- 172.21.15.133:0/3214407408 wait complete. 2024-04-25T10:12:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b8def2640 1 Processor -- start 2024-04-25T10:12:57.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.564+0000 7f6b8def2640 1 -- start start 2024-04-25T10:12:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b8def2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b880b3f40 0x7f6b88147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b8def2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b880b4b70 0x7f6b881f4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b8def2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b8814b030 0x7f6b881f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b8def2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b88165590 con 0x7f6b880b4b70 2024-04-25T10:12:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b8def2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b880aed30 con 0x7f6b8814b030 2024-04-25T10:12:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b8def2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b88165710 con 0x7f6b880b3f40 2024-04-25T10:12:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b86ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b880b4b70 0x7f6b881f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b86ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b880b4b70 0x7f6b881f4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52560/0 (socket says 172.21.15.133:52560) 2024-04-25T10:12:57.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b86ffd640 1 -- 172.21.15.133:0/307302880 learned_addr learned my addr 172.21.15.133:0/307302880 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b87fff640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b8814b030 0x7f6b881f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b877fe640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b880b3f40 0x7f6b88147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b86ffd640 1 -- 172.21.15.133:0/307302880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b880b3f40 msgr2=0x7f6b88147930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b86ffd640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b880b3f40 0x7f6b88147930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b86ffd640 1 -- 172.21.15.133:0/307302880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b8814b030 msgr2=0x7f6b881f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b86ffd640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b8814b030 0x7f6b881f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b86ffd640 1 -- 172.21.15.133:0/307302880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8814b4c0 con 0x7f6b880b4b70 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b87fff640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b8814b030 0x7f6b881f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b86ffd640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b880b4b70 0x7f6b881f4980 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00d800 tx=0x7f6b7c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b6ffff640 1 -- 172.21.15.133:0/307302880 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00b6c0 con 0x7f6b880b4b70 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b6ffff640 1 -- 172.21.15.133:0/307302880 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7c019070 con 0x7f6b880b4b70 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b6ffff640 1 -- 172.21.15.133:0/307302880 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00ba60 con 0x7f6b880b4b70 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b8def2640 1 -- 172.21.15.133:0/307302880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8814b7a0 con 0x7f6b880b4b70 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b8def2640 1 -- 172.21.15.133:0/307302880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8814bd60 con 0x7f6b880b4b70 2024-04-25T10:12:57.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b8def2640 1 -- 172.21.15.133:0/307302880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b88067c70 con 0x7f6b880b4b70 2024-04-25T10:12:57.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b6ffff640 1 -- 172.21.15.133:0/307302880 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b7c00bc00 con 0x7f6b880b4b70 2024-04-25T10:12:57.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b6ffff640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b74066f60 0x7f6b74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b6ffff640 1 -- 172.21.15.133:0/307302880 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b7c085590 con 0x7f6b880b4b70 2024-04-25T10:12:57.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b877fe640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b74066f60 0x7f6b74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.568+0000 7f6b877fe640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b74066f60 0x7f6b74069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6b800ad000 tx=0x7f6b80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.572+0000 7f6b6ffff640 1 -- 172.21.15.133:0/307302880 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b7c0564a0 con 0x7f6b880b4b70 2024-04-25T10:12:57.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.752+0000 7f6b8def2640 1 -- 172.21.15.133:0/307302880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b88073dd0 con 0x7f6b880b4b70 2024-04-25T10:12:57.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.752+0000 7f6b6ffff640 1 -- 172.21.15.133:0/307302880 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f6b7c055da0 con 0x7f6b880b4b70 2024-04-25T10:12:57.756 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:57.756 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:57.757 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 -- 172.21.15.133:0/307302880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b74066f60 msgr2=0x7f6b74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b74066f60 0x7f6b74069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6b800ad000 tx=0x7f6b80008040 comp rx=0 tx=0).stop 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 -- 172.21.15.133:0/307302880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b880b4b70 msgr2=0x7f6b881f4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b880b4b70 0x7f6b881f4980 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00d800 tx=0x7f6b7c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 -- 172.21.15.133:0/307302880 shutdown_connections 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b8814b030 0x7f6b881f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b74066f60 0x7f6b74069420 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b880b4b70 0x7f6b881f4980 secure :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00d800 tx=0x7f6b7c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 --2- 172.21.15.133:0/307302880 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b880b3f40 0x7f6b88147930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 -- 172.21.15.133:0/307302880 >> 172.21.15.133:0/307302880 conn(0x7f6b88071aa0 msgr2=0x7f6b88165b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 -- 172.21.15.133:0/307302880 shutdown_connections 2024-04-25T10:12:57.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.756+0000 7f6b8def2640 1 -- 172.21.15.133:0/307302880 wait complete. 2024-04-25T10:12:57.775 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:57.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb26bb640 1 Processor -- start 2024-04-25T10:12:57.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb26bb640 1 -- start start 2024-04-25T10:12:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb26bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac158110 0x7f5eac0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb26bb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5eac1584f0 0x7f5eac15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb26bb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eac0cbe60 con 0x7f5eac06e460 2024-04-25T10:12:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb26bb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac0b19b0 con 0x7f5eac1584f0 2024-04-25T10:12:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb26bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac15b360 con 0x7f5eac158110 2024-04-25T10:12:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb0c58640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5eac06e460 0x7f5eac15e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35332/0 (socket says 172.21.15.133:35332) 2024-04-25T10:12:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb0c58640 1 -- 172.21.15.133:0/2935220234 learned_addr learned my addr 172.21.15.133:0/2935220234 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eabfff640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac158110 0x7f5eac0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eab7fe640 1 --2- 172.21.15.133:0/2935220234 >> v2:172.21.15.162:3300/0 conn(0x7f5eac1584f0 0x7f5eac15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966708509 0 0) 0x7f5eac0cbe60 con 0x7f5eac06e460 2024-04-25T10:12:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e98003530 con 0x7f5eac06e460 2024-04-25T10:12:57.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 728880798 0 0) 0x7f5e98003530 con 0x7f5eac06e460 2024-04-25T10:12:57.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac158110 msgr2=0x7f5eac0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac158110 0x7f5eac0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 >> v2:172.21.15.162:3300/0 conn(0x7f5eac1584f0 msgr2=0x7f5eac15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 --2- 172.21.15.133:0/2935220234 >> v2:172.21.15.162:3300/0 conn(0x7f5eac1584f0 0x7f5eac15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac15b640 con 0x7f5eac06e460 2024-04-25T10:12:57.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5e9c001680 con 0x7f5eac06e460 2024-04-25T10:12:57.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 >> v1:172.21.15.133:6789/0 conn(0x7f5eac06e460 legacy=0x7f5eac15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac158110 0x7f5e980039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e98003f40 0x7f5e98004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e98004d50 0x7f5e980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0b19b0 con 0x7f5e98004d50 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac15b360 con 0x7f5e98003f40 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9800b9d0 con 0x7f5eac158110 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb0c58640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e98004d50 0x7f5e980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eabfff640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac158110 0x7f5e980039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eab7fe640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e98003f40 0x7f5e98004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb0c58640 1 -- 172.21.15.133:0/2935220234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac158110 msgr2=0x7f5e980039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb0c58640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac158110 0x7f5e980039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb0c58640 1 -- 172.21.15.133:0/2935220234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e98003f40 msgr2=0x7f5e98004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb0c58640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5e98003f40 0x7f5e98004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.892+0000 7f5eb0c58640 1 -- 172.21.15.133:0/2935220234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98003530 con 0x7f5e98004d50 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb0c58640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e98004d50 0x7f5e980051d0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c2c0 tx=0x7f5e9c01cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad3fd967c0c10a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c007d60 con 0x7f5e98004d50 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c0a9020 con 0x7f5e98004d50 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eaaffd640 1 -- 172.21.15.133:0/2935220234 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0a5bb0 con 0x7f5e98004d50 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 -- 172.21.15.133:0/2935220234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e98004d50 msgr2=0x7f5e980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 --2- 172.21.15.133:0/2935220234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e98004d50 0x7f5e980051d0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c2c0 tx=0x7f5e9c01cce0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb0c58640 1 -- 172.21.15.133:0/2935220234 reap_dead start 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 -- 172.21.15.133:0/2935220234 shutdown_connections 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 -- 172.21.15.133:0/2935220234 >> 172.21.15.133:0/2935220234 conn(0x7f5eac067000 msgr2=0x7f5eac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 -- 172.21.15.133:0/2935220234 shutdown_connections 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 -- 172.21.15.133:0/2935220234 wait complete. 2024-04-25T10:12:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 Processor -- start 2024-04-25T10:12:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 -- start start 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac14dfe0 0x7f5eac0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eac0b40e0 0x7f5eac0c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac0c7130 0x7f5eac06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0cd670 con 0x7f5eac14dfe0 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac15bc00 con 0x7f5eac0b40e0 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eb26bb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0cd7f0 con 0x7f5eac0c7130 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac14dfe0 0x7f5eac0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac14dfe0 0x7f5eac0b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52576/0 (socket says 172.21.15.133:52576) 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eabfff640 1 -- 172.21.15.133:0/1149474275 learned_addr learned my addr 172.21.15.133:0/1149474275 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eabfff640 1 -- 172.21.15.133:0/1149474275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac0c7130 msgr2=0x7f5eac06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eabfff640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac0c7130 0x7f5eac06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eabfff640 1 -- 172.21.15.133:0/1149474275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eac0b40e0 msgr2=0x7f5eac0c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eabfff640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eac0b40e0 0x7f5eac0c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.896+0000 7f5eabfff640 1 -- 172.21.15.133:0/1149474275 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac1f3560 con 0x7f5eac14dfe0 2024-04-25T10:12:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5eabfff640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac14dfe0 0x7f5eac0b3ba0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4007f60 tx=0x7f5ea4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1149474275 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea401a020 con 0x7f5eac14dfe0 2024-04-25T10:12:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1149474275 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea4016470 con 0x7f5eac14dfe0 2024-04-25T10:12:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5eb26bb640 1 -- 172.21.15.133:0/1149474275 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac0c7920 con 0x7f5eac14dfe0 2024-04-25T10:12:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1149474275 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4016860 con 0x7f5eac14dfe0 2024-04-25T10:12:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5eb26bb640 1 -- 172.21.15.133:0/1149474275 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac0c7ee0 con 0x7f5eac14dfe0 2024-04-25T10:12:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5eb26bb640 1 -- 172.21.15.133:0/1149474275 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac06bc20 con 0x7f5eac14dfe0 2024-04-25T10:12:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1149474275 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ea401d020 con 0x7f5eac14dfe0 2024-04-25T10:12:57.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5ea8ff9640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e8c066f60 0x7f5e8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:57.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1149474275 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ea4085fe0 con 0x7f5eac14dfe0 2024-04-25T10:12:57.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5eab7fe640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e8c066f60 0x7f5e8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:57.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.900+0000 7f5eab7fe640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e8c066f60 0x7f5e8c069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5eac156a60 tx=0x7f5e94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:57.904+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1149474275 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ea4024030 con 0x7f5eac14dfe0 2024-04-25T10:12:58.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.084+0000 7f5eb26bb640 1 -- 172.21.15.133:0/1149474275 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5eac068ef0 con 0x7f5eac14dfe0 2024-04-25T10:12:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.084+0000 7f5ea8ff9640 1 -- 172.21.15.133:0/1149474275 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f5ea4056e60 con 0x7f5eac14dfe0 2024-04-25T10:12:58.088 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:58.088 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:17.925393+0000","modified":"2024-04-25T10:12:19.956686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/4218771669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4218771669},{"type":"v1","addr":"172.21.15.133:6837","nonce":4218771669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 72 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 -- 172.21.15.133:0/1149474275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e8c066f60 msgr2=0x7f5e8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e8c066f60 0x7f5e8c069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5eac156a60 tx=0x7f5e94008040 comp rx=0 tx=0).stop 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 -- 172.21.15.133:0/1149474275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac14dfe0 msgr2=0x7f5eac0b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac14dfe0 0x7f5eac0b3ba0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4007f60 tx=0x7f5ea4007f90 comp rx=0 tx=0).stop 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 -- 172.21.15.133:0/1149474275 shutdown_connections 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5eac0c7130 0x7f5eac06d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5eac0b40e0 0x7f5eac0c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5e8c066f60 0x7f5e8c069420 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 --2- 172.21.15.133:0/1149474275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5eac14dfe0 0x7f5eac0b3ba0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 -- 172.21.15.133:0/1149474275 >> 172.21.15.133:0/1149474275 conn(0x7f5eac067000 msgr2=0x7f5eac15ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 -- 172.21.15.133:0/1149474275 shutdown_connections 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.088+0000 7f5eb26bb640 1 -- 172.21.15.133:0/1149474275 wait complete. 2024-04-25T10:12:58.107 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:58.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f3472252640 1 Processor -- start 2024-04-25T10:12:58.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f3472252640 1 -- start start 2024-04-25T10:12:58.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f3472252640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f346c15c9a0 0x7f346c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f3472252640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0b1f70 0x7f346c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f3472252640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346c0c61d0 con 0x7f346c0ce330 2024-04-25T10:12:58.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f3472252640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346c0c64b0 con 0x7f346c15c9a0 2024-04-25T10:12:58.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f3472252640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c06d4f0 con 0x7f346c0b1f70 2024-04-25T10:12:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f346affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0b1f70 0x7f346c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f346b7fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f346c15c9a0 0x7f346c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f346affd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0b1f70 0x7f346c06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:42080/0 (socket says 172.21.15.133:42080) 2024-04-25T10:12:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f346affd640 1 -- 172.21.15.133:0/84429867 learned_addr learned my addr 172.21.15.133:0/84429867 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f346affd640 1 -- 172.21.15.133:0/84429867 >> v2:172.21.15.162:3300/0 conn(0x7f346c15c9a0 msgr2=0x7f346c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f346affd640 1 --2- 172.21.15.133:0/84429867 >> v2:172.21.15.162:3300/0 conn(0x7f346c15c9a0 0x7f346c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f346affd640 1 -- 172.21.15.133:0/84429867 >> v1:172.21.15.162:6789/0 conn(0x7f346c0ce330 legacy=0x7f346c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f346affd640 1 -- 172.21.15.133:0/84429867 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c06d7d0 con 0x7f346c0b1f70 2024-04-25T10:12:58.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.212+0000 7f346b7fe640 1 --2- 172.21.15.133:0/84429867 >> v2:172.21.15.162:3300/0 conn(0x7f346c15c9a0 0x7f346c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:58.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f346affd640 1 --2- 172.21.15.133:0/84429867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0b1f70 0x7f346c06dcf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3460009870 tx=0x7f346001c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=309027eace19800b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f346a7fc640 1 -- 172.21.15.133:0/84429867 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34600aa680 con 0x7f346c0b1f70 2024-04-25T10:12:58.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f346a7fc640 1 -- 172.21.15.133:0/84429867 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34600af070 con 0x7f346c0b1f70 2024-04-25T10:12:58.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f346a7fc640 1 -- 172.21.15.133:0/84429867 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34600aaa40 con 0x7f346c0b1f70 2024-04-25T10:12:58.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 -- 172.21.15.133:0/84429867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0b1f70 msgr2=0x7f346c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 --2- 172.21.15.133:0/84429867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0b1f70 0x7f346c06dcf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3460009870 tx=0x7f346001c500 comp rx=0 tx=0).stop 2024-04-25T10:12:58.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 -- 172.21.15.133:0/84429867 shutdown_connections 2024-04-25T10:12:58.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 --2- 172.21.15.133:0/84429867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0b1f70 0x7f346c06dcf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 --2- 172.21.15.133:0/84429867 >> v2:172.21.15.162:3300/0 conn(0x7f346c15c9a0 0x7f346c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 -- 172.21.15.133:0/84429867 >> 172.21.15.133:0/84429867 conn(0x7f346c067000 msgr2=0x7f346c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 -- 172.21.15.133:0/84429867 shutdown_connections 2024-04-25T10:12:58.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 -- 172.21.15.133:0/84429867 wait complete. 2024-04-25T10:12:58.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 Processor -- start 2024-04-25T10:12:58.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 -- start start 2024-04-25T10:12:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f346c0b1f70 0x7f346c152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0ce330 0x7f346c154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f346c15c9a0 0x7f346c1450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c0ceb50 con 0x7f346c15c9a0 2024-04-25T10:12:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c15cd80 con 0x7f346c0b1f70 2024-04-25T10:12:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f3472252640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c145b20 con 0x7f346c0ce330 2024-04-25T10:12:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f346bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f346c15c9a0 0x7f346c1450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f346bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f346c15c9a0 0x7f346c1450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52590/0 (socket says 172.21.15.133:52590) 2024-04-25T10:12:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.216+0000 7f346bfff640 1 -- 172.21.15.133:0/3143700528 learned_addr learned my addr 172.21.15.133:0/3143700528 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f346affd640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0ce330 0x7f346c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f346b7fe640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f346c0b1f70 0x7f346c152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f346bfff640 1 -- 172.21.15.133:0/3143700528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0ce330 msgr2=0x7f346c154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f346bfff640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0ce330 0x7f346c154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f346bfff640 1 -- 172.21.15.133:0/3143700528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f346c0b1f70 msgr2=0x7f346c152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f346bfff640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f346c0b1f70 0x7f346c152eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f346bfff640 1 -- 172.21.15.133:0/3143700528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c145e00 con 0x7f346c15c9a0 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f346bfff640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f346c15c9a0 0x7f346c1450e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f346400c730 tx=0x7f346400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f3453fff640 1 -- 172.21.15.133:0/3143700528 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3464003b80 con 0x7f346c15c9a0 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f3453fff640 1 -- 172.21.15.133:0/3143700528 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3464003d20 con 0x7f346c15c9a0 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f3472252640 1 -- 172.21.15.133:0/3143700528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346c1460e0 con 0x7f346c15c9a0 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f3453fff640 1 -- 172.21.15.133:0/3143700528 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346400aa30 con 0x7f346c15c9a0 2024-04-25T10:12:58.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f3472252640 1 -- 172.21.15.133:0/3143700528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346c154c90 con 0x7f346c15c9a0 2024-04-25T10:12:58.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f3472252640 1 -- 172.21.15.133:0/3143700528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346c06bc20 con 0x7f346c15c9a0 2024-04-25T10:12:58.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f3453fff640 1 -- 172.21.15.133:0/3143700528 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f346400abd0 con 0x7f346c15c9a0 2024-04-25T10:12:58.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f3453fff640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3458066f60 0x7f3458069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f346b7fe640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3458066f60 0x7f3458069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f3453fff640 1 -- 172.21.15.133:0/3143700528 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3464085020 con 0x7f346c15c9a0 2024-04-25T10:12:58.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.220+0000 7f346b7fe640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3458066f60 0x7f3458069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f345c009580 tx=0x7f345c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.224+0000 7f3453fff640 1 -- 172.21.15.133:0/3143700528 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3464055ea0 con 0x7f346c15c9a0 2024-04-25T10:12:58.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.372+0000 7f3472252640 1 -- 172.21.15.133:0/3143700528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f346c0737e0 con 0x7f346c15c9a0 2024-04-25T10:12:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.376+0000 7f3453fff640 1 -- 172.21.15.133:0/3143700528 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f34640557a0 con 0x7f346c15c9a0 2024-04-25T10:12:58.380 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:12:58.380 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":120,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:12:19.584218+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:16.197174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:16.904937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 -- 172.21.15.133:0/3143700528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3458066f60 msgr2=0x7f3458069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3458066f60 0x7f3458069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f345c009580 tx=0x7f345c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 -- 172.21.15.133:0/3143700528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f346c15c9a0 msgr2=0x7f346c1450e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f346c15c9a0 0x7f346c1450e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f346400c730 tx=0x7f346400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 -- 172.21.15.133:0/3143700528 shutdown_connections 2024-04-25T10:12:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3458066f60 0x7f3458069420 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f346c15c9a0 0x7f346c1450e0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f346c0ce330 0x7f346c154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 --2- 172.21.15.133:0/3143700528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f346c0b1f70 0x7f346c152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 -- 172.21.15.133:0/3143700528 >> 172.21.15.133:0/3143700528 conn(0x7f346c067000 msgr2=0x7f346c150cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 -- 172.21.15.133:0/3143700528 shutdown_connections 2024-04-25T10:12:58.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.380+0000 7f3472252640 1 -- 172.21.15.133:0/3143700528 wait complete. 2024-04-25T10:12:58.398 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:12:58.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e79e6c640 1 Processor -- start 2024-04-25T10:12:58.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e79e6c640 1 -- start start 2024-04-25T10:12:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e79e6c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e7415d5e0 0x7f4e740cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e79e6c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e740ce2e0 con 0x7f4e74145170 2024-04-25T10:12:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e79e6c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e7406d4b0 con 0x7f4e740cd820 2024-04-25T10:12:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e79e6c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7406db00 con 0x7f4e7415d5e0 2024-04-25T10:12:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e737fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4e74145170 0x7f4e7406df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35336/0 (socket says 172.21.15.133:35336) 2024-04-25T10:12:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e737fe640 1 -- 172.21.15.133:0/884452754 learned_addr learned my addr 172.21.15.133:0/884452754 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e73fff640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e7415d5e0 0x7f4e740cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398049863 0 0) 0x7f4e740ce2e0 con 0x7f4e74145170 2024-04-25T10:12:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e60003530 con 0x7f4e74145170 2024-04-25T10:12:58.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2696038077 0 0) 0x7f4e60003530 con 0x7f4e74145170 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e7415d5e0 msgr2=0x7f4e740cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e7415d5e0 0x7f4e740cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 >> v1:172.21.15.162:6789/0 conn(0x7f4e740cd820 legacy=0x7f4e7415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7406d790 con 0x7f4e74145170 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4e64001680 con 0x7f4e74145170 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 >> v1:172.21.15.133:6789/0 conn(0x7f4e74145170 legacy=0x7f4e7406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e600039f0 0x7f4e60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e60004350 0x7f4e60004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e60005030 0x7f4e6000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7406db00 con 0x7f4e60004350 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6000bf10 con 0x7f4e60005030 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6000c0c0 con 0x7f4e600039f0 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e6bfff640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e60004350 0x7f4e60004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e737fe640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e600039f0 0x7f4e60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e73fff640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e60005030 0x7f4e6000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e6bfff640 1 -- 172.21.15.133:0/884452754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e600039f0 msgr2=0x7f4e60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e6bfff640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e600039f0 0x7f4e60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e6bfff640 1 -- 172.21.15.133:0/884452754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e60005030 msgr2=0x7f4e6000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e6bfff640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e60005030 0x7f4e6000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.516+0000 7f4e6bfff640 1 -- 172.21.15.133:0/884452754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e60003530 con 0x7f4e60004350 2024-04-25T10:12:58.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e6bfff640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e60004350 0x7f4e60004830 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c001780 tx=0x7f4e5c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55da9463ab4770c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0013a0 con 0x7f4e60004350 2024-04-25T10:12:58.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c0052f0 con 0x7f4e60004350 2024-04-25T10:12:58.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e72ffd640 1 -- 172.21.15.133:0/884452754 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c003840 con 0x7f4e60004350 2024-04-25T10:12:58.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e79e6c640 1 -- 172.21.15.133:0/884452754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e60004350 msgr2=0x7f4e60004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e79e6c640 1 --2- 172.21.15.133:0/884452754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e60004350 0x7f4e60004830 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c001780 tx=0x7f4e5c01c420 comp rx=0 tx=0).stop 2024-04-25T10:12:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e73fff640 1 -- 172.21.15.133:0/884452754 reap_dead start 2024-04-25T10:12:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e79e6c640 1 -- 172.21.15.133:0/884452754 shutdown_connections 2024-04-25T10:12:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e79e6c640 1 -- 172.21.15.133:0/884452754 >> 172.21.15.133:0/884452754 conn(0x7f4e74066f70 msgr2=0x7f4e7406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e79e6c640 1 -- 172.21.15.133:0/884452754 shutdown_connections 2024-04-25T10:12:58.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e79e6c640 1 -- 172.21.15.133:0/884452754 wait complete. 2024-04-25T10:12:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e79e6c640 1 Processor -- start 2024-04-25T10:12:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.520+0000 7f4e79e6c640 1 -- start start 2024-04-25T10:12:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e79e6c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e60004350 0x7f4e74139170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e79e6c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e7415fea0 0x7f4e7413fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e79e6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e7413ea80 0x7f4e7413d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e79e6c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7415cb70 con 0x7f4e7413ea80 2024-04-25T10:12:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e79e6c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e74145550 con 0x7f4e60004350 2024-04-25T10:12:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e79e6c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740cdca0 con 0x7f4e7415fea0 2024-04-25T10:12:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e73fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e7413ea80 0x7f4e7413d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e73fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e7413ea80 0x7f4e7413d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52598/0 (socket says 172.21.15.133:52598) 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e73fff640 1 -- 172.21.15.133:0/2999710693 learned_addr learned my addr 172.21.15.133:0/2999710693 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e6bfff640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e7415fea0 0x7f4e7413fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e737fe640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e60004350 0x7f4e74139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e73fff640 1 -- 172.21.15.133:0/2999710693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e7415fea0 msgr2=0x7f4e7413fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e73fff640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e7415fea0 0x7f4e7413fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e73fff640 1 -- 172.21.15.133:0/2999710693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e60004350 msgr2=0x7f4e74139170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e73fff640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e60004350 0x7f4e74139170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e73fff640 1 -- 172.21.15.133:0/2999710693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e741401a0 con 0x7f4e7413ea80 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e73fff640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e7413ea80 0x7f4e7413d840 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f4e740b2810 tx=0x7f4e6c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e70ff9640 1 -- 172.21.15.133:0/2999710693 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0039a0 con 0x7f4e7413ea80 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e70ff9640 1 -- 172.21.15.133:0/2999710693 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c01b070 con 0x7f4e7413ea80 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e79e6c640 1 -- 172.21.15.133:0/2999710693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7415f3e0 con 0x7f4e7413ea80 2024-04-25T10:12:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e70ff9640 1 -- 172.21.15.133:0/2999710693 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c003b80 con 0x7f4e7413ea80 2024-04-25T10:12:58.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e79e6c640 1 -- 172.21.15.133:0/2999710693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7415f9a0 con 0x7f4e7413ea80 2024-04-25T10:12:58.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e79e6c640 1 -- 172.21.15.133:0/2999710693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e74068e80 con 0x7f4e7413ea80 2024-04-25T10:12:58.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e70ff9640 1 -- 172.21.15.133:0/2999710693 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e6c004030 con 0x7f4e7413ea80 2024-04-25T10:12:58.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e70ff9640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e54066f60 0x7f4e54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e70ff9640 1 -- 172.21.15.133:0/2999710693 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4e6c085a70 con 0x7f4e7413ea80 2024-04-25T10:12:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e737fe640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e54066f60 0x7f4e54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.524+0000 7f4e737fe640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e54066f60 0x7f4e54069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4e64001ec0 tx=0x7f4e64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.528+0000 7f4e70ff9640 1 -- 172.21.15.133:0/2999710693 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e6c056980 con 0x7f4e7413ea80 2024-04-25T10:12:58.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.728+0000 7f4e79e6c640 1 -- 172.21.15.133:0/2999710693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4e74155fb0 con 0x7f4e7413ea80 2024-04-25T10:12:58.731 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:12:58.728+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:12:58.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.728+0000 7f4e70ff9640 1 -- 172.21.15.133:0/2999710693 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4e6c056280 con 0x7f4e7413ea80 2024-04-25T10:12:58.733 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:12:58.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 -- 172.21.15.133:0/2999710693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e54066f60 msgr2=0x7f4e54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e54066f60 0x7f4e54069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4e64001ec0 tx=0x7f4e64009040 comp rx=0 tx=0).stop 2024-04-25T10:12:58.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 -- 172.21.15.133:0/2999710693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e7413ea80 msgr2=0x7f4e7413d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e7413ea80 0x7f4e7413d840 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f4e740b2810 tx=0x7f4e6c0115a0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 -- 172.21.15.133:0/2999710693 shutdown_connections 2024-04-25T10:12:58.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e54066f60 0x7f4e54069420 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e7413ea80 0x7f4e7413d840 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e7415fea0 0x7f4e7413fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 --2- 172.21.15.133:0/2999710693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e60004350 0x7f4e74139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 -- 172.21.15.133:0/2999710693 >> 172.21.15.133:0/2999710693 conn(0x7f4e74066f70 msgr2=0x7f4e7406af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 -- 172.21.15.133:0/2999710693 shutdown_connections 2024-04-25T10:12:58.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.736+0000 7f4e79e6c640 1 -- 172.21.15.133:0/2999710693 wait complete. 2024-04-25T10:12:58.758 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:12:58.783 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:12:58.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae14eeb640 1 Processor -- start 2024-04-25T10:12:58.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae14eeb640 1 -- start start 2024-04-25T10:12:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae14eeb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fae1015b8e0 0x7fae1015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae14eeb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae1006ac90 0x7fae100b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae14eeb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1006b0f0 0x7fae100b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae14eeb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1015cd60 con 0x7fae1006ac90 2024-04-25T10:12:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae14eeb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1015c390 con 0x7fae1015b8e0 2024-04-25T10:12:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae14eeb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1006b650 con 0x7fae1006b0f0 2024-04-25T10:12:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae0dd9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae1006ac90 0x7fae100b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae0dd9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae1006ac90 0x7fae100b14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52622/0 (socket says 172.21.15.133:52622) 2024-04-25T10:12:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae0dd9b640 1 -- 172.21.15.133:0/202337238 learned_addr learned my addr 172.21.15.133:0/202337238 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae0e59c640 1 --2- 172.21.15.133:0/202337238 >> v2:172.21.15.162:3300/0 conn(0x7fae1015b8e0 0x7fae1015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae0ed9d640 1 --2- 172.21.15.133:0/202337238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1006b0f0 0x7fae100b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae0dd9b640 1 -- 172.21.15.133:0/202337238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1006b0f0 msgr2=0x7fae100b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae0dd9b640 1 --2- 172.21.15.133:0/202337238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1006b0f0 0x7fae100b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae0dd9b640 1 -- 172.21.15.133:0/202337238 >> v2:172.21.15.162:3300/0 conn(0x7fae1015b8e0 msgr2=0x7fae1015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae0dd9b640 1 --2- 172.21.15.133:0/202337238 >> v2:172.21.15.162:3300/0 conn(0x7fae1015b8e0 0x7fae1015c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae0dd9b640 1 -- 172.21.15.133:0/202337238 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1006b930 con 0x7fae1006ac90 2024-04-25T10:12:58.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.868+0000 7fae0e59c640 1 --2- 172.21.15.133:0/202337238 >> v2:172.21.15.162:3300/0 conn(0x7fae1015b8e0 0x7fae1015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:58.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae0dd9b640 1 --2- 172.21.15.133:0/202337238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1006ac90 0x7fae100b14d0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fae100709c0 tx=0x7fae0401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43b4543ad96fa32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae0d59a640 1 -- 172.21.15.133:0/202337238 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040ab5f0 con 0x7fae1006ac90 2024-04-25T10:12:58.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae0d59a640 1 -- 172.21.15.133:0/202337238 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae040b0070 con 0x7fae1006ac90 2024-04-25T10:12:58.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae0d59a640 1 -- 172.21.15.133:0/202337238 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040ab9b0 con 0x7fae1006ac90 2024-04-25T10:12:58.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae14eeb640 1 -- 172.21.15.133:0/202337238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1006ac90 msgr2=0x7fae100b14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae14eeb640 1 --2- 172.21.15.133:0/202337238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1006ac90 0x7fae100b14d0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fae100709c0 tx=0x7fae0401c520 comp rx=0 tx=0).stop 2024-04-25T10:12:58.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae14eeb640 1 -- 172.21.15.133:0/202337238 shutdown_connections 2024-04-25T10:12:58.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae14eeb640 1 --2- 172.21.15.133:0/202337238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1006b0f0 0x7fae100b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae14eeb640 1 --2- 172.21.15.133:0/202337238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1006ac90 0x7fae100b14d0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae14eeb640 1 --2- 172.21.15.133:0/202337238 >> v2:172.21.15.162:3300/0 conn(0x7fae1015b8e0 0x7fae1015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae14eeb640 1 -- 172.21.15.133:0/202337238 >> 172.21.15.133:0/202337238 conn(0x7fae100999d0 msgr2=0x7fae100d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae14eeb640 1 -- 172.21.15.133:0/202337238 shutdown_connections 2024-04-25T10:12:58.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae14eeb640 1 -- 172.21.15.133:0/202337238 wait complete. 2024-04-25T10:12:58.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae14eeb640 1 Processor -- start 2024-04-25T10:12:58.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.872+0000 7fae14eeb640 1 -- start start 2024-04-25T10:12:58.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae14eeb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1006ac90 0x7fae101539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae14eeb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1006b0f0 0x7fae1015a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae14eeb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1015b8e0 0x7fae10151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae14eeb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1015fce0 con 0x7fae1006b0f0 2024-04-25T10:12:58.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae14eeb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1015fb60 con 0x7fae1006ac90 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae14eeb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1015fe60 con 0x7fae1015b8e0 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1006b0f0 0x7fae1015a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1006b0f0 0x7fae1015a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52630/0 (socket says 172.21.15.133:52630) 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0dd9b640 1 -- 172.21.15.133:0/285650780 learned_addr learned my addr 172.21.15.133:0/285650780 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0e59c640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1006ac90 0x7fae101539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0ed9d640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1015b8e0 0x7fae10151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0dd9b640 1 -- 172.21.15.133:0/285650780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1015b8e0 msgr2=0x7fae10151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0dd9b640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1015b8e0 0x7fae10151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0dd9b640 1 -- 172.21.15.133:0/285650780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1006ac90 msgr2=0x7fae101539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0dd9b640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1006ac90 0x7fae101539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0dd9b640 1 -- 172.21.15.133:0/285650780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae100afe00 con 0x7fae1006b0f0 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0dd9b640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1006b0f0 0x7fae1015a2d0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fae0401abd0 tx=0x7fae0401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fadfeffd640 1 -- 172.21.15.133:0/285650780 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040be5f0 con 0x7fae1006b0f0 2024-04-25T10:12:58.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fadfeffd640 1 -- 172.21.15.133:0/285650780 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae040be7b0 con 0x7fae1006b0f0 2024-04-25T10:12:58.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fadfeffd640 1 -- 172.21.15.133:0/285650780 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040beba0 con 0x7fae1006b0f0 2024-04-25T10:12:58.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae14eeb640 1 -- 172.21.15.133:0/285650780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae100b00e0 con 0x7fae1006b0f0 2024-04-25T10:12:58.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae14eeb640 1 -- 172.21.15.133:0/285650780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae100b06a0 con 0x7fae1006b0f0 2024-04-25T10:12:58.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae14eeb640 1 -- 172.21.15.133:0/285650780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae10067c70 con 0x7fae1006b0f0 2024-04-25T10:12:58.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fadfeffd640 1 -- 172.21.15.133:0/285650780 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae040b43f0 con 0x7fae1006b0f0 2024-04-25T10:12:58.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fadfeffd640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf0066f90 0x7fadf0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0e59c640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf0066f90 0x7fadf0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fadfeffd640 1 -- 172.21.15.133:0/285650780 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fae0411f540 con 0x7fae1006b0f0 2024-04-25T10:12:58.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.876+0000 7fae0e59c640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf0066f90 0x7fadf0069450 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fae100b2460 tx=0x7fae08008450 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:58.880+0000 7fadfeffd640 1 -- 172.21.15.133:0/285650780 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae040c4060 con 0x7fae1006b0f0 2024-04-25T10:12:59.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.060+0000 7fae14eeb640 1 -- 172.21.15.133:0/285650780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fae10158880 con 0x7fae1006b0f0 2024-04-25T10:12:59.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.732+0000 7fadfeffd640 1 -- 172.21.15.133:0/285650780 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) v1 ==== 104+0+0 (secure 0 0 0) 0x7fae04016110 con 0x7fae1006b0f0 2024-04-25T10:12:59.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 -- 172.21.15.133:0/285650780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf0066f90 msgr2=0x7fadf0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf0066f90 0x7fadf0069450 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fae100b2460 tx=0x7fae08008450 comp rx=0 tx=0).stop 2024-04-25T10:12:59.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 -- 172.21.15.133:0/285650780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1006b0f0 msgr2=0x7fae1015a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1006b0f0 0x7fae1015a2d0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fae0401abd0 tx=0x7fae0401ad60 comp rx=0 tx=0).stop 2024-04-25T10:12:59.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 -- 172.21.15.133:0/285650780 shutdown_connections 2024-04-25T10:12:59.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae1015b8e0 0x7fae10151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf0066f90 0x7fadf0069450 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1006b0f0 0x7fae1015a2d0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 --2- 172.21.15.133:0/285650780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae1006ac90 0x7fae101539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 -- 172.21.15.133:0/285650780 >> 172.21.15.133:0/285650780 conn(0x7fae100999d0 msgr2=0x7fae100caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 -- 172.21.15.133:0/285650780 shutdown_connections 2024-04-25T10:12:59.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.736+0000 7fae14eeb640 1 -- 172.21.15.133:0/285650780 wait complete. 2024-04-25T10:12:59.761 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:12:59.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc63888640 1 Processor -- start 2024-04-25T10:12:59.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc63888640 1 -- start start 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc63888640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbc5c15d8c0 0x7fbc5c161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc63888640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc5c0cef10 0x7fbc5c0d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc63888640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5c0d1cf0 con 0x7fbc5c161c60 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc63888640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5c1626f0 con 0x7fbc5c15d8c0 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc63888640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c162130 con 0x7fbc5c0cef10 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc61624640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbc5c15d8c0 0x7fbc5c161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc61e25640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc5c0cef10 0x7fbc5c0d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc60e23640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fbc5c161c60 0x7fbc5c0ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:48256/0 (socket says 172.21.15.133:48256) 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc61624640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbc5c15d8c0 0x7fbc5c161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:45382/0 (socket says 172.21.15.133:45382) 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc60e23640 1 -- 172.21.15.133:0/4235018840 learned_addr learned my addr 172.21.15.133:0/4235018840 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:12:59.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc61e25640 1 -- 172.21.15.133:0/4235018840 >> v2:172.21.15.162:3300/0 conn(0x7fbc5c15d8c0 msgr2=0x7fbc5c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc61e25640 1 --2- 172.21.15.133:0/4235018840 >> v2:172.21.15.162:3300/0 conn(0x7fbc5c15d8c0 0x7fbc5c161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc61e25640 1 -- 172.21.15.133:0/4235018840 >> v1:172.21.15.162:6789/0 conn(0x7fbc5c161c60 legacy=0x7fbc5c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.936+0000 7fbc61e25640 1 -- 172.21.15.133:0/4235018840 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c162410 con 0x7fbc5c0cef10 2024-04-25T10:12:59.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc61e25640 1 --2- 172.21.15.133:0/4235018840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc5c0cef10 0x7fbc5c0d24b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009dc0 tx=0x7fbc5401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11cb6e18be24f222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc53fff640 1 -- 172.21.15.133:0/4235018840 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc540b2070 con 0x7fbc5c0cef10 2024-04-25T10:12:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc53fff640 1 -- 172.21.15.133:0/4235018840 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c0b9a20 con 0x7fbc5c0cef10 2024-04-25T10:12:59.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc53fff640 1 -- 172.21.15.133:0/4235018840 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc540a5b90 con 0x7fbc5c0cef10 2024-04-25T10:12:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc63888640 1 -- 172.21.15.133:0/4235018840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc5c0cef10 msgr2=0x7fbc5c0d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc63888640 1 --2- 172.21.15.133:0/4235018840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc5c0cef10 0x7fbc5c0d24b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009dc0 tx=0x7fbc5401c890 comp rx=0 tx=0).stop 2024-04-25T10:12:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc63888640 1 -- 172.21.15.133:0/4235018840 shutdown_connections 2024-04-25T10:12:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc63888640 1 --2- 172.21.15.133:0/4235018840 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc5c0cef10 0x7fbc5c0d24b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc63888640 1 --2- 172.21.15.133:0/4235018840 >> v2:172.21.15.162:3300/0 conn(0x7fbc5c15d8c0 0x7fbc5c161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc63888640 1 -- 172.21.15.133:0/4235018840 >> 172.21.15.133:0/4235018840 conn(0x7fbc5c0723c0 msgr2=0x7fbc5c0727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc63888640 1 -- 172.21.15.133:0/4235018840 shutdown_connections 2024-04-25T10:12:59.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc63888640 1 -- 172.21.15.133:0/4235018840 wait complete. 2024-04-25T10:12:59.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc63888640 1 Processor -- start 2024-04-25T10:12:59.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.940+0000 7fbc63888640 1 -- start start 2024-04-25T10:12:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc63888640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc5c0cef10 0x7fbc5c156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc63888640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc5c15d8c0 0x7fbc5c150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc63888640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c161c60 0x7fbc5c14c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc63888640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0d35d0 con 0x7fbc5c161c60 2024-04-25T10:12:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc63888640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0d3450 con 0x7fbc5c0cef10 2024-04-25T10:12:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc63888640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0d3750 con 0x7fbc5c15d8c0 2024-04-25T10:12:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61e25640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c161c60 0x7fbc5c14c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61e25640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c161c60 0x7fbc5c14c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52636/0 (socket says 172.21.15.133:52636) 2024-04-25T10:12:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61e25640 1 -- 172.21.15.133:0/3840940566 learned_addr learned my addr 172.21.15.133:0/3840940566 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:12:59.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc60e23640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc5c15d8c0 0x7fbc5c150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61624640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc5c0cef10 0x7fbc5c156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61e25640 1 -- 172.21.15.133:0/3840940566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc5c15d8c0 msgr2=0x7fbc5c150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61e25640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc5c15d8c0 0x7fbc5c150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61e25640 1 -- 172.21.15.133:0/3840940566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc5c0cef10 msgr2=0x7fbc5c156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61e25640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc5c0cef10 0x7fbc5c156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61e25640 1 -- 172.21.15.133:0/3840940566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c1557d0 con 0x7fbc5c161c60 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61624640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc5c0cef10 0x7fbc5c156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61e25640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c161c60 0x7fbc5c14c850 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009dc0 tx=0x7fbc54019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc51ffb640 1 -- 172.21.15.133:0/3840940566 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc540b2050 con 0x7fbc5c161c60 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc51ffb640 1 -- 172.21.15.133:0/3840940566 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc540be070 con 0x7fbc5c161c60 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc63888640 1 -- 172.21.15.133:0/3840940566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c155ab0 con 0x7fbc5c161c60 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc51ffb640 1 -- 172.21.15.133:0/3840940566 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54009450 con 0x7fbc5c161c60 2024-04-25T10:12:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc63888640 1 -- 172.21.15.133:0/3840940566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c156070 con 0x7fbc5c161c60 2024-04-25T10:12:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc63888640 1 -- 172.21.15.133:0/3840940566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c069000 con 0x7fbc5c161c60 2024-04-25T10:12:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc51ffb640 1 -- 172.21.15.133:0/3840940566 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc540ada90 con 0x7fbc5c161c60 2024-04-25T10:12:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc51ffb640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbc40066f60 0x7fbc40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc51ffb640 1 -- 172.21.15.133:0/3840940566 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbc541203c0 con 0x7fbc5c161c60 2024-04-25T10:12:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61624640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbc40066f60 0x7fbc40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.944+0000 7fbc61624640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbc40066f60 0x7fbc40069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009ae0 tx=0x7fbc58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:12:59.948+0000 7fbc51ffb640 1 -- 172.21.15.133:0/3840940566 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc54016540 con 0x7fbc5c161c60 2024-04-25T10:13:00.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:00.112+0000 7fbc63888640 1 -- 172.21.15.133:0/3840940566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbc5c0d1700 con 0x7fbc5c161c60 2024-04-25T10:13:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.092+0000 7fbc51ffb640 1 -- 172.21.15.133:0/3840940566 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbc540f12b0 con 0x7fbc5c161c60 2024-04-25T10:13:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:13:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 -- 172.21.15.133:0/3840940566 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbc40066f60 msgr2=0x7fbc40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbc40066f60 0x7fbc40069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009ae0 tx=0x7fbc58008040 comp rx=0 tx=0).stop 2024-04-25T10:13:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 -- 172.21.15.133:0/3840940566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c161c60 msgr2=0x7fbc5c14c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c161c60 0x7fbc5c14c850 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009dc0 tx=0x7fbc54019630 comp rx=0 tx=0).stop 2024-04-25T10:13:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 -- 172.21.15.133:0/3840940566 shutdown_connections 2024-04-25T10:13:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbc40066f60 0x7fbc40069420 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc5c161c60 0x7fbc5c14c850 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbc5c15d8c0 0x7fbc5c150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 --2- 172.21.15.133:0/3840940566 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc5c0cef10 0x7fbc5c156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 -- 172.21.15.133:0/3840940566 >> 172.21.15.133:0/3840940566 conn(0x7fbc5c0723c0 msgr2=0x7fbc5c15aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 -- 172.21.15.133:0/3840940566 shutdown_connections 2024-04-25T10:13:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.096+0000 7fbc63888640 1 -- 172.21.15.133:0/3840940566 wait complete. 2024-04-25T10:13:01.125 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:13:01.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec0a9a3640 1 Processor -- start 2024-04-25T10:13:01.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec0a9a3640 1 -- start start 2024-04-25T10:13:01.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec0a9a3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fec0415d9d0 0x7fec040b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec0a9a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec040d3360 0x7fec040ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec0a9a3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec040d22c0 con 0x7fec040d1d00 2024-04-25T10:13:01.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec0a9a3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec040d25a0 con 0x7fec0415d9d0 2024-04-25T10:13:01.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec0a9a3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec040d2880 con 0x7fec040d3360 2024-04-25T10:13:01.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec03fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fec0415d9d0 0x7fec040b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec08f40640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fec040d1d00 0x7fec040b53b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35358/0 (socket says 172.21.15.133:35358) 2024-04-25T10:13:01.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec08f40640 1 -- 172.21.15.133:0/1409988527 learned_addr learned my addr 172.21.15.133:0/1409988527 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:01.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec037fe640 1 --2- 172.21.15.133:0/1409988527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec040d3360 0x7fec040ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec03fff640 1 -- 172.21.15.133:0/1409988527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec040d3360 msgr2=0x7fec040ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec03fff640 1 --2- 172.21.15.133:0/1409988527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec040d3360 0x7fec040ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec03fff640 1 -- 172.21.15.133:0/1409988527 >> v1:172.21.15.133:6789/0 conn(0x7fec040d1d00 legacy=0x7fec040b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.248+0000 7fec03fff640 1 -- 172.21.15.133:0/1409988527 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec040d2b60 con 0x7fec0415d9d0 2024-04-25T10:13:01.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec03fff640 1 --2- 172.21.15.133:0/1409988527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec0415d9d0 0x7fec040b92e0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7febfc009dc0 tx=0x7febfc01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff72e3062165cb8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec02ffd640 1 -- 172.21.15.133:0/1409988527 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc0b2070 con 0x7fec0415d9d0 2024-04-25T10:13:01.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec02ffd640 1 -- 172.21.15.133:0/1409988527 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc0a5810 con 0x7fec0415d9d0 2024-04-25T10:13:01.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec02ffd640 1 -- 172.21.15.133:0/1409988527 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc0a5be0 con 0x7fec0415d9d0 2024-04-25T10:13:01.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1409988527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec0415d9d0 msgr2=0x7fec040b92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec0a9a3640 1 --2- 172.21.15.133:0/1409988527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec0415d9d0 0x7fec040b92e0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7febfc009dc0 tx=0x7febfc01c950 comp rx=0 tx=0).stop 2024-04-25T10:13:01.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1409988527 shutdown_connections 2024-04-25T10:13:01.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec0a9a3640 1 --2- 172.21.15.133:0/1409988527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec040d3360 0x7fec040ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec0a9a3640 1 --2- 172.21.15.133:0/1409988527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec0415d9d0 0x7fec040b92e0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1409988527 >> 172.21.15.133:0/1409988527 conn(0x7fec0406c080 msgr2=0x7fec0406c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1409988527 shutdown_connections 2024-04-25T10:13:01.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1409988527 wait complete. 2024-04-25T10:13:01.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec0a9a3640 1 Processor -- start 2024-04-25T10:13:01.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.252+0000 7fec0a9a3640 1 -- start start 2024-04-25T10:13:01.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec0a9a3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec040d1d00 0x7fec0415a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec0a9a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040d3360 0x7fec040c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec0a9a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec0415d9d0 0x7fec04163a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec0a9a3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0416a760 con 0x7fec040d3360 2024-04-25T10:13:01.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec0a9a3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0416a5e0 con 0x7fec040d1d00 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec0a9a3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0416a8e0 con 0x7fec0415d9d0 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec037fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040d3360 0x7fec040c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec037fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040d3360 0x7fec040c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52654/0 (socket says 172.21.15.133:52654) 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec037fe640 1 -- 172.21.15.133:0/1410168331 learned_addr learned my addr 172.21.15.133:0/1410168331 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec03fff640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec040d1d00 0x7fec0415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec08f40640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec0415d9d0 0x7fec04163a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec037fe640 1 -- 172.21.15.133:0/1410168331 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec0415d9d0 msgr2=0x7fec04163a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec037fe640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec0415d9d0 0x7fec04163a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec037fe640 1 -- 172.21.15.133:0/1410168331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec040d1d00 msgr2=0x7fec0415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec037fe640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec040d1d00 0x7fec0415a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec037fe640 1 -- 172.21.15.133:0/1410168331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec04157ae0 con 0x7fec040d3360 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec037fe640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040d3360 0x7fec040c9dc0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7febf800d7d0 tx=0x7febf800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec00ff9640 1 -- 172.21.15.133:0/1410168331 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8004020 con 0x7fec040d3360 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec00ff9640 1 -- 172.21.15.133:0/1410168331 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf80041c0 con 0x7fec040d3360 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec00ff9640 1 -- 172.21.15.133:0/1410168331 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf80043b0 con 0x7fec040d3360 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1410168331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0415a790 con 0x7fec040d3360 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1410168331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0415ad50 con 0x7fec040d3360 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1410168331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec04069000 con 0x7fec040d3360 2024-04-25T10:13:01.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec00ff9640 1 -- 172.21.15.133:0/1410168331 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7febf801b020 con 0x7fec040d3360 2024-04-25T10:13:01.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec00ff9640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf0066f60 0x7febf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec00ff9640 1 -- 172.21.15.133:0/1410168331 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5749+0+0 (secure 0 0 0) 0x7febf8084fb0 con 0x7fec040d3360 2024-04-25T10:13:01.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec03fff640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf0066f60 0x7febf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.256+0000 7fec03fff640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf0066f60 0x7febf0069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7febfc0af000 tx=0x7febfc0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.260+0000 7fec00ff9640 1 -- 172.21.15.133:0/1410168331 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec04157ae0 con 0x7fec040d3360 2024-04-25T10:13:01.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:01.416+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1410168331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fec04073b10 con 0x7fec040d3360 2024-04-25T10:13:02.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.104+0000 7fec00ff9640 1 -- 172.21.15.133:0/1410168331 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) v1 ==== 169+0+0 (secure 0 0 0) 0x7febf8055e60 con 0x7fec040d3360 2024-04-25T10:13:02.107 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1410168331 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf0066f60 msgr2=0x7febf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf0066f60 0x7febf0069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7febfc0af000 tx=0x7febfc0a7070 comp rx=0 tx=0).stop 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1410168331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040d3360 msgr2=0x7fec040c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040d3360 0x7fec040c9dc0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7febf800d7d0 tx=0x7febf800dc90 comp rx=0 tx=0).stop 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1410168331 shutdown_connections 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec0415d9d0 0x7fec04163a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf0066f60 0x7febf0069420 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040d3360 0x7fec040c9dc0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 --2- 172.21.15.133:0/1410168331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec040d1d00 0x7fec0415a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1410168331 >> 172.21.15.133:0/1410168331 conn(0x7fec0406c080 msgr2=0x7fec041600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1410168331 shutdown_connections 2024-04-25T10:13:02.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.108+0000 7fec0a9a3640 1 -- 172.21.15.133:0/1410168331 wait complete. 2024-04-25T10:13:02.141 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:13:02.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.296+0000 7fa357864640 1 Processor -- start 2024-04-25T10:13:02.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.296+0000 7fa357864640 1 -- start start 2024-04-25T10:13:02.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.296+0000 7fa357864640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa3500b2150 0x7fa35006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.296+0000 7fa357864640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35015b9b0 0x7fa35006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.296+0000 7fa355600640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fa350143ea0 0x7fa3500cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:48272/0 (socket says 172.21.15.133:48272) 2024-04-25T10:13:02.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.296+0000 7fa355600640 1 -- 172.21.15.133:0/4098743556 learned_addr learned my addr 172.21.15.133:0/4098743556 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:02.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.296+0000 7fa355e01640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35015b9b0 0x7fa35006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.296+0000 7fa357864640 1 -- 172.21.15.133:0/4098743556 --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35015ac50 con 0x7fa350143ea0 2024-04-25T10:13:02.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.296+0000 7fa357864640 1 -- 172.21.15.133:0/4098743556 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3500ccf20 con 0x7fa3500b2150 2024-04-25T10:13:02.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.296+0000 7fa357864640 1 -- 172.21.15.133:0/4098743556 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3500cd180 con 0x7fa35015b9b0 2024-04-25T10:13:02.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6392092 0 0) 0x7fa34c001120 con 0x7fa350143ea0 2024-04-25T10:13:02.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa33c003530 con 0x7fa350143ea0 2024-04-25T10:13:02.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa354dff640 1 --2- 172.21.15.133:0/4098743556 >> v2:172.21.15.133:3300/0 conn(0x7fa3500b2150 0x7fa35006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 <== mon.1 v1:172.21.15.162:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3129188899 0 0) 0x7fa33c003530 con 0x7fa350143ea0 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35015b9b0 msgr2=0x7fa35006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35015b9b0 0x7fa35006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 >> v2:172.21.15.133:3300/0 conn(0x7fa3500b2150 msgr2=0x7fa35006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 --2- 172.21.15.133:0/4098743556 >> v2:172.21.15.133:3300/0 conn(0x7fa3500b2150 0x7fa35006e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 --> v1:172.21.15.162:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35015aed0 con 0x7fa350143ea0 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 <== mon.1 v1:172.21.15.162:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa34c001850 con 0x7fa350143ea0 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 >> v1:172.21.15.162:6789/0 conn(0x7fa350143ea0 legacy=0x7fa3500cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35015b9b0 0x7fa33c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa33c003f40 0x7fa33c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 0x7fa33c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3500ccf20 con 0x7fa33c004d50 2024-04-25T10:13:02.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3500cd180 con 0x7fa33c003f40 2024-04-25T10:13:02.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c00b9d0 con 0x7fa35015b9b0 2024-04-25T10:13:02.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa354dff640 1 --2- 172.21.15.133:0/4098743556 >> v2:172.21.15.133:3300/0 conn(0x7fa3500b2150 0x7fa35006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:02.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa355e01640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 0x7fa33c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa354dff640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa33c003f40 0x7fa33c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa355600640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35015b9b0 0x7fa33c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa355e01640 1 -- 172.21.15.133:0/4098743556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35015b9b0 msgr2=0x7fa33c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa355e01640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35015b9b0 0x7fa33c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa355e01640 1 -- 172.21.15.133:0/4098743556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa33c003f40 msgr2=0x7fa33c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa355e01640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa33c003f40 0x7fa33c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.300+0000 7fa355e01640 1 -- 172.21.15.133:0/4098743556 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35015aed0 con 0x7fa33c004d50 2024-04-25T10:13:02.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.304+0000 7fa355e01640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 0x7fa33c0051d0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fa35015bdc0 tx=0x7fa34801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9990e7dd0a8b85f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.304+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3480b2070 con 0x7fa33c004d50 2024-04-25T10:13:02.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.304+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3480a5c30 con 0x7fa33c004d50 2024-04-25T10:13:02.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.304+0000 7fa347fff640 1 -- 172.21.15.133:0/4098743556 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348009c10 con 0x7fa33c004d50 2024-04-25T10:13:02.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.304+0000 7fa357864640 1 -- 172.21.15.133:0/4098743556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 msgr2=0x7fa33c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.304+0000 7fa357864640 1 --2- 172.21.15.133:0/4098743556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 0x7fa33c0051d0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fa35015bdc0 tx=0x7fa34801cbc0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.304+0000 7fa355e01640 1 -- 172.21.15.133:0/4098743556 reap_dead start 2024-04-25T10:13:02.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.304+0000 7fa357864640 1 -- 172.21.15.133:0/4098743556 shutdown_connections 2024-04-25T10:13:02.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.304+0000 7fa357864640 1 -- 172.21.15.133:0/4098743556 >> 172.21.15.133:0/4098743556 conn(0x7fa350066f70 msgr2=0x7fa35006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.304+0000 7fa357864640 1 -- 172.21.15.133:0/4098743556 shutdown_connections 2024-04-25T10:13:02.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.304+0000 7fa357864640 1 -- 172.21.15.133:0/4098743556 wait complete. 2024-04-25T10:13:02.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.308+0000 7fa357864640 1 Processor -- start 2024-04-25T10:13:02.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.308+0000 7fa357864640 1 -- start start 2024-04-25T10:13:02.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.308+0000 7fa357864640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 0x7fa3501f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.308+0000 7fa357864640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35015a160 0x7fa35015de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.308+0000 7fa357864640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35013d310 0x7fa35015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.308+0000 7fa357864640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa350144280 con 0x7fa33c004d50 2024-04-25T10:13:02.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.308+0000 7fa357864640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35015b490 con 0x7fa35015a160 2024-04-25T10:13:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.308+0000 7fa357864640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa350144400 con 0x7fa35013d310 2024-04-25T10:13:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa355600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 0x7fa3501f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa355600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 0x7fa3501f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52678/0 (socket says 172.21.15.133:52678) 2024-04-25T10:13:02.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa355600640 1 -- 172.21.15.133:0/1963954703 learned_addr learned my addr 172.21.15.133:0/1963954703 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa354dff640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35015a160 0x7fa35015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa355e01640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35013d310 0x7fa35015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa355600640 1 -- 172.21.15.133:0/1963954703 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35013d310 msgr2=0x7fa35015e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa355600640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35013d310 0x7fa35015e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa355600640 1 -- 172.21.15.133:0/1963954703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35015a160 msgr2=0x7fa35015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa355600640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35015a160 0x7fa35015de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa355600640 1 -- 172.21.15.133:0/1963954703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3501f7830 con 0x7fa33c004d50 2024-04-25T10:13:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa354dff640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35015a160 0x7fa35015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa355600640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 0x7fa3501f72f0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fa34c001c50 tx=0x7fa34c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa345ffb640 1 -- 172.21.15.133:0/1963954703 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00e8c0 con 0x7fa33c004d50 2024-04-25T10:13:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa345ffb640 1 -- 172.21.15.133:0/1963954703 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c018070 con 0x7fa33c004d50 2024-04-25T10:13:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa345ffb640 1 -- 172.21.15.133:0/1963954703 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00eca0 con 0x7fa33c004d50 2024-04-25T10:13:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa357864640 1 -- 172.21.15.133:0/1963954703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35015a5f0 con 0x7fa33c004d50 2024-04-25T10:13:02.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa357864640 1 -- 172.21.15.133:0/1963954703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35013ac90 con 0x7fa33c004d50 2024-04-25T10:13:02.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.312+0000 7fa357864640 1 -- 172.21.15.133:0/1963954703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa350068e80 con 0x7fa33c004d50 2024-04-25T10:13:02.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.316+0000 7fa345ffb640 1 -- 172.21.15.133:0/1963954703 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa34c01d460 con 0x7fa33c004d50 2024-04-25T10:13:02.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.316+0000 7fa345ffb640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa334066f60 0x7fa334069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.316+0000 7fa345ffb640 1 -- 172.21.15.133:0/1963954703 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa34c085020 con 0x7fa33c004d50 2024-04-25T10:13:02.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.316+0000 7fa354dff640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa334066f60 0x7fa334069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.316+0000 7fa354dff640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa334066f60 0x7fa334069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa350155090 tx=0x7fa340008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.324+0000 7fa345ffb640 1 -- 172.21.15.133:0/1963954703 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa35015ac50 con 0x7fa33c004d50 2024-04-25T10:13:02.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.492+0000 7fa357864640 1 -- 172.21.15.133:0/1963954703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa3500b10a0 con 0x7fa33c004d50 2024-04-25T10:13:02.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.496+0000 7fa345ffb640 1 -- 172.21.15.133:0/1963954703 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa34c056260 con 0x7fa33c004d50 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 -- 172.21.15.133:0/1963954703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa334066f60 msgr2=0x7fa334069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa334066f60 0x7fa334069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa350155090 tx=0x7fa340008040 comp rx=0 tx=0).stop 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 -- 172.21.15.133:0/1963954703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 msgr2=0x7fa3501f72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 0x7fa3501f72f0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fa34c001c50 tx=0x7fa34c0077e0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 -- 172.21.15.133:0/1963954703 shutdown_connections 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35013d310 0x7fa35015e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35015a160 0x7fa35015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa334066f60 0x7fa334069420 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 --2- 172.21.15.133:0/1963954703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa33c004d50 0x7fa3501f72f0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 -- 172.21.15.133:0/1963954703 >> 172.21.15.133:0/1963954703 conn(0x7fa350066f70 msgr2=0x7fa3500b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 -- 172.21.15.133:0/1963954703 shutdown_connections 2024-04-25T10:13:02.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.500+0000 7fa357864640 1 -- 172.21.15.133:0/1963954703 wait complete. 2024-04-25T10:13:02.520 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:13:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87ff062640 1 Processor -- start 2024-04-25T10:13:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87ff062640 1 -- start start 2024-04-25T10:13:02.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87ff062640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87f815c9a0 0x7f87f80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87ff062640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87f80b1f70 0x7f87f806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87ff062640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f80c61d0 con 0x7f87f80ce330 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87ff062640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f80c64b0 con 0x7f87f815c9a0 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87ff062640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f806d4f0 con 0x7f87f80b1f70 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87fcdfe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87f815c9a0 0x7f87f80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87fd5ff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f87f80ce330 0x7f87f815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35360/0 (socket says 172.21.15.133:35360) 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87fd5ff640 1 -- 172.21.15.133:0/481905599 learned_addr learned my addr 172.21.15.133:0/481905599 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87f7fff640 1 --2- 172.21.15.133:0/481905599 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87f80b1f70 0x7f87f806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87f77fe640 1 -- 172.21.15.133:0/481905599 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835589372 0 0) 0x7f87f80c61d0 con 0x7f87f80ce330 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87f77fe640 1 -- 172.21.15.133:0/481905599 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87e8003530 con 0x7f87f80ce330 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87fcdfe640 1 -- 172.21.15.133:0/481905599 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87f80b1f70 msgr2=0x7f87f806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87fcdfe640 1 --2- 172.21.15.133:0/481905599 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87f80b1f70 0x7f87f806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87fcdfe640 1 -- 172.21.15.133:0/481905599 >> v1:172.21.15.133:6789/0 conn(0x7f87f80ce330 legacy=0x7f87f815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.636+0000 7f87fcdfe640 1 -- 172.21.15.133:0/481905599 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f806d7d0 con 0x7f87f815c9a0 2024-04-25T10:13:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87fcdfe640 1 --2- 172.21.15.133:0/481905599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f815c9a0 0x7f87f80c5980 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0097c0 tx=0x7f87ec01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac9e4688e3ee36e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87f77fe640 1 -- 172.21.15.133:0/481905599 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0b0070 con 0x7f87f815c9a0 2024-04-25T10:13:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87f77fe640 1 -- 172.21.15.133:0/481905599 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ec004230 con 0x7f87f815c9a0 2024-04-25T10:13:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87f77fe640 1 -- 172.21.15.133:0/481905599 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec003620 con 0x7f87f815c9a0 2024-04-25T10:13:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87ff062640 1 -- 172.21.15.133:0/481905599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f815c9a0 msgr2=0x7f87f80c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87ff062640 1 --2- 172.21.15.133:0/481905599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f815c9a0 0x7f87f80c5980 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0097c0 tx=0x7f87ec01c440 comp rx=0 tx=0).stop 2024-04-25T10:13:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87ff062640 1 -- 172.21.15.133:0/481905599 shutdown_connections 2024-04-25T10:13:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87ff062640 1 --2- 172.21.15.133:0/481905599 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87f80b1f70 0x7f87f806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87ff062640 1 --2- 172.21.15.133:0/481905599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f815c9a0 0x7f87f80c5980 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87ff062640 1 -- 172.21.15.133:0/481905599 >> 172.21.15.133:0/481905599 conn(0x7f87f8067000 msgr2=0x7f87f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87ff062640 1 -- 172.21.15.133:0/481905599 shutdown_connections 2024-04-25T10:13:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87ff062640 1 -- 172.21.15.133:0/481905599 wait complete. 2024-04-25T10:13:02.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87ff062640 1 Processor -- start 2024-04-25T10:13:02.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.640+0000 7f87ff062640 1 -- start start 2024-04-25T10:13:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87ff062640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87f80b1f70 0x7f87f815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87ff062640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87f80ce330 0x7f87f8155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87ff062640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f815c9a0 0x7f87f80cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87ff062640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f80ceb50 con 0x7f87f815c9a0 2024-04-25T10:13:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87ff062640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f815cd80 con 0x7f87f80ce330 2024-04-25T10:13:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87ff062640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f81471f0 con 0x7f87f80b1f70 2024-04-25T10:13:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fd5ff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f815c9a0 0x7f87f80cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fd5ff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f815c9a0 0x7f87f80cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52700/0 (socket says 172.21.15.133:52700) 2024-04-25T10:13:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fd5ff640 1 -- 172.21.15.133:0/1335646511 learned_addr learned my addr 172.21.15.133:0/1335646511 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87f7fff640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87f80ce330 0x7f87f8155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fcdfe640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87f80b1f70 0x7f87f815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fd5ff640 1 -- 172.21.15.133:0/1335646511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87f80b1f70 msgr2=0x7f87f815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fd5ff640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87f80b1f70 0x7f87f815e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fd5ff640 1 -- 172.21.15.133:0/1335646511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87f80ce330 msgr2=0x7f87f8155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fd5ff640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87f80ce330 0x7f87f8155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fd5ff640 1 -- 172.21.15.133:0/1335646511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f8146950 con 0x7f87f815c9a0 2024-04-25T10:13:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fd5ff640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f815c9a0 0x7f87f80cd9c0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f87f000caa0 tx=0x7f87f000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87f57fa640 1 -- 172.21.15.133:0/1335646511 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0017070 con 0x7f87f815c9a0 2024-04-25T10:13:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87f57fa640 1 -- 172.21.15.133:0/1335646511 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f000ac00 con 0x7f87f815c9a0 2024-04-25T10:13:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87ff062640 1 -- 172.21.15.133:0/1335646511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f8146c30 con 0x7f87f815c9a0 2024-04-25T10:13:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87f57fa640 1 -- 172.21.15.133:0/1335646511 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0012400 con 0x7f87f815c9a0 2024-04-25T10:13:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87ff062640 1 -- 172.21.15.133:0/1335646511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f8146570 con 0x7f87f815c9a0 2024-04-25T10:13:02.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87ff062640 1 -- 172.21.15.133:0/1335646511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f806bc20 con 0x7f87f815c9a0 2024-04-25T10:13:02.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87f57fa640 1 -- 172.21.15.133:0/1335646511 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87f001a070 con 0x7f87f815c9a0 2024-04-25T10:13:02.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87f57fa640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87dc066f60 0x7f87dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:02.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87f57fa640 1 -- 172.21.15.133:0/1335646511 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f87f0084c40 con 0x7f87f815c9a0 2024-04-25T10:13:02.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fcdfe640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87dc066f60 0x7f87dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:02.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.644+0000 7f87fcdfe640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87dc066f60 0x7f87dc069420 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0ad000 tx=0x7f87ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.648+0000 7f87f57fa640 1 -- 172.21.15.133:0/1335646511 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87f0055dd0 con 0x7f87f815c9a0 2024-04-25T10:13:02.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:02.868+0000 7f87ff062640 1 -- 172.21.15.133:0/1335646511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f87f8068ef0 con 0x7f87f815c9a0 2024-04-25T10:13:03.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.104+0000 7f87f57fa640 1 -- 172.21.15.133:0/1335646511 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v504) v1 ==== 166+0+0 (secure 0 0 0) 0x7f87f00556d0 con 0x7f87f815c9a0 2024-04-25T10:13:03.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 -- 172.21.15.133:0/1335646511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87dc066f60 msgr2=0x7f87dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87dc066f60 0x7f87dc069420 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0ad000 tx=0x7f87ec008040 comp rx=0 tx=0).stop 2024-04-25T10:13:03.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 -- 172.21.15.133:0/1335646511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f815c9a0 msgr2=0x7f87f80cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f815c9a0 0x7f87f80cd9c0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f87f000caa0 tx=0x7f87f000cf60 comp rx=0 tx=0).stop 2024-04-25T10:13:03.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 -- 172.21.15.133:0/1335646511 shutdown_connections 2024-04-25T10:13:03.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f87dc066f60 0x7f87dc069420 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f815c9a0 0x7f87f80cd9c0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f87f80ce330 0x7f87f8155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 --2- 172.21.15.133:0/1335646511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f87f80b1f70 0x7f87f815e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 -- 172.21.15.133:0/1335646511 >> 172.21.15.133:0/1335646511 conn(0x7f87f8067000 msgr2=0x7f87f80b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 -- 172.21.15.133:0/1335646511 shutdown_connections 2024-04-25T10:13:03.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.108+0000 7f87ff062640 1 -- 172.21.15.133:0/1335646511 wait complete. 2024-04-25T10:13:03.133 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:13:03.133 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:13:03.134 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:03.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.252+0000 7f1612db1640 1 Processor -- start 2024-04-25T10:13:03.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.252+0000 7f1612db1640 1 -- start start 2024-04-25T10:13:03.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f1612db1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f160c15dd90 0x7f160c0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f1612db1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160c0ca9b0 con 0x7f160c0b19f0 2024-04-25T10:13:03.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f1612db1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160c0cac90 con 0x7f160c15d9b0 2024-04-25T10:13:03.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f1612db1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f160c0cbe30 con 0x7f160c15dd90 2024-04-25T10:13:03.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f160bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f160c15dd90 0x7f160c0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f160bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f160c15dd90 0x7f160c0ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52716/0 (socket says 172.21.15.133:52716) 2024-04-25T10:13:03.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f160bfff640 1 -- 172.21.15.133:0/3615072759 learned_addr learned my addr 172.21.15.133:0/3615072759 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:03.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f160bfff640 1 -- 172.21.15.133:0/3615072759 >> v1:172.21.15.162:6789/0 conn(0x7f160c15d9b0 legacy=0x7f160c0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f160bfff640 1 -- 172.21.15.133:0/3615072759 >> v1:172.21.15.133:6789/0 conn(0x7f160c0b19f0 legacy=0x7f160c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f160b7fe640 1 -- 172.21.15.133:0/3615072759 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660982612 0 0) 0x7f160c0ca9b0 con 0x7f160c0b19f0 2024-04-25T10:13:03.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f160bfff640 1 -- 172.21.15.133:0/3615072759 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c0caf70 con 0x7f160c15dd90 2024-04-25T10:13:03.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f160bfff640 1 --2- 172.21.15.133:0/3615072759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160c15dd90 0x7f160c0ce770 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f1600009870 tx=0x7f160001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0253123f3147dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f160b7fe640 1 -- 172.21.15.133:0/3615072759 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16000aa610 con 0x7f160c15dd90 2024-04-25T10:13:03.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f160b7fe640 1 -- 172.21.15.133:0/3615072759 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16000af070 con 0x7f160c15dd90 2024-04-25T10:13:03.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.256+0000 7f160b7fe640 1 -- 172.21.15.133:0/3615072759 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16000aa9d0 con 0x7f160c15dd90 2024-04-25T10:13:03.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 -- 172.21.15.133:0/3615072759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160c15dd90 msgr2=0x7f160c0ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 --2- 172.21.15.133:0/3615072759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160c15dd90 0x7f160c0ce770 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f1600009870 tx=0x7f160001c490 comp rx=0 tx=0).stop 2024-04-25T10:13:03.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 -- 172.21.15.133:0/3615072759 shutdown_connections 2024-04-25T10:13:03.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 --2- 172.21.15.133:0/3615072759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160c15dd90 0x7f160c0ce770 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 -- 172.21.15.133:0/3615072759 >> 172.21.15.133:0/3615072759 conn(0x7f160c067000 msgr2=0x7f160c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 -- 172.21.15.133:0/3615072759 shutdown_connections 2024-04-25T10:13:03.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 -- 172.21.15.133:0/3615072759 wait complete. 2024-04-25T10:13:03.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 Processor -- start 2024-04-25T10:13:03.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 -- start start 2024-04-25T10:13:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f160c0b19f0 0x7f160c15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160c15d9b0 0x7f160c14a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f160c15dd90 0x7f160c14fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c06ea20 con 0x7f160c15d9b0 2024-04-25T10:13:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c06e8a0 con 0x7f160c0b19f0 2024-04-25T10:13:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c16b870 con 0x7f160c15dd90 2024-04-25T10:13:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f160bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160c15d9b0 0x7f160c14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f160bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160c15d9b0 0x7f160c14a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52732/0 (socket says 172.21.15.133:52732) 2024-04-25T10:13:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f160bfff640 1 -- 172.21.15.133:0/1193714732 learned_addr learned my addr 172.21.15.133:0/1193714732 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f160bfff640 1 -- 172.21.15.133:0/1193714732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f160c15dd90 msgr2=0x7f160c14fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f161134e640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f160c15dd90 0x7f160c14fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1610b4d640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f160c0b19f0 0x7f160c15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f160bfff640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f160c15dd90 0x7f160c14fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f160bfff640 1 -- 172.21.15.133:0/1193714732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f160c0b19f0 msgr2=0x7f160c15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f160bfff640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f160c0b19f0 0x7f160c15cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f160bfff640 1 -- 172.21.15.133:0/1193714732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c14d360 con 0x7f160c15d9b0 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1610b4d640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f160c0b19f0 0x7f160c15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f160bfff640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160c15d9b0 0x7f160c14a780 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f16000190a0 tx=0x7f160001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f16097fa640 1 -- 172.21.15.133:0/1193714732 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16000b38c0 con 0x7f160c15d9b0 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f16097fa640 1 -- 172.21.15.133:0/1193714732 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16000b3a60 con 0x7f160c15d9b0 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 -- 172.21.15.133:0/1193714732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160c14d080 con 0x7f160c15d9b0 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f16097fa640 1 -- 172.21.15.133:0/1193714732 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16000b3e50 con 0x7f160c15d9b0 2024-04-25T10:13:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.260+0000 7f1612db1640 1 -- 172.21.15.133:0/1193714732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160c1623f0 con 0x7f160c15d9b0 2024-04-25T10:13:03.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.264+0000 7f1612db1640 1 -- 172.21.15.133:0/1193714732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f160c068ef0 con 0x7f160c15d9b0 2024-04-25T10:13:03.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.264+0000 7f16097fa640 1 -- 172.21.15.133:0/1193714732 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16000c6430 con 0x7f160c15d9b0 2024-04-25T10:13:03.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.264+0000 7f16097fa640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f15f8067130 0x7f15f80695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.264+0000 7f16097fa640 1 -- 172.21.15.133:0/1193714732 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f160011fab0 con 0x7f160c15d9b0 2024-04-25T10:13:03.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.264+0000 7f1610b4d640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f15f8067130 0x7f15f80695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.264+0000 7f1610b4d640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f15f8067130 0x7f15f80695f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f160c150d10 tx=0x7f15fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.268+0000 7f16097fa640 1 -- 172.21.15.133:0/1193714732 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f160c14d360 con 0x7f160c15d9b0 2024-04-25T10:13:03.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.428+0000 7f1612db1640 1 -- 172.21.15.133:0/1193714732 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f160c0ce350 con 0x7f160c15d9b0 2024-04-25T10:13:03.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.428+0000 7f16097fa640 1 -- 172.21.15.133:0/1193714732 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+13329 (secure 0 0 0) 0x7f16000a7070 con 0x7f160c15d9b0 2024-04-25T10:13:03.487 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:03.487 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":123,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:13:02.095907+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/4218771669":"2024-04-26T10:12:58.732212+0000","172.21.15.133:6835/4218771669":"2024-04-26T10:12:58.732212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 -- 172.21.15.133:0/1193714732 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f15f8067130 msgr2=0x7f15f80695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f15f8067130 0x7f15f80695f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f160c150d10 tx=0x7f15fc009040 comp rx=0 tx=0).stop 2024-04-25T10:13:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 -- 172.21.15.133:0/1193714732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160c15d9b0 msgr2=0x7f160c14a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160c15d9b0 0x7f160c14a780 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f16000190a0 tx=0x7f160001acd0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 -- 172.21.15.133:0/1193714732 shutdown_connections 2024-04-25T10:13:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f160c15dd90 0x7f160c14fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f15f8067130 0x7f15f80695f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f160c15d9b0 0x7f160c14a780 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 --2- 172.21.15.133:0/1193714732 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f160c0b19f0 0x7f160c15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 -- 172.21.15.133:0/1193714732 >> 172.21.15.133:0/1193714732 conn(0x7f160c067000 msgr2=0x7f160c151140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 -- 172.21.15.133:0/1193714732 shutdown_connections 2024-04-25T10:13:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.432+0000 7f1612db1640 1 -- 172.21.15.133:0/1193714732 wait complete. 2024-04-25T10:13:03.490 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:03.490 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:03.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8fa034640 1 Processor -- start 2024-04-25T10:13:03.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8fa034640 1 -- start start 2024-04-25T10:13:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8fa034640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f4143ea0 0x7fb8f40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8fa034640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb8f40b2150 0x7fb8f406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8fa034640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb8f415b9b0 0x7fb8f406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8fa034640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f415ac50 con 0x7fb8f415b9b0 2024-04-25T10:13:03.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8fa034640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f40cd180 con 0x7fb8f40b2150 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8fa034640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f415aed0 con 0x7fb8f4143ea0 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8f3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb8f415b9b0 0x7fb8f406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8f3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb8f415b9b0 0x7fb8f406e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52736/0 (socket says 172.21.15.133:52736) 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8f3fff640 1 -- 172.21.15.133:0/2516856826 learned_addr learned my addr 172.21.15.133:0/2516856826 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8f37fe640 1 --2- 172.21.15.133:0/2516856826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f4143ea0 0x7fb8f40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8ebfff640 1 --2- 172.21.15.133:0/2516856826 >> v2:172.21.15.162:3300/0 conn(0x7fb8f40b2150 0x7fb8f406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8f3fff640 1 -- 172.21.15.133:0/2516856826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f4143ea0 msgr2=0x7fb8f40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8f3fff640 1 --2- 172.21.15.133:0/2516856826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f4143ea0 0x7fb8f40cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8f3fff640 1 -- 172.21.15.133:0/2516856826 >> v2:172.21.15.162:3300/0 conn(0x7fb8f40b2150 msgr2=0x7fb8f406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8f3fff640 1 --2- 172.21.15.133:0/2516856826 >> v2:172.21.15.162:3300/0 conn(0x7fb8f40b2150 0x7fb8f406e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8f3fff640 1 -- 172.21.15.133:0/2516856826 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f415b1b0 con 0x7fb8f415b9b0 2024-04-25T10:13:03.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8ebfff640 1 --2- 172.21.15.133:0/2516856826 >> v2:172.21.15.162:3300/0 conn(0x7fb8f40b2150 0x7fb8f406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.616+0000 7fb8f3fff640 1 --2- 172.21.15.133:0/2516856826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f415b9b0 0x7fb8f406e8f0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fb8f406eff0 tx=0x7fb8ec01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799c518224e71c36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f2ffd640 1 -- 172.21.15.133:0/2516856826 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00b2d0 con 0x7fb8f415b9b0 2024-04-25T10:13:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f2ffd640 1 -- 172.21.15.133:0/2516856826 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec00b470 con 0x7fb8f415b9b0 2024-04-25T10:13:03.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f2ffd640 1 -- 172.21.15.133:0/2516856826 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0a7bb0 con 0x7fb8f415b9b0 2024-04-25T10:13:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 -- 172.21.15.133:0/2516856826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f415b9b0 msgr2=0x7fb8f406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 --2- 172.21.15.133:0/2516856826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f415b9b0 0x7fb8f406e8f0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fb8f406eff0 tx=0x7fb8ec01ebf0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 -- 172.21.15.133:0/2516856826 shutdown_connections 2024-04-25T10:13:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 --2- 172.21.15.133:0/2516856826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f415b9b0 0x7fb8f406e8f0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 --2- 172.21.15.133:0/2516856826 >> v2:172.21.15.162:3300/0 conn(0x7fb8f40b2150 0x7fb8f406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 --2- 172.21.15.133:0/2516856826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f4143ea0 0x7fb8f40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 -- 172.21.15.133:0/2516856826 >> 172.21.15.133:0/2516856826 conn(0x7fb8f4066f70 msgr2=0x7fb8f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 -- 172.21.15.133:0/2516856826 shutdown_connections 2024-04-25T10:13:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 -- 172.21.15.133:0/2516856826 wait complete. 2024-04-25T10:13:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 Processor -- start 2024-04-25T10:13:03.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 -- start start 2024-04-25T10:13:03.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8f40b2150 0x7fb8f41544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f4143ea0 0x7fb8f41f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f415b9b0 0x7fb8f413f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f415ede0 con 0x7fb8f415b9b0 2024-04-25T10:13:03.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f415ec60 con 0x7fb8f40b2150 2024-04-25T10:13:03.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8fa034640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f415ef60 con 0x7fb8f4143ea0 2024-04-25T10:13:03.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f415b9b0 0x7fb8f413f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f415b9b0 0x7fb8f413f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52738/0 (socket says 172.21.15.133:52738) 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f3fff640 1 -- 172.21.15.133:0/1420952824 learned_addr learned my addr 172.21.15.133:0/1420952824 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8ebfff640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f4143ea0 0x7fb8f41f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f3fff640 1 -- 172.21.15.133:0/1420952824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f4143ea0 msgr2=0x7fb8f41f9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f3fff640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f4143ea0 0x7fb8f41f9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f37fe640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8f40b2150 0x7fb8f41544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f3fff640 1 -- 172.21.15.133:0/1420952824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8f40b2150 msgr2=0x7fb8f41544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f3fff640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8f40b2150 0x7fb8f41544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f3fff640 1 -- 172.21.15.133:0/1420952824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f4151d70 con 0x7fb8f415b9b0 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8ebfff640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f4143ea0 0x7fb8f41f9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f3fff640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f415b9b0 0x7fb8f413f8f0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0b1040 tx=0x7fb8ec01b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f0ff9640 1 -- 172.21.15.133:0/1420952824 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec009600 con 0x7fb8f415b9b0 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f0ff9640 1 -- 172.21.15.133:0/1420952824 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec0097c0 con 0x7fb8f415b9b0 2024-04-25T10:13:03.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.620+0000 7fb8f0ff9640 1 -- 172.21.15.133:0/1420952824 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec009bb0 con 0x7fb8f415b9b0 2024-04-25T10:13:03.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.624+0000 7fb8fa034640 1 -- 172.21.15.133:0/1420952824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f41fa700 con 0x7fb8f415b9b0 2024-04-25T10:13:03.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.624+0000 7fb8fa034640 1 -- 172.21.15.133:0/1420952824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f41facc0 con 0x7fb8f415b9b0 2024-04-25T10:13:03.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.624+0000 7fb8fa034640 1 -- 172.21.15.133:0/1420952824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f4068e80 con 0x7fb8f415b9b0 2024-04-25T10:13:03.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.624+0000 7fb8f0ff9640 1 -- 172.21.15.133:0/1420952824 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8ec0c0030 con 0x7fb8f415b9b0 2024-04-25T10:13:03.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.624+0000 7fb8f0ff9640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8d4066f60 0x7fb8d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.624+0000 7fb8f0ff9640 1 -- 172.21.15.133:0/1420952824 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb8ec11fcf0 con 0x7fb8f415b9b0 2024-04-25T10:13:03.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.624+0000 7fb8f37fe640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8d4066f60 0x7fb8d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.624+0000 7fb8f37fe640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8d4066f60 0x7fb8d4069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009710 tx=0x7fb8e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.628+0000 7fb8f0ff9640 1 -- 172.21.15.133:0/1420952824 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8ec0c7020 con 0x7fb8f415b9b0 2024-04-25T10:13:03.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.776+0000 7fb8fa034640 1 -- 172.21.15.133:0/1420952824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8f40cb740 con 0x7fb8f415b9b0 2024-04-25T10:13:03.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.776+0000 7fb8f0ff9640 1 -- 172.21.15.133:0/1420952824 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8ec0187e0 con 0x7fb8f415b9b0 2024-04-25T10:13:03.780 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:13:03.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 -- 172.21.15.133:0/1420952824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8d4066f60 msgr2=0x7fb8d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8d4066f60 0x7fb8d4069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009710 tx=0x7fb8e4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:03.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 -- 172.21.15.133:0/1420952824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f415b9b0 msgr2=0x7fb8f413f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f415b9b0 0x7fb8f413f8f0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0b1040 tx=0x7fb8ec01b990 comp rx=0 tx=0).stop 2024-04-25T10:13:03.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 -- 172.21.15.133:0/1420952824 shutdown_connections 2024-04-25T10:13:03.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8d4066f60 0x7fb8d4069420 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f415b9b0 0x7fb8f413f8f0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb8f4143ea0 0x7fb8f41f9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 --2- 172.21.15.133:0/1420952824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8f40b2150 0x7fb8f41544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 -- 172.21.15.133:0/1420952824 >> 172.21.15.133:0/1420952824 conn(0x7fb8f4066f70 msgr2=0x7fb8f406e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 -- 172.21.15.133:0/1420952824 shutdown_connections 2024-04-25T10:13:03.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.780+0000 7fb8fa034640 1 -- 172.21.15.133:0/1420952824 wait complete. 2024-04-25T10:13:03.799 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:13:03.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6bdf94640 1 Processor -- start 2024-04-25T10:13:03.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6bdf94640 1 -- start start 2024-04-25T10:13:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6bdf94640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff6b80ce330 0x7ff6b80ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6bdf94640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b80c61d0 con 0x7ff6b80b1f70 2024-04-25T10:13:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6bdf94640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b80c64b0 con 0x7ff6b815c9a0 2024-04-25T10:13:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6bdf94640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b80cecc0 con 0x7ff6b80ce330 2024-04-25T10:13:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b6ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff6b80b1f70 0x7ff6b806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35376/0 (socket says 172.21.15.133:35376) 2024-04-25T10:13:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b6ffd640 1 -- 172.21.15.133:0/3998282831 learned_addr learned my addr 172.21.15.133:0/3998282831 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b7fff640 1 --2- 172.21.15.133:0/3998282831 >> v2:172.21.15.162:3300/0 conn(0x7ff6b80ce330 0x7ff6b80ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185077991 0 0) 0x7ff6b80c61d0 con 0x7ff6b80b1f70 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a0003530 con 0x7ff6b80b1f70 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 180158948 0 0) 0x7ff6a0003530 con 0x7ff6b80b1f70 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 >> v2:172.21.15.162:3300/0 conn(0x7ff6b80ce330 msgr2=0x7ff6b80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 --2- 172.21.15.133:0/3998282831 >> v2:172.21.15.162:3300/0 conn(0x7ff6b80ce330 0x7ff6b80ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 >> v1:172.21.15.162:6789/0 conn(0x7ff6b815c9a0 legacy=0x7ff6b80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b806d4f0 con 0x7ff6b80b1f70 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff6ac001680 con 0x7ff6b80b1f70 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 >> v1:172.21.15.133:6789/0 conn(0x7ff6b80b1f70 legacy=0x7ff6b806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0003a10 0x7ff6a0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6a0004810 0x7ff6a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6a00051d0 0x7ff6a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b80cecc0 con 0x7ff6a0004810 2024-04-25T10:13:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a000c0d0 con 0x7ff6a0003a10 2024-04-25T10:13:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a000c2d0 con 0x7ff6a00051d0 2024-04-25T10:13:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b6ffd640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6a0004810 0x7ff6a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.904+0000 7ff6b77fe640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0003a10 0x7ff6a0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b7fff640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6a00051d0 0x7ff6a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b6ffd640 1 -- 172.21.15.133:0/3998282831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6a00051d0 msgr2=0x7ff6a000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b6ffd640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6a00051d0 0x7ff6a000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b6ffd640 1 -- 172.21.15.133:0/3998282831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0003a10 msgr2=0x7ff6a0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b6ffd640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0003a10 0x7ff6a0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b6ffd640 1 -- 172.21.15.133:0/3998282831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0003530 con 0x7ff6a0004810 2024-04-25T10:13:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b7fff640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6a00051d0 0x7ff6a000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b7fff640 1 -- 172.21.15.133:0/3998282831 reap_dead start 2024-04-25T10:13:03.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b6ffd640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6a0004810 0x7ff6a0004c90 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c0e0 tx=0x7ff6ac01f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac63e96775542c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac007c50 con 0x7ff6a0004810 2024-04-25T10:13:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac0ac020 con 0x7ff6a0004810 2024-04-25T10:13:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6b67fc640 1 -- 172.21.15.133:0/3998282831 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0058e0 con 0x7ff6a0004810 2024-04-25T10:13:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6bdf94640 1 -- 172.21.15.133:0/3998282831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6a0004810 msgr2=0x7ff6a0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6bdf94640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6a0004810 0x7ff6a0004c90 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c0e0 tx=0x7ff6ac01f270 comp rx=0 tx=0).stop 2024-04-25T10:13:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6bdf94640 1 -- 172.21.15.133:0/3998282831 shutdown_connections 2024-04-25T10:13:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6bdf94640 1 --2- 172.21.15.133:0/3998282831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6a0004810 0x7ff6a0004c90 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6bdf94640 1 -- 172.21.15.133:0/3998282831 >> 172.21.15.133:0/3998282831 conn(0x7ff6b8067000 msgr2=0x7ff6b806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6bdf94640 1 -- 172.21.15.133:0/3998282831 shutdown_connections 2024-04-25T10:13:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.908+0000 7ff6bdf94640 1 -- 172.21.15.133:0/3998282831 wait complete. 2024-04-25T10:13:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6bdf94640 1 Processor -- start 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6bdf94640 1 -- start start 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6bdf94640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0004810 0x7ff6b8152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6bdf94640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b80ca9d0 0x7ff6b80c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6bdf94640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6b814f0d0 0x7ff6b81579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6bdf94640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b815cf00 con 0x7ff6b80ca9d0 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6bdf94640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b815cd80 con 0x7ff6a0004810 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6bdf94640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b80ce780 con 0x7ff6b814f0d0 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6b6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b80ca9d0 0x7ff6b80c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6b6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b80ca9d0 0x7ff6b80c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52756/0 (socket says 172.21.15.133:52756) 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6b6ffd640 1 -- 172.21.15.133:0/977936846 learned_addr learned my addr 172.21.15.133:0/977936846 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6b77fe640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0004810 0x7ff6b8152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6b7fff640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6b814f0d0 0x7ff6b81579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6b6ffd640 1 -- 172.21.15.133:0/977936846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6b814f0d0 msgr2=0x7ff6b81579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6b6ffd640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6b814f0d0 0x7ff6b81579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6b6ffd640 1 -- 172.21.15.133:0/977936846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0004810 msgr2=0x7ff6b8152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6b6ffd640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0004810 0x7ff6b8152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6b6ffd640 1 -- 172.21.15.133:0/977936846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b80c9690 con 0x7ff6b80ca9d0 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6b6ffd640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b80ca9d0 0x7ff6b80c9150 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac01f0a0 tx=0x7ff6a0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff69ffff640 1 -- 172.21.15.133:0/977936846 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac005b40 con 0x7ff6b80ca9d0 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff69ffff640 1 -- 172.21.15.133:0/977936846 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac0b5070 con 0x7ff6b80ca9d0 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff69ffff640 1 -- 172.21.15.133:0/977936846 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0070d0 con 0x7ff6b80ca9d0 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6bdf94640 1 -- 172.21.15.133:0/977936846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b80c9930 con 0x7ff6b80ca9d0 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6bdf94640 1 -- 172.21.15.133:0/977936846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b80c9ef0 con 0x7ff6b80ca9d0 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.912+0000 7ff6bdf94640 1 -- 172.21.15.133:0/977936846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b8068ef0 con 0x7ff6b80ca9d0 2024-04-25T10:13:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.916+0000 7ff69ffff640 1 -- 172.21.15.133:0/977936846 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6ac007270 con 0x7ff6b80ca9d0 2024-04-25T10:13:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.916+0000 7ff69ffff640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6a4066f90 0x7ff6a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.916+0000 7ff69ffff640 1 -- 172.21.15.133:0/977936846 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff6ac1217d0 con 0x7ff6b80ca9d0 2024-04-25T10:13:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.916+0000 7ff6b77fe640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6a4066f90 0x7ff6a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.916+0000 7ff6b77fe640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6a4066f90 0x7ff6a4069450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8000c10 tx=0x7ff6a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:03.920+0000 7ff69ffff640 1 -- 172.21.15.133:0/977936846 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff6ac018f80 con 0x7ff6b80ca9d0 2024-04-25T10:13:04.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:04.136+0000 7ff6bdf94640 1 -- 172.21.15.133:0/977936846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7ff6b80b4210 con 0x7ff6b80ca9d0 2024-04-25T10:13:05.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.116+0000 7ff69ffff640 1 -- 172.21.15.133:0/977936846 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v506) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff6ac0f2950 con 0x7ff6b80ca9d0 2024-04-25T10:13:05.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 -- 172.21.15.133:0/977936846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6a4066f90 msgr2=0x7ff6a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6a4066f90 0x7ff6a4069450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8000c10 tx=0x7ff6a8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:05.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 -- 172.21.15.133:0/977936846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b80ca9d0 msgr2=0x7ff6b80c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b80ca9d0 0x7ff6b80c9150 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac01f0a0 tx=0x7ff6a0005840 comp rx=0 tx=0).stop 2024-04-25T10:13:05.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 -- 172.21.15.133:0/977936846 shutdown_connections 2024-04-25T10:13:05.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff6b814f0d0 0x7ff6b81579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6a4066f90 0x7ff6a4069450 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6b80ca9d0 0x7ff6b80c9150 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 --2- 172.21.15.133:0/977936846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff6a0004810 0x7ff6b8152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 -- 172.21.15.133:0/977936846 >> 172.21.15.133:0/977936846 conn(0x7ff6b8067000 msgr2=0x7ff6b80c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 -- 172.21.15.133:0/977936846 shutdown_connections 2024-04-25T10:13:05.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.120+0000 7ff6bdf94640 1 -- 172.21.15.133:0/977936846 wait complete. 2024-04-25T10:13:05.144 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:13:05.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.268+0000 7fc10264c640 1 Processor -- start 2024-04-25T10:13:05.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.268+0000 7fc10264c640 1 -- start start 2024-04-25T10:13:05.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc0fc145170 0x7fc0fc06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc0fc15ced0 0x7fc0fc15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0fc0ce2e0 con 0x7fc0fc06e4d0 2024-04-25T10:13:05.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc06db00 con 0x7fc0fc145170 2024-04-25T10:13:05.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc145550 con 0x7fc0fc15ced0 2024-04-25T10:13:05.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0fbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc0fc145170 0x7fc0fc06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0fb7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc0fc06e4d0 0x7fc0fc0cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35384/0 (socket says 172.21.15.133:35384) 2024-04-25T10:13:05.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0fb7fe640 1 -- 172.21.15.133:0/3231485035 learned_addr learned my addr 172.21.15.133:0/3231485035 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:05.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc100be9640 1 --2- 172.21.15.133:0/3231485035 >> v2:172.21.15.162:3300/0 conn(0x7fc0fc15ced0 0x7fc0fc15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0fbfff640 1 -- 172.21.15.133:0/3231485035 >> v2:172.21.15.162:3300/0 conn(0x7fc0fc15ced0 msgr2=0x7fc0fc15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:05.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0fbfff640 1 --2- 172.21.15.133:0/3231485035 >> v2:172.21.15.162:3300/0 conn(0x7fc0fc15ced0 0x7fc0fc15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0fbfff640 1 -- 172.21.15.133:0/3231485035 >> v1:172.21.15.133:6789/0 conn(0x7fc0fc06e4d0 legacy=0x7fc0fc0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0fbfff640 1 -- 172.21.15.133:0/3231485035 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc06d4b0 con 0x7fc0fc145170 2024-04-25T10:13:05.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0faffd640 1 -- 172.21.15.133:0/3231485035 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416653047 0 0) 0x7fc0fc0ce2e0 con 0x7fc0fc06e4d0 2024-04-25T10:13:05.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0fbfff640 1 --2- 172.21.15.133:0/3231485035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc145170 0x7fc0fc06df90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0b27f0 tx=0x7fc0ec01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0dfc37f81bb7390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0faffd640 1 -- 172.21.15.133:0/3231485035 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec0b0070 con 0x7fc0fc145170 2024-04-25T10:13:05.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0faffd640 1 -- 172.21.15.133:0/3231485035 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec0ab600 con 0x7fc0fc145170 2024-04-25T10:13:05.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc0faffd640 1 -- 172.21.15.133:0/3231485035 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec0ab9d0 con 0x7fc0fc145170 2024-04-25T10:13:05.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 -- 172.21.15.133:0/3231485035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc145170 msgr2=0x7fc0fc06df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 --2- 172.21.15.133:0/3231485035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc145170 0x7fc0fc06df90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0b27f0 tx=0x7fc0ec01c480 comp rx=0 tx=0).stop 2024-04-25T10:13:05.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 -- 172.21.15.133:0/3231485035 shutdown_connections 2024-04-25T10:13:05.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 --2- 172.21.15.133:0/3231485035 >> v2:172.21.15.162:3300/0 conn(0x7fc0fc15ced0 0x7fc0fc15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 --2- 172.21.15.133:0/3231485035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc145170 0x7fc0fc06df90 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 -- 172.21.15.133:0/3231485035 >> 172.21.15.133:0/3231485035 conn(0x7fc0fc066f70 msgr2=0x7fc0fc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 -- 172.21.15.133:0/3231485035 shutdown_connections 2024-04-25T10:13:05.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.272+0000 7fc10264c640 1 -- 172.21.15.133:0/3231485035 wait complete. 2024-04-25T10:13:05.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc10264c640 1 Processor -- start 2024-04-25T10:13:05.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc10264c640 1 -- start start 2024-04-25T10:13:05.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc10264c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0fc06e4d0 0x7fc0fc0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc10264c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0fc145170 0x7fc0fc0af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc10264c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc15ced0 0x7fc0fc0af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc10264c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc168610 con 0x7fc0fc15ced0 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc10264c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc168490 con 0x7fc0fc06e4d0 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc10264c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc168790 con 0x7fc0fc145170 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc100be9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc15ced0 0x7fc0fc0af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc100be9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc15ced0 0x7fc0fc0af9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52784/0 (socket says 172.21.15.133:52784) 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc100be9640 1 -- 172.21.15.133:0/2136232225 learned_addr learned my addr 172.21.15.133:0/2136232225 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc0fbfff640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0fc06e4d0 0x7fc0fc0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc0fb7fe640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0fc145170 0x7fc0fc0af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc100be9640 1 -- 172.21.15.133:0/2136232225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0fc145170 msgr2=0x7fc0fc0af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc100be9640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0fc145170 0x7fc0fc0af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc100be9640 1 -- 172.21.15.133:0/2136232225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0fc06e4d0 msgr2=0x7fc0fc0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc100be9640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0fc06e4d0 0x7fc0fc0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc100be9640 1 -- 172.21.15.133:0/2136232225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc165ac0 con 0x7fc0fc15ced0 2024-04-25T10:13:05.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc100be9640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc15ced0 0x7fc0fc0af9f0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4007e90 tx=0x7fc0f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc0f8ff9640 1 -- 172.21.15.133:0/2136232225 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4014420 con 0x7fc0fc15ced0 2024-04-25T10:13:05.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc0f8ff9640 1 -- 172.21.15.133:0/2136232225 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f4019070 con 0x7fc0fc15ced0 2024-04-25T10:13:05.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc10264c640 1 -- 172.21.15.133:0/2136232225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0fc140740 con 0x7fc0fc15ced0 2024-04-25T10:13:05.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc0f8ff9640 1 -- 172.21.15.133:0/2136232225 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4014800 con 0x7fc0fc15ced0 2024-04-25T10:13:05.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc10264c640 1 -- 172.21.15.133:0/2136232225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0fc140d00 con 0x7fc0fc15ced0 2024-04-25T10:13:05.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.276+0000 7fc10264c640 1 -- 172.21.15.133:0/2136232225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0fc068e80 con 0x7fc0fc15ced0 2024-04-25T10:13:05.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.280+0000 7fc0f8ff9640 1 -- 172.21.15.133:0/2136232225 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0f40149a0 con 0x7fc0fc15ced0 2024-04-25T10:13:05.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.280+0000 7fc0f8ff9640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0e8066f60 0x7fc0e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.280+0000 7fc0fbfff640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0e8066f60 0x7fc0e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.280+0000 7fc0f8ff9640 1 -- 172.21.15.133:0/2136232225 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc0f4085630 con 0x7fc0fc15ced0 2024-04-25T10:13:05.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.280+0000 7fc0fbfff640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0e8066f60 0x7fc0e8069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0ad000 tx=0x7fc0ec0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.284+0000 7fc0f8ff9640 1 -- 172.21.15.133:0/2136232225 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0f4011260 con 0x7fc0fc15ced0 2024-04-25T10:13:05.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.428+0000 7fc10264c640 1 -- 172.21.15.133:0/2136232225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc0fc13d3a0 con 0x7fc0fc15ced0 2024-04-25T10:13:05.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.436+0000 7fc0f8ff9640 1 -- 172.21.15.133:0/2136232225 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc0f405c0c0 con 0x7fc0fc15ced0 2024-04-25T10:13:05.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 -- 172.21.15.133:0/2136232225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0e8066f60 msgr2=0x7fc0e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0e8066f60 0x7fc0e8069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0ad000 tx=0x7fc0ec0a6000 comp rx=0 tx=0).stop 2024-04-25T10:13:05.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 -- 172.21.15.133:0/2136232225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc15ced0 msgr2=0x7fc0fc0af9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc15ced0 0x7fc0fc0af9f0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4007e90 tx=0x7fc0f400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 -- 172.21.15.133:0/2136232225 shutdown_connections 2024-04-25T10:13:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0e8066f60 0x7fc0e8069420 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc15ced0 0x7fc0fc0af9f0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0fc145170 0x7fc0fc0af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 --2- 172.21.15.133:0/2136232225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0fc06e4d0 0x7fc0fc0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 -- 172.21.15.133:0/2136232225 >> 172.21.15.133:0/2136232225 conn(0x7fc0fc066f70 msgr2=0x7fc0fc13d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 -- 172.21.15.133:0/2136232225 shutdown_connections 2024-04-25T10:13:05.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.440+0000 7fc10264c640 1 -- 172.21.15.133:0/2136232225 wait complete. 2024-04-25T10:13:05.457 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:13:05.457 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:05.502 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:05.557 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:13:05.558 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:13:05.558 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:05.562 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:05.563 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:13:05.563 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:05.569 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:05.569 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:13:05.569 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:05.609 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:05.610 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:13:05.610 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:05.616 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:05.616 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:13:05.617 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:05.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee3587640 1 Processor -- start 2024-04-25T10:13:05.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee3587640 1 -- start start 2024-04-25T10:13:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee3587640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3edc15c9a0 0x7f3edc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee3587640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3edc0ce330 0x7f3edc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee3587640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3edc0c61d0 con 0x7f3edc0b1f70 2024-04-25T10:13:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee3587640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3edc0cecc0 con 0x7f3edc15c9a0 2024-04-25T10:13:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee3587640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3edc0c64b0 con 0x7f3edc0ce330 2024-04-25T10:13:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee1323640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3edc15c9a0 0x7f3edc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee1323640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3edc15c9a0 0x7f3edc0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52790/0 (socket says 172.21.15.133:52790) 2024-04-25T10:13:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee1323640 1 -- 172.21.15.133:0/1626709792 learned_addr learned my addr 172.21.15.133:0/1626709792 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee1b24640 1 --2- 172.21.15.133:0/1626709792 >> v2:172.21.15.162:3300/0 conn(0x7f3edc0ce330 0x7f3edc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee1323640 1 -- 172.21.15.133:0/1626709792 >> v2:172.21.15.162:3300/0 conn(0x7f3edc0ce330 msgr2=0x7f3edc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:05.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee1323640 1 --2- 172.21.15.133:0/1626709792 >> v2:172.21.15.162:3300/0 conn(0x7f3edc0ce330 0x7f3edc0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee1323640 1 -- 172.21.15.133:0/1626709792 >> v1:172.21.15.162:6789/0 conn(0x7f3edc0b1f70 legacy=0x7f3edc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.772+0000 7f3ee1323640 1 -- 172.21.15.133:0/1626709792 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc06d4f0 con 0x7f3edc15c9a0 2024-04-25T10:13:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee1323640 1 --2- 172.21.15.133:0/1626709792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc15c9a0 0x7f3edc0c5980 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0cc0f0 tx=0x7f3ed801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afadae09a845eca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ed3fff640 1 -- 172.21.15.133:0/1626709792 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed80b0070 con 0x7f3edc15c9a0 2024-04-25T10:13:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ed3fff640 1 -- 172.21.15.133:0/1626709792 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed80ab600 con 0x7f3edc15c9a0 2024-04-25T10:13:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ed3fff640 1 -- 172.21.15.133:0/1626709792 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed80ab9d0 con 0x7f3edc15c9a0 2024-04-25T10:13:05.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 -- 172.21.15.133:0/1626709792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc15c9a0 msgr2=0x7f3edc0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 --2- 172.21.15.133:0/1626709792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc15c9a0 0x7f3edc0c5980 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0cc0f0 tx=0x7f3ed801c520 comp rx=0 tx=0).stop 2024-04-25T10:13:05.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 -- 172.21.15.133:0/1626709792 shutdown_connections 2024-04-25T10:13:05.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 --2- 172.21.15.133:0/1626709792 >> v2:172.21.15.162:3300/0 conn(0x7f3edc0ce330 0x7f3edc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 --2- 172.21.15.133:0/1626709792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc15c9a0 0x7f3edc0c5980 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 -- 172.21.15.133:0/1626709792 >> 172.21.15.133:0/1626709792 conn(0x7f3edc067000 msgr2=0x7f3edc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 -- 172.21.15.133:0/1626709792 shutdown_connections 2024-04-25T10:13:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 -- 172.21.15.133:0/1626709792 wait complete. 2024-04-25T10:13:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 Processor -- start 2024-04-25T10:13:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 -- start start 2024-04-25T10:13:05.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc0b1f70 0x7f3edc14c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3edc0ce330 0x7f3edc148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3edc15c9a0 0x7f3edc151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc15dad0 con 0x7f3edc0b1f70 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc15d950 con 0x7f3edc15c9a0 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee3587640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc15dc50 con 0x7f3edc0ce330 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee1323640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc0b1f70 0x7f3edc14c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee1323640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc0b1f70 0x7f3edc14c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52804/0 (socket says 172.21.15.133:52804) 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.776+0000 7f3ee1323640 1 -- 172.21.15.133:0/2031352605 learned_addr learned my addr 172.21.15.133:0/2031352605 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ee1323640 1 -- 172.21.15.133:0/2031352605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3edc0ce330 msgr2=0x7f3edc148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ee1323640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3edc0ce330 0x7f3edc148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ee1323640 1 -- 172.21.15.133:0/2031352605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3edc15c9a0 msgr2=0x7f3edc151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ee1323640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3edc15c9a0 0x7f3edc151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ee1323640 1 -- 172.21.15.133:0/2031352605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc14b6b0 con 0x7f3edc0b1f70 2024-04-25T10:13:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ee1323640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc0b1f70 0x7f3edc14c470 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f3ed801ca00 tx=0x7f3ed80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/2031352605 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed80b0040 con 0x7f3edc0b1f70 2024-04-25T10:13:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/2031352605 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed80b9070 con 0x7f3edc0b1f70 2024-04-25T10:13:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ee3587640 1 -- 172.21.15.133:0/2031352605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3edc14b990 con 0x7f3edc0b1f70 2024-04-25T10:13:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/2031352605 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed80b4660 con 0x7f3edc0b1f70 2024-04-25T10:13:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ee3587640 1 -- 172.21.15.133:0/2031352605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3edc14bf50 con 0x7f3edc0b1f70 2024-04-25T10:13:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ee3587640 1 -- 172.21.15.133:0/2031352605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3edc06bc20 con 0x7f3edc0b1f70 2024-04-25T10:13:05.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/2031352605 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ed80b4800 con 0x7f3edc0b1f70 2024-04-25T10:13:05.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ed1ffb640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ec0066f90 0x7f3ec0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/2031352605 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3ed811eb10 con 0x7f3edc0b1f70 2024-04-25T10:13:05.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ee0b22640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ec0066f90 0x7f3ec0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.780+0000 7f3ee0b22640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ec0066f90 0x7f3ec0069450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc000f70 tx=0x7f3ecc0079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.784+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/2031352605 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ed8016110 con 0x7f3edc0b1f70 2024-04-25T10:13:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 -- 172.21.15.133:0/2031352605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3edc068ef0 con 0x7f3edc0b1f70 2024-04-25T10:13:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ed1ffb640 1 -- 172.21.15.133:0/2031352605 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f3ed80efc00 con 0x7f3edc0b1f70 2024-04-25T10:13:05.979 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:05.979 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":74,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6062,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/598914819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6833","nonce":598914819},{"type":"v1","addr":"172.21.15.162:6835","nonce":598914819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6834/1858521876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1858521876},{"type":"v1","addr":"172.21.15.162:6834","nonce":1858521876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1855134958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1855134958},{"type":"v1","addr":"172.21.15.133:6836","nonce":1855134958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9411,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3848813407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3848813407},{"type":"v1","addr":"172.21.15.133:6837","nonce":3848813407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[]} 2024-04-25T10:13:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 74 2024-04-25T10:13:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 -- 172.21.15.133:0/2031352605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ec0066f90 msgr2=0x7f3ec0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ec0066f90 0x7f3ec0069450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc000f70 tx=0x7f3ecc0079e0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 -- 172.21.15.133:0/2031352605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc0b1f70 msgr2=0x7f3edc14c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc0b1f70 0x7f3edc14c470 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f3ed801ca00 tx=0x7f3ed80192c0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 -- 172.21.15.133:0/2031352605 shutdown_connections 2024-04-25T10:13:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3edc15c9a0 0x7f3edc151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3edc0ce330 0x7f3edc148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ec0066f90 0x7f3ec0069450 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 --2- 172.21.15.133:0/2031352605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3edc0b1f70 0x7f3edc14c470 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 -- 172.21.15.133:0/2031352605 >> 172.21.15.133:0/2031352605 conn(0x7f3edc067000 msgr2=0x7f3edc0b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 -- 172.21.15.133:0/2031352605 shutdown_connections 2024-04-25T10:13:05.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:05.976+0000 7f3ee3587640 1 -- 172.21.15.133:0/2031352605 wait complete. 2024-04-25T10:13:05.997 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:13:05.997 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:13:05.998 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:13:05.998 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:05.998 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:13:05.998 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:13:05.998 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:13:05.998 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:05.998 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:13:05.999 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:13:05.999 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:13:05.999 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:05.999 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:13:05.999 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:13:05.999 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:13:05.999 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:06.007 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:13:06.007 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:13:06.008 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:13:06.008 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:13:06.013 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:13:06.014 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:13:06.014 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:13:06.015 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:13:06.050 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:13:06.051 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:06.051 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:13:06.051 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:06.057 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:13:06.057 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:06.058 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:13:06.058 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:06.058 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:13:06.109 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:13:06.118 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:13:06.129 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:13:06.131 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:13:06.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df701f640 1 Processor -- start 2024-04-25T10:13:06.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df701f640 1 -- start start 2024-04-25T10:13:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df701f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df00d2030 0x7f7df00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df701f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df00d24c0 con 0x7f7df015d9a0 2024-04-25T10:13:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df701f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df015ebd0 con 0x7f7df00ce600 2024-04-25T10:13:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df701f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df015eeb0 con 0x7f7df00d2030 2024-04-25T10:13:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df4dbb640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7df015d9a0 0x7f7df00d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38670/0 (socket says 172.21.15.133:38670) 2024-04-25T10:13:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df4dbb640 1 -- 172.21.15.133:0/3048861281 learned_addr learned my addr 172.21.15.133:0/3048861281 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df55bc640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df00d2030 0x7f7df00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203746292 0 0) 0x7f7df00d24c0 con 0x7f7df015d9a0 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dd8003530 con 0x7f7df015d9a0 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3954154680 0 0) 0x7f7dd8003530 con 0x7f7df015d9a0 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df00d2030 msgr2=0x7f7df00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df00d2030 0x7f7df00cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 >> v1:172.21.15.162:6789/0 conn(0x7f7df00ce600 legacy=0x7f7df00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df00d0330 con 0x7f7df015d9a0 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7de0001680 con 0x7f7df015d9a0 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 >> v1:172.21.15.133:6789/0 conn(0x7f7df015d9a0 legacy=0x7f7df00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df00d2030 0x7f7dd80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dd8003f70 0x7f7dd8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dd8004d50 0x7f7dd80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df015eeb0 con 0x7f7dd8004d50 2024-04-25T10:13:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd800bb60 con 0x7f7dd8003f70 2024-04-25T10:13:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd800bd60 con 0x7f7df00d2030 2024-04-25T10:13:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df55bc640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dd8004d50 0x7f7dd80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df4dbb640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df00d2030 0x7f7dd80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7deffff640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dd8003f70 0x7f7dd8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df55bc640 1 -- 172.21.15.133:0/3048861281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df00d2030 msgr2=0x7f7dd80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df55bc640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df00d2030 0x7f7dd80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df55bc640 1 -- 172.21.15.133:0/3048861281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dd8003f70 msgr2=0x7f7dd8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df55bc640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7dd8003f70 0x7f7dd8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.164+0000 7f7df55bc640 1 -- 172.21.15.133:0/3048861281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8003530 con 0x7f7dd8004d50 2024-04-25T10:13:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df55bc640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dd8004d50 0x7f7dd80051d0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009fc0 tx=0x7f7de801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4208993f7ce91530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de80b2070 con 0x7f7dd8004d50 2024-04-25T10:13:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de80a5c90 con 0x7f7dd8004d50 2024-04-25T10:13:06.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7def7fe640 1 -- 172.21.15.133:0/3048861281 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de8009c10 con 0x7f7dd8004d50 2024-04-25T10:13:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 -- 172.21.15.133:0/3048861281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dd8004d50 msgr2=0x7f7dd80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 --2- 172.21.15.133:0/3048861281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7dd8004d50 0x7f7dd80051d0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009fc0 tx=0x7f7de801cc80 comp rx=0 tx=0).stop 2024-04-25T10:13:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df55bc640 1 -- 172.21.15.133:0/3048861281 reap_dead start 2024-04-25T10:13:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 -- 172.21.15.133:0/3048861281 shutdown_connections 2024-04-25T10:13:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 -- 172.21.15.133:0/3048861281 >> 172.21.15.133:0/3048861281 conn(0x7f7df006bf80 msgr2=0x7f7df006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 -- 172.21.15.133:0/3048861281 shutdown_connections 2024-04-25T10:13:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 -- 172.21.15.133:0/3048861281 wait complete. 2024-04-25T10:13:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 Processor -- start 2024-04-25T10:13:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 -- start start 2024-04-25T10:13:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df00c9a70 0x7f7df00cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df00c9e50 0x7f7df01604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df0160a00 0x7f7df01f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00b96a0 con 0x7f7df00c9a70 2024-04-25T10:13:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00cffd0 con 0x7f7df00c9e50 2024-04-25T10:13:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df701f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df00b9820 con 0x7f7df0160a00 2024-04-25T10:13:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df4dbb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df00c9a70 0x7f7df00cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df4dbb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df00c9a70 0x7f7df00cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37768/0 (socket says 172.21.15.133:37768) 2024-04-25T10:13:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df4dbb640 1 -- 172.21.15.133:0/3162290054 learned_addr learned my addr 172.21.15.133:0/3162290054 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7df55bc640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df0160a00 0x7f7df01f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.168+0000 7f7deffff640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df00c9e50 0x7f7df01604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7df4dbb640 1 -- 172.21.15.133:0/3162290054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df0160a00 msgr2=0x7f7df01f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7df4dbb640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df0160a00 0x7f7df01f31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7df4dbb640 1 -- 172.21.15.133:0/3162290054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df00c9e50 msgr2=0x7f7df01604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7df4dbb640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df00c9e50 0x7f7df01604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7df4dbb640 1 -- 172.21.15.133:0/3162290054 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df00cbad0 con 0x7f7df00c9a70 2024-04-25T10:13:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7df4dbb640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df00c9a70 0x7f7df00cc590 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f7de0012490 tx=0x7f7de0012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7ded7fa640 1 -- 172.21.15.133:0/3162290054 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0014810 con 0x7f7df00c9a70 2024-04-25T10:13:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7ded7fa640 1 -- 172.21.15.133:0/3162290054 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de001b070 con 0x7f7df00c9a70 2024-04-25T10:13:06.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7df701f640 1 -- 172.21.15.133:0/3162290054 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df01f3f80 con 0x7f7df00c9a70 2024-04-25T10:13:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7ded7fa640 1 -- 172.21.15.133:0/3162290054 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de0014c30 con 0x7f7df00c9a70 2024-04-25T10:13:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7df701f640 1 -- 172.21.15.133:0/3162290054 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df00cc070 con 0x7f7df00c9a70 2024-04-25T10:13:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7df701f640 1 -- 172.21.15.133:0/3162290054 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df00cc270 con 0x7f7df00c9a70 2024-04-25T10:13:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7ded7fa640 1 -- 172.21.15.133:0/3162290054 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7de0016480 con 0x7f7df00c9a70 2024-04-25T10:13:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7ded7fa640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ddc066f90 0x7f7ddc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7ded7fa640 1 -- 172.21.15.133:0/3162290054 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7de00880f0 con 0x7f7df00c9a70 2024-04-25T10:13:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7deffff640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ddc066f90 0x7f7ddc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.172+0000 7f7deffff640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ddc066f90 0x7f7ddc069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7df00cd180 tx=0x7f7de4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.176+0000 7f7ded7fa640 1 -- 172.21.15.133:0/3162290054 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7df00cc270 con 0x7f7df00c9a70 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:06.336+0000 7f7df701f640 1 -- 172.21.15.133:0/3162290054 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7df0068e80 con 0x7f7df00c9a70 2024-04-25T10:13:07.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.460+0000 7f7ded7fa640 1 -- 172.21.15.133:0/3162290054 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v128) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7de0058c20 con 0x7f7df00c9a70 2024-04-25T10:13:07.464 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:13:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 -- 172.21.15.133:0/3162290054 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ddc066f90 msgr2=0x7f7ddc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ddc066f90 0x7f7ddc069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7df00cd180 tx=0x7f7de4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 -- 172.21.15.133:0/3162290054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df00c9a70 msgr2=0x7f7df00cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df00c9a70 0x7f7df00cc590 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f7de0012490 tx=0x7f7de0012950 comp rx=0 tx=0).stop 2024-04-25T10:13:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 -- 172.21.15.133:0/3162290054 shutdown_connections 2024-04-25T10:13:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7df0160a00 0x7f7df01f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7df00c9e50 0x7f7df01604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7ddc066f90 0x7f7ddc069450 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 --2- 172.21.15.133:0/3162290054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7df00c9a70 0x7f7df00cc590 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 -- 172.21.15.133:0/3162290054 >> 172.21.15.133:0/3162290054 conn(0x7f7df006bf80 msgr2=0x7f7df00b8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 -- 172.21.15.133:0/3162290054 shutdown_connections 2024-04-25T10:13:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.464+0000 7f7df701f640 1 -- 172.21.15.133:0/3162290054 wait complete. 2024-04-25T10:13:07.488 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:13:07.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869f988640 1 Processor -- start 2024-04-25T10:13:07.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869f988640 1 -- start start 2024-04-25T10:13:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869f988640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8698159dc0 0x7f86980cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869f988640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f869815a1a0 0x7f869815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869f988640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86980ce880 con 0x7f869815e050 2024-04-25T10:13:07.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869f988640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869815e4a0 con 0x7f869815a1a0 2024-04-25T10:13:07.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869f988640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869806e360 con 0x7f8698159dc0 2024-04-25T10:13:07.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869cf23640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f869815a1a0 0x7f869815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869cf23640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f869815a1a0 0x7f869815db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37774/0 (socket says 172.21.15.133:37774) 2024-04-25T10:13:07.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869cf23640 1 -- 172.21.15.133:0/2438246305 learned_addr learned my addr 172.21.15.133:0/2438246305 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:07.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869d724640 1 --2- 172.21.15.133:0/2438246305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8698159dc0 0x7f86980cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869cf23640 1 -- 172.21.15.133:0/2438246305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8698159dc0 msgr2=0x7f86980cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869cf23640 1 --2- 172.21.15.133:0/2438246305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8698159dc0 0x7f86980cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869cf23640 1 -- 172.21.15.133:0/2438246305 >> v1:172.21.15.162:6789/0 conn(0x7f869815e050 legacy=0x7f86980ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.612+0000 7f869cf23640 1 -- 172.21.15.133:0/2438246305 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86980c49e0 con 0x7f869815a1a0 2024-04-25T10:13:07.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869cf23640 1 --2- 172.21.15.133:0/2438246305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f869815a1a0 0x7f869815db10 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f868c00aa10 tx=0x7f868c01d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15743283296ffa86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f8697fff640 1 -- 172.21.15.133:0/2438246305 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c00b040 con 0x7f869815a1a0 2024-04-25T10:13:07.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f8697fff640 1 -- 172.21.15.133:0/2438246305 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c0a7710 con 0x7f869815a1a0 2024-04-25T10:13:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f8697fff640 1 -- 172.21.15.133:0/2438246305 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c0a7ae0 con 0x7f869815a1a0 2024-04-25T10:13:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 -- 172.21.15.133:0/2438246305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f869815a1a0 msgr2=0x7f869815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 --2- 172.21.15.133:0/2438246305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f869815a1a0 0x7f869815db10 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f868c00aa10 tx=0x7f868c01d8f0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 -- 172.21.15.133:0/2438246305 shutdown_connections 2024-04-25T10:13:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 --2- 172.21.15.133:0/2438246305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f869815a1a0 0x7f869815db10 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 --2- 172.21.15.133:0/2438246305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8698159dc0 0x7f86980cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 -- 172.21.15.133:0/2438246305 >> 172.21.15.133:0/2438246305 conn(0x7f8698067000 msgr2=0x7f869806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 -- 172.21.15.133:0/2438246305 shutdown_connections 2024-04-25T10:13:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 -- 172.21.15.133:0/2438246305 wait complete. 2024-04-25T10:13:07.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 Processor -- start 2024-04-25T10:13:07.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 -- start start 2024-04-25T10:13:07.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8698159dc0 0x7f869814dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f869815e050 0x7f8698152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f869814ffe0 0x7f869814bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980c4e40 con 0x7f869815e050 2024-04-25T10:13:07.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980c4cc0 con 0x7f8698159dc0 2024-04-25T10:13:07.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869f988640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86980cf300 con 0x7f869814ffe0 2024-04-25T10:13:07.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869cf23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f869815e050 0x7f8698152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869cf23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f869815e050 0x7f8698152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37784/0 (socket says 172.21.15.133:37784) 2024-04-25T10:13:07.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869cf23640 1 -- 172.21.15.133:0/2940256479 learned_addr learned my addr 172.21.15.133:0/2940256479 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:07.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869d724640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8698159dc0 0x7f869814dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869df25640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f869814ffe0 0x7f869814bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869cf23640 1 -- 172.21.15.133:0/2940256479 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f869814ffe0 msgr2=0x7f869814bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869cf23640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f869814ffe0 0x7f869814bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869cf23640 1 -- 172.21.15.133:0/2940256479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8698159dc0 msgr2=0x7f869814dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869cf23640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8698159dc0 0x7f869814dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869cf23640 1 -- 172.21.15.133:0/2940256479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86981519a0 con 0x7f869815e050 2024-04-25T10:13:07.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.616+0000 7f869cf23640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f869815e050 0x7f8698152810 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f868c01bfa0 tx=0x7f868c01c130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.620+0000 7f8695ffb640 1 -- 172.21.15.133:0/2940256479 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c0af830 con 0x7f869815e050 2024-04-25T10:13:07.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.620+0000 7f8695ffb640 1 -- 172.21.15.133:0/2940256479 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c0bf070 con 0x7f869815e050 2024-04-25T10:13:07.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.620+0000 7f869f988640 1 -- 172.21.15.133:0/2940256479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869814e2b0 con 0x7f869815e050 2024-04-25T10:13:07.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.620+0000 7f8695ffb640 1 -- 172.21.15.133:0/2940256479 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c0afc10 con 0x7f869815e050 2024-04-25T10:13:07.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.620+0000 7f869f988640 1 -- 172.21.15.133:0/2940256479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869814e870 con 0x7f869815e050 2024-04-25T10:13:07.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.620+0000 7f869f988640 1 -- 172.21.15.133:0/2940256479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8698068ef0 con 0x7f869815e050 2024-04-25T10:13:07.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.620+0000 7f8695ffb640 1 -- 172.21.15.133:0/2940256479 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f868c0b9400 con 0x7f869815e050 2024-04-25T10:13:07.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.620+0000 7f8695ffb640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8684066f60 0x7f8684069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.620+0000 7f8695ffb640 1 -- 172.21.15.133:0/2940256479 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f868c120840 con 0x7f869815e050 2024-04-25T10:13:07.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.620+0000 7f869d724640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8684066f60 0x7f8684069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.620+0000 7f869d724640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8684066f60 0x7f8684069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f869000b9d0 tx=0x7f8690008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.624+0000 7f8695ffb640 1 -- 172.21.15.133:0/2940256479 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f868c0174e0 con 0x7f869815e050 2024-04-25T10:13:07.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.840+0000 7f869f988640 1 -- 172.21.15.133:0/2940256479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f86980cd7b0 con 0x7f869815e050 2024-04-25T10:13:07.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.840+0000 7f8695ffb640 1 -- 172.21.15.133:0/2940256479 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f868c0f1af0 con 0x7f869815e050 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.844 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:13:07.845 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:13:07.846 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.847 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:13:07.848 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:13:07.849 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.850 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.850 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.850 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.850 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:13:07.850 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:13:07.850 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.850 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:07.850 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:13:07.850 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.850 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.850 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.851 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:07.852 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:13:07.852 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.852 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.852 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.852 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.852 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:13:07.852 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:13:07.852 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.852 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:07.852 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:13:07.852 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.853 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.853 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.853 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.853 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:13:07.853 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:13:07.853 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.853 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:13:07.853 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:13:07.853 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:13:07.853 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.853 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:13:07.855 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.856 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.857 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:13:07.857 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:13:07.857 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.857 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:13:07.857 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:13:07.857 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.857 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.857 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:07.857 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:07.857 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:13:07.857 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:13:07.858 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:13:07.858 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:13:07.858 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:13:07.858 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:13:07.858 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.858 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:07.858 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:13:07.858 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:13:07.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 -- 172.21.15.133:0/2940256479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8684066f60 msgr2=0x7f8684069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8684066f60 0x7f8684069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f869000b9d0 tx=0x7f8690008040 comp rx=0 tx=0).stop 2024-04-25T10:13:07.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 -- 172.21.15.133:0/2940256479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f869815e050 msgr2=0x7f8698152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f869815e050 0x7f8698152810 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f868c01bfa0 tx=0x7f868c01c130 comp rx=0 tx=0).stop 2024-04-25T10:13:07.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 -- 172.21.15.133:0/2940256479 shutdown_connections 2024-04-25T10:13:07.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f869814ffe0 0x7f869814bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8684066f60 0x7f8684069420 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f869815e050 0x7f8698152810 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 --2- 172.21.15.133:0/2940256479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8698159dc0 0x7f869814dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 -- 172.21.15.133:0/2940256479 >> 172.21.15.133:0/2940256479 conn(0x7f8698067000 msgr2=0x7f869814cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 -- 172.21.15.133:0/2940256479 shutdown_connections 2024-04-25T10:13:07.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.844+0000 7f869f988640 1 -- 172.21.15.133:0/2940256479 wait complete. 2024-04-25T10:13:07.863 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:07.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fe92c640 1 Processor -- start 2024-04-25T10:13:07.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fe92c640 1 -- start start 2024-04-25T10:13:07.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fe92c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8152750 0x7f34f80ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fe92c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f34f8152b30 0x7f34f80ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fe92c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f806d350 con 0x7f34f80b17b0 2024-04-25T10:13:07.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fe92c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f815bf90 con 0x7f34f8152b30 2024-04-25T10:13:07.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fe92c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f806d5d0 con 0x7f34f8152750 2024-04-25T10:13:07.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fcec9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f34f80b17b0 0x7f34f80b1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38682/0 (socket says 172.21.15.133:38682) 2024-04-25T10:13:07.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fcec9640 1 -- 172.21.15.133:0/421870172 learned_addr learned my addr 172.21.15.133:0/421870172 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:07.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f7fff640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8152750 0x7f34f80ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f77fe640 1 --2- 172.21.15.133:0/421870172 >> v2:172.21.15.162:3300/0 conn(0x7f34f8152b30 0x7f34f80ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4044963302 0 0) 0x7f34f00014a0 con 0x7f34f80b17b0 2024-04-25T10:13:07.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e0003530 con 0x7f34f80b17b0 2024-04-25T10:13:07.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 747646396 0 0) 0x7f34e0003530 con 0x7f34f80b17b0 2024-04-25T10:13:07.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8152750 msgr2=0x7f34f80ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8152750 0x7f34f80ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 >> v2:172.21.15.162:3300/0 conn(0x7f34f8152b30 msgr2=0x7f34f80ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 --2- 172.21.15.133:0/421870172 >> v2:172.21.15.162:3300/0 conn(0x7f34f8152b30 0x7f34f80ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f815c230 con 0x7f34f80b17b0 2024-04-25T10:13:07.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f34f0001bd0 con 0x7f34f80b17b0 2024-04-25T10:13:07.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 >> v1:172.21.15.133:6789/0 conn(0x7f34f80b17b0 legacy=0x7f34f80b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8152750 0x7f34e00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e0003f40 0x7f34e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 0x7f34e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f815bf90 con 0x7f34e0004d50 2024-04-25T10:13:07.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f806d5d0 con 0x7f34e0003f40 2024-04-25T10:13:07.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e000b9d0 con 0x7f34f8152750 2024-04-25T10:13:07.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fcec9640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 0x7f34e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f77fe640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e0003f40 0x7f34e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34f7fff640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8152750 0x7f34e00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fcec9640 1 -- 172.21.15.133:0/421870172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8152750 msgr2=0x7f34e00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fcec9640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8152750 0x7f34e00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fcec9640 1 -- 172.21.15.133:0/421870172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e0003f40 msgr2=0x7f34e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fcec9640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34e0003f40 0x7f34e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.980+0000 7f34fcec9640 1 -- 172.21.15.133:0/421870172 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e0003530 con 0x7f34e0004d50 2024-04-25T10:13:07.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fcec9640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 0x7f34e00051d0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f34f000aa50 tx=0x7f34f001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71d81e4280294a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000a670 con 0x7f34e0004d50 2024-04-25T10:13:07.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f00ad630 con 0x7f34e0004d50 2024-04-25T10:13:07.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f6ffd640 1 -- 172.21.15.133:0/421870172 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f00ada20 con 0x7f34e0004d50 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 -- 172.21.15.133:0/421870172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 msgr2=0x7f34e00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 --2- 172.21.15.133:0/421870172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 0x7f34e00051d0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f34f000aa50 tx=0x7f34f001d340 comp rx=0 tx=0).stop 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fcec9640 1 -- 172.21.15.133:0/421870172 reap_dead start 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 -- 172.21.15.133:0/421870172 shutdown_connections 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 -- 172.21.15.133:0/421870172 >> 172.21.15.133:0/421870172 conn(0x7f34f8067000 msgr2=0x7f34f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 -- 172.21.15.133:0/421870172 shutdown_connections 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 -- 172.21.15.133:0/421870172 wait complete. 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 Processor -- start 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 -- start start 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 0x7f34f8145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8145410 0x7f34f8157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34f80711d0 0x7f34f8071650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f80ce330 con 0x7f34e0004d50 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f815c7f0 con 0x7f34f80711d0 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fe92c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f80ce4b0 con 0x7f34f8145410 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 0x7f34f8145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 0x7f34f8145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37812/0 (socket says 172.21.15.133:37812) 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f7fff640 1 -- 172.21.15.133:0/3564950795 learned_addr learned my addr 172.21.15.133:0/3564950795 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f77fe640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8145410 0x7f34f8157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34fcec9640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34f80711d0 0x7f34f8071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f7fff640 1 -- 172.21.15.133:0/3564950795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8145410 msgr2=0x7f34f8157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f7fff640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8145410 0x7f34f8157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f7fff640 1 -- 172.21.15.133:0/3564950795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34f80711d0 msgr2=0x7f34f8071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f7fff640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34f80711d0 0x7f34f8071650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.984+0000 7f34f7fff640 1 -- 172.21.15.133:0/3564950795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f81583e0 con 0x7f34e0004d50 2024-04-25T10:13:07.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34f7fff640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 0x7f34f8145ef0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f34e8011640 tx=0x7f34e8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34f4ff9640 1 -- 172.21.15.133:0/3564950795 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e800f920 con 0x7f34e0004d50 2024-04-25T10:13:07.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34f4ff9640 1 -- 172.21.15.133:0/3564950795 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e800fac0 con 0x7f34e0004d50 2024-04-25T10:13:07.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34fe92c640 1 -- 172.21.15.133:0/3564950795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f8153090 con 0x7f34e0004d50 2024-04-25T10:13:07.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34f4ff9640 1 -- 172.21.15.133:0/3564950795 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e800fe90 con 0x7f34e0004d50 2024-04-25T10:13:07.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34fe92c640 1 -- 172.21.15.133:0/3564950795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f8158940 con 0x7f34e0004d50 2024-04-25T10:13:07.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34fe92c640 1 -- 172.21.15.133:0/3564950795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f8071b90 con 0x7f34e0004d50 2024-04-25T10:13:07.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34f4ff9640 1 -- 172.21.15.133:0/3564950795 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34e8015460 con 0x7f34e0004d50 2024-04-25T10:13:07.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34f4ff9640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f34e4067100 0x7f34e40695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34f4ff9640 1 -- 172.21.15.133:0/3564950795 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f34e8085800 con 0x7f34e0004d50 2024-04-25T10:13:07.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34f77fe640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f34e4067100 0x7f34e40695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.988+0000 7f34f77fe640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f34e4067100 0x7f34e40695c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f34f81f4840 tx=0x7f34ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:07.992+0000 7f34f4ff9640 1 -- 172.21.15.133:0/3564950795 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f34f81583e0 con 0x7f34e0004d50 2024-04-25T10:13:08.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.152+0000 7f34fe92c640 1 -- 172.21.15.133:0/3564950795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34f8068ef0 con 0x7f34e0004d50 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.152+0000 7f34f4ff9640 1 -- 172.21.15.133:0/3564950795 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+13198 (secure 0 0 0) 0x7f34e8019030 con 0x7f34e0004d50 2024-04-25T10:13:08.156 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:08.157 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":128,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:13:07.459693+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 -- 172.21.15.133:0/3564950795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f34e4067100 msgr2=0x7f34e40695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f34e4067100 0x7f34e40695c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f34f81f4840 tx=0x7f34ec008040 comp rx=0 tx=0).stop 2024-04-25T10:13:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 -- 172.21.15.133:0/3564950795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 msgr2=0x7f34f8145ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 0x7f34f8145ef0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f34e8011640 tx=0x7f34e8011b00 comp rx=0 tx=0).stop 2024-04-25T10:13:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 -- 172.21.15.133:0/3564950795 shutdown_connections 2024-04-25T10:13:08.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34f80711d0 0x7f34f8071650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34f8145410 0x7f34f8157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f34e4067100 0x7f34e40695c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 --2- 172.21.15.133:0/3564950795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e0004d50 0x7f34f8145ef0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 -- 172.21.15.133:0/3564950795 >> 172.21.15.133:0/3564950795 conn(0x7f34f8067000 msgr2=0x7f34f815a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 -- 172.21.15.133:0/3564950795 shutdown_connections 2024-04-25T10:13:08.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.156+0000 7f34fe92c640 1 -- 172.21.15.133:0/3564950795 wait complete. 2024-04-25T10:13:08.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:08.176 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.292+0000 7f0b5bd5c640 1 Processor -- start 2024-04-25T10:13:08.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.292+0000 7f0b5bd5c640 1 -- start start 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5bd5c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b5406e8b0 0x7f0b5415eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5bd5c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0b540b1e50 0x7f0b54146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5bd5c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0b540aebc0 0x7f0b540af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5bd5c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b54147490 con 0x7f0b540aebc0 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5bd5c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b540b19c0 con 0x7f0b540b1e50 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5bd5c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540b1090 con 0x7f0b5406e8b0 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5a2f9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0b540aebc0 0x7f0b540af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5a2f9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0b540aebc0 0x7f0b540af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37828/0 (socket says 172.21.15.133:37828) 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5a2f9640 1 -- 172.21.15.133:0/3008330098 learned_addr learned my addr 172.21.15.133:0/3008330098 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b59af8640 1 --2- 172.21.15.133:0/3008330098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b5406e8b0 0x7f0b5415eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5a2f9640 1 -- 172.21.15.133:0/3008330098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b5406e8b0 msgr2=0x7f0b5415eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5a2f9640 1 --2- 172.21.15.133:0/3008330098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b5406e8b0 0x7f0b5415eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5a2f9640 1 -- 172.21.15.133:0/3008330098 >> v2:172.21.15.162:3300/0 conn(0x7f0b540b1e50 msgr2=0x7f0b54146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5a2f9640 1 --2- 172.21.15.133:0/3008330098 >> v2:172.21.15.162:3300/0 conn(0x7f0b540b1e50 0x7f0b54146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5a2f9640 1 -- 172.21.15.133:0/3008330098 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540b1370 con 0x7f0b540aebc0 2024-04-25T10:13:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b5a2f9640 1 --2- 172.21.15.133:0/3008330098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b540aebc0 0x7f0b540af050 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009e50 tx=0x7f0b4c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e064b74e696ced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b58af6640 1 -- 172.21.15.133:0/3008330098 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0a55e0 con 0x7f0b540aebc0 2024-04-25T10:13:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b58af6640 1 -- 172.21.15.133:0/3008330098 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c0a5780 con 0x7f0b540aebc0 2024-04-25T10:13:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.296+0000 7f0b58af6640 1 -- 172.21.15.133:0/3008330098 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0a5b50 con 0x7f0b540aebc0 2024-04-25T10:13:08.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3008330098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b540aebc0 msgr2=0x7f0b540af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 --2- 172.21.15.133:0/3008330098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b540aebc0 0x7f0b540af050 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009e50 tx=0x7f0b4c01c960 comp rx=0 tx=0).stop 2024-04-25T10:13:08.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3008330098 shutdown_connections 2024-04-25T10:13:08.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 --2- 172.21.15.133:0/3008330098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b540aebc0 0x7f0b540af050 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 --2- 172.21.15.133:0/3008330098 >> v2:172.21.15.162:3300/0 conn(0x7f0b540b1e50 0x7f0b54146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 --2- 172.21.15.133:0/3008330098 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b5406e8b0 0x7f0b5415eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3008330098 >> 172.21.15.133:0/3008330098 conn(0x7f0b54066f70 msgr2=0x7f0b5406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3008330098 shutdown_connections 2024-04-25T10:13:08.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3008330098 wait complete. 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 Processor -- start 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 -- start start 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b5406e8b0 0x7f0b541367b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b540aebc0 0x7f0b54144980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b540b1e50 0x7f0b54144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5406dab0 con 0x7f0b540aebc0 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54147230 con 0x7f0b540b1e50 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5406dc30 con 0x7f0b5406e8b0 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b592f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b540aebc0 0x7f0b54144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b592f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b540aebc0 0x7f0b54144980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37830/0 (socket says 172.21.15.133:37830) 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b592f7640 1 -- 172.21.15.133:0/3557623081 learned_addr learned my addr 172.21.15.133:0/3557623081 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b59af8640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b5406e8b0 0x7f0b541367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5a2f9640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b540b1e50 0x7f0b54144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b592f7640 1 -- 172.21.15.133:0/3557623081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b5406e8b0 msgr2=0x7f0b541367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b592f7640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b5406e8b0 0x7f0b541367b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b592f7640 1 -- 172.21.15.133:0/3557623081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b540b1e50 msgr2=0x7f0b54144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b592f7640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b540b1e50 0x7f0b54144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b592f7640 1 -- 172.21.15.133:0/3557623081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b54139b20 con 0x7f0b540aebc0 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5a2f9640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b540b1e50 0x7f0b54144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b592f7640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b540aebc0 0x7f0b54144980 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400b440 tx=0x7f0b4400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b4a7fc640 1 -- 172.21.15.133:0/3557623081 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44015020 con 0x7f0b540aebc0 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b4a7fc640 1 -- 172.21.15.133:0/3557623081 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b440118b0 con 0x7f0b540aebc0 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b4a7fc640 1 -- 172.21.15.133:0/3557623081 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44011ca0 con 0x7f0b540aebc0 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3557623081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b54143ee0 con 0x7f0b540aebc0 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.300+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3557623081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b54144180 con 0x7f0b540aebc0 2024-04-25T10:13:08.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.304+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3557623081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b54068e80 con 0x7f0b540aebc0 2024-04-25T10:13:08.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.304+0000 7f0b4a7fc640 1 -- 172.21.15.133:0/3557623081 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b44018020 con 0x7f0b540aebc0 2024-04-25T10:13:08.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.304+0000 7f0b4a7fc640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b38066f60 0x7f0b38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.304+0000 7f0b4a7fc640 1 -- 172.21.15.133:0/3557623081 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0b44084ee0 con 0x7f0b540aebc0 2024-04-25T10:13:08.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.304+0000 7f0b59af8640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b38066f60 0x7f0b38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.304+0000 7f0b59af8640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b38066f60 0x7f0b38069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000a970 tx=0x7f0b50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.308+0000 7f0b4a7fc640 1 -- 172.21.15.133:0/3557623081 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b4401f020 con 0x7f0b540aebc0 2024-04-25T10:13:08.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.468+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3557623081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b5406b960 con 0x7f0b540aebc0 2024-04-25T10:13:08.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.468+0000 7f0b4a7fc640 1 -- 172.21.15.133:0/3557623081 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b4400d040 con 0x7f0b540aebc0 2024-04-25T10:13:08.472 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:13:08.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3557623081 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b38066f60 msgr2=0x7f0b38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b38066f60 0x7f0b38069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000a970 tx=0x7f0b50008040 comp rx=0 tx=0).stop 2024-04-25T10:13:08.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3557623081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b540aebc0 msgr2=0x7f0b54144980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b540aebc0 0x7f0b54144980 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400b440 tx=0x7f0b4400b900 comp rx=0 tx=0).stop 2024-04-25T10:13:08.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3557623081 shutdown_connections 2024-04-25T10:13:08.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b540b1e50 0x7f0b54144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b38066f60 0x7f0b38069420 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b540aebc0 0x7f0b54144980 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 --2- 172.21.15.133:0/3557623081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b5406e8b0 0x7f0b541367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3557623081 >> 172.21.15.133:0/3557623081 conn(0x7f0b54066f70 msgr2=0x7f0b54073870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3557623081 shutdown_connections 2024-04-25T10:13:08.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.472+0000 7f0b5bd5c640 1 -- 172.21.15.133:0/3557623081 wait complete. 2024-04-25T10:13:08.491 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:13:08.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.608+0000 7f1588e1f640 1 Processor -- start 2024-04-25T10:13:08.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.608+0000 7f1588e1f640 1 -- start start 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1588e1f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f158415db00 0x7f15840b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1588e1f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840ceb20 con 0x7f15840d0770 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1588e1f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840b97f0 con 0x7f15840d0bd0 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1588e1f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15841622e0 con 0x7f158415db00 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1581d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f15840d0770 0x7f15841614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38684/0 (socket says 172.21.15.133:38684) 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1581d9b640 1 -- 172.21.15.133:0/3380708943 learned_addr learned my addr 172.21.15.133:0/3380708943 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158259c640 1 --2- 172.21.15.133:0/3380708943 >> v2:172.21.15.162:3300/0 conn(0x7f158415db00 0x7f15840b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271987166 0 0) 0x7f15840ceb20 con 0x7f15840d0770 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f156c003530 con 0x7f15840d0770 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 872885916 0 0) 0x7f156c003530 con 0x7f15840d0770 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 >> v2:172.21.15.162:3300/0 conn(0x7f158415db00 msgr2=0x7f15840b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 --2- 172.21.15.133:0/3380708943 >> v2:172.21.15.162:3300/0 conn(0x7f158415db00 0x7f15840b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 >> v1:172.21.15.162:6789/0 conn(0x7f15840d0bd0 legacy=0x7f1584161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15840b7740 con 0x7f15840d0770 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1578001680 con 0x7f15840d0770 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 >> v1:172.21.15.133:6789/0 conn(0x7f15840d0770 legacy=0x7f15841614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 --2- 172.21.15.133:0/3380708943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f156c0039c0 0x7f156c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 --2- 172.21.15.133:0/3380708943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f156c004810 0x7f156c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 --2- 172.21.15.133:0/3380708943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c0051d0 0x7f156c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15841622e0 con 0x7f156c004810 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c00c070 con 0x7f156c0039c0 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c00c270 con 0x7f156c0051d0 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1581d9b640 1 --2- 172.21.15.133:0/3380708943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f156c004810 0x7f156c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f158259c640 1 --2- 172.21.15.133:0/3380708943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f156c0039c0 0x7f156c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1582d9d640 1 --2- 172.21.15.133:0/3380708943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c0051d0 0x7f156c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1581d9b640 1 -- 172.21.15.133:0/3380708943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c0051d0 msgr2=0x7f156c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1581d9b640 1 --2- 172.21.15.133:0/3380708943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c0051d0 0x7f156c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1581d9b640 1 -- 172.21.15.133:0/3380708943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f156c0039c0 msgr2=0x7f156c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1581d9b640 1 --2- 172.21.15.133:0/3380708943 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f156c0039c0 0x7f156c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1581d9b640 1 -- 172.21.15.133:0/3380708943 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c003530 con 0x7f156c004810 2024-04-25T10:13:08.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.612+0000 7f1582d9d640 1 -- 172.21.15.133:0/3380708943 reap_dead start 2024-04-25T10:13:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1581d9b640 1 --2- 172.21.15.133:0/3380708943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f156c004810 0x7f156c004c90 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f157800c1c0 tx=0x7f157801f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c52e144793d86c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578007d00 con 0x7f156c004810 2024-04-25T10:13:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1578005b00 con 0x7f156c004810 2024-04-25T10:13:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f158159a640 1 -- 172.21.15.133:0/3380708943 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578007070 con 0x7f156c004810 2024-04-25T10:13:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 -- 172.21.15.133:0/3380708943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f156c004810 msgr2=0x7f156c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 --2- 172.21.15.133:0/3380708943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f156c004810 0x7f156c004c90 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f157800c1c0 tx=0x7f157801f3d0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 -- 172.21.15.133:0/3380708943 shutdown_connections 2024-04-25T10:13:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 --2- 172.21.15.133:0/3380708943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f156c004810 0x7f156c004c90 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 -- 172.21.15.133:0/3380708943 >> 172.21.15.133:0/3380708943 conn(0x7f15840727c0 msgr2=0x7f1584072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 -- 172.21.15.133:0/3380708943 shutdown_connections 2024-04-25T10:13:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 -- 172.21.15.133:0/3380708943 wait complete. 2024-04-25T10:13:08.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 Processor -- start 2024-04-25T10:13:08.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 -- start start 2024-04-25T10:13:08.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c004810 0x7f158415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15841e8c10 0x7f15841f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f158415e960 0x7f158415ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840b7fe0 con 0x7f15841e8c10 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840b7d00 con 0x7f158415e960 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15840b82c0 con 0x7f156c004810 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1581d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15841e8c10 0x7f15841f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1581d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15841e8c10 0x7f15841f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37836/0 (socket says 172.21.15.133:37836) 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1581d9b640 1 -- 172.21.15.133:0/1790803723 learned_addr learned my addr 172.21.15.133:0/1790803723 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1581d9b640 1 -- 172.21.15.133:0/1790803723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c004810 msgr2=0x7f158415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1581d9b640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c004810 0x7f158415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1581d9b640 1 -- 172.21.15.133:0/1790803723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f158415e960 msgr2=0x7f158415ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1581d9b640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f158415e960 0x7f158415ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1581d9b640 1 -- 172.21.15.133:0/1790803723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1584160df0 con 0x7f15841e8c10 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1581d9b640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15841e8c10 0x7f15841f30b0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f157800c1c0 tx=0x7f156c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1572ffd640 1 -- 172.21.15.133:0/1790803723 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15780047f0 con 0x7f15841e8c10 2024-04-25T10:13:08.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1572ffd640 1 -- 172.21.15.133:0/1790803723 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1578004990 con 0x7f15841e8c10 2024-04-25T10:13:08.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 -- 172.21.15.133:0/1790803723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15840d4170 con 0x7f15841e8c10 2024-04-25T10:13:08.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1572ffd640 1 -- 172.21.15.133:0/1790803723 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578004d40 con 0x7f15841e8c10 2024-04-25T10:13:08.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.616+0000 7f1588e1f640 1 -- 172.21.15.133:0/1790803723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15841f35f0 con 0x7f15841e8c10 2024-04-25T10:13:08.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.620+0000 7f1588e1f640 1 -- 172.21.15.133:0/1790803723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1584068ef0 con 0x7f15841e8c10 2024-04-25T10:13:08.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.620+0000 7f1572ffd640 1 -- 172.21.15.133:0/1790803723 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15780b9030 con 0x7f15841e8c10 2024-04-25T10:13:08.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.620+0000 7f1572ffd640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1564066f60 0x7f1564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.620+0000 7f1572ffd640 1 -- 172.21.15.133:0/1790803723 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1578121dc0 con 0x7f15841e8c10 2024-04-25T10:13:08.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.620+0000 7f158259c640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1564066f60 0x7f1564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.620+0000 7f158259c640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1564066f60 0x7f1564069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f158404f450 tx=0x7f1574008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.624+0000 7f1572ffd640 1 -- 172.21.15.133:0/1790803723 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1584160df0 con 0x7f15841e8c10 2024-04-25T10:13:08.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.772+0000 7f1588e1f640 1 -- 172.21.15.133:0/1790803723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f15840b91c0 con 0x7f15841e8c10 2024-04-25T10:13:08.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1572ffd640 1 -- 172.21.15.133:0/1790803723 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) v1 ==== 84+0+65 (secure 0 0 0) 0x7f15780190e0 con 0x7f15841e8c10 2024-04-25T10:13:08.778 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:08.778 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:13:08.778 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:13:08.778 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:13:08.778 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:13:08.778 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:08.778 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:13:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 -- 172.21.15.133:0/1790803723 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1564066f60 msgr2=0x7f1564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1564066f60 0x7f1564069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f158404f450 tx=0x7f1574008040 comp rx=0 tx=0).stop 2024-04-25T10:13:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 -- 172.21.15.133:0/1790803723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15841e8c10 msgr2=0x7f15841f30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15841e8c10 0x7f15841f30b0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f157800c1c0 tx=0x7f156c005840 comp rx=0 tx=0).stop 2024-04-25T10:13:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 -- 172.21.15.133:0/1790803723 shutdown_connections 2024-04-25T10:13:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f158415e960 0x7f158415ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1564066f60 0x7f1564069420 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15841e8c10 0x7f15841f30b0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 --2- 172.21.15.133:0/1790803723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f156c004810 0x7f158415dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 -- 172.21.15.133:0/1790803723 >> 172.21.15.133:0/1790803723 conn(0x7f15840727c0 msgr2=0x7f15840b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 -- 172.21.15.133:0/1790803723 shutdown_connections 2024-04-25T10:13:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.776+0000 7f1588e1f640 1 -- 172.21.15.133:0/1790803723 wait complete. 2024-04-25T10:13:08.795 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:13:08.795 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:13:08.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f661543e640 1 Processor -- start 2024-04-25T10:13:08.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f661543e640 1 -- start start 2024-04-25T10:13:08.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f661543e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f661015dca0 0x7f66100ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f661543e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f66100ced40 0x7f66100d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f661543e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66100d1cf0 con 0x7f661015d8c0 2024-04-25T10:13:08.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f661543e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66101626f0 con 0x7f66100ced40 2024-04-25T10:13:08.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f661543e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66100cf1b0 con 0x7f661015dca0 2024-04-25T10:13:08.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f660f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f66100ced40 0x7f66100d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f660f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f66100ced40 0x7f66100d24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37848/0 (socket says 172.21.15.133:37848) 2024-04-25T10:13:08.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f660f7fe640 1 -- 172.21.15.133:0/4189501146 learned_addr learned my addr 172.21.15.133:0/4189501146 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:08.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f66067fc640 1 --2- 172.21.15.133:0/4189501146 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f661015dca0 0x7f66100ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f660f7fe640 1 -- 172.21.15.133:0/4189501146 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f661015dca0 msgr2=0x7f66100ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f660f7fe640 1 --2- 172.21.15.133:0/4189501146 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f661015dca0 0x7f66100ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f660f7fe640 1 -- 172.21.15.133:0/4189501146 >> v1:172.21.15.162:6789/0 conn(0x7f661015d8c0 legacy=0x7f6610161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.912+0000 7f660f7fe640 1 -- 172.21.15.133:0/4189501146 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610162130 con 0x7f66100ced40 2024-04-25T10:13:08.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f660f7fe640 1 --2- 172.21.15.133:0/4189501146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66100ced40 0x7f66100d24b0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f6608009e50 tx=0x7f660801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e977c74dc7c470a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f660e7fc640 1 -- 172.21.15.133:0/4189501146 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66080a5840 con 0x7f66100ced40 2024-04-25T10:13:08.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f660e7fc640 1 -- 172.21.15.133:0/4189501146 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66080a59e0 con 0x7f66100ced40 2024-04-25T10:13:08.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f660e7fc640 1 -- 172.21.15.133:0/4189501146 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66080a5db0 con 0x7f66100ced40 2024-04-25T10:13:08.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f661543e640 1 -- 172.21.15.133:0/4189501146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66100ced40 msgr2=0x7f66100d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f661543e640 1 --2- 172.21.15.133:0/4189501146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66100ced40 0x7f66100d24b0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f6608009e50 tx=0x7f660801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f661543e640 1 -- 172.21.15.133:0/4189501146 shutdown_connections 2024-04-25T10:13:08.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f661543e640 1 --2- 172.21.15.133:0/4189501146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66100ced40 0x7f66100d24b0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f661543e640 1 --2- 172.21.15.133:0/4189501146 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f661015dca0 0x7f66100ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f661543e640 1 -- 172.21.15.133:0/4189501146 >> 172.21.15.133:0/4189501146 conn(0x7f66100723c0 msgr2=0x7f66100727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f661543e640 1 -- 172.21.15.133:0/4189501146 shutdown_connections 2024-04-25T10:13:08.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.916+0000 7f661543e640 1 -- 172.21.15.133:0/4189501146 wait complete. 2024-04-25T10:13:08.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f661543e640 1 Processor -- start 2024-04-25T10:13:08.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f661543e640 1 -- start start 2024-04-25T10:13:08.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f661543e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ced40 0x7f66100cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f661543e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f661015d8c0 0x7f6610159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f661543e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015dca0 0x7f66101542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f661543e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66100b9990 con 0x7f661015d8c0 2024-04-25T10:13:08.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f661543e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66100b9810 con 0x7f661015dca0 2024-04-25T10:13:08.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f661543e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66100d3200 con 0x7f66100ced40 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f66067fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f661015d8c0 0x7f6610159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f66067fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f661015d8c0 0x7f6610159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37862/0 (socket says 172.21.15.133:37862) 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f66067fc640 1 -- 172.21.15.133:0/1987949473 learned_addr learned my addr 172.21.15.133:0/1987949473 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f660effd640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ced40 0x7f66100cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f660f7fe640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015dca0 0x7f66101542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f66067fc640 1 -- 172.21.15.133:0/1987949473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ced40 msgr2=0x7f66100cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f66067fc640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ced40 0x7f66100cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f66067fc640 1 -- 172.21.15.133:0/1987949473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015dca0 msgr2=0x7f66101542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f66067fc640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015dca0 0x7f66101542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f66067fc640 1 -- 172.21.15.133:0/1987949473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66100cb200 con 0x7f661015d8c0 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f660f7fe640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015dca0 0x7f66101542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f66067fc640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f661015d8c0 0x7f6610159330 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f65f800d5f0 tx=0x7f65f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f6607fff640 1 -- 172.21.15.133:0/1987949473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80125f0 con 0x7f661015d8c0 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f6607fff640 1 -- 172.21.15.133:0/1987949473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f8019070 con 0x7f661015d8c0 2024-04-25T10:13:08.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f6607fff640 1 -- 172.21.15.133:0/1987949473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80129b0 con 0x7f661015d8c0 2024-04-25T10:13:08.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f661543e640 1 -- 172.21.15.133:0/1987949473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66100cb4e0 con 0x7f661015d8c0 2024-04-25T10:13:08.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f661543e640 1 -- 172.21.15.133:0/1987949473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6610158b30 con 0x7f661015d8c0 2024-04-25T10:13:08.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f661543e640 1 -- 172.21.15.133:0/1987949473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6610069000 con 0x7f661015d8c0 2024-04-25T10:13:08.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.920+0000 7f6607fff640 1 -- 172.21.15.133:0/1987949473 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65f8012b50 con 0x7f661015d8c0 2024-04-25T10:13:08.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.924+0000 7f6607fff640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f0066f60 0x7f65f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.924+0000 7f6607fff640 1 -- 172.21.15.133:0/1987949473 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f65f80852b0 con 0x7f661015d8c0 2024-04-25T10:13:08.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.924+0000 7f660effd640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f0066f60 0x7f65f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.924+0000 7f660effd640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f0066f60 0x7f65f0069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6600004e30 tx=0x7f6600009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:08.928+0000 7f6607fff640 1 -- 172.21.15.133:0/1987949473 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65f8056480 con 0x7f661015d8c0 2024-04-25T10:13:09.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.084+0000 7f661543e640 1 -- 172.21.15.133:0/1987949473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f66100d1700 con 0x7f661015d8c0 2024-04-25T10:13:09.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.464+0000 7f6607fff640 1 -- 172.21.15.133:0/1987949473 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) v1 ==== 138+0+0 (secure 0 0 0) 0x7f65f80850d0 con 0x7f661015d8c0 2024-04-25T10:13:09.469 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:13:09.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 -- 172.21.15.133:0/1987949473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f0066f60 msgr2=0x7f65f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f0066f60 0x7f65f0069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6600004e30 tx=0x7f6600009040 comp rx=0 tx=0).stop 2024-04-25T10:13:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 -- 172.21.15.133:0/1987949473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f661015d8c0 msgr2=0x7f6610159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f661015d8c0 0x7f6610159330 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f65f800d5f0 tx=0x7f65f800dab0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 -- 172.21.15.133:0/1987949473 shutdown_connections 2024-04-25T10:13:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015dca0 0x7f66101542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f0066f60 0x7f65f0069420 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f661015d8c0 0x7f6610159330 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 --2- 172.21.15.133:0/1987949473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ced40 0x7f66100cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 -- 172.21.15.133:0/1987949473 >> 172.21.15.133:0/1987949473 conn(0x7f66100723c0 msgr2=0x7f661015aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 -- 172.21.15.133:0/1987949473 shutdown_connections 2024-04-25T10:13:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.468+0000 7f661543e640 1 -- 172.21.15.133:0/1987949473 wait complete. 2024-04-25T10:13:09.497 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:13:09.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b5cb17640 1 Processor -- start 2024-04-25T10:13:09.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b5cb17640 1 -- start start 2024-04-25T10:13:09.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b5cb17640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b580cd820 0x7f9b580cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b5cb17640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9b5815ced0 0x7f9b5815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b5cb17640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b580ce2e0 con 0x7f9b58145170 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b5cb17640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5806db00 con 0x7f9b5815ced0 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b5cb17640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b58145550 con 0x7f9b580cd820 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b56d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9b5815ced0 0x7f9b5815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b56d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9b5815ced0 0x7f9b5815d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37874/0 (socket says 172.21.15.133:37874) 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b56d9d640 1 -- 172.21.15.133:0/606339938 learned_addr learned my addr 172.21.15.133:0/606339938 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b55d9b640 1 --2- 172.21.15.133:0/606339938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b580cd820 0x7f9b580cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b56d9d640 1 -- 172.21.15.133:0/606339938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b580cd820 msgr2=0x7f9b580cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b56d9d640 1 --2- 172.21.15.133:0/606339938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b580cd820 0x7f9b580cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b56d9d640 1 -- 172.21.15.133:0/606339938 >> v1:172.21.15.162:6789/0 conn(0x7f9b58145170 legacy=0x7f9b5806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.616+0000 7f9b56d9d640 1 -- 172.21.15.133:0/606339938 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5806d4b0 con 0x7f9b5815ced0 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b56d9d640 1 --2- 172.21.15.133:0/606339938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b5815ced0 0x7f9b5815d360 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009e50 tx=0x7f9b5001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6777cea33760d67e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5559a640 1 -- 172.21.15.133:0/606339938 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b500a5670 con 0x7f9b5815ced0 2024-04-25T10:13:09.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5559a640 1 -- 172.21.15.133:0/606339938 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b500a5810 con 0x7f9b5815ced0 2024-04-25T10:13:09.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5559a640 1 -- 172.21.15.133:0/606339938 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b500a5be0 con 0x7f9b5815ced0 2024-04-25T10:13:09.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5cb17640 1 -- 172.21.15.133:0/606339938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b5815ced0 msgr2=0x7f9b5815d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5cb17640 1 --2- 172.21.15.133:0/606339938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b5815ced0 0x7f9b5815d360 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009e50 tx=0x7f9b5001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5cb17640 1 -- 172.21.15.133:0/606339938 shutdown_connections 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5cb17640 1 --2- 172.21.15.133:0/606339938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b5815ced0 0x7f9b5815d360 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5cb17640 1 --2- 172.21.15.133:0/606339938 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b580cd820 0x7f9b580cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5cb17640 1 -- 172.21.15.133:0/606339938 >> 172.21.15.133:0/606339938 conn(0x7f9b58066f70 msgr2=0x7f9b5806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5cb17640 1 -- 172.21.15.133:0/606339938 shutdown_connections 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5cb17640 1 -- 172.21.15.133:0/606339938 wait complete. 2024-04-25T10:13:09.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5cb17640 1 Processor -- start 2024-04-25T10:13:09.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.620+0000 7f9b5cb17640 1 -- start start 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b5cb17640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b580cd820 0x7f9b58137ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b5cb17640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b58145170 0x7f9b5813ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b5cb17640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b5815ced0 0x7f9b58135640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b5cb17640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b58168610 con 0x7f9b58145170 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b5cb17640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b58168490 con 0x7f9b580cd820 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b5cb17640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b58168790 con 0x7f9b5815ced0 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b55d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b58145170 0x7f9b5813ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b55d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b58145170 0x7f9b5813ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37882/0 (socket says 172.21.15.133:37882) 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b55d9b640 1 -- 172.21.15.133:0/1360929036 learned_addr learned my addr 172.21.15.133:0/1360929036 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b56d9d640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b5815ced0 0x7f9b58135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b55d9b640 1 -- 172.21.15.133:0/1360929036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b5815ced0 msgr2=0x7f9b58135640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b55d9b640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b5815ced0 0x7f9b58135640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b55d9b640 1 -- 172.21.15.133:0/1360929036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b580cd820 msgr2=0x7f9b58137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b55d9b640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b580cd820 0x7f9b58137ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b55d9b640 1 -- 172.21.15.133:0/1360929036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5813b0a0 con 0x7f9b58145170 2024-04-25T10:13:09.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b55d9b640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b58145170 0x7f9b5813ab60 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00d620 tx=0x7f9b4c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b46ffd640 1 -- 172.21.15.133:0/1360929036 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00b620 con 0x7f9b58145170 2024-04-25T10:13:09.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b46ffd640 1 -- 172.21.15.133:0/1360929036 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c019070 con 0x7f9b58145170 2024-04-25T10:13:09.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b5cb17640 1 -- 172.21.15.133:0/1360929036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5813db60 con 0x7f9b58145170 2024-04-25T10:13:09.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b46ffd640 1 -- 172.21.15.133:0/1360929036 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00b9e0 con 0x7f9b58145170 2024-04-25T10:13:09.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b5cb17640 1 -- 172.21.15.133:0/1360929036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5813e120 con 0x7f9b58145170 2024-04-25T10:13:09.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b5cb17640 1 -- 172.21.15.133:0/1360929036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b58068e80 con 0x7f9b58145170 2024-04-25T10:13:09.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b46ffd640 1 -- 172.21.15.133:0/1360929036 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b4c00bb80 con 0x7f9b58145170 2024-04-25T10:13:09.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b46ffd640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b38066f60 0x7f9b38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b46ffd640 1 -- 172.21.15.133:0/1360929036 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f9b4c085e70 con 0x7f9b58145170 2024-04-25T10:13:09.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b5659c640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b38066f60 0x7f9b38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.624+0000 7f9b5659c640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b38066f60 0x7f9b38069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9b48003cf0 tx=0x7f9b480079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.628+0000 7f9b46ffd640 1 -- 172.21.15.133:0/1360929036 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b4c056fb0 con 0x7f9b58145170 2024-04-25T10:13:09.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:09.776+0000 7f9b5cb17640 1 -- 172.21.15.133:0/1360929036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9b5813b360 con 0x7f9b58145170 2024-04-25T10:13:10.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.472+0000 7f9b46ffd640 1 -- 172.21.15.133:0/1360929036 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9b4c0568b0 con 0x7f9b58145170 2024-04-25T10:13:10.474 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 -- 172.21.15.133:0/1360929036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b38066f60 msgr2=0x7f9b38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b38066f60 0x7f9b38069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9b48003cf0 tx=0x7f9b480079c0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 -- 172.21.15.133:0/1360929036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b58145170 msgr2=0x7f9b5813ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b58145170 0x7f9b5813ab60 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00d620 tx=0x7f9b4c00dae0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 -- 172.21.15.133:0/1360929036 shutdown_connections 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b5815ced0 0x7f9b58135640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b38066f60 0x7f9b38069420 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b58145170 0x7f9b5813ab60 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 --2- 172.21.15.133:0/1360929036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b580cd820 0x7f9b58137ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 -- 172.21.15.133:0/1360929036 >> 172.21.15.133:0/1360929036 conn(0x7f9b58066f70 msgr2=0x7f9b58144180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 -- 172.21.15.133:0/1360929036 shutdown_connections 2024-04-25T10:13:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.476+0000 7f9b5cb17640 1 -- 172.21.15.133:0/1360929036 wait complete. 2024-04-25T10:13:10.518 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:13:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3aee98b640 1 Processor -- start 2024-04-25T10:13:10.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3aee98b640 1 -- start start 2024-04-25T10:13:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3aee98b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ae80ce600 0x7f3ae80cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3aee98b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae815ebd0 con 0x7f3ae80d1920 2024-04-25T10:13:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3aee98b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae815eeb0 con 0x7f3ae815d9a0 2024-04-25T10:13:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3aee98b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae80d0330 con 0x7f3ae80ce600 2024-04-25T10:13:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3ae77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ae80ce600 0x7f3ae80cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3aecf28640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3ae80d1920 0x7f3ae80d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38688/0 (socket says 172.21.15.133:38688) 2024-04-25T10:13:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3aecf28640 1 -- 172.21.15.133:0/1225477109 learned_addr learned my addr 172.21.15.133:0/1225477109 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3ae77fe640 1 -- 172.21.15.133:0/1225477109 >> v1:172.21.15.162:6789/0 conn(0x7f3ae815d9a0 legacy=0x7f3ae80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3ae77fe640 1 -- 172.21.15.133:0/1225477109 >> v1:172.21.15.133:6789/0 conn(0x7f3ae80d1920 legacy=0x7f3ae80d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.640+0000 7f3ae77fe640 1 -- 172.21.15.133:0/1225477109 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae80d0610 con 0x7f3ae80ce600 2024-04-25T10:13:10.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3ae77fe640 1 --2- 172.21.15.133:0/1225477109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ae80ce600 0x7f3ae80cea90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00097c0 tx=0x7f3ad001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f509aa82c720484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3ae6ffd640 1 -- 172.21.15.133:0/1225477109 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00b0070 con 0x7f3ae80ce600 2024-04-25T10:13:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3ae6ffd640 1 -- 172.21.15.133:0/1225477109 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad00ab600 con 0x7f3ae80ce600 2024-04-25T10:13:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3ae6ffd640 1 -- 172.21.15.133:0/1225477109 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00ab9d0 con 0x7f3ae80ce600 2024-04-25T10:13:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 -- 172.21.15.133:0/1225477109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ae80ce600 msgr2=0x7f3ae80cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 --2- 172.21.15.133:0/1225477109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ae80ce600 0x7f3ae80cea90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00097c0 tx=0x7f3ad001c3d0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 -- 172.21.15.133:0/1225477109 shutdown_connections 2024-04-25T10:13:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 --2- 172.21.15.133:0/1225477109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ae80ce600 0x7f3ae80cea90 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 -- 172.21.15.133:0/1225477109 >> 172.21.15.133:0/1225477109 conn(0x7f3ae806bf80 msgr2=0x7f3ae806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 -- 172.21.15.133:0/1225477109 shutdown_connections 2024-04-25T10:13:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 -- 172.21.15.133:0/1225477109 wait complete. 2024-04-25T10:13:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 Processor -- start 2024-04-25T10:13:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 -- start start 2024-04-25T10:13:10.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ae80ce600 0x7f3ae80b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ae80d1920 0x7f3ae80ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ae815d9a0 0x7f3ae80cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae815e710 con 0x7f3ae815d9a0 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae815e590 con 0x7f3ae80d1920 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aee98b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae815e890 con 0x7f3ae80ce600 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aecf28640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ae815d9a0 0x7f3ae80cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aecf28640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ae815d9a0 0x7f3ae80cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37900/0 (socket says 172.21.15.133:37900) 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aecf28640 1 -- 172.21.15.133:0/3267969516 learned_addr learned my addr 172.21.15.133:0/3267969516 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aecf28640 1 -- 172.21.15.133:0/3267969516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ae80ce600 msgr2=0x7f3ae80b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3ae7fff640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ae80ce600 0x7f3ae80b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3ae77fe640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ae80d1920 0x7f3ae80ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aecf28640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ae80ce600 0x7f3ae80b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aecf28640 1 -- 172.21.15.133:0/3267969516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ae80d1920 msgr2=0x7f3ae80ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aecf28640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ae80d1920 0x7f3ae80ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aecf28640 1 -- 172.21.15.133:0/3267969516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae80cb350 con 0x7f3ae815d9a0 2024-04-25T10:13:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.644+0000 7f3aecf28640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ae815d9a0 0x7f3ae80cc3f0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000ca80 tx=0x7f3ae000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.648+0000 7f3ae4ff9640 1 -- 172.21.15.133:0/3267969516 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0017070 con 0x7f3ae815d9a0 2024-04-25T10:13:10.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.648+0000 7f3ae4ff9640 1 -- 172.21.15.133:0/3267969516 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae000ac30 con 0x7f3ae815d9a0 2024-04-25T10:13:10.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.648+0000 7f3ae4ff9640 1 -- 172.21.15.133:0/3267969516 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0012600 con 0x7f3ae815d9a0 2024-04-25T10:13:10.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.648+0000 7f3aee98b640 1 -- 172.21.15.133:0/3267969516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae80cb630 con 0x7f3ae815d9a0 2024-04-25T10:13:10.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.648+0000 7f3aee98b640 1 -- 172.21.15.133:0/3267969516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae80c9300 con 0x7f3ae815d9a0 2024-04-25T10:13:10.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.648+0000 7f3aee98b640 1 -- 172.21.15.133:0/3267969516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae8068e80 con 0x7f3ae815d9a0 2024-04-25T10:13:10.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.648+0000 7f3ae4ff9640 1 -- 172.21.15.133:0/3267969516 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ae001a070 con 0x7f3ae815d9a0 2024-04-25T10:13:10.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.648+0000 7f3ae4ff9640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ac8066f60 0x7f3ac8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.648+0000 7f3ae4ff9640 1 -- 172.21.15.133:0/3267969516 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f3ae0084d50 con 0x7f3ae815d9a0 2024-04-25T10:13:10.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.648+0000 7f3ae7fff640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ac8066f60 0x7f3ac8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.648+0000 7f3ae7fff640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ac8066f60 0x7f3ac8069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8003d60 tx=0x7f3ad8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.652+0000 7f3ae4ff9640 1 -- 172.21.15.133:0/3267969516 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ae0088030 con 0x7f3ae815d9a0 2024-04-25T10:13:10.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:10.852+0000 7f3aee98b640 1 -- 172.21.15.133:0/3267969516 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3ae80d1700 con 0x7f3ae815d9a0 2024-04-25T10:13:11.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.496+0000 7f3ae4ff9640 1 -- 172.21.15.133:0/3267969516 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v76) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3ae000ae20 con 0x7f3ae815d9a0 2024-04-25T10:13:11.499 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 19 and data pool 20 2024-04-25T10:13:11.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.500+0000 7f3aee98b640 1 -- 172.21.15.133:0/3267969516 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ac8066f60 msgr2=0x7f3ac8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.500+0000 7f3aee98b640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ac8066f60 0x7f3ac8069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8003d60 tx=0x7f3ad8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:11.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.500+0000 7f3aee98b640 1 -- 172.21.15.133:0/3267969516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ae815d9a0 msgr2=0x7f3ae80cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.500+0000 7f3aee98b640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ae815d9a0 0x7f3ae80cc3f0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000ca80 tx=0x7f3ae000cf40 comp rx=0 tx=0).stop 2024-04-25T10:13:11.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.500+0000 7f3aee98b640 1 -- 172.21.15.133:0/3267969516 shutdown_connections 2024-04-25T10:13:11.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.500+0000 7f3aee98b640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ac8066f60 0x7f3ac8069420 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.500+0000 7f3aee98b640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ae815d9a0 0x7f3ae80cc3f0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.500+0000 7f3aee98b640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ae80d1920 0x7f3ae80ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.500+0000 7f3aee98b640 1 --2- 172.21.15.133:0/3267969516 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ae80ce600 0x7f3ae80b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.500+0000 7f3aee98b640 1 -- 172.21.15.133:0/3267969516 >> 172.21.15.133:0/3267969516 conn(0x7f3ae806bf80 msgr2=0x7f3ae80b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.504+0000 7f3aee98b640 1 -- 172.21.15.133:0/3267969516 shutdown_connections 2024-04-25T10:13:11.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.504+0000 7f3aee98b640 1 -- 172.21.15.133:0/3267969516 wait complete. 2024-04-25T10:13:11.546 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:11.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f0085dbb640 1 Processor -- start 2024-04-25T10:13:11.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f0085dbb640 1 -- start start 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f0085dbb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f008015b990 0x7f00800d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f0085dbb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 0x7f00800b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f0085dbb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f008015f7c0 0x7f008015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f0085dbb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00800b2230 con 0x7f008015f7c0 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f0085dbb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00800cf490 con 0x7f008015b990 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f0085dbb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00800cf770 con 0x7f00800b1890 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f007ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f008015f7c0 0x7f008015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f007ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f008015f7c0 0x7f008015fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37910/0 (socket says 172.21.15.133:37910) 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f007ffff640 1 -- 172.21.15.133:0/4150017302 learned_addr learned my addr 172.21.15.133:0/4150017302 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f007effd640 1 --2- 172.21.15.133:0/4150017302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 0x7f00800b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f007f7fe640 1 --2- 172.21.15.133:0/4150017302 >> v2:172.21.15.162:3300/0 conn(0x7f008015b990 0x7f00800d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f007ffff640 1 -- 172.21.15.133:0/4150017302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 msgr2=0x7f00800b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f007ffff640 1 --2- 172.21.15.133:0/4150017302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 0x7f00800b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f007ffff640 1 -- 172.21.15.133:0/4150017302 >> v2:172.21.15.162:3300/0 conn(0x7f008015b990 msgr2=0x7f00800d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f007ffff640 1 --2- 172.21.15.133:0/4150017302 >> v2:172.21.15.162:3300/0 conn(0x7f008015b990 0x7f00800d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f007ffff640 1 -- 172.21.15.133:0/4150017302 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00800cfa50 con 0x7f008015f7c0 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.664+0000 7f007effd640 1 --2- 172.21.15.133:0/4150017302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 0x7f00800b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f007ffff640 1 --2- 172.21.15.133:0/4150017302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f008015f7c0 0x7f008015fc50 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f0078009e50 tx=0x7f007801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15df45f7938974b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f007e7fc640 1 -- 172.21.15.133:0/4150017302 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780a5690 con 0x7f008015f7c0 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f007e7fc640 1 -- 172.21.15.133:0/4150017302 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00780a5830 con 0x7f008015f7c0 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f007e7fc640 1 -- 172.21.15.133:0/4150017302 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780a5c00 con 0x7f008015f7c0 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f0085dbb640 1 -- 172.21.15.133:0/4150017302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f008015f7c0 msgr2=0x7f008015fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f0085dbb640 1 --2- 172.21.15.133:0/4150017302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f008015f7c0 0x7f008015fc50 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f0078009e50 tx=0x7f007801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f0085dbb640 1 -- 172.21.15.133:0/4150017302 shutdown_connections 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f0085dbb640 1 --2- 172.21.15.133:0/4150017302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f008015f7c0 0x7f008015fc50 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f0085dbb640 1 --2- 172.21.15.133:0/4150017302 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 0x7f00800b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f0085dbb640 1 --2- 172.21.15.133:0/4150017302 >> v2:172.21.15.162:3300/0 conn(0x7f008015b990 0x7f00800d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f0085dbb640 1 -- 172.21.15.133:0/4150017302 >> 172.21.15.133:0/4150017302 conn(0x7f008009aa10 msgr2=0x7f008008af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f0085dbb640 1 -- 172.21.15.133:0/4150017302 shutdown_connections 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f0085dbb640 1 -- 172.21.15.133:0/4150017302 wait complete. 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f0085dbb640 1 Processor -- start 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.668+0000 7f0085dbb640 1 -- start start 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0085dbb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 0x7f00801486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0085dbb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008015b990 0x7f00800c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0085dbb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f008015f7c0 0x7f00800c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0085dbb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00800ccbc0 con 0x7f008015f7c0 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0085dbb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00800cca40 con 0x7f008015b990 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0085dbb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00800ccd40 con 0x7f00800b1890 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f008015f7c0 0x7f00800c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f008015f7c0 0x7f00800c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37912/0 (socket says 172.21.15.133:37912) 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007ffff640 1 -- 172.21.15.133:0/2293888400 learned_addr learned my addr 172.21.15.133:0/2293888400 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:11.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007effd640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008015b990 0x7f00800c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007f7fe640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 0x7f00801486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007ffff640 1 -- 172.21.15.133:0/2293888400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 msgr2=0x7f00801486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007ffff640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 0x7f00801486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007ffff640 1 -- 172.21.15.133:0/2293888400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008015b990 msgr2=0x7f00800c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007ffff640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008015b990 0x7f00800c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007ffff640 1 -- 172.21.15.133:0/2293888400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008014a8e0 con 0x7f008015f7c0 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007f7fe640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 0x7f00801486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007ffff640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f008015f7c0 0x7f00800c74e0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f00780af040 tx=0x7f0078019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0067fff640 1 -- 172.21.15.133:0/2293888400 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00780bc070 con 0x7f008015f7c0 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0067fff640 1 -- 172.21.15.133:0/2293888400 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00780b8640 con 0x7f008015f7c0 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0085dbb640 1 -- 172.21.15.133:0/2293888400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008014abc0 con 0x7f008015f7c0 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0067fff640 1 -- 172.21.15.133:0/2293888400 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078009510 con 0x7f008015f7c0 2024-04-25T10:13:11.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0085dbb640 1 -- 172.21.15.133:0/2293888400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008014b180 con 0x7f008015f7c0 2024-04-25T10:13:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0085dbb640 1 -- 172.21.15.133:0/2293888400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0080068ef0 con 0x7f008015f7c0 2024-04-25T10:13:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0067fff640 1 -- 172.21.15.133:0/2293888400 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00780b89f0 con 0x7f008015f7c0 2024-04-25T10:13:11.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0067fff640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f006c066f60 0x7f006c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007f7fe640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f006c066f60 0x7f006c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f0067fff640 1 -- 172.21.15.133:0/2293888400 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0078120130 con 0x7f008015f7c0 2024-04-25T10:13:11.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.672+0000 7f007f7fe640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f006c066f60 0x7f006c069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f00700079f0 tx=0x7f0070008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.676+0000 7f0067fff640 1 -- 172.21.15.133:0/2293888400 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00780165e0 con 0x7f008015f7c0 2024-04-25T10:13:11.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.844+0000 7f0085dbb640 1 -- 172.21.15.133:0/2293888400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0080068100 con 0x7f008015f7c0 2024-04-25T10:13:11.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0067fff640 1 -- 172.21.15.133:0/2293888400 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16937 (secure 0 0 0) 0x7f00780f1040 con 0x7f008015f7c0 2024-04-25T10:13:11.850 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:11.850 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":131,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:13:10.855906+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:09.128791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:09.816841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 -- 172.21.15.133:0/2293888400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f006c066f60 msgr2=0x7f006c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f006c066f60 0x7f006c069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f00700079f0 tx=0x7f0070008040 comp rx=0 tx=0).stop 2024-04-25T10:13:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 -- 172.21.15.133:0/2293888400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f008015f7c0 msgr2=0x7f00800c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f008015f7c0 0x7f00800c74e0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f00780af040 tx=0x7f0078019790 comp rx=0 tx=0).stop 2024-04-25T10:13:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 -- 172.21.15.133:0/2293888400 shutdown_connections 2024-04-25T10:13:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f006c066f60 0x7f006c069420 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f008015f7c0 0x7f00800c74e0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f008015b990 0x7f00800c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 --2- 172.21.15.133:0/2293888400 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f00800b1890 0x7f00801486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 -- 172.21.15.133:0/2293888400 >> 172.21.15.133:0/2293888400 conn(0x7f008009aa10 msgr2=0x7f0080070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 -- 172.21.15.133:0/2293888400 shutdown_connections 2024-04-25T10:13:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.848+0000 7f0085dbb640 1 -- 172.21.15.133:0/2293888400 wait complete. 2024-04-25T10:13:11.870 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:11.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.992+0000 7f523ca42640 1 Processor -- start 2024-04-25T10:13:11.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.992+0000 7f523ca42640 1 -- start start 2024-04-25T10:13:11.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523ca42640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5238152b30 0x7f52380b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523ca42640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f52380cea40 0x7f52380b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523ca42640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523806d350 con 0x7f5238152750 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523ca42640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523815bf90 con 0x7f52380cea40 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523ca42640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523806d5d0 con 0x7f5238152b30 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523659c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5238152750 0x7f52380ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38700/0 (socket says 172.21.15.133:38700) 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523659c640 1 -- 172.21.15.133:0/2570546225 learned_addr learned my addr 172.21.15.133:0/2570546225 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f5236d9d640 1 --2- 172.21.15.133:0/2570546225 >> v2:172.21.15.133:3300/0 conn(0x7f52380cea40 0x7f52380b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523559a640 1 -- 172.21.15.133:0/2570546225 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990259951 0 0) 0x7f52300014a0 con 0x7f5238152750 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f5235d9b640 1 --2- 172.21.15.133:0/2570546225 >> v2:172.21.15.162:3300/0 conn(0x7f5238152b30 0x7f52380b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523559a640 1 -- 172.21.15.133:0/2570546225 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5224003530 con 0x7f5238152750 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f5236d9d640 1 -- 172.21.15.133:0/2570546225 >> v2:172.21.15.162:3300/0 conn(0x7f5238152b30 msgr2=0x7f52380b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f5236d9d640 1 --2- 172.21.15.133:0/2570546225 >> v2:172.21.15.162:3300/0 conn(0x7f5238152b30 0x7f52380b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f5236d9d640 1 -- 172.21.15.133:0/2570546225 >> v1:172.21.15.133:6789/0 conn(0x7f5238152750 legacy=0x7f52380ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f5236d9d640 1 -- 172.21.15.133:0/2570546225 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523815c230 con 0x7f52380cea40 2024-04-25T10:13:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f5236d9d640 1 --2- 172.21.15.133:0/2570546225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52380cea40 0x7f52380b1cf0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f523806f1c0 tx=0x7f522801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8efc2eb060f5368 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523559a640 1 -- 172.21.15.133:0/2570546225 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280b0070 con 0x7f52380cea40 2024-04-25T10:13:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523559a640 1 -- 172.21.15.133:0/2570546225 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5228004230 con 0x7f52380cea40 2024-04-25T10:13:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:11.996+0000 7f523559a640 1 -- 172.21.15.133:0/2570546225 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228003620 con 0x7f52380cea40 2024-04-25T10:13:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 -- 172.21.15.133:0/2570546225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52380cea40 msgr2=0x7f52380b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 --2- 172.21.15.133:0/2570546225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52380cea40 0x7f52380b1cf0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f523806f1c0 tx=0x7f522801c460 comp rx=0 tx=0).stop 2024-04-25T10:13:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 -- 172.21.15.133:0/2570546225 shutdown_connections 2024-04-25T10:13:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 --2- 172.21.15.133:0/2570546225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52380cea40 0x7f52380b1cf0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 --2- 172.21.15.133:0/2570546225 >> v2:172.21.15.162:3300/0 conn(0x7f5238152b30 0x7f52380b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 -- 172.21.15.133:0/2570546225 >> 172.21.15.133:0/2570546225 conn(0x7f5238067000 msgr2=0x7f523806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 -- 172.21.15.133:0/2570546225 shutdown_connections 2024-04-25T10:13:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 -- 172.21.15.133:0/2570546225 wait complete. 2024-04-25T10:13:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 Processor -- start 2024-04-25T10:13:12.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 -- start start 2024-04-25T10:13:12.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52380cea40 0x7f5238148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5238152750 0x7f5238148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5238152b30 0x7f523814d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523815fe70 con 0x7f5238152b30 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523815fcf0 con 0x7f5238152750 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523ca42640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523815fff0 con 0x7f52380cea40 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f5236d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5238152b30 0x7f523814d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f5236d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5238152b30 0x7f523814d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37926/0 (socket says 172.21.15.133:37926) 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f5236d9d640 1 -- 172.21.15.133:0/2865716867 learned_addr learned my addr 172.21.15.133:0/2865716867 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523659c640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52380cea40 0x7f5238148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f5235d9b640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5238152750 0x7f5238148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f5236d9d640 1 -- 172.21.15.133:0/2865716867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52380cea40 msgr2=0x7f5238148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f5236d9d640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52380cea40 0x7f5238148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f5236d9d640 1 -- 172.21.15.133:0/2865716867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5238152750 msgr2=0x7f5238148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f5236d9d640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5238152750 0x7f5238148080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f5236d9d640 1 -- 172.21.15.133:0/2865716867 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380b3630 con 0x7f5238152b30 2024-04-25T10:13:12.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f523659c640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52380cea40 0x7f5238148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:12.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.000+0000 7f5236d9d640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5238152b30 0x7f523814d150 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f5228019070 tx=0x7f5228019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.004+0000 7f522effd640 1 -- 172.21.15.133:0/2865716867 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280b9020 con 0x7f5238152b30 2024-04-25T10:13:12.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.004+0000 7f522effd640 1 -- 172.21.15.133:0/2865716867 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52280043e0 con 0x7f5238152b30 2024-04-25T10:13:12.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.004+0000 7f523ca42640 1 -- 172.21.15.133:0/2865716867 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5238149b80 con 0x7f5238152b30 2024-04-25T10:13:12.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.004+0000 7f522effd640 1 -- 172.21.15.133:0/2865716867 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280b0040 con 0x7f5238152b30 2024-04-25T10:13:12.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.004+0000 7f523ca42640 1 -- 172.21.15.133:0/2865716867 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52381492e0 con 0x7f5238152b30 2024-04-25T10:13:12.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.004+0000 7f523ca42640 1 -- 172.21.15.133:0/2865716867 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5238068ef0 con 0x7f5238152b30 2024-04-25T10:13:12.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.004+0000 7f522effd640 1 -- 172.21.15.133:0/2865716867 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52280bc020 con 0x7f5238152b30 2024-04-25T10:13:12.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.004+0000 7f522effd640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5218066f90 0x7f5218069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.004+0000 7f522effd640 1 -- 172.21.15.133:0/2865716867 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f522811efd0 con 0x7f5238152b30 2024-04-25T10:13:12.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.004+0000 7f523659c640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5218066f90 0x7f5218069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.004+0000 7f523659c640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5218066f90 0x7f5218069450 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5230001640 tx=0x7f5230009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.008+0000 7f522effd640 1 -- 172.21.15.133:0/2865716867 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f52280c3030 con 0x7f5238152b30 2024-04-25T10:13:12.142 INFO:tasks.ceph.mds.c.smithi133.stderr:2024-04-25T10:13:12.140+0000 7efe59555640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.172+0000 7f523ca42640 1 -- 172.21.15.133:0/2865716867 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f523815ed30 con 0x7f5238152b30 2024-04-25T10:13:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.172+0000 7f522effd640 1 -- 172.21.15.133:0/2865716867 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16937 (secure 0 0 0) 0x7f5228016050 con 0x7f5238152b30 2024-04-25T10:13:12.177 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:12.177 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":131,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:13:10.855906+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:09.128791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:09.816841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 -- 172.21.15.133:0/2865716867 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5218066f90 msgr2=0x7f5218069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5218066f90 0x7f5218069450 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5230001640 tx=0x7f5230009040 comp rx=0 tx=0).stop 2024-04-25T10:13:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 -- 172.21.15.133:0/2865716867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5238152b30 msgr2=0x7f523814d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5238152b30 0x7f523814d150 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f5228019070 tx=0x7f5228019200 comp rx=0 tx=0).stop 2024-04-25T10:13:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 -- 172.21.15.133:0/2865716867 shutdown_connections 2024-04-25T10:13:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5218066f90 0x7f5218069450 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5238152b30 0x7f523814d150 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5238152750 0x7f5238148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 --2- 172.21.15.133:0/2865716867 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f52380cea40 0x7f5238148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 -- 172.21.15.133:0/2865716867 >> 172.21.15.133:0/2865716867 conn(0x7f5238067000 msgr2=0x7f523815dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 -- 172.21.15.133:0/2865716867 shutdown_connections 2024-04-25T10:13:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.176+0000 7f523ca42640 1 -- 172.21.15.133:0/2865716867 wait complete. 2024-04-25T10:13:12.197 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:13:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5003bdd640 1 Processor -- start 2024-04-25T10:13:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5003bdd640 1 -- start start 2024-04-25T10:13:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5003bdd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc0b2150 0x7f4ffc06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5003bdd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc15ac50 con 0x7f4ffc15b9b0 2024-04-25T10:13:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5003bdd640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc0cd180 con 0x7f4ffc143ea0 2024-04-25T10:13:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5003bdd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0ccf20 con 0x7f4ffc0b2150 2024-04-25T10:13:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f500217a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4ffc15b9b0 0x7f4ffc0b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38712/0 (socket says 172.21.15.133:38712) 2024-04-25T10:13:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f500217a640 1 -- 172.21.15.133:0/403867597 learned_addr learned my addr 172.21.15.133:0/403867597 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5001178640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc0b2150 0x7f4ffc06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212402806 0 0) 0x7f4ff40014a0 con 0x7f4ffc15b9b0 2024-04-25T10:13:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe8003530 con 0x7f4ffc15b9b0 2024-04-25T10:13:12.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1047021416 0 0) 0x7f4fe8003530 con 0x7f4ffc15b9b0 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc0b2150 msgr2=0x7f4ffc06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc0b2150 0x7f4ffc06e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 >> v1:172.21.15.162:6789/0 conn(0x7f4ffc143ea0 legacy=0x7f4ffc0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc15aed0 con 0x7f4ffc15b9b0 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4ff4001bd0 con 0x7f4ffc15b9b0 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 >> v1:172.21.15.133:6789/0 conn(0x7f4ffc15b9b0 legacy=0x7f4ffc0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc0b2150 0x7f4fe80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 0x7f4fe8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fe8004d50 0x7f4fe80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0ccf20 con 0x7f4fe8003f70 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe800bab0 con 0x7f4fe8004d50 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe800bcb0 con 0x7f4ffc0b2150 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5001178640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 0x7f4fe8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f500217a640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fe8004d50 0x7f4fe80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5001979640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc0b2150 0x7f4fe80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5001178640 1 -- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc0b2150 msgr2=0x7f4fe80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5001178640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc0b2150 0x7f4fe80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5001178640 1 -- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fe8004d50 msgr2=0x7f4fe80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5001178640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fe8004d50 0x7f4fe80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f5001178640 1 -- 172.21.15.133:0/403867597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe8003530 con 0x7f4fe8003f70 2024-04-25T10:13:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.320+0000 7f500217a640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4fe8004d50 0x7f4fe80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5001178640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 0x7f4fe8004810 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009ac0 tx=0x7f4fec01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fe10c02a056a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0b2070 con 0x7f4fe8003f70 2024-04-25T10:13:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec0a5640 con 0x7f4fe8003f70 2024-04-25T10:13:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5000977640 1 -- 172.21.15.133:0/403867597 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec009690 con 0x7f4fe8003f70 2024-04-25T10:13:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5003bdd640 1 -- 172.21.15.133:0/403867597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 msgr2=0x7f4fe8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5003bdd640 1 --2- 172.21.15.133:0/403867597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 0x7f4fe8004810 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009ac0 tx=0x7f4fec01ca50 comp rx=0 tx=0).stop 2024-04-25T10:13:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f500217a640 1 -- 172.21.15.133:0/403867597 reap_dead start 2024-04-25T10:13:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5003bdd640 1 -- 172.21.15.133:0/403867597 shutdown_connections 2024-04-25T10:13:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5003bdd640 1 -- 172.21.15.133:0/403867597 >> 172.21.15.133:0/403867597 conn(0x7f4ffc066f70 msgr2=0x7f4ffc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5003bdd640 1 -- 172.21.15.133:0/403867597 shutdown_connections 2024-04-25T10:13:12.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5003bdd640 1 -- 172.21.15.133:0/403867597 wait complete. 2024-04-25T10:13:12.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5003bdd640 1 Processor -- start 2024-04-25T10:13:12.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.324+0000 7f5003bdd640 1 -- start start 2024-04-25T10:13:12.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5003bdd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 0x7f4ffc1403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5003bdd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ffc151d40 0x7f4ffc141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5003bdd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc1521c0 0x7f4ffc141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5003bdd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc144280 con 0x7f4fe8003f70 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5003bdd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc15b490 con 0x7f4ffc151d40 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5003bdd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc144400 con 0x7f4ffc1521c0 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5001979640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 0x7f4ffc1403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5001979640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 0x7f4ffc1403b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37944/0 (socket says 172.21.15.133:37944) 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5001979640 1 -- 172.21.15.133:0/2446147409 learned_addr learned my addr 172.21.15.133:0/2446147409 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f500217a640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc1521c0 0x7f4ffc141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5001178640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ffc151d40 0x7f4ffc141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5001979640 1 -- 172.21.15.133:0/2446147409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc1521c0 msgr2=0x7f4ffc141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5001979640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc1521c0 0x7f4ffc141f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5001979640 1 -- 172.21.15.133:0/2446147409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ffc151d40 msgr2=0x7f4ffc141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5001979640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ffc151d40 0x7f4ffc141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5001979640 1 -- 172.21.15.133:0/2446147409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc157ed0 con 0x7f4fe8003f70 2024-04-25T10:13:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f500217a640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc1521c0 0x7f4ffc141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5001979640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 0x7f4ffc1403b0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800e580 tx=0x7f4ff800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f4ff27fc640 1 -- 172.21.15.133:0/2446147409 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80052e0 con 0x7f4fe8003f70 2024-04-25T10:13:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f4ff27fc640 1 -- 172.21.15.133:0/2446147409 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff8018070 con 0x7f4fe8003f70 2024-04-25T10:13:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5003bdd640 1 -- 172.21.15.133:0/2446147409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc0c8ae0 con 0x7f4fe8003f70 2024-04-25T10:13:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f4ff27fc640 1 -- 172.21.15.133:0/2446147409 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8003820 con 0x7f4fe8003f70 2024-04-25T10:13:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5003bdd640 1 -- 172.21.15.133:0/2446147409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc0c90a0 con 0x7f4fe8003f70 2024-04-25T10:13:12.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5003bdd640 1 -- 172.21.15.133:0/2446147409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ffc068e80 con 0x7f4fe8003f70 2024-04-25T10:13:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f4ff27fc640 1 -- 172.21.15.133:0/2446147409 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ff80039c0 con 0x7f4fe8003f70 2024-04-25T10:13:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f4ff27fc640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4fe0066f60 0x7f4fe0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f4ff27fc640 1 -- 172.21.15.133:0/2446147409 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ff8084ec0 con 0x7f4fe8003f70 2024-04-25T10:13:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.328+0000 7f5001178640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4fe0066f60 0x7f4fe0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.332+0000 7f5001178640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4fe0066f60 0x7f4fe0069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0af000 tx=0x7f4fec0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.332+0000 7f4ff27fc640 1 -- 172.21.15.133:0/2446147409 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ff8055d50 con 0x7f4fe8003f70 2024-04-25T10:13:12.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:12.532+0000 7f5003bdd640 1 -- 172.21.15.133:0/2446147409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4ffc158c50 con 0x7f4fe8003f70 2024-04-25T10:13:13.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.500+0000 7f4ff27fc640 1 -- 172.21.15.133:0/2446147409 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v78) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4ff8005480 con 0x7f4fe8003f70 2024-04-25T10:13:13.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 -- 172.21.15.133:0/2446147409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4fe0066f60 msgr2=0x7f4fe0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4fe0066f60 0x7f4fe0069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0af000 tx=0x7f4fec0046d0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 -- 172.21.15.133:0/2446147409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 msgr2=0x7f4ffc1403b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 0x7f4ffc1403b0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800e580 tx=0x7f4ff800ea40 comp rx=0 tx=0).stop 2024-04-25T10:13:13.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 -- 172.21.15.133:0/2446147409 shutdown_connections 2024-04-25T10:13:13.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ffc1521c0 0x7f4ffc141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4ffc151d40 0x7f4ffc141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4fe0066f60 0x7f4fe0069420 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 --2- 172.21.15.133:0/2446147409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fe8003f70 0x7f4ffc1403b0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 -- 172.21.15.133:0/2446147409 >> 172.21.15.133:0/2446147409 conn(0x7f4ffc066f70 msgr2=0x7f4ffc15e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 -- 172.21.15.133:0/2446147409 shutdown_connections 2024-04-25T10:13:13.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.504+0000 7f5003bdd640 1 -- 172.21.15.133:0/2446147409 wait complete. 2024-04-25T10:13:13.527 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:13:13.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:13:13.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81f0187640 1 Processor -- start 2024-04-25T10:13:13.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81f0187640 1 -- start start 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81f0187640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f81e80ce600 0x7f81e80cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81f0187640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f81e80d1920 0x7f81e80d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81f0187640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e815ebd0 con 0x7f81e815d9a0 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81f0187640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e815eeb0 con 0x7f81e80d1920 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81f0187640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e80d0330 con 0x7f81e80ce600 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81ee724640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f81e80d1920 0x7f81e80d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81ee724640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f81e80d1920 0x7f81e80d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37950/0 (socket says 172.21.15.133:37950) 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81ee724640 1 -- 172.21.15.133:0/2674134984 learned_addr learned my addr 172.21.15.133:0/2674134984 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81ed722640 1 --2- 172.21.15.133:0/2674134984 >> v2:172.21.15.162:3300/0 conn(0x7f81e80ce600 0x7f81e80cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81ee724640 1 -- 172.21.15.133:0/2674134984 >> v2:172.21.15.162:3300/0 conn(0x7f81e80ce600 msgr2=0x7f81e80cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81ee724640 1 --2- 172.21.15.133:0/2674134984 >> v2:172.21.15.162:3300/0 conn(0x7f81e80ce600 0x7f81e80cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81ee724640 1 -- 172.21.15.133:0/2674134984 >> v1:172.21.15.162:6789/0 conn(0x7f81e815d9a0 legacy=0x7f81e80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81ee724640 1 -- 172.21.15.133:0/2674134984 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e80d0610 con 0x7f81e80d1920 2024-04-25T10:13:13.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.648+0000 7f81ed722640 1 --2- 172.21.15.133:0/2674134984 >> v2:172.21.15.162:3300/0 conn(0x7f81e80ce600 0x7f81e80cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:13.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81ee724640 1 --2- 172.21.15.133:0/2674134984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e80d1920 0x7f81e80d1d70 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f81e80ce080 tx=0x7f81e001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5adbd0aebf80dba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81ecf21640 1 -- 172.21.15.133:0/2674134984 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e00a5600 con 0x7f81e80d1920 2024-04-25T10:13:13.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81ecf21640 1 -- 172.21.15.133:0/2674134984 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e00a57a0 con 0x7f81e80d1920 2024-04-25T10:13:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81ecf21640 1 -- 172.21.15.133:0/2674134984 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e00a5b70 con 0x7f81e80d1920 2024-04-25T10:13:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81f0187640 1 -- 172.21.15.133:0/2674134984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e80d1920 msgr2=0x7f81e80d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81f0187640 1 --2- 172.21.15.133:0/2674134984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e80d1920 0x7f81e80d1d70 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f81e80ce080 tx=0x7f81e001c920 comp rx=0 tx=0).stop 2024-04-25T10:13:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81f0187640 1 -- 172.21.15.133:0/2674134984 shutdown_connections 2024-04-25T10:13:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81f0187640 1 --2- 172.21.15.133:0/2674134984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e80d1920 0x7f81e80d1d70 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81f0187640 1 --2- 172.21.15.133:0/2674134984 >> v2:172.21.15.162:3300/0 conn(0x7f81e80ce600 0x7f81e80cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81f0187640 1 -- 172.21.15.133:0/2674134984 >> 172.21.15.133:0/2674134984 conn(0x7f81e806bf80 msgr2=0x7f81e806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81f0187640 1 -- 172.21.15.133:0/2674134984 shutdown_connections 2024-04-25T10:13:13.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81f0187640 1 -- 172.21.15.133:0/2674134984 wait complete. 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81f0187640 1 Processor -- start 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.652+0000 7f81f0187640 1 -- start start 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81f0187640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e80ce600 0x7f81e80c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81f0187640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81e80d1920 0x7f81e814a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81f0187640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e815d9a0 0x7f81e8162810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81f0187640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e815e960 con 0x7f81e80ce600 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81f0187640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e815e7e0 con 0x7f81e815d9a0 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81f0187640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e816a7e0 con 0x7f81e80d1920 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81edf23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e80ce600 0x7f81e80c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81edf23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e80ce600 0x7f81e80c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37964/0 (socket says 172.21.15.133:37964) 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81edf23640 1 -- 172.21.15.133:0/1288179919 learned_addr learned my addr 172.21.15.133:0/1288179919 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:13.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81ee724640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e815d9a0 0x7f81e8162810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81ed722640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81e80d1920 0x7f81e814a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81edf23640 1 -- 172.21.15.133:0/1288179919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81e80d1920 msgr2=0x7f81e814a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81edf23640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81e80d1920 0x7f81e814a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81edf23640 1 -- 172.21.15.133:0/1288179919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e815d9a0 msgr2=0x7f81e8162810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81edf23640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e815d9a0 0x7f81e8162810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81edf23640 1 -- 172.21.15.133:0/1288179919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e80caea0 con 0x7f81e80ce600 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81edf23640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e80ce600 0x7f81e80c6d70 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f81d800c480 tx=0x7f81d800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81de7fc640 1 -- 172.21.15.133:0/1288179919 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8012830 con 0x7f81e80ce600 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81de7fc640 1 -- 172.21.15.133:0/1288179919 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d8017070 con 0x7f81e80ce600 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81f0187640 1 -- 172.21.15.133:0/1288179919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e80ca900 con 0x7f81e80ce600 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81de7fc640 1 -- 172.21.15.133:0/1288179919 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8012c30 con 0x7f81e80ce600 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81f0187640 1 -- 172.21.15.133:0/1288179919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e815b900 con 0x7f81e80ce600 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81f0187640 1 -- 172.21.15.133:0/1288179919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e8068e80 con 0x7f81e80ce600 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81de7fc640 1 -- 172.21.15.133:0/1288179919 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81d801c460 con 0x7f81e80ce600 2024-04-25T10:13:13.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81de7fc640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81cc066f60 0x7f81cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81de7fc640 1 -- 172.21.15.133:0/1288179919 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81d8084f70 con 0x7f81e80ce600 2024-04-25T10:13:13.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81ed722640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81cc066f60 0x7f81cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.656+0000 7f81ed722640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81cc066f60 0x7f81cc069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f81e4006fd0 tx=0x7f81e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.660+0000 7f81de7fc640 1 -- 172.21.15.133:0/1288179919 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81d8055e80 con 0x7f81e80ce600 2024-04-25T10:13:13.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:13.856+0000 7f81f0187640 1 -- 172.21.15.133:0/1288179919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f81e80cb070 con 0x7f81e80ce600 2024-04-25T10:13:14.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.548+0000 7f81de7fc640 1 -- 172.21.15.133:0/1288179919 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v79) v1 ==== 122+0+0 (secure 0 0 0) 0x7f81d8055780 con 0x7f81e80ce600 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 -- 172.21.15.133:0/1288179919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81cc066f60 msgr2=0x7f81cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81cc066f60 0x7f81cc069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f81e4006fd0 tx=0x7f81e4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 -- 172.21.15.133:0/1288179919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e80ce600 msgr2=0x7f81e80c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e80ce600 0x7f81e80c6d70 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f81d800c480 tx=0x7f81d800c940 comp rx=0 tx=0).stop 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 -- 172.21.15.133:0/1288179919 shutdown_connections 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81e815d9a0 0x7f81e8162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f81e80d1920 0x7f81e814a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f81cc066f60 0x7f81cc069420 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 --2- 172.21.15.133:0/1288179919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81e80ce600 0x7f81e80c6d70 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 -- 172.21.15.133:0/1288179919 >> 172.21.15.133:0/1288179919 conn(0x7f81e806bf80 msgr2=0x7f81e806c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 -- 172.21.15.133:0/1288179919 shutdown_connections 2024-04-25T10:13:14.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.552+0000 7f81f0187640 1 -- 172.21.15.133:0/1288179919 wait complete. 2024-04-25T10:13:14.573 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:13:14.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:14.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f640373d640 1 Processor -- start 2024-04-25T10:13:14.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f640373d640 1 -- start start 2024-04-25T10:13:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f640373d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f63fc06eb70 0x7f63fc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f640373d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63fc0cbe60 con 0x7f63fc158110 2024-04-25T10:13:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f640373d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63fc0b19b0 con 0x7f63fc1584f0 2024-04-25T10:13:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f640373d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63fc15b360 con 0x7f63fc06eb70 2024-04-25T10:13:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f6400cd8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f63fc06eb70 0x7f63fc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f64014d9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f63fc158110 0x7f63fc06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38724/0 (socket says 172.21.15.133:38724) 2024-04-25T10:13:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f64014d9640 1 -- 172.21.15.133:0/761418360 learned_addr learned my addr 172.21.15.133:0/761418360 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f6400cd8640 1 -- 172.21.15.133:0/761418360 >> v1:172.21.15.162:6789/0 conn(0x7f63fc1584f0 legacy=0x7f63fc0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f6400cd8640 1 -- 172.21.15.133:0/761418360 >> v1:172.21.15.133:6789/0 conn(0x7f63fc158110 legacy=0x7f63fc06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f63effff640 1 -- 172.21.15.133:0/761418360 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556430956 0 0) 0x7f63fc0cbe60 con 0x7f63fc158110 2024-04-25T10:13:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.692+0000 7f6400cd8640 1 -- 172.21.15.133:0/761418360 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc15b640 con 0x7f63fc06eb70 2024-04-25T10:13:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f6400cd8640 1 --2- 172.21.15.133:0/761418360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc06eb70 0x7f63fc0b1f00 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f63f00097c0 tx=0x7f63f001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd36a2b9c713591b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f63effff640 1 -- 172.21.15.133:0/761418360 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00b0070 con 0x7f63fc06eb70 2024-04-25T10:13:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f63effff640 1 -- 172.21.15.133:0/761418360 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f00ab600 con 0x7f63fc06eb70 2024-04-25T10:13:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f63effff640 1 -- 172.21.15.133:0/761418360 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00ab9d0 con 0x7f63fc06eb70 2024-04-25T10:13:14.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 -- 172.21.15.133:0/761418360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc06eb70 msgr2=0x7f63fc0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 --2- 172.21.15.133:0/761418360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc06eb70 0x7f63fc0b1f00 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f63f00097c0 tx=0x7f63f001c430 comp rx=0 tx=0).stop 2024-04-25T10:13:14.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 -- 172.21.15.133:0/761418360 shutdown_connections 2024-04-25T10:13:14.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 --2- 172.21.15.133:0/761418360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc06eb70 0x7f63fc0b1f00 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 -- 172.21.15.133:0/761418360 >> 172.21.15.133:0/761418360 conn(0x7f63fc067000 msgr2=0x7f63fc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:14.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 -- 172.21.15.133:0/761418360 shutdown_connections 2024-04-25T10:13:14.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 -- 172.21.15.133:0/761418360 wait complete. 2024-04-25T10:13:14.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 Processor -- start 2024-04-25T10:13:14.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 -- start start 2024-04-25T10:13:14.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc06eb70 0x7f63fc156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63fc158110 0x7f63fc13c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63fc1584f0 0x7f63fc0c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc168600 con 0x7f63fc06eb70 2024-04-25T10:13:14.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc168480 con 0x7f63fc1584f0 2024-04-25T10:13:14.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f640373d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc168780 con 0x7f63fc158110 2024-04-25T10:13:14.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f64014d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc06eb70 0x7f63fc156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f64014d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc06eb70 0x7f63fc156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37980/0 (socket says 172.21.15.133:37980) 2024-04-25T10:13:14.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f64014d9640 1 -- 172.21.15.133:0/977077562 learned_addr learned my addr 172.21.15.133:0/977077562 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:14.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f64014d9640 1 -- 172.21.15.133:0/977077562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63fc158110 msgr2=0x7f63fc13c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f64014d9640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63fc158110 0x7f63fc13c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f64014d9640 1 -- 172.21.15.133:0/977077562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63fc1584f0 msgr2=0x7f63fc0c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f64014d9640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63fc1584f0 0x7f63fc0c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f64014d9640 1 -- 172.21.15.133:0/977077562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc139ca0 con 0x7f63fc06eb70 2024-04-25T10:13:14.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f64014d9640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc06eb70 0x7f63fc156dc0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f63f80011f0 tx=0x7f63f800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f63edffb640 1 -- 172.21.15.133:0/977077562 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8018070 con 0x7f63fc06eb70 2024-04-25T10:13:14.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f63edffb640 1 -- 172.21.15.133:0/977077562 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f80074e0 con 0x7f63fc06eb70 2024-04-25T10:13:14.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.696+0000 7f63edffb640 1 -- 172.21.15.133:0/977077562 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8004a30 con 0x7f63fc06eb70 2024-04-25T10:13:14.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.700+0000 7f640373d640 1 -- 172.21.15.133:0/977077562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63fc13cc30 con 0x7f63fc06eb70 2024-04-25T10:13:14.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.700+0000 7f640373d640 1 -- 172.21.15.133:0/977077562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63fc13a220 con 0x7f63fc06eb70 2024-04-25T10:13:14.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.700+0000 7f640373d640 1 -- 172.21.15.133:0/977077562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63fc06bc20 con 0x7f63fc06eb70 2024-04-25T10:13:14.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.700+0000 7f63edffb640 1 -- 172.21.15.133:0/977077562 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63f8004bd0 con 0x7f63fc06eb70 2024-04-25T10:13:14.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.700+0000 7f63edffb640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 0x7f63e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.700+0000 7f63edffb640 1 -- 172.21.15.133:0/977077562 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63f80845d0 con 0x7f63fc06eb70 2024-04-25T10:13:14.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.700+0000 7f6400cd8640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 0x7f63e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.700+0000 7f6400cd8640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 0x7f63e0069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0b4d40 tx=0x7f63f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.704+0000 7f63edffb640 1 -- 172.21.15.133:0/977077562 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63f8055450 con 0x7f63fc06eb70 2024-04-25T10:13:14.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 -- 172.21.15.133:0/977077562 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63fc0cb490 con 0x7f63fc06eb70 2024-04-25T10:13:14.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f63edffb640 1 -- 172.21.15.133:0/977077562 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f63f8054d50 con 0x7f63fc06eb70 2024-04-25T10:13:14.902 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:14.903 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:13:14.903 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:13:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 -- 172.21.15.133:0/977077562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 msgr2=0x7f63e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 0x7f63e0069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0b4d40 tx=0x7f63f0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 -- 172.21.15.133:0/977077562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc06eb70 msgr2=0x7f63fc156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc06eb70 0x7f63fc156dc0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f63f80011f0 tx=0x7f63f800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 -- 172.21.15.133:0/977077562 shutdown_connections 2024-04-25T10:13:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63fc1584f0 0x7f63fc0c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63fc158110 0x7f63fc13c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 0x7f63e0069420 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 --2- 172.21.15.133:0/977077562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc06eb70 0x7f63fc156dc0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 -- 172.21.15.133:0/977077562 >> 172.21.15.133:0/977077562 conn(0x7f63fc067000 msgr2=0x7f63fc0ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:14.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 -- 172.21.15.133:0/977077562 shutdown_connections 2024-04-25T10:13:14.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:14.900+0000 7f640373d640 1 -- 172.21.15.133:0/977077562 wait complete. 2024-04-25T10:13:14.921 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:15.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.028+0000 7fd726bf5640 1 Processor -- start 2024-04-25T10:13:15.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.028+0000 7fd726bf5640 1 -- start start 2024-04-25T10:13:15.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.028+0000 7fd726bf5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd720159dc0 0x7fd7200cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.028+0000 7fd726bf5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7200ce880 con 0x7fd72015e220 2024-04-25T10:13:15.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.028+0000 7fd726bf5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7200c49e0 con 0x7fd72015a1a0 2024-04-25T10:13:15.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.028+0000 7fd726bf5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7200ceb20 con 0x7fd720159dc0 2024-04-25T10:13:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.028+0000 7fd725192640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd72015e220 0x7fd72006d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38730/0 (socket says 172.21.15.133:38730) 2024-04-25T10:13:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.028+0000 7fd725192640 1 -- 172.21.15.133:0/3720602730 learned_addr learned my addr 172.21.15.133:0/3720602730 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.028+0000 7fd724991640 1 --2- 172.21.15.133:0/3720602730 >> v2:172.21.15.162:3300/0 conn(0x7fd720159dc0 0x7fd7200cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.028+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138079145 0 0) 0x7fd7200ce880 con 0x7fd72015e220 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.028+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd708003530 con 0x7fd72015e220 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 16115611 0 0) 0x7fd708003530 con 0x7fd72015e220 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 >> v2:172.21.15.162:3300/0 conn(0x7fd720159dc0 msgr2=0x7fd7200cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 --2- 172.21.15.133:0/3720602730 >> v2:172.21.15.162:3300/0 conn(0x7fd720159dc0 0x7fd7200cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 >> v1:172.21.15.162:6789/0 conn(0x7fd72015a1a0 legacy=0x7fd72015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7200cbfa0 con 0x7fd72015e220 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fd718001a00 con 0x7fd72015e220 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 >> v1:172.21.15.133:6789/0 conn(0x7fd72015e220 legacy=0x7fd72006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 --2- 172.21.15.133:0/3720602730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7080039c0 0x7fd708003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 --2- 172.21.15.133:0/3720602730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd708004810 0x7fd708004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 --2- 172.21.15.133:0/3720602730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7080051d0 0x7fd70800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200ceb20 con 0x7fd708004810 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70800c070 con 0x7fd7080039c0 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70800c270 con 0x7fd7080051d0 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71ffff640 1 --2- 172.21.15.133:0/3720602730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd708004810 0x7fd708004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd724991640 1 --2- 172.21.15.133:0/3720602730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7080039c0 0x7fd708003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd725192640 1 --2- 172.21.15.133:0/3720602730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7080051d0 0x7fd70800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71ffff640 1 -- 172.21.15.133:0/3720602730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7080051d0 msgr2=0x7fd70800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71ffff640 1 --2- 172.21.15.133:0/3720602730 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7080051d0 0x7fd70800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71ffff640 1 -- 172.21.15.133:0/3720602730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7080039c0 msgr2=0x7fd708003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71ffff640 1 --2- 172.21.15.133:0/3720602730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7080039c0 0x7fd708003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd725192640 1 -- 172.21.15.133:0/3720602730 reap_dead start 2024-04-25T10:13:15.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71ffff640 1 -- 172.21.15.133:0/3720602730 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd708003530 con 0x7fd708004810 2024-04-25T10:13:15.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71ffff640 1 --2- 172.21.15.133:0/3720602730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd708004810 0x7fd708004c90 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fd714001780 tx=0x7fd71401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cddf069ea68522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7140013a0 con 0x7fd708004810 2024-04-25T10:13:15.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7140052f0 con 0x7fd708004810 2024-04-25T10:13:15.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.032+0000 7fd71f7fe640 1 -- 172.21.15.133:0/3720602730 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714003840 con 0x7fd708004810 2024-04-25T10:13:15.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 -- 172.21.15.133:0/3720602730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd708004810 msgr2=0x7fd708004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 --2- 172.21.15.133:0/3720602730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd708004810 0x7fd708004c90 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fd714001780 tx=0x7fd71401c400 comp rx=0 tx=0).stop 2024-04-25T10:13:15.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 -- 172.21.15.133:0/3720602730 shutdown_connections 2024-04-25T10:13:15.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 --2- 172.21.15.133:0/3720602730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd708004810 0x7fd708004c90 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 -- 172.21.15.133:0/3720602730 >> 172.21.15.133:0/3720602730 conn(0x7fd720067000 msgr2=0x7fd72006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 -- 172.21.15.133:0/3720602730 shutdown_connections 2024-04-25T10:13:15.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 -- 172.21.15.133:0/3720602730 wait complete. 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 Processor -- start 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 -- start start 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd708004810 0x7fd720132bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd720145640 0x7fd72014f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72014fbe0 0x7fd7201520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200c47c0 con 0x7fd72014fbe0 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200c4640 con 0x7fd720145640 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7200c4cc0 con 0x7fd708004810 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd725192640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72014fbe0 0x7fd7201520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd725192640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72014fbe0 0x7fd7201520d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37998/0 (socket says 172.21.15.133:37998) 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd725192640 1 -- 172.21.15.133:0/2731884735 learned_addr learned my addr 172.21.15.133:0/2731884735 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:15.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd724991640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd708004810 0x7fd720132bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd71ffff640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd720145640 0x7fd72014f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd725192640 1 -- 172.21.15.133:0/2731884735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd708004810 msgr2=0x7fd720132bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd725192640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd708004810 0x7fd720132bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd725192640 1 -- 172.21.15.133:0/2731884735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd720145640 msgr2=0x7fd72014f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd725192640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd720145640 0x7fd72014f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd725192640 1 -- 172.21.15.133:0/2731884735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7200b1760 con 0x7fd72014fbe0 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd724991640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd708004810 0x7fd720132bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd71ffff640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd720145640 0x7fd72014f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd725192640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72014fbe0 0x7fd7201520d0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fd7200b4990 tx=0x7fd718011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd71d7fa640 1 -- 172.21.15.133:0/2731884735 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718010040 con 0x7fd72014fbe0 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd71d7fa640 1 -- 172.21.15.133:0/2731884735 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71801a070 con 0x7fd72014fbe0 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd71d7fa640 1 -- 172.21.15.133:0/2731884735 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71800fb40 con 0x7fd72014fbe0 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 -- 172.21.15.133:0/2731884735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd720150080 con 0x7fd72014fbe0 2024-04-25T10:13:15.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.036+0000 7fd726bf5640 1 -- 172.21.15.133:0/2731884735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd720150640 con 0x7fd72014fbe0 2024-04-25T10:13:15.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.040+0000 7fd726bf5640 1 -- 172.21.15.133:0/2731884735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd720068ef0 con 0x7fd72014fbe0 2024-04-25T10:13:15.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.040+0000 7fd71d7fa640 1 -- 172.21.15.133:0/2731884735 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd71800fce0 con 0x7fd72014fbe0 2024-04-25T10:13:15.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.040+0000 7fd71d7fa640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd70c066f60 0x7fd70c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.040+0000 7fd724991640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd70c066f60 0x7fd70c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.040+0000 7fd71d7fa640 1 -- 172.21.15.133:0/2731884735 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd718088050 con 0x7fd72014fbe0 2024-04-25T10:13:15.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.040+0000 7fd724991640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd70c066f60 0x7fd70c069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd72004f450 tx=0x7fd710008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.044+0000 7fd71d7fa640 1 -- 172.21.15.133:0/2731884735 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd718058ed0 con 0x7fd72014fbe0 2024-04-25T10:13:15.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.192+0000 7fd726bf5640 1 -- 172.21.15.133:0/2731884735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7200cd530 con 0x7fd72014fbe0 2024-04-25T10:13:15.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd71d7fa640 1 -- 172.21.15.133:0/2731884735 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16924 (secure 0 0 0) 0x7fd7180587d0 con 0x7fd72014fbe0 2024-04-25T10:13:15.199 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:15.199 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":132,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:13:12.499841+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:09.128791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:09.816841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:15.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 -- 172.21.15.133:0/2731884735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd70c066f60 msgr2=0x7fd70c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd70c066f60 0x7fd70c069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd72004f450 tx=0x7fd710008040 comp rx=0 tx=0).stop 2024-04-25T10:13:15.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 -- 172.21.15.133:0/2731884735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72014fbe0 msgr2=0x7fd7201520d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72014fbe0 0x7fd7201520d0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fd7200b4990 tx=0x7fd718011c20 comp rx=0 tx=0).stop 2024-04-25T10:13:15.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 -- 172.21.15.133:0/2731884735 shutdown_connections 2024-04-25T10:13:15.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd70c066f60 0x7fd70c069420 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72014fbe0 0x7fd7201520d0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd720145640 0x7fd72014f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 --2- 172.21.15.133:0/2731884735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd708004810 0x7fd720132bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 -- 172.21.15.133:0/2731884735 >> 172.21.15.133:0/2731884735 conn(0x7fd720067000 msgr2=0x7fd72006d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 -- 172.21.15.133:0/2731884735 shutdown_connections 2024-04-25T10:13:15.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.196+0000 7fd726bf5640 1 -- 172.21.15.133:0/2731884735 wait complete. 2024-04-25T10:13:15.216 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:13:15.217 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:15.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.320+0000 7f4e97b5f640 1 Processor -- start 2024-04-25T10:13:15.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.320+0000 7f4e97b5f640 1 -- start start 2024-04-25T10:13:15.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e97b5f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9015c9a0 0x7f4e900c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e97b5f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4e900ce330 0x7f4e900ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e97b5f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e900c61d0 con 0x7f4e900b1f70 2024-04-25T10:13:15.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e97b5f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e900cece0 con 0x7f4e900ce330 2024-04-25T10:13:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e97b5f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e900c64b0 con 0x7f4e9015c9a0 2024-04-25T10:13:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e958fb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9015c9a0 0x7f4e900c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e958fb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9015c9a0 0x7f4e900c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:60082/0 (socket says 172.21.15.133:60082) 2024-04-25T10:13:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e960fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4e900ce330 0x7f4e900ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e958fb640 1 -- 172.21.15.133:0/3223244103 learned_addr learned my addr 172.21.15.133:0/3223244103 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e958fb640 1 -- 172.21.15.133:0/3223244103 >> v2:172.21.15.162:3300/0 conn(0x7f4e900ce330 msgr2=0x7f4e900ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e958fb640 1 --2- 172.21.15.133:0/3223244103 >> v2:172.21.15.162:3300/0 conn(0x7f4e900ce330 0x7f4e900ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e958fb640 1 -- 172.21.15.133:0/3223244103 >> v1:172.21.15.162:6789/0 conn(0x7f4e900b1f70 legacy=0x7f4e9006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e958fb640 1 -- 172.21.15.133:0/3223244103 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9006d4f0 con 0x7f4e9015c9a0 2024-04-25T10:13:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.324+0000 7f4e958fb640 1 --2- 172.21.15.133:0/3223244103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9015c9a0 0x7f4e900c5980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009870 tx=0x7f4e8801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcd6772114f80733 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e948f9640 1 -- 172.21.15.133:0/3223244103 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e880b2070 con 0x7f4e9015c9a0 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e948f9640 1 -- 172.21.15.133:0/3223244103 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e880a9070 con 0x7f4e9015c9a0 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e948f9640 1 -- 172.21.15.133:0/3223244103 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e880ad880 con 0x7f4e9015c9a0 2024-04-25T10:13:15.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3223244103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9015c9a0 msgr2=0x7f4e900c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 --2- 172.21.15.133:0/3223244103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9015c9a0 0x7f4e900c5980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009870 tx=0x7f4e8801c560 comp rx=0 tx=0).stop 2024-04-25T10:13:15.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3223244103 shutdown_connections 2024-04-25T10:13:15.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 --2- 172.21.15.133:0/3223244103 >> v2:172.21.15.162:3300/0 conn(0x7f4e900ce330 0x7f4e900ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 --2- 172.21.15.133:0/3223244103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e9015c9a0 0x7f4e900c5980 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3223244103 >> 172.21.15.133:0/3223244103 conn(0x7f4e90067000 msgr2=0x7f4e9006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3223244103 shutdown_connections 2024-04-25T10:13:15.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3223244103 wait complete. 2024-04-25T10:13:15.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 Processor -- start 2024-04-25T10:13:15.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 -- start start 2024-04-25T10:13:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e900b1f70 0x7f4e900cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e900ce330 0x7f4e90152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e9015c9a0 0x7f4e900ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9015d950 con 0x7f4e9015c9a0 2024-04-25T10:13:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9015dad0 con 0x7f4e900ce330 2024-04-25T10:13:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9015dc50 con 0x7f4e900b1f70 2024-04-25T10:13:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e960fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e9015c9a0 0x7f4e900ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e960fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e9015c9a0 0x7f4e900ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38006/0 (socket says 172.21.15.133:38006) 2024-04-25T10:13:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e960fc640 1 -- 172.21.15.133:0/3112445172 learned_addr learned my addr 172.21.15.133:0/3112445172 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e958fb640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e900b1f70 0x7f4e900cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e950fa640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e900ce330 0x7f4e90152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e960fc640 1 -- 172.21.15.133:0/3112445172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e900b1f70 msgr2=0x7f4e900cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e960fc640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e900b1f70 0x7f4e900cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e960fc640 1 -- 172.21.15.133:0/3112445172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e900ce330 msgr2=0x7f4e90152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e960fc640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e900ce330 0x7f4e90152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e960fc640 1 -- 172.21.15.133:0/3112445172 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9014f660 con 0x7f4e9015c9a0 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e960fc640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e9015c9a0 0x7f4e900ca600 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000efc0 tx=0x7f4e8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e8e7fc640 1 -- 172.21.15.133:0/3112445172 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80018020 con 0x7f4e9015c9a0 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e8e7fc640 1 -- 172.21.15.133:0/3112445172 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e80014480 con 0x7f4e9015c9a0 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3112445172 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e90153a70 con 0x7f4e9015c9a0 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e8e7fc640 1 -- 172.21.15.133:0/3112445172 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80014870 con 0x7f4e9015c9a0 2024-04-25T10:13:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.328+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3112445172 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9014f8d0 con 0x7f4e9015c9a0 2024-04-25T10:13:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.332+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3112445172 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9006bc20 con 0x7f4e9015c9a0 2024-04-25T10:13:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.332+0000 7f4e8e7fc640 1 -- 172.21.15.133:0/3112445172 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e8001b020 con 0x7f4e9015c9a0 2024-04-25T10:13:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.332+0000 7f4e8e7fc640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e7c066f60 0x7f4e7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.332+0000 7f4e8e7fc640 1 -- 172.21.15.133:0/3112445172 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4e80085b90 con 0x7f4e9015c9a0 2024-04-25T10:13:15.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.332+0000 7f4e958fb640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e7c066f60 0x7f4e7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.332+0000 7f4e958fb640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e7c066f60 0x7f4e7c069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4e880af000 tx=0x7f4e880a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.336+0000 7f4e8e7fc640 1 -- 172.21.15.133:0/3112445172 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e80022070 con 0x7f4e9015c9a0 2024-04-25T10:13:15.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.488+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3112445172 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e90068ef0 con 0x7f4e7c066f60 2024-04-25T10:13:15.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.504+0000 7f4e8e7fc640 1 -- 172.21.15.133:0/3112445172 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240065 (secure 0 0 0) 0x7f4e90068ef0 con 0x7f4e7c066f60 2024-04-25T10:13:15.511 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:15.513 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:13:15.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3112445172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e7c066f60 msgr2=0x7f4e7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e7c066f60 0x7f4e7c069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4e880af000 tx=0x7f4e880a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:15.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3112445172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e9015c9a0 msgr2=0x7f4e900ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e9015c9a0 0x7f4e900ca600 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000efc0 tx=0x7f4e8000c490 comp rx=0 tx=0).stop 2024-04-25T10:13:15.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3112445172 shutdown_connections 2024-04-25T10:13:15.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e7c066f60 0x7f4e7c069420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e9015c9a0 0x7f4e900ca600 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e900ce330 0x7f4e90152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 --2- 172.21.15.133:0/3112445172 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e900b1f70 0x7f4e900cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3112445172 >> 172.21.15.133:0/3112445172 conn(0x7f4e90067000 msgr2=0x7f4e900b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3112445172 shutdown_connections 2024-04-25T10:13:15.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.508+0000 7f4e97b5f640 1 -- 172.21.15.133:0/3112445172 wait complete. 2024-04-25T10:13:15.527 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-04-25T10:13:13.925423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":496396,"kb_used_data":13720,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754478324,"statfs":{"total":773094113280,"available":772585803776,"internally_reserved":0,"allocated":14049280,"data_stored":7164372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.453869"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476901+0000","last_change":"2024-04-25T10:13:09.476901+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476901+0000","last_undegraded":"2024-04-25T10:13:09.476901+0000","last_fullsized":"2024-04-25T10:13:09.476901+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.477508+0000","last_change":"2024-04-25T10:13:09.477508+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.477508+0000","last_undegraded":"2024-04-25T10:13:09.477508+0000","last_fullsized":"2024-04-25T10:13:09.477508+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.485195+0000","last_change":"2024-04-25T10:13:09.485195+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.485195+0000","last_undegraded":"2024-04-25T10:13:09.485195+0000","last_fullsized":"2024-04-25T10:13:09.485195+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.483556+0000","last_change":"2024-04-25T10:13:09.483556+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.483556+0000","last_undegraded":"2024-04-25T10:13:09.483556+0000","last_fullsized":"2024-04-25T10:13:09.483556+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476739+0000","last_change":"2024-04-25T10:13:09.476739+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476739+0000","last_undegraded":"2024-04-25T10:13:09.476739+0000","last_fullsized":"2024-04-25T10:13:09.476739+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.482549+0000","last_change":"2024-04-25T10:13:09.482549+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.482549+0000","last_undegraded":"2024-04-25T10:13:09.482549+0000","last_fullsized":"2024-04-25T10:13:09.482549+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.477740+0000","last_change":"2024-04-25T10:13:09.477740+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.477740+0000","last_undegraded":"2024-04-25T10:13:09.477740+0000","last_fullsized":"2024-04-25T10:13:09.477740+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.475372+0000","last_change":"2024-04-25T10:13:09.475372+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.475372+0000","last_undegraded":"2024-04-25T10:13:09.475372+0000","last_fullsized":"2024-04-25T10:13:09.475372+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.483182+0000","last_change":"2024-04-25T10:13:09.483182+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.483182+0000","last_undegraded":"2024-04-25T10:13:09.483182+0000","last_fullsized":"2024-04-25T10:13:09.483182+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.479125+0000","last_change":"2024-04-25T10:13:09.479125+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.479125+0000","last_undegraded":"2024-04-25T10:13:09.479125+0000","last_fullsized":"2024-04-25T10:13:09.479125+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.481327+0000","last_change":"2024-04-25T10:13:09.481327+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.481327+0000","last_undegraded":"2024-04-25T10:13:09.481327+0000","last_fullsized":"2024-04-25T10:13:09.481327+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.481327+0000","last_change":"2024-04-25T10:13:09.481327+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.481327+0000","last_undegraded":"2024-04-25T10:13:09.481327+0000","last_fullsized":"2024-04-25T10:13:09.481327+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.479767+0000","last_change":"2024-04-25T10:13:09.479767+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.479767+0000","last_undegraded":"2024-04-25T10:13:09.479767+0000","last_fullsized":"2024-04-25T10:13:09.479767+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.477942+0000","last_change":"2024-04-25T10:13:09.477942+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.477942+0000","last_undegraded":"2024-04-25T10:13:09.477942+0000","last_fullsized":"2024-04-25T10:13:09.477942+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476971+0000","last_change":"2024-04-25T10:13:09.476971+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476971+0000","last_undegraded":"2024-04-25T10:13:09.476971+0000","last_fullsized":"2024-04-25T10:13:09.476971+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.482136+0000","last_change":"2024-04-25T10:13:09.482136+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.482136+0000","last_undegraded":"2024-04-25T10:13:09.482136+0000","last_fullsized":"2024-04-25T10:13:09.482136+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.475496+0000","last_change":"2024-04-25T10:13:09.475496+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.475496+0000","last_undegraded":"2024-04-25T10:13:09.475496+0000","last_fullsized":"2024-04-25T10:13:09.475496+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476512+0000","last_change":"2024-04-25T10:13:09.476512+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476512+0000","last_undegraded":"2024-04-25T10:13:09.476512+0000","last_fullsized":"2024-04-25T10:13:09.476512+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.482639+0000","last_change":"2024-04-25T10:13:09.482639+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.482639+0000","last_undegraded":"2024-04-25T10:13:09.482639+0000","last_fullsized":"2024-04-25T10:13:09.482639+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.475496+0000","last_change":"2024-04-25T10:13:09.475496+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.475496+0000","last_undegraded":"2024-04-25T10:13:09.475496+0000","last_fullsized":"2024-04-25T10:13:09.475496+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.478457+0000","last_change":"2024-04-25T10:13:09.478457+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.478457+0000","last_undegraded":"2024-04-25T10:13:09.478457+0000","last_fullsized":"2024-04-25T10:13:09.478457+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.481223+0000","last_change":"2024-04-25T10:13:09.481223+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.481223+0000","last_undegraded":"2024-04-25T10:13:09.481223+0000","last_fullsized":"2024-04-25T10:13:09.481223+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476368+0000","last_change":"2024-04-25T10:13:09.476368+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476368+0000","last_undegraded":"2024-04-25T10:13:09.476368+0000","last_fullsized":"2024-04-25T10:13:09.476368+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.478770+0000","last_change":"2024-04-25T10:13:09.478770+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.478770+0000","last_undegraded":"2024-04-25T10:13:09.478770+0000","last_fullsized":"2024-04-25T10:13:09.478770+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.475498+0000","last_change":"2024-04-25T10:13:09.475498+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.475498+0000","last_undegraded":"2024-04-25T10:13:09.475498+0000","last_fullsized":"2024-04-25T10:13:09.475498+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476791+0000","last_change":"2024-04-25T10:13:09.476791+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476791+0000","last_undegraded":"2024-04-25T10:13:09.476791+0000","last_fullsized":"2024-04-25T10:13:09.476791+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.477778+0000","last_change":"2024-04-25T10:13:09.477778+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.477778+0000","last_undegraded":"2024-04-25T10:13:09.477778+0000","last_fullsized":"2024-04-25T10:13:09.477778+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.481745+0000","last_change":"2024-04-25T10:13:09.481745+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.481745+0000","last_undegraded":"2024-04-25T10:13:09.481745+0000","last_fullsized":"2024-04-25T10:13:09.481745+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.481283+0000","last_change":"2024-04-25T10:13:09.481283+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.481283+0000","last_undegraded":"2024-04-25T10:13:09.481283+0000","last_fullsized":"2024-04-25T10:13:09.481283+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.477301+0000","last_change":"2024-04-25T10:13:09.477301+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.477301+0000","last_undegraded":"2024-04-25T10:13:09.477301+0000","last_fullsized":"2024-04-25T10:13:09.477301+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.482105+0000","last_change":"2024-04-25T10:13:09.482105+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.482105+0000","last_undegraded":"2024-04-25T10:13:09.482105+0000","last_fullsized":"2024-04-25T10:13:09.482105+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.486045+0000","last_change":"2024-04-25T10:13:09.486045+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.486045+0000","last_undegraded":"2024-04-25T10:13:09.486045+0000","last_fullsized":"2024-04-25T10:13:09.486045+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":374,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:13:09.486683+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:13:09.486683+0000","last_peered":"2024-04-25T10:13:09.486683+0000","last_clean":"2024-04-25T10:13:09.486683+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:13:09.486683+0000","last_undegraded":"2024-04-25T10:13:09.486683+0000","last_fullsized":"2024-04-25T10:13:09.486683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69440,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302400,"statfs":{"total":96636764160,"available":96565657600,"internally_reserved":0,"allocated":2207744,"data_stored":1341427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46644,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325196,"statfs":{"total":96636764160,"available":96589000704,"internally_reserved":0,"allocated":1613824,"data_stored":751059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59112,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312728,"statfs":{"total":96636764160,"available":96576233472,"internally_reserved":0,"allocated":1601536,"data_stored":744850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574990,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73012,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298828,"statfs":{"total":96636764160,"available":96561999872,"internally_reserved":0,"allocated":1613824,"data_stored":751059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73004,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1601536,"data_stored":744850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46632,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325208,"statfs":{"total":96636764160,"available":96589012992,"internally_reserved":0,"allocated":1601536,"data_stored":744850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59124,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312716,"statfs":{"total":96636764160,"available":96576221184,"internally_reserved":0,"allocated":1613824,"data_stored":751059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69428,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302412,"statfs":{"total":96636764160,"available":96565669888,"internally_reserved":0,"allocated":2195456,"data_stored":1335218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:15.532 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:15.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d533ee640 1 Processor -- start 2024-04-25T10:13:15.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d533ee640 1 -- start start 2024-04-25T10:13:15.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d533ee640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d4c15db00 0x7f4d4c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d533ee640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4d4c0d0bd0 0x7f4d4c161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d533ee640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4c0ceb20 con 0x7f4d4c0d0770 2024-04-25T10:13:15.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d533ee640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4c0b7740 con 0x7f4d4c15db00 2024-04-25T10:13:15.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d533ee640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4c0b7a20 con 0x7f4d4c0d0bd0 2024-04-25T10:13:15.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d5118a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d4c15db00 0x7f4d4c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d5118a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d4c15db00 0x7f4d4c0b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38016/0 (socket says 172.21.15.133:38016) 2024-04-25T10:13:15.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d50989640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4d4c0d0770 0x7f4d4c1614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38736/0 (socket says 172.21.15.133:38736) 2024-04-25T10:13:15.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d5118a640 1 -- 172.21.15.133:0/2669204054 learned_addr learned my addr 172.21.15.133:0/2669204054 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:15.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d5198b640 1 --2- 172.21.15.133:0/2669204054 >> v2:172.21.15.162:3300/0 conn(0x7f4d4c0d0bd0 0x7f4d4c161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d5118a640 1 -- 172.21.15.133:0/2669204054 >> v2:172.21.15.162:3300/0 conn(0x7f4d4c0d0bd0 msgr2=0x7f4d4c161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d5118a640 1 --2- 172.21.15.133:0/2669204054 >> v2:172.21.15.162:3300/0 conn(0x7f4d4c0d0bd0 0x7f4d4c161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d5118a640 1 -- 172.21.15.133:0/2669204054 >> v1:172.21.15.133:6789/0 conn(0x7f4d4c0d0770 legacy=0x7f4d4c1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d3ffff640 1 -- 172.21.15.133:0/2669204054 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346839939 0 0) 0x7f4d4c0ceb20 con 0x7f4d4c0d0770 2024-04-25T10:13:15.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d5118a640 1 -- 172.21.15.133:0/2669204054 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c0b97f0 con 0x7f4d4c15db00 2024-04-25T10:13:15.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.684+0000 7f4d5118a640 1 --2- 172.21.15.133:0/2669204054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d4c15db00 0x7f4d4c0b2990 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0b5a00 tx=0x7f4d4801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2988e10c0cbadf5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d3ffff640 1 -- 172.21.15.133:0/2669204054 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d480a5880 con 0x7f4d4c15db00 2024-04-25T10:13:15.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d3ffff640 1 -- 172.21.15.133:0/2669204054 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d480a5a20 con 0x7f4d4c15db00 2024-04-25T10:13:15.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d3ffff640 1 -- 172.21.15.133:0/2669204054 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d480a5df0 con 0x7f4d4c15db00 2024-04-25T10:13:15.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 -- 172.21.15.133:0/2669204054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d4c15db00 msgr2=0x7f4d4c0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 --2- 172.21.15.133:0/2669204054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d4c15db00 0x7f4d4c0b2990 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0b5a00 tx=0x7f4d4801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 -- 172.21.15.133:0/2669204054 shutdown_connections 2024-04-25T10:13:15.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 --2- 172.21.15.133:0/2669204054 >> v2:172.21.15.162:3300/0 conn(0x7f4d4c0d0bd0 0x7f4d4c161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 --2- 172.21.15.133:0/2669204054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d4c15db00 0x7f4d4c0b2990 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 -- 172.21.15.133:0/2669204054 >> 172.21.15.133:0/2669204054 conn(0x7f4d4c0727c0 msgr2=0x7f4d4c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 -- 172.21.15.133:0/2669204054 shutdown_connections 2024-04-25T10:13:15.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 -- 172.21.15.133:0/2669204054 wait complete. 2024-04-25T10:13:15.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 Processor -- start 2024-04-25T10:13:15.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 -- start start 2024-04-25T10:13:15.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d4c0d0770 0x7f4d4c14ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4c0d0bd0 0x7f4d4c14b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d4c15db00 0x7f4d4c14b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c0d0fe0 con 0x7f4d4c15db00 2024-04-25T10:13:15.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c0b7430 con 0x7f4d4c0d0770 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c0d18a0 con 0x7f4d4c0d0bd0 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d5198b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d4c15db00 0x7f4d4c14b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d5198b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d4c15db00 0x7f4d4c14b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38024/0 (socket says 172.21.15.133:38024) 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d5198b640 1 -- 172.21.15.133:0/4113076129 learned_addr learned my addr 172.21.15.133:0/4113076129 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d5118a640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d4c0d0770 0x7f4d4c14ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d50989640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4c0d0bd0 0x7f4d4c14b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d5198b640 1 -- 172.21.15.133:0/4113076129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4c0d0bd0 msgr2=0x7f4d4c14b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d5198b640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4c0d0bd0 0x7f4d4c14b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d5198b640 1 -- 172.21.15.133:0/4113076129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d4c0d0770 msgr2=0x7f4d4c14ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d5198b640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d4c0d0770 0x7f4d4c14ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d5198b640 1 -- 172.21.15.133:0/4113076129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c14f080 con 0x7f4d4c15db00 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d50989640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4c0d0bd0 0x7f4d4c14b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:15.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d5198b640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d4c15db00 0x7f4d4c14b7e0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400d560 tx=0x7f4d4400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d3dffb640 1 -- 172.21.15.133:0/4113076129 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440125f0 con 0x7f4d4c15db00 2024-04-25T10:13:15.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d3dffb640 1 -- 172.21.15.133:0/4113076129 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d44019070 con 0x7f4d4c15db00 2024-04-25T10:13:15.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 -- 172.21.15.133:0/4113076129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4c14b060 con 0x7f4d4c15db00 2024-04-25T10:13:15.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d3dffb640 1 -- 172.21.15.133:0/4113076129 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440129f0 con 0x7f4d4c15db00 2024-04-25T10:13:15.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.688+0000 7f4d533ee640 1 -- 172.21.15.133:0/4113076129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4c14f300 con 0x7f4d4c15db00 2024-04-25T10:13:15.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.692+0000 7f4d533ee640 1 -- 172.21.15.133:0/4113076129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4c068ef0 con 0x7f4d4c15db00 2024-04-25T10:13:15.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.692+0000 7f4d3dffb640 1 -- 172.21.15.133:0/4113076129 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d44012b90 con 0x7f4d4c15db00 2024-04-25T10:13:15.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.692+0000 7f4d3dffb640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d30066f60 0x7f4d30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.692+0000 7f4d3dffb640 1 -- 172.21.15.133:0/4113076129 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d440854b0 con 0x7f4d4c15db00 2024-04-25T10:13:15.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.692+0000 7f4d5118a640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d30066f60 0x7f4d30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.692+0000 7f4d5118a640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d30066f60 0x7f4d30069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4d480ae000 tx=0x7f4d480a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.696+0000 7f4d3dffb640 1 -- 172.21.15.133:0/4113076129 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d4c14f080 con 0x7f4d4c15db00 2024-04-25T10:13:15.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.840+0000 7f4d533ee640 1 -- 172.21.15.133:0/4113076129 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d4c0b27c0 con 0x7f4d30066f60 2024-04-25T10:13:15.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.852+0000 7f4d3dffb640 1 -- 172.21.15.133:0/4113076129 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240065 (secure 0 0 0) 0x7f4d4c0b27c0 con 0x7f4d30066f60 2024-04-25T10:13:15.859 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:15.861 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:13:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 -- 172.21.15.133:0/4113076129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d30066f60 msgr2=0x7f4d30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d30066f60 0x7f4d30069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4d480ae000 tx=0x7f4d480a7070 comp rx=0 tx=0).stop 2024-04-25T10:13:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 -- 172.21.15.133:0/4113076129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d4c15db00 msgr2=0x7f4d4c14b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d4c15db00 0x7f4d4c14b7e0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400d560 tx=0x7f4d4400da20 comp rx=0 tx=0).stop 2024-04-25T10:13:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 -- 172.21.15.133:0/4113076129 shutdown_connections 2024-04-25T10:13:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d30066f60 0x7f4d30069420 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d4c15db00 0x7f4d4c14b7e0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4c0d0bd0 0x7f4d4c14b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 --2- 172.21.15.133:0/4113076129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d4c0d0770 0x7f4d4c14ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 -- 172.21.15.133:0/4113076129 >> 172.21.15.133:0/4113076129 conn(0x7f4d4c0727c0 msgr2=0x7f4d4c0b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 -- 172.21.15.133:0/4113076129 shutdown_connections 2024-04-25T10:13:15.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:15.856+0000 7f4d533ee640 1 -- 172.21.15.133:0/4113076129 wait complete. 2024-04-25T10:13:15.877 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-04-25T10:13:13.925423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":496396,"kb_used_data":13720,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754478324,"statfs":{"total":773094113280,"available":772585803776,"internally_reserved":0,"allocated":14049280,"data_stored":7164372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.453869"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476901+0000","last_change":"2024-04-25T10:13:09.476901+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476901+0000","last_undegraded":"2024-04-25T10:13:09.476901+0000","last_fullsized":"2024-04-25T10:13:09.476901+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.477508+0000","last_change":"2024-04-25T10:13:09.477508+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.477508+0000","last_undegraded":"2024-04-25T10:13:09.477508+0000","last_fullsized":"2024-04-25T10:13:09.477508+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.485195+0000","last_change":"2024-04-25T10:13:09.485195+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.485195+0000","last_undegraded":"2024-04-25T10:13:09.485195+0000","last_fullsized":"2024-04-25T10:13:09.485195+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.483556+0000","last_change":"2024-04-25T10:13:09.483556+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.483556+0000","last_undegraded":"2024-04-25T10:13:09.483556+0000","last_fullsized":"2024-04-25T10:13:09.483556+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476739+0000","last_change":"2024-04-25T10:13:09.476739+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476739+0000","last_undegraded":"2024-04-25T10:13:09.476739+0000","last_fullsized":"2024-04-25T10:13:09.476739+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.482549+0000","last_change":"2024-04-25T10:13:09.482549+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.482549+0000","last_undegraded":"2024-04-25T10:13:09.482549+0000","last_fullsized":"2024-04-25T10:13:09.482549+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.477740+0000","last_change":"2024-04-25T10:13:09.477740+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.477740+0000","last_undegraded":"2024-04-25T10:13:09.477740+0000","last_fullsized":"2024-04-25T10:13:09.477740+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.475372+0000","last_change":"2024-04-25T10:13:09.475372+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.475372+0000","last_undegraded":"2024-04-25T10:13:09.475372+0000","last_fullsized":"2024-04-25T10:13:09.475372+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.483182+0000","last_change":"2024-04-25T10:13:09.483182+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.483182+0000","last_undegraded":"2024-04-25T10:13:09.483182+0000","last_fullsized":"2024-04-25T10:13:09.483182+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.479125+0000","last_change":"2024-04-25T10:13:09.479125+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.479125+0000","last_undegraded":"2024-04-25T10:13:09.479125+0000","last_fullsized":"2024-04-25T10:13:09.479125+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.481327+0000","last_change":"2024-04-25T10:13:09.481327+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.481327+0000","last_undegraded":"2024-04-25T10:13:09.481327+0000","last_fullsized":"2024-04-25T10:13:09.481327+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.481327+0000","last_change":"2024-04-25T10:13:09.481327+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.481327+0000","last_undegraded":"2024-04-25T10:13:09.481327+0000","last_fullsized":"2024-04-25T10:13:09.481327+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.479767+0000","last_change":"2024-04-25T10:13:09.479767+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.479767+0000","last_undegraded":"2024-04-25T10:13:09.479767+0000","last_fullsized":"2024-04-25T10:13:09.479767+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.477942+0000","last_change":"2024-04-25T10:13:09.477942+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.477942+0000","last_undegraded":"2024-04-25T10:13:09.477942+0000","last_fullsized":"2024-04-25T10:13:09.477942+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476971+0000","last_change":"2024-04-25T10:13:09.476971+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476971+0000","last_undegraded":"2024-04-25T10:13:09.476971+0000","last_fullsized":"2024-04-25T10:13:09.476971+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.482136+0000","last_change":"2024-04-25T10:13:09.482136+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.482136+0000","last_undegraded":"2024-04-25T10:13:09.482136+0000","last_fullsized":"2024-04-25T10:13:09.482136+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.475496+0000","last_change":"2024-04-25T10:13:09.475496+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.475496+0000","last_undegraded":"2024-04-25T10:13:09.475496+0000","last_fullsized":"2024-04-25T10:13:09.475496+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476512+0000","last_change":"2024-04-25T10:13:09.476512+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476512+0000","last_undegraded":"2024-04-25T10:13:09.476512+0000","last_fullsized":"2024-04-25T10:13:09.476512+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.482639+0000","last_change":"2024-04-25T10:13:09.482639+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.482639+0000","last_undegraded":"2024-04-25T10:13:09.482639+0000","last_fullsized":"2024-04-25T10:13:09.482639+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.475496+0000","last_change":"2024-04-25T10:13:09.475496+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.475496+0000","last_undegraded":"2024-04-25T10:13:09.475496+0000","last_fullsized":"2024-04-25T10:13:09.475496+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.478457+0000","last_change":"2024-04-25T10:13:09.478457+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.478457+0000","last_undegraded":"2024-04-25T10:13:09.478457+0000","last_fullsized":"2024-04-25T10:13:09.478457+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.481223+0000","last_change":"2024-04-25T10:13:09.481223+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.481223+0000","last_undegraded":"2024-04-25T10:13:09.481223+0000","last_fullsized":"2024-04-25T10:13:09.481223+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476368+0000","last_change":"2024-04-25T10:13:09.476368+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476368+0000","last_undegraded":"2024-04-25T10:13:09.476368+0000","last_fullsized":"2024-04-25T10:13:09.476368+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.478770+0000","last_change":"2024-04-25T10:13:09.478770+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.478770+0000","last_undegraded":"2024-04-25T10:13:09.478770+0000","last_fullsized":"2024-04-25T10:13:09.478770+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.475498+0000","last_change":"2024-04-25T10:13:09.475498+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.475498+0000","last_undegraded":"2024-04-25T10:13:09.475498+0000","last_fullsized":"2024-04-25T10:13:09.475498+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.476791+0000","last_change":"2024-04-25T10:13:09.476791+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.476791+0000","last_undegraded":"2024-04-25T10:13:09.476791+0000","last_fullsized":"2024-04-25T10:13:09.476791+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.477778+0000","last_change":"2024-04-25T10:13:09.477778+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.477778+0000","last_undegraded":"2024-04-25T10:13:09.477778+0000","last_fullsized":"2024-04-25T10:13:09.477778+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.481745+0000","last_change":"2024-04-25T10:13:09.481745+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.481745+0000","last_undegraded":"2024-04-25T10:13:09.481745+0000","last_fullsized":"2024-04-25T10:13:09.481745+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.481283+0000","last_change":"2024-04-25T10:13:09.481283+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.481283+0000","last_undegraded":"2024-04-25T10:13:09.481283+0000","last_fullsized":"2024-04-25T10:13:09.481283+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.477301+0000","last_change":"2024-04-25T10:13:09.477301+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.477301+0000","last_undegraded":"2024-04-25T10:13:09.477301+0000","last_fullsized":"2024-04-25T10:13:09.477301+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.482105+0000","last_change":"2024-04-25T10:13:09.482105+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.482105+0000","last_undegraded":"2024-04-25T10:13:09.482105+0000","last_fullsized":"2024-04-25T10:13:09.482105+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:13:10.469777+0000","last_change":"2024-04-25T10:13:10.469777+0000","last_active":"2024-04-25T10:13:10.469777+0000","last_peered":"2024-04-25T10:13:10.469777+0000","last_clean":"2024-04-25T10:13:10.469777+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:10.469777+0000","last_undegraded":"2024-04-25T10:13:10.469777+0000","last_fullsized":"2024-04-25T10:13:10.469777+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":129,"state":"creating+peering","last_fresh":"2024-04-25T10:13:09.486045+0000","last_change":"2024-04-25T10:13:09.486045+0000","last_active":"2024-04-25T10:13:09.466674+0000","last_peered":"2024-04-25T10:13:09.466674+0000","last_clean":"2024-04-25T10:13:09.466674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:13:09.486045+0000","last_undegraded":"2024-04-25T10:13:09.486045+0000","last_fullsized":"2024-04-25T10:13:09.486045+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":374,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T10:13:09.486683+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:13:09.486683+0000","last_peered":"2024-04-25T10:13:09.486683+0000","last_clean":"2024-04-25T10:13:09.486683+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:13:09.486683+0000","last_undegraded":"2024-04-25T10:13:09.486683+0000","last_fullsized":"2024-04-25T10:13:09.486683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69440,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302400,"statfs":{"total":96636764160,"available":96565657600,"internally_reserved":0,"allocated":2207744,"data_stored":1341427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46644,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325196,"statfs":{"total":96636764160,"available":96589000704,"internally_reserved":0,"allocated":1613824,"data_stored":751059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59112,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312728,"statfs":{"total":96636764160,"available":96576233472,"internally_reserved":0,"allocated":1601536,"data_stored":744850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574990,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73012,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298828,"statfs":{"total":96636764160,"available":96561999872,"internally_reserved":0,"allocated":1613824,"data_stored":751059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73004,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1601536,"data_stored":744850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46632,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325208,"statfs":{"total":96636764160,"available":96589012992,"internally_reserved":0,"allocated":1601536,"data_stored":744850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59124,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312716,"statfs":{"total":96636764160,"available":96576221184,"internally_reserved":0,"allocated":1613824,"data_stored":751059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69428,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302412,"statfs":{"total":96636764160,"available":96565669888,"internally_reserved":0,"allocated":2195456,"data_stored":1335218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:15.881 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:16.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.032+0000 7fef0daf5640 1 Processor -- start 2024-04-25T10:13:16.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.032+0000 7fef0daf5640 1 -- start start 2024-04-25T10:13:16.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.032+0000 7fef0daf5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef08152750 0x7fef080ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.032+0000 7fef0daf5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fef08152b30 0x7fef080ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.032+0000 7fef0daf5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0806d350 con 0x7fef080b17b0 2024-04-25T10:13:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.032+0000 7fef0daf5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0815bf90 con 0x7fef08152b30 2024-04-25T10:13:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.032+0000 7fef0daf5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0806d5d0 con 0x7fef08152750 2024-04-25T10:13:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7fef077fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef08152750 0x7fef080ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7fef07fff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fef080b17b0 0x7fef080b1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:43076/0 (socket says 172.21.15.133:43076) 2024-04-25T10:13:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7feefffff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fef08152b30 0x7fef080ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7fef07fff640 1 -- 172.21.15.133:0/4109592930 learned_addr learned my addr 172.21.15.133:0/4109592930 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7fef077fe640 1 -- 172.21.15.133:0/4109592930 >> v2:172.21.15.162:3300/0 conn(0x7fef08152b30 msgr2=0x7fef080ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7fef077fe640 1 --2- 172.21.15.133:0/4109592930 >> v2:172.21.15.162:3300/0 conn(0x7fef08152b30 0x7fef080ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7fef077fe640 1 -- 172.21.15.133:0/4109592930 >> v1:172.21.15.162:6789/0 conn(0x7fef080b17b0 legacy=0x7fef080b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7fef077fe640 1 -- 172.21.15.133:0/4109592930 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0815c230 con 0x7fef08152750 2024-04-25T10:13:16.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7fef077fe640 1 --2- 172.21.15.133:0/4109592930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef08152750 0x7fef080ce330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feef80097c0 tx=0x7feef801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c47d31f3dcc364ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7fef06ffd640 1 -- 172.21.15.133:0/4109592930 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef80b0070 con 0x7fef08152750 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7fef06ffd640 1 -- 172.21.15.133:0/4109592930 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef80ab600 con 0x7fef08152750 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.036+0000 7fef06ffd640 1 -- 172.21.15.133:0/4109592930 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef80ab9d0 con 0x7fef08152750 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- 172.21.15.133:0/4109592930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef08152750 msgr2=0x7fef080ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 --2- 172.21.15.133:0/4109592930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef08152750 0x7fef080ce330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feef80097c0 tx=0x7feef801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- 172.21.15.133:0/4109592930 shutdown_connections 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 --2- 172.21.15.133:0/4109592930 >> v2:172.21.15.162:3300/0 conn(0x7fef08152b30 0x7fef080ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 --2- 172.21.15.133:0/4109592930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef08152750 0x7fef080ce330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- 172.21.15.133:0/4109592930 >> 172.21.15.133:0/4109592930 conn(0x7fef08067000 msgr2=0x7fef0806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- 172.21.15.133:0/4109592930 shutdown_connections 2024-04-25T10:13:16.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- 172.21.15.133:0/4109592930 wait complete. 2024-04-25T10:13:16.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 Processor -- start 2024-04-25T10:13:16.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- start start 2024-04-25T10:13:16.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef080b17b0 0x7fef081f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef08152750 0x7fef0814c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef08152b30 0x7fef081517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef081600a0 con 0x7fef08152b30 2024-04-25T10:13:16.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0815ff20 con 0x7fef080b17b0 2024-04-25T10:13:16.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef08160220 con 0x7fef08152750 2024-04-25T10:13:16.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef08152b30 0x7fef081517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef07fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef08152b30 0x7fef081517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38036/0 (socket says 172.21.15.133:38036) 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef07fff640 1 -- 172.21.15.133:0/4071235505 learned_addr learned my addr 172.21.15.133:0/4071235505 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef077fe640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef080b17b0 0x7fef081f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7feefffff640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef08152750 0x7fef0814c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef07fff640 1 -- 172.21.15.133:0/4071235505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef08152750 msgr2=0x7fef0814c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef07fff640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef08152750 0x7fef0814c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef07fff640 1 -- 172.21.15.133:0/4071235505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef080b17b0 msgr2=0x7fef081f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef07fff640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef080b17b0 0x7fef081f8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef07fff640 1 -- 172.21.15.133:0/4071235505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef080c7510 con 0x7fef08152b30 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef07fff640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef08152b30 0x7fef081517b0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fef0000ca20 tx=0x7fef0000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef04ff9640 1 -- 172.21.15.133:0/4071235505 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00017070 con 0x7fef08152b30 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef04ff9640 1 -- 172.21.15.133:0/4071235505 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0000abd0 con 0x7fef08152b30 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- 172.21.15.133:0/4071235505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0814d730 con 0x7fef08152b30 2024-04-25T10:13:16.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef04ff9640 1 -- 172.21.15.133:0/4071235505 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00012400 con 0x7fef08152b30 2024-04-25T10:13:16.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- 172.21.15.133:0/4071235505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef081569e0 con 0x7fef08152b30 2024-04-25T10:13:16.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.040+0000 7fef0daf5640 1 -- 172.21.15.133:0/4071235505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef08068ef0 con 0x7fef08152b30 2024-04-25T10:13:16.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.044+0000 7fef04ff9640 1 -- 172.21.15.133:0/4071235505 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef0001a070 con 0x7fef08152b30 2024-04-25T10:13:16.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.044+0000 7fef04ff9640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feee8066f60 0x7feee8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.044+0000 7fef04ff9640 1 -- 172.21.15.133:0/4071235505 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fef00084db0 con 0x7fef08152b30 2024-04-25T10:13:16.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.044+0000 7fef077fe640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feee8066f60 0x7feee8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.044+0000 7fef077fe640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feee8066f60 0x7feee8069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7feef80ad000 tx=0x7feef8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.048+0000 7fef04ff9640 1 -- 172.21.15.133:0/4071235505 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef0814d730 con 0x7fef08152b30 2024-04-25T10:13:16.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.200+0000 7fef0daf5640 1 -- 172.21.15.133:0/4071235505 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef0815a550 con 0x7feee8066f60 2024-04-25T10:13:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.212+0000 7fef04ff9640 1 -- 172.21.15.133:0/4071235505 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7fef0815a550 con 0x7feee8066f60 2024-04-25T10:13:16.216 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:13:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 -- 172.21.15.133:0/4071235505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feee8066f60 msgr2=0x7feee8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feee8066f60 0x7feee8069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7feef80ad000 tx=0x7feef8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 -- 172.21.15.133:0/4071235505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef08152b30 msgr2=0x7fef081517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef08152b30 0x7fef081517b0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fef0000ca20 tx=0x7fef0000cee0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 -- 172.21.15.133:0/4071235505 shutdown_connections 2024-04-25T10:13:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feee8066f60 0x7feee8069420 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef08152b30 0x7fef081517b0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fef08152750 0x7fef0814c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 --2- 172.21.15.133:0/4071235505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef080b17b0 0x7fef081f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 -- 172.21.15.133:0/4071235505 >> 172.21.15.133:0/4071235505 conn(0x7fef08067000 msgr2=0x7fef0815ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 -- 172.21.15.133:0/4071235505 shutdown_connections 2024-04-25T10:13:16.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.216+0000 7fef0daf5640 1 -- 172.21.15.133:0/4071235505 wait complete. 2024-04-25T10:13:16.233 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-04-25T10:13:15.925916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":496792,"kb_used_data":14120,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754477928,"statfs":{"total":773094113280,"available":772585398272,"internally_reserved":0,"allocated":14458880,"data_stored":7358172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001598"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759324+0000","last_change":"2024-04-25T10:13:11.518523+0000","last_active":"2024-04-25T10:13:13.759324+0000","last_peered":"2024-04-25T10:13:13.759324+0000","last_clean":"2024-04-25T10:13:13.759324+0000","last_became_active":"2024-04-25T10:13:11.518246+0000","last_became_peered":"2024-04-25T10:13:11.518246+0000","last_unstale":"2024-04-25T10:13:13.759324+0000","last_undegraded":"2024-04-25T10:13:13.759324+0000","last_fullsized":"2024-04-25T10:13:13.759324+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:59.087475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514796+0000","last_change":"2024-04-25T10:13:11.566029+0000","last_active":"2024-04-25T10:13:12.514796+0000","last_peered":"2024-04-25T10:13:12.514796+0000","last_clean":"2024-04-25T10:13:12.514796+0000","last_became_active":"2024-04-25T10:13:11.565395+0000","last_became_peered":"2024-04-25T10:13:11.565395+0000","last_unstale":"2024-04-25T10:13:12.514796+0000","last_undegraded":"2024-04-25T10:13:12.514796+0000","last_fullsized":"2024-04-25T10:13:12.514796+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:46.637364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509560+0000","last_change":"2024-04-25T10:13:11.529012+0000","last_active":"2024-04-25T10:13:12.509560+0000","last_peered":"2024-04-25T10:13:12.509560+0000","last_clean":"2024-04-25T10:13:12.509560+0000","last_became_active":"2024-04-25T10:13:11.527515+0000","last_became_peered":"2024-04-25T10:13:11.527515+0000","last_unstale":"2024-04-25T10:13:12.509560+0000","last_undegraded":"2024-04-25T10:13:12.509560+0000","last_fullsized":"2024-04-25T10:13:12.509560+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:22.965494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514436+0000","last_change":"2024-04-25T10:13:11.551694+0000","last_active":"2024-04-25T10:13:12.514436+0000","last_peered":"2024-04-25T10:13:12.514436+0000","last_clean":"2024-04-25T10:13:12.514436+0000","last_became_active":"2024-04-25T10:13:11.551054+0000","last_became_peered":"2024-04-25T10:13:11.551054+0000","last_unstale":"2024-04-25T10:13:12.514436+0000","last_undegraded":"2024-04-25T10:13:12.514436+0000","last_fullsized":"2024-04-25T10:13:12.514436+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:38.132411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760155+0000","last_change":"2024-04-25T10:13:11.524769+0000","last_active":"2024-04-25T10:13:13.760155+0000","last_peered":"2024-04-25T10:13:13.760155+0000","last_clean":"2024-04-25T10:13:13.760155+0000","last_became_active":"2024-04-25T10:13:11.524473+0000","last_became_peered":"2024-04-25T10:13:11.524473+0000","last_unstale":"2024-04-25T10:13:13.760155+0000","last_undegraded":"2024-04-25T10:13:13.760155+0000","last_fullsized":"2024-04-25T10:13:13.760155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:52.612669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810454+0000","last_change":"2024-04-25T10:13:11.542185+0000","last_active":"2024-04-25T10:13:12.810454+0000","last_peered":"2024-04-25T10:13:12.810454+0000","last_clean":"2024-04-25T10:13:12.810454+0000","last_became_active":"2024-04-25T10:13:11.541886+0000","last_became_peered":"2024-04-25T10:13:11.541886+0000","last_unstale":"2024-04-25T10:13:12.810454+0000","last_undegraded":"2024-04-25T10:13:12.810454+0000","last_fullsized":"2024-04-25T10:13:12.810454+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:58.188235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780810+0000","last_change":"2024-04-25T10:13:11.564656+0000","last_active":"2024-04-25T10:13:12.780810+0000","last_peered":"2024-04-25T10:13:12.780810+0000","last_clean":"2024-04-25T10:13:12.780810+0000","last_became_active":"2024-04-25T10:13:11.564126+0000","last_became_peered":"2024-04-25T10:13:11.564126+0000","last_unstale":"2024-04-25T10:13:12.780810+0000","last_undegraded":"2024-04-25T10:13:12.780810+0000","last_fullsized":"2024-04-25T10:13:12.780810+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:07.102379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507173+0000","last_change":"2024-04-25T10:13:11.519215+0000","last_active":"2024-04-25T10:13:12.507173+0000","last_peered":"2024-04-25T10:13:12.507173+0000","last_clean":"2024-04-25T10:13:12.507173+0000","last_became_active":"2024-04-25T10:13:11.518497+0000","last_became_peered":"2024-04-25T10:13:11.518497+0000","last_unstale":"2024-04-25T10:13:12.507173+0000","last_undegraded":"2024-04-25T10:13:12.507173+0000","last_fullsized":"2024-04-25T10:13:12.507173+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:33.385792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.811190+0000","last_change":"2024-04-25T10:13:11.575005+0000","last_active":"2024-04-25T10:13:12.811190+0000","last_peered":"2024-04-25T10:13:12.811190+0000","last_clean":"2024-04-25T10:13:12.811190+0000","last_became_active":"2024-04-25T10:13:11.574701+0000","last_became_peered":"2024-04-25T10:13:11.574701+0000","last_unstale":"2024-04-25T10:13:12.811190+0000","last_undegraded":"2024-04-25T10:13:12.811190+0000","last_fullsized":"2024-04-25T10:13:12.811190+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:39.559386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506878+0000","last_change":"2024-04-25T10:13:11.523813+0000","last_active":"2024-04-25T10:13:12.506878+0000","last_peered":"2024-04-25T10:13:12.506878+0000","last_clean":"2024-04-25T10:13:12.506878+0000","last_became_active":"2024-04-25T10:13:11.523508+0000","last_became_peered":"2024-04-25T10:13:11.523508+0000","last_unstale":"2024-04-25T10:13:12.506878+0000","last_undegraded":"2024-04-25T10:13:12.506878+0000","last_fullsized":"2024-04-25T10:13:12.506878+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:56.849747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514527+0000","last_change":"2024-04-25T10:13:11.564687+0000","last_active":"2024-04-25T10:13:12.514527+0000","last_peered":"2024-04-25T10:13:12.514527+0000","last_clean":"2024-04-25T10:13:12.514527+0000","last_became_active":"2024-04-25T10:13:11.564124+0000","last_became_peered":"2024-04-25T10:13:11.564124+0000","last_unstale":"2024-04-25T10:13:12.514527+0000","last_undegraded":"2024-04-25T10:13:12.514527+0000","last_fullsized":"2024-04-25T10:13:12.514527+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:10.162659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759606+0000","last_change":"2024-04-25T10:13:11.518996+0000","last_active":"2024-04-25T10:13:13.759606+0000","last_peered":"2024-04-25T10:13:13.759606+0000","last_clean":"2024-04-25T10:13:13.759606+0000","last_became_active":"2024-04-25T10:13:11.518243+0000","last_became_peered":"2024-04-25T10:13:11.518243+0000","last_unstale":"2024-04-25T10:13:13.759606+0000","last_undegraded":"2024-04-25T10:13:13.759606+0000","last_fullsized":"2024-04-25T10:13:13.759606+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:18.646411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810982+0000","last_change":"2024-04-25T10:13:11.546228+0000","last_active":"2024-04-25T10:13:12.810982+0000","last_peered":"2024-04-25T10:13:12.810982+0000","last_clean":"2024-04-25T10:13:12.810982+0000","last_became_active":"2024-04-25T10:13:11.545888+0000","last_became_peered":"2024-04-25T10:13:11.545888+0000","last_unstale":"2024-04-25T10:13:12.810982+0000","last_undegraded":"2024-04-25T10:13:12.810982+0000","last_fullsized":"2024-04-25T10:13:12.810982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:52.612669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759949+0000","last_change":"2024-04-25T10:13:11.556074+0000","last_active":"2024-04-25T10:13:13.759949+0000","last_peered":"2024-04-25T10:13:13.759949+0000","last_clean":"2024-04-25T10:13:13.759949+0000","last_became_active":"2024-04-25T10:13:11.555629+0000","last_became_peered":"2024-04-25T10:13:11.555629+0000","last_unstale":"2024-04-25T10:13:13.759949+0000","last_undegraded":"2024-04-25T10:13:13.759949+0000","last_fullsized":"2024-04-25T10:13:13.759949+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:47.123922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.758838+0000","last_change":"2024-04-25T10:13:11.526526+0000","last_active":"2024-04-25T10:13:13.758838+0000","last_peered":"2024-04-25T10:13:13.758838+0000","last_clean":"2024-04-25T10:13:13.758838+0000","last_became_active":"2024-04-25T10:13:11.526250+0000","last_became_peered":"2024-04-25T10:13:11.526250+0000","last_unstale":"2024-04-25T10:13:13.758838+0000","last_undegraded":"2024-04-25T10:13:13.758838+0000","last_fullsized":"2024-04-25T10:13:13.758838+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:12.458362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.809981+0000","last_change":"2024-04-25T10:13:11.522012+0000","last_active":"2024-04-25T10:13:12.809981+0000","last_peered":"2024-04-25T10:13:12.809981+0000","last_clean":"2024-04-25T10:13:12.809981+0000","last_became_active":"2024-04-25T10:13:11.521053+0000","last_became_peered":"2024-04-25T10:13:11.521053+0000","last_unstale":"2024-04-25T10:13:12.809981+0000","last_undegraded":"2024-04-25T10:13:12.809981+0000","last_fullsized":"2024-04-25T10:13:12.809981+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:38.132411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.811195+0000","last_change":"2024-04-25T10:13:11.521369+0000","last_active":"2024-04-25T10:13:12.811195+0000","last_peered":"2024-04-25T10:13:12.811195+0000","last_clean":"2024-04-25T10:13:12.811195+0000","last_became_active":"2024-04-25T10:13:11.520948+0000","last_became_peered":"2024-04-25T10:13:11.520948+0000","last_unstale":"2024-04-25T10:13:12.811195+0000","last_undegraded":"2024-04-25T10:13:12.811195+0000","last_fullsized":"2024-04-25T10:13:12.811195+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:43.271736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760046+0000","last_change":"2024-04-25T10:13:11.525396+0000","last_active":"2024-04-25T10:13:13.760046+0000","last_peered":"2024-04-25T10:13:13.760046+0000","last_clean":"2024-04-25T10:13:13.760046+0000","last_became_active":"2024-04-25T10:13:11.525018+0000","last_became_peered":"2024-04-25T10:13:11.525018+0000","last_unstale":"2024-04-25T10:13:13.760046+0000","last_undegraded":"2024-04-25T10:13:13.760046+0000","last_fullsized":"2024-04-25T10:13:13.760046+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:46.637364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781628+0000","last_change":"2024-04-25T10:13:11.566279+0000","last_active":"2024-04-25T10:13:12.781628+0000","last_peered":"2024-04-25T10:13:12.781628+0000","last_clean":"2024-04-25T10:13:12.781628+0000","last_became_active":"2024-04-25T10:13:11.562536+0000","last_became_peered":"2024-04-25T10:13:11.562536+0000","last_unstale":"2024-04-25T10:13:12.781628+0000","last_undegraded":"2024-04-25T10:13:12.781628+0000","last_fullsized":"2024-04-25T10:13:12.781628+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:47.607552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761069+0000","last_change":"2024-04-25T10:13:11.525258+0000","last_active":"2024-04-25T10:13:13.761069+0000","last_peered":"2024-04-25T10:13:13.761069+0000","last_clean":"2024-04-25T10:13:13.761069+0000","last_became_active":"2024-04-25T10:13:11.524739+0000","last_became_peered":"2024-04-25T10:13:11.524739+0000","last_unstale":"2024-04-25T10:13:13.761069+0000","last_undegraded":"2024-04-25T10:13:13.761069+0000","last_fullsized":"2024-04-25T10:13:13.761069+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:52.434397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514594+0000","last_change":"2024-04-25T10:13:11.564376+0000","last_active":"2024-04-25T10:13:12.514594+0000","last_peered":"2024-04-25T10:13:12.514594+0000","last_clean":"2024-04-25T10:13:12.514594+0000","last_became_active":"2024-04-25T10:13:11.563826+0000","last_became_peered":"2024-04-25T10:13:11.563826+0000","last_unstale":"2024-04-25T10:13:12.514594+0000","last_undegraded":"2024-04-25T10:13:12.514594+0000","last_fullsized":"2024-04-25T10:13:12.514594+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:58.188235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759164+0000","last_change":"2024-04-25T10:13:11.567489+0000","last_active":"2024-04-25T10:13:13.759164+0000","last_peered":"2024-04-25T10:13:13.759164+0000","last_clean":"2024-04-25T10:13:13.759164+0000","last_became_active":"2024-04-25T10:13:11.567145+0000","last_became_peered":"2024-04-25T10:13:11.567145+0000","last_unstale":"2024-04-25T10:13:13.759164+0000","last_undegraded":"2024-04-25T10:13:13.759164+0000","last_fullsized":"2024-04-25T10:13:13.759164+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:37.208933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514296+0000","last_change":"2024-04-25T10:13:11.556816+0000","last_active":"2024-04-25T10:13:12.514296+0000","last_peered":"2024-04-25T10:13:12.514296+0000","last_clean":"2024-04-25T10:13:12.514296+0000","last_became_active":"2024-04-25T10:13:11.556288+0000","last_became_peered":"2024-04-25T10:13:11.556288+0000","last_unstale":"2024-04-25T10:13:12.514296+0000","last_undegraded":"2024-04-25T10:13:12.514296+0000","last_fullsized":"2024-04-25T10:13:12.514296+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:36.478303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810170+0000","last_change":"2024-04-25T10:13:11.567581+0000","last_active":"2024-04-25T10:13:12.810170+0000","last_peered":"2024-04-25T10:13:12.810170+0000","last_clean":"2024-04-25T10:13:12.810170+0000","last_became_active":"2024-04-25T10:13:11.567284+0000","last_became_peered":"2024-04-25T10:13:11.567284+0000","last_unstale":"2024-04-25T10:13:12.810170+0000","last_undegraded":"2024-04-25T10:13:12.810170+0000","last_fullsized":"2024-04-25T10:13:12.810170+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:10.162659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510104+0000","last_change":"2024-04-25T10:13:11.562038+0000","last_active":"2024-04-25T10:13:12.510104+0000","last_peered":"2024-04-25T10:13:12.510104+0000","last_clean":"2024-04-25T10:13:12.510104+0000","last_became_active":"2024-04-25T10:13:11.555255+0000","last_became_peered":"2024-04-25T10:13:11.555255+0000","last_unstale":"2024-04-25T10:13:12.510104+0000","last_undegraded":"2024-04-25T10:13:12.510104+0000","last_fullsized":"2024-04-25T10:13:12.510104+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:53.723716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509065+0000","last_change":"2024-04-25T10:13:11.564767+0000","last_active":"2024-04-25T10:13:12.509065+0000","last_peered":"2024-04-25T10:13:12.509065+0000","last_clean":"2024-04-25T10:13:12.509065+0000","last_became_active":"2024-04-25T10:13:11.564094+0000","last_became_peered":"2024-04-25T10:13:11.564094+0000","last_unstale":"2024-04-25T10:13:12.509065+0000","last_undegraded":"2024-04-25T10:13:12.509065+0000","last_fullsized":"2024-04-25T10:13:12.509065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.305600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781823+0000","last_change":"2024-04-25T10:13:11.570693+0000","last_active":"2024-04-25T10:13:12.781823+0000","last_peered":"2024-04-25T10:13:12.781823+0000","last_clean":"2024-04-25T10:13:12.781823+0000","last_became_active":"2024-04-25T10:13:11.563159+0000","last_became_peered":"2024-04-25T10:13:11.563159+0000","last_unstale":"2024-04-25T10:13:12.781823+0000","last_undegraded":"2024-04-25T10:13:12.781823+0000","last_fullsized":"2024-04-25T10:13:12.781823+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:27.760139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506676+0000","last_change":"2024-04-25T10:13:11.524297+0000","last_active":"2024-04-25T10:13:12.506676+0000","last_peered":"2024-04-25T10:13:12.506676+0000","last_clean":"2024-04-25T10:13:12.506676+0000","last_became_active":"2024-04-25T10:13:11.523906+0000","last_became_peered":"2024-04-25T10:13:11.523906+0000","last_unstale":"2024-04-25T10:13:12.506676+0000","last_undegraded":"2024-04-25T10:13:12.506676+0000","last_fullsized":"2024-04-25T10:13:12.506676+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:42.816079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508821+0000","last_change":"2024-04-25T10:13:11.552751+0000","last_active":"2024-04-25T10:13:12.508821+0000","last_peered":"2024-04-25T10:13:12.508821+0000","last_clean":"2024-04-25T10:13:12.508821+0000","last_became_active":"2024-04-25T10:13:11.551644+0000","last_became_peered":"2024-04-25T10:13:11.551644+0000","last_unstale":"2024-04-25T10:13:12.508821+0000","last_undegraded":"2024-04-25T10:13:12.508821+0000","last_fullsized":"2024-04-25T10:13:12.508821+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:33.385792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509621+0000","last_change":"2024-04-25T10:13:11.558991+0000","last_active":"2024-04-25T10:13:12.509621+0000","last_peered":"2024-04-25T10:13:12.509621+0000","last_clean":"2024-04-25T10:13:12.509621+0000","last_became_active":"2024-04-25T10:13:11.553029+0000","last_became_peered":"2024-04-25T10:13:11.553029+0000","last_unstale":"2024-04-25T10:13:12.509621+0000","last_undegraded":"2024-04-25T10:13:12.509621+0000","last_fullsized":"2024-04-25T10:13:12.509621+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:49.372602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507272+0000","last_change":"2024-04-25T10:13:11.525057+0000","last_active":"2024-04-25T10:13:12.507272+0000","last_peered":"2024-04-25T10:13:12.507272+0000","last_clean":"2024-04-25T10:13:12.507272+0000","last_became_active":"2024-04-25T10:13:11.522942+0000","last_became_peered":"2024-04-25T10:13:11.522942+0000","last_unstale":"2024-04-25T10:13:12.507272+0000","last_undegraded":"2024-04-25T10:13:12.507272+0000","last_fullsized":"2024-04-25T10:13:12.507272+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.305600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506251+0000","last_change":"2024-04-25T10:13:11.522239+0000","last_active":"2024-04-25T10:13:12.506251+0000","last_peered":"2024-04-25T10:13:12.506251+0000","last_clean":"2024-04-25T10:13:12.506251+0000","last_became_active":"2024-04-25T10:13:11.521940+0000","last_became_peered":"2024-04-25T10:13:11.521940+0000","last_unstale":"2024-04-25T10:13:12.506251+0000","last_undegraded":"2024-04-25T10:13:12.506251+0000","last_fullsized":"2024-04-25T10:13:12.506251+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:42.493948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507440+0000","last_change":"2024-04-25T10:13:11.524308+0000","last_active":"2024-04-25T10:13:12.507440+0000","last_peered":"2024-04-25T10:13:12.507440+0000","last_clean":"2024-04-25T10:13:12.507440+0000","last_became_active":"2024-04-25T10:13:11.523488+0000","last_became_peered":"2024-04-25T10:13:11.523488+0000","last_unstale":"2024-04-25T10:13:12.507440+0000","last_undegraded":"2024-04-25T10:13:12.507440+0000","last_fullsized":"2024-04-25T10:13:12.507440+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:38.172285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"131'4","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510113+0000","last_change":"2024-04-25T10:13:10.509075+0000","last_active":"2024-04-25T10:13:12.510113+0000","last_peered":"2024-04-25T10:13:12.510113+0000","last_clean":"2024-04-25T10:13:12.510113+0000","last_became_active":"2024-04-25T10:13:10.506901+0000","last_became_peered":"2024-04-25T10:13:10.506901+0000","last_unstale":"2024-04-25T10:13:12.510113+0000","last_undegraded":"2024-04-25T10:13:12.510113+0000","last_fullsized":"2024-04-25T10:13:12.510113+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:43.483874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759499+0000","last_change":"2024-04-25T10:13:11.567350+0000","last_active":"2024-04-25T10:13:13.759499+0000","last_peered":"2024-04-25T10:13:13.759499+0000","last_clean":"2024-04-25T10:13:13.759499+0000","last_became_active":"2024-04-25T10:13:11.567065+0000","last_became_peered":"2024-04-25T10:13:11.567065+0000","last_unstale":"2024-04-25T10:13:13.759499+0000","last_undegraded":"2024-04-25T10:13:13.759499+0000","last_fullsized":"2024-04-25T10:13:13.759499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:14.942690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760310+0000","last_change":"2024-04-25T10:13:10.504453+0000","last_active":"2024-04-25T10:13:13.760310+0000","last_peered":"2024-04-25T10:13:13.760310+0000","last_clean":"2024-04-25T10:13:13.760310+0000","last_became_active":"2024-04-25T10:13:10.504177+0000","last_became_peered":"2024-04-25T10:13:10.504177+0000","last_unstale":"2024-04-25T10:13:13.760310+0000","last_undegraded":"2024-04-25T10:13:13.760310+0000","last_fullsized":"2024-04-25T10:13:13.760310+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:47.372814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514629+0000","last_change":"2024-04-25T10:13:11.573265+0000","last_active":"2024-04-25T10:13:12.514629+0000","last_peered":"2024-04-25T10:13:12.514629+0000","last_clean":"2024-04-25T10:13:12.514629+0000","last_became_active":"2024-04-25T10:13:11.568213+0000","last_became_peered":"2024-04-25T10:13:11.568213+0000","last_unstale":"2024-04-25T10:13:12.514629+0000","last_undegraded":"2024-04-25T10:13:12.514629+0000","last_fullsized":"2024-04-25T10:13:12.514629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:10.130196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781827+0000","last_change":"2024-04-25T10:13:10.520416+0000","last_active":"2024-04-25T10:13:12.781827+0000","last_peered":"2024-04-25T10:13:12.781827+0000","last_clean":"2024-04-25T10:13:12.781827+0000","last_became_active":"2024-04-25T10:13:10.519694+0000","last_became_peered":"2024-04-25T10:13:10.519694+0000","last_unstale":"2024-04-25T10:13:12.781827+0000","last_undegraded":"2024-04-25T10:13:12.781827+0000","last_fullsized":"2024-04-25T10:13:12.781827+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:33.252824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759314+0000","last_change":"2024-04-25T10:13:11.526936+0000","last_active":"2024-04-25T10:13:13.759314+0000","last_peered":"2024-04-25T10:13:13.759314+0000","last_clean":"2024-04-25T10:13:13.759314+0000","last_became_active":"2024-04-25T10:13:11.526618+0000","last_became_peered":"2024-04-25T10:13:11.526618+0000","last_unstale":"2024-04-25T10:13:13.759314+0000","last_undegraded":"2024-04-25T10:13:13.759314+0000","last_fullsized":"2024-04-25T10:13:13.759314+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:20.954445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781022+0000","last_change":"2024-04-25T10:13:10.519322+0000","last_active":"2024-04-25T10:13:12.781022+0000","last_peered":"2024-04-25T10:13:12.781022+0000","last_clean":"2024-04-25T10:13:12.781022+0000","last_became_active":"2024-04-25T10:13:10.518685+0000","last_became_peered":"2024-04-25T10:13:10.518685+0000","last_unstale":"2024-04-25T10:13:12.781022+0000","last_undegraded":"2024-04-25T10:13:12.781022+0000","last_fullsized":"2024-04-25T10:13:12.781022+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:03.662949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510070+0000","last_change":"2024-04-25T10:13:11.523386+0000","last_active":"2024-04-25T10:13:12.510070+0000","last_peered":"2024-04-25T10:13:12.510070+0000","last_clean":"2024-04-25T10:13:12.510070+0000","last_became_active":"2024-04-25T10:13:11.522629+0000","last_became_peered":"2024-04-25T10:13:11.522629+0000","last_unstale":"2024-04-25T10:13:12.510070+0000","last_undegraded":"2024-04-25T10:13:12.510070+0000","last_fullsized":"2024-04-25T10:13:12.510070+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:49.372602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507928+0000","last_change":"2024-04-25T10:13:10.499060+0000","last_active":"2024-04-25T10:13:12.507928+0000","last_peered":"2024-04-25T10:13:12.507928+0000","last_clean":"2024-04-25T10:13:12.507928+0000","last_became_active":"2024-04-25T10:13:10.494838+0000","last_became_peered":"2024-04-25T10:13:10.494838+0000","last_unstale":"2024-04-25T10:13:12.507928+0000","last_undegraded":"2024-04-25T10:13:12.507928+0000","last_fullsized":"2024-04-25T10:13:12.507928+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:51.109162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508381+0000","last_change":"2024-04-25T10:13:11.526929+0000","last_active":"2024-04-25T10:13:12.508381+0000","last_peered":"2024-04-25T10:13:12.508381+0000","last_clean":"2024-04-25T10:13:12.508381+0000","last_became_active":"2024-04-25T10:13:11.526184+0000","last_became_peered":"2024-04-25T10:13:11.526184+0000","last_unstale":"2024-04-25T10:13:12.508381+0000","last_undegraded":"2024-04-25T10:13:12.508381+0000","last_fullsized":"2024-04-25T10:13:12.508381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:22.965494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759768+0000","last_change":"2024-04-25T10:13:10.500210+0000","last_active":"2024-04-25T10:13:13.759768+0000","last_peered":"2024-04-25T10:13:13.759768+0000","last_clean":"2024-04-25T10:13:13.759768+0000","last_became_active":"2024-04-25T10:13:10.499928+0000","last_became_peered":"2024-04-25T10:13:10.499928+0000","last_unstale":"2024-04-25T10:13:13.759768+0000","last_undegraded":"2024-04-25T10:13:13.759768+0000","last_fullsized":"2024-04-25T10:13:13.759768+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:48.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760330+0000","last_change":"2024-04-25T10:13:11.555023+0000","last_active":"2024-04-25T10:13:13.760330+0000","last_peered":"2024-04-25T10:13:13.760330+0000","last_clean":"2024-04-25T10:13:13.760330+0000","last_became_active":"2024-04-25T10:13:11.554728+0000","last_became_peered":"2024-04-25T10:13:11.554728+0000","last_unstale":"2024-04-25T10:13:13.760330+0000","last_undegraded":"2024-04-25T10:13:13.760330+0000","last_fullsized":"2024-04-25T10:13:13.760330+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:39.559386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.809762+0000","last_change":"2024-04-25T10:13:10.504035+0000","last_active":"2024-04-25T10:13:12.809762+0000","last_peered":"2024-04-25T10:13:12.809762+0000","last_clean":"2024-04-25T10:13:12.809762+0000","last_became_active":"2024-04-25T10:13:10.502964+0000","last_became_peered":"2024-04-25T10:13:10.502964+0000","last_unstale":"2024-04-25T10:13:12.809762+0000","last_undegraded":"2024-04-25T10:13:12.809762+0000","last_fullsized":"2024-04-25T10:13:12.809762+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:04.997195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781330+0000","last_change":"2024-04-25T10:13:11.559414+0000","last_active":"2024-04-25T10:13:12.781330+0000","last_peered":"2024-04-25T10:13:12.781330+0000","last_clean":"2024-04-25T10:13:12.781330+0000","last_became_active":"2024-04-25T10:13:11.558823+0000","last_became_peered":"2024-04-25T10:13:11.558823+0000","last_unstale":"2024-04-25T10:13:12.781330+0000","last_undegraded":"2024-04-25T10:13:12.781330+0000","last_fullsized":"2024-04-25T10:13:12.781330+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:06.795281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508653+0000","last_change":"2024-04-25T10:13:10.507874+0000","last_active":"2024-04-25T10:13:12.508653+0000","last_peered":"2024-04-25T10:13:12.508653+0000","last_clean":"2024-04-25T10:13:12.508653+0000","last_became_active":"2024-04-25T10:13:10.507221+0000","last_became_peered":"2024-04-25T10:13:10.507221+0000","last_unstale":"2024-04-25T10:13:12.508653+0000","last_undegraded":"2024-04-25T10:13:12.508653+0000","last_fullsized":"2024-04-25T10:13:12.508653+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:45.472460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.758603+0000","last_change":"2024-04-25T10:13:11.548771+0000","last_active":"2024-04-25T10:13:13.758603+0000","last_peered":"2024-04-25T10:13:13.758603+0000","last_clean":"2024-04-25T10:13:13.758603+0000","last_became_active":"2024-04-25T10:13:11.548373+0000","last_became_peered":"2024-04-25T10:13:11.548373+0000","last_unstale":"2024-04-25T10:13:13.758603+0000","last_undegraded":"2024-04-25T10:13:13.758603+0000","last_fullsized":"2024-04-25T10:13:13.758603+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:59.266348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514921+0000","last_change":"2024-04-25T10:13:10.494868+0000","last_active":"2024-04-25T10:13:12.514921+0000","last_peered":"2024-04-25T10:13:12.514921+0000","last_clean":"2024-04-25T10:13:12.514921+0000","last_became_active":"2024-04-25T10:13:10.494288+0000","last_became_peered":"2024-04-25T10:13:10.494288+0000","last_unstale":"2024-04-25T10:13:12.514921+0000","last_undegraded":"2024-04-25T10:13:12.514921+0000","last_fullsized":"2024-04-25T10:13:12.514921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:37.169182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810475+0000","last_change":"2024-04-25T10:13:11.513098+0000","last_active":"2024-04-25T10:13:12.810475+0000","last_peered":"2024-04-25T10:13:12.810475+0000","last_clean":"2024-04-25T10:13:12.810475+0000","last_became_active":"2024-04-25T10:13:11.512776+0000","last_became_peered":"2024-04-25T10:13:11.512776+0000","last_unstale":"2024-04-25T10:13:12.810475+0000","last_undegraded":"2024-04-25T10:13:12.810475+0000","last_fullsized":"2024-04-25T10:13:12.810475+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:10.130196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759446+0000","last_change":"2024-04-25T10:13:10.497439+0000","last_active":"2024-04-25T10:13:13.759446+0000","last_peered":"2024-04-25T10:13:13.759446+0000","last_clean":"2024-04-25T10:13:13.759446+0000","last_became_active":"2024-04-25T10:13:10.496013+0000","last_became_peered":"2024-04-25T10:13:10.496013+0000","last_unstale":"2024-04-25T10:13:13.759446+0000","last_undegraded":"2024-04-25T10:13:13.759446+0000","last_fullsized":"2024-04-25T10:13:13.759446+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:19.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510044+0000","last_change":"2024-04-25T10:13:11.565816+0000","last_active":"2024-04-25T10:13:12.510044+0000","last_peered":"2024-04-25T10:13:12.510044+0000","last_clean":"2024-04-25T10:13:12.510044+0000","last_became_active":"2024-04-25T10:13:11.565141+0000","last_became_peered":"2024-04-25T10:13:11.565141+0000","last_unstale":"2024-04-25T10:13:12.510044+0000","last_undegraded":"2024-04-25T10:13:12.510044+0000","last_fullsized":"2024-04-25T10:13:12.510044+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.375918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"131'3","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781913+0000","last_change":"2024-04-25T10:13:10.520399+0000","last_active":"2024-04-25T10:13:12.781913+0000","last_peered":"2024-04-25T10:13:12.781913+0000","last_clean":"2024-04-25T10:13:12.781913+0000","last_became_active":"2024-04-25T10:13:10.519892+0000","last_became_peered":"2024-04-25T10:13:10.519892+0000","last_unstale":"2024-04-25T10:13:12.781913+0000","last_undegraded":"2024-04-25T10:13:12.781913+0000","last_fullsized":"2024-04-25T10:13:12.781913+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:21.502204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509200+0000","last_change":"2024-04-25T10:13:11.529758+0000","last_active":"2024-04-25T10:13:12.509200+0000","last_peered":"2024-04-25T10:13:12.509200+0000","last_clean":"2024-04-25T10:13:12.509200+0000","last_became_active":"2024-04-25T10:13:11.528989+0000","last_became_peered":"2024-04-25T10:13:11.528989+0000","last_unstale":"2024-04-25T10:13:12.509200+0000","last_undegraded":"2024-04-25T10:13:12.509200+0000","last_fullsized":"2024-04-25T10:13:12.509200+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:38.172285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"131'6","reported_seq":20,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781106+0000","last_change":"2024-04-25T10:13:10.523732+0000","last_active":"2024-04-25T10:13:12.781106+0000","last_peered":"2024-04-25T10:13:12.781106+0000","last_clean":"2024-04-25T10:13:12.781106+0000","last_became_active":"2024-04-25T10:13:10.523019+0000","last_became_peered":"2024-04-25T10:13:10.523019+0000","last_unstale":"2024-04-25T10:13:12.781106+0000","last_undegraded":"2024-04-25T10:13:12.781106+0000","last_fullsized":"2024-04-25T10:13:12.781106+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:35.743803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507053+0000","last_change":"2024-04-25T10:13:11.520435+0000","last_active":"2024-04-25T10:13:12.507053+0000","last_peered":"2024-04-25T10:13:12.507053+0000","last_clean":"2024-04-25T10:13:12.507053+0000","last_became_active":"2024-04-25T10:13:11.520147+0000","last_became_peered":"2024-04-25T10:13:11.520147+0000","last_unstale":"2024-04-25T10:13:12.507053+0000","last_undegraded":"2024-04-25T10:13:12.507053+0000","last_fullsized":"2024-04-25T10:13:12.507053+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:02.552382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759467+0000","last_change":"2024-04-25T10:13:10.497816+0000","last_active":"2024-04-25T10:13:13.759467+0000","last_peered":"2024-04-25T10:13:13.759467+0000","last_clean":"2024-04-25T10:13:13.759467+0000","last_became_active":"2024-04-25T10:13:10.497547+0000","last_became_peered":"2024-04-25T10:13:10.497547+0000","last_unstale":"2024-04-25T10:13:13.759467+0000","last_undegraded":"2024-04-25T10:13:13.759467+0000","last_fullsized":"2024-04-25T10:13:13.759467+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:07.860557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781622+0000","last_change":"2024-04-25T10:13:11.570950+0000","last_active":"2024-04-25T10:13:12.781622+0000","last_peered":"2024-04-25T10:13:12.781622+0000","last_clean":"2024-04-25T10:13:12.781622+0000","last_became_active":"2024-04-25T10:13:11.570376+0000","last_became_peered":"2024-04-25T10:13:11.570376+0000","last_unstale":"2024-04-25T10:13:12.781622+0000","last_undegraded":"2024-04-25T10:13:12.781622+0000","last_fullsized":"2024-04-25T10:13:12.781622+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:51.592552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508365+0000","last_change":"2024-04-25T10:13:10.507142+0000","last_active":"2024-04-25T10:13:12.508365+0000","last_peered":"2024-04-25T10:13:12.508365+0000","last_clean":"2024-04-25T10:13:12.508365+0000","last_became_active":"2024-04-25T10:13:10.506580+0000","last_became_peered":"2024-04-25T10:13:10.506580+0000","last_unstale":"2024-04-25T10:13:12.508365+0000","last_undegraded":"2024-04-25T10:13:12.508365+0000","last_fullsized":"2024-04-25T10:13:12.508365+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:41.464459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760601+0000","last_change":"2024-04-25T10:13:11.524605+0000","last_active":"2024-04-25T10:13:13.760601+0000","last_peered":"2024-04-25T10:13:13.760601+0000","last_clean":"2024-04-25T10:13:13.760601+0000","last_became_active":"2024-04-25T10:13:11.523134+0000","last_became_peered":"2024-04-25T10:13:11.523134+0000","last_unstale":"2024-04-25T10:13:13.760601+0000","last_undegraded":"2024-04-25T10:13:13.760601+0000","last_fullsized":"2024-04-25T10:13:13.760601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:43.271736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"131'8","reported_seq":22,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810197+0000","last_change":"2024-04-25T10:13:10.503861+0000","last_active":"2024-04-25T10:13:12.810197+0000","last_peered":"2024-04-25T10:13:12.810197+0000","last_clean":"2024-04-25T10:13:12.810197+0000","last_became_active":"2024-04-25T10:13:10.503476+0000","last_became_peered":"2024-04-25T10:13:10.503476+0000","last_unstale":"2024-04-25T10:13:12.810197+0000","last_undegraded":"2024-04-25T10:13:12.810197+0000","last_fullsized":"2024-04-25T10:13:12.810197+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.405882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508510+0000","last_change":"2024-04-25T10:13:11.547495+0000","last_active":"2024-04-25T10:13:12.508510+0000","last_peered":"2024-04-25T10:13:12.508510+0000","last_clean":"2024-04-25T10:13:12.508510+0000","last_became_active":"2024-04-25T10:13:11.541513+0000","last_became_peered":"2024-04-25T10:13:11.541513+0000","last_unstale":"2024-04-25T10:13:12.508510+0000","last_undegraded":"2024-04-25T10:13:12.508510+0000","last_fullsized":"2024-04-25T10:13:12.508510+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:59.408986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514982+0000","last_change":"2024-04-25T10:13:10.501668+0000","last_active":"2024-04-25T10:13:12.514982+0000","last_peered":"2024-04-25T10:13:12.514982+0000","last_clean":"2024-04-25T10:13:12.514982+0000","last_became_active":"2024-04-25T10:13:10.501112+0000","last_became_peered":"2024-04-25T10:13:10.501112+0000","last_unstale":"2024-04-25T10:13:12.514982+0000","last_undegraded":"2024-04-25T10:13:12.514982+0000","last_fullsized":"2024-04-25T10:13:12.514982+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:21.962390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780949+0000","last_change":"2024-04-25T10:13:11.559201+0000","last_active":"2024-04-25T10:13:12.780949+0000","last_peered":"2024-04-25T10:13:12.780949+0000","last_clean":"2024-04-25T10:13:12.780949+0000","last_became_active":"2024-04-25T10:13:11.558584+0000","last_became_peered":"2024-04-25T10:13:11.558584+0000","last_unstale":"2024-04-25T10:13:12.780949+0000","last_undegraded":"2024-04-25T10:13:12.780949+0000","last_fullsized":"2024-04-25T10:13:12.780949+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:47.947133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509197+0000","last_change":"2024-04-25T10:13:10.507714+0000","last_active":"2024-04-25T10:13:12.509197+0000","last_peered":"2024-04-25T10:13:12.509197+0000","last_clean":"2024-04-25T10:13:12.509197+0000","last_became_active":"2024-04-25T10:13:10.506915+0000","last_became_peered":"2024-04-25T10:13:10.506915+0000","last_unstale":"2024-04-25T10:13:12.509197+0000","last_undegraded":"2024-04-25T10:13:12.509197+0000","last_fullsized":"2024-04-25T10:13:12.509197+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:31.176627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781867+0000","last_change":"2024-04-25T10:13:11.559225+0000","last_active":"2024-04-25T10:13:12.781867+0000","last_peered":"2024-04-25T10:13:12.781867+0000","last_clean":"2024-04-25T10:13:12.781867+0000","last_became_active":"2024-04-25T10:13:11.558665+0000","last_became_peered":"2024-04-25T10:13:11.558665+0000","last_unstale":"2024-04-25T10:13:12.781867+0000","last_undegraded":"2024-04-25T10:13:12.781867+0000","last_fullsized":"2024-04-25T10:13:12.781867+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:29.750424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810921+0000","last_change":"2024-04-25T10:13:10.502878+0000","last_active":"2024-04-25T10:13:12.810921+0000","last_peered":"2024-04-25T10:13:12.810921+0000","last_clean":"2024-04-25T10:13:12.810921+0000","last_became_active":"2024-04-25T10:13:10.501740+0000","last_became_peered":"2024-04-25T10:13:10.501740+0000","last_unstale":"2024-04-25T10:13:12.810921+0000","last_undegraded":"2024-04-25T10:13:12.810921+0000","last_fullsized":"2024-04-25T10:13:12.810921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:48.234733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506278+0000","last_change":"2024-04-25T10:13:11.548117+0000","last_active":"2024-04-25T10:13:12.506278+0000","last_peered":"2024-04-25T10:13:12.506278+0000","last_clean":"2024-04-25T10:13:12.506278+0000","last_became_active":"2024-04-25T10:13:11.547691+0000","last_became_peered":"2024-04-25T10:13:11.547691+0000","last_unstale":"2024-04-25T10:13:12.506278+0000","last_undegraded":"2024-04-25T10:13:12.506278+0000","last_fullsized":"2024-04-25T10:13:12.506278+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:27.760139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761144+0000","last_change":"2024-04-25T10:13:10.499039+0000","last_active":"2024-04-25T10:13:13.761144+0000","last_peered":"2024-04-25T10:13:13.761144+0000","last_clean":"2024-04-25T10:13:13.761144+0000","last_became_active":"2024-04-25T10:13:10.498545+0000","last_became_peered":"2024-04-25T10:13:10.498545+0000","last_unstale":"2024-04-25T10:13:13.761144+0000","last_undegraded":"2024-04-25T10:13:13.761144+0000","last_fullsized":"2024-04-25T10:13:13.761144+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:52.720613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761304+0000","last_change":"2024-04-25T10:13:11.518690+0000","last_active":"2024-04-25T10:13:13.761304+0000","last_peered":"2024-04-25T10:13:13.761304+0000","last_clean":"2024-04-25T10:13:13.761304+0000","last_became_active":"2024-04-25T10:13:11.518411+0000","last_became_peered":"2024-04-25T10:13:11.518411+0000","last_unstale":"2024-04-25T10:13:13.761304+0000","last_undegraded":"2024-04-25T10:13:13.761304+0000","last_fullsized":"2024-04-25T10:13:13.761304+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:09.135552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508272+0000","last_change":"2024-04-25T10:13:10.504158+0000","last_active":"2024-04-25T10:13:12.508272+0000","last_peered":"2024-04-25T10:13:12.508272+0000","last_clean":"2024-04-25T10:13:12.508272+0000","last_became_active":"2024-04-25T10:13:10.502316+0000","last_became_peered":"2024-04-25T10:13:10.502316+0000","last_unstale":"2024-04-25T10:13:12.508272+0000","last_undegraded":"2024-04-25T10:13:12.508272+0000","last_fullsized":"2024-04-25T10:13:12.508272+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:39.042840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759831+0000","last_change":"2024-04-25T10:13:11.562497+0000","last_active":"2024-04-25T10:13:13.759831+0000","last_peered":"2024-04-25T10:13:13.759831+0000","last_clean":"2024-04-25T10:13:13.759831+0000","last_became_active":"2024-04-25T10:13:11.562192+0000","last_became_peered":"2024-04-25T10:13:11.562192+0000","last_unstale":"2024-04-25T10:13:13.759831+0000","last_undegraded":"2024-04-25T10:13:13.759831+0000","last_fullsized":"2024-04-25T10:13:13.759831+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:27.488802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760075+0000","last_change":"2024-04-25T10:13:10.500218+0000","last_active":"2024-04-25T10:13:13.760075+0000","last_peered":"2024-04-25T10:13:13.760075+0000","last_clean":"2024-04-25T10:13:13.760075+0000","last_became_active":"2024-04-25T10:13:10.499839+0000","last_became_peered":"2024-04-25T10:13:10.499839+0000","last_unstale":"2024-04-25T10:13:13.760075+0000","last_undegraded":"2024-04-25T10:13:13.760075+0000","last_fullsized":"2024-04-25T10:13:13.760075+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:15.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760253+0000","last_change":"2024-04-25T10:13:11.547740+0000","last_active":"2024-04-25T10:13:13.760253+0000","last_peered":"2024-04-25T10:13:13.760253+0000","last_clean":"2024-04-25T10:13:13.760253+0000","last_became_active":"2024-04-25T10:13:11.547457+0000","last_became_peered":"2024-04-25T10:13:11.547457+0000","last_unstale":"2024-04-25T10:13:13.760253+0000","last_undegraded":"2024-04-25T10:13:13.760253+0000","last_fullsized":"2024-04-25T10:13:13.760253+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:10.791929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781126+0000","last_change":"2024-04-25T10:13:10.523607+0000","last_active":"2024-04-25T10:13:12.781126+0000","last_peered":"2024-04-25T10:13:12.781126+0000","last_clean":"2024-04-25T10:13:12.781126+0000","last_became_active":"2024-04-25T10:13:10.522750+0000","last_became_peered":"2024-04-25T10:13:10.522750+0000","last_unstale":"2024-04-25T10:13:12.781126+0000","last_undegraded":"2024-04-25T10:13:12.781126+0000","last_fullsized":"2024-04-25T10:13:12.781126+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:37.642788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514651+0000","last_change":"2024-04-25T10:13:11.574680+0000","last_active":"2024-04-25T10:13:12.514651+0000","last_peered":"2024-04-25T10:13:12.514651+0000","last_clean":"2024-04-25T10:13:12.514651+0000","last_became_active":"2024-04-25T10:13:11.574085+0000","last_became_peered":"2024-04-25T10:13:11.574085+0000","last_unstale":"2024-04-25T10:13:12.514651+0000","last_undegraded":"2024-04-25T10:13:12.514651+0000","last_fullsized":"2024-04-25T10:13:12.514651+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.168445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810338+0000","last_change":"2024-04-25T10:13:10.496461+0000","last_active":"2024-04-25T10:13:12.810338+0000","last_peered":"2024-04-25T10:13:12.810338+0000","last_clean":"2024-04-25T10:13:12.810338+0000","last_became_active":"2024-04-25T10:13:10.495439+0000","last_became_peered":"2024-04-25T10:13:10.495439+0000","last_unstale":"2024-04-25T10:13:12.810338+0000","last_undegraded":"2024-04-25T10:13:12.810338+0000","last_fullsized":"2024-04-25T10:13:12.810338+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:32.382689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761590+0000","last_change":"2024-04-25T10:13:11.525150+0000","last_active":"2024-04-25T10:13:13.761590+0000","last_peered":"2024-04-25T10:13:13.761590+0000","last_clean":"2024-04-25T10:13:13.761590+0000","last_became_active":"2024-04-25T10:13:11.524852+0000","last_became_peered":"2024-04-25T10:13:11.524852+0000","last_unstale":"2024-04-25T10:13:13.761590+0000","last_undegraded":"2024-04-25T10:13:13.761590+0000","last_fullsized":"2024-04-25T10:13:13.761590+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:36.746906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759373+0000","last_change":"2024-04-25T10:13:10.504888+0000","last_active":"2024-04-25T10:13:13.759373+0000","last_peered":"2024-04-25T10:13:13.759373+0000","last_clean":"2024-04-25T10:13:13.759373+0000","last_became_active":"2024-04-25T10:13:10.504352+0000","last_became_peered":"2024-04-25T10:13:10.504352+0000","last_unstale":"2024-04-25T10:13:13.759373+0000","last_undegraded":"2024-04-25T10:13:13.759373+0000","last_fullsized":"2024-04-25T10:13:13.759373+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:51.056571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508240+0000","last_change":"2024-04-25T10:13:11.541243+0000","last_active":"2024-04-25T10:13:12.508240+0000","last_peered":"2024-04-25T10:13:12.508240+0000","last_clean":"2024-04-25T10:13:12.508240+0000","last_became_active":"2024-04-25T10:13:11.540480+0000","last_became_peered":"2024-04-25T10:13:11.540480+0000","last_unstale":"2024-04-25T10:13:12.508240+0000","last_undegraded":"2024-04-25T10:13:12.508240+0000","last_fullsized":"2024-04-25T10:13:12.508240+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:49.237836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"132'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509465+0000","last_change":"2024-04-25T10:13:10.505272+0000","last_active":"2024-04-25T10:13:12.509465+0000","last_peered":"2024-04-25T10:13:12.509465+0000","last_clean":"2024-04-25T10:13:12.509465+0000","last_became_active":"2024-04-25T10:13:10.504580+0000","last_became_peered":"2024-04-25T10:13:10.504580+0000","last_unstale":"2024-04-25T10:13:12.509465+0000","last_undegraded":"2024-04-25T10:13:12.509465+0000","last_fullsized":"2024-04-25T10:13:12.509465+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:55.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780865+0000","last_change":"2024-04-25T10:13:11.571071+0000","last_active":"2024-04-25T10:13:12.780865+0000","last_peered":"2024-04-25T10:13:12.780865+0000","last_clean":"2024-04-25T10:13:12.780865+0000","last_became_active":"2024-04-25T10:13:11.569555+0000","last_became_peered":"2024-04-25T10:13:11.569555+0000","last_unstale":"2024-04-25T10:13:12.780865+0000","last_undegraded":"2024-04-25T10:13:12.780865+0000","last_fullsized":"2024-04-25T10:13:12.780865+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:03.874094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508513+0000","last_change":"2024-04-25T10:13:10.502034+0000","last_active":"2024-04-25T10:13:12.508513+0000","last_peered":"2024-04-25T10:13:12.508513+0000","last_clean":"2024-04-25T10:13:12.508513+0000","last_became_active":"2024-04-25T10:13:10.501199+0000","last_became_peered":"2024-04-25T10:13:10.501199+0000","last_unstale":"2024-04-25T10:13:12.508513+0000","last_undegraded":"2024-04-25T10:13:12.508513+0000","last_fullsized":"2024-04-25T10:13:12.508513+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:02.941844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506669+0000","last_change":"2024-04-25T10:13:11.524061+0000","last_active":"2024-04-25T10:13:12.506669+0000","last_peered":"2024-04-25T10:13:12.506669+0000","last_clean":"2024-04-25T10:13:12.506669+0000","last_became_active":"2024-04-25T10:13:11.523588+0000","last_became_peered":"2024-04-25T10:13:11.523588+0000","last_unstale":"2024-04-25T10:13:12.506669+0000","last_undegraded":"2024-04-25T10:13:12.506669+0000","last_fullsized":"2024-04-25T10:13:12.506669+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:29.750424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810692+0000","last_change":"2024-04-25T10:13:10.501983+0000","last_active":"2024-04-25T10:13:12.810692+0000","last_peered":"2024-04-25T10:13:12.810692+0000","last_clean":"2024-04-25T10:13:12.810692+0000","last_became_active":"2024-04-25T10:13:10.501561+0000","last_became_peered":"2024-04-25T10:13:10.501561+0000","last_unstale":"2024-04-25T10:13:12.810692+0000","last_undegraded":"2024-04-25T10:13:12.810692+0000","last_fullsized":"2024-04-25T10:13:12.810692+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:53.326070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810602+0000","last_change":"2024-04-25T10:13:11.515291+0000","last_active":"2024-04-25T10:13:12.810602+0000","last_peered":"2024-04-25T10:13:12.810602+0000","last_clean":"2024-04-25T10:13:12.810602+0000","last_became_active":"2024-04-25T10:13:11.514943+0000","last_became_peered":"2024-04-25T10:13:11.514943+0000","last_unstale":"2024-04-25T10:13:12.810602+0000","last_undegraded":"2024-04-25T10:13:12.810602+0000","last_fullsized":"2024-04-25T10:13:12.810602+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.168445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514140+0000","last_change":"2024-04-25T10:13:10.509756+0000","last_active":"2024-04-25T10:13:12.514140+0000","last_peered":"2024-04-25T10:13:12.514140+0000","last_clean":"2024-04-25T10:13:12.514140+0000","last_became_active":"2024-04-25T10:13:10.508989+0000","last_became_peered":"2024-04-25T10:13:10.508989+0000","last_unstale":"2024-04-25T10:13:12.514140+0000","last_undegraded":"2024-04-25T10:13:12.514140+0000","last_fullsized":"2024-04-25T10:13:12.514140+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:45.302496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.811001+0000","last_change":"2024-04-25T10:13:11.521300+0000","last_active":"2024-04-25T10:13:12.811001+0000","last_peered":"2024-04-25T10:13:12.811001+0000","last_clean":"2024-04-25T10:13:12.811001+0000","last_became_active":"2024-04-25T10:13:11.521002+0000","last_became_peered":"2024-04-25T10:13:11.521002+0000","last_unstale":"2024-04-25T10:13:12.811001+0000","last_undegraded":"2024-04-25T10:13:12.811001+0000","last_fullsized":"2024-04-25T10:13:12.811001+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:10.791929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781666+0000","last_change":"2024-04-25T10:13:10.520125+0000","last_active":"2024-04-25T10:13:12.781666+0000","last_peered":"2024-04-25T10:13:12.781666+0000","last_clean":"2024-04-25T10:13:12.781666+0000","last_became_active":"2024-04-25T10:13:10.519603+0000","last_became_peered":"2024-04-25T10:13:10.519603+0000","last_unstale":"2024-04-25T10:13:12.781666+0000","last_undegraded":"2024-04-25T10:13:12.781666+0000","last_fullsized":"2024-04-25T10:13:12.781666+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:42.431009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810604+0000","last_change":"2024-04-25T10:13:11.554463+0000","last_active":"2024-04-25T10:13:12.810604+0000","last_peered":"2024-04-25T10:13:12.810604+0000","last_clean":"2024-04-25T10:13:12.810604+0000","last_became_active":"2024-04-25T10:13:11.554189+0000","last_became_peered":"2024-04-25T10:13:11.554189+0000","last_unstale":"2024-04-25T10:13:12.810604+0000","last_undegraded":"2024-04-25T10:13:12.810604+0000","last_fullsized":"2024-04-25T10:13:12.810604+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:46.637364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810630+0000","last_change":"2024-04-25T10:13:10.518854+0000","last_active":"2024-04-25T10:13:12.810630+0000","last_peered":"2024-04-25T10:13:12.810630+0000","last_clean":"2024-04-25T10:13:12.810630+0000","last_became_active":"2024-04-25T10:13:10.518358+0000","last_became_peered":"2024-04-25T10:13:10.518358+0000","last_unstale":"2024-04-25T10:13:12.810630+0000","last_undegraded":"2024-04-25T10:13:12.810630+0000","last_fullsized":"2024-04-25T10:13:12.810630+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:18.036486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781697+0000","last_change":"2024-04-25T10:13:11.570138+0000","last_active":"2024-04-25T10:13:12.781697+0000","last_peered":"2024-04-25T10:13:12.781697+0000","last_clean":"2024-04-25T10:13:12.781697+0000","last_became_active":"2024-04-25T10:13:11.569565+0000","last_became_peered":"2024-04-25T10:13:11.569565+0000","last_unstale":"2024-04-25T10:13:12.781697+0000","last_undegraded":"2024-04-25T10:13:12.781697+0000","last_fullsized":"2024-04-25T10:13:12.781697+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:42.493948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760048+0000","last_change":"2024-04-25T10:13:10.508016+0000","last_active":"2024-04-25T10:13:13.760048+0000","last_peered":"2024-04-25T10:13:13.760048+0000","last_clean":"2024-04-25T10:13:13.760048+0000","last_became_active":"2024-04-25T10:13:10.507618+0000","last_became_peered":"2024-04-25T10:13:10.507618+0000","last_unstale":"2024-04-25T10:13:13.760048+0000","last_undegraded":"2024-04-25T10:13:13.760048+0000","last_fullsized":"2024-04-25T10:13:13.760048+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:57.522399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509790+0000","last_change":"2024-04-25T10:13:11.527767+0000","last_active":"2024-04-25T10:13:12.509790+0000","last_peered":"2024-04-25T10:13:12.509790+0000","last_clean":"2024-04-25T10:13:12.509790+0000","last_became_active":"2024-04-25T10:13:11.527021+0000","last_became_peered":"2024-04-25T10:13:11.527021+0000","last_unstale":"2024-04-25T10:13:12.509790+0000","last_undegraded":"2024-04-25T10:13:12.509790+0000","last_fullsized":"2024-04-25T10:13:12.509790+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:58.525502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781388+0000","last_change":"2024-04-25T10:13:10.523416+0000","last_active":"2024-04-25T10:13:12.781388+0000","last_peered":"2024-04-25T10:13:12.781388+0000","last_clean":"2024-04-25T10:13:12.781388+0000","last_became_active":"2024-04-25T10:13:10.522665+0000","last_became_peered":"2024-04-25T10:13:10.522665+0000","last_unstale":"2024-04-25T10:13:12.781388+0000","last_undegraded":"2024-04-25T10:13:12.781388+0000","last_fullsized":"2024-04-25T10:13:12.781388+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:44.313595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":380,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780419+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:13:12.780419+0000","last_peered":"2024-04-25T10:13:12.780419+0000","last_clean":"2024-04-25T10:13:12.780419+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:13:12.780419+0000","last_undegraded":"2024-04-25T10:13:12.780419+0000","last_fullsized":"2024-04-25T10:13:12.780419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574991,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69484,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302356,"statfs":{"total":96636764160,"available":96565612544,"internally_reserved":0,"allocated":2260992,"data_stored":1363647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574992,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46692,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325148,"statfs":{"total":96636764160,"available":96588951552,"internally_reserved":0,"allocated":1658880,"data_stored":772249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1654784,"data_stored":771599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73052,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298788,"statfs":{"total":96636764160,"available":96561958912,"internally_reserved":0,"allocated":1650688,"data_stored":771577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73064,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298776,"statfs":{"total":96636764160,"available":96561946624,"internally_reserved":0,"allocated":1667072,"data_stored":773259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46692,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325148,"statfs":{"total":96636764160,"available":96588951552,"internally_reserved":0,"allocated":1658880,"data_stored":772193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59164,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312676,"statfs":{"total":96636764160,"available":96576180224,"internally_reserved":0,"allocated":1654784,"data_stored":771647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574992,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69476,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302364,"statfs":{"total":96636764160,"available":96565620736,"internally_reserved":0,"allocated":2252800,"data_stored":1362001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:16.276 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:16.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.380+0000 7fe821d3e640 1 Processor -- start 2024-04-25T10:13:16.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.380+0000 7fe821d3e640 1 -- start start 2024-04-25T10:13:16.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe821d3e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe81c0d0770 0x7fe81c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe821d3e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe81c0b87e0 0x7fe81c0b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe821d3e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c0ceb20 con 0x7fe81c15db00 2024-04-25T10:13:16.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe821d3e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c0d1110 con 0x7fe81c0d0770 2024-04-25T10:13:16.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe821d3e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c0b30a0 con 0x7fe81c0b87e0 2024-04-25T10:13:16.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81b7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe81c15db00 0x7fe81c0b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58538/0 (socket says 172.21.15.133:58538) 2024-04-25T10:13:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81b7fe640 1 -- 172.21.15.133:0/1006441018 learned_addr learned my addr 172.21.15.133:0/1006441018 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81affd640 1 --2- 172.21.15.133:0/1006441018 >> v2:172.21.15.133:3300/0 conn(0x7fe81c0d0770 0x7fe81c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81a7fc640 1 -- 172.21.15.133:0/1006441018 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749888248 0 0) 0x7fe81c0ceb20 con 0x7fe81c15db00 2024-04-25T10:13:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81bfff640 1 --2- 172.21.15.133:0/1006441018 >> v2:172.21.15.162:3300/0 conn(0x7fe81c0b87e0 0x7fe81c0b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81a7fc640 1 -- 172.21.15.133:0/1006441018 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe804003530 con 0x7fe81c15db00 2024-04-25T10:13:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81affd640 1 -- 172.21.15.133:0/1006441018 >> v2:172.21.15.162:3300/0 conn(0x7fe81c0b87e0 msgr2=0x7fe81c0b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81affd640 1 --2- 172.21.15.133:0/1006441018 >> v2:172.21.15.162:3300/0 conn(0x7fe81c0b87e0 0x7fe81c0b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81affd640 1 -- 172.21.15.133:0/1006441018 >> v1:172.21.15.133:6789/0 conn(0x7fe81c15db00 legacy=0x7fe81c0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81a7fc640 1 -- 172.21.15.133:0/1006441018 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 597347298 0 0) 0x7fe804003530 con 0x7fe81c15db00 2024-04-25T10:13:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81affd640 1 -- 172.21.15.133:0/1006441018 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c0b7740 con 0x7fe81c0d0770 2024-04-25T10:13:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81bfff640 1 --2- 172.21.15.133:0/1006441018 >> v2:172.21.15.162:3300/0 conn(0x7fe81c0b87e0 0x7fe81c0b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:16.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81affd640 1 --2- 172.21.15.133:0/1006441018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c0d0770 0x7fe81c0d0bd0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0b5a00 tx=0x7fe81001c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7184eaadd79f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81a7fc640 1 -- 172.21.15.133:0/1006441018 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8100b0070 con 0x7fe81c0d0770 2024-04-25T10:13:16.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81a7fc640 1 -- 172.21.15.133:0/1006441018 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8100ab400 con 0x7fe81c0d0770 2024-04-25T10:13:16.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.384+0000 7fe81a7fc640 1 -- 172.21.15.133:0/1006441018 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c001a00 con 0x7fe81c0d0770 2024-04-25T10:13:16.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 -- 172.21.15.133:0/1006441018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c0d0770 msgr2=0x7fe81c0d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 --2- 172.21.15.133:0/1006441018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c0d0770 0x7fe81c0d0bd0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0b5a00 tx=0x7fe81001c370 comp rx=0 tx=0).stop 2024-04-25T10:13:16.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 -- 172.21.15.133:0/1006441018 shutdown_connections 2024-04-25T10:13:16.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 --2- 172.21.15.133:0/1006441018 >> v2:172.21.15.162:3300/0 conn(0x7fe81c0b87e0 0x7fe81c0b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 --2- 172.21.15.133:0/1006441018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c0d0770 0x7fe81c0d0bd0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 -- 172.21.15.133:0/1006441018 >> 172.21.15.133:0/1006441018 conn(0x7fe81c0727c0 msgr2=0x7fe81c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 -- 172.21.15.133:0/1006441018 shutdown_connections 2024-04-25T10:13:16.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 -- 172.21.15.133:0/1006441018 wait complete. 2024-04-25T10:13:16.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 Processor -- start 2024-04-25T10:13:16.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 -- start start 2024-04-25T10:13:16.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe81c0b87e0 0x7fe81c15edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c0d0770 0x7fe81c155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe81c15db00 0x7fe81c153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c0b7430 con 0x7fe81c0d0770 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c0b82c0 con 0x7fe81c0b87e0 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c0d18a0 con 0x7fe81c15db00 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe81affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c0d0770 0x7fe81c155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe81affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c0d0770 0x7fe81c155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34500/0 (socket says 172.21.15.133:34500) 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe81affd640 1 -- 172.21.15.133:0/306135186 learned_addr learned my addr 172.21.15.133:0/306135186 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe81affd640 1 -- 172.21.15.133:0/306135186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe81c15db00 msgr2=0x7fe81c153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe81affd640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe81c15db00 0x7fe81c153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe81affd640 1 -- 172.21.15.133:0/306135186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe81c0b87e0 msgr2=0x7fe81c15edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe81b7fe640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe81c0b87e0 0x7fe81c15edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe81affd640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe81c0b87e0 0x7fe81c15edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe81affd640 1 -- 172.21.15.133:0/306135186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c15e1c0 con 0x7fe81c0d0770 2024-04-25T10:13:16.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe81affd640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c0d0770 0x7fe81c155900 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe81001aa20 tx=0x7fe81001abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe803fff640 1 -- 172.21.15.133:0/306135186 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8100b0050 con 0x7fe81c0d0770 2024-04-25T10:13:16.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe803fff640 1 -- 172.21.15.133:0/306135186 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8100bc070 con 0x7fe81c0d0770 2024-04-25T10:13:16.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 -- 172.21.15.133:0/306135186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c15fb80 con 0x7fe81c0d0770 2024-04-25T10:13:16.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe803fff640 1 -- 172.21.15.133:0/306135186 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8100b78b0 con 0x7fe81c0d0770 2024-04-25T10:13:16.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.388+0000 7fe821d3e640 1 -- 172.21.15.133:0/306135186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c160140 con 0x7fe81c0d0770 2024-04-25T10:13:16.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.392+0000 7fe821d3e640 1 -- 172.21.15.133:0/306135186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c068ef0 con 0x7fe81c0d0770 2024-04-25T10:13:16.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.392+0000 7fe803fff640 1 -- 172.21.15.133:0/306135186 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8100b7a50 con 0x7fe81c0d0770 2024-04-25T10:13:16.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.392+0000 7fe803fff640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe808066f60 0x7fe808069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.392+0000 7fe81b7fe640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe808066f60 0x7fe808069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.392+0000 7fe803fff640 1 -- 172.21.15.133:0/306135186 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe810120420 con 0x7fe81c0d0770 2024-04-25T10:13:16.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.392+0000 7fe81b7fe640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe808066f60 0x7fe808069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe80c005c10 tx=0x7fe80c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.396+0000 7fe803fff640 1 -- 172.21.15.133:0/306135186 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe810015f90 con 0x7fe81c0d0770 2024-04-25T10:13:16.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.600+0000 7fe821d3e640 1 -- 172.21.15.133:0/306135186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe81c0b2750 con 0x7fe81c0d0770 2024-04-25T10:13:16.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.600+0000 7fe803fff640 1 -- 172.21.15.133:0/306135186 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1563 (secure 0 0 0) 0x7fe8100f1410 con 0x7fe81c0d0770 2024-04-25T10:13:16.603 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:16.603 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":709,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":508715008,"bytes_avail":772585398272,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":79,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9495}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:12:51.919939+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:13:16.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 -- 172.21.15.133:0/306135186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe808066f60 msgr2=0x7fe808069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe808066f60 0x7fe808069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe80c005c10 tx=0x7fe80c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:16.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 -- 172.21.15.133:0/306135186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c0d0770 msgr2=0x7fe81c155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c0d0770 0x7fe81c155900 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe81001aa20 tx=0x7fe81001abb0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 -- 172.21.15.133:0/306135186 shutdown_connections 2024-04-25T10:13:16.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe81c15db00 0x7fe81c153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe808066f60 0x7fe808069420 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe81c0d0770 0x7fe81c155900 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 --2- 172.21.15.133:0/306135186 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe81c0b87e0 0x7fe81c15edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 -- 172.21.15.133:0/306135186 >> 172.21.15.133:0/306135186 conn(0x7fe81c0727c0 msgr2=0x7fe81c0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 -- 172.21.15.133:0/306135186 shutdown_connections 2024-04-25T10:13:16.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.604+0000 7fe821d3e640 1 -- 172.21.15.133:0/306135186 wait complete. 2024-04-25T10:13:16.621 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:13:16.621 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:16.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.724+0000 7f8ff0e34640 1 Processor -- start 2024-04-25T10:13:16.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.724+0000 7f8ff0e34640 1 -- start start 2024-04-25T10:13:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8ff0e34640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fec15dd90 0x7f8fec0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8ff0e34640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fec0b19f0 0x7f8fec0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8ff0e34640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec0ca9b0 con 0x7f8fec15d9b0 2024-04-25T10:13:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8ff0e34640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec0cbe30 con 0x7f8fec15dd90 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8ff0e34640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0cac90 con 0x7f8fec0b19f0 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fe9d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fec15dd90 0x7f8fec0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fea59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8fec15d9b0 0x7f8fec0cb5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58540/0 (socket says 172.21.15.133:58540) 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fe9d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fec15dd90 0x7f8fec0ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34512/0 (socket says 172.21.15.133:34512) 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fea59c640 1 -- 172.21.15.133:0/835905230 learned_addr learned my addr 172.21.15.133:0/835905230 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fead9d640 1 --2- 172.21.15.133:0/835905230 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fec0b19f0 0x7f8fec0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fe9d9b640 1 -- 172.21.15.133:0/835905230 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fec0b19f0 msgr2=0x7f8fec0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fe959a640 1 -- 172.21.15.133:0/835905230 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976121834 0 0) 0x7f8fec0ca9b0 con 0x7f8fec15d9b0 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fe9d9b640 1 --2- 172.21.15.133:0/835905230 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fec0b19f0 0x7f8fec0cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fe9d9b640 1 -- 172.21.15.133:0/835905230 >> v1:172.21.15.133:6789/0 conn(0x7f8fec15d9b0 legacy=0x7f8fec0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fe9d9b640 1 -- 172.21.15.133:0/835905230 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec0caf70 con 0x7f8fec15dd90 2024-04-25T10:13:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fead9d640 1 --2- 172.21.15.133:0/835905230 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fec0b19f0 0x7f8fec0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:16.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fe9d9b640 1 --2- 172.21.15.133:0/835905230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fec15dd90 0x7f8fec0ce770 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00097c0 tx=0x7f8fe001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee8254b7a3441ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fe959a640 1 -- 172.21.15.133:0/835905230 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00ab430 con 0x7f8fec15dd90 2024-04-25T10:13:16.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fe959a640 1 -- 172.21.15.133:0/835905230 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe00b0070 con 0x7f8fec15dd90 2024-04-25T10:13:16.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8fe959a640 1 -- 172.21.15.133:0/835905230 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00ab7f0 con 0x7f8fec15dd90 2024-04-25T10:13:16.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8ff0e34640 1 -- 172.21.15.133:0/835905230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fec15dd90 msgr2=0x7f8fec0ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.728+0000 7f8ff0e34640 1 --2- 172.21.15.133:0/835905230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fec15dd90 0x7f8fec0ce770 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00097c0 tx=0x7f8fe001c410 comp rx=0 tx=0).stop 2024-04-25T10:13:16.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 -- 172.21.15.133:0/835905230 shutdown_connections 2024-04-25T10:13:16.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 --2- 172.21.15.133:0/835905230 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fec0b19f0 0x7f8fec0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 --2- 172.21.15.133:0/835905230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fec15dd90 0x7f8fec0ce770 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 -- 172.21.15.133:0/835905230 >> 172.21.15.133:0/835905230 conn(0x7f8fec067000 msgr2=0x7f8fec06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 -- 172.21.15.133:0/835905230 shutdown_connections 2024-04-25T10:13:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 -- 172.21.15.133:0/835905230 wait complete. 2024-04-25T10:13:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 Processor -- start 2024-04-25T10:13:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 -- start start 2024-04-25T10:13:16.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fec0b19f0 0x7f8fec15eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fec15d9b0 0x7f8fec1606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fec15dd90 0x7f8fec14cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec06e290 con 0x7f8fec15d9b0 2024-04-25T10:13:16.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec06e110 con 0x7f8fec0b19f0 2024-04-25T10:13:16.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec06e410 con 0x7f8fec15dd90 2024-04-25T10:13:16.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fe9d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fec15d9b0 0x7f8fec1606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fe9d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fec15d9b0 0x7f8fec1606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34518/0 (socket says 172.21.15.133:34518) 2024-04-25T10:13:16.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fe9d9b640 1 -- 172.21.15.133:0/2276738212 learned_addr learned my addr 172.21.15.133:0/2276738212 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fea59c640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fec0b19f0 0x7f8fec15eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fead9d640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fec15dd90 0x7f8fec14cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fe9d9b640 1 -- 172.21.15.133:0/2276738212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fec15dd90 msgr2=0x7f8fec14cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fe9d9b640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fec15dd90 0x7f8fec14cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fe9d9b640 1 -- 172.21.15.133:0/2276738212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fec0b19f0 msgr2=0x7f8fec15eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fe9d9b640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fec0b19f0 0x7f8fec15eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fe9d9b640 1 -- 172.21.15.133:0/2276738212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec160c30 con 0x7f8fec15d9b0 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fea59c640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fec0b19f0 0x7f8fec15eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fe9d9b640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fec15d9b0 0x7f8fec1606f0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00097c0 tx=0x7f8fe001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fdaffd640 1 -- 172.21.15.133:0/2276738212 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0004020 con 0x7f8fec15d9b0 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fdaffd640 1 -- 172.21.15.133:0/2276738212 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe00041c0 con 0x7f8fec15d9b0 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 -- 172.21.15.133:0/2276738212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec15f1a0 con 0x7f8fec15d9b0 2024-04-25T10:13:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8fdaffd640 1 -- 172.21.15.133:0/2276738212 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00b56d0 con 0x7f8fec15d9b0 2024-04-25T10:13:16.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.732+0000 7f8ff0e34640 1 -- 172.21.15.133:0/2276738212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec15f760 con 0x7f8fec15d9b0 2024-04-25T10:13:16.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.736+0000 7f8ff0e34640 1 -- 172.21.15.133:0/2276738212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec068ef0 con 0x7f8fec15d9b0 2024-04-25T10:13:16.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.736+0000 7f8fdaffd640 1 -- 172.21.15.133:0/2276738212 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fe00bd070 con 0x7f8fec15d9b0 2024-04-25T10:13:16.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.736+0000 7f8fdaffd640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fcc066f60 0x7f8fcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.736+0000 7f8fea59c640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fcc066f60 0x7f8fcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.736+0000 7f8fdaffd640 1 -- 172.21.15.133:0/2276738212 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8fe011f9f0 con 0x7f8fec15d9b0 2024-04-25T10:13:16.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.736+0000 7f8fea59c640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fcc066f60 0x7f8fcc069420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000f70 tx=0x7f8fdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.740+0000 7f8fdaffd640 1 -- 172.21.15.133:0/2276738212 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fe0016090 con 0x7f8fec15d9b0 2024-04-25T10:13:16.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.892+0000 7f8ff0e34640 1 -- 172.21.15.133:0/2276738212 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8fec0cd140 con 0x7f8fcc066f60 2024-04-25T10:13:16.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.900+0000 7f8fdaffd640 1 -- 172.21.15.133:0/2276738212 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7f8fec0cd140 con 0x7f8fcc066f60 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:13:16.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 -- 172.21.15.133:0/2276738212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fcc066f60 msgr2=0x7f8fcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fcc066f60 0x7f8fcc069420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000f70 tx=0x7f8fdc009040 comp rx=0 tx=0).stop 2024-04-25T10:13:16.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 -- 172.21.15.133:0/2276738212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fec15d9b0 msgr2=0x7f8fec1606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fec15d9b0 0x7f8fec1606f0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00097c0 tx=0x7f8fe001ac50 comp rx=0 tx=0).stop 2024-04-25T10:13:16.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 -- 172.21.15.133:0/2276738212 shutdown_connections 2024-04-25T10:13:16.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fec15dd90 0x7f8fec14cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8fcc066f60 0x7f8fcc069420 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fec15d9b0 0x7f8fec1606f0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 --2- 172.21.15.133:0/2276738212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fec0b19f0 0x7f8fec15eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 -- 172.21.15.133:0/2276738212 >> 172.21.15.133:0/2276738212 conn(0x7f8fec067000 msgr2=0x7f8fec0b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 -- 172.21.15.133:0/2276738212 shutdown_connections 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:16.904+0000 7f8ff0e34640 1 -- 172.21.15.133:0/2276738212 wait complete. 2024-04-25T10:13:16.923 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-04-25T10:13:15.925916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":496792,"kb_used_data":14120,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754477928,"statfs":{"total":773094113280,"available":772585398272,"internally_reserved":0,"allocated":14458880,"data_stored":7358172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001598"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759324+0000","last_change":"2024-04-25T10:13:11.518523+0000","last_active":"2024-04-25T10:13:13.759324+0000","last_peered":"2024-04-25T10:13:13.759324+0000","last_clean":"2024-04-25T10:13:13.759324+0000","last_became_active":"2024-04-25T10:13:11.518246+0000","last_became_peered":"2024-04-25T10:13:11.518246+0000","last_unstale":"2024-04-25T10:13:13.759324+0000","last_undegraded":"2024-04-25T10:13:13.759324+0000","last_fullsized":"2024-04-25T10:13:13.759324+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:59.087475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514796+0000","last_change":"2024-04-25T10:13:11.566029+0000","last_active":"2024-04-25T10:13:12.514796+0000","last_peered":"2024-04-25T10:13:12.514796+0000","last_clean":"2024-04-25T10:13:12.514796+0000","last_became_active":"2024-04-25T10:13:11.565395+0000","last_became_peered":"2024-04-25T10:13:11.565395+0000","last_unstale":"2024-04-25T10:13:12.514796+0000","last_undegraded":"2024-04-25T10:13:12.514796+0000","last_fullsized":"2024-04-25T10:13:12.514796+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:46.637364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509560+0000","last_change":"2024-04-25T10:13:11.529012+0000","last_active":"2024-04-25T10:13:12.509560+0000","last_peered":"2024-04-25T10:13:12.509560+0000","last_clean":"2024-04-25T10:13:12.509560+0000","last_became_active":"2024-04-25T10:13:11.527515+0000","last_became_peered":"2024-04-25T10:13:11.527515+0000","last_unstale":"2024-04-25T10:13:12.509560+0000","last_undegraded":"2024-04-25T10:13:12.509560+0000","last_fullsized":"2024-04-25T10:13:12.509560+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:22.965494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514436+0000","last_change":"2024-04-25T10:13:11.551694+0000","last_active":"2024-04-25T10:13:12.514436+0000","last_peered":"2024-04-25T10:13:12.514436+0000","last_clean":"2024-04-25T10:13:12.514436+0000","last_became_active":"2024-04-25T10:13:11.551054+0000","last_became_peered":"2024-04-25T10:13:11.551054+0000","last_unstale":"2024-04-25T10:13:12.514436+0000","last_undegraded":"2024-04-25T10:13:12.514436+0000","last_fullsized":"2024-04-25T10:13:12.514436+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:38.132411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760155+0000","last_change":"2024-04-25T10:13:11.524769+0000","last_active":"2024-04-25T10:13:13.760155+0000","last_peered":"2024-04-25T10:13:13.760155+0000","last_clean":"2024-04-25T10:13:13.760155+0000","last_became_active":"2024-04-25T10:13:11.524473+0000","last_became_peered":"2024-04-25T10:13:11.524473+0000","last_unstale":"2024-04-25T10:13:13.760155+0000","last_undegraded":"2024-04-25T10:13:13.760155+0000","last_fullsized":"2024-04-25T10:13:13.760155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:52.612669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810454+0000","last_change":"2024-04-25T10:13:11.542185+0000","last_active":"2024-04-25T10:13:12.810454+0000","last_peered":"2024-04-25T10:13:12.810454+0000","last_clean":"2024-04-25T10:13:12.810454+0000","last_became_active":"2024-04-25T10:13:11.541886+0000","last_became_peered":"2024-04-25T10:13:11.541886+0000","last_unstale":"2024-04-25T10:13:12.810454+0000","last_undegraded":"2024-04-25T10:13:12.810454+0000","last_fullsized":"2024-04-25T10:13:12.810454+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:58.188235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780810+0000","last_change":"2024-04-25T10:13:11.564656+0000","last_active":"2024-04-25T10:13:12.780810+0000","last_peered":"2024-04-25T10:13:12.780810+0000","last_clean":"2024-04-25T10:13:12.780810+0000","last_became_active":"2024-04-25T10:13:11.564126+0000","last_became_peered":"2024-04-25T10:13:11.564126+0000","last_unstale":"2024-04-25T10:13:12.780810+0000","last_undegraded":"2024-04-25T10:13:12.780810+0000","last_fullsized":"2024-04-25T10:13:12.780810+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:07.102379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507173+0000","last_change":"2024-04-25T10:13:11.519215+0000","last_active":"2024-04-25T10:13:12.507173+0000","last_peered":"2024-04-25T10:13:12.507173+0000","last_clean":"2024-04-25T10:13:12.507173+0000","last_became_active":"2024-04-25T10:13:11.518497+0000","last_became_peered":"2024-04-25T10:13:11.518497+0000","last_unstale":"2024-04-25T10:13:12.507173+0000","last_undegraded":"2024-04-25T10:13:12.507173+0000","last_fullsized":"2024-04-25T10:13:12.507173+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:33.385792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.811190+0000","last_change":"2024-04-25T10:13:11.575005+0000","last_active":"2024-04-25T10:13:12.811190+0000","last_peered":"2024-04-25T10:13:12.811190+0000","last_clean":"2024-04-25T10:13:12.811190+0000","last_became_active":"2024-04-25T10:13:11.574701+0000","last_became_peered":"2024-04-25T10:13:11.574701+0000","last_unstale":"2024-04-25T10:13:12.811190+0000","last_undegraded":"2024-04-25T10:13:12.811190+0000","last_fullsized":"2024-04-25T10:13:12.811190+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:39.559386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506878+0000","last_change":"2024-04-25T10:13:11.523813+0000","last_active":"2024-04-25T10:13:12.506878+0000","last_peered":"2024-04-25T10:13:12.506878+0000","last_clean":"2024-04-25T10:13:12.506878+0000","last_became_active":"2024-04-25T10:13:11.523508+0000","last_became_peered":"2024-04-25T10:13:11.523508+0000","last_unstale":"2024-04-25T10:13:12.506878+0000","last_undegraded":"2024-04-25T10:13:12.506878+0000","last_fullsized":"2024-04-25T10:13:12.506878+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:56.849747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514527+0000","last_change":"2024-04-25T10:13:11.564687+0000","last_active":"2024-04-25T10:13:12.514527+0000","last_peered":"2024-04-25T10:13:12.514527+0000","last_clean":"2024-04-25T10:13:12.514527+0000","last_became_active":"2024-04-25T10:13:11.564124+0000","last_became_peered":"2024-04-25T10:13:11.564124+0000","last_unstale":"2024-04-25T10:13:12.514527+0000","last_undegraded":"2024-04-25T10:13:12.514527+0000","last_fullsized":"2024-04-25T10:13:12.514527+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:10.162659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759606+0000","last_change":"2024-04-25T10:13:11.518996+0000","last_active":"2024-04-25T10:13:13.759606+0000","last_peered":"2024-04-25T10:13:13.759606+0000","last_clean":"2024-04-25T10:13:13.759606+0000","last_became_active":"2024-04-25T10:13:11.518243+0000","last_became_peered":"2024-04-25T10:13:11.518243+0000","last_unstale":"2024-04-25T10:13:13.759606+0000","last_undegraded":"2024-04-25T10:13:13.759606+0000","last_fullsized":"2024-04-25T10:13:13.759606+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:18.646411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810982+0000","last_change":"2024-04-25T10:13:11.546228+0000","last_active":"2024-04-25T10:13:12.810982+0000","last_peered":"2024-04-25T10:13:12.810982+0000","last_clean":"2024-04-25T10:13:12.810982+0000","last_became_active":"2024-04-25T10:13:11.545888+0000","last_became_peered":"2024-04-25T10:13:11.545888+0000","last_unstale":"2024-04-25T10:13:12.810982+0000","last_undegraded":"2024-04-25T10:13:12.810982+0000","last_fullsized":"2024-04-25T10:13:12.810982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:52.612669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759949+0000","last_change":"2024-04-25T10:13:11.556074+0000","last_active":"2024-04-25T10:13:13.759949+0000","last_peered":"2024-04-25T10:13:13.759949+0000","last_clean":"2024-04-25T10:13:13.759949+0000","last_became_active":"2024-04-25T10:13:11.555629+0000","last_became_peered":"2024-04-25T10:13:11.555629+0000","last_unstale":"2024-04-25T10:13:13.759949+0000","last_undegraded":"2024-04-25T10:13:13.759949+0000","last_fullsized":"2024-04-25T10:13:13.759949+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:47.123922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.758838+0000","last_change":"2024-04-25T10:13:11.526526+0000","last_active":"2024-04-25T10:13:13.758838+0000","last_peered":"2024-04-25T10:13:13.758838+0000","last_clean":"2024-04-25T10:13:13.758838+0000","last_became_active":"2024-04-25T10:13:11.526250+0000","last_became_peered":"2024-04-25T10:13:11.526250+0000","last_unstale":"2024-04-25T10:13:13.758838+0000","last_undegraded":"2024-04-25T10:13:13.758838+0000","last_fullsized":"2024-04-25T10:13:13.758838+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:12.458362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.809981+0000","last_change":"2024-04-25T10:13:11.522012+0000","last_active":"2024-04-25T10:13:12.809981+0000","last_peered":"2024-04-25T10:13:12.809981+0000","last_clean":"2024-04-25T10:13:12.809981+0000","last_became_active":"2024-04-25T10:13:11.521053+0000","last_became_peered":"2024-04-25T10:13:11.521053+0000","last_unstale":"2024-04-25T10:13:12.809981+0000","last_undegraded":"2024-04-25T10:13:12.809981+0000","last_fullsized":"2024-04-25T10:13:12.809981+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:38.132411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.811195+0000","last_change":"2024-04-25T10:13:11.521369+0000","last_active":"2024-04-25T10:13:12.811195+0000","last_peered":"2024-04-25T10:13:12.811195+0000","last_clean":"2024-04-25T10:13:12.811195+0000","last_became_active":"2024-04-25T10:13:11.520948+0000","last_became_peered":"2024-04-25T10:13:11.520948+0000","last_unstale":"2024-04-25T10:13:12.811195+0000","last_undegraded":"2024-04-25T10:13:12.811195+0000","last_fullsized":"2024-04-25T10:13:12.811195+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:43.271736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760046+0000","last_change":"2024-04-25T10:13:11.525396+0000","last_active":"2024-04-25T10:13:13.760046+0000","last_peered":"2024-04-25T10:13:13.760046+0000","last_clean":"2024-04-25T10:13:13.760046+0000","last_became_active":"2024-04-25T10:13:11.525018+0000","last_became_peered":"2024-04-25T10:13:11.525018+0000","last_unstale":"2024-04-25T10:13:13.760046+0000","last_undegraded":"2024-04-25T10:13:13.760046+0000","last_fullsized":"2024-04-25T10:13:13.760046+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:46.637364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781628+0000","last_change":"2024-04-25T10:13:11.566279+0000","last_active":"2024-04-25T10:13:12.781628+0000","last_peered":"2024-04-25T10:13:12.781628+0000","last_clean":"2024-04-25T10:13:12.781628+0000","last_became_active":"2024-04-25T10:13:11.562536+0000","last_became_peered":"2024-04-25T10:13:11.562536+0000","last_unstale":"2024-04-25T10:13:12.781628+0000","last_undegraded":"2024-04-25T10:13:12.781628+0000","last_fullsized":"2024-04-25T10:13:12.781628+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:47.607552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761069+0000","last_change":"2024-04-25T10:13:11.525258+0000","last_active":"2024-04-25T10:13:13.761069+0000","last_peered":"2024-04-25T10:13:13.761069+0000","last_clean":"2024-04-25T10:13:13.761069+0000","last_became_active":"2024-04-25T10:13:11.524739+0000","last_became_peered":"2024-04-25T10:13:11.524739+0000","last_unstale":"2024-04-25T10:13:13.761069+0000","last_undegraded":"2024-04-25T10:13:13.761069+0000","last_fullsized":"2024-04-25T10:13:13.761069+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:52.434397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514594+0000","last_change":"2024-04-25T10:13:11.564376+0000","last_active":"2024-04-25T10:13:12.514594+0000","last_peered":"2024-04-25T10:13:12.514594+0000","last_clean":"2024-04-25T10:13:12.514594+0000","last_became_active":"2024-04-25T10:13:11.563826+0000","last_became_peered":"2024-04-25T10:13:11.563826+0000","last_unstale":"2024-04-25T10:13:12.514594+0000","last_undegraded":"2024-04-25T10:13:12.514594+0000","last_fullsized":"2024-04-25T10:13:12.514594+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:58.188235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759164+0000","last_change":"2024-04-25T10:13:11.567489+0000","last_active":"2024-04-25T10:13:13.759164+0000","last_peered":"2024-04-25T10:13:13.759164+0000","last_clean":"2024-04-25T10:13:13.759164+0000","last_became_active":"2024-04-25T10:13:11.567145+0000","last_became_peered":"2024-04-25T10:13:11.567145+0000","last_unstale":"2024-04-25T10:13:13.759164+0000","last_undegraded":"2024-04-25T10:13:13.759164+0000","last_fullsized":"2024-04-25T10:13:13.759164+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:37.208933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514296+0000","last_change":"2024-04-25T10:13:11.556816+0000","last_active":"2024-04-25T10:13:12.514296+0000","last_peered":"2024-04-25T10:13:12.514296+0000","last_clean":"2024-04-25T10:13:12.514296+0000","last_became_active":"2024-04-25T10:13:11.556288+0000","last_became_peered":"2024-04-25T10:13:11.556288+0000","last_unstale":"2024-04-25T10:13:12.514296+0000","last_undegraded":"2024-04-25T10:13:12.514296+0000","last_fullsized":"2024-04-25T10:13:12.514296+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:36.478303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810170+0000","last_change":"2024-04-25T10:13:11.567581+0000","last_active":"2024-04-25T10:13:12.810170+0000","last_peered":"2024-04-25T10:13:12.810170+0000","last_clean":"2024-04-25T10:13:12.810170+0000","last_became_active":"2024-04-25T10:13:11.567284+0000","last_became_peered":"2024-04-25T10:13:11.567284+0000","last_unstale":"2024-04-25T10:13:12.810170+0000","last_undegraded":"2024-04-25T10:13:12.810170+0000","last_fullsized":"2024-04-25T10:13:12.810170+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:10.162659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510104+0000","last_change":"2024-04-25T10:13:11.562038+0000","last_active":"2024-04-25T10:13:12.510104+0000","last_peered":"2024-04-25T10:13:12.510104+0000","last_clean":"2024-04-25T10:13:12.510104+0000","last_became_active":"2024-04-25T10:13:11.555255+0000","last_became_peered":"2024-04-25T10:13:11.555255+0000","last_unstale":"2024-04-25T10:13:12.510104+0000","last_undegraded":"2024-04-25T10:13:12.510104+0000","last_fullsized":"2024-04-25T10:13:12.510104+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:53.723716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509065+0000","last_change":"2024-04-25T10:13:11.564767+0000","last_active":"2024-04-25T10:13:12.509065+0000","last_peered":"2024-04-25T10:13:12.509065+0000","last_clean":"2024-04-25T10:13:12.509065+0000","last_became_active":"2024-04-25T10:13:11.564094+0000","last_became_peered":"2024-04-25T10:13:11.564094+0000","last_unstale":"2024-04-25T10:13:12.509065+0000","last_undegraded":"2024-04-25T10:13:12.509065+0000","last_fullsized":"2024-04-25T10:13:12.509065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.305600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781823+0000","last_change":"2024-04-25T10:13:11.570693+0000","last_active":"2024-04-25T10:13:12.781823+0000","last_peered":"2024-04-25T10:13:12.781823+0000","last_clean":"2024-04-25T10:13:12.781823+0000","last_became_active":"2024-04-25T10:13:11.563159+0000","last_became_peered":"2024-04-25T10:13:11.563159+0000","last_unstale":"2024-04-25T10:13:12.781823+0000","last_undegraded":"2024-04-25T10:13:12.781823+0000","last_fullsized":"2024-04-25T10:13:12.781823+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:27.760139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506676+0000","last_change":"2024-04-25T10:13:11.524297+0000","last_active":"2024-04-25T10:13:12.506676+0000","last_peered":"2024-04-25T10:13:12.506676+0000","last_clean":"2024-04-25T10:13:12.506676+0000","last_became_active":"2024-04-25T10:13:11.523906+0000","last_became_peered":"2024-04-25T10:13:11.523906+0000","last_unstale":"2024-04-25T10:13:12.506676+0000","last_undegraded":"2024-04-25T10:13:12.506676+0000","last_fullsized":"2024-04-25T10:13:12.506676+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:42.816079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508821+0000","last_change":"2024-04-25T10:13:11.552751+0000","last_active":"2024-04-25T10:13:12.508821+0000","last_peered":"2024-04-25T10:13:12.508821+0000","last_clean":"2024-04-25T10:13:12.508821+0000","last_became_active":"2024-04-25T10:13:11.551644+0000","last_became_peered":"2024-04-25T10:13:11.551644+0000","last_unstale":"2024-04-25T10:13:12.508821+0000","last_undegraded":"2024-04-25T10:13:12.508821+0000","last_fullsized":"2024-04-25T10:13:12.508821+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:33.385792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509621+0000","last_change":"2024-04-25T10:13:11.558991+0000","last_active":"2024-04-25T10:13:12.509621+0000","last_peered":"2024-04-25T10:13:12.509621+0000","last_clean":"2024-04-25T10:13:12.509621+0000","last_became_active":"2024-04-25T10:13:11.553029+0000","last_became_peered":"2024-04-25T10:13:11.553029+0000","last_unstale":"2024-04-25T10:13:12.509621+0000","last_undegraded":"2024-04-25T10:13:12.509621+0000","last_fullsized":"2024-04-25T10:13:12.509621+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:49.372602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507272+0000","last_change":"2024-04-25T10:13:11.525057+0000","last_active":"2024-04-25T10:13:12.507272+0000","last_peered":"2024-04-25T10:13:12.507272+0000","last_clean":"2024-04-25T10:13:12.507272+0000","last_became_active":"2024-04-25T10:13:11.522942+0000","last_became_peered":"2024-04-25T10:13:11.522942+0000","last_unstale":"2024-04-25T10:13:12.507272+0000","last_undegraded":"2024-04-25T10:13:12.507272+0000","last_fullsized":"2024-04-25T10:13:12.507272+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.305600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506251+0000","last_change":"2024-04-25T10:13:11.522239+0000","last_active":"2024-04-25T10:13:12.506251+0000","last_peered":"2024-04-25T10:13:12.506251+0000","last_clean":"2024-04-25T10:13:12.506251+0000","last_became_active":"2024-04-25T10:13:11.521940+0000","last_became_peered":"2024-04-25T10:13:11.521940+0000","last_unstale":"2024-04-25T10:13:12.506251+0000","last_undegraded":"2024-04-25T10:13:12.506251+0000","last_fullsized":"2024-04-25T10:13:12.506251+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:42.493948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507440+0000","last_change":"2024-04-25T10:13:11.524308+0000","last_active":"2024-04-25T10:13:12.507440+0000","last_peered":"2024-04-25T10:13:12.507440+0000","last_clean":"2024-04-25T10:13:12.507440+0000","last_became_active":"2024-04-25T10:13:11.523488+0000","last_became_peered":"2024-04-25T10:13:11.523488+0000","last_unstale":"2024-04-25T10:13:12.507440+0000","last_undegraded":"2024-04-25T10:13:12.507440+0000","last_fullsized":"2024-04-25T10:13:12.507440+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:38.172285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"131'4","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510113+0000","last_change":"2024-04-25T10:13:10.509075+0000","last_active":"2024-04-25T10:13:12.510113+0000","last_peered":"2024-04-25T10:13:12.510113+0000","last_clean":"2024-04-25T10:13:12.510113+0000","last_became_active":"2024-04-25T10:13:10.506901+0000","last_became_peered":"2024-04-25T10:13:10.506901+0000","last_unstale":"2024-04-25T10:13:12.510113+0000","last_undegraded":"2024-04-25T10:13:12.510113+0000","last_fullsized":"2024-04-25T10:13:12.510113+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:43.483874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759499+0000","last_change":"2024-04-25T10:13:11.567350+0000","last_active":"2024-04-25T10:13:13.759499+0000","last_peered":"2024-04-25T10:13:13.759499+0000","last_clean":"2024-04-25T10:13:13.759499+0000","last_became_active":"2024-04-25T10:13:11.567065+0000","last_became_peered":"2024-04-25T10:13:11.567065+0000","last_unstale":"2024-04-25T10:13:13.759499+0000","last_undegraded":"2024-04-25T10:13:13.759499+0000","last_fullsized":"2024-04-25T10:13:13.759499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:14.942690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760310+0000","last_change":"2024-04-25T10:13:10.504453+0000","last_active":"2024-04-25T10:13:13.760310+0000","last_peered":"2024-04-25T10:13:13.760310+0000","last_clean":"2024-04-25T10:13:13.760310+0000","last_became_active":"2024-04-25T10:13:10.504177+0000","last_became_peered":"2024-04-25T10:13:10.504177+0000","last_unstale":"2024-04-25T10:13:13.760310+0000","last_undegraded":"2024-04-25T10:13:13.760310+0000","last_fullsized":"2024-04-25T10:13:13.760310+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:47.372814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514629+0000","last_change":"2024-04-25T10:13:11.573265+0000","last_active":"2024-04-25T10:13:12.514629+0000","last_peered":"2024-04-25T10:13:12.514629+0000","last_clean":"2024-04-25T10:13:12.514629+0000","last_became_active":"2024-04-25T10:13:11.568213+0000","last_became_peered":"2024-04-25T10:13:11.568213+0000","last_unstale":"2024-04-25T10:13:12.514629+0000","last_undegraded":"2024-04-25T10:13:12.514629+0000","last_fullsized":"2024-04-25T10:13:12.514629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:10.130196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781827+0000","last_change":"2024-04-25T10:13:10.520416+0000","last_active":"2024-04-25T10:13:12.781827+0000","last_peered":"2024-04-25T10:13:12.781827+0000","last_clean":"2024-04-25T10:13:12.781827+0000","last_became_active":"2024-04-25T10:13:10.519694+0000","last_became_peered":"2024-04-25T10:13:10.519694+0000","last_unstale":"2024-04-25T10:13:12.781827+0000","last_undegraded":"2024-04-25T10:13:12.781827+0000","last_fullsized":"2024-04-25T10:13:12.781827+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:33.252824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759314+0000","last_change":"2024-04-25T10:13:11.526936+0000","last_active":"2024-04-25T10:13:13.759314+0000","last_peered":"2024-04-25T10:13:13.759314+0000","last_clean":"2024-04-25T10:13:13.759314+0000","last_became_active":"2024-04-25T10:13:11.526618+0000","last_became_peered":"2024-04-25T10:13:11.526618+0000","last_unstale":"2024-04-25T10:13:13.759314+0000","last_undegraded":"2024-04-25T10:13:13.759314+0000","last_fullsized":"2024-04-25T10:13:13.759314+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:20.954445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781022+0000","last_change":"2024-04-25T10:13:10.519322+0000","last_active":"2024-04-25T10:13:12.781022+0000","last_peered":"2024-04-25T10:13:12.781022+0000","last_clean":"2024-04-25T10:13:12.781022+0000","last_became_active":"2024-04-25T10:13:10.518685+0000","last_became_peered":"2024-04-25T10:13:10.518685+0000","last_unstale":"2024-04-25T10:13:12.781022+0000","last_undegraded":"2024-04-25T10:13:12.781022+0000","last_fullsized":"2024-04-25T10:13:12.781022+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:03.662949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510070+0000","last_change":"2024-04-25T10:13:11.523386+0000","last_active":"2024-04-25T10:13:12.510070+0000","last_peered":"2024-04-25T10:13:12.510070+0000","last_clean":"2024-04-25T10:13:12.510070+0000","last_became_active":"2024-04-25T10:13:11.522629+0000","last_became_peered":"2024-04-25T10:13:11.522629+0000","last_unstale":"2024-04-25T10:13:12.510070+0000","last_undegraded":"2024-04-25T10:13:12.510070+0000","last_fullsized":"2024-04-25T10:13:12.510070+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:49.372602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507928+0000","last_change":"2024-04-25T10:13:10.499060+0000","last_active":"2024-04-25T10:13:12.507928+0000","last_peered":"2024-04-25T10:13:12.507928+0000","last_clean":"2024-04-25T10:13:12.507928+0000","last_became_active":"2024-04-25T10:13:10.494838+0000","last_became_peered":"2024-04-25T10:13:10.494838+0000","last_unstale":"2024-04-25T10:13:12.507928+0000","last_undegraded":"2024-04-25T10:13:12.507928+0000","last_fullsized":"2024-04-25T10:13:12.507928+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:51.109162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508381+0000","last_change":"2024-04-25T10:13:11.526929+0000","last_active":"2024-04-25T10:13:12.508381+0000","last_peered":"2024-04-25T10:13:12.508381+0000","last_clean":"2024-04-25T10:13:12.508381+0000","last_became_active":"2024-04-25T10:13:11.526184+0000","last_became_peered":"2024-04-25T10:13:11.526184+0000","last_unstale":"2024-04-25T10:13:12.508381+0000","last_undegraded":"2024-04-25T10:13:12.508381+0000","last_fullsized":"2024-04-25T10:13:12.508381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:22.965494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759768+0000","last_change":"2024-04-25T10:13:10.500210+0000","last_active":"2024-04-25T10:13:13.759768+0000","last_peered":"2024-04-25T10:13:13.759768+0000","last_clean":"2024-04-25T10:13:13.759768+0000","last_became_active":"2024-04-25T10:13:10.499928+0000","last_became_peered":"2024-04-25T10:13:10.499928+0000","last_unstale":"2024-04-25T10:13:13.759768+0000","last_undegraded":"2024-04-25T10:13:13.759768+0000","last_fullsized":"2024-04-25T10:13:13.759768+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:48.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760330+0000","last_change":"2024-04-25T10:13:11.555023+0000","last_active":"2024-04-25T10:13:13.760330+0000","last_peered":"2024-04-25T10:13:13.760330+0000","last_clean":"2024-04-25T10:13:13.760330+0000","last_became_active":"2024-04-25T10:13:11.554728+0000","last_became_peered":"2024-04-25T10:13:11.554728+0000","last_unstale":"2024-04-25T10:13:13.760330+0000","last_undegraded":"2024-04-25T10:13:13.760330+0000","last_fullsized":"2024-04-25T10:13:13.760330+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:39.559386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.809762+0000","last_change":"2024-04-25T10:13:10.504035+0000","last_active":"2024-04-25T10:13:12.809762+0000","last_peered":"2024-04-25T10:13:12.809762+0000","last_clean":"2024-04-25T10:13:12.809762+0000","last_became_active":"2024-04-25T10:13:10.502964+0000","last_became_peered":"2024-04-25T10:13:10.502964+0000","last_unstale":"2024-04-25T10:13:12.809762+0000","last_undegraded":"2024-04-25T10:13:12.809762+0000","last_fullsized":"2024-04-25T10:13:12.809762+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:04.997195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781330+0000","last_change":"2024-04-25T10:13:11.559414+0000","last_active":"2024-04-25T10:13:12.781330+0000","last_peered":"2024-04-25T10:13:12.781330+0000","last_clean":"2024-04-25T10:13:12.781330+0000","last_became_active":"2024-04-25T10:13:11.558823+0000","last_became_peered":"2024-04-25T10:13:11.558823+0000","last_unstale":"2024-04-25T10:13:12.781330+0000","last_undegraded":"2024-04-25T10:13:12.781330+0000","last_fullsized":"2024-04-25T10:13:12.781330+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:06.795281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508653+0000","last_change":"2024-04-25T10:13:10.507874+0000","last_active":"2024-04-25T10:13:12.508653+0000","last_peered":"2024-04-25T10:13:12.508653+0000","last_clean":"2024-04-25T10:13:12.508653+0000","last_became_active":"2024-04-25T10:13:10.507221+0000","last_became_peered":"2024-04-25T10:13:10.507221+0000","last_unstale":"2024-04-25T10:13:12.508653+0000","last_undegraded":"2024-04-25T10:13:12.508653+0000","last_fullsized":"2024-04-25T10:13:12.508653+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:45.472460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.758603+0000","last_change":"2024-04-25T10:13:11.548771+0000","last_active":"2024-04-25T10:13:13.758603+0000","last_peered":"2024-04-25T10:13:13.758603+0000","last_clean":"2024-04-25T10:13:13.758603+0000","last_became_active":"2024-04-25T10:13:11.548373+0000","last_became_peered":"2024-04-25T10:13:11.548373+0000","last_unstale":"2024-04-25T10:13:13.758603+0000","last_undegraded":"2024-04-25T10:13:13.758603+0000","last_fullsized":"2024-04-25T10:13:13.758603+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:59.266348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514921+0000","last_change":"2024-04-25T10:13:10.494868+0000","last_active":"2024-04-25T10:13:12.514921+0000","last_peered":"2024-04-25T10:13:12.514921+0000","last_clean":"2024-04-25T10:13:12.514921+0000","last_became_active":"2024-04-25T10:13:10.494288+0000","last_became_peered":"2024-04-25T10:13:10.494288+0000","last_unstale":"2024-04-25T10:13:12.514921+0000","last_undegraded":"2024-04-25T10:13:12.514921+0000","last_fullsized":"2024-04-25T10:13:12.514921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:37.169182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810475+0000","last_change":"2024-04-25T10:13:11.513098+0000","last_active":"2024-04-25T10:13:12.810475+0000","last_peered":"2024-04-25T10:13:12.810475+0000","last_clean":"2024-04-25T10:13:12.810475+0000","last_became_active":"2024-04-25T10:13:11.512776+0000","last_became_peered":"2024-04-25T10:13:11.512776+0000","last_unstale":"2024-04-25T10:13:12.810475+0000","last_undegraded":"2024-04-25T10:13:12.810475+0000","last_fullsized":"2024-04-25T10:13:12.810475+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:10.130196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759446+0000","last_change":"2024-04-25T10:13:10.497439+0000","last_active":"2024-04-25T10:13:13.759446+0000","last_peered":"2024-04-25T10:13:13.759446+0000","last_clean":"2024-04-25T10:13:13.759446+0000","last_became_active":"2024-04-25T10:13:10.496013+0000","last_became_peered":"2024-04-25T10:13:10.496013+0000","last_unstale":"2024-04-25T10:13:13.759446+0000","last_undegraded":"2024-04-25T10:13:13.759446+0000","last_fullsized":"2024-04-25T10:13:13.759446+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:19.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510044+0000","last_change":"2024-04-25T10:13:11.565816+0000","last_active":"2024-04-25T10:13:12.510044+0000","last_peered":"2024-04-25T10:13:12.510044+0000","last_clean":"2024-04-25T10:13:12.510044+0000","last_became_active":"2024-04-25T10:13:11.565141+0000","last_became_peered":"2024-04-25T10:13:11.565141+0000","last_unstale":"2024-04-25T10:13:12.510044+0000","last_undegraded":"2024-04-25T10:13:12.510044+0000","last_fullsized":"2024-04-25T10:13:12.510044+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.375918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"131'3","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781913+0000","last_change":"2024-04-25T10:13:10.520399+0000","last_active":"2024-04-25T10:13:12.781913+0000","last_peered":"2024-04-25T10:13:12.781913+0000","last_clean":"2024-04-25T10:13:12.781913+0000","last_became_active":"2024-04-25T10:13:10.519892+0000","last_became_peered":"2024-04-25T10:13:10.519892+0000","last_unstale":"2024-04-25T10:13:12.781913+0000","last_undegraded":"2024-04-25T10:13:12.781913+0000","last_fullsized":"2024-04-25T10:13:12.781913+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:21.502204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509200+0000","last_change":"2024-04-25T10:13:11.529758+0000","last_active":"2024-04-25T10:13:12.509200+0000","last_peered":"2024-04-25T10:13:12.509200+0000","last_clean":"2024-04-25T10:13:12.509200+0000","last_became_active":"2024-04-25T10:13:11.528989+0000","last_became_peered":"2024-04-25T10:13:11.528989+0000","last_unstale":"2024-04-25T10:13:12.509200+0000","last_undegraded":"2024-04-25T10:13:12.509200+0000","last_fullsized":"2024-04-25T10:13:12.509200+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:38.172285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"131'6","reported_seq":20,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781106+0000","last_change":"2024-04-25T10:13:10.523732+0000","last_active":"2024-04-25T10:13:12.781106+0000","last_peered":"2024-04-25T10:13:12.781106+0000","last_clean":"2024-04-25T10:13:12.781106+0000","last_became_active":"2024-04-25T10:13:10.523019+0000","last_became_peered":"2024-04-25T10:13:10.523019+0000","last_unstale":"2024-04-25T10:13:12.781106+0000","last_undegraded":"2024-04-25T10:13:12.781106+0000","last_fullsized":"2024-04-25T10:13:12.781106+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:35.743803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507053+0000","last_change":"2024-04-25T10:13:11.520435+0000","last_active":"2024-04-25T10:13:12.507053+0000","last_peered":"2024-04-25T10:13:12.507053+0000","last_clean":"2024-04-25T10:13:12.507053+0000","last_became_active":"2024-04-25T10:13:11.520147+0000","last_became_peered":"2024-04-25T10:13:11.520147+0000","last_unstale":"2024-04-25T10:13:12.507053+0000","last_undegraded":"2024-04-25T10:13:12.507053+0000","last_fullsized":"2024-04-25T10:13:12.507053+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:02.552382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759467+0000","last_change":"2024-04-25T10:13:10.497816+0000","last_active":"2024-04-25T10:13:13.759467+0000","last_peered":"2024-04-25T10:13:13.759467+0000","last_clean":"2024-04-25T10:13:13.759467+0000","last_became_active":"2024-04-25T10:13:10.497547+0000","last_became_peered":"2024-04-25T10:13:10.497547+0000","last_unstale":"2024-04-25T10:13:13.759467+0000","last_undegraded":"2024-04-25T10:13:13.759467+0000","last_fullsized":"2024-04-25T10:13:13.759467+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:07.860557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781622+0000","last_change":"2024-04-25T10:13:11.570950+0000","last_active":"2024-04-25T10:13:12.781622+0000","last_peered":"2024-04-25T10:13:12.781622+0000","last_clean":"2024-04-25T10:13:12.781622+0000","last_became_active":"2024-04-25T10:13:11.570376+0000","last_became_peered":"2024-04-25T10:13:11.570376+0000","last_unstale":"2024-04-25T10:13:12.781622+0000","last_undegraded":"2024-04-25T10:13:12.781622+0000","last_fullsized":"2024-04-25T10:13:12.781622+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:51.592552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508365+0000","last_change":"2024-04-25T10:13:10.507142+0000","last_active":"2024-04-25T10:13:12.508365+0000","last_peered":"2024-04-25T10:13:12.508365+0000","last_clean":"2024-04-25T10:13:12.508365+0000","last_became_active":"2024-04-25T10:13:10.506580+0000","last_became_peered":"2024-04-25T10:13:10.506580+0000","last_unstale":"2024-04-25T10:13:12.508365+0000","last_undegraded":"2024-04-25T10:13:12.508365+0000","last_fullsized":"2024-04-25T10:13:12.508365+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:41.464459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760601+0000","last_change":"2024-04-25T10:13:11.524605+0000","last_active":"2024-04-25T10:13:13.760601+0000","last_peered":"2024-04-25T10:13:13.760601+0000","last_clean":"2024-04-25T10:13:13.760601+0000","last_became_active":"2024-04-25T10:13:11.523134+0000","last_became_peered":"2024-04-25T10:13:11.523134+0000","last_unstale":"2024-04-25T10:13:13.760601+0000","last_undegraded":"2024-04-25T10:13:13.760601+0000","last_fullsized":"2024-04-25T10:13:13.760601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:43.271736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"131'8","reported_seq":22,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810197+0000","last_change":"2024-04-25T10:13:10.503861+0000","last_active":"2024-04-25T10:13:12.810197+0000","last_peered":"2024-04-25T10:13:12.810197+0000","last_clean":"2024-04-25T10:13:12.810197+0000","last_became_active":"2024-04-25T10:13:10.503476+0000","last_became_peered":"2024-04-25T10:13:10.503476+0000","last_unstale":"2024-04-25T10:13:12.810197+0000","last_undegraded":"2024-04-25T10:13:12.810197+0000","last_fullsized":"2024-04-25T10:13:12.810197+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.405882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508510+0000","last_change":"2024-04-25T10:13:11.547495+0000","last_active":"2024-04-25T10:13:12.508510+0000","last_peered":"2024-04-25T10:13:12.508510+0000","last_clean":"2024-04-25T10:13:12.508510+0000","last_became_active":"2024-04-25T10:13:11.541513+0000","last_became_peered":"2024-04-25T10:13:11.541513+0000","last_unstale":"2024-04-25T10:13:12.508510+0000","last_undegraded":"2024-04-25T10:13:12.508510+0000","last_fullsized":"2024-04-25T10:13:12.508510+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:59.408986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514982+0000","last_change":"2024-04-25T10:13:10.501668+0000","last_active":"2024-04-25T10:13:12.514982+0000","last_peered":"2024-04-25T10:13:12.514982+0000","last_clean":"2024-04-25T10:13:12.514982+0000","last_became_active":"2024-04-25T10:13:10.501112+0000","last_became_peered":"2024-04-25T10:13:10.501112+0000","last_unstale":"2024-04-25T10:13:12.514982+0000","last_undegraded":"2024-04-25T10:13:12.514982+0000","last_fullsized":"2024-04-25T10:13:12.514982+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:21.962390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780949+0000","last_change":"2024-04-25T10:13:11.559201+0000","last_active":"2024-04-25T10:13:12.780949+0000","last_peered":"2024-04-25T10:13:12.780949+0000","last_clean":"2024-04-25T10:13:12.780949+0000","last_became_active":"2024-04-25T10:13:11.558584+0000","last_became_peered":"2024-04-25T10:13:11.558584+0000","last_unstale":"2024-04-25T10:13:12.780949+0000","last_undegraded":"2024-04-25T10:13:12.780949+0000","last_fullsized":"2024-04-25T10:13:12.780949+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:47.947133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509197+0000","last_change":"2024-04-25T10:13:10.507714+0000","last_active":"2024-04-25T10:13:12.509197+0000","last_peered":"2024-04-25T10:13:12.509197+0000","last_clean":"2024-04-25T10:13:12.509197+0000","last_became_active":"2024-04-25T10:13:10.506915+0000","last_became_peered":"2024-04-25T10:13:10.506915+0000","last_unstale":"2024-04-25T10:13:12.509197+0000","last_undegraded":"2024-04-25T10:13:12.509197+0000","last_fullsized":"2024-04-25T10:13:12.509197+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:31.176627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781867+0000","last_change":"2024-04-25T10:13:11.559225+0000","last_active":"2024-04-25T10:13:12.781867+0000","last_peered":"2024-04-25T10:13:12.781867+0000","last_clean":"2024-04-25T10:13:12.781867+0000","last_became_active":"2024-04-25T10:13:11.558665+0000","last_became_peered":"2024-04-25T10:13:11.558665+0000","last_unstale":"2024-04-25T10:13:12.781867+0000","last_undegraded":"2024-04-25T10:13:12.781867+0000","last_fullsized":"2024-04-25T10:13:12.781867+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:29.750424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810921+0000","last_change":"2024-04-25T10:13:10.502878+0000","last_active":"2024-04-25T10:13:12.810921+0000","last_peered":"2024-04-25T10:13:12.810921+0000","last_clean":"2024-04-25T10:13:12.810921+0000","last_became_active":"2024-04-25T10:13:10.501740+0000","last_became_peered":"2024-04-25T10:13:10.501740+0000","last_unstale":"2024-04-25T10:13:12.810921+0000","last_undegraded":"2024-04-25T10:13:12.810921+0000","last_fullsized":"2024-04-25T10:13:12.810921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:48.234733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506278+0000","last_change":"2024-04-25T10:13:11.548117+0000","last_active":"2024-04-25T10:13:12.506278+0000","last_peered":"2024-04-25T10:13:12.506278+0000","last_clean":"2024-04-25T10:13:12.506278+0000","last_became_active":"2024-04-25T10:13:11.547691+0000","last_became_peered":"2024-04-25T10:13:11.547691+0000","last_unstale":"2024-04-25T10:13:12.506278+0000","last_undegraded":"2024-04-25T10:13:12.506278+0000","last_fullsized":"2024-04-25T10:13:12.506278+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:27.760139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761144+0000","last_change":"2024-04-25T10:13:10.499039+0000","last_active":"2024-04-25T10:13:13.761144+0000","last_peered":"2024-04-25T10:13:13.761144+0000","last_clean":"2024-04-25T10:13:13.761144+0000","last_became_active":"2024-04-25T10:13:10.498545+0000","last_became_peered":"2024-04-25T10:13:10.498545+0000","last_unstale":"2024-04-25T10:13:13.761144+0000","last_undegraded":"2024-04-25T10:13:13.761144+0000","last_fullsized":"2024-04-25T10:13:13.761144+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:52.720613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761304+0000","last_change":"2024-04-25T10:13:11.518690+0000","last_active":"2024-04-25T10:13:13.761304+0000","last_peered":"2024-04-25T10:13:13.761304+0000","last_clean":"2024-04-25T10:13:13.761304+0000","last_became_active":"2024-04-25T10:13:11.518411+0000","last_became_peered":"2024-04-25T10:13:11.518411+0000","last_unstale":"2024-04-25T10:13:13.761304+0000","last_undegraded":"2024-04-25T10:13:13.761304+0000","last_fullsized":"2024-04-25T10:13:13.761304+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:09.135552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508272+0000","last_change":"2024-04-25T10:13:10.504158+0000","last_active":"2024-04-25T10:13:12.508272+0000","last_peered":"2024-04-25T10:13:12.508272+0000","last_clean":"2024-04-25T10:13:12.508272+0000","last_became_active":"2024-04-25T10:13:10.502316+0000","last_became_peered":"2024-04-25T10:13:10.502316+0000","last_unstale":"2024-04-25T10:13:12.508272+0000","last_undegraded":"2024-04-25T10:13:12.508272+0000","last_fullsized":"2024-04-25T10:13:12.508272+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:39.042840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759831+0000","last_change":"2024-04-25T10:13:11.562497+0000","last_active":"2024-04-25T10:13:13.759831+0000","last_peered":"2024-04-25T10:13:13.759831+0000","last_clean":"2024-04-25T10:13:13.759831+0000","last_became_active":"2024-04-25T10:13:11.562192+0000","last_became_peered":"2024-04-25T10:13:11.562192+0000","last_unstale":"2024-04-25T10:13:13.759831+0000","last_undegraded":"2024-04-25T10:13:13.759831+0000","last_fullsized":"2024-04-25T10:13:13.759831+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:27.488802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760075+0000","last_change":"2024-04-25T10:13:10.500218+0000","last_active":"2024-04-25T10:13:13.760075+0000","last_peered":"2024-04-25T10:13:13.760075+0000","last_clean":"2024-04-25T10:13:13.760075+0000","last_became_active":"2024-04-25T10:13:10.499839+0000","last_became_peered":"2024-04-25T10:13:10.499839+0000","last_unstale":"2024-04-25T10:13:13.760075+0000","last_undegraded":"2024-04-25T10:13:13.760075+0000","last_fullsized":"2024-04-25T10:13:13.760075+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:15.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760253+0000","last_change":"2024-04-25T10:13:11.547740+0000","last_active":"2024-04-25T10:13:13.760253+0000","last_peered":"2024-04-25T10:13:13.760253+0000","last_clean":"2024-04-25T10:13:13.760253+0000","last_became_active":"2024-04-25T10:13:11.547457+0000","last_became_peered":"2024-04-25T10:13:11.547457+0000","last_unstale":"2024-04-25T10:13:13.760253+0000","last_undegraded":"2024-04-25T10:13:13.760253+0000","last_fullsized":"2024-04-25T10:13:13.760253+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:10.791929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781126+0000","last_change":"2024-04-25T10:13:10.523607+0000","last_active":"2024-04-25T10:13:12.781126+0000","last_peered":"2024-04-25T10:13:12.781126+0000","last_clean":"2024-04-25T10:13:12.781126+0000","last_became_active":"2024-04-25T10:13:10.522750+0000","last_became_peered":"2024-04-25T10:13:10.522750+0000","last_unstale":"2024-04-25T10:13:12.781126+0000","last_undegraded":"2024-04-25T10:13:12.781126+0000","last_fullsized":"2024-04-25T10:13:12.781126+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:37.642788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514651+0000","last_change":"2024-04-25T10:13:11.574680+0000","last_active":"2024-04-25T10:13:12.514651+0000","last_peered":"2024-04-25T10:13:12.514651+0000","last_clean":"2024-04-25T10:13:12.514651+0000","last_became_active":"2024-04-25T10:13:11.574085+0000","last_became_peered":"2024-04-25T10:13:11.574085+0000","last_unstale":"2024-04-25T10:13:12.514651+0000","last_undegraded":"2024-04-25T10:13:12.514651+0000","last_fullsized":"2024-04-25T10:13:12.514651+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.168445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810338+0000","last_change":"2024-04-25T10:13:10.496461+0000","last_active":"2024-04-25T10:13:12.810338+0000","last_peered":"2024-04-25T10:13:12.810338+0000","last_clean":"2024-04-25T10:13:12.810338+0000","last_became_active":"2024-04-25T10:13:10.495439+0000","last_became_peered":"2024-04-25T10:13:10.495439+0000","last_unstale":"2024-04-25T10:13:12.810338+0000","last_undegraded":"2024-04-25T10:13:12.810338+0000","last_fullsized":"2024-04-25T10:13:12.810338+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:32.382689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761590+0000","last_change":"2024-04-25T10:13:11.525150+0000","last_active":"2024-04-25T10:13:13.761590+0000","last_peered":"2024-04-25T10:13:13.761590+0000","last_clean":"2024-04-25T10:13:13.761590+0000","last_became_active":"2024-04-25T10:13:11.524852+0000","last_became_peered":"2024-04-25T10:13:11.524852+0000","last_unstale":"2024-04-25T10:13:13.761590+0000","last_undegraded":"2024-04-25T10:13:13.761590+0000","last_fullsized":"2024-04-25T10:13:13.761590+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:36.746906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759373+0000","last_change":"2024-04-25T10:13:10.504888+0000","last_active":"2024-04-25T10:13:13.759373+0000","last_peered":"2024-04-25T10:13:13.759373+0000","last_clean":"2024-04-25T10:13:13.759373+0000","last_became_active":"2024-04-25T10:13:10.504352+0000","last_became_peered":"2024-04-25T10:13:10.504352+0000","last_unstale":"2024-04-25T10:13:13.759373+0000","last_undegraded":"2024-04-25T10:13:13.759373+0000","last_fullsized":"2024-04-25T10:13:13.759373+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:51.056571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508240+0000","last_change":"2024-04-25T10:13:11.541243+0000","last_active":"2024-04-25T10:13:12.508240+0000","last_peered":"2024-04-25T10:13:12.508240+0000","last_clean":"2024-04-25T10:13:12.508240+0000","last_became_active":"2024-04-25T10:13:11.540480+0000","last_became_peered":"2024-04-25T10:13:11.540480+0000","last_unstale":"2024-04-25T10:13:12.508240+0000","last_undegraded":"2024-04-25T10:13:12.508240+0000","last_fullsized":"2024-04-25T10:13:12.508240+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:49.237836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"132'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509465+0000","last_change":"2024-04-25T10:13:10.505272+0000","last_active":"2024-04-25T10:13:12.509465+0000","last_peered":"2024-04-25T10:13:12.509465+0000","last_clean":"2024-04-25T10:13:12.509465+0000","last_became_active":"2024-04-25T10:13:10.504580+0000","last_became_peered":"2024-04-25T10:13:10.504580+0000","last_unstale":"2024-04-25T10:13:12.509465+0000","last_undegraded":"2024-04-25T10:13:12.509465+0000","last_fullsized":"2024-04-25T10:13:12.509465+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:55.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780865+0000","last_change":"2024-04-25T10:13:11.571071+0000","last_active":"2024-04-25T10:13:12.780865+0000","last_peered":"2024-04-25T10:13:12.780865+0000","last_clean":"2024-04-25T10:13:12.780865+0000","last_became_active":"2024-04-25T10:13:11.569555+0000","last_became_peered":"2024-04-25T10:13:11.569555+0000","last_unstale":"2024-04-25T10:13:12.780865+0000","last_undegraded":"2024-04-25T10:13:12.780865+0000","last_fullsized":"2024-04-25T10:13:12.780865+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:03.874094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508513+0000","last_change":"2024-04-25T10:13:10.502034+0000","last_active":"2024-04-25T10:13:12.508513+0000","last_peered":"2024-04-25T10:13:12.508513+0000","last_clean":"2024-04-25T10:13:12.508513+0000","last_became_active":"2024-04-25T10:13:10.501199+0000","last_became_peered":"2024-04-25T10:13:10.501199+0000","last_unstale":"2024-04-25T10:13:12.508513+0000","last_undegraded":"2024-04-25T10:13:12.508513+0000","last_fullsized":"2024-04-25T10:13:12.508513+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:02.941844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506669+0000","last_change":"2024-04-25T10:13:11.524061+0000","last_active":"2024-04-25T10:13:12.506669+0000","last_peered":"2024-04-25T10:13:12.506669+0000","last_clean":"2024-04-25T10:13:12.506669+0000","last_became_active":"2024-04-25T10:13:11.523588+0000","last_became_peered":"2024-04-25T10:13:11.523588+0000","last_unstale":"2024-04-25T10:13:12.506669+0000","last_undegraded":"2024-04-25T10:13:12.506669+0000","last_fullsized":"2024-04-25T10:13:12.506669+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:29.750424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810692+0000","last_change":"2024-04-25T10:13:10.501983+0000","last_active":"2024-04-25T10:13:12.810692+0000","last_peered":"2024-04-25T10:13:12.810692+0000","last_clean":"2024-04-25T10:13:12.810692+0000","last_became_active":"2024-04-25T10:13:10.501561+0000","last_became_peered":"2024-04-25T10:13:10.501561+0000","last_unstale":"2024-04-25T10:13:12.810692+0000","last_undegraded":"2024-04-25T10:13:12.810692+0000","last_fullsized":"2024-04-25T10:13:12.810692+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:53.326070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810602+0000","last_change":"2024-04-25T10:13:11.515291+0000","last_active":"2024-04-25T10:13:12.810602+0000","last_peered":"2024-04-25T10:13:12.810602+0000","last_clean":"2024-04-25T10:13:12.810602+0000","last_became_active":"2024-04-25T10:13:11.514943+0000","last_became_peered":"2024-04-25T10:13:11.514943+0000","last_unstale":"2024-04-25T10:13:12.810602+0000","last_undegraded":"2024-04-25T10:13:12.810602+0000","last_fullsized":"2024-04-25T10:13:12.810602+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.168445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514140+0000","last_change":"2024-04-25T10:13:10.509756+0000","last_active":"2024-04-25T10:13:12.514140+0000","last_peered":"2024-04-25T10:13:12.514140+0000","last_clean":"2024-04-25T10:13:12.514140+0000","last_became_active":"2024-04-25T10:13:10.508989+0000","last_became_peered":"2024-04-25T10:13:10.508989+0000","last_unstale":"2024-04-25T10:13:12.514140+0000","last_undegraded":"2024-04-25T10:13:12.514140+0000","last_fullsized":"2024-04-25T10:13:12.514140+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:45.302496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.811001+0000","last_change":"2024-04-25T10:13:11.521300+0000","last_active":"2024-04-25T10:13:12.811001+0000","last_peered":"2024-04-25T10:13:12.811001+0000","last_clean":"2024-04-25T10:13:12.811001+0000","last_became_active":"2024-04-25T10:13:11.521002+0000","last_became_peered":"2024-04-25T10:13:11.521002+0000","last_unstale":"2024-04-25T10:13:12.811001+0000","last_undegraded":"2024-04-25T10:13:12.811001+0000","last_fullsized":"2024-04-25T10:13:12.811001+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:10.791929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781666+0000","last_change":"2024-04-25T10:13:10.520125+0000","last_active":"2024-04-25T10:13:12.781666+0000","last_peered":"2024-04-25T10:13:12.781666+0000","last_clean":"2024-04-25T10:13:12.781666+0000","last_became_active":"2024-04-25T10:13:10.519603+0000","last_became_peered":"2024-04-25T10:13:10.519603+0000","last_unstale":"2024-04-25T10:13:12.781666+0000","last_undegraded":"2024-04-25T10:13:12.781666+0000","last_fullsized":"2024-04-25T10:13:12.781666+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:42.431009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810604+0000","last_change":"2024-04-25T10:13:11.554463+0000","last_active":"2024-04-25T10:13:12.810604+0000","last_peered":"2024-04-25T10:13:12.810604+0000","last_clean":"2024-04-25T10:13:12.810604+0000","last_became_active":"2024-04-25T10:13:11.554189+0000","last_became_peered":"2024-04-25T10:13:11.554189+0000","last_unstale":"2024-04-25T10:13:12.810604+0000","last_undegraded":"2024-04-25T10:13:12.810604+0000","last_fullsized":"2024-04-25T10:13:12.810604+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:46.637364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810630+0000","last_change":"2024-04-25T10:13:10.518854+0000","last_active":"2024-04-25T10:13:12.810630+0000","last_peered":"2024-04-25T10:13:12.810630+0000","last_clean":"2024-04-25T10:13:12.810630+0000","last_became_active":"2024-04-25T10:13:10.518358+0000","last_became_peered":"2024-04-25T10:13:10.518358+0000","last_unstale":"2024-04-25T10:13:12.810630+0000","last_undegraded":"2024-04-25T10:13:12.810630+0000","last_fullsized":"2024-04-25T10:13:12.810630+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:18.036486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781697+0000","last_change":"2024-04-25T10:13:11.570138+0000","last_active":"2024-04-25T10:13:12.781697+0000","last_peered":"2024-04-25T10:13:12.781697+0000","last_clean":"2024-04-25T10:13:12.781697+0000","last_became_active":"2024-04-25T10:13:11.569565+0000","last_became_peered":"2024-04-25T10:13:11.569565+0000","last_unstale":"2024-04-25T10:13:12.781697+0000","last_undegraded":"2024-04-25T10:13:12.781697+0000","last_fullsized":"2024-04-25T10:13:12.781697+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:42.493948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760048+0000","last_change":"2024-04-25T10:13:10.508016+0000","last_active":"2024-04-25T10:13:13.760048+0000","last_peered":"2024-04-25T10:13:13.760048+0000","last_clean":"2024-04-25T10:13:13.760048+0000","last_became_active":"2024-04-25T10:13:10.507618+0000","last_became_peered":"2024-04-25T10:13:10.507618+0000","last_unstale":"2024-04-25T10:13:13.760048+0000","last_undegraded":"2024-04-25T10:13:13.760048+0000","last_fullsized":"2024-04-25T10:13:13.760048+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:57.522399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509790+0000","last_change":"2024-04-25T10:13:11.527767+0000","last_active":"2024-04-25T10:13:12.509790+0000","last_peered":"2024-04-25T10:13:12.509790+0000","last_clean":"2024-04-25T10:13:12.509790+0000","last_became_active":"2024-04-25T10:13:11.527021+0000","last_became_peered":"2024-04-25T10:13:11.527021+0000","last_unstale":"2024-04-25T10:13:12.509790+0000","last_undegraded":"2024-04-25T10:13:12.509790+0000","last_fullsized":"2024-04-25T10:13:12.509790+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:58.525502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781388+0000","last_change":"2024-04-25T10:13:10.523416+0000","last_active":"2024-04-25T10:13:12.781388+0000","last_peered":"2024-04-25T10:13:12.781388+0000","last_clean":"2024-04-25T10:13:12.781388+0000","last_became_active":"2024-04-25T10:13:10.522665+0000","last_became_peered":"2024-04-25T10:13:10.522665+0000","last_unstale":"2024-04-25T10:13:12.781388+0000","last_undegraded":"2024-04-25T10:13:12.781388+0000","last_fullsized":"2024-04-25T10:13:12.781388+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:44.313595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":380,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780419+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:13:12.780419+0000","last_peered":"2024-04-25T10:13:12.780419+0000","last_clean":"2024-04-25T10:13:12.780419+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:13:12.780419+0000","last_undegraded":"2024-04-25T10:13:12.780419+0000","last_fullsized":"2024-04-25T10:13:12.780419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574991,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69484,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302356,"statfs":{"total":96636764160,"available":96565612544,"internally_reserved":0,"allocated":2260992,"data_stored":1363647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574992,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46692,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325148,"statfs":{"total":96636764160,"available":96588951552,"internally_reserved":0,"allocated":1658880,"data_stored":772249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1654784,"data_stored":771599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73052,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298788,"statfs":{"total":96636764160,"available":96561958912,"internally_reserved":0,"allocated":1650688,"data_stored":771577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73064,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298776,"statfs":{"total":96636764160,"available":96561946624,"internally_reserved":0,"allocated":1667072,"data_stored":773259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46692,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325148,"statfs":{"total":96636764160,"available":96588951552,"internally_reserved":0,"allocated":1658880,"data_stored":772193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59164,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312676,"statfs":{"total":96636764160,"available":96576180224,"internally_reserved":0,"allocated":1654784,"data_stored":771647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574992,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69476,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302364,"statfs":{"total":96636764160,"available":96565620736,"internally_reserved":0,"allocated":2252800,"data_stored":1362001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:19.928 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:20.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.056+0000 7f55c301e640 1 Processor -- start 2024-04-25T10:13:20.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.056+0000 7f55c301e640 1 -- start start 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c301e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0ce330 0x7f55bc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c301e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0c61d0 con 0x7f55bc0b1f70 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c301e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0c64b0 con 0x7f55bc15c9a0 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c301e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0cecc0 con 0x7f55bc0ce330 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bbfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f55bc0b1f70 0x7f55bc06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58548/0 (socket says 172.21.15.133:58548) 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bbfff640 1 -- 172.21.15.133:0/985271313 learned_addr learned my addr 172.21.15.133:0/985271313 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c15bb640 1 --2- 172.21.15.133:0/985271313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0ce330 0x7f55bc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983523166 0 0) 0x7f55bc0c61d0 con 0x7f55bc0b1f70 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a4003530 con 0x7f55bc0b1f70 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 200590477 0 0) 0x7f55a4003530 con 0x7f55bc0b1f70 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0ce330 msgr2=0x7f55bc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 --2- 172.21.15.133:0/985271313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0ce330 0x7f55bc0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 >> v1:172.21.15.162:6789/0 conn(0x7f55bc15c9a0 legacy=0x7f55bc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc06d4f0 con 0x7f55bc0b1f70 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f55ac001680 con 0x7f55bc0b1f70 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 >> v1:172.21.15.133:6789/0 conn(0x7f55bc0b1f70 legacy=0x7f55bc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 --2- 172.21.15.133:0/985271313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55a40039f0 0x7f55a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 --2- 172.21.15.133:0/985271313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0ce330 0x7f55a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 --2- 172.21.15.133:0/985271313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55a4004d50 0x7f55a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0cecc0 con 0x7f55a4004d50 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a400bab0 con 0x7f55a40039f0 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a400bcb0 con 0x7f55bc0ce330 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c15bb640 1 --2- 172.21.15.133:0/985271313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55a4004d50 0x7f55a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c0dba640 1 --2- 172.21.15.133:0/985271313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55a40039f0 0x7f55a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c15bb640 1 -- 172.21.15.133:0/985271313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0ce330 msgr2=0x7f55a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c15bb640 1 --2- 172.21.15.133:0/985271313 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0ce330 0x7f55a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c15bb640 1 -- 172.21.15.133:0/985271313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55a40039f0 msgr2=0x7f55a4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c15bb640 1 --2- 172.21.15.133:0/985271313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55a40039f0 0x7f55a4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.060+0000 7f55c15bb640 1 -- 172.21.15.133:0/985271313 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4003530 con 0x7f55a4004d50 2024-04-25T10:13:20.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c15bb640 1 --2- 172.21.15.133:0/985271313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55a4004d50 0x7f55a40051d0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009f90 tx=0x7f55b401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e311614907d23527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40b2070 con 0x7f55a4004d50 2024-04-25T10:13:20.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b40a5c90 con 0x7f55a4004d50 2024-04-25T10:13:20.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55bb7fe640 1 -- 172.21.15.133:0/985271313 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4009c10 con 0x7f55a4004d50 2024-04-25T10:13:20.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 -- 172.21.15.133:0/985271313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55a4004d50 msgr2=0x7f55a40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 --2- 172.21.15.133:0/985271313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55a4004d50 0x7f55a40051d0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009f90 tx=0x7f55b401cc80 comp rx=0 tx=0).stop 2024-04-25T10:13:20.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c15bb640 1 -- 172.21.15.133:0/985271313 reap_dead start 2024-04-25T10:13:20.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 -- 172.21.15.133:0/985271313 shutdown_connections 2024-04-25T10:13:20.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 -- 172.21.15.133:0/985271313 >> 172.21.15.133:0/985271313 conn(0x7f55bc067000 msgr2=0x7f55bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 -- 172.21.15.133:0/985271313 shutdown_connections 2024-04-25T10:13:20.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 -- 172.21.15.133:0/985271313 wait complete. 2024-04-25T10:13:20.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 Processor -- start 2024-04-25T10:13:20.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 -- start start 2024-04-25T10:13:20.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55bc155c10 0x7f55bc1f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0cc120 0x7f55bc0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55bc15b000 0x7f55bc15b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc15cf00 con 0x7f55bc15b000 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc15cd80 con 0x7f55bc155c10 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c301e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0ce780 con 0x7f55bc0cc120 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c15bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55bc15b000 0x7f55bc15b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c15bb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55bc15b000 0x7f55bc15b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34536/0 (socket says 172.21.15.133:34536) 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c15bb640 1 -- 172.21.15.133:0/1387476272 learned_addr learned my addr 172.21.15.133:0/1387476272 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c0dba640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55bc155c10 0x7f55bc1f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55bbfff640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0cc120 0x7f55bc0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c15bb640 1 -- 172.21.15.133:0/1387476272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0cc120 msgr2=0x7f55bc0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c15bb640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0cc120 0x7f55bc0cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c15bb640 1 -- 172.21.15.133:0/1387476272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55bc155c10 msgr2=0x7f55bc1f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c15bb640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55bc155c10 0x7f55bc1f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.064+0000 7f55c15bb640 1 -- 172.21.15.133:0/1387476272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0c7cf0 con 0x7f55bc15b000 2024-04-25T10:13:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55c15bb640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55bc15b000 0x7f55bc15b480 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f55b40b0040 tx=0x7f55b4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55b97fa640 1 -- 172.21.15.133:0/1387476272 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40bd070 con 0x7f55bc15b000 2024-04-25T10:13:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55b97fa640 1 -- 172.21.15.133:0/1387476272 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b40a5a90 con 0x7f55bc15b000 2024-04-25T10:13:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55c301e640 1 -- 172.21.15.133:0/1387476272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc0ccc90 con 0x7f55bc15b000 2024-04-25T10:13:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55b97fa640 1 -- 172.21.15.133:0/1387476272 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40b2040 con 0x7f55bc15b000 2024-04-25T10:13:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55c301e640 1 -- 172.21.15.133:0/1387476272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc0cd250 con 0x7f55bc15b000 2024-04-25T10:13:20.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55c301e640 1 -- 172.21.15.133:0/1387476272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc068ef0 con 0x7f55bc15b000 2024-04-25T10:13:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55b97fa640 1 -- 172.21.15.133:0/1387476272 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55b40c0070 con 0x7f55bc15b000 2024-04-25T10:13:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55b97fa640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a8066f60 0x7f55a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55c0dba640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a8066f60 0x7f55a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55b97fa640 1 -- 172.21.15.133:0/1387476272 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55b411ee00 con 0x7f55bc15b000 2024-04-25T10:13:20.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.068+0000 7f55c0dba640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a8066f60 0x7f55a8069420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f55b0000c10 tx=0x7f55b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.072+0000 7f55b97fa640 1 -- 172.21.15.133:0/1387476272 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55b4016990 con 0x7f55bc15b000 2024-04-25T10:13:20.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.228+0000 7f55c301e640 1 -- 172.21.15.133:0/1387476272 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55bc0b15f0 con 0x7f55a8066f60 2024-04-25T10:13:20.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.236+0000 7f55b97fa640 1 -- 172.21.15.133:0/1387476272 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252374 (secure 0 0 0) 0x7f55bc0b15f0 con 0x7f55a8066f60 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:13:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 -- 172.21.15.133:0/1387476272 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a8066f60 msgr2=0x7f55a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a8066f60 0x7f55a8069420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f55b0000c10 tx=0x7f55b0009040 comp rx=0 tx=0).stop 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 -- 172.21.15.133:0/1387476272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55bc15b000 msgr2=0x7f55bc15b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55bc15b000 0x7f55bc15b480 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f55b40b0040 tx=0x7f55b4009750 comp rx=0 tx=0).stop 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 -- 172.21.15.133:0/1387476272 shutdown_connections 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a8066f60 0x7f55a8069420 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55bc15b000 0x7f55bc15b480 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55bc0cc120 0x7f55bc0cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 --2- 172.21.15.133:0/1387476272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55bc155c10 0x7f55bc1f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 -- 172.21.15.133:0/1387476272 >> 172.21.15.133:0/1387476272 conn(0x7f55bc067000 msgr2=0x7f55bc0c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 -- 172.21.15.133:0/1387476272 shutdown_connections 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.240+0000 7f55c301e640 1 -- 172.21.15.133:0/1387476272 wait complete. 2024-04-25T10:13:20.261 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-04-25T10:13:19.927035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":496792,"kb_used_data":14120,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754477928,"statfs":{"total":773094113280,"available":772585398272,"internally_reserved":0,"allocated":14458880,"data_stored":7358172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.421252"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759324+0000","last_change":"2024-04-25T10:13:11.518523+0000","last_active":"2024-04-25T10:13:13.759324+0000","last_peered":"2024-04-25T10:13:13.759324+0000","last_clean":"2024-04-25T10:13:13.759324+0000","last_became_active":"2024-04-25T10:13:11.518246+0000","last_became_peered":"2024-04-25T10:13:11.518246+0000","last_unstale":"2024-04-25T10:13:13.759324+0000","last_undegraded":"2024-04-25T10:13:13.759324+0000","last_fullsized":"2024-04-25T10:13:13.759324+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:59.087475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514796+0000","last_change":"2024-04-25T10:13:11.566029+0000","last_active":"2024-04-25T10:13:12.514796+0000","last_peered":"2024-04-25T10:13:12.514796+0000","last_clean":"2024-04-25T10:13:12.514796+0000","last_became_active":"2024-04-25T10:13:11.565395+0000","last_became_peered":"2024-04-25T10:13:11.565395+0000","last_unstale":"2024-04-25T10:13:12.514796+0000","last_undegraded":"2024-04-25T10:13:12.514796+0000","last_fullsized":"2024-04-25T10:13:12.514796+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:46.637364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509560+0000","last_change":"2024-04-25T10:13:11.529012+0000","last_active":"2024-04-25T10:13:12.509560+0000","last_peered":"2024-04-25T10:13:12.509560+0000","last_clean":"2024-04-25T10:13:12.509560+0000","last_became_active":"2024-04-25T10:13:11.527515+0000","last_became_peered":"2024-04-25T10:13:11.527515+0000","last_unstale":"2024-04-25T10:13:12.509560+0000","last_undegraded":"2024-04-25T10:13:12.509560+0000","last_fullsized":"2024-04-25T10:13:12.509560+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:22.965494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514436+0000","last_change":"2024-04-25T10:13:11.551694+0000","last_active":"2024-04-25T10:13:12.514436+0000","last_peered":"2024-04-25T10:13:12.514436+0000","last_clean":"2024-04-25T10:13:12.514436+0000","last_became_active":"2024-04-25T10:13:11.551054+0000","last_became_peered":"2024-04-25T10:13:11.551054+0000","last_unstale":"2024-04-25T10:13:12.514436+0000","last_undegraded":"2024-04-25T10:13:12.514436+0000","last_fullsized":"2024-04-25T10:13:12.514436+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:38.132411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760155+0000","last_change":"2024-04-25T10:13:11.524769+0000","last_active":"2024-04-25T10:13:13.760155+0000","last_peered":"2024-04-25T10:13:13.760155+0000","last_clean":"2024-04-25T10:13:13.760155+0000","last_became_active":"2024-04-25T10:13:11.524473+0000","last_became_peered":"2024-04-25T10:13:11.524473+0000","last_unstale":"2024-04-25T10:13:13.760155+0000","last_undegraded":"2024-04-25T10:13:13.760155+0000","last_fullsized":"2024-04-25T10:13:13.760155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:52.612669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810454+0000","last_change":"2024-04-25T10:13:11.542185+0000","last_active":"2024-04-25T10:13:12.810454+0000","last_peered":"2024-04-25T10:13:12.810454+0000","last_clean":"2024-04-25T10:13:12.810454+0000","last_became_active":"2024-04-25T10:13:11.541886+0000","last_became_peered":"2024-04-25T10:13:11.541886+0000","last_unstale":"2024-04-25T10:13:12.810454+0000","last_undegraded":"2024-04-25T10:13:12.810454+0000","last_fullsized":"2024-04-25T10:13:12.810454+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:58.188235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780810+0000","last_change":"2024-04-25T10:13:11.564656+0000","last_active":"2024-04-25T10:13:12.780810+0000","last_peered":"2024-04-25T10:13:12.780810+0000","last_clean":"2024-04-25T10:13:12.780810+0000","last_became_active":"2024-04-25T10:13:11.564126+0000","last_became_peered":"2024-04-25T10:13:11.564126+0000","last_unstale":"2024-04-25T10:13:12.780810+0000","last_undegraded":"2024-04-25T10:13:12.780810+0000","last_fullsized":"2024-04-25T10:13:12.780810+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:07.102379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507173+0000","last_change":"2024-04-25T10:13:11.519215+0000","last_active":"2024-04-25T10:13:12.507173+0000","last_peered":"2024-04-25T10:13:12.507173+0000","last_clean":"2024-04-25T10:13:12.507173+0000","last_became_active":"2024-04-25T10:13:11.518497+0000","last_became_peered":"2024-04-25T10:13:11.518497+0000","last_unstale":"2024-04-25T10:13:12.507173+0000","last_undegraded":"2024-04-25T10:13:12.507173+0000","last_fullsized":"2024-04-25T10:13:12.507173+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:33.385792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.811190+0000","last_change":"2024-04-25T10:13:11.575005+0000","last_active":"2024-04-25T10:13:12.811190+0000","last_peered":"2024-04-25T10:13:12.811190+0000","last_clean":"2024-04-25T10:13:12.811190+0000","last_became_active":"2024-04-25T10:13:11.574701+0000","last_became_peered":"2024-04-25T10:13:11.574701+0000","last_unstale":"2024-04-25T10:13:12.811190+0000","last_undegraded":"2024-04-25T10:13:12.811190+0000","last_fullsized":"2024-04-25T10:13:12.811190+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:39.559386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506878+0000","last_change":"2024-04-25T10:13:11.523813+0000","last_active":"2024-04-25T10:13:12.506878+0000","last_peered":"2024-04-25T10:13:12.506878+0000","last_clean":"2024-04-25T10:13:12.506878+0000","last_became_active":"2024-04-25T10:13:11.523508+0000","last_became_peered":"2024-04-25T10:13:11.523508+0000","last_unstale":"2024-04-25T10:13:12.506878+0000","last_undegraded":"2024-04-25T10:13:12.506878+0000","last_fullsized":"2024-04-25T10:13:12.506878+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:56.849747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514527+0000","last_change":"2024-04-25T10:13:11.564687+0000","last_active":"2024-04-25T10:13:12.514527+0000","last_peered":"2024-04-25T10:13:12.514527+0000","last_clean":"2024-04-25T10:13:12.514527+0000","last_became_active":"2024-04-25T10:13:11.564124+0000","last_became_peered":"2024-04-25T10:13:11.564124+0000","last_unstale":"2024-04-25T10:13:12.514527+0000","last_undegraded":"2024-04-25T10:13:12.514527+0000","last_fullsized":"2024-04-25T10:13:12.514527+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:10.162659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759606+0000","last_change":"2024-04-25T10:13:11.518996+0000","last_active":"2024-04-25T10:13:13.759606+0000","last_peered":"2024-04-25T10:13:13.759606+0000","last_clean":"2024-04-25T10:13:13.759606+0000","last_became_active":"2024-04-25T10:13:11.518243+0000","last_became_peered":"2024-04-25T10:13:11.518243+0000","last_unstale":"2024-04-25T10:13:13.759606+0000","last_undegraded":"2024-04-25T10:13:13.759606+0000","last_fullsized":"2024-04-25T10:13:13.759606+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:18.646411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810982+0000","last_change":"2024-04-25T10:13:11.546228+0000","last_active":"2024-04-25T10:13:12.810982+0000","last_peered":"2024-04-25T10:13:12.810982+0000","last_clean":"2024-04-25T10:13:12.810982+0000","last_became_active":"2024-04-25T10:13:11.545888+0000","last_became_peered":"2024-04-25T10:13:11.545888+0000","last_unstale":"2024-04-25T10:13:12.810982+0000","last_undegraded":"2024-04-25T10:13:12.810982+0000","last_fullsized":"2024-04-25T10:13:12.810982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:52.612669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759949+0000","last_change":"2024-04-25T10:13:11.556074+0000","last_active":"2024-04-25T10:13:13.759949+0000","last_peered":"2024-04-25T10:13:13.759949+0000","last_clean":"2024-04-25T10:13:13.759949+0000","last_became_active":"2024-04-25T10:13:11.555629+0000","last_became_peered":"2024-04-25T10:13:11.555629+0000","last_unstale":"2024-04-25T10:13:13.759949+0000","last_undegraded":"2024-04-25T10:13:13.759949+0000","last_fullsized":"2024-04-25T10:13:13.759949+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:47.123922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.758838+0000","last_change":"2024-04-25T10:13:11.526526+0000","last_active":"2024-04-25T10:13:13.758838+0000","last_peered":"2024-04-25T10:13:13.758838+0000","last_clean":"2024-04-25T10:13:13.758838+0000","last_became_active":"2024-04-25T10:13:11.526250+0000","last_became_peered":"2024-04-25T10:13:11.526250+0000","last_unstale":"2024-04-25T10:13:13.758838+0000","last_undegraded":"2024-04-25T10:13:13.758838+0000","last_fullsized":"2024-04-25T10:13:13.758838+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:12.458362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.809981+0000","last_change":"2024-04-25T10:13:11.522012+0000","last_active":"2024-04-25T10:13:12.809981+0000","last_peered":"2024-04-25T10:13:12.809981+0000","last_clean":"2024-04-25T10:13:12.809981+0000","last_became_active":"2024-04-25T10:13:11.521053+0000","last_became_peered":"2024-04-25T10:13:11.521053+0000","last_unstale":"2024-04-25T10:13:12.809981+0000","last_undegraded":"2024-04-25T10:13:12.809981+0000","last_fullsized":"2024-04-25T10:13:12.809981+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:38.132411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.811195+0000","last_change":"2024-04-25T10:13:11.521369+0000","last_active":"2024-04-25T10:13:12.811195+0000","last_peered":"2024-04-25T10:13:12.811195+0000","last_clean":"2024-04-25T10:13:12.811195+0000","last_became_active":"2024-04-25T10:13:11.520948+0000","last_became_peered":"2024-04-25T10:13:11.520948+0000","last_unstale":"2024-04-25T10:13:12.811195+0000","last_undegraded":"2024-04-25T10:13:12.811195+0000","last_fullsized":"2024-04-25T10:13:12.811195+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:43.271736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760046+0000","last_change":"2024-04-25T10:13:11.525396+0000","last_active":"2024-04-25T10:13:13.760046+0000","last_peered":"2024-04-25T10:13:13.760046+0000","last_clean":"2024-04-25T10:13:13.760046+0000","last_became_active":"2024-04-25T10:13:11.525018+0000","last_became_peered":"2024-04-25T10:13:11.525018+0000","last_unstale":"2024-04-25T10:13:13.760046+0000","last_undegraded":"2024-04-25T10:13:13.760046+0000","last_fullsized":"2024-04-25T10:13:13.760046+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:46.637364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781628+0000","last_change":"2024-04-25T10:13:11.566279+0000","last_active":"2024-04-25T10:13:12.781628+0000","last_peered":"2024-04-25T10:13:12.781628+0000","last_clean":"2024-04-25T10:13:12.781628+0000","last_became_active":"2024-04-25T10:13:11.562536+0000","last_became_peered":"2024-04-25T10:13:11.562536+0000","last_unstale":"2024-04-25T10:13:12.781628+0000","last_undegraded":"2024-04-25T10:13:12.781628+0000","last_fullsized":"2024-04-25T10:13:12.781628+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:47.607552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761069+0000","last_change":"2024-04-25T10:13:11.525258+0000","last_active":"2024-04-25T10:13:13.761069+0000","last_peered":"2024-04-25T10:13:13.761069+0000","last_clean":"2024-04-25T10:13:13.761069+0000","last_became_active":"2024-04-25T10:13:11.524739+0000","last_became_peered":"2024-04-25T10:13:11.524739+0000","last_unstale":"2024-04-25T10:13:13.761069+0000","last_undegraded":"2024-04-25T10:13:13.761069+0000","last_fullsized":"2024-04-25T10:13:13.761069+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:52.434397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514594+0000","last_change":"2024-04-25T10:13:11.564376+0000","last_active":"2024-04-25T10:13:12.514594+0000","last_peered":"2024-04-25T10:13:12.514594+0000","last_clean":"2024-04-25T10:13:12.514594+0000","last_became_active":"2024-04-25T10:13:11.563826+0000","last_became_peered":"2024-04-25T10:13:11.563826+0000","last_unstale":"2024-04-25T10:13:12.514594+0000","last_undegraded":"2024-04-25T10:13:12.514594+0000","last_fullsized":"2024-04-25T10:13:12.514594+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:58.188235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759164+0000","last_change":"2024-04-25T10:13:11.567489+0000","last_active":"2024-04-25T10:13:13.759164+0000","last_peered":"2024-04-25T10:13:13.759164+0000","last_clean":"2024-04-25T10:13:13.759164+0000","last_became_active":"2024-04-25T10:13:11.567145+0000","last_became_peered":"2024-04-25T10:13:11.567145+0000","last_unstale":"2024-04-25T10:13:13.759164+0000","last_undegraded":"2024-04-25T10:13:13.759164+0000","last_fullsized":"2024-04-25T10:13:13.759164+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:37.208933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514296+0000","last_change":"2024-04-25T10:13:11.556816+0000","last_active":"2024-04-25T10:13:12.514296+0000","last_peered":"2024-04-25T10:13:12.514296+0000","last_clean":"2024-04-25T10:13:12.514296+0000","last_became_active":"2024-04-25T10:13:11.556288+0000","last_became_peered":"2024-04-25T10:13:11.556288+0000","last_unstale":"2024-04-25T10:13:12.514296+0000","last_undegraded":"2024-04-25T10:13:12.514296+0000","last_fullsized":"2024-04-25T10:13:12.514296+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:36.478303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810170+0000","last_change":"2024-04-25T10:13:11.567581+0000","last_active":"2024-04-25T10:13:12.810170+0000","last_peered":"2024-04-25T10:13:12.810170+0000","last_clean":"2024-04-25T10:13:12.810170+0000","last_became_active":"2024-04-25T10:13:11.567284+0000","last_became_peered":"2024-04-25T10:13:11.567284+0000","last_unstale":"2024-04-25T10:13:12.810170+0000","last_undegraded":"2024-04-25T10:13:12.810170+0000","last_fullsized":"2024-04-25T10:13:12.810170+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:10.162659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510104+0000","last_change":"2024-04-25T10:13:11.562038+0000","last_active":"2024-04-25T10:13:12.510104+0000","last_peered":"2024-04-25T10:13:12.510104+0000","last_clean":"2024-04-25T10:13:12.510104+0000","last_became_active":"2024-04-25T10:13:11.555255+0000","last_became_peered":"2024-04-25T10:13:11.555255+0000","last_unstale":"2024-04-25T10:13:12.510104+0000","last_undegraded":"2024-04-25T10:13:12.510104+0000","last_fullsized":"2024-04-25T10:13:12.510104+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:53.723716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509065+0000","last_change":"2024-04-25T10:13:11.564767+0000","last_active":"2024-04-25T10:13:12.509065+0000","last_peered":"2024-04-25T10:13:12.509065+0000","last_clean":"2024-04-25T10:13:12.509065+0000","last_became_active":"2024-04-25T10:13:11.564094+0000","last_became_peered":"2024-04-25T10:13:11.564094+0000","last_unstale":"2024-04-25T10:13:12.509065+0000","last_undegraded":"2024-04-25T10:13:12.509065+0000","last_fullsized":"2024-04-25T10:13:12.509065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.305600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781823+0000","last_change":"2024-04-25T10:13:11.570693+0000","last_active":"2024-04-25T10:13:12.781823+0000","last_peered":"2024-04-25T10:13:12.781823+0000","last_clean":"2024-04-25T10:13:12.781823+0000","last_became_active":"2024-04-25T10:13:11.563159+0000","last_became_peered":"2024-04-25T10:13:11.563159+0000","last_unstale":"2024-04-25T10:13:12.781823+0000","last_undegraded":"2024-04-25T10:13:12.781823+0000","last_fullsized":"2024-04-25T10:13:12.781823+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:27.760139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506676+0000","last_change":"2024-04-25T10:13:11.524297+0000","last_active":"2024-04-25T10:13:12.506676+0000","last_peered":"2024-04-25T10:13:12.506676+0000","last_clean":"2024-04-25T10:13:12.506676+0000","last_became_active":"2024-04-25T10:13:11.523906+0000","last_became_peered":"2024-04-25T10:13:11.523906+0000","last_unstale":"2024-04-25T10:13:12.506676+0000","last_undegraded":"2024-04-25T10:13:12.506676+0000","last_fullsized":"2024-04-25T10:13:12.506676+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:42.816079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508821+0000","last_change":"2024-04-25T10:13:11.552751+0000","last_active":"2024-04-25T10:13:12.508821+0000","last_peered":"2024-04-25T10:13:12.508821+0000","last_clean":"2024-04-25T10:13:12.508821+0000","last_became_active":"2024-04-25T10:13:11.551644+0000","last_became_peered":"2024-04-25T10:13:11.551644+0000","last_unstale":"2024-04-25T10:13:12.508821+0000","last_undegraded":"2024-04-25T10:13:12.508821+0000","last_fullsized":"2024-04-25T10:13:12.508821+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:33.385792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509621+0000","last_change":"2024-04-25T10:13:11.558991+0000","last_active":"2024-04-25T10:13:12.509621+0000","last_peered":"2024-04-25T10:13:12.509621+0000","last_clean":"2024-04-25T10:13:12.509621+0000","last_became_active":"2024-04-25T10:13:11.553029+0000","last_became_peered":"2024-04-25T10:13:11.553029+0000","last_unstale":"2024-04-25T10:13:12.509621+0000","last_undegraded":"2024-04-25T10:13:12.509621+0000","last_fullsized":"2024-04-25T10:13:12.509621+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:49.372602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507272+0000","last_change":"2024-04-25T10:13:11.525057+0000","last_active":"2024-04-25T10:13:12.507272+0000","last_peered":"2024-04-25T10:13:12.507272+0000","last_clean":"2024-04-25T10:13:12.507272+0000","last_became_active":"2024-04-25T10:13:11.522942+0000","last_became_peered":"2024-04-25T10:13:11.522942+0000","last_unstale":"2024-04-25T10:13:12.507272+0000","last_undegraded":"2024-04-25T10:13:12.507272+0000","last_fullsized":"2024-04-25T10:13:12.507272+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.305600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506251+0000","last_change":"2024-04-25T10:13:11.522239+0000","last_active":"2024-04-25T10:13:12.506251+0000","last_peered":"2024-04-25T10:13:12.506251+0000","last_clean":"2024-04-25T10:13:12.506251+0000","last_became_active":"2024-04-25T10:13:11.521940+0000","last_became_peered":"2024-04-25T10:13:11.521940+0000","last_unstale":"2024-04-25T10:13:12.506251+0000","last_undegraded":"2024-04-25T10:13:12.506251+0000","last_fullsized":"2024-04-25T10:13:12.506251+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:42.493948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507440+0000","last_change":"2024-04-25T10:13:11.524308+0000","last_active":"2024-04-25T10:13:12.507440+0000","last_peered":"2024-04-25T10:13:12.507440+0000","last_clean":"2024-04-25T10:13:12.507440+0000","last_became_active":"2024-04-25T10:13:11.523488+0000","last_became_peered":"2024-04-25T10:13:11.523488+0000","last_unstale":"2024-04-25T10:13:12.507440+0000","last_undegraded":"2024-04-25T10:13:12.507440+0000","last_fullsized":"2024-04-25T10:13:12.507440+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:38.172285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"131'4","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510113+0000","last_change":"2024-04-25T10:13:10.509075+0000","last_active":"2024-04-25T10:13:12.510113+0000","last_peered":"2024-04-25T10:13:12.510113+0000","last_clean":"2024-04-25T10:13:12.510113+0000","last_became_active":"2024-04-25T10:13:10.506901+0000","last_became_peered":"2024-04-25T10:13:10.506901+0000","last_unstale":"2024-04-25T10:13:12.510113+0000","last_undegraded":"2024-04-25T10:13:12.510113+0000","last_fullsized":"2024-04-25T10:13:12.510113+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:43.483874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759499+0000","last_change":"2024-04-25T10:13:11.567350+0000","last_active":"2024-04-25T10:13:13.759499+0000","last_peered":"2024-04-25T10:13:13.759499+0000","last_clean":"2024-04-25T10:13:13.759499+0000","last_became_active":"2024-04-25T10:13:11.567065+0000","last_became_peered":"2024-04-25T10:13:11.567065+0000","last_unstale":"2024-04-25T10:13:13.759499+0000","last_undegraded":"2024-04-25T10:13:13.759499+0000","last_fullsized":"2024-04-25T10:13:13.759499+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:14.942690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760310+0000","last_change":"2024-04-25T10:13:10.504453+0000","last_active":"2024-04-25T10:13:13.760310+0000","last_peered":"2024-04-25T10:13:13.760310+0000","last_clean":"2024-04-25T10:13:13.760310+0000","last_became_active":"2024-04-25T10:13:10.504177+0000","last_became_peered":"2024-04-25T10:13:10.504177+0000","last_unstale":"2024-04-25T10:13:13.760310+0000","last_undegraded":"2024-04-25T10:13:13.760310+0000","last_fullsized":"2024-04-25T10:13:13.760310+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:47.372814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514629+0000","last_change":"2024-04-25T10:13:11.573265+0000","last_active":"2024-04-25T10:13:12.514629+0000","last_peered":"2024-04-25T10:13:12.514629+0000","last_clean":"2024-04-25T10:13:12.514629+0000","last_became_active":"2024-04-25T10:13:11.568213+0000","last_became_peered":"2024-04-25T10:13:11.568213+0000","last_unstale":"2024-04-25T10:13:12.514629+0000","last_undegraded":"2024-04-25T10:13:12.514629+0000","last_fullsized":"2024-04-25T10:13:12.514629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:10.130196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781827+0000","last_change":"2024-04-25T10:13:10.520416+0000","last_active":"2024-04-25T10:13:12.781827+0000","last_peered":"2024-04-25T10:13:12.781827+0000","last_clean":"2024-04-25T10:13:12.781827+0000","last_became_active":"2024-04-25T10:13:10.519694+0000","last_became_peered":"2024-04-25T10:13:10.519694+0000","last_unstale":"2024-04-25T10:13:12.781827+0000","last_undegraded":"2024-04-25T10:13:12.781827+0000","last_fullsized":"2024-04-25T10:13:12.781827+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:33.252824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759314+0000","last_change":"2024-04-25T10:13:11.526936+0000","last_active":"2024-04-25T10:13:13.759314+0000","last_peered":"2024-04-25T10:13:13.759314+0000","last_clean":"2024-04-25T10:13:13.759314+0000","last_became_active":"2024-04-25T10:13:11.526618+0000","last_became_peered":"2024-04-25T10:13:11.526618+0000","last_unstale":"2024-04-25T10:13:13.759314+0000","last_undegraded":"2024-04-25T10:13:13.759314+0000","last_fullsized":"2024-04-25T10:13:13.759314+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:20.954445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781022+0000","last_change":"2024-04-25T10:13:10.519322+0000","last_active":"2024-04-25T10:13:12.781022+0000","last_peered":"2024-04-25T10:13:12.781022+0000","last_clean":"2024-04-25T10:13:12.781022+0000","last_became_active":"2024-04-25T10:13:10.518685+0000","last_became_peered":"2024-04-25T10:13:10.518685+0000","last_unstale":"2024-04-25T10:13:12.781022+0000","last_undegraded":"2024-04-25T10:13:12.781022+0000","last_fullsized":"2024-04-25T10:13:12.781022+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:03.662949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510070+0000","last_change":"2024-04-25T10:13:11.523386+0000","last_active":"2024-04-25T10:13:12.510070+0000","last_peered":"2024-04-25T10:13:12.510070+0000","last_clean":"2024-04-25T10:13:12.510070+0000","last_became_active":"2024-04-25T10:13:11.522629+0000","last_became_peered":"2024-04-25T10:13:11.522629+0000","last_unstale":"2024-04-25T10:13:12.510070+0000","last_undegraded":"2024-04-25T10:13:12.510070+0000","last_fullsized":"2024-04-25T10:13:12.510070+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:49.372602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507928+0000","last_change":"2024-04-25T10:13:10.499060+0000","last_active":"2024-04-25T10:13:12.507928+0000","last_peered":"2024-04-25T10:13:12.507928+0000","last_clean":"2024-04-25T10:13:12.507928+0000","last_became_active":"2024-04-25T10:13:10.494838+0000","last_became_peered":"2024-04-25T10:13:10.494838+0000","last_unstale":"2024-04-25T10:13:12.507928+0000","last_undegraded":"2024-04-25T10:13:12.507928+0000","last_fullsized":"2024-04-25T10:13:12.507928+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:51.109162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508381+0000","last_change":"2024-04-25T10:13:11.526929+0000","last_active":"2024-04-25T10:13:12.508381+0000","last_peered":"2024-04-25T10:13:12.508381+0000","last_clean":"2024-04-25T10:13:12.508381+0000","last_became_active":"2024-04-25T10:13:11.526184+0000","last_became_peered":"2024-04-25T10:13:11.526184+0000","last_unstale":"2024-04-25T10:13:12.508381+0000","last_undegraded":"2024-04-25T10:13:12.508381+0000","last_fullsized":"2024-04-25T10:13:12.508381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:22.965494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759768+0000","last_change":"2024-04-25T10:13:10.500210+0000","last_active":"2024-04-25T10:13:13.759768+0000","last_peered":"2024-04-25T10:13:13.759768+0000","last_clean":"2024-04-25T10:13:13.759768+0000","last_became_active":"2024-04-25T10:13:10.499928+0000","last_became_peered":"2024-04-25T10:13:10.499928+0000","last_unstale":"2024-04-25T10:13:13.759768+0000","last_undegraded":"2024-04-25T10:13:13.759768+0000","last_fullsized":"2024-04-25T10:13:13.759768+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:48.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760330+0000","last_change":"2024-04-25T10:13:11.555023+0000","last_active":"2024-04-25T10:13:13.760330+0000","last_peered":"2024-04-25T10:13:13.760330+0000","last_clean":"2024-04-25T10:13:13.760330+0000","last_became_active":"2024-04-25T10:13:11.554728+0000","last_became_peered":"2024-04-25T10:13:11.554728+0000","last_unstale":"2024-04-25T10:13:13.760330+0000","last_undegraded":"2024-04-25T10:13:13.760330+0000","last_fullsized":"2024-04-25T10:13:13.760330+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:39.559386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.809762+0000","last_change":"2024-04-25T10:13:10.504035+0000","last_active":"2024-04-25T10:13:12.809762+0000","last_peered":"2024-04-25T10:13:12.809762+0000","last_clean":"2024-04-25T10:13:12.809762+0000","last_became_active":"2024-04-25T10:13:10.502964+0000","last_became_peered":"2024-04-25T10:13:10.502964+0000","last_unstale":"2024-04-25T10:13:12.809762+0000","last_undegraded":"2024-04-25T10:13:12.809762+0000","last_fullsized":"2024-04-25T10:13:12.809762+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:04.997195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781330+0000","last_change":"2024-04-25T10:13:11.559414+0000","last_active":"2024-04-25T10:13:12.781330+0000","last_peered":"2024-04-25T10:13:12.781330+0000","last_clean":"2024-04-25T10:13:12.781330+0000","last_became_active":"2024-04-25T10:13:11.558823+0000","last_became_peered":"2024-04-25T10:13:11.558823+0000","last_unstale":"2024-04-25T10:13:12.781330+0000","last_undegraded":"2024-04-25T10:13:12.781330+0000","last_fullsized":"2024-04-25T10:13:12.781330+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:06.795281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508653+0000","last_change":"2024-04-25T10:13:10.507874+0000","last_active":"2024-04-25T10:13:12.508653+0000","last_peered":"2024-04-25T10:13:12.508653+0000","last_clean":"2024-04-25T10:13:12.508653+0000","last_became_active":"2024-04-25T10:13:10.507221+0000","last_became_peered":"2024-04-25T10:13:10.507221+0000","last_unstale":"2024-04-25T10:13:12.508653+0000","last_undegraded":"2024-04-25T10:13:12.508653+0000","last_fullsized":"2024-04-25T10:13:12.508653+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:45.472460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.758603+0000","last_change":"2024-04-25T10:13:11.548771+0000","last_active":"2024-04-25T10:13:13.758603+0000","last_peered":"2024-04-25T10:13:13.758603+0000","last_clean":"2024-04-25T10:13:13.758603+0000","last_became_active":"2024-04-25T10:13:11.548373+0000","last_became_peered":"2024-04-25T10:13:11.548373+0000","last_unstale":"2024-04-25T10:13:13.758603+0000","last_undegraded":"2024-04-25T10:13:13.758603+0000","last_fullsized":"2024-04-25T10:13:13.758603+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:59.266348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514921+0000","last_change":"2024-04-25T10:13:10.494868+0000","last_active":"2024-04-25T10:13:12.514921+0000","last_peered":"2024-04-25T10:13:12.514921+0000","last_clean":"2024-04-25T10:13:12.514921+0000","last_became_active":"2024-04-25T10:13:10.494288+0000","last_became_peered":"2024-04-25T10:13:10.494288+0000","last_unstale":"2024-04-25T10:13:12.514921+0000","last_undegraded":"2024-04-25T10:13:12.514921+0000","last_fullsized":"2024-04-25T10:13:12.514921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:37.169182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810475+0000","last_change":"2024-04-25T10:13:11.513098+0000","last_active":"2024-04-25T10:13:12.810475+0000","last_peered":"2024-04-25T10:13:12.810475+0000","last_clean":"2024-04-25T10:13:12.810475+0000","last_became_active":"2024-04-25T10:13:11.512776+0000","last_became_peered":"2024-04-25T10:13:11.512776+0000","last_unstale":"2024-04-25T10:13:12.810475+0000","last_undegraded":"2024-04-25T10:13:12.810475+0000","last_fullsized":"2024-04-25T10:13:12.810475+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:10.130196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759446+0000","last_change":"2024-04-25T10:13:10.497439+0000","last_active":"2024-04-25T10:13:13.759446+0000","last_peered":"2024-04-25T10:13:13.759446+0000","last_clean":"2024-04-25T10:13:13.759446+0000","last_became_active":"2024-04-25T10:13:10.496013+0000","last_became_peered":"2024-04-25T10:13:10.496013+0000","last_unstale":"2024-04-25T10:13:13.759446+0000","last_undegraded":"2024-04-25T10:13:13.759446+0000","last_fullsized":"2024-04-25T10:13:13.759446+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:19.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.510044+0000","last_change":"2024-04-25T10:13:11.565816+0000","last_active":"2024-04-25T10:13:12.510044+0000","last_peered":"2024-04-25T10:13:12.510044+0000","last_clean":"2024-04-25T10:13:12.510044+0000","last_became_active":"2024-04-25T10:13:11.565141+0000","last_became_peered":"2024-04-25T10:13:11.565141+0000","last_unstale":"2024-04-25T10:13:12.510044+0000","last_undegraded":"2024-04-25T10:13:12.510044+0000","last_fullsized":"2024-04-25T10:13:12.510044+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.375918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"131'3","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781913+0000","last_change":"2024-04-25T10:13:10.520399+0000","last_active":"2024-04-25T10:13:12.781913+0000","last_peered":"2024-04-25T10:13:12.781913+0000","last_clean":"2024-04-25T10:13:12.781913+0000","last_became_active":"2024-04-25T10:13:10.519892+0000","last_became_peered":"2024-04-25T10:13:10.519892+0000","last_unstale":"2024-04-25T10:13:12.781913+0000","last_undegraded":"2024-04-25T10:13:12.781913+0000","last_fullsized":"2024-04-25T10:13:12.781913+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:21.502204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509200+0000","last_change":"2024-04-25T10:13:11.529758+0000","last_active":"2024-04-25T10:13:12.509200+0000","last_peered":"2024-04-25T10:13:12.509200+0000","last_clean":"2024-04-25T10:13:12.509200+0000","last_became_active":"2024-04-25T10:13:11.528989+0000","last_became_peered":"2024-04-25T10:13:11.528989+0000","last_unstale":"2024-04-25T10:13:12.509200+0000","last_undegraded":"2024-04-25T10:13:12.509200+0000","last_fullsized":"2024-04-25T10:13:12.509200+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:38.172285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"131'6","reported_seq":20,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781106+0000","last_change":"2024-04-25T10:13:10.523732+0000","last_active":"2024-04-25T10:13:12.781106+0000","last_peered":"2024-04-25T10:13:12.781106+0000","last_clean":"2024-04-25T10:13:12.781106+0000","last_became_active":"2024-04-25T10:13:10.523019+0000","last_became_peered":"2024-04-25T10:13:10.523019+0000","last_unstale":"2024-04-25T10:13:12.781106+0000","last_undegraded":"2024-04-25T10:13:12.781106+0000","last_fullsized":"2024-04-25T10:13:12.781106+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:35.743803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.507053+0000","last_change":"2024-04-25T10:13:11.520435+0000","last_active":"2024-04-25T10:13:12.507053+0000","last_peered":"2024-04-25T10:13:12.507053+0000","last_clean":"2024-04-25T10:13:12.507053+0000","last_became_active":"2024-04-25T10:13:11.520147+0000","last_became_peered":"2024-04-25T10:13:11.520147+0000","last_unstale":"2024-04-25T10:13:12.507053+0000","last_undegraded":"2024-04-25T10:13:12.507053+0000","last_fullsized":"2024-04-25T10:13:12.507053+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:02.552382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759467+0000","last_change":"2024-04-25T10:13:10.497816+0000","last_active":"2024-04-25T10:13:13.759467+0000","last_peered":"2024-04-25T10:13:13.759467+0000","last_clean":"2024-04-25T10:13:13.759467+0000","last_became_active":"2024-04-25T10:13:10.497547+0000","last_became_peered":"2024-04-25T10:13:10.497547+0000","last_unstale":"2024-04-25T10:13:13.759467+0000","last_undegraded":"2024-04-25T10:13:13.759467+0000","last_fullsized":"2024-04-25T10:13:13.759467+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:07.860557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781622+0000","last_change":"2024-04-25T10:13:11.570950+0000","last_active":"2024-04-25T10:13:12.781622+0000","last_peered":"2024-04-25T10:13:12.781622+0000","last_clean":"2024-04-25T10:13:12.781622+0000","last_became_active":"2024-04-25T10:13:11.570376+0000","last_became_peered":"2024-04-25T10:13:11.570376+0000","last_unstale":"2024-04-25T10:13:12.781622+0000","last_undegraded":"2024-04-25T10:13:12.781622+0000","last_fullsized":"2024-04-25T10:13:12.781622+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:51.592552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508365+0000","last_change":"2024-04-25T10:13:10.507142+0000","last_active":"2024-04-25T10:13:12.508365+0000","last_peered":"2024-04-25T10:13:12.508365+0000","last_clean":"2024-04-25T10:13:12.508365+0000","last_became_active":"2024-04-25T10:13:10.506580+0000","last_became_peered":"2024-04-25T10:13:10.506580+0000","last_unstale":"2024-04-25T10:13:12.508365+0000","last_undegraded":"2024-04-25T10:13:12.508365+0000","last_fullsized":"2024-04-25T10:13:12.508365+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:41.464459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760601+0000","last_change":"2024-04-25T10:13:11.524605+0000","last_active":"2024-04-25T10:13:13.760601+0000","last_peered":"2024-04-25T10:13:13.760601+0000","last_clean":"2024-04-25T10:13:13.760601+0000","last_became_active":"2024-04-25T10:13:11.523134+0000","last_became_peered":"2024-04-25T10:13:11.523134+0000","last_unstale":"2024-04-25T10:13:13.760601+0000","last_undegraded":"2024-04-25T10:13:13.760601+0000","last_fullsized":"2024-04-25T10:13:13.760601+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:43.271736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"131'8","reported_seq":22,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810197+0000","last_change":"2024-04-25T10:13:10.503861+0000","last_active":"2024-04-25T10:13:12.810197+0000","last_peered":"2024-04-25T10:13:12.810197+0000","last_clean":"2024-04-25T10:13:12.810197+0000","last_became_active":"2024-04-25T10:13:10.503476+0000","last_became_peered":"2024-04-25T10:13:10.503476+0000","last_unstale":"2024-04-25T10:13:12.810197+0000","last_undegraded":"2024-04-25T10:13:12.810197+0000","last_fullsized":"2024-04-25T10:13:12.810197+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.405882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508510+0000","last_change":"2024-04-25T10:13:11.547495+0000","last_active":"2024-04-25T10:13:12.508510+0000","last_peered":"2024-04-25T10:13:12.508510+0000","last_clean":"2024-04-25T10:13:12.508510+0000","last_became_active":"2024-04-25T10:13:11.541513+0000","last_became_peered":"2024-04-25T10:13:11.541513+0000","last_unstale":"2024-04-25T10:13:12.508510+0000","last_undegraded":"2024-04-25T10:13:12.508510+0000","last_fullsized":"2024-04-25T10:13:12.508510+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:59.408986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514982+0000","last_change":"2024-04-25T10:13:10.501668+0000","last_active":"2024-04-25T10:13:12.514982+0000","last_peered":"2024-04-25T10:13:12.514982+0000","last_clean":"2024-04-25T10:13:12.514982+0000","last_became_active":"2024-04-25T10:13:10.501112+0000","last_became_peered":"2024-04-25T10:13:10.501112+0000","last_unstale":"2024-04-25T10:13:12.514982+0000","last_undegraded":"2024-04-25T10:13:12.514982+0000","last_fullsized":"2024-04-25T10:13:12.514982+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:21.962390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780949+0000","last_change":"2024-04-25T10:13:11.559201+0000","last_active":"2024-04-25T10:13:12.780949+0000","last_peered":"2024-04-25T10:13:12.780949+0000","last_clean":"2024-04-25T10:13:12.780949+0000","last_became_active":"2024-04-25T10:13:11.558584+0000","last_became_peered":"2024-04-25T10:13:11.558584+0000","last_unstale":"2024-04-25T10:13:12.780949+0000","last_undegraded":"2024-04-25T10:13:12.780949+0000","last_fullsized":"2024-04-25T10:13:12.780949+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:47.947133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509197+0000","last_change":"2024-04-25T10:13:10.507714+0000","last_active":"2024-04-25T10:13:12.509197+0000","last_peered":"2024-04-25T10:13:12.509197+0000","last_clean":"2024-04-25T10:13:12.509197+0000","last_became_active":"2024-04-25T10:13:10.506915+0000","last_became_peered":"2024-04-25T10:13:10.506915+0000","last_unstale":"2024-04-25T10:13:12.509197+0000","last_undegraded":"2024-04-25T10:13:12.509197+0000","last_fullsized":"2024-04-25T10:13:12.509197+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:31.176627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781867+0000","last_change":"2024-04-25T10:13:11.559225+0000","last_active":"2024-04-25T10:13:12.781867+0000","last_peered":"2024-04-25T10:13:12.781867+0000","last_clean":"2024-04-25T10:13:12.781867+0000","last_became_active":"2024-04-25T10:13:11.558665+0000","last_became_peered":"2024-04-25T10:13:11.558665+0000","last_unstale":"2024-04-25T10:13:12.781867+0000","last_undegraded":"2024-04-25T10:13:12.781867+0000","last_fullsized":"2024-04-25T10:13:12.781867+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:29.750424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810921+0000","last_change":"2024-04-25T10:13:10.502878+0000","last_active":"2024-04-25T10:13:12.810921+0000","last_peered":"2024-04-25T10:13:12.810921+0000","last_clean":"2024-04-25T10:13:12.810921+0000","last_became_active":"2024-04-25T10:13:10.501740+0000","last_became_peered":"2024-04-25T10:13:10.501740+0000","last_unstale":"2024-04-25T10:13:12.810921+0000","last_undegraded":"2024-04-25T10:13:12.810921+0000","last_fullsized":"2024-04-25T10:13:12.810921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:48.234733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506278+0000","last_change":"2024-04-25T10:13:11.548117+0000","last_active":"2024-04-25T10:13:12.506278+0000","last_peered":"2024-04-25T10:13:12.506278+0000","last_clean":"2024-04-25T10:13:12.506278+0000","last_became_active":"2024-04-25T10:13:11.547691+0000","last_became_peered":"2024-04-25T10:13:11.547691+0000","last_unstale":"2024-04-25T10:13:12.506278+0000","last_undegraded":"2024-04-25T10:13:12.506278+0000","last_fullsized":"2024-04-25T10:13:12.506278+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:27.760139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761144+0000","last_change":"2024-04-25T10:13:10.499039+0000","last_active":"2024-04-25T10:13:13.761144+0000","last_peered":"2024-04-25T10:13:13.761144+0000","last_clean":"2024-04-25T10:13:13.761144+0000","last_became_active":"2024-04-25T10:13:10.498545+0000","last_became_peered":"2024-04-25T10:13:10.498545+0000","last_unstale":"2024-04-25T10:13:13.761144+0000","last_undegraded":"2024-04-25T10:13:13.761144+0000","last_fullsized":"2024-04-25T10:13:13.761144+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:52.720613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761304+0000","last_change":"2024-04-25T10:13:11.518690+0000","last_active":"2024-04-25T10:13:13.761304+0000","last_peered":"2024-04-25T10:13:13.761304+0000","last_clean":"2024-04-25T10:13:13.761304+0000","last_became_active":"2024-04-25T10:13:11.518411+0000","last_became_peered":"2024-04-25T10:13:11.518411+0000","last_unstale":"2024-04-25T10:13:13.761304+0000","last_undegraded":"2024-04-25T10:13:13.761304+0000","last_fullsized":"2024-04-25T10:13:13.761304+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:09.135552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508272+0000","last_change":"2024-04-25T10:13:10.504158+0000","last_active":"2024-04-25T10:13:12.508272+0000","last_peered":"2024-04-25T10:13:12.508272+0000","last_clean":"2024-04-25T10:13:12.508272+0000","last_became_active":"2024-04-25T10:13:10.502316+0000","last_became_peered":"2024-04-25T10:13:10.502316+0000","last_unstale":"2024-04-25T10:13:12.508272+0000","last_undegraded":"2024-04-25T10:13:12.508272+0000","last_fullsized":"2024-04-25T10:13:12.508272+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:39.042840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759831+0000","last_change":"2024-04-25T10:13:11.562497+0000","last_active":"2024-04-25T10:13:13.759831+0000","last_peered":"2024-04-25T10:13:13.759831+0000","last_clean":"2024-04-25T10:13:13.759831+0000","last_became_active":"2024-04-25T10:13:11.562192+0000","last_became_peered":"2024-04-25T10:13:11.562192+0000","last_unstale":"2024-04-25T10:13:13.759831+0000","last_undegraded":"2024-04-25T10:13:13.759831+0000","last_fullsized":"2024-04-25T10:13:13.759831+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:27.488802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760075+0000","last_change":"2024-04-25T10:13:10.500218+0000","last_active":"2024-04-25T10:13:13.760075+0000","last_peered":"2024-04-25T10:13:13.760075+0000","last_clean":"2024-04-25T10:13:13.760075+0000","last_became_active":"2024-04-25T10:13:10.499839+0000","last_became_peered":"2024-04-25T10:13:10.499839+0000","last_unstale":"2024-04-25T10:13:13.760075+0000","last_undegraded":"2024-04-25T10:13:13.760075+0000","last_fullsized":"2024-04-25T10:13:13.760075+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:15.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760253+0000","last_change":"2024-04-25T10:13:11.547740+0000","last_active":"2024-04-25T10:13:13.760253+0000","last_peered":"2024-04-25T10:13:13.760253+0000","last_clean":"2024-04-25T10:13:13.760253+0000","last_became_active":"2024-04-25T10:13:11.547457+0000","last_became_peered":"2024-04-25T10:13:11.547457+0000","last_unstale":"2024-04-25T10:13:13.760253+0000","last_undegraded":"2024-04-25T10:13:13.760253+0000","last_fullsized":"2024-04-25T10:13:13.760253+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:10.791929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781126+0000","last_change":"2024-04-25T10:13:10.523607+0000","last_active":"2024-04-25T10:13:12.781126+0000","last_peered":"2024-04-25T10:13:12.781126+0000","last_clean":"2024-04-25T10:13:12.781126+0000","last_became_active":"2024-04-25T10:13:10.522750+0000","last_became_peered":"2024-04-25T10:13:10.522750+0000","last_unstale":"2024-04-25T10:13:12.781126+0000","last_undegraded":"2024-04-25T10:13:12.781126+0000","last_fullsized":"2024-04-25T10:13:12.781126+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:37.642788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514651+0000","last_change":"2024-04-25T10:13:11.574680+0000","last_active":"2024-04-25T10:13:12.514651+0000","last_peered":"2024-04-25T10:13:12.514651+0000","last_clean":"2024-04-25T10:13:12.514651+0000","last_became_active":"2024-04-25T10:13:11.574085+0000","last_became_peered":"2024-04-25T10:13:11.574085+0000","last_unstale":"2024-04-25T10:13:12.514651+0000","last_undegraded":"2024-04-25T10:13:12.514651+0000","last_fullsized":"2024-04-25T10:13:12.514651+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.168445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810338+0000","last_change":"2024-04-25T10:13:10.496461+0000","last_active":"2024-04-25T10:13:12.810338+0000","last_peered":"2024-04-25T10:13:12.810338+0000","last_clean":"2024-04-25T10:13:12.810338+0000","last_became_active":"2024-04-25T10:13:10.495439+0000","last_became_peered":"2024-04-25T10:13:10.495439+0000","last_unstale":"2024-04-25T10:13:12.810338+0000","last_undegraded":"2024-04-25T10:13:12.810338+0000","last_fullsized":"2024-04-25T10:13:12.810338+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:32.382689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.761590+0000","last_change":"2024-04-25T10:13:11.525150+0000","last_active":"2024-04-25T10:13:13.761590+0000","last_peered":"2024-04-25T10:13:13.761590+0000","last_clean":"2024-04-25T10:13:13.761590+0000","last_became_active":"2024-04-25T10:13:11.524852+0000","last_became_peered":"2024-04-25T10:13:11.524852+0000","last_unstale":"2024-04-25T10:13:13.761590+0000","last_undegraded":"2024-04-25T10:13:13.761590+0000","last_fullsized":"2024-04-25T10:13:13.761590+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:36.746906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.759373+0000","last_change":"2024-04-25T10:13:10.504888+0000","last_active":"2024-04-25T10:13:13.759373+0000","last_peered":"2024-04-25T10:13:13.759373+0000","last_clean":"2024-04-25T10:13:13.759373+0000","last_became_active":"2024-04-25T10:13:10.504352+0000","last_became_peered":"2024-04-25T10:13:10.504352+0000","last_unstale":"2024-04-25T10:13:13.759373+0000","last_undegraded":"2024-04-25T10:13:13.759373+0000","last_fullsized":"2024-04-25T10:13:13.759373+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:51.056571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508240+0000","last_change":"2024-04-25T10:13:11.541243+0000","last_active":"2024-04-25T10:13:12.508240+0000","last_peered":"2024-04-25T10:13:12.508240+0000","last_clean":"2024-04-25T10:13:12.508240+0000","last_became_active":"2024-04-25T10:13:11.540480+0000","last_became_peered":"2024-04-25T10:13:11.540480+0000","last_unstale":"2024-04-25T10:13:12.508240+0000","last_undegraded":"2024-04-25T10:13:12.508240+0000","last_fullsized":"2024-04-25T10:13:12.508240+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:49.237836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"132'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509465+0000","last_change":"2024-04-25T10:13:10.505272+0000","last_active":"2024-04-25T10:13:12.509465+0000","last_peered":"2024-04-25T10:13:12.509465+0000","last_clean":"2024-04-25T10:13:12.509465+0000","last_became_active":"2024-04-25T10:13:10.504580+0000","last_became_peered":"2024-04-25T10:13:10.504580+0000","last_unstale":"2024-04-25T10:13:12.509465+0000","last_undegraded":"2024-04-25T10:13:12.509465+0000","last_fullsized":"2024-04-25T10:13:12.509465+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:55.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780865+0000","last_change":"2024-04-25T10:13:11.571071+0000","last_active":"2024-04-25T10:13:12.780865+0000","last_peered":"2024-04-25T10:13:12.780865+0000","last_clean":"2024-04-25T10:13:12.780865+0000","last_became_active":"2024-04-25T10:13:11.569555+0000","last_became_peered":"2024-04-25T10:13:11.569555+0000","last_unstale":"2024-04-25T10:13:12.780865+0000","last_undegraded":"2024-04-25T10:13:12.780865+0000","last_fullsized":"2024-04-25T10:13:12.780865+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:03.874094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.508513+0000","last_change":"2024-04-25T10:13:10.502034+0000","last_active":"2024-04-25T10:13:12.508513+0000","last_peered":"2024-04-25T10:13:12.508513+0000","last_clean":"2024-04-25T10:13:12.508513+0000","last_became_active":"2024-04-25T10:13:10.501199+0000","last_became_peered":"2024-04-25T10:13:10.501199+0000","last_unstale":"2024-04-25T10:13:12.508513+0000","last_undegraded":"2024-04-25T10:13:12.508513+0000","last_fullsized":"2024-04-25T10:13:12.508513+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:02.941844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.506669+0000","last_change":"2024-04-25T10:13:11.524061+0000","last_active":"2024-04-25T10:13:12.506669+0000","last_peered":"2024-04-25T10:13:12.506669+0000","last_clean":"2024-04-25T10:13:12.506669+0000","last_became_active":"2024-04-25T10:13:11.523588+0000","last_became_peered":"2024-04-25T10:13:11.523588+0000","last_unstale":"2024-04-25T10:13:12.506669+0000","last_undegraded":"2024-04-25T10:13:12.506669+0000","last_fullsized":"2024-04-25T10:13:12.506669+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:29.750424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810692+0000","last_change":"2024-04-25T10:13:10.501983+0000","last_active":"2024-04-25T10:13:12.810692+0000","last_peered":"2024-04-25T10:13:12.810692+0000","last_clean":"2024-04-25T10:13:12.810692+0000","last_became_active":"2024-04-25T10:13:10.501561+0000","last_became_peered":"2024-04-25T10:13:10.501561+0000","last_unstale":"2024-04-25T10:13:12.810692+0000","last_undegraded":"2024-04-25T10:13:12.810692+0000","last_fullsized":"2024-04-25T10:13:12.810692+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:53.326070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810602+0000","last_change":"2024-04-25T10:13:11.515291+0000","last_active":"2024-04-25T10:13:12.810602+0000","last_peered":"2024-04-25T10:13:12.810602+0000","last_clean":"2024-04-25T10:13:12.810602+0000","last_became_active":"2024-04-25T10:13:11.514943+0000","last_became_peered":"2024-04-25T10:13:11.514943+0000","last_unstale":"2024-04-25T10:13:12.810602+0000","last_undegraded":"2024-04-25T10:13:12.810602+0000","last_fullsized":"2024-04-25T10:13:12.810602+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:46.168445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.514140+0000","last_change":"2024-04-25T10:13:10.509756+0000","last_active":"2024-04-25T10:13:12.514140+0000","last_peered":"2024-04-25T10:13:12.514140+0000","last_clean":"2024-04-25T10:13:12.514140+0000","last_became_active":"2024-04-25T10:13:10.508989+0000","last_became_peered":"2024-04-25T10:13:10.508989+0000","last_unstale":"2024-04-25T10:13:12.514140+0000","last_undegraded":"2024-04-25T10:13:12.514140+0000","last_fullsized":"2024-04-25T10:13:12.514140+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:45.302496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.811001+0000","last_change":"2024-04-25T10:13:11.521300+0000","last_active":"2024-04-25T10:13:12.811001+0000","last_peered":"2024-04-25T10:13:12.811001+0000","last_clean":"2024-04-25T10:13:12.811001+0000","last_became_active":"2024-04-25T10:13:11.521002+0000","last_became_peered":"2024-04-25T10:13:11.521002+0000","last_unstale":"2024-04-25T10:13:12.811001+0000","last_undegraded":"2024-04-25T10:13:12.811001+0000","last_fullsized":"2024-04-25T10:13:12.811001+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:10.791929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781666+0000","last_change":"2024-04-25T10:13:10.520125+0000","last_active":"2024-04-25T10:13:12.781666+0000","last_peered":"2024-04-25T10:13:12.781666+0000","last_clean":"2024-04-25T10:13:12.781666+0000","last_became_active":"2024-04-25T10:13:10.519603+0000","last_became_peered":"2024-04-25T10:13:10.519603+0000","last_unstale":"2024-04-25T10:13:12.781666+0000","last_undegraded":"2024-04-25T10:13:12.781666+0000","last_fullsized":"2024-04-25T10:13:12.781666+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:42.431009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810604+0000","last_change":"2024-04-25T10:13:11.554463+0000","last_active":"2024-04-25T10:13:12.810604+0000","last_peered":"2024-04-25T10:13:12.810604+0000","last_clean":"2024-04-25T10:13:12.810604+0000","last_became_active":"2024-04-25T10:13:11.554189+0000","last_became_peered":"2024-04-25T10:13:11.554189+0000","last_unstale":"2024-04-25T10:13:12.810604+0000","last_undegraded":"2024-04-25T10:13:12.810604+0000","last_fullsized":"2024-04-25T10:13:12.810604+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:46.637364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.810630+0000","last_change":"2024-04-25T10:13:10.518854+0000","last_active":"2024-04-25T10:13:12.810630+0000","last_peered":"2024-04-25T10:13:12.810630+0000","last_clean":"2024-04-25T10:13:12.810630+0000","last_became_active":"2024-04-25T10:13:10.518358+0000","last_became_peered":"2024-04-25T10:13:10.518358+0000","last_unstale":"2024-04-25T10:13:12.810630+0000","last_undegraded":"2024-04-25T10:13:12.810630+0000","last_fullsized":"2024-04-25T10:13:12.810630+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:18.036486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781697+0000","last_change":"2024-04-25T10:13:11.570138+0000","last_active":"2024-04-25T10:13:12.781697+0000","last_peered":"2024-04-25T10:13:12.781697+0000","last_clean":"2024-04-25T10:13:12.781697+0000","last_became_active":"2024-04-25T10:13:11.569565+0000","last_became_peered":"2024-04-25T10:13:11.569565+0000","last_unstale":"2024-04-25T10:13:12.781697+0000","last_undegraded":"2024-04-25T10:13:12.781697+0000","last_fullsized":"2024-04-25T10:13:12.781697+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:42.493948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:13.760048+0000","last_change":"2024-04-25T10:13:10.508016+0000","last_active":"2024-04-25T10:13:13.760048+0000","last_peered":"2024-04-25T10:13:13.760048+0000","last_clean":"2024-04-25T10:13:13.760048+0000","last_became_active":"2024-04-25T10:13:10.507618+0000","last_became_peered":"2024-04-25T10:13:10.507618+0000","last_unstale":"2024-04-25T10:13:13.760048+0000","last_undegraded":"2024-04-25T10:13:13.760048+0000","last_fullsized":"2024-04-25T10:13:13.760048+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:57.522399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.509790+0000","last_change":"2024-04-25T10:13:11.527767+0000","last_active":"2024-04-25T10:13:12.509790+0000","last_peered":"2024-04-25T10:13:12.509790+0000","last_clean":"2024-04-25T10:13:12.509790+0000","last_became_active":"2024-04-25T10:13:11.527021+0000","last_became_peered":"2024-04-25T10:13:11.527021+0000","last_unstale":"2024-04-25T10:13:12.509790+0000","last_undegraded":"2024-04-25T10:13:12.509790+0000","last_fullsized":"2024-04-25T10:13:12.509790+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:10.469777+0000","last_clean_scrub_stamp":"2024-04-25T10:13:10.469777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:58.525502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.781388+0000","last_change":"2024-04-25T10:13:10.523416+0000","last_active":"2024-04-25T10:13:12.781388+0000","last_peered":"2024-04-25T10:13:12.781388+0000","last_clean":"2024-04-25T10:13:12.781388+0000","last_became_active":"2024-04-25T10:13:10.522665+0000","last_became_peered":"2024-04-25T10:13:10.522665+0000","last_unstale":"2024-04-25T10:13:12.781388+0000","last_undegraded":"2024-04-25T10:13:12.781388+0000","last_fullsized":"2024-04-25T10:13:12.781388+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:09.466674+0000","last_clean_scrub_stamp":"2024-04-25T10:13:09.466674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:44.313595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":380,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:13:12.780419+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:13:12.780419+0000","last_peered":"2024-04-25T10:13:12.780419+0000","last_clean":"2024-04-25T10:13:12.780419+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:13:12.780419+0000","last_undegraded":"2024-04-25T10:13:12.780419+0000","last_fullsized":"2024-04-25T10:13:12.780419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69484,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302356,"statfs":{"total":96636764160,"available":96565612544,"internally_reserved":0,"allocated":2260992,"data_stored":1363647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574993,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46692,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325148,"statfs":{"total":96636764160,"available":96588951552,"internally_reserved":0,"allocated":1658880,"data_stored":772249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574992,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1654784,"data_stored":771599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574992,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73052,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298788,"statfs":{"total":96636764160,"available":96561958912,"internally_reserved":0,"allocated":1650688,"data_stored":771577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574992,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73064,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298776,"statfs":{"total":96636764160,"available":96561946624,"internally_reserved":0,"allocated":1667072,"data_stored":773259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574992,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46692,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325148,"statfs":{"total":96636764160,"available":96588951552,"internally_reserved":0,"allocated":1658880,"data_stored":772193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59164,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312676,"statfs":{"total":96636764160,"available":96576180224,"internally_reserved":0,"allocated":1654784,"data_stored":771647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69476,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302364,"statfs":{"total":96636764160,"available":96565620736,"internally_reserved":0,"allocated":2252800,"data_stored":1362001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:20.266 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:13:20.266 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:13:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82beeab640 1 Processor -- start 2024-04-25T10:13:20.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82beeab640 1 -- start start 2024-04-25T10:13:20.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82beeab640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f82b8143ea0 0x7f82b80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82beeab640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b806eac0 0x7f82b80b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82beeab640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b815ac50 con 0x7f82b80b2150 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82beeab640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b80cd180 con 0x7f82b8143ea0 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82beeab640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b815aed0 con 0x7f82b806eac0 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f82b80b2150 0x7f82b806e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58558/0 (socket says 172.21.15.133:58558) 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b7fff640 1 -- 172.21.15.133:0/1475095361 learned_addr learned my addr 172.21.15.133:0/1475095361 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82bcc47640 1 --2- 172.21.15.133:0/1475095361 >> v2:172.21.15.162:3300/0 conn(0x7f82b8143ea0 0x7f82b80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82bd448640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b806eac0 0x7f82b80b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11291429 0 0) 0x7f82ac001120 con 0x7f82b80b2150 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82a0003530 con 0x7f82b80b2150 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3979862025 0 0) 0x7f82a0003530 con 0x7f82b80b2150 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b806eac0 msgr2=0x7f82b80b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b806eac0 0x7f82b80b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 >> v2:172.21.15.162:3300/0 conn(0x7f82b8143ea0 msgr2=0x7f82b80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 --2- 172.21.15.133:0/1475095361 >> v2:172.21.15.162:3300/0 conn(0x7f82b8143ea0 0x7f82b80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b815b1b0 con 0x7f82b80b2150 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f82ac001850 con 0x7f82b80b2150 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 >> v1:172.21.15.133:6789/0 conn(0x7f82b80b2150 legacy=0x7f82b806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b806eac0 0x7f82a00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a0003f40 0x7f82a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a0004d50 0x7f82a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80cd180 con 0x7f82a0004d50 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b815aed0 con 0x7f82a0003f40 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a000b9d0 con 0x7f82b806eac0 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82bd448640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a0004d50 0x7f82a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b7fff640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a0003f40 0x7f82a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82bd448640 1 -- 172.21.15.133:0/1475095361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b806eac0 msgr2=0x7f82a00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82bd448640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b806eac0 0x7f82a00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82bd448640 1 -- 172.21.15.133:0/1475095361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a0003f40 msgr2=0x7f82a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82bd448640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a0003f40 0x7f82a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82bd448640 1 -- 172.21.15.133:0/1475095361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a0003530 con 0x7f82a0004d50 2024-04-25T10:13:20.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.436+0000 7f82b7fff640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a0003f40 0x7f82a0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:20.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82bd448640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a0004d50 0x7f82a00051d0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f82b0010320 tx=0x7f82b001ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc17b5a500eb28a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000be00 con 0x7f82a0004d50 2024-04-25T10:13:20.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b000b0d0 con 0x7f82a0004d50 2024-04-25T10:13:20.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82b77fe640 1 -- 172.21.15.133:0/1475095361 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0009540 con 0x7f82a0004d50 2024-04-25T10:13:20.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82beeab640 1 -- 172.21.15.133:0/1475095361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a0004d50 msgr2=0x7f82a00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82beeab640 1 --2- 172.21.15.133:0/1475095361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82a0004d50 0x7f82a00051d0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f82b0010320 tx=0x7f82b001ee00 comp rx=0 tx=0).stop 2024-04-25T10:13:20.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82bd448640 1 -- 172.21.15.133:0/1475095361 reap_dead start 2024-04-25T10:13:20.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82beeab640 1 -- 172.21.15.133:0/1475095361 shutdown_connections 2024-04-25T10:13:20.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82beeab640 1 -- 172.21.15.133:0/1475095361 >> 172.21.15.133:0/1475095361 conn(0x7f82b8066f70 msgr2=0x7f82b806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82beeab640 1 -- 172.21.15.133:0/1475095361 shutdown_connections 2024-04-25T10:13:20.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82beeab640 1 -- 172.21.15.133:0/1475095361 wait complete. 2024-04-25T10:13:20.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82beeab640 1 Processor -- start 2024-04-25T10:13:20.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.440+0000 7f82beeab640 1 -- start start 2024-04-25T10:13:20.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82beeab640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a0004d50 0x7f82b81ee740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82beeab640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b8153ff0 0x7f82b8151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82beeab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8154450 0x7f82b80b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82beeab640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80cced0 con 0x7f82b8154450 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82beeab640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80ccd50 con 0x7f82a0004d50 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82beeab640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b815b9b0 con 0x7f82b8153ff0 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bd448640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8154450 0x7f82b80b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bd448640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8154450 0x7f82b80b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34546/0 (socket says 172.21.15.133:34546) 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bd448640 1 -- 172.21.15.133:0/392396854 learned_addr learned my addr 172.21.15.133:0/392396854 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bcc47640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a0004d50 0x7f82b81ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bd448640 1 -- 172.21.15.133:0/392396854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b8153ff0 msgr2=0x7f82b8151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82b7fff640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b8153ff0 0x7f82b8151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bd448640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b8153ff0 0x7f82b8151d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bd448640 1 -- 172.21.15.133:0/392396854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a0004d50 msgr2=0x7f82b81ee740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bd448640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a0004d50 0x7f82b81ee740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bd448640 1 -- 172.21.15.133:0/392396854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b8153530 con 0x7f82b8154450 2024-04-25T10:13:20.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bcc47640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a0004d50 0x7f82b81ee740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:20.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bd448640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8154450 0x7f82b80b3f50 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f82b00b1040 tx=0x7f82b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82b57fa640 1 -- 172.21.15.133:0/392396854 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000be00 con 0x7f82b8154450 2024-04-25T10:13:20.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82b57fa640 1 -- 172.21.15.133:0/392396854 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b0009c50 con 0x7f82b8154450 2024-04-25T10:13:20.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82beeab640 1 -- 172.21.15.133:0/392396854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b81527f0 con 0x7f82b8154450 2024-04-25T10:13:20.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82b57fa640 1 -- 172.21.15.133:0/392396854 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b00b8460 con 0x7f82b8154450 2024-04-25T10:13:20.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82beeab640 1 -- 172.21.15.133:0/392396854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b81537f0 con 0x7f82b8154450 2024-04-25T10:13:20.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82beeab640 1 -- 172.21.15.133:0/392396854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b8068e80 con 0x7f82b8154450 2024-04-25T10:13:20.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82b57fa640 1 -- 172.21.15.133:0/392396854 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82b00c0020 con 0x7f82b8154450 2024-04-25T10:13:20.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82b57fa640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82a4066f90 0x7f82a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82b57fa640 1 -- 172.21.15.133:0/392396854 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f82b0121a80 con 0x7f82b8154450 2024-04-25T10:13:20.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bcc47640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82a4066f90 0x7f82a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.444+0000 7f82bcc47640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82a4066f90 0x7f82a4069450 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f82a80047c0 tx=0x7f82a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.448+0000 7f82b57fa640 1 -- 172.21.15.133:0/392396854 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82b00c7020 con 0x7f82b8154450 2024-04-25T10:13:20.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.668+0000 7f82beeab640 1 -- 172.21.15.133:0/392396854 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f82b8157a60 con 0x7f82b8154450 2024-04-25T10:13:20.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.668+0000 7f82b57fa640 1 -- 172.21.15.133:0/392396854 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7f82b0018b70 con 0x7f82b8154450 2024-04-25T10:13:20.673 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:13:20.673 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:13:20.673 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:13:20.673 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:13:20.673 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:13:20.674 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 -- 172.21.15.133:0/392396854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82a4066f90 msgr2=0x7f82a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82a4066f90 0x7f82a4069450 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f82a80047c0 tx=0x7f82a8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 -- 172.21.15.133:0/392396854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8154450 msgr2=0x7f82b80b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8154450 0x7f82b80b3f50 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f82b00b1040 tx=0x7f82b000b960 comp rx=0 tx=0).stop 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 -- 172.21.15.133:0/392396854 shutdown_connections 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82a4066f90 0x7f82a4069450 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b8154450 0x7f82b80b3f50 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82b8153ff0 0x7f82b8151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 --2- 172.21.15.133:0/392396854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82a0004d50 0x7f82b81ee740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 -- 172.21.15.133:0/392396854 >> 172.21.15.133:0/392396854 conn(0x7f82b8066f70 msgr2=0x7f82b81431d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 -- 172.21.15.133:0/392396854 shutdown_connections 2024-04-25T10:13:20.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.672+0000 7f82beeab640 1 -- 172.21.15.133:0/392396854 wait complete. 2024-04-25T10:13:20.693 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:20.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124d0dc640 1 Processor -- start 2024-04-25T10:13:20.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124d0dc640 1 -- start start 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124d0dc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f124815dbf0 0x7f12480afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124d0dc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f12480d0f80 0x7f124806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124d0dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124806ac90 0x7f124806b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124d0dc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12480b3e40 con 0x7f12480d0f80 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124d0dc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124806f280 con 0x7f124815dbf0 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124d0dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12480b3880 con 0x7f124806ac90 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f12480d0f80 0x7f124806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f12480d0f80 0x7f124806ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34554/0 (socket says 172.21.15.133:34554) 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124659c640 1 -- 172.21.15.133:0/2942412154 learned_addr learned my addr 172.21.15.133:0/2942412154 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124759e640 1 --2- 172.21.15.133:0/2942412154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124806ac90 0x7f124806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f1246d9d640 1 --2- 172.21.15.133:0/2942412154 >> v2:172.21.15.162:3300/0 conn(0x7f124815dbf0 0x7f12480afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124659c640 1 -- 172.21.15.133:0/2942412154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124806ac90 msgr2=0x7f124806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124659c640 1 --2- 172.21.15.133:0/2942412154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124806ac90 0x7f124806b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124659c640 1 -- 172.21.15.133:0/2942412154 >> v2:172.21.15.162:3300/0 conn(0x7f124815dbf0 msgr2=0x7f12480afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124659c640 1 --2- 172.21.15.133:0/2942412154 >> v2:172.21.15.162:3300/0 conn(0x7f124815dbf0 0x7f12480afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.812+0000 7f124659c640 1 -- 172.21.15.133:0/2942412154 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12480b3b60 con 0x7f12480d0f80 2024-04-25T10:13:20.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124659c640 1 --2- 172.21.15.133:0/2942412154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12480d0f80 0x7f124806ed40 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f124806f510 tx=0x7f123001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2a498b53de1eea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1245d9b640 1 -- 172.21.15.133:0/2942412154 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12300b0070 con 0x7f12480d0f80 2024-04-25T10:13:20.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1245d9b640 1 -- 172.21.15.133:0/2942412154 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12300ab600 con 0x7f12480d0f80 2024-04-25T10:13:20.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1245d9b640 1 -- 172.21.15.133:0/2942412154 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12300ab9d0 con 0x7f12480d0f80 2024-04-25T10:13:20.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 -- 172.21.15.133:0/2942412154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12480d0f80 msgr2=0x7f124806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 --2- 172.21.15.133:0/2942412154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12480d0f80 0x7f124806ed40 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f124806f510 tx=0x7f123001c490 comp rx=0 tx=0).stop 2024-04-25T10:13:20.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 -- 172.21.15.133:0/2942412154 shutdown_connections 2024-04-25T10:13:20.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 --2- 172.21.15.133:0/2942412154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124806ac90 0x7f124806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 --2- 172.21.15.133:0/2942412154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12480d0f80 0x7f124806ed40 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 --2- 172.21.15.133:0/2942412154 >> v2:172.21.15.162:3300/0 conn(0x7f124815dbf0 0x7f12480afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 -- 172.21.15.133:0/2942412154 >> 172.21.15.133:0/2942412154 conn(0x7f1248072430 msgr2=0x7f1248072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 -- 172.21.15.133:0/2942412154 shutdown_connections 2024-04-25T10:13:20.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 -- 172.21.15.133:0/2942412154 wait complete. 2024-04-25T10:13:20.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 Processor -- start 2024-04-25T10:13:20.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 -- start start 2024-04-25T10:13:20.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124806ac90 0x7f124814bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12480d0f80 0x7f124815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124815dbf0 0x7f124815e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1248162280 con 0x7f124806ac90 2024-04-25T10:13:20.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1248162100 con 0x7f12480d0f80 2024-04-25T10:13:20.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124806fc90 con 0x7f124815dbf0 2024-04-25T10:13:20.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1246d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124806ac90 0x7f124814bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1246d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124806ac90 0x7f124814bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34570/0 (socket says 172.21.15.133:34570) 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1246d9d640 1 -- 172.21.15.133:0/2833973826 learned_addr learned my addr 172.21.15.133:0/2833973826 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1246d9d640 1 -- 172.21.15.133:0/2833973826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124815dbf0 msgr2=0x7f124815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1246d9d640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124815dbf0 0x7f124815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1246d9d640 1 -- 172.21.15.133:0/2833973826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12480d0f80 msgr2=0x7f124815dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1246d9d640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12480d0f80 0x7f124815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1246d9d640 1 -- 172.21.15.133:0/2833973826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1248150b60 con 0x7f124806ac90 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f1246d9d640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124806ac90 0x7f124814bb80 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f124000e8a0 tx=0x7f124000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f123f7fe640 1 -- 172.21.15.133:0/2833973826 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240017070 con 0x7f124806ac90 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f123f7fe640 1 -- 172.21.15.133:0/2833973826 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124000c940 con 0x7f124806ac90 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f124d0dc640 1 -- 172.21.15.133:0/2833973826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1248158c20 con 0x7f124806ac90 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.816+0000 7f123f7fe640 1 -- 172.21.15.133:0/2833973826 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124000ccf0 con 0x7f124806ac90 2024-04-25T10:13:20.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.820+0000 7f124d0dc640 1 -- 172.21.15.133:0/2833973826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1248157ef0 con 0x7f124806ac90 2024-04-25T10:13:20.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.820+0000 7f124d0dc640 1 -- 172.21.15.133:0/2833973826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1248067c70 con 0x7f124806ac90 2024-04-25T10:13:20.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.820+0000 7f123f7fe640 1 -- 172.21.15.133:0/2833973826 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f124001a070 con 0x7f124806ac90 2024-04-25T10:13:20.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.820+0000 7f123f7fe640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1228066f90 0x7f1228069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.820+0000 7f123f7fe640 1 -- 172.21.15.133:0/2833973826 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1240085110 con 0x7f124806ac90 2024-04-25T10:13:20.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.820+0000 7f124659c640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1228066f90 0x7f1228069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.820+0000 7f124659c640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1228066f90 0x7f1228069450 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f12300ad000 tx=0x7f1230004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:20.824+0000 7f123f7fe640 1 -- 172.21.15.133:0/2833973826 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1240055f90 con 0x7f124806ac90 2024-04-25T10:13:21.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.016+0000 7f124d0dc640 1 -- 172.21.15.133:0/2833973826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12480b5750 con 0x7f124806ac90 2024-04-25T10:13:21.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.016+0000 7f123f7fe640 1 -- 172.21.15.133:0/2833973826 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f1240055890 con 0x7f124806ac90 2024-04-25T10:13:21.020 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:21.020 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:13:21.020 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:13:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 -- 172.21.15.133:0/2833973826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1228066f90 msgr2=0x7f1228069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1228066f90 0x7f1228069450 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f12300ad000 tx=0x7f1230004740 comp rx=0 tx=0).stop 2024-04-25T10:13:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 -- 172.21.15.133:0/2833973826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124806ac90 msgr2=0x7f124814bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124806ac90 0x7f124814bb80 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f124000e8a0 tx=0x7f124000eca0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 -- 172.21.15.133:0/2833973826 shutdown_connections 2024-04-25T10:13:21.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f124815dbf0 0x7f124815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12480d0f80 0x7f124815dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1228066f90 0x7f1228069450 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 --2- 172.21.15.133:0/2833973826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f124806ac90 0x7f124814bb80 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 -- 172.21.15.133:0/2833973826 >> 172.21.15.133:0/2833973826 conn(0x7f1248072430 msgr2=0x7f12480cdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 -- 172.21.15.133:0/2833973826 shutdown_connections 2024-04-25T10:13:21.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.020+0000 7f124d0dc640 1 -- 172.21.15.133:0/2833973826 wait complete. 2024-04-25T10:13:21.442 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:10.856393+0000', 'modified': '2024-04-25T10:13:13.860376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9495}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9495': {'gid': 9495, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/2351700644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 2351700644}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 2351700644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:21.442 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:13:21.442 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:13:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2f947640 1 Processor -- start 2024-04-25T10:13:21.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2f947640 1 -- start start 2024-04-25T10:13:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2f947640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd28142920 0x7fdd28143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2f947640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdd2815d9f0 0x7fdd2815de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2f947640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd28140ea0 con 0x7fdd28140a40 2024-04-25T10:13:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2f947640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd280cf280 con 0x7fdd2815d9f0 2024-04-25T10:13:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2f947640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2806d440 con 0x7fdd28142920 2024-04-25T10:13:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2cee2640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdd28140a40 0x7fdd280ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58560/0 (socket says 172.21.15.133:58560) 2024-04-25T10:13:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2cee2640 1 -- 172.21.15.133:0/1621085395 learned_addr learned my addr 172.21.15.133:0/1621085395 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:21.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389869290 0 0) 0x7fdd28140ea0 con 0x7fdd28140a40 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd10004640 con 0x7fdd28140a40 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2dee4640 1 --2- 172.21.15.133:0/1621085395 >> v2:172.21.15.162:3300/0 conn(0x7fdd2815d9f0 0x7fdd2815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd28142920 0x7fdd28143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 521048923 0 0) 0x7fdd10004640 con 0x7fdd28140a40 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd28142920 msgr2=0x7fdd28143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd28142920 0x7fdd28143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 >> v2:172.21.15.162:3300/0 conn(0x7fdd2815d9f0 msgr2=0x7fdd2815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 --2- 172.21.15.133:0/1621085395 >> v2:172.21.15.162:3300/0 conn(0x7fdd2815d9f0 0x7fdd2815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2806d720 con 0x7fdd28140a40 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fdd1c001680 con 0x7fdd28140a40 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 >> v1:172.21.15.133:6789/0 conn(0x7fdd28140a40 legacy=0x7fdd280ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd10004ad0 0x7fdd10004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd28142920 0x7fdd10005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd10005e60 0x7fdd100062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd280cf280 con 0x7fdd10004ad0 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2806d440 con 0x7fdd10005e60 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1000cb90 con 0x7fdd28142920 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd10004ad0 0x7fdd10004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2cee2640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd28142920 0x7fdd10005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2dee4640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd10005e60 0x7fdd100062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2d6e3640 1 -- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd28142920 msgr2=0x7fdd10005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd28142920 0x7fdd10005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2d6e3640 1 -- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd10005e60 msgr2=0x7fdd100062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd10005e60 0x7fdd100062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2d6e3640 1 -- 172.21.15.133:0/1621085395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd10004640 con 0x7fdd10004ad0 2024-04-25T10:13:21.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.572+0000 7fdd2dee4640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd10005e60 0x7fdd100062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd10004ad0 0x7fdd10004ef0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000ba30 tx=0x7fdd2001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724b2390fced3865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd200b2070 con 0x7fdd10004ad0 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd200ad410 con 0x7fdd10004ad0 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd27fff640 1 -- 172.21.15.133:0/1621085395 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000b600 con 0x7fdd10004ad0 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd2f947640 1 -- 172.21.15.133:0/1621085395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd10004ad0 msgr2=0x7fdd10004ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd2f947640 1 --2- 172.21.15.133:0/1621085395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd10004ad0 0x7fdd10004ef0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000ba30 tx=0x7fdd2001c6a0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd2dee4640 1 -- 172.21.15.133:0/1621085395 reap_dead start 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd2f947640 1 -- 172.21.15.133:0/1621085395 shutdown_connections 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd2f947640 1 -- 172.21.15.133:0/1621085395 >> 172.21.15.133:0/1621085395 conn(0x7fdd28066f70 msgr2=0x7fdd2806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd2f947640 1 -- 172.21.15.133:0/1621085395 shutdown_connections 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd2f947640 1 -- 172.21.15.133:0/1621085395 wait complete. 2024-04-25T10:13:21.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd2f947640 1 Processor -- start 2024-04-25T10:13:21.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.576+0000 7fdd2f947640 1 -- start start 2024-04-25T10:13:21.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2f947640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd10004ad0 0x7fdd2813fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2f947640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd280c7c50 0x7fdd280c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2f947640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd280ad180 0x7fdd280ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2f947640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2806e2b0 con 0x7fdd280c7c50 2024-04-25T10:13:21.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2f947640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2806e130 con 0x7fdd280ad180 2024-04-25T10:13:21.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2f947640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2806e430 con 0x7fdd10004ad0 2024-04-25T10:13:21.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2cee2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd280c7c50 0x7fdd280c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2cee2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd280c7c50 0x7fdd280c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34596/0 (socket says 172.21.15.133:34596) 2024-04-25T10:13:21.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2cee2640 1 -- 172.21.15.133:0/3615132101 learned_addr learned my addr 172.21.15.133:0/3615132101 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:21.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2dee4640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd280ad180 0x7fdd280ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2cee2640 1 -- 172.21.15.133:0/3615132101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd10004ad0 msgr2=0x7fdd2813fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2cee2640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd10004ad0 0x7fdd2813fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2cee2640 1 -- 172.21.15.133:0/3615132101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd280ad180 msgr2=0x7fdd280ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2cee2640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd280ad180 0x7fdd280ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2cee2640 1 -- 172.21.15.133:0/3615132101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd280ce4a0 con 0x7fdd280c7c50 2024-04-25T10:13:21.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2cee2640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd280c7c50 0x7fdd280c80d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00e7f0 tx=0x7fdd1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd25ffb640 1 -- 172.21.15.133:0/3615132101 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00c870 con 0x7fdd280c7c50 2024-04-25T10:13:21.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd25ffb640 1 -- 172.21.15.133:0/3615132101 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c018070 con 0x7fdd280c7c50 2024-04-25T10:13:21.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2f947640 1 -- 172.21.15.133:0/3615132101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd280c5400 con 0x7fdd280c7c50 2024-04-25T10:13:21.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd25ffb640 1 -- 172.21.15.133:0/3615132101 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00cc50 con 0x7fdd280c7c50 2024-04-25T10:13:21.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2f947640 1 -- 172.21.15.133:0/3615132101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd280c59c0 con 0x7fdd280c7c50 2024-04-25T10:13:21.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd25ffb640 1 -- 172.21.15.133:0/3615132101 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd1c01d460 con 0x7fdd280c7c50 2024-04-25T10:13:21.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd25ffb640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd14066f90 0x7fdd14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd25ffb640 1 -- 172.21.15.133:0/3615132101 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd1c085000 con 0x7fdd280c7c50 2024-04-25T10:13:21.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd14066f90 0x7fdd14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.580+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd14066f90 0x7fdd14069450 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000ba00 tx=0x7fdd20004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 Processor -- start 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- start start 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558346469920 0x558346469d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x55834641f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834641ff30 0x558346420310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583463075f0 con 0x55834646a240 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834626eda0 con 0x55834641ff30 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583461f7150 con 0x558346469920 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd2cee2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x55834641f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd2cee2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x55834641f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34606/0 (socket says 172.21.15.133:34606) 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd2cee2640 1 -- 172.21.15.133:0/2633981704 learned_addr learned my addr 172.21.15.133:0/2633981704 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd2cee2640 1 -- 172.21.15.133:0/2633981704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558346469920 msgr2=0x558346469d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd2cee2640 1 --2- 172.21.15.133:0/2633981704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558346469920 0x558346469d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd2cee2640 1 -- 172.21.15.133:0/2633981704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834641ff30 msgr2=0x558346420310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd2cee2640 1 --2- 172.21.15.133:0/2633981704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834641ff30 0x558346420310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd2cee2640 1 -- 172.21.15.133:0/2633981704 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558346210ca0 con 0x55834646a240 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd2cee2640 1 --2- 172.21.15.133:0/2633981704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x55834641f9f0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c051be0 tx=0x7fdd1c055f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a7bf6c397556ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd0effd640 1 -- 172.21.15.133:0/2633981704 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c06d070 con 0x55834646a240 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd0effd640 1 -- 172.21.15.133:0/2633981704 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c0598d0 con 0x55834646a240 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd0effd640 1 -- 172.21.15.133:0/2633981704 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c05de90 con 0x55834646a240 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2633981704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 msgr2=0x55834641f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/2633981704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x55834641f9f0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c051be0 tx=0x7fdd1c055f40 comp rx=0 tx=0).stop 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2633981704 shutdown_connections 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/2633981704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834641ff30 0x558346420310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/2633981704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x55834641f9f0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/2633981704 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558346469920 0x558346469d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2633981704 >> 172.21.15.133:0/2633981704 conn(0x5583464262a0 msgr2=0x55834641e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2633981704 shutdown_connections 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2633981704 wait complete. 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- start start 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 Processor -- start 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55834641ff30 0x558346470e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558346469920 0x5583464713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x558346423aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558346229990 con 0x55834646a240 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834626ae40 con 0x558346469920 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583461fa900 con 0x55834641ff30 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.584+0000 7fdd32ab4000 10 client.0 _ll_register_callbacks cb 0x5583463221c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2dee4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x558346423aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2dee4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x558346423aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34610/0 (socket says 172.21.15.133:34610) 2024-04-25T10:13:21.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2dee4640 1 -- 172.21.15.133:0/3378229525 learned_addr learned my addr 172.21.15.133:0/3378229525 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:21.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2cee2640 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558346469920 0x5583464713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55834641ff30 0x558346470e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2dee4640 1 -- 172.21.15.133:0/3378229525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55834641ff30 msgr2=0x558346470e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2dee4640 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55834641ff30 0x558346470e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2dee4640 1 -- 172.21.15.133:0/3378229525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558346469920 msgr2=0x5583464713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2dee4640 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558346469920 0x5583464713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2dee4640 1 -- 172.21.15.133:0/3378229525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558346227630 con 0x55834646a240 2024-04-25T10:13:21.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2dee4640 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x558346423aa0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fdd2815ddd0 tx=0x7fdd180046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438778ac57ea3195 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd07fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:13:21.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd07fff640 1 -- 172.21.15.133:0/3378229525 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18007930 con 0x55834646a240 2024-04-25T10:13:21.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 -- 172.21.15.133:0/3378229525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd1c0598d0 con 0x55834646a240 2024-04-25T10:13:21.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd07fff640 1 -- 172.21.15.133:0/3378229525 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd18007ad0 con 0x55834646a240 2024-04-25T10:13:21.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd07fff640 1 -- 172.21.15.133:0/3378229525 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd18022050 con 0x55834646a240 2024-04-25T10:13:21.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 10 client.9669 fetch_fsmap learned FSMap version 79 2024-04-25T10:13:21.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 -- 172.21.15.133:0/3378229525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5583461f73a0 con 0x55834646a240 2024-04-25T10:13:21.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd07fff640 1 -- 172.21.15.133:0/3378229525 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c0598d0 con 0x55834646a240 2024-04-25T10:13:21.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd07fff640 1 -- 172.21.15.133:0/3378229525 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdd18025020 con 0x55834646a240 2024-04-25T10:13:21.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 10 client.9669 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:13:21.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 10 client.9669 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 20 client.9669 populate_metadata read hostname 'smithi133' 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x558346425160 0x55834647aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 4 client.9669 mds_command: new command op to 9495 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 -- 172.21.15.133:0/3378229525 --> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55834620aa50 con 0x558346425160 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x558346425160 0x55834647aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd07fff640 10 client.9669 ms_handle_connect on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x558346425160 0x55834647aec0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2651e74cfd37100 server_cookie=baaaa55fc7f6ba6a in_seq=0 out_seq=0 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd07fff640 1 -- 172.21.15.133:0/3378229525 <== mds.0 v2:172.21.15.133:6836/2351700644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55834620aa50 con 0x558346425160 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd07fff640 10 client.9669 handle_command_reply: tid=0 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 client.9669 shutdown 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 -- 172.21.15.133:0/3378229525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 msgr2=0x558346423aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x558346423aa0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fdd2815ddd0 tx=0x7fdd180046e0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 -- 172.21.15.133:0/3378229525 shutdown_connections 2024-04-25T10:13:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x558346425160 0x55834647aec0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834646a240 0x558346423aa0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558346469920 0x5583464713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/3378229525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55834641ff30 0x558346470e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd07fff640 0 client.9669 ms_handle_reset on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 -- 172.21.15.133:0/3378229525 >> 172.21.15.133:0/3378229525 conn(0x5583464262a0 msgr2=0x55834641e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 -- 172.21.15.133:0/3378229525 shutdown_connections 2024-04-25T10:13:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 1 -- 172.21.15.133:0/3378229525 wait complete. 2024-04-25T10:13:21.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.588+0000 7fdd32ab4000 20 client.9669 trim_cache size 0 max 16384 2024-04-25T10:13:21.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd32ab4000 1 Processor -- start 2024-04-25T10:13:21.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd32ab4000 1 -- start start 2024-04-25T10:13:21.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834641ff30 0x5583464cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558346469920 0x5583464cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834646a240 0x5583464cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28200bb0 con 0x55834641ff30 2024-04-25T10:13:21.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28200030 con 0x55834646a240 2024-04-25T10:13:21.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd282005b0 con 0x558346469920 2024-04-25T10:13:21.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd2d6e3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834641ff30 0x5583464cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd2d6e3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834641ff30 0x5583464cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34622/0 (socket says 172.21.15.133:34622) 2024-04-25T10:13:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd2d6e3640 1 -- 172.21.15.133:0/852898488 learned_addr learned my addr 172.21.15.133:0/852898488 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd2d6e3640 1 -- 172.21.15.133:0/852898488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558346469920 msgr2=0x5583464cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/852898488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558346469920 0x5583464cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd2d6e3640 1 -- 172.21.15.133:0/852898488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834646a240 msgr2=0x5583464cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/852898488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834646a240 0x5583464cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd2d6e3640 1 -- 172.21.15.133:0/852898488 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55834646a070 con 0x55834641ff30 2024-04-25T10:13:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/852898488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834641ff30 0x5583464cd550 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fdd2001c6e0 tx=0x7fdd200a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbea45a5b378c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.604+0000 7fdd0e7fc640 1 -- 172.21.15.133:0/852898488 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd200ca030 con 0x55834641ff30 2024-04-25T10:13:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd0e7fc640 1 -- 172.21.15.133:0/852898488 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd200c5430 con 0x55834641ff30 2024-04-25T10:13:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd0e7fc640 1 -- 172.21.15.133:0/852898488 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd200b2040 con 0x55834641ff30 2024-04-25T10:13:21.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- 172.21.15.133:0/852898488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834641ff30 msgr2=0x5583464cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/852898488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834641ff30 0x5583464cd550 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fdd2001c6e0 tx=0x7fdd200a6000 comp rx=0 tx=0).stop 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- 172.21.15.133:0/852898488 shutdown_connections 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/852898488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834646a240 0x5583464cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/852898488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558346469920 0x5583464cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/852898488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55834641ff30 0x5583464cd550 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- 172.21.15.133:0/852898488 >> 172.21.15.133:0/852898488 conn(0x5583464262a0 msgr2=0x558346468730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- 172.21.15.133:0/852898488 shutdown_connections 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- 172.21.15.133:0/852898488 wait complete. 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- start start 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 Processor -- start 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834641ff30 0x558346475630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558346469920 0x558346475b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55834646a240 0x5583464b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583462733d0 con 0x558346469920 2024-04-25T10:13:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834632ad50 con 0x55834641ff30 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x558346209980 con 0x55834646a240 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 10 client.0 _ll_register_callbacks cb 0x7fdd1c001690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2cee2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558346469920 0x558346475b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2cee2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558346469920 0x558346475b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34624/0 (socket says 172.21.15.133:34624) 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2cee2640 1 -- 172.21.15.133:0/2691371683 learned_addr learned my addr 172.21.15.133:0/2691371683 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834641ff30 0x558346475630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2cee2640 1 -- 172.21.15.133:0/2691371683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55834646a240 msgr2=0x5583464b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2cee2640 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55834646a240 0x5583464b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2cee2640 1 -- 172.21.15.133:0/2691371683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834641ff30 msgr2=0x558346475630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2cee2640 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834641ff30 0x558346475630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2cee2640 1 -- 172.21.15.133:0/2691371683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583462f0600 con 0x558346469920 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2cee2640 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558346469920 0x558346475b70 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c001740 tx=0x7fdd1c083270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9a14c6bb2c9b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd07fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd07fff640 1 -- 172.21.15.133:0/2691371683 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c06d040 con 0x558346469920 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd07fff640 1 -- 172.21.15.133:0/2691371683 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd1c06d1e0 con 0x558346469920 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2691371683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd200c5430 con 0x558346469920 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd07fff640 1 -- 172.21.15.133:0/2691371683 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd1c0830d0 con 0x558346469920 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 10 client.9675 fetch_fsmap learned FSMap version 79 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2691371683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5583462039b0 con 0x558346469920 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd07fff640 1 -- 172.21.15.133:0/2691371683 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd200c5430 con 0x558346469920 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd07fff640 1 -- 172.21.15.133:0/2691371683 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdd1c051df0 con 0x558346469920 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 10 client.9675 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 10 client.9675 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 20 client.9675 populate_metadata read hostname 'smithi133' 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5583464b6380 0x5583464b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 4 client.9675 mds_command: new command op to 9495 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2691371683 --> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55834627c2e0 con 0x5583464b6380 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5583464b6380 0x5583464b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd2d6e3640 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5583464b6380 0x5583464b6c20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1f22389f830f9ac server_cookie=6279a193b4d8dc64 in_seq=0 out_seq=0 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd07fff640 10 client.9675 ms_handle_connect on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd07fff640 1 -- 172.21.15.133:0/2691371683 <== mds.0 v2:172.21.15.133:6836/2351700644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55834627c2e0 con 0x5583464b6380 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd07fff640 10 client.9675 handle_command_reply: tid=0 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 client.9675 shutdown 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2691371683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558346469920 msgr2=0x558346475b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558346469920 0x558346475b70 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c001740 tx=0x7fdd1c083270 comp rx=0 tx=0).stop 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2691371683 shutdown_connections 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55834646a240 0x5583464b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5583464b6380 0x5583464b6c20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558346469920 0x558346475b70 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 --2- 172.21.15.133:0/2691371683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55834641ff30 0x558346475630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd07fff640 0 client.9675 ms_handle_reset on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.608+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2691371683 >> 172.21.15.133:0/2691371683 conn(0x5583464262a0 msgr2=0x558346467b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2691371683 shutdown_connections 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd32ab4000 1 -- 172.21.15.133:0/2691371683 wait complete. 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd32ab4000 20 client.9675 trim_cache size 0 max 16384 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 9495, 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 10.113314203, 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 79, 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 132, 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 132, 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 15.493461610000001 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 -- 172.21.15.133:0/3615132101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd14066f90 msgr2=0x7fdd14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd14066f90 0x7fdd14069450 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000ba00 tx=0x7fdd20004740 comp rx=0 tx=0).stop 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 -- 172.21.15.133:0/3615132101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd280c7c50 msgr2=0x7fdd280c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd280c7c50 0x7fdd280c80d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00e7f0 tx=0x7fdd1c00ecb0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 -- 172.21.15.133:0/3615132101 shutdown_connections 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd280ad180 0x7fdd280ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd14066f90 0x7fdd14069450 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd280c7c50 0x7fdd280c80d0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 --2- 172.21.15.133:0/3615132101 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd10004ad0 0x7fdd2813fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 -- 172.21.15.133:0/3615132101 >> 172.21.15.133:0/3615132101 conn(0x7fdd28066f70 msgr2=0x7fdd280b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 -- 172.21.15.133:0/3615132101 shutdown_connections 2024-04-25T10:13:21.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.612+0000 7fdd2f947640 1 -- 172.21.15.133:0/3615132101 wait complete. 2024-04-25T10:13:21.629 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:21.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a12a5c640 1 Processor -- start 2024-04-25T10:13:21.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a12a5c640 1 -- start start 2024-04-25T10:13:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a12a5c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2a0c0b1f70 0x7f2a0c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a12a5c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0c0c61d0 con 0x7f2a0c15c9a0 2024-04-25T10:13:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a12a5c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0c0c64b0 con 0x7f2a0c0ce330 2024-04-25T10:13:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a12a5c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0c06d4f0 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a0bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2a0c15c9a0 0x7f2a0c0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58568/0 (socket says 172.21.15.133:58568) 2024-04-25T10:13:21.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a0bfff640 1 -- 172.21.15.133:0/600311548 learned_addr learned my addr 172.21.15.133:0/600311548 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:21.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a03fff640 1 --2- 172.21.15.133:0/600311548 >> v2:172.21.15.162:3300/0 conn(0x7f2a0c0b1f70 0x7f2a0c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a0b7fe640 1 -- 172.21.15.133:0/600311548 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315191083 0 0) 0x7f2a0c0c61d0 con 0x7f2a0c15c9a0 2024-04-25T10:13:21.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a0b7fe640 1 -- 172.21.15.133:0/600311548 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29ec003530 con 0x7f2a0c15c9a0 2024-04-25T10:13:21.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a03fff640 1 -- 172.21.15.133:0/600311548 >> v1:172.21.15.162:6789/0 conn(0x7f2a0c0ce330 legacy=0x7f2a0c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a03fff640 1 -- 172.21.15.133:0/600311548 >> v1:172.21.15.133:6789/0 conn(0x7f2a0c15c9a0 legacy=0x7f2a0c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a0b7fe640 1 -- 172.21.15.133:0/600311548 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3735763149 0 0) 0x7f29ec003530 con 0x7f2a0c15c9a0 2024-04-25T10:13:21.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.740+0000 7f2a03fff640 1 -- 172.21.15.133:0/600311548 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c06d7d0 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a03fff640 1 --2- 172.21.15.133:0/600311548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0b1f70 0x7f2a0c06dcf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f29f40097c0 tx=0x7f29f401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=969703624e64c7fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a0b7fe640 1 -- 172.21.15.133:0/600311548 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40b2070 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a0b7fe640 1 -- 172.21.15.133:0/600311548 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f40a9070 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a0b7fe640 1 -- 172.21.15.133:0/600311548 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc001680 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 -- 172.21.15.133:0/600311548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0b1f70 msgr2=0x7f2a0c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 --2- 172.21.15.133:0/600311548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0b1f70 0x7f2a0c06dcf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f29f40097c0 tx=0x7f29f401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 -- 172.21.15.133:0/600311548 shutdown_connections 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 --2- 172.21.15.133:0/600311548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c0b1f70 0x7f2a0c06dcf0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 -- 172.21.15.133:0/600311548 >> 172.21.15.133:0/600311548 conn(0x7f2a0c067000 msgr2=0x7f2a0c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 -- 172.21.15.133:0/600311548 shutdown_connections 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 -- 172.21.15.133:0/600311548 wait complete. 2024-04-25T10:13:21.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 Processor -- start 2024-04-25T10:13:21.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 -- start start 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a0c0b1f70 0x7f2a0c0ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a0c0ce330 0x7f2a0c0b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c15c9a0 0x7f2a0c147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c15cf00 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c15cd80 con 0x7f2a0c15c9a0 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a12a5c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c169de0 con 0x7f2a0c0ce330 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.744+0000 7f2a0bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a0c0b1f70 0x7f2a0c0ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a0bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a0c0b1f70 0x7f2a0c0ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34636/0 (socket says 172.21.15.133:34636) 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a0bfff640 1 -- 172.21.15.133:0/2288067905 learned_addr learned my addr 172.21.15.133:0/2288067905 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a03fff640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a0c0ce330 0x7f2a0c0b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a10ff9640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c15c9a0 0x7f2a0c147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a0bfff640 1 -- 172.21.15.133:0/2288067905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a0c0ce330 msgr2=0x7f2a0c0b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a0bfff640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a0c0ce330 0x7f2a0c0b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a0bfff640 1 -- 172.21.15.133:0/2288067905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c15c9a0 msgr2=0x7f2a0c147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a0bfff640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c15c9a0 0x7f2a0c147f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a0bfff640 1 -- 172.21.15.133:0/2288067905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c0c9580 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a0bfff640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a0c0b1f70 0x7f2a0c0ca340 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00c6f0 tx=0x7f29fc00caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a097fa640 1 -- 172.21.15.133:0/2288067905 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc005cf0 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a12a5c640 1 -- 172.21.15.133:0/2288067905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c0c9860 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a097fa640 1 -- 172.21.15.133:0/2288067905 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc007070 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a097fa640 1 -- 172.21.15.133:0/2288067905 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0074a0 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a12a5c640 1 -- 172.21.15.133:0/2288067905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c0c9e20 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a12a5c640 1 -- 172.21.15.133:0/2288067905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0c068ef0 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a097fa640 1 -- 172.21.15.133:0/2288067905 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29fc01b400 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a097fa640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29f0066f60 0x7f29f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a03fff640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29f0066f60 0x7f29f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a097fa640 1 -- 172.21.15.133:0/2288067905 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29fc084f90 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.748+0000 7f2a03fff640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29f0066f60 0x7f29f0069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f29f40af000 tx=0x7f29f40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.752+0000 7f2a097fa640 1 -- 172.21.15.133:0/2288067905 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29fc055ea0 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 -- 172.21.15.133:0/2288067905 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2a0c15e840 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a097fa640 1 -- 172.21.15.133:0/2288067905 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) v1 ==== 53+0+73 (secure 0 0 0) 0x7f29fc0557a0 con 0x7f2a0c0b1f70 2024-04-25T10:13:21.946 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 -- 172.21.15.133:0/2288067905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29f0066f60 msgr2=0x7f29f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29f0066f60 0x7f29f0069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f29f40af000 tx=0x7f29f40a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 -- 172.21.15.133:0/2288067905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a0c0b1f70 msgr2=0x7f2a0c0ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a0c0b1f70 0x7f2a0c0ca340 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00c6f0 tx=0x7f29fc00caf0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 -- 172.21.15.133:0/2288067905 shutdown_connections 2024-04-25T10:13:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a0c15c9a0 0x7f2a0c147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2a0c0ce330 0x7f2a0c0b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f29f0066f60 0x7f29f0069420 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 --2- 172.21.15.133:0/2288067905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a0c0b1f70 0x7f2a0c0ca340 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 -- 172.21.15.133:0/2288067905 >> 172.21.15.133:0/2288067905 conn(0x7f2a0c067000 msgr2=0x7f2a0c15de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 -- 172.21.15.133:0/2288067905 shutdown_connections 2024-04-25T10:13:21.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:21.944+0000 7f2a12a5c640 1 -- 172.21.15.133:0/2288067905 wait complete. 2024-04-25T10:13:21.963 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:13:21.963 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:21.963 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:13:21.963 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:21.964 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:21.964 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:13:21.964 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:13:21.964 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:21.964 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:21.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:21.964 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:13:21.970 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:21.970 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:21.970 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:21.970 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:21.970 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:21.970 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:21.970 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:21.970 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41830sec preferred_lft 41830sec 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:13:21.971 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:21.972 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:13:21.972 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:21.972 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:21.972 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:21.972 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:21.972 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:21.972 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:13:21.972 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:21.972 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:21.972 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:21.972 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:13:21.973 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:21.973 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:21.973 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:21.973 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:13:21.973 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:21.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:21.973 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:22.023 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:13:22.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:22.024 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:13:22.071 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:13:22.072 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:22.072 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:13:22.072 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:13:22.072 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:13:22.072 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:13:22.072 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:13:22.072 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:22.072 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:22.072 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:22.072 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:22.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:22.073 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:13:22.073 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:22.073 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:13:22.073 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:13:22.074 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:13:22.074 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:22.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:22.141 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:13:22.193 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:22.194 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:22.194 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:22.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:22.194 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:13:22.253 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:13:22.253 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:22.253 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:22.302 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:22.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:22.302 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:22.349 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:13:22.407 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:22.455 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:13:22.455 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:13:22.455 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:13:22.455 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:13:22.455 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:13:22.455 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:13:22.455 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:13:22.455 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:13:22.455 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:13:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:13:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:13:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:13:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:13:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:13:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:13:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:13:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:13:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:22.457 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:22.458 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:22.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:22.459 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:13:22.505 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:13:22.506 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:13:22.551 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:13:22.565 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:22.614 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:22.615 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:13:22.617 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:22.617 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:22.617 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:22.617 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:22.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:22.617 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:13:22.628 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:13:22.624+0000 7f96eb5d0580 -1 init, newargv = 0x56220c3fa2c0 newargc=15 2024-04-25T10:13:22.628 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[81613]: starting ceph client 2024-04-25T10:13:22.634 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[81613]: starting fuse 2024-04-25T10:13:23.312 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:23.315 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:23.734 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:13:23.750 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:23.800 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:13:23.800 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:13:23.800 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:13:23.800 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:13:23.800 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:13:23.800 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:13:23.801 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:23.802 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:23.803 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:23.803 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:23.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:23.804 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:13:23.850 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:13:23.850 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:13:23.850 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:13:23.850 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:13:23.850 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:13:23.850 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:13:23.851 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:13:23.852 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:13:23.852 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:13:23.852 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:13:23.852 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:13:23.852 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:13:23.852 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:13:23.852 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:13:23.975 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.81613.asok 2024-04-25T10:13:23.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:23.986 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.81613.asok 2024-04-25T10:13:23.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:23.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.81613.asok status 2024-04-25T10:13:24.146 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:13:24.146 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:13:24.146 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:13:24.146 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:13:24.146 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:13:24.146 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "81613", 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 9687, 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 9687 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:13:24.147 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2526059431 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2526059431 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.9687 192.168.0.1:0/2526059431", 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/2526059431", 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 79, 2024-04-25T10:13:24.148 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 132, 2024-04-25T10:13:24.149 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:13:24.149 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:13:24.149 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:13:24.149 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:13:24.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:24.160 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:13:24.160 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:24.166 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:13:24.167 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:24.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:24.167 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:24.222 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:24.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.384+0000 7f0eea24e640 1 Processor -- start 2024-04-25T10:13:24.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.384+0000 7f0eea24e640 1 -- start start 2024-04-25T10:13:24.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0eea24e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ee415d9b0 0x7f0ee40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0eea24e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0ee415dd90 0x7f0ee40ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0eea24e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee40ca9b0 con 0x7f0ee40b19f0 2024-04-25T10:13:24.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0eea24e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee40cbe30 con 0x7f0ee415dd90 2024-04-25T10:13:24.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0eea24e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee40cac90 con 0x7f0ee415d9b0 2024-04-25T10:13:24.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0ee37fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ee415d9b0 0x7f0ee40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0ee2ffd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0ee415dd90 0x7f0ee40ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0ee3fff640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f0ee40b19f0 0x7f0ee41616f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:49988/0 (socket says 172.21.15.133:49988) 2024-04-25T10:13:24.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0ee3fff640 1 -- 172.21.15.133:0/393909239 learned_addr learned my addr 172.21.15.133:0/393909239 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:24.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0ee27fc640 1 -- 172.21.15.133:0/393909239 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685097666 0 0) 0x7f0ee40ca9b0 con 0x7f0ee40b19f0 2024-04-25T10:13:24.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0ee27fc640 1 -- 172.21.15.133:0/393909239 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ecc003530 con 0x7f0ee40b19f0 2024-04-25T10:13:24.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0ee37fe640 1 -- 172.21.15.133:0/393909239 >> v2:172.21.15.162:3300/0 conn(0x7f0ee415dd90 msgr2=0x7f0ee40ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0ee37fe640 1 --2- 172.21.15.133:0/393909239 >> v2:172.21.15.162:3300/0 conn(0x7f0ee415dd90 0x7f0ee40ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0ee37fe640 1 -- 172.21.15.133:0/393909239 >> v1:172.21.15.162:6789/0 conn(0x7f0ee40b19f0 legacy=0x7f0ee41616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.388+0000 7f0ee37fe640 1 -- 172.21.15.133:0/393909239 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee40caf70 con 0x7f0ee415d9b0 2024-04-25T10:13:24.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0ee37fe640 1 --2- 172.21.15.133:0/393909239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ee415d9b0 0x7f0ee40cb5e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80097c0 tx=0x7f0ed801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b97032fc049e06c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0ee27fc640 1 -- 172.21.15.133:0/393909239 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80b1070 con 0x7f0ee415d9b0 2024-04-25T10:13:24.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0ee27fc640 1 -- 172.21.15.133:0/393909239 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed80ac630 con 0x7f0ee415d9b0 2024-04-25T10:13:24.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0ee27fc640 1 -- 172.21.15.133:0/393909239 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80a8050 con 0x7f0ee415d9b0 2024-04-25T10:13:24.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0eea24e640 1 -- 172.21.15.133:0/393909239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ee415d9b0 msgr2=0x7f0ee40cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0eea24e640 1 --2- 172.21.15.133:0/393909239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ee415d9b0 0x7f0ee40cb5e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80097c0 tx=0x7f0ed801c3b0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0eea24e640 1 -- 172.21.15.133:0/393909239 shutdown_connections 2024-04-25T10:13:24.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0eea24e640 1 --2- 172.21.15.133:0/393909239 >> v2:172.21.15.162:3300/0 conn(0x7f0ee415dd90 0x7f0ee40ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0eea24e640 1 --2- 172.21.15.133:0/393909239 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ee415d9b0 0x7f0ee40cb5e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0eea24e640 1 -- 172.21.15.133:0/393909239 >> 172.21.15.133:0/393909239 conn(0x7f0ee4067000 msgr2=0x7f0ee406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0eea24e640 1 -- 172.21.15.133:0/393909239 shutdown_connections 2024-04-25T10:13:24.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.392+0000 7f0eea24e640 1 -- 172.21.15.133:0/393909239 wait complete. 2024-04-25T10:13:24.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0eea24e640 1 Processor -- start 2024-04-25T10:13:24.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0eea24e640 1 -- start start 2024-04-25T10:13:24.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0eea24e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee40b19f0 0x7f0ee4151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0eea24e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ee415d9b0 0x7f0ee4149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0eea24e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee415dd90 0x7f0ee4149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0eea24e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee416abf0 con 0x7f0ee40b19f0 2024-04-25T10:13:24.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0eea24e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee416aa70 con 0x7f0ee415dd90 2024-04-25T10:13:24.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0eea24e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee416ad70 con 0x7f0ee415d9b0 2024-04-25T10:13:24.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ee37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee40b19f0 0x7f0ee4151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ee37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee40b19f0 0x7f0ee4151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34642/0 (socket says 172.21.15.133:34642) 2024-04-25T10:13:24.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ee37fe640 1 -- 172.21.15.133:0/1443471381 learned_addr learned my addr 172.21.15.133:0/1443471381 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:24.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ee3fff640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee415dd90 0x7f0ee4149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ee2ffd640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ee415d9b0 0x7f0ee4149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ee37fe640 1 -- 172.21.15.133:0/1443471381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ee415d9b0 msgr2=0x7f0ee4149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ee37fe640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ee415d9b0 0x7f0ee4149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ee37fe640 1 -- 172.21.15.133:0/1443471381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee415dd90 msgr2=0x7f0ee4149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ee37fe640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee415dd90 0x7f0ee4149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ee37fe640 1 -- 172.21.15.133:0/1443471381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee414a640 con 0x7f0ee40b19f0 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ee37fe640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee40b19f0 0x7f0ee4151850 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f0ed801a040 tx=0x7f0ed80191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ecbfff640 1 -- 172.21.15.133:0/1443471381 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80b9020 con 0x7f0ee40b19f0 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ecbfff640 1 -- 172.21.15.133:0/1443471381 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed80b5410 con 0x7f0ee40b19f0 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0eea24e640 1 -- 172.21.15.133:0/1443471381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee414a920 con 0x7f0ee40b19f0 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0ecbfff640 1 -- 172.21.15.133:0/1443471381 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80b1040 con 0x7f0ee40b19f0 2024-04-25T10:13:24.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0eea24e640 1 -- 172.21.15.133:0/1443471381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee414aee0 con 0x7f0ee40b19f0 2024-04-25T10:13:24.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.396+0000 7f0eea24e640 1 -- 172.21.15.133:0/1443471381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee4068ef0 con 0x7f0ee40b19f0 2024-04-25T10:13:24.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.400+0000 7f0ecbfff640 1 -- 172.21.15.133:0/1443471381 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ed80bc020 con 0x7f0ee40b19f0 2024-04-25T10:13:24.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.400+0000 7f0ecbfff640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0ed0066f60 0x7f0ed0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.400+0000 7f0ee2ffd640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0ed0066f60 0x7f0ed0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.400+0000 7f0ecbfff640 1 -- 172.21.15.133:0/1443471381 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ed811ec70 con 0x7f0ee40b19f0 2024-04-25T10:13:24.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.400+0000 7f0ee2ffd640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0ed0066f60 0x7f0ed0069420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0ee41538c0 tx=0x7f0ed400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.404+0000 7f0ecbfff640 1 -- 172.21.15.133:0/1443471381 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ed80c3030 con 0x7f0ee40b19f0 2024-04-25T10:13:24.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0eea24e640 1 -- 172.21.15.133:0/1443471381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ee40c9630 con 0x7f0ee40b19f0 2024-04-25T10:13:24.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0ecbfff640 1 -- 172.21.15.133:0/1443471381 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0ed8016090 con 0x7f0ee40b19f0 2024-04-25T10:13:24.603 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:24.603 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:13:24.603 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:13:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0eea24e640 1 -- 172.21.15.133:0/1443471381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0ed0066f60 msgr2=0x7f0ed0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0eea24e640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0ed0066f60 0x7f0ed0069420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0ee41538c0 tx=0x7f0ed400b040 comp rx=0 tx=0).stop 2024-04-25T10:13:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0eea24e640 1 -- 172.21.15.133:0/1443471381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee40b19f0 msgr2=0x7f0ee4151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0eea24e640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee40b19f0 0x7f0ee4151850 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f0ed801a040 tx=0x7f0ed80191b0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0eea24e640 1 -- 172.21.15.133:0/1443471381 shutdown_connections 2024-04-25T10:13:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0eea24e640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0ee415dd90 0x7f0ee4149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0eea24e640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0ee415d9b0 0x7f0ee4149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0eea24e640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0ed0066f60 0x7f0ed0069420 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0eea24e640 1 --2- 172.21.15.133:0/1443471381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ee40b19f0 0x7f0ee4151850 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.600+0000 7f0eea24e640 1 -- 172.21.15.133:0/1443471381 >> 172.21.15.133:0/1443471381 conn(0x7f0ee4067000 msgr2=0x7f0ee4156880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.604+0000 7f0eea24e640 1 -- 172.21.15.133:0/1443471381 shutdown_connections 2024-04-25T10:13:24.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.604+0000 7f0eea24e640 1 -- 172.21.15.133:0/1443471381 wait complete. 2024-04-25T10:13:24.621 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:24.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.728+0000 7f48f3544640 1 Processor -- start 2024-04-25T10:13:24.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.728+0000 7f48f3544640 1 -- start start 2024-04-25T10:13:24.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f3544640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f48ec15c9a0 0x7f48ec0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f3544640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f48ec0ce330 0x7f48ec0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f3544640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec0c61d0 con 0x7f48ec0b1f70 2024-04-25T10:13:24.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f3544640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec0cecc0 con 0x7f48ec15c9a0 2024-04-25T10:13:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f3544640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec0c64b0 con 0x7f48ec0ce330 2024-04-25T10:13:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f12e0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f48ec15c9a0 0x7f48ec0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f12e0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f48ec15c9a0 0x7f48ec0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34654/0 (socket says 172.21.15.133:34654) 2024-04-25T10:13:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f12e0640 1 -- 172.21.15.133:0/532320924 learned_addr learned my addr 172.21.15.133:0/532320924 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f1ae1640 1 --2- 172.21.15.133:0/532320924 >> v2:172.21.15.162:3300/0 conn(0x7f48ec0ce330 0x7f48ec0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f12e0640 1 -- 172.21.15.133:0/532320924 >> v2:172.21.15.162:3300/0 conn(0x7f48ec0ce330 msgr2=0x7f48ec0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f12e0640 1 --2- 172.21.15.133:0/532320924 >> v2:172.21.15.162:3300/0 conn(0x7f48ec0ce330 0x7f48ec0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f12e0640 1 -- 172.21.15.133:0/532320924 >> v1:172.21.15.162:6789/0 conn(0x7f48ec0b1f70 legacy=0x7f48ec06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.732+0000 7f48f12e0640 1 -- 172.21.15.133:0/532320924 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec06d4f0 con 0x7f48ec15c9a0 2024-04-25T10:13:24.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f12e0640 1 --2- 172.21.15.133:0/532320924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48ec15c9a0 0x7f48ec0c5980 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0cc0f0 tx=0x7f48e401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc286e591462265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48e3fff640 1 -- 172.21.15.133:0/532320924 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40a56c0 con 0x7f48ec15c9a0 2024-04-25T10:13:24.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48e3fff640 1 -- 172.21.15.133:0/532320924 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e40a5860 con 0x7f48ec15c9a0 2024-04-25T10:13:24.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48e3fff640 1 -- 172.21.15.133:0/532320924 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40a5c30 con 0x7f48ec15c9a0 2024-04-25T10:13:24.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 -- 172.21.15.133:0/532320924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48ec15c9a0 msgr2=0x7f48ec0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 --2- 172.21.15.133:0/532320924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48ec15c9a0 0x7f48ec0c5980 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0cc0f0 tx=0x7f48e401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 -- 172.21.15.133:0/532320924 shutdown_connections 2024-04-25T10:13:24.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 --2- 172.21.15.133:0/532320924 >> v2:172.21.15.162:3300/0 conn(0x7f48ec0ce330 0x7f48ec0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 --2- 172.21.15.133:0/532320924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48ec15c9a0 0x7f48ec0c5980 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 -- 172.21.15.133:0/532320924 >> 172.21.15.133:0/532320924 conn(0x7f48ec067000 msgr2=0x7f48ec06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 -- 172.21.15.133:0/532320924 shutdown_connections 2024-04-25T10:13:24.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 -- 172.21.15.133:0/532320924 wait complete. 2024-04-25T10:13:24.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 Processor -- start 2024-04-25T10:13:24.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 -- start start 2024-04-25T10:13:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48ec0b1f70 0x7f48ec0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48ec0ce330 0x7f48ec164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48ec15c9a0 0x7f48ec148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec15dad0 con 0x7f48ec0b1f70 2024-04-25T10:13:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec15d950 con 0x7f48ec0ce330 2024-04-25T10:13:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f3544640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec15dc50 con 0x7f48ec15c9a0 2024-04-25T10:13:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f12e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48ec0b1f70 0x7f48ec0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f12e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48ec0b1f70 0x7f48ec0cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34668/0 (socket says 172.21.15.133:34668) 2024-04-25T10:13:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f12e0640 1 -- 172.21.15.133:0/1051686944 learned_addr learned my addr 172.21.15.133:0/1051686944 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:24.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f1ae1640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48ec15c9a0 0x7f48ec148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f12e0640 1 -- 172.21.15.133:0/1051686944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48ec15c9a0 msgr2=0x7f48ec148b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f12e0640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48ec15c9a0 0x7f48ec148b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f12e0640 1 -- 172.21.15.133:0/1051686944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48ec0ce330 msgr2=0x7f48ec164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f12e0640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48ec0ce330 0x7f48ec164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.736+0000 7f48f12e0640 1 -- 172.21.15.133:0/1051686944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec165250 con 0x7f48ec0b1f70 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48f12e0640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48ec0b1f70 0x7f48ec0cdd90 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f48e40af040 tx=0x7f48e4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48e1ffb640 1 -- 172.21.15.133:0/1051686944 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40b8670 con 0x7f48ec0b1f70 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48e1ffb640 1 -- 172.21.15.133:0/1051686944 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e40be070 con 0x7f48ec0b1f70 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48f3544640 1 -- 172.21.15.133:0/1051686944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec164f70 con 0x7f48ec0b1f70 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48e1ffb640 1 -- 172.21.15.133:0/1051686944 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40b8a90 con 0x7f48ec0b1f70 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48f3544640 1 -- 172.21.15.133:0/1051686944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec0cd590 con 0x7f48ec0b1f70 2024-04-25T10:13:24.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48f3544640 1 -- 172.21.15.133:0/1051686944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec06bc20 con 0x7f48ec0b1f70 2024-04-25T10:13:24.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48e1ffb640 1 -- 172.21.15.133:0/1051686944 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48e40b8c30 con 0x7f48ec0b1f70 2024-04-25T10:13:24.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48e1ffb640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48d0066f60 0x7f48d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48e1ffb640 1 -- 172.21.15.133:0/1051686944 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f48e4120450 con 0x7f48ec0b1f70 2024-04-25T10:13:24.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48f0adf640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48d0066f60 0x7f48d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.740+0000 7f48f0adf640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48d0066f60 0x7f48d0069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f48dc000f70 tx=0x7f48dc0079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.744+0000 7f48e1ffb640 1 -- 172.21.15.133:0/1051686944 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48e40165e0 con 0x7f48ec0b1f70 2024-04-25T10:13:24.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 -- 172.21.15.133:0/1051686944 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48ec068ef0 con 0x7f48ec0b1f70 2024-04-25T10:13:24.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48e1ffb640 1 -- 172.21.15.133:0/1051686944 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f48e40f1360 con 0x7f48ec0b1f70 2024-04-25T10:13:24.942 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:24.943 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:13:24.943 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:13:24.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 -- 172.21.15.133:0/1051686944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48d0066f60 msgr2=0x7f48d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48d0066f60 0x7f48d0069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f48dc000f70 tx=0x7f48dc0079e0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 -- 172.21.15.133:0/1051686944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48ec0b1f70 msgr2=0x7f48ec0cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48ec0b1f70 0x7f48ec0cdd90 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f48e40af040 tx=0x7f48e4019790 comp rx=0 tx=0).stop 2024-04-25T10:13:24.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 -- 172.21.15.133:0/1051686944 shutdown_connections 2024-04-25T10:13:24.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48ec15c9a0 0x7f48ec148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48ec0ce330 0x7f48ec164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f48d0066f60 0x7f48d0069420 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 --2- 172.21.15.133:0/1051686944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48ec0b1f70 0x7f48ec0cdd90 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 -- 172.21.15.133:0/1051686944 >> 172.21.15.133:0/1051686944 conn(0x7f48ec067000 msgr2=0x7f48ec1560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 -- 172.21.15.133:0/1051686944 shutdown_connections 2024-04-25T10:13:24.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:24.940+0000 7f48f3544640 1 -- 172.21.15.133:0/1051686944 wait complete. 2024-04-25T10:13:24.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:24.961 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:24.961 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:24.961 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:13:24.961 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:24.961 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:24.961 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:24.961 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:24.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:24.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:24.999 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194832 s, 53.8 MB/s 2024-04-25T10:13:25.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129278 s, 81.1 MB/s 2024-04-25T10:13:25.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.027 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113416 s, 92.5 MB/s 2024-04-25T10:13:25.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.039 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009938 s, 106 MB/s 2024-04-25T10:13:25.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.049 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795041 s, 132 MB/s 2024-04-25T10:13:25.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.062 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113473 s, 92.4 MB/s 2024-04-25T10:13:25.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.073 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919408 s, 114 MB/s 2024-04-25T10:13:25.087 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.087 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.088 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120348 s, 87.1 MB/s 2024-04-25T10:13:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.101 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117294 s, 89.4 MB/s 2024-04-25T10:13:25.115 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.116 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121618 s, 86.2 MB/s 2024-04-25T10:13:25.130 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.130 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.130 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127498 s, 82.2 MB/s 2024-04-25T10:13:25.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.146 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134307 s, 78.1 MB/s 2024-04-25T10:13:25.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.160 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120014 s, 87.4 MB/s 2024-04-25T10:13:25.177 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.177 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.177 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144563 s, 72.5 MB/s 2024-04-25T10:13:25.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.188 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00935457 s, 112 MB/s 2024-04-25T10:13:25.202 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.202 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.202 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116703 s, 89.8 MB/s 2024-04-25T10:13:25.216 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.216 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.216 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010789 s, 97.2 MB/s 2024-04-25T10:13:25.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.231 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136256 s, 77.0 MB/s 2024-04-25T10:13:25.244 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.245 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121198 s, 86.5 MB/s 2024-04-25T10:13:25.258 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.258 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.258 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116907 s, 89.7 MB/s 2024-04-25T10:13:25.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.273 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113992 s, 92.0 MB/s 2024-04-25T10:13:25.283 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.283 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.283 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911218 s, 115 MB/s 2024-04-25T10:13:25.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.296 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105338 s, 99.5 MB/s 2024-04-25T10:13:25.310 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.310 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.310 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119211 s, 88.0 MB/s 2024-04-25T10:13:25.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.321 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899915 s, 117 MB/s 2024-04-25T10:13:25.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.335 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011831 s, 88.6 MB/s 2024-04-25T10:13:25.349 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.349 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.349 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120451 s, 87.1 MB/s 2024-04-25T10:13:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866008 s, 121 MB/s 2024-04-25T10:13:25.373 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.373 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.373 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112147 s, 93.5 MB/s 2024-04-25T10:13:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114023 s, 92.0 MB/s 2024-04-25T10:13:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123277 s, 85.1 MB/s 2024-04-25T10:13:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.413 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936545 s, 112 MB/s 2024-04-25T10:13:25.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:25.414 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:25.414 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:25.414 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:13:25.414 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:25.414 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:25.414 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:25.414 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:25.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.482 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119893 s, 87.5 MB/s 2024-04-25T10:13:25.497 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.498 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131049 s, 80.0 MB/s 2024-04-25T10:13:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134317 s, 78.1 MB/s 2024-04-25T10:13:25.530 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.530 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.530 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132543 s, 79.1 MB/s 2024-04-25T10:13:25.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.546 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136046 s, 77.1 MB/s 2024-04-25T10:13:25.561 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.561 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.562 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127293 s, 82.4 MB/s 2024-04-25T10:13:25.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.576 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114274 s, 91.8 MB/s 2024-04-25T10:13:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.592 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125134 s, 83.8 MB/s 2024-04-25T10:13:25.609 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.609 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.609 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149086 s, 70.3 MB/s 2024-04-25T10:13:25.623 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.624 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117065 s, 89.6 MB/s 2024-04-25T10:13:25.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.637 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111431 s, 94.1 MB/s 2024-04-25T10:13:25.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.654 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140183 s, 74.8 MB/s 2024-04-25T10:13:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.671 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013506 s, 77.6 MB/s 2024-04-25T10:13:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139833 s, 75.0 MB/s 2024-04-25T10:13:25.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.703 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125802 s, 83.4 MB/s 2024-04-25T10:13:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.720 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142243 s, 73.7 MB/s 2024-04-25T10:13:25.735 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.735 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.735 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114551 s, 91.5 MB/s 2024-04-25T10:13:25.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.751 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132288 s, 79.3 MB/s 2024-04-25T10:13:25.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.769 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146366 s, 71.6 MB/s 2024-04-25T10:13:25.786 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.786 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.786 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140872 s, 74.4 MB/s 2024-04-25T10:13:25.801 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.801 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.801 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127853 s, 82.0 MB/s 2024-04-25T10:13:25.818 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.818 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.818 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133764 s, 78.4 MB/s 2024-04-25T10:13:25.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.835 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142549 s, 73.6 MB/s 2024-04-25T10:13:25.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.853 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014839 s, 70.7 MB/s 2024-04-25T10:13:25.869 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.869 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.869 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134324 s, 78.1 MB/s 2024-04-25T10:13:25.887 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148979 s, 70.4 MB/s 2024-04-25T10:13:25.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.904 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132501 s, 79.1 MB/s 2024-04-25T10:13:25.919 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.919 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.919 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114549 s, 91.5 MB/s 2024-04-25T10:13:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132687 s, 79.0 MB/s 2024-04-25T10:13:25.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.951 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126224 s, 83.1 MB/s 2024-04-25T10:13:25.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.968 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138931 s, 75.5 MB/s 2024-04-25T10:13:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129874 s, 80.7 MB/s 2024-04-25T10:13:25.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:25.985 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:25.985 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:25.985 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:13:25.985 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:25.985 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:25.985 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:25.986 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:26.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.059 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143679 s, 73.0 MB/s 2024-04-25T10:13:26.076 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.076 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.077 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137394 s, 76.3 MB/s 2024-04-25T10:13:26.093 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.093 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.093 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138629 s, 75.6 MB/s 2024-04-25T10:13:26.109 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.109 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.109 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128798 s, 81.4 MB/s 2024-04-25T10:13:26.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.124 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125075 s, 83.8 MB/s 2024-04-25T10:13:26.143 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.143 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.143 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154944 s, 67.7 MB/s 2024-04-25T10:13:26.157 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.157 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.157 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117935 s, 88.9 MB/s 2024-04-25T10:13:26.172 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.172 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.172 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121822 s, 86.1 MB/s 2024-04-25T10:13:26.189 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.189 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.189 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141797 s, 73.9 MB/s 2024-04-25T10:13:26.204 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.204 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.204 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120554 s, 87.0 MB/s 2024-04-25T10:13:26.220 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.220 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.220 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134016 s, 78.2 MB/s 2024-04-25T10:13:26.235 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.235 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.235 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118338 s, 88.6 MB/s 2024-04-25T10:13:26.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.248 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106145 s, 98.8 MB/s 2024-04-25T10:13:26.263 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.263 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.263 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115659 s, 90.7 MB/s 2024-04-25T10:13:26.280 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.281 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.281 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147887 s, 70.9 MB/s 2024-04-25T10:13:26.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.296 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129737 s, 80.8 MB/s 2024-04-25T10:13:26.313 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.313 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.313 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139688 s, 75.1 MB/s 2024-04-25T10:13:26.328 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.328 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.328 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121806 s, 86.1 MB/s 2024-04-25T10:13:26.345 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.345 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.345 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135735 s, 77.3 MB/s 2024-04-25T10:13:26.359 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.360 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118269 s, 88.7 MB/s 2024-04-25T10:13:26.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.376 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131049 s, 80.0 MB/s 2024-04-25T10:13:26.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.392 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127447 s, 82.3 MB/s 2024-04-25T10:13:26.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.410 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124288 s, 84.4 MB/s 2024-04-25T10:13:26.423 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.424 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.424 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130548 s, 80.3 MB/s 2024-04-25T10:13:26.439 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.439 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.439 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125054 s, 83.8 MB/s 2024-04-25T10:13:26.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.455 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012931 s, 81.1 MB/s 2024-04-25T10:13:26.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.470 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121833 s, 86.1 MB/s 2024-04-25T10:13:26.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.484 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110159 s, 95.2 MB/s 2024-04-25T10:13:26.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.498 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115814 s, 90.5 MB/s 2024-04-25T10:13:26.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.516 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132777 s, 79.0 MB/s 2024-04-25T10:13:26.532 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.532 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.532 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135392 s, 77.4 MB/s 2024-04-25T10:13:26.550 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.550 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.550 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147635 s, 71.0 MB/s 2024-04-25T10:13:26.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:26.552 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:26.552 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:26.552 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:13:26.552 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:26.552 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:26.552 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:26.553 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:26.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.619 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.619 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131397 s, 79.8 MB/s 2024-04-25T10:13:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148844 s, 70.4 MB/s 2024-04-25T10:13:26.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.654 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014412 s, 72.8 MB/s 2024-04-25T10:13:26.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.669 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120584 s, 87.0 MB/s 2024-04-25T10:13:26.687 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.688 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.688 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149566 s, 70.1 MB/s 2024-04-25T10:13:26.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.703 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129213 s, 81.2 MB/s 2024-04-25T10:13:26.718 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.719 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125392 s, 83.6 MB/s 2024-04-25T10:13:26.732 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.732 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.732 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984759 s, 106 MB/s 2024-04-25T10:13:26.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.747 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128182 s, 81.8 MB/s 2024-04-25T10:13:26.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.763 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013402 s, 78.2 MB/s 2024-04-25T10:13:26.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.775 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936252 s, 112 MB/s 2024-04-25T10:13:26.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.789 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112229 s, 93.4 MB/s 2024-04-25T10:13:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119444 s, 87.8 MB/s 2024-04-25T10:13:26.819 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.819 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.819 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122332 s, 85.7 MB/s 2024-04-25T10:13:26.834 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.835 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122427 s, 85.6 MB/s 2024-04-25T10:13:26.851 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.851 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.851 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135211 s, 77.6 MB/s 2024-04-25T10:13:26.863 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.863 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104881 s, 100 MB/s 2024-04-25T10:13:26.879 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.879 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.879 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127396 s, 82.3 MB/s 2024-04-25T10:13:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112817 s, 92.9 MB/s 2024-04-25T10:13:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123456 s, 84.9 MB/s 2024-04-25T10:13:26.923 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.923 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.923 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122556 s, 85.6 MB/s 2024-04-25T10:13:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147228 s, 71.2 MB/s 2024-04-25T10:13:26.956 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124956 s, 83.9 MB/s 2024-04-25T10:13:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.970 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945416 s, 111 MB/s 2024-04-25T10:13:26.987 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:26.987 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:26.987 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013951 s, 75.2 MB/s 2024-04-25T10:13:27.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.006 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150304 s, 69.8 MB/s 2024-04-25T10:13:27.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.022 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135205 s, 77.6 MB/s 2024-04-25T10:13:27.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.039 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141074 s, 74.3 MB/s 2024-04-25T10:13:27.057 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.058 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147203 s, 71.2 MB/s 2024-04-25T10:13:27.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.073 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127914 s, 82.0 MB/s 2024-04-25T10:13:27.090 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.090 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.090 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136779 s, 76.7 MB/s 2024-04-25T10:13:27.105 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.106 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.106 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012534 s, 83.7 MB/s 2024-04-25T10:13:27.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.107 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:27.107 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:27.107 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:13:27.107 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:27.107 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:27.107 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:27.107 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:27.182 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.182 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.182 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159371 s, 65.8 MB/s 2024-04-25T10:13:27.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.199 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130721 s, 80.2 MB/s 2024-04-25T10:13:27.216 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.216 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.216 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146981 s, 71.3 MB/s 2024-04-25T10:13:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126453 s, 82.9 MB/s 2024-04-25T10:13:27.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.247 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111632 s, 93.9 MB/s 2024-04-25T10:13:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139176 s, 75.3 MB/s 2024-04-25T10:13:27.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.277 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105443 s, 99.4 MB/s 2024-04-25T10:13:27.295 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.295 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.295 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014389 s, 72.9 MB/s 2024-04-25T10:13:27.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.311 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114564 s, 91.5 MB/s 2024-04-25T10:13:27.327 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.327 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.327 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117654 s, 89.1 MB/s 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.344 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129056 s, 81.2 MB/s 2024-04-25T10:13:27.362 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.362 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.362 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013535 s, 77.5 MB/s 2024-04-25T10:13:27.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.377 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108464 s, 96.7 MB/s 2024-04-25T10:13:27.394 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.394 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.394 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012856 s, 81.6 MB/s 2024-04-25T10:13:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010549 s, 99.4 MB/s 2024-04-25T10:13:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112659 s, 93.1 MB/s 2024-04-25T10:13:27.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.441 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126565 s, 82.8 MB/s 2024-04-25T10:13:27.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.457 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121882 s, 86.0 MB/s 2024-04-25T10:13:27.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.473 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114027 s, 92.0 MB/s 2024-04-25T10:13:27.489 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.490 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.490 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124523 s, 84.2 MB/s 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125387 s, 83.6 MB/s 2024-04-25T10:13:27.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.525 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152021 s, 69.0 MB/s 2024-04-25T10:13:27.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.544 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147549 s, 71.1 MB/s 2024-04-25T10:13:27.558 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.559 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119202 s, 88.0 MB/s 2024-04-25T10:13:27.572 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.572 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.572 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110934 s, 94.5 MB/s 2024-04-25T10:13:27.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.584 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090903 s, 115 MB/s 2024-04-25T10:13:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130674 s, 80.2 MB/s 2024-04-25T10:13:27.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.615 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130245 s, 80.5 MB/s 2024-04-25T10:13:27.633 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.633 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.634 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132485 s, 79.1 MB/s 2024-04-25T10:13:27.645 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.645 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.645 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121516 s, 86.3 MB/s 2024-04-25T10:13:27.660 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.660 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.660 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125088 s, 83.8 MB/s 2024-04-25T10:13:27.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.675 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.675 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012288 s, 85.3 MB/s 2024-04-25T10:13:27.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:27.676 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:27.676 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:27.677 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:13:27.677 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:27.677 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:27.677 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:27.677 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.739 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107888 s, 97.2 MB/s 2024-04-25T10:13:27.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.752 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.752 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107746 s, 97.3 MB/s 2024-04-25T10:13:27.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.764 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108301 s, 96.8 MB/s 2024-04-25T10:13:27.778 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.779 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113308 s, 92.5 MB/s 2024-04-25T10:13:27.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.790 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955889 s, 110 MB/s 2024-04-25T10:13:27.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.804 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121739 s, 86.1 MB/s 2024-04-25T10:13:27.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.816 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882312 s, 119 MB/s 2024-04-25T10:13:27.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.825 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772996 s, 136 MB/s 2024-04-25T10:13:27.839 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.839 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.839 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115843 s, 90.5 MB/s 2024-04-25T10:13:27.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.853 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117228 s, 89.4 MB/s 2024-04-25T10:13:27.863 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.863 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.863 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789103 s, 133 MB/s 2024-04-25T10:13:27.877 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.878 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.878 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012691 s, 82.6 MB/s 2024-04-25T10:13:27.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.890 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104632 s, 100 MB/s 2024-04-25T10:13:27.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.904 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111833 s, 93.8 MB/s 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.917 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.917 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107041 s, 98.0 MB/s 2024-04-25T10:13:27.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.930 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106902 s, 98.1 MB/s 2024-04-25T10:13:27.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.945 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131771 s, 79.6 MB/s 2024-04-25T10:13:27.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.959 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113216 s, 92.6 MB/s 2024-04-25T10:13:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.969 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867615 s, 121 MB/s 2024-04-25T10:13:27.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.984 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122863 s, 85.3 MB/s 2024-04-25T10:13:27.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:27.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:27.998 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120709 s, 86.9 MB/s 2024-04-25T10:13:28.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129167 s, 81.2 MB/s 2024-04-25T10:13:28.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.026 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105757 s, 99.1 MB/s 2024-04-25T10:13:28.041 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.041 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.041 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126821 s, 82.7 MB/s 2024-04-25T10:13:28.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.054 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107202 s, 97.8 MB/s 2024-04-25T10:13:28.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.066 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940444 s, 111 MB/s 2024-04-25T10:13:28.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.078 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108691 s, 96.5 MB/s 2024-04-25T10:13:28.091 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.091 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.091 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105038 s, 99.8 MB/s 2024-04-25T10:13:28.102 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.103 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914907 s, 115 MB/s 2024-04-25T10:13:28.115 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.115 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.116 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109569 s, 95.7 MB/s 2024-04-25T10:13:28.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.129 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115216 s, 91.0 MB/s 2024-04-25T10:13:28.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.141 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924134 s, 113 MB/s 2024-04-25T10:13:28.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.142 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:28.142 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:28.142 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:13:28.143 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:28.143 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:28.143 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:28.143 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:28.206 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.206 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.206 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991089 s, 106 MB/s 2024-04-25T10:13:28.220 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.220 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.220 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117412 s, 89.3 MB/s 2024-04-25T10:13:28.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.232 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910224 s, 115 MB/s 2024-04-25T10:13:28.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.246 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118684 s, 88.4 MB/s 2024-04-25T10:13:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.258 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103666 s, 101 MB/s 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115853 s, 90.5 MB/s 2024-04-25T10:13:28.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.286 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120916 s, 86.7 MB/s 2024-04-25T10:13:28.299 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.299 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.300 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011279 s, 93.0 MB/s 2024-04-25T10:13:28.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.311 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930958 s, 113 MB/s 2024-04-25T10:13:28.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.321 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786328 s, 133 MB/s 2024-04-25T10:13:28.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.335 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119123 s, 88.0 MB/s 2024-04-25T10:13:28.348 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.348 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.348 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104501 s, 100 MB/s 2024-04-25T10:13:28.361 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.361 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.361 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110359 s, 95.0 MB/s 2024-04-25T10:13:28.374 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.374 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.374 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010742 s, 97.6 MB/s 2024-04-25T10:13:28.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.385 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867057 s, 121 MB/s 2024-04-25T10:13:28.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.397 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102004 s, 103 MB/s 2024-04-25T10:13:28.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.410 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107598 s, 97.5 MB/s 2024-04-25T10:13:28.420 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.420 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.420 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729678 s, 144 MB/s 2024-04-25T10:13:28.432 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.432 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.433 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103576 s, 101 MB/s 2024-04-25T10:13:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110771 s, 94.7 MB/s 2024-04-25T10:13:28.459 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.459 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.459 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107135 s, 97.9 MB/s 2024-04-25T10:13:28.471 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.471 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.471 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103889 s, 101 MB/s 2024-04-25T10:13:28.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.482 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881295 s, 119 MB/s 2024-04-25T10:13:28.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.494 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970346 s, 108 MB/s 2024-04-25T10:13:28.505 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.505 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.505 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879343 s, 119 MB/s 2024-04-25T10:13:28.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.517 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994101 s, 105 MB/s 2024-04-25T10:13:28.532 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.532 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.532 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012878 s, 81.4 MB/s 2024-04-25T10:13:28.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.544 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101875 s, 103 MB/s 2024-04-25T10:13:28.555 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.555 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.555 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827063 s, 127 MB/s 2024-04-25T10:13:28.568 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.568 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.569 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113988 s, 92.0 MB/s 2024-04-25T10:13:28.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.584 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126264 s, 83.0 MB/s 2024-04-25T10:13:28.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.599 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124917 s, 83.9 MB/s 2024-04-25T10:13:28.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.601 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:28.601 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:28.601 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:13:28.601 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:28.601 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:28.601 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:28.601 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:28.664 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.665 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781772 s, 134 MB/s 2024-04-25T10:13:28.678 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.678 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.678 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115676 s, 90.6 MB/s 2024-04-25T10:13:28.690 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.691 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.691 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100431 s, 104 MB/s 2024-04-25T10:13:28.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.707 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142329 s, 73.7 MB/s 2024-04-25T10:13:28.721 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.721 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.721 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112486 s, 93.2 MB/s 2024-04-25T10:13:28.732 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.733 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.733 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091883 s, 114 MB/s 2024-04-25T10:13:28.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.745 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979987 s, 107 MB/s 2024-04-25T10:13:28.759 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.759 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.759 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116378 s, 90.1 MB/s 2024-04-25T10:13:28.772 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.773 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.773 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113265 s, 92.6 MB/s 2024-04-25T10:13:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122037 s, 85.9 MB/s 2024-04-25T10:13:28.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.798 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.798 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812641 s, 129 MB/s 2024-04-25T10:13:28.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.807 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074364 s, 141 MB/s 2024-04-25T10:13:28.820 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.820 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.820 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105257 s, 99.6 MB/s 2024-04-25T10:13:28.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.831 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834606 s, 126 MB/s 2024-04-25T10:13:28.845 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.845 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.845 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121515 s, 86.3 MB/s 2024-04-25T10:13:28.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01023 s, 103 MB/s 2024-04-25T10:13:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113362 s, 92.5 MB/s 2024-04-25T10:13:28.884 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.884 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.884 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115174 s, 91.0 MB/s 2024-04-25T10:13:28.896 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.896 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.896 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984691 s, 106 MB/s 2024-04-25T10:13:28.908 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.908 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.908 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983577 s, 107 MB/s 2024-04-25T10:13:28.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.920 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925977 s, 113 MB/s 2024-04-25T10:13:28.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.930 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779433 s, 135 MB/s 2024-04-25T10:13:28.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.944 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117039 s, 89.6 MB/s 2024-04-25T10:13:28.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.957 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110276 s, 95.1 MB/s 2024-04-25T10:13:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.970 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102879 s, 102 MB/s 2024-04-25T10:13:28.982 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.983 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110234 s, 95.1 MB/s 2024-04-25T10:13:28.997 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:28.997 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:28.998 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114095 s, 91.9 MB/s 2024-04-25T10:13:29.011 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.011 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.012 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133 s, 78.8 MB/s 2024-04-25T10:13:29.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.026 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122544 s, 85.6 MB/s 2024-04-25T10:13:29.036 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.036 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.036 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787898 s, 133 MB/s 2024-04-25T10:13:29.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.050 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117888 s, 88.9 MB/s 2024-04-25T10:13:29.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.062 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996701 s, 105 MB/s 2024-04-25T10:13:29.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.063 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:29.063 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:29.063 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:13:29.064 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:29.064 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:29.064 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:29.064 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:29.126 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.126 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.126 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100812 s, 104 MB/s 2024-04-25T10:13:29.140 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.140 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.140 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122094 s, 85.9 MB/s 2024-04-25T10:13:29.151 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.152 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.152 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982811 s, 107 MB/s 2024-04-25T10:13:29.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.167 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.167 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128909 s, 81.3 MB/s 2024-04-25T10:13:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102193 s, 103 MB/s 2024-04-25T10:13:29.193 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121877 s, 86.0 MB/s 2024-04-25T10:13:29.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.205 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.205 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964917 s, 109 MB/s 2024-04-25T10:13:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749256 s, 140 MB/s 2024-04-25T10:13:29.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.225 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749629 s, 140 MB/s 2024-04-25T10:13:29.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.238 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108934 s, 96.3 MB/s 2024-04-25T10:13:29.251 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.251 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.251 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112197 s, 93.5 MB/s 2024-04-25T10:13:29.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.261 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828335 s, 127 MB/s 2024-04-25T10:13:29.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.273 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995354 s, 105 MB/s 2024-04-25T10:13:29.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.287 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112545 s, 93.2 MB/s 2024-04-25T10:13:29.299 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.299 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.299 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102023 s, 103 MB/s 2024-04-25T10:13:29.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.311 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975706 s, 107 MB/s 2024-04-25T10:13:29.323 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.323 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.323 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987003 s, 106 MB/s 2024-04-25T10:13:29.334 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.335 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966538 s, 108 MB/s 2024-04-25T10:13:29.348 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.348 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.348 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114743 s, 91.4 MB/s 2024-04-25T10:13:29.362 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.362 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.362 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012403 s, 84.5 MB/s 2024-04-25T10:13:29.374 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.374 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.374 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986489 s, 106 MB/s 2024-04-25T10:13:29.388 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.388 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.388 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118807 s, 88.3 MB/s 2024-04-25T10:13:29.400 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.400 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.400 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994957 s, 105 MB/s 2024-04-25T10:13:29.414 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.414 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.414 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120112 s, 87.3 MB/s 2024-04-25T10:13:29.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.427 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106641 s, 98.3 MB/s 2024-04-25T10:13:29.439 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.439 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.439 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991193 s, 106 MB/s 2024-04-25T10:13:29.452 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.452 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.452 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112935 s, 92.8 MB/s 2024-04-25T10:13:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116172 s, 90.3 MB/s 2024-04-25T10:13:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757777 s, 138 MB/s 2024-04-25T10:13:29.489 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.489 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.489 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110645 s, 94.8 MB/s 2024-04-25T10:13:29.503 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.503 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.503 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116357 s, 90.1 MB/s 2024-04-25T10:13:29.516 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.516 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.516 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112809 s, 93.0 MB/s 2024-04-25T10:13:29.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.517 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:29.518 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:29.518 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:13:29.518 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:29.518 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:29.518 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:29.518 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864607 s, 121 MB/s 2024-04-25T10:13:29.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.592 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999169 s, 105 MB/s 2024-04-25T10:13:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.602 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804485 s, 130 MB/s 2024-04-25T10:13:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108307 s, 96.8 MB/s 2024-04-25T10:13:29.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.630 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130589 s, 80.3 MB/s 2024-04-25T10:13:29.644 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.644 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.644 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121247 s, 86.5 MB/s 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.657 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.657 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104552 s, 100 MB/s 2024-04-25T10:13:29.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.670 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111212 s, 94.3 MB/s 2024-04-25T10:13:29.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.681 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953353 s, 110 MB/s 2024-04-25T10:13:29.694 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010976 s, 95.5 MB/s 2024-04-25T10:13:29.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.708 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.708 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109375 s, 95.9 MB/s 2024-04-25T10:13:29.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.720 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010425 s, 101 MB/s 2024-04-25T10:13:29.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.735 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120292 s, 87.2 MB/s 2024-04-25T10:13:29.744 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.744 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.744 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798134 s, 131 MB/s 2024-04-25T10:13:29.755 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.756 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.756 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879041 s, 119 MB/s 2024-04-25T10:13:29.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.768 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100771 s, 104 MB/s 2024-04-25T10:13:29.782 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.782 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.782 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012696 s, 82.6 MB/s 2024-04-25T10:13:29.794 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.794 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.794 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968253 s, 108 MB/s 2024-04-25T10:13:29.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.808 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.808 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111563 s, 94.0 MB/s 2024-04-25T10:13:29.820 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.821 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.821 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109104 s, 96.1 MB/s 2024-04-25T10:13:29.834 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.834 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.834 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117971 s, 88.9 MB/s 2024-04-25T10:13:29.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.849 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126329 s, 83.0 MB/s 2024-04-25T10:13:29.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.861 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00998457 s, 105 MB/s 2024-04-25T10:13:29.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.875 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010899 s, 96.2 MB/s 2024-04-25T10:13:29.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.888 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112303 s, 93.4 MB/s 2024-04-25T10:13:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110236 s, 95.1 MB/s 2024-04-25T10:13:29.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.914 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.914 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010365 s, 101 MB/s 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132146 s, 79.3 MB/s 2024-04-25T10:13:29.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.944 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123713 s, 84.8 MB/s 2024-04-25T10:13:29.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.957 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109204 s, 96.0 MB/s 2024-04-25T10:13:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105028 s, 99.8 MB/s 2024-04-25T10:13:29.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:29.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:29.983 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111283 s, 94.2 MB/s 2024-04-25T10:13:29.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.985 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:29.985 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:29.985 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:13:29.985 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:29.985 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:29.985 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:29.985 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:30.056 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.056 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.056 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126922 s, 82.6 MB/s 2024-04-25T10:13:30.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.071 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011906 s, 88.1 MB/s 2024-04-25T10:13:30.094 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.094 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.094 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202201 s, 51.9 MB/s 2024-04-25T10:13:30.110 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.110 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.110 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131638 s, 79.7 MB/s 2024-04-25T10:13:30.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.129 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159504 s, 65.7 MB/s 2024-04-25T10:13:30.142 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.143 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.143 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105817 s, 99.1 MB/s 2024-04-25T10:13:30.158 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.158 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.159 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128765 s, 81.4 MB/s 2024-04-25T10:13:30.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.174 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125478 s, 83.6 MB/s 2024-04-25T10:13:30.189 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.189 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.189 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011779 s, 89.0 MB/s 2024-04-25T10:13:30.207 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.207 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.207 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014571 s, 72.0 MB/s 2024-04-25T10:13:30.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.224 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139829 s, 75.0 MB/s 2024-04-25T10:13:30.240 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.241 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013359 s, 78.5 MB/s 2024-04-25T10:13:30.256 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.257 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129201 s, 81.2 MB/s 2024-04-25T10:13:30.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.273 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129496 s, 81.0 MB/s 2024-04-25T10:13:30.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.288 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012166 s, 86.2 MB/s 2024-04-25T10:13:30.302 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.302 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.302 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107137 s, 97.9 MB/s 2024-04-25T10:13:30.319 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.319 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.319 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141801 s, 73.9 MB/s 2024-04-25T10:13:30.334 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.334 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.334 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120344 s, 87.1 MB/s 2024-04-25T10:13:30.348 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.349 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.349 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118463 s, 88.5 MB/s 2024-04-25T10:13:30.365 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.366 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.366 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134772 s, 77.8 MB/s 2024-04-25T10:13:30.380 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.380 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.380 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123988 s, 84.6 MB/s 2024-04-25T10:13:30.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.397 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136095 s, 77.0 MB/s 2024-04-25T10:13:30.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.413 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131713 s, 79.6 MB/s 2024-04-25T10:13:30.430 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.430 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.430 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135271 s, 77.5 MB/s 2024-04-25T10:13:30.445 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.446 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.446 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126936 s, 82.6 MB/s 2024-04-25T10:13:30.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.465 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152425 s, 68.8 MB/s 2024-04-25T10:13:30.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.480 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126018 s, 83.2 MB/s 2024-04-25T10:13:30.495 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.496 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.496 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127742 s, 82.1 MB/s 2024-04-25T10:13:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.513 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143677 s, 73.0 MB/s 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126046 s, 83.2 MB/s 2024-04-25T10:13:30.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.546 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136707 s, 76.7 MB/s 2024-04-25T10:13:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111289 s, 94.2 MB/s 2024-04-25T10:13:30.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:30.561 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:30.561 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:30.561 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:13:30.561 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:30.562 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:30.562 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:30.562 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.631 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124974 s, 83.9 MB/s 2024-04-25T10:13:30.648 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.649 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136775 s, 76.7 MB/s 2024-04-25T10:13:30.668 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.668 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.668 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109274 s, 96.0 MB/s 2024-04-25T10:13:30.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.680 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985954 s, 106 MB/s 2024-04-25T10:13:30.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.695 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116603 s, 89.9 MB/s 2024-04-25T10:13:30.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116883 s, 89.7 MB/s 2024-04-25T10:13:30.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.724 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.724 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111176 s, 94.3 MB/s 2024-04-25T10:13:30.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.738 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114118 s, 91.9 MB/s 2024-04-25T10:13:30.750 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.750 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.750 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949871 s, 110 MB/s 2024-04-25T10:13:30.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.763 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103734 s, 101 MB/s 2024-04-25T10:13:30.774 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.774 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.774 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841373 s, 125 MB/s 2024-04-25T10:13:30.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.787 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104738 s, 100 MB/s 2024-04-25T10:13:30.801 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.801 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.801 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113017 s, 92.8 MB/s 2024-04-25T10:13:30.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.816 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115935 s, 90.4 MB/s 2024-04-25T10:13:30.828 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.828 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.828 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994149 s, 105 MB/s 2024-04-25T10:13:30.839 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.839 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843748 s, 124 MB/s 2024-04-25T10:13:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123529 s, 84.9 MB/s 2024-04-25T10:13:30.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.871 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122968 s, 85.3 MB/s 2024-04-25T10:13:30.883 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.883 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.883 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103634 s, 101 MB/s 2024-04-25T10:13:30.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.898 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120601 s, 86.9 MB/s 2024-04-25T10:13:30.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.912 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114877 s, 91.3 MB/s 2024-04-25T10:13:30.925 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.926 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.926 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111359 s, 94.2 MB/s 2024-04-25T10:13:30.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.940 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106648 s, 98.3 MB/s 2024-04-25T10:13:30.954 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.954 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.955 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117707 s, 89.1 MB/s 2024-04-25T10:13:30.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.969 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117774 s, 89.0 MB/s 2024-04-25T10:13:30.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.985 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123869 s, 84.7 MB/s 2024-04-25T10:13:30.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:30.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:30.998 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105814 s, 99.1 MB/s 2024-04-25T10:13:31.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124742 s, 84.1 MB/s 2024-04-25T10:13:31.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.027 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106862 s, 98.1 MB/s 2024-04-25T10:13:31.040 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.041 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.041 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109567 s, 95.7 MB/s 2024-04-25T10:13:31.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.054 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113035 s, 92.8 MB/s 2024-04-25T10:13:31.067 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.067 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.067 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010124 s, 104 MB/s 2024-04-25T10:13:31.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:31.068 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:31.069 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:31.069 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:13:31.069 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:31.069 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:31.069 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:31.069 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:31.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.134 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122524 s, 85.6 MB/s 2024-04-25T10:13:31.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143166 s, 73.2 MB/s 2024-04-25T10:13:31.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.165 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117374 s, 89.3 MB/s 2024-04-25T10:13:31.177 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101021 s, 104 MB/s 2024-04-25T10:13:31.191 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.192 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.192 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112452 s, 93.2 MB/s 2024-04-25T10:13:31.204 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.204 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.204 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100102 s, 105 MB/s 2024-04-25T10:13:31.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.219 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.219 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012075 s, 86.8 MB/s 2024-04-25T10:13:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111168 s, 94.3 MB/s 2024-04-25T10:13:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107133 s, 97.9 MB/s 2024-04-25T10:13:31.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.255 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792523 s, 132 MB/s 2024-04-25T10:13:31.269 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.269 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.269 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113804 s, 92.1 MB/s 2024-04-25T10:13:31.283 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.284 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122672 s, 85.5 MB/s 2024-04-25T10:13:31.294 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.294 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.294 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848441 s, 124 MB/s 2024-04-25T10:13:31.308 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.308 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.308 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114274 s, 91.8 MB/s 2024-04-25T10:13:31.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.321 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102285 s, 103 MB/s 2024-04-25T10:13:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104934 s, 99.9 MB/s 2024-04-25T10:13:31.346 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.346 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.346 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105906 s, 99.0 MB/s 2024-04-25T10:13:31.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.361 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119255 s, 87.9 MB/s 2024-04-25T10:13:31.373 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.373 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.374 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010773 s, 97.3 MB/s 2024-04-25T10:13:31.387 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.387 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.387 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109355 s, 95.9 MB/s 2024-04-25T10:13:31.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.401 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117004 s, 89.6 MB/s 2024-04-25T10:13:31.414 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.415 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.415 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113063 s, 92.7 MB/s 2024-04-25T10:13:31.430 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.431 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.431 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127999 s, 81.9 MB/s 2024-04-25T10:13:31.445 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.445 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.445 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110106 s, 95.2 MB/s 2024-04-25T10:13:31.458 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.458 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.458 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117668 s, 89.1 MB/s 2024-04-25T10:13:31.471 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.471 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.471 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106059 s, 98.9 MB/s 2024-04-25T10:13:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118356 s, 88.6 MB/s 2024-04-25T10:13:31.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.501 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012551 s, 83.5 MB/s 2024-04-25T10:13:31.516 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127304 s, 82.4 MB/s 2024-04-25T10:13:31.528 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.528 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.528 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00913329 s, 115 MB/s 2024-04-25T10:13:31.542 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.542 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.542 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113003 s, 92.8 MB/s 2024-04-25T10:13:31.557 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.557 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.558 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125928 s, 83.3 MB/s 2024-04-25T10:13:31.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:31.559 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:31.559 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:31.559 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:13:31.559 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:31.559 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:31.559 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:31.559 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:31.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.625 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120067 s, 87.3 MB/s 2024-04-25T10:13:31.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.643 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149296 s, 70.2 MB/s 2024-04-25T10:13:31.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.670 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130607 s, 80.3 MB/s 2024-04-25T10:13:31.684 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.684 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.685 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119564 s, 87.7 MB/s 2024-04-25T10:13:31.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.698 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109143 s, 96.1 MB/s 2024-04-25T10:13:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100716 s, 104 MB/s 2024-04-25T10:13:31.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.726 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128447 s, 81.6 MB/s 2024-04-25T10:13:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.737 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883769 s, 119 MB/s 2024-04-25T10:13:31.752 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.752 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.752 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012057 s, 87.0 MB/s 2024-04-25T10:13:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112883 s, 92.9 MB/s 2024-04-25T10:13:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.778 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.778 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985445 s, 106 MB/s 2024-04-25T10:13:31.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.791 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109765 s, 95.5 MB/s 2024-04-25T10:13:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.805 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111456 s, 94.1 MB/s 2024-04-25T10:13:31.818 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.818 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.819 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111581 s, 94.0 MB/s 2024-04-25T10:13:31.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967852 s, 108 MB/s 2024-04-25T10:13:31.848 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.848 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.848 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146795 s, 71.4 MB/s 2024-04-25T10:13:31.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.861 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010825 s, 96.9 MB/s 2024-04-25T10:13:31.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.875 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108495 s, 96.6 MB/s 2024-04-25T10:13:31.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.890 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012398 s, 84.6 MB/s 2024-04-25T10:13:31.903 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.903 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.904 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114367 s, 91.7 MB/s 2024-04-25T10:13:31.919 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.919 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.919 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123231 s, 85.1 MB/s 2024-04-25T10:13:31.932 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.933 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108711 s, 96.5 MB/s 2024-04-25T10:13:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126019 s, 83.2 MB/s 2024-04-25T10:13:31.963 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.964 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131306 s, 79.9 MB/s 2024-04-25T10:13:31.978 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.978 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.978 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116711 s, 89.8 MB/s 2024-04-25T10:13:31.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:31.993 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:31.993 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118376 s, 88.6 MB/s 2024-04-25T10:13:32.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.007 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.007 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011693 s, 89.7 MB/s 2024-04-25T10:13:32.021 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.022 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.022 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119366 s, 87.8 MB/s 2024-04-25T10:13:32.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.037 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123899 s, 84.6 MB/s 2024-04-25T10:13:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.049 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990427 s, 106 MB/s 2024-04-25T10:13:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137084 s, 76.5 MB/s 2024-04-25T10:13:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.082 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124472 s, 84.2 MB/s 2024-04-25T10:13:32.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:32.083 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:32.083 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:32.083 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:13:32.083 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:32.083 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:32.083 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:32.083 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:32.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.151 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.151 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125869 s, 83.3 MB/s 2024-04-25T10:13:32.167 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.167 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.167 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139606 s, 75.1 MB/s 2024-04-25T10:13:32.182 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.182 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.182 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117903 s, 88.9 MB/s 2024-04-25T10:13:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01233 s, 85.0 MB/s 2024-04-25T10:13:32.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.211 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114221 s, 91.8 MB/s 2024-04-25T10:13:32.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.227 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132768 s, 79.0 MB/s 2024-04-25T10:13:32.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.241 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.241 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114976 s, 91.2 MB/s 2024-04-25T10:13:32.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.255 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.255 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106683 s, 98.3 MB/s 2024-04-25T10:13:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121881 s, 86.0 MB/s 2024-04-25T10:13:32.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.284 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112787 s, 93.0 MB/s 2024-04-25T10:13:32.298 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.298 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.298 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118116 s, 88.8 MB/s 2024-04-25T10:13:32.313 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.313 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.313 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124161 s, 84.5 MB/s 2024-04-25T10:13:32.328 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.329 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.329 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123957 s, 84.6 MB/s 2024-04-25T10:13:32.343 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.343 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.343 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117587 s, 89.2 MB/s 2024-04-25T10:13:32.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.357 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115447 s, 90.8 MB/s 2024-04-25T10:13:32.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.372 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.372 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113637 s, 92.3 MB/s 2024-04-25T10:13:32.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.385 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109353 s, 95.9 MB/s 2024-04-25T10:13:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.396 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084076 s, 125 MB/s 2024-04-25T10:13:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130006 s, 80.7 MB/s 2024-04-25T10:13:32.426 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.426 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.426 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113139 s, 92.7 MB/s 2024-04-25T10:13:32.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.442 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125134 s, 83.8 MB/s 2024-04-25T10:13:32.456 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.457 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124828 s, 84.0 MB/s 2024-04-25T10:13:32.471 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.471 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.471 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116067 s, 90.3 MB/s 2024-04-25T10:13:32.487 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.487 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.487 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128192 s, 81.8 MB/s 2024-04-25T10:13:32.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.501 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122379 s, 85.7 MB/s 2024-04-25T10:13:32.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.518 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.518 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132198 s, 79.3 MB/s 2024-04-25T10:13:32.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.532 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109697 s, 95.6 MB/s 2024-04-25T10:13:32.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.547 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126743 s, 82.7 MB/s 2024-04-25T10:13:32.558 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.559 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893561 s, 117 MB/s 2024-04-25T10:13:32.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.575 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135477 s, 77.4 MB/s 2024-04-25T10:13:32.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.591 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128312 s, 81.7 MB/s 2024-04-25T10:13:32.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.606 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125322 s, 83.7 MB/s 2024-04-25T10:13:32.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:32.607 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:32.607 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:32.607 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:13:32.607 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:32.607 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:32.607 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:32.608 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:32.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.690 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138146 s, 75.9 MB/s 2024-04-25T10:13:32.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.704 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116161 s, 90.3 MB/s 2024-04-25T10:13:32.716 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.716 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.717 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941873 s, 111 MB/s 2024-04-25T10:13:32.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.730 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109775 s, 95.5 MB/s 2024-04-25T10:13:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129614 s, 80.9 MB/s 2024-04-25T10:13:32.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.761 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121939 s, 86.0 MB/s 2024-04-25T10:13:32.773 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.773 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.773 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972913 s, 108 MB/s 2024-04-25T10:13:32.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.785 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840703 s, 125 MB/s 2024-04-25T10:13:32.798 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.799 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111195 s, 94.3 MB/s 2024-04-25T10:13:32.812 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.812 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.812 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110316 s, 95.1 MB/s 2024-04-25T10:13:32.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.825 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103006 s, 102 MB/s 2024-04-25T10:13:32.837 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.837 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.837 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983456 s, 107 MB/s 2024-04-25T10:13:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.849 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085804 s, 122 MB/s 2024-04-25T10:13:32.862 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.862 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.863 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113795 s, 92.1 MB/s 2024-04-25T10:13:32.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.875 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00963509 s, 109 MB/s 2024-04-25T10:13:32.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.888 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106425 s, 98.5 MB/s 2024-04-25T10:13:32.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.900 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970627 s, 108 MB/s 2024-04-25T10:13:32.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.912 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884751 s, 119 MB/s 2024-04-25T10:13:32.926 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.927 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117965 s, 88.9 MB/s 2024-04-25T10:13:32.941 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.941 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.941 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115609 s, 90.7 MB/s 2024-04-25T10:13:32.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.953 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878158 s, 119 MB/s 2024-04-25T10:13:32.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.968 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124182 s, 84.4 MB/s 2024-04-25T10:13:32.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.983 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120166 s, 87.3 MB/s 2024-04-25T10:13:32.997 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:32.997 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:32.997 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011609 s, 90.3 MB/s 2024-04-25T10:13:33.011 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.011 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.011 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109522 s, 95.7 MB/s 2024-04-25T10:13:33.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.025 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112187 s, 93.5 MB/s 2024-04-25T10:13:33.041 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.041 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.041 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132459 s, 79.2 MB/s 2024-04-25T10:13:33.055 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.055 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.055 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117925 s, 88.9 MB/s 2024-04-25T10:13:33.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124935 s, 83.9 MB/s 2024-04-25T10:13:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104062 s, 101 MB/s 2024-04-25T10:13:33.098 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.099 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108802 s, 96.4 MB/s 2024-04-25T10:13:33.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.114 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125906 s, 83.3 MB/s 2024-04-25T10:13:33.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:33.115 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:33.115 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:33.115 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:13:33.115 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:33.115 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:33.115 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:33.116 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:33.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.180 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109955 s, 95.4 MB/s 2024-04-25T10:13:33.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.197 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.197 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131405 s, 79.8 MB/s 2024-04-25T10:13:33.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.210 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107922 s, 97.2 MB/s 2024-04-25T10:13:33.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.222 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911844 s, 115 MB/s 2024-04-25T10:13:33.237 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.237 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.237 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125258 s, 83.7 MB/s 2024-04-25T10:13:33.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.248 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849367 s, 123 MB/s 2024-04-25T10:13:33.258 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.259 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082636 s, 127 MB/s 2024-04-25T10:13:33.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.273 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120283 s, 87.2 MB/s 2024-04-25T10:13:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.288 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126502 s, 82.9 MB/s 2024-04-25T10:13:33.302 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.303 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115407 s, 90.9 MB/s 2024-04-25T10:13:33.315 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.315 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.315 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979193 s, 107 MB/s 2024-04-25T10:13:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885923 s, 118 MB/s 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095208 s, 110 MB/s 2024-04-25T10:13:33.352 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.352 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.353 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109379 s, 95.9 MB/s 2024-04-25T10:13:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123813 s, 84.7 MB/s 2024-04-25T10:13:33.380 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.380 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.380 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112795 s, 93.0 MB/s 2024-04-25T10:13:33.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.392 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911438 s, 115 MB/s 2024-04-25T10:13:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125328 s, 83.7 MB/s 2024-04-25T10:13:33.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.422 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119924 s, 87.4 MB/s 2024-04-25T10:13:33.437 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.437 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.437 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128511 s, 81.6 MB/s 2024-04-25T10:13:33.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.454 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137376 s, 76.3 MB/s 2024-04-25T10:13:33.469 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.469 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.469 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123155 s, 85.1 MB/s 2024-04-25T10:13:33.485 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.485 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.485 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127249 s, 82.4 MB/s 2024-04-25T10:13:33.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.501 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129737 s, 80.8 MB/s 2024-04-25T10:13:33.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.517 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.517 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129564 s, 80.9 MB/s 2024-04-25T10:13:33.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.533 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128119 s, 81.8 MB/s 2024-04-25T10:13:33.549 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.549 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.549 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135098 s, 77.6 MB/s 2024-04-25T10:13:33.566 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.566 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.566 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139305 s, 75.3 MB/s 2024-04-25T10:13:33.581 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.581 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.581 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125388 s, 83.6 MB/s 2024-04-25T10:13:33.598 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.598 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.598 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137788 s, 76.1 MB/s 2024-04-25T10:13:33.613 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.613 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.613 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012453 s, 84.2 MB/s 2024-04-25T10:13:33.628 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.628 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.628 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119767 s, 87.6 MB/s 2024-04-25T10:13:33.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:33.629 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:33.629 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:33.629 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:13:33.629 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:33.630 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:33.630 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:33.630 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:33.700 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.700 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.700 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124314 s, 84.3 MB/s 2024-04-25T10:13:33.716 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.716 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.716 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126942 s, 82.6 MB/s 2024-04-25T10:13:33.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.730 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111076 s, 94.4 MB/s 2024-04-25T10:13:33.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.745 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123109 s, 85.2 MB/s 2024-04-25T10:13:33.759 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.759 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.759 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111787 s, 93.8 MB/s 2024-04-25T10:13:33.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.776 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135216 s, 77.5 MB/s 2024-04-25T10:13:33.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.790 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115524 s, 90.8 MB/s 2024-04-25T10:13:33.803 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.803 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.803 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104553 s, 100 MB/s 2024-04-25T10:13:33.819 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.819 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.819 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131811 s, 79.6 MB/s 2024-04-25T10:13:33.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.835 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126732 s, 82.7 MB/s 2024-04-25T10:13:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.851 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.851 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126847 s, 82.7 MB/s 2024-04-25T10:13:33.863 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.863 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.863 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993868 s, 106 MB/s 2024-04-25T10:13:33.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.876 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.876 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931015 s, 113 MB/s 2024-04-25T10:13:33.891 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.891 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.891 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123404 s, 85.0 MB/s 2024-04-25T10:13:33.905 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.905 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.905 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113103 s, 92.7 MB/s 2024-04-25T10:13:33.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.920 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116697 s, 89.9 MB/s 2024-04-25T10:13:33.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.936 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130843 s, 80.1 MB/s 2024-04-25T10:13:33.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.951 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109966 s, 95.4 MB/s 2024-04-25T10:13:33.966 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.966 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.966 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118517 s, 88.5 MB/s 2024-04-25T10:13:33.982 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122751 s, 85.4 MB/s 2024-04-25T10:13:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111776 s, 93.8 MB/s 2024-04-25T10:13:34.014 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.014 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.014 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122 s, 85.9 MB/s 2024-04-25T10:13:34.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101885 s, 103 MB/s 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.040 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010949 s, 95.8 MB/s 2024-04-25T10:13:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.056 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134392 s, 78.0 MB/s 2024-04-25T10:13:34.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.071 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118372 s, 88.6 MB/s 2024-04-25T10:13:34.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.087 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128783 s, 81.4 MB/s 2024-04-25T10:13:34.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.101 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120392 s, 87.1 MB/s 2024-04-25T10:13:34.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.212 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.212 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128961 s, 81.3 MB/s 2024-04-25T10:13:34.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.214 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120469 s, 87.0 MB/s 2024-04-25T10:13:34.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.214 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106957 s, 98.0 MB/s 2024-04-25T10:13:34.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.215 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115652 s, 90.7 MB/s 2024-04-25T10:13:34.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:34.215 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:34.215 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:34.215 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:13:34.215 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:34.215 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:34.215 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:34.215 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146759 s, 71.4 MB/s 2024-04-25T10:13:34.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.306 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144702 s, 72.5 MB/s 2024-04-25T10:13:34.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.324 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140791 s, 74.5 MB/s 2024-04-25T10:13:34.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.342 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.342 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132547 s, 79.1 MB/s 2024-04-25T10:13:34.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.360 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138825 s, 75.5 MB/s 2024-04-25T10:13:34.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.376 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136221 s, 77.0 MB/s 2024-04-25T10:13:34.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.391 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113585 s, 92.3 MB/s 2024-04-25T10:13:34.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.406 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123603 s, 84.8 MB/s 2024-04-25T10:13:34.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.421 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124187 s, 84.4 MB/s 2024-04-25T10:13:34.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.435 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104989 s, 99.9 MB/s 2024-04-25T10:13:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.450 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120853 s, 86.8 MB/s 2024-04-25T10:13:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116429 s, 90.1 MB/s 2024-04-25T10:13:34.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.480 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128718 s, 81.5 MB/s 2024-04-25T10:13:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.495 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119599 s, 87.7 MB/s 2024-04-25T10:13:34.509 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.509 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.509 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115112 s, 91.1 MB/s 2024-04-25T10:13:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140089 s, 74.9 MB/s 2024-04-25T10:13:34.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.541 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120463 s, 87.0 MB/s 2024-04-25T10:13:34.555 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.555 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.555 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113522 s, 92.4 MB/s 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.571 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123245 s, 85.1 MB/s 2024-04-25T10:13:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125663 s, 83.4 MB/s 2024-04-25T10:13:34.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140157 s, 74.8 MB/s 2024-04-25T10:13:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.620 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128301 s, 81.7 MB/s 2024-04-25T10:13:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.637 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135238 s, 77.5 MB/s 2024-04-25T10:13:34.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.653 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128086 s, 81.9 MB/s 2024-04-25T10:13:34.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.670 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134243 s, 78.1 MB/s 2024-04-25T10:13:34.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.708 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.708 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127687 s, 82.1 MB/s 2024-04-25T10:13:34.725 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.725 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.725 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150534 s, 69.7 MB/s 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123401 s, 85.0 MB/s 2024-04-25T10:13:34.756 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.757 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.757 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130484 s, 80.4 MB/s 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128636 s, 81.5 MB/s 2024-04-25T10:13:34.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.787 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118268 s, 88.7 MB/s 2024-04-25T10:13:34.803 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.804 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134277 s, 78.1 MB/s 2024-04-25T10:13:34.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:34.805 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:34.805 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:34.805 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:13:34.805 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:34.805 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:34.805 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:34.805 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:34.882 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.882 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.882 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149371 s, 70.2 MB/s 2024-04-25T10:13:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148901 s, 70.4 MB/s 2024-04-25T10:13:34.918 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.918 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.918 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140092 s, 74.8 MB/s 2024-04-25T10:13:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124852 s, 84.0 MB/s 2024-04-25T10:13:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135183 s, 77.6 MB/s 2024-04-25T10:13:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.967 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.967 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134189 s, 78.1 MB/s 2024-04-25T10:13:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:34.984 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140819 s, 74.5 MB/s 2024-04-25T10:13:35.001 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.001 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.001 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136303 s, 76.9 MB/s 2024-04-25T10:13:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.017 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133012 s, 78.8 MB/s 2024-04-25T10:13:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134336 s, 78.1 MB/s 2024-04-25T10:13:35.051 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.051 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.051 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129278 s, 81.1 MB/s 2024-04-25T10:13:35.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.068 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138971 s, 75.5 MB/s 2024-04-25T10:13:35.085 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.085 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.085 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138308 s, 75.8 MB/s 2024-04-25T10:13:35.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.101 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.102 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135245 s, 77.5 MB/s 2024-04-25T10:13:35.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.116 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110893 s, 94.6 MB/s 2024-04-25T10:13:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.132 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010986 s, 95.4 MB/s 2024-04-25T10:13:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.146 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134432 s, 78.0 MB/s 2024-04-25T10:13:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127332 s, 82.4 MB/s 2024-04-25T10:13:35.177 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.178 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.178 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012103 s, 86.6 MB/s 2024-04-25T10:13:35.195 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.195 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.195 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141129 s, 74.3 MB/s 2024-04-25T10:13:35.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.211 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127561 s, 82.2 MB/s 2024-04-25T10:13:35.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.228 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142274 s, 73.7 MB/s 2024-04-25T10:13:35.242 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.242 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.242 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111852 s, 93.7 MB/s 2024-04-25T10:13:35.258 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.258 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.258 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125395 s, 83.6 MB/s 2024-04-25T10:13:35.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.275 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144498 s, 72.6 MB/s 2024-04-25T10:13:35.289 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.289 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.289 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111126 s, 94.4 MB/s 2024-04-25T10:13:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120305 s, 87.2 MB/s 2024-04-25T10:13:35.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.322 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145721 s, 72.0 MB/s 2024-04-25T10:13:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.338 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129928 s, 80.7 MB/s 2024-04-25T10:13:35.352 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.352 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.352 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110394 s, 95.0 MB/s 2024-04-25T10:13:35.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.368 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122784 s, 85.4 MB/s 2024-04-25T10:13:35.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.384 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133788 s, 78.4 MB/s 2024-04-25T10:13:35.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:35.385 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:35.385 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:35.385 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:13:35.385 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:35.385 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:35.385 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:35.386 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:35.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.455 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124423 s, 84.3 MB/s 2024-04-25T10:13:35.471 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.471 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.471 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136338 s, 76.9 MB/s 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.486 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121216 s, 86.5 MB/s 2024-04-25T10:13:35.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.502 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119678 s, 87.6 MB/s 2024-04-25T10:13:35.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.516 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110169 s, 95.2 MB/s 2024-04-25T10:13:35.532 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.532 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.532 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131777 s, 79.6 MB/s 2024-04-25T10:13:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126193 s, 83.1 MB/s 2024-04-25T10:13:35.563 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.563 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.563 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129147 s, 81.2 MB/s 2024-04-25T10:13:35.579 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.579 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.579 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130862 s, 80.1 MB/s 2024-04-25T10:13:35.595 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.595 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.595 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134369 s, 78.0 MB/s 2024-04-25T10:13:35.611 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.611 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.611 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126695 s, 82.8 MB/s 2024-04-25T10:13:35.626 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.627 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.627 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125458 s, 83.6 MB/s 2024-04-25T10:13:35.641 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.641 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.641 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111972 s, 93.6 MB/s 2024-04-25T10:13:35.657 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.657 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.657 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135521 s, 77.4 MB/s 2024-04-25T10:13:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151781 s, 69.1 MB/s 2024-04-25T10:13:35.706 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.706 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.706 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135597 s, 77.3 MB/s 2024-04-25T10:13:35.721 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.721 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.721 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125435 s, 83.6 MB/s 2024-04-25T10:13:35.736 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.737 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.737 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122111 s, 85.9 MB/s 2024-04-25T10:13:35.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.751 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117082 s, 89.6 MB/s 2024-04-25T10:13:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128816 s, 81.4 MB/s 2024-04-25T10:13:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115034 s, 91.2 MB/s 2024-04-25T10:13:35.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.799 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123781 s, 84.7 MB/s 2024-04-25T10:13:35.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.815 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.815 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121188 s, 86.5 MB/s 2024-04-25T10:13:35.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.831 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125092 s, 83.8 MB/s 2024-04-25T10:13:35.843 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.843 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.843 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924807 s, 113 MB/s 2024-04-25T10:13:35.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.859 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.859 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126982 s, 82.6 MB/s 2024-04-25T10:13:35.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.874 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012645 s, 82.9 MB/s 2024-04-25T10:13:35.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.890 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125991 s, 83.2 MB/s 2024-04-25T10:13:35.906 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.906 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.906 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130098 s, 80.6 MB/s 2024-04-25T10:13:35.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.922 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013583 s, 77.2 MB/s 2024-04-25T10:13:35.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.936 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114483 s, 91.6 MB/s 2024-04-25T10:13:35.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:35.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:35.952 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132261 s, 79.3 MB/s 2024-04-25T10:13:35.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:35.954 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:35.954 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:35.954 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:13:35.954 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:35.954 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:35.954 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:35.954 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:36.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.025 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.025 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108422 s, 96.7 MB/s 2024-04-25T10:13:36.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.038 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101902 s, 103 MB/s 2024-04-25T10:13:36.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.053 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129005 s, 81.3 MB/s 2024-04-25T10:13:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.070 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130902 s, 80.1 MB/s 2024-04-25T10:13:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994827 s, 105 MB/s 2024-04-25T10:13:36.097 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.097 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.097 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105401 s, 99.5 MB/s 2024-04-25T10:13:36.112 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.112 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.112 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120885 s, 86.7 MB/s 2024-04-25T10:13:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.127 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124072 s, 84.5 MB/s 2024-04-25T10:13:36.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.141 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118568 s, 88.4 MB/s 2024-04-25T10:13:36.156 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.156 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.156 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127032 s, 82.5 MB/s 2024-04-25T10:13:36.171 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.171 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.172 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125878 s, 83.3 MB/s 2024-04-25T10:13:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107131 s, 97.9 MB/s 2024-04-25T10:13:36.198 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.198 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.198 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106736 s, 98.2 MB/s 2024-04-25T10:13:36.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.214 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130951 s, 80.1 MB/s 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119675 s, 87.6 MB/s 2024-04-25T10:13:36.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.246 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145195 s, 72.2 MB/s 2024-04-25T10:13:36.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.260 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.260 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104762 s, 100 MB/s 2024-04-25T10:13:36.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.270 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787675 s, 133 MB/s 2024-04-25T10:13:36.283 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.283 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.283 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106128 s, 98.8 MB/s 2024-04-25T10:13:36.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.296 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105137 s, 99.7 MB/s 2024-04-25T10:13:36.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.306 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795539 s, 132 MB/s 2024-04-25T10:13:36.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.318 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972267 s, 108 MB/s 2024-04-25T10:13:36.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.332 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119355 s, 87.9 MB/s 2024-04-25T10:13:36.344 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105903 s, 99.0 MB/s 2024-04-25T10:13:36.356 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.356 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.356 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971976 s, 108 MB/s 2024-04-25T10:13:36.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.370 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107918 s, 97.2 MB/s 2024-04-25T10:13:36.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.383 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111406 s, 94.1 MB/s 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114773 s, 91.4 MB/s 2024-04-25T10:13:36.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.407 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813276 s, 129 MB/s 2024-04-25T10:13:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125204 s, 83.7 MB/s 2024-04-25T10:13:36.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.436 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113298 s, 92.6 MB/s 2024-04-25T10:13:36.448 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.448 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.448 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00954723 s, 110 MB/s 2024-04-25T10:13:36.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:36.449 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:36.449 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:36.449 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:13:36.449 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:36.449 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:36.450 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:36.450 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:36.516 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.516 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.516 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103646 s, 101 MB/s 2024-04-25T10:13:36.530 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.531 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.531 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115655 s, 90.7 MB/s 2024-04-25T10:13:36.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.544 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105022 s, 99.8 MB/s 2024-04-25T10:13:36.557 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.557 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.558 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112896 s, 92.9 MB/s 2024-04-25T10:13:36.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.571 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.571 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106237 s, 98.7 MB/s 2024-04-25T10:13:36.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.584 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113524 s, 92.4 MB/s 2024-04-25T10:13:36.600 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.600 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.600 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131973 s, 79.5 MB/s 2024-04-25T10:13:36.614 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.614 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.614 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011107 s, 94.4 MB/s 2024-04-25T10:13:36.626 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.626 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.626 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102651 s, 102 MB/s 2024-04-25T10:13:36.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.638 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897455 s, 117 MB/s 2024-04-25T10:13:36.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.651 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112158 s, 93.5 MB/s 2024-04-25T10:13:36.661 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.662 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.662 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787396 s, 133 MB/s 2024-04-25T10:13:36.682 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187123 s, 56.0 MB/s 2024-04-25T10:13:36.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.730 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015729 s, 66.7 MB/s 2024-04-25T10:13:36.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.745 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012443 s, 84.3 MB/s 2024-04-25T10:13:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.761 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118336 s, 88.6 MB/s 2024-04-25T10:13:36.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.778 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.778 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135881 s, 77.2 MB/s 2024-04-25T10:13:36.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.792 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110974 s, 94.5 MB/s 2024-04-25T10:13:36.806 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.806 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.806 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118353 s, 88.6 MB/s 2024-04-25T10:13:36.820 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.820 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.820 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111984 s, 93.6 MB/s 2024-04-25T10:13:36.834 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.834 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.834 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903394 s, 116 MB/s 2024-04-25T10:13:36.846 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.846 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.846 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113792 s, 92.1 MB/s 2024-04-25T10:13:36.862 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.862 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.862 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127965 s, 81.9 MB/s 2024-04-25T10:13:36.876 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113539 s, 92.4 MB/s 2024-04-25T10:13:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903355 s, 116 MB/s 2024-04-25T10:13:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117441 s, 89.3 MB/s 2024-04-25T10:13:36.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.913 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905312 s, 116 MB/s 2024-04-25T10:13:36.928 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.928 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.928 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114765 s, 91.4 MB/s 2024-04-25T10:13:36.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.942 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117319 s, 89.4 MB/s 2024-04-25T10:13:36.954 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.954 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.954 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911515 s, 115 MB/s 2024-04-25T10:13:36.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.968 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114708 s, 91.4 MB/s 2024-04-25T10:13:36.982 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:36.982 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:36.982 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111586 s, 94.0 MB/s 2024-04-25T10:13:36.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:36.983 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:36.983 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:36.983 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:13:36.983 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:36.984 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:36.984 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:36.984 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:37.047 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.047 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.047 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108197 s, 96.9 MB/s 2024-04-25T10:13:37.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.063 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127924 s, 82.0 MB/s 2024-04-25T10:13:37.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.078 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125919 s, 83.3 MB/s 2024-04-25T10:13:37.091 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.091 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.091 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961558 s, 109 MB/s 2024-04-25T10:13:37.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.107 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129723 s, 80.8 MB/s 2024-04-25T10:13:37.120 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.120 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.120 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010559 s, 99.3 MB/s 2024-04-25T10:13:37.135 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.136 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.136 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123647 s, 84.8 MB/s 2024-04-25T10:13:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112692 s, 93.0 MB/s 2024-04-25T10:13:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123615 s, 84.8 MB/s 2024-04-25T10:13:37.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.180 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125821 s, 83.3 MB/s 2024-04-25T10:13:37.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.195 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.195 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115903 s, 90.5 MB/s 2024-04-25T10:13:37.209 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.209 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.209 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111072 s, 94.4 MB/s 2024-04-25T10:13:37.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.225 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129813 s, 80.8 MB/s 2024-04-25T10:13:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109803 s, 95.5 MB/s 2024-04-25T10:13:37.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.254 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.254 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117128 s, 89.5 MB/s 2024-04-25T10:13:37.266 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.267 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.267 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108696 s, 96.5 MB/s 2024-04-25T10:13:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857305 s, 122 MB/s 2024-04-25T10:13:37.291 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.291 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.291 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106732 s, 98.2 MB/s 2024-04-25T10:13:37.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.307 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131449 s, 79.8 MB/s 2024-04-25T10:13:37.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.322 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121917 s, 86.0 MB/s 2024-04-25T10:13:37.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.335 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993711 s, 106 MB/s 2024-04-25T10:13:37.349 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.349 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.349 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119352 s, 87.9 MB/s 2024-04-25T10:13:37.364 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.364 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.364 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120651 s, 86.9 MB/s 2024-04-25T10:13:37.378 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.378 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.378 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109823 s, 95.5 MB/s 2024-04-25T10:13:37.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.391 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100902 s, 104 MB/s 2024-04-25T10:13:37.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.406 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124486 s, 84.2 MB/s 2024-04-25T10:13:37.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.421 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01225 s, 85.6 MB/s 2024-04-25T10:13:37.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.436 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117733 s, 89.1 MB/s 2024-04-25T10:13:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111974 s, 93.6 MB/s 2024-04-25T10:13:37.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.462 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960873 s, 109 MB/s 2024-04-25T10:13:37.474 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.474 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.474 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914688 s, 115 MB/s 2024-04-25T10:13:37.490 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.490 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.490 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129211 s, 81.2 MB/s 2024-04-25T10:13:37.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:37.491 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:37.491 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:37.492 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:13:37.492 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:37.492 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:37.492 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:37.492 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:37.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.559 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128729 s, 81.5 MB/s 2024-04-25T10:13:37.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.575 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126604 s, 82.8 MB/s 2024-04-25T10:13:37.589 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.589 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.589 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117326 s, 89.4 MB/s 2024-04-25T10:13:37.601 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.602 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974997 s, 108 MB/s 2024-04-25T10:13:37.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.616 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114052 s, 91.9 MB/s 2024-04-25T10:13:37.631 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.631 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.631 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012771 s, 82.1 MB/s 2024-04-25T10:13:37.644 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.644 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.644 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107423 s, 97.6 MB/s 2024-04-25T10:13:37.658 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.658 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.658 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110758 s, 94.7 MB/s 2024-04-25T10:13:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124192 s, 84.4 MB/s 2024-04-25T10:13:37.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.692 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118476 s, 88.5 MB/s 2024-04-25T10:13:37.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.739 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992642 s, 106 MB/s 2024-04-25T10:13:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124115 s, 84.5 MB/s 2024-04-25T10:13:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.771 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120577 s, 87.0 MB/s 2024-04-25T10:13:37.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.787 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126248 s, 83.1 MB/s 2024-04-25T10:13:37.801 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.801 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.801 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110118 s, 95.2 MB/s 2024-04-25T10:13:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123108 s, 85.2 MB/s 2024-04-25T10:13:37.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.832 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116428 s, 90.1 MB/s 2024-04-25T10:13:37.850 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.850 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.850 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136133 s, 77.0 MB/s 2024-04-25T10:13:37.866 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.866 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.866 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124584 s, 84.2 MB/s 2024-04-25T10:13:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.881 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109659 s, 95.6 MB/s 2024-04-25T10:13:37.896 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.896 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.896 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129045 s, 81.3 MB/s 2024-04-25T10:13:37.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.913 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132102 s, 79.4 MB/s 2024-04-25T10:13:37.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.930 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128133 s, 81.8 MB/s 2024-04-25T10:13:37.946 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.947 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136351 s, 76.9 MB/s 2024-04-25T10:13:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.960 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104342 s, 100 MB/s 2024-04-25T10:13:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.974 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011419 s, 91.8 MB/s 2024-04-25T10:13:37.990 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:37.990 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:37.991 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134852 s, 77.8 MB/s 2024-04-25T10:13:38.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.005 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120099 s, 87.3 MB/s 2024-04-25T10:13:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867897 s, 121 MB/s 2024-04-25T10:13:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011099 s, 94.5 MB/s 2024-04-25T10:13:38.045 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.045 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.045 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113918 s, 92.0 MB/s 2024-04-25T10:13:38.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.059 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010909 s, 96.1 MB/s 2024-04-25T10:13:38.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:38.062 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:38.062 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:38.062 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:13:38.062 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:38.062 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:38.062 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:38.062 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:38.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.137 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145868 s, 71.9 MB/s 2024-04-25T10:13:38.153 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.153 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.153 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013139 s, 79.8 MB/s 2024-04-25T10:13:38.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.169 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123384 s, 85.0 MB/s 2024-04-25T10:13:38.183 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.183 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.183 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103027 s, 102 MB/s 2024-04-25T10:13:38.195 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.195 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.195 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881635 s, 119 MB/s 2024-04-25T10:13:38.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.210 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116729 s, 89.8 MB/s 2024-04-25T10:13:38.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.224 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100386 s, 104 MB/s 2024-04-25T10:13:38.239 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.239 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.240 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120688 s, 86.9 MB/s 2024-04-25T10:13:38.254 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.254 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.254 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113809 s, 92.1 MB/s 2024-04-25T10:13:38.267 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.268 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.268 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103064 s, 102 MB/s 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114056 s, 91.9 MB/s 2024-04-25T10:13:38.294 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.294 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.294 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962532 s, 109 MB/s 2024-04-25T10:13:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.306 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907619 s, 116 MB/s 2024-04-25T10:13:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112248 s, 93.4 MB/s 2024-04-25T10:13:38.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.335 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122411 s, 85.7 MB/s 2024-04-25T10:13:38.350 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.350 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.351 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124675 s, 84.1 MB/s 2024-04-25T10:13:38.365 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.365 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.365 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112585 s, 93.1 MB/s 2024-04-25T10:13:38.380 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.380 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.380 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117259 s, 89.4 MB/s 2024-04-25T10:13:38.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.396 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126538 s, 82.9 MB/s 2024-04-25T10:13:38.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.410 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108582 s, 96.6 MB/s 2024-04-25T10:13:38.425 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.425 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.425 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120138 s, 87.3 MB/s 2024-04-25T10:13:38.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.438 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103355 s, 101 MB/s 2024-04-25T10:13:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.451 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931653 s, 113 MB/s 2024-04-25T10:13:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117287 s, 89.4 MB/s 2024-04-25T10:13:38.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.480 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121422 s, 86.4 MB/s 2024-04-25T10:13:38.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.495 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.495 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124713 s, 84.1 MB/s 2024-04-25T10:13:38.510 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.510 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.510 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125847 s, 83.3 MB/s 2024-04-25T10:13:38.523 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.523 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.523 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106902 s, 98.1 MB/s 2024-04-25T10:13:38.537 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.537 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.537 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113647 s, 92.3 MB/s 2024-04-25T10:13:38.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.552 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122949 s, 85.3 MB/s 2024-04-25T10:13:38.567 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.567 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.567 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128047 s, 81.9 MB/s 2024-04-25T10:13:38.581 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.581 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.581 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116351 s, 90.1 MB/s 2024-04-25T10:13:38.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:38.583 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:38.583 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:38.583 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:13:38.583 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:38.583 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:38.583 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:38.583 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:38.652 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.653 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128747 s, 81.4 MB/s 2024-04-25T10:13:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013037 s, 80.4 MB/s 2024-04-25T10:13:38.685 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.686 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134462 s, 78.0 MB/s 2024-04-25T10:13:38.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.723 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011692 s, 89.7 MB/s 2024-04-25T10:13:38.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.738 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117377 s, 89.3 MB/s 2024-04-25T10:13:38.755 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.756 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.756 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139653 s, 75.1 MB/s 2024-04-25T10:13:38.770 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.771 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123816 s, 84.7 MB/s 2024-04-25T10:13:38.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.785 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010548 s, 99.4 MB/s 2024-04-25T10:13:38.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.800 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106819 s, 98.2 MB/s 2024-04-25T10:13:38.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.816 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126972 s, 82.6 MB/s 2024-04-25T10:13:38.831 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.832 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121203 s, 86.5 MB/s 2024-04-25T10:13:38.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.849 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142473 s, 73.6 MB/s 2024-04-25T10:13:38.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.867 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143459 s, 73.1 MB/s 2024-04-25T10:13:38.883 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.884 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.884 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116936 s, 89.7 MB/s 2024-04-25T10:13:38.899 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.899 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.900 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107445 s, 97.6 MB/s 2024-04-25T10:13:38.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.916 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129485 s, 81.0 MB/s 2024-04-25T10:13:38.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.930 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111813 s, 93.8 MB/s 2024-04-25T10:13:38.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.947 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.947 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130941 s, 80.1 MB/s 2024-04-25T10:13:38.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.964 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014349 s, 73.1 MB/s 2024-04-25T10:13:38.980 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.980 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.980 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012289 s, 85.3 MB/s 2024-04-25T10:13:38.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:38.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:38.998 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147182 s, 71.2 MB/s 2024-04-25T10:13:39.015 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.015 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.015 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014043 s, 74.7 MB/s 2024-04-25T10:13:39.029 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.029 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.029 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109161 s, 96.1 MB/s 2024-04-25T10:13:39.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.043 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.044 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114131 s, 91.9 MB/s 2024-04-25T10:13:39.057 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.057 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.057 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107683 s, 97.4 MB/s 2024-04-25T10:13:39.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.071 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103283 s, 102 MB/s 2024-04-25T10:13:39.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.087 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.087 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981488 s, 107 MB/s 2024-04-25T10:13:39.098 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.098 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.098 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122118 s, 85.9 MB/s 2024-04-25T10:13:39.112 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.112 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.112 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116156 s, 90.3 MB/s 2024-04-25T10:13:39.127 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.127 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.127 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117842 s, 89.0 MB/s 2024-04-25T10:13:39.142 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.143 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.143 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125915 s, 83.3 MB/s 2024-04-25T10:13:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.158 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122928 s, 85.3 MB/s 2024-04-25T10:13:39.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:39.159 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:39.159 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:39.159 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:13:39.159 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:39.159 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:39.159 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:39.159 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:39.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.227 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124214 s, 84.4 MB/s 2024-04-25T10:13:39.242 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.243 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124093 s, 84.5 MB/s 2024-04-25T10:13:39.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.257 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116011 s, 90.4 MB/s 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140532 s, 74.6 MB/s 2024-04-25T10:13:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014439 s, 72.6 MB/s 2024-04-25T10:13:39.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.308 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125927 s, 83.3 MB/s 2024-04-25T10:13:39.325 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.325 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.325 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141878 s, 73.9 MB/s 2024-04-25T10:13:39.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.341 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131357 s, 79.8 MB/s 2024-04-25T10:13:39.358 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.359 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.359 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133828 s, 78.4 MB/s 2024-04-25T10:13:39.374 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.374 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.374 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118015 s, 88.9 MB/s 2024-04-25T10:13:39.390 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.390 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.391 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013236 s, 79.2 MB/s 2024-04-25T10:13:39.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.407 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133195 s, 78.7 MB/s 2024-04-25T10:13:39.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.422 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116291 s, 90.2 MB/s 2024-04-25T10:13:39.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.438 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139357 s, 75.2 MB/s 2024-04-25T10:13:39.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.454 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126106 s, 83.2 MB/s 2024-04-25T10:13:39.469 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.469 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.470 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123804 s, 84.7 MB/s 2024-04-25T10:13:39.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.486 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132672 s, 79.0 MB/s 2024-04-25T10:13:39.502 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.503 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.503 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136447 s, 76.8 MB/s 2024-04-25T10:13:39.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.516 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.516 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010052 s, 104 MB/s 2024-04-25T10:13:39.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.530 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.530 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106893 s, 98.1 MB/s 2024-04-25T10:13:39.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.546 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126694 s, 82.8 MB/s 2024-04-25T10:13:39.561 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.561 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.561 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120362 s, 87.1 MB/s 2024-04-25T10:13:39.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.578 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.578 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132898 s, 78.9 MB/s 2024-04-25T10:13:39.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.592 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010836 s, 96.8 MB/s 2024-04-25T10:13:39.608 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.608 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.608 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130192 s, 80.5 MB/s 2024-04-25T10:13:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.620 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.621 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924933 s, 113 MB/s 2024-04-25T10:13:39.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.637 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131234 s, 79.9 MB/s 2024-04-25T10:13:39.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.650 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994855 s, 105 MB/s 2024-04-25T10:13:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130674 s, 80.2 MB/s 2024-04-25T10:13:39.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.682 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011831 s, 88.6 MB/s 2024-04-25T10:13:39.699 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.700 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.700 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131551 s, 79.7 MB/s 2024-04-25T10:13:39.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.740 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142779 s, 73.4 MB/s 2024-04-25T10:13:39.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:39.741 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:39.741 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:39.742 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:13:39.742 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:39.742 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:39.742 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:39.742 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:39.812 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.812 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.812 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127759 s, 82.1 MB/s 2024-04-25T10:13:39.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.826 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103168 s, 102 MB/s 2024-04-25T10:13:39.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.843 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141629 s, 74.0 MB/s 2024-04-25T10:13:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.858 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122471 s, 85.6 MB/s 2024-04-25T10:13:39.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.874 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011914 s, 88.0 MB/s 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012719 s, 82.4 MB/s 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.907 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137373 s, 76.3 MB/s 2024-04-25T10:13:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.922 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120695 s, 86.9 MB/s 2024-04-25T10:13:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.939 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.939 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136217 s, 77.0 MB/s 2024-04-25T10:13:39.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.953 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115704 s, 90.6 MB/s 2024-04-25T10:13:39.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.968 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120617 s, 86.9 MB/s 2024-04-25T10:13:39.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:39.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:39.983 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012276 s, 85.4 MB/s 2024-04-25T10:13:40.001 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.001 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.001 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151485 s, 69.2 MB/s 2024-04-25T10:13:40.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.020 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154442 s, 67.9 MB/s 2024-04-25T10:13:40.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.037 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141822 s, 73.9 MB/s 2024-04-25T10:13:40.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.053 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131263 s, 79.9 MB/s 2024-04-25T10:13:40.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.070 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139395 s, 75.2 MB/s 2024-04-25T10:13:40.086 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.087 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130697 s, 80.2 MB/s 2024-04-25T10:13:40.106 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.108 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.108 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013132 s, 79.8 MB/s 2024-04-25T10:13:40.119 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.119 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.120 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133652 s, 78.5 MB/s 2024-04-25T10:13:40.134 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.135 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.135 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121909 s, 86.0 MB/s 2024-04-25T10:13:40.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126071 s, 83.2 MB/s 2024-04-25T10:13:40.167 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.167 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.167 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137411 s, 76.3 MB/s 2024-04-25T10:13:40.181 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.181 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.182 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010993 s, 95.4 MB/s 2024-04-25T10:13:40.198 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.198 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.198 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013169 s, 79.6 MB/s 2024-04-25T10:13:40.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.212 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.212 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102737 s, 102 MB/s 2024-04-25T10:13:40.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.228 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127172 s, 82.5 MB/s 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136385 s, 76.9 MB/s 2024-04-25T10:13:40.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.261 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133937 s, 78.3 MB/s 2024-04-25T10:13:40.277 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.277 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.277 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126592 s, 82.8 MB/s 2024-04-25T10:13:40.292 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.292 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.292 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012506 s, 83.8 MB/s 2024-04-25T10:13:40.310 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.311 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131496 s, 79.7 MB/s 2024-04-25T10:13:40.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:40.311 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:40.311 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:40.311 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:13:40.311 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:40.311 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:40.311 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:40.312 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:40.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.379 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012485 s, 84.0 MB/s 2024-04-25T10:13:40.394 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.394 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.394 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012448 s, 84.2 MB/s 2024-04-25T10:13:40.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.410 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119655 s, 87.6 MB/s 2024-04-25T10:13:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126831 s, 82.7 MB/s 2024-04-25T10:13:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108349 s, 96.8 MB/s 2024-04-25T10:13:40.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128201 s, 81.8 MB/s 2024-04-25T10:13:40.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.471 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.471 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125523 s, 83.5 MB/s 2024-04-25T10:13:40.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.488 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145285 s, 72.2 MB/s 2024-04-25T10:13:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.504 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.504 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128064 s, 81.9 MB/s 2024-04-25T10:13:40.520 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.520 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.520 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136025 s, 77.1 MB/s 2024-04-25T10:13:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135583 s, 77.3 MB/s 2024-04-25T10:13:40.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.552 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134503 s, 78.0 MB/s 2024-04-25T10:13:40.567 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.567 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.567 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118258 s, 88.7 MB/s 2024-04-25T10:13:40.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.582 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117651 s, 89.1 MB/s 2024-04-25T10:13:40.597 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.597 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.597 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126065 s, 83.2 MB/s 2024-04-25T10:13:40.614 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.614 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.614 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135664 s, 77.3 MB/s 2024-04-25T10:13:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966668 s, 108 MB/s 2024-04-25T10:13:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997807 s, 105 MB/s 2024-04-25T10:13:40.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.654 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127354 s, 82.3 MB/s 2024-04-25T10:13:40.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.672 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.672 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146354 s, 71.6 MB/s 2024-04-25T10:13:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117422 s, 89.3 MB/s 2024-04-25T10:13:40.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.703 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145383 s, 72.1 MB/s 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.729 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122552 s, 85.6 MB/s 2024-04-25T10:13:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.745 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132669 s, 79.0 MB/s 2024-04-25T10:13:40.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.760 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118176 s, 88.7 MB/s 2024-04-25T10:13:40.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.776 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135986 s, 77.1 MB/s 2024-04-25T10:13:40.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.792 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127172 s, 82.5 MB/s 2024-04-25T10:13:40.806 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.806 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.806 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112228 s, 93.4 MB/s 2024-04-25T10:13:40.821 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.821 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.821 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124443 s, 84.3 MB/s 2024-04-25T10:13:40.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.835 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114986 s, 91.2 MB/s 2024-04-25T10:13:40.851 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.851 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.851 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130177 s, 80.5 MB/s 2024-04-25T10:13:40.866 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.867 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125958 s, 83.2 MB/s 2024-04-25T10:13:40.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:40.868 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:40.868 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:40.868 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:13:40.868 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:40.868 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:40.868 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:40.868 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:40.934 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.934 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.934 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121233 s, 86.5 MB/s 2024-04-25T10:13:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.952 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149427 s, 70.2 MB/s 2024-04-25T10:13:40.968 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.969 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137627 s, 76.2 MB/s 2024-04-25T10:13:40.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:40.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:40.985 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134589 s, 77.9 MB/s 2024-04-25T10:13:41.002 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.002 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.002 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141434 s, 74.1 MB/s 2024-04-25T10:13:41.018 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.018 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.018 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135635 s, 77.3 MB/s 2024-04-25T10:13:41.033 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.034 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012797 s, 81.9 MB/s 2024-04-25T10:13:41.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.049 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121892 s, 86.0 MB/s 2024-04-25T10:13:41.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.063 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.063 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106996 s, 98.0 MB/s 2024-04-25T10:13:41.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.078 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.078 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125386 s, 83.6 MB/s 2024-04-25T10:13:41.093 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.094 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.094 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127477 s, 82.3 MB/s 2024-04-25T10:13:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102943 s, 102 MB/s 2024-04-25T10:13:41.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.124 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140463 s, 74.7 MB/s 2024-04-25T10:13:41.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.138 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101556 s, 103 MB/s 2024-04-25T10:13:41.155 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.155 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.156 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130526 s, 80.3 MB/s 2024-04-25T10:13:41.168 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.169 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103356 s, 101 MB/s 2024-04-25T10:13:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119232 s, 87.9 MB/s 2024-04-25T10:13:41.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.196 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111325 s, 94.2 MB/s 2024-04-25T10:13:41.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.211 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118217 s, 88.7 MB/s 2024-04-25T10:13:41.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.224 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108522 s, 96.6 MB/s 2024-04-25T10:13:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.239 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125672 s, 83.4 MB/s 2024-04-25T10:13:41.254 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.254 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.254 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125875 s, 83.3 MB/s 2024-04-25T10:13:41.266 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.266 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.267 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100597 s, 104 MB/s 2024-04-25T10:13:41.281 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.281 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.281 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124882 s, 84.0 MB/s 2024-04-25T10:13:41.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.296 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117309 s, 89.4 MB/s 2024-04-25T10:13:41.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.307 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846487 s, 124 MB/s 2024-04-25T10:13:41.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.321 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116806 s, 89.8 MB/s 2024-04-25T10:13:41.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.339 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114202 s, 91.8 MB/s 2024-04-25T10:13:41.348 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.348 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.348 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106607 s, 98.4 MB/s 2024-04-25T10:13:41.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.360 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.360 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102057 s, 103 MB/s 2024-04-25T10:13:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817321 s, 128 MB/s 2024-04-25T10:13:41.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.384 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109603 s, 95.7 MB/s 2024-04-25T10:13:41.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:41.385 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:41.385 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:13:41.385 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:13:41.385 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:41.385 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:41.385 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:13:41.385 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:13:41.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.455 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121323 s, 86.4 MB/s 2024-04-25T10:13:41.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.470 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122306 s, 85.7 MB/s 2024-04-25T10:13:41.485 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.485 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.485 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123195 s, 85.1 MB/s 2024-04-25T10:13:41.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.501 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.501 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129925 s, 80.7 MB/s 2024-04-25T10:13:41.514 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.515 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111662 s, 93.9 MB/s 2024-04-25T10:13:41.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.530 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125956 s, 83.2 MB/s 2024-04-25T10:13:41.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.544 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113373 s, 92.5 MB/s 2024-04-25T10:13:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138865 s, 75.5 MB/s 2024-04-25T10:13:41.574 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.575 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.575 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120702 s, 86.9 MB/s 2024-04-25T10:13:41.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.592 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014444 s, 72.6 MB/s 2024-04-25T10:13:41.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.607 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121043 s, 86.6 MB/s 2024-04-25T10:13:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132723 s, 79.0 MB/s 2024-04-25T10:13:41.640 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.640 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.640 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130406 s, 80.4 MB/s 2024-04-25T10:13:41.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.655 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118701 s, 88.3 MB/s 2024-04-25T10:13:41.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.670 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125413 s, 83.6 MB/s 2024-04-25T10:13:41.687 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.688 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.688 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144046 s, 72.8 MB/s 2024-04-25T10:13:41.706 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.706 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.706 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155435 s, 67.5 MB/s 2024-04-25T10:13:41.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.729 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108145 s, 97.0 MB/s 2024-04-25T10:13:41.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.745 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123448 s, 84.9 MB/s 2024-04-25T10:13:41.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.761 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.761 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136812 s, 76.6 MB/s 2024-04-25T10:13:41.774 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.775 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108824 s, 96.4 MB/s 2024-04-25T10:13:41.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.790 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.790 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127189 s, 82.4 MB/s 2024-04-25T10:13:41.803 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.804 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010828 s, 96.8 MB/s 2024-04-25T10:13:41.818 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.818 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.818 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114841 s, 91.3 MB/s 2024-04-25T10:13:41.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.830 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.830 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979046 s, 107 MB/s 2024-04-25T10:13:41.845 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.845 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.845 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117648 s, 89.1 MB/s 2024-04-25T10:13:41.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.858 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101337 s, 103 MB/s 2024-04-25T10:13:41.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.874 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124442 s, 84.3 MB/s 2024-04-25T10:13:41.887 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.888 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118111 s, 88.8 MB/s 2024-04-25T10:13:41.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.900 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101193 s, 104 MB/s 2024-04-25T10:13:41.915 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.915 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.916 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121083 s, 86.6 MB/s 2024-04-25T10:13:41.932 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:13:41.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:13:41.933 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137364 s, 76.3 MB/s 2024-04-25T10:13:41.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:41.934 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:13:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb860ad3640 1 Processor -- start 2024-04-25T10:13:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb860ad3640 1 -- start start 2024-04-25T10:13:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb860ad3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb85c15b530 0x7fb85c15b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb860ad3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb85c0cc890 0x7fb85c0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb860ad3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb85c0ce690 0x7fb85c0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb860ad3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85c1589c0 con 0x7fb85c0cc890 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb860ad3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85c15bf40 con 0x7fb85c15b530 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb860ad3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c15c200 con 0x7fb85c0ce690 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb859d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb85c0cc890 0x7fb85c0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb859d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb85c0cc890 0x7fb85c0ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50466/0 (socket says 172.21.15.133:50466) 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb859d9b640 1 -- 172.21.15.133:0/2834917038 learned_addr learned my addr 172.21.15.133:0/2834917038 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/2834917038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb85c0ce690 0x7fb85c0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb85a59c640 1 --2- 172.21.15.133:0/2834917038 >> v2:172.21.15.162:3300/0 conn(0x7fb85c15b530 0x7fb85c15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb859d9b640 1 -- 172.21.15.133:0/2834917038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb85c0ce690 msgr2=0x7fb85c0ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb859d9b640 1 --2- 172.21.15.133:0/2834917038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb85c0ce690 0x7fb85c0ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb859d9b640 1 -- 172.21.15.133:0/2834917038 >> v2:172.21.15.162:3300/0 conn(0x7fb85c15b530 msgr2=0x7fb85c15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb859d9b640 1 --2- 172.21.15.133:0/2834917038 >> v2:172.21.15.162:3300/0 conn(0x7fb85c15b530 0x7fb85c15b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb859d9b640 1 -- 172.21.15.133:0/2834917038 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c15c4e0 con 0x7fb85c0cc890 2024-04-25T10:13:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.108+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/2834917038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb85c0ce690 0x7fb85c0ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb859d9b640 1 --2- 172.21.15.133:0/2834917038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb85c0cc890 0x7fb85c0ccd40 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fb85c070b40 tx=0x7fb85001cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e1bcf86af630ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb85959a640 1 -- 172.21.15.133:0/2834917038 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8500b2070 con 0x7fb85c0cc890 2024-04-25T10:13:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb85959a640 1 -- 172.21.15.133:0/2834917038 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8500a9070 con 0x7fb85c0cc890 2024-04-25T10:13:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb85959a640 1 -- 172.21.15.133:0/2834917038 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8500ad880 con 0x7fb85c0cc890 2024-04-25T10:13:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb860ad3640 1 -- 172.21.15.133:0/2834917038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb85c0cc890 msgr2=0x7fb85c0ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb860ad3640 1 --2- 172.21.15.133:0/2834917038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb85c0cc890 0x7fb85c0ccd40 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fb85c070b40 tx=0x7fb85001cef0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb860ad3640 1 -- 172.21.15.133:0/2834917038 shutdown_connections 2024-04-25T10:13:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb860ad3640 1 --2- 172.21.15.133:0/2834917038 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb85c0ce690 0x7fb85c0ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb860ad3640 1 --2- 172.21.15.133:0/2834917038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb85c0cc890 0x7fb85c0ccd40 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb860ad3640 1 --2- 172.21.15.133:0/2834917038 >> v2:172.21.15.162:3300/0 conn(0x7fb85c15b530 0x7fb85c15b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb860ad3640 1 -- 172.21.15.133:0/2834917038 >> 172.21.15.133:0/2834917038 conn(0x7fb85c06d370 msgr2=0x7fb85c06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb860ad3640 1 -- 172.21.15.133:0/2834917038 shutdown_connections 2024-04-25T10:13:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb860ad3640 1 -- 172.21.15.133:0/2834917038 wait complete. 2024-04-25T10:13:42.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb860ad3640 1 Processor -- start 2024-04-25T10:13:42.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.112+0000 7fb860ad3640 1 -- start start 2024-04-25T10:13:42.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb860ad3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb85c0cc890 0x7fb85c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb860ad3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c0ce690 0x7fb85c156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb860ad3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb85c15b530 0x7fb85c1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb860ad3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c15ef10 con 0x7fb85c15b530 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb860ad3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c15ed90 con 0x7fb85c0ce690 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb860ad3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c15f090 con 0x7fb85c0cc890 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb85c15b530 0x7fb85c1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb85c15b530 0x7fb85c1574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50476/0 (socket says 172.21.15.133:50476) 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85ad9d640 1 -- 172.21.15.133:0/1727637934 learned_addr learned my addr 172.21.15.133:0/1727637934 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb859d9b640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c0ce690 0x7fb85c156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85a59c640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb85c0cc890 0x7fb85c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85ad9d640 1 -- 172.21.15.133:0/1727637934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb85c0cc890 msgr2=0x7fb85c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb85c0cc890 0x7fb85c15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85ad9d640 1 -- 172.21.15.133:0/1727637934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c0ce690 msgr2=0x7fb85c156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c0ce690 0x7fb85c156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85ad9d640 1 -- 172.21.15.133:0/1727637934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c155440 con 0x7fb85c15b530 2024-04-25T10:13:42.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb85c15b530 0x7fb85c1574f0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fb85400cd10 tx=0x7fb85400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb84affd640 1 -- 172.21.15.133:0/1727637934 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85400f680 con 0x7fb85c15b530 2024-04-25T10:13:42.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb84affd640 1 -- 172.21.15.133:0/1727637934 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85400f840 con 0x7fb85c15b530 2024-04-25T10:13:42.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb860ad3640 1 -- 172.21.15.133:0/1727637934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c1556e0 con 0x7fb85c15b530 2024-04-25T10:13:42.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb84affd640 1 -- 172.21.15.133:0/1727637934 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85400fc30 con 0x7fb85c15b530 2024-04-25T10:13:42.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb860ad3640 1 -- 172.21.15.133:0/1727637934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c155ca0 con 0x7fb85c15b530 2024-04-25T10:13:42.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb84affd640 1 -- 172.21.15.133:0/1727637934 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8540213f0 con 0x7fb85c15b530 2024-04-25T10:13:42.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb84affd640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb83c066f60 0x7fb83c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85a59c640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb83c066f60 0x7fb83c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb84affd640 1 -- 172.21.15.133:0/1727637934 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb854085db0 con 0x7fb85c15b530 2024-04-25T10:13:42.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.116+0000 7fb85a59c640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb83c066f60 0x7fb83c069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00a8d0 tx=0x7fb84c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 Processor -- start 2024-04-25T10:13:42.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 -- start start 2024-04-25T10:13:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae873f0 0x562a1ae877d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae3dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 0x562a1ae3e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1ac8bd50 con 0x562a1ae87d10 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1ac13e40 con 0x562a1ae3e520 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a1ac87df0 con 0x562a1ae873f0 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb859d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae3dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb859d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae3dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50480/0 (socket says 172.21.15.133:50480) 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb859d9b640 1 -- 172.21.15.133:0/1513972797 learned_addr learned my addr 172.21.15.133:0/1513972797 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb85a59c640 1 --2- 172.21.15.133:0/1513972797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae873f0 0x562a1ae877d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb859d9b640 1 -- 172.21.15.133:0/1513972797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae873f0 msgr2=0x562a1ae877d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/1513972797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 0x562a1ae3e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb859d9b640 1 --2- 172.21.15.133:0/1513972797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae873f0 0x562a1ae877d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb859d9b640 1 -- 172.21.15.133:0/1513972797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 msgr2=0x562a1ae3e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb859d9b640 1 --2- 172.21.15.133:0/1513972797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 0x562a1ae3e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb859d9b640 1 -- 172.21.15.133:0/1513972797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a1ac28d90 con 0x562a1ae87d10 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb859d9b640 1 --2- 172.21.15.133:0/1513972797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae3dfe0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fb85001cd30 tx=0x7fb8500046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7e724f910f7e33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb83bfff640 1 -- 172.21.15.133:0/1513972797 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8500b2040 con 0x562a1ae87d10 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb83bfff640 1 -- 172.21.15.133:0/1513972797 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8500a9040 con 0x562a1ae87d10 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb83bfff640 1 -- 172.21.15.133:0/1513972797 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8500ada40 con 0x562a1ae87d10 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 -- 172.21.15.133:0/1513972797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 msgr2=0x562a1ae3dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 --2- 172.21.15.133:0/1513972797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae3dfe0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fb85001cd30 tx=0x7fb8500046d0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 -- 172.21.15.133:0/1513972797 shutdown_connections 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 --2- 172.21.15.133:0/1513972797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 0x562a1ae3e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 --2- 172.21.15.133:0/1513972797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae3dfe0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 --2- 172.21.15.133:0/1513972797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae873f0 0x562a1ae877d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 -- 172.21.15.133:0/1513972797 >> 172.21.15.133:0/1513972797 conn(0x562a1ae3a4d0 msgr2=0x562a1ae3c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 -- 172.21.15.133:0/1513972797 shutdown_connections 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 -- 172.21.15.133:0/1513972797 wait complete. 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 -- start start 2024-04-25T10:13:42.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.120+0000 7fb863c40000 1 Processor -- start 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae3e520 0x562a1ae8e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae873f0 0x562a1ae8ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae48bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1ac31610 con 0x562a1ae87d10 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1ac469a0 con 0x562a1ae873f0 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a1ac90380 con 0x562a1ae3e520 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 10 client.0 _ll_register_callbacks cb 0x562a1ad400d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae48bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae48bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50496/0 (socket says 172.21.15.133:50496) 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85ad9d640 1 -- 172.21.15.133:0/1059195037 learned_addr learned my addr 172.21.15.133:0/1059195037 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85a59c640 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae3e520 0x562a1ae8e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb859d9b640 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae873f0 0x562a1ae8ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85ad9d640 1 -- 172.21.15.133:0/1059195037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae3e520 msgr2=0x562a1ae8e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae3e520 0x562a1ae8e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85ad9d640 1 -- 172.21.15.133:0/1059195037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae873f0 msgr2=0x562a1ae8ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae873f0 0x562a1ae8ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85ad9d640 1 -- 172.21.15.133:0/1059195037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a1ac86d40 con 0x562a1ae87d10 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb859d9b640 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae873f0 0x562a1ae8ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae48bf0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fb85405e7a0 tx=0x7fb854056e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e77dbf3a433d170 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb8397fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb8397fa640 1 -- 172.21.15.133:0/1059195037 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85405ecd0 con 0x562a1ae87d10 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb8397fa640 1 -- 172.21.15.133:0/1059195037 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb85405e840 con 0x562a1ae87d10 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 -- 172.21.15.133:0/1059195037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8500a9040 con 0x562a1ae87d10 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb8397fa640 1 -- 172.21.15.133:0/1059195037 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb85405ea40 con 0x562a1ae87d10 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 10 client.9708 fetch_fsmap learned FSMap version 79 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 -- 172.21.15.133:0/1059195037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x562a1ad0ef00 con 0x562a1ae87d10 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb8397fa640 1 -- 172.21.15.133:0/1059195037 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8500a9040 con 0x562a1ae87d10 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb8397fa640 1 -- 172.21.15.133:0/1059195037 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb85407db40 con 0x562a1ae87d10 2024-04-25T10:13:42.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 10 client.9708 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 10 client.9708 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 20 client.9708 populate_metadata read hostname 'smithi133' 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x562a1ae49930 0x562a1ae49d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 4 client.9708 mds_command: new command op to 9495 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 -- 172.21.15.133:0/1059195037 --> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a1ac27a20 con 0x562a1ae49930 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85a59c640 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x562a1ae49930 0x562a1ae49d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb85a59c640 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x562a1ae49930 0x562a1ae49d10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83d30865184e6ef5 server_cookie=59563748f13a48e in_seq=0 out_seq=0 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb8397fa640 10 client.9708 ms_handle_connect on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb8397fa640 1 -- 172.21.15.133:0/1059195037 <== mds.0 v2:172.21.15.133:6836/2351700644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a1ac27a20 con 0x562a1ae49930 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb8397fa640 10 client.9708 handle_command_reply: tid=0 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 client.9708 shutdown 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 -- 172.21.15.133:0/1059195037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 msgr2=0x562a1ae48bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae48bf0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fb85405e7a0 tx=0x7fb854056e00 comp rx=0 tx=0).stop 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 -- 172.21.15.133:0/1059195037 shutdown_connections 2024-04-25T10:13:42.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x562a1ae49930 0x562a1ae49d10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae48bf0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae873f0 0x562a1ae8ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb8397fa640 0 client.9708 ms_handle_reset on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 --2- 172.21.15.133:0/1059195037 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae3e520 0x562a1ae8e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 -- 172.21.15.133:0/1059195037 >> 172.21.15.133:0/1059195037 conn(0x562a1ae3a4d0 msgr2=0x562a1ae3d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 -- 172.21.15.133:0/1059195037 shutdown_connections 2024-04-25T10:13:42.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 1 -- 172.21.15.133:0/1059195037 wait complete. 2024-04-25T10:13:42.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.124+0000 7fb863c40000 20 client.9708 trim_cache size 0 max 16384 2024-04-25T10:13:42.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb863c40000 1 Processor -- start 2024-04-25T10:13:42.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb863c40000 1 -- start start 2024-04-25T10:13:42.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb863c40000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 0x562a1ae3df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb863c40000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae873f0 0x562a1ae42d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb863c40000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae432d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb863c40000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c200e00 con 0x562a1ae87d10 2024-04-25T10:13:42.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb863c40000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c200060 con 0x562a1ae3e520 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb863c40000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0727f0 con 0x562a1ae873f0 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb85ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae432d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50510/0 (socket says 172.21.15.133:50510) 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb85ad9d640 1 -- 172.21.15.133:0/3626143170 learned_addr learned my addr 172.21.15.133:0/3626143170 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb85ad9d640 1 -- 172.21.15.133:0/3626143170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae873f0 msgr2=0x562a1ae42d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/3626143170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae873f0 0x562a1ae42d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb85ad9d640 1 -- 172.21.15.133:0/3626143170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 msgr2=0x562a1ae3df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/3626143170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 0x562a1ae3df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb85ad9d640 1 -- 172.21.15.133:0/3626143170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a1ace5330 con 0x562a1ae87d10 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/3626143170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae432d0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fb85406e200 tx=0x7fb85401a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58e5d3454db548 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb83b7fe640 1 -- 172.21.15.133:0/3626143170 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85405ecd0 con 0x562a1ae87d10 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.140+0000 7fb83b7fe640 1 -- 172.21.15.133:0/3626143170 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8540548c0 con 0x562a1ae87d10 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb83b7fe640 1 -- 172.21.15.133:0/3626143170 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854055f20 con 0x562a1ae87d10 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- 172.21.15.133:0/3626143170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 msgr2=0x562a1ae432d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 --2- 172.21.15.133:0/3626143170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae432d0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fb85406e200 tx=0x7fb85401a070 comp rx=0 tx=0).stop 2024-04-25T10:13:42.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- 172.21.15.133:0/3626143170 shutdown_connections 2024-04-25T10:13:42.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 --2- 172.21.15.133:0/3626143170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae87d10 0x562a1ae432d0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 --2- 172.21.15.133:0/3626143170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae873f0 0x562a1ae42d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 --2- 172.21.15.133:0/3626143170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 0x562a1ae3df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- 172.21.15.133:0/3626143170 >> 172.21.15.133:0/3626143170 conn(0x562a1ae3a4d0 msgr2=0x562a1ae94c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- 172.21.15.133:0/3626143170 shutdown_connections 2024-04-25T10:13:42.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- 172.21.15.133:0/3626143170 wait complete. 2024-04-25T10:13:42.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- start start 2024-04-25T10:13:42.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 Processor -- start 2024-04-25T10:13:42.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 0x562a1ae8b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae873f0 0x562a1aea4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae87d10 0x562a1aea5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1ac42dc0 con 0x562a1ae873f0 2024-04-25T10:13:42.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0b3f40 con 0x562a1ae3e520 2024-04-25T10:13:42.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a1ac8b060 con 0x562a1ae87d10 2024-04-25T10:13:42.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 10 client.0 _ll_register_callbacks cb 0x7fb8500096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:42.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb859d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae873f0 0x562a1aea4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb859d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae873f0 0x562a1aea4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50518/0 (socket says 172.21.15.133:50518) 2024-04-25T10:13:42.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb859d9b640 1 -- 172.21.15.133:0/2204076680 learned_addr learned my addr 172.21.15.133:0/2204076680 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb85ad9d640 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae87d10 0x562a1aea5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb85a59c640 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 0x562a1ae8b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb859d9b640 1 -- 172.21.15.133:0/2204076680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae87d10 msgr2=0x562a1aea5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb859d9b640 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae87d10 0x562a1aea5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb859d9b640 1 -- 172.21.15.133:0/2204076680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 msgr2=0x562a1ae8b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb859d9b640 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 0x562a1ae8b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb859d9b640 1 -- 172.21.15.133:0/2204076680 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a1ac585f0 con 0x562a1ae873f0 2024-04-25T10:13:42.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb859d9b640 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae873f0 0x562a1aea4c00 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fb85001a0c0 tx=0x7fb850019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddec078a6e1981a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb8397fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:13:42.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb8397fa640 1 -- 172.21.15.133:0/2204076680 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8500b2050 con 0x562a1ae873f0 2024-04-25T10:13:42.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb8397fa640 1 -- 172.21.15.133:0/2204076680 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8500b8880 con 0x562a1ae873f0 2024-04-25T10:13:42.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- 172.21.15.133:0/2204076680 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8540548c0 con 0x562a1ae873f0 2024-04-25T10:13:42.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb8397fa640 1 -- 172.21.15.133:0/2204076680 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8500b8a80 con 0x562a1ae873f0 2024-04-25T10:13:42.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 10 client.9714 fetch_fsmap learned FSMap version 79 2024-04-25T10:13:42.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- 172.21.15.133:0/2204076680 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x562a1ac32fe0 con 0x562a1ae873f0 2024-04-25T10:13:42.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb8397fa640 1 -- 172.21.15.133:0/2204076680 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8540548c0 con 0x562a1ae873f0 2024-04-25T10:13:42.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb8397fa640 1 -- 172.21.15.133:0/2204076680 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb8500b5050 con 0x562a1ae873f0 2024-04-25T10:13:42.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 10 client.9714 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:13:42.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 10 client.9714 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:13:42.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 20 client.9714 populate_metadata read hostname 'smithi133' 2024-04-25T10:13:42.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x562a1aea5680 0x562a1aef0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 4 client.9714 mds_command: new command op to 9495 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:13:42.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb863c40000 1 -- 172.21.15.133:0/2204076680 --> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x562a1ac99290 con 0x562a1aea5680 2024-04-25T10:13:42.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.144+0000 7fb85a59c640 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x562a1aea5680 0x562a1aef0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb8397fa640 10 client.9714 ms_handle_connect on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb85a59c640 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x562a1aea5680 0x562a1aef0070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd7469bc95c9d5b server_cookie=574ef7e025ace88d in_seq=0 out_seq=0 2024-04-25T10:13:42.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb8397fa640 1 -- 172.21.15.133:0/2204076680 <== mds.0 v2:172.21.15.133:6836/2351700644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562a1ac99290 con 0x562a1aea5680 2024-04-25T10:13:42.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb8397fa640 10 client.9714 handle_command_reply: tid=0 2024-04-25T10:13:42.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 1 client.9714 shutdown 2024-04-25T10:13:42.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 1 -- 172.21.15.133:0/2204076680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae873f0 msgr2=0x562a1aea4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae873f0 0x562a1aea4c00 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fb85001a0c0 tx=0x7fb850019230 comp rx=0 tx=0).stop 2024-04-25T10:13:42.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 1 -- 172.21.15.133:0/2204076680 shutdown_connections 2024-04-25T10:13:42.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562a1ae87d10 0x562a1aea5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x562a1aea5680 0x562a1aef0070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562a1ae873f0 0x562a1aea4c00 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 1 --2- 172.21.15.133:0/2204076680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562a1ae3e520 0x562a1ae8b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb8397fa640 0 client.9714 ms_handle_reset on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 1 -- 172.21.15.133:0/2204076680 >> 172.21.15.133:0/2204076680 conn(0x562a1ae3a4d0 msgr2=0x562a1ae85ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 1 -- 172.21.15.133:0/2204076680 shutdown_connections 2024-04-25T10:13:42.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 1 -- 172.21.15.133:0/2204076680 wait complete. 2024-04-25T10:13:42.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb863c40000 20 client.9714 trim_cache size 0 max 16384 2024-04-25T10:13:42.164 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:13:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:13:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "1b70c6d4-1b74-439e-b12a-15eb7bc46d90", 2024-04-25T10:13:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:13:42.165 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:13:42.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb860ad3640 1 -- 172.21.15.133:0/1727637934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb83c066f60 msgr2=0x7fb83c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb860ad3640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb83c066f60 0x7fb83c069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00a8d0 tx=0x7fb84c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:42.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb860ad3640 1 -- 172.21.15.133:0/1727637934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb85c15b530 msgr2=0x7fb85c1574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb860ad3640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb85c15b530 0x7fb85c1574f0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fb85400cd10 tx=0x7fb85400a650 comp rx=0 tx=0).stop 2024-04-25T10:13:42.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb860ad3640 1 -- 172.21.15.133:0/1727637934 shutdown_connections 2024-04-25T10:13:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb860ad3640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb83c066f60 0x7fb83c069420 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb860ad3640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb85c15b530 0x7fb85c1574f0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb860ad3640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb85c0ce690 0x7fb85c156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb860ad3640 1 --2- 172.21.15.133:0/1727637934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb85c0cc890 0x7fb85c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.148+0000 7fb860ad3640 1 -- 172.21.15.133:0/1727637934 >> 172.21.15.133:0/1727637934 conn(0x7fb85c06d370 msgr2=0x7fb85c0b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.152+0000 7fb860ad3640 1 -- 172.21.15.133:0/1727637934 shutdown_connections 2024-04-25T10:13:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.152+0000 7fb860ad3640 1 -- 172.21.15.133:0/1727637934 wait complete. 2024-04-25T10:13:42.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:42.172 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T10:13:42.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.308+0000 7fe58a4be640 1 Processor -- start 2024-04-25T10:13:42.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.308+0000 7fe58a4be640 1 -- start start 2024-04-25T10:13:42.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe58a4be640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe5840b5ef0 0x7fe5840ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe58a4be640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe5840d1060 0x7fe58406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe58a4be640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5840d0300 con 0x7fe58406b1d0 2024-04-25T10:13:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe58a4be640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58406b640 con 0x7fe5840d1060 2024-04-25T10:13:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe58a4be640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5840cfaa0 con 0x7fe5840b5ef0 2024-04-25T10:13:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe5837fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe5840d1060 0x7fe58406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe5837fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe5840d1060 0x7fe58406ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50520/0 (socket says 172.21.15.133:50520) 2024-04-25T10:13:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe5837fe640 1 -- 172.21.15.133:0/2606767380 learned_addr learned my addr 172.21.15.133:0/2606767380 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe5837fe640 1 -- 172.21.15.133:0/2606767380 >> v2:172.21.15.162:3300/0 conn(0x7fe5840b5ef0 msgr2=0x7fe5840ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe583fff640 1 --2- 172.21.15.133:0/2606767380 >> v2:172.21.15.162:3300/0 conn(0x7fe5840b5ef0 0x7fe5840ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe5837fe640 1 --2- 172.21.15.133:0/2606767380 >> v2:172.21.15.162:3300/0 conn(0x7fe5840b5ef0 0x7fe5840ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe5837fe640 1 -- 172.21.15.133:0/2606767380 >> v1:172.21.15.162:6789/0 conn(0x7fe58406b1d0 legacy=0x7fe5840ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe5837fe640 1 -- 172.21.15.133:0/2606767380 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5840cc4e0 con 0x7fe5840d1060 2024-04-25T10:13:42.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe5837fe640 1 --2- 172.21.15.133:0/2606767380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5840d1060 0x7fe58406ac90 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fe5840d04d0 tx=0x7fe57401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a05fffd57bc7d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe582ffd640 1 -- 172.21.15.133:0/2606767380 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5740a5760 con 0x7fe5840d1060 2024-04-25T10:13:42.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe582ffd640 1 -- 172.21.15.133:0/2606767380 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5740a5900 con 0x7fe5840d1060 2024-04-25T10:13:42.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe582ffd640 1 -- 172.21.15.133:0/2606767380 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5740a5cd0 con 0x7fe5840d1060 2024-04-25T10:13:42.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe58a4be640 1 -- 172.21.15.133:0/2606767380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5840d1060 msgr2=0x7fe58406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.312+0000 7fe58a4be640 1 --2- 172.21.15.133:0/2606767380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5840d1060 0x7fe58406ac90 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fe5840d04d0 tx=0x7fe57401c980 comp rx=0 tx=0).stop 2024-04-25T10:13:42.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 -- 172.21.15.133:0/2606767380 shutdown_connections 2024-04-25T10:13:42.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 --2- 172.21.15.133:0/2606767380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5840d1060 0x7fe58406ac90 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 --2- 172.21.15.133:0/2606767380 >> v2:172.21.15.162:3300/0 conn(0x7fe5840b5ef0 0x7fe5840ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 -- 172.21.15.133:0/2606767380 >> 172.21.15.133:0/2606767380 conn(0x7fe5840d0570 msgr2=0x7fe5840d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 -- 172.21.15.133:0/2606767380 shutdown_connections 2024-04-25T10:13:42.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 -- 172.21.15.133:0/2606767380 wait complete. 2024-04-25T10:13:42.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 Processor -- start 2024-04-25T10:13:42.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 -- start start 2024-04-25T10:13:42.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe58406b1d0 0x7fe58415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5840b5ef0 0x7fe58414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5840d1060 0x7fe5841513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840b33c0 con 0x7fe5840d1060 2024-04-25T10:13:42.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840cafb0 con 0x7fe58406b1d0 2024-04-25T10:13:42.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840b62d0 con 0x7fe5840b5ef0 2024-04-25T10:13:42.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe588a5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5840d1060 0x7fe5841513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe588a5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5840d1060 0x7fe5841513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50522/0 (socket says 172.21.15.133:50522) 2024-04-25T10:13:42.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe588a5b640 1 -- 172.21.15.133:0/3309684560 learned_addr learned my addr 172.21.15.133:0/3309684560 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe5837fe640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5840b5ef0 0x7fe58414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe588a5b640 1 -- 172.21.15.133:0/3309684560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5840b5ef0 msgr2=0x7fe58414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe588a5b640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5840b5ef0 0x7fe58414feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe583fff640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe58406b1d0 0x7fe58415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe588a5b640 1 -- 172.21.15.133:0/3309684560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe58406b1d0 msgr2=0x7fe58415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe588a5b640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe58406b1d0 0x7fe58415b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe588a5b640 1 -- 172.21.15.133:0/3309684560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe584150630 con 0x7fe5840d1060 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe588a5b640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5840d1060 0x7fe5841513f0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00c410 tx=0x7fe57c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe580ff9640 1 -- 172.21.15.133:0/3309684560 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c017070 con 0x7fe5840d1060 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe580ff9640 1 -- 172.21.15.133:0/3309684560 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c003600 con 0x7fe5840d1060 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 -- 172.21.15.133:0/3309684560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe584150910 con 0x7fe5840d1060 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe580ff9640 1 -- 172.21.15.133:0/3309684560 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0039f0 con 0x7fe5840d1060 2024-04-25T10:13:42.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.316+0000 7fe58a4be640 1 -- 172.21.15.133:0/3309684560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe584151930 con 0x7fe5840d1060 2024-04-25T10:13:42.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe580ff9640 1 -- 172.21.15.133:0/3309684560 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe57c01a070 con 0x7fe5840d1060 2024-04-25T10:13:42.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe580ff9640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe570066f60 0x7fe570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe580ff9640 1 -- 172.21.15.133:0/3309684560 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe57c0844d0 con 0x7fe5840d1060 2024-04-25T10:13:42.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe583fff640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe570066f60 0x7fe570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe583fff640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe570066f60 0x7fe570069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe58406ccd0 tx=0x7fe578008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 Processor -- start 2024-04-25T10:13:42.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 -- start start 2024-04-25T10:13:42.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197ca40 0x55614197ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55614197d360 0x556141937850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556141937d90 0x556141938170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55614181a1f0 con 0x55614197ca40 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556141780e90 con 0x55614197d360 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556141708f50 con 0x556141937d90 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe583fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197ca40 0x55614197ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe583fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197ca40 0x55614197ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50536/0 (socket says 172.21.15.133:50536) 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe583fff640 1 -- 172.21.15.133:0/2702505389 learned_addr learned my addr 172.21.15.133:0/2702505389 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe588a5b640 1 --2- 172.21.15.133:0/2702505389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556141937d90 0x556141938170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe583fff640 1 -- 172.21.15.133:0/2702505389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556141937d90 msgr2=0x556141938170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe5837fe640 1 --2- 172.21.15.133:0/2702505389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55614197d360 0x556141937850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe583fff640 1 --2- 172.21.15.133:0/2702505389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556141937d90 0x556141938170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe583fff640 1 -- 172.21.15.133:0/2702505389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55614197d360 msgr2=0x556141937850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe583fff640 1 --2- 172.21.15.133:0/2702505389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55614197d360 0x556141937850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe583fff640 1 -- 172.21.15.133:0/2702505389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561417588b0 con 0x55614197ca40 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe588a5b640 1 --2- 172.21.15.133:0/2702505389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556141937d90 0x556141938170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe583fff640 1 --2- 172.21.15.133:0/2702505389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197ca40 0x55614197ce20 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fe57800db60 tx=0x7fe578010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd209e799cb027e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe569ffb640 1 -- 172.21.15.133:0/2702505389 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578019050 con 0x55614197ca40 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe569ffb640 1 -- 172.21.15.133:0/2702505389 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578023070 con 0x55614197ca40 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe569ffb640 1 -- 172.21.15.133:0/2702505389 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578012c10 con 0x55614197ca40 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 -- 172.21.15.133:0/2702505389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197ca40 msgr2=0x55614197ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 --2- 172.21.15.133:0/2702505389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197ca40 0x55614197ce20 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fe57800db60 tx=0x7fe578010000 comp rx=0 tx=0).stop 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 -- 172.21.15.133:0/2702505389 shutdown_connections 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 --2- 172.21.15.133:0/2702505389 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556141937d90 0x556141938170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 --2- 172.21.15.133:0/2702505389 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55614197d360 0x556141937850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 --2- 172.21.15.133:0/2702505389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197ca40 0x55614197ce20 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 -- 172.21.15.133:0/2702505389 >> 172.21.15.133:0/2702505389 conn(0x556141933d40 msgr2=0x556141936130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 -- 172.21.15.133:0/2702505389 shutdown_connections 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.320+0000 7fe58d62b000 1 -- 172.21.15.133:0/2702505389 wait complete. 2024-04-25T10:13:42.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- start start 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 Processor -- start 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556141937d90 0x5561419841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197ca40 0x556141984700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x55614193dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55614173b540 con 0x55614197d360 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55614177cf30 con 0x556141937d90 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55614174d7c0 con 0x55614197ca40 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 10 client.0 _ll_register_callbacks cb 0x556141835180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe588a5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x55614193dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe588a5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x55614193dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50548/0 (socket says 172.21.15.133:50548) 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe588a5b640 1 -- 172.21.15.133:0/424173492 learned_addr learned my addr 172.21.15.133:0/424173492 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe5837fe640 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197ca40 0x556141984700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe583fff640 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556141937d90 0x5561419841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe588a5b640 1 -- 172.21.15.133:0/424173492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197ca40 msgr2=0x556141984700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe588a5b640 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197ca40 0x556141984700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe588a5b640 1 -- 172.21.15.133:0/424173492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556141937d90 msgr2=0x5561419841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe588a5b640 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556141937d90 0x5561419841c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe588a5b640 1 -- 172.21.15.133:0/424173492 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561417091a0 con 0x55614197d360 2024-04-25T10:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe588a5b640 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x55614193dd80 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fe57c05cec0 tx=0x7fe57c054ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827516ffa0a2464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe562ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe562ffd640 1 -- 172.21.15.133:0/424173492 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c084070 con 0x55614197d360 2024-04-25T10:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- 172.21.15.133:0/424173492 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe578023070 con 0x55614197d360 2024-04-25T10:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe562ffd640 1 -- 172.21.15.133:0/424173492 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe57c084210 con 0x55614197d360 2024-04-25T10:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe562ffd640 1 -- 172.21.15.133:0/424173492 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe57c003b90 con 0x55614197d360 2024-04-25T10:13:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 10 client.9726 fetch_fsmap learned FSMap version 79 2024-04-25T10:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- 172.21.15.133:0/424173492 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x556141803840 con 0x55614197d360 2024-04-25T10:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe562ffd640 1 -- 172.21.15.133:0/424173492 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578023070 con 0x55614197d360 2024-04-25T10:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe562ffd640 1 -- 172.21.15.133:0/424173492 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe57c07e020 con 0x55614197d360 2024-04-25T10:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 10 client.9726 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 10 client.9726 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 20 client.9726 populate_metadata read hostname 'smithi133' 2024-04-25T10:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x55614193eaf0 0x55614193eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 4 client.9726 mds_command: new command op to 9495 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- 172.21.15.133:0/424173492 --> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55614171cbe0 con 0x55614193eaf0 2024-04-25T10:13:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe583fff640 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x55614193eaf0 0x55614193eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe583fff640 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x55614193eaf0 0x55614193eed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d2178b66c9c65 server_cookie=2400de675a0fa700 in_seq=0 out_seq=0 2024-04-25T10:13:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe562ffd640 10 client.9726 ms_handle_connect on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe562ffd640 1 -- 172.21.15.133:0/424173492 <== mds.0 v2:172.21.15.133:6836/2351700644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55614171cbe0 con 0x55614193eaf0 2024-04-25T10:13:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe562ffd640 10 client.9726 handle_command_reply: tid=0 2024-04-25T10:13:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 client.9726 shutdown 2024-04-25T10:13:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- 172.21.15.133:0/424173492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 msgr2=0x55614193dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x55614193dd80 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fe57c05cec0 tx=0x7fe57c054ce0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- 172.21.15.133:0/424173492 shutdown_connections 2024-04-25T10:13:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x55614193eaf0 0x55614193eed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x55614193dd80 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197ca40 0x556141984700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe562ffd640 0 client.9726 ms_handle_reset on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 --2- 172.21.15.133:0/424173492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556141937d90 0x5561419841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- 172.21.15.133:0/424173492 >> 172.21.15.133:0/424173492 conn(0x556141933d40 msgr2=0x5561419369c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- 172.21.15.133:0/424173492 shutdown_connections 2024-04-25T10:13:42.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 1 -- 172.21.15.133:0/424173492 wait complete. 2024-04-25T10:13:42.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.324+0000 7fe58d62b000 20 client.9726 trim_cache size 0 max 16384 2024-04-25T10:13:42.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe58d62b000 1 Processor -- start 2024-04-25T10:13:42.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe58d62b000 1 -- start start 2024-04-25T10:13:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556141937d90 0x5561419d9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55614197ca40 0x5561419d97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197d360 0x5561419d9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe58d62b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe584203080 con 0x556141937d90 2024-04-25T10:13:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe58d62b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe584202280 con 0x55614197ca40 2024-04-25T10:13:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe58d62b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe584153030 con 0x55614197d360 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe583fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556141937d90 0x5561419d9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe583fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556141937d90 0x5561419d9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50554/0 (socket says 172.21.15.133:50554) 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe583fff640 1 -- 172.21.15.133:0/3340669419 learned_addr learned my addr 172.21.15.133:0/3340669419 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe588a5b640 1 --2- 172.21.15.133:0/3340669419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197d360 0x5561419d9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe583fff640 1 -- 172.21.15.133:0/3340669419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197d360 msgr2=0x5561419d9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe583fff640 1 --2- 172.21.15.133:0/3340669419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197d360 0x5561419d9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe583fff640 1 -- 172.21.15.133:0/3340669419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55614197ca40 msgr2=0x5561419d97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe5837fe640 1 --2- 172.21.15.133:0/3340669419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55614197ca40 0x5561419d97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe583fff640 1 --2- 172.21.15.133:0/3340669419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55614197ca40 0x5561419d97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe583fff640 1 -- 172.21.15.133:0/3340669419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55614197d190 con 0x556141937d90 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe583fff640 1 --2- 172.21.15.133:0/3340669419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556141937d90 0x5561419d9290 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe57800b1d0 tx=0x7fe578012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908a4acd8c4327b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe5697fa640 1 -- 172.21.15.133:0/3340669419 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578036070 con 0x556141937d90 2024-04-25T10:13:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe5697fa640 1 -- 172.21.15.133:0/3340669419 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578023050 con 0x556141937d90 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.340+0000 7fe5697fa640 1 -- 172.21.15.133:0/3340669419 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578019040 con 0x556141937d90 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/3340669419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556141937d90 msgr2=0x5561419d9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- 172.21.15.133:0/3340669419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556141937d90 0x5561419d9290 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fe57800b1d0 tx=0x7fe578012000 comp rx=0 tx=0).stop 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/3340669419 shutdown_connections 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- 172.21.15.133:0/3340669419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197d360 0x5561419d9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- 172.21.15.133:0/3340669419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55614197ca40 0x5561419d97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- 172.21.15.133:0/3340669419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556141937d90 0x5561419d9290 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/3340669419 >> 172.21.15.133:0/3340669419 conn(0x556141933d40 msgr2=0x55614197b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/3340669419 shutdown_connections 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/3340669419 wait complete. 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- start start 2024-04-25T10:13:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 Processor -- start 2024-04-25T10:13:42.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556141937d90 0x5561419dd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197ca40 0x5561419dd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x556141981ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561417801a0 con 0x55614197d360 2024-04-25T10:13:42.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe584201cb0 con 0x556141937d90 2024-04-25T10:13:42.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556141727c00 con 0x55614197ca40 2024-04-25T10:13:42.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 10 client.0 _ll_register_callbacks cb 0x556141743a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:42.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe588a5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x556141981ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe588a5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x556141981ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50556/0 (socket says 172.21.15.133:50556) 2024-04-25T10:13:42.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe588a5b640 1 -- 172.21.15.133:0/2913082247 learned_addr learned my addr 172.21.15.133:0/2913082247 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe5837fe640 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197ca40 0x5561419dd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe583fff640 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556141937d90 0x5561419dd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe588a5b640 1 -- 172.21.15.133:0/2913082247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197ca40 msgr2=0x5561419dd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe588a5b640 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197ca40 0x5561419dd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe588a5b640 1 -- 172.21.15.133:0/2913082247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556141937d90 msgr2=0x5561419dd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe588a5b640 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556141937d90 0x5561419dd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe588a5b640 1 -- 172.21.15.133:0/2913082247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561417daa10 con 0x55614197d360 2024-04-25T10:13:42.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe588a5b640 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x556141981ed0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fe57c001000 tx=0x7fe57c014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30b43ca64f5b16a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe562ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:13:42.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe562ffd640 1 -- 172.21.15.133:0/2913082247 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c054640 con 0x55614197d360 2024-04-25T10:13:42.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe562ffd640 1 -- 172.21.15.133:0/2913082247 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe57c050a30 con 0x55614197d360 2024-04-25T10:13:42.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/2913082247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe578023050 con 0x55614197d360 2024-04-25T10:13:42.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe562ffd640 1 -- 172.21.15.133:0/2913082247 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe57c07b050 con 0x55614197d360 2024-04-25T10:13:42.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 10 client.9732 fetch_fsmap learned FSMap version 79 2024-04-25T10:13:42.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/2913082247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x556141715b00 con 0x55614197d360 2024-04-25T10:13:42.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe562ffd640 1 -- 172.21.15.133:0/2913082247 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578023050 con 0x55614197d360 2024-04-25T10:13:42.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe562ffd640 1 -- 172.21.15.133:0/2913082247 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe57c07e070 con 0x55614197d360 2024-04-25T10:13:42.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 10 client.9732 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:13:42.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 10 client.9732 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:13:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 20 client.9732 populate_metadata read hostname 'smithi133' 2024-04-25T10:13:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x556141982bc0 0x556141982fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 4 client.9732 mds_command: new command op to 9495 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:13:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/2913082247 --> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55614178e3d0 con 0x556141982bc0 2024-04-25T10:13:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe583fff640 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x556141982bc0 0x556141982fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe562ffd640 10 client.9732 ms_handle_connect on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe583fff640 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x556141982bc0 0x556141982fd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781bf82908147a4c server_cookie=31b6b6d0de5e81b5 in_seq=0 out_seq=0 2024-04-25T10:13:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe562ffd640 1 -- 172.21.15.133:0/2913082247 <== mds.0 v2:172.21.15.133:6836/2351700644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55614178e3d0 con 0x556141982bc0 2024-04-25T10:13:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe562ffd640 10 client.9732 handle_command_reply: tid=0 2024-04-25T10:13:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 client.9732 shutdown 2024-04-25T10:13:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/2913082247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 msgr2=0x556141981ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x556141981ed0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fe57c001000 tx=0x7fe57c014070 comp rx=0 tx=0).stop 2024-04-25T10:13:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/2913082247 shutdown_connections 2024-04-25T10:13:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x556141982bc0 0x556141982fd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55614197d360 0x556141981ed0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe562ffd640 0 client.9732 ms_handle_reset on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55614197ca40 0x5561419dd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 --2- 172.21.15.133:0/2913082247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556141937d90 0x5561419dd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/2913082247 >> 172.21.15.133:0/2913082247 conn(0x556141933d40 msgr2=0x55614197b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/2913082247 shutdown_connections 2024-04-25T10:13:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 1 -- 172.21.15.133:0/2913082247 wait complete. 2024-04-25T10:13:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.344+0000 7fe58d62b000 20 client.9732 trim_cache size 0 max 16384 2024-04-25T10:13:42.361 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:13:42.361 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0 2024-04-25T10:13:42.361 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:13:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 -- 172.21.15.133:0/3309684560 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe570066f60 msgr2=0x7fe570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe570066f60 0x7fe570069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe58406ccd0 tx=0x7fe578008040 comp rx=0 tx=0).stop 2024-04-25T10:13:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 -- 172.21.15.133:0/3309684560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5840d1060 msgr2=0x7fe5841513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5840d1060 0x7fe5841513f0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00c410 tx=0x7fe57c00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 -- 172.21.15.133:0/3309684560 shutdown_connections 2024-04-25T10:13:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe570066f60 0x7fe570069420 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5840d1060 0x7fe5841513f0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5840b5ef0 0x7fe58414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 --2- 172.21.15.133:0/3309684560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe58406b1d0 0x7fe58415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 -- 172.21.15.133:0/3309684560 >> 172.21.15.133:0/3309684560 conn(0x7fe5840d0570 msgr2=0x7fe5840d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 -- 172.21.15.133:0/3309684560 shutdown_connections 2024-04-25T10:13:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.348+0000 7fe58a4be640 1 -- 172.21.15.133:0/3309684560 wait complete. 2024-04-25T10:13:42.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:42.364 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T10:13:42.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.464+0000 7f85f20b6640 1 Processor -- start 2024-04-25T10:13:42.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.464+0000 7f85f20b6640 1 -- start start 2024-04-25T10:13:42.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85f20b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ec15f030 0x7f85ec09a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85f20b6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f85ec09a0e0 0x7f85ec0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85f20b6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ec0cf320 con 0x7f85ec0b2550 2024-04-25T10:13:42.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85f20b6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ec0ccdd0 con 0x7f85ec09a0e0 2024-04-25T10:13:42.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85f20b6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec15f410 con 0x7f85ec15f030 2024-04-25T10:13:42.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85ebfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f85ec09a0e0 0x7f85ec0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85ebfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f85ec09a0e0 0x7f85ec0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50568/0 (socket says 172.21.15.133:50568) 2024-04-25T10:13:42.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85eaffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f85ec0b2550 0x7f85ec0999d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54642/0 (socket says 172.21.15.133:54642) 2024-04-25T10:13:42.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85ebfff640 1 -- 172.21.15.133:0/748887712 learned_addr learned my addr 172.21.15.133:0/748887712 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/748887712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ec15f030 0x7f85ec09a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85ebfff640 1 -- 172.21.15.133:0/748887712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ec15f030 msgr2=0x7f85ec09a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85ebfff640 1 --2- 172.21.15.133:0/748887712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ec15f030 0x7f85ec09a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85ebfff640 1 -- 172.21.15.133:0/748887712 >> v1:172.21.15.133:6789/0 conn(0x7f85ec0b2550 legacy=0x7f85ec0999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85ebfff640 1 -- 172.21.15.133:0/748887712 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec0cd020 con 0x7f85ec09a0e0 2024-04-25T10:13:42.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85ea7fc640 1 -- 172.21.15.133:0/748887712 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803671977 0 0) 0x7f85ec0cf320 con 0x7f85ec0b2550 2024-04-25T10:13:42.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.468+0000 7f85ebfff640 1 --2- 172.21.15.133:0/748887712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec09a0e0 0x7f85ec0cf7d0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009e50 tx=0x7f85e401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f59a09d2d4bcbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ea7fc640 1 -- 172.21.15.133:0/748887712 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e40a5850 con 0x7f85ec09a0e0 2024-04-25T10:13:42.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ea7fc640 1 -- 172.21.15.133:0/748887712 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e40a59f0 con 0x7f85ec09a0e0 2024-04-25T10:13:42.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ea7fc640 1 -- 172.21.15.133:0/748887712 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e40a5dc0 con 0x7f85ec09a0e0 2024-04-25T10:13:42.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 -- 172.21.15.133:0/748887712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec09a0e0 msgr2=0x7f85ec0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 --2- 172.21.15.133:0/748887712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec09a0e0 0x7f85ec0cf7d0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009e50 tx=0x7f85e401c970 comp rx=0 tx=0).stop 2024-04-25T10:13:42.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 -- 172.21.15.133:0/748887712 shutdown_connections 2024-04-25T10:13:42.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 --2- 172.21.15.133:0/748887712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec09a0e0 0x7f85ec0cf7d0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 --2- 172.21.15.133:0/748887712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ec15f030 0x7f85ec09a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 -- 172.21.15.133:0/748887712 >> 172.21.15.133:0/748887712 conn(0x7f85ec0b31a0 msgr2=0x7f85ec0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 -- 172.21.15.133:0/748887712 shutdown_connections 2024-04-25T10:13:42.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 -- 172.21.15.133:0/748887712 wait complete. 2024-04-25T10:13:42.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 Processor -- start 2024-04-25T10:13:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 -- start start 2024-04-25T10:13:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ec09a0e0 0x7f85ec15e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ec0b2550 0x7f85ec0b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec15f030 0x7f85ec156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec0c5200 con 0x7f85ec15f030 2024-04-25T10:13:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec0b2b80 con 0x7f85ec0b2550 2024-04-25T10:13:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec0c5380 con 0x7f85ec09a0e0 2024-04-25T10:13:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec15f030 0x7f85ec156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec15f030 0x7f85ec156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50574/0 (socket says 172.21.15.133:50574) 2024-04-25T10:13:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ebfff640 1 -- 172.21.15.133:0/1389089677 learned_addr learned my addr 172.21.15.133:0/1389089677 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ec09a0e0 0x7f85ec15e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85eaffd640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ec0b2550 0x7f85ec0b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ebfff640 1 -- 172.21.15.133:0/1389089677 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ec09a0e0 msgr2=0x7f85ec15e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ebfff640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ec09a0e0 0x7f85ec15e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ebfff640 1 -- 172.21.15.133:0/1389089677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ec0b2550 msgr2=0x7f85ec0b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ebfff640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ec0b2550 0x7f85ec0b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ebfff640 1 -- 172.21.15.133:0/1389089677 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec157930 con 0x7f85ec15f030 2024-04-25T10:13:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85eaffd640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ec0b2550 0x7f85ec0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85ebfff640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec15f030 0x7f85ec156060 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f85e40af040 tx=0x7f85e4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85e3fff640 1 -- 172.21.15.133:0/1389089677 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e40b8670 con 0x7f85ec15f030 2024-04-25T10:13:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85e3fff640 1 -- 172.21.15.133:0/1389089677 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e40be070 con 0x7f85ec15f030 2024-04-25T10:13:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85e3fff640 1 -- 172.21.15.133:0/1389089677 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e40b8a70 con 0x7f85ec15f030 2024-04-25T10:13:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 -- 172.21.15.133:0/1389089677 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ec157c10 con 0x7f85ec15f030 2024-04-25T10:13:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.472+0000 7f85f20b6640 1 -- 172.21.15.133:0/1389089677 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec0c8b00 con 0x7f85ec15f030 2024-04-25T10:13:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85e3fff640 1 -- 172.21.15.133:0/1389089677 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85e40b8c10 con 0x7f85ec15f030 2024-04-25T10:13:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85e3fff640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85cc066f60 0x7f85cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85cc066f60 0x7f85cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85e3fff640 1 -- 172.21.15.133:0/1389089677 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85e4120490 con 0x7f85ec15f030 2024-04-25T10:13:42.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85cc066f60 0x7f85cc069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f85d400b9d0 tx=0x7f85d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85f5223000 1 Processor -- start 2024-04-25T10:13:42.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85f5223000 1 -- start start 2024-04-25T10:13:42.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85f5223000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850e9d0 0x5603d850edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85f5223000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850f2f0 0x5603d84c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85f5223000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d84c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85f5223000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d8311ff0 con 0x5603d850f2f0 2024-04-25T10:13:42.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85f5223000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d829a150 con 0x5603d850e9d0 2024-04-25T10:13:42.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85f5223000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603d830e090 con 0x5603d84c50a0 2024-04-25T10:13:42.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850f2f0 0x5603d84c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850f2f0 0x5603d84c4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50586/0 (socket says 172.21.15.133:50586) 2024-04-25T10:13:42.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eaffd640 1 -- 172.21.15.133:0/1944311579 learned_addr learned my addr 172.21.15.133:0/1944311579 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85ebfff640 1 --2- 172.21.15.133:0/1944311579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d84c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/1944311579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850e9d0 0x5603d850edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eaffd640 1 -- 172.21.15.133:0/1944311579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 msgr2=0x5603d84c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eaffd640 1 --2- 172.21.15.133:0/1944311579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d84c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eaffd640 1 -- 172.21.15.133:0/1944311579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850e9d0 msgr2=0x5603d850edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eaffd640 1 --2- 172.21.15.133:0/1944311579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850e9d0 0x5603d850edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eaffd640 1 -- 172.21.15.133:0/1944311579 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603d84a80d0 con 0x5603d850f2f0 2024-04-25T10:13:42.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85eaffd640 1 --2- 172.21.15.133:0/1944311579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850f2f0 0x5603d84c4b60 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f85dc001780 tx=0x7f85dc001c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c34e796f9cefe4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85e17fa640 1 -- 172.21.15.133:0/1944311579 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0074e0 con 0x5603d850f2f0 2024-04-25T10:13:42.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85e17fa640 1 -- 172.21.15.133:0/1944311579 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc0047f0 con 0x5603d850f2f0 2024-04-25T10:13:42.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.476+0000 7f85e17fa640 1 -- 172.21.15.133:0/1944311579 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc007680 con 0x5603d850f2f0 2024-04-25T10:13:42.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- 172.21.15.133:0/1944311579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850f2f0 msgr2=0x5603d84c4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 --2- 172.21.15.133:0/1944311579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850f2f0 0x5603d84c4b60 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f85dc001780 tx=0x7f85dc001c40 comp rx=0 tx=0).stop 2024-04-25T10:13:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- 172.21.15.133:0/1944311579 shutdown_connections 2024-04-25T10:13:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 --2- 172.21.15.133:0/1944311579 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d84c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 --2- 172.21.15.133:0/1944311579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850f2f0 0x5603d84c4b60 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 --2- 172.21.15.133:0/1944311579 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850e9d0 0x5603d850edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- 172.21.15.133:0/1944311579 >> 172.21.15.133:0/1944311579 conn(0x5603d84c1020 msgr2=0x5603d84c3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- 172.21.15.133:0/1944311579 shutdown_connections 2024-04-25T10:13:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- 172.21.15.133:0/1944311579 wait complete. 2024-04-25T10:13:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- start start 2024-04-25T10:13:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 Processor -- start 2024-04-25T10:13:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d84c50a0 0x5603d84f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d850e9d0 0x5603d84f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d84f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d82d6250 con 0x5603d84c50a0 2024-04-25T10:13:42.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d82cc690 con 0x5603d850f2f0 2024-04-25T10:13:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603d829aa40 con 0x5603d850e9d0 2024-04-25T10:13:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 10 client.0 _ll_register_callbacks cb 0x5603d84afe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d84c50a0 0x5603d84f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d84c50a0 0x5603d84f37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50594/0 (socket says 172.21.15.133:50594) 2024-04-25T10:13:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eb7fe640 1 -- 172.21.15.133:0/1640138422 learned_addr learned my addr 172.21.15.133:0/1640138422 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eaffd640 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d850e9d0 0x5603d84f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85ebfff640 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d84f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eb7fe640 1 -- 172.21.15.133:0/1640138422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d850e9d0 msgr2=0x5603d84f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d850e9d0 0x5603d84f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eb7fe640 1 -- 172.21.15.133:0/1640138422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 msgr2=0x5603d84f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d84f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eb7fe640 1 -- 172.21.15.133:0/1640138422 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603d83bf1c0 con 0x5603d84c50a0 2024-04-25T10:13:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eaffd640 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d850e9d0 0x5603d84f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d84c50a0 0x5603d84f37f0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f85d40146e0 tx=0x7f85d4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bdcf9ea81d7087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:13:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85ca7fc640 1 -- 172.21.15.133:0/1640138422 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d4025020 con 0x5603d84c50a0 2024-04-25T10:13:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85ca7fc640 1 -- 172.21.15.133:0/1640138422 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85d4015960 con 0x5603d84c50a0 2024-04-25T10:13:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- 172.21.15.133:0/1640138422 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85dc0047f0 con 0x5603d84c50a0 2024-04-25T10:13:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85ca7fc640 1 -- 172.21.15.133:0/1640138422 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85d402f050 con 0x5603d84c50a0 2024-04-25T10:13:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 10 client.9747 fetch_fsmap learned FSMap version 79 2024-04-25T10:13:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- 172.21.15.133:0/1640138422 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5603d83bb170 con 0x5603d84c50a0 2024-04-25T10:13:42.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85ca7fc640 1 -- 172.21.15.133:0/1640138422 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc0047f0 con 0x5603d84c50a0 2024-04-25T10:13:42.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85ca7fc640 1 -- 172.21.15.133:0/1640138422 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f85d4032020 con 0x5603d84c50a0 2024-04-25T10:13:42.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 10 client.9747 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:13:42.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 10 client.9747 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:13:42.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 20 client.9747 populate_metadata read hostname 'smithi133' 2024-04-25T10:13:42.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5603d851ec80 0x5603d8521070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 4 client.9747 mds_command: new command op to 9495 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:42.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85f5223000 1 -- 172.21.15.133:0/1640138422 --> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603d82adc60 con 0x5603d851ec80 2024-04-25T10:13:42.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eaffd640 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5603d851ec80 0x5603d8521070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85eaffd640 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5603d851ec80 0x5603d8521070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d107dc2771f140d2 server_cookie=fc646f735a1426ea in_seq=0 out_seq=0 2024-04-25T10:13:42.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85ca7fc640 10 client.9747 ms_handle_connect on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85ca7fc640 1 -- 172.21.15.133:0/1640138422 <== mds.0 v2:172.21.15.133:6836/2351700644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5603d82adc60 con 0x5603d851ec80 2024-04-25T10:13:42.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.480+0000 7f85ca7fc640 10 client.9747 handle_command_reply: tid=0 2024-04-25T10:13:42.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 1 client.9747 shutdown 2024-04-25T10:13:42.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 1 -- 172.21.15.133:0/1640138422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d84c50a0 msgr2=0x5603d84f37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d84c50a0 0x5603d84f37f0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f85d40146e0 tx=0x7f85d4004740 comp rx=0 tx=0).stop 2024-04-25T10:13:42.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 1 -- 172.21.15.133:0/1640138422 shutdown_connections 2024-04-25T10:13:42.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d84f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d850e9d0 0x5603d84f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5603d851ec80 0x5603d8521070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 1 --2- 172.21.15.133:0/1640138422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d84c50a0 0x5603d84f37f0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85ca7fc640 0 client.9747 ms_handle_reset on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 1 -- 172.21.15.133:0/1640138422 >> 172.21.15.133:0/1640138422 conn(0x5603d84c1020 msgr2=0x5603d84c3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 1 -- 172.21.15.133:0/1640138422 shutdown_connections 2024-04-25T10:13:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 1 -- 172.21.15.133:0/1640138422 wait complete. 2024-04-25T10:13:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.484+0000 7f85f5223000 20 client.9747 trim_cache size 0 max 16384 2024-04-25T10:13:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.496+0000 7f85f5223000 1 Processor -- start 2024-04-25T10:13:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.496+0000 7f85f5223000 1 -- start start 2024-04-25T10:13:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d856b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d856b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d856bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec200be0 con 0x5603d850e9d0 2024-04-25T10:13:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec0ca5a0 con 0x5603d850f2f0 2024-04-25T10:13:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1f88c0 con 0x5603d84c50a0 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d856b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d856b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50598/0 (socket says 172.21.15.133:50598) 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 -- 172.21.15.133:0/856112413 learned_addr learned my addr 172.21.15.133:0/856112413 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/856112413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d856b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85ebfff640 1 --2- 172.21.15.133:0/856112413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d856bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 -- 172.21.15.133:0/856112413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 msgr2=0x5603d856b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 --2- 172.21.15.133:0/856112413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d856b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 -- 172.21.15.133:0/856112413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 msgr2=0x5603d856bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 --2- 172.21.15.133:0/856112413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d856bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 -- 172.21.15.133:0/856112413 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603d8358de0 con 0x5603d850e9d0 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/856112413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d856b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 --2- 172.21.15.133:0/856112413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d856b630 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009010 tx=0x7f85dc00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f471d18ce72586a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85e0ff9640 1 -- 172.21.15.133:0/856112413 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc026d00 con 0x5603d850e9d0 2024-04-25T10:13:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85e0ff9640 1 -- 172.21.15.133:0/856112413 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc0047f0 con 0x5603d850e9d0 2024-04-25T10:13:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85e0ff9640 1 -- 172.21.15.133:0/856112413 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc004be0 con 0x5603d850e9d0 2024-04-25T10:13:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- 172.21.15.133:0/856112413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 msgr2=0x5603d856b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 --2- 172.21.15.133:0/856112413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d856b630 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009010 tx=0x7f85dc00a000 comp rx=0 tx=0).stop 2024-04-25T10:13:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- 172.21.15.133:0/856112413 shutdown_connections 2024-04-25T10:13:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 --2- 172.21.15.133:0/856112413 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d856bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 --2- 172.21.15.133:0/856112413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d856b630 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 --2- 172.21.15.133:0/856112413 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d856b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- 172.21.15.133:0/856112413 >> 172.21.15.133:0/856112413 conn(0x5603d84c1020 msgr2=0x5603d851bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- 172.21.15.133:0/856112413 shutdown_connections 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- 172.21.15.133:0/856112413 wait complete. 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- start start 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 Processor -- start 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d8521380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d85218c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d8521e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603d82ace00 con 0x5603d850e9d0 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec0c9fa0 con 0x5603d850f2f0 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603d8311300 con 0x5603d84c50a0 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 10 client.0 _ll_register_callbacks cb 0x5603d8490040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d85218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d85218c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50604/0 (socket says 172.21.15.133:50604) 2024-04-25T10:13:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 -- 172.21.15.133:0/175144134 learned_addr learned my addr 172.21.15.133:0/175144134 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d8521380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85ebfff640 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d8521e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 -- 172.21.15.133:0/175144134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 msgr2=0x5603d8521380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d8521380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 -- 172.21.15.133:0/175144134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 msgr2=0x5603d8521e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d8521e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 -- 172.21.15.133:0/175144134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603d82b38e0 con 0x5603d850e9d0 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85eaffd640 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d85218c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009010 tx=0x7f85dc000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a577871bfeb3191 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85ca7fc640 1 -- 172.21.15.133:0/175144134 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc02a040 con 0x5603d850e9d0 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85ca7fc640 1 -- 172.21.15.133:0/175144134 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85dc026d00 con 0x5603d850e9d0 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- 172.21.15.133:0/175144134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85dc0047f0 con 0x5603d850e9d0 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85ca7fc640 1 -- 172.21.15.133:0/175144134 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85dc03b050 con 0x5603d850e9d0 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 10 client.9753 fetch_fsmap learned FSMap version 79 2024-04-25T10:13:42.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.500+0000 7f85f5223000 1 -- 172.21.15.133:0/175144134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x5603d836b8f0 con 0x5603d850e9d0 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85ca7fc640 1 -- 172.21.15.133:0/175144134 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc0047f0 con 0x5603d850e9d0 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85ca7fc640 1 -- 172.21.15.133:0/175144134 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 79) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f85dc03e020 con 0x5603d850e9d0 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 10 client.9753 fetch_fsmap finished waiting for FSMap version 79 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 10 client.9753 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 20 client.9753 populate_metadata read hostname 'smithi133' 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5603d85227f0 0x5603d8576f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 4 client.9753 mds_command: new command op to 9495 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 -- 172.21.15.133:0/175144134 --> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5603d831f530 con 0x5603d85227f0 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5603d85227f0 0x5603d8576f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85ca7fc640 10 client.9753 ms_handle_connect on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85eb7fe640 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5603d85227f0 0x5603d8576f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8317125a3311ae06 server_cookie=37507df42db94237 in_seq=0 out_seq=0 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85ca7fc640 1 -- 172.21.15.133:0/175144134 <== mds.0 v2:172.21.15.133:6836/2351700644 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x5603d831f530 con 0x5603d85227f0 2024-04-25T10:13:42.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85ca7fc640 10 client.9753 handle_command_reply: tid=0 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 client.9753 shutdown 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 -- 172.21.15.133:0/175144134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 msgr2=0x5603d85218c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d85218c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009010 tx=0x7f85dc000ee0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 -- 172.21.15.133:0/175144134 shutdown_connections 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5603d850f2f0 0x5603d8521e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.133:6836/2351700644,v1:172.21.15.133:6837/2351700644] conn(0x5603d85227f0 0x5603d8576f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5603d850e9d0 0x5603d85218c0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 --2- 172.21.15.133:0/175144134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5603d84c50a0 0x5603d8521380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85ca7fc640 0 client.9753 ms_handle_reset on v2:172.21.15.133:6836/2351700644 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 -- 172.21.15.133:0/175144134 >> 172.21.15.133:0/175144134 conn(0x5603d84c1020 msgr2=0x5603d850d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 -- 172.21.15.133:0/175144134 shutdown_connections 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 1 -- 172.21.15.133:0/175144134 wait complete. 2024-04-25T10:13:42.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f5223000 20 client.9753 trim_cache size 0 max 16384 2024-04-25T10:13:42.519 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:13:42.519 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:13:42.519 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-04-25T10:13:42.519 INFO:teuthology.orchestra.run.smithi133.stdout: "1b70c6d4-1b74-439e-b12a-15eb7bc46d90": { 2024-04-25T10:13:42.519 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:13:42.519 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "1b70c6d4-1b74-439e-b12a-15eb7bc46d90", 2024-04-25T10:13:42.519 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive" 2024-04-25T10:13:42.519 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:42.519 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:13:42.519 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:13:42.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 -- 172.21.15.133:0/1389089677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85cc066f60 msgr2=0x7f85cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85cc066f60 0x7f85cc069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f85d400b9d0 tx=0x7f85d4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:42.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 -- 172.21.15.133:0/1389089677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec15f030 msgr2=0x7f85ec156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec15f030 0x7f85ec156060 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f85e40af040 tx=0x7f85e4019710 comp rx=0 tx=0).stop 2024-04-25T10:13:42.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 -- 172.21.15.133:0/1389089677 shutdown_connections 2024-04-25T10:13:42.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f85cc066f60 0x7f85cc069420 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec15f030 0x7f85ec156060 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ec0b2550 0x7f85ec0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 --2- 172.21.15.133:0/1389089677 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85ec09a0e0 0x7f85ec15e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 -- 172.21.15.133:0/1389089677 >> 172.21.15.133:0/1389089677 conn(0x7f85ec0b31a0 msgr2=0x7f85ec06f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 -- 172.21.15.133:0/1389089677 shutdown_connections 2024-04-25T10:13:42.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.504+0000 7f85f20b6640 1 -- 172.21.15.133:0/1389089677 wait complete. 2024-04-25T10:13:42.522 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:42.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b7a60a640 1 Processor -- start 2024-04-25T10:13:42.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b7a60a640 1 -- start start 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b7a60a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4b7415b990 0x7f4b740d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b7a60a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b740b1890 0x7f4b740b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b7a60a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4b7415f7c0 0x7f4b7415fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b7a60a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b740b2230 con 0x7f4b7415f7c0 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b7a60a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b740cf490 con 0x7f4b7415b990 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b7a60a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740cf770 con 0x7f4b740b1890 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b78ba7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4b7415f7c0 0x7f4b7415fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b78ba7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4b7415f7c0 0x7f4b7415fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50618/0 (socket says 172.21.15.133:50618) 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b78ba7640 1 -- 172.21.15.133:0/1510508343 learned_addr learned my addr 172.21.15.133:0/1510508343 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b73fff640 1 --2- 172.21.15.133:0/1510508343 >> v2:172.21.15.162:3300/0 conn(0x7f4b7415b990 0x7f4b740d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b737fe640 1 --2- 172.21.15.133:0/1510508343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b740b1890 0x7f4b740b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b78ba7640 1 -- 172.21.15.133:0/1510508343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b740b1890 msgr2=0x7f4b740b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b78ba7640 1 --2- 172.21.15.133:0/1510508343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b740b1890 0x7f4b740b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b78ba7640 1 -- 172.21.15.133:0/1510508343 >> v2:172.21.15.162:3300/0 conn(0x7f4b7415b990 msgr2=0x7f4b740d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b78ba7640 1 --2- 172.21.15.133:0/1510508343 >> v2:172.21.15.162:3300/0 conn(0x7f4b7415b990 0x7f4b740d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b78ba7640 1 -- 172.21.15.133:0/1510508343 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b740cfa50 con 0x7f4b7415f7c0 2024-04-25T10:13:42.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.632+0000 7f4b73fff640 1 --2- 172.21.15.133:0/1510508343 >> v2:172.21.15.162:3300/0 conn(0x7f4b7415b990 0x7f4b740d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:42.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b78ba7640 1 --2- 172.21.15.133:0/1510508343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b7415f7c0 0x7f4b7415fc50 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009e50 tx=0x7f4b6401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c0ebf94087ff63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b72ffd640 1 -- 172.21.15.133:0/1510508343 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640a5690 con 0x7f4b7415f7c0 2024-04-25T10:13:42.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b72ffd640 1 -- 172.21.15.133:0/1510508343 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b640a5830 con 0x7f4b7415f7c0 2024-04-25T10:13:42.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b72ffd640 1 -- 172.21.15.133:0/1510508343 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640a5c00 con 0x7f4b7415f7c0 2024-04-25T10:13:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 -- 172.21.15.133:0/1510508343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b7415f7c0 msgr2=0x7f4b7415fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 --2- 172.21.15.133:0/1510508343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b7415f7c0 0x7f4b7415fc50 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009e50 tx=0x7f4b6401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 -- 172.21.15.133:0/1510508343 shutdown_connections 2024-04-25T10:13:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 --2- 172.21.15.133:0/1510508343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b7415f7c0 0x7f4b7415fc50 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 --2- 172.21.15.133:0/1510508343 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b740b1890 0x7f4b740b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 --2- 172.21.15.133:0/1510508343 >> v2:172.21.15.162:3300/0 conn(0x7f4b7415b990 0x7f4b740d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 -- 172.21.15.133:0/1510508343 >> 172.21.15.133:0/1510508343 conn(0x7f4b7409aa10 msgr2=0x7f4b7408af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 -- 172.21.15.133:0/1510508343 shutdown_connections 2024-04-25T10:13:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 -- 172.21.15.133:0/1510508343 wait complete. 2024-04-25T10:13:42.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 Processor -- start 2024-04-25T10:13:42.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 -- start start 2024-04-25T10:13:42.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b740b1890 0x7f4b740b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b7415b990 0x7f4b7414b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b7415f7c0 0x7f4b7406f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740ccbc0 con 0x7f4b7415b990 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740cca40 con 0x7f4b740b1890 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740ccd40 con 0x7f4b7415f7c0 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b737fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b7415b990 0x7f4b7414b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b737fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b7415b990 0x7f4b7414b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50622/0 (socket says 172.21.15.133:50622) 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b737fe640 1 -- 172.21.15.133:0/2140256886 learned_addr learned my addr 172.21.15.133:0/2140256886 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b78ba7640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b7415f7c0 0x7f4b7406f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b73fff640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b740b1890 0x7f4b740b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b737fe640 1 -- 172.21.15.133:0/2140256886 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b7415f7c0 msgr2=0x7f4b7406f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b737fe640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b7415f7c0 0x7f4b7406f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b737fe640 1 -- 172.21.15.133:0/2140256886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b740b1890 msgr2=0x7f4b740b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b737fe640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b740b1890 0x7f4b740b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b737fe640 1 -- 172.21.15.133:0/2140256886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7414bc10 con 0x7f4b7415b990 2024-04-25T10:13:42.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b73fff640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b740b1890 0x7f4b740b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:42.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b737fe640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b7415b990 0x7f4b7414b290 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00d830 tx=0x7f4b5c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b70ff9640 1 -- 172.21.15.133:0/2140256886 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c018020 con 0x7f4b7415b990 2024-04-25T10:13:42.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b70ff9640 1 -- 172.21.15.133:0/2140256886 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c0096a0 con 0x7f4b7415b990 2024-04-25T10:13:42.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 -- 172.21.15.133:0/2140256886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b740c8890 con 0x7f4b7415b990 2024-04-25T10:13:42.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b70ff9640 1 -- 172.21.15.133:0/2140256886 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c009a50 con 0x7f4b7415b990 2024-04-25T10:13:42.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.636+0000 7f4b7a60a640 1 -- 172.21.15.133:0/2140256886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7414beb0 con 0x7f4b7415b990 2024-04-25T10:13:42.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.640+0000 7f4b7a60a640 1 -- 172.21.15.133:0/2140256886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b74068ef0 con 0x7f4b7415b990 2024-04-25T10:13:42.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.640+0000 7f4b70ff9640 1 -- 172.21.15.133:0/2140256886 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b5c01b020 con 0x7f4b7415b990 2024-04-25T10:13:42.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.640+0000 7f4b70ff9640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 0x7f4b54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.640+0000 7f4b70ff9640 1 -- 172.21.15.133:0/2140256886 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b5c085880 con 0x7f4b7415b990 2024-04-25T10:13:42.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.640+0000 7f4b73fff640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 0x7f4b54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.640+0000 7f4b73fff640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 0x7f4b54069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009980 tx=0x7f4b6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.644+0000 7f4b70ff9640 1 -- 172.21.15.133:0/2140256886 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b5c022020 con 0x7f4b7415b990 2024-04-25T10:13:42.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.828+0000 7f4b7a60a640 1 -- 172.21.15.133:0/2140256886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b740b1360 con 0x7f4b7415b990 2024-04-25T10:13:42.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b70ff9640 1 -- 172.21.15.133:0/2140256886 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f4b5c011340 con 0x7f4b7415b990 2024-04-25T10:13:42.833 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:42.834 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:13:42.834 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:13:42.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 -- 172.21.15.133:0/2140256886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 msgr2=0x7f4b54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 0x7f4b54069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009980 tx=0x7f4b6c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:42.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 -- 172.21.15.133:0/2140256886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b7415b990 msgr2=0x7f4b7414b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b7415b990 0x7f4b7414b290 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00d830 tx=0x7f4b5c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 -- 172.21.15.133:0/2140256886 shutdown_connections 2024-04-25T10:13:42.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4b7415f7c0 0x7f4b7406f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4b54066f60 0x7f4b54069420 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b7415b990 0x7f4b7414b290 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 --2- 172.21.15.133:0/2140256886 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4b740b1890 0x7f4b740b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 -- 172.21.15.133:0/2140256886 >> 172.21.15.133:0/2140256886 conn(0x7f4b7409aa10 msgr2=0x7f4b74070710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 -- 172.21.15.133:0/2140256886 shutdown_connections 2024-04-25T10:13:42.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.832+0000 7f4b7a60a640 1 -- 172.21.15.133:0/2140256886 wait complete. 2024-04-25T10:13:42.851 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:13:42.851 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:42.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f7d096640 1 Processor -- start 2024-04-25T10:13:42.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f7d096640 1 -- start start 2024-04-25T10:13:42.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f7d096640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f780ce330 0x7f2f780ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f7d096640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f780c61d0 con 0x7f2f7815c9a0 2024-04-25T10:13:42.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f7d096640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f780c64b0 con 0x7f2f780b1f70 2024-04-25T10:13:42.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f7d096640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f780cecc0 con 0x7f2f780ce330 2024-04-25T10:13:42.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f7759e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f780ce330 0x7f2f780ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f76d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2f7815c9a0 0x7f2f780c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54646/0 (socket says 172.21.15.133:54646) 2024-04-25T10:13:42.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f76d9d640 1 -- 172.21.15.133:0/597790750 learned_addr learned my addr 172.21.15.133:0/597790750 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:42.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f7759e640 1 -- 172.21.15.133:0/597790750 >> v1:172.21.15.162:6789/0 conn(0x7f2f780b1f70 legacy=0x7f2f7806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f7759e640 1 -- 172.21.15.133:0/597790750 >> v1:172.21.15.133:6789/0 conn(0x7f2f7815c9a0 legacy=0x7f2f780c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f7759e640 1 -- 172.21.15.133:0/597790750 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7806d4f0 con 0x7f2f780ce330 2024-04-25T10:13:42.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.960+0000 7f2f75d9b640 1 -- 172.21.15.133:0/597790750 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927730886 0 0) 0x7f2f780c61d0 con 0x7f2f7815c9a0 2024-04-25T10:13:42.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f7759e640 1 --2- 172.21.15.133:0/597790750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f780ce330 0x7f2f780ce780 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009d90 tx=0x7f2f7001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68ce8ac9fcef3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f75d9b640 1 -- 172.21.15.133:0/597790750 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700b2070 con 0x7f2f780ce330 2024-04-25T10:13:42.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f75d9b640 1 -- 172.21.15.133:0/597790750 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f700a5900 con 0x7f2f780ce330 2024-04-25T10:13:42.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f75d9b640 1 -- 172.21.15.133:0/597790750 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700a5cd0 con 0x7f2f780ce330 2024-04-25T10:13:42.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f7d096640 1 -- 172.21.15.133:0/597790750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f780ce330 msgr2=0x7f2f780ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f7d096640 1 --2- 172.21.15.133:0/597790750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f780ce330 0x7f2f780ce780 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009d90 tx=0x7f2f7001c930 comp rx=0 tx=0).stop 2024-04-25T10:13:42.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f7d096640 1 -- 172.21.15.133:0/597790750 shutdown_connections 2024-04-25T10:13:42.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f7d096640 1 --2- 172.21.15.133:0/597790750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f780ce330 0x7f2f780ce780 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f7d096640 1 -- 172.21.15.133:0/597790750 >> 172.21.15.133:0/597790750 conn(0x7f2f78067000 msgr2=0x7f2f7806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f7d096640 1 -- 172.21.15.133:0/597790750 shutdown_connections 2024-04-25T10:13:42.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f7d096640 1 -- 172.21.15.133:0/597790750 wait complete. 2024-04-25T10:13:42.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f7d096640 1 Processor -- start 2024-04-25T10:13:42.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.964+0000 7f2f7d096640 1 -- start start 2024-04-25T10:13:42.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7d096640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f780b1f70 0x7f2f781488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7d096640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f780ce330 0x7f2f7814b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7d096640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7815c9a0 0x7f2f7814e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7d096640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f781695d0 con 0x7f2f7815c9a0 2024-04-25T10:13:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7d096640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7815dc20 con 0x7f2f780b1f70 2024-04-25T10:13:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7d096640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f78169750 con 0x7f2f780ce330 2024-04-25T10:13:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7815c9a0 0x7f2f7814e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7815c9a0 0x7f2f7814e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50638/0 (socket says 172.21.15.133:50638) 2024-04-25T10:13:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7759e640 1 -- 172.21.15.133:0/679100030 learned_addr learned my addr 172.21.15.133:0/679100030 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7759e640 1 -- 172.21.15.133:0/679100030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f780ce330 msgr2=0x7f2f7814b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7659c640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f780ce330 0x7f2f7814b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7759e640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f780ce330 0x7f2f7814b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f76d9d640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f780b1f70 0x7f2f781488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7759e640 1 -- 172.21.15.133:0/679100030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f780b1f70 msgr2=0x7f2f781488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7759e640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f780b1f70 0x7f2f781488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7759e640 1 -- 172.21.15.133:0/679100030 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f780c7e80 con 0x7f2f7815c9a0 2024-04-25T10:13:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7659c640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f780ce330 0x7f2f7814b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7759e640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7815c9a0 0x7f2f7814e0e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f2f700b0040 tx=0x7f2f700196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f5f7fe640 1 -- 172.21.15.133:0/679100030 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700b2050 con 0x7f2f7815c9a0 2024-04-25T10:13:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f5f7fe640 1 -- 172.21.15.133:0/679100030 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f700b3040 con 0x7f2f7815c9a0 2024-04-25T10:13:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7d096640 1 -- 172.21.15.133:0/679100030 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f780c8160 con 0x7f2f7815c9a0 2024-04-25T10:13:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f5f7fe640 1 -- 172.21.15.133:0/679100030 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f700ad930 con 0x7f2f7815c9a0 2024-04-25T10:13:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7d096640 1 -- 172.21.15.133:0/679100030 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f780c8720 con 0x7f2f7815c9a0 2024-04-25T10:13:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f7d096640 1 -- 172.21.15.133:0/679100030 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f78068ef0 con 0x7f2f7815c9a0 2024-04-25T10:13:42.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f5f7fe640 1 -- 172.21.15.133:0/679100030 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f700c0020 con 0x7f2f7815c9a0 2024-04-25T10:13:42.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f5f7fe640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 0x7f2f64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f5f7fe640 1 -- 172.21.15.133:0/679100030 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f701203c0 con 0x7f2f7815c9a0 2024-04-25T10:13:42.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.968+0000 7f2f76d9d640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 0x7f2f64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.972+0000 7f2f76d9d640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 0x7f2f64069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2f68000f70 tx=0x7f2f68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:42.972+0000 7f2f5f7fe640 1 -- 172.21.15.133:0/679100030 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f700c7020 con 0x7f2f7815c9a0 2024-04-25T10:13:43.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.196+0000 7f2f7d096640 1 -- 172.21.15.133:0/679100030 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2f780cbb50 con 0x7f2f7815c9a0 2024-04-25T10:13:43.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.196+0000 7f2f5f7fe640 1 -- 172.21.15.133:0/679100030 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f2f70016520 con 0x7f2f7815c9a0 2024-04-25T10:13:43.200 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:43.200 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":736,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":923,"data_bytes":945346677,"bytes_used":2360557568,"bytes_avail":770733555712,"bytes_total":773094113280,"write_bytes_sec":51710221,"read_op_per_sec":0,"write_op_per_sec":52},"fsmap":{"epoch":79,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9495}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:12:51.919939+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:13:43.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 -- 172.21.15.133:0/679100030 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 msgr2=0x7f2f64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 0x7f2f64069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2f68000f70 tx=0x7f2f68009040 comp rx=0 tx=0).stop 2024-04-25T10:13:43.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 -- 172.21.15.133:0/679100030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7815c9a0 msgr2=0x7f2f7814e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7815c9a0 0x7f2f7814e0e0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f2f700b0040 tx=0x7f2f700196d0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 -- 172.21.15.133:0/679100030 shutdown_connections 2024-04-25T10:13:43.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f64066f60 0x7f2f64069420 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7815c9a0 0x7f2f7814e0e0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f780ce330 0x7f2f7814b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 --2- 172.21.15.133:0/679100030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f780b1f70 0x7f2f781488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 -- 172.21.15.133:0/679100030 >> 172.21.15.133:0/679100030 conn(0x7f2f78067000 msgr2=0x7f2f7806dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 -- 172.21.15.133:0/679100030 shutdown_connections 2024-04-25T10:13:43.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:43.200+0000 7f2f7d096640 1 -- 172.21.15.133:0/679100030 wait complete. 2024-04-25T10:13:43.219 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:13:44.220 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:44.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.348+0000 7f654e2c5640 1 Processor -- start 2024-04-25T10:13:44.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.348+0000 7f654e2c5640 1 -- start start 2024-04-25T10:13:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654e2c5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f654815db00 0x7f65480b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654e2c5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65480d0bd0 0x7f6548161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654e2c5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65480ceb20 con 0x7f65480d0770 2024-04-25T10:13:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654e2c5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65480b7740 con 0x7f65480d0bd0 2024-04-25T10:13:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654e2c5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65480b7a20 con 0x7f654815db00 2024-04-25T10:13:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654c862640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65480d0bd0 0x7f6548161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654c862640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65480d0bd0 0x7f6548161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50640/0 (socket says 172.21.15.133:50640) 2024-04-25T10:13:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654c862640 1 -- 172.21.15.133:0/1226031199 learned_addr learned my addr 172.21.15.133:0/1226031199 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f6547fff640 1 --2- 172.21.15.133:0/1226031199 >> v2:172.21.15.162:3300/0 conn(0x7f654815db00 0x7f65480b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654c862640 1 -- 172.21.15.133:0/1226031199 >> v2:172.21.15.162:3300/0 conn(0x7f654815db00 msgr2=0x7f65480b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654c862640 1 --2- 172.21.15.133:0/1226031199 >> v2:172.21.15.162:3300/0 conn(0x7f654815db00 0x7f65480b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654c862640 1 -- 172.21.15.133:0/1226031199 >> v1:172.21.15.162:6789/0 conn(0x7f65480d0770 legacy=0x7f65481614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654c862640 1 -- 172.21.15.133:0/1226031199 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65480b97f0 con 0x7f65480d0bd0 2024-04-25T10:13:44.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f654c862640 1 --2- 172.21.15.133:0/1226031199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65480d0bd0 0x7f6548161bd0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f65480b53d0 tx=0x7f654001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aaf7324bea3f731 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f65477fe640 1 -- 172.21.15.133:0/1226031199 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65400a5880 con 0x7f65480d0bd0 2024-04-25T10:13:44.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f65477fe640 1 -- 172.21.15.133:0/1226031199 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65400a5a20 con 0x7f65480d0bd0 2024-04-25T10:13:44.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.352+0000 7f65477fe640 1 -- 172.21.15.133:0/1226031199 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65400a5df0 con 0x7f65480d0bd0 2024-04-25T10:13:44.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 -- 172.21.15.133:0/1226031199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65480d0bd0 msgr2=0x7f6548161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 --2- 172.21.15.133:0/1226031199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65480d0bd0 0x7f6548161bd0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f65480b53d0 tx=0x7f654001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 -- 172.21.15.133:0/1226031199 shutdown_connections 2024-04-25T10:13:44.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 --2- 172.21.15.133:0/1226031199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65480d0bd0 0x7f6548161bd0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 --2- 172.21.15.133:0/1226031199 >> v2:172.21.15.162:3300/0 conn(0x7f654815db00 0x7f65480b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 -- 172.21.15.133:0/1226031199 >> 172.21.15.133:0/1226031199 conn(0x7f65480727c0 msgr2=0x7f6548072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 -- 172.21.15.133:0/1226031199 shutdown_connections 2024-04-25T10:13:44.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 -- 172.21.15.133:0/1226031199 wait complete. 2024-04-25T10:13:44.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 Processor -- start 2024-04-25T10:13:44.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 -- start start 2024-04-25T10:13:44.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65480d0770 0x7f65480c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65480d0bd0 0x7f654814c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f654815db00 0x7f6548156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65480b7430 con 0x7f65480d0bd0 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65480d1060 con 0x7f65480d0770 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65480d18a0 con 0x7f654815db00 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f653f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65480d0bd0 0x7f654814c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f653f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65480d0bd0 0x7f654814c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50656/0 (socket says 172.21.15.133:50656) 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f653f7fe640 1 -- 172.21.15.133:0/1257361726 learned_addr learned my addr 172.21.15.133:0/1257361726 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654c862640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f654815db00 0x7f6548156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f6547fff640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65480d0770 0x7f65480c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f653f7fe640 1 -- 172.21.15.133:0/1257361726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f654815db00 msgr2=0x7f6548156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f653f7fe640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f654815db00 0x7f6548156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f653f7fe640 1 -- 172.21.15.133:0/1257361726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65480d0770 msgr2=0x7f65480c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f653f7fe640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65480d0770 0x7f65480c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f653f7fe640 1 -- 172.21.15.133:0/1257361726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6548156fd0 con 0x7f65480d0bd0 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f6547fff640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65480d0770 0x7f65480c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:44.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f653f7fe640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65480d0bd0 0x7f654814c380 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f653000c370 tx=0x7f653000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f65457fa640 1 -- 172.21.15.133:0/1257361726 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65300052d0 con 0x7f65480d0bd0 2024-04-25T10:13:44.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f65457fa640 1 -- 172.21.15.133:0/1257361726 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6530005470 con 0x7f65480d0bd0 2024-04-25T10:13:44.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f65457fa640 1 -- 172.21.15.133:0/1257361726 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530007260 con 0x7f65480d0bd0 2024-04-25T10:13:44.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 -- 172.21.15.133:0/1257361726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654814ef40 con 0x7f65480d0bd0 2024-04-25T10:13:44.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.356+0000 7f654e2c5640 1 -- 172.21.15.133:0/1257361726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654814f500 con 0x7f65480d0bd0 2024-04-25T10:13:44.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.360+0000 7f654e2c5640 1 -- 172.21.15.133:0/1257361726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6548068ef0 con 0x7f65480d0bd0 2024-04-25T10:13:44.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.360+0000 7f65457fa640 1 -- 172.21.15.133:0/1257361726 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6530007400 con 0x7f65480d0bd0 2024-04-25T10:13:44.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.360+0000 7f65457fa640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6528066f60 0x7f6528069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.360+0000 7f65457fa640 1 -- 172.21.15.133:0/1257361726 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6530084560 con 0x7f65480d0bd0 2024-04-25T10:13:44.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.360+0000 7f6547fff640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6528066f60 0x7f6528069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.360+0000 7f6547fff640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6528066f60 0x7f6528069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f65481559b0 tx=0x7f6538008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.364+0000 7f65457fa640 1 -- 172.21.15.133:0/1257361726 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6530054d90 con 0x7f65480d0bd0 2024-04-25T10:13:44.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 -- 172.21.15.133:0/1257361726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65480b2750 con 0x7f65480d0bd0 2024-04-25T10:13:44.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f65457fa640 1 -- 172.21.15.133:0/1257361726 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f6530058e30 con 0x7f65480d0bd0 2024-04-25T10:13:44.553 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:44.553 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:13:44.553 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:13:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 -- 172.21.15.133:0/1257361726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6528066f60 msgr2=0x7f6528069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6528066f60 0x7f6528069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f65481559b0 tx=0x7f6538008040 comp rx=0 tx=0).stop 2024-04-25T10:13:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 -- 172.21.15.133:0/1257361726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65480d0bd0 msgr2=0x7f654814c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65480d0bd0 0x7f654814c380 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f653000c370 tx=0x7f653000c830 comp rx=0 tx=0).stop 2024-04-25T10:13:44.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 -- 172.21.15.133:0/1257361726 shutdown_connections 2024-04-25T10:13:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f654815db00 0x7f6548156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6528066f60 0x7f6528069420 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65480d0bd0 0x7f654814c380 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 --2- 172.21.15.133:0/1257361726 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65480d0770 0x7f65480c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 -- 172.21.15.133:0/1257361726 >> 172.21.15.133:0/1257361726 conn(0x7f65480727c0 msgr2=0x7f65480d3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 -- 172.21.15.133:0/1257361726 shutdown_connections 2024-04-25T10:13:44.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.544+0000 7f654e2c5640 1 -- 172.21.15.133:0/1257361726 wait complete. 2024-04-25T10:13:44.565 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:13:44.565 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:44.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.668+0000 7f8265486640 1 Processor -- start 2024-04-25T10:13:44.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.668+0000 7f8265486640 1 -- start start 2024-04-25T10:13:44.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.668+0000 7f8265486640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f826015c9a0 0x7f82600c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.668+0000 7f8265486640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82600b1f70 0x7f826006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.668+0000 7f8265486640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82600c61d0 con 0x7f82600ce330 2024-04-25T10:13:44.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.668+0000 7f8265486640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82600c64b0 con 0x7f826015c9a0 2024-04-25T10:13:44.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.668+0000 7f8265486640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826006d4f0 con 0x7f82600b1f70 2024-04-25T10:13:44.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f826015c9a0 0x7f82600c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f826015c9a0 0x7f82600c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50666/0 (socket says 172.21.15.133:50666) 2024-04-25T10:13:44.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825effd640 1 -- 172.21.15.133:0/3364269336 learned_addr learned my addr 172.21.15.133:0/3364269336 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:44.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825e7fc640 1 --2- 172.21.15.133:0/3364269336 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82600b1f70 0x7f826006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825effd640 1 -- 172.21.15.133:0/3364269336 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82600b1f70 msgr2=0x7f826006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825effd640 1 --2- 172.21.15.133:0/3364269336 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82600b1f70 0x7f826006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825effd640 1 -- 172.21.15.133:0/3364269336 >> v1:172.21.15.162:6789/0 conn(0x7f82600ce330 legacy=0x7f826015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825effd640 1 -- 172.21.15.133:0/3364269336 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826006d7d0 con 0x7f826015c9a0 2024-04-25T10:13:44.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825effd640 1 --2- 172.21.15.133:0/3364269336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f826015c9a0 0x7f82600c5980 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f8250009870 tx=0x7f825001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd1dba72e10f594 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825dffb640 1 -- 172.21.15.133:0/3364269336 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500b2070 con 0x7f826015c9a0 2024-04-25T10:13:44.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825dffb640 1 -- 172.21.15.133:0/3364269336 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82500ad640 con 0x7f826015c9a0 2024-04-25T10:13:44.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f825dffb640 1 -- 172.21.15.133:0/3364269336 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500ada10 con 0x7f826015c9a0 2024-04-25T10:13:44.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f8265486640 1 -- 172.21.15.133:0/3364269336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f826015c9a0 msgr2=0x7f82600c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f8265486640 1 --2- 172.21.15.133:0/3364269336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f826015c9a0 0x7f82600c5980 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f8250009870 tx=0x7f825001c4c0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f8265486640 1 -- 172.21.15.133:0/3364269336 shutdown_connections 2024-04-25T10:13:44.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f8265486640 1 --2- 172.21.15.133:0/3364269336 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82600b1f70 0x7f826006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f8265486640 1 --2- 172.21.15.133:0/3364269336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f826015c9a0 0x7f82600c5980 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f8265486640 1 -- 172.21.15.133:0/3364269336 >> 172.21.15.133:0/3364269336 conn(0x7f8260067000 msgr2=0x7f826006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f8265486640 1 -- 172.21.15.133:0/3364269336 shutdown_connections 2024-04-25T10:13:44.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.672+0000 7f8265486640 1 -- 172.21.15.133:0/3364269336 wait complete. 2024-04-25T10:13:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f8265486640 1 Processor -- start 2024-04-25T10:13:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f8265486640 1 -- start start 2024-04-25T10:13:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f8265486640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82600b1f70 0x7f826013ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f8265486640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82600ce330 0x7f8260157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f8265486640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f826015c9a0 0x7f8260152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f8265486640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826015cd80 con 0x7f82600ce330 2024-04-25T10:13:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f8265486640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826006e230 con 0x7f82600b1f70 2024-04-25T10:13:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f8265486640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826015cf00 con 0x7f826015c9a0 2024-04-25T10:13:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f825e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82600ce330 0x7f8260157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f825e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82600ce330 0x7f8260157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50670/0 (socket says 172.21.15.133:50670) 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f825e7fc640 1 -- 172.21.15.133:0/1051118830 learned_addr learned my addr 172.21.15.133:0/1051118830 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f825effd640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82600b1f70 0x7f826013ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f825f7fe640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f826015c9a0 0x7f8260152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f825e7fc640 1 -- 172.21.15.133:0/1051118830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f826015c9a0 msgr2=0x7f8260152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f825e7fc640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f826015c9a0 0x7f8260152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f825e7fc640 1 -- 172.21.15.133:0/1051118830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82600b1f70 msgr2=0x7f826013ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f825e7fc640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82600b1f70 0x7f826013ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f825e7fc640 1 -- 172.21.15.133:0/1051118830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8260157d80 con 0x7f82600ce330 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f825e7fc640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82600ce330 0x7f8260157680 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f824800d620 tx=0x7f824800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f82577fe640 1 -- 172.21.15.133:0/1051118830 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824800b620 con 0x7f82600ce330 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f82577fe640 1 -- 172.21.15.133:0/1051118830 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8248019070 con 0x7f82600ce330 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f8265486640 1 -- 172.21.15.133:0/1051118830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8260158060 con 0x7f82600ce330 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f82577fe640 1 -- 172.21.15.133:0/1051118830 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824800b9e0 con 0x7f82600ce330 2024-04-25T10:13:44.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f8265486640 1 -- 172.21.15.133:0/1051118830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8260158620 con 0x7f82600ce330 2024-04-25T10:13:44.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f8265486640 1 -- 172.21.15.133:0/1051118830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826006bc20 con 0x7f82600ce330 2024-04-25T10:13:44.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.676+0000 7f82577fe640 1 -- 172.21.15.133:0/1051118830 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f824800bb80 con 0x7f82600ce330 2024-04-25T10:13:44.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.680+0000 7f82577fe640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8240066f60 0x7f8240069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.680+0000 7f82577fe640 1 -- 172.21.15.133:0/1051118830 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8248085680 con 0x7f82600ce330 2024-04-25T10:13:44.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.680+0000 7f825effd640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8240066f60 0x7f8240069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.680+0000 7f825effd640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8240066f60 0x7f8240069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f82500af000 tx=0x7f8250008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.684+0000 7f82577fe640 1 -- 172.21.15.133:0/1051118830 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8248056450 con 0x7f82600ce330 2024-04-25T10:13:44.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.896+0000 7f8265486640 1 -- 172.21.15.133:0/1051118830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8260068ef0 con 0x7f82600ce330 2024-04-25T10:13:44.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.896+0000 7f82577fe640 1 -- 172.21.15.133:0/1051118830 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f8248083e20 con 0x7f82600ce330 2024-04-25T10:13:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:44.901 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":737,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":923,"data_bytes":945346677,"bytes_used":2360557568,"bytes_avail":770733555712,"bytes_total":773094113280,"write_bytes_sec":51710462,"read_op_per_sec":0,"write_op_per_sec":52},"fsmap":{"epoch":79,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9495}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:12:51.919939+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:13:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 -- 172.21.15.133:0/1051118830 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8240066f60 msgr2=0x7f8240069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8240066f60 0x7f8240069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f82500af000 tx=0x7f8250008040 comp rx=0 tx=0).stop 2024-04-25T10:13:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 -- 172.21.15.133:0/1051118830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82600ce330 msgr2=0x7f8260157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82600ce330 0x7f8260157680 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f824800d620 tx=0x7f824800dae0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 -- 172.21.15.133:0/1051118830 shutdown_connections 2024-04-25T10:13:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f826015c9a0 0x7f8260152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8240066f60 0x7f8240069420 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82600ce330 0x7f8260157680 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 --2- 172.21.15.133:0/1051118830 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82600b1f70 0x7f826013ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 -- 172.21.15.133:0/1051118830 >> 172.21.15.133:0/1051118830 conn(0x7f8260067000 msgr2=0x7f8260153970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 -- 172.21.15.133:0/1051118830 shutdown_connections 2024-04-25T10:13:44.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:44.900+0000 7f8265486640 1 -- 172.21.15.133:0/1051118830 wait complete. 2024-04-25T10:13:44.917 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:13:45.919 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:46.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb07eb7640 1 Processor -- start 2024-04-25T10:13:46.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb07eb7640 1 -- start start 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb07eb7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb00159dc0 0x7ffb000cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb07eb7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb0015e220 0x7ffb0006d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb07eb7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb000ce880 con 0x7ffb0015a1a0 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb07eb7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb000c49e0 con 0x7ffb00159dc0 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb07eb7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000ceb20 con 0x7ffb0015e220 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb05452640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ffb0015a1a0 0x7ffb0015db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54658/0 (socket says 172.21.15.133:54658) 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb05452640 1 -- 172.21.15.133:0/3709661286 learned_addr learned my addr 172.21.15.133:0/3709661286 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb06454640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb0015e220 0x7ffb0006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb05c53640 1 --2- 172.21.15.133:0/3709661286 >> v2:172.21.15.162:3300/0 conn(0x7ffb00159dc0 0x7ffb000cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869444507 0 0) 0x7ffb000ce880 con 0x7ffb0015a1a0 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaec003530 con 0x7ffb0015a1a0 2024-04-25T10:13:46.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3900568002 0 0) 0x7ffaec003530 con 0x7ffb0015a1a0 2024-04-25T10:13:46.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb0015e220 msgr2=0x7ffb0006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:46.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb0015e220 0x7ffb0006d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 >> v2:172.21.15.162:3300/0 conn(0x7ffb00159dc0 msgr2=0x7ffb000cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:46.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 --2- 172.21.15.133:0/3709661286 >> v2:172.21.15.162:3300/0 conn(0x7ffb00159dc0 0x7ffb000cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb000cbfa0 con 0x7ffb0015a1a0 2024-04-25T10:13:46.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ffaf00016a0 con 0x7ffb0015a1a0 2024-04-25T10:13:46.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 >> v1:172.21.15.133:6789/0 conn(0x7ffb0015a1a0 legacy=0x7ffb0015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec0039c0 0x7ffaec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb0015e220 0x7ffaec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaec004d50 0x7ffaec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000c49e0 con 0x7ffaec004d50 2024-04-25T10:13:46.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000ceb20 con 0x7ffaec0039c0 2024-04-25T10:13:46.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec00b9d0 con 0x7ffb0015e220 2024-04-25T10:13:46.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb06454640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaec004d50 0x7ffaec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb05c53640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec0039c0 0x7ffaec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb05452640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb0015e220 0x7ffaec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb06454640 1 -- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb0015e220 msgr2=0x7ffaec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:46.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb06454640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb0015e220 0x7ffaec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb06454640 1 -- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec0039c0 msgr2=0x7ffaec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:46.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb06454640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffaec0039c0 0x7ffaec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb06454640 1 -- 172.21.15.133:0/3709661286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec003530 con 0x7ffaec004d50 2024-04-25T10:13:46.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.048+0000 7ffb05452640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb0015e220 0x7ffaec004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb06454640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaec004d50 0x7ffaec0051d0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800f270 tx=0x7ffaf801dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278e11ac221ccb97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800ad20 con 0x7ffaec004d50 2024-04-25T10:13:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf80a6b40 con 0x7ffaec004d50 2024-04-25T10:13:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb04c51640 1 -- 172.21.15.133:0/3709661286 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8008440 con 0x7ffaec004d50 2024-04-25T10:13:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb07eb7640 1 -- 172.21.15.133:0/3709661286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaec004d50 msgr2=0x7ffaec0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb07eb7640 1 --2- 172.21.15.133:0/3709661286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaec004d50 0x7ffaec0051d0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800f270 tx=0x7ffaf801dd40 comp rx=0 tx=0).stop 2024-04-25T10:13:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb06454640 1 -- 172.21.15.133:0/3709661286 reap_dead start 2024-04-25T10:13:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb07eb7640 1 -- 172.21.15.133:0/3709661286 shutdown_connections 2024-04-25T10:13:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb07eb7640 1 -- 172.21.15.133:0/3709661286 >> 172.21.15.133:0/3709661286 conn(0x7ffb00067000 msgr2=0x7ffb0006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb07eb7640 1 -- 172.21.15.133:0/3709661286 shutdown_connections 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb07eb7640 1 -- 172.21.15.133:0/3709661286 wait complete. 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb07eb7640 1 Processor -- start 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.052+0000 7ffb07eb7640 1 -- start start 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb07eb7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb00155a10 0x7ffb00150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb07eb7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb00155df0 0x7ffb001511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb07eb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb00154470 0x7ffb001548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb07eb7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000c47c0 con 0x7ffb00154470 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb07eb7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000c4640 con 0x7ffb00155df0 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb07eb7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000c4cc0 con 0x7ffb00155a10 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb06454640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb00154470 0x7ffb001548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb06454640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb00154470 0x7ffb001548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50684/0 (socket says 172.21.15.133:50684) 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb06454640 1 -- 172.21.15.133:0/2620037469 learned_addr learned my addr 172.21.15.133:0/2620037469 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb05c53640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb00155a10 0x7ffb00150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb05452640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb00155df0 0x7ffb001511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb06454640 1 -- 172.21.15.133:0/2620037469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb00155a10 msgr2=0x7ffb00150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb06454640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb00155a10 0x7ffb00150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb06454640 1 -- 172.21.15.133:0/2620037469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb00155df0 msgr2=0x7ffb001511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb06454640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb00155df0 0x7ffb001511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb06454640 1 -- 172.21.15.133:0/2620037469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb00151a20 con 0x7ffb00154470 2024-04-25T10:13:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb05c53640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb00155a10 0x7ffb00150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb06454640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb00154470 0x7ffb001548d0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800f270 tx=0x7ffaf800a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffaf67fc640 1 -- 172.21.15.133:0/2620037469 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800ad20 con 0x7ffb00154470 2024-04-25T10:13:46.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffaf67fc640 1 -- 172.21.15.133:0/2620037469 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf80bb070 con 0x7ffb00154470 2024-04-25T10:13:46.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb07eb7640 1 -- 172.21.15.133:0/2620037469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb00153990 con 0x7ffb00154470 2024-04-25T10:13:46.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffaf67fc640 1 -- 172.21.15.133:0/2620037469 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800a2e0 con 0x7ffb00154470 2024-04-25T10:13:46.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb07eb7640 1 -- 172.21.15.133:0/2620037469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb00151700 con 0x7ffb00154470 2024-04-25T10:13:46.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb07eb7640 1 -- 172.21.15.133:0/2620037469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb00068ef0 con 0x7ffb00154470 2024-04-25T10:13:46.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffaf67fc640 1 -- 172.21.15.133:0/2620037469 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffaf80b5070 con 0x7ffb00154470 2024-04-25T10:13:46.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffaf67fc640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffae4066f60 0x7ffae4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffaf67fc640 1 -- 172.21.15.133:0/2620037469 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffaf811fd90 con 0x7ffb00154470 2024-04-25T10:13:46.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb05c53640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffae4066f60 0x7ffae4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.056+0000 7ffb05c53640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffae4066f60 0x7ffae4069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009ad0 tx=0x7ffafc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.060+0000 7ffaf67fc640 1 -- 172.21.15.133:0/2620037469 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffaf8017a90 con 0x7ffb00154470 2024-04-25T10:13:46.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.268+0000 7ffb07eb7640 1 -- 172.21.15.133:0/2620037469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb0015dab0 con 0x7ffb00154470 2024-04-25T10:13:46.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.268+0000 7ffaf67fc640 1 -- 172.21.15.133:0/2620037469 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ffaf80f0b80 con 0x7ffb00154470 2024-04-25T10:13:46.272 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:46.272 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:13:46.272 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:13:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 -- 172.21.15.133:0/2620037469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffae4066f60 msgr2=0x7ffae4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffae4066f60 0x7ffae4069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009ad0 tx=0x7ffafc008040 comp rx=0 tx=0).stop 2024-04-25T10:13:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 -- 172.21.15.133:0/2620037469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb00154470 msgr2=0x7ffb001548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb00154470 0x7ffb001548d0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800f270 tx=0x7ffaf800a860 comp rx=0 tx=0).stop 2024-04-25T10:13:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 -- 172.21.15.133:0/2620037469 shutdown_connections 2024-04-25T10:13:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffae4066f60 0x7ffae4069420 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb00154470 0x7ffb001548d0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb00155df0 0x7ffb001511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 --2- 172.21.15.133:0/2620037469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb00155a10 0x7ffb00150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 -- 172.21.15.133:0/2620037469 >> 172.21.15.133:0/2620037469 conn(0x7ffb00067000 msgr2=0x7ffb0015a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 -- 172.21.15.133:0/2620037469 shutdown_connections 2024-04-25T10:13:46.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.272+0000 7ffb07eb7640 1 -- 172.21.15.133:0/2620037469 wait complete. 2024-04-25T10:13:46.291 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:13:46.292 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:46.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.400+0000 7fc53a08b640 1 Processor -- start 2024-04-25T10:13:46.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.400+0000 7fc53a08b640 1 -- start start 2024-04-25T10:13:46.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc53a08b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc53415db00 0x7fc5340b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc53a08b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5340ceb20 con 0x7fc5340d0770 2024-04-25T10:13:46.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc53a08b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5340b97f0 con 0x7fc5340d0bd0 2024-04-25T10:13:46.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc53a08b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5341622e0 con 0x7fc53415db00 2024-04-25T10:13:46.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc52affd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc5340d0770 0x7fc5341614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42576/0 (socket says 172.21.15.133:42576) 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc52affd640 1 -- 172.21.15.133:0/1765715048 learned_addr learned my addr 172.21.15.133:0/1765715048 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc5337fe640 1 --2- 172.21.15.133:0/1765715048 >> v2:172.21.15.162:3300/0 conn(0x7fc53415db00 0x7fc5340b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674177695 0 0) 0x7fc5340ceb20 con 0x7fc5340d0770 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc520003530 con 0x7fc5340d0770 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3900445574 0 0) 0x7fc520003530 con 0x7fc5340d0770 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 >> v2:172.21.15.162:3300/0 conn(0x7fc53415db00 msgr2=0x7fc5340b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 --2- 172.21.15.133:0/1765715048 >> v2:172.21.15.162:3300/0 conn(0x7fc53415db00 0x7fc5340b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 >> v1:172.21.15.162:6789/0 conn(0x7fc5340d0bd0 legacy=0x7fc534161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5340b7740 con 0x7fc5340d0770 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fc51c001680 con 0x7fc5340d0770 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 >> v1:172.21.15.133:6789/0 conn(0x7fc5340d0770 legacy=0x7fc5341614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 --2- 172.21.15.133:0/1765715048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5200039c0 0x7fc520003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 --2- 172.21.15.133:0/1765715048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc520004810 0x7fc520004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 --2- 172.21.15.133:0/1765715048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5200051d0 0x7fc52000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5341622e0 con 0x7fc5200039c0 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52000c120 con 0x7fc520004810 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52000c320 con 0x7fc5200051d0 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc5337fe640 1 --2- 172.21.15.133:0/1765715048 >> v2:172.21.15.162:3300/0 conn(0x7fc53415db00 0x7fc5340b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc5337fe640 1 --2- 172.21.15.133:0/1765715048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5200039c0 0x7fc520003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc52affd640 1 --2- 172.21.15.133:0/1765715048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc520004810 0x7fc520004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc533fff640 1 --2- 172.21.15.133:0/1765715048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5200051d0 0x7fc52000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc5337fe640 1 -- 172.21.15.133:0/1765715048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5200051d0 msgr2=0x7fc52000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc5337fe640 1 --2- 172.21.15.133:0/1765715048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5200051d0 0x7fc52000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc5337fe640 1 -- 172.21.15.133:0/1765715048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc520004810 msgr2=0x7fc520004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc5337fe640 1 --2- 172.21.15.133:0/1765715048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc520004810 0x7fc520004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc5337fe640 1 -- 172.21.15.133:0/1765715048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc520003530 con 0x7fc5200039c0 2024-04-25T10:13:46.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.404+0000 7fc533fff640 1 -- 172.21.15.133:0/1765715048 reap_dead start 2024-04-25T10:13:46.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc5337fe640 1 --2- 172.21.15.133:0/1765715048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5200039c0 0x7fc520003de0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fc5240098b0 tx=0x7fc52401c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7d520a73366a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400f040 con 0x7fc5200039c0 2024-04-25T10:13:46.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc524004020 con 0x7fc5200039c0 2024-04-25T10:13:46.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc532ffd640 1 -- 172.21.15.133:0/1765715048 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5240043f0 con 0x7fc5200039c0 2024-04-25T10:13:46.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 -- 172.21.15.133:0/1765715048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5200039c0 msgr2=0x7fc520003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 --2- 172.21.15.133:0/1765715048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5200039c0 0x7fc520003de0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fc5240098b0 tx=0x7fc52401c5a0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 -- 172.21.15.133:0/1765715048 shutdown_connections 2024-04-25T10:13:46.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 --2- 172.21.15.133:0/1765715048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5200039c0 0x7fc520003de0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 -- 172.21.15.133:0/1765715048 >> 172.21.15.133:0/1765715048 conn(0x7fc5340727c0 msgr2=0x7fc534072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:46.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 -- 172.21.15.133:0/1765715048 shutdown_connections 2024-04-25T10:13:46.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 -- 172.21.15.133:0/1765715048 wait complete. 2024-04-25T10:13:46.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 Processor -- start 2024-04-25T10:13:46.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 -- start start 2024-04-25T10:13:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5200039c0 0x7fc5340ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc53415b120 0x7fc5340ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5340ac9b0 0x7fc5341f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340b7430 con 0x7fc53415b120 2024-04-25T10:13:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340b82c0 con 0x7fc5200039c0 2024-04-25T10:13:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc53a08b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340d0fe0 con 0x7fc5340ac9b0 2024-04-25T10:13:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc52affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc53415b120 0x7fc5340ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc52affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc53415b120 0x7fc5340ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52524/0 (socket says 172.21.15.133:52524) 2024-04-25T10:13:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc52affd640 1 -- 172.21.15.133:0/413616829 learned_addr learned my addr 172.21.15.133:0/413616829 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:46.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc5337fe640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5200039c0 0x7fc5340ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc533fff640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5340ac9b0 0x7fc5341f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc52affd640 1 -- 172.21.15.133:0/413616829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5340ac9b0 msgr2=0x7fc5341f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc52affd640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5340ac9b0 0x7fc5341f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc52affd640 1 -- 172.21.15.133:0/413616829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5200039c0 msgr2=0x7fc5340ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc52affd640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5200039c0 0x7fc5340ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc52affd640 1 -- 172.21.15.133:0/413616829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc534160f00 con 0x7fc53415b120 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc5337fe640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5200039c0 0x7fc5340ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.408+0000 7fc52affd640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc53415b120 0x7fc5340ac470 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0072c0 tx=0x7fc51c007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.412+0000 7fc530ff9640 1 -- 172.21.15.133:0/413616829 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c001640 con 0x7fc53415b120 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.412+0000 7fc530ff9640 1 -- 172.21.15.133:0/413616829 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c018070 con 0x7fc53415b120 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.412+0000 7fc53a08b640 1 -- 172.21.15.133:0/413616829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5340cc3f0 con 0x7fc53415b120 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.412+0000 7fc530ff9640 1 -- 172.21.15.133:0/413616829 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00fbe0 con 0x7fc53415b120 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.412+0000 7fc53a08b640 1 -- 172.21.15.133:0/413616829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5340cc970 con 0x7fc53415b120 2024-04-25T10:13:46.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.412+0000 7fc53a08b640 1 -- 172.21.15.133:0/413616829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5340acdc0 con 0x7fc53415b120 2024-04-25T10:13:46.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.412+0000 7fc530ff9640 1 -- 172.21.15.133:0/413616829 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc51c01d460 con 0x7fc53415b120 2024-04-25T10:13:46.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.412+0000 7fc530ff9640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc514066f60 0x7fc514069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.412+0000 7fc530ff9640 1 -- 172.21.15.133:0/413616829 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc51c0850b0 con 0x7fc53415b120 2024-04-25T10:13:46.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.412+0000 7fc5337fe640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc514066f60 0x7fc514069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.412+0000 7fc5337fe640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc514066f60 0x7fc514069420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc5240ad000 tx=0x7fc5240046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.416+0000 7fc530ff9640 1 -- 172.21.15.133:0/413616829 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5340acdc0 con 0x7fc53415b120 2024-04-25T10:13:46.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.632+0000 7fc53a08b640 1 -- 172.21.15.133:0/413616829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc5340b2750 con 0x7fc53415b120 2024-04-25T10:13:46.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.632+0000 7fc530ff9640 1 -- 172.21.15.133:0/413616829 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7fc51c055fc0 con 0x7fc53415b120 2024-04-25T10:13:46.636 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:46.636 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":739,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077628953,"bytes_used":2679775232,"bytes_avail":770414338048,"bytes_total":773094113280,"write_bytes_sec":62732893,"read_op_per_sec":0,"write_op_per_sec":64},"fsmap":{"epoch":79,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9495}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:12:51.919939+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:13:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 -- 172.21.15.133:0/413616829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc514066f60 msgr2=0x7fc514069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc514066f60 0x7fc514069420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc5240ad000 tx=0x7fc5240046d0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 -- 172.21.15.133:0/413616829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc53415b120 msgr2=0x7fc5340ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc53415b120 0x7fc5340ac470 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0072c0 tx=0x7fc51c007780 comp rx=0 tx=0).stop 2024-04-25T10:13:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 -- 172.21.15.133:0/413616829 shutdown_connections 2024-04-25T10:13:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5340ac9b0 0x7fc5341f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc514066f60 0x7fc514069420 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc53415b120 0x7fc5340ac470 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 --2- 172.21.15.133:0/413616829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5200039c0 0x7fc5340ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 -- 172.21.15.133:0/413616829 >> 172.21.15.133:0/413616829 conn(0x7fc5340727c0 msgr2=0x7fc534161a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 -- 172.21.15.133:0/413616829 shutdown_connections 2024-04-25T10:13:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:46.636+0000 7fc53a08b640 1 -- 172.21.15.133:0/413616829 wait complete. 2024-04-25T10:13:46.656 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:13:47.657 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:47.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.788+0000 7f032259a640 1 Processor -- start 2024-04-25T10:13:47.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.788+0000 7f032259a640 1 -- start start 2024-04-25T10:13:47.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f032259a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f031c0ce330 0x7f031c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f032259a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031c0c61d0 con 0x7f031c0b1f70 2024-04-25T10:13:47.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f032259a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031c0c64b0 con 0x7f031c15c9a0 2024-04-25T10:13:47.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f032259a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031c0cecc0 con 0x7f031c0ce330 2024-04-25T10:13:47.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f031b7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f031c0b1f70 0x7f031c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42590/0 (socket says 172.21.15.133:42590) 2024-04-25T10:13:47.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f031b7fe640 1 -- 172.21.15.133:0/245309193 learned_addr learned my addr 172.21.15.133:0/245309193 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:47.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f0320b37640 1 --2- 172.21.15.133:0/245309193 >> v2:172.21.15.133:3300/0 conn(0x7f031c0ce330 0x7f031c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f0320b37640 1 -- 172.21.15.133:0/245309193 >> v1:172.21.15.162:6789/0 conn(0x7f031c15c9a0 legacy=0x7f031c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f0320b37640 1 -- 172.21.15.133:0/245309193 >> v1:172.21.15.133:6789/0 conn(0x7f031c0b1f70 legacy=0x7f031c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f0320b37640 1 -- 172.21.15.133:0/245309193 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c06d4f0 con 0x7f031c0ce330 2024-04-25T10:13:47.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f0320b37640 1 --2- 172.21.15.133:0/245309193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031c0ce330 0x7f031c0ce780 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f0314009d90 tx=0x7f031401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8261817d957474d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f031affd640 1 -- 172.21.15.133:0/245309193 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140b2070 con 0x7f031c0ce330 2024-04-25T10:13:47.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f031affd640 1 -- 172.21.15.133:0/245309193 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03140a57f0 con 0x7f031c0ce330 2024-04-25T10:13:47.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f031affd640 1 -- 172.21.15.133:0/245309193 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140a5bc0 con 0x7f031c0ce330 2024-04-25T10:13:47.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f032259a640 1 -- 172.21.15.133:0/245309193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031c0ce330 msgr2=0x7f031c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f032259a640 1 --2- 172.21.15.133:0/245309193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031c0ce330 0x7f031c0ce780 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f0314009d90 tx=0x7f031401c930 comp rx=0 tx=0).stop 2024-04-25T10:13:47.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f032259a640 1 -- 172.21.15.133:0/245309193 shutdown_connections 2024-04-25T10:13:47.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f032259a640 1 --2- 172.21.15.133:0/245309193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031c0ce330 0x7f031c0ce780 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.792+0000 7f032259a640 1 -- 172.21.15.133:0/245309193 >> 172.21.15.133:0/245309193 conn(0x7f031c067000 msgr2=0x7f031c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:47.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 -- 172.21.15.133:0/245309193 shutdown_connections 2024-04-25T10:13:47.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 -- 172.21.15.133:0/245309193 wait complete. 2024-04-25T10:13:47.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 Processor -- start 2024-04-25T10:13:47.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 -- start start 2024-04-25T10:13:47.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f031c0b1f70 0x7f031c14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f031c0ce330 0x7f031c139450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031c15c9a0 0x7f031c14e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c15dc20 con 0x7f031c15c9a0 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c1695d0 con 0x7f031c0b1f70 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c14d3a0 con 0x7f031c0ce330 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0320b37640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031c15c9a0 0x7f031c14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0320b37640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031c15c9a0 0x7f031c14e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52538/0 (socket says 172.21.15.133:52538) 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0320b37640 1 -- 172.21.15.133:0/77614314 learned_addr learned my addr 172.21.15.133:0/77614314 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f031b7fe640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f031c0ce330 0x7f031c139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f031bfff640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f031c0b1f70 0x7f031c14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0320b37640 1 -- 172.21.15.133:0/77614314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f031c0ce330 msgr2=0x7f031c139450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0320b37640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f031c0ce330 0x7f031c139450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0320b37640 1 -- 172.21.15.133:0/77614314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f031c0b1f70 msgr2=0x7f031c14c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0320b37640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f031c0b1f70 0x7f031c14c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0320b37640 1 -- 172.21.15.133:0/77614314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c139990 con 0x7f031c15c9a0 2024-04-25T10:13:47.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0320b37640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031c15c9a0 0x7f031c14e910 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f0314009d90 tx=0x7f03140196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0318ff9640 1 -- 172.21.15.133:0/77614314 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140b2040 con 0x7f031c15c9a0 2024-04-25T10:13:47.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0318ff9640 1 -- 172.21.15.133:0/77614314 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03140bf070 con 0x7f031c15c9a0 2024-04-25T10:13:47.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f0318ff9640 1 -- 172.21.15.133:0/77614314 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140b8890 con 0x7f031c15c9a0 2024-04-25T10:13:47.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 -- 172.21.15.133:0/77614314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031c0c9dc0 con 0x7f031c15c9a0 2024-04-25T10:13:47.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.796+0000 7f032259a640 1 -- 172.21.15.133:0/77614314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031c0ca380 con 0x7f031c15c9a0 2024-04-25T10:13:47.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.800+0000 7f032259a640 1 -- 172.21.15.133:0/77614314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031c068ef0 con 0x7f031c15c9a0 2024-04-25T10:13:47.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.800+0000 7f0318ff9640 1 -- 172.21.15.133:0/77614314 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03140b8a30 con 0x7f031c15c9a0 2024-04-25T10:13:47.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.800+0000 7f0318ff9640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0308066f60 0x7f0308069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.800+0000 7f0318ff9640 1 -- 172.21.15.133:0/77614314 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0314120400 con 0x7f031c15c9a0 2024-04-25T10:13:47.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.800+0000 7f031bfff640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0308066f60 0x7f0308069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.800+0000 7f031bfff640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0308066f60 0x7f0308069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f031c06f120 tx=0x7f030c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.804+0000 7f0318ff9640 1 -- 172.21.15.133:0/77614314 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0314016520 con 0x7f031c15c9a0 2024-04-25T10:13:48.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:47.996+0000 7f032259a640 1 -- 172.21.15.133:0/77614314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f031c152cb0 con 0x7f031c15c9a0 2024-04-25T10:13:48.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f0318ff9640 1 -- 172.21.15.133:0/77614314 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f03140f13a0 con 0x7f031c15c9a0 2024-04-25T10:13:48.001 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:48.001 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:13:48.001 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:13:48.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 -- 172.21.15.133:0/77614314 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0308066f60 msgr2=0x7f0308069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0308066f60 0x7f0308069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f031c06f120 tx=0x7f030c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:48.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 -- 172.21.15.133:0/77614314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031c15c9a0 msgr2=0x7f031c14e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031c15c9a0 0x7f031c14e910 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f0314009d90 tx=0x7f03140196d0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 -- 172.21.15.133:0/77614314 shutdown_connections 2024-04-25T10:13:48.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0308066f60 0x7f0308069420 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f031c15c9a0 0x7f031c14e910 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f031c0ce330 0x7f031c139450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 --2- 172.21.15.133:0/77614314 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f031c0b1f70 0x7f031c14c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 -- 172.21.15.133:0/77614314 >> 172.21.15.133:0/77614314 conn(0x7f031c067000 msgr2=0x7f031c15e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 -- 172.21.15.133:0/77614314 shutdown_connections 2024-04-25T10:13:48.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.000+0000 7f032259a640 1 -- 172.21.15.133:0/77614314 wait complete. 2024-04-25T10:13:48.020 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:13:48.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:48.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f66178ac640 1 Processor -- start 2024-04-25T10:13:48.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f66178ac640 1 -- start start 2024-04-25T10:13:48.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f66178ac640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f661015c9a0 0x7f66100c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f66178ac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 0x7f66100ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f66178ac640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66100c61d0 con 0x7f66100b1f70 2024-04-25T10:13:48.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f66178ac640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66100cecc0 con 0x7f661015c9a0 2024-04-25T10:13:48.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f66178ac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66100c64b0 con 0x7f66100ce330 2024-04-25T10:13:48.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f6615e49640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 0x7f66100ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f6615e49640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 0x7f66100ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39880/0 (socket says 172.21.15.133:39880) 2024-04-25T10:13:48.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f6615648640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f661015c9a0 0x7f66100c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f6614e47640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f66100b1f70 0x7f661006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:48918/0 (socket says 172.21.15.133:48918) 2024-04-25T10:13:48.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.136+0000 7f6615e49640 1 -- 172.21.15.133:0/2317951566 learned_addr learned my addr 172.21.15.133:0/2317951566 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:48.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f6615e49640 1 -- 172.21.15.133:0/2317951566 >> v2:172.21.15.162:3300/0 conn(0x7f661015c9a0 msgr2=0x7f66100c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:48.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f6615e49640 1 --2- 172.21.15.133:0/2317951566 >> v2:172.21.15.162:3300/0 conn(0x7f661015c9a0 0x7f66100c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f6615e49640 1 -- 172.21.15.133:0/2317951566 >> v1:172.21.15.162:6789/0 conn(0x7f66100b1f70 legacy=0x7f661006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f6615e49640 1 -- 172.21.15.133:0/2317951566 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661006d4f0 con 0x7f66100ce330 2024-04-25T10:13:48.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f6615648640 1 --2- 172.21.15.133:0/2317951566 >> v2:172.21.15.162:3300/0 conn(0x7f661015c9a0 0x7f66100c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:48.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f6615e49640 1 --2- 172.21.15.133:0/2317951566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 0x7f66100ce780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6608009e50 tx=0x7f660801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56dd5d7f62d6a50e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f6607fff640 1 -- 172.21.15.133:0/2317951566 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66080a5700 con 0x7f66100ce330 2024-04-25T10:13:48.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f6607fff640 1 -- 172.21.15.133:0/2317951566 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66080a58a0 con 0x7f66100ce330 2024-04-25T10:13:48.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f6607fff640 1 -- 172.21.15.133:0/2317951566 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66080a5c70 con 0x7f66100ce330 2024-04-25T10:13:48.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f66178ac640 1 -- 172.21.15.133:0/2317951566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 msgr2=0x7f66100ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f66178ac640 1 --2- 172.21.15.133:0/2317951566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 0x7f66100ce780 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6608009e50 tx=0x7f660801ca30 comp rx=0 tx=0).stop 2024-04-25T10:13:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f66178ac640 1 -- 172.21.15.133:0/2317951566 shutdown_connections 2024-04-25T10:13:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f66178ac640 1 --2- 172.21.15.133:0/2317951566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 0x7f66100ce780 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f66178ac640 1 --2- 172.21.15.133:0/2317951566 >> v2:172.21.15.162:3300/0 conn(0x7f661015c9a0 0x7f66100c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f66178ac640 1 -- 172.21.15.133:0/2317951566 >> 172.21.15.133:0/2317951566 conn(0x7f6610067000 msgr2=0x7f661006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f66178ac640 1 -- 172.21.15.133:0/2317951566 shutdown_connections 2024-04-25T10:13:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.140+0000 7f66178ac640 1 -- 172.21.15.133:0/2317951566 wait complete. 2024-04-25T10:13:48.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f66178ac640 1 Processor -- start 2024-04-25T10:13:48.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f66178ac640 1 -- start start 2024-04-25T10:13:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f66178ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66100b1f70 0x7f661013ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f66178ac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 0x7f6610151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f66178ac640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015c9a0 0x7f66101515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f66178ac640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661015dad0 con 0x7f66100b1f70 2024-04-25T10:13:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f66178ac640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661015d950 con 0x7f661015c9a0 2024-04-25T10:13:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f66178ac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661015dc50 con 0x7f66100ce330 2024-04-25T10:13:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6615648640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66100b1f70 0x7f661013ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6615648640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66100b1f70 0x7f661013ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52544/0 (socket says 172.21.15.133:52544) 2024-04-25T10:13:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6615648640 1 -- 172.21.15.133:0/445596328 learned_addr learned my addr 172.21.15.133:0/445596328 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6614e47640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 0x7f6610151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6615e49640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015c9a0 0x7f66101515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6615648640 1 -- 172.21.15.133:0/445596328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 msgr2=0x7f6610151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6615648640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 0x7f6610151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6615648640 1 -- 172.21.15.133:0/445596328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015c9a0 msgr2=0x7f66101515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6615648640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015c9a0 0x7f66101515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6615648640 1 -- 172.21.15.133:0/445596328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610157c80 con 0x7f66100b1f70 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6615e49640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015c9a0 0x7f66101515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6615648640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66100b1f70 0x7f661013ef60 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f660c00ebe0 tx=0x7f660c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6605ffb640 1 -- 172.21.15.133:0/445596328 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c017070 con 0x7f66100b1f70 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6605ffb640 1 -- 172.21.15.133:0/445596328 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c00cc30 con 0x7f66100b1f70 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f66178ac640 1 -- 172.21.15.133:0/445596328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6610157f60 con 0x7f66100b1f70 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f6605ffb640 1 -- 172.21.15.133:0/445596328 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c013470 con 0x7f66100b1f70 2024-04-25T10:13:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f66178ac640 1 -- 172.21.15.133:0/445596328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6610158520 con 0x7f66100b1f70 2024-04-25T10:13:48.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.144+0000 7f66178ac640 1 -- 172.21.15.133:0/445596328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661006bc20 con 0x7f66100b1f70 2024-04-25T10:13:48.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.148+0000 7f6605ffb640 1 -- 172.21.15.133:0/445596328 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f660c01a070 con 0x7f66100b1f70 2024-04-25T10:13:48.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.148+0000 7f6605ffb640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f4066f60 0x7f65f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.148+0000 7f6605ffb640 1 -- 172.21.15.133:0/445596328 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f660c0851b0 con 0x7f66100b1f70 2024-04-25T10:13:48.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.148+0000 7f6614e47640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f4066f60 0x7f65f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.148+0000 7f6614e47640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f4066f60 0x7f65f4069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f661014fb10 tx=0x7f6600007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.152+0000 7f6605ffb640 1 -- 172.21.15.133:0/445596328 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f660c055fa0 con 0x7f66100b1f70 2024-04-25T10:13:48.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.368+0000 7f66178ac640 1 -- 172.21.15.133:0/445596328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6610068ef0 con 0x7f66100b1f70 2024-04-25T10:13:48.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.368+0000 7f6605ffb640 1 -- 172.21.15.133:0/445596328 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f660c0558a0 con 0x7f66100b1f70 2024-04-25T10:13:48.370 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:48.371 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":741,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077628953,"bytes_used":2679775232,"bytes_avail":770414338048,"bytes_total":773094113280,"write_bytes_sec":36544859,"read_op_per_sec":0,"write_op_per_sec":37},"fsmap":{"epoch":79,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9495}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T10:13:47.935271+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:13:48.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 -- 172.21.15.133:0/445596328 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f4066f60 msgr2=0x7f65f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f4066f60 0x7f65f4069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f661014fb10 tx=0x7f6600007cf0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 -- 172.21.15.133:0/445596328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66100b1f70 msgr2=0x7f661013ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66100b1f70 0x7f661013ef60 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f660c00ebe0 tx=0x7f660c00c520 comp rx=0 tx=0).stop 2024-04-25T10:13:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 -- 172.21.15.133:0/445596328 shutdown_connections 2024-04-25T10:13:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f661015c9a0 0x7f66101515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66100ce330 0x7f6610151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65f4066f60 0x7f65f4069420 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 --2- 172.21.15.133:0/445596328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66100b1f70 0x7f661013ef60 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 -- 172.21.15.133:0/445596328 >> 172.21.15.133:0/445596328 conn(0x7f6610067000 msgr2=0x7f661015afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 -- 172.21.15.133:0/445596328 shutdown_connections 2024-04-25T10:13:48.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.372+0000 7f66178ac640 1 -- 172.21.15.133:0/445596328 wait complete. 2024-04-25T10:13:48.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:13:48.389 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:13:48.389 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:48.400 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:13:48.400 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'d', 'a', 'b'} 2024-04-25T10:13:48.400 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T10:13:48.565 INFO:teuthology.orchestra.run.smithi133.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T10:13:48.565 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T10:13:48.566 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:48.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.684+0000 7f6a381d7640 1 Processor -- start 2024-04-25T10:13:48.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.684+0000 7f6a381d7640 1 -- start start 2024-04-25T10:13:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a381d7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6a300ce330 0x7f6a300ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a381d7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a300c61d0 con 0x7f6a300b1f70 2024-04-25T10:13:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a381d7640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a300c64b0 con 0x7f6a3015c9a0 2024-04-25T10:13:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a381d7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a300cecc0 con 0x7f6a300ce330 2024-04-25T10:13:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a35772640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6a300b1f70 0x7f6a3006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42606/0 (socket says 172.21.15.133:42606) 2024-04-25T10:13:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a35772640 1 -- 172.21.15.133:0/3497562820 learned_addr learned my addr 172.21.15.133:0/3497562820 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:48.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a36774640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6a300ce330 0x7f6a300ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a34f71640 1 -- 172.21.15.133:0/3497562820 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802723351 0 0) 0x7f6a300c61d0 con 0x7f6a300b1f70 2024-04-25T10:13:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a34f71640 1 -- 172.21.15.133:0/3497562820 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a1c003530 con 0x7f6a300b1f70 2024-04-25T10:13:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a36774640 1 -- 172.21.15.133:0/3497562820 >> v1:172.21.15.162:6789/0 conn(0x7f6a3015c9a0 legacy=0x7f6a300c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a36774640 1 -- 172.21.15.133:0/3497562820 >> v1:172.21.15.133:6789/0 conn(0x7f6a300b1f70 legacy=0x7f6a3006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a36774640 1 -- 172.21.15.133:0/3497562820 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3006d4f0 con 0x7f6a300ce330 2024-04-25T10:13:48.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a36774640 1 --2- 172.21.15.133:0/3497562820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a300ce330 0x7f6a300ce780 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009d90 tx=0x7f6a2801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9b73a9bb21eac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a34f71640 1 -- 172.21.15.133:0/3497562820 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a280b2070 con 0x7f6a300ce330 2024-04-25T10:13:48.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a34f71640 1 -- 172.21.15.133:0/3497562820 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a280a5830 con 0x7f6a300ce330 2024-04-25T10:13:48.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a34f71640 1 -- 172.21.15.133:0/3497562820 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a280a5be0 con 0x7f6a300ce330 2024-04-25T10:13:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a381d7640 1 -- 172.21.15.133:0/3497562820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a300ce330 msgr2=0x7f6a300ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a381d7640 1 --2- 172.21.15.133:0/3497562820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a300ce330 0x7f6a300ce780 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009d90 tx=0x7f6a2801c970 comp rx=0 tx=0).stop 2024-04-25T10:13:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a381d7640 1 -- 172.21.15.133:0/3497562820 shutdown_connections 2024-04-25T10:13:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a381d7640 1 --2- 172.21.15.133:0/3497562820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a300ce330 0x7f6a300ce780 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a381d7640 1 -- 172.21.15.133:0/3497562820 >> 172.21.15.133:0/3497562820 conn(0x7f6a30067000 msgr2=0x7f6a3006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a381d7640 1 -- 172.21.15.133:0/3497562820 shutdown_connections 2024-04-25T10:13:48.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.688+0000 7f6a381d7640 1 -- 172.21.15.133:0/3497562820 wait complete. 2024-04-25T10:13:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a381d7640 1 Processor -- start 2024-04-25T10:13:48.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a381d7640 1 -- start start 2024-04-25T10:13:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a381d7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a300b1f70 0x7f6a3014eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a381d7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a300ce330 0x7f6a30151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a381d7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3015c9a0 0x7f6a300c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a381d7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3015dc20 con 0x7f6a300ce330 2024-04-25T10:13:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a381d7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a301695d0 con 0x7f6a3015c9a0 2024-04-25T10:13:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a381d7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a30152240 con 0x7f6a300b1f70 2024-04-25T10:13:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a35772640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a300ce330 0x7f6a30151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a35772640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a300ce330 0x7f6a30151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52558/0 (socket says 172.21.15.133:52558) 2024-04-25T10:13:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a35772640 1 -- 172.21.15.133:0/1919225972 learned_addr learned my addr 172.21.15.133:0/1919225972 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a36774640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3015c9a0 0x7f6a300c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a35f73640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a300b1f70 0x7f6a3014eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a35772640 1 -- 172.21.15.133:0/1919225972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a300b1f70 msgr2=0x7f6a3014eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a35772640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a300b1f70 0x7f6a3014eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a35772640 1 -- 172.21.15.133:0/1919225972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3015c9a0 msgr2=0x7f6a300c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a35772640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3015c9a0 0x7f6a300c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a35772640 1 -- 172.21.15.133:0/1919225972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30150500 con 0x7f6a300ce330 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a36774640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3015c9a0 0x7f6a300c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a35f73640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a300b1f70 0x7f6a3014eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a35772640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a300ce330 0x7f6a30151d00 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000c420 tx=0x7f6a2000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a267fc640 1 -- 172.21.15.133:0/1919225972 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20017070 con 0x7f6a300ce330 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a267fc640 1 -- 172.21.15.133:0/1919225972 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a20005cf0 con 0x7f6a300ce330 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a381d7640 1 -- 172.21.15.133:0/1919225972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a301507e0 con 0x7f6a300ce330 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a267fc640 1 -- 172.21.15.133:0/1919225972 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a200072d0 con 0x7f6a300ce330 2024-04-25T10:13:48.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a381d7640 1 -- 172.21.15.133:0/1919225972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a30150da0 con 0x7f6a300ce330 2024-04-25T10:13:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.692+0000 7f6a381d7640 1 -- 172.21.15.133:0/1919225972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a30068ef0 con 0x7f6a300ce330 2024-04-25T10:13:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.696+0000 7f6a267fc640 1 -- 172.21.15.133:0/1919225972 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a2001a070 con 0x7f6a300ce330 2024-04-25T10:13:48.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.696+0000 7f6a267fc640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a14066f60 0x7f6a14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.696+0000 7f6a267fc640 1 -- 172.21.15.133:0/1919225972 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a200844b0 con 0x7f6a300ce330 2024-04-25T10:13:48.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.696+0000 7f6a35f73640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a14066f60 0x7f6a14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.696+0000 7f6a35f73640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a14066f60 0x7f6a14069420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6a3015e360 tx=0x7f6a2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.700+0000 7f6a267fc640 1 -- 172.21.15.133:0/1919225972 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a20054ce0 con 0x7f6a300ce330 2024-04-25T10:13:48.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.888+0000 7f6a381d7640 1 -- 172.21.15.133:0/1919225972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a30152a60 con 0x7f6a300ce330 2024-04-25T10:13:48.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.888+0000 7f6a267fc640 1 -- 172.21.15.133:0/1919225972 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f6a20058d80 con 0x7f6a300ce330 2024-04-25T10:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:48.893 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:13:48.893 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:13:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 -- 172.21.15.133:0/1919225972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a14066f60 msgr2=0x7f6a14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a14066f60 0x7f6a14069420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6a3015e360 tx=0x7f6a2c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 -- 172.21.15.133:0/1919225972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a300ce330 msgr2=0x7f6a30151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a300ce330 0x7f6a30151d00 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000c420 tx=0x7f6a2000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 -- 172.21.15.133:0/1919225972 shutdown_connections 2024-04-25T10:13:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a3015c9a0 0x7f6a300c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6a14066f60 0x7f6a14069420 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a300ce330 0x7f6a30151d00 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 --2- 172.21.15.133:0/1919225972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6a300b1f70 0x7f6a3014eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 -- 172.21.15.133:0/1919225972 >> 172.21.15.133:0/1919225972 conn(0x7f6a30067000 msgr2=0x7f6a3015e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 -- 172.21.15.133:0/1919225972 shutdown_connections 2024-04-25T10:13:48.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:48.892+0000 7f6a381d7640 1 -- 172.21.15.133:0/1919225972 wait complete. 2024-04-25T10:13:48.911 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:13:53.357 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:53.360 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:53.912 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:54.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e52d97640 1 Processor -- start 2024-04-25T10:13:54.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e52d97640 1 -- start start 2024-04-25T10:13:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e52d97640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8e4c0cfc70 0x7f8e4c0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e52d97640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0c5aa0 0x7f8e4c0c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e52d97640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c15bd90 con 0x7f8e4c15b9b0 2024-04-25T10:13:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e52d97640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c0d0610 con 0x7f8e4c0cfc70 2024-04-25T10:13:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e52d97640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0c5610 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e51334640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0c5aa0 0x7f8e4c0c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e4bfff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8e4c0cfc70 0x7f8e4c0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e51334640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0c5aa0 0x7f8e4c0c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:39918/0 (socket says 172.21.15.133:39918) 2024-04-25T10:13:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e51334640 1 -- 172.21.15.133:0/3311435075 learned_addr learned my addr 172.21.15.133:0/3311435075 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e4bfff640 1 -- 172.21.15.133:0/3311435075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0c5aa0 msgr2=0x7f8e4c0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e4bfff640 1 --2- 172.21.15.133:0/3311435075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0c5aa0 0x7f8e4c0c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e4bfff640 1 -- 172.21.15.133:0/3311435075 >> v1:172.21.15.162:6789/0 conn(0x7f8e4c15b9b0 legacy=0x7f8e4c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.040+0000 7f8e4bfff640 1 -- 172.21.15.133:0/3311435075 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c09ad10 con 0x7f8e4c0cfc70 2024-04-25T10:13:54.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e4bfff640 1 --2- 172.21.15.133:0/3311435075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c0cfc70 0x7f8e4c0d00d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8e380097c0 tx=0x7f8e3801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb0899687700a557 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e4b7fe640 1 -- 172.21.15.133:0/3311435075 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e380a7070 con 0x7f8e4c0cfc70 2024-04-25T10:13:54.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e4b7fe640 1 -- 172.21.15.133:0/3311435075 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e380b0070 con 0x7f8e4c0cfc70 2024-04-25T10:13:54.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e4b7fe640 1 -- 172.21.15.133:0/3311435075 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e380ab890 con 0x7f8e4c0cfc70 2024-04-25T10:13:54.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 -- 172.21.15.133:0/3311435075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c0cfc70 msgr2=0x7f8e4c0d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 --2- 172.21.15.133:0/3311435075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c0cfc70 0x7f8e4c0d00d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8e380097c0 tx=0x7f8e3801c470 comp rx=0 tx=0).stop 2024-04-25T10:13:54.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 -- 172.21.15.133:0/3311435075 shutdown_connections 2024-04-25T10:13:54.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 --2- 172.21.15.133:0/3311435075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0c5aa0 0x7f8e4c0c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 --2- 172.21.15.133:0/3311435075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c0cfc70 0x7f8e4c0d00d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 -- 172.21.15.133:0/3311435075 >> 172.21.15.133:0/3311435075 conn(0x7f8e4c0b31a0 msgr2=0x7f8e4c0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:54.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 -- 172.21.15.133:0/3311435075 shutdown_connections 2024-04-25T10:13:54.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 -- 172.21.15.133:0/3311435075 wait complete. 2024-04-25T10:13:54.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 Processor -- start 2024-04-25T10:13:54.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 -- start start 2024-04-25T10:13:54.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c0c5aa0 0x7f8e4c14fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0cfc70 0x7f8e4c0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c15b9b0 0x7f8e4c0b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c09a3e0 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0cf990 con 0x7f8e4c15b9b0 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e52d97640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1684e0 con 0x7f8e4c0cfc70 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e50b33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c0c5aa0 0x7f8e4c14fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e50b33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c0c5aa0 0x7f8e4c14fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52572/0 (socket says 172.21.15.133:52572) 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.044+0000 7f8e50b33640 1 -- 172.21.15.133:0/1039536212 learned_addr learned my addr 172.21.15.133:0/1039536212 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e4bfff640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0cfc70 0x7f8e4c0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e51334640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c15b9b0 0x7f8e4c0b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e50b33640 1 -- 172.21.15.133:0/1039536212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0cfc70 msgr2=0x7f8e4c0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e50b33640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0cfc70 0x7f8e4c0c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e50b33640 1 -- 172.21.15.133:0/1039536212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c15b9b0 msgr2=0x7f8e4c0b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e50b33640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c15b9b0 0x7f8e4c0b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e50b33640 1 -- 172.21.15.133:0/1039536212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c1502d0 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e50b33640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c0c5aa0 0x7f8e4c14fb20 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000c450 tx=0x7f8e4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e497fa640 1 -- 172.21.15.133:0/1039536212 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40016020 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e497fa640 1 -- 172.21.15.133:0/1039536212 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e400052d0 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e52d97640 1 -- 172.21.15.133:0/1039536212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c0c9890 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e497fa640 1 -- 172.21.15.133:0/1039536212 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40007260 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e52d97640 1 -- 172.21.15.133:0/1039536212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c0c9e50 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e52d97640 1 -- 172.21.15.133:0/1039536212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c067c70 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e497fa640 1 -- 172.21.15.133:0/1039536212 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e40007400 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e497fa640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e30066f60 0x7f8e30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e497fa640 1 -- 172.21.15.133:0/1039536212 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e40084b00 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e4bfff640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e30066f60 0x7f8e30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.048+0000 7f8e4bfff640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e30066f60 0x7f8e30069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8e380af000 tx=0x7f8e380a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.052+0000 7f8e497fa640 1 -- 172.21.15.133:0/1039536212 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8e40020020 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.240+0000 7f8e52d97640 1 -- 172.21.15.133:0/1039536212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e4c0cf350 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.240+0000 7f8e497fa640 1 -- 172.21.15.133:0/1039536212 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f8e40005480 con 0x7f8e4c0c5aa0 2024-04-25T10:13:54.244 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:54.244 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:13:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:13:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 -- 172.21.15.133:0/1039536212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e30066f60 msgr2=0x7f8e30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e30066f60 0x7f8e30069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8e380af000 tx=0x7f8e380a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 -- 172.21.15.133:0/1039536212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c0c5aa0 msgr2=0x7f8e4c14fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c0c5aa0 0x7f8e4c14fb20 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000c450 tx=0x7f8e4000c910 comp rx=0 tx=0).stop 2024-04-25T10:13:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 -- 172.21.15.133:0/1039536212 shutdown_connections 2024-04-25T10:13:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e4c15b9b0 0x7f8e4c0b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8e4c0cfc70 0x7f8e4c0c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8e30066f60 0x7f8e30069420 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 --2- 172.21.15.133:0/1039536212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c0c5aa0 0x7f8e4c14fb20 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 -- 172.21.15.133:0/1039536212 >> 172.21.15.133:0/1039536212 conn(0x7f8e4c0b31a0 msgr2=0x7f8e4c06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 -- 172.21.15.133:0/1039536212 shutdown_connections 2024-04-25T10:13:54.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:54.244+0000 7f8e52d97640 1 -- 172.21.15.133:0/1039536212 wait complete. 2024-04-25T10:13:54.262 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:13:59.263 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b7bac640 1 Processor -- start 2024-04-25T10:13:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b7bac640 1 -- start start 2024-04-25T10:13:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b7bac640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe2b00cc860 0x7fe2b00cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b7bac640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b006e640 con 0x7fe2b0162090 2024-04-25T10:13:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b7bac640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b006e920 con 0x7fe2b0161cb0 2024-04-25T10:13:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b7bac640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b006ec00 con 0x7fe2b00cc860 2024-04-25T10:13:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b6149640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe2b0162090 0x7fe2b0162db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43364/0 (socket says 172.21.15.133:43364) 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b6149640 1 -- 172.21.15.133:0/2076556069 learned_addr learned my addr 172.21.15.133:0/2076556069 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b5147640 1 --2- 172.21.15.133:0/2076556069 >> v2:172.21.15.162:3300/0 conn(0x7fe2b00cc860 0x7fe2b00cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490880046 0 0) 0x7fe2b006e640 con 0x7fe2b0162090 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe29c003530 con 0x7fe2b0162090 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 83932292 0 0) 0x7fe29c003530 con 0x7fe2b0162090 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 >> v2:172.21.15.162:3300/0 conn(0x7fe2b00cc860 msgr2=0x7fe2b00cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 --2- 172.21.15.133:0/2076556069 >> v2:172.21.15.162:3300/0 conn(0x7fe2b00cc860 0x7fe2b00cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 >> v1:172.21.15.162:6789/0 conn(0x7fe2b0161cb0 legacy=0x7fe2b006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b01625b0 con 0x7fe2b0162090 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe2a8001a00 con 0x7fe2b0162090 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 >> v1:172.21.15.133:6789/0 conn(0x7fe2b0162090 legacy=0x7fe2b0162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe29c0039c0 0x7fe29c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe29c004810 0x7fe29c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe29c0051d0 0x7fe29c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b006ec00 con 0x7fe29c004810 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c00c0a0 con 0x7fe29c0051d0 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c00c2a0 con 0x7fe29c0039c0 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b5147640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe29c004810 0x7fe29c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b5948640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe29c0039c0 0x7fe29c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b6149640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe29c0051d0 0x7fe29c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b5147640 1 -- 172.21.15.133:0/2076556069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe29c0039c0 msgr2=0x7fe29c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b5147640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe29c0039c0 0x7fe29c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b5147640 1 -- 172.21.15.133:0/2076556069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe29c0051d0 msgr2=0x7fe29c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b5147640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe29c0051d0 0x7fe29c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b5147640 1 -- 172.21.15.133:0/2076556069 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c003530 con 0x7fe29c004810 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b6149640 1 -- 172.21.15.133:0/2076556069 reap_dead start 2024-04-25T10:13:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.392+0000 7fe2b5948640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe29c0039c0 0x7fe29c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:59.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b5147640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe29c004810 0x7fe29c004c90 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009ac0 tx=0x7fe2a001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d3c2b441dab7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00b2070 con 0x7fe29c004810 2024-04-25T10:13:59.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a00a5530 con 0x7fe29c004810 2024-04-25T10:13:59.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b4946640 1 -- 172.21.15.133:0/2076556069 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0009690 con 0x7fe29c004810 2024-04-25T10:13:59.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b7bac640 1 -- 172.21.15.133:0/2076556069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe29c004810 msgr2=0x7fe29c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b7bac640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe29c004810 0x7fe29c004c90 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009ac0 tx=0x7fe2a001ca50 comp rx=0 tx=0).stop 2024-04-25T10:13:59.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b7bac640 1 -- 172.21.15.133:0/2076556069 shutdown_connections 2024-04-25T10:13:59.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b7bac640 1 --2- 172.21.15.133:0/2076556069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe29c004810 0x7fe29c004c90 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b7bac640 1 -- 172.21.15.133:0/2076556069 >> 172.21.15.133:0/2076556069 conn(0x7fe2b0067000 msgr2=0x7fe2b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b7bac640 1 -- 172.21.15.133:0/2076556069 shutdown_connections 2024-04-25T10:13:59.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b7bac640 1 -- 172.21.15.133:0/2076556069 wait complete. 2024-04-25T10:13:59.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b7bac640 1 Processor -- start 2024-04-25T10:13:59.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.396+0000 7fe2b7bac640 1 -- start start 2024-04-25T10:13:59.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b7bac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe29c004810 0x7fe2b015f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b7bac640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b0145bd0 0x7fe2b0146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b7bac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b0158560 0x7fe2b01589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b7bac640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b006dee0 con 0x7fe2b0158560 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b7bac640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b006dd60 con 0x7fe2b0145bd0 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b7bac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b006e060 con 0x7fe29c004810 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b6149640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b0158560 0x7fe2b01589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b6149640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b0158560 0x7fe2b01589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57032/0 (socket says 172.21.15.133:57032) 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b6149640 1 -- 172.21.15.133:0/512640462 learned_addr learned my addr 172.21.15.133:0/512640462 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b5948640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe29c004810 0x7fe2b015f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b5147640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b0145bd0 0x7fe2b0146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b6149640 1 -- 172.21.15.133:0/512640462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe29c004810 msgr2=0x7fe2b015f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b6149640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe29c004810 0x7fe2b015f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b6149640 1 -- 172.21.15.133:0/512640462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b0145bd0 msgr2=0x7fe2b0146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b6149640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b0145bd0 0x7fe2b0146030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b6149640 1 -- 172.21.15.133:0/512640462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b01468a0 con 0x7fe2b0158560 2024-04-25T10:13:59.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b5147640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b0145bd0 0x7fe2b0146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:59.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b6149640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b0158560 0x7fe2b01589e0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fe2b006f4d0 tx=0x7fe2a8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2a67fc640 1 -- 172.21.15.133:0/512640462 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8010040 con 0x7fe2b0158560 2024-04-25T10:13:59.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2a67fc640 1 -- 172.21.15.133:0/512640462 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a801a070 con 0x7fe2b0158560 2024-04-25T10:13:59.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b7bac640 1 -- 172.21.15.133:0/512640462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b0146b40 con 0x7fe2b0158560 2024-04-25T10:13:59.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2a67fc640 1 -- 172.21.15.133:0/512640462 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a800fa20 con 0x7fe2b0158560 2024-04-25T10:13:59.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b7bac640 1 -- 172.21.15.133:0/512640462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b0147100 con 0x7fe2b0158560 2024-04-25T10:13:59.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b7bac640 1 -- 172.21.15.133:0/512640462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b0068ef0 con 0x7fe2b0158560 2024-04-25T10:13:59.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2a67fc640 1 -- 172.21.15.133:0/512640462 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2a800fbc0 con 0x7fe2b0158560 2024-04-25T10:13:59.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2a67fc640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe294066f90 0x7fe294069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2a67fc640 1 -- 172.21.15.133:0/512640462 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe2a80881b0 con 0x7fe2b0158560 2024-04-25T10:13:59.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b5948640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe294066f90 0x7fe294069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.400+0000 7fe2b5948640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe294066f90 0x7fe294069450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac000c10 tx=0x7fe2ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.404+0000 7fe2a67fc640 1 -- 172.21.15.133:0/512640462 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2a8058f00 con 0x7fe2b0158560 2024-04-25T10:13:59.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.600+0000 7fe2b7bac640 1 -- 172.21.15.133:0/512640462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2b00008e0 con 0x7fe2b0158560 2024-04-25T10:13:59.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.600+0000 7fe2a67fc640 1 -- 172.21.15.133:0/512640462 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe2a8058800 con 0x7fe2b0158560 2024-04-25T10:13:59.604 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:13:59.604 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:13:59.605 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:13:59.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 -- 172.21.15.133:0/512640462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe294066f90 msgr2=0x7fe294069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe294066f90 0x7fe294069450 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac000c10 tx=0x7fe2ac009040 comp rx=0 tx=0).stop 2024-04-25T10:13:59.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 -- 172.21.15.133:0/512640462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b0158560 msgr2=0x7fe2b01589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b0158560 0x7fe2b01589e0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fe2b006f4d0 tx=0x7fe2a8011a50 comp rx=0 tx=0).stop 2024-04-25T10:13:59.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 -- 172.21.15.133:0/512640462 shutdown_connections 2024-04-25T10:13:59.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe294066f90 0x7fe294069450 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b0158560 0x7fe2b01589e0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b0145bd0 0x7fe2b0146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 --2- 172.21.15.133:0/512640462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe29c004810 0x7fe2b015f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 -- 172.21.15.133:0/512640462 >> 172.21.15.133:0/512640462 conn(0x7fe2b0067000 msgr2=0x7fe2b0164cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 -- 172.21.15.133:0/512640462 shutdown_connections 2024-04-25T10:13:59.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:13:59.604+0000 7fe2b7bac640 1 -- 172.21.15.133:0/512640462 wait complete. 2024-04-25T10:13:59.622 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:14:04.623 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:04.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.752+0000 7fcfccaec640 1 Processor -- start 2024-04-25T10:14:04.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfccaec640 1 -- start start 2024-04-25T10:14:04.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfccaec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc815c9a0 0x7fcfc80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfccaec640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcfc80b1f70 0x7fcfc806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfccaec640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc80c61d0 con 0x7fcfc80ce330 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfccaec640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc80c64b0 con 0x7fcfc80b1f70 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfccaec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc806d4f0 con 0x7fcfc815c9a0 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfc6d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcfc80ce330 0x7fcfc815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43368/0 (socket says 172.21.15.133:43368) 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfc6d9d640 1 -- 172.21.15.133:0/1844367457 learned_addr learned my addr 172.21.15.133:0/1844367457 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfc659c640 1 --2- 172.21.15.133:0/1844367457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc815c9a0 0x7fcfc80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfc5d9b640 1 --2- 172.21.15.133:0/1844367457 >> v2:172.21.15.162:3300/0 conn(0x7fcfc80b1f70 0x7fcfc806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfc559a640 1 -- 172.21.15.133:0/1844367457 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087517107 0 0) 0x7fcfc80c61d0 con 0x7fcfc80ce330 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfc559a640 1 -- 172.21.15.133:0/1844367457 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfb0003530 con 0x7fcfc80ce330 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfc659c640 1 -- 172.21.15.133:0/1844367457 >> v2:172.21.15.162:3300/0 conn(0x7fcfc80b1f70 msgr2=0x7fcfc806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfc659c640 1 --2- 172.21.15.133:0/1844367457 >> v2:172.21.15.162:3300/0 conn(0x7fcfc80b1f70 0x7fcfc806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfc559a640 1 -- 172.21.15.133:0/1844367457 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2560107430 0 0) 0x7fcfb0003530 con 0x7fcfc80ce330 2024-04-25T10:14:04.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfc659c640 1 -- 172.21.15.133:0/1844367457 >> v1:172.21.15.133:6789/0 conn(0x7fcfc80ce330 legacy=0x7fcfc815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.756+0000 7fcfc659c640 1 -- 172.21.15.133:0/1844367457 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc806d7d0 con 0x7fcfc815c9a0 2024-04-25T10:14:04.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfc659c640 1 --2- 172.21.15.133:0/1844367457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc815c9a0 0x7fcfc80c5980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80097c0 tx=0x7fcfb801c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58645829e6205b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:04.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfc559a640 1 -- 172.21.15.133:0/1844367457 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80b0070 con 0x7fcfc815c9a0 2024-04-25T10:14:04.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfc559a640 1 -- 172.21.15.133:0/1844367457 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb80ab400 con 0x7fcfc815c9a0 2024-04-25T10:14:04.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfc559a640 1 -- 172.21.15.133:0/1844367457 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0001a00 con 0x7fcfc815c9a0 2024-04-25T10:14:04.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 -- 172.21.15.133:0/1844367457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc815c9a0 msgr2=0x7fcfc80c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 --2- 172.21.15.133:0/1844367457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc815c9a0 0x7fcfc80c5980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80097c0 tx=0x7fcfb801c360 comp rx=0 tx=0).stop 2024-04-25T10:14:04.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 -- 172.21.15.133:0/1844367457 shutdown_connections 2024-04-25T10:14:04.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 --2- 172.21.15.133:0/1844367457 >> v2:172.21.15.162:3300/0 conn(0x7fcfc80b1f70 0x7fcfc806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 --2- 172.21.15.133:0/1844367457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc815c9a0 0x7fcfc80c5980 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 -- 172.21.15.133:0/1844367457 >> 172.21.15.133:0/1844367457 conn(0x7fcfc8067000 msgr2=0x7fcfc806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:04.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 -- 172.21.15.133:0/1844367457 shutdown_connections 2024-04-25T10:14:04.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 -- 172.21.15.133:0/1844367457 wait complete. 2024-04-25T10:14:04.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 Processor -- start 2024-04-25T10:14:04.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 -- start start 2024-04-25T10:14:04.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc80b1f70 0x7fcfc81f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfc80ce330 0x7fcfc81f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfc815c9a0 0x7fcfc80cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc815cd80 con 0x7fcfc815c9a0 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc806e230 con 0x7fcfc80ce330 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfccaec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc815cf00 con 0x7fcfc80b1f70 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfc6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfc815c9a0 0x7fcfc80cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfc6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfc815c9a0 0x7fcfc80cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57038/0 (socket says 172.21.15.133:57038) 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.760+0000 7fcfc6d9d640 1 -- 172.21.15.133:0/78309871 learned_addr learned my addr 172.21.15.133:0/78309871 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc659c640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc80b1f70 0x7fcfc81f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc5d9b640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfc80ce330 0x7fcfc81f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc6d9d640 1 -- 172.21.15.133:0/78309871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc80b1f70 msgr2=0x7fcfc81f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc6d9d640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc80b1f70 0x7fcfc81f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc6d9d640 1 -- 172.21.15.133:0/78309871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfc80ce330 msgr2=0x7fcfc81f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc6d9d640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfc80ce330 0x7fcfc81f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc6d9d640 1 -- 172.21.15.133:0/78309871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80c82c0 con 0x7fcfc815c9a0 2024-04-25T10:14:04.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc5d9b640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfc80ce330 0x7fcfc81f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:04.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc659c640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc80b1f70 0x7fcfc81f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:04.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc6d9d640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfc815c9a0 0x7fcfc80cabd0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000cc90 tx=0x7fcfc000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:04.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfb6ffd640 1 -- 172.21.15.133:0/78309871 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000ce40 con 0x7fcfc815c9a0 2024-04-25T10:14:04.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfb6ffd640 1 -- 172.21.15.133:0/78309871 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc0012400 con 0x7fcfc815c9a0 2024-04-25T10:14:04.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfccaec640 1 -- 172.21.15.133:0/78309871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc80c85a0 con 0x7fcfc815c9a0 2024-04-25T10:14:04.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfb6ffd640 1 -- 172.21.15.133:0/78309871 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00127f0 con 0x7fcfc815c9a0 2024-04-25T10:14:04.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfccaec640 1 -- 172.21.15.133:0/78309871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc80c8b60 con 0x7fcfc815c9a0 2024-04-25T10:14:04.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfccaec640 1 -- 172.21.15.133:0/78309871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc806bc20 con 0x7fcfc815c9a0 2024-04-25T10:14:04.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfb6ffd640 1 -- 172.21.15.133:0/78309871 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfc0012990 con 0x7fcfc815c9a0 2024-04-25T10:14:04.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfb6ffd640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfa8066f90 0x7fcfa8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:04.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfb6ffd640 1 -- 172.21.15.133:0/78309871 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcfc00858e0 con 0x7fcfc815c9a0 2024-04-25T10:14:04.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc659c640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfa8066f90 0x7fcfa8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:04.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.764+0000 7fcfc659c640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfa8066f90 0x7fcfa8069450 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80ad000 tx=0x7fcfb80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.768+0000 7fcfb6ffd640 1 -- 172.21.15.133:0/78309871 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfc0056770 con 0x7fcfc815c9a0 2024-04-25T10:14:04.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.960+0000 7fcfccaec640 1 -- 172.21.15.133:0/78309871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfc8068ef0 con 0x7fcfc815c9a0 2024-04-25T10:14:04.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.964+0000 7fcfb6ffd640 1 -- 172.21.15.133:0/78309871 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fcfc0056070 con 0x7fcfc815c9a0 2024-04-25T10:14:04.965 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:04.965 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:13:13.860376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9495},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9495":{"gid":9495,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/2351700644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2351700644},{"type":"v1","addr":"172.21.15.133:6837","nonce":2351700644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:14:04.966 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 79 2024-04-25T10:14:04.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 -- 172.21.15.133:0/78309871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfa8066f90 msgr2=0x7fcfa8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfa8066f90 0x7fcfa8069450 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80ad000 tx=0x7fcfb80a6000 comp rx=0 tx=0).stop 2024-04-25T10:14:04.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 -- 172.21.15.133:0/78309871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfc815c9a0 msgr2=0x7fcfc80cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:04.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfc815c9a0 0x7fcfc80cabd0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000cc90 tx=0x7fcfc000a430 comp rx=0 tx=0).stop 2024-04-25T10:14:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 -- 172.21.15.133:0/78309871 shutdown_connections 2024-04-25T10:14:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcfa8066f90 0x7fcfa8069450 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfc815c9a0 0x7fcfc80cabd0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcfc80ce330 0x7fcfc81f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 --2- 172.21.15.133:0/78309871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcfc80b1f70 0x7fcfc81f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 -- 172.21.15.133:0/78309871 >> 172.21.15.133:0/78309871 conn(0x7fcfc8067000 msgr2=0x7fcfc8159190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 -- 172.21.15.133:0/78309871 shutdown_connections 2024-04-25T10:14:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:04.968+0000 7fcfccaec640 1 -- 172.21.15.133:0/78309871 wait complete. 2024-04-25T10:14:04.987 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:14:09.988 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:10.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07d9167640 1 Processor -- start 2024-04-25T10:14:10.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07d9167640 1 -- start start 2024-04-25T10:14:10.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07d9167640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07d40d08e0 0x7f07d40d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07d9167640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f07d40d22c0 0x7f07d40d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07d9167640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d40d1b30 con 0x7f07d415dc80 2024-04-25T10:14:10.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07d9167640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d40b95f0 con 0x7f07d40d08e0 2024-04-25T10:14:10.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07d9167640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d415e060 con 0x7f07d40d22c0 2024-04-25T10:14:10.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07cbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07d40d08e0 0x7f07d40d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07cbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07d40d08e0 0x7f07d40d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55812/0 (socket says 172.21.15.133:55812) 2024-04-25T10:14:10.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07cbfff640 1 -- 172.21.15.133:0/321219808 learned_addr learned my addr 172.21.15.133:0/321219808 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:10.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07d359e640 1 --2- 172.21.15.133:0/321219808 >> v2:172.21.15.162:3300/0 conn(0x7f07d40d22c0 0x7f07d40d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07cbfff640 1 -- 172.21.15.133:0/321219808 >> v2:172.21.15.162:3300/0 conn(0x7f07d40d22c0 msgr2=0x7f07d40d2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07cbfff640 1 --2- 172.21.15.133:0/321219808 >> v2:172.21.15.162:3300/0 conn(0x7f07d40d22c0 0x7f07d40d2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07cbfff640 1 -- 172.21.15.133:0/321219808 >> v1:172.21.15.162:6789/0 conn(0x7f07d415dc80 legacy=0x7f07d40ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.124+0000 7f07cbfff640 1 -- 172.21.15.133:0/321219808 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d40b8560 con 0x7f07d40d08e0 2024-04-25T10:14:10.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07cbfff640 1 --2- 172.21.15.133:0/321219808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d40d08e0 0x7f07d40d0d90 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009870 tx=0x7f07bc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773df0b0aa13e8b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07d259c640 1 -- 172.21.15.133:0/321219808 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0b2070 con 0x7f07d40d08e0 2024-04-25T10:14:10.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07d259c640 1 -- 172.21.15.133:0/321219808 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07bc0ad680 con 0x7f07d40d08e0 2024-04-25T10:14:10.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07d259c640 1 -- 172.21.15.133:0/321219808 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0ada50 con 0x7f07d40d08e0 2024-04-25T10:14:10.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07d9167640 1 -- 172.21.15.133:0/321219808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d40d08e0 msgr2=0x7f07d40d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07d9167640 1 --2- 172.21.15.133:0/321219808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d40d08e0 0x7f07d40d0d90 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009870 tx=0x7f07bc01c490 comp rx=0 tx=0).stop 2024-04-25T10:14:10.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07d9167640 1 -- 172.21.15.133:0/321219808 shutdown_connections 2024-04-25T10:14:10.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07d9167640 1 --2- 172.21.15.133:0/321219808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d40d08e0 0x7f07d40d0d90 secure :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009870 tx=0x7f07bc01c490 comp rx=0 tx=0).stop 2024-04-25T10:14:10.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07d9167640 1 --2- 172.21.15.133:0/321219808 >> v2:172.21.15.162:3300/0 conn(0x7f07d40d22c0 0x7f07d40d2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07d9167640 1 -- 172.21.15.133:0/321219808 >> 172.21.15.133:0/321219808 conn(0x7f07d406c550 msgr2=0x7f07d406c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07d9167640 1 -- 172.21.15.133:0/321219808 shutdown_connections 2024-04-25T10:14:10.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.128+0000 7f07d9167640 1 -- 172.21.15.133:0/321219808 wait complete. 2024-04-25T10:14:10.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07d9167640 1 Processor -- start 2024-04-25T10:14:10.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07d9167640 1 -- start start 2024-04-25T10:14:10.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07d9167640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07d40d22c0 0x7f07d4156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07d9167640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d415dc80 0x7f07d4156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07d9167640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07d4157800 0x7f07d414ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07d9167640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40ceef0 con 0x7f07d415dc80 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07d9167640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40b8e00 con 0x7f07d4157800 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07d9167640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40cf070 con 0x7f07d40d22c0 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d415dc80 0x7f07d4156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d415dc80 0x7f07d4156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55824/0 (socket says 172.21.15.133:55824) 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cbfff640 1 -- 172.21.15.133:0/3739284645 learned_addr learned my addr 172.21.15.133:0/3739284645 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07d2d9d640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07d40d22c0 0x7f07d4156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07d359e640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07d4157800 0x7f07d414ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cbfff640 1 -- 172.21.15.133:0/3739284645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07d40d22c0 msgr2=0x7f07d4156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cbfff640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07d40d22c0 0x7f07d4156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cbfff640 1 -- 172.21.15.133:0/3739284645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07d4157800 msgr2=0x7f07d414ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cbfff640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07d4157800 0x7f07d414ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cbfff640 1 -- 172.21.15.133:0/3739284645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d415e240 con 0x7f07d415dc80 2024-04-25T10:14:10.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cbfff640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d415dc80 0x7f07d4156760 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0b0040 tx=0x7f07bc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cb7fe640 1 -- 172.21.15.133:0/3739284645 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0b2040 con 0x7f07d415dc80 2024-04-25T10:14:10.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cb7fe640 1 -- 172.21.15.133:0/3739284645 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07bc004240 con 0x7f07d415dc80 2024-04-25T10:14:10.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.132+0000 7f07cb7fe640 1 -- 172.21.15.133:0/3739284645 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0ad8b0 con 0x7f07d415dc80 2024-04-25T10:14:10.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.136+0000 7f07d9167640 1 -- 172.21.15.133:0/3739284645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d415ba50 con 0x7f07d415dc80 2024-04-25T10:14:10.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.136+0000 7f07d9167640 1 -- 172.21.15.133:0/3739284645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d4158350 con 0x7f07d415dc80 2024-04-25T10:14:10.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.136+0000 7f07d9167640 1 -- 172.21.15.133:0/3739284645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d4068ef0 con 0x7f07d415dc80 2024-04-25T10:14:10.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.136+0000 7f07cb7fe640 1 -- 172.21.15.133:0/3739284645 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07bc0bf060 con 0x7f07d415dc80 2024-04-25T10:14:10.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.136+0000 7f07cb7fe640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07b4066f90 0x7f07b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.136+0000 7f07cb7fe640 1 -- 172.21.15.133:0/3739284645 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f07bc11e860 con 0x7f07d415dc80 2024-04-25T10:14:10.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.136+0000 7f07d2d9d640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07b4066f90 0x7f07b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.136+0000 7f07d2d9d640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07b4066f90 0x7f07b4069450 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f07c4001650 tx=0x7f07c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.140+0000 7f07cb7fe640 1 -- 172.21.15.133:0/3739284645 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07bc016110 con 0x7f07d415dc80 2024-04-25T10:14:10.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.352+0000 7f07d9167640 1 -- 172.21.15.133:0/3739284645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07d4157e50 con 0x7f07d415dc80 2024-04-25T10:14:10.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.352+0000 7f07cb7fe640 1 -- 172.21.15.133:0/3739284645 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f07bc0ef800 con 0x7f07d415dc80 2024-04-25T10:14:10.355 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:10.355 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:14:09.106064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":19,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:14:10.355 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 83 2024-04-25T10:14:10.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 -- 172.21.15.133:0/3739284645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07b4066f90 msgr2=0x7f07b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07b4066f90 0x7f07b4069450 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f07c4001650 tx=0x7f07c4009040 comp rx=0 tx=0).stop 2024-04-25T10:14:10.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 -- 172.21.15.133:0/3739284645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d415dc80 msgr2=0x7f07d4156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d415dc80 0x7f07d4156760 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0b0040 tx=0x7f07bc01acd0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 -- 172.21.15.133:0/3739284645 shutdown_connections 2024-04-25T10:14:10.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07d4157800 0x7f07d414ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07b4066f90 0x7f07b4069450 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d415dc80 0x7f07d4156760 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 --2- 172.21.15.133:0/3739284645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07d40d22c0 0x7f07d4156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 -- 172.21.15.133:0/3739284645 >> 172.21.15.133:0/3739284645 conn(0x7f07d406c550 msgr2=0x7f07d415c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 -- 172.21.15.133:0/3739284645 shutdown_connections 2024-04-25T10:14:10.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.356+0000 7f07d9167640 1 -- 172.21.15.133:0/3739284645 wait complete. 2024-04-25T10:14:10.374 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-25T10:14:10.374 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:10.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.492+0000 7f6c134a3640 1 Processor -- start 2024-04-25T10:14:10.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.492+0000 7f6c134a3640 1 -- start start 2024-04-25T10:14:10.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c134a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c0c06eb70 0x7f6c0c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c134a3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6c0c1584f0 0x7f6c0c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c134a3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0c0cbe60 con 0x7f6c0c158110 2024-04-25T10:14:10.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c134a3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0c0b19b0 con 0x7f6c0c1584f0 2024-04-25T10:14:10.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c134a3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c15b360 con 0x7f6c0c06eb70 2024-04-25T10:14:10.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c11a40640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6c0c1584f0 0x7f6c0c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c11a40640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6c0c1584f0 0x7f6c0c15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55828/0 (socket says 172.21.15.133:55828) 2024-04-25T10:14:10.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c11a40640 1 -- 172.21.15.133:0/108682611 learned_addr learned my addr 172.21.15.133:0/108682611 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:10.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c10a3e640 1 --2- 172.21.15.133:0/108682611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c0c06eb70 0x7f6c0c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c11a40640 1 -- 172.21.15.133:0/108682611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c0c06eb70 msgr2=0x7f6c0c0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c11a40640 1 --2- 172.21.15.133:0/108682611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c0c06eb70 0x7f6c0c0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c11a40640 1 -- 172.21.15.133:0/108682611 >> v1:172.21.15.162:6789/0 conn(0x7f6c0c158110 legacy=0x7f6c0c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c11a40640 1 -- 172.21.15.133:0/108682611 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c15b640 con 0x7f6c0c1584f0 2024-04-25T10:14:10.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c11a40640 1 --2- 172.21.15.133:0/108682611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c0c1584f0 0x7f6c0c15ad10 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f6c04009e50 tx=0x7f6c0401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abdb7cacfe5655b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c03fff640 1 -- 172.21.15.133:0/108682611 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c040a58f0 con 0x7f6c0c1584f0 2024-04-25T10:14:10.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c03fff640 1 -- 172.21.15.133:0/108682611 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c040a5a90 con 0x7f6c0c1584f0 2024-04-25T10:14:10.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c03fff640 1 -- 172.21.15.133:0/108682611 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c040a5e60 con 0x7f6c0c1584f0 2024-04-25T10:14:10.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c134a3640 1 -- 172.21.15.133:0/108682611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c0c1584f0 msgr2=0x7f6c0c15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.496+0000 7f6c134a3640 1 --2- 172.21.15.133:0/108682611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c0c1584f0 0x7f6c0c15ad10 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f6c04009e50 tx=0x7f6c0401ca00 comp rx=0 tx=0).stop 2024-04-25T10:14:10.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 -- 172.21.15.133:0/108682611 shutdown_connections 2024-04-25T10:14:10.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 --2- 172.21.15.133:0/108682611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c0c1584f0 0x7f6c0c15ad10 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 --2- 172.21.15.133:0/108682611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c0c06eb70 0x7f6c0c0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 -- 172.21.15.133:0/108682611 >> 172.21.15.133:0/108682611 conn(0x7f6c0c067000 msgr2=0x7f6c0c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 -- 172.21.15.133:0/108682611 shutdown_connections 2024-04-25T10:14:10.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 -- 172.21.15.133:0/108682611 wait complete. 2024-04-25T10:14:10.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 Processor -- start 2024-04-25T10:14:10.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 -- start start 2024-04-25T10:14:10.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c0c06eb70 0x7f6c0c150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c0c158110 0x7f6c0c156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0c1584f0 0x7f6c0c1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0ccca0 con 0x7f6c0c158110 2024-04-25T10:14:10.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0ccb20 con 0x7f6c0c1584f0 2024-04-25T10:14:10.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0cce20 con 0x7f6c0c06eb70 2024-04-25T10:14:10.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c10a3e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c0c158110 0x7f6c0c156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c10a3e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c0c158110 0x7f6c0c156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55840/0 (socket says 172.21.15.133:55840) 2024-04-25T10:14:10.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c10a3e640 1 -- 172.21.15.133:0/1697460889 learned_addr learned my addr 172.21.15.133:0/1697460889 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:10.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c1123f640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c0c06eb70 0x7f6c0c150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c11a40640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0c1584f0 0x7f6c0c1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c10a3e640 1 -- 172.21.15.133:0/1697460889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c0c06eb70 msgr2=0x7f6c0c150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c10a3e640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c0c06eb70 0x7f6c0c150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c10a3e640 1 -- 172.21.15.133:0/1697460889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0c1584f0 msgr2=0x7f6c0c1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c10a3e640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0c1584f0 0x7f6c0c1566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c10a3e640 1 -- 172.21.15.133:0/1697460889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c0b4130 con 0x7f6c0c158110 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c11a40640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0c1584f0 0x7f6c0c1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c10a3e640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c0c158110 0x7f6c0c156190 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00e960 tx=0x7f6bfc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c01ffb640 1 -- 172.21.15.133:0/1697460889 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc017070 con 0x7f6c0c158110 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c01ffb640 1 -- 172.21.15.133:0/1697460889 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc00ca00 con 0x7f6c0c158110 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 -- 172.21.15.133:0/1697460889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c0c6d00 con 0x7f6c0c158110 2024-04-25T10:14:10.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c01ffb640 1 -- 172.21.15.133:0/1697460889 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00cdb0 con 0x7f6c0c158110 2024-04-25T10:14:10.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 -- 172.21.15.133:0/1697460889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c0c72c0 con 0x7f6c0c158110 2024-04-25T10:14:10.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.500+0000 7f6c134a3640 1 -- 172.21.15.133:0/1697460889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c068ef0 con 0x7f6c0c158110 2024-04-25T10:14:10.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.504+0000 7f6c01ffb640 1 -- 172.21.15.133:0/1697460889 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6bfc01a070 con 0x7f6c0c158110 2024-04-25T10:14:10.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.504+0000 7f6c01ffb640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6bf0066f60 0x7f6bf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.504+0000 7f6c01ffb640 1 -- 172.21.15.133:0/1697460889 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6bfc085590 con 0x7f6c0c158110 2024-04-25T10:14:10.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.504+0000 7f6c1123f640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6bf0066f60 0x7f6bf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.504+0000 7f6c1123f640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6bf0066f60 0x7f6bf0069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6c08001650 tx=0x7f6c08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.508+0000 7f6c01ffb640 1 -- 172.21.15.133:0/1697460889 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6bfc056450 con 0x7f6c0c158110 2024-04-25T10:14:10.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.700+0000 7f6c134a3640 1 -- 172.21.15.133:0/1697460889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c0c0c57e0 con 0x7f6c0c158110 2024-04-25T10:14:10.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c01ffb640 1 -- 172.21.15.133:0/1697460889 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f6bfc055d50 con 0x7f6c0c158110 2024-04-25T10:14:10.705 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:10.705 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:14:09.106064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":19,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:14:10.705 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 83 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 -- 172.21.15.133:0/1697460889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6bf0066f60 msgr2=0x7f6bf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6bf0066f60 0x7f6bf0069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6c08001650 tx=0x7f6c08009040 comp rx=0 tx=0).stop 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 -- 172.21.15.133:0/1697460889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c0c158110 msgr2=0x7f6c0c156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c0c158110 0x7f6c0c156190 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00e960 tx=0x7f6bfc00ee20 comp rx=0 tx=0).stop 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 -- 172.21.15.133:0/1697460889 shutdown_connections 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c0c1584f0 0x7f6c0c1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6bf0066f60 0x7f6bf0069420 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c0c158110 0x7f6c0c156190 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 --2- 172.21.15.133:0/1697460889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6c0c06eb70 0x7f6c0c150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 -- 172.21.15.133:0/1697460889 >> 172.21.15.133:0/1697460889 conn(0x7f6c0c067000 msgr2=0x7f6c0c0ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 -- 172.21.15.133:0/1697460889 shutdown_connections 2024-04-25T10:14:10.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.704+0000 7f6c134a3640 1 -- 172.21.15.133:0/1697460889 wait complete. 2024-04-25T10:14:10.724 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:14:10.725 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:14:10.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bf94f640 1 Processor -- start 2024-04-25T10:14:10.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bf94f640 1 -- start start 2024-04-25T10:14:10.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bf94f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f34b8158110 0x7f34b80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bf94f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b80cbe60 con 0x7f34b806eb70 2024-04-25T10:14:10.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bf94f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b80b19b0 con 0x7f34b81584f0 2024-04-25T10:14:10.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bf94f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b815b360 con 0x7f34b8158110 2024-04-25T10:14:10.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bd6eb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f34b8158110 0x7f34b80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bdeec640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f34b806eb70 0x7f34b80cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43916/0 (socket says 172.21.15.133:43916) 2024-04-25T10:14:10.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bdeec640 1 -- 172.21.15.133:0/2667685636 learned_addr learned my addr 172.21.15.133:0/2667685636 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:10.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34affff640 1 -- 172.21.15.133:0/2667685636 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040515975 0 0) 0x7f34b80cbe60 con 0x7f34b806eb70 2024-04-25T10:14:10.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34affff640 1 -- 172.21.15.133:0/2667685636 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34a4003530 con 0x7f34b806eb70 2024-04-25T10:14:10.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bd6eb640 1 -- 172.21.15.133:0/2667685636 >> v1:172.21.15.162:6789/0 conn(0x7f34b81584f0 legacy=0x7f34b806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:10.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bd6eb640 1 -- 172.21.15.133:0/2667685636 >> v1:172.21.15.133:6789/0 conn(0x7f34b806eb70 legacy=0x7f34b80cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.836+0000 7f34bd6eb640 1 -- 172.21.15.133:0/2667685636 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b815b640 con 0x7f34b8158110 2024-04-25T10:14:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34bd6eb640 1 --2- 172.21.15.133:0/2667685636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34b8158110 0x7f34b80b1f00 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f34a80097c0 tx=0x7f34a801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bbb90405f2f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34affff640 1 -- 172.21.15.133:0/2667685636 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a80b0070 con 0x7f34b8158110 2024-04-25T10:14:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34affff640 1 -- 172.21.15.133:0/2667685636 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a80ab600 con 0x7f34b8158110 2024-04-25T10:14:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34affff640 1 -- 172.21.15.133:0/2667685636 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a80ab9d0 con 0x7f34b8158110 2024-04-25T10:14:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34bf94f640 1 -- 172.21.15.133:0/2667685636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34b8158110 msgr2=0x7f34b80b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34bf94f640 1 --2- 172.21.15.133:0/2667685636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34b8158110 0x7f34b80b1f00 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f34a80097c0 tx=0x7f34a801c430 comp rx=0 tx=0).stop 2024-04-25T10:14:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34bf94f640 1 -- 172.21.15.133:0/2667685636 shutdown_connections 2024-04-25T10:14:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34bf94f640 1 --2- 172.21.15.133:0/2667685636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34b8158110 0x7f34b80b1f00 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34bf94f640 1 -- 172.21.15.133:0/2667685636 >> 172.21.15.133:0/2667685636 conn(0x7f34b8067000 msgr2=0x7f34b806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34bf94f640 1 -- 172.21.15.133:0/2667685636 shutdown_connections 2024-04-25T10:14:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34bf94f640 1 -- 172.21.15.133:0/2667685636 wait complete. 2024-04-25T10:14:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34bf94f640 1 Processor -- start 2024-04-25T10:14:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.840+0000 7f34bf94f640 1 -- start start 2024-04-25T10:14:10.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bf94f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34b806eb70 0x7f34b8157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bf94f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34b8158110 0x7f34b80cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bf94f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34b81584f0 0x7f34b8155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bf94f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b8168600 con 0x7f34b81584f0 2024-04-25T10:14:10.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bf94f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b8168480 con 0x7f34b806eb70 2024-04-25T10:14:10.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bf94f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b8168780 con 0x7f34b8158110 2024-04-25T10:14:10.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bdeec640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34b81584f0 0x7f34b8155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bdeec640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34b81584f0 0x7f34b8155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55860/0 (socket says 172.21.15.133:55860) 2024-04-25T10:14:10.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bdeec640 1 -- 172.21.15.133:0/4184992317 learned_addr learned my addr 172.21.15.133:0/4184992317 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:10.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bdeec640 1 -- 172.21.15.133:0/4184992317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34b8158110 msgr2=0x7f34b80cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bceea640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34b8158110 0x7f34b80cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bd6eb640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34b806eb70 0x7f34b8157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bdeec640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34b8158110 0x7f34b80cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bdeec640 1 -- 172.21.15.133:0/4184992317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34b806eb70 msgr2=0x7f34b8157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:10.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bdeec640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34b806eb70 0x7f34b8157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:10.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bdeec640 1 -- 172.21.15.133:0/4184992317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b80b24a0 con 0x7f34b81584f0 2024-04-25T10:14:10.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bceea640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34b8158110 0x7f34b80cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:10.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bdeec640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34b81584f0 0x7f34b8155be0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f34b000cb10 tx=0x7f34b000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34adffb640 1 -- 172.21.15.133:0/4184992317 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0017070 con 0x7f34b81584f0 2024-04-25T10:14:10.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34adffb640 1 -- 172.21.15.133:0/4184992317 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b000ac70 con 0x7f34b81584f0 2024-04-25T10:14:10.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bf94f640 1 -- 172.21.15.133:0/4184992317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b8156430 con 0x7f34b81584f0 2024-04-25T10:14:10.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34adffb640 1 -- 172.21.15.133:0/4184992317 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b000ae60 con 0x7f34b81584f0 2024-04-25T10:14:10.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bf94f640 1 -- 172.21.15.133:0/4184992317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b80b2a20 con 0x7f34b81584f0 2024-04-25T10:14:10.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bf94f640 1 -- 172.21.15.133:0/4184992317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b806bc20 con 0x7f34b81584f0 2024-04-25T10:14:10.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34adffb640 1 -- 172.21.15.133:0/4184992317 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34b001a070 con 0x7f34b81584f0 2024-04-25T10:14:10.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34adffb640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f349c066f60 0x7f349c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:10.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34bd6eb640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f349c066f60 0x7f349c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:10.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.844+0000 7f34adffb640 1 -- 172.21.15.133:0/4184992317 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f34b0084f40 con 0x7f34b81584f0 2024-04-25T10:14:10.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.848+0000 7f34bd6eb640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f349c066f60 0x7f349c069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f34b80b2470 tx=0x7f34a80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:10.852+0000 7f34adffb640 1 -- 172.21.15.133:0/4184992317 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f34b0055e00 con 0x7f34b81584f0 2024-04-25T10:14:11.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.052+0000 7f34bf94f640 1 -- 172.21.15.133:0/4184992317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f34b81513c0 con 0x7f34b81584f0 2024-04-25T10:14:11.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.052+0000 7f34adffb640 1 -- 172.21.15.133:0/4184992317 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1045 (secure 0 0 0) 0x7f34b0055700 con 0x7f34b81584f0 2024-04-25T10:14:11.055 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:11.055 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":764,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":133,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039298,"num_in_osds":8,"osd_in_since":1714039289,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078283627,"bytes_used":2681217024,"bytes_avail":770412896256,"bytes_total":773094113280,"read_bytes_sec":396342,"write_bytes_sec":204,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":83,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9489}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-04-25T10:14:07.942736+0000","services":{}},"progress_events":{}} 2024-04-25T10:14:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 -- 172.21.15.133:0/4184992317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f349c066f60 msgr2=0x7f349c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f349c066f60 0x7f349c069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f34b80b2470 tx=0x7f34a80a6000 comp rx=0 tx=0).stop 2024-04-25T10:14:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 -- 172.21.15.133:0/4184992317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34b81584f0 msgr2=0x7f34b8155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34b81584f0 0x7f34b8155be0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f34b000cb10 tx=0x7f34b000a060 comp rx=0 tx=0).stop 2024-04-25T10:14:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 -- 172.21.15.133:0/4184992317 shutdown_connections 2024-04-25T10:14:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f349c066f60 0x7f349c069420 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34b81584f0 0x7f34b8155be0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f34b8158110 0x7f34b80cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 --2- 172.21.15.133:0/4184992317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f34b806eb70 0x7f34b8157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 -- 172.21.15.133:0/4184992317 >> 172.21.15.133:0/4184992317 conn(0x7f34b8067000 msgr2=0x7f34b8154810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 -- 172.21.15.133:0/4184992317 shutdown_connections 2024-04-25T10:14:11.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.056+0000 7f34bf94f640 1 -- 172.21.15.133:0/4184992317 wait complete. 2024-04-25T10:14:11.073 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:14:11.073 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:11.118 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:11.156 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:14:11.157 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:14:11.157 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:11.170 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.328+0000 7fceb3c56640 1 Processor -- start 2024-04-25T10:14:11.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.328+0000 7fceb3c56640 1 -- start start 2024-04-25T10:14:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb3c56640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fceac15dd90 0x7fceac0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb3c56640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fceac0b19f0 0x7fceac0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb3c56640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceac0ca9b0 con 0x7fceac15d9b0 2024-04-25T10:14:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb3c56640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceac0cbe30 con 0x7fceac0b19f0 2024-04-25T10:14:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb3c56640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceac0cac90 con 0x7fceac15dd90 2024-04-25T10:14:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb21f3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fceac0b19f0 0x7fceac0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb21f3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fceac0b19f0 0x7fceac0cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55870/0 (socket says 172.21.15.133:55870) 2024-04-25T10:14:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb21f3640 1 -- 172.21.15.133:0/1762531689 learned_addr learned my addr 172.21.15.133:0/1762531689 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb11f1640 1 --2- 172.21.15.133:0/1762531689 >> v2:172.21.15.162:3300/0 conn(0x7fceac15dd90 0x7fceac0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb21f3640 1 -- 172.21.15.133:0/1762531689 >> v2:172.21.15.162:3300/0 conn(0x7fceac15dd90 msgr2=0x7fceac0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb21f3640 1 --2- 172.21.15.133:0/1762531689 >> v2:172.21.15.162:3300/0 conn(0x7fceac15dd90 0x7fceac0ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb21f3640 1 -- 172.21.15.133:0/1762531689 >> v1:172.21.15.162:6789/0 conn(0x7fceac15d9b0 legacy=0x7fceac0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb21f3640 1 -- 172.21.15.133:0/1762531689 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac0caf70 con 0x7fceac0b19f0 2024-04-25T10:14:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb21f3640 1 --2- 172.21.15.133:0/1762531689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fceac0b19f0 0x7fceac0cecb0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fceac0cc000 tx=0x7fcea401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ac850777f1a080 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb09f0640 1 -- 172.21.15.133:0/1762531689 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea40a5800 con 0x7fceac0b19f0 2024-04-25T10:14:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb09f0640 1 -- 172.21.15.133:0/1762531689 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea40a59a0 con 0x7fceac0b19f0 2024-04-25T10:14:11.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.332+0000 7fceb09f0640 1 -- 172.21.15.133:0/1762531689 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea40a5d70 con 0x7fceac0b19f0 2024-04-25T10:14:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 -- 172.21.15.133:0/1762531689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fceac0b19f0 msgr2=0x7fceac0cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 --2- 172.21.15.133:0/1762531689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fceac0b19f0 0x7fceac0cecb0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fceac0cc000 tx=0x7fcea401c920 comp rx=0 tx=0).stop 2024-04-25T10:14:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 -- 172.21.15.133:0/1762531689 shutdown_connections 2024-04-25T10:14:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 --2- 172.21.15.133:0/1762531689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fceac0b19f0 0x7fceac0cecb0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 --2- 172.21.15.133:0/1762531689 >> v2:172.21.15.162:3300/0 conn(0x7fceac15dd90 0x7fceac0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 -- 172.21.15.133:0/1762531689 >> 172.21.15.133:0/1762531689 conn(0x7fceac067000 msgr2=0x7fceac06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 -- 172.21.15.133:0/1762531689 shutdown_connections 2024-04-25T10:14:11.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 -- 172.21.15.133:0/1762531689 wait complete. 2024-04-25T10:14:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 Processor -- start 2024-04-25T10:14:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 -- start start 2024-04-25T10:14:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fceac0b19f0 0x7fceac0cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fceac15d9b0 0x7fceac0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fceac15dd90 0x7fceac150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac06e290 con 0x7fceac15dd90 2024-04-25T10:14:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac06e110 con 0x7fceac15d9b0 2024-04-25T10:14:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceac06e410 con 0x7fceac0b19f0 2024-04-25T10:14:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb21f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fceac15dd90 0x7fceac150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb21f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fceac15dd90 0x7fceac150050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55878/0 (socket says 172.21.15.133:55878) 2024-04-25T10:14:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb21f3640 1 -- 172.21.15.133:0/2506814778 learned_addr learned my addr 172.21.15.133:0/2506814778 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb19f2640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fceac0b19f0 0x7fceac0cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb11f1640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fceac15d9b0 0x7fceac0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb21f3640 1 -- 172.21.15.133:0/2506814778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fceac0b19f0 msgr2=0x7fceac0cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb21f3640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fceac0b19f0 0x7fceac0cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb21f3640 1 -- 172.21.15.133:0/2506814778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fceac15d9b0 msgr2=0x7fceac0cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb21f3640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fceac15d9b0 0x7fceac0cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb21f3640 1 -- 172.21.15.133:0/2506814778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac150d10 con 0x7fceac15dd90 2024-04-25T10:14:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb21f3640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fceac15dd90 0x7fceac150050 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fcea401a550 tx=0x7fcea40196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fcea27fc640 1 -- 172.21.15.133:0/2506814778 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea40bc020 con 0x7fceac15dd90 2024-04-25T10:14:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fcea27fc640 1 -- 172.21.15.133:0/2506814778 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea4007b30 con 0x7fceac15dd90 2024-04-25T10:14:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 -- 172.21.15.133:0/2506814778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceac150ff0 con 0x7fceac15dd90 2024-04-25T10:14:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fcea27fc640 1 -- 172.21.15.133:0/2506814778 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea4009510 con 0x7fceac15dd90 2024-04-25T10:14:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.336+0000 7fceb3c56640 1 -- 172.21.15.133:0/2506814778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceac1515b0 con 0x7fceac15dd90 2024-04-25T10:14:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.340+0000 7fceb3c56640 1 -- 172.21.15.133:0/2506814778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceac068ef0 con 0x7fceac15dd90 2024-04-25T10:14:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.340+0000 7fcea27fc640 1 -- 172.21.15.133:0/2506814778 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcea40bf020 con 0x7fceac15dd90 2024-04-25T10:14:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.340+0000 7fcea27fc640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f60 0x7fce90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.340+0000 7fceb19f2640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f60 0x7fce90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.340+0000 7fcea27fc640 1 -- 172.21.15.133:0/2506814778 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcea4120b70 con 0x7fceac15dd90 2024-04-25T10:14:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.340+0000 7fceb19f2640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f60 0x7fce90069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcea8004e30 tx=0x7fcea8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.344+0000 7fcea27fc640 1 -- 172.21.15.133:0/2506814778 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcea40c6030 con 0x7fceac15dd90 2024-04-25T10:14:11.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.500+0000 7fceb3c56640 1 -- 172.21.15.133:0/2506814778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fceac0b2c00 con 0x7fceac15dd90 2024-04-25T10:14:11.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fcea27fc640 1 -- 172.21.15.133:0/2506814778 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17055 (secure 0 0 0) 0x7fcea40165a0 con 0x7fceac15dd90 2024-04-25T10:14:11.505 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:11.505 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":133,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:06.927526+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:09.128791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:09.816841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/2351700644":"2024-04-26T10:14:06.927420+0000","172.21.15.133:6836/2351700644":"2024-04-26T10:14:06.927420+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:11.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 -- 172.21.15.133:0/2506814778 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f60 msgr2=0x7fce90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f60 0x7fce90069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcea8004e30 tx=0x7fcea8009040 comp rx=0 tx=0).stop 2024-04-25T10:14:11.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 -- 172.21.15.133:0/2506814778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fceac15dd90 msgr2=0x7fceac150050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fceac15dd90 0x7fceac150050 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fcea401a550 tx=0x7fcea40196c0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 -- 172.21.15.133:0/2506814778 shutdown_connections 2024-04-25T10:14:11.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce90066f60 0x7fce90069420 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fceac15dd90 0x7fceac150050 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fceac15d9b0 0x7fceac0cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 --2- 172.21.15.133:0/2506814778 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fceac0b19f0 0x7fceac0cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 -- 172.21.15.133:0/2506814778 >> 172.21.15.133:0/2506814778 conn(0x7fceac067000 msgr2=0x7fceac0711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 -- 172.21.15.133:0/2506814778 shutdown_connections 2024-04-25T10:14:11.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.504+0000 7fceb3c56640 1 -- 172.21.15.133:0/2506814778 wait complete. 2024-04-25T10:14:11.523 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:09.128791+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:09.816841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:11.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:11.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.644+0000 7f65ead8c640 1 Processor -- start 2024-04-25T10:14:11.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.644+0000 7f65ead8c640 1 -- start start 2024-04-25T10:14:11.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65ead8c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f65e4145170 0x7f65e406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65ead8c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65e406e4d0 0x7f65e40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65ead8c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e40ce2e0 con 0x7f65e40cdd60 2024-04-25T10:14:11.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65ead8c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e406db00 con 0x7f65e406e4d0 2024-04-25T10:14:11.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65ead8c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e4145550 con 0x7f65e4145170 2024-04-25T10:14:11.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65e3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65e406e4d0 0x7f65e40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65e3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65e406e4d0 0x7f65e40cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55880/0 (socket says 172.21.15.133:55880) 2024-04-25T10:14:11.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65e3fff640 1 -- 172.21.15.133:0/2982344141 learned_addr learned my addr 172.21.15.133:0/2982344141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:11.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65e8b28640 1 --2- 172.21.15.133:0/2982344141 >> v2:172.21.15.162:3300/0 conn(0x7f65e4145170 0x7f65e406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65e3fff640 1 -- 172.21.15.133:0/2982344141 >> v2:172.21.15.162:3300/0 conn(0x7f65e4145170 msgr2=0x7f65e406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65e3fff640 1 --2- 172.21.15.133:0/2982344141 >> v2:172.21.15.162:3300/0 conn(0x7f65e4145170 0x7f65e406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65e3fff640 1 -- 172.21.15.133:0/2982344141 >> v1:172.21.15.162:6789/0 conn(0x7f65e40cdd60 legacy=0x7f65e415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65e3fff640 1 -- 172.21.15.133:0/2982344141 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e406d4b0 con 0x7f65e406e4d0 2024-04-25T10:14:11.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65e3fff640 1 --2- 172.21.15.133:0/2982344141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e406e4d0 0x7f65e40cd820 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f65e406b790 tx=0x7f65d801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f6f17f20f77d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.648+0000 7f65e37fe640 1 -- 172.21.15.133:0/2982344141 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80b0070 con 0x7f65e406e4d0 2024-04-25T10:14:11.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e37fe640 1 -- 172.21.15.133:0/2982344141 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d80ab600 con 0x7f65e406e4d0 2024-04-25T10:14:11.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e37fe640 1 -- 172.21.15.133:0/2982344141 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80ab9d0 con 0x7f65e406e4d0 2024-04-25T10:14:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 -- 172.21.15.133:0/2982344141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e406e4d0 msgr2=0x7f65e40cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 --2- 172.21.15.133:0/2982344141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e406e4d0 0x7f65e40cd820 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f65e406b790 tx=0x7f65d801c490 comp rx=0 tx=0).stop 2024-04-25T10:14:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 -- 172.21.15.133:0/2982344141 shutdown_connections 2024-04-25T10:14:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 --2- 172.21.15.133:0/2982344141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e406e4d0 0x7f65e40cd820 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 --2- 172.21.15.133:0/2982344141 >> v2:172.21.15.162:3300/0 conn(0x7f65e4145170 0x7f65e406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 -- 172.21.15.133:0/2982344141 >> 172.21.15.133:0/2982344141 conn(0x7f65e4066f70 msgr2=0x7f65e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 -- 172.21.15.133:0/2982344141 shutdown_connections 2024-04-25T10:14:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 -- 172.21.15.133:0/2982344141 wait complete. 2024-04-25T10:14:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 Processor -- start 2024-04-25T10:14:11.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 -- start start 2024-04-25T10:14:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e406e4d0 0x7f65e40ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65e40cdd60 0x7f65e40b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65e4145170 0x7f65e40acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4168490 con 0x7f65e406e4d0 2024-04-25T10:14:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e415d7f0 con 0x7f65e40cdd60 2024-04-25T10:14:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4168610 con 0x7f65e4145170 2024-04-25T10:14:11.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e8b28640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e406e4d0 0x7f65e40ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e8b28640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e406e4d0 0x7f65e40ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55896/0 (socket says 172.21.15.133:55896) 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e8b28640 1 -- 172.21.15.133:0/622681447 learned_addr learned my addr 172.21.15.133:0/622681447 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e3fff640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65e40cdd60 0x7f65e40b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e8b28640 1 -- 172.21.15.133:0/622681447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65e4145170 msgr2=0x7f65e40acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e9329640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65e4145170 0x7f65e40acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e8b28640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65e4145170 0x7f65e40acc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e8b28640 1 -- 172.21.15.133:0/622681447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65e40cdd60 msgr2=0x7f65e40b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e8b28640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65e40cdd60 0x7f65e40b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e8b28640 1 -- 172.21.15.133:0/622681447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40aff60 con 0x7f65e406e4d0 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e3fff640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65e40cdd60 0x7f65e40b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e8b28640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e406e4d0 0x7f65e40ae0d0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f65d400d830 tx=0x7f65d400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e17fa640 1 -- 172.21.15.133:0/622681447 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400b6c0 con 0x7f65e406e4d0 2024-04-25T10:14:11.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e17fa640 1 -- 172.21.15.133:0/622681447 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d4019070 con 0x7f65e406e4d0 2024-04-25T10:14:11.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 -- 172.21.15.133:0/622681447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e40b0240 con 0x7f65e406e4d0 2024-04-25T10:14:11.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65e17fa640 1 -- 172.21.15.133:0/622681447 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400bae0 con 0x7f65e406e4d0 2024-04-25T10:14:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.652+0000 7f65ead8c640 1 -- 172.21.15.133:0/622681447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e40b0800 con 0x7f65e406e4d0 2024-04-25T10:14:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.656+0000 7f65ead8c640 1 -- 172.21.15.133:0/622681447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e4068e80 con 0x7f65e406e4d0 2024-04-25T10:14:11.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.656+0000 7f65e17fa640 1 -- 172.21.15.133:0/622681447 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65d401e460 con 0x7f65e406e4d0 2024-04-25T10:14:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.656+0000 7f65e17fa640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65d0066f60 0x7f65d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.656+0000 7f65e17fa640 1 -- 172.21.15.133:0/622681447 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f65d40858c0 con 0x7f65e406e4d0 2024-04-25T10:14:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.656+0000 7f65e3fff640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65d0066f60 0x7f65d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.656+0000 7f65e3fff640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65d0066f60 0x7f65d0069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f65d80ad000 tx=0x7f65d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.660+0000 7f65e17fa640 1 -- 172.21.15.133:0/622681447 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65d4056780 con 0x7f65e406e4d0 2024-04-25T10:14:11.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.804+0000 7f65ead8c640 1 -- 172.21.15.133:0/622681447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65e415e360 con 0x7f65e406e4d0 2024-04-25T10:14:11.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65e17fa640 1 -- 172.21.15.133:0/622681447 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65d4056080 con 0x7f65e406e4d0 2024-04-25T10:14:11.809 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:14:11.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 -- 172.21.15.133:0/622681447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65d0066f60 msgr2=0x7f65d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65d0066f60 0x7f65d0069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f65d80ad000 tx=0x7f65d8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:11.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 -- 172.21.15.133:0/622681447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e406e4d0 msgr2=0x7f65e40ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e406e4d0 0x7f65e40ae0d0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f65d400d830 tx=0x7f65d400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 -- 172.21.15.133:0/622681447 shutdown_connections 2024-04-25T10:14:11.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f65e4145170 0x7f65e40acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65e40cdd60 0x7f65e40b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f65d0066f60 0x7f65d0069420 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 --2- 172.21.15.133:0/622681447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65e406e4d0 0x7f65e40ae0d0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 -- 172.21.15.133:0/622681447 >> 172.21.15.133:0/622681447 conn(0x7f65e4066f70 msgr2=0x7f65e4153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 -- 172.21.15.133:0/622681447 shutdown_connections 2024-04-25T10:14:11.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.808+0000 7f65ead8c640 1 -- 172.21.15.133:0/622681447 wait complete. 2024-04-25T10:14:11.826 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b67eb640 1 Processor -- start 2024-04-25T10:14:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b67eb640 1 -- start start 2024-04-25T10:14:11.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b67eb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f38b0145170 0x7f38b006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b67eb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f38b015ced0 0x7f38b015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b67eb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b00ce2e0 con 0x7f38b006e4d0 2024-04-25T10:14:11.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b67eb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b006db00 con 0x7f38b015ced0 2024-04-25T10:14:11.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b67eb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b0145550 con 0x7f38b0145170 2024-04-25T10:14:11.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b4d88640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f38b015ced0 0x7f38b015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b4d88640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f38b015ced0 0x7f38b015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55900/0 (socket says 172.21.15.133:55900) 2024-04-25T10:14:11.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38a77fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f38b006e4d0 0x7f38b00cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43926/0 (socket says 172.21.15.133:43926) 2024-04-25T10:14:11.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b4d88640 1 -- 172.21.15.133:0/1765938367 learned_addr learned my addr 172.21.15.133:0/1765938367 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:11.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b4d88640 1 -- 172.21.15.133:0/1765938367 >> v2:172.21.15.162:3300/0 conn(0x7f38b0145170 msgr2=0x7f38b006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b4d88640 1 --2- 172.21.15.133:0/1765938367 >> v2:172.21.15.162:3300/0 conn(0x7f38b0145170 0x7f38b006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38af7fe640 1 -- 172.21.15.133:0/1765938367 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511462172 0 0) 0x7f38b00ce2e0 con 0x7f38b006e4d0 2024-04-25T10:14:11.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b4d88640 1 -- 172.21.15.133:0/1765938367 >> v1:172.21.15.133:6789/0 conn(0x7f38b006e4d0 legacy=0x7f38b00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.944+0000 7f38b4d88640 1 -- 172.21.15.133:0/1765938367 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b006d4b0 con 0x7f38b015ced0 2024-04-25T10:14:11.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b4d88640 1 --2- 172.21.15.133:0/1765938367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38b015ced0 0x7f38b015d320 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f38b006b790 tx=0x7f38a801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f14355a53391dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38af7fe640 1 -- 172.21.15.133:0/1765938367 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80a56c0 con 0x7f38b015ced0 2024-04-25T10:14:11.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38af7fe640 1 -- 172.21.15.133:0/1765938367 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a80a5860 con 0x7f38b015ced0 2024-04-25T10:14:11.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38af7fe640 1 -- 172.21.15.133:0/1765938367 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80a5c30 con 0x7f38b015ced0 2024-04-25T10:14:11.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 -- 172.21.15.133:0/1765938367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38b015ced0 msgr2=0x7f38b015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 --2- 172.21.15.133:0/1765938367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38b015ced0 0x7f38b015d320 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f38b006b790 tx=0x7f38a801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 -- 172.21.15.133:0/1765938367 shutdown_connections 2024-04-25T10:14:11.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 --2- 172.21.15.133:0/1765938367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38b015ced0 0x7f38b015d320 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 --2- 172.21.15.133:0/1765938367 >> v2:172.21.15.162:3300/0 conn(0x7f38b0145170 0x7f38b006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 -- 172.21.15.133:0/1765938367 >> 172.21.15.133:0/1765938367 conn(0x7f38b0066f70 msgr2=0x7f38b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 -- 172.21.15.133:0/1765938367 shutdown_connections 2024-04-25T10:14:11.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 -- 172.21.15.133:0/1765938367 wait complete. 2024-04-25T10:14:11.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 Processor -- start 2024-04-25T10:14:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 -- start start 2024-04-25T10:14:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38b006e4d0 0x7f38b0153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38b0145170 0x7f38b01f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38b015ced0 0x7f38b006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0168610 con 0x7f38b0145170 2024-04-25T10:14:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0168490 con 0x7f38b006e4d0 2024-04-25T10:14:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38b67eb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0168790 con 0x7f38b015ced0 2024-04-25T10:14:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38b0145170 0x7f38b01f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38b0145170 0x7f38b01f4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55904/0 (socket says 172.21.15.133:55904) 2024-04-25T10:14:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.948+0000 7f38a77fe640 1 -- 172.21.15.133:0/786640040 learned_addr learned my addr 172.21.15.133:0/786640040 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38b4d88640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38b015ced0 0x7f38b006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38a77fe640 1 -- 172.21.15.133:0/786640040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38b015ced0 msgr2=0x7f38b006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38affff640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38b006e4d0 0x7f38b0153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38a77fe640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38b015ced0 0x7f38b006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38a77fe640 1 -- 172.21.15.133:0/786640040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38b006e4d0 msgr2=0x7f38b0153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38a77fe640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38b006e4d0 0x7f38b0153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38a77fe640 1 -- 172.21.15.133:0/786640040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b0154250 con 0x7f38b0145170 2024-04-25T10:14:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38affff640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38b006e4d0 0x7f38b0153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38a77fe640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38b0145170 0x7f38b01f4970 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f389800c450 tx=0x7f389800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38ad7fa640 1 -- 172.21.15.133:0/786640040 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898018070 con 0x7f38b0145170 2024-04-25T10:14:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38ad7fa640 1 -- 172.21.15.133:0/786640040 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38980074e0 con 0x7f38b0145170 2024-04-25T10:14:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38b67eb640 1 -- 172.21.15.133:0/786640040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b01547e0 con 0x7f38b0145170 2024-04-25T10:14:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38ad7fa640 1 -- 172.21.15.133:0/786640040 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38980049f0 con 0x7f38b0145170 2024-04-25T10:14:11.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38b67eb640 1 -- 172.21.15.133:0/786640040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b013d380 con 0x7f38b0145170 2024-04-25T10:14:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38b67eb640 1 -- 172.21.15.133:0/786640040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b0068e80 con 0x7f38b0145170 2024-04-25T10:14:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38ad7fa640 1 -- 172.21.15.133:0/786640040 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f389801b070 con 0x7f38b0145170 2024-04-25T10:14:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38ad7fa640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3890066f60 0x7f3890069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38ad7fa640 1 -- 172.21.15.133:0/786640040 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3898084570 con 0x7f38b0145170 2024-04-25T10:14:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38affff640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3890066f60 0x7f3890069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.952+0000 7f38affff640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3890066f60 0x7f3890069420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f38a000a870 tx=0x7f38a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:11.956+0000 7f38ad7fa640 1 -- 172.21.15.133:0/786640040 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38980553a0 con 0x7f38b0145170 2024-04-25T10:14:12.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.104+0000 7f38b67eb640 1 -- 172.21.15.133:0/786640040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38b0154f70 con 0x7f38b0145170 2024-04-25T10:14:12.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.104+0000 7f38ad7fa640 1 -- 172.21.15.133:0/786640040 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3898054ca0 con 0x7f38b0145170 2024-04-25T10:14:12.107 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:14:12.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 -- 172.21.15.133:0/786640040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3890066f60 msgr2=0x7f3890069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3890066f60 0x7f3890069420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f38a000a870 tx=0x7f38a0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 -- 172.21.15.133:0/786640040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38b0145170 msgr2=0x7f38b01f4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38b0145170 0x7f38b01f4970 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f389800c450 tx=0x7f389800c910 comp rx=0 tx=0).stop 2024-04-25T10:14:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 -- 172.21.15.133:0/786640040 shutdown_connections 2024-04-25T10:14:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f38b015ced0 0x7f38b006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3890066f60 0x7f3890069420 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38b0145170 0x7f38b01f4970 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 --2- 172.21.15.133:0/786640040 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38b006e4d0 0x7f38b0153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 -- 172.21.15.133:0/786640040 >> 172.21.15.133:0/786640040 conn(0x7f38b0066f70 msgr2=0x7f38b015c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 -- 172.21.15.133:0/786640040 shutdown_connections 2024-04-25T10:14:12.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.108+0000 7f38b67eb640 1 -- 172.21.15.133:0/786640040 wait complete. 2024-04-25T10:14:12.124 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:12.143 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-04-25T10:14:12.140+0000 7f9db3a9f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:12.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.232+0000 7fa465b8b640 1 Processor -- start 2024-04-25T10:14:12.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.232+0000 7fa465b8b640 1 -- start start 2024-04-25T10:14:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.232+0000 7fa465b8b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa46015d5e0 0x7fa4600cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.232+0000 7fa465b8b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4600ce2e0 con 0x7fa4600cd820 2024-04-25T10:14:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.232+0000 7fa465b8b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46006d4b0 con 0x7fa460145170 2024-04-25T10:14:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.232+0000 7fa465b8b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46006db00 con 0x7fa46015d5e0 2024-04-25T10:14:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.232+0000 7fa457fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa4600cd820 0x7fa46015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43932/0 (socket says 172.21.15.133:43932) 2024-04-25T10:14:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa457fff640 1 -- 172.21.15.133:0/1889422961 learned_addr learned my addr 172.21.15.133:0/1889422961 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:12.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45ffff640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa46015d5e0 0x7fa4600cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968861284 0 0) 0x7fa4600ce2e0 con 0x7fa4600cd820 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa44c003530 con 0x7fa4600cd820 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4134259341 0 0) 0x7fa44c003530 con 0x7fa4600cd820 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa46015d5e0 msgr2=0x7fa4600cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa46015d5e0 0x7fa4600cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 >> v1:172.21.15.162:6789/0 conn(0x7fa460145170 legacy=0x7fa46006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46006d790 con 0x7fa4600cd820 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa450001680 con 0x7fa4600cd820 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 >> v1:172.21.15.133:6789/0 conn(0x7fa4600cd820 legacy=0x7fa46015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 0x7fa44c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa44c004810 0x7fa44c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa44c0051d0 0x7fa44c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46006db00 con 0x7fa44c0039f0 2024-04-25T10:14:12.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c00bf10 con 0x7fa44c004810 2024-04-25T10:14:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c00c0e0 con 0x7fa44c0051d0 2024-04-25T10:14:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45f7fe640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 0x7fa44c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa457fff640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa44c004810 0x7fa44c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45ffff640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa44c0051d0 0x7fa44c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45f7fe640 1 -- 172.21.15.133:0/1889422961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa44c0051d0 msgr2=0x7fa44c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45f7fe640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa44c0051d0 0x7fa44c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45f7fe640 1 -- 172.21.15.133:0/1889422961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa44c004810 msgr2=0x7fa44c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45f7fe640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa44c004810 0x7fa44c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45f7fe640 1 -- 172.21.15.133:0/1889422961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c003530 con 0x7fa44c0039f0 2024-04-25T10:14:12.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45f7fe640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 0x7fa44c003e10 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fa448001780 tx=0x7fa44801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e6274b26a364cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480013a0 con 0x7fa44c0039f0 2024-04-25T10:14:12.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4480052f0 con 0x7fa44c0039f0 2024-04-25T10:14:12.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45effd640 1 -- 172.21.15.133:0/1889422961 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448003840 con 0x7fa44c0039f0 2024-04-25T10:14:12.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa465b8b640 1 -- 172.21.15.133:0/1889422961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 msgr2=0x7fa44c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa465b8b640 1 --2- 172.21.15.133:0/1889422961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 0x7fa44c003e10 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fa448001780 tx=0x7fa44801c420 comp rx=0 tx=0).stop 2024-04-25T10:14:12.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa45ffff640 1 -- 172.21.15.133:0/1889422961 reap_dead start 2024-04-25T10:14:12.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa465b8b640 1 -- 172.21.15.133:0/1889422961 shutdown_connections 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa465b8b640 1 -- 172.21.15.133:0/1889422961 >> 172.21.15.133:0/1889422961 conn(0x7fa460066f70 msgr2=0x7fa46006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa465b8b640 1 -- 172.21.15.133:0/1889422961 shutdown_connections 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.236+0000 7fa465b8b640 1 -- 172.21.15.133:0/1889422961 wait complete. 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa465b8b640 1 Processor -- start 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa465b8b640 1 -- start start 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa465b8b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 0x7fa4600a9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa465b8b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa460152440 0x7fa4600ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa465b8b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4600ae390 0x7fa460143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa465b8b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa460145550 con 0x7fa44c0039f0 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa465b8b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4600ce030 con 0x7fa4600ae390 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa465b8b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4601676a0 con 0x7fa460152440 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 0x7fa4600a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 0x7fa4600a9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55920/0 (socket says 172.21.15.133:55920) 2024-04-25T10:14:12.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45f7fe640 1 -- 172.21.15.133:0/3514977687 learned_addr learned my addr 172.21.15.133:0/3514977687 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45ffff640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4600ae390 0x7fa460143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa457fff640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa460152440 0x7fa4600ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45f7fe640 1 -- 172.21.15.133:0/3514977687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa460152440 msgr2=0x7fa4600ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45f7fe640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa460152440 0x7fa4600ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45f7fe640 1 -- 172.21.15.133:0/3514977687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4600ae390 msgr2=0x7fa460143c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45f7fe640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4600ae390 0x7fa460143c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45f7fe640 1 -- 172.21.15.133:0/3514977687 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600a98f0 con 0x7fa44c0039f0 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa457fff640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa460152440 0x7fa4600ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45f7fe640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 0x7fa4600a9150 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fa448001780 tx=0x7fa44c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45cff9640 1 -- 172.21.15.133:0/3514977687 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448007a40 con 0x7fa44c0039f0 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45cff9640 1 -- 172.21.15.133:0/3514977687 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4480bb070 con 0x7fa44c0039f0 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa465b8b640 1 -- 172.21.15.133:0/3514977687 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4600a9bd0 con 0x7fa44c0039f0 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45cff9640 1 -- 172.21.15.133:0/3514977687 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480013a0 con 0x7fa44c0039f0 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa465b8b640 1 -- 172.21.15.133:0/3514977687 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4600aa190 con 0x7fa44c0039f0 2024-04-25T10:14:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa465b8b640 1 -- 172.21.15.133:0/3514977687 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa460068e80 con 0x7fa44c0039f0 2024-04-25T10:14:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.240+0000 7fa45cff9640 1 -- 172.21.15.133:0/3514977687 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa4480b4040 con 0x7fa44c0039f0 2024-04-25T10:14:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.244+0000 7fa45cff9640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa440066f60 0x7fa440069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.244+0000 7fa45cff9640 1 -- 172.21.15.133:0/3514977687 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa44811f310 con 0x7fa44c0039f0 2024-04-25T10:14:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.244+0000 7fa457fff640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa440066f60 0x7fa440069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.244+0000 7fa457fff640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa440066f60 0x7fa440069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa450001ec0 tx=0x7fa450001d90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.248+0000 7fa45cff9640 1 -- 172.21.15.133:0/3514977687 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa448016130 con 0x7fa44c0039f0 2024-04-25T10:14:12.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.392+0000 7fa465b8b640 1 -- 172.21.15.133:0/3514977687 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4601545a0 con 0x7fa44c0039f0 2024-04-25T10:14:12.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.392+0000 7fa45cff9640 1 -- 172.21.15.133:0/3514977687 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4480f01e0 con 0x7fa44c0039f0 2024-04-25T10:14:12.396 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:14:12.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 -- 172.21.15.133:0/3514977687 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa440066f60 msgr2=0x7fa440069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa440066f60 0x7fa440069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa450001ec0 tx=0x7fa450001d90 comp rx=0 tx=0).stop 2024-04-25T10:14:12.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 -- 172.21.15.133:0/3514977687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 msgr2=0x7fa4600a9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 0x7fa4600a9150 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fa448001780 tx=0x7fa44c005840 comp rx=0 tx=0).stop 2024-04-25T10:14:12.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 -- 172.21.15.133:0/3514977687 shutdown_connections 2024-04-25T10:14:12.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4600ae390 0x7fa460143c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa460152440 0x7fa4600ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa440066f60 0x7fa440069420 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 --2- 172.21.15.133:0/3514977687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa44c0039f0 0x7fa4600a9150 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 -- 172.21.15.133:0/3514977687 >> 172.21.15.133:0/3514977687 conn(0x7fa460066f70 msgr2=0x7fa46006df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 -- 172.21.15.133:0/3514977687 shutdown_connections 2024-04-25T10:14:12.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.396+0000 7fa465b8b640 1 -- 172.21.15.133:0/3514977687 wait complete. 2024-04-25T10:14:12.414 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:12.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f7580311640 1 Processor -- start 2024-04-25T10:14:12.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f7580311640 1 -- start start 2024-04-25T10:14:12.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f7580311640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7578145170 0x7f757806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f7580311640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f757806e4d0 0x7f75780cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f7580311640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75780ce2e0 con 0x7f75780cdd60 2024-04-25T10:14:12.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f7580311640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757806db00 con 0x7f7578145170 2024-04-25T10:14:12.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f7580311640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7578145550 con 0x7f757806e4d0 2024-04-25T10:14:12.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f757e0ad640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7578145170 0x7f757806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f757e0ad640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7578145170 0x7f757806df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55924/0 (socket says 172.21.15.133:55924) 2024-04-25T10:14:12.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f757e0ad640 1 -- 172.21.15.133:0/714520056 learned_addr learned my addr 172.21.15.133:0/714520056 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:12.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f757d8ac640 1 --2- 172.21.15.133:0/714520056 >> v2:172.21.15.162:3300/0 conn(0x7f757806e4d0 0x7f75780cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f757e0ad640 1 -- 172.21.15.133:0/714520056 >> v2:172.21.15.162:3300/0 conn(0x7f757806e4d0 msgr2=0x7f75780cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f757e0ad640 1 --2- 172.21.15.133:0/714520056 >> v2:172.21.15.162:3300/0 conn(0x7f757806e4d0 0x7f75780cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f757e0ad640 1 -- 172.21.15.133:0/714520056 >> v1:172.21.15.162:6789/0 conn(0x7f75780cdd60 legacy=0x7f757815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f757e0ad640 1 -- 172.21.15.133:0/714520056 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757806d4b0 con 0x7f7578145170 2024-04-25T10:14:12.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.564+0000 7f757d8ac640 1 --2- 172.21.15.133:0/714520056 >> v2:172.21.15.162:3300/0 conn(0x7f757806e4d0 0x7f75780cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:12.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f757e0ad640 1 --2- 172.21.15.133:0/714520056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7578145170 0x7f757806df90 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f75780b4dd0 tx=0x7f757001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3f3030b3ed7241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f757d0ab640 1 -- 172.21.15.133:0/714520056 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700b0070 con 0x7f7578145170 2024-04-25T10:14:12.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f757d0ab640 1 -- 172.21.15.133:0/714520056 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75700ab600 con 0x7f7578145170 2024-04-25T10:14:12.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f757d0ab640 1 -- 172.21.15.133:0/714520056 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700ab9d0 con 0x7f7578145170 2024-04-25T10:14:12.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f7580311640 1 -- 172.21.15.133:0/714520056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7578145170 msgr2=0x7f757806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f7580311640 1 --2- 172.21.15.133:0/714520056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7578145170 0x7f757806df90 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f75780b4dd0 tx=0x7f757001c450 comp rx=0 tx=0).stop 2024-04-25T10:14:12.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f7580311640 1 -- 172.21.15.133:0/714520056 shutdown_connections 2024-04-25T10:14:12.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f7580311640 1 --2- 172.21.15.133:0/714520056 >> v2:172.21.15.162:3300/0 conn(0x7f757806e4d0 0x7f75780cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f7580311640 1 --2- 172.21.15.133:0/714520056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7578145170 0x7f757806df90 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f7580311640 1 -- 172.21.15.133:0/714520056 >> 172.21.15.133:0/714520056 conn(0x7f7578066f70 msgr2=0x7f757806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f7580311640 1 -- 172.21.15.133:0/714520056 shutdown_connections 2024-04-25T10:14:12.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f7580311640 1 -- 172.21.15.133:0/714520056 wait complete. 2024-04-25T10:14:12.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f7580311640 1 Processor -- start 2024-04-25T10:14:12.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.568+0000 7f7580311640 1 -- start start 2024-04-25T10:14:12.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7580311640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f757806e4d0 0x7f75781526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7580311640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cdd60 0x7f7578138330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7580311640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7578145170 0x7f7578143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7580311640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7578168490 con 0x7f75780cdd60 2024-04-25T10:14:12.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7580311640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757815d830 con 0x7f757806e4d0 2024-04-25T10:14:12.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7580311640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7578168610 con 0x7f7578145170 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757d8ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cdd60 0x7f7578138330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757d8ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cdd60 0x7f7578138330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55932/0 (socket says 172.21.15.133:55932) 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757d8ac640 1 -- 172.21.15.133:0/1670616028 learned_addr learned my addr 172.21.15.133:0/1670616028 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757e0ad640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f757806e4d0 0x7f75781526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757e8ae640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7578145170 0x7f7578143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757d8ac640 1 -- 172.21.15.133:0/1670616028 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7578145170 msgr2=0x7f7578143630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757d8ac640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7578145170 0x7f7578143630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757d8ac640 1 -- 172.21.15.133:0/1670616028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f757806e4d0 msgr2=0x7f75781526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757d8ac640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f757806e4d0 0x7f75781526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757d8ac640 1 -- 172.21.15.133:0/1670616028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757813f420 con 0x7f75780cdd60 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757d8ac640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cdd60 0x7f7578138330 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f756c00c7f0 tx=0x7f756c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7576ffd640 1 -- 172.21.15.133:0/1670616028 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c015070 con 0x7f75780cdd60 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7576ffd640 1 -- 172.21.15.133:0/1670616028 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c004220 con 0x7f75780cdd60 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7580311640 1 -- 172.21.15.133:0/1670616028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757813f700 con 0x7f75780cdd60 2024-04-25T10:14:12.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7576ffd640 1 -- 172.21.15.133:0/1670616028 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c003620 con 0x7f75780cdd60 2024-04-25T10:14:12.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7580311640 1 -- 172.21.15.133:0/1670616028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757813fcc0 con 0x7f75780cdd60 2024-04-25T10:14:12.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7580311640 1 -- 172.21.15.133:0/1670616028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7578068e80 con 0x7f75780cdd60 2024-04-25T10:14:12.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7576ffd640 1 -- 172.21.15.133:0/1670616028 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f756c0039d0 con 0x7f75780cdd60 2024-04-25T10:14:12.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7576ffd640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7564066f60 0x7f7564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f7576ffd640 1 -- 172.21.15.133:0/1670616028 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f756c0856b0 con 0x7f75780cdd60 2024-04-25T10:14:12.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757e0ad640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7564066f60 0x7f7564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.572+0000 7f757e0ad640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7564066f60 0x7f7564069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f75700ad000 tx=0x7f75700a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.576+0000 7f7576ffd640 1 -- 172.21.15.133:0/1670616028 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f756c0565c0 con 0x7f75780cdd60 2024-04-25T10:14:12.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 -- 172.21.15.133:0/1670616028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f75781543e0 con 0x7f75780cdd60 2024-04-25T10:14:12.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7576ffd640 1 -- 172.21.15.133:0/1670616028 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v133) v1 ==== 80+0+124 (secure 0 0 0) 0x7f756c055ec0 con 0x7f75780cdd60 2024-04-25T10:14:12.738 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6837/2351700644 2024-04-26T10:14:06.927420+0000 2024-04-25T10:14:12.738 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6836/2351700644 2024-04-26T10:14:06.927420+0000 2024-04-25T10:14:12.738 INFO:teuthology.orchestra.run.smithi133.stderr:listed 2 entries 2024-04-25T10:14:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 -- 172.21.15.133:0/1670616028 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7564066f60 msgr2=0x7f7564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7564066f60 0x7f7564069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f75700ad000 tx=0x7f75700a6000 comp rx=0 tx=0).stop 2024-04-25T10:14:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 -- 172.21.15.133:0/1670616028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cdd60 msgr2=0x7f7578138330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cdd60 0x7f7578138330 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f756c00c7f0 tx=0x7f756c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 -- 172.21.15.133:0/1670616028 shutdown_connections 2024-04-25T10:14:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7578145170 0x7f7578143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7564066f60 0x7f7564069420 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75780cdd60 0x7f7578138330 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 --2- 172.21.15.133:0/1670616028 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f757806e4d0 0x7f75781526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 -- 172.21.15.133:0/1670616028 >> 172.21.15.133:0/1670616028 conn(0x7f7578066f70 msgr2=0x7f757815f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.736+0000 7f7580311640 1 -- 172.21.15.133:0/1670616028 shutdown_connections 2024-04-25T10:14:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:12.740+0000 7f7580311640 1 -- 172.21.15.133:0/1670616028 wait complete. 2024-04-25T10:14:12.756 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:14:12.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:12.757 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:12.768 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[81613]: fuse finished with error 0 and tester_r 0 2024-04-25T10:14:12.769 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:14:12.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:12.769 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:12.884 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:14:12.884 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:12.891 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:12.891 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:14:12.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:12.891 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:14:12.939 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:14:12.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:12.940 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:14:12.940 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:12.946 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:12.946 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:14:12.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:12.946 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:14:12.993 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:14:12.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:12.994 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:14:12.994 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:13.000 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:13.000 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:14:13.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:13.000 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:14:13.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:13.048 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:14:13.048 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:13.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc45749f640 1 Processor -- start 2024-04-25T10:14:13.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc45749f640 1 -- start start 2024-04-25T10:14:13.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc45749f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc45015c9a0 0x7fc4500c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc45749f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4500ce330 0x7fc4500ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc45749f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4500c61d0 con 0x7fc4500b1f70 2024-04-25T10:14:13.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc45749f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4500cecc0 con 0x7fc4500ce330 2024-04-25T10:14:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc45749f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4500c64b0 con 0x7fc45015c9a0 2024-04-25T10:14:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc455a3c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4500ce330 0x7fc4500ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc454a3a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc4500b1f70 0x7fc45006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43940/0 (socket says 172.21.15.133:43940) 2024-04-25T10:14:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc454a3a640 1 -- 172.21.15.133:0/1990779719 learned_addr learned my addr 172.21.15.133:0/1990779719 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc455a3c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc4500ce330 0x7fc4500ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55944/0 (socket says 172.21.15.133:55944) 2024-04-25T10:14:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc45523b640 1 --2- 172.21.15.133:0/1990779719 >> v2:172.21.15.162:3300/0 conn(0x7fc45015c9a0 0x7fc4500c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc455a3c640 1 -- 172.21.15.133:0/1990779719 >> v2:172.21.15.162:3300/0 conn(0x7fc45015c9a0 msgr2=0x7fc4500c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc455a3c640 1 --2- 172.21.15.133:0/1990779719 >> v2:172.21.15.162:3300/0 conn(0x7fc45015c9a0 0x7fc4500c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc455a3c640 1 -- 172.21.15.133:0/1990779719 >> v1:172.21.15.133:6789/0 conn(0x7fc4500b1f70 legacy=0x7fc45006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc455a3c640 1 -- 172.21.15.133:0/1990779719 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45006d4f0 con 0x7fc4500ce330 2024-04-25T10:14:13.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.172+0000 7fc45523b640 1 --2- 172.21.15.133:0/1990779719 >> v2:172.21.15.162:3300/0 conn(0x7fc45015c9a0 0x7fc4500c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:13.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc455a3c640 1 --2- 172.21.15.133:0/1990779719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4500ce330 0x7fc4500ce780 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fc45006f5f0 tx=0x7fc44801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64e488a21c3545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc447fff640 1 -- 172.21.15.133:0/1990779719 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4480b0070 con 0x7fc4500ce330 2024-04-25T10:14:13.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc447fff640 1 -- 172.21.15.133:0/1990779719 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4480ab400 con 0x7fc4500ce330 2024-04-25T10:14:13.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc447fff640 1 -- 172.21.15.133:0/1990779719 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4480ab7d0 con 0x7fc4500ce330 2024-04-25T10:14:13.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 -- 172.21.15.133:0/1990779719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4500ce330 msgr2=0x7fc4500ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 --2- 172.21.15.133:0/1990779719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4500ce330 0x7fc4500ce780 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fc45006f5f0 tx=0x7fc44801c480 comp rx=0 tx=0).stop 2024-04-25T10:14:13.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 -- 172.21.15.133:0/1990779719 shutdown_connections 2024-04-25T10:14:13.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 --2- 172.21.15.133:0/1990779719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4500ce330 0x7fc4500ce780 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 --2- 172.21.15.133:0/1990779719 >> v2:172.21.15.162:3300/0 conn(0x7fc45015c9a0 0x7fc4500c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 -- 172.21.15.133:0/1990779719 >> 172.21.15.133:0/1990779719 conn(0x7fc450067000 msgr2=0x7fc45006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 -- 172.21.15.133:0/1990779719 shutdown_connections 2024-04-25T10:14:13.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 -- 172.21.15.133:0/1990779719 wait complete. 2024-04-25T10:14:13.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 Processor -- start 2024-04-25T10:14:13.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 -- start start 2024-04-25T10:14:13.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4500b1f70 0x7fc45014e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4500ce330 0x7fc45014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc45015c9a0 0x7fc450154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45015dad0 con 0x7fc4500ce330 2024-04-25T10:14:13.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45015d950 con 0x7fc45015c9a0 2024-04-25T10:14:13.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45015dc50 con 0x7fc4500b1f70 2024-04-25T10:14:13.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc454a3a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4500ce330 0x7fc45014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc454a3a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4500ce330 0x7fc45014d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55950/0 (socket says 172.21.15.133:55950) 2024-04-25T10:14:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc454a3a640 1 -- 172.21.15.133:0/3661664620 learned_addr learned my addr 172.21.15.133:0/3661664620 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45523b640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4500b1f70 0x7fc45014e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc455a3c640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc45015c9a0 0x7fc450154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc454a3a640 1 -- 172.21.15.133:0/3661664620 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4500b1f70 msgr2=0x7fc45014e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc454a3a640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4500b1f70 0x7fc45014e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc454a3a640 1 -- 172.21.15.133:0/3661664620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc45015c9a0 msgr2=0x7fc450154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc454a3a640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc45015c9a0 0x7fc450154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc454a3a640 1 -- 172.21.15.133:0/3661664620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4501534f0 con 0x7fc4500ce330 2024-04-25T10:14:13.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45523b640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4500b1f70 0x7fc45014e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc455a3c640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc45015c9a0 0x7fc450154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc454a3a640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4500ce330 0x7fc45014d6b0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fc44000c430 tx=0x7fc44000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc445ffb640 1 -- 172.21.15.133:0/3661664620 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440017070 con 0x7fc4500ce330 2024-04-25T10:14:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc445ffb640 1 -- 172.21.15.133:0/3661664620 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc440005cf0 con 0x7fc4500ce330 2024-04-25T10:14:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 -- 172.21.15.133:0/3661664620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4501537d0 con 0x7fc4500ce330 2024-04-25T10:14:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc445ffb640 1 -- 172.21.15.133:0/3661664620 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400072d0 con 0x7fc4500ce330 2024-04-25T10:14:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.176+0000 7fc45749f640 1 -- 172.21.15.133:0/3661664620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc450153d90 con 0x7fc4500ce330 2024-04-25T10:14:13.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.180+0000 7fc45749f640 1 -- 172.21.15.133:0/3661664620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45006bc20 con 0x7fc4500ce330 2024-04-25T10:14:13.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.180+0000 7fc445ffb640 1 -- 172.21.15.133:0/3661664620 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc44001a070 con 0x7fc4500ce330 2024-04-25T10:14:13.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.180+0000 7fc445ffb640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc434066f90 0x7fc434069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.180+0000 7fc445ffb640 1 -- 172.21.15.133:0/3661664620 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc440084530 con 0x7fc4500ce330 2024-04-25T10:14:13.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.180+0000 7fc45523b640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc434066f90 0x7fc434069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.180+0000 7fc45523b640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc434066f90 0x7fc434069450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00bfd0 tx=0x7fc44c00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.184+0000 7fc445ffb640 1 -- 172.21.15.133:0/3661664620 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc440054c70 con 0x7fc4500ce330 2024-04-25T10:14:13.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.380+0000 7fc45749f640 1 -- 172.21.15.133:0/3661664620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4500ceaa0 con 0x7fc4500ce330 2024-04-25T10:14:13.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.380+0000 7fc445ffb640 1 -- 172.21.15.133:0/3661664620 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4155 (secure 0 0 0) 0x7fc440058d10 con 0x7fc4500ce330 2024-04-25T10:14:13.382 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:13.382 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:14:09.106064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":19,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:14:13.382 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 83 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 -- 172.21.15.133:0/3661664620 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc434066f90 msgr2=0x7fc434069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc434066f90 0x7fc434069450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00bfd0 tx=0x7fc44c00d040 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 -- 172.21.15.133:0/3661664620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4500ce330 msgr2=0x7fc45014d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4500ce330 0x7fc45014d6b0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fc44000c430 tx=0x7fc44000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 -- 172.21.15.133:0/3661664620 shutdown_connections 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc45015c9a0 0x7fc450154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc434066f90 0x7fc434069450 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4500ce330 0x7fc45014d6b0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 --2- 172.21.15.133:0/3661664620 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc4500b1f70 0x7fc45014e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 -- 172.21.15.133:0/3661664620 >> 172.21.15.133:0/3661664620 conn(0x7fc450067000 msgr2=0x7fc4500b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 -- 172.21.15.133:0/3661664620 shutdown_connections 2024-04-25T10:14:13.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.384+0000 7fc45749f640 1 -- 172.21.15.133:0/3661664620 wait complete. 2024-04-25T10:14:13.402 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:13.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.516+0000 7f7565b43640 1 Processor -- start 2024-04-25T10:14:13.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.516+0000 7f7565b43640 1 -- start start 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f7565b43640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75600ce330 0x7f75600ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f7565b43640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75600c61d0 con 0x7f756015c9a0 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f7565b43640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75600c64b0 con 0x7f75600b1f70 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f7565b43640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75600cecc0 con 0x7f75600ce330 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f756015c9a0 0x7f75600c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43956/0 (socket says 172.21.15.133:43956) 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755f7fe640 1 -- 172.21.15.133:0/2985167364 learned_addr learned my addr 172.21.15.133:0/2985167364 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755ffff640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75600ce330 0x7f75600ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915645984 0 0) 0x7f75600c61d0 con 0x7f756015c9a0 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7548003530 con 0x7f756015c9a0 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2163948003 0 0) 0x7f7548003530 con 0x7f756015c9a0 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75600ce330 msgr2=0x7f75600ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75600ce330 0x7f75600ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 >> v1:172.21.15.162:6789/0 conn(0x7f75600b1f70 legacy=0x7f756006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756006d4f0 con 0x7f756015c9a0 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7550001680 con 0x7f756015c9a0 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 >> v1:172.21.15.133:6789/0 conn(0x7f756015c9a0 legacy=0x7f75600c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75600ce330 0x7f75480039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7548003f70 0x7f7548004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7548004d50 0x7f75480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75600cecc0 con 0x7f7548003f70 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754800bab0 con 0x7f7548004d50 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754800bcb0 con 0x7f75600ce330 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755effd640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7548003f70 0x7f7548004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755f7fe640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75600ce330 0x7f75480039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755ffff640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7548004d50 0x7f75480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755effd640 1 -- 172.21.15.133:0/2985167364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75600ce330 msgr2=0x7f75480039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755effd640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75600ce330 0x7f75480039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755effd640 1 -- 172.21.15.133:0/2985167364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7548004d50 msgr2=0x7f75480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755effd640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7548004d50 0x7f75480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.520+0000 7f755effd640 1 -- 172.21.15.133:0/2985167364 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7548003530 con 0x7f7548003f70 2024-04-25T10:14:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755effd640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7548003f70 0x7f7548004810 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f7554001780 tx=0x7f755401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16779c59b3bf657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75540013a0 con 0x7f7548003f70 2024-04-25T10:14:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75540052f0 con 0x7f7548003f70 2024-04-25T10:14:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755e7fc640 1 -- 172.21.15.133:0/2985167364 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554003840 con 0x7f7548003f70 2024-04-25T10:14:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 -- 172.21.15.133:0/2985167364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7548003f70 msgr2=0x7f7548004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 --2- 172.21.15.133:0/2985167364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7548003f70 0x7f7548004810 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f7554001780 tx=0x7f755401c420 comp rx=0 tx=0).stop 2024-04-25T10:14:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755ffff640 1 -- 172.21.15.133:0/2985167364 reap_dead start 2024-04-25T10:14:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 -- 172.21.15.133:0/2985167364 shutdown_connections 2024-04-25T10:14:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 -- 172.21.15.133:0/2985167364 >> 172.21.15.133:0/2985167364 conn(0x7f7560067000 msgr2=0x7f756006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 -- 172.21.15.133:0/2985167364 shutdown_connections 2024-04-25T10:14:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 -- 172.21.15.133:0/2985167364 wait complete. 2024-04-25T10:14:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 Processor -- start 2024-04-25T10:14:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 -- start start 2024-04-25T10:14:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7548003f70 0x7f75600c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75601500a0 0x7f7560145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7560157b80 0x7f7560158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756015cf00 con 0x7f75601500a0 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756015cd80 con 0x7f7560157b80 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f7565b43640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75600ce780 con 0x7f7548003f70 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75601500a0 0x7f7560145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75601500a0 0x7f7560145ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55974/0 (socket says 172.21.15.133:55974) 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755effd640 1 -- 172.21.15.133:0/3014154373 learned_addr learned my addr 172.21.15.133:0/3014154373 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755f7fe640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7548003f70 0x7f75600c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755ffff640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7560157b80 0x7f7560158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755effd640 1 -- 172.21.15.133:0/3014154373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7548003f70 msgr2=0x7f75600c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755effd640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7548003f70 0x7f75600c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755effd640 1 -- 172.21.15.133:0/3014154373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7560157b80 msgr2=0x7f7560158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755effd640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7560157b80 0x7f7560158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755effd640 1 -- 172.21.15.133:0/3014154373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75600c8530 con 0x7f75601500a0 2024-04-25T10:14:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755f7fe640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7548003f70 0x7f75600c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:13.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755ffff640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7560157b80 0x7f7560158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:13.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.524+0000 7f755effd640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75601500a0 0x7f7560145ed0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f7554005040 tx=0x7f7548005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.528+0000 7f7547fff640 1 -- 172.21.15.133:0/3014154373 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554000ee0 con 0x7f75601500a0 2024-04-25T10:14:13.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.528+0000 7f7547fff640 1 -- 172.21.15.133:0/3014154373 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75540bb070 con 0x7f75601500a0 2024-04-25T10:14:13.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.528+0000 7f7565b43640 1 -- 172.21.15.133:0/3014154373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75600c8810 con 0x7f75601500a0 2024-04-25T10:14:13.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.528+0000 7f7547fff640 1 -- 172.21.15.133:0/3014154373 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75540069e0 con 0x7f75601500a0 2024-04-25T10:14:13.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.528+0000 7f7565b43640 1 -- 172.21.15.133:0/3014154373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75600c8220 con 0x7f75601500a0 2024-04-25T10:14:13.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.528+0000 7f7565b43640 1 -- 172.21.15.133:0/3014154373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7560068ef0 con 0x7f75601500a0 2024-04-25T10:14:13.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.528+0000 7f7547fff640 1 -- 172.21.15.133:0/3014154373 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75540b4040 con 0x7f75601500a0 2024-04-25T10:14:13.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.528+0000 7f7547fff640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f754c067100 0x7f754c0695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.528+0000 7f755f7fe640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f754c067100 0x7f754c0695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.528+0000 7f7547fff640 1 -- 172.21.15.133:0/3014154373 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f755411f590 con 0x7f75601500a0 2024-04-25T10:14:13.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.528+0000 7f755f7fe640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f754c067100 0x7f754c0695c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7550005380 tx=0x7f7550009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.532+0000 7f7547fff640 1 -- 172.21.15.133:0/3014154373 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7554016130 con 0x7f75601500a0 2024-04-25T10:14:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.692+0000 7f7565b43640 1 -- 172.21.15.133:0/3014154373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f756006ec70 con 0x7f75601500a0 2024-04-25T10:14:13.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.696+0000 7f7547fff640 1 -- 172.21.15.133:0/3014154373 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17055 (secure 0 0 0) 0x7f75540aa3a0 con 0x7f75601500a0 2024-04-25T10:14:13.699 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:13.699 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":133,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:06.927526+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:09.128791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:09.816841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/2351700644":"2024-04-26T10:14:06.927420+0000","172.21.15.133:6836/2351700644":"2024-04-26T10:14:06.927420+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 -- 172.21.15.133:0/3014154373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f754c067100 msgr2=0x7f754c0695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f754c067100 0x7f754c0695c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7550005380 tx=0x7f7550009040 comp rx=0 tx=0).stop 2024-04-25T10:14:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 -- 172.21.15.133:0/3014154373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75601500a0 msgr2=0x7f7560145ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75601500a0 0x7f7560145ed0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f7554005040 tx=0x7f7548005840 comp rx=0 tx=0).stop 2024-04-25T10:14:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 -- 172.21.15.133:0/3014154373 shutdown_connections 2024-04-25T10:14:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7560157b80 0x7f7560158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f754c067100 0x7f754c0695c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75601500a0 0x7f7560145ed0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 --2- 172.21.15.133:0/3014154373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7548003f70 0x7f75600c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 -- 172.21.15.133:0/3014154373 >> 172.21.15.133:0/3014154373 conn(0x7f7560067000 msgr2=0x7f756015b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 -- 172.21.15.133:0/3014154373 shutdown_connections 2024-04-25T10:14:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.700+0000 7f7565b43640 1 -- 172.21.15.133:0/3014154373 wait complete. 2024-04-25T10:14:13.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:09.128791+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:09.816841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:13.719 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:13.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338f7fd640 1 Processor -- start 2024-04-25T10:14:13.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338f7fd640 1 -- start start 2024-04-25T10:14:13.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338f7fd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3388143ea0 0x7f33880cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338f7fd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f33880b2150 0x7f338806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338f7fd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f338815b9b0 0x7f338806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338f7fd640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338815ac50 con 0x7f338815b9b0 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338f7fd640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33880cd180 con 0x7f33880b2150 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338f7fd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338815aed0 con 0x7f3388143ea0 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338dd9a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f338815b9b0 0x7f338806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338dd9a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f338815b9b0 0x7f338806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55980/0 (socket says 172.21.15.133:55980) 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338dd9a640 1 -- 172.21.15.133:0/1011116006 learned_addr learned my addr 172.21.15.133:0/1011116006 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338d599640 1 --2- 172.21.15.133:0/1011116006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3388143ea0 0x7f33880cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338cd98640 1 --2- 172.21.15.133:0/1011116006 >> v2:172.21.15.162:3300/0 conn(0x7f33880b2150 0x7f338806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338dd9a640 1 -- 172.21.15.133:0/1011116006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3388143ea0 msgr2=0x7f33880cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338dd9a640 1 --2- 172.21.15.133:0/1011116006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3388143ea0 0x7f33880cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338dd9a640 1 -- 172.21.15.133:0/1011116006 >> v2:172.21.15.162:3300/0 conn(0x7f33880b2150 msgr2=0x7f338806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338dd9a640 1 --2- 172.21.15.133:0/1011116006 >> v2:172.21.15.162:3300/0 conn(0x7f33880b2150 0x7f338806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.844+0000 7f338dd9a640 1 -- 172.21.15.133:0/1011116006 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338815b1b0 con 0x7f338815b9b0 2024-04-25T10:14:13.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338dd9a640 1 --2- 172.21.15.133:0/1011116006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f338815b9b0 0x7f338806e8f0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f338806eff0 tx=0x7f338001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0f51647d14b67b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f337ffff640 1 -- 172.21.15.133:0/1011116006 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800a5780 con 0x7f338815b9b0 2024-04-25T10:14:13.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f337ffff640 1 -- 172.21.15.133:0/1011116006 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33800a5920 con 0x7f338815b9b0 2024-04-25T10:14:13.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f337ffff640 1 -- 172.21.15.133:0/1011116006 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800a5cf0 con 0x7f338815b9b0 2024-04-25T10:14:13.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 -- 172.21.15.133:0/1011116006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f338815b9b0 msgr2=0x7f338806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 --2- 172.21.15.133:0/1011116006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f338815b9b0 0x7f338806e8f0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f338806eff0 tx=0x7f338001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 -- 172.21.15.133:0/1011116006 shutdown_connections 2024-04-25T10:14:13.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 --2- 172.21.15.133:0/1011116006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f338815b9b0 0x7f338806e8f0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 --2- 172.21.15.133:0/1011116006 >> v2:172.21.15.162:3300/0 conn(0x7f33880b2150 0x7f338806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 --2- 172.21.15.133:0/1011116006 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3388143ea0 0x7f33880cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 -- 172.21.15.133:0/1011116006 >> 172.21.15.133:0/1011116006 conn(0x7f3388066f70 msgr2=0x7f338806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 -- 172.21.15.133:0/1011116006 shutdown_connections 2024-04-25T10:14:13.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 -- 172.21.15.133:0/1011116006 wait complete. 2024-04-25T10:14:13.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 Processor -- start 2024-04-25T10:14:13.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 -- start start 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33880b2150 0x7f338815d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3388143ea0 0x7f3388132910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f338815b9b0 0x7f33881576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338815ede0 con 0x7f3388143ea0 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338815ec60 con 0x7f33880b2150 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338f7fd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338815ef60 con 0x7f338815b9b0 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338cd98640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3388143ea0 0x7f3388132910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338cd98640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3388143ea0 0x7f3388132910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55990/0 (socket says 172.21.15.133:55990) 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338cd98640 1 -- 172.21.15.133:0/2054182405 learned_addr learned my addr 172.21.15.133:0/2054182405 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338d599640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33880b2150 0x7f338815d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.848+0000 7f338dd9a640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f338815b9b0 0x7f33881576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338cd98640 1 -- 172.21.15.133:0/2054182405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f338815b9b0 msgr2=0x7f33881576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338cd98640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f338815b9b0 0x7f33881576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338cd98640 1 -- 172.21.15.133:0/2054182405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33880b2150 msgr2=0x7f338815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338cd98640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33880b2150 0x7f338815d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338cd98640 1 -- 172.21.15.133:0/2054182405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3388156610 con 0x7f3388143ea0 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338d599640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33880b2150 0x7f338815d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338cd98640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3388143ea0 0x7f3388132910 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f337800d5f0 tx=0x7f337800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f337dffb640 1 -- 172.21.15.133:0/2054182405 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33780125f0 con 0x7f3388143ea0 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f337dffb640 1 -- 172.21.15.133:0/2054182405 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3378019070 con 0x7f3388143ea0 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338f7fd640 1 -- 172.21.15.133:0/2054182405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33881568f0 con 0x7f3388143ea0 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f337dffb640 1 -- 172.21.15.133:0/2054182405 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33780129b0 con 0x7f3388143ea0 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338f7fd640 1 -- 172.21.15.133:0/2054182405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3388156eb0 con 0x7f3388143ea0 2024-04-25T10:14:13.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338f7fd640 1 -- 172.21.15.133:0/2054182405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3388068e80 con 0x7f3388143ea0 2024-04-25T10:14:13.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f337dffb640 1 -- 172.21.15.133:0/2054182405 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3378012b50 con 0x7f3388143ea0 2024-04-25T10:14:13.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f337dffb640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f336c066f60 0x7f336c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338d599640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f336c066f60 0x7f336c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f337dffb640 1 -- 172.21.15.133:0/2054182405 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f33780855a0 con 0x7f3388143ea0 2024-04-25T10:14:13.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.852+0000 7f338d599640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f336c066f60 0x7f336c069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f33840079f0 tx=0x7f3384008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:13.856+0000 7f337dffb640 1 -- 172.21.15.133:0/2054182405 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3378056460 con 0x7f3388143ea0 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.016+0000 7f338f7fd640 1 -- 172.21.15.133:0/2054182405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33880b1e10 con 0x7f3388143ea0 2024-04-25T10:14:14.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f337dffb640 1 -- 172.21.15.133:0/2054182405 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3378055d60 con 0x7f3388143ea0 2024-04-25T10:14:14.021 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:14:14.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 -- 172.21.15.133:0/2054182405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f336c066f60 msgr2=0x7f336c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f336c066f60 0x7f336c069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f33840079f0 tx=0x7f3384008040 comp rx=0 tx=0).stop 2024-04-25T10:14:14.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 -- 172.21.15.133:0/2054182405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3388143ea0 msgr2=0x7f3388132910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3388143ea0 0x7f3388132910 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f337800d5f0 tx=0x7f337800dab0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 -- 172.21.15.133:0/2054182405 shutdown_connections 2024-04-25T10:14:14.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f338815b9b0 0x7f33881576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f336c066f60 0x7f336c069420 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3388143ea0 0x7f3388132910 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 --2- 172.21.15.133:0/2054182405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33880b2150 0x7f338815d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 -- 172.21.15.133:0/2054182405 >> 172.21.15.133:0/2054182405 conn(0x7f3388066f70 msgr2=0x7f3388158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 -- 172.21.15.133:0/2054182405 shutdown_connections 2024-04-25T10:14:14.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.020+0000 7f338f7fd640 1 -- 172.21.15.133:0/2054182405 wait complete. 2024-04-25T10:14:14.040 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:14.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.156+0000 7f01eb803640 1 Processor -- start 2024-04-25T10:14:14.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.156+0000 7f01eb803640 1 -- start start 2024-04-25T10:14:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01eb803640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f01e4145170 0x7f01e406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01eb803640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f01e406e4d0 0x7f01e40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01eb803640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e40ce2e0 con 0x7f01e40cdd60 2024-04-25T10:14:14.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01eb803640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e406db00 con 0x7f01e406e4d0 2024-04-25T10:14:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01eb803640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e4145550 con 0x7f01e4145170 2024-04-25T10:14:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01e8d9e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f01e406e4d0 0x7f01e40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01e8d9e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f01e406e4d0 0x7f01e40cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56000/0 (socket says 172.21.15.133:56000) 2024-04-25T10:14:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01e8d9e640 1 -- 172.21.15.133:0/119460946 learned_addr learned my addr 172.21.15.133:0/119460946 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01e959f640 1 --2- 172.21.15.133:0/119460946 >> v2:172.21.15.162:3300/0 conn(0x7f01e4145170 0x7f01e406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01e8d9e640 1 -- 172.21.15.133:0/119460946 >> v2:172.21.15.162:3300/0 conn(0x7f01e4145170 msgr2=0x7f01e406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01e8d9e640 1 --2- 172.21.15.133:0/119460946 >> v2:172.21.15.162:3300/0 conn(0x7f01e4145170 0x7f01e406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01e8d9e640 1 -- 172.21.15.133:0/119460946 >> v1:172.21.15.133:6789/0 conn(0x7f01e40cdd60 legacy=0x7f01e415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01e8d9e640 1 -- 172.21.15.133:0/119460946 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e406d4b0 con 0x7f01e406e4d0 2024-04-25T10:14:14.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01e959f640 1 --2- 172.21.15.133:0/119460946 >> v2:172.21.15.162:3300/0 conn(0x7f01e4145170 0x7f01e406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:14.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01e8d9e640 1 --2- 172.21.15.133:0/119460946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01e406e4d0 0x7f01e40cd820 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f01e406b790 tx=0x7f01d401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b898bf2705fb00d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01dbfff640 1 -- 172.21.15.133:0/119460946 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40b0070 con 0x7f01e406e4d0 2024-04-25T10:14:14.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01dbfff640 1 -- 172.21.15.133:0/119460946 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d40ab600 con 0x7f01e406e4d0 2024-04-25T10:14:14.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.160+0000 7f01dbfff640 1 -- 172.21.15.133:0/119460946 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40ab9d0 con 0x7f01e406e4d0 2024-04-25T10:14:14.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 -- 172.21.15.133:0/119460946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01e406e4d0 msgr2=0x7f01e40cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 --2- 172.21.15.133:0/119460946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01e406e4d0 0x7f01e40cd820 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f01e406b790 tx=0x7f01d401c490 comp rx=0 tx=0).stop 2024-04-25T10:14:14.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 -- 172.21.15.133:0/119460946 shutdown_connections 2024-04-25T10:14:14.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 --2- 172.21.15.133:0/119460946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01e406e4d0 0x7f01e40cd820 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 --2- 172.21.15.133:0/119460946 >> v2:172.21.15.162:3300/0 conn(0x7f01e4145170 0x7f01e406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 -- 172.21.15.133:0/119460946 >> 172.21.15.133:0/119460946 conn(0x7f01e4066f70 msgr2=0x7f01e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 -- 172.21.15.133:0/119460946 shutdown_connections 2024-04-25T10:14:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 -- 172.21.15.133:0/119460946 wait complete. 2024-04-25T10:14:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 Processor -- start 2024-04-25T10:14:14.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 -- start start 2024-04-25T10:14:14.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01e406e4d0 0x7f01e41fa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01e40cdd60 0x7f01e4142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01e4145170 0x7f01e40b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e4168490 con 0x7f01e406e4d0 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e415d7f0 con 0x7f01e4145170 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e4168610 con 0x7f01e40cdd60 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e959f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01e406e4d0 0x7f01e41fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e959f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01e406e4d0 0x7f01e41fa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56006/0 (socket says 172.21.15.133:56006) 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e959f640 1 -- 172.21.15.133:0/173608646 learned_addr learned my addr 172.21.15.133:0/173608646 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e9da0640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01e4145170 0x7f01e40b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e8d9e640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01e40cdd60 0x7f01e4142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e959f640 1 -- 172.21.15.133:0/173608646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01e40cdd60 msgr2=0x7f01e4142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e959f640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01e40cdd60 0x7f01e4142820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e959f640 1 -- 172.21.15.133:0/173608646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01e4145170 msgr2=0x7f01e40b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e959f640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01e4145170 0x7f01e40b2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e959f640 1 -- 172.21.15.133:0/173608646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e40b3320 con 0x7f01e406e4d0 2024-04-25T10:14:14.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e8d9e640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01e40cdd60 0x7f01e4142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:14.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01e959f640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01e406e4d0 0x7f01e41fa180 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f01e000c7c0 tx=0x7f01e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01d9ffb640 1 -- 172.21.15.133:0/173608646 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0007a30 con 0x7f01e406e4d0 2024-04-25T10:14:14.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01d9ffb640 1 -- 172.21.15.133:0/173608646 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e0007bd0 con 0x7f01e406e4d0 2024-04-25T10:14:14.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 -- 172.21.15.133:0/173608646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e41fa880 con 0x7f01e406e4d0 2024-04-25T10:14:14.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01d9ffb640 1 -- 172.21.15.133:0/173608646 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e00113f0 con 0x7f01e406e4d0 2024-04-25T10:14:14.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.164+0000 7f01eb803640 1 -- 172.21.15.133:0/173608646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e41fae40 con 0x7f01e406e4d0 2024-04-25T10:14:14.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.168+0000 7f01eb803640 1 -- 172.21.15.133:0/173608646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e4068e80 con 0x7f01e406e4d0 2024-04-25T10:14:14.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.168+0000 7f01d9ffb640 1 -- 172.21.15.133:0/173608646 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01e0011590 con 0x7f01e406e4d0 2024-04-25T10:14:14.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.168+0000 7f01d9ffb640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01c8066f60 0x7f01c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.168+0000 7f01d9ffb640 1 -- 172.21.15.133:0/173608646 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f01e0085410 con 0x7f01e406e4d0 2024-04-25T10:14:14.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.168+0000 7f01e8d9e640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01c8066f60 0x7f01c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.168+0000 7f01e8d9e640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01c8066f60 0x7f01c8069420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f01d40ad000 tx=0x7f01d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.172+0000 7f01d9ffb640 1 -- 172.21.15.133:0/173608646 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01e0015020 con 0x7f01e406e4d0 2024-04-25T10:14:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.332+0000 7f01eb803640 1 -- 172.21.15.133:0/173608646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01e41fb080 con 0x7f01e406e4d0 2024-04-25T10:14:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.332+0000 7f01d9ffb640 1 -- 172.21.15.133:0/173608646 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) v1 ==== 104+0+11 (secure 0 0 0) 0x7f01e0056240 con 0x7f01e406e4d0 2024-04-25T10:14:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:14:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 -- 172.21.15.133:0/173608646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01c8066f60 msgr2=0x7f01c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01c8066f60 0x7f01c8069420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f01d40ad000 tx=0x7f01d4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 -- 172.21.15.133:0/173608646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01e406e4d0 msgr2=0x7f01e41fa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01e406e4d0 0x7f01e41fa180 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f01e000c7c0 tx=0x7f01e000cc80 comp rx=0 tx=0).stop 2024-04-25T10:14:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 -- 172.21.15.133:0/173608646 shutdown_connections 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01e4145170 0x7f01e40b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f01e40cdd60 0x7f01e4142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f01c8066f60 0x7f01c8069420 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 --2- 172.21.15.133:0/173608646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01e406e4d0 0x7f01e41fa180 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 -- 172.21.15.133:0/173608646 >> 172.21.15.133:0/173608646 conn(0x7f01e4066f70 msgr2=0x7f01e41531a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 -- 172.21.15.133:0/173608646 shutdown_connections 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.336+0000 7f01eb803640 1 -- 172.21.15.133:0/173608646 wait complete. 2024-04-25T10:14:14.355 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:14.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f4c00c640 1 Processor -- start 2024-04-25T10:14:14.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f4c00c640 1 -- start start 2024-04-25T10:14:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f4c00c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f4406bda0 0x7f2f4415e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f4c00c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4415e750 con 0x7f2f4409a0e0 2024-04-25T10:14:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f4c00c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4415ea30 con 0x7f2f4415dd00 2024-04-25T10:14:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f4c00c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4415ed10 con 0x7f2f4406bda0 2024-04-25T10:14:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f4a5a9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f4406bda0 0x7f2f4415e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f495a7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2f4409a0e0 0x7f2f440cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43972/0 (socket says 172.21.15.133:43972) 2024-04-25T10:14:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f495a7640 1 -- 172.21.15.133:0/1982190418 learned_addr learned my addr 172.21.15.133:0/1982190418 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f4a5a9640 1 -- 172.21.15.133:0/1982190418 >> v1:172.21.15.162:6789/0 conn(0x7f2f4415dd00 legacy=0x7f2f440999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f4a5a9640 1 -- 172.21.15.133:0/1982190418 >> v1:172.21.15.133:6789/0 conn(0x7f2f4409a0e0 legacy=0x7f2f440cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f4a5a9640 1 -- 172.21.15.133:0/1982190418 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4415eff0 con 0x7f2f4406bda0 2024-04-25T10:14:14.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.472+0000 7f2f48da6640 1 -- 172.21.15.133:0/1982190418 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092234739 0 0) 0x7f2f4415e750 con 0x7f2f4409a0e0 2024-04-25T10:14:14.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4a5a9640 1 --2- 172.21.15.133:0/1982190418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4406bda0 0x7f2f4415e0e0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009d90 tx=0x7f2f3c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c24083b92a3810d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f48da6640 1 -- 172.21.15.133:0/1982190418 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c0b2070 con 0x7f2f4406bda0 2024-04-25T10:14:14.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f48da6640 1 -- 172.21.15.133:0/1982190418 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c0a5a20 con 0x7f2f4406bda0 2024-04-25T10:14:14.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f48da6640 1 -- 172.21.15.133:0/1982190418 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c0a5df0 con 0x7f2f4406bda0 2024-04-25T10:14:14.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 -- 172.21.15.133:0/1982190418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4406bda0 msgr2=0x7f2f4415e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 --2- 172.21.15.133:0/1982190418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4406bda0 0x7f2f4415e0e0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009d90 tx=0x7f2f3c01c940 comp rx=0 tx=0).stop 2024-04-25T10:14:14.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 -- 172.21.15.133:0/1982190418 shutdown_connections 2024-04-25T10:14:14.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 --2- 172.21.15.133:0/1982190418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4406bda0 0x7f2f4415e0e0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 -- 172.21.15.133:0/1982190418 >> 172.21.15.133:0/1982190418 conn(0x7f2f440b31a0 msgr2=0x7f2f440b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 -- 172.21.15.133:0/1982190418 shutdown_connections 2024-04-25T10:14:14.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 -- 172.21.15.133:0/1982190418 wait complete. 2024-04-25T10:14:14.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 Processor -- start 2024-04-25T10:14:14.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 -- start start 2024-04-25T10:14:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f4406bda0 0x7f2f4414b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f4409a0e0 0x7f2f440b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4415dd00 0x7f2f44153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4416b9e0 con 0x7f2f4415dd00 2024-04-25T10:14:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440d10e0 con 0x7f2f4406bda0 2024-04-25T10:14:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4c00c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4416bb60 con 0x7f2f4409a0e0 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4a5a9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4415dd00 0x7f2f44153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4a5a9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4415dd00 0x7f2f44153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56016/0 (socket says 172.21.15.133:56016) 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.476+0000 7f2f4a5a9640 1 -- 172.21.15.133:0/382212639 learned_addr learned my addr 172.21.15.133:0/382212639 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f49da8640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f4406bda0 0x7f2f4414b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f495a7640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f4409a0e0 0x7f2f440b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f4a5a9640 1 -- 172.21.15.133:0/382212639 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f4409a0e0 msgr2=0x7f2f440b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f4a5a9640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f4409a0e0 0x7f2f440b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f4a5a9640 1 -- 172.21.15.133:0/382212639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f4406bda0 msgr2=0x7f2f4414b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f4a5a9640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f4406bda0 0x7f2f4414b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f4a5a9640 1 -- 172.21.15.133:0/382212639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f440b4e80 con 0x7f2f4415dd00 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f4a5a9640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4415dd00 0x7f2f44153c40 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c01a570 tx=0x7f2f3c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f3a7fc640 1 -- 172.21.15.133:0/382212639 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c0b2040 con 0x7f2f4415dd00 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f3a7fc640 1 -- 172.21.15.133:0/382212639 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c0bf070 con 0x7f2f4415dd00 2024-04-25T10:14:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f4c00c640 1 -- 172.21.15.133:0/382212639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f440b5160 con 0x7f2f4415dd00 2024-04-25T10:14:14.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f3a7fc640 1 -- 172.21.15.133:0/382212639 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c0adad0 con 0x7f2f4415dd00 2024-04-25T10:14:14.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f4c00c640 1 -- 172.21.15.133:0/382212639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f440b5720 con 0x7f2f4415dd00 2024-04-25T10:14:14.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f4c00c640 1 -- 172.21.15.133:0/382212639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f44068e80 con 0x7f2f4415dd00 2024-04-25T10:14:14.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f3a7fc640 1 -- 172.21.15.133:0/382212639 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f3c0adc70 con 0x7f2f4415dd00 2024-04-25T10:14:14.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f3a7fc640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f28066f60 0x7f2f28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f3a7fc640 1 -- 172.21.15.133:0/382212639 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2f3c1206a0 con 0x7f2f4415dd00 2024-04-25T10:14:14.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f49da8640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f28066f60 0x7f2f28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.480+0000 7f2f49da8640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f28066f60 0x7f2f28069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000f70 tx=0x7f2f40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.484+0000 7f2f3a7fc640 1 -- 172.21.15.133:0/382212639 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f3c016560 con 0x7f2f4415dd00 2024-04-25T10:14:14.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.640+0000 7f2f4c00c640 1 -- 172.21.15.133:0/382212639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f440b7630 con 0x7f2f4415dd00 2024-04-25T10:14:14.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.640+0000 7f2f3a7fc640 1 -- 172.21.15.133:0/382212639 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f3c0ab340 con 0x7f2f4415dd00 2024-04-25T10:14:14.642 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.640+0000 7f2f4c00c640 1 -- 172.21.15.133:0/382212639 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f28066f60 msgr2=0x7f2f28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.640+0000 7f2f4c00c640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f28066f60 0x7f2f28069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000f70 tx=0x7f2f40009040 comp rx=0 tx=0).stop 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.640+0000 7f2f4c00c640 1 -- 172.21.15.133:0/382212639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4415dd00 msgr2=0x7f2f44153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.640+0000 7f2f4c00c640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4415dd00 0x7f2f44153c40 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c01a570 tx=0x7f2f3c0196e0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.640+0000 7f2f4c00c640 1 -- 172.21.15.133:0/382212639 shutdown_connections 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.640+0000 7f2f4c00c640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2f28066f60 0x7f2f28069420 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.644+0000 7f2f4c00c640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f4415dd00 0x7f2f44153c40 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.644+0000 7f2f4c00c640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2f4409a0e0 0x7f2f440b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.644+0000 7f2f4c00c640 1 --2- 172.21.15.133:0/382212639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f4406bda0 0x7f2f4414b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.644+0000 7f2f4c00c640 1 -- 172.21.15.133:0/382212639 >> 172.21.15.133:0/382212639 conn(0x7f2f440b31a0 msgr2=0x7f2f441520c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.644+0000 7f2f4c00c640 1 -- 172.21.15.133:0/382212639 shutdown_connections 2024-04-25T10:14:14.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.644+0000 7f2f4c00c640 1 -- 172.21.15.133:0/382212639 wait complete. 2024-04-25T10:14:14.661 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:14.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.772+0000 7f759bf26640 1 Processor -- start 2024-04-25T10:14:14.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.772+0000 7f759bf26640 1 -- start start 2024-04-25T10:14:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759bf26640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f759415a5f0 0x7f75940c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759bf26640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f759406eb70 0x7f75940cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759bf26640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75940cc5a0 con 0x7f759415a9d0 2024-04-25T10:14:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759bf26640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75940cc880 con 0x7f759406eb70 2024-04-25T10:14:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759bf26640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75940ccb60 con 0x7f759415a5f0 2024-04-25T10:14:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759a4c3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f759406eb70 0x7f75940cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759a4c3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f759406eb70 0x7f75940cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56028/0 (socket says 172.21.15.133:56028) 2024-04-25T10:14:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759a4c3640 1 -- 172.21.15.133:0/3892879999 learned_addr learned my addr 172.21.15.133:0/3892879999 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f7599cc2640 1 --2- 172.21.15.133:0/3892879999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f759415a5f0 0x7f75940c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759a4c3640 1 -- 172.21.15.133:0/3892879999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f759415a5f0 msgr2=0x7f75940c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759a4c3640 1 --2- 172.21.15.133:0/3892879999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f759415a5f0 0x7f75940c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759a4c3640 1 -- 172.21.15.133:0/3892879999 >> v1:172.21.15.162:6789/0 conn(0x7f759415a9d0 legacy=0x7f759406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759a4c3640 1 -- 172.21.15.133:0/3892879999 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75940cce40 con 0x7f759406eb70 2024-04-25T10:14:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f7599cc2640 1 --2- 172.21.15.133:0/3892879999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f759415a5f0 0x7f75940c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:14.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f759a4c3640 1 --2- 172.21.15.133:0/3892879999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759406eb70 0x7f75940cbe50 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f758c009e50 tx=0x7f758c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2062295f97b74bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f7598cc0640 1 -- 172.21.15.133:0/3892879999 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c0a5670 con 0x7f759406eb70 2024-04-25T10:14:14.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f7598cc0640 1 -- 172.21.15.133:0/3892879999 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758c0a5810 con 0x7f759406eb70 2024-04-25T10:14:14.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.776+0000 7f7598cc0640 1 -- 172.21.15.133:0/3892879999 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c0a5be0 con 0x7f759406eb70 2024-04-25T10:14:14.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 -- 172.21.15.133:0/3892879999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759406eb70 msgr2=0x7f75940cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 --2- 172.21.15.133:0/3892879999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759406eb70 0x7f75940cbe50 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f758c009e50 tx=0x7f758c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 -- 172.21.15.133:0/3892879999 shutdown_connections 2024-04-25T10:14:14.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 --2- 172.21.15.133:0/3892879999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759406eb70 0x7f75940cbe50 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 --2- 172.21.15.133:0/3892879999 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f759415a5f0 0x7f75940c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 -- 172.21.15.133:0/3892879999 >> 172.21.15.133:0/3892879999 conn(0x7f7594067000 msgr2=0x7f759406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 -- 172.21.15.133:0/3892879999 shutdown_connections 2024-04-25T10:14:14.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 -- 172.21.15.133:0/3892879999 wait complete. 2024-04-25T10:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 Processor -- start 2024-04-25T10:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 -- start start 2024-04-25T10:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f759406eb70 0x7f759413e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759415a5f0 0x7f7594154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f759415a9d0 0x7f7594154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75940ce430 con 0x7f759415a5f0 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75940ce2b0 con 0x7f759415a9d0 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75940ce5b0 con 0x7f759406eb70 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f75994c1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759415a5f0 0x7f7594154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f75994c1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759415a5f0 0x7f7594154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56042/0 (socket says 172.21.15.133:56042) 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f75994c1640 1 -- 172.21.15.133:0/3006015376 learned_addr learned my addr 172.21.15.133:0/3006015376 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f7599cc2640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f759406eb70 0x7f759413e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759a4c3640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f759415a9d0 0x7f7594154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f75994c1640 1 -- 172.21.15.133:0/3006015376 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f759406eb70 msgr2=0x7f759413e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f75994c1640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f759406eb70 0x7f759413e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f75994c1640 1 -- 172.21.15.133:0/3006015376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f759415a9d0 msgr2=0x7f7594154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f75994c1640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f759415a9d0 0x7f7594154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f75994c1640 1 -- 172.21.15.133:0/3006015376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7594158c80 con 0x7f759415a5f0 2024-04-25T10:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f75994c1640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759415a5f0 0x7f7594154540 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f758400b3e0 tx=0x7f758400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f758a7fc640 1 -- 172.21.15.133:0/3006015376 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584015020 con 0x7f759415a5f0 2024-04-25T10:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f758a7fc640 1 -- 172.21.15.133:0/3006015376 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7584011830 con 0x7f759415a5f0 2024-04-25T10:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 -- 172.21.15.133:0/3006015376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7594158f60 con 0x7f759415a5f0 2024-04-25T10:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f758a7fc640 1 -- 172.21.15.133:0/3006015376 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584011c60 con 0x7f759415a5f0 2024-04-25T10:14:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.780+0000 7f759bf26640 1 -- 172.21.15.133:0/3006015376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7594159520 con 0x7f759415a5f0 2024-04-25T10:14:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.784+0000 7f759bf26640 1 -- 172.21.15.133:0/3006015376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7594068ef0 con 0x7f759415a5f0 2024-04-25T10:14:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.784+0000 7f758a7fc640 1 -- 172.21.15.133:0/3006015376 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7584018020 con 0x7f759415a5f0 2024-04-25T10:14:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.784+0000 7f758a7fc640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7578066f60 0x7f7578069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.784+0000 7f758a7fc640 1 -- 172.21.15.133:0/3006015376 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7584085120 con 0x7f759415a5f0 2024-04-25T10:14:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.784+0000 7f7599cc2640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7578066f60 0x7f7578069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.784+0000 7f7599cc2640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7578066f60 0x7f7578069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f75940b2630 tx=0x7f7590008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.788+0000 7f758a7fc640 1 -- 172.21.15.133:0/3006015376 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f758401f020 con 0x7f759415a5f0 2024-04-25T10:14:14.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.976+0000 7f759bf26640 1 -- 172.21.15.133:0/3006015376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75940caeb0 con 0x7f759415a5f0 2024-04-25T10:14:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.976+0000 7f758a7fc640 1 -- 172.21.15.133:0/3006015376 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f75840172c0 con 0x7f759415a5f0 2024-04-25T10:14:14.979 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:14.980 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:14:09.106064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":19,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:14:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 83 2024-04-25T10:14:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 -- 172.21.15.133:0/3006015376 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7578066f60 msgr2=0x7f7578069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7578066f60 0x7f7578069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f75940b2630 tx=0x7f7590008040 comp rx=0 tx=0).stop 2024-04-25T10:14:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 -- 172.21.15.133:0/3006015376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759415a5f0 msgr2=0x7f7594154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759415a5f0 0x7f7594154540 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f758400b3e0 tx=0x7f758400b8a0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 -- 172.21.15.133:0/3006015376 shutdown_connections 2024-04-25T10:14:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f759415a9d0 0x7f7594154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7578066f60 0x7f7578069420 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759415a5f0 0x7f7594154540 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 --2- 172.21.15.133:0/3006015376 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f759406eb70 0x7f759413e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 -- 172.21.15.133:0/3006015376 >> 172.21.15.133:0/3006015376 conn(0x7f7594067000 msgr2=0x7f75940b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 -- 172.21.15.133:0/3006015376 shutdown_connections 2024-04-25T10:14:14.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:14.980+0000 7f759bf26640 1 -- 172.21.15.133:0/3006015376 wait complete. 2024-04-25T10:14:14.998 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5c430640 1 Processor -- start 2024-04-25T10:14:15.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5c430640 1 -- start start 2024-04-25T10:14:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5c430640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5f54161280 0x7f5f540cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5c430640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5f540b06f0 0x7f5f541618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5c430640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f540b0b30 0x7f5f54161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5c430640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f540cfc60 con 0x7f5f54161280 2024-04-25T10:14:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5c430640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f54161660 con 0x7f5f540b06f0 2024-04-25T10:14:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5c430640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540cf930 con 0x7f5f540b0b30 2024-04-25T10:14:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5a1cc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5f54161280 0x7f5f540cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5a1cc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5f54161280 0x7f5f540cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56046/0 (socket says 172.21.15.133:56046) 2024-04-25T10:14:15.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5a1cc640 1 -- 172.21.15.133:0/3821517780 learned_addr learned my addr 172.21.15.133:0/3821517780 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:15.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5a9cd640 1 --2- 172.21.15.133:0/3821517780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f540b0b30 0x7f5f54161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f599cb640 1 --2- 172.21.15.133:0/3821517780 >> v2:172.21.15.162:3300/0 conn(0x7f5f540b06f0 0x7f5f541618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5a1cc640 1 -- 172.21.15.133:0/3821517780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f540b0b30 msgr2=0x7f5f54161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5a1cc640 1 --2- 172.21.15.133:0/3821517780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f540b0b30 0x7f5f54161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5a1cc640 1 -- 172.21.15.133:0/3821517780 >> v2:172.21.15.162:3300/0 conn(0x7f5f540b06f0 msgr2=0x7f5f541618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5a1cc640 1 --2- 172.21.15.133:0/3821517780 >> v2:172.21.15.162:3300/0 conn(0x7f5f540b06f0 0x7f5f541618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.116+0000 7f5f5a1cc640 1 -- 172.21.15.133:0/3821517780 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5406c930 con 0x7f5f54161280 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5a1cc640 1 --2- 172.21.15.133:0/3821517780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f54161280 0x7f5f540cbd90 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f5f540cdd80 tx=0x7f5f4401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8be2e249898be54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f591ca640 1 -- 172.21.15.133:0/3821517780 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440ab5f0 con 0x7f5f54161280 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f591ca640 1 -- 172.21.15.133:0/3821517780 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f440b0070 con 0x7f5f54161280 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f591ca640 1 -- 172.21.15.133:0/3821517780 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440ab9b0 con 0x7f5f54161280 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 -- 172.21.15.133:0/3821517780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f54161280 msgr2=0x7f5f540cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 --2- 172.21.15.133:0/3821517780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f54161280 0x7f5f540cbd90 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f5f540cdd80 tx=0x7f5f4401c520 comp rx=0 tx=0).stop 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 -- 172.21.15.133:0/3821517780 shutdown_connections 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 --2- 172.21.15.133:0/3821517780 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f540b0b30 0x7f5f54161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 --2- 172.21.15.133:0/3821517780 >> v2:172.21.15.162:3300/0 conn(0x7f5f540b06f0 0x7f5f541618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 --2- 172.21.15.133:0/3821517780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f54161280 0x7f5f540cbd90 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 -- 172.21.15.133:0/3821517780 >> 172.21.15.133:0/3821517780 conn(0x7f5f540b3230 msgr2=0x7f5f540b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 -- 172.21.15.133:0/3821517780 shutdown_connections 2024-04-25T10:14:15.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 -- 172.21.15.133:0/3821517780 wait complete. 2024-04-25T10:14:15.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 Processor -- start 2024-04-25T10:14:15.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 -- start start 2024-04-25T10:14:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f540b06f0 0x7f5f540c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f540b0b30 0x7f5f5406fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f54161280 0x7f5f54154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540cc450 con 0x7f5f54161280 2024-04-25T10:14:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540cc2d0 con 0x7f5f540b06f0 2024-04-25T10:14:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5c430640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540d0900 con 0x7f5f540b0b30 2024-04-25T10:14:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5a9cd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f54161280 0x7f5f54154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5a9cd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f54161280 0x7f5f54154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56060/0 (socket says 172.21.15.133:56060) 2024-04-25T10:14:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5a9cd640 1 -- 172.21.15.133:0/2403656032 learned_addr learned my addr 172.21.15.133:0/2403656032 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:15.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5a1cc640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f540b06f0 0x7f5f540c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f599cb640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f540b0b30 0x7f5f5406fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5a9cd640 1 -- 172.21.15.133:0/2403656032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f540b0b30 msgr2=0x7f5f5406fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5a9cd640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f540b0b30 0x7f5f5406fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5a9cd640 1 -- 172.21.15.133:0/2403656032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f540b06f0 msgr2=0x7f5f540c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5a9cd640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f540b06f0 0x7f5f540c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f5a9cd640 1 -- 172.21.15.133:0/2403656032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f540c53a0 con 0x7f5f54161280 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.120+0000 7f5f599cb640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f540b0b30 0x7f5f5406fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f5a9cd640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f54161280 0x7f5f54154c00 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c007e30 tx=0x7f5f4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f4affd640 1 -- 172.21.15.133:0/2403656032 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c014620 con 0x7f5f54161280 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f4affd640 1 -- 172.21.15.133:0/2403656032 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c019070 con 0x7f5f54161280 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f5c430640 1 -- 172.21.15.133:0/2403656032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f540c5680 con 0x7f5f54161280 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f4affd640 1 -- 172.21.15.133:0/2403656032 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c014a00 con 0x7f5f54161280 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f5c430640 1 -- 172.21.15.133:0/2403656032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f54156ac0 con 0x7f5f54161280 2024-04-25T10:14:15.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f5c430640 1 -- 172.21.15.133:0/2403656032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f54068ef0 con 0x7f5f54161280 2024-04-25T10:14:15.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f4affd640 1 -- 172.21.15.133:0/2403656032 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f4c014ba0 con 0x7f5f54161280 2024-04-25T10:14:15.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f4affd640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f38066f60 0x7f5f38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f4affd640 1 -- 172.21.15.133:0/2403656032 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5f4c085a90 con 0x7f5f54161280 2024-04-25T10:14:15.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f5a1cc640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f38066f60 0x7f5f38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.124+0000 7f5f5a1cc640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f38066f60 0x7f5f38069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5f440af000 tx=0x7f5f440a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.128+0000 7f5f4affd640 1 -- 172.21.15.133:0/2403656032 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f4c056950 con 0x7f5f54161280 2024-04-25T10:14:15.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.276+0000 7f5f5c430640 1 -- 172.21.15.133:0/2403656032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f541572a0 con 0x7f5f54161280 2024-04-25T10:14:15.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f4affd640 1 -- 172.21.15.133:0/2403656032 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17055 (secure 0 0 0) 0x7f5f4c056250 con 0x7f5f54161280 2024-04-25T10:14:15.281 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:15.281 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":133,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:06.927526+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:09.128791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:09.816841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/2351700644":"2024-04-26T10:14:06.927420+0000","172.21.15.133:6836/2351700644":"2024-04-26T10:14:06.927420+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 -- 172.21.15.133:0/2403656032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f38066f60 msgr2=0x7f5f38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f38066f60 0x7f5f38069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5f440af000 tx=0x7f5f440a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 -- 172.21.15.133:0/2403656032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f54161280 msgr2=0x7f5f54154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f54161280 0x7f5f54154c00 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c007e30 tx=0x7f5f4c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 -- 172.21.15.133:0/2403656032 shutdown_connections 2024-04-25T10:14:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5f38066f60 0x7f5f38069420 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f54161280 0x7f5f54154c00 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5f540b0b30 0x7f5f5406fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 --2- 172.21.15.133:0/2403656032 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f540b06f0 0x7f5f540c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 -- 172.21.15.133:0/2403656032 >> 172.21.15.133:0/2403656032 conn(0x7f5f540b3230 msgr2=0x7f5f540ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 -- 172.21.15.133:0/2403656032 shutdown_connections 2024-04-25T10:14:15.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.280+0000 7f5f5c430640 1 -- 172.21.15.133:0/2403656032 wait complete. 2024-04-25T10:14:15.299 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:14:15.299 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:15.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.408+0000 7f7a1b007640 1 Processor -- start 2024-04-25T10:14:15.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.408+0000 7f7a1b007640 1 -- start start 2024-04-25T10:14:15.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a1b007640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7a14158110 0x7f7a140b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a1b007640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a140cbe60 con 0x7f7a1406eb70 2024-04-25T10:14:15.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a1b007640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a140b19b0 con 0x7f7a141584f0 2024-04-25T10:14:15.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a1b007640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a1415b360 con 0x7f7a14158110 2024-04-25T10:14:15.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a195a4640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7a1406eb70 0x7f7a140cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43984/0 (socket says 172.21.15.133:43984) 2024-04-25T10:14:15.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a195a4640 1 -- 172.21.15.133:0/4085017855 learned_addr learned my addr 172.21.15.133:0/4085017855 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:15.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a18da3640 1 --2- 172.21.15.133:0/4085017855 >> v2:172.21.15.162:3300/0 conn(0x7f7a14158110 0x7f7a140b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856071963 0 0) 0x7f7a140cbe60 con 0x7f7a1406eb70 2024-04-25T10:14:15.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a00003530 con 0x7f7a1406eb70 2024-04-25T10:14:15.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3505041306 0 0) 0x7f7a00003530 con 0x7f7a1406eb70 2024-04-25T10:14:15.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 >> v2:172.21.15.162:3300/0 conn(0x7f7a14158110 msgr2=0x7f7a140b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 --2- 172.21.15.133:0/4085017855 >> v2:172.21.15.162:3300/0 conn(0x7f7a14158110 0x7f7a140b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 >> v1:172.21.15.162:6789/0 conn(0x7f7a141584f0 legacy=0x7f7a1406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1415b640 con 0x7f7a1406eb70 2024-04-25T10:14:15.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7a0c001a00 con 0x7f7a1406eb70 2024-04-25T10:14:15.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 >> v1:172.21.15.133:6789/0 conn(0x7f7a1406eb70 legacy=0x7f7a140cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 --2- 172.21.15.133:0/4085017855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a000039c0 0x7f7a00003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 --2- 172.21.15.133:0/4085017855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 0x7f7a00004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 --2- 172.21.15.133:0/4085017855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a000051d0 0x7f7a0000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1415b360 con 0x7f7a00004810 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0000c070 con 0x7f7a000051d0 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0000c270 con 0x7f7a000039c0 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a13fff640 1 --2- 172.21.15.133:0/4085017855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 0x7f7a00004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a195a4640 1 --2- 172.21.15.133:0/4085017855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a000051d0 0x7f7a0000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a18da3640 1 --2- 172.21.15.133:0/4085017855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a000039c0 0x7f7a00003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a13fff640 1 -- 172.21.15.133:0/4085017855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a000039c0 msgr2=0x7f7a00003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a13fff640 1 --2- 172.21.15.133:0/4085017855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a000039c0 0x7f7a00003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a13fff640 1 -- 172.21.15.133:0/4085017855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a000051d0 msgr2=0x7f7a0000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a13fff640 1 --2- 172.21.15.133:0/4085017855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a000051d0 0x7f7a0000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a195a4640 1 -- 172.21.15.133:0/4085017855 reap_dead start 2024-04-25T10:14:15.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.412+0000 7f7a13fff640 1 -- 172.21.15.133:0/4085017855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a00003530 con 0x7f7a00004810 2024-04-25T10:14:15.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a13fff640 1 --2- 172.21.15.133:0/4085017855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 0x7f7a00004c90 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f7a140cf5f0 tx=0x7f7a0401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f622dbe4fcc2399 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a040013a0 con 0x7f7a00004810 2024-04-25T10:14:15.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a040052f0 con 0x7f7a00004810 2024-04-25T10:14:15.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a137fe640 1 -- 172.21.15.133:0/4085017855 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a040a4e60 con 0x7f7a00004810 2024-04-25T10:14:15.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 -- 172.21.15.133:0/4085017855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 msgr2=0x7f7a00004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 --2- 172.21.15.133:0/4085017855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 0x7f7a00004c90 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f7a140cf5f0 tx=0x7f7a0401c3a0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 -- 172.21.15.133:0/4085017855 shutdown_connections 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 --2- 172.21.15.133:0/4085017855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 0x7f7a00004c90 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 -- 172.21.15.133:0/4085017855 >> 172.21.15.133:0/4085017855 conn(0x7f7a14067000 msgr2=0x7f7a1406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 -- 172.21.15.133:0/4085017855 shutdown_connections 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 -- 172.21.15.133:0/4085017855 wait complete. 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 Processor -- start 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 -- start start 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 0x7f7a141368f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a1414b620 0x7f7a1414baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a1415ce90 0x7f7a1415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140cd670 con 0x7f7a00004810 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1415bc00 con 0x7f7a1415ce90 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a1b007640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140cd7f0 con 0x7f7a1414b620 2024-04-25T10:14:15.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a18da3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 0x7f7a141368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a18da3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 0x7f7a141368f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56076/0 (socket says 172.21.15.133:56076) 2024-04-25T10:14:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.416+0000 7f7a18da3640 1 -- 172.21.15.133:0/452048044 learned_addr learned my addr 172.21.15.133:0/452048044 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a13fff640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a1414b620 0x7f7a1414baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a195a4640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a1415ce90 0x7f7a1415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a18da3640 1 -- 172.21.15.133:0/452048044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a1414b620 msgr2=0x7f7a1414baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a18da3640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a1414b620 0x7f7a1414baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a18da3640 1 -- 172.21.15.133:0/452048044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a1415ce90 msgr2=0x7f7a1415d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a18da3640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a1415ce90 0x7f7a1415d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a18da3640 1 -- 172.21.15.133:0/452048044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1414c940 con 0x7f7a00004810 2024-04-25T10:14:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a13fff640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a1414b620 0x7f7a1414baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:15.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a195a4640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a1415ce90 0x7f7a1415d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:15.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a18da3640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 0x7f7a141368f0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f79fc011690 tx=0x7f79fc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a117fa640 1 -- 172.21.15.133:0/452048044 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc00f9c0 con 0x7f7a00004810 2024-04-25T10:14:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a117fa640 1 -- 172.21.15.133:0/452048044 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc00fb60 con 0x7f7a00004810 2024-04-25T10:14:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a117fa640 1 -- 172.21.15.133:0/452048044 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc015460 con 0x7f7a00004810 2024-04-25T10:14:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a1b007640 1 -- 172.21.15.133:0/452048044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1414c0a0 con 0x7f7a00004810 2024-04-25T10:14:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a1b007640 1 -- 172.21.15.133:0/452048044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1414c660 con 0x7f7a00004810 2024-04-25T10:14:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a1b007640 1 -- 172.21.15.133:0/452048044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1406bc20 con 0x7f7a00004810 2024-04-25T10:14:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a117fa640 1 -- 172.21.15.133:0/452048044 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79fc015600 con 0x7f7a00004810 2024-04-25T10:14:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a117fa640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f4066f90 0x7f79f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a117fa640 1 -- 172.21.15.133:0/452048044 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f79fc085730 con 0x7f7a00004810 2024-04-25T10:14:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a13fff640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f4066f90 0x7f79f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.420+0000 7f7a13fff640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f4066f90 0x7f79f4069450 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7a14158900 tx=0x7f7a040011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.424+0000 7f7a117fa640 1 -- 172.21.15.133:0/452048044 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79fc019030 con 0x7f7a00004810 2024-04-25T10:14:15.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 -- 172.21.15.133:0/452048044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a14169160 con 0x7f7a00004810 2024-04-25T10:14:15.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a117fa640 1 -- 172.21.15.133:0/452048044 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f79fc055fa0 con 0x7f7a00004810 2024-04-25T10:14:15.618 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:15.618 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:14:09.106064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":19,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:14:15.618 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 83 2024-04-25T10:14:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 -- 172.21.15.133:0/452048044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f4066f90 msgr2=0x7f79f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f4066f90 0x7f79f4069450 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7a14158900 tx=0x7f7a040011a0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 -- 172.21.15.133:0/452048044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 msgr2=0x7f7a141368f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 0x7f7a141368f0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f79fc011690 tx=0x7f79fc011b50 comp rx=0 tx=0).stop 2024-04-25T10:14:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 -- 172.21.15.133:0/452048044 shutdown_connections 2024-04-25T10:14:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a1415ce90 0x7f7a1415d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a1414b620 0x7f7a1414baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f4066f90 0x7f79f4069450 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 --2- 172.21.15.133:0/452048044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a00004810 0x7f7a141368f0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 -- 172.21.15.133:0/452048044 >> 172.21.15.133:0/452048044 conn(0x7f7a14067000 msgr2=0x7f7a140caff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 -- 172.21.15.133:0/452048044 shutdown_connections 2024-04-25T10:14:15.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.616+0000 7f7a1b007640 1 -- 172.21.15.133:0/452048044 wait complete. 2024-04-25T10:14:15.636 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:15.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.740+0000 7faee812b640 1 Processor -- start 2024-04-25T10:14:15.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.740+0000 7faee812b640 1 -- start start 2024-04-25T10:14:15.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee812b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faee00d0bd0 0x7faee00b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee812b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee00ceb20 con 0x7faee00d0770 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee812b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee00b30a0 con 0x7faee015db00 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee812b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee00b7740 con 0x7faee00d0bd0 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee56c6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7faee00d0770 0x7faee00b87e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44000/0 (socket says 172.21.15.133:44000) 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee56c6640 1 -- 172.21.15.133:0/2558652813 learned_addr learned my addr 172.21.15.133:0/2558652813 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee66c8640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faee00d0bd0 0x7faee00b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422851215 0 0) 0x7faee00ceb20 con 0x7faee00d0770 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faecc003530 con 0x7faee00d0770 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 225219352 0 0) 0x7faecc003530 con 0x7faee00d0770 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faee00d0bd0 msgr2=0x7faee00b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faee00d0bd0 0x7faee00b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 >> v1:172.21.15.162:6789/0 conn(0x7faee015db00 legacy=0x7faee00b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee00b7a20 con 0x7faee00d0770 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7faedc001a00 con 0x7faee00d0770 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee66c8640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faee00d0bd0 0x7faee00b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 >> v1:172.21.15.133:6789/0 conn(0x7faee00d0770 legacy=0x7faee00b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faecc0039f0 0x7faecc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faecc004810 0x7faecc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faee00d0bd0 0x7faecc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee00b7740 con 0x7faecc0039f0 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc00bb00 con 0x7faecc004810 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faecc00bd00 con 0x7faee00d0bd0 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee5ec7640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faecc0039f0 0x7faecc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee66c8640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faee00d0bd0 0x7faecc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee56c6640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faecc004810 0x7faecc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee5ec7640 1 -- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faee00d0bd0 msgr2=0x7faecc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee5ec7640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faee00d0bd0 0x7faecc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee5ec7640 1 -- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faecc004810 msgr2=0x7faecc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee5ec7640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faecc004810 0x7faecc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.744+0000 7faee5ec7640 1 -- 172.21.15.133:0/2558652813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc003530 con 0x7faecc0039f0 2024-04-25T10:14:15.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee5ec7640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faecc0039f0 0x7faecc003e10 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7faed0001780 tx=0x7faed001bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8b2d2338580bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed00013a0 con 0x7faecc0039f0 2024-04-25T10:14:15.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed00a4d60 con 0x7faecc0039f0 2024-04-25T10:14:15.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee4ec5640 1 -- 172.21.15.133:0/2558652813 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0005310 con 0x7faecc0039f0 2024-04-25T10:14:15.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 -- 172.21.15.133:0/2558652813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faecc0039f0 msgr2=0x7faecc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 --2- 172.21.15.133:0/2558652813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faecc0039f0 0x7faecc003e10 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7faed0001780 tx=0x7faed001bf50 comp rx=0 tx=0).stop 2024-04-25T10:14:15.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee66c8640 1 -- 172.21.15.133:0/2558652813 reap_dead start 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 -- 172.21.15.133:0/2558652813 shutdown_connections 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 -- 172.21.15.133:0/2558652813 >> 172.21.15.133:0/2558652813 conn(0x7faee00727c0 msgr2=0x7faee0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 -- 172.21.15.133:0/2558652813 shutdown_connections 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 -- 172.21.15.133:0/2558652813 wait complete. 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 Processor -- start 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 -- start start 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faecc0039f0 0x7faee0163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faee014db30 0x7faee0162550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faee0162a90 0x7faee00ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee00d0fe0 con 0x7faee0162a90 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee00b7430 con 0x7faee014db30 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee00d18a0 con 0x7faecc0039f0 2024-04-25T10:14:15.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee66c8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faee0162a90 0x7faee00ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee66c8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faee0162a90 0x7faee00ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56088/0 (socket says 172.21.15.133:56088) 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee66c8640 1 -- 172.21.15.133:0/1328135248 learned_addr learned my addr 172.21.15.133:0/1328135248 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee66c8640 1 -- 172.21.15.133:0/1328135248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faecc0039f0 msgr2=0x7faee0163fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee5ec7640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faecc0039f0 0x7faee0163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee56c6640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faee014db30 0x7faee0162550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee66c8640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faecc0039f0 0x7faee0163fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee66c8640 1 -- 172.21.15.133:0/1328135248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faee014db30 msgr2=0x7faee0162550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee66c8640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faee014db30 0x7faee0162550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee66c8640 1 -- 172.21.15.133:0/1328135248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee0162f10 con 0x7faee0162a90 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee66c8640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faee0162a90 0x7faee00ccac0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7faee00b5840 tx=0x7faed800f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faed67fc640 1 -- 172.21.15.133:0/1328135248 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8004440 con 0x7faee0162a90 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faed67fc640 1 -- 172.21.15.133:0/1328135248 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed801a070 con 0x7faee0162a90 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 -- 172.21.15.133:0/1328135248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee01631f0 con 0x7faee0162a90 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faed67fc640 1 -- 172.21.15.133:0/1328135248 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8003860 con 0x7faee0162a90 2024-04-25T10:14:15.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.748+0000 7faee812b640 1 -- 172.21.15.133:0/1328135248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee01637b0 con 0x7faee0162a90 2024-04-25T10:14:15.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.752+0000 7faee812b640 1 -- 172.21.15.133:0/1328135248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee0068ef0 con 0x7faee0162a90 2024-04-25T10:14:15.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.752+0000 7faed67fc640 1 -- 172.21.15.133:0/1328135248 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faed8012070 con 0x7faee0162a90 2024-04-25T10:14:15.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.752+0000 7faed67fc640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faec4066f60 0x7faec4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.752+0000 7faee5ec7640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faec4066f60 0x7faec4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.752+0000 7faed67fc640 1 -- 172.21.15.133:0/1328135248 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7faed8084d00 con 0x7faee0162a90 2024-04-25T10:14:15.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.752+0000 7faee5ec7640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faec4066f60 0x7faec4069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7faed0001750 tx=0x7faed00011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.756+0000 7faed67fc640 1 -- 172.21.15.133:0/1328135248 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faed8055450 con 0x7faee0162a90 2024-04-25T10:14:15.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.932+0000 7faee812b640 1 -- 172.21.15.133:0/1328135248 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faee00b2750 con 0x7faee0162a90 2024-04-25T10:14:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.932+0000 7faed67fc640 1 -- 172.21.15.133:0/1328135248 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4155 (secure 0 0 0) 0x7faed80594f0 con 0x7faee0162a90 2024-04-25T10:14:15.936 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:15.936 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:10.856393+0000","modified":"2024-04-25T10:14:09.106064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":19,"addr":"172.21.15.133:6835/111426777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":111426777},{"type":"v1","addr":"172.21.15.133:6835","nonce":111426777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:14:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 83 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 -- 172.21.15.133:0/1328135248 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faec4066f60 msgr2=0x7faec4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faec4066f60 0x7faec4069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7faed0001750 tx=0x7faed00011a0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 -- 172.21.15.133:0/1328135248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faee0162a90 msgr2=0x7faee00ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faee0162a90 0x7faee00ccac0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7faee00b5840 tx=0x7faed800f6f0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 -- 172.21.15.133:0/1328135248 shutdown_connections 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7faec4066f60 0x7faec4069420 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faee0162a90 0x7faee00ccac0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faee014db30 0x7faee0162550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 --2- 172.21.15.133:0/1328135248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7faecc0039f0 0x7faee0163fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 -- 172.21.15.133:0/1328135248 >> 172.21.15.133:0/1328135248 conn(0x7faee00727c0 msgr2=0x7faee015c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 -- 172.21.15.133:0/1328135248 shutdown_connections 2024-04-25T10:14:15.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:15.936+0000 7faee812b640 1 -- 172.21.15.133:0/1328135248 wait complete. 2024-04-25T10:14:15.955 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:16.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8558f1b640 1 Processor -- start 2024-04-25T10:14:16.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8558f1b640 1 -- start start 2024-04-25T10:14:16.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8558f1b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f85540b5350 0x7f85540b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8558f1b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85540b6b00 0x7f85540b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8558f1b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85540d20f0 con 0x7f855415db40 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8558f1b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85540b5cf0 con 0x7f85540b5350 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8558f1b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85540d2fd0 con 0x7f85540b6b00 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8551d9b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f85540b5350 0x7f85540b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8552d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85540b6b00 0x7f85540b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8551d9b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f85540b5350 0x7f85540b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55238/0 (socket says 172.21.15.133:55238) 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8551d9b640 1 -- 172.21.15.133:0/58852411 learned_addr learned my addr 172.21.15.133:0/58852411 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:16.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8551d9b640 1 -- 172.21.15.133:0/58852411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85540b6b00 msgr2=0x7f85540b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8551d9b640 1 --2- 172.21.15.133:0/58852411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85540b6b00 0x7f85540b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8551d9b640 1 -- 172.21.15.133:0/58852411 >> v1:172.21.15.162:6789/0 conn(0x7f855415db40 legacy=0x7f85540d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8551d9b640 1 -- 172.21.15.133:0/58852411 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85540d23c0 con 0x7f85540b5350 2024-04-25T10:14:16.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.064+0000 7f8552d9d640 1 --2- 172.21.15.133:0/58852411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85540b6b00 0x7f85540b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:16.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f8551d9b640 1 --2- 172.21.15.133:0/58852411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85540b5350 0x7f85540b57b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8548009870 tx=0x7f854801c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97e46640485af77e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f855159a640 1 -- 172.21.15.133:0/58852411 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480a95f0 con 0x7f85540b5350 2024-04-25T10:14:16.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f855159a640 1 -- 172.21.15.133:0/58852411 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85480a9790 con 0x7f85540b5350 2024-04-25T10:14:16.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f855159a640 1 -- 172.21.15.133:0/58852411 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480a9b60 con 0x7f85540b5350 2024-04-25T10:14:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f8558f1b640 1 -- 172.21.15.133:0/58852411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85540b5350 msgr2=0x7f85540b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f8558f1b640 1 --2- 172.21.15.133:0/58852411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85540b5350 0x7f85540b57b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8548009870 tx=0x7f854801c530 comp rx=0 tx=0).stop 2024-04-25T10:14:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f8558f1b640 1 -- 172.21.15.133:0/58852411 shutdown_connections 2024-04-25T10:14:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f8558f1b640 1 --2- 172.21.15.133:0/58852411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85540b6b00 0x7f85540b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f8558f1b640 1 --2- 172.21.15.133:0/58852411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85540b5350 0x7f85540b57b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f8558f1b640 1 -- 172.21.15.133:0/58852411 >> 172.21.15.133:0/58852411 conn(0x7f85540d0350 msgr2=0x7f85540d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f8558f1b640 1 -- 172.21.15.133:0/58852411 shutdown_connections 2024-04-25T10:14:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f8558f1b640 1 -- 172.21.15.133:0/58852411 wait complete. 2024-04-25T10:14:16.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f8558f1b640 1 Processor -- start 2024-04-25T10:14:16.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.068+0000 7f8558f1b640 1 -- start start 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8558f1b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85540b5350 0x7f85541546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8558f1b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85540b6b00 0x7f8554155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8558f1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855415db40 0x7f85540d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8558f1b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855415e0a0 con 0x7f855415db40 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8558f1b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855415df20 con 0x7f85540b5350 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8558f1b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85540d3440 con 0x7f85540b6b00 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8552d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855415db40 0x7f85540d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8552d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855415db40 0x7f85540d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56104/0 (socket says 172.21.15.133:56104) 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8552d9d640 1 -- 172.21.15.133:0/472455425 learned_addr learned my addr 172.21.15.133:0/472455425 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:16.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f855259c640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85540b5350 0x7f85541546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8551d9b640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85540b6b00 0x7f8554155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8552d9d640 1 -- 172.21.15.133:0/472455425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85540b6b00 msgr2=0x7f8554155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8552d9d640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85540b6b00 0x7f8554155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8552d9d640 1 -- 172.21.15.133:0/472455425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85540b5350 msgr2=0x7f85541546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8552d9d640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85540b5350 0x7f85541546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8552d9d640 1 -- 172.21.15.133:0/472455425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85541566c0 con 0x7f855415db40 2024-04-25T10:14:16.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f855259c640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85540b5350 0x7f85541546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:16.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8552d9d640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855415db40 0x7f85540d36f0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f854c009e60 tx=0x7f854c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8542ffd640 1 -- 172.21.15.133:0/472455425 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c018030 con 0x7f855415db40 2024-04-25T10:14:16.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8542ffd640 1 -- 172.21.15.133:0/472455425 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854c00cdf0 con 0x7f855415db40 2024-04-25T10:14:16.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8542ffd640 1 -- 172.21.15.133:0/472455425 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c003bc0 con 0x7f855415db40 2024-04-25T10:14:16.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8558f1b640 1 -- 172.21.15.133:0/472455425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85541569a0 con 0x7f855415db40 2024-04-25T10:14:16.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8558f1b640 1 -- 172.21.15.133:0/472455425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8554154bf0 con 0x7f855415db40 2024-04-25T10:14:16.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8558f1b640 1 -- 172.21.15.133:0/472455425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8554068ef0 con 0x7f855415db40 2024-04-25T10:14:16.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8542ffd640 1 -- 172.21.15.133:0/472455425 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f854c01b070 con 0x7f855415db40 2024-04-25T10:14:16.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8542ffd640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8534066f60 0x7f8534069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f8542ffd640 1 -- 172.21.15.133:0/472455425 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f854c085b30 con 0x7f855415db40 2024-04-25T10:14:16.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.072+0000 7f855259c640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8534066f60 0x7f8534069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.076+0000 7f855259c640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8534066f60 0x7f8534069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8544001710 tx=0x7f8544009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.076+0000 7f8542ffd640 1 -- 172.21.15.133:0/472455425 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f854c022070 con 0x7f855415db40 2024-04-25T10:14:16.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.228+0000 7f8558f1b640 1 -- 172.21.15.133:0/472455425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85540b3260 con 0x7f855415db40 2024-04-25T10:14:16.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8542ffd640 1 -- 172.21.15.133:0/472455425 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17055 (secure 0 0 0) 0x7f854c010040 con 0x7f855415db40 2024-04-25T10:14:16.233 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:16.233 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":133,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:06.927526+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:09.128791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:09.816841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/2351700644":"2024-04-26T10:14:06.927420+0000","172.21.15.133:6836/2351700644":"2024-04-26T10:14:06.927420+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:16.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 -- 172.21.15.133:0/472455425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8534066f60 msgr2=0x7f8534069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8534066f60 0x7f8534069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8544001710 tx=0x7f8544009040 comp rx=0 tx=0).stop 2024-04-25T10:14:16.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 -- 172.21.15.133:0/472455425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855415db40 msgr2=0x7f85540d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855415db40 0x7f85540d36f0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f854c009e60 tx=0x7f854c00c490 comp rx=0 tx=0).stop 2024-04-25T10:14:16.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 -- 172.21.15.133:0/472455425 shutdown_connections 2024-04-25T10:14:16.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8534066f60 0x7f8534069420 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855415db40 0x7f85540d36f0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85540b6b00 0x7f8554155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 --2- 172.21.15.133:0/472455425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85540b5350 0x7f85541546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 -- 172.21.15.133:0/472455425 >> 172.21.15.133:0/472455425 conn(0x7f85540d0350 msgr2=0x7f85540b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 -- 172.21.15.133:0/472455425 shutdown_connections 2024-04-25T10:14:16.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.232+0000 7f8558f1b640 1 -- 172.21.15.133:0/472455425 wait complete. 2024-04-25T10:14:16.251 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:14:16.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a86983640 1 Processor -- start 2024-04-25T10:14:16.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a86983640 1 -- start start 2024-04-25T10:14:16.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a86983640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3a8015d9a0 0x7f3a800d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a86983640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a800ce600 0x7f3a800cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a86983640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3a800d1920 0x7f3a800d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a86983640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a800cefa0 con 0x7f3a800d1920 2024-04-25T10:14:16.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a86983640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8015ebd0 con 0x7f3a8015d9a0 2024-04-25T10:14:16.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a86983640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8015eeb0 con 0x7f3a800ce600 2024-04-25T10:14:16.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a84f20640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3a800d1920 0x7f3a800d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a84f20640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3a800d1920 0x7f3a800d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55186/0 (socket says 172.21.15.133:55186) 2024-04-25T10:14:16.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a84f20640 1 -- 172.21.15.133:0/1406819559 learned_addr learned my addr 172.21.15.133:0/1406819559 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:16.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a7f7fe640 1 --2- 172.21.15.133:0/1406819559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a800ce600 0x7f3a800cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a7ffff640 1 --2- 172.21.15.133:0/1406819559 >> v2:172.21.15.162:3300/0 conn(0x7f3a8015d9a0 0x7f3a800d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a84f20640 1 -- 172.21.15.133:0/1406819559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a800ce600 msgr2=0x7f3a800cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a84f20640 1 --2- 172.21.15.133:0/1406819559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a800ce600 0x7f3a800cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a84f20640 1 -- 172.21.15.133:0/1406819559 >> v2:172.21.15.162:3300/0 conn(0x7f3a8015d9a0 msgr2=0x7f3a800d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a84f20640 1 --2- 172.21.15.133:0/1406819559 >> v2:172.21.15.162:3300/0 conn(0x7f3a8015d9a0 0x7f3a800d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a84f20640 1 -- 172.21.15.133:0/1406819559 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a800d0330 con 0x7f3a800d1920 2024-04-25T10:14:16.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.372+0000 7f3a7ffff640 1 --2- 172.21.15.133:0/1406819559 >> v2:172.21.15.162:3300/0 conn(0x7f3a8015d9a0 0x7f3a800d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:16.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a84f20640 1 --2- 172.21.15.133:0/1406819559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a800d1920 0x7f3a800d1db0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f3a800d4750 tx=0x7f3a7801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de12866bac47e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7effd640 1 -- 172.21.15.133:0/1406819559 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a780a5790 con 0x7f3a800d1920 2024-04-25T10:14:16.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7effd640 1 -- 172.21.15.133:0/1406819559 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a780a5930 con 0x7f3a800d1920 2024-04-25T10:14:16.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7effd640 1 -- 172.21.15.133:0/1406819559 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a780a5d00 con 0x7f3a800d1920 2024-04-25T10:14:16.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 -- 172.21.15.133:0/1406819559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a800d1920 msgr2=0x7f3a800d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 --2- 172.21.15.133:0/1406819559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a800d1920 0x7f3a800d1db0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f3a800d4750 tx=0x7f3a7801c930 comp rx=0 tx=0).stop 2024-04-25T10:14:16.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 -- 172.21.15.133:0/1406819559 shutdown_connections 2024-04-25T10:14:16.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 --2- 172.21.15.133:0/1406819559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a800d1920 0x7f3a800d1db0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 --2- 172.21.15.133:0/1406819559 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a800ce600 0x7f3a800cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 --2- 172.21.15.133:0/1406819559 >> v2:172.21.15.162:3300/0 conn(0x7f3a8015d9a0 0x7f3a800d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 -- 172.21.15.133:0/1406819559 >> 172.21.15.133:0/1406819559 conn(0x7f3a8006bf80 msgr2=0x7f3a8006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 -- 172.21.15.133:0/1406819559 shutdown_connections 2024-04-25T10:14:16.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 -- 172.21.15.133:0/1406819559 wait complete. 2024-04-25T10:14:16.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 Processor -- start 2024-04-25T10:14:16.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 -- start start 2024-04-25T10:14:16.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a800ce600 0x7f3a800ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a800d1920 0x7f3a80157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a8015d9a0 0x7f3a80158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800b9820 con 0x7f3a800d1920 2024-04-25T10:14:16.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800b96a0 con 0x7f3a800ce600 2024-04-25T10:14:16.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a86983640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800d22f0 con 0x7f3a8015d9a0 2024-04-25T10:14:16.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a800d1920 0x7f3a80157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a800d1920 0x7f3a80157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55196/0 (socket says 172.21.15.133:55196) 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7f7fe640 1 -- 172.21.15.133:0/3515986755 learned_addr learned my addr 172.21.15.133:0/3515986755 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a84f20640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a8015d9a0 0x7f3a80158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7ffff640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a800ce600 0x7f3a800ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7f7fe640 1 -- 172.21.15.133:0/3515986755 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a8015d9a0 msgr2=0x7f3a80158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7f7fe640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a8015d9a0 0x7f3a80158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7f7fe640 1 -- 172.21.15.133:0/3515986755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a800ce600 msgr2=0x7f3a800ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7f7fe640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a800ce600 0x7f3a800ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.376+0000 7f3a7f7fe640 1 -- 172.21.15.133:0/3515986755 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a800c8550 con 0x7f3a800d1920 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a7f7fe640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a800d1920 0x7f3a80157190 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400d800 tx=0x7f3a7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a7cff9640 1 -- 172.21.15.133:0/3515986755 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74018030 con 0x7f3a800d1920 2024-04-25T10:14:16.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a7cff9640 1 -- 172.21.15.133:0/3515986755 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a740096a0 con 0x7f3a800d1920 2024-04-25T10:14:16.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a86983640 1 -- 172.21.15.133:0/3515986755 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a800c87f0 con 0x7f3a800d1920 2024-04-25T10:14:16.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a7cff9640 1 -- 172.21.15.133:0/3515986755 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74009870 con 0x7f3a800d1920 2024-04-25T10:14:16.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a86983640 1 -- 172.21.15.133:0/3515986755 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a800c8db0 con 0x7f3a800d1920 2024-04-25T10:14:16.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a86983640 1 -- 172.21.15.133:0/3515986755 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a80068e80 con 0x7f3a800d1920 2024-04-25T10:14:16.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a7cff9640 1 -- 172.21.15.133:0/3515986755 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a7401b070 con 0x7f3a800d1920 2024-04-25T10:14:16.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a7cff9640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a6c066f90 0x7f3a6c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a7ffff640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a6c066f90 0x7f3a6c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a7cff9640 1 -- 172.21.15.133:0/3515986755 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3a740853b0 con 0x7f3a800d1920 2024-04-25T10:14:16.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.380+0000 7f3a7ffff640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a6c066f90 0x7f3a6c069450 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3a8015af30 tx=0x7f3a70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.384+0000 7f3a7cff9640 1 -- 172.21.15.133:0/3515986755 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a74056270 con 0x7f3a800d1920 2024-04-25T10:14:16.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.572+0000 7f3a86983640 1 -- 172.21.15.133:0/3515986755 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3a80167a20 con 0x7f3a800d1920 2024-04-25T10:14:16.574 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:14:16.572+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:14:16.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.576+0000 7f3a7cff9640 1 -- 172.21.15.133:0/3515986755 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3a74055b70 con 0x7f3a800d1920 2024-04-25T10:14:16.576 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:14:16.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.576+0000 7f3a86983640 1 -- 172.21.15.133:0/3515986755 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a6c066f90 msgr2=0x7f3a6c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.576+0000 7f3a86983640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a6c066f90 0x7f3a6c069450 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3a8015af30 tx=0x7f3a70008040 comp rx=0 tx=0).stop 2024-04-25T10:14:16.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.576+0000 7f3a86983640 1 -- 172.21.15.133:0/3515986755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a800d1920 msgr2=0x7f3a80157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.576+0000 7f3a86983640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a800d1920 0x7f3a80157190 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400d800 tx=0x7f3a7400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.580+0000 7f3a86983640 1 -- 172.21.15.133:0/3515986755 shutdown_connections 2024-04-25T10:14:16.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.580+0000 7f3a86983640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3a8015d9a0 0x7f3a80158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.580+0000 7f3a86983640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3a6c066f90 0x7f3a6c069450 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.580+0000 7f3a86983640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a800d1920 0x7f3a80157190 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.580+0000 7f3a86983640 1 --2- 172.21.15.133:0/3515986755 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a800ce600 0x7f3a800ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.580+0000 7f3a86983640 1 -- 172.21.15.133:0/3515986755 >> 172.21.15.133:0/3515986755 conn(0x7f3a8006bf80 msgr2=0x7f3a800735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.580+0000 7f3a86983640 1 -- 172.21.15.133:0/3515986755 shutdown_connections 2024-04-25T10:14:16.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.580+0000 7f3a86983640 1 -- 172.21.15.133:0/3515986755 wait complete. 2024-04-25T10:14:16.595 INFO:tasks.ceph.mds.a.smithi133.stderr: -2628> 2024-04-25T10:14:12.140+0000 7f9db3a9f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:16.598 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:14:16.630 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:14:16.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.708+0000 7f753d46b640 1 Processor -- start 2024-04-25T10:14:16.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.708+0000 7f753d46b640 1 -- start start 2024-04-25T10:14:16.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f753d46b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7538145170 0x7f753806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f753d46b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f753806e4d0 0x7f75380cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f753d46b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75380ce2e0 con 0x7f75380cdd60 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f753d46b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753806db00 con 0x7f753806e4d0 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f753d46b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7538145550 con 0x7f7538145170 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f75367fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f753806e4d0 0x7f75380cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f75377fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f75380cdd60 0x7f753815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45404/0 (socket says 172.21.15.133:45404) 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f75377fe640 1 -- 172.21.15.133:0/3663225661 learned_addr learned my addr 172.21.15.133:0/3663225661 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f7536ffd640 1 --2- 172.21.15.133:0/3663225661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7538145170 0x7f753806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f75367fc640 1 -- 172.21.15.133:0/3663225661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7538145170 msgr2=0x7f753806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f75367fc640 1 --2- 172.21.15.133:0/3663225661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7538145170 0x7f753806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f7535ffb640 1 -- 172.21.15.133:0/3663225661 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959995722 0 0) 0x7f75380ce2e0 con 0x7f75380cdd60 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f75367fc640 1 -- 172.21.15.133:0/3663225661 >> v1:172.21.15.133:6789/0 conn(0x7f75380cdd60 legacy=0x7f753815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f75367fc640 1 -- 172.21.15.133:0/3663225661 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753806d4b0 con 0x7f753806e4d0 2024-04-25T10:14:16.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f75367fc640 1 --2- 172.21.15.133:0/3663225661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f753806e4d0 0x7f75380cd820 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f75280097c0 tx=0x7f752801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5d43cb4aa71d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f7535ffb640 1 -- 172.21.15.133:0/3663225661 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75280b0070 con 0x7f753806e4d0 2024-04-25T10:14:16.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f7535ffb640 1 -- 172.21.15.133:0/3663225661 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75280ab600 con 0x7f753806e4d0 2024-04-25T10:14:16.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.712+0000 7f7535ffb640 1 -- 172.21.15.133:0/3663225661 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75280ab9d0 con 0x7f753806e4d0 2024-04-25T10:14:16.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 -- 172.21.15.133:0/3663225661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f753806e4d0 msgr2=0x7f75380cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 --2- 172.21.15.133:0/3663225661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f753806e4d0 0x7f75380cd820 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f75280097c0 tx=0x7f752801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 -- 172.21.15.133:0/3663225661 shutdown_connections 2024-04-25T10:14:16.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 --2- 172.21.15.133:0/3663225661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f753806e4d0 0x7f75380cd820 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 --2- 172.21.15.133:0/3663225661 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7538145170 0x7f753806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 -- 172.21.15.133:0/3663225661 >> 172.21.15.133:0/3663225661 conn(0x7f7538066f70 msgr2=0x7f753806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 -- 172.21.15.133:0/3663225661 shutdown_connections 2024-04-25T10:14:16.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 -- 172.21.15.133:0/3663225661 wait complete. 2024-04-25T10:14:16.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 Processor -- start 2024-04-25T10:14:16.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 -- start start 2024-04-25T10:14:16.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f753806e4d0 0x7f75380b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75380cdd60 0x7f75381411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7538145170 0x7f7538164e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7538168490 con 0x7f753806e4d0 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753815d7f0 con 0x7f75380cdd60 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7538168610 con 0x7f7538145170 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f7536ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f753806e4d0 0x7f75380b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f7536ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f753806e4d0 0x7f75380b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55244/0 (socket says 172.21.15.133:55244) 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f7536ffd640 1 -- 172.21.15.133:0/3883028373 learned_addr learned my addr 172.21.15.133:0/3883028373 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f7536ffd640 1 -- 172.21.15.133:0/3883028373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7538145170 msgr2=0x7f7538164e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f7536ffd640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7538145170 0x7f7538164e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f7536ffd640 1 -- 172.21.15.133:0/3883028373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75380cdd60 msgr2=0x7f75381411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f7536ffd640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75380cdd60 0x7f75381411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f7536ffd640 1 -- 172.21.15.133:0/3883028373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75380af650 con 0x7f753806e4d0 2024-04-25T10:14:16.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f7536ffd640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f753806e4d0 0x7f75380b0410 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f752c00d5f0 tx=0x7f752c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f751f7fe640 1 -- 172.21.15.133:0/3883028373 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c0125f0 con 0x7f753806e4d0 2024-04-25T10:14:16.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f751f7fe640 1 -- 172.21.15.133:0/3883028373 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752c019070 con 0x7f753806e4d0 2024-04-25T10:14:16.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 -- 172.21.15.133:0/3883028373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75380af930 con 0x7f753806e4d0 2024-04-25T10:14:16.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f751f7fe640 1 -- 172.21.15.133:0/3883028373 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c0129d0 con 0x7f753806e4d0 2024-04-25T10:14:16.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.716+0000 7f753d46b640 1 -- 172.21.15.133:0/3883028373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75380afef0 con 0x7f753806e4d0 2024-04-25T10:14:16.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.720+0000 7f753d46b640 1 -- 172.21.15.133:0/3883028373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75380b00f0 con 0x7f753806e4d0 2024-04-25T10:14:16.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.720+0000 7f751f7fe640 1 -- 172.21.15.133:0/3883028373 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f752c012b70 con 0x7f753806e4d0 2024-04-25T10:14:16.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.720+0000 7f751f7fe640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7524066f60 0x7f7524069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.720+0000 7f751f7fe640 1 -- 172.21.15.133:0/3883028373 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f752c085770 con 0x7f753806e4d0 2024-04-25T10:14:16.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.720+0000 7f75367fc640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7524066f60 0x7f7524069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.720+0000 7f75367fc640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7524066f60 0x7f7524069420 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f75280ad000 tx=0x7f7528008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.724+0000 7f751f7fe640 1 -- 172.21.15.133:0/3883028373 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75380b00f0 con 0x7f753806e4d0 2024-04-25T10:14:16.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:16.916+0000 7f753d46b640 1 -- 172.21.15.133:0/3883028373 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7538068e80 con 0x7f753806e4d0 2024-04-25T10:14:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.576+0000 7f751f7fe640 1 -- 172.21.15.133:0/3883028373 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7f752c0564a0 con 0x7f753806e4d0 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 -- 172.21.15.133:0/3883028373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7524066f60 msgr2=0x7f7524069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7524066f60 0x7f7524069420 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f75280ad000 tx=0x7f7528008040 comp rx=0 tx=0).stop 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 -- 172.21.15.133:0/3883028373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f753806e4d0 msgr2=0x7f75380b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f753806e4d0 0x7f75380b0410 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f752c00d5f0 tx=0x7f752c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 -- 172.21.15.133:0/3883028373 shutdown_connections 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7538145170 0x7f7538164e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75380cdd60 0x7f75381411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7524066f60 0x7f7524069420 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 --2- 172.21.15.133:0/3883028373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f753806e4d0 0x7f75380b0410 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 -- 172.21.15.133:0/3883028373 >> 172.21.15.133:0/3883028373 conn(0x7f7538066f70 msgr2=0x7f753815f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 -- 172.21.15.133:0/3883028373 shutdown_connections 2024-04-25T10:14:17.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.580+0000 7f753d46b640 1 -- 172.21.15.133:0/3883028373 wait complete. 2024-04-25T10:14:17.605 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:14:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.776+0000 7fa50724b640 1 Processor -- start 2024-04-25T10:14:17.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.776+0000 7fa50724b640 1 -- start start 2024-04-25T10:14:17.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa50724b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa50015b640 0x7fa50015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa50724b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5000d03e0 0x7fa5000b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa50724b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5000cc300 con 0x7fa5000cf2c0 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa50724b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50015f460 con 0x7fa50015b640 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa50724b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5001602c0 con 0x7fa5000d03e0 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa504fe7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa50015b640 0x7fa50015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa504fe7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa50015b640 0x7fa50015ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55248/0 (socket says 172.21.15.133:55248) 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa504fe7640 1 -- 172.21.15.133:0/2119767801 learned_addr learned my addr 172.21.15.133:0/2119767801 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa5057e8640 1 --2- 172.21.15.133:0/2119767801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5000d03e0 0x7fa5000b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa504fe7640 1 -- 172.21.15.133:0/2119767801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5000d03e0 msgr2=0x7fa5000b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa504fe7640 1 --2- 172.21.15.133:0/2119767801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5000d03e0 0x7fa5000b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa504fe7640 1 -- 172.21.15.133:0/2119767801 >> v1:172.21.15.162:6789/0 conn(0x7fa5000cf2c0 legacy=0x7fa5000cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa504fe7640 1 -- 172.21.15.133:0/2119767801 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50015f6b0 con 0x7fa50015b640 2024-04-25T10:14:17.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa504fe7640 1 --2- 172.21.15.133:0/2119767801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa50015b640 0x7fa50015ba20 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009870 tx=0x7fa4f001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2a66d154e1aded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa4ff7fe640 1 -- 172.21.15.133:0/2119767801 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f00b0070 con 0x7fa50015b640 2024-04-25T10:14:17.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa4ff7fe640 1 -- 172.21.15.133:0/2119767801 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f00ab600 con 0x7fa50015b640 2024-04-25T10:14:17.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.780+0000 7fa4ff7fe640 1 -- 172.21.15.133:0/2119767801 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f00ab9d0 con 0x7fa50015b640 2024-04-25T10:14:17.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- 172.21.15.133:0/2119767801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa50015b640 msgr2=0x7fa50015ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 --2- 172.21.15.133:0/2119767801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa50015b640 0x7fa50015ba20 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009870 tx=0x7fa4f001c340 comp rx=0 tx=0).stop 2024-04-25T10:14:17.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- 172.21.15.133:0/2119767801 shutdown_connections 2024-04-25T10:14:17.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 --2- 172.21.15.133:0/2119767801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5000d03e0 0x7fa5000b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 --2- 172.21.15.133:0/2119767801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa50015b640 0x7fa50015ba20 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- 172.21.15.133:0/2119767801 >> 172.21.15.133:0/2119767801 conn(0x7fa50009aa00 msgr2=0x7fa50008af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- 172.21.15.133:0/2119767801 shutdown_connections 2024-04-25T10:14:17.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- 172.21.15.133:0/2119767801 wait complete. 2024-04-25T10:14:17.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 Processor -- start 2024-04-25T10:14:17.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- start start 2024-04-25T10:14:17.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5000cf2c0 0x7fa500158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5000d03e0 0x7fa500154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa50015b640 0x7fa50015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000b26b0 con 0x7fa50015b640 2024-04-25T10:14:17.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000b2530 con 0x7fa5000cf2c0 2024-04-25T10:14:17.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000b2830 con 0x7fa5000d03e0 2024-04-25T10:14:17.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa5057e8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa50015b640 0x7fa50015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa5057e8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa50015b640 0x7fa50015e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55262/0 (socket says 172.21.15.133:55262) 2024-04-25T10:14:17.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa5057e8640 1 -- 172.21.15.133:0/1449095275 learned_addr learned my addr 172.21.15.133:0/1449095275 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:17.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa5057e8640 1 -- 172.21.15.133:0/1449095275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5000d03e0 msgr2=0x7fa500154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa504fe7640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5000cf2c0 0x7fa500158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa4fffff640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5000d03e0 0x7fa500154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa5057e8640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5000d03e0 0x7fa500154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa5057e8640 1 -- 172.21.15.133:0/1449095275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5000cf2c0 msgr2=0x7fa500158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa5057e8640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5000cf2c0 0x7fa500158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa5057e8640 1 -- 172.21.15.133:0/1449095275 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50015d120 con 0x7fa50015b640 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa504fe7640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5000cf2c0 0x7fa500158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa5057e8640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa50015b640 0x7fa50015e1c0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800cd20 tx=0x7fa4f800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa4fd7fa640 1 -- 172.21.15.133:0/1449095275 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8003ba0 con 0x7fa50015b640 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa4fd7fa640 1 -- 172.21.15.133:0/1449095275 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f8003d40 con 0x7fa50015b640 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- 172.21.15.133:0/1449095275 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50015d400 con 0x7fa50015b640 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa4fd7fa640 1 -- 172.21.15.133:0/1449095275 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8004240 con 0x7fa50015b640 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- 172.21.15.133:0/1449095275 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50015d9c0 con 0x7fa50015b640 2024-04-25T10:14:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.784+0000 7fa50724b640 1 -- 172.21.15.133:0/1449095275 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa500069000 con 0x7fa50015b640 2024-04-25T10:14:17.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.788+0000 7fa4fd7fa640 1 -- 172.21.15.133:0/1449095275 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa4f80043e0 con 0x7fa50015b640 2024-04-25T10:14:17.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.788+0000 7fa4fd7fa640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa4ec066f60 0x7fa4ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.788+0000 7fa4fd7fa640 1 -- 172.21.15.133:0/1449095275 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fa4f8085d00 con 0x7fa50015b640 2024-04-25T10:14:17.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.788+0000 7fa504fe7640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa4ec066f60 0x7fa4ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.788+0000 7fa504fe7640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa4ec066f60 0x7fa4ec069420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00ad000 tx=0x7fa4f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.792+0000 7fa4fd7fa640 1 -- 172.21.15.133:0/1449095275 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa4f8056ad0 con 0x7fa50015b640 2024-04-25T10:14:17.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:17.952+0000 7fa50724b640 1 -- 172.21.15.133:0/1449095275 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa50006e220 con 0x7fa50015b640 2024-04-25T10:14:18.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.584+0000 7fa4fd7fa640 1 -- 172.21.15.133:0/1449095275 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v135) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa4f80563d0 con 0x7fa50015b640 2024-04-25T10:14:18.587 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:14:18.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.588+0000 7fa50724b640 1 -- 172.21.15.133:0/1449095275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa4ec066f60 msgr2=0x7fa4ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.588+0000 7fa50724b640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa4ec066f60 0x7fa4ec069420 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00ad000 tx=0x7fa4f0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:18.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.588+0000 7fa50724b640 1 -- 172.21.15.133:0/1449095275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa50015b640 msgr2=0x7fa50015e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.588+0000 7fa50724b640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa50015b640 0x7fa50015e1c0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800cd20 tx=0x7fa4f800a660 comp rx=0 tx=0).stop 2024-04-25T10:14:18.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.588+0000 7fa50724b640 1 -- 172.21.15.133:0/1449095275 shutdown_connections 2024-04-25T10:14:18.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.588+0000 7fa50724b640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa4ec066f60 0x7fa4ec069420 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.588+0000 7fa50724b640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa50015b640 0x7fa50015e1c0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.588+0000 7fa50724b640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5000d03e0 0x7fa500154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.588+0000 7fa50724b640 1 --2- 172.21.15.133:0/1449095275 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5000cf2c0 0x7fa500158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.588+0000 7fa50724b640 1 -- 172.21.15.133:0/1449095275 >> 172.21.15.133:0/1449095275 conn(0x7fa50009aa00 msgr2=0x7fa50015a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.588+0000 7fa50724b640 1 -- 172.21.15.133:0/1449095275 shutdown_connections 2024-04-25T10:14:18.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.592+0000 7fa50724b640 1 -- 172.21.15.133:0/1449095275 wait complete. 2024-04-25T10:14:18.616 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:14:18.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f489052d640 1 Processor -- start 2024-04-25T10:14:18.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f489052d640 1 -- start start 2024-04-25T10:14:18.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f489052d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f488806e0c0 0x7f488806e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f489052d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488806eb80 con 0x7f488815dc20 2024-04-25T10:14:18.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f489052d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48880caed0 con 0x7f488815a040 2024-04-25T10:14:18.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f489052d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48880cb1b0 con 0x7f488806e0c0 2024-04-25T10:14:18.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f488dac8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f488806e0c0 0x7f488806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f488eaca640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f488815dc20 0x7f48880cb9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45420/0 (socket says 172.21.15.133:45420) 2024-04-25T10:14:18.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f488dac8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f488806e0c0 0x7f488806e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55264/0 (socket says 172.21.15.133:55264) 2024-04-25T10:14:18.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f488eaca640 1 -- 172.21.15.133:0/764784398 learned_addr learned my addr 172.21.15.133:0/764784398 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:18.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f488dac8640 1 -- 172.21.15.133:0/764784398 >> v1:172.21.15.162:6789/0 conn(0x7f488815a040 legacy=0x7f488806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f488dac8640 1 -- 172.21.15.133:0/764784398 >> v1:172.21.15.133:6789/0 conn(0x7f488815dc20 legacy=0x7f48880cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f488d2c7640 1 -- 172.21.15.133:0/764784398 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953806764 0 0) 0x7f488806eb80 con 0x7f488815dc20 2024-04-25T10:14:18.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.740+0000 7f488dac8640 1 -- 172.21.15.133:0/764784398 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48880cb490 con 0x7f488806e0c0 2024-04-25T10:14:18.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f488dac8640 1 --2- 172.21.15.133:0/764784398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f488806e0c0 0x7f488806e520 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f487800a960 tx=0x7f487801d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ac2d48f4f4cc54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f488d2c7640 1 -- 172.21.15.133:0/764784398 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48780b3070 con 0x7f488806e0c0 2024-04-25T10:14:18.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f488d2c7640 1 -- 172.21.15.133:0/764784398 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48780a6780 con 0x7f488806e0c0 2024-04-25T10:14:18.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f488d2c7640 1 -- 172.21.15.133:0/764784398 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800b040 con 0x7f488806e0c0 2024-04-25T10:14:18.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f489052d640 1 -- 172.21.15.133:0/764784398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f488806e0c0 msgr2=0x7f488806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f489052d640 1 --2- 172.21.15.133:0/764784398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f488806e0c0 0x7f488806e520 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f487800a960 tx=0x7f487801d830 comp rx=0 tx=0).stop 2024-04-25T10:14:18.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f489052d640 1 -- 172.21.15.133:0/764784398 shutdown_connections 2024-04-25T10:14:18.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f489052d640 1 --2- 172.21.15.133:0/764784398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f488806e0c0 0x7f488806e520 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f489052d640 1 -- 172.21.15.133:0/764784398 >> 172.21.15.133:0/764784398 conn(0x7f48880cc7c0 msgr2=0x7f48880ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f489052d640 1 -- 172.21.15.133:0/764784398 shutdown_connections 2024-04-25T10:14:18.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f489052d640 1 -- 172.21.15.133:0/764784398 wait complete. 2024-04-25T10:14:18.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f489052d640 1 Processor -- start 2024-04-25T10:14:18.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.744+0000 7f489052d640 1 -- start start 2024-04-25T10:14:18.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f489052d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f488806e0c0 0x7f4888151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f489052d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488815a040 0x7f48880c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f489052d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f488815dc20 0x7f48880b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f489052d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488815a420 con 0x7f488815dc20 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f489052d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488815e480 con 0x7f488806e0c0 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f489052d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488815a5a0 con 0x7f488815a040 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488eaca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f488815dc20 0x7f48880b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488eaca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f488815dc20 0x7f48880b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55274/0 (socket says 172.21.15.133:55274) 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488eaca640 1 -- 172.21.15.133:0/1990486691 learned_addr learned my addr 172.21.15.133:0/1990486691 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488dac8640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488815a040 0x7f48880c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488e2c9640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f488806e0c0 0x7f4888151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488eaca640 1 -- 172.21.15.133:0/1990486691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488815a040 msgr2=0x7f48880c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488eaca640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488815a040 0x7f48880c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488eaca640 1 -- 172.21.15.133:0/1990486691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f488806e0c0 msgr2=0x7f4888151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488eaca640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f488806e0c0 0x7f4888151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488eaca640 1 -- 172.21.15.133:0/1990486691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4888152470 con 0x7f488815dc20 2024-04-25T10:14:18.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488dac8640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488815a040 0x7f48880c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:18.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488e2c9640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f488806e0c0 0x7f4888151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:18.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488eaca640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f488815dc20 0x7f48880b0190 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f488400c730 tx=0x7f488400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f487effd640 1 -- 172.21.15.133:0/1990486691 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884018070 con 0x7f488815dc20 2024-04-25T10:14:18.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f487effd640 1 -- 172.21.15.133:0/1990486691 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400ce40 con 0x7f488815dc20 2024-04-25T10:14:18.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f489052d640 1 -- 172.21.15.133:0/1990486691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488809d530 con 0x7f488815dc20 2024-04-25T10:14:18.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f487effd640 1 -- 172.21.15.133:0/1990486691 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400a820 con 0x7f488815dc20 2024-04-25T10:14:18.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f489052d640 1 -- 172.21.15.133:0/1990486691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488815bbd0 con 0x7f488815dc20 2024-04-25T10:14:18.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f489052d640 1 -- 172.21.15.133:0/1990486691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4888069000 con 0x7f488815dc20 2024-04-25T10:14:18.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f487effd640 1 -- 172.21.15.133:0/1990486691 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f488401b070 con 0x7f488815dc20 2024-04-25T10:14:18.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f487effd640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f486c066f60 0x7f486c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f487effd640 1 -- 172.21.15.133:0/1990486691 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f4884084660 con 0x7f488815dc20 2024-04-25T10:14:18.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.748+0000 7f488e2c9640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f486c066f60 0x7f486c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.752+0000 7f488e2c9640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f486c066f60 0x7f486c069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f488000a850 tx=0x7f4880008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.752+0000 7f487effd640 1 -- 172.21.15.133:0/1990486691 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4884055550 con 0x7f488815dc20 2024-04-25T10:14:18.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:18.916+0000 7f489052d640 1 -- 172.21.15.133:0/1990486691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f488806da60 con 0x7f488815dc20 2024-04-25T10:14:19.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.588+0000 7f487effd640 1 -- 172.21.15.133:0/1990486691 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v136) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4884054e50 con 0x7f488815dc20 2024-04-25T10:14:19.590 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:14:19.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.592+0000 7f489052d640 1 -- 172.21.15.133:0/1990486691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f486c066f60 msgr2=0x7f486c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.592+0000 7f489052d640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f486c066f60 0x7f486c069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f488000a850 tx=0x7f4880008040 comp rx=0 tx=0).stop 2024-04-25T10:14:19.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.592+0000 7f489052d640 1 -- 172.21.15.133:0/1990486691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f488815dc20 msgr2=0x7f48880b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.592+0000 7f489052d640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f488815dc20 0x7f48880b0190 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f488400c730 tx=0x7f488400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.596+0000 7f489052d640 1 -- 172.21.15.133:0/1990486691 shutdown_connections 2024-04-25T10:14:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.596+0000 7f489052d640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f486c066f60 0x7f486c069420 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.596+0000 7f489052d640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f488815dc20 0x7f48880b0190 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.596+0000 7f489052d640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f488815a040 0x7f48880c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.596+0000 7f489052d640 1 --2- 172.21.15.133:0/1990486691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f488806e0c0 0x7f4888151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.596+0000 7f489052d640 1 -- 172.21.15.133:0/1990486691 >> 172.21.15.133:0/1990486691 conn(0x7f48880cc7c0 msgr2=0x7f48880af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.596+0000 7f489052d640 1 -- 172.21.15.133:0/1990486691 shutdown_connections 2024-04-25T10:14:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.596+0000 7f489052d640 1 -- 172.21.15.133:0/1990486691 wait complete. 2024-04-25T10:14:19.623 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:14:19.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.756+0000 7f3f4ed5a640 1 Processor -- start 2024-04-25T10:14:19.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.756+0000 7f3f4ed5a640 1 -- start start 2024-04-25T10:14:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4ed5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f4815d9a0 0x7f3f480d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4ed5a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3f480d2030 0x7f3f480cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4ed5a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f480d24c0 con 0x7f3f480ce600 2024-04-25T10:14:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4ed5a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f480cefa0 con 0x7f3f480d2030 2024-04-25T10:14:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4ed5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4815ebd0 con 0x7f3f4815d9a0 2024-04-25T10:14:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4d2f7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3f480d2030 0x7f3f480cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f47fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3f480ce600 0x7f3f480d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45430/0 (socket says 172.21.15.133:45430) 2024-04-25T10:14:19.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f47fff640 1 -- 172.21.15.133:0/4116043990 learned_addr learned my addr 172.21.15.133:0/4116043990 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:19.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4caf6640 1 --2- 172.21.15.133:0/4116043990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f4815d9a0 0x7f3f480d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4d2f7640 1 -- 172.21.15.133:0/4116043990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f4815d9a0 msgr2=0x7f3f480d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4d2f7640 1 --2- 172.21.15.133:0/4116043990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f4815d9a0 0x7f3f480d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4d2f7640 1 -- 172.21.15.133:0/4116043990 >> v1:172.21.15.133:6789/0 conn(0x7f3f480ce600 legacy=0x7f3f480d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f477fe640 1 -- 172.21.15.133:0/4116043990 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101100116 0 0) 0x7f3f480d24c0 con 0x7f3f480ce600 2024-04-25T10:14:19.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4d2f7640 1 -- 172.21.15.133:0/4116043990 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4815eeb0 con 0x7f3f480d2030 2024-04-25T10:14:19.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4caf6640 1 --2- 172.21.15.133:0/4116043990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f4815d9a0 0x7f3f480d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:19.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f4d2f7640 1 --2- 172.21.15.133:0/4116043990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f480d2030 0x7f3f480cea60 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f3f400097c0 tx=0x7f3f4001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cef170535e5de04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f477fe640 1 -- 172.21.15.133:0/4116043990 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f400b1070 con 0x7f3f480d2030 2024-04-25T10:14:19.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f477fe640 1 -- 172.21.15.133:0/4116043990 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f400a8050 con 0x7f3f480d2030 2024-04-25T10:14:19.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.760+0000 7f3f477fe640 1 -- 172.21.15.133:0/4116043990 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f400ac860 con 0x7f3f480d2030 2024-04-25T10:14:19.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/4116043990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f480d2030 msgr2=0x7f3f480cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 --2- 172.21.15.133:0/4116043990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f480d2030 0x7f3f480cea60 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f3f400097c0 tx=0x7f3f4001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/4116043990 shutdown_connections 2024-04-25T10:14:19.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 --2- 172.21.15.133:0/4116043990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f480d2030 0x7f3f480cea60 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 --2- 172.21.15.133:0/4116043990 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f4815d9a0 0x7f3f480d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/4116043990 >> 172.21.15.133:0/4116043990 conn(0x7f3f4806bf80 msgr2=0x7f3f4806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/4116043990 shutdown_connections 2024-04-25T10:14:19.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/4116043990 wait complete. 2024-04-25T10:14:19.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 Processor -- start 2024-04-25T10:14:19.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 -- start start 2024-04-25T10:14:19.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f480ce600 0x7f3f481575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f480d2030 0x7f3f480cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4815d9a0 0x7f3f480c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480b96a0 con 0x7f3f4815d9a0 2024-04-25T10:14:19.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480cffd0 con 0x7f3f480d2030 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480b9820 con 0x7f3f480ce600 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4d2f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4815d9a0 0x7f3f480c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4d2f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4815d9a0 0x7f3f480c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55296/0 (socket says 172.21.15.133:55296) 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4d2f7640 1 -- 172.21.15.133:0/1930158934 learned_addr learned my addr 172.21.15.133:0/1930158934 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4caf6640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f480ce600 0x7f3f481575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f47fff640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f480d2030 0x7f3f480cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4d2f7640 1 -- 172.21.15.133:0/1930158934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f480ce600 msgr2=0x7f3f481575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4d2f7640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f480ce600 0x7f3f481575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4d2f7640 1 -- 172.21.15.133:0/1930158934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f480d2030 msgr2=0x7f3f480cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4d2f7640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f480d2030 0x7f3f480cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4d2f7640 1 -- 172.21.15.133:0/1930158934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f48157b10 con 0x7f3f4815d9a0 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4caf6640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f480ce600 0x7f3f481575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4d2f7640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4815d9a0 0x7f3f480c7320 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f3f400af040 tx=0x7f3f4001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f457fa640 1 -- 172.21.15.133:0/1930158934 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f400bd070 con 0x7f3f4815d9a0 2024-04-25T10:14:19.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f457fa640 1 -- 172.21.15.133:0/1930158934 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f400b2040 con 0x7f3f4815d9a0 2024-04-25T10:14:19.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/1930158934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f48158da0 con 0x7f3f4815d9a0 2024-04-25T10:14:19.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f457fa640 1 -- 172.21.15.133:0/1930158934 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f400b1050 con 0x7f3f4815d9a0 2024-04-25T10:14:19.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.764+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/1930158934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f48159360 con 0x7f3f4815d9a0 2024-04-25T10:14:19.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.768+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/1930158934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f48068e80 con 0x7f3f4815d9a0 2024-04-25T10:14:19.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.768+0000 7f3f457fa640 1 -- 172.21.15.133:0/1930158934 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f400c0070 con 0x7f3f4815d9a0 2024-04-25T10:14:19.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.768+0000 7f3f457fa640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f28066f90 0x7f3f28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.768+0000 7f3f457fa640 1 -- 172.21.15.133:0/1930158934 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3f4011ed60 con 0x7f3f4815d9a0 2024-04-25T10:14:19.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.768+0000 7f3f4caf6640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f28066f90 0x7f3f28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.768+0000 7f3f4caf6640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f28066f90 0x7f3f28069450 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800e3d0 tx=0x7f3f38007a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.772+0000 7f3f457fa640 1 -- 172.21.15.133:0/1930158934 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f40016090 con 0x7f3f4815d9a0 2024-04-25T10:14:19.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.932+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/1930158934 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3f480b5020 con 0x7f3f4815d9a0 2024-04-25T10:14:19.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.936+0000 7f3f457fa640 1 -- 172.21.15.133:0/1930158934 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3f400efe10 con 0x7f3f4815d9a0 2024-04-25T10:14:19.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/1930158934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f28066f90 msgr2=0x7f3f28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f28066f90 0x7f3f28069450 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800e3d0 tx=0x7f3f38007a10 comp rx=0 tx=0).stop 2024-04-25T10:14:19.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/1930158934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4815d9a0 msgr2=0x7f3f480c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4815d9a0 0x7f3f480c7320 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f3f400af040 tx=0x7f3f4001ac20 comp rx=0 tx=0).stop 2024-04-25T10:14:19.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/1930158934 shutdown_connections 2024-04-25T10:14:19.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f28066f90 0x7f3f28069450 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4815d9a0 0x7f3f480c7320 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f480d2030 0x7f3f480cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 --2- 172.21.15.133:0/1930158934 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f480ce600 0x7f3f481575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/1930158934 >> 172.21.15.133:0/1930158934 conn(0x7f3f4806bf80 msgr2=0x7f3f4806c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/1930158934 shutdown_connections 2024-04-25T10:14:19.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:19.940+0000 7f3f4ed5a640 1 -- 172.21.15.133:0/1930158934 wait complete. 2024-04-25T10:14:19.959 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:14:20.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.076+0000 7f981aca7640 1 Processor -- start 2024-04-25T10:14:20.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.076+0000 7f981aca7640 1 -- start start 2024-04-25T10:14:20.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f981aca7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f981415c9a0 0x7f98140c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f981aca7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140ce330 0x7f98140ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f981aca7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98140c61d0 con 0x7f98140b1f70 2024-04-25T10:14:20.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f981aca7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98140cecc0 con 0x7f981415c9a0 2024-04-25T10:14:20.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f981aca7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98140c64b0 con 0x7f98140ce330 2024-04-25T10:14:20.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9813fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f98140b1f70 0x7f981406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45432/0 (socket says 172.21.15.133:45432) 2024-04-25T10:14:20.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9813fff640 1 -- 172.21.15.133:0/2268681210 learned_addr learned my addr 172.21.15.133:0/2268681210 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:20.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9819244640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140ce330 0x7f98140ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9818a43640 1 --2- 172.21.15.133:0/2268681210 >> v2:172.21.15.162:3300/0 conn(0x7f981415c9a0 0x7f98140c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938930090 0 0) 0x7f98140c61d0 con 0x7f98140b1f70 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9800003530 con 0x7f98140b1f70 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464606188 0 0) 0x7f9800003530 con 0x7f98140b1f70 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140ce330 msgr2=0x7f98140ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140ce330 0x7f98140ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 >> v2:172.21.15.162:3300/0 conn(0x7f981415c9a0 msgr2=0x7f98140c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 --2- 172.21.15.133:0/2268681210 >> v2:172.21.15.162:3300/0 conn(0x7f981415c9a0 0x7f98140c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981406d4f0 con 0x7f98140b1f70 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f980c001a00 con 0x7f98140b1f70 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 >> v1:172.21.15.133:6789/0 conn(0x7f98140b1f70 legacy=0x7f981406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98000039c0 0x7f9800003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140ce330 0x7f9800004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9800004d50 0x7f98000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98140cecc0 con 0x7f9800004d50 2024-04-25T10:14:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98140c64b0 con 0x7f98000039c0 2024-04-25T10:14:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980000b9d0 con 0x7f98140ce330 2024-04-25T10:14:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9819244640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9800004d50 0x7f98000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9818a43640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98000039c0 0x7f9800003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9813fff640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140ce330 0x7f9800004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9819244640 1 -- 172.21.15.133:0/2268681210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140ce330 msgr2=0x7f9800004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9819244640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140ce330 0x7f9800004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9819244640 1 -- 172.21.15.133:0/2268681210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98000039c0 msgr2=0x7f9800003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9819244640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98000039c0 0x7f9800003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.080+0000 7f9819244640 1 -- 172.21.15.133:0/2268681210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9800003530 con 0x7f9800004d50 2024-04-25T10:14:20.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f9819244640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9800004d50 0x7f98000051d0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f9804009b10 tx=0x7f980401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e13c9c6d230283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98040b2070 con 0x7f9800004d50 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98040a5660 con 0x7f9800004d50 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f98137fe640 1 -- 172.21.15.133:0/2268681210 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98040096b0 con 0x7f9800004d50 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 -- 172.21.15.133:0/2268681210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9800004d50 msgr2=0x7f98000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 --2- 172.21.15.133:0/2268681210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9800004d50 0x7f98000051d0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f9804009b10 tx=0x7f980401ca70 comp rx=0 tx=0).stop 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f9819244640 1 -- 172.21.15.133:0/2268681210 reap_dead start 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 -- 172.21.15.133:0/2268681210 shutdown_connections 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 -- 172.21.15.133:0/2268681210 >> 172.21.15.133:0/2268681210 conn(0x7f9814067000 msgr2=0x7f981406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 -- 172.21.15.133:0/2268681210 shutdown_connections 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 -- 172.21.15.133:0/2268681210 wait complete. 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 Processor -- start 2024-04-25T10:14:20.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 -- start start 2024-04-25T10:14:20.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9814149fa0 0x7f98140c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98141529d0 0x7f98140c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140c8760 0x7f98141f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981415cd80 con 0x7f98141529d0 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98140c5ec0 con 0x7f9814149fa0 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f981aca7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981415cf00 con 0x7f98140c8760 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f9813fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98141529d0 0x7f98140c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f9813fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98141529d0 0x7f98140c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55306/0 (socket says 172.21.15.133:55306) 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f9813fff640 1 -- 172.21.15.133:0/228139878 learned_addr learned my addr 172.21.15.133:0/228139878 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f9818a43640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9814149fa0 0x7f98140c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f9819244640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140c8760 0x7f98141f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f9813fff640 1 -- 172.21.15.133:0/228139878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140c8760 msgr2=0x7f98141f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f9813fff640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140c8760 0x7f98141f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f9813fff640 1 -- 172.21.15.133:0/228139878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9814149fa0 msgr2=0x7f98140c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.084+0000 7f9813fff640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9814149fa0 0x7f98140c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f9813fff640 1 -- 172.21.15.133:0/228139878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98140c9f20 con 0x7f98141529d0 2024-04-25T10:14:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f9813fff640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98141529d0 0x7f98140c8220 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f980c007b30 tx=0x7f980c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f98117fa640 1 -- 172.21.15.133:0/228139878 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c013420 con 0x7f98141529d0 2024-04-25T10:14:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f98117fa640 1 -- 172.21.15.133:0/228139878 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c018070 con 0x7f98141529d0 2024-04-25T10:14:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f981aca7640 1 -- 172.21.15.133:0/228139878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98140ca200 con 0x7f98141529d0 2024-04-25T10:14:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f98117fa640 1 -- 172.21.15.133:0/228139878 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c013800 con 0x7f98141529d0 2024-04-25T10:14:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f981aca7640 1 -- 172.21.15.133:0/228139878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9814153ad0 con 0x7f98141529d0 2024-04-25T10:14:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f981aca7640 1 -- 172.21.15.133:0/228139878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98141531e0 con 0x7f98141529d0 2024-04-25T10:14:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f98117fa640 1 -- 172.21.15.133:0/228139878 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f980c0139a0 con 0x7f98141529d0 2024-04-25T10:14:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f98117fa640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97f4066f60 0x7f97f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f98117fa640 1 -- 172.21.15.133:0/228139878 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f980c0855c0 con 0x7f98141529d0 2024-04-25T10:14:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f9818a43640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97f4066f60 0x7f97f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.088+0000 7f9818a43640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97f4066f60 0x7f97f4069420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009b20 tx=0x7f97fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.092+0000 7f98117fa640 1 -- 172.21.15.133:0/228139878 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f980c056780 con 0x7f98141529d0 2024-04-25T10:14:20.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.344+0000 7f981aca7640 1 -- 172.21.15.133:0/228139878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f9814068ef0 con 0x7f98141529d0 2024-04-25T10:14:20.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.588+0000 7f98117fa640 1 -- 172.21.15.133:0/228139878 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v559) v1 ==== 166+0+0 (secure 0 0 0) 0x7f980c056080 con 0x7f98141529d0 2024-04-25T10:14:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 -- 172.21.15.133:0/228139878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97f4066f60 msgr2=0x7f97f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97f4066f60 0x7f97f4069420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009b20 tx=0x7f97fc008040 comp rx=0 tx=0).stop 2024-04-25T10:14:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 -- 172.21.15.133:0/228139878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98141529d0 msgr2=0x7f98140c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98141529d0 0x7f98140c8220 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f980c007b30 tx=0x7f980c00e530 comp rx=0 tx=0).stop 2024-04-25T10:14:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 -- 172.21.15.133:0/228139878 shutdown_connections 2024-04-25T10:14:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98140c8760 0x7f98141f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f97f4066f60 0x7f97f4069420 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98141529d0 0x7f98140c8220 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 --2- 172.21.15.133:0/228139878 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9814149fa0 0x7f98140c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 -- 172.21.15.133:0/228139878 >> 172.21.15.133:0/228139878 conn(0x7f9814067000 msgr2=0x7f98140c4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 -- 172.21.15.133:0/228139878 shutdown_connections 2024-04-25T10:14:20.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.592+0000 7f981aca7640 1 -- 172.21.15.133:0/228139878 wait complete. 2024-04-25T10:14:20.618 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:14:20.618 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:14:20.619 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:20.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.740+0000 7f6380e80640 1 Processor -- start 2024-04-25T10:14:20.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.740+0000 7f6380e80640 1 -- start start 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f6380e80640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f637c15dd90 0x7f637c0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f6380e80640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f637c0b19f0 0x7f637c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f6380e80640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637c0ca9b0 con 0x7f637c15d9b0 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f6380e80640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637c0cbe30 con 0x7f637c15dd90 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f6380e80640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0cac90 con 0x7f637c0b19f0 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f6379d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f637c15dd90 0x7f637c0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f637a59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f637c15d9b0 0x7f637c0cb5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45444/0 (socket says 172.21.15.133:45444) 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f637a59c640 1 -- 172.21.15.133:0/3429467541 learned_addr learned my addr 172.21.15.133:0/3429467541 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f637ad9d640 1 --2- 172.21.15.133:0/3429467541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f637c0b19f0 0x7f637c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f6379d9b640 1 -- 172.21.15.133:0/3429467541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f637c0b19f0 msgr2=0x7f637c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f6379d9b640 1 --2- 172.21.15.133:0/3429467541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f637c0b19f0 0x7f637c0cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f6379d9b640 1 -- 172.21.15.133:0/3429467541 >> v1:172.21.15.133:6789/0 conn(0x7f637c15d9b0 legacy=0x7f637c0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f6379d9b640 1 -- 172.21.15.133:0/3429467541 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c0caf70 con 0x7f637c15dd90 2024-04-25T10:14:20.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f637959a640 1 -- 172.21.15.133:0/3429467541 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345200258 0 0) 0x7f637c0ca9b0 con 0x7f637c15d9b0 2024-04-25T10:14:20.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f6379d9b640 1 --2- 172.21.15.133:0/3429467541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f637c15dd90 0x7f637c0ce770 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f63700097c0 tx=0x7f637001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb314443522f1337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f637959a640 1 -- 172.21.15.133:0/3429467541 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63700ab430 con 0x7f637c15dd90 2024-04-25T10:14:20.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f637959a640 1 -- 172.21.15.133:0/3429467541 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63700b0070 con 0x7f637c15dd90 2024-04-25T10:14:20.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.744+0000 7f637959a640 1 -- 172.21.15.133:0/3429467541 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63700ab7f0 con 0x7f637c15dd90 2024-04-25T10:14:20.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 -- 172.21.15.133:0/3429467541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f637c15dd90 msgr2=0x7f637c0ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 --2- 172.21.15.133:0/3429467541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f637c15dd90 0x7f637c0ce770 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f63700097c0 tx=0x7f637001c410 comp rx=0 tx=0).stop 2024-04-25T10:14:20.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 -- 172.21.15.133:0/3429467541 shutdown_connections 2024-04-25T10:14:20.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 --2- 172.21.15.133:0/3429467541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f637c0b19f0 0x7f637c0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 --2- 172.21.15.133:0/3429467541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f637c15dd90 0x7f637c0ce770 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 -- 172.21.15.133:0/3429467541 >> 172.21.15.133:0/3429467541 conn(0x7f637c067000 msgr2=0x7f637c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 -- 172.21.15.133:0/3429467541 shutdown_connections 2024-04-25T10:14:20.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 -- 172.21.15.133:0/3429467541 wait complete. 2024-04-25T10:14:20.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 Processor -- start 2024-04-25T10:14:20.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 -- start start 2024-04-25T10:14:20.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f637c0b19f0 0x7f637c15f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f637c15d9b0 0x7f637c155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f637c15dd90 0x7f637c1556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c06e290 con 0x7f637c15dd90 2024-04-25T10:14:20.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c06e110 con 0x7f637c15d9b0 2024-04-25T10:14:20.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637c06e410 con 0x7f637c0b19f0 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f637ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f637c15dd90 0x7f637c1556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f637ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f637c15dd90 0x7f637c1556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55314/0 (socket says 172.21.15.133:55314) 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f637ad9d640 1 -- 172.21.15.133:0/472593148 learned_addr learned my addr 172.21.15.133:0/472593148 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f637a59c640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f637c0b19f0 0x7f637c15f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6379d9b640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f637c15d9b0 0x7f637c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f637ad9d640 1 -- 172.21.15.133:0/472593148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f637c0b19f0 msgr2=0x7f637c15f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f637ad9d640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f637c0b19f0 0x7f637c15f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f637ad9d640 1 -- 172.21.15.133:0/472593148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f637c15d9b0 msgr2=0x7f637c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f637ad9d640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f637c15d9b0 0x7f637c155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f637ad9d640 1 -- 172.21.15.133:0/472593148 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c15feb0 con 0x7f637c15dd90 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6379d9b640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f637c15d9b0 0x7f637c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f637ad9d640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f637c15dd90 0x7f637c1556a0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f637400d5d0 tx=0x7f637400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f636affd640 1 -- 172.21.15.133:0/472593148 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63740125f0 con 0x7f637c15dd90 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f636affd640 1 -- 172.21.15.133:0/472593148 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6374019070 con 0x7f637c15dd90 2024-04-25T10:14:20.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 -- 172.21.15.133:0/472593148 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c160190 con 0x7f637c15dd90 2024-04-25T10:14:20.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f636affd640 1 -- 172.21.15.133:0/472593148 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63740129f0 con 0x7f637c15dd90 2024-04-25T10:14:20.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.748+0000 7f6380e80640 1 -- 172.21.15.133:0/472593148 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c160750 con 0x7f637c15dd90 2024-04-25T10:14:20.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.752+0000 7f6380e80640 1 -- 172.21.15.133:0/472593148 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637c068ef0 con 0x7f637c15dd90 2024-04-25T10:14:20.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.752+0000 7f636affd640 1 -- 172.21.15.133:0/472593148 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6374012b90 con 0x7f637c15dd90 2024-04-25T10:14:20.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.752+0000 7f636affd640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f635c066f60 0x7f635c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.752+0000 7f636affd640 1 -- 172.21.15.133:0/472593148 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f63740853a0 con 0x7f637c15dd90 2024-04-25T10:14:20.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.752+0000 7f637a59c640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f635c066f60 0x7f635c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.752+0000 7f637a59c640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f635c066f60 0x7f635c069420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f636c0012f0 tx=0x7f636c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.756+0000 7f636affd640 1 -- 172.21.15.133:0/472593148 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63740564b0 con 0x7f637c15dd90 2024-04-25T10:14:20.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.916+0000 7f6380e80640 1 -- 172.21.15.133:0/472593148 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f637c06c2d0 con 0x7f637c15dd90 2024-04-25T10:14:20.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f636affd640 1 -- 172.21.15.133:0/472593148 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+13459 (secure 0 0 0) 0x7f637405ae20 con 0x7f637c15dd90 2024-04-25T10:14:20.922 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:20.922 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":136,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:19.589027+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/2351700644":"2024-04-26T10:14:06.927420+0000","172.21.15.133:6835/111426777":"2024-04-26T10:14:16.576868+0000","172.21.15.133:6834/111426777":"2024-04-26T10:14:16.576868+0000","172.21.15.133:6836/2351700644":"2024-04-26T10:14:06.927420+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:20.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 -- 172.21.15.133:0/472593148 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f635c066f60 msgr2=0x7f635c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f635c066f60 0x7f635c069420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f636c0012f0 tx=0x7f636c009040 comp rx=0 tx=0).stop 2024-04-25T10:14:20.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 -- 172.21.15.133:0/472593148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f637c15dd90 msgr2=0x7f637c1556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f637c15dd90 0x7f637c1556a0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f637400d5d0 tx=0x7f637400da90 comp rx=0 tx=0).stop 2024-04-25T10:14:20.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 -- 172.21.15.133:0/472593148 shutdown_connections 2024-04-25T10:14:20.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f635c066f60 0x7f635c069420 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f637c15dd90 0x7f637c1556a0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f637c15d9b0 0x7f637c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 --2- 172.21.15.133:0/472593148 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f637c0b19f0 0x7f637c15f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 -- 172.21.15.133:0/472593148 >> 172.21.15.133:0/472593148 conn(0x7f637c067000 msgr2=0x7f637c0b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 -- 172.21.15.133:0/472593148 shutdown_connections 2024-04-25T10:14:20.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:20.920+0000 7f6380e80640 1 -- 172.21.15.133:0/472593148 wait complete. 2024-04-25T10:14:20.942 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:20.942 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:21.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.068+0000 7f40e50fa640 1 Processor -- start 2024-04-25T10:14:21.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.068+0000 7f40e50fa640 1 -- start start 2024-04-25T10:14:21.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.068+0000 7f40e50fa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40e0145170 0x7f40e006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.068+0000 7f40e50fa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f40e015ced0 0x7f40e015d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.068+0000 7f40e50fa640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e00ce2e0 con 0x7f40e006e4d0 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.068+0000 7f40e50fa640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e006db00 con 0x7f40e015ced0 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.068+0000 7f40e50fa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e0145550 con 0x7f40e0145170 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.068+0000 7f40df59e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f40e015ced0 0x7f40e015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.068+0000 7f40de59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f40e006e4d0 0x7f40e00cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45448/0 (socket says 172.21.15.133:45448) 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.068+0000 7f40de59c640 1 -- 172.21.15.133:0/3576414112 learned_addr learned my addr 172.21.15.133:0/3576414112 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40ded9d640 1 --2- 172.21.15.133:0/3576414112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40e0145170 0x7f40e006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40ddd9b640 1 -- 172.21.15.133:0/3576414112 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152464472 0 0) 0x7f40e00ce2e0 con 0x7f40e006e4d0 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40ddd9b640 1 -- 172.21.15.133:0/3576414112 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40c8003530 con 0x7f40e006e4d0 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40df59e640 1 -- 172.21.15.133:0/3576414112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40e0145170 msgr2=0x7f40e006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40df59e640 1 --2- 172.21.15.133:0/3576414112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40e0145170 0x7f40e006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40df59e640 1 -- 172.21.15.133:0/3576414112 >> v1:172.21.15.133:6789/0 conn(0x7f40e006e4d0 legacy=0x7f40e00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40df59e640 1 -- 172.21.15.133:0/3576414112 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e006d4b0 con 0x7f40e015ced0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40df59e640 1 --2- 172.21.15.133:0/3576414112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e015ced0 0x7f40e015d360 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009d90 tx=0x7f40d801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a382194cd8bad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40ddd9b640 1 -- 172.21.15.133:0/3576414112 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d80b2070 con 0x7f40e015ced0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40ddd9b640 1 -- 172.21.15.133:0/3576414112 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d80a5830 con 0x7f40e015ced0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40ddd9b640 1 -- 172.21.15.133:0/3576414112 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d80a5c00 con 0x7f40e015ced0 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40e50fa640 1 -- 172.21.15.133:0/3576414112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e015ced0 msgr2=0x7f40e015d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40e50fa640 1 --2- 172.21.15.133:0/3576414112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e015ced0 0x7f40e015d360 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009d90 tx=0x7f40d801c970 comp rx=0 tx=0).stop 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40e50fa640 1 -- 172.21.15.133:0/3576414112 shutdown_connections 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40e50fa640 1 --2- 172.21.15.133:0/3576414112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e015ced0 0x7f40e015d360 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40e50fa640 1 --2- 172.21.15.133:0/3576414112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40e0145170 0x7f40e006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40e50fa640 1 -- 172.21.15.133:0/3576414112 >> 172.21.15.133:0/3576414112 conn(0x7f40e0066f70 msgr2=0x7f40e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40e50fa640 1 -- 172.21.15.133:0/3576414112 shutdown_connections 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.072+0000 7f40e50fa640 1 -- 172.21.15.133:0/3576414112 wait complete. 2024-04-25T10:14:21.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40e50fa640 1 Processor -- start 2024-04-25T10:14:21.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40e50fa640 1 -- start start 2024-04-25T10:14:21.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40e50fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e006e4d0 0x7f40e0164f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40e50fa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40e0145170 0x7f40e0153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40e50fa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40e015ced0 0x7f40e0160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40e50fa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e0168610 con 0x7f40e006e4d0 2024-04-25T10:14:21.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40e50fa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e0168490 con 0x7f40e0145170 2024-04-25T10:14:21.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40e50fa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e0168790 con 0x7f40e015ced0 2024-04-25T10:14:21.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40ded9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e006e4d0 0x7f40e0164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40ded9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e006e4d0 0x7f40e0164f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55320/0 (socket says 172.21.15.133:55320) 2024-04-25T10:14:21.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40ded9d640 1 -- 172.21.15.133:0/191051124 learned_addr learned my addr 172.21.15.133:0/191051124 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40ded9d640 1 -- 172.21.15.133:0/191051124 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40e015ced0 msgr2=0x7f40e0160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40ded9d640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40e015ced0 0x7f40e0160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40ded9d640 1 -- 172.21.15.133:0/191051124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40e0145170 msgr2=0x7f40e0153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40ded9d640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40e0145170 0x7f40e0153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40ded9d640 1 -- 172.21.15.133:0/191051124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e0163ec0 con 0x7f40e006e4d0 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40ded9d640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e006e4d0 0x7f40e0164f60 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f40d000d830 tx=0x7f40d000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40c77fe640 1 -- 172.21.15.133:0/191051124 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000b6c0 con 0x7f40e006e4d0 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40c77fe640 1 -- 172.21.15.133:0/191051124 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d0019070 con 0x7f40e006e4d0 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40e50fa640 1 -- 172.21.15.133:0/191051124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e01641a0 con 0x7f40e006e4d0 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40c77fe640 1 -- 172.21.15.133:0/191051124 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000ba60 con 0x7f40e006e4d0 2024-04-25T10:14:21.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40e50fa640 1 -- 172.21.15.133:0/191051124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e0164760 con 0x7f40e006e4d0 2024-04-25T10:14:21.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.076+0000 7f40e50fa640 1 -- 172.21.15.133:0/191051124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e0068e80 con 0x7f40e006e4d0 2024-04-25T10:14:21.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.080+0000 7f40c77fe640 1 -- 172.21.15.133:0/191051124 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40d001e460 con 0x7f40e006e4d0 2024-04-25T10:14:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.080+0000 7f40c77fe640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40cc066f60 0x7f40cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.080+0000 7f40c77fe640 1 -- 172.21.15.133:0/191051124 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f40d00855e0 con 0x7f40e006e4d0 2024-04-25T10:14:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.080+0000 7f40de59c640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40cc066f60 0x7f40cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.080+0000 7f40de59c640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40cc066f60 0x7f40cc069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f40d4007fd0 tx=0x7f40d4007940 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.084+0000 7f40c77fe640 1 -- 172.21.15.133:0/191051124 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40d0056710 con 0x7f40e006e4d0 2024-04-25T10:14:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 -- 172.21.15.133:0/191051124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40e0153e50 con 0x7f40e006e4d0 2024-04-25T10:14:21.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40c77fe640 1 -- 172.21.15.133:0/191051124 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40d0056010 con 0x7f40e006e4d0 2024-04-25T10:14:21.246 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:14:21.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 -- 172.21.15.133:0/191051124 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40cc066f60 msgr2=0x7f40cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40cc066f60 0x7f40cc069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f40d4007fd0 tx=0x7f40d4007940 comp rx=0 tx=0).stop 2024-04-25T10:14:21.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 -- 172.21.15.133:0/191051124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e006e4d0 msgr2=0x7f40e0164f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e006e4d0 0x7f40e0164f60 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f40d000d830 tx=0x7f40d000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 -- 172.21.15.133:0/191051124 shutdown_connections 2024-04-25T10:14:21.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f40e015ced0 0x7f40e0160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f40e0145170 0x7f40e0153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f40cc066f60 0x7f40cc069420 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 --2- 172.21.15.133:0/191051124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e006e4d0 0x7f40e0164f60 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 -- 172.21.15.133:0/191051124 >> 172.21.15.133:0/191051124 conn(0x7f40e0066f70 msgr2=0x7f40e015f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 -- 172.21.15.133:0/191051124 shutdown_connections 2024-04-25T10:14:21.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.244+0000 7f40e50fa640 1 -- 172.21.15.133:0/191051124 wait complete. 2024-04-25T10:14:21.265 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:14:21.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f2d77640 1 Processor -- start 2024-04-25T10:14:21.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f2d77640 1 -- start start 2024-04-25T10:14:21.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f2d77640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0b4b70 0x7f16ec15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f2d77640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f16ec0ae410 0x7f16ec0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f2d77640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ec15f080 con 0x7f16ec0b3f40 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f2d77640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ec155150 con 0x7f16ec0ae410 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f2d77640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec15e200 con 0x7f16ec0b4b70 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16ebfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f16ec0b3f40 0x7f16ec15e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45464/0 (socket says 172.21.15.133:45464) 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16ebfff640 1 -- 172.21.15.133:0/1337314497 learned_addr learned my addr 172.21.15.133:0/1337314497 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f1314640 1 --2- 172.21.15.133:0/1337314497 >> v2:172.21.15.162:3300/0 conn(0x7f16ec0ae410 0x7f16ec0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f0b13640 1 --2- 172.21.15.133:0/1337314497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0b4b70 0x7f16ec15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3486141145 0 0) 0x7f16ec15f080 con 0x7f16ec0b3f40 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16d8003530 con 0x7f16ec0b3f40 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 665053474 0 0) 0x7f16d8003530 con 0x7f16ec0b3f40 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0b4b70 msgr2=0x7f16ec15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 --2- 172.21.15.133:0/1337314497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0b4b70 0x7f16ec15ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 >> v2:172.21.15.162:3300/0 conn(0x7f16ec0ae410 msgr2=0x7f16ec0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 --2- 172.21.15.133:0/1337314497 >> v2:172.21.15.162:3300/0 conn(0x7f16ec0ae410 0x7f16ec0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec15b490 con 0x7f16ec0b3f40 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f16dc001680 con 0x7f16ec0b3f40 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 >> v1:172.21.15.133:6789/0 conn(0x7f16ec0b3f40 legacy=0x7f16ec15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 --2- 172.21.15.133:0/1337314497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d80039c0 0x7f16d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 --2- 172.21.15.133:0/1337314497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d8004810 0x7f16d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 --2- 172.21.15.133:0/1337314497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0b4b70 0x7f16d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec155150 con 0x7f16d80039c0 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec15e200 con 0x7f16d8004810 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d8007a30 con 0x7f16ec0b4b70 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f0b13640 1 --2- 172.21.15.133:0/1337314497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d80039c0 0x7f16d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16ebfff640 1 --2- 172.21.15.133:0/1337314497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d8004810 0x7f16d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f0b13640 1 -- 172.21.15.133:0/1337314497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0b4b70 msgr2=0x7f16d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f0b13640 1 --2- 172.21.15.133:0/1337314497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0b4b70 0x7f16d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f0b13640 1 -- 172.21.15.133:0/1337314497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d8004810 msgr2=0x7f16d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f0b13640 1 --2- 172.21.15.133:0/1337314497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d8004810 0x7f16d8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.380+0000 7f16f0b13640 1 -- 172.21.15.133:0/1337314497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d8003530 con 0x7f16d80039c0 2024-04-25T10:14:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16f0b13640 1 --2- 172.21.15.133:0/1337314497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d80039c0 0x7f16d8003de0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009ae0 tx=0x7f16d401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fa941c623d2f0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d40096b0 con 0x7f16d80039c0 2024-04-25T10:14:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d40a6490 con 0x7f16d80039c0 2024-04-25T10:14:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16eb7fe640 1 -- 172.21.15.133:0/1337314497 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d40a6880 con 0x7f16d80039c0 2024-04-25T10:14:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16f2d77640 1 -- 172.21.15.133:0/1337314497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d80039c0 msgr2=0x7f16d8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16f2d77640 1 --2- 172.21.15.133:0/1337314497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d80039c0 0x7f16d8003de0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009ae0 tx=0x7f16d401cab0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16f1314640 1 -- 172.21.15.133:0/1337314497 reap_dead start 2024-04-25T10:14:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16f2d77640 1 -- 172.21.15.133:0/1337314497 shutdown_connections 2024-04-25T10:14:21.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16f2d77640 1 -- 172.21.15.133:0/1337314497 >> 172.21.15.133:0/1337314497 conn(0x7f16ec071aa0 msgr2=0x7f16ec071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16f2d77640 1 -- 172.21.15.133:0/1337314497 shutdown_connections 2024-04-25T10:14:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16f2d77640 1 -- 172.21.15.133:0/1337314497 wait complete. 2024-04-25T10:14:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16f2d77640 1 Processor -- start 2024-04-25T10:14:21.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.384+0000 7f16f2d77640 1 -- start start 2024-04-25T10:14:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f2d77640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d80039c0 0x7f16ec1e6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f2d77640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0ceb40 0x7f16ec153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f2d77640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16ec154050 0x7f16ec1f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f2d77640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec0b2e30 con 0x7f16ec154050 2024-04-25T10:14:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f2d77640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec15b230 con 0x7f16d80039c0 2024-04-25T10:14:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f2d77640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec0b2fb0 con 0x7f16ec0ceb40 2024-04-25T10:14:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f1314640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16ec154050 0x7f16ec1f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f1314640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16ec154050 0x7f16ec1f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55344/0 (socket says 172.21.15.133:55344) 2024-04-25T10:14:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f1314640 1 -- 172.21.15.133:0/975169053 learned_addr learned my addr 172.21.15.133:0/975169053 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f0b13640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d80039c0 0x7f16ec1e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16ebfff640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0ceb40 0x7f16ec153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f1314640 1 -- 172.21.15.133:0/975169053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0ceb40 msgr2=0x7f16ec153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f1314640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0ceb40 0x7f16ec153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f1314640 1 -- 172.21.15.133:0/975169053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d80039c0 msgr2=0x7f16ec1e6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f1314640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d80039c0 0x7f16ec1e6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f1314640 1 -- 172.21.15.133:0/975169053 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec1f38c0 con 0x7f16ec154050 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16ebfff640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0ceb40 0x7f16ec153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f0b13640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d80039c0 0x7f16ec1e6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f1314640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16ec154050 0x7f16ec1f23a0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0b2990 tx=0x7f16e400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16e97fa640 1 -- 172.21.15.133:0/975169053 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4018020 con 0x7f16ec154050 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16e97fa640 1 -- 172.21.15.133:0/975169053 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e400bc80 con 0x7f16ec154050 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16e97fa640 1 -- 172.21.15.133:0/975169053 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4014640 con 0x7f16ec154050 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f2d77640 1 -- 172.21.15.133:0/975169053 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec0cdaa0 con 0x7f16ec154050 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f2d77640 1 -- 172.21.15.133:0/975169053 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec0ce060 con 0x7f16ec154050 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f2d77640 1 -- 172.21.15.133:0/975169053 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ec1544d0 con 0x7f16ec154050 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16e97fa640 1 -- 172.21.15.133:0/975169053 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16e401b020 con 0x7f16ec154050 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16e97fa640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f16cc066f60 0x7f16cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16e97fa640 1 -- 172.21.15.133:0/975169053 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f16e40854f0 con 0x7f16ec154050 2024-04-25T10:14:21.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.388+0000 7f16f0b13640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f16cc066f60 0x7f16cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.392+0000 7f16f0b13640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f16cc066f60 0x7f16cc069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f16d40ad000 tx=0x7f16d40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.392+0000 7f16e97fa640 1 -- 172.21.15.133:0/975169053 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16e4022030 con 0x7f16ec154050 2024-04-25T10:14:21.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:21.608+0000 7f16f2d77640 1 -- 172.21.15.133:0/975169053 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f16ec067c70 con 0x7f16ec154050 2024-04-25T10:14:22.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.600+0000 7f16e97fa640 1 -- 172.21.15.133:0/975169053 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v561) v1 ==== 162+0+0 (secure 0 0 0) 0x7f16e40232b0 con 0x7f16ec154050 2024-04-25T10:14:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 -- 172.21.15.133:0/975169053 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f16cc066f60 msgr2=0x7f16cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f16cc066f60 0x7f16cc069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f16d40ad000 tx=0x7f16d40046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 -- 172.21.15.133:0/975169053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16ec154050 msgr2=0x7f16ec1f23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16ec154050 0x7f16ec1f23a0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0b2990 tx=0x7f16e400b480 comp rx=0 tx=0).stop 2024-04-25T10:14:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 -- 172.21.15.133:0/975169053 shutdown_connections 2024-04-25T10:14:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f16cc066f60 0x7f16cc069420 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16ec154050 0x7f16ec1f23a0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16ec0ceb40 0x7f16ec153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 --2- 172.21.15.133:0/975169053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16d80039c0 0x7f16ec1e6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 -- 172.21.15.133:0/975169053 >> 172.21.15.133:0/975169053 conn(0x7f16ec071aa0 msgr2=0x7f16ec06f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 -- 172.21.15.133:0/975169053 shutdown_connections 2024-04-25T10:14:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.604+0000 7f16f2d77640 1 -- 172.21.15.133:0/975169053 wait complete. 2024-04-25T10:14:22.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:14:22.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa167ed0640 1 Processor -- start 2024-04-25T10:14:22.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa167ed0640 1 -- start start 2024-04-25T10:14:22.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa167ed0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1600ce8e0 0x7fa16006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa167ed0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1600cd6f0 con 0x7fa16006e970 2024-04-25T10:14:22.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa167ed0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1600cf270 con 0x7fa1600ce450 2024-04-25T10:14:22.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa167ed0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1600cd970 con 0x7fa1600ce8e0 2024-04-25T10:14:22.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa165c6c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa16006e970 0x7fa160162e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45466/0 (socket says 172.21.15.133:45466) 2024-04-25T10:14:22.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa165c6c640 1 -- 172.21.15.133:0/1719946064 learned_addr learned my addr 172.21.15.133:0/1719946064 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa16646d640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1600ce8e0 0x7fa16006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510654593 0 0) 0x7fa15c001120 con 0x7fa16006e970 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa14c003530 con 0x7fa16006e970 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4179386095 0 0) 0x7fa14c003530 con 0x7fa16006e970 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1600ce8e0 msgr2=0x7fa16006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1600ce8e0 0x7fa16006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 >> v1:172.21.15.162:6789/0 conn(0x7fa1600ce450 legacy=0x7fa1600b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1600cdc50 con 0x7fa16006e970 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa15c001850 con 0x7fa16006e970 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 >> v1:172.21.15.133:6789/0 conn(0x7fa16006e970 legacy=0x7fa160162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 0x7fa14c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa14c004810 0x7fa14c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1600ce8e0 0x7fa14c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1600cd970 con 0x7fa14c0039f0 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c00bb00 con 0x7fa14c004810 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c00bd00 con 0x7fa1600ce8e0 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa165c6c640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 0x7fa14c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa16546b640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa14c004810 0x7fa14c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa16646d640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1600ce8e0 0x7fa14c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa165c6c640 1 -- 172.21.15.133:0/1719946064 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1600ce8e0 msgr2=0x7fa14c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa165c6c640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1600ce8e0 0x7fa14c005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa165c6c640 1 -- 172.21.15.133:0/1719946064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa14c004810 msgr2=0x7fa14c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa165c6c640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa14c004810 0x7fa14c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.752+0000 7fa165c6c640 1 -- 172.21.15.133:0/1719946064 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c003530 con 0x7fa14c0039f0 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa165c6c640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 0x7fa14c003e10 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c4c0 tx=0x7fa15c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948c01c815643fba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00c0e0 con 0x7fa14c0039f0 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c0a9070 con 0x7fa14c0039f0 2024-04-25T10:14:22.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa164c6a640 1 -- 172.21.15.133:0/1719946064 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c005d40 con 0x7fa14c0039f0 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa167ed0640 1 -- 172.21.15.133:0/1719946064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 msgr2=0x7fa14c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa167ed0640 1 --2- 172.21.15.133:0/1719946064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 0x7fa14c003e10 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c4c0 tx=0x7fa15c01cc80 comp rx=0 tx=0).stop 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa16646d640 1 -- 172.21.15.133:0/1719946064 reap_dead start 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa167ed0640 1 -- 172.21.15.133:0/1719946064 shutdown_connections 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa167ed0640 1 -- 172.21.15.133:0/1719946064 >> 172.21.15.133:0/1719946064 conn(0x7fa160066f70 msgr2=0x7fa16006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa167ed0640 1 -- 172.21.15.133:0/1719946064 shutdown_connections 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa167ed0640 1 -- 172.21.15.133:0/1719946064 wait complete. 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa167ed0640 1 Processor -- start 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.756+0000 7fa167ed0640 1 -- start start 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa167ed0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 0x7fa160142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa167ed0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa16015f980 0x7fa16015fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa167ed0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa160155470 0x7fa1601558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa167ed0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16006dc30 con 0x7fa14c0039f0 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa167ed0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16006dab0 con 0x7fa160155470 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa167ed0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16015f030 con 0x7fa16015f980 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa165c6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 0x7fa160142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa165c6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 0x7fa160142740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55366/0 (socket says 172.21.15.133:55366) 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa165c6c640 1 -- 172.21.15.133:0/2771855971 learned_addr learned my addr 172.21.15.133:0/2771855971 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa16546b640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa16015f980 0x7fa16015fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa16646d640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa160155470 0x7fa1601558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa165c6c640 1 -- 172.21.15.133:0/2771855971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa16015f980 msgr2=0x7fa16015fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa165c6c640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa16015f980 0x7fa16015fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa165c6c640 1 -- 172.21.15.133:0/2771855971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa160155470 msgr2=0x7fa1601558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa165c6c640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa160155470 0x7fa1601558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa165c6c640 1 -- 172.21.15.133:0/2771855971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1601614e0 con 0x7fa14c0039f0 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa165c6c640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 0x7fa160142740 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fa15c01c6f0 tx=0x7fa14c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa1567fc640 1 -- 172.21.15.133:0/2771855971 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0bd070 con 0x7fa14c0039f0 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa1567fc640 1 -- 172.21.15.133:0/2771855971 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c005d20 con 0x7fa14c0039f0 2024-04-25T10:14:22.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa167ed0640 1 -- 172.21.15.133:0/2771855971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa160160440 con 0x7fa14c0039f0 2024-04-25T10:14:22.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa1567fc640 1 -- 172.21.15.133:0/2771855971 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c001610 con 0x7fa14c0039f0 2024-04-25T10:14:22.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa167ed0640 1 -- 172.21.15.133:0/2771855971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa160160a00 con 0x7fa14c0039f0 2024-04-25T10:14:22.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa167ed0640 1 -- 172.21.15.133:0/2771855971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa160068e80 con 0x7fa14c0039f0 2024-04-25T10:14:22.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa1567fc640 1 -- 172.21.15.133:0/2771855971 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa15c0c0070 con 0x7fa14c0039f0 2024-04-25T10:14:22.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa1567fc640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa144066f60 0x7fa144069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa1567fc640 1 -- 172.21.15.133:0/2771855971 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa15c11e020 con 0x7fa14c0039f0 2024-04-25T10:14:22.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa16546b640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa144066f60 0x7fa144069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.760+0000 7fa16546b640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa144066f60 0x7fa144069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa160154cb0 tx=0x7fa150009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.764+0000 7fa1567fc640 1 -- 172.21.15.133:0/2771855971 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1600cd6f0 con 0x7fa14c0039f0 2024-04-25T10:14:22.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.916+0000 7fa167ed0640 1 -- 172.21.15.133:0/2771855971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa16006b960 con 0x7fa14c0039f0 2024-04-25T10:14:22.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.924+0000 7fa1567fc640 1 -- 172.21.15.133:0/2771855971 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa15c0ef1e0 con 0x7fa14c0039f0 2024-04-25T10:14:22.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.924+0000 7fa167ed0640 1 -- 172.21.15.133:0/2771855971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa144066f60 msgr2=0x7fa144069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.924+0000 7fa167ed0640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa144066f60 0x7fa144069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa160154cb0 tx=0x7fa150009040 comp rx=0 tx=0).stop 2024-04-25T10:14:22.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.924+0000 7fa167ed0640 1 -- 172.21.15.133:0/2771855971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 msgr2=0x7fa160142740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.924+0000 7fa167ed0640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 0x7fa160142740 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fa15c01c6f0 tx=0x7fa14c005840 comp rx=0 tx=0).stop 2024-04-25T10:14:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.924+0000 7fa167ed0640 1 -- 172.21.15.133:0/2771855971 shutdown_connections 2024-04-25T10:14:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.924+0000 7fa167ed0640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa160155470 0x7fa1601558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.924+0000 7fa167ed0640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa16015f980 0x7fa16015fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.924+0000 7fa167ed0640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa144066f60 0x7fa144069420 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.924+0000 7fa167ed0640 1 --2- 172.21.15.133:0/2771855971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14c0039f0 0x7fa160142740 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.928+0000 7fa167ed0640 1 -- 172.21.15.133:0/2771855971 >> 172.21.15.133:0/2771855971 conn(0x7fa160066f70 msgr2=0x7fa1600b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.928+0000 7fa167ed0640 1 -- 172.21.15.133:0/2771855971 shutdown_connections 2024-04-25T10:14:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:22.928+0000 7fa167ed0640 1 -- 172.21.15.133:0/2771855971 wait complete. 2024-04-25T10:14:22.945 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:14:22.945 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:22.986 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:23.028 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:14:23.028 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:14:23.029 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:23.033 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:23.034 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:14:23.034 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:23.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:23.075 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:14:23.075 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:23.080 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:23.081 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:14:23.081 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:23.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:23.127 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:14:23.127 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:23.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa00906a640 1 Processor -- start 2024-04-25T10:14:23.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa00906a640 1 -- start start 2024-04-25T10:14:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa00906a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa00415d9b0 0x7fa0040cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa00906a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa00415dd90 0x7fa0040ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa00906a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0040b19f0 0x7fa0040cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa00906a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0040cbe30 con 0x7fa00415d9b0 2024-04-25T10:14:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa00906a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0040ca9b0 con 0x7fa00415dd90 2024-04-25T10:14:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa00906a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0040cac90 con 0x7fa0040b19f0 2024-04-25T10:14:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa002d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa00415d9b0 0x7fa0040cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa002d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa00415d9b0 0x7fa0040cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55378/0 (socket says 172.21.15.133:55378) 2024-04-25T10:14:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa002d9d640 1 -- 172.21.15.133:0/2040014635 learned_addr learned my addr 172.21.15.133:0/2040014635 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa00359e640 1 --2- 172.21.15.133:0/2040014635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0040b19f0 0x7fa0040cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7f9ffa59c640 1 --2- 172.21.15.133:0/2040014635 >> v2:172.21.15.162:3300/0 conn(0x7fa00415dd90 0x7fa0040ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa002d9d640 1 -- 172.21.15.133:0/2040014635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0040b19f0 msgr2=0x7fa0040cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa002d9d640 1 --2- 172.21.15.133:0/2040014635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0040b19f0 0x7fa0040cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa002d9d640 1 -- 172.21.15.133:0/2040014635 >> v2:172.21.15.162:3300/0 conn(0x7fa00415dd90 msgr2=0x7fa0040ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa002d9d640 1 --2- 172.21.15.133:0/2040014635 >> v2:172.21.15.162:3300/0 conn(0x7fa00415dd90 0x7fa0040ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa002d9d640 1 -- 172.21.15.133:0/2040014635 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0040caf70 con 0x7fa00415d9b0 2024-04-25T10:14:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7f9ffa59c640 1 --2- 172.21.15.133:0/2040014635 >> v2:172.21.15.162:3300/0 conn(0x7fa00415dd90 0x7fa0040ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:23.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.248+0000 7fa00359e640 1 --2- 172.21.15.133:0/2040014635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0040b19f0 0x7fa0040cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:23.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa002d9d640 1 --2- 172.21.15.133:0/2040014635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa00415d9b0 0x7fa0040cb5e0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fa0040cc000 tx=0x7f9ff401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452fc1806fd3fe36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00259c640 1 -- 172.21.15.133:0/2040014635 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff40ab610 con 0x7fa00415d9b0 2024-04-25T10:14:23.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00259c640 1 -- 172.21.15.133:0/2040014635 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff40b0070 con 0x7fa00415d9b0 2024-04-25T10:14:23.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00259c640 1 -- 172.21.15.133:0/2040014635 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff40ab9d0 con 0x7fa00415d9b0 2024-04-25T10:14:23.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00906a640 1 -- 172.21.15.133:0/2040014635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa00415d9b0 msgr2=0x7fa0040cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00906a640 1 --2- 172.21.15.133:0/2040014635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa00415d9b0 0x7fa0040cb5e0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7fa0040cc000 tx=0x7f9ff401c430 comp rx=0 tx=0).stop 2024-04-25T10:14:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00906a640 1 -- 172.21.15.133:0/2040014635 shutdown_connections 2024-04-25T10:14:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00906a640 1 --2- 172.21.15.133:0/2040014635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0040b19f0 0x7fa0040cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00906a640 1 --2- 172.21.15.133:0/2040014635 >> v2:172.21.15.162:3300/0 conn(0x7fa00415dd90 0x7fa0040ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00906a640 1 --2- 172.21.15.133:0/2040014635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa00415d9b0 0x7fa0040cb5e0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00906a640 1 -- 172.21.15.133:0/2040014635 >> 172.21.15.133:0/2040014635 conn(0x7fa004067000 msgr2=0x7fa00406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00906a640 1 -- 172.21.15.133:0/2040014635 shutdown_connections 2024-04-25T10:14:23.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00906a640 1 -- 172.21.15.133:0/2040014635 wait complete. 2024-04-25T10:14:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00906a640 1 Processor -- start 2024-04-25T10:14:23.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.252+0000 7fa00906a640 1 -- start start 2024-04-25T10:14:23.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa00906a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0040b19f0 0x7fa00415e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa00906a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa00415d9b0 0x7fa0041fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa00906a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa00415dd90 0x7fa0041432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa00906a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa004162290 con 0x7fa0040b19f0 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa00906a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa004162110 con 0x7fa00415dd90 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa00906a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa004162410 con 0x7fa00415d9b0 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa002d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0040b19f0 0x7fa00415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa002d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0040b19f0 0x7fa00415e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55394/0 (socket says 172.21.15.133:55394) 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa002d9d640 1 -- 172.21.15.133:0/108549861 learned_addr learned my addr 172.21.15.133:0/108549861 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa002d9d640 1 -- 172.21.15.133:0/108549861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa00415d9b0 msgr2=0x7fa0041fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa00359e640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa00415dd90 0x7fa0041432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7f9ffa59c640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa00415d9b0 0x7fa0041fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa002d9d640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa00415d9b0 0x7fa0041fa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa002d9d640 1 -- 172.21.15.133:0/108549861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa00415dd90 msgr2=0x7fa0041432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa002d9d640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa00415dd90 0x7fa0041432c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa002d9d640 1 -- 172.21.15.133:0/108549861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00414e970 con 0x7fa0040b19f0 2024-04-25T10:14:23.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa002d9d640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0040b19f0 0x7fa00415e170 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f9ff401ab40 tx=0x7f9ff401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7f9ffbfff640 1 -- 172.21.15.133:0/108549861 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff40b48c0 con 0x7fa0040b19f0 2024-04-25T10:14:23.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7f9ffbfff640 1 -- 172.21.15.133:0/108549861 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff40b4a80 con 0x7fa0040b19f0 2024-04-25T10:14:23.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7f9ffbfff640 1 -- 172.21.15.133:0/108549861 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff40be3f0 con 0x7fa0040b19f0 2024-04-25T10:14:23.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa00906a640 1 -- 172.21.15.133:0/108549861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0040cc3c0 con 0x7fa0040b19f0 2024-04-25T10:14:23.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa00906a640 1 -- 172.21.15.133:0/108549861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00414ec50 con 0x7fa0040b19f0 2024-04-25T10:14:23.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7fa00906a640 1 -- 172.21.15.133:0/108549861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa004068ef0 con 0x7fa0040b19f0 2024-04-25T10:14:23.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7f9ffbfff640 1 -- 172.21.15.133:0/108549861 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ff40c7440 con 0x7fa0040b19f0 2024-04-25T10:14:23.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7f9ffbfff640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fe4066f60 0x7f9fe4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7f9ffa59c640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fe4066f60 0x7f9fe4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7f9ffbfff640 1 -- 172.21.15.133:0/108549861 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f9ff411f980 con 0x7fa0040b19f0 2024-04-25T10:14:23.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.256+0000 7f9ffa59c640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fe4066f60 0x7f9fe4069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9fec006fd0 tx=0x7f9fec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.260+0000 7f9ffbfff640 1 -- 172.21.15.133:0/108549861 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ff40a8050 con 0x7fa0040b19f0 2024-04-25T10:14:23.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.452+0000 7fa00906a640 1 -- 172.21.15.133:0/108549861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa00406f060 con 0x7fa0040b19f0 2024-04-25T10:14:23.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.452+0000 7f9ffbfff640 1 -- 172.21.15.133:0/108549861 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+2914 (secure 0 0 0) 0x7f9ff4016110 con 0x7fa0040b19f0 2024-04-25T10:14:23.456 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:23.456 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":85,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1739399697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1739399697},{"type":"v1","addr":"172.21.15.162:6833","nonce":1739399697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/740779706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":740779706},{"type":"v1","addr":"172.21.15.162:6835","nonce":740779706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9960,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3779816148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3779816148},{"type":"v1","addr":"172.21.15.133:6835","nonce":3779816148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[]} 2024-04-25T10:14:23.457 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 85 2024-04-25T10:14:23.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 -- 172.21.15.133:0/108549861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fe4066f60 msgr2=0x7f9fe4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fe4066f60 0x7f9fe4069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9fec006fd0 tx=0x7f9fec008040 comp rx=0 tx=0).stop 2024-04-25T10:14:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 -- 172.21.15.133:0/108549861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0040b19f0 msgr2=0x7fa00415e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0040b19f0 0x7fa00415e170 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f9ff401ab40 tx=0x7f9ff401acd0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 -- 172.21.15.133:0/108549861 shutdown_connections 2024-04-25T10:14:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa00415dd90 0x7fa0041432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa00415d9b0 0x7fa0041fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9fe4066f60 0x7f9fe4069420 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 --2- 172.21.15.133:0/108549861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0040b19f0 0x7fa00415e170 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 -- 172.21.15.133:0/108549861 >> 172.21.15.133:0/108549861 conn(0x7fa004067000 msgr2=0x7fa00406e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 -- 172.21.15.133:0/108549861 shutdown_connections 2024-04-25T10:14:23.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.456+0000 7fa00906a640 1 -- 172.21.15.133:0/108549861 wait complete. 2024-04-25T10:14:23.474 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:14:23.474 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:14:23.475 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:14:23.475 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:23.475 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:14:23.475 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:14:23.475 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:14:23.476 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:14:23.476 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:14:23.476 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:23.476 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:14:23.476 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:14:23.476 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:14:23.476 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:23.478 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:14:23.478 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:23.484 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:14:23.484 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:14:23.489 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:14:23.489 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:14:23.493 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:14:23.493 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:14:23.510 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:23.530 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:14:23.530 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:23.531 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:23.535 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:14:23.536 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:23.537 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:14:23.537 DEBUG:teuthology.parallel:result is None 2024-04-25T10:14:23.537 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:14:23.540 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:14:23.588 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:14:23.605 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:14:23.612 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:14:23.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f8459a32640 1 Processor -- start 2024-04-25T10:14:23.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f8459a32640 1 -- start start 2024-04-25T10:14:23.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f8459a32640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f84540ce600 0x7f84540cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f8459a32640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f84540d1920 0x7f84540d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f8459a32640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845415ebd0 con 0x7f845415d9a0 2024-04-25T10:14:23.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f8459a32640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845415eeb0 con 0x7f84540d1920 2024-04-25T10:14:23.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f8459a32640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84540d0330 con 0x7f84540ce600 2024-04-25T10:14:23.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f84537fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f84540d1920 0x7f84540d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f8452ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f845415d9a0 0x7f84540d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45498/0 (socket says 172.21.15.133:45498) 2024-04-25T10:14:23.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f8452ffd640 1 -- 172.21.15.133:0/3041020306 learned_addr learned my addr 172.21.15.133:0/3041020306 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:23.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f84527fc640 1 --2- 172.21.15.133:0/3041020306 >> v2:172.21.15.162:3300/0 conn(0x7f84540ce600 0x7f84540cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f84537fe640 1 -- 172.21.15.133:0/3041020306 >> v2:172.21.15.162:3300/0 conn(0x7f84540ce600 msgr2=0x7f84540cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:23.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f84537fe640 1 --2- 172.21.15.133:0/3041020306 >> v2:172.21.15.162:3300/0 conn(0x7f84540ce600 0x7f84540cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f84537fe640 1 -- 172.21.15.133:0/3041020306 >> v1:172.21.15.133:6789/0 conn(0x7f845415d9a0 legacy=0x7f84540d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f84537fe640 1 -- 172.21.15.133:0/3041020306 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84540d0610 con 0x7f84540d1920 2024-04-25T10:14:23.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f8451ffb640 1 -- 172.21.15.133:0/3041020306 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143238405 0 0) 0x7f845415ebd0 con 0x7f845415d9a0 2024-04-25T10:14:23.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.648+0000 7f84537fe640 1 --2- 172.21.15.133:0/3041020306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540d1920 0x7f84540d1d70 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f84540ce080 tx=0x7f844c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d38b049d49cfb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8451ffb640 1 -- 172.21.15.133:0/3041020306 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0b2070 con 0x7f84540d1920 2024-04-25T10:14:23.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8451ffb640 1 -- 172.21.15.133:0/3041020306 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c0a5780 con 0x7f84540d1920 2024-04-25T10:14:23.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8451ffb640 1 -- 172.21.15.133:0/3041020306 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0a5b50 con 0x7f84540d1920 2024-04-25T10:14:23.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 -- 172.21.15.133:0/3041020306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540d1920 msgr2=0x7f84540d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 --2- 172.21.15.133:0/3041020306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540d1920 0x7f84540d1d70 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f84540ce080 tx=0x7f844c01c8a0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 -- 172.21.15.133:0/3041020306 shutdown_connections 2024-04-25T10:14:23.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 --2- 172.21.15.133:0/3041020306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540d1920 0x7f84540d1d70 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 --2- 172.21.15.133:0/3041020306 >> v2:172.21.15.162:3300/0 conn(0x7f84540ce600 0x7f84540cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 -- 172.21.15.133:0/3041020306 >> 172.21.15.133:0/3041020306 conn(0x7f845406bf80 msgr2=0x7f845406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:23.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 -- 172.21.15.133:0/3041020306 shutdown_connections 2024-04-25T10:14:23.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 -- 172.21.15.133:0/3041020306 wait complete. 2024-04-25T10:14:23.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 Processor -- start 2024-04-25T10:14:23.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 -- start start 2024-04-25T10:14:23.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540ce600 0x7f8454155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84540d1920 0x7f84540b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f845415d9a0 0x7f8454158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845415e960 con 0x7f84540ce600 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845415e7e0 con 0x7f84540d1920 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845416a7e0 con 0x7f845415d9a0 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8452ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540ce600 0x7f8454155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8452ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540ce600 0x7f8454155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55444/0 (socket says 172.21.15.133:55444) 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8452ffd640 1 -- 172.21.15.133:0/1825259254 learned_addr learned my addr 172.21.15.133:0/1825259254 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8452ffd640 1 -- 172.21.15.133:0/1825259254 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f845415d9a0 msgr2=0x7f8454158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8452ffd640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f845415d9a0 0x7f8454158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8452ffd640 1 -- 172.21.15.133:0/1825259254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84540d1920 msgr2=0x7f84540b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8452ffd640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84540d1920 0x7f84540b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8452ffd640 1 -- 172.21.15.133:0/1825259254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84540b3220 con 0x7f84540ce600 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8452ffd640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540ce600 0x7f8454155520 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f84440011f0 tx=0x7f844400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f843bfff640 1 -- 172.21.15.133:0/1825259254 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444018070 con 0x7f84540ce600 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f843bfff640 1 -- 172.21.15.133:0/1825259254 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84440074e0 con 0x7f84540ce600 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 -- 172.21.15.133:0/1825259254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84540b3500 con 0x7f84540ce600 2024-04-25T10:14:23.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f843bfff640 1 -- 172.21.15.133:0/1825259254 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444004a10 con 0x7f84540ce600 2024-04-25T10:14:23.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.652+0000 7f8459a32640 1 -- 172.21.15.133:0/1825259254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84540b3ac0 con 0x7f84540ce600 2024-04-25T10:14:23.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.656+0000 7f8459a32640 1 -- 172.21.15.133:0/1825259254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8454068e80 con 0x7f84540ce600 2024-04-25T10:14:23.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.656+0000 7f843bfff640 1 -- 172.21.15.133:0/1825259254 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f844401b070 con 0x7f84540ce600 2024-04-25T10:14:23.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.656+0000 7f843bfff640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8440066f60 0x7f8440069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:23.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.656+0000 7f843bfff640 1 -- 172.21.15.133:0/1825259254 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8444084230 con 0x7f84540ce600 2024-04-25T10:14:23.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.656+0000 7f84527fc640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8440066f60 0x7f8440069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:23.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.656+0000 7f84527fc640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8440066f60 0x7f8440069420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8454158a50 tx=0x7f8448008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:23.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.660+0000 7f843bfff640 1 -- 172.21.15.133:0/1825259254 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84440553f0 con 0x7f84540ce600 2024-04-25T10:14:23.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:23.828+0000 7f8459a32640 1 -- 172.21.15.133:0/1825259254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8454156ec0 con 0x7f84540ce600 2024-04-25T10:14:24.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.611+0000 7f843bfff640 1 -- 172.21.15.133:0/1825259254 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v138) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8454156ec0 con 0x7f84540ce600 2024-04-25T10:14:24.614 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:14:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 -- 172.21.15.133:0/1825259254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8440066f60 msgr2=0x7f8440069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8440066f60 0x7f8440069420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8454158a50 tx=0x7f8448008040 comp rx=0 tx=0).stop 2024-04-25T10:14:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 -- 172.21.15.133:0/1825259254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540ce600 msgr2=0x7f8454155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540ce600 0x7f8454155520 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f84440011f0 tx=0x7f844400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 -- 172.21.15.133:0/1825259254 shutdown_connections 2024-04-25T10:14:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f845415d9a0 0x7f8454158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84540d1920 0x7f84540b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8440066f60 0x7f8440069420 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 --2- 172.21.15.133:0/1825259254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540ce600 0x7f8454155520 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 -- 172.21.15.133:0/1825259254 >> 172.21.15.133:0/1825259254 conn(0x7f845406bf80 msgr2=0x7f8454167f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:24.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 -- 172.21.15.133:0/1825259254 shutdown_connections 2024-04-25T10:14:24.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.615+0000 7f8459a32640 1 -- 172.21.15.133:0/1825259254 wait complete. 2024-04-25T10:14:24.637 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:14:24.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.799+0000 7fea52415640 1 Processor -- start 2024-04-25T10:14:24.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.799+0000 7fea52415640 1 -- start start 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea52415640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea4c0b1f70 0x7fea4c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea52415640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fea4c0ce330 0x7fea4c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea52415640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c0c61d0 con 0x7fea4c15c9a0 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea52415640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c0cece0 con 0x7fea4c0ce330 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea52415640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0c64b0 con 0x7fea4c0b1f70 2024-04-25T10:14:24.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fea4c15c9a0 0x7fea4c0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45506/0 (socket says 172.21.15.133:45506) 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4bfff640 1 -- 172.21.15.133:0/2987171771 learned_addr learned my addr 172.21.15.133:0/2987171771 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508636761 0 0) 0x7fea4c0c61d0 con 0x7fea4c15c9a0 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea38003530 con 0x7fea4c15c9a0 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea509b2640 1 --2- 172.21.15.133:0/2987171771 >> v2:172.21.15.162:3300/0 conn(0x7fea4c0ce330 0x7fea4c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea437fe640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea4c0b1f70 0x7fea4c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3379365878 0 0) 0x7fea38003530 con 0x7fea4c15c9a0 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea4c0b1f70 msgr2=0x7fea4c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea4c0b1f70 0x7fea4c06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 >> v2:172.21.15.162:3300/0 conn(0x7fea4c0ce330 msgr2=0x7fea4c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 --2- 172.21.15.133:0/2987171771 >> v2:172.21.15.162:3300/0 conn(0x7fea4c0ce330 0x7fea4c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c06d4f0 con 0x7fea4c15c9a0 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fea3c001680 con 0x7fea4c15c9a0 2024-04-25T10:14:24.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 >> v1:172.21.15.133:6789/0 conn(0x7fea4c15c9a0 legacy=0x7fea4c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea380039c0 0x7fea38003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38004810 0x7fea38004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea4c0b1f70 0x7fea380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0cece0 con 0x7fea38004810 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0c64b0 con 0x7fea380039c0 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3800ba80 con 0x7fea4c0b1f70 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea437fe640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38004810 0x7fea38004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea4bfff640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea380039c0 0x7fea38003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea509b2640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea4c0b1f70 0x7fea380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea437fe640 1 -- 172.21.15.133:0/2987171771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea4c0b1f70 msgr2=0x7fea380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea437fe640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea4c0b1f70 0x7fea380051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea437fe640 1 -- 172.21.15.133:0/2987171771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea380039c0 msgr2=0x7fea38003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea437fe640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea380039c0 0x7fea38003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.803+0000 7fea437fe640 1 -- 172.21.15.133:0/2987171771 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea38003530 con 0x7fea38004810 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea437fe640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38004810 0x7fea38004c90 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fea3400ba00 tx=0x7fea3401c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cade4ff86732a7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea340b2070 con 0x7fea38004810 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea340ad640 con 0x7fea38004810 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea4b7fe640 1 -- 172.21.15.133:0/2987171771 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3400b600 con 0x7fea38004810 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 -- 172.21.15.133:0/2987171771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38004810 msgr2=0x7fea38004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 --2- 172.21.15.133:0/2987171771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea38004810 0x7fea38004c90 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fea3400ba00 tx=0x7fea3401c640 comp rx=0 tx=0).stop 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea509b2640 1 -- 172.21.15.133:0/2987171771 reap_dead start 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 -- 172.21.15.133:0/2987171771 shutdown_connections 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 -- 172.21.15.133:0/2987171771 >> 172.21.15.133:0/2987171771 conn(0x7fea4c067000 msgr2=0x7fea4c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:24.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 -- 172.21.15.133:0/2987171771 shutdown_connections 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 -- 172.21.15.133:0/2987171771 wait complete. 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 Processor -- start 2024-04-25T10:14:24.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 -- start start 2024-04-25T10:14:24.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea38004810 0x7fea4c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c157a70 0x7fea4c157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c0cc120 0x7fea4c0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c15cf00 con 0x7fea4c157a70 2024-04-25T10:14:24.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c15cd80 con 0x7fea4c0cc120 2024-04-25T10:14:24.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea52415640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c06dcf0 con 0x7fea38004810 2024-04-25T10:14:24.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea437fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c157a70 0x7fea4c157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea437fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c157a70 0x7fea4c157ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55460/0 (socket says 172.21.15.133:55460) 2024-04-25T10:14:24.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea437fe640 1 -- 172.21.15.133:0/2457329419 learned_addr learned my addr 172.21.15.133:0/2457329419 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:24.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.807+0000 7fea4bfff640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea38004810 0x7fea4c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea509b2640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c0cc120 0x7fea4c0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea437fe640 1 -- 172.21.15.133:0/2457329419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea38004810 msgr2=0x7fea4c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea437fe640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea38004810 0x7fea4c0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea437fe640 1 -- 172.21.15.133:0/2457329419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c0cc120 msgr2=0x7fea4c0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea437fe640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c0cc120 0x7fea4c0cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea437fe640 1 -- 172.21.15.133:0/2457329419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c0cce50 con 0x7fea4c157a70 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea4bfff640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea38004810 0x7fea4c0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea509b2640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c0cc120 0x7fea4c0cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea437fe640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c157a70 0x7fea4c157ed0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fea340b0040 tx=0x7fea38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea497fa640 1 -- 172.21.15.133:0/2457329419 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea340bc030 con 0x7fea4c157a70 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea497fa640 1 -- 172.21.15.133:0/2457329419 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea340ad440 con 0x7fea4c157a70 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea52415640 1 -- 172.21.15.133:0/2457329419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c0cd130 con 0x7fea4c157a70 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea497fa640 1 -- 172.21.15.133:0/2457329419 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea340b2040 con 0x7fea4c157a70 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea52415640 1 -- 172.21.15.133:0/2457329419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c0cd6f0 con 0x7fea4c157a70 2024-04-25T10:14:24.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea52415640 1 -- 172.21.15.133:0/2457329419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c068ef0 con 0x7fea4c157a70 2024-04-25T10:14:24.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea497fa640 1 -- 172.21.15.133:0/2457329419 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea340bf070 con 0x7fea4c157a70 2024-04-25T10:14:24.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea497fa640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fea2c066f60 0x7fea2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea497fa640 1 -- 172.21.15.133:0/2457329419 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fea3411f2b0 con 0x7fea4c157a70 2024-04-25T10:14:24.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea4bfff640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fea2c066f60 0x7fea2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.811+0000 7fea4bfff640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fea2c066f60 0x7fea2c069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fea3c001060 tx=0x7fea3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:24.815+0000 7fea497fa640 1 -- 172.21.15.133:0/2457329419 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea34016350 con 0x7fea4c157a70 2024-04-25T10:14:25.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.031+0000 7fea52415640 1 -- 172.21.15.133:0/2457329419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fea4c15dfd0 con 0x7fea4c157a70 2024-04-25T10:14:25.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.031+0000 7fea497fa640 1 -- 172.21.15.133:0/2457329419 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fea340f0430 con 0x7fea4c157a70 2024-04-25T10:14:25.036 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:25.036 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:14:25.036 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:14:25.036 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.036 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:14:25.036 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:14:25.036 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.036 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:14:25.036 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:14:25.036 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.037 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.039 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.041 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:14:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:14:25.044 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.045 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:14:25.046 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.047 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:14:25.048 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:14:25.049 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.049 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.049 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:14:25.049 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 -- 172.21.15.133:0/2457329419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fea2c066f60 msgr2=0x7fea2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fea2c066f60 0x7fea2c069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fea3c001060 tx=0x7fea3c009040 comp rx=0 tx=0).stop 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 -- 172.21.15.133:0/2457329419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c157a70 msgr2=0x7fea4c157ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c157a70 0x7fea4c157ed0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fea340b0040 tx=0x7fea38005840 comp rx=0 tx=0).stop 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 -- 172.21.15.133:0/2457329419 shutdown_connections 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c0cc120 0x7fea4c0cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fea2c066f60 0x7fea2c069420 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c157a70 0x7fea4c157ed0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 --2- 172.21.15.133:0/2457329419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fea38004810 0x7fea4c0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 -- 172.21.15.133:0/2457329419 >> 172.21.15.133:0/2457329419 conn(0x7fea4c067000 msgr2=0x7fea4c15e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 -- 172.21.15.133:0/2457329419 shutdown_connections 2024-04-25T10:14:25.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.035+0000 7fea52415640 1 -- 172.21.15.133:0/2457329419 wait complete. 2024-04-25T10:14:25.055 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:25.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.167+0000 7fd2cacef640 1 Processor -- start 2024-04-25T10:14:25.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.167+0000 7fd2cacef640 1 -- start start 2024-04-25T10:14:25.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2cacef640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2c40ce330 0x7fd2c40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2cacef640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c40c61d0 con 0x7fd2c415c9a0 2024-04-25T10:14:25.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2cacef640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c40c64b0 con 0x7fd2c40b1f70 2024-04-25T10:14:25.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2cacef640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c40cecc0 con 0x7fd2c40ce330 2024-04-25T10:14:25.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c928c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2c40ce330 0x7fd2c40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c8a8b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd2c415c9a0 0x7fd2c40c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45512/0 (socket says 172.21.15.133:45512) 2024-04-25T10:14:25.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c8a8b640 1 -- 172.21.15.133:0/1600382574 learned_addr learned my addr 172.21.15.133:0/1600382574 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:25.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c37fe640 1 -- 172.21.15.133:0/1600382574 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027701999 0 0) 0x7fd2c40c61d0 con 0x7fd2c415c9a0 2024-04-25T10:14:25.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c37fe640 1 -- 172.21.15.133:0/1600382574 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ac003530 con 0x7fd2c415c9a0 2024-04-25T10:14:25.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c928c640 1 -- 172.21.15.133:0/1600382574 >> v1:172.21.15.162:6789/0 conn(0x7fd2c40b1f70 legacy=0x7fd2c406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:25.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c928c640 1 -- 172.21.15.133:0/1600382574 >> v1:172.21.15.133:6789/0 conn(0x7fd2c415c9a0 legacy=0x7fd2c40c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c928c640 1 -- 172.21.15.133:0/1600382574 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c406d4f0 con 0x7fd2c40ce330 2024-04-25T10:14:25.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c928c640 1 --2- 172.21.15.133:0/1600382574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c40ce330 0x7fd2c40ce780 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009d90 tx=0x7fd2bc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e9e291ec9006b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c37fe640 1 -- 172.21.15.133:0/1600382574 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0b2070 con 0x7fd2c40ce330 2024-04-25T10:14:25.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c37fe640 1 -- 172.21.15.133:0/1600382574 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc0a5830 con 0x7fd2c40ce330 2024-04-25T10:14:25.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.171+0000 7fd2c37fe640 1 -- 172.21.15.133:0/1600382574 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0a5c00 con 0x7fd2c40ce330 2024-04-25T10:14:25.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- 172.21.15.133:0/1600382574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c40ce330 msgr2=0x7fd2c40ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 --2- 172.21.15.133:0/1600382574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c40ce330 0x7fd2c40ce780 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009d90 tx=0x7fd2bc01c970 comp rx=0 tx=0).stop 2024-04-25T10:14:25.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- 172.21.15.133:0/1600382574 shutdown_connections 2024-04-25T10:14:25.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 --2- 172.21.15.133:0/1600382574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c40ce330 0x7fd2c40ce780 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- 172.21.15.133:0/1600382574 >> 172.21.15.133:0/1600382574 conn(0x7fd2c4067000 msgr2=0x7fd2c406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- 172.21.15.133:0/1600382574 shutdown_connections 2024-04-25T10:14:25.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- 172.21.15.133:0/1600382574 wait complete. 2024-04-25T10:14:25.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 Processor -- start 2024-04-25T10:14:25.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- start start 2024-04-25T10:14:25.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c40b1f70 0x7fd2c4163910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c40ce330 0x7fd2c4166f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2c415c9a0 0x7fd2c4155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c415dc20 con 0x7fd2c40b1f70 2024-04-25T10:14:25.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c41695d0 con 0x7fd2c40ce330 2024-04-25T10:14:25.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c4153eb0 con 0x7fd2c415c9a0 2024-04-25T10:14:25.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c8a8b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c40b1f70 0x7fd2c4163910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c8a8b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c40b1f70 0x7fd2c4163910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55472/0 (socket says 172.21.15.133:55472) 2024-04-25T10:14:25.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c8a8b640 1 -- 172.21.15.133:0/1775072176 learned_addr learned my addr 172.21.15.133:0/1775072176 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c3fff640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c40ce330 0x7fd2c4166f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c8a8b640 1 -- 172.21.15.133:0/1775072176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2c415c9a0 msgr2=0x7fd2c4155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c928c640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2c415c9a0 0x7fd2c4155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c8a8b640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2c415c9a0 0x7fd2c4155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c8a8b640 1 -- 172.21.15.133:0/1775072176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c40ce330 msgr2=0x7fd2c4166f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c8a8b640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c40ce330 0x7fd2c4166f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c8a8b640 1 -- 172.21.15.133:0/1775072176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c4154170 con 0x7fd2c40b1f70 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c3fff640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c40ce330 0x7fd2c4166f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c8a8b640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c40b1f70 0x7fd2c4163910 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400c500 tx=0x7fd2b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c17fa640 1 -- 172.21.15.133:0/1775072176 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4017070 con 0x7fd2c40b1f70 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c17fa640 1 -- 172.21.15.133:0/1775072176 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b4005cf0 con 0x7fd2c40b1f70 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- 172.21.15.133:0/1775072176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c40cc800 con 0x7fd2c40b1f70 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2c17fa640 1 -- 172.21.15.133:0/1775072176 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b40072f0 con 0x7fd2c40b1f70 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- 172.21.15.133:0/1775072176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c4156b60 con 0x7fd2c40b1f70 2024-04-25T10:14:25.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.175+0000 7fd2cacef640 1 -- 172.21.15.133:0/1775072176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c4068ef0 con 0x7fd2c40b1f70 2024-04-25T10:14:25.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.179+0000 7fd2c17fa640 1 -- 172.21.15.133:0/1775072176 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2b401a070 con 0x7fd2c40b1f70 2024-04-25T10:14:25.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.179+0000 7fd2c17fa640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b0066f60 0x7fd2b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.179+0000 7fd2c3fff640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b0066f60 0x7fd2b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.179+0000 7fd2c17fa640 1 -- 172.21.15.133:0/1775072176 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd2b4084200 con 0x7fd2c40b1f70 2024-04-25T10:14:25.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.179+0000 7fd2c3fff640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b0066f60 0x7fd2b0069420 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40cc5f0 tx=0x7fd2b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.183+0000 7fd2c17fa640 1 -- 172.21.15.133:0/1775072176 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2b4054d70 con 0x7fd2c40b1f70 2024-04-25T10:14:25.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.343+0000 7fd2cacef640 1 -- 172.21.15.133:0/1775072176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2c40b44a0 con 0x7fd2c40b1f70 2024-04-25T10:14:25.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.343+0000 7fd2c17fa640 1 -- 172.21.15.133:0/1775072176 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+13198 (secure 0 0 0) 0x7fd2b4058e10 con 0x7fd2c40b1f70 2024-04-25T10:14:25.348 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:25.348 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":139,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:24.621275+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 -- 172.21.15.133:0/1775072176 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b0066f60 msgr2=0x7fd2b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b0066f60 0x7fd2b0069420 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40cc5f0 tx=0x7fd2b8009040 comp rx=0 tx=0).stop 2024-04-25T10:14:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 -- 172.21.15.133:0/1775072176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c40b1f70 msgr2=0x7fd2c4163910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c40b1f70 0x7fd2c4163910 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400c500 tx=0x7fd2b400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 -- 172.21.15.133:0/1775072176 shutdown_connections 2024-04-25T10:14:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2c415c9a0 0x7fd2c4155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2c40ce330 0x7fd2c4166f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2b0066f60 0x7fd2b0069420 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 --2- 172.21.15.133:0/1775072176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2c40b1f70 0x7fd2c4163910 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 -- 172.21.15.133:0/1775072176 >> 172.21.15.133:0/1775072176 conn(0x7fd2c4067000 msgr2=0x7fd2c415afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 -- 172.21.15.133:0/1775072176 shutdown_connections 2024-04-25T10:14:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.347+0000 7fd2cacef640 1 -- 172.21.15.133:0/1775072176 wait complete. 2024-04-25T10:14:25.367 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:25.368 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:25.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.491+0000 7f08778b8640 1 Processor -- start 2024-04-25T10:14:25.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.491+0000 7f08778b8640 1 -- start start 2024-04-25T10:14:25.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f08778b8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08700cd820 0x7f08700cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f08778b8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08700ce2e0 con 0x7f087015ced0 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f08778b8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087006d4b0 con 0x7f0870145170 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f08778b8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087006db00 con 0x7f08700cd820 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0875e55640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f087015ced0 0x7f08700b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45524/0 (socket says 172.21.15.133:45524) 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0875e55640 1 -- 172.21.15.133:0/2197613099 learned_addr learned my addr 172.21.15.133:0/2197613099 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614148870 0 0) 0x7f08700ce2e0 con 0x7f087015ced0 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0874e53640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08700cd820 0x7f08700cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f085c003530 con 0x7f087015ced0 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3636243530 0 0) 0x7f085c003530 con 0x7f087015ced0 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08700cd820 msgr2=0x7f08700cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08700cd820 0x7f08700cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 >> v1:172.21.15.162:6789/0 conn(0x7f0870145170 legacy=0x7f087006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087006d790 con 0x7f087015ced0 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0868001a00 con 0x7f087015ced0 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 >> v1:172.21.15.133:6789/0 conn(0x7f087015ced0 legacy=0x7f08700b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08700cd820 0x7f085c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 0x7f085c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f085c004d50 0x7f085c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087006db00 con 0x7f085c003f70 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c00bab0 con 0x7f085c004d50 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c00bcb0 con 0x7f08700cd820 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0874e53640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 0x7f085c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0875654640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08700cd820 0x7f085c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0875e55640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f085c004d50 0x7f085c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0874e53640 1 -- 172.21.15.133:0/2197613099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08700cd820 msgr2=0x7f085c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:25.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0874e53640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08700cd820 0x7f085c0039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0874e53640 1 -- 172.21.15.133:0/2197613099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f085c004d50 msgr2=0x7f085c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:25.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0874e53640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f085c004d50 0x7f085c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.495+0000 7f0874e53640 1 -- 172.21.15.133:0/2197613099 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c003530 con 0x7f085c003f70 2024-04-25T10:14:25.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0874e53640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 0x7f085c004810 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f0860009ae0 tx=0x7f086001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbc49fda64bd6a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600b2070 con 0x7f085c003f70 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08600a5660 con 0x7f085c003f70 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0867fff640 1 -- 172.21.15.133:0/2197613099 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600096b0 con 0x7f085c003f70 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 -- 172.21.15.133:0/2197613099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 msgr2=0x7f085c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 --2- 172.21.15.133:0/2197613099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 0x7f085c004810 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f0860009ae0 tx=0x7f086001ca70 comp rx=0 tx=0).stop 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875e55640 1 -- 172.21.15.133:0/2197613099 reap_dead start 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 -- 172.21.15.133:0/2197613099 shutdown_connections 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 -- 172.21.15.133:0/2197613099 >> 172.21.15.133:0/2197613099 conn(0x7f0870066f70 msgr2=0x7f087006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 -- 172.21.15.133:0/2197613099 shutdown_connections 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 -- 172.21.15.133:0/2197613099 wait complete. 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 Processor -- start 2024-04-25T10:14:25.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 -- start start 2024-04-25T10:14:25.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 0x7f0870151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f087013b450 0x7f087015dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f087013b890 0x7f087015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087015cb70 con 0x7f085c003f70 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0870145550 con 0x7f087013b890 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f08778b8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08700b1dd0 con 0x7f087013b450 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875654640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 0x7f0870151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875654640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 0x7f0870151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55478/0 (socket says 172.21.15.133:55478) 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875654640 1 -- 172.21.15.133:0/3045694260 learned_addr learned my addr 172.21.15.133:0/3045694260 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875e55640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f087013b890 0x7f087015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0874e53640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f087013b450 0x7f087015dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875654640 1 -- 172.21.15.133:0/3045694260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f087013b450 msgr2=0x7f087015dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875654640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f087013b450 0x7f087015dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875654640 1 -- 172.21.15.133:0/3045694260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f087013b890 msgr2=0x7f087015e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875654640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f087013b890 0x7f087015e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875654640 1 -- 172.21.15.133:0/3045694260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08701f1670 con 0x7f085c003f70 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875e55640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f087013b890 0x7f087015e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0874e53640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f087013b450 0x7f087015dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0875654640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 0x7f0870151eb0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f086c00d7a0 tx=0x7f086c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.499+0000 7f0865ffb640 1 -- 172.21.15.133:0/3045694260 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c005270 con 0x7f085c003f70 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.503+0000 7f0865ffb640 1 -- 172.21.15.133:0/3045694260 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c017070 con 0x7f085c003f70 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.503+0000 7f08778b8640 1 -- 172.21.15.133:0/3045694260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08701523f0 con 0x7f085c003f70 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.503+0000 7f0865ffb640 1 -- 172.21.15.133:0/3045694260 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c00b650 con 0x7f085c003f70 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.503+0000 7f08778b8640 1 -- 172.21.15.133:0/3045694260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087013c0f0 con 0x7f085c003f70 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.503+0000 7f08778b8640 1 -- 172.21.15.133:0/3045694260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0870068e80 con 0x7f085c003f70 2024-04-25T10:14:25.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.503+0000 7f0865ffb640 1 -- 172.21.15.133:0/3045694260 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f086c00b7f0 con 0x7f085c003f70 2024-04-25T10:14:25.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.503+0000 7f0865ffb640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0854066f60 0x7f0854069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.503+0000 7f0865ffb640 1 -- 172.21.15.133:0/3045694260 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f086c084cc0 con 0x7f085c003f70 2024-04-25T10:14:25.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.503+0000 7f0874e53640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0854066f60 0x7f0854069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.503+0000 7f0874e53640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0854066f60 0x7f0854069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f08600af000 tx=0x7f08600046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.507+0000 7f0865ffb640 1 -- 172.21.15.133:0/3045694260 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f086c055e00 con 0x7f085c003f70 2024-04-25T10:14:25.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.671+0000 7f08778b8640 1 -- 172.21.15.133:0/3045694260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f087015d6f0 con 0x7f085c003f70 2024-04-25T10:14:25.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.671+0000 7f0865ffb640 1 -- 172.21.15.133:0/3045694260 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7f086c055700 con 0x7f085c003f70 2024-04-25T10:14:25.676 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:14:25.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 -- 172.21.15.133:0/3045694260 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0854066f60 msgr2=0x7f0854069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0854066f60 0x7f0854069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f08600af000 tx=0x7f08600046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 -- 172.21.15.133:0/3045694260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 msgr2=0x7f0870151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 0x7f0870151eb0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f086c00d7a0 tx=0x7f086c00dc60 comp rx=0 tx=0).stop 2024-04-25T10:14:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 -- 172.21.15.133:0/3045694260 shutdown_connections 2024-04-25T10:14:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f087013b890 0x7f087015e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f087013b450 0x7f087015dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0854066f60 0x7f0854069420 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 --2- 172.21.15.133:0/3045694260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f085c003f70 0x7f0870151eb0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 -- 172.21.15.133:0/3045694260 >> 172.21.15.133:0/3045694260 conn(0x7f0870066f70 msgr2=0x7f08701421d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 -- 172.21.15.133:0/3045694260 shutdown_connections 2024-04-25T10:14:25.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.675+0000 7f08778b8640 1 -- 172.21.15.133:0/3045694260 wait complete. 2024-04-25T10:14:25.696 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:14:25.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.815+0000 7fb87b75f640 1 Processor -- start 2024-04-25T10:14:25.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.815+0000 7fb87b75f640 1 -- start start 2024-04-25T10:14:25.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb87b75f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb874152750 0x7fb8740ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb87b75f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb8740ce870 0x7fb8740b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb87b75f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87406d350 con 0x7fb874152b30 2024-04-25T10:14:25.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb87b75f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87415bf90 con 0x7fb8740ce870 2024-04-25T10:14:25.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb87b75f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87406d5d0 con 0x7fb874152750 2024-04-25T10:14:25.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb8794fb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb874152750 0x7fb8740ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb879cfc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb8740ce870 0x7fb8740b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb879cfc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb8740ce870 0x7fb8740b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:34552/0 (socket says 172.21.15.133:34552) 2024-04-25T10:14:25.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb879cfc640 1 -- 172.21.15.133:0/3265188182 learned_addr learned my addr 172.21.15.133:0/3265188182 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:25.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb879cfc640 1 -- 172.21.15.133:0/3265188182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb874152750 msgr2=0x7fb8740ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:25.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb879cfc640 1 --2- 172.21.15.133:0/3265188182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb874152750 0x7fb8740ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb879cfc640 1 -- 172.21.15.133:0/3265188182 >> v1:172.21.15.162:6789/0 conn(0x7fb874152b30 legacy=0x7fb8740b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb879cfc640 1 -- 172.21.15.133:0/3265188182 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87415c230 con 0x7fb8740ce870 2024-04-25T10:14:25.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb879cfc640 1 --2- 172.21.15.133:0/3265188182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8740ce870 0x7fb8740b1ec0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009e50 tx=0x7fb86c01ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2657256fad95cec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.819+0000 7fb86bfff640 1 -- 172.21.15.133:0/3265188182 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0a56e0 con 0x7fb8740ce870 2024-04-25T10:14:25.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb86bfff640 1 -- 172.21.15.133:0/3265188182 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c0a5880 con 0x7fb8740ce870 2024-04-25T10:14:25.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb86bfff640 1 -- 172.21.15.133:0/3265188182 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0a5c50 con 0x7fb8740ce870 2024-04-25T10:14:25.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 -- 172.21.15.133:0/3265188182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8740ce870 msgr2=0x7fb8740b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 --2- 172.21.15.133:0/3265188182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8740ce870 0x7fb8740b1ec0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009e50 tx=0x7fb86c01ca10 comp rx=0 tx=0).stop 2024-04-25T10:14:25.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 -- 172.21.15.133:0/3265188182 shutdown_connections 2024-04-25T10:14:25.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 --2- 172.21.15.133:0/3265188182 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8740ce870 0x7fb8740b1ec0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 --2- 172.21.15.133:0/3265188182 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb874152750 0x7fb8740ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 -- 172.21.15.133:0/3265188182 >> 172.21.15.133:0/3265188182 conn(0x7fb874067000 msgr2=0x7fb87406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 -- 172.21.15.133:0/3265188182 shutdown_connections 2024-04-25T10:14:25.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 -- 172.21.15.133:0/3265188182 wait complete. 2024-04-25T10:14:25.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 Processor -- start 2024-04-25T10:14:25.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 -- start start 2024-04-25T10:14:25.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8740ce870 0x7fb87413c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb874152750 0x7fb874154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb874152b30 0x7fb874155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb874160080 con 0x7fb874152b30 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87415ff00 con 0x7fb8740ce870 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb874160200 con 0x7fb874152750 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb879cfc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb874152b30 0x7fb874155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb879cfc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb874152b30 0x7fb874155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55490/0 (socket says 172.21.15.133:55490) 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb879cfc640 1 -- 172.21.15.133:0/2095656824 learned_addr learned my addr 172.21.15.133:0/2095656824 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb8794fb640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8740ce870 0x7fb87413c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb878cfa640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb874152750 0x7fb874154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb879cfc640 1 -- 172.21.15.133:0/2095656824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb874152750 msgr2=0x7fb874154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb879cfc640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb874152750 0x7fb874154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb879cfc640 1 -- 172.21.15.133:0/2095656824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8740ce870 msgr2=0x7fb87413c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb879cfc640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8740ce870 0x7fb87413c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb879cfc640 1 -- 172.21.15.133:0/2095656824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8741518c0 con 0x7fb874152b30 2024-04-25T10:14:25.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb878cfa640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb874152750 0x7fb874154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:25.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb879cfc640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb874152b30 0x7fb874155460 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fb86c01a640 tx=0x7fb86c0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb869ffb640 1 -- 172.21.15.133:0/2095656824 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0bc020 con 0x7fb874152b30 2024-04-25T10:14:25.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb869ffb640 1 -- 172.21.15.133:0/2095656824 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c0b8670 con 0x7fb874152b30 2024-04-25T10:14:25.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 -- 172.21.15.133:0/2095656824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb874151ba0 con 0x7fb874152b30 2024-04-25T10:14:25.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb869ffb640 1 -- 172.21.15.133:0/2095656824 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0b8a60 con 0x7fb874152b30 2024-04-25T10:14:25.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.823+0000 7fb87b75f640 1 -- 172.21.15.133:0/2095656824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87413c770 con 0x7fb874152b30 2024-04-25T10:14:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.827+0000 7fb87b75f640 1 -- 172.21.15.133:0/2095656824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb874068ef0 con 0x7fb874152b30 2024-04-25T10:14:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.827+0000 7fb869ffb640 1 -- 172.21.15.133:0/2095656824 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb86c0bf020 con 0x7fb874152b30 2024-04-25T10:14:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.827+0000 7fb869ffb640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb858066f60 0x7fb858069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.827+0000 7fb869ffb640 1 -- 172.21.15.133:0/2095656824 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb86c11fd50 con 0x7fb874152b30 2024-04-25T10:14:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.827+0000 7fb8794fb640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb858066f60 0x7fb858069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.827+0000 7fb8794fb640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb858066f60 0x7fb858069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb870009a50 tx=0x7fb870008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.831+0000 7fb869ffb640 1 -- 172.21.15.133:0/2095656824 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb86c123030 con 0x7fb874152b30 2024-04-25T10:14:25.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.983+0000 7fb87b75f640 1 -- 172.21.15.133:0/2095656824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb8740c61c0 con 0x7fb874152b30 2024-04-25T10:14:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.983+0000 7fb869ffb640 1 -- 172.21.15.133:0/2095656824 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v139) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb86c0c6030 con 0x7fb874152b30 2024-04-25T10:14:25.987 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:25.987 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:14:25.987 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:14:25.987 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:14:25.987 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:14:25.988 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:25.988 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:14:25.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 -- 172.21.15.133:0/2095656824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb858066f60 msgr2=0x7fb858069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb858066f60 0x7fb858069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb870009a50 tx=0x7fb870008040 comp rx=0 tx=0).stop 2024-04-25T10:14:25.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 -- 172.21.15.133:0/2095656824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb874152b30 msgr2=0x7fb874155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb874152b30 0x7fb874155460 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fb86c01a640 tx=0x7fb86c0197b0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 -- 172.21.15.133:0/2095656824 shutdown_connections 2024-04-25T10:14:25.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb858066f60 0x7fb858069420 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb874152b30 0x7fb874155460 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb874152750 0x7fb874154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 --2- 172.21.15.133:0/2095656824 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8740ce870 0x7fb87413c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 -- 172.21.15.133:0/2095656824 >> 172.21.15.133:0/2095656824 conn(0x7fb874067000 msgr2=0x7fb87415adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 -- 172.21.15.133:0/2095656824 shutdown_connections 2024-04-25T10:14:25.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:25.987+0000 7fb87b75f640 1 -- 172.21.15.133:0/2095656824 wait complete. 2024-04-25T10:14:26.006 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:14:26.006 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:14:26.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3259c36640 1 Processor -- start 2024-04-25T10:14:26.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3259c36640 1 -- start start 2024-04-25T10:14:26.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3259c36640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32540b4940 0x7f32540b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3259c36640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f325415c420 0x7f32540cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3259c36640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32540b5260 con 0x7f3254071cb0 2024-04-25T10:14:26.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3259c36640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3254072a90 con 0x7f325415c420 2024-04-25T10:14:26.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3259c36640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32540a85d0 con 0x7f32540b4940 2024-04-25T10:14:26.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3253fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f325415c420 0x7f32540cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f32537fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32540b4940 0x7f32540b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3253fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f325415c420 0x7f32540cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:34570/0 (socket says 172.21.15.133:34570) 2024-04-25T10:14:26.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3253fff640 1 -- 172.21.15.133:0/2859050248 learned_addr learned my addr 172.21.15.133:0/2859050248 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:26.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3253fff640 1 -- 172.21.15.133:0/2859050248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32540b4940 msgr2=0x7f32540b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3253fff640 1 --2- 172.21.15.133:0/2859050248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32540b4940 0x7f32540b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3253fff640 1 -- 172.21.15.133:0/2859050248 >> v1:172.21.15.162:6789/0 conn(0x7f3254071cb0 legacy=0x7f325415bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f3253fff640 1 -- 172.21.15.133:0/2859050248 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325415c9b0 con 0x7f325415c420 2024-04-25T10:14:26.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.123+0000 7f32537fe640 1 --2- 172.21.15.133:0/2859050248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32540b4940 0x7f32540b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:26.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3253fff640 1 --2- 172.21.15.133:0/2859050248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325415c420 0x7f32540cc010 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f324c009e50 tx=0x7f324c01ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1021c887f397fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3252ffd640 1 -- 172.21.15.133:0/2859050248 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c0a58d0 con 0x7f325415c420 2024-04-25T10:14:26.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3252ffd640 1 -- 172.21.15.133:0/2859050248 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324c0a5a70 con 0x7f325415c420 2024-04-25T10:14:26.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3252ffd640 1 -- 172.21.15.133:0/2859050248 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c0a5e40 con 0x7f325415c420 2024-04-25T10:14:26.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3259c36640 1 -- 172.21.15.133:0/2859050248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325415c420 msgr2=0x7f32540cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3259c36640 1 --2- 172.21.15.133:0/2859050248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325415c420 0x7f32540cc010 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f324c009e50 tx=0x7f324c01ca00 comp rx=0 tx=0).stop 2024-04-25T10:14:26.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3259c36640 1 -- 172.21.15.133:0/2859050248 shutdown_connections 2024-04-25T10:14:26.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3259c36640 1 --2- 172.21.15.133:0/2859050248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f325415c420 0x7f32540cc010 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3259c36640 1 --2- 172.21.15.133:0/2859050248 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32540b4940 0x7f32540b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3259c36640 1 -- 172.21.15.133:0/2859050248 >> 172.21.15.133:0/2859050248 conn(0x7f325406d370 msgr2=0x7f325406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3259c36640 1 -- 172.21.15.133:0/2859050248 shutdown_connections 2024-04-25T10:14:26.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.127+0000 7f3259c36640 1 -- 172.21.15.133:0/2859050248 wait complete. 2024-04-25T10:14:26.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3259c36640 1 Processor -- start 2024-04-25T10:14:26.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3259c36640 1 -- start start 2024-04-25T10:14:26.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3259c36640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3254071cb0 0x7f32540af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3259c36640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32540b4940 0x7f3254140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3259c36640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325415c420 0x7f3254158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3259c36640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32540cc960 con 0x7f32540b4940 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3259c36640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3254071790 con 0x7f3254071cb0 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3259c36640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32540ccae0 con 0x7f325415c420 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f324affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32540b4940 0x7f3254140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f324affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32540b4940 0x7f3254140210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55498/0 (socket says 172.21.15.133:55498) 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f324affd640 1 -- 172.21.15.133:0/1698221416 learned_addr learned my addr 172.21.15.133:0/1698221416 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3253fff640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325415c420 0x7f3254158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f32537fe640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3254071cb0 0x7f32540af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f324affd640 1 -- 172.21.15.133:0/1698221416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325415c420 msgr2=0x7f3254158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f324affd640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325415c420 0x7f3254158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f324affd640 1 -- 172.21.15.133:0/1698221416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3254071cb0 msgr2=0x7f32540af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f324affd640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3254071cb0 0x7f32540af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f324affd640 1 -- 172.21.15.133:0/1698221416 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32540af960 con 0x7f32540b4940 2024-04-25T10:14:26.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f324affd640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32540b4940 0x7f3254140210 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c500 tx=0x7f323c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3250ff9640 1 -- 172.21.15.133:0/1698221416 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c016020 con 0x7f32540b4940 2024-04-25T10:14:26.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3259c36640 1 -- 172.21.15.133:0/1698221416 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32540afc40 con 0x7f32540b4940 2024-04-25T10:14:26.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3250ff9640 1 -- 172.21.15.133:0/1698221416 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c005220 con 0x7f32540b4940 2024-04-25T10:14:26.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3250ff9640 1 -- 172.21.15.133:0/1698221416 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c007030 con 0x7f32540b4940 2024-04-25T10:14:26.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3259c36640 1 -- 172.21.15.133:0/1698221416 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32540b0200 con 0x7f32540b4940 2024-04-25T10:14:26.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3259c36640 1 -- 172.21.15.133:0/1698221416 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32540b0440 con 0x7f32540b4940 2024-04-25T10:14:26.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.131+0000 7f3250ff9640 1 -- 172.21.15.133:0/1698221416 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f323c0071f0 con 0x7f32540b4940 2024-04-25T10:14:26.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.135+0000 7f3250ff9640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3234066f90 0x7f3234069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.135+0000 7f3250ff9640 1 -- 172.21.15.133:0/1698221416 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f323c084960 con 0x7f32540b4940 2024-04-25T10:14:26.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.135+0000 7f32537fe640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3234066f90 0x7f3234069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.135+0000 7f32537fe640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3234066f90 0x7f3234069450 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3244009580 tx=0x7f3244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.139+0000 7f3250ff9640 1 -- 172.21.15.133:0/1698221416 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f323c020020 con 0x7f32540b4940 2024-04-25T10:14:26.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.295+0000 7f3259c36640 1 -- 172.21.15.133:0/1698221416 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3254069000 con 0x7f32540b4940 2024-04-25T10:14:26.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.631+0000 7f3250ff9640 1 -- 172.21.15.133:0/1698221416 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v140) v1 ==== 138+0+0 (secure 0 0 0) 0x7f323c055aa0 con 0x7f32540b4940 2024-04-25T10:14:26.633 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:14:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 -- 172.21.15.133:0/1698221416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3234066f90 msgr2=0x7f3234069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3234066f90 0x7f3234069450 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3244009580 tx=0x7f3244008040 comp rx=0 tx=0).stop 2024-04-25T10:14:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 -- 172.21.15.133:0/1698221416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32540b4940 msgr2=0x7f3254140210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32540b4940 0x7f3254140210 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c500 tx=0x7f323c00c9c0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 -- 172.21.15.133:0/1698221416 shutdown_connections 2024-04-25T10:14:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f325415c420 0x7f3254158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3234066f90 0x7f3234069450 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32540b4940 0x7f3254140210 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 --2- 172.21.15.133:0/1698221416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3254071cb0 0x7f32540af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 -- 172.21.15.133:0/1698221416 >> 172.21.15.133:0/1698221416 conn(0x7f325406d370 msgr2=0x7f32540737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 -- 172.21.15.133:0/1698221416 shutdown_connections 2024-04-25T10:14:26.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.635+0000 7f3259c36640 1 -- 172.21.15.133:0/1698221416 wait complete. 2024-04-25T10:14:26.662 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:14:26.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67fe674640 1 Processor -- start 2024-04-25T10:14:26.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67fe674640 1 -- start start 2024-04-25T10:14:26.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67fe674640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f80ced70 0x7f67f80ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67fe674640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f67f80ce1e0 0x7f67f815da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67fe674640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f80cb2e0 con 0x7f67f80ce990 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67fe674640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f80cb5e0 con 0x7f67f80ce1e0 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67fe674640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f80cd920 con 0x7f67f80ced70 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67f7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f67f80ce990 0x7f67f80d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48544/0 (socket says 172.21.15.133:48544) 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67f7fff640 1 -- 172.21.15.133:0/4142499394 learned_addr learned my addr 172.21.15.133:0/4142499394 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122341435 0 0) 0x7f67f80cb2e0 con 0x7f67f80ce990 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67f77fe640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f80ced70 0x7f67f80ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67e4003530 con 0x7f67f80ce990 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67fcc11640 1 --2- 172.21.15.133:0/4142499394 >> v2:172.21.15.162:3300/0 conn(0x7f67f80ce1e0 0x7f67f815da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 346572879 0 0) 0x7f67e4003530 con 0x7f67f80ce990 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f80ced70 msgr2=0x7f67f80ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67f6ffd640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f80ced70 0x7f67f80ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 >> v2:172.21.15.162:3300/0 conn(0x7f67f80ce1e0 msgr2=0x7f67f815da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67f6ffd640 1 --2- 172.21.15.133:0/4142499394 >> v2:172.21.15.162:3300/0 conn(0x7f67f80ce1e0 0x7f67f815da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.791+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f80cdbc0 con 0x7f67f80ce990 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f67e8001a00 con 0x7f67f80ce990 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 >> v1:172.21.15.133:6789/0 conn(0x7f67f80ce990 legacy=0x7f67f80d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f6ffd640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f80ced70 0x7f67e40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f6ffd640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 0x7f67e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f6ffd640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67e4004d50 0x7f67e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f80cb5e0 con 0x7f67e4003ff0 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f80cd920 con 0x7f67e4004d50 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e400bb90 con 0x7f67f80ced70 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67fcc11640 1 --2- 172.21.15.133:0/4142499394 >> v2:172.21.15.162:3300/0 conn(0x7f67f80ce1e0 0x7f67f815da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f77fe640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 0x7f67e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f7fff640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f80ced70 0x7f67e40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67fcc11640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67e4004d50 0x7f67e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f77fe640 1 -- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f80ced70 msgr2=0x7f67e40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f77fe640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f80ced70 0x7f67e40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f77fe640 1 -- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67e4004d50 msgr2=0x7f67e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f77fe640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67e4004d50 0x7f67e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f77fe640 1 -- 172.21.15.133:0/4142499394 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e4003530 con 0x7f67e4003ff0 2024-04-25T10:14:26.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f7fff640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f80ced70 0x7f67e40039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:26.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67fcc11640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67e4004d50 0x7f67e40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f77fe640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 0x7f67e4004810 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f67e000ba30 tx=0x7f67e001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4a818837d4c841 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e000b600 con 0x7f67e4003ff0 2024-04-25T10:14:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e00b0070 con 0x7f67e4003ff0 2024-04-25T10:14:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67f6ffd640 1 -- 172.21.15.133:0/4142499394 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e00ab870 con 0x7f67e4003ff0 2024-04-25T10:14:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67fe674640 1 -- 172.21.15.133:0/4142499394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 msgr2=0x7f67e4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67fe674640 1 --2- 172.21.15.133:0/4142499394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 0x7f67e4004810 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f67e000ba30 tx=0x7f67e001c6a0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67fcc11640 1 -- 172.21.15.133:0/4142499394 reap_dead start 2024-04-25T10:14:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67fe674640 1 -- 172.21.15.133:0/4142499394 shutdown_connections 2024-04-25T10:14:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67fe674640 1 -- 172.21.15.133:0/4142499394 >> 172.21.15.133:0/4142499394 conn(0x7f67f80999d0 msgr2=0x7f67f80d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67fe674640 1 -- 172.21.15.133:0/4142499394 shutdown_connections 2024-04-25T10:14:26.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.795+0000 7f67fe674640 1 -- 172.21.15.133:0/4142499394 wait complete. 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fe674640 1 Processor -- start 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fe674640 1 -- start start 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fe674640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 0x7f67f80b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fe674640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f81518d0 0x7f67f80c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fe674640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f8151380 0x7f67f80c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fe674640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f80cbe00 con 0x7f67e4003ff0 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fe674640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f80cbc80 con 0x7f67f8151380 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fe674640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f80cbf80 con 0x7f67f81518d0 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 0x7f67f80b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 0x7f67f80b5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37258/0 (socket says 172.21.15.133:37258) 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f7fff640 1 -- 172.21.15.133:0/14737409 learned_addr learned my addr 172.21.15.133:0/14737409 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fcc11640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f8151380 0x7f67f80c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f7fff640 1 -- 172.21.15.133:0/14737409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f81518d0 msgr2=0x7f67f80c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f7fff640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f81518d0 0x7f67f80c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f7fff640 1 -- 172.21.15.133:0/14737409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f8151380 msgr2=0x7f67f80c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f7fff640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f8151380 0x7f67f80c44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f7fff640 1 -- 172.21.15.133:0/14737409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f80c62a0 con 0x7f67e4003ff0 2024-04-25T10:14:26.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f7fff640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 0x7f67f80b5f60 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f67e8001b20 tx=0x7f67e800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f4ff9640 1 -- 172.21.15.133:0/14737409 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8019070 con 0x7f67e4003ff0 2024-04-25T10:14:26.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f4ff9640 1 -- 172.21.15.133:0/14737409 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e8004d70 con 0x7f67e4003ff0 2024-04-25T10:14:26.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fe674640 1 -- 172.21.15.133:0/14737409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f80c6580 con 0x7f67e4003ff0 2024-04-25T10:14:26.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67f4ff9640 1 -- 172.21.15.133:0/14737409 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8007310 con 0x7f67e4003ff0 2024-04-25T10:14:26.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fe674640 1 -- 172.21.15.133:0/14737409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f80c6b40 con 0x7f67e4003ff0 2024-04-25T10:14:26.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.799+0000 7f67fe674640 1 -- 172.21.15.133:0/14737409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f8067c70 con 0x7f67e4003ff0 2024-04-25T10:14:26.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.803+0000 7f67f4ff9640 1 -- 172.21.15.133:0/14737409 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67e801c070 con 0x7f67e4003ff0 2024-04-25T10:14:26.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.803+0000 7f67f4ff9640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67d8066f60 0x7f67d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.803+0000 7f67f4ff9640 1 -- 172.21.15.133:0/14737409 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f67e8085260 con 0x7f67e4003ff0 2024-04-25T10:14:26.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.803+0000 7f67f77fe640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67d8066f60 0x7f67d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.803+0000 7f67f77fe640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67d8066f60 0x7f67d8069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f67e00af000 tx=0x7f67e0004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.807+0000 7f67f4ff9640 1 -- 172.21.15.133:0/14737409 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67e8056220 con 0x7f67e4003ff0 2024-04-25T10:14:26.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:26.951+0000 7f67fe674640 1 -- 172.21.15.133:0/14737409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f67f80b6520 con 0x7f67e4003ff0 2024-04-25T10:14:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.635+0000 7f67f4ff9640 1 -- 172.21.15.133:0/14737409 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v141) v1 ==== 170+0+0 (secure 0 0 0) 0x7f67e8055b20 con 0x7f67e4003ff0 2024-04-25T10:14:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:14:27.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.635+0000 7f67fe674640 1 -- 172.21.15.133:0/14737409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67d8066f60 msgr2=0x7f67d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.635+0000 7f67fe674640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67d8066f60 0x7f67d8069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f67e00af000 tx=0x7f67e0004740 comp rx=0 tx=0).stop 2024-04-25T10:14:27.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.635+0000 7f67fe674640 1 -- 172.21.15.133:0/14737409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 msgr2=0x7f67f80b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.635+0000 7f67fe674640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 0x7f67f80b5f60 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f67e8001b20 tx=0x7f67e800b560 comp rx=0 tx=0).stop 2024-04-25T10:14:27.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.639+0000 7f67fe674640 1 -- 172.21.15.133:0/14737409 shutdown_connections 2024-04-25T10:14:27.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.639+0000 7f67fe674640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f67f8151380 0x7f67f80c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.639+0000 7f67fe674640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f67f81518d0 0x7f67f80c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.639+0000 7f67fe674640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f67d8066f60 0x7f67d8069420 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.639+0000 7f67fe674640 1 --2- 172.21.15.133:0/14737409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67e4003ff0 0x7f67f80b5f60 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.639+0000 7f67fe674640 1 -- 172.21.15.133:0/14737409 >> 172.21.15.133:0/14737409 conn(0x7f67f80999d0 msgr2=0x7f67f815bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.639+0000 7f67fe674640 1 -- 172.21.15.133:0/14737409 shutdown_connections 2024-04-25T10:14:27.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.639+0000 7f67fe674640 1 -- 172.21.15.133:0/14737409 wait complete. 2024-04-25T10:14:27.673 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:14:27.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121c78e640 1 Processor -- start 2024-04-25T10:14:27.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121c78e640 1 -- start start 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121c78e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140cd820 0x7f12140cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121c78e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12140ce2e0 con 0x7f1214145170 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121c78e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f121406d4b0 con 0x7f121415ced0 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121c78e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121406db00 con 0x7f12140cd820 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121a52a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1214145170 0x7f121406df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48554/0 (socket says 172.21.15.133:48554) 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121a52a640 1 -- 172.21.15.133:0/253279877 learned_addr learned my addr 172.21.15.133:0/253279877 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14587745 0 0) 0x7f12140ce2e0 con 0x7f1214145170 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1200003530 con 0x7f1214145170 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219d29640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140cd820 0x7f12140cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1782995584 0 0) 0x7f1200003530 con 0x7f1214145170 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140cd820 msgr2=0x7f12140cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140cd820 0x7f12140cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 >> v1:172.21.15.162:6789/0 conn(0x7f121415ced0 legacy=0x7f12140b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121406d790 con 0x7f1214145170 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1210001680 con 0x7f1214145170 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 >> v1:172.21.15.133:6789/0 conn(0x7f1214145170 legacy=0x7f121406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140cd820 0x7f12000039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1200004020 0x7f1200004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 0x7f12000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121406db00 con 0x7f1200004d50 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120000bb90 con 0x7f1200004020 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120000bd90 con 0x7f12140cd820 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121ad2b640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 0x7f12000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121a52a640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140cd820 0x7f12000039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f1219d29640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1200004020 0x7f1200004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121ad2b640 1 -- 172.21.15.133:0/253279877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140cd820 msgr2=0x7f12000039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121ad2b640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140cd820 0x7f12000039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121ad2b640 1 -- 172.21.15.133:0/253279877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1200004020 msgr2=0x7f1200004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:27.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121ad2b640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1200004020 0x7f1200004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.791+0000 7f121ad2b640 1 -- 172.21.15.133:0/253279877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1200003530 con 0x7f1200004d50 2024-04-25T10:14:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f121ad2b640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 0x7f12000051d0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f120c001e70 tx=0x7f120c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2602d1d36e308bed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c001a90 con 0x7f1200004d50 2024-04-25T10:14:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c003b80 con 0x7f1200004d50 2024-04-25T10:14:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f1219528640 1 -- 172.21.15.133:0/253279877 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c005080 con 0x7f1200004d50 2024-04-25T10:14:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f121c78e640 1 -- 172.21.15.133:0/253279877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 msgr2=0x7f12000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f121c78e640 1 --2- 172.21.15.133:0/253279877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 0x7f12000051d0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f120c001e70 tx=0x7f120c01c780 comp rx=0 tx=0).stop 2024-04-25T10:14:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f121ad2b640 1 -- 172.21.15.133:0/253279877 reap_dead start 2024-04-25T10:14:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f121c78e640 1 -- 172.21.15.133:0/253279877 shutdown_connections 2024-04-25T10:14:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f121c78e640 1 -- 172.21.15.133:0/253279877 >> 172.21.15.133:0/253279877 conn(0x7f1214066f70 msgr2=0x7f121406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f121c78e640 1 -- 172.21.15.133:0/253279877 shutdown_connections 2024-04-25T10:14:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f121c78e640 1 -- 172.21.15.133:0/253279877 wait complete. 2024-04-25T10:14:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f121c78e640 1 Processor -- start 2024-04-25T10:14:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.795+0000 7f121c78e640 1 -- start start 2024-04-25T10:14:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121c78e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 0x7f12141f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121c78e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12140b10f0 0x7f121413cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121c78e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140afe30 0x7f12140acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121c78e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121415cb70 con 0x7f1200004d50 2024-04-25T10:14:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121c78e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1214145550 con 0x7f12140b10f0 2024-04-25T10:14:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121c78e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12140b1dd0 con 0x7f12140afe30 2024-04-25T10:14:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121a52a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 0x7f12141f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121a52a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 0x7f12141f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37278/0 (socket says 172.21.15.133:37278) 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121a52a640 1 -- 172.21.15.133:0/1047175044 learned_addr learned my addr 172.21.15.133:0/1047175044 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f1219d29640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12140b10f0 0x7f121413cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121ad2b640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140afe30 0x7f12140acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121a52a640 1 -- 172.21.15.133:0/1047175044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140afe30 msgr2=0x7f12140acb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121a52a640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140afe30 0x7f12140acb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121a52a640 1 -- 172.21.15.133:0/1047175044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12140b10f0 msgr2=0x7f121413cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121a52a640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12140b10f0 0x7f121413cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121a52a640 1 -- 172.21.15.133:0/1047175044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12141f42c0 con 0x7f1200004d50 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121a52a640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 0x7f12141f3d80 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f1210005fd0 tx=0x7f121000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f120affd640 1 -- 172.21.15.133:0/1047175044 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121000c8d0 con 0x7f1200004d50 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f120affd640 1 -- 172.21.15.133:0/1047175044 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1210018070 con 0x7f1200004d50 2024-04-25T10:14:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121c78e640 1 -- 172.21.15.133:0/1047175044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12140c48e0 con 0x7f1200004d50 2024-04-25T10:14:27.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f120affd640 1 -- 172.21.15.133:0/1047175044 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121000ccd0 con 0x7f1200004d50 2024-04-25T10:14:27.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121c78e640 1 -- 172.21.15.133:0/1047175044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12140af630 con 0x7f1200004d50 2024-04-25T10:14:27.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f121c78e640 1 -- 172.21.15.133:0/1047175044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1214068e80 con 0x7f1200004d50 2024-04-25T10:14:27.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f120affd640 1 -- 172.21.15.133:0/1047175044 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f121001d460 con 0x7f1200004d50 2024-04-25T10:14:27.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f120affd640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11f8066f60 0x7f11f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f1219d29640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11f8066f60 0x7f11f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f120affd640 1 -- 172.21.15.133:0/1047175044 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f12100851c0 con 0x7f1200004d50 2024-04-25T10:14:27.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.799+0000 7f1219d29640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11f8066f60 0x7f11f8069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f12140af0b0 tx=0x7f1204008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.803+0000 7f120affd640 1 -- 172.21.15.133:0/1047175044 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12141f42c0 con 0x7f1200004d50 2024-04-25T10:14:28.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:27.999+0000 7f121c78e640 1 -- 172.21.15.133:0/1047175044 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f12140b1b90 con 0x7f1200004d50 2024-04-25T10:14:28.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.667+0000 7f120affd640 1 -- 172.21.15.133:0/1047175044 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v88) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1210056160 con 0x7f1200004d50 2024-04-25T10:14:28.669 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 21 and data pool 22 2024-04-25T10:14:28.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.667+0000 7f121c78e640 1 -- 172.21.15.133:0/1047175044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11f8066f60 msgr2=0x7f11f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.667+0000 7f121c78e640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11f8066f60 0x7f11f8069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f12140af0b0 tx=0x7f1204008040 comp rx=0 tx=0).stop 2024-04-25T10:14:28.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.667+0000 7f121c78e640 1 -- 172.21.15.133:0/1047175044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 msgr2=0x7f12141f3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.667+0000 7f121c78e640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 0x7f12141f3d80 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f1210005fd0 tx=0x7f121000ed50 comp rx=0 tx=0).stop 2024-04-25T10:14:28.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.667+0000 7f121c78e640 1 -- 172.21.15.133:0/1047175044 shutdown_connections 2024-04-25T10:14:28.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.667+0000 7f121c78e640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f12140afe30 0x7f12140acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.667+0000 7f121c78e640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12140b10f0 0x7f121413cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.667+0000 7f121c78e640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f11f8066f60 0x7f11f8069420 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.667+0000 7f121c78e640 1 --2- 172.21.15.133:0/1047175044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1200004d50 0x7f12141f3d80 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.667+0000 7f121c78e640 1 -- 172.21.15.133:0/1047175044 >> 172.21.15.133:0/1047175044 conn(0x7f1214066f70 msgr2=0x7f121415f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.671+0000 7f121c78e640 1 -- 172.21.15.133:0/1047175044 shutdown_connections 2024-04-25T10:14:28.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.671+0000 7f121c78e640 1 -- 172.21.15.133:0/1047175044 wait complete. 2024-04-25T10:14:28.707 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:28.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b77cb640 1 Processor -- start 2024-04-25T10:14:28.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b77cb640 1 -- start start 2024-04-25T10:14:28.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b77cb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd4b015db00 0x7fd4b00b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b77cb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd4b00d0770 0x7fd4b00d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b77cb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b00ceb20 con 0x7fd4b01614c0 2024-04-25T10:14:28.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b77cb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b00d1110 con 0x7fd4b00d0770 2024-04-25T10:14:28.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b77cb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b00b7740 con 0x7fd4b015db00 2024-04-25T10:14:28.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b4d66640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd4b00d0770 0x7fd4b00d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b5d68640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd4b01614c0 0x7fd4b0161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48570/0 (socket says 172.21.15.133:48570) 2024-04-25T10:14:28.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b5d68640 1 -- 172.21.15.133:0/3670686322 learned_addr learned my addr 172.21.15.133:0/3670686322 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:28.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b5567640 1 --2- 172.21.15.133:0/3670686322 >> v2:172.21.15.162:3300/0 conn(0x7fd4b015db00 0x7fd4b00b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b4d66640 1 -- 172.21.15.133:0/3670686322 >> v2:172.21.15.162:3300/0 conn(0x7fd4b015db00 msgr2=0x7fd4b00b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b4d66640 1 --2- 172.21.15.133:0/3670686322 >> v2:172.21.15.162:3300/0 conn(0x7fd4b015db00 0x7fd4b00b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b4d66640 1 -- 172.21.15.133:0/3670686322 >> v1:172.21.15.133:6789/0 conn(0x7fd4b01614c0 legacy=0x7fd4b0161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4a7fff640 1 -- 172.21.15.133:0/3670686322 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046004358 0 0) 0x7fd4b00ceb20 con 0x7fd4b01614c0 2024-04-25T10:14:28.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.831+0000 7fd4b4d66640 1 -- 172.21.15.133:0/3670686322 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00b97f0 con 0x7fd4b00d0770 2024-04-25T10:14:28.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b4d66640 1 --2- 172.21.15.133:0/3670686322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b00d0770 0x7fd4b00d0bd0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00b53d0 tx=0x7fd4a001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d5cc4b6f59bf66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4a7fff640 1 -- 172.21.15.133:0/3670686322 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00ab650 con 0x7fd4b00d0770 2024-04-25T10:14:28.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4a7fff640 1 -- 172.21.15.133:0/3670686322 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a00b0070 con 0x7fd4b00d0770 2024-04-25T10:14:28.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4a7fff640 1 -- 172.21.15.133:0/3670686322 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00aba10 con 0x7fd4b00d0770 2024-04-25T10:14:28.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 -- 172.21.15.133:0/3670686322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b00d0770 msgr2=0x7fd4b00d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 --2- 172.21.15.133:0/3670686322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b00d0770 0x7fd4b00d0bd0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00b53d0 tx=0x7fd4a001c450 comp rx=0 tx=0).stop 2024-04-25T10:14:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 -- 172.21.15.133:0/3670686322 shutdown_connections 2024-04-25T10:14:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 --2- 172.21.15.133:0/3670686322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b00d0770 0x7fd4b00d0bd0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 --2- 172.21.15.133:0/3670686322 >> v2:172.21.15.162:3300/0 conn(0x7fd4b015db00 0x7fd4b00b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 -- 172.21.15.133:0/3670686322 >> 172.21.15.133:0/3670686322 conn(0x7fd4b00727c0 msgr2=0x7fd4b0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 -- 172.21.15.133:0/3670686322 shutdown_connections 2024-04-25T10:14:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 -- 172.21.15.133:0/3670686322 wait complete. 2024-04-25T10:14:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 Processor -- start 2024-04-25T10:14:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 -- start start 2024-04-25T10:14:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b00d0770 0x7fd4b00b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b015db00 0x7fd4b015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01614c0 0x7fd4b015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00b7430 con 0x7fd4b01614c0 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00b82c0 con 0x7fd4b00d0770 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b77cb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00d18a0 con 0x7fd4b015db00 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b5d68640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01614c0 0x7fd4b015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b5d68640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01614c0 0x7fd4b015bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37282/0 (socket says 172.21.15.133:37282) 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b5d68640 1 -- 172.21.15.133:0/1498338527 learned_addr learned my addr 172.21.15.133:0/1498338527 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b5567640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b00d0770 0x7fd4b00b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b5d68640 1 -- 172.21.15.133:0/1498338527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b015db00 msgr2=0x7fd4b015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b4d66640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b015db00 0x7fd4b015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b5d68640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b015db00 0x7fd4b015b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b5d68640 1 -- 172.21.15.133:0/1498338527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b00d0770 msgr2=0x7fd4b00b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b5d68640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b00d0770 0x7fd4b00b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b5d68640 1 -- 172.21.15.133:0/1498338527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00b3f00 con 0x7fd4b01614c0 2024-04-25T10:14:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.835+0000 7fd4b5567640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b00d0770 0x7fd4b00b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4b5d68640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01614c0 0x7fd4b015bb10 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800c7d0 tx=0x7fd4a800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4a5ffb640 1 -- 172.21.15.133:0/1498338527 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8018070 con 0x7fd4b01614c0 2024-04-25T10:14:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4a5ffb640 1 -- 172.21.15.133:0/1498338527 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a800a640 con 0x7fd4b01614c0 2024-04-25T10:14:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4b77cb640 1 -- 172.21.15.133:0/1498338527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b00b41e0 con 0x7fd4b01614c0 2024-04-25T10:14:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4a5ffb640 1 -- 172.21.15.133:0/1498338527 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800aa30 con 0x7fd4b01614c0 2024-04-25T10:14:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4b77cb640 1 -- 172.21.15.133:0/1498338527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b00b47a0 con 0x7fd4b01614c0 2024-04-25T10:14:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4b77cb640 1 -- 172.21.15.133:0/1498338527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b00b4f80 con 0x7fd4b01614c0 2024-04-25T10:14:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4a5ffb640 1 -- 172.21.15.133:0/1498338527 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4a801b070 con 0x7fd4b01614c0 2024-04-25T10:14:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4a5ffb640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd494066f60 0x7fd494069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4a5ffb640 1 -- 172.21.15.133:0/1498338527 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4a8084760 con 0x7fd4b01614c0 2024-04-25T10:14:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4b5567640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd494066f60 0x7fd494069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.839+0000 7fd4b5567640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd494066f60 0x7fd494069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac006fd0 tx=0x7fd4ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:28.843+0000 7fd4a5ffb640 1 -- 172.21.15.133:0/1498338527 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4a8055670 con 0x7fd4b01614c0 2024-04-25T10:14:29.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.007+0000 7fd4b77cb640 1 -- 172.21.15.133:0/1498338527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4b0068ef0 con 0x7fd4b01614c0 2024-04-25T10:14:29.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4a5ffb640 1 -- 172.21.15.133:0/1498338527 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+16929 (secure 0 0 0) 0x7fd4a8054f70 con 0x7fd4b01614c0 2024-04-25T10:14:29.013 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:29.013 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":142,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:28.005491+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:26.340459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:26.996481+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 -- 172.21.15.133:0/1498338527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd494066f60 msgr2=0x7fd494069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd494066f60 0x7fd494069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac006fd0 tx=0x7fd4ac008040 comp rx=0 tx=0).stop 2024-04-25T10:14:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 -- 172.21.15.133:0/1498338527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01614c0 msgr2=0x7fd4b015bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01614c0 0x7fd4b015bb10 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800c7d0 tx=0x7fd4a800cc90 comp rx=0 tx=0).stop 2024-04-25T10:14:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 -- 172.21.15.133:0/1498338527 shutdown_connections 2024-04-25T10:14:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd494066f60 0x7fd494069420 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01614c0 0x7fd4b015bb10 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd4b015db00 0x7fd4b015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 --2- 172.21.15.133:0/1498338527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4b00d0770 0x7fd4b00b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 -- 172.21.15.133:0/1498338527 >> 172.21.15.133:0/1498338527 conn(0x7fd4b00727c0 msgr2=0x7fd4b00d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 -- 172.21.15.133:0/1498338527 shutdown_connections 2024-04-25T10:14:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.011+0000 7fd4b77cb640 1 -- 172.21.15.133:0/1498338527 wait complete. 2024-04-25T10:14:29.033 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:29.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.151+0000 7f8658c1d640 1 Processor -- start 2024-04-25T10:14:29.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.151+0000 7f8658c1d640 1 -- start start 2024-04-25T10:14:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86540b1f70 0x7f865406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86540ce330 0x7f86540ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86540c61d0 con 0x7f865415c9a0 2024-04-25T10:14:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86540cecc0 con 0x7f86540b1f70 2024-04-25T10:14:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c64b0 con 0x7f86540ce330 2024-04-25T10:14:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8651d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86540b1f70 0x7f865406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8651d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86540b1f70 0x7f865406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37286/0 (socket says 172.21.15.133:37286) 2024-04-25T10:14:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8651d9b640 1 -- 172.21.15.133:0/4097851984 learned_addr learned my addr 172.21.15.133:0/4097851984 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8652d9d640 1 --2- 172.21.15.133:0/4097851984 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86540ce330 0x7f86540ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8651d9b640 1 -- 172.21.15.133:0/4097851984 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86540ce330 msgr2=0x7f86540ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8651d9b640 1 --2- 172.21.15.133:0/4097851984 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86540ce330 0x7f86540ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8651d9b640 1 -- 172.21.15.133:0/4097851984 >> v1:172.21.15.162:6789/0 conn(0x7f865415c9a0 legacy=0x7f86540c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8651d9b640 1 -- 172.21.15.133:0/4097851984 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865406d4f0 con 0x7f86540b1f70 2024-04-25T10:14:29.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8651d9b640 1 --2- 172.21.15.133:0/4097851984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86540b1f70 0x7f865406dcf0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f8648009870 tx=0x7f864801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba977b85740b4eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f865159a640 1 -- 172.21.15.133:0/4097851984 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86480aa430 con 0x7f86540b1f70 2024-04-25T10:14:29.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f865159a640 1 -- 172.21.15.133:0/4097851984 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86480af070 con 0x7f86540b1f70 2024-04-25T10:14:29.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f865159a640 1 -- 172.21.15.133:0/4097851984 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86480aa7f0 con 0x7f86540b1f70 2024-04-25T10:14:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 -- 172.21.15.133:0/4097851984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86540b1f70 msgr2=0x7f865406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 --2- 172.21.15.133:0/4097851984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86540b1f70 0x7f865406dcf0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f8648009870 tx=0x7f864801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 -- 172.21.15.133:0/4097851984 shutdown_connections 2024-04-25T10:14:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 --2- 172.21.15.133:0/4097851984 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f86540ce330 0x7f86540ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 --2- 172.21.15.133:0/4097851984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86540b1f70 0x7f865406dcf0 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 -- 172.21.15.133:0/4097851984 >> 172.21.15.133:0/4097851984 conn(0x7f8654067000 msgr2=0x7f865406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 -- 172.21.15.133:0/4097851984 shutdown_connections 2024-04-25T10:14:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.155+0000 7f8658c1d640 1 -- 172.21.15.133:0/4097851984 wait complete. 2024-04-25T10:14:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8658c1d640 1 Processor -- start 2024-04-25T10:14:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8658c1d640 1 -- start start 2024-04-25T10:14:29.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8658c1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86540b1f70 0x7f8654144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8658c1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86540ce330 0x7f865414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8658c1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865415c9a0 0x7f8654153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8658c1d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865415d950 con 0x7f86540b1f70 2024-04-25T10:14:29.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8658c1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865415dad0 con 0x7f86540ce330 2024-04-25T10:14:29.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8658c1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865415dc50 con 0x7f865415c9a0 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f865259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86540b1f70 0x7f8654144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f865259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86540b1f70 0x7f8654144fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37288/0 (socket says 172.21.15.133:37288) 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f865259c640 1 -- 172.21.15.133:0/738296965 learned_addr learned my addr 172.21.15.133:0/738296965 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8651d9b640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86540ce330 0x7f865414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8652d9d640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865415c9a0 0x7f8654153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f865259c640 1 -- 172.21.15.133:0/738296965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865415c9a0 msgr2=0x7f8654153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f865259c640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865415c9a0 0x7f8654153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f865259c640 1 -- 172.21.15.133:0/738296965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86540ce330 msgr2=0x7f865414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f865259c640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86540ce330 0x7f865414c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f865259c640 1 -- 172.21.15.133:0/738296965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654145500 con 0x7f86540b1f70 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f865259c640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86540b1f70 0x7f8654144fc0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f864400c480 tx=0x7f864400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8642ffd640 1 -- 172.21.15.133:0/738296965 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644012830 con 0x7f86540b1f70 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8642ffd640 1 -- 172.21.15.133:0/738296965 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8644017070 con 0x7f86540b1f70 2024-04-25T10:14:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8658c1d640 1 -- 172.21.15.133:0/738296965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865414ccd0 con 0x7f86540b1f70 2024-04-25T10:14:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8642ffd640 1 -- 172.21.15.133:0/738296965 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644012c30 con 0x7f86540b1f70 2024-04-25T10:14:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8658c1d640 1 -- 172.21.15.133:0/738296965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865414d290 con 0x7f86540b1f70 2024-04-25T10:14:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.159+0000 7f8658c1d640 1 -- 172.21.15.133:0/738296965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865406bc20 con 0x7f86540b1f70 2024-04-25T10:14:29.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.163+0000 7f8642ffd640 1 -- 172.21.15.133:0/738296965 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f864401c460 con 0x7f86540b1f70 2024-04-25T10:14:29.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.163+0000 7f8642ffd640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8634066f60 0x7f8634069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.163+0000 7f8642ffd640 1 -- 172.21.15.133:0/738296965 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8644085010 con 0x7f86540b1f70 2024-04-25T10:14:29.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.163+0000 7f8651d9b640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8634066f60 0x7f8634069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.163+0000 7f8651d9b640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8634066f60 0x7f8634069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f86480ae000 tx=0x7f86480a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.167+0000 7f8642ffd640 1 -- 172.21.15.133:0/738296965 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8644055e80 con 0x7f86540b1f70 2024-04-25T10:14:29.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.323+0000 7f8658c1d640 1 -- 172.21.15.133:0/738296965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8654068ef0 con 0x7f86540b1f70 2024-04-25T10:14:29.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.327+0000 7f8642ffd640 1 -- 172.21.15.133:0/738296965 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+16929 (secure 0 0 0) 0x7f8644055780 con 0x7f86540b1f70 2024-04-25T10:14:29.330 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:29.331 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":142,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:28.005491+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:26.340459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:26.996481+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:29.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 -- 172.21.15.133:0/738296965 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8634066f60 msgr2=0x7f8634069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8634066f60 0x7f8634069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f86480ae000 tx=0x7f86480a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 -- 172.21.15.133:0/738296965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86540b1f70 msgr2=0x7f8654144fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86540b1f70 0x7f8654144fc0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f864400c480 tx=0x7f864400c940 comp rx=0 tx=0).stop 2024-04-25T10:14:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 -- 172.21.15.133:0/738296965 shutdown_connections 2024-04-25T10:14:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f865415c9a0 0x7f8654153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f86540ce330 0x7f865414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8634066f60 0x7f8634069420 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 --2- 172.21.15.133:0/738296965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86540b1f70 0x7f8654144fc0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 -- 172.21.15.133:0/738296965 >> 172.21.15.133:0/738296965 conn(0x7f8654067000 msgr2=0x7f86540b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 -- 172.21.15.133:0/738296965 shutdown_connections 2024-04-25T10:14:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.331+0000 7f8658c1d640 1 -- 172.21.15.133:0/738296965 wait complete. 2024-04-25T10:14:29.348 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:14:29.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fe439640 1 Processor -- start 2024-04-25T10:14:29.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fe439640 1 -- start start 2024-04-25T10:14:29.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fe439640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f80f80cd820 0x7f80f80cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fe439640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f815ced0 0x7f80f815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fe439640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f80ce2e0 con 0x7f80f8145170 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fe439640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f806db00 con 0x7f80f80cd820 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fe439640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f8145550 con 0x7f80f815ced0 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f80f8145170 0x7f80f806df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48586/0 (socket says 172.21.15.133:48586) 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f7fff640 1 -- 172.21.15.133:0/523883377 learned_addr learned my addr 172.21.15.133:0/523883377 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f77fe640 1 --2- 172.21.15.133:0/523883377 >> v2:172.21.15.162:3300/0 conn(0x7f80f80cd820 0x7f80f80cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574217359 0 0) 0x7f80f80ce2e0 con 0x7f80f8145170 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fc9d6640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f815ced0 0x7f80f815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80e0003530 con 0x7f80f8145170 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1251952804 0 0) 0x7f80e0003530 con 0x7f80f8145170 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f815ced0 msgr2=0x7f80f815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f815ced0 0x7f80f815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 >> v2:172.21.15.162:3300/0 conn(0x7f80f80cd820 msgr2=0x7f80f80cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 --2- 172.21.15.133:0/523883377 >> v2:172.21.15.162:3300/0 conn(0x7f80f80cd820 0x7f80f80cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f806d4b0 con 0x7f80f8145170 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f80e8001680 con 0x7f80f8145170 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 >> v1:172.21.15.133:6789/0 conn(0x7f80f8145170 legacy=0x7f80f806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e00039c0 0x7f80e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f815ced0 0x7f80e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80e0004d50 0x7f80e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f806db00 con 0x7f80e0004d50 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f8145550 con 0x7f80e00039c0 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e000b9d0 con 0x7f80f815ced0 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fc9d6640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80e0004d50 0x7f80e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f7fff640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e00039c0 0x7f80e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f77fe640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f815ced0 0x7f80e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fc9d6640 1 -- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f815ced0 msgr2=0x7f80e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fc9d6640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f815ced0 0x7f80e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fc9d6640 1 -- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e00039c0 msgr2=0x7f80e0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fc9d6640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e00039c0 0x7f80e0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80fc9d6640 1 -- 172.21.15.133:0/523883377 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0003530 con 0x7f80e0004d50 2024-04-25T10:14:29.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.471+0000 7f80f7fff640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e00039c0 0x7f80e0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:29.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fc9d6640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80e0004d50 0x7f80e00051d0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009fc0 tx=0x7f80f001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d2db800347aef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f00b2070 con 0x7f80e0004d50 2024-04-25T10:14:29.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f00a5c90 con 0x7f80e0004d50 2024-04-25T10:14:29.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80f6ffd640 1 -- 172.21.15.133:0/523883377 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0009c10 con 0x7f80e0004d50 2024-04-25T10:14:29.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 -- 172.21.15.133:0/523883377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80e0004d50 msgr2=0x7f80e00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 --2- 172.21.15.133:0/523883377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80e0004d50 0x7f80e00051d0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009fc0 tx=0x7f80f001cc80 comp rx=0 tx=0).stop 2024-04-25T10:14:29.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fc9d6640 1 -- 172.21.15.133:0/523883377 reap_dead start 2024-04-25T10:14:29.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 -- 172.21.15.133:0/523883377 shutdown_connections 2024-04-25T10:14:29.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 -- 172.21.15.133:0/523883377 >> 172.21.15.133:0/523883377 conn(0x7f80f8066f70 msgr2=0x7f80f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 -- 172.21.15.133:0/523883377 shutdown_connections 2024-04-25T10:14:29.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 -- 172.21.15.133:0/523883377 wait complete. 2024-04-25T10:14:29.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 Processor -- start 2024-04-25T10:14:29.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 -- start start 2024-04-25T10:14:29.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e0004d50 0x7f80f8138f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f80afec0 0x7f80f81f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f80b03d0 0x7f80f80b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f815cb70 con 0x7f80f80afec0 2024-04-25T10:14:29.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f806e6a0 con 0x7f80e0004d50 2024-04-25T10:14:29.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80fe439640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f815d360 con 0x7f80f80b03d0 2024-04-25T10:14:29.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80f77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f80afec0 0x7f80f81f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80f77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f80afec0 0x7f80f81f46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37316/0 (socket says 172.21.15.133:37316) 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.475+0000 7f80f77fe640 1 -- 172.21.15.133:0/2871346980 learned_addr learned my addr 172.21.15.133:0/2871346980 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80fc9d6640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f80b03d0 0x7f80f80b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f7fff640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e0004d50 0x7f80f8138f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f77fe640 1 -- 172.21.15.133:0/2871346980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f80b03d0 msgr2=0x7f80f80b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f77fe640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f80b03d0 0x7f80f80b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f77fe640 1 -- 172.21.15.133:0/2871346980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e0004d50 msgr2=0x7f80f8138f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f77fe640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e0004d50 0x7f80f8138f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f77fe640 1 -- 172.21.15.133:0/2871346980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f81394d0 con 0x7f80f80afec0 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f77fe640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f80afec0 0x7f80f81f46d0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f80ec011660 tx=0x7f80ec011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f4ff9640 1 -- 172.21.15.133:0/2871346980 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec00f9c0 con 0x7f80f80afec0 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f4ff9640 1 -- 172.21.15.133:0/2871346980 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ec00fb60 con 0x7f80f80afec0 2024-04-25T10:14:29.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f4ff9640 1 -- 172.21.15.133:0/2871346980 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec015460 con 0x7f80f80afec0 2024-04-25T10:14:29.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80fe439640 1 -- 172.21.15.133:0/2871346980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f813aba0 con 0x7f80f80afec0 2024-04-25T10:14:29.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80fe439640 1 -- 172.21.15.133:0/2871346980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f813b160 con 0x7f80f80afec0 2024-04-25T10:14:29.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80fe439640 1 -- 172.21.15.133:0/2871346980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f8068e80 con 0x7f80f80afec0 2024-04-25T10:14:29.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f4ff9640 1 -- 172.21.15.133:0/2871346980 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80ec015600 con 0x7f80f80afec0 2024-04-25T10:14:29.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f4ff9640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f80e4066f60 0x7f80e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f7fff640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f80e4066f60 0x7f80e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f4ff9640 1 -- 172.21.15.133:0/2871346980 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80ec085b00 con 0x7f80f80afec0 2024-04-25T10:14:29.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.479+0000 7f80f7fff640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f80e4066f60 0x7f80e4069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f80e8000c10 tx=0x7f80e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.483+0000 7f80f4ff9640 1 -- 172.21.15.133:0/2871346980 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f80ec019030 con 0x7f80f80afec0 2024-04-25T10:14:29.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.679+0000 7f80fe439640 1 -- 172.21.15.133:0/2871346980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f80f81442c0 con 0x7f80f80afec0 2024-04-25T10:14:29.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.683+0000 7f80f4ff9640 1 -- 172.21.15.133:0/2871346980 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) v1 ==== 118+0+0 (secure 0 0 0) 0x7f80ec056a10 con 0x7f80f80afec0 2024-04-25T10:14:29.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 -- 172.21.15.133:0/2871346980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f80e4066f60 msgr2=0x7f80e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f80e4066f60 0x7f80e4069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f80e8000c10 tx=0x7f80e8009040 comp rx=0 tx=0).stop 2024-04-25T10:14:29.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 -- 172.21.15.133:0/2871346980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f80afec0 msgr2=0x7f80f81f46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f80afec0 0x7f80f81f46d0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f80ec011660 tx=0x7f80ec011b20 comp rx=0 tx=0).stop 2024-04-25T10:14:29.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 -- 172.21.15.133:0/2871346980 shutdown_connections 2024-04-25T10:14:29.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f80f80b03d0 0x7f80f80b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f80e4066f60 0x7f80e4069420 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f80afec0 0x7f80f81f46d0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 --2- 172.21.15.133:0/2871346980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e0004d50 0x7f80f8138f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 -- 172.21.15.133:0/2871346980 >> 172.21.15.133:0/2871346980 conn(0x7f80f8066f70 msgr2=0x7f80f806df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 -- 172.21.15.133:0/2871346980 shutdown_connections 2024-04-25T10:14:29.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.687+0000 7f80fe439640 1 -- 172.21.15.133:0/2871346980 wait complete. 2024-04-25T10:14:29.709 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:14:29.709 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:14:29.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.823+0000 7f71a501a640 1 Processor -- start 2024-04-25T10:14:29.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.823+0000 7f71a501a640 1 -- start start 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f71a501a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f71a0143ea0 0x7f71a00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f71a501a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a015ac50 con 0x7f71a00b2150 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f71a501a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a00cd180 con 0x7f71a006eac0 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f71a501a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a015aed0 con 0x7f71a0143ea0 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f7197fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f71a00b2150 0x7f71a006e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48588/0 (socket says 172.21.15.133:48588) 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f7197fff640 1 -- 172.21.15.133:0/516002266 learned_addr learned my addr 172.21.15.133:0/516002266 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833495598 0 0) 0x7f7188001120 con 0x7f71a00b2150 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f718c003530 con 0x7f71a00b2150 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719ed9d640 1 --2- 172.21.15.133:0/516002266 >> v2:172.21.15.162:3300/0 conn(0x7f71a0143ea0 0x7f71a00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1054856113 0 0) 0x7f718c003530 con 0x7f71a00b2150 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 >> v2:172.21.15.162:3300/0 conn(0x7f71a0143ea0 msgr2=0x7f71a00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 --2- 172.21.15.133:0/516002266 >> v2:172.21.15.162:3300/0 conn(0x7f71a0143ea0 0x7f71a00cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 >> v1:172.21.15.162:6789/0 conn(0x7f71a006eac0 legacy=0x7f71a00b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a015b1b0 con 0x7f71a00b2150 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7188001850 con 0x7f71a00b2150 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 >> v1:172.21.15.133:6789/0 conn(0x7f71a00b2150 legacy=0x7f71a006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f718c0039c0 0x7f718c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718c004810 0x7f718c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f718c0052f0 0x7f718c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a015aed0 con 0x7f718c0039c0 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c00c1c0 con 0x7f718c0052f0 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718c00c3c0 con 0x7f718c004810 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719ed9d640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f718c0039c0 0x7f718c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719f59e640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f718c0052f0 0x7f718c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719ed9d640 1 -- 172.21.15.133:0/516002266 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718c004810 msgr2=0x7f718c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f7197fff640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718c004810 0x7f718c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719ed9d640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718c004810 0x7f718c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719ed9d640 1 -- 172.21.15.133:0/516002266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f718c0052f0 msgr2=0x7f718c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719ed9d640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f718c0052f0 0x7f718c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719ed9d640 1 -- 172.21.15.133:0/516002266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c003530 con 0x7f718c0039c0 2024-04-25T10:14:29.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719f59e640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f718c0052f0 0x7f718c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:29.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719f59e640 1 -- 172.21.15.133:0/516002266 reap_dead start 2024-04-25T10:14:29.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.827+0000 7f719ed9d640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f718c0039c0 0x7f718c003de0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f719000b9e0 tx=0x7f719001c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6be5d26dcc8bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71900b2070 con 0x7f718c0039c0 2024-04-25T10:14:29.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71900ad640 con 0x7f718c0039c0 2024-04-25T10:14:29.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f719e59c640 1 -- 172.21.15.133:0/516002266 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719000b5e0 con 0x7f718c0039c0 2024-04-25T10:14:29.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 -- 172.21.15.133:0/516002266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f718c0039c0 msgr2=0x7f718c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f718c0039c0 0x7f718c003de0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f719000b9e0 tx=0x7f719001c640 comp rx=0 tx=0).stop 2024-04-25T10:14:29.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 -- 172.21.15.133:0/516002266 shutdown_connections 2024-04-25T10:14:29.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 --2- 172.21.15.133:0/516002266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f718c0039c0 0x7f718c003de0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 -- 172.21.15.133:0/516002266 >> 172.21.15.133:0/516002266 conn(0x7f71a0066f70 msgr2=0x7f71a006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 -- 172.21.15.133:0/516002266 shutdown_connections 2024-04-25T10:14:29.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 -- 172.21.15.133:0/516002266 wait complete. 2024-04-25T10:14:29.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 Processor -- start 2024-04-25T10:14:29.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 -- start start 2024-04-25T10:14:29.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718c0039c0 0x7f71a01f9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71a013b990 0x7f71a013b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71a013e600 0x7f71a0153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a015e3c0 con 0x7f71a013b990 2024-04-25T10:14:29.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a015e240 con 0x7f71a013e600 2024-04-25T10:14:29.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a015e540 con 0x7f718c0039c0 2024-04-25T10:14:29.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f7197fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71a013b990 0x7f71a013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f7197fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71a013b990 0x7f71a013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37338/0 (socket says 172.21.15.133:37338) 2024-04-25T10:14:29.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f7197fff640 1 -- 172.21.15.133:0/1780833475 learned_addr learned my addr 172.21.15.133:0/1780833475 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:29.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f719ed9d640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718c0039c0 0x7f71a01f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f719f59e640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71a013e600 0x7f71a0153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f7197fff640 1 -- 172.21.15.133:0/1780833475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718c0039c0 msgr2=0x7f71a01f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f7197fff640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718c0039c0 0x7f71a01f9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f7197fff640 1 -- 172.21.15.133:0/1780833475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71a013e600 msgr2=0x7f71a0153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f7197fff640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71a013e600 0x7f71a0153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f7197fff640 1 -- 172.21.15.133:0/1780833475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a01f9780 con 0x7f71a013b990 2024-04-25T10:14:29.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f719f59e640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71a013e600 0x7f71a0153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:29.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f7197fff640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71a013b990 0x7f71a013b160 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f7188007fd0 tx=0x7f718800dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71977fe640 1 -- 172.21.15.133:0/1780833475 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188019070 con 0x7f71a013b990 2024-04-25T10:14:29.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71977fe640 1 -- 172.21.15.133:0/1780833475 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71880074e0 con 0x7f71a013b990 2024-04-25T10:14:29.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 -- 172.21.15.133:0/1780833475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a00cb030 con 0x7f71a013b990 2024-04-25T10:14:29.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71977fe640 1 -- 172.21.15.133:0/1780833475 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71880049f0 con 0x7f71a013b990 2024-04-25T10:14:29.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.831+0000 7f71a501a640 1 -- 172.21.15.133:0/1780833475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a01f8a40 con 0x7f71a013b990 2024-04-25T10:14:29.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.835+0000 7f71a501a640 1 -- 172.21.15.133:0/1780833475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a00b1810 con 0x7f71a013b990 2024-04-25T10:14:29.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.835+0000 7f71977fe640 1 -- 172.21.15.133:0/1780833475 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f718801c070 con 0x7f71a013b990 2024-04-25T10:14:29.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.835+0000 7f71977fe640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7180066f60 0x7f7180069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.835+0000 7f71977fe640 1 -- 172.21.15.133:0/1780833475 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7188084c10 con 0x7f71a013b990 2024-04-25T10:14:29.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.835+0000 7f719ed9d640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7180066f60 0x7f7180069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.835+0000 7f719ed9d640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7180066f60 0x7f7180069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f71900af000 tx=0x7f71900046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:29.839+0000 7f71977fe640 1 -- 172.21.15.133:0/1780833475 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71a015ac50 con 0x7f71a013b990 2024-04-25T10:14:30.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.035+0000 7f71a501a640 1 -- 172.21.15.133:0/1780833475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f71a0068e80 con 0x7f71a013b990 2024-04-25T10:14:30.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.703+0000 7f71977fe640 1 -- 172.21.15.133:0/1780833475 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7188055b20 con 0x7f71a013b990 2024-04-25T10:14:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 -- 172.21.15.133:0/1780833475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7180066f60 msgr2=0x7f7180069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7180066f60 0x7f7180069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f71900af000 tx=0x7f71900046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 -- 172.21.15.133:0/1780833475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71a013b990 msgr2=0x7f71a013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71a013b990 0x7f71a013b160 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f7188007fd0 tx=0x7f718800dc70 comp rx=0 tx=0).stop 2024-04-25T10:14:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 -- 172.21.15.133:0/1780833475 shutdown_connections 2024-04-25T10:14:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f71a013e600 0x7f71a0153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7180066f60 0x7f7180069420 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71a013b990 0x7f71a013b160 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 --2- 172.21.15.133:0/1780833475 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f718c0039c0 0x7f71a01f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 -- 172.21.15.133:0/1780833475 >> 172.21.15.133:0/1780833475 conn(0x7f71a0066f70 msgr2=0x7f71a015d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 -- 172.21.15.133:0/1780833475 shutdown_connections 2024-04-25T10:14:30.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.707+0000 7f71a501a640 1 -- 172.21.15.133:0/1780833475 wait complete. 2024-04-25T10:14:30.732 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:14:30.732 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:30.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.847+0000 7fe3614f4640 1 Processor -- start 2024-04-25T10:14:30.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe3614f4640 1 -- start start 2024-04-25T10:14:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe3614f4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe35c15c9a0 0x7fe35c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe3614f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe35c0b1f70 0x7fe35c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe3614f4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c0c61d0 con 0x7fe35c0ce330 2024-04-25T10:14:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe3614f4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35c0c64b0 con 0x7fe35c15c9a0 2024-04-25T10:14:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe3614f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c06d4f0 con 0x7fe35c0b1f70 2024-04-25T10:14:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35b7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe35c0ce330 0x7fe35c15d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48604/0 (socket says 172.21.15.133:48604) 2024-04-25T10:14:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35b7fe640 1 -- 172.21.15.133:0/3420549795 learned_addr learned my addr 172.21.15.133:0/3420549795 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985584464 0 0) 0x7fe35c0c61d0 con 0x7fe35c0ce330 2024-04-25T10:14:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe344003530 con 0x7fe35c0ce330 2024-04-25T10:14:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35a7fc640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe35c0b1f70 0x7fe35c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35affd640 1 --2- 172.21.15.133:0/3420549795 >> v2:172.21.15.162:3300/0 conn(0x7fe35c15c9a0 0x7fe35c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3260593908 0 0) 0x7fe344003530 con 0x7fe35c0ce330 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe35c0b1f70 msgr2=0x7fe35c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe35c0b1f70 0x7fe35c06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 >> v2:172.21.15.162:3300/0 conn(0x7fe35c15c9a0 msgr2=0x7fe35c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 --2- 172.21.15.133:0/3420549795 >> v2:172.21.15.162:3300/0 conn(0x7fe35c15c9a0 0x7fe35c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c06d7d0 con 0x7fe35c0ce330 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe354001a00 con 0x7fe35c0ce330 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 >> v1:172.21.15.133:6789/0 conn(0x7fe35c0ce330 legacy=0x7fe35c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3440039c0 0x7fe344003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe344004810 0x7fe344004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe35c0b1f70 0x7fe3440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0c64b0 con 0x7fe344004810 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c06d4f0 con 0x7fe3440039c0 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34400ba80 con 0x7fe35c0b1f70 2024-04-25T10:14:30.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35a7fc640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe344004810 0x7fe344004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35affd640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3440039c0 0x7fe344003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35a7fc640 1 -- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe35c0b1f70 msgr2=0x7fe3440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35b7fe640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe35c0b1f70 0x7fe3440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35a7fc640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe35c0b1f70 0x7fe3440051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35a7fc640 1 -- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3440039c0 msgr2=0x7fe344003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35a7fc640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3440039c0 0x7fe344003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35a7fc640 1 -- 172.21.15.133:0/3420549795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe344003530 con 0x7fe344004810 2024-04-25T10:14:30.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.851+0000 7fe35affd640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3440039c0 0x7fe344003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35a7fc640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe344004810 0x7fe344004c90 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00ba00 tx=0x7fe34c01c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866eeab99e2f1b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c00b600 con 0x7fe344004810 2024-04-25T10:14:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0b0070 con 0x7fe344004810 2024-04-25T10:14:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe359ffb640 1 -- 172.21.15.133:0/3420549795 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0ab890 con 0x7fe344004810 2024-04-25T10:14:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 -- 172.21.15.133:0/3420549795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe344004810 msgr2=0x7fe344004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 --2- 172.21.15.133:0/3420549795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe344004810 0x7fe344004c90 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00ba00 tx=0x7fe34c01c640 comp rx=0 tx=0).stop 2024-04-25T10:14:30.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35b7fe640 1 -- 172.21.15.133:0/3420549795 reap_dead start 2024-04-25T10:14:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 -- 172.21.15.133:0/3420549795 shutdown_connections 2024-04-25T10:14:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 -- 172.21.15.133:0/3420549795 >> 172.21.15.133:0/3420549795 conn(0x7fe35c067000 msgr2=0x7fe35c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 -- 172.21.15.133:0/3420549795 shutdown_connections 2024-04-25T10:14:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 -- 172.21.15.133:0/3420549795 wait complete. 2024-04-25T10:14:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 Processor -- start 2024-04-25T10:14:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 -- start start 2024-04-25T10:14:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe344004810 0x7fe35c0a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c1488e0 0x7fe35c0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe35c0ab810 0x7fe35c0abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c15cf00 con 0x7fe35c1488e0 2024-04-25T10:14:30.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c15cd80 con 0x7fe35c0ab810 2024-04-25T10:14:30.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe3614f4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c15d850 con 0x7fe344004810 2024-04-25T10:14:30.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c1488e0 0x7fe35c0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c1488e0 0x7fe35c0ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37366/0 (socket says 172.21.15.133:37366) 2024-04-25T10:14:30.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35a7fc640 1 -- 172.21.15.133:0/1387043806 learned_addr learned my addr 172.21.15.133:0/1387043806 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:30.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35affd640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe344004810 0x7fe35c0a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35b7fe640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe35c0ab810 0x7fe35c0abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35a7fc640 1 -- 172.21.15.133:0/1387043806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe344004810 msgr2=0x7fe35c0a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35a7fc640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe344004810 0x7fe35c0a9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35a7fc640 1 -- 172.21.15.133:0/1387043806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe35c0ab810 msgr2=0x7fe35c0abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35a7fc640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe35c0ab810 0x7fe35c0abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35a7fc640 1 -- 172.21.15.133:0/1387043806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c148ec0 con 0x7fe35c1488e0 2024-04-25T10:14:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35affd640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe344004810 0x7fe35c0a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.855+0000 7fe35a7fc640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c1488e0 0x7fe35c0ca830 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fe34c01c470 tx=0x7fe344005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.859+0000 7fe3437fe640 1 -- 172.21.15.133:0/1387043806 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0ba070 con 0x7fe35c1488e0 2024-04-25T10:14:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.859+0000 7fe3437fe640 1 -- 172.21.15.133:0/1387043806 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c004220 con 0x7fe35c1488e0 2024-04-25T10:14:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.859+0000 7fe3614f4640 1 -- 172.21.15.133:0/1387043806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c0cb080 con 0x7fe35c1488e0 2024-04-25T10:14:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.859+0000 7fe3437fe640 1 -- 172.21.15.133:0/1387043806 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0ab820 con 0x7fe35c1488e0 2024-04-25T10:14:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.859+0000 7fe3614f4640 1 -- 172.21.15.133:0/1387043806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c0cb640 con 0x7fe35c1488e0 2024-04-25T10:14:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.859+0000 7fe3614f4640 1 -- 172.21.15.133:0/1387043806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c068ef0 con 0x7fe35c1488e0 2024-04-25T10:14:30.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.859+0000 7fe3437fe640 1 -- 172.21.15.133:0/1387043806 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe34c0ab9c0 con 0x7fe35c1488e0 2024-04-25T10:14:30.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.859+0000 7fe3437fe640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe348066f90 0x7fe348069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.859+0000 7fe3437fe640 1 -- 172.21.15.133:0/1387043806 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe34c11f600 con 0x7fe35c1488e0 2024-04-25T10:14:30.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.859+0000 7fe35affd640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe348066f90 0x7fe348069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.859+0000 7fe35affd640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe348066f90 0x7fe348069450 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe350009980 tx=0x7fe350008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:30.863+0000 7fe3437fe640 1 -- 172.21.15.133:0/1387043806 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe34c0c4060 con 0x7fe35c1488e0 2024-04-25T10:14:31.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 -- 172.21.15.133:0/1387043806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe35c0a9590 con 0x7fe35c1488e0 2024-04-25T10:14:31.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3437fe640 1 -- 172.21.15.133:0/1387043806 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4958 (secure 0 0 0) 0x7fe34c016350 con 0x7fe35c1488e0 2024-04-25T10:14:31.058 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:31.058 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3448301688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3448301688},{"type":"v1","addr":"172.21.15.162:6833","nonce":3448301688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/4188704231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":4188704231},{"type":"v1","addr":"172.21.15.162:6835","nonce":4188704231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":10041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2626366082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2626366082},{"type":"v1","addr":"172.21.15.133:6835","nonce":2626366082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:28.005823+0000","modified":"2024-04-25T10:14:30.040268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10050":{"gid":10050,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3234096063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3234096063},{"type":"v1","addr":"172.21.15.133:6837","nonce":3234096063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:14:31.058 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 90 2024-04-25T10:14:31.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 -- 172.21.15.133:0/1387043806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe348066f90 msgr2=0x7fe348069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe348066f90 0x7fe348069450 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe350009980 tx=0x7fe350008040 comp rx=0 tx=0).stop 2024-04-25T10:14:31.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 -- 172.21.15.133:0/1387043806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c1488e0 msgr2=0x7fe35c0ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c1488e0 0x7fe35c0ca830 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fe34c01c470 tx=0x7fe344005840 comp rx=0 tx=0).stop 2024-04-25T10:14:31.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 -- 172.21.15.133:0/1387043806 shutdown_connections 2024-04-25T10:14:31.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe35c0ab810 0x7fe35c0abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe348066f90 0x7fe348069450 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c1488e0 0x7fe35c0ca830 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 --2- 172.21.15.133:0/1387043806 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe344004810 0x7fe35c0a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 -- 172.21.15.133:0/1387043806 >> 172.21.15.133:0/1387043806 conn(0x7fe35c067000 msgr2=0x7fe35c0b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 -- 172.21.15.133:0/1387043806 shutdown_connections 2024-04-25T10:14:31.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.055+0000 7fe3614f4640 1 -- 172.21.15.133:0/1387043806 wait complete. 2024-04-25T10:14:31.076 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:31.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabe0a17640 1 Processor -- start 2024-04-25T10:14:31.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabe0a17640 1 -- start start 2024-04-25T10:14:31.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabe0a17640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabdc0b1f70 0x7fabdc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabe0a17640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fabdc0ce330 0x7fabdc0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabe0a17640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc0c61d0 con 0x7fabdc15c9a0 2024-04-25T10:14:31.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabe0a17640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc0cece0 con 0x7fabdc0ce330 2024-04-25T10:14:31.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabe0a17640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0c64b0 con 0x7fabdc0b1f70 2024-04-25T10:14:31.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabdad9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fabdc0ce330 0x7fabdc0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabda59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fabdc15c9a0 0x7fabdc0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48608/0 (socket says 172.21.15.133:48608) 2024-04-25T10:14:31.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabda59c640 1 -- 172.21.15.133:0/2044724429 learned_addr learned my addr 172.21.15.133:0/2044724429 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:31.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabd1d9b640 1 --2- 172.21.15.133:0/2044724429 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabdc0b1f70 0x7fabdc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabdad9d640 1 -- 172.21.15.133:0/2044724429 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabdc0b1f70 msgr2=0x7fabdc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabdad9d640 1 --2- 172.21.15.133:0/2044724429 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabdc0b1f70 0x7fabdc06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabdad9d640 1 -- 172.21.15.133:0/2044724429 >> v1:172.21.15.133:6789/0 conn(0x7fabdc15c9a0 legacy=0x7fabdc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabd9d9b640 1 -- 172.21.15.133:0/2044724429 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801987899 0 0) 0x7fabdc0c61d0 con 0x7fabdc15c9a0 2024-04-25T10:14:31.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabdad9d640 1 -- 172.21.15.133:0/2044724429 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc06d4f0 con 0x7fabdc0ce330 2024-04-25T10:14:31.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabdad9d640 1 --2- 172.21.15.133:0/2044724429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc0ce330 0x7fabdc0ce7a0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009d90 tx=0x7fabd401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feffea42aae3be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabd9d9b640 1 -- 172.21.15.133:0/2044724429 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd40b2070 con 0x7fabdc0ce330 2024-04-25T10:14:31.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabd9d9b640 1 -- 172.21.15.133:0/2044724429 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40a5910 con 0x7fabdc0ce330 2024-04-25T10:14:31.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.187+0000 7fabd9d9b640 1 -- 172.21.15.133:0/2044724429 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd40a5ce0 con 0x7fabdc0ce330 2024-04-25T10:14:31.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 -- 172.21.15.133:0/2044724429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc0ce330 msgr2=0x7fabdc0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 --2- 172.21.15.133:0/2044724429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc0ce330 0x7fabdc0ce7a0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009d90 tx=0x7fabd401c940 comp rx=0 tx=0).stop 2024-04-25T10:14:31.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 -- 172.21.15.133:0/2044724429 shutdown_connections 2024-04-25T10:14:31.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 --2- 172.21.15.133:0/2044724429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc0ce330 0x7fabdc0ce7a0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 --2- 172.21.15.133:0/2044724429 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabdc0b1f70 0x7fabdc06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 -- 172.21.15.133:0/2044724429 >> 172.21.15.133:0/2044724429 conn(0x7fabdc067000 msgr2=0x7fabdc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 -- 172.21.15.133:0/2044724429 shutdown_connections 2024-04-25T10:14:31.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 -- 172.21.15.133:0/2044724429 wait complete. 2024-04-25T10:14:31.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 Processor -- start 2024-04-25T10:14:31.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 -- start start 2024-04-25T10:14:31.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc0b1f70 0x7fabdc153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabdc0ce330 0x7fabdc14e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabdc15c9a0 0x7fabdc14ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc15dad0 con 0x7fabdc0b1f70 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc15d950 con 0x7fabdc15c9a0 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc15dc50 con 0x7fabdc0ce330 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabda59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc0b1f70 0x7fabdc153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabda59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc0b1f70 0x7fabdc153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37374/0 (socket says 172.21.15.133:37374) 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabda59c640 1 -- 172.21.15.133:0/3522990980 learned_addr learned my addr 172.21.15.133:0/3522990980 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabd1d9b640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabdc0ce330 0x7fabdc14e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabda59c640 1 -- 172.21.15.133:0/3522990980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabdc0ce330 msgr2=0x7fabdc14e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabda59c640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabdc0ce330 0x7fabdc14e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabda59c640 1 -- 172.21.15.133:0/3522990980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabdc15c9a0 msgr2=0x7fabdc14ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabdad9d640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabdc15c9a0 0x7fabdc14ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabda59c640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabdc15c9a0 0x7fabdc14ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabda59c640 1 -- 172.21.15.133:0/3522990980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc14f960 con 0x7fabdc0b1f70 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabda59c640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc0b1f70 0x7fabdc153740 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00c480 tx=0x7fabcc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabd37fe640 1 -- 172.21.15.133:0/3522990980 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc018070 con 0x7fabdc0b1f70 2024-04-25T10:14:31.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabd37fe640 1 -- 172.21.15.133:0/3522990980 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc007500 con 0x7fabdc0b1f70 2024-04-25T10:14:31.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 -- 172.21.15.133:0/3522990980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc14f680 con 0x7fabdc0b1f70 2024-04-25T10:14:31.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabd37fe640 1 -- 172.21.15.133:0/3522990980 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc004a30 con 0x7fabdc0b1f70 2024-04-25T10:14:31.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.191+0000 7fabe0a17640 1 -- 172.21.15.133:0/3522990980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc1590a0 con 0x7fabdc0b1f70 2024-04-25T10:14:31.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.195+0000 7fabe0a17640 1 -- 172.21.15.133:0/3522990980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc158bf0 con 0x7fabdc0b1f70 2024-04-25T10:14:31.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.195+0000 7fabd37fe640 1 -- 172.21.15.133:0/3522990980 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fabcc01b070 con 0x7fabdc0b1f70 2024-04-25T10:14:31.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.195+0000 7fabd37fe640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fabbc066f60 0x7fabbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.195+0000 7fabd37fe640 1 -- 172.21.15.133:0/3522990980 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fabcc0844f0 con 0x7fabdc0b1f70 2024-04-25T10:14:31.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.195+0000 7fabd1d9b640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fabbc066f60 0x7fabbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.195+0000 7fabd1d9b640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fabbc066f60 0x7fabbc069420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fabdc1507e0 tx=0x7fabc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.199+0000 7fabd37fe640 1 -- 172.21.15.133:0/3522990980 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fabcc055400 con 0x7fabdc0b1f70 2024-04-25T10:14:31.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.359+0000 7fabe0a17640 1 -- 172.21.15.133:0/3522990980 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabdc06bc20 con 0x7fabdc0b1f70 2024-04-25T10:14:31.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabd37fe640 1 -- 172.21.15.133:0/3522990980 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16916 (secure 0 0 0) 0x7fabcc054d00 con 0x7fabdc0b1f70 2024-04-25T10:14:31.365 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:31.365 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":143,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:29.688089+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:26.340459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:26.996481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 -- 172.21.15.133:0/3522990980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fabbc066f60 msgr2=0x7fabbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fabbc066f60 0x7fabbc069420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fabdc1507e0 tx=0x7fabc4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 -- 172.21.15.133:0/3522990980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc0b1f70 msgr2=0x7fabdc153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc0b1f70 0x7fabdc153740 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00c480 tx=0x7fabcc00c940 comp rx=0 tx=0).stop 2024-04-25T10:14:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 -- 172.21.15.133:0/3522990980 shutdown_connections 2024-04-25T10:14:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabdc15c9a0 0x7fabdc14ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fabdc0ce330 0x7fabdc14e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fabbc066f60 0x7fabbc069420 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 --2- 172.21.15.133:0/3522990980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc0b1f70 0x7fabdc153740 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 -- 172.21.15.133:0/3522990980 >> 172.21.15.133:0/3522990980 conn(0x7fabdc067000 msgr2=0x7fabdc15de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 -- 172.21.15.133:0/3522990980 shutdown_connections 2024-04-25T10:14:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.363+0000 7fabe0a17640 1 -- 172.21.15.133:0/3522990980 wait complete. 2024-04-25T10:14:31.384 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:14:31.385 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:14:31.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.503+0000 7f1b1208a640 1 Processor -- start 2024-04-25T10:14:31.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.503+0000 7f1b1208a640 1 -- start start 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b1208a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1b0c0b1f70 0x7f1b0c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b1208a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b0c0ce330 0x7f1b0c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b1208a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0c0c61d0 con 0x7f1b0c15c9a0 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b1208a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c0cecc0 con 0x7f1b0c0b1f70 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b1208a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0c64b0 con 0x7f1b0c0ce330 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b0affd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1b0c0b1f70 0x7f1b0c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b0affd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1b0c0b1f70 0x7f1b0c06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37386/0 (socket says 172.21.15.133:37386) 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b0affd640 1 -- 172.21.15.133:0/1676549047 learned_addr learned my addr 172.21.15.133:0/1676549047 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b0bfff640 1 --2- 172.21.15.133:0/1676549047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b0c0ce330 0x7f1b0c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b0affd640 1 -- 172.21.15.133:0/1676549047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b0c0ce330 msgr2=0x7f1b0c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b0affd640 1 --2- 172.21.15.133:0/1676549047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b0c0ce330 0x7f1b0c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b0affd640 1 -- 172.21.15.133:0/1676549047 >> v1:172.21.15.162:6789/0 conn(0x7f1b0c15c9a0 legacy=0x7f1b0c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b0affd640 1 -- 172.21.15.133:0/1676549047 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c06d4f0 con 0x7f1b0c0b1f70 2024-04-25T10:14:31.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.507+0000 7f1b0affd640 1 --2- 172.21.15.133:0/1676549047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0b1f70 0x7f1b0c06dcf0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009870 tx=0x7f1af401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d600dc21cce873d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0a7fc640 1 -- 172.21.15.133:0/1676549047 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af40a9610 con 0x7f1b0c0b1f70 2024-04-25T10:14:31.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0a7fc640 1 -- 172.21.15.133:0/1676549047 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af40a97b0 con 0x7f1b0c0b1f70 2024-04-25T10:14:31.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0a7fc640 1 -- 172.21.15.133:0/1676549047 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af40a9b80 con 0x7f1b0c0b1f70 2024-04-25T10:14:31.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 -- 172.21.15.133:0/1676549047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0b1f70 msgr2=0x7f1b0c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 --2- 172.21.15.133:0/1676549047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0b1f70 0x7f1b0c06dcf0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009870 tx=0x7f1af401c530 comp rx=0 tx=0).stop 2024-04-25T10:14:31.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 -- 172.21.15.133:0/1676549047 shutdown_connections 2024-04-25T10:14:31.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 --2- 172.21.15.133:0/1676549047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b0c0ce330 0x7f1b0c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 --2- 172.21.15.133:0/1676549047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0b1f70 0x7f1b0c06dcf0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 -- 172.21.15.133:0/1676549047 >> 172.21.15.133:0/1676549047 conn(0x7f1b0c067000 msgr2=0x7f1b0c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 -- 172.21.15.133:0/1676549047 shutdown_connections 2024-04-25T10:14:31.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 -- 172.21.15.133:0/1676549047 wait complete. 2024-04-25T10:14:31.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 Processor -- start 2024-04-25T10:14:31.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 -- start start 2024-04-25T10:14:31.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b0c0b1f70 0x7f1b0c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce330 0x7f1b0c138d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b0c15c9a0 0x7f1b0c14a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c15dad0 con 0x7f1b0c0ce330 2024-04-25T10:14:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c15d950 con 0x7f1b0c0b1f70 2024-04-25T10:14:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b1208a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c15dc50 con 0x7f1b0c15c9a0 2024-04-25T10:14:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce330 0x7f1b0c138d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0b7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b0c0b1f70 0x7f1b0c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b0c15c9a0 0x7f1b0c14a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0b7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b0c0b1f70 0x7f1b0c1562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55410/0 (socket says 172.21.15.133:55410) 2024-04-25T10:14:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0b7fe640 1 -- 172.21.15.133:0/3317805659 learned_addr learned my addr 172.21.15.133:0/3317805659 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0affd640 1 -- 172.21.15.133:0/3317805659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b0c15c9a0 msgr2=0x7f1b0c14a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0affd640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b0c15c9a0 0x7f1b0c14a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0affd640 1 -- 172.21.15.133:0/3317805659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b0c0b1f70 msgr2=0x7f1b0c1562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0affd640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b0c0b1f70 0x7f1b0c1562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0affd640 1 -- 172.21.15.133:0/3317805659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c155290 con 0x7f1b0c0ce330 2024-04-25T10:14:31.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0bfff640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b0c15c9a0 0x7f1b0c14a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:31.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.511+0000 7f1b0affd640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce330 0x7f1b0c138d50 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0b4920 tx=0x7f1af40192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.515+0000 7f1b03fff640 1 -- 172.21.15.133:0/3317805659 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af40b9630 con 0x7f1b0c0ce330 2024-04-25T10:14:31.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.515+0000 7f1b03fff640 1 -- 172.21.15.133:0/3317805659 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af40b9810 con 0x7f1b0c0ce330 2024-04-25T10:14:31.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.515+0000 7f1b1208a640 1 -- 172.21.15.133:0/3317805659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c155550 con 0x7f1b0c0ce330 2024-04-25T10:14:31.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.515+0000 7f1b03fff640 1 -- 172.21.15.133:0/3317805659 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af40b9c00 con 0x7f1b0c0ce330 2024-04-25T10:14:31.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.515+0000 7f1b1208a640 1 -- 172.21.15.133:0/3317805659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c155b10 con 0x7f1b0c0ce330 2024-04-25T10:14:31.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.515+0000 7f1b1208a640 1 -- 172.21.15.133:0/3317805659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0c068ef0 con 0x7f1b0c0ce330 2024-04-25T10:14:31.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.515+0000 7f1b03fff640 1 -- 172.21.15.133:0/3317805659 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1af40a9650 con 0x7f1b0c0ce330 2024-04-25T10:14:31.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.515+0000 7f1b03fff640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1aec066f60 0x7f1aec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.515+0000 7f1b0b7fe640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1aec066f60 0x7f1aec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.515+0000 7f1b03fff640 1 -- 172.21.15.133:0/3317805659 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1af411f2c0 con 0x7f1b0c0ce330 2024-04-25T10:14:31.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.515+0000 7f1b0b7fe640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1aec066f60 0x7f1aec069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1afc001f60 tx=0x7f1afc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.519+0000 7f1b03fff640 1 -- 172.21.15.133:0/3317805659 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1af40ad020 con 0x7f1b0c0ce330 2024-04-25T10:14:31.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.675+0000 7f1b1208a640 1 -- 172.21.15.133:0/3317805659 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b0c0c5a90 con 0x7f1aec066f60 2024-04-25T10:14:31.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.683+0000 7f1b03fff640 1 -- 172.21.15.133:0/3317805659 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252453 (secure 0 0 0) 0x7f1b0c0c5a90 con 0x7f1aec066f60 2024-04-25T10:14:31.691 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:31.693 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.687+0000 7f1b1208a640 1 -- 172.21.15.133:0/3317805659 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1aec066f60 msgr2=0x7f1aec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.687+0000 7f1b1208a640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1aec066f60 0x7f1aec069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1afc001f60 tx=0x7f1afc009040 comp rx=0 tx=0).stop 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.687+0000 7f1b1208a640 1 -- 172.21.15.133:0/3317805659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce330 msgr2=0x7f1b0c138d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.687+0000 7f1b1208a640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce330 0x7f1b0c138d50 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0b4920 tx=0x7f1af40192d0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.687+0000 7f1b1208a640 1 -- 172.21.15.133:0/3317805659 shutdown_connections 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.687+0000 7f1b1208a640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1b0c15c9a0 0x7f1b0c14a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.687+0000 7f1b1208a640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1aec066f60 0x7f1aec069420 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.691+0000 7f1b1208a640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b0c0ce330 0x7f1b0c138d50 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.691+0000 7f1b1208a640 1 --2- 172.21.15.133:0/3317805659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1b0c0b1f70 0x7f1b0c1562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.691+0000 7f1b1208a640 1 -- 172.21.15.133:0/3317805659 >> 172.21.15.133:0/3317805659 conn(0x7f1b0c067000 msgr2=0x7f1b0c15b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.691+0000 7f1b1208a640 1 -- 172.21.15.133:0/3317805659 shutdown_connections 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.691+0000 7f1b1208a640 1 -- 172.21.15.133:0/3317805659 wait complete. 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2024-04-25T10:14:29.948919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":502020,"kb_used_data":15156,"kb_used_omap":12,"kb_used_meta":486707,"kb_avail":754472700,"statfs":{"total":773094113280,"available":772580044800,"internally_reserved":0,"allocated":15519744,"data_stored":7862928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":498388529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001097"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710816+0000","last_change":"2024-04-25T10:14:28.735551+0000","last_active":"2024-04-25T10:14:29.710816+0000","last_peered":"2024-04-25T10:14:29.710816+0000","last_clean":"2024-04-25T10:14:29.710816+0000","last_became_active":"2024-04-25T10:14:28.735031+0000","last_became_peered":"2024-04-25T10:14:28.735031+0000","last_unstale":"2024-04-25T10:14:29.710816+0000","last_undegraded":"2024-04-25T10:14:29.710816+0000","last_fullsized":"2024-04-25T10:14:29.710816+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:44.556550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.698117+0000","last_change":"2024-04-25T10:14:28.695866+0000","last_active":"2024-04-25T10:14:28.698117+0000","last_peered":"2024-04-25T10:14:28.698117+0000","last_clean":"2024-04-25T10:14:28.698117+0000","last_became_active":"2024-04-25T10:14:28.695235+0000","last_became_peered":"2024-04-25T10:14:28.695235+0000","last_unstale":"2024-04-25T10:14:28.698117+0000","last_undegraded":"2024-04-25T10:14:28.698117+0000","last_fullsized":"2024-04-25T10:14:28.698117+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:59.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.699303+0000","last_change":"2024-04-25T10:14:28.697833+0000","last_active":"2024-04-25T10:14:28.699303+0000","last_peered":"2024-04-25T10:14:28.699303+0000","last_clean":"2024-04-25T10:14:28.699303+0000","last_became_active":"2024-04-25T10:14:28.695386+0000","last_became_peered":"2024-04-25T10:14:28.695386+0000","last_unstale":"2024-04-25T10:14:28.699303+0000","last_undegraded":"2024-04-25T10:14:28.699303+0000","last_fullsized":"2024-04-25T10:14:28.699303+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:45.995012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.706511+0000","last_change":"2024-04-25T10:14:28.711671+0000","last_active":"2024-04-25T10:14:29.706511+0000","last_peered":"2024-04-25T10:14:29.706511+0000","last_clean":"2024-04-25T10:14:29.706511+0000","last_became_active":"2024-04-25T10:14:28.710920+0000","last_became_peered":"2024-04-25T10:14:28.710920+0000","last_unstale":"2024-04-25T10:14:29.706511+0000","last_undegraded":"2024-04-25T10:14:29.706511+0000","last_fullsized":"2024-04-25T10:14:29.706511+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:46.915924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.719956+0000","last_change":"2024-04-25T10:14:28.713697+0000","last_active":"2024-04-25T10:14:28.719956+0000","last_peered":"2024-04-25T10:14:28.719956+0000","last_clean":"2024-04-25T10:14:28.719956+0000","last_became_active":"2024-04-25T10:14:28.713268+0000","last_became_peered":"2024-04-25T10:14:28.713268+0000","last_unstale":"2024-04-25T10:14:28.719956+0000","last_undegraded":"2024-04-25T10:14:28.719956+0000","last_fullsized":"2024-04-25T10:14:28.719956+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:36.344770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.719761+0000","last_change":"2024-04-25T10:14:28.712694+0000","last_active":"2024-04-25T10:14:28.719761+0000","last_peered":"2024-04-25T10:14:28.719761+0000","last_clean":"2024-04-25T10:14:28.719761+0000","last_became_active":"2024-04-25T10:14:28.711745+0000","last_became_peered":"2024-04-25T10:14:28.711745+0000","last_unstale":"2024-04-25T10:14:28.719761+0000","last_undegraded":"2024-04-25T10:14:28.719761+0000","last_fullsized":"2024-04-25T10:14:28.719761+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:44.556550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710333+0000","last_change":"2024-04-25T10:14:28.728500+0000","last_active":"2024-04-25T10:14:29.710333+0000","last_peered":"2024-04-25T10:14:29.710333+0000","last_clean":"2024-04-25T10:14:29.710333+0000","last_became_active":"2024-04-25T10:14:28.728027+0000","last_became_peered":"2024-04-25T10:14:28.728027+0000","last_unstale":"2024-04-25T10:14:29.710333+0000","last_undegraded":"2024-04-25T10:14:29.710333+0000","last_fullsized":"2024-04-25T10:14:29.710333+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:48.641809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.703879+0000","last_change":"2024-04-25T10:14:28.736090+0000","last_active":"2024-04-25T10:14:29.703879+0000","last_peered":"2024-04-25T10:14:29.703879+0000","last_clean":"2024-04-25T10:14:29.703879+0000","last_became_active":"2024-04-25T10:14:28.733147+0000","last_became_peered":"2024-04-25T10:14:28.733147+0000","last_unstale":"2024-04-25T10:14:29.703879+0000","last_undegraded":"2024-04-25T10:14:29.703879+0000","last_fullsized":"2024-04-25T10:14:29.703879+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.758052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.692062+0000","last_change":"2024-04-25T10:14:28.687342+0000","last_active":"2024-04-25T10:14:28.692062+0000","last_peered":"2024-04-25T10:14:28.692062+0000","last_clean":"2024-04-25T10:14:28.692062+0000","last_became_active":"2024-04-25T10:14:28.684251+0000","last_became_peered":"2024-04-25T10:14:28.684251+0000","last_unstale":"2024-04-25T10:14:28.692062+0000","last_undegraded":"2024-04-25T10:14:28.692062+0000","last_fullsized":"2024-04-25T10:14:28.692062+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:06.877297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.691426+0000","last_change":"2024-04-25T10:14:28.689473+0000","last_active":"2024-04-25T10:14:28.691426+0000","last_peered":"2024-04-25T10:14:28.691426+0000","last_clean":"2024-04-25T10:14:28.691426+0000","last_became_active":"2024-04-25T10:14:28.689113+0000","last_became_peered":"2024-04-25T10:14:28.689113+0000","last_unstale":"2024-04-25T10:14:28.691426+0000","last_undegraded":"2024-04-25T10:14:28.691426+0000","last_fullsized":"2024-04-25T10:14:28.691426+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:22.157823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.726010+0000","last_change":"2024-04-25T10:14:28.725325+0000","last_active":"2024-04-25T10:14:28.726010+0000","last_peered":"2024-04-25T10:14:28.726010+0000","last_clean":"2024-04-25T10:14:28.726010+0000","last_became_active":"2024-04-25T10:14:28.725124+0000","last_became_peered":"2024-04-25T10:14:28.725124+0000","last_unstale":"2024-04-25T10:14:28.726010+0000","last_undegraded":"2024-04-25T10:14:28.726010+0000","last_fullsized":"2024-04-25T10:14:28.726010+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:16.574486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.706917+0000","last_change":"2024-04-25T10:14:28.701620+0000","last_active":"2024-04-25T10:14:29.706917+0000","last_peered":"2024-04-25T10:14:29.706917+0000","last_clean":"2024-04-25T10:14:29.706917+0000","last_became_active":"2024-04-25T10:14:28.700878+0000","last_became_peered":"2024-04-25T10:14:28.700878+0000","last_unstale":"2024-04-25T10:14:29.706917+0000","last_undegraded":"2024-04-25T10:14:29.706917+0000","last_fullsized":"2024-04-25T10:14:29.706917+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:59.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.693496+0000","last_change":"2024-04-25T10:14:28.692680+0000","last_active":"2024-04-25T10:14:28.693496+0000","last_peered":"2024-04-25T10:14:28.693496+0000","last_clean":"2024-04-25T10:14:28.693496+0000","last_became_active":"2024-04-25T10:14:28.692327+0000","last_became_peered":"2024-04-25T10:14:28.692327+0000","last_unstale":"2024-04-25T10:14:28.693496+0000","last_undegraded":"2024-04-25T10:14:28.693496+0000","last_fullsized":"2024-04-25T10:14:28.693496+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:48.641809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.706307+0000","last_change":"2024-04-25T10:14:28.702406+0000","last_active":"2024-04-25T10:14:29.706307+0000","last_peered":"2024-04-25T10:14:29.706307+0000","last_clean":"2024-04-25T10:14:29.706307+0000","last_became_active":"2024-04-25T10:14:28.701610+0000","last_became_peered":"2024-04-25T10:14:28.701610+0000","last_unstale":"2024-04-25T10:14:29.706307+0000","last_undegraded":"2024-04-25T10:14:29.706307+0000","last_fullsized":"2024-04-25T10:14:29.706307+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:44.925639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.713520+0000","last_change":"2024-04-25T10:14:28.698707+0000","last_active":"2024-04-25T10:14:28.713520+0000","last_peered":"2024-04-25T10:14:28.713520+0000","last_clean":"2024-04-25T10:14:28.713520+0000","last_became_active":"2024-04-25T10:14:28.697985+0000","last_became_peered":"2024-04-25T10:14:28.697985+0000","last_unstale":"2024-04-25T10:14:28.713520+0000","last_undegraded":"2024-04-25T10:14:28.713520+0000","last_fullsized":"2024-04-25T10:14:28.713520+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:06.877297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.734273+0000","last_change":"2024-04-25T10:14:28.733839+0000","last_active":"2024-04-25T10:14:28.734273+0000","last_peered":"2024-04-25T10:14:28.734273+0000","last_clean":"2024-04-25T10:14:28.734273+0000","last_became_active":"2024-04-25T10:14:28.733589+0000","last_became_peered":"2024-04-25T10:14:28.733589+0000","last_unstale":"2024-04-25T10:14:28.734273+0000","last_undegraded":"2024-04-25T10:14:28.734273+0000","last_fullsized":"2024-04-25T10:14:28.734273+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:09.909912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.707887+0000","last_change":"2024-04-25T10:14:28.706734+0000","last_active":"2024-04-25T10:14:29.707887+0000","last_peered":"2024-04-25T10:14:29.707887+0000","last_clean":"2024-04-25T10:14:29.707887+0000","last_became_active":"2024-04-25T10:14:28.700630+0000","last_became_peered":"2024-04-25T10:14:28.700630+0000","last_unstale":"2024-04-25T10:14:29.707887+0000","last_undegraded":"2024-04-25T10:14:29.707887+0000","last_fullsized":"2024-04-25T10:14:29.707887+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:19.717882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.716463+0000","last_change":"2024-04-25T10:14:28.715825+0000","last_active":"2024-04-25T10:14:28.716463+0000","last_peered":"2024-04-25T10:14:28.716463+0000","last_clean":"2024-04-25T10:14:28.716463+0000","last_became_active":"2024-04-25T10:14:28.714478+0000","last_became_peered":"2024-04-25T10:14:28.714478+0000","last_unstale":"2024-04-25T10:14:28.716463+0000","last_undegraded":"2024-04-25T10:14:28.716463+0000","last_fullsized":"2024-04-25T10:14:28.716463+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:06.403336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.716385+0000","last_change":"2024-04-25T10:14:28.704667+0000","last_active":"2024-04-25T10:14:29.716385+0000","last_peered":"2024-04-25T10:14:29.716385+0000","last_clean":"2024-04-25T10:14:29.716385+0000","last_became_active":"2024-04-25T10:14:28.703614+0000","last_became_peered":"2024-04-25T10:14:28.703614+0000","last_unstale":"2024-04-25T10:14:29.716385+0000","last_undegraded":"2024-04-25T10:14:29.716385+0000","last_fullsized":"2024-04-25T10:14:29.716385+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:25.424823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.700240+0000","last_change":"2024-04-25T10:14:28.700047+0000","last_active":"2024-04-25T10:14:29.700240+0000","last_peered":"2024-04-25T10:14:29.700240+0000","last_clean":"2024-04-25T10:14:29.700240+0000","last_became_active":"2024-04-25T10:14:28.699718+0000","last_became_peered":"2024-04-25T10:14:28.699718+0000","last_unstale":"2024-04-25T10:14:29.700240+0000","last_undegraded":"2024-04-25T10:14:29.700240+0000","last_fullsized":"2024-04-25T10:14:29.700240+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:06.300000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.693447+0000","last_change":"2024-04-25T10:14:28.691844+0000","last_active":"2024-04-25T10:14:28.693447+0000","last_peered":"2024-04-25T10:14:28.693447+0000","last_clean":"2024-04-25T10:14:28.693447+0000","last_became_active":"2024-04-25T10:14:28.691375+0000","last_became_peered":"2024-04-25T10:14:28.691375+0000","last_unstale":"2024-04-25T10:14:28.693447+0000","last_undegraded":"2024-04-25T10:14:28.693447+0000","last_fullsized":"2024-04-25T10:14:28.693447+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:44.628062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710269+0000","last_change":"2024-04-25T10:14:28.727722+0000","last_active":"2024-04-25T10:14:29.710269+0000","last_peered":"2024-04-25T10:14:29.710269+0000","last_clean":"2024-04-25T10:14:29.710269+0000","last_became_active":"2024-04-25T10:14:28.727252+0000","last_became_peered":"2024-04-25T10:14:28.727252+0000","last_unstale":"2024-04-25T10:14:29.710269+0000","last_undegraded":"2024-04-25T10:14:29.710269+0000","last_fullsized":"2024-04-25T10:14:29.710269+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:57.109328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.703941+0000","last_change":"2024-04-25T10:14:28.721605+0000","last_active":"2024-04-25T10:14:29.703941+0000","last_peered":"2024-04-25T10:14:29.703941+0000","last_clean":"2024-04-25T10:14:29.703941+0000","last_became_active":"2024-04-25T10:14:28.719956+0000","last_became_peered":"2024-04-25T10:14:28.719956+0000","last_unstale":"2024-04-25T10:14:29.703941+0000","last_undegraded":"2024-04-25T10:14:29.703941+0000","last_fullsized":"2024-04-25T10:14:29.703941+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:04.773052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.707527+0000","last_change":"2024-04-25T10:14:28.705785+0000","last_active":"2024-04-25T10:14:28.707527+0000","last_peered":"2024-04-25T10:14:28.707527+0000","last_clean":"2024-04-25T10:14:28.707527+0000","last_became_active":"2024-04-25T10:14:28.703647+0000","last_became_peered":"2024-04-25T10:14:28.703647+0000","last_unstale":"2024-04-25T10:14:28.707527+0000","last_undegraded":"2024-04-25T10:14:28.707527+0000","last_fullsized":"2024-04-25T10:14:28.707527+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:23.165799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.711025+0000","last_change":"2024-04-25T10:14:28.712571+0000","last_active":"2024-04-25T10:14:29.711025+0000","last_peered":"2024-04-25T10:14:29.711025+0000","last_clean":"2024-04-25T10:14:29.711025+0000","last_became_active":"2024-04-25T10:14:28.710510+0000","last_became_peered":"2024-04-25T10:14:28.710510+0000","last_unstale":"2024-04-25T10:14:29.711025+0000","last_undegraded":"2024-04-25T10:14:29.711025+0000","last_fullsized":"2024-04-25T10:14:29.711025+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:15.661063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710858+0000","last_change":"2024-04-25T10:14:28.714854+0000","last_active":"2024-04-25T10:14:29.710858+0000","last_peered":"2024-04-25T10:14:29.710858+0000","last_clean":"2024-04-25T10:14:29.710858+0000","last_became_active":"2024-04-25T10:14:28.713819+0000","last_became_peered":"2024-04-25T10:14:28.713819+0000","last_unstale":"2024-04-25T10:14:29.710858+0000","last_undegraded":"2024-04-25T10:14:29.710858+0000","last_fullsized":"2024-04-25T10:14:29.710858+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:36.344770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.691484+0000","last_change":"2024-04-25T10:14:28.689143+0000","last_active":"2024-04-25T10:14:28.691484+0000","last_peered":"2024-04-25T10:14:28.691484+0000","last_clean":"2024-04-25T10:14:28.691484+0000","last_became_active":"2024-04-25T10:14:28.688839+0000","last_became_peered":"2024-04-25T10:14:28.688839+0000","last_unstale":"2024-04-25T10:14:28.691484+0000","last_undegraded":"2024-04-25T10:14:28.691484+0000","last_fullsized":"2024-04-25T10:14:28.691484+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:20.841913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.706961+0000","last_change":"2024-04-25T10:14:28.704358+0000","last_active":"2024-04-25T10:14:28.706961+0000","last_peered":"2024-04-25T10:14:28.706961+0000","last_clean":"2024-04-25T10:14:28.706961+0000","last_became_active":"2024-04-25T10:14:28.703956+0000","last_became_peered":"2024-04-25T10:14:28.703956+0000","last_unstale":"2024-04-25T10:14:28.706961+0000","last_undegraded":"2024-04-25T10:14:28.706961+0000","last_fullsized":"2024-04-25T10:14:28.706961+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:13.846771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705121+0000","last_change":"2024-04-25T10:14:28.730615+0000","last_active":"2024-04-25T10:14:29.705121+0000","last_peered":"2024-04-25T10:14:29.705121+0000","last_clean":"2024-04-25T10:14:29.705121+0000","last_became_active":"2024-04-25T10:14:28.720383+0000","last_became_peered":"2024-04-25T10:14:28.720383+0000","last_unstale":"2024-04-25T10:14:29.705121+0000","last_undegraded":"2024-04-25T10:14:29.705121+0000","last_fullsized":"2024-04-25T10:14:29.705121+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:41.946199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.706114+0000","last_change":"2024-04-25T10:14:28.704098+0000","last_active":"2024-04-25T10:14:28.706114+0000","last_peered":"2024-04-25T10:14:28.706114+0000","last_clean":"2024-04-25T10:14:28.706114+0000","last_became_active":"2024-04-25T10:14:28.703842+0000","last_became_peered":"2024-04-25T10:14:28.703842+0000","last_unstale":"2024-04-25T10:14:28.706114+0000","last_undegraded":"2024-04-25T10:14:28.706114+0000","last_fullsized":"2024-04-25T10:14:28.706114+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:21.110448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.730732+0000","last_change":"2024-04-25T10:14:28.730293+0000","last_active":"2024-04-25T10:14:28.730732+0000","last_peered":"2024-04-25T10:14:28.730732+0000","last_clean":"2024-04-25T10:14:28.730732+0000","last_became_active":"2024-04-25T10:14:28.730056+0000","last_became_peered":"2024-04-25T10:14:28.730056+0000","last_unstale":"2024-04-25T10:14:28.730732+0000","last_undegraded":"2024-04-25T10:14:28.730732+0000","last_fullsized":"2024-04-25T10:14:28.730732+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:15.661063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704901+0000","last_change":"2024-04-25T10:14:28.733718+0000","last_active":"2024-04-25T10:14:29.704901+0000","last_peered":"2024-04-25T10:14:29.704901+0000","last_clean":"2024-04-25T10:14:29.704901+0000","last_became_active":"2024-04-25T10:14:28.732712+0000","last_became_peered":"2024-04-25T10:14:28.732712+0000","last_unstale":"2024-04-25T10:14:29.704901+0000","last_undegraded":"2024-04-25T10:14:29.704901+0000","last_fullsized":"2024-04-25T10:14:29.704901+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:14.015247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.699233+0000","last_change":"2024-04-25T10:14:28.697898+0000","last_active":"2024-04-25T10:14:28.699233+0000","last_peered":"2024-04-25T10:14:28.699233+0000","last_clean":"2024-04-25T10:14:28.699233+0000","last_became_active":"2024-04-25T10:14:28.695278+0000","last_became_peered":"2024-04-25T10:14:28.695278+0000","last_unstale":"2024-04-25T10:14:28.699233+0000","last_undegraded":"2024-04-25T10:14:28.699233+0000","last_fullsized":"2024-04-25T10:14:28.699233+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:19.717882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"142'4","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.719852+0000","last_change":"2024-04-25T10:14:27.653333+0000","last_active":"2024-04-25T10:14:28.719852+0000","last_peered":"2024-04-25T10:14:28.719852+0000","last_clean":"2024-04-25T10:14:28.719852+0000","last_became_active":"2024-04-25T10:14:27.653012+0000","last_became_peered":"2024-04-25T10:14:27.653012+0000","last_unstale":"2024-04-25T10:14:28.719852+0000","last_undegraded":"2024-04-25T10:14:28.719852+0000","last_fullsized":"2024-04-25T10:14:28.719852+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:44.980700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.699165+0000","last_change":"2024-04-25T10:14:28.697817+0000","last_active":"2024-04-25T10:14:28.699165+0000","last_peered":"2024-04-25T10:14:28.699165+0000","last_clean":"2024-04-25T10:14:28.699165+0000","last_became_active":"2024-04-25T10:14:28.695159+0000","last_became_peered":"2024-04-25T10:14:28.695159+0000","last_unstale":"2024-04-25T10:14:28.699165+0000","last_undegraded":"2024-04-25T10:14:28.699165+0000","last_fullsized":"2024-04-25T10:14:28.699165+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:14.015247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.717275+0000","last_change":"2024-04-25T10:14:27.674043+0000","last_active":"2024-04-25T10:14:29.717275+0000","last_peered":"2024-04-25T10:14:29.717275+0000","last_clean":"2024-04-25T10:14:29.717275+0000","last_became_active":"2024-04-25T10:14:27.672434+0000","last_became_peered":"2024-04-25T10:14:27.672434+0000","last_unstale":"2024-04-25T10:14:29.717275+0000","last_undegraded":"2024-04-25T10:14:29.717275+0000","last_fullsized":"2024-04-25T10:14:29.717275+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:03.615957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704851+0000","last_change":"2024-04-25T10:14:28.730968+0000","last_active":"2024-04-25T10:14:29.704851+0000","last_peered":"2024-04-25T10:14:29.704851+0000","last_clean":"2024-04-25T10:14:29.704851+0000","last_became_active":"2024-04-25T10:14:28.730560+0000","last_became_peered":"2024-04-25T10:14:28.730560+0000","last_unstale":"2024-04-25T10:14:29.704851+0000","last_undegraded":"2024-04-25T10:14:29.704851+0000","last_fullsized":"2024-04-25T10:14:29.704851+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:59.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.716996+0000","last_change":"2024-04-25T10:14:27.673638+0000","last_active":"2024-04-25T10:14:29.716996+0000","last_peered":"2024-04-25T10:14:29.716996+0000","last_clean":"2024-04-25T10:14:29.716996+0000","last_became_active":"2024-04-25T10:14:27.672946+0000","last_became_peered":"2024-04-25T10:14:27.672946+0000","last_unstale":"2024-04-25T10:14:29.716996+0000","last_undegraded":"2024-04-25T10:14:29.716996+0000","last_fullsized":"2024-04-25T10:14:29.716996+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:20.239965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.717520+0000","last_change":"2024-04-25T10:14:28.718512+0000","last_active":"2024-04-25T10:14:29.717520+0000","last_peered":"2024-04-25T10:14:29.717520+0000","last_clean":"2024-04-25T10:14:29.717520+0000","last_became_active":"2024-04-25T10:14:28.717781+0000","last_became_peered":"2024-04-25T10:14:28.717781+0000","last_unstale":"2024-04-25T10:14:29.717520+0000","last_undegraded":"2024-04-25T10:14:29.717520+0000","last_fullsized":"2024-04-25T10:14:29.717520+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:22.743612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"142'2","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.735868+0000","last_change":"2024-04-25T10:14:27.688102+0000","last_active":"2024-04-25T10:14:28.735868+0000","last_peered":"2024-04-25T10:14:28.735868+0000","last_clean":"2024-04-25T10:14:28.735868+0000","last_became_active":"2024-04-25T10:14:27.687749+0000","last_became_peered":"2024-04-25T10:14:27.687749+0000","last_unstale":"2024-04-25T10:14:28.735868+0000","last_undegraded":"2024-04-25T10:14:28.735868+0000","last_fullsized":"2024-04-25T10:14:28.735868+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:03.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.700029+0000","last_change":"2024-04-25T10:14:28.706628+0000","last_active":"2024-04-25T10:14:29.700029+0000","last_peered":"2024-04-25T10:14:29.700029+0000","last_clean":"2024-04-25T10:14:29.700029+0000","last_became_active":"2024-04-25T10:14:28.706311+0000","last_became_peered":"2024-04-25T10:14:28.706311+0000","last_unstale":"2024-04-25T10:14:29.700029+0000","last_undegraded":"2024-04-25T10:14:29.700029+0000","last_fullsized":"2024-04-25T10:14:29.700029+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:18.941190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705212+0000","last_change":"2024-04-25T10:14:27.687454+0000","last_active":"2024-04-25T10:14:29.705212+0000","last_peered":"2024-04-25T10:14:29.705212+0000","last_clean":"2024-04-25T10:14:29.705212+0000","last_became_active":"2024-04-25T10:14:27.686708+0000","last_became_peered":"2024-04-25T10:14:27.686708+0000","last_unstale":"2024-04-25T10:14:29.705212+0000","last_undegraded":"2024-04-25T10:14:29.705212+0000","last_fullsized":"2024-04-25T10:14:29.705212+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:35.421123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705651+0000","last_change":"2024-04-25T10:14:28.689632+0000","last_active":"2024-04-25T10:14:29.705651+0000","last_peered":"2024-04-25T10:14:29.705651+0000","last_clean":"2024-04-25T10:14:29.705651+0000","last_became_active":"2024-04-25T10:14:28.688933+0000","last_became_peered":"2024-04-25T10:14:28.688933+0000","last_unstale":"2024-04-25T10:14:29.705651+0000","last_undegraded":"2024-04-25T10:14:29.705651+0000","last_fullsized":"2024-04-25T10:14:29.705651+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:04.773052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"142'1","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.720092+0000","last_change":"2024-04-25T10:14:27.669065+0000","last_active":"2024-04-25T10:14:28.720092+0000","last_peered":"2024-04-25T10:14:28.720092+0000","last_clean":"2024-04-25T10:14:28.720092+0000","last_became_active":"2024-04-25T10:14:27.668777+0000","last_became_peered":"2024-04-25T10:14:27.668777+0000","last_unstale":"2024-04-25T10:14:28.720092+0000","last_undegraded":"2024-04-25T10:14:28.720092+0000","last_fullsized":"2024-04-25T10:14:28.720092+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:40.931888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.713382+0000","last_change":"2024-04-25T10:14:28.694888+0000","last_active":"2024-04-25T10:14:28.713382+0000","last_peered":"2024-04-25T10:14:28.713382+0000","last_clean":"2024-04-25T10:14:28.713382+0000","last_became_active":"2024-04-25T10:14:28.687566+0000","last_became_peered":"2024-04-25T10:14:28.687566+0000","last_unstale":"2024-04-25T10:14:28.713382+0000","last_undegraded":"2024-04-25T10:14:28.713382+0000","last_fullsized":"2024-04-25T10:14:28.713382+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:46.915924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710251+0000","last_change":"2024-04-25T10:14:27.664262+0000","last_active":"2024-04-25T10:14:29.710251+0000","last_peered":"2024-04-25T10:14:29.710251+0000","last_clean":"2024-04-25T10:14:29.710251+0000","last_became_active":"2024-04-25T10:14:27.663602+0000","last_became_peered":"2024-04-25T10:14:27.663602+0000","last_unstale":"2024-04-25T10:14:29.710251+0000","last_undegraded":"2024-04-25T10:14:29.710251+0000","last_fullsized":"2024-04-25T10:14:29.710251+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:58.645137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710396+0000","last_change":"2024-04-25T10:14:28.735780+0000","last_active":"2024-04-25T10:14:29.710396+0000","last_peered":"2024-04-25T10:14:29.710396+0000","last_clean":"2024-04-25T10:14:29.710396+0000","last_became_active":"2024-04-25T10:14:28.734958+0000","last_became_peered":"2024-04-25T10:14:28.734958+0000","last_unstale":"2024-04-25T10:14:29.710396+0000","last_undegraded":"2024-04-25T10:14:29.710396+0000","last_fullsized":"2024-04-25T10:14:29.710396+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:36.116720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.676058+0000","last_change":"2024-04-25T10:14:27.662758+0000","last_active":"2024-04-25T10:14:28.676058+0000","last_peered":"2024-04-25T10:14:28.676058+0000","last_clean":"2024-04-25T10:14:28.676058+0000","last_became_active":"2024-04-25T10:14:27.662472+0000","last_became_peered":"2024-04-25T10:14:27.662472+0000","last_unstale":"2024-04-25T10:14:28.676058+0000","last_undegraded":"2024-04-25T10:14:28.676058+0000","last_fullsized":"2024-04-25T10:14:28.676058+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:18.703571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704261+0000","last_change":"2024-04-25T10:14:28.711268+0000","last_active":"2024-04-25T10:14:29.704261+0000","last_peered":"2024-04-25T10:14:29.704261+0000","last_clean":"2024-04-25T10:14:29.704261+0000","last_became_active":"2024-04-25T10:14:28.709576+0000","last_became_peered":"2024-04-25T10:14:28.709576+0000","last_unstale":"2024-04-25T10:14:29.704261+0000","last_undegraded":"2024-04-25T10:14:29.704261+0000","last_fullsized":"2024-04-25T10:14:29.704261+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:59.659448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.717556+0000","last_change":"2024-04-25T10:14:27.660215+0000","last_active":"2024-04-25T10:14:29.717556+0000","last_peered":"2024-04-25T10:14:29.717556+0000","last_clean":"2024-04-25T10:14:29.717556+0000","last_became_active":"2024-04-25T10:14:27.659448+0000","last_became_peered":"2024-04-25T10:14:27.659448+0000","last_unstale":"2024-04-25T10:14:29.717556+0000","last_undegraded":"2024-04-25T10:14:29.717556+0000","last_fullsized":"2024-04-25T10:14:29.717556+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:52.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.708626+0000","last_change":"2024-04-25T10:14:28.707374+0000","last_active":"2024-04-25T10:14:28.708626+0000","last_peered":"2024-04-25T10:14:28.708626+0000","last_clean":"2024-04-25T10:14:28.708626+0000","last_became_active":"2024-04-25T10:14:28.706874+0000","last_became_peered":"2024-04-25T10:14:28.706874+0000","last_unstale":"2024-04-25T10:14:28.708626+0000","last_undegraded":"2024-04-25T10:14:28.708626+0000","last_fullsized":"2024-04-25T10:14:28.708626+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:36.205090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.711509+0000","last_change":"2024-04-25T10:14:27.667663+0000","last_active":"2024-04-25T10:14:29.711509+0000","last_peered":"2024-04-25T10:14:29.711509+0000","last_clean":"2024-04-25T10:14:29.711509+0000","last_became_active":"2024-04-25T10:14:27.666982+0000","last_became_peered":"2024-04-25T10:14:27.666982+0000","last_unstale":"2024-04-25T10:14:29.711509+0000","last_undegraded":"2024-04-25T10:14:29.711509+0000","last_fullsized":"2024-04-25T10:14:29.711509+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:13.000936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.716335+0000","last_change":"2024-04-25T10:14:28.714480+0000","last_active":"2024-04-25T10:14:28.716335+0000","last_peered":"2024-04-25T10:14:28.716335+0000","last_clean":"2024-04-25T10:14:28.716335+0000","last_became_active":"2024-04-25T10:14:28.714209+0000","last_became_peered":"2024-04-25T10:14:28.714209+0000","last_unstale":"2024-04-25T10:14:28.716335+0000","last_undegraded":"2024-04-25T10:14:28.716335+0000","last_fullsized":"2024-04-25T10:14:28.716335+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:11.494674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"142'3","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.717750+0000","last_change":"2024-04-25T10:14:27.656191+0000","last_active":"2024-04-25T10:14:28.717750+0000","last_peered":"2024-04-25T10:14:28.717750+0000","last_clean":"2024-04-25T10:14:28.717750+0000","last_became_active":"2024-04-25T10:14:27.655900+0000","last_became_peered":"2024-04-25T10:14:27.655900+0000","last_unstale":"2024-04-25T10:14:28.717750+0000","last_undegraded":"2024-04-25T10:14:28.717750+0000","last_fullsized":"2024-04-25T10:14:28.717750+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:08.263455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704655+0000","last_change":"2024-04-25T10:14:28.732317+0000","last_active":"2024-04-25T10:14:29.704655+0000","last_peered":"2024-04-25T10:14:29.704655+0000","last_clean":"2024-04-25T10:14:29.704655+0000","last_became_active":"2024-04-25T10:14:28.731900+0000","last_became_peered":"2024-04-25T10:14:28.731900+0000","last_unstale":"2024-04-25T10:14:29.704655+0000","last_undegraded":"2024-04-25T10:14:29.704655+0000","last_fullsized":"2024-04-25T10:14:29.704655+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:46.915924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"142'2","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.719886+0000","last_change":"2024-04-25T10:14:27.654880+0000","last_active":"2024-04-25T10:14:28.719886+0000","last_peered":"2024-04-25T10:14:28.719886+0000","last_clean":"2024-04-25T10:14:28.719886+0000","last_became_active":"2024-04-25T10:14:27.654414+0000","last_became_peered":"2024-04-25T10:14:27.654414+0000","last_unstale":"2024-04-25T10:14:28.719886+0000","last_undegraded":"2024-04-25T10:14:28.719886+0000","last_fullsized":"2024-04-25T10:14:28.719886+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:11.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.713491+0000","last_change":"2024-04-25T10:14:28.698386+0000","last_active":"2024-04-25T10:14:28.713491+0000","last_peered":"2024-04-25T10:14:28.713491+0000","last_clean":"2024-04-25T10:14:28.713491+0000","last_became_active":"2024-04-25T10:14:28.696419+0000","last_became_peered":"2024-04-25T10:14:28.696419+0000","last_unstale":"2024-04-25T10:14:28.713491+0000","last_undegraded":"2024-04-25T10:14:28.713491+0000","last_fullsized":"2024-04-25T10:14:28.713491+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:10.122313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.706750+0000","last_change":"2024-04-25T10:14:27.685045+0000","last_active":"2024-04-25T10:14:29.706750+0000","last_peered":"2024-04-25T10:14:29.706750+0000","last_clean":"2024-04-25T10:14:29.706750+0000","last_became_active":"2024-04-25T10:14:27.684316+0000","last_became_peered":"2024-04-25T10:14:27.684316+0000","last_unstale":"2024-04-25T10:14:29.706750+0000","last_undegraded":"2024-04-25T10:14:29.706750+0000","last_fullsized":"2024-04-25T10:14:29.706750+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:51.974348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.693449+0000","last_change":"2024-04-25T10:14:28.692061+0000","last_active":"2024-04-25T10:14:28.693449+0000","last_peered":"2024-04-25T10:14:28.693449+0000","last_clean":"2024-04-25T10:14:28.693449+0000","last_became_active":"2024-04-25T10:14:28.691813+0000","last_became_peered":"2024-04-25T10:14:28.691813+0000","last_unstale":"2024-04-25T10:14:28.693449+0000","last_undegraded":"2024-04-25T10:14:28.693449+0000","last_fullsized":"2024-04-25T10:14:28.693449+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:57.109328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705928+0000","last_change":"2024-04-25T10:14:27.670240+0000","last_active":"2024-04-25T10:14:29.705928+0000","last_peered":"2024-04-25T10:14:29.705928+0000","last_clean":"2024-04-25T10:14:29.705928+0000","last_became_active":"2024-04-25T10:14:27.669570+0000","last_became_peered":"2024-04-25T10:14:27.669570+0000","last_unstale":"2024-04-25T10:14:29.705928+0000","last_undegraded":"2024-04-25T10:14:29.705928+0000","last_fullsized":"2024-04-25T10:14:29.705928+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:06.165329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.699255+0000","last_change":"2024-04-25T10:14:28.697944+0000","last_active":"2024-04-25T10:14:28.699255+0000","last_peered":"2024-04-25T10:14:28.699255+0000","last_clean":"2024-04-25T10:14:28.699255+0000","last_became_active":"2024-04-25T10:14:28.695583+0000","last_became_peered":"2024-04-25T10:14:28.695583+0000","last_unstale":"2024-04-25T10:14:28.699255+0000","last_undegraded":"2024-04-25T10:14:28.699255+0000","last_fullsized":"2024-04-25T10:14:28.699255+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:41.946199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"142'8","reported_seq":22,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.709870+0000","last_change":"2024-04-25T10:14:27.670240+0000","last_active":"2024-04-25T10:14:29.709870+0000","last_peered":"2024-04-25T10:14:29.709870+0000","last_clean":"2024-04-25T10:14:29.709870+0000","last_became_active":"2024-04-25T10:14:27.669471+0000","last_became_peered":"2024-04-25T10:14:27.669471+0000","last_unstale":"2024-04-25T10:14:29.709870+0000","last_undegraded":"2024-04-25T10:14:29.709870+0000","last_fullsized":"2024-04-25T10:14:29.709870+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:58.967268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.715612+0000","last_change":"2024-04-25T10:14:28.709296+0000","last_active":"2024-04-25T10:14:29.715612+0000","last_peered":"2024-04-25T10:14:29.715612+0000","last_clean":"2024-04-25T10:14:29.715612+0000","last_became_active":"2024-04-25T10:14:28.708559+0000","last_became_peered":"2024-04-25T10:14:28.708559+0000","last_unstale":"2024-04-25T10:14:29.715612+0000","last_undegraded":"2024-04-25T10:14:29.715612+0000","last_fullsized":"2024-04-25T10:14:29.715612+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:14.262169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"142'1","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.724543+0000","last_change":"2024-04-25T10:14:27.667141+0000","last_active":"2024-04-25T10:14:28.724543+0000","last_peered":"2024-04-25T10:14:28.724543+0000","last_clean":"2024-04-25T10:14:28.724543+0000","last_became_active":"2024-04-25T10:14:27.666848+0000","last_became_peered":"2024-04-25T10:14:27.666848+0000","last_unstale":"2024-04-25T10:14:28.724543+0000","last_undegraded":"2024-04-25T10:14:28.724543+0000","last_fullsized":"2024-04-25T10:14:28.724543+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:56.197132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.716741+0000","last_change":"2024-04-25T10:14:28.703857+0000","last_active":"2024-04-25T10:14:29.716741+0000","last_peered":"2024-04-25T10:14:29.716741+0000","last_clean":"2024-04-25T10:14:29.716741+0000","last_became_active":"2024-04-25T10:14:28.702623+0000","last_became_peered":"2024-04-25T10:14:28.702623+0000","last_unstale":"2024-04-25T10:14:29.716741+0000","last_undegraded":"2024-04-25T10:14:29.716741+0000","last_fullsized":"2024-04-25T10:14:29.716741+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:18.754906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.717384+0000","last_change":"2024-04-25T10:14:27.660304+0000","last_active":"2024-04-25T10:14:29.717384+0000","last_peered":"2024-04-25T10:14:29.717384+0000","last_clean":"2024-04-25T10:14:29.717384+0000","last_became_active":"2024-04-25T10:14:27.659613+0000","last_became_peered":"2024-04-25T10:14:27.659613+0000","last_unstale":"2024-04-25T10:14:29.717384+0000","last_undegraded":"2024-04-25T10:14:29.717384+0000","last_fullsized":"2024-04-25T10:14:29.717384+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:33.859995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.711181+0000","last_change":"2024-04-25T10:14:28.722896+0000","last_active":"2024-04-25T10:14:29.711181+0000","last_peered":"2024-04-25T10:14:29.711181+0000","last_clean":"2024-04-25T10:14:29.711181+0000","last_became_active":"2024-04-25T10:14:28.716643+0000","last_became_peered":"2024-04-25T10:14:28.716643+0000","last_unstale":"2024-04-25T10:14:29.711181+0000","last_undegraded":"2024-04-25T10:14:29.711181+0000","last_fullsized":"2024-04-25T10:14:29.711181+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:09.909912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"142'1","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.721757+0000","last_change":"2024-04-25T10:14:27.669742+0000","last_active":"2024-04-25T10:14:28.721757+0000","last_peered":"2024-04-25T10:14:28.721757+0000","last_clean":"2024-04-25T10:14:28.721757+0000","last_became_active":"2024-04-25T10:14:27.669472+0000","last_became_peered":"2024-04-25T10:14:27.669472+0000","last_unstale":"2024-04-25T10:14:28.721757+0000","last_undegraded":"2024-04-25T10:14:28.721757+0000","last_fullsized":"2024-04-25T10:14:28.721757+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:58.645137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.711259+0000","last_change":"2024-04-25T10:14:28.720299+0000","last_active":"2024-04-25T10:14:29.711259+0000","last_peered":"2024-04-25T10:14:29.711259+0000","last_clean":"2024-04-25T10:14:29.711259+0000","last_became_active":"2024-04-25T10:14:28.719933+0000","last_became_peered":"2024-04-25T10:14:28.719933+0000","last_unstale":"2024-04-25T10:14:29.711259+0000","last_undegraded":"2024-04-25T10:14:29.711259+0000","last_fullsized":"2024-04-25T10:14:29.711259+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.301711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.707020+0000","last_change":"2024-04-25T10:14:27.689022+0000","last_active":"2024-04-25T10:14:29.707020+0000","last_peered":"2024-04-25T10:14:29.707020+0000","last_clean":"2024-04-25T10:14:29.707020+0000","last_became_active":"2024-04-25T10:14:27.688317+0000","last_became_peered":"2024-04-25T10:14:27.688317+0000","last_unstale":"2024-04-25T10:14:29.707020+0000","last_undegraded":"2024-04-25T10:14:29.707020+0000","last_fullsized":"2024-04-25T10:14:29.707020+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:14.068208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.715446+0000","last_change":"2024-04-25T10:14:28.734635+0000","last_active":"2024-04-25T10:14:29.715446+0000","last_peered":"2024-04-25T10:14:29.715446+0000","last_clean":"2024-04-25T10:14:29.715446+0000","last_became_active":"2024-04-25T10:14:28.734312+0000","last_became_peered":"2024-04-25T10:14:28.734312+0000","last_unstale":"2024-04-25T10:14:29.715446+0000","last_undegraded":"2024-04-25T10:14:29.715446+0000","last_fullsized":"2024-04-25T10:14:29.715446+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:57.748394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.699974+0000","last_change":"2024-04-25T10:14:27.687304+0000","last_active":"2024-04-25T10:14:29.699974+0000","last_peered":"2024-04-25T10:14:29.699974+0000","last_clean":"2024-04-25T10:14:29.699974+0000","last_became_active":"2024-04-25T10:14:27.686967+0000","last_became_peered":"2024-04-25T10:14:27.686967+0000","last_unstale":"2024-04-25T10:14:29.699974+0000","last_undegraded":"2024-04-25T10:14:29.699974+0000","last_fullsized":"2024-04-25T10:14:29.699974+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:52.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.720378+0000","last_change":"2024-04-25T10:14:28.719716+0000","last_active":"2024-04-25T10:14:28.720378+0000","last_peered":"2024-04-25T10:14:28.720378+0000","last_clean":"2024-04-25T10:14:28.720378+0000","last_became_active":"2024-04-25T10:14:28.719312+0000","last_became_peered":"2024-04-25T10:14:28.719312+0000","last_unstale":"2024-04-25T10:14:28.720378+0000","last_undegraded":"2024-04-25T10:14:28.720378+0000","last_fullsized":"2024-04-25T10:14:28.720378+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:22.157823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704352+0000","last_change":"2024-04-25T10:14:27.694637+0000","last_active":"2024-04-25T10:14:29.704352+0000","last_peered":"2024-04-25T10:14:29.704352+0000","last_clean":"2024-04-25T10:14:29.704352+0000","last_became_active":"2024-04-25T10:14:27.693879+0000","last_became_peered":"2024-04-25T10:14:27.693879+0000","last_unstale":"2024-04-25T10:14:29.704352+0000","last_undegraded":"2024-04-25T10:14:29.704352+0000","last_fullsized":"2024-04-25T10:14:29.704352+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:18.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.715442+0000","last_change":"2024-04-25T10:14:28.707634+0000","last_active":"2024-04-25T10:14:29.715442+0000","last_peered":"2024-04-25T10:14:29.715442+0000","last_clean":"2024-04-25T10:14:29.715442+0000","last_became_active":"2024-04-25T10:14:28.706813+0000","last_became_peered":"2024-04-25T10:14:28.706813+0000","last_unstale":"2024-04-25T10:14:29.715442+0000","last_undegraded":"2024-04-25T10:14:29.715442+0000","last_fullsized":"2024-04-25T10:14:29.715442+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:55.950752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.699894+0000","last_change":"2024-04-25T10:14:27.687256+0000","last_active":"2024-04-25T10:14:29.699894+0000","last_peered":"2024-04-25T10:14:29.699894+0000","last_clean":"2024-04-25T10:14:29.699894+0000","last_became_active":"2024-04-25T10:14:27.686763+0000","last_became_peered":"2024-04-25T10:14:27.686763+0000","last_unstale":"2024-04-25T10:14:29.699894+0000","last_undegraded":"2024-04-25T10:14:29.699894+0000","last_fullsized":"2024-04-25T10:14:29.699894+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:33.859995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.699284+0000","last_change":"2024-04-25T10:14:28.697823+0000","last_active":"2024-04-25T10:14:28.699284+0000","last_peered":"2024-04-25T10:14:28.699284+0000","last_clean":"2024-04-25T10:14:28.699284+0000","last_became_active":"2024-04-25T10:14:28.695402+0000","last_became_peered":"2024-04-25T10:14:28.695402+0000","last_unstale":"2024-04-25T10:14:28.699284+0000","last_undegraded":"2024-04-25T10:14:28.699284+0000","last_fullsized":"2024-04-25T10:14:28.699284+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:12.681021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705070+0000","last_change":"2024-04-25T10:14:27.691808+0000","last_active":"2024-04-25T10:14:29.705070+0000","last_peered":"2024-04-25T10:14:29.705070+0000","last_clean":"2024-04-25T10:14:29.705070+0000","last_became_active":"2024-04-25T10:14:27.690407+0000","last_became_peered":"2024-04-25T10:14:27.690407+0000","last_unstale":"2024-04-25T10:14:29.705070+0000","last_undegraded":"2024-04-25T10:14:29.705070+0000","last_fullsized":"2024-04-25T10:14:29.705070+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:01.572491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.712767+0000","last_change":"2024-04-25T10:14:28.710904+0000","last_active":"2024-04-25T10:14:28.712767+0000","last_peered":"2024-04-25T10:14:28.712767+0000","last_clean":"2024-04-25T10:14:28.712767+0000","last_became_active":"2024-04-25T10:14:28.710418+0000","last_became_peered":"2024-04-25T10:14:28.710418+0000","last_unstale":"2024-04-25T10:14:28.712767+0000","last_undegraded":"2024-04-25T10:14:28.712767+0000","last_fullsized":"2024-04-25T10:14:28.712767+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:36.116720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.707746+0000","last_change":"2024-04-25T10:14:27.684947+0000","last_active":"2024-04-25T10:14:29.707746+0000","last_peered":"2024-04-25T10:14:29.707746+0000","last_clean":"2024-04-25T10:14:29.707746+0000","last_became_active":"2024-04-25T10:14:27.684161+0000","last_became_peered":"2024-04-25T10:14:27.684161+0000","last_unstale":"2024-04-25T10:14:29.707746+0000","last_undegraded":"2024-04-25T10:14:29.707746+0000","last_fullsized":"2024-04-25T10:14:29.707746+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:14.323948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.707319+0000","last_change":"2024-04-25T10:14:28.701381+0000","last_active":"2024-04-25T10:14:29.707319+0000","last_peered":"2024-04-25T10:14:29.707319+0000","last_clean":"2024-04-25T10:14:29.707319+0000","last_became_active":"2024-04-25T10:14:28.700480+0000","last_became_peered":"2024-04-25T10:14:28.700480+0000","last_unstale":"2024-04-25T10:14:29.707319+0000","last_undegraded":"2024-04-25T10:14:29.707319+0000","last_fullsized":"2024-04-25T10:14:29.707319+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:14.015247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"143'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.714333+0000","last_change":"2024-04-25T10:14:27.670622+0000","last_active":"2024-04-25T10:14:29.714333+0000","last_peered":"2024-04-25T10:14:29.714333+0000","last_clean":"2024-04-25T10:14:29.714333+0000","last_became_active":"2024-04-25T10:14:27.669279+0000","last_became_peered":"2024-04-25T10:14:27.669279+0000","last_unstale":"2024-04-25T10:14:29.714333+0000","last_undegraded":"2024-04-25T10:14:29.714333+0000","last_fullsized":"2024-04-25T10:14:29.714333+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:18.703571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.711385+0000","last_change":"2024-04-25T10:14:28.734963+0000","last_active":"2024-04-25T10:14:29.711385+0000","last_peered":"2024-04-25T10:14:29.711385+0000","last_clean":"2024-04-25T10:14:29.711385+0000","last_became_active":"2024-04-25T10:14:28.734583+0000","last_became_peered":"2024-04-25T10:14:28.734583+0000","last_unstale":"2024-04-25T10:14:29.711385+0000","last_undegraded":"2024-04-25T10:14:29.711385+0000","last_fullsized":"2024-04-25T10:14:29.711385+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:43.056854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.682766+0000","last_change":"2024-04-25T10:14:27.681632+0000","last_active":"2024-04-25T10:14:28.682766+0000","last_peered":"2024-04-25T10:14:28.682766+0000","last_clean":"2024-04-25T10:14:28.682766+0000","last_became_active":"2024-04-25T10:14:27.681255+0000","last_became_peered":"2024-04-25T10:14:27.681255+0000","last_unstale":"2024-04-25T10:14:28.682766+0000","last_undegraded":"2024-04-25T10:14:28.682766+0000","last_fullsized":"2024-04-25T10:14:28.682766+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:43.911328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.692091+0000","last_change":"2024-04-25T10:14:28.689844+0000","last_active":"2024-04-25T10:14:28.692091+0000","last_peered":"2024-04-25T10:14:28.692091+0000","last_clean":"2024-04-25T10:14:28.692091+0000","last_became_active":"2024-04-25T10:14:28.689559+0000","last_became_peered":"2024-04-25T10:14:28.689559+0000","last_unstale":"2024-04-25T10:14:28.692091+0000","last_undegraded":"2024-04-25T10:14:28.692091+0000","last_fullsized":"2024-04-25T10:14:28.692091+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:39.841652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.677668+0000","last_change":"2024-04-25T10:14:27.691854+0000","last_active":"2024-04-25T10:14:28.677668+0000","last_peered":"2024-04-25T10:14:28.677668+0000","last_clean":"2024-04-25T10:14:28.677668+0000","last_became_active":"2024-04-25T10:14:27.691496+0000","last_became_peered":"2024-04-25T10:14:27.691496+0000","last_unstale":"2024-04-25T10:14:28.677668+0000","last_undegraded":"2024-04-25T10:14:28.677668+0000","last_fullsized":"2024-04-25T10:14:28.677668+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:09.108002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705124+0000","last_change":"2024-04-25T10:14:28.722151+0000","last_active":"2024-04-25T10:14:29.705124+0000","last_peered":"2024-04-25T10:14:29.705124+0000","last_clean":"2024-04-25T10:14:29.705124+0000","last_became_active":"2024-04-25T10:14:28.720912+0000","last_became_peered":"2024-04-25T10:14:28.720912+0000","last_unstale":"2024-04-25T10:14:29.705124+0000","last_undegraded":"2024-04-25T10:14:29.705124+0000","last_fullsized":"2024-04-25T10:14:29.705124+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:12.681021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710392+0000","last_change":"2024-04-25T10:14:27.672590+0000","last_active":"2024-04-25T10:14:29.710392+0000","last_peered":"2024-04-25T10:14:29.710392+0000","last_clean":"2024-04-25T10:14:29.710392+0000","last_became_active":"2024-04-25T10:14:27.671920+0000","last_became_peered":"2024-04-25T10:14:27.671920+0000","last_unstale":"2024-04-25T10:14:29.710392+0000","last_undegraded":"2024-04-25T10:14:29.710392+0000","last_fullsized":"2024-04-25T10:14:29.710392+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:45.901612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705800+0000","last_change":"2024-04-25T10:14:28.707811+0000","last_active":"2024-04-25T10:14:29.705800+0000","last_peered":"2024-04-25T10:14:29.705800+0000","last_clean":"2024-04-25T10:14:29.705800+0000","last_became_active":"2024-04-25T10:14:28.702188+0000","last_became_peered":"2024-04-25T10:14:28.702188+0000","last_unstale":"2024-04-25T10:14:29.705800+0000","last_undegraded":"2024-04-25T10:14:29.705800+0000","last_fullsized":"2024-04-25T10:14:29.705800+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:59.659448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.684964+0000","last_change":"2024-04-25T10:14:27.687468+0000","last_active":"2024-04-25T10:14:28.684964+0000","last_peered":"2024-04-25T10:14:28.684964+0000","last_clean":"2024-04-25T10:14:28.684964+0000","last_became_active":"2024-04-25T10:14:27.687077+0000","last_became_peered":"2024-04-25T10:14:27.687077+0000","last_unstale":"2024-04-25T10:14:28.684964+0000","last_undegraded":"2024-04-25T10:14:28.684964+0000","last_fullsized":"2024-04-25T10:14:28.684964+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:17.280053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.714413+0000","last_change":"2024-04-25T10:14:28.712963+0000","last_active":"2024-04-25T10:14:28.714413+0000","last_peered":"2024-04-25T10:14:28.714413+0000","last_clean":"2024-04-25T10:14:28.714413+0000","last_became_active":"2024-04-25T10:14:28.699156+0000","last_became_peered":"2024-04-25T10:14:28.699156+0000","last_unstale":"2024-04-25T10:14:28.714413+0000","last_undegraded":"2024-04-25T10:14:28.714413+0000","last_fullsized":"2024-04-25T10:14:28.714413+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:20.841913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710513+0000","last_change":"2024-04-25T10:14:27.667498+0000","last_active":"2024-04-25T10:14:29.710513+0000","last_peered":"2024-04-25T10:14:29.710513+0000","last_clean":"2024-04-25T10:14:29.710513+0000","last_became_active":"2024-04-25T10:14:27.666488+0000","last_became_peered":"2024-04-25T10:14:27.666488+0000","last_unstale":"2024-04-25T10:14:29.710513+0000","last_undegraded":"2024-04-25T10:14:29.710513+0000","last_fullsized":"2024-04-25T10:14:29.710513+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:43.911328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704433+0000","last_change":"2024-04-25T10:14:28.728969+0000","last_active":"2024-04-25T10:14:29.704433+0000","last_peered":"2024-04-25T10:14:29.704433+0000","last_clean":"2024-04-25T10:14:29.704433+0000","last_became_active":"2024-04-25T10:14:28.720777+0000","last_became_peered":"2024-04-25T10:14:28.720777+0000","last_unstale":"2024-04-25T10:14:29.704433+0000","last_undegraded":"2024-04-25T10:14:29.704433+0000","last_fullsized":"2024-04-25T10:14:29.704433+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:44.925639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.715330+0000","last_change":"2024-04-25T10:14:27.660245+0000","last_active":"2024-04-25T10:14:29.715330+0000","last_peered":"2024-04-25T10:14:29.715330+0000","last_clean":"2024-04-25T10:14:29.715330+0000","last_became_active":"2024-04-25T10:14:27.659553+0000","last_became_peered":"2024-04-25T10:14:27.659553+0000","last_unstale":"2024-04-25T10:14:29.715330+0000","last_undegraded":"2024-04-25T10:14:29.715330+0000","last_fullsized":"2024-04-25T10:14:29.715330+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:31.269675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704985+0000","last_change":"2024-04-25T10:14:28.732501+0000","last_active":"2024-04-25T10:14:29.704985+0000","last_peered":"2024-04-25T10:14:29.704985+0000","last_clean":"2024-04-25T10:14:29.704985+0000","last_became_active":"2024-04-25T10:14:28.732062+0000","last_became_peered":"2024-04-25T10:14:28.732062+0000","last_unstale":"2024-04-25T10:14:29.704985+0000","last_undegraded":"2024-04-25T10:14:29.704985+0000","last_fullsized":"2024-04-25T10:14:29.704985+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:19.717882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"142'6","reported_seq":20,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.700134+0000","last_change":"2024-04-25T10:14:27.687376+0000","last_active":"2024-04-25T10:14:29.700134+0000","last_peered":"2024-04-25T10:14:29.700134+0000","last_clean":"2024-04-25T10:14:29.700134+0000","last_became_active":"2024-04-25T10:14:27.687042+0000","last_became_peered":"2024-04-25T10:14:27.687042+0000","last_unstale":"2024-04-25T10:14:29.700134+0000","last_undegraded":"2024-04-25T10:14:29.700134+0000","last_fullsized":"2024-04-25T10:14:29.700134+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:03.615957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":402,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.715488+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:14:29.715488+0000","last_peered":"2024-04-25T10:14:29.715488+0000","last_clean":"2024-04-25T10:14:29.715488+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:14:29.715488+0000","last_undegraded":"2024-04-25T10:14:29.715488+0000","last_fullsized":"2024-04-25T10:14:29.715488+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69608,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302232,"statfs":{"total":96636764160,"available":96565485568,"internally_reserved":0,"allocated":2379776,"data_stored":1424225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50992,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320848,"statfs":{"total":96636764160,"available":96584548352,"internally_reserved":0,"allocated":1798144,"data_stored":840597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575006,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59296,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312544,"statfs":{"total":96636764160,"available":96576045056,"internally_reserved":0,"allocated":1781760,"data_stored":833835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73196,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298644,"statfs":{"total":96636764160,"available":96561811456,"internally_reserved":0,"allocated":1794048,"data_stored":835447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575006,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77344,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294496,"statfs":{"total":96636764160,"available":96557563904,"internally_reserved":0,"allocated":1789952,"data_stored":834451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50984,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320856,"statfs":{"total":96636764160,"available":96584556544,"internally_reserved":0,"allocated":1789952,"data_stored":833947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46816,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325024,"statfs":{"total":96636764160,"available":96588824576,"internally_reserved":0,"allocated":1785856,"data_stored":833905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575007,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73784,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298056,"statfs":{"total":96636764160,"available":96561209344,"internally_reserved":0,"allocated":2400256,"data_stored":1426521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:14:31.716 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:14:31.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.871+0000 7f1f1a664640 1 Processor -- start 2024-04-25T10:14:31.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f1a664640 1 -- start start 2024-04-25T10:14:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f1a664640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140b1f70 0x7f1f1406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f1a664640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1f140ce330 0x7f1f140ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f1a664640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f140c61d0 con 0x7f1f1415c9a0 2024-04-25T10:14:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f1a664640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f140cece0 con 0x7f1f140ce330 2024-04-25T10:14:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f1a664640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140c64b0 con 0x7f1f140b1f70 2024-04-25T10:14:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f18c01640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1f140ce330 0x7f1f140ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f13fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1f1415c9a0 0x7f1f140c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48620/0 (socket says 172.21.15.133:48620) 2024-04-25T10:14:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f13fff640 1 -- 172.21.15.133:0/1695101868 learned_addr learned my addr 172.21.15.133:0/1695101868 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f137fe640 1 --2- 172.21.15.133:0/1695101868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140b1f70 0x7f1f1406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f12ffd640 1 -- 172.21.15.133:0/1695101868 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543043699 0 0) 0x7f1f140c61d0 con 0x7f1f1415c9a0 2024-04-25T10:14:31.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f12ffd640 1 -- 172.21.15.133:0/1695101868 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1efc003530 con 0x7f1f1415c9a0 2024-04-25T10:14:31.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f18c01640 1 -- 172.21.15.133:0/1695101868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140b1f70 msgr2=0x7f1f1406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f18c01640 1 --2- 172.21.15.133:0/1695101868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140b1f70 0x7f1f1406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f18c01640 1 -- 172.21.15.133:0/1695101868 >> v1:172.21.15.133:6789/0 conn(0x7f1f1415c9a0 legacy=0x7f1f140c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.875+0000 7f1f18c01640 1 -- 172.21.15.133:0/1695101868 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1406d4f0 con 0x7f1f140ce330 2024-04-25T10:14:31.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f18c01640 1 --2- 172.21.15.133:0/1695101868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f140ce330 0x7f1f140ce7a0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009d90 tx=0x7f1f0c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c9ff490c09f147 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f12ffd640 1 -- 172.21.15.133:0/1695101868 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0b2070 con 0x7f1f140ce330 2024-04-25T10:14:31.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f12ffd640 1 -- 172.21.15.133:0/1695101868 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c0a5910 con 0x7f1f140ce330 2024-04-25T10:14:31.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f12ffd640 1 -- 172.21.15.133:0/1695101868 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0a5ce0 con 0x7f1f140ce330 2024-04-25T10:14:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695101868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f140ce330 msgr2=0x7f1f140ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 --2- 172.21.15.133:0/1695101868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f140ce330 0x7f1f140ce7a0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009d90 tx=0x7f1f0c01c940 comp rx=0 tx=0).stop 2024-04-25T10:14:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695101868 shutdown_connections 2024-04-25T10:14:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 --2- 172.21.15.133:0/1695101868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f140ce330 0x7f1f140ce7a0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 --2- 172.21.15.133:0/1695101868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140b1f70 0x7f1f1406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695101868 >> 172.21.15.133:0/1695101868 conn(0x7f1f14067000 msgr2=0x7f1f1406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695101868 shutdown_connections 2024-04-25T10:14:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695101868 wait complete. 2024-04-25T10:14:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 Processor -- start 2024-04-25T10:14:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 -- start start 2024-04-25T10:14:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140b1f70 0x7f1f1414a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f140ce330 0x7f1f14146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f1415c9a0 0x7f1f14152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1415dad0 con 0x7f1f140ce330 2024-04-25T10:14:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1415d950 con 0x7f1f1415c9a0 2024-04-25T10:14:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1415dc50 con 0x7f1f140b1f70 2024-04-25T10:14:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f140ce330 0x7f1f14146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f140ce330 0x7f1f14146b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37398/0 (socket says 172.21.15.133:37398) 2024-04-25T10:14:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f137fe640 1 -- 172.21.15.133:0/1695358572 learned_addr learned my addr 172.21.15.133:0/1695358572 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f13fff640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140b1f70 0x7f1f1414a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f137fe640 1 -- 172.21.15.133:0/1695358572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140b1f70 msgr2=0x7f1f1414a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f137fe640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140b1f70 0x7f1f1414a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f18c01640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f1415c9a0 0x7f1f14152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f137fe640 1 -- 172.21.15.133:0/1695358572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f1415c9a0 msgr2=0x7f1f14152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f137fe640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f1415c9a0 0x7f1f14152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f137fe640 1 -- 172.21.15.133:0/1695358572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f14147040 con 0x7f1f140ce330 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f13fff640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140b1f70 0x7f1f1414a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f137fe640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f140ce330 0x7f1f14146b00 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800d5f0 tx=0x7f1f0800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f10ff9640 1 -- 172.21.15.133:0/1695358572 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080125f0 con 0x7f1f140ce330 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f10ff9640 1 -- 172.21.15.133:0/1695358572 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f08019070 con 0x7f1f140ce330 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695358572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f141532e0 con 0x7f1f140ce330 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f10ff9640 1 -- 172.21.15.133:0/1695358572 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080129b0 con 0x7f1f140ce330 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.879+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695358572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f14148ff0 con 0x7f1f140ce330 2024-04-25T10:14:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.883+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695358572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f14068ef0 con 0x7f1f140ce330 2024-04-25T10:14:31.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.883+0000 7f1f10ff9640 1 -- 172.21.15.133:0/1695358572 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f08012b50 con 0x7f1f140ce330 2024-04-25T10:14:31.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.883+0000 7f1f10ff9640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f00066f90 0x7f1f00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.883+0000 7f1f10ff9640 1 -- 172.21.15.133:0/1695358572 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f08085610 con 0x7f1f140ce330 2024-04-25T10:14:31.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.883+0000 7f1f13fff640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f00066f90 0x7f1f00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.883+0000 7f1f13fff640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f00066f90 0x7f1f00069450 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1f040011f0 tx=0x7f1f04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:31.887+0000 7f1f10ff9640 1 -- 172.21.15.133:0/1695358572 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f080563f0 con 0x7f1f140ce330 2024-04-25T10:14:32.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.043+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695358572 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f140ce8b0 con 0x7f1f00066f90 2024-04-25T10:14:32.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.059+0000 7f1f10ff9640 1 -- 172.21.15.133:0/1695358572 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252453 (secure 0 0 0) 0x7f1f140ce8b0 con 0x7f1f00066f90 2024-04-25T10:14:32.064 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695358572 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f00066f90 msgr2=0x7f1f00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f00066f90 0x7f1f00069450 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1f040011f0 tx=0x7f1f04009040 comp rx=0 tx=0).stop 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695358572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f140ce330 msgr2=0x7f1f14146b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f140ce330 0x7f1f14146b00 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800d5f0 tx=0x7f1f0800dab0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695358572 shutdown_connections 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f1415c9a0 0x7f1f14152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f00066f90 0x7f1f00069450 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f140ce330 0x7f1f14146b00 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 --2- 172.21.15.133:0/1695358572 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140b1f70 0x7f1f1414a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695358572 >> 172.21.15.133:0/1695358572 conn(0x7f1f14067000 msgr2=0x7f1f140c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695358572 shutdown_connections 2024-04-25T10:14:32.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.063+0000 7f1f1a664640 1 -- 172.21.15.133:0/1695358572 wait complete. 2024-04-25T10:14:32.084 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-04-25T10:14:31.949721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":502020,"kb_used_data":15156,"kb_used_omap":12,"kb_used_meta":486707,"kb_avail":754472700,"statfs":{"total":773094113280,"available":772580044800,"internally_reserved":0,"allocated":15519744,"data_stored":7862928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":498388529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.312595"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710816+0000","last_change":"2024-04-25T10:14:28.735551+0000","last_active":"2024-04-25T10:14:29.710816+0000","last_peered":"2024-04-25T10:14:29.710816+0000","last_clean":"2024-04-25T10:14:29.710816+0000","last_became_active":"2024-04-25T10:14:28.735031+0000","last_became_peered":"2024-04-25T10:14:28.735031+0000","last_unstale":"2024-04-25T10:14:29.710816+0000","last_undegraded":"2024-04-25T10:14:29.710816+0000","last_fullsized":"2024-04-25T10:14:29.710816+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:44.556550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.698117+0000","last_change":"2024-04-25T10:14:28.695866+0000","last_active":"2024-04-25T10:14:28.698117+0000","last_peered":"2024-04-25T10:14:28.698117+0000","last_clean":"2024-04-25T10:14:28.698117+0000","last_became_active":"2024-04-25T10:14:28.695235+0000","last_became_peered":"2024-04-25T10:14:28.695235+0000","last_unstale":"2024-04-25T10:14:28.698117+0000","last_undegraded":"2024-04-25T10:14:28.698117+0000","last_fullsized":"2024-04-25T10:14:28.698117+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:59.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.699303+0000","last_change":"2024-04-25T10:14:28.697833+0000","last_active":"2024-04-25T10:14:28.699303+0000","last_peered":"2024-04-25T10:14:28.699303+0000","last_clean":"2024-04-25T10:14:28.699303+0000","last_became_active":"2024-04-25T10:14:28.695386+0000","last_became_peered":"2024-04-25T10:14:28.695386+0000","last_unstale":"2024-04-25T10:14:28.699303+0000","last_undegraded":"2024-04-25T10:14:28.699303+0000","last_fullsized":"2024-04-25T10:14:28.699303+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:45.995012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.706511+0000","last_change":"2024-04-25T10:14:28.711671+0000","last_active":"2024-04-25T10:14:29.706511+0000","last_peered":"2024-04-25T10:14:29.706511+0000","last_clean":"2024-04-25T10:14:29.706511+0000","last_became_active":"2024-04-25T10:14:28.710920+0000","last_became_peered":"2024-04-25T10:14:28.710920+0000","last_unstale":"2024-04-25T10:14:29.706511+0000","last_undegraded":"2024-04-25T10:14:29.706511+0000","last_fullsized":"2024-04-25T10:14:29.706511+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:46.915924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.719956+0000","last_change":"2024-04-25T10:14:28.713697+0000","last_active":"2024-04-25T10:14:28.719956+0000","last_peered":"2024-04-25T10:14:28.719956+0000","last_clean":"2024-04-25T10:14:28.719956+0000","last_became_active":"2024-04-25T10:14:28.713268+0000","last_became_peered":"2024-04-25T10:14:28.713268+0000","last_unstale":"2024-04-25T10:14:28.719956+0000","last_undegraded":"2024-04-25T10:14:28.719956+0000","last_fullsized":"2024-04-25T10:14:28.719956+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:36.344770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.719761+0000","last_change":"2024-04-25T10:14:28.712694+0000","last_active":"2024-04-25T10:14:28.719761+0000","last_peered":"2024-04-25T10:14:28.719761+0000","last_clean":"2024-04-25T10:14:28.719761+0000","last_became_active":"2024-04-25T10:14:28.711745+0000","last_became_peered":"2024-04-25T10:14:28.711745+0000","last_unstale":"2024-04-25T10:14:28.719761+0000","last_undegraded":"2024-04-25T10:14:28.719761+0000","last_fullsized":"2024-04-25T10:14:28.719761+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:44.556550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710333+0000","last_change":"2024-04-25T10:14:28.728500+0000","last_active":"2024-04-25T10:14:29.710333+0000","last_peered":"2024-04-25T10:14:29.710333+0000","last_clean":"2024-04-25T10:14:29.710333+0000","last_became_active":"2024-04-25T10:14:28.728027+0000","last_became_peered":"2024-04-25T10:14:28.728027+0000","last_unstale":"2024-04-25T10:14:29.710333+0000","last_undegraded":"2024-04-25T10:14:29.710333+0000","last_fullsized":"2024-04-25T10:14:29.710333+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:48.641809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.703879+0000","last_change":"2024-04-25T10:14:28.736090+0000","last_active":"2024-04-25T10:14:29.703879+0000","last_peered":"2024-04-25T10:14:29.703879+0000","last_clean":"2024-04-25T10:14:29.703879+0000","last_became_active":"2024-04-25T10:14:28.733147+0000","last_became_peered":"2024-04-25T10:14:28.733147+0000","last_unstale":"2024-04-25T10:14:29.703879+0000","last_undegraded":"2024-04-25T10:14:29.703879+0000","last_fullsized":"2024-04-25T10:14:29.703879+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.758052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.692062+0000","last_change":"2024-04-25T10:14:28.687342+0000","last_active":"2024-04-25T10:14:28.692062+0000","last_peered":"2024-04-25T10:14:28.692062+0000","last_clean":"2024-04-25T10:14:28.692062+0000","last_became_active":"2024-04-25T10:14:28.684251+0000","last_became_peered":"2024-04-25T10:14:28.684251+0000","last_unstale":"2024-04-25T10:14:28.692062+0000","last_undegraded":"2024-04-25T10:14:28.692062+0000","last_fullsized":"2024-04-25T10:14:28.692062+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:06.877297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.691426+0000","last_change":"2024-04-25T10:14:28.689473+0000","last_active":"2024-04-25T10:14:28.691426+0000","last_peered":"2024-04-25T10:14:28.691426+0000","last_clean":"2024-04-25T10:14:28.691426+0000","last_became_active":"2024-04-25T10:14:28.689113+0000","last_became_peered":"2024-04-25T10:14:28.689113+0000","last_unstale":"2024-04-25T10:14:28.691426+0000","last_undegraded":"2024-04-25T10:14:28.691426+0000","last_fullsized":"2024-04-25T10:14:28.691426+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:22.157823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.726010+0000","last_change":"2024-04-25T10:14:28.725325+0000","last_active":"2024-04-25T10:14:28.726010+0000","last_peered":"2024-04-25T10:14:28.726010+0000","last_clean":"2024-04-25T10:14:28.726010+0000","last_became_active":"2024-04-25T10:14:28.725124+0000","last_became_peered":"2024-04-25T10:14:28.725124+0000","last_unstale":"2024-04-25T10:14:28.726010+0000","last_undegraded":"2024-04-25T10:14:28.726010+0000","last_fullsized":"2024-04-25T10:14:28.726010+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:16.574486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.706917+0000","last_change":"2024-04-25T10:14:28.701620+0000","last_active":"2024-04-25T10:14:29.706917+0000","last_peered":"2024-04-25T10:14:29.706917+0000","last_clean":"2024-04-25T10:14:29.706917+0000","last_became_active":"2024-04-25T10:14:28.700878+0000","last_became_peered":"2024-04-25T10:14:28.700878+0000","last_unstale":"2024-04-25T10:14:29.706917+0000","last_undegraded":"2024-04-25T10:14:29.706917+0000","last_fullsized":"2024-04-25T10:14:29.706917+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:59.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.693496+0000","last_change":"2024-04-25T10:14:28.692680+0000","last_active":"2024-04-25T10:14:28.693496+0000","last_peered":"2024-04-25T10:14:28.693496+0000","last_clean":"2024-04-25T10:14:28.693496+0000","last_became_active":"2024-04-25T10:14:28.692327+0000","last_became_peered":"2024-04-25T10:14:28.692327+0000","last_unstale":"2024-04-25T10:14:28.693496+0000","last_undegraded":"2024-04-25T10:14:28.693496+0000","last_fullsized":"2024-04-25T10:14:28.693496+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:48.641809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.706307+0000","last_change":"2024-04-25T10:14:28.702406+0000","last_active":"2024-04-25T10:14:29.706307+0000","last_peered":"2024-04-25T10:14:29.706307+0000","last_clean":"2024-04-25T10:14:29.706307+0000","last_became_active":"2024-04-25T10:14:28.701610+0000","last_became_peered":"2024-04-25T10:14:28.701610+0000","last_unstale":"2024-04-25T10:14:29.706307+0000","last_undegraded":"2024-04-25T10:14:29.706307+0000","last_fullsized":"2024-04-25T10:14:29.706307+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:44.925639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.713520+0000","last_change":"2024-04-25T10:14:28.698707+0000","last_active":"2024-04-25T10:14:28.713520+0000","last_peered":"2024-04-25T10:14:28.713520+0000","last_clean":"2024-04-25T10:14:28.713520+0000","last_became_active":"2024-04-25T10:14:28.697985+0000","last_became_peered":"2024-04-25T10:14:28.697985+0000","last_unstale":"2024-04-25T10:14:28.713520+0000","last_undegraded":"2024-04-25T10:14:28.713520+0000","last_fullsized":"2024-04-25T10:14:28.713520+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:06.877297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.734273+0000","last_change":"2024-04-25T10:14:28.733839+0000","last_active":"2024-04-25T10:14:28.734273+0000","last_peered":"2024-04-25T10:14:28.734273+0000","last_clean":"2024-04-25T10:14:28.734273+0000","last_became_active":"2024-04-25T10:14:28.733589+0000","last_became_peered":"2024-04-25T10:14:28.733589+0000","last_unstale":"2024-04-25T10:14:28.734273+0000","last_undegraded":"2024-04-25T10:14:28.734273+0000","last_fullsized":"2024-04-25T10:14:28.734273+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:09.909912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.707887+0000","last_change":"2024-04-25T10:14:28.706734+0000","last_active":"2024-04-25T10:14:29.707887+0000","last_peered":"2024-04-25T10:14:29.707887+0000","last_clean":"2024-04-25T10:14:29.707887+0000","last_became_active":"2024-04-25T10:14:28.700630+0000","last_became_peered":"2024-04-25T10:14:28.700630+0000","last_unstale":"2024-04-25T10:14:29.707887+0000","last_undegraded":"2024-04-25T10:14:29.707887+0000","last_fullsized":"2024-04-25T10:14:29.707887+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:19.717882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.716463+0000","last_change":"2024-04-25T10:14:28.715825+0000","last_active":"2024-04-25T10:14:28.716463+0000","last_peered":"2024-04-25T10:14:28.716463+0000","last_clean":"2024-04-25T10:14:28.716463+0000","last_became_active":"2024-04-25T10:14:28.714478+0000","last_became_peered":"2024-04-25T10:14:28.714478+0000","last_unstale":"2024-04-25T10:14:28.716463+0000","last_undegraded":"2024-04-25T10:14:28.716463+0000","last_fullsized":"2024-04-25T10:14:28.716463+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:06.403336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.716385+0000","last_change":"2024-04-25T10:14:28.704667+0000","last_active":"2024-04-25T10:14:29.716385+0000","last_peered":"2024-04-25T10:14:29.716385+0000","last_clean":"2024-04-25T10:14:29.716385+0000","last_became_active":"2024-04-25T10:14:28.703614+0000","last_became_peered":"2024-04-25T10:14:28.703614+0000","last_unstale":"2024-04-25T10:14:29.716385+0000","last_undegraded":"2024-04-25T10:14:29.716385+0000","last_fullsized":"2024-04-25T10:14:29.716385+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:25.424823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.700240+0000","last_change":"2024-04-25T10:14:28.700047+0000","last_active":"2024-04-25T10:14:29.700240+0000","last_peered":"2024-04-25T10:14:29.700240+0000","last_clean":"2024-04-25T10:14:29.700240+0000","last_became_active":"2024-04-25T10:14:28.699718+0000","last_became_peered":"2024-04-25T10:14:28.699718+0000","last_unstale":"2024-04-25T10:14:29.700240+0000","last_undegraded":"2024-04-25T10:14:29.700240+0000","last_fullsized":"2024-04-25T10:14:29.700240+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:06.300000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.693447+0000","last_change":"2024-04-25T10:14:28.691844+0000","last_active":"2024-04-25T10:14:28.693447+0000","last_peered":"2024-04-25T10:14:28.693447+0000","last_clean":"2024-04-25T10:14:28.693447+0000","last_became_active":"2024-04-25T10:14:28.691375+0000","last_became_peered":"2024-04-25T10:14:28.691375+0000","last_unstale":"2024-04-25T10:14:28.693447+0000","last_undegraded":"2024-04-25T10:14:28.693447+0000","last_fullsized":"2024-04-25T10:14:28.693447+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:44.628062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710269+0000","last_change":"2024-04-25T10:14:28.727722+0000","last_active":"2024-04-25T10:14:29.710269+0000","last_peered":"2024-04-25T10:14:29.710269+0000","last_clean":"2024-04-25T10:14:29.710269+0000","last_became_active":"2024-04-25T10:14:28.727252+0000","last_became_peered":"2024-04-25T10:14:28.727252+0000","last_unstale":"2024-04-25T10:14:29.710269+0000","last_undegraded":"2024-04-25T10:14:29.710269+0000","last_fullsized":"2024-04-25T10:14:29.710269+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:57.109328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.703941+0000","last_change":"2024-04-25T10:14:28.721605+0000","last_active":"2024-04-25T10:14:29.703941+0000","last_peered":"2024-04-25T10:14:29.703941+0000","last_clean":"2024-04-25T10:14:29.703941+0000","last_became_active":"2024-04-25T10:14:28.719956+0000","last_became_peered":"2024-04-25T10:14:28.719956+0000","last_unstale":"2024-04-25T10:14:29.703941+0000","last_undegraded":"2024-04-25T10:14:29.703941+0000","last_fullsized":"2024-04-25T10:14:29.703941+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:04.773052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.707527+0000","last_change":"2024-04-25T10:14:28.705785+0000","last_active":"2024-04-25T10:14:28.707527+0000","last_peered":"2024-04-25T10:14:28.707527+0000","last_clean":"2024-04-25T10:14:28.707527+0000","last_became_active":"2024-04-25T10:14:28.703647+0000","last_became_peered":"2024-04-25T10:14:28.703647+0000","last_unstale":"2024-04-25T10:14:28.707527+0000","last_undegraded":"2024-04-25T10:14:28.707527+0000","last_fullsized":"2024-04-25T10:14:28.707527+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:23.165799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.711025+0000","last_change":"2024-04-25T10:14:28.712571+0000","last_active":"2024-04-25T10:14:29.711025+0000","last_peered":"2024-04-25T10:14:29.711025+0000","last_clean":"2024-04-25T10:14:29.711025+0000","last_became_active":"2024-04-25T10:14:28.710510+0000","last_became_peered":"2024-04-25T10:14:28.710510+0000","last_unstale":"2024-04-25T10:14:29.711025+0000","last_undegraded":"2024-04-25T10:14:29.711025+0000","last_fullsized":"2024-04-25T10:14:29.711025+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:15.661063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710858+0000","last_change":"2024-04-25T10:14:28.714854+0000","last_active":"2024-04-25T10:14:29.710858+0000","last_peered":"2024-04-25T10:14:29.710858+0000","last_clean":"2024-04-25T10:14:29.710858+0000","last_became_active":"2024-04-25T10:14:28.713819+0000","last_became_peered":"2024-04-25T10:14:28.713819+0000","last_unstale":"2024-04-25T10:14:29.710858+0000","last_undegraded":"2024-04-25T10:14:29.710858+0000","last_fullsized":"2024-04-25T10:14:29.710858+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:36.344770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.691484+0000","last_change":"2024-04-25T10:14:28.689143+0000","last_active":"2024-04-25T10:14:28.691484+0000","last_peered":"2024-04-25T10:14:28.691484+0000","last_clean":"2024-04-25T10:14:28.691484+0000","last_became_active":"2024-04-25T10:14:28.688839+0000","last_became_peered":"2024-04-25T10:14:28.688839+0000","last_unstale":"2024-04-25T10:14:28.691484+0000","last_undegraded":"2024-04-25T10:14:28.691484+0000","last_fullsized":"2024-04-25T10:14:28.691484+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:20.841913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.706961+0000","last_change":"2024-04-25T10:14:28.704358+0000","last_active":"2024-04-25T10:14:28.706961+0000","last_peered":"2024-04-25T10:14:28.706961+0000","last_clean":"2024-04-25T10:14:28.706961+0000","last_became_active":"2024-04-25T10:14:28.703956+0000","last_became_peered":"2024-04-25T10:14:28.703956+0000","last_unstale":"2024-04-25T10:14:28.706961+0000","last_undegraded":"2024-04-25T10:14:28.706961+0000","last_fullsized":"2024-04-25T10:14:28.706961+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:13.846771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705121+0000","last_change":"2024-04-25T10:14:28.730615+0000","last_active":"2024-04-25T10:14:29.705121+0000","last_peered":"2024-04-25T10:14:29.705121+0000","last_clean":"2024-04-25T10:14:29.705121+0000","last_became_active":"2024-04-25T10:14:28.720383+0000","last_became_peered":"2024-04-25T10:14:28.720383+0000","last_unstale":"2024-04-25T10:14:29.705121+0000","last_undegraded":"2024-04-25T10:14:29.705121+0000","last_fullsized":"2024-04-25T10:14:29.705121+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:41.946199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.706114+0000","last_change":"2024-04-25T10:14:28.704098+0000","last_active":"2024-04-25T10:14:28.706114+0000","last_peered":"2024-04-25T10:14:28.706114+0000","last_clean":"2024-04-25T10:14:28.706114+0000","last_became_active":"2024-04-25T10:14:28.703842+0000","last_became_peered":"2024-04-25T10:14:28.703842+0000","last_unstale":"2024-04-25T10:14:28.706114+0000","last_undegraded":"2024-04-25T10:14:28.706114+0000","last_fullsized":"2024-04-25T10:14:28.706114+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:21.110448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.730732+0000","last_change":"2024-04-25T10:14:28.730293+0000","last_active":"2024-04-25T10:14:28.730732+0000","last_peered":"2024-04-25T10:14:28.730732+0000","last_clean":"2024-04-25T10:14:28.730732+0000","last_became_active":"2024-04-25T10:14:28.730056+0000","last_became_peered":"2024-04-25T10:14:28.730056+0000","last_unstale":"2024-04-25T10:14:28.730732+0000","last_undegraded":"2024-04-25T10:14:28.730732+0000","last_fullsized":"2024-04-25T10:14:28.730732+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:15.661063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704901+0000","last_change":"2024-04-25T10:14:28.733718+0000","last_active":"2024-04-25T10:14:29.704901+0000","last_peered":"2024-04-25T10:14:29.704901+0000","last_clean":"2024-04-25T10:14:29.704901+0000","last_became_active":"2024-04-25T10:14:28.732712+0000","last_became_peered":"2024-04-25T10:14:28.732712+0000","last_unstale":"2024-04-25T10:14:29.704901+0000","last_undegraded":"2024-04-25T10:14:29.704901+0000","last_fullsized":"2024-04-25T10:14:29.704901+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:14.015247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.699233+0000","last_change":"2024-04-25T10:14:28.697898+0000","last_active":"2024-04-25T10:14:28.699233+0000","last_peered":"2024-04-25T10:14:28.699233+0000","last_clean":"2024-04-25T10:14:28.699233+0000","last_became_active":"2024-04-25T10:14:28.695278+0000","last_became_peered":"2024-04-25T10:14:28.695278+0000","last_unstale":"2024-04-25T10:14:28.699233+0000","last_undegraded":"2024-04-25T10:14:28.699233+0000","last_fullsized":"2024-04-25T10:14:28.699233+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:19.717882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"142'4","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.719852+0000","last_change":"2024-04-25T10:14:27.653333+0000","last_active":"2024-04-25T10:14:28.719852+0000","last_peered":"2024-04-25T10:14:28.719852+0000","last_clean":"2024-04-25T10:14:28.719852+0000","last_became_active":"2024-04-25T10:14:27.653012+0000","last_became_peered":"2024-04-25T10:14:27.653012+0000","last_unstale":"2024-04-25T10:14:28.719852+0000","last_undegraded":"2024-04-25T10:14:28.719852+0000","last_fullsized":"2024-04-25T10:14:28.719852+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:44.980700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.699165+0000","last_change":"2024-04-25T10:14:28.697817+0000","last_active":"2024-04-25T10:14:28.699165+0000","last_peered":"2024-04-25T10:14:28.699165+0000","last_clean":"2024-04-25T10:14:28.699165+0000","last_became_active":"2024-04-25T10:14:28.695159+0000","last_became_peered":"2024-04-25T10:14:28.695159+0000","last_unstale":"2024-04-25T10:14:28.699165+0000","last_undegraded":"2024-04-25T10:14:28.699165+0000","last_fullsized":"2024-04-25T10:14:28.699165+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:14.015247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.717275+0000","last_change":"2024-04-25T10:14:27.674043+0000","last_active":"2024-04-25T10:14:29.717275+0000","last_peered":"2024-04-25T10:14:29.717275+0000","last_clean":"2024-04-25T10:14:29.717275+0000","last_became_active":"2024-04-25T10:14:27.672434+0000","last_became_peered":"2024-04-25T10:14:27.672434+0000","last_unstale":"2024-04-25T10:14:29.717275+0000","last_undegraded":"2024-04-25T10:14:29.717275+0000","last_fullsized":"2024-04-25T10:14:29.717275+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:03.615957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704851+0000","last_change":"2024-04-25T10:14:28.730968+0000","last_active":"2024-04-25T10:14:29.704851+0000","last_peered":"2024-04-25T10:14:29.704851+0000","last_clean":"2024-04-25T10:14:29.704851+0000","last_became_active":"2024-04-25T10:14:28.730560+0000","last_became_peered":"2024-04-25T10:14:28.730560+0000","last_unstale":"2024-04-25T10:14:29.704851+0000","last_undegraded":"2024-04-25T10:14:29.704851+0000","last_fullsized":"2024-04-25T10:14:29.704851+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:59.981579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.716996+0000","last_change":"2024-04-25T10:14:27.673638+0000","last_active":"2024-04-25T10:14:29.716996+0000","last_peered":"2024-04-25T10:14:29.716996+0000","last_clean":"2024-04-25T10:14:29.716996+0000","last_became_active":"2024-04-25T10:14:27.672946+0000","last_became_peered":"2024-04-25T10:14:27.672946+0000","last_unstale":"2024-04-25T10:14:29.716996+0000","last_undegraded":"2024-04-25T10:14:29.716996+0000","last_fullsized":"2024-04-25T10:14:29.716996+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:20.239965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.717520+0000","last_change":"2024-04-25T10:14:28.718512+0000","last_active":"2024-04-25T10:14:29.717520+0000","last_peered":"2024-04-25T10:14:29.717520+0000","last_clean":"2024-04-25T10:14:29.717520+0000","last_became_active":"2024-04-25T10:14:28.717781+0000","last_became_peered":"2024-04-25T10:14:28.717781+0000","last_unstale":"2024-04-25T10:14:29.717520+0000","last_undegraded":"2024-04-25T10:14:29.717520+0000","last_fullsized":"2024-04-25T10:14:29.717520+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:22.743612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"142'2","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.735868+0000","last_change":"2024-04-25T10:14:27.688102+0000","last_active":"2024-04-25T10:14:28.735868+0000","last_peered":"2024-04-25T10:14:28.735868+0000","last_clean":"2024-04-25T10:14:28.735868+0000","last_became_active":"2024-04-25T10:14:27.687749+0000","last_became_peered":"2024-04-25T10:14:27.687749+0000","last_unstale":"2024-04-25T10:14:28.735868+0000","last_undegraded":"2024-04-25T10:14:28.735868+0000","last_fullsized":"2024-04-25T10:14:28.735868+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:03.758740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.700029+0000","last_change":"2024-04-25T10:14:28.706628+0000","last_active":"2024-04-25T10:14:29.700029+0000","last_peered":"2024-04-25T10:14:29.700029+0000","last_clean":"2024-04-25T10:14:29.700029+0000","last_became_active":"2024-04-25T10:14:28.706311+0000","last_became_peered":"2024-04-25T10:14:28.706311+0000","last_unstale":"2024-04-25T10:14:29.700029+0000","last_undegraded":"2024-04-25T10:14:29.700029+0000","last_fullsized":"2024-04-25T10:14:29.700029+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:18.941190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705212+0000","last_change":"2024-04-25T10:14:27.687454+0000","last_active":"2024-04-25T10:14:29.705212+0000","last_peered":"2024-04-25T10:14:29.705212+0000","last_clean":"2024-04-25T10:14:29.705212+0000","last_became_active":"2024-04-25T10:14:27.686708+0000","last_became_peered":"2024-04-25T10:14:27.686708+0000","last_unstale":"2024-04-25T10:14:29.705212+0000","last_undegraded":"2024-04-25T10:14:29.705212+0000","last_fullsized":"2024-04-25T10:14:29.705212+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:35.421123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705651+0000","last_change":"2024-04-25T10:14:28.689632+0000","last_active":"2024-04-25T10:14:29.705651+0000","last_peered":"2024-04-25T10:14:29.705651+0000","last_clean":"2024-04-25T10:14:29.705651+0000","last_became_active":"2024-04-25T10:14:28.688933+0000","last_became_peered":"2024-04-25T10:14:28.688933+0000","last_unstale":"2024-04-25T10:14:29.705651+0000","last_undegraded":"2024-04-25T10:14:29.705651+0000","last_fullsized":"2024-04-25T10:14:29.705651+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:04.773052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"142'1","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.720092+0000","last_change":"2024-04-25T10:14:27.669065+0000","last_active":"2024-04-25T10:14:28.720092+0000","last_peered":"2024-04-25T10:14:28.720092+0000","last_clean":"2024-04-25T10:14:28.720092+0000","last_became_active":"2024-04-25T10:14:27.668777+0000","last_became_peered":"2024-04-25T10:14:27.668777+0000","last_unstale":"2024-04-25T10:14:28.720092+0000","last_undegraded":"2024-04-25T10:14:28.720092+0000","last_fullsized":"2024-04-25T10:14:28.720092+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:40.931888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.713382+0000","last_change":"2024-04-25T10:14:28.694888+0000","last_active":"2024-04-25T10:14:28.713382+0000","last_peered":"2024-04-25T10:14:28.713382+0000","last_clean":"2024-04-25T10:14:28.713382+0000","last_became_active":"2024-04-25T10:14:28.687566+0000","last_became_peered":"2024-04-25T10:14:28.687566+0000","last_unstale":"2024-04-25T10:14:28.713382+0000","last_undegraded":"2024-04-25T10:14:28.713382+0000","last_fullsized":"2024-04-25T10:14:28.713382+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:46.915924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710251+0000","last_change":"2024-04-25T10:14:27.664262+0000","last_active":"2024-04-25T10:14:29.710251+0000","last_peered":"2024-04-25T10:14:29.710251+0000","last_clean":"2024-04-25T10:14:29.710251+0000","last_became_active":"2024-04-25T10:14:27.663602+0000","last_became_peered":"2024-04-25T10:14:27.663602+0000","last_unstale":"2024-04-25T10:14:29.710251+0000","last_undegraded":"2024-04-25T10:14:29.710251+0000","last_fullsized":"2024-04-25T10:14:29.710251+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:58.645137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710396+0000","last_change":"2024-04-25T10:14:28.735780+0000","last_active":"2024-04-25T10:14:29.710396+0000","last_peered":"2024-04-25T10:14:29.710396+0000","last_clean":"2024-04-25T10:14:29.710396+0000","last_became_active":"2024-04-25T10:14:28.734958+0000","last_became_peered":"2024-04-25T10:14:28.734958+0000","last_unstale":"2024-04-25T10:14:29.710396+0000","last_undegraded":"2024-04-25T10:14:29.710396+0000","last_fullsized":"2024-04-25T10:14:29.710396+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:36.116720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.676058+0000","last_change":"2024-04-25T10:14:27.662758+0000","last_active":"2024-04-25T10:14:28.676058+0000","last_peered":"2024-04-25T10:14:28.676058+0000","last_clean":"2024-04-25T10:14:28.676058+0000","last_became_active":"2024-04-25T10:14:27.662472+0000","last_became_peered":"2024-04-25T10:14:27.662472+0000","last_unstale":"2024-04-25T10:14:28.676058+0000","last_undegraded":"2024-04-25T10:14:28.676058+0000","last_fullsized":"2024-04-25T10:14:28.676058+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:18.703571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704261+0000","last_change":"2024-04-25T10:14:28.711268+0000","last_active":"2024-04-25T10:14:29.704261+0000","last_peered":"2024-04-25T10:14:29.704261+0000","last_clean":"2024-04-25T10:14:29.704261+0000","last_became_active":"2024-04-25T10:14:28.709576+0000","last_became_peered":"2024-04-25T10:14:28.709576+0000","last_unstale":"2024-04-25T10:14:29.704261+0000","last_undegraded":"2024-04-25T10:14:29.704261+0000","last_fullsized":"2024-04-25T10:14:29.704261+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:59.659448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.717556+0000","last_change":"2024-04-25T10:14:27.660215+0000","last_active":"2024-04-25T10:14:29.717556+0000","last_peered":"2024-04-25T10:14:29.717556+0000","last_clean":"2024-04-25T10:14:29.717556+0000","last_became_active":"2024-04-25T10:14:27.659448+0000","last_became_peered":"2024-04-25T10:14:27.659448+0000","last_unstale":"2024-04-25T10:14:29.717556+0000","last_undegraded":"2024-04-25T10:14:29.717556+0000","last_fullsized":"2024-04-25T10:14:29.717556+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:52.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.708626+0000","last_change":"2024-04-25T10:14:28.707374+0000","last_active":"2024-04-25T10:14:28.708626+0000","last_peered":"2024-04-25T10:14:28.708626+0000","last_clean":"2024-04-25T10:14:28.708626+0000","last_became_active":"2024-04-25T10:14:28.706874+0000","last_became_peered":"2024-04-25T10:14:28.706874+0000","last_unstale":"2024-04-25T10:14:28.708626+0000","last_undegraded":"2024-04-25T10:14:28.708626+0000","last_fullsized":"2024-04-25T10:14:28.708626+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:36.205090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.711509+0000","last_change":"2024-04-25T10:14:27.667663+0000","last_active":"2024-04-25T10:14:29.711509+0000","last_peered":"2024-04-25T10:14:29.711509+0000","last_clean":"2024-04-25T10:14:29.711509+0000","last_became_active":"2024-04-25T10:14:27.666982+0000","last_became_peered":"2024-04-25T10:14:27.666982+0000","last_unstale":"2024-04-25T10:14:29.711509+0000","last_undegraded":"2024-04-25T10:14:29.711509+0000","last_fullsized":"2024-04-25T10:14:29.711509+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:13.000936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.716335+0000","last_change":"2024-04-25T10:14:28.714480+0000","last_active":"2024-04-25T10:14:28.716335+0000","last_peered":"2024-04-25T10:14:28.716335+0000","last_clean":"2024-04-25T10:14:28.716335+0000","last_became_active":"2024-04-25T10:14:28.714209+0000","last_became_peered":"2024-04-25T10:14:28.714209+0000","last_unstale":"2024-04-25T10:14:28.716335+0000","last_undegraded":"2024-04-25T10:14:28.716335+0000","last_fullsized":"2024-04-25T10:14:28.716335+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:11.494674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"142'3","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.717750+0000","last_change":"2024-04-25T10:14:27.656191+0000","last_active":"2024-04-25T10:14:28.717750+0000","last_peered":"2024-04-25T10:14:28.717750+0000","last_clean":"2024-04-25T10:14:28.717750+0000","last_became_active":"2024-04-25T10:14:27.655900+0000","last_became_peered":"2024-04-25T10:14:27.655900+0000","last_unstale":"2024-04-25T10:14:28.717750+0000","last_undegraded":"2024-04-25T10:14:28.717750+0000","last_fullsized":"2024-04-25T10:14:28.717750+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:08.263455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704655+0000","last_change":"2024-04-25T10:14:28.732317+0000","last_active":"2024-04-25T10:14:29.704655+0000","last_peered":"2024-04-25T10:14:29.704655+0000","last_clean":"2024-04-25T10:14:29.704655+0000","last_became_active":"2024-04-25T10:14:28.731900+0000","last_became_peered":"2024-04-25T10:14:28.731900+0000","last_unstale":"2024-04-25T10:14:29.704655+0000","last_undegraded":"2024-04-25T10:14:29.704655+0000","last_fullsized":"2024-04-25T10:14:29.704655+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:46.915924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"142'2","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.719886+0000","last_change":"2024-04-25T10:14:27.654880+0000","last_active":"2024-04-25T10:14:28.719886+0000","last_peered":"2024-04-25T10:14:28.719886+0000","last_clean":"2024-04-25T10:14:28.719886+0000","last_became_active":"2024-04-25T10:14:27.654414+0000","last_became_peered":"2024-04-25T10:14:27.654414+0000","last_unstale":"2024-04-25T10:14:28.719886+0000","last_undegraded":"2024-04-25T10:14:28.719886+0000","last_fullsized":"2024-04-25T10:14:28.719886+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:11.666710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.713491+0000","last_change":"2024-04-25T10:14:28.698386+0000","last_active":"2024-04-25T10:14:28.713491+0000","last_peered":"2024-04-25T10:14:28.713491+0000","last_clean":"2024-04-25T10:14:28.713491+0000","last_became_active":"2024-04-25T10:14:28.696419+0000","last_became_peered":"2024-04-25T10:14:28.696419+0000","last_unstale":"2024-04-25T10:14:28.713491+0000","last_undegraded":"2024-04-25T10:14:28.713491+0000","last_fullsized":"2024-04-25T10:14:28.713491+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:10.122313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.706750+0000","last_change":"2024-04-25T10:14:27.685045+0000","last_active":"2024-04-25T10:14:29.706750+0000","last_peered":"2024-04-25T10:14:29.706750+0000","last_clean":"2024-04-25T10:14:29.706750+0000","last_became_active":"2024-04-25T10:14:27.684316+0000","last_became_peered":"2024-04-25T10:14:27.684316+0000","last_unstale":"2024-04-25T10:14:29.706750+0000","last_undegraded":"2024-04-25T10:14:29.706750+0000","last_fullsized":"2024-04-25T10:14:29.706750+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:51.974348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.693449+0000","last_change":"2024-04-25T10:14:28.692061+0000","last_active":"2024-04-25T10:14:28.693449+0000","last_peered":"2024-04-25T10:14:28.693449+0000","last_clean":"2024-04-25T10:14:28.693449+0000","last_became_active":"2024-04-25T10:14:28.691813+0000","last_became_peered":"2024-04-25T10:14:28.691813+0000","last_unstale":"2024-04-25T10:14:28.693449+0000","last_undegraded":"2024-04-25T10:14:28.693449+0000","last_fullsized":"2024-04-25T10:14:28.693449+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:57.109328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705928+0000","last_change":"2024-04-25T10:14:27.670240+0000","last_active":"2024-04-25T10:14:29.705928+0000","last_peered":"2024-04-25T10:14:29.705928+0000","last_clean":"2024-04-25T10:14:29.705928+0000","last_became_active":"2024-04-25T10:14:27.669570+0000","last_became_peered":"2024-04-25T10:14:27.669570+0000","last_unstale":"2024-04-25T10:14:29.705928+0000","last_undegraded":"2024-04-25T10:14:29.705928+0000","last_fullsized":"2024-04-25T10:14:29.705928+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:06.165329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.699255+0000","last_change":"2024-04-25T10:14:28.697944+0000","last_active":"2024-04-25T10:14:28.699255+0000","last_peered":"2024-04-25T10:14:28.699255+0000","last_clean":"2024-04-25T10:14:28.699255+0000","last_became_active":"2024-04-25T10:14:28.695583+0000","last_became_peered":"2024-04-25T10:14:28.695583+0000","last_unstale":"2024-04-25T10:14:28.699255+0000","last_undegraded":"2024-04-25T10:14:28.699255+0000","last_fullsized":"2024-04-25T10:14:28.699255+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:41.946199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"142'8","reported_seq":22,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.709870+0000","last_change":"2024-04-25T10:14:27.670240+0000","last_active":"2024-04-25T10:14:29.709870+0000","last_peered":"2024-04-25T10:14:29.709870+0000","last_clean":"2024-04-25T10:14:29.709870+0000","last_became_active":"2024-04-25T10:14:27.669471+0000","last_became_peered":"2024-04-25T10:14:27.669471+0000","last_unstale":"2024-04-25T10:14:29.709870+0000","last_undegraded":"2024-04-25T10:14:29.709870+0000","last_fullsized":"2024-04-25T10:14:29.709870+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:58.967268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.715612+0000","last_change":"2024-04-25T10:14:28.709296+0000","last_active":"2024-04-25T10:14:29.715612+0000","last_peered":"2024-04-25T10:14:29.715612+0000","last_clean":"2024-04-25T10:14:29.715612+0000","last_became_active":"2024-04-25T10:14:28.708559+0000","last_became_peered":"2024-04-25T10:14:28.708559+0000","last_unstale":"2024-04-25T10:14:29.715612+0000","last_undegraded":"2024-04-25T10:14:29.715612+0000","last_fullsized":"2024-04-25T10:14:29.715612+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:14.262169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"142'1","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.724543+0000","last_change":"2024-04-25T10:14:27.667141+0000","last_active":"2024-04-25T10:14:28.724543+0000","last_peered":"2024-04-25T10:14:28.724543+0000","last_clean":"2024-04-25T10:14:28.724543+0000","last_became_active":"2024-04-25T10:14:27.666848+0000","last_became_peered":"2024-04-25T10:14:27.666848+0000","last_unstale":"2024-04-25T10:14:28.724543+0000","last_undegraded":"2024-04-25T10:14:28.724543+0000","last_fullsized":"2024-04-25T10:14:28.724543+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:56.197132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.716741+0000","last_change":"2024-04-25T10:14:28.703857+0000","last_active":"2024-04-25T10:14:29.716741+0000","last_peered":"2024-04-25T10:14:29.716741+0000","last_clean":"2024-04-25T10:14:29.716741+0000","last_became_active":"2024-04-25T10:14:28.702623+0000","last_became_peered":"2024-04-25T10:14:28.702623+0000","last_unstale":"2024-04-25T10:14:29.716741+0000","last_undegraded":"2024-04-25T10:14:29.716741+0000","last_fullsized":"2024-04-25T10:14:29.716741+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:18.754906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.717384+0000","last_change":"2024-04-25T10:14:27.660304+0000","last_active":"2024-04-25T10:14:29.717384+0000","last_peered":"2024-04-25T10:14:29.717384+0000","last_clean":"2024-04-25T10:14:29.717384+0000","last_became_active":"2024-04-25T10:14:27.659613+0000","last_became_peered":"2024-04-25T10:14:27.659613+0000","last_unstale":"2024-04-25T10:14:29.717384+0000","last_undegraded":"2024-04-25T10:14:29.717384+0000","last_fullsized":"2024-04-25T10:14:29.717384+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:33.859995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.711181+0000","last_change":"2024-04-25T10:14:28.722896+0000","last_active":"2024-04-25T10:14:29.711181+0000","last_peered":"2024-04-25T10:14:29.711181+0000","last_clean":"2024-04-25T10:14:29.711181+0000","last_became_active":"2024-04-25T10:14:28.716643+0000","last_became_peered":"2024-04-25T10:14:28.716643+0000","last_unstale":"2024-04-25T10:14:29.711181+0000","last_undegraded":"2024-04-25T10:14:29.711181+0000","last_fullsized":"2024-04-25T10:14:29.711181+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:09.909912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"142'1","reported_seq":13,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.721757+0000","last_change":"2024-04-25T10:14:27.669742+0000","last_active":"2024-04-25T10:14:28.721757+0000","last_peered":"2024-04-25T10:14:28.721757+0000","last_clean":"2024-04-25T10:14:28.721757+0000","last_became_active":"2024-04-25T10:14:27.669472+0000","last_became_peered":"2024-04-25T10:14:27.669472+0000","last_unstale":"2024-04-25T10:14:28.721757+0000","last_undegraded":"2024-04-25T10:14:28.721757+0000","last_fullsized":"2024-04-25T10:14:28.721757+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:58.645137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.711259+0000","last_change":"2024-04-25T10:14:28.720299+0000","last_active":"2024-04-25T10:14:29.711259+0000","last_peered":"2024-04-25T10:14:29.711259+0000","last_clean":"2024-04-25T10:14:29.711259+0000","last_became_active":"2024-04-25T10:14:28.719933+0000","last_became_peered":"2024-04-25T10:14:28.719933+0000","last_unstale":"2024-04-25T10:14:29.711259+0000","last_undegraded":"2024-04-25T10:14:29.711259+0000","last_fullsized":"2024-04-25T10:14:29.711259+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.301711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.707020+0000","last_change":"2024-04-25T10:14:27.689022+0000","last_active":"2024-04-25T10:14:29.707020+0000","last_peered":"2024-04-25T10:14:29.707020+0000","last_clean":"2024-04-25T10:14:29.707020+0000","last_became_active":"2024-04-25T10:14:27.688317+0000","last_became_peered":"2024-04-25T10:14:27.688317+0000","last_unstale":"2024-04-25T10:14:29.707020+0000","last_undegraded":"2024-04-25T10:14:29.707020+0000","last_fullsized":"2024-04-25T10:14:29.707020+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:14.068208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.715446+0000","last_change":"2024-04-25T10:14:28.734635+0000","last_active":"2024-04-25T10:14:29.715446+0000","last_peered":"2024-04-25T10:14:29.715446+0000","last_clean":"2024-04-25T10:14:29.715446+0000","last_became_active":"2024-04-25T10:14:28.734312+0000","last_became_peered":"2024-04-25T10:14:28.734312+0000","last_unstale":"2024-04-25T10:14:29.715446+0000","last_undegraded":"2024-04-25T10:14:29.715446+0000","last_fullsized":"2024-04-25T10:14:29.715446+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:57.748394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.699974+0000","last_change":"2024-04-25T10:14:27.687304+0000","last_active":"2024-04-25T10:14:29.699974+0000","last_peered":"2024-04-25T10:14:29.699974+0000","last_clean":"2024-04-25T10:14:29.699974+0000","last_became_active":"2024-04-25T10:14:27.686967+0000","last_became_peered":"2024-04-25T10:14:27.686967+0000","last_unstale":"2024-04-25T10:14:29.699974+0000","last_undegraded":"2024-04-25T10:14:29.699974+0000","last_fullsized":"2024-04-25T10:14:29.699974+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:52.606118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.720378+0000","last_change":"2024-04-25T10:14:28.719716+0000","last_active":"2024-04-25T10:14:28.720378+0000","last_peered":"2024-04-25T10:14:28.720378+0000","last_clean":"2024-04-25T10:14:28.720378+0000","last_became_active":"2024-04-25T10:14:28.719312+0000","last_became_peered":"2024-04-25T10:14:28.719312+0000","last_unstale":"2024-04-25T10:14:28.720378+0000","last_undegraded":"2024-04-25T10:14:28.720378+0000","last_fullsized":"2024-04-25T10:14:28.720378+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:22.157823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704352+0000","last_change":"2024-04-25T10:14:27.694637+0000","last_active":"2024-04-25T10:14:29.704352+0000","last_peered":"2024-04-25T10:14:29.704352+0000","last_clean":"2024-04-25T10:14:29.704352+0000","last_became_active":"2024-04-25T10:14:27.693879+0000","last_became_peered":"2024-04-25T10:14:27.693879+0000","last_unstale":"2024-04-25T10:14:29.704352+0000","last_undegraded":"2024-04-25T10:14:29.704352+0000","last_fullsized":"2024-04-25T10:14:29.704352+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:18.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.715442+0000","last_change":"2024-04-25T10:14:28.707634+0000","last_active":"2024-04-25T10:14:29.715442+0000","last_peered":"2024-04-25T10:14:29.715442+0000","last_clean":"2024-04-25T10:14:29.715442+0000","last_became_active":"2024-04-25T10:14:28.706813+0000","last_became_peered":"2024-04-25T10:14:28.706813+0000","last_unstale":"2024-04-25T10:14:29.715442+0000","last_undegraded":"2024-04-25T10:14:29.715442+0000","last_fullsized":"2024-04-25T10:14:29.715442+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:55.950752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.699894+0000","last_change":"2024-04-25T10:14:27.687256+0000","last_active":"2024-04-25T10:14:29.699894+0000","last_peered":"2024-04-25T10:14:29.699894+0000","last_clean":"2024-04-25T10:14:29.699894+0000","last_became_active":"2024-04-25T10:14:27.686763+0000","last_became_peered":"2024-04-25T10:14:27.686763+0000","last_unstale":"2024-04-25T10:14:29.699894+0000","last_undegraded":"2024-04-25T10:14:29.699894+0000","last_fullsized":"2024-04-25T10:14:29.699894+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:33.859995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.699284+0000","last_change":"2024-04-25T10:14:28.697823+0000","last_active":"2024-04-25T10:14:28.699284+0000","last_peered":"2024-04-25T10:14:28.699284+0000","last_clean":"2024-04-25T10:14:28.699284+0000","last_became_active":"2024-04-25T10:14:28.695402+0000","last_became_peered":"2024-04-25T10:14:28.695402+0000","last_unstale":"2024-04-25T10:14:28.699284+0000","last_undegraded":"2024-04-25T10:14:28.699284+0000","last_fullsized":"2024-04-25T10:14:28.699284+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:12.681021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705070+0000","last_change":"2024-04-25T10:14:27.691808+0000","last_active":"2024-04-25T10:14:29.705070+0000","last_peered":"2024-04-25T10:14:29.705070+0000","last_clean":"2024-04-25T10:14:29.705070+0000","last_became_active":"2024-04-25T10:14:27.690407+0000","last_became_peered":"2024-04-25T10:14:27.690407+0000","last_unstale":"2024-04-25T10:14:29.705070+0000","last_undegraded":"2024-04-25T10:14:29.705070+0000","last_fullsized":"2024-04-25T10:14:29.705070+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:01.572491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.712767+0000","last_change":"2024-04-25T10:14:28.710904+0000","last_active":"2024-04-25T10:14:28.712767+0000","last_peered":"2024-04-25T10:14:28.712767+0000","last_clean":"2024-04-25T10:14:28.712767+0000","last_became_active":"2024-04-25T10:14:28.710418+0000","last_became_peered":"2024-04-25T10:14:28.710418+0000","last_unstale":"2024-04-25T10:14:28.712767+0000","last_undegraded":"2024-04-25T10:14:28.712767+0000","last_fullsized":"2024-04-25T10:14:28.712767+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:36.116720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.707746+0000","last_change":"2024-04-25T10:14:27.684947+0000","last_active":"2024-04-25T10:14:29.707746+0000","last_peered":"2024-04-25T10:14:29.707746+0000","last_clean":"2024-04-25T10:14:29.707746+0000","last_became_active":"2024-04-25T10:14:27.684161+0000","last_became_peered":"2024-04-25T10:14:27.684161+0000","last_unstale":"2024-04-25T10:14:29.707746+0000","last_undegraded":"2024-04-25T10:14:29.707746+0000","last_fullsized":"2024-04-25T10:14:29.707746+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:14.323948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.707319+0000","last_change":"2024-04-25T10:14:28.701381+0000","last_active":"2024-04-25T10:14:29.707319+0000","last_peered":"2024-04-25T10:14:29.707319+0000","last_clean":"2024-04-25T10:14:29.707319+0000","last_became_active":"2024-04-25T10:14:28.700480+0000","last_became_peered":"2024-04-25T10:14:28.700480+0000","last_unstale":"2024-04-25T10:14:29.707319+0000","last_undegraded":"2024-04-25T10:14:29.707319+0000","last_fullsized":"2024-04-25T10:14:29.707319+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:14.015247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"143'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.714333+0000","last_change":"2024-04-25T10:14:27.670622+0000","last_active":"2024-04-25T10:14:29.714333+0000","last_peered":"2024-04-25T10:14:29.714333+0000","last_clean":"2024-04-25T10:14:29.714333+0000","last_became_active":"2024-04-25T10:14:27.669279+0000","last_became_peered":"2024-04-25T10:14:27.669279+0000","last_unstale":"2024-04-25T10:14:29.714333+0000","last_undegraded":"2024-04-25T10:14:29.714333+0000","last_fullsized":"2024-04-25T10:14:29.714333+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:18.703571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.711385+0000","last_change":"2024-04-25T10:14:28.734963+0000","last_active":"2024-04-25T10:14:29.711385+0000","last_peered":"2024-04-25T10:14:29.711385+0000","last_clean":"2024-04-25T10:14:29.711385+0000","last_became_active":"2024-04-25T10:14:28.734583+0000","last_became_peered":"2024-04-25T10:14:28.734583+0000","last_unstale":"2024-04-25T10:14:29.711385+0000","last_undegraded":"2024-04-25T10:14:29.711385+0000","last_fullsized":"2024-04-25T10:14:29.711385+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:43.056854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.682766+0000","last_change":"2024-04-25T10:14:27.681632+0000","last_active":"2024-04-25T10:14:28.682766+0000","last_peered":"2024-04-25T10:14:28.682766+0000","last_clean":"2024-04-25T10:14:28.682766+0000","last_became_active":"2024-04-25T10:14:27.681255+0000","last_became_peered":"2024-04-25T10:14:27.681255+0000","last_unstale":"2024-04-25T10:14:28.682766+0000","last_undegraded":"2024-04-25T10:14:28.682766+0000","last_fullsized":"2024-04-25T10:14:28.682766+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:43.911328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.692091+0000","last_change":"2024-04-25T10:14:28.689844+0000","last_active":"2024-04-25T10:14:28.692091+0000","last_peered":"2024-04-25T10:14:28.692091+0000","last_clean":"2024-04-25T10:14:28.692091+0000","last_became_active":"2024-04-25T10:14:28.689559+0000","last_became_peered":"2024-04-25T10:14:28.689559+0000","last_unstale":"2024-04-25T10:14:28.692091+0000","last_undegraded":"2024-04-25T10:14:28.692091+0000","last_fullsized":"2024-04-25T10:14:28.692091+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:39.841652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.677668+0000","last_change":"2024-04-25T10:14:27.691854+0000","last_active":"2024-04-25T10:14:28.677668+0000","last_peered":"2024-04-25T10:14:28.677668+0000","last_clean":"2024-04-25T10:14:28.677668+0000","last_became_active":"2024-04-25T10:14:27.691496+0000","last_became_peered":"2024-04-25T10:14:27.691496+0000","last_unstale":"2024-04-25T10:14:28.677668+0000","last_undegraded":"2024-04-25T10:14:28.677668+0000","last_fullsized":"2024-04-25T10:14:28.677668+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:09.108002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705124+0000","last_change":"2024-04-25T10:14:28.722151+0000","last_active":"2024-04-25T10:14:29.705124+0000","last_peered":"2024-04-25T10:14:29.705124+0000","last_clean":"2024-04-25T10:14:29.705124+0000","last_became_active":"2024-04-25T10:14:28.720912+0000","last_became_peered":"2024-04-25T10:14:28.720912+0000","last_unstale":"2024-04-25T10:14:29.705124+0000","last_undegraded":"2024-04-25T10:14:29.705124+0000","last_fullsized":"2024-04-25T10:14:29.705124+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:12.681021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710392+0000","last_change":"2024-04-25T10:14:27.672590+0000","last_active":"2024-04-25T10:14:29.710392+0000","last_peered":"2024-04-25T10:14:29.710392+0000","last_clean":"2024-04-25T10:14:29.710392+0000","last_became_active":"2024-04-25T10:14:27.671920+0000","last_became_peered":"2024-04-25T10:14:27.671920+0000","last_unstale":"2024-04-25T10:14:29.710392+0000","last_undegraded":"2024-04-25T10:14:29.710392+0000","last_fullsized":"2024-04-25T10:14:29.710392+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:45.901612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.705800+0000","last_change":"2024-04-25T10:14:28.707811+0000","last_active":"2024-04-25T10:14:29.705800+0000","last_peered":"2024-04-25T10:14:29.705800+0000","last_clean":"2024-04-25T10:14:29.705800+0000","last_became_active":"2024-04-25T10:14:28.702188+0000","last_became_peered":"2024-04-25T10:14:28.702188+0000","last_unstale":"2024-04-25T10:14:29.705800+0000","last_undegraded":"2024-04-25T10:14:29.705800+0000","last_fullsized":"2024-04-25T10:14:29.705800+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:59.659448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.684964+0000","last_change":"2024-04-25T10:14:27.687468+0000","last_active":"2024-04-25T10:14:28.684964+0000","last_peered":"2024-04-25T10:14:28.684964+0000","last_clean":"2024-04-25T10:14:28.684964+0000","last_became_active":"2024-04-25T10:14:27.687077+0000","last_became_peered":"2024-04-25T10:14:27.687077+0000","last_unstale":"2024-04-25T10:14:28.684964+0000","last_undegraded":"2024-04-25T10:14:28.684964+0000","last_fullsized":"2024-04-25T10:14:28.684964+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:17.280053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T10:14:28.714413+0000","last_change":"2024-04-25T10:14:28.712963+0000","last_active":"2024-04-25T10:14:28.714413+0000","last_peered":"2024-04-25T10:14:28.714413+0000","last_clean":"2024-04-25T10:14:28.714413+0000","last_became_active":"2024-04-25T10:14:28.699156+0000","last_became_peered":"2024-04-25T10:14:28.699156+0000","last_unstale":"2024-04-25T10:14:28.714413+0000","last_undegraded":"2024-04-25T10:14:28.714413+0000","last_fullsized":"2024-04-25T10:14:28.714413+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:20.841913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.710513+0000","last_change":"2024-04-25T10:14:27.667498+0000","last_active":"2024-04-25T10:14:29.710513+0000","last_peered":"2024-04-25T10:14:29.710513+0000","last_clean":"2024-04-25T10:14:29.710513+0000","last_became_active":"2024-04-25T10:14:27.666488+0000","last_became_peered":"2024-04-25T10:14:27.666488+0000","last_unstale":"2024-04-25T10:14:29.710513+0000","last_undegraded":"2024-04-25T10:14:29.710513+0000","last_fullsized":"2024-04-25T10:14:29.710513+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:43.911328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704433+0000","last_change":"2024-04-25T10:14:28.728969+0000","last_active":"2024-04-25T10:14:29.704433+0000","last_peered":"2024-04-25T10:14:29.704433+0000","last_clean":"2024-04-25T10:14:29.704433+0000","last_became_active":"2024-04-25T10:14:28.720777+0000","last_became_peered":"2024-04-25T10:14:28.720777+0000","last_unstale":"2024-04-25T10:14:29.704433+0000","last_undegraded":"2024-04-25T10:14:29.704433+0000","last_fullsized":"2024-04-25T10:14:29.704433+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:44.925639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.715330+0000","last_change":"2024-04-25T10:14:27.660245+0000","last_active":"2024-04-25T10:14:29.715330+0000","last_peered":"2024-04-25T10:14:29.715330+0000","last_clean":"2024-04-25T10:14:29.715330+0000","last_became_active":"2024-04-25T10:14:27.659553+0000","last_became_peered":"2024-04-25T10:14:27.659553+0000","last_unstale":"2024-04-25T10:14:29.715330+0000","last_undegraded":"2024-04-25T10:14:29.715330+0000","last_fullsized":"2024-04-25T10:14:29.715330+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:31.269675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.704985+0000","last_change":"2024-04-25T10:14:28.732501+0000","last_active":"2024-04-25T10:14:29.704985+0000","last_peered":"2024-04-25T10:14:29.704985+0000","last_clean":"2024-04-25T10:14:29.704985+0000","last_became_active":"2024-04-25T10:14:28.732062+0000","last_became_peered":"2024-04-25T10:14:28.732062+0000","last_unstale":"2024-04-25T10:14:29.704985+0000","last_undegraded":"2024-04-25T10:14:29.704985+0000","last_fullsized":"2024-04-25T10:14:29.704985+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:27.635277+0000","last_clean_scrub_stamp":"2024-04-25T10:14:27.635277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:19.717882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"142'6","reported_seq":20,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.700134+0000","last_change":"2024-04-25T10:14:27.687376+0000","last_active":"2024-04-25T10:14:29.700134+0000","last_peered":"2024-04-25T10:14:29.700134+0000","last_clean":"2024-04-25T10:14:29.700134+0000","last_became_active":"2024-04-25T10:14:27.687042+0000","last_became_peered":"2024-04-25T10:14:27.687042+0000","last_unstale":"2024-04-25T10:14:29.700134+0000","last_undegraded":"2024-04-25T10:14:29.700134+0000","last_fullsized":"2024-04-25T10:14:29.700134+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:14:26.620966+0000","last_clean_scrub_stamp":"2024-04-25T10:14:26.620966+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:03.615957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":402,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:14:29.715488+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:14:29.715488+0000","last_peered":"2024-04-25T10:14:29.715488+0000","last_clean":"2024-04-25T10:14:29.715488+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:14:29.715488+0000","last_undegraded":"2024-04-25T10:14:29.715488+0000","last_fullsized":"2024-04-25T10:14:29.715488+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69608,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302232,"statfs":{"total":96636764160,"available":96565485568,"internally_reserved":0,"allocated":2379776,"data_stored":1424225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50992,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320848,"statfs":{"total":96636764160,"available":96584548352,"internally_reserved":0,"allocated":1798144,"data_stored":840597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575006,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59296,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312544,"statfs":{"total":96636764160,"available":96576045056,"internally_reserved":0,"allocated":1781760,"data_stored":833835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73196,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298644,"statfs":{"total":96636764160,"available":96561811456,"internally_reserved":0,"allocated":1794048,"data_stored":835447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575006,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77344,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294496,"statfs":{"total":96636764160,"available":96557563904,"internally_reserved":0,"allocated":1789952,"data_stored":834451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50984,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320856,"statfs":{"total":96636764160,"available":96584556544,"internally_reserved":0,"allocated":1789952,"data_stored":833947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46816,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325024,"statfs":{"total":96636764160,"available":96588824576,"internally_reserved":0,"allocated":1785856,"data_stored":833905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575007,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73784,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298056,"statfs":{"total":96636764160,"available":96561209344,"internally_reserved":0,"allocated":2400256,"data_stored":1426521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:14:32.088 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:14:32.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:14:32.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.255+0000 7fa5d57fe640 1 Processor -- start 2024-04-25T10:14:32.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.255+0000 7fa5d57fe640 1 -- start start 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5d57fe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa5d00cd820 0x7fa5d00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5d57fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d015ced0 0x7fa5d015d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5d57fe640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d00ce2e0 con 0x7fa5d0145170 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5d57fe640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d006db00 con 0x7fa5d00cd820 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5d57fe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d0145550 con 0x7fa5d015ced0 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5ceffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa5d0145170 0x7fa5d006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48636/0 (socket says 172.21.15.133:48636) 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5ceffd640 1 -- 172.21.15.133:0/1223251288 learned_addr learned my addr 172.21.15.133:0/1223251288 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5ce7fc640 1 --2- 172.21.15.133:0/1223251288 >> v2:172.21.15.162:3300/0 conn(0x7fa5d00cd820 0x7fa5d00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cf7fe640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d015ced0 0x7fa5d015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156993474 0 0) 0x7fa5d00ce2e0 con 0x7fa5d0145170 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b8003530 con 0x7fa5d0145170 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 700467460 0 0) 0x7fa5b8003530 con 0x7fa5d0145170 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d015ced0 msgr2=0x7fa5d015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d015ced0 0x7fa5d015d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 >> v2:172.21.15.162:3300/0 conn(0x7fa5d00cd820 msgr2=0x7fa5d00cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 --2- 172.21.15.133:0/1223251288 >> v2:172.21.15.162:3300/0 conn(0x7fa5d00cd820 0x7fa5d00cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d006d4b0 con 0x7fa5d0145170 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa5c0001680 con 0x7fa5d0145170 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 >> v1:172.21.15.133:6789/0 conn(0x7fa5d0145170 legacy=0x7fa5d006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d015ced0 0x7fa5b80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 0x7fa5b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5b8004d50 0x7fa5b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d006db00 con 0x7fa5b8003f40 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d0145550 con 0x7fa5b8004d50 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b800b9d0 con 0x7fa5d015ced0 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5ce7fc640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 0x7fa5b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5ceffd640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d015ced0 0x7fa5b80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5cf7fe640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5b8004d50 0x7fa5b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5ce7fc640 1 -- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d015ced0 msgr2=0x7fa5b80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5ce7fc640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d015ced0 0x7fa5b80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5ce7fc640 1 -- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5b8004d50 msgr2=0x7fa5b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5ce7fc640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5b8004d50 0x7fa5b80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5ce7fc640 1 -- 172.21.15.133:0/1223251288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b8003530 con 0x7fa5b8003f40 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.259+0000 7fa5ceffd640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d015ced0 0x7fa5b80039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:32.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ce7fc640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 0x7fa5b8004810 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009ae0 tx=0x7fa5c401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a568f2fe87f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40b2070 con 0x7fa5b8003f40 2024-04-25T10:14:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40a56a0 con 0x7fa5b8003f40 2024-04-25T10:14:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5cdffb640 1 -- 172.21.15.133:0/1223251288 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40096b0 con 0x7fa5b8003f40 2024-04-25T10:14:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 -- 172.21.15.133:0/1223251288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 msgr2=0x7fa5b8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 --2- 172.21.15.133:0/1223251288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 0x7fa5b8004810 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009ae0 tx=0x7fa5c401ca50 comp rx=0 tx=0).stop 2024-04-25T10:14:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5cf7fe640 1 -- 172.21.15.133:0/1223251288 reap_dead start 2024-04-25T10:14:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 -- 172.21.15.133:0/1223251288 shutdown_connections 2024-04-25T10:14:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 -- 172.21.15.133:0/1223251288 >> 172.21.15.133:0/1223251288 conn(0x7fa5d0066f70 msgr2=0x7fa5d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 -- 172.21.15.133:0/1223251288 shutdown_connections 2024-04-25T10:14:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 -- 172.21.15.133:0/1223251288 wait complete. 2024-04-25T10:14:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 Processor -- start 2024-04-25T10:14:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 -- start start 2024-04-25T10:14:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 0x7fa5d013ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d0131900 0x7fa5d0131d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d013c9a0 0x7fa5d013ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d015cb70 con 0x7fa5b8003f40 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d006e6a0 con 0x7fa5d013c9a0 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5d57fe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d015d360 con 0x7fa5d0131900 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ceffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 0x7fa5d013ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ceffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 0x7fa5d013ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37418/0 (socket says 172.21.15.133:37418) 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ceffd640 1 -- 172.21.15.133:0/2755373651 learned_addr learned my addr 172.21.15.133:0/2755373651 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5cf7fe640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d013c9a0 0x7fa5d013ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ce7fc640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d0131900 0x7fa5d0131d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ceffd640 1 -- 172.21.15.133:0/2755373651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d0131900 msgr2=0x7fa5d0131d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ceffd640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d0131900 0x7fa5d0131d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ceffd640 1 -- 172.21.15.133:0/2755373651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d013c9a0 msgr2=0x7fa5d013ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ceffd640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d013c9a0 0x7fa5d013ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ceffd640 1 -- 172.21.15.133:0/2755373651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d013d360 con 0x7fa5b8003f40 2024-04-25T10:14:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ce7fc640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d0131900 0x7fa5d0131d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.263+0000 7fa5ceffd640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 0x7fa5d013ea60 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0011790 tx=0x7fa5c0011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.267+0000 7fa5b77fe640 1 -- 172.21.15.133:0/2755373651 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0010040 con 0x7fa5b8003f40 2024-04-25T10:14:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.267+0000 7fa5b77fe640 1 -- 172.21.15.133:0/2755373651 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c001a070 con 0x7fa5b8003f40 2024-04-25T10:14:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.267+0000 7fa5d57fe640 1 -- 172.21.15.133:0/2755373651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d013d5f0 con 0x7fa5b8003f40 2024-04-25T10:14:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.267+0000 7fa5b77fe640 1 -- 172.21.15.133:0/2755373651 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000fb80 con 0x7fa5b8003f40 2024-04-25T10:14:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.267+0000 7fa5d57fe640 1 -- 172.21.15.133:0/2755373651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d013dbb0 con 0x7fa5b8003f40 2024-04-25T10:14:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.267+0000 7fa5d57fe640 1 -- 172.21.15.133:0/2755373651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d0068e80 con 0x7fa5b8003f40 2024-04-25T10:14:32.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.267+0000 7fa5b77fe640 1 -- 172.21.15.133:0/2755373651 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5c000fd20 con 0x7fa5b8003f40 2024-04-25T10:14:32.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.267+0000 7fa5b77fe640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5bc066f60 0x7fa5bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.267+0000 7fa5b77fe640 1 -- 172.21.15.133:0/2755373651 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5c0088310 con 0x7fa5b8003f40 2024-04-25T10:14:32.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.267+0000 7fa5ce7fc640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5bc066f60 0x7fa5bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.267+0000 7fa5ce7fc640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5bc066f60 0x7fa5bc069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40af000 tx=0x7fa5c40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.271+0000 7fa5b77fe640 1 -- 172.21.15.133:0/2755373651 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5c0059190 con 0x7fa5b8003f40 2024-04-25T10:14:32.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.483+0000 7fa5d57fe640 1 -- 172.21.15.133:0/2755373651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa5d015da50 con 0x7fa5b8003f40 2024-04-25T10:14:32.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.483+0000 7fa5b77fe640 1 -- 172.21.15.133:0/2755373651 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa5c0058a90 con 0x7fa5b8003f40 2024-04-25T10:14:32.488 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:14:32.488 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:14:32.488 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:14:32.488 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:14:32.488 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:14:32.488 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:14:32.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 -- 172.21.15.133:0/2755373651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5bc066f60 msgr2=0x7fa5bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5bc066f60 0x7fa5bc069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40af000 tx=0x7fa5c40046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 -- 172.21.15.133:0/2755373651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 msgr2=0x7fa5d013ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 0x7fa5d013ea60 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0011790 tx=0x7fa5c0011c50 comp rx=0 tx=0).stop 2024-04-25T10:14:32.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 -- 172.21.15.133:0/2755373651 shutdown_connections 2024-04-25T10:14:32.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5d013c9a0 0x7fa5d013ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5d0131900 0x7fa5d0131d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa5bc066f60 0x7fa5bc069420 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 --2- 172.21.15.133:0/2755373651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b8003f40 0x7fa5d013ea60 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 -- 172.21.15.133:0/2755373651 >> 172.21.15.133:0/2755373651 conn(0x7fa5d0066f70 msgr2=0x7fa5d00ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 -- 172.21.15.133:0/2755373651 shutdown_connections 2024-04-25T10:14:32.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.487+0000 7fa5d57fe640 1 -- 172.21.15.133:0/2755373651 wait complete. 2024-04-25T10:14:32.505 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:32.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.607+0000 7ff0fc613640 1 Processor -- start 2024-04-25T10:14:32.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.607+0000 7ff0fc613640 1 -- start start 2024-04-25T10:14:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f415d9b0 0x7ff0f40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0f40cee80 0x7ff0f406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f40ca9b0 con 0x7ff0f415dd90 2024-04-25T10:14:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f40cbe30 con 0x7ff0f40cee80 2024-04-25T10:14:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40cac90 con 0x7ff0f415d9b0 2024-04-25T10:14:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fabb0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0f40cee80 0x7ff0f406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0f9bae640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff0f415dd90 0x7ff0f40ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48650/0 (socket says 172.21.15.133:48650) 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0f9bae640 1 -- 172.21.15.133:0/2839010684 learned_addr learned my addr 172.21.15.133:0/2839010684 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fabb0640 1 -- 172.21.15.133:0/2839010684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f415d9b0 msgr2=0x7ff0f40cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fabb0640 1 --2- 172.21.15.133:0/2839010684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f415d9b0 0x7ff0f40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fabb0640 1 -- 172.21.15.133:0/2839010684 >> v1:172.21.15.133:6789/0 conn(0x7ff0f415dd90 legacy=0x7ff0f40ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fabb0640 1 -- 172.21.15.133:0/2839010684 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f40caf70 con 0x7ff0f40cee80 2024-04-25T10:14:32.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fabb0640 1 --2- 172.21.15.133:0/2839010684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f40cee80 0x7ff0f406d6f0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40cbfb0 tx=0x7ff0ec01c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e11c0de46d4c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0f93ad640 1 -- 172.21.15.133:0/2839010684 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0b2070 con 0x7ff0f40cee80 2024-04-25T10:14:32.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0f93ad640 1 -- 172.21.15.133:0/2839010684 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec0a5940 con 0x7ff0f40cee80 2024-04-25T10:14:32.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0f93ad640 1 -- 172.21.15.133:0/2839010684 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0a5d10 con 0x7ff0f40cee80 2024-04-25T10:14:32.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 -- 172.21.15.133:0/2839010684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f40cee80 msgr2=0x7ff0f406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 --2- 172.21.15.133:0/2839010684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f40cee80 0x7ff0f406d6f0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40cbfb0 tx=0x7ff0ec01c860 comp rx=0 tx=0).stop 2024-04-25T10:14:32.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 -- 172.21.15.133:0/2839010684 shutdown_connections 2024-04-25T10:14:32.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 --2- 172.21.15.133:0/2839010684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f40cee80 0x7ff0f406d6f0 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 --2- 172.21.15.133:0/2839010684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f415d9b0 0x7ff0f40cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 -- 172.21.15.133:0/2839010684 >> 172.21.15.133:0/2839010684 conn(0x7ff0f4067000 msgr2=0x7ff0f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 -- 172.21.15.133:0/2839010684 shutdown_connections 2024-04-25T10:14:32.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.611+0000 7ff0fc613640 1 -- 172.21.15.133:0/2839010684 wait complete. 2024-04-25T10:14:32.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fc613640 1 Processor -- start 2024-04-25T10:14:32.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fc613640 1 -- start start 2024-04-25T10:14:32.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fc613640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0f40cee80 0x7ff0f413ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fc613640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f415d9b0 0x7ff0f415ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fc613640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f415dd90 0x7ff0f415f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fc613640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f406e7d0 con 0x7ff0f415dd90 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fc613640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f406e650 con 0x7ff0f40cee80 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fc613640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f406e950 con 0x7ff0f415d9b0 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fabb0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f415dd90 0x7ff0f415f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fabb0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f415dd90 0x7ff0f415f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37438/0 (socket says 172.21.15.133:37438) 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fabb0640 1 -- 172.21.15.133:0/2191479479 learned_addr learned my addr 172.21.15.133:0/2191479479 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fabb0640 1 -- 172.21.15.133:0/2191479479 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f415d9b0 msgr2=0x7ff0f415ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fabb0640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f415d9b0 0x7ff0f415ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fabb0640 1 -- 172.21.15.133:0/2191479479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0f40cee80 msgr2=0x7ff0f413ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fabb0640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0f40cee80 0x7ff0f413ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fabb0640 1 -- 172.21.15.133:0/2191479479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f415f840 con 0x7ff0f415dd90 2024-04-25T10:14:32.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fabb0640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f415dd90 0x7ff0f415f230 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec01a4f0 tx=0x7ff0ec019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0eaffd640 1 -- 172.21.15.133:0/2191479479 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0bd020 con 0x7ff0f415dd90 2024-04-25T10:14:32.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0eaffd640 1 -- 172.21.15.133:0/2191479479 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec0ada70 con 0x7ff0f415dd90 2024-04-25T10:14:32.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fc613640 1 -- 172.21.15.133:0/2191479479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f415fb20 con 0x7ff0f415dd90 2024-04-25T10:14:32.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0eaffd640 1 -- 172.21.15.133:0/2191479479 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0b2040 con 0x7ff0f415dd90 2024-04-25T10:14:32.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fc613640 1 -- 172.21.15.133:0/2191479479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f41600e0 con 0x7ff0f415dd90 2024-04-25T10:14:32.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.615+0000 7ff0fc613640 1 -- 172.21.15.133:0/2191479479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f4068ef0 con 0x7ff0f415dd90 2024-04-25T10:14:32.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.619+0000 7ff0eaffd640 1 -- 172.21.15.133:0/2191479479 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0ec0c0020 con 0x7ff0f415dd90 2024-04-25T10:14:32.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.619+0000 7ff0eaffd640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d8066f90 0x7ff0d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.619+0000 7ff0eaffd640 1 -- 172.21.15.133:0/2191479479 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff0ec1205c0 con 0x7ff0f415dd90 2024-04-25T10:14:32.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.619+0000 7ff0fa3af640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d8066f90 0x7ff0d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.619+0000 7ff0fa3af640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d8066f90 0x7ff0d8069450 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009730 tx=0x7ff0f0006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.623+0000 7ff0eaffd640 1 -- 172.21.15.133:0/2191479479 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0ec016540 con 0x7ff0f415dd90 2024-04-25T10:14:32.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.807+0000 7ff0fc613640 1 -- 172.21.15.133:0/2191479479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0f406b520 con 0x7ff0f415dd90 2024-04-25T10:14:32.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0eaffd640 1 -- 172.21.15.133:0/2191479479 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4958 (secure 0 0 0) 0x7ff0ec0f14d0 con 0x7ff0f415dd90 2024-04-25T10:14:32.813 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:32.813 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3448301688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3448301688},{"type":"v1","addr":"172.21.15.162:6833","nonce":3448301688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/4188704231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":4188704231},{"type":"v1","addr":"172.21.15.162:6835","nonce":4188704231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":10041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2626366082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2626366082},{"type":"v1","addr":"172.21.15.133:6835","nonce":2626366082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:14:28.005823+0000","modified":"2024-04-25T10:14:30.040268+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10050":{"gid":10050,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3234096063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3234096063},{"type":"v1","addr":"172.21.15.133:6837","nonce":3234096063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:14:32.813 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 90 2024-04-25T10:14:32.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 -- 172.21.15.133:0/2191479479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d8066f90 msgr2=0x7ff0d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d8066f90 0x7ff0d8069450 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009730 tx=0x7ff0f0006ca0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 -- 172.21.15.133:0/2191479479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f415dd90 msgr2=0x7ff0f415f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f415dd90 0x7ff0f415f230 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec01a4f0 tx=0x7ff0ec019660 comp rx=0 tx=0).stop 2024-04-25T10:14:32.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 -- 172.21.15.133:0/2191479479 shutdown_connections 2024-04-25T10:14:32.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d8066f90 0x7ff0d8069450 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0f415dd90 0x7ff0f415f230 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0f415d9b0 0x7ff0f415ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 --2- 172.21.15.133:0/2191479479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0f40cee80 0x7ff0f413ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 -- 172.21.15.133:0/2191479479 >> 172.21.15.133:0/2191479479 conn(0x7ff0f4067000 msgr2=0x7ff0f40ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 -- 172.21.15.133:0/2191479479 shutdown_connections 2024-04-25T10:14:32.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:32.811+0000 7ff0fc613640 1 -- 172.21.15.133:0/2191479479 wait complete. 2024-04-25T10:14:33.232 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:14:28.005823+0000', 'modified': '2024-04-25T10:14:30.040268+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10050}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10050': {'gid': 10050, 'name': 'a', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/3234096063', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 3234096063}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3234096063}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:33.233 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:14:33.233 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:14:33.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4ccb6f2640 1 Processor -- start 2024-04-25T10:14:33.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4ccb6f2640 1 -- start start 2024-04-25T10:14:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4ccb6f2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc40b2150 0x7f4cc406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4ccb6f2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4cc415b9b0 0x7f4cc406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4ccb6f2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc415ac50 con 0x7f4cc4143ea0 2024-04-25T10:14:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4ccb6f2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc40ccf20 con 0x7f4cc415b9b0 2024-04-25T10:14:33.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4ccb6f2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc40cd180 con 0x7f4cc40b2150 2024-04-25T10:14:33.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4cc9c8f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4cc415b9b0 0x7f4cc406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4cc8c8d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc40b2150 0x7f4cc406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4cc9c8f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4cc415b9b0 0x7f4cc406e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55454/0 (socket says 172.21.15.133:55454) 2024-04-25T10:14:33.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4cc8c8d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc40b2150 0x7f4cc406e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:41042/0 (socket says 172.21.15.133:41042) 2024-04-25T10:14:33.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4cc948e640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f4cc4143ea0 0x7f4cc40cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:57356/0 (socket says 172.21.15.133:57356) 2024-04-25T10:14:33.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.359+0000 7f4cc9c8f640 1 -- 172.21.15.133:0/839339267 learned_addr learned my addr 172.21.15.133:0/839339267 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:33.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4cc8c8d640 1 -- 172.21.15.133:0/839339267 >> v2:172.21.15.162:3300/0 conn(0x7f4cc415b9b0 msgr2=0x7f4cc406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/839339267 >> v2:172.21.15.162:3300/0 conn(0x7f4cc415b9b0 0x7f4cc406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4cc8c8d640 1 -- 172.21.15.133:0/839339267 >> v1:172.21.15.162:6789/0 conn(0x7f4cc4143ea0 legacy=0x7f4cc40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4cc8c8d640 1 -- 172.21.15.133:0/839339267 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc415aed0 con 0x7f4cc40b2150 2024-04-25T10:14:33.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4cc9c8f640 1 --2- 172.21.15.133:0/839339267 >> v2:172.21.15.162:3300/0 conn(0x7f4cc415b9b0 0x7f4cc406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/839339267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc40b2150 0x7f4cc406e3b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40097c0 tx=0x7f4cb401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc24f02a4995f9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4cbbfff640 1 -- 172.21.15.133:0/839339267 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb40b1070 con 0x7f4cc40b2150 2024-04-25T10:14:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4cbbfff640 1 -- 172.21.15.133:0/839339267 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb40ac640 con 0x7f4cc40b2150 2024-04-25T10:14:33.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4cbbfff640 1 -- 172.21.15.133:0/839339267 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb40aca10 con 0x7f4cc40b2150 2024-04-25T10:14:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/839339267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc40b2150 msgr2=0x7f4cc406e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4ccb6f2640 1 --2- 172.21.15.133:0/839339267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc40b2150 0x7f4cc406e3b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40097c0 tx=0x7f4cb401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/839339267 shutdown_connections 2024-04-25T10:14:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4ccb6f2640 1 --2- 172.21.15.133:0/839339267 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc40b2150 0x7f4cc406e3b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4ccb6f2640 1 --2- 172.21.15.133:0/839339267 >> v2:172.21.15.162:3300/0 conn(0x7f4cc415b9b0 0x7f4cc406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/839339267 >> 172.21.15.133:0/839339267 conn(0x7f4cc4066f70 msgr2=0x7f4cc406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/839339267 shutdown_connections 2024-04-25T10:14:33.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.363+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/839339267 wait complete. 2024-04-25T10:14:33.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4ccb6f2640 1 Processor -- start 2024-04-25T10:14:33.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4ccb6f2640 1 -- start start 2024-04-25T10:14:33.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4ccb6f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc40b2150 0x7f4cc41567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4ccb6f2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc4143ea0 0x7f4cc413ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4ccb6f2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cc415b9b0 0x7f4cc4155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4ccb6f2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc40b1270 con 0x7f4cc40b2150 2024-04-25T10:14:33.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4ccb6f2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc40b10f0 con 0x7f4cc415b9b0 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4ccb6f2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc40b13f0 con 0x7f4cc4143ea0 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc948e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc40b2150 0x7f4cc41567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc948e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc40b2150 0x7f4cc41567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37446/0 (socket says 172.21.15.133:37446) 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc948e640 1 -- 172.21.15.133:0/12159173 learned_addr learned my addr 172.21.15.133:0/12159173 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc4143ea0 0x7f4cc413ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc9c8f640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cc415b9b0 0x7f4cc4155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc948e640 1 -- 172.21.15.133:0/12159173 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc4143ea0 msgr2=0x7f4cc413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc948e640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc4143ea0 0x7f4cc413ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc948e640 1 -- 172.21.15.133:0/12159173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cc415b9b0 msgr2=0x7f4cc4155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc948e640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cc415b9b0 0x7f4cc4155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc948e640 1 -- 172.21.15.133:0/12159173 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc4156db0 con 0x7f4cc40b2150 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc9c8f640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cc415b9b0 0x7f4cc4155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cc948e640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc40b2150 0x7f4cc41567d0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000c420 tx=0x7f4cc000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cb9ffb640 1 -- 172.21.15.133:0/12159173 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00074e0 con 0x7f4cc40b2150 2024-04-25T10:14:33.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cb9ffb640 1 -- 172.21.15.133:0/12159173 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc0007680 con 0x7f4cc40b2150 2024-04-25T10:14:33.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/12159173 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc4157090 con 0x7f4cc40b2150 2024-04-25T10:14:33.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4cb9ffb640 1 -- 172.21.15.133:0/12159173 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0004a30 con 0x7f4cc40b2150 2024-04-25T10:14:33.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.367+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/12159173 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc4157650 con 0x7f4cc40b2150 2024-04-25T10:14:33.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cb9ffb640 1 -- 172.21.15.133:0/12159173 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4cc001a020 con 0x7f4cc40b2150 2024-04-25T10:14:33.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cb9ffb640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca8066f60 0x7f4ca8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cb9ffb640 1 -- 172.21.15.133:0/12159173 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4cc0084a30 con 0x7f4cc40b2150 2024-04-25T10:14:33.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca8066f60 0x7f4ca8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca8066f60 0x7f4ca8069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40ae000 tx=0x7f4cb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cce85f000 1 Processor -- start 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cce85f000 1 -- start start 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 0x564886af6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886af7240 0x564886aac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886aad310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cce85f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648869945f0 con 0x564886aacf30 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cce85f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648868fbda0 con 0x564886af7240 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cce85f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564886884150 con 0x564886af6920 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc9c8f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886aad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc9c8f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886aad310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37460/0 (socket says 172.21.15.133:37460) 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc9c8f640 1 -- 172.21.15.133:0/1071776712 learned_addr learned my addr 172.21.15.133:0/1071776712 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/1071776712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886af7240 0x564886aac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc948e640 1 --2- 172.21.15.133:0/1071776712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 0x564886af6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc9c8f640 1 -- 172.21.15.133:0/1071776712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 msgr2=0x564886af6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc9c8f640 1 --2- 172.21.15.133:0/1071776712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 0x564886af6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc9c8f640 1 -- 172.21.15.133:0/1071776712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886af7240 msgr2=0x564886aac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc9c8f640 1 --2- 172.21.15.133:0/1071776712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886af7240 0x564886aac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.371+0000 7f4cc9c8f640 1 -- 172.21.15.133:0/1071776712 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56488689dca0 con 0x564886aacf30 2024-04-25T10:14:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cc9c8f640 1 --2- 172.21.15.133:0/1071776712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886aad310 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009d50 tx=0x7f4cbc009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d645c45423a5288f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4caeffd640 1 -- 172.21.15.133:0/1071776712 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc014610 con 0x564886aacf30 2024-04-25T10:14:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4caeffd640 1 -- 172.21.15.133:0/1071776712 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cbc019070 con 0x564886aacf30 2024-04-25T10:14:33.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4caeffd640 1 -- 172.21.15.133:0/1071776712 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0149f0 con 0x564886aacf30 2024-04-25T10:14:33.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- 172.21.15.133:0/1071776712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 msgr2=0x564886aad310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 --2- 172.21.15.133:0/1071776712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886aad310 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009d50 tx=0x7f4cbc009810 comp rx=0 tx=0).stop 2024-04-25T10:14:33.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- 172.21.15.133:0/1071776712 shutdown_connections 2024-04-25T10:14:33.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 --2- 172.21.15.133:0/1071776712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886aad310 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 --2- 172.21.15.133:0/1071776712 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886af7240 0x564886aac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 --2- 172.21.15.133:0/1071776712 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 0x564886af6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- 172.21.15.133:0/1071776712 >> 172.21.15.133:0/1071776712 conn(0x564886ab32a0 msgr2=0x564886aab2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- 172.21.15.133:0/1071776712 shutdown_connections 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- 172.21.15.133:0/1071776712 wait complete. 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- start start 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 Processor -- start 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886afddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 0x564886afe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886af7240 0x564886ab0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648868b6990 con 0x564886aacf30 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648868f7e40 con 0x564886af7240 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564886887900 con 0x564886af6920 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 10 client.0 _ll_register_callbacks cb 0x5648869af1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:33.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cc948e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886afddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cc948e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886afddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37474/0 (socket says 172.21.15.133:37474) 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cc948e640 1 -- 172.21.15.133:0/4276596521 learned_addr learned my addr 172.21.15.133:0/4276596521 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cc948e640 1 -- 172.21.15.133:0/4276596521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 msgr2=0x564886afe310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cc948e640 1 --2- 172.21.15.133:0/4276596521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 0x564886afe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cc948e640 1 -- 172.21.15.133:0/4276596521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886af7240 msgr2=0x564886ab0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cc948e640 1 --2- 172.21.15.133:0/4276596521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886af7240 0x564886ab0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cc948e640 1 -- 172.21.15.133:0/4276596521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648868b4630 con 0x564886aacf30 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cc948e640 1 --2- 172.21.15.133:0/4276596521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886afddd0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0082db0 tx=0x7f4cc00552d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67957b23ef3ab6e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4ca3fff640 1 -- 172.21.15.133:0/4276596521 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0081060 con 0x564886aacf30 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4ca3fff640 1 -- 172.21.15.133:0/4276596521 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4cc0082a40 con 0x564886aacf30 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- 172.21.15.133:0/4276596521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cbc019070 con 0x564886aacf30 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4ca3fff640 1 -- 172.21.15.133:0/4276596521 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cc0071050 con 0x564886aacf30 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 10 client.10191 fetch_fsmap learned FSMap version 90 2024-04-25T10:14:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- 172.21.15.133:0/4276596521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5648868843a0 con 0x564886aacf30 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4ca3fff640 1 -- 172.21.15.133:0/4276596521 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cbc019070 con 0x564886aacf30 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4ca3fff640 1 -- 172.21.15.133:0/4276596521 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4cc00553f0 con 0x564886aacf30 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 10 client.10191 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 10 client.10191 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 20 client.10191 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 --2- 172.21.15.133:0/4276596521 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x564886ab14b0 0x564886ab1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 4 client.10191 mds_command: new command op to 10050 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cce85f000 1 -- 172.21.15.133:0/4276596521 --> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564886897a50 con 0x564886ab14b0 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.375+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/4276596521 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x564886ab14b0 0x564886ab1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4ca3fff640 10 client.10191 ms_handle_connect on v2:172.21.15.133:6836/3234096063 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/4276596521 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x564886ab14b0 0x564886ab1890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c16046ca2a1cf71 server_cookie=bdf5809789c7c3fb in_seq=0 out_seq=0 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4ca3fff640 1 -- 172.21.15.133:0/4276596521 <== mds.0 v2:172.21.15.133:6836/3234096063 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564886897a50 con 0x564886ab14b0 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4ca3fff640 10 client.10191 handle_command_reply: tid=0 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 1 client.10191 shutdown 2024-04-25T10:14:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 1 -- 172.21.15.133:0/4276596521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 msgr2=0x564886afddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 1 --2- 172.21.15.133:0/4276596521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886afddd0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0082db0 tx=0x7f4cc00552d0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 1 -- 172.21.15.133:0/4276596521 shutdown_connections 2024-04-25T10:14:33.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 1 --2- 172.21.15.133:0/4276596521 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x564886ab14b0 0x564886ab1890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 1 --2- 172.21.15.133:0/4276596521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886af7240 0x564886ab0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 1 --2- 172.21.15.133:0/4276596521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 0x564886afe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4ca3fff640 0 client.10191 ms_handle_reset on v2:172.21.15.133:6836/3234096063 2024-04-25T10:14:33.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 1 --2- 172.21.15.133:0/4276596521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886aacf30 0x564886afddd0 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 1 -- 172.21.15.133:0/4276596521 >> 172.21.15.133:0/4276596521 conn(0x564886ab32a0 msgr2=0x564886aab9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 1 -- 172.21.15.133:0/4276596521 shutdown_connections 2024-04-25T10:14:33.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 1 -- 172.21.15.133:0/4276596521 wait complete. 2024-04-25T10:14:33.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.379+0000 7f4cce85f000 20 client.10191 trim_cache size 0 max 16384 2024-04-25T10:14:33.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 Processor -- start 2024-04-25T10:14:33.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 -- start start 2024-04-25T10:14:33.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886aacf30 0x564886ab0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af6920 0x564886ab0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af7240 0x564886b008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc4200640 con 0x564886af6920 2024-04-25T10:14:33.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc4140830 con 0x564886aacf30 2024-04-25T10:14:33.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc4200070 con 0x564886af7240 2024-04-25T10:14:33.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cc8c8d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af6920 0x564886ab0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cc8c8d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af6920 0x564886ab0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37490/0 (socket says 172.21.15.133:37490) 2024-04-25T10:14:33.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cc8c8d640 1 -- 172.21.15.133:0/3896143835 learned_addr learned my addr 172.21.15.133:0/3896143835 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cc8c8d640 1 -- 172.21.15.133:0/3896143835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af7240 msgr2=0x564886b008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/3896143835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af7240 0x564886b008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cc8c8d640 1 -- 172.21.15.133:0/3896143835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886aacf30 msgr2=0x564886ab0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/3896143835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886aacf30 0x564886ab0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cc8c8d640 1 -- 172.21.15.133:0/3896143835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56488697d600 con 0x564886af6920 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/3896143835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af6920 0x564886ab0870 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40af040 tx=0x7f4cb4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b52a9ca9301f5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cae7fc640 1 -- 172.21.15.133:0/3896143835 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb40b1040 con 0x564886af6920 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cae7fc640 1 -- 172.21.15.133:0/3896143835 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb40ca070 con 0x564886af6920 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cae7fc640 1 -- 172.21.15.133:0/3896143835 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb40c4630 con 0x564886af6920 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 -- 172.21.15.133:0/3896143835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af6920 msgr2=0x564886ab0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 --2- 172.21.15.133:0/3896143835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af6920 0x564886ab0870 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40af040 tx=0x7f4cb4004740 comp rx=0 tx=0).stop 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 -- 172.21.15.133:0/3896143835 shutdown_connections 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 --2- 172.21.15.133:0/3896143835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af7240 0x564886b008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 --2- 172.21.15.133:0/3896143835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af6920 0x564886ab0870 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 --2- 172.21.15.133:0/3896143835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886aacf30 0x564886ab0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 -- 172.21.15.133:0/3896143835 >> 172.21.15.133:0/3896143835 conn(0x564886ab32a0 msgr2=0x564886ae3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 -- 172.21.15.133:0/3896143835 shutdown_connections 2024-04-25T10:14:33.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 -- 172.21.15.133:0/3896143835 wait complete. 2024-04-25T10:14:33.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 -- start start 2024-04-25T10:14:33.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.395+0000 7f4cce85f000 1 Processor -- start 2024-04-25T10:14:33.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886aacf30 0x564886b03ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 0x564886b124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af7240 0x564886b12a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648869003d0 con 0x564886af7240 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648869b7d50 con 0x564886aacf30 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564886896980 con 0x564886af6920 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 10 client.0 _ll_register_callbacks cb 0x7f4cbc0090f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc9c8f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af7240 0x564886b12a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc9c8f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af7240 0x564886b12a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37498/0 (socket says 172.21.15.133:37498) 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc9c8f640 1 -- 172.21.15.133:0/445203761 learned_addr learned my addr 172.21.15.133:0/445203761 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc948e640 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886aacf30 0x564886b03ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc8c8d640 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 0x564886b124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc9c8f640 1 -- 172.21.15.133:0/445203761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 msgr2=0x564886b124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc9c8f640 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 0x564886b124f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc9c8f640 1 -- 172.21.15.133:0/445203761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886aacf30 msgr2=0x564886b03ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc9c8f640 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886aacf30 0x564886b03ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc9c8f640 1 -- 172.21.15.133:0/445203761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648868909b0 con 0x564886af7240 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc948e640 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886aacf30 0x564886b03ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc9c8f640 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af7240 0x564886b12a30 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009d20 tx=0x7f4cbc014d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b5ebfa14598859 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ca3fff640 1 -- 172.21.15.133:0/445203761 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc022070 con 0x564886af7240 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ca3fff640 1 -- 172.21.15.133:0/445203761 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4cbc027660 con 0x564886af7240 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 -- 172.21.15.133:0/445203761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cb40ca070 con 0x564886af7240 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ca3fff640 1 -- 172.21.15.133:0/445203761 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cbc0278a0 con 0x564886af7240 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 10 client.10197 fetch_fsmap learned FSMap version 90 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 -- 172.21.15.133:0/445203761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5648868a30b0 con 0x564886af7240 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ca3fff640 1 -- 172.21.15.133:0/445203761 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb40ca070 con 0x564886af7240 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ca3fff640 1 -- 172.21.15.133:0/445203761 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4cbc027a40 con 0x564886af7240 2024-04-25T10:14:33.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 10 client.10197 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 10 client.10197 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 20 client.10197 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x564886b12f70 0x564886b57460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 4 client.10197 mds_command: new command op to 10050 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 -- 172.21.15.133:0/445203761 --> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5648869092e0 con 0x564886b12f70 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc948e640 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x564886b12f70 0x564886b57460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cc948e640 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x564886b12f70 0x564886b57460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d819669177c55 server_cookie=768f8a3039368561 in_seq=0 out_seq=0 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ca3fff640 10 client.10197 ms_handle_connect on v2:172.21.15.133:6836/3234096063 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ca3fff640 1 -- 172.21.15.133:0/445203761 <== mds.0 v2:172.21.15.133:6836/3234096063 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5648869092e0 con 0x564886b12f70 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ca3fff640 10 client.10197 handle_command_reply: tid=0 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 client.10197 shutdown 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 -- 172.21.15.133:0/445203761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af7240 msgr2=0x564886b12a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af7240 0x564886b12a30 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009d20 tx=0x7f4cbc014d00 comp rx=0 tx=0).stop 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 -- 172.21.15.133:0/445203761 shutdown_connections 2024-04-25T10:14:33.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564886af7240 0x564886b12a30 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564886af6920 0x564886b124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x564886b12f70 0x564886b57460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 --2- 172.21.15.133:0/445203761 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564886aacf30 0x564886b03ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ca3fff640 0 client.10197 ms_handle_reset on v2:172.21.15.133:6836/3234096063 2024-04-25T10:14:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 -- 172.21.15.133:0/445203761 >> 172.21.15.133:0/445203761 conn(0x564886ab32a0 msgr2=0x564886aaf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 -- 172.21.15.133:0/445203761 shutdown_connections 2024-04-25T10:14:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 1 -- 172.21.15.133:0/445203761 wait complete. 2024-04-25T10:14:33.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4cce85f000 20 client.10197 trim_cache size 0 max 16384 2024-04-25T10:14:33.409 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:14:33.409 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:14:33.409 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:14:33.410 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 10050, 2024-04-25T10:14:33.410 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:14:33.410 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:14:33.410 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:14:33.410 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 4.7322081620000001, 2024-04-25T10:14:33.410 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 90, 2024-04-25T10:14:33.410 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 143, 2024-04-25T10:14:33.410 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 143, 2024-04-25T10:14:33.410 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 9.8132524050000001 2024-04-25T10:14:33.410 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:14:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/12159173 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca8066f60 msgr2=0x7f4ca8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca8066f60 0x7f4ca8069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40ae000 tx=0x7f4cb4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/12159173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc40b2150 msgr2=0x7f4cc41567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc40b2150 0x7f4cc41567d0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000c420 tx=0x7f4cc000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/12159173 shutdown_connections 2024-04-25T10:14:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cc415b9b0 0x7f4cc4155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cc4143ea0 0x7f4cc413ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca8066f60 0x7f4ca8069420 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 --2- 172.21.15.133:0/12159173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc40b2150 0x7f4cc41567d0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/12159173 >> 172.21.15.133:0/12159173 conn(0x7f4cc4066f70 msgr2=0x7f4cc406db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/12159173 shutdown_connections 2024-04-25T10:14:33.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.399+0000 7f4ccb6f2640 1 -- 172.21.15.133:0/12159173 wait complete. 2024-04-25T10:14:33.419 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:14:33.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.515+0000 7efee1dbe640 1 Processor -- start 2024-04-25T10:14:33.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.515+0000 7efee1dbe640 1 -- start start 2024-04-25T10:14:33.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efee1dbe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efedc15c9a0 0x7efedc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efee1dbe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efedc0b1f70 0x7efedc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efee1dbe640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efedc0ce330 0x7efedc0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efee1dbe640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc0cece0 con 0x7efedc15c9a0 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efee1dbe640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc0c61d0 con 0x7efedc0ce330 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efee1dbe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc0c64b0 con 0x7efedc0b1f70 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efedc15c9a0 0x7efedc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efedc15c9a0 0x7efedc0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37506/0 (socket says 172.21.15.133:37506) 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedb7fe640 1 -- 172.21.15.133:0/3885320481 learned_addr learned my addr 172.21.15.133:0/3885320481 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedb7fe640 1 -- 172.21.15.133:0/3885320481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efedc0b1f70 msgr2=0x7efedc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efed3fff640 1 --2- 172.21.15.133:0/3885320481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efedc0b1f70 0x7efedc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedbfff640 1 --2- 172.21.15.133:0/3885320481 >> v2:172.21.15.162:3300/0 conn(0x7efedc0ce330 0x7efedc0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedb7fe640 1 --2- 172.21.15.133:0/3885320481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efedc0b1f70 0x7efedc06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedb7fe640 1 -- 172.21.15.133:0/3885320481 >> v2:172.21.15.162:3300/0 conn(0x7efedc0ce330 msgr2=0x7efedc0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedb7fe640 1 --2- 172.21.15.133:0/3885320481 >> v2:172.21.15.162:3300/0 conn(0x7efedc0ce330 0x7efedc0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedb7fe640 1 -- 172.21.15.133:0/3885320481 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc06d4f0 con 0x7efedc15c9a0 2024-04-25T10:14:33.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedb7fe640 1 --2- 172.21.15.133:0/3885320481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efedc15c9a0 0x7efedc0c5980 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7efedc0ceef0 tx=0x7efec401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1a2fc5a407f90b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedaffd640 1 -- 172.21.15.133:0/3885320481 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec40b1070 con 0x7efedc15c9a0 2024-04-25T10:14:33.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedaffd640 1 -- 172.21.15.133:0/3885320481 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec40ac620 con 0x7efedc15c9a0 2024-04-25T10:14:33.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.519+0000 7efedaffd640 1 -- 172.21.15.133:0/3885320481 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec40ac9f0 con 0x7efedc15c9a0 2024-04-25T10:14:33.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- 172.21.15.133:0/3885320481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efedc15c9a0 msgr2=0x7efedc0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 --2- 172.21.15.133:0/3885320481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efedc15c9a0 0x7efedc0c5980 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7efedc0ceef0 tx=0x7efec401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- 172.21.15.133:0/3885320481 shutdown_connections 2024-04-25T10:14:33.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 --2- 172.21.15.133:0/3885320481 >> v2:172.21.15.162:3300/0 conn(0x7efedc0ce330 0x7efedc0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 --2- 172.21.15.133:0/3885320481 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efedc0b1f70 0x7efedc06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 --2- 172.21.15.133:0/3885320481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efedc15c9a0 0x7efedc0c5980 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- 172.21.15.133:0/3885320481 >> 172.21.15.133:0/3885320481 conn(0x7efedc067000 msgr2=0x7efedc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- 172.21.15.133:0/3885320481 shutdown_connections 2024-04-25T10:14:33.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- 172.21.15.133:0/3885320481 wait complete. 2024-04-25T10:14:33.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 Processor -- start 2024-04-25T10:14:33.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- start start 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efedc0b1f70 0x7efedc0b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efedc0ce330 0x7efedc133e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efedc15c9a0 0x7efedc1343b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc15cf00 con 0x7efedc0ce330 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc15cd80 con 0x7efedc0b1f70 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc15d950 con 0x7efedc15c9a0 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efedc0ce330 0x7efedc133e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efedc0ce330 0x7efedc133e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37518/0 (socket says 172.21.15.133:37518) 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed3fff640 1 -- 172.21.15.133:0/3313323846 learned_addr learned my addr 172.21.15.133:0/3313323846 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efedb7fe640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efedc0b1f70 0x7efedc0b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efedbfff640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efedc15c9a0 0x7efedc1343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed3fff640 1 -- 172.21.15.133:0/3313323846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efedc15c9a0 msgr2=0x7efedc1343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed3fff640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efedc15c9a0 0x7efedc1343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed3fff640 1 -- 172.21.15.133:0/3313323846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efedc0b1f70 msgr2=0x7efedc0b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed3fff640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efedc0b1f70 0x7efedc0b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed3fff640 1 -- 172.21.15.133:0/3313323846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc145bc0 con 0x7efedc0ce330 2024-04-25T10:14:33.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efedbfff640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efedc15c9a0 0x7efedc1343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:33.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed3fff640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efedc0ce330 0x7efedc133e70 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7efecc00c7f0 tx=0x7efecc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed8ff9640 1 -- 172.21.15.133:0/3313323846 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc015070 con 0x7efedc0ce330 2024-04-25T10:14:33.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed8ff9640 1 -- 172.21.15.133:0/3313323846 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc004220 con 0x7efedc0ce330 2024-04-25T10:14:33.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- 172.21.15.133:0/3313323846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc145e60 con 0x7efedc0ce330 2024-04-25T10:14:33.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efed8ff9640 1 -- 172.21.15.133:0/3313323846 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc003620 con 0x7efedc0ce330 2024-04-25T10:14:33.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- 172.21.15.133:0/3313323846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc146420 con 0x7efedc0ce330 2024-04-25T10:14:33.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.523+0000 7efee1dbe640 1 -- 172.21.15.133:0/3313323846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efedc06bc20 con 0x7efedc0ce330 2024-04-25T10:14:33.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.527+0000 7efed8ff9640 1 -- 172.21.15.133:0/3313323846 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efecc0039d0 con 0x7efedc0ce330 2024-04-25T10:14:33.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.527+0000 7efed8ff9640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efebc066f60 0x7efebc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.527+0000 7efed8ff9640 1 -- 172.21.15.133:0/3313323846 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efecc0851a0 con 0x7efedc0ce330 2024-04-25T10:14:33.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.527+0000 7efedb7fe640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efebc066f60 0x7efebc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.527+0000 7efedb7fe640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efebc066f60 0x7efebc069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7efec40ae000 tx=0x7efec40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.531+0000 7efed8ff9640 1 -- 172.21.15.133:0/3313323846 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efecc0559d0 con 0x7efedc0ce330 2024-04-25T10:14:33.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.723+0000 7efee1dbe640 1 -- 172.21.15.133:0/3313323846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efedc0c6c10 con 0x7efedc0ce330 2024-04-25T10:14:33.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.723+0000 7efed8ff9640 1 -- 172.21.15.133:0/3313323846 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7efecc059a70 con 0x7efedc0ce330 2024-04-25T10:14:33.727 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:14:33.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 -- 172.21.15.133:0/3313323846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efebc066f60 msgr2=0x7efebc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efebc066f60 0x7efebc069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7efec40ae000 tx=0x7efec40a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:33.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 -- 172.21.15.133:0/3313323846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efedc0ce330 msgr2=0x7efedc133e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efedc0ce330 0x7efedc133e70 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7efecc00c7f0 tx=0x7efecc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 -- 172.21.15.133:0/3313323846 shutdown_connections 2024-04-25T10:14:33.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efedc15c9a0 0x7efedc1343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efebc066f60 0x7efebc069420 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efedc0ce330 0x7efedc133e70 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 --2- 172.21.15.133:0/3313323846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efedc0b1f70 0x7efedc0b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 -- 172.21.15.133:0/3313323846 >> 172.21.15.133:0/3313323846 conn(0x7efedc067000 msgr2=0x7efedc155f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 -- 172.21.15.133:0/3313323846 shutdown_connections 2024-04-25T10:14:33.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:33.727+0000 7efee1dbe640 1 -- 172.21.15.133:0/3313323846 wait complete. 2024-04-25T10:14:33.744 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:14:33.745 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:14:33.745 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:14:33.745 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:14:33.745 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:33.745 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:14:33.745 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:14:33.745 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:14:33.745 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:33.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:33.746 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:14:33.751 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:14:33.751 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:14:33.751 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41758sec preferred_lft 41758sec 2024-04-25T10:14:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:14:33.753 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:33.754 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:33.754 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:33.754 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:14:33.754 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:33.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:33.754 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:14:33.800 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:14:33.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:33.803 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:14:33.852 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:14:33.852 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:14:33.852 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:14:33.852 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:14:33.852 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:14:33.853 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:14:33.853 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:14:33.853 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:14:33.853 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:14:33.853 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:14:33.853 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:14:33.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:33.853 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:14:33.853 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:33.853 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:14:33.853 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:14:33.853 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:14:33.854 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:33.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:33.929 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:14:33.936 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-04-25T10:14:33.935+0000 7fd000d31640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:33.942 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:14:33.942 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:14:33.942 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:33.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:33.942 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:14:33.996 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:14:33.997 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:33.997 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:34.044 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:34.045 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:34.045 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:34.092 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:14:34.150 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:14:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:14:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:14:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:14:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:14:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:34.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:34.201 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:14:34.249 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:14:34.249 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:14:34.294 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:14:34.308 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:34.357 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:14:34.357 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:14:34.357 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:14:34.357 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:14:34.357 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:14:34.357 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:14:34.358 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:34.359 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:34.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:34.360 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:14:34.374 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:14:34.371+0000 7f74cacbd580 -1 init, newargv = 0x55aa0d549ef0 newargc=15 2024-04-25T10:14:34.374 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[84760]: starting ceph client 2024-04-25T10:14:34.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[84760]: starting fuse 2024-04-25T10:14:35.481 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:14:35.498 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:35.546 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:14:35.546 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:14:35.546 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:14:35.546 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:14:35.546 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:14:35.546 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:14:35.546 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:14:35.546 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:14:35.546 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:14:35.546 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:14:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:14:35.548 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:35.548 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:35.548 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:35.548 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:35.548 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:14:35.548 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:35.548 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:35.548 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:35.548 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:35.548 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:35.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:35.549 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:14:35.597 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:14:35.597 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:14:35.597 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:14:35.597 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:14:35.597 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:14:35.597 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:14:35.598 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:14:35.598 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:14:35.598 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:14:35.598 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:14:35.598 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:14:35.598 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:14:35.598 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:14:35.598 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:14:35.598 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:14:35.598 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:14:35.598 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:14:35.601 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:14:35.601 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:14:35.602 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:14:35.602 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:14:35.602 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:14:35.602 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:14:35.602 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:14:35.602 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:14:35.602 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:14:35.602 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:14:35.602 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:14:35.602 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:14:35.722 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.84760.asok 2024-04-25T10:14:35.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:35.735 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84760.asok 2024-04-25T10:14:35.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:35.735 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84760.asok status 2024-04-25T10:14:35.889 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:14:35.889 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:14:35.889 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:14:35.889 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:14:35.889 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:14:35.889 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:14:35.889 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:14:35.889 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "84760", 2024-04-25T10:14:35.889 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 10212, 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 10212 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2147721144 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2147721144 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.10212 192.168.0.1:0/2147721144", 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/2147721144", 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 90, 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 144, 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:14:35.891 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:14:35.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:35.904 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:14:35.904 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:35.910 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:14:35.911 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:35.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:35.911 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:35.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:35.966 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:35.966 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:35.966 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:14:35.966 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:35.966 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:35.966 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:35.967 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:36.047 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.047 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.048 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124796 s, 84.0 MB/s 2024-04-25T10:14:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112987 s, 92.8 MB/s 2024-04-25T10:14:36.081 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.081 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.082 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160712 s, 65.2 MB/s 2024-04-25T10:14:36.095 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.095 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.095 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105194 s, 99.7 MB/s 2024-04-25T10:14:36.113 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.113 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.113 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015069 s, 69.6 MB/s 2024-04-25T10:14:36.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.129 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121942 s, 86.0 MB/s 2024-04-25T10:14:36.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.147 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153631 s, 68.3 MB/s 2024-04-25T10:14:36.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.166 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.166 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155719 s, 67.3 MB/s 2024-04-25T10:14:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.184 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141251 s, 74.2 MB/s 2024-04-25T10:14:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.200 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123702 s, 84.8 MB/s 2024-04-25T10:14:36.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.215 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127871 s, 82.0 MB/s 2024-04-25T10:14:36.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.231 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128844 s, 81.4 MB/s 2024-04-25T10:14:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.245 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100684 s, 104 MB/s 2024-04-25T10:14:36.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.261 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134932 s, 77.7 MB/s 2024-04-25T10:14:36.277 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.278 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.278 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131357 s, 79.8 MB/s 2024-04-25T10:14:36.293 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.294 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.294 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129265 s, 81.1 MB/s 2024-04-25T10:14:36.310 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.310 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.310 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129934 s, 80.7 MB/s 2024-04-25T10:14:36.325 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.325 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.325 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118819 s, 88.2 MB/s 2024-04-25T10:14:36.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.340 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.340 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122518 s, 85.6 MB/s 2024-04-25T10:14:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.353 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104116 s, 101 MB/s 2024-04-25T10:14:36.368 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.368 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.368 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012274 s, 85.4 MB/s 2024-04-25T10:14:36.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.384 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129995 s, 80.7 MB/s 2024-04-25T10:14:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143448 s, 73.1 MB/s 2024-04-25T10:14:36.417 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.417 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.417 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130387 s, 80.4 MB/s 2024-04-25T10:14:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131522 s, 79.7 MB/s 2024-04-25T10:14:36.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.449 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013412 s, 78.2 MB/s 2024-04-25T10:14:36.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.465 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129477 s, 81.0 MB/s 2024-04-25T10:14:36.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.480 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121847 s, 86.1 MB/s 2024-04-25T10:14:36.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.494 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107064 s, 97.9 MB/s 2024-04-25T10:14:36.509 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.509 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.509 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128743 s, 81.4 MB/s 2024-04-25T10:14:36.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.526 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.526 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135608 s, 77.3 MB/s 2024-04-25T10:14:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126929 s, 82.6 MB/s 2024-04-25T10:14:36.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:36.542 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:36.542 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:36.542 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:14:36.543 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:36.543 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:36.543 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:36.543 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:36.609 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.609 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.609 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128309 s, 81.7 MB/s 2024-04-25T10:14:36.624 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.625 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127488 s, 82.2 MB/s 2024-04-25T10:14:36.639 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.639 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.639 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117556 s, 89.2 MB/s 2024-04-25T10:14:36.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.655 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.655 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130249 s, 80.5 MB/s 2024-04-25T10:14:36.670 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.671 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.671 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132366 s, 79.2 MB/s 2024-04-25T10:14:36.685 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.686 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125338 s, 83.7 MB/s 2024-04-25T10:14:36.701 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.701 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.701 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012706 s, 82.5 MB/s 2024-04-25T10:14:36.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.716 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.716 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011926 s, 87.9 MB/s 2024-04-25T10:14:36.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.731 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119403 s, 87.8 MB/s 2024-04-25T10:14:36.746 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.746 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.746 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132978 s, 78.9 MB/s 2024-04-25T10:14:36.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153502 s, 68.3 MB/s 2024-04-25T10:14:36.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.779 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120777 s, 86.8 MB/s 2024-04-25T10:14:36.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.793 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.793 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107316 s, 97.7 MB/s 2024-04-25T10:14:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.808 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131954 s, 79.5 MB/s 2024-04-25T10:14:36.823 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.823 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.823 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121563 s, 86.3 MB/s 2024-04-25T10:14:36.837 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.837 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.837 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111724 s, 93.9 MB/s 2024-04-25T10:14:36.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.853 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127197 s, 82.4 MB/s 2024-04-25T10:14:36.869 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.869 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.869 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125387 s, 83.6 MB/s 2024-04-25T10:14:36.882 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.883 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.883 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111819 s, 93.8 MB/s 2024-04-25T10:14:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127808 s, 82.0 MB/s 2024-04-25T10:14:36.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.913 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128489 s, 81.6 MB/s 2024-04-25T10:14:36.926 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.926 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.926 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929935 s, 113 MB/s 2024-04-25T10:14:36.941 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.941 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.941 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126165 s, 83.1 MB/s 2024-04-25T10:14:36.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.957 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131082 s, 80.0 MB/s 2024-04-25T10:14:36.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.974 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.974 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131784 s, 79.6 MB/s 2024-04-25T10:14:36.988 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:36.988 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:36.988 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116526 s, 90.0 MB/s 2024-04-25T10:14:37.001 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.001 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.001 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104612 s, 100 MB/s 2024-04-25T10:14:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012705 s, 82.5 MB/s 2024-04-25T10:14:37.028 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.029 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.029 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949805 s, 110 MB/s 2024-04-25T10:14:37.042 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.042 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.042 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107102 s, 97.9 MB/s 2024-04-25T10:14:37.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.058 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.058 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127742 s, 82.1 MB/s 2024-04-25T10:14:37.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.071 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011154 s, 94.0 MB/s 2024-04-25T10:14:37.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:37.073 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:37.073 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:37.073 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:14:37.073 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:37.073 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:37.073 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:37.073 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:37.142 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.142 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.142 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135521 s, 77.4 MB/s 2024-04-25T10:14:37.155 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.155 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.155 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990566 s, 106 MB/s 2024-04-25T10:14:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111815 s, 93.8 MB/s 2024-04-25T10:14:37.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.185 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013058 s, 80.3 MB/s 2024-04-25T10:14:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00990291 s, 106 MB/s 2024-04-25T10:14:37.213 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.213 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.213 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120242 s, 87.2 MB/s 2024-04-25T10:14:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.227 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120724 s, 86.9 MB/s 2024-04-25T10:14:37.242 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.242 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.242 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011868 s, 88.4 MB/s 2024-04-25T10:14:37.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.257 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125145 s, 83.8 MB/s 2024-04-25T10:14:37.269 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.269 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.270 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957602 s, 110 MB/s 2024-04-25T10:14:37.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.284 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120909 s, 86.7 MB/s 2024-04-25T10:14:37.298 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.298 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.298 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011362 s, 92.3 MB/s 2024-04-25T10:14:37.314 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.314 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.314 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122524 s, 85.6 MB/s 2024-04-25T10:14:37.328 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.328 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.329 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122001 s, 85.9 MB/s 2024-04-25T10:14:37.343 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.343 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.343 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121094 s, 86.6 MB/s 2024-04-25T10:14:37.358 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.358 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.358 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119833 s, 87.5 MB/s 2024-04-25T10:14:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.373 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113942 s, 92.0 MB/s 2024-04-25T10:14:37.388 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.388 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.388 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120856 s, 86.8 MB/s 2024-04-25T10:14:37.404 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.404 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.404 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012471 s, 84.1 MB/s 2024-04-25T10:14:37.420 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.420 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.420 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127581 s, 82.2 MB/s 2024-04-25T10:14:37.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.435 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118078 s, 88.8 MB/s 2024-04-25T10:14:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.450 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121277 s, 86.5 MB/s 2024-04-25T10:14:37.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.467 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.468 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146794 s, 71.4 MB/s 2024-04-25T10:14:37.483 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.484 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134075 s, 78.2 MB/s 2024-04-25T10:14:37.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.498 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.498 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117706 s, 89.1 MB/s 2024-04-25T10:14:37.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.513 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117367 s, 89.3 MB/s 2024-04-25T10:14:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136166 s, 77.0 MB/s 2024-04-25T10:14:37.544 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.545 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126596 s, 82.8 MB/s 2024-04-25T10:14:37.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125847 s, 83.3 MB/s 2024-04-25T10:14:37.573 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.573 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.573 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983284 s, 107 MB/s 2024-04-25T10:14:37.589 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.589 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.589 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131392 s, 79.8 MB/s 2024-04-25T10:14:37.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125708 s, 83.4 MB/s 2024-04-25T10:14:37.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:37.605 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:37.606 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:37.606 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:14:37.606 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:37.606 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:37.606 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:37.606 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:37.676 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.677 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138655 s, 75.6 MB/s 2024-04-25T10:14:37.690 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.690 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.690 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109085 s, 96.1 MB/s 2024-04-25T10:14:37.706 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.706 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.706 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125866 s, 83.3 MB/s 2024-04-25T10:14:37.721 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.721 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.721 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012588 s, 83.3 MB/s 2024-04-25T10:14:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.736 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129333 s, 81.1 MB/s 2024-04-25T10:14:37.753 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.753 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.753 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129185 s, 81.2 MB/s 2024-04-25T10:14:37.767 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.768 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124373 s, 84.3 MB/s 2024-04-25T10:14:37.783 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.783 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.783 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129441 s, 81.0 MB/s 2024-04-25T10:14:37.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.799 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129702 s, 80.8 MB/s 2024-04-25T10:14:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123155 s, 85.1 MB/s 2024-04-25T10:14:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127899 s, 82.0 MB/s 2024-04-25T10:14:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.841 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980584 s, 107 MB/s 2024-04-25T10:14:37.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.853 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925597 s, 113 MB/s 2024-04-25T10:14:37.865 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.865 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.865 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984357 s, 107 MB/s 2024-04-25T10:14:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.880 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124404 s, 84.3 MB/s 2024-04-25T10:14:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014639 s, 71.6 MB/s 2024-04-25T10:14:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.911 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104471 s, 100 MB/s 2024-04-25T10:14:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.927 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126104 s, 83.2 MB/s 2024-04-25T10:14:37.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.942 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130906 s, 80.1 MB/s 2024-04-25T10:14:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120645 s, 86.9 MB/s 2024-04-25T10:14:37.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.970 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113173 s, 92.7 MB/s 2024-04-25T10:14:37.987 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:37.987 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:37.987 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143203 s, 73.2 MB/s 2024-04-25T10:14:38.002 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.002 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.002 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127259 s, 82.4 MB/s 2024-04-25T10:14:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121084 s, 86.6 MB/s 2024-04-25T10:14:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115997 s, 90.4 MB/s 2024-04-25T10:14:38.042 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.042 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.043 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883616 s, 119 MB/s 2024-04-25T10:14:38.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.059 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.059 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136206 s, 77.0 MB/s 2024-04-25T10:14:38.072 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.072 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.072 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106941 s, 98.1 MB/s 2024-04-25T10:14:38.088 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.088 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.088 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132133 s, 79.4 MB/s 2024-04-25T10:14:38.104 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.104 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.104 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129258 s, 81.1 MB/s 2024-04-25T10:14:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.117 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984136 s, 107 MB/s 2024-04-25T10:14:38.130 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.131 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111873 s, 93.7 MB/s 2024-04-25T10:14:38.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:38.132 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:38.132 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:38.132 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:14:38.132 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:38.132 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:38.133 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:38.133 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:38.197 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.198 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.198 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111474 s, 94.1 MB/s 2024-04-25T10:14:38.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.211 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.211 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108028 s, 97.1 MB/s 2024-04-25T10:14:38.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.224 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.225 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012015 s, 87.3 MB/s 2024-04-25T10:14:38.233 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.234 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.234 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074422 s, 141 MB/s 2024-04-25T10:14:38.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.248 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.248 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117214 s, 89.5 MB/s 2024-04-25T10:14:38.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.259 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100346 s, 104 MB/s 2024-04-25T10:14:38.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.272 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109449 s, 95.8 MB/s 2024-04-25T10:14:38.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.285 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.285 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103632 s, 101 MB/s 2024-04-25T10:14:38.296 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.297 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.297 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982113 s, 107 MB/s 2024-04-25T10:14:38.309 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.309 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.309 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01088 s, 96.4 MB/s 2024-04-25T10:14:38.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.321 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984636 s, 106 MB/s 2024-04-25T10:14:38.333 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.333 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.333 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103182 s, 102 MB/s 2024-04-25T10:14:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.346 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104931 s, 99.9 MB/s 2024-04-25T10:14:38.358 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.358 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.358 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100947 s, 104 MB/s 2024-04-25T10:14:38.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.370 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102239 s, 103 MB/s 2024-04-25T10:14:38.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.383 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010745 s, 97.6 MB/s 2024-04-25T10:14:38.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.397 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.397 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112805 s, 93.0 MB/s 2024-04-25T10:14:38.408 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.408 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.408 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814357 s, 129 MB/s 2024-04-25T10:14:38.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.421 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112174 s, 93.5 MB/s 2024-04-25T10:14:38.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.436 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122266 s, 85.8 MB/s 2024-04-25T10:14:38.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.449 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.449 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010857 s, 96.6 MB/s 2024-04-25T10:14:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120135 s, 87.3 MB/s 2024-04-25T10:14:38.478 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.478 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.479 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115765 s, 90.6 MB/s 2024-04-25T10:14:38.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.492 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.492 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103815 s, 101 MB/s 2024-04-25T10:14:38.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.507 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125576 s, 83.5 MB/s 2024-04-25T10:14:38.522 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.522 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.522 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120416 s, 87.1 MB/s 2024-04-25T10:14:38.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.534 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898349 s, 117 MB/s 2024-04-25T10:14:38.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.547 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.547 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103333 s, 101 MB/s 2024-04-25T10:14:38.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111602 s, 94.0 MB/s 2024-04-25T10:14:38.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.570 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.570 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079071 s, 133 MB/s 2024-04-25T10:14:38.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.584 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.584 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111767 s, 93.8 MB/s 2024-04-25T10:14:38.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.599 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.599 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120872 s, 86.8 MB/s 2024-04-25T10:14:38.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:38.600 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:38.600 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:38.600 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:14:38.600 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:38.600 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:38.600 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:38.600 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:38.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.668 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.668 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110231 s, 95.1 MB/s 2024-04-25T10:14:38.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.681 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107273 s, 97.7 MB/s 2024-04-25T10:14:38.696 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.696 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.696 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123151 s, 85.1 MB/s 2024-04-25T10:14:38.710 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.710 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.710 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112072 s, 93.6 MB/s 2024-04-25T10:14:38.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.723 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111863 s, 93.7 MB/s 2024-04-25T10:14:38.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.739 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128077 s, 81.9 MB/s 2024-04-25T10:14:38.752 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.753 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.753 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118456 s, 88.5 MB/s 2024-04-25T10:14:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.766 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113513 s, 92.4 MB/s 2024-04-25T10:14:38.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.777 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.777 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843474 s, 124 MB/s 2024-04-25T10:14:38.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.791 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01215 s, 86.3 MB/s 2024-04-25T10:14:38.804 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.805 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010853 s, 96.6 MB/s 2024-04-25T10:14:38.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.816 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008803 s, 119 MB/s 2024-04-25T10:14:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112638 s, 93.1 MB/s 2024-04-25T10:14:38.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.842 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105842 s, 99.1 MB/s 2024-04-25T10:14:38.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.854 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.854 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897079 s, 117 MB/s 2024-04-25T10:14:38.864 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.865 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.865 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848865 s, 124 MB/s 2024-04-25T10:14:38.881 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.881 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.881 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142494 s, 73.6 MB/s 2024-04-25T10:14:38.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.894 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104629 s, 100 MB/s 2024-04-25T10:14:38.908 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.908 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.908 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118992 s, 88.1 MB/s 2024-04-25T10:14:38.921 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.921 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.921 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010546 s, 99.4 MB/s 2024-04-25T10:14:38.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.935 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114509 s, 91.6 MB/s 2024-04-25T10:14:38.946 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.946 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.946 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900264 s, 116 MB/s 2024-04-25T10:14:38.963 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.963 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.963 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142573 s, 73.5 MB/s 2024-04-25T10:14:38.977 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.977 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.977 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011624 s, 90.2 MB/s 2024-04-25T10:14:38.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:38.992 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:38.992 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012146 s, 86.3 MB/s 2024-04-25T10:14:39.006 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.006 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.006 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119776 s, 87.5 MB/s 2024-04-25T10:14:39.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115769 s, 90.6 MB/s 2024-04-25T10:14:39.036 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.037 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133629 s, 78.5 MB/s 2024-04-25T10:14:39.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.054 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135397 s, 77.4 MB/s 2024-04-25T10:14:39.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.070 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140497 s, 74.6 MB/s 2024-04-25T10:14:39.087 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.087 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.087 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134334 s, 78.1 MB/s 2024-04-25T10:14:39.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.103 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132295 s, 79.3 MB/s 2024-04-25T10:14:39.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:39.104 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:39.104 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:39.104 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:14:39.104 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:39.104 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:39.105 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:39.105 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:39.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.175 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129843 s, 80.8 MB/s 2024-04-25T10:14:39.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.188 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102762 s, 102 MB/s 2024-04-25T10:14:39.203 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.203 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.203 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127187 s, 82.4 MB/s 2024-04-25T10:14:39.216 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.216 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.216 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969389 s, 108 MB/s 2024-04-25T10:14:39.229 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.230 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.230 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110402 s, 95.0 MB/s 2024-04-25T10:14:39.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.246 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133471 s, 78.6 MB/s 2024-04-25T10:14:39.260 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.260 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.261 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118349 s, 88.6 MB/s 2024-04-25T10:14:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117852 s, 89.0 MB/s 2024-04-25T10:14:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124926 s, 83.9 MB/s 2024-04-25T10:14:39.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.307 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137712 s, 76.1 MB/s 2024-04-25T10:14:39.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.321 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109471 s, 95.8 MB/s 2024-04-25T10:14:39.337 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.337 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.337 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127919 s, 82.0 MB/s 2024-04-25T10:14:39.352 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.352 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.352 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120229 s, 87.2 MB/s 2024-04-25T10:14:39.365 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.365 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.365 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109195 s, 96.0 MB/s 2024-04-25T10:14:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.379 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115269 s, 91.0 MB/s 2024-04-25T10:14:39.393 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.393 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.393 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107243 s, 97.8 MB/s 2024-04-25T10:14:39.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.407 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111916 s, 93.7 MB/s 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116716 s, 89.8 MB/s 2024-04-25T10:14:39.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.436 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115412 s, 90.9 MB/s 2024-04-25T10:14:39.447 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.447 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.447 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829007 s, 126 MB/s 2024-04-25T10:14:39.460 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.460 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.460 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107486 s, 97.6 MB/s 2024-04-25T10:14:39.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.473 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104318 s, 101 MB/s 2024-04-25T10:14:39.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.487 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.487 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107163 s, 97.8 MB/s 2024-04-25T10:14:39.499 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102115 s, 103 MB/s 2024-04-25T10:14:39.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.512 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970035 s, 108 MB/s 2024-04-25T10:14:39.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.525 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106697 s, 98.3 MB/s 2024-04-25T10:14:39.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.539 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117729 s, 89.1 MB/s 2024-04-25T10:14:39.554 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.554 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.554 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118497 s, 88.5 MB/s 2024-04-25T10:14:39.565 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.565 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.566 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983231 s, 107 MB/s 2024-04-25T10:14:39.579 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.579 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.580 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115933 s, 90.4 MB/s 2024-04-25T10:14:39.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.591 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00979907 s, 107 MB/s 2024-04-25T10:14:39.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102517 s, 102 MB/s 2024-04-25T10:14:39.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:39.605 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:39.605 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:39.605 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:14:39.605 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:39.605 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:39.606 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:39.606 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:39.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.669 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111763 s, 93.8 MB/s 2024-04-25T10:14:39.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.679 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719772 s, 146 MB/s 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108594 s, 96.6 MB/s 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.705 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103685 s, 101 MB/s 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.720 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106933 s, 98.1 MB/s 2024-04-25T10:14:39.735 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.735 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.735 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107115 s, 97.9 MB/s 2024-04-25T10:14:39.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.749 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110415 s, 95.0 MB/s 2024-04-25T10:14:39.759 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.760 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854168 s, 123 MB/s 2024-04-25T10:14:39.770 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.770 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.770 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767889 s, 137 MB/s 2024-04-25T10:14:39.783 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.784 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.784 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107446 s, 97.6 MB/s 2024-04-25T10:14:39.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.797 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010461 s, 100 MB/s 2024-04-25T10:14:39.810 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.811 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109068 s, 96.1 MB/s 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.824 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110712 s, 94.7 MB/s 2024-04-25T10:14:39.838 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.839 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.839 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116957 s, 89.7 MB/s 2024-04-25T10:14:39.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.853 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115296 s, 90.9 MB/s 2024-04-25T10:14:39.865 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.865 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.866 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105596 s, 99.3 MB/s 2024-04-25T10:14:39.879 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.879 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.879 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011204 s, 93.6 MB/s 2024-04-25T10:14:39.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.890 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.890 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726424 s, 144 MB/s 2024-04-25T10:14:39.901 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.902 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.902 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110385 s, 95.0 MB/s 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.913 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891656 s, 118 MB/s 2024-04-25T10:14:39.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.927 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.927 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119224 s, 88.0 MB/s 2024-04-25T10:14:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108461 s, 96.7 MB/s 2024-04-25T10:14:39.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.953 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111141 s, 94.3 MB/s 2024-04-25T10:14:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989602 s, 106 MB/s 2024-04-25T10:14:39.978 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.979 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.979 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108543 s, 96.6 MB/s 2024-04-25T10:14:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:39.990 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933301 s, 112 MB/s 2024-04-25T10:14:40.001 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.001 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.001 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901168 s, 116 MB/s 2024-04-25T10:14:40.012 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925044 s, 113 MB/s 2024-04-25T10:14:40.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.026 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112227 s, 93.4 MB/s 2024-04-25T10:14:40.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.038 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010033 s, 105 MB/s 2024-04-25T10:14:40.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.048 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758054 s, 138 MB/s 2024-04-25T10:14:40.061 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.061 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.061 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111132 s, 94.4 MB/s 2024-04-25T10:14:40.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:40.063 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:40.063 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:40.063 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:14:40.063 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:40.063 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:40.063 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:40.063 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:40.129 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.130 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.130 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106971 s, 98.0 MB/s 2024-04-25T10:14:40.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.146 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125228 s, 83.7 MB/s 2024-04-25T10:14:40.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.161 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.161 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119373 s, 87.8 MB/s 2024-04-25T10:14:40.173 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.174 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.174 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103889 s, 101 MB/s 2024-04-25T10:14:40.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.187 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.187 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107096 s, 97.9 MB/s 2024-04-25T10:14:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.202 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013126 s, 79.9 MB/s 2024-04-25T10:14:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897537 s, 117 MB/s 2024-04-25T10:14:40.226 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.226 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.227 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106871 s, 98.1 MB/s 2024-04-25T10:14:40.240 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.240 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.240 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114186 s, 91.8 MB/s 2024-04-25T10:14:40.250 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.250 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.250 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761134 s, 138 MB/s 2024-04-25T10:14:40.264 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.264 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.264 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116791 s, 89.8 MB/s 2024-04-25T10:14:40.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.276 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101961 s, 103 MB/s 2024-04-25T10:14:40.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.288 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966336 s, 109 MB/s 2024-04-25T10:14:40.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.300 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955648 s, 110 MB/s 2024-04-25T10:14:40.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.312 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.312 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0093282 s, 112 MB/s 2024-04-25T10:14:40.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.324 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103928 s, 101 MB/s 2024-04-25T10:14:40.334 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.335 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.335 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079915 s, 131 MB/s 2024-04-25T10:14:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.348 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01089 s, 96.3 MB/s 2024-04-25T10:14:40.361 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.361 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.361 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111038 s, 94.4 MB/s 2024-04-25T10:14:40.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.371 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741129 s, 141 MB/s 2024-04-25T10:14:40.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.383 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105482 s, 99.4 MB/s 2024-04-25T10:14:40.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.396 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101227 s, 104 MB/s 2024-04-25T10:14:40.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.410 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.410 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120939 s, 86.7 MB/s 2024-04-25T10:14:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012157 s, 86.3 MB/s 2024-04-25T10:14:40.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.438 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106126 s, 98.8 MB/s 2024-04-25T10:14:40.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.450 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971333 s, 108 MB/s 2024-04-25T10:14:40.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.464 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119034 s, 88.1 MB/s 2024-04-25T10:14:40.478 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.478 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.478 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114906 s, 91.3 MB/s 2024-04-25T10:14:40.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.491 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109936 s, 95.4 MB/s 2024-04-25T10:14:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124175 s, 84.4 MB/s 2024-04-25T10:14:40.518 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.518 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.518 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933745 s, 112 MB/s 2024-04-25T10:14:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124964 s, 83.9 MB/s 2024-04-25T10:14:40.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:40.534 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:40.534 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:40.534 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:14:40.535 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:40.535 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:40.535 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:40.535 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:40.600 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.601 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.601 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125612 s, 83.5 MB/s 2024-04-25T10:14:40.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.615 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.615 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119317 s, 87.9 MB/s 2024-04-25T10:14:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.630 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127899 s, 82.0 MB/s 2024-04-25T10:14:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941699 s, 111 MB/s 2024-04-25T10:14:40.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.654 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00908757 s, 115 MB/s 2024-04-25T10:14:40.668 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.668 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.668 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118424 s, 88.5 MB/s 2024-04-25T10:14:40.682 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.682 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.682 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114426 s, 91.6 MB/s 2024-04-25T10:14:40.697 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.698 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131104 s, 80.0 MB/s 2024-04-25T10:14:40.712 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.712 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.712 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121695 s, 86.2 MB/s 2024-04-25T10:14:40.728 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.729 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136718 s, 76.7 MB/s 2024-04-25T10:14:40.742 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.742 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.742 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107741 s, 97.3 MB/s 2024-04-25T10:14:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.757 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129851 s, 80.8 MB/s 2024-04-25T10:14:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.772 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122204 s, 85.8 MB/s 2024-04-25T10:14:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139627 s, 75.1 MB/s 2024-04-25T10:14:40.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.805 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129707 s, 80.8 MB/s 2024-04-25T10:14:40.821 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.821 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.821 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117133 s, 89.5 MB/s 2024-04-25T10:14:40.837 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.838 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.838 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128213 s, 81.8 MB/s 2024-04-25T10:14:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.857 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150652 s, 69.6 MB/s 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135683 s, 77.3 MB/s 2024-04-25T10:14:40.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.889 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119487 s, 87.8 MB/s 2024-04-25T10:14:40.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.904 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.904 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125446 s, 83.6 MB/s 2024-04-25T10:14:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.920 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.921 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131409 s, 79.8 MB/s 2024-04-25T10:14:40.934 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.935 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114433 s, 91.6 MB/s 2024-04-25T10:14:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108226 s, 96.9 MB/s 2024-04-25T10:14:40.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.962 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113662 s, 92.3 MB/s 2024-04-25T10:14:40.978 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.978 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.978 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130259 s, 80.5 MB/s 2024-04-25T10:14:40.994 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:40.995 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:40.995 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134521 s, 77.9 MB/s 2024-04-25T10:14:41.010 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.010 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.010 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122292 s, 85.7 MB/s 2024-04-25T10:14:41.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.026 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012979 s, 80.8 MB/s 2024-04-25T10:14:41.041 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.041 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.041 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123857 s, 84.7 MB/s 2024-04-25T10:14:41.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.054 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.054 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996068 s, 105 MB/s 2024-04-25T10:14:41.069 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.069 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.069 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117604 s, 89.2 MB/s 2024-04-25T10:14:41.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:41.070 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:41.070 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:41.070 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:14:41.071 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:41.071 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:41.071 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:41.071 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:41.135 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.135 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.135 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105 s, 99.9 MB/s 2024-04-25T10:14:41.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.147 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950857 s, 110 MB/s 2024-04-25T10:14:41.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.162 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.162 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121512 s, 86.3 MB/s 2024-04-25T10:14:41.176 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.176 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.176 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116745 s, 89.8 MB/s 2024-04-25T10:14:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090958 s, 115 MB/s 2024-04-25T10:14:41.203 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.203 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.203 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118934 s, 88.2 MB/s 2024-04-25T10:14:41.217 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.217 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.217 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122816 s, 85.4 MB/s 2024-04-25T10:14:41.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.365 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.365 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126983 s, 82.6 MB/s 2024-04-25T10:14:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112784 s, 93.0 MB/s 2024-04-25T10:14:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943238 s, 111 MB/s 2024-04-25T10:14:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084619 s, 124 MB/s 2024-04-25T10:14:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120145 s, 87.3 MB/s 2024-04-25T10:14:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109952 s, 95.4 MB/s 2024-04-25T10:14:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.369 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111339 s, 94.2 MB/s 2024-04-25T10:14:41.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.369 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944358 s, 111 MB/s 2024-04-25T10:14:41.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.369 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108932 s, 96.3 MB/s 2024-04-25T10:14:41.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801843 s, 131 MB/s 2024-04-25T10:14:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106556 s, 98.4 MB/s 2024-04-25T10:14:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.371 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108183 s, 96.9 MB/s 2024-04-25T10:14:41.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.379 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738862 s, 142 MB/s 2024-04-25T10:14:41.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.407 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.407 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101671 s, 103 MB/s 2024-04-25T10:14:41.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.422 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.422 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124181 s, 84.4 MB/s 2024-04-25T10:14:41.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.436 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.436 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112813 s, 92.9 MB/s 2024-04-25T10:14:41.446 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.446 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.446 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081054 s, 129 MB/s 2024-04-25T10:14:41.460 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.460 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.460 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110243 s, 95.1 MB/s 2024-04-25T10:14:41.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.473 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105312 s, 99.6 MB/s 2024-04-25T10:14:41.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.486 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.487 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010991 s, 95.4 MB/s 2024-04-25T10:14:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.499 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977062 s, 107 MB/s 2024-04-25T10:14:41.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.512 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107188 s, 97.8 MB/s 2024-04-25T10:14:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.528 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118097 s, 88.8 MB/s 2024-04-25T10:14:41.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.541 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.542 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112342 s, 93.3 MB/s 2024-04-25T10:14:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.555 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113321 s, 92.5 MB/s 2024-04-25T10:14:41.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:41.556 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:41.556 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:41.556 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:14:41.556 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:41.556 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:41.557 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:41.557 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.624 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971797 s, 108 MB/s 2024-04-25T10:14:41.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.636 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.636 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101145 s, 104 MB/s 2024-04-25T10:14:41.648 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.649 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.649 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00967726 s, 108 MB/s 2024-04-25T10:14:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.662 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010588 s, 99.0 MB/s 2024-04-25T10:14:41.673 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.673 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.673 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937623 s, 112 MB/s 2024-04-25T10:14:41.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.686 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105166 s, 99.7 MB/s 2024-04-25T10:14:41.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.698 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100887 s, 104 MB/s 2024-04-25T10:14:41.712 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.712 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.712 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114406 s, 91.7 MB/s 2024-04-25T10:14:41.725 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.726 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010934 s, 95.9 MB/s 2024-04-25T10:14:41.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.738 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103141 s, 102 MB/s 2024-04-25T10:14:41.750 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.750 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.750 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976907 s, 107 MB/s 2024-04-25T10:14:41.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.763 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106137 s, 98.8 MB/s 2024-04-25T10:14:41.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.776 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108106 s, 97.0 MB/s 2024-04-25T10:14:41.786 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.786 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.786 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791056 s, 133 MB/s 2024-04-25T10:14:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.800 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111045 s, 94.4 MB/s 2024-04-25T10:14:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.813 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115841 s, 90.5 MB/s 2024-04-25T10:14:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.829 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013277 s, 79.0 MB/s 2024-04-25T10:14:41.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.841 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.841 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102455 s, 102 MB/s 2024-04-25T10:14:41.852 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.852 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.853 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091636 s, 114 MB/s 2024-04-25T10:14:41.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.867 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119679 s, 87.6 MB/s 2024-04-25T10:14:41.881 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.881 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.881 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113487 s, 92.4 MB/s 2024-04-25T10:14:41.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.895 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.895 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01108 s, 94.6 MB/s 2024-04-25T10:14:41.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.909 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118531 s, 88.5 MB/s 2024-04-25T10:14:41.924 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.924 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.924 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123881 s, 84.6 MB/s 2024-04-25T10:14:41.938 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.938 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.938 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107418 s, 97.6 MB/s 2024-04-25T10:14:41.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.952 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.952 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114782 s, 91.4 MB/s 2024-04-25T10:14:41.967 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.967 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.967 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122627 s, 85.5 MB/s 2024-04-25T10:14:41.982 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:41.982 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:41.983 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126386 s, 83.0 MB/s 2024-04-25T10:14:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.000 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.000 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137567 s, 76.2 MB/s 2024-04-25T10:14:42.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.017 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142683 s, 73.5 MB/s 2024-04-25T10:14:42.033 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.033 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.033 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129239 s, 81.1 MB/s 2024-04-25T10:14:42.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.048 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011899 s, 88.1 MB/s 2024-04-25T10:14:42.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:42.049 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:42.050 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:42.050 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:14:42.050 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:42.050 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:42.050 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:42.050 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:42.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.118 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012476 s, 84.0 MB/s 2024-04-25T10:14:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.130 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987315 s, 106 MB/s 2024-04-25T10:14:42.145 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.145 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.145 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121292 s, 86.5 MB/s 2024-04-25T10:14:42.159 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.159 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.159 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109461 s, 95.8 MB/s 2024-04-25T10:14:42.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.175 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.175 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122764 s, 85.4 MB/s 2024-04-25T10:14:42.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.188 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110414 s, 95.0 MB/s 2024-04-25T10:14:42.202 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.202 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.202 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111622 s, 93.9 MB/s 2024-04-25T10:14:42.216 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.216 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.216 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112673 s, 93.1 MB/s 2024-04-25T10:14:42.230 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.230 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.230 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113432 s, 92.4 MB/s 2024-04-25T10:14:42.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.243 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.243 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107838 s, 97.2 MB/s 2024-04-25T10:14:42.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.257 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.258 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116861 s, 89.7 MB/s 2024-04-25T10:14:42.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.270 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01025 s, 102 MB/s 2024-04-25T10:14:42.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.284 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.284 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113129 s, 92.7 MB/s 2024-04-25T10:14:42.298 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.298 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118705 s, 88.3 MB/s 2024-04-25T10:14:42.312 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.312 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.312 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110872 s, 94.6 MB/s 2024-04-25T10:14:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.326 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.326 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113082 s, 92.7 MB/s 2024-04-25T10:14:42.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.339 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110495 s, 94.9 MB/s 2024-04-25T10:14:42.354 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.354 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.354 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117338 s, 89.4 MB/s 2024-04-25T10:14:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.370 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127287 s, 82.4 MB/s 2024-04-25T10:14:42.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.384 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121325 s, 86.4 MB/s 2024-04-25T10:14:42.416 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.416 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.416 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110829 s, 94.6 MB/s 2024-04-25T10:14:42.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.427 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870052 s, 121 MB/s 2024-04-25T10:14:42.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.441 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114587 s, 91.5 MB/s 2024-04-25T10:14:42.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.454 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105896 s, 99.0 MB/s 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.468 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117193 s, 89.5 MB/s 2024-04-25T10:14:42.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.482 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116044 s, 90.4 MB/s 2024-04-25T10:14:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.494 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104242 s, 101 MB/s 2024-04-25T10:14:42.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.507 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104803 s, 100 MB/s 2024-04-25T10:14:42.520 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.520 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.520 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103479 s, 101 MB/s 2024-04-25T10:14:42.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.533 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106612 s, 98.4 MB/s 2024-04-25T10:14:42.545 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.546 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010544 s, 99.4 MB/s 2024-04-25T10:14:42.558 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.558 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.559 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107567 s, 97.5 MB/s 2024-04-25T10:14:42.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:42.560 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:42.560 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:42.560 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:14:42.560 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:42.560 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:42.560 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:42.560 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:42.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.626 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.626 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996276 s, 105 MB/s 2024-04-25T10:14:42.638 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.639 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.639 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108104 s, 97.0 MB/s 2024-04-25T10:14:42.653 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.654 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131098 s, 80.0 MB/s 2024-04-25T10:14:42.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.668 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114821 s, 91.3 MB/s 2024-04-25T10:14:42.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.680 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100568 s, 104 MB/s 2024-04-25T10:14:42.693 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.693 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.693 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108287 s, 96.8 MB/s 2024-04-25T10:14:42.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.704 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944831 s, 111 MB/s 2024-04-25T10:14:42.717 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.718 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.718 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107153 s, 97.9 MB/s 2024-04-25T10:14:42.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.731 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.731 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108773 s, 96.4 MB/s 2024-04-25T10:14:42.743 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.743 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.743 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103461 s, 101 MB/s 2024-04-25T10:14:42.756 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114451 s, 91.6 MB/s 2024-04-25T10:14:42.767 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.767 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.767 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800661 s, 131 MB/s 2024-04-25T10:14:42.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.779 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.779 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982555 s, 107 MB/s 2024-04-25T10:14:42.794 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.794 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.794 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130271 s, 80.5 MB/s 2024-04-25T10:14:42.806 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.806 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.806 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994477 s, 105 MB/s 2024-04-25T10:14:42.821 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.821 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.821 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125897 s, 83.3 MB/s 2024-04-25T10:14:42.836 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.836 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.836 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121046 s, 86.6 MB/s 2024-04-25T10:14:42.851 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.851 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.851 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011928 s, 87.9 MB/s 2024-04-25T10:14:42.865 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.865 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.865 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108873 s, 96.3 MB/s 2024-04-25T10:14:42.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.875 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.875 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825885 s, 127 MB/s 2024-04-25T10:14:42.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.889 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.889 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118026 s, 88.8 MB/s 2024-04-25T10:14:42.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.900 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836176 s, 125 MB/s 2024-04-25T10:14:42.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.912 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.912 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953806 s, 110 MB/s 2024-04-25T10:14:42.925 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.925 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.925 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106914 s, 98.1 MB/s 2024-04-25T10:14:42.937 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.937 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.937 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960985 s, 109 MB/s 2024-04-25T10:14:42.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.951 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.951 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117343 s, 89.4 MB/s 2024-04-25T10:14:42.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.962 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.962 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850819 s, 123 MB/s 2024-04-25T10:14:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101616 s, 103 MB/s 2024-04-25T10:14:42.989 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:42.989 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:42.989 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117339 s, 89.4 MB/s 2024-04-25T10:14:43.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.004 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126168 s, 83.1 MB/s 2024-04-25T10:14:43.016 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.017 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.017 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010532 s, 99.6 MB/s 2024-04-25T10:14:43.031 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.031 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.031 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115247 s, 91.0 MB/s 2024-04-25T10:14:43.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:43.032 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:43.032 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:43.032 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:14:43.032 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:43.032 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:43.032 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:43.032 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113083 s, 92.7 MB/s 2024-04-25T10:14:43.112 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.112 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.112 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011079 s, 94.6 MB/s 2024-04-25T10:14:43.126 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.126 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.126 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120118 s, 87.3 MB/s 2024-04-25T10:14:43.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.138 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.138 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989374 s, 106 MB/s 2024-04-25T10:14:43.151 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.151 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.151 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110944 s, 94.5 MB/s 2024-04-25T10:14:43.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.165 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.165 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010718 s, 97.8 MB/s 2024-04-25T10:14:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958239 s, 109 MB/s 2024-04-25T10:14:43.192 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.192 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.192 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962681 s, 109 MB/s 2024-04-25T10:14:43.204 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.204 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.204 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900559 s, 116 MB/s 2024-04-25T10:14:43.220 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.220 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.220 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012974 s, 80.8 MB/s 2024-04-25T10:14:43.233 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.233 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.233 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104208 s, 101 MB/s 2024-04-25T10:14:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.246 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106244 s, 98.7 MB/s 2024-04-25T10:14:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.259 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113553 s, 92.3 MB/s 2024-04-25T10:14:43.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.273 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116436 s, 90.1 MB/s 2024-04-25T10:14:43.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.287 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112871 s, 92.9 MB/s 2024-04-25T10:14:43.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.300 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101622 s, 103 MB/s 2024-04-25T10:14:43.314 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.314 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.314 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115386 s, 90.9 MB/s 2024-04-25T10:14:43.327 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.327 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.327 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108198 s, 96.9 MB/s 2024-04-25T10:14:43.342 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.342 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.342 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122805 s, 85.4 MB/s 2024-04-25T10:14:43.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.357 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122545 s, 85.6 MB/s 2024-04-25T10:14:43.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.370 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109158 s, 96.1 MB/s 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110873 s, 94.6 MB/s 2024-04-25T10:14:43.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.436 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113065 s, 92.7 MB/s 2024-04-25T10:14:43.451 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.451 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.451 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127643 s, 82.1 MB/s 2024-04-25T10:14:43.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.462 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884698 s, 119 MB/s 2024-04-25T10:14:43.477 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.477 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.477 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121387 s, 86.4 MB/s 2024-04-25T10:14:43.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.488 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.488 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880529 s, 119 MB/s 2024-04-25T10:14:43.503 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.503 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.503 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117927 s, 88.9 MB/s 2024-04-25T10:14:43.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.515 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.515 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965644 s, 109 MB/s 2024-04-25T10:14:43.528 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.529 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.529 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104473 s, 100 MB/s 2024-04-25T10:14:43.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.539 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814257 s, 129 MB/s 2024-04-25T10:14:43.554 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.554 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.554 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117424 s, 89.3 MB/s 2024-04-25T10:14:43.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:43.555 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:43.556 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:43.556 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:14:43.556 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:43.556 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:43.556 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:43.556 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:43.782 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.782 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.782 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105753 s, 99.2 MB/s 2024-04-25T10:14:43.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.797 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.798 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119301 s, 87.9 MB/s 2024-04-25T10:14:43.811 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.811 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.812 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107008 s, 98.0 MB/s 2024-04-25T10:14:43.821 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.821 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.821 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834148 s, 126 MB/s 2024-04-25T10:14:43.831 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.831 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.831 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074566 s, 141 MB/s 2024-04-25T10:14:43.844 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.844 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.844 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109205 s, 96.0 MB/s 2024-04-25T10:14:43.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.858 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.858 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118308 s, 88.6 MB/s 2024-04-25T10:14:43.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.871 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.871 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106281 s, 98.7 MB/s 2024-04-25T10:14:43.883 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.883 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.883 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995503 s, 105 MB/s 2024-04-25T10:14:43.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.898 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123704 s, 84.8 MB/s 2024-04-25T10:14:43.911 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.911 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.911 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011335 s, 92.5 MB/s 2024-04-25T10:14:43.924 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.925 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.925 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109511 s, 95.8 MB/s 2024-04-25T10:14:43.937 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.937 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.937 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994704 s, 105 MB/s 2024-04-25T10:14:43.949 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.949 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.949 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102272 s, 103 MB/s 2024-04-25T10:14:43.963 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.964 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.964 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120585 s, 87.0 MB/s 2024-04-25T10:14:43.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.975 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.975 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944107 s, 111 MB/s 2024-04-25T10:14:43.988 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:43.989 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:43.989 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110438 s, 94.9 MB/s 2024-04-25T10:14:44.002 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.003 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.003 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945748 s, 111 MB/s 2024-04-25T10:14:44.014 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.014 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.015 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116225 s, 90.2 MB/s 2024-04-25T10:14:44.026 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.027 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.027 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100027 s, 105 MB/s 2024-04-25T10:14:44.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.039 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.040 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105003 s, 99.9 MB/s 2024-04-25T10:14:44.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.050 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.050 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870468 s, 120 MB/s 2024-04-25T10:14:44.060 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.060 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.060 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773309 s, 136 MB/s 2024-04-25T10:14:44.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.073 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.073 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104039 s, 101 MB/s 2024-04-25T10:14:44.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.082 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741417 s, 141 MB/s 2024-04-25T10:14:44.096 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.097 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.097 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120326 s, 87.1 MB/s 2024-04-25T10:14:44.106 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.106 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.107 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743938 s, 141 MB/s 2024-04-25T10:14:44.119 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.119 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.119 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010126 s, 104 MB/s 2024-04-25T10:14:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102768 s, 102 MB/s 2024-04-25T10:14:44.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.141 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.142 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771329 s, 136 MB/s 2024-04-25T10:14:44.154 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.154 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.155 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104931 s, 99.9 MB/s 2024-04-25T10:14:44.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.169 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114049 s, 91.9 MB/s 2024-04-25T10:14:44.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:44.170 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:44.170 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:44.170 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:14:44.170 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:44.171 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:44.171 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:44.171 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:44.193 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.193 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.193 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113725 s, 92.2 MB/s 2024-04-25T10:14:44.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.210 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117863 s, 89.0 MB/s 2024-04-25T10:14:44.218 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.218 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.218 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092122 s, 114 MB/s 2024-04-25T10:14:44.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.232 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.232 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010896 s, 96.2 MB/s 2024-04-25T10:14:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.246 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114693 s, 91.4 MB/s 2024-04-25T10:14:44.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.259 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104774 s, 100 MB/s 2024-04-25T10:14:44.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.272 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.272 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105783 s, 99.1 MB/s 2024-04-25T10:14:44.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.286 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118984 s, 88.1 MB/s 2024-04-25T10:14:44.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.300 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.300 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116792 s, 89.8 MB/s 2024-04-25T10:14:44.314 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.314 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.314 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114258 s, 91.8 MB/s 2024-04-25T10:14:44.327 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.327 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.327 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103131 s, 102 MB/s 2024-04-25T10:14:44.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.341 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118931 s, 88.2 MB/s 2024-04-25T10:14:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.354 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109128 s, 96.1 MB/s 2024-04-25T10:14:44.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.367 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.368 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102523 s, 102 MB/s 2024-04-25T10:14:44.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.383 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125418 s, 83.6 MB/s 2024-04-25T10:14:44.401 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.402 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.402 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161645 s, 64.9 MB/s 2024-04-25T10:14:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111083 s, 94.4 MB/s 2024-04-25T10:14:44.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.442 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.442 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115875 s, 90.5 MB/s 2024-04-25T10:14:44.456 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.457 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.457 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119101 s, 88.0 MB/s 2024-04-25T10:14:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106614 s, 98.4 MB/s 2024-04-25T10:14:44.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.485 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121292 s, 86.5 MB/s 2024-04-25T10:14:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.500 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136295 s, 76.9 MB/s 2024-04-25T10:14:44.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.512 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087373 s, 120 MB/s 2024-04-25T10:14:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125019 s, 83.9 MB/s 2024-04-25T10:14:44.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.539 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866872 s, 121 MB/s 2024-04-25T10:14:44.553 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.553 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.553 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01172 s, 89.5 MB/s 2024-04-25T10:14:44.565 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.565 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.565 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974542 s, 108 MB/s 2024-04-25T10:14:44.580 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.580 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.580 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119191 s, 88.0 MB/s 2024-04-25T10:14:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.592 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984472 s, 107 MB/s 2024-04-25T10:14:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.604 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917119 s, 114 MB/s 2024-04-25T10:14:44.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.618 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810313 s, 129 MB/s 2024-04-25T10:14:44.629 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.629 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.629 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122292 s, 85.7 MB/s 2024-04-25T10:14:44.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:44.631 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:44.631 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:44.631 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:14:44.631 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:44.631 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:44.631 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:44.631 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:44.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.692 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865766 s, 121 MB/s 2024-04-25T10:14:44.702 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.702 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.702 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831416 s, 126 MB/s 2024-04-25T10:14:44.716 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.717 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.717 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118665 s, 88.4 MB/s 2024-04-25T10:14:44.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.726 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.726 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736106 s, 142 MB/s 2024-04-25T10:14:44.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.739 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.739 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105407 s, 99.5 MB/s 2024-04-25T10:14:44.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.751 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.751 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101968 s, 103 MB/s 2024-04-25T10:14:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.763 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969786 s, 108 MB/s 2024-04-25T10:14:44.774 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.775 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976122 s, 107 MB/s 2024-04-25T10:14:44.786 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.787 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.787 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964881 s, 109 MB/s 2024-04-25T10:14:44.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.800 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.800 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117813 s, 89.0 MB/s 2024-04-25T10:14:44.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.814 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.814 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011096 s, 94.5 MB/s 2024-04-25T10:14:44.826 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.826 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.827 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993353 s, 106 MB/s 2024-04-25T10:14:44.838 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.838 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.839 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010113 s, 104 MB/s 2024-04-25T10:14:44.850 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.851 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.851 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100032 s, 105 MB/s 2024-04-25T10:14:44.862 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.863 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.863 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994393 s, 105 MB/s 2024-04-25T10:14:44.877 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.877 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.877 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117879 s, 89.0 MB/s 2024-04-25T10:14:44.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.886 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.886 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724187 s, 145 MB/s 2024-04-25T10:14:44.897 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.897 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.897 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924686 s, 113 MB/s 2024-04-25T10:14:44.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.909 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00988578 s, 106 MB/s 2024-04-25T10:14:44.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.922 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992185 s, 106 MB/s 2024-04-25T10:14:44.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.936 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.936 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119428 s, 87.8 MB/s 2024-04-25T10:14:44.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.946 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746093 s, 141 MB/s 2024-04-25T10:14:44.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.959 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112397 s, 93.3 MB/s 2024-04-25T10:14:44.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.970 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0093758 s, 112 MB/s 2024-04-25T10:14:44.983 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.984 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.984 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110082 s, 95.3 MB/s 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:44.996 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101589 s, 103 MB/s 2024-04-25T10:14:45.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.005 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755776 s, 139 MB/s 2024-04-25T10:14:45.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.019 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118509 s, 88.5 MB/s 2024-04-25T10:14:45.032 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.032 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.032 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103788 s, 101 MB/s 2024-04-25T10:14:45.045 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.045 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.045 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108613 s, 96.5 MB/s 2024-04-25T10:14:45.056 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.057 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.057 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00981183 s, 107 MB/s 2024-04-25T10:14:45.070 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.071 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.071 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117264 s, 89.4 MB/s 2024-04-25T10:14:45.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:45.072 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:45.072 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:45.072 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:14:45.072 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:45.072 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:45.072 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:45.072 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:45.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.137 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112287 s, 93.4 MB/s 2024-04-25T10:14:45.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.150 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.150 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105917 s, 99.0 MB/s 2024-04-25T10:14:45.159 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.159 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.159 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715979 s, 146 MB/s 2024-04-25T10:14:45.170 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.171 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.171 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00935428 s, 112 MB/s 2024-04-25T10:14:45.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.180 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.180 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729487 s, 144 MB/s 2024-04-25T10:14:45.189 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.190 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734014 s, 143 MB/s 2024-04-25T10:14:45.203 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.203 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.203 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108692 s, 96.5 MB/s 2024-04-25T10:14:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.213 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763851 s, 137 MB/s 2024-04-25T10:14:45.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.225 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.225 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102198 s, 103 MB/s 2024-04-25T10:14:45.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.238 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.238 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105977 s, 98.9 MB/s 2024-04-25T10:14:45.251 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.251 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.251 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103335 s, 101 MB/s 2024-04-25T10:14:45.262 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.262 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.263 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939626 s, 112 MB/s 2024-04-25T10:14:45.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.276 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110673 s, 94.7 MB/s 2024-04-25T10:14:45.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.288 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101305 s, 104 MB/s 2024-04-25T10:14:45.299 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.299 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.299 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875992 s, 120 MB/s 2024-04-25T10:14:45.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.311 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.311 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983692 s, 107 MB/s 2024-04-25T10:14:45.325 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.325 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.325 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108327 s, 96.8 MB/s 2024-04-25T10:14:45.337 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108283 s, 96.8 MB/s 2024-04-25T10:14:45.349 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.349 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.349 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943379 s, 111 MB/s 2024-04-25T10:14:45.364 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.364 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.364 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119746 s, 87.6 MB/s 2024-04-25T10:14:45.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.376 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103773 s, 101 MB/s 2024-04-25T10:14:45.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.391 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.391 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121498 s, 86.3 MB/s 2024-04-25T10:14:45.405 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.405 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.405 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102002 s, 103 MB/s 2024-04-25T10:14:45.434 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.435 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.435 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087312 s, 120 MB/s 2024-04-25T10:14:45.446 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.447 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.447 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00994866 s, 105 MB/s 2024-04-25T10:14:45.461 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.461 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.461 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116459 s, 90.0 MB/s 2024-04-25T10:14:45.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.473 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.473 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102527 s, 102 MB/s 2024-04-25T10:14:45.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.484 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.484 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083155 s, 126 MB/s 2024-04-25T10:14:45.497 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.497 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.497 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111851 s, 93.7 MB/s 2024-04-25T10:14:45.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.513 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.513 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130548 s, 80.3 MB/s 2024-04-25T10:14:45.522 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.522 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.522 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725949 s, 144 MB/s 2024-04-25T10:14:45.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.536 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120892 s, 86.7 MB/s 2024-04-25T10:14:45.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:45.537 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:45.537 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:45.538 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:14:45.538 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:45.538 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:45.538 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:45.538 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:45.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.606 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.606 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104436 s, 100 MB/s 2024-04-25T10:14:45.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.616 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748125 s, 140 MB/s 2024-04-25T10:14:45.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.631 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012078 s, 86.8 MB/s 2024-04-25T10:14:45.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.643 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918598 s, 114 MB/s 2024-04-25T10:14:45.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.654 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.655 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110104 s, 95.2 MB/s 2024-04-25T10:14:45.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.667 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.667 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110793 s, 94.6 MB/s 2024-04-25T10:14:45.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.680 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.680 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105854 s, 99.1 MB/s 2024-04-25T10:14:45.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.692 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.692 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980761 s, 107 MB/s 2024-04-25T10:14:45.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.704 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.704 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103525 s, 101 MB/s 2024-04-25T10:14:45.718 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.718 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.718 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114235 s, 91.8 MB/s 2024-04-25T10:14:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.731 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113746 s, 92.2 MB/s 2024-04-25T10:14:45.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.741 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788357 s, 133 MB/s 2024-04-25T10:14:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107377 s, 97.7 MB/s 2024-04-25T10:14:45.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.764 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753834 s, 139 MB/s 2024-04-25T10:14:45.778 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.778 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.778 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114678 s, 91.4 MB/s 2024-04-25T10:14:45.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.792 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118468 s, 88.5 MB/s 2024-04-25T10:14:45.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.807 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129736 s, 80.8 MB/s 2024-04-25T10:14:45.819 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.819 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.819 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00958112 s, 109 MB/s 2024-04-25T10:14:45.834 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.834 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.834 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129898 s, 80.7 MB/s 2024-04-25T10:14:45.848 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.848 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.848 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119183 s, 88.0 MB/s 2024-04-25T10:14:45.862 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.862 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.862 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115805 s, 90.5 MB/s 2024-04-25T10:14:45.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.874 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.874 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00996331 s, 105 MB/s 2024-04-25T10:14:45.887 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.888 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.888 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110457 s, 94.9 MB/s 2024-04-25T10:14:45.901 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.902 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.902 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011517 s, 91.0 MB/s 2024-04-25T10:14:45.915 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.916 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119771 s, 87.5 MB/s 2024-04-25T10:14:45.928 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.928 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.928 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997029 s, 105 MB/s 2024-04-25T10:14:45.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.942 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.942 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116851 s, 89.7 MB/s 2024-04-25T10:14:45.956 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.956 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.957 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123512 s, 84.9 MB/s 2024-04-25T10:14:45.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.970 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109438 s, 95.8 MB/s 2024-04-25T10:14:45.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.981 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.981 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974702 s, 108 MB/s 2024-04-25T10:14:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:45.992 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871877 s, 120 MB/s 2024-04-25T10:14:46.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.006 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103624 s, 101 MB/s 2024-04-25T10:14:46.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:46.007 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:46.007 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:46.007 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:14:46.007 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:46.007 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:46.007 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:46.007 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:46.068 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.069 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.069 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092473 s, 113 MB/s 2024-04-25T10:14:46.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.082 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115986 s, 90.4 MB/s 2024-04-25T10:14:46.095 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.095 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.095 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100815 s, 104 MB/s 2024-04-25T10:14:46.109 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.109 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.109 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011524 s, 91.0 MB/s 2024-04-25T10:14:46.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.124 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125218 s, 83.7 MB/s 2024-04-25T10:14:46.139 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.139 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.139 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126012 s, 83.2 MB/s 2024-04-25T10:14:46.155 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.156 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.156 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134166 s, 78.2 MB/s 2024-04-25T10:14:46.170 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.171 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.171 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123981 s, 84.6 MB/s 2024-04-25T10:14:46.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.186 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.186 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121996 s, 86.0 MB/s 2024-04-25T10:14:46.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.201 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012566 s, 83.4 MB/s 2024-04-25T10:14:46.213 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.214 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952104 s, 110 MB/s 2024-04-25T10:14:46.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.228 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119388 s, 87.8 MB/s 2024-04-25T10:14:46.244 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.245 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013472 s, 77.8 MB/s 2024-04-25T10:14:46.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.259 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120201 s, 87.2 MB/s 2024-04-25T10:14:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.274 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116835 s, 89.7 MB/s 2024-04-25T10:14:46.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.287 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.287 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107101 s, 97.9 MB/s 2024-04-25T10:14:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.298 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833022 s, 126 MB/s 2024-04-25T10:14:46.312 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.312 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.312 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124234 s, 84.4 MB/s 2024-04-25T10:14:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969151 s, 108 MB/s 2024-04-25T10:14:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.336 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.337 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950908 s, 110 MB/s 2024-04-25T10:14:46.350 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.350 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.350 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107735 s, 97.3 MB/s 2024-04-25T10:14:46.364 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.364 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.364 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119522 s, 87.7 MB/s 2024-04-25T10:14:46.378 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.379 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111241 s, 94.3 MB/s 2024-04-25T10:14:46.390 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.390 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985365 s, 106 MB/s 2024-04-25T10:14:46.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.406 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.406 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133362 s, 78.6 MB/s 2024-04-25T10:14:46.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.438 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.438 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126036 s, 83.2 MB/s 2024-04-25T10:14:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.453 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.454 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123533 s, 84.9 MB/s 2024-04-25T10:14:46.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.465 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092526 s, 113 MB/s 2024-04-25T10:14:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114185 s, 91.8 MB/s 2024-04-25T10:14:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.491 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916159 s, 114 MB/s 2024-04-25T10:14:46.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011899 s, 88.1 MB/s 2024-04-25T10:14:46.519 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.519 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.520 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111375 s, 94.1 MB/s 2024-04-25T10:14:46.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:46.521 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:46.521 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:46.521 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:14:46.521 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:46.521 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:46.521 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:46.521 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:46.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.590 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.590 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118884 s, 88.2 MB/s 2024-04-25T10:14:46.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.606 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128524 s, 81.6 MB/s 2024-04-25T10:14:46.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.618 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103934 s, 101 MB/s 2024-04-25T10:14:46.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.634 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012393 s, 84.6 MB/s 2024-04-25T10:14:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116847 s, 89.7 MB/s 2024-04-25T10:14:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114057 s, 91.9 MB/s 2024-04-25T10:14:46.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.679 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.679 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144296 s, 72.7 MB/s 2024-04-25T10:14:46.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.698 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158021 s, 66.4 MB/s 2024-04-25T10:14:46.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.715 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.715 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139446 s, 75.2 MB/s 2024-04-25T10:14:46.731 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.731 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.731 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125942 s, 83.3 MB/s 2024-04-25T10:14:46.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.748 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.748 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141006 s, 74.4 MB/s 2024-04-25T10:14:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110971 s, 94.5 MB/s 2024-04-25T10:14:46.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.776 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.776 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104597 s, 100 MB/s 2024-04-25T10:14:46.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.793 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127565 s, 82.2 MB/s 2024-04-25T10:14:46.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.808 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.808 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012297 s, 85.3 MB/s 2024-04-25T10:14:46.824 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.824 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.824 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130613 s, 80.3 MB/s 2024-04-25T10:14:46.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.840 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.840 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129523 s, 81.0 MB/s 2024-04-25T10:14:46.855 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.855 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.855 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012835 s, 81.7 MB/s 2024-04-25T10:14:46.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.867 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917974 s, 114 MB/s 2024-04-25T10:14:46.883 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.883 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.883 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130229 s, 80.5 MB/s 2024-04-25T10:14:46.898 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.899 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.899 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124281 s, 84.4 MB/s 2024-04-25T10:14:46.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.913 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.914 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124424 s, 84.3 MB/s 2024-04-25T10:14:46.929 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.929 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.929 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127283 s, 82.4 MB/s 2024-04-25T10:14:46.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.944 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118154 s, 88.7 MB/s 2024-04-25T10:14:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.959 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126917 s, 82.6 MB/s 2024-04-25T10:14:46.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.973 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113605 s, 92.3 MB/s 2024-04-25T10:14:46.989 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:46.989 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:46.989 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130462 s, 80.4 MB/s 2024-04-25T10:14:47.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.004 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124027 s, 84.5 MB/s 2024-04-25T10:14:47.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.019 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119836 s, 87.5 MB/s 2024-04-25T10:14:47.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.034 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121343 s, 86.4 MB/s 2024-04-25T10:14:47.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.048 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114532 s, 91.6 MB/s 2024-04-25T10:14:47.063 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.063 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.063 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120674 s, 86.9 MB/s 2024-04-25T10:14:47.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:47.064 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:47.064 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:47.065 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:14:47.065 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:47.065 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:47.065 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:47.065 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:47.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.137 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.137 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139601 s, 75.1 MB/s 2024-04-25T10:14:47.153 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.153 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.153 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117353 s, 89.4 MB/s 2024-04-25T10:14:47.168 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.168 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.168 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124667 s, 84.1 MB/s 2024-04-25T10:14:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.184 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130374 s, 80.4 MB/s 2024-04-25T10:14:47.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.199 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114827 s, 91.3 MB/s 2024-04-25T10:14:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.214 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.215 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012577 s, 83.4 MB/s 2024-04-25T10:14:47.230 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.230 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.230 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116618 s, 89.9 MB/s 2024-04-25T10:14:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121351 s, 86.4 MB/s 2024-04-25T10:14:47.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.260 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114162 s, 91.8 MB/s 2024-04-25T10:14:47.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.276 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135779 s, 77.2 MB/s 2024-04-25T10:14:47.292 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.292 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.292 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127363 s, 82.3 MB/s 2024-04-25T10:14:47.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.307 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.307 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117263 s, 89.4 MB/s 2024-04-25T10:14:47.323 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.323 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.323 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135643 s, 77.3 MB/s 2024-04-25T10:14:47.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.339 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.339 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129214 s, 81.2 MB/s 2024-04-25T10:14:47.356 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.356 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.356 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136538 s, 76.8 MB/s 2024-04-25T10:14:47.372 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.372 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.372 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125839 s, 83.3 MB/s 2024-04-25T10:14:47.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.386 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.386 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110515 s, 94.9 MB/s 2024-04-25T10:14:47.402 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.402 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.403 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131477 s, 79.8 MB/s 2024-04-25T10:14:47.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.441 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.441 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130633 s, 80.3 MB/s 2024-04-25T10:14:47.456 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.456 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.456 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120942 s, 86.7 MB/s 2024-04-25T10:14:47.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.470 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.470 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011154 s, 94.0 MB/s 2024-04-25T10:14:47.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.482 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989602 s, 106 MB/s 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108508 s, 96.6 MB/s 2024-04-25T10:14:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012361 s, 84.8 MB/s 2024-04-25T10:14:47.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.527 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.527 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113136 s, 92.7 MB/s 2024-04-25T10:14:47.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.543 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.544 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125865 s, 83.3 MB/s 2024-04-25T10:14:47.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.559 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.559 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116526 s, 90.0 MB/s 2024-04-25T10:14:47.576 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.576 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.576 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121975 s, 86.0 MB/s 2024-04-25T10:14:47.591 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.592 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.592 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118605 s, 88.4 MB/s 2024-04-25T10:14:47.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.607 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.607 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119752 s, 87.6 MB/s 2024-04-25T10:14:47.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.622 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.622 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123298 s, 85.0 MB/s 2024-04-25T10:14:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117389 s, 89.3 MB/s 2024-04-25T10:14:47.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:47.638 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:47.638 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:47.638 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:14:47.638 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:47.638 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:47.638 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:47.638 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:47.706 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.706 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.707 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120332 s, 87.1 MB/s 2024-04-25T10:14:47.723 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.724 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.724 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141055 s, 74.3 MB/s 2024-04-25T10:14:47.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.738 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.738 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011601 s, 90.4 MB/s 2024-04-25T10:14:47.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.755 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133062 s, 78.8 MB/s 2024-04-25T10:14:47.770 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.771 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.771 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012494 s, 83.9 MB/s 2024-04-25T10:14:47.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.785 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.786 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120108 s, 87.3 MB/s 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.802 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134352 s, 78.0 MB/s 2024-04-25T10:14:47.817 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.817 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.817 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012759 s, 82.2 MB/s 2024-04-25T10:14:47.833 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.833 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.833 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128536 s, 81.6 MB/s 2024-04-25T10:14:47.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.849 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.850 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013475 s, 77.8 MB/s 2024-04-25T10:14:47.865 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.866 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.866 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130519 s, 80.3 MB/s 2024-04-25T10:14:47.880 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.881 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.881 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115471 s, 90.8 MB/s 2024-04-25T10:14:47.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.894 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.894 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107704 s, 97.4 MB/s 2024-04-25T10:14:47.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.909 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.909 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113713 s, 92.2 MB/s 2024-04-25T10:14:47.923 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.924 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.924 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119511 s, 87.7 MB/s 2024-04-25T10:14:47.940 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.941 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.941 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140708 s, 74.5 MB/s 2024-04-25T10:14:47.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.957 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131898 s, 79.5 MB/s 2024-04-25T10:14:47.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.973 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.973 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013274 s, 79.0 MB/s 2024-04-25T10:14:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:47.989 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:47.989 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123633 s, 84.8 MB/s 2024-04-25T10:14:48.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.005 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.005 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135064 s, 77.6 MB/s 2024-04-25T10:14:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.021 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128916 s, 81.3 MB/s 2024-04-25T10:14:48.036 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.036 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.036 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126474 s, 82.9 MB/s 2024-04-25T10:14:48.051 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.052 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.052 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125725 s, 83.4 MB/s 2024-04-25T10:14:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125093 s, 83.8 MB/s 2024-04-25T10:14:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.080 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100176 s, 105 MB/s 2024-04-25T10:14:48.094 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.094 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.094 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010896 s, 96.2 MB/s 2024-04-25T10:14:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.109 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.109 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121189 s, 86.5 MB/s 2024-04-25T10:14:48.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.124 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122736 s, 85.4 MB/s 2024-04-25T10:14:48.139 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.139 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.139 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123413 s, 85.0 MB/s 2024-04-25T10:14:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122563 s, 85.6 MB/s 2024-04-25T10:14:48.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.170 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.170 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01215 s, 86.3 MB/s 2024-04-25T10:14:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.183 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103454 s, 101 MB/s 2024-04-25T10:14:48.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:48.184 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:48.184 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:48.184 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:14:48.184 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:48.184 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:48.184 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:48.184 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:48.253 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.254 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.254 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114329 s, 91.7 MB/s 2024-04-25T10:14:48.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.270 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.270 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131494 s, 79.7 MB/s 2024-04-25T10:14:48.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.286 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.286 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126283 s, 83.0 MB/s 2024-04-25T10:14:48.301 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.301 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.301 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124103 s, 84.5 MB/s 2024-04-25T10:14:48.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.318 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134699 s, 77.8 MB/s 2024-04-25T10:14:48.334 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.334 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.334 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012089 s, 86.7 MB/s 2024-04-25T10:14:48.349 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.350 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.350 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128662 s, 81.5 MB/s 2024-04-25T10:14:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.362 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101446 s, 103 MB/s 2024-04-25T10:14:48.378 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.378 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.378 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125551 s, 83.5 MB/s 2024-04-25T10:14:48.394 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.394 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.394 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135331 s, 77.5 MB/s 2024-04-25T10:14:48.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.409 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120061 s, 87.3 MB/s 2024-04-25T10:14:48.447 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.447 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.448 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120452 s, 87.1 MB/s 2024-04-25T10:14:48.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.462 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.462 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011922 s, 88.0 MB/s 2024-04-25T10:14:48.477 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.477 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.477 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120808 s, 86.8 MB/s 2024-04-25T10:14:48.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.491 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.492 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116919 s, 89.7 MB/s 2024-04-25T10:14:48.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114216 s, 91.8 MB/s 2024-04-25T10:14:48.521 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.522 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.522 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121475 s, 86.3 MB/s 2024-04-25T10:14:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.536 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111766 s, 93.8 MB/s 2024-04-25T10:14:48.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.552 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.552 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125879 s, 83.3 MB/s 2024-04-25T10:14:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125803 s, 83.4 MB/s 2024-04-25T10:14:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118407 s, 88.6 MB/s 2024-04-25T10:14:48.598 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.598 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.598 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115228 s, 91.0 MB/s 2024-04-25T10:14:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114685 s, 91.4 MB/s 2024-04-25T10:14:48.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.630 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.630 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141173 s, 74.3 MB/s 2024-04-25T10:14:48.645 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.645 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.645 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127095 s, 82.5 MB/s 2024-04-25T10:14:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115617 s, 90.7 MB/s 2024-04-25T10:14:48.672 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.672 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.672 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100601 s, 104 MB/s 2024-04-25T10:14:48.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.686 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.686 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113723 s, 92.2 MB/s 2024-04-25T10:14:48.699 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.699 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.699 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108087 s, 97.0 MB/s 2024-04-25T10:14:48.713 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.713 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.713 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111525 s, 94.0 MB/s 2024-04-25T10:14:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132538 s, 79.1 MB/s 2024-04-25T10:14:48.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.741 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.741 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995086 s, 105 MB/s 2024-04-25T10:14:48.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:48.742 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:48.742 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:48.742 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:14:48.743 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:48.743 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:48.743 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:48.743 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:48.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.805 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.805 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116168 s, 90.3 MB/s 2024-04-25T10:14:48.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.817 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.817 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983991 s, 107 MB/s 2024-04-25T10:14:48.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.829 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.829 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100401 s, 104 MB/s 2024-04-25T10:14:48.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.842 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.842 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107999 s, 97.1 MB/s 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.856 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012239 s, 85.7 MB/s 2024-04-25T10:14:48.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.870 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.870 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119288 s, 87.9 MB/s 2024-04-25T10:14:48.883 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.883 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.883 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108268 s, 96.9 MB/s 2024-04-25T10:14:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.895 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100183 s, 105 MB/s 2024-04-25T10:14:48.907 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.910 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00947759 s, 111 MB/s 2024-04-25T10:14:48.919 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.919 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.919 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102693 s, 102 MB/s 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.933 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118293 s, 88.6 MB/s 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.945 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971057 s, 108 MB/s 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.957 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101795 s, 103 MB/s 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.969 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986097 s, 106 MB/s 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770715 s, 136 MB/s 2024-04-25T10:14:48.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:48.992 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:48.992 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011202 s, 93.6 MB/s 2024-04-25T10:14:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.004 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102243 s, 103 MB/s 2024-04-25T10:14:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.018 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117517 s, 89.2 MB/s 2024-04-25T10:14:49.028 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.028 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.028 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834547 s, 126 MB/s 2024-04-25T10:14:49.040 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.040 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.040 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103108 s, 102 MB/s 2024-04-25T10:14:49.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.053 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.053 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010215 s, 103 MB/s 2024-04-25T10:14:49.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.065 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100319 s, 105 MB/s 2024-04-25T10:14:49.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.077 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100403 s, 104 MB/s 2024-04-25T10:14:49.090 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.091 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.091 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110401 s, 95.0 MB/s 2024-04-25T10:14:49.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.103 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103176 s, 102 MB/s 2024-04-25T10:14:49.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.118 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012818 s, 81.8 MB/s 2024-04-25T10:14:49.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.131 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011051 s, 94.9 MB/s 2024-04-25T10:14:49.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.146 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121806 s, 86.1 MB/s 2024-04-25T10:14:49.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.160 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115003 s, 91.2 MB/s 2024-04-25T10:14:49.173 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.173 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.173 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109839 s, 95.5 MB/s 2024-04-25T10:14:49.183 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.183 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.183 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775833 s, 135 MB/s 2024-04-25T10:14:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.196 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104408 s, 100 MB/s 2024-04-25T10:14:49.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:49.197 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:49.197 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:49.198 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:14:49.198 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:49.198 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:49.198 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:49.198 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:49.262 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.262 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.262 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103951 s, 101 MB/s 2024-04-25T10:14:49.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.276 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121784 s, 86.1 MB/s 2024-04-25T10:14:49.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.290 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.290 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107985 s, 97.1 MB/s 2024-04-25T10:14:49.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.303 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109527 s, 95.7 MB/s 2024-04-25T10:14:49.316 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.316 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.316 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102254 s, 103 MB/s 2024-04-25T10:14:49.329 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.330 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.330 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104877 s, 100 MB/s 2024-04-25T10:14:49.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.341 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.341 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100267 s, 105 MB/s 2024-04-25T10:14:49.354 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.354 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.354 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106361 s, 98.6 MB/s 2024-04-25T10:14:49.368 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.369 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.369 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116955 s, 89.7 MB/s 2024-04-25T10:14:49.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.383 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119708 s, 87.6 MB/s 2024-04-25T10:14:49.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.396 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.396 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109415 s, 95.8 MB/s 2024-04-25T10:14:49.412 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.413 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.413 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140409 s, 74.7 MB/s 2024-04-25T10:14:49.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.450 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.450 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121985 s, 86.0 MB/s 2024-04-25T10:14:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.465 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116446 s, 90.0 MB/s 2024-04-25T10:14:49.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.479 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.479 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108005 s, 97.1 MB/s 2024-04-25T10:14:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124164 s, 84.5 MB/s 2024-04-25T10:14:49.508 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.508 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.509 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122033 s, 85.9 MB/s 2024-04-25T10:14:49.526 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.526 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.526 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01108 s, 94.6 MB/s 2024-04-25T10:14:49.537 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.537 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.537 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122211 s, 85.8 MB/s 2024-04-25T10:14:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.551 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118317 s, 88.6 MB/s 2024-04-25T10:14:49.568 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.568 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.568 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140441 s, 74.7 MB/s 2024-04-25T10:14:49.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.582 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106829 s, 98.2 MB/s 2024-04-25T10:14:49.596 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.596 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.596 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109259 s, 96.0 MB/s 2024-04-25T10:14:49.612 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.613 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137537 s, 76.2 MB/s 2024-04-25T10:14:49.627 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.627 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.627 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120744 s, 86.8 MB/s 2024-04-25T10:14:49.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.643 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.643 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128887 s, 81.4 MB/s 2024-04-25T10:14:49.658 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.658 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.658 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125103 s, 83.8 MB/s 2024-04-25T10:14:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129908 s, 80.7 MB/s 2024-04-25T10:14:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115351 s, 90.9 MB/s 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.703 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121576 s, 86.2 MB/s 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.719 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132331 s, 79.2 MB/s 2024-04-25T10:14:49.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.734 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.734 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110824 s, 94.6 MB/s 2024-04-25T10:14:49.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:49.735 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:49.735 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:49.735 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:14:49.735 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:49.735 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:49.735 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:49.735 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:49.802 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.802 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.802 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128884 s, 81.4 MB/s 2024-04-25T10:14:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.816 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120392 s, 87.1 MB/s 2024-04-25T10:14:49.831 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.832 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.832 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123825 s, 84.7 MB/s 2024-04-25T10:14:49.847 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.848 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.848 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127027 s, 82.5 MB/s 2024-04-25T10:14:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.864 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139784 s, 75.0 MB/s 2024-04-25T10:14:49.882 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.882 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.882 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146716 s, 71.5 MB/s 2024-04-25T10:14:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.900 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146088 s, 71.8 MB/s 2024-04-25T10:14:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012631 s, 83.0 MB/s 2024-04-25T10:14:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110757 s, 94.7 MB/s 2024-04-25T10:14:49.944 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.945 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.945 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117154 s, 89.5 MB/s 2024-04-25T10:14:49.956 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.956 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.956 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962698 s, 109 MB/s 2024-04-25T10:14:49.971 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.971 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.971 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118479 s, 88.5 MB/s 2024-04-25T10:14:49.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:49.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:49.986 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123377 s, 85.0 MB/s 2024-04-25T10:14:50.002 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.002 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.002 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135056 s, 77.6 MB/s 2024-04-25T10:14:50.018 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.019 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.019 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133165 s, 78.7 MB/s 2024-04-25T10:14:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.035 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130415 s, 80.4 MB/s 2024-04-25T10:14:50.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.048 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.048 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107827 s, 97.2 MB/s 2024-04-25T10:14:50.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.062 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.062 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112503 s, 93.2 MB/s 2024-04-25T10:14:50.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.077 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126309 s, 83.0 MB/s 2024-04-25T10:14:50.092 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.092 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.092 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118392 s, 88.6 MB/s 2024-04-25T10:14:50.103 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.104 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.104 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922899 s, 114 MB/s 2024-04-25T10:14:50.118 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.119 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.119 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123715 s, 84.8 MB/s 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.133 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123514 s, 84.9 MB/s 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.148 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117853 s, 89.0 MB/s 2024-04-25T10:14:50.163 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.163 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.164 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012863 s, 81.5 MB/s 2024-04-25T10:14:50.179 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.179 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.179 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124934 s, 83.9 MB/s 2024-04-25T10:14:50.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.194 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.194 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124039 s, 84.5 MB/s 2024-04-25T10:14:50.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.210 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.210 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134547 s, 77.9 MB/s 2024-04-25T10:14:50.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.222 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.223 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0096555 s, 109 MB/s 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.237 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115058 s, 91.1 MB/s 2024-04-25T10:14:50.252 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.252 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.252 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125566 s, 83.5 MB/s 2024-04-25T10:14:50.269 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.269 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.269 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136882 s, 76.6 MB/s 2024-04-25T10:14:50.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:50.270 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:50.270 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:50.270 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:14:50.270 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:50.270 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:50.270 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:50.271 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:50.345 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.345 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.345 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144963 s, 72.3 MB/s 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137672 s, 76.2 MB/s 2024-04-25T10:14:50.378 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.379 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137686 s, 76.2 MB/s 2024-04-25T10:14:50.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.392 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.393 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106484 s, 98.5 MB/s 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.411 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145475 s, 72.1 MB/s 2024-04-25T10:14:50.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.455 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.455 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014491 s, 72.4 MB/s 2024-04-25T10:14:50.471 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.472 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.472 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132834 s, 78.9 MB/s 2024-04-25T10:14:50.490 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.490 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.490 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154021 s, 68.1 MB/s 2024-04-25T10:14:50.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.507 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.507 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123112 s, 85.2 MB/s 2024-04-25T10:14:50.521 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.521 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.521 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011313 s, 92.7 MB/s 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115728 s, 90.6 MB/s 2024-04-25T10:14:50.550 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.551 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123678 s, 84.8 MB/s 2024-04-25T10:14:50.566 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.566 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.566 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012563 s, 83.5 MB/s 2024-04-25T10:14:50.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.582 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128167 s, 81.8 MB/s 2024-04-25T10:14:50.596 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.596 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.596 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120605 s, 86.9 MB/s 2024-04-25T10:14:50.611 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.611 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.611 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112524 s, 93.2 MB/s 2024-04-25T10:14:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010734 s, 97.7 MB/s 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137136 s, 76.5 MB/s 2024-04-25T10:14:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.656 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121579 s, 86.2 MB/s 2024-04-25T10:14:50.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.669 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.669 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101882 s, 103 MB/s 2024-04-25T10:14:50.684 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.684 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.684 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121176 s, 86.5 MB/s 2024-04-25T10:14:50.698 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.699 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011896 s, 88.1 MB/s 2024-04-25T10:14:50.716 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.717 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.717 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015286 s, 68.6 MB/s 2024-04-25T10:14:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.730 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109108 s, 96.1 MB/s 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.745 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124184 s, 84.4 MB/s 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.760 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120827 s, 86.8 MB/s 2024-04-25T10:14:50.774 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.775 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012094 s, 86.7 MB/s 2024-04-25T10:14:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.792 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138767 s, 75.6 MB/s 2024-04-25T10:14:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.807 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129762 s, 80.8 MB/s 2024-04-25T10:14:50.823 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.823 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.823 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138762 s, 75.6 MB/s 2024-04-25T10:14:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118457 s, 88.5 MB/s 2024-04-25T10:14:50.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.853 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116189 s, 90.2 MB/s 2024-04-25T10:14:50.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:50.854 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:50.854 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:50.854 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:14:50.854 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:50.855 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:50.855 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:50.855 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108524 s, 96.6 MB/s 2024-04-25T10:14:50.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.935 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.935 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103543 s, 101 MB/s 2024-04-25T10:14:50.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.953 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149036 s, 70.4 MB/s 2024-04-25T10:14:50.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.970 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.970 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135849 s, 77.2 MB/s 2024-04-25T10:14:50.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.985 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.986 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013036 s, 80.4 MB/s 2024-04-25T10:14:50.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:50.999 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:50.999 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010003 s, 105 MB/s 2024-04-25T10:14:51.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.013 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.013 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109865 s, 95.4 MB/s 2024-04-25T10:14:51.028 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.029 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.029 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124438 s, 84.3 MB/s 2024-04-25T10:14:51.046 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.046 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.046 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140601 s, 74.6 MB/s 2024-04-25T10:14:51.061 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.061 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.061 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119268 s, 87.9 MB/s 2024-04-25T10:14:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135087 s, 77.6 MB/s 2024-04-25T10:14:51.092 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.093 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.093 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121939 s, 86.0 MB/s 2024-04-25T10:14:51.111 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.111 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.111 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121746 s, 86.1 MB/s 2024-04-25T10:14:51.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.124 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.124 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130051 s, 80.6 MB/s 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131283 s, 79.9 MB/s 2024-04-25T10:14:51.154 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.154 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.154 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104949 s, 99.9 MB/s 2024-04-25T10:14:51.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.169 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.169 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122719 s, 85.4 MB/s 2024-04-25T10:14:51.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.185 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.185 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131869 s, 79.5 MB/s 2024-04-25T10:14:51.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.199 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.199 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107659 s, 97.4 MB/s 2024-04-25T10:14:51.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.215 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.216 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136052 s, 77.1 MB/s 2024-04-25T10:14:51.229 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.229 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.230 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109971 s, 95.3 MB/s 2024-04-25T10:14:51.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.245 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126547 s, 82.9 MB/s 2024-04-25T10:14:51.261 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.262 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.262 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133552 s, 78.5 MB/s 2024-04-25T10:14:51.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.276 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.276 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112861 s, 92.9 MB/s 2024-04-25T10:14:51.292 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.292 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.292 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013139 s, 79.8 MB/s 2024-04-25T10:14:51.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.306 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.306 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104621 s, 100 MB/s 2024-04-25T10:14:51.321 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.322 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124397 s, 84.3 MB/s 2024-04-25T10:14:51.336 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.336 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.337 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119147 s, 88.0 MB/s 2024-04-25T10:14:51.352 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.352 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.352 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121509 s, 86.3 MB/s 2024-04-25T10:14:51.370 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.371 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.371 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135782 s, 77.2 MB/s 2024-04-25T10:14:51.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.383 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.383 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117183 s, 89.5 MB/s 2024-04-25T10:14:51.399 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.399 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.399 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121501 s, 86.3 MB/s 2024-04-25T10:14:51.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:51.400 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:51.400 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:51.400 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:14:51.400 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:51.400 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:51.400 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:51.400 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:51.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.469 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.469 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119674 s, 87.6 MB/s 2024-04-25T10:14:51.482 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.483 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.483 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115959 s, 90.4 MB/s 2024-04-25T10:14:51.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.494 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.494 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929806 s, 113 MB/s 2024-04-25T10:14:51.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.506 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.506 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895935 s, 117 MB/s 2024-04-25T10:14:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.519 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114126 s, 91.9 MB/s 2024-04-25T10:14:51.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.533 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.533 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109616 s, 95.7 MB/s 2024-04-25T10:14:51.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.546 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.546 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111299 s, 94.2 MB/s 2024-04-25T10:14:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.560 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111211 s, 94.3 MB/s 2024-04-25T10:14:51.574 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.574 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.574 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117639 s, 89.1 MB/s 2024-04-25T10:14:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011322 s, 92.6 MB/s 2024-04-25T10:14:51.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.602 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.602 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108594 s, 96.6 MB/s 2024-04-25T10:14:51.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.618 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.618 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129531 s, 81.0 MB/s 2024-04-25T10:14:51.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.634 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.634 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130333 s, 80.5 MB/s 2024-04-25T10:14:51.650 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.651 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.651 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129145 s, 81.2 MB/s 2024-04-25T10:14:51.665 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.666 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.666 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117733 s, 89.1 MB/s 2024-04-25T10:14:51.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.681 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.681 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125199 s, 83.8 MB/s 2024-04-25T10:14:51.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.695 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.695 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113609 s, 92.3 MB/s 2024-04-25T10:14:51.707 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.708 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.708 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955634 s, 110 MB/s 2024-04-25T10:14:51.722 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.722 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.722 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119609 s, 87.7 MB/s 2024-04-25T10:14:51.735 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.735 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010911 s, 96.1 MB/s 2024-04-25T10:14:51.750 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.750 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.750 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115489 s, 90.8 MB/s 2024-04-25T10:14:51.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.764 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.764 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112953 s, 92.8 MB/s 2024-04-25T10:14:51.780 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139835 s, 75.0 MB/s 2024-04-25T10:14:51.795 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.795 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.795 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011028 s, 95.1 MB/s 2024-04-25T10:14:51.810 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.810 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.810 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125704 s, 83.4 MB/s 2024-04-25T10:14:51.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.825 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.825 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124065 s, 84.5 MB/s 2024-04-25T10:14:51.838 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.839 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.839 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111027 s, 94.4 MB/s 2024-04-25T10:14:51.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.853 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.853 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121799 s, 86.1 MB/s 2024-04-25T10:14:51.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.867 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.867 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112563 s, 93.2 MB/s 2024-04-25T10:14:51.882 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.882 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.882 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116127 s, 90.3 MB/s 2024-04-25T10:14:51.899 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.899 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.900 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147512 s, 71.1 MB/s 2024-04-25T10:14:51.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.916 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.916 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134125 s, 78.2 MB/s 2024-04-25T10:14:51.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:51.917 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:51.917 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:14:51.917 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:14:51.917 DEBUG:teuthology.orchestra.run.smithi133:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:51.917 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:51.917 DEBUG:teuthology.orchestra.run.smithi133:> done 2024-04-25T10:14:51.917 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:14:51.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:51.986 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:51.986 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121793 s, 86.1 MB/s 2024-04-25T10:14:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.004 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151325 s, 69.3 MB/s 2024-04-25T10:14:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136263 s, 77.0 MB/s 2024-04-25T10:14:52.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.034 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.034 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010719 s, 97.8 MB/s 2024-04-25T10:14:52.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.049 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.049 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132439 s, 79.2 MB/s 2024-04-25T10:14:52.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.065 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.066 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013341 s, 78.6 MB/s 2024-04-25T10:14:52.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.082 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.082 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134366 s, 78.0 MB/s 2024-04-25T10:14:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.099 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139503 s, 75.2 MB/s 2024-04-25T10:14:52.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.114 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.114 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123807 s, 84.7 MB/s 2024-04-25T10:14:52.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.131 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134593 s, 77.9 MB/s 2024-04-25T10:14:52.145 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.145 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.145 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110525 s, 94.9 MB/s 2024-04-25T10:14:52.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.160 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.160 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118074 s, 88.8 MB/s 2024-04-25T10:14:52.173 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.173 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.173 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100333 s, 105 MB/s 2024-04-25T10:14:52.188 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123599 s, 84.8 MB/s 2024-04-25T10:14:52.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.201 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.201 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961534 s, 109 MB/s 2024-04-25T10:14:52.216 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.216 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.216 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012399 s, 84.6 MB/s 2024-04-25T10:14:52.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.231 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.231 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125822 s, 83.3 MB/s 2024-04-25T10:14:52.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.245 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.245 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113257 s, 92.6 MB/s 2024-04-25T10:14:52.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.259 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.259 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112015 s, 93.6 MB/s 2024-04-25T10:14:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.273 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116616 s, 89.9 MB/s 2024-04-25T10:14:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117315 s, 89.4 MB/s 2024-04-25T10:14:52.303 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.304 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.304 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128612 s, 81.5 MB/s 2024-04-25T10:14:52.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.318 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.318 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122902 s, 85.3 MB/s 2024-04-25T10:14:52.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.332 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.333 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115266 s, 91.0 MB/s 2024-04-25T10:14:52.347 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.347 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.347 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118225 s, 88.7 MB/s 2024-04-25T10:14:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012302 s, 85.2 MB/s 2024-04-25T10:14:52.376 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.377 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.377 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122019 s, 85.9 MB/s 2024-04-25T10:14:52.394 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.394 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.394 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128167 s, 81.8 MB/s 2024-04-25T10:14:52.408 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.409 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.409 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138534 s, 75.7 MB/s 2024-04-25T10:14:52.426 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.426 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.426 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148166 s, 70.8 MB/s 2024-04-25T10:14:52.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.454 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.454 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100349 s, 104 MB/s 2024-04-25T10:14:52.468 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T10:14:52.469 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T10:14:52.469 INFO:teuthology.orchestra.run.smithi133.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012032 s, 87.1 MB/s 2024-04-25T10:14:52.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T10:14:52.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.639+0000 7feb62ca5640 1 Processor -- start 2024-04-25T10:14:52.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb62ca5640 1 -- start start 2024-04-25T10:14:52.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb62ca5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feb5c0cd820 0x7feb5c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb62ca5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5c0ce2e0 con 0x7feb5c15ced0 2024-04-25T10:14:52.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb62ca5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5c06d4b0 con 0x7feb5c145170 2024-04-25T10:14:52.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb62ca5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5c06db00 con 0x7feb5c0cd820 2024-04-25T10:14:52.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb5bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feb5c0cd820 0x7feb5c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb5bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feb5c0cd820 0x7feb5c0cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43744/0 (socket says 172.21.15.133:43744) 2024-04-25T10:14:52.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb61242640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7feb5c15ced0 0x7feb5c0b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46792/0 (socket says 172.21.15.133:46792) 2024-04-25T10:14:52.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb5bfff640 1 -- 172.21.15.133:0/4084796822 learned_addr learned my addr 172.21.15.133:0/4084796822 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:52.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb5bfff640 1 -- 172.21.15.133:0/4084796822 >> v1:172.21.15.162:6789/0 conn(0x7feb5c145170 legacy=0x7feb5c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb5bfff640 1 -- 172.21.15.133:0/4084796822 >> v1:172.21.15.133:6789/0 conn(0x7feb5c15ced0 legacy=0x7feb5c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.643+0000 7feb5bfff640 1 -- 172.21.15.133:0/4084796822 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c06d790 con 0x7feb5c0cd820 2024-04-25T10:14:52.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb5bfff640 1 --2- 172.21.15.133:0/4084796822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb5c0cd820 0x7feb5c0cdca0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7feb50009870 tx=0x7feb5001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1438df77f2cb5715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb5b7fe640 1 -- 172.21.15.133:0/4084796822 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb500b0070 con 0x7feb5c0cd820 2024-04-25T10:14:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb5b7fe640 1 -- 172.21.15.133:0/4084796822 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb500ab600 con 0x7feb5c0cd820 2024-04-25T10:14:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb5b7fe640 1 -- 172.21.15.133:0/4084796822 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb500ab9d0 con 0x7feb5c0cd820 2024-04-25T10:14:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 -- 172.21.15.133:0/4084796822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb5c0cd820 msgr2=0x7feb5c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 --2- 172.21.15.133:0/4084796822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb5c0cd820 0x7feb5c0cdca0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7feb50009870 tx=0x7feb5001c520 comp rx=0 tx=0).stop 2024-04-25T10:14:52.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 -- 172.21.15.133:0/4084796822 shutdown_connections 2024-04-25T10:14:52.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 --2- 172.21.15.133:0/4084796822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb5c0cd820 0x7feb5c0cdca0 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 -- 172.21.15.133:0/4084796822 >> 172.21.15.133:0/4084796822 conn(0x7feb5c066f70 msgr2=0x7feb5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 -- 172.21.15.133:0/4084796822 shutdown_connections 2024-04-25T10:14:52.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 -- 172.21.15.133:0/4084796822 wait complete. 2024-04-25T10:14:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 Processor -- start 2024-04-25T10:14:52.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 -- start start 2024-04-25T10:14:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb5c0cd820 0x7feb5c1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb5c145170 0x7feb5c13d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb5c15ced0 0x7feb5c0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c15d2e0 con 0x7feb5c0cd820 2024-04-25T10:14:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c145550 con 0x7feb5c15ced0 2024-04-25T10:14:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb62ca5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c15d460 con 0x7feb5c145170 2024-04-25T10:14:52.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb60a41640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb5c0cd820 0x7feb5c1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb60a41640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb5c0cd820 0x7feb5c1563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43752/0 (socket says 172.21.15.133:43752) 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.647+0000 7feb60a41640 1 -- 172.21.15.133:0/1500817567 learned_addr learned my addr 172.21.15.133:0/1500817567 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb61242640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb5c15ced0 0x7feb5c0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb60a41640 1 -- 172.21.15.133:0/1500817567 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb5c145170 msgr2=0x7feb5c13d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb60a41640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb5c145170 0x7feb5c13d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb60a41640 1 -- 172.21.15.133:0/1500817567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb5c15ced0 msgr2=0x7feb5c0b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb60a41640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb5c15ced0 0x7feb5c0b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb60a41640 1 -- 172.21.15.133:0/1500817567 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c1f6b40 con 0x7feb5c0cd820 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb60a41640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb5c0cd820 0x7feb5c1563d0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00c450 tx=0x7feb4c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb597fa640 1 -- 172.21.15.133:0/1500817567 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c012830 con 0x7feb5c0cd820 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb597fa640 1 -- 172.21.15.133:0/1500817567 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c017070 con 0x7feb5c0cd820 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb62ca5640 1 -- 172.21.15.133:0/1500817567 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5c154280 con 0x7feb5c0cd820 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb597fa640 1 -- 172.21.15.133:0/1500817567 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c012c30 con 0x7feb5c0cd820 2024-04-25T10:14:52.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb62ca5640 1 -- 172.21.15.133:0/1500817567 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c154840 con 0x7feb5c0cd820 2024-04-25T10:14:52.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb62ca5640 1 -- 172.21.15.133:0/1500817567 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5c068e80 con 0x7feb5c0cd820 2024-04-25T10:14:52.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb597fa640 1 -- 172.21.15.133:0/1500817567 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb4c01c460 con 0x7feb5c0cd820 2024-04-25T10:14:52.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb597fa640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb48066f60 0x7feb48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb597fa640 1 -- 172.21.15.133:0/1500817567 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7feb4c084fa0 con 0x7feb5c0cd820 2024-04-25T10:14:52.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb5bfff640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb48066f60 0x7feb48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.651+0000 7feb5bfff640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb48066f60 0x7feb48069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7feb500ad000 tx=0x7feb50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.655+0000 7feb597fa640 1 -- 172.21.15.133:0/1500817567 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb4c055df0 con 0x7feb5c0cd820 2024-04-25T10:14:52.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:52.855+0000 7feb62ca5640 1 -- 172.21.15.133:0/1500817567 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7feb5c142250 con 0x7feb5c0cd820 2024-04-25T10:14:53.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.035+0000 7feb597fa640 1 -- 172.21.15.133:0/1500817567 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v91) v1 ==== 105+0+0 (secure 0 0 0) 0x7feb4c0556f0 con 0x7feb5c0cd820 2024-04-25T10:14:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 -- 172.21.15.133:0/1500817567 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb48066f60 msgr2=0x7feb48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb48066f60 0x7feb48069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7feb500ad000 tx=0x7feb50008040 comp rx=0 tx=0).stop 2024-04-25T10:14:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 -- 172.21.15.133:0/1500817567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb5c0cd820 msgr2=0x7feb5c1563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb5c0cd820 0x7feb5c1563d0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00c450 tx=0x7feb4c00c910 comp rx=0 tx=0).stop 2024-04-25T10:14:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 -- 172.21.15.133:0/1500817567 shutdown_connections 2024-04-25T10:14:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feb5c15ced0 0x7feb5c0b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feb5c145170 0x7feb5c13d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7feb48066f60 0x7feb48069420 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 --2- 172.21.15.133:0/1500817567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb5c0cd820 0x7feb5c1563d0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 -- 172.21.15.133:0/1500817567 >> 172.21.15.133:0/1500817567 conn(0x7feb5c066f70 msgr2=0x7feb5c0acb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 -- 172.21.15.133:0/1500817567 shutdown_connections 2024-04-25T10:14:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.039+0000 7feb62ca5640 1 -- 172.21.15.133:0/1500817567 wait complete. 2024-04-25T10:14:53.060 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T10:14:53.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f937aca3640 1 Processor -- start 2024-04-25T10:14:53.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f937aca3640 1 -- start start 2024-04-25T10:14:53.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f937aca3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937415eb40 0x7f93740b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f937aca3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93741471f0 con 0x7f937406e8b0 2024-04-25T10:14:53.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f937aca3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93740b1090 con 0x7f93740aebc0 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f937aca3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9374147490 con 0x7f937415eb40 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9378a3f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f937406e8b0 0x7f9374146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46808/0 (socket says 172.21.15.133:46808) 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9378a3f640 1 -- 172.21.15.133:0/2692664829 learned_addr learned my addr 172.21.15.133:0/2692664829 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006747553 0 0) 0x7f93741471f0 con 0x7f937406e8b0 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9360003530 con 0x7f937406e8b0 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9373fff640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937415eb40 0x7f93740b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1803403171 0 0) 0x7f9360003530 con 0x7f937406e8b0 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937415eb40 msgr2=0x7f93740b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937415eb40 0x7f93740b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 >> v1:172.21.15.162:6789/0 conn(0x7f93740aebc0 legacy=0x7f937406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93740b1370 con 0x7f937406e8b0 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f9364001680 con 0x7f937406e8b0 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 >> v1:172.21.15.133:6789/0 conn(0x7f937406e8b0 legacy=0x7f9374146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93600039f0 0x7f9360003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9360004810 0x7f9360004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937415eb40 0x7f936000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9374147490 con 0x7f9360004810 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9360005530 con 0x7f93600039f0 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936000bf10 con 0x7f937415eb40 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9373fff640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9360004810 0x7f9360004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9378a3f640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93600039f0 0x7f9360003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9379240640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937415eb40 0x7f936000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9373fff640 1 -- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937415eb40 msgr2=0x7f936000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9373fff640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937415eb40 0x7f936000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9373fff640 1 -- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93600039f0 msgr2=0x7f9360003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9373fff640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f93600039f0 0x7f9360003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9373fff640 1 -- 172.21.15.133:0/2692664829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9360003530 con 0x7f9360004810 2024-04-25T10:14:53.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.183+0000 7f9379240640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f937415eb40 0x7f936000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f9373fff640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9360004810 0x7f9360004c90 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f935c007360 tx=0x7f935c01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c3b81fe5cdf00e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c0b2070 con 0x7f9360004810 2024-04-25T10:14:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c0a5b20 con 0x7f9360004810 2024-04-25T10:14:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f93737fe640 1 -- 172.21.15.133:0/2692664829 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c009be0 con 0x7f9360004810 2024-04-25T10:14:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f937aca3640 1 -- 172.21.15.133:0/2692664829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9360004810 msgr2=0x7f9360004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f937aca3640 1 --2- 172.21.15.133:0/2692664829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9360004810 0x7f9360004c90 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f935c007360 tx=0x7f935c01ce40 comp rx=0 tx=0).stop 2024-04-25T10:14:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f9379240640 1 -- 172.21.15.133:0/2692664829 reap_dead start 2024-04-25T10:14:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f937aca3640 1 -- 172.21.15.133:0/2692664829 shutdown_connections 2024-04-25T10:14:53.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f937aca3640 1 -- 172.21.15.133:0/2692664829 >> 172.21.15.133:0/2692664829 conn(0x7f9374066f70 msgr2=0x7f937406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f937aca3640 1 -- 172.21.15.133:0/2692664829 shutdown_connections 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f937aca3640 1 -- 172.21.15.133:0/2692664829 wait complete. 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f937aca3640 1 Processor -- start 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.187+0000 7f937aca3640 1 -- start start 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f937aca3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9360004810 0x7f9374142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f937aca3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93741625e0 0x7f9374140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f937aca3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9374163b10 0x7f937415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f937aca3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93740b1e30 con 0x7f93741625e0 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f937aca3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93740b19c0 con 0x7f9360004810 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f937aca3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93740b1fb0 con 0x7f9374163b10 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9373fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93741625e0 0x7f9374140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9373fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93741625e0 0x7f9374140990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43760/0 (socket says 172.21.15.133:43760) 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9373fff640 1 -- 172.21.15.133:0/2365033501 learned_addr learned my addr 172.21.15.133:0/2365033501 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:53.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9378a3f640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9360004810 0x7f9374142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9379240640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9374163b10 0x7f937415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9373fff640 1 -- 172.21.15.133:0/2365033501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9374163b10 msgr2=0x7f937415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9373fff640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9374163b10 0x7f937415cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9373fff640 1 -- 172.21.15.133:0/2365033501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9360004810 msgr2=0x7f9374142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9373fff640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9360004810 0x7f9374142290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9373fff640 1 -- 172.21.15.133:0/2365033501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9374162a70 con 0x7f93741625e0 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9378a3f640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9360004810 0x7f9374142290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9373fff640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93741625e0 0x7f9374140990 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f935c000f90 tx=0x7f9360005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f93717fa640 1 -- 172.21.15.133:0/2365033501 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c0b2040 con 0x7f93741625e0 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f937aca3640 1 -- 172.21.15.133:0/2365033501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9374162d50 con 0x7f93741625e0 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f937aca3640 1 -- 172.21.15.133:0/2365033501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9374163310 con 0x7f93741625e0 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f93717fa640 1 -- 172.21.15.133:0/2365033501 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c0a5b20 con 0x7f93741625e0 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f93717fa640 1 -- 172.21.15.133:0/2365033501 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c0adb20 con 0x7f93741625e0 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f937aca3640 1 -- 172.21.15.133:0/2365033501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93580052a0 con 0x7f93741625e0 2024-04-25T10:14:53.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f93717fa640 1 -- 172.21.15.133:0/2365033501 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f935c0bf060 con 0x7f93741625e0 2024-04-25T10:14:53.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f93717fa640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9354066f60 0x7f9354069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f93717fa640 1 -- 172.21.15.133:0/2365033501 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f935c11f300 con 0x7f93741625e0 2024-04-25T10:14:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9378a3f640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9354066f60 0x7f9354069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.191+0000 7f9378a3f640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9354066f60 0x7f9354069420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9364000f70 tx=0x7f9364000ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.195+0000 7f93717fa640 1 -- 172.21.15.133:0/2365033501 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f935c016b50 con 0x7f93741625e0 2024-04-25T10:14:53.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:53.391+0000 7f937aca3640 1 -- 172.21.15.133:0/2365033501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f9358005860 con 0x7f93741625e0 2024-04-25T10:14:53.646 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:53.650 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:54.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.047+0000 7f93717fa640 1 -- 172.21.15.133:0/2365033501 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v92) v1 ==== 121+0+0 (secure 0 0 0) 0x7f935c0f0160 con 0x7f93741625e0 2024-04-25T10:14:54.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 -- 172.21.15.133:0/2365033501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9354066f60 msgr2=0x7f9354069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9354066f60 0x7f9354069420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9364000f70 tx=0x7f9364000ee0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 -- 172.21.15.133:0/2365033501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93741625e0 msgr2=0x7f9374140990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93741625e0 0x7f9374140990 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f935c000f90 tx=0x7f9360005840 comp rx=0 tx=0).stop 2024-04-25T10:14:54.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 -- 172.21.15.133:0/2365033501 shutdown_connections 2024-04-25T10:14:54.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9374163b10 0x7f937415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9354066f60 0x7f9354069420 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93741625e0 0x7f9374140990 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 --2- 172.21.15.133:0/2365033501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9360004810 0x7f9374142290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 -- 172.21.15.133:0/2365033501 >> 172.21.15.133:0/2365033501 conn(0x7f9374066f70 msgr2=0x7f937406d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 -- 172.21.15.133:0/2365033501 shutdown_connections 2024-04-25T10:14:54.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.051+0000 7f937aca3640 1 -- 172.21.15.133:0/2365033501 wait complete. 2024-04-25T10:14:54.080 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:54.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.195+0000 7fac89101640 1 Processor -- start 2024-04-25T10:14:54.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.195+0000 7fac89101640 1 -- start start 2024-04-25T10:14:54.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.195+0000 7fac89101640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fac8415db00 0x7fac840b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.195+0000 7fac89101640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fac840d0770 0x7fac840d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.195+0000 7fac89101640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac841614c0 0x7fac84161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.195+0000 7fac89101640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac840d1110 con 0x7fac840d0770 2024-04-25T10:14:54.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.195+0000 7fac89101640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac840ceb20 con 0x7fac8415db00 2024-04-25T10:14:54.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.195+0000 7fac89101640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac840b7740 con 0x7fac841614c0 2024-04-25T10:14:54.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.195+0000 7fac8259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fac840d0770 0x7fac840d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac8259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fac840d0770 0x7fac840d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43772/0 (socket says 172.21.15.133:43772) 2024-04-25T10:14:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac8259c640 1 -- 172.21.15.133:0/2567761298 learned_addr learned my addr 172.21.15.133:0/2567761298 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac82d9d640 1 --2- 172.21.15.133:0/2567761298 >> v2:172.21.15.162:3300/0 conn(0x7fac8415db00 0x7fac840b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac8359e640 1 --2- 172.21.15.133:0/2567761298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac841614c0 0x7fac84161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac8259c640 1 -- 172.21.15.133:0/2567761298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac841614c0 msgr2=0x7fac84161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac8259c640 1 --2- 172.21.15.133:0/2567761298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac841614c0 0x7fac84161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac8259c640 1 -- 172.21.15.133:0/2567761298 >> v2:172.21.15.162:3300/0 conn(0x7fac8415db00 msgr2=0x7fac840b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac8259c640 1 --2- 172.21.15.133:0/2567761298 >> v2:172.21.15.162:3300/0 conn(0x7fac8415db00 0x7fac840b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac8259c640 1 -- 172.21.15.133:0/2567761298 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac840b7a20 con 0x7fac840d0770 2024-04-25T10:14:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac82d9d640 1 --2- 172.21.15.133:0/2567761298 >> v2:172.21.15.162:3300/0 conn(0x7fac8415db00 0x7fac840b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac8259c640 1 --2- 172.21.15.133:0/2567761298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac840d0770 0x7fac840d0bd0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fac840b53d0 tx=0x7fac6c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eec1d86a35eba86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac81d9b640 1 -- 172.21.15.133:0/2567761298 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0aa610 con 0x7fac840d0770 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac81d9b640 1 -- 172.21.15.133:0/2567761298 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c0af070 con 0x7fac840d0770 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac81d9b640 1 -- 172.21.15.133:0/2567761298 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0aa9d0 con 0x7fac840d0770 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac89101640 1 -- 172.21.15.133:0/2567761298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac840d0770 msgr2=0x7fac840d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac89101640 1 --2- 172.21.15.133:0/2567761298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac840d0770 0x7fac840d0bd0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fac840b53d0 tx=0x7fac6c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac89101640 1 -- 172.21.15.133:0/2567761298 shutdown_connections 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac89101640 1 --2- 172.21.15.133:0/2567761298 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac841614c0 0x7fac84161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac89101640 1 --2- 172.21.15.133:0/2567761298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac840d0770 0x7fac840d0bd0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac89101640 1 --2- 172.21.15.133:0/2567761298 >> v2:172.21.15.162:3300/0 conn(0x7fac8415db00 0x7fac840b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac89101640 1 -- 172.21.15.133:0/2567761298 >> 172.21.15.133:0/2567761298 conn(0x7fac840727c0 msgr2=0x7fac84072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac89101640 1 -- 172.21.15.133:0/2567761298 shutdown_connections 2024-04-25T10:14:54.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.199+0000 7fac89101640 1 -- 172.21.15.133:0/2567761298 wait complete. 2024-04-25T10:14:54.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac89101640 1 Processor -- start 2024-04-25T10:14:54.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac89101640 1 -- start start 2024-04-25T10:14:54.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac89101640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac840d0770 0x7fac84142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac89101640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac8415db00 0x7fac841561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac89101640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac841614c0 0x7fac841566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac89101640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac840b7430 con 0x7fac841614c0 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac89101640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac840b82c0 con 0x7fac8415db00 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac89101640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac840d18a0 con 0x7fac840d0770 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac8359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac841614c0 0x7fac841566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac8359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac841614c0 0x7fac841566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43788/0 (socket says 172.21.15.133:43788) 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac8359e640 1 -- 172.21.15.133:0/3925520365 learned_addr learned my addr 172.21.15.133:0/3925520365 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac82d9d640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac840d0770 0x7fac84142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac8259c640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac8415db00 0x7fac841561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac8359e640 1 -- 172.21.15.133:0/3925520365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac840d0770 msgr2=0x7fac84142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac8359e640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac840d0770 0x7fac84142da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac8359e640 1 -- 172.21.15.133:0/3925520365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac8415db00 msgr2=0x7fac841561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac8359e640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac8415db00 0x7fac841561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac8359e640 1 -- 172.21.15.133:0/3925520365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac84155fb0 con 0x7fac841614c0 2024-04-25T10:14:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac8259c640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac8415db00 0x7fac841561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac8359e640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac841614c0 0x7fac841566f0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00c7f0 tx=0x7fac7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac7b7fe640 1 -- 172.21.15.133:0/3925520365 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c015070 con 0x7fac841614c0 2024-04-25T10:14:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac7b7fe640 1 -- 172.21.15.133:0/3925520365 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c004220 con 0x7fac841614c0 2024-04-25T10:14:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac89101640 1 -- 172.21.15.133:0/3925520365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac84158c60 con 0x7fac841614c0 2024-04-25T10:14:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac7b7fe640 1 -- 172.21.15.133:0/3925520365 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c003620 con 0x7fac841614c0 2024-04-25T10:14:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac89101640 1 -- 172.21.15.133:0/3925520365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac84159220 con 0x7fac841614c0 2024-04-25T10:14:54.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac89101640 1 -- 172.21.15.133:0/3925520365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac84068ef0 con 0x7fac841614c0 2024-04-25T10:14:54.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.203+0000 7fac7b7fe640 1 -- 172.21.15.133:0/3925520365 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac7c0039d0 con 0x7fac841614c0 2024-04-25T10:14:54.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.207+0000 7fac7b7fe640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac64066f60 0x7fac64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.207+0000 7fac7b7fe640 1 -- 172.21.15.133:0/3925520365 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fac7c085200 con 0x7fac841614c0 2024-04-25T10:14:54.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.207+0000 7fac82d9d640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac64066f60 0x7fac64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.207+0000 7fac82d9d640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac64066f60 0x7fac64069420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fac74007fc0 tx=0x7fac74007550 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.211+0000 7fac7b7fe640 1 -- 172.21.15.133:0/3925520365 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac84155fb0 con 0x7fac841614c0 2024-04-25T10:14:54.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 -- 172.21.15.133:0/3925520365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac8415c620 con 0x7fac841614c0 2024-04-25T10:14:54.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac7b7fe640 1 -- 172.21.15.133:0/3925520365 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4966 (secure 0 0 0) 0x7fac7c056110 con 0x7fac841614c0 2024-04-25T10:14:54.406 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:54.406 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":93,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3448301688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3448301688},{"type":"v1","addr":"172.21.15.162:6833","nonce":3448301688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/4188704231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":4188704231},{"type":"v1","addr":"172.21.15.162:6835","nonce":4188704231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":93,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:14:28.005823+0000","modified":"2024-04-25T10:14:54.057373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2626366082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2626366082},{"type":"v1","addr":"172.21.15.133:6835","nonce":2626366082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10050":{"gid":10050,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3234096063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3234096063},{"type":"v1","addr":"172.21.15.133:6837","nonce":3234096063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:14:54.407 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 93 2024-04-25T10:14:54.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 -- 172.21.15.133:0/3925520365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac64066f60 msgr2=0x7fac64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac64066f60 0x7fac64069420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fac74007fc0 tx=0x7fac74007550 comp rx=0 tx=0).stop 2024-04-25T10:14:54.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 -- 172.21.15.133:0/3925520365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac841614c0 msgr2=0x7fac841566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac841614c0 0x7fac841566f0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00c7f0 tx=0x7fac7c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 -- 172.21.15.133:0/3925520365 shutdown_connections 2024-04-25T10:14:54.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac64066f60 0x7fac64069420 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac841614c0 0x7fac841566f0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac8415db00 0x7fac841561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 --2- 172.21.15.133:0/3925520365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac840d0770 0x7fac84142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 -- 172.21.15.133:0/3925520365 >> 172.21.15.133:0/3925520365 conn(0x7fac840727c0 msgr2=0x7fac8415ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 -- 172.21.15.133:0/3925520365 shutdown_connections 2024-04-25T10:14:54.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.403+0000 7fac89101640 1 -- 172.21.15.133:0/3925520365 wait complete. 2024-04-25T10:14:54.828 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T10:14:28.005823+0000', 'modified': '2024-04-25T10:14:54.057373+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10050}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10041': {'gid': 10041, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/2626366082', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 2626366082}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 2626366082}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10050': {'gid': 10050, 'name': 'a', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/3234096063', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 3234096063}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3234096063}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:54.828 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:14:54.829 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:14:54.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a64d40640 1 Processor -- start 2024-04-25T10:14:54.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a64d40640 1 -- start start 2024-04-25T10:14:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a64d40640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1a6015aa50 0x7f1a600b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a64d40640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1a600cc7f0 0x7f1a600caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a64d40640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a600cb490 con 0x7f1a600ccd00 2024-04-25T10:14:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a64d40640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a600cc360 con 0x7f1a600cc7f0 2024-04-25T10:14:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a64d40640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a600b2f10 con 0x7f1a6015aa50 2024-04-25T10:14:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a5e59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1a600ccd00 0x7f1a600cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46810/0 (socket says 172.21.15.133:46810) 2024-04-25T10:14:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a5e59c640 1 -- 172.21.15.133:0/3003343101 learned_addr learned my addr 172.21.15.133:0/3003343101 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/3003343101 >> v2:172.21.15.133:3300/0 conn(0x7f1a600cc7f0 0x7f1a600caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a5d59a640 1 -- 172.21.15.133:0/3003343101 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251451296 0 0) 0x7f1a600cb490 con 0x7f1a600ccd00 2024-04-25T10:14:54.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a5d59a640 1 -- 172.21.15.133:0/3003343101 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a4c003530 con 0x7f1a600ccd00 2024-04-25T10:14:54.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a5dd9b640 1 --2- 172.21.15.133:0/3003343101 >> v2:172.21.15.162:3300/0 conn(0x7f1a6015aa50 0x7f1a600b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/3003343101 >> v2:172.21.15.162:3300/0 conn(0x7f1a6015aa50 msgr2=0x7f1a600b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/3003343101 >> v2:172.21.15.162:3300/0 conn(0x7f1a6015aa50 0x7f1a600b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/3003343101 >> v1:172.21.15.133:6789/0 conn(0x7f1a600ccd00 legacy=0x7f1a600cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.959+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/3003343101 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600cb750 con 0x7f1a600cc7f0 2024-04-25T10:14:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/3003343101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a600cc7f0 0x7f1a600caf50 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f1a600726f0 tx=0x7f1a5001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be6a65297c8eb44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a5d59a640 1 -- 172.21.15.133:0/3003343101 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500ab650 con 0x7f1a600cc7f0 2024-04-25T10:14:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a5d59a640 1 -- 172.21.15.133:0/3003343101 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a500b0070 con 0x7f1a600cc7f0 2024-04-25T10:14:54.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a5d59a640 1 -- 172.21.15.133:0/3003343101 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500aba10 con 0x7f1a600cc7f0 2024-04-25T10:14:54.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 -- 172.21.15.133:0/3003343101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a600cc7f0 msgr2=0x7f1a600caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 --2- 172.21.15.133:0/3003343101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a600cc7f0 0x7f1a600caf50 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f1a600726f0 tx=0x7f1a5001c450 comp rx=0 tx=0).stop 2024-04-25T10:14:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 -- 172.21.15.133:0/3003343101 shutdown_connections 2024-04-25T10:14:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 --2- 172.21.15.133:0/3003343101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a600cc7f0 0x7f1a600caf50 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 --2- 172.21.15.133:0/3003343101 >> v2:172.21.15.162:3300/0 conn(0x7f1a6015aa50 0x7f1a600b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 -- 172.21.15.133:0/3003343101 >> 172.21.15.133:0/3003343101 conn(0x7f1a600999d0 msgr2=0x7f1a600b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 -- 172.21.15.133:0/3003343101 shutdown_connections 2024-04-25T10:14:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 -- 172.21.15.133:0/3003343101 wait complete. 2024-04-25T10:14:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 Processor -- start 2024-04-25T10:14:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 -- start start 2024-04-25T10:14:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a600cc7f0 0x7f1a601548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.963+0000 7f1a64d40640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a600ccd00 0x7f1a6006b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a64d40640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6015aa50 0x7f1a600b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a64d40640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6006ec50 con 0x7f1a600ccd00 2024-04-25T10:14:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a64d40640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6006ead0 con 0x7f1a6015aa50 2024-04-25T10:14:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a64d40640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6006edd0 con 0x7f1a600cc7f0 2024-04-25T10:14:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a600ccd00 0x7f1a6006b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a600ccd00 0x7f1a6006b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43812/0 (socket says 172.21.15.133:43812) 2024-04-25T10:14:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5dd9b640 1 -- 172.21.15.133:0/4160281420 learned_addr learned my addr 172.21.15.133:0/4160281420 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a600cc7f0 0x7f1a601548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6015aa50 0x7f1a600b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5dd9b640 1 -- 172.21.15.133:0/4160281420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a600cc7f0 msgr2=0x7f1a601548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5dd9b640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a600cc7f0 0x7f1a601548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5dd9b640 1 -- 172.21.15.133:0/4160281420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6015aa50 msgr2=0x7f1a600b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5dd9b640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6015aa50 0x7f1a600b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5dd9b640 1 -- 172.21.15.133:0/4160281420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600c87e0 con 0x7f1a600ccd00 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a600cc7f0 0x7f1a601548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5dd9b640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a600ccd00 0x7f1a6006b450 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800d5f0 tx=0x7f1a4800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a56ffd640 1 -- 172.21.15.133:0/4160281420 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480125f0 con 0x7f1a600ccd00 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a56ffd640 1 -- 172.21.15.133:0/4160281420 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a48019070 con 0x7f1a600ccd00 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a64d40640 1 -- 172.21.15.133:0/4160281420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a600c6ee0 con 0x7f1a600ccd00 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a56ffd640 1 -- 172.21.15.133:0/4160281420 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480129b0 con 0x7f1a600ccd00 2024-04-25T10:14:54.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a64d40640 1 -- 172.21.15.133:0/4160281420 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a600c6120 con 0x7f1a600ccd00 2024-04-25T10:14:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a56ffd640 1 -- 172.21.15.133:0/4160281420 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a48012b50 con 0x7f1a600ccd00 2024-04-25T10:14:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a56ffd640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a40066f60 0x7f1a40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a56ffd640 1 -- 172.21.15.133:0/4160281420 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1a480855a0 con 0x7f1a600ccd00 2024-04-25T10:14:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a40066f60 0x7f1a40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a40066f60 0x7f1a40069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1a58005c10 tx=0x7f1a58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a67ead000 1 Processor -- start 2024-04-25T10:14:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.967+0000 7f1a67ead000 1 -- start start 2024-04-25T10:14:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc17e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e1240 0x55adc17969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc1797310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adc167e5f0 con 0x55adc1796f30 2024-04-25T10:14:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adc15e5da0 con 0x55adc17e1240 2024-04-25T10:14:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55adc156e150 con 0x55adc17e0920 2024-04-25T10:14:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc1797310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc1797310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43818/0 (socket says 172.21.15.133:43818) 2024-04-25T10:14:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/334792295 learned_addr learned my addr 172.21.15.133:0/334792295 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/334792295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc17e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5dd9b640 1 --2- 172.21.15.133:0/334792295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e1240 0x55adc17969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/334792295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 msgr2=0x55adc17e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/334792295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc17e0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/334792295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e1240 msgr2=0x55adc17969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/334792295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e1240 0x55adc17969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/334792295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adc1587ca0 con 0x55adc1796f30 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/334792295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc17e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/334792295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc1797310 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f1a50019060 tx=0x7f1a500a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e5ff1a52085e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a3ffff640 1 -- 172.21.15.133:0/334792295 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500ab6d0 con 0x55adc1796f30 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a3ffff640 1 -- 172.21.15.133:0/334792295 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a500b0040 con 0x55adc1796f30 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a3ffff640 1 -- 172.21.15.133:0/334792295 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500aba70 con 0x55adc1796f30 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- 172.21.15.133:0/334792295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 msgr2=0x55adc1797310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 --2- 172.21.15.133:0/334792295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc1797310 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f1a50019060 tx=0x7f1a500a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- 172.21.15.133:0/334792295 shutdown_connections 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 --2- 172.21.15.133:0/334792295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc1797310 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 --2- 172.21.15.133:0/334792295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e1240 0x55adc17969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 --2- 172.21.15.133:0/334792295 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc17e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- 172.21.15.133:0/334792295 >> 172.21.15.133:0/334792295 conn(0x55adc179d2a0 msgr2=0x55adc17952d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- 172.21.15.133:0/334792295 shutdown_connections 2024-04-25T10:14:54.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- 172.21.15.133:0/334792295 wait complete. 2024-04-25T10:14:54.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- start start 2024-04-25T10:14:54.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 Processor -- start 2024-04-25T10:14:54.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc1796f30 0x55adc17e7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc17e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc179aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adc15a0990 con 0x55adc17e1240 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adc15e1e40 con 0x55adc1796f30 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55adc1571900 con 0x55adc17e0920 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a67ead000 10 client.0 _ll_register_callbacks cb 0x55adc16991c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc179aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc179aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43832/0 (socket says 172.21.15.133:43832) 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/2381603238 learned_addr learned my addr 172.21.15.133:0/2381603238 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc1796f30 0x55adc17e7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/2381603238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 msgr2=0x55adc17e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc17e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/2381603238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc1796f30 msgr2=0x55adc17e7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc1796f30 0x55adc17e7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/2381603238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adc159e630 con 0x55adc17e1240 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.971+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc1796f30 0x55adc17e7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc179aaa0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f1a50019060 tx=0x7f1a5001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f53b4cac3e0002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/2381603238 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500038a0 con 0x55adc17e1240 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/2381603238 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1a50003a40 con 0x55adc17e1240 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 -- 172.21.15.133:0/2381603238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a500b0040 con 0x55adc17e1240 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/2381603238 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a500d0050 con 0x55adc17e1240 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 10 client.10248 fetch_fsmap learned FSMap version 93 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 -- 172.21.15.133:0/2381603238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55adc156e3a0 con 0x55adc17e1240 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/2381603238 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a500b0040 con 0x55adc17e1240 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/2381603238 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 93) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f1a500d3020 con 0x55adc17e1240 2024-04-25T10:14:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 10 client.10248 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 10 client.10248 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 20 client.10248 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x55adc179c160 0x55adc17f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 4 client.10248 mds_command: new command op to 10050 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 -- 172.21.15.133:0/2381603238 --> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adc1581a50 con 0x55adc179c160 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x55adc179c160 0x55adc17f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a3d7fa640 10 client.10248 ms_handle_connect on v2:172.21.15.133:6836/3234096063 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x55adc179c160 0x55adc17f1ce0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e18ce318caf2e813 server_cookie=ec8dfa290811d4a in_seq=0 out_seq=0 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/2381603238 <== mds.0 v2:172.21.15.133:6836/3234096063 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55adc1581a50 con 0x55adc179c160 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a3d7fa640 10 client.10248 handle_command_reply: tid=0 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 client.10248 shutdown 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 -- 172.21.15.133:0/2381603238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 msgr2=0x55adc179aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc179aaa0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f1a50019060 tx=0x7f1a5001ac90 comp rx=0 tx=0).stop 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 -- 172.21.15.133:0/2381603238 shutdown_connections 2024-04-25T10:14:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc179aaa0 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc17e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x55adc179c160 0x55adc17f1ce0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a3d7fa640 0 client.10248 ms_handle_reset on v2:172.21.15.133:6836/3234096063 2024-04-25T10:14:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 --2- 172.21.15.133:0/2381603238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc1796f30 0x55adc17e7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 -- 172.21.15.133:0/2381603238 >> 172.21.15.133:0/2381603238 conn(0x55adc179d2a0 msgr2=0x55adc17959e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 -- 172.21.15.133:0/2381603238 shutdown_connections 2024-04-25T10:14:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 1 -- 172.21.15.133:0/2381603238 wait complete. 2024-04-25T10:14:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.975+0000 7f1a67ead000 20 client.10248 trim_cache size 0 max 16384 2024-04-25T10:14:54.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a67ead000 1 Processor -- start 2024-04-25T10:14:54.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a67ead000 1 -- start start 2024-04-25T10:14:54.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc1796f30 0x55adc17ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e0920 0x55adc17ea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc17eacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a67ead000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a60202d50 con 0x55adc17e1240 2024-04-25T10:14:54.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a67ead000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600c9d80 con 0x55adc17e0920 2024-04-25T10:14:54.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a67ead000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a60201940 con 0x55adc1796f30 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a5ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc17eacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a5ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc17eacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43834/0 (socket says 172.21.15.133:43834) 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/1407986003 learned_addr learned my addr 172.21.15.133:0/1407986003 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a5dd9b640 1 --2- 172.21.15.133:0/1407986003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e0920 0x55adc17ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/1407986003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc1796f30 msgr2=0x55adc17ed2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/1407986003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc1796f30 0x55adc17ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/1407986003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e0920 msgr2=0x55adc17ea790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/1407986003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e0920 0x55adc17ea790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a5ed9d640 1 -- 172.21.15.133:0/1407986003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adc17e1070 con 0x55adc17e1240 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/1407986003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc17eacd0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f1a50000c10 tx=0x7f1a5000b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1a5e4b69e23d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a3f7fe640 1 -- 172.21.15.133:0/1407986003 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500b9040 con 0x55adc17e1240 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a3f7fe640 1 -- 172.21.15.133:0/1407986003 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a500b0040 con 0x55adc17e1240 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a3f7fe640 1 -- 172.21.15.133:0/1407986003 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50003840 con 0x55adc17e1240 2024-04-25T10:14:54.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a67ead000 1 -- 172.21.15.133:0/1407986003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 msgr2=0x55adc17eacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.991+0000 7f1a67ead000 1 --2- 172.21.15.133:0/1407986003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc17eacd0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f1a50000c10 tx=0x7f1a5000b050 comp rx=0 tx=0).stop 2024-04-25T10:14:54.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1407986003 shutdown_connections 2024-04-25T10:14:54.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- 172.21.15.133:0/1407986003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc17e1240 0x55adc17eacd0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- 172.21.15.133:0/1407986003 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e0920 0x55adc17ea790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- 172.21.15.133:0/1407986003 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc1796f30 0x55adc17ed2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1407986003 >> 172.21.15.133:0/1407986003 conn(0x55adc179d2a0 msgr2=0x55adc179a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1407986003 shutdown_connections 2024-04-25T10:14:54.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1407986003 wait complete. 2024-04-25T10:14:55.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- start start 2024-04-25T10:14:55.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 Processor -- start 2024-04-25T10:14:55.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc17ed3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc182da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e1240 0x55adc182df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adc15ea3d0 con 0x55adc1796f30 2024-04-25T10:14:55.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adc16a1d50 con 0x55adc17e1240 2024-04-25T10:14:55.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55adc1580980 con 0x55adc17e0920 2024-04-25T10:14:55.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 10 client.0 _ll_register_callbacks cb 0x7f1a4c003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:55.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc17ed3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc17ed3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43840/0 (socket says 172.21.15.133:43840) 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5e59c640 1 -- 172.21.15.133:0/1856365421 learned_addr learned my addr 172.21.15.133:0/1856365421 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5dd9b640 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc182da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5ed9d640 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e1240 0x55adc182df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5e59c640 1 -- 172.21.15.133:0/1856365421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 msgr2=0x55adc182da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc182da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5e59c640 1 -- 172.21.15.133:0/1856365421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e1240 msgr2=0x55adc182df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e1240 0x55adc182df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5e59c640 1 -- 172.21.15.133:0/1856365421 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adc1667600 con 0x55adc1796f30 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5e59c640 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc17ed3b0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f1a58007140 tx=0x7f1a58001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8fe13657d9d16e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/1856365421 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58012610 con 0x55adc1796f30 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/1856365421 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1a58027a50 con 0x55adc1796f30 2024-04-25T10:14:55.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1856365421 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a500b0040 con 0x55adc1796f30 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/1856365421 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a58032050 con 0x55adc1796f30 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 10 client.10254 fetch_fsmap learned FSMap version 93 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1856365421 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55adc157a9b0 con 0x55adc1796f30 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/1856365421 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a500b0040 con 0x55adc1796f30 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/1856365421 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 93) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f1a58035020 con 0x55adc1796f30 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 10 client.10254 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 10 client.10254 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 20 client.10254 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x55adc182e4c0 0x55adc1849910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 4 client.10254 mds_command: new command op to 10050 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1856365421 --> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55adc15f32e0 con 0x55adc182e4c0 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5dd9b640 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x55adc182e4c0 0x55adc1849910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a5dd9b640 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x55adc182e4c0 0x55adc1849910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce2f364bc29d923 server_cookie=4737f409b80eb6e5 in_seq=0 out_seq=0 2024-04-25T10:14:55.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a3d7fa640 10 client.10254 ms_handle_connect on v2:172.21.15.133:6836/3234096063 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a3d7fa640 1 -- 172.21.15.133:0/1856365421 <== mds.0 v2:172.21.15.133:6836/3234096063 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55adc15f32e0 con 0x55adc182e4c0 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a3d7fa640 10 client.10254 handle_command_reply: tid=0 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 client.10254 shutdown 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1856365421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 msgr2=0x55adc17ed3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc17ed3b0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f1a58007140 tx=0x7f1a58001260 comp rx=0 tx=0).stop 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1856365421 shutdown_connections 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.133:6836/3234096063,v1:172.21.15.133:6837/3234096063] conn(0x55adc182e4c0 0x55adc1849910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55adc17e1240 0x55adc182df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55adc17e0920 0x55adc182da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 --2- 172.21.15.133:0/1856365421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55adc1796f30 0x55adc17ed3b0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a3d7fa640 0 client.10254 ms_handle_reset on v2:172.21.15.133:6836/3234096063 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1856365421 >> 172.21.15.133:0/1856365421 conn(0x55adc179d2a0 msgr2=0x55adc17de300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1856365421 shutdown_connections 2024-04-25T10:14:55.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 1 -- 172.21.15.133:0/1856365421 wait complete. 2024-04-25T10:14:55.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.995+0000 7f1a67ead000 20 client.10254 trim_cache size 0 max 16384 2024-04-25T10:14:55.006 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:14:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:14:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:14:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 10050, 2024-04-25T10:14:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:14:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:14:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:14:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 26.32959683, 2024-04-25T10:14:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 93, 2024-04-25T10:14:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 145, 2024-04-25T10:14:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 145, 2024-04-25T10:14:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 31.410642355 2024-04-25T10:14:55.007 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 -- 172.21.15.133:0/4160281420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a40066f60 msgr2=0x7f1a40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a40066f60 0x7f1a40069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1a58005c10 tx=0x7f1a58009040 comp rx=0 tx=0).stop 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 -- 172.21.15.133:0/4160281420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a600ccd00 msgr2=0x7f1a6006b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a600ccd00 0x7f1a6006b450 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800d5f0 tx=0x7f1a4800dab0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 -- 172.21.15.133:0/4160281420 shutdown_connections 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6015aa50 0x7f1a600b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a40066f60 0x7f1a40069420 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a600ccd00 0x7f1a6006b450 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 --2- 172.21.15.133:0/4160281420 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a600cc7f0 0x7f1a601548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 -- 172.21.15.133:0/4160281420 >> 172.21.15.133:0/4160281420 conn(0x7f1a600999d0 msgr2=0x7f1a600d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 -- 172.21.15.133:0/4160281420 shutdown_connections 2024-04-25T10:14:55.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:54.999+0000 7f1a64d40640 1 -- 172.21.15.133:0/4160281420 wait complete. 2024-04-25T10:14:55.017 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub start abs_test_path recursive 2024-04-25T10:14:55.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a3780640 1 Processor -- start 2024-04-25T10:14:55.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a3780640 1 -- start start 2024-04-25T10:14:55.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a3780640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c071cb0 0x7f779c15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a3780640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f779c15c250 0x7f779c0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a3780640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779c15c9b0 con 0x7f779c0b4940 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a3780640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779c072a90 con 0x7f779c15c250 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a3780640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779c0a85d0 con 0x7f779c071cb0 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a151c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f779c0b4940 0x7f779c0b4d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46814/0 (socket says 172.21.15.133:46814) 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a151c640 1 -- 172.21.15.133:0/1803149531 learned_addr learned my addr 172.21.15.133:0/1803149531 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a1d1d640 1 --2- 172.21.15.133:0/1803149531 >> v2:172.21.15.162:3300/0 conn(0x7f779c15c250 0x7f779c0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c071cb0 0x7f779c15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336553127 0 0) 0x7f779c15c9b0 con 0x7f779c0b4940 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7788003530 con 0x7f779c0b4940 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 568736309 0 0) 0x7f7788003530 con 0x7f779c0b4940 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c071cb0 msgr2=0x7f779c15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c071cb0 0x7f779c15bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 >> v2:172.21.15.162:3300/0 conn(0x7f779c15c250 msgr2=0x7f779c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 --2- 172.21.15.133:0/1803149531 >> v2:172.21.15.162:3300/0 conn(0x7f779c15c250 0x7f779c0cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c15cc90 con 0x7f779c0b4940 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7798001680 con 0x7f779c0b4940 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 >> v1:172.21.15.133:6789/0 conn(0x7f779c0b4940 legacy=0x7f779c0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77880039c0 0x7f7788003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c071cb0 0x7f7788004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7788004d50 0x7f77880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778800b9d0 con 0x7f7788004d50 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778800bb50 con 0x7f77880039c0 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778800bcd0 con 0x7f779c071cb0 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a1d1d640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7788004d50 0x7f77880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a151c640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77880039c0 0x7f7788003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c071cb0 0x7f7788004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a1d1d640 1 -- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c071cb0 msgr2=0x7f7788004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a1d1d640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c071cb0 0x7f7788004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a1d1d640 1 -- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77880039c0 msgr2=0x7f7788003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a1d1d640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77880039c0 0x7f7788003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a1d1d640 1 -- 172.21.15.133:0/1803149531 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7788003530 con 0x7f7788004d50 2024-04-25T10:14:55.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.139+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c071cb0 0x7f7788004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f77a1d1d640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7788004d50 0x7f77880051d0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f7794009fc0 tx=0x7f779401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3ad18b763ebb53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77940b2070 con 0x7f7788004d50 2024-04-25T10:14:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77940a5cd0 con 0x7f7788004d50 2024-04-25T10:14:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f7793fff640 1 -- 172.21.15.133:0/1803149531 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794009c10 con 0x7f7788004d50 2024-04-25T10:14:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f77a3780640 1 -- 172.21.15.133:0/1803149531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7788004d50 msgr2=0x7f77880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f77a3780640 1 --2- 172.21.15.133:0/1803149531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7788004d50 0x7f77880051d0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f7794009fc0 tx=0x7f779401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f77a1d1d640 1 -- 172.21.15.133:0/1803149531 reap_dead start 2024-04-25T10:14:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f77a3780640 1 -- 172.21.15.133:0/1803149531 shutdown_connections 2024-04-25T10:14:55.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f77a3780640 1 -- 172.21.15.133:0/1803149531 >> 172.21.15.133:0/1803149531 conn(0x7f779c06d370 msgr2=0x7f779c06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f77a3780640 1 -- 172.21.15.133:0/1803149531 shutdown_connections 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f77a3780640 1 -- 172.21.15.133:0/1803149531 wait complete. 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f77a3780640 1 Processor -- start 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.143+0000 7f77a3780640 1 -- start start 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a3780640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c15ae00 0x7f779c0aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a3780640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f779c0a90c0 0x7f779c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a3780640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c06dcc0 0x7f779c0ac560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a3780640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c071790 con 0x7f779c15ae00 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a3780640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c0714b0 con 0x7f779c0a90c0 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a3780640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779c15c6a0 con 0x7f779c06dcc0 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a151c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c15ae00 0x7f779c0aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a151c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c15ae00 0x7f779c0aa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43872/0 (socket says 172.21.15.133:43872) 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a151c640 1 -- 172.21.15.133:0/3921020419 learned_addr learned my addr 172.21.15.133:0/3921020419 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a1d1d640 1 --2- 172.21.15.133:0/3921020419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c06dcc0 0x7f779c0ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/3921020419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f779c0a90c0 0x7f779c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a151c640 1 -- 172.21.15.133:0/3921020419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c06dcc0 msgr2=0x7f779c0ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a151c640 1 --2- 172.21.15.133:0/3921020419 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f779c06dcc0 0x7f779c0ac560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a151c640 1 -- 172.21.15.133:0/3921020419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f779c0a90c0 msgr2=0x7f779c06d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a151c640 1 --2- 172.21.15.133:0/3921020419 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f779c0a90c0 0x7f779c06d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a151c640 1 -- 172.21.15.133:0/3921020419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c0ab180 con 0x7f779c15ae00 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a151c640 1 --2- 172.21.15.133:0/3921020419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c15ae00 0x7f779c0aa8f0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f7798000fc0 tx=0x7f779800ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f7791ffb640 1 -- 172.21.15.133:0/3921020419 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779800c8d0 con 0x7f779c15ae00 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f7791ffb640 1 -- 172.21.15.133:0/3921020419 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798018070 con 0x7f779c15ae00 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a3780640 1 -- 172.21.15.133:0/3921020419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c0ab460 con 0x7f779c15ae00 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f7791ffb640 1 -- 172.21.15.133:0/3921020419 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779800ccb0 con 0x7f779c15ae00 2024-04-25T10:14:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a3780640 1 -- 172.21.15.133:0/3921020419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c0aba20 con 0x7f779c15ae00 2024-04-25T10:14:55.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f7791ffb640 1 -- 172.21.15.133:0/3921020419 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f779801d460 con 0x7f779c15ae00 2024-04-25T10:14:55.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f7791ffb640 1 --2- 172.21.15.133:0/3921020419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7780066f60 0x7f7780069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f7791ffb640 1 -- 172.21.15.133:0/3921020419 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7798085140 con 0x7f779c15ae00 2024-04-25T10:14:55.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.147+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/3921020419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7780066f60 0x7f7780069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/3921020419 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7780066f60 0x7f7780069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ac1c0 tx=0x7f778c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 Processor -- start 2024-04-25T10:14:55.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 -- start start 2024-04-25T10:14:55.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7d460 0x55c89ff7d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7dd80 0x55c89ff30cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff31210 0x55c89ff315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89fd81c30 con 0x55c89ff7dd80 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89fd0a8c0 con 0x55c89ff7d460 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c89fd7dcd0 con 0x55c89ff31210 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a0d1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7dd80 0x55c89ff30cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a0d1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7dd80 0x55c89ff30cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43880/0 (socket says 172.21.15.133:43880) 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a0d1b640 1 -- 172.21.15.133:0/3015349237 learned_addr learned my addr 172.21.15.133:0/3015349237 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a151c640 1 --2- 172.21.15.133:0/3015349237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7d460 0x55c89ff7d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a1d1d640 1 --2- 172.21.15.133:0/3015349237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff31210 0x55c89ff315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a0d1b640 1 -- 172.21.15.133:0/3015349237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff31210 msgr2=0x55c89ff315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/3015349237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff31210 0x55c89ff315f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a0d1b640 1 -- 172.21.15.133:0/3015349237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7d460 msgr2=0x55c89ff7d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/3015349237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7d460 0x55c89ff7d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a0d1b640 1 -- 172.21.15.133:0/3015349237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c89fe0a130 con 0x55c89ff7dd80 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/3015349237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7dd80 0x55c89ff30cd0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f778c012d70 tx=0x7f778c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9896abcf3758be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f7786ffd640 1 -- 172.21.15.133:0/3015349237 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c00f040 con 0x55c89ff7dd80 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f7786ffd640 1 -- 172.21.15.133:0/3015349237 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c027070 con 0x55c89ff7dd80 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f7786ffd640 1 -- 172.21.15.133:0/3015349237 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c022660 con 0x55c89ff7dd80 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 -- 172.21.15.133:0/3015349237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7dd80 msgr2=0x55c89ff30cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 --2- 172.21.15.133:0/3015349237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7dd80 0x55c89ff30cd0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f778c012d70 tx=0x7f778c015000 comp rx=0 tx=0).stop 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 -- 172.21.15.133:0/3015349237 shutdown_connections 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 --2- 172.21.15.133:0/3015349237 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff31210 0x55c89ff315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 --2- 172.21.15.133:0/3015349237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7dd80 0x55c89ff30cd0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 --2- 172.21.15.133:0/3015349237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7d460 0x55c89ff7d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 -- 172.21.15.133:0/3015349237 >> 172.21.15.133:0/3015349237 conn(0x55c89fe1bc80 msgr2=0x55c89ff60420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 -- 172.21.15.133:0/3015349237 shutdown_connections 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 -- 172.21.15.133:0/3015349237 wait complete. 2024-04-25T10:14:55.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 -- start start 2024-04-25T10:14:55.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.151+0000 7f77a68ed000 1 Processor -- start 2024-04-25T10:14:55.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff31210 0x55c89ff7ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7d460 0x55c89ff3e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 0x55c89ff3e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89fd2aad0 con 0x55c89ff7d460 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89fd3cfa0 con 0x55c89ff7dd80 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c89fd80c00 con 0x55c89ff31210 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 10 client.0 _ll_register_callbacks cb 0x55c89ff1eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a0d1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7d460 0x55c89ff3e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a0d1b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7d460 0x55c89ff3e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43896/0 (socket says 172.21.15.133:43896) 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a0d1b640 1 -- 172.21.15.133:0/1426460546 learned_addr learned my addr 172.21.15.133:0/1426460546 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a151c640 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff31210 0x55c89ff7ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a1d1d640 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 0x55c89ff3e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a0d1b640 1 -- 172.21.15.133:0/1426460546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff31210 msgr2=0x55c89ff7ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff31210 0x55c89ff7ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a0d1b640 1 -- 172.21.15.133:0/1426460546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 msgr2=0x55c89ff3e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 0x55c89ff3e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a0d1b640 1 -- 172.21.15.133:0/1426460546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c89fdc7f90 con 0x55c89ff7d460 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7d460 0x55c89ff3e000 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f778c009960 tx=0x7f778c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ad50b11264a399 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f777bfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f777bfff640 1 -- 172.21.15.133:0/1426460546 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c022910 con 0x55c89ff7d460 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 -- 172.21.15.133:0/1426460546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f778c027070 con 0x55c89ff7d460 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f777bfff640 1 -- 172.21.15.133:0/1426460546 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f778c022ab0 con 0x55c89ff7d460 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f777bfff640 1 -- 172.21.15.133:0/1426460546 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f778c03e050 con 0x55c89ff7d460 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 10 client.10266 fetch_fsmap learned FSMap version 93 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 -- 172.21.15.133:0/1426460546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55c89fd46630 con 0x55c89ff7d460 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f777bfff640 1 -- 172.21.15.133:0/1426460546 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c027070 con 0x55c89ff7d460 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f777bfff640 1 -- 172.21.15.133:0/1426460546 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 93) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f778c041020 con 0x55c89ff7d460 2024-04-25T10:14:55.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 10 client.10266 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 10 client.10266 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 20 client.10266 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x55c89ff3f280 0x55c89ff8f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 4 client.10266 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 -- 172.21.15.133:0/1426460546 --> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c89fd1df80 con 0x55c89ff3f280 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a151c640 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x55c89ff3f280 0x55c89ff8f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a151c640 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x55c89ff3f280 0x55c89ff8f720 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.10041 client_cookie=d88ad23821b71c71 server_cookie=6df3aa31f888817b in_seq=0 out_seq=0 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f777bfff640 10 client.10266 ms_handle_connect on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f777bfff640 1 -- 172.21.15.133:0/1426460546 <== mds.10041 v2:172.21.15.133:6834/2626366082 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c89fd1df80 con 0x55c89ff3f280 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f777bfff640 10 client.10266 handle_command_reply: tid=0 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 client.10266 shutdown 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 -- 172.21.15.133:0/1426460546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7d460 msgr2=0x55c89ff3e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7d460 0x55c89ff3e000 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f778c009960 tx=0x7f778c004740 comp rx=0 tx=0).stop 2024-04-25T10:14:55.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 -- 172.21.15.133:0/1426460546 shutdown_connections 2024-04-25T10:14:55.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 0x55c89ff3e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x55c89ff3f280 0x55c89ff8f720 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff7d460 0x55c89ff3e000 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 --2- 172.21.15.133:0/1426460546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff31210 0x55c89ff7ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 -- 172.21.15.133:0/1426460546 >> 172.21.15.133:0/1426460546 conn(0x55c89fe1bc80 msgr2=0x55c89ff303d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f777bfff640 0 client.10266 ms_handle_reset on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 -- 172.21.15.133:0/1426460546 shutdown_connections 2024-04-25T10:14:55.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 1 -- 172.21.15.133:0/1426460546 wait complete. 2024-04-25T10:14:55.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.155+0000 7f77a68ed000 20 client.10266 trim_cache size 0 max 16384 2024-04-25T10:14:55.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a68ed000 1 Processor -- start 2024-04-25T10:14:55.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a68ed000 1 -- start start 2024-04-25T10:14:55.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ff30590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff7d460 0x55c89ff2f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 0x55c89ff2f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a68ed000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c1fe210 con 0x55c89ff31210 2024-04-25T10:14:55.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a68ed000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c1fe870 con 0x55c89ff7dd80 2024-04-25T10:14:55.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a68ed000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779c1fcdd0 con 0x55c89ff7d460 2024-04-25T10:14:55.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a151c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ff30590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a151c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ff30590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43898/0 (socket says 172.21.15.133:43898) 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a151c640 1 -- 172.21.15.133:0/2519580531 learned_addr learned my addr 172.21.15.133:0/2519580531 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a151c640 1 -- 172.21.15.133:0/2519580531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff7d460 msgr2=0x55c89ff2f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a151c640 1 --2- 172.21.15.133:0/2519580531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff7d460 0x55c89ff2f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a151c640 1 -- 172.21.15.133:0/2519580531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 msgr2=0x55c89ff2f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a151c640 1 --2- 172.21.15.133:0/2519580531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 0x55c89ff2f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a151c640 1 -- 172.21.15.133:0/2519580531 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c89fd0ab10 con 0x55c89ff31210 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77a151c640 1 --2- 172.21.15.133:0/2519580531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ff30590 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f7798000c10 tx=0x7f7798083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825010aca3af68a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77867fc640 1 -- 172.21.15.133:0/2519580531 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779806d070 con 0x55c89ff31210 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77867fc640 1 -- 172.21.15.133:0/2519580531 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77980599e0 con 0x55c89ff31210 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.171+0000 7f77867fc640 1 -- 172.21.15.133:0/2519580531 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779805dfa0 con 0x55c89ff31210 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- 172.21.15.133:0/2519580531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 msgr2=0x55c89ff30590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 --2- 172.21.15.133:0/2519580531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ff30590 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f7798000c10 tx=0x7f7798083040 comp rx=0 tx=0).stop 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- 172.21.15.133:0/2519580531 shutdown_connections 2024-04-25T10:14:55.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 --2- 172.21.15.133:0/2519580531 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 0x55c89ff2f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 --2- 172.21.15.133:0/2519580531 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff7d460 0x55c89ff2f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 --2- 172.21.15.133:0/2519580531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ff30590 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- 172.21.15.133:0/2519580531 >> 172.21.15.133:0/2519580531 conn(0x55c89fe1bc80 msgr2=0x55c89ff8aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- 172.21.15.133:0/2519580531 shutdown_connections 2024-04-25T10:14:55.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- 172.21.15.133:0/2519580531 wait complete. 2024-04-25T10:14:55.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- start start 2024-04-25T10:14:55.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 Processor -- start 2024-04-25T10:14:55.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ffe18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff7d460 0x55c89ff9d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 0x55c89ff9daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89ff30d80 con 0x55c89ff31210 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c1fd3d0 con 0x55c89ff7dd80 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c89fd2aad0 con 0x55c89ff7d460 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 10 client.0 _ll_register_callbacks cb 0x7f778c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a151c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ffe18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a151c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ffe18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43906/0 (socket says 172.21.15.133:43906) 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a151c640 1 -- 172.21.15.133:0/2314808658 learned_addr learned my addr 172.21.15.133:0/2314808658 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a1d1d640 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 0x55c89ff9daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff7d460 0x55c89ff9d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a151c640 1 -- 172.21.15.133:0/2314808658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff7d460 msgr2=0x55c89ff9d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a151c640 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff7d460 0x55c89ff9d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a151c640 1 -- 172.21.15.133:0/2314808658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 msgr2=0x55c89ff9daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a151c640 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 0x55c89ff9daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a151c640 1 -- 172.21.15.133:0/2314808658 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c89fddacf0 con 0x55c89ff31210 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a151c640 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ffe18f0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f7798083450 tx=0x7f7798007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81edb6da337d3ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f777bfff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f777bfff640 1 -- 172.21.15.133:0/2314808658 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798080070 con 0x55c89ff31210 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f777bfff640 1 -- 172.21.15.133:0/2314808658 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7798051f00 con 0x55c89ff31210 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- 172.21.15.133:0/2314808658 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77980599e0 con 0x55c89ff31210 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f777bfff640 1 -- 172.21.15.133:0/2314808658 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f779806d040 con 0x55c89ff31210 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 10 client.10272 fetch_fsmap learned FSMap version 93 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- 172.21.15.133:0/2314808658 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55c89fb67a30 con 0x55c89ff31210 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f777bfff640 1 -- 172.21.15.133:0/2314808658 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77980599e0 con 0x55c89ff31210 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f777bfff640 1 -- 172.21.15.133:0/2314808658 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 93) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f779806d1e0 con 0x55c89ff31210 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 10 client.10272 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 10 client.10272 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 20 client.10272 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x55c89ff9e030 0x55c89ff9e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 4 client.10272 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- 172.21.15.133:0/2314808658 --> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] -- command(tid 0: {"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}) v1 -- 0x55c89fd8f170 con 0x55c89ff9e030 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x55c89ff9e030 0x55c89ff9e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a0d1b640 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x55c89ff9e030 0x55c89ff9e410 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.10041 client_cookie=c4f64ed6798de807 server_cookie=588292fce6f75585 in_seq=0 out_seq=0 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f777bfff640 10 client.10272 ms_handle_connect on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f777bfff640 1 -- 172.21.15.133:0/2314808658 <== mds.10041 v2:172.21.15.133:6834/2626366082 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x55c89fd8f170 con 0x55c89ff9e030 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f777bfff640 10 client.10272 handle_command_reply: tid=0 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 client.10272 shutdown 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 -- 172.21.15.133:0/2314808658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 msgr2=0x55c89ffe18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.175+0000 7f77a68ed000 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ffe18f0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f7798083450 tx=0x7f7798007040 comp rx=0 tx=0).stop 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.179+0000 7f77a68ed000 1 -- 172.21.15.133:0/2314808658 shutdown_connections 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.179+0000 7f77a68ed000 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c89ff7dd80 0x55c89ff9daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.179+0000 7f77a68ed000 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55c89ff7d460 0x55c89ff9d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.179+0000 7f77a68ed000 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x55c89ff9e030 0x55c89ff9e410 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.179+0000 7f77a68ed000 1 --2- 172.21.15.133:0/2314808658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c89ff31210 0x55c89ffe18f0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.179+0000 7f777bfff640 0 client.10272 ms_handle_reset on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.179+0000 7f77a68ed000 1 -- 172.21.15.133:0/2314808658 >> 172.21.15.133:0/2314808658 conn(0x55c89fe1bc80 msgr2=0x55c89ff7b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.179+0000 7f77a68ed000 1 -- 172.21.15.133:0/2314808658 shutdown_connections 2024-04-25T10:14:55.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.179+0000 7f77a68ed000 1 -- 172.21.15.133:0/2314808658 wait complete. 2024-04-25T10:14:55.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.179+0000 7f77a68ed000 20 client.10272 trim_cache size 0 max 16384 2024-04-25T10:14:55.186 INFO:teuthology.orchestra.run.smithi133.stderr:Error EINVAL: MDS is not active 2024-04-25T10:14:55.186 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:14:55.187 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub pause 2024-04-25T10:14:55.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1147076640 1 Processor -- start 2024-04-25T10:14:55.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1147076640 1 -- start start 2024-04-25T10:14:55.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1147076640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f114006f020 0x7f114006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1147076640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11400701a0 con 0x7f114006b3a0 2024-04-25T10:14:55.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1147076640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114006e540 con 0x7f114015db70 2024-04-25T10:14:55.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1147076640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114006eb90 con 0x7f114006f020 2024-04-25T10:14:55.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1145613640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f114006f020 0x7f114006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1145613640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f114006f020 0x7f114006fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43914/0 (socket says 172.21.15.133:43914) 2024-04-25T10:14:55.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1145613640 1 -- 172.21.15.133:0/2881882911 learned_addr learned my addr 172.21.15.133:0/2881882911 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1145613640 1 -- 172.21.15.133:0/2881882911 >> v1:172.21.15.162:6789/0 conn(0x7f114015db70 legacy=0x7f114006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1145613640 1 -- 172.21.15.133:0/2881882911 >> v1:172.21.15.133:6789/0 conn(0x7f114006b3a0 legacy=0x7f11400ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.299+0000 7f1145613640 1 -- 172.21.15.133:0/2881882911 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114006e820 con 0x7f114006f020 2024-04-25T10:14:55.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1145613640 1 --2- 172.21.15.133:0/2881882911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f020 0x7f114006fbc0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f1138009e50 tx=0x7f113801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1fc5047903a1a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f113f7fe640 1 -- 172.21.15.133:0/2881882911 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380a57d0 con 0x7f114006f020 2024-04-25T10:14:55.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f113f7fe640 1 -- 172.21.15.133:0/2881882911 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11380a5970 con 0x7f114006f020 2024-04-25T10:14:55.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f113f7fe640 1 -- 172.21.15.133:0/2881882911 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380a5d40 con 0x7f114006f020 2024-04-25T10:14:55.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 -- 172.21.15.133:0/2881882911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f020 msgr2=0x7f114006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 --2- 172.21.15.133:0/2881882911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f020 0x7f114006fbc0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f1138009e50 tx=0x7f113801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 -- 172.21.15.133:0/2881882911 shutdown_connections 2024-04-25T10:14:55.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 --2- 172.21.15.133:0/2881882911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f020 0x7f114006fbc0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 -- 172.21.15.133:0/2881882911 >> 172.21.15.133:0/2881882911 conn(0x7f11400999d0 msgr2=0x7f11400b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 -- 172.21.15.133:0/2881882911 shutdown_connections 2024-04-25T10:14:55.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 -- 172.21.15.133:0/2881882911 wait complete. 2024-04-25T10:14:55.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 Processor -- start 2024-04-25T10:14:55.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 -- start start 2024-04-25T10:14:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114006b3a0 0x7f11400c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f020 0x7f1140152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f114015db70 0x7f11400c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114016bc50 con 0x7f114006f020 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114016bad0 con 0x7f114006b3a0 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f1147076640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114016bdd0 con 0x7f114015db70 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.303+0000 7f113ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f020 0x7f1140152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f020 0x7f1140152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43920/0 (socket says 172.21.15.133:43920) 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113ffff640 1 -- 172.21.15.133:0/392681823 learned_addr learned my addr 172.21.15.133:0/392681823 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f1145613640 1 --2- 172.21.15.133:0/392681823 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f114015db70 0x7f11400c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f1144e12640 1 --2- 172.21.15.133:0/392681823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114006b3a0 0x7f11400c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113ffff640 1 -- 172.21.15.133:0/392681823 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f114015db70 msgr2=0x7f11400c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113ffff640 1 --2- 172.21.15.133:0/392681823 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f114015db70 0x7f11400c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113ffff640 1 -- 172.21.15.133:0/392681823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114006b3a0 msgr2=0x7f11400c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113ffff640 1 --2- 172.21.15.133:0/392681823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114006b3a0 0x7f11400c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113ffff640 1 -- 172.21.15.133:0/392681823 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11400c6040 con 0x7f114006f020 2024-04-25T10:14:55.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f1144e12640 1 --2- 172.21.15.133:0/392681823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f114006b3a0 0x7f11400c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113ffff640 1 --2- 172.21.15.133:0/392681823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f020 0x7f1140152950 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f113400c460 tx=0x7f113400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113d7fa640 1 -- 172.21.15.133:0/392681823 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134016020 con 0x7f114006f020 2024-04-25T10:14:55.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113d7fa640 1 -- 172.21.15.133:0/392681823 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11340052d0 con 0x7f114006f020 2024-04-25T10:14:55.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f1147076640 1 -- 172.21.15.133:0/392681823 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11400c6320 con 0x7f114006f020 2024-04-25T10:14:55.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113d7fa640 1 -- 172.21.15.133:0/392681823 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134007260 con 0x7f114006f020 2024-04-25T10:14:55.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f1147076640 1 -- 172.21.15.133:0/392681823 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11400c68e0 con 0x7f114006f020 2024-04-25T10:14:55.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113d7fa640 1 -- 172.21.15.133:0/392681823 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1134007400 con 0x7f114006f020 2024-04-25T10:14:55.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113d7fa640 1 --2- 172.21.15.133:0/392681823 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f112c066f60 0x7f112c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f113d7fa640 1 -- 172.21.15.133:0/392681823 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1134084b40 con 0x7f114006f020 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f1144e12640 1 --2- 172.21.15.133:0/392681823 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f112c066f60 0x7f112c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f1144e12640 1 --2- 172.21.15.133:0/392681823 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f112c066f60 0x7f112c069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f113000a850 tx=0x7f1130008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f114a1e3000 1 Processor -- start 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.307+0000 7f114a1e3000 1 -- start start 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 0x555e2f6adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 0x555e2f6634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f663dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f54aad0 con 0x555e2f6639e0 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f4b1f80 con 0x555e2f6ae0d0 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e2f439ee0 con 0x555e2f6ad7b0 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1145613640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f663dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1145613640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f663dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43928/0 (socket says 172.21.15.133:43928) 2024-04-25T10:14:55.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1145613640 1 -- 172.21.15.133:0/93456207 learned_addr learned my addr 172.21.15.133:0/93456207 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1144e12640 1 --2- 172.21.15.133:0/93456207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 0x555e2f6adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f113ffff640 1 --2- 172.21.15.133:0/93456207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 0x555e2f6634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1145613640 1 -- 172.21.15.133:0/93456207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 msgr2=0x555e2f6adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1145613640 1 --2- 172.21.15.133:0/93456207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 0x555e2f6adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1145613640 1 -- 172.21.15.133:0/93456207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 msgr2=0x555e2f6634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1145613640 1 --2- 172.21.15.133:0/93456207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 0x555e2f6634a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1145613640 1 -- 172.21.15.133:0/93456207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e2f46a930 con 0x555e2f6639e0 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f113ffff640 1 --2- 172.21.15.133:0/93456207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 0x555e2f6634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1145613640 1 --2- 172.21.15.133:0/93456207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f663dc0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f113801a620 tx=0x7f1138004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134eda326e733f8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f11267fc640 1 -- 172.21.15.133:0/93456207 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380a58b0 con 0x555e2f6639e0 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f11267fc640 1 -- 172.21.15.133:0/93456207 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11380a5a50 con 0x555e2f6639e0 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f11267fc640 1 -- 172.21.15.133:0/93456207 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380a5e40 con 0x555e2f6639e0 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- 172.21.15.133:0/93456207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 msgr2=0x555e2f663dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 --2- 172.21.15.133:0/93456207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f663dc0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f113801a620 tx=0x7f1138004ac0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- 172.21.15.133:0/93456207 shutdown_connections 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 --2- 172.21.15.133:0/93456207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f663dc0 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 --2- 172.21.15.133:0/93456207 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 0x555e2f6634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 --2- 172.21.15.133:0/93456207 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 0x555e2f6adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- 172.21.15.133:0/93456207 >> 172.21.15.133:0/93456207 conn(0x555e2f669f60 msgr2=0x555e2f661d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- 172.21.15.133:0/93456207 shutdown_connections 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- 172.21.15.133:0/93456207 wait complete. 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- start start 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 Processor -- start 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6639e0 0x555e2f6ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ad7b0 0x555e2f668a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ae0d0 0x555e2f668fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f46c390 con 0x555e2f6ad7b0 2024-04-25T10:14:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f4ae020 con 0x555e2f6639e0 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e2f4aeff0 con 0x555e2f6ae0d0 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f114a1e3000 10 client.0 _ll_register_callbacks cb 0x555e2f5656a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f113ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ad7b0 0x555e2f668a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f113ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ad7b0 0x555e2f668a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43942/0 (socket says 172.21.15.133:43942) 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f113ffff640 1 -- 172.21.15.133:0/1386749171 learned_addr learned my addr 172.21.15.133:0/1386749171 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1144e12640 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6639e0 0x555e2f6ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f1145613640 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ae0d0 0x555e2f668fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f113ffff640 1 -- 172.21.15.133:0/1386749171 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ae0d0 msgr2=0x555e2f668fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f113ffff640 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ae0d0 0x555e2f668fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f113ffff640 1 -- 172.21.15.133:0/1386749171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6639e0 msgr2=0x555e2f6ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f113ffff640 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6639e0 0x555e2f6ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.311+0000 7f113ffff640 1 -- 172.21.15.133:0/1386749171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e2f43a130 con 0x555e2f6ad7b0 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f113ffff640 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ad7b0 0x555e2f668a90 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f113400c430 tx=0x7f1134055a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a7628637cdc1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f111f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f111f7fe640 1 -- 172.21.15.133:0/1386749171 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113405d9b0 con 0x555e2f6ad7b0 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f111f7fe640 1 -- 172.21.15.133:0/1386749171 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1134019da0 con 0x555e2f6ad7b0 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 -- 172.21.15.133:0/1386749171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11380a5a50 con 0x555e2f6ad7b0 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f111f7fe640 1 -- 172.21.15.133:0/1386749171 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1134051910 con 0x555e2f6ad7b0 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 10 client.10287 fetch_fsmap learned FSMap version 93 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 -- 172.21.15.133:0/1386749171 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x555e2f46caa0 con 0x555e2f6ad7b0 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f111f7fe640 1 -- 172.21.15.133:0/1386749171 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11380a5a50 con 0x555e2f6ad7b0 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f111f7fe640 1 -- 172.21.15.133:0/1386749171 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 93) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f11340546f0 con 0x555e2f6ad7b0 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 10 client.10287 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 10 client.10287 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 20 client.10287 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x555e2f66ff60 0x555e2f6bf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 4 client.10287 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 -- 172.21.15.133:0/1386749171 --> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e2f44da30 con 0x555e2f66ff60 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f1144e12640 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x555e2f66ff60 0x555e2f6bf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f111f7fe640 10 client.10287 ms_handle_connect on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f1144e12640 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x555e2f66ff60 0x555e2f6bf860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.10041 client_cookie=966162c54db40363 server_cookie=389bde989ff76098 in_seq=0 out_seq=0 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f111f7fe640 1 -- 172.21.15.133:0/1386749171 <== mds.10041 v2:172.21.15.133:6834/2626366082 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e2f44da30 con 0x555e2f66ff60 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f111f7fe640 10 client.10287 handle_command_reply: tid=0 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 client.10287 shutdown 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 -- 172.21.15.133:0/1386749171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ad7b0 msgr2=0x555e2f668a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ad7b0 0x555e2f668a90 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f113400c430 tx=0x7f1134055a60 comp rx=0 tx=0).stop 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 -- 172.21.15.133:0/1386749171 shutdown_connections 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ae0d0 0x555e2f668fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x555e2f66ff60 0x555e2f6bf860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ad7b0 0x555e2f668a90 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1386749171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6639e0 0x555e2f6ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f111f7fe640 0 client.10287 ms_handle_reset on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 -- 172.21.15.133:0/1386749171 >> 172.21.15.133:0/1386749171 conn(0x555e2f669f60 msgr2=0x555e2f662490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 -- 172.21.15.133:0/1386749171 shutdown_connections 2024-04-25T10:14:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 1 -- 172.21.15.133:0/1386749171 wait complete. 2024-04-25T10:14:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.315+0000 7f114a1e3000 20 client.10287 trim_cache size 0 max 16384 2024-04-25T10:14:55.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 Processor -- start 2024-04-25T10:14:55.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 -- start start 2024-04-25T10:14:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6639e0 0x555e2f711ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 0x555e2f712430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ae0d0 0x555e2f712970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140203010 con 0x555e2f6ae0d0 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140203650 con 0x555e2f6639e0 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1140154080 con 0x555e2f6ad7b0 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1145613640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ae0d0 0x555e2f712970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1145613640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ae0d0 0x555e2f712970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43958/0 (socket says 172.21.15.133:43958) 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1145613640 1 -- 172.21.15.133:0/3915013865 learned_addr learned my addr 172.21.15.133:0/3915013865 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1145613640 1 -- 172.21.15.133:0/3915013865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 msgr2=0x555e2f712430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1145613640 1 --2- 172.21.15.133:0/3915013865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 0x555e2f712430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1145613640 1 -- 172.21.15.133:0/3915013865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6639e0 msgr2=0x555e2f711ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1145613640 1 --2- 172.21.15.133:0/3915013865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6639e0 0x555e2f711ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1145613640 1 -- 172.21.15.133:0/3915013865 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e2f533af0 con 0x555e2f6ae0d0 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1145613640 1 --2- 172.21.15.133:0/3915013865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ae0d0 0x555e2f712970 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f11380b9040 tx=0x7f11380a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d62e44b79722b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1125ffb640 1 -- 172.21.15.133:0/3915013865 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380ba070 con 0x555e2f6ae0d0 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1125ffb640 1 -- 172.21.15.133:0/3915013865 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11380bb040 con 0x555e2f6ae0d0 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f1125ffb640 1 -- 172.21.15.133:0/3915013865 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138009af0 con 0x555e2f6ae0d0 2024-04-25T10:14:55.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 -- 172.21.15.133:0/3915013865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ae0d0 msgr2=0x555e2f712970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 --2- 172.21.15.133:0/3915013865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ae0d0 0x555e2f712970 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f11380b9040 tx=0x7f11380a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 -- 172.21.15.133:0/3915013865 shutdown_connections 2024-04-25T10:14:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 --2- 172.21.15.133:0/3915013865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6ae0d0 0x555e2f712970 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 --2- 172.21.15.133:0/3915013865 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 0x555e2f712430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 --2- 172.21.15.133:0/3915013865 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6639e0 0x555e2f711ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 -- 172.21.15.133:0/3915013865 >> 172.21.15.133:0/3915013865 conn(0x555e2f669f60 msgr2=0x555e2f6ac520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 -- 172.21.15.133:0/3915013865 shutdown_connections 2024-04-25T10:14:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 -- 172.21.15.133:0/3915013865 wait complete. 2024-04-25T10:14:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 -- start start 2024-04-25T10:14:55.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.331+0000 7f114a1e3000 1 Processor -- start 2024-04-25T10:14:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f6c1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 0x555e2f6fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 0x555e2f6fe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e2f56e230 con 0x555e2f6639e0 2024-04-25T10:14:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140154680 con 0x555e2f6ae0d0 2024-04-25T10:14:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e2f66cf30 con 0x555e2f6ad7b0 2024-04-25T10:14:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 10 client.0 _ll_register_callbacks cb 0x555e2f4b8450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f1144e12640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f6c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f1144e12640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f6c1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43966/0 (socket says 172.21.15.133:43966) 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f1144e12640 1 -- 172.21.15.133:0/1801579941 learned_addr learned my addr 172.21.15.133:0/1801579941 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f113ffff640 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 0x555e2f6fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f1145613640 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 0x555e2f6fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f1144e12640 1 -- 172.21.15.133:0/1801579941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 msgr2=0x555e2f6fdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f1144e12640 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 0x555e2f6fdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f1144e12640 1 -- 172.21.15.133:0/1801579941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 msgr2=0x555e2f6fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f1144e12640 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 0x555e2f6fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f1144e12640 1 -- 172.21.15.133:0/1801579941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e2f509d50 con 0x555e2f6639e0 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f1145613640 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 0x555e2f6fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f1144e12640 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f6c1480 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f1130006fd0 tx=0x7f1130004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d7b23ac48f16c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f111f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f111f7fe640 1 -- 172.21.15.133:0/1801579941 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130025e60 con 0x555e2f6639e0 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f111f7fe640 1 -- 172.21.15.133:0/1801579941 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1130023b10 con 0x555e2f6639e0 2024-04-25T10:14:55.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 -- 172.21.15.133:0/1801579941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11380bb040 con 0x555e2f6639e0 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f111f7fe640 1 -- 172.21.15.133:0/1801579941 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1130036050 con 0x555e2f6639e0 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 10 client.10293 fetch_fsmap learned FSMap version 93 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 -- 172.21.15.133:0/1801579941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x555e2f458e90 con 0x555e2f6639e0 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f111f7fe640 1 -- 172.21.15.133:0/1801579941 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11380bb040 con 0x555e2f6639e0 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f111f7fe640 1 -- 172.21.15.133:0/1801579941 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 93) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f1130039020 con 0x555e2f6639e0 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 10 client.10293 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 10 client.10293 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 20 client.10293 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x555e2f6fea40 0x555e2f716bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 4 client.10293 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 -- 172.21.15.133:0/1801579941 --> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x555e2f4bf4c0 con 0x555e2f6fea40 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f113ffff640 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x555e2f6fea40 0x555e2f716bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f113ffff640 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x555e2f6fea40 0x555e2f716bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.10041 client_cookie=67599353b64a02e4 server_cookie=408b4ac56f87b4bf in_seq=0 out_seq=0 2024-04-25T10:14:55.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f111f7fe640 10 client.10293 ms_handle_connect on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f111f7fe640 1 -- 172.21.15.133:0/1801579941 <== mds.10041 v2:172.21.15.133:6834/2626366082 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x555e2f4bf4c0 con 0x555e2f6fea40 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f111f7fe640 10 client.10293 handle_command_reply: tid=0 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 client.10293 shutdown 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 -- 172.21.15.133:0/1801579941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 msgr2=0x555e2f6c1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f6c1480 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f1130006fd0 tx=0x7f1130004700 comp rx=0 tx=0).stop 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 -- 172.21.15.133:0/1801579941 shutdown_connections 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555e2f6ae0d0 0x555e2f6fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x555e2f6ad7b0 0x555e2f6fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x555e2f6fea40 0x555e2f716bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 --2- 172.21.15.133:0/1801579941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555e2f6639e0 0x555e2f6c1480 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f111f7fe640 0 client.10293 ms_handle_reset on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 -- 172.21.15.133:0/1801579941 >> 172.21.15.133:0/1801579941 conn(0x555e2f669f60 msgr2=0x555e2f6ac520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 -- 172.21.15.133:0/1801579941 shutdown_connections 2024-04-25T10:14:55.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 1 -- 172.21.15.133:0/1801579941 wait complete. 2024-04-25T10:14:55.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.335+0000 7f114a1e3000 20 client.10293 trim_cache size 0 max 16384 2024-04-25T10:14:55.345 INFO:teuthology.orchestra.run.smithi133.stderr:Error EINVAL: MDS is not active 2024-04-25T10:14:55.345 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:14:55.346 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub abort 2024-04-25T10:14:55.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4f7df640 1 Processor -- start 2024-04-25T10:14:55.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4f7df640 1 -- start start 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4f7df640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0b4815b910 0x7f0b4806b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4f7df640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0b480af070 0x7f0b480acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4f7df640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4806bc80 con 0x7f0b480ad460 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4f7df640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b480af4d0 con 0x7f0b480af070 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4f7df640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4806bf60 con 0x7f0b4815b910 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4cd7a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0b480af070 0x7f0b480acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4cd7a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0b480af070 0x7f0b480acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43970/0 (socket says 172.21.15.133:43970) 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4dd7c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0b480ad460 0x7f0b480b35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46828/0 (socket says 172.21.15.133:46828) 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4cd7a640 1 -- 172.21.15.133:0/824135622 learned_addr learned my addr 172.21.15.133:0/824135622 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4cd7a640 1 -- 172.21.15.133:0/824135622 >> v2:172.21.15.162:3300/0 conn(0x7f0b4815b910 msgr2=0x7f0b4806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/824135622 >> v2:172.21.15.162:3300/0 conn(0x7f0b4815b910 0x7f0b4806b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4cd7a640 1 -- 172.21.15.133:0/824135622 >> v1:172.21.15.133:6789/0 conn(0x7f0b480ad460 legacy=0x7f0b480b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b3bfff640 1 -- 172.21.15.133:0/824135622 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968797828 0 0) 0x7f0b4806bc80 con 0x7f0b480ad460 2024-04-25T10:14:55.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.459+0000 7f0b4cd7a640 1 -- 172.21.15.133:0/824135622 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4806c240 con 0x7f0b480af070 2024-04-25T10:14:55.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/824135622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b480af070 0x7f0b480acf20 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f0b480d0de0 tx=0x7f0b3c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffba34cb34db3dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b3bfff640 1 -- 172.21.15.133:0/824135622 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0b0070 con 0x7f0b480af070 2024-04-25T10:14:55.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b3bfff640 1 -- 172.21.15.133:0/824135622 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c0ab600 con 0x7f0b480af070 2024-04-25T10:14:55.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b3bfff640 1 -- 172.21.15.133:0/824135622 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0ab9d0 con 0x7f0b480af070 2024-04-25T10:14:55.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 -- 172.21.15.133:0/824135622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b480af070 msgr2=0x7f0b480acf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 --2- 172.21.15.133:0/824135622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b480af070 0x7f0b480acf20 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f0b480d0de0 tx=0x7f0b3c01c520 comp rx=0 tx=0).stop 2024-04-25T10:14:55.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 -- 172.21.15.133:0/824135622 shutdown_connections 2024-04-25T10:14:55.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 --2- 172.21.15.133:0/824135622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b480af070 0x7f0b480acf20 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 --2- 172.21.15.133:0/824135622 >> v2:172.21.15.162:3300/0 conn(0x7f0b4815b910 0x7f0b4806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 -- 172.21.15.133:0/824135622 >> 172.21.15.133:0/824135622 conn(0x7f0b480cfd50 msgr2=0x7f0b480d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 -- 172.21.15.133:0/824135622 shutdown_connections 2024-04-25T10:14:55.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 -- 172.21.15.133:0/824135622 wait complete. 2024-04-25T10:14:55.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 Processor -- start 2024-04-25T10:14:55.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 -- start start 2024-04-25T10:14:55.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b480ad460 0x7f0b481589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b480af070 0x7f0b4815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4815b910 0x7f0b48151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481685c0 con 0x7f0b4815b910 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480ad8b0 con 0x7f0b480af070 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.463+0000 7f0b4f7df640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b48168740 con 0x7f0b480ad460 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4dd7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4815b910 0x7f0b48151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4dd7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4815b910 0x7f0b48151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43978/0 (socket says 172.21.15.133:43978) 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4dd7c640 1 -- 172.21.15.133:0/3564999351 learned_addr learned my addr 172.21.15.133:0/3564999351 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/3564999351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b480af070 0x7f0b4815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4dd7c640 1 -- 172.21.15.133:0/3564999351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b480ad460 msgr2=0x7f0b481589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/3564999351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b480ad460 0x7f0b481589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4dd7c640 1 --2- 172.21.15.133:0/3564999351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b480ad460 0x7f0b481589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4dd7c640 1 -- 172.21.15.133:0/3564999351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b480af070 msgr2=0x7f0b4815acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4dd7c640 1 --2- 172.21.15.133:0/3564999351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b480af070 0x7f0b4815acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4dd7c640 1 -- 172.21.15.133:0/3564999351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b48153090 con 0x7f0b4815b910 2024-04-25T10:14:55.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/3564999351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0b480af070 0x7f0b4815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/3564999351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0b480ad460 0x7f0b481589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4dd7c640 1 --2- 172.21.15.133:0/3564999351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4815b910 0x7f0b48151b70 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000c7a0 tx=0x7f0b4000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b39ffb640 1 -- 172.21.15.133:0/3564999351 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40018070 con 0x7f0b4815b910 2024-04-25T10:14:55.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b39ffb640 1 -- 172.21.15.133:0/3564999351 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4000a640 con 0x7f0b4815b910 2024-04-25T10:14:55.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4f7df640 1 -- 172.21.15.133:0/3564999351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b48153370 con 0x7f0b4815b910 2024-04-25T10:14:55.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b39ffb640 1 -- 172.21.15.133:0/3564999351 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4000aab0 con 0x7f0b4815b910 2024-04-25T10:14:55.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4f7df640 1 -- 172.21.15.133:0/3564999351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b48099a70 con 0x7f0b4815b910 2024-04-25T10:14:55.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b39ffb640 1 -- 172.21.15.133:0/3564999351 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b4001b070 con 0x7f0b4815b910 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b39ffb640 1 --2- 172.21.15.133:0/3564999351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b2c066f60 0x7f0b2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b39ffb640 1 -- 172.21.15.133:0/3564999351 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0b40084840 con 0x7f0b4815b910 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/3564999351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b2c066f60 0x7f0b2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b5294c000 1 Processor -- start 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/3564999351 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0b2c066f60 0x7f0b2c069420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0b4809d410 tx=0x7f0b44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.467+0000 7f0b5294c000 1 -- start start 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d9b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7d96b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d96b9e0 0x564b7d96bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b7d852ad0 con 0x564b7d9b60d0 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b7d7b9f80 con 0x564b7d9b57b0 2024-04-25T10:14:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b7d741ee0 con 0x564b7d96b9e0 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4cd7a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7d96b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4cd7a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7d96b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43994/0 (socket says 172.21.15.133:43994) 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4cd7a640 1 -- 172.21.15.133:0/3300390644 learned_addr learned my addr 172.21.15.133:0/3300390644 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4dd7c640 1 --2- 172.21.15.133:0/3300390644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d96b9e0 0x564b7d96bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/3300390644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d9b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4cd7a640 1 -- 172.21.15.133:0/3300390644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d96b9e0 msgr2=0x564b7d96bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/3300390644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d96b9e0 0x564b7d96bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4cd7a640 1 -- 172.21.15.133:0/3300390644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 msgr2=0x564b7d9b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/3300390644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d9b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4cd7a640 1 -- 172.21.15.133:0/3300390644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b7d772930 con 0x564b7d9b60d0 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/3300390644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d9b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/3300390644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7d96b4a0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0ae040 tx=0x7f0b3c0a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab57aaa3e3abfc5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b32ffd640 1 -- 172.21.15.133:0/3300390644 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0b0040 con 0x564b7d9b60d0 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b32ffd640 1 -- 172.21.15.133:0/3300390644 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c0ab840 con 0x564b7d9b60d0 2024-04-25T10:14:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b32ffd640 1 -- 172.21.15.133:0/3300390644 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0abbf0 con 0x564b7d9b60d0 2024-04-25T10:14:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- 172.21.15.133:0/3300390644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 msgr2=0x564b7d96b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 --2- 172.21.15.133:0/3300390644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7d96b4a0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0ae040 tx=0x7f0b3c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:14:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- 172.21.15.133:0/3300390644 shutdown_connections 2024-04-25T10:14:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 --2- 172.21.15.133:0/3300390644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d96b9e0 0x564b7d96bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 --2- 172.21.15.133:0/3300390644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7d96b4a0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 --2- 172.21.15.133:0/3300390644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d9b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- 172.21.15.133:0/3300390644 >> 172.21.15.133:0/3300390644 conn(0x564b7d971f60 msgr2=0x564b7d969d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- 172.21.15.133:0/3300390644 shutdown_connections 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- 172.21.15.133:0/3300390644 wait complete. 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- start start 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 Processor -- start 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d970a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b60d0 0x564b7d970fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b7d774390 con 0x564b7d96b9e0 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b7d7b6020 con 0x564b7d9b57b0 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.471+0000 7f0b5294c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b7d7b6ff0 con 0x564b7d9b60d0 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4d57b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 10 client.0 _ll_register_callbacks cb 0x564b7d86d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4d57b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44010/0 (socket says 172.21.15.133:44010) 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4d57b640 1 -- 172.21.15.133:0/1422229551 learned_addr learned my addr 172.21.15.133:0/1422229551 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4dd7c640 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b60d0 0x564b7d970fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d970a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4d57b640 1 -- 172.21.15.133:0/1422229551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b60d0 msgr2=0x564b7d970fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b60d0 0x564b7d970fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4d57b640 1 -- 172.21.15.133:0/1422229551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 msgr2=0x564b7d970a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d970a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4d57b640 1 -- 172.21.15.133:0/1422229551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b7d742130 con 0x564b7d96b9e0 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9b5200 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400a180 tx=0x7f0b440046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1419aa9cf1f7cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b27fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b27fff640 1 -- 172.21.15.133:0/1422229551 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b440117e0 con 0x564b7d96b9e0 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b27fff640 1 -- 172.21.15.133:0/1422229551 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0b44011980 con 0x564b7d96b9e0 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 -- 172.21.15.133:0/1422229551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b3c0ab840 con 0x564b7d96b9e0 2024-04-25T10:14:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b27fff640 1 -- 172.21.15.133:0/1422229551 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b4402b050 con 0x564b7d96b9e0 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 10 client.10308 fetch_fsmap learned FSMap version 93 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 -- 172.21.15.133:0/1422229551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x564b7d774aa0 con 0x564b7d96b9e0 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b27fff640 1 -- 172.21.15.133:0/1422229551 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c0ab840 con 0x564b7d96b9e0 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b27fff640 1 -- 172.21.15.133:0/1422229551 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 93) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f0b4402e020 con 0x564b7d96b9e0 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 10 client.10308 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 10 client.10308 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 20 client.10308 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x564b7d977ff0 0x564b7d9c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 4 client.10308 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 -- 172.21.15.133:0/1422229551 --> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b7d755a30 con 0x564b7d977ff0 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x564b7d977ff0 0x564b7d9c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b27fff640 10 client.10308 ms_handle_connect on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x564b7d977ff0 0x564b7d9c7860 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.10041 client_cookie=e3882716715f49e2 server_cookie=51562d0f9bf94f02 in_seq=0 out_seq=0 2024-04-25T10:14:55.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b27fff640 1 -- 172.21.15.133:0/1422229551 <== mds.10041 v2:172.21.15.133:6834/2626366082 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b7d755a30 con 0x564b7d977ff0 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b27fff640 10 client.10308 handle_command_reply: tid=0 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 client.10308 shutdown 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 -- 172.21.15.133:0/1422229551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 msgr2=0x564b7d9b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9b5200 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400a180 tx=0x7f0b440046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 -- 172.21.15.133:0/1422229551 shutdown_connections 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b60d0 0x564b7d970fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d970a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x564b7d977ff0 0x564b7d9c7860 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 --2- 172.21.15.133:0/1422229551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9b5200 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b27fff640 0 client.10308 ms_handle_reset on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 -- 172.21.15.133:0/1422229551 >> 172.21.15.133:0/1422229551 conn(0x564b7d971f60 msgr2=0x564b7d96a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 -- 172.21.15.133:0/1422229551 shutdown_connections 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 1 -- 172.21.15.133:0/1422229551 wait complete. 2024-04-25T10:14:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.475+0000 7f0b5294c000 20 client.10308 trim_cache size 0 max 16384 2024-04-25T10:14:55.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 Processor -- start 2024-04-25T10:14:55.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 -- start start 2024-04-25T10:14:55.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9c6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d9c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b60d0 0x564b7d9c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b48200ce0 con 0x564b7d96b9e0 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b482012e0 con 0x564b7d9b57b0 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481f8620 con 0x564b7d9b60d0 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b4d57b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b4d57b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44012/0 (socket says 172.21.15.133:44012) 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b4d57b640 1 -- 172.21.15.133:0/2830822711 learned_addr learned my addr 172.21.15.133:0/2830822711 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/2830822711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d9c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b4dd7c640 1 --2- 172.21.15.133:0/2830822711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b60d0 0x564b7d9c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b4d57b640 1 -- 172.21.15.133:0/2830822711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b60d0 msgr2=0x564b7d9c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/2830822711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b60d0 0x564b7d9c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b4d57b640 1 -- 172.21.15.133:0/2830822711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 msgr2=0x564b7d9c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/2830822711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d9c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b4d57b640 1 -- 172.21.15.133:0/2830822711 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b7d83baf0 con 0x564b7d96b9e0 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/2830822711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9c6860 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400c520 tx=0x7f0b4401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fea915dc2338ec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b327fc640 1 -- 172.21.15.133:0/2830822711 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b440115e0 con 0x564b7d96b9e0 2024-04-25T10:14:55.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b327fc640 1 -- 172.21.15.133:0/2830822711 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44033070 con 0x564b7d96b9e0 2024-04-25T10:14:55.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b327fc640 1 -- 172.21.15.133:0/2830822711 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44011a00 con 0x564b7d96b9e0 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 -- 172.21.15.133:0/2830822711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 msgr2=0x564b7d9c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 --2- 172.21.15.133:0/2830822711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9c6860 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400c520 tx=0x7f0b4401c000 comp rx=0 tx=0).stop 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 -- 172.21.15.133:0/2830822711 shutdown_connections 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 --2- 172.21.15.133:0/2830822711 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b60d0 0x564b7d9c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 --2- 172.21.15.133:0/2830822711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d9b57b0 0x564b7d9c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 --2- 172.21.15.133:0/2830822711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d96b9e0 0x564b7d9c6860 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 -- 172.21.15.133:0/2830822711 >> 172.21.15.133:0/2830822711 conn(0x564b7d971f60 msgr2=0x564b7d976770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 -- 172.21.15.133:0/2830822711 shutdown_connections 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 -- 172.21.15.133:0/2830822711 wait complete. 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 -- start start 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.499+0000 7f0b5294c000 1 Processor -- start 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d96b9e0 0x564b7da16d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b57b0 0x564b7da17290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7da13380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b7d760a50 con 0x564b7d9b60d0 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481fff20 con 0x564b7d96b9e0 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b7d7b9290 con 0x564b7d9b57b0 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 10 client.0 _ll_register_callbacks cb 0x564b7d7c0450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4dd7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7da13380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4dd7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7da13380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44014/0 (socket says 172.21.15.133:44014) 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4dd7c640 1 -- 172.21.15.133:0/618674152 learned_addr learned my addr 172.21.15.133:0/618674152 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d96b9e0 0x564b7da16d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4cd7a640 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b57b0 0x564b7da17290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4dd7c640 1 -- 172.21.15.133:0/618674152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b57b0 msgr2=0x564b7da17290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4dd7c640 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b57b0 0x564b7da17290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4dd7c640 1 -- 172.21.15.133:0/618674152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d96b9e0 msgr2=0x564b7da16d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4dd7c640 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d96b9e0 0x564b7da16d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4dd7c640 1 -- 172.21.15.133:0/618674152 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b7d811d50 con 0x564b7d9b60d0 2024-04-25T10:14:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4dd7c640 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7da13380 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f0b40082bb0 tx=0x7f0b40055750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864d776b7682dce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b27fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b27fff640 1 -- 172.21.15.133:0/618674152 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4000ae50 con 0x564b7d9b60d0 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b27fff640 1 -- 172.21.15.133:0/618674152 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0b4005b0c0 con 0x564b7d9b60d0 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 -- 172.21.15.133:0/618674152 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b44033070 con 0x564b7d9b60d0 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b27fff640 1 -- 172.21.15.133:0/618674152 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b4005d6a0 con 0x564b7d9b60d0 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 10 client.10314 fetch_fsmap learned FSMap version 93 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 -- 172.21.15.133:0/618674152 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x564b7d760e90 con 0x564b7d9b60d0 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b27fff640 1 -- 172.21.15.133:0/618674152 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44033070 con 0x564b7d9b60d0 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b27fff640 1 -- 172.21.15.133:0/618674152 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 93) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f0b400543e0 con 0x564b7d9b60d0 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 10 client.10314 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 10 client.10314 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 20 client.10314 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x564b7da138c0 0x564b7da13ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 4 client.10314 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 -- 172.21.15.133:0/618674152 --> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x564b7d7c74c0 con 0x564b7da138c0 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x564b7da138c0 0x564b7da13ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b4d57b640 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x564b7da138c0 0x564b7da13ca0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.10041 client_cookie=9d0aa3f8d6fe3d9b server_cookie=dd4bcea642ac631d in_seq=0 out_seq=0 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b27fff640 10 client.10314 ms_handle_connect on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b27fff640 1 -- 172.21.15.133:0/618674152 <== mds.10041 v2:172.21.15.133:6834/2626366082 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x564b7d7c74c0 con 0x564b7da138c0 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b27fff640 10 client.10314 handle_command_reply: tid=0 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 client.10314 shutdown 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 -- 172.21.15.133:0/618674152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 msgr2=0x564b7da13380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7da13380 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f0b40082bb0 tx=0x7f0b40055750 comp rx=0 tx=0).stop 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 -- 172.21.15.133:0/618674152 shutdown_connections 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x564b7da138c0 0x564b7da13ca0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b7d9b60d0 0x564b7da13380 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564b7d9b57b0 0x564b7da17290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b27fff640 0 client.10314 ms_handle_reset on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 --2- 172.21.15.133:0/618674152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564b7d96b9e0 0x564b7da16d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 -- 172.21.15.133:0/618674152 >> 172.21.15.133:0/618674152 conn(0x564b7d971f60 msgr2=0x564b7d977000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 -- 172.21.15.133:0/618674152 shutdown_connections 2024-04-25T10:14:55.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 1 -- 172.21.15.133:0/618674152 wait complete. 2024-04-25T10:14:55.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.503+0000 7f0b5294c000 20 client.10314 trim_cache size 0 max 16384 2024-04-25T10:14:55.512 INFO:teuthology.orchestra.run.smithi133.stderr:Error EINVAL: MDS is not active 2024-04-25T10:14:55.512 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:14:55.513 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c scrub resume 2024-04-25T10:14:55.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.623+0000 7fa5ba214640 1 Processor -- start 2024-04-25T10:14:55.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.623+0000 7fa5ba214640 1 -- start start 2024-04-25T10:14:55.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.623+0000 7fa5ba214640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5b40cd820 0x7fa5b40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.623+0000 7fa5ba214640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5b415ced0 0x7fa5b415d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.623+0000 7fa5ba214640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b40ce2e0 con 0x7fa5b4145170 2024-04-25T10:14:55.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.623+0000 7fa5ba214640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b406db00 con 0x7fa5b40cd820 2024-04-25T10:14:55.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.623+0000 7fa5ba214640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b4145550 con 0x7fa5b415ced0 2024-04-25T10:14:55.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.623+0000 7fa5b2ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5b40cd820 0x7fa5b40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.623+0000 7fa5b2ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa5b40cd820 0x7fa5b40cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44016/0 (socket says 172.21.15.133:44016) 2024-04-25T10:14:55.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.623+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/536328218 learned_addr learned my addr 172.21.15.133:0/536328218 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/536328218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5b415ced0 0x7fa5b415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/536328218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5b415ced0 msgr2=0x7fa5b415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/536328218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5b415ced0 0x7fa5b415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/536328218 >> v1:172.21.15.162:6789/0 conn(0x7fa5b4145170 legacy=0x7fa5b406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/536328218 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b406d4b0 con 0x7fa5b40cd820 2024-04-25T10:14:55.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/536328218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b40cd820 0x7fa5b40cdca0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009870 tx=0x7fa59c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ebb9ee4c4c45bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5b27fc640 1 -- 172.21.15.133:0/536328218 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0b0070 con 0x7fa5b40cd820 2024-04-25T10:14:55.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5b27fc640 1 -- 172.21.15.133:0/536328218 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c0ab600 con 0x7fa5b40cd820 2024-04-25T10:14:55.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5b27fc640 1 -- 172.21.15.133:0/536328218 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0ab9d0 con 0x7fa5b40cd820 2024-04-25T10:14:55.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5ba214640 1 -- 172.21.15.133:0/536328218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b40cd820 msgr2=0x7fa5b40cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5ba214640 1 --2- 172.21.15.133:0/536328218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b40cd820 0x7fa5b40cdca0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009870 tx=0x7fa59c01c520 comp rx=0 tx=0).stop 2024-04-25T10:14:55.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5ba214640 1 -- 172.21.15.133:0/536328218 shutdown_connections 2024-04-25T10:14:55.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5ba214640 1 --2- 172.21.15.133:0/536328218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5b415ced0 0x7fa5b415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5ba214640 1 --2- 172.21.15.133:0/536328218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b40cd820 0x7fa5b40cdca0 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5ba214640 1 -- 172.21.15.133:0/536328218 >> 172.21.15.133:0/536328218 conn(0x7fa5b4066f70 msgr2=0x7fa5b406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5ba214640 1 -- 172.21.15.133:0/536328218 shutdown_connections 2024-04-25T10:14:55.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.627+0000 7fa5ba214640 1 -- 172.21.15.133:0/536328218 wait complete. 2024-04-25T10:14:55.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5ba214640 1 Processor -- start 2024-04-25T10:14:55.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5ba214640 1 -- start start 2024-04-25T10:14:55.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5ba214640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5b40cd820 0x7fa5b4127ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5ba214640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b4145170 0x7fa5b41388e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5ba214640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5b415ced0 0x7fa5b4143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5ba214640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40b1e50 con 0x7fa5b4145170 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5ba214640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40b1cd0 con 0x7fa5b415ced0 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5ba214640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40b1fd0 con 0x7fa5b40cd820 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b4145170 0x7fa5b41388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b4145170 0x7fa5b41388e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44024/0 (socket says 172.21.15.133:44024) 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/2547240754 learned_addr learned my addr 172.21.15.133:0/2547240754 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/2547240754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5b415ced0 0x7fa5b4143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/2547240754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5b40cd820 0x7fa5b4127ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/2547240754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5b40cd820 msgr2=0x7fa5b4127ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/2547240754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa5b40cd820 0x7fa5b4127ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/2547240754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5b415ced0 msgr2=0x7fa5b4143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/2547240754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5b415ced0 0x7fa5b4143900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/2547240754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b40ae4f0 con 0x7fa5b4145170 2024-04-25T10:14:55.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/2547240754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa5b415ced0 0x7fa5b4143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/2547240754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5b4145170 0x7fa5b41388e0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0ae040 tx=0x7fa59c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5abfff640 1 -- 172.21.15.133:0/2547240754 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0ba020 con 0x7fa5b4145170 2024-04-25T10:14:55.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5abfff640 1 -- 172.21.15.133:0/2547240754 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c0b6420 con 0x7fa5b4145170 2024-04-25T10:14:55.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5abfff640 1 -- 172.21.15.133:0/2547240754 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0b0040 con 0x7fa5b4145170 2024-04-25T10:14:55.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5ba214640 1 -- 172.21.15.133:0/2547240754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b406e040 con 0x7fa5b4145170 2024-04-25T10:14:55.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.631+0000 7fa5ba214640 1 -- 172.21.15.133:0/2547240754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b4128be0 con 0x7fa5b4145170 2024-04-25T10:14:55.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5abfff640 1 -- 172.21.15.133:0/2547240754 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa59c0bd020 con 0x7fa5b4145170 2024-04-25T10:14:55.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5abfff640 1 --2- 172.21.15.133:0/2547240754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa594066f60 0x7fa594069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5abfff640 1 -- 172.21.15.133:0/2547240754 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa59c11f410 con 0x7fa5b4145170 2024-04-25T10:14:55.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/2547240754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa594066f60 0x7fa594069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/2547240754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa594066f60 0x7fa594069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009f20 tx=0x7fa5ac0094d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 Processor -- start 2024-04-25T10:14:55.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 -- start start 2024-04-25T10:14:55.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf1de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 0x561dcced8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dcced8d90 0x561dcced9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dccdbb1f0 con 0x561dccf1da40 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dccd21e90 con 0x561dccf1e360 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561dccca9f50 con 0x561dcced8d90 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf1de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf1de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44034/0 (socket says 172.21.15.133:44034) 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b37fe640 1 -- 172.21.15.133:0/236413746 learned_addr learned my addr 172.21.15.133:0/236413746 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/236413746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dcced8d90 0x561dcced9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/236413746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 0x561dcced8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b37fe640 1 -- 172.21.15.133:0/236413746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dcced8d90 msgr2=0x561dcced9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/236413746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dcced8d90 0x561dcced9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b37fe640 1 -- 172.21.15.133:0/236413746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 msgr2=0x561dcced8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/236413746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 0x561dcced8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b37fe640 1 -- 172.21.15.133:0/236413746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dcccf98b0 con 0x561dccf1da40 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/236413746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 0x561dcced8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/236413746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf1de20 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009b60 tx=0x7fa5ac011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b19508cc9c1f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5a97fa640 1 -- 172.21.15.133:0/236413746 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac017dd0 con 0x561dccf1da40 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5a97fa640 1 -- 172.21.15.133:0/236413746 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac003980 con 0x561dccf1da40 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5a97fa640 1 -- 172.21.15.133:0/236413746 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac003b50 con 0x561dccf1da40 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 -- 172.21.15.133:0/236413746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 msgr2=0x561dccf1de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 --2- 172.21.15.133:0/236413746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf1de20 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009b60 tx=0x7fa5ac011000 comp rx=0 tx=0).stop 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 -- 172.21.15.133:0/236413746 shutdown_connections 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 --2- 172.21.15.133:0/236413746 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dcced8d90 0x561dcced9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 --2- 172.21.15.133:0/236413746 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 0x561dcced8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 --2- 172.21.15.133:0/236413746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf1de20 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.635+0000 7fa5bd381000 1 -- 172.21.15.133:0/236413746 >> 172.21.15.133:0/236413746 conn(0x561dcced4d40 msgr2=0x561dcced7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 -- 172.21.15.133:0/236413746 shutdown_connections 2024-04-25T10:14:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 -- 172.21.15.133:0/236413746 wait complete. 2024-04-25T10:14:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 -- start start 2024-04-25T10:14:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 Processor -- start 2024-04-25T10:14:55.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dcced8d90 0x561dccf251c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf25700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 0x561dcceded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcccdc540 con 0x561dccf1da40 2024-04-25T10:14:55.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dccd1df30 con 0x561dccf1e360 2024-04-25T10:14:55.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561dcccc52d0 con 0x561dcced8d90 2024-04-25T10:14:55.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 10 client.0 _ll_register_callbacks cb 0x561dccdd6180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:55.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf25700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf25700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44050/0 (socket says 172.21.15.133:44050) 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/2446392411 learned_addr learned my addr 172.21.15.133:0/2446392411 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dcced8d90 0x561dccf251c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 0x561dcceded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/2446392411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dcced8d90 msgr2=0x561dccf251c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dcced8d90 0x561dccf251c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/2446392411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 msgr2=0x561dcceded70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 0x561dcceded70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b2ffd640 1 -- 172.21.15.133:0/2446392411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dcccaa1a0 con 0x561dccf1da40 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 0x561dcceded70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf25700 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fa59c01c350 tx=0x7fa59c0ef430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db792c2df2a5c0e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5927fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5927fc640 1 -- 172.21.15.133:0/2446392411 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0efad0 con 0x561dccf1da40 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5927fc640 1 -- 172.21.15.133:0/2446392411 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa59c11f0b0 con 0x561dccf1da40 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 -- 172.21.15.133:0/2446392411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5ac003980 con 0x561dccf1da40 2024-04-25T10:14:55.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5927fc640 1 -- 172.21.15.133:0/2446392411 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa59c0f3c20 con 0x561dccf1da40 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 10 client.10326 fetch_fsmap learned FSMap version 93 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 -- 172.21.15.133:0/2446392411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x561dccda4840 con 0x561dccf1da40 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5927fc640 1 -- 172.21.15.133:0/2446392411 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac003980 con 0x561dccf1da40 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5927fc640 1 -- 172.21.15.133:0/2446392411 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 93) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fa59c0b3030 con 0x561dccf1da40 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 10 client.10326 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 10 client.10326 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 20 client.10326 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x561dccedfb10 0x561dccedfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 4 client.10326 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 -- 172.21.15.133:0/2446392411 --> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561dcccbdbe0 con 0x561dccedfb10 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x561dccedfb10 0x561dccedfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x561dccedfb10 0x561dccedfef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.10041 client_cookie=92c52845509b137 server_cookie=b7ec5192981055b2 in_seq=0 out_seq=0 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5927fc640 10 client.10326 ms_handle_connect on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5927fc640 1 -- 172.21.15.133:0/2446392411 <== mds.10041 v2:172.21.15.133:6834/2626366082 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561dcccbdbe0 con 0x561dccedfb10 2024-04-25T10:14:55.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5927fc640 10 client.10326 handle_command_reply: tid=0 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 client.10326 shutdown 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 -- 172.21.15.133:0/2446392411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 msgr2=0x561dccf25700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.639+0000 7fa5bd381000 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf25700 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fa59c01c350 tx=0x7fa59c0ef430 comp rx=0 tx=0).stop 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.643+0000 7fa5bd381000 1 -- 172.21.15.133:0/2446392411 shutdown_connections 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.643+0000 7fa5bd381000 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dccf1e360 0x561dcceded70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.643+0000 7fa5bd381000 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x561dccedfb10 0x561dccedfef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.643+0000 7fa5bd381000 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1da40 0x561dccf25700 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.643+0000 7fa5bd381000 1 --2- 172.21.15.133:0/2446392411 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dcced8d90 0x561dccf251c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.643+0000 7fa5927fc640 0 client.10326 ms_handle_reset on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.643+0000 7fa5bd381000 1 -- 172.21.15.133:0/2446392411 >> 172.21.15.133:0/2446392411 conn(0x561dcced4d40 msgr2=0x561dcced79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.643+0000 7fa5bd381000 1 -- 172.21.15.133:0/2446392411 shutdown_connections 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.643+0000 7fa5bd381000 1 -- 172.21.15.133:0/2446392411 wait complete. 2024-04-25T10:14:55.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.643+0000 7fa5bd381000 20 client.10326 trim_cache size 0 max 16384 2024-04-25T10:14:55.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.655+0000 7fa5bd381000 1 Processor -- start 2024-04-25T10:14:55.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.655+0000 7fa5bd381000 1 -- start start 2024-04-25T10:14:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.655+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dcced8d90 0x561dcced84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.655+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dccf1da40 0x561dcced7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.655+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dcced7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.655+0000 7fa5bd381000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b4138250 con 0x561dccf1e360 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.655+0000 7fa5bd381000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b42031c0 con 0x561dcced8d90 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.655+0000 7fa5bd381000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b4133520 con 0x561dccf1da40 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.655+0000 7fa5b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dcced7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.655+0000 7fa5b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dcced7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44052/0 (socket says 172.21.15.133:44052) 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.655+0000 7fa5b3fff640 1 -- 172.21.15.133:0/2996828932 learned_addr learned my addr 172.21.15.133:0/2996828932 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/2996828932 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dccf1da40 0x561dcced7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 -- 172.21.15.133:0/2996828932 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dccf1da40 msgr2=0x561dcced7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/2996828932 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dccf1da40 0x561dcced7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 -- 172.21.15.133:0/2996828932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dcced8d90 msgr2=0x561dcced84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/2996828932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dcced8d90 0x561dcced84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/2996828932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dcced8d90 0x561dcced84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 -- 172.21.15.133:0/2996828932 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dccd7ba10 con 0x561dccf1e360 2024-04-25T10:14:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/2996828932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dcced7670 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400ca90 tx=0x7fa5a400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f95a5da352122b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5a8ff9640 1 -- 172.21.15.133:0/2996828932 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4019070 con 0x561dccf1e360 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5a8ff9640 1 -- 172.21.15.133:0/2996828932 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a4010040 con 0x561dccf1e360 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5a8ff9640 1 -- 172.21.15.133:0/2996828932 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4015400 con 0x561dccf1e360 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- 172.21.15.133:0/2996828932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 msgr2=0x561dcced7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 --2- 172.21.15.133:0/2996828932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dcced7670 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400ca90 tx=0x7fa5a400cf50 comp rx=0 tx=0).stop 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- 172.21.15.133:0/2996828932 shutdown_connections 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 --2- 172.21.15.133:0/2996828932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dcced7670 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 --2- 172.21.15.133:0/2996828932 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dccf1da40 0x561dcced7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 --2- 172.21.15.133:0/2996828932 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dcced8d90 0x561dcced84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- 172.21.15.133:0/2996828932 >> 172.21.15.133:0/2996828932 conn(0x561dcced4d40 msgr2=0x561dccf1c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- 172.21.15.133:0/2996828932 shutdown_connections 2024-04-25T10:14:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- 172.21.15.133:0/2996828932 wait complete. 2024-04-25T10:14:55.668 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:14:55.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- start start 2024-04-25T10:14:55.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 Processor -- start 2024-04-25T10:14:55.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dcced8d90 0x561dccf7e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dccf1da40 0x561dccf7ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dccf22ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dccd211a0 con 0x561dccf1e360 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b413f070 con 0x561dcced8d90 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561dcccdc540 con 0x561dccf1da40 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 10 client.0 _ll_register_callbacks cb 0x561dccce4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dccf22ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dccf22ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44058/0 (socket says 172.21.15.133:44058) 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 -- 172.21.15.133:0/3088124580 learned_addr learned my addr 172.21.15.133:0/3088124580 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dcced8d90 0x561dccf7e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dccf1da40 0x561dccf7ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 -- 172.21.15.133:0/3088124580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dccf1da40 msgr2=0x561dccf7ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dccf1da40 0x561dccf7ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 -- 172.21.15.133:0/3088124580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dcced8d90 msgr2=0x561dccf7e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dcced8d90 0x561dccf7e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 -- 172.21.15.133:0/3088124580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dcccb6b00 con 0x561dccf1e360 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dcced8d90 0x561dccf7e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b2ffd640 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dccf1da40 0x561dccf7ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5b3fff640 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dccf22ed0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4000c10 tx=0x7fa5a4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da751ea31490d5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5927fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5927fc640 1 -- 172.21.15.133:0/3088124580 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a402d060 con 0x561dccf1e360 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- 172.21.15.133:0/3088124580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5a4010040 con 0x561dccf1e360 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5927fc640 1 -- 172.21.15.133:0/3088124580 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa5a4004250 con 0x561dccf1e360 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5927fc640 1 -- 172.21.15.133:0/3088124580 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a4033050 con 0x561dccf1e360 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 10 client.10332 fetch_fsmap learned FSMap version 93 2024-04-25T10:14:55.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- 172.21.15.133:0/3088124580 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x561dcccc8f80 con 0x561dccf1e360 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5927fc640 1 -- 172.21.15.133:0/3088124580 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a4010040 con 0x561dccf1e360 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5927fc640 1 -- 172.21.15.133:0/3088124580 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 93) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fa5a4036020 con 0x561dccf1e360 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 10 client.10332 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 10 client.10332 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 20 client.10332 populate_metadata read hostname 'smithi133' 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x561dccf85520 0x561dccf87910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 4 client.10332 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.659+0000 7fa5bd381000 1 -- 172.21.15.133:0/3088124580 --> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x561dccd2f3d0 con 0x561dccf85520 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x561dccf85520 0x561dccf87910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5b37fe640 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x561dccf85520 0x561dccf87910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.10041 client_cookie=c1a8dfeee56b1d93 server_cookie=bbb0eff8cbf496b1 in_seq=0 out_seq=0 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5927fc640 10 client.10332 ms_handle_connect on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5927fc640 1 -- 172.21.15.133:0/3088124580 <== mds.10041 v2:172.21.15.133:6834/2626366082 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x561dccd2f3d0 con 0x561dccf85520 2024-04-25T10:14:55.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5927fc640 10 client.10332 handle_command_reply: tid=0 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 1 client.10332 shutdown 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 1 -- 172.21.15.133:0/3088124580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 msgr2=0x561dccf22ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dccf22ed0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4000c10 tx=0x7fa5a4004740 comp rx=0 tx=0).stop 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 1 -- 172.21.15.133:0/3088124580 shutdown_connections 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.133:6834/2626366082,v1:172.21.15.133:6835/2626366082] conn(0x561dccf85520 0x561dccf87910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561dccf1e360 0x561dccf22ed0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561dccf1da40 0x561dccf7ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 1 --2- 172.21.15.133:0/3088124580 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561dcced8d90 0x561dccf7e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5927fc640 0 client.10332 ms_handle_reset on v2:172.21.15.133:6834/2626366082 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 1 -- 172.21.15.133:0/3088124580 >> 172.21.15.133:0/3088124580 conn(0x561dcced4d40 msgr2=0x561dccf1c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 1 -- 172.21.15.133:0/3088124580 shutdown_connections 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 1 -- 172.21.15.133:0/3088124580 wait complete. 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.663+0000 7fa5bd381000 20 client.10332 trim_cache size 0 max 16384 2024-04-25T10:14:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:Error EINVAL: MDS is not active 2024-04-25T10:14:55.673 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:14:55.673 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:55.705 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:55.760 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:14:55.760 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:14:55.760 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:55.766 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:55.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3233168640 1 Processor -- start 2024-04-25T10:14:55.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3233168640 1 -- start start 2024-04-25T10:14:55.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3233168640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f322c15db00 0x7f322c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3233168640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322c0d0770 0x7f322c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3233168640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c0ceb20 con 0x7f322c1614c0 2024-04-25T10:14:55.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3233168640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0d1110 con 0x7f322c15db00 2024-04-25T10:14:55.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3233168640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b7740 con 0x7f322c0d0770 2024-04-25T10:14:55.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3230f04640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f322c15db00 0x7f322c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3231705640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f322c1614c0 0x7f322c161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46842/0 (socket says 172.21.15.133:46842) 2024-04-25T10:14:55.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3231705640 1 -- 172.21.15.133:0/1763529164 learned_addr learned my addr 172.21.15.133:0/1763529164 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:55.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3223fff640 1 --2- 172.21.15.133:0/1763529164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322c0d0770 0x7f322c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3230f04640 1 -- 172.21.15.133:0/1763529164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322c0d0770 msgr2=0x7f322c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3230f04640 1 --2- 172.21.15.133:0/1763529164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322c0d0770 0x7f322c0d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3230f04640 1 -- 172.21.15.133:0/1763529164 >> v1:172.21.15.133:6789/0 conn(0x7f322c1614c0 legacy=0x7f322c161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f32237fe640 1 -- 172.21.15.133:0/1763529164 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487623539 0 0) 0x7f322c0ceb20 con 0x7f322c1614c0 2024-04-25T10:14:55.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.931+0000 7f3230f04640 1 -- 172.21.15.133:0/1763529164 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0b97f0 con 0x7f322c15db00 2024-04-25T10:14:55.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3230f04640 1 --2- 172.21.15.133:0/1763529164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322c15db00 0x7f322c0b2990 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f32280097c0 tx=0x7f322801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d5c030a7266e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f32237fe640 1 -- 172.21.15.133:0/1763529164 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280a7070 con 0x7f322c15db00 2024-04-25T10:14:55.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f32237fe640 1 -- 172.21.15.133:0/1763529164 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32280b0070 con 0x7f322c15db00 2024-04-25T10:14:55.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f32237fe640 1 -- 172.21.15.133:0/1763529164 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280ab890 con 0x7f322c15db00 2024-04-25T10:14:55.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 -- 172.21.15.133:0/1763529164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322c15db00 msgr2=0x7f322c0b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 --2- 172.21.15.133:0/1763529164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322c15db00 0x7f322c0b2990 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f32280097c0 tx=0x7f322801c470 comp rx=0 tx=0).stop 2024-04-25T10:14:55.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 -- 172.21.15.133:0/1763529164 shutdown_connections 2024-04-25T10:14:55.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 --2- 172.21.15.133:0/1763529164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322c0d0770 0x7f322c0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 --2- 172.21.15.133:0/1763529164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322c15db00 0x7f322c0b2990 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 -- 172.21.15.133:0/1763529164 >> 172.21.15.133:0/1763529164 conn(0x7f322c0727c0 msgr2=0x7f322c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 -- 172.21.15.133:0/1763529164 shutdown_connections 2024-04-25T10:14:55.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 -- 172.21.15.133:0/1763529164 wait complete. 2024-04-25T10:14:55.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 Processor -- start 2024-04-25T10:14:55.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 -- start start 2024-04-25T10:14:55.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322c0d0770 0x7f322c0b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322c1614c0 0x7f322c0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322c1534a0 0x7f322c0c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b7430 con 0x7f322c1614c0 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b82c0 con 0x7f322c1534a0 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3233168640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0d18a0 con 0x7f322c0d0770 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3223fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322c1614c0 0x7f322c0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3223fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322c1614c0 0x7f322c0b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44072/0 (socket says 172.21.15.133:44072) 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3223fff640 1 -- 172.21.15.133:0/3454236846 learned_addr learned my addr 172.21.15.133:0/3454236846 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3230f04640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322c0d0770 0x7f322c0b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3223fff640 1 -- 172.21.15.133:0/3454236846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322c0d0770 msgr2=0x7f322c0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3223fff640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322c0d0770 0x7f322c0b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3231705640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322c1534a0 0x7f322c0c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3223fff640 1 -- 172.21.15.133:0/3454236846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322c1534a0 msgr2=0x7f322c0c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3223fff640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322c1534a0 0x7f322c0c7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3223fff640 1 -- 172.21.15.133:0/3454236846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c15eb60 con 0x7f322c1614c0 2024-04-25T10:14:55.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3230f04640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322c0d0770 0x7f322c0b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f3223fff640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322c1614c0 0x7f322c0b2eb0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f321c00d5f0 tx=0x7f321c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.935+0000 7f32217fa640 1 -- 172.21.15.133:0/3454236846 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0125f0 con 0x7f322c1614c0 2024-04-25T10:14:55.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.939+0000 7f32217fa640 1 -- 172.21.15.133:0/3454236846 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c019070 con 0x7f322c1614c0 2024-04-25T10:14:55.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.939+0000 7f3233168640 1 -- 172.21.15.133:0/3454236846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c1557d0 con 0x7f322c1614c0 2024-04-25T10:14:55.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.939+0000 7f32217fa640 1 -- 172.21.15.133:0/3454236846 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0129b0 con 0x7f322c1614c0 2024-04-25T10:14:55.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.939+0000 7f3233168640 1 -- 172.21.15.133:0/3454236846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c155d90 con 0x7f322c1614c0 2024-04-25T10:14:55.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.939+0000 7f3233168640 1 -- 172.21.15.133:0/3454236846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c068ef0 con 0x7f322c1614c0 2024-04-25T10:14:55.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.939+0000 7f32217fa640 1 -- 172.21.15.133:0/3454236846 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f321c012b50 con 0x7f322c1614c0 2024-04-25T10:14:55.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.939+0000 7f32217fa640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3218066f60 0x7f3218069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.939+0000 7f32217fa640 1 -- 172.21.15.133:0/3454236846 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f321c085510 con 0x7f322c1614c0 2024-04-25T10:14:55.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.939+0000 7f3230f04640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3218066f60 0x7f3218069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.939+0000 7f3230f04640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3218066f60 0x7f3218069420 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f32280af000 tx=0x7f3228008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:55.943+0000 7f32217fa640 1 -- 172.21.15.133:0/3454236846 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f322c15eb60 con 0x7f322c1614c0 2024-04-25T10:14:56.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.103+0000 7f3233168640 1 -- 172.21.15.133:0/3454236846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f322c155460 con 0x7f322c1614c0 2024-04-25T10:14:56.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f32217fa640 1 -- 172.21.15.133:0/3454236846 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f321c0563f0 con 0x7f322c1614c0 2024-04-25T10:14:56.109 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:56.110 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":145,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:34.721848+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:26.340459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:26.996481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.28","mappings":[{"from":3,"to":0}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:56.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 -- 172.21.15.133:0/3454236846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3218066f60 msgr2=0x7f3218069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3218066f60 0x7f3218069420 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f32280af000 tx=0x7f3228008040 comp rx=0 tx=0).stop 2024-04-25T10:14:56.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 -- 172.21.15.133:0/3454236846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322c1614c0 msgr2=0x7f322c0b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322c1614c0 0x7f322c0b2eb0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f321c00d5f0 tx=0x7f321c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 -- 172.21.15.133:0/3454236846 shutdown_connections 2024-04-25T10:14:56.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f322c1534a0 0x7f322c0c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3218066f60 0x7f3218069420 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f322c1614c0 0x7f322c0b2eb0 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 --2- 172.21.15.133:0/3454236846 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f322c0d0770 0x7f322c0b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 -- 172.21.15.133:0/3454236846 >> 172.21.15.133:0/3454236846 conn(0x7f322c0727c0 msgr2=0x7f322c0b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 -- 172.21.15.133:0/3454236846 shutdown_connections 2024-04-25T10:14:56.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.107+0000 7f3233168640 1 -- 172.21.15.133:0/3454236846 wait complete. 2024-04-25T10:14:56.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:26.340459+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:26.996481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:56.129 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:56.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.247+0000 7fa698675640 1 Processor -- start 2024-04-25T10:14:56.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.247+0000 7fa698675640 1 -- start start 2024-04-25T10:14:56.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa698675640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa690145170 0x7fa69006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa698675640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa69015ced0 0x7fa69015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa698675640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6900ce2e0 con 0x7fa69006e4d0 2024-04-25T10:14:56.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa698675640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69006db00 con 0x7fa69015ced0 2024-04-25T10:14:56.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa698675640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa690145550 con 0x7fa690145170 2024-04-25T10:14:56.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa696c12640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa69015ced0 0x7fa69015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa696c12640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa69015ced0 0x7fa69015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47260/0 (socket says 172.21.15.133:47260) 2024-04-25T10:14:56.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa696c12640 1 -- 172.21.15.133:0/1800624246 learned_addr learned my addr 172.21.15.133:0/1800624246 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:56.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa695c10640 1 --1- 172.21.15.133:0/1800624246 >> v1:172.21.15.133:6789/0 conn(0x7fa69006e4d0 0x7fa6900cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36088/0 (socket says 172.21.15.133:36088) 2024-04-25T10:14:56.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa696c12640 1 -- 172.21.15.133:0/1800624246 >> v2:172.21.15.162:3300/0 conn(0x7fa690145170 msgr2=0x7fa69006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa696411640 1 --2- 172.21.15.133:0/1800624246 >> v2:172.21.15.162:3300/0 conn(0x7fa690145170 0x7fa69006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa696c12640 1 --2- 172.21.15.133:0/1800624246 >> v2:172.21.15.162:3300/0 conn(0x7fa690145170 0x7fa69006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa696c12640 1 -- 172.21.15.133:0/1800624246 >> v1:172.21.15.133:6789/0 conn(0x7fa69006e4d0 legacy=0x7fa6900cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa69540f640 1 -- 172.21.15.133:0/1800624246 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760070819 0 0) 0x7fa6900ce2e0 con 0x7fa69006e4d0 2024-04-25T10:14:56.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa696c12640 1 -- 172.21.15.133:0/1800624246 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69006d4b0 con 0x7fa69015ced0 2024-04-25T10:14:56.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa696411640 1 --2- 172.21.15.133:0/1800624246 >> v2:172.21.15.162:3300/0 conn(0x7fa690145170 0x7fa69006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:56.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa696c12640 1 --2- 172.21.15.133:0/1800624246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa69015ced0 0x7fa69015d320 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fa69006b790 tx=0x7fa68801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a436f4db9748a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa69540f640 1 -- 172.21.15.133:0/1800624246 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6880a56c0 con 0x7fa69015ced0 2024-04-25T10:14:56.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa69540f640 1 -- 172.21.15.133:0/1800624246 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6880a5860 con 0x7fa69015ced0 2024-04-25T10:14:56.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.251+0000 7fa69540f640 1 -- 172.21.15.133:0/1800624246 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6880a5c30 con 0x7fa69015ced0 2024-04-25T10:14:56.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 -- 172.21.15.133:0/1800624246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa69015ced0 msgr2=0x7fa69015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 --2- 172.21.15.133:0/1800624246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa69015ced0 0x7fa69015d320 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fa69006b790 tx=0x7fa68801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 -- 172.21.15.133:0/1800624246 shutdown_connections 2024-04-25T10:14:56.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 --2- 172.21.15.133:0/1800624246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa69015ced0 0x7fa69015d320 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 --2- 172.21.15.133:0/1800624246 >> v2:172.21.15.162:3300/0 conn(0x7fa690145170 0x7fa69006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 -- 172.21.15.133:0/1800624246 >> 172.21.15.133:0/1800624246 conn(0x7fa690066f70 msgr2=0x7fa69006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 -- 172.21.15.133:0/1800624246 shutdown_connections 2024-04-25T10:14:56.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 -- 172.21.15.133:0/1800624246 wait complete. 2024-04-25T10:14:56.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 Processor -- start 2024-04-25T10:14:56.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 -- start start 2024-04-25T10:14:56.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa69006e4d0 0x7fa69013e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa690145170 0x7fa6901553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa69015ced0 0x7fa6901558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa690168610 con 0x7fa69006e4d0 2024-04-25T10:14:56.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa690168490 con 0x7fa69015ced0 2024-04-25T10:14:56.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa690168790 con 0x7fa690145170 2024-04-25T10:14:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa696411640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa69006e4d0 0x7fa69013e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa696411640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa69006e4d0 0x7fa69013e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47276/0 (socket says 172.21.15.133:47276) 2024-04-25T10:14:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa696411640 1 -- 172.21.15.133:0/3351123500 learned_addr learned my addr 172.21.15.133:0/3351123500 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa696c12640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa69015ced0 0x7fa6901558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa695c10640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa690145170 0x7fa6901553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa696411640 1 -- 172.21.15.133:0/3351123500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa690145170 msgr2=0x7fa6901553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa696411640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa690145170 0x7fa6901553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa696411640 1 -- 172.21.15.133:0/3351123500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa69015ced0 msgr2=0x7fa6901558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa696411640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa69015ced0 0x7fa6901558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa696411640 1 -- 172.21.15.133:0/3351123500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69013f310 con 0x7fa69006e4d0 2024-04-25T10:14:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa696411640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa69006e4d0 0x7fa69013e8e0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00c7f0 tx=0x7fa68c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa686ffd640 1 -- 172.21.15.133:0/3351123500 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c015070 con 0x7fa69006e4d0 2024-04-25T10:14:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa686ffd640 1 -- 172.21.15.133:0/3351123500 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c007820 con 0x7fa69006e4d0 2024-04-25T10:14:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 -- 172.21.15.133:0/3351123500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69013f5f0 con 0x7fa69006e4d0 2024-04-25T10:14:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa686ffd640 1 -- 172.21.15.133:0/3351123500 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c007c10 con 0x7fa69006e4d0 2024-04-25T10:14:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.255+0000 7fa698675640 1 -- 172.21.15.133:0/3351123500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69013fbb0 con 0x7fa69006e4d0 2024-04-25T10:14:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.259+0000 7fa698675640 1 -- 172.21.15.133:0/3351123500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa690068e80 con 0x7fa69006e4d0 2024-04-25T10:14:56.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.259+0000 7fa686ffd640 1 -- 172.21.15.133:0/3351123500 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa68c018070 con 0x7fa69006e4d0 2024-04-25T10:14:56.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.259+0000 7fa686ffd640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.259+0000 7fa686ffd640 1 -- 172.21.15.133:0/3351123500 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa68c0852d0 con 0x7fa69006e4d0 2024-04-25T10:14:56.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.259+0000 7fa695c10640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.259+0000 7fa695c10640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa674066f60 0x7fa674069420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa690155160 tx=0x7fa680007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.263+0000 7fa686ffd640 1 -- 172.21.15.133:0/3351123500 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa68c0561b0 con 0x7fa69006e4d0 2024-04-25T10:14:56.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.415+0000 7fa698675640 1 -- 172.21.15.133:0/3351123500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6900b3220 con 0x7fa69006e4d0 2024-04-25T10:14:56.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.415+0000 7fa686ffd640 1 -- 172.21.15.133:0/3351123500 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa68c055ab0 con 0x7fa69006e4d0 2024-04-25T10:14:56.420 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:14:56.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 -- 172.21.15.133:0/3351123500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa674066f60 msgr2=0x7fa674069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa674066f60 0x7fa674069420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa690155160 tx=0x7fa680007c10 comp rx=0 tx=0).stop 2024-04-25T10:14:56.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 -- 172.21.15.133:0/3351123500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa69006e4d0 msgr2=0x7fa69013e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa69006e4d0 0x7fa69013e8e0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00c7f0 tx=0x7fa68c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 -- 172.21.15.133:0/3351123500 shutdown_connections 2024-04-25T10:14:56.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa69015ced0 0x7fa6901558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa690145170 0x7fa6901553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 --2- 172.21.15.133:0/3351123500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa69006e4d0 0x7fa69013e8e0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 -- 172.21.15.133:0/3351123500 >> 172.21.15.133:0/3351123500 conn(0x7fa690066f70 msgr2=0x7fa69006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 -- 172.21.15.133:0/3351123500 shutdown_connections 2024-04-25T10:14:56.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.419+0000 7fa698675640 1 -- 172.21.15.133:0/3351123500 wait complete. 2024-04-25T10:14:56.438 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:56.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0798102640 1 Processor -- start 2024-04-25T10:14:56.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0798102640 1 -- start start 2024-04-25T10:14:56.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0798102640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0790145170 0x7f079006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0798102640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f079006e4d0 0x7f07900cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0798102640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07900ce2e0 con 0x7f07900cdd60 2024-04-25T10:14:56.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0798102640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079006db00 con 0x7f079006e4d0 2024-04-25T10:14:56.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0798102640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0790145550 con 0x7f0790145170 2024-04-25T10:14:56.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f079669f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f07900cdd60 0x7f079015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36094/0 (socket says 172.21.15.133:36094) 2024-04-25T10:14:56.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f079669f640 1 -- 172.21.15.133:0/2714471686 learned_addr learned my addr 172.21.15.133:0/2714471686 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:56.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0795e9e640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0790145170 0x7f079006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f079569d640 1 --2- 172.21.15.133:0/2714471686 >> v2:172.21.15.162:3300/0 conn(0x7f079006e4d0 0x7f07900cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157979581 0 0) 0x7f07900ce2e0 con 0x7f07900cdd60 2024-04-25T10:14:56.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f077c003530 con 0x7f07900cdd60 2024-04-25T10:14:56.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2044177753 0 0) 0x7f077c003530 con 0x7f07900cdd60 2024-04-25T10:14:56.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0790145170 msgr2=0x7f079006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0790145170 0x7f079006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 >> v2:172.21.15.162:3300/0 conn(0x7f079006e4d0 msgr2=0x7f07900cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 --2- 172.21.15.133:0/2714471686 >> v2:172.21.15.162:3300/0 conn(0x7f079006e4d0 0x7f07900cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079006d4b0 con 0x7f07900cdd60 2024-04-25T10:14:56.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0788001a00 con 0x7f07900cdd60 2024-04-25T10:14:56.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 >> v1:172.21.15.133:6789/0 conn(0x7f07900cdd60 legacy=0x7f079015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0790145170 0x7f077c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 0x7f077c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f077c004d50 0x7f077c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079006db00 con 0x7f077c003f40 2024-04-25T10:14:56.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790145550 con 0x7f077c004d50 2024-04-25T10:14:56.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c00b9d0 con 0x7f0790145170 2024-04-25T10:14:56.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f079569d640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 0x7f077c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f0795e9e640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0790145170 0x7f077c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f079669f640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f077c004d50 0x7f077c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f079569d640 1 -- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0790145170 msgr2=0x7f077c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f079569d640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0790145170 0x7f077c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f079569d640 1 -- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f077c004d50 msgr2=0x7f077c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f079569d640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f077c004d50 0x7f077c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f079569d640 1 -- 172.21.15.133:0/2714471686 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c003530 con 0x7f077c003f40 2024-04-25T10:14:56.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.555+0000 7f079669f640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f077c004d50 0x7f077c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:56.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f079569d640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 0x7f077c004810 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f0780009ab0 tx=0x7f078001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7d20dc21ffcaf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07800b2070 con 0x7f077c003f40 2024-04-25T10:14:56.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07800a5680 con 0x7f077c003f40 2024-04-25T10:14:56.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0794e9c640 1 -- 172.21.15.133:0/2714471686 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07800096b0 con 0x7f077c003f40 2024-04-25T10:14:56.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 -- 172.21.15.133:0/2714471686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 msgr2=0x7f077c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 --2- 172.21.15.133:0/2714471686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 0x7f077c004810 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f0780009ab0 tx=0x7f078001ca30 comp rx=0 tx=0).stop 2024-04-25T10:14:56.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f079669f640 1 -- 172.21.15.133:0/2714471686 reap_dead start 2024-04-25T10:14:56.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 -- 172.21.15.133:0/2714471686 shutdown_connections 2024-04-25T10:14:56.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 -- 172.21.15.133:0/2714471686 >> 172.21.15.133:0/2714471686 conn(0x7f0790066f70 msgr2=0x7f079006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 -- 172.21.15.133:0/2714471686 shutdown_connections 2024-04-25T10:14:56.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 -- 172.21.15.133:0/2714471686 wait complete. 2024-04-25T10:14:56.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 Processor -- start 2024-04-25T10:14:56.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 -- start start 2024-04-25T10:14:56.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 0x7f0790139000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07901e8720 0x7f07901e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f079013e020 0x7f079013e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079015d5e0 con 0x7f077c003f40 2024-04-25T10:14:56.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079015cb70 con 0x7f07901e8720 2024-04-25T10:14:56.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0798102640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079015d760 con 0x7f079013e020 2024-04-25T10:14:56.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0795e9e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 0x7f0790139000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0795e9e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 0x7f0790139000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47282/0 (socket says 172.21.15.133:47282) 2024-04-25T10:14:56.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0795e9e640 1 -- 172.21.15.133:0/2911328835 learned_addr learned my addr 172.21.15.133:0/2911328835 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:56.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f079569d640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07901e8720 0x7f07901e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f079669f640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f079013e020 0x7f079013e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0795e9e640 1 -- 172.21.15.133:0/2911328835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f079013e020 msgr2=0x7f079013e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0795e9e640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f079013e020 0x7f079013e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0795e9e640 1 -- 172.21.15.133:0/2911328835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07901e8720 msgr2=0x7f07901e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0795e9e640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07901e8720 0x7f07901e8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0795e9e640 1 -- 172.21.15.133:0/2911328835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07901e7680 con 0x7f077c003f40 2024-04-25T10:14:56.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.559+0000 7f0795e9e640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 0x7f0790139000 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f078c0115e0 tx=0x7f078c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.563+0000 7f07867fc640 1 -- 172.21.15.133:0/2911328835 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00f8f0 con 0x7f077c003f40 2024-04-25T10:14:56.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.563+0000 7f07867fc640 1 -- 172.21.15.133:0/2911328835 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c00fa90 con 0x7f077c003f40 2024-04-25T10:14:56.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.563+0000 7f0798102640 1 -- 172.21.15.133:0/2911328835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07901e7960 con 0x7f077c003f40 2024-04-25T10:14:56.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.563+0000 7f07867fc640 1 -- 172.21.15.133:0/2911328835 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00fe40 con 0x7f077c003f40 2024-04-25T10:14:56.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.563+0000 7f0798102640 1 -- 172.21.15.133:0/2911328835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07901e7f20 con 0x7f077c003f40 2024-04-25T10:14:56.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.563+0000 7f0798102640 1 -- 172.21.15.133:0/2911328835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0790068e80 con 0x7f077c003f40 2024-04-25T10:14:56.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.563+0000 7f07867fc640 1 -- 172.21.15.133:0/2911328835 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f078c015460 con 0x7f077c003f40 2024-04-25T10:14:56.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.563+0000 7f07867fc640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0774066f90 0x7f0774069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.563+0000 7f07867fc640 1 -- 172.21.15.133:0/2911328835 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f078c085ad0 con 0x7f077c003f40 2024-04-25T10:14:56.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.563+0000 7f079569d640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0774066f90 0x7f0774069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.563+0000 7f079569d640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0774066f90 0x7f0774069450 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f07800af000 tx=0x7f07800046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.567+0000 7f07867fc640 1 -- 172.21.15.133:0/2911328835 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f078c019030 con 0x7f077c003f40 2024-04-25T10:14:56.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 -- 172.21.15.133:0/2911328835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f079015ebb0 con 0x7f077c003f40 2024-04-25T10:14:56.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f07867fc640 1 -- 172.21.15.133:0/2911328835 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7f078c056920 con 0x7f077c003f40 2024-04-25T10:14:56.717 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 -- 172.21.15.133:0/2911328835 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0774066f90 msgr2=0x7f0774069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0774066f90 0x7f0774069450 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f07800af000 tx=0x7f07800046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 -- 172.21.15.133:0/2911328835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 msgr2=0x7f0790139000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 0x7f0790139000 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f078c0115e0 tx=0x7f078c011aa0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 -- 172.21.15.133:0/2911328835 shutdown_connections 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f079013e020 0x7f079013e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07901e8720 0x7f07901e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0774066f90 0x7f0774069450 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 --2- 172.21.15.133:0/2911328835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077c003f40 0x7f0790139000 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 -- 172.21.15.133:0/2911328835 >> 172.21.15.133:0/2911328835 conn(0x7f0790066f70 msgr2=0x7f0790144490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 -- 172.21.15.133:0/2911328835 shutdown_connections 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.715+0000 7f0798102640 1 -- 172.21.15.133:0/2911328835 wait complete. 2024-04-25T10:14:56.734 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:56.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6de7af640 1 Processor -- start 2024-04-25T10:14:56.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6de7af640 1 -- start start 2024-04-25T10:14:56.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6de7af640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc6d80d02d0 0x7fc6d80b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6de7af640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6d80b2cf0 0x7fc6d80cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6de7af640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d80cc530 con 0x7fc6d815b850 2024-04-25T10:14:56.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6de7af640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d80b3160 con 0x7fc6d80d02d0 2024-04-25T10:14:56.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6de7af640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d80cd140 con 0x7fc6d80b2cf0 2024-04-25T10:14:56.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6d7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc6d815b850 0x7fc6d80cfbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36104/0 (socket says 172.21.15.133:36104) 2024-04-25T10:14:56.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6d77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc6d80d02d0 0x7fc6d80b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6d7fff640 1 -- 172.21.15.133:0/3759811489 learned_addr learned my addr 172.21.15.133:0/3759811489 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:56.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6d77fe640 1 -- 172.21.15.133:0/3759811489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6d80b2cf0 msgr2=0x7fc6d80cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6dcd4c640 1 --2- 172.21.15.133:0/3759811489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6d80b2cf0 0x7fc6d80cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6d77fe640 1 --2- 172.21.15.133:0/3759811489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6d80b2cf0 0x7fc6d80cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6d6ffd640 1 -- 172.21.15.133:0/3759811489 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908504432 0 0) 0x7fc6d80cc530 con 0x7fc6d815b850 2024-04-25T10:14:56.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6d77fe640 1 -- 172.21.15.133:0/3759811489 >> v1:172.21.15.133:6789/0 conn(0x7fc6d815b850 legacy=0x7fc6d80cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6d77fe640 1 -- 172.21.15.133:0/3759811489 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80cc810 con 0x7fc6d80d02d0 2024-04-25T10:14:56.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.855+0000 7fc6dcd4c640 1 --2- 172.21.15.133:0/3759811489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6d80b2cf0 0x7fc6d80cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:56.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6d77fe640 1 --2- 172.21.15.133:0/3759811489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d80d02d0 0x7fc6d80b27b0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009d90 tx=0x7fc6cc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bea37156be65830 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6d6ffd640 1 -- 172.21.15.133:0/3759811489 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0b2070 con 0x7fc6d80d02d0 2024-04-25T10:14:56.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6d6ffd640 1 -- 172.21.15.133:0/3759811489 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc0a5a00 con 0x7fc6d80d02d0 2024-04-25T10:14:56.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6d6ffd640 1 -- 172.21.15.133:0/3759811489 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0a5dd0 con 0x7fc6d80d02d0 2024-04-25T10:14:56.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 -- 172.21.15.133:0/3759811489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d80d02d0 msgr2=0x7fc6d80b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 --2- 172.21.15.133:0/3759811489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d80d02d0 0x7fc6d80b27b0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009d90 tx=0x7fc6cc01c940 comp rx=0 tx=0).stop 2024-04-25T10:14:56.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 -- 172.21.15.133:0/3759811489 shutdown_connections 2024-04-25T10:14:56.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 --2- 172.21.15.133:0/3759811489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6d80b2cf0 0x7fc6d80cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 --2- 172.21.15.133:0/3759811489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d80d02d0 0x7fc6d80b27b0 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 -- 172.21.15.133:0/3759811489 >> 172.21.15.133:0/3759811489 conn(0x7fc6d809ab90 msgr2=0x7fc6d80d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 -- 172.21.15.133:0/3759811489 shutdown_connections 2024-04-25T10:14:56.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 -- 172.21.15.133:0/3759811489 wait complete. 2024-04-25T10:14:56.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 Processor -- start 2024-04-25T10:14:56.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 -- start start 2024-04-25T10:14:56.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6d80b2cf0 0x7fc6d8153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6d80d02d0 0x7fc6d8149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d815b850 0x7fc6d80c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d815bdb0 con 0x7fc6d815b850 2024-04-25T10:14:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d815bc30 con 0x7fc6d80b2cf0 2024-04-25T10:14:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d80ce320 con 0x7fc6d80d02d0 2024-04-25T10:14:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6dcd4c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d815b850 0x7fc6d80c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6dcd4c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d815b850 0x7fc6d80c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47300/0 (socket says 172.21.15.133:47300) 2024-04-25T10:14:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6dcd4c640 1 -- 172.21.15.133:0/3566516515 learned_addr learned my addr 172.21.15.133:0/3566516515 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6d7fff640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6d80b2cf0 0x7fc6d8153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6dcd4c640 1 -- 172.21.15.133:0/3566516515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6d80d02d0 msgr2=0x7fc6d8149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6d77fe640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6d80d02d0 0x7fc6d8149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6dcd4c640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6d80d02d0 0x7fc6d8149d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6dcd4c640 1 -- 172.21.15.133:0/3566516515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6d80b2cf0 msgr2=0x7fc6d8153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6dcd4c640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6d80b2cf0 0x7fc6d8153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6dcd4c640 1 -- 172.21.15.133:0/3566516515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d81537d0 con 0x7fc6d815b850 2024-04-25T10:14:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6dcd4c640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d815b850 0x7fc6d80c5e30 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c820 tx=0x7fc6d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6d4ff9640 1 -- 172.21.15.133:0/3566516515 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0015070 con 0x7fc6d815b850 2024-04-25T10:14:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6d4ff9640 1 -- 172.21.15.133:0/3566516515 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d0007a70 con 0x7fc6d815b850 2024-04-25T10:14:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6d4ff9640 1 -- 172.21.15.133:0/3566516515 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0007e60 con 0x7fc6d815b850 2024-04-25T10:14:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 -- 172.21.15.133:0/3566516515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d814e720 con 0x7fc6d815b850 2024-04-25T10:14:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.859+0000 7fc6de7af640 1 -- 172.21.15.133:0/3566516515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d814ece0 con 0x7fc6d815b850 2024-04-25T10:14:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.863+0000 7fc6de7af640 1 -- 172.21.15.133:0/3566516515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d8068e80 con 0x7fc6d815b850 2024-04-25T10:14:56.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.863+0000 7fc6d4ff9640 1 -- 172.21.15.133:0/3566516515 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6d0018070 con 0x7fc6d815b850 2024-04-25T10:14:56.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.863+0000 7fc6d4ff9640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc6c4066f60 0x7fc6c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.863+0000 7fc6d4ff9640 1 -- 172.21.15.133:0/3566516515 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc6d00850c0 con 0x7fc6d815b850 2024-04-25T10:14:56.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.863+0000 7fc6d7fff640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc6c4066f60 0x7fc6c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.863+0000 7fc6d7fff640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc6c4066f60 0x7fc6c4069420 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8007fd0 tx=0x7fc6c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:56.867+0000 7fc6d4ff9640 1 -- 172.21.15.133:0/3566516515 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc6d0055fa0 con 0x7fc6d815b850 2024-04-25T10:14:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.019+0000 7fc6de7af640 1 -- 172.21.15.133:0/3566516515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6d80d10e0 con 0x7fc6d815b850 2024-04-25T10:14:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.019+0000 7fc6d4ff9640 1 -- 172.21.15.133:0/3566516515 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6d00558a0 con 0x7fc6d815b850 2024-04-25T10:14:57.022 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:14:57.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 -- 172.21.15.133:0/3566516515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc6c4066f60 msgr2=0x7fc6c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc6c4066f60 0x7fc6c4069420 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8007fd0 tx=0x7fc6c8009040 comp rx=0 tx=0).stop 2024-04-25T10:14:57.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 -- 172.21.15.133:0/3566516515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d815b850 msgr2=0x7fc6d80c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d815b850 0x7fc6d80c5e30 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c820 tx=0x7fc6d000cce0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 -- 172.21.15.133:0/3566516515 shutdown_connections 2024-04-25T10:14:57.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc6c4066f60 0x7fc6c4069420 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d815b850 0x7fc6d80c5e30 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc6d80d02d0 0x7fc6d8149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 --2- 172.21.15.133:0/3566516515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6d80b2cf0 0x7fc6d8153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 -- 172.21.15.133:0/3566516515 >> 172.21.15.133:0/3566516515 conn(0x7fc6d809ab90 msgr2=0x7fc6d806f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 -- 172.21.15.133:0/3566516515 shutdown_connections 2024-04-25T10:14:57.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.023+0000 7fc6de7af640 1 -- 172.21.15.133:0/3566516515 wait complete. 2024-04-25T10:14:57.040 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:57.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f958392d640 1 Processor -- start 2024-04-25T10:14:57.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f958392d640 1 -- start start 2024-04-25T10:14:57.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f958392d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f957c143ea0 0x7f957c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f958392d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f957c06eac0 0x7f957c0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f958392d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957c15ac50 con 0x7f957c0b2150 2024-04-25T10:14:57.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f958392d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957c0cd180 con 0x7f957c143ea0 2024-04-25T10:14:57.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f958392d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957c15aed0 con 0x7f957c06eac0 2024-04-25T10:14:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f95816c9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f957c143ea0 0x7f957c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f95816c9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f957c143ea0 0x7f957c0cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47304/0 (socket says 172.21.15.133:47304) 2024-04-25T10:14:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f95816c9640 1 -- 172.21.15.133:0/1897284121 learned_addr learned my addr 172.21.15.133:0/1897284121 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f9581eca640 1 --2- 172.21.15.133:0/1897284121 >> v2:172.21.15.162:3300/0 conn(0x7f957c06eac0 0x7f957c0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f95816c9640 1 -- 172.21.15.133:0/1897284121 >> v2:172.21.15.162:3300/0 conn(0x7f957c06eac0 msgr2=0x7f957c0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f95816c9640 1 --2- 172.21.15.133:0/1897284121 >> v2:172.21.15.162:3300/0 conn(0x7f957c06eac0 0x7f957c0b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f95816c9640 1 -- 172.21.15.133:0/1897284121 >> v1:172.21.15.162:6789/0 conn(0x7f957c0b2150 legacy=0x7f957c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f95816c9640 1 -- 172.21.15.133:0/1897284121 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c15b1b0 con 0x7f957c143ea0 2024-04-25T10:14:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.155+0000 7f9581eca640 1 --2- 172.21.15.133:0/1897284121 >> v2:172.21.15.162:3300/0 conn(0x7f957c06eac0 0x7f957c0b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f95816c9640 1 --2- 172.21.15.133:0/1897284121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f957c143ea0 0x7f957c0cc810 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f957c0cd820 tx=0x7f957801e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d93804887488e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f9573fff640 1 -- 172.21.15.133:0/1897284121 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780b4070 con 0x7f957c143ea0 2024-04-25T10:14:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f9573fff640 1 -- 172.21.15.133:0/1897284121 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95780a7920 con 0x7f957c143ea0 2024-04-25T10:14:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f9573fff640 1 -- 172.21.15.133:0/1897284121 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780a7cf0 con 0x7f957c143ea0 2024-04-25T10:14:57.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f958392d640 1 -- 172.21.15.133:0/1897284121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f957c143ea0 msgr2=0x7f957c0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f958392d640 1 --2- 172.21.15.133:0/1897284121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f957c143ea0 0x7f957c0cc810 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f957c0cd820 tx=0x7f957801e940 comp rx=0 tx=0).stop 2024-04-25T10:14:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f958392d640 1 -- 172.21.15.133:0/1897284121 shutdown_connections 2024-04-25T10:14:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f958392d640 1 --2- 172.21.15.133:0/1897284121 >> v2:172.21.15.162:3300/0 conn(0x7f957c06eac0 0x7f957c0b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f958392d640 1 --2- 172.21.15.133:0/1897284121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f957c143ea0 0x7f957c0cc810 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f958392d640 1 -- 172.21.15.133:0/1897284121 >> 172.21.15.133:0/1897284121 conn(0x7f957c066f70 msgr2=0x7f957c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f958392d640 1 -- 172.21.15.133:0/1897284121 shutdown_connections 2024-04-25T10:14:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.159+0000 7f958392d640 1 -- 172.21.15.133:0/1897284121 wait complete. 2024-04-25T10:14:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f958392d640 1 Processor -- start 2024-04-25T10:14:57.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f958392d640 1 -- start start 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f958392d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c06eac0 0x7f957c1f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f958392d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f957c0b2150 0x7f957c155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f958392d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f957c143ea0 0x7f957c163780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f958392d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c06d420 con 0x7f957c0b2150 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f958392d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c06d2a0 con 0x7f957c06eac0 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f958392d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957c06d5a0 con 0x7f957c143ea0 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9580ec8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f957c0b2150 0x7f957c155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9580ec8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f957c0b2150 0x7f957c155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47318/0 (socket says 172.21.15.133:47318) 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9580ec8640 1 -- 172.21.15.133:0/69824935 learned_addr learned my addr 172.21.15.133:0/69824935 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f95816c9640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c06eac0 0x7f957c1f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9581eca640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f957c143ea0 0x7f957c163780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9580ec8640 1 -- 172.21.15.133:0/69824935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f957c143ea0 msgr2=0x7f957c163780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9580ec8640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f957c143ea0 0x7f957c163780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9580ec8640 1 -- 172.21.15.133:0/69824935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c06eac0 msgr2=0x7f957c1f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9580ec8640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c06eac0 0x7f957c1f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9580ec8640 1 -- 172.21.15.133:0/69824935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c157dd0 con 0x7f957c0b2150 2024-04-25T10:14:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9580ec8640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f957c0b2150 0x7f957c155270 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f956c00c430 tx=0x7f956c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9571ffb640 1 -- 172.21.15.133:0/69824935 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c007020 con 0x7f957c0b2150 2024-04-25T10:14:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9571ffb640 1 -- 172.21.15.133:0/69824935 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c0071c0 con 0x7f957c0b2150 2024-04-25T10:14:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f958392d640 1 -- 172.21.15.133:0/69824935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c131df0 con 0x7f957c0b2150 2024-04-25T10:14:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9571ffb640 1 -- 172.21.15.133:0/69824935 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0075b0 con 0x7f957c0b2150 2024-04-25T10:14:57.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f958392d640 1 -- 172.21.15.133:0/69824935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957c1557b0 con 0x7f957c0b2150 2024-04-25T10:14:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f958392d640 1 -- 172.21.15.133:0/69824935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957c068e80 con 0x7f957c0b2150 2024-04-25T10:14:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.163+0000 7f9571ffb640 1 -- 172.21.15.133:0/69824935 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f956c019020 con 0x7f957c0b2150 2024-04-25T10:14:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.167+0000 7f9571ffb640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9560066f90 0x7f9560069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.167+0000 7f9571ffb640 1 -- 172.21.15.133:0/69824935 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f956c084940 con 0x7f957c0b2150 2024-04-25T10:14:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.167+0000 7f95816c9640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9560066f90 0x7f9560069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.167+0000 7f95816c9640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9560066f90 0x7f9560069450 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f95780b1000 tx=0x7f9578008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.171+0000 7f9571ffb640 1 -- 172.21.15.133:0/69824935 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f956c055820 con 0x7f957c0b2150 2024-04-25T10:14:57.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.327+0000 7f958392d640 1 -- 172.21.15.133:0/69824935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f957c06c2d0 con 0x7f957c0b2150 2024-04-25T10:14:57.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f9571ffb640 1 -- 172.21.15.133:0/69824935 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) v1 ==== 80+0+0 (secure 0 0 0) 0x7f956c055120 con 0x7f957c0b2150 2024-04-25T10:14:57.332 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:14:57.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 -- 172.21.15.133:0/69824935 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9560066f90 msgr2=0x7f9560069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9560066f90 0x7f9560069450 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f95780b1000 tx=0x7f9578008040 comp rx=0 tx=0).stop 2024-04-25T10:14:57.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 -- 172.21.15.133:0/69824935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f957c0b2150 msgr2=0x7f957c155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f957c0b2150 0x7f957c155270 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f956c00c430 tx=0x7f956c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 -- 172.21.15.133:0/69824935 shutdown_connections 2024-04-25T10:14:57.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f957c143ea0 0x7f957c163780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9560066f90 0x7f9560069450 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f957c0b2150 0x7f957c155270 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 --2- 172.21.15.133:0/69824935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f957c06eac0 0x7f957c1f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 -- 172.21.15.133:0/69824935 >> 172.21.15.133:0/69824935 conn(0x7f957c066f70 msgr2=0x7f957c15e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 -- 172.21.15.133:0/69824935 shutdown_connections 2024-04-25T10:14:57.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.331+0000 7f958392d640 1 -- 172.21.15.133:0/69824935 wait complete. 2024-04-25T10:14:57.350 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:14:57.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.351 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:57.361 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[84760]: fuse finished with error 0 and tester_r 0 2024-04-25T10:14:57.434 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:14:57.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.435 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:57.590 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:14:57.590 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:57.597 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:57.597 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:14:57.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.597 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:14:57.647 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:14:57.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:57.648 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:14:57.649 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:57.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:57.654 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.654 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:14:57.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:57.700 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:14:57.701 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:14:57.701 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:57.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:57.707 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:14:57.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:57.708 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:14:57.756 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:14:57.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:57.757 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:57.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.875+0000 7fa0c9b09640 1 Processor -- start 2024-04-25T10:14:57.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.875+0000 7fa0c9b09640 1 -- start start 2024-04-25T10:14:57.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c9b09640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c40cfc70 0x7fa0c40d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c9b09640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c415bd90 con 0x7fa0c415b9b0 2024-04-25T10:14:57.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c9b09640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c409ad10 con 0x7fa0c40c5aa0 2024-04-25T10:14:57.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c9b09640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40d0610 con 0x7fa0c40cfc70 2024-04-25T10:14:57.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c37fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa0c415b9b0 0x7fa0c40cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36114/0 (socket says 172.21.15.133:36114) 2024-04-25T10:14:57.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c37fe640 1 -- 172.21.15.133:0/923773660 learned_addr learned my addr 172.21.15.133:0/923773660 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:57.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503361893 0 0) 0x7fa0c415bd90 con 0x7fa0c415b9b0 2024-04-25T10:14:57.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0baffd640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c40cfc70 0x7fa0c40d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b0003530 con 0x7fa0c415b9b0 2024-04-25T10:14:57.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3212886914 0 0) 0x7fa0b0003530 con 0x7fa0c415b9b0 2024-04-25T10:14:57.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c40cfc70 msgr2=0x7fa0c40d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c40cfc70 0x7fa0c40d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 >> v1:172.21.15.162:6789/0 conn(0x7fa0c40c5aa0 legacy=0x7fa0c40c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c409a750 con 0x7fa0c415b9b0 2024-04-25T10:14:57.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa0b4001680 con 0x7fa0c415b9b0 2024-04-25T10:14:57.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 >> v1:172.21.15.133:6789/0 conn(0x7fa0c415b9b0 legacy=0x7fa0c40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c40cfc70 0x7fa0b00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0b0003fc0 0x7fa0b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b0004d50 0x7fa0b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b000bae0 con 0x7fa0b0004d50 2024-04-25T10:14:57.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b000bc90 con 0x7fa0b0003fc0 2024-04-25T10:14:57.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b000be90 con 0x7fa0c40cfc70 2024-04-25T10:14:57.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c3fff640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b0004d50 0x7fa0b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c37fe640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c40cfc70 0x7fa0b00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0baffd640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0b0003fc0 0x7fa0b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c3fff640 1 -- 172.21.15.133:0/923773660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c40cfc70 msgr2=0x7fa0b00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c3fff640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c40cfc70 0x7fa0b00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c3fff640 1 -- 172.21.15.133:0/923773660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0b0003fc0 msgr2=0x7fa0b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c3fff640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0b0003fc0 0x7fa0b0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.879+0000 7fa0c3fff640 1 -- 172.21.15.133:0/923773660 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b0003530 con 0x7fa0b0004d50 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c3fff640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b0004d50 0x7fa0b00051d0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc001e70 tx=0x7fa0bc01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831baabc8e602204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc001a90 con 0x7fa0b0004d50 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc003b80 con 0x7fa0b0004d50 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c2ffd640 1 -- 172.21.15.133:0/923773660 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc005080 con 0x7fa0b0004d50 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 -- 172.21.15.133:0/923773660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b0004d50 msgr2=0x7fa0b00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 --2- 172.21.15.133:0/923773660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0b0004d50 0x7fa0b00051d0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc001e70 tx=0x7fa0bc01c780 comp rx=0 tx=0).stop 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c3fff640 1 -- 172.21.15.133:0/923773660 reap_dead start 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 -- 172.21.15.133:0/923773660 shutdown_connections 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 -- 172.21.15.133:0/923773660 >> 172.21.15.133:0/923773660 conn(0x7fa0c40b31a0 msgr2=0x7fa0c40b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 -- 172.21.15.133:0/923773660 shutdown_connections 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 -- 172.21.15.133:0/923773660 wait complete. 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 Processor -- start 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 -- start start 2024-04-25T10:14:57.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0b0004d50 0x7fa0c40b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0c414c420 0x7fa0c40b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c414bb70 0x7fa0c406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40c52a0 con 0x7fa0c414c420 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40c5610 con 0x7fa0b0004d50 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c9b09640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40cf990 con 0x7fa0c414bb70 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0baffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0c414c420 0x7fa0c40b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0baffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0c414c420 0x7fa0c40b6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47330/0 (socket says 172.21.15.133:47330) 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0baffd640 1 -- 172.21.15.133:0/3576645285 learned_addr learned my addr 172.21.15.133:0/3576645285 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c37fe640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0b0004d50 0x7fa0c40b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0c3fff640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c414bb70 0x7fa0c406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0baffd640 1 -- 172.21.15.133:0/3576645285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c414bb70 msgr2=0x7fa0c406fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0baffd640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c414bb70 0x7fa0c406fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0baffd640 1 -- 172.21.15.133:0/3576645285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0b0004d50 msgr2=0x7fa0c40b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0baffd640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0b0004d50 0x7fa0c40b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.883+0000 7fa0baffd640 1 -- 172.21.15.133:0/3576645285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c414c880 con 0x7fa0c414c420 2024-04-25T10:14:57.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0baffd640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0c414c420 0x7fa0c40b6f90 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac010890 tx=0x7fa0ac010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0c0ff9640 1 -- 172.21.15.133:0/3576645285 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00f040 con 0x7fa0c414c420 2024-04-25T10:14:57.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0c0ff9640 1 -- 172.21.15.133:0/3576645285 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac00eb60 con 0x7fa0c414c420 2024-04-25T10:14:57.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0c9b09640 1 -- 172.21.15.133:0/3576645285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c414bff0 con 0x7fa0c414c420 2024-04-25T10:14:57.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0c0ff9640 1 -- 172.21.15.133:0/3576645285 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac01c400 con 0x7fa0c414c420 2024-04-25T10:14:57.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0c9b09640 1 -- 172.21.15.133:0/3576645285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40ca020 con 0x7fa0c414c420 2024-04-25T10:14:57.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0c9b09640 1 -- 172.21.15.133:0/3576645285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c4067c70 con 0x7fa0c414c420 2024-04-25T10:14:57.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0c0ff9640 1 -- 172.21.15.133:0/3576645285 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0ac01c5a0 con 0x7fa0c414c420 2024-04-25T10:14:57.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0c0ff9640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa0a4066f90 0x7fa0a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0c0ff9640 1 -- 172.21.15.133:0/3576645285 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa0ac085870 con 0x7fa0c414c420 2024-04-25T10:14:57.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0c37fe640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa0a4066f90 0x7fa0a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.887+0000 7fa0c37fe640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa0a4066f90 0x7fa0a4069450 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4001ec0 tx=0x7fa0b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:57.891+0000 7fa0c0ff9640 1 -- 172.21.15.133:0/3576645285 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0ac018030 con 0x7fa0c414c420 2024-04-25T10:14:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.091+0000 7fa0c9b09640 1 -- 172.21.15.133:0/3576645285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0c40cf3c0 con 0x7fa0c414c420 2024-04-25T10:14:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.091+0000 7fa0c0ff9640 1 -- 172.21.15.133:0/3576645285 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4966 (secure 0 0 0) 0x7fa0ac056750 con 0x7fa0c414c420 2024-04-25T10:14:58.096 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:58.096 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":93,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3448301688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3448301688},{"type":"v1","addr":"172.21.15.162:6833","nonce":3448301688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/4188704231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":4188704231},{"type":"v1","addr":"172.21.15.162:6835","nonce":4188704231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":93,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:14:28.005823+0000","modified":"2024-04-25T10:14:54.057373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2626366082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2626366082},{"type":"v1","addr":"172.21.15.133:6835","nonce":2626366082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10050":{"gid":10050,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3234096063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3234096063},{"type":"v1","addr":"172.21.15.133:6837","nonce":3234096063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:14:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 93 2024-04-25T10:14:58.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 -- 172.21.15.133:0/3576645285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa0a4066f90 msgr2=0x7fa0a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa0a4066f90 0x7fa0a4069450 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4001ec0 tx=0x7fa0b4009040 comp rx=0 tx=0).stop 2024-04-25T10:14:58.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 -- 172.21.15.133:0/3576645285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0c414c420 msgr2=0x7fa0c40b6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0c414c420 0x7fa0c40b6f90 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac010890 tx=0x7fa0ac010d50 comp rx=0 tx=0).stop 2024-04-25T10:14:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 -- 172.21.15.133:0/3576645285 shutdown_connections 2024-04-25T10:14:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa0c414bb70 0x7fa0c406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa0a4066f90 0x7fa0a4069450 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0c414c420 0x7fa0c40b6f90 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 --2- 172.21.15.133:0/3576645285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa0b0004d50 0x7fa0c40b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 -- 172.21.15.133:0/3576645285 >> 172.21.15.133:0/3576645285 conn(0x7fa0c40b31a0 msgr2=0x7fa0c415ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 -- 172.21.15.133:0/3576645285 shutdown_connections 2024-04-25T10:14:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.095+0000 7fa0c9b09640 1 -- 172.21.15.133:0/3576645285 wait complete. 2024-04-25T10:14:58.115 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:58.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.231+0000 7f7cea2be640 1 Processor -- start 2024-04-25T10:14:58.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.231+0000 7f7cea2be640 1 -- start start 2024-04-25T10:14:58.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.231+0000 7f7cea2be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce4158110 0x7f7ce40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.231+0000 7f7cea2be640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7ce406eb70 0x7f7ce415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.231+0000 7f7cea2be640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce40cbe60 con 0x7f7ce41584f0 2024-04-25T10:14:58.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.231+0000 7f7cea2be640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce40b19b0 con 0x7f7ce406eb70 2024-04-25T10:14:58.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.231+0000 7f7cea2be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce415b360 con 0x7f7ce4158110 2024-04-25T10:14:58.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.231+0000 7f7ce37fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7ce41584f0 0x7f7ce406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36118/0 (socket says 172.21.15.133:36118) 2024-04-25T10:14:58.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.231+0000 7f7ce37fe640 1 -- 172.21.15.133:0/2730949802 learned_addr learned my addr 172.21.15.133:0/2730949802 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:58.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce885b640 1 --2- 172.21.15.133:0/2730949802 >> v2:172.21.15.162:3300/0 conn(0x7f7ce406eb70 0x7f7ce415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce3fff640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce4158110 0x7f7ce40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336968398 0 0) 0x7f7ce40cbe60 con 0x7f7ce41584f0 2024-04-25T10:14:58.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ccc003530 con 0x7f7ce41584f0 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1969795984 0 0) 0x7f7ccc003530 con 0x7f7ce41584f0 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce4158110 msgr2=0x7f7ce40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce4158110 0x7f7ce40b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 >> v2:172.21.15.162:3300/0 conn(0x7f7ce406eb70 msgr2=0x7f7ce415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 --2- 172.21.15.133:0/2730949802 >> v2:172.21.15.162:3300/0 conn(0x7f7ce406eb70 0x7f7ce415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce415b640 con 0x7f7ce41584f0 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7cd8001680 con 0x7f7ce41584f0 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 >> v1:172.21.15.133:6789/0 conn(0x7f7ce41584f0 legacy=0x7f7ce406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0039c0 0x7f7ccc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce4158110 0x7f7ccc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ccc004d50 0x7f7ccc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce40b19b0 con 0x7f7ccc004d50 2024-04-25T10:14:58.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce415b360 con 0x7f7ccc0039c0 2024-04-25T10:14:58.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc00b9d0 con 0x7f7ce4158110 2024-04-25T10:14:58.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce885b640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ccc004d50 0x7f7ccc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce3fff640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0039c0 0x7f7ccc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce37fe640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce4158110 0x7f7ccc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce885b640 1 -- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce4158110 msgr2=0x7f7ccc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce885b640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce4158110 0x7f7ccc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce885b640 1 -- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0039c0 msgr2=0x7f7ccc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce885b640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0039c0 0x7f7ccc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce885b640 1 -- 172.21.15.133:0/2730949802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc003530 con 0x7f7ccc004d50 2024-04-25T10:14:58.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce3fff640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ccc0039c0 0x7f7ccc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:58.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce37fe640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce4158110 0x7f7ccc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:58.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce885b640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ccc004d50 0x7f7ccc0051d0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc009fc0 tx=0x7f7cdc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43ce1ad61181cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc0b2070 con 0x7f7ccc004d50 2024-04-25T10:14:58.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc0a5cd0 con 0x7f7ccc004d50 2024-04-25T10:14:58.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce2ffd640 1 -- 172.21.15.133:0/2730949802 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc009c10 con 0x7f7ccc004d50 2024-04-25T10:14:58.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7cea2be640 1 -- 172.21.15.133:0/2730949802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ccc004d50 msgr2=0x7f7ccc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7cea2be640 1 --2- 172.21.15.133:0/2730949802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ccc004d50 0x7f7ccc0051d0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc009fc0 tx=0x7f7cdc01ccc0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7ce885b640 1 -- 172.21.15.133:0/2730949802 reap_dead start 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7cea2be640 1 -- 172.21.15.133:0/2730949802 shutdown_connections 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7cea2be640 1 -- 172.21.15.133:0/2730949802 >> 172.21.15.133:0/2730949802 conn(0x7f7ce4067000 msgr2=0x7f7ce406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7cea2be640 1 -- 172.21.15.133:0/2730949802 shutdown_connections 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.235+0000 7f7cea2be640 1 -- 172.21.15.133:0/2730949802 wait complete. 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7cea2be640 1 Processor -- start 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7cea2be640 1 -- start start 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7cea2be640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4136180 0x7f7ce415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7cea2be640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce415e910 0x7f7ce40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7cea2be640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ce415ed90 0x7f7ce40cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7cea2be640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce40cd670 con 0x7f7ce4136180 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7cea2be640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce415bc00 con 0x7f7ce415ed90 2024-04-25T10:14:58.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7cea2be640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce40cd7f0 con 0x7f7ce415e910 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4136180 0x7f7ce415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4136180 0x7f7ce415bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47348/0 (socket says 172.21.15.133:47348) 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce3fff640 1 -- 172.21.15.133:0/559876111 learned_addr learned my addr 172.21.15.133:0/559876111 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce37fe640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce415e910 0x7f7ce40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce3fff640 1 -- 172.21.15.133:0/559876111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce415e910 msgr2=0x7f7ce40cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce3fff640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce415e910 0x7f7ce40cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce3fff640 1 -- 172.21.15.133:0/559876111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ce415ed90 msgr2=0x7f7ce40cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce3fff640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ce415ed90 0x7f7ce40cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce3fff640 1 -- 172.21.15.133:0/559876111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce41369f0 con 0x7f7ce4136180 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce3fff640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4136180 0x7f7ce415bd80 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4010900 tx=0x7f7cd4010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce0ff9640 1 -- 172.21.15.133:0/559876111 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd400f040 con 0x7f7ce4136180 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce0ff9640 1 -- 172.21.15.133:0/559876111 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd4004220 con 0x7f7ce4136180 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7cea2be640 1 -- 172.21.15.133:0/559876111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce4136c60 con 0x7f7ce4136180 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7ce0ff9640 1 -- 172.21.15.133:0/559876111 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd400e9a0 con 0x7f7ce4136180 2024-04-25T10:14:58.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7cea2be640 1 -- 172.21.15.133:0/559876111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce4137220 con 0x7f7ce4136180 2024-04-25T10:14:58.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.239+0000 7f7cea2be640 1 -- 172.21.15.133:0/559876111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce4068ef0 con 0x7f7ce4136180 2024-04-25T10:14:58.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.243+0000 7f7ce0ff9640 1 -- 172.21.15.133:0/559876111 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7cd400eb40 con 0x7f7ce4136180 2024-04-25T10:14:58.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.243+0000 7f7ce0ff9640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7cd0066f90 0x7f7cd0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.243+0000 7f7ce0ff9640 1 -- 172.21.15.133:0/559876111 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7cd40854d0 con 0x7f7ce4136180 2024-04-25T10:14:58.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.243+0000 7f7ce37fe640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7cd0066f90 0x7f7cd0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.243+0000 7f7ce37fe640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7cd0066f90 0x7f7cd0069450 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8000fc0 tx=0x7f7cd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.247+0000 7f7ce0ff9640 1 -- 172.21.15.133:0/559876111 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7cd4018030 con 0x7f7ce4136180 2024-04-25T10:14:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.391+0000 7f7cea2be640 1 -- 172.21.15.133:0/559876111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ce414ea20 con 0x7f7ce4136180 2024-04-25T10:14:58.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7ce0ff9640 1 -- 172.21.15.133:0/559876111 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f7cd40563b0 con 0x7f7ce4136180 2024-04-25T10:14:58.398 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:58.398 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":145,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:34.721848+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:26.340459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:26.996481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.28","mappings":[{"from":3,"to":0}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:58.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 -- 172.21.15.133:0/559876111 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7cd0066f90 msgr2=0x7f7cd0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7cd0066f90 0x7f7cd0069450 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8000fc0 tx=0x7f7cd8009040 comp rx=0 tx=0).stop 2024-04-25T10:14:58.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 -- 172.21.15.133:0/559876111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4136180 msgr2=0x7f7ce415bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4136180 0x7f7ce415bd80 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4010900 tx=0x7f7cd4010dc0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 -- 172.21.15.133:0/559876111 shutdown_connections 2024-04-25T10:14:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7ce415ed90 0x7f7ce40cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7ce415e910 0x7f7ce40cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7cd0066f90 0x7f7cd0069450 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 --2- 172.21.15.133:0/559876111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4136180 0x7f7ce415bd80 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 -- 172.21.15.133:0/559876111 >> 172.21.15.133:0/559876111 conn(0x7f7ce4067000 msgr2=0x7f7ce4151c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 -- 172.21.15.133:0/559876111 shutdown_connections 2024-04-25T10:14:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.395+0000 7f7cea2be640 1 -- 172.21.15.133:0/559876111 wait complete. 2024-04-25T10:14:58.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:14:26.340459+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:14:26.996481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:58.417 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:58.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa9954ac640 1 Processor -- start 2024-04-25T10:14:58.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa9954ac640 1 -- start start 2024-04-25T10:14:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa9954ac640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa99006ac90 0x7fa99015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa9954ac640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa9900cce30 0x7fa9900cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa9954ac640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9900b2f80 con 0x7fa9900b2770 2024-04-25T10:14:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa9954ac640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99015cc30 con 0x7fa9900cce30 2024-04-25T10:14:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa9954ac640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9900b2ca0 con 0x7fa99006ac90 2024-04-25T10:14:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa98f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa9900cce30 0x7fa9900cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa98f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa9900cce30 0x7fa9900cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47354/0 (socket says 172.21.15.133:47354) 2024-04-25T10:14:58.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa98f7fe640 1 -- 172.21.15.133:0/2590080655 learned_addr learned my addr 172.21.15.133:0/2590080655 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:58.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa98effd640 1 --2- 172.21.15.133:0/2590080655 >> v2:172.21.15.162:3300/0 conn(0x7fa99006ac90 0x7fa99015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa98f7fe640 1 -- 172.21.15.133:0/2590080655 >> v2:172.21.15.162:3300/0 conn(0x7fa99006ac90 msgr2=0x7fa99015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa98f7fe640 1 --2- 172.21.15.133:0/2590080655 >> v2:172.21.15.162:3300/0 conn(0x7fa99006ac90 0x7fa99015c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa98f7fe640 1 -- 172.21.15.133:0/2590080655 >> v1:172.21.15.133:6789/0 conn(0x7fa9900b2770 legacy=0x7fa99006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.539+0000 7fa98f7fe640 1 -- 172.21.15.133:0/2590080655 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9900cc350 con 0x7fa9900cce30 2024-04-25T10:14:58.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa98f7fe640 1 --2- 172.21.15.133:0/2590080655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900cce30 0x7fa9900cd260 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fa9900d0470 tx=0x7fa98801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33d8dafcf4fd084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa98e7fc640 1 -- 172.21.15.133:0/2590080655 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9880a5880 con 0x7fa9900cce30 2024-04-25T10:14:58.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa98e7fc640 1 -- 172.21.15.133:0/2590080655 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9880a5a20 con 0x7fa9900cce30 2024-04-25T10:14:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa98e7fc640 1 -- 172.21.15.133:0/2590080655 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9880a5df0 con 0x7fa9900cce30 2024-04-25T10:14:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 -- 172.21.15.133:0/2590080655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900cce30 msgr2=0x7fa9900cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 --2- 172.21.15.133:0/2590080655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900cce30 0x7fa9900cd260 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fa9900d0470 tx=0x7fa98801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 -- 172.21.15.133:0/2590080655 shutdown_connections 2024-04-25T10:14:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 --2- 172.21.15.133:0/2590080655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900cce30 0x7fa9900cd260 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 --2- 172.21.15.133:0/2590080655 >> v2:172.21.15.162:3300/0 conn(0x7fa99006ac90 0x7fa99015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 -- 172.21.15.133:0/2590080655 >> 172.21.15.133:0/2590080655 conn(0x7fa9900999d0 msgr2=0x7fa9900b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 -- 172.21.15.133:0/2590080655 shutdown_connections 2024-04-25T10:14:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 -- 172.21.15.133:0/2590080655 wait complete. 2024-04-25T10:14:58.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 Processor -- start 2024-04-25T10:14:58.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 -- start start 2024-04-25T10:14:58.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa99006ac90 0x7fa99014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9900b2770 0x7fa99014f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900cce30 0x7fa9900b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99015efa0 con 0x7fa9900cce30 2024-04-25T10:14:58.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99015ee20 con 0x7fa9900b2770 2024-04-25T10:14:58.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa9954ac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900cdbb0 con 0x7fa99006ac90 2024-04-25T10:14:58.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.543+0000 7fa98f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900cce30 0x7fa9900b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900cce30 0x7fa9900b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47356/0 (socket says 172.21.15.133:47356) 2024-04-25T10:14:58.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98f7fe640 1 -- 172.21.15.133:0/2789078257 learned_addr learned my addr 172.21.15.133:0/2789078257 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:58.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa987fff640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9900b2770 0x7fa99014f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98effd640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa99006ac90 0x7fa99014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98f7fe640 1 -- 172.21.15.133:0/2789078257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa99006ac90 msgr2=0x7fa99014fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98f7fe640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa99006ac90 0x7fa99014fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98f7fe640 1 -- 172.21.15.133:0/2789078257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9900b2770 msgr2=0x7fa99014f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98f7fe640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9900b2770 0x7fa99014f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98f7fe640 1 -- 172.21.15.133:0/2789078257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9900b5ac0 con 0x7fa9900cce30 2024-04-25T10:14:58.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98effd640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa99006ac90 0x7fa99014fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:58.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98f7fe640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900cce30 0x7fa9900b3150 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fa9880195d0 tx=0x7fa988019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa9877fe640 1 -- 172.21.15.133:0/2789078257 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9880b8670 con 0x7fa9900cce30 2024-04-25T10:14:58.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa9877fe640 1 -- 172.21.15.133:0/2789078257 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9880be070 con 0x7fa9900cce30 2024-04-25T10:14:58.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa9954ac640 1 -- 172.21.15.133:0/2789078257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9900b5d60 con 0x7fa9900cce30 2024-04-25T10:14:58.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa9877fe640 1 -- 172.21.15.133:0/2789078257 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9880b8a70 con 0x7fa9900cce30 2024-04-25T10:14:58.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa9954ac640 1 -- 172.21.15.133:0/2789078257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9900c4d40 con 0x7fa9900cce30 2024-04-25T10:14:58.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa9954ac640 1 -- 172.21.15.133:0/2789078257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa990068e80 con 0x7fa9900cce30 2024-04-25T10:14:58.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa9877fe640 1 -- 172.21.15.133:0/2789078257 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9880b8c10 con 0x7fa9900cce30 2024-04-25T10:14:58.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa9877fe640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa970066f60 0x7fa970069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa9877fe640 1 -- 172.21.15.133:0/2789078257 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa988120f50 con 0x7fa9900cce30 2024-04-25T10:14:58.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98effd640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa970066f60 0x7fa970069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.547+0000 7fa98effd640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa970066f60 0x7fa970069420 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa9900d0ab0 tx=0x7fa980008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.551+0000 7fa9877fe640 1 -- 172.21.15.133:0/2789078257 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9880165e0 con 0x7fa9900cce30 2024-04-25T10:14:58.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.707+0000 7fa9954ac640 1 -- 172.21.15.133:0/2789078257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa99006b7c0 con 0x7fa9900cce30 2024-04-25T10:14:58.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9877fe640 1 -- 172.21.15.133:0/2789078257 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9880c23c0 con 0x7fa9900cce30 2024-04-25T10:14:58.713 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:14:58.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 -- 172.21.15.133:0/2789078257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa970066f60 msgr2=0x7fa970069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa970066f60 0x7fa970069420 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa9900d0ab0 tx=0x7fa980008040 comp rx=0 tx=0).stop 2024-04-25T10:14:58.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 -- 172.21.15.133:0/2789078257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900cce30 msgr2=0x7fa9900b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900cce30 0x7fa9900b3150 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fa9880195d0 tx=0x7fa988019760 comp rx=0 tx=0).stop 2024-04-25T10:14:58.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 -- 172.21.15.133:0/2789078257 shutdown_connections 2024-04-25T10:14:58.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa970066f60 0x7fa970069420 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900cce30 0x7fa9900b3150 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa9900b2770 0x7fa99014f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 --2- 172.21.15.133:0/2789078257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa99006ac90 0x7fa99014fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 -- 172.21.15.133:0/2789078257 >> 172.21.15.133:0/2789078257 conn(0x7fa9900999d0 msgr2=0x7fa9900b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 -- 172.21.15.133:0/2789078257 shutdown_connections 2024-04-25T10:14:58.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.711+0000 7fa9954ac640 1 -- 172.21.15.133:0/2789078257 wait complete. 2024-04-25T10:14:58.732 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:58.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f72147640 1 Processor -- start 2024-04-25T10:14:58.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f72147640 1 -- start start 2024-04-25T10:14:58.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f72147640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7f6c15d5e0 0x7f7f6c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f72147640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c0ce2e0 con 0x7f7f6c0cd820 2024-04-25T10:14:58.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f72147640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c06d4b0 con 0x7f7f6c145170 2024-04-25T10:14:58.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f72147640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6c06db00 con 0x7f7f6c15d5e0 2024-04-25T10:14:58.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6affd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7f6c0cd820 0x7f7f6c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36130/0 (socket says 172.21.15.133:36130) 2024-04-25T10:14:58.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6affd640 1 -- 172.21.15.133:0/950611283 learned_addr learned my addr 172.21.15.133:0/950611283 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:58.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6bfff640 1 --2- 172.21.15.133:0/950611283 >> v2:172.21.15.162:3300/0 conn(0x7f7f6c15d5e0 0x7f7f6c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217833343 0 0) 0x7f7f6c0ce2e0 con 0x7f7f6c0cd820 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f54003530 con 0x7f7f6c0cd820 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3514184105 0 0) 0x7f7f54003530 con 0x7f7f6c0cd820 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 >> v2:172.21.15.162:3300/0 conn(0x7f7f6c15d5e0 msgr2=0x7f7f6c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 --2- 172.21.15.133:0/950611283 >> v2:172.21.15.162:3300/0 conn(0x7f7f6c15d5e0 0x7f7f6c0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 >> v1:172.21.15.162:6789/0 conn(0x7f7f6c145170 legacy=0x7f7f6c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c06d790 con 0x7f7f6c0cd820 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7f60001680 con 0x7f7f6c0cd820 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 >> v1:172.21.15.133:6789/0 conn(0x7f7f6c0cd820 legacy=0x7f7f6c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f540039c0 0x7f7f54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f54004810 0x7f7f54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f540051d0 0x7f7f5400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c06db00 con 0x7f7f540051d0 2024-04-25T10:14:58.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400c070 con 0x7f7f54004810 2024-04-25T10:14:58.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400c270 con 0x7f7f540039c0 2024-04-25T10:14:58.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.855+0000 7f7f6bfff640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f540051d0 0x7f7f5400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6affd640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f54004810 0x7f7f54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6b7fe640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f540039c0 0x7f7f54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6bfff640 1 -- 172.21.15.133:0/950611283 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f540039c0 msgr2=0x7f7f54003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6bfff640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f540039c0 0x7f7f54003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6bfff640 1 -- 172.21.15.133:0/950611283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f54004810 msgr2=0x7f7f54004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6bfff640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f54004810 0x7f7f54004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6bfff640 1 -- 172.21.15.133:0/950611283 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54003530 con 0x7f7f540051d0 2024-04-25T10:14:58.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6b7fe640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f540039c0 0x7f7f54003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6bfff640 1 -- 172.21.15.133:0/950611283 reap_dead start 2024-04-25T10:14:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6bfff640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f540051d0 0x7f7f5400b9d0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009f90 tx=0x7f7f6401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54603909210174bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f640b2070 con 0x7f7f540051d0 2024-04-25T10:14:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f640a5c90 con 0x7f7f540051d0 2024-04-25T10:14:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f6a7fc640 1 -- 172.21.15.133:0/950611283 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64009c10 con 0x7f7f540051d0 2024-04-25T10:14:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f72147640 1 -- 172.21.15.133:0/950611283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f540051d0 msgr2=0x7f7f5400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f72147640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f540051d0 0x7f7f5400b9d0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009f90 tx=0x7f7f6401cc80 comp rx=0 tx=0).stop 2024-04-25T10:14:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f72147640 1 -- 172.21.15.133:0/950611283 shutdown_connections 2024-04-25T10:14:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f72147640 1 --2- 172.21.15.133:0/950611283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f540051d0 0x7f7f5400b9d0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f72147640 1 -- 172.21.15.133:0/950611283 >> 172.21.15.133:0/950611283 conn(0x7f7f6c066f70 msgr2=0x7f7f6c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f72147640 1 -- 172.21.15.133:0/950611283 shutdown_connections 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.859+0000 7f7f72147640 1 -- 172.21.15.133:0/950611283 wait complete. 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f72147640 1 Processor -- start 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f72147640 1 -- start start 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f72147640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f540051d0 0x7f7f6c134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f72147640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c1f1f20 0x7f7f6c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f72147640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f6c152e30 0x7f7f6c155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f72147640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cdca0 con 0x7f7f6c1f1f20 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f72147640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c145550 con 0x7f7f6c152e30 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f72147640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cde20 con 0x7f7f540051d0 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c1f1f20 0x7f7f6c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c1f1f20 0x7f7f6c156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47378/0 (socket says 172.21.15.133:47378) 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f6affd640 1 -- 172.21.15.133:0/3321666805 learned_addr learned my addr 172.21.15.133:0/3321666805 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f6b7fe640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f540051d0 0x7f7f6c134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f6bfff640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f6c152e30 0x7f7f6c155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f6affd640 1 -- 172.21.15.133:0/3321666805 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f540051d0 msgr2=0x7f7f6c134f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f6affd640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f540051d0 0x7f7f6c134f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f6affd640 1 -- 172.21.15.133:0/3321666805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f6c152e30 msgr2=0x7f7f6c155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f6affd640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f6c152e30 0x7f7f6c155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f6affd640 1 -- 172.21.15.133:0/3321666805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c15a0c0 con 0x7f7f6c1f1f20 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f6affd640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c1f1f20 0x7f7f6c156de0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000e7b0 tx=0x7f7f6000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f53fff640 1 -- 172.21.15.133:0/3321666805 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6000c6c0 con 0x7f7f6c1f1f20 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f53fff640 1 -- 172.21.15.133:0/3321666805 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f60018070 con 0x7f7f6c1f1f20 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f72147640 1 -- 172.21.15.133:0/3321666805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c15a360 con 0x7f7f6c1f1f20 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f53fff640 1 -- 172.21.15.133:0/3321666805 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6000ca80 con 0x7f7f6c1f1f20 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f72147640 1 -- 172.21.15.133:0/3321666805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c15a920 con 0x7f7f6c1f1f20 2024-04-25T10:14:58.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f72147640 1 -- 172.21.15.133:0/3321666805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c068e80 con 0x7f7f6c1f1f20 2024-04-25T10:14:58.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.863+0000 7f7f53fff640 1 -- 172.21.15.133:0/3321666805 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f6000cc20 con 0x7f7f6c1f1f20 2024-04-25T10:14:58.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.867+0000 7f7f53fff640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f58066f60 0x7f7f58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.867+0000 7f7f6b7fe640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f58066f60 0x7f7f58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.867+0000 7f7f53fff640 1 -- 172.21.15.133:0/3321666805 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7f60084fc0 con 0x7f7f6c1f1f20 2024-04-25T10:14:58.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.867+0000 7f7f6b7fe640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f58066f60 0x7f7f58069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c06e230 tx=0x7f7f5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:58.871+0000 7f7f53fff640 1 -- 172.21.15.133:0/3321666805 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f60055e10 con 0x7f7f6c1f1f20 2024-04-25T10:14:59.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 -- 172.21.15.133:0/3321666805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f6c144610 con 0x7f7f6c1f1f20 2024-04-25T10:14:59.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f53fff640 1 -- 172.21.15.133:0/3321666805 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f60055710 con 0x7f7f6c1f1f20 2024-04-25T10:14:59.033 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:14:59.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 -- 172.21.15.133:0/3321666805 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f58066f60 msgr2=0x7f7f58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f58066f60 0x7f7f58069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c06e230 tx=0x7f7f5c009040 comp rx=0 tx=0).stop 2024-04-25T10:14:59.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 -- 172.21.15.133:0/3321666805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c1f1f20 msgr2=0x7f7f6c156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c1f1f20 0x7f7f6c156de0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000e7b0 tx=0x7f7f6000ec70 comp rx=0 tx=0).stop 2024-04-25T10:14:59.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 -- 172.21.15.133:0/3321666805 shutdown_connections 2024-04-25T10:14:59.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f6c152e30 0x7f7f6c155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7f58066f60 0x7f7f58069420 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c1f1f20 0x7f7f6c156de0 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 --2- 172.21.15.133:0/3321666805 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7f540051d0 0x7f7f6c134f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 -- 172.21.15.133:0/3321666805 >> 172.21.15.133:0/3321666805 conn(0x7f7f6c066f70 msgr2=0x7f7f6c15e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 -- 172.21.15.133:0/3321666805 shutdown_connections 2024-04-25T10:14:59.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.031+0000 7f7f72147640 1 -- 172.21.15.133:0/3321666805 wait complete. 2024-04-25T10:14:59.052 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:59.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb4210c6640 1 Processor -- start 2024-04-25T10:14:59.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb4210c6640 1 -- start start 2024-04-25T10:14:59.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb4210c6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb41c142920 0x7fb41c143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb4210c6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb41c140a40 0x7fb41c0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb4210c6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c0cf000 con 0x7fb41c15d9f0 2024-04-25T10:14:59.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb4210c6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41c140ea0 con 0x7fb41c140a40 2024-04-25T10:14:59.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb4210c6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41c0cf280 con 0x7fb41c142920 2024-04-25T10:14:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb41a59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb41c140a40 0x7fb41c0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb41a59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb41c140a40 0x7fb41c0ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47392/0 (socket says 172.21.15.133:47392) 2024-04-25T10:14:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb41a59c640 1 -- 172.21.15.133:0/366299725 learned_addr learned my addr 172.21.15.133:0/366299725 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb41ad9d640 1 --2- 172.21.15.133:0/366299725 >> v2:172.21.15.162:3300/0 conn(0x7fb41c142920 0x7fb41c143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb41a59c640 1 -- 172.21.15.133:0/366299725 >> v2:172.21.15.162:3300/0 conn(0x7fb41c142920 msgr2=0x7fb41c143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb41a59c640 1 --2- 172.21.15.133:0/366299725 >> v2:172.21.15.162:3300/0 conn(0x7fb41c142920 0x7fb41c143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb41a59c640 1 -- 172.21.15.133:0/366299725 >> v1:172.21.15.162:6789/0 conn(0x7fb41c15d9f0 legacy=0x7fb41c06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb41a59c640 1 -- 172.21.15.133:0/366299725 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c06d440 con 0x7fb41c140a40 2024-04-25T10:14:59.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.171+0000 7fb41ad9d640 1 --2- 172.21.15.133:0/366299725 >> v2:172.21.15.162:3300/0 conn(0x7fb41c142920 0x7fb41c143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb41a59c640 1 --2- 172.21.15.133:0/366299725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb41c140a40 0x7fb41c0ceac0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fb41c06ee50 tx=0x7fb41001e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5cdaefc2e18d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb419d9b640 1 -- 172.21.15.133:0/366299725 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4100ad070 con 0x7fb41c140a40 2024-04-25T10:14:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb419d9b640 1 -- 172.21.15.133:0/366299725 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb410004230 con 0x7fb41c140a40 2024-04-25T10:14:59.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb419d9b640 1 -- 172.21.15.133:0/366299725 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4100a7700 con 0x7fb41c140a40 2024-04-25T10:14:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb4210c6640 1 -- 172.21.15.133:0/366299725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb41c140a40 msgr2=0x7fb41c0ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb4210c6640 1 --2- 172.21.15.133:0/366299725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb41c140a40 0x7fb41c0ceac0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fb41c06ee50 tx=0x7fb41001e940 comp rx=0 tx=0).stop 2024-04-25T10:14:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb4210c6640 1 -- 172.21.15.133:0/366299725 shutdown_connections 2024-04-25T10:14:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb4210c6640 1 --2- 172.21.15.133:0/366299725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb41c140a40 0x7fb41c0ceac0 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb4210c6640 1 --2- 172.21.15.133:0/366299725 >> v2:172.21.15.162:3300/0 conn(0x7fb41c142920 0x7fb41c143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb4210c6640 1 -- 172.21.15.133:0/366299725 >> 172.21.15.133:0/366299725 conn(0x7fb41c066f70 msgr2=0x7fb41c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb4210c6640 1 -- 172.21.15.133:0/366299725 shutdown_connections 2024-04-25T10:14:59.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.175+0000 7fb4210c6640 1 -- 172.21.15.133:0/366299725 wait complete. 2024-04-25T10:14:59.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4210c6640 1 Processor -- start 2024-04-25T10:14:59.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4210c6640 1 -- start start 2024-04-25T10:14:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4210c6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb41c140a40 0x7fb41c13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4210c6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb41c142920 0x7fb41c0b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4210c6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb41c15d9f0 0x7fb41c0b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4210c6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c06ebc0 con 0x7fb41c15d9f0 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4210c6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c06ea40 con 0x7fb41c140a40 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4210c6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c142d00 con 0x7fb41c142920 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41b59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb41c15d9f0 0x7fb41c0b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41b59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb41c15d9f0 0x7fb41c0b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47406/0 (socket says 172.21.15.133:47406) 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41b59e640 1 -- 172.21.15.133:0/1346715375 learned_addr learned my addr 172.21.15.133:0/1346715375 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41a59c640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb41c142920 0x7fb41c0b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41ad9d640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb41c140a40 0x7fb41c13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41b59e640 1 -- 172.21.15.133:0/1346715375 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb41c142920 msgr2=0x7fb41c0b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41b59e640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb41c142920 0x7fb41c0b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41b59e640 1 -- 172.21.15.133:0/1346715375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb41c140a40 msgr2=0x7fb41c13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41b59e640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb41c140a40 0x7fb41c13e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41b59e640 1 -- 172.21.15.133:0/1346715375 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c15a810 con 0x7fb41c15d9f0 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41b59e640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb41c15d9f0 0x7fb41c0b3d80 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c730 tx=0x7fb41400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4037fe640 1 -- 172.21.15.133:0/1346715375 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb414003b80 con 0x7fb41c15d9f0 2024-04-25T10:14:59.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4037fe640 1 -- 172.21.15.133:0/1346715375 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb414003d20 con 0x7fb41c15d9f0 2024-04-25T10:14:59.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4210c6640 1 -- 172.21.15.133:0/1346715375 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41c15aaf0 con 0x7fb41c15d9f0 2024-04-25T10:14:59.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4037fe640 1 -- 172.21.15.133:0/1346715375 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41400aa30 con 0x7fb41c15d9f0 2024-04-25T10:14:59.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4210c6640 1 -- 172.21.15.133:0/1346715375 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c15b0b0 con 0x7fb41c15d9f0 2024-04-25T10:14:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4210c6640 1 -- 172.21.15.133:0/1346715375 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c068e80 con 0x7fb41c15d9f0 2024-04-25T10:14:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4037fe640 1 -- 172.21.15.133:0/1346715375 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb41400abd0 con 0x7fb41c15d9f0 2024-04-25T10:14:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4037fe640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb408066f60 0x7fb408069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb4037fe640 1 -- 172.21.15.133:0/1346715375 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb414084ff0 con 0x7fb41c15d9f0 2024-04-25T10:14:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.179+0000 7fb41ad9d640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb408066f60 0x7fb408069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.183+0000 7fb41ad9d640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb408066f60 0x7fb408069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00d8b0 tx=0x7fb40c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.187+0000 7fb4037fe640 1 -- 172.21.15.133:0/1346715375 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb41c0cf000 con 0x7fb41c15d9f0 2024-04-25T10:14:59.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.347+0000 7fb4210c6640 1 -- 172.21.15.133:0/1346715375 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb41c0b0bf0 con 0x7fb41c15d9f0 2024-04-25T10:14:59.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4037fe640 1 -- 172.21.15.133:0/1346715375 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb414055ed0 con 0x7fb41c15d9f0 2024-04-25T10:14:59.352 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 -- 172.21.15.133:0/1346715375 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb408066f60 msgr2=0x7fb408069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb408066f60 0x7fb408069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00d8b0 tx=0x7fb40c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 -- 172.21.15.133:0/1346715375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb41c15d9f0 msgr2=0x7fb41c0b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb41c15d9f0 0x7fb41c0b3d80 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c730 tx=0x7fb41400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 -- 172.21.15.133:0/1346715375 shutdown_connections 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb408066f60 0x7fb408069420 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb41c15d9f0 0x7fb41c0b3d80 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb41c142920 0x7fb41c0b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 --2- 172.21.15.133:0/1346715375 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb41c140a40 0x7fb41c13e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 -- 172.21.15.133:0/1346715375 >> 172.21.15.133:0/1346715375 conn(0x7fb41c066f70 msgr2=0x7fb41c141080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 -- 172.21.15.133:0/1346715375 shutdown_connections 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.351+0000 7fb4210c6640 1 -- 172.21.15.133:0/1346715375 wait complete. 2024-04-25T10:14:59.375 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:59.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb269081640 1 Processor -- start 2024-04-25T10:14:59.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb269081640 1 -- start start 2024-04-25T10:14:59.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb269081640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb26406dca0 0x7fb2640b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb269081640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2640cfdc0 con 0x7fb2640cc250 2024-04-25T10:14:59.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb269081640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2640d00a0 con 0x7fb26406bda0 2024-04-25T10:14:59.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb269081640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2640d0660 con 0x7fb26406dca0 2024-04-25T10:14:59.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb26359e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb2640cc250 0x7fb2640cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36136/0 (socket says 172.21.15.133:36136) 2024-04-25T10:14:59.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb26359e640 1 -- 172.21.15.133:0/3097860650 learned_addr learned my addr 172.21.15.133:0/3097860650 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:59.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb262d9d640 1 --2- 172.21.15.133:0/3097860650 >> v2:172.21.15.162:3300/0 conn(0x7fb26406dca0 0x7fb2640b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229503654 0 0) 0x7fb2640cfdc0 con 0x7fb2640cc250 2024-04-25T10:14:59.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb24c003530 con 0x7fb2640cc250 2024-04-25T10:14:59.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 495428892 0 0) 0x7fb24c003530 con 0x7fb2640cc250 2024-04-25T10:14:59.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 >> v2:172.21.15.162:3300/0 conn(0x7fb26406dca0 msgr2=0x7fb2640b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 --2- 172.21.15.133:0/3097860650 >> v2:172.21.15.162:3300/0 conn(0x7fb26406dca0 0x7fb2640b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 >> v1:172.21.15.162:6789/0 conn(0x7fb26406bda0 legacy=0x7fb2640b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2640d0380 con 0x7fb2640cc250 2024-04-25T10:14:59.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb25c001a00 con 0x7fb2640cc250 2024-04-25T10:14:59.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb262d9d640 1 --2- 172.21.15.133:0/3097860650 >> v2:172.21.15.162:3300/0 conn(0x7fb26406dca0 0x7fb2640b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:59.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 >> v1:172.21.15.133:6789/0 conn(0x7fb2640cc250 legacy=0x7fb2640cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 0x7fb24c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb24c004810 0x7fb24c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb24c0052f0 0x7fb24c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2640d0660 con 0x7fb24c003a10 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c00c1f0 con 0x7fb24c004810 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24c00c3f0 con 0x7fb24c0052f0 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb262d9d640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 0x7fb24c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb26259c640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb24c004810 0x7fb24c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb26359e640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb24c0052f0 0x7fb24c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb262d9d640 1 -- 172.21.15.133:0/3097860650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb24c0052f0 msgr2=0x7fb24c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb262d9d640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb24c0052f0 0x7fb24c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb262d9d640 1 -- 172.21.15.133:0/3097860650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb24c004810 msgr2=0x7fb24c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb262d9d640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb24c004810 0x7fb24c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb262d9d640 1 -- 172.21.15.133:0/3097860650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c003530 con 0x7fb24c003a10 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb26359e640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb24c0052f0 0x7fb24c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:59.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.495+0000 7fb26359e640 1 -- 172.21.15.133:0/3097860650 reap_dead start 2024-04-25T10:14:59.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb262d9d640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 0x7fb24c003e30 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fb254009cb0 tx=0x7fb25401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6206f529e8c1e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2540b2070 con 0x7fb24c003a10 2024-04-25T10:14:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2540a56b0 con 0x7fb24c003a10 2024-04-25T10:14:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb261d9b640 1 -- 172.21.15.133:0/3097860650 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254009880 con 0x7fb24c003a10 2024-04-25T10:14:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb269081640 1 -- 172.21.15.133:0/3097860650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 msgr2=0x7fb24c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb269081640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 0x7fb24c003e30 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fb254009cb0 tx=0x7fb25401cc10 comp rx=0 tx=0).stop 2024-04-25T10:14:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb269081640 1 -- 172.21.15.133:0/3097860650 shutdown_connections 2024-04-25T10:14:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb269081640 1 --2- 172.21.15.133:0/3097860650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 0x7fb24c003e30 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb269081640 1 -- 172.21.15.133:0/3097860650 >> 172.21.15.133:0/3097860650 conn(0x7fb2640b31a0 msgr2=0x7fb2640b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb269081640 1 -- 172.21.15.133:0/3097860650 shutdown_connections 2024-04-25T10:14:59.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb269081640 1 -- 172.21.15.133:0/3097860650 wait complete. 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.499+0000 7fb269081640 1 Processor -- start 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb269081640 1 -- start start 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb269081640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 0x7fb2641ea730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb269081640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2640c93e0 0x7fb264155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb269081640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb264155ed0 0x7fb2640cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb269081640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2640b21c0 con 0x7fb24c003a10 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb269081640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2640b2040 con 0x7fb2640c93e0 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb269081640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2640b2340 con 0x7fb264155ed0 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb262d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 0x7fb2641ea730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb262d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 0x7fb2641ea730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47428/0 (socket says 172.21.15.133:47428) 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb262d9d640 1 -- 172.21.15.133:0/4252212082 learned_addr learned my addr 172.21.15.133:0/4252212082 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:59.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb26259c640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2640c93e0 0x7fb264155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb26359e640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb264155ed0 0x7fb2640cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb262d9d640 1 -- 172.21.15.133:0/4252212082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb264155ed0 msgr2=0x7fb2640cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb262d9d640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb264155ed0 0x7fb2640cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb262d9d640 1 -- 172.21.15.133:0/4252212082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2640c93e0 msgr2=0x7fb264155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb262d9d640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2640c93e0 0x7fb264155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb262d9d640 1 -- 172.21.15.133:0/4252212082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb264156350 con 0x7fb24c003a10 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb26359e640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb264155ed0 0x7fb2640cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb26259c640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2640c93e0 0x7fb264155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb262d9d640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 0x7fb2641ea730 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fb25401ca40 tx=0x7fb2540199b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb24b7fe640 1 -- 172.21.15.133:0/4252212082 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2540bd070 con 0x7fb24c003a10 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb24b7fe640 1 -- 172.21.15.133:0/4252212082 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2540ad440 con 0x7fb24c003a10 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb269081640 1 -- 172.21.15.133:0/4252212082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2640cea70 con 0x7fb24c003a10 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb24b7fe640 1 -- 172.21.15.133:0/4252212082 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2540b2040 con 0x7fb24c003a10 2024-04-25T10:14:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb269081640 1 -- 172.21.15.133:0/4252212082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2640c8be0 con 0x7fb24c003a10 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb269081640 1 -- 172.21.15.133:0/4252212082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb264067c70 con 0x7fb24c003a10 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb24b7fe640 1 -- 172.21.15.133:0/4252212082 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2540ad810 con 0x7fb24c003a10 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb24b7fe640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 0x7fb250069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb24b7fe640 1 -- 172.21.15.133:0/4252212082 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb25411ed00 con 0x7fb24c003a10 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.503+0000 7fb26259c640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 0x7fb250069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.507+0000 7fb26259c640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 0x7fb250069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb2641f9dc0 tx=0x7fb258009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.507+0000 7fb24b7fe640 1 -- 172.21.15.133:0/4252212082 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb254016920 con 0x7fb24c003a10 2024-04-25T10:14:59.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.703+0000 7fb269081640 1 -- 172.21.15.133:0/4252212082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb26409a5f0 con 0x7fb24c003a10 2024-04-25T10:14:59.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb24b7fe640 1 -- 172.21.15.133:0/4252212082 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4966 (secure 0 0 0) 0x7fb2540ef3f0 con 0x7fb24c003a10 2024-04-25T10:14:59.710 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:14:59.710 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":93,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3448301688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3448301688},{"type":"v1","addr":"172.21.15.162:6833","nonce":3448301688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/4188704231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":4188704231},{"type":"v1","addr":"172.21.15.162:6835","nonce":4188704231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":93,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:14:28.005823+0000","modified":"2024-04-25T10:14:54.057373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2626366082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2626366082},{"type":"v1","addr":"172.21.15.133:6835","nonce":2626366082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10050":{"gid":10050,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3234096063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3234096063},{"type":"v1","addr":"172.21.15.133:6837","nonce":3234096063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:14:59.710 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 93 2024-04-25T10:14:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 -- 172.21.15.133:0/4252212082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 msgr2=0x7fb250069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 0x7fb250069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb2641f9dc0 tx=0x7fb258009040 comp rx=0 tx=0).stop 2024-04-25T10:14:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 -- 172.21.15.133:0/4252212082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 msgr2=0x7fb2641ea730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 0x7fb2641ea730 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fb25401ca40 tx=0x7fb2540199b0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 -- 172.21.15.133:0/4252212082 shutdown_connections 2024-04-25T10:14:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb264155ed0 0x7fb2640cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2640c93e0 0x7fb264155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb250066f60 0x7fb250069420 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 --2- 172.21.15.133:0/4252212082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb24c003a10 0x7fb2641ea730 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 -- 172.21.15.133:0/4252212082 >> 172.21.15.133:0/4252212082 conn(0x7fb2640b31a0 msgr2=0x7fb2640aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 -- 172.21.15.133:0/4252212082 shutdown_connections 2024-04-25T10:14:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.707+0000 7fb269081640 1 -- 172.21.15.133:0/4252212082 wait complete. 2024-04-25T10:14:59.729 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:59.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.847+0000 7fe58b3ef640 1 Processor -- start 2024-04-25T10:14:59.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.847+0000 7fe58b3ef640 1 -- start start 2024-04-25T10:14:59.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58b3ef640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe58415c9a0 0x7fe5840c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58b3ef640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5840c61d0 con 0x7fe5840ce330 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58b3ef640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5840c64b0 con 0x7fe5840b1f70 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58b3ef640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58406d4f0 con 0x7fe58415c9a0 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58998c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe5840ce330 0x7fe58415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36138/0 (socket says 172.21.15.133:36138) 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58998c640 1 -- 172.21.15.133:0/335845410 learned_addr learned my addr 172.21.15.133:0/335845410 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58918b640 1 --2- 172.21.15.133:0/335845410 >> v2:172.21.15.162:3300/0 conn(0x7fe58415c9a0 0x7fe5840c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780074945 0 0) 0x7fe5840c61d0 con 0x7fe5840ce330 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe570003530 con 0x7fe5840ce330 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3123269282 0 0) 0x7fe570003530 con 0x7fe5840ce330 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 >> v2:172.21.15.162:3300/0 conn(0x7fe58415c9a0 msgr2=0x7fe5840c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 --2- 172.21.15.133:0/335845410 >> v2:172.21.15.162:3300/0 conn(0x7fe58415c9a0 0x7fe5840c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 >> v1:172.21.15.162:6789/0 conn(0x7fe5840b1f70 legacy=0x7fe58406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58406d7d0 con 0x7fe5840ce330 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe57c001a00 con 0x7fe5840ce330 2024-04-25T10:14:59.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 >> v1:172.21.15.133:6789/0 conn(0x7fe5840ce330 legacy=0x7fe58415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 --2- 172.21.15.133:0/335845410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5700039c0 0x7fe570003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 --2- 172.21.15.133:0/335845410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570004810 0x7fe570004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 --2- 172.21.15.133:0/335845410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5700051d0 0x7fe57000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58406d4f0 con 0x7fe5700051d0 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe570010480 con 0x7fe570004810 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe570010680 con 0x7fe5700039c0 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58998c640 1 --2- 172.21.15.133:0/335845410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5700051d0 0x7fe57000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58918b640 1 --2- 172.21.15.133:0/335845410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5700039c0 0x7fe570003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58898a640 1 --2- 172.21.15.133:0/335845410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570004810 0x7fe570004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58998c640 1 -- 172.21.15.133:0/335845410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5700039c0 msgr2=0x7fe570003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58998c640 1 --2- 172.21.15.133:0/335845410 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe5700039c0 0x7fe570003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58998c640 1 -- 172.21.15.133:0/335845410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570004810 msgr2=0x7fe570004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58998c640 1 --2- 172.21.15.133:0/335845410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe570004810 0x7fe570004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.851+0000 7fe58998c640 1 -- 172.21.15.133:0/335845410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe570003530 con 0x7fe5700051d0 2024-04-25T10:14:59.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe58998c640 1 -- 172.21.15.133:0/335845410 reap_dead start 2024-04-25T10:14:59.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe58998c640 1 --2- 172.21.15.133:0/335845410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5700051d0 0x7fe57000b9d0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00a860 tx=0x7fe57c01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0148ed58b0bdfbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00a480 con 0x7fe5700051d0 2024-04-25T10:14:59.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c0ad630 con 0x7fe5700051d0 2024-04-25T10:14:59.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe57bfff640 1 -- 172.21.15.133:0/335845410 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0ada20 con 0x7fe5700051d0 2024-04-25T10:14:59.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe58b3ef640 1 -- 172.21.15.133:0/335845410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5700051d0 msgr2=0x7fe57000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe58b3ef640 1 --2- 172.21.15.133:0/335845410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5700051d0 0x7fe57000b9d0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00a860 tx=0x7fe57c01d380 comp rx=0 tx=0).stop 2024-04-25T10:14:59.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe58b3ef640 1 -- 172.21.15.133:0/335845410 shutdown_connections 2024-04-25T10:14:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe58b3ef640 1 --2- 172.21.15.133:0/335845410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5700051d0 0x7fe57000b9d0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe58b3ef640 1 -- 172.21.15.133:0/335845410 >> 172.21.15.133:0/335845410 conn(0x7fe584067000 msgr2=0x7fe58406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe58b3ef640 1 -- 172.21.15.133:0/335845410 shutdown_connections 2024-04-25T10:14:59.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe58b3ef640 1 -- 172.21.15.133:0/335845410 wait complete. 2024-04-25T10:14:59.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe58b3ef640 1 Processor -- start 2024-04-25T10:14:59.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.855+0000 7fe58b3ef640 1 -- start start 2024-04-25T10:14:59.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58b3ef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700051d0 0x7fe58415a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58b3ef640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe584150c20 0x7fe5841512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58b3ef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe58413db80 0x7fe58413e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58b3ef640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840ce780 con 0x7fe58413db80 2024-04-25T10:14:59.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58b3ef640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840ce900 con 0x7fe5700051d0 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58b3ef640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840cea80 con 0x7fe584150c20 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58998c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe58413db80 0x7fe58413e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58998c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe58413db80 0x7fe58413e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47438/0 (socket says 172.21.15.133:47438) 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58998c640 1 -- 172.21.15.133:0/972265545 learned_addr learned my addr 172.21.15.133:0/972265545 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58918b640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700051d0 0x7fe58415a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58898a640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe584150c20 0x7fe5841512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58998c640 1 -- 172.21.15.133:0/972265545 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe584150c20 msgr2=0x7fe5841512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58998c640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe584150c20 0x7fe5841512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58998c640 1 -- 172.21.15.133:0/972265545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700051d0 msgr2=0x7fe58415a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58998c640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700051d0 0x7fe58415a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58998c640 1 -- 172.21.15.133:0/972265545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe584154620 con 0x7fe58413db80 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58918b640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700051d0 0x7fe58415a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:59.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58998c640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe58413db80 0x7fe58413e000 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fe57c01dc20 tx=0x7fe570005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe579ffb640 1 -- 172.21.15.133:0/972265545 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0bc070 con 0x7fe58413db80 2024-04-25T10:14:59.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe579ffb640 1 -- 172.21.15.133:0/972265545 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c001c20 con 0x7fe58413db80 2024-04-25T10:14:59.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58b3ef640 1 -- 172.21.15.133:0/972265545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5841510a0 con 0x7fe58413db80 2024-04-25T10:14:59.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe579ffb640 1 -- 172.21.15.133:0/972265545 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0ad9e0 con 0x7fe58413db80 2024-04-25T10:14:59.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58b3ef640 1 -- 172.21.15.133:0/972265545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe584159df0 con 0x7fe58413db80 2024-04-25T10:14:59.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58b3ef640 1 -- 172.21.15.133:0/972265545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe584068ef0 con 0x7fe58413db80 2024-04-25T10:14:59.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe579ffb640 1 -- 172.21.15.133:0/972265545 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe57c0bf070 con 0x7fe58413db80 2024-04-25T10:14:59.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe579ffb640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe568066f90 0x7fe568069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe58918b640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe568066f90 0x7fe568069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.859+0000 7fe579ffb640 1 -- 172.21.15.133:0/972265545 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe57c11f090 con 0x7fe58413db80 2024-04-25T10:14:59.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.863+0000 7fe58918b640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe568066f90 0x7fe568069450 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe5740099a0 tx=0x7fe574008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:14:59.863+0000 7fe579ffb640 1 -- 172.21.15.133:0/972265545 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe57c0b4070 con 0x7fe58413db80 2024-04-25T10:15:00.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.023+0000 7fe58b3ef640 1 -- 172.21.15.133:0/972265545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe584158dd0 con 0x7fe58413db80 2024-04-25T10:15:00.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.027+0000 7fe579ffb640 1 -- 172.21.15.133:0/972265545 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+17074 (secure 0 0 0) 0x7fe57c017090 con 0x7fe58413db80 2024-04-25T10:15:00.031 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:00.031 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":145,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:34.721848+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:26.340459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:26.996481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.28","mappings":[{"from":3,"to":0}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:00.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 -- 172.21.15.133:0/972265545 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe568066f90 msgr2=0x7fe568069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe568066f90 0x7fe568069450 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe5740099a0 tx=0x7fe574008040 comp rx=0 tx=0).stop 2024-04-25T10:15:00.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 -- 172.21.15.133:0/972265545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe58413db80 msgr2=0x7fe58413e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe58413db80 0x7fe58413e000 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fe57c01dc20 tx=0x7fe570005840 comp rx=0 tx=0).stop 2024-04-25T10:15:00.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 -- 172.21.15.133:0/972265545 shutdown_connections 2024-04-25T10:15:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe568066f90 0x7fe568069450 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe58413db80 0x7fe58413e000 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe584150c20 0x7fe5841512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 --2- 172.21.15.133:0/972265545 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe5700051d0 0x7fe58415a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 -- 172.21.15.133:0/972265545 >> 172.21.15.133:0/972265545 conn(0x7fe584067000 msgr2=0x7fe5841575c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 -- 172.21.15.133:0/972265545 shutdown_connections 2024-04-25T10:15:00.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.031+0000 7fe58b3ef640 1 -- 172.21.15.133:0/972265545 wait complete. 2024-04-25T10:15:00.050 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:15:00.050 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:00.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bee04640 1 Processor -- start 2024-04-25T10:15:00.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bee04640 1 -- start start 2024-04-25T10:15:00.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bee04640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b8158110 0x7f08b80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bee04640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f08b81584f0 0x7f08b815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bee04640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b80cbe60 con 0x7f08b806e460 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bee04640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b80b19b0 con 0x7f08b81584f0 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bee04640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b815b360 con 0x7f08b8158110 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bd3a1640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f08b806e460 0x7f08b815e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36142/0 (socket says 172.21.15.133:36142) 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bd3a1640 1 -- 172.21.15.133:0/1538793743 learned_addr learned my addr 172.21.15.133:0/1538793743 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bcba0640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b8158110 0x7f08b80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b7fff640 1 --2- 172.21.15.133:0/1538793743 >> v2:172.21.15.162:3300/0 conn(0x7f08b81584f0 0x7f08b815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573616895 0 0) 0x7f08b80cbe60 con 0x7f08b806e460 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08a0003530 con 0x7f08b806e460 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2931844578 0 0) 0x7f08a0003530 con 0x7f08b806e460 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b8158110 msgr2=0x7f08b80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b8158110 0x7f08b80b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 >> v2:172.21.15.162:3300/0 conn(0x7f08b81584f0 msgr2=0x7f08b815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 --2- 172.21.15.133:0/1538793743 >> v2:172.21.15.162:3300/0 conn(0x7f08b81584f0 0x7f08b815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b815b640 con 0x7f08b806e460 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f08b0001a00 con 0x7f08b806e460 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 >> v1:172.21.15.133:6789/0 conn(0x7f08b806e460 legacy=0x7f08b815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b8158110 0x7f08a00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08a0003f00 0x7f08a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08a0004d50 0x7f08a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80b19b0 con 0x7f08a0003f00 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b815b360 con 0x7f08a0004d50 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a000b9d0 con 0x7f08b8158110 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b7fff640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08a0003f00 0x7f08a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bcba0640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b8158110 0x7f08a00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bd3a1640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08a0004d50 0x7f08a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b7fff640 1 -- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b8158110 msgr2=0x7f08a00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b7fff640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08b8158110 0x7f08a00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b7fff640 1 -- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08a0004d50 msgr2=0x7f08a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b7fff640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08a0004d50 0x7f08a00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08b7fff640 1 -- 172.21.15.133:0/1538793743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a0003530 con 0x7f08a0003f00 2024-04-25T10:15:00.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.155+0000 7f08bd3a1640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08a0004d50 0x7f08a00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b7fff640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08a0003f00 0x7f08a0004810 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009ae0 tx=0x7f08ac01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a37fdb8db2a9e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0b2070 con 0x7f08a0003f00 2024-04-25T10:15:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac0a56a0 con 0x7f08a0003f00 2024-04-25T10:15:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b77fe640 1 -- 172.21.15.133:0/1538793743 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0096b0 con 0x7f08a0003f00 2024-04-25T10:15:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 -- 172.21.15.133:0/1538793743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08a0003f00 msgr2=0x7f08a0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 --2- 172.21.15.133:0/1538793743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08a0003f00 0x7f08a0004810 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009ae0 tx=0x7f08ac01cab0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bd3a1640 1 -- 172.21.15.133:0/1538793743 reap_dead start 2024-04-25T10:15:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 -- 172.21.15.133:0/1538793743 shutdown_connections 2024-04-25T10:15:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 -- 172.21.15.133:0/1538793743 >> 172.21.15.133:0/1538793743 conn(0x7f08b8067000 msgr2=0x7f08b806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 -- 172.21.15.133:0/1538793743 shutdown_connections 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 -- 172.21.15.133:0/1538793743 wait complete. 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 Processor -- start 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 -- start start 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08a0003f00 0x7f08b80a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81f6d50 0x7f08b81f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08b81499f0 0x7f08b8149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80cd670 con 0x7f08b81f6d50 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b815bc00 con 0x7f08b81499f0 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bee04640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80cd7f0 con 0x7f08a0003f00 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81f6d50 0x7f08b81f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81f6d50 0x7f08b81f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47450/0 (socket says 172.21.15.133:47450) 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b7fff640 1 -- 172.21.15.133:0/3660171544 learned_addr learned my addr 172.21.15.133:0/3660171544 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bcba0640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08a0003f00 0x7f08b80a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08bd3a1640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08b81499f0 0x7f08b8149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b7fff640 1 -- 172.21.15.133:0/3660171544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08a0003f00 msgr2=0x7f08b80a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b7fff640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08a0003f00 0x7f08b80a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b7fff640 1 -- 172.21.15.133:0/3660171544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08b81499f0 msgr2=0x7f08b8149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b7fff640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08b81499f0 0x7f08b8149e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.159+0000 7f08b7fff640 1 -- 172.21.15.133:0/3660171544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b80b24a0 con 0x7f08b81f6d50 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08b7fff640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81f6d50 0x7f08b81f71b0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009ae0 tx=0x7f08a0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08b57fa640 1 -- 172.21.15.133:0/3660171544 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0b2040 con 0x7f08b81f6d50 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08b57fa640 1 -- 172.21.15.133:0/3660171544 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac004230 con 0x7f08b81f6d50 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08bee04640 1 -- 172.21.15.133:0/3660171544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b81f7a80 con 0x7f08b81f6d50 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08b57fa640 1 -- 172.21.15.133:0/3660171544 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac003620 con 0x7f08b81f6d50 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08bee04640 1 -- 172.21.15.133:0/3660171544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b80b2a40 con 0x7f08b81f6d50 2024-04-25T10:15:00.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08bee04640 1 -- 172.21.15.133:0/3660171544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b8068ef0 con 0x7f08b81f6d50 2024-04-25T10:15:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08b57fa640 1 -- 172.21.15.133:0/3660171544 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08ac0037c0 con 0x7f08b81f6d50 2024-04-25T10:15:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08b57fa640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08a4066f60 0x7f08a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08b57fa640 1 -- 172.21.15.133:0/3660171544 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f08ac120590 con 0x7f08b81f6d50 2024-04-25T10:15:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08bcba0640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08a4066f60 0x7f08a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.163+0000 7f08bcba0640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08a4066f60 0x7f08a4069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009950 tx=0x7f08a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.167+0000 7f08b57fa640 1 -- 172.21.15.133:0/3660171544 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08ac0167c0 con 0x7f08b81f6d50 2024-04-25T10:15:00.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.347+0000 7f08bee04640 1 -- 172.21.15.133:0/3660171544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08b8158b90 con 0x7f08b81f6d50 2024-04-25T10:15:00.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.347+0000 7f08b57fa640 1 -- 172.21.15.133:0/3660171544 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4966 (secure 0 0 0) 0x7f08ac0f1670 con 0x7f08b81f6d50 2024-04-25T10:15:00.351 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":93,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3448301688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3448301688},{"type":"v1","addr":"172.21.15.162:6833","nonce":3448301688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/4188704231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":4188704231},{"type":"v1","addr":"172.21.15.162:6835","nonce":4188704231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":93,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:14:28.005823+0000","modified":"2024-04-25T10:14:54.057373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2626366082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2626366082},{"type":"v1","addr":"172.21.15.133:6835","nonce":2626366082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10050":{"gid":10050,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3234096063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3234096063},{"type":"v1","addr":"172.21.15.133:6837","nonce":3234096063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:15:00.352 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 93 2024-04-25T10:15:00.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 -- 172.21.15.133:0/3660171544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08a4066f60 msgr2=0x7f08a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08a4066f60 0x7f08a4069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009950 tx=0x7f08a8008040 comp rx=0 tx=0).stop 2024-04-25T10:15:00.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 -- 172.21.15.133:0/3660171544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81f6d50 msgr2=0x7f08b81f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81f6d50 0x7f08b81f71b0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009ae0 tx=0x7f08a0005840 comp rx=0 tx=0).stop 2024-04-25T10:15:00.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 -- 172.21.15.133:0/3660171544 shutdown_connections 2024-04-25T10:15:00.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f08b81499f0 0x7f08b8149e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f08a4066f60 0x7f08a4069420 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81f6d50 0x7f08b81f71b0 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 --2- 172.21.15.133:0/3660171544 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f08a0003f00 0x7f08b80a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 -- 172.21.15.133:0/3660171544 >> 172.21.15.133:0/3660171544 conn(0x7f08b8067000 msgr2=0x7f08b80c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 -- 172.21.15.133:0/3660171544 shutdown_connections 2024-04-25T10:15:00.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.351+0000 7f08bee04640 1 -- 172.21.15.133:0/3660171544 wait complete. 2024-04-25T10:15:00.369 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:00.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfc0fb640 1 Processor -- start 2024-04-25T10:15:00.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfc0fb640 1 -- start start 2024-04-25T10:15:00.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfc0fb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcf415db00 0x7fdcf40b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfc0fb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdcf40d0bd0 0x7fdcf4161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfc0fb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf40ceb20 con 0x7fdcf40d0770 2024-04-25T10:15:00.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfc0fb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf40b7740 con 0x7fdcf40d0bd0 2024-04-25T10:15:00.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfc0fb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40b7a20 con 0x7fdcf415db00 2024-04-25T10:15:00.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfa698640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdcf40d0bd0 0x7fdcf4161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfa698640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdcf40d0bd0 0x7fdcf4161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47454/0 (socket says 172.21.15.133:47454) 2024-04-25T10:15:00.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfa698640 1 -- 172.21.15.133:0/3648994976 learned_addr learned my addr 172.21.15.133:0/3648994976 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:00.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfa698640 1 -- 172.21.15.133:0/3648994976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcf415db00 msgr2=0x7fdcf40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfa698640 1 --2- 172.21.15.133:0/3648994976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcf415db00 0x7fdcf40b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfa698640 1 -- 172.21.15.133:0/3648994976 >> v1:172.21.15.162:6789/0 conn(0x7fdcf40d0770 legacy=0x7fdcf41614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.475+0000 7fdcfa698640 1 -- 172.21.15.133:0/3648994976 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf40b97f0 con 0x7fdcf40d0bd0 2024-04-25T10:15:00.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfa698640 1 --2- 172.21.15.133:0/3648994976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcf40d0bd0 0x7fdcf4161bd0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009e50 tx=0x7fdcec01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e2b6b62b09f779 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcf8e95640 1 -- 172.21.15.133:0/3648994976 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0a5840 con 0x7fdcf40d0bd0 2024-04-25T10:15:00.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcf8e95640 1 -- 172.21.15.133:0/3648994976 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec0a59e0 con 0x7fdcf40d0bd0 2024-04-25T10:15:00.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcf8e95640 1 -- 172.21.15.133:0/3648994976 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0a5db0 con 0x7fdcf40d0bd0 2024-04-25T10:15:00.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/3648994976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcf40d0bd0 msgr2=0x7fdcf4161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 --2- 172.21.15.133:0/3648994976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcf40d0bd0 0x7fdcf4161bd0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009e50 tx=0x7fdcec01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/3648994976 shutdown_connections 2024-04-25T10:15:00.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 --2- 172.21.15.133:0/3648994976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcf40d0bd0 0x7fdcf4161bd0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 --2- 172.21.15.133:0/3648994976 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcf415db00 0x7fdcf40b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/3648994976 >> 172.21.15.133:0/3648994976 conn(0x7fdcf40727c0 msgr2=0x7fdcf4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/3648994976 shutdown_connections 2024-04-25T10:15:00.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/3648994976 wait complete. 2024-04-25T10:15:00.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 Processor -- start 2024-04-25T10:15:00.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 -- start start 2024-04-25T10:15:00.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcf40d0770 0x7fdcf40c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcf40d0bd0 0x7fdcf40c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcf415db00 0x7fdcf4153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40b7430 con 0x7fdcf415db00 2024-04-25T10:15:00.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40d1060 con 0x7fdcf40d0bd0 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.479+0000 7fdcfc0fb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40d18a0 con 0x7fdcf40d0770 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfa698640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcf415db00 0x7fdcf4153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfa698640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcf415db00 0x7fdcf4153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47470/0 (socket says 172.21.15.133:47470) 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfa698640 1 -- 172.21.15.133:0/465744103 learned_addr learned my addr 172.21.15.133:0/465744103 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcf9696640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcf40d0bd0 0x7fdcf40c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcf9e97640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcf40d0770 0x7fdcf40c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfa698640 1 -- 172.21.15.133:0/465744103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcf40d0770 msgr2=0x7fdcf40c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfa698640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcf40d0770 0x7fdcf40c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfa698640 1 -- 172.21.15.133:0/465744103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcf40d0bd0 msgr2=0x7fdcf40c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfa698640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcf40d0bd0 0x7fdcf40c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfa698640 1 -- 172.21.15.133:0/465744103 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf40c8dc0 con 0x7fdcf415db00 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcf9e97640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcf40d0770 0x7fdcf40c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:00.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcf9696640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcf40d0bd0 0x7fdcf40c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfa698640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcf415db00 0x7fdcf4153f10 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0af040 tx=0x7fdcec019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcea7fc640 1 -- 172.21.15.133:0/465744103 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0b8880 con 0x7fdcf415db00 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcea7fc640 1 -- 172.21.15.133:0/465744103 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec0be070 con 0x7fdcf415db00 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/465744103 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf40c90a0 con 0x7fdcf415db00 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcea7fc640 1 -- 172.21.15.133:0/465744103 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0b8c60 con 0x7fdcf415db00 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/465744103 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf40c9660 con 0x7fdcf415db00 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/465744103 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf4068ef0 con 0x7fdcf415db00 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcea7fc640 1 -- 172.21.15.133:0/465744103 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdcec0c3460 con 0x7fdcf415db00 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcea7fc640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdcd8066f60 0x7fdcd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcea7fc640 1 -- 172.21.15.133:0/465744103 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fdcec1205d0 con 0x7fdcf415db00 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcf9e97640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdcd8066f60 0x7fdcd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.483+0000 7fdcf9e97640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdcd8066f60 0x7fdcd8069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000a8b0 tx=0x7fdcf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.487+0000 7fdcea7fc640 1 -- 172.21.15.133:0/465744103 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdcec0165e0 con 0x7fdcf415db00 2024-04-25T10:15:00.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.675+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/465744103 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcf40b2290 con 0x7fdcf415db00 2024-04-25T10:15:00.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.675+0000 7fdcea7fc640 1 -- 172.21.15.133:0/465744103 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4966 (secure 0 0 0) 0x7fdcec0f1540 con 0x7fdcf415db00 2024-04-25T10:15:00.679 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:00.679 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":93,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3448301688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3448301688},{"type":"v1","addr":"172.21.15.162:6833","nonce":3448301688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/4188704231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":4188704231},{"type":"v1","addr":"172.21.15.162:6835","nonce":4188704231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":93,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:14:28.005823+0000","modified":"2024-04-25T10:14:54.057373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2626366082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2626366082},{"type":"v1","addr":"172.21.15.133:6835","nonce":2626366082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10050":{"gid":10050,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3234096063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3234096063},{"type":"v1","addr":"172.21.15.133:6837","nonce":3234096063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:15:00.679 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 93 2024-04-25T10:15:00.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/465744103 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdcd8066f60 msgr2=0x7fdcd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdcd8066f60 0x7fdcd8069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000a8b0 tx=0x7fdcf0008040 comp rx=0 tx=0).stop 2024-04-25T10:15:00.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/465744103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcf415db00 msgr2=0x7fdcf4153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcf415db00 0x7fdcf4153f10 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0af040 tx=0x7fdcec019760 comp rx=0 tx=0).stop 2024-04-25T10:15:00.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/465744103 shutdown_connections 2024-04-25T10:15:00.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdcd8066f60 0x7fdcd8069420 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcf415db00 0x7fdcf4153f10 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdcf40d0bd0 0x7fdcf40c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 --2- 172.21.15.133:0/465744103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdcf40d0770 0x7fdcf40c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/465744103 >> 172.21.15.133:0/465744103 conn(0x7fdcf40727c0 msgr2=0x7fdcf40b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/465744103 shutdown_connections 2024-04-25T10:15:00.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.679+0000 7fdcfc0fb640 1 -- 172.21.15.133:0/465744103 wait complete. 2024-04-25T10:15:00.697 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:00.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.803+0000 7f7fdbf24640 1 Processor -- start 2024-04-25T10:15:00.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.803+0000 7f7fdbf24640 1 -- start start 2024-04-25T10:15:00.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fdbf24640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7fd4161cb0 0x7f7fd406dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fdbf24640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fd40cd5e0 0x7f7fd4162db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fdbf24640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd406e640 con 0x7f7fd40cc860 2024-04-25T10:15:00.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fdbf24640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd406ec00 con 0x7f7fd4161cb0 2024-04-25T10:15:00.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fdbf24640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd406e920 con 0x7f7fd40cd5e0 2024-04-25T10:15:00.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fd9cc0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7fd4161cb0 0x7f7fd406dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fd9cc0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7fd4161cb0 0x7f7fd406dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47476/0 (socket says 172.21.15.133:47476) 2024-04-25T10:15:00.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fd9cc0640 1 -- 172.21.15.133:0/1233155228 learned_addr learned my addr 172.21.15.133:0/1233155228 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:00.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fda4c1640 1 --2- 172.21.15.133:0/1233155228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fd40cd5e0 0x7f7fd4162db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fd9cc0640 1 -- 172.21.15.133:0/1233155228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fd40cd5e0 msgr2=0x7f7fd4162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fd9cc0640 1 --2- 172.21.15.133:0/1233155228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fd40cd5e0 0x7f7fd4162db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fd9cc0640 1 -- 172.21.15.133:0/1233155228 >> v1:172.21.15.162:6789/0 conn(0x7f7fd40cc860 legacy=0x7f7fd40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fd9cc0640 1 -- 172.21.15.133:0/1233155228 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd41625b0 con 0x7f7fd4161cb0 2024-04-25T10:15:00.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fd9cc0640 1 --2- 172.21.15.133:0/1233155228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fd4161cb0 0x7f7fd406dd60 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009e50 tx=0x7f7fc401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda93c1a8cb0529b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fd8cbe640 1 -- 172.21.15.133:0/1233155228 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40a5770 con 0x7f7fd4161cb0 2024-04-25T10:15:00.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fd8cbe640 1 -- 172.21.15.133:0/1233155228 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc40a5910 con 0x7f7fd4161cb0 2024-04-25T10:15:00.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.807+0000 7f7fd8cbe640 1 -- 172.21.15.133:0/1233155228 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40a5ce0 con 0x7f7fd4161cb0 2024-04-25T10:15:00.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1233155228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fd4161cb0 msgr2=0x7f7fd406dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 --2- 172.21.15.133:0/1233155228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fd4161cb0 0x7f7fd406dd60 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009e50 tx=0x7f7fc401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1233155228 shutdown_connections 2024-04-25T10:15:00.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 --2- 172.21.15.133:0/1233155228 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fd40cd5e0 0x7f7fd4162db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 --2- 172.21.15.133:0/1233155228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fd4161cb0 0x7f7fd406dd60 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1233155228 >> 172.21.15.133:0/1233155228 conn(0x7f7fd4067000 msgr2=0x7f7fd406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1233155228 shutdown_connections 2024-04-25T10:15:00.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1233155228 wait complete. 2024-04-25T10:15:00.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 Processor -- start 2024-04-25T10:15:00.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 -- start start 2024-04-25T10:15:00.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fd40cc860 0x7f7fd414cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fd40cd5e0 0x7f7fd415d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fd4161cb0 0x7f7fd415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4163e90 con 0x7f7fd40cc860 2024-04-25T10:15:00.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4163d10 con 0x7f7fd4161cb0 2024-04-25T10:15:00.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4164010 con 0x7f7fd40cd5e0 2024-04-25T10:15:00.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd9cc0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fd40cc860 0x7f7fd414cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd9cc0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fd40cc860 0x7f7fd414cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47490/0 (socket says 172.21.15.133:47490) 2024-04-25T10:15:00.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd9cc0640 1 -- 172.21.15.133:0/1746712605 learned_addr learned my addr 172.21.15.133:0/1746712605 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:00.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd9cc0640 1 -- 172.21.15.133:0/1746712605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fd40cd5e0 msgr2=0x7f7fd415d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd9cc0640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fd40cd5e0 0x7f7fd415d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd9cc0640 1 -- 172.21.15.133:0/1746712605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fd4161cb0 msgr2=0x7f7fd415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd9cc0640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fd4161cb0 0x7f7fd415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd9cc0640 1 -- 172.21.15.133:0/1746712605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd415a520 con 0x7f7fd40cc860 2024-04-25T10:15:00.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd9cc0640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fd40cc860 0x7f7fd414cf60 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f7fc401b070 tx=0x7f7fc4019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd27fc640 1 -- 172.21.15.133:0/1746712605 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40b8470 con 0x7f7fd40cc860 2024-04-25T10:15:00.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd27fc640 1 -- 172.21.15.133:0/1746712605 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc40b8610 con 0x7f7fd40cc860 2024-04-25T10:15:00.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fd27fc640 1 -- 172.21.15.133:0/1746712605 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40b89e0 con 0x7f7fd40cc860 2024-04-25T10:15:00.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1746712605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd415a800 con 0x7f7fd40cc860 2024-04-25T10:15:00.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.811+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1746712605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd415adc0 con 0x7f7fd40cc860 2024-04-25T10:15:00.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.815+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1746712605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd40b2730 con 0x7f7fd40cc860 2024-04-25T10:15:00.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.815+0000 7f7fd27fc640 1 -- 172.21.15.133:0/1746712605 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fc40c0030 con 0x7f7fd40cc860 2024-04-25T10:15:00.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.815+0000 7f7fd27fc640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fc0066f90 0x7f7fc0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.815+0000 7f7fd94bf640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fc0066f90 0x7f7fc0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.815+0000 7f7fd27fc640 1 -- 172.21.15.133:0/1746712605 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7fc411fb10 con 0x7f7fd40cc860 2024-04-25T10:15:00.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.815+0000 7f7fd94bf640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fc0066f90 0x7f7fc0069450 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7fd41fe580 tx=0x7f7fc80079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.819+0000 7f7fd27fc640 1 -- 172.21.15.133:0/1746712605 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7fc40c7060 con 0x7f7fd40cc860 2024-04-25T10:15:00.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.967+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1746712605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fd4068ef0 con 0x7f7fd40cc860 2024-04-25T10:15:00.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fd27fc640 1 -- 172.21.15.133:0/1746712605 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+17074 (secure 0 0 0) 0x7f7fc40165e0 con 0x7f7fd40cc860 2024-04-25T10:15:00.972 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:00.973 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":145,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:14:34.721848+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:14:26.340459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:14:26.996481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.28","mappings":[{"from":3,"to":0}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:00.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1746712605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fc0066f90 msgr2=0x7f7fc0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fc0066f90 0x7f7fc0069450 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7fd41fe580 tx=0x7f7fc80079c0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1746712605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fd40cc860 msgr2=0x7f7fd414cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fd40cc860 0x7f7fd414cf60 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f7fc401b070 tx=0x7f7fc4019760 comp rx=0 tx=0).stop 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1746712605 shutdown_connections 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fd4161cb0 0x7f7fd415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7fd40cd5e0 0x7f7fd415d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7fc0066f90 0x7f7fc0069450 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 --2- 172.21.15.133:0/1746712605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fd40cc860 0x7f7fd414cf60 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1746712605 >> 172.21.15.133:0/1746712605 conn(0x7f7fd4067000 msgr2=0x7f7fd4160700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1746712605 shutdown_connections 2024-04-25T10:15:00.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:00.971+0000 7f7fdbf24640 1 -- 172.21.15.133:0/1746712605 wait complete. 2024-04-25T10:15:00.991 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:15:01.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3bab52d640 1 Processor -- start 2024-04-25T10:15:01.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3bab52d640 1 -- start start 2024-04-25T10:15:01.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3bab52d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ba40b2b70 0x7f3ba406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3bab52d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba406f4a0 con 0x7f3ba40acbd0 2024-04-25T10:15:01.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3bab52d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba40b2fd0 con 0x7f3ba4146f90 2024-04-25T10:15:01.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3bab52d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba40725e0 con 0x7f3ba40b2b70 2024-04-25T10:15:01.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3ba8ac8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ba40b2b70 0x7f3ba406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3ba9aca640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3ba40acbd0 0x7f3ba40ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36144/0 (socket says 172.21.15.133:36144) 2024-04-25T10:15:01.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3ba9aca640 1 -- 172.21.15.133:0/2333555752 learned_addr learned my addr 172.21.15.133:0/2333555752 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:01.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3ba8ac8640 1 -- 172.21.15.133:0/2333555752 >> v1:172.21.15.162:6789/0 conn(0x7f3ba4146f90 legacy=0x7f3ba406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3ba8ac8640 1 -- 172.21.15.133:0/2333555752 >> v1:172.21.15.133:6789/0 conn(0x7f3ba40acbd0 legacy=0x7f3ba40ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.111+0000 7f3ba8ac8640 1 -- 172.21.15.133:0/2333555752 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba406e6e0 con 0x7f3ba40b2b70 2024-04-25T10:15:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3ba8ac8640 1 --2- 172.21.15.133:0/2333555752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ba40b2b70 0x7f3ba406dd40 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0097c0 tx=0x7f3b9c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bf499b01f88fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3b97fff640 1 -- 172.21.15.133:0/2333555752 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0b0070 con 0x7f3ba40b2b70 2024-04-25T10:15:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3b97fff640 1 -- 172.21.15.133:0/2333555752 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c0ab600 con 0x7f3ba40b2b70 2024-04-25T10:15:01.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3b97fff640 1 -- 172.21.15.133:0/2333555752 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0ab9d0 con 0x7f3ba40b2b70 2024-04-25T10:15:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3bab52d640 1 -- 172.21.15.133:0/2333555752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ba40b2b70 msgr2=0x7f3ba406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3bab52d640 1 --2- 172.21.15.133:0/2333555752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ba40b2b70 0x7f3ba406dd40 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0097c0 tx=0x7f3b9c01c330 comp rx=0 tx=0).stop 2024-04-25T10:15:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3bab52d640 1 -- 172.21.15.133:0/2333555752 shutdown_connections 2024-04-25T10:15:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3bab52d640 1 --2- 172.21.15.133:0/2333555752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ba40b2b70 0x7f3ba406dd40 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3bab52d640 1 -- 172.21.15.133:0/2333555752 >> 172.21.15.133:0/2333555752 conn(0x7f3ba40b15d0 msgr2=0x7f3ba40b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3bab52d640 1 -- 172.21.15.133:0/2333555752 shutdown_connections 2024-04-25T10:15:01.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3bab52d640 1 -- 172.21.15.133:0/2333555752 wait complete. 2024-04-25T10:15:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3bab52d640 1 Processor -- start 2024-04-25T10:15:01.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.115+0000 7f3bab52d640 1 -- start start 2024-04-25T10:15:01.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3bab52d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ba40acbd0 0x7f3ba4145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3bab52d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ba40b2b70 0x7f3ba4131d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3bab52d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba4146f90 0x7f3ba4132240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3bab52d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba406ef80 con 0x7f3ba40b2b70 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3bab52d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba406eca0 con 0x7f3ba4146f90 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3bab52d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba4072800 con 0x7f3ba40acbd0 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba8ac8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ba40b2b70 0x7f3ba4131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba8ac8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ba40b2b70 0x7f3ba4131d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47520/0 (socket says 172.21.15.133:47520) 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba8ac8640 1 -- 172.21.15.133:0/3255484753 learned_addr learned my addr 172.21.15.133:0/3255484753 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba9aca640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba4146f90 0x7f3ba4132240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba92c9640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ba40acbd0 0x7f3ba4145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba8ac8640 1 -- 172.21.15.133:0/3255484753 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ba40acbd0 msgr2=0x7f3ba4145fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba8ac8640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ba40acbd0 0x7f3ba4145fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba8ac8640 1 -- 172.21.15.133:0/3255484753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba4146f90 msgr2=0x7f3ba4132240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba8ac8640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba4146f90 0x7f3ba4132240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba8ac8640 1 -- 172.21.15.133:0/3255484753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba415ee40 con 0x7f3ba40b2b70 2024-04-25T10:15:01.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba9aca640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba4146f90 0x7f3ba4132240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:01.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba8ac8640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ba40b2b70 0x7f3ba4131d00 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c018f40 tx=0x7f3b9c0190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3b95ffb640 1 -- 172.21.15.133:0/3255484753 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0b0040 con 0x7f3ba40b2b70 2024-04-25T10:15:01.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3b95ffb640 1 -- 172.21.15.133:0/3255484753 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c0ba070 con 0x7f3ba40b2b70 2024-04-25T10:15:01.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3bab52d640 1 -- 172.21.15.133:0/3255484753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba415f120 con 0x7f3ba40b2b70 2024-04-25T10:15:01.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3b95ffb640 1 -- 172.21.15.133:0/3255484753 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0aac30 con 0x7f3ba40b2b70 2024-04-25T10:15:01.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3bab52d640 1 -- 172.21.15.133:0/3255484753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba415f6e0 con 0x7f3ba40b2b70 2024-04-25T10:15:01.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3bab52d640 1 -- 172.21.15.133:0/3255484753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba4067c70 con 0x7f3ba40b2b70 2024-04-25T10:15:01.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3b95ffb640 1 -- 172.21.15.133:0/3255484753 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b9c0ab420 con 0x7f3ba40b2b70 2024-04-25T10:15:01.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3b95ffb640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b88066f90 0x7f3b88069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba92c9640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b88066f90 0x7f3b88069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3b95ffb640 1 -- 172.21.15.133:0/3255484753 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3b9c11f350 con 0x7f3ba40b2b70 2024-04-25T10:15:01.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.119+0000 7f3ba92c9640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b88066f90 0x7f3b88069450 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3b98001c20 tx=0x7f3b98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.123+0000 7f3b95ffb640 1 -- 172.21.15.133:0/3255484753 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b9c015fb0 con 0x7f3ba40b2b70 2024-04-25T10:15:01.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.319+0000 7f3bab52d640 1 -- 172.21.15.133:0/3255484753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3ba406c5f0 con 0x7f3ba40b2b70 2024-04-25T10:15:01.323 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:15:01.319+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:15:01.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.323+0000 7f3b95ffb640 1 -- 172.21.15.133:0/3255484753 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3b9c0f0230 con 0x7f3ba40b2b70 2024-04-25T10:15:01.327 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:15:01.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 -- 172.21.15.133:0/3255484753 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b88066f90 msgr2=0x7f3b88069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b88066f90 0x7f3b88069450 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3b98001c20 tx=0x7f3b98009040 comp rx=0 tx=0).stop 2024-04-25T10:15:01.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 -- 172.21.15.133:0/3255484753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ba40b2b70 msgr2=0x7f3ba4131d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ba40b2b70 0x7f3ba4131d00 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c018f40 tx=0x7f3b9c0190d0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 -- 172.21.15.133:0/3255484753 shutdown_connections 2024-04-25T10:15:01.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3ba4146f90 0x7f3ba4132240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3b88066f90 0x7f3b88069450 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ba40b2b70 0x7f3ba4131d00 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 --2- 172.21.15.133:0/3255484753 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3ba40acbd0 0x7f3ba4145fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 -- 172.21.15.133:0/3255484753 >> 172.21.15.133:0/3255484753 conn(0x7f3ba40b15d0 msgr2=0x7f3ba406d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 -- 172.21.15.133:0/3255484753 shutdown_connections 2024-04-25T10:15:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.327+0000 7f3bab52d640 1 -- 172.21.15.133:0/3255484753 wait complete. 2024-04-25T10:15:01.357 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:15:01.378 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:15:01.379 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:15:01.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.467+0000 7f1f84618640 1 Processor -- start 2024-04-25T10:15:01.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.467+0000 7f1f84618640 1 -- start start 2024-04-25T10:15:01.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.467+0000 7f1f84618640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1f7c15e100 0x7f1f7c140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.467+0000 7f1f84618640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c0cf280 con 0x7f1f7c143560 2024-04-25T10:15:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.467+0000 7f1f84618640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c06d440 con 0x7f1f7c142920 2024-04-25T10:15:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.467+0000 7f1f84618640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c06d720 con 0x7f1f7c15e100 2024-04-25T10:15:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.467+0000 7f1f81bb3640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1f7c143560 0x7f1f7c15d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36152/0 (socket says 172.21.15.133:36152) 2024-04-25T10:15:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.467+0000 7f1f81bb3640 1 -- 172.21.15.133:0/3613345155 learned_addr learned my addr 172.21.15.133:0/3613345155 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f82bb5640 1 --2- 172.21.15.133:0/3613345155 >> v2:172.21.15.133:3300/0 conn(0x7f1f7c15e100 0x7f1f7c140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f813b2640 1 -- 172.21.15.133:0/3613345155 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566013186 0 0) 0x7f1f7c0cf280 con 0x7f1f7c143560 2024-04-25T10:15:01.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f813b2640 1 -- 172.21.15.133:0/3613345155 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f64003450 con 0x7f1f7c143560 2024-04-25T10:15:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f82bb5640 1 -- 172.21.15.133:0/3613345155 >> v1:172.21.15.162:6789/0 conn(0x7f1f7c142920 legacy=0x7f1f7c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f82bb5640 1 -- 172.21.15.133:0/3613345155 >> v1:172.21.15.133:6789/0 conn(0x7f1f7c143560 legacy=0x7f1f7c15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f82bb5640 1 -- 172.21.15.133:0/3613345155 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c06da00 con 0x7f1f7c15e100 2024-04-25T10:15:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f82bb5640 1 --2- 172.21.15.133:0/3613345155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f7c15e100 0x7f1f7c140a40 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009d90 tx=0x7f1f6c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaeecb8a3b59420 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f813b2640 1 -- 172.21.15.133:0/3613345155 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c0b2070 con 0x7f1f7c15e100 2024-04-25T10:15:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f813b2640 1 -- 172.21.15.133:0/3613345155 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c0a57f0 con 0x7f1f7c15e100 2024-04-25T10:15:01.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f813b2640 1 -- 172.21.15.133:0/3613345155 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c0a5bc0 con 0x7f1f7c15e100 2024-04-25T10:15:01.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f84618640 1 -- 172.21.15.133:0/3613345155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f7c15e100 msgr2=0x7f1f7c140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f84618640 1 --2- 172.21.15.133:0/3613345155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f7c15e100 0x7f1f7c140a40 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009d90 tx=0x7f1f6c01c8a0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f84618640 1 -- 172.21.15.133:0/3613345155 shutdown_connections 2024-04-25T10:15:01.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f84618640 1 --2- 172.21.15.133:0/3613345155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f7c15e100 0x7f1f7c140a40 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f84618640 1 -- 172.21.15.133:0/3613345155 >> 172.21.15.133:0/3613345155 conn(0x7f1f7c066f70 msgr2=0x7f1f7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f84618640 1 -- 172.21.15.133:0/3613345155 shutdown_connections 2024-04-25T10:15:01.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.471+0000 7f1f84618640 1 -- 172.21.15.133:0/3613345155 wait complete. 2024-04-25T10:15:01.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f84618640 1 Processor -- start 2024-04-25T10:15:01.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f84618640 1 -- start start 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f84618640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f7c142920 0x7f1f7c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f84618640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f7c143560 0x7f1f7c0b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f84618640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f7c15e100 0x7f1f7c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f84618640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c06eb40 con 0x7f1f7c15e100 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f84618640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c06e9c0 con 0x7f1f7c142920 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f84618640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c143940 con 0x7f1f7c143560 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f82bb5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f7c15e100 0x7f1f7c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f82bb5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f7c15e100 0x7f1f7c0b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47568/0 (socket says 172.21.15.133:47568) 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f82bb5640 1 -- 172.21.15.133:0/961417017 learned_addr learned my addr 172.21.15.133:0/961417017 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f82bb5640 1 -- 172.21.15.133:0/961417017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f7c143560 msgr2=0x7f1f7c0b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f82bb5640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f7c143560 0x7f1f7c0b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f82bb5640 1 -- 172.21.15.133:0/961417017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f7c142920 msgr2=0x7f1f7c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f82bb5640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f7c142920 0x7f1f7c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f82bb5640 1 -- 172.21.15.133:0/961417017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c1373c0 con 0x7f1f7c15e100 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f82bb5640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f7c15e100 0x7f1f7c0b2a90 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0b0040 tx=0x7f1f6c019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f7affd640 1 -- 172.21.15.133:0/961417017 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c0b2040 con 0x7f1f7c15e100 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f7affd640 1 -- 172.21.15.133:0/961417017 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c0bf070 con 0x7f1f7c15e100 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f84618640 1 -- 172.21.15.133:0/961417017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c0b1280 con 0x7f1f7c15e100 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f7affd640 1 -- 172.21.15.133:0/961417017 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c009450 con 0x7f1f7c15e100 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f84618640 1 -- 172.21.15.133:0/961417017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c0b1840 con 0x7f1f7c15e100 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f84618640 1 -- 172.21.15.133:0/961417017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c068e80 con 0x7f1f7c15e100 2024-04-25T10:15:01.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f7affd640 1 -- 172.21.15.133:0/961417017 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f6c0ada30 con 0x7f1f7c15e100 2024-04-25T10:15:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f7affd640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f68066f60 0x7f1f68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f7affd640 1 -- 172.21.15.133:0/961417017 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f1f6c1202b0 con 0x7f1f7c15e100 2024-04-25T10:15:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.475+0000 7f1f823b4640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f68066f60 0x7f1f68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.479+0000 7f1f823b4640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f68066f60 0x7f1f68069420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1f74003cf0 tx=0x7f1f740079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.479+0000 7f1f7affd640 1 -- 172.21.15.133:0/961417017 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f7c1373c0 con 0x7f1f7c15e100 2024-04-25T10:15:01.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:01.675+0000 7f1f84618640 1 -- 172.21.15.133:0/961417017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1f7c0af170 con 0x7f1f7c15e100 2024-04-25T10:15:02.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.327+0000 7f1f7affd640 1 -- 172.21.15.133:0/961417017 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v95) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1f6c016520 con 0x7f1f7c15e100 2024-04-25T10:15:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 -- 172.21.15.133:0/961417017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f68066f60 msgr2=0x7f1f68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f68066f60 0x7f1f68069420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1f74003cf0 tx=0x7f1f740079c0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 -- 172.21.15.133:0/961417017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f7c15e100 msgr2=0x7f1f7c0b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f7c15e100 0x7f1f7c0b2a90 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0b0040 tx=0x7f1f6c019640 comp rx=0 tx=0).stop 2024-04-25T10:15:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 -- 172.21.15.133:0/961417017 shutdown_connections 2024-04-25T10:15:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1f68066f60 0x7f1f68069420 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f7c15e100 0x7f1f7c0b2a90 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f7c143560 0x7f1f7c0b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 --2- 172.21.15.133:0/961417017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f7c142920 0x7f1f7c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 -- 172.21.15.133:0/961417017 >> 172.21.15.133:0/961417017 conn(0x7f1f7c066f70 msgr2=0x7f1f7c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 -- 172.21.15.133:0/961417017 shutdown_connections 2024-04-25T10:15:02.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.331+0000 7f1f84618640 1 -- 172.21.15.133:0/961417017 wait complete. 2024-04-25T10:15:02.357 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:15:02.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.527+0000 7f37c5104640 1 Processor -- start 2024-04-25T10:15:02.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.527+0000 7f37c5104640 1 -- start start 2024-04-25T10:15:02.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.527+0000 7f37c5104640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f37c015b910 0x7f37c00ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.527+0000 7f37c5104640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f37c00cebd0 0x7f37c015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.527+0000 7f37c5104640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c015bf40 con 0x7f37c015b530 2024-04-25T10:15:02.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.527+0000 7f37c5104640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c01589c0 con 0x7f37c00cebd0 2024-04-25T10:15:02.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.527+0000 7f37c5104640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c015c200 con 0x7f37c015b910 2024-04-25T10:15:02.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.527+0000 7f37bf59e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f37c00cebd0 0x7f37c015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.531+0000 7f37bf59e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f37c00cebd0 0x7f37c015cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47584/0 (socket says 172.21.15.133:47584) 2024-04-25T10:15:02.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.531+0000 7f37bf59e640 1 -- 172.21.15.133:0/2936563903 learned_addr learned my addr 172.21.15.133:0/2936563903 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:02.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.531+0000 7f37be59c640 1 --2- 172.21.15.133:0/2936563903 >> v2:172.21.15.162:3300/0 conn(0x7f37c015b910 0x7f37c00ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.531+0000 7f37bf59e640 1 -- 172.21.15.133:0/2936563903 >> v2:172.21.15.162:3300/0 conn(0x7f37c015b910 msgr2=0x7f37c00ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.531+0000 7f37bf59e640 1 --2- 172.21.15.133:0/2936563903 >> v2:172.21.15.162:3300/0 conn(0x7f37c015b910 0x7f37c00ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.531+0000 7f37bf59e640 1 -- 172.21.15.133:0/2936563903 >> v1:172.21.15.162:6789/0 conn(0x7f37c015b530 legacy=0x7f37c00cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.531+0000 7f37bf59e640 1 -- 172.21.15.133:0/2936563903 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c015c4e0 con 0x7f37c00cebd0 2024-04-25T10:15:02.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bf59e640 1 --2- 172.21.15.133:0/2936563903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37c00cebd0 0x7f37c015cfc0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f37c006e370 tx=0x7f37b801eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec3f66aebc1d364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bdd9b640 1 -- 172.21.15.133:0/2936563903 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800b2d0 con 0x7f37c00cebd0 2024-04-25T10:15:02.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bdd9b640 1 -- 172.21.15.133:0/2936563903 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b800b470 con 0x7f37c00cebd0 2024-04-25T10:15:02.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bdd9b640 1 -- 172.21.15.133:0/2936563903 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b80a7a10 con 0x7f37c00cebd0 2024-04-25T10:15:02.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 -- 172.21.15.133:0/2936563903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37c00cebd0 msgr2=0x7f37c015cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 --2- 172.21.15.133:0/2936563903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37c00cebd0 0x7f37c015cfc0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f37c006e370 tx=0x7f37b801eb20 comp rx=0 tx=0).stop 2024-04-25T10:15:02.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 -- 172.21.15.133:0/2936563903 shutdown_connections 2024-04-25T10:15:02.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 --2- 172.21.15.133:0/2936563903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37c00cebd0 0x7f37c015cfc0 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 --2- 172.21.15.133:0/2936563903 >> v2:172.21.15.162:3300/0 conn(0x7f37c015b910 0x7f37c00ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 -- 172.21.15.133:0/2936563903 >> 172.21.15.133:0/2936563903 conn(0x7f37c006d370 msgr2=0x7f37c006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 -- 172.21.15.133:0/2936563903 shutdown_connections 2024-04-25T10:15:02.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 -- 172.21.15.133:0/2936563903 wait complete. 2024-04-25T10:15:02.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 Processor -- start 2024-04-25T10:15:02.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 -- start start 2024-04-25T10:15:02.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37c00cebd0 0x7f37c0141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37c015b530 0x7f37c00b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f37c015b910 0x7f37c01428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c015de90 con 0x7f37c00cebd0 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c015dd10 con 0x7f37c015b530 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37c5104640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c015e010 con 0x7f37c015b910 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37c00cebd0 0x7f37c0141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37c00cebd0 0x7f37c0141d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47588/0 (socket says 172.21.15.133:47588) 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bed9d640 1 -- 172.21.15.133:0/483543861 learned_addr learned my addr 172.21.15.133:0/483543861 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bed9d640 1 -- 172.21.15.133:0/483543861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f37c015b910 msgr2=0x7f37c01428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bed9d640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f37c015b910 0x7f37c01428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37be59c640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37c015b530 0x7f37c00b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bed9d640 1 -- 172.21.15.133:0/483543861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37c015b530 msgr2=0x7f37c00b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bed9d640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37c015b530 0x7f37c00b44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bed9d640 1 -- 172.21.15.133:0/483543861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c00b4df0 con 0x7f37c00cebd0 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.535+0000 7f37bed9d640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37c00cebd0 0x7f37c0141d80 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f37b000b440 tx=0x7f37b000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.539+0000 7f37a77fe640 1 -- 172.21.15.133:0/483543861 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0015020 con 0x7f37c00cebd0 2024-04-25T10:15:02.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.539+0000 7f37a77fe640 1 -- 172.21.15.133:0/483543861 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b0011890 con 0x7f37c00cebd0 2024-04-25T10:15:02.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.539+0000 7f37a77fe640 1 -- 172.21.15.133:0/483543861 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0011c80 con 0x7f37c00cebd0 2024-04-25T10:15:02.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.539+0000 7f37c5104640 1 -- 172.21.15.133:0/483543861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c00b50d0 con 0x7f37c00cebd0 2024-04-25T10:15:02.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.539+0000 7f37c5104640 1 -- 172.21.15.133:0/483543861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c0154d50 con 0x7f37c00cebd0 2024-04-25T10:15:02.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.539+0000 7f37c5104640 1 -- 172.21.15.133:0/483543861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c0069000 con 0x7f37c00cebd0 2024-04-25T10:15:02.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.539+0000 7f37a77fe640 1 -- 172.21.15.133:0/483543861 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37b0018020 con 0x7f37c00cebd0 2024-04-25T10:15:02.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.539+0000 7f37a77fe640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f37ac066f60 0x7f37ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.539+0000 7f37be59c640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f37ac066f60 0x7f37ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.539+0000 7f37a77fe640 1 -- 172.21.15.133:0/483543861 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f37b0085090 con 0x7f37c00cebd0 2024-04-25T10:15:02.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.539+0000 7f37be59c640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f37ac066f60 0x7f37ac069420 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f37c00b58c0 tx=0x7f37b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.543+0000 7f37a77fe640 1 -- 172.21.15.133:0/483543861 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37b001f030 con 0x7f37c00cebd0 2024-04-25T10:15:02.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:02.703+0000 7f37c5104640 1 -- 172.21.15.133:0/483543861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f37c00ad8f0 con 0x7f37c00cebd0 2024-04-25T10:15:03.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.335+0000 7f37a77fe640 1 -- 172.21.15.133:0/483543861 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) v1 ==== 181+0+0 (secure 0 0 0) 0x7f37b000d040 con 0x7f37c00cebd0 2024-04-25T10:15:03.340 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:15:03.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.339+0000 7f37c5104640 1 -- 172.21.15.133:0/483543861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f37ac066f60 msgr2=0x7f37ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.339+0000 7f37c5104640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f37ac066f60 0x7f37ac069420 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f37c00b58c0 tx=0x7f37b4008040 comp rx=0 tx=0).stop 2024-04-25T10:15:03.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.339+0000 7f37c5104640 1 -- 172.21.15.133:0/483543861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37c00cebd0 msgr2=0x7f37c0141d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.339+0000 7f37c5104640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37c00cebd0 0x7f37c0141d80 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f37b000b440 tx=0x7f37b000b900 comp rx=0 tx=0).stop 2024-04-25T10:15:03.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.339+0000 7f37c5104640 1 -- 172.21.15.133:0/483543861 shutdown_connections 2024-04-25T10:15:03.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.339+0000 7f37c5104640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f37c015b910 0x7f37c01428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.339+0000 7f37c5104640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37c015b530 0x7f37c00b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.339+0000 7f37c5104640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f37ac066f60 0x7f37ac069420 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.339+0000 7f37c5104640 1 --2- 172.21.15.133:0/483543861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37c00cebd0 0x7f37c0141d80 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.343+0000 7f37c5104640 1 -- 172.21.15.133:0/483543861 >> 172.21.15.133:0/483543861 conn(0x7f37c006d370 msgr2=0x7f37c00af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.343+0000 7f37c5104640 1 -- 172.21.15.133:0/483543861 shutdown_connections 2024-04-25T10:15:03.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.343+0000 7f37c5104640 1 -- 172.21.15.133:0/483543861 wait complete. 2024-04-25T10:15:03.366 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:15:03.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07eabfd640 1 Processor -- start 2024-04-25T10:15:03.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07eabfd640 1 -- start start 2024-04-25T10:15:03.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07eabfd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e41407f0 0x7f07e40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07eabfd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e415edd0 con 0x7f07e40b2d40 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07eabfd640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e40b47d0 con 0x7f07e4140bd0 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07eabfd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b4ab0 con 0x7f07e41407f0 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e919a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f07e40b2d40 0x7f07e415e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36158/0 (socket says 172.21.15.133:36158) 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e919a640 1 -- 172.21.15.133:0/3056581650 learned_addr learned my addr 172.21.15.133:0/3056581650 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e8999640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e41407f0 0x7f07e40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330026731 0 0) 0x7f07e415edd0 con 0x7f07e40b2d40 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07cc003530 con 0x7f07e40b2d40 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1867038461 0 0) 0x7f07cc003530 con 0x7f07e40b2d40 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e41407f0 msgr2=0x7f07e40b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e41407f0 0x7f07e40b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 >> v1:172.21.15.162:6789/0 conn(0x7f07e4140bd0 legacy=0x7f07e40b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e40b4d90 con 0x7f07e40b2d40 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f07dc001a00 con 0x7f07e40b2d40 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 >> v1:172.21.15.133:6789/0 conn(0x7f07e40b2d40 legacy=0x7f07e415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e41407f0 0x7f07cc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07cc003f70 0x7f07cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc004d50 0x7f07cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b4ab0 con 0x7f07cc004d50 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc00bab0 con 0x7f07cc003f70 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc00bcb0 con 0x7f07e41407f0 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.483+0000 7f07e919a640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc004d50 0x7f07cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e3fff640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07cc003f70 0x7f07cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e8999640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e41407f0 0x7f07cc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e919a640 1 -- 172.21.15.133:0/3056581650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e41407f0 msgr2=0x7f07cc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e919a640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07e41407f0 0x7f07cc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e919a640 1 -- 172.21.15.133:0/3056581650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07cc003f70 msgr2=0x7f07cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e919a640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07cc003f70 0x7f07cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e919a640 1 -- 172.21.15.133:0/3056581650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc003530 con 0x7f07cc004d50 2024-04-25T10:15:03.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e919a640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc004d50 0x7f07cc0051d0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f07e4070c30 tx=0x7f07dc01d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43857210cdf5885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc00a480 con 0x7f07cc004d50 2024-04-25T10:15:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc0ad630 con 0x7f07cc004d50 2024-04-25T10:15:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e37fe640 1 -- 172.21.15.133:0/3056581650 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0ada60 con 0x7f07cc004d50 2024-04-25T10:15:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07eabfd640 1 -- 172.21.15.133:0/3056581650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc004d50 msgr2=0x7f07cc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07eabfd640 1 --2- 172.21.15.133:0/3056581650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc004d50 0x7f07cc0051d0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f07e4070c30 tx=0x7f07dc01d300 comp rx=0 tx=0).stop 2024-04-25T10:15:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07e919a640 1 -- 172.21.15.133:0/3056581650 reap_dead start 2024-04-25T10:15:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07eabfd640 1 -- 172.21.15.133:0/3056581650 shutdown_connections 2024-04-25T10:15:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07eabfd640 1 -- 172.21.15.133:0/3056581650 >> 172.21.15.133:0/3056581650 conn(0x7f07e406d370 msgr2=0x7f07e406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07eabfd640 1 -- 172.21.15.133:0/3056581650 shutdown_connections 2024-04-25T10:15:03.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.487+0000 7f07eabfd640 1 -- 172.21.15.133:0/3056581650 wait complete. 2024-04-25T10:15:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07eabfd640 1 Processor -- start 2024-04-25T10:15:03.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07eabfd640 1 -- start start 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07eabfd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07cc004d50 0x7f07e4134730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07eabfd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4138500 0x7f07e4138980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07eabfd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e415f140 0x7f07e415f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07eabfd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e415e930 con 0x7f07e4138500 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07eabfd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e415e7b0 con 0x7f07e415f140 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07eabfd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e415eab0 con 0x7f07cc004d50 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4138500 0x7f07e4138980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4138500 0x7f07e4138980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47616/0 (socket says 172.21.15.133:47616) 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e3fff640 1 -- 172.21.15.133:0/422754317 learned_addr learned my addr 172.21.15.133:0/422754317 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e8999640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07cc004d50 0x7f07e4134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e919a640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e415f140 0x7f07e415f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e3fff640 1 -- 172.21.15.133:0/422754317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07cc004d50 msgr2=0x7f07e4134730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e3fff640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07cc004d50 0x7f07e4134730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e3fff640 1 -- 172.21.15.133:0/422754317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e415f140 msgr2=0x7f07e415f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e3fff640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e415f140 0x7f07e415f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e3fff640 1 -- 172.21.15.133:0/422754317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e4136600 con 0x7f07e4138500 2024-04-25T10:15:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e3fff640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4138500 0x7f07e4138980 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f07d800e550 tx=0x7f07d800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e17fa640 1 -- 172.21.15.133:0/422754317 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80052e0 con 0x7f07e4138500 2024-04-25T10:15:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e17fa640 1 -- 172.21.15.133:0/422754317 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d8018070 con 0x7f07e4138500 2024-04-25T10:15:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07eabfd640 1 -- 172.21.15.133:0/422754317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e4137a60 con 0x7f07e4138500 2024-04-25T10:15:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e17fa640 1 -- 172.21.15.133:0/422754317 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8003820 con 0x7f07e4138500 2024-04-25T10:15:03.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07eabfd640 1 -- 172.21.15.133:0/422754317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e415fc70 con 0x7f07e4138500 2024-04-25T10:15:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07eabfd640 1 -- 172.21.15.133:0/422754317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e4069000 con 0x7f07e4138500 2024-04-25T10:15:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e17fa640 1 -- 172.21.15.133:0/422754317 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07d80039c0 con 0x7f07e4138500 2024-04-25T10:15:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.491+0000 7f07e17fa640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07d0066f90 0x7f07d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.495+0000 7f07e17fa640 1 -- 172.21.15.133:0/422754317 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5807+0+0 (secure 0 0 0) 0x7f07d80848e0 con 0x7f07e4138500 2024-04-25T10:15:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.495+0000 7f07e8999640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07d0066f90 0x7f07d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.495+0000 7f07e8999640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07d0066f90 0x7f07d0069450 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009ae0 tx=0x7f07d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.499+0000 7f07e17fa640 1 -- 172.21.15.133:0/422754317 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07e4136600 con 0x7f07e4138500 2024-04-25T10:15:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:03.659+0000 7f07eabfd640 1 -- 172.21.15.133:0/422754317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f07e40acaf0 con 0x7f07e4138500 2024-04-25T10:15:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.347+0000 7f07e17fa640 1 -- 172.21.15.133:0/422754317 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) v1 ==== 169+0+0 (secure 0 0 0) 0x7f07d8055890 con 0x7f07e4138500 2024-04-25T10:15:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:15:04.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.351+0000 7f07eabfd640 1 -- 172.21.15.133:0/422754317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07d0066f90 msgr2=0x7f07d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.351+0000 7f07eabfd640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07d0066f90 0x7f07d0069450 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009ae0 tx=0x7f07d4008040 comp rx=0 tx=0).stop 2024-04-25T10:15:04.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.351+0000 7f07eabfd640 1 -- 172.21.15.133:0/422754317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4138500 msgr2=0x7f07e4138980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.351+0000 7f07eabfd640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4138500 0x7f07e4138980 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f07d800e550 tx=0x7f07d800ea10 comp rx=0 tx=0).stop 2024-04-25T10:15:04.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.351+0000 7f07eabfd640 1 -- 172.21.15.133:0/422754317 shutdown_connections 2024-04-25T10:15:04.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.351+0000 7f07eabfd640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f07e415f140 0x7f07e415f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.351+0000 7f07eabfd640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f07d0066f90 0x7f07d0069450 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.351+0000 7f07eabfd640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07e4138500 0x7f07e4138980 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.351+0000 7f07eabfd640 1 --2- 172.21.15.133:0/422754317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f07cc004d50 0x7f07e4134730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.351+0000 7f07eabfd640 1 -- 172.21.15.133:0/422754317 >> 172.21.15.133:0/422754317 conn(0x7f07e406d370 msgr2=0x7f07e40add80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.355+0000 7f07eabfd640 1 -- 172.21.15.133:0/422754317 shutdown_connections 2024-04-25T10:15:04.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.355+0000 7f07eabfd640 1 -- 172.21.15.133:0/422754317 wait complete. 2024-04-25T10:15:04.379 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:15:04.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.507+0000 7fa1f8bd0640 1 Processor -- start 2024-04-25T10:15:04.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.507+0000 7fa1f8bd0640 1 -- start start 2024-04-25T10:15:04.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f8bd0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1f40cd820 0x7fa1f40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f8bd0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa1f415ced0 0x7fa1f415d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f8bd0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f40ce2e0 con 0x7fa1f4145170 2024-04-25T10:15:04.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f8bd0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f406db00 con 0x7fa1f415ced0 2024-04-25T10:15:04.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f8bd0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f4145550 con 0x7fa1f40cd820 2024-04-25T10:15:04.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f259c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa1f4145170 0x7fa1f406df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36172/0 (socket says 172.21.15.133:36172) 2024-04-25T10:15:04.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f259c640 1 -- 172.21.15.133:0/19470812 learned_addr learned my addr 172.21.15.133:0/19470812 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:04.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f2d9d640 1 --2- 172.21.15.133:0/19470812 >> v2:172.21.15.133:3300/0 conn(0x7fa1f415ced0 0x7fa1f415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f159a640 1 -- 172.21.15.133:0/19470812 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966850900 0 0) 0x7fa1f40ce2e0 con 0x7fa1f4145170 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f159a640 1 -- 172.21.15.133:0/19470812 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1dc003530 con 0x7fa1f4145170 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f1d9b640 1 --2- 172.21.15.133:0/19470812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1f40cd820 0x7fa1f40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f2d9d640 1 -- 172.21.15.133:0/19470812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1f40cd820 msgr2=0x7fa1f40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f2d9d640 1 --2- 172.21.15.133:0/19470812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1f40cd820 0x7fa1f40cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f2d9d640 1 -- 172.21.15.133:0/19470812 >> v1:172.21.15.133:6789/0 conn(0x7fa1f4145170 legacy=0x7fa1f406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f2d9d640 1 -- 172.21.15.133:0/19470812 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f406d4b0 con 0x7fa1f415ced0 2024-04-25T10:15:04.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f2d9d640 1 --2- 172.21.15.133:0/19470812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1f415ced0 0x7fa1f415d360 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009d90 tx=0x7fa1ec020d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5037d5cbc28c89da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f159a640 1 -- 172.21.15.133:0/19470812 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0b6070 con 0x7fa1f415ced0 2024-04-25T10:15:04.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f159a640 1 -- 172.21.15.133:0/19470812 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec0a9c50 con 0x7fa1f415ced0 2024-04-25T10:15:04.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.511+0000 7fa1f159a640 1 -- 172.21.15.133:0/19470812 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0b1440 con 0x7fa1f415ced0 2024-04-25T10:15:04.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/19470812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1f415ced0 msgr2=0x7fa1f415d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 --2- 172.21.15.133:0/19470812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1f415ced0 0x7fa1f415d360 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009d90 tx=0x7fa1ec020d80 comp rx=0 tx=0).stop 2024-04-25T10:15:04.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/19470812 shutdown_connections 2024-04-25T10:15:04.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 --2- 172.21.15.133:0/19470812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1f415ced0 0x7fa1f415d360 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 --2- 172.21.15.133:0/19470812 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1f40cd820 0x7fa1f40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/19470812 >> 172.21.15.133:0/19470812 conn(0x7fa1f4066f70 msgr2=0x7fa1f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/19470812 shutdown_connections 2024-04-25T10:15:04.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/19470812 wait complete. 2024-04-25T10:15:04.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 Processor -- start 2024-04-25T10:15:04.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 -- start start 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1f40cd820 0x7fa1f40ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1f4145170 0x7fa1f4152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1f415ced0 0x7fa1f4153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40b1e50 con 0x7fa1f4145170 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40b1cd0 con 0x7fa1f40cd820 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f8bd0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40b1fd0 con 0x7fa1f415ced0 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1f4145170 0x7fa1f4152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f1d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1f4145170 0x7fa1f4152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47636/0 (socket says 172.21.15.133:47636) 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f1d9b640 1 -- 172.21.15.133:0/1328445625 learned_addr learned my addr 172.21.15.133:0/1328445625 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f2d9d640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1f415ced0 0x7fa1f4153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f259c640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1f40cd820 0x7fa1f40ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f1d9b640 1 -- 172.21.15.133:0/1328445625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1f415ced0 msgr2=0x7fa1f4153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f1d9b640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1f415ced0 0x7fa1f4153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f1d9b640 1 -- 172.21.15.133:0/1328445625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1f40cd820 msgr2=0x7fa1f40ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f1d9b640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1f40cd820 0x7fa1f40ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f1d9b640 1 -- 172.21.15.133:0/1328445625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f41566d0 con 0x7fa1f4145170 2024-04-25T10:15:04.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.515+0000 7fa1f259c640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1f40cd820 0x7fa1f40ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:04.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1f1d9b640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1f4145170 0x7fa1f4152e30 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800d830 tx=0x7fa1e800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1e2ffd640 1 -- 172.21.15.133:0/1328445625 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800b6c0 con 0x7fa1f4145170 2024-04-25T10:15:04.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1e2ffd640 1 -- 172.21.15.133:0/1328445625 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e8019070 con 0x7fa1f4145170 2024-04-25T10:15:04.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1e2ffd640 1 -- 172.21.15.133:0/1328445625 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800ba60 con 0x7fa1f4145170 2024-04-25T10:15:04.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/1328445625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f41539f0 con 0x7fa1f4145170 2024-04-25T10:15:04.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/1328445625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f4154350 con 0x7fa1f4145170 2024-04-25T10:15:04.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/1328445625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f4068e80 con 0x7fa1f4145170 2024-04-25T10:15:04.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1e2ffd640 1 -- 172.21.15.133:0/1328445625 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1e801e460 con 0x7fa1f4145170 2024-04-25T10:15:04.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1e2ffd640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1d4066f60 0x7fa1d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1e2ffd640 1 -- 172.21.15.133:0/1328445625 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa1e8085510 con 0x7fa1f4145170 2024-04-25T10:15:04.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1f259c640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1d4066f60 0x7fa1d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.519+0000 7fa1f259c640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1d4066f60 0x7fa1d4069420 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4005890 tx=0x7fa1e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.523+0000 7fa1e2ffd640 1 -- 172.21.15.133:0/1328445625 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1f41566d0 con 0x7fa1f4145170 2024-04-25T10:15:04.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.683+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/1328445625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa1f406c2d0 con 0x7fa1f4145170 2024-04-25T10:15:04.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.687+0000 7fa1e2ffd640 1 -- 172.21.15.133:0/1328445625 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa1e8056690 con 0x7fa1f4145170 2024-04-25T10:15:04.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/1328445625 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1d4066f60 msgr2=0x7fa1d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1d4066f60 0x7fa1d4069420 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4005890 tx=0x7fa1e4009040 comp rx=0 tx=0).stop 2024-04-25T10:15:04.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/1328445625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1f4145170 msgr2=0x7fa1f4152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1f4145170 0x7fa1f4152e30 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800d830 tx=0x7fa1e800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/1328445625 shutdown_connections 2024-04-25T10:15:04.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa1f415ced0 0x7fa1f4153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa1d4066f60 0x7fa1d4069420 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1f4145170 0x7fa1f4152e30 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 --2- 172.21.15.133:0/1328445625 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa1f40cd820 0x7fa1f40ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/1328445625 >> 172.21.15.133:0/1328445625 conn(0x7fa1f4066f70 msgr2=0x7fa1f4144300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/1328445625 shutdown_connections 2024-04-25T10:15:04.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.691+0000 7fa1f8bd0640 1 -- 172.21.15.133:0/1328445625 wait complete. 2024-04-25T10:15:04.711 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:15:04.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f27015c4640 1 Processor -- start 2024-04-25T10:15:04.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f27015c4640 1 -- start start 2024-04-25T10:15:04.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f27015c4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f26fc06eb70 0x7f26fc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f27015c4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26fc1584f0 0x7f26fc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f27015c4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26fc0cbe60 con 0x7f26fc158110 2024-04-25T10:15:04.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f27015c4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26fc0b19b0 con 0x7f26fc06eb70 2024-04-25T10:15:04.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f27015c4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc15b360 con 0x7f26fc1584f0 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26faffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f26fc158110 0x7f26fc06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36186/0 (socket says 172.21.15.133:36186) 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26faffd640 1 -- 172.21.15.133:0/2878086097 learned_addr learned my addr 172.21.15.133:0/2878086097 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fb7fe640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26fc1584f0 0x7f26fc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26f3fff640 1 --2- 172.21.15.133:0/2878086097 >> v2:172.21.15.162:3300/0 conn(0x7f26fc06eb70 0x7f26fc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009032855 0 0) 0x7f26fc0cbe60 con 0x7f26fc158110 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e8003530 con 0x7f26fc158110 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2636775470 0 0) 0x7f26e8003530 con 0x7f26fc158110 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26fc1584f0 msgr2=0x7f26fc15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26fc1584f0 0x7f26fc15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 >> v2:172.21.15.162:3300/0 conn(0x7f26fc06eb70 msgr2=0x7f26fc0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 --2- 172.21.15.133:0/2878086097 >> v2:172.21.15.162:3300/0 conn(0x7f26fc06eb70 0x7f26fc0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc15b640 con 0x7f26fc158110 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f26e4001680 con 0x7f26fc158110 2024-04-25T10:15:04.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 >> v1:172.21.15.133:6789/0 conn(0x7f26fc158110 legacy=0x7f26fc06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26fc1584f0 0x7f26e80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e8003f40 0x7f26e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26e8004d50 0x7f26e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0b19b0 con 0x7f26e8004d50 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc15b360 con 0x7f26e8003f40 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e800b9d0 con 0x7f26fc1584f0 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fb7fe640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26e8004d50 0x7f26e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26f3fff640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e8003f40 0x7f26e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26faffd640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26fc1584f0 0x7f26e80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fb7fe640 1 -- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26fc1584f0 msgr2=0x7f26e80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fb7fe640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26fc1584f0 0x7f26e80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fb7fe640 1 -- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e8003f40 msgr2=0x7f26e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fb7fe640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26e8003f40 0x7f26e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26fb7fe640 1 -- 172.21.15.133:0/2878086097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8003530 con 0x7f26e8004d50 2024-04-25T10:15:04.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.823+0000 7f26faffd640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26fc1584f0 0x7f26e80039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:04.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f26fb7fe640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26e8004d50 0x7f26e80051d0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009f90 tx=0x7f26f401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283e6e2314686bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40b2070 con 0x7f26e8004d50 2024-04-25T10:15:04.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f40a5c50 con 0x7f26e8004d50 2024-04-25T10:15:04.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f26fa7fc640 1 -- 172.21.15.133:0/2878086097 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4009c10 con 0x7f26e8004d50 2024-04-25T10:15:04.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 -- 172.21.15.133:0/2878086097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26e8004d50 msgr2=0x7f26e80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 --2- 172.21.15.133:0/2878086097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26e8004d50 0x7f26e80051d0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f26f4009f90 tx=0x7f26f401cbe0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f26fb7fe640 1 -- 172.21.15.133:0/2878086097 reap_dead start 2024-04-25T10:15:04.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 -- 172.21.15.133:0/2878086097 shutdown_connections 2024-04-25T10:15:04.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 -- 172.21.15.133:0/2878086097 >> 172.21.15.133:0/2878086097 conn(0x7f26fc067000 msgr2=0x7f26fc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 -- 172.21.15.133:0/2878086097 shutdown_connections 2024-04-25T10:15:04.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 -- 172.21.15.133:0/2878086097 wait complete. 2024-04-25T10:15:04.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 Processor -- start 2024-04-25T10:15:04.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 -- start start 2024-04-25T10:15:04.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26e8004d50 0x7f26fc149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26fc158f90 0x7f26fc15d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26fc155590 0x7f26fc155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0cd670 con 0x7f26fc158f90 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc15bc00 con 0x7f26fc155590 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f27015c4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0cd7f0 con 0x7f26e8004d50 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f26f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26fc158f90 0x7f26fc15d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f26f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26fc158f90 0x7f26fc15d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47652/0 (socket says 172.21.15.133:47652) 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f26f3fff640 1 -- 172.21.15.133:0/3305075048 learned_addr learned my addr 172.21.15.133:0/3305075048 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f26faffd640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26e8004d50 0x7f26fc149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f26fb7fe640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26fc155590 0x7f26fc155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.827+0000 7f26f3fff640 1 -- 172.21.15.133:0/3305075048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26e8004d50 msgr2=0x7f26fc149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26f3fff640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26e8004d50 0x7f26fc149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26f3fff640 1 -- 172.21.15.133:0/3305075048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26fc155590 msgr2=0x7f26fc155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26f3fff640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26fc155590 0x7f26fc155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26f3fff640 1 -- 172.21.15.133:0/3305075048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc154530 con 0x7f26fc158f90 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26fb7fe640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26fc155590 0x7f26fc155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26f3fff640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26fc158f90 0x7f26fc15d7a0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f26ec011600 tx=0x7f26ec011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26f37fe640 1 -- 172.21.15.133:0/3305075048 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec004220 con 0x7f26fc158f90 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26f37fe640 1 -- 172.21.15.133:0/3305075048 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec0043c0 con 0x7f26fc158f90 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26f37fe640 1 -- 172.21.15.133:0/3305075048 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0158b0 con 0x7f26fc158f90 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f27015c4640 1 -- 172.21.15.133:0/3305075048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26fc1547d0 con 0x7f26fc158f90 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f27015c4640 1 -- 172.21.15.133:0/3305075048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc154d90 con 0x7f26fc158f90 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f27015c4640 1 -- 172.21.15.133:0/3305075048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26fc06bc20 con 0x7f26fc158f90 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26f37fe640 1 -- 172.21.15.133:0/3305075048 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26ec015a50 con 0x7f26fc158f90 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26f37fe640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f26dc066f90 0x7f26dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26faffd640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f26dc066f90 0x7f26dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26f37fe640 1 -- 172.21.15.133:0/3305075048 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f26ec085460 con 0x7f26fc158f90 2024-04-25T10:15:04.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.831+0000 7f26faffd640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f26dc066f90 0x7f26dc069450 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f26e4001ec0 tx=0x7f26e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:04.835+0000 7f26f37fe640 1 -- 172.21.15.133:0/3305075048 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26ec019030 con 0x7f26fc158f90 2024-04-25T10:15:05.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.055+0000 7f27015c4640 1 -- 172.21.15.133:0/3305075048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f26fc068ef0 con 0x7f26fc158f90 2024-04-25T10:15:05.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.347+0000 7f26f37fe640 1 -- 172.21.15.133:0/3305075048 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v594) v1 ==== 159+0+0 (secure 0 0 0) 0x7f26ec056670 con 0x7f26fc158f90 2024-04-25T10:15:05.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 -- 172.21.15.133:0/3305075048 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f26dc066f90 msgr2=0x7f26dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f26dc066f90 0x7f26dc069450 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f26e4001ec0 tx=0x7f26e4009040 comp rx=0 tx=0).stop 2024-04-25T10:15:05.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 -- 172.21.15.133:0/3305075048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26fc158f90 msgr2=0x7f26fc15d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26fc158f90 0x7f26fc15d7a0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f26ec011600 tx=0x7f26ec011ac0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 -- 172.21.15.133:0/3305075048 shutdown_connections 2024-04-25T10:15:05.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26fc155590 0x7f26fc155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f26dc066f90 0x7f26dc069450 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26fc158f90 0x7f26fc15d7a0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 --2- 172.21.15.133:0/3305075048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f26e8004d50 0x7f26fc149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 -- 172.21.15.133:0/3305075048 >> 172.21.15.133:0/3305075048 conn(0x7f26fc067000 msgr2=0x7f26fc0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:05.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 -- 172.21.15.133:0/3305075048 shutdown_connections 2024-04-25T10:15:05.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.351+0000 7f27015c4640 1 -- 172.21.15.133:0/3305075048 wait complete. 2024-04-25T10:15:05.376 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:15:05.376 INFO:tasks.cephfs_test_runner:Starting test: test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:15:05.377 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:05.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.495+0000 7efdf9eec640 1 Processor -- start 2024-04-25T10:15:05.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.495+0000 7efdf9eec640 1 -- start start 2024-04-25T10:15:05.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf9eec640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efdf415c9a0 0x7efdf40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf9eec640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf40c61d0 con 0x7efdf40b1f70 2024-04-25T10:15:05.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf9eec640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf40c64b0 con 0x7efdf40ce330 2024-04-25T10:15:05.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf9eec640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf406d4f0 con 0x7efdf415c9a0 2024-04-25T10:15:05.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf2ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7efdf40b1f70 0x7efdf406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36194/0 (socket says 172.21.15.133:36194) 2024-04-25T10:15:05.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf2ffd640 1 -- 172.21.15.133:0/600557673 learned_addr learned my addr 172.21.15.133:0/600557673 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:05.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621170824 0 0) 0x7efdf40c61d0 con 0x7efdf40b1f70 2024-04-25T10:15:05.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efddc003530 con 0x7efdf40b1f70 2024-04-25T10:15:05.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf37fe640 1 --2- 172.21.15.133:0/600557673 >> v2:172.21.15.162:3300/0 conn(0x7efdf415c9a0 0x7efdf40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1074364652 0 0) 0x7efddc003530 con 0x7efdf40b1f70 2024-04-25T10:15:05.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 >> v2:172.21.15.162:3300/0 conn(0x7efdf415c9a0 msgr2=0x7efdf40c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:05.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 --2- 172.21.15.133:0/600557673 >> v2:172.21.15.162:3300/0 conn(0x7efdf415c9a0 0x7efdf40c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 >> v1:172.21.15.162:6789/0 conn(0x7efdf40ce330 legacy=0x7efdf415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf406d7d0 con 0x7efdf40b1f70 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7efde8001680 con 0x7efdf40b1f70 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 >> v1:172.21.15.133:6789/0 conn(0x7efdf40b1f70 legacy=0x7efdf406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efddc0039c0 0x7efddc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efddc004810 0x7efddc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efddc0051d0 0x7efddc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf406d4f0 con 0x7efddc004810 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc00c120 con 0x7efddc0051d0 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc00c320 con 0x7efddc0039c0 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf2ffd640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efddc004810 0x7efddc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf37fe640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efddc0039c0 0x7efddc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf3fff640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efddc0051d0 0x7efddc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf2ffd640 1 -- 172.21.15.133:0/600557673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efddc0039c0 msgr2=0x7efddc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf2ffd640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efddc0039c0 0x7efddc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf2ffd640 1 -- 172.21.15.133:0/600557673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efddc0051d0 msgr2=0x7efddc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf2ffd640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efddc0051d0 0x7efddc00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf2ffd640 1 -- 172.21.15.133:0/600557673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc003530 con 0x7efddc004810 2024-04-25T10:15:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf3fff640 1 -- 172.21.15.133:0/600557673 reap_dead start 2024-04-25T10:15:05.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.499+0000 7efdf37fe640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efddc0039c0 0x7efddc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf2ffd640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efddc004810 0x7efddc004c90 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7efde800c240 tx=0x7efde801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6b17ec4e366379 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8007d20 con 0x7efddc004810 2024-04-25T10:15:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde8005ae0 con 0x7efddc004810 2024-04-25T10:15:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf27fc640 1 -- 172.21.15.133:0/600557673 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8007070 con 0x7efddc004810 2024-04-25T10:15:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 -- 172.21.15.133:0/600557673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efddc004810 msgr2=0x7efddc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efddc004810 0x7efddc004c90 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7efde800c240 tx=0x7efde801f3f0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 -- 172.21.15.133:0/600557673 shutdown_connections 2024-04-25T10:15:05.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 --2- 172.21.15.133:0/600557673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efddc004810 0x7efddc004c90 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 -- 172.21.15.133:0/600557673 >> 172.21.15.133:0/600557673 conn(0x7efdf4067000 msgr2=0x7efdf406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:05.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 -- 172.21.15.133:0/600557673 shutdown_connections 2024-04-25T10:15:05.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 -- 172.21.15.133:0/600557673 wait complete. 2024-04-25T10:15:05.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 Processor -- start 2024-04-25T10:15:05.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 -- start start 2024-04-25T10:15:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efddc004810 0x7efdf40c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf4158460 0x7efdf40caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdf41588c0 0x7efdf4141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40ce780 con 0x7efdf4158460 2024-04-25T10:15:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40ce900 con 0x7efddc004810 2024-04-25T10:15:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40cea80 con 0x7efdf41588c0 2024-04-25T10:15:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf4158460 0x7efdf40caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf4158460 0x7efdf40caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47678/0 (socket says 172.21.15.133:47678) 2024-04-25T10:15:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf2ffd640 1 -- 172.21.15.133:0/3205449553 learned_addr learned my addr 172.21.15.133:0/3205449553 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf3fff640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdf41588c0 0x7efdf4141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf37fe640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efddc004810 0x7efdf40c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf2ffd640 1 -- 172.21.15.133:0/3205449553 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdf41588c0 msgr2=0x7efdf4141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf2ffd640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdf41588c0 0x7efdf4141640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf2ffd640 1 -- 172.21.15.133:0/3205449553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efddc004810 msgr2=0x7efdf40c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf2ffd640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efddc004810 0x7efdf40c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf2ffd640 1 -- 172.21.15.133:0/3205449553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf40c86e0 con 0x7efdf4158460 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf37fe640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efddc004810 0x7efdf40c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf2ffd640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf4158460 0x7efdf40caa00 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7efde801f220 tx=0x7efddc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efddbfff640 1 -- 172.21.15.133:0/3205449553 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80049f0 con 0x7efdf4158460 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efddbfff640 1 -- 172.21.15.133:0/3205449553 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde80b6070 con 0x7efdf4158460 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 -- 172.21.15.133:0/3205449553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf41521d0 con 0x7efdf4158460 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efddbfff640 1 -- 172.21.15.133:0/3205449553 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8004db0 con 0x7efdf4158460 2024-04-25T10:15:05.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.503+0000 7efdf9eec640 1 -- 172.21.15.133:0/3205449553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf4152790 con 0x7efdf4158460 2024-04-25T10:15:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.507+0000 7efdf9eec640 1 -- 172.21.15.133:0/3205449553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf4068ed0 con 0x7efdf4158460 2024-04-25T10:15:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.507+0000 7efddbfff640 1 -- 172.21.15.133:0/3205449553 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efde80b8020 con 0x7efdf4158460 2024-04-25T10:15:05.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.507+0000 7efddbfff640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 0x7efde0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.507+0000 7efdf37fe640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 0x7efde0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.507+0000 7efddbfff640 1 -- 172.21.15.133:0/3205449553 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efde8122370 con 0x7efdf4158460 2024-04-25T10:15:05.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.507+0000 7efdf37fe640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 0x7efde0069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7efdf404f450 tx=0x7efde4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.511+0000 7efddbfff640 1 -- 172.21.15.133:0/3205449553 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efde8019100 con 0x7efdf4158460 2024-04-25T10:15:05.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.671+0000 7efdf9eec640 1 -- 172.21.15.133:0/3205449553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdf406db00 con 0x7efdf4158460 2024-04-25T10:15:05.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.671+0000 7efddbfff640 1 -- 172.21.15.133:0/3205449553 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+13329 (secure 0 0 0) 0x7efde80f3450 con 0x7efdf4158460 2024-04-25T10:15:05.674 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:05.675 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":148,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:15:04.342639+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/3234096063":"2024-04-26T10:15:01.325685+0000","172.21.15.133:6836/3234096063":"2024-04-26T10:15:01.325685+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:05.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.671+0000 7efdf9eec640 1 -- 172.21.15.133:0/3205449553 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 msgr2=0x7efde0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.671+0000 7efdf9eec640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 0x7efde0069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7efdf404f450 tx=0x7efde4008040 comp rx=0 tx=0).stop 2024-04-25T10:15:05.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.675+0000 7efdf9eec640 1 -- 172.21.15.133:0/3205449553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf4158460 msgr2=0x7efdf40caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.675+0000 7efdf9eec640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf4158460 0x7efdf40caa00 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7efde801f220 tx=0x7efddc005840 comp rx=0 tx=0).stop 2024-04-25T10:15:05.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.675+0000 7efdf9eec640 1 -- 172.21.15.133:0/3205449553 shutdown_connections 2024-04-25T10:15:05.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.675+0000 7efdf9eec640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efdf41588c0 0x7efdf4141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.675+0000 7efdf9eec640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efde0066f60 0x7efde0069420 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.675+0000 7efdf9eec640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf4158460 0x7efdf40caa00 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.675+0000 7efdf9eec640 1 --2- 172.21.15.133:0/3205449553 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efddc004810 0x7efdf40c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.675+0000 7efdf9eec640 1 -- 172.21.15.133:0/3205449553 >> 172.21.15.133:0/3205449553 conn(0x7efdf4067000 msgr2=0x7efdf415d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:05.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.675+0000 7efdf9eec640 1 -- 172.21.15.133:0/3205449553 shutdown_connections 2024-04-25T10:15:05.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.675+0000 7efdf9eec640 1 -- 172.21.15.133:0/3205449553 wait complete. 2024-04-25T10:15:05.694 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:05.694 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:05.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f2786d21640 1 Processor -- start 2024-04-25T10:15:05.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f2786d21640 1 -- start start 2024-04-25T10:15:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f2786d21640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2780143ea0 0x7f27800cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f2786d21640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f27800b2150 0x7f278006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f2786d21640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278015ac50 con 0x7f278015b9b0 2024-04-25T10:15:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f2786d21640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27800cd180 con 0x7f27800b2150 2024-04-25T10:15:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f2786d21640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278015aed0 con 0x7f2780143ea0 2024-04-25T10:15:05.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f277ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f27800b2150 0x7f278006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f27852be640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f278015b9b0 0x7f278015e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36196/0 (socket says 172.21.15.133:36196) 2024-04-25T10:15:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f27852be640 1 -- 172.21.15.133:0/1020019218 learned_addr learned my addr 172.21.15.133:0/1020019218 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f277ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f27800b2150 0x7f278006e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47688/0 (socket says 172.21.15.133:47688) 2024-04-25T10:15:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f2784abd640 1 --2- 172.21.15.133:0/1020019218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2780143ea0 0x7f27800cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f277f7fe640 1 -- 172.21.15.133:0/1020019218 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301302059 0 0) 0x7f27780014a0 con 0x7f278015b9b0 2024-04-25T10:15:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f277f7fe640 1 -- 172.21.15.133:0/1020019218 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2770003530 con 0x7f278015b9b0 2024-04-25T10:15:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f277ffff640 1 -- 172.21.15.133:0/1020019218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2780143ea0 msgr2=0x7f27800cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f277ffff640 1 --2- 172.21.15.133:0/1020019218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2780143ea0 0x7f27800cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f277ffff640 1 -- 172.21.15.133:0/1020019218 >> v1:172.21.15.133:6789/0 conn(0x7f278015b9b0 legacy=0x7f278015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.823+0000 7f277ffff640 1 -- 172.21.15.133:0/1020019218 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278015b1b0 con 0x7f27800b2150 2024-04-25T10:15:05.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f277ffff640 1 --2- 172.21.15.133:0/1020019218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27800b2150 0x7f278006e3b0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f276c0097c0 tx=0x7f276c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06b925e8b027390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f277f7fe640 1 -- 172.21.15.133:0/1020019218 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0b1070 con 0x7f27800b2150 2024-04-25T10:15:05.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f277f7fe640 1 -- 172.21.15.133:0/1020019218 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c0ac410 con 0x7f27800b2150 2024-04-25T10:15:05.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f277f7fe640 1 -- 172.21.15.133:0/1020019218 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0ac7e0 con 0x7f27800b2150 2024-04-25T10:15:05.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 -- 172.21.15.133:0/1020019218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27800b2150 msgr2=0x7f278006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 --2- 172.21.15.133:0/1020019218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27800b2150 0x7f278006e3b0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f276c0097c0 tx=0x7f276c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 -- 172.21.15.133:0/1020019218 shutdown_connections 2024-04-25T10:15:05.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 --2- 172.21.15.133:0/1020019218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27800b2150 0x7f278006e3b0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 --2- 172.21.15.133:0/1020019218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2780143ea0 0x7f27800cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 -- 172.21.15.133:0/1020019218 >> 172.21.15.133:0/1020019218 conn(0x7f2780066f70 msgr2=0x7f278006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:05.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 -- 172.21.15.133:0/1020019218 shutdown_connections 2024-04-25T10:15:05.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 -- 172.21.15.133:0/1020019218 wait complete. 2024-04-25T10:15:05.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 Processor -- start 2024-04-25T10:15:05.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 -- start start 2024-04-25T10:15:05.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27800b2150 0x7f2780151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2780143ea0 0x7f278013cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f278015b9b0 0x7f2780141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2780165750 con 0x7f278015b9b0 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27801655d0 con 0x7f2780143ea0 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.827+0000 7f2786d21640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27801658d0 con 0x7f27800b2150 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f27852be640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f278015b9b0 0x7f2780141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f27852be640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f278015b9b0 0x7f2780141890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47690/0 (socket says 172.21.15.133:47690) 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f27852be640 1 -- 172.21.15.133:0/2900832062 learned_addr learned my addr 172.21.15.133:0/2900832062 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f27852be640 1 -- 172.21.15.133:0/2900832062 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27800b2150 msgr2=0x7f2780151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f2784abd640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27800b2150 0x7f2780151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f277ffff640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2780143ea0 0x7f278013cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f27852be640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27800b2150 0x7f2780151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f27852be640 1 -- 172.21.15.133:0/2900832062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2780143ea0 msgr2=0x7f278013cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f27852be640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2780143ea0 0x7f278013cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f27852be640 1 -- 172.21.15.133:0/2900832062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2780142550 con 0x7f278015b9b0 2024-04-25T10:15:05.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f27852be640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f278015b9b0 0x7f2780141890 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f277800caa0 tx=0x7f277800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f277d7fa640 1 -- 172.21.15.133:0/2900832062 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778017070 con 0x7f278015b9b0 2024-04-25T10:15:05.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f277d7fa640 1 -- 172.21.15.133:0/2900832062 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277800ac00 con 0x7f278015b9b0 2024-04-25T10:15:05.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f2786d21640 1 -- 172.21.15.133:0/2900832062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2780142270 con 0x7f278015b9b0 2024-04-25T10:15:05.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f277d7fa640 1 -- 172.21.15.133:0/2900832062 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778012400 con 0x7f278015b9b0 2024-04-25T10:15:05.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f2786d21640 1 -- 172.21.15.133:0/2900832062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278013b020 con 0x7f278015b9b0 2024-04-25T10:15:05.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f2786d21640 1 -- 172.21.15.133:0/2900832062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278013a940 con 0x7f278015b9b0 2024-04-25T10:15:05.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f277d7fa640 1 -- 172.21.15.133:0/2900832062 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2778007070 con 0x7f278015b9b0 2024-04-25T10:15:05.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f277d7fa640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2764066f60 0x7f2764069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f277d7fa640 1 -- 172.21.15.133:0/2900832062 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2778084bf0 con 0x7f278015b9b0 2024-04-25T10:15:05.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f2784abd640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2764066f60 0x7f2764069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.831+0000 7f2784abd640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2764066f60 0x7f2764069420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2774009790 tx=0x7f2774008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.839+0000 7f277d7fa640 1 -- 172.21.15.133:0/2900832062 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f278013a940 con 0x7f278015b9b0 2024-04-25T10:15:05.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.995+0000 7f2786d21640 1 -- 172.21.15.133:0/2900832062 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2780068e80 con 0x7f278015b9b0 2024-04-25T10:15:06.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:05.995+0000 7f277d7fa640 1 -- 172.21.15.133:0/2900832062 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7f278015ac50 con 0x7f278015b9b0 2024-04-25T10:15:06.000 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:15:06.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 -- 172.21.15.133:0/2900832062 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2764066f60 msgr2=0x7f2764069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2764066f60 0x7f2764069420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2774009790 tx=0x7f2774008040 comp rx=0 tx=0).stop 2024-04-25T10:15:06.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 -- 172.21.15.133:0/2900832062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f278015b9b0 msgr2=0x7f2780141890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f278015b9b0 0x7f2780141890 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f277800caa0 tx=0x7f277800cf60 comp rx=0 tx=0).stop 2024-04-25T10:15:06.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 -- 172.21.15.133:0/2900832062 shutdown_connections 2024-04-25T10:15:06.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2764066f60 0x7f2764069420 secure :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2774009790 tx=0x7f2774008040 comp rx=0 tx=0).stop 2024-04-25T10:15:06.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f278015b9b0 0x7f2780141890 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2780143ea0 0x7f278013cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 --2- 172.21.15.133:0/2900832062 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f27800b2150 0x7f2780151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 -- 172.21.15.133:0/2900832062 >> 172.21.15.133:0/2900832062 conn(0x7f2780066f70 msgr2=0x7f278015d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:06.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 -- 172.21.15.133:0/2900832062 shutdown_connections 2024-04-25T10:15:06.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.003+0000 7f2786d21640 1 -- 172.21.15.133:0/2900832062 wait complete. 2024-04-25T10:15:06.028 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:15:06.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.139+0000 7fd08b718640 1 Processor -- start 2024-04-25T10:15:06.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.139+0000 7fd08b718640 1 -- start start 2024-04-25T10:15:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd08b718640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd08415c9a0 0x7fd0840c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd08b718640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0840ce330 0x7fd0840ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd08b718640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0840c61d0 con 0x7fd0840b1f70 2024-04-25T10:15:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd08b718640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0840cecc0 con 0x7fd0840ce330 2024-04-25T10:15:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd08b718640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0840c64b0 con 0x7fd08415c9a0 2024-04-25T10:15:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd089cb5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0840ce330 0x7fd0840ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd089cb5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0840ce330 0x7fd0840ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47702/0 (socket says 172.21.15.133:47702) 2024-04-25T10:15:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd089cb5640 1 -- 172.21.15.133:0/4123319582 learned_addr learned my addr 172.21.15.133:0/4123319582 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd0894b4640 1 --2- 172.21.15.133:0/4123319582 >> v2:172.21.15.162:3300/0 conn(0x7fd08415c9a0 0x7fd0840c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd089cb5640 1 -- 172.21.15.133:0/4123319582 >> v2:172.21.15.162:3300/0 conn(0x7fd08415c9a0 msgr2=0x7fd0840c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd089cb5640 1 --2- 172.21.15.133:0/4123319582 >> v2:172.21.15.162:3300/0 conn(0x7fd08415c9a0 0x7fd0840c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd089cb5640 1 -- 172.21.15.133:0/4123319582 >> v1:172.21.15.162:6789/0 conn(0x7fd0840b1f70 legacy=0x7fd08406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd089cb5640 1 -- 172.21.15.133:0/4123319582 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08406d4f0 con 0x7fd0840ce330 2024-04-25T10:15:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.143+0000 7fd089cb5640 1 --2- 172.21.15.133:0/4123319582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0840ce330 0x7fd0840ce780 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fd08406f5f0 tx=0x7fd07c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2789ea9b01253ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd07bfff640 1 -- 172.21.15.133:0/4123319582 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0b0070 con 0x7fd0840ce330 2024-04-25T10:15:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd07bfff640 1 -- 172.21.15.133:0/4123319582 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c0ab600 con 0x7fd0840ce330 2024-04-25T10:15:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd07bfff640 1 -- 172.21.15.133:0/4123319582 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0ab9d0 con 0x7fd0840ce330 2024-04-25T10:15:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 -- 172.21.15.133:0/4123319582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0840ce330 msgr2=0x7fd0840ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 --2- 172.21.15.133:0/4123319582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0840ce330 0x7fd0840ce780 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fd08406f5f0 tx=0x7fd07c01c520 comp rx=0 tx=0).stop 2024-04-25T10:15:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 -- 172.21.15.133:0/4123319582 shutdown_connections 2024-04-25T10:15:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 --2- 172.21.15.133:0/4123319582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0840ce330 0x7fd0840ce780 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 --2- 172.21.15.133:0/4123319582 >> v2:172.21.15.162:3300/0 conn(0x7fd08415c9a0 0x7fd0840c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 -- 172.21.15.133:0/4123319582 >> 172.21.15.133:0/4123319582 conn(0x7fd084067000 msgr2=0x7fd08406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 -- 172.21.15.133:0/4123319582 shutdown_connections 2024-04-25T10:15:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 -- 172.21.15.133:0/4123319582 wait complete. 2024-04-25T10:15:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 Processor -- start 2024-04-25T10:15:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 -- start start 2024-04-25T10:15:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0840b1f70 0x7fd084153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0840ce330 0x7fd084145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd08415c9a0 0x7fd084157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08415dad0 con 0x7fd08415c9a0 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08415d950 con 0x7fd0840b1f70 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08415dc50 con 0x7fd0840ce330 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd089cb5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd08415c9a0 0x7fd084157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd089cb5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd08415c9a0 0x7fd084157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47706/0 (socket says 172.21.15.133:47706) 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd089cb5640 1 -- 172.21.15.133:0/164683856 learned_addr learned my addr 172.21.15.133:0/164683856 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd089cb5640 1 -- 172.21.15.133:0/164683856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0840ce330 msgr2=0x7fd084145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd089cb5640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0840ce330 0x7fd084145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd089cb5640 1 -- 172.21.15.133:0/164683856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0840b1f70 msgr2=0x7fd084153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd089cb5640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0840b1f70 0x7fd084153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd089cb5640 1 -- 172.21.15.133:0/164683856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd084146610 con 0x7fd08415c9a0 2024-04-25T10:15:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd089cb5640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd08415c9a0 0x7fd084157110 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fd07c01ca00 tx=0x7fd07c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd079ffb640 1 -- 172.21.15.133:0/164683856 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0b0040 con 0x7fd08415c9a0 2024-04-25T10:15:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd079ffb640 1 -- 172.21.15.133:0/164683856 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c0b9070 con 0x7fd08415c9a0 2024-04-25T10:15:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 -- 172.21.15.133:0/164683856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0841468f0 con 0x7fd08415c9a0 2024-04-25T10:15:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd079ffb640 1 -- 172.21.15.133:0/164683856 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0b4670 con 0x7fd08415c9a0 2024-04-25T10:15:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.147+0000 7fd08b718640 1 -- 172.21.15.133:0/164683856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd084146eb0 con 0x7fd08415c9a0 2024-04-25T10:15:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.151+0000 7fd08b718640 1 -- 172.21.15.133:0/164683856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08406bc20 con 0x7fd08415c9a0 2024-04-25T10:15:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.151+0000 7fd079ffb640 1 -- 172.21.15.133:0/164683856 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd07c0b4810 con 0x7fd08415c9a0 2024-04-25T10:15:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.151+0000 7fd079ffb640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd068066f60 0x7fd068069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.151+0000 7fd079ffb640 1 -- 172.21.15.133:0/164683856 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd07c11ea10 con 0x7fd08415c9a0 2024-04-25T10:15:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.151+0000 7fd0894b4640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd068066f60 0x7fd068069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.151+0000 7fd0894b4640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd068066f60 0x7fd068069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd080009fa0 tx=0x7fd08000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.155+0000 7fd079ffb640 1 -- 172.21.15.133:0/164683856 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd07c016110 con 0x7fd08415c9a0 2024-04-25T10:15:06.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:06.391+0000 7fd08b718640 1 -- 172.21.15.133:0/164683856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7fd084147290 con 0x7fd08415c9a0 2024-04-25T10:15:07.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.359+0000 7fd079ffb640 1 -- 172.21.15.133:0/164683856 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v596) v1 ==== 143+0+0 (secure 0 0 0) 0x7fd084147290 con 0x7fd08415c9a0 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.359+0000 7fd08b718640 1 -- 172.21.15.133:0/164683856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd068066f60 msgr2=0x7fd068069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.359+0000 7fd08b718640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd068066f60 0x7fd068069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd080009fa0 tx=0x7fd08000b040 comp rx=0 tx=0).stop 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.359+0000 7fd08b718640 1 -- 172.21.15.133:0/164683856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd08415c9a0 msgr2=0x7fd084157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.363+0000 7fd08b718640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd08415c9a0 0x7fd084157110 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fd07c01ca00 tx=0x7fd07c0192c0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.363+0000 7fd08b718640 1 -- 172.21.15.133:0/164683856 shutdown_connections 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.363+0000 7fd08b718640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd068066f60 0x7fd068069420 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.363+0000 7fd08b718640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd08415c9a0 0x7fd084157110 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.363+0000 7fd08b718640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0840ce330 0x7fd084145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.363+0000 7fd08b718640 1 --2- 172.21.15.133:0/164683856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0840b1f70 0x7fd084153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.363+0000 7fd08b718640 1 -- 172.21.15.133:0/164683856 >> 172.21.15.133:0/164683856 conn(0x7fd084067000 msgr2=0x7fd08415b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.363+0000 7fd08b718640 1 -- 172.21.15.133:0/164683856 shutdown_connections 2024-04-25T10:15:07.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.363+0000 7fd08b718640 1 -- 172.21.15.133:0/164683856 wait complete. 2024-04-25T10:15:07.385 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:15:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b6dcc2640 1 Processor -- start 2024-04-25T10:15:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b6dcc2640 1 -- start start 2024-04-25T10:15:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b6dcc2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5b6815b8b0 0x7f5b680ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b6dcc2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5b6815bc90 0x7f5b680cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b6dcc2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680cc6a0 con 0x7f5b680d01d0 2024-04-25T10:15:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b6dcc2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b680d0600 con 0x7f5b6815b8b0 2024-04-25T10:15:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b6dcc2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b680cc980 con 0x7f5b6815bc90 2024-04-25T10:15:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b677fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5b6815b8b0 0x7f5b680ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b67fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5b680d01d0 0x7f5b6815f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38132/0 (socket says 172.21.15.133:38132) 2024-04-25T10:15:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b67fff640 1 -- 172.21.15.133:0/2194750505 learned_addr learned my addr 172.21.15.133:0/2194750505 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b66ffd640 1 --2- 172.21.15.133:0/2194750505 >> v2:172.21.15.162:3300/0 conn(0x7f5b6815bc90 0x7f5b680cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b677fe640 1 -- 172.21.15.133:0/2194750505 >> v2:172.21.15.162:3300/0 conn(0x7f5b6815bc90 msgr2=0x7f5b680cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b677fe640 1 --2- 172.21.15.133:0/2194750505 >> v2:172.21.15.162:3300/0 conn(0x7f5b6815bc90 0x7f5b680cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b677fe640 1 -- 172.21.15.133:0/2194750505 >> v1:172.21.15.133:6789/0 conn(0x7f5b680d01d0 legacy=0x7f5b6815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b677fe640 1 -- 172.21.15.133:0/2194750505 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680ccc60 con 0x7f5b6815b8b0 2024-04-25T10:15:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.511+0000 7f5b667fc640 1 -- 172.21.15.133:0/2194750505 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975466583 0 0) 0x7f5b680cc6a0 con 0x7f5b680d01d0 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b677fe640 1 --2- 172.21.15.133:0/2194750505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6815b8b0 0x7f5b680ce6d0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f5b6806ec50 tx=0x7f5b5c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaed2041c0109a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b667fc640 1 -- 172.21.15.133:0/2194750505 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c0b0070 con 0x7f5b6815b8b0 2024-04-25T10:15:07.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b667fc640 1 -- 172.21.15.133:0/2194750505 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c0a4e50 con 0x7f5b6815b8b0 2024-04-25T10:15:07.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b667fc640 1 -- 172.21.15.133:0/2194750505 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c0ab820 con 0x7f5b6815b8b0 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/2194750505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6815b8b0 msgr2=0x7f5b680ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b6dcc2640 1 --2- 172.21.15.133:0/2194750505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6815b8b0 0x7f5b680ce6d0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f5b6806ec50 tx=0x7f5b5c01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/2194750505 shutdown_connections 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b6dcc2640 1 --2- 172.21.15.133:0/2194750505 >> v2:172.21.15.162:3300/0 conn(0x7f5b6815bc90 0x7f5b680cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b6dcc2640 1 --2- 172.21.15.133:0/2194750505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6815b8b0 0x7f5b680ce6d0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/2194750505 >> 172.21.15.133:0/2194750505 conn(0x7f5b6809a990 msgr2=0x7f5b680d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/2194750505 shutdown_connections 2024-04-25T10:15:07.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.515+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/2194750505 wait complete. 2024-04-25T10:15:07.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b6dcc2640 1 Processor -- start 2024-04-25T10:15:07.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b6dcc2640 1 -- start start 2024-04-25T10:15:07.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b6dcc2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b680d01d0 0x7f5b680c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b6dcc2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6815b8b0 0x7f5b68158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b6dcc2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b6815bc90 0x7f5b68152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b6dcc2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680cc3b0 con 0x7f5b6815b8b0 2024-04-25T10:15:07.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b6dcc2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680cc230 con 0x7f5b6815bc90 2024-04-25T10:15:07.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b6dcc2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680b1ab0 con 0x7f5b680d01d0 2024-04-25T10:15:07.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b66ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6815b8b0 0x7f5b68158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b66ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6815b8b0 0x7f5b68158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37530/0 (socket says 172.21.15.133:37530) 2024-04-25T10:15:07.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b66ffd640 1 -- 172.21.15.133:0/4106383151 learned_addr learned my addr 172.21.15.133:0/4106383151 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:07.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b67fff640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b6815bc90 0x7f5b68152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b677fe640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b680d01d0 0x7f5b680c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b66ffd640 1 -- 172.21.15.133:0/4106383151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b680d01d0 msgr2=0x7f5b680c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b66ffd640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b680d01d0 0x7f5b680c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b66ffd640 1 -- 172.21.15.133:0/4106383151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b6815bc90 msgr2=0x7f5b68152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b66ffd640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b6815bc90 0x7f5b68152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b66ffd640 1 -- 172.21.15.133:0/4106383151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b68159400 con 0x7f5b6815b8b0 2024-04-25T10:15:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b677fe640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b680d01d0 0x7f5b680c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b66ffd640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6815b8b0 0x7f5b68158a90 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800d620 tx=0x7f5b5800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b4ffff640 1 -- 172.21.15.133:0/4106383151 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b580125f0 con 0x7f5b6815b8b0 2024-04-25T10:15:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b4ffff640 1 -- 172.21.15.133:0/4106383151 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b58019070 con 0x7f5b6815b8b0 2024-04-25T10:15:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/4106383151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b681596e0 con 0x7f5b6815b8b0 2024-04-25T10:15:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b4ffff640 1 -- 172.21.15.133:0/4106383151 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b580129b0 con 0x7f5b6815b8b0 2024-04-25T10:15:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/4106383151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b680c8300 con 0x7f5b6815b8b0 2024-04-25T10:15:07.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.519+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/4106383151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b68067c70 con 0x7f5b6815b8b0 2024-04-25T10:15:07.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.523+0000 7f5b4ffff640 1 -- 172.21.15.133:0/4106383151 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b58012b50 con 0x7f5b6815b8b0 2024-04-25T10:15:07.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.523+0000 7f5b4ffff640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b54066f60 0x7f5b54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.523+0000 7f5b677fe640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b54066f60 0x7f5b54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.523+0000 7f5b4ffff640 1 -- 172.21.15.133:0/4106383151 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5b58085220 con 0x7f5b6815b8b0 2024-04-25T10:15:07.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.523+0000 7f5b677fe640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b54066f60 0x7f5b54069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0ad000 tx=0x7f5b5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.527+0000 7f5b4ffff640 1 -- 172.21.15.133:0/4106383151 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b58056430 con 0x7f5b6815b8b0 2024-04-25T10:15:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.687+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/4106383151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5b680d1dc0 con 0x7f5b6815b8b0 2024-04-25T10:15:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b4ffff640 1 -- 172.21.15.133:0/4106383151 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5b58055d30 con 0x7f5b6815b8b0 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/4106383151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b54066f60 msgr2=0x7f5b54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b54066f60 0x7f5b54069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0ad000 tx=0x7f5b5c008040 comp rx=0 tx=0).stop 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/4106383151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6815b8b0 msgr2=0x7f5b68158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6815b8b0 0x7f5b68158a90 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800d620 tx=0x7f5b5800dae0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/4106383151 shutdown_connections 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5b6815bc90 0x7f5b68152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5b54066f60 0x7f5b54069420 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6815b8b0 0x7f5b68158a90 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 --2- 172.21.15.133:0/4106383151 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5b680d01d0 0x7f5b680c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/4106383151 >> 172.21.15.133:0/4106383151 conn(0x7f5b6809a990 msgr2=0x7f5b6806f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/4106383151 shutdown_connections 2024-04-25T10:15:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:07.691+0000 7f5b6dcc2640 1 -- 172.21.15.133:0/4106383151 wait complete. 2024-04-25T10:15:07.713 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:15:07.713 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:07.758 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:07.816 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:15:07.816 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:15:07.817 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:07.823 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:07.823 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:15:07.823 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:07.830 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:07.830 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:15:07.830 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:07.869 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:07.870 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:15:07.870 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:07.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:07.878 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:15:07.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:08.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e713b5640 1 Processor -- start 2024-04-25T10:15:08.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e713b5640 1 -- start start 2024-04-25T10:15:08.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e713b5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15c9a0 0x7f6e6c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e713b5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6e6c0ce330 0x7f6e6c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e713b5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c0c61d0 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e713b5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c0cece0 con 0x7f6e6c0ce330 2024-04-25T10:15:08.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e713b5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0c64b0 con 0x7f6e6c15c9a0 2024-04-25T10:15:08.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e6b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6e6c0ce330 0x7f6e6c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e6b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6e6c0ce330 0x7f6e6c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37544/0 (socket says 172.21.15.133:37544) 2024-04-25T10:15:08.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e6b7fe640 1 -- 172.21.15.133:0/2939183015 learned_addr learned my addr 172.21.15.133:0/2939183015 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e6affd640 1 --2- 172.21.15.133:0/2939183015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15c9a0 0x7f6e6c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e6b7fe640 1 -- 172.21.15.133:0/2939183015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15c9a0 msgr2=0x7f6e6c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e6b7fe640 1 --2- 172.21.15.133:0/2939183015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15c9a0 0x7f6e6c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e6b7fe640 1 -- 172.21.15.133:0/2939183015 >> v1:172.21.15.162:6789/0 conn(0x7f6e6c0b1f70 legacy=0x7f6e6c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.043+0000 7f6e6b7fe640 1 -- 172.21.15.133:0/2939183015 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c06d4f0 con 0x7f6e6c0ce330 2024-04-25T10:15:08.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6b7fe640 1 --2- 172.21.15.133:0/2939183015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c0ce330 0x7f6e6c0ce7a0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009e50 tx=0x7f6e6401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba65e76386df99a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e69ffb640 1 -- 172.21.15.133:0/2939183015 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640a5700 con 0x7f6e6c0ce330 2024-04-25T10:15:08.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e69ffb640 1 -- 172.21.15.133:0/2939183015 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e640a58a0 con 0x7f6e6c0ce330 2024-04-25T10:15:08.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e69ffb640 1 -- 172.21.15.133:0/2939183015 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640a5c70 con 0x7f6e6c0ce330 2024-04-25T10:15:08.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 -- 172.21.15.133:0/2939183015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c0ce330 msgr2=0x7f6e6c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 --2- 172.21.15.133:0/2939183015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c0ce330 0x7f6e6c0ce7a0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009e50 tx=0x7f6e6401ca30 comp rx=0 tx=0).stop 2024-04-25T10:15:08.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 -- 172.21.15.133:0/2939183015 shutdown_connections 2024-04-25T10:15:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 --2- 172.21.15.133:0/2939183015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c0ce330 0x7f6e6c0ce7a0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 --2- 172.21.15.133:0/2939183015 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15c9a0 0x7f6e6c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 -- 172.21.15.133:0/2939183015 >> 172.21.15.133:0/2939183015 conn(0x7f6e6c067000 msgr2=0x7f6e6c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 -- 172.21.15.133:0/2939183015 shutdown_connections 2024-04-25T10:15:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 -- 172.21.15.133:0/2939183015 wait complete. 2024-04-25T10:15:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 Processor -- start 2024-04-25T10:15:08.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 -- start start 2024-04-25T10:15:08.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c0b1f70 0x7f6e6c136f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e6c0ce330 0x7f6e6c0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15c9a0 0x7f6e6c0b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15dad0 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15d950 con 0x7f6e6c0ce330 2024-04-25T10:15:08.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15dc50 con 0x7f6e6c15c9a0 2024-04-25T10:15:08.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c0b1f70 0x7f6e6c136f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c0b1f70 0x7f6e6c136f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37552/0 (socket says 172.21.15.133:37552) 2024-04-25T10:15:08.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6affd640 1 -- 172.21.15.133:0/4282944831 learned_addr learned my addr 172.21.15.133:0/4282944831 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6a7fc640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e6c0ce330 0x7f6e6c0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6affd640 1 -- 172.21.15.133:0/4282944831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15c9a0 msgr2=0x7f6e6c0b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6b7fe640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15c9a0 0x7f6e6c0b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6affd640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15c9a0 0x7f6e6c0b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6affd640 1 -- 172.21.15.133:0/4282944831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e6c0ce330 msgr2=0x7f6e6c0b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6affd640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e6c0ce330 0x7f6e6c0b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6affd640 1 -- 172.21.15.133:0/4282944831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c0cd020 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6a7fc640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e6c0ce330 0x7f6e6c0b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e6affd640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c0b1f70 0x7f6e6c136f70 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00d830 tx=0x7f6e5c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e637fe640 1 -- 172.21.15.133:0/4282944831 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c00b6c0 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e637fe640 1 -- 172.21.15.133:0/4282944831 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c019070 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 -- 172.21.15.133:0/4282944831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c0cd2c0 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e637fe640 1 -- 172.21.15.133:0/4282944831 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c00ba60 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.047+0000 7f6e713b5640 1 -- 172.21.15.133:0/4282944831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c0cd880 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.051+0000 7f6e713b5640 1 -- 172.21.15.133:0/4282944831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c06bc20 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.051+0000 7f6e637fe640 1 -- 172.21.15.133:0/4282944831 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e5c01e460 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.051+0000 7f6e637fe640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 0x7f6e4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.051+0000 7f6e637fe640 1 -- 172.21.15.133:0/4282944831 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6e5c085510 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.051+0000 7f6e6a7fc640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 0x7f6e4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.051+0000 7f6e6a7fc640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 0x7f6e4c069420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0b4680 tx=0x7f6e54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.055+0000 7f6e637fe640 1 -- 172.21.15.133:0/4282944831 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e5c056720 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.251+0000 7f6e713b5640 1 -- 172.21.15.133:0/4282944831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e6c151680 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.251+0000 7f6e637fe640 1 -- 172.21.15.133:0/4282944831 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+3710 (secure 0 0 0) 0x7f6e5c056020 con 0x7f6e6c0b1f70 2024-04-25T10:15:08.254 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:08.255 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":95,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3448301688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3448301688},{"type":"v1","addr":"172.21.15.162:6833","nonce":3448301688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/4188704231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":4188704231},{"type":"v1","addr":"172.21.15.162:6835","nonce":4188704231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":10455,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/885581611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":885581611},{"type":"v1","addr":"172.21.15.133:6835","nonce":885581611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":10458,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2338331740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2338331740},{"type":"v1","addr":"172.21.15.133:6837","nonce":2338331740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[]} 2024-04-25T10:15:08.255 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 95 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 -- 172.21.15.133:0/4282944831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 msgr2=0x7f6e4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 0x7f6e4c069420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0b4680 tx=0x7f6e54009040 comp rx=0 tx=0).stop 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 -- 172.21.15.133:0/4282944831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c0b1f70 msgr2=0x7f6e6c136f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c0b1f70 0x7f6e6c136f70 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00d830 tx=0x7f6e5c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 -- 172.21.15.133:0/4282944831 shutdown_connections 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e6c15c9a0 0x7f6e6c0b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e6c0ce330 0x7f6e6c0b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e4c066f60 0x7f6e4c069420 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 --2- 172.21.15.133:0/4282944831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c0b1f70 0x7f6e6c136f70 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 -- 172.21.15.133:0/4282944831 >> 172.21.15.133:0/4282944831 conn(0x7f6e6c067000 msgr2=0x7f6e6c154860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 -- 172.21.15.133:0/4282944831 shutdown_connections 2024-04-25T10:15:08.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.255+0000 7f6e713b5640 1 -- 172.21.15.133:0/4282944831 wait complete. 2024-04-25T10:15:08.274 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:15:08.275 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:15:08.275 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:15:08.275 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:08.275 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:15:08.275 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:15:08.275 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:15:08.276 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:08.276 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:15:08.276 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:15:08.276 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:15:08.276 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:08.276 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:15:08.276 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:15:08.277 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:15:08.277 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:08.286 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:15:08.286 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:15:08.287 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:15:08.287 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:15:08.289 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:15:08.289 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:15:08.290 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:15:08.290 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:15:08.329 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:15:08.330 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:08.330 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:15:08.331 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:08.335 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:15:08.335 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:08.337 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:15:08.337 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:08.337 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:15:08.392 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:15:08.398 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:15:08.406 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:15:08.407 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:15:08.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.447+0000 7f51e5e25640 1 Processor -- start 2024-04-25T10:15:08.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.447+0000 7f51e5e25640 1 -- start start 2024-04-25T10:15:08.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.447+0000 7f51e5e25640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51e0099c30 0x7f51e009a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.447+0000 7f51e5e25640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f51e009a010 0x7f51e00b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.447+0000 7f51e5e25640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e00cd6b0 con 0x7f51e015ea60 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.447+0000 7f51e5e25640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e00cd0f0 con 0x7f51e0099c30 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.447+0000 7f51e5e25640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e00cd3d0 con 0x7f51e009a010 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.447+0000 7f51df7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51e0099c30 0x7f51e009a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.447+0000 7f51dffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f51e015ea60 0x7f51e015eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38158/0 (socket says 172.21.15.133:38158) 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.447+0000 7f51dffff640 1 -- 172.21.15.133:0/2013416498 learned_addr learned my addr 172.21.15.133:0/2013416498 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51deffd640 1 --2- 172.21.15.133:0/2013416498 >> v2:172.21.15.162:3300/0 conn(0x7f51e009a010 0x7f51e00b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51df7fe640 1 -- 172.21.15.133:0/2013416498 >> v2:172.21.15.162:3300/0 conn(0x7f51e009a010 msgr2=0x7f51e00b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51df7fe640 1 --2- 172.21.15.133:0/2013416498 >> v2:172.21.15.162:3300/0 conn(0x7f51e009a010 0x7f51e00b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51df7fe640 1 -- 172.21.15.133:0/2013416498 >> v1:172.21.15.133:6789/0 conn(0x7f51e015ea60 legacy=0x7f51e015eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51df7fe640 1 -- 172.21.15.133:0/2013416498 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e00cf360 con 0x7f51e0099c30 2024-04-25T10:15:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51de7fc640 1 -- 172.21.15.133:0/2013416498 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361249236 0 0) 0x7f51e00cd6b0 con 0x7f51e015ea60 2024-04-25T10:15:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51df7fe640 1 --2- 172.21.15.133:0/2013416498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e0099c30 0x7f51e009a710 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f51e009a4a0 tx=0x7f51d001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b8f21ef7611fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51de7fc640 1 -- 172.21.15.133:0/2013416498 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00b2070 con 0x7f51e0099c30 2024-04-25T10:15:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51de7fc640 1 -- 172.21.15.133:0/2013416498 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d00a57e0 con 0x7f51e0099c30 2024-04-25T10:15:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51de7fc640 1 -- 172.21.15.133:0/2013416498 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00a5bb0 con 0x7f51e0099c30 2024-04-25T10:15:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51e5e25640 1 -- 172.21.15.133:0/2013416498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e0099c30 msgr2=0x7f51e009a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51e5e25640 1 --2- 172.21.15.133:0/2013416498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e0099c30 0x7f51e009a710 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f51e009a4a0 tx=0x7f51d001c8a0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51e5e25640 1 -- 172.21.15.133:0/2013416498 shutdown_connections 2024-04-25T10:15:08.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51e5e25640 1 --2- 172.21.15.133:0/2013416498 >> v2:172.21.15.162:3300/0 conn(0x7f51e009a010 0x7f51e00b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51e5e25640 1 --2- 172.21.15.133:0/2013416498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e0099c30 0x7f51e009a710 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51e5e25640 1 -- 172.21.15.133:0/2013416498 >> 172.21.15.133:0/2013416498 conn(0x7f51e00b3170 msgr2=0x7f51e00b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:08.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51e5e25640 1 -- 172.21.15.133:0/2013416498 shutdown_connections 2024-04-25T10:15:08.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.451+0000 7f51e5e25640 1 -- 172.21.15.133:0/2013416498 wait complete. 2024-04-25T10:15:08.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51e5e25640 1 Processor -- start 2024-04-25T10:15:08.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51e5e25640 1 -- start start 2024-04-25T10:15:08.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51e5e25640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51e0099c30 0x7f51e0156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51e5e25640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e009a010 0x7f51e00ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51e5e25640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e015ea60 0x7f51e00ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51e5e25640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00b4b90 con 0x7f51e009a010 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51e5e25640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00b4a10 con 0x7f51e015ea60 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51e5e25640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00d0370 con 0x7f51e0099c30 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51deffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e009a010 0x7f51e00ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51deffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e009a010 0x7f51e00ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37598/0 (socket says 172.21.15.133:37598) 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51deffd640 1 -- 172.21.15.133:0/876250161 learned_addr learned my addr 172.21.15.133:0/876250161 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51df7fe640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51e0099c30 0x7f51e0156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51dffff640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e015ea60 0x7f51e00ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51deffd640 1 -- 172.21.15.133:0/876250161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51e0099c30 msgr2=0x7f51e0156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51deffd640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51e0099c30 0x7f51e0156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51deffd640 1 -- 172.21.15.133:0/876250161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e015ea60 msgr2=0x7f51e00ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51deffd640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e015ea60 0x7f51e00ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51deffd640 1 -- 172.21.15.133:0/876250161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e0159b10 con 0x7f51e009a010 2024-04-25T10:15:08.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51dffff640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e015ea60 0x7f51e00ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:08.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51deffd640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e009a010 0x7f51e00ca2f0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f51c800e930 tx=0x7f51c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51d7fff640 1 -- 172.21.15.133:0/876250161 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8017070 con 0x7f51e009a010 2024-04-25T10:15:08.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51d7fff640 1 -- 172.21.15.133:0/876250161 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c800ca00 con 0x7f51e009a010 2024-04-25T10:15:08.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51e5e25640 1 -- 172.21.15.133:0/876250161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e009eda0 con 0x7f51e009a010 2024-04-25T10:15:08.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51d7fff640 1 -- 172.21.15.133:0/876250161 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c800cdb0 con 0x7f51e009a010 2024-04-25T10:15:08.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51e5e25640 1 -- 172.21.15.133:0/876250161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e0159dd0 con 0x7f51e009a010 2024-04-25T10:15:08.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51e5e25640 1 -- 172.21.15.133:0/876250161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e0067c70 con 0x7f51e009a010 2024-04-25T10:15:08.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51d7fff640 1 -- 172.21.15.133:0/876250161 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51c801a070 con 0x7f51e009a010 2024-04-25T10:15:08.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51d7fff640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f60 0x7f51c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51d7fff640 1 -- 172.21.15.133:0/876250161 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f51c8084e40 con 0x7f51e009a010 2024-04-25T10:15:08.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.455+0000 7f51df7fe640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f60 0x7f51c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.459+0000 7f51df7fe640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f60 0x7f51c0069420 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f51d00af000 tx=0x7f51d00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.463+0000 7f51d7fff640 1 -- 172.21.15.133:0/876250161 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51c8055a30 con 0x7f51e009a010 2024-04-25T10:15:08.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:15:08.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:08.631+0000 7f51e5e25640 1 -- 172.21.15.133:0/876250161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f51e00b2300 con 0x7f51e009a010 2024-04-25T10:15:09.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.719+0000 7f51d7fff640 1 -- 172.21.15.133:0/876250161 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v153) v1 ==== 97+0+0 (secure 0 0 0) 0x7f51c8059ad0 con 0x7f51e009a010 2024-04-25T10:15:09.724 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:15:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 -- 172.21.15.133:0/876250161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f60 msgr2=0x7f51c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f60 0x7f51c0069420 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f51d00af000 tx=0x7f51d00a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 -- 172.21.15.133:0/876250161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e009a010 msgr2=0x7f51e00ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e009a010 0x7f51e00ca2f0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f51c800e930 tx=0x7f51c800edf0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 -- 172.21.15.133:0/876250161 shutdown_connections 2024-04-25T10:15:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51e015ea60 0x7f51e00ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f51c0066f60 0x7f51c0069420 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e009a010 0x7f51e00ca2f0 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 --2- 172.21.15.133:0/876250161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f51e0099c30 0x7f51e0156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 -- 172.21.15.133:0/876250161 >> 172.21.15.133:0/876250161 conn(0x7f51e00b3170 msgr2=0x7f51e0157370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 -- 172.21.15.133:0/876250161 shutdown_connections 2024-04-25T10:15:09.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.723+0000 7f51e5e25640 1 -- 172.21.15.133:0/876250161 wait complete. 2024-04-25T10:15:09.750 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:15:09.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.871+0000 7ff0ee719640 1 Processor -- start 2024-04-25T10:15:09.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.871+0000 7ff0ee719640 1 -- start start 2024-04-25T10:15:09.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0ee719640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e815c9a0 0x7ff0e80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0ee719640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff0e80b1f70 0x7ff0e806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0ee719640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e80c61d0 con 0x7ff0e80ce330 2024-04-25T10:15:09.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0ee719640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e80c64b0 con 0x7ff0e80b1f70 2024-04-25T10:15:09.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0ee719640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e806d4f0 con 0x7ff0e815c9a0 2024-04-25T10:15:09.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0eccb6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff0e80ce330 0x7ff0e815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38170/0 (socket says 172.21.15.133:38170) 2024-04-25T10:15:09.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0eccb6640 1 -- 172.21.15.133:0/4010557048 learned_addr learned my addr 172.21.15.133:0/4010557048 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:09.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e7fff640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e815c9a0 0x7ff0e80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e77fe640 1 --2- 172.21.15.133:0/4010557048 >> v2:172.21.15.162:3300/0 conn(0x7ff0e80b1f70 0x7ff0e806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413310674 0 0) 0x7ff0e80c61d0 con 0x7ff0e80ce330 2024-04-25T10:15:09.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d0003530 con 0x7ff0e80ce330 2024-04-25T10:15:09.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1851474925 0 0) 0x7ff0d0003530 con 0x7ff0e80ce330 2024-04-25T10:15:09.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e815c9a0 msgr2=0x7ff0e80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e815c9a0 0x7ff0e80c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 >> v2:172.21.15.162:3300/0 conn(0x7ff0e80b1f70 msgr2=0x7ff0e806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 --2- 172.21.15.133:0/4010557048 >> v2:172.21.15.162:3300/0 conn(0x7ff0e80b1f70 0x7ff0e806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e806d7d0 con 0x7ff0e80ce330 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff0e0001a00 con 0x7ff0e80ce330 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 >> v1:172.21.15.133:6789/0 conn(0x7ff0e80ce330 legacy=0x7ff0e815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0d00039c0 0x7ff0d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e815c9a0 0x7ff0d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0d0004d50 0x7ff0d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e80c64b0 con 0x7ff0d0004d50 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e806d4f0 con 0x7ff0d00039c0 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d000b9d0 con 0x7ff0e815c9a0 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0eccb6640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0d0004d50 0x7ff0d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e7fff640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0d00039c0 0x7ff0d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0e77fe640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e815c9a0 0x7ff0d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0eccb6640 1 -- 172.21.15.133:0/4010557048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e815c9a0 msgr2=0x7ff0d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0eccb6640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e815c9a0 0x7ff0d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0eccb6640 1 -- 172.21.15.133:0/4010557048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0d00039c0 msgr2=0x7ff0d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0eccb6640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0d00039c0 0x7ff0d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.875+0000 7ff0eccb6640 1 -- 172.21.15.133:0/4010557048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d0003530 con 0x7ff0d0004d50 2024-04-25T10:15:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0eccb6640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0d0004d50 0x7ff0d00051d0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000a870 tx=0x7ff0e001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7bc3e88508f867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000a4c0 con 0x7ff0d0004d50 2024-04-25T10:15:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e00072d0 con 0x7ff0d0004d50 2024-04-25T10:15:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0e6ffd640 1 -- 172.21.15.133:0/4010557048 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00076c0 con 0x7ff0d0004d50 2024-04-25T10:15:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 -- 172.21.15.133:0/4010557048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0d0004d50 msgr2=0x7ff0d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 --2- 172.21.15.133:0/4010557048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0d0004d50 0x7ff0d00051d0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000a870 tx=0x7ff0e001d3a0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0eccb6640 1 -- 172.21.15.133:0/4010557048 reap_dead start 2024-04-25T10:15:09.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 -- 172.21.15.133:0/4010557048 shutdown_connections 2024-04-25T10:15:09.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 -- 172.21.15.133:0/4010557048 >> 172.21.15.133:0/4010557048 conn(0x7ff0e8067000 msgr2=0x7ff0e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 -- 172.21.15.133:0/4010557048 shutdown_connections 2024-04-25T10:15:09.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 -- 172.21.15.133:0/4010557048 wait complete. 2024-04-25T10:15:09.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 Processor -- start 2024-04-25T10:15:09.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 -- start start 2024-04-25T10:15:09.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0e814cf60 0x7ff0e80ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e814d340 0x7ff0e80a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e80b2350 0x7ff0e80b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e815cf00 con 0x7ff0e814d340 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e815cd80 con 0x7ff0e814cf60 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0ee719640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e815d850 con 0x7ff0e80b2350 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0e77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e814d340 0x7ff0e80a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0e77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e814d340 0x7ff0e80a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37624/0 (socket says 172.21.15.133:37624) 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.879+0000 7ff0e77fe640 1 -- 172.21.15.133:0/314314488 learned_addr learned my addr 172.21.15.133:0/314314488 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e7fff640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0e814cf60 0x7ff0e80ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0eccb6640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e80b2350 0x7ff0e80b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e77fe640 1 -- 172.21.15.133:0/314314488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e80b2350 msgr2=0x7ff0e80b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e77fe640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e80b2350 0x7ff0e80b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e77fe640 1 -- 172.21.15.133:0/314314488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0e814cf60 msgr2=0x7ff0e80ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e77fe640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0e814cf60 0x7ff0e80ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e77fe640 1 -- 172.21.15.133:0/314314488 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e80aa650 con 0x7ff0e814d340 2024-04-25T10:15:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e77fe640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e814d340 0x7ff0e80a8e40 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc011600 tx=0x7ff0dc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e4ff9640 1 -- 172.21.15.133:0/314314488 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00f920 con 0x7ff0e814d340 2024-04-25T10:15:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e4ff9640 1 -- 172.21.15.133:0/314314488 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc00fac0 con 0x7ff0e814d340 2024-04-25T10:15:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0ee719640 1 -- 172.21.15.133:0/314314488 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e80a95b0 con 0x7ff0e814d340 2024-04-25T10:15:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e4ff9640 1 -- 172.21.15.133:0/314314488 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00fe70 con 0x7ff0e814d340 2024-04-25T10:15:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0ee719640 1 -- 172.21.15.133:0/314314488 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e80a9b70 con 0x7ff0e814d340 2024-04-25T10:15:09.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0ee719640 1 -- 172.21.15.133:0/314314488 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e8068ef0 con 0x7ff0e814d340 2024-04-25T10:15:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e4ff9640 1 -- 172.21.15.133:0/314314488 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0dc015460 con 0x7ff0e814d340 2024-04-25T10:15:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e4ff9640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d4066f60 0x7ff0d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e4ff9640 1 -- 172.21.15.133:0/314314488 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff0dc0857f0 con 0x7ff0e814d340 2024-04-25T10:15:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e7fff640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d4066f60 0x7ff0d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.883+0000 7ff0e7fff640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d4066f60 0x7ff0d4069420 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80099b0 tx=0x7ff0d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:09.887+0000 7ff0e4ff9640 1 -- 172.21.15.133:0/314314488 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0dc019030 con 0x7ff0e814d340 2024-04-25T10:15:10.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.099+0000 7ff0ee719640 1 -- 172.21.15.133:0/314314488 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff0e806edb0 con 0x7ff0e814d340 2024-04-25T10:15:10.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.099+0000 7ff0e4ff9640 1 -- 172.21.15.133:0/314314488 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff0dc0569c0 con 0x7ff0e814d340 2024-04-25T10:15:10.103 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:10.103 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:15:10.103 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:15:10.105 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.106 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:15:10.107 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:15:10.108 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.109 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.110 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.111 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:15:10.112 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.113 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:15:10.114 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:10.115 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:10.116 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:15:10.116 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:15:10.116 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:15:10.116 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:15:10.116 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:15:10.116 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:15:10.116 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.116 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:10.116 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:15:10.116 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:15:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 -- 172.21.15.133:0/314314488 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d4066f60 msgr2=0x7ff0d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d4066f60 0x7ff0d4069420 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80099b0 tx=0x7ff0d8008040 comp rx=0 tx=0).stop 2024-04-25T10:15:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 -- 172.21.15.133:0/314314488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e814d340 msgr2=0x7ff0e80a8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e814d340 0x7ff0e80a8e40 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc011600 tx=0x7ff0dc011ac0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 -- 172.21.15.133:0/314314488 shutdown_connections 2024-04-25T10:15:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0e80b2350 0x7ff0e80b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0d4066f60 0x7ff0d4069420 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0e814d340 0x7ff0e80a8e40 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 --2- 172.21.15.133:0/314314488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0e814cf60 0x7ff0e80ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 -- 172.21.15.133:0/314314488 >> 172.21.15.133:0/314314488 conn(0x7ff0e8067000 msgr2=0x7ff0e815e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 -- 172.21.15.133:0/314314488 shutdown_connections 2024-04-25T10:15:10.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.103+0000 7ff0ee719640 1 -- 172.21.15.133:0/314314488 wait complete. 2024-04-25T10:15:10.121 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:10.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc412a1640 1 Processor -- start 2024-04-25T10:15:10.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc412a1640 1 -- start start 2024-04-25T10:15:10.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc412a1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcc3c0ccf90 0x7fcc3c06fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc412a1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcc3c0702d0 0x7fcc3c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc412a1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3c070700 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc412a1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3c15bda0 con 0x7fcc3c0702d0 2024-04-25T10:15:10.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc412a1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3c0cd450 con 0x7fcc3c0ccf90 2024-04-25T10:15:10.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc3b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcc3c0702d0 0x7fcc3c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc3b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcc3c0702d0 0x7fcc3c0cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37626/0 (socket says 172.21.15.133:37626) 2024-04-25T10:15:10.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc3b7fe640 1 -- 172.21.15.133:0/597895534 learned_addr learned my addr 172.21.15.133:0/597895534 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:10.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc3b7fe640 1 -- 172.21.15.133:0/597895534 >> v2:172.21.15.162:3300/0 conn(0x7fcc3c0ccf90 msgr2=0x7fcc3c06fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:10.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc3a7fc640 1 --2- 172.21.15.133:0/597895534 >> v2:172.21.15.162:3300/0 conn(0x7fcc3c0ccf90 0x7fcc3c06fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc3b7fe640 1 --2- 172.21.15.133:0/597895534 >> v2:172.21.15.162:3300/0 conn(0x7fcc3c0ccf90 0x7fcc3c06fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc3b7fe640 1 -- 172.21.15.133:0/597895534 >> v1:172.21.15.162:6789/0 conn(0x7fcc3c15b9c0 legacy=0x7fcc3c072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:10.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.227+0000 7fcc3b7fe640 1 -- 172.21.15.133:0/597895534 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c0cd730 con 0x7fcc3c0702d0 2024-04-25T10:15:10.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3b7fe640 1 --2- 172.21.15.133:0/597895534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c0702d0 0x7fcc3c0cf950 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c06bad0 tx=0x7fcc3401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17614cb50c6124f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc39ffb640 1 -- 172.21.15.133:0/597895534 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc340ac020 con 0x7fcc3c0702d0 2024-04-25T10:15:10.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc39ffb640 1 -- 172.21.15.133:0/597895534 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc340a6410 con 0x7fcc3c0702d0 2024-04-25T10:15:10.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc39ffb640 1 -- 172.21.15.133:0/597895534 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc340a67e0 con 0x7fcc3c0702d0 2024-04-25T10:15:10.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 -- 172.21.15.133:0/597895534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c0702d0 msgr2=0x7fcc3c0cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 --2- 172.21.15.133:0/597895534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c0702d0 0x7fcc3c0cf950 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c06bad0 tx=0x7fcc3401c740 comp rx=0 tx=0).stop 2024-04-25T10:15:10.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 -- 172.21.15.133:0/597895534 shutdown_connections 2024-04-25T10:15:10.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 --2- 172.21.15.133:0/597895534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c0702d0 0x7fcc3c0cf950 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 --2- 172.21.15.133:0/597895534 >> v2:172.21.15.162:3300/0 conn(0x7fcc3c0ccf90 0x7fcc3c06fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 -- 172.21.15.133:0/597895534 >> 172.21.15.133:0/597895534 conn(0x7fcc3c0b2370 msgr2=0x7fcc3c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 -- 172.21.15.133:0/597895534 shutdown_connections 2024-04-25T10:15:10.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 -- 172.21.15.133:0/597895534 wait complete. 2024-04-25T10:15:10.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 Processor -- start 2024-04-25T10:15:10.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 -- start start 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc3c0702d0 0x7fcc3c14e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc3c0ccf90 0x7fcc3c164f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c15b9c0 0x7fcc3c1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c0d0a70 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c0d08f0 con 0x7fcc3c0ccf90 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc412a1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c0d0bf0 con 0x7fcc3c0702d0 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c15b9c0 0x7fcc3c1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c15b9c0 0x7fcc3c1567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37636/0 (socket says 172.21.15.133:37636) 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3b7fe640 1 -- 172.21.15.133:0/2878112309 learned_addr learned my addr 172.21.15.133:0/2878112309 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3a7fc640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc3c0ccf90 0x7fcc3c164f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3affd640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc3c0702d0 0x7fcc3c14e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3b7fe640 1 -- 172.21.15.133:0/2878112309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc3c0702d0 msgr2=0x7fcc3c14e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3b7fe640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc3c0702d0 0x7fcc3c14e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3b7fe640 1 -- 172.21.15.133:0/2878112309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc3c0ccf90 msgr2=0x7fcc3c164f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3b7fe640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc3c0ccf90 0x7fcc3c164f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3b7fe640 1 -- 172.21.15.133:0/2878112309 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c165870 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.231+0000 7fcc3b7fe640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c15b9c0 0x7fcc3c1567a0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fcc3401a370 tx=0x7fcc340194e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.235+0000 7fcc237fe640 1 -- 172.21.15.133:0/2878112309 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc340b7070 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.235+0000 7fcc237fe640 1 -- 172.21.15.133:0/2878112309 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc340b1420 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.235+0000 7fcc412a1640 1 -- 172.21.15.133:0/2878112309 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3c165b50 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.235+0000 7fcc237fe640 1 -- 172.21.15.133:0/2878112309 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc340b1810 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.235+0000 7fcc412a1640 1 -- 172.21.15.133:0/2878112309 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3c166110 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.235+0000 7fcc412a1640 1 -- 172.21.15.133:0/2878112309 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3c067c70 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.235+0000 7fcc237fe640 1 -- 172.21.15.133:0/2878112309 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc340af020 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.235+0000 7fcc237fe640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc28066f60 0x7fcc28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.235+0000 7fcc237fe640 1 -- 172.21.15.133:0/2878112309 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcc3411eff0 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.235+0000 7fcc3affd640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc28066f60 0x7fcc28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.235+0000 7fcc3affd640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc28066f60 0x7fcc28069420 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c1fa300 tx=0x7fcc2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.239+0000 7fcc237fe640 1 -- 172.21.15.133:0/2878112309 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc3c070700 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.386+0000 7fcc412a1640 1 -- 172.21.15.133:0/2878112309 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc3c070ba0 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.386+0000 7fcc237fe640 1 -- 172.21.15.133:0/2878112309 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+13198 (secure 0 0 0) 0x7fcc340f0280 con 0x7fcc3c15b9c0 2024-04-25T10:15:10.391 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:10.392 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":153,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:15:09.722790+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 -- 172.21.15.133:0/2878112309 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc28066f60 msgr2=0x7fcc28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc28066f60 0x7fcc28069420 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c1fa300 tx=0x7fcc2c009040 comp rx=0 tx=0).stop 2024-04-25T10:15:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 -- 172.21.15.133:0/2878112309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c15b9c0 msgr2=0x7fcc3c1567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c15b9c0 0x7fcc3c1567a0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fcc3401a370 tx=0x7fcc340194e0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 -- 172.21.15.133:0/2878112309 shutdown_connections 2024-04-25T10:15:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc28066f60 0x7fcc28069420 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c15b9c0 0x7fcc3c1567a0 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc3c0ccf90 0x7fcc3c164f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 --2- 172.21.15.133:0/2878112309 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc3c0702d0 0x7fcc3c14e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 -- 172.21.15.133:0/2878112309 >> 172.21.15.133:0/2878112309 conn(0x7fcc3c0b2370 msgr2=0x7fcc3c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 -- 172.21.15.133:0/2878112309 shutdown_connections 2024-04-25T10:15:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.390+0000 7fcc412a1640 1 -- 172.21.15.133:0/2878112309 wait complete. 2024-04-25T10:15:10.409 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:10.410 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:10.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f65175d5640 1 Processor -- start 2024-04-25T10:15:10.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f65175d5640 1 -- start start 2024-04-25T10:15:10.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f65175d5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f65100cd820 0x7f65100cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f65175d5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f651015ced0 0x7f651015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f65175d5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65100ce2e0 con 0x7f6510145170 2024-04-25T10:15:10.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f65175d5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651006db00 con 0x7f651015ced0 2024-04-25T10:15:10.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f65175d5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6510145550 con 0x7f65100cd820 2024-04-25T10:15:10.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f6515b72640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f651015ced0 0x7f651015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f6515b72640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f651015ced0 0x7f651015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37646/0 (socket says 172.21.15.133:37646) 2024-04-25T10:15:10.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f6515b72640 1 -- 172.21.15.133:0/2948871445 learned_addr learned my addr 172.21.15.133:0/2948871445 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:10.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f6515371640 1 --1- 172.21.15.133:0/2948871445 >> v1:172.21.15.133:6789/0 conn(0x7f6510145170 0x7f651006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38180/0 (socket says 172.21.15.133:38180) 2024-04-25T10:15:10.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f6514b70640 1 --2- 172.21.15.133:0/2948871445 >> v2:172.21.15.162:3300/0 conn(0x7f65100cd820 0x7f65100cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f6515b72640 1 -- 172.21.15.133:0/2948871445 >> v2:172.21.15.162:3300/0 conn(0x7f65100cd820 msgr2=0x7f65100cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:10.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f6515b72640 1 --2- 172.21.15.133:0/2948871445 >> v2:172.21.15.162:3300/0 conn(0x7f65100cd820 0x7f65100cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f6515b72640 1 -- 172.21.15.133:0/2948871445 >> v1:172.21.15.133:6789/0 conn(0x7f6510145170 legacy=0x7f651006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.526+0000 7f6515b72640 1 -- 172.21.15.133:0/2948871445 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651006d4b0 con 0x7f651015ced0 2024-04-25T10:15:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f6515b72640 1 --2- 172.21.15.133:0/2948871445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f651015ced0 0x7f651015d320 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f65100b4dd0 tx=0x7f650001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eff6af9e9939e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f650ffff640 1 -- 172.21.15.133:0/2948871445 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65000a56c0 con 0x7f651015ced0 2024-04-25T10:15:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f650ffff640 1 -- 172.21.15.133:0/2948871445 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65000a5860 con 0x7f651015ced0 2024-04-25T10:15:10.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f650ffff640 1 -- 172.21.15.133:0/2948871445 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65000a5c30 con 0x7f651015ced0 2024-04-25T10:15:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 -- 172.21.15.133:0/2948871445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f651015ced0 msgr2=0x7f651015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 --2- 172.21.15.133:0/2948871445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f651015ced0 0x7f651015d320 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f65100b4dd0 tx=0x7f650001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 -- 172.21.15.133:0/2948871445 shutdown_connections 2024-04-25T10:15:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 --2- 172.21.15.133:0/2948871445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f651015ced0 0x7f651015d320 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 --2- 172.21.15.133:0/2948871445 >> v2:172.21.15.162:3300/0 conn(0x7f65100cd820 0x7f65100cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 -- 172.21.15.133:0/2948871445 >> 172.21.15.133:0/2948871445 conn(0x7f6510066f70 msgr2=0x7f651006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 -- 172.21.15.133:0/2948871445 shutdown_connections 2024-04-25T10:15:10.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 -- 172.21.15.133:0/2948871445 wait complete. 2024-04-25T10:15:10.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 Processor -- start 2024-04-25T10:15:10.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 -- start start 2024-04-25T10:15:10.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65100cd820 0x7f6510140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6510145170 0x7f6510150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f651015ced0 0x7f6510133b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6510168610 con 0x7f651015ced0 2024-04-25T10:15:10.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6510168490 con 0x7f65100cd820 2024-04-25T10:15:10.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f65175d5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6510168790 con 0x7f6510145170 2024-04-25T10:15:10.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f6515b72640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f651015ced0 0x7f6510133b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f6515b72640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f651015ced0 0x7f6510133b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37662/0 (socket says 172.21.15.133:37662) 2024-04-25T10:15:10.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f6515b72640 1 -- 172.21.15.133:0/2145979681 learned_addr learned my addr 172.21.15.133:0/2145979681 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:10.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f6515371640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65100cd820 0x7f6510140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f6514b70640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6510145170 0x7f6510150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f6515b72640 1 -- 172.21.15.133:0/2145979681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6510145170 msgr2=0x7f6510150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.530+0000 7f6515b72640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6510145170 0x7f6510150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f6515b72640 1 -- 172.21.15.133:0/2145979681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65100cd820 msgr2=0x7f6510140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f6515b72640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65100cd820 0x7f6510140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f6515b72640 1 -- 172.21.15.133:0/2145979681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6510139260 con 0x7f651015ced0 2024-04-25T10:15:10.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f6515b72640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f651015ced0 0x7f6510133b20 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f6500019600 tx=0x7f6500019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f650dffb640 1 -- 172.21.15.133:0/2145979681 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65000bc020 con 0x7f651015ced0 2024-04-25T10:15:10.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f650dffb640 1 -- 172.21.15.133:0/2145979681 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65000b8670 con 0x7f651015ced0 2024-04-25T10:15:10.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f650dffb640 1 -- 172.21.15.133:0/2145979681 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500009510 con 0x7f651015ced0 2024-04-25T10:15:10.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f65175d5640 1 -- 172.21.15.133:0/2145979681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651013e5b0 con 0x7f651015ced0 2024-04-25T10:15:10.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f65175d5640 1 -- 172.21.15.133:0/2145979681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65101397e0 con 0x7f651015ced0 2024-04-25T10:15:10.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f65175d5640 1 -- 172.21.15.133:0/2145979681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6510068e80 con 0x7f651015ced0 2024-04-25T10:15:10.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f650dffb640 1 -- 172.21.15.133:0/2145979681 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65000bf020 con 0x7f651015ced0 2024-04-25T10:15:10.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f650dffb640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64fc066f90 0x7f64fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f650dffb640 1 -- 172.21.15.133:0/2145979681 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f650011fec0 con 0x7f651015ced0 2024-04-25T10:15:10.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f6515371640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64fc066f90 0x7f64fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.534+0000 7f6515371640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64fc066f90 0x7f64fc069450 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6508000f70 tx=0x7f6508007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.538+0000 7f650dffb640 1 -- 172.21.15.133:0/2145979681 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65000c6030 con 0x7f651015ced0 2024-04-25T10:15:10.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 -- 172.21.15.133:0/2145979681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6510141d80 con 0x7f651015ced0 2024-04-25T10:15:10.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f650dffb640 1 -- 172.21.15.133:0/2145979681 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v153) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65000165e0 con 0x7f651015ced0 2024-04-25T10:15:10.701 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:15:10.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 -- 172.21.15.133:0/2145979681 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64fc066f90 msgr2=0x7f64fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64fc066f90 0x7f64fc069450 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6508000f70 tx=0x7f6508007cf0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 -- 172.21.15.133:0/2145979681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f651015ced0 msgr2=0x7f6510133b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f651015ced0 0x7f6510133b20 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f6500019600 tx=0x7f6500019790 comp rx=0 tx=0).stop 2024-04-25T10:15:10.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 -- 172.21.15.133:0/2145979681 shutdown_connections 2024-04-25T10:15:10.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64fc066f90 0x7f64fc069450 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f651015ced0 0x7f6510133b20 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6510145170 0x7f6510150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 --2- 172.21.15.133:0/2145979681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f65100cd820 0x7f6510140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 -- 172.21.15.133:0/2145979681 >> 172.21.15.133:0/2145979681 conn(0x7f6510066f70 msgr2=0x7f651015e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 -- 172.21.15.133:0/2145979681 shutdown_connections 2024-04-25T10:15:10.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.698+0000 7f65175d5640 1 -- 172.21.15.133:0/2145979681 wait complete. 2024-04-25T10:15:10.720 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:15:10.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa363be8640 1 Processor -- start 2024-04-25T10:15:10.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa363be8640 1 -- start start 2024-04-25T10:15:10.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa363be8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa35c0cf310 0x7fa35c09a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa363be8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa35c0d02a0 0x7fa35c0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa363be8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c0cc2c0 con 0x7fa35c15ef50 2024-04-25T10:15:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa363be8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35c0cc560 con 0x7fa35c0cf310 2024-04-25T10:15:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa363be8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35c0cc840 con 0x7fa35c0d02a0 2024-04-25T10:15:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa361183640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa35c0cf310 0x7fa35c09a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa361984640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa35c15ef50 0x7fa35c0cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38186/0 (socket says 172.21.15.133:38186) 2024-04-25T10:15:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa361984640 1 -- 172.21.15.133:0/3015765135 learned_addr learned my addr 172.21.15.133:0/3015765135 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa362185640 1 --2- 172.21.15.133:0/3015765135 >> v2:172.21.15.162:3300/0 conn(0x7fa35c0d02a0 0x7fa35c0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa361183640 1 -- 172.21.15.133:0/3015765135 >> v2:172.21.15.162:3300/0 conn(0x7fa35c0d02a0 msgr2=0x7fa35c0cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa361183640 1 --2- 172.21.15.133:0/3015765135 >> v2:172.21.15.162:3300/0 conn(0x7fa35c0d02a0 0x7fa35c0cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa361183640 1 -- 172.21.15.133:0/3015765135 >> v1:172.21.15.133:6789/0 conn(0x7fa35c15ef50 legacy=0x7fa35c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa360982640 1 -- 172.21.15.133:0/3015765135 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137842285 0 0) 0x7fa35c0cc2c0 con 0x7fa35c15ef50 2024-04-25T10:15:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa361183640 1 -- 172.21.15.133:0/3015765135 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0ccb20 con 0x7fa35c0cf310 2024-04-25T10:15:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.838+0000 7fa362185640 1 --2- 172.21.15.133:0/3015765135 >> v2:172.21.15.162:3300/0 conn(0x7fa35c0d02a0 0x7fa35c0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361183640 1 --2- 172.21.15.133:0/3015765135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c0cf310 0x7fa35c09a690 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fa35c072dd0 tx=0x7fa34c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84ec50f759fa568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa360982640 1 -- 172.21.15.133:0/3015765135 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0b0070 con 0x7fa35c0cf310 2024-04-25T10:15:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa360982640 1 -- 172.21.15.133:0/3015765135 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c0ab600 con 0x7fa35c0cf310 2024-04-25T10:15:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa360982640 1 -- 172.21.15.133:0/3015765135 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0ab9d0 con 0x7fa35c0cf310 2024-04-25T10:15:10.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 -- 172.21.15.133:0/3015765135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c0cf310 msgr2=0x7fa35c09a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 --2- 172.21.15.133:0/3015765135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c0cf310 0x7fa35c09a690 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fa35c072dd0 tx=0x7fa34c01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 -- 172.21.15.133:0/3015765135 shutdown_connections 2024-04-25T10:15:10.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 --2- 172.21.15.133:0/3015765135 >> v2:172.21.15.162:3300/0 conn(0x7fa35c0d02a0 0x7fa35c0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 --2- 172.21.15.133:0/3015765135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c0cf310 0x7fa35c09a690 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 -- 172.21.15.133:0/3015765135 >> 172.21.15.133:0/3015765135 conn(0x7fa35c0b3230 msgr2=0x7fa35c0b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 -- 172.21.15.133:0/3015765135 shutdown_connections 2024-04-25T10:15:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 -- 172.21.15.133:0/3015765135 wait complete. 2024-04-25T10:15:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 Processor -- start 2024-04-25T10:15:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 -- start start 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35c0d02a0 0x7fa35c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c15ef50 0x7fa35c1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35c1540b0 0x7fa35c15db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0c5e70 con 0x7fa35c15ef50 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0c5cf0 con 0x7fa35c1540b0 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa363be8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c168a00 con 0x7fa35c0d02a0 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361183640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c15ef50 0x7fa35c1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361183640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c15ef50 0x7fa35c1530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37682/0 (socket says 172.21.15.133:37682) 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361183640 1 -- 172.21.15.133:0/2067148316 learned_addr learned my addr 172.21.15.133:0/2067148316 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa362185640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35c1540b0 0x7fa35c15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361984640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35c0d02a0 0x7fa35c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361183640 1 -- 172.21.15.133:0/2067148316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35c0d02a0 msgr2=0x7fa35c154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361183640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35c0d02a0 0x7fa35c154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361183640 1 -- 172.21.15.133:0/2067148316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35c1540b0 msgr2=0x7fa35c15db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361183640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35c1540b0 0x7fa35c15db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361183640 1 -- 172.21.15.133:0/2067148316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c154850 con 0x7fa35c15ef50 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361984640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35c0d02a0 0x7fa35c154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa362185640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35c1540b0 0x7fa35c15db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa361183640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c15ef50 0x7fa35c1530a0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0ae040 tx=0x7fa34c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa3527fc640 1 -- 172.21.15.133:0/2067148316 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0bc070 con 0x7fa35c15ef50 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.842+0000 7fa3527fc640 1 -- 172.21.15.133:0/2067148316 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c0b1040 con 0x7fa35c15ef50 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.846+0000 7fa363be8640 1 -- 172.21.15.133:0/2067148316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c0c8a20 con 0x7fa35c15ef50 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.846+0000 7fa3527fc640 1 -- 172.21.15.133:0/2067148316 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0b0050 con 0x7fa35c15ef50 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.846+0000 7fa363be8640 1 -- 172.21.15.133:0/2067148316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c154540 con 0x7fa35c15ef50 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.846+0000 7fa363be8640 1 -- 172.21.15.133:0/2067148316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c068ef0 con 0x7fa35c15ef50 2024-04-25T10:15:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.846+0000 7fa3527fc640 1 -- 172.21.15.133:0/2067148316 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa34c0bf070 con 0x7fa35c15ef50 2024-04-25T10:15:10.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.846+0000 7fa3527fc640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa340066f90 0x7fa340069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.846+0000 7fa3527fc640 1 -- 172.21.15.133:0/2067148316 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa34c11ea80 con 0x7fa35c15ef50 2024-04-25T10:15:10.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.846+0000 7fa361984640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa340066f90 0x7fa340069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.846+0000 7fa361984640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa340066f90 0x7fa340069450 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa358007fd0 tx=0x7fa358009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.850+0000 7fa3527fc640 1 -- 172.21.15.133:0/2067148316 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa34c016070 con 0x7fa35c15ef50 2024-04-25T10:15:11.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 -- 172.21.15.133:0/2067148316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa35c0b3ec0 con 0x7fa35c15ef50 2024-04-25T10:15:11.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa3527fc640 1 -- 172.21.15.133:0/2067148316 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v153) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa34c0efc50 con 0x7fa35c15ef50 2024-04-25T10:15:11.002 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:11.002 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:15:11.002 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:15:11.002 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:15:11.002 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:15:11.002 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:11.002 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:15:11.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 -- 172.21.15.133:0/2067148316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa340066f90 msgr2=0x7fa340069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa340066f90 0x7fa340069450 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa358007fd0 tx=0x7fa358009040 comp rx=0 tx=0).stop 2024-04-25T10:15:11.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 -- 172.21.15.133:0/2067148316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c15ef50 msgr2=0x7fa35c1530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c15ef50 0x7fa35c1530a0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0ae040 tx=0x7fa34c019190 comp rx=0 tx=0).stop 2024-04-25T10:15:11.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 -- 172.21.15.133:0/2067148316 shutdown_connections 2024-04-25T10:15:11.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa35c1540b0 0x7fa35c15db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa340066f90 0x7fa340069450 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa35c15ef50 0x7fa35c1530a0 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 --2- 172.21.15.133:0/2067148316 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa35c0d02a0 0x7fa35c154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 -- 172.21.15.133:0/2067148316 >> 172.21.15.133:0/2067148316 conn(0x7fa35c0b3230 msgr2=0x7fa35c070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 -- 172.21.15.133:0/2067148316 shutdown_connections 2024-04-25T10:15:11.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:10.998+0000 7fa363be8640 1 -- 172.21.15.133:0/2067148316 wait complete. 2024-04-25T10:15:11.019 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:15:11.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:15:11.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fb49d1640 1 Processor -- start 2024-04-25T10:15:11.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fb49d1640 1 -- start start 2024-04-25T10:15:11.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fb49d1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fb00b4940 0x7f8fb00b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fb49d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fb0071cb0 0x7f8fb015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fb49d1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f8fb015c250 0x7f8fb00cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fb49d1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb0072a90 con 0x7f8fb00b4940 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fb49d1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb00a85d0 con 0x7f8fb015c250 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fb49d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb00b5260 con 0x7f8fb0071cb0 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fae59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fb00b4940 0x7f8fb00b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fae59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8fb00b4940 0x7f8fb00b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37684/0 (socket says 172.21.15.133:37684) 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fae59c640 1 -- 172.21.15.133:0/3420161713 learned_addr learned my addr 172.21.15.133:0/3420161713 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fadd9b640 1 --2- 172.21.15.133:0/3420161713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fb0071cb0 0x7f8fb015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8faed9d640 1 --2- 172.21.15.133:0/3420161713 >> v2:172.21.15.162:3300/0 conn(0x7f8fb015c250 0x7f8fb00cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fae59c640 1 -- 172.21.15.133:0/3420161713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fb0071cb0 msgr2=0x7f8fb015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fae59c640 1 --2- 172.21.15.133:0/3420161713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fb0071cb0 0x7f8fb015bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fae59c640 1 -- 172.21.15.133:0/3420161713 >> v2:172.21.15.162:3300/0 conn(0x7f8fb015c250 msgr2=0x7f8fb00cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.126+0000 7f8fae59c640 1 --2- 172.21.15.133:0/3420161713 >> v2:172.21.15.162:3300/0 conn(0x7f8fb015c250 0x7f8fb00cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fae59c640 1 -- 172.21.15.133:0/3420161713 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb015c9b0 con 0x7f8fb00b4940 2024-04-25T10:15:11.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fae59c640 1 --2- 172.21.15.133:0/3420161713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fb00b4940 0x7f8fb00b4d20 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0072fb0 tx=0x7f8fa001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0a5ff60a1b4f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fad59a640 1 -- 172.21.15.133:0/3420161713 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00aa610 con 0x7f8fb00b4940 2024-04-25T10:15:11.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fad59a640 1 -- 172.21.15.133:0/3420161713 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa00af070 con 0x7f8fb00b4940 2024-04-25T10:15:11.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fad59a640 1 -- 172.21.15.133:0/3420161713 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00aa9d0 con 0x7f8fb00b4940 2024-04-25T10:15:11.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fb49d1640 1 -- 172.21.15.133:0/3420161713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fb00b4940 msgr2=0x7f8fb00b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fb49d1640 1 --2- 172.21.15.133:0/3420161713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fb00b4940 0x7f8fb00b4d20 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0072fb0 tx=0x7f8fa001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fb49d1640 1 -- 172.21.15.133:0/3420161713 shutdown_connections 2024-04-25T10:15:11.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fb49d1640 1 --2- 172.21.15.133:0/3420161713 >> v2:172.21.15.162:3300/0 conn(0x7f8fb015c250 0x7f8fb00cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fb49d1640 1 --2- 172.21.15.133:0/3420161713 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fb0071cb0 0x7f8fb015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fb49d1640 1 --2- 172.21.15.133:0/3420161713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fb00b4940 0x7f8fb00b4d20 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fb49d1640 1 -- 172.21.15.133:0/3420161713 >> 172.21.15.133:0/3420161713 conn(0x7f8fb006d370 msgr2=0x7f8fb006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fb49d1640 1 -- 172.21.15.133:0/3420161713 shutdown_connections 2024-04-25T10:15:11.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.130+0000 7f8fb49d1640 1 -- 172.21.15.133:0/3420161713 wait complete. 2024-04-25T10:15:11.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fb49d1640 1 Processor -- start 2024-04-25T10:15:11.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fb49d1640 1 -- start start 2024-04-25T10:15:11.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fb49d1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fb0071cb0 0x7f8fb015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fb49d1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fb00b4940 0x7f8fb0143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fb49d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fb015c250 0x7f8fb00ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fb49d1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb015c660 con 0x7f8fb0071cb0 2024-04-25T10:15:11.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fb49d1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb0071790 con 0x7f8fb00b4940 2024-04-25T10:15:11.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fb49d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb00cc960 con 0x7f8fb015c250 2024-04-25T10:15:11.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fae59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fb0071cb0 0x7f8fb015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fae59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fb0071cb0 0x7f8fb015a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37686/0 (socket says 172.21.15.133:37686) 2024-04-25T10:15:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fae59c640 1 -- 172.21.15.133:0/1811758505 learned_addr learned my addr 172.21.15.133:0/1811758505 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fadd9b640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fb00b4940 0x7f8fb0143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8faed9d640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fb015c250 0x7f8fb00ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fae59c640 1 -- 172.21.15.133:0/1811758505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fb015c250 msgr2=0x7f8fb00ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fae59c640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fb015c250 0x7f8fb00ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fae59c640 1 -- 172.21.15.133:0/1811758505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fb00b4940 msgr2=0x7f8fb0143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fae59c640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fb00b4940 0x7f8fb0143970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fae59c640 1 -- 172.21.15.133:0/1811758505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb01426c0 con 0x7f8fb0071cb0 2024-04-25T10:15:11.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8faed9d640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fb015c250 0x7f8fb00ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fae59c640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fb0071cb0 0x7f8fb015a4d0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f8fa001c320 tx=0x7f8fa001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8f9effd640 1 -- 172.21.15.133:0/1811758505 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00bd5f0 con 0x7f8fb0071cb0 2024-04-25T10:15:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8f9effd640 1 -- 172.21.15.133:0/1811758505 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa00bd790 con 0x7f8fb0071cb0 2024-04-25T10:15:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8f9effd640 1 -- 172.21.15.133:0/1811758505 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00bdb60 con 0x7f8fb0071cb0 2024-04-25T10:15:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fb49d1640 1 -- 172.21.15.133:0/1811758505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb01429a0 con 0x7f8fb0071cb0 2024-04-25T10:15:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.134+0000 7f8fb49d1640 1 -- 172.21.15.133:0/1811758505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb0142f60 con 0x7f8fb0071cb0 2024-04-25T10:15:11.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.138+0000 7f8fb49d1640 1 -- 172.21.15.133:0/1811758505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb0069000 con 0x7f8fb0071cb0 2024-04-25T10:15:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.138+0000 7f8f9effd640 1 -- 172.21.15.133:0/1811758505 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fa00b33f0 con 0x7f8fb0071cb0 2024-04-25T10:15:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.138+0000 7f8f9effd640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f90066f60 0x7f8f90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.138+0000 7f8f9effd640 1 -- 172.21.15.133:0/1811758505 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8fa011f730 con 0x7f8fb0071cb0 2024-04-25T10:15:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.138+0000 7f8fadd9b640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f90066f60 0x7f8f90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.138+0000 7f8fadd9b640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f90066f60 0x7f8f90069420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8fb01445c0 tx=0x7f8fa4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.142+0000 7f8f9effd640 1 -- 172.21.15.133:0/1811758505 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fa00a7070 con 0x7f8fb0071cb0 2024-04-25T10:15:11.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.286+0000 7f8fb49d1640 1 -- 172.21.15.133:0/1811758505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8fb00af440 con 0x7f8fb0071cb0 2024-04-25T10:15:11.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.726+0000 7f8f9effd640 1 -- 172.21.15.133:0/1811758505 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v154) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8fa0016110 con 0x7f8fb0071cb0 2024-04-25T10:15:11.729 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:15:11.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 -- 172.21.15.133:0/1811758505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f90066f60 msgr2=0x7f8f90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f90066f60 0x7f8f90069420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8fb01445c0 tx=0x7f8fa4008040 comp rx=0 tx=0).stop 2024-04-25T10:15:11.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 -- 172.21.15.133:0/1811758505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fb0071cb0 msgr2=0x7f8fb015a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fb0071cb0 0x7f8fb015a4d0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f8fa001c320 tx=0x7f8fa001ad30 comp rx=0 tx=0).stop 2024-04-25T10:15:11.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 -- 172.21.15.133:0/1811758505 shutdown_connections 2024-04-25T10:15:11.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8fb015c250 0x7f8fb00ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fb00b4940 0x7f8fb0143970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8f90066f60 0x7f8f90069420 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 --2- 172.21.15.133:0/1811758505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fb0071cb0 0x7f8fb015a4d0 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 -- 172.21.15.133:0/1811758505 >> 172.21.15.133:0/1811758505 conn(0x7f8fb006d370 msgr2=0x7f8fb015ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 -- 172.21.15.133:0/1811758505 shutdown_connections 2024-04-25T10:15:11.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.730+0000 7f8fb49d1640 1 -- 172.21.15.133:0/1811758505 wait complete. 2024-04-25T10:15:11.763 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:15:11.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.890+0000 7f91f5d60640 1 Processor -- start 2024-04-25T10:15:11.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.890+0000 7f91f5d60640 1 -- start start 2024-04-25T10:15:11.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.890+0000 7f91f5d60640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f0142920 0x7f91f0143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.890+0000 7f91f5d60640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91f015d9f0 0x7f91f015de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.890+0000 7f91f5d60640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f0140ea0 con 0x7f91f0140a40 2024-04-25T10:15:11.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.890+0000 7f91f5d60640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00cf280 con 0x7f91f015d9f0 2024-04-25T10:15:11.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.890+0000 7f91f5d60640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f006d440 con 0x7f91f0142920 2024-04-25T10:15:11.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.890+0000 7f91eeffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f91f0140a40 0x7f91f00ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38200/0 (socket says 172.21.15.133:38200) 2024-04-25T10:15:11.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.890+0000 7f91eeffd640 1 -- 172.21.15.133:0/3688418084 learned_addr learned my addr 172.21.15.133:0/3688418084 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:11.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ef7fe640 1 --2- 172.21.15.133:0/3688418084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f0142920 0x7f91f0143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91effff640 1 --2- 172.21.15.133:0/3688418084 >> v2:172.21.15.162:3300/0 conn(0x7f91f015d9f0 0x7f91f015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844327909 0 0) 0x7f91f0140ea0 con 0x7f91f0140a40 2024-04-25T10:15:11.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91d8003530 con 0x7f91f0140a40 2024-04-25T10:15:11.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1541731442 0 0) 0x7f91d8003530 con 0x7f91f0140a40 2024-04-25T10:15:11.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f0142920 msgr2=0x7f91f0143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 --2- 172.21.15.133:0/3688418084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f0142920 0x7f91f0143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 >> v2:172.21.15.162:3300/0 conn(0x7f91f015d9f0 msgr2=0x7f91f015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 --2- 172.21.15.133:0/3688418084 >> v2:172.21.15.162:3300/0 conn(0x7f91f015d9f0 0x7f91f015de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f006d720 con 0x7f91f0140a40 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f91e4001680 con 0x7f91f0140a40 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 >> v1:172.21.15.133:6789/0 conn(0x7f91f0140a40 legacy=0x7f91f00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 --2- 172.21.15.133:0/3688418084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 0x7f91d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 --2- 172.21.15.133:0/3688418084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91d8004810 0x7f91d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 --2- 172.21.15.133:0/3688418084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f0142920 0x7f91d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f00cf280 con 0x7f91d80039c0 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f006d440 con 0x7f91d8004810 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d800b9d0 con 0x7f91f0142920 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ef7fe640 1 --2- 172.21.15.133:0/3688418084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 0x7f91d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91eeffd640 1 --2- 172.21.15.133:0/3688418084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91d8004810 0x7f91d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ef7fe640 1 -- 172.21.15.133:0/3688418084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f0142920 msgr2=0x7f91d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ef7fe640 1 --2- 172.21.15.133:0/3688418084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f0142920 0x7f91d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ef7fe640 1 -- 172.21.15.133:0/3688418084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91d8004810 msgr2=0x7f91d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ef7fe640 1 --2- 172.21.15.133:0/3688418084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91d8004810 0x7f91d8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ef7fe640 1 -- 172.21.15.133:0/3688418084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8003530 con 0x7f91d80039c0 2024-04-25T10:15:11.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ef7fe640 1 --2- 172.21.15.133:0/3688418084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 0x7f91d8003de0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f91e000f2a0 tx=0x7f91e001dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7671a355e36ffa99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e000ad20 con 0x7f91d80039c0 2024-04-25T10:15:11.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e00a6820 con 0x7f91d80039c0 2024-04-25T10:15:11.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.894+0000 7f91ee7fc640 1 -- 172.21.15.133:0/3688418084 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e00a6c10 con 0x7f91d80039c0 2024-04-25T10:15:11.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 -- 172.21.15.133:0/3688418084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 msgr2=0x7f91d8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 --2- 172.21.15.133:0/3688418084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 0x7f91d8003de0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f91e000f2a0 tx=0x7f91e001dda0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91effff640 1 -- 172.21.15.133:0/3688418084 reap_dead start 2024-04-25T10:15:11.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 -- 172.21.15.133:0/3688418084 shutdown_connections 2024-04-25T10:15:11.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 -- 172.21.15.133:0/3688418084 >> 172.21.15.133:0/3688418084 conn(0x7f91f0066f70 msgr2=0x7f91f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 -- 172.21.15.133:0/3688418084 shutdown_connections 2024-04-25T10:15:11.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 -- 172.21.15.133:0/3688418084 wait complete. 2024-04-25T10:15:11.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 Processor -- start 2024-04-25T10:15:11.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 -- start start 2024-04-25T10:15:11.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 0x7f91f00b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91f0138d80 0x7f91f00b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f00b34e0 0x7f91f00ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f006e2b0 con 0x7f91d80039c0 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f006e130 con 0x7f91f0138d80 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f006e430 con 0x7f91f00b34e0 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 0x7f91f00b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 0x7f91f00b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37694/0 (socket says 172.21.15.133:37694) 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91ef7fe640 1 -- 172.21.15.133:0/894915066 learned_addr learned my addr 172.21.15.133:0/894915066 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91eeffd640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91f0138d80 0x7f91f00b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91effff640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f00b34e0 0x7f91f00ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91ef7fe640 1 -- 172.21.15.133:0/894915066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f00b34e0 msgr2=0x7f91f00ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91ef7fe640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f00b34e0 0x7f91f00ac840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91ef7fe640 1 -- 172.21.15.133:0/894915066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91f0138d80 msgr2=0x7f91f00b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91ef7fe640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91f0138d80 0x7f91f00b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91ef7fe640 1 -- 172.21.15.133:0/894915066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00afc90 con 0x7f91d80039c0 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91ef7fe640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 0x7f91f00b16d0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f91e000a040 tx=0x7f91e000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91d7fff640 1 -- 172.21.15.133:0/894915066 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e00bd020 con 0x7f91d80039c0 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91d7fff640 1 -- 172.21.15.133:0/894915066 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e0008440 con 0x7f91d80039c0 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 -- 172.21.15.133:0/894915066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f00aff70 con 0x7f91d80039c0 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91d7fff640 1 -- 172.21.15.133:0/894915066 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e000ad20 con 0x7f91d80039c0 2024-04-25T10:15:11.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.898+0000 7f91f5d60640 1 -- 172.21.15.133:0/894915066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f00b0530 con 0x7f91d80039c0 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.902+0000 7f91f5d60640 1 -- 172.21.15.133:0/894915066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f0068e80 con 0x7f91d80039c0 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.902+0000 7f91d7fff640 1 -- 172.21.15.133:0/894915066 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91e00c0070 con 0x7f91d80039c0 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.902+0000 7f91d7fff640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91dc066f60 0x7f91dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.902+0000 7f91d7fff640 1 -- 172.21.15.133:0/894915066 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f91e011f420 con 0x7f91d80039c0 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.902+0000 7f91eeffd640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91dc066f60 0x7f91dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.902+0000 7f91eeffd640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91dc066f60 0x7f91dc069420 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f91e4001830 tx=0x7f91e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:11.906+0000 7f91d7fff640 1 -- 172.21.15.133:0/894915066 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91e00c7080 con 0x7f91d80039c0 2024-04-25T10:15:12.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.058+0000 7f91f5d60640 1 -- 172.21.15.133:0/894915066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f91f00b38c0 con 0x7f91d80039c0 2024-04-25T10:15:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.734+0000 7f91d7fff640 1 -- 172.21.15.133:0/894915066 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v155) v1 ==== 170+0+0 (secure 0 0 0) 0x7f91e0017af0 con 0x7f91d80039c0 2024-04-25T10:15:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:15:12.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.738+0000 7f91f5d60640 1 -- 172.21.15.133:0/894915066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91dc066f60 msgr2=0x7f91dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.738+0000 7f91f5d60640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91dc066f60 0x7f91dc069420 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f91e4001830 tx=0x7f91e4009040 comp rx=0 tx=0).stop 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.742+0000 7f91f5d60640 1 -- 172.21.15.133:0/894915066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 msgr2=0x7f91f00b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.742+0000 7f91f5d60640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 0x7f91f00b16d0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f91e000a040 tx=0x7f91e000a860 comp rx=0 tx=0).stop 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.742+0000 7f91f5d60640 1 -- 172.21.15.133:0/894915066 shutdown_connections 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.742+0000 7f91f5d60640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91f00b34e0 0x7f91f00ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.742+0000 7f91f5d60640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91f0138d80 0x7f91f00b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.742+0000 7f91f5d60640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91dc066f60 0x7f91dc069420 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.742+0000 7f91f5d60640 1 --2- 172.21.15.133:0/894915066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d80039c0 0x7f91f00b16d0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.742+0000 7f91f5d60640 1 -- 172.21.15.133:0/894915066 >> 172.21.15.133:0/894915066 conn(0x7f91f0066f70 msgr2=0x7f91f0140050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.742+0000 7f91f5d60640 1 -- 172.21.15.133:0/894915066 shutdown_connections 2024-04-25T10:15:12.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.742+0000 7f91f5d60640 1 -- 172.21.15.133:0/894915066 wait complete. 2024-04-25T10:15:12.781 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:15:12.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91d20da640 1 Processor -- start 2024-04-25T10:15:12.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91d20da640 1 -- start start 2024-04-25T10:15:12.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91d20da640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f91cc143560 0x7f91cc140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91d20da640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc0cf280 con 0x7f91cc0ceac0 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91d20da640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc06d440 con 0x7f91cc06b3a0 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91d20da640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc143940 con 0x7f91cc143560 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91caffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f91cc0ceac0 0x7f91cc142920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38214/0 (socket says 172.21.15.133:38214) 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91caffd640 1 -- 172.21.15.133:0/565415022 learned_addr learned my addr 172.21.15.133:0/565415022 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91cb7fe640 1 --2- 172.21.15.133:0/565415022 >> v2:172.21.15.162:3300/0 conn(0x7f91cc143560 0x7f91cc140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276970974 0 0) 0x7f91cc0cf280 con 0x7f91cc0ceac0 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b4005740 con 0x7f91cc0ceac0 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1771423592 0 0) 0x7f91b4005740 con 0x7f91cc0ceac0 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 >> v2:172.21.15.162:3300/0 conn(0x7f91cc143560 msgr2=0x7f91cc140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 --2- 172.21.15.133:0/565415022 >> v2:172.21.15.162:3300/0 conn(0x7f91cc143560 0x7f91cc140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 >> v1:172.21.15.162:6789/0 conn(0x7f91cc06b3a0 legacy=0x7f91cc15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc06d720 con 0x7f91cc0ceac0 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f91c0001680 con 0x7f91cc0ceac0 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 >> v1:172.21.15.133:6789/0 conn(0x7f91cc0ceac0 legacy=0x7f91cc142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b4005bd0 0x7f91b4005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91b4006a20 0x7f91b4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b40073e0 0x7f91b400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc143940 con 0x7f91b4005bd0 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b400e380 con 0x7f91b40073e0 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b400e580 con 0x7f91b4006a20 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91cb7fe640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b4005bd0 0x7f91b4005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91cbfff640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b40073e0 0x7f91b400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91caffd640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91b4006a20 0x7f91b4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91cb7fe640 1 -- 172.21.15.133:0/565415022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91b4006a20 msgr2=0x7f91b4006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91cb7fe640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91b4006a20 0x7f91b4006ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91cb7fe640 1 -- 172.21.15.133:0/565415022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b40073e0 msgr2=0x7f91b400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91cb7fe640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b40073e0 0x7f91b400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91cb7fe640 1 -- 172.21.15.133:0/565415022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b4005740 con 0x7f91b4005bd0 2024-04-25T10:15:12.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91cbfff640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b40073e0 0x7f91b400dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:12.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.898+0000 7f91cbfff640 1 -- 172.21.15.133:0/565415022 reap_dead start 2024-04-25T10:15:12.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91cb7fe640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b4005bd0 0x7f91b4005ff0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009ac0 tx=0x7f91bc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b84f11d7bd23618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc009690 con 0x7f91b4005bd0 2024-04-25T10:15:12.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc0a8460 con 0x7f91b4005bd0 2024-04-25T10:15:12.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91ca7fc640 1 -- 172.21.15.133:0/565415022 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc0a8830 con 0x7f91b4005bd0 2024-04-25T10:15:12.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 -- 172.21.15.133:0/565415022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b4005bd0 msgr2=0x7f91b4005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b4005bd0 0x7f91b4005ff0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009ac0 tx=0x7f91bc01ca50 comp rx=0 tx=0).stop 2024-04-25T10:15:12.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 -- 172.21.15.133:0/565415022 shutdown_connections 2024-04-25T10:15:12.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 --2- 172.21.15.133:0/565415022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91b4005bd0 0x7f91b4005ff0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 -- 172.21.15.133:0/565415022 >> 172.21.15.133:0/565415022 conn(0x7f91cc066f70 msgr2=0x7f91cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 -- 172.21.15.133:0/565415022 shutdown_connections 2024-04-25T10:15:12.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 -- 172.21.15.133:0/565415022 wait complete. 2024-04-25T10:15:12.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 Processor -- start 2024-04-25T10:15:12.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 -- start start 2024-04-25T10:15:12.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b4005bd0 0x7f91cc156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91cc156b10 0x7f91cc136d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91cc1f2f00 0x7f91cc15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc15e380 con 0x7f91cc156b10 2024-04-25T10:15:12.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc15e200 con 0x7f91b4005bd0 2024-04-25T10:15:12.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91d20da640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc142920 con 0x7f91cc1f2f00 2024-04-25T10:15:12.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91caffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91cc156b10 0x7f91cc136d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91caffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91cc156b10 0x7f91cc136d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37712/0 (socket says 172.21.15.133:37712) 2024-04-25T10:15:12.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.902+0000 7f91caffd640 1 -- 172.21.15.133:0/250431986 learned_addr learned my addr 172.21.15.133:0/250431986 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:12.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91cb7fe640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b4005bd0 0x7f91cc156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91cbfff640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91cc1f2f00 0x7f91cc15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91caffd640 1 -- 172.21.15.133:0/250431986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91cc1f2f00 msgr2=0x7f91cc15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91caffd640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91cc1f2f00 0x7f91cc15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91caffd640 1 -- 172.21.15.133:0/250431986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b4005bd0 msgr2=0x7f91cc156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91caffd640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b4005bd0 0x7f91cc156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91caffd640 1 -- 172.21.15.133:0/250431986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc1372a0 con 0x7f91cc156b10 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91cbfff640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91cc1f2f00 0x7f91cc15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91caffd640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91cc156b10 0x7f91cc136d60 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f91c000d810 tx=0x7f91c000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91b3fff640 1 -- 172.21.15.133:0/250431986 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0019070 con 0x7f91cc156b10 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91b3fff640 1 -- 172.21.15.133:0/250431986 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c00074e0 con 0x7f91cc156b10 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91d20da640 1 -- 172.21.15.133:0/250431986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc1571d0 con 0x7f91cc156b10 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91b3fff640 1 -- 172.21.15.133:0/250431986 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0004a10 con 0x7f91cc156b10 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91d20da640 1 -- 172.21.15.133:0/250431986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc157790 con 0x7f91cc156b10 2024-04-25T10:15:12.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91d20da640 1 -- 172.21.15.133:0/250431986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc068e80 con 0x7f91cc156b10 2024-04-25T10:15:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91b3fff640 1 -- 172.21.15.133:0/250431986 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91c001c070 con 0x7f91cc156b10 2024-04-25T10:15:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91b3fff640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91b8066f60 0x7f91b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91b3fff640 1 -- 172.21.15.133:0/250431986 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f91c00852e0 con 0x7f91cc156b10 2024-04-25T10:15:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91cb7fe640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91b8066f60 0x7f91b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.906+0000 7f91cb7fe640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91b8066f60 0x7f91b8069420 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0af000 tx=0x7f91bc0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:12.910+0000 7f91b3fff640 1 -- 172.21.15.133:0/250431986 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91c0023070 con 0x7f91cc156b10 2024-04-25T10:15:13.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.090+0000 7f91d20da640 1 -- 172.21.15.133:0/250431986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f91cc0afa70 con 0x7f91cc156b10 2024-04-25T10:15:13.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.754+0000 7f91b3fff640 1 -- 172.21.15.133:0/250431986 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v97) v1 ==== 174+0+0 (secure 0 0 0) 0x7f91c00562a0 con 0x7f91cc156b10 2024-04-25T10:15:13.763 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 23 and data pool 24 2024-04-25T10:15:13.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.774+0000 7f91d20da640 1 -- 172.21.15.133:0/250431986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91b8066f60 msgr2=0x7f91b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.774+0000 7f91d20da640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91b8066f60 0x7f91b8069420 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0af000 tx=0x7f91bc0046d0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.774+0000 7f91d20da640 1 -- 172.21.15.133:0/250431986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91cc156b10 msgr2=0x7f91cc136d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.774+0000 7f91d20da640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91cc156b10 0x7f91cc136d60 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f91c000d810 tx=0x7f91c000dcd0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.778+0000 7f91d20da640 1 -- 172.21.15.133:0/250431986 shutdown_connections 2024-04-25T10:15:13.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.778+0000 7f91d20da640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91cc1f2f00 0x7f91cc15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.778+0000 7f91d20da640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91b8066f60 0x7f91b8069420 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.778+0000 7f91d20da640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91cc156b10 0x7f91cc136d60 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.778+0000 7f91d20da640 1 --2- 172.21.15.133:0/250431986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91b4005bd0 0x7f91cc156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.778+0000 7f91d20da640 1 -- 172.21.15.133:0/250431986 >> 172.21.15.133:0/250431986 conn(0x7f91cc066f70 msgr2=0x7f91cc0b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.778+0000 7f91d20da640 1 -- 172.21.15.133:0/250431986 shutdown_connections 2024-04-25T10:15:13.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.778+0000 7f91d20da640 1 -- 172.21.15.133:0/250431986 wait complete. 2024-04-25T10:15:13.810 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.914+0000 7f22f533c640 1 Processor -- start 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.914+0000 7f22f533c640 1 -- start start 2024-04-25T10:15:13.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.914+0000 7f22f533c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f22f00ce330 0x7f22f00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.914+0000 7f22f533c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f00c61d0 con 0x7f22f00b1f70 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.914+0000 7f22f533c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f00c64b0 con 0x7f22f015c9a0 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.914+0000 7f22f533c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f00cecc0 con 0x7f22f00ce330 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.914+0000 7f22ee7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f22f00b1f70 0x7f22f006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38220/0 (socket says 172.21.15.133:38220) 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.914+0000 7f22ee7fc640 1 -- 172.21.15.133:0/66194430 learned_addr learned my addr 172.21.15.133:0/66194430 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.914+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1372774381 0 0) 0x7f22f00c61d0 con 0x7f22f00b1f70 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22dc003530 con 0x7f22f00b1f70 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22ef7fe640 1 --2- 172.21.15.133:0/66194430 >> v2:172.21.15.162:3300/0 conn(0x7f22f00ce330 0x7f22f00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1620689374 0 0) 0x7f22dc003530 con 0x7f22f00b1f70 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 >> v2:172.21.15.162:3300/0 conn(0x7f22f00ce330 msgr2=0x7f22f00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 --2- 172.21.15.133:0/66194430 >> v2:172.21.15.162:3300/0 conn(0x7f22f00ce330 0x7f22f00ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 >> v1:172.21.15.162:6789/0 conn(0x7f22f015c9a0 legacy=0x7f22f00c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f006d4f0 con 0x7f22f00b1f70 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f22d8001680 con 0x7f22f00b1f70 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 >> v1:172.21.15.133:6789/0 conn(0x7f22f00b1f70 legacy=0x7f22f006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 --2- 172.21.15.133:0/66194430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22dc0039c0 0x7f22dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 --2- 172.21.15.133:0/66194430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22dc004810 0x7f22dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 --2- 172.21.15.133:0/66194430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc0051d0 0x7f22dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f00cecc0 con 0x7f22dc0051d0 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc00c120 con 0x7f22dc004810 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc00c320 con 0x7f22dc0039c0 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22ef7fe640 1 --2- 172.21.15.133:0/66194430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc0051d0 0x7f22dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22ee7fc640 1 --2- 172.21.15.133:0/66194430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22dc004810 0x7f22dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22eeffd640 1 --2- 172.21.15.133:0/66194430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22dc0039c0 0x7f22dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22ef7fe640 1 -- 172.21.15.133:0/66194430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22dc0039c0 msgr2=0x7f22dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22ef7fe640 1 --2- 172.21.15.133:0/66194430 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22dc0039c0 0x7f22dc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22ef7fe640 1 -- 172.21.15.133:0/66194430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22dc004810 msgr2=0x7f22dc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22ef7fe640 1 --2- 172.21.15.133:0/66194430 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22dc004810 0x7f22dc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22ef7fe640 1 -- 172.21.15.133:0/66194430 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc003530 con 0x7f22dc0051d0 2024-04-25T10:15:13.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22ef7fe640 1 -- 172.21.15.133:0/66194430 reap_dead start 2024-04-25T10:15:13.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22ef7fe640 1 --2- 172.21.15.133:0/66194430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc0051d0 0x7f22dc00b9d0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f22e800ba10 tx=0x7f22e801c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffee60ab76638cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e800b5e0 con 0x7f22dc0051d0 2024-04-25T10:15:13.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e80aa670 con 0x7f22dc0051d0 2024-04-25T10:15:13.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22edffb640 1 -- 172.21.15.133:0/66194430 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e80aaa60 con 0x7f22dc0051d0 2024-04-25T10:15:13.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22f533c640 1 -- 172.21.15.133:0/66194430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc0051d0 msgr2=0x7f22dc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22f533c640 1 --2- 172.21.15.133:0/66194430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc0051d0 0x7f22dc00b9d0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f22e800ba10 tx=0x7f22e801c6c0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22f533c640 1 -- 172.21.15.133:0/66194430 shutdown_connections 2024-04-25T10:15:13.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22f533c640 1 --2- 172.21.15.133:0/66194430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc0051d0 0x7f22dc00b9d0 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22f533c640 1 -- 172.21.15.133:0/66194430 >> 172.21.15.133:0/66194430 conn(0x7f22f0067000 msgr2=0x7f22f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22f533c640 1 -- 172.21.15.133:0/66194430 shutdown_connections 2024-04-25T10:15:13.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.918+0000 7f22f533c640 1 -- 172.21.15.133:0/66194430 wait complete. 2024-04-25T10:15:13.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22f533c640 1 Processor -- start 2024-04-25T10:15:13.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22f533c640 1 -- start start 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22f533c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22dc0051d0 0x7f22f014ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22f533c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f0158f00 0x7f22f01f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22f533c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22f015b330 0x7f22f015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22f533c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f015cf00 con 0x7f22f0158f00 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22f533c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f015cd80 con 0x7f22dc0051d0 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22f533c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f00ce780 con 0x7f22f015b330 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22ee7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f0158f00 0x7f22f01f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22ee7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f0158f00 0x7f22f01f1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37740/0 (socket says 172.21.15.133:37740) 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22ee7fc640 1 -- 172.21.15.133:0/1733488912 learned_addr learned my addr 172.21.15.133:0/1733488912 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22ef7fe640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22f015b330 0x7f22f015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22eeffd640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22dc0051d0 0x7f22f014ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22ee7fc640 1 -- 172.21.15.133:0/1733488912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22f015b330 msgr2=0x7f22f015b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22ee7fc640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22f015b330 0x7f22f015b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22ee7fc640 1 -- 172.21.15.133:0/1733488912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22dc0051d0 msgr2=0x7f22f014ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22ee7fc640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22dc0051d0 0x7f22f014ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22ee7fc640 1 -- 172.21.15.133:0/1733488912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f01513a0 con 0x7f22f0158f00 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22eeffd640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22dc0051d0 0x7f22f014ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22ef7fe640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22f015b330 0x7f22f015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22ee7fc640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f0158f00 0x7f22f01f1800 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f22d800e7e0 tx=0x7f22d800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22e77fe640 1 -- 172.21.15.133:0/1733488912 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d800c6c0 con 0x7f22f0158f00 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22e77fe640 1 -- 172.21.15.133:0/1733488912 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d8018070 con 0x7f22f0158f00 2024-04-25T10:15:13.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22f533c640 1 -- 172.21.15.133:0/1733488912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f01f2110 con 0x7f22f0158f00 2024-04-25T10:15:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22e77fe640 1 -- 172.21.15.133:0/1733488912 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d800ca80 con 0x7f22f0158f00 2024-04-25T10:15:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22f533c640 1 -- 172.21.15.133:0/1733488912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f01f26d0 con 0x7f22f0158f00 2024-04-25T10:15:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.922+0000 7f22f533c640 1 -- 172.21.15.133:0/1733488912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f0068ef0 con 0x7f22f0158f00 2024-04-25T10:15:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.926+0000 7f22e77fe640 1 -- 172.21.15.133:0/1733488912 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22d800cc20 con 0x7f22f0158f00 2024-04-25T10:15:13.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.926+0000 7f22e77fe640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f90 0x7f22d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.926+0000 7f22e77fe640 1 -- 172.21.15.133:0/1733488912 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f22d8084be0 con 0x7f22f0158f00 2024-04-25T10:15:13.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.926+0000 7f22eeffd640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f90 0x7f22d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.926+0000 7f22eeffd640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f90 0x7f22d0069450 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f22f015b120 tx=0x7f22e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:13.930+0000 7f22e77fe640 1 -- 172.21.15.133:0/1733488912 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22d8055a60 con 0x7f22f0158f00 2024-04-25T10:15:14.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.078+0000 7f22f533c640 1 -- 172.21.15.133:0/1733488912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22f00b3f50 con 0x7f22f0158f00 2024-04-25T10:15:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22e77fe640 1 -- 172.21.15.133:0/1733488912 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+16921 (secure 0 0 0) 0x7f22d8055360 con 0x7f22f0158f00 2024-04-25T10:15:14.084 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:14.085 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":156,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:15:13.097375+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:11.332375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:12.104303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 -- 172.21.15.133:0/1733488912 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f90 msgr2=0x7f22d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f90 0x7f22d0069450 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f22f015b120 tx=0x7f22e0009040 comp rx=0 tx=0).stop 2024-04-25T10:15:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 -- 172.21.15.133:0/1733488912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f0158f00 msgr2=0x7f22f01f1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f0158f00 0x7f22f01f1800 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f22d800e7e0 tx=0x7f22d800eca0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 -- 172.21.15.133:0/1733488912 shutdown_connections 2024-04-25T10:15:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22f015b330 0x7f22f015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22d0066f90 0x7f22d0069450 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22f0158f00 0x7f22f01f1800 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 --2- 172.21.15.133:0/1733488912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22dc0051d0 0x7f22f014ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 -- 172.21.15.133:0/1733488912 >> 172.21.15.133:0/1733488912 conn(0x7f22f0067000 msgr2=0x7f22f00c4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 -- 172.21.15.133:0/1733488912 shutdown_connections 2024-04-25T10:15:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.082+0000 7f22f533c640 1 -- 172.21.15.133:0/1733488912 wait complete. 2024-04-25T10:15:14.103 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68ee68f640 1 Processor -- start 2024-04-25T10:15:14.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68ee68f640 1 -- start start 2024-04-25T10:15:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68ee68f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68e815c9a0 0x7f68e80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68ee68f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f68e80b1f70 0x7f68e806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68ee68f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e80c61d0 con 0x7f68e80ce330 2024-04-25T10:15:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68ee68f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e80c64b0 con 0x7f68e80b1f70 2024-04-25T10:15:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68ee68f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e806d4f0 con 0x7f68e815c9a0 2024-04-25T10:15:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68ecc2c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f68e80ce330 0x7f68e815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38236/0 (socket says 172.21.15.133:38236) 2024-04-25T10:15:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68ecc2c640 1 -- 172.21.15.133:0/1695478462 learned_addr learned my addr 172.21.15.133:0/1695478462 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e77fe640 1 --2- 172.21.15.133:0/1695478462 >> v2:172.21.15.162:3300/0 conn(0x7f68e80b1f70 0x7f68e806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e7fff640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68e815c9a0 0x7f68e80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183845256 0 0) 0x7f68e80c61d0 con 0x7f68e80ce330 2024-04-25T10:15:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d0003530 con 0x7f68e80ce330 2024-04-25T10:15:14.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 59449054 0 0) 0x7f68d0003530 con 0x7f68e80ce330 2024-04-25T10:15:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68e815c9a0 msgr2=0x7f68e80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68e815c9a0 0x7f68e80c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 >> v2:172.21.15.162:3300/0 conn(0x7f68e80b1f70 msgr2=0x7f68e806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 --2- 172.21.15.133:0/1695478462 >> v2:172.21.15.162:3300/0 conn(0x7f68e80b1f70 0x7f68e806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e806d7d0 con 0x7f68e80ce330 2024-04-25T10:15:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f68d8001a00 con 0x7f68e80ce330 2024-04-25T10:15:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 >> v1:172.21.15.133:6789/0 conn(0x7f68e80ce330 legacy=0x7f68e815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68d00039c0 0x7f68d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68d0004810 0x7f68d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68e815c9a0 0x7f68d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80c64b0 con 0x7f68d0004810 2024-04-25T10:15:14.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e806d4f0 con 0x7f68d00039c0 2024-04-25T10:15:14.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d000b9d0 con 0x7f68e815c9a0 2024-04-25T10:15:14.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e77fe640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68d0004810 0x7f68d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e7fff640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68d00039c0 0x7f68d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68ecc2c640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68e815c9a0 0x7f68d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e77fe640 1 -- 172.21.15.133:0/1695478462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68e815c9a0 msgr2=0x7f68d00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e77fe640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68e815c9a0 0x7f68d00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e77fe640 1 -- 172.21.15.133:0/1695478462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68d00039c0 msgr2=0x7f68d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e77fe640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68d00039c0 0x7f68d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.218+0000 7f68e77fe640 1 -- 172.21.15.133:0/1695478462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d0003530 con 0x7f68d0004810 2024-04-25T10:15:14.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68e77fe640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68d0004810 0x7f68d0004c90 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009ae0 tx=0x7f68dc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b236a8cf7da240 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc0096b0 con 0x7f68d0004810 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc0a64c0 con 0x7f68d0004810 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68e6ffd640 1 -- 172.21.15.133:0/1695478462 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc0a68b0 con 0x7f68d0004810 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 -- 172.21.15.133:0/1695478462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68d0004810 msgr2=0x7f68d0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 --2- 172.21.15.133:0/1695478462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68d0004810 0x7f68d0004c90 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009ae0 tx=0x7f68dc01cab0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ecc2c640 1 -- 172.21.15.133:0/1695478462 reap_dead start 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 -- 172.21.15.133:0/1695478462 shutdown_connections 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 -- 172.21.15.133:0/1695478462 >> 172.21.15.133:0/1695478462 conn(0x7f68e8067000 msgr2=0x7f68e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 -- 172.21.15.133:0/1695478462 shutdown_connections 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 -- 172.21.15.133:0/1695478462 wait complete. 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 Processor -- start 2024-04-25T10:15:14.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 -- start start 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68d0004810 0x7f68e81f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68e814cf60 0x7f68e8157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e8158200 0x7f68e80c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e815cf00 con 0x7f68e8158200 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e815cd80 con 0x7f68e814cf60 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.222+0000 7f68ee68f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e815d850 con 0x7f68d0004810 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ecc2c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e8158200 0x7f68e80c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ecc2c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e8158200 0x7f68e80c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37764/0 (socket says 172.21.15.133:37764) 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ecc2c640 1 -- 172.21.15.133:0/44275679 learned_addr learned my addr 172.21.15.133:0/44275679 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68e7fff640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68d0004810 0x7f68e81f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ecc2c640 1 -- 172.21.15.133:0/44275679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68d0004810 msgr2=0x7f68e81f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68e77fe640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68e814cf60 0x7f68e8157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ecc2c640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68d0004810 0x7f68e81f24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ecc2c640 1 -- 172.21.15.133:0/44275679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68e814cf60 msgr2=0x7f68e8157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ecc2c640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68e814cf60 0x7f68e8157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ecc2c640 1 -- 172.21.15.133:0/44275679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e80a84d0 con 0x7f68e8158200 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ecc2c640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e8158200 0x7f68e80c95d0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f68e80cc0f0 tx=0x7f68d8011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68e4ff9640 1 -- 172.21.15.133:0/44275679 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8010040 con 0x7f68e8158200 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68e4ff9640 1 -- 172.21.15.133:0/44275679 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d801a070 con 0x7f68e8158200 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ee68f640 1 -- 172.21.15.133:0/44275679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e80a8770 con 0x7f68e8158200 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68e4ff9640 1 -- 172.21.15.133:0/44275679 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d800fb20 con 0x7f68e8158200 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ee68f640 1 -- 172.21.15.133:0/44275679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e80a8d30 con 0x7f68e8158200 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68ee68f640 1 -- 172.21.15.133:0/44275679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e8158680 con 0x7f68e8158200 2024-04-25T10:15:14.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68e4ff9640 1 -- 172.21.15.133:0/44275679 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68d800fcc0 con 0x7f68e8158200 2024-04-25T10:15:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68e4ff9640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68d4066f90 0x7f68d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68e7fff640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68d4066f90 0x7f68d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68e4ff9640 1 -- 172.21.15.133:0/44275679 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68d8087fd0 con 0x7f68e8158200 2024-04-25T10:15:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.226+0000 7f68e7fff640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68d4066f90 0x7f68d4069450 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009890 tx=0x7f68e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.230+0000 7f68e4ff9640 1 -- 172.21.15.133:0/44275679 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68d80132d0 con 0x7f68e8158200 2024-04-25T10:15:14.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.378+0000 7f68ee68f640 1 -- 172.21.15.133:0/44275679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68e8068ef0 con 0x7f68e8158200 2024-04-25T10:15:14.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.378+0000 7f68e4ff9640 1 -- 172.21.15.133:0/44275679 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+16921 (secure 0 0 0) 0x7f68d8058ee0 con 0x7f68e8158200 2024-04-25T10:15:14.384 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:14.384 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":156,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:15:13.097375+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:11.332375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:12.104303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:14.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 -- 172.21.15.133:0/44275679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68d4066f90 msgr2=0x7f68d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68d4066f90 0x7f68d4069450 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009890 tx=0x7f68e0008040 comp rx=0 tx=0).stop 2024-04-25T10:15:14.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 -- 172.21.15.133:0/44275679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e8158200 msgr2=0x7f68e80c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e8158200 0x7f68e80c95d0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f68e80cc0f0 tx=0x7f68d8011c20 comp rx=0 tx=0).stop 2024-04-25T10:15:14.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 -- 172.21.15.133:0/44275679 shutdown_connections 2024-04-25T10:15:14.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f68d4066f90 0x7f68d4069450 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68e8158200 0x7f68e80c95d0 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f68e814cf60 0x7f68e8157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 --2- 172.21.15.133:0/44275679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f68d0004810 0x7f68e81f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 -- 172.21.15.133:0/44275679 >> 172.21.15.133:0/44275679 conn(0x7f68e8067000 msgr2=0x7f68e80b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 -- 172.21.15.133:0/44275679 shutdown_connections 2024-04-25T10:15:14.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.382+0000 7f68ee68f640 1 -- 172.21.15.133:0/44275679 wait complete. 2024-04-25T10:15:14.402 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:15:14.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56c62a5640 1 Processor -- start 2024-04-25T10:15:14.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56c62a5640 1 -- start start 2024-04-25T10:15:14.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56c62a5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56c0142920 0x7f56c0143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56c62a5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c0140ea0 con 0x7f56c015d9f0 2024-04-25T10:15:14.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56c62a5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c00cf280 con 0x7f56c0140a40 2024-04-25T10:15:14.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56c62a5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c006d440 con 0x7f56c0142920 2024-04-25T10:15:14.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56bffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56c0142920 0x7f56c0143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56c4842640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f56c015d9f0 0x7f56c006df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38252/0 (socket says 172.21.15.133:38252) 2024-04-25T10:15:14.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56c4842640 1 -- 172.21.15.133:0/459554211 learned_addr learned my addr 172.21.15.133:0/459554211 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:14.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56bffff640 1 -- 172.21.15.133:0/459554211 >> v1:172.21.15.162:6789/0 conn(0x7f56c0140a40 legacy=0x7f56c00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56bffff640 1 -- 172.21.15.133:0/459554211 >> v1:172.21.15.133:6789/0 conn(0x7f56c015d9f0 legacy=0x7f56c006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.514+0000 7f56bffff640 1 -- 172.21.15.133:0/459554211 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c006d720 con 0x7f56c0142920 2024-04-25T10:15:14.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56bffff640 1 --2- 172.21.15.133:0/459554211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c0142920 0x7f56c0143560 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f56b00097c0 tx=0x7f56b001c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4dae48eca46fec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56bf7fe640 1 -- 172.21.15.133:0/459554211 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b00b0070 con 0x7f56c0142920 2024-04-25T10:15:14.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56bf7fe640 1 -- 172.21.15.133:0/459554211 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b00ab600 con 0x7f56c0142920 2024-04-25T10:15:14.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56bf7fe640 1 -- 172.21.15.133:0/459554211 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b00ab9d0 con 0x7f56c0142920 2024-04-25T10:15:14.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 -- 172.21.15.133:0/459554211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c0142920 msgr2=0x7f56c0143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 --2- 172.21.15.133:0/459554211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c0142920 0x7f56c0143560 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f56b00097c0 tx=0x7f56b001c2a0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 -- 172.21.15.133:0/459554211 shutdown_connections 2024-04-25T10:15:14.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 --2- 172.21.15.133:0/459554211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c0142920 0x7f56c0143560 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 -- 172.21.15.133:0/459554211 >> 172.21.15.133:0/459554211 conn(0x7f56c0066f70 msgr2=0x7f56c006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 -- 172.21.15.133:0/459554211 shutdown_connections 2024-04-25T10:15:14.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 -- 172.21.15.133:0/459554211 wait complete. 2024-04-25T10:15:14.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 Processor -- start 2024-04-25T10:15:14.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 -- start start 2024-04-25T10:15:14.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56c0140a40 0x7f56c015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c0142920 0x7f56c01321d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c015d9f0 0x7f56c01413d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c0142e80 con 0x7f56c015d9f0 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c0142d00 con 0x7f56c0140a40 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c62a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c015de30 con 0x7f56c0142920 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c4842640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c015d9f0 0x7f56c01413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c4842640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c015d9f0 0x7f56c01413d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37778/0 (socket says 172.21.15.133:37778) 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c4842640 1 -- 172.21.15.133:0/2484496141 learned_addr learned my addr 172.21.15.133:0/2484496141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56bffff640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56c0140a40 0x7f56c015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56b7fff640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c0142920 0x7f56c01321d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c4842640 1 -- 172.21.15.133:0/2484496141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c0142920 msgr2=0x7f56c01321d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c4842640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c0142920 0x7f56c01321d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c4842640 1 -- 172.21.15.133:0/2484496141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56c0140a40 msgr2=0x7f56c015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c4842640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56c0140a40 0x7f56c015cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c4842640 1 -- 172.21.15.133:0/2484496141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c00b2550 con 0x7f56c015d9f0 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56b7fff640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c0142920 0x7f56c01321d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:14.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.518+0000 7f56c4842640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c015d9f0 0x7f56c01413d0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f56b800e550 tx=0x7f56b800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.522+0000 7f56bd7fa640 1 -- 172.21.15.133:0/2484496141 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8009500 con 0x7f56c015d9f0 2024-04-25T10:15:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.522+0000 7f56bd7fa640 1 -- 172.21.15.133:0/2484496141 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b8018070 con 0x7f56c015d9f0 2024-04-25T10:15:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.522+0000 7f56bd7fa640 1 -- 172.21.15.133:0/2484496141 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80098e0 con 0x7f56c015d9f0 2024-04-25T10:15:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.522+0000 7f56c62a5640 1 -- 172.21.15.133:0/2484496141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c014fcb0 con 0x7f56c015d9f0 2024-04-25T10:15:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.522+0000 7f56c62a5640 1 -- 172.21.15.133:0/2484496141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c0150270 con 0x7f56c015d9f0 2024-04-25T10:15:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.522+0000 7f56c62a5640 1 -- 172.21.15.133:0/2484496141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c00b2720 con 0x7f56c015d9f0 2024-04-25T10:15:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.522+0000 7f56bd7fa640 1 -- 172.21.15.133:0/2484496141 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56b8013480 con 0x7f56c015d9f0 2024-04-25T10:15:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.522+0000 7f56bd7fa640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a0066f60 0x7f56a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.522+0000 7f56bd7fa640 1 -- 172.21.15.133:0/2484496141 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56b80882a0 con 0x7f56c015d9f0 2024-04-25T10:15:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.522+0000 7f56bffff640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a0066f60 0x7f56a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.522+0000 7f56bffff640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a0066f60 0x7f56a0069420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f56b00ad000 tx=0x7f56b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.526+0000 7f56bd7fa640 1 -- 172.21.15.133:0/2484496141 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f56b80591b0 con 0x7f56c015d9f0 2024-04-25T10:15:14.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.706+0000 7f56c62a5640 1 -- 172.21.15.133:0/2484496141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f56c00afa70 con 0x7f56c015d9f0 2024-04-25T10:15:14.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.762+0000 7f56bd7fa640 1 -- 172.21.15.133:0/2484496141 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) v1 ==== 118+0+0 (secure 0 0 0) 0x7f56b8058ab0 con 0x7f56c015d9f0 2024-04-25T10:15:14.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 -- 172.21.15.133:0/2484496141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a0066f60 msgr2=0x7f56a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a0066f60 0x7f56a0069420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f56b00ad000 tx=0x7f56b0008040 comp rx=0 tx=0).stop 2024-04-25T10:15:14.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 -- 172.21.15.133:0/2484496141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c015d9f0 msgr2=0x7f56c01413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c015d9f0 0x7f56c01413d0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f56b800e550 tx=0x7f56b800ea10 comp rx=0 tx=0).stop 2024-04-25T10:15:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 -- 172.21.15.133:0/2484496141 shutdown_connections 2024-04-25T10:15:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56a0066f60 0x7f56a0069420 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56c015d9f0 0x7f56c01413d0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f56c0142920 0x7f56c01321d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 --2- 172.21.15.133:0/2484496141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56c0140a40 0x7f56c015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 -- 172.21.15.133:0/2484496141 >> 172.21.15.133:0/2484496141 conn(0x7f56c0066f70 msgr2=0x7f56c0138390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 -- 172.21.15.133:0/2484496141 shutdown_connections 2024-04-25T10:15:14.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.766+0000 7f56c62a5640 1 -- 172.21.15.133:0/2484496141 wait complete. 2024-04-25T10:15:14.788 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:15:14.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2ce23b9640 1 Processor -- start 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2ce23b9640 1 -- start start 2024-04-25T10:15:14.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2ce23b9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cdc145170 0x7f2cdc06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2ce23b9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2cdc06e4d0 0x7f2cdc0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2ce23b9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2cdc0cdd60 0x7f2cdc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2ce23b9640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc06db00 con 0x7f2cdc06e4d0 2024-04-25T10:15:14.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2ce23b9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc145550 con 0x7f2cdc0cdd60 2024-04-25T10:15:14.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2ce23b9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0ce2e0 con 0x7f2cdc145170 2024-04-25T10:15:14.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2cdb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2cdc06e4d0 0x7f2cdc0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2cdb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2cdc06e4d0 0x7f2cdc0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37788/0 (socket says 172.21.15.133:37788) 2024-04-25T10:15:14.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2cdb7fe640 1 -- 172.21.15.133:0/2157820755 learned_addr learned my addr 172.21.15.133:0/2157820755 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:14.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2cdbfff640 1 --2- 172.21.15.133:0/2157820755 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cdc145170 0x7f2cdc06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2cdb7fe640 1 -- 172.21.15.133:0/2157820755 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cdc145170 msgr2=0x7f2cdc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2cdb7fe640 1 --2- 172.21.15.133:0/2157820755 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cdc145170 0x7f2cdc06df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2ce0956640 1 --2- 172.21.15.133:0/2157820755 >> v2:172.21.15.162:3300/0 conn(0x7f2cdc0cdd60 0x7f2cdc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2cdb7fe640 1 -- 172.21.15.133:0/2157820755 >> v2:172.21.15.162:3300/0 conn(0x7f2cdc0cdd60 msgr2=0x7f2cdc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2cdb7fe640 1 --2- 172.21.15.133:0/2157820755 >> v2:172.21.15.162:3300/0 conn(0x7f2cdc0cdd60 0x7f2cdc15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.910+0000 7f2cdb7fe640 1 -- 172.21.15.133:0/2157820755 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc06d4b0 con 0x7f2cdc06e4d0 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2cdb7fe640 1 --2- 172.21.15.133:0/2157820755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cdc06e4d0 0x7f2cdc0cd820 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0a8550 tx=0x7f2ccc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c38444c1370824 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2cdaffd640 1 -- 172.21.15.133:0/2157820755 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0b0070 con 0x7f2cdc06e4d0 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2cdaffd640 1 -- 172.21.15.133:0/2157820755 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc0ab600 con 0x7f2cdc06e4d0 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2cdaffd640 1 -- 172.21.15.133:0/2157820755 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0ab9d0 con 0x7f2cdc06e4d0 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 -- 172.21.15.133:0/2157820755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cdc06e4d0 msgr2=0x7f2cdc0cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 --2- 172.21.15.133:0/2157820755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cdc06e4d0 0x7f2cdc0cd820 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0a8550 tx=0x7f2ccc01c520 comp rx=0 tx=0).stop 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 -- 172.21.15.133:0/2157820755 shutdown_connections 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 --2- 172.21.15.133:0/2157820755 >> v2:172.21.15.162:3300/0 conn(0x7f2cdc0cdd60 0x7f2cdc15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 --2- 172.21.15.133:0/2157820755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cdc06e4d0 0x7f2cdc0cd820 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 --2- 172.21.15.133:0/2157820755 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cdc145170 0x7f2cdc06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 -- 172.21.15.133:0/2157820755 >> 172.21.15.133:0/2157820755 conn(0x7f2cdc066f70 msgr2=0x7f2cdc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 -- 172.21.15.133:0/2157820755 shutdown_connections 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 -- 172.21.15.133:0/2157820755 wait complete. 2024-04-25T10:15:14.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 Processor -- start 2024-04-25T10:15:14.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 -- start start 2024-04-25T10:15:14.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cdc06e4d0 0x7f2cdc13a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cdc0cdd60 0x7f2cdc163b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cdc145170 0x7f2cdc165010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc15f690 con 0x7f2cdc145170 2024-04-25T10:15:14.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc15f510 con 0x7f2cdc06e4d0 2024-04-25T10:15:14.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce23b9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc15f810 con 0x7f2cdc0cdd60 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce0956640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cdc145170 0x7f2cdc165010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce0956640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cdc145170 0x7f2cdc165010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37800/0 (socket says 172.21.15.133:37800) 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2ce0956640 1 -- 172.21.15.133:0/4023717611 learned_addr learned my addr 172.21.15.133:0/4023717611 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2cdbfff640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cdc06e4d0 0x7f2cdc13a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.914+0000 7f2cdb7fe640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cdc0cdd60 0x7f2cdc163b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2ce0956640 1 -- 172.21.15.133:0/4023717611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cdc0cdd60 msgr2=0x7f2cdc163b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2ce0956640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cdc0cdd60 0x7f2cdc163b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2ce0956640 1 -- 172.21.15.133:0/4023717611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cdc06e4d0 msgr2=0x7f2cdc13a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2ce0956640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cdc06e4d0 0x7f2cdc13a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2ce0956640 1 -- 172.21.15.133:0/4023717611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc1644f0 con 0x7f2cdc145170 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2cdb7fe640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cdc0cdd60 0x7f2cdc163b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:14.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2cdbfff640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cdc06e4d0 0x7f2cdc13a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:14.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2ce0956640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cdc145170 0x7f2cdc165010 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400ccf0 tx=0x7f2cd400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2cd8ff9640 1 -- 172.21.15.133:0/4023717611 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4003ba0 con 0x7f2cdc145170 2024-04-25T10:15:14.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2cd8ff9640 1 -- 172.21.15.133:0/4023717611 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd4003d40 con 0x7f2cdc145170 2024-04-25T10:15:14.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2ce23b9640 1 -- 172.21.15.133:0/4023717611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc1647d0 con 0x7f2cdc145170 2024-04-25T10:15:14.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2cd8ff9640 1 -- 172.21.15.133:0/4023717611 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4004240 con 0x7f2cdc145170 2024-04-25T10:15:14.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2ce23b9640 1 -- 172.21.15.133:0/4023717611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc0a8990 con 0x7f2cdc145170 2024-04-25T10:15:14.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2ce23b9640 1 -- 172.21.15.133:0/4023717611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc0a87a0 con 0x7f2cdc145170 2024-04-25T10:15:14.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2cd8ff9640 1 -- 172.21.15.133:0/4023717611 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2cd40043e0 con 0x7f2cdc145170 2024-04-25T10:15:14.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2cd8ff9640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cbc066f90 0x7f2cbc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2cdbfff640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cbc066f90 0x7f2cbc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2cd8ff9640 1 -- 172.21.15.133:0/4023717611 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2cd4085cd0 con 0x7f2cdc145170 2024-04-25T10:15:14.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.918+0000 7f2cdbfff640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cbc066f90 0x7f2cbc069450 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4006fd0 tx=0x7f2cc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:14.922+0000 7f2cd8ff9640 1 -- 172.21.15.133:0/4023717611 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2cd401d340 con 0x7f2cdc145170 2024-04-25T10:15:15.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.122+0000 7f2ce23b9640 1 -- 172.21.15.133:0/4023717611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2cdc141ea0 con 0x7f2cdc145170 2024-04-25T10:15:15.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:15:15.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.766+0000 7f2cd8ff9640 1 -- 172.21.15.133:0/4023717611 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2cd4056ac0 con 0x7f2cdc145170 2024-04-25T10:15:15.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 -- 172.21.15.133:0/4023717611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cbc066f90 msgr2=0x7f2cbc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cbc066f90 0x7f2cbc069450 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4006fd0 tx=0x7f2cc4008040 comp rx=0 tx=0).stop 2024-04-25T10:15:15.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 -- 172.21.15.133:0/4023717611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cdc145170 msgr2=0x7f2cdc165010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cdc145170 0x7f2cdc165010 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400ccf0 tx=0x7f2cd400a660 comp rx=0 tx=0).stop 2024-04-25T10:15:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 -- 172.21.15.133:0/4023717611 shutdown_connections 2024-04-25T10:15:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2cbc066f90 0x7f2cbc069450 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cdc145170 0x7f2cdc165010 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2cdc0cdd60 0x7f2cdc163b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 --2- 172.21.15.133:0/4023717611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2cdc06e4d0 0x7f2cdc13a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 -- 172.21.15.133:0/4023717611 >> 172.21.15.133:0/4023717611 conn(0x7f2cdc066f70 msgr2=0x7f2cdc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 -- 172.21.15.133:0/4023717611 shutdown_connections 2024-04-25T10:15:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.770+0000 7f2ce23b9640 1 -- 172.21.15.133:0/4023717611 wait complete. 2024-04-25T10:15:15.797 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:15:15.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:15.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 Processor -- start 2024-04-25T10:15:15.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- start start 2024-04-25T10:15:15.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd6780cfc70 0x7fd6780d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 0x7fd6780c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67815bd90 con 0x7fd67815b9b0 2024-04-25T10:15:15.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6780d0610 con 0x7fd6780cfc70 2024-04-25T10:15:15.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780c5610 con 0x7fd6780c5aa0 2024-04-25T10:15:15.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd6780cfc70 0x7fd6780d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd6780cfc70 0x7fd6780d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37804/0 (socket says 172.21.15.133:37804) 2024-04-25T10:15:15.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 -- 172.21.15.133:0/1393657090 learned_addr learned my addr 172.21.15.133:0/1393657090 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:15.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd677fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd67815b9b0 0x7fd6780cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38254/0 (socket says 172.21.15.133:38254) 2024-04-25T10:15:15.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 -- 172.21.15.133:0/1393657090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 msgr2=0x7fd6780c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:15.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67cadb640 1 --2- 172.21.15.133:0/1393657090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 0x7fd6780c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 --2- 172.21.15.133:0/1393657090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 0x7fd6780c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd676ffd640 1 -- 172.21.15.133:0/1393657090 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779839944 0 0) 0x7fd67815bd90 con 0x7fd67815b9b0 2024-04-25T10:15:15.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 -- 172.21.15.133:0/1393657090 >> v1:172.21.15.133:6789/0 conn(0x7fd67815b9b0 legacy=0x7fd6780cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 -- 172.21.15.133:0/1393657090 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67809ad10 con 0x7fd6780cfc70 2024-04-25T10:15:15.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67cadb640 1 --2- 172.21.15.133:0/1393657090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 0x7fd6780c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:15.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 --2- 172.21.15.133:0/1393657090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6780cfc70 0x7fd6780d00d0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009870 tx=0x7fd66c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d48dfe801caa3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd676ffd640 1 -- 172.21.15.133:0/1393657090 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0a95f0 con 0x7fd6780cfc70 2024-04-25T10:15:15.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd676ffd640 1 -- 172.21.15.133:0/1393657090 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c0a9790 con 0x7fd6780cfc70 2024-04-25T10:15:15.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd676ffd640 1 -- 172.21.15.133:0/1393657090 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0a9b60 con 0x7fd6780cfc70 2024-04-25T10:15:15.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- 172.21.15.133:0/1393657090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6780cfc70 msgr2=0x7fd6780d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 --2- 172.21.15.133:0/1393657090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6780cfc70 0x7fd6780d00d0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009870 tx=0x7fd66c01c530 comp rx=0 tx=0).stop 2024-04-25T10:15:15.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- 172.21.15.133:0/1393657090 shutdown_connections 2024-04-25T10:15:15.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 --2- 172.21.15.133:0/1393657090 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 0x7fd6780c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 --2- 172.21.15.133:0/1393657090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6780cfc70 0x7fd6780d00d0 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- 172.21.15.133:0/1393657090 >> 172.21.15.133:0/1393657090 conn(0x7fd6780b31a0 msgr2=0x7fd6780b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- 172.21.15.133:0/1393657090 shutdown_connections 2024-04-25T10:15:15.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- 172.21.15.133:0/1393657090 wait complete. 2024-04-25T10:15:15.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 Processor -- start 2024-04-25T10:15:15.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- start start 2024-04-25T10:15:15.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 0x7fd67806cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6780cfc70 0x7fd678152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd67815b9b0 0x7fd678155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67809a3e0 con 0x7fd6780cfc70 2024-04-25T10:15:15.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780cf990 con 0x7fd67815b9b0 2024-04-25T10:15:15.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781684e0 con 0x7fd6780c5aa0 2024-04-25T10:15:15.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6780cfc70 0x7fd678152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6780cfc70 0x7fd678152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37808/0 (socket says 172.21.15.133:37808) 2024-04-25T10:15:15.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 -- 172.21.15.133:0/396774647 learned_addr learned my addr 172.21.15.133:0/396774647 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:15.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd677fff640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 0x7fd67806cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67cadb640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd67815b9b0 0x7fd678155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 -- 172.21.15.133:0/396774647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 msgr2=0x7fd67806cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 0x7fd67806cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 -- 172.21.15.133:0/396774647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd67815b9b0 msgr2=0x7fd678155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd67815b9b0 0x7fd678155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 -- 172.21.15.133:0/396774647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67806dd60 con 0x7fd6780cfc70 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd677fff640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 0x7fd67806cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd6777fe640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6780cfc70 0x7fd678152ed0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fd66c01a160 tx=0x7fd66c0192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd674ff9640 1 -- 172.21.15.133:0/396774647 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0b1800 con 0x7fd6780cfc70 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd674ff9640 1 -- 172.21.15.133:0/396774647 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c0b19a0 con 0x7fd6780cfc70 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- 172.21.15.133:0/396774647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67806be40 con 0x7fd6780cfc70 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd674ff9640 1 -- 172.21.15.133:0/396774647 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0b1d90 con 0x7fd6780cfc70 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.926+0000 7fd67e53e640 1 -- 172.21.15.133:0/396774647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67806c400 con 0x7fd6780cfc70 2024-04-25T10:15:15.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.930+0000 7fd67e53e640 1 -- 172.21.15.133:0/396774647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd678067c70 con 0x7fd6780cfc70 2024-04-25T10:15:15.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.930+0000 7fd674ff9640 1 -- 172.21.15.133:0/396774647 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6780c6c80 con 0x7fd6780cfc70 2024-04-25T10:15:15.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.930+0000 7fd674ff9640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd664066f60 0x7fd664069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.930+0000 7fd674ff9640 1 -- 172.21.15.133:0/396774647 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd66c11f830 con 0x7fd6780cfc70 2024-04-25T10:15:15.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.930+0000 7fd677fff640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd664066f60 0x7fd664069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.930+0000 7fd677fff640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd664066f60 0x7fd664069420 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd668000f70 tx=0x7fd668009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:15.934+0000 7fd674ff9640 1 -- 172.21.15.133:0/396774647 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd66c0ad020 con 0x7fd6780cfc70 2024-04-25T10:15:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.126+0000 7fd67e53e640 1 -- 172.21.15.133:0/396774647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6780b2e40 con 0x7fd6780cfc70 2024-04-25T10:15:16.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd674ff9640 1 -- 172.21.15.133:0/396774647 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4959 (secure 0 0 0) 0x7fd66c016150 con 0x7fd6780cfc70 2024-04-25T10:15:16.132 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:16.132 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3454110484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3454110484},{"type":"v1","addr":"172.21.15.162:6835","nonce":3454110484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3345416592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3345416592},{"type":"v1","addr":"172.21.15.133:6835","nonce":3345416592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2924224565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2924224565},{"type":"v1","addr":"172.21.15.133:6837","nonce":2924224565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:13.097799+0000","modified":"2024-04-25T10:15:15.128366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10554":{"gid":10554,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/3763981385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3763981385},{"type":"v1","addr":"172.21.15.162:6833","nonce":3763981385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:15:16.133 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 99 2024-04-25T10:15:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 -- 172.21.15.133:0/396774647 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd664066f60 msgr2=0x7fd664069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd664066f60 0x7fd664069420 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd668000f70 tx=0x7fd668009040 comp rx=0 tx=0).stop 2024-04-25T10:15:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 -- 172.21.15.133:0/396774647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6780cfc70 msgr2=0x7fd678152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6780cfc70 0x7fd678152ed0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fd66c01a160 tx=0x7fd66c0192d0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 -- 172.21.15.133:0/396774647 shutdown_connections 2024-04-25T10:15:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd67815b9b0 0x7fd678155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd664066f60 0x7fd664069420 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6780cfc70 0x7fd678152ed0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 --2- 172.21.15.133:0/396774647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6780c5aa0 0x7fd67806cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 -- 172.21.15.133:0/396774647 >> 172.21.15.133:0/396774647 conn(0x7fd6780b31a0 msgr2=0x7fd6780cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 -- 172.21.15.133:0/396774647 shutdown_connections 2024-04-25T10:15:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.134+0000 7fd67e53e640 1 -- 172.21.15.133:0/396774647 wait complete. 2024-04-25T10:15:16.152 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:16.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 Processor -- start 2024-04-25T10:15:16.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- start start 2024-04-25T10:15:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f287015c9a0 0x7f28700c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28700c61d0 con 0x7f28700b1f70 2024-04-25T10:15:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28700c64b0 con 0x7f28700ce330 2024-04-25T10:15:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287006d4f0 con 0x7f287015c9a0 2024-04-25T10:15:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f287015c9a0 0x7f28700c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286effd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f28700b1f70 0x7f287006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58584/0 (socket says 172.21.15.133:58584) 2024-04-25T10:15:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286effd640 1 -- 172.21.15.133:0/4126592195 learned_addr learned my addr 172.21.15.133:0/4126592195 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286e7fc640 1 -- 172.21.15.133:0/4126592195 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945329024 0 0) 0x7f28700c61d0 con 0x7f28700b1f70 2024-04-25T10:15:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286e7fc640 1 -- 172.21.15.133:0/4126592195 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2858003530 con 0x7f28700b1f70 2024-04-25T10:15:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286f7fe640 1 -- 172.21.15.133:0/4126592195 >> v1:172.21.15.162:6789/0 conn(0x7f28700ce330 legacy=0x7f287015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286f7fe640 1 -- 172.21.15.133:0/4126592195 >> v1:172.21.15.133:6789/0 conn(0x7f28700b1f70 legacy=0x7f287006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286f7fe640 1 -- 172.21.15.133:0/4126592195 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287006d7d0 con 0x7f287015c9a0 2024-04-25T10:15:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286f7fe640 1 --2- 172.21.15.133:0/4126592195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287015c9a0 0x7f28700c5980 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f28600097c0 tx=0x7f286001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434b36c0f0c3a0c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286e7fc640 1 -- 172.21.15.133:0/4126592195 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600b0070 con 0x7f287015c9a0 2024-04-25T10:15:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286e7fc640 1 -- 172.21.15.133:0/4126592195 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28600ab600 con 0x7f287015c9a0 2024-04-25T10:15:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286e7fc640 1 -- 172.21.15.133:0/4126592195 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600ab9d0 con 0x7f287015c9a0 2024-04-25T10:15:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- 172.21.15.133:0/4126592195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287015c9a0 msgr2=0x7f28700c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 --2- 172.21.15.133:0/4126592195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287015c9a0 0x7f28700c5980 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f28600097c0 tx=0x7f286001c460 comp rx=0 tx=0).stop 2024-04-25T10:15:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- 172.21.15.133:0/4126592195 shutdown_connections 2024-04-25T10:15:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 --2- 172.21.15.133:0/4126592195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f287015c9a0 0x7f28700c5980 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- 172.21.15.133:0/4126592195 >> 172.21.15.133:0/4126592195 conn(0x7f2870067000 msgr2=0x7f287006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- 172.21.15.133:0/4126592195 shutdown_connections 2024-04-25T10:15:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- 172.21.15.133:0/4126592195 wait complete. 2024-04-25T10:15:16.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 Processor -- start 2024-04-25T10:15:16.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- start start 2024-04-25T10:15:16.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28700b1f70 0x7f287015b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28700ce330 0x7f287014baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f287015c9a0 0x7f28701474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287015cf00 con 0x7f28700ce330 2024-04-25T10:15:16.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287015cd80 con 0x7f287015c9a0 2024-04-25T10:15:16.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f2876231640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2870169de0 con 0x7f28700b1f70 2024-04-25T10:15:16.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28700ce330 0x7f287014baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28700ce330 0x7f287014baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54024/0 (socket says 172.21.15.133:54024) 2024-04-25T10:15:16.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286effd640 1 -- 172.21.15.133:0/55065371 learned_addr learned my addr 172.21.15.133:0/55065371 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:16.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286ffff640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f287015c9a0 0x7f28701474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.274+0000 7f286f7fe640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28700b1f70 0x7f287015b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f286effd640 1 -- 172.21.15.133:0/55065371 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28700b1f70 msgr2=0x7f287015b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f286effd640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28700b1f70 0x7f287015b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f286effd640 1 -- 172.21.15.133:0/55065371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f287015c9a0 msgr2=0x7f28701474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f286effd640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f287015c9a0 0x7f28701474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f286effd640 1 -- 172.21.15.133:0/55065371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287015c0a0 con 0x7f28700ce330 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f286f7fe640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28700b1f70 0x7f287015b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f286effd640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28700ce330 0x7f287014baf0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f286400c3c0 tx=0x7f286400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f2857fff640 1 -- 172.21.15.133:0/55065371 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864017070 con 0x7f28700ce330 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f2857fff640 1 -- 172.21.15.133:0/55065371 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2864005cf0 con 0x7f28700ce330 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f2876231640 1 -- 172.21.15.133:0/55065371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287014ad30 con 0x7f28700ce330 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f2857fff640 1 -- 172.21.15.133:0/55065371 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28640072f0 con 0x7f28700ce330 2024-04-25T10:15:16.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f2876231640 1 -- 172.21.15.133:0/55065371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287014b2f0 con 0x7f28700ce330 2024-04-25T10:15:16.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f2876231640 1 -- 172.21.15.133:0/55065371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2870068ef0 con 0x7f28700ce330 2024-04-25T10:15:16.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f2857fff640 1 -- 172.21.15.133:0/55065371 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f286401a070 con 0x7f28700ce330 2024-04-25T10:15:16.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f2857fff640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f285c066f90 0x7f285c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f2857fff640 1 -- 172.21.15.133:0/55065371 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2864084580 con 0x7f28700ce330 2024-04-25T10:15:16.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f286f7fe640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f285c066f90 0x7f285c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.278+0000 7f286f7fe640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f285c066f90 0x7f285c069450 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f28600ad000 tx=0x7f2860008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.282+0000 7f2857fff640 1 -- 172.21.15.133:0/55065371 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2864055400 con 0x7f28700ce330 2024-04-25T10:15:16.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.442+0000 7f2876231640 1 -- 172.21.15.133:0/55065371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28701479e0 con 0x7f28700ce330 2024-04-25T10:15:16.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2857fff640 1 -- 172.21.15.133:0/55065371 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16908 (secure 0 0 0) 0x7f2864054d00 con 0x7f28700ce330 2024-04-25T10:15:16.449 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:16.449 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":157,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:15:14.762327+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:11.332375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:12.104303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:16.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 -- 172.21.15.133:0/55065371 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f285c066f90 msgr2=0x7f285c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f285c066f90 0x7f285c069450 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f28600ad000 tx=0x7f2860008040 comp rx=0 tx=0).stop 2024-04-25T10:15:16.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 -- 172.21.15.133:0/55065371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28700ce330 msgr2=0x7f287014baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28700ce330 0x7f287014baf0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f286400c3c0 tx=0x7f286400c880 comp rx=0 tx=0).stop 2024-04-25T10:15:16.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 -- 172.21.15.133:0/55065371 shutdown_connections 2024-04-25T10:15:16.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f287015c9a0 0x7f28701474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f285c066f90 0x7f285c069450 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28700ce330 0x7f287014baf0 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 --2- 172.21.15.133:0/55065371 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f28700b1f70 0x7f287015b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 -- 172.21.15.133:0/55065371 >> 172.21.15.133:0/55065371 conn(0x7f2870067000 msgr2=0x7f28700b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 -- 172.21.15.133:0/55065371 shutdown_connections 2024-04-25T10:15:16.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.450+0000 7f2876231640 1 -- 172.21.15.133:0/55065371 wait complete. 2024-04-25T10:15:16.470 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:15:16.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc31d542640 1 Processor -- start 2024-04-25T10:15:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc31d542640 1 -- start start 2024-04-25T10:15:16.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc31d542640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3180d0bd0 0x7fc3180b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc31d542640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3180ceb20 con 0x7fc3180d0770 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc31d542640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3180b30a0 con 0x7fc31815db00 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc31d542640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180b7740 con 0x7fc3180d0bd0 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc30e7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc3180d0770 0x7fc3180b87e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58596/0 (socket says 172.21.15.133:58596) 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc30e7fc640 1 -- 172.21.15.133:0/2549149939 learned_addr learned my addr 172.21.15.133:0/2549149939 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3177fe640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3180d0bd0 0x7fc3180b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197924732 0 0) 0x7fc3180ceb20 con 0x7fc3180d0770 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc304003530 con 0x7fc3180d0770 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4037917727 0 0) 0x7fc304003530 con 0x7fc3180d0770 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3180d0bd0 msgr2=0x7fc3180b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3180d0bd0 0x7fc3180b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 >> v1:172.21.15.162:6789/0 conn(0x7fc31815db00 legacy=0x7fc3180b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3180b7a20 con 0x7fc3180d0770 2024-04-25T10:15:16.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fc300001680 con 0x7fc3180d0770 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 >> v1:172.21.15.133:6789/0 conn(0x7fc3180d0770 legacy=0x7fc3180b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3040039f0 0x7fc304003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3180d0bd0 0x7fc304004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc304004d50 0x7fc3040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180b7740 con 0x7fc3040039f0 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30400bab0 con 0x7fc304004d50 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30400bcb0 con 0x7fc3180d0bd0 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc316ffd640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3040039f0 0x7fc304003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc3177fe640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc304004d50 0x7fc3040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc30e7fc640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3180d0bd0 0x7fc304004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc316ffd640 1 -- 172.21.15.133:0/2549149939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3180d0bd0 msgr2=0x7fc304004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc316ffd640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc3180d0bd0 0x7fc304004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc316ffd640 1 -- 172.21.15.133:0/2549149939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc304004d50 msgr2=0x7fc3040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc316ffd640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc304004d50 0x7fc3040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.586+0000 7fc316ffd640 1 -- 172.21.15.133:0/2549149939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc304003530 con 0x7fc3040039f0 2024-04-25T10:15:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc316ffd640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3040039f0 0x7fc304003e10 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fc308001750 tx=0x7fc30801bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141af9f89db6c19b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3080013a0 con 0x7fc3040039f0 2024-04-25T10:15:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3080a4ce0 con 0x7fc3040039f0 2024-04-25T10:15:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc3167fc640 1 -- 172.21.15.133:0/2549149939 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308005310 con 0x7fc3040039f0 2024-04-25T10:15:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 -- 172.21.15.133:0/2549149939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3040039f0 msgr2=0x7fc304003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 --2- 172.21.15.133:0/2549149939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3040039f0 0x7fc304003e10 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fc308001750 tx=0x7fc30801bf10 comp rx=0 tx=0).stop 2024-04-25T10:15:16.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc3177fe640 1 -- 172.21.15.133:0/2549149939 reap_dead start 2024-04-25T10:15:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 -- 172.21.15.133:0/2549149939 shutdown_connections 2024-04-25T10:15:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 -- 172.21.15.133:0/2549149939 >> 172.21.15.133:0/2549149939 conn(0x7fc3180727c0 msgr2=0x7fc318072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 -- 172.21.15.133:0/2549149939 shutdown_connections 2024-04-25T10:15:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 -- 172.21.15.133:0/2549149939 wait complete. 2024-04-25T10:15:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 Processor -- start 2024-04-25T10:15:16.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 -- start start 2024-04-25T10:15:16.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3040039f0 0x7fc31814ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc318165e40 0x7fc3181662c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318161ed0 0x7fc318162350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180d0fe0 con 0x7fc318161ed0 2024-04-25T10:15:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180b7430 con 0x7fc3040039f0 2024-04-25T10:15:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc31d542640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180d18a0 con 0x7fc318165e40 2024-04-25T10:15:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc3177fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318161ed0 0x7fc318162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc3177fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318161ed0 0x7fc318162350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54044/0 (socket says 172.21.15.133:54044) 2024-04-25T10:15:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc3177fe640 1 -- 172.21.15.133:0/3634201714 learned_addr learned my addr 172.21.15.133:0/3634201714 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.590+0000 7fc30e7fc640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc318165e40 0x7fc3181662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc316ffd640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3040039f0 0x7fc31814ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc3177fe640 1 -- 172.21.15.133:0/3634201714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc318165e40 msgr2=0x7fc3181662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc3177fe640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc318165e40 0x7fc3181662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc3177fe640 1 -- 172.21.15.133:0/3634201714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3040039f0 msgr2=0x7fc31814ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc3177fe640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3040039f0 0x7fc31814ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc3177fe640 1 -- 172.21.15.133:0/3634201714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc318162c50 con 0x7fc318161ed0 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc30e7fc640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc318165e40 0x7fc3181662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc3177fe640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318161ed0 0x7fc318162350 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fc3180b5840 tx=0x7fc3100115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc30ffff640 1 -- 172.21.15.133:0/3634201714 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310011e10 con 0x7fc318161ed0 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc30ffff640 1 -- 172.21.15.133:0/3634201714 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31001b070 con 0x7fc318161ed0 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc31d542640 1 -- 172.21.15.133:0/3634201714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc318162f30 con 0x7fc318161ed0 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc30ffff640 1 -- 172.21.15.133:0/3634201714 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100039a0 con 0x7fc318161ed0 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc31d542640 1 -- 172.21.15.133:0/3634201714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc318163ca0 con 0x7fc318161ed0 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc31d542640 1 -- 172.21.15.133:0/3634201714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc318068ef0 con 0x7fc318161ed0 2024-04-25T10:15:16.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc30ffff640 1 -- 172.21.15.133:0/3634201714 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc310003b60 con 0x7fc318161ed0 2024-04-25T10:15:16.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc30ffff640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f8066f60 0x7fc2f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc30ffff640 1 -- 172.21.15.133:0/3634201714 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc310085860 con 0x7fc318161ed0 2024-04-25T10:15:16.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc316ffd640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f8066f60 0x7fc2f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.594+0000 7fc316ffd640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f8066f60 0x7fc2f8069420 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc3040057f0 tx=0x7fc3080011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.598+0000 7fc30ffff640 1 -- 172.21.15.133:0/3634201714 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3100566e0 con 0x7fc318161ed0 2024-04-25T10:15:16.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.754+0000 7fc31d542640 1 -- 172.21.15.133:0/3634201714 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc3180b2750 con 0x7fc2f8066f60 2024-04-25T10:15:16.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.762+0000 7fc30ffff640 1 -- 172.21.15.133:0/3634201714 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252429 (secure 0 0 0) 0x7fc3180b2750 con 0x7fc2f8066f60 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:16.766 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:15:16.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 -- 172.21.15.133:0/3634201714 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f8066f60 msgr2=0x7fc2f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f8066f60 0x7fc2f8069420 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc3040057f0 tx=0x7fc3080011a0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 -- 172.21.15.133:0/3634201714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318161ed0 msgr2=0x7fc318162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318161ed0 0x7fc318162350 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fc3180b5840 tx=0x7fc3100115a0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 -- 172.21.15.133:0/3634201714 shutdown_connections 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f8066f60 0x7fc2f8069420 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318161ed0 0x7fc318162350 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc318165e40 0x7fc3181662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 --2- 172.21.15.133:0/3634201714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc3040039f0 0x7fc31814ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 -- 172.21.15.133:0/3634201714 >> 172.21.15.133:0/3634201714 conn(0x7fc3180727c0 msgr2=0x7fc3180d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 -- 172.21.15.133:0/3634201714 shutdown_connections 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.766+0000 7fc31d542640 1 -- 172.21.15.133:0/3634201714 wait complete. 2024-04-25T10:15:16.783 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":556,"stamp":"2024-04-25T10:15:15.960544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":499252,"kb_used_data":16552,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754475468,"statfs":{"total":773094113280,"available":772582879232,"internally_reserved":0,"allocated":16949248,"data_stored":8545460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.227892"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776390+0000","last_change":"2024-04-25T10:15:13.777248+0000","last_active":"2024-04-25T10:15:14.776390+0000","last_peered":"2024-04-25T10:15:14.776390+0000","last_clean":"2024-04-25T10:15:14.776390+0000","last_became_active":"2024-04-25T10:15:13.776504+0000","last_became_peered":"2024-04-25T10:15:13.776504+0000","last_unstale":"2024-04-25T10:15:14.776390+0000","last_undegraded":"2024-04-25T10:15:14.776390+0000","last_fullsized":"2024-04-25T10:15:14.776390+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:51.396232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.783993+0000","last_change":"2024-04-25T10:15:13.772066+0000","last_active":"2024-04-25T10:15:13.783993+0000","last_peered":"2024-04-25T10:15:13.783993+0000","last_clean":"2024-04-25T10:15:13.783993+0000","last_became_active":"2024-04-25T10:15:13.771713+0000","last_became_peered":"2024-04-25T10:15:13.771713+0000","last_unstale":"2024-04-25T10:15:13.783993+0000","last_undegraded":"2024-04-25T10:15:13.783993+0000","last_fullsized":"2024-04-25T10:15:13.783993+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:07.491218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776056+0000","last_change":"2024-04-25T10:15:13.789783+0000","last_active":"2024-04-25T10:15:14.776056+0000","last_peered":"2024-04-25T10:15:14.776056+0000","last_clean":"2024-04-25T10:15:14.776056+0000","last_became_active":"2024-04-25T10:15:13.789394+0000","last_became_peered":"2024-04-25T10:15:13.789394+0000","last_unstale":"2024-04-25T10:15:14.776056+0000","last_undegraded":"2024-04-25T10:15:14.776056+0000","last_fullsized":"2024-04-25T10:15:14.776056+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:06.350508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776339+0000","last_change":"2024-04-25T10:15:13.778540+0000","last_active":"2024-04-25T10:15:14.776339+0000","last_peered":"2024-04-25T10:15:14.776339+0000","last_clean":"2024-04-25T10:15:14.776339+0000","last_became_active":"2024-04-25T10:15:13.777912+0000","last_became_peered":"2024-04-25T10:15:13.777912+0000","last_unstale":"2024-04-25T10:15:14.776339+0000","last_undegraded":"2024-04-25T10:15:14.776339+0000","last_fullsized":"2024-04-25T10:15:14.776339+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:04.037423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.791875+0000","last_change":"2024-04-25T10:15:13.783438+0000","last_active":"2024-04-25T10:15:13.791875+0000","last_peered":"2024-04-25T10:15:13.791875+0000","last_clean":"2024-04-25T10:15:13.791875+0000","last_became_active":"2024-04-25T10:15:13.782662+0000","last_became_peered":"2024-04-25T10:15:13.782662+0000","last_unstale":"2024-04-25T10:15:13.791875+0000","last_undegraded":"2024-04-25T10:15:13.791875+0000","last_fullsized":"2024-04-25T10:15:13.791875+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:07.893119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784093+0000","last_change":"2024-04-25T10:15:13.777716+0000","last_active":"2024-04-25T10:15:13.784093+0000","last_peered":"2024-04-25T10:15:13.784093+0000","last_clean":"2024-04-25T10:15:13.784093+0000","last_became_active":"2024-04-25T10:15:13.777321+0000","last_became_peered":"2024-04-25T10:15:13.777321+0000","last_unstale":"2024-04-25T10:15:13.784093+0000","last_undegraded":"2024-04-25T10:15:13.784093+0000","last_fullsized":"2024-04-25T10:15:13.784093+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:37.604335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.797459+0000","last_change":"2024-04-25T10:15:13.788631+0000","last_active":"2024-04-25T10:15:13.797459+0000","last_peered":"2024-04-25T10:15:13.797459+0000","last_clean":"2024-04-25T10:15:13.797459+0000","last_became_active":"2024-04-25T10:15:13.788272+0000","last_became_peered":"2024-04-25T10:15:13.788272+0000","last_unstale":"2024-04-25T10:15:13.797459+0000","last_undegraded":"2024-04-25T10:15:13.797459+0000","last_fullsized":"2024-04-25T10:15:13.797459+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:01.806338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.794435+0000","last_change":"2024-04-25T10:15:13.777268+0000","last_active":"2024-04-25T10:15:13.794435+0000","last_peered":"2024-04-25T10:15:13.794435+0000","last_clean":"2024-04-25T10:15:13.794435+0000","last_became_active":"2024-04-25T10:15:13.776983+0000","last_became_peered":"2024-04-25T10:15:13.776983+0000","last_unstale":"2024-04-25T10:15:13.794435+0000","last_undegraded":"2024-04-25T10:15:13.794435+0000","last_fullsized":"2024-04-25T10:15:13.794435+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:55.865610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.793938+0000","last_change":"2024-04-25T10:15:13.783548+0000","last_active":"2024-04-25T10:15:13.793938+0000","last_peered":"2024-04-25T10:15:13.793938+0000","last_clean":"2024-04-25T10:15:13.793938+0000","last_became_active":"2024-04-25T10:15:13.783226+0000","last_became_peered":"2024-04-25T10:15:13.783226+0000","last_unstale":"2024-04-25T10:15:13.793938+0000","last_undegraded":"2024-04-25T10:15:13.793938+0000","last_fullsized":"2024-04-25T10:15:13.793938+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.227044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.797858+0000","last_change":"2024-04-25T10:15:13.792924+0000","last_active":"2024-04-25T10:15:13.797858+0000","last_peered":"2024-04-25T10:15:13.797858+0000","last_clean":"2024-04-25T10:15:13.797858+0000","last_became_active":"2024-04-25T10:15:13.788945+0000","last_became_peered":"2024-04-25T10:15:13.788945+0000","last_unstale":"2024-04-25T10:15:13.797858+0000","last_undegraded":"2024-04-25T10:15:13.797858+0000","last_fullsized":"2024-04-25T10:15:13.797858+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:28.526079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.788559+0000","last_change":"2024-04-25T10:15:13.786249+0000","last_active":"2024-04-25T10:15:13.788559+0000","last_peered":"2024-04-25T10:15:13.788559+0000","last_clean":"2024-04-25T10:15:13.788559+0000","last_became_active":"2024-04-25T10:15:13.785597+0000","last_became_peered":"2024-04-25T10:15:13.785597+0000","last_unstale":"2024-04-25T10:15:13.788559+0000","last_undegraded":"2024-04-25T10:15:13.788559+0000","last_fullsized":"2024-04-25T10:15:13.788559+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.227044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.787357+0000","last_change":"2024-04-25T10:15:13.782743+0000","last_active":"2024-04-25T10:15:13.787357+0000","last_peered":"2024-04-25T10:15:13.787357+0000","last_clean":"2024-04-25T10:15:13.787357+0000","last_became_active":"2024-04-25T10:15:13.782385+0000","last_became_peered":"2024-04-25T10:15:13.782385+0000","last_unstale":"2024-04-25T10:15:13.787357+0000","last_undegraded":"2024-04-25T10:15:13.787357+0000","last_fullsized":"2024-04-25T10:15:13.787357+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:14.818954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.773824+0000","last_change":"2024-04-25T10:15:13.766168+0000","last_active":"2024-04-25T10:15:14.773824+0000","last_peered":"2024-04-25T10:15:14.773824+0000","last_clean":"2024-04-25T10:15:14.773824+0000","last_became_active":"2024-04-25T10:15:13.765672+0000","last_became_peered":"2024-04-25T10:15:13.765672+0000","last_unstale":"2024-04-25T10:15:14.773824+0000","last_undegraded":"2024-04-25T10:15:14.773824+0000","last_fullsized":"2024-04-25T10:15:14.773824+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:14.818954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795370+0000","last_change":"2024-04-25T10:15:13.787363+0000","last_active":"2024-04-25T10:15:13.795370+0000","last_peered":"2024-04-25T10:15:13.795370+0000","last_clean":"2024-04-25T10:15:13.795370+0000","last_became_active":"2024-04-25T10:15:13.787089+0000","last_became_peered":"2024-04-25T10:15:13.787089+0000","last_unstale":"2024-04-25T10:15:13.795370+0000","last_undegraded":"2024-04-25T10:15:13.795370+0000","last_fullsized":"2024-04-25T10:15:13.795370+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:28.192082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.787761+0000","last_change":"2024-04-25T10:15:13.783393+0000","last_active":"2024-04-25T10:15:13.787761+0000","last_peered":"2024-04-25T10:15:13.787761+0000","last_clean":"2024-04-25T10:15:13.787761+0000","last_became_active":"2024-04-25T10:15:13.783069+0000","last_became_peered":"2024-04-25T10:15:13.783069+0000","last_unstale":"2024-04-25T10:15:13.787761+0000","last_undegraded":"2024-04-25T10:15:13.787761+0000","last_fullsized":"2024-04-25T10:15:13.787761+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:12.560631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.775547+0000","last_change":"2024-04-25T10:15:13.781303+0000","last_active":"2024-04-25T10:15:14.775547+0000","last_peered":"2024-04-25T10:15:14.775547+0000","last_clean":"2024-04-25T10:15:14.775547+0000","last_became_active":"2024-04-25T10:15:13.780471+0000","last_became_peered":"2024-04-25T10:15:13.780471+0000","last_unstale":"2024-04-25T10:15:14.775547+0000","last_undegraded":"2024-04-25T10:15:14.775547+0000","last_fullsized":"2024-04-25T10:15:14.775547+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:38.716662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.793763+0000","last_change":"2024-04-25T10:15:13.776284+0000","last_active":"2024-04-25T10:15:13.793763+0000","last_peered":"2024-04-25T10:15:13.793763+0000","last_clean":"2024-04-25T10:15:13.793763+0000","last_became_active":"2024-04-25T10:15:13.775903+0000","last_became_peered":"2024-04-25T10:15:13.775903+0000","last_unstale":"2024-04-25T10:15:13.793763+0000","last_undegraded":"2024-04-25T10:15:13.793763+0000","last_fullsized":"2024-04-25T10:15:13.793763+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:27.442307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774838+0000","last_change":"2024-04-25T10:15:13.790271+0000","last_active":"2024-04-25T10:15:14.774838+0000","last_peered":"2024-04-25T10:15:14.774838+0000","last_clean":"2024-04-25T10:15:14.774838+0000","last_became_active":"2024-04-25T10:15:13.789942+0000","last_became_peered":"2024-04-25T10:15:13.789942+0000","last_unstale":"2024-04-25T10:15:14.774838+0000","last_undegraded":"2024-04-25T10:15:14.774838+0000","last_fullsized":"2024-04-25T10:15:14.774838+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:04.820906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.799256+0000","last_change":"2024-04-25T10:15:13.798652+0000","last_active":"2024-04-25T10:15:13.799256+0000","last_peered":"2024-04-25T10:15:13.799256+0000","last_clean":"2024-04-25T10:15:13.799256+0000","last_became_active":"2024-04-25T10:15:13.798281+0000","last_became_peered":"2024-04-25T10:15:13.798281+0000","last_unstale":"2024-04-25T10:15:13.799256+0000","last_undegraded":"2024-04-25T10:15:13.799256+0000","last_fullsized":"2024-04-25T10:15:13.799256+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:11.287206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.798225+0000","last_change":"2024-04-25T10:15:13.792698+0000","last_active":"2024-04-25T10:15:13.798225+0000","last_peered":"2024-04-25T10:15:13.798225+0000","last_clean":"2024-04-25T10:15:13.798225+0000","last_became_active":"2024-04-25T10:15:13.792451+0000","last_became_peered":"2024-04-25T10:15:13.792451+0000","last_unstale":"2024-04-25T10:15:13.798225+0000","last_undegraded":"2024-04-25T10:15:13.798225+0000","last_fullsized":"2024-04-25T10:15:13.798225+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:52.275872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.773873+0000","last_change":"2024-04-25T10:15:13.776270+0000","last_active":"2024-04-25T10:15:14.773873+0000","last_peered":"2024-04-25T10:15:14.773873+0000","last_clean":"2024-04-25T10:15:14.773873+0000","last_became_active":"2024-04-25T10:15:13.775699+0000","last_became_peered":"2024-04-25T10:15:13.775699+0000","last_unstale":"2024-04-25T10:15:14.773873+0000","last_undegraded":"2024-04-25T10:15:14.773873+0000","last_fullsized":"2024-04-25T10:15:14.773873+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:33.998118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784061+0000","last_change":"2024-04-25T10:15:13.776160+0000","last_active":"2024-04-25T10:15:13.784061+0000","last_peered":"2024-04-25T10:15:13.784061+0000","last_clean":"2024-04-25T10:15:13.784061+0000","last_became_active":"2024-04-25T10:15:13.775795+0000","last_became_peered":"2024-04-25T10:15:13.775795+0000","last_unstale":"2024-04-25T10:15:13.784061+0000","last_undegraded":"2024-04-25T10:15:13.784061+0000","last_fullsized":"2024-04-25T10:15:13.784061+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:36.208593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795823+0000","last_change":"2024-04-25T10:15:13.791580+0000","last_active":"2024-04-25T10:15:13.795823+0000","last_peered":"2024-04-25T10:15:13.795823+0000","last_clean":"2024-04-25T10:15:13.795823+0000","last_became_active":"2024-04-25T10:15:13.790925+0000","last_became_peered":"2024-04-25T10:15:13.790925+0000","last_unstale":"2024-04-25T10:15:13.795823+0000","last_undegraded":"2024-04-25T10:15:13.795823+0000","last_fullsized":"2024-04-25T10:15:13.795823+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:37.604335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.775661+0000","last_change":"2024-04-25T10:15:13.781331+0000","last_active":"2024-04-25T10:15:14.775661+0000","last_peered":"2024-04-25T10:15:14.775661+0000","last_clean":"2024-04-25T10:15:14.775661+0000","last_became_active":"2024-04-25T10:15:13.780994+0000","last_became_peered":"2024-04-25T10:15:13.780994+0000","last_unstale":"2024-04-25T10:15:14.775661+0000","last_undegraded":"2024-04-25T10:15:14.775661+0000","last_fullsized":"2024-04-25T10:15:14.775661+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:49.726501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774549+0000","last_change":"2024-04-25T10:15:13.770157+0000","last_active":"2024-04-25T10:15:14.774549+0000","last_peered":"2024-04-25T10:15:14.774549+0000","last_clean":"2024-04-25T10:15:14.774549+0000","last_became_active":"2024-04-25T10:15:13.769866+0000","last_became_peered":"2024-04-25T10:15:13.769866+0000","last_unstale":"2024-04-25T10:15:14.774549+0000","last_undegraded":"2024-04-25T10:15:14.774549+0000","last_fullsized":"2024-04-25T10:15:14.774549+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:48.708916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784937+0000","last_change":"2024-04-25T10:15:13.778869+0000","last_active":"2024-04-25T10:15:13.784937+0000","last_peered":"2024-04-25T10:15:13.784937+0000","last_clean":"2024-04-25T10:15:13.784937+0000","last_became_active":"2024-04-25T10:15:13.778584+0000","last_became_peered":"2024-04-25T10:15:13.778584+0000","last_unstale":"2024-04-25T10:15:13.784937+0000","last_undegraded":"2024-04-25T10:15:13.784937+0000","last_fullsized":"2024-04-25T10:15:13.784937+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:04.220167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.788021+0000","last_change":"2024-04-25T10:15:13.787035+0000","last_active":"2024-04-25T10:15:13.788021+0000","last_peered":"2024-04-25T10:15:13.788021+0000","last_clean":"2024-04-25T10:15:13.788021+0000","last_became_active":"2024-04-25T10:15:13.786291+0000","last_became_peered":"2024-04-25T10:15:13.786291+0000","last_unstale":"2024-04-25T10:15:13.788021+0000","last_undegraded":"2024-04-25T10:15:13.788021+0000","last_fullsized":"2024-04-25T10:15:13.788021+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:28.723944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.773690+0000","last_change":"2024-04-25T10:15:13.780046+0000","last_active":"2024-04-25T10:15:14.773690+0000","last_peered":"2024-04-25T10:15:14.773690+0000","last_clean":"2024-04-25T10:15:14.773690+0000","last_became_active":"2024-04-25T10:15:13.779674+0000","last_became_peered":"2024-04-25T10:15:13.779674+0000","last_unstale":"2024-04-25T10:15:14.773690+0000","last_undegraded":"2024-04-25T10:15:14.773690+0000","last_fullsized":"2024-04-25T10:15:14.773690+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:36.208593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784324+0000","last_change":"2024-04-25T10:15:13.770250+0000","last_active":"2024-04-25T10:15:13.784324+0000","last_peered":"2024-04-25T10:15:13.784324+0000","last_clean":"2024-04-25T10:15:13.784324+0000","last_became_active":"2024-04-25T10:15:13.769894+0000","last_became_peered":"2024-04-25T10:15:13.769894+0000","last_unstale":"2024-04-25T10:15:13.784324+0000","last_undegraded":"2024-04-25T10:15:13.784324+0000","last_fullsized":"2024-04-25T10:15:13.784324+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:07.893119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774914+0000","last_change":"2024-04-25T10:15:13.784969+0000","last_active":"2024-04-25T10:15:14.774914+0000","last_peered":"2024-04-25T10:15:14.774914+0000","last_clean":"2024-04-25T10:15:14.774914+0000","last_became_active":"2024-04-25T10:15:13.784635+0000","last_became_peered":"2024-04-25T10:15:13.784635+0000","last_unstale":"2024-04-25T10:15:14.774914+0000","last_undegraded":"2024-04-25T10:15:14.774914+0000","last_fullsized":"2024-04-25T10:15:14.774914+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:49.030545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.781667+0000","last_change":"2024-04-25T10:15:13.780351+0000","last_active":"2024-04-25T10:15:13.781667+0000","last_peered":"2024-04-25T10:15:13.781667+0000","last_clean":"2024-04-25T10:15:13.781667+0000","last_became_active":"2024-04-25T10:15:13.779997+0000","last_became_peered":"2024-04-25T10:15:13.779997+0000","last_unstale":"2024-04-25T10:15:13.781667+0000","last_undegraded":"2024-04-25T10:15:13.781667+0000","last_fullsized":"2024-04-25T10:15:13.781667+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:27.442307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774432+0000","last_change":"2024-04-25T10:15:13.763480+0000","last_active":"2024-04-25T10:15:14.774432+0000","last_peered":"2024-04-25T10:15:14.774432+0000","last_clean":"2024-04-25T10:15:14.774432+0000","last_became_active":"2024-04-25T10:15:13.762642+0000","last_became_peered":"2024-04-25T10:15:13.762642+0000","last_unstale":"2024-04-25T10:15:14.774432+0000","last_undegraded":"2024-04-25T10:15:14.774432+0000","last_fullsized":"2024-04-25T10:15:14.774432+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:52.397943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.783874+0000","last_change":"2024-04-25T10:15:13.769289+0000","last_active":"2024-04-25T10:15:13.783874+0000","last_peered":"2024-04-25T10:15:13.783874+0000","last_clean":"2024-04-25T10:15:13.783874+0000","last_became_active":"2024-04-25T10:15:13.768386+0000","last_became_peered":"2024-04-25T10:15:13.768386+0000","last_unstale":"2024-04-25T10:15:13.783874+0000","last_undegraded":"2024-04-25T10:15:13.783874+0000","last_fullsized":"2024-04-25T10:15:13.783874+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:49.584476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"156'4","reported_seq":18,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776640+0000","last_change":"2024-04-25T10:15:12.775976+0000","last_active":"2024-04-25T10:15:14.776640+0000","last_peered":"2024-04-25T10:15:14.776640+0000","last_clean":"2024-04-25T10:15:14.776640+0000","last_became_active":"2024-04-25T10:15:12.775370+0000","last_became_peered":"2024-04-25T10:15:12.775370+0000","last_unstale":"2024-04-25T10:15:14.776640+0000","last_undegraded":"2024-04-25T10:15:14.776640+0000","last_fullsized":"2024-04-25T10:15:14.776640+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:48.026615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774032+0000","last_change":"2024-04-25T10:15:13.760187+0000","last_active":"2024-04-25T10:15:14.774032+0000","last_peered":"2024-04-25T10:15:14.774032+0000","last_clean":"2024-04-25T10:15:14.774032+0000","last_became_active":"2024-04-25T10:15:13.759450+0000","last_became_peered":"2024-04-25T10:15:13.759450+0000","last_unstale":"2024-04-25T10:15:14.774032+0000","last_undegraded":"2024-04-25T10:15:14.774032+0000","last_fullsized":"2024-04-25T10:15:14.774032+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:28.723944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"156'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.798538+0000","last_change":"2024-04-25T10:15:12.776401+0000","last_active":"2024-04-25T10:15:13.798538+0000","last_peered":"2024-04-25T10:15:13.798538+0000","last_clean":"2024-04-25T10:15:13.798538+0000","last_became_active":"2024-04-25T10:15:12.775255+0000","last_became_peered":"2024-04-25T10:15:12.775255+0000","last_unstale":"2024-04-25T10:15:13.798538+0000","last_undegraded":"2024-04-25T10:15:13.798538+0000","last_fullsized":"2024-04-25T10:15:13.798538+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:48.580547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795773+0000","last_change":"2024-04-25T10:15:13.791547+0000","last_active":"2024-04-25T10:15:13.795773+0000","last_peered":"2024-04-25T10:15:13.795773+0000","last_clean":"2024-04-25T10:15:13.795773+0000","last_became_active":"2024-04-25T10:15:13.791203+0000","last_became_peered":"2024-04-25T10:15:13.791203+0000","last_unstale":"2024-04-25T10:15:13.795773+0000","last_undegraded":"2024-04-25T10:15:13.795773+0000","last_fullsized":"2024-04-25T10:15:13.795773+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:36.208593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.787232+0000","last_change":"2024-04-25T10:15:12.766821+0000","last_active":"2024-04-25T10:15:13.787232+0000","last_peered":"2024-04-25T10:15:13.787232+0000","last_clean":"2024-04-25T10:15:13.787232+0000","last_became_active":"2024-04-25T10:15:12.766179+0000","last_became_peered":"2024-04-25T10:15:12.766179+0000","last_unstale":"2024-04-25T10:15:13.787232+0000","last_undegraded":"2024-04-25T10:15:13.787232+0000","last_fullsized":"2024-04-25T10:15:13.787232+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:09.517126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.783792+0000","last_change":"2024-04-25T10:15:13.769091+0000","last_active":"2024-04-25T10:15:13.783792+0000","last_peered":"2024-04-25T10:15:13.783792+0000","last_clean":"2024-04-25T10:15:13.783792+0000","last_became_active":"2024-04-25T10:15:13.768722+0000","last_became_peered":"2024-04-25T10:15:13.768722+0000","last_unstale":"2024-04-25T10:15:13.783792+0000","last_undegraded":"2024-04-25T10:15:13.783792+0000","last_fullsized":"2024-04-25T10:15:13.783792+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:34.100190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.787359+0000","last_change":"2024-04-25T10:15:12.774941+0000","last_active":"2024-04-25T10:15:13.787359+0000","last_peered":"2024-04-25T10:15:13.787359+0000","last_clean":"2024-04-25T10:15:13.787359+0000","last_became_active":"2024-04-25T10:15:12.774655+0000","last_became_peered":"2024-04-25T10:15:12.774655+0000","last_unstale":"2024-04-25T10:15:13.787359+0000","last_undegraded":"2024-04-25T10:15:13.787359+0000","last_fullsized":"2024-04-25T10:15:13.787359+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:44.691915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.794077+0000","last_change":"2024-04-25T10:15:13.782845+0000","last_active":"2024-04-25T10:15:13.794077+0000","last_peered":"2024-04-25T10:15:13.794077+0000","last_clean":"2024-04-25T10:15:13.794077+0000","last_became_active":"2024-04-25T10:15:13.782172+0000","last_became_peered":"2024-04-25T10:15:13.782172+0000","last_unstale":"2024-04-25T10:15:13.794077+0000","last_undegraded":"2024-04-25T10:15:13.794077+0000","last_fullsized":"2024-04-25T10:15:13.794077+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:09.716962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"156'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.793913+0000","last_change":"2024-04-25T10:15:12.771422+0000","last_active":"2024-04-25T10:15:13.793913+0000","last_peered":"2024-04-25T10:15:13.793913+0000","last_clean":"2024-04-25T10:15:13.793913+0000","last_became_active":"2024-04-25T10:15:12.770843+0000","last_became_peered":"2024-04-25T10:15:12.770843+0000","last_unstale":"2024-04-25T10:15:13.793913+0000","last_undegraded":"2024-04-25T10:15:13.793913+0000","last_fullsized":"2024-04-25T10:15:13.793913+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:06.835915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.785104+0000","last_change":"2024-04-25T10:15:13.778440+0000","last_active":"2024-04-25T10:15:13.785104+0000","last_peered":"2024-04-25T10:15:13.785104+0000","last_clean":"2024-04-25T10:15:13.785104+0000","last_became_active":"2024-04-25T10:15:13.777800+0000","last_became_peered":"2024-04-25T10:15:13.777800+0000","last_unstale":"2024-04-25T10:15:13.785104+0000","last_undegraded":"2024-04-25T10:15:13.785104+0000","last_fullsized":"2024-04-25T10:15:13.785104+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:21.531667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"156'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795400+0000","last_change":"2024-04-25T10:15:12.765558+0000","last_active":"2024-04-25T10:15:13.795400+0000","last_peered":"2024-04-25T10:15:13.795400+0000","last_clean":"2024-04-25T10:15:13.795400+0000","last_became_active":"2024-04-25T10:15:12.765227+0000","last_became_peered":"2024-04-25T10:15:12.765227+0000","last_unstale":"2024-04-25T10:15:13.795400+0000","last_undegraded":"2024-04-25T10:15:13.795400+0000","last_fullsized":"2024-04-25T10:15:13.795400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:41.790368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.782024+0000","last_change":"2024-04-25T10:15:13.777079+0000","last_active":"2024-04-25T10:15:13.782024+0000","last_peered":"2024-04-25T10:15:13.782024+0000","last_clean":"2024-04-25T10:15:13.782024+0000","last_became_active":"2024-04-25T10:15:13.775146+0000","last_became_peered":"2024-04-25T10:15:13.775146+0000","last_unstale":"2024-04-25T10:15:13.782024+0000","last_undegraded":"2024-04-25T10:15:13.782024+0000","last_fullsized":"2024-04-25T10:15:13.782024+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:24.767040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.775739+0000","last_change":"2024-04-25T10:15:12.768937+0000","last_active":"2024-04-25T10:15:13.775739+0000","last_peered":"2024-04-25T10:15:13.775739+0000","last_clean":"2024-04-25T10:15:13.775739+0000","last_became_active":"2024-04-25T10:15:12.768546+0000","last_became_peered":"2024-04-25T10:15:12.768546+0000","last_unstale":"2024-04-25T10:15:13.775739+0000","last_undegraded":"2024-04-25T10:15:13.775739+0000","last_fullsized":"2024-04-25T10:15:13.775739+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:05.923856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776509+0000","last_change":"2024-04-25T10:15:13.776821+0000","last_active":"2024-04-25T10:15:14.776509+0000","last_peered":"2024-04-25T10:15:14.776509+0000","last_clean":"2024-04-25T10:15:14.776509+0000","last_became_active":"2024-04-25T10:15:13.776085+0000","last_became_peered":"2024-04-25T10:15:13.776085+0000","last_unstale":"2024-04-25T10:15:14.776509+0000","last_undegraded":"2024-04-25T10:15:14.776509+0000","last_fullsized":"2024-04-25T10:15:14.776509+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:49.158981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.771020+0000","last_change":"2024-04-25T10:15:12.774878+0000","last_active":"2024-04-25T10:15:13.771020+0000","last_peered":"2024-04-25T10:15:13.771020+0000","last_clean":"2024-04-25T10:15:13.771020+0000","last_became_active":"2024-04-25T10:15:12.774467+0000","last_became_peered":"2024-04-25T10:15:12.774467+0000","last_unstale":"2024-04-25T10:15:13.771020+0000","last_undegraded":"2024-04-25T10:15:13.771020+0000","last_fullsized":"2024-04-25T10:15:13.771020+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:58.354472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.783904+0000","last_change":"2024-04-25T10:15:13.769687+0000","last_active":"2024-04-25T10:15:13.783904+0000","last_peered":"2024-04-25T10:15:13.783904+0000","last_clean":"2024-04-25T10:15:13.783904+0000","last_became_active":"2024-04-25T10:15:13.768960+0000","last_became_peered":"2024-04-25T10:15:13.768960+0000","last_unstale":"2024-04-25T10:15:13.783904+0000","last_undegraded":"2024-04-25T10:15:13.783904+0000","last_fullsized":"2024-04-25T10:15:13.783904+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:58.557955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774345+0000","last_change":"2024-04-25T10:15:12.773471+0000","last_active":"2024-04-25T10:15:14.774345+0000","last_peered":"2024-04-25T10:15:14.774345+0000","last_clean":"2024-04-25T10:15:14.774345+0000","last_became_active":"2024-04-25T10:15:12.771864+0000","last_became_peered":"2024-04-25T10:15:12.771864+0000","last_unstale":"2024-04-25T10:15:14.774345+0000","last_undegraded":"2024-04-25T10:15:14.774345+0000","last_fullsized":"2024-04-25T10:15:14.774345+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:04.795357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.773770+0000","last_change":"2024-04-25T10:15:13.769915+0000","last_active":"2024-04-25T10:15:14.773770+0000","last_peered":"2024-04-25T10:15:14.773770+0000","last_clean":"2024-04-25T10:15:14.773770+0000","last_became_active":"2024-04-25T10:15:13.769378+0000","last_became_peered":"2024-04-25T10:15:13.769378+0000","last_unstale":"2024-04-25T10:15:14.773770+0000","last_undegraded":"2024-04-25T10:15:14.773770+0000","last_fullsized":"2024-04-25T10:15:14.773770+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:37.604335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"156'3","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776327+0000","last_change":"2024-04-25T10:15:12.777785+0000","last_active":"2024-04-25T10:15:14.776327+0000","last_peered":"2024-04-25T10:15:14.776327+0000","last_clean":"2024-04-25T10:15:14.776327+0000","last_became_active":"2024-04-25T10:15:12.777045+0000","last_became_peered":"2024-04-25T10:15:12.777045+0000","last_unstale":"2024-04-25T10:15:14.776327+0000","last_undegraded":"2024-04-25T10:15:14.776327+0000","last_fullsized":"2024-04-25T10:15:14.776327+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:27.149157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.785233+0000","last_change":"2024-04-25T10:15:13.784254+0000","last_active":"2024-04-25T10:15:13.785233+0000","last_peered":"2024-04-25T10:15:13.785233+0000","last_clean":"2024-04-25T10:15:13.785233+0000","last_became_active":"2024-04-25T10:15:13.783932+0000","last_became_peered":"2024-04-25T10:15:13.783932+0000","last_unstale":"2024-04-25T10:15:13.785233+0000","last_undegraded":"2024-04-25T10:15:13.785233+0000","last_fullsized":"2024-04-25T10:15:13.785233+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:00.434491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"156'6","reported_seq":20,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774212+0000","last_change":"2024-04-25T10:15:12.773735+0000","last_active":"2024-04-25T10:15:14.774212+0000","last_peered":"2024-04-25T10:15:14.774212+0000","last_clean":"2024-04-25T10:15:14.774212+0000","last_became_active":"2024-04-25T10:15:12.773371+0000","last_became_peered":"2024-04-25T10:15:12.773371+0000","last_unstale":"2024-04-25T10:15:14.774212+0000","last_undegraded":"2024-04-25T10:15:14.774212+0000","last_fullsized":"2024-04-25T10:15:14.774212+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:37.073666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774727+0000","last_change":"2024-04-25T10:15:13.763459+0000","last_active":"2024-04-25T10:15:14.774727+0000","last_peered":"2024-04-25T10:15:14.774727+0000","last_clean":"2024-04-25T10:15:14.774727+0000","last_became_active":"2024-04-25T10:15:13.760892+0000","last_became_peered":"2024-04-25T10:15:13.760892+0000","last_unstale":"2024-04-25T10:15:14.774727+0000","last_undegraded":"2024-04-25T10:15:14.774727+0000","last_fullsized":"2024-04-25T10:15:14.774727+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:28.153086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795557+0000","last_change":"2024-04-25T10:15:12.777614+0000","last_active":"2024-04-25T10:15:13.795557+0000","last_peered":"2024-04-25T10:15:13.795557+0000","last_clean":"2024-04-25T10:15:13.795557+0000","last_became_active":"2024-04-25T10:15:12.777135+0000","last_became_peered":"2024-04-25T10:15:12.777135+0000","last_unstale":"2024-04-25T10:15:13.795557+0000","last_undegraded":"2024-04-25T10:15:13.795557+0000","last_fullsized":"2024-04-25T10:15:13.795557+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:09.915844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.798640+0000","last_change":"2024-04-25T10:15:13.793815+0000","last_active":"2024-04-25T10:15:13.798640+0000","last_peered":"2024-04-25T10:15:13.798640+0000","last_clean":"2024-04-25T10:15:13.798640+0000","last_became_active":"2024-04-25T10:15:13.793459+0000","last_became_peered":"2024-04-25T10:15:13.793459+0000","last_unstale":"2024-04-25T10:15:13.798640+0000","last_undegraded":"2024-04-25T10:15:13.798640+0000","last_fullsized":"2024-04-25T10:15:13.798640+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:31.373069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.773954+0000","last_change":"2024-04-25T10:15:12.762355+0000","last_active":"2024-04-25T10:15:14.773954+0000","last_peered":"2024-04-25T10:15:14.773954+0000","last_clean":"2024-04-25T10:15:14.773954+0000","last_became_active":"2024-04-25T10:15:12.762056+0000","last_became_peered":"2024-04-25T10:15:12.762056+0000","last_unstale":"2024-04-25T10:15:14.773954+0000","last_undegraded":"2024-04-25T10:15:14.773954+0000","last_fullsized":"2024-04-25T10:15:14.773954+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:30.369140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.788393+0000","last_change":"2024-04-25T10:15:13.784694+0000","last_active":"2024-04-25T10:15:13.788393+0000","last_peered":"2024-04-25T10:15:13.788393+0000","last_clean":"2024-04-25T10:15:13.788393+0000","last_became_active":"2024-04-25T10:15:13.784401+0000","last_became_peered":"2024-04-25T10:15:13.784401+0000","last_unstale":"2024-04-25T10:15:13.788393+0000","last_undegraded":"2024-04-25T10:15:13.788393+0000","last_fullsized":"2024-04-25T10:15:13.788393+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:55.865610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.769277+0000","last_change":"2024-04-25T10:15:12.767004+0000","last_active":"2024-04-25T10:15:13.769277+0000","last_peered":"2024-04-25T10:15:13.769277+0000","last_clean":"2024-04-25T10:15:13.769277+0000","last_became_active":"2024-04-25T10:15:12.766468+0000","last_became_peered":"2024-04-25T10:15:12.766468+0000","last_unstale":"2024-04-25T10:15:13.769277+0000","last_undegraded":"2024-04-25T10:15:13.769277+0000","last_fullsized":"2024-04-25T10:15:13.769277+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:40.043055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784195+0000","last_change":"2024-04-25T10:15:13.778256+0000","last_active":"2024-04-25T10:15:13.784195+0000","last_peered":"2024-04-25T10:15:13.784195+0000","last_clean":"2024-04-25T10:15:13.784195+0000","last_became_active":"2024-04-25T10:15:13.777826+0000","last_became_peered":"2024-04-25T10:15:13.777826+0000","last_unstale":"2024-04-25T10:15:13.784195+0000","last_undegraded":"2024-04-25T10:15:13.784195+0000","last_fullsized":"2024-04-25T10:15:13.784195+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:47.683035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"156'8","reported_seq":20,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.798774+0000","last_change":"2024-04-25T10:15:12.769778+0000","last_active":"2024-04-25T10:15:13.798774+0000","last_peered":"2024-04-25T10:15:13.798774+0000","last_clean":"2024-04-25T10:15:13.798774+0000","last_became_active":"2024-04-25T10:15:12.769118+0000","last_became_peered":"2024-04-25T10:15:12.769118+0000","last_unstale":"2024-04-25T10:15:13.798774+0000","last_undegraded":"2024-04-25T10:15:13.798774+0000","last_fullsized":"2024-04-25T10:15:13.798774+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:09.517126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784223+0000","last_change":"2024-04-25T10:15:13.769771+0000","last_active":"2024-04-25T10:15:13.784223+0000","last_peered":"2024-04-25T10:15:13.784223+0000","last_clean":"2024-04-25T10:15:13.784223+0000","last_became_active":"2024-04-25T10:15:13.769351+0000","last_became_peered":"2024-04-25T10:15:13.769351+0000","last_unstale":"2024-04-25T10:15:13.784223+0000","last_undegraded":"2024-04-25T10:15:13.784223+0000","last_fullsized":"2024-04-25T10:15:13.784223+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:42.844627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"156'1","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774685+0000","last_change":"2024-04-25T10:15:12.773689+0000","last_active":"2024-04-25T10:15:14.774685+0000","last_peered":"2024-04-25T10:15:14.774685+0000","last_clean":"2024-04-25T10:15:14.774685+0000","last_became_active":"2024-04-25T10:15:12.771536+0000","last_became_peered":"2024-04-25T10:15:12.771536+0000","last_unstale":"2024-04-25T10:15:14.774685+0000","last_undegraded":"2024-04-25T10:15:14.774685+0000","last_fullsized":"2024-04-25T10:15:14.774685+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:41.790368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774918+0000","last_change":"2024-04-25T10:15:13.771636+0000","last_active":"2024-04-25T10:15:14.774918+0000","last_peered":"2024-04-25T10:15:14.774918+0000","last_clean":"2024-04-25T10:15:14.774918+0000","last_became_active":"2024-04-25T10:15:13.771323+0000","last_became_peered":"2024-04-25T10:15:13.771323+0000","last_unstale":"2024-04-25T10:15:14.774918+0000","last_undegraded":"2024-04-25T10:15:14.774918+0000","last_fullsized":"2024-04-25T10:15:14.774918+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.557403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.797777+0000","last_change":"2024-04-25T10:15:12.772343+0000","last_active":"2024-04-25T10:15:13.797777+0000","last_peered":"2024-04-25T10:15:13.797777+0000","last_clean":"2024-04-25T10:15:13.797777+0000","last_became_active":"2024-04-25T10:15:12.771722+0000","last_became_peered":"2024-04-25T10:15:12.771722+0000","last_unstale":"2024-04-25T10:15:13.797777+0000","last_undegraded":"2024-04-25T10:15:13.797777+0000","last_fullsized":"2024-04-25T10:15:13.797777+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:11.556702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.778822+0000","last_change":"2024-04-25T10:15:13.776141+0000","last_active":"2024-04-25T10:15:13.778822+0000","last_peered":"2024-04-25T10:15:13.778822+0000","last_clean":"2024-04-25T10:15:13.778822+0000","last_became_active":"2024-04-25T10:15:13.775796+0000","last_became_peered":"2024-04-25T10:15:13.775796+0000","last_unstale":"2024-04-25T10:15:13.778822+0000","last_undegraded":"2024-04-25T10:15:13.778822+0000","last_fullsized":"2024-04-25T10:15:13.778822+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:03.851138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"156'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795901+0000","last_change":"2024-04-25T10:15:12.775505+0000","last_active":"2024-04-25T10:15:13.795901+0000","last_peered":"2024-04-25T10:15:13.795901+0000","last_clean":"2024-04-25T10:15:13.795901+0000","last_became_active":"2024-04-25T10:15:12.775176+0000","last_became_peered":"2024-04-25T10:15:12.775176+0000","last_unstale":"2024-04-25T10:15:13.795901+0000","last_undegraded":"2024-04-25T10:15:13.795901+0000","last_fullsized":"2024-04-25T10:15:13.795901+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:57.554026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.788433+0000","last_change":"2024-04-25T10:15:13.785874+0000","last_active":"2024-04-25T10:15:13.788433+0000","last_peered":"2024-04-25T10:15:13.788433+0000","last_clean":"2024-04-25T10:15:13.788433+0000","last_became_active":"2024-04-25T10:15:13.785522+0000","last_became_peered":"2024-04-25T10:15:13.785522+0000","last_unstale":"2024-04-25T10:15:13.788433+0000","last_undegraded":"2024-04-25T10:15:13.788433+0000","last_fullsized":"2024-04-25T10:15:13.788433+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:09.716962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.774716+0000","last_change":"2024-04-25T10:15:12.768337+0000","last_active":"2024-04-25T10:15:13.774716+0000","last_peered":"2024-04-25T10:15:13.774716+0000","last_clean":"2024-04-25T10:15:13.774716+0000","last_became_active":"2024-04-25T10:15:12.767913+0000","last_became_peered":"2024-04-25T10:15:12.767913+0000","last_unstale":"2024-04-25T10:15:13.774716+0000","last_undegraded":"2024-04-25T10:15:13.774716+0000","last_fullsized":"2024-04-25T10:15:13.774716+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:02.847209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774807+0000","last_change":"2024-04-25T10:15:13.780085+0000","last_active":"2024-04-25T10:15:14.774807+0000","last_peered":"2024-04-25T10:15:14.774807+0000","last_clean":"2024-04-25T10:15:14.774807+0000","last_became_active":"2024-04-25T10:15:13.779334+0000","last_became_peered":"2024-04-25T10:15:13.779334+0000","last_unstale":"2024-04-25T10:15:14.774807+0000","last_undegraded":"2024-04-25T10:15:14.774807+0000","last_fullsized":"2024-04-25T10:15:14.774807+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:28.553065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.796260+0000","last_change":"2024-04-25T10:15:12.777020+0000","last_active":"2024-04-25T10:15:13.796260+0000","last_peered":"2024-04-25T10:15:13.796260+0000","last_clean":"2024-04-25T10:15:13.796260+0000","last_became_active":"2024-04-25T10:15:12.776611+0000","last_became_peered":"2024-04-25T10:15:12.776611+0000","last_unstale":"2024-04-25T10:15:13.796260+0000","last_undegraded":"2024-04-25T10:15:13.796260+0000","last_fullsized":"2024-04-25T10:15:13.796260+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:21.126555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795177+0000","last_change":"2024-04-25T10:15:13.786921+0000","last_active":"2024-04-25T10:15:13.795177+0000","last_peered":"2024-04-25T10:15:13.795177+0000","last_clean":"2024-04-25T10:15:13.795177+0000","last_became_active":"2024-04-25T10:15:13.783503+0000","last_became_peered":"2024-04-25T10:15:13.783503+0000","last_unstale":"2024-04-25T10:15:13.795177+0000","last_undegraded":"2024-04-25T10:15:13.795177+0000","last_fullsized":"2024-04-25T10:15:13.795177+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:07.491218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.769608+0000","last_change":"2024-04-25T10:15:12.761779+0000","last_active":"2024-04-25T10:15:13.769608+0000","last_peered":"2024-04-25T10:15:13.769608+0000","last_clean":"2024-04-25T10:15:13.769608+0000","last_became_active":"2024-04-25T10:15:12.760878+0000","last_became_peered":"2024-04-25T10:15:12.760878+0000","last_unstale":"2024-04-25T10:15:13.769608+0000","last_undegraded":"2024-04-25T10:15:13.769608+0000","last_fullsized":"2024-04-25T10:15:13.769608+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:58.354472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795895+0000","last_change":"2024-04-25T10:15:13.794092+0000","last_active":"2024-04-25T10:15:13.795895+0000","last_peered":"2024-04-25T10:15:13.795895+0000","last_clean":"2024-04-25T10:15:13.795895+0000","last_became_active":"2024-04-25T10:15:13.791933+0000","last_became_peered":"2024-04-25T10:15:13.791933+0000","last_unstale":"2024-04-25T10:15:13.795895+0000","last_undegraded":"2024-04-25T10:15:13.795895+0000","last_fullsized":"2024-04-25T10:15:13.795895+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:14.818954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.775942+0000","last_change":"2024-04-25T10:15:12.764264+0000","last_active":"2024-04-25T10:15:13.775942+0000","last_peered":"2024-04-25T10:15:13.775942+0000","last_clean":"2024-04-25T10:15:13.775942+0000","last_became_active":"2024-04-25T10:15:12.763996+0000","last_became_peered":"2024-04-25T10:15:12.763996+0000","last_unstale":"2024-04-25T10:15:13.775942+0000","last_undegraded":"2024-04-25T10:15:13.775942+0000","last_fullsized":"2024-04-25T10:15:13.775942+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:46.574502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.787414+0000","last_change":"2024-04-25T10:15:13.780395+0000","last_active":"2024-04-25T10:15:13.787414+0000","last_peered":"2024-04-25T10:15:13.787414+0000","last_clean":"2024-04-25T10:15:13.787414+0000","last_became_active":"2024-04-25T10:15:13.780075+0000","last_became_peered":"2024-04-25T10:15:13.780075+0000","last_unstale":"2024-04-25T10:15:13.787414+0000","last_undegraded":"2024-04-25T10:15:13.787414+0000","last_fullsized":"2024-04-25T10:15:13.787414+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:33.998118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.754187+0000","last_change":"2024-04-25T10:15:12.781594+0000","last_active":"2024-04-25T10:15:13.754187+0000","last_peered":"2024-04-25T10:15:13.754187+0000","last_clean":"2024-04-25T10:15:13.754187+0000","last_became_active":"2024-04-25T10:15:12.781033+0000","last_became_peered":"2024-04-25T10:15:12.781033+0000","last_unstale":"2024-04-25T10:15:13.754187+0000","last_undegraded":"2024-04-25T10:15:13.754187+0000","last_fullsized":"2024-04-25T10:15:13.754187+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:09.915844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.779036+0000","last_change":"2024-04-25T10:15:13.776857+0000","last_active":"2024-04-25T10:15:13.779036+0000","last_peered":"2024-04-25T10:15:13.779036+0000","last_clean":"2024-04-25T10:15:13.779036+0000","last_became_active":"2024-04-25T10:15:13.776552+0000","last_became_peered":"2024-04-25T10:15:13.776552+0000","last_unstale":"2024-04-25T10:15:13.779036+0000","last_undegraded":"2024-04-25T10:15:13.779036+0000","last_fullsized":"2024-04-25T10:15:13.779036+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:07.839844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.796149+0000","last_change":"2024-04-25T10:15:12.764289+0000","last_active":"2024-04-25T10:15:13.796149+0000","last_peered":"2024-04-25T10:15:13.796149+0000","last_clean":"2024-04-25T10:15:13.796149+0000","last_became_active":"2024-04-25T10:15:12.763956+0000","last_became_peered":"2024-04-25T10:15:12.763956+0000","last_unstale":"2024-04-25T10:15:13.796149+0000","last_undegraded":"2024-04-25T10:15:13.796149+0000","last_fullsized":"2024-04-25T10:15:13.796149+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:33.096261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.797580+0000","last_change":"2024-04-25T10:15:13.789318+0000","last_active":"2024-04-25T10:15:13.797580+0000","last_peered":"2024-04-25T10:15:13.797580+0000","last_clean":"2024-04-25T10:15:13.797580+0000","last_became_active":"2024-04-25T10:15:13.787734+0000","last_became_peered":"2024-04-25T10:15:13.787734+0000","last_unstale":"2024-04-25T10:15:13.797580+0000","last_undegraded":"2024-04-25T10:15:13.797580+0000","last_fullsized":"2024-04-25T10:15:13.797580+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:20.113079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774415+0000","last_change":"2024-04-25T10:15:12.774334+0000","last_active":"2024-04-25T10:15:14.774415+0000","last_peered":"2024-04-25T10:15:14.774415+0000","last_clean":"2024-04-25T10:15:14.774415+0000","last_became_active":"2024-04-25T10:15:12.773988+0000","last_became_peered":"2024-04-25T10:15:12.773988+0000","last_unstale":"2024-04-25T10:15:14.774415+0000","last_undegraded":"2024-04-25T10:15:14.774415+0000","last_fullsized":"2024-04-25T10:15:14.774415+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:03.058798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784988+0000","last_change":"2024-04-25T10:15:13.779576+0000","last_active":"2024-04-25T10:15:13.784988+0000","last_peered":"2024-04-25T10:15:13.784988+0000","last_clean":"2024-04-25T10:15:13.784988+0000","last_became_active":"2024-04-25T10:15:13.778670+0000","last_became_peered":"2024-04-25T10:15:13.778670+0000","last_unstale":"2024-04-25T10:15:13.784988+0000","last_undegraded":"2024-04-25T10:15:13.784988+0000","last_fullsized":"2024-04-25T10:15:13.784988+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:00.178751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774554+0000","last_change":"2024-04-25T10:15:12.770515+0000","last_active":"2024-04-25T10:15:14.774554+0000","last_peered":"2024-04-25T10:15:14.774554+0000","last_clean":"2024-04-25T10:15:14.774554+0000","last_became_active":"2024-04-25T10:15:12.770063+0000","last_became_peered":"2024-04-25T10:15:12.770063+0000","last_unstale":"2024-04-25T10:15:14.774554+0000","last_undegraded":"2024-04-25T10:15:14.774554+0000","last_fullsized":"2024-04-25T10:15:14.774554+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:00.146668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.797357+0000","last_change":"2024-04-25T10:15:13.787786+0000","last_active":"2024-04-25T10:15:13.797357+0000","last_peered":"2024-04-25T10:15:13.797357+0000","last_clean":"2024-04-25T10:15:13.797357+0000","last_became_active":"2024-04-25T10:15:13.787515+0000","last_became_peered":"2024-04-25T10:15:13.787515+0000","last_unstale":"2024-04-25T10:15:13.797357+0000","last_undegraded":"2024-04-25T10:15:13.797357+0000","last_fullsized":"2024-04-25T10:15:13.797357+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:11.287206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.777466+0000","last_change":"2024-04-25T10:15:12.781787+0000","last_active":"2024-04-25T10:15:13.777466+0000","last_peered":"2024-04-25T10:15:13.777466+0000","last_clean":"2024-04-25T10:15:13.777466+0000","last_became_active":"2024-04-25T10:15:12.781472+0000","last_became_peered":"2024-04-25T10:15:12.781472+0000","last_unstale":"2024-04-25T10:15:13.777466+0000","last_undegraded":"2024-04-25T10:15:13.777466+0000","last_fullsized":"2024-04-25T10:15:13.777466+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:35.513730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776210+0000","last_change":"2024-04-25T10:15:13.789390+0000","last_active":"2024-04-25T10:15:14.776210+0000","last_peered":"2024-04-25T10:15:14.776210+0000","last_clean":"2024-04-25T10:15:14.776210+0000","last_became_active":"2024-04-25T10:15:13.789147+0000","last_became_peered":"2024-04-25T10:15:13.789147+0000","last_unstale":"2024-04-25T10:15:14.776210+0000","last_undegraded":"2024-04-25T10:15:14.776210+0000","last_fullsized":"2024-04-25T10:15:14.776210+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.799267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.775724+0000","last_change":"2024-04-25T10:15:12.779302+0000","last_active":"2024-04-25T10:15:14.775724+0000","last_peered":"2024-04-25T10:15:14.775724+0000","last_clean":"2024-04-25T10:15:14.775724+0000","last_became_active":"2024-04-25T10:15:12.778801+0000","last_became_peered":"2024-04-25T10:15:12.778801+0000","last_unstale":"2024-04-25T10:15:14.775724+0000","last_undegraded":"2024-04-25T10:15:14.775724+0000","last_fullsized":"2024-04-25T10:15:14.775724+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:03.816976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784003+0000","last_change":"2024-04-25T10:15:13.775189+0000","last_active":"2024-04-25T10:15:13.784003+0000","last_peered":"2024-04-25T10:15:13.784003+0000","last_clean":"2024-04-25T10:15:13.784003+0000","last_became_active":"2024-04-25T10:15:13.774842+0000","last_became_peered":"2024-04-25T10:15:13.774842+0000","last_unstale":"2024-04-25T10:15:13.784003+0000","last_undegraded":"2024-04-25T10:15:13.784003+0000","last_fullsized":"2024-04-25T10:15:13.784003+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:28.192082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774084+0000","last_change":"2024-04-25T10:15:12.765151+0000","last_active":"2024-04-25T10:15:14.774084+0000","last_peered":"2024-04-25T10:15:14.774084+0000","last_clean":"2024-04-25T10:15:14.774084+0000","last_became_active":"2024-04-25T10:15:12.764613+0000","last_became_peered":"2024-04-25T10:15:12.764613+0000","last_unstale":"2024-04-25T10:15:14.774084+0000","last_undegraded":"2024-04-25T10:15:14.774084+0000","last_fullsized":"2024-04-25T10:15:14.774084+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:09.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774332+0000","last_change":"2024-04-25T10:15:13.779475+0000","last_active":"2024-04-25T10:15:14.774332+0000","last_peered":"2024-04-25T10:15:14.774332+0000","last_clean":"2024-04-25T10:15:14.774332+0000","last_became_active":"2024-04-25T10:15:13.778959+0000","last_became_peered":"2024-04-25T10:15:13.778959+0000","last_unstale":"2024-04-25T10:15:14.774332+0000","last_undegraded":"2024-04-25T10:15:14.774332+0000","last_fullsized":"2024-04-25T10:15:14.774332+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:09.927288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.769389+0000","last_change":"2024-04-25T10:15:12.765440+0000","last_active":"2024-04-25T10:15:13.769389+0000","last_peered":"2024-04-25T10:15:13.769389+0000","last_clean":"2024-04-25T10:15:13.769389+0000","last_became_active":"2024-04-25T10:15:12.764781+0000","last_became_peered":"2024-04-25T10:15:12.764781+0000","last_unstale":"2024-04-25T10:15:13.769389+0000","last_undegraded":"2024-04-25T10:15:13.769389+0000","last_fullsized":"2024-04-25T10:15:13.769389+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:41.840697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.798299+0000","last_change":"2024-04-25T10:15:13.795477+0000","last_active":"2024-04-25T10:15:13.798299+0000","last_peered":"2024-04-25T10:15:13.798299+0000","last_clean":"2024-04-25T10:15:13.798299+0000","last_became_active":"2024-04-25T10:15:13.795197+0000","last_became_peered":"2024-04-25T10:15:13.795197+0000","last_unstale":"2024-04-25T10:15:13.798299+0000","last_undegraded":"2024-04-25T10:15:13.798299+0000","last_fullsized":"2024-04-25T10:15:13.798299+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:38.084892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.761168+0000","last_change":"2024-04-25T10:15:12.778958+0000","last_active":"2024-04-25T10:15:13.761168+0000","last_peered":"2024-04-25T10:15:13.761168+0000","last_clean":"2024-04-25T10:15:13.761168+0000","last_became_active":"2024-04-25T10:15:12.778435+0000","last_became_peered":"2024-04-25T10:15:12.778435+0000","last_unstale":"2024-04-25T10:15:13.761168+0000","last_undegraded":"2024-04-25T10:15:13.761168+0000","last_fullsized":"2024-04-25T10:15:13.761168+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:03.058798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.764031+0000","last_change":"2024-04-25T10:15:13.760884+0000","last_active":"2024-04-25T10:15:13.764031+0000","last_peered":"2024-04-25T10:15:13.764031+0000","last_clean":"2024-04-25T10:15:13.764031+0000","last_became_active":"2024-04-25T10:15:13.760131+0000","last_became_peered":"2024-04-25T10:15:13.760131+0000","last_unstale":"2024-04-25T10:15:13.764031+0000","last_undegraded":"2024-04-25T10:15:13.764031+0000","last_fullsized":"2024-04-25T10:15:13.764031+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:22.130484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774305+0000","last_change":"2024-04-25T10:15:12.773685+0000","last_active":"2024-04-25T10:15:14.774305+0000","last_peered":"2024-04-25T10:15:14.774305+0000","last_clean":"2024-04-25T10:15:14.774305+0000","last_became_active":"2024-04-25T10:15:12.771759+0000","last_became_peered":"2024-04-25T10:15:12.771759+0000","last_unstale":"2024-04-25T10:15:14.774305+0000","last_undegraded":"2024-04-25T10:15:14.774305+0000","last_fullsized":"2024-04-25T10:15:14.774305+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:16.296846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":425,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.754326+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:15:13.754326+0000","last_peered":"2024-04-25T10:15:13.754326+0000","last_clean":"2024-04-25T10:15:13.754326+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:15:13.754326+0000","last_undegraded":"2024-04-25T10:15:13.754326+0000","last_fullsized":"2024-04-25T10:15:13.754326+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575015,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73940,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297900,"statfs":{"total":96636764160,"available":96561049600,"internally_reserved":0,"allocated":2555904,"data_stored":1510356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575016,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51148,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320692,"statfs":{"total":96636764160,"available":96584388608,"internally_reserved":0,"allocated":1961984,"data_stored":919988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47000,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324840,"statfs":{"total":96636764160,"available":96588636160,"internally_reserved":0,"allocated":1970176,"data_stored":920112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575015,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77536,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294304,"statfs":{"total":96636764160,"available":96557367296,"internally_reserved":0,"allocated":1978368,"data_stored":920752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575015,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77532,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294308,"statfs":{"total":96636764160,"available":96557371392,"internally_reserved":0,"allocated":1982464,"data_stored":922204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575015,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51152,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320688,"statfs":{"total":96636764160,"available":96584384512,"internally_reserved":0,"allocated":1961984,"data_stored":919988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47000,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324840,"statfs":{"total":96636764160,"available":96588636160,"internally_reserved":0,"allocated":1974272,"data_stored":921600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575016,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73944,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297896,"statfs":{"total":96636764160,"available":96561045504,"internally_reserved":0,"allocated":2564096,"data_stored":1510460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:16.788 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:16.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.942+0000 7f0311673640 1 Processor -- start 2024-04-25T10:15:16.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.942+0000 7f0311673640 1 -- start start 2024-04-25T10:15:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f0311673640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f030c15db00 0x7f030c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f0311673640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f030c0d0770 0x7f030c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f0311673640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f030c1614c0 0x7f030c161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f0311673640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030c0d1110 con 0x7f030c0d0770 2024-04-25T10:15:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f0311673640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030c0ceb20 con 0x7f030c1614c0 2024-04-25T10:15:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f0311673640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0b7740 con 0x7f030c15db00 2024-04-25T10:15:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030a7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f030c0d0770 0x7f030c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030a7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f030c0d0770 0x7f030c0d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54054/0 (socket says 172.21.15.133:54054) 2024-04-25T10:15:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030a7fc640 1 -- 172.21.15.133:0/2578404234 learned_addr learned my addr 172.21.15.133:0/2578404234 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:16.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030a7fc640 1 -- 172.21.15.133:0/2578404234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f030c15db00 msgr2=0x7f030c0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030affd640 1 --2- 172.21.15.133:0/2578404234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f030c15db00 0x7f030c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030b7fe640 1 --2- 172.21.15.133:0/2578404234 >> v2:172.21.15.162:3300/0 conn(0x7f030c1614c0 0x7f030c161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030a7fc640 1 --2- 172.21.15.133:0/2578404234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f030c15db00 0x7f030c0b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030a7fc640 1 -- 172.21.15.133:0/2578404234 >> v2:172.21.15.162:3300/0 conn(0x7f030c1614c0 msgr2=0x7f030c161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030a7fc640 1 --2- 172.21.15.133:0/2578404234 >> v2:172.21.15.162:3300/0 conn(0x7f030c1614c0 0x7f030c161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030a7fc640 1 -- 172.21.15.133:0/2578404234 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c0b7a20 con 0x7f030c0d0770 2024-04-25T10:15:16.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030affd640 1 --2- 172.21.15.133:0/2578404234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f030c15db00 0x7f030c0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:16.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f030a7fc640 1 --2- 172.21.15.133:0/2578404234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f030c0d0770 0x7f030c0d0bd0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f030c06cf70 tx=0x7f030001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e975e892d22c5d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f0309ffb640 1 -- 172.21.15.133:0/2578404234 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03000aa610 con 0x7f030c0d0770 2024-04-25T10:15:16.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f0309ffb640 1 -- 172.21.15.133:0/2578404234 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03000af070 con 0x7f030c0d0770 2024-04-25T10:15:16.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.946+0000 7f0309ffb640 1 -- 172.21.15.133:0/2578404234 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03000aa9d0 con 0x7f030c0d0770 2024-04-25T10:15:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 -- 172.21.15.133:0/2578404234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f030c0d0770 msgr2=0x7f030c0d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 --2- 172.21.15.133:0/2578404234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f030c0d0770 0x7f030c0d0bd0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f030c06cf70 tx=0x7f030001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 -- 172.21.15.133:0/2578404234 shutdown_connections 2024-04-25T10:15:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 --2- 172.21.15.133:0/2578404234 >> v2:172.21.15.162:3300/0 conn(0x7f030c1614c0 0x7f030c161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 --2- 172.21.15.133:0/2578404234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f030c0d0770 0x7f030c0d0bd0 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 --2- 172.21.15.133:0/2578404234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f030c15db00 0x7f030c0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 -- 172.21.15.133:0/2578404234 >> 172.21.15.133:0/2578404234 conn(0x7f030c0727c0 msgr2=0x7f030c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 -- 172.21.15.133:0/2578404234 shutdown_connections 2024-04-25T10:15:16.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 -- 172.21.15.133:0/2578404234 wait complete. 2024-04-25T10:15:16.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 Processor -- start 2024-04-25T10:15:16.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 -- start start 2024-04-25T10:15:16.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f030c0d0770 0x7f030c15dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f030c15db00 0x7f030c15e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f030c1614c0 0x7f030c1f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0b7430 con 0x7f030c1614c0 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0b82c0 con 0x7f030c0d0770 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0d18a0 con 0x7f030c15db00 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f030b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f030c1614c0 0x7f030c1f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f030b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f030c1614c0 0x7f030c1f6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54056/0 (socket says 172.21.15.133:54056) 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f030b7fe640 1 -- 172.21.15.133:0/1961391277 learned_addr learned my addr 172.21.15.133:0/1961391277 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f030affd640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f030c0d0770 0x7f030c15dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f030a7fc640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f030c15db00 0x7f030c15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f030b7fe640 1 -- 172.21.15.133:0/1961391277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f030c15db00 msgr2=0x7f030c15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f030b7fe640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f030c15db00 0x7f030c15e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f030b7fe640 1 -- 172.21.15.133:0/1961391277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f030c0d0770 msgr2=0x7f030c15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f030b7fe640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f030c0d0770 0x7f030c15dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f030b7fe640 1 -- 172.21.15.133:0/1961391277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c15f5d0 con 0x7f030c1614c0 2024-04-25T10:15:16.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f030b7fe640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f030c1614c0 0x7f030c1f6d70 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f030400cda0 tx=0x7f030400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f02f37fe640 1 -- 172.21.15.133:0/1961391277 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304003ba0 con 0x7f030c1614c0 2024-04-25T10:15:16.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f02f37fe640 1 -- 172.21.15.133:0/1961391277 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0304003d40 con 0x7f030c1614c0 2024-04-25T10:15:16.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 -- 172.21.15.133:0/1961391277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030c15ea50 con 0x7f030c1614c0 2024-04-25T10:15:16.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f02f37fe640 1 -- 172.21.15.133:0/1961391277 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304004240 con 0x7f030c1614c0 2024-04-25T10:15:16.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.950+0000 7f0311673640 1 -- 172.21.15.133:0/1961391277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030c15f010 con 0x7f030c1614c0 2024-04-25T10:15:16.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.954+0000 7f0311673640 1 -- 172.21.15.133:0/1961391277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030c068ef0 con 0x7f030c1614c0 2024-04-25T10:15:16.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.954+0000 7f02f37fe640 1 -- 172.21.15.133:0/1961391277 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03040043e0 con 0x7f030c1614c0 2024-04-25T10:15:16.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.954+0000 7f02f37fe640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f8066f90 0x7f02f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.954+0000 7f02f37fe640 1 -- 172.21.15.133:0/1961391277 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0304085c30 con 0x7f030c1614c0 2024-04-25T10:15:16.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.954+0000 7f030affd640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f8066f90 0x7f02f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.954+0000 7f030affd640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f8066f90 0x7f02f8069450 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00a990 tx=0x7f02fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:16.958+0000 7f02f37fe640 1 -- 172.21.15.133:0/1961391277 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0304056b40 con 0x7f030c1614c0 2024-04-25T10:15:17.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.110+0000 7f0311673640 1 -- 172.21.15.133:0/1961391277 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f030c0b2750 con 0x7f02f8066f90 2024-04-25T10:15:17.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.130+0000 7f02f37fe640 1 -- 172.21.15.133:0/1961391277 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252429 (secure 0 0 0) 0x7f030c0b2750 con 0x7f02f8066f90 2024-04-25T10:15:17.131 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:17.133 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:15:17.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 -- 172.21.15.133:0/1961391277 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f8066f90 msgr2=0x7f02f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f8066f90 0x7f02f8069450 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00a990 tx=0x7f02fc008040 comp rx=0 tx=0).stop 2024-04-25T10:15:17.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 -- 172.21.15.133:0/1961391277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f030c1614c0 msgr2=0x7f030c1f6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f030c1614c0 0x7f030c1f6d70 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f030400cda0 tx=0x7f030400a660 comp rx=0 tx=0).stop 2024-04-25T10:15:17.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 -- 172.21.15.133:0/1961391277 shutdown_connections 2024-04-25T10:15:17.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02f8066f90 0x7f02f8069450 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f030c1614c0 0x7f030c1f6d70 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f030c15db00 0x7f030c15e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 --2- 172.21.15.133:0/1961391277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f030c0d0770 0x7f030c15dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 -- 172.21.15.133:0/1961391277 >> 172.21.15.133:0/1961391277 conn(0x7f030c0727c0 msgr2=0x7f030c0c7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 -- 172.21.15.133:0/1961391277 shutdown_connections 2024-04-25T10:15:17.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.134+0000 7f0311673640 1 -- 172.21.15.133:0/1961391277 wait complete. 2024-04-25T10:15:17.150 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":556,"stamp":"2024-04-25T10:15:15.960544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":499252,"kb_used_data":16552,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754475468,"statfs":{"total":773094113280,"available":772582879232,"internally_reserved":0,"allocated":16949248,"data_stored":8545460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.227892"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776390+0000","last_change":"2024-04-25T10:15:13.777248+0000","last_active":"2024-04-25T10:15:14.776390+0000","last_peered":"2024-04-25T10:15:14.776390+0000","last_clean":"2024-04-25T10:15:14.776390+0000","last_became_active":"2024-04-25T10:15:13.776504+0000","last_became_peered":"2024-04-25T10:15:13.776504+0000","last_unstale":"2024-04-25T10:15:14.776390+0000","last_undegraded":"2024-04-25T10:15:14.776390+0000","last_fullsized":"2024-04-25T10:15:14.776390+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:51.396232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.783993+0000","last_change":"2024-04-25T10:15:13.772066+0000","last_active":"2024-04-25T10:15:13.783993+0000","last_peered":"2024-04-25T10:15:13.783993+0000","last_clean":"2024-04-25T10:15:13.783993+0000","last_became_active":"2024-04-25T10:15:13.771713+0000","last_became_peered":"2024-04-25T10:15:13.771713+0000","last_unstale":"2024-04-25T10:15:13.783993+0000","last_undegraded":"2024-04-25T10:15:13.783993+0000","last_fullsized":"2024-04-25T10:15:13.783993+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:07.491218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776056+0000","last_change":"2024-04-25T10:15:13.789783+0000","last_active":"2024-04-25T10:15:14.776056+0000","last_peered":"2024-04-25T10:15:14.776056+0000","last_clean":"2024-04-25T10:15:14.776056+0000","last_became_active":"2024-04-25T10:15:13.789394+0000","last_became_peered":"2024-04-25T10:15:13.789394+0000","last_unstale":"2024-04-25T10:15:14.776056+0000","last_undegraded":"2024-04-25T10:15:14.776056+0000","last_fullsized":"2024-04-25T10:15:14.776056+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:06.350508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776339+0000","last_change":"2024-04-25T10:15:13.778540+0000","last_active":"2024-04-25T10:15:14.776339+0000","last_peered":"2024-04-25T10:15:14.776339+0000","last_clean":"2024-04-25T10:15:14.776339+0000","last_became_active":"2024-04-25T10:15:13.777912+0000","last_became_peered":"2024-04-25T10:15:13.777912+0000","last_unstale":"2024-04-25T10:15:14.776339+0000","last_undegraded":"2024-04-25T10:15:14.776339+0000","last_fullsized":"2024-04-25T10:15:14.776339+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:04.037423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.791875+0000","last_change":"2024-04-25T10:15:13.783438+0000","last_active":"2024-04-25T10:15:13.791875+0000","last_peered":"2024-04-25T10:15:13.791875+0000","last_clean":"2024-04-25T10:15:13.791875+0000","last_became_active":"2024-04-25T10:15:13.782662+0000","last_became_peered":"2024-04-25T10:15:13.782662+0000","last_unstale":"2024-04-25T10:15:13.791875+0000","last_undegraded":"2024-04-25T10:15:13.791875+0000","last_fullsized":"2024-04-25T10:15:13.791875+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:07.893119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784093+0000","last_change":"2024-04-25T10:15:13.777716+0000","last_active":"2024-04-25T10:15:13.784093+0000","last_peered":"2024-04-25T10:15:13.784093+0000","last_clean":"2024-04-25T10:15:13.784093+0000","last_became_active":"2024-04-25T10:15:13.777321+0000","last_became_peered":"2024-04-25T10:15:13.777321+0000","last_unstale":"2024-04-25T10:15:13.784093+0000","last_undegraded":"2024-04-25T10:15:13.784093+0000","last_fullsized":"2024-04-25T10:15:13.784093+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:37.604335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.797459+0000","last_change":"2024-04-25T10:15:13.788631+0000","last_active":"2024-04-25T10:15:13.797459+0000","last_peered":"2024-04-25T10:15:13.797459+0000","last_clean":"2024-04-25T10:15:13.797459+0000","last_became_active":"2024-04-25T10:15:13.788272+0000","last_became_peered":"2024-04-25T10:15:13.788272+0000","last_unstale":"2024-04-25T10:15:13.797459+0000","last_undegraded":"2024-04-25T10:15:13.797459+0000","last_fullsized":"2024-04-25T10:15:13.797459+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:01.806338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.794435+0000","last_change":"2024-04-25T10:15:13.777268+0000","last_active":"2024-04-25T10:15:13.794435+0000","last_peered":"2024-04-25T10:15:13.794435+0000","last_clean":"2024-04-25T10:15:13.794435+0000","last_became_active":"2024-04-25T10:15:13.776983+0000","last_became_peered":"2024-04-25T10:15:13.776983+0000","last_unstale":"2024-04-25T10:15:13.794435+0000","last_undegraded":"2024-04-25T10:15:13.794435+0000","last_fullsized":"2024-04-25T10:15:13.794435+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:55.865610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.793938+0000","last_change":"2024-04-25T10:15:13.783548+0000","last_active":"2024-04-25T10:15:13.793938+0000","last_peered":"2024-04-25T10:15:13.793938+0000","last_clean":"2024-04-25T10:15:13.793938+0000","last_became_active":"2024-04-25T10:15:13.783226+0000","last_became_peered":"2024-04-25T10:15:13.783226+0000","last_unstale":"2024-04-25T10:15:13.793938+0000","last_undegraded":"2024-04-25T10:15:13.793938+0000","last_fullsized":"2024-04-25T10:15:13.793938+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.227044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.797858+0000","last_change":"2024-04-25T10:15:13.792924+0000","last_active":"2024-04-25T10:15:13.797858+0000","last_peered":"2024-04-25T10:15:13.797858+0000","last_clean":"2024-04-25T10:15:13.797858+0000","last_became_active":"2024-04-25T10:15:13.788945+0000","last_became_peered":"2024-04-25T10:15:13.788945+0000","last_unstale":"2024-04-25T10:15:13.797858+0000","last_undegraded":"2024-04-25T10:15:13.797858+0000","last_fullsized":"2024-04-25T10:15:13.797858+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:28.526079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.788559+0000","last_change":"2024-04-25T10:15:13.786249+0000","last_active":"2024-04-25T10:15:13.788559+0000","last_peered":"2024-04-25T10:15:13.788559+0000","last_clean":"2024-04-25T10:15:13.788559+0000","last_became_active":"2024-04-25T10:15:13.785597+0000","last_became_peered":"2024-04-25T10:15:13.785597+0000","last_unstale":"2024-04-25T10:15:13.788559+0000","last_undegraded":"2024-04-25T10:15:13.788559+0000","last_fullsized":"2024-04-25T10:15:13.788559+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.227044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.787357+0000","last_change":"2024-04-25T10:15:13.782743+0000","last_active":"2024-04-25T10:15:13.787357+0000","last_peered":"2024-04-25T10:15:13.787357+0000","last_clean":"2024-04-25T10:15:13.787357+0000","last_became_active":"2024-04-25T10:15:13.782385+0000","last_became_peered":"2024-04-25T10:15:13.782385+0000","last_unstale":"2024-04-25T10:15:13.787357+0000","last_undegraded":"2024-04-25T10:15:13.787357+0000","last_fullsized":"2024-04-25T10:15:13.787357+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:14.818954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.773824+0000","last_change":"2024-04-25T10:15:13.766168+0000","last_active":"2024-04-25T10:15:14.773824+0000","last_peered":"2024-04-25T10:15:14.773824+0000","last_clean":"2024-04-25T10:15:14.773824+0000","last_became_active":"2024-04-25T10:15:13.765672+0000","last_became_peered":"2024-04-25T10:15:13.765672+0000","last_unstale":"2024-04-25T10:15:14.773824+0000","last_undegraded":"2024-04-25T10:15:14.773824+0000","last_fullsized":"2024-04-25T10:15:14.773824+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:14.818954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795370+0000","last_change":"2024-04-25T10:15:13.787363+0000","last_active":"2024-04-25T10:15:13.795370+0000","last_peered":"2024-04-25T10:15:13.795370+0000","last_clean":"2024-04-25T10:15:13.795370+0000","last_became_active":"2024-04-25T10:15:13.787089+0000","last_became_peered":"2024-04-25T10:15:13.787089+0000","last_unstale":"2024-04-25T10:15:13.795370+0000","last_undegraded":"2024-04-25T10:15:13.795370+0000","last_fullsized":"2024-04-25T10:15:13.795370+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:28.192082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.787761+0000","last_change":"2024-04-25T10:15:13.783393+0000","last_active":"2024-04-25T10:15:13.787761+0000","last_peered":"2024-04-25T10:15:13.787761+0000","last_clean":"2024-04-25T10:15:13.787761+0000","last_became_active":"2024-04-25T10:15:13.783069+0000","last_became_peered":"2024-04-25T10:15:13.783069+0000","last_unstale":"2024-04-25T10:15:13.787761+0000","last_undegraded":"2024-04-25T10:15:13.787761+0000","last_fullsized":"2024-04-25T10:15:13.787761+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:12.560631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.775547+0000","last_change":"2024-04-25T10:15:13.781303+0000","last_active":"2024-04-25T10:15:14.775547+0000","last_peered":"2024-04-25T10:15:14.775547+0000","last_clean":"2024-04-25T10:15:14.775547+0000","last_became_active":"2024-04-25T10:15:13.780471+0000","last_became_peered":"2024-04-25T10:15:13.780471+0000","last_unstale":"2024-04-25T10:15:14.775547+0000","last_undegraded":"2024-04-25T10:15:14.775547+0000","last_fullsized":"2024-04-25T10:15:14.775547+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:38.716662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.793763+0000","last_change":"2024-04-25T10:15:13.776284+0000","last_active":"2024-04-25T10:15:13.793763+0000","last_peered":"2024-04-25T10:15:13.793763+0000","last_clean":"2024-04-25T10:15:13.793763+0000","last_became_active":"2024-04-25T10:15:13.775903+0000","last_became_peered":"2024-04-25T10:15:13.775903+0000","last_unstale":"2024-04-25T10:15:13.793763+0000","last_undegraded":"2024-04-25T10:15:13.793763+0000","last_fullsized":"2024-04-25T10:15:13.793763+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:27.442307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774838+0000","last_change":"2024-04-25T10:15:13.790271+0000","last_active":"2024-04-25T10:15:14.774838+0000","last_peered":"2024-04-25T10:15:14.774838+0000","last_clean":"2024-04-25T10:15:14.774838+0000","last_became_active":"2024-04-25T10:15:13.789942+0000","last_became_peered":"2024-04-25T10:15:13.789942+0000","last_unstale":"2024-04-25T10:15:14.774838+0000","last_undegraded":"2024-04-25T10:15:14.774838+0000","last_fullsized":"2024-04-25T10:15:14.774838+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:04.820906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.799256+0000","last_change":"2024-04-25T10:15:13.798652+0000","last_active":"2024-04-25T10:15:13.799256+0000","last_peered":"2024-04-25T10:15:13.799256+0000","last_clean":"2024-04-25T10:15:13.799256+0000","last_became_active":"2024-04-25T10:15:13.798281+0000","last_became_peered":"2024-04-25T10:15:13.798281+0000","last_unstale":"2024-04-25T10:15:13.799256+0000","last_undegraded":"2024-04-25T10:15:13.799256+0000","last_fullsized":"2024-04-25T10:15:13.799256+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:11.287206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.798225+0000","last_change":"2024-04-25T10:15:13.792698+0000","last_active":"2024-04-25T10:15:13.798225+0000","last_peered":"2024-04-25T10:15:13.798225+0000","last_clean":"2024-04-25T10:15:13.798225+0000","last_became_active":"2024-04-25T10:15:13.792451+0000","last_became_peered":"2024-04-25T10:15:13.792451+0000","last_unstale":"2024-04-25T10:15:13.798225+0000","last_undegraded":"2024-04-25T10:15:13.798225+0000","last_fullsized":"2024-04-25T10:15:13.798225+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:52.275872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.773873+0000","last_change":"2024-04-25T10:15:13.776270+0000","last_active":"2024-04-25T10:15:14.773873+0000","last_peered":"2024-04-25T10:15:14.773873+0000","last_clean":"2024-04-25T10:15:14.773873+0000","last_became_active":"2024-04-25T10:15:13.775699+0000","last_became_peered":"2024-04-25T10:15:13.775699+0000","last_unstale":"2024-04-25T10:15:14.773873+0000","last_undegraded":"2024-04-25T10:15:14.773873+0000","last_fullsized":"2024-04-25T10:15:14.773873+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:33.998118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784061+0000","last_change":"2024-04-25T10:15:13.776160+0000","last_active":"2024-04-25T10:15:13.784061+0000","last_peered":"2024-04-25T10:15:13.784061+0000","last_clean":"2024-04-25T10:15:13.784061+0000","last_became_active":"2024-04-25T10:15:13.775795+0000","last_became_peered":"2024-04-25T10:15:13.775795+0000","last_unstale":"2024-04-25T10:15:13.784061+0000","last_undegraded":"2024-04-25T10:15:13.784061+0000","last_fullsized":"2024-04-25T10:15:13.784061+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:36.208593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795823+0000","last_change":"2024-04-25T10:15:13.791580+0000","last_active":"2024-04-25T10:15:13.795823+0000","last_peered":"2024-04-25T10:15:13.795823+0000","last_clean":"2024-04-25T10:15:13.795823+0000","last_became_active":"2024-04-25T10:15:13.790925+0000","last_became_peered":"2024-04-25T10:15:13.790925+0000","last_unstale":"2024-04-25T10:15:13.795823+0000","last_undegraded":"2024-04-25T10:15:13.795823+0000","last_fullsized":"2024-04-25T10:15:13.795823+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:37.604335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.775661+0000","last_change":"2024-04-25T10:15:13.781331+0000","last_active":"2024-04-25T10:15:14.775661+0000","last_peered":"2024-04-25T10:15:14.775661+0000","last_clean":"2024-04-25T10:15:14.775661+0000","last_became_active":"2024-04-25T10:15:13.780994+0000","last_became_peered":"2024-04-25T10:15:13.780994+0000","last_unstale":"2024-04-25T10:15:14.775661+0000","last_undegraded":"2024-04-25T10:15:14.775661+0000","last_fullsized":"2024-04-25T10:15:14.775661+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:49.726501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774549+0000","last_change":"2024-04-25T10:15:13.770157+0000","last_active":"2024-04-25T10:15:14.774549+0000","last_peered":"2024-04-25T10:15:14.774549+0000","last_clean":"2024-04-25T10:15:14.774549+0000","last_became_active":"2024-04-25T10:15:13.769866+0000","last_became_peered":"2024-04-25T10:15:13.769866+0000","last_unstale":"2024-04-25T10:15:14.774549+0000","last_undegraded":"2024-04-25T10:15:14.774549+0000","last_fullsized":"2024-04-25T10:15:14.774549+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:48.708916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784937+0000","last_change":"2024-04-25T10:15:13.778869+0000","last_active":"2024-04-25T10:15:13.784937+0000","last_peered":"2024-04-25T10:15:13.784937+0000","last_clean":"2024-04-25T10:15:13.784937+0000","last_became_active":"2024-04-25T10:15:13.778584+0000","last_became_peered":"2024-04-25T10:15:13.778584+0000","last_unstale":"2024-04-25T10:15:13.784937+0000","last_undegraded":"2024-04-25T10:15:13.784937+0000","last_fullsized":"2024-04-25T10:15:13.784937+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:04.220167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.788021+0000","last_change":"2024-04-25T10:15:13.787035+0000","last_active":"2024-04-25T10:15:13.788021+0000","last_peered":"2024-04-25T10:15:13.788021+0000","last_clean":"2024-04-25T10:15:13.788021+0000","last_became_active":"2024-04-25T10:15:13.786291+0000","last_became_peered":"2024-04-25T10:15:13.786291+0000","last_unstale":"2024-04-25T10:15:13.788021+0000","last_undegraded":"2024-04-25T10:15:13.788021+0000","last_fullsized":"2024-04-25T10:15:13.788021+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:28.723944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.773690+0000","last_change":"2024-04-25T10:15:13.780046+0000","last_active":"2024-04-25T10:15:14.773690+0000","last_peered":"2024-04-25T10:15:14.773690+0000","last_clean":"2024-04-25T10:15:14.773690+0000","last_became_active":"2024-04-25T10:15:13.779674+0000","last_became_peered":"2024-04-25T10:15:13.779674+0000","last_unstale":"2024-04-25T10:15:14.773690+0000","last_undegraded":"2024-04-25T10:15:14.773690+0000","last_fullsized":"2024-04-25T10:15:14.773690+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:36.208593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784324+0000","last_change":"2024-04-25T10:15:13.770250+0000","last_active":"2024-04-25T10:15:13.784324+0000","last_peered":"2024-04-25T10:15:13.784324+0000","last_clean":"2024-04-25T10:15:13.784324+0000","last_became_active":"2024-04-25T10:15:13.769894+0000","last_became_peered":"2024-04-25T10:15:13.769894+0000","last_unstale":"2024-04-25T10:15:13.784324+0000","last_undegraded":"2024-04-25T10:15:13.784324+0000","last_fullsized":"2024-04-25T10:15:13.784324+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:07.893119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774914+0000","last_change":"2024-04-25T10:15:13.784969+0000","last_active":"2024-04-25T10:15:14.774914+0000","last_peered":"2024-04-25T10:15:14.774914+0000","last_clean":"2024-04-25T10:15:14.774914+0000","last_became_active":"2024-04-25T10:15:13.784635+0000","last_became_peered":"2024-04-25T10:15:13.784635+0000","last_unstale":"2024-04-25T10:15:14.774914+0000","last_undegraded":"2024-04-25T10:15:14.774914+0000","last_fullsized":"2024-04-25T10:15:14.774914+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:49.030545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.781667+0000","last_change":"2024-04-25T10:15:13.780351+0000","last_active":"2024-04-25T10:15:13.781667+0000","last_peered":"2024-04-25T10:15:13.781667+0000","last_clean":"2024-04-25T10:15:13.781667+0000","last_became_active":"2024-04-25T10:15:13.779997+0000","last_became_peered":"2024-04-25T10:15:13.779997+0000","last_unstale":"2024-04-25T10:15:13.781667+0000","last_undegraded":"2024-04-25T10:15:13.781667+0000","last_fullsized":"2024-04-25T10:15:13.781667+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:27.442307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774432+0000","last_change":"2024-04-25T10:15:13.763480+0000","last_active":"2024-04-25T10:15:14.774432+0000","last_peered":"2024-04-25T10:15:14.774432+0000","last_clean":"2024-04-25T10:15:14.774432+0000","last_became_active":"2024-04-25T10:15:13.762642+0000","last_became_peered":"2024-04-25T10:15:13.762642+0000","last_unstale":"2024-04-25T10:15:14.774432+0000","last_undegraded":"2024-04-25T10:15:14.774432+0000","last_fullsized":"2024-04-25T10:15:14.774432+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:52.397943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.783874+0000","last_change":"2024-04-25T10:15:13.769289+0000","last_active":"2024-04-25T10:15:13.783874+0000","last_peered":"2024-04-25T10:15:13.783874+0000","last_clean":"2024-04-25T10:15:13.783874+0000","last_became_active":"2024-04-25T10:15:13.768386+0000","last_became_peered":"2024-04-25T10:15:13.768386+0000","last_unstale":"2024-04-25T10:15:13.783874+0000","last_undegraded":"2024-04-25T10:15:13.783874+0000","last_fullsized":"2024-04-25T10:15:13.783874+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:49.584476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"156'4","reported_seq":18,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776640+0000","last_change":"2024-04-25T10:15:12.775976+0000","last_active":"2024-04-25T10:15:14.776640+0000","last_peered":"2024-04-25T10:15:14.776640+0000","last_clean":"2024-04-25T10:15:14.776640+0000","last_became_active":"2024-04-25T10:15:12.775370+0000","last_became_peered":"2024-04-25T10:15:12.775370+0000","last_unstale":"2024-04-25T10:15:14.776640+0000","last_undegraded":"2024-04-25T10:15:14.776640+0000","last_fullsized":"2024-04-25T10:15:14.776640+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:48.026615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774032+0000","last_change":"2024-04-25T10:15:13.760187+0000","last_active":"2024-04-25T10:15:14.774032+0000","last_peered":"2024-04-25T10:15:14.774032+0000","last_clean":"2024-04-25T10:15:14.774032+0000","last_became_active":"2024-04-25T10:15:13.759450+0000","last_became_peered":"2024-04-25T10:15:13.759450+0000","last_unstale":"2024-04-25T10:15:14.774032+0000","last_undegraded":"2024-04-25T10:15:14.774032+0000","last_fullsized":"2024-04-25T10:15:14.774032+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:28.723944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"156'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.798538+0000","last_change":"2024-04-25T10:15:12.776401+0000","last_active":"2024-04-25T10:15:13.798538+0000","last_peered":"2024-04-25T10:15:13.798538+0000","last_clean":"2024-04-25T10:15:13.798538+0000","last_became_active":"2024-04-25T10:15:12.775255+0000","last_became_peered":"2024-04-25T10:15:12.775255+0000","last_unstale":"2024-04-25T10:15:13.798538+0000","last_undegraded":"2024-04-25T10:15:13.798538+0000","last_fullsized":"2024-04-25T10:15:13.798538+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:48.580547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795773+0000","last_change":"2024-04-25T10:15:13.791547+0000","last_active":"2024-04-25T10:15:13.795773+0000","last_peered":"2024-04-25T10:15:13.795773+0000","last_clean":"2024-04-25T10:15:13.795773+0000","last_became_active":"2024-04-25T10:15:13.791203+0000","last_became_peered":"2024-04-25T10:15:13.791203+0000","last_unstale":"2024-04-25T10:15:13.795773+0000","last_undegraded":"2024-04-25T10:15:13.795773+0000","last_fullsized":"2024-04-25T10:15:13.795773+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:36.208593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.787232+0000","last_change":"2024-04-25T10:15:12.766821+0000","last_active":"2024-04-25T10:15:13.787232+0000","last_peered":"2024-04-25T10:15:13.787232+0000","last_clean":"2024-04-25T10:15:13.787232+0000","last_became_active":"2024-04-25T10:15:12.766179+0000","last_became_peered":"2024-04-25T10:15:12.766179+0000","last_unstale":"2024-04-25T10:15:13.787232+0000","last_undegraded":"2024-04-25T10:15:13.787232+0000","last_fullsized":"2024-04-25T10:15:13.787232+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:09.517126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.783792+0000","last_change":"2024-04-25T10:15:13.769091+0000","last_active":"2024-04-25T10:15:13.783792+0000","last_peered":"2024-04-25T10:15:13.783792+0000","last_clean":"2024-04-25T10:15:13.783792+0000","last_became_active":"2024-04-25T10:15:13.768722+0000","last_became_peered":"2024-04-25T10:15:13.768722+0000","last_unstale":"2024-04-25T10:15:13.783792+0000","last_undegraded":"2024-04-25T10:15:13.783792+0000","last_fullsized":"2024-04-25T10:15:13.783792+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:34.100190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.787359+0000","last_change":"2024-04-25T10:15:12.774941+0000","last_active":"2024-04-25T10:15:13.787359+0000","last_peered":"2024-04-25T10:15:13.787359+0000","last_clean":"2024-04-25T10:15:13.787359+0000","last_became_active":"2024-04-25T10:15:12.774655+0000","last_became_peered":"2024-04-25T10:15:12.774655+0000","last_unstale":"2024-04-25T10:15:13.787359+0000","last_undegraded":"2024-04-25T10:15:13.787359+0000","last_fullsized":"2024-04-25T10:15:13.787359+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:44.691915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.794077+0000","last_change":"2024-04-25T10:15:13.782845+0000","last_active":"2024-04-25T10:15:13.794077+0000","last_peered":"2024-04-25T10:15:13.794077+0000","last_clean":"2024-04-25T10:15:13.794077+0000","last_became_active":"2024-04-25T10:15:13.782172+0000","last_became_peered":"2024-04-25T10:15:13.782172+0000","last_unstale":"2024-04-25T10:15:13.794077+0000","last_undegraded":"2024-04-25T10:15:13.794077+0000","last_fullsized":"2024-04-25T10:15:13.794077+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:09.716962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"156'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.793913+0000","last_change":"2024-04-25T10:15:12.771422+0000","last_active":"2024-04-25T10:15:13.793913+0000","last_peered":"2024-04-25T10:15:13.793913+0000","last_clean":"2024-04-25T10:15:13.793913+0000","last_became_active":"2024-04-25T10:15:12.770843+0000","last_became_peered":"2024-04-25T10:15:12.770843+0000","last_unstale":"2024-04-25T10:15:13.793913+0000","last_undegraded":"2024-04-25T10:15:13.793913+0000","last_fullsized":"2024-04-25T10:15:13.793913+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:06.835915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.785104+0000","last_change":"2024-04-25T10:15:13.778440+0000","last_active":"2024-04-25T10:15:13.785104+0000","last_peered":"2024-04-25T10:15:13.785104+0000","last_clean":"2024-04-25T10:15:13.785104+0000","last_became_active":"2024-04-25T10:15:13.777800+0000","last_became_peered":"2024-04-25T10:15:13.777800+0000","last_unstale":"2024-04-25T10:15:13.785104+0000","last_undegraded":"2024-04-25T10:15:13.785104+0000","last_fullsized":"2024-04-25T10:15:13.785104+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:21.531667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"156'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795400+0000","last_change":"2024-04-25T10:15:12.765558+0000","last_active":"2024-04-25T10:15:13.795400+0000","last_peered":"2024-04-25T10:15:13.795400+0000","last_clean":"2024-04-25T10:15:13.795400+0000","last_became_active":"2024-04-25T10:15:12.765227+0000","last_became_peered":"2024-04-25T10:15:12.765227+0000","last_unstale":"2024-04-25T10:15:13.795400+0000","last_undegraded":"2024-04-25T10:15:13.795400+0000","last_fullsized":"2024-04-25T10:15:13.795400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:41.790368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.782024+0000","last_change":"2024-04-25T10:15:13.777079+0000","last_active":"2024-04-25T10:15:13.782024+0000","last_peered":"2024-04-25T10:15:13.782024+0000","last_clean":"2024-04-25T10:15:13.782024+0000","last_became_active":"2024-04-25T10:15:13.775146+0000","last_became_peered":"2024-04-25T10:15:13.775146+0000","last_unstale":"2024-04-25T10:15:13.782024+0000","last_undegraded":"2024-04-25T10:15:13.782024+0000","last_fullsized":"2024-04-25T10:15:13.782024+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:24.767040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.775739+0000","last_change":"2024-04-25T10:15:12.768937+0000","last_active":"2024-04-25T10:15:13.775739+0000","last_peered":"2024-04-25T10:15:13.775739+0000","last_clean":"2024-04-25T10:15:13.775739+0000","last_became_active":"2024-04-25T10:15:12.768546+0000","last_became_peered":"2024-04-25T10:15:12.768546+0000","last_unstale":"2024-04-25T10:15:13.775739+0000","last_undegraded":"2024-04-25T10:15:13.775739+0000","last_fullsized":"2024-04-25T10:15:13.775739+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:05.923856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776509+0000","last_change":"2024-04-25T10:15:13.776821+0000","last_active":"2024-04-25T10:15:14.776509+0000","last_peered":"2024-04-25T10:15:14.776509+0000","last_clean":"2024-04-25T10:15:14.776509+0000","last_became_active":"2024-04-25T10:15:13.776085+0000","last_became_peered":"2024-04-25T10:15:13.776085+0000","last_unstale":"2024-04-25T10:15:14.776509+0000","last_undegraded":"2024-04-25T10:15:14.776509+0000","last_fullsized":"2024-04-25T10:15:14.776509+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:49.158981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.771020+0000","last_change":"2024-04-25T10:15:12.774878+0000","last_active":"2024-04-25T10:15:13.771020+0000","last_peered":"2024-04-25T10:15:13.771020+0000","last_clean":"2024-04-25T10:15:13.771020+0000","last_became_active":"2024-04-25T10:15:12.774467+0000","last_became_peered":"2024-04-25T10:15:12.774467+0000","last_unstale":"2024-04-25T10:15:13.771020+0000","last_undegraded":"2024-04-25T10:15:13.771020+0000","last_fullsized":"2024-04-25T10:15:13.771020+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:58.354472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.783904+0000","last_change":"2024-04-25T10:15:13.769687+0000","last_active":"2024-04-25T10:15:13.783904+0000","last_peered":"2024-04-25T10:15:13.783904+0000","last_clean":"2024-04-25T10:15:13.783904+0000","last_became_active":"2024-04-25T10:15:13.768960+0000","last_became_peered":"2024-04-25T10:15:13.768960+0000","last_unstale":"2024-04-25T10:15:13.783904+0000","last_undegraded":"2024-04-25T10:15:13.783904+0000","last_fullsized":"2024-04-25T10:15:13.783904+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:58.557955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774345+0000","last_change":"2024-04-25T10:15:12.773471+0000","last_active":"2024-04-25T10:15:14.774345+0000","last_peered":"2024-04-25T10:15:14.774345+0000","last_clean":"2024-04-25T10:15:14.774345+0000","last_became_active":"2024-04-25T10:15:12.771864+0000","last_became_peered":"2024-04-25T10:15:12.771864+0000","last_unstale":"2024-04-25T10:15:14.774345+0000","last_undegraded":"2024-04-25T10:15:14.774345+0000","last_fullsized":"2024-04-25T10:15:14.774345+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:04.795357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.773770+0000","last_change":"2024-04-25T10:15:13.769915+0000","last_active":"2024-04-25T10:15:14.773770+0000","last_peered":"2024-04-25T10:15:14.773770+0000","last_clean":"2024-04-25T10:15:14.773770+0000","last_became_active":"2024-04-25T10:15:13.769378+0000","last_became_peered":"2024-04-25T10:15:13.769378+0000","last_unstale":"2024-04-25T10:15:14.773770+0000","last_undegraded":"2024-04-25T10:15:14.773770+0000","last_fullsized":"2024-04-25T10:15:14.773770+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:37.604335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"156'3","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776327+0000","last_change":"2024-04-25T10:15:12.777785+0000","last_active":"2024-04-25T10:15:14.776327+0000","last_peered":"2024-04-25T10:15:14.776327+0000","last_clean":"2024-04-25T10:15:14.776327+0000","last_became_active":"2024-04-25T10:15:12.777045+0000","last_became_peered":"2024-04-25T10:15:12.777045+0000","last_unstale":"2024-04-25T10:15:14.776327+0000","last_undegraded":"2024-04-25T10:15:14.776327+0000","last_fullsized":"2024-04-25T10:15:14.776327+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:27.149157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.785233+0000","last_change":"2024-04-25T10:15:13.784254+0000","last_active":"2024-04-25T10:15:13.785233+0000","last_peered":"2024-04-25T10:15:13.785233+0000","last_clean":"2024-04-25T10:15:13.785233+0000","last_became_active":"2024-04-25T10:15:13.783932+0000","last_became_peered":"2024-04-25T10:15:13.783932+0000","last_unstale":"2024-04-25T10:15:13.785233+0000","last_undegraded":"2024-04-25T10:15:13.785233+0000","last_fullsized":"2024-04-25T10:15:13.785233+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:00.434491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"156'6","reported_seq":20,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774212+0000","last_change":"2024-04-25T10:15:12.773735+0000","last_active":"2024-04-25T10:15:14.774212+0000","last_peered":"2024-04-25T10:15:14.774212+0000","last_clean":"2024-04-25T10:15:14.774212+0000","last_became_active":"2024-04-25T10:15:12.773371+0000","last_became_peered":"2024-04-25T10:15:12.773371+0000","last_unstale":"2024-04-25T10:15:14.774212+0000","last_undegraded":"2024-04-25T10:15:14.774212+0000","last_fullsized":"2024-04-25T10:15:14.774212+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:37.073666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774727+0000","last_change":"2024-04-25T10:15:13.763459+0000","last_active":"2024-04-25T10:15:14.774727+0000","last_peered":"2024-04-25T10:15:14.774727+0000","last_clean":"2024-04-25T10:15:14.774727+0000","last_became_active":"2024-04-25T10:15:13.760892+0000","last_became_peered":"2024-04-25T10:15:13.760892+0000","last_unstale":"2024-04-25T10:15:14.774727+0000","last_undegraded":"2024-04-25T10:15:14.774727+0000","last_fullsized":"2024-04-25T10:15:14.774727+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:28.153086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795557+0000","last_change":"2024-04-25T10:15:12.777614+0000","last_active":"2024-04-25T10:15:13.795557+0000","last_peered":"2024-04-25T10:15:13.795557+0000","last_clean":"2024-04-25T10:15:13.795557+0000","last_became_active":"2024-04-25T10:15:12.777135+0000","last_became_peered":"2024-04-25T10:15:12.777135+0000","last_unstale":"2024-04-25T10:15:13.795557+0000","last_undegraded":"2024-04-25T10:15:13.795557+0000","last_fullsized":"2024-04-25T10:15:13.795557+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:09.915844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.798640+0000","last_change":"2024-04-25T10:15:13.793815+0000","last_active":"2024-04-25T10:15:13.798640+0000","last_peered":"2024-04-25T10:15:13.798640+0000","last_clean":"2024-04-25T10:15:13.798640+0000","last_became_active":"2024-04-25T10:15:13.793459+0000","last_became_peered":"2024-04-25T10:15:13.793459+0000","last_unstale":"2024-04-25T10:15:13.798640+0000","last_undegraded":"2024-04-25T10:15:13.798640+0000","last_fullsized":"2024-04-25T10:15:13.798640+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:31.373069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.773954+0000","last_change":"2024-04-25T10:15:12.762355+0000","last_active":"2024-04-25T10:15:14.773954+0000","last_peered":"2024-04-25T10:15:14.773954+0000","last_clean":"2024-04-25T10:15:14.773954+0000","last_became_active":"2024-04-25T10:15:12.762056+0000","last_became_peered":"2024-04-25T10:15:12.762056+0000","last_unstale":"2024-04-25T10:15:14.773954+0000","last_undegraded":"2024-04-25T10:15:14.773954+0000","last_fullsized":"2024-04-25T10:15:14.773954+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:30.369140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.788393+0000","last_change":"2024-04-25T10:15:13.784694+0000","last_active":"2024-04-25T10:15:13.788393+0000","last_peered":"2024-04-25T10:15:13.788393+0000","last_clean":"2024-04-25T10:15:13.788393+0000","last_became_active":"2024-04-25T10:15:13.784401+0000","last_became_peered":"2024-04-25T10:15:13.784401+0000","last_unstale":"2024-04-25T10:15:13.788393+0000","last_undegraded":"2024-04-25T10:15:13.788393+0000","last_fullsized":"2024-04-25T10:15:13.788393+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:55.865610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.769277+0000","last_change":"2024-04-25T10:15:12.767004+0000","last_active":"2024-04-25T10:15:13.769277+0000","last_peered":"2024-04-25T10:15:13.769277+0000","last_clean":"2024-04-25T10:15:13.769277+0000","last_became_active":"2024-04-25T10:15:12.766468+0000","last_became_peered":"2024-04-25T10:15:12.766468+0000","last_unstale":"2024-04-25T10:15:13.769277+0000","last_undegraded":"2024-04-25T10:15:13.769277+0000","last_fullsized":"2024-04-25T10:15:13.769277+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:40.043055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784195+0000","last_change":"2024-04-25T10:15:13.778256+0000","last_active":"2024-04-25T10:15:13.784195+0000","last_peered":"2024-04-25T10:15:13.784195+0000","last_clean":"2024-04-25T10:15:13.784195+0000","last_became_active":"2024-04-25T10:15:13.777826+0000","last_became_peered":"2024-04-25T10:15:13.777826+0000","last_unstale":"2024-04-25T10:15:13.784195+0000","last_undegraded":"2024-04-25T10:15:13.784195+0000","last_fullsized":"2024-04-25T10:15:13.784195+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:47.683035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"156'8","reported_seq":20,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.798774+0000","last_change":"2024-04-25T10:15:12.769778+0000","last_active":"2024-04-25T10:15:13.798774+0000","last_peered":"2024-04-25T10:15:13.798774+0000","last_clean":"2024-04-25T10:15:13.798774+0000","last_became_active":"2024-04-25T10:15:12.769118+0000","last_became_peered":"2024-04-25T10:15:12.769118+0000","last_unstale":"2024-04-25T10:15:13.798774+0000","last_undegraded":"2024-04-25T10:15:13.798774+0000","last_fullsized":"2024-04-25T10:15:13.798774+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:09.517126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784223+0000","last_change":"2024-04-25T10:15:13.769771+0000","last_active":"2024-04-25T10:15:13.784223+0000","last_peered":"2024-04-25T10:15:13.784223+0000","last_clean":"2024-04-25T10:15:13.784223+0000","last_became_active":"2024-04-25T10:15:13.769351+0000","last_became_peered":"2024-04-25T10:15:13.769351+0000","last_unstale":"2024-04-25T10:15:13.784223+0000","last_undegraded":"2024-04-25T10:15:13.784223+0000","last_fullsized":"2024-04-25T10:15:13.784223+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:42.844627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"156'1","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774685+0000","last_change":"2024-04-25T10:15:12.773689+0000","last_active":"2024-04-25T10:15:14.774685+0000","last_peered":"2024-04-25T10:15:14.774685+0000","last_clean":"2024-04-25T10:15:14.774685+0000","last_became_active":"2024-04-25T10:15:12.771536+0000","last_became_peered":"2024-04-25T10:15:12.771536+0000","last_unstale":"2024-04-25T10:15:14.774685+0000","last_undegraded":"2024-04-25T10:15:14.774685+0000","last_fullsized":"2024-04-25T10:15:14.774685+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:41.790368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774918+0000","last_change":"2024-04-25T10:15:13.771636+0000","last_active":"2024-04-25T10:15:14.774918+0000","last_peered":"2024-04-25T10:15:14.774918+0000","last_clean":"2024-04-25T10:15:14.774918+0000","last_became_active":"2024-04-25T10:15:13.771323+0000","last_became_peered":"2024-04-25T10:15:13.771323+0000","last_unstale":"2024-04-25T10:15:14.774918+0000","last_undegraded":"2024-04-25T10:15:14.774918+0000","last_fullsized":"2024-04-25T10:15:14.774918+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:21.557403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.797777+0000","last_change":"2024-04-25T10:15:12.772343+0000","last_active":"2024-04-25T10:15:13.797777+0000","last_peered":"2024-04-25T10:15:13.797777+0000","last_clean":"2024-04-25T10:15:13.797777+0000","last_became_active":"2024-04-25T10:15:12.771722+0000","last_became_peered":"2024-04-25T10:15:12.771722+0000","last_unstale":"2024-04-25T10:15:13.797777+0000","last_undegraded":"2024-04-25T10:15:13.797777+0000","last_fullsized":"2024-04-25T10:15:13.797777+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:11.556702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.778822+0000","last_change":"2024-04-25T10:15:13.776141+0000","last_active":"2024-04-25T10:15:13.778822+0000","last_peered":"2024-04-25T10:15:13.778822+0000","last_clean":"2024-04-25T10:15:13.778822+0000","last_became_active":"2024-04-25T10:15:13.775796+0000","last_became_peered":"2024-04-25T10:15:13.775796+0000","last_unstale":"2024-04-25T10:15:13.778822+0000","last_undegraded":"2024-04-25T10:15:13.778822+0000","last_fullsized":"2024-04-25T10:15:13.778822+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:03.851138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"156'1","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795901+0000","last_change":"2024-04-25T10:15:12.775505+0000","last_active":"2024-04-25T10:15:13.795901+0000","last_peered":"2024-04-25T10:15:13.795901+0000","last_clean":"2024-04-25T10:15:13.795901+0000","last_became_active":"2024-04-25T10:15:12.775176+0000","last_became_peered":"2024-04-25T10:15:12.775176+0000","last_unstale":"2024-04-25T10:15:13.795901+0000","last_undegraded":"2024-04-25T10:15:13.795901+0000","last_fullsized":"2024-04-25T10:15:13.795901+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:57.554026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.788433+0000","last_change":"2024-04-25T10:15:13.785874+0000","last_active":"2024-04-25T10:15:13.788433+0000","last_peered":"2024-04-25T10:15:13.788433+0000","last_clean":"2024-04-25T10:15:13.788433+0000","last_became_active":"2024-04-25T10:15:13.785522+0000","last_became_peered":"2024-04-25T10:15:13.785522+0000","last_unstale":"2024-04-25T10:15:13.788433+0000","last_undegraded":"2024-04-25T10:15:13.788433+0000","last_fullsized":"2024-04-25T10:15:13.788433+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:09.716962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.774716+0000","last_change":"2024-04-25T10:15:12.768337+0000","last_active":"2024-04-25T10:15:13.774716+0000","last_peered":"2024-04-25T10:15:13.774716+0000","last_clean":"2024-04-25T10:15:13.774716+0000","last_became_active":"2024-04-25T10:15:12.767913+0000","last_became_peered":"2024-04-25T10:15:12.767913+0000","last_unstale":"2024-04-25T10:15:13.774716+0000","last_undegraded":"2024-04-25T10:15:13.774716+0000","last_fullsized":"2024-04-25T10:15:13.774716+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:02.847209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774807+0000","last_change":"2024-04-25T10:15:13.780085+0000","last_active":"2024-04-25T10:15:14.774807+0000","last_peered":"2024-04-25T10:15:14.774807+0000","last_clean":"2024-04-25T10:15:14.774807+0000","last_became_active":"2024-04-25T10:15:13.779334+0000","last_became_peered":"2024-04-25T10:15:13.779334+0000","last_unstale":"2024-04-25T10:15:14.774807+0000","last_undegraded":"2024-04-25T10:15:14.774807+0000","last_fullsized":"2024-04-25T10:15:14.774807+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:28.553065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.796260+0000","last_change":"2024-04-25T10:15:12.777020+0000","last_active":"2024-04-25T10:15:13.796260+0000","last_peered":"2024-04-25T10:15:13.796260+0000","last_clean":"2024-04-25T10:15:13.796260+0000","last_became_active":"2024-04-25T10:15:12.776611+0000","last_became_peered":"2024-04-25T10:15:12.776611+0000","last_unstale":"2024-04-25T10:15:13.796260+0000","last_undegraded":"2024-04-25T10:15:13.796260+0000","last_fullsized":"2024-04-25T10:15:13.796260+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:21.126555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795177+0000","last_change":"2024-04-25T10:15:13.786921+0000","last_active":"2024-04-25T10:15:13.795177+0000","last_peered":"2024-04-25T10:15:13.795177+0000","last_clean":"2024-04-25T10:15:13.795177+0000","last_became_active":"2024-04-25T10:15:13.783503+0000","last_became_peered":"2024-04-25T10:15:13.783503+0000","last_unstale":"2024-04-25T10:15:13.795177+0000","last_undegraded":"2024-04-25T10:15:13.795177+0000","last_fullsized":"2024-04-25T10:15:13.795177+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:07.491218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.769608+0000","last_change":"2024-04-25T10:15:12.761779+0000","last_active":"2024-04-25T10:15:13.769608+0000","last_peered":"2024-04-25T10:15:13.769608+0000","last_clean":"2024-04-25T10:15:13.769608+0000","last_became_active":"2024-04-25T10:15:12.760878+0000","last_became_peered":"2024-04-25T10:15:12.760878+0000","last_unstale":"2024-04-25T10:15:13.769608+0000","last_undegraded":"2024-04-25T10:15:13.769608+0000","last_fullsized":"2024-04-25T10:15:13.769608+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:58.354472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.795895+0000","last_change":"2024-04-25T10:15:13.794092+0000","last_active":"2024-04-25T10:15:13.795895+0000","last_peered":"2024-04-25T10:15:13.795895+0000","last_clean":"2024-04-25T10:15:13.795895+0000","last_became_active":"2024-04-25T10:15:13.791933+0000","last_became_peered":"2024-04-25T10:15:13.791933+0000","last_unstale":"2024-04-25T10:15:13.795895+0000","last_undegraded":"2024-04-25T10:15:13.795895+0000","last_fullsized":"2024-04-25T10:15:13.795895+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:14.818954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.775942+0000","last_change":"2024-04-25T10:15:12.764264+0000","last_active":"2024-04-25T10:15:13.775942+0000","last_peered":"2024-04-25T10:15:13.775942+0000","last_clean":"2024-04-25T10:15:13.775942+0000","last_became_active":"2024-04-25T10:15:12.763996+0000","last_became_peered":"2024-04-25T10:15:12.763996+0000","last_unstale":"2024-04-25T10:15:13.775942+0000","last_undegraded":"2024-04-25T10:15:13.775942+0000","last_fullsized":"2024-04-25T10:15:13.775942+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:46.574502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.787414+0000","last_change":"2024-04-25T10:15:13.780395+0000","last_active":"2024-04-25T10:15:13.787414+0000","last_peered":"2024-04-25T10:15:13.787414+0000","last_clean":"2024-04-25T10:15:13.787414+0000","last_became_active":"2024-04-25T10:15:13.780075+0000","last_became_peered":"2024-04-25T10:15:13.780075+0000","last_unstale":"2024-04-25T10:15:13.787414+0000","last_undegraded":"2024-04-25T10:15:13.787414+0000","last_fullsized":"2024-04-25T10:15:13.787414+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:33.998118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.754187+0000","last_change":"2024-04-25T10:15:12.781594+0000","last_active":"2024-04-25T10:15:13.754187+0000","last_peered":"2024-04-25T10:15:13.754187+0000","last_clean":"2024-04-25T10:15:13.754187+0000","last_became_active":"2024-04-25T10:15:12.781033+0000","last_became_peered":"2024-04-25T10:15:12.781033+0000","last_unstale":"2024-04-25T10:15:13.754187+0000","last_undegraded":"2024-04-25T10:15:13.754187+0000","last_fullsized":"2024-04-25T10:15:13.754187+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:09.915844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.779036+0000","last_change":"2024-04-25T10:15:13.776857+0000","last_active":"2024-04-25T10:15:13.779036+0000","last_peered":"2024-04-25T10:15:13.779036+0000","last_clean":"2024-04-25T10:15:13.779036+0000","last_became_active":"2024-04-25T10:15:13.776552+0000","last_became_peered":"2024-04-25T10:15:13.776552+0000","last_unstale":"2024-04-25T10:15:13.779036+0000","last_undegraded":"2024-04-25T10:15:13.779036+0000","last_fullsized":"2024-04-25T10:15:13.779036+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:07.839844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"156'2","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.796149+0000","last_change":"2024-04-25T10:15:12.764289+0000","last_active":"2024-04-25T10:15:13.796149+0000","last_peered":"2024-04-25T10:15:13.796149+0000","last_clean":"2024-04-25T10:15:13.796149+0000","last_became_active":"2024-04-25T10:15:12.763956+0000","last_became_peered":"2024-04-25T10:15:12.763956+0000","last_unstale":"2024-04-25T10:15:13.796149+0000","last_undegraded":"2024-04-25T10:15:13.796149+0000","last_fullsized":"2024-04-25T10:15:13.796149+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:33.096261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.797580+0000","last_change":"2024-04-25T10:15:13.789318+0000","last_active":"2024-04-25T10:15:13.797580+0000","last_peered":"2024-04-25T10:15:13.797580+0000","last_clean":"2024-04-25T10:15:13.797580+0000","last_became_active":"2024-04-25T10:15:13.787734+0000","last_became_peered":"2024-04-25T10:15:13.787734+0000","last_unstale":"2024-04-25T10:15:13.797580+0000","last_undegraded":"2024-04-25T10:15:13.797580+0000","last_fullsized":"2024-04-25T10:15:13.797580+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:20.113079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774415+0000","last_change":"2024-04-25T10:15:12.774334+0000","last_active":"2024-04-25T10:15:14.774415+0000","last_peered":"2024-04-25T10:15:14.774415+0000","last_clean":"2024-04-25T10:15:14.774415+0000","last_became_active":"2024-04-25T10:15:12.773988+0000","last_became_peered":"2024-04-25T10:15:12.773988+0000","last_unstale":"2024-04-25T10:15:14.774415+0000","last_undegraded":"2024-04-25T10:15:14.774415+0000","last_fullsized":"2024-04-25T10:15:14.774415+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:03.058798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784988+0000","last_change":"2024-04-25T10:15:13.779576+0000","last_active":"2024-04-25T10:15:13.784988+0000","last_peered":"2024-04-25T10:15:13.784988+0000","last_clean":"2024-04-25T10:15:13.784988+0000","last_became_active":"2024-04-25T10:15:13.778670+0000","last_became_peered":"2024-04-25T10:15:13.778670+0000","last_unstale":"2024-04-25T10:15:13.784988+0000","last_undegraded":"2024-04-25T10:15:13.784988+0000","last_fullsized":"2024-04-25T10:15:13.784988+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:00.178751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774554+0000","last_change":"2024-04-25T10:15:12.770515+0000","last_active":"2024-04-25T10:15:14.774554+0000","last_peered":"2024-04-25T10:15:14.774554+0000","last_clean":"2024-04-25T10:15:14.774554+0000","last_became_active":"2024-04-25T10:15:12.770063+0000","last_became_peered":"2024-04-25T10:15:12.770063+0000","last_unstale":"2024-04-25T10:15:14.774554+0000","last_undegraded":"2024-04-25T10:15:14.774554+0000","last_fullsized":"2024-04-25T10:15:14.774554+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:00.146668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.797357+0000","last_change":"2024-04-25T10:15:13.787786+0000","last_active":"2024-04-25T10:15:13.797357+0000","last_peered":"2024-04-25T10:15:13.797357+0000","last_clean":"2024-04-25T10:15:13.797357+0000","last_became_active":"2024-04-25T10:15:13.787515+0000","last_became_peered":"2024-04-25T10:15:13.787515+0000","last_unstale":"2024-04-25T10:15:13.797357+0000","last_undegraded":"2024-04-25T10:15:13.797357+0000","last_fullsized":"2024-04-25T10:15:13.797357+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:11.287206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.777466+0000","last_change":"2024-04-25T10:15:12.781787+0000","last_active":"2024-04-25T10:15:13.777466+0000","last_peered":"2024-04-25T10:15:13.777466+0000","last_clean":"2024-04-25T10:15:13.777466+0000","last_became_active":"2024-04-25T10:15:12.781472+0000","last_became_peered":"2024-04-25T10:15:12.781472+0000","last_unstale":"2024-04-25T10:15:13.777466+0000","last_undegraded":"2024-04-25T10:15:13.777466+0000","last_fullsized":"2024-04-25T10:15:13.777466+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:35.513730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.776210+0000","last_change":"2024-04-25T10:15:13.789390+0000","last_active":"2024-04-25T10:15:14.776210+0000","last_peered":"2024-04-25T10:15:14.776210+0000","last_clean":"2024-04-25T10:15:14.776210+0000","last_became_active":"2024-04-25T10:15:13.789147+0000","last_became_peered":"2024-04-25T10:15:13.789147+0000","last_unstale":"2024-04-25T10:15:14.776210+0000","last_undegraded":"2024-04-25T10:15:14.776210+0000","last_fullsized":"2024-04-25T10:15:14.776210+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.799267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.775724+0000","last_change":"2024-04-25T10:15:12.779302+0000","last_active":"2024-04-25T10:15:14.775724+0000","last_peered":"2024-04-25T10:15:14.775724+0000","last_clean":"2024-04-25T10:15:14.775724+0000","last_became_active":"2024-04-25T10:15:12.778801+0000","last_became_peered":"2024-04-25T10:15:12.778801+0000","last_unstale":"2024-04-25T10:15:14.775724+0000","last_undegraded":"2024-04-25T10:15:14.775724+0000","last_fullsized":"2024-04-25T10:15:14.775724+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:03.816976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.784003+0000","last_change":"2024-04-25T10:15:13.775189+0000","last_active":"2024-04-25T10:15:13.784003+0000","last_peered":"2024-04-25T10:15:13.784003+0000","last_clean":"2024-04-25T10:15:13.784003+0000","last_became_active":"2024-04-25T10:15:13.774842+0000","last_became_peered":"2024-04-25T10:15:13.774842+0000","last_unstale":"2024-04-25T10:15:13.784003+0000","last_undegraded":"2024-04-25T10:15:13.784003+0000","last_fullsized":"2024-04-25T10:15:13.784003+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:28.192082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774084+0000","last_change":"2024-04-25T10:15:12.765151+0000","last_active":"2024-04-25T10:15:14.774084+0000","last_peered":"2024-04-25T10:15:14.774084+0000","last_clean":"2024-04-25T10:15:14.774084+0000","last_became_active":"2024-04-25T10:15:12.764613+0000","last_became_peered":"2024-04-25T10:15:12.764613+0000","last_unstale":"2024-04-25T10:15:14.774084+0000","last_undegraded":"2024-04-25T10:15:14.774084+0000","last_fullsized":"2024-04-25T10:15:14.774084+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:09.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774332+0000","last_change":"2024-04-25T10:15:13.779475+0000","last_active":"2024-04-25T10:15:14.774332+0000","last_peered":"2024-04-25T10:15:14.774332+0000","last_clean":"2024-04-25T10:15:14.774332+0000","last_became_active":"2024-04-25T10:15:13.778959+0000","last_became_peered":"2024-04-25T10:15:13.778959+0000","last_unstale":"2024-04-25T10:15:14.774332+0000","last_undegraded":"2024-04-25T10:15:14.774332+0000","last_fullsized":"2024-04-25T10:15:14.774332+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:09.927288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.769389+0000","last_change":"2024-04-25T10:15:12.765440+0000","last_active":"2024-04-25T10:15:13.769389+0000","last_peered":"2024-04-25T10:15:13.769389+0000","last_clean":"2024-04-25T10:15:13.769389+0000","last_became_active":"2024-04-25T10:15:12.764781+0000","last_became_peered":"2024-04-25T10:15:12.764781+0000","last_unstale":"2024-04-25T10:15:13.769389+0000","last_undegraded":"2024-04-25T10:15:13.769389+0000","last_fullsized":"2024-04-25T10:15:13.769389+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:41.840697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.798299+0000","last_change":"2024-04-25T10:15:13.795477+0000","last_active":"2024-04-25T10:15:13.798299+0000","last_peered":"2024-04-25T10:15:13.798299+0000","last_clean":"2024-04-25T10:15:13.798299+0000","last_became_active":"2024-04-25T10:15:13.795197+0000","last_became_peered":"2024-04-25T10:15:13.795197+0000","last_unstale":"2024-04-25T10:15:13.798299+0000","last_undegraded":"2024-04-25T10:15:13.798299+0000","last_fullsized":"2024-04-25T10:15:13.798299+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:38.084892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.761168+0000","last_change":"2024-04-25T10:15:12.778958+0000","last_active":"2024-04-25T10:15:13.761168+0000","last_peered":"2024-04-25T10:15:13.761168+0000","last_clean":"2024-04-25T10:15:13.761168+0000","last_became_active":"2024-04-25T10:15:12.778435+0000","last_became_peered":"2024-04-25T10:15:12.778435+0000","last_unstale":"2024-04-25T10:15:13.761168+0000","last_undegraded":"2024-04-25T10:15:13.761168+0000","last_fullsized":"2024-04-25T10:15:13.761168+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:03.058798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.764031+0000","last_change":"2024-04-25T10:15:13.760884+0000","last_active":"2024-04-25T10:15:13.764031+0000","last_peered":"2024-04-25T10:15:13.764031+0000","last_clean":"2024-04-25T10:15:13.764031+0000","last_became_active":"2024-04-25T10:15:13.760131+0000","last_became_peered":"2024-04-25T10:15:13.760131+0000","last_unstale":"2024-04-25T10:15:13.764031+0000","last_undegraded":"2024-04-25T10:15:13.764031+0000","last_fullsized":"2024-04-25T10:15:13.764031+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:12.731510+0000","last_clean_scrub_stamp":"2024-04-25T10:15:12.731510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:22.130484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:15:14.774305+0000","last_change":"2024-04-25T10:15:12.773685+0000","last_active":"2024-04-25T10:15:14.774305+0000","last_peered":"2024-04-25T10:15:14.774305+0000","last_clean":"2024-04-25T10:15:14.774305+0000","last_became_active":"2024-04-25T10:15:12.771759+0000","last_became_peered":"2024-04-25T10:15:12.771759+0000","last_unstale":"2024-04-25T10:15:14.774305+0000","last_undegraded":"2024-04-25T10:15:14.774305+0000","last_fullsized":"2024-04-25T10:15:14.774305+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:11.727580+0000","last_clean_scrub_stamp":"2024-04-25T10:15:11.727580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:16.296846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":425,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:15:13.754326+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:15:13.754326+0000","last_peered":"2024-04-25T10:15:13.754326+0000","last_clean":"2024-04-25T10:15:13.754326+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:15:13.754326+0000","last_undegraded":"2024-04-25T10:15:13.754326+0000","last_fullsized":"2024-04-25T10:15:13.754326+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575015,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73940,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297900,"statfs":{"total":96636764160,"available":96561049600,"internally_reserved":0,"allocated":2555904,"data_stored":1510356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575016,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51148,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320692,"statfs":{"total":96636764160,"available":96584388608,"internally_reserved":0,"allocated":1961984,"data_stored":919988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47000,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324840,"statfs":{"total":96636764160,"available":96588636160,"internally_reserved":0,"allocated":1970176,"data_stored":920112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575015,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77536,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294304,"statfs":{"total":96636764160,"available":96557367296,"internally_reserved":0,"allocated":1978368,"data_stored":920752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575015,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77532,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294308,"statfs":{"total":96636764160,"available":96557371392,"internally_reserved":0,"allocated":1982464,"data_stored":922204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575015,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51152,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320688,"statfs":{"total":96636764160,"available":96584384512,"internally_reserved":0,"allocated":1961984,"data_stored":919988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47000,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324840,"statfs":{"total":96636764160,"available":96588636160,"internally_reserved":0,"allocated":1974272,"data_stored":921600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575016,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73944,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297896,"statfs":{"total":96636764160,"available":96561045504,"internally_reserved":0,"allocated":2564096,"data_stored":1510460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:17.153 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:15:17.154 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.326+0000 7f5417f85640 1 Processor -- start 2024-04-25T10:15:17.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.326+0000 7f5417f85640 1 -- start start 2024-04-25T10:15:17.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.326+0000 7f5417f85640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5410143560 0x7f5410140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.326+0000 7f5417f85640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f541015d9f0 0x7f541015de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.326+0000 7f5417f85640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54100cf280 con 0x7f5410142920 2024-04-25T10:15:17.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.326+0000 7f5417f85640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541006d440 con 0x7f541015d9f0 2024-04-25T10:15:17.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.326+0000 7f5417f85640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541006d720 con 0x7f5410143560 2024-04-25T10:15:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5415d21640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f5410142920 0x7f54100ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:37608/0 (socket says 172.21.15.133:37608) 2024-04-25T10:15:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5415d21640 1 -- 172.21.15.133:0/2039954450 learned_addr learned my addr 172.21.15.133:0/2039954450 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5415520640 1 --2- 172.21.15.133:0/2039954450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5410143560 0x7f5410140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5416522640 1 --2- 172.21.15.133:0/2039954450 >> v2:172.21.15.162:3300/0 conn(0x7f541015d9f0 0x7f541015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5414d1f640 1 -- 172.21.15.133:0/2039954450 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572602798 0 0) 0x7f54100cf280 con 0x7f5410142920 2024-04-25T10:15:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5414d1f640 1 -- 172.21.15.133:0/2039954450 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53fc003530 con 0x7f5410142920 2024-04-25T10:15:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5415520640 1 -- 172.21.15.133:0/2039954450 >> v2:172.21.15.162:3300/0 conn(0x7f541015d9f0 msgr2=0x7f541015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5415520640 1 --2- 172.21.15.133:0/2039954450 >> v2:172.21.15.162:3300/0 conn(0x7f541015d9f0 0x7f541015de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5415520640 1 -- 172.21.15.133:0/2039954450 >> v1:172.21.15.162:6789/0 conn(0x7f5410142920 legacy=0x7f54100ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5415520640 1 -- 172.21.15.133:0/2039954450 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541006da00 con 0x7f5410143560 2024-04-25T10:15:17.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5415520640 1 --2- 172.21.15.133:0/2039954450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5410143560 0x7f5410140a40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f540000a960 tx=0x7f540001d7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f665eec7662bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5414d1f640 1 -- 172.21.15.133:0/2039954450 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54000b3070 con 0x7f5410143560 2024-04-25T10:15:17.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5414d1f640 1 -- 172.21.15.133:0/2039954450 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54000a6740 con 0x7f5410143560 2024-04-25T10:15:17.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.330+0000 7f5414d1f640 1 -- 172.21.15.133:0/2039954450 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540000b040 con 0x7f5410143560 2024-04-25T10:15:17.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 -- 172.21.15.133:0/2039954450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5410143560 msgr2=0x7f5410140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 --2- 172.21.15.133:0/2039954450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5410143560 0x7f5410140a40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f540000a960 tx=0x7f540001d7e0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 -- 172.21.15.133:0/2039954450 shutdown_connections 2024-04-25T10:15:17.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 --2- 172.21.15.133:0/2039954450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5410143560 0x7f5410140a40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 --2- 172.21.15.133:0/2039954450 >> v2:172.21.15.162:3300/0 conn(0x7f541015d9f0 0x7f541015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 -- 172.21.15.133:0/2039954450 >> 172.21.15.133:0/2039954450 conn(0x7f5410066f70 msgr2=0x7f541006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 -- 172.21.15.133:0/2039954450 shutdown_connections 2024-04-25T10:15:17.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 -- 172.21.15.133:0/2039954450 wait complete. 2024-04-25T10:15:17.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 Processor -- start 2024-04-25T10:15:17.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 -- start start 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5410142920 0x7f54101f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5410143560 0x7f541013cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f541015d9f0 0x7f54100b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541006eac0 con 0x7f5410142920 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541006e940 con 0x7f541015d9f0 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5410166d90 con 0x7f5410143560 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5415d21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5410142920 0x7f54101f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5415d21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5410142920 0x7f54101f6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54072/0 (socket says 172.21.15.133:54072) 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5415d21640 1 -- 172.21.15.133:0/405070897 learned_addr learned my addr 172.21.15.133:0/405070897 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5415520640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5410143560 0x7f541013cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5416522640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f541015d9f0 0x7f54100b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5415d21640 1 -- 172.21.15.133:0/405070897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5410143560 msgr2=0x7f541013cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5415d21640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5410143560 0x7f541013cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5415d21640 1 -- 172.21.15.133:0/405070897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f541015d9f0 msgr2=0x7f54100b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5415d21640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f541015d9f0 0x7f54100b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5415d21640 1 -- 172.21.15.133:0/405070897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541013d500 con 0x7f5410142920 2024-04-25T10:15:17.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5416522640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f541015d9f0 0x7f54100b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:17.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5415d21640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5410142920 0x7f54101f6400 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c450 tx=0x7f540c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f54067fc640 1 -- 172.21.15.133:0/405070897 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c017070 con 0x7f5410142920 2024-04-25T10:15:17.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f54067fc640 1 -- 172.21.15.133:0/405070897 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c005cf0 con 0x7f5410142920 2024-04-25T10:15:17.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 -- 172.21.15.133:0/405070897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54100ae030 con 0x7f5410142920 2024-04-25T10:15:17.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f54067fc640 1 -- 172.21.15.133:0/405070897 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c007270 con 0x7f5410142920 2024-04-25T10:15:17.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.334+0000 7f5417f85640 1 -- 172.21.15.133:0/405070897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541015bc60 con 0x7f5410142920 2024-04-25T10:15:17.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.338+0000 7f5417f85640 1 -- 172.21.15.133:0/405070897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5410068e80 con 0x7f5410142920 2024-04-25T10:15:17.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.338+0000 7f54067fc640 1 -- 172.21.15.133:0/405070897 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f540c01a070 con 0x7f5410142920 2024-04-25T10:15:17.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.338+0000 7f54067fc640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53f4066f60 0x7f53f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.338+0000 7f54067fc640 1 -- 172.21.15.133:0/405070897 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f540c084770 con 0x7f5410142920 2024-04-25T10:15:17.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.338+0000 7f5415520640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53f4066f60 0x7f53f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.338+0000 7f5415520640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53f4066f60 0x7f53f4069420 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f54000b0000 tx=0x7f540000c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.342+0000 7f54067fc640 1 -- 172.21.15.133:0/405070897 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f540c055110 con 0x7f5410142920 2024-04-25T10:15:17.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.566+0000 7f5417f85640 1 -- 172.21.15.133:0/405070897 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f541006aea0 con 0x7f5410142920 2024-04-25T10:15:17.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f54067fc640 1 -- 172.21.15.133:0/405070897 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+147 (secure 0 0 0) 0x7f540c0591b0 con 0x7f5410142920 2024-04-25T10:15:17.569 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:15:17.569 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:15:17.569 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:15:17.569 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:15:17.569 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:15:17.569 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:15:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 -- 172.21.15.133:0/405070897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53f4066f60 msgr2=0x7f53f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53f4066f60 0x7f53f4069420 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f54000b0000 tx=0x7f540000c040 comp rx=0 tx=0).stop 2024-04-25T10:15:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 -- 172.21.15.133:0/405070897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5410142920 msgr2=0x7f54101f6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5410142920 0x7f54101f6400 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c450 tx=0x7f540c00c910 comp rx=0 tx=0).stop 2024-04-25T10:15:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 -- 172.21.15.133:0/405070897 shutdown_connections 2024-04-25T10:15:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f541015d9f0 0x7f54100b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5410143560 0x7f541013cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f53f4066f60 0x7f53f4069420 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 --2- 172.21.15.133:0/405070897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5410142920 0x7f54101f6400 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 -- 172.21.15.133:0/405070897 >> 172.21.15.133:0/405070897 conn(0x7f5410066f70 msgr2=0x7f541013b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 -- 172.21.15.133:0/405070897 shutdown_connections 2024-04-25T10:15:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.570+0000 7f5417f85640 1 -- 172.21.15.133:0/405070897 wait complete. 2024-04-25T10:15:17.587 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:17.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.694+0000 7f4f3d0b1640 1 Processor -- start 2024-04-25T10:15:17.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.694+0000 7f4f3d0b1640 1 -- start start 2024-04-25T10:15:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4f38158110 0x7f4f380b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f381584f0 0x7f4f3815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4f3806e460 0x7f4f3806e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f3806edf0 con 0x7f4f38158110 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f380cbe60 con 0x7f4f3806e460 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380b19b0 con 0x7f4f381584f0 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f36d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4f38158110 0x7f4f380b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f36d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4f38158110 0x7f4f380b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54086/0 (socket says 172.21.15.133:54086) 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f36d9d640 1 -- 172.21.15.133:0/4006818014 learned_addr learned my addr 172.21.15.133:0/4006818014 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3759e640 1 --2- 172.21.15.133:0/4006818014 >> v2:172.21.15.162:3300/0 conn(0x7f4f3806e460 0x7f4f3806e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f36d9d640 1 -- 172.21.15.133:0/4006818014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f381584f0 msgr2=0x7f4f3815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f2e59c640 1 --2- 172.21.15.133:0/4006818014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f381584f0 0x7f4f3815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f36d9d640 1 --2- 172.21.15.133:0/4006818014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f381584f0 0x7f4f3815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f36d9d640 1 -- 172.21.15.133:0/4006818014 >> v2:172.21.15.162:3300/0 conn(0x7f4f3806e460 msgr2=0x7f4f3806e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f36d9d640 1 --2- 172.21.15.133:0/4006818014 >> v2:172.21.15.162:3300/0 conn(0x7f4f3806e460 0x7f4f3806e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f36d9d640 1 -- 172.21.15.133:0/4006818014 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3815b360 con 0x7f4f38158110 2024-04-25T10:15:17.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3759e640 1 --2- 172.21.15.133:0/4006818014 >> v2:172.21.15.162:3300/0 conn(0x7f4f3806e460 0x7f4f3806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:17.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f2e59c640 1 --2- 172.21.15.133:0/4006818014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f381584f0 0x7f4f3815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f36d9d640 1 --2- 172.21.15.133:0/4006818014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f38158110 0x7f4f380b1f00 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f4f3815f3a0 tx=0x7f4f2001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c900bd117b00e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3659c640 1 -- 172.21.15.133:0/4006818014 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f200aa610 con 0x7f4f38158110 2024-04-25T10:15:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3659c640 1 -- 172.21.15.133:0/4006818014 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f200af070 con 0x7f4f38158110 2024-04-25T10:15:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3659c640 1 -- 172.21.15.133:0/4006818014 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f200aa9d0 con 0x7f4f38158110 2024-04-25T10:15:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/4006818014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f38158110 msgr2=0x7f4f380b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 --2- 172.21.15.133:0/4006818014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f38158110 0x7f4f380b1f00 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f4f3815f3a0 tx=0x7f4f2001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/4006818014 shutdown_connections 2024-04-25T10:15:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 --2- 172.21.15.133:0/4006818014 >> v2:172.21.15.162:3300/0 conn(0x7f4f3806e460 0x7f4f3806e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 --2- 172.21.15.133:0/4006818014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f381584f0 0x7f4f3815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 --2- 172.21.15.133:0/4006818014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f38158110 0x7f4f380b1f00 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/4006818014 >> 172.21.15.133:0/4006818014 conn(0x7f4f38067000 msgr2=0x7f4f3806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/4006818014 shutdown_connections 2024-04-25T10:15:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.698+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/4006818014 wait complete. 2024-04-25T10:15:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3d0b1640 1 Processor -- start 2024-04-25T10:15:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3d0b1640 1 -- start start 2024-04-25T10:15:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3d0b1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f3806e460 0x7f4f3814eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3d0b1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38158110 0x7f4f38155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3d0b1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f381584f0 0x7f4f38156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3d0b1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380cd7f0 con 0x7f4f381584f0 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3d0b1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380cd670 con 0x7f4f38158110 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3d0b1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3815eb10 con 0x7f4f3806e460 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f381584f0 0x7f4f38156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f381584f0 0x7f4f38156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54100/0 (socket says 172.21.15.133:54100) 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3759e640 1 -- 172.21.15.133:0/2434547701 learned_addr learned my addr 172.21.15.133:0/2434547701 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f2e59c640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38158110 0x7f4f38155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f36d9d640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f3806e460 0x7f4f3814eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3759e640 1 -- 172.21.15.133:0/2434547701 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f3806e460 msgr2=0x7f4f3814eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3759e640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f3806e460 0x7f4f3814eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3759e640 1 -- 172.21.15.133:0/2434547701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38158110 msgr2=0x7f4f38155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3759e640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38158110 0x7f4f38155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3759e640 1 -- 172.21.15.133:0/2434547701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38151b20 con 0x7f4f381584f0 2024-04-25T10:15:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f2e59c640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38158110 0x7f4f38155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3759e640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f381584f0 0x7f4f38156110 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000cef0 tx=0x7f4f3000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f2ffff640 1 -- 172.21.15.133:0/2434547701 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3001a070 con 0x7f4f381584f0 2024-04-25T10:15:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f2ffff640 1 -- 172.21.15.133:0/2434547701 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3000d840 con 0x7f4f381584f0 2024-04-25T10:15:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/2434547701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f38151e00 con 0x7f4f381584f0 2024-04-25T10:15:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f2ffff640 1 -- 172.21.15.133:0/2434547701 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000dc30 con 0x7f4f381584f0 2024-04-25T10:15:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/2434547701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3815c020 con 0x7f4f381584f0 2024-04-25T10:15:17.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/2434547701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3806bc20 con 0x7f4f381584f0 2024-04-25T10:15:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.702+0000 7f4f2ffff640 1 -- 172.21.15.133:0/2434547701 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f3001d070 con 0x7f4f381584f0 2024-04-25T10:15:17.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.706+0000 7f4f2ffff640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f1c066f60 0x7f4f1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.706+0000 7f4f36d9d640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f1c066f60 0x7f4f1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.706+0000 7f4f2ffff640 1 -- 172.21.15.133:0/2434547701 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4f300859f0 con 0x7f4f381584f0 2024-04-25T10:15:17.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.706+0000 7f4f36d9d640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f1c066f60 0x7f4f1c069420 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4f200ae000 tx=0x7f4f20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.710+0000 7f4f2ffff640 1 -- 172.21.15.133:0/2434547701 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f30056870 con 0x7f4f381584f0 2024-04-25T10:15:17.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.886+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/2434547701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f38068ef0 con 0x7f4f381584f0 2024-04-25T10:15:17.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.886+0000 7f4f2ffff640 1 -- 172.21.15.133:0/2434547701 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4959 (secure 0 0 0) 0x7f4f30085070 con 0x7f4f381584f0 2024-04-25T10:15:17.887 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:17.887 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3454110484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3454110484},{"type":"v1","addr":"172.21.15.162:6835","nonce":3454110484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3345416592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3345416592},{"type":"v1","addr":"172.21.15.133:6835","nonce":3345416592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2924224565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2924224565},{"type":"v1","addr":"172.21.15.133:6837","nonce":2924224565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:13.097799+0000","modified":"2024-04-25T10:15:15.128366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10554":{"gid":10554,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/3763981385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3763981385},{"type":"v1","addr":"172.21.15.162:6833","nonce":3763981385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:15:17.888 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 99 2024-04-25T10:15:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/2434547701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f1c066f60 msgr2=0x7f4f1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f1c066f60 0x7f4f1c069420 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4f200ae000 tx=0x7f4f20008040 comp rx=0 tx=0).stop 2024-04-25T10:15:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/2434547701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f381584f0 msgr2=0x7f4f38156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f381584f0 0x7f4f38156110 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000cef0 tx=0x7f4f3000a710 comp rx=0 tx=0).stop 2024-04-25T10:15:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/2434547701 shutdown_connections 2024-04-25T10:15:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f1c066f60 0x7f4f1c069420 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f381584f0 0x7f4f38156110 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f38158110 0x7f4f38155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 --2- 172.21.15.133:0/2434547701 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f3806e460 0x7f4f3814eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/2434547701 >> 172.21.15.133:0/2434547701 conn(0x7f4f38067000 msgr2=0x7f4f380cada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/2434547701 shutdown_connections 2024-04-25T10:15:17.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:17.890+0000 7f4f3d0b1640 1 -- 172.21.15.133:0/2434547701 wait complete. 2024-04-25T10:15:18.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:15:13.097799+0000', 'modified': '2024-04-25T10:15:15.128366+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10554}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10554': {'gid': 10554, 'name': 'b', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6833/3763981385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6832', 'nonce': 3763981385}, {'type': 'v1', 'addr': '172.21.15.162:6833', 'nonce': 3763981385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:15:18.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:15:18.307 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:15:18.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7feff5f8a640 1 Processor -- start 2024-04-25T10:15:18.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7feff5f8a640 1 -- start start 2024-04-25T10:15:18.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7feff5f8a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7feff0145170 0x7feff006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7feff5f8a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff00ce2e0 con 0x7feff006e4d0 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7feff5f8a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff006d4b0 con 0x7feff015ced0 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7feff5f8a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff006db00 con 0x7feff0145170 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefe7fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7feff006e4d0 0x7feff00cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58612/0 (socket says 172.21.15.133:58612) 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefe7fff640 1 -- 172.21.15.133:0/2635325205 learned_addr learned my addr 172.21.15.133:0/2635325205 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefef7fe640 1 --2- 172.21.15.133:0/2635325205 >> v2:172.21.15.162:3300/0 conn(0x7feff0145170 0x7feff006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1833930435 0 0) 0x7feff00ce2e0 con 0x7feff006e4d0 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefdc003530 con 0x7feff006e4d0 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1962470532 0 0) 0x7fefdc003530 con 0x7feff006e4d0 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 >> v2:172.21.15.162:3300/0 conn(0x7feff0145170 msgr2=0x7feff006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 --2- 172.21.15.133:0/2635325205 >> v2:172.21.15.162:3300/0 conn(0x7feff0145170 0x7feff006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 >> v1:172.21.15.162:6789/0 conn(0x7feff015ced0 legacy=0x7feff00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff006d790 con 0x7feff006e4d0 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fefd8001680 con 0x7feff006e4d0 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 >> v1:172.21.15.133:6789/0 conn(0x7feff006e4d0 legacy=0x7feff00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 --2- 172.21.15.133:0/2635325205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefdc0039c0 0x7fefdc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 --2- 172.21.15.133:0/2635325205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 0x7fefdc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 --2- 172.21.15.133:0/2635325205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc0051d0 0x7fefdc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff006db00 con 0x7fefdc004810 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc00c070 con 0x7fefdc0051d0 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc00c270 con 0x7fefdc0039c0 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefe7fff640 1 --2- 172.21.15.133:0/2635325205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 0x7fefdc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefef7fe640 1 --2- 172.21.15.133:0/2635325205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefdc0039c0 0x7fefdc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeffff640 1 --2- 172.21.15.133:0/2635325205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc0051d0 0x7fefdc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefe7fff640 1 -- 172.21.15.133:0/2635325205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefdc0039c0 msgr2=0x7fefdc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefe7fff640 1 --2- 172.21.15.133:0/2635325205 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fefdc0039c0 0x7fefdc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefe7fff640 1 -- 172.21.15.133:0/2635325205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc0051d0 msgr2=0x7fefdc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefe7fff640 1 --2- 172.21.15.133:0/2635325205 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fefdc0051d0 0x7fefdc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefeffff640 1 -- 172.21.15.133:0/2635325205 reap_dead start 2024-04-25T10:15:18.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.438+0000 7fefe7fff640 1 -- 172.21.15.133:0/2635325205 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc003530 con 0x7fefdc004810 2024-04-25T10:15:18.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.442+0000 7fefe7fff640 1 --2- 172.21.15.133:0/2635325205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 0x7fefdc004c90 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7feff006e930 tx=0x7fefd801f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd91fe7ba170258d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.442+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8007d40 con 0x7fefdc004810 2024-04-25T10:15:18.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.442+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd8005b00 con 0x7fefdc004810 2024-04-25T10:15:18.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.442+0000 7fefeeffd640 1 -- 172.21.15.133:0/2635325205 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8007070 con 0x7fefdc004810 2024-04-25T10:15:18.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.442+0000 7feff5f8a640 1 -- 172.21.15.133:0/2635325205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 msgr2=0x7fefdc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.442+0000 7feff5f8a640 1 --2- 172.21.15.133:0/2635325205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 0x7fefdc004c90 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7feff006e930 tx=0x7fefd801f3d0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.442+0000 7feff5f8a640 1 -- 172.21.15.133:0/2635325205 shutdown_connections 2024-04-25T10:15:18.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.442+0000 7feff5f8a640 1 --2- 172.21.15.133:0/2635325205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 0x7fefdc004c90 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.442+0000 7feff5f8a640 1 -- 172.21.15.133:0/2635325205 >> 172.21.15.133:0/2635325205 conn(0x7feff0066f70 msgr2=0x7feff006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.442+0000 7feff5f8a640 1 -- 172.21.15.133:0/2635325205 shutdown_connections 2024-04-25T10:15:18.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.442+0000 7feff5f8a640 1 -- 172.21.15.133:0/2635325205 wait complete. 2024-04-25T10:15:18.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7feff5f8a640 1 Processor -- start 2024-04-25T10:15:18.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7feff5f8a640 1 -- start start 2024-04-25T10:15:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7feff5f8a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 0x7feff013db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7feff5f8a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff0142290 0x7feff0139140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7feff5f8a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feff00adce0 0x7feff00ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7feff5f8a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff006df90 con 0x7fefdc004810 2024-04-25T10:15:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7feff5f8a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff0145550 con 0x7feff0142290 2024-04-25T10:15:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7feff5f8a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff006e110 con 0x7feff00adce0 2024-04-25T10:15:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 0x7feff013db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 0x7feff013db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54130/0 (socket says 172.21.15.133:54130) 2024-04-25T10:15:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefef7fe640 1 -- 172.21.15.133:0/217983230 learned_addr learned my addr 172.21.15.133:0/217983230 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefe7fff640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff0142290 0x7feff0139140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefeffff640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feff00adce0 0x7feff00ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefef7fe640 1 -- 172.21.15.133:0/217983230 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feff00adce0 msgr2=0x7feff00ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefef7fe640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feff00adce0 0x7feff00ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefef7fe640 1 -- 172.21.15.133:0/217983230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff0142290 msgr2=0x7feff0139140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefef7fe640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff0142290 0x7feff0139140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefef7fe640 1 -- 172.21.15.133:0/217983230 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff0142a00 con 0x7fefdc004810 2024-04-25T10:15:18.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefef7fe640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 0x7feff013db80 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fefe0011690 tx=0x7fefe0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefecff9640 1 -- 172.21.15.133:0/217983230 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe000f9c0 con 0x7fefdc004810 2024-04-25T10:15:18.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefecff9640 1 -- 172.21.15.133:0/217983230 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe000fb60 con 0x7fefdc004810 2024-04-25T10:15:18.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7feff5f8a640 1 -- 172.21.15.133:0/217983230 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff0142ce0 con 0x7fefdc004810 2024-04-25T10:15:18.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefecff9640 1 -- 172.21.15.133:0/217983230 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0015460 con 0x7fefdc004810 2024-04-25T10:15:18.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7feff5f8a640 1 -- 172.21.15.133:0/217983230 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff013aa40 con 0x7fefdc004810 2024-04-25T10:15:18.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.446+0000 7fefecff9640 1 -- 172.21.15.133:0/217983230 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fefe0015600 con 0x7fefdc004810 2024-04-25T10:15:18.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefecff9640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fefd0066f90 0x7fefd0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefecff9640 1 -- 172.21.15.133:0/217983230 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fefe0085c80 con 0x7fefdc004810 2024-04-25T10:15:18.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefe7fff640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fefd0066f90 0x7fefd0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefe7fff640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fefd0066f90 0x7fefd0069450 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fefd80af000 tx=0x7fefd80aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 Processor -- start 2024-04-25T10:15:18.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 -- start start 2024-04-25T10:15:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b48f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 0x560a99afee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99aff720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a999e25f0 con 0x560a99b48b70 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a99949da0 con 0x560a99b49490 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a998d2150 con 0x560a99aff340 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b48f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b48f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54134/0 (socket says 172.21.15.133:54134) 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefef7fe640 1 -- 172.21.15.133:0/2460607612 learned_addr learned my addr 172.21.15.133:0/2460607612 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefef7fe640 1 -- 172.21.15.133:0/2460607612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 msgr2=0x560a99aff720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefef7fe640 1 --2- 172.21.15.133:0/2460607612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99aff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefef7fe640 1 -- 172.21.15.133:0/2460607612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 msgr2=0x560a99afee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefef7fe640 1 --2- 172.21.15.133:0/2460607612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 0x560a99afee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefef7fe640 1 -- 172.21.15.133:0/2460607612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a998ebca0 con 0x560a99b48b70 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefef7fe640 1 --2- 172.21.15.133:0/2460607612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b48f50 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fefe0052770 tx=0x7fefe0056360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e537a3463aa5c17d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefe57fa640 1 -- 172.21.15.133:0/2460607612 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe005ea70 con 0x560a99b48b70 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefe57fa640 1 -- 172.21.15.133:0/2460607612 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0054660 con 0x560a99b48b70 2024-04-25T10:15:18.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7fefe57fa640 1 -- 172.21.15.133:0/2460607612 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe00529d0 con 0x560a99b48b70 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 -- 172.21.15.133:0/2460607612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 msgr2=0x560a99b48f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 --2- 172.21.15.133:0/2460607612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b48f50 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fefe0052770 tx=0x7fefe0056360 comp rx=0 tx=0).stop 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 -- 172.21.15.133:0/2460607612 shutdown_connections 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 --2- 172.21.15.133:0/2460607612 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99aff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 --2- 172.21.15.133:0/2460607612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 0x560a99afee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 --2- 172.21.15.133:0/2460607612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b48f50 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 -- 172.21.15.133:0/2460607612 >> 172.21.15.133:0/2460607612 conn(0x560a99b012a0 msgr2=0x560a99af92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 -- 172.21.15.133:0/2460607612 shutdown_connections 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 -- 172.21.15.133:0/2460607612 wait complete. 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.450+0000 7feff90f7000 1 -- start start 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 1 Processor -- start 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99b503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b508e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 0x560a99b55260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a99904990 con 0x560a99b48b70 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a99945e40 con 0x560a99b49490 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a998d5900 con 0x560a99aff340 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 10 client.0 _ll_register_callbacks cb 0x560a999fd1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefe7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefe7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54144/0 (socket says 172.21.15.133:54144) 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefe7fff640 1 -- 172.21.15.133:0/4138586202 learned_addr learned my addr 172.21.15.133:0/4138586202 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefef7fe640 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99b503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefeffff640 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 0x560a99b55260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefe7fff640 1 -- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 msgr2=0x560a99b503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefe7fff640 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99b503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefe7fff640 1 -- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 msgr2=0x560a99b55260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefe7fff640 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 0x560a99b55260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefe7fff640 1 -- 172.21.15.133:0/4138586202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a99902630 con 0x560a99b48b70 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefe7fff640 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b508e0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fefd801f200 tx=0x7fefdc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dab3347cee5eea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefce7fc640 1 -- 172.21.15.133:0/4138586202 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd80cb060 con 0x560a99b48b70 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 1 -- 172.21.15.133:0/4138586202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefe0054660 con 0x560a99b48b70 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefce7fc640 1 -- 172.21.15.133:0/4138586202 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fefd8007070 con 0x560a99b48b70 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefce7fc640 1 -- 172.21.15.133:0/4138586202 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefd80d1050 con 0x560a99b48b70 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 10 client.10692 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 1 -- 172.21.15.133:0/4138586202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x560a998d23a0 con 0x560a99b48b70 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefce7fc640 1 -- 172.21.15.133:0/4138586202 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0054660 con 0x560a99b48b70 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefce7fc640 1 -- 172.21.15.133:0/4138586202 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fefd80d4020 con 0x560a99b48b70 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 10 client.10692 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 10 client.10692 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:15:18.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 20 client.10692 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:18.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x560a99b57aa0 0x560a99b59e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 4 client.10692 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:18.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7feff90f7000 1 -- 172.21.15.133:0/4138586202 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a998e5a50 con 0x560a99b57aa0 2024-04-25T10:15:18.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefef7fe640 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x560a99b57aa0 0x560a99b59e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefce7fc640 10 client.10692 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:18.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.454+0000 7fefef7fe640 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x560a99b57aa0 0x560a99b59e90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f58dfb7ba2beb236 server_cookie=6ca26c08043d09fd in_seq=0 out_seq=0 2024-04-25T10:15:18.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7fefce7fc640 1 -- 172.21.15.133:0/4138586202 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a998e5a50 con 0x560a99b57aa0 2024-04-25T10:15:18.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7fefce7fc640 10 client.10692 handle_command_reply: tid=0 2024-04-25T10:15:18.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 1 client.10692 shutdown 2024-04-25T10:15:18.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 1 -- 172.21.15.133:0/4138586202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 msgr2=0x560a99b508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b508e0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7fefd801f200 tx=0x7fefdc005840 comp rx=0 tx=0).stop 2024-04-25T10:15:18.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 1 -- 172.21.15.133:0/4138586202 shutdown_connections 2024-04-25T10:15:18.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 0x560a99b55260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99b508e0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x560a99b57aa0 0x560a99b59e90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 1 --2- 172.21.15.133:0/4138586202 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99b503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7fefce7fc640 0 client.10692 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:18.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 1 -- 172.21.15.133:0/4138586202 >> 172.21.15.133:0/4138586202 conn(0x560a99b012a0 msgr2=0x560a99b006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 1 -- 172.21.15.133:0/4138586202 shutdown_connections 2024-04-25T10:15:18.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 1 -- 172.21.15.133:0/4138586202 wait complete. 2024-04-25T10:15:18.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.458+0000 7feff90f7000 20 client.10692 trim_cache size 0 max 16384 2024-04-25T10:15:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.470+0000 7feff90f7000 1 Processor -- start 2024-04-25T10:15:18.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.470+0000 7feff90f7000 1 -- start start 2024-04-25T10:15:18.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99ba3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99ba4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 0x560a99ba4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00b13b0 con 0x560a99b48b70 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00b07b0 con 0x560a99b49490 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff00b0db0 con 0x560a99aff340 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99ba4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99ba4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54154/0 (socket says 172.21.15.133:54154) 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe7fff640 1 -- 172.21.15.133:0/2754084057 learned_addr learned my addr 172.21.15.133:0/2754084057 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe7fff640 1 -- 172.21.15.133:0/2754084057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 msgr2=0x560a99ba3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe7fff640 1 --2- 172.21.15.133:0/2754084057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99ba3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe7fff640 1 -- 172.21.15.133:0/2754084057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 msgr2=0x560a99ba4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe7fff640 1 --2- 172.21.15.133:0/2754084057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 0x560a99ba4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe7fff640 1 -- 172.21.15.133:0/2754084057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a99aff170 con 0x560a99b48b70 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe7fff640 1 --2- 172.21.15.133:0/2754084057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99ba4250 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fefd80b0040 tx=0x7fefd80c4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc75b0ec3391fef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe4ff9640 1 -- 172.21.15.133:0/2754084057 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8007070 con 0x560a99b48b70 2024-04-25T10:15:18.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe4ff9640 1 -- 172.21.15.133:0/2754084057 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd80d9070 con 0x560a99b48b70 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe4ff9640 1 -- 172.21.15.133:0/2754084057 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8007210 con 0x560a99b48b70 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- 172.21.15.133:0/2754084057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 msgr2=0x560a99ba4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 --2- 172.21.15.133:0/2754084057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99ba4250 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fefd80b0040 tx=0x7fefd80c4000 comp rx=0 tx=0).stop 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- 172.21.15.133:0/2754084057 shutdown_connections 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 --2- 172.21.15.133:0/2754084057 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b49490 0x560a99ba4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 --2- 172.21.15.133:0/2754084057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b48b70 0x560a99ba4250 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 --2- 172.21.15.133:0/2754084057 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99ba3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- 172.21.15.133:0/2754084057 >> 172.21.15.133:0/2754084057 conn(0x560a99b012a0 msgr2=0x560a99af8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- 172.21.15.133:0/2754084057 shutdown_connections 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- 172.21.15.133:0/2754084057 wait complete. 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- start start 2024-04-25T10:15:18.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 Processor -- start 2024-04-25T10:15:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99b8c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b48b70 0x560a99b5b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b49490 0x560a99b5bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9994e3d0 con 0x560a99b49490 2024-04-25T10:15:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a99a05d50 con 0x560a99b48b70 2024-04-25T10:15:18.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a998e4980 con 0x560a99aff340 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7feff90f7000 10 client.0 _ll_register_callbacks cb 0x7fefe0052600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefeffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b49490 0x560a99b5bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefeffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b49490 0x560a99b5bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54162/0 (socket says 172.21.15.133:54162) 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefeffff640 1 -- 172.21.15.133:0/1997225747 learned_addr learned my addr 172.21.15.133:0/1997225747 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefef7fe640 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99b8c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefe7fff640 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b48b70 0x560a99b5b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefeffff640 1 -- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 msgr2=0x560a99b8c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefeffff640 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99b8c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefeffff640 1 -- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b48b70 msgr2=0x560a99b5b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefeffff640 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b48b70 0x560a99b5b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefeffff640 1 -- 172.21.15.133:0/1997225747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a999cb600 con 0x560a99b49490 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefeffff640 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b49490 0x560a99b5bba0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7feff015d2b0 tx=0x7fefe80019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa117b463b268886 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.474+0000 7fefce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:18.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7fefce7fc640 1 -- 172.21.15.133:0/1997225747 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe801a060 con 0x560a99b49490 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7fefce7fc640 1 -- 172.21.15.133:0/1997225747 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fefe8015410 con 0x560a99b49490 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 -- 172.21.15.133:0/1997225747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefd80d9070 con 0x560a99b49490 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7fefce7fc640 1 -- 172.21.15.133:0/1997225747 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefe8020050 con 0x560a99b49490 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 10 client.10698 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 -- 172.21.15.133:0/1997225747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x560a998de9b0 con 0x560a99b49490 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7fefce7fc640 1 -- 172.21.15.133:0/1997225747 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd80d9070 con 0x560a99b49490 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7fefce7fc640 1 -- 172.21.15.133:0/1997225747 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fefe8023020 con 0x560a99b49490 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 10 client.10698 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 10 client.10698 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 20 client.10698 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x560a99b5c0e0 0x560a99b8d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 4 client.10698 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 -- 172.21.15.133:0/1997225747 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560a999572e0 con 0x560a99b5c0e0 2024-04-25T10:15:18.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7fefef7fe640 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x560a99b5c0e0 0x560a99b8d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7fefce7fc640 10 client.10698 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7fefef7fe640 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x560a99b5c0e0 0x560a99b8d1d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d16b58ea779183d server_cookie=73dbd6d47c7c9603 in_seq=0 out_seq=0 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7fefce7fc640 1 -- 172.21.15.133:0/1997225747 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x560a999572e0 con 0x560a99b5c0e0 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7fefce7fc640 10 client.10698 handle_command_reply: tid=0 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 client.10698 shutdown 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 -- 172.21.15.133:0/1997225747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b49490 msgr2=0x560a99b5bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b49490 0x560a99b5bba0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7feff015d2b0 tx=0x7fefe80019f0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 -- 172.21.15.133:0/1997225747 shutdown_connections 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a99b49490 0x560a99b5bba0 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560a99b48b70 0x560a99b5b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x560a99b5c0e0 0x560a99b8d1d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 --2- 172.21.15.133:0/1997225747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x560a99aff340 0x560a99b8c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7fefce7fc640 0 client.10698 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:18.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 -- 172.21.15.133:0/1997225747 >> 172.21.15.133:0/1997225747 conn(0x560a99b012a0 msgr2=0x560a99b50870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 -- 172.21.15.133:0/1997225747 shutdown_connections 2024-04-25T10:15:18.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 1 -- 172.21.15.133:0/1997225747 wait complete. 2024-04-25T10:15:18.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff90f7000 20 client.10698 trim_cache size 0 max 16384 2024-04-25T10:15:18.484 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:15:18.484 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:15:18.484 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:15:18.484 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 10554, 2024-04-25T10:15:18.484 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:15:18.484 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:15:18.484 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:15:18.485 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 4.7165689159999999, 2024-04-25T10:15:18.485 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 99, 2024-04-25T10:15:18.485 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 157, 2024-04-25T10:15:18.485 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 157, 2024-04-25T10:15:18.485 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 10.070580412 2024-04-25T10:15:18.485 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:15:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 -- 172.21.15.133:0/217983230 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fefd0066f90 msgr2=0x7fefd0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fefd0066f90 0x7fefd0069450 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fefd80af000 tx=0x7fefd80aa000 comp rx=0 tx=0).stop 2024-04-25T10:15:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 -- 172.21.15.133:0/217983230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 msgr2=0x7feff013db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 0x7feff013db80 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fefe0011690 tx=0x7fefe0011b50 comp rx=0 tx=0).stop 2024-04-25T10:15:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 -- 172.21.15.133:0/217983230 shutdown_connections 2024-04-25T10:15:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7feff00adce0 0x7feff00ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feff0142290 0x7feff0139140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fefd0066f90 0x7fefd0069450 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 --2- 172.21.15.133:0/217983230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefdc004810 0x7feff013db80 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 -- 172.21.15.133:0/217983230 >> 172.21.15.133:0/217983230 conn(0x7feff0066f70 msgr2=0x7feff00cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 -- 172.21.15.133:0/217983230 shutdown_connections 2024-04-25T10:15:18.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.478+0000 7feff5f8a640 1 -- 172.21.15.133:0/217983230 wait complete. 2024-04-25T10:15:18.495 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:15:18.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.606+0000 7f896ce0d640 1 Processor -- start 2024-04-25T10:15:18.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.606+0000 7f896ce0d640 1 -- start start 2024-04-25T10:15:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.606+0000 7f896ce0d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8968152750 0x7f89680ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.606+0000 7f896ce0d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8968152b30 0x7f89680ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.606+0000 7f896ce0d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f89680b17b0 0x7f89680b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.606+0000 7f896ce0d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896806d350 con 0x7f8968152750 2024-04-25T10:15:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.606+0000 7f896ce0d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896815bf90 con 0x7f89680b17b0 2024-04-25T10:15:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.606+0000 7f896ce0d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896806d5d0 con 0x7f8968152b30 2024-04-25T10:15:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8968152750 0x7f89680ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8968152750 0x7f89680ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54174/0 (socket says 172.21.15.133:54174) 2024-04-25T10:15:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896659c640 1 -- 172.21.15.133:0/743179643 learned_addr learned my addr 172.21.15.133:0/743179643 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f8966d9d640 1 --2- 172.21.15.133:0/743179643 >> v2:172.21.15.162:3300/0 conn(0x7f89680b17b0 0x7f89680b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f8965d9b640 1 --2- 172.21.15.133:0/743179643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8968152b30 0x7f89680ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896659c640 1 -- 172.21.15.133:0/743179643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8968152b30 msgr2=0x7f89680ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896659c640 1 --2- 172.21.15.133:0/743179643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8968152b30 0x7f89680ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896659c640 1 -- 172.21.15.133:0/743179643 >> v2:172.21.15.162:3300/0 conn(0x7f89680b17b0 msgr2=0x7f89680b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896659c640 1 --2- 172.21.15.133:0/743179643 >> v2:172.21.15.162:3300/0 conn(0x7f89680b17b0 0x7f89680b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896659c640 1 -- 172.21.15.133:0/743179643 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896815c230 con 0x7f8968152750 2024-04-25T10:15:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f8966d9d640 1 --2- 172.21.15.133:0/743179643 >> v2:172.21.15.162:3300/0 conn(0x7f89680b17b0 0x7f89680b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:18.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f8965d9b640 1 --2- 172.21.15.133:0/743179643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8968152b30 0x7f89680ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:18.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896659c640 1 --2- 172.21.15.133:0/743179643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8968152750 0x7f89680ce330 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f896806f670 tx=0x7f895801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37d59b487271792 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896559a640 1 -- 172.21.15.133:0/743179643 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89580b1070 con 0x7f8968152750 2024-04-25T10:15:18.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896559a640 1 -- 172.21.15.133:0/743179643 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89580ac630 con 0x7f8968152750 2024-04-25T10:15:18.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896559a640 1 -- 172.21.15.133:0/743179643 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89580a8050 con 0x7f8968152750 2024-04-25T10:15:18.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896ce0d640 1 -- 172.21.15.133:0/743179643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8968152750 msgr2=0x7f89680ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896ce0d640 1 --2- 172.21.15.133:0/743179643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8968152750 0x7f89680ce330 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f896806f670 tx=0x7f895801c520 comp rx=0 tx=0).stop 2024-04-25T10:15:18.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896ce0d640 1 -- 172.21.15.133:0/743179643 shutdown_connections 2024-04-25T10:15:18.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896ce0d640 1 --2- 172.21.15.133:0/743179643 >> v2:172.21.15.162:3300/0 conn(0x7f89680b17b0 0x7f89680b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896ce0d640 1 --2- 172.21.15.133:0/743179643 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8968152b30 0x7f89680ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896ce0d640 1 --2- 172.21.15.133:0/743179643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8968152750 0x7f89680ce330 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896ce0d640 1 -- 172.21.15.133:0/743179643 >> 172.21.15.133:0/743179643 conn(0x7f8968067000 msgr2=0x7f896806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896ce0d640 1 -- 172.21.15.133:0/743179643 shutdown_connections 2024-04-25T10:15:18.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.610+0000 7f896ce0d640 1 -- 172.21.15.133:0/743179643 wait complete. 2024-04-25T10:15:18.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896ce0d640 1 Processor -- start 2024-04-25T10:15:18.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896ce0d640 1 -- start start 2024-04-25T10:15:18.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896ce0d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89680b17b0 0x7f8968149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896ce0d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8968152750 0x7f89681f9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896ce0d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8968152b30 0x7f896815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896ce0d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89680c4fd0 con 0x7f89680b17b0 2024-04-25T10:15:18.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896ce0d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89680c4e50 con 0x7f8968152750 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896ce0d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89680c5150 con 0x7f8968152b30 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89680b17b0 0x7f8968149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89680b17b0 0x7f8968149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54190/0 (socket says 172.21.15.133:54190) 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896659c640 1 -- 172.21.15.133:0/2237144416 learned_addr learned my addr 172.21.15.133:0/2237144416 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f8966d9d640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8968152b30 0x7f896815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896659c640 1 -- 172.21.15.133:0/2237144416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8968152b30 msgr2=0x7f896815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896659c640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8968152b30 0x7f896815aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896659c640 1 -- 172.21.15.133:0/2237144416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8968152750 msgr2=0x7f89681f9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f8965d9b640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8968152750 0x7f89681f9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896659c640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8968152750 0x7f89681f9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896659c640 1 -- 172.21.15.133:0/2237144416 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89681fa7a0 con 0x7f89680b17b0 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896659c640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89680b17b0 0x7f8968149920 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f895801cdc0 tx=0x7f89580192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f8956ffd640 1 -- 172.21.15.133:0/2237144416 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89580bc070 con 0x7f89680b17b0 2024-04-25T10:15:18.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f8956ffd640 1 -- 172.21.15.133:0/2237144416 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89580ab800 con 0x7f89680b17b0 2024-04-25T10:15:18.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896ce0d640 1 -- 172.21.15.133:0/2237144416 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89681faa80 con 0x7f89680b17b0 2024-04-25T10:15:18.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f8956ffd640 1 -- 172.21.15.133:0/2237144416 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89580b1070 con 0x7f89680b17b0 2024-04-25T10:15:18.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896ce0d640 1 -- 172.21.15.133:0/2237144416 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896815a4f0 con 0x7f89680b17b0 2024-04-25T10:15:18.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.614+0000 7f896ce0d640 1 -- 172.21.15.133:0/2237144416 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8968068ef0 con 0x7f89680b17b0 2024-04-25T10:15:18.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.618+0000 7f8956ffd640 1 -- 172.21.15.133:0/2237144416 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89580bf070 con 0x7f89680b17b0 2024-04-25T10:15:18.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.618+0000 7f8956ffd640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8948066f90 0x7f8948069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.618+0000 7f8965d9b640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8948066f90 0x7f8948069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.618+0000 7f8956ffd640 1 -- 172.21.15.133:0/2237144416 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f895811ec90 con 0x7f89680b17b0 2024-04-25T10:15:18.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.618+0000 7f8965d9b640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8948066f90 0x7f8948069450 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f895c006fd0 tx=0x7f895c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.622+0000 7f8956ffd640 1 -- 172.21.15.133:0/2237144416 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8958016110 con 0x7f89680b17b0 2024-04-25T10:15:18.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.814+0000 7f896ce0d640 1 -- 172.21.15.133:0/2237144416 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f896815b440 con 0x7f89680b17b0 2024-04-25T10:15:18.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.814+0000 7f8956ffd640 1 -- 172.21.15.133:0/2237144416 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7f89580efba0 con 0x7f89680b17b0 2024-04-25T10:15:18.818 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:15:18.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 -- 172.21.15.133:0/2237144416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8948066f90 msgr2=0x7f8948069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8948066f90 0x7f8948069450 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f895c006fd0 tx=0x7f895c008040 comp rx=0 tx=0).stop 2024-04-25T10:15:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 -- 172.21.15.133:0/2237144416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89680b17b0 msgr2=0x7f8968149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89680b17b0 0x7f8968149920 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f895801cdc0 tx=0x7f89580192c0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 -- 172.21.15.133:0/2237144416 shutdown_connections 2024-04-25T10:15:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8968152b30 0x7f896815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8968152750 0x7f89681f9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8948066f90 0x7f8948069450 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 --2- 172.21.15.133:0/2237144416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89680b17b0 0x7f8968149920 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 -- 172.21.15.133:0/2237144416 >> 172.21.15.133:0/2237144416 conn(0x7f8968067000 msgr2=0x7f8968165ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 -- 172.21.15.133:0/2237144416 shutdown_connections 2024-04-25T10:15:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:18.818+0000 7f896ce0d640 1 -- 172.21.15.133:0/2237144416 wait complete. 2024-04-25T10:15:18.833 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:15:18.833 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:15:18.833 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:15:18.833 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:15:18.833 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:18.833 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:15:18.833 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:15:18.833 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:15:18.833 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:18.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.834 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:15:18.840 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:15:18.840 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:15:18.840 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:15:18.840 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.840 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:15:18.840 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.840 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:15:18.840 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.841 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:15:18.841 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:15:18.841 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.841 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:15:18.841 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:15:18.841 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.841 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:15:18.841 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.841 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:15:18.841 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41713sec preferred_lft 41713sec 2024-04-25T10:15:18.841 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:15:18.842 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:15:18.843 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:15:18.843 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.843 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:15:18.843 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:15:18.843 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:15:18.843 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.843 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:15:18.894 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:15:18.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.895 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:15:18.948 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:15:18.948 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:15:18.948 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:15:18.948 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:15:18.948 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:15:18.948 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:15:18.948 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:15:18.948 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:15:18.948 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:15:18.948 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:15:18.949 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:15:18.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.949 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:15:18.949 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:15:18.949 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:15:18.949 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:15:18.949 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:15:18.949 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:15:19.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:19.027 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:15:19.072 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:15:19.073 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:15:19.073 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:19.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:19.073 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:15:19.130 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:15:19.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:15:19.130 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:19.177 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:19.177 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:15:19.177 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:19.229 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:15:19.286 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:15:19.340 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:15:19.340 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:15:19.341 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:15:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:15:19.343 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:15:19.343 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:15:19.343 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:15:19.343 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:15:19.343 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:15:19.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:19.344 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:15:19.383 INFO:tasks.ceph.mds.b.smithi162.stderr:2024-04-25T10:15:19.381+0000 7f2cd23bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:15:19.393 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:15:19.393 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:15:19.438 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:15:19.452 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:15:19.501 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:15:19.502 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:15:19.503 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:15:19.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:19.504 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:15:19.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:15:19.518+0000 7f870ad27580 -1 init, newargv = 0x5560fdc6add0 newargc=15 2024-04-25T10:15:19.519 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[87636]: starting ceph client 2024-04-25T10:15:19.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[87636]: starting fuse 2024-04-25T10:15:20.625 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:15:20.640 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:15:20.694 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:15:20.694 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:15:20.694 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:15:20.694 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:15:20.694 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:15:20.694 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:15:20.694 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:15:20.694 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:15:20.694 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:15:20.695 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:15:20.696 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:15:20.696 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:15:20.696 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:15:20.696 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:15:20.696 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:15:20.696 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:15:20.696 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:15:20.696 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:15:20.696 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:15:20.696 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:15:20.696 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:20.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:20.697 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:15:20.745 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:15:20.745 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:15:20.746 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:15:20.747 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:15:20.868 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.87636.asok 2024-04-25T10:15:20.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:15:20.879 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.87636.asok 2024-04-25T10:15:20.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:20.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.87636.asok status 2024-04-25T10:15:21.033 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:15:21.033 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:15:21.033 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:15:21.033 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:15:21.033 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:15:21.033 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:15:21.033 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:15:21.033 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "87636", 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 10713, 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 10713 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:15:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1326527298 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1326527298 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.10713 192.168.0.1:0/1326527298", 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/1326527298", 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 99, 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 157, 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:15:21.035 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:15:21.036 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:15:21.036 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:15:21.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:21.047 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:15:21.047 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:21.054 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:15:21.054 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:21.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:21.055 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:21.109 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:21.109 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:15:21.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.278+0000 7f1e65487640 1 Processor -- start 2024-04-25T10:15:21.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e65487640 1 -- start start 2024-04-25T10:15:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e65487640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e6015e100 0x7f1e60140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e65487640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e600cf280 con 0x7f1e60143560 2024-04-25T10:15:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e65487640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6006d440 con 0x7f1e60142920 2024-04-25T10:15:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e65487640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6006d720 con 0x7f1e6015e100 2024-04-25T10:15:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e5f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e6015e100 0x7f1e60140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e5e7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1e60143560 0x7f1e6015d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58624/0 (socket says 172.21.15.133:58624) 2024-04-25T10:15:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e5e7fc640 1 -- 172.21.15.133:0/818654383 learned_addr learned my addr 172.21.15.133:0/818654383 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e5f7fe640 1 -- 172.21.15.133:0/818654383 >> v1:172.21.15.162:6789/0 conn(0x7f1e60142920 legacy=0x7f1e600ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e5f7fe640 1 -- 172.21.15.133:0/818654383 >> v1:172.21.15.133:6789/0 conn(0x7f1e60143560 legacy=0x7f1e6015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e5f7fe640 1 -- 172.21.15.133:0/818654383 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6006da00 con 0x7f1e6015e100 2024-04-25T10:15:21.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e5dffb640 1 -- 172.21.15.133:0/818654383 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179944675 0 0) 0x7f1e600cf280 con 0x7f1e60143560 2024-04-25T10:15:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.282+0000 7f1e5f7fe640 1 --2- 172.21.15.133:0/818654383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e6015e100 0x7f1e60140a40 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800aea0 tx=0x7f1e5801d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7575983f227e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5dffb640 1 -- 172.21.15.133:0/818654383 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580a6710 con 0x7f1e6015e100 2024-04-25T10:15:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5dffb640 1 -- 172.21.15.133:0/818654383 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e580a68b0 con 0x7f1e6015e100 2024-04-25T10:15:21.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5dffb640 1 -- 172.21.15.133:0/818654383 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580a6c80 con 0x7f1e6015e100 2024-04-25T10:15:21.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 -- 172.21.15.133:0/818654383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e6015e100 msgr2=0x7f1e60140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 --2- 172.21.15.133:0/818654383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e6015e100 0x7f1e60140a40 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800aea0 tx=0x7f1e5801d9a0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 -- 172.21.15.133:0/818654383 shutdown_connections 2024-04-25T10:15:21.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 --2- 172.21.15.133:0/818654383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e6015e100 0x7f1e60140a40 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 -- 172.21.15.133:0/818654383 >> 172.21.15.133:0/818654383 conn(0x7f1e60066f70 msgr2=0x7f1e6006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 -- 172.21.15.133:0/818654383 shutdown_connections 2024-04-25T10:15:21.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 -- 172.21.15.133:0/818654383 wait complete. 2024-04-25T10:15:21.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 Processor -- start 2024-04-25T10:15:21.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 -- start start 2024-04-25T10:15:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e60142920 0x7f1e6015a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60143560 0x7f1e6015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e6015e100 0x7f1e60156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6006e9c0 con 0x7f1e60143560 2024-04-25T10:15:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e60143980 con 0x7f1e60142920 2024-04-25T10:15:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e65487640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6006eb40 con 0x7f1e6015e100 2024-04-25T10:15:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60143560 0x7f1e6015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60143560 0x7f1e6015c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54212/0 (socket says 172.21.15.133:54212) 2024-04-25T10:15:21.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5e7fc640 1 -- 172.21.15.133:0/594934397 learned_addr learned my addr 172.21.15.133:0/594934397 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5e7fc640 1 -- 172.21.15.133:0/594934397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e6015e100 msgr2=0x7f1e60156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5e7fc640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e6015e100 0x7f1e60156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5e7fc640 1 -- 172.21.15.133:0/594934397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e60142920 msgr2=0x7f1e6015a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5e7fc640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e60142920 0x7f1e6015a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5e7fc640 1 -- 172.21.15.133:0/594934397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6015ab10 con 0x7f1e60143560 2024-04-25T10:15:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e5e7fc640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60143560 0x7f1e6015c5b0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400c4d0 tx=0x7f1e5400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e477fe640 1 -- 172.21.15.133:0/594934397 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54018070 con 0x7f1e60143560 2024-04-25T10:15:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e477fe640 1 -- 172.21.15.133:0/594934397 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e540074e0 con 0x7f1e60143560 2024-04-25T10:15:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.286+0000 7f1e477fe640 1 -- 172.21.15.133:0/594934397 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54004a30 con 0x7f1e60143560 2024-04-25T10:15:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e65487640 1 -- 172.21.15.133:0/594934397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6015adf0 con 0x7f1e60143560 2024-04-25T10:15:21.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e65487640 1 -- 172.21.15.133:0/594934397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6015b3b0 con 0x7f1e60143560 2024-04-25T10:15:21.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e477fe640 1 -- 172.21.15.133:0/594934397 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e54004bd0 con 0x7f1e60143560 2024-04-25T10:15:21.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e477fe640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e4c066f60 0x7f1e4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e477fe640 1 -- 172.21.15.133:0/594934397 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e54084500 con 0x7f1e60143560 2024-04-25T10:15:21.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e5effd640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e4c066f60 0x7f1e4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e5effd640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e4c066f60 0x7f1e4c069420 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1e50007fd0 tx=0x7f1e500079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e685f4000 1 Processor -- start 2024-04-25T10:15:21.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e685f4000 1 -- start start 2024-04-25T10:15:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b308560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b308aa0 0x55d42b2be0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2be9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e685f4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42b10bf20 con 0x55d42b308180 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e685f4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42b094000 con 0x55d42b308aa0 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e685f4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d42b107fc0 con 0x55d42b2be600 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b308560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b308560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54228/0 (socket says 172.21.15.133:54228) 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e5effd640 1 -- 172.21.15.133:0/1405113511 learned_addr learned my addr 172.21.15.133:0/1405113511 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e5e7fc640 1 --2- 172.21.15.133:0/1405113511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b308aa0 0x55d42b2be0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e5f7fe640 1 --2- 172.21.15.133:0/1405113511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2be9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.290+0000 7f1e5effd640 1 -- 172.21.15.133:0/1405113511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 msgr2=0x55d42b2be9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5effd640 1 --2- 172.21.15.133:0/1405113511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2be9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5effd640 1 -- 172.21.15.133:0/1405113511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b308aa0 msgr2=0x55d42b2be0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5effd640 1 --2- 172.21.15.133:0/1405113511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b308aa0 0x55d42b2be0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5effd640 1 -- 172.21.15.133:0/1405113511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d42b0e7ba0 con 0x55d42b308180 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5e7fc640 1 --2- 172.21.15.133:0/1405113511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b308aa0 0x55d42b2be0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:21.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5effd640 1 --2- 172.21.15.133:0/1405113511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b308560 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005540 tx=0x7f1e5000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd6640109293370 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e44ff9640 1 -- 172.21.15.133:0/1405113511 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50016510 con 0x55d42b308180 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e44ff9640 1 -- 172.21.15.133:0/1405113511 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e500166b0 con 0x55d42b308180 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e44ff9640 1 -- 172.21.15.133:0/1405113511 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50016a80 con 0x55d42b308180 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- 172.21.15.133:0/1405113511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 msgr2=0x55d42b308560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1405113511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b308560 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f1e50005540 tx=0x7f1e5000f000 comp rx=0 tx=0).stop 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- 172.21.15.133:0/1405113511 shutdown_connections 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1405113511 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2be9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1405113511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b308aa0 0x55d42b2be0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1405113511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b308560 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- 172.21.15.133:0/1405113511 >> 172.21.15.133:0/1405113511 conn(0x55d42b2c4760 msgr2=0x55d42b2bc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- 172.21.15.133:0/1405113511 shutdown_connections 2024-04-25T10:15:21.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- 172.21.15.133:0/1405113511 wait complete. 2024-04-25T10:15:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- start start 2024-04-25T10:15:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 Processor -- start 2024-04-25T10:15:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b308aa0 0x55d42b2f6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42b1c99c0 con 0x55d42b308180 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42b0c65d0 con 0x55d42b308aa0 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d42b0b72f0 con 0x55d42b2be600 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 10 client.0 _ll_register_callbacks cb 0x55d42b2a9a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54238/0 (socket says 172.21.15.133:54238) 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5e7fc640 1 -- 172.21.15.133:0/1430476301 learned_addr learned my addr 172.21.15.133:0/1430476301 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5effd640 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5f7fe640 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b308aa0 0x55d42b2f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5e7fc640 1 -- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 msgr2=0x55d42b2c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5e7fc640 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5e7fc640 1 -- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b308aa0 msgr2=0x55d42b2f6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5e7fc640 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b308aa0 0x55d42b2f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5e7fc640 1 -- 172.21.15.133:0/1430476301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d42b1525a0 con 0x55d42b308180 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5effd640 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e5e7fc640 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2c3be0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f1e5401f030 tx=0x7f1e540553a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8910f96b4e8c3211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1430476301 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5405d060 con 0x55d42b308180 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1430476301 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e54054030 con 0x55d42b308180 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- 172.21.15.133:0/1430476301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e500166b0 con 0x55d42b308180 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1430476301 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e5405d2f0 con 0x55d42b308180 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 10 client.10728 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- 172.21.15.133:0/1430476301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55d42b0e32b0 con 0x55d42b308180 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1430476301 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e500166b0 con 0x55d42b308180 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1430476301 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1e54068ba0 con 0x55d42b308180 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 10 client.10728 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 10 client.10728 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T10:15:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 20 client.10728 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55d42b2f5e60 0x55d42b2f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 4 client.10728 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.294+0000 7f1e685f4000 1 -- 172.21.15.133:0/1430476301 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d42b0a7c70 con 0x55d42b2f5e60 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e5effd640 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55d42b2f5e60 0x55d42b2f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e3dffb640 10 client.10728 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e5effd640 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55d42b2f5e60 0x55d42b2f6e60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dcd9ae893b4c50c server_cookie=a9393a6bb372a058 in_seq=0 out_seq=0 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1430476301 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d42b0a7c70 con 0x55d42b2f5e60 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e3dffb640 10 client.10728 handle_command_reply: tid=0 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 1 client.10728 shutdown 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 1 -- 172.21.15.133:0/1430476301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 msgr2=0x55d42b2c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2c3be0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f1e5401f030 tx=0x7f1e540553a0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 1 -- 172.21.15.133:0/1430476301 shutdown_connections 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b308aa0 0x55d42b2f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2c3be0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55d42b2f5e60 0x55d42b2f6e60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1430476301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e3dffb640 0 client.10728 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 1 -- 172.21.15.133:0/1430476301 >> 172.21.15.133:0/1430476301 conn(0x55d42b2c4760 msgr2=0x55d42b2bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 1 -- 172.21.15.133:0/1430476301 shutdown_connections 2024-04-25T10:15:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 1 -- 172.21.15.133:0/1430476301 wait complete. 2024-04-25T10:15:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.298+0000 7f1e685f4000 20 client.10728 trim_cache size 0 max 16384 2024-04-25T10:15:21.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e685f4000 1 Processor -- start 2024-04-25T10:15:21.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e685f4000 1 -- start start 2024-04-25T10:15:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b2c4760 0x55d42b31e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2bd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e685f4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601ff350 con 0x55d42b308180 2024-04-25T10:15:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e685f4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601fe5b0 con 0x55d42b2c4760 2024-04-25T10:15:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e685f4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6013b240 con 0x55d42b2be600 2024-04-25T10:15:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2bd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2bd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54240/0 (socket says 172.21.15.133:54240) 2024-04-25T10:15:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e5f7fe640 1 -- 172.21.15.133:0/1617938042 learned_addr learned my addr 172.21.15.133:0/1617938042 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e5f7fe640 1 -- 172.21.15.133:0/1617938042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 msgr2=0x55d42b2bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e5f7fe640 1 --2- 172.21.15.133:0/1617938042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e5f7fe640 1 -- 172.21.15.133:0/1617938042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b2c4760 msgr2=0x55d42b31e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e5effd640 1 --2- 172.21.15.133:0/1617938042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b2c4760 0x55d42b31e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e5f7fe640 1 --2- 172.21.15.133:0/1617938042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b2c4760 0x55d42b31e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e5f7fe640 1 -- 172.21.15.133:0/1617938042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d42b2f71d0 con 0x55d42b308180 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.310+0000 7f1e5f7fe640 1 --2- 172.21.15.133:0/1617938042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2bd660 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f1e5801a5b0 tx=0x7f1e58004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498000043f035f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e3ffff640 1 -- 172.21.15.133:0/1617938042 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580a6710 con 0x55d42b308180 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e3ffff640 1 -- 172.21.15.133:0/1617938042 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e580bf070 con 0x55d42b308180 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e3ffff640 1 -- 172.21.15.133:0/1617938042 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580a6ad0 con 0x55d42b308180 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- 172.21.15.133:0/1617938042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 msgr2=0x55d42b2bd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1617938042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2bd660 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f1e5801a5b0 tx=0x7f1e58004a50 comp rx=0 tx=0).stop 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- 172.21.15.133:0/1617938042 shutdown_connections 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1617938042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b308180 0x55d42b2bd660 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1617938042 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b2be600 0x55d42b2bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1617938042 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b2c4760 0x55d42b31e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- 172.21.15.133:0/1617938042 >> 172.21.15.133:0/1617938042 conn(0x55d42b2f65b0 msgr2=0x55d42b315770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- 172.21.15.133:0/1617938042 shutdown_connections 2024-04-25T10:15:21.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- 172.21.15.133:0/1617938042 wait complete. 2024-04-25T10:15:21.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- start start 2024-04-25T10:15:21.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 Processor -- start 2024-04-25T10:15:21.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b2be600 0x55d42b370bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b2c4760 0x55d42b371100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b308180 0x55d42b31d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d42b0c2dc0 con 0x55d42b2be600 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6013b840 con 0x55d42b2c4760 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d42b0c8080 con 0x55d42b308180 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 10 client.0 _ll_register_callbacks cb 0x55d42b28a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b2be600 0x55d42b370bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b2be600 0x55d42b370bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54252/0 (socket says 172.21.15.133:54252) 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5effd640 1 -- 172.21.15.133:0/1586957330 learned_addr learned my addr 172.21.15.133:0/1586957330 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5f7fe640 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b308180 0x55d42b31d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5e7fc640 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b2c4760 0x55d42b371100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5effd640 1 -- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b308180 msgr2=0x55d42b31d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5effd640 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b308180 0x55d42b31d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5effd640 1 -- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b2c4760 msgr2=0x55d42b371100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5effd640 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b2c4760 0x55d42b371100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5effd640 1 -- 172.21.15.133:0/1586957330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d42b094250 con 0x55d42b2be600 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5effd640 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b2be600 0x55d42b370bc0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f1e500054e0 tx=0x7f1e500010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259093f857835784 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1586957330 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50029550 con 0x55d42b2be600 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1586957330 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e500296f0 con 0x55d42b2be600 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- 172.21.15.133:0/1586957330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e580bf070 con 0x55d42b2be600 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1586957330 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e50031050 con 0x55d42b2be600 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 10 client.10734 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- 172.21.15.133:0/1586957330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55d42b0a0b90 con 0x55d42b2be600 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1586957330 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e580bf070 con 0x55d42b2be600 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1586957330 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1e50034020 con 0x55d42b2be600 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 10 client.10734 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 10 client.10734 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 20 client.10734 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55d42b308aa0 0x55d42b31db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 4 client.10734 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e685f4000 1 -- 172.21.15.133:0/1586957330 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d42b119460 con 0x55d42b308aa0 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.314+0000 7f1e5e7fc640 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55d42b308aa0 0x55d42b31db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e3dffb640 10 client.10734 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e5e7fc640 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55d42b308aa0 0x55d42b31db20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc88971464cdf801 server_cookie=fa57b2a5bdc868e9 in_seq=0 out_seq=0 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e3dffb640 1 -- 172.21.15.133:0/1586957330 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d42b119460 con 0x55d42b308aa0 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e3dffb640 10 client.10734 handle_command_reply: tid=0 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 1 client.10734 shutdown 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 1 -- 172.21.15.133:0/1586957330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b2be600 msgr2=0x55d42b370bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b2be600 0x55d42b370bc0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f1e500054e0 tx=0x7f1e500010a0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 1 -- 172.21.15.133:0/1586957330 shutdown_connections 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55d42b308aa0 0x55d42b31db20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d42b308180 0x55d42b31d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d42b2c4760 0x55d42b371100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e3dffb640 0 client.10734 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:21.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 1 --2- 172.21.15.133:0/1586957330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d42b2be600 0x55d42b370bc0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 1 -- 172.21.15.133:0/1586957330 >> 172.21.15.133:0/1586957330 conn(0x55d42b2f65b0 msgr2=0x55d42b2bc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 1 -- 172.21.15.133:0/1586957330 shutdown_connections 2024-04-25T10:15:21.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 1 -- 172.21.15.133:0/1586957330 wait complete. 2024-04-25T10:15:21.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e685f4000 20 client.10734 trim_cache size 0 max 16384 2024-04-25T10:15:21.322 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-04-25T10:15:21.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 -- 172.21.15.133:0/594934397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e4c066f60 msgr2=0x7f1e4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e4c066f60 0x7f1e4c069420 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1e50007fd0 tx=0x7f1e500079a0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 -- 172.21.15.133:0/594934397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60143560 msgr2=0x7f1e6015c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60143560 0x7f1e6015c5b0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400c4d0 tx=0x7f1e5400c990 comp rx=0 tx=0).stop 2024-04-25T10:15:21.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 -- 172.21.15.133:0/594934397 shutdown_connections 2024-04-25T10:15:21.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1e6015e100 0x7f1e60156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1e4c066f60 0x7f1e4c069420 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e60143560 0x7f1e6015c5b0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 --2- 172.21.15.133:0/594934397 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1e60142920 0x7f1e6015a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 -- 172.21.15.133:0/594934397 >> 172.21.15.133:0/594934397 conn(0x7f1e60066f70 msgr2=0x7f1e600b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 -- 172.21.15.133:0/594934397 shutdown_connections 2024-04-25T10:15:21.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.318+0000 7f1e65487640 1 -- 172.21.15.133:0/594934397 wait complete. 2024-04-25T10:15:21.334 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:15:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d0456d640 1 Processor -- start 2024-04-25T10:15:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d0456d640 1 -- start start 2024-04-25T10:15:21.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d0456d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9cfc0ce6b0 0x7f9cfc06cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d0456d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cfc0cec40 con 0x7f9cfc0b2540 2024-04-25T10:15:21.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d0456d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cfc15e2a0 con 0x7f9cfc06c7b0 2024-04-25T10:15:21.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d0456d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cfc06d150 con 0x7f9cfc0ce6b0 2024-04-25T10:15:21.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d02309640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9cfc0b2540 0x7f9cfc15ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58628/0 (socket says 172.21.15.133:58628) 2024-04-25T10:15:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d02309640 1 -- 172.21.15.133:0/3516055176 learned_addr learned my addr 172.21.15.133:0/3516055176 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d02b0a640 1 --2- 172.21.15.133:0/3516055176 >> v2:172.21.15.162:3300/0 conn(0x7f9cfc0ce6b0 0x7f9cfc06cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219498125 0 0) 0x7f9cfc0cec40 con 0x7f9cfc0b2540 2024-04-25T10:15:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce8003530 con 0x7f9cfc0b2540 2024-04-25T10:15:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3164910939 0 0) 0x7f9ce8003530 con 0x7f9cfc0b2540 2024-04-25T10:15:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 >> v2:172.21.15.162:3300/0 conn(0x7f9cfc0ce6b0 msgr2=0x7f9cfc06cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d01307640 1 --2- 172.21.15.133:0/3516055176 >> v2:172.21.15.162:3300/0 conn(0x7f9cfc0ce6b0 0x7f9cfc06cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 >> v1:172.21.15.162:6789/0 conn(0x7f9cfc06c7b0 legacy=0x7f9cfc0cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc15e580 con 0x7f9cfc0b2540 2024-04-25T10:15:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.446+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f9cf8001680 con 0x7f9cfc0b2540 2024-04-25T10:15:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 >> v1:172.21.15.133:6789/0 conn(0x7f9cfc0b2540 legacy=0x7f9cfc15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01307640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 0x7f9ce8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01307640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ce8004810 0x7f9ce8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01307640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce8005220 0x7f9ce800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce800c140 con 0x7f9ce8003a10 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce800c2f0 con 0x7f9ce8004810 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce800c4f0 con 0x7f9ce8005220 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d02309640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 0x7f9ce8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01b08640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ce8004810 0x7f9ce8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d02b0a640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce8005220 0x7f9ce800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d02309640 1 -- 172.21.15.133:0/3516055176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce8005220 msgr2=0x7f9ce800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d02309640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9ce8005220 0x7f9ce800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d02309640 1 -- 172.21.15.133:0/3516055176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ce8004810 msgr2=0x7f9ce8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d02309640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ce8004810 0x7f9ce8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d02309640 1 -- 172.21.15.133:0/3516055176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce8003530 con 0x7f9ce8003a10 2024-04-25T10:15:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01b08640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9ce8004810 0x7f9ce8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d02b0a640 1 -- 172.21.15.133:0/3516055176 reap_dead start 2024-04-25T10:15:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d02309640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 0x7f9ce8003e30 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800c160 tx=0x7f9cf801cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be123dacc6d82b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8007c70 con 0x7f9ce8003a10 2024-04-25T10:15:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf80a9070 con 0x7f9ce8003a10 2024-04-25T10:15:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d01307640 1 -- 172.21.15.133:0/3516055176 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8005dc0 con 0x7f9ce8003a10 2024-04-25T10:15:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d0456d640 1 -- 172.21.15.133:0/3516055176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 msgr2=0x7f9ce8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d0456d640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 0x7f9ce8003e30 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800c160 tx=0x7f9cf801cb20 comp rx=0 tx=0).stop 2024-04-25T10:15:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d0456d640 1 -- 172.21.15.133:0/3516055176 shutdown_connections 2024-04-25T10:15:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d0456d640 1 --2- 172.21.15.133:0/3516055176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 0x7f9ce8003e30 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d0456d640 1 -- 172.21.15.133:0/3516055176 >> 172.21.15.133:0/3516055176 conn(0x7f9cfc09aa10 msgr2=0x7f9cfc08af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d0456d640 1 -- 172.21.15.133:0/3516055176 shutdown_connections 2024-04-25T10:15:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.450+0000 7f9d0456d640 1 -- 172.21.15.133:0/3516055176 wait complete. 2024-04-25T10:15:21.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d0456d640 1 Processor -- start 2024-04-25T10:15:21.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d0456d640 1 -- start start 2024-04-25T10:15:21.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d0456d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 0x7f9cfc0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d0456d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1f74c0 0x7f9cfc15f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d0456d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cfc163300 0x7f9cfc15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d0456d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0d00a0 con 0x7f9ce8003a10 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d0456d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc06cfa0 con 0x7f9cfc1f74c0 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d0456d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0d0220 con 0x7f9cfc163300 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d02309640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 0x7f9cfc0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d02309640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 0x7f9cfc0c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54268/0 (socket says 172.21.15.133:54268) 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d02309640 1 -- 172.21.15.133:0/1777594645 learned_addr learned my addr 172.21.15.133:0/1777594645 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d02b0a640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cfc163300 0x7f9cfc15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d01b08640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1f74c0 0x7f9cfc15f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d02309640 1 -- 172.21.15.133:0/1777594645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cfc163300 msgr2=0x7f9cfc15fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d02309640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cfc163300 0x7f9cfc15fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d02309640 1 -- 172.21.15.133:0/1777594645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1f74c0 msgr2=0x7f9cfc15f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d02309640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1f74c0 0x7f9cfc15f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d02309640 1 -- 172.21.15.133:0/1777594645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc0ca1b0 con 0x7f9ce8003a10 2024-04-25T10:15:21.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d02309640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 0x7f9cfc0c9840 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800c160 tx=0x7f9ce8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9cf2ffd640 1 -- 172.21.15.133:0/1777594645 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8000ee0 con 0x7f9ce8003a10 2024-04-25T10:15:21.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9cf2ffd640 1 -- 172.21.15.133:0/1777594645 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf8004e50 con 0x7f9ce8003a10 2024-04-25T10:15:21.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9cf2ffd640 1 -- 172.21.15.133:0/1777594645 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8007380 con 0x7f9ce8003a10 2024-04-25T10:15:21.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d0456d640 1 -- 172.21.15.133:0/1777594645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cfc0ca490 con 0x7f9ce8003a10 2024-04-25T10:15:21.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d0456d640 1 -- 172.21.15.133:0/1777594645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc0caa50 con 0x7f9ce8003a10 2024-04-25T10:15:21.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.454+0000 7f9d0456d640 1 -- 172.21.15.133:0/1777594645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cfc068ef0 con 0x7f9ce8003a10 2024-04-25T10:15:21.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.458+0000 7f9cf2ffd640 1 -- 172.21.15.133:0/1777594645 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9cf8007520 con 0x7f9ce8003a10 2024-04-25T10:15:21.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.458+0000 7f9cf2ffd640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce0067100 0x7f9ce00695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.458+0000 7f9cf2ffd640 1 -- 172.21.15.133:0/1777594645 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9cf811e820 con 0x7f9ce8003a10 2024-04-25T10:15:21.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.458+0000 7f9d01b08640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce0067100 0x7f9ce00695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.458+0000 7f9d01b08640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce0067100 0x7f9ce00695c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc1f7900 tx=0x7f9cec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.462+0000 7f9cf2ffd640 1 -- 172.21.15.133:0/1777594645 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9cf80bf030 con 0x7f9ce8003a10 2024-04-25T10:15:21.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.654+0000 7f9d0456d640 1 -- 172.21.15.133:0/1777594645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9cfc0b5350 con 0x7f9ce8003a10 2024-04-25T10:15:21.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9cf2ffd640 1 -- 172.21.15.133:0/1777594645 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9cf8016830 con 0x7f9ce8003a10 2024-04-25T10:15:21.657 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:21.657 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:15:21.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 -- 172.21.15.133:0/1777594645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce0067100 msgr2=0x7f9ce00695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce0067100 0x7f9ce00695c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc1f7900 tx=0x7f9cec009040 comp rx=0 tx=0).stop 2024-04-25T10:15:21.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 -- 172.21.15.133:0/1777594645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 msgr2=0x7f9cfc0c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 0x7f9cfc0c9840 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800c160 tx=0x7f9ce8005840 comp rx=0 tx=0).stop 2024-04-25T10:15:21.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 -- 172.21.15.133:0/1777594645 shutdown_connections 2024-04-25T10:15:21.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cfc163300 0x7f9cfc15fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cfc1f74c0 0x7f9cfc15f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9ce0067100 0x7f9ce00695c0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 --2- 172.21.15.133:0/1777594645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ce8003a10 0x7f9cfc0c9840 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 -- 172.21.15.133:0/1777594645 >> 172.21.15.133:0/1777594645 conn(0x7f9cfc09aa10 msgr2=0x7f9cfc0701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 -- 172.21.15.133:0/1777594645 shutdown_connections 2024-04-25T10:15:21.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.658+0000 7f9d0456d640 1 -- 172.21.15.133:0/1777594645 wait complete. 2024-04-25T10:15:21.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:21.679 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub start '~mdsdir' recursive 2024-04-25T10:15:21.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f928f10f640 1 Processor -- start 2024-04-25T10:15:21.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f928f10f640 1 -- start start 2024-04-25T10:15:21.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f928f10f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9288146f20 0x7f928806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f928f10f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92880cdc40 con 0x7f928806aff0 2024-04-25T10:15:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f928f10f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928806bc10 con 0x7f928806dbf0 2024-04-25T10:15:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f928f10f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928806e020 con 0x7f9288146f20 2024-04-25T10:15:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f928ceab640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f928806aff0 0x7f928806b3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58640/0 (socket says 172.21.15.133:58640) 2024-04-25T10:15:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f928ceab640 1 -- 172.21.15.133:0/646033755 learned_addr learned my addr 172.21.15.133:0/646033755 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f9287fff640 1 --2- 172.21.15.133:0/646033755 >> v2:172.21.15.133:3300/0 conn(0x7f9288146f20 0x7f928806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f92877fe640 1 -- 172.21.15.133:0/646033755 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643935694 0 0) 0x7f92880cdc40 con 0x7f928806aff0 2024-04-25T10:15:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f92877fe640 1 -- 172.21.15.133:0/646033755 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9274003530 con 0x7f928806aff0 2024-04-25T10:15:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f9287fff640 1 -- 172.21.15.133:0/646033755 >> v1:172.21.15.162:6789/0 conn(0x7f928806dbf0 legacy=0x7f92880ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f9287fff640 1 -- 172.21.15.133:0/646033755 >> v1:172.21.15.133:6789/0 conn(0x7f928806aff0 legacy=0x7f928806b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.806+0000 7f9287fff640 1 -- 172.21.15.133:0/646033755 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928806bef0 con 0x7f9288146f20 2024-04-25T10:15:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f9287fff640 1 --2- 172.21.15.133:0/646033755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9288146f20 0x7f928806d6b0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f92780097c0 tx=0x7f927801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f471f2aa782131a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f92877fe640 1 -- 172.21.15.133:0/646033755 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780b1070 con 0x7f9288146f20 2024-04-25T10:15:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f92877fe640 1 -- 172.21.15.133:0/646033755 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92780a8050 con 0x7f9288146f20 2024-04-25T10:15:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f92877fe640 1 -- 172.21.15.133:0/646033755 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780ac860 con 0x7f9288146f20 2024-04-25T10:15:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f928f10f640 1 -- 172.21.15.133:0/646033755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9288146f20 msgr2=0x7f928806d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f928f10f640 1 --2- 172.21.15.133:0/646033755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9288146f20 0x7f928806d6b0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f92780097c0 tx=0x7f927801c3e0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f928f10f640 1 -- 172.21.15.133:0/646033755 shutdown_connections 2024-04-25T10:15:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f928f10f640 1 --2- 172.21.15.133:0/646033755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9288146f20 0x7f928806d6b0 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f928f10f640 1 -- 172.21.15.133:0/646033755 >> 172.21.15.133:0/646033755 conn(0x7f92880cc7c0 msgr2=0x7f92880ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f928f10f640 1 -- 172.21.15.133:0/646033755 shutdown_connections 2024-04-25T10:15:21.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f928f10f640 1 -- 172.21.15.133:0/646033755 wait complete. 2024-04-25T10:15:21.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.810+0000 7f928f10f640 1 Processor -- start 2024-04-25T10:15:21.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928f10f640 1 -- start start 2024-04-25T10:15:21.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928f10f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f928806aff0 0x7f9288156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928f10f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f928806dbf0 0x7f92880b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928f10f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9288146f20 0x7f9288146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928f10f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92880cf040 con 0x7f928806aff0 2024-04-25T10:15:21.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928f10f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92881475c0 con 0x7f928806dbf0 2024-04-25T10:15:21.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928f10f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92880cf1c0 con 0x7f9288146f20 2024-04-25T10:15:21.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928ceab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f928806aff0 0x7f9288156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928ceab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f928806aff0 0x7f9288156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54294/0 (socket says 172.21.15.133:54294) 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928ceab640 1 -- 172.21.15.133:0/2184350900 learned_addr learned my addr 172.21.15.133:0/2184350900 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f9287fff640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f928806dbf0 0x7f92880b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928d6ac640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9288146f20 0x7f9288146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928ceab640 1 -- 172.21.15.133:0/2184350900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9288146f20 msgr2=0x7f9288146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928ceab640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9288146f20 0x7f9288146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928ceab640 1 -- 172.21.15.133:0/2184350900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f928806dbf0 msgr2=0x7f92880b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928ceab640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f928806dbf0 0x7f92880b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928ceab640 1 -- 172.21.15.133:0/2184350900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928815e3e0 con 0x7f928806aff0 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f9287fff640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f928806dbf0 0x7f92880b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928ceab640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f928806aff0 0x7f9288156be0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f928000cbb0 tx=0x7f928000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f92857fa640 1 -- 172.21.15.133:0/2184350900 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280017070 con 0x7f928806aff0 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f92857fa640 1 -- 172.21.15.133:0/2184350900 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928000ad40 con 0x7f928806aff0 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928f10f640 1 -- 172.21.15.133:0/2184350900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92880b3f30 con 0x7f928806aff0 2024-04-25T10:15:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f92857fa640 1 -- 172.21.15.133:0/2184350900 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280012600 con 0x7f928806aff0 2024-04-25T10:15:21.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f928f10f640 1 -- 172.21.15.133:0/2184350900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92880b44f0 con 0x7f928806aff0 2024-04-25T10:15:21.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f92857fa640 1 -- 172.21.15.133:0/2184350900 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f928001a070 con 0x7f928806aff0 2024-04-25T10:15:21.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f92857fa640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9268066f60 0x7f9268069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f92857fa640 1 -- 172.21.15.133:0/2184350900 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9280084ec0 con 0x7f928806aff0 2024-04-25T10:15:21.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.814+0000 7f9287fff640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9268066f60 0x7f9268069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f9287fff640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9268066f60 0x7f9268069420 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f92780ae000 tx=0x7f92780a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 Processor -- start 2024-04-25T10:15:21.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 -- start start 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x5569525ecca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569523f0dd0 con 0x5569525a2650 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556952379010 con 0x5569525ec8c0 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569523ece70 con 0x5569525ed1e0 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f928d6ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f928d6ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54310/0 (socket says 172.21.15.133:54310) 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f928d6ac640 1 -- 172.21.15.133:0/2144159056 learned_addr learned my addr 172.21.15.133:0/2144159056 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f9287fff640 1 --2- 172.21.15.133:0/2144159056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f928ceab640 1 --2- 172.21.15.133:0/2144159056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x5569525ecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f928d6ac640 1 -- 172.21.15.133:0/2144159056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 msgr2=0x5569525a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f928d6ac640 1 --2- 172.21.15.133:0/2144159056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f928d6ac640 1 -- 172.21.15.133:0/2144159056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 msgr2=0x5569525ecca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f928d6ac640 1 --2- 172.21.15.133:0/2144159056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x5569525ecca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f928d6ac640 1 -- 172.21.15.133:0/2144159056 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569523a99c0 con 0x5569525a2650 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f928ceab640 1 --2- 172.21.15.133:0/2144159056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x5569525ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f928d6ac640 1 --2- 172.21.15.133:0/2144159056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a2a30 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f9270001540 tx=0x7f9270001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c8a7b0c1d285dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f927e7fc640 1 -- 172.21.15.133:0/2144159056 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270019070 con 0x5569525a2650 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f927e7fc640 1 -- 172.21.15.133:0/2144159056 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92700050d0 con 0x5569525a2650 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f927e7fc640 1 -- 172.21.15.133:0/2144159056 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92700054a0 con 0x5569525a2650 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 -- 172.21.15.133:0/2144159056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 msgr2=0x5569525a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 --2- 172.21.15.133:0/2144159056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a2a30 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f9270001540 tx=0x7f9270001a00 comp rx=0 tx=0).stop 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 -- 172.21.15.133:0/2144159056 shutdown_connections 2024-04-25T10:15:21.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 --2- 172.21.15.133:0/2144159056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a2a30 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 --2- 172.21.15.133:0/2144159056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 --2- 172.21.15.133:0/2144159056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x5569525ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 -- 172.21.15.133:0/2144159056 >> 172.21.15.133:0/2144159056 conn(0x55695248a870 msgr2=0x5569525a09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 -- 172.21.15.133:0/2144159056 shutdown_connections 2024-04-25T10:15:21.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 -- 172.21.15.133:0/2144159056 wait complete. 2024-04-25T10:15:21.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 -- start start 2024-04-25T10:15:21.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.818+0000 7f929227c000 1 Processor -- start 2024-04-25T10:15:21.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x5569525d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569523ab420 con 0x5569525a2650 2024-04-25T10:15:21.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569523f00e0 con 0x5569525ec8c0 2024-04-25T10:15:21.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556952473470 con 0x5569525ed1e0 2024-04-25T10:15:21.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 10 client.0 _ll_register_callbacks cb 0x55695258df90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:21.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f928ceab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f928ceab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54326/0 (socket says 172.21.15.133:54326) 2024-04-25T10:15:21.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f928ceab640 1 -- 172.21.15.133:0/804971550 learned_addr learned my addr 172.21.15.133:0/804971550 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f928d6ac640 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f9287fff640 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x5569525d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f928ceab640 1 -- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 msgr2=0x5569525d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f928ceab640 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525d0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f928ceab640 1 -- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 msgr2=0x5569525d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f928ceab640 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x5569525d08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f928ceab640 1 -- 172.21.15.133:0/804971550 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556952379260 con 0x5569525a2650 2024-04-25T10:15:21.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f928d6ac640 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:21.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f928ceab640 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a7c10 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f928000cfd0 tx=0x7f928005dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85053e0c22466775 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f92677fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f92677fe640 1 -- 172.21.15.133:0/804971550 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928006d1d0 con 0x5569525a2650 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 1 -- 172.21.15.133:0/804971550 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92700050d0 con 0x5569525a2650 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f92677fe640 1 -- 172.21.15.133:0/804971550 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f928007cbc0 con 0x5569525a2650 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f92677fe640 1 -- 172.21.15.133:0/804971550 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f928007cdc0 con 0x5569525a2650 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 10 client.10755 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 1 -- 172.21.15.133:0/804971550 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5569523abb30 con 0x5569525a2650 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f92677fe640 1 -- 172.21.15.133:0/804971550 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92700050d0 con 0x5569525a2650 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f92677fe640 1 -- 172.21.15.133:0/804971550 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9280004b70 con 0x5569525a2650 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 10 client.10755 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 10 client.10755 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 20 client.10755 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5569525f3af0 0x5569525f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 4 client.10755 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f929227c000 1 -- 172.21.15.133:0/804971550 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55695238cc80 con 0x5569525f3af0 2024-04-25T10:15:21.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f9287fff640 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5569525f3af0 0x5569525f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f9287fff640 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5569525f3af0 0x5569525f3ed0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f829322e509a1da1 server_cookie=5774a6fa792bd437 in_seq=0 out_seq=0 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f92677fe640 10 client.10755 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f92677fe640 1 -- 172.21.15.133:0/804971550 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55695238cc80 con 0x5569525f3af0 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.822+0000 7f92677fe640 10 client.10755 handle_command_reply: tid=0 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 1 client.10755 shutdown 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 1 -- 172.21.15.133:0/804971550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 msgr2=0x5569525a7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a7c10 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f928000cfd0 tx=0x7f928005dd50 comp rx=0 tx=0).stop 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 1 -- 172.21.15.133:0/804971550 shutdown_connections 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5569525f3af0 0x5569525f3ed0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x5569525d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 1 --2- 172.21.15.133:0/804971550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569525a7c10 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f92677fe640 0 client.10755 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 1 -- 172.21.15.133:0/804971550 >> 172.21.15.133:0/804971550 conn(0x55695248a870 msgr2=0x5569525a09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 1 -- 172.21.15.133:0/804971550 shutdown_connections 2024-04-25T10:15:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 1 -- 172.21.15.133:0/804971550 wait complete. 2024-04-25T10:15:21.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.826+0000 7f929227c000 20 client.10755 trim_cache size 0 max 16384 2024-04-25T10:15:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.838+0000 7f929227c000 1 Processor -- start 2024-04-25T10:15:21.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.838+0000 7f929227c000 1 -- start start 2024-04-25T10:15:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569526486c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ec8c0 0x556952648c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ed1e0 0x556952649140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288202cc0 con 0x5569525a2650 2024-04-25T10:15:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288201ec0 con 0x5569525ed1e0 2024-04-25T10:15:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92881610c0 con 0x5569525ec8c0 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569526486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569526486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54330/0 (socket says 172.21.15.133:54330) 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 -- 172.21.15.133:0/276530046 learned_addr learned my addr 172.21.15.133:0/276530046 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 -- 172.21.15.133:0/276530046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ec8c0 msgr2=0x556952648c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 --2- 172.21.15.133:0/276530046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ec8c0 0x556952648c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 -- 172.21.15.133:0/276530046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ed1e0 msgr2=0x556952649140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 --2- 172.21.15.133:0/276530046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ed1e0 0x556952649140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 -- 172.21.15.133:0/276530046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569524766d0 con 0x5569525a2650 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 --2- 172.21.15.133:0/276530046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569526486c0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f92800012f0 tx=0x7f9280057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666ef44863152620 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f927dffb640 1 -- 172.21.15.133:0/276530046 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280051990 con 0x5569525a2650 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f927dffb640 1 -- 172.21.15.133:0/276530046 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928007cbc0 con 0x5569525a2650 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f927dffb640 1 -- 172.21.15.133:0/276530046 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280004b70 con 0x5569525a2650 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- 172.21.15.133:0/276530046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 msgr2=0x5569526486c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 --2- 172.21.15.133:0/276530046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569526486c0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f92800012f0 tx=0x7f9280057000 comp rx=0 tx=0).stop 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- 172.21.15.133:0/276530046 shutdown_connections 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 --2- 172.21.15.133:0/276530046 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ed1e0 0x556952649140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 --2- 172.21.15.133:0/276530046 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ec8c0 0x556952648c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 --2- 172.21.15.133:0/276530046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x5569526486c0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- 172.21.15.133:0/276530046 >> 172.21.15.133:0/276530046 conn(0x55695248a870 msgr2=0x5569525a63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- 172.21.15.133:0/276530046 shutdown_connections 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- 172.21.15.133:0/276530046 wait complete. 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- start start 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 Processor -- start 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x55695264d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x55695264d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525f0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569523f5400 con 0x5569525a2650 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92881616c0 con 0x5569525ec8c0 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556952397ae0 con 0x5569525ed1e0 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 10 client.0 _ll_register_callbacks cb 0x55695238d200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x55695264d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x55695264d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54338/0 (socket says 172.21.15.133:54338) 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 -- 172.21.15.133:0/4184123089 learned_addr learned my addr 172.21.15.133:0/4184123089 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f9287fff640 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x55695264d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 -- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 msgr2=0x5569525f0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928d6ac640 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525f0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525f0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 -- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 msgr2=0x55695264d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x55695264d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 -- 172.21.15.133:0/4184123089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556952473900 con 0x5569525a2650 2024-04-25T10:15:21.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f928ceab640 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x55695264d490 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f928000cfd0 tx=0x7f9280083130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdb570ccaf32efc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f92677fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f92677fe640 1 -- 172.21.15.133:0/4184123089 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280081070 con 0x5569525a2650 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f92677fe640 1 -- 172.21.15.133:0/4184123089 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9280051440 con 0x5569525a2650 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f929227c000 1 -- 172.21.15.133:0/4184123089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f928007cbc0 con 0x5569525a2650 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.842+0000 7f92677fe640 1 -- 172.21.15.133:0/4184123089 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9280054e20 con 0x5569525a2650 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 10 client.10761 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 -- 172.21.15.133:0/4184123089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x556952397f20 con 0x5569525a2650 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f92677fe640 1 -- 172.21.15.133:0/4184123089 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928007cbc0 con 0x5569525a2650 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f92677fe640 1 -- 172.21.15.133:0/4184123089 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9280051640 con 0x5569525a2650 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 10 client.10761 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 10 client.10761 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 20 client.10761 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5569525f14d0 0x5569525f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 4 client.10761 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 -- 172.21.15.133:0/4184123089 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x5569523fe310 con 0x5569525f14d0 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f9287fff640 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5569525f14d0 0x5569525f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f9287fff640 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5569525f14d0 0x5569525f18e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27557dbd1e0ebb47 server_cookie=d2a7ae1fb295dc2c in_seq=0 out_seq=0 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f92677fe640 10 client.10761 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f92677fe640 1 -- 172.21.15.133:0/4184123089 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5569523fe310 con 0x5569525f14d0 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f92677fe640 10 client.10761 handle_command_reply: tid=0 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 client.10761 shutdown 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 -- 172.21.15.133:0/4184123089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 msgr2=0x55695264d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x55695264d490 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f928000cfd0 tx=0x7f9280083130 comp rx=0 tx=0).stop 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 -- 172.21.15.133:0/4184123089 shutdown_connections 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5569525f14d0 0x5569525f18e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5569525ed1e0 0x5569525f0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569525ec8c0 0x55695264d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f92677fe640 0 client.10761 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 --2- 172.21.15.133:0/4184123089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5569525a2650 0x55695264d490 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 -- 172.21.15.133:0/4184123089 >> 172.21.15.133:0/4184123089 conn(0x55695248a870 msgr2=0x5569525a6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 -- 172.21.15.133:0/4184123089 shutdown_connections 2024-04-25T10:15:21.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 1 -- 172.21.15.133:0/4184123089 wait complete. 2024-04-25T10:15:21.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f929227c000 20 client.10761 trim_cache size 0 max 16384 2024-04-25T10:15:21.856 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:15:21.856 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:15:21.856 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "2f105e50-67eb-4381-9e1d-a150de070c13", 2024-04-25T10:15:21.856 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:15:21.856 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:15:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 -- 172.21.15.133:0/2184350900 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9268066f60 msgr2=0x7f9268069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9268066f60 0x7f9268069420 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f92780ae000 tx=0x7f92780a7000 comp rx=0 tx=0).stop 2024-04-25T10:15:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 -- 172.21.15.133:0/2184350900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f928806aff0 msgr2=0x7f9288156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f928806aff0 0x7f9288156be0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f928000cbb0 tx=0x7f928000a430 comp rx=0 tx=0).stop 2024-04-25T10:15:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 -- 172.21.15.133:0/2184350900 shutdown_connections 2024-04-25T10:15:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9288146f20 0x7f9288146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f928806dbf0 0x7f92880b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9268066f60 0x7f9268069420 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 --2- 172.21.15.133:0/2184350900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f928806aff0 0x7f9288156be0 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 -- 172.21.15.133:0/2184350900 >> 172.21.15.133:0/2184350900 conn(0x7f92880cc7c0 msgr2=0x7f92880ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 -- 172.21.15.133:0/2184350900 shutdown_connections 2024-04-25T10:15:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.846+0000 7f928f10f640 1 -- 172.21.15.133:0/2184350900 wait complete. 2024-04-25T10:15:21.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:21.864 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:15:21.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.978+0000 7f6733ebd640 1 Processor -- start 2024-04-25T10:15:21.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.978+0000 7f6733ebd640 1 -- start start 2024-04-25T10:15:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672c145170 0x7f672c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f672c06e4d0 0x7f672c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672c0ce2e0 con 0x7f672c0cdd60 2024-04-25T10:15:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672c06db00 con 0x7f672c06e4d0 2024-04-25T10:15:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c145550 con 0x7f672c145170 2024-04-25T10:15:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6731458640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f672c06e4d0 0x7f672c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f673245a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f672c0cdd60 0x7f672c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58650/0 (socket says 172.21.15.133:58650) 2024-04-25T10:15:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f673245a640 1 -- 172.21.15.133:0/1334753120 learned_addr learned my addr 172.21.15.133:0/1334753120 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6731c59640 1 --2- 172.21.15.133:0/1334753120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672c145170 0x7f672c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6731458640 1 -- 172.21.15.133:0/1334753120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672c145170 msgr2=0x7f672c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6731458640 1 --2- 172.21.15.133:0/1334753120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672c145170 0x7f672c06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6731458640 1 -- 172.21.15.133:0/1334753120 >> v1:172.21.15.133:6789/0 conn(0x7f672c0cdd60 legacy=0x7f672c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6730c57640 1 -- 172.21.15.133:0/1334753120 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812630853 0 0) 0x7f672c0ce2e0 con 0x7f672c0cdd60 2024-04-25T10:15:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6731458640 1 -- 172.21.15.133:0/1334753120 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c06d4b0 con 0x7f672c06e4d0 2024-04-25T10:15:21.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6731c59640 1 --2- 172.21.15.133:0/1334753120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672c145170 0x7f672c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:21.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6731458640 1 --2- 172.21.15.133:0/1334753120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f672c06e4d0 0x7f672c0cd820 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f671c0097c0 tx=0x7f671c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a568527a326b4b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6730c57640 1 -- 172.21.15.133:0/1334753120 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0b0070 con 0x7f672c06e4d0 2024-04-25T10:15:21.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6730c57640 1 -- 172.21.15.133:0/1334753120 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c0ab600 con 0x7f672c06e4d0 2024-04-25T10:15:21.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6730c57640 1 -- 172.21.15.133:0/1334753120 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0ab9d0 con 0x7f672c06e4d0 2024-04-25T10:15:21.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 -- 172.21.15.133:0/1334753120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f672c06e4d0 msgr2=0x7f672c0cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 --2- 172.21.15.133:0/1334753120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f672c06e4d0 0x7f672c0cd820 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f671c0097c0 tx=0x7f671c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 -- 172.21.15.133:0/1334753120 shutdown_connections 2024-04-25T10:15:21.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 --2- 172.21.15.133:0/1334753120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f672c06e4d0 0x7f672c0cd820 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 --2- 172.21.15.133:0/1334753120 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672c145170 0x7f672c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 -- 172.21.15.133:0/1334753120 >> 172.21.15.133:0/1334753120 conn(0x7f672c066f70 msgr2=0x7f672c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 -- 172.21.15.133:0/1334753120 shutdown_connections 2024-04-25T10:15:21.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.982+0000 7f6733ebd640 1 -- 172.21.15.133:0/1334753120 wait complete. 2024-04-25T10:15:21.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f6733ebd640 1 Processor -- start 2024-04-25T10:15:21.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f6733ebd640 1 -- start start 2024-04-25T10:15:21.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f6733ebd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f672c06e4d0 0x7f672c13cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f6733ebd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672c0cdd60 0x7f672c163210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f6733ebd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f672c145170 0x7f672c163750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f6733ebd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c168490 con 0x7f672c145170 2024-04-25T10:15:21.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f6733ebd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c15d7f0 con 0x7f672c06e4d0 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f6733ebd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c168610 con 0x7f672c0cdd60 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f673245a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f672c145170 0x7f672c163750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f673245a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f672c145170 0x7f672c163750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54352/0 (socket says 172.21.15.133:54352) 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f673245a640 1 -- 172.21.15.133:0/1669056952 learned_addr learned my addr 172.21.15.133:0/1669056952 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f673245a640 1 -- 172.21.15.133:0/1669056952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672c0cdd60 msgr2=0x7f672c163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f673245a640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672c0cdd60 0x7f672c163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f673245a640 1 -- 172.21.15.133:0/1669056952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f672c06e4d0 msgr2=0x7f672c13cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f6731c59640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f672c06e4d0 0x7f672c13cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f673245a640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f672c06e4d0 0x7f672c13cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f673245a640 1 -- 172.21.15.133:0/1669056952 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c1641d0 con 0x7f672c145170 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f673245a640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f672c145170 0x7f672c163750 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f672400c7a0 tx=0x7f672400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f67227fc640 1 -- 172.21.15.133:0/1669056952 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724018070 con 0x7f672c145170 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f67227fc640 1 -- 172.21.15.133:0/1669056952 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672400a640 con 0x7f672c145170 2024-04-25T10:15:21.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f6733ebd640 1 -- 172.21.15.133:0/1669056952 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672c161830 con 0x7f672c145170 2024-04-25T10:15:21.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f67227fc640 1 -- 172.21.15.133:0/1669056952 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400aab0 con 0x7f672c145170 2024-04-25T10:15:21.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.986+0000 7f6733ebd640 1 -- 172.21.15.133:0/1669056952 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c161df0 con 0x7f672c145170 2024-04-25T10:15:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f67227fc640 1 -- 172.21.15.133:0/1669056952 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f672401b070 con 0x7f672c145170 2024-04-25T10:15:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f67227fc640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6710066f60 0x7f6710069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f67227fc640 1 -- 172.21.15.133:0/1669056952 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6724084850 con 0x7f672c145170 2024-04-25T10:15:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f6731c59640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6710066f60 0x7f6710069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f6731c59640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6710066f60 0x7f6710069420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6728006fd0 tx=0x7f6728008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 Processor -- start 2024-04-25T10:15:21.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 -- start start 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d606db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 0x55e42d5bcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5bd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42d409ff0 con 0x55e42d5bd0a0 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42d392150 con 0x55e42d6072f0 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e42d406090 con 0x55e42d6069d0 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673245a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5bd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673245a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5bd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54360/0 (socket says 172.21.15.133:54360) 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673245a640 1 -- 172.21.15.133:0/3780839137 learned_addr learned my addr 172.21.15.133:0/3780839137 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f6731c59640 1 --2- 172.21.15.133:0/3780839137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d606db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f6731458640 1 --2- 172.21.15.133:0/3780839137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 0x55e42d5bcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673245a640 1 -- 172.21.15.133:0/3780839137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 msgr2=0x55e42d606db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673245a640 1 --2- 172.21.15.133:0/3780839137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d606db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673245a640 1 -- 172.21.15.133:0/3780839137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 msgr2=0x55e42d5bcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673245a640 1 --2- 172.21.15.133:0/3780839137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 0x55e42d5bcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673245a640 1 -- 172.21.15.133:0/3780839137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e42d5a00d0 con 0x55e42d5bd0a0 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673245a640 1 --2- 172.21.15.133:0/3780839137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5bd480 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f6724001340 tx=0x7f6724055770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d43310637aeea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f670f7fe640 1 -- 172.21.15.133:0/3780839137 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672405d640 con 0x55e42d5bd0a0 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f670f7fe640 1 -- 172.21.15.133:0/3780839137 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6724053230 con 0x55e42d5bd0a0 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f670f7fe640 1 -- 172.21.15.133:0/3780839137 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400ac50 con 0x55e42d5bd0a0 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 -- 172.21.15.133:0/3780839137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 msgr2=0x55e42d5bd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 --2- 172.21.15.133:0/3780839137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5bd480 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f6724001340 tx=0x7f6724055770 comp rx=0 tx=0).stop 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 -- 172.21.15.133:0/3780839137 shutdown_connections 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 --2- 172.21.15.133:0/3780839137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5bd480 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 --2- 172.21.15.133:0/3780839137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 0x55e42d5bcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 --2- 172.21.15.133:0/3780839137 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d606db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:21.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 -- 172.21.15.133:0/3780839137 >> 172.21.15.133:0/3780839137 conn(0x55e42d5b9020 msgr2=0x55e42d5bb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:21.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 -- 172.21.15.133:0/3780839137 shutdown_connections 2024-04-25T10:15:21.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.990+0000 7f673702a000 1 -- 172.21.15.133:0/3780839137 wait complete. 2024-04-25T10:15:21.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 -- start start 2024-04-25T10:15:21.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 Processor -- start 2024-04-25T10:15:22.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d5ebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 0x55e42d5ec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42d3ce250 con 0x55e42d5bd0a0 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42d3c4690 con 0x55e42d6072f0 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e42d392a40 con 0x55e42d6069d0 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 10 client.0 _ll_register_callbacks cb 0x55e42d5a7e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731c59640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731c59640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5eb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54376/0 (socket says 172.21.15.133:54376) 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731c59640 1 -- 172.21.15.133:0/2653503378 learned_addr learned my addr 172.21.15.133:0/2653503378 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731458640 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d5ebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673245a640 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 0x55e42d5ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731c59640 1 -- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 msgr2=0x55e42d5ebd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731c59640 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d5ebd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731c59640 1 -- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 msgr2=0x55e42d5ec270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731c59640 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 0x55e42d5ec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731c59640 1 -- 172.21.15.133:0/2653503378 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e42d4b71c0 con 0x55e42d5bd0a0 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731c59640 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5eb7f0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f6728014720 tx=0x7f6728004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a1b3068d5a6617 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f670cff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f670cff9640 1 -- 172.21.15.133:0/2653503378 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728020460 con 0x55e42d5bd0a0 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f670cff9640 1 -- 172.21.15.133:0/2653503378 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6728020600 con 0x55e42d5bd0a0 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 -- 172.21.15.133:0/2653503378 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6724053230 con 0x55e42d5bd0a0 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f670cff9640 1 -- 172.21.15.133:0/2653503378 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f672802d050 con 0x55e42d5bd0a0 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 10 client.10776 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:22.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 -- 172.21.15.133:0/2653503378 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55e42d4b3170 con 0x55e42d5bd0a0 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f670cff9640 1 -- 172.21.15.133:0/2653503378 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6724053230 con 0x55e42d5bd0a0 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f670cff9640 1 -- 172.21.15.133:0/2653503378 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6728030020 con 0x55e42d5bd0a0 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 10 client.10776 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 10 client.10776 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 20 client.10776 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55e42d616c80 0x55e42d619070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 4 client.10776 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f673702a000 1 -- 172.21.15.133:0/2653503378 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e42d3a5c60 con 0x55e42d616c80 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731458640 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55e42d616c80 0x55e42d619070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f670cff9640 10 client.10776 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.994+0000 7f6731458640 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55e42d616c80 0x55e42d619070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d13206131194e190 server_cookie=466adbe5096b916 in_seq=0 out_seq=0 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f670cff9640 1 -- 172.21.15.133:0/2653503378 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e42d3a5c60 con 0x55e42d616c80 2024-04-25T10:15:22.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f670cff9640 10 client.10776 handle_command_reply: tid=0 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 1 client.10776 shutdown 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 1 -- 172.21.15.133:0/2653503378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 msgr2=0x55e42d5eb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5eb7f0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f6728014720 tx=0x7f6728004740 comp rx=0 tx=0).stop 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 1 -- 172.21.15.133:0/2653503378 shutdown_connections 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 0x55e42d5ec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55e42d616c80 0x55e42d619070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d5ebd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 1 --2- 172.21.15.133:0/2653503378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d5bd0a0 0x55e42d5eb7f0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 1 -- 172.21.15.133:0/2653503378 >> 172.21.15.133:0/2653503378 conn(0x55e42d5b9020 msgr2=0x55e42d5bbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f670cff9640 0 client.10776 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 1 -- 172.21.15.133:0/2653503378 shutdown_connections 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 1 -- 172.21.15.133:0/2653503378 wait complete. 2024-04-25T10:15:22.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:21.998+0000 7f673702a000 20 client.10776 trim_cache size 0 max 16384 2024-04-25T10:15:22.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.010+0000 7f673702a000 1 Processor -- start 2024-04-25T10:15:22.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- start start 2024-04-25T10:15:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d5bd0a0 0x55e42d6630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6069d0 0x55e42d663630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 0x55e42d663b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c201200 con 0x55e42d6069d0 2024-04-25T10:15:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c2003a0 con 0x55e42d6072f0 2024-04-25T10:15:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c13c2e0 con 0x55e42d5bd0a0 2024-04-25T10:15:22.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f6731458640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6069d0 0x55e42d663630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f6731458640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6069d0 0x55e42d663630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54384/0 (socket says 172.21.15.133:54384) 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f6731458640 1 -- 172.21.15.133:0/694165125 learned_addr learned my addr 172.21.15.133:0/694165125 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f6731458640 1 -- 172.21.15.133:0/694165125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d5bd0a0 msgr2=0x55e42d6630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f6731458640 1 --2- 172.21.15.133:0/694165125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d5bd0a0 0x55e42d6630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f6731458640 1 -- 172.21.15.133:0/694165125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 msgr2=0x55e42d663b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f6731458640 1 --2- 172.21.15.133:0/694165125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 0x55e42d663b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f6731458640 1 -- 172.21.15.133:0/694165125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e42d450de0 con 0x55e42d6069d0 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f6731458640 1 --2- 172.21.15.133:0/694165125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6069d0 0x55e42d663630 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f671c000c10 tx=0x7f671c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470cc6009304cd1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f670effd640 1 -- 172.21.15.133:0/694165125 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0c1030 con 0x55e42d6069d0 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f670effd640 1 -- 172.21.15.133:0/694165125 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c0ab780 con 0x55e42d6069d0 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f670effd640 1 -- 172.21.15.133:0/694165125 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0b0040 con 0x55e42d6069d0 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- 172.21.15.133:0/694165125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6069d0 msgr2=0x55e42d663630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 --2- 172.21.15.133:0/694165125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6069d0 0x55e42d663630 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f671c000c10 tx=0x7f671c004740 comp rx=0 tx=0).stop 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- 172.21.15.133:0/694165125 shutdown_connections 2024-04-25T10:15:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 --2- 172.21.15.133:0/694165125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d6072f0 0x55e42d663b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 --2- 172.21.15.133:0/694165125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6069d0 0x55e42d663630 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 --2- 172.21.15.133:0/694165125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d5bd0a0 0x55e42d6630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- 172.21.15.133:0/694165125 >> 172.21.15.133:0/694165125 conn(0x55e42d5b9020 msgr2=0x55e42d613fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- 172.21.15.133:0/694165125 shutdown_connections 2024-04-25T10:15:22.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- 172.21.15.133:0/694165125 wait complete. 2024-04-25T10:15:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- start start 2024-04-25T10:15:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 Processor -- start 2024-04-25T10:15:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d5bd0a0 0x55e42d618fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d6194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6072f0 0x55e42d619a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42d3a4e00 con 0x55e42d6072f0 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c1ffda0 con 0x55e42d5bd0a0 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e42d409300 con 0x55e42d6069d0 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673702a000 10 client.0 _ll_register_callbacks cb 0x55e42d588040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673245a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6072f0 0x55e42d619a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673245a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6072f0 0x55e42d619a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54400/0 (socket says 172.21.15.133:54400) 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673245a640 1 -- 172.21.15.133:0/2955210868 learned_addr learned my addr 172.21.15.133:0/2955210868 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f6731458640 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d6194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f6731c59640 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d5bd0a0 0x55e42d618fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673245a640 1 -- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 msgr2=0x55e42d6194e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673245a640 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d6194e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673245a640 1 -- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d5bd0a0 msgr2=0x55e42d618fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673245a640 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d5bd0a0 0x55e42d618fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.014+0000 7f673245a640 1 -- 172.21.15.133:0/2955210868 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e42d3ab8e0 con 0x55e42d6072f0 2024-04-25T10:15:22.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673245a640 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6072f0 0x55e42d619a20 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f6724054c30 tx=0x7f6724082a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4991f370f7a2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f670cff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f670cff9640 1 -- 172.21.15.133:0/2955210868 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724071060 con 0x55e42d6072f0 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f670cff9640 1 -- 172.21.15.133:0/2955210868 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f672405b070 con 0x55e42d6072f0 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 -- 172.21.15.133:0/2955210868 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f671c0ab780 con 0x55e42d6072f0 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f670cff9640 1 -- 172.21.15.133:0/2955210868 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6724054d00 con 0x55e42d6072f0 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 10 client.10782 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 -- 172.21.15.133:0/2955210868 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55e42d4638f0 con 0x55e42d6072f0 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f670cff9640 1 -- 172.21.15.133:0/2955210868 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c0ab780 con 0x55e42d6072f0 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f670cff9640 1 -- 172.21.15.133:0/2955210868 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6724054380 con 0x55e42d6072f0 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 10 client.10782 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 10 client.10782 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T10:15:22.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 20 client.10782 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55e42d61a350 0x55e42d66ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 4 client.10782 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 -- 172.21.15.133:0/2955210868 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e42d417530 con 0x55e42d61a350 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6731c59640 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55e42d61a350 0x55e42d66ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f670cff9640 10 client.10782 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6731c59640 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55e42d61a350 0x55e42d66ef60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3eb98d2715099ae server_cookie=5e4617841e4d011a in_seq=0 out_seq=0 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f670cff9640 1 -- 172.21.15.133:0/2955210868 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+257 (crc 0 0 0) 0x55e42d417530 con 0x55e42d61a350 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f670cff9640 10 client.10782 handle_command_reply: tid=0 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 client.10782 shutdown 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 -- 172.21.15.133:0/2955210868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6072f0 msgr2=0x55e42d619a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6072f0 0x55e42d619a20 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f6724054c30 tx=0x7f6724082a20 comp rx=0 tx=0).stop 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 -- 172.21.15.133:0/2955210868 shutdown_connections 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e42d6072f0 0x55e42d619a20 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x55e42d61a350 0x55e42d66ef60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55e42d6069d0 0x55e42d6194e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 --2- 172.21.15.133:0/2955210868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55e42d5bd0a0 0x55e42d618fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f670cff9640 0 client.10782 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 -- 172.21.15.133:0/2955210868 >> 172.21.15.133:0/2955210868 conn(0x55e42d5b9020 msgr2=0x55e42d6050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 -- 172.21.15.133:0/2955210868 shutdown_connections 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 1 -- 172.21.15.133:0/2955210868 wait complete. 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f673702a000 20 client.10782 trim_cache size 0 max 16384 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stdout: "2f105e50-67eb-4381-9e1d-a150de070c13": { 2024-04-25T10:15:22.024 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "~mds0", 2024-04-25T10:15:22.025 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "2f105e50-67eb-4381-9e1d-a150de070c13", 2024-04-25T10:15:22.025 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive" 2024-04-25T10:15:22.025 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:22.025 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:15:22.025 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 -- 172.21.15.133:0/1669056952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6710066f60 msgr2=0x7f6710069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6710066f60 0x7f6710069420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6728006fd0 tx=0x7f6728008040 comp rx=0 tx=0).stop 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 -- 172.21.15.133:0/1669056952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f672c145170 msgr2=0x7f672c163750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f672c145170 0x7f672c163750 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f672400c7a0 tx=0x7f672400cc60 comp rx=0 tx=0).stop 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 -- 172.21.15.133:0/1669056952 shutdown_connections 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6710066f60 0x7f6710069420 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f672c145170 0x7f672c163750 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f672c0cdd60 0x7f672c163210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 --2- 172.21.15.133:0/1669056952 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f672c06e4d0 0x7f672c13cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 -- 172.21.15.133:0/1669056952 >> 172.21.15.133:0/1669056952 conn(0x7f672c066f70 msgr2=0x7f672c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 -- 172.21.15.133:0/1669056952 shutdown_connections 2024-04-25T10:15:22.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:22.018+0000 7f6733ebd640 1 -- 172.21.15.133:0/1669056952 wait complete. 2024-04-25T10:15:22.035 INFO:tasks.cephfs.filesystem:scrub status for tag:2f105e50-67eb-4381-9e1d-a150de070c13 - {'path': '~mds0', 'tag': '2f105e50-67eb-4381-9e1d-a150de070c13', 'options': 'recursive'} 2024-04-25T10:15:23.700 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:23.703 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:52.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:52.037 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:15:52.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.170+0000 7f4f6c15c640 1 Processor -- start 2024-04-25T10:15:52.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.170+0000 7f4f6c15c640 1 -- start start 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f6c15c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4f6415b9d0 0x7f4f640b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f6c15c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4f6406ac90 0x7f4f6406b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f6c15c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6406ba60 con 0x7f4f6406b680 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f6c15c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6415bdb0 con 0x7f4f6415b9d0 2024-04-25T10:15:52.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f6c15c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f640b1e70 con 0x7f4f6406ac90 2024-04-25T10:15:52.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f69ef8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4f6415b9d0 0x7f4f640b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f69ef8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4f6415b9d0 0x7f4f640b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55464/0 (socket says 172.21.15.133:55464) 2024-04-25T10:15:52.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f69ef8640 1 -- 172.21.15.133:0/3685625052 learned_addr learned my addr 172.21.15.133:0/3685625052 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f696f7640 1 --2- 172.21.15.133:0/3685625052 >> v2:172.21.15.162:3300/0 conn(0x7f4f6406ac90 0x7f4f6406b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f69ef8640 1 -- 172.21.15.133:0/3685625052 >> v2:172.21.15.162:3300/0 conn(0x7f4f6406ac90 msgr2=0x7f4f6406b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/3685625052 >> v2:172.21.15.162:3300/0 conn(0x7f4f6406ac90 0x7f4f6406b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f69ef8640 1 -- 172.21.15.133:0/3685625052 >> v1:172.21.15.162:6789/0 conn(0x7f4f6406b680 legacy=0x7f4f6415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f69ef8640 1 -- 172.21.15.133:0/3685625052 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f640b1b90 con 0x7f4f6415b9d0 2024-04-25T10:15:52.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/3685625052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6415b9d0 0x7f4f640b15b0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f4f64070b50 tx=0x7f4f5c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76759105e733bf7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f68ef6640 1 -- 172.21.15.133:0/3685625052 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0a55e0 con 0x7f4f6415b9d0 2024-04-25T10:15:52.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f68ef6640 1 -- 172.21.15.133:0/3685625052 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c0a5780 con 0x7f4f6415b9d0 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.174+0000 7f4f68ef6640 1 -- 172.21.15.133:0/3685625052 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0a5b50 con 0x7f4f6415b9d0 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 -- 172.21.15.133:0/3685625052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6415b9d0 msgr2=0x7f4f640b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 --2- 172.21.15.133:0/3685625052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6415b9d0 0x7f4f640b15b0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f4f64070b50 tx=0x7f4f5c01c920 comp rx=0 tx=0).stop 2024-04-25T10:15:52.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 -- 172.21.15.133:0/3685625052 shutdown_connections 2024-04-25T10:15:52.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 --2- 172.21.15.133:0/3685625052 >> v2:172.21.15.162:3300/0 conn(0x7f4f6406ac90 0x7f4f6406b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 --2- 172.21.15.133:0/3685625052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6415b9d0 0x7f4f640b15b0 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 -- 172.21.15.133:0/3685625052 >> 172.21.15.133:0/3685625052 conn(0x7f4f640999d0 msgr2=0x7f4f640d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 -- 172.21.15.133:0/3685625052 shutdown_connections 2024-04-25T10:15:52.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 -- 172.21.15.133:0/3685625052 wait complete. 2024-04-25T10:15:52.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 Processor -- start 2024-04-25T10:15:52.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 -- start start 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6406ac90 0x7f4f640c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f6406b680 0x7f4f64156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f6415b9d0 0x7f4f640c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f64168920 con 0x7f4f6406ac90 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f641687a0 con 0x7f4f6415b9d0 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f64168aa0 con 0x7f4f6406b680 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f69ef8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6406ac90 0x7f4f640c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f69ef8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6406ac90 0x7f4f640c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55476/0 (socket says 172.21.15.133:55476) 2024-04-25T10:15:52.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f69ef8640 1 -- 172.21.15.133:0/1850494221 learned_addr learned my addr 172.21.15.133:0/1850494221 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f696f7640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f6406b680 0x7f4f64156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6a6f9640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f6415b9d0 0x7f4f640c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f69ef8640 1 -- 172.21.15.133:0/1850494221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f6406b680 msgr2=0x7f4f64156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f6406b680 0x7f4f64156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f69ef8640 1 -- 172.21.15.133:0/1850494221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f6415b9d0 msgr2=0x7f4f640c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f6415b9d0 0x7f4f640c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f69ef8640 1 -- 172.21.15.133:0/1850494221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f64155850 con 0x7f4f6406ac90 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f696f7640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f6406b680 0x7f4f64156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6406ac90 0x7f4f640c4530 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0af040 tx=0x7f4f5c0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f627fc640 1 -- 172.21.15.133:0/1850494221 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0bc020 con 0x7f4f6406ac90 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f627fc640 1 -- 172.21.15.133:0/1850494221 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c0b8690 con 0x7f4f6406ac90 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 -- 172.21.15.133:0/1850494221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f64155b30 con 0x7f4f6406ac90 2024-04-25T10:15:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f627fc640 1 -- 172.21.15.133:0/1850494221 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0b8a80 con 0x7f4f6406ac90 2024-04-25T10:15:52.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.178+0000 7f4f6c15c640 1 -- 172.21.15.133:0/1850494221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f641560f0 con 0x7f4f6406ac90 2024-04-25T10:15:52.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f627fc640 1 -- 172.21.15.133:0/1850494221 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f5c0bf020 con 0x7f4f6406ac90 2024-04-25T10:15:52.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f627fc640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f50066f60 0x7f4f50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f627fc640 1 -- 172.21.15.133:0/1850494221 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4f5c120250 con 0x7f4f6406ac90 2024-04-25T10:15:52.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f696f7640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f50066f60 0x7f4f50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f696f7640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f50066f60 0x7f4f50069420 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4f58006fd0 tx=0x7f4f58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6f2c9000 1 Processor -- start 2024-04-25T10:15:52.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6f2c9000 1 -- start start 2024-04-25T10:15:52.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c71139d0 0x5576c7113db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 0x5576c70c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c6f16ff0 con 0x5576c70ca0a0 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c6e9f150 con 0x5576c71142f0 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576c6f13090 con 0x5576c71139d0 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6a6f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6a6f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55478/0 (socket says 172.21.15.133:55478) 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6a6f9640 1 -- 172.21.15.133:0/1625959227 learned_addr learned my addr 172.21.15.133:0/1625959227 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f696f7640 1 --2- 172.21.15.133:0/1625959227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 0x5576c70c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/1625959227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c71139d0 0x5576c7113db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6a6f9640 1 -- 172.21.15.133:0/1625959227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c71139d0 msgr2=0x5576c7113db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6a6f9640 1 --2- 172.21.15.133:0/1625959227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c71139d0 0x5576c7113db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6a6f9640 1 -- 172.21.15.133:0/1625959227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 msgr2=0x5576c70c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6a6f9640 1 --2- 172.21.15.133:0/1625959227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 0x5576c70c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6a6f9640 1 -- 172.21.15.133:0/1625959227 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576c70ad0d0 con 0x5576c70ca0a0 2024-04-25T10:15:52.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f6a6f9640 1 --2- 172.21.15.133:0/1625959227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70ca480 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f4f54003dc0 tx=0x7f4f54004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de0b35a9ebdd6cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f4b7fe640 1 -- 172.21.15.133:0/1625959227 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54016020 con 0x5576c70ca0a0 2024-04-25T10:15:52.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f4b7fe640 1 -- 172.21.15.133:0/1625959227 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5400db60 con 0x5576c70ca0a0 2024-04-25T10:15:52.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.182+0000 7f4f4b7fe640 1 -- 172.21.15.133:0/1625959227 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f540123f0 con 0x5576c70ca0a0 2024-04-25T10:15:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/1625959227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 msgr2=0x5576c70ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/1625959227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70ca480 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f4f54003dc0 tx=0x7f4f54004280 comp rx=0 tx=0).stop 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/1625959227 shutdown_connections 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/1625959227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70ca480 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/1625959227 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 0x5576c70c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/1625959227 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c71139d0 0x5576c7113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/1625959227 >> 172.21.15.133:0/1625959227 conn(0x5576c70c6020 msgr2=0x5576c70c8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/1625959227 shutdown_connections 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/1625959227 wait complete. 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- start start 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 Processor -- start 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71139d0 0x5576c70f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c71142f0 0x5576c70f9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c6edb250 con 0x5576c70ca0a0 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c6ed1690 con 0x5576c71139d0 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576c6e9fa40 con 0x5576c71142f0 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 10 client.0 _ll_register_callbacks cb 0x5576c70b4e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f69ef8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f69ef8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70f87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55494/0 (socket says 172.21.15.133:55494) 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f69ef8640 1 -- 172.21.15.133:0/2886586036 learned_addr learned my addr 172.21.15.133:0/2886586036 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6a6f9640 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c71142f0 0x5576c70f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f696f7640 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71139d0 0x5576c70f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f69ef8640 1 -- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c71142f0 msgr2=0x5576c70f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c71142f0 0x5576c70f9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f69ef8640 1 -- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71139d0 msgr2=0x5576c70f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71139d0 0x5576c70f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f69ef8640 1 -- 172.21.15.133:0/2886586036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576c6fc41c0 con 0x5576c70ca0a0 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70f87f0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c11e680 tx=0x7f4f5c0f04e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6403e8546905d31d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f48ff9640 1 -- 172.21.15.133:0/2886586036 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0f0fe0 con 0x5576c70ca0a0 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f48ff9640 1 -- 172.21.15.133:0/2886586036 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4f5c0f8e00 con 0x5576c70ca0a0 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/2886586036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f5400db60 con 0x5576c70ca0a0 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f48ff9640 1 -- 172.21.15.133:0/2886586036 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f5c0f0b80 con 0x5576c70ca0a0 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 10 client.10794 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/2886586036 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5576c6fc0170 con 0x5576c70ca0a0 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f48ff9640 1 -- 172.21.15.133:0/2886586036 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5400db60 con 0x5576c70ca0a0 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f48ff9640 1 -- 172.21.15.133:0/2886586036 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4f5c104b40 con 0x5576c70ca0a0 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 10 client.10794 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 10 client.10794 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 20 client.10794 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:52.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5576c7123c80 0x5576c7126070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 4 client.10794 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:52.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/2886586036 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576c6eb2c60 con 0x5576c7123c80 2024-04-25T10:15:52.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.186+0000 7f4f696f7640 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5576c7123c80 0x5576c7126070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f48ff9640 10 client.10794 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:52.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f696f7640 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5576c7123c80 0x5576c7126070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e878e97c9271f415 server_cookie=999a82ae778e8ee in_seq=0 out_seq=0 2024-04-25T10:15:52.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f48ff9640 1 -- 172.21.15.133:0/2886586036 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5576c6eb2c60 con 0x5576c7123c80 2024-04-25T10:15:52.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f48ff9640 10 client.10794 handle_command_reply: tid=0 2024-04-25T10:15:52.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 1 client.10794 shutdown 2024-04-25T10:15:52.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/2886586036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 msgr2=0x5576c70f87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70f87f0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c11e680 tx=0x7f4f5c0f04e0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/2886586036 shutdown_connections 2024-04-25T10:15:52.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5576c7123c80 0x5576c7126070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c71142f0 0x5576c70f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71139d0 0x5576c70f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/2886586036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c70ca0a0 0x5576c70f87f0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f48ff9640 0 client.10794 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:52.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/2886586036 >> 172.21.15.133:0/2886586036 conn(0x5576c70c6020 msgr2=0x5576c70c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/2886586036 shutdown_connections 2024-04-25T10:15:52.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/2886586036 wait complete. 2024-04-25T10:15:52.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.190+0000 7f4f6f2c9000 20 client.10794 trim_cache size 0 max 16384 2024-04-25T10:15:52.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 Processor -- start 2024-04-25T10:15:52.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 -- start start 2024-04-25T10:15:52.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c70ca0a0 0x5576c7112e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c70c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 0x5576c70c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f64203560 con 0x5576c71139d0 2024-04-25T10:15:52.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f641598f0 con 0x5576c71142f0 2024-04-25T10:15:52.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f64158cf0 con 0x5576c70ca0a0 2024-04-25T10:15:52.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f696f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c70c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f696f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c70c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55496/0 (socket says 172.21.15.133:55496) 2024-04-25T10:15:52.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f696f7640 1 -- 172.21.15.133:0/471180597 learned_addr learned my addr 172.21.15.133:0/471180597 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f696f7640 1 -- 172.21.15.133:0/471180597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c70ca0a0 msgr2=0x5576c7112e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f696f7640 1 --2- 172.21.15.133:0/471180597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c70ca0a0 0x5576c7112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f696f7640 1 -- 172.21.15.133:0/471180597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 msgr2=0x5576c70c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f696f7640 1 --2- 172.21.15.133:0/471180597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 0x5576c70c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f696f7640 1 -- 172.21.15.133:0/471180597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576c6f5dde0 con 0x5576c71139d0 2024-04-25T10:15:52.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f696f7640 1 --2- 172.21.15.133:0/471180597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c70c9420 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f4f640c6c50 tx=0x7f4f58014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94207a15701aaff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f4affd640 1 -- 172.21.15.133:0/471180597 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5800dea0 con 0x5576c71139d0 2024-04-25T10:15:52.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f4affd640 1 -- 172.21.15.133:0/471180597 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f58029030 con 0x5576c71139d0 2024-04-25T10:15:52.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f4affd640 1 -- 172.21.15.133:0/471180597 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58014cc0 con 0x5576c71139d0 2024-04-25T10:15:52.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/471180597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 msgr2=0x5576c70c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/471180597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c70c9420 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f4f640c6c50 tx=0x7f4f58014000 comp rx=0 tx=0).stop 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/471180597 shutdown_connections 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/471180597 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 0x5576c70c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/471180597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c70c9420 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/471180597 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c70ca0a0 0x5576c7112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/471180597 >> 172.21.15.133:0/471180597 conn(0x5576c70c6020 msgr2=0x5576c71120d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/471180597 shutdown_connections 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/471180597 wait complete. 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 -- start start 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.206+0000 7f4f6f2c9000 1 Processor -- start 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c70ca0a0 0x5576c717cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c717d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 0x5576c7126380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576c6eb1e00 con 0x5576c71139d0 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f641592f0 con 0x5576c71142f0 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576c6f16300 con 0x5576c70ca0a0 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 10 client.0 _ll_register_callbacks cb 0x5576c7095040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f696f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c717d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f696f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c717d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55508/0 (socket says 172.21.15.133:55508) 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f696f7640 1 -- 172.21.15.133:0/3262018764 learned_addr learned my addr 172.21.15.133:0/3262018764 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c70ca0a0 0x5576c717cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f696f7640 1 -- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c70ca0a0 msgr2=0x5576c717cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f696f7640 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c70ca0a0 0x5576c717cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6a6f9640 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 0x5576c7126380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f696f7640 1 -- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 msgr2=0x5576c7126380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f696f7640 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 0x5576c7126380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f696f7640 1 -- 172.21.15.133:0/3262018764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576c6eb88e0 con 0x5576c71139d0 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c70ca0a0 0x5576c717cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f696f7640 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c717d270 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f4f58008010 tx=0x7f4f58004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e97fb421d8416f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f48ff9640 1 -- 172.21.15.133:0/3262018764 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f58032070 con 0x5576c71139d0 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f48ff9640 1 -- 172.21.15.133:0/3262018764 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4f58014cd0 con 0x5576c71139d0 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/3262018764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f58029030 con 0x5576c71139d0 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f48ff9640 1 -- 172.21.15.133:0/3262018764 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f5803d050 con 0x5576c71139d0 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 10 client.10800 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/3262018764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5576c6f708f0 con 0x5576c71139d0 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f48ff9640 1 -- 172.21.15.133:0/3262018764 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f58029030 con 0x5576c71139d0 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f48ff9640 1 -- 172.21.15.133:0/3262018764 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4f58040020 con 0x5576c71139d0 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 10 client.10800 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 10 client.10800 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T10:15:52.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 20 client.10800 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:52.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5576c7126f50 0x5576c7127360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 4 client.10800 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:15:52.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/3262018764 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5576c6f24530 con 0x5576c7126f50 2024-04-25T10:15:52.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5576c7126f50 0x5576c7127360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f69ef8640 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5576c7126f50 0x5576c7127360 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce93db7f603ce27 server_cookie=ee964b9de0277c8a in_seq=0 out_seq=0 2024-04-25T10:15:52.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f48ff9640 10 client.10800 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:52.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f48ff9640 1 -- 172.21.15.133:0/3262018764 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5576c6f24530 con 0x5576c7126f50 2024-04-25T10:15:52.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f48ff9640 10 client.10800 handle_command_reply: tid=0 2024-04-25T10:15:52.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 client.10800 shutdown 2024-04-25T10:15:52.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/3262018764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 msgr2=0x5576c717d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c717d270 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f4f58008010 tx=0x7f4f58004740 comp rx=0 tx=0).stop 2024-04-25T10:15:52.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/3262018764 shutdown_connections 2024-04-25T10:15:52.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5576c71142f0 0x5576c7126380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576c71139d0 0x5576c717d270 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x5576c7126f50 0x5576c7127360 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 --2- 172.21.15.133:0/3262018764 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5576c70ca0a0 0x5576c717cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/3262018764 >> 172.21.15.133:0/3262018764 conn(0x5576c70c6020 msgr2=0x5576c7112960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f48ff9640 0 client.10800 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:52.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/3262018764 shutdown_connections 2024-04-25T10:15:52.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 1 -- 172.21.15.133:0/3262018764 wait complete. 2024-04-25T10:15:52.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.210+0000 7f4f6f2c9000 20 client.10800 trim_cache size 0 max 16384 2024-04-25T10:15:52.222 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:15:52.222 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-04-25T10:15:52.222 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:15:52.222 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:15:52.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 -- 172.21.15.133:0/1850494221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f50066f60 msgr2=0x7f4f50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f50066f60 0x7f4f50069420 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4f58006fd0 tx=0x7f4f58008040 comp rx=0 tx=0).stop 2024-04-25T10:15:52.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 -- 172.21.15.133:0/1850494221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6406ac90 msgr2=0x7f4f640c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6406ac90 0x7f4f640c4530 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0af040 tx=0x7f4f5c0196c0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 -- 172.21.15.133:0/1850494221 shutdown_connections 2024-04-25T10:15:52.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f6415b9d0 0x7f4f640c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f6406b680 0x7f4f64156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f50066f60 0x7f4f50069420 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 --2- 172.21.15.133:0/1850494221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6406ac90 0x7f4f640c4530 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 -- 172.21.15.133:0/1850494221 >> 172.21.15.133:0/1850494221 conn(0x7f4f640999d0 msgr2=0x7f4f641655d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 -- 172.21.15.133:0/1850494221 shutdown_connections 2024-04-25T10:15:52.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.214+0000 7f4f6c15c640 1 -- 172.21.15.133:0/1850494221 wait complete. 2024-04-25T10:15:52.229 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:15:52.229 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:15:52.229 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:15:52.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.346+0000 7f3657324640 1 Processor -- start 2024-04-25T10:15:52.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.346+0000 7f3657324640 1 -- start start 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3657324640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f365006eac0 0x7f36501653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3657324640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365015ac50 con 0x7f36500b2150 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3657324640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36500cd180 con 0x7f3650143ea0 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3657324640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36500ccf20 con 0x7f365006eac0 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f36548bf640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f36500b2150 0x7f365006e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45798/0 (socket says 172.21.15.133:45798) 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f36548bf640 1 -- 172.21.15.133:0/2591859475 learned_addr learned my addr 172.21.15.133:0/2591859475 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f36558c1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f365006eac0 0x7f36501653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3647fff640 1 -- 172.21.15.133:0/2591859475 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7023839 0 0) 0x7f3640001120 con 0x7f36500b2150 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3647fff640 1 -- 172.21.15.133:0/2591859475 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f363c003530 con 0x7f36500b2150 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f36558c1640 1 -- 172.21.15.133:0/2591859475 >> v1:172.21.15.162:6789/0 conn(0x7f3650143ea0 legacy=0x7f36500cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f36558c1640 1 -- 172.21.15.133:0/2591859475 >> v1:172.21.15.133:6789/0 conn(0x7f36500b2150 legacy=0x7f365006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f36558c1640 1 -- 172.21.15.133:0/2591859475 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365015aed0 con 0x7f365006eac0 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f36558c1640 1 --2- 172.21.15.133:0/2591859475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365006eac0 0x7f36501653c0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f3648009d90 tx=0x7f364801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f7e4557f831ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3647fff640 1 -- 172.21.15.133:0/2591859475 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36480b2070 con 0x7f365006eac0 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3647fff640 1 -- 172.21.15.133:0/2591859475 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36480a58e0 con 0x7f365006eac0 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3647fff640 1 -- 172.21.15.133:0/2591859475 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36480a5cb0 con 0x7f365006eac0 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3657324640 1 -- 172.21.15.133:0/2591859475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365006eac0 msgr2=0x7f36501653c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3657324640 1 --2- 172.21.15.133:0/2591859475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365006eac0 0x7f36501653c0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f3648009d90 tx=0x7f364801c930 comp rx=0 tx=0).stop 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3657324640 1 -- 172.21.15.133:0/2591859475 shutdown_connections 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3657324640 1 --2- 172.21.15.133:0/2591859475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365006eac0 0x7f36501653c0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3657324640 1 -- 172.21.15.133:0/2591859475 >> 172.21.15.133:0/2591859475 conn(0x7f3650066f70 msgr2=0x7f365006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3657324640 1 -- 172.21.15.133:0/2591859475 shutdown_connections 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.350+0000 7f3657324640 1 -- 172.21.15.133:0/2591859475 wait complete. 2024-04-25T10:15:52.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3657324640 1 Processor -- start 2024-04-25T10:15:52.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3657324640 1 -- start start 2024-04-25T10:15:52.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3657324640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f365006eac0 0x7f365015df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3657324640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36500b2150 0x7f36501fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3657324640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3650143ea0 0x7f36501fe540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3657324640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36501684b0 con 0x7f3650143ea0 2024-04-25T10:15:52.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3657324640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3650168330 con 0x7f365006eac0 2024-04-25T10:15:52.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3657324640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3650168630 con 0x7f36500b2150 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f36558c1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3650143ea0 0x7f36501fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f36558c1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3650143ea0 0x7f36501fe540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55526/0 (socket says 172.21.15.133:55526) 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f36558c1640 1 -- 172.21.15.133:0/2534919822 learned_addr learned my addr 172.21.15.133:0/2534919822 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f36548bf640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36500b2150 0x7f36501fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f36550c0640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f365006eac0 0x7f365015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f36558c1640 1 -- 172.21.15.133:0/2534919822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36500b2150 msgr2=0x7f36501fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f36558c1640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36500b2150 0x7f36501fe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f36558c1640 1 -- 172.21.15.133:0/2534919822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f365006eac0 msgr2=0x7f365015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f36558c1640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f365006eac0 0x7f365015df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f36558c1640 1 -- 172.21.15.133:0/2534919822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36500b3dd0 con 0x7f3650143ea0 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f36558c1640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3650143ea0 0x7f36501fe540 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f36480b0040 tx=0x7f36480196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3645ffb640 1 -- 172.21.15.133:0/2534919822 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36480bd020 con 0x7f3650143ea0 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3645ffb640 1 -- 172.21.15.133:0/2534919822 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36480b8670 con 0x7f3650143ea0 2024-04-25T10:15:52.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3657324640 1 -- 172.21.15.133:0/2534919822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36500b40b0 con 0x7f3650143ea0 2024-04-25T10:15:52.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3645ffb640 1 -- 172.21.15.133:0/2534919822 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36480b2040 con 0x7f3650143ea0 2024-04-25T10:15:52.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.354+0000 7f3657324640 1 -- 172.21.15.133:0/2534919822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36500b4670 con 0x7f3650143ea0 2024-04-25T10:15:52.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f3645ffb640 1 -- 172.21.15.133:0/2534919822 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36480c0020 con 0x7f3650143ea0 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f3645ffb640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3634066f60 0x7f3634069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f3645ffb640 1 -- 172.21.15.133:0/2534919822 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3648120220 con 0x7f3650143ea0 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36550c0640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3634066f60 0x7f3634069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36550c0640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3634066f60 0x7f3634069420 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f364c00a750 tx=0x7f364c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 Processor -- start 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 -- start start 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354180 0x564d60354560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d6030a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 0x564d6030a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d60157f20 con 0x564d60354aa0 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d600e0000 con 0x564d6030a600 2024-04-25T10:15:52.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d60153fc0 con 0x564d60354180 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36548bf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d6030a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36548bf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d6030a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55532/0 (socket says 172.21.15.133:55532) 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36548bf640 1 -- 172.21.15.133:0/192358152 learned_addr learned my addr 172.21.15.133:0/192358152 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36550c0640 1 --2- 172.21.15.133:0/192358152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354180 0x564d60354560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36558c1640 1 --2- 172.21.15.133:0/192358152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 0x564d6030a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36548bf640 1 -- 172.21.15.133:0/192358152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354180 msgr2=0x564d60354560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36548bf640 1 --2- 172.21.15.133:0/192358152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354180 0x564d60354560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36548bf640 1 -- 172.21.15.133:0/192358152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 msgr2=0x564d6030a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36548bf640 1 --2- 172.21.15.133:0/192358152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 0x564d6030a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36548bf640 1 -- 172.21.15.133:0/192358152 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d60133ba0 con 0x564d60354aa0 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36558c1640 1 --2- 172.21.15.133:0/192358152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 0x564d6030a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f36548bf640 1 --2- 172.21.15.133:0/192358152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d6030a0c0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f3640001670 tx=0x7f3640001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587835b8963e522d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f363affd640 1 -- 172.21.15.133:0/192358152 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640018070 con 0x564d60354aa0 2024-04-25T10:15:52.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f363affd640 1 -- 172.21.15.133:0/192358152 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640005cb0 con 0x564d60354aa0 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f363affd640 1 -- 172.21.15.133:0/192358152 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640007310 con 0x564d60354aa0 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 -- 172.21.15.133:0/192358152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 msgr2=0x564d6030a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 --2- 172.21.15.133:0/192358152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d6030a0c0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f3640001670 tx=0x7f3640001b30 comp rx=0 tx=0).stop 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 -- 172.21.15.133:0/192358152 shutdown_connections 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 --2- 172.21.15.133:0/192358152 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 0x564d6030a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 --2- 172.21.15.133:0/192358152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d6030a0c0 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 --2- 172.21.15.133:0/192358152 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354180 0x564d60354560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 -- 172.21.15.133:0/192358152 >> 172.21.15.133:0/192358152 conn(0x564d60310760 msgr2=0x564d603089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 -- 172.21.15.133:0/192358152 shutdown_connections 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.358+0000 7f365a491000 1 -- 172.21.15.133:0/192358152 wait complete. 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 -- start start 2024-04-25T10:15:52.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 Processor -- start 2024-04-25T10:15:52.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d6030a600 0x564d6030fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d60354180 0x564d60315f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d603164c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d602159c0 con 0x564d60354aa0 2024-04-25T10:15:52.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d601125d0 con 0x564d60354180 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d601224b0 con 0x564d6030a600 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 10 client.0 _ll_register_callbacks cb 0x564d602f5a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36558c1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d603164c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36558c1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d603164c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55548/0 (socket says 172.21.15.133:55548) 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36558c1640 1 -- 172.21.15.133:0/4048370135 learned_addr learned my addr 172.21.15.133:0/4048370135 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36550c0640 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d6030a600 0x564d6030fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36548bf640 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d60354180 0x564d60315f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36558c1640 1 -- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d6030a600 msgr2=0x564d6030fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36558c1640 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d6030a600 0x564d6030fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36558c1640 1 -- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d60354180 msgr2=0x564d60315f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36558c1640 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d60354180 0x564d60315f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36558c1640 1 -- 172.21.15.133:0/4048370135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d6019e5a0 con 0x564d60354aa0 2024-04-25T10:15:52.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36548bf640 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d60354180 0x564d60315f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36558c1640 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d603164c0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f364801cea0 tx=0x7f36480f0af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad3bf90dc3e5926 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f362ffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:52.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f362ffff640 1 -- 172.21.15.133:0/4048370135 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648120070 con 0x564d60354aa0 2024-04-25T10:15:52.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f362ffff640 1 -- 172.21.15.133:0/4048370135 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f36480a56e0 con 0x564d60354aa0 2024-04-25T10:15:52.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 -- 172.21.15.133:0/4048370135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3640005cb0 con 0x564d60354aa0 2024-04-25T10:15:52.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f362ffff640 1 -- 172.21.15.133:0/4048370135 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f364811e050 con 0x564d60354aa0 2024-04-25T10:15:52.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 10 client.10815 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:52.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 -- 172.21.15.133:0/4048370135 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x564d6012f2b0 con 0x564d60354aa0 2024-04-25T10:15:52.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f362ffff640 1 -- 172.21.15.133:0/4048370135 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3640005cb0 con 0x564d60354aa0 2024-04-25T10:15:52.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f362ffff640 1 -- 172.21.15.133:0/4048370135 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f364811e1f0 con 0x564d60354aa0 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 10 client.10815 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 10 client.10815 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 20 client.10815 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x564d6035b9e0 0x564d6035bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 4 client.10815 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f365a491000 1 -- 172.21.15.133:0/4048370135 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d600f3c70 con 0x564d6035b9e0 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36550c0640 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x564d6035b9e0 0x564d6035bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f362ffff640 10 client.10815 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.362+0000 7f36550c0640 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x564d6035b9e0 0x564d6035bdf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5cc69596702ac2a server_cookie=bfcc4d477c9cbf18 in_seq=0 out_seq=0 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f362ffff640 1 -- 172.21.15.133:0/4048370135 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564d600f3c70 con 0x564d6035b9e0 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f362ffff640 10 client.10815 handle_command_reply: tid=0 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 1 client.10815 shutdown 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 1 -- 172.21.15.133:0/4048370135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 msgr2=0x564d603164c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d603164c0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f364801cea0 tx=0x7f36480f0af0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 1 -- 172.21.15.133:0/4048370135 shutdown_connections 2024-04-25T10:15:52.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354aa0 0x564d603164c0 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d60354180 0x564d60315f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x564d6035b9e0 0x564d6035bdf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 1 --2- 172.21.15.133:0/4048370135 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d6030a600 0x564d6030fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f362ffff640 0 client.10815 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:52.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 1 -- 172.21.15.133:0/4048370135 >> 172.21.15.133:0/4048370135 conn(0x564d60310760 msgr2=0x564d603090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 1 -- 172.21.15.133:0/4048370135 shutdown_connections 2024-04-25T10:15:52.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 1 -- 172.21.15.133:0/4048370135 wait complete. 2024-04-25T10:15:52.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.366+0000 7f365a491000 20 client.10815 trim_cache size 0 max 16384 2024-04-25T10:15:52.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 Processor -- start 2024-04-25T10:15:52.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 -- start start 2024-04-25T10:15:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d6030a600 0x564d60308110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354180 0x564d60308650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d60354aa0 0x564d60308b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3650207cb0 con 0x564d6030a600 2024-04-25T10:15:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3650206f00 con 0x564d60354aa0 2024-04-25T10:15:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36500b3510 con 0x564d60354180 2024-04-25T10:15:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f36550c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d6030a600 0x564d60308110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f36550c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d6030a600 0x564d60308110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55562/0 (socket says 172.21.15.133:55562) 2024-04-25T10:15:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f36550c0640 1 -- 172.21.15.133:0/282909517 learned_addr learned my addr 172.21.15.133:0/282909517 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f36550c0640 1 -- 172.21.15.133:0/282909517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354180 msgr2=0x564d60308650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f36550c0640 1 --2- 172.21.15.133:0/282909517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354180 0x564d60308650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f36550c0640 1 -- 172.21.15.133:0/282909517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d60354aa0 msgr2=0x564d60308b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f36550c0640 1 --2- 172.21.15.133:0/282909517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d60354aa0 0x564d60308b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f36550c0640 1 -- 172.21.15.133:0/282909517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d600e0250 con 0x564d6030a600 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f36550c0640 1 --2- 172.21.15.133:0/282909517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d6030a600 0x564d60308110 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f364c00cad0 tx=0x7f364c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d673f4dd53d58cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f363a7fc640 1 -- 172.21.15.133:0/282909517 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c019040 con 0x564d6030a600 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f363a7fc640 1 -- 172.21.15.133:0/282909517 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c029030 con 0x564d6030a600 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f363a7fc640 1 -- 172.21.15.133:0/282909517 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c020e10 con 0x564d6030a600 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 -- 172.21.15.133:0/282909517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d6030a600 msgr2=0x564d60308110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 --2- 172.21.15.133:0/282909517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d6030a600 0x564d60308110 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f364c00cad0 tx=0x7f364c013000 comp rx=0 tx=0).stop 2024-04-25T10:15:52.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 -- 172.21.15.133:0/282909517 shutdown_connections 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 --2- 172.21.15.133:0/282909517 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d60354aa0 0x564d60308b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 --2- 172.21.15.133:0/282909517 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354180 0x564d60308650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 --2- 172.21.15.133:0/282909517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d6030a600 0x564d60308110 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 -- 172.21.15.133:0/282909517 >> 172.21.15.133:0/282909517 conn(0x564d60310760 msgr2=0x564d60314f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 -- 172.21.15.133:0/282909517 shutdown_connections 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 -- 172.21.15.133:0/282909517 wait complete. 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 -- start start 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.378+0000 7f365a491000 1 Processor -- start 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 0x564d60359390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354180 0x564d603598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354aa0 0x564d60359e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d60157230 con 0x564d60354180 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36501fc250 con 0x564d6030a600 2024-04-25T10:15:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d6011b7b0 con 0x564d60354aa0 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 10 client.0 _ll_register_callbacks cb 0x564d602d6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36548bf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354180 0x564d603598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36548bf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354180 0x564d603598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55572/0 (socket says 172.21.15.133:55572) 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36548bf640 1 -- 172.21.15.133:0/2601019961 learned_addr learned my addr 172.21.15.133:0/2601019961 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36550c0640 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 0x564d60359390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36558c1640 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354aa0 0x564d60359e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36548bf640 1 -- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354aa0 msgr2=0x564d60359e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36548bf640 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354aa0 0x564d60359e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36548bf640 1 -- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 msgr2=0x564d60359390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36548bf640 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 0x564d60359390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36548bf640 1 -- 172.21.15.133:0/2601019961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d600ecb90 con 0x564d60354180 2024-04-25T10:15:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36550c0640 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 0x564d60359390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36548bf640 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354180 0x564d603598d0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x564d6015df00 tx=0x7f3640000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b768d348a86e376d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f362ffff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:15:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f362ffff640 1 -- 172.21.15.133:0/2601019961 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640018070 con 0x564d60354180 2024-04-25T10:15:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f362ffff640 1 -- 172.21.15.133:0/2601019961 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3640005040 con 0x564d60354180 2024-04-25T10:15:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 -- 172.21.15.133:0/2601019961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f364c029030 con 0x564d60354180 2024-04-25T10:15:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f362ffff640 1 -- 172.21.15.133:0/2601019961 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f364002b050 con 0x564d60354180 2024-04-25T10:15:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 10 client.10821 fetch_fsmap learned FSMap version 99 2024-04-25T10:15:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 -- 172.21.15.133:0/2601019961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x564d600ff010 con 0x564d60354180 2024-04-25T10:15:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f362ffff640 1 -- 172.21.15.133:0/2601019961 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c029030 con 0x564d60354180 2024-04-25T10:15:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f362ffff640 1 -- 172.21.15.133:0/2601019961 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f364002e020 con 0x564d60354180 2024-04-25T10:15:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 10 client.10821 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 10 client.10821 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.b 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 20 client.10821 populate_metadata read hostname 'smithi133' 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x564d6035a7d0 0x564d603bc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 4 client.10821 mds_command: new command op to 10554 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 -- 172.21.15.133:0/2601019961 --> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564d60165460 con 0x564d6035a7d0 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36550c0640 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x564d6035a7d0 0x564d603bc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f362ffff640 10 client.10821 ms_handle_connect on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f36550c0640 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x564d6035a7d0 0x564d603bc420 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c6f504a8e680d9 server_cookie=3a01320de0f660d8 in_seq=0 out_seq=0 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f362ffff640 1 -- 172.21.15.133:0/2601019961 <== mds.0 v2:172.21.15.162:6832/3763981385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564d60165460 con 0x564d6035a7d0 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f362ffff640 10 client.10821 handle_command_reply: tid=0 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 client.10821 shutdown 2024-04-25T10:15:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 -- 172.21.15.133:0/2601019961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354180 msgr2=0x564d603598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354180 0x564d603598d0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x564d6015df00 tx=0x7f3640000ee0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 -- 172.21.15.133:0/2601019961 shutdown_connections 2024-04-25T10:15:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:6832/3763981385,v1:172.21.15.162:6833/3763981385] conn(0x564d6035a7d0 0x564d603bc420 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564d60354aa0 0x564d60359e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d60354180 0x564d603598d0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 --2- 172.21.15.133:0/2601019961 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564d6030a600 0x564d60359390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 -- 172.21.15.133:0/2601019961 >> 172.21.15.133:0/2601019961 conn(0x564d60310760 msgr2=0x564d603089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f362ffff640 0 client.10821 ms_handle_reset on v2:172.21.15.162:6832/3763981385 2024-04-25T10:15:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 -- 172.21.15.133:0/2601019961 shutdown_connections 2024-04-25T10:15:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 1 -- 172.21.15.133:0/2601019961 wait complete. 2024-04-25T10:15:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.382+0000 7f365a491000 20 client.10821 trim_cache size 0 max 16384 2024-04-25T10:15:52.392 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-04-25T10:15:52.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 -- 172.21.15.133:0/2534919822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3634066f60 msgr2=0x7f3634069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3634066f60 0x7f3634069420 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f364c00a750 tx=0x7f364c008040 comp rx=0 tx=0).stop 2024-04-25T10:15:52.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 -- 172.21.15.133:0/2534919822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3650143ea0 msgr2=0x7f36501fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3650143ea0 0x7f36501fe540 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f36480b0040 tx=0x7f36480196d0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 -- 172.21.15.133:0/2534919822 shutdown_connections 2024-04-25T10:15:52.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3634066f60 0x7f3634069420 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3650143ea0 0x7f36501fe540 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36500b2150 0x7f36501fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 --2- 172.21.15.133:0/2534919822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f365006eac0 0x7f365015df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 -- 172.21.15.133:0/2534919822 >> 172.21.15.133:0/2534919822 conn(0x7f3650066f70 msgr2=0x7f365015cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 -- 172.21.15.133:0/2534919822 shutdown_connections 2024-04-25T10:15:52.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.386+0000 7f3657324640 1 -- 172.21.15.133:0/2534919822 wait complete. 2024-04-25T10:15:52.401 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:15:52.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.510+0000 7fe77ee84640 1 Processor -- start 2024-04-25T10:15:52.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.510+0000 7fe77ee84640 1 -- start start 2024-04-25T10:15:52.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77ee84640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 0x7fe7780b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77ee84640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe7781584f0 0x7fe77815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77ee84640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7780cbe60 con 0x7fe778158110 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77ee84640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780b19b0 con 0x7fe7781584f0 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77ee84640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77815b360 con 0x7fe77806eb70 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe777fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 0x7fe7780b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77d421640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe7781584f0 0x7fe77815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77cc20640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fe778158110 0x7fe77806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:42686/0 (socket says 172.21.15.133:42686) 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77d421640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe7781584f0 0x7fe77815ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:60944/0 (socket says 172.21.15.133:60944) 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe777fff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 0x7fe7780b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:49658/0 (socket says 172.21.15.133:49658) 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77cc20640 1 -- 172.21.15.133:0/770856916 learned_addr learned my addr 172.21.15.133:0/770856916 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77d421640 1 -- 172.21.15.133:0/770856916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 msgr2=0x7fe7780b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77d421640 1 --2- 172.21.15.133:0/770856916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 0x7fe7780b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77d421640 1 -- 172.21.15.133:0/770856916 >> v1:172.21.15.162:6789/0 conn(0x7fe778158110 legacy=0x7fe77806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77d421640 1 -- 172.21.15.133:0/770856916 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77815b640 con 0x7fe7781584f0 2024-04-25T10:15:52.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe777fff640 1 --2- 172.21.15.133:0/770856916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 0x7fe7780b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:52.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe77d421640 1 --2- 172.21.15.133:0/770856916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7781584f0 0x7fe77815ad10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe770009d90 tx=0x7fe77001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53beda715ef495cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.514+0000 7fe7777fe640 1 -- 172.21.15.133:0/770856916 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700b2070 con 0x7fe7781584f0 2024-04-25T10:15:52.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe7777fe640 1 -- 172.21.15.133:0/770856916 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7700a5a20 con 0x7fe7781584f0 2024-04-25T10:15:52.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe7777fe640 1 -- 172.21.15.133:0/770856916 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700a5df0 con 0x7fe7781584f0 2024-04-25T10:15:52.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 -- 172.21.15.133:0/770856916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7781584f0 msgr2=0x7fe77815ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 --2- 172.21.15.133:0/770856916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7781584f0 0x7fe77815ad10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe770009d90 tx=0x7fe77001c940 comp rx=0 tx=0).stop 2024-04-25T10:15:52.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 -- 172.21.15.133:0/770856916 shutdown_connections 2024-04-25T10:15:52.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 --2- 172.21.15.133:0/770856916 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 0x7fe7780b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 --2- 172.21.15.133:0/770856916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7781584f0 0x7fe77815ad10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 -- 172.21.15.133:0/770856916 >> 172.21.15.133:0/770856916 conn(0x7fe778067000 msgr2=0x7fe77806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 -- 172.21.15.133:0/770856916 shutdown_connections 2024-04-25T10:15:52.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 -- 172.21.15.133:0/770856916 wait complete. 2024-04-25T10:15:52.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 Processor -- start 2024-04-25T10:15:52.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 -- start start 2024-04-25T10:15:52.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 0x7fe7781f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe778158110 0x7fe7781503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7781584f0 0x7fe77815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780ccca0 con 0x7fe7781584f0 2024-04-25T10:15:52.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780ccb20 con 0x7fe778158110 2024-04-25T10:15:52.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780cce20 con 0x7fe77806eb70 2024-04-25T10:15:52.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77d421640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7781584f0 0x7fe77815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77d421640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7781584f0 0x7fe77815ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55574/0 (socket says 172.21.15.133:55574) 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77d421640 1 -- 172.21.15.133:0/1818527243 learned_addr learned my addr 172.21.15.133:0/1818527243 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe777fff640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe778158110 0x7fe7781503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77cc20640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 0x7fe7781f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77d421640 1 -- 172.21.15.133:0/1818527243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 msgr2=0x7fe7781f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77d421640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 0x7fe7781f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77d421640 1 -- 172.21.15.133:0/1818527243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe778158110 msgr2=0x7fe7781503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77d421640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe778158110 0x7fe7781503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77d421640 1 -- 172.21.15.133:0/1818527243 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7781508f0 con 0x7fe7781584f0 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe777fff640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe778158110 0x7fe7781503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77cc20640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 0x7fe7781f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77d421640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7781584f0 0x7fe77815ce60 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b0040 tx=0x7fe7700196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe7757fa640 1 -- 172.21.15.133:0/1818527243 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700b2050 con 0x7fe7781584f0 2024-04-25T10:15:52.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe7757fa640 1 -- 172.21.15.133:0/1818527243 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7700be070 con 0x7fe7781584f0 2024-04-25T10:15:52.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 -- 172.21.15.133:0/1818527243 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7780b24a0 con 0x7fe7781584f0 2024-04-25T10:15:52.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe7757fa640 1 -- 172.21.15.133:0/1818527243 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700adb10 con 0x7fe7781584f0 2024-04-25T10:15:52.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.518+0000 7fe77ee84640 1 -- 172.21.15.133:0/1818527243 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7780b2a60 con 0x7fe7781584f0 2024-04-25T10:15:52.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.522+0000 7fe77ee84640 1 -- 172.21.15.133:0/1818527243 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe778068ef0 con 0x7fe7781584f0 2024-04-25T10:15:52.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.522+0000 7fe7757fa640 1 -- 172.21.15.133:0/1818527243 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7700adcb0 con 0x7fe7781584f0 2024-04-25T10:15:52.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.522+0000 7fe7757fa640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe764066f90 0x7fe764069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.522+0000 7fe77cc20640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe764066f90 0x7fe764069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.522+0000 7fe7757fa640 1 -- 172.21.15.133:0/1818527243 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe770120590 con 0x7fe7781584f0 2024-04-25T10:15:52.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.522+0000 7fe77cc20640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe764066f90 0x7fe764069450 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe768000f70 tx=0x7fe768009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.526+0000 7fe7757fa640 1 -- 172.21.15.133:0/1818527243 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe770016560 con 0x7fe7781584f0 2024-04-25T10:15:52.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.714+0000 7fe77ee84640 1 -- 172.21.15.133:0/1818527243 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe77806c2d0 con 0x7fe7781584f0 2024-04-25T10:15:52.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.714+0000 7fe7757fa640 1 -- 172.21.15.133:0/1818527243 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe7700ab340 con 0x7fe7781584f0 2024-04-25T10:15:52.716 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:52.716 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:15:52.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 -- 172.21.15.133:0/1818527243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe764066f90 msgr2=0x7fe764069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe764066f90 0x7fe764069450 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe768000f70 tx=0x7fe768009040 comp rx=0 tx=0).stop 2024-04-25T10:15:52.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 -- 172.21.15.133:0/1818527243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7781584f0 msgr2=0x7fe77815ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7781584f0 0x7fe77815ce60 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b0040 tx=0x7fe7700196e0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 -- 172.21.15.133:0/1818527243 shutdown_connections 2024-04-25T10:15:52.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe764066f90 0x7fe764069450 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7781584f0 0x7fe77815ce60 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe778158110 0x7fe7781503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 --2- 172.21.15.133:0/1818527243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe77806eb70 0x7fe7781f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 -- 172.21.15.133:0/1818527243 >> 172.21.15.133:0/1818527243 conn(0x7fe778067000 msgr2=0x7fe7780ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 -- 172.21.15.133:0/1818527243 shutdown_connections 2024-04-25T10:15:52.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.718+0000 7fe77ee84640 1 -- 172.21.15.133:0/1818527243 wait complete. 2024-04-25T10:15:52.733 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:15:52.734 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:52.774 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:52.820 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:15:52.821 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:15:52.821 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:52.827 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.986+0000 7f4ec56b3640 1 Processor -- start 2024-04-25T10:15:52.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.986+0000 7f4ec56b3640 1 -- start start 2024-04-25T10:15:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ec56b3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec01584f0 0x7f4ec00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ec56b3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec00cbe60 con 0x7f4ec006eb70 2024-04-25T10:15:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ec56b3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec00b19b0 con 0x7f4ec0158110 2024-04-25T10:15:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ec56b3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec015b360 con 0x7f4ec01584f0 2024-04-25T10:15:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4eb67fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4ec006eb70 0x7f4ec00cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45812/0 (socket says 172.21.15.133:45812) 2024-04-25T10:15:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4eb67fc640 1 -- 172.21.15.133:0/3176217492 learned_addr learned my addr 172.21.15.133:0/3176217492 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:52.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968579549 0 0) 0x7f4ec00cbe60 con 0x7f4ec006eb70 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebf7fe640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec01584f0 0x7f4ec00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eac005740 con 0x7f4ec006eb70 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1552477409 0 0) 0x7f4eac005740 con 0x7f4ec006eb70 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec01584f0 msgr2=0x7f4ec00b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec01584f0 0x7f4ec00b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 >> v1:172.21.15.162:6789/0 conn(0x7f4ec0158110 legacy=0x7f4ec006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec015b640 con 0x7f4ec006eb70 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4ea8001680 con 0x7f4ec006eb70 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 >> v1:172.21.15.133:6789/0 conn(0x7f4ec006eb70 legacy=0x7f4ec00cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eac005c00 0x7f4eac006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4eac006a20 0x7f4eac006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec01584f0 0x7f4eac007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec015b360 con 0x7f4eac005c00 2024-04-25T10:15:52.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac00dd10 con 0x7f4eac006a20 2024-04-25T10:15:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac00df10 con 0x7f4ec01584f0 2024-04-25T10:15:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebeffd640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eac005c00 0x7f4eac006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4eb67fc640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4eac006a20 0x7f4eac006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebf7fe640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec01584f0 0x7f4eac007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebeffd640 1 -- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec01584f0 msgr2=0x7f4eac007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebeffd640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec01584f0 0x7f4eac007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebeffd640 1 -- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4eac006a20 msgr2=0x7f4eac006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebeffd640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4eac006a20 0x7f4eac006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebeffd640 1 -- 172.21.15.133:0/3176217492 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac005740 con 0x7f4eac005c00 2024-04-25T10:15:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4ebf7fe640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec01584f0 0x7f4eac007420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.990+0000 7f4eb67fc640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4eac006a20 0x7f4eac006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebeffd640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eac005c00 0x7f4eac006020 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0001780 tx=0x7f4eb001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0957e290ac8352c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00013a0 con 0x7f4eac005c00 2024-04-25T10:15:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb00052f0 con 0x7f4eac005c00 2024-04-25T10:15:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebe7fc640 1 -- 172.21.15.133:0/3176217492 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0003840 con 0x7f4eac005c00 2024-04-25T10:15:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 -- 172.21.15.133:0/3176217492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eac005c00 msgr2=0x7f4eac006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 --2- 172.21.15.133:0/3176217492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4eac005c00 0x7f4eac006020 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0001780 tx=0x7f4eb001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebf7fe640 1 -- 172.21.15.133:0/3176217492 reap_dead start 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 -- 172.21.15.133:0/3176217492 shutdown_connections 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 -- 172.21.15.133:0/3176217492 >> 172.21.15.133:0/3176217492 conn(0x7f4ec0067000 msgr2=0x7f4ec006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 -- 172.21.15.133:0/3176217492 shutdown_connections 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 -- 172.21.15.133:0/3176217492 wait complete. 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 Processor -- start 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 -- start start 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4eac005c00 0x7f4ec015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec015f3f0 0x7f4ec015f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec0141f60 0x7f4ec01423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec00cd670 con 0x7f4ec0141f60 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec015b120 con 0x7f4eac005c00 2024-04-25T10:15:52.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec00cd7f0 con 0x7f4ec015f3f0 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec0141f60 0x7f4ec01423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec0141f60 0x7f4ec01423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55590/0 (socket says 172.21.15.133:55590) 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebf7fe640 1 -- 172.21.15.133:0/4016929240 learned_addr learned my addr 172.21.15.133:0/4016929240 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebeffd640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4eac005c00 0x7f4ec015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4eb67fc640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec015f3f0 0x7f4ec015f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebf7fe640 1 -- 172.21.15.133:0/4016929240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec015f3f0 msgr2=0x7f4ec015f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebf7fe640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec015f3f0 0x7f4ec015f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebf7fe640 1 -- 172.21.15.133:0/4016929240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4eac005c00 msgr2=0x7f4ec015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebf7fe640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4eac005c00 0x7f4ec015cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebf7fe640 1 -- 172.21.15.133:0/4016929240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec0140ec0 con 0x7f4ec0141f60 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ebf7fe640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec0141f60 0x7f4ec01423e0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00b4d70 tx=0x7f4eb8009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4eb7fff640 1 -- 172.21.15.133:0/4016929240 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8019070 con 0x7f4ec0141f60 2024-04-25T10:15:52.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4eb7fff640 1 -- 172.21.15.133:0/4016929240 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb8007cb0 con 0x7f4ec0141f60 2024-04-25T10:15:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 -- 172.21.15.133:0/4016929240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec01411a0 con 0x7f4ec0141f60 2024-04-25T10:15:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4eb7fff640 1 -- 172.21.15.133:0/4016929240 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8007e60 con 0x7f4ec0141f60 2024-04-25T10:15:52.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.994+0000 7f4ec56b3640 1 -- 172.21.15.133:0/4016929240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec0141760 con 0x7f4ec0141f60 2024-04-25T10:15:53.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.998+0000 7f4ec56b3640 1 -- 172.21.15.133:0/4016929240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec006bc20 con 0x7f4ec0141f60 2024-04-25T10:15:53.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.998+0000 7f4eb7fff640 1 -- 172.21.15.133:0/4016929240 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4eb801c070 con 0x7f4ec0141f60 2024-04-25T10:15:53.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.998+0000 7f4eb7fff640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ea0066f60 0x7f4ea0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.998+0000 7f4eb7fff640 1 -- 172.21.15.133:0/4016929240 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4eb8084b70 con 0x7f4ec0141f60 2024-04-25T10:15:53.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.998+0000 7f4ebeffd640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ea0066f60 0x7f4ea0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:52.998+0000 7f4ebeffd640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ea0066f60 0x7f4ea0069420 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0001750 tx=0x7f4eb00011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.002+0000 7f4eb7fff640 1 -- 172.21.15.133:0/4016929240 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4eb80559f0 con 0x7f4ec0141f60 2024-04-25T10:15:53.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.162+0000 7f4ec56b3640 1 -- 172.21.15.133:0/4016929240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ec00cb1a0 con 0x7f4ec0141f60 2024-04-25T10:15:53.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.162+0000 7f4eb7fff640 1 -- 172.21.15.133:0/4016929240 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16908 (secure 0 0 0) 0x7f4eb80552f0 con 0x7f4ec0141f60 2024-04-25T10:15:53.164 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:53.164 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":157,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:15:14.762327+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:11.332375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:12.104303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:53.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 -- 172.21.15.133:0/4016929240 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ea0066f60 msgr2=0x7f4ea0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ea0066f60 0x7f4ea0069420 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0001750 tx=0x7f4eb00011a0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 -- 172.21.15.133:0/4016929240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec0141f60 msgr2=0x7f4ec01423e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec0141f60 0x7f4ec01423e0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00b4d70 tx=0x7f4eb8009d50 comp rx=0 tx=0).stop 2024-04-25T10:15:53.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 -- 172.21.15.133:0/4016929240 shutdown_connections 2024-04-25T10:15:53.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ea0066f60 0x7f4ea0069420 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec0141f60 0x7f4ec01423e0 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4ec015f3f0 0x7f4ec015f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 --2- 172.21.15.133:0/4016929240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4eac005c00 0x7f4ec015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 -- 172.21.15.133:0/4016929240 >> 172.21.15.133:0/4016929240 conn(0x7f4ec0067000 msgr2=0x7f4ec00ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 -- 172.21.15.133:0/4016929240 shutdown_connections 2024-04-25T10:15:53.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.166+0000 7f4ec56b3640 1 -- 172.21.15.133:0/4016929240 wait complete. 2024-04-25T10:15:53.184 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:15:11.332375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:15:12.104303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:53.184 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:53.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.302+0000 7f358ea90640 1 Processor -- start 2024-04-25T10:15:53.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f358ea90640 1 -- start start 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f358ea90640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3588145170 0x7f358806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f358ea90640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35880ce2e0 con 0x7f358806e4d0 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f358ea90640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358806d4b0 con 0x7f358815ced0 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f358ea90640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358806db00 con 0x7f3588145170 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f358c82c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3588145170 0x7f358806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f357ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f358806e4d0 0x7f35880cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45824/0 (socket says 172.21.15.133:45824) 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f357ffff640 1 -- 172.21.15.133:0/1330143313 learned_addr learned my addr 172.21.15.133:0/1330143313 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f358c82c640 1 -- 172.21.15.133:0/1330143313 >> v1:172.21.15.162:6789/0 conn(0x7f358815ced0 legacy=0x7f35880b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f358c82c640 1 -- 172.21.15.133:0/1330143313 >> v1:172.21.15.133:6789/0 conn(0x7f358806e4d0 legacy=0x7f35880cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f357f7fe640 1 -- 172.21.15.133:0/1330143313 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003469423 0 0) 0x7f35880ce2e0 con 0x7f358806e4d0 2024-04-25T10:15:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.306+0000 7f358c82c640 1 -- 172.21.15.133:0/1330143313 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358806d790 con 0x7f3588145170 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358c82c640 1 --2- 172.21.15.133:0/1330143313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3588145170 0x7f358806df90 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f35840097c0 tx=0x7f358401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b1ba3a81ac1d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f357f7fe640 1 -- 172.21.15.133:0/1330143313 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840b0070 con 0x7f3588145170 2024-04-25T10:15:53.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f357f7fe640 1 -- 172.21.15.133:0/1330143313 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35840ab600 con 0x7f3588145170 2024-04-25T10:15:53.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f357f7fe640 1 -- 172.21.15.133:0/1330143313 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840ab9d0 con 0x7f3588145170 2024-04-25T10:15:53.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 -- 172.21.15.133:0/1330143313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3588145170 msgr2=0x7f358806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 --2- 172.21.15.133:0/1330143313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3588145170 0x7f358806df90 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f35840097c0 tx=0x7f358401c460 comp rx=0 tx=0).stop 2024-04-25T10:15:53.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 -- 172.21.15.133:0/1330143313 shutdown_connections 2024-04-25T10:15:53.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 --2- 172.21.15.133:0/1330143313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3588145170 0x7f358806df90 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 -- 172.21.15.133:0/1330143313 >> 172.21.15.133:0/1330143313 conn(0x7f3588066f70 msgr2=0x7f358806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 -- 172.21.15.133:0/1330143313 shutdown_connections 2024-04-25T10:15:53.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 -- 172.21.15.133:0/1330143313 wait complete. 2024-04-25T10:15:53.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 Processor -- start 2024-04-25T10:15:53.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 -- start start 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358806e4d0 0x7f35881553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3588145170 0x7f35880ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358815ced0 0x7f35880b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3588145550 con 0x7f358815ced0 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880b2020 con 0x7f358806e4d0 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358ea90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358815d320 con 0x7f3588145170 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358d02d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358815ced0 0x7f35880b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358d02d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358815ced0 0x7f35880b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55602/0 (socket says 172.21.15.133:55602) 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358d02d640 1 -- 172.21.15.133:0/2257653850 learned_addr learned my addr 172.21.15.133:0/2257653850 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358c82c640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358806e4d0 0x7f35881553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358d02d640 1 -- 172.21.15.133:0/2257653850 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3588145170 msgr2=0x7f35880ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358d02d640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3588145170 0x7f35880ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358d02d640 1 -- 172.21.15.133:0/2257653850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358806e4d0 msgr2=0x7f35881553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358d02d640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358806e4d0 0x7f35881553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358d02d640 1 -- 172.21.15.133:0/2257653850 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35880b0a60 con 0x7f358815ced0 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f358d02d640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358815ced0 0x7f35880b41c0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f358000b970 tx=0x7f358000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.310+0000 7f357d7fa640 1 -- 172.21.15.133:0/2257653850 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580019070 con 0x7f358815ced0 2024-04-25T10:15:53.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.314+0000 7f357d7fa640 1 -- 172.21.15.133:0/2257653850 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358000cb40 con 0x7f358815ced0 2024-04-25T10:15:53.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.314+0000 7f357d7fa640 1 -- 172.21.15.133:0/2257653850 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580014440 con 0x7f358815ced0 2024-04-25T10:15:53.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.314+0000 7f358ea90640 1 -- 172.21.15.133:0/2257653850 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35881559c0 con 0x7f358815ced0 2024-04-25T10:15:53.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.314+0000 7f358ea90640 1 -- 172.21.15.133:0/2257653850 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3588155f80 con 0x7f358815ced0 2024-04-25T10:15:53.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.314+0000 7f358ea90640 1 -- 172.21.15.133:0/2257653850 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3588068e80 con 0x7f358815ced0 2024-04-25T10:15:53.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.314+0000 7f357d7fa640 1 -- 172.21.15.133:0/2257653850 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f358001c070 con 0x7f358815ced0 2024-04-25T10:15:53.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.314+0000 7f357d7fa640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3574066f90 0x7f3574069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.314+0000 7f358c82c640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3574066f90 0x7f3574069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.314+0000 7f357d7fa640 1 -- 172.21.15.133:0/2257653850 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3580084d90 con 0x7f358815ced0 2024-04-25T10:15:53.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.314+0000 7f358c82c640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3574066f90 0x7f3574069450 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f35840ad000 tx=0x7f3584008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.318+0000 7f357d7fa640 1 -- 172.21.15.133:0/2257653850 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3580055ca0 con 0x7f358815ced0 2024-04-25T10:15:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.470+0000 7f358ea90640 1 -- 172.21.15.133:0/2257653850 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f358815e190 con 0x7f358815ced0 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.470+0000 7f357d7fa640 1 -- 172.21.15.133:0/2257653850 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35800555a0 con 0x7f358815ced0 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:15:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 -- 172.21.15.133:0/2257653850 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3574066f90 msgr2=0x7f3574069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3574066f90 0x7f3574069450 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f35840ad000 tx=0x7f3584008040 comp rx=0 tx=0).stop 2024-04-25T10:15:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 -- 172.21.15.133:0/2257653850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358815ced0 msgr2=0x7f35880b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358815ced0 0x7f35880b41c0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f358000b970 tx=0x7f358000be30 comp rx=0 tx=0).stop 2024-04-25T10:15:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 -- 172.21.15.133:0/2257653850 shutdown_connections 2024-04-25T10:15:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3574066f90 0x7f3574069450 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358815ced0 0x7f35880b41c0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3588145170 0x7f35880ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 --2- 172.21.15.133:0/2257653850 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f358806e4d0 0x7f35881553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 -- 172.21.15.133:0/2257653850 >> 172.21.15.133:0/2257653850 conn(0x7f3588066f70 msgr2=0x7f358813c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 -- 172.21.15.133:0/2257653850 shutdown_connections 2024-04-25T10:15:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.474+0000 7f358ea90640 1 -- 172.21.15.133:0/2257653850 wait complete. 2024-04-25T10:15:53.490 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:15:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.598+0000 7fd52e0f2640 1 Processor -- start 2024-04-25T10:15:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.598+0000 7fd52e0f2640 1 -- start start 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd52e0f2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd52806e8b0 0x7fd52815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd52e0f2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd5280aebc0 0x7fd5280af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd52e0f2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5281471f0 con 0x7fd5280b1e50 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd52e0f2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd528147490 con 0x7fd5280aebc0 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd52e0f2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280b19c0 con 0x7fd52806e8b0 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd526ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd5280b1e50 0x7fd528146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45836/0 (socket says 172.21.15.133:45836) 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd526ffd640 1 -- 172.21.15.133:0/3716786585 learned_addr learned my addr 172.21.15.133:0/3716786585 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd527fff640 1 --2- 172.21.15.133:0/3716786585 >> v2:172.21.15.162:3300/0 conn(0x7fd5280aebc0 0x7fd5280af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5277fe640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd52806e8b0 0x7fd52815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584477379 0 0) 0x7fd5281471f0 con 0x7fd5280b1e50 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd510003530 con 0x7fd5280b1e50 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 36968767 0 0) 0x7fd510003530 con 0x7fd5280b1e50 2024-04-25T10:15:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd52806e8b0 msgr2=0x7fd52815eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd52806e8b0 0x7fd52815eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 >> v2:172.21.15.162:3300/0 conn(0x7fd5280aebc0 msgr2=0x7fd5280af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 --2- 172.21.15.133:0/3716786585 >> v2:172.21.15.162:3300/0 conn(0x7fd5280aebc0 0x7fd5280af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5280b1090 con 0x7fd5280b1e50 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fd51c001680 con 0x7fd5280b1e50 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 >> v1:172.21.15.133:6789/0 conn(0x7fd5280b1e50 legacy=0x7fd528146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd52806e8b0 0x7fd5100039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 0x7fd510004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd510004d50 0x7fd5100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd528147490 con 0x7fd510003f40 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280b19c0 con 0x7fd510004d50 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51000ba20 con 0x7fd52806e8b0 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd526ffd640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 0x7fd510004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd5277fe640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd52806e8b0 0x7fd5100039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd526ffd640 1 -- 172.21.15.133:0/3716786585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd52806e8b0 msgr2=0x7fd5100039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd527fff640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd510004d50 0x7fd5100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd526ffd640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd52806e8b0 0x7fd5100039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd526ffd640 1 -- 172.21.15.133:0/3716786585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd510004d50 msgr2=0x7fd5100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd526ffd640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd510004d50 0x7fd5100051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.602+0000 7fd526ffd640 1 -- 172.21.15.133:0/3716786585 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd510003530 con 0x7fd510003f40 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd526ffd640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 0x7fd510004810 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c270 tx=0x7fd51c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d8f49f89e8148f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c007d20 con 0x7fd510003f40 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c0a9070 con 0x7fd510003f40 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5267fc640 1 -- 172.21.15.133:0/3716786585 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c005da0 con 0x7fd510003f40 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 -- 172.21.15.133:0/3716786585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 msgr2=0x7fd510004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 --2- 172.21.15.133:0/3716786585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 0x7fd510004810 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c270 tx=0x7fd51c01cc80 comp rx=0 tx=0).stop 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd527fff640 1 -- 172.21.15.133:0/3716786585 reap_dead start 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 -- 172.21.15.133:0/3716786585 shutdown_connections 2024-04-25T10:15:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 -- 172.21.15.133:0/3716786585 >> 172.21.15.133:0/3716786585 conn(0x7fd528066f70 msgr2=0x7fd52806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 -- 172.21.15.133:0/3716786585 shutdown_connections 2024-04-25T10:15:53.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 -- 172.21.15.133:0/3716786585 wait complete. 2024-04-25T10:15:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 Processor -- start 2024-04-25T10:15:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 -- start start 2024-04-25T10:15:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 0x7fd52815d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5281f7200 0x7fd528144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd528144c90 0x7fd5281f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52806dc30 con 0x7fd510003f40 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52806dab0 con 0x7fd528144c90 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52806ddb0 con 0x7fd5281f7200 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5277fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 0x7fd52815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5277fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 0x7fd52815d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55626/0 (socket says 172.21.15.133:55626) 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5277fe640 1 -- 172.21.15.133:0/1717697011 learned_addr learned my addr 172.21.15.133:0/1717697011 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd526ffd640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5281f7200 0x7fd528144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd527fff640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd528144c90 0x7fd5281f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5277fe640 1 -- 172.21.15.133:0/1717697011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5281f7200 msgr2=0x7fd528144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5277fe640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5281f7200 0x7fd528144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5277fe640 1 -- 172.21.15.133:0/1717697011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd528144c90 msgr2=0x7fd5281f9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5277fe640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd528144c90 0x7fd5281f9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5277fe640 1 -- 172.21.15.133:0/1717697011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5281f7680 con 0x7fd510003f40 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd526ffd640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5281f7200 0x7fd528144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd527fff640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd528144c90 0x7fd5281f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd5277fe640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 0x7fd52815d280 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7fd5180108c0 tx=0x7fd518010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd50ffff640 1 -- 172.21.15.133:0/1717697011 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51800f040 con 0x7fd510003f40 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd50ffff640 1 -- 172.21.15.133:0/1717697011 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51800eb60 con 0x7fd510003f40 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 -- 172.21.15.133:0/1717697011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5281404a0 con 0x7fd510003f40 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd50ffff640 1 -- 172.21.15.133:0/1717697011 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51801c400 con 0x7fd510003f40 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.606+0000 7fd52e0f2640 1 -- 172.21.15.133:0/1717697011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd528140a60 con 0x7fd510003f40 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.610+0000 7fd52e0f2640 1 -- 172.21.15.133:0/1717697011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd528068e80 con 0x7fd510003f40 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.610+0000 7fd50ffff640 1 -- 172.21.15.133:0/1717697011 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd51801c5a0 con 0x7fd510003f40 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.610+0000 7fd50ffff640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd514066f90 0x7fd514069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.610+0000 7fd50ffff640 1 -- 172.21.15.133:0/1717697011 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd518085770 con 0x7fd510003f40 2024-04-25T10:15:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.610+0000 7fd526ffd640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd514066f90 0x7fd514069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.610+0000 7fd526ffd640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd514066f90 0x7fd514069450 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0ab000 tx=0x7fd51c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.614+0000 7fd50ffff640 1 -- 172.21.15.133:0/1717697011 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd518018030 con 0x7fd510003f40 2024-04-25T10:15:53.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.758+0000 7fd52e0f2640 1 -- 172.21.15.133:0/1717697011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd52806b960 con 0x7fd510003f40 2024-04-25T10:15:53.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.758+0000 7fd50ffff640 1 -- 172.21.15.133:0/1717697011 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v157) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5180565e0 con 0x7fd510003f40 2024-04-25T10:15:53.758 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:15:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 -- 172.21.15.133:0/1717697011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd514066f90 msgr2=0x7fd514069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd514066f90 0x7fd514069450 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0ab000 tx=0x7fd51c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:15:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 -- 172.21.15.133:0/1717697011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 msgr2=0x7fd52815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 0x7fd52815d280 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7fd5180108c0 tx=0x7fd518010d80 comp rx=0 tx=0).stop 2024-04-25T10:15:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 -- 172.21.15.133:0/1717697011 shutdown_connections 2024-04-25T10:15:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd528144c90 0x7fd5281f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd5281f7200 0x7fd528144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd514066f90 0x7fd514069450 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 --2- 172.21.15.133:0/1717697011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd510003f40 0x7fd52815d280 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 -- 172.21.15.133:0/1717697011 >> 172.21.15.133:0/1717697011 conn(0x7fd528066f70 msgr2=0x7fd528161bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 -- 172.21.15.133:0/1717697011 shutdown_connections 2024-04-25T10:15:53.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.762+0000 7fd52e0f2640 1 -- 172.21.15.133:0/1717697011 wait complete. 2024-04-25T10:15:53.777 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:15:53.820 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:53.822 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:53.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff3475fa640 1 Processor -- start 2024-04-25T10:15:53.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff3475fa640 1 -- start start 2024-04-25T10:15:53.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff3475fa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff3401595f0 0x7ff3400b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff3475fa640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3400ccda0 con 0x7ff34006b3a0 2024-04-25T10:15:53.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff3475fa640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3400afec0 con 0x7ff3400b1770 2024-04-25T10:15:53.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff3475fa640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff340160340 con 0x7ff3401595f0 2024-04-25T10:15:53.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff345396640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff3401595f0 0x7ff3400b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff345b97640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff34006b3a0 0x7ff34015bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45842/0 (socket says 172.21.15.133:45842) 2024-04-25T10:15:53.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff345b97640 1 -- 172.21.15.133:0/391784247 learned_addr learned my addr 172.21.15.133:0/391784247 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:53.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff345396640 1 -- 172.21.15.133:0/391784247 >> v1:172.21.15.162:6789/0 conn(0x7ff3400b1770 legacy=0x7ff34006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff345396640 1 -- 172.21.15.133:0/391784247 >> v1:172.21.15.133:6789/0 conn(0x7ff34006b3a0 legacy=0x7ff34015bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff345396640 1 -- 172.21.15.133:0/391784247 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400b1db0 con 0x7ff3401595f0 2024-04-25T10:15:53.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.898+0000 7ff337fff640 1 -- 172.21.15.133:0/391784247 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360390820 0 0) 0x7ff3400ccda0 con 0x7ff34006b3a0 2024-04-25T10:15:53.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff345396640 1 --2- 172.21.15.133:0/391784247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3401595f0 0x7ff3400b1230 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7ff3400d15c0 tx=0x7ff33c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b13d70a1ada321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff337fff640 1 -- 172.21.15.133:0/391784247 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0b0070 con 0x7ff3401595f0 2024-04-25T10:15:53.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff337fff640 1 -- 172.21.15.133:0/391784247 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c0ab600 con 0x7ff3401595f0 2024-04-25T10:15:53.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff337fff640 1 -- 172.21.15.133:0/391784247 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0ab9d0 con 0x7ff3401595f0 2024-04-25T10:15:53.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 -- 172.21.15.133:0/391784247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3401595f0 msgr2=0x7ff3400b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 --2- 172.21.15.133:0/391784247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3401595f0 0x7ff3400b1230 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7ff3400d15c0 tx=0x7ff33c01c440 comp rx=0 tx=0).stop 2024-04-25T10:15:53.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 -- 172.21.15.133:0/391784247 shutdown_connections 2024-04-25T10:15:53.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 --2- 172.21.15.133:0/391784247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3401595f0 0x7ff3400b1230 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 -- 172.21.15.133:0/391784247 >> 172.21.15.133:0/391784247 conn(0x7ff340072710 msgr2=0x7ff3400999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 -- 172.21.15.133:0/391784247 shutdown_connections 2024-04-25T10:15:53.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 -- 172.21.15.133:0/391784247 wait complete. 2024-04-25T10:15:53.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 Processor -- start 2024-04-25T10:15:53.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 -- start start 2024-04-25T10:15:53.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34006b3a0 0x7ff340155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3400b1770 0x7ff340156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3401595f0 0x7ff34014f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340169890 con 0x7ff3400b1770 2024-04-25T10:15:53.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34015fe20 con 0x7ff34006b3a0 2024-04-25T10:15:53.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff3475fa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff340169a10 con 0x7ff3401595f0 2024-04-25T10:15:53.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff344b95640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3400b1770 0x7ff340156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff344b95640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3400b1770 0x7ff340156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55646/0 (socket says 172.21.15.133:55646) 2024-04-25T10:15:53.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff344b95640 1 -- 172.21.15.133:0/4071921542 learned_addr learned my addr 172.21.15.133:0/4071921542 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:53.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff345396640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34006b3a0 0x7ff340155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff345b97640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3401595f0 0x7ff34014f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff344b95640 1 -- 172.21.15.133:0/4071921542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3401595f0 msgr2=0x7ff34014f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff344b95640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3401595f0 0x7ff34014f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff344b95640 1 -- 172.21.15.133:0/4071921542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34006b3a0 msgr2=0x7ff340155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff344b95640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34006b3a0 0x7ff340155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff344b95640 1 -- 172.21.15.133:0/4071921542 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34014e2d0 con 0x7ff3400b1770 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.902+0000 7ff345b97640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3401595f0 0x7ff34014f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff344b95640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3400b1770 0x7ff340156100 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c480 tx=0x7ff33000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff335ffb640 1 -- 172.21.15.133:0/4071921542 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330012830 con 0x7ff3400b1770 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff335ffb640 1 -- 172.21.15.133:0/4071921542 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330017070 con 0x7ff3400b1770 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff335ffb640 1 -- 172.21.15.133:0/4071921542 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330012c30 con 0x7ff3400b1770 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff3475fa640 1 -- 172.21.15.133:0/4071921542 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34014e5b0 con 0x7ff3400b1770 2024-04-25T10:15:53.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff3475fa640 1 -- 172.21.15.133:0/4071921542 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34014eb70 con 0x7ff3400b1770 2024-04-25T10:15:53.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff3475fa640 1 -- 172.21.15.133:0/4071921542 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff340068e80 con 0x7ff3400b1770 2024-04-25T10:15:53.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff335ffb640 1 -- 172.21.15.133:0/4071921542 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff33001c460 con 0x7ff3400b1770 2024-04-25T10:15:53.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff335ffb640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff324066f60 0x7ff324069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff335ffb640 1 -- 172.21.15.133:0/4071921542 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff330085080 con 0x7ff3400b1770 2024-04-25T10:15:53.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff345396640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff324066f60 0x7ff324069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.906+0000 7ff345396640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff324066f60 0x7ff324069420 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0ad000 tx=0x7ff33c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:53.910+0000 7ff335ffb640 1 -- 172.21.15.133:0/4071921542 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff330055ef0 con 0x7ff3400b1770 2024-04-25T10:15:54.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.058+0000 7ff3475fa640 1 -- 172.21.15.133:0/4071921542 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff340099cb0 con 0x7ff3400b1770 2024-04-25T10:15:54.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.058+0000 7ff335ffb640 1 -- 172.21.15.133:0/4071921542 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v157) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3300557f0 con 0x7ff3400b1770 2024-04-25T10:15:54.060 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:15:54.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 -- 172.21.15.133:0/4071921542 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff324066f60 msgr2=0x7ff324069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff324066f60 0x7ff324069420 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0ad000 tx=0x7ff33c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:15:54.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 -- 172.21.15.133:0/4071921542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3400b1770 msgr2=0x7ff340156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3400b1770 0x7ff340156100 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c480 tx=0x7ff33000c940 comp rx=0 tx=0).stop 2024-04-25T10:15:54.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 -- 172.21.15.133:0/4071921542 shutdown_connections 2024-04-25T10:15:54.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3401595f0 0x7ff34014f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff324066f60 0x7ff324069420 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3400b1770 0x7ff340156100 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 --2- 172.21.15.133:0/4071921542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff34006b3a0 0x7ff340155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 -- 172.21.15.133:0/4071921542 >> 172.21.15.133:0/4071921542 conn(0x7ff340072710 msgr2=0x7ff34006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 -- 172.21.15.133:0/4071921542 shutdown_connections 2024-04-25T10:15:54.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.062+0000 7ff3475fa640 1 -- 172.21.15.133:0/4071921542 wait complete. 2024-04-25T10:15:54.078 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:15:54.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05b0345640 1 Processor -- start 2024-04-25T10:15:54.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05b0345640 1 -- start start 2024-04-25T10:15:54.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05b0345640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815eb40 0x7f05a80b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05b0345640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a81471f0 con 0x7f05a80aebc0 2024-04-25T10:15:54.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05b0345640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a80b1090 con 0x7f05a806e8b0 2024-04-25T10:15:54.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05b0345640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8147490 con 0x7f05a815eb40 2024-04-25T10:15:54.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ae8e2640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f05a80aebc0 0x7f05a806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45856/0 (socket says 172.21.15.133:45856) 2024-04-25T10:15:54.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ae8e2640 1 -- 172.21.15.133:0/751315423 learned_addr learned my addr 172.21.15.133:0/751315423 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:54.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad8e0640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815eb40 0x7f05a80b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841362552 0 0) 0x7f05a81471f0 con 0x7f05a80aebc0 2024-04-25T10:15:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0594003530 con 0x7f05a80aebc0 2024-04-25T10:15:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1063770398 0 0) 0x7f0594003530 con 0x7f05a80aebc0 2024-04-25T10:15:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815eb40 msgr2=0x7f05a80b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815eb40 0x7f05a80b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 >> v1:172.21.15.162:6789/0 conn(0x7f05a806e8b0 legacy=0x7f05a8146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80b1370 con 0x7f05a80aebc0 2024-04-25T10:15:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f05a0001a00 con 0x7f05a80aebc0 2024-04-25T10:15:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 >> v1:172.21.15.133:6789/0 conn(0x7f05a80aebc0 legacy=0x7f05a806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05940039f0 0x7f0594003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815eb40 0x7f0594004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0594004d50 0x7f05940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8147490 con 0x7f0594004d50 2024-04-25T10:15:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059400bb00 con 0x7f05940039f0 2024-04-25T10:15:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059400bd00 con 0x7f05a815eb40 2024-04-25T10:15:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ae8e2640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0594004d50 0x7f05940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ae0e1640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05940039f0 0x7f0594003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.198+0000 7f05ad8e0640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815eb40 0x7f0594004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05ae8e2640 1 -- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815eb40 msgr2=0x7f0594004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:54.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05ae8e2640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815eb40 0x7f0594004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05ae8e2640 1 -- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05940039f0 msgr2=0x7f0594003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05ae8e2640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05940039f0 0x7f0594003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05ae8e2640 1 -- 172.21.15.133:0/751315423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0594003530 con 0x7f0594004d50 2024-04-25T10:15:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05ae0e1640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05940039f0 0x7f0594003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05ae8e2640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0594004d50 0x7f05940051d0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f05a000a860 tx=0x7f05a001fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd8a9cecb871596 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a000a480 con 0x7f0594004d50 2024-04-25T10:15:54.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a00aa710 con 0x7f0594004d50 2024-04-25T10:15:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05ad0df640 1 -- 172.21.15.133:0/751315423 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a00aab20 con 0x7f0594004d50 2024-04-25T10:15:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05b0345640 1 -- 172.21.15.133:0/751315423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0594004d50 msgr2=0x7f05940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05b0345640 1 --2- 172.21.15.133:0/751315423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0594004d50 0x7f05940051d0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f05a000a860 tx=0x7f05a001fae0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05ae8e2640 1 -- 172.21.15.133:0/751315423 reap_dead start 2024-04-25T10:15:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05b0345640 1 -- 172.21.15.133:0/751315423 shutdown_connections 2024-04-25T10:15:54.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05b0345640 1 -- 172.21.15.133:0/751315423 >> 172.21.15.133:0/751315423 conn(0x7f05a8066f70 msgr2=0x7f05a806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05b0345640 1 -- 172.21.15.133:0/751315423 shutdown_connections 2024-04-25T10:15:54.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.202+0000 7f05b0345640 1 -- 172.21.15.133:0/751315423 wait complete. 2024-04-25T10:15:54.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05b0345640 1 Processor -- start 2024-04-25T10:15:54.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05b0345640 1 -- start start 2024-04-25T10:15:54.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05b0345640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05a813aed0 0x7f05a8141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05b0345640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a8141670 0x7f05a8143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05b0345640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a81444e0 0x7f05a81601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05b0345640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80af050 con 0x7f05a81444e0 2024-04-25T10:15:54.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05b0345640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80b19c0 con 0x7f05a813aed0 2024-04-25T10:15:54.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05b0345640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80af1d0 con 0x7f05a8141670 2024-04-25T10:15:54.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05ae8e2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a81444e0 0x7f05a81601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05ae8e2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a81444e0 0x7f05a81601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55674/0 (socket says 172.21.15.133:55674) 2024-04-25T10:15:54.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05ae8e2640 1 -- 172.21.15.133:0/1047942113 learned_addr learned my addr 172.21.15.133:0/1047942113 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:54.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05ae0e1640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05a813aed0 0x7f05a8141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05ad8e0640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a8141670 0x7f05a8143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05ae8e2640 1 -- 172.21.15.133:0/1047942113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a8141670 msgr2=0x7f05a8143fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05ae8e2640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a8141670 0x7f05a8143fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05ae8e2640 1 -- 172.21.15.133:0/1047942113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05a813aed0 msgr2=0x7f05a8141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05ae8e2640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05a813aed0 0x7f05a8141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05ae8e2640 1 -- 172.21.15.133:0/1047942113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a81431e0 con 0x7f05a81444e0 2024-04-25T10:15:54.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05ae8e2640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a81444e0 0x7f05a81601b0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f05a0001d00 tx=0x7f0594005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f059affd640 1 -- 172.21.15.133:0/1047942113 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a000a480 con 0x7f05a81444e0 2024-04-25T10:15:54.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f059affd640 1 -- 172.21.15.133:0/1047942113 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a00b8070 con 0x7f05a81444e0 2024-04-25T10:15:54.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05b0345640 1 -- 172.21.15.133:0/1047942113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a81434c0 con 0x7f05a81444e0 2024-04-25T10:15:54.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f059affd640 1 -- 172.21.15.133:0/1047942113 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0004d70 con 0x7f05a81444e0 2024-04-25T10:15:54.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05b0345640 1 -- 172.21.15.133:0/1047942113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a8143a80 con 0x7f05a81444e0 2024-04-25T10:15:54.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.206+0000 7f05b0345640 1 -- 172.21.15.133:0/1047942113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a8068e80 con 0x7f05a81444e0 2024-04-25T10:15:54.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.210+0000 7f059affd640 1 -- 172.21.15.133:0/1047942113 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05a00ba030 con 0x7f05a81444e0 2024-04-25T10:15:54.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.210+0000 7f059affd640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f058c066f60 0x7f058c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.210+0000 7f059affd640 1 -- 172.21.15.133:0/1047942113 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f05a01221e0 con 0x7f05a81444e0 2024-04-25T10:15:54.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.210+0000 7f05ae0e1640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f058c066f60 0x7f058c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.210+0000 7f05ae0e1640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f058c066f60 0x7f058c069420 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f059c0054b0 tx=0x7f059c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.214+0000 7f059affd640 1 -- 172.21.15.133:0/1047942113 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05a00ae030 con 0x7f05a81444e0 2024-04-25T10:15:54.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.378+0000 7f05b0345640 1 -- 172.21.15.133:0/1047942113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f05a80b4aa0 con 0x7f05a81444e0 2024-04-25T10:15:54.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.378+0000 7f059affd640 1 -- 172.21.15.133:0/1047942113 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v157) v1 ==== 80+0+0 (secure 0 0 0) 0x7f05a00197f0 con 0x7f05a81444e0 2024-04-25T10:15:54.380 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:15:54.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 -- 172.21.15.133:0/1047942113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f058c066f60 msgr2=0x7f058c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f058c066f60 0x7f058c069420 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f059c0054b0 tx=0x7f059c009040 comp rx=0 tx=0).stop 2024-04-25T10:15:54.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 -- 172.21.15.133:0/1047942113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a81444e0 msgr2=0x7f05a81601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a81444e0 0x7f05a81601b0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f05a0001d00 tx=0x7f0594005840 comp rx=0 tx=0).stop 2024-04-25T10:15:54.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 -- 172.21.15.133:0/1047942113 shutdown_connections 2024-04-25T10:15:54.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f058c066f60 0x7f058c069420 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a81444e0 0x7f05a81601b0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a8141670 0x7f05a8143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 --2- 172.21.15.133:0/1047942113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05a813aed0 0x7f05a8141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 -- 172.21.15.133:0/1047942113 >> 172.21.15.133:0/1047942113 conn(0x7f05a8066f70 msgr2=0x7f05a8145ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 -- 172.21.15.133:0/1047942113 shutdown_connections 2024-04-25T10:15:54.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.382+0000 7f05b0345640 1 -- 172.21.15.133:0/1047942113 wait complete. 2024-04-25T10:15:54.397 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:15:54.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:54.398 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:54.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[87636]: fuse finished with error 0 and tester_r 0 2024-04-25T10:15:54.410 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:15:54.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:54.410 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:15:54.563 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:15:54.563 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:54.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:54.570 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:15:54.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:54.570 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:15:54.615 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:15:54.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:54.617 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:15:54.617 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:54.624 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:54.624 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:15:54.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:54.624 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:15:54.678 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:15:54.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:54.679 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:15:54.680 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:54.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:54.686 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:15:54.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:54.686 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:15:54.735 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:15:54.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:54.736 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:54.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.866+0000 7fcbc7283640 1 Processor -- start 2024-04-25T10:15:54.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc7283640 1 -- start start 2024-04-25T10:15:54.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc7283640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcbc015c9a0 0x7fcbc00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc7283640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcbc00b1f70 0x7fcbc006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc7283640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc00c61d0 con 0x7fcbc00ce330 2024-04-25T10:15:54.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc7283640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc00c64b0 con 0x7fcbc015c9a0 2024-04-25T10:15:54.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc7283640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc006d4f0 con 0x7fcbc00b1f70 2024-04-25T10:15:54.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc501f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcbc015c9a0 0x7fcbc00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc501f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcbc015c9a0 0x7fcbc00c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55680/0 (socket says 172.21.15.133:55680) 2024-04-25T10:15:54.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc501f640 1 -- 172.21.15.133:0/2327349591 learned_addr learned my addr 172.21.15.133:0/2327349591 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:54.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc5820640 1 --1- 172.21.15.133:0/2327349591 >> v1:172.21.15.133:6789/0 conn(0x7fcbc00ce330 0x7fcbc015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45866/0 (socket says 172.21.15.133:45866) 2024-04-25T10:15:54.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc481e640 1 --2- 172.21.15.133:0/2327349591 >> v2:172.21.15.162:3300/0 conn(0x7fcbc00b1f70 0x7fcbc006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc501f640 1 -- 172.21.15.133:0/2327349591 >> v2:172.21.15.162:3300/0 conn(0x7fcbc00b1f70 msgr2=0x7fcbc006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:54.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc501f640 1 --2- 172.21.15.133:0/2327349591 >> v2:172.21.15.162:3300/0 conn(0x7fcbc00b1f70 0x7fcbc006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc501f640 1 -- 172.21.15.133:0/2327349591 >> v1:172.21.15.133:6789/0 conn(0x7fcbc00ce330 legacy=0x7fcbc015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbbffff640 1 -- 172.21.15.133:0/2327349591 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621346329 0 0) 0x7fcbc00c61d0 con 0x7fcbc00ce330 2024-04-25T10:15:54.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc501f640 1 -- 172.21.15.133:0/2327349591 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc006d7d0 con 0x7fcbc015c9a0 2024-04-25T10:15:54.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.870+0000 7fcbc501f640 1 --2- 172.21.15.133:0/2327349591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc015c9a0 0x7fcbc00c5980 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0072670 tx=0x7fcbb001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f939c1df98eb3ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbbffff640 1 -- 172.21.15.133:0/2327349591 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb00b1070 con 0x7fcbc015c9a0 2024-04-25T10:15:54.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbbffff640 1 -- 172.21.15.133:0/2327349591 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb00ac630 con 0x7fcbc015c9a0 2024-04-25T10:15:54.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbbffff640 1 -- 172.21.15.133:0/2327349591 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb00a8050 con 0x7fcbc015c9a0 2024-04-25T10:15:54.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 -- 172.21.15.133:0/2327349591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc015c9a0 msgr2=0x7fcbc00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 --2- 172.21.15.133:0/2327349591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc015c9a0 0x7fcbc00c5980 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0072670 tx=0x7fcbb001c4c0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 -- 172.21.15.133:0/2327349591 shutdown_connections 2024-04-25T10:15:54.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 --2- 172.21.15.133:0/2327349591 >> v2:172.21.15.162:3300/0 conn(0x7fcbc00b1f70 0x7fcbc006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 --2- 172.21.15.133:0/2327349591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc015c9a0 0x7fcbc00c5980 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 -- 172.21.15.133:0/2327349591 >> 172.21.15.133:0/2327349591 conn(0x7fcbc0067000 msgr2=0x7fcbc006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 -- 172.21.15.133:0/2327349591 shutdown_connections 2024-04-25T10:15:54.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 -- 172.21.15.133:0/2327349591 wait complete. 2024-04-25T10:15:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 Processor -- start 2024-04-25T10:15:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 -- start start 2024-04-25T10:15:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc00b1f70 0x7fcbc00ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc00ce330 0x7fcbc0148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbc015c9a0 0x7fcbc00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00ceb50 con 0x7fcbc00b1f70 2024-04-25T10:15:54.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc015cd80 con 0x7fcbc00ce330 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00ce150 con 0x7fcbc015c9a0 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc501f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc00b1f70 0x7fcbc00ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc501f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc00b1f70 0x7fcbc00ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55688/0 (socket says 172.21.15.133:55688) 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc501f640 1 -- 172.21.15.133:0/122026807 learned_addr learned my addr 172.21.15.133:0/122026807 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc501f640 1 -- 172.21.15.133:0/122026807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbc015c9a0 msgr2=0x7fcbc00cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc501f640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbc015c9a0 0x7fcbc00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc501f640 1 -- 172.21.15.133:0/122026807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc00ce330 msgr2=0x7fcbc0148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc501f640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc00ce330 0x7fcbc0148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc501f640 1 -- 172.21.15.133:0/122026807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc00cde70 con 0x7fcbc00b1f70 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc501f640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc00b1f70 0x7fcbc00ccd30 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fcbb001a150 tx=0x7fcbb001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbbdffb640 1 -- 172.21.15.133:0/122026807 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb00bb020 con 0x7fcbc00b1f70 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbbdffb640 1 -- 172.21.15.133:0/122026807 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb00b7420 con 0x7fcbc00b1f70 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 -- 172.21.15.133:0/122026807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc00cc250 con 0x7fcbc00b1f70 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbbdffb640 1 -- 172.21.15.133:0/122026807 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb00b1040 con 0x7fcbc00b1f70 2024-04-25T10:15:54.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.874+0000 7fcbc7283640 1 -- 172.21.15.133:0/122026807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc00cc810 con 0x7fcbc00b1f70 2024-04-25T10:15:54.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.878+0000 7fcbc7283640 1 -- 172.21.15.133:0/122026807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc006bc20 con 0x7fcbc00b1f70 2024-04-25T10:15:54.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.878+0000 7fcbbdffb640 1 -- 172.21.15.133:0/122026807 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcbb00be020 con 0x7fcbc00b1f70 2024-04-25T10:15:54.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.878+0000 7fcbbdffb640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcbac066f60 0x7fcbac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.878+0000 7fcbbdffb640 1 -- 172.21.15.133:0/122026807 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcbb011f420 con 0x7fcbc00b1f70 2024-04-25T10:15:54.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.878+0000 7fcbc481e640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcbac066f60 0x7fcbac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.878+0000 7fcbc481e640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcbac066f60 0x7fcbac069420 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0156cb0 tx=0x7fcbb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:54.882+0000 7fcbbdffb640 1 -- 172.21.15.133:0/122026807 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcbb0016110 con 0x7fcbc00b1f70 2024-04-25T10:15:55.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.074+0000 7fcbc7283640 1 -- 172.21.15.133:0/122026807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbc0068ef0 con 0x7fcbc00b1f70 2024-04-25T10:15:55.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.074+0000 7fcbbdffb640 1 -- 172.21.15.133:0/122026807 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4959 (secure 0 0 0) 0x7fcbb00f0330 con 0x7fcbc00b1f70 2024-04-25T10:15:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:55.074 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3454110484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3454110484},{"type":"v1","addr":"172.21.15.162:6835","nonce":3454110484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3345416592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3345416592},{"type":"v1","addr":"172.21.15.133:6835","nonce":3345416592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2924224565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2924224565},{"type":"v1","addr":"172.21.15.133:6837","nonce":2924224565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:13.097799+0000","modified":"2024-04-25T10:15:15.128366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10554":{"gid":10554,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/3763981385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3763981385},{"type":"v1","addr":"172.21.15.162:6833","nonce":3763981385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:15:55.075 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 99 2024-04-25T10:15:55.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 -- 172.21.15.133:0/122026807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcbac066f60 msgr2=0x7fcbac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcbac066f60 0x7fcbac069420 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0156cb0 tx=0x7fcbb4008040 comp rx=0 tx=0).stop 2024-04-25T10:15:55.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 -- 172.21.15.133:0/122026807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc00b1f70 msgr2=0x7fcbc00ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc00b1f70 0x7fcbc00ccd30 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fcbb001a150 tx=0x7fcbb001ad60 comp rx=0 tx=0).stop 2024-04-25T10:15:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 -- 172.21.15.133:0/122026807 shutdown_connections 2024-04-25T10:15:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcbc015c9a0 0x7fcbc00cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcbc00ce330 0x7fcbc0148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcbac066f60 0x7fcbac069420 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 --2- 172.21.15.133:0/122026807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc00b1f70 0x7fcbc00ccd30 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 -- 172.21.15.133:0/122026807 >> 172.21.15.133:0/122026807 conn(0x7fcbc0067000 msgr2=0x7fcbc0073790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 -- 172.21.15.133:0/122026807 shutdown_connections 2024-04-25T10:15:55.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.078+0000 7fcbc7283640 1 -- 172.21.15.133:0/122026807 wait complete. 2024-04-25T10:15:55.094 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:55.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.210+0000 7fee34588640 1 Processor -- start 2024-04-25T10:15:55.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.210+0000 7fee34588640 1 -- start start 2024-04-25T10:15:55.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee34588640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fee2c15db00 0x7fee2c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee34588640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee2c0d0bd0 0x7fee2c161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee34588640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee2c0ceb20 con 0x7fee2c0d0770 2024-04-25T10:15:55.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee34588640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c0b7740 con 0x7fee2c15db00 2024-04-25T10:15:55.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee34588640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0b7a20 con 0x7fee2c0d0bd0 2024-04-25T10:15:55.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31b23640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fee2c0d0770 0x7fee2c1614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45876/0 (socket says 172.21.15.133:45876) 2024-04-25T10:15:55.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31b23640 1 -- 172.21.15.133:0/867797423 learned_addr learned my addr 172.21.15.133:0/867797423 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:55.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036077346 0 0) 0x7fee2c0ceb20 con 0x7fee2c0d0770 2024-04-25T10:15:55.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee32324640 1 --2- 172.21.15.133:0/867797423 >> v2:172.21.15.162:3300/0 conn(0x7fee2c15db00 0x7fee2c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee14003530 con 0x7fee2c0d0770 2024-04-25T10:15:55.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee32b25640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee2c0d0bd0 0x7fee2c161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1642348132 0 0) 0x7fee14003530 con 0x7fee2c0d0770 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee2c0d0bd0 msgr2=0x7fee2c161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee2c0d0bd0 0x7fee2c161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 >> v2:172.21.15.162:3300/0 conn(0x7fee2c15db00 msgr2=0x7fee2c0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 --2- 172.21.15.133:0/867797423 >> v2:172.21.15.162:3300/0 conn(0x7fee2c15db00 0x7fee2c0b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c0b97f0 con 0x7fee2c0d0770 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fee20001680 con 0x7fee2c0d0770 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 >> v1:172.21.15.133:6789/0 conn(0x7fee2c0d0770 legacy=0x7fee2c1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee140039c0 0x7fee14003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee14004810 0x7fee14004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee2c0d0bd0 0x7fee140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0b7740 con 0x7fee140039c0 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0b7a20 con 0x7fee14004810 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1400bab0 con 0x7fee2c0d0bd0 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee32324640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee140039c0 0x7fee14003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee31b23640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee14004810 0x7fee14004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee32b25640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee2c0d0bd0 0x7fee140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee32324640 1 -- 172.21.15.133:0/867797423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee2c0d0bd0 msgr2=0x7fee140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee32324640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee2c0d0bd0 0x7fee140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee32324640 1 -- 172.21.15.133:0/867797423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee14004810 msgr2=0x7fee14004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee32324640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee14004810 0x7fee14004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.214+0000 7fee32324640 1 -- 172.21.15.133:0/867797423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee14003530 con 0x7fee140039c0 2024-04-25T10:15:55.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee32324640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee140039c0 0x7fee14003de0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7fee24007330 tx=0x7fee2401ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75995a7d77e0a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee240b2070 con 0x7fee140039c0 2024-04-25T10:15:55.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee240a5c10 con 0x7fee140039c0 2024-04-25T10:15:55.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31322640 1 -- 172.21.15.133:0/867797423 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24009be0 con 0x7fee140039c0 2024-04-25T10:15:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 -- 172.21.15.133:0/867797423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee140039c0 msgr2=0x7fee14003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 --2- 172.21.15.133:0/867797423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee140039c0 0x7fee14003de0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7fee24007330 tx=0x7fee2401ce10 comp rx=0 tx=0).stop 2024-04-25T10:15:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee32b25640 1 -- 172.21.15.133:0/867797423 reap_dead start 2024-04-25T10:15:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 -- 172.21.15.133:0/867797423 shutdown_connections 2024-04-25T10:15:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 -- 172.21.15.133:0/867797423 >> 172.21.15.133:0/867797423 conn(0x7fee2c0727c0 msgr2=0x7fee2c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 -- 172.21.15.133:0/867797423 shutdown_connections 2024-04-25T10:15:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 -- 172.21.15.133:0/867797423 wait complete. 2024-04-25T10:15:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 Processor -- start 2024-04-25T10:15:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 -- start start 2024-04-25T10:15:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee140039c0 0x7fee2c164e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee2c1653c0 0x7fee2c146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee2c1470a0 0x7fee2c147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0d0fe0 con 0x7fee2c1653c0 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0b7430 con 0x7fee2c1470a0 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee34588640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0d18a0 con 0x7fee140039c0 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31b23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee2c1653c0 0x7fee2c146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31b23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee2c1653c0 0x7fee2c146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55700/0 (socket says 172.21.15.133:55700) 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31b23640 1 -- 172.21.15.133:0/1252123312 learned_addr learned my addr 172.21.15.133:0/1252123312 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31b23640 1 -- 172.21.15.133:0/1252123312 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee140039c0 msgr2=0x7fee2c164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31b23640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee140039c0 0x7fee2c164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31b23640 1 -- 172.21.15.133:0/1252123312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee2c1470a0 msgr2=0x7fee2c147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31b23640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee2c1470a0 0x7fee2c147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31b23640 1 -- 172.21.15.133:0/1252123312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c067e80 con 0x7fee2c1653c0 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee31b23640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee2c1653c0 0x7fee2c146b60 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fee2000e7e0 tx=0x7fee2000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee2affd640 1 -- 172.21.15.133:0/1252123312 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2000c8d0 con 0x7fee2c1653c0 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee2affd640 1 -- 172.21.15.133:0/1252123312 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee20018070 con 0x7fee2c1653c0 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.218+0000 7fee2affd640 1 -- 172.21.15.133:0/1252123312 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2000ccd0 con 0x7fee2c1653c0 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.222+0000 7fee34588640 1 -- 172.21.15.133:0/1252123312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2c0cd400 con 0x7fee2c1653c0 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.222+0000 7fee34588640 1 -- 172.21.15.133:0/1252123312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c144e50 con 0x7fee2c1653c0 2024-04-25T10:15:55.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.222+0000 7fee34588640 1 -- 172.21.15.133:0/1252123312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c068ef0 con 0x7fee2c1653c0 2024-04-25T10:15:55.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.222+0000 7fee2affd640 1 -- 172.21.15.133:0/1252123312 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee2001d460 con 0x7fee2c1653c0 2024-04-25T10:15:55.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.222+0000 7fee2affd640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee18066f60 0x7fee18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.222+0000 7fee2affd640 1 -- 172.21.15.133:0/1252123312 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fee200850a0 con 0x7fee2c1653c0 2024-04-25T10:15:55.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.222+0000 7fee32324640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee18066f60 0x7fee18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.222+0000 7fee32324640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee18066f60 0x7fee18069420 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fee240af000 tx=0x7fee24004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.226+0000 7fee2affd640 1 -- 172.21.15.133:0/1252123312 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee20055f20 con 0x7fee2c1653c0 2024-04-25T10:15:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.373+0000 7fee34588640 1 -- 172.21.15.133:0/1252123312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee2c0b2750 con 0x7fee2c1653c0 2024-04-25T10:15:55.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.377+0000 7fee2affd640 1 -- 172.21.15.133:0/1252123312 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16908 (secure 0 0 0) 0x7fee20055820 con 0x7fee2c1653c0 2024-04-25T10:15:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:55.378 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":157,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:15:14.762327+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:11.332375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:12.104303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:55.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 -- 172.21.15.133:0/1252123312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee18066f60 msgr2=0x7fee18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee18066f60 0x7fee18069420 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fee240af000 tx=0x7fee24004ac0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 -- 172.21.15.133:0/1252123312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee2c1653c0 msgr2=0x7fee2c146b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee2c1653c0 0x7fee2c146b60 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fee2000e7e0 tx=0x7fee2000eca0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 -- 172.21.15.133:0/1252123312 shutdown_connections 2024-04-25T10:15:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee2c1470a0 0x7fee2c147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee18066f60 0x7fee18069420 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee2c1653c0 0x7fee2c146b60 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 --2- 172.21.15.133:0/1252123312 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee140039c0 0x7fee2c164e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 -- 172.21.15.133:0/1252123312 >> 172.21.15.133:0/1252123312 conn(0x7fee2c0727c0 msgr2=0x7fee2c0d2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 -- 172.21.15.133:0/1252123312 shutdown_connections 2024-04-25T10:15:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.381+0000 7fee34588640 1 -- 172.21.15.133:0/1252123312 wait complete. 2024-04-25T10:15:55.396 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:15:11.332375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:15:12.104303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:55.397 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:55.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56ef0f640 1 Processor -- start 2024-04-25T10:15:55.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56ef0f640 1 -- start start 2024-04-25T10:15:55.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56ef0f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd568145170 0x7fd56806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56ef0f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd56815ced0 0x7fd56815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56ef0f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5680ce2e0 con 0x7fd56806e4d0 2024-04-25T10:15:55.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56ef0f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56806db00 con 0x7fd56815ced0 2024-04-25T10:15:55.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56ef0f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd568145550 con 0x7fd568145170 2024-04-25T10:15:55.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56d4ac640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd56815ced0 0x7fd56815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56d4ac640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd56815ced0 0x7fd56815d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55716/0 (socket says 172.21.15.133:55716) 2024-04-25T10:15:55.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56d4ac640 1 -- 172.21.15.133:0/353363732 learned_addr learned my addr 172.21.15.133:0/353363732 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:55.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56ccab640 1 --2- 172.21.15.133:0/353363732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd568145170 0x7fd56806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56d4ac640 1 -- 172.21.15.133:0/353363732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd568145170 msgr2=0x7fd56806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56d4ac640 1 --2- 172.21.15.133:0/353363732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd568145170 0x7fd56806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56d4ac640 1 -- 172.21.15.133:0/353363732 >> v1:172.21.15.162:6789/0 conn(0x7fd56806e4d0 legacy=0x7fd5680cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.517+0000 7fd56d4ac640 1 -- 172.21.15.133:0/353363732 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56806d4b0 con 0x7fd56815ced0 2024-04-25T10:15:55.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56d4ac640 1 --2- 172.21.15.133:0/353363732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd56815ced0 0x7fd56815d360 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7fd560009e50 tx=0x7fd56001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143d9411dc92599e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd5677fe640 1 -- 172.21.15.133:0/353363732 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5600a5700 con 0x7fd56815ced0 2024-04-25T10:15:55.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd5677fe640 1 -- 172.21.15.133:0/353363732 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5600a58a0 con 0x7fd56815ced0 2024-04-25T10:15:55.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd5677fe640 1 -- 172.21.15.133:0/353363732 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5600a5c70 con 0x7fd56815ced0 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 -- 172.21.15.133:0/353363732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd56815ced0 msgr2=0x7fd56815d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 --2- 172.21.15.133:0/353363732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd56815ced0 0x7fd56815d360 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7fd560009e50 tx=0x7fd56001ca30 comp rx=0 tx=0).stop 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 -- 172.21.15.133:0/353363732 shutdown_connections 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 --2- 172.21.15.133:0/353363732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd56815ced0 0x7fd56815d360 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 --2- 172.21.15.133:0/353363732 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd568145170 0x7fd56806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 -- 172.21.15.133:0/353363732 >> 172.21.15.133:0/353363732 conn(0x7fd568066f70 msgr2=0x7fd56806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 -- 172.21.15.133:0/353363732 shutdown_connections 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 -- 172.21.15.133:0/353363732 wait complete. 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 Processor -- start 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 -- start start 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd56806e4d0 0x7fd5681521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd568145170 0x7fd568136a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd56815ced0 0x7fd568134050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd568168610 con 0x7fd56815ced0 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd568168490 con 0x7fd568145170 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ef0f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd568168790 con 0x7fd56806e4d0 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56d4ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd56815ced0 0x7fd568134050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56d4ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd56815ced0 0x7fd568134050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55730/0 (socket says 172.21.15.133:55730) 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56d4ac640 1 -- 172.21.15.133:0/554567971 learned_addr learned my addr 172.21.15.133:0/554567971 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd567fff640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd568145170 0x7fd568136a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56ccab640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd56806e4d0 0x7fd5681521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56d4ac640 1 -- 172.21.15.133:0/554567971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd56806e4d0 msgr2=0x7fd5681521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56d4ac640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd56806e4d0 0x7fd5681521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56d4ac640 1 -- 172.21.15.133:0/554567971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd568145170 msgr2=0x7fd568136a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56d4ac640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd568145170 0x7fd568136a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56d4ac640 1 -- 172.21.15.133:0/554567971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5681307d0 con 0x7fd56815ced0 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd56d4ac640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd56815ced0 0x7fd568134050 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fd5600af040 tx=0x7fd5600197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd5657fa640 1 -- 172.21.15.133:0/554567971 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5600b8470 con 0x7fd56815ced0 2024-04-25T10:15:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.521+0000 7fd5657fa640 1 -- 172.21.15.133:0/554567971 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5600b8650 con 0x7fd56815ced0 2024-04-25T10:15:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.525+0000 7fd56ef0f640 1 -- 172.21.15.133:0/554567971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5681373a0 con 0x7fd56815ced0 2024-04-25T10:15:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.525+0000 7fd5657fa640 1 -- 172.21.15.133:0/554567971 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5600b8a40 con 0x7fd56815ced0 2024-04-25T10:15:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.525+0000 7fd56ef0f640 1 -- 172.21.15.133:0/554567971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd568130a70 con 0x7fd56815ced0 2024-04-25T10:15:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.525+0000 7fd56ef0f640 1 -- 172.21.15.133:0/554567971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd568068e80 con 0x7fd56815ced0 2024-04-25T10:15:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.525+0000 7fd5657fa640 1 -- 172.21.15.133:0/554567971 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5600c0020 con 0x7fd56815ced0 2024-04-25T10:15:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.525+0000 7fd5657fa640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd54c066f90 0x7fd54c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.525+0000 7fd5657fa640 1 -- 172.21.15.133:0/554567971 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd56011ff30 con 0x7fd56815ced0 2024-04-25T10:15:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.525+0000 7fd56ccab640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd54c066f90 0x7fd54c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.525+0000 7fd56ccab640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd54c066f90 0x7fd54c069450 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00b930 tx=0x7fd55c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.529+0000 7fd5657fa640 1 -- 172.21.15.133:0/554567971 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd560016620 con 0x7fd56815ced0 2024-04-25T10:15:55.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.685+0000 7fd56ef0f640 1 -- 172.21.15.133:0/554567971 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd56813e670 con 0x7fd56815ced0 2024-04-25T10:15:55.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.689+0000 7fd5657fa640 1 -- 172.21.15.133:0/554567971 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5600f0ed0 con 0x7fd56815ced0 2024-04-25T10:15:55.689 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:15:55.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.689+0000 7fd56ef0f640 1 -- 172.21.15.133:0/554567971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd54c066f90 msgr2=0x7fd54c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.689+0000 7fd56ef0f640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd54c066f90 0x7fd54c069450 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00b930 tx=0x7fd55c008040 comp rx=0 tx=0).stop 2024-04-25T10:15:55.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.689+0000 7fd56ef0f640 1 -- 172.21.15.133:0/554567971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd56815ced0 msgr2=0x7fd568134050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.689+0000 7fd56ef0f640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd56815ced0 0x7fd568134050 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7fd5600af040 tx=0x7fd5600197d0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.689+0000 7fd56ef0f640 1 -- 172.21.15.133:0/554567971 shutdown_connections 2024-04-25T10:15:55.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.689+0000 7fd56ef0f640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd54c066f90 0x7fd54c069450 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.689+0000 7fd56ef0f640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd56815ced0 0x7fd568134050 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.689+0000 7fd56ef0f640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd568145170 0x7fd568136a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.689+0000 7fd56ef0f640 1 --2- 172.21.15.133:0/554567971 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd56806e4d0 0x7fd5681521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.689+0000 7fd56ef0f640 1 -- 172.21.15.133:0/554567971 >> 172.21.15.133:0/554567971 conn(0x7fd568066f70 msgr2=0x7fd56813dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.693+0000 7fd56ef0f640 1 -- 172.21.15.133:0/554567971 shutdown_connections 2024-04-25T10:15:55.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.693+0000 7fd56ef0f640 1 -- 172.21.15.133:0/554567971 wait complete. 2024-04-25T10:15:55.709 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:15:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f40b7640 1 Processor -- start 2024-04-25T10:15:55.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f40b7640 1 -- start start 2024-04-25T10:15:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f40b7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec0d02d0 0x7f43ec0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f40b7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f43ec0b2cf0 0x7f43ec0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f40b7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ec0cc530 con 0x7f43ec15b850 2024-04-25T10:15:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f40b7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ec0cd140 con 0x7f43ec0b2cf0 2024-04-25T10:15:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f40b7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec0cc810 con 0x7f43ec0d02d0 2024-04-25T10:15:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f1e53640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f43ec15b850 0x7f43ec0cfbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45892/0 (socket says 172.21.15.133:45892) 2024-04-25T10:15:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f1e53640 1 -- 172.21.15.133:0/2564571193 learned_addr learned my addr 172.21.15.133:0/2564571193 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f1652640 1 --2- 172.21.15.133:0/2564571193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec0d02d0 0x7f43ec0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f2654640 1 --2- 172.21.15.133:0/2564571193 >> v2:172.21.15.162:3300/0 conn(0x7f43ec0b2cf0 0x7f43ec0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999877875 0 0) 0x7f43ec0cc530 con 0x7f43ec15b850 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d8003530 con 0x7f43ec15b850 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 847883121 0 0) 0x7f43d8003530 con 0x7f43ec15b850 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec0d02d0 msgr2=0x7f43ec0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 --2- 172.21.15.133:0/2564571193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec0d02d0 0x7f43ec0b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 >> v2:172.21.15.162:3300/0 conn(0x7f43ec0b2cf0 msgr2=0x7f43ec0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 --2- 172.21.15.133:0/2564571193 >> v2:172.21.15.162:3300/0 conn(0x7f43ec0b2cf0 0x7f43ec0cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec0ccaf0 con 0x7f43ec15b850 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f43e8001820 con 0x7f43ec15b850 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 >> v1:172.21.15.133:6789/0 conn(0x7f43ec15b850 legacy=0x7f43ec0cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 --2- 172.21.15.133:0/2564571193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec0d02d0 0x7f43d80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 --2- 172.21.15.133:0/2564571193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d8003f40 0x7f43d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 --2- 172.21.15.133:0/2564571193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43d8004d50 0x7f43d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec0cd140 con 0x7f43d8004d50 2024-04-25T10:15:55.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec0cc810 con 0x7f43d8003f40 2024-04-25T10:15:55.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d800b9d0 con 0x7f43ec0d02d0 2024-04-25T10:15:55.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f2654640 1 --2- 172.21.15.133:0/2564571193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43d8004d50 0x7f43d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f1e53640 1 --2- 172.21.15.133:0/2564571193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec0d02d0 0x7f43d80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f2654640 1 -- 172.21.15.133:0/2564571193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec0d02d0 msgr2=0x7f43d80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f2654640 1 --2- 172.21.15.133:0/2564571193 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec0d02d0 0x7f43d80039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f2654640 1 -- 172.21.15.133:0/2564571193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d8003f40 msgr2=0x7f43d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f2654640 1 --2- 172.21.15.133:0/2564571193 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43d8003f40 0x7f43d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.821+0000 7f43f2654640 1 -- 172.21.15.133:0/2564571193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d8003530 con 0x7f43d8004d50 2024-04-25T10:15:55.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f2654640 1 --2- 172.21.15.133:0/2564571193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43d8004d50 0x7f43d80051d0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009fc0 tx=0x7f43e401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b091728d77af9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40b2070 con 0x7f43d8004d50 2024-04-25T10:15:55.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e40a5cd0 con 0x7f43d8004d50 2024-04-25T10:15:55.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f0e51640 1 -- 172.21.15.133:0/2564571193 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4009c10 con 0x7f43d8004d50 2024-04-25T10:15:55.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 -- 172.21.15.133:0/2564571193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43d8004d50 msgr2=0x7f43d80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 --2- 172.21.15.133:0/2564571193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43d8004d50 0x7f43d80051d0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009fc0 tx=0x7f43e401cc60 comp rx=0 tx=0).stop 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f2654640 1 -- 172.21.15.133:0/2564571193 reap_dead start 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 -- 172.21.15.133:0/2564571193 shutdown_connections 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 -- 172.21.15.133:0/2564571193 >> 172.21.15.133:0/2564571193 conn(0x7f43ec09ab90 msgr2=0x7f43ec0d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 -- 172.21.15.133:0/2564571193 shutdown_connections 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 -- 172.21.15.133:0/2564571193 wait complete. 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 Processor -- start 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 -- start start 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43ec1ee5a0 0x7f43ec15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec1f1bc0 0x7f43ec0b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43ec0b69f0 0x7f43ec1f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec15bc30 con 0x7f43ec0b69f0 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec0cc1e0 con 0x7f43ec1ee5a0 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f40b7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec15bdb0 con 0x7f43ec1f1bc0 2024-04-25T10:15:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f2654640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43ec0b69f0 0x7f43ec1f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f2654640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43ec0b69f0 0x7f43ec1f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55746/0 (socket says 172.21.15.133:55746) 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.825+0000 7f43f2654640 1 -- 172.21.15.133:0/1098656668 learned_addr learned my addr 172.21.15.133:0/1098656668 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f1652640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec1f1bc0 0x7f43ec0b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f1e53640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43ec1ee5a0 0x7f43ec15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f2654640 1 -- 172.21.15.133:0/1098656668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec1f1bc0 msgr2=0x7f43ec0b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f2654640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec1f1bc0 0x7f43ec0b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f2654640 1 -- 172.21.15.133:0/1098656668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43ec1ee5a0 msgr2=0x7f43ec15bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f2654640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43ec1ee5a0 0x7f43ec15bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f2654640 1 -- 172.21.15.133:0/1098656668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec1f2040 con 0x7f43ec0b69f0 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f2654640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43ec0b69f0 0x7f43ec1f46a0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f43e4002240 tx=0x7f43e4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43e27fc640 1 -- 172.21.15.133:0/1098656668 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40bd070 con 0x7f43ec0b69f0 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43e27fc640 1 -- 172.21.15.133:0/1098656668 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e40a5ad0 con 0x7f43ec0b69f0 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f40b7640 1 -- 172.21.15.133:0/1098656668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ec15c630 con 0x7f43ec0b69f0 2024-04-25T10:15:55.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43e27fc640 1 -- 172.21.15.133:0/1098656668 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40b2040 con 0x7f43ec0b69f0 2024-04-25T10:15:55.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f40b7640 1 -- 172.21.15.133:0/1098656668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ec15cbf0 con 0x7f43ec0b69f0 2024-04-25T10:15:55.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f40b7640 1 -- 172.21.15.133:0/1098656668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ec068e80 con 0x7f43ec0b69f0 2024-04-25T10:15:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43e27fc640 1 -- 172.21.15.133:0/1098656668 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43e40c0070 con 0x7f43ec0b69f0 2024-04-25T10:15:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43e27fc640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43d0066f90 0x7f43d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43e27fc640 1 -- 172.21.15.133:0/1098656668 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f43e411ece0 con 0x7f43ec0b69f0 2024-04-25T10:15:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f1e53640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43d0066f90 0x7f43d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.829+0000 7f43f1e53640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43d0066f90 0x7f43d0069450 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000c10 tx=0x7f43e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.833+0000 7f43e27fc640 1 -- 172.21.15.133:0/1098656668 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43e40169d0 con 0x7f43ec0b69f0 2024-04-25T10:15:55.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 -- 172.21.15.133:0/1098656668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43ec0b3d50 con 0x7f43ec0b69f0 2024-04-25T10:15:55.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43e27fc640 1 -- 172.21.15.133:0/1098656668 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v157) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43e40ef510 con 0x7f43ec0b69f0 2024-04-25T10:15:55.993 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 -- 172.21.15.133:0/1098656668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43d0066f90 msgr2=0x7f43d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43d0066f90 0x7f43d0069450 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000c10 tx=0x7f43e8009040 comp rx=0 tx=0).stop 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 -- 172.21.15.133:0/1098656668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43ec0b69f0 msgr2=0x7f43ec1f46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43ec0b69f0 0x7f43ec1f46a0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f43e4002240 tx=0x7f43e4009750 comp rx=0 tx=0).stop 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 -- 172.21.15.133:0/1098656668 shutdown_connections 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f43d0066f90 0x7f43d0069450 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43ec0b69f0 0x7f43ec1f46a0 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f43ec1f1bc0 0x7f43ec0b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 --2- 172.21.15.133:0/1098656668 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f43ec1ee5a0 0x7f43ec15bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 -- 172.21.15.133:0/1098656668 >> 172.21.15.133:0/1098656668 conn(0x7f43ec09ab90 msgr2=0x7f43ec15a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 -- 172.21.15.133:0/1098656668 shutdown_connections 2024-04-25T10:15:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:55.993+0000 7f43f40b7640 1 -- 172.21.15.133:0/1098656668 wait complete. 2024-04-25T10:15:56.012 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:15:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47efbb7640 1 Processor -- start 2024-04-25T10:15:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47efbb7640 1 -- start start 2024-04-25T10:15:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47efbb7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e80cd820 0x7f47e80cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47efbb7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f47e815ced0 0x7f47e815d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47efbb7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e80ce2e0 con 0x7f47e8145170 2024-04-25T10:15:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47efbb7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e806db00 con 0x7f47e815ced0 2024-04-25T10:15:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47efbb7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e8145550 con 0x7f47e80cd820 2024-04-25T10:15:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ed953640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f47e8145170 0x7f47e806df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:45902/0 (socket says 172.21.15.133:45902) 2024-04-25T10:15:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ed953640 1 -- 172.21.15.133:0/191758085 learned_addr learned my addr 172.21.15.133:0/191758085 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ee154640 1 --2- 172.21.15.133:0/191758085 >> v2:172.21.15.162:3300/0 conn(0x7f47e815ced0 0x7f47e815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547446377 0 0) 0x7f47e80ce2e0 con 0x7f47e8145170 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ed152640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e80cd820 0x7f47e80cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d4003530 con 0x7f47e8145170 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3435850232 0 0) 0x7f47d4003530 con 0x7f47e8145170 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e80cd820 msgr2=0x7f47e80cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e80cd820 0x7f47e80cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 >> v2:172.21.15.162:3300/0 conn(0x7f47e815ced0 msgr2=0x7f47e815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 --2- 172.21.15.133:0/191758085 >> v2:172.21.15.162:3300/0 conn(0x7f47e815ced0 0x7f47e815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e806d4b0 con 0x7f47e8145170 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f47e4001680 con 0x7f47e8145170 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 >> v1:172.21.15.133:6789/0 conn(0x7f47e8145170 legacy=0x7f47e806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d40039c0 0x7f47d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e80cd820 0x7f47d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 0x7f47d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e806db00 con 0x7f47d4004d50 2024-04-25T10:15:56.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e8145550 con 0x7f47d40039c0 2024-04-25T10:15:56.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d400b9d0 con 0x7f47e80cd820 2024-04-25T10:15:56.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ee154640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 0x7f47d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ed953640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d40039c0 0x7f47d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ee154640 1 -- 172.21.15.133:0/191758085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e80cd820 msgr2=0x7f47d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ee154640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e80cd820 0x7f47d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ee154640 1 -- 172.21.15.133:0/191758085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d40039c0 msgr2=0x7f47d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ee154640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d40039c0 0x7f47d4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ee154640 1 -- 172.21.15.133:0/191758085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4003530 con 0x7f47d4004d50 2024-04-25T10:15:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.137+0000 7f47ed953640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47d40039c0 0x7f47d4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ee154640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 0x7f47d40051d0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009fc0 tx=0x7f47e001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e8d2a6d119b3e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e00b2070 con 0x7f47d4004d50 2024-04-25T10:15:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e00a5cd0 con 0x7f47d4004d50 2024-04-25T10:15:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ec951640 1 -- 172.21.15.133:0/191758085 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0009c10 con 0x7f47d4004d50 2024-04-25T10:15:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 -- 172.21.15.133:0/191758085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 msgr2=0x7f47d40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 --2- 172.21.15.133:0/191758085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 0x7f47d40051d0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009fc0 tx=0x7f47e001ccc0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ee154640 1 -- 172.21.15.133:0/191758085 reap_dead start 2024-04-25T10:15:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 -- 172.21.15.133:0/191758085 shutdown_connections 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 -- 172.21.15.133:0/191758085 >> 172.21.15.133:0/191758085 conn(0x7f47e8066f70 msgr2=0x7f47e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 -- 172.21.15.133:0/191758085 shutdown_connections 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 -- 172.21.15.133:0/191758085 wait complete. 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 Processor -- start 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 -- start start 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 0x7f47e813bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e8138ae0 0x7f47e8138f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e813d090 0x7f47e813d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e815cb70 con 0x7f47d4004d50 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e806e6a0 con 0x7f47e813d090 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47efbb7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80cdca0 con 0x7f47e8138ae0 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ed953640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 0x7f47e813bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ed953640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 0x7f47e813bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55760/0 (socket says 172.21.15.133:55760) 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ed953640 1 -- 172.21.15.133:0/1765018871 learned_addr learned my addr 172.21.15.133:0/1765018871 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ee154640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e813d090 0x7f47e813d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ed152640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e8138ae0 0x7f47e8138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ed953640 1 -- 172.21.15.133:0/1765018871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e8138ae0 msgr2=0x7f47e8138f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ed953640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e8138ae0 0x7f47e8138f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ed953640 1 -- 172.21.15.133:0/1765018871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e813d090 msgr2=0x7f47e813d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ed953640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e813d090 0x7f47e813d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ed953640 1 -- 172.21.15.133:0/1765018871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e81394a0 con 0x7f47d4004d50 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ed152640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e8138ae0 0x7f47e8138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ee154640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e813d090 0x7f47e813d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47ed953640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 0x7f47e813bb00 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f47e400dae0 tx=0x7f47e4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.141+0000 7f47de7fc640 1 -- 172.21.15.133:0/1765018871 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4019070 con 0x7f47d4004d50 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.145+0000 7f47de7fc640 1 -- 172.21.15.133:0/1765018871 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e40074e0 con 0x7f47d4004d50 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.145+0000 7f47efbb7640 1 -- 172.21.15.133:0/1765018871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e813de00 con 0x7f47d4004d50 2024-04-25T10:15:56.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.145+0000 7f47de7fc640 1 -- 172.21.15.133:0/1765018871 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e40049f0 con 0x7f47d4004d50 2024-04-25T10:15:56.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.145+0000 7f47efbb7640 1 -- 172.21.15.133:0/1765018871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e813e3c0 con 0x7f47d4004d50 2024-04-25T10:15:56.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.145+0000 7f47efbb7640 1 -- 172.21.15.133:0/1765018871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e8068e80 con 0x7f47d4004d50 2024-04-25T10:15:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.145+0000 7f47de7fc640 1 -- 172.21.15.133:0/1765018871 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47e401c070 con 0x7f47d4004d50 2024-04-25T10:15:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.145+0000 7f47de7fc640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47cc066f60 0x7f47cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.145+0000 7f47ed152640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47cc066f60 0x7f47cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.145+0000 7f47de7fc640 1 -- 172.21.15.133:0/1765018871 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f47e4084d90 con 0x7f47d4004d50 2024-04-25T10:15:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.145+0000 7f47ed152640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47cc066f60 0x7f47cc069420 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f47d80047c0 tx=0x7f47d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.149+0000 7f47de7fc640 1 -- 172.21.15.133:0/1765018871 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47e4055ca0 con 0x7f47d4004d50 2024-04-25T10:15:56.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.297+0000 7f47efbb7640 1 -- 172.21.15.133:0/1765018871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47e815f270 con 0x7f47d4004d50 2024-04-25T10:15:56.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.297+0000 7f47de7fc640 1 -- 172.21.15.133:0/1765018871 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v157) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47e40555a0 con 0x7f47d4004d50 2024-04-25T10:15:56.298 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.297+0000 7f47efbb7640 1 -- 172.21.15.133:0/1765018871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47cc066f60 msgr2=0x7f47cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.297+0000 7f47efbb7640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47cc066f60 0x7f47cc069420 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f47d80047c0 tx=0x7f47d8008040 comp rx=0 tx=0).stop 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.297+0000 7f47efbb7640 1 -- 172.21.15.133:0/1765018871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 msgr2=0x7f47e813bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.297+0000 7f47efbb7640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 0x7f47e813bb00 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f47e400dae0 tx=0x7f47e4007e00 comp rx=0 tx=0).stop 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.301+0000 7f47efbb7640 1 -- 172.21.15.133:0/1765018871 shutdown_connections 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.301+0000 7f47efbb7640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47e813d090 0x7f47e813d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.301+0000 7f47efbb7640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47e8138ae0 0x7f47e8138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.301+0000 7f47efbb7640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f47cc066f60 0x7f47cc069420 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.301+0000 7f47efbb7640 1 --2- 172.21.15.133:0/1765018871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47d4004d50 0x7f47e813bb00 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.301+0000 7f47efbb7640 1 -- 172.21.15.133:0/1765018871 >> 172.21.15.133:0/1765018871 conn(0x7f47e8066f70 msgr2=0x7f47e815e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.301+0000 7f47efbb7640 1 -- 172.21.15.133:0/1765018871 shutdown_connections 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.301+0000 7f47efbb7640 1 -- 172.21.15.133:0/1765018871 wait complete. 2024-04-25T10:15:56.316 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.433+0000 7f7232390640 1 Processor -- start 2024-04-25T10:15:56.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.433+0000 7f7232390640 1 -- start start 2024-04-25T10:15:56.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f7232390640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c159dc0 0x7f722c0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f7232390640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c0ce880 con 0x7f722c15e220 2024-04-25T10:15:56.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f7232390640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c0c49e0 con 0x7f722c15a1a0 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f7232390640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0ceb20 con 0x7f722c159dc0 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f723092d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f722c15e220 0x7f722c06d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44090/0 (socket says 172.21.15.133:44090) 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f723092d640 1 -- 172.21.15.133:0/3704923211 learned_addr learned my addr 172.21.15.133:0/3704923211 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317333548 0 0) 0x7f722c0ce880 con 0x7f722c15e220 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722bfff640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c159dc0 0x7f722c0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7214003530 con 0x7f722c15e220 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 845270065 0 0) 0x7f7214003530 con 0x7f722c15e220 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c159dc0 msgr2=0x7f722c0cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c159dc0 0x7f722c0cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 >> v1:172.21.15.162:6789/0 conn(0x7f722c15a1a0 legacy=0x7f722c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c0cbfa0 con 0x7f722c15e220 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7224001a20 con 0x7f722c15e220 2024-04-25T10:15:56.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 >> v1:172.21.15.133:6789/0 conn(0x7f722c15e220 legacy=0x7f722c06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c159dc0 0x7f72140039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7214003f70 0x7f7214004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7214004d50 0x7f72140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0ceb20 con 0x7f7214003f70 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721400bab0 con 0x7f7214004d50 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721400dec0 con 0x7f722c159dc0 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722b7fe640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7214003f70 0x7f7214004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722bfff640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c159dc0 0x7f72140039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f723092d640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7214004d50 0x7f72140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722b7fe640 1 -- 172.21.15.133:0/3704923211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c159dc0 msgr2=0x7f72140039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722b7fe640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f722c159dc0 0x7f72140039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722b7fe640 1 -- 172.21.15.133:0/3704923211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7214004d50 msgr2=0x7f72140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722b7fe640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7214004d50 0x7f72140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.437+0000 7f722b7fe640 1 -- 172.21.15.133:0/3704923211 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214003530 con 0x7f7214003f70 2024-04-25T10:15:56.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722b7fe640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7214003f70 0x7f7214004810 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f721c001780 tx=0x7f721c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b32ed9c2c51cd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0013a0 con 0x7f7214003f70 2024-04-25T10:15:56.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c0052f0 con 0x7f7214003f70 2024-04-25T10:15:56.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722affd640 1 -- 172.21.15.133:0/3704923211 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c003840 con 0x7f7214003f70 2024-04-25T10:15:56.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 -- 172.21.15.133:0/3704923211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7214003f70 msgr2=0x7f7214004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 --2- 172.21.15.133:0/3704923211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7214003f70 0x7f7214004810 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f721c001780 tx=0x7f721c01c420 comp rx=0 tx=0).stop 2024-04-25T10:15:56.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f723092d640 1 -- 172.21.15.133:0/3704923211 reap_dead start 2024-04-25T10:15:56.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 -- 172.21.15.133:0/3704923211 shutdown_connections 2024-04-25T10:15:56.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 -- 172.21.15.133:0/3704923211 >> 172.21.15.133:0/3704923211 conn(0x7f722c067000 msgr2=0x7f722c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 -- 172.21.15.133:0/3704923211 shutdown_connections 2024-04-25T10:15:56.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 -- 172.21.15.133:0/3704923211 wait complete. 2024-04-25T10:15:56.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 Processor -- start 2024-04-25T10:15:56.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 -- start start 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7214003f70 0x7f722c155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f722c0c68f0 0x7f722c0c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c14f710 0x7f722c14fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0c47c0 con 0x7f722c0c68f0 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0c4640 con 0x7f722c14f710 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f7232390640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0c4cc0 con 0x7f7214003f70 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f722c0c68f0 0x7f722c0c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f722c0c68f0 0x7f722c0c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38946/0 (socket says 172.21.15.133:38946) 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722b7fe640 1 -- 172.21.15.133:0/523671478 learned_addr learned my addr 172.21.15.133:0/523671478 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722b7fe640 1 -- 172.21.15.133:0/523671478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7214003f70 msgr2=0x7f722c155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722b7fe640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7214003f70 0x7f722c155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722b7fe640 1 -- 172.21.15.133:0/523671478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c14f710 msgr2=0x7f722c14fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722b7fe640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c14f710 0x7f722c14fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.441+0000 7f722b7fe640 1 -- 172.21.15.133:0/523671478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c150100 con 0x7f722c0c68f0 2024-04-25T10:15:56.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f722b7fe640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f722c0c68f0 0x7f722c0c6d70 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f721c001780 tx=0x7f7214005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f7228ff9640 1 -- 172.21.15.133:0/523671478 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0054a0 con 0x7f722c0c68f0 2024-04-25T10:15:56.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f7228ff9640 1 -- 172.21.15.133:0/523671478 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c0bb070 con 0x7f722c0c68f0 2024-04-25T10:15:56.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f7232390640 1 -- 172.21.15.133:0/523671478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722c1505e0 con 0x7f722c0c68f0 2024-04-25T10:15:56.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f7228ff9640 1 -- 172.21.15.133:0/523671478 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0069e0 con 0x7f722c0c68f0 2024-04-25T10:15:56.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f7232390640 1 -- 172.21.15.133:0/523671478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722c0c77b0 con 0x7f722c0c68f0 2024-04-25T10:15:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f7232390640 1 -- 172.21.15.133:0/523671478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722c068ef0 con 0x7f722c0c68f0 2024-04-25T10:15:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f7228ff9640 1 -- 172.21.15.133:0/523671478 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f721c0b4040 con 0x7f722c0c68f0 2024-04-25T10:15:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f7228ff9640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7218066f60 0x7f7218069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f7228ff9640 1 -- 172.21.15.133:0/523671478 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f721c11f480 con 0x7f722c0c68f0 2024-04-25T10:15:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f722bfff640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7218066f60 0x7f7218069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.445+0000 7f722bfff640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7218066f60 0x7f7218069420 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f722c06ed20 tx=0x7f7220008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.449+0000 7f7228ff9640 1 -- 172.21.15.133:0/523671478 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f721c016130 con 0x7f722c0c68f0 2024-04-25T10:15:56.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.625+0000 7f7232390640 1 -- 172.21.15.133:0/523671478 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f722c1568d0 con 0x7f722c0c68f0 2024-04-25T10:15:56.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7228ff9640 1 -- 172.21.15.133:0/523671478 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4959 (secure 0 0 0) 0x7f721c0aa3a0 con 0x7f722c0c68f0 2024-04-25T10:15:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:56.629 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3454110484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3454110484},{"type":"v1","addr":"172.21.15.162:6835","nonce":3454110484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3345416592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3345416592},{"type":"v1","addr":"172.21.15.133:6835","nonce":3345416592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2924224565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2924224565},{"type":"v1","addr":"172.21.15.133:6837","nonce":2924224565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:13.097799+0000","modified":"2024-04-25T10:15:15.128366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10554":{"gid":10554,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/3763981385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3763981385},{"type":"v1","addr":"172.21.15.162:6833","nonce":3763981385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:15:56.629 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 99 2024-04-25T10:15:56.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 -- 172.21.15.133:0/523671478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7218066f60 msgr2=0x7f7218069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7218066f60 0x7f7218069420 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f722c06ed20 tx=0x7f7220008040 comp rx=0 tx=0).stop 2024-04-25T10:15:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 -- 172.21.15.133:0/523671478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f722c0c68f0 msgr2=0x7f722c0c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f722c0c68f0 0x7f722c0c6d70 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f721c001780 tx=0x7f7214005840 comp rx=0 tx=0).stop 2024-04-25T10:15:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 -- 172.21.15.133:0/523671478 shutdown_connections 2024-04-25T10:15:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f722c14f710 0x7f722c14fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7218066f60 0x7f7218069420 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f722c0c68f0 0x7f722c0c6d70 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 --2- 172.21.15.133:0/523671478 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7214003f70 0x7f722c155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 -- 172.21.15.133:0/523671478 >> 172.21.15.133:0/523671478 conn(0x7f722c067000 msgr2=0x7f722c0ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 -- 172.21.15.133:0/523671478 shutdown_connections 2024-04-25T10:15:56.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.629+0000 7f7232390640 1 -- 172.21.15.133:0/523671478 wait complete. 2024-04-25T10:15:56.647 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:56.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83ddd82640 1 Processor -- start 2024-04-25T10:15:56.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83ddd82640 1 -- start start 2024-04-25T10:15:56.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83ddd82640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d815db00 0x7f83d80b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83ddd82640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f83d80d0bd0 0x7f83d8161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83ddd82640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d80ceb20 con 0x7f83d80d0770 2024-04-25T10:15:56.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83ddd82640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d80b7740 con 0x7f83d80d0bd0 2024-04-25T10:15:56.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83ddd82640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80b7a20 con 0x7f83d815db00 2024-04-25T10:15:56.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83d77fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d815db00 0x7f83d80b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83d7fff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f83d80d0bd0 0x7f83d8161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83d77fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d815db00 0x7f83d80b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:43674/0 (socket says 172.21.15.133:43674) 2024-04-25T10:15:56.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83d77fe640 1 -- 172.21.15.133:0/1119311317 learned_addr learned my addr 172.21.15.133:0/1119311317 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:56.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83d77fe640 1 -- 172.21.15.133:0/1119311317 >> v2:172.21.15.162:3300/0 conn(0x7f83d80d0bd0 msgr2=0x7f83d8161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83d77fe640 1 --2- 172.21.15.133:0/1119311317 >> v2:172.21.15.162:3300/0 conn(0x7f83d80d0bd0 0x7f83d8161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83d77fe640 1 -- 172.21.15.133:0/1119311317 >> v1:172.21.15.162:6789/0 conn(0x7f83d80d0770 legacy=0x7f83d81614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.757+0000 7f83d77fe640 1 -- 172.21.15.133:0/1119311317 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d80b97f0 con 0x7f83d815db00 2024-04-25T10:15:56.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d77fe640 1 --2- 172.21.15.133:0/1119311317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d815db00 0x7f83d80b2990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009870 tx=0x7f83c801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1211b1a5c705a02c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d67fc640 1 -- 172.21.15.133:0/1119311317 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c80a95f0 con 0x7f83d815db00 2024-04-25T10:15:56.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d67fc640 1 -- 172.21.15.133:0/1119311317 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c80a9790 con 0x7f83d815db00 2024-04-25T10:15:56.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d67fc640 1 -- 172.21.15.133:0/1119311317 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c80a9b60 con 0x7f83d815db00 2024-04-25T10:15:56.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 -- 172.21.15.133:0/1119311317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d815db00 msgr2=0x7f83d80b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 --2- 172.21.15.133:0/1119311317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d815db00 0x7f83d80b2990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009870 tx=0x7f83c801c530 comp rx=0 tx=0).stop 2024-04-25T10:15:56.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 -- 172.21.15.133:0/1119311317 shutdown_connections 2024-04-25T10:15:56.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 --2- 172.21.15.133:0/1119311317 >> v2:172.21.15.162:3300/0 conn(0x7f83d80d0bd0 0x7f83d8161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 --2- 172.21.15.133:0/1119311317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d815db00 0x7f83d80b2990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 -- 172.21.15.133:0/1119311317 >> 172.21.15.133:0/1119311317 conn(0x7f83d80727c0 msgr2=0x7f83d8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 -- 172.21.15.133:0/1119311317 shutdown_connections 2024-04-25T10:15:56.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 -- 172.21.15.133:0/1119311317 wait complete. 2024-04-25T10:15:56.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 Processor -- start 2024-04-25T10:15:56.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 -- start start 2024-04-25T10:15:56.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d80d0770 0x7f83d80ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d80d0bd0 0x7f83d81fa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d815db00 0x7f83d8161330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80d0fe0 con 0x7f83d80d0bd0 2024-04-25T10:15:56.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80b7430 con 0x7f83d815db00 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83ddd82640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80d18a0 con 0x7f83d80d0770 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d80d0bd0 0x7f83d81fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d80d0bd0 0x7f83d81fa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38956/0 (socket says 172.21.15.133:38956) 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d6ffd640 1 -- 172.21.15.133:0/1792277901 learned_addr learned my addr 172.21.15.133:0/1792277901 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d77fe640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d80d0770 0x7f83d80ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d7fff640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d815db00 0x7f83d8161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d6ffd640 1 -- 172.21.15.133:0/1792277901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d80d0770 msgr2=0x7f83d80ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d6ffd640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d80d0770 0x7f83d80ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d6ffd640 1 -- 172.21.15.133:0/1792277901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d815db00 msgr2=0x7f83d8161330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d6ffd640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d815db00 0x7f83d8161330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.761+0000 7f83d6ffd640 1 -- 172.21.15.133:0/1792277901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d80c76c0 con 0x7f83d80d0bd0 2024-04-25T10:15:56.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83d6ffd640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d80d0bd0 0x7f83d81fa350 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c3c0 tx=0x7f83c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83cffff640 1 -- 172.21.15.133:0/1792277901 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0017070 con 0x7f83d80d0bd0 2024-04-25T10:15:56.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83cffff640 1 -- 172.21.15.133:0/1792277901 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c00052d0 con 0x7f83d80d0bd0 2024-04-25T10:15:56.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83ddd82640 1 -- 172.21.15.133:0/1792277901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d80c79a0 con 0x7f83d80d0bd0 2024-04-25T10:15:56.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83cffff640 1 -- 172.21.15.133:0/1792277901 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0003840 con 0x7f83d80d0bd0 2024-04-25T10:15:56.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83ddd82640 1 -- 172.21.15.133:0/1792277901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d80c7f60 con 0x7f83d80d0bd0 2024-04-25T10:15:56.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83ddd82640 1 -- 172.21.15.133:0/1792277901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d8068ef0 con 0x7f83d80d0bd0 2024-04-25T10:15:56.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83cffff640 1 -- 172.21.15.133:0/1792277901 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83c001a070 con 0x7f83d80d0bd0 2024-04-25T10:15:56.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83cffff640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83b8066f60 0x7f83b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83d77fe640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83b8066f60 0x7f83b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83cffff640 1 -- 172.21.15.133:0/1792277901 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83c00844a0 con 0x7f83d80d0bd0 2024-04-25T10:15:56.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.765+0000 7f83d77fe640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83b8066f60 0x7f83b8069420 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f83c80ae000 tx=0x7f83c80b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.769+0000 7f83cffff640 1 -- 172.21.15.133:0/1792277901 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83c0005480 con 0x7f83d80d0bd0 2024-04-25T10:15:56.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.929+0000 7f83ddd82640 1 -- 172.21.15.133:0/1792277901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83d80b2750 con 0x7f83d80d0bd0 2024-04-25T10:15:56.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.929+0000 7f83cffff640 1 -- 172.21.15.133:0/1792277901 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16908 (secure 0 0 0) 0x7f83c00553b0 con 0x7f83d80d0bd0 2024-04-25T10:15:56.932 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:56.932 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":157,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:15:14.762327+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:11.332375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:12.104303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:56.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 -- 172.21.15.133:0/1792277901 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83b8066f60 msgr2=0x7f83b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83b8066f60 0x7f83b8069420 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f83c80ae000 tx=0x7f83c80b3040 comp rx=0 tx=0).stop 2024-04-25T10:15:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 -- 172.21.15.133:0/1792277901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d80d0bd0 msgr2=0x7f83d81fa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d80d0bd0 0x7f83d81fa350 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c3c0 tx=0x7f83c000c880 comp rx=0 tx=0).stop 2024-04-25T10:15:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 -- 172.21.15.133:0/1792277901 shutdown_connections 2024-04-25T10:15:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f83d815db00 0x7f83d8161330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f83b8066f60 0x7f83b8069420 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83d80d0bd0 0x7f83d81fa350 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 --2- 172.21.15.133:0/1792277901 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83d80d0770 0x7f83d80ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 -- 172.21.15.133:0/1792277901 >> 172.21.15.133:0/1792277901 conn(0x7f83d80727c0 msgr2=0x7f83d80b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 -- 172.21.15.133:0/1792277901 shutdown_connections 2024-04-25T10:15:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:56.933+0000 7f83ddd82640 1 -- 172.21.15.133:0/1792277901 wait complete. 2024-04-25T10:15:56.951 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:15:56.951 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:57.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f3fad640 1 Processor -- start 2024-04-25T10:15:57.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f3fad640 1 -- start start 2024-04-25T10:15:57.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f3fad640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f63ec0d0bd0 0x7f63ec0b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f3fad640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ec0ceb20 con 0x7f63ec15db00 2024-04-25T10:15:57.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f3fad640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ec0b30a0 con 0x7f63ec0d0770 2024-04-25T10:15:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f3fad640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ec0b7740 con 0x7f63ec0d0bd0 2024-04-25T10:15:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f1d49640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f63ec15db00 0x7f63ec0b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44098/0 (socket says 172.21.15.133:44098) 2024-04-25T10:15:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f1d49640 1 -- 172.21.15.133:0/4293657262 learned_addr learned my addr 172.21.15.133:0/4293657262 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:57.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897270757 0 0) 0x7f63ec0ceb20 con 0x7f63ec15db00 2024-04-25T10:15:57.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63d8003530 con 0x7f63ec15db00 2024-04-25T10:15:57.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f254a640 1 --2- 172.21.15.133:0/4293657262 >> v2:172.21.15.162:3300/0 conn(0x7f63ec0d0bd0 0x7f63ec0b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 395929647 0 0) 0x7f63d8003530 con 0x7f63ec15db00 2024-04-25T10:15:57.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 >> v2:172.21.15.162:3300/0 conn(0x7f63ec0d0bd0 msgr2=0x7f63ec0b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 --2- 172.21.15.133:0/4293657262 >> v2:172.21.15.162:3300/0 conn(0x7f63ec0d0bd0 0x7f63ec0b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 >> v1:172.21.15.162:6789/0 conn(0x7f63ec0d0770 legacy=0x7f63ec0b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec0b7a20 con 0x7f63ec15db00 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f63e0001680 con 0x7f63ec15db00 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 >> v1:172.21.15.133:6789/0 conn(0x7f63ec15db00 legacy=0x7f63ec0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80039c0 0x7f63d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d8004810 0x7f63d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d80051d0 0x7f63d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec0b7740 con 0x7f63d80051d0 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d800c070 con 0x7f63d80039c0 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d800c270 con 0x7f63d8004810 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f254a640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d80051d0 0x7f63d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f1d49640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80039c0 0x7f63d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f254a640 1 -- 172.21.15.133:0/4293657262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d8004810 msgr2=0x7f63d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f1548640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d8004810 0x7f63d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f254a640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63d8004810 0x7f63d8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f254a640 1 -- 172.21.15.133:0/4293657262 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80039c0 msgr2=0x7f63d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f254a640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80039c0 0x7f63d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f254a640 1 -- 172.21.15.133:0/4293657262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8003530 con 0x7f63d80051d0 2024-04-25T10:15:57.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.069+0000 7f63f1d49640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80039c0 0x7f63d8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f254a640 1 -- 172.21.15.133:0/4293657262 reap_dead start 2024-04-25T10:15:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f254a640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d80051d0 0x7f63d800b9d0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009fa0 tx=0x7f63e401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb2fcb2390dd82b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e40b2070 con 0x7f63d80051d0 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e40a5d50 con 0x7f63d80051d0 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f0d47640 1 -- 172.21.15.133:0/4293657262 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4009bf0 con 0x7f63d80051d0 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 -- 172.21.15.133:0/4293657262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d80051d0 msgr2=0x7f63d800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d80051d0 0x7f63d800b9d0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009fa0 tx=0x7f63e401cc80 comp rx=0 tx=0).stop 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 -- 172.21.15.133:0/4293657262 shutdown_connections 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 --2- 172.21.15.133:0/4293657262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d80051d0 0x7f63d800b9d0 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 -- 172.21.15.133:0/4293657262 >> 172.21.15.133:0/4293657262 conn(0x7f63ec0727c0 msgr2=0x7f63ec072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 -- 172.21.15.133:0/4293657262 shutdown_connections 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 -- 172.21.15.133:0/4293657262 wait complete. 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 Processor -- start 2024-04-25T10:15:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 -- start start 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80051d0 0x7f63ec160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ec1550e0 0x7f63ec155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63ec0caa40 0x7f63ec0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec0b82c0 con 0x7f63ec1550e0 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec0b7fe0 con 0x7f63d80051d0 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f3fad640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec0d1060 con 0x7f63ec0caa40 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f1548640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ec1550e0 0x7f63ec155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f1548640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ec1550e0 0x7f63ec155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38972/0 (socket says 172.21.15.133:38972) 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f1548640 1 -- 172.21.15.133:0/3285307527 learned_addr learned my addr 172.21.15.133:0/3285307527 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f254a640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63ec0caa40 0x7f63ec0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f1d49640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80051d0 0x7f63ec160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f1548640 1 -- 172.21.15.133:0/3285307527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63ec0caa40 msgr2=0x7f63ec0caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f1548640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63ec0caa40 0x7f63ec0caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f1548640 1 -- 172.21.15.133:0/3285307527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80051d0 msgr2=0x7f63ec160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f1548640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80051d0 0x7f63ec160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f1548640 1 -- 172.21.15.133:0/3285307527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec0cc330 con 0x7f63ec1550e0 2024-04-25T10:15:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.073+0000 7f63f254a640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63ec0caa40 0x7f63ec0caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63f1548640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ec1550e0 0x7f63ec155560 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f63e800d490 tx=0x7f63e800d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63de7fc640 1 -- 172.21.15.133:0/3285307527 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8005480 con 0x7f63ec1550e0 2024-04-25T10:15:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63de7fc640 1 -- 172.21.15.133:0/3285307527 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e8017070 con 0x7f63ec1550e0 2024-04-25T10:15:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63f3fad640 1 -- 172.21.15.133:0/3285307527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ec0cb7b0 con 0x7f63ec1550e0 2024-04-25T10:15:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63de7fc640 1 -- 172.21.15.133:0/3285307527 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8003820 con 0x7f63ec1550e0 2024-04-25T10:15:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63f3fad640 1 -- 172.21.15.133:0/3285307527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ec0cbd70 con 0x7f63ec1550e0 2024-04-25T10:15:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63f3fad640 1 -- 172.21.15.133:0/3285307527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ec068ef0 con 0x7f63ec1550e0 2024-04-25T10:15:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63de7fc640 1 -- 172.21.15.133:0/3285307527 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63e80039c0 con 0x7f63ec1550e0 2024-04-25T10:15:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63de7fc640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63d0066f90 0x7f63d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63de7fc640 1 -- 172.21.15.133:0/3285307527 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63e8084ae0 con 0x7f63ec1550e0 2024-04-25T10:15:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63f1d49640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63d0066f90 0x7f63d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.077+0000 7f63f1d49640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63d0066f90 0x7f63d0069450 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f63e0001930 tx=0x7f63e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.081+0000 7f63de7fc640 1 -- 172.21.15.133:0/3285307527 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63ec0b30a0 con 0x7f63ec1550e0 2024-04-25T10:15:57.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.261+0000 7f63f3fad640 1 -- 172.21.15.133:0/3285307527 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63ec153570 con 0x7f63ec1550e0 2024-04-25T10:15:57.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63de7fc640 1 -- 172.21.15.133:0/3285307527 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4959 (secure 0 0 0) 0x7f63e8055a40 con 0x7f63ec1550e0 2024-04-25T10:15:57.264 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:57.264 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3454110484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3454110484},{"type":"v1","addr":"172.21.15.162:6835","nonce":3454110484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3345416592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3345416592},{"type":"v1","addr":"172.21.15.133:6835","nonce":3345416592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2924224565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2924224565},{"type":"v1","addr":"172.21.15.133:6837","nonce":2924224565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:13.097799+0000","modified":"2024-04-25T10:15:15.128366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10554":{"gid":10554,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/3763981385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3763981385},{"type":"v1","addr":"172.21.15.162:6833","nonce":3763981385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:15:57.265 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 99 2024-04-25T10:15:57.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 -- 172.21.15.133:0/3285307527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63d0066f90 msgr2=0x7f63d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63d0066f90 0x7f63d0069450 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f63e0001930 tx=0x7f63e0009040 comp rx=0 tx=0).stop 2024-04-25T10:15:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 -- 172.21.15.133:0/3285307527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ec1550e0 msgr2=0x7f63ec155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ec1550e0 0x7f63ec155560 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f63e800d490 tx=0x7f63e800d950 comp rx=0 tx=0).stop 2024-04-25T10:15:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 -- 172.21.15.133:0/3285307527 shutdown_connections 2024-04-25T10:15:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63ec0caa40 0x7f63ec0caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63d0066f90 0x7f63d0069450 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ec1550e0 0x7f63ec155560 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 --2- 172.21.15.133:0/3285307527 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63d80051d0 0x7f63ec160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 -- 172.21.15.133:0/3285307527 >> 172.21.15.133:0/3285307527 conn(0x7f63ec0727c0 msgr2=0x7f63ec0d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 -- 172.21.15.133:0/3285307527 shutdown_connections 2024-04-25T10:15:57.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.265+0000 7f63f3fad640 1 -- 172.21.15.133:0/3285307527 wait complete. 2024-04-25T10:15:57.282 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:57.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.401+0000 7f8c44cc8640 1 Processor -- start 2024-04-25T10:15:57.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.401+0000 7f8c44cc8640 1 -- start start 2024-04-25T10:15:57.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c44cc8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c400b1f70 0x7f8c4006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c44cc8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c400c61d0 con 0x7f8c4015c9a0 2024-04-25T10:15:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c44cc8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c400c64b0 con 0x7f8c400ce330 2024-04-25T10:15:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c44cc8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4006d4f0 con 0x7f8c400b1f70 2024-04-25T10:15:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c3dd9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c400b1f70 0x7f8c4006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c3e59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8c4015c9a0 0x7f8c400c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44106/0 (socket says 172.21.15.133:44106) 2024-04-25T10:15:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c3e59c640 1 -- 172.21.15.133:0/949286591 learned_addr learned my addr 172.21.15.133:0/949286591 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c3dd9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c400b1f70 0x7f8c4006dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38980/0 (socket says 172.21.15.133:38980) 2024-04-25T10:15:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c3dd9b640 1 -- 172.21.15.133:0/949286591 >> v1:172.21.15.162:6789/0 conn(0x7f8c400ce330 legacy=0x7f8c4015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c3dd9b640 1 -- 172.21.15.133:0/949286591 >> v1:172.21.15.133:6789/0 conn(0x7f8c4015c9a0 legacy=0x7f8c400c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c3dd9b640 1 -- 172.21.15.133:0/949286591 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4006d7d0 con 0x7f8c400b1f70 2024-04-25T10:15:57.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c3dd9b640 1 --2- 172.21.15.133:0/949286591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c400b1f70 0x7f8c4006dcf0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f8c340097c0 tx=0x7f8c3401c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2a96a80d6a977d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c3d59a640 1 -- 172.21.15.133:0/949286591 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c340b0070 con 0x7f8c400b1f70 2024-04-25T10:15:57.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c3d59a640 1 -- 172.21.15.133:0/949286591 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c340ab600 con 0x7f8c400b1f70 2024-04-25T10:15:57.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.405+0000 7f8c3d59a640 1 -- 172.21.15.133:0/949286591 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c340ab9d0 con 0x7f8c400b1f70 2024-04-25T10:15:57.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- 172.21.15.133:0/949286591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c400b1f70 msgr2=0x7f8c4006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 --2- 172.21.15.133:0/949286591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c400b1f70 0x7f8c4006dcf0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f8c340097c0 tx=0x7f8c3401c2f0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- 172.21.15.133:0/949286591 shutdown_connections 2024-04-25T10:15:57.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 --2- 172.21.15.133:0/949286591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c400b1f70 0x7f8c4006dcf0 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- 172.21.15.133:0/949286591 >> 172.21.15.133:0/949286591 conn(0x7f8c40067000 msgr2=0x7f8c4006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- 172.21.15.133:0/949286591 shutdown_connections 2024-04-25T10:15:57.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- 172.21.15.133:0/949286591 wait complete. 2024-04-25T10:15:57.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 Processor -- start 2024-04-25T10:15:57.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- start start 2024-04-25T10:15:57.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400b1f70 0x7f8c401328a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c400ce330 0x7f8c40145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4015c9a0 0x7f8c4014e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4015cf00 con 0x7f8c4015c9a0 2024-04-25T10:15:57.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4015cd80 con 0x7f8c400ce330 2024-04-25T10:15:57.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40169de0 con 0x7f8c400b1f70 2024-04-25T10:15:57.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4015c9a0 0x7f8c4014e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4015c9a0 0x7f8c4014e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38996/0 (socket says 172.21.15.133:38996) 2024-04-25T10:15:57.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3ed9d640 1 -- 172.21.15.133:0/4062881244 learned_addr learned my addr 172.21.15.133:0/4062881244 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:57.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3e59c640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400b1f70 0x7f8c401328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3dd9b640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c400ce330 0x7f8c40145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3ed9d640 1 -- 172.21.15.133:0/4062881244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400b1f70 msgr2=0x7f8c401328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3ed9d640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400b1f70 0x7f8c401328a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3ed9d640 1 -- 172.21.15.133:0/4062881244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c400ce330 msgr2=0x7f8c40145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3ed9d640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c400ce330 0x7f8c40145cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3ed9d640 1 -- 172.21.15.133:0/4062881244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4014d670 con 0x7f8c4015c9a0 2024-04-25T10:15:57.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3e59c640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400b1f70 0x7f8c401328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:57.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c3ed9d640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4015c9a0 0x7f8c4014e710 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800b9c0 tx=0x7f8c3800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c2effd640 1 -- 172.21.15.133:0/4062881244 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38019070 con 0x7f8c4015c9a0 2024-04-25T10:15:57.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c2effd640 1 -- 172.21.15.133:0/4062881244 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3800cba0 con 0x7f8c4015c9a0 2024-04-25T10:15:57.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- 172.21.15.133:0/4062881244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4014d950 con 0x7f8c4015c9a0 2024-04-25T10:15:57.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c2effd640 1 -- 172.21.15.133:0/4062881244 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38014440 con 0x7f8c4015c9a0 2024-04-25T10:15:57.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- 172.21.15.133:0/4062881244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4014df10 con 0x7f8c4015c9a0 2024-04-25T10:15:57.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.409+0000 7f8c44cc8640 1 -- 172.21.15.133:0/4062881244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c40068ef0 con 0x7f8c4015c9a0 2024-04-25T10:15:57.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.413+0000 7f8c2effd640 1 -- 172.21.15.133:0/4062881244 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c3801c070 con 0x7f8c4015c9a0 2024-04-25T10:15:57.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.413+0000 7f8c2effd640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c20066f90 0x7f8c20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.413+0000 7f8c2effd640 1 -- 172.21.15.133:0/4062881244 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c38084f00 con 0x7f8c4015c9a0 2024-04-25T10:15:57.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.413+0000 7f8c3e59c640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c20066f90 0x7f8c20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.413+0000 7f8c3e59c640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c20066f90 0x7f8c20069450 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8c30000f70 tx=0x7f8c30009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.417+0000 7f8c2effd640 1 -- 172.21.15.133:0/4062881244 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c38055d80 con 0x7f8c4015c9a0 2024-04-25T10:15:57.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.593+0000 7f8c44cc8640 1 -- 172.21.15.133:0/4062881244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c40150cd0 con 0x7f8c4015c9a0 2024-04-25T10:15:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.593+0000 7f8c2effd640 1 -- 172.21.15.133:0/4062881244 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4959 (secure 0 0 0) 0x7f8c38055680 con 0x7f8c4015c9a0 2024-04-25T10:15:57.595 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:57.596 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":99,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3454110484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3454110484},{"type":"v1","addr":"172.21.15.162:6835","nonce":3454110484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3345416592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3345416592},{"type":"v1","addr":"172.21.15.133:6835","nonce":3345416592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2924224565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2924224565},{"type":"v1","addr":"172.21.15.133:6837","nonce":2924224565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:13.097799+0000","modified":"2024-04-25T10:15:15.128366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10554":{"gid":10554,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/3763981385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3763981385},{"type":"v1","addr":"172.21.15.162:6833","nonce":3763981385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:15:57.596 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 99 2024-04-25T10:15:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 -- 172.21.15.133:0/4062881244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c20066f90 msgr2=0x7f8c20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c20066f90 0x7f8c20069450 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8c30000f70 tx=0x7f8c30009040 comp rx=0 tx=0).stop 2024-04-25T10:15:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 -- 172.21.15.133:0/4062881244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4015c9a0 msgr2=0x7f8c4014e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4015c9a0 0x7f8c4014e710 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800b9c0 tx=0x7f8c3800be80 comp rx=0 tx=0).stop 2024-04-25T10:15:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 -- 172.21.15.133:0/4062881244 shutdown_connections 2024-04-25T10:15:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c20066f90 0x7f8c20069450 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4015c9a0 0x7f8c4014e710 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c400ce330 0x7f8c40145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 --2- 172.21.15.133:0/4062881244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c400b1f70 0x7f8c401328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 -- 172.21.15.133:0/4062881244 >> 172.21.15.133:0/4062881244 conn(0x7f8c40067000 msgr2=0x7f8c4006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 -- 172.21.15.133:0/4062881244 shutdown_connections 2024-04-25T10:15:57.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.597+0000 7f8c44cc8640 1 -- 172.21.15.133:0/4062881244 wait complete. 2024-04-25T10:15:57.614 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:57.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.725+0000 7f7a37e28640 1 Processor -- start 2024-04-25T10:15:57.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.725+0000 7f7a37e28640 1 -- start start 2024-04-25T10:15:57.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a37e28640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a3015db00 0x7f7a300b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a37e28640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7a300d0bd0 0x7f7a30161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a37e28640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a300ceb20 con 0x7f7a300d0770 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a37e28640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a300b7740 con 0x7f7a300d0bd0 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a37e28640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300b7a20 con 0x7f7a3015db00 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a363c5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7a300d0bd0 0x7f7a30161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a363c5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7a300d0bd0 0x7f7a30161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39010/0 (socket says 172.21.15.133:39010) 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a363c5640 1 -- 172.21.15.133:0/67102457 learned_addr learned my addr 172.21.15.133:0/67102457 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a35bc4640 1 --2- 172.21.15.133:0/67102457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a3015db00 0x7f7a300b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a363c5640 1 -- 172.21.15.133:0/67102457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a3015db00 msgr2=0x7f7a300b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a363c5640 1 --2- 172.21.15.133:0/67102457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a3015db00 0x7f7a300b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a363c5640 1 -- 172.21.15.133:0/67102457 >> v1:172.21.15.162:6789/0 conn(0x7f7a300d0770 legacy=0x7f7a301614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a363c5640 1 -- 172.21.15.133:0/67102457 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300b97f0 con 0x7f7a300d0bd0 2024-04-25T10:15:57.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.729+0000 7f7a363c5640 1 --2- 172.21.15.133:0/67102457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a300d0bd0 0x7f7a30161bd0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009e50 tx=0x7f7a2801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa1916de021bdf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a34bc2640 1 -- 172.21.15.133:0/67102457 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280a5840 con 0x7f7a300d0bd0 2024-04-25T10:15:57.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a34bc2640 1 -- 172.21.15.133:0/67102457 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a280a59e0 con 0x7f7a300d0bd0 2024-04-25T10:15:57.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a34bc2640 1 -- 172.21.15.133:0/67102457 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280a5db0 con 0x7f7a300d0bd0 2024-04-25T10:15:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 -- 172.21.15.133:0/67102457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a300d0bd0 msgr2=0x7f7a30161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 --2- 172.21.15.133:0/67102457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a300d0bd0 0x7f7a30161bd0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009e50 tx=0x7f7a2801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 -- 172.21.15.133:0/67102457 shutdown_connections 2024-04-25T10:15:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 --2- 172.21.15.133:0/67102457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a300d0bd0 0x7f7a30161bd0 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 --2- 172.21.15.133:0/67102457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a3015db00 0x7f7a300b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 -- 172.21.15.133:0/67102457 >> 172.21.15.133:0/67102457 conn(0x7f7a300727c0 msgr2=0x7f7a30072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 -- 172.21.15.133:0/67102457 shutdown_connections 2024-04-25T10:15:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 -- 172.21.15.133:0/67102457 wait complete. 2024-04-25T10:15:57.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 Processor -- start 2024-04-25T10:15:57.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 -- start start 2024-04-25T10:15:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a300d0770 0x7f7a3015e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a300d0bd0 0x7f7a30158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a3015db00 0x7f7a300cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300b7430 con 0x7f7a300d0770 2024-04-25T10:15:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300d1060 con 0x7f7a3015db00 2024-04-25T10:15:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300d18a0 con 0x7f7a300d0bd0 2024-04-25T10:15:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a35bc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a300d0770 0x7f7a3015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a35bc4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a300d0770 0x7f7a3015e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39026/0 (socket says 172.21.15.133:39026) 2024-04-25T10:15:57.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a35bc4640 1 -- 172.21.15.133:0/2282965931 learned_addr learned my addr 172.21.15.133:0/2282965931 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a35bc4640 1 -- 172.21.15.133:0/2282965931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a300d0bd0 msgr2=0x7f7a30158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a35bc4640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a300d0bd0 0x7f7a30158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a35bc4640 1 -- 172.21.15.133:0/2282965931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a3015db00 msgr2=0x7f7a300cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a363c5640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a3015db00 0x7f7a300cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a35bc4640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a3015db00 0x7f7a300cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a35bc4640 1 -- 172.21.15.133:0/2282965931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3015f760 con 0x7f7a300d0770 2024-04-25T10:15:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a35bc4640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a300d0770 0x7f7a3015e290 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00d800 tx=0x7f7a2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a267fc640 1 -- 172.21.15.133:0/2282965931 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00b6c0 con 0x7f7a300d0770 2024-04-25T10:15:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a267fc640 1 -- 172.21.15.133:0/2282965931 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c019070 con 0x7f7a300d0770 2024-04-25T10:15:57.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 -- 172.21.15.133:0/2282965931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3015e7d0 con 0x7f7a300d0770 2024-04-25T10:15:57.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a267fc640 1 -- 172.21.15.133:0/2282965931 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00ba60 con 0x7f7a300d0770 2024-04-25T10:15:57.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.733+0000 7f7a37e28640 1 -- 172.21.15.133:0/2282965931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3015fa00 con 0x7f7a300d0770 2024-04-25T10:15:57.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.737+0000 7f7a37e28640 1 -- 172.21.15.133:0/2282965931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a30068ef0 con 0x7f7a300d0770 2024-04-25T10:15:57.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.737+0000 7f7a267fc640 1 -- 172.21.15.133:0/2282965931 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a2c00bc00 con 0x7f7a300d0770 2024-04-25T10:15:57.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.737+0000 7f7a267fc640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a14066f60 0x7f7a14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.737+0000 7f7a267fc640 1 -- 172.21.15.133:0/2282965931 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7a2c0855d0 con 0x7f7a300d0770 2024-04-25T10:15:57.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.737+0000 7f7a353c3640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a14066f60 0x7f7a14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.737+0000 7f7a353c3640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a14066f60 0x7f7a14069420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009670 tx=0x7f7a20006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.741+0000 7f7a267fc640 1 -- 172.21.15.133:0/2282965931 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a2c0564e0 con 0x7f7a300d0770 2024-04-25T10:15:57.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.893+0000 7f7a37e28640 1 -- 172.21.15.133:0/2282965931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a3015ea00 con 0x7f7a300d0770 2024-04-25T10:15:57.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a267fc640 1 -- 172.21.15.133:0/2282965931 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16908 (secure 0 0 0) 0x7f7a2c055de0 con 0x7f7a300d0770 2024-04-25T10:15:57.898 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:15:57.898 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":157,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:15:14.762327+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:11.332375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:12.104303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 -- 172.21.15.133:0/2282965931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a14066f60 msgr2=0x7f7a14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a14066f60 0x7f7a14069420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009670 tx=0x7f7a20006bd0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 -- 172.21.15.133:0/2282965931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a300d0770 msgr2=0x7f7a3015e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a300d0770 0x7f7a3015e290 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00d800 tx=0x7f7a2c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 -- 172.21.15.133:0/2282965931 shutdown_connections 2024-04-25T10:15:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a3015db00 0x7f7a300cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a300d0bd0 0x7f7a30158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7a14066f60 0x7f7a14069420 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 --2- 172.21.15.133:0/2282965931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a300d0770 0x7f7a3015e290 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 -- 172.21.15.133:0/2282965931 >> 172.21.15.133:0/2282965931 conn(0x7f7a300727c0 msgr2=0x7f7a300b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 -- 172.21.15.133:0/2282965931 shutdown_connections 2024-04-25T10:15:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:57.897+0000 7f7a37e28640 1 -- 172.21.15.133:0/2282965931 wait complete. 2024-04-25T10:15:57.916 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:15:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f374ed66640 1 Processor -- start 2024-04-25T10:15:58.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f374ed66640 1 -- start start 2024-04-25T10:15:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f374ed66640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3748145170 0x7f374806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f374ed66640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37480ce2e0 con 0x7f374806e4d0 2024-04-25T10:15:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f374ed66640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374806d4b0 con 0x7f374815ced0 2024-04-25T10:15:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f374ed66640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f374806db00 con 0x7f3748145170 2024-04-25T10:15:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f374cb02640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3748145170 0x7f374806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f3747fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f374806e4d0 0x7f37480cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44112/0 (socket says 172.21.15.133:44112) 2024-04-25T10:15:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f3747fff640 1 -- 172.21.15.133:0/1960841606 learned_addr learned my addr 172.21.15.133:0/1960841606 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f374cb02640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3748145170 0x7f374806df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39028/0 (socket says 172.21.15.133:39028) 2024-04-25T10:15:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f374cb02640 1 -- 172.21.15.133:0/1960841606 >> v1:172.21.15.162:6789/0 conn(0x7f374815ced0 legacy=0x7f37480b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f374cb02640 1 -- 172.21.15.133:0/1960841606 >> v1:172.21.15.133:6789/0 conn(0x7f374806e4d0 legacy=0x7f37480cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f374cb02640 1 -- 172.21.15.133:0/1960841606 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374806d790 con 0x7f3748145170 2024-04-25T10:15:58.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.025+0000 7f37477fe640 1 -- 172.21.15.133:0/1960841606 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094253393 0 0) 0x7f37480ce2e0 con 0x7f374806e4d0 2024-04-25T10:15:58.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f374cb02640 1 --2- 172.21.15.133:0/1960841606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3748145170 0x7f374806df90 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f37380097c0 tx=0x7f373801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5a19e7174f605e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f37477fe640 1 -- 172.21.15.133:0/1960841606 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37380b0070 con 0x7f3748145170 2024-04-25T10:15:58.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f37477fe640 1 -- 172.21.15.133:0/1960841606 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37380ab600 con 0x7f3748145170 2024-04-25T10:15:58.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f37477fe640 1 -- 172.21.15.133:0/1960841606 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37380ab9d0 con 0x7f3748145170 2024-04-25T10:15:58.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f374ed66640 1 -- 172.21.15.133:0/1960841606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3748145170 msgr2=0x7f374806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f374ed66640 1 --2- 172.21.15.133:0/1960841606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3748145170 0x7f374806df90 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f37380097c0 tx=0x7f373801c460 comp rx=0 tx=0).stop 2024-04-25T10:15:58.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f374ed66640 1 -- 172.21.15.133:0/1960841606 shutdown_connections 2024-04-25T10:15:58.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f374ed66640 1 --2- 172.21.15.133:0/1960841606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3748145170 0x7f374806df90 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f374ed66640 1 -- 172.21.15.133:0/1960841606 >> 172.21.15.133:0/1960841606 conn(0x7f3748066f70 msgr2=0x7f374806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f374ed66640 1 -- 172.21.15.133:0/1960841606 shutdown_connections 2024-04-25T10:15:58.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f374ed66640 1 -- 172.21.15.133:0/1960841606 wait complete. 2024-04-25T10:15:58.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f374ed66640 1 Processor -- start 2024-04-25T10:15:58.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.029+0000 7f374ed66640 1 -- start start 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374ed66640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f374806e4d0 0x7f374815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374ed66640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3748145170 0x7f37481356b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374ed66640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f374815ced0 0x7f3748154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374ed66640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3748145550 con 0x7f374815ced0 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374ed66640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37480b2020 con 0x7f374806e4d0 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374ed66640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374815d320 con 0x7f3748145170 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374d303640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f374815ced0 0x7f3748154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374d303640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f374815ced0 0x7f3748154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39030/0 (socket says 172.21.15.133:39030) 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374d303640 1 -- 172.21.15.133:0/1925394945 learned_addr learned my addr 172.21.15.133:0/1925394945 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374cb02640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f374806e4d0 0x7f374815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374d303640 1 -- 172.21.15.133:0/1925394945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3748145170 msgr2=0x7f37481356b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374d303640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3748145170 0x7f37481356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374d303640 1 -- 172.21.15.133:0/1925394945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f374806e4d0 msgr2=0x7f374815f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374d303640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f374806e4d0 0x7f374815f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374d303640 1 -- 172.21.15.133:0/1925394945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37481557b0 con 0x7f374815ced0 2024-04-25T10:15:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374cb02640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f374806e4d0 0x7f374815f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374d303640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f374815ced0 0x7f3748154c30 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f374000b970 tx=0x7f374000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f37457fa640 1 -- 172.21.15.133:0/1925394945 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740019070 con 0x7f374815ced0 2024-04-25T10:15:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f37457fa640 1 -- 172.21.15.133:0/1925394945 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374000cb40 con 0x7f374815ced0 2024-04-25T10:15:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374ed66640 1 -- 172.21.15.133:0/1925394945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3748155a90 con 0x7f374815ced0 2024-04-25T10:15:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f37457fa640 1 -- 172.21.15.133:0/1925394945 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740014440 con 0x7f374815ced0 2024-04-25T10:15:58.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374ed66640 1 -- 172.21.15.133:0/1925394945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3748135190 con 0x7f374815ced0 2024-04-25T10:15:58.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374ed66640 1 -- 172.21.15.133:0/1925394945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3748135390 con 0x7f374815ced0 2024-04-25T10:15:58.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f37457fa640 1 -- 172.21.15.133:0/1925394945 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f374001c070 con 0x7f374815ced0 2024-04-25T10:15:58.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f37457fa640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3734066f60 0x7f3734069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374cb02640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3734066f60 0x7f3734069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f37457fa640 1 -- 172.21.15.133:0/1925394945 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3740084de0 con 0x7f374815ced0 2024-04-25T10:15:58.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.033+0000 7f374cb02640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3734066f60 0x7f3734069420 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f37380ad000 tx=0x7f3738008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.037+0000 7f37457fa640 1 -- 172.21.15.133:0/1925394945 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3748135390 con 0x7f374815ced0 2024-04-25T10:15:58.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.229+0000 7f374ed66640 1 -- 172.21.15.133:0/1925394945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f374815e9a0 con 0x7f374815ced0 2024-04-25T10:15:58.230 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:15:58.229+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:15:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f37457fa640 1 -- 172.21.15.133:0/1925394945 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3740055cf0 con 0x7f374815ced0 2024-04-25T10:15:58.232 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:15:58.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 -- 172.21.15.133:0/1925394945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3734066f60 msgr2=0x7f3734069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3734066f60 0x7f3734069420 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f37380ad000 tx=0x7f3738008040 comp rx=0 tx=0).stop 2024-04-25T10:15:58.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 -- 172.21.15.133:0/1925394945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f374815ced0 msgr2=0x7f3748154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f374815ced0 0x7f3748154c30 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f374000b970 tx=0x7f374000be30 comp rx=0 tx=0).stop 2024-04-25T10:15:58.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 -- 172.21.15.133:0/1925394945 shutdown_connections 2024-04-25T10:15:58.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3734066f60 0x7f3734069420 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f374815ced0 0x7f3748154c30 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3748145170 0x7f37481356b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 --2- 172.21.15.133:0/1925394945 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f374806e4d0 0x7f374815f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 -- 172.21.15.133:0/1925394945 >> 172.21.15.133:0/1925394945 conn(0x7f3748066f70 msgr2=0x7f374813b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 -- 172.21.15.133:0/1925394945 shutdown_connections 2024-04-25T10:15:58.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.233+0000 7f374ed66640 1 -- 172.21.15.133:0/1925394945 wait complete. 2024-04-25T10:15:58.244 INFO:tasks.ceph.mds.b.smithi162.stderr: -2201> 2024-04-25T10:15:19.381+0000 7f2cd23bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:15:58.252 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:15:58.303 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:15:58.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6fe637640 1 Processor -- start 2024-04-25T10:15:58.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6fe637640 1 -- start start 2024-04-25T10:15:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6fe637640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f8145170 0x7fb6f806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6fe637640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6f806e4d0 0x7fb6f80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6fe637640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f80ce2e0 con 0x7fb6f80cdd60 2024-04-25T10:15:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6fe637640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f806db00 con 0x7fb6f806e4d0 2024-04-25T10:15:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6fe637640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f8145550 con 0x7fb6f8145170 2024-04-25T10:15:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6fcbd4640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb6f80cdd60 0x7fb6f815ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44124/0 (socket says 172.21.15.133:44124) 2024-04-25T10:15:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6fcbd4640 1 -- 172.21.15.133:0/4053636515 learned_addr learned my addr 172.21.15.133:0/4053636515 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f77fe640 1 --2- 172.21.15.133:0/4053636515 >> v2:172.21.15.162:3300/0 conn(0x7fb6f806e4d0 0x7fb6f80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f7fff640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f8145170 0x7fb6f806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036719673 0 0) 0x7fb6f80ce2e0 con 0x7fb6f80cdd60 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6e0003530 con 0x7fb6f80cdd60 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707282314 0 0) 0x7fb6e0003530 con 0x7fb6f80cdd60 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f8145170 msgr2=0x7fb6f806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f8145170 0x7fb6f806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 >> v2:172.21.15.162:3300/0 conn(0x7fb6f806e4d0 msgr2=0x7fb6f80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 --2- 172.21.15.133:0/4053636515 >> v2:172.21.15.162:3300/0 conn(0x7fb6f806e4d0 0x7fb6f80cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f806d4b0 con 0x7fb6f80cdd60 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb6f0001a00 con 0x7fb6f80cdd60 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 >> v1:172.21.15.133:6789/0 conn(0x7fb6f80cdd60 legacy=0x7fb6f815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e00039c0 0x7fb6e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e0004810 0x7fb6e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f8145170 0x7fb6e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f806db00 con 0x7fb6e0004810 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f8145550 con 0x7fb6e00039c0 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.369+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e000b9d0 con 0x7fb6f8145170 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f77fe640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e0004810 0x7fb6e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f7fff640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e00039c0 0x7fb6e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6fcbd4640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f8145170 0x7fb6e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f77fe640 1 -- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f8145170 msgr2=0x7fb6e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f77fe640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f8145170 0x7fb6e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f77fe640 1 -- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e00039c0 msgr2=0x7fb6e0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f77fe640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e00039c0 0x7fb6e0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f77fe640 1 -- 172.21.15.133:0/4053636515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e0003530 con 0x7fb6e0004810 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6fcbd4640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f8145170 0x7fb6e00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:58.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f7fff640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e00039c0 0x7fb6e0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f77fe640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e0004810 0x7fb6e0004c90 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009ab0 tx=0x7fb6ec01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae98a414941d8b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0096b0 con 0x7fb6e0004810 2024-04-25T10:15:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0a64b0 con 0x7fb6e0004810 2024-04-25T10:15:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6f6ffd640 1 -- 172.21.15.133:0/4053636515 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0a68a0 con 0x7fb6e0004810 2024-04-25T10:15:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6fe637640 1 -- 172.21.15.133:0/4053636515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e0004810 msgr2=0x7fb6e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6fe637640 1 --2- 172.21.15.133:0/4053636515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e0004810 0x7fb6e0004c90 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009ab0 tx=0x7fb6ec01ca90 comp rx=0 tx=0).stop 2024-04-25T10:15:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6fcbd4640 1 -- 172.21.15.133:0/4053636515 reap_dead start 2024-04-25T10:15:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6fe637640 1 -- 172.21.15.133:0/4053636515 shutdown_connections 2024-04-25T10:15:58.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6fe637640 1 -- 172.21.15.133:0/4053636515 >> 172.21.15.133:0/4053636515 conn(0x7fb6f8066f70 msgr2=0x7fb6f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6fe637640 1 -- 172.21.15.133:0/4053636515 shutdown_connections 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.373+0000 7fb6fe637640 1 -- 172.21.15.133:0/4053636515 wait complete. 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fe637640 1 Processor -- start 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fe637640 1 -- start start 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fe637640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e0004810 0x7fb6f8152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fe637640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f813a720 0x7fb6f813c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fe637640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f815e430 0x7fb6f815e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fe637640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f815d5e0 con 0x7fb6f813a720 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fe637640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f815cb70 con 0x7fb6e0004810 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fe637640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f815d760 con 0x7fb6f815e430 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f813a720 0x7fb6f813c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f813a720 0x7fb6f813c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39060/0 (socket says 172.21.15.133:39060) 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f77fe640 1 -- 172.21.15.133:0/4151695928 learned_addr learned my addr 172.21.15.133:0/4151695928 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fcbd4640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f815e430 0x7fb6f815e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f7fff640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e0004810 0x7fb6f8152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f77fe640 1 -- 172.21.15.133:0/4151695928 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f815e430 msgr2=0x7fb6f815e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f77fe640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f815e430 0x7fb6f815e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f77fe640 1 -- 172.21.15.133:0/4151695928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e0004810 msgr2=0x7fb6f8152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f77fe640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e0004810 0x7fb6f8152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f77fe640 1 -- 172.21.15.133:0/4151695928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f81526b0 con 0x7fb6f813a720 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f77fe640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f813a720 0x7fb6f813c380 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0b0040 tx=0x7fb6e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f4ff9640 1 -- 172.21.15.133:0/4151695928 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec005c70 con 0x7fb6f813a720 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f4ff9640 1 -- 172.21.15.133:0/4151695928 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec005e30 con 0x7fb6f813a720 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fe637640 1 -- 172.21.15.133:0/4151695928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f813abb0 con 0x7fb6f813a720 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f4ff9640 1 -- 172.21.15.133:0/4151695928 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0a6660 con 0x7fb6f813a720 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fe637640 1 -- 172.21.15.133:0/4151695928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f813b170 con 0x7fb6f813a720 2024-04-25T10:15:58.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6fe637640 1 -- 172.21.15.133:0/4151695928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f8068e80 con 0x7fb6f813a720 2024-04-25T10:15:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.377+0000 7fb6f4ff9640 1 -- 172.21.15.133:0/4151695928 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6ec0c1020 con 0x7fb6f813a720 2024-04-25T10:15:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.381+0000 7fb6f4ff9640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6e4066f60 0x7fb6e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.381+0000 7fb6f4ff9640 1 -- 172.21.15.133:0/4151695928 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb6ec11f1f0 con 0x7fb6f813a720 2024-04-25T10:15:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.381+0000 7fb6f7fff640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6e4066f60 0x7fb6e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.381+0000 7fb6f7fff640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6e4066f60 0x7fb6e4069420 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80099e0 tx=0x7fb6e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.385+0000 7fb6f4ff9640 1 -- 172.21.15.133:0/4151695928 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6ec0c8020 con 0x7fb6f813a720 2024-04-25T10:15:58.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:58.577+0000 7fb6fe637640 1 -- 172.21.15.133:0/4151695928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb6f81f1370 con 0x7fb6f813a720 2024-04-25T10:15:59.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.233+0000 7fb6f4ff9640 1 -- 172.21.15.133:0/4151695928 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v101) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb6ec0167a0 con 0x7fb6f813a720 2024-04-25T10:15:59.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 -- 172.21.15.133:0/4151695928 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6e4066f60 msgr2=0x7fb6e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6e4066f60 0x7fb6e4069420 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80099e0 tx=0x7fb6e8008040 comp rx=0 tx=0).stop 2024-04-25T10:15:59.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 -- 172.21.15.133:0/4151695928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f813a720 msgr2=0x7fb6f813c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f813a720 0x7fb6f813c380 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0b0040 tx=0x7fb6e0005840 comp rx=0 tx=0).stop 2024-04-25T10:15:59.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 -- 172.21.15.133:0/4151695928 shutdown_connections 2024-04-25T10:15:59.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6f815e430 0x7fb6f815e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6e4066f60 0x7fb6e4069420 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f813a720 0x7fb6f813c380 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 --2- 172.21.15.133:0/4151695928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6e0004810 0x7fb6f8152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 -- 172.21.15.133:0/4151695928 >> 172.21.15.133:0/4151695928 conn(0x7fb6f8066f70 msgr2=0x7fb6f806adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 -- 172.21.15.133:0/4151695928 shutdown_connections 2024-04-25T10:15:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.237+0000 7fb6fe637640 1 -- 172.21.15.133:0/4151695928 wait complete. 2024-04-25T10:15:59.259 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:15:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec18768640 1 Processor -- start 2024-04-25T10:15:59.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec18768640 1 -- start start 2024-04-25T10:15:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec18768640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fec1015b8f0 0x7fec100b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec18768640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec100d1360 con 0x7fec1006ac90 2024-04-25T10:15:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec18768640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1015c060 con 0x7fec1006b110 2024-04-25T10:15:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec18768640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1015cc30 con 0x7fec1015b8f0 2024-04-25T10:15:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15d03640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fec1006ac90 0x7fec100d0540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44134/0 (socket says 172.21.15.133:44134) 2024-04-25T10:15:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15d03640 1 -- 172.21.15.133:0/1978418762 learned_addr learned my addr 172.21.15.133:0/1978418762 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:15:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736209314 0 0) 0x7fec100d1360 con 0x7fec1006ac90 2024-04-25T10:15:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec16504640 1 --2- 172.21.15.133:0/1978418762 >> v2:172.21.15.162:3300/0 conn(0x7fec1015b8f0 0x7fec100b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febfc003530 con 0x7fec1006ac90 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3865200776 0 0) 0x7febfc003530 con 0x7fec1006ac90 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 >> v2:172.21.15.162:3300/0 conn(0x7fec1015b8f0 msgr2=0x7fec100b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 --2- 172.21.15.133:0/1978418762 >> v2:172.21.15.162:3300/0 conn(0x7fec1015b8f0 0x7fec100b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 >> v1:172.21.15.162:6789/0 conn(0x7fec1006b110 legacy=0x7fec100d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec100b0df0 con 0x7fec1006ac90 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fec00001680 con 0x7fec1006ac90 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec16504640 1 --2- 172.21.15.133:0/1978418762 >> v2:172.21.15.162:3300/0 conn(0x7fec1015b8f0 0x7fec100b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 >> v1:172.21.15.133:6789/0 conn(0x7fec1006ac90 legacy=0x7fec100d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 --2- 172.21.15.133:0/1978418762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febfc0039c0 0x7febfc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 --2- 172.21.15.133:0/1978418762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febfc004810 0x7febfc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 --2- 172.21.15.133:0/1978418762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febfc0052f0 0x7febfc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1015cc30 con 0x7febfc004810 2024-04-25T10:15:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc00c1c0 con 0x7febfc0039c0 2024-04-25T10:15:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febfc00c3c0 con 0x7febfc0052f0 2024-04-25T10:15:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec15d03640 1 --2- 172.21.15.133:0/1978418762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febfc004810 0x7febfc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.385+0000 7fec16504640 1 --2- 172.21.15.133:0/1978418762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febfc0039c0 0x7febfc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec15d03640 1 -- 172.21.15.133:0/1978418762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febfc0052f0 msgr2=0x7febfc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec15d03640 1 --2- 172.21.15.133:0/1978418762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7febfc0052f0 0x7febfc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec15d03640 1 -- 172.21.15.133:0/1978418762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febfc0039c0 msgr2=0x7febfc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec15d03640 1 --2- 172.21.15.133:0/1978418762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febfc0039c0 0x7febfc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec15d03640 1 -- 172.21.15.133:0/1978418762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc003530 con 0x7febfc004810 2024-04-25T10:15:59.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec16d05640 1 -- 172.21.15.133:0/1978418762 reap_dead start 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec15d03640 1 --2- 172.21.15.133:0/1978418762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febfc004810 0x7febfc004c90 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fec00001ec0 tx=0x7fec0001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d3fc07fd41aba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00007d40 con 0x7febfc004810 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec000a9070 con 0x7febfc004810 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec15502640 1 -- 172.21.15.133:0/1978418762 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00005d80 con 0x7febfc004810 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec18768640 1 -- 172.21.15.133:0/1978418762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febfc004810 msgr2=0x7febfc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec18768640 1 --2- 172.21.15.133:0/1978418762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febfc004810 0x7febfc004c90 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fec00001ec0 tx=0x7fec0001cc40 comp rx=0 tx=0).stop 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec18768640 1 -- 172.21.15.133:0/1978418762 shutdown_connections 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec18768640 1 --2- 172.21.15.133:0/1978418762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febfc004810 0x7febfc004c90 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec18768640 1 -- 172.21.15.133:0/1978418762 >> 172.21.15.133:0/1978418762 conn(0x7fec100726f0 msgr2=0x7fec100999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec18768640 1 -- 172.21.15.133:0/1978418762 shutdown_connections 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.389+0000 7fec18768640 1 -- 172.21.15.133:0/1978418762 wait complete. 2024-04-25T10:15:59.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec18768640 1 Processor -- start 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec18768640 1 -- start start 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec18768640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febfc004810 0x7fec10158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec18768640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec101f9aa0 0x7fec100cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec18768640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec1014f5a0 0x7fec1014fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec18768640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1015bcd0 con 0x7fec101f9aa0 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec18768640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1006b4f0 con 0x7febfc004810 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec18768640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec100b13f0 con 0x7fec1014f5a0 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec15d03640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec101f9aa0 0x7fec100cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec15d03640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec101f9aa0 0x7fec100cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39082/0 (socket says 172.21.15.133:39082) 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec15d03640 1 -- 172.21.15.133:0/3787364287 learned_addr learned my addr 172.21.15.133:0/3787364287 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec16d05640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec1014f5a0 0x7fec1014fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec16504640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febfc004810 0x7fec10158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec15d03640 1 -- 172.21.15.133:0/3787364287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec1014f5a0 msgr2=0x7fec1014fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec15d03640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec1014f5a0 0x7fec1014fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec15d03640 1 -- 172.21.15.133:0/3787364287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febfc004810 msgr2=0x7fec10158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec15d03640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febfc004810 0x7fec10158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec15d03640 1 -- 172.21.15.133:0/3787364287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec10158c60 con 0x7fec101f9aa0 2024-04-25T10:15:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec16504640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febfc004810 0x7fec10158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec15d03640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec101f9aa0 0x7fec100cf430 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fec0001c6b0 tx=0x7febfc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec06ffd640 1 -- 172.21.15.133:0/3787364287 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00007d40 con 0x7fec101f9aa0 2024-04-25T10:15:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec06ffd640 1 -- 172.21.15.133:0/3787364287 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec00000ee0 con 0x7fec101f9aa0 2024-04-25T10:15:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec06ffd640 1 -- 172.21.15.133:0/3787364287 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00007180 con 0x7fec101f9aa0 2024-04-25T10:15:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec18768640 1 -- 172.21.15.133:0/3787364287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1014ff60 con 0x7fec101f9aa0 2024-04-25T10:15:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec18768640 1 -- 172.21.15.133:0/3787364287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec101fa2f0 con 0x7fec101f9aa0 2024-04-25T10:15:59.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec18768640 1 -- 172.21.15.133:0/3787364287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec10069000 con 0x7fec101f9aa0 2024-04-25T10:15:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec06ffd640 1 -- 172.21.15.133:0/3787364287 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec00007320 con 0x7fec101f9aa0 2024-04-25T10:15:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec06ffd640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf4066f60 0x7febf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec06ffd640 1 -- 172.21.15.133:0/3787364287 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fec0011eb00 con 0x7fec101f9aa0 2024-04-25T10:15:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.393+0000 7fec16504640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf4066f60 0x7febf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:59.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.397+0000 7fec16504640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf4066f60 0x7febf4069420 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fec0800c950 tx=0x7fec08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:59.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.401+0000 7fec06ffd640 1 -- 172.21.15.133:0/3787364287 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec00016950 con 0x7fec101f9aa0 2024-04-25T10:15:59.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:15:59.557+0000 7fec18768640 1 -- 172.21.15.133:0/3787364287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fec1006f9d0 con 0x7fec101f9aa0 2024-04-25T10:16:00.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.245+0000 7fec06ffd640 1 -- 172.21.15.133:0/3787364287 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v159) v1 ==== 181+0+0 (secure 0 0 0) 0x7fec000b6320 con 0x7fec101f9aa0 2024-04-25T10:16:00.248 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:16:00.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 -- 172.21.15.133:0/3787364287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf4066f60 msgr2=0x7febf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf4066f60 0x7febf4069420 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fec0800c950 tx=0x7fec08008040 comp rx=0 tx=0).stop 2024-04-25T10:16:00.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 -- 172.21.15.133:0/3787364287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec101f9aa0 msgr2=0x7fec100cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec101f9aa0 0x7fec100cf430 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fec0001c6b0 tx=0x7febfc005840 comp rx=0 tx=0).stop 2024-04-25T10:16:00.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 -- 172.21.15.133:0/3787364287 shutdown_connections 2024-04-25T10:16:00.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fec1014f5a0 0x7fec1014fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7febf4066f60 0x7febf4069420 secure :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fec0800c950 tx=0x7fec08008040 comp rx=0 tx=0).stop 2024-04-25T10:16:00.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec101f9aa0 0x7fec100cf430 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 --2- 172.21.15.133:0/3787364287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7febfc004810 0x7fec10158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 -- 172.21.15.133:0/3787364287 >> 172.21.15.133:0/3787364287 conn(0x7fec100726f0 msgr2=0x7fec1006f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 -- 172.21.15.133:0/3787364287 shutdown_connections 2024-04-25T10:16:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.253+0000 7fec18768640 1 -- 172.21.15.133:0/3787364287 wait complete. 2024-04-25T10:16:00.274 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:16:00.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2e5a0e640 1 Processor -- start 2024-04-25T10:16:00.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2e5a0e640 1 -- start start 2024-04-25T10:16:00.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2e5a0e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2e00b4940 0x7fd2e00b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2e5a0e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e00b5260 con 0x7fd2e015c420 2024-04-25T10:16:00.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2e5a0e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e015c9b0 con 0x7fd2e0071cb0 2024-04-25T10:16:00.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2e5a0e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e0072a90 con 0x7fd2e00b4940 2024-04-25T10:16:00.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2df7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd2e015c420 0x7fd2e00cc010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44146/0 (socket says 172.21.15.133:44146) 2024-04-25T10:16:00.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2deffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2e00b4940 0x7fd2e00b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2df7fe640 1 -- 172.21.15.133:0/1180525883 learned_addr learned my addr 172.21.15.133:0/1180525883 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:00.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2de7fc640 1 -- 172.21.15.133:0/1180525883 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818822174 0 0) 0x7fd2e00b5260 con 0x7fd2e015c420 2024-04-25T10:16:00.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2de7fc640 1 -- 172.21.15.133:0/1180525883 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2cc003530 con 0x7fd2e015c420 2024-04-25T10:16:00.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2deffd640 1 -- 172.21.15.133:0/1180525883 >> v1:172.21.15.162:6789/0 conn(0x7fd2e0071cb0 legacy=0x7fd2e015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:00.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2deffd640 1 -- 172.21.15.133:0/1180525883 >> v1:172.21.15.133:6789/0 conn(0x7fd2e015c420 legacy=0x7fd2e00cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2deffd640 1 -- 172.21.15.133:0/1180525883 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e015cc90 con 0x7fd2e00b4940 2024-04-25T10:16:00.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.397+0000 7fd2deffd640 1 --2- 172.21.15.133:0/1180525883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e00b4940 0x7fd2e00b4d20 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00097c0 tx=0x7fd2d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d722f48b93417ffd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2de7fc640 1 -- 172.21.15.133:0/1180525883 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d00b0070 con 0x7fd2e00b4940 2024-04-25T10:16:00.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2de7fc640 1 -- 172.21.15.133:0/1180525883 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d00ab600 con 0x7fd2e00b4940 2024-04-25T10:16:00.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2de7fc640 1 -- 172.21.15.133:0/1180525883 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d00ab9d0 con 0x7fd2e00b4940 2024-04-25T10:16:00.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/1180525883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e00b4940 msgr2=0x7fd2e00b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 --2- 172.21.15.133:0/1180525883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e00b4940 0x7fd2e00b4d20 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00097c0 tx=0x7fd2d001c460 comp rx=0 tx=0).stop 2024-04-25T10:16:00.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/1180525883 shutdown_connections 2024-04-25T10:16:00.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 --2- 172.21.15.133:0/1180525883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e00b4940 0x7fd2e00b4d20 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/1180525883 >> 172.21.15.133:0/1180525883 conn(0x7fd2e006d370 msgr2=0x7fd2e006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/1180525883 shutdown_connections 2024-04-25T10:16:00.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/1180525883 wait complete. 2024-04-25T10:16:00.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 Processor -- start 2024-04-25T10:16:00.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 -- start start 2024-04-25T10:16:00.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2e0071cb0 0x7fd2e00ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e00b4940 0x7fd2e00ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e015c420 0x7fd2e00acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e0145450 con 0x7fd2e00b4940 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e01452d0 con 0x7fd2e0071cb0 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e0072160 con 0x7fd2e015c420 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2d7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e00b4940 0x7fd2e00ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2d7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e00b4940 0x7fd2e00ac6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39102/0 (socket says 172.21.15.133:39102) 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2d7fff640 1 -- 172.21.15.133:0/4020374061 learned_addr learned my addr 172.21.15.133:0/4020374061 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2df7fe640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e015c420 0x7fd2e00acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2deffd640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2e0071cb0 0x7fd2e00ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2d7fff640 1 -- 172.21.15.133:0/4020374061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e015c420 msgr2=0x7fd2e00acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2d7fff640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e015c420 0x7fd2e00acc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2d7fff640 1 -- 172.21.15.133:0/4020374061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2e0071cb0 msgr2=0x7fd2e00ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2d7fff640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2e0071cb0 0x7fd2e00ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2d7fff640 1 -- 172.21.15.133:0/4020374061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e01351e0 con 0x7fd2e00b4940 2024-04-25T10:16:00.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2d7fff640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e00b4940 0x7fd2e00ac6c0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800c450 tx=0x7fd2c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2dcff9640 1 -- 172.21.15.133:0/4020374061 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8012830 con 0x7fd2e00b4940 2024-04-25T10:16:00.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2dcff9640 1 -- 172.21.15.133:0/4020374061 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8017070 con 0x7fd2e00b4940 2024-04-25T10:16:00.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/4020374061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e00a89b0 con 0x7fd2e00b4940 2024-04-25T10:16:00.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2dcff9640 1 -- 172.21.15.133:0/4020374061 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8012c30 con 0x7fd2e00b4940 2024-04-25T10:16:00.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.401+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/4020374061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e00a8f70 con 0x7fd2e00b4940 2024-04-25T10:16:00.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.405+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/4020374061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e00a9170 con 0x7fd2e00b4940 2024-04-25T10:16:00.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.405+0000 7fd2dcff9640 1 -- 172.21.15.133:0/4020374061 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2c801c460 con 0x7fd2e00b4940 2024-04-25T10:16:00.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.405+0000 7fd2dcff9640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c0066f60 0x7fd2c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.405+0000 7fd2dcff9640 1 -- 172.21.15.133:0/4020374061 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fd2c8084dd0 con 0x7fd2e00b4940 2024-04-25T10:16:00.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.405+0000 7fd2deffd640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c0066f60 0x7fd2c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.405+0000 7fd2deffd640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c0066f60 0x7fd2c0069420 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00ad000 tx=0x7fd2d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.409+0000 7fd2dcff9640 1 -- 172.21.15.133:0/4020374061 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2e00a9170 con 0x7fd2e00b4940 2024-04-25T10:16:00.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:00.565+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/4020374061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd2e0069000 con 0x7fd2e00b4940 2024-04-25T10:16:01.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.257+0000 7fd2dcff9640 1 -- 172.21.15.133:0/4020374061 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v160) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd2c8086020 con 0x7fd2e00b4940 2024-04-25T10:16:01.258 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:16:01.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/4020374061 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c0066f60 msgr2=0x7fd2c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c0066f60 0x7fd2c0069420 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00ad000 tx=0x7fd2d0008040 comp rx=0 tx=0).stop 2024-04-25T10:16:01.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/4020374061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e00b4940 msgr2=0x7fd2e00ac6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e00b4940 0x7fd2e00ac6c0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800c450 tx=0x7fd2c800c910 comp rx=0 tx=0).stop 2024-04-25T10:16:01.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/4020374061 shutdown_connections 2024-04-25T10:16:01.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2e015c420 0x7fd2e00acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2c0066f60 0x7fd2c0069420 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2e00b4940 0x7fd2e00ac6c0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 --2- 172.21.15.133:0/4020374061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2e0071cb0 0x7fd2e00ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/4020374061 >> 172.21.15.133:0/4020374061 conn(0x7fd2e006d370 msgr2=0x7fd2e015d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:01.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/4020374061 shutdown_connections 2024-04-25T10:16:01.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.261+0000 7fd2e5a0e640 1 -- 172.21.15.133:0/4020374061 wait complete. 2024-04-25T10:16:01.285 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:16:01.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.405+0000 7f920cd11640 1 Processor -- start 2024-04-25T10:16:01.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.405+0000 7f920cd11640 1 -- start start 2024-04-25T10:16:01.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.405+0000 7f920cd11640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f920806e8b0 0x7f920815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.405+0000 7f920cd11640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f92080b1e50 0x7f9208146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.405+0000 7f920cd11640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92080b1090 con 0x7f92080aebc0 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.405+0000 7f920cd11640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9208147490 con 0x7f920806e8b0 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.405+0000 7f920cd11640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92080b19c0 con 0x7f92080b1e50 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.405+0000 7f920659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f920806e8b0 0x7f920815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.405+0000 7f920659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f920806e8b0 0x7f920815eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39110/0 (socket says 172.21.15.133:39110) 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920659c640 1 -- 172.21.15.133:0/3080160202 learned_addr learned my addr 172.21.15.133:0/3080160202 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920659c640 1 -- 172.21.15.133:0/3080160202 >> v2:172.21.15.162:3300/0 conn(0x7f92080b1e50 msgr2=0x7f9208146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f9205d9b640 1 --2- 172.21.15.133:0/3080160202 >> v2:172.21.15.162:3300/0 conn(0x7f92080b1e50 0x7f9208146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920659c640 1 --2- 172.21.15.133:0/3080160202 >> v2:172.21.15.162:3300/0 conn(0x7f92080b1e50 0x7f9208146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920559a640 1 -- 172.21.15.133:0/3080160202 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875696357 0 0) 0x7f92080b1090 con 0x7f92080aebc0 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920659c640 1 -- 172.21.15.133:0/3080160202 >> v1:172.21.15.133:6789/0 conn(0x7f92080aebc0 legacy=0x7f920806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920659c640 1 -- 172.21.15.133:0/3080160202 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92080b1370 con 0x7f920806e8b0 2024-04-25T10:16:01.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920659c640 1 --2- 172.21.15.133:0/3080160202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f920806e8b0 0x7f920815eb40 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f92080b4e30 tx=0x7f91f801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a943ab880e58e820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920559a640 1 -- 172.21.15.133:0/3080160202 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f80b1070 con 0x7f920806e8b0 2024-04-25T10:16:01.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920559a640 1 -- 172.21.15.133:0/3080160202 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f80ac600 con 0x7f920806e8b0 2024-04-25T10:16:01.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920559a640 1 -- 172.21.15.133:0/3080160202 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f80ac9d0 con 0x7f920806e8b0 2024-04-25T10:16:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920cd11640 1 -- 172.21.15.133:0/3080160202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f920806e8b0 msgr2=0x7f920815eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920cd11640 1 --2- 172.21.15.133:0/3080160202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f920806e8b0 0x7f920815eb40 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f92080b4e30 tx=0x7f91f801c490 comp rx=0 tx=0).stop 2024-04-25T10:16:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920cd11640 1 -- 172.21.15.133:0/3080160202 shutdown_connections 2024-04-25T10:16:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920cd11640 1 --2- 172.21.15.133:0/3080160202 >> v2:172.21.15.162:3300/0 conn(0x7f92080b1e50 0x7f9208146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920cd11640 1 --2- 172.21.15.133:0/3080160202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f920806e8b0 0x7f920815eb40 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920cd11640 1 -- 172.21.15.133:0/3080160202 >> 172.21.15.133:0/3080160202 conn(0x7f9208066f70 msgr2=0x7f920806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920cd11640 1 -- 172.21.15.133:0/3080160202 shutdown_connections 2024-04-25T10:16:01.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.409+0000 7f920cd11640 1 -- 172.21.15.133:0/3080160202 wait complete. 2024-04-25T10:16:01.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920cd11640 1 Processor -- start 2024-04-25T10:16:01.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920cd11640 1 -- start start 2024-04-25T10:16:01.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920cd11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f920806e8b0 0x7f9208142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920cd11640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92080aebc0 0x7f92080cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920cd11640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92080b1e50 0x7f92080adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920cd11640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92080b23b0 con 0x7f920806e8b0 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920cd11640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92080b2230 con 0x7f92080b1e50 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920cd11640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9208147020 con 0x7f92080aebc0 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f920806e8b0 0x7f9208142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f920806e8b0 0x7f9208142b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39112/0 (socket says 172.21.15.133:39112) 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920659c640 1 -- 172.21.15.133:0/176465842 learned_addr learned my addr 172.21.15.133:0/176465842 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f9205d9b640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92080aebc0 0x7f92080cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920659c640 1 -- 172.21.15.133:0/176465842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92080aebc0 msgr2=0x7f92080cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920659c640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92080aebc0 0x7f92080cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920659c640 1 -- 172.21.15.133:0/176465842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92080b1e50 msgr2=0x7f92080adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f9206d9d640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92080b1e50 0x7f92080adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920659c640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92080b1e50 0x7f92080adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920659c640 1 -- 172.21.15.133:0/176465842 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92081646f0 con 0x7f920806e8b0 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920659c640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f920806e8b0 0x7f9208142b00 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f91f801a0c0 tx=0x7f91f801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f91f6ffd640 1 -- 172.21.15.133:0/176465842 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f80b1040 con 0x7f920806e8b0 2024-04-25T10:16:01.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f91f6ffd640 1 -- 172.21.15.133:0/176465842 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f80bd070 con 0x7f920806e8b0 2024-04-25T10:16:01.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f91f6ffd640 1 -- 172.21.15.133:0/176465842 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f80b7820 con 0x7f920806e8b0 2024-04-25T10:16:01.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920cd11640 1 -- 172.21.15.133:0/176465842 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9208141d40 con 0x7f920806e8b0 2024-04-25T10:16:01.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920cd11640 1 -- 172.21.15.133:0/176465842 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9208142300 con 0x7f920806e8b0 2024-04-25T10:16:01.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f920cd11640 1 -- 172.21.15.133:0/176465842 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9208068e80 con 0x7f920806e8b0 2024-04-25T10:16:01.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f91f6ffd640 1 -- 172.21.15.133:0/176465842 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91f80b79c0 con 0x7f920806e8b0 2024-04-25T10:16:01.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.413+0000 7f91f6ffd640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91e8066f60 0x7f91e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.417+0000 7f91f6ffd640 1 -- 172.21.15.133:0/176465842 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f91f811ed70 con 0x7f920806e8b0 2024-04-25T10:16:01.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.417+0000 7f9205d9b640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91e8066f60 0x7f91e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.417+0000 7f9205d9b640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91e8066f60 0x7f91e8069420 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f92080b05e0 tx=0x7f91fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.421+0000 7f91f6ffd640 1 -- 172.21.15.133:0/176465842 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91f80c6020 con 0x7f920806e8b0 2024-04-25T10:16:01.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.577+0000 7f920cd11640 1 -- 172.21.15.133:0/176465842 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f920806b960 con 0x7f920806e8b0 2024-04-25T10:16:01.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.581+0000 7f91f6ffd640 1 -- 172.21.15.133:0/176465842 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f91f8016110 con 0x7f920806e8b0 2024-04-25T10:16:01.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 -- 172.21.15.133:0/176465842 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91e8066f60 msgr2=0x7f91e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91e8066f60 0x7f91e8069420 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f92080b05e0 tx=0x7f91fc008040 comp rx=0 tx=0).stop 2024-04-25T10:16:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 -- 172.21.15.133:0/176465842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f920806e8b0 msgr2=0x7f9208142b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f920806e8b0 0x7f9208142b00 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f91f801a0c0 tx=0x7f91f801acd0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 -- 172.21.15.133:0/176465842 shutdown_connections 2024-04-25T10:16:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92080b1e50 0x7f92080adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f92080aebc0 0x7f92080cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f91e8066f60 0x7f91e8069420 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 --2- 172.21.15.133:0/176465842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f920806e8b0 0x7f9208142b00 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 -- 172.21.15.133:0/176465842 >> 172.21.15.133:0/176465842 conn(0x7f9208066f70 msgr2=0x7f92080737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 -- 172.21.15.133:0/176465842 shutdown_connections 2024-04-25T10:16:01.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.585+0000 7f920cd11640 1 -- 172.21.15.133:0/176465842 wait complete. 2024-04-25T10:16:01.603 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:16:01.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e64353640 1 Processor -- start 2024-04-25T10:16:01.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e64353640 1 -- start start 2024-04-25T10:16:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e64353640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3e5c15c9a0 0x7f3e5c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e64353640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c0ce330 0x7f3e5c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e64353640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5c0c61d0 con 0x7f3e5c0b1f70 2024-04-25T10:16:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e64353640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5c0cecc0 con 0x7f3e5c15c9a0 2024-04-25T10:16:01.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e64353640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0c64b0 con 0x7f3e5c0ce330 2024-04-25T10:16:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e618ee640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3e5c0b1f70 0x7f3e5c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44162/0 (socket says 172.21.15.133:44162) 2024-04-25T10:16:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e618ee640 1 -- 172.21.15.133:0/725094935 learned_addr learned my addr 172.21.15.133:0/725094935 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e620ef640 1 --2- 172.21.15.133:0/725094935 >> v2:172.21.15.162:3300/0 conn(0x7f3e5c15c9a0 0x7f3e5c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e628f0640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c0ce330 0x7f3e5c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820616803 0 0) 0x7f3e5c0c61d0 con 0x7f3e5c0b1f70 2024-04-25T10:16:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e48003530 con 0x7f3e5c0b1f70 2024-04-25T10:16:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 466018426 0 0) 0x7f3e48003530 con 0x7f3e5c0b1f70 2024-04-25T10:16:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c0ce330 msgr2=0x7f3e5c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c0ce330 0x7f3e5c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 >> v2:172.21.15.162:3300/0 conn(0x7f3e5c15c9a0 msgr2=0x7f3e5c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 --2- 172.21.15.133:0/725094935 >> v2:172.21.15.162:3300/0 conn(0x7f3e5c15c9a0 0x7f3e5c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c06d4f0 con 0x7f3e5c0b1f70 2024-04-25T10:16:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f3e4c001680 con 0x7f3e5c0b1f70 2024-04-25T10:16:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 >> v1:172.21.15.133:6789/0 conn(0x7f3e5c0b1f70 legacy=0x7f3e5c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480039c0 0x7f3e48003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c0ce330 0x7f3e48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e48004d50 0x7f3e480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0cecc0 con 0x7f3e48004d50 2024-04-25T10:16:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0c64b0 con 0x7f3e480039c0 2024-04-25T10:16:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4800b9d0 con 0x7f3e5c0ce330 2024-04-25T10:16:01.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e628f0640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e48004d50 0x7f3e480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e618ee640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c0ce330 0x7f3e48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e620ef640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480039c0 0x7f3e48003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e628f0640 1 -- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c0ce330 msgr2=0x7f3e48004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e628f0640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c0ce330 0x7f3e48004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e628f0640 1 -- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480039c0 msgr2=0x7f3e48003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e628f0640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480039c0 0x7f3e48003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e628f0640 1 -- 172.21.15.133:0/725094935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e48003530 con 0x7f3e48004d50 2024-04-25T10:16:01.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e618ee640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c0ce330 0x7f3e48004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:01.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.721+0000 7f3e620ef640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e480039c0 0x7f3e48003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e628f0640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e48004d50 0x7f3e480051d0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400e170 tx=0x7f3e5401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00051b569eaf539 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540b2070 con 0x7f3e48004d50 2024-04-25T10:16:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e540a5c70 con 0x7f3e48004d50 2024-04-25T10:16:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e610ed640 1 -- 172.21.15.133:0/725094935 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54009c10 con 0x7f3e48004d50 2024-04-25T10:16:01.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 -- 172.21.15.133:0/725094935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e48004d50 msgr2=0x7f3e480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 --2- 172.21.15.133:0/725094935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e48004d50 0x7f3e480051d0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400e170 tx=0x7f3e5401cc60 comp rx=0 tx=0).stop 2024-04-25T10:16:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e628f0640 1 -- 172.21.15.133:0/725094935 reap_dead start 2024-04-25T10:16:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 -- 172.21.15.133:0/725094935 shutdown_connections 2024-04-25T10:16:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 -- 172.21.15.133:0/725094935 >> 172.21.15.133:0/725094935 conn(0x7f3e5c067000 msgr2=0x7f3e5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 -- 172.21.15.133:0/725094935 shutdown_connections 2024-04-25T10:16:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 -- 172.21.15.133:0/725094935 wait complete. 2024-04-25T10:16:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 Processor -- start 2024-04-25T10:16:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 -- start start 2024-04-25T10:16:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c142d90 0x7f3e5c0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e5c143170 0x7f3e5c145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e5c145c50 0x7f3e5c15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c15cd80 con 0x7f3e5c143170 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0c5ec0 con 0x7f3e5c145c50 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e64353640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c15cf00 con 0x7f3e5c142d90 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e618ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e5c143170 0x7f3e5c145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.725+0000 7f3e618ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e5c143170 0x7f3e5c145710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39142/0 (socket says 172.21.15.133:39142) 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e618ee640 1 -- 172.21.15.133:0/2538784757 learned_addr learned my addr 172.21.15.133:0/2538784757 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e620ef640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c142d90 0x7f3e5c0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e628f0640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e5c145c50 0x7f3e5c15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e618ee640 1 -- 172.21.15.133:0/2538784757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c142d90 msgr2=0x7f3e5c0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e618ee640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c142d90 0x7f3e5c0c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e618ee640 1 -- 172.21.15.133:0/2538784757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e5c145c50 msgr2=0x7f3e5c15ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e618ee640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e5c145c50 0x7f3e5c15ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e618ee640 1 -- 172.21.15.133:0/2538784757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c1461d0 con 0x7f3e5c143170 2024-04-25T10:16:01.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e618ee640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e5c143170 0x7f3e5c145710 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00d870 tx=0x7f3e4c00dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e52ffd640 1 -- 172.21.15.133:0/2538784757 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c019070 con 0x7f3e5c143170 2024-04-25T10:16:01.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e52ffd640 1 -- 172.21.15.133:0/2538784757 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c0074e0 con 0x7f3e5c143170 2024-04-25T10:16:01.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e64353640 1 -- 172.21.15.133:0/2538784757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c1464b0 con 0x7f3e5c143170 2024-04-25T10:16:01.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e52ffd640 1 -- 172.21.15.133:0/2538784757 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c0049f0 con 0x7f3e5c143170 2024-04-25T10:16:01.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e64353640 1 -- 172.21.15.133:0/2538784757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c0c81d0 con 0x7f3e5c143170 2024-04-25T10:16:01.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e64353640 1 -- 172.21.15.133:0/2538784757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5c068ef0 con 0x7f3e5c143170 2024-04-25T10:16:01.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e52ffd640 1 -- 172.21.15.133:0/2538784757 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e4c01c070 con 0x7f3e5c143170 2024-04-25T10:16:01.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e52ffd640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e40066f60 0x7f3e40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e52ffd640 1 -- 172.21.15.133:0/2538784757 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3e4c084af0 con 0x7f3e5c143170 2024-04-25T10:16:01.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e620ef640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e40066f60 0x7f3e40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.729+0000 7f3e620ef640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e40066f60 0x7f3e40069420 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009b40 tx=0x7f3e58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.733+0000 7f3e52ffd640 1 -- 172.21.15.133:0/2538784757 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e4c055be0 con 0x7f3e5c143170 2024-04-25T10:16:01.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:01.949+0000 7f3e64353640 1 -- 172.21.15.133:0/2538784757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7f3e5c0b3d10 con 0x7f3e5c143170 2024-04-25T10:16:02.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.261+0000 7f3e52ffd640 1 -- 172.21.15.133:0/2538784757 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v635) v1 ==== 140+0+0 (secure 0 0 0) 0x7f3e4c0554e0 con 0x7f3e5c143170 2024-04-25T10:16:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 -- 172.21.15.133:0/2538784757 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e40066f60 msgr2=0x7f3e40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e40066f60 0x7f3e40069420 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009b40 tx=0x7f3e58008040 comp rx=0 tx=0).stop 2024-04-25T10:16:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 -- 172.21.15.133:0/2538784757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e5c143170 msgr2=0x7f3e5c145710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e5c143170 0x7f3e5c145710 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00d870 tx=0x7f3e4c00dd30 comp rx=0 tx=0).stop 2024-04-25T10:16:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 -- 172.21.15.133:0/2538784757 shutdown_connections 2024-04-25T10:16:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e5c145c50 0x7f3e5c15ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3e40066f60 0x7f3e40069420 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e5c143170 0x7f3e5c145710 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 --2- 172.21.15.133:0/2538784757 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3e5c142d90 0x7f3e5c0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 -- 172.21.15.133:0/2538784757 >> 172.21.15.133:0/2538784757 conn(0x7f3e5c067000 msgr2=0x7f3e5c0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 -- 172.21.15.133:0/2538784757 shutdown_connections 2024-04-25T10:16:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.265+0000 7f3e64353640 1 -- 172.21.15.133:0/2538784757 wait complete. 2024-04-25T10:16:02.287 INFO:tasks.cephfs_test_runner:test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:16:02.287 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:16:02.287 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:02.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee64801640 1 Processor -- start 2024-04-25T10:16:02.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee64801640 1 -- start start 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee64801640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fee6015c9a0 0x7fee600c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee64801640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fee600b1f70 0x7fee6006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee64801640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee600c61d0 con 0x7fee600ce330 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee64801640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee600c64b0 con 0x7fee600b1f70 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee64801640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6006d4f0 con 0x7fee6015c9a0 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee5dd9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fee600b1f70 0x7fee6006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee5ed9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fee600ce330 0x7fee6015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44164/0 (socket says 172.21.15.133:44164) 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee5ed9d640 1 -- 172.21.15.133:0/780442148 learned_addr learned my addr 172.21.15.133:0/780442148 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee5dd9b640 1 -- 172.21.15.133:0/780442148 >> v2:172.21.15.162:3300/0 conn(0x7fee6015c9a0 msgr2=0x7fee600c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee5dd9b640 1 --2- 172.21.15.133:0/780442148 >> v2:172.21.15.162:3300/0 conn(0x7fee6015c9a0 0x7fee600c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee5dd9b640 1 -- 172.21.15.133:0/780442148 >> v1:172.21.15.133:6789/0 conn(0x7fee600ce330 legacy=0x7fee6015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee5d59a640 1 -- 172.21.15.133:0/780442148 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593825146 0 0) 0x7fee600c61d0 con 0x7fee600ce330 2024-04-25T10:16:02.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.409+0000 7fee5dd9b640 1 -- 172.21.15.133:0/780442148 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6006d7d0 con 0x7fee600b1f70 2024-04-25T10:16:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee5dd9b640 1 --2- 172.21.15.133:0/780442148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee600b1f70 0x7fee6006dcf0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fee6006f5f0 tx=0x7fee5401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388dc308e3cbccc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee5d59a640 1 -- 172.21.15.133:0/780442148 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee540ab640 con 0x7fee600b1f70 2024-04-25T10:16:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee5d59a640 1 -- 172.21.15.133:0/780442148 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee540b0070 con 0x7fee600b1f70 2024-04-25T10:16:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee5d59a640 1 -- 172.21.15.133:0/780442148 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee540aba00 con 0x7fee600b1f70 2024-04-25T10:16:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee64801640 1 -- 172.21.15.133:0/780442148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee600b1f70 msgr2=0x7fee6006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee64801640 1 --2- 172.21.15.133:0/780442148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee600b1f70 0x7fee6006dcf0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fee6006f5f0 tx=0x7fee5401c420 comp rx=0 tx=0).stop 2024-04-25T10:16:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee64801640 1 -- 172.21.15.133:0/780442148 shutdown_connections 2024-04-25T10:16:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee64801640 1 --2- 172.21.15.133:0/780442148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee600b1f70 0x7fee6006dcf0 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee64801640 1 --2- 172.21.15.133:0/780442148 >> v2:172.21.15.162:3300/0 conn(0x7fee6015c9a0 0x7fee600c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee64801640 1 -- 172.21.15.133:0/780442148 >> 172.21.15.133:0/780442148 conn(0x7fee60067000 msgr2=0x7fee6006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee64801640 1 -- 172.21.15.133:0/780442148 shutdown_connections 2024-04-25T10:16:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee64801640 1 -- 172.21.15.133:0/780442148 wait complete. 2024-04-25T10:16:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee64801640 1 Processor -- start 2024-04-25T10:16:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.413+0000 7fee64801640 1 -- start start 2024-04-25T10:16:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee64801640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee600b1f70 0x7fee600cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee64801640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee600ce330 0x7fee601599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee64801640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee6015c9a0 0x7fee60159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee64801640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee600ceb50 con 0x7fee600b1f70 2024-04-25T10:16:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee64801640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6015cd80 con 0x7fee6015c9a0 2024-04-25T10:16:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee64801640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee600cb720 con 0x7fee600ce330 2024-04-25T10:16:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee5e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee600b1f70 0x7fee600cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee5e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee600b1f70 0x7fee600cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39164/0 (socket says 172.21.15.133:39164) 2024-04-25T10:16:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee5e59c640 1 -- 172.21.15.133:0/296504325 learned_addr learned my addr 172.21.15.133:0/296504325 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee5e59c640 1 -- 172.21.15.133:0/296504325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee600ce330 msgr2=0x7fee601599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee5e59c640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee600ce330 0x7fee601599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee5e59c640 1 -- 172.21.15.133:0/296504325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee6015c9a0 msgr2=0x7fee60159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee5e59c640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee6015c9a0 0x7fee60159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee5e59c640 1 -- 172.21.15.133:0/296504325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee600b3a30 con 0x7fee600b1f70 2024-04-25T10:16:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee5e59c640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee600b1f70 0x7fee600cb0b0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fee5000b550 tx=0x7fee5000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee4effd640 1 -- 172.21.15.133:0/296504325 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50014070 con 0x7fee600b1f70 2024-04-25T10:16:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee4effd640 1 -- 172.21.15.133:0/296504325 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee50010890 con 0x7fee600b1f70 2024-04-25T10:16:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee64801640 1 -- 172.21.15.133:0/296504325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee600b3d10 con 0x7fee600b1f70 2024-04-25T10:16:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee4effd640 1 -- 172.21.15.133:0/296504325 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50010c80 con 0x7fee600b1f70 2024-04-25T10:16:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee64801640 1 -- 172.21.15.133:0/296504325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee601471f0 con 0x7fee600b1f70 2024-04-25T10:16:02.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee64801640 1 -- 172.21.15.133:0/296504325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee60068ef0 con 0x7fee600b1f70 2024-04-25T10:16:02.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee4effd640 1 -- 172.21.15.133:0/296504325 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee50017070 con 0x7fee600b1f70 2024-04-25T10:16:02.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee4effd640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee40066f60 0x7fee40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee5dd9b640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee40066f60 0x7fee40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee4effd640 1 -- 172.21.15.133:0/296504325 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fee50084c20 con 0x7fee600b1f70 2024-04-25T10:16:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.417+0000 7fee5dd9b640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee40066f60 0x7fee40069420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fee540af000 tx=0x7fee54004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.421+0000 7fee4effd640 1 -- 172.21.15.133:0/296504325 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee50055e30 con 0x7fee600b1f70 2024-04-25T10:16:02.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.581+0000 7fee64801640 1 -- 172.21.15.133:0/296504325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee6006e6d0 con 0x7fee600b1f70 2024-04-25T10:16:02.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.581+0000 7fee4effd640 1 -- 172.21.15.133:0/296504325 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+13329 (secure 0 0 0) 0x7fee50010e20 con 0x7fee600b1f70 2024-04-25T10:16:02.584 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:02.584 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":160,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:16:01.250109+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6833/3763981385":"2024-04-26T10:15:58.232263+0000","172.21.15.162:6832/3763981385":"2024-04-26T10:15:58.232263+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:02.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 -- 172.21.15.133:0/296504325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee40066f60 msgr2=0x7fee40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee40066f60 0x7fee40069420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fee540af000 tx=0x7fee54004740 comp rx=0 tx=0).stop 2024-04-25T10:16:02.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 -- 172.21.15.133:0/296504325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee600b1f70 msgr2=0x7fee600cb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee600b1f70 0x7fee600cb0b0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fee5000b550 tx=0x7fee5000ba10 comp rx=0 tx=0).stop 2024-04-25T10:16:02.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 -- 172.21.15.133:0/296504325 shutdown_connections 2024-04-25T10:16:02.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee6015c9a0 0x7fee60159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee600ce330 0x7fee601599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee40066f60 0x7fee40069420 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 --2- 172.21.15.133:0/296504325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee600b1f70 0x7fee600cb0b0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 -- 172.21.15.133:0/296504325 >> 172.21.15.133:0/296504325 conn(0x7fee60067000 msgr2=0x7fee6015afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 -- 172.21.15.133:0/296504325 shutdown_connections 2024-04-25T10:16:02.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.585+0000 7fee64801640 1 -- 172.21.15.133:0/296504325 wait complete. 2024-04-25T10:16:02.603 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:02.603 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:02.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.721+0000 7fe927f70640 1 Processor -- start 2024-04-25T10:16:02.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe927f70640 1 -- start start 2024-04-25T10:16:02.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe927f70640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fe920145170 0x7fe92006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe927f70640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe92006e4d0 0x7fe9200cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe927f70640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9200ce2e0 con 0x7fe9200cdd60 2024-04-25T10:16:02.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe927f70640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92006db00 con 0x7fe92006e4d0 2024-04-25T10:16:02.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe927f70640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe920145550 con 0x7fe920145170 2024-04-25T10:16:02.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe92650d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe9200cdd60 0x7fe92015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44178/0 (socket says 172.21.15.133:44178) 2024-04-25T10:16:02.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe92550b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe92006e4d0 0x7fe9200cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe92650d640 1 -- 172.21.15.133:0/708396786 learned_addr learned my addr 172.21.15.133:0/708396786 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:02.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe924d0a640 1 -- 172.21.15.133:0/708396786 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639935323 0 0) 0x7fe9200ce2e0 con 0x7fe9200cdd60 2024-04-25T10:16:02.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe925d0c640 1 --2- 172.21.15.133:0/708396786 >> v2:172.21.15.162:3300/0 conn(0x7fe920145170 0x7fe92006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe924d0a640 1 -- 172.21.15.133:0/708396786 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe90c003530 con 0x7fe9200cdd60 2024-04-25T10:16:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe92550b640 1 -- 172.21.15.133:0/708396786 >> v2:172.21.15.162:3300/0 conn(0x7fe920145170 msgr2=0x7fe92006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe92550b640 1 --2- 172.21.15.133:0/708396786 >> v2:172.21.15.162:3300/0 conn(0x7fe920145170 0x7fe92006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe92550b640 1 -- 172.21.15.133:0/708396786 >> v1:172.21.15.133:6789/0 conn(0x7fe9200cdd60 legacy=0x7fe92015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe92550b640 1 -- 172.21.15.133:0/708396786 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92006d4b0 con 0x7fe92006e4d0 2024-04-25T10:16:02.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.725+0000 7fe92550b640 1 --2- 172.21.15.133:0/708396786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe92006e4d0 0x7fe9200cd820 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fe92006b790 tx=0x7fe91001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aaa2767475e138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe924d0a640 1 -- 172.21.15.133:0/708396786 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9100b0070 con 0x7fe92006e4d0 2024-04-25T10:16:02.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe924d0a640 1 -- 172.21.15.133:0/708396786 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9100ab600 con 0x7fe92006e4d0 2024-04-25T10:16:02.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe924d0a640 1 -- 172.21.15.133:0/708396786 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9100ab9d0 con 0x7fe92006e4d0 2024-04-25T10:16:02.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 -- 172.21.15.133:0/708396786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe92006e4d0 msgr2=0x7fe9200cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 --2- 172.21.15.133:0/708396786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe92006e4d0 0x7fe9200cd820 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fe92006b790 tx=0x7fe91001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 -- 172.21.15.133:0/708396786 shutdown_connections 2024-04-25T10:16:02.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 --2- 172.21.15.133:0/708396786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe92006e4d0 0x7fe9200cd820 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 --2- 172.21.15.133:0/708396786 >> v2:172.21.15.162:3300/0 conn(0x7fe920145170 0x7fe92006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 -- 172.21.15.133:0/708396786 >> 172.21.15.133:0/708396786 conn(0x7fe920066f70 msgr2=0x7fe92006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 -- 172.21.15.133:0/708396786 shutdown_connections 2024-04-25T10:16:02.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 -- 172.21.15.133:0/708396786 wait complete. 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 Processor -- start 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 -- start start 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe92006e4d0 0x7fe920143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200cdd60 0x7fe92013ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe920145170 0x7fe92013f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe920168490 con 0x7fe9200cdd60 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92015d7f0 con 0x7fe92006e4d0 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe920168610 con 0x7fe920145170 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe92550b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200cdd60 0x7fe92013ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe92550b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200cdd60 0x7fe92013ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39174/0 (socket says 172.21.15.133:39174) 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe92550b640 1 -- 172.21.15.133:0/464383385 learned_addr learned my addr 172.21.15.133:0/464383385 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe92550b640 1 -- 172.21.15.133:0/464383385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe920145170 msgr2=0x7fe92013f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe92550b640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe920145170 0x7fe92013f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe92550b640 1 -- 172.21.15.133:0/464383385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe92006e4d0 msgr2=0x7fe920143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe92550b640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe92006e4d0 0x7fe920143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe92550b640 1 -- 172.21.15.133:0/464383385 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92013b2d0 con 0x7fe9200cdd60 2024-04-25T10:16:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe92550b640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200cdd60 0x7fe92013ee30 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fe91001a020 tx=0x7fe910019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe9167fc640 1 -- 172.21.15.133:0/464383385 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9100b0040 con 0x7fe9200cdd60 2024-04-25T10:16:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe9167fc640 1 -- 172.21.15.133:0/464383385 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9100b9070 con 0x7fe9200cdd60 2024-04-25T10:16:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 -- 172.21.15.133:0/464383385 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe920142360 con 0x7fe9200cdd60 2024-04-25T10:16:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe9167fc640 1 -- 172.21.15.133:0/464383385 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910006e20 con 0x7fe9200cdd60 2024-04-25T10:16:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.729+0000 7fe927f70640 1 -- 172.21.15.133:0/464383385 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe920142920 con 0x7fe9200cdd60 2024-04-25T10:16:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.733+0000 7fe927f70640 1 -- 172.21.15.133:0/464383385 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe920068e80 con 0x7fe9200cdd60 2024-04-25T10:16:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.733+0000 7fe9167fc640 1 -- 172.21.15.133:0/464383385 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9100b4650 con 0x7fe9200cdd60 2024-04-25T10:16:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.733+0000 7fe9167fc640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe904066f60 0x7fe904069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.733+0000 7fe9167fc640 1 -- 172.21.15.133:0/464383385 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe91011eb40 con 0x7fe9200cdd60 2024-04-25T10:16:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.733+0000 7fe925d0c640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe904066f60 0x7fe904069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.733+0000 7fe925d0c640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe904066f60 0x7fe904069420 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0070b0 tx=0x7fe91c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.737+0000 7fe9167fc640 1 -- 172.21.15.133:0/464383385 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe910016070 con 0x7fe9200cdd60 2024-04-25T10:16:02.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.897+0000 7fe927f70640 1 -- 172.21.15.133:0/464383385 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe920073a80 con 0x7fe9200cdd60 2024-04-25T10:16:02.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.897+0000 7fe9167fc640 1 -- 172.21.15.133:0/464383385 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9100efcb0 con 0x7fe9200cdd60 2024-04-25T10:16:02.898 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:16:02.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.897+0000 7fe927f70640 1 -- 172.21.15.133:0/464383385 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe904066f60 msgr2=0x7fe904069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.897+0000 7fe927f70640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe904066f60 0x7fe904069420 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0070b0 tx=0x7fe91c007040 comp rx=0 tx=0).stop 2024-04-25T10:16:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.897+0000 7fe927f70640 1 -- 172.21.15.133:0/464383385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200cdd60 msgr2=0x7fe92013ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.897+0000 7fe927f70640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200cdd60 0x7fe92013ee30 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fe91001a020 tx=0x7fe910019190 comp rx=0 tx=0).stop 2024-04-25T10:16:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.897+0000 7fe927f70640 1 -- 172.21.15.133:0/464383385 shutdown_connections 2024-04-25T10:16:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.897+0000 7fe927f70640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe920145170 0x7fe92013f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.897+0000 7fe927f70640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe904066f60 0x7fe904069420 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.897+0000 7fe927f70640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9200cdd60 0x7fe92013ee30 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.897+0000 7fe927f70640 1 --2- 172.21.15.133:0/464383385 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe92006e4d0 0x7fe920143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.901+0000 7fe927f70640 1 -- 172.21.15.133:0/464383385 >> 172.21.15.133:0/464383385 conn(0x7fe920066f70 msgr2=0x7fe92015e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.901+0000 7fe927f70640 1 -- 172.21.15.133:0/464383385 shutdown_connections 2024-04-25T10:16:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:02.901+0000 7fe927f70640 1 -- 172.21.15.133:0/464383385 wait complete. 2024-04-25T10:16:02.917 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:16:03.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3169928640 1 Processor -- start 2024-04-25T10:16:03.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3169928640 1 -- start start 2024-04-25T10:16:03.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3169928640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f316415c9a0 0x7f31640c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3169928640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f31640ce330 0x7f31640ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3169928640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31640c61d0 con 0x7f31640b1f70 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3169928640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31640cecc0 con 0x7f316415c9a0 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3169928640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31640c64b0 con 0x7f31640ce330 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3162ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f316415c9a0 0x7f31640c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f31627fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f31640b1f70 0x7f316406dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44180/0 (socket says 172.21.15.133:44180) 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3162ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f316415c9a0 0x7f31640c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39186/0 (socket says 172.21.15.133:39186) 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f31627fc640 1 -- 172.21.15.133:0/3212096686 learned_addr learned my addr 172.21.15.133:0/3212096686 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f31637fe640 1 --2- 172.21.15.133:0/3212096686 >> v2:172.21.15.162:3300/0 conn(0x7f31640ce330 0x7f31640ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3162ffd640 1 -- 172.21.15.133:0/3212096686 >> v2:172.21.15.162:3300/0 conn(0x7f31640ce330 msgr2=0x7f31640ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3162ffd640 1 --2- 172.21.15.133:0/3212096686 >> v2:172.21.15.162:3300/0 conn(0x7f31640ce330 0x7f31640ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3162ffd640 1 -- 172.21.15.133:0/3212096686 >> v1:172.21.15.133:6789/0 conn(0x7f31640b1f70 legacy=0x7f316406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3162ffd640 1 -- 172.21.15.133:0/3212096686 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316406d4f0 con 0x7f316415c9a0 2024-04-25T10:16:03.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.037+0000 7f3161ffb640 1 -- 172.21.15.133:0/3212096686 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069092104 0 0) 0x7f31640c61d0 con 0x7f31640b1f70 2024-04-25T10:16:03.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3162ffd640 1 --2- 172.21.15.133:0/3212096686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316415c9a0 0x7f31640c5980 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f31640cc0f0 tx=0x7f315801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3da6a5741c4168 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3161ffb640 1 -- 172.21.15.133:0/3212096686 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31580b0070 con 0x7f316415c9a0 2024-04-25T10:16:03.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3161ffb640 1 -- 172.21.15.133:0/3212096686 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31580ab600 con 0x7f316415c9a0 2024-04-25T10:16:03.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3161ffb640 1 -- 172.21.15.133:0/3212096686 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31580ab9d0 con 0x7f316415c9a0 2024-04-25T10:16:03.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 -- 172.21.15.133:0/3212096686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316415c9a0 msgr2=0x7f31640c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 --2- 172.21.15.133:0/3212096686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316415c9a0 0x7f31640c5980 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f31640cc0f0 tx=0x7f315801c480 comp rx=0 tx=0).stop 2024-04-25T10:16:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 -- 172.21.15.133:0/3212096686 shutdown_connections 2024-04-25T10:16:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 --2- 172.21.15.133:0/3212096686 >> v2:172.21.15.162:3300/0 conn(0x7f31640ce330 0x7f31640ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 --2- 172.21.15.133:0/3212096686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316415c9a0 0x7f31640c5980 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 -- 172.21.15.133:0/3212096686 >> 172.21.15.133:0/3212096686 conn(0x7f3164067000 msgr2=0x7f316406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 -- 172.21.15.133:0/3212096686 shutdown_connections 2024-04-25T10:16:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 -- 172.21.15.133:0/3212096686 wait complete. 2024-04-25T10:16:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 Processor -- start 2024-04-25T10:16:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 -- start start 2024-04-25T10:16:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31640b1f70 0x7f316414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31640ce330 0x7f316414b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316415c9a0 0x7f3164149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316415dad0 con 0x7f316415c9a0 2024-04-25T10:16:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316415d950 con 0x7f31640ce330 2024-04-25T10:16:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3169928640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316415dc50 con 0x7f31640b1f70 2024-04-25T10:16:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f31637fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316415c9a0 0x7f3164149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f31637fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316415c9a0 0x7f3164149740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39188/0 (socket says 172.21.15.133:39188) 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f31637fe640 1 -- 172.21.15.133:0/1745896792 learned_addr learned my addr 172.21.15.133:0/1745896792 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f31637fe640 1 -- 172.21.15.133:0/1745896792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31640b1f70 msgr2=0x7f316414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f3162ffd640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31640b1f70 0x7f316414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f31627fc640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31640ce330 0x7f316414b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f31637fe640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31640b1f70 0x7f316414cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f31637fe640 1 -- 172.21.15.133:0/1745896792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31640ce330 msgr2=0x7f316414b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f31637fe640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31640ce330 0x7f316414b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f31637fe640 1 -- 172.21.15.133:0/1745896792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316414fd70 con 0x7f316415c9a0 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.041+0000 7f31637fe640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316415c9a0 0x7f3164149740 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f315c007e60 tx=0x7f315c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.045+0000 7f314bfff640 1 -- 172.21.15.133:0/1745896792 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c014420 con 0x7f316415c9a0 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.045+0000 7f314bfff640 1 -- 172.21.15.133:0/1745896792 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c019070 con 0x7f316415c9a0 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.045+0000 7f3169928640 1 -- 172.21.15.133:0/1745896792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3164150050 con 0x7f316415c9a0 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.045+0000 7f314bfff640 1 -- 172.21.15.133:0/1745896792 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c014820 con 0x7f316415c9a0 2024-04-25T10:16:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.045+0000 7f3169928640 1 -- 172.21.15.133:0/1745896792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3164150610 con 0x7f316415c9a0 2024-04-25T10:16:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.045+0000 7f3169928640 1 -- 172.21.15.133:0/1745896792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3164068ef0 con 0x7f316415c9a0 2024-04-25T10:16:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.045+0000 7f314bfff640 1 -- 172.21.15.133:0/1745896792 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f315c0149c0 con 0x7f316415c9a0 2024-04-25T10:16:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.045+0000 7f314bfff640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3150066f60 0x7f3150069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.045+0000 7f314bfff640 1 -- 172.21.15.133:0/1745896792 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f315c085760 con 0x7f316415c9a0 2024-04-25T10:16:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.045+0000 7f3162ffd640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3150066f60 0x7f3150069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.045+0000 7f3162ffd640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3150066f60 0x7f3150069420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f31580ad000 tx=0x7f31580a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.049+0000 7f314bfff640 1 -- 172.21.15.133:0/1745896792 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f315c0568e0 con 0x7f316415c9a0 2024-04-25T10:16:03.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.249+0000 7f3169928640 1 -- 172.21.15.133:0/1745896792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7f316415df10 con 0x7f316415c9a0 2024-04-25T10:16:03.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.261+0000 7f314bfff640 1 -- 172.21.15.133:0/1745896792 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v636) v1 ==== 136+0+0 (secure 0 0 0) 0x7f315c0561e0 con 0x7f316415c9a0 2024-04-25T10:16:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 -- 172.21.15.133:0/1745896792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3150066f60 msgr2=0x7f3150069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3150066f60 0x7f3150069420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f31580ad000 tx=0x7f31580a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 -- 172.21.15.133:0/1745896792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316415c9a0 msgr2=0x7f3164149740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316415c9a0 0x7f3164149740 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f315c007e60 tx=0x7f315c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 -- 172.21.15.133:0/1745896792 shutdown_connections 2024-04-25T10:16:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3150066f60 0x7f3150069420 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316415c9a0 0x7f3164149740 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f31640ce330 0x7f316414b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 --2- 172.21.15.133:0/1745896792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f31640b1f70 0x7f316414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 -- 172.21.15.133:0/1745896792 >> 172.21.15.133:0/1745896792 conn(0x7f3164067000 msgr2=0x7f31640b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 -- 172.21.15.133:0/1745896792 shutdown_connections 2024-04-25T10:16:03.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.265+0000 7f3169928640 1 -- 172.21.15.133:0/1745896792 wait complete. 2024-04-25T10:16:03.281 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:16:03.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61cd110640 1 Processor -- start 2024-04-25T10:16:03.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61cd110640 1 -- start start 2024-04-25T10:16:03.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61cd110640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f61c8145170 0x7f61c806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61cd110640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c806e4d0 0x7f61c80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61cd110640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61c80cdd60 0x7f61c815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61cd110640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c806db00 con 0x7f61c80cdd60 2024-04-25T10:16:03.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61cd110640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c8145550 con 0x7f61c8145170 2024-04-25T10:16:03.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61cd110640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80ce2e0 con 0x7f61c806e4d0 2024-04-25T10:16:03.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61c759e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61c80cdd60 0x7f61c815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61c759e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61c80cdd60 0x7f61c815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39196/0 (socket says 172.21.15.133:39196) 2024-04-25T10:16:03.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61c759e640 1 -- 172.21.15.133:0/11741513 learned_addr learned my addr 172.21.15.133:0/11741513 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:03.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61c6d9d640 1 --2- 172.21.15.133:0/11741513 >> v2:172.21.15.162:3300/0 conn(0x7f61c8145170 0x7f61c806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61bffff640 1 --2- 172.21.15.133:0/11741513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c806e4d0 0x7f61c80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61c759e640 1 -- 172.21.15.133:0/11741513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c806e4d0 msgr2=0x7f61c80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61c759e640 1 --2- 172.21.15.133:0/11741513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c806e4d0 0x7f61c80cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61c759e640 1 -- 172.21.15.133:0/11741513 >> v2:172.21.15.162:3300/0 conn(0x7f61c8145170 msgr2=0x7f61c806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61c759e640 1 --2- 172.21.15.133:0/11741513 >> v2:172.21.15.162:3300/0 conn(0x7f61c8145170 0x7f61c806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.401+0000 7f61c759e640 1 -- 172.21.15.133:0/11741513 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c806d4b0 con 0x7f61c80cdd60 2024-04-25T10:16:03.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c759e640 1 --2- 172.21.15.133:0/11741513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c80cdd60 0x7f61c815ced0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f61c806b790 tx=0x7f61c001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45aaa8a4f7a27cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c659c640 1 -- 172.21.15.133:0/11741513 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00a5670 con 0x7f61c80cdd60 2024-04-25T10:16:03.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c659c640 1 -- 172.21.15.133:0/11741513 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c00a5810 con 0x7f61c80cdd60 2024-04-25T10:16:03.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c659c640 1 -- 172.21.15.133:0/11741513 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00a5be0 con 0x7f61c80cdd60 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 -- 172.21.15.133:0/11741513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c80cdd60 msgr2=0x7f61c815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 --2- 172.21.15.133:0/11741513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c80cdd60 0x7f61c815ced0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f61c806b790 tx=0x7f61c001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 -- 172.21.15.133:0/11741513 shutdown_connections 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 --2- 172.21.15.133:0/11741513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c80cdd60 0x7f61c815ced0 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 --2- 172.21.15.133:0/11741513 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c806e4d0 0x7f61c80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 --2- 172.21.15.133:0/11741513 >> v2:172.21.15.162:3300/0 conn(0x7f61c8145170 0x7f61c806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 -- 172.21.15.133:0/11741513 >> 172.21.15.133:0/11741513 conn(0x7f61c8066f70 msgr2=0x7f61c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 -- 172.21.15.133:0/11741513 shutdown_connections 2024-04-25T10:16:03.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 -- 172.21.15.133:0/11741513 wait complete. 2024-04-25T10:16:03.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 Processor -- start 2024-04-25T10:16:03.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 -- start start 2024-04-25T10:16:03.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c806e4d0 0x7f61c8135eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c80cdd60 0x7f61c813bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c8145170 0x7f61c813fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c815f690 con 0x7f61c8145170 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c815f510 con 0x7f61c80cdd60 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61cd110640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c815f810 con 0x7f61c806e4d0 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c8145170 0x7f61c813fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c8145170 0x7f61c813fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39200/0 (socket says 172.21.15.133:39200) 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c759e640 1 -- 172.21.15.133:0/2008746671 learned_addr learned my addr 172.21.15.133:0/2008746671 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61bffff640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c80cdd60 0x7f61c813bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c6d9d640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c806e4d0 0x7f61c8135eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c759e640 1 -- 172.21.15.133:0/2008746671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c806e4d0 msgr2=0x7f61c8135eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c759e640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c806e4d0 0x7f61c8135eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c759e640 1 -- 172.21.15.133:0/2008746671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c80cdd60 msgr2=0x7f61c813bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c759e640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c80cdd60 0x7f61c813bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61c759e640 1 -- 172.21.15.133:0/2008746671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c8156060 con 0x7f61c8145170 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.405+0000 7f61bffff640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c80cdd60 0x7f61c813bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:03.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61c759e640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c8145170 0x7f61c813fff0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f61c001b0a0 tx=0x7f61c0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61bf7fe640 1 -- 172.21.15.133:0/2008746671 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00bc020 con 0x7f61c8145170 2024-04-25T10:16:03.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61bf7fe640 1 -- 172.21.15.133:0/2008746671 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c00b8670 con 0x7f61c8145170 2024-04-25T10:16:03.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61cd110640 1 -- 172.21.15.133:0/2008746671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c81554e0 con 0x7f61c8145170 2024-04-25T10:16:03.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61bf7fe640 1 -- 172.21.15.133:0/2008746671 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0009510 con 0x7f61c8145170 2024-04-25T10:16:03.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61cd110640 1 -- 172.21.15.133:0/2008746671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c8155aa0 con 0x7f61c8145170 2024-04-25T10:16:03.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61cd110640 1 -- 172.21.15.133:0/2008746671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c8068e80 con 0x7f61c8145170 2024-04-25T10:16:03.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61bf7fe640 1 -- 172.21.15.133:0/2008746671 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61c00bf020 con 0x7f61c8145170 2024-04-25T10:16:03.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61bf7fe640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61a8066f60 0x7f61a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61bf7fe640 1 -- 172.21.15.133:0/2008746671 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f61c011fff0 con 0x7f61c8145170 2024-04-25T10:16:03.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61c6d9d640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61a8066f60 0x7f61a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.409+0000 7f61c6d9d640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61a8066f60 0x7f61a8069420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f61b800b930 tx=0x7f61b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.413+0000 7f61bf7fe640 1 -- 172.21.15.133:0/2008746671 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61c00c6030 con 0x7f61c8145170 2024-04-25T10:16:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.569+0000 7f61cd110640 1 -- 172.21.15.133:0/2008746671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f61c813a110 con 0x7f61c8145170 2024-04-25T10:16:03.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.573+0000 7f61bf7fe640 1 -- 172.21.15.133:0/2008746671 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f61c00165e0 con 0x7f61c8145170 2024-04-25T10:16:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 -- 172.21.15.133:0/2008746671 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61a8066f60 msgr2=0x7f61a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61a8066f60 0x7f61a8069420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f61b800b930 tx=0x7f61b8008040 comp rx=0 tx=0).stop 2024-04-25T10:16:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 -- 172.21.15.133:0/2008746671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c8145170 msgr2=0x7f61c813fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c8145170 0x7f61c813fff0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f61c001b0a0 tx=0x7f61c0019790 comp rx=0 tx=0).stop 2024-04-25T10:16:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 -- 172.21.15.133:0/2008746671 shutdown_connections 2024-04-25T10:16:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61a8066f60 0x7f61a8069420 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c8145170 0x7f61c813fff0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c80cdd60 0x7f61c813bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 --2- 172.21.15.133:0/2008746671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c806e4d0 0x7f61c8135eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 -- 172.21.15.133:0/2008746671 >> 172.21.15.133:0/2008746671 conn(0x7f61c8066f70 msgr2=0x7f61c8152e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 -- 172.21.15.133:0/2008746671 shutdown_connections 2024-04-25T10:16:03.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.577+0000 7f61cd110640 1 -- 172.21.15.133:0/2008746671 wait complete. 2024-04-25T10:16:03.594 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:16:03.594 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:03.642 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:03.684 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:16:03.684 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:16:03.685 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:03.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:03.689 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:16:03.689 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:03.736 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:03.736 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:16:03.736 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:03.741 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:03.741 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:16:03.741 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:03.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:03.783 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:16:03.784 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:03.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd8dbce640 1 Processor -- start 2024-04-25T10:16:03.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd8dbce640 1 -- start start 2024-04-25T10:16:03.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd8dbce640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdd88159dc0 0x7fdd880cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd8dbce640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdd8815e220 0x7fdd8806d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd8dbce640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd880ce880 con 0x7fdd8815a1a0 2024-04-25T10:16:03.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd8dbce640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd880c49e0 con 0x7fdd88159dc0 2024-04-25T10:16:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd8dbce640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd880ceb20 con 0x7fdd8815e220 2024-04-25T10:16:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd877fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdd88159dc0 0x7fdd880cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd877fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdd88159dc0 0x7fdd880cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39216/0 (socket says 172.21.15.133:39216) 2024-04-25T10:16:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd877fe640 1 -- 172.21.15.133:0/376431748 learned_addr learned my addr 172.21.15.133:0/376431748 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd877fe640 1 -- 172.21.15.133:0/376431748 >> v2:172.21.15.162:3300/0 conn(0x7fdd8815e220 msgr2=0x7fdd8806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd87fff640 1 --2- 172.21.15.133:0/376431748 >> v2:172.21.15.162:3300/0 conn(0x7fdd8815e220 0x7fdd8806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd877fe640 1 --2- 172.21.15.133:0/376431748 >> v2:172.21.15.162:3300/0 conn(0x7fdd8815e220 0x7fdd8806d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd877fe640 1 -- 172.21.15.133:0/376431748 >> v1:172.21.15.162:6789/0 conn(0x7fdd8815a1a0 legacy=0x7fdd8815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.909+0000 7fdd877fe640 1 -- 172.21.15.133:0/376431748 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd880cbfa0 con 0x7fdd88159dc0 2024-04-25T10:16:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd877fe640 1 --2- 172.21.15.133:0/376431748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd88159dc0 0x7fdd880cba60 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fdd880cee00 tx=0x7fdd7801e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa3de6fce356f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd86ffd640 1 -- 172.21.15.133:0/376431748 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd780ad070 con 0x7fdd88159dc0 2024-04-25T10:16:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd86ffd640 1 -- 172.21.15.133:0/376431748 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd78004230 con 0x7fdd88159dc0 2024-04-25T10:16:03.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd86ffd640 1 -- 172.21.15.133:0/376431748 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78003620 con 0x7fdd88159dc0 2024-04-25T10:16:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 -- 172.21.15.133:0/376431748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd88159dc0 msgr2=0x7fdd880cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 --2- 172.21.15.133:0/376431748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd88159dc0 0x7fdd880cba60 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fdd880cee00 tx=0x7fdd7801e990 comp rx=0 tx=0).stop 2024-04-25T10:16:03.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 -- 172.21.15.133:0/376431748 shutdown_connections 2024-04-25T10:16:03.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 --2- 172.21.15.133:0/376431748 >> v2:172.21.15.162:3300/0 conn(0x7fdd8815e220 0x7fdd8806d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 --2- 172.21.15.133:0/376431748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd88159dc0 0x7fdd880cba60 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 -- 172.21.15.133:0/376431748 >> 172.21.15.133:0/376431748 conn(0x7fdd88067000 msgr2=0x7fdd8806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 -- 172.21.15.133:0/376431748 shutdown_connections 2024-04-25T10:16:03.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 -- 172.21.15.133:0/376431748 wait complete. 2024-04-25T10:16:03.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 Processor -- start 2024-04-25T10:16:03.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 -- start start 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88159dc0 0x7fdd881588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd8815e220 0x7fdd88150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd88142650 0x7fdd88142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8806da10 con 0x7fdd8815e220 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8806d890 con 0x7fdd88159dc0 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8806db90 con 0x7fdd88142650 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd7effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd8815e220 0x7fdd88150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd7effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd8815e220 0x7fdd88150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39228/0 (socket says 172.21.15.133:39228) 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd7effd640 1 -- 172.21.15.133:0/3286121939 learned_addr learned my addr 172.21.15.133:0/3286121939 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd877fe640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88159dc0 0x7fdd881588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd87fff640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd88142650 0x7fdd88142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd7effd640 1 -- 172.21.15.133:0/3286121939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd88142650 msgr2=0x7fdd88142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd7effd640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd88142650 0x7fdd88142b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd7effd640 1 -- 172.21.15.133:0/3286121939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88159dc0 msgr2=0x7fdd881588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd7effd640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88159dc0 0x7fdd881588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd7effd640 1 -- 172.21.15.133:0/3286121939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd88148d10 con 0x7fdd8815e220 2024-04-25T10:16:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd87fff640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd88142650 0x7fdd88142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd7effd640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd8815e220 0x7fdd88150d90 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c4c0 tx=0x7fdd70001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd84ff9640 1 -- 172.21.15.133:0/3286121939 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70017070 con 0x7fdd8815e220 2024-04-25T10:16:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd84ff9640 1 -- 172.21.15.133:0/3286121939 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70003600 con 0x7fdd8815e220 2024-04-25T10:16:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 -- 172.21.15.133:0/3286121939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd88158e30 con 0x7fdd8815e220 2024-04-25T10:16:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd84ff9640 1 -- 172.21.15.133:0/3286121939 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd700039f0 con 0x7fdd8815e220 2024-04-25T10:16:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.913+0000 7fdd8dbce640 1 -- 172.21.15.133:0/3286121939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8815bcf0 con 0x7fdd8815e220 2024-04-25T10:16:03.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.917+0000 7fdd8dbce640 1 -- 172.21.15.133:0/3286121939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd88068ef0 con 0x7fdd8815e220 2024-04-25T10:16:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.917+0000 7fdd84ff9640 1 -- 172.21.15.133:0/3286121939 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd7001a070 con 0x7fdd8815e220 2024-04-25T10:16:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.917+0000 7fdd84ff9640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd68066f60 0x7fdd68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.917+0000 7fdd84ff9640 1 -- 172.21.15.133:0/3286121939 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fdd70084240 con 0x7fdd8815e220 2024-04-25T10:16:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.917+0000 7fdd877fe640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd68066f60 0x7fdd68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.917+0000 7fdd877fe640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd68066f60 0x7fdd68069420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b1000 tx=0x7fdd780a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:03.921+0000 7fdd84ff9640 1 -- 172.21.15.133:0/3286121939 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd700553b0 con 0x7fdd8815e220 2024-04-25T10:16:04.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 -- 172.21.15.133:0/3286121939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd880cccc0 con 0x7fdd8815e220 2024-04-25T10:16:04.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd84ff9640 1 -- 172.21.15.133:0/3286121939 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+3715 (secure 0 0 0) 0x7fdd70054cb0 con 0x7fdd8815e220 2024-04-25T10:16:04.102 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:04.102 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":101,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3454110484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3454110484},{"type":"v1","addr":"172.21.15.162:6835","nonce":3454110484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":6857,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/2777128039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2777128039},{"type":"v1","addr":"172.21.15.162:6833","nonce":2777128039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":10539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3345416592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3345416592},{"type":"v1","addr":"172.21.15.133:6835","nonce":3345416592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10545,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2924224565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2924224565},{"type":"v1","addr":"172.21.15.133:6837","nonce":2924224565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[]} 2024-04-25T10:16:04.102 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 101 2024-04-25T10:16:04.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 -- 172.21.15.133:0/3286121939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd68066f60 msgr2=0x7fdd68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd68066f60 0x7fdd68069420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b1000 tx=0x7fdd780a9070 comp rx=0 tx=0).stop 2024-04-25T10:16:04.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 -- 172.21.15.133:0/3286121939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd8815e220 msgr2=0x7fdd88150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd8815e220 0x7fdd88150d90 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c4c0 tx=0x7fdd70001e80 comp rx=0 tx=0).stop 2024-04-25T10:16:04.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 -- 172.21.15.133:0/3286121939 shutdown_connections 2024-04-25T10:16:04.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdd88142650 0x7fdd88142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdd68066f60 0x7fdd68069420 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd8815e220 0x7fdd88150d90 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 --2- 172.21.15.133:0/3286121939 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdd88159dc0 0x7fdd881588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 -- 172.21.15.133:0/3286121939 >> 172.21.15.133:0/3286121939 conn(0x7fdd88067000 msgr2=0x7fdd880b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 -- 172.21.15.133:0/3286121939 shutdown_connections 2024-04-25T10:16:04.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.101+0000 7fdd8dbce640 1 -- 172.21.15.133:0/3286121939 wait complete. 2024-04-25T10:16:04.120 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:16:04.120 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:16:04.120 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:16:04.120 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:04.120 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:16:04.121 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:16:04.121 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:16:04.121 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:04.121 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:16:04.121 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:16:04.121 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:16:04.121 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:04.121 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:16:04.122 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:16:04.122 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:16:04.122 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:04.130 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:16:04.131 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:16:04.136 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:16:04.136 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:16:04.144 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:16:04.144 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:16:04.145 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:16:04.145 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:16:04.181 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:16:04.182 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:04.183 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:16:04.183 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:04.192 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:16:04.192 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:04.193 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:16:04.193 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:04.193 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:16:04.238 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:16:04.252 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:16:04.260 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:16:04.270 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:16:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.297+0000 7f1432b3e640 1 Processor -- start 2024-04-25T10:16:04.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.297+0000 7f1432b3e640 1 -- start start 2024-04-25T10:16:04.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f1432b3e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f142c0d2030 0x7f142c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f1432b3e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142c0d24c0 con 0x7f142c15d9a0 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f1432b3e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142c15ebd0 con 0x7f142c0ce600 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f1432b3e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142c15eeb0 con 0x7f142c0d2030 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f14308da640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f142c15d9a0 0x7f142c0d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44228/0 (socket says 172.21.15.133:44228) 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f14308da640 1 -- 172.21.15.133:0/943557538 learned_addr learned my addr 172.21.15.133:0/943557538 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f14310db640 1 --2- 172.21.15.133:0/943557538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f142c0d2030 0x7f142c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f142b7fe640 1 -- 172.21.15.133:0/943557538 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497329107 0 0) 0x7f142c0d24c0 con 0x7f142c15d9a0 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f142b7fe640 1 -- 172.21.15.133:0/943557538 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1414003530 con 0x7f142c15d9a0 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f142b7fe640 1 -- 172.21.15.133:0/943557538 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611096657 0 0) 0x7f142c15ebd0 con 0x7f142c0ce600 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f142b7fe640 1 -- 172.21.15.133:0/943557538 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f142c0d24c0 con 0x7f142c0ce600 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f14310db640 1 -- 172.21.15.133:0/943557538 >> v1:172.21.15.162:6789/0 conn(0x7f142c0ce600 legacy=0x7f142c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f14310db640 1 -- 172.21.15.133:0/943557538 >> v1:172.21.15.133:6789/0 conn(0x7f142c15d9a0 legacy=0x7f142c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f14310db640 1 -- 172.21.15.133:0/943557538 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c0d0330 con 0x7f142c0d2030 2024-04-25T10:16:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f14310db640 1 --2- 172.21.15.133:0/943557538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f142c0d2030 0x7f142c0cea90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1424009dc0 tx=0x7f142401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d30346e9ac3f64ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f142b7fe640 1 -- 172.21.15.133:0/943557538 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14240b2070 con 0x7f142c0d2030 2024-04-25T10:16:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f142b7fe640 1 -- 172.21.15.133:0/943557538 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14240a58d0 con 0x7f142c0d2030 2024-04-25T10:16:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f142b7fe640 1 -- 172.21.15.133:0/943557538 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14240a5c80 con 0x7f142c0d2030 2024-04-25T10:16:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f1432b3e640 1 -- 172.21.15.133:0/943557538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f142c0d2030 msgr2=0x7f142c0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.301+0000 7f1432b3e640 1 --2- 172.21.15.133:0/943557538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f142c0d2030 0x7f142c0cea90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1424009dc0 tx=0x7f142401c900 comp rx=0 tx=0).stop 2024-04-25T10:16:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 -- 172.21.15.133:0/943557538 shutdown_connections 2024-04-25T10:16:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 --2- 172.21.15.133:0/943557538 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f142c0d2030 0x7f142c0cea90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 -- 172.21.15.133:0/943557538 >> 172.21.15.133:0/943557538 conn(0x7f142c06bf80 msgr2=0x7f142c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 -- 172.21.15.133:0/943557538 shutdown_connections 2024-04-25T10:16:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 -- 172.21.15.133:0/943557538 wait complete. 2024-04-25T10:16:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 Processor -- start 2024-04-25T10:16:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 -- start start 2024-04-25T10:16:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f142c0ce600 0x7f142c153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f142c0d2030 0x7f142c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c15d9a0 0x7f142c0cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c15e710 con 0x7f142c15d9a0 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c15e590 con 0x7f142c0d2030 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142c15e890 con 0x7f142c0ce600 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14310db640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c15d9a0 0x7f142c0cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14310db640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c15d9a0 0x7f142c0cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39254/0 (socket says 172.21.15.133:39254) 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14308da640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f142c0ce600 0x7f142c153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14310db640 1 -- 172.21.15.133:0/2144974673 learned_addr learned my addr 172.21.15.133:0/2144974673 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f142bfff640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f142c0d2030 0x7f142c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14310db640 1 -- 172.21.15.133:0/2144974673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f142c0ce600 msgr2=0x7f142c153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14310db640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f142c0ce600 0x7f142c153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14310db640 1 -- 172.21.15.133:0/2144974673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f142c0d2030 msgr2=0x7f142c0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14310db640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f142c0d2030 0x7f142c0cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14310db640 1 -- 172.21.15.133:0/2144974673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c153840 con 0x7f142c15d9a0 2024-04-25T10:16:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14310db640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c15d9a0 0x7f142c0cc840 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f14240b0040 tx=0x7f14240196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14297fa640 1 -- 172.21.15.133:0/2144974673 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14240b2050 con 0x7f142c15d9a0 2024-04-25T10:16:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14297fa640 1 -- 172.21.15.133:0/2144974673 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14240be070 con 0x7f142c15d9a0 2024-04-25T10:16:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 -- 172.21.15.133:0/2144974673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c0c80b0 con 0x7f142c15d9a0 2024-04-25T10:16:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f14297fa640 1 -- 172.21.15.133:0/2144974673 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14240ad850 con 0x7f142c15d9a0 2024-04-25T10:16:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 -- 172.21.15.133:0/2144974673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c0c8670 con 0x7f142c15d9a0 2024-04-25T10:16:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.305+0000 7f1432b3e640 1 -- 172.21.15.133:0/2144974673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c068e80 con 0x7f142c15d9a0 2024-04-25T10:16:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.309+0000 7f14297fa640 1 -- 172.21.15.133:0/2144974673 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14240ad9f0 con 0x7f142c15d9a0 2024-04-25T10:16:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.309+0000 7f14297fa640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1418066f60 0x7f1418069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.309+0000 7f14297fa640 1 -- 172.21.15.133:0/2144974673 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f142411fd80 con 0x7f142c15d9a0 2024-04-25T10:16:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.309+0000 7f14308da640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1418066f60 0x7f1418069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.309+0000 7f14308da640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1418066f60 0x7f1418069420 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1420007fd0 tx=0x7f1420007d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.313+0000 7f14297fa640 1 -- 172.21.15.133:0/2144974673 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f142c153840 con 0x7f142c15d9a0 2024-04-25T10:16:04.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:04.469+0000 7f1432b3e640 1 -- 172.21.15.133:0/2144974673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f142c0d3ed0 con 0x7f142c15d9a0 2024-04-25T10:16:05.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.605+0000 7f14297fa640 1 -- 172.21.15.133:0/2144974673 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v165) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1424016580 con 0x7f142c15d9a0 2024-04-25T10:16:05.606 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:16:05.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 -- 172.21.15.133:0/2144974673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1418066f60 msgr2=0x7f1418069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1418066f60 0x7f1418069420 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1420007fd0 tx=0x7f1420007d80 comp rx=0 tx=0).stop 2024-04-25T10:16:05.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 -- 172.21.15.133:0/2144974673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c15d9a0 msgr2=0x7f142c0cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c15d9a0 0x7f142c0cc840 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f14240b0040 tx=0x7f14240196a0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 -- 172.21.15.133:0/2144974673 shutdown_connections 2024-04-25T10:16:05.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1418066f60 0x7f1418069420 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f142c15d9a0 0x7f142c0cc840 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f142c0d2030 0x7f142c0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 --2- 172.21.15.133:0/2144974673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f142c0ce600 0x7f142c153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 -- 172.21.15.133:0/2144974673 >> 172.21.15.133:0/2144974673 conn(0x7f142c06bf80 msgr2=0x7f142c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:05.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 -- 172.21.15.133:0/2144974673 shutdown_connections 2024-04-25T10:16:05.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.609+0000 7f1432b3e640 1 -- 172.21.15.133:0/2144974673 wait complete. 2024-04-25T10:16:05.632 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:16:05.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f979b00c640 1 Processor -- start 2024-04-25T10:16:05.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f979b00c640 1 -- start start 2024-04-25T10:16:05.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f979b00c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f979415dd90 0x7f97940ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f979b00c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f97940b19f0 0x7f97940cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f979b00c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97940ca9b0 con 0x7f979415d9b0 2024-04-25T10:16:05.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f979b00c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97940cbe30 con 0x7f97940b19f0 2024-04-25T10:16:05.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f979b00c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97940cac90 con 0x7f979415dd90 2024-04-25T10:16:05.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f97995a9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f97940b19f0 0x7f97940cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f97995a9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f97940b19f0 0x7f97940cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39266/0 (socket says 172.21.15.133:39266) 2024-04-25T10:16:05.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f97995a9640 1 -- 172.21.15.133:0/4016034605 learned_addr learned my addr 172.21.15.133:0/4016034605 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:05.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f978bfff640 1 --2- 172.21.15.133:0/4016034605 >> v2:172.21.15.162:3300/0 conn(0x7f979415dd90 0x7f97940ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f97995a9640 1 -- 172.21.15.133:0/4016034605 >> v2:172.21.15.162:3300/0 conn(0x7f979415dd90 msgr2=0x7f97940ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:05.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f97995a9640 1 --2- 172.21.15.133:0/4016034605 >> v2:172.21.15.162:3300/0 conn(0x7f979415dd90 0x7f97940ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f97995a9640 1 -- 172.21.15.133:0/4016034605 >> v1:172.21.15.162:6789/0 conn(0x7f979415d9b0 legacy=0x7f97940cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:05.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f97995a9640 1 -- 172.21.15.133:0/4016034605 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940caf70 con 0x7f97940b19f0 2024-04-25T10:16:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.801+0000 7f97995a9640 1 --2- 172.21.15.133:0/4016034605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97940b19f0 0x7f97940cecb0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f97940cc000 tx=0x7f978c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132a86864d220bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f978b7fe640 1 -- 172.21.15.133:0/4016034605 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c0a5800 con 0x7f97940b19f0 2024-04-25T10:16:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f978b7fe640 1 -- 172.21.15.133:0/4016034605 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978c0a59a0 con 0x7f97940b19f0 2024-04-25T10:16:05.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f978b7fe640 1 -- 172.21.15.133:0/4016034605 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978c0a5d70 con 0x7f97940b19f0 2024-04-25T10:16:05.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 -- 172.21.15.133:0/4016034605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97940b19f0 msgr2=0x7f97940cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 --2- 172.21.15.133:0/4016034605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97940b19f0 0x7f97940cecb0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f97940cc000 tx=0x7f978c01c920 comp rx=0 tx=0).stop 2024-04-25T10:16:05.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 -- 172.21.15.133:0/4016034605 shutdown_connections 2024-04-25T10:16:05.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 --2- 172.21.15.133:0/4016034605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97940b19f0 0x7f97940cecb0 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 --2- 172.21.15.133:0/4016034605 >> v2:172.21.15.162:3300/0 conn(0x7f979415dd90 0x7f97940ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 -- 172.21.15.133:0/4016034605 >> 172.21.15.133:0/4016034605 conn(0x7f9794067000 msgr2=0x7f979406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:05.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 -- 172.21.15.133:0/4016034605 shutdown_connections 2024-04-25T10:16:05.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 -- 172.21.15.133:0/4016034605 wait complete. 2024-04-25T10:16:05.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 Processor -- start 2024-04-25T10:16:05.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 -- start start 2024-04-25T10:16:05.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97940b19f0 0x7f9794144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f979415d9b0 0x7f979414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f979415dd90 0x7f979414b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979406e290 con 0x7f97940b19f0 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979406e110 con 0x7f979415dd90 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979406e410 con 0x7f979415d9b0 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f9798da8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97940b19f0 0x7f9794144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f9798da8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97940b19f0 0x7f9794144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39276/0 (socket says 172.21.15.133:39276) 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f9798da8640 1 -- 172.21.15.133:0/790683611 learned_addr learned my addr 172.21.15.133:0/790683611 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f97995a9640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f979415dd90 0x7f979414b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f978bfff640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f979415d9b0 0x7f979414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f9798da8640 1 -- 172.21.15.133:0/790683611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f979415d9b0 msgr2=0x7f979414aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f9798da8640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f979415d9b0 0x7f979414aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f9798da8640 1 -- 172.21.15.133:0/790683611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f979415dd90 msgr2=0x7f979414b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:05.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f9798da8640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f979415dd90 0x7f979414b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:05.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f9798da8640 1 -- 172.21.15.133:0/790683611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9794147c60 con 0x7f97940b19f0 2024-04-25T10:16:05.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f978bfff640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f979415d9b0 0x7f979414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:05.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f9798da8640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97940b19f0 0x7f9794144320 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f979000c450 tx=0x7f979000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f97897fa640 1 -- 172.21.15.133:0/790683611 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790012830 con 0x7f97940b19f0 2024-04-25T10:16:05.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f97897fa640 1 -- 172.21.15.133:0/790683611 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9790017070 con 0x7f97940b19f0 2024-04-25T10:16:05.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f979b00c640 1 -- 172.21.15.133:0/790683611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9794147f40 con 0x7f97940b19f0 2024-04-25T10:16:05.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.805+0000 7f97897fa640 1 -- 172.21.15.133:0/790683611 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790012c30 con 0x7f97940b19f0 2024-04-25T10:16:05.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.809+0000 7f979b00c640 1 -- 172.21.15.133:0/790683611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9794148500 con 0x7f97940b19f0 2024-04-25T10:16:05.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.809+0000 7f979b00c640 1 -- 172.21.15.133:0/790683611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9794148840 con 0x7f97940b19f0 2024-04-25T10:16:05.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.809+0000 7f97897fa640 1 -- 172.21.15.133:0/790683611 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f979001c460 con 0x7f97940b19f0 2024-04-25T10:16:05.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.809+0000 7f97897fa640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9780066f60 0x7f9780069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:05.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.809+0000 7f97897fa640 1 -- 172.21.15.133:0/790683611 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9790084cb0 con 0x7f97940b19f0 2024-04-25T10:16:05.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.809+0000 7f978bfff640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9780066f60 0x7f9780069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:05.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.809+0000 7f978bfff640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9780066f60 0x7f9780069420 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f979414bd70 tx=0x7f9784008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:05.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:05.813+0000 7f97897fa640 1 -- 172.21.15.133:0/790683611 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9794148840 con 0x7f97940b19f0 2024-04-25T10:16:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 -- 172.21.15.133:0/790683611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f97940cc420 con 0x7f97940b19f0 2024-04-25T10:16:06.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f97897fa640 1 -- 172.21.15.133:0/790683611 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9790055e80 con 0x7f97940b19f0 2024-04-25T10:16:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:06.022 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:16:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:16:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:16:06.022 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:16:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:16:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:16:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:16:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:16:06.024 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:16:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.026 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.027 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:16:06.028 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.029 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:16:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:16:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:16:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:16:06.030 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:16:06.031 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.032 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:16:06.033 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.034 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:16:06.035 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:16:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:16:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:16:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.036 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:16:06.036 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:16:06.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 -- 172.21.15.133:0/790683611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9780066f60 msgr2=0x7f9780069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9780066f60 0x7f9780069420 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f979414bd70 tx=0x7f9784008040 comp rx=0 tx=0).stop 2024-04-25T10:16:06.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 -- 172.21.15.133:0/790683611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97940b19f0 msgr2=0x7f9794144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97940b19f0 0x7f9794144320 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f979000c450 tx=0x7f979000c910 comp rx=0 tx=0).stop 2024-04-25T10:16:06.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 -- 172.21.15.133:0/790683611 shutdown_connections 2024-04-25T10:16:06.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f979415dd90 0x7f979414b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f979415d9b0 0x7f979414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9780066f60 0x7f9780069420 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 --2- 172.21.15.133:0/790683611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97940b19f0 0x7f9794144320 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 -- 172.21.15.133:0/790683611 >> 172.21.15.133:0/790683611 conn(0x7f9794067000 msgr2=0x7f97940711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 -- 172.21.15.133:0/790683611 shutdown_connections 2024-04-25T10:16:06.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.021+0000 7f979b00c640 1 -- 172.21.15.133:0/790683611 wait complete. 2024-04-25T10:16:06.039 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.145+0000 7fd14efda640 1 Processor -- start 2024-04-25T10:16:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.145+0000 7fd14efda640 1 -- start start 2024-04-25T10:16:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd14efda640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd14815db00 0x7fd1480b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd14efda640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1480d0770 0x7fd1480d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd14efda640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1481614c0 0x7fd148161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd14efda640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1480d1110 con 0x7fd1480d0770 2024-04-25T10:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd14efda640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1480ceb20 con 0x7fd14815db00 2024-04-25T10:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd14efda640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480b7740 con 0x7fd1481614c0 2024-04-25T10:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd147fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1480d0770 0x7fd1480d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd14cd76640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd14815db00 0x7fd1480b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd147fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1480d0770 0x7fd1480d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39292/0 (socket says 172.21.15.133:39292) 2024-04-25T10:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd147fff640 1 -- 172.21.15.133:0/2674743301 learned_addr learned my addr 172.21.15.133:0/2674743301 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd14d577640 1 --2- 172.21.15.133:0/2674743301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1481614c0 0x7fd148161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd147fff640 1 -- 172.21.15.133:0/2674743301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1481614c0 msgr2=0x7fd148161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:06.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd147fff640 1 --2- 172.21.15.133:0/2674743301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1481614c0 0x7fd148161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd147fff640 1 -- 172.21.15.133:0/2674743301 >> v2:172.21.15.162:3300/0 conn(0x7fd14815db00 msgr2=0x7fd1480b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:06.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd147fff640 1 --2- 172.21.15.133:0/2674743301 >> v2:172.21.15.162:3300/0 conn(0x7fd14815db00 0x7fd1480b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd147fff640 1 -- 172.21.15.133:0/2674743301 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480b7a20 con 0x7fd1480d0770 2024-04-25T10:16:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.149+0000 7fd147fff640 1 --2- 172.21.15.133:0/2674743301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1480d0770 0x7fd1480d0bd0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fd1480b53d0 tx=0x7fd13801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decbeb5fc0679ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd1477fe640 1 -- 172.21.15.133:0/2674743301 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1380aa610 con 0x7fd1480d0770 2024-04-25T10:16:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd1477fe640 1 -- 172.21.15.133:0/2674743301 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1380af070 con 0x7fd1480d0770 2024-04-25T10:16:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd1477fe640 1 -- 172.21.15.133:0/2674743301 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1380aa9d0 con 0x7fd1480d0770 2024-04-25T10:16:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 -- 172.21.15.133:0/2674743301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1480d0770 msgr2=0x7fd1480d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 --2- 172.21.15.133:0/2674743301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1480d0770 0x7fd1480d0bd0 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fd1480b53d0 tx=0x7fd13801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 -- 172.21.15.133:0/2674743301 shutdown_connections 2024-04-25T10:16:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 --2- 172.21.15.133:0/2674743301 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1481614c0 0x7fd148161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 --2- 172.21.15.133:0/2674743301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1480d0770 0x7fd1480d0bd0 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 --2- 172.21.15.133:0/2674743301 >> v2:172.21.15.162:3300/0 conn(0x7fd14815db00 0x7fd1480b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 -- 172.21.15.133:0/2674743301 >> 172.21.15.133:0/2674743301 conn(0x7fd1480727c0 msgr2=0x7fd148072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 -- 172.21.15.133:0/2674743301 shutdown_connections 2024-04-25T10:16:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 -- 172.21.15.133:0/2674743301 wait complete. 2024-04-25T10:16:06.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 Processor -- start 2024-04-25T10:16:06.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 -- start start 2024-04-25T10:16:06.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1480d0770 0x7fd1480d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd14815db00 0x7fd14814d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1481614c0 0x7fd148157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480b7430 con 0x7fd1480d0770 2024-04-25T10:16:06.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480b82c0 con 0x7fd14815db00 2024-04-25T10:16:06.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480d18a0 con 0x7fd1481614c0 2024-04-25T10:16:06.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14cd76640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1480d0770 0x7fd1480d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14cd76640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1480d0770 0x7fd1480d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39304/0 (socket says 172.21.15.133:39304) 2024-04-25T10:16:06.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14cd76640 1 -- 172.21.15.133:0/3021578014 learned_addr learned my addr 172.21.15.133:0/3021578014 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:06.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14d577640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1481614c0 0x7fd148157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd147fff640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd14815db00 0x7fd14814d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14cd76640 1 -- 172.21.15.133:0/3021578014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1481614c0 msgr2=0x7fd148157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14cd76640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1481614c0 0x7fd148157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14cd76640 1 -- 172.21.15.133:0/3021578014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd14815db00 msgr2=0x7fd14814d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14cd76640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd14815db00 0x7fd14814d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14cd76640 1 -- 172.21.15.133:0/3021578014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480cc090 con 0x7fd1480d0770 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14d577640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1481614c0 0x7fd148157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14cd76640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1480d0770 0x7fd1480d1b50 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00e8a0 tx=0x7fd13c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd1457fa640 1 -- 172.21.15.133:0/3021578014 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c017070 con 0x7fd1480d0770 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd1457fa640 1 -- 172.21.15.133:0/3021578014 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c00ee90 con 0x7fd1480d0770 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 -- 172.21.15.133:0/3021578014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14814ed00 con 0x7fd1480d0770 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd1457fa640 1 -- 172.21.15.133:0/3021578014 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00cb80 con 0x7fd1480d0770 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.153+0000 7fd14efda640 1 -- 172.21.15.133:0/3021578014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1480cc310 con 0x7fd1480d0770 2024-04-25T10:16:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.157+0000 7fd14efda640 1 -- 172.21.15.133:0/3021578014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14814f220 con 0x7fd1480d0770 2024-04-25T10:16:06.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.157+0000 7fd1457fa640 1 -- 172.21.15.133:0/3021578014 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd13c01a070 con 0x7fd1480d0770 2024-04-25T10:16:06.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.157+0000 7fd1457fa640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd134066f60 0x7fd134069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.157+0000 7fd1457fa640 1 -- 172.21.15.133:0/3021578014 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd13c084d90 con 0x7fd1480d0770 2024-04-25T10:16:06.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.157+0000 7fd147fff640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd134066f60 0x7fd134069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.157+0000 7fd147fff640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd134066f60 0x7fd134069420 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd1380ae000 tx=0x7fd1380a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.161+0000 7fd1457fa640 1 -- 172.21.15.133:0/3021578014 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd14814f220 con 0x7fd1480d0770 2024-04-25T10:16:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.321+0000 7fd14efda640 1 -- 172.21.15.133:0/3021578014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd148068ef0 con 0x7fd1480d0770 2024-04-25T10:16:06.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.321+0000 7fd1457fa640 1 -- 172.21.15.133:0/3021578014 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+13198 (secure 0 0 0) 0x7fd1480cc090 con 0x7fd1480d0770 2024-04-25T10:16:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:06.322 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":165,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:16:05.605413+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.321+0000 7fd14efda640 1 -- 172.21.15.133:0/3021578014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd134066f60 msgr2=0x7fd134069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.321+0000 7fd14efda640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd134066f60 0x7fd134069420 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd1380ae000 tx=0x7fd1380a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.321+0000 7fd14efda640 1 -- 172.21.15.133:0/3021578014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1480d0770 msgr2=0x7fd1480d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.321+0000 7fd14efda640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1480d0770 0x7fd1480d1b50 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00e8a0 tx=0x7fd13c00ed60 comp rx=0 tx=0).stop 2024-04-25T10:16:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.321+0000 7fd14efda640 1 -- 172.21.15.133:0/3021578014 shutdown_connections 2024-04-25T10:16:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.321+0000 7fd14efda640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd1481614c0 0x7fd148157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.325+0000 7fd14efda640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd14815db00 0x7fd14814d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.325+0000 7fd14efda640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd134066f60 0x7fd134069420 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.325+0000 7fd14efda640 1 --2- 172.21.15.133:0/3021578014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1480d0770 0x7fd1480d1b50 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.325+0000 7fd14efda640 1 -- 172.21.15.133:0/3021578014 >> 172.21.15.133:0/3021578014 conn(0x7fd1480727c0 msgr2=0x7fd14815cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.325+0000 7fd14efda640 1 -- 172.21.15.133:0/3021578014 shutdown_connections 2024-04-25T10:16:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.325+0000 7fd14efda640 1 -- 172.21.15.133:0/3021578014 wait complete. 2024-04-25T10:16:06.340 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:06.341 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:06.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae1bdd6640 1 Processor -- start 2024-04-25T10:16:06.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae1bdd6640 1 -- start start 2024-04-25T10:16:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae1bdd6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fae140ad630 0x7fae1406b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae1bdd6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1406bc80 con 0x7fae1415b910 2024-04-25T10:16:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae1bdd6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1406bf60 con 0x7fae140af070 2024-04-25T10:16:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae1bdd6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1406c240 con 0x7fae140ad630 2024-04-25T10:16:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae19b72640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fae1415b910 0x7fae140acf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40892/0 (socket says 172.21.15.133:40892) 2024-04-25T10:16:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae19b72640 1 -- 172.21.15.133:0/2386148520 learned_addr learned my addr 172.21.15.133:0/2386148520 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae19371640 1 --2- 172.21.15.133:0/2386148520 >> v2:172.21.15.133:3300/0 conn(0x7fae140ad630 0x7fae1406b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae18b70640 1 -- 172.21.15.133:0/2386148520 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879962544 0 0) 0x7fae1406bc80 con 0x7fae1415b910 2024-04-25T10:16:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae18b70640 1 -- 172.21.15.133:0/2386148520 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae00003530 con 0x7fae1415b910 2024-04-25T10:16:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae19371640 1 -- 172.21.15.133:0/2386148520 >> v1:172.21.15.162:6789/0 conn(0x7fae140af070 legacy=0x7fae1415c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae19371640 1 -- 172.21.15.133:0/2386148520 >> v1:172.21.15.133:6789/0 conn(0x7fae1415b910 legacy=0x7fae140acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.465+0000 7fae19371640 1 -- 172.21.15.133:0/2386148520 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae140b4fc0 con 0x7fae140ad630 2024-04-25T10:16:06.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae19371640 1 --2- 172.21.15.133:0/2386148520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae140ad630 0x7fae1406b6a0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7fae040097c0 tx=0x7fae0401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3510f5250486a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae18b70640 1 -- 172.21.15.133:0/2386148520 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040b0070 con 0x7fae140ad630 2024-04-25T10:16:06.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae18b70640 1 -- 172.21.15.133:0/2386148520 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae040ab600 con 0x7fae140ad630 2024-04-25T10:16:06.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae18b70640 1 -- 172.21.15.133:0/2386148520 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040ab9d0 con 0x7fae140ad630 2024-04-25T10:16:06.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2386148520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae140ad630 msgr2=0x7fae1406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 --2- 172.21.15.133:0/2386148520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae140ad630 0x7fae1406b6a0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7fae040097c0 tx=0x7fae0401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2386148520 shutdown_connections 2024-04-25T10:16:06.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 --2- 172.21.15.133:0/2386148520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae140ad630 0x7fae1406b6a0 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2386148520 >> 172.21.15.133:0/2386148520 conn(0x7fae140cfd50 msgr2=0x7fae140d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2386148520 shutdown_connections 2024-04-25T10:16:06.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2386148520 wait complete. 2024-04-25T10:16:06.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 Processor -- start 2024-04-25T10:16:06.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 -- start start 2024-04-25T10:16:06.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae140ad630 0x7fae14150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae140af070 0x7fae14156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1415b910 0x7fae1415d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1415d5e0 con 0x7fae1415b910 2024-04-25T10:16:06.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1415d460 con 0x7fae140af070 2024-04-25T10:16:06.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae140af480 con 0x7fae140ad630 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1a373640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1415b910 0x7fae1415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1a373640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1415b910 0x7fae1415d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55544/0 (socket says 172.21.15.133:55544) 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1a373640 1 -- 172.21.15.133:0/2084271255 learned_addr learned my addr 172.21.15.133:0/2084271255 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae19371640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae140af070 0x7fae14156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1a373640 1 -- 172.21.15.133:0/2084271255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae140ad630 msgr2=0x7fae14150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1a373640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae140ad630 0x7fae14150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1a373640 1 -- 172.21.15.133:0/2084271255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae140af070 msgr2=0x7fae14156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1a373640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae140af070 0x7fae14156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1a373640 1 -- 172.21.15.133:0/2084271255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1413c880 con 0x7fae1415b910 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1a373640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1415b910 0x7fae1415d760 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c740 tx=0x7fae0c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae0a7fc640 1 -- 172.21.15.133:0/2084271255 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c016020 con 0x7fae1415b910 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae0a7fc640 1 -- 172.21.15.133:0/2084271255 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c00ce50 con 0x7fae1415b910 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2084271255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1413cb60 con 0x7fae1415b910 2024-04-25T10:16:06.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae0a7fc640 1 -- 172.21.15.133:0/2084271255 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c003980 con 0x7fae1415b910 2024-04-25T10:16:06.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.469+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2084271255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1413d120 con 0x7fae1415b910 2024-04-25T10:16:06.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.473+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2084271255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae14067c70 con 0x7fae1415b910 2024-04-25T10:16:06.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.473+0000 7fae0a7fc640 1 -- 172.21.15.133:0/2084271255 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae0c003b20 con 0x7fae1415b910 2024-04-25T10:16:06.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.473+0000 7fae0a7fc640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf8066f60 0x7fadf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.473+0000 7fae0a7fc640 1 -- 172.21.15.133:0/2084271255 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fae0c084a90 con 0x7fae1415b910 2024-04-25T10:16:06.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.473+0000 7fae19b72640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf8066f60 0x7fadf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.473+0000 7fae19b72640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf8066f60 0x7fadf8069420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fae10007fd0 tx=0x7fae10007940 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.477+0000 7fae0a7fc640 1 -- 172.21.15.133:0/2084271255 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae0c020020 con 0x7fae1415b910 2024-04-25T10:16:06.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.625+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2084271255 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae14153d90 con 0x7fae1415b910 2024-04-25T10:16:06.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.625+0000 7fae0a7fc640 1 -- 172.21.15.133:0/2084271255 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae0c019310 con 0x7fae1415b910 2024-04-25T10:16:06.627 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:16:06.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2084271255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf8066f60 msgr2=0x7fadf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf8066f60 0x7fadf8069420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fae10007fd0 tx=0x7fae10007940 comp rx=0 tx=0).stop 2024-04-25T10:16:06.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2084271255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1415b910 msgr2=0x7fae1415d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1415b910 0x7fae1415d760 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c740 tx=0x7fae0c00cc00 comp rx=0 tx=0).stop 2024-04-25T10:16:06.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2084271255 shutdown_connections 2024-04-25T10:16:06.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fadf8066f60 0x7fadf8069420 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae1415b910 0x7fae1415d760 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae140af070 0x7fae14156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 --2- 172.21.15.133:0/2084271255 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fae140ad630 0x7fae14150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2084271255 >> 172.21.15.133:0/2084271255 conn(0x7fae140cfd50 msgr2=0x7fae14099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2084271255 shutdown_connections 2024-04-25T10:16:06.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.629+0000 7fae1bdd6640 1 -- 172.21.15.133:0/2084271255 wait complete. 2024-04-25T10:16:06.644 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:16:06.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61af8c4640 1 Processor -- start 2024-04-25T10:16:06.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61af8c4640 1 -- start start 2024-04-25T10:16:06.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61af8c4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61a815db00 0x7f61a80b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61af8c4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f61a80d0bd0 0x7f61a8161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61af8c4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a80ceb20 con 0x7f61a80d0770 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61af8c4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a80b7740 con 0x7f61a815db00 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61af8c4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a80b7a20 con 0x7f61a80d0bd0 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61ace5f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f61a80d0770 0x7f61a81614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40908/0 (socket says 172.21.15.133:40908) 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61ace5f640 1 -- 172.21.15.133:0/621189368 learned_addr learned my addr 172.21.15.133:0/621189368 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61ad660640 1 --2- 172.21.15.133:0/621189368 >> v2:172.21.15.133:3300/0 conn(0x7f61a815db00 0x7f61a80b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61ade61640 1 --2- 172.21.15.133:0/621189368 >> v2:172.21.15.162:3300/0 conn(0x7f61a80d0bd0 0x7f61a8161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f619ffff640 1 -- 172.21.15.133:0/621189368 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928079304 0 0) 0x7f61a80ceb20 con 0x7f61a80d0770 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f619ffff640 1 -- 172.21.15.133:0/621189368 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6194003530 con 0x7f61a80d0770 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61ad660640 1 -- 172.21.15.133:0/621189368 >> v2:172.21.15.162:3300/0 conn(0x7f61a80d0bd0 msgr2=0x7f61a8161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61ad660640 1 --2- 172.21.15.133:0/621189368 >> v2:172.21.15.162:3300/0 conn(0x7f61a80d0bd0 0x7f61a8161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61ad660640 1 -- 172.21.15.133:0/621189368 >> v1:172.21.15.133:6789/0 conn(0x7f61a80d0770 legacy=0x7f61a81614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.761+0000 7f61ad660640 1 -- 172.21.15.133:0/621189368 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a80b97f0 con 0x7f61a815db00 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61ad660640 1 --2- 172.21.15.133:0/621189368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a815db00 0x7f61a80b2990 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f61a80b5a00 tx=0x7f61a401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caf0227d8b27d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f619ffff640 1 -- 172.21.15.133:0/621189368 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a40ab650 con 0x7f61a815db00 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f619ffff640 1 -- 172.21.15.133:0/621189368 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a40b0070 con 0x7f61a815db00 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f619ffff640 1 -- 172.21.15.133:0/621189368 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a40aba10 con 0x7f61a815db00 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 -- 172.21.15.133:0/621189368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a815db00 msgr2=0x7f61a80b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 --2- 172.21.15.133:0/621189368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a815db00 0x7f61a80b2990 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f61a80b5a00 tx=0x7f61a401c450 comp rx=0 tx=0).stop 2024-04-25T10:16:06.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 -- 172.21.15.133:0/621189368 shutdown_connections 2024-04-25T10:16:06.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 --2- 172.21.15.133:0/621189368 >> v2:172.21.15.162:3300/0 conn(0x7f61a80d0bd0 0x7f61a8161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 --2- 172.21.15.133:0/621189368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a815db00 0x7f61a80b2990 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 -- 172.21.15.133:0/621189368 >> 172.21.15.133:0/621189368 conn(0x7f61a80727c0 msgr2=0x7f61a8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 -- 172.21.15.133:0/621189368 shutdown_connections 2024-04-25T10:16:06.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 -- 172.21.15.133:0/621189368 wait complete. 2024-04-25T10:16:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 Processor -- start 2024-04-25T10:16:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 -- start start 2024-04-25T10:16:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a80d0770 0x7f61a814adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61a80d0bd0 0x7f61a8160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61a815db00 0x7f61a815f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a80d0fe0 con 0x7f61a80d0770 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a80b7430 con 0x7f61a80d0bd0 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a80d18a0 con 0x7f61a815db00 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61ad660640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a80d0770 0x7f61a814adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61ad660640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a80d0770 0x7f61a814adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55564/0 (socket says 172.21.15.133:55564) 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61ad660640 1 -- 172.21.15.133:0/3809082922 learned_addr learned my addr 172.21.15.133:0/3809082922 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61ad660640 1 -- 172.21.15.133:0/3809082922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61a815db00 msgr2=0x7f61a815f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61ade61640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61a815db00 0x7f61a815f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61ad660640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61a815db00 0x7f61a815f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61ad660640 1 -- 172.21.15.133:0/3809082922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61a80d0bd0 msgr2=0x7f61a8160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61ad660640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61a80d0bd0 0x7f61a8160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61ad660640 1 -- 172.21.15.133:0/3809082922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a814f290 con 0x7f61a80d0770 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61ad660640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a80d0770 0x7f61a814adb0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f61a4019060 tx=0x7f61a401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f619dffb640 1 -- 172.21.15.133:0/3809082922 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a40bf610 con 0x7f61a80d0770 2024-04-25T10:16:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f619dffb640 1 -- 172.21.15.133:0/3809082922 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a40bf7d0 con 0x7f61a80d0770 2024-04-25T10:16:06.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f619dffb640 1 -- 172.21.15.133:0/3809082922 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a40bfbc0 con 0x7f61a80d0770 2024-04-25T10:16:06.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.765+0000 7f61af8c4640 1 -- 172.21.15.133:0/3809082922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a814f570 con 0x7f61a80d0770 2024-04-25T10:16:06.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.769+0000 7f61af8c4640 1 -- 172.21.15.133:0/3809082922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a814ed80 con 0x7f61a80d0770 2024-04-25T10:16:06.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.769+0000 7f61af8c4640 1 -- 172.21.15.133:0/3809082922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a8068ef0 con 0x7f61a80d0770 2024-04-25T10:16:06.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.769+0000 7f619dffb640 1 -- 172.21.15.133:0/3809082922 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61a40bd020 con 0x7f61a80d0770 2024-04-25T10:16:06.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.769+0000 7f619dffb640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f618c066f90 0x7f618c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:06.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.769+0000 7f61ace5f640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f618c066f90 0x7f618c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:06.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.769+0000 7f619dffb640 1 -- 172.21.15.133:0/3809082922 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f61a411f7e0 con 0x7f61a80d0770 2024-04-25T10:16:06.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.769+0000 7f61ace5f640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f618c066f90 0x7f618c069450 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6198007fd0 tx=0x7f6198007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:06.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.773+0000 7f619dffb640 1 -- 172.21.15.133:0/3809082922 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61a4016070 con 0x7f61a80d0770 2024-04-25T10:16:06.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 -- 172.21.15.133:0/3809082922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f61a80b2750 con 0x7f61a80d0770 2024-04-25T10:16:06.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f619dffb640 1 -- 172.21.15.133:0/3809082922 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v165) v1 ==== 84+0+65 (secure 0 0 0) 0x7f61a40f0ad0 con 0x7f61a80d0770 2024-04-25T10:16:06.925 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:06.925 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:16:06.925 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:06.925 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:16:06.926 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:16:06.926 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:06.926 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 -- 172.21.15.133:0/3809082922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f618c066f90 msgr2=0x7f618c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f618c066f90 0x7f618c069450 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6198007fd0 tx=0x7f6198007900 comp rx=0 tx=0).stop 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 -- 172.21.15.133:0/3809082922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a80d0770 msgr2=0x7f61a814adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a80d0770 0x7f61a814adb0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f61a4019060 tx=0x7f61a401ac90 comp rx=0 tx=0).stop 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 -- 172.21.15.133:0/3809082922 shutdown_connections 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61a815db00 0x7f61a815f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61a80d0bd0 0x7f61a8160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f618c066f90 0x7f618c069450 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 --2- 172.21.15.133:0/3809082922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a80d0770 0x7f61a814adb0 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 -- 172.21.15.133:0/3809082922 >> 172.21.15.133:0/3809082922 conn(0x7f61a80727c0 msgr2=0x7f61a80b87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 -- 172.21.15.133:0/3809082922 shutdown_connections 2024-04-25T10:16:06.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:06.925+0000 7f61af8c4640 1 -- 172.21.15.133:0/3809082922 wait complete. 2024-04-25T10:16:06.943 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:16:06.944 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:16:07.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd70075a640 1 Processor -- start 2024-04-25T10:16:07.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd70075a640 1 -- start start 2024-04-25T10:16:07.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd70075a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd6f80ca390 0x7fd6f806bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd70075a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd6f815bae0 0x7fd6f80ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd70075a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f806c5e0 con 0x7fd6f806ac90 2024-04-25T10:16:07.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd70075a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f8066ed0 con 0x7fd6f80ca390 2024-04-25T10:16:07.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd70075a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f806c0c0 con 0x7fd6f815bae0 2024-04-25T10:16:07.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd6fe4f6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd6f80ca390 0x7fd6f806bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd6fe4f6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd6f80ca390 0x7fd6f806bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55578/0 (socket says 172.21.15.133:55578) 2024-04-25T10:16:07.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd6fe4f6640 1 -- 172.21.15.133:0/965668149 learned_addr learned my addr 172.21.15.133:0/965668149 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:07.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd6fe4f6640 1 -- 172.21.15.133:0/965668149 >> v2:172.21.15.162:3300/0 conn(0x7fd6f815bae0 msgr2=0x7fd6f80ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd6fe4f6640 1 --2- 172.21.15.133:0/965668149 >> v2:172.21.15.162:3300/0 conn(0x7fd6f815bae0 0x7fd6f80ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd6fe4f6640 1 -- 172.21.15.133:0/965668149 >> v1:172.21.15.162:6789/0 conn(0x7fd6f806ac90 legacy=0x7fd6f815f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.057+0000 7fd6fe4f6640 1 -- 172.21.15.133:0/965668149 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f80cdd70 con 0x7fd6f80ca390 2024-04-25T10:16:07.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd6fe4f6640 1 --2- 172.21.15.133:0/965668149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f80ca390 0x7fd6f806bb80 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8067720 tx=0x7fd6f401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afede3f6166d88c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd6fd4f4640 1 -- 172.21.15.133:0/965668149 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40b2070 con 0x7fd6f80ca390 2024-04-25T10:16:07.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd6fd4f4640 1 -- 172.21.15.133:0/965668149 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f40a9070 con 0x7fd6f80ca390 2024-04-25T10:16:07.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd6fd4f4640 1 -- 172.21.15.133:0/965668149 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40ad880 con 0x7fd6f80ca390 2024-04-25T10:16:07.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd70075a640 1 -- 172.21.15.133:0/965668149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f80ca390 msgr2=0x7fd6f806bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd70075a640 1 --2- 172.21.15.133:0/965668149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f80ca390 0x7fd6f806bb80 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8067720 tx=0x7fd6f401c450 comp rx=0 tx=0).stop 2024-04-25T10:16:07.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd70075a640 1 -- 172.21.15.133:0/965668149 shutdown_connections 2024-04-25T10:16:07.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd70075a640 1 --2- 172.21.15.133:0/965668149 >> v2:172.21.15.162:3300/0 conn(0x7fd6f815bae0 0x7fd6f80ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd70075a640 1 --2- 172.21.15.133:0/965668149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f80ca390 0x7fd6f806bb80 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd70075a640 1 -- 172.21.15.133:0/965668149 >> 172.21.15.133:0/965668149 conn(0x7fd6f80cc7c0 msgr2=0x7fd6f80ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd70075a640 1 -- 172.21.15.133:0/965668149 shutdown_connections 2024-04-25T10:16:07.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd70075a640 1 -- 172.21.15.133:0/965668149 wait complete. 2024-04-25T10:16:07.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd70075a640 1 Processor -- start 2024-04-25T10:16:07.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.061+0000 7fd70075a640 1 -- start start 2024-04-25T10:16:07.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd70075a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f806ac90 0x7fd6f815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd70075a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6f80ca390 0x7fd6f81f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd70075a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6f815bae0 0x7fd6f81f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd70075a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8168610 con 0x7fd6f806ac90 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd70075a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80ce2f0 con 0x7fd6f80ca390 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd70075a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8168790 con 0x7fd6f815bae0 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6fe4f6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f806ac90 0x7fd6f815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6fe4f6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f806ac90 0x7fd6f815ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55584/0 (socket says 172.21.15.133:55584) 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6fe4f6640 1 -- 172.21.15.133:0/2749105164 learned_addr learned my addr 172.21.15.133:0/2749105164 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6fe4f6640 1 -- 172.21.15.133:0/2749105164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6f815bae0 msgr2=0x7fd6f81f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6fe4f6640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6f815bae0 0x7fd6f81f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6fe4f6640 1 -- 172.21.15.133:0/2749105164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6f80ca390 msgr2=0x7fd6f81f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6fe4f6640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6f80ca390 0x7fd6f81f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6fe4f6640 1 -- 172.21.15.133:0/2749105164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f806b410 con 0x7fd6f806ac90 2024-04-25T10:16:07.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6fe4f6640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f806ac90 0x7fd6f815ae80 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4000c10 tx=0x7fd6f401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6eeffd640 1 -- 172.21.15.133:0/2749105164 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40bc020 con 0x7fd6f806ac90 2024-04-25T10:16:07.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6eeffd640 1 -- 172.21.15.133:0/2749105164 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f40b8420 con 0x7fd6f806ac90 2024-04-25T10:16:07.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd70075a640 1 -- 172.21.15.133:0/2749105164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f8165180 con 0x7fd6f806ac90 2024-04-25T10:16:07.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6eeffd640 1 -- 172.21.15.133:0/2749105164 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40b2040 con 0x7fd6f806ac90 2024-04-25T10:16:07.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd70075a640 1 -- 172.21.15.133:0/2749105164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f8165420 con 0x7fd6f806ac90 2024-04-25T10:16:07.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd70075a640 1 -- 172.21.15.133:0/2749105164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f8069000 con 0x7fd6f806ac90 2024-04-25T10:16:07.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6eeffd640 1 -- 172.21.15.133:0/2749105164 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6f40bf020 con 0x7fd6f806ac90 2024-04-25T10:16:07.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6eeffd640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd6dc066f60 0x7fd6dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6eeffd640 1 -- 172.21.15.133:0/2749105164 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd6f411f250 con 0x7fd6f806ac90 2024-04-25T10:16:07.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6fdcf5640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd6dc066f60 0x7fd6dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.065+0000 7fd6fdcf5640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd6dc066f60 0x7fd6dc069420 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80add80 tx=0x7fd6e8006c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.069+0000 7fd6eeffd640 1 -- 172.21.15.133:0/2749105164 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6f40c6030 con 0x7fd6f806ac90 2024-04-25T10:16:07.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.217+0000 7fd70075a640 1 -- 172.21.15.133:0/2749105164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd6f80ca910 con 0x7fd6f806ac90 2024-04-25T10:16:07.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.609+0000 7fd6eeffd640 1 -- 172.21.15.133:0/2749105164 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v166) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd6f40160d0 con 0x7fd6f806ac90 2024-04-25T10:16:07.611 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:16:07.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 -- 172.21.15.133:0/2749105164 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd6dc066f60 msgr2=0x7fd6dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd6dc066f60 0x7fd6dc069420 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80add80 tx=0x7fd6e8006c00 comp rx=0 tx=0).stop 2024-04-25T10:16:07.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 -- 172.21.15.133:0/2749105164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f806ac90 msgr2=0x7fd6f815ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f806ac90 0x7fd6f815ae80 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4000c10 tx=0x7fd6f401ac90 comp rx=0 tx=0).stop 2024-04-25T10:16:07.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 -- 172.21.15.133:0/2749105164 shutdown_connections 2024-04-25T10:16:07.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd6f815bae0 0x7fd6f81f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6f80ca390 0x7fd6f81f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd6dc066f60 0x7fd6dc069420 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 --2- 172.21.15.133:0/2749105164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f806ac90 0x7fd6f815ae80 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 -- 172.21.15.133:0/2749105164 >> 172.21.15.133:0/2749105164 conn(0x7fd6f80cc7c0 msgr2=0x7fd6f80ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 -- 172.21.15.133:0/2749105164 shutdown_connections 2024-04-25T10:16:07.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.613+0000 7fd70075a640 1 -- 172.21.15.133:0/2749105164 wait complete. 2024-04-25T10:16:07.639 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:16:07.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f2100175640 1 Processor -- start 2024-04-25T10:16:07.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f2100175640 1 -- start start 2024-04-25T10:16:07.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f2100175640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f20f80d0a10 0x7f20f80b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f2100175640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f20f8161570 0x7f20f8161970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f2100175640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f815de20 con 0x7f20f815da40 2024-04-25T10:16:07.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f2100175640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f81624a0 con 0x7f20f80d0a10 2024-04-25T10:16:07.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f2100175640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f80b7060 con 0x7f20f8161570 2024-04-25T10:16:07.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f20fd710640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f20f80d0a10 0x7f20f80b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f20fd710640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f20f80d0a10 0x7f20f80b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55592/0 (socket says 172.21.15.133:55592) 2024-04-25T10:16:07.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f20fd710640 1 -- 172.21.15.133:0/167447081 learned_addr learned my addr 172.21.15.133:0/167447081 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:07.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f20fdf11640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f20f815da40 0x7f20f80d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40918/0 (socket says 172.21.15.133:40918) 2024-04-25T10:16:07.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f20fe712640 1 --2- 172.21.15.133:0/167447081 >> v2:172.21.15.162:3300/0 conn(0x7f20f8161570 0x7f20f8161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f20fd710640 1 -- 172.21.15.133:0/167447081 >> v2:172.21.15.162:3300/0 conn(0x7f20f8161570 msgr2=0x7f20f8161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f20fd710640 1 --2- 172.21.15.133:0/167447081 >> v2:172.21.15.162:3300/0 conn(0x7f20f8161570 0x7f20f8161970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f20fd710640 1 -- 172.21.15.133:0/167447081 >> v1:172.21.15.133:6789/0 conn(0x7f20f815da40 legacy=0x7f20f80d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f20fd710640 1 -- 172.21.15.133:0/167447081 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f81621c0 con 0x7f20f80d0a10 2024-04-25T10:16:07.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f20fcf0f640 1 -- 172.21.15.133:0/167447081 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633027821 0 0) 0x7f20f815de20 con 0x7f20f815da40 2024-04-25T10:16:07.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.761+0000 7f20fe712640 1 --2- 172.21.15.133:0/167447081 >> v2:172.21.15.162:3300/0 conn(0x7f20f8161570 0x7f20f8161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:07.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f20fd710640 1 --2- 172.21.15.133:0/167447081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f80d0a10 0x7f20f80b6b20 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f20f80d1420 tx=0x7f20e801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859b8785b550d1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f20fcf0f640 1 -- 172.21.15.133:0/167447081 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80aa5f0 con 0x7f20f80d0a10 2024-04-25T10:16:07.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f20fcf0f640 1 -- 172.21.15.133:0/167447081 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e80af070 con 0x7f20f80d0a10 2024-04-25T10:16:07.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f20fcf0f640 1 -- 172.21.15.133:0/167447081 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80aa9b0 con 0x7f20f80d0a10 2024-04-25T10:16:07.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 -- 172.21.15.133:0/167447081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f80d0a10 msgr2=0x7f20f80b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 --2- 172.21.15.133:0/167447081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f80d0a10 0x7f20f80b6b20 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f20f80d1420 tx=0x7f20e801c490 comp rx=0 tx=0).stop 2024-04-25T10:16:07.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 -- 172.21.15.133:0/167447081 shutdown_connections 2024-04-25T10:16:07.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 --2- 172.21.15.133:0/167447081 >> v2:172.21.15.162:3300/0 conn(0x7f20f8161570 0x7f20f8161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 --2- 172.21.15.133:0/167447081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f80d0a10 0x7f20f80b6b20 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 -- 172.21.15.133:0/167447081 >> 172.21.15.133:0/167447081 conn(0x7f20f80723c0 msgr2=0x7f20f80727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 -- 172.21.15.133:0/167447081 shutdown_connections 2024-04-25T10:16:07.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 -- 172.21.15.133:0/167447081 wait complete. 2024-04-25T10:16:07.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 Processor -- start 2024-04-25T10:16:07.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 -- start start 2024-04-25T10:16:07.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f80d0a10 0x7f20f81565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f815da40 0x7f20f815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8161570 0x7f20f815e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80d31f0 con 0x7f20f8161570 2024-04-25T10:16:07.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8161eb0 con 0x7f20f80d0a10 2024-04-25T10:16:07.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.765+0000 7f2100175640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80d3370 con 0x7f20f815da40 2024-04-25T10:16:07.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fe712640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8161570 0x7f20f815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fe712640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8161570 0x7f20f815e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55598/0 (socket says 172.21.15.133:55598) 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fe712640 1 -- 172.21.15.133:0/2558586785 learned_addr learned my addr 172.21.15.133:0/2558586785 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fdf11640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f80d0a10 0x7f20f81565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fd710640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f815da40 0x7f20f815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fe712640 1 -- 172.21.15.133:0/2558586785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f815da40 msgr2=0x7f20f815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fe712640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f815da40 0x7f20f815e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fe712640 1 -- 172.21.15.133:0/2558586785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f80d0a10 msgr2=0x7f20f81565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fe712640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f80d0a10 0x7f20f81565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fe712640 1 -- 172.21.15.133:0/2558586785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f81551a0 con 0x7f20f8161570 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fe712640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8161570 0x7f20f815e650 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f20f000cd80 tx=0x7f20f000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20ee7fc640 1 -- 172.21.15.133:0/2558586785 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f000f680 con 0x7f20f8161570 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20ee7fc640 1 -- 172.21.15.133:0/2558586785 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f000f840 con 0x7f20f8161570 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f2100175640 1 -- 172.21.15.133:0/2558586785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f8155480 con 0x7f20f8161570 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20ee7fc640 1 -- 172.21.15.133:0/2558586785 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f000fc30 con 0x7f20f8161570 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f2100175640 1 -- 172.21.15.133:0/2558586785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f8155a40 con 0x7f20f8161570 2024-04-25T10:16:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f2100175640 1 -- 172.21.15.133:0/2558586785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f8154f30 con 0x7f20f8161570 2024-04-25T10:16:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20ee7fc640 1 -- 172.21.15.133:0/2558586785 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20f00213f0 con 0x7f20f8161570 2024-04-25T10:16:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20ee7fc640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20dc066f90 0x7f20dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fdf11640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20dc066f90 0x7f20dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20ee7fc640 1 -- 172.21.15.133:0/2558586785 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f20f0085da0 con 0x7f20f8161570 2024-04-25T10:16:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.769+0000 7f20fdf11640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20dc066f90 0x7f20dc069450 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f20f4007fd0 tx=0x7f20f4007ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.773+0000 7f20ee7fc640 1 -- 172.21.15.133:0/2558586785 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20f0014030 con 0x7f20f8161570 2024-04-25T10:16:07.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:07.933+0000 7f2100175640 1 -- 172.21.15.133:0/2558586785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f20f8068e80 con 0x7f20f8161570 2024-04-25T10:16:08.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.613+0000 7f20ee7fc640 1 -- 172.21.15.133:0/2558586785 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v167) v1 ==== 170+0+0 (secure 0 0 0) 0x7f20f0056e00 con 0x7f20f8161570 2024-04-25T10:16:08.615 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 -- 172.21.15.133:0/2558586785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20dc066f90 msgr2=0x7f20dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20dc066f90 0x7f20dc069450 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f20f4007fd0 tx=0x7f20f4007ba0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 -- 172.21.15.133:0/2558586785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8161570 msgr2=0x7f20f815e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8161570 0x7f20f815e650 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f20f000cd80 tx=0x7f20f000a650 comp rx=0 tx=0).stop 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 -- 172.21.15.133:0/2558586785 shutdown_connections 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f20dc066f90 0x7f20dc069450 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20f8161570 0x7f20f815e650 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f20f815da40 0x7f20f815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 --2- 172.21.15.133:0/2558586785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f20f80d0a10 0x7f20f81565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 -- 172.21.15.133:0/2558586785 >> 172.21.15.133:0/2558586785 conn(0x7f20f80723c0 msgr2=0x7f20f80cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 -- 172.21.15.133:0/2558586785 shutdown_connections 2024-04-25T10:16:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.617+0000 7f2100175640 1 -- 172.21.15.133:0/2558586785 wait complete. 2024-04-25T10:16:08.650 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:16:08.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a81e2640 1 Processor -- start 2024-04-25T10:16:08.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a81e2640 1 -- start start 2024-04-25T10:16:08.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a81e2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe3a0161c80 0x7fe3a00b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a81e2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a015de20 con 0x7fe3a015da40 2024-04-25T10:16:08.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a81e2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a01621c0 con 0x7fe3a00d0a10 2024-04-25T10:16:08.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a81e2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a01624a0 con 0x7fe3a0161c80 2024-04-25T10:16:08.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a677f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe3a0161c80 0x7fe3a00b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a5f7e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe3a015da40 0x7fe3a00d0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40930/0 (socket says 172.21.15.133:40930) 2024-04-25T10:16:08.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a5f7e640 1 -- 172.21.15.133:0/374822089 learned_addr learned my addr 172.21.15.133:0/374822089 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:08.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a677f640 1 -- 172.21.15.133:0/374822089 >> v1:172.21.15.162:6789/0 conn(0x7fe3a00d0a10 legacy=0x7fe3a0161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a677f640 1 -- 172.21.15.133:0/374822089 >> v1:172.21.15.133:6789/0 conn(0x7fe3a015da40 legacy=0x7fe3a00d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a4f7c640 1 -- 172.21.15.133:0/374822089 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705911846 0 0) 0x7fe3a015de20 con 0x7fe3a015da40 2024-04-25T10:16:08.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a677f640 1 -- 172.21.15.133:0/374822089 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00d1f00 con 0x7fe3a0161c80 2024-04-25T10:16:08.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.773+0000 7fe3a677f640 1 --2- 172.21.15.133:0/374822089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a0161c80 0x7fe3a00b6b20 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7fe398009d90 tx=0x7fe39801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114cfd1665c3ff67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a4f7c640 1 -- 172.21.15.133:0/374822089 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980b2070 con 0x7fe3a0161c80 2024-04-25T10:16:08.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a4f7c640 1 -- 172.21.15.133:0/374822089 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3980a5870 con 0x7fe3a0161c80 2024-04-25T10:16:08.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a4f7c640 1 -- 172.21.15.133:0/374822089 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980a5c40 con 0x7fe3a0161c80 2024-04-25T10:16:08.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 -- 172.21.15.133:0/374822089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a0161c80 msgr2=0x7fe3a00b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 --2- 172.21.15.133:0/374822089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a0161c80 0x7fe3a00b6b20 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7fe398009d90 tx=0x7fe39801c8a0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 -- 172.21.15.133:0/374822089 shutdown_connections 2024-04-25T10:16:08.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 --2- 172.21.15.133:0/374822089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a0161c80 0x7fe3a00b6b20 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 -- 172.21.15.133:0/374822089 >> 172.21.15.133:0/374822089 conn(0x7fe3a00723c0 msgr2=0x7fe3a00727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 -- 172.21.15.133:0/374822089 shutdown_connections 2024-04-25T10:16:08.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 -- 172.21.15.133:0/374822089 wait complete. 2024-04-25T10:16:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 Processor -- start 2024-04-25T10:16:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 -- start start 2024-04-25T10:16:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3a00d0a10 0x7fe3a00b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3a015da40 0x7fe3a014cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a0161c80 0x7fe3a015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00d3600 con 0x7fe3a0161c80 2024-04-25T10:16:08.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00d3480 con 0x7fe3a015da40 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a016b920 con 0x7fe3a00d0a10 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a677f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a0161c80 0x7fe3a015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a677f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a0161c80 0x7fe3a015ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55620/0 (socket says 172.21.15.133:55620) 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a677f640 1 -- 172.21.15.133:0/497416762 learned_addr learned my addr 172.21.15.133:0/497416762 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a5f7e640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3a00d0a10 0x7fe3a00b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a577d640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3a015da40 0x7fe3a014cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a677f640 1 -- 172.21.15.133:0/497416762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3a00d0a10 msgr2=0x7fe3a00b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a677f640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3a00d0a10 0x7fe3a00b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a677f640 1 -- 172.21.15.133:0/497416762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3a015da40 msgr2=0x7fe3a014cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a677f640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3a015da40 0x7fe3a014cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a677f640 1 -- 172.21.15.133:0/497416762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a014d2c0 con 0x7fe3a0161c80 2024-04-25T10:16:08.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a677f640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a0161c80 0x7fe3a015ee70 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fe398009d90 tx=0x7fe398019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3967fc640 1 -- 172.21.15.133:0/497416762 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980b9980 con 0x7fe3a0161c80 2024-04-25T10:16:08.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3967fc640 1 -- 172.21.15.133:0/497416762 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3980b9b20 con 0x7fe3a0161c80 2024-04-25T10:16:08.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 -- 172.21.15.133:0/497416762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a015ec40 con 0x7fe3a0161c80 2024-04-25T10:16:08.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3967fc640 1 -- 172.21.15.133:0/497416762 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980ad440 con 0x7fe3a0161c80 2024-04-25T10:16:08.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.777+0000 7fe3a81e2640 1 -- 172.21.15.133:0/497416762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a00b5ba0 con 0x7fe3a0161c80 2024-04-25T10:16:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.781+0000 7fe3a81e2640 1 -- 172.21.15.133:0/497416762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a0068e80 con 0x7fe3a0161c80 2024-04-25T10:16:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.781+0000 7fe3967fc640 1 -- 172.21.15.133:0/497416762 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3980cb060 con 0x7fe3a0161c80 2024-04-25T10:16:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.781+0000 7fe3967fc640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe384066f60 0x7fe384069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.781+0000 7fe3967fc640 1 -- 172.21.15.133:0/497416762 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fe398127600 con 0x7fe3a0161c80 2024-04-25T10:16:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.781+0000 7fe3a5f7e640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe384066f60 0x7fe384069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.781+0000 7fe3a5f7e640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe384066f60 0x7fe384069420 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe39c000f70 tx=0x7fe39c007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.785+0000 7fe3967fc640 1 -- 172.21.15.133:0/497416762 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe398016520 con 0x7fe3a0161c80 2024-04-25T10:16:08.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:08.977+0000 7fe3a81e2640 1 -- 172.21.15.133:0/497416762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe3a00d16a0 con 0x7fe3a0161c80 2024-04-25T10:16:09.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.637+0000 7fe3967fc640 1 -- 172.21.15.133:0/497416762 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v103) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe3980f84f0 con 0x7fe3a0161c80 2024-04-25T10:16:09.640 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 25 and data pool 26 2024-04-25T10:16:09.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.641+0000 7fe3a81e2640 1 -- 172.21.15.133:0/497416762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe384066f60 msgr2=0x7fe384069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.641+0000 7fe3a81e2640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe384066f60 0x7fe384069420 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe39c000f70 tx=0x7fe39c007b60 comp rx=0 tx=0).stop 2024-04-25T10:16:09.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.641+0000 7fe3a81e2640 1 -- 172.21.15.133:0/497416762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a0161c80 msgr2=0x7fe3a015ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.641+0000 7fe3a81e2640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a0161c80 0x7fe3a015ee70 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fe398009d90 tx=0x7fe398019640 comp rx=0 tx=0).stop 2024-04-25T10:16:09.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.641+0000 7fe3a81e2640 1 -- 172.21.15.133:0/497416762 shutdown_connections 2024-04-25T10:16:09.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.641+0000 7fe3a81e2640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe384066f60 0x7fe384069420 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.641+0000 7fe3a81e2640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3a0161c80 0x7fe3a015ee70 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.641+0000 7fe3a81e2640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3a015da40 0x7fe3a014cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.641+0000 7fe3a81e2640 1 --2- 172.21.15.133:0/497416762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3a00d0a10 0x7fe3a00b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.641+0000 7fe3a81e2640 1 -- 172.21.15.133:0/497416762 >> 172.21.15.133:0/497416762 conn(0x7fe3a00723c0 msgr2=0x7fe3a0153da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.645+0000 7fe3a81e2640 1 -- 172.21.15.133:0/497416762 shutdown_connections 2024-04-25T10:16:09.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.645+0000 7fe3a81e2640 1 -- 172.21.15.133:0/497416762 wait complete. 2024-04-25T10:16:09.699 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:09.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02d23eb640 1 Processor -- start 2024-04-25T10:16:09.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02d23eb640 1 -- start start 2024-04-25T10:16:09.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02d23eb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f02cc0b1f70 0x7f02cc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02d23eb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f02cc0ce330 0x7f02cc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02d23eb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02cc0c61d0 con 0x7f02cc15c9a0 2024-04-25T10:16:09.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02d23eb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02cc0cecc0 con 0x7f02cc0b1f70 2024-04-25T10:16:09.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02d23eb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02cc0c64b0 con 0x7f02cc0ce330 2024-04-25T10:16:09.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02cb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f02cc0b1f70 0x7f02cc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02cb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f02cc0b1f70 0x7f02cc06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55632/0 (socket says 172.21.15.133:55632) 2024-04-25T10:16:09.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02cb7fe640 1 -- 172.21.15.133:0/4124488694 learned_addr learned my addr 172.21.15.133:0/4124488694 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:09.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02d0988640 1 --2- 172.21.15.133:0/4124488694 >> v2:172.21.15.162:3300/0 conn(0x7f02cc0ce330 0x7f02cc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02cb7fe640 1 -- 172.21.15.133:0/4124488694 >> v2:172.21.15.162:3300/0 conn(0x7f02cc0ce330 msgr2=0x7f02cc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02cb7fe640 1 --2- 172.21.15.133:0/4124488694 >> v2:172.21.15.162:3300/0 conn(0x7f02cc0ce330 0x7f02cc0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02cb7fe640 1 -- 172.21.15.133:0/4124488694 >> v1:172.21.15.162:6789/0 conn(0x7f02cc15c9a0 legacy=0x7f02cc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.813+0000 7f02cb7fe640 1 -- 172.21.15.133:0/4124488694 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc06d4f0 con 0x7f02cc0b1f70 2024-04-25T10:16:09.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02cb7fe640 1 --2- 172.21.15.133:0/4124488694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02cc0b1f70 0x7f02cc06dcf0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0cc0f0 tx=0x7f02b401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381756e7a83893e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02caffd640 1 -- 172.21.15.133:0/4124488694 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40a58a0 con 0x7f02cc0b1f70 2024-04-25T10:16:09.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02caffd640 1 -- 172.21.15.133:0/4124488694 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b40a5a40 con 0x7f02cc0b1f70 2024-04-25T10:16:09.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02caffd640 1 -- 172.21.15.133:0/4124488694 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40a5e10 con 0x7f02cc0b1f70 2024-04-25T10:16:09.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 -- 172.21.15.133:0/4124488694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02cc0b1f70 msgr2=0x7f02cc06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 --2- 172.21.15.133:0/4124488694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02cc0b1f70 0x7f02cc06dcf0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0cc0f0 tx=0x7f02b401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 -- 172.21.15.133:0/4124488694 shutdown_connections 2024-04-25T10:16:09.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 --2- 172.21.15.133:0/4124488694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02cc0b1f70 0x7f02cc06dcf0 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 --2- 172.21.15.133:0/4124488694 >> v2:172.21.15.162:3300/0 conn(0x7f02cc0ce330 0x7f02cc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 -- 172.21.15.133:0/4124488694 >> 172.21.15.133:0/4124488694 conn(0x7f02cc067000 msgr2=0x7f02cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 -- 172.21.15.133:0/4124488694 shutdown_connections 2024-04-25T10:16:09.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 -- 172.21.15.133:0/4124488694 wait complete. 2024-04-25T10:16:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 Processor -- start 2024-04-25T10:16:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 -- start start 2024-04-25T10:16:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02cc0b1f70 0x7f02cc157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02cc0ce330 0x7f02cc153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc15c9a0 0x7f02cc13cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc15dad0 con 0x7f02cc0b1f70 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc15d950 con 0x7f02cc15c9a0 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02d23eb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc15dc50 con 0x7f02cc0ce330 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02cbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02cc0b1f70 0x7f02cc157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02cbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02cc0b1f70 0x7f02cc157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55648/0 (socket says 172.21.15.133:55648) 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02cbfff640 1 -- 172.21.15.133:0/542939366 learned_addr learned my addr 172.21.15.133:0/542939366 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02cb7fe640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02cc0ce330 0x7f02cc153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02cbfff640 1 -- 172.21.15.133:0/542939366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02cc0ce330 msgr2=0x7f02cc153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02cbfff640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02cc0ce330 0x7f02cc153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02cbfff640 1 -- 172.21.15.133:0/542939366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc15c9a0 msgr2=0x7f02cc13cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02cbfff640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc15c9a0 0x7f02cc13cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02cbfff640 1 -- 172.21.15.133:0/542939366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc1576d0 con 0x7f02cc0b1f70 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.817+0000 7f02cbfff640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02cc0b1f70 0x7f02cc157190 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00c3a0 tx=0x7f02bc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.821+0000 7f02c8ff9640 1 -- 172.21.15.133:0/542939366 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc0052d0 con 0x7f02cc0b1f70 2024-04-25T10:16:09.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.821+0000 7f02c8ff9640 1 -- 172.21.15.133:0/542939366 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02bc005470 con 0x7f02cc0b1f70 2024-04-25T10:16:09.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.821+0000 7f02d23eb640 1 -- 172.21.15.133:0/542939366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc155cc0 con 0x7f02cc0b1f70 2024-04-25T10:16:09.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.821+0000 7f02c8ff9640 1 -- 172.21.15.133:0/542939366 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc007260 con 0x7f02cc0b1f70 2024-04-25T10:16:09.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.821+0000 7f02d23eb640 1 -- 172.21.15.133:0/542939366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc156280 con 0x7f02cc0b1f70 2024-04-25T10:16:09.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.821+0000 7f02d23eb640 1 -- 172.21.15.133:0/542939366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02cc06bc20 con 0x7f02cc0b1f70 2024-04-25T10:16:09.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.821+0000 7f02c8ff9640 1 -- 172.21.15.133:0/542939366 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02bc007400 con 0x7f02cc0b1f70 2024-04-25T10:16:09.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.821+0000 7f02c8ff9640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02ac066f90 0x7f02ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.821+0000 7f02c8ff9640 1 -- 172.21.15.133:0/542939366 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f02bc0845c0 con 0x7f02cc0b1f70 2024-04-25T10:16:09.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.821+0000 7f02cb7fe640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02ac066f90 0x7f02ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.821+0000 7f02cb7fe640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02ac066f90 0x7f02ac069450 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f02b40ae000 tx=0x7f02b40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.825+0000 7f02c8ff9640 1 -- 172.21.15.133:0/542939366 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f02bc055440 con 0x7f02cc0b1f70 2024-04-25T10:16:09.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.977+0000 7f02d23eb640 1 -- 172.21.15.133:0/542939366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02cc068ef0 con 0x7f02cc0b1f70 2024-04-25T10:16:09.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02c8ff9640 1 -- 172.21.15.133:0/542939366 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+16925 (secure 0 0 0) 0x7f02bc054d40 con 0x7f02cc0b1f70 2024-04-25T10:16:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:09.981 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":168,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:16:08.982458+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:07.256836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:07.975826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:09.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 -- 172.21.15.133:0/542939366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02ac066f90 msgr2=0x7f02ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02ac066f90 0x7f02ac069450 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f02b40ae000 tx=0x7f02b40a7070 comp rx=0 tx=0).stop 2024-04-25T10:16:09.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 -- 172.21.15.133:0/542939366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02cc0b1f70 msgr2=0x7f02cc157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02cc0b1f70 0x7f02cc157190 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00c3a0 tx=0x7f02bc00c860 comp rx=0 tx=0).stop 2024-04-25T10:16:09.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 -- 172.21.15.133:0/542939366 shutdown_connections 2024-04-25T10:16:09.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f02cc15c9a0 0x7f02cc13cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f02cc0ce330 0x7f02cc153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f02ac066f90 0x7f02ac069450 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 --2- 172.21.15.133:0/542939366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02cc0b1f70 0x7f02cc157190 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 -- 172.21.15.133:0/542939366 >> 172.21.15.133:0/542939366 conn(0x7f02cc067000 msgr2=0x7f02cc0c5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 -- 172.21.15.133:0/542939366 shutdown_connections 2024-04-25T10:16:09.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:09.981+0000 7f02d23eb640 1 -- 172.21.15.133:0/542939366 wait complete. 2024-04-25T10:16:09.999 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:10.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.113+0000 7f7708976640 1 Processor -- start 2024-04-25T10:16:10.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.113+0000 7f7708976640 1 -- start start 2024-04-25T10:16:10.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.113+0000 7f7708976640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f77040cea40 0x7f77040b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.113+0000 7f7708976640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770406d350 con 0x7f7704152750 2024-04-25T10:16:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.113+0000 7f7708976640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770415bf90 con 0x7f7704152b30 2024-04-25T10:16:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.113+0000 7f7708976640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770406d5d0 con 0x7f77040cea40 2024-04-25T10:16:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f770259c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7704152750 0x7f77040ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40940/0 (socket says 172.21.15.133:40940) 2024-04-25T10:16:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f770259c640 1 -- 172.21.15.133:0/3993732892 learned_addr learned my addr 172.21.15.133:0/3993732892 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7702d9d640 1 --2- 172.21.15.133:0/3993732892 >> v2:172.21.15.133:3300/0 conn(0x7f77040cea40 0x7f77040b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7702d9d640 1 -- 172.21.15.133:0/3993732892 >> v1:172.21.15.162:6789/0 conn(0x7f7704152b30 legacy=0x7f77040b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7702d9d640 1 -- 172.21.15.133:0/3993732892 >> v1:172.21.15.133:6789/0 conn(0x7f7704152750 legacy=0x7f77040ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f770159a640 1 -- 172.21.15.133:0/3993732892 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551242664 0 0) 0x7f76f4001120 con 0x7f7704152750 2024-04-25T10:16:10.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7702d9d640 1 -- 172.21.15.133:0/3993732892 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770415c230 con 0x7f77040cea40 2024-04-25T10:16:10.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7702d9d640 1 --2- 172.21.15.133:0/3993732892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77040cea40 0x7f77040b1ec0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009d90 tx=0x7f76fc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c626627cebc57f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f770159a640 1 -- 172.21.15.133:0/3993732892 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0b2070 con 0x7f77040cea40 2024-04-25T10:16:10.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f770159a640 1 -- 172.21.15.133:0/3993732892 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76fc0a58e0 con 0x7f77040cea40 2024-04-25T10:16:10.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f770159a640 1 -- 172.21.15.133:0/3993732892 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0a5cb0 con 0x7f77040cea40 2024-04-25T10:16:10.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7708976640 1 -- 172.21.15.133:0/3993732892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77040cea40 msgr2=0x7f77040b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7708976640 1 --2- 172.21.15.133:0/3993732892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77040cea40 0x7f77040b1ec0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009d90 tx=0x7f76fc01c930 comp rx=0 tx=0).stop 2024-04-25T10:16:10.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7708976640 1 -- 172.21.15.133:0/3993732892 shutdown_connections 2024-04-25T10:16:10.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7708976640 1 --2- 172.21.15.133:0/3993732892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77040cea40 0x7f77040b1ec0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7708976640 1 -- 172.21.15.133:0/3993732892 >> 172.21.15.133:0/3993732892 conn(0x7f7704067000 msgr2=0x7f770406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7708976640 1 -- 172.21.15.133:0/3993732892 shutdown_connections 2024-04-25T10:16:10.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.117+0000 7f7708976640 1 -- 172.21.15.133:0/3993732892 wait complete. 2024-04-25T10:16:10.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7708976640 1 Processor -- start 2024-04-25T10:16:10.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7708976640 1 -- start start 2024-04-25T10:16:10.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7708976640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77040cea40 0x7f77041518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7708976640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7704152750 0x7f7704161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7708976640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7704152b30 0x7f7704155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7708976640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7704169180 con 0x7f7704152750 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7708976640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7704169000 con 0x7f77040cea40 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7708976640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7704169300 con 0x7f7704152b30 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7701d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7704152750 0x7f7704161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7701d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7704152750 0x7f7704161000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55662/0 (socket says 172.21.15.133:55662) 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7701d9b640 1 -- 172.21.15.133:0/2858997781 learned_addr learned my addr 172.21.15.133:0/2858997781 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f770259c640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77040cea40 0x7f77041518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7701d9b640 1 -- 172.21.15.133:0/2858997781 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7704152b30 msgr2=0x7f7704155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7701d9b640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7704152b30 0x7f7704155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7701d9b640 1 -- 172.21.15.133:0/2858997781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77040cea40 msgr2=0x7f77041518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7701d9b640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77040cea40 0x7f77041518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7701d9b640 1 -- 172.21.15.133:0/2858997781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7704151e30 con 0x7f7704152750 2024-04-25T10:16:10.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f770259c640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77040cea40 0x7f77041518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:10.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7701d9b640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7704152750 0x7f7704161000 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f76f800c450 tx=0x7f76f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f76f2ffd640 1 -- 172.21.15.133:0/2858997781 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8012830 con 0x7f7704152750 2024-04-25T10:16:10.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f76f2ffd640 1 -- 172.21.15.133:0/2858997781 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8017070 con 0x7f7704152750 2024-04-25T10:16:10.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7708976640 1 -- 172.21.15.133:0/2858997781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77040ca410 con 0x7f7704152750 2024-04-25T10:16:10.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f76f2ffd640 1 -- 172.21.15.133:0/2858997781 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8012c30 con 0x7f7704152750 2024-04-25T10:16:10.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7708976640 1 -- 172.21.15.133:0/2858997781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77040ca9d0 con 0x7f7704152750 2024-04-25T10:16:10.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f7708976640 1 -- 172.21.15.133:0/2858997781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77040cb190 con 0x7f7704152750 2024-04-25T10:16:10.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.121+0000 7f76f2ffd640 1 -- 172.21.15.133:0/2858997781 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76f801c460 con 0x7f7704152750 2024-04-25T10:16:10.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.125+0000 7f76f2ffd640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76e4066f60 0x7f76e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.125+0000 7f76f2ffd640 1 -- 172.21.15.133:0/2858997781 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f76f8085000 con 0x7f7704152750 2024-04-25T10:16:10.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.125+0000 7f770259c640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76e4066f60 0x7f76e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.125+0000 7f770259c640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76e4066f60 0x7f76e4069420 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f76f4007fd0 tx=0x7f76f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.129+0000 7f76f2ffd640 1 -- 172.21.15.133:0/2858997781 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77040cb190 con 0x7f7704152750 2024-04-25T10:16:10.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.285+0000 7f7708976640 1 -- 172.21.15.133:0/2858997781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7704068ef0 con 0x7f7704152750 2024-04-25T10:16:10.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.289+0000 7f76f2ffd640 1 -- 172.21.15.133:0/2858997781 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+16925 (secure 0 0 0) 0x7f76f8055e80 con 0x7f7704152750 2024-04-25T10:16:10.290 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:10.291 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":168,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:16:08.982458+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:07.256836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:07.975826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 -- 172.21.15.133:0/2858997781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76e4066f60 msgr2=0x7f76e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76e4066f60 0x7f76e4069420 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f76f4007fd0 tx=0x7f76f4009040 comp rx=0 tx=0).stop 2024-04-25T10:16:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 -- 172.21.15.133:0/2858997781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7704152750 msgr2=0x7f7704161000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7704152750 0x7f7704161000 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f76f800c450 tx=0x7f76f800c910 comp rx=0 tx=0).stop 2024-04-25T10:16:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 -- 172.21.15.133:0/2858997781 shutdown_connections 2024-04-25T10:16:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7704152b30 0x7f7704155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f76e4066f60 0x7f76e4069420 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7704152750 0x7f7704161000 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 --2- 172.21.15.133:0/2858997781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f77040cea40 0x7f77041518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 -- 172.21.15.133:0/2858997781 >> 172.21.15.133:0/2858997781 conn(0x7f7704067000 msgr2=0x7f77040c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 -- 172.21.15.133:0/2858997781 shutdown_connections 2024-04-25T10:16:10.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.293+0000 7f7708976640 1 -- 172.21.15.133:0/2858997781 wait complete. 2024-04-25T10:16:10.310 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:16:10.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.433+0000 7fcf8bf9d640 1 Processor -- start 2024-04-25T10:16:10.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.433+0000 7fcf8bf9d640 1 -- start start 2024-04-25T10:16:10.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf8bf9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcf8415d5e0 0x7fcf840cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf8bf9d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf840ce2e0 con 0x7fcf840cd820 2024-04-25T10:16:10.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf8bf9d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8406d4b0 con 0x7fcf84145170 2024-04-25T10:16:10.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf8bf9d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf8406db00 con 0x7fcf8415d5e0 2024-04-25T10:16:10.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf89538640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcf840cd820 0x7fcf8415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40956/0 (socket says 172.21.15.133:40956) 2024-04-25T10:16:10.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf89538640 1 -- 172.21.15.133:0/874719075 learned_addr learned my addr 172.21.15.133:0/874719075 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:10.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf8a53a640 1 --2- 172.21.15.133:0/874719075 >> v2:172.21.15.162:3300/0 conn(0x7fcf8415d5e0 0x7fcf840cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867670758 0 0) 0x7fcf840ce2e0 con 0x7fcf840cd820 2024-04-25T10:16:10.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf70003530 con 0x7fcf840cd820 2024-04-25T10:16:10.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 864536741 0 0) 0x7fcf70003530 con 0x7fcf840cd820 2024-04-25T10:16:10.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 >> v2:172.21.15.162:3300/0 conn(0x7fcf8415d5e0 msgr2=0x7fcf840cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 --2- 172.21.15.133:0/874719075 >> v2:172.21.15.162:3300/0 conn(0x7fcf8415d5e0 0x7fcf840cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 >> v1:172.21.15.162:6789/0 conn(0x7fcf84145170 legacy=0x7fcf8406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8406d790 con 0x7fcf840cd820 2024-04-25T10:16:10.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fcf74001680 con 0x7fcf840cd820 2024-04-25T10:16:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf8a53a640 1 --2- 172.21.15.133:0/874719075 >> v2:172.21.15.162:3300/0 conn(0x7fcf8415d5e0 0x7fcf840cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 >> v1:172.21.15.133:6789/0 conn(0x7fcf840cd820 legacy=0x7fcf8415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf700039c0 0x7fcf70003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf70004810 0x7fcf70004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf700051d0 0x7fcf7000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8406db00 con 0x7fcf700039c0 2024-04-25T10:16:10.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7000c120 con 0x7fcf700051d0 2024-04-25T10:16:10.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7000c320 con 0x7fcf70004810 2024-04-25T10:16:10.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf89d39640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf700039c0 0x7fcf70003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf8a53a640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf700051d0 0x7fcf7000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf89d39640 1 -- 172.21.15.133:0/874719075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf70004810 msgr2=0x7fcf70004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf89538640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf70004810 0x7fcf70004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf89d39640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf70004810 0x7fcf70004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf89d39640 1 -- 172.21.15.133:0/874719075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf700051d0 msgr2=0x7fcf7000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf89d39640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf700051d0 0x7fcf7000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf89d39640 1 -- 172.21.15.133:0/874719075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf70003530 con 0x7fcf700039c0 2024-04-25T10:16:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf8a53a640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf700051d0 0x7fcf7000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.437+0000 7fcf8a53a640 1 -- 172.21.15.133:0/874719075 reap_dead start 2024-04-25T10:16:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89d39640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf700039c0 0x7fcf70003de0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fcf80001780 tx=0x7fcf8001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5677e8d45acf6a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf800013a0 con 0x7fcf700039c0 2024-04-25T10:16:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf800052f0 con 0x7fcf700039c0 2024-04-25T10:16:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf88d37640 1 -- 172.21.15.133:0/874719075 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80003840 con 0x7fcf700039c0 2024-04-25T10:16:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/874719075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf700039c0 msgr2=0x7fcf70003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf700039c0 0x7fcf70003de0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fcf80001780 tx=0x7fcf8001c400 comp rx=0 tx=0).stop 2024-04-25T10:16:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/874719075 shutdown_connections 2024-04-25T10:16:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 --2- 172.21.15.133:0/874719075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf700039c0 0x7fcf70003de0 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/874719075 >> 172.21.15.133:0/874719075 conn(0x7fcf84066f70 msgr2=0x7fcf8406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/874719075 shutdown_connections 2024-04-25T10:16:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/874719075 wait complete. 2024-04-25T10:16:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 Processor -- start 2024-04-25T10:16:10.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 -- start start 2024-04-25T10:16:10.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf700039c0 0x7fcf841f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf841f4f10 0x7fcf8413c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf8413ce70 0x7fcf84143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840b1a50 con 0x7fcf841f4f10 2024-04-25T10:16:10.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840b18d0 con 0x7fcf8413ce70 2024-04-25T10:16:10.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840b1bd0 con 0x7fcf700039c0 2024-04-25T10:16:10.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89538640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf841f4f10 0x7fcf8413c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89538640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf841f4f10 0x7fcf8413c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55682/0 (socket says 172.21.15.133:55682) 2024-04-25T10:16:10.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89538640 1 -- 172.21.15.133:0/2542315469 learned_addr learned my addr 172.21.15.133:0/2542315469 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:10.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8a53a640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf8413ce70 0x7fcf84143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89d39640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf700039c0 0x7fcf841f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89538640 1 -- 172.21.15.133:0/2542315469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf700039c0 msgr2=0x7fcf841f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89538640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf700039c0 0x7fcf841f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89538640 1 -- 172.21.15.133:0/2542315469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf8413ce70 msgr2=0x7fcf84143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89538640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf8413ce70 0x7fcf84143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89538640 1 -- 172.21.15.133:0/2542315469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf841f5470 con 0x7fcf841f4f10 2024-04-25T10:16:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89d39640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf700039c0 0x7fcf841f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf89538640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf841f4f10 0x7fcf8413c930 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400d840 tx=0x7fcf7400dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf7a7fc640 1 -- 172.21.15.133:0/2542315469 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74019070 con 0x7fcf841f4f10 2024-04-25T10:16:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf7a7fc640 1 -- 172.21.15.133:0/2542315469 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf740074e0 con 0x7fcf841f4f10 2024-04-25T10:16:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/2542315469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf841f5750 con 0x7fcf841f4f10 2024-04-25T10:16:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf7a7fc640 1 -- 172.21.15.133:0/2542315469 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74004a30 con 0x7fcf841f4f10 2024-04-25T10:16:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.441+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/2542315469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf841f5d10 con 0x7fcf841f4f10 2024-04-25T10:16:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.445+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/2542315469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8413d250 con 0x7fcf841f4f10 2024-04-25T10:16:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.445+0000 7fcf7a7fc640 1 -- 172.21.15.133:0/2542315469 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf7401c070 con 0x7fcf841f4f10 2024-04-25T10:16:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.445+0000 7fcf7a7fc640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf68066f60 0x7fcf68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.445+0000 7fcf7a7fc640 1 -- 172.21.15.133:0/2542315469 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcf74084ce0 con 0x7fcf841f4f10 2024-04-25T10:16:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.445+0000 7fcf89d39640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf68066f60 0x7fcf68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.445+0000 7fcf89d39640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf68066f60 0x7fcf68069420 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fcf80001750 tx=0x7fcf800011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.449+0000 7fcf7a7fc640 1 -- 172.21.15.133:0/2542315469 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcf74055ad0 con 0x7fcf841f4f10 2024-04-25T10:16:10.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.649+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/2542315469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcf840b38b0 con 0x7fcf841f4f10 2024-04-25T10:16:10.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.673+0000 7fcf7a7fc640 1 -- 172.21.15.133:0/2542315469 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v104) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcf740553d0 con 0x7fcf841f4f10 2024-04-25T10:16:10.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/2542315469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf68066f60 msgr2=0x7fcf68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf68066f60 0x7fcf68069420 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fcf80001750 tx=0x7fcf800011a0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/2542315469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf841f4f10 msgr2=0x7fcf8413c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf841f4f10 0x7fcf8413c930 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400d840 tx=0x7fcf7400dd00 comp rx=0 tx=0).stop 2024-04-25T10:16:10.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/2542315469 shutdown_connections 2024-04-25T10:16:10.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf8413ce70 0x7fcf84143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf68066f60 0x7fcf68069420 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf841f4f10 0x7fcf8413c930 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 --2- 172.21.15.133:0/2542315469 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf700039c0 0x7fcf841f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/2542315469 >> 172.21.15.133:0/2542315469 conn(0x7fcf84066f70 msgr2=0x7fcf8415bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/2542315469 shutdown_connections 2024-04-25T10:16:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.677+0000 7fcf8bf9d640 1 -- 172.21.15.133:0/2542315469 wait complete. 2024-04-25T10:16:10.695 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:16:10.696 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e74f8640 1 Processor -- start 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e74f8640 1 -- start start 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e74f8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f42e0145170 0x7f42e006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e74f8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f42e006e4d0 0x7f42e00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e74f8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42e00cdd60 0x7f42e015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e74f8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e006db00 con 0x7f42e006e4d0 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e74f8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e0145550 con 0x7f42e0145170 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e74f8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00ce2e0 con 0x7f42e00cdd60 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e4a93640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f42e006e4d0 0x7f42e00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e4a93640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f42e006e4d0 0x7f42e00cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55684/0 (socket says 172.21.15.133:55684) 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e4a93640 1 -- 172.21.15.133:0/2907695832 learned_addr learned my addr 172.21.15.133:0/2907695832 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e5294640 1 --2- 172.21.15.133:0/2907695832 >> v2:172.21.15.162:3300/0 conn(0x7f42e0145170 0x7f42e006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e5a95640 1 --2- 172.21.15.133:0/2907695832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42e00cdd60 0x7f42e015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e4a93640 1 -- 172.21.15.133:0/2907695832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42e00cdd60 msgr2=0x7f42e015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e4a93640 1 --2- 172.21.15.133:0/2907695832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42e00cdd60 0x7f42e015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e4a93640 1 -- 172.21.15.133:0/2907695832 >> v2:172.21.15.162:3300/0 conn(0x7f42e0145170 msgr2=0x7f42e006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e4a93640 1 --2- 172.21.15.133:0/2907695832 >> v2:172.21.15.162:3300/0 conn(0x7f42e0145170 0x7f42e006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e4a93640 1 -- 172.21.15.133:0/2907695832 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e006d4b0 con 0x7f42e006e4d0 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e5a95640 1 --2- 172.21.15.133:0/2907695832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42e00cdd60 0x7f42e015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.817+0000 7f42e5294640 1 --2- 172.21.15.133:0/2907695832 >> v2:172.21.15.162:3300/0 conn(0x7f42e0145170 0x7f42e006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e4a93640 1 --2- 172.21.15.133:0/2907695832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e006e4d0 0x7f42e00cd820 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f42e006b790 tx=0x7f42d001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfbea9113ddb6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42d7fff640 1 -- 172.21.15.133:0/2907695832 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00b0070 con 0x7f42e006e4d0 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42d7fff640 1 -- 172.21.15.133:0/2907695832 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d00ab600 con 0x7f42e006e4d0 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42d7fff640 1 -- 172.21.15.133:0/2907695832 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00ab9d0 con 0x7f42e006e4d0 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e74f8640 1 -- 172.21.15.133:0/2907695832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e006e4d0 msgr2=0x7f42e00cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e74f8640 1 --2- 172.21.15.133:0/2907695832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e006e4d0 0x7f42e00cd820 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f42e006b790 tx=0x7f42d001c520 comp rx=0 tx=0).stop 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e74f8640 1 -- 172.21.15.133:0/2907695832 shutdown_connections 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e74f8640 1 --2- 172.21.15.133:0/2907695832 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42e00cdd60 0x7f42e015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e74f8640 1 --2- 172.21.15.133:0/2907695832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e006e4d0 0x7f42e00cd820 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e74f8640 1 --2- 172.21.15.133:0/2907695832 >> v2:172.21.15.162:3300/0 conn(0x7f42e0145170 0x7f42e006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e74f8640 1 -- 172.21.15.133:0/2907695832 >> 172.21.15.133:0/2907695832 conn(0x7f42e0066f70 msgr2=0x7f42e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e74f8640 1 -- 172.21.15.133:0/2907695832 shutdown_connections 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e74f8640 1 -- 172.21.15.133:0/2907695832 wait complete. 2024-04-25T10:16:10.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e74f8640 1 Processor -- start 2024-04-25T10:16:10.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.821+0000 7f42e74f8640 1 -- start start 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e74f8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42e006e4d0 0x7f42e0134db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e74f8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42e00cdd60 0x7f42e0140d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e74f8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e0145170 0x7f42e0141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e74f8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e015f690 con 0x7f42e0145170 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e74f8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e015f510 con 0x7f42e00cdd60 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e74f8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e015f810 con 0x7f42e006e4d0 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5a95640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e0145170 0x7f42e0141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5a95640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e0145170 0x7f42e0141270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55692/0 (socket says 172.21.15.133:55692) 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5a95640 1 -- 172.21.15.133:0/4211417129 learned_addr learned my addr 172.21.15.133:0/4211417129 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e4a93640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42e00cdd60 0x7f42e0140d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5294640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42e006e4d0 0x7f42e0134db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5a95640 1 -- 172.21.15.133:0/4211417129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42e006e4d0 msgr2=0x7f42e0134db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5a95640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42e006e4d0 0x7f42e0134db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5a95640 1 -- 172.21.15.133:0/4211417129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42e00cdd60 msgr2=0x7f42e0140d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5a95640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42e00cdd60 0x7f42e0140d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5a95640 1 -- 172.21.15.133:0/4211417129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e01f7970 con 0x7f42e0145170 2024-04-25T10:16:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e4a93640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42e00cdd60 0x7f42e0140d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5a95640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e0145170 0x7f42e0141270 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f42d800cef0 tx=0x7f42d800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42d5ffb640 1 -- 172.21.15.133:0/4211417129 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d801a070 con 0x7f42e0145170 2024-04-25T10:16:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42d5ffb640 1 -- 172.21.15.133:0/4211417129 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d800d840 con 0x7f42e0145170 2024-04-25T10:16:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e74f8640 1 -- 172.21.15.133:0/4211417129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e01f7c50 con 0x7f42e0145170 2024-04-25T10:16:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42d5ffb640 1 -- 172.21.15.133:0/4211417129 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d800dc30 con 0x7f42e0145170 2024-04-25T10:16:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e74f8640 1 -- 172.21.15.133:0/4211417129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e0141f80 con 0x7f42e0145170 2024-04-25T10:16:10.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e74f8640 1 -- 172.21.15.133:0/4211417129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e0068e80 con 0x7f42e0145170 2024-04-25T10:16:10.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42d5ffb640 1 -- 172.21.15.133:0/4211417129 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42d801d070 con 0x7f42e0145170 2024-04-25T10:16:10.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42d5ffb640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f42c4066f60 0x7f42c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42d5ffb640 1 -- 172.21.15.133:0/4211417129 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f42d8085960 con 0x7f42e0145170 2024-04-25T10:16:10.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5294640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f42c4066f60 0x7f42c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.825+0000 7f42e5294640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f42c4066f60 0x7f42c4069420 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f42dc006fd0 tx=0x7f42dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:10.829+0000 7f42d5ffb640 1 -- 172.21.15.133:0/4211417129 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42d8056870 con 0x7f42e0145170 2024-04-25T10:16:11.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.013+0000 7f42e74f8640 1 -- 172.21.15.133:0/4211417129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f42e0161e70 con 0x7f42e0145170 2024-04-25T10:16:11.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.693+0000 7f42d5ffb640 1 -- 172.21.15.133:0/4211417129 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v105) v1 ==== 122+0+0 (secure 0 0 0) 0x7f42d8056170 con 0x7f42e0145170 2024-04-25T10:16:11.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 -- 172.21.15.133:0/4211417129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f42c4066f60 msgr2=0x7f42c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f42c4066f60 0x7f42c4069420 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f42dc006fd0 tx=0x7f42dc008040 comp rx=0 tx=0).stop 2024-04-25T10:16:11.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 -- 172.21.15.133:0/4211417129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e0145170 msgr2=0x7f42e0141270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e0145170 0x7f42e0141270 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f42d800cef0 tx=0x7f42d800a710 comp rx=0 tx=0).stop 2024-04-25T10:16:11.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 -- 172.21.15.133:0/4211417129 shutdown_connections 2024-04-25T10:16:11.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f42c4066f60 0x7f42c4069420 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42e0145170 0x7f42e0141270 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42e00cdd60 0x7f42e0140d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 --2- 172.21.15.133:0/4211417129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f42e006e4d0 0x7f42e0134db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 -- 172.21.15.133:0/4211417129 >> 172.21.15.133:0/4211417129 conn(0x7f42e0066f70 msgr2=0x7f42e013e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 -- 172.21.15.133:0/4211417129 shutdown_connections 2024-04-25T10:16:11.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.697+0000 7f42e74f8640 1 -- 172.21.15.133:0/4211417129 wait complete. 2024-04-25T10:16:11.720 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:16:11.720 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.845+0000 7fb68f8e5640 1 Processor -- start 2024-04-25T10:16:11.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.845+0000 7fb68f8e5640 1 -- start start 2024-04-25T10:16:11.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68f8e5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb68815c9a0 0x7fb6880c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68f8e5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6880ce330 0x7fb6880ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68f8e5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6880c61d0 con 0x7fb6880b1f70 2024-04-25T10:16:11.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68f8e5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6880cecc0 con 0x7fb6880ce330 2024-04-25T10:16:11.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68f8e5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6880c64b0 con 0x7fb68815c9a0 2024-04-25T10:16:11.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68de82640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6880ce330 0x7fb6880ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68de82640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6880ce330 0x7fb6880ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55702/0 (socket says 172.21.15.133:55702) 2024-04-25T10:16:11.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68de82640 1 -- 172.21.15.133:0/3185833971 learned_addr learned my addr 172.21.15.133:0/3185833971 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:11.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68d681640 1 --2- 172.21.15.133:0/3185833971 >> v2:172.21.15.162:3300/0 conn(0x7fb68815c9a0 0x7fb6880c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68de82640 1 -- 172.21.15.133:0/3185833971 >> v2:172.21.15.162:3300/0 conn(0x7fb68815c9a0 msgr2=0x7fb6880c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:11.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68de82640 1 --2- 172.21.15.133:0/3185833971 >> v2:172.21.15.162:3300/0 conn(0x7fb68815c9a0 0x7fb6880c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68de82640 1 -- 172.21.15.133:0/3185833971 >> v1:172.21.15.162:6789/0 conn(0x7fb6880b1f70 legacy=0x7fb68806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68de82640 1 -- 172.21.15.133:0/3185833971 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68806d4f0 con 0x7fb6880ce330 2024-04-25T10:16:11.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb68de82640 1 --2- 172.21.15.133:0/3185833971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6880ce330 0x7fb6880ce780 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fb68806f5f0 tx=0x7fb68001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6083c969d62ae03d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb67ffff640 1 -- 172.21.15.133:0/3185833971 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6800a56c0 con 0x7fb6880ce330 2024-04-25T10:16:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb67ffff640 1 -- 172.21.15.133:0/3185833971 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6800a5860 con 0x7fb6880ce330 2024-04-25T10:16:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.849+0000 7fb67ffff640 1 -- 172.21.15.133:0/3185833971 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6800a5c30 con 0x7fb6880ce330 2024-04-25T10:16:11.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- 172.21.15.133:0/3185833971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6880ce330 msgr2=0x7fb6880ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 --2- 172.21.15.133:0/3185833971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6880ce330 0x7fb6880ce780 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fb68806f5f0 tx=0x7fb68001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- 172.21.15.133:0/3185833971 shutdown_connections 2024-04-25T10:16:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 --2- 172.21.15.133:0/3185833971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6880ce330 0x7fb6880ce780 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 --2- 172.21.15.133:0/3185833971 >> v2:172.21.15.162:3300/0 conn(0x7fb68815c9a0 0x7fb6880c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- 172.21.15.133:0/3185833971 >> 172.21.15.133:0/3185833971 conn(0x7fb688067000 msgr2=0x7fb68806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- 172.21.15.133:0/3185833971 shutdown_connections 2024-04-25T10:16:11.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- 172.21.15.133:0/3185833971 wait complete. 2024-04-25T10:16:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 Processor -- start 2024-04-25T10:16:11.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- start start 2024-04-25T10:16:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6880b1f70 0x7fb6881656d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6880ce330 0x7fb68814eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb68815c9a0 0x7fb688164960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68815dad0 con 0x7fb6880ce330 2024-04-25T10:16:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68815d950 con 0x7fb6880b1f70 2024-04-25T10:16:11.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68815dc50 con 0x7fb68815c9a0 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68ce80640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6880ce330 0x7fb68814eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68ce80640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6880ce330 0x7fb68814eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55708/0 (socket says 172.21.15.133:55708) 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68ce80640 1 -- 172.21.15.133:0/59382196 learned_addr learned my addr 172.21.15.133:0/59382196 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68d681640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6880b1f70 0x7fb6881656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68de82640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb68815c9a0 0x7fb688164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68ce80640 1 -- 172.21.15.133:0/59382196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb68815c9a0 msgr2=0x7fb688164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68ce80640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb68815c9a0 0x7fb688164960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68ce80640 1 -- 172.21.15.133:0/59382196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6880b1f70 msgr2=0x7fb6881656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68ce80640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6880b1f70 0x7fb6881656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68ce80640 1 -- 172.21.15.133:0/59382196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68814fb70 con 0x7fb6880ce330 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68ce80640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6880ce330 0x7fb68814eff0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c370 tx=0x7fb67800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb67dffb640 1 -- 172.21.15.133:0/59382196 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6780052d0 con 0x7fb6880ce330 2024-04-25T10:16:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb67dffb640 1 -- 172.21.15.133:0/59382196 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb678005470 con 0x7fb6880ce330 2024-04-25T10:16:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- 172.21.15.133:0/59382196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68814f890 con 0x7fb6880ce330 2024-04-25T10:16:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb67dffb640 1 -- 172.21.15.133:0/59382196 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678007260 con 0x7fb6880ce330 2024-04-25T10:16:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- 172.21.15.133:0/59382196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6880cdfa0 con 0x7fb6880ce330 2024-04-25T10:16:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.853+0000 7fb68f8e5640 1 -- 172.21.15.133:0/59382196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68806bc20 con 0x7fb6880ce330 2024-04-25T10:16:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.857+0000 7fb67dffb640 1 -- 172.21.15.133:0/59382196 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb678007400 con 0x7fb6880ce330 2024-04-25T10:16:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.857+0000 7fb67dffb640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb66c066f60 0x7fb66c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.857+0000 7fb67dffb640 1 -- 172.21.15.133:0/59382196 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb6780849d0 con 0x7fb6880ce330 2024-04-25T10:16:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.857+0000 7fb68d681640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb66c066f60 0x7fb66c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.857+0000 7fb68d681640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb66c066f60 0x7fb66c069420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb6881f97c0 tx=0x7fb684008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:11.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:11.861+0000 7fb67dffb640 1 -- 172.21.15.133:0/59382196 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6780559c0 con 0x7fb6880ce330 2024-04-25T10:16:12.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.037+0000 7fb68f8e5640 1 -- 172.21.15.133:0/59382196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6880cd830 con 0x7fb6880ce330 2024-04-25T10:16:12.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.037+0000 7fb67dffb640 1 -- 172.21.15.133:0/59382196 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4962 (secure 0 0 0) 0x7fb6780552c0 con 0x7fb6880ce330 2024-04-25T10:16:12.038 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:12.038 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1498957356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1498957356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1498957356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/883424012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":883424012},{"type":"v1","addr":"172.21.15.133:6835","nonce":883424012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2040533186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2040533186},{"type":"v1","addr":"172.21.15.133:6837","nonce":2040533186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:08.983208+0000","modified":"2024-04-25T10:16:11.017848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11037":{"gid":11037,"name":"d","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/2658062772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2658062772},{"type":"v1","addr":"172.21.15.162:6833","nonce":2658062772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:16:12.039 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 105 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 -- 172.21.15.133:0/59382196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb66c066f60 msgr2=0x7fb66c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb66c066f60 0x7fb66c069420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb6881f97c0 tx=0x7fb684008040 comp rx=0 tx=0).stop 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 -- 172.21.15.133:0/59382196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6880ce330 msgr2=0x7fb68814eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6880ce330 0x7fb68814eff0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c370 tx=0x7fb67800c830 comp rx=0 tx=0).stop 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 -- 172.21.15.133:0/59382196 shutdown_connections 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb68815c9a0 0x7fb688164960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb66c066f60 0x7fb66c069420 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6880ce330 0x7fb68814eff0 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 --2- 172.21.15.133:0/59382196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6880b1f70 0x7fb6881656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 -- 172.21.15.133:0/59382196 >> 172.21.15.133:0/59382196 conn(0x7fb688067000 msgr2=0x7fb68815de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 -- 172.21.15.133:0/59382196 shutdown_connections 2024-04-25T10:16:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.041+0000 7fb68f8e5640 1 -- 172.21.15.133:0/59382196 wait complete. 2024-04-25T10:16:12.057 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbabd563640 1 Processor -- start 2024-04-25T10:16:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbabd563640 1 -- start start 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbabd563640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbab8159dc0 0x7fbab80cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbabd563640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbab815a1a0 0x7fbab815db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbabd563640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab80ce880 con 0x7fbab815e050 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbabd563640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab815e4a0 con 0x7fbab815a1a0 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbabd563640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab806e360 con 0x7fbab8159dc0 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbab67fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbab815a1a0 0x7fbab815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbab67fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbab815a1a0 0x7fbab815db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55718/0 (socket says 172.21.15.133:55718) 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbab67fc640 1 -- 172.21.15.133:0/3638832234 learned_addr learned my addr 172.21.15.133:0/3638832234 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbab67fc640 1 -- 172.21.15.133:0/3638832234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbab8159dc0 msgr2=0x7fbab80cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbab67fc640 1 --2- 172.21.15.133:0/3638832234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbab8159dc0 0x7fbab80cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbab67fc640 1 -- 172.21.15.133:0/3638832234 >> v1:172.21.15.162:6789/0 conn(0x7fbab815e050 legacy=0x7fbab80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.165+0000 7fbab67fc640 1 -- 172.21.15.133:0/3638832234 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab80c49e0 con 0x7fbab815a1a0 2024-04-25T10:16:12.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbab67fc640 1 --2- 172.21.15.133:0/3638832234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbab815a1a0 0x7fbab815db10 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fbaac006fd0 tx=0x7fbaac01e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abb758ce757e14d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbab5ffb640 1 -- 172.21.15.133:0/3638832234 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0ad070 con 0x7fbab815a1a0 2024-04-25T10:16:12.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbab5ffb640 1 -- 172.21.15.133:0/3638832234 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac004230 con 0x7fbab815a1a0 2024-04-25T10:16:12.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbab5ffb640 1 -- 172.21.15.133:0/3638832234 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac003620 con 0x7fbab815a1a0 2024-04-25T10:16:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbabd563640 1 -- 172.21.15.133:0/3638832234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbab815a1a0 msgr2=0x7fbab815db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbabd563640 1 --2- 172.21.15.133:0/3638832234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbab815a1a0 0x7fbab815db10 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fbaac006fd0 tx=0x7fbaac01e9b0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbabd563640 1 -- 172.21.15.133:0/3638832234 shutdown_connections 2024-04-25T10:16:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbabd563640 1 --2- 172.21.15.133:0/3638832234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbab815a1a0 0x7fbab815db10 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbabd563640 1 --2- 172.21.15.133:0/3638832234 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbab8159dc0 0x7fbab80cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbabd563640 1 -- 172.21.15.133:0/3638832234 >> 172.21.15.133:0/3638832234 conn(0x7fbab8067000 msgr2=0x7fbab806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbabd563640 1 -- 172.21.15.133:0/3638832234 shutdown_connections 2024-04-25T10:16:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbabd563640 1 -- 172.21.15.133:0/3638832234 wait complete. 2024-04-25T10:16:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbabd563640 1 Processor -- start 2024-04-25T10:16:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.169+0000 7fbabd563640 1 -- start start 2024-04-25T10:16:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbabd563640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbab8159dc0 0x7fbab81f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbabd563640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbab815e050 0x7fbab8148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbabd563640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbab815d490 0x7fbab81510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbabd563640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab80c47c0 con 0x7fbab815e050 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbabd563640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab80c4640 con 0x7fbab8159dc0 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbabd563640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab80c4cc0 con 0x7fbab815d490 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbab815e050 0x7fbab8148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbab815e050 0x7fbab8148910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55728/0 (socket says 172.21.15.133:55728) 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab67fc640 1 -- 172.21.15.133:0/3356133087 learned_addr learned my addr 172.21.15.133:0/3356133087 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab6ffd640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbab8159dc0 0x7fbab81f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab67fc640 1 -- 172.21.15.133:0/3356133087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbab815d490 msgr2=0x7fbab81510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab77fe640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbab815d490 0x7fbab81510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab67fc640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbab815d490 0x7fbab81510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab67fc640 1 -- 172.21.15.133:0/3356133087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbab8159dc0 msgr2=0x7fbab81f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab67fc640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbab8159dc0 0x7fbab81f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab67fc640 1 -- 172.21.15.133:0/3356133087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab8158760 con 0x7fbab815e050 2024-04-25T10:16:12.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab67fc640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbab815e050 0x7fbab8148910 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fbaac006fd0 tx=0x7fbaac01b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fba9f7fe640 1 -- 172.21.15.133:0/3356133087 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0c0070 con 0x7fbab815e050 2024-04-25T10:16:12.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fba9f7fe640 1 -- 172.21.15.133:0/3356133087 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac004030 con 0x7fbab815e050 2024-04-25T10:16:12.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbabd563640 1 -- 172.21.15.133:0/3356133087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab8158a40 con 0x7fbab815e050 2024-04-25T10:16:12.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fba9f7fe640 1 -- 172.21.15.133:0/3356133087 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0ad040 con 0x7fbab815e050 2024-04-25T10:16:12.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbabd563640 1 -- 172.21.15.133:0/3356133087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab8159000 con 0x7fbab815e050 2024-04-25T10:16:12.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbabd563640 1 -- 172.21.15.133:0/3356133087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab8068ef0 con 0x7fbab815e050 2024-04-25T10:16:12.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fba9f7fe640 1 -- 172.21.15.133:0/3356133087 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbaac0c3070 con 0x7fbab815e050 2024-04-25T10:16:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fba9f7fe640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaa4066f60 0x7fbaa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fba9f7fe640 1 -- 172.21.15.133:0/3356133087 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbaac121120 con 0x7fbab815e050 2024-04-25T10:16:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab6ffd640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaa4066f60 0x7fbaa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.173+0000 7fbab6ffd640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaa4066f60 0x7fbaa4069420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009f20 tx=0x7fbaa80094f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.177+0000 7fba9f7fe640 1 -- 172.21.15.133:0/3356133087 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbaac0185a0 con 0x7fbab815e050 2024-04-25T10:16:12.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.333+0000 7fbabd563640 1 -- 172.21.15.133:0/3356133087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbab80cc970 con 0x7fbab815e050 2024-04-25T10:16:12.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fba9f7fe640 1 -- 172.21.15.133:0/3356133087 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+16912 (secure 0 0 0) 0x7fbaac118e20 con 0x7fbab815e050 2024-04-25T10:16:12.337 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:12.337 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":169,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:16:10.641770+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:07.256836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:07.975826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:12.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 -- 172.21.15.133:0/3356133087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaa4066f60 msgr2=0x7fbaa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaa4066f60 0x7fbaa4069420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009f20 tx=0x7fbaa80094f0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 -- 172.21.15.133:0/3356133087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbab815e050 msgr2=0x7fbab8148910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbab815e050 0x7fbab8148910 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fbaac006fd0 tx=0x7fbaac01b750 comp rx=0 tx=0).stop 2024-04-25T10:16:12.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 -- 172.21.15.133:0/3356133087 shutdown_connections 2024-04-25T10:16:12.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbab815d490 0x7fbab81510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbaa4066f60 0x7fbaa4069420 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbab815e050 0x7fbab8148910 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 --2- 172.21.15.133:0/3356133087 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbab8159dc0 0x7fbab81f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 -- 172.21.15.133:0/3356133087 >> 172.21.15.133:0/3356133087 conn(0x7fbab8067000 msgr2=0x7fbab8155500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 -- 172.21.15.133:0/3356133087 shutdown_connections 2024-04-25T10:16:12.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.337+0000 7fbabd563640 1 -- 172.21.15.133:0/3356133087 wait complete. 2024-04-25T10:16:12.354 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:16:12.355 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:12.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d301c640 1 Processor -- start 2024-04-25T10:16:12.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d301c640 1 -- start start 2024-04-25T10:16:12.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d301c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb3cc15c9a0 0x7fb3cc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d301c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb3cc0b1f70 0x7fb3cc06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d301c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3cc0ce330 0x7fb3cc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d301c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc0cecc0 con 0x7fb3cc15c9a0 2024-04-25T10:16:12.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d301c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc0c61d0 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d301c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0c64b0 con 0x7fb3cc0ce330 2024-04-25T10:16:12.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d0db8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb3cc15c9a0 0x7fb3cc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d0db8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb3cc15c9a0 0x7fb3cc0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55740/0 (socket says 172.21.15.133:55740) 2024-04-25T10:16:12.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d0db8640 1 -- 172.21.15.133:0/4125417560 learned_addr learned my addr 172.21.15.133:0/4125417560 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:12.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3cbfff640 1 --2- 172.21.15.133:0/4125417560 >> v2:172.21.15.162:3300/0 conn(0x7fb3cc0b1f70 0x7fb3cc06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d15b9640 1 --2- 172.21.15.133:0/4125417560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3cc0ce330 0x7fb3cc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d0db8640 1 -- 172.21.15.133:0/4125417560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3cc0ce330 msgr2=0x7fb3cc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d0db8640 1 --2- 172.21.15.133:0/4125417560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3cc0ce330 0x7fb3cc0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d0db8640 1 -- 172.21.15.133:0/4125417560 >> v2:172.21.15.162:3300/0 conn(0x7fb3cc0b1f70 msgr2=0x7fb3cc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d0db8640 1 --2- 172.21.15.133:0/4125417560 >> v2:172.21.15.162:3300/0 conn(0x7fb3cc0b1f70 0x7fb3cc06dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3d0db8640 1 -- 172.21.15.133:0/4125417560 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc06d4f0 con 0x7fb3cc15c9a0 2024-04-25T10:16:12.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.473+0000 7fb3cbfff640 1 --2- 172.21.15.133:0/4125417560 >> v2:172.21.15.162:3300/0 conn(0x7fb3cc0b1f70 0x7fb3cc06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:12.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3d0db8640 1 --2- 172.21.15.133:0/4125417560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3cc15c9a0 0x7fb3cc0c5980 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc072670 tx=0x7fb3bc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2fa1d2ef967ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3cb7fe640 1 -- 172.21.15.133:0/4125417560 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc0b1070 con 0x7fb3cc15c9a0 2024-04-25T10:16:12.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3cb7fe640 1 -- 172.21.15.133:0/4125417560 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc0ac620 con 0x7fb3cc15c9a0 2024-04-25T10:16:12.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3cb7fe640 1 -- 172.21.15.133:0/4125417560 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc0ac9f0 con 0x7fb3cc15c9a0 2024-04-25T10:16:12.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3d301c640 1 -- 172.21.15.133:0/4125417560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3cc15c9a0 msgr2=0x7fb3cc0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3d301c640 1 --2- 172.21.15.133:0/4125417560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3cc15c9a0 0x7fb3cc0c5980 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc072670 tx=0x7fb3bc01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3d301c640 1 -- 172.21.15.133:0/4125417560 shutdown_connections 2024-04-25T10:16:12.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3d301c640 1 --2- 172.21.15.133:0/4125417560 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3cc0ce330 0x7fb3cc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3d301c640 1 --2- 172.21.15.133:0/4125417560 >> v2:172.21.15.162:3300/0 conn(0x7fb3cc0b1f70 0x7fb3cc06dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3d301c640 1 --2- 172.21.15.133:0/4125417560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3cc15c9a0 0x7fb3cc0c5980 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3d301c640 1 -- 172.21.15.133:0/4125417560 >> 172.21.15.133:0/4125417560 conn(0x7fb3cc067000 msgr2=0x7fb3cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3d301c640 1 -- 172.21.15.133:0/4125417560 shutdown_connections 2024-04-25T10:16:12.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.477+0000 7fb3d301c640 1 -- 172.21.15.133:0/4125417560 wait complete. 2024-04-25T10:16:12.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d301c640 1 Processor -- start 2024-04-25T10:16:12.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d301c640 1 -- start start 2024-04-25T10:16:12.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d301c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3cc0b1f70 0x7fb3cc1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d301c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3cc0ce330 0x7fb3cc14f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d301c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3cc15c9a0 0x7fb3cc13cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d301c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc15cf00 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d301c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc15cd80 con 0x7fb3cc15c9a0 2024-04-25T10:16:12.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d301c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc15d950 con 0x7fb3cc0ce330 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d0db8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3cc0b1f70 0x7fb3cc1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d0db8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3cc0b1f70 0x7fb3cc1569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55744/0 (socket says 172.21.15.133:55744) 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d0db8640 1 -- 172.21.15.133:0/646799047 learned_addr learned my addr 172.21.15.133:0/646799047 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3cbfff640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3cc0ce330 0x7fb3cc14f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d15b9640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3cc15c9a0 0x7fb3cc13cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d0db8640 1 -- 172.21.15.133:0/646799047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3cc0ce330 msgr2=0x7fb3cc14f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d0db8640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3cc0ce330 0x7fb3cc14f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d0db8640 1 -- 172.21.15.133:0/646799047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3cc15c9a0 msgr2=0x7fb3cc13cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d0db8640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3cc15c9a0 0x7fb3cc13cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d0db8640 1 -- 172.21.15.133:0/646799047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc14edd0 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3cbfff640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3cc0ce330 0x7fb3cc14f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d15b9640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3cc15c9a0 0x7fb3cc13cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:12.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d0db8640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3cc0b1f70 0x7fb3cc1569b0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0af040 tx=0x7fb3bc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3c97fa640 1 -- 172.21.15.133:0/646799047 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc0b1040 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3c97fa640 1 -- 172.21.15.133:0/646799047 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc0bd070 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d301c640 1 -- 172.21.15.133:0/646799047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc13d900 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3c97fa640 1 -- 172.21.15.133:0/646799047 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc0b7880 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d301c640 1 -- 172.21.15.133:0/646799047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc14f370 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.481+0000 7fb3d301c640 1 -- 172.21.15.133:0/646799047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc06bc20 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.485+0000 7fb3c97fa640 1 -- 172.21.15.133:0/646799047 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3bc0ac4e0 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.485+0000 7fb3c97fa640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3b8066f60 0x7fb3b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.485+0000 7fb3cbfff640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3b8066f60 0x7fb3b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.485+0000 7fb3c97fa640 1 -- 172.21.15.133:0/646799047 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3bc11f6c0 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.485+0000 7fb3cbfff640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3b8066f60 0x7fb3b8069420 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0006fd0 tx=0x7fb3c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.489+0000 7fb3c97fa640 1 -- 172.21.15.133:0/646799047 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3bc016110 con 0x7fb3cc0b1f70 2024-04-25T10:16:12.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.629+0000 7fb3d301c640 1 -- 172.21.15.133:0/646799047 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3cc156230 con 0x7fb3b8066f60 2024-04-25T10:16:12.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3c97fa640 1 -- 172.21.15.133:0/646799047 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252413 (secure 0 0 0) 0x7fb3cc156230 con 0x7fb3b8066f60 2024-04-25T10:16:12.652 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:12.653 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:16:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 -- 172.21.15.133:0/646799047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3b8066f60 msgr2=0x7fb3b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3b8066f60 0x7fb3b8069420 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0006fd0 tx=0x7fb3c0008040 comp rx=0 tx=0).stop 2024-04-25T10:16:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 -- 172.21.15.133:0/646799047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3cc0b1f70 msgr2=0x7fb3cc1569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3cc0b1f70 0x7fb3cc1569b0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0af040 tx=0x7fb3bc01ad30 comp rx=0 tx=0).stop 2024-04-25T10:16:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 -- 172.21.15.133:0/646799047 shutdown_connections 2024-04-25T10:16:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb3cc15c9a0 0x7fb3cc13cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb3cc0ce330 0x7fb3cc14f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb3b8066f60 0x7fb3b8069420 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 --2- 172.21.15.133:0/646799047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3cc0b1f70 0x7fb3cc1569b0 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 -- 172.21.15.133:0/646799047 >> 172.21.15.133:0/646799047 conn(0x7fb3cc067000 msgr2=0x7fb3cc0b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 -- 172.21.15.133:0/646799047 shutdown_connections 2024-04-25T10:16:12.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.649+0000 7fb3d301c640 1 -- 172.21.15.133:0/646799047 wait complete. 2024-04-25T10:16:12.668 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":596,"stamp":"2024-04-25T10:16:11.979833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":500324,"kb_used_data":17636,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754474396,"statfs":{"total":773094113280,"available":772581781504,"internally_reserved":0,"allocated":18059264,"data_stored":9077869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.360482"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.688199+0000","last_change":"2024-04-25T10:16:09.687114+0000","last_active":"2024-04-25T10:16:09.688199+0000","last_peered":"2024-04-25T10:16:09.688199+0000","last_clean":"2024-04-25T10:16:09.688199+0000","last_became_active":"2024-04-25T10:16:09.681399+0000","last_became_peered":"2024-04-25T10:16:09.681399+0000","last_unstale":"2024-04-25T10:16:09.688199+0000","last_undegraded":"2024-04-25T10:16:09.688199+0000","last_fullsized":"2024-04-25T10:16:09.688199+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:31.820766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687435+0000","last_change":"2024-04-25T10:16:09.667761+0000","last_active":"2024-04-25T10:16:09.687435+0000","last_peered":"2024-04-25T10:16:09.687435+0000","last_clean":"2024-04-25T10:16:09.687435+0000","last_became_active":"2024-04-25T10:16:09.667475+0000","last_became_peered":"2024-04-25T10:16:09.667475+0000","last_unstale":"2024-04-25T10:16:09.687435+0000","last_undegraded":"2024-04-25T10:16:09.687435+0000","last_fullsized":"2024-04-25T10:16:09.687435+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.210290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685279+0000","last_change":"2024-04-25T10:16:09.684653+0000","last_active":"2024-04-25T10:16:09.685279+0000","last_peered":"2024-04-25T10:16:09.685279+0000","last_clean":"2024-04-25T10:16:09.685279+0000","last_became_active":"2024-04-25T10:16:09.684288+0000","last_became_peered":"2024-04-25T10:16:09.684288+0000","last_unstale":"2024-04-25T10:16:09.685279+0000","last_undegraded":"2024-04-25T10:16:09.685279+0000","last_fullsized":"2024-04-25T10:16:09.685279+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:07.168646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.682410+0000","last_change":"2024-04-25T10:16:09.671709+0000","last_active":"2024-04-25T10:16:09.682410+0000","last_peered":"2024-04-25T10:16:09.682410+0000","last_clean":"2024-04-25T10:16:09.682410+0000","last_became_active":"2024-04-25T10:16:09.671403+0000","last_became_peered":"2024-04-25T10:16:09.671403+0000","last_unstale":"2024-04-25T10:16:09.682410+0000","last_undegraded":"2024-04-25T10:16:09.682410+0000","last_fullsized":"2024-04-25T10:16:09.682410+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:21.825271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.662101+0000","last_change":"2024-04-25T10:16:09.660728+0000","last_active":"2024-04-25T10:16:09.662101+0000","last_peered":"2024-04-25T10:16:09.662101+0000","last_clean":"2024-04-25T10:16:09.662101+0000","last_became_active":"2024-04-25T10:16:09.660389+0000","last_became_peered":"2024-04-25T10:16:09.660389+0000","last_unstale":"2024-04-25T10:16:09.662101+0000","last_undegraded":"2024-04-25T10:16:09.662101+0000","last_fullsized":"2024-04-25T10:16:09.662101+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:41.488428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683183+0000","last_change":"2024-04-25T10:16:09.682229+0000","last_active":"2024-04-25T10:16:09.683183+0000","last_peered":"2024-04-25T10:16:09.683183+0000","last_clean":"2024-04-25T10:16:09.683183+0000","last_became_active":"2024-04-25T10:16:09.681906+0000","last_became_peered":"2024-04-25T10:16:09.681906+0000","last_unstale":"2024-04-25T10:16:09.683183+0000","last_undegraded":"2024-04-25T10:16:09.683183+0000","last_fullsized":"2024-04-25T10:16:09.683183+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:21.825271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.674217+0000","last_change":"2024-04-25T10:16:09.673038+0000","last_active":"2024-04-25T10:16:09.674217+0000","last_peered":"2024-04-25T10:16:09.674217+0000","last_clean":"2024-04-25T10:16:09.674217+0000","last_became_active":"2024-04-25T10:16:09.672234+0000","last_became_peered":"2024-04-25T10:16:09.672234+0000","last_unstale":"2024-04-25T10:16:09.674217+0000","last_undegraded":"2024-04-25T10:16:09.674217+0000","last_fullsized":"2024-04-25T10:16:09.674217+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:15.317455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.671262+0000","last_change":"2024-04-25T10:16:09.664558+0000","last_active":"2024-04-25T10:16:09.671262+0000","last_peered":"2024-04-25T10:16:09.671262+0000","last_clean":"2024-04-25T10:16:09.671262+0000","last_became_active":"2024-04-25T10:16:09.663971+0000","last_became_peered":"2024-04-25T10:16:09.663971+0000","last_unstale":"2024-04-25T10:16:09.671262+0000","last_undegraded":"2024-04-25T10:16:09.671262+0000","last_fullsized":"2024-04-25T10:16:09.671262+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:21.421172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.689367+0000","last_change":"2024-04-25T10:16:09.688239+0000","last_active":"2024-04-25T10:16:09.689367+0000","last_peered":"2024-04-25T10:16:09.689367+0000","last_clean":"2024-04-25T10:16:09.689367+0000","last_became_active":"2024-04-25T10:16:09.687942+0000","last_became_peered":"2024-04-25T10:16:09.687942+0000","last_unstale":"2024-04-25T10:16:09.689367+0000","last_undegraded":"2024-04-25T10:16:09.689367+0000","last_fullsized":"2024-04-25T10:16:09.689367+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:01.194919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.670988+0000","last_change":"2024-04-25T10:16:09.670365+0000","last_active":"2024-04-25T10:16:09.670988+0000","last_peered":"2024-04-25T10:16:09.670988+0000","last_clean":"2024-04-25T10:16:09.670988+0000","last_became_active":"2024-04-25T10:16:09.669994+0000","last_became_peered":"2024-04-25T10:16:09.669994+0000","last_unstale":"2024-04-25T10:16:09.670988+0000","last_undegraded":"2024-04-25T10:16:09.670988+0000","last_fullsized":"2024-04-25T10:16:09.670988+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:29.226767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.663550+0000","last_change":"2024-04-25T10:16:09.659496+0000","last_active":"2024-04-25T10:16:09.663550+0000","last_peered":"2024-04-25T10:16:09.663550+0000","last_clean":"2024-04-25T10:16:09.663550+0000","last_became_active":"2024-04-25T10:16:09.659172+0000","last_became_peered":"2024-04-25T10:16:09.659172+0000","last_unstale":"2024-04-25T10:16:09.663550+0000","last_undegraded":"2024-04-25T10:16:09.663550+0000","last_fullsized":"2024-04-25T10:16:09.663550+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:47.498485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683873+0000","last_change":"2024-04-25T10:16:09.679406+0000","last_active":"2024-04-25T10:16:09.683873+0000","last_peered":"2024-04-25T10:16:09.683873+0000","last_clean":"2024-04-25T10:16:09.683873+0000","last_became_active":"2024-04-25T10:16:09.679138+0000","last_became_peered":"2024-04-25T10:16:09.679138+0000","last_unstale":"2024-04-25T10:16:09.683873+0000","last_undegraded":"2024-04-25T10:16:09.683873+0000","last_fullsized":"2024-04-25T10:16:09.683873+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:47.587222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.669595+0000","last_change":"2024-04-25T10:16:09.668009+0000","last_active":"2024-04-25T10:16:09.669595+0000","last_peered":"2024-04-25T10:16:09.669595+0000","last_clean":"2024-04-25T10:16:09.669595+0000","last_became_active":"2024-04-25T10:16:09.665994+0000","last_became_peered":"2024-04-25T10:16:09.665994+0000","last_unstale":"2024-04-25T10:16:09.669595+0000","last_undegraded":"2024-04-25T10:16:09.669595+0000","last_fullsized":"2024-04-25T10:16:09.669595+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:24.073522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.668467+0000","last_change":"2024-04-25T10:16:09.668036+0000","last_active":"2024-04-25T10:16:09.668467+0000","last_peered":"2024-04-25T10:16:09.668467+0000","last_clean":"2024-04-25T10:16:09.668467+0000","last_became_active":"2024-04-25T10:16:09.667800+0000","last_became_peered":"2024-04-25T10:16:09.667800+0000","last_unstale":"2024-04-25T10:16:09.668467+0000","last_undegraded":"2024-04-25T10:16:09.668467+0000","last_fullsized":"2024-04-25T10:16:09.668467+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:41.237468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687694+0000","last_change":"2024-04-25T10:16:09.668837+0000","last_active":"2024-04-25T10:16:09.687694+0000","last_peered":"2024-04-25T10:16:09.687694+0000","last_clean":"2024-04-25T10:16:09.687694+0000","last_became_active":"2024-04-25T10:16:09.668350+0000","last_became_peered":"2024-04-25T10:16:09.668350+0000","last_unstale":"2024-04-25T10:16:09.687694+0000","last_undegraded":"2024-04-25T10:16:09.687694+0000","last_fullsized":"2024-04-25T10:16:09.687694+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:01.211474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.663567+0000","last_change":"2024-04-25T10:16:09.661024+0000","last_active":"2024-04-25T10:16:09.663567+0000","last_peered":"2024-04-25T10:16:09.663567+0000","last_clean":"2024-04-25T10:16:09.663567+0000","last_became_active":"2024-04-25T10:16:09.660729+0000","last_became_peered":"2024-04-25T10:16:09.660729+0000","last_unstale":"2024-04-25T10:16:09.663567+0000","last_undegraded":"2024-04-25T10:16:09.663567+0000","last_fullsized":"2024-04-25T10:16:09.663567+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:35.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664227+0000","last_change":"2024-04-25T10:16:09.661977+0000","last_active":"2024-04-25T10:16:09.664227+0000","last_peered":"2024-04-25T10:16:09.664227+0000","last_clean":"2024-04-25T10:16:09.664227+0000","last_became_active":"2024-04-25T10:16:09.661717+0000","last_became_peered":"2024-04-25T10:16:09.661717+0000","last_unstale":"2024-04-25T10:16:09.664227+0000","last_undegraded":"2024-04-25T10:16:09.664227+0000","last_fullsized":"2024-04-25T10:16:09.664227+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:03.526115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.684987+0000","last_change":"2024-04-25T10:16:09.678827+0000","last_active":"2024-04-25T10:16:09.684987+0000","last_peered":"2024-04-25T10:16:09.684987+0000","last_clean":"2024-04-25T10:16:09.684987+0000","last_became_active":"2024-04-25T10:16:09.674760+0000","last_became_peered":"2024-04-25T10:16:09.674760+0000","last_unstale":"2024-04-25T10:16:09.684987+0000","last_undegraded":"2024-04-25T10:16:09.684987+0000","last_fullsized":"2024-04-25T10:16:09.684987+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:24.605384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685074+0000","last_change":"2024-04-25T10:16:09.678709+0000","last_active":"2024-04-25T10:16:09.685074+0000","last_peered":"2024-04-25T10:16:09.685074+0000","last_clean":"2024-04-25T10:16:09.685074+0000","last_became_active":"2024-04-25T10:16:09.675827+0000","last_became_peered":"2024-04-25T10:16:09.675827+0000","last_unstale":"2024-04-25T10:16:09.685074+0000","last_undegraded":"2024-04-25T10:16:09.685074+0000","last_fullsized":"2024-04-25T10:16:09.685074+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:29.879558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.670669+0000","last_change":"2024-04-25T10:16:09.669962+0000","last_active":"2024-04-25T10:16:09.670669+0000","last_peered":"2024-04-25T10:16:09.670669+0000","last_clean":"2024-04-25T10:16:09.670669+0000","last_became_active":"2024-04-25T10:16:09.669669+0000","last_became_peered":"2024-04-25T10:16:09.669669+0000","last_unstale":"2024-04-25T10:16:09.670669+0000","last_undegraded":"2024-04-25T10:16:09.670669+0000","last_fullsized":"2024-04-25T10:16:09.670669+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:44.205361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687698+0000","last_change":"2024-04-25T10:16:09.668989+0000","last_active":"2024-04-25T10:16:09.687698+0000","last_peered":"2024-04-25T10:16:09.687698+0000","last_clean":"2024-04-25T10:16:09.687698+0000","last_became_active":"2024-04-25T10:16:09.668771+0000","last_became_peered":"2024-04-25T10:16:09.668771+0000","last_unstale":"2024-04-25T10:16:09.687698+0000","last_undegraded":"2024-04-25T10:16:09.687698+0000","last_fullsized":"2024-04-25T10:16:09.687698+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:56.779556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687845+0000","last_change":"2024-04-25T10:16:09.680455+0000","last_active":"2024-04-25T10:16:09.687845+0000","last_peered":"2024-04-25T10:16:09.687845+0000","last_clean":"2024-04-25T10:16:09.687845+0000","last_became_active":"2024-04-25T10:16:09.679782+0000","last_became_peered":"2024-04-25T10:16:09.679782+0000","last_unstale":"2024-04-25T10:16:09.687845+0000","last_undegraded":"2024-04-25T10:16:09.687845+0000","last_fullsized":"2024-04-25T10:16:09.687845+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:54.967800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.678084+0000","last_change":"2024-04-25T10:16:09.671269+0000","last_active":"2024-04-25T10:16:09.678084+0000","last_peered":"2024-04-25T10:16:09.678084+0000","last_clean":"2024-04-25T10:16:09.678084+0000","last_became_active":"2024-04-25T10:16:09.670540+0000","last_became_peered":"2024-04-25T10:16:09.670540+0000","last_unstale":"2024-04-25T10:16:09.678084+0000","last_undegraded":"2024-04-25T10:16:09.678084+0000","last_fullsized":"2024-04-25T10:16:09.678084+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:44.205361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.661682+0000","last_change":"2024-04-25T10:16:09.657110+0000","last_active":"2024-04-25T10:16:09.661682+0000","last_peered":"2024-04-25T10:16:09.661682+0000","last_clean":"2024-04-25T10:16:09.661682+0000","last_became_active":"2024-04-25T10:16:09.656671+0000","last_became_peered":"2024-04-25T10:16:09.656671+0000","last_unstale":"2024-04-25T10:16:09.661682+0000","last_undegraded":"2024-04-25T10:16:09.661682+0000","last_fullsized":"2024-04-25T10:16:09.661682+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:19.724589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.674127+0000","last_change":"2024-04-25T10:16:09.673169+0000","last_active":"2024-04-25T10:16:09.674127+0000","last_peered":"2024-04-25T10:16:09.674127+0000","last_clean":"2024-04-25T10:16:09.674127+0000","last_became_active":"2024-04-25T10:16:09.672080+0000","last_became_peered":"2024-04-25T10:16:09.672080+0000","last_unstale":"2024-04-25T10:16:09.674127+0000","last_undegraded":"2024-04-25T10:16:09.674127+0000","last_fullsized":"2024-04-25T10:16:09.674127+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:13.261659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664271+0000","last_change":"2024-04-25T10:16:09.662065+0000","last_active":"2024-04-25T10:16:09.664271+0000","last_peered":"2024-04-25T10:16:09.664271+0000","last_clean":"2024-04-25T10:16:09.664271+0000","last_became_active":"2024-04-25T10:16:09.661589+0000","last_became_peered":"2024-04-25T10:16:09.661589+0000","last_unstale":"2024-04-25T10:16:09.664271+0000","last_undegraded":"2024-04-25T10:16:09.664271+0000","last_fullsized":"2024-04-25T10:16:09.664271+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:19.724589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.680258+0000","last_change":"2024-04-25T10:16:09.677095+0000","last_active":"2024-04-25T10:16:09.680258+0000","last_peered":"2024-04-25T10:16:09.680258+0000","last_clean":"2024-04-25T10:16:09.680258+0000","last_became_active":"2024-04-25T10:16:09.676578+0000","last_became_peered":"2024-04-25T10:16:09.676578+0000","last_unstale":"2024-04-25T10:16:09.680258+0000","last_undegraded":"2024-04-25T10:16:09.680258+0000","last_fullsized":"2024-04-25T10:16:09.680258+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:50.230673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.662475+0000","last_change":"2024-04-25T10:16:09.660900+0000","last_active":"2024-04-25T10:16:09.662475+0000","last_peered":"2024-04-25T10:16:09.662475+0000","last_clean":"2024-04-25T10:16:09.662475+0000","last_became_active":"2024-04-25T10:16:09.660234+0000","last_became_peered":"2024-04-25T10:16:09.660234+0000","last_unstale":"2024-04-25T10:16:09.662475+0000","last_undegraded":"2024-04-25T10:16:09.662475+0000","last_fullsized":"2024-04-25T10:16:09.662475+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:18.584380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.684776+0000","last_change":"2024-04-25T10:16:09.682493+0000","last_active":"2024-04-25T10:16:09.684776+0000","last_peered":"2024-04-25T10:16:09.684776+0000","last_clean":"2024-04-25T10:16:09.684776+0000","last_became_active":"2024-04-25T10:16:09.682209+0000","last_became_peered":"2024-04-25T10:16:09.682209+0000","last_unstale":"2024-04-25T10:16:09.684776+0000","last_undegraded":"2024-04-25T10:16:09.684776+0000","last_fullsized":"2024-04-25T10:16:09.684776+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:08.143697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664447+0000","last_change":"2024-04-25T10:16:09.663926+0000","last_active":"2024-04-25T10:16:09.664447+0000","last_peered":"2024-04-25T10:16:09.664447+0000","last_clean":"2024-04-25T10:16:09.664447+0000","last_became_active":"2024-04-25T10:16:09.663595+0000","last_became_peered":"2024-04-25T10:16:09.663595+0000","last_unstale":"2024-04-25T10:16:09.664447+0000","last_undegraded":"2024-04-25T10:16:09.664447+0000","last_fullsized":"2024-04-25T10:16:09.664447+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.292196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.667808+0000","last_change":"2024-04-25T10:16:09.666375+0000","last_active":"2024-04-25T10:16:09.667808+0000","last_peered":"2024-04-25T10:16:09.667808+0000","last_clean":"2024-04-25T10:16:09.667808+0000","last_became_active":"2024-04-25T10:16:09.665958+0000","last_became_peered":"2024-04-25T10:16:09.665958+0000","last_unstale":"2024-04-25T10:16:09.667808+0000","last_undegraded":"2024-04-25T10:16:09.667808+0000","last_fullsized":"2024-04-25T10:16:09.667808+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:17.438843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.668164+0000","last_change":"2024-04-25T10:16:09.666873+0000","last_active":"2024-04-25T10:16:09.668164+0000","last_peered":"2024-04-25T10:16:09.668164+0000","last_clean":"2024-04-25T10:16:09.668164+0000","last_became_active":"2024-04-25T10:16:09.666255+0000","last_became_peered":"2024-04-25T10:16:09.666255+0000","last_unstale":"2024-04-25T10:16:09.668164+0000","last_undegraded":"2024-04-25T10:16:09.668164+0000","last_fullsized":"2024-04-25T10:16:09.668164+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:21.421172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.680316+0000","last_change":"2024-04-25T10:16:09.677648+0000","last_active":"2024-04-25T10:16:09.680316+0000","last_peered":"2024-04-25T10:16:09.680316+0000","last_clean":"2024-04-25T10:16:09.680316+0000","last_became_active":"2024-04-25T10:16:09.677260+0000","last_became_peered":"2024-04-25T10:16:09.677260+0000","last_unstale":"2024-04-25T10:16:09.680316+0000","last_undegraded":"2024-04-25T10:16:09.680316+0000","last_fullsized":"2024-04-25T10:16:09.680316+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:21.825271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"168'4","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.684029+0000","last_change":"2024-04-25T10:16:08.659302+0000","last_active":"2024-04-25T10:16:09.684029+0000","last_peered":"2024-04-25T10:16:09.684029+0000","last_clean":"2024-04-25T10:16:09.684029+0000","last_became_active":"2024-04-25T10:16:08.658876+0000","last_became_peered":"2024-04-25T10:16:08.658876+0000","last_unstale":"2024-04-25T10:16:09.684029+0000","last_undegraded":"2024-04-25T10:16:09.684029+0000","last_fullsized":"2024-04-25T10:16:09.684029+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:20.418213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.655302+0000","last_change":"2024-04-25T10:16:09.650330+0000","last_active":"2024-04-25T10:16:09.655302+0000","last_peered":"2024-04-25T10:16:09.655302+0000","last_clean":"2024-04-25T10:16:09.655302+0000","last_became_active":"2024-04-25T10:16:09.649113+0000","last_became_peered":"2024-04-25T10:16:09.649113+0000","last_unstale":"2024-04-25T10:16:09.655302+0000","last_undegraded":"2024-04-25T10:16:09.655302+0000","last_fullsized":"2024-04-25T10:16:09.655302+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:24.034526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.680542+0000","last_change":"2024-04-25T10:16:08.647667+0000","last_active":"2024-04-25T10:16:09.680542+0000","last_peered":"2024-04-25T10:16:09.680542+0000","last_clean":"2024-04-25T10:16:09.680542+0000","last_became_active":"2024-04-25T10:16:08.647314+0000","last_became_peered":"2024-04-25T10:16:08.647314+0000","last_unstale":"2024-04-25T10:16:09.680542+0000","last_undegraded":"2024-04-25T10:16:09.680542+0000","last_fullsized":"2024-04-25T10:16:09.680542+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:14.935716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664480+0000","last_change":"2024-04-25T10:16:09.663845+0000","last_active":"2024-04-25T10:16:09.664480+0000","last_peered":"2024-04-25T10:16:09.664480+0000","last_clean":"2024-04-25T10:16:09.664480+0000","last_became_active":"2024-04-25T10:16:09.663450+0000","last_became_peered":"2024-04-25T10:16:09.663450+0000","last_unstale":"2024-04-25T10:16:09.664480+0000","last_undegraded":"2024-04-25T10:16:09.664480+0000","last_fullsized":"2024-04-25T10:16:09.664480+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:54.880074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685332+0000","last_change":"2024-04-25T10:16:08.660011+0000","last_active":"2024-04-25T10:16:09.685332+0000","last_peered":"2024-04-25T10:16:09.685332+0000","last_clean":"2024-04-25T10:16:09.685332+0000","last_became_active":"2024-04-25T10:16:08.659221+0000","last_became_peered":"2024-04-25T10:16:08.659221+0000","last_unstale":"2024-04-25T10:16:09.685332+0000","last_undegraded":"2024-04-25T10:16:09.685332+0000","last_fullsized":"2024-04-25T10:16:09.685332+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:40.572850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.660403+0000","last_change":"2024-04-25T10:16:09.657996+0000","last_active":"2024-04-25T10:16:09.660403+0000","last_peered":"2024-04-25T10:16:09.660403+0000","last_clean":"2024-04-25T10:16:09.660403+0000","last_became_active":"2024-04-25T10:16:09.657718+0000","last_became_peered":"2024-04-25T10:16:09.657718+0000","last_unstale":"2024-04-25T10:16:09.660403+0000","last_undegraded":"2024-04-25T10:16:09.660403+0000","last_fullsized":"2024-04-25T10:16:09.660403+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:51.747050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685461+0000","last_change":"2024-04-25T10:16:08.661916+0000","last_active":"2024-04-25T10:16:09.685461+0000","last_peered":"2024-04-25T10:16:09.685461+0000","last_clean":"2024-04-25T10:16:09.685461+0000","last_became_active":"2024-04-25T10:16:08.658155+0000","last_became_peered":"2024-04-25T10:16:08.658155+0000","last_unstale":"2024-04-25T10:16:09.685461+0000","last_undegraded":"2024-04-25T10:16:09.685461+0000","last_fullsized":"2024-04-25T10:16:09.685461+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:28.978672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.671642+0000","last_change":"2024-04-25T10:16:09.670313+0000","last_active":"2024-04-25T10:16:09.671642+0000","last_peered":"2024-04-25T10:16:09.671642+0000","last_clean":"2024-04-25T10:16:09.671642+0000","last_became_active":"2024-04-25T10:16:09.670046+0000","last_became_peered":"2024-04-25T10:16:09.670046+0000","last_unstale":"2024-04-25T10:16:09.671642+0000","last_undegraded":"2024-04-25T10:16:09.671642+0000","last_fullsized":"2024-04-25T10:16:09.671642+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.700394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.681337+0000","last_change":"2024-04-25T10:16:08.646440+0000","last_active":"2024-04-25T10:16:09.681337+0000","last_peered":"2024-04-25T10:16:09.681337+0000","last_clean":"2024-04-25T10:16:09.681337+0000","last_became_active":"2024-04-25T10:16:08.646012+0000","last_became_peered":"2024-04-25T10:16:08.646012+0000","last_unstale":"2024-04-25T10:16:09.681337+0000","last_undegraded":"2024-04-25T10:16:09.681337+0000","last_fullsized":"2024-04-25T10:16:09.681337+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:31.545225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687731+0000","last_change":"2024-04-25T10:16:09.669241+0000","last_active":"2024-04-25T10:16:09.687731+0000","last_peered":"2024-04-25T10:16:09.687731+0000","last_clean":"2024-04-25T10:16:09.687731+0000","last_became_active":"2024-04-25T10:16:09.668714+0000","last_became_peered":"2024-04-25T10:16:09.668714+0000","last_unstale":"2024-04-25T10:16:09.687731+0000","last_undegraded":"2024-04-25T10:16:09.687731+0000","last_fullsized":"2024-04-25T10:16:09.687731+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:05.969499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683593+0000","last_change":"2024-04-25T10:16:08.649906+0000","last_active":"2024-04-25T10:16:09.683593+0000","last_peered":"2024-04-25T10:16:09.683593+0000","last_clean":"2024-04-25T10:16:09.683593+0000","last_became_active":"2024-04-25T10:16:08.649477+0000","last_became_peered":"2024-04-25T10:16:08.649477+0000","last_unstale":"2024-04-25T10:16:09.683593+0000","last_undegraded":"2024-04-25T10:16:09.683593+0000","last_fullsized":"2024-04-25T10:16:09.683593+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:34.151109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687765+0000","last_change":"2024-04-25T10:16:09.679979+0000","last_active":"2024-04-25T10:16:09.687765+0000","last_peered":"2024-04-25T10:16:09.687765+0000","last_clean":"2024-04-25T10:16:09.687765+0000","last_became_active":"2024-04-25T10:16:09.669194+0000","last_became_peered":"2024-04-25T10:16:09.669194+0000","last_unstale":"2024-04-25T10:16:09.687765+0000","last_undegraded":"2024-04-25T10:16:09.687765+0000","last_fullsized":"2024-04-25T10:16:09.687765+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:31.274242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.661599+0000","last_change":"2024-04-25T10:16:08.664827+0000","last_active":"2024-04-25T10:16:09.661599+0000","last_peered":"2024-04-25T10:16:09.661599+0000","last_clean":"2024-04-25T10:16:09.661599+0000","last_became_active":"2024-04-25T10:16:08.664221+0000","last_became_peered":"2024-04-25T10:16:08.664221+0000","last_unstale":"2024-04-25T10:16:09.661599+0000","last_undegraded":"2024-04-25T10:16:09.661599+0000","last_fullsized":"2024-04-25T10:16:09.661599+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:53.436437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.678076+0000","last_change":"2024-04-25T10:16:09.671723+0000","last_active":"2024-04-25T10:16:09.678076+0000","last_peered":"2024-04-25T10:16:09.678076+0000","last_clean":"2024-04-25T10:16:09.678076+0000","last_became_active":"2024-04-25T10:16:09.671394+0000","last_became_peered":"2024-04-25T10:16:09.671394+0000","last_unstale":"2024-04-25T10:16:09.678076+0000","last_undegraded":"2024-04-25T10:16:09.678076+0000","last_fullsized":"2024-04-25T10:16:09.678076+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:29.226767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.658963+0000","last_change":"2024-04-25T10:16:08.645210+0000","last_active":"2024-04-25T10:16:09.658963+0000","last_peered":"2024-04-25T10:16:09.658963+0000","last_clean":"2024-04-25T10:16:09.658963+0000","last_became_active":"2024-04-25T10:16:08.644570+0000","last_became_peered":"2024-04-25T10:16:08.644570+0000","last_unstale":"2024-04-25T10:16:09.658963+0000","last_undegraded":"2024-04-25T10:16:09.658963+0000","last_fullsized":"2024-04-25T10:16:09.658963+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:06.442796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.674084+0000","last_change":"2024-04-25T10:16:09.672960+0000","last_active":"2024-04-25T10:16:09.674084+0000","last_peered":"2024-04-25T10:16:09.674084+0000","last_clean":"2024-04-25T10:16:09.674084+0000","last_became_active":"2024-04-25T10:16:09.672734+0000","last_became_peered":"2024-04-25T10:16:09.672734+0000","last_unstale":"2024-04-25T10:16:09.674084+0000","last_undegraded":"2024-04-25T10:16:09.674084+0000","last_fullsized":"2024-04-25T10:16:09.674084+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:05.018533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.657712+0000","last_change":"2024-04-25T10:16:08.642261+0000","last_active":"2024-04-25T10:16:09.657712+0000","last_peered":"2024-04-25T10:16:09.657712+0000","last_clean":"2024-04-25T10:16:09.657712+0000","last_became_active":"2024-04-25T10:16:08.641042+0000","last_became_peered":"2024-04-25T10:16:08.641042+0000","last_unstale":"2024-04-25T10:16:09.657712+0000","last_undegraded":"2024-04-25T10:16:09.657712+0000","last_fullsized":"2024-04-25T10:16:09.657712+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:53.877115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.669805+0000","last_change":"2024-04-25T10:16:09.668618+0000","last_active":"2024-04-25T10:16:09.669805+0000","last_peered":"2024-04-25T10:16:09.669805+0000","last_clean":"2024-04-25T10:16:09.669805+0000","last_became_active":"2024-04-25T10:16:09.668318+0000","last_became_peered":"2024-04-25T10:16:09.668318+0000","last_unstale":"2024-04-25T10:16:09.669805+0000","last_undegraded":"2024-04-25T10:16:09.669805+0000","last_fullsized":"2024-04-25T10:16:09.669805+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:32.090033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"168'3","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683744+0000","last_change":"2024-04-25T10:16:08.650684+0000","last_active":"2024-04-25T10:16:09.683744+0000","last_peered":"2024-04-25T10:16:09.683744+0000","last_clean":"2024-04-25T10:16:09.683744+0000","last_became_active":"2024-04-25T10:16:08.650302+0000","last_became_peered":"2024-04-25T10:16:08.650302+0000","last_unstale":"2024-04-25T10:16:09.683744+0000","last_undegraded":"2024-04-25T10:16:09.683744+0000","last_fullsized":"2024-04-25T10:16:09.683744+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:28.693060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.688160+0000","last_change":"2024-04-25T10:16:09.686754+0000","last_active":"2024-04-25T10:16:09.688160+0000","last_peered":"2024-04-25T10:16:09.688160+0000","last_clean":"2024-04-25T10:16:09.688160+0000","last_became_active":"2024-04-25T10:16:09.680912+0000","last_became_peered":"2024-04-25T10:16:09.680912+0000","last_unstale":"2024-04-25T10:16:09.688160+0000","last_undegraded":"2024-04-25T10:16:09.688160+0000","last_fullsized":"2024-04-25T10:16:09.688160+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:52.642922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"168'6","reported_seq":18,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.688029+0000","last_change":"2024-04-25T10:16:08.663000+0000","last_active":"2024-04-25T10:16:09.688029+0000","last_peered":"2024-04-25T10:16:09.688029+0000","last_clean":"2024-04-25T10:16:09.688029+0000","last_became_active":"2024-04-25T10:16:08.658822+0000","last_became_peered":"2024-04-25T10:16:08.658822+0000","last_unstale":"2024-04-25T10:16:09.688029+0000","last_undegraded":"2024-04-25T10:16:09.688029+0000","last_fullsized":"2024-04-25T10:16:09.688029+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:44.462957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.666714+0000","last_change":"2024-04-25T10:16:09.664532+0000","last_active":"2024-04-25T10:16:09.666714+0000","last_peered":"2024-04-25T10:16:09.666714+0000","last_clean":"2024-04-25T10:16:09.666714+0000","last_became_active":"2024-04-25T10:16:09.664150+0000","last_became_peered":"2024-04-25T10:16:09.664150+0000","last_unstale":"2024-04-25T10:16:09.666714+0000","last_undegraded":"2024-04-25T10:16:09.666714+0000","last_fullsized":"2024-04-25T10:16:09.666714+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:00.702346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685995+0000","last_change":"2024-04-25T10:16:08.657294+0000","last_active":"2024-04-25T10:16:09.685995+0000","last_peered":"2024-04-25T10:16:09.685995+0000","last_clean":"2024-04-25T10:16:09.685995+0000","last_became_active":"2024-04-25T10:16:08.655030+0000","last_became_peered":"2024-04-25T10:16:08.655030+0000","last_unstale":"2024-04-25T10:16:09.685995+0000","last_undegraded":"2024-04-25T10:16:09.685995+0000","last_fullsized":"2024-04-25T10:16:09.685995+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:02.771600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.661408+0000","last_change":"2024-04-25T10:16:09.660672+0000","last_active":"2024-04-25T10:16:09.661408+0000","last_peered":"2024-04-25T10:16:09.661408+0000","last_clean":"2024-04-25T10:16:09.661408+0000","last_became_active":"2024-04-25T10:16:09.660097+0000","last_became_peered":"2024-04-25T10:16:09.660097+0000","last_unstale":"2024-04-25T10:16:09.661408+0000","last_undegraded":"2024-04-25T10:16:09.661408+0000","last_fullsized":"2024-04-25T10:16:09.661408+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:56.268533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.655661+0000","last_change":"2024-04-25T10:16:08.634116+0000","last_active":"2024-04-25T10:16:09.655661+0000","last_peered":"2024-04-25T10:16:09.655661+0000","last_clean":"2024-04-25T10:16:09.655661+0000","last_became_active":"2024-04-25T10:16:08.633654+0000","last_became_peered":"2024-04-25T10:16:08.633654+0000","last_unstale":"2024-04-25T10:16:09.655661+0000","last_undegraded":"2024-04-25T10:16:09.655661+0000","last_fullsized":"2024-04-25T10:16:09.655661+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:29.647658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685170+0000","last_change":"2024-04-25T10:16:09.679744+0000","last_active":"2024-04-25T10:16:09.685170+0000","last_peered":"2024-04-25T10:16:09.685170+0000","last_clean":"2024-04-25T10:16:09.685170+0000","last_became_active":"2024-04-25T10:16:09.676445+0000","last_became_peered":"2024-04-25T10:16:09.676445+0000","last_unstale":"2024-04-25T10:16:09.685170+0000","last_undegraded":"2024-04-25T10:16:09.685170+0000","last_fullsized":"2024-04-25T10:16:09.685170+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:05.808729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.659392+0000","last_change":"2024-04-25T10:16:08.660720+0000","last_active":"2024-04-25T10:16:09.659392+0000","last_peered":"2024-04-25T10:16:09.659392+0000","last_clean":"2024-04-25T10:16:09.659392+0000","last_became_active":"2024-04-25T10:16:08.660299+0000","last_became_peered":"2024-04-25T10:16:08.660299+0000","last_unstale":"2024-04-25T10:16:09.659392+0000","last_undegraded":"2024-04-25T10:16:09.659392+0000","last_fullsized":"2024-04-25T10:16:09.659392+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:04.015574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.689055+0000","last_change":"2024-04-25T10:16:09.688609+0000","last_active":"2024-04-25T10:16:09.689055+0000","last_peered":"2024-04-25T10:16:09.689055+0000","last_clean":"2024-04-25T10:16:09.689055+0000","last_became_active":"2024-04-25T10:16:09.688354+0000","last_became_peered":"2024-04-25T10:16:09.688354+0000","last_unstale":"2024-04-25T10:16:09.689055+0000","last_undegraded":"2024-04-25T10:16:09.689055+0000","last_fullsized":"2024-04-25T10:16:09.689055+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:44.590356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"168'8","reported_seq":20,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685489+0000","last_change":"2024-04-25T10:16:08.641352+0000","last_active":"2024-04-25T10:16:09.685489+0000","last_peered":"2024-04-25T10:16:09.685489+0000","last_clean":"2024-04-25T10:16:09.685489+0000","last_became_active":"2024-04-25T10:16:08.640634+0000","last_became_peered":"2024-04-25T10:16:08.640634+0000","last_unstale":"2024-04-25T10:16:09.685489+0000","last_undegraded":"2024-04-25T10:16:09.685489+0000","last_fullsized":"2024-04-25T10:16:09.685489+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:23.431546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.671369+0000","last_change":"2024-04-25T10:16:09.666851+0000","last_active":"2024-04-25T10:16:09.671369+0000","last_peered":"2024-04-25T10:16:09.671369+0000","last_clean":"2024-04-25T10:16:09.671369+0000","last_became_active":"2024-04-25T10:16:09.665403+0000","last_became_peered":"2024-04-25T10:16:09.665403+0000","last_unstale":"2024-04-25T10:16:09.671369+0000","last_undegraded":"2024-04-25T10:16:09.671369+0000","last_fullsized":"2024-04-25T10:16:09.671369+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:58.561617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683791+0000","last_change":"2024-04-25T10:16:08.643698+0000","last_active":"2024-04-25T10:16:09.683791+0000","last_peered":"2024-04-25T10:16:09.683791+0000","last_clean":"2024-04-25T10:16:09.683791+0000","last_became_active":"2024-04-25T10:16:08.642974+0000","last_became_peered":"2024-04-25T10:16:08.642974+0000","last_unstale":"2024-04-25T10:16:09.683791+0000","last_undegraded":"2024-04-25T10:16:09.683791+0000","last_fullsized":"2024-04-25T10:16:09.683791+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:02.927081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.667516+0000","last_change":"2024-04-25T10:16:09.666603+0000","last_active":"2024-04-25T10:16:09.667516+0000","last_peered":"2024-04-25T10:16:09.667516+0000","last_clean":"2024-04-25T10:16:09.667516+0000","last_became_active":"2024-04-25T10:16:09.666280+0000","last_became_peered":"2024-04-25T10:16:09.666280+0000","last_unstale":"2024-04-25T10:16:09.667516+0000","last_undegraded":"2024-04-25T10:16:09.667516+0000","last_fullsized":"2024-04-25T10:16:09.667516+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:29.696019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685538+0000","last_change":"2024-04-25T10:16:08.660345+0000","last_active":"2024-04-25T10:16:09.685538+0000","last_peered":"2024-04-25T10:16:09.685538+0000","last_clean":"2024-04-25T10:16:09.685538+0000","last_became_active":"2024-04-25T10:16:08.660066+0000","last_became_peered":"2024-04-25T10:16:08.660066+0000","last_unstale":"2024-04-25T10:16:09.685538+0000","last_undegraded":"2024-04-25T10:16:09.685538+0000","last_fullsized":"2024-04-25T10:16:09.685538+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:17.581421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664249+0000","last_change":"2024-04-25T10:16:09.661838+0000","last_active":"2024-04-25T10:16:09.664249+0000","last_peered":"2024-04-25T10:16:09.664249+0000","last_clean":"2024-04-25T10:16:09.664249+0000","last_became_active":"2024-04-25T10:16:09.661469+0000","last_became_peered":"2024-04-25T10:16:09.661469+0000","last_unstale":"2024-04-25T10:16:09.664249+0000","last_undegraded":"2024-04-25T10:16:09.664249+0000","last_fullsized":"2024-04-25T10:16:09.664249+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:30.650616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.682058+0000","last_change":"2024-04-25T10:16:08.647360+0000","last_active":"2024-04-25T10:16:09.682058+0000","last_peered":"2024-04-25T10:16:09.682058+0000","last_clean":"2024-04-25T10:16:09.682058+0000","last_became_active":"2024-04-25T10:16:08.646610+0000","last_became_peered":"2024-04-25T10:16:08.646610+0000","last_unstale":"2024-04-25T10:16:09.682058+0000","last_undegraded":"2024-04-25T10:16:09.682058+0000","last_fullsized":"2024-04-25T10:16:09.682058+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:07.439112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.688115+0000","last_change":"2024-04-25T10:16:09.680473+0000","last_active":"2024-04-25T10:16:09.688115+0000","last_peered":"2024-04-25T10:16:09.688115+0000","last_clean":"2024-04-25T10:16:09.688115+0000","last_became_active":"2024-04-25T10:16:09.680151+0000","last_became_peered":"2024-04-25T10:16:09.680151+0000","last_unstale":"2024-04-25T10:16:09.688115+0000","last_undegraded":"2024-04-25T10:16:09.688115+0000","last_fullsized":"2024-04-25T10:16:09.688115+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:15.368473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.653279+0000","last_change":"2024-04-25T10:16:08.649787+0000","last_active":"2024-04-25T10:16:09.653279+0000","last_peered":"2024-04-25T10:16:09.653279+0000","last_clean":"2024-04-25T10:16:09.653279+0000","last_became_active":"2024-04-25T10:16:08.649013+0000","last_became_peered":"2024-04-25T10:16:08.649013+0000","last_unstale":"2024-04-25T10:16:09.653279+0000","last_undegraded":"2024-04-25T10:16:09.653279+0000","last_fullsized":"2024-04-25T10:16:09.653279+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:47.939850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.680723+0000","last_change":"2024-04-25T10:16:09.662397+0000","last_active":"2024-04-25T10:16:09.680723+0000","last_peered":"2024-04-25T10:16:09.680723+0000","last_clean":"2024-04-25T10:16:09.680723+0000","last_became_active":"2024-04-25T10:16:09.662071+0000","last_became_peered":"2024-04-25T10:16:09.662071+0000","last_unstale":"2024-04-25T10:16:09.680723+0000","last_undegraded":"2024-04-25T10:16:09.680723+0000","last_fullsized":"2024-04-25T10:16:09.680723+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.292196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685685+0000","last_change":"2024-04-25T10:16:08.659010+0000","last_active":"2024-04-25T10:16:09.685685+0000","last_peered":"2024-04-25T10:16:09.685685+0000","last_clean":"2024-04-25T10:16:09.685685+0000","last_became_active":"2024-04-25T10:16:08.657905+0000","last_became_peered":"2024-04-25T10:16:08.657905+0000","last_unstale":"2024-04-25T10:16:09.685685+0000","last_undegraded":"2024-04-25T10:16:09.685685+0000","last_fullsized":"2024-04-25T10:16:09.685685+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:02.369700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.682185+0000","last_change":"2024-04-25T10:16:09.672257+0000","last_active":"2024-04-25T10:16:09.682185+0000","last_peered":"2024-04-25T10:16:09.682185+0000","last_clean":"2024-04-25T10:16:09.682185+0000","last_became_active":"2024-04-25T10:16:09.671973+0000","last_became_peered":"2024-04-25T10:16:09.671973+0000","last_unstale":"2024-04-25T10:16:09.682185+0000","last_undegraded":"2024-04-25T10:16:09.682185+0000","last_fullsized":"2024-04-25T10:16:09.682185+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:53.185119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.650116+0000","last_change":"2024-04-25T10:16:08.658908+0000","last_active":"2024-04-25T10:16:09.650116+0000","last_peered":"2024-04-25T10:16:09.650116+0000","last_clean":"2024-04-25T10:16:09.650116+0000","last_became_active":"2024-04-25T10:16:08.658614+0000","last_became_peered":"2024-04-25T10:16:08.658614+0000","last_unstale":"2024-04-25T10:16:09.650116+0000","last_undegraded":"2024-04-25T10:16:09.650116+0000","last_fullsized":"2024-04-25T10:16:09.650116+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:02.523156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.659904+0000","last_change":"2024-04-25T10:16:09.654012+0000","last_active":"2024-04-25T10:16:09.659904+0000","last_peered":"2024-04-25T10:16:09.659904+0000","last_clean":"2024-04-25T10:16:09.659904+0000","last_became_active":"2024-04-25T10:16:09.653563+0000","last_became_peered":"2024-04-25T10:16:09.653563+0000","last_unstale":"2024-04-25T10:16:09.659904+0000","last_undegraded":"2024-04-25T10:16:09.659904+0000","last_fullsized":"2024-04-25T10:16:09.659904+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:29.226767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.646966+0000","last_change":"2024-04-25T10:16:08.653087+0000","last_active":"2024-04-25T10:16:09.646966+0000","last_peered":"2024-04-25T10:16:09.646966+0000","last_clean":"2024-04-25T10:16:09.646966+0000","last_became_active":"2024-04-25T10:16:08.652267+0000","last_became_peered":"2024-04-25T10:16:08.652267+0000","last_unstale":"2024-04-25T10:16:09.646966+0000","last_undegraded":"2024-04-25T10:16:09.646966+0000","last_fullsized":"2024-04-25T10:16:09.646966+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:49.884625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.678681+0000","last_change":"2024-04-25T10:16:09.674875+0000","last_active":"2024-04-25T10:16:09.678681+0000","last_peered":"2024-04-25T10:16:09.678681+0000","last_clean":"2024-04-25T10:16:09.678681+0000","last_became_active":"2024-04-25T10:16:09.674500+0000","last_became_peered":"2024-04-25T10:16:09.674500+0000","last_unstale":"2024-04-25T10:16:09.678681+0000","last_undegraded":"2024-04-25T10:16:09.678681+0000","last_fullsized":"2024-04-25T10:16:09.678681+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:50.230673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.649613+0000","last_change":"2024-04-25T10:16:08.651364+0000","last_active":"2024-04-25T10:16:09.649613+0000","last_peered":"2024-04-25T10:16:09.649613+0000","last_clean":"2024-04-25T10:16:09.649613+0000","last_became_active":"2024-04-25T10:16:08.650588+0000","last_became_peered":"2024-04-25T10:16:08.650588+0000","last_unstale":"2024-04-25T10:16:09.649613+0000","last_undegraded":"2024-04-25T10:16:09.649613+0000","last_fullsized":"2024-04-25T10:16:09.649613+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:02.718326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.671114+0000","last_change":"2024-04-25T10:16:09.667103+0000","last_active":"2024-04-25T10:16:09.671114+0000","last_peered":"2024-04-25T10:16:09.671114+0000","last_clean":"2024-04-25T10:16:09.671114+0000","last_became_active":"2024-04-25T10:16:09.666267+0000","last_became_peered":"2024-04-25T10:16:09.666267+0000","last_unstale":"2024-04-25T10:16:09.671114+0000","last_undegraded":"2024-04-25T10:16:09.671114+0000","last_fullsized":"2024-04-25T10:16:09.671114+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:38.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683886+0000","last_change":"2024-04-25T10:16:08.651567+0000","last_active":"2024-04-25T10:16:09.683886+0000","last_peered":"2024-04-25T10:16:09.683886+0000","last_clean":"2024-04-25T10:16:09.683886+0000","last_became_active":"2024-04-25T10:16:08.650547+0000","last_became_peered":"2024-04-25T10:16:08.650547+0000","last_unstale":"2024-04-25T10:16:09.683886+0000","last_undegraded":"2024-04-25T10:16:09.683886+0000","last_fullsized":"2024-04-25T10:16:09.683886+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:43.202403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687469+0000","last_change":"2024-04-25T10:16:09.667922+0000","last_active":"2024-04-25T10:16:09.687469+0000","last_peered":"2024-04-25T10:16:09.687469+0000","last_clean":"2024-04-25T10:16:09.687469+0000","last_became_active":"2024-04-25T10:16:09.662755+0000","last_became_peered":"2024-04-25T10:16:09.662755+0000","last_unstale":"2024-04-25T10:16:09.687469+0000","last_undegraded":"2024-04-25T10:16:09.687469+0000","last_fullsized":"2024-04-25T10:16:09.687469+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:14.638281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.646615+0000","last_change":"2024-04-25T10:16:08.655603+0000","last_active":"2024-04-25T10:16:09.646615+0000","last_peered":"2024-04-25T10:16:09.646615+0000","last_clean":"2024-04-25T10:16:09.646615+0000","last_became_active":"2024-04-25T10:16:08.655227+0000","last_became_peered":"2024-04-25T10:16:08.655227+0000","last_unstale":"2024-04-25T10:16:09.646615+0000","last_undegraded":"2024-04-25T10:16:09.646615+0000","last_fullsized":"2024-04-25T10:16:09.646615+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:55.635776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664542+0000","last_change":"2024-04-25T10:16:09.662444+0000","last_active":"2024-04-25T10:16:09.664542+0000","last_peered":"2024-04-25T10:16:09.664542+0000","last_clean":"2024-04-25T10:16:09.664542+0000","last_became_active":"2024-04-25T10:16:09.662108+0000","last_became_peered":"2024-04-25T10:16:09.662108+0000","last_unstale":"2024-04-25T10:16:09.664542+0000","last_undegraded":"2024-04-25T10:16:09.664542+0000","last_fullsized":"2024-04-25T10:16:09.664542+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:29.696019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.659201+0000","last_change":"2024-04-25T10:16:08.651246+0000","last_active":"2024-04-25T10:16:09.659201+0000","last_peered":"2024-04-25T10:16:09.659201+0000","last_clean":"2024-04-25T10:16:09.659201+0000","last_became_active":"2024-04-25T10:16:08.650658+0000","last_became_peered":"2024-04-25T10:16:08.650658+0000","last_unstale":"2024-04-25T10:16:09.659201+0000","last_undegraded":"2024-04-25T10:16:09.659201+0000","last_fullsized":"2024-04-25T10:16:09.659201+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:07.522631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.667744+0000","last_change":"2024-04-25T10:16:09.665694+0000","last_active":"2024-04-25T10:16:09.667744+0000","last_peered":"2024-04-25T10:16:09.667744+0000","last_clean":"2024-04-25T10:16:09.667744+0000","last_became_active":"2024-04-25T10:16:09.665125+0000","last_became_peered":"2024-04-25T10:16:09.665125+0000","last_unstale":"2024-04-25T10:16:09.667744+0000","last_undegraded":"2024-04-25T10:16:09.667744+0000","last_fullsized":"2024-04-25T10:16:09.667744+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:44.911985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.659413+0000","last_change":"2024-04-25T10:16:08.641461+0000","last_active":"2024-04-25T10:16:09.659413+0000","last_peered":"2024-04-25T10:16:09.659413+0000","last_clean":"2024-04-25T10:16:09.659413+0000","last_became_active":"2024-04-25T10:16:08.641065+0000","last_became_peered":"2024-04-25T10:16:08.641065+0000","last_unstale":"2024-04-25T10:16:09.659413+0000","last_undegraded":"2024-04-25T10:16:09.659413+0000","last_fullsized":"2024-04-25T10:16:09.659413+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:22.264331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.668835+0000","last_change":"2024-04-25T10:16:09.667553+0000","last_active":"2024-04-25T10:16:09.668835+0000","last_peered":"2024-04-25T10:16:09.668835+0000","last_clean":"2024-04-25T10:16:09.668835+0000","last_became_active":"2024-04-25T10:16:09.667299+0000","last_became_peered":"2024-04-25T10:16:09.667299+0000","last_unstale":"2024-04-25T10:16:09.668835+0000","last_undegraded":"2024-04-25T10:16:09.668835+0000","last_fullsized":"2024-04-25T10:16:09.668835+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:48.108484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.651309+0000","last_change":"2024-04-25T10:16:08.660279+0000","last_active":"2024-04-25T10:16:09.651309+0000","last_peered":"2024-04-25T10:16:09.651309+0000","last_clean":"2024-04-25T10:16:09.651309+0000","last_became_active":"2024-04-25T10:16:08.659536+0000","last_became_peered":"2024-04-25T10:16:08.659536+0000","last_unstale":"2024-04-25T10:16:09.651309+0000","last_undegraded":"2024-04-25T10:16:09.651309+0000","last_fullsized":"2024-04-25T10:16:09.651309+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:34.151109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.669894+0000","last_change":"2024-04-25T10:16:09.668702+0000","last_active":"2024-04-25T10:16:09.669894+0000","last_peered":"2024-04-25T10:16:09.669894+0000","last_clean":"2024-04-25T10:16:09.669894+0000","last_became_active":"2024-04-25T10:16:09.668381+0000","last_became_peered":"2024-04-25T10:16:09.668381+0000","last_unstale":"2024-04-25T10:16:09.669894+0000","last_undegraded":"2024-04-25T10:16:09.669894+0000","last_fullsized":"2024-04-25T10:16:09.669894+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:33.485775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.652359+0000","last_change":"2024-04-25T10:16:08.650806+0000","last_active":"2024-04-25T10:16:09.652359+0000","last_peered":"2024-04-25T10:16:09.652359+0000","last_clean":"2024-04-25T10:16:09.652359+0000","last_became_active":"2024-04-25T10:16:08.650004+0000","last_became_peered":"2024-04-25T10:16:08.650004+0000","last_unstale":"2024-04-25T10:16:09.652359+0000","last_undegraded":"2024-04-25T10:16:09.652359+0000","last_fullsized":"2024-04-25T10:16:09.652359+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:02.523156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687395+0000","last_change":"2024-04-25T10:16:09.663171+0000","last_active":"2024-04-25T10:16:09.687395+0000","last_peered":"2024-04-25T10:16:09.687395+0000","last_clean":"2024-04-25T10:16:09.687395+0000","last_became_active":"2024-04-25T10:16:09.662888+0000","last_became_peered":"2024-04-25T10:16:09.662888+0000","last_unstale":"2024-04-25T10:16:09.687395+0000","last_undegraded":"2024-04-25T10:16:09.687395+0000","last_fullsized":"2024-04-25T10:16:09.687395+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:08.143697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.647265+0000","last_change":"2024-04-25T10:16:08.652691+0000","last_active":"2024-04-25T10:16:09.647265+0000","last_peered":"2024-04-25T10:16:09.647265+0000","last_clean":"2024-04-25T10:16:09.647265+0000","last_became_active":"2024-04-25T10:16:08.651694+0000","last_became_peered":"2024-04-25T10:16:08.651694+0000","last_unstale":"2024-04-25T10:16:09.647265+0000","last_undegraded":"2024-04-25T10:16:09.647265+0000","last_fullsized":"2024-04-25T10:16:09.647265+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:47.276424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.680221+0000","last_change":"2024-04-25T10:16:09.672755+0000","last_active":"2024-04-25T10:16:09.680221+0000","last_peered":"2024-04-25T10:16:09.680221+0000","last_clean":"2024-04-25T10:16:09.680221+0000","last_became_active":"2024-04-25T10:16:09.672278+0000","last_became_peered":"2024-04-25T10:16:09.672278+0000","last_unstale":"2024-04-25T10:16:09.680221+0000","last_undegraded":"2024-04-25T10:16:09.680221+0000","last_fullsized":"2024-04-25T10:16:09.680221+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:05.018533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.649998+0000","last_change":"2024-04-25T10:16:08.643862+0000","last_active":"2024-04-25T10:16:09.649998+0000","last_peered":"2024-04-25T10:16:09.649998+0000","last_clean":"2024-04-25T10:16:09.649998+0000","last_became_active":"2024-04-25T10:16:08.642583+0000","last_became_peered":"2024-04-25T10:16:08.642583+0000","last_unstale":"2024-04-25T10:16:09.649998+0000","last_undegraded":"2024-04-25T10:16:09.649998+0000","last_fullsized":"2024-04-25T10:16:09.649998+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:22.320789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.667033+0000","last_change":"2024-04-25T10:16:09.666613+0000","last_active":"2024-04-25T10:16:09.667033+0000","last_peered":"2024-04-25T10:16:09.667033+0000","last_clean":"2024-04-25T10:16:09.667033+0000","last_became_active":"2024-04-25T10:16:09.663230+0000","last_became_peered":"2024-04-25T10:16:09.663230+0000","last_unstale":"2024-04-25T10:16:09.667033+0000","last_undegraded":"2024-04-25T10:16:09.667033+0000","last_fullsized":"2024-04-25T10:16:09.667033+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:05.598402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.659756+0000","last_change":"2024-04-25T10:16:08.649365+0000","last_active":"2024-04-25T10:16:09.659756+0000","last_peered":"2024-04-25T10:16:09.659756+0000","last_clean":"2024-04-25T10:16:09.659756+0000","last_became_active":"2024-04-25T10:16:08.648780+0000","last_became_peered":"2024-04-25T10:16:08.648780+0000","last_unstale":"2024-04-25T10:16:09.659756+0000","last_undegraded":"2024-04-25T10:16:09.659756+0000","last_fullsized":"2024-04-25T10:16:09.659756+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:49.227714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":449,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.650184+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:16:09.650184+0000","last_peered":"2024-04-25T10:16:09.650184+0000","last_clean":"2024-04-25T10:16:09.650184+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:16:09.650184+0000","last_undegraded":"2024-04-25T10:16:09.650184+0000","last_fullsized":"2024-04-25T10:16:09.650184+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575026,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74072,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297768,"statfs":{"total":96636764160,"available":96560914432,"internally_reserved":0,"allocated":2699264,"data_stored":1576408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51288,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320552,"statfs":{"total":96636764160,"available":96584245248,"internally_reserved":0,"allocated":2105344,"data_stored":986040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575026,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47124,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324716,"statfs":{"total":96636764160,"available":96588509184,"internally_reserved":0,"allocated":2105344,"data_stored":986040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575026,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77684,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294156,"statfs":{"total":96636764160,"available":96557215744,"internally_reserved":0,"allocated":2134016,"data_stored":994853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575026,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77652,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294188,"statfs":{"total":96636764160,"available":96557248512,"internally_reserved":0,"allocated":2105344,"data_stored":986040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575026,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51292,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320548,"statfs":{"total":96636764160,"available":96584241152,"internally_reserved":0,"allocated":2105344,"data_stored":986040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575026,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47132,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324708,"statfs":{"total":96636764160,"available":96588500992,"internally_reserved":0,"allocated":2105344,"data_stored":986040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575027,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74080,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297760,"statfs":{"total":96636764160,"available":96560906240,"internally_reserved":0,"allocated":2699264,"data_stored":1576408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:12.673 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.833+0000 7f7d8dad4640 1 Processor -- start 2024-04-25T10:16:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.833+0000 7f7d8dad4640 1 -- start start 2024-04-25T10:16:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d8dad4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d8815b9c0 0x7f7d88072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d8dad4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7d880704a0 0x7f7d880cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d8dad4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8815bda0 con 0x7f7d880ccf90 2024-04-25T10:16:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d8dad4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d880cd450 con 0x7f7d880704a0 2024-04-25T10:16:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d8dad4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d880cd730 con 0x7f7d8815b9c0 2024-04-25T10:16:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d87fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7d880704a0 0x7f7d880cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d87fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7d880704a0 0x7f7d880cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55748/0 (socket says 172.21.15.133:55748) 2024-04-25T10:16:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d87fff640 1 -- 172.21.15.133:0/1334360426 learned_addr learned my addr 172.21.15.133:0/1334360426 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d87fff640 1 -- 172.21.15.133:0/1334360426 >> v2:172.21.15.162:3300/0 conn(0x7f7d8815b9c0 msgr2=0x7f7d88072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d877fe640 1 --2- 172.21.15.133:0/1334360426 >> v2:172.21.15.162:3300/0 conn(0x7f7d8815b9c0 0x7f7d88072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d87fff640 1 --2- 172.21.15.133:0/1334360426 >> v2:172.21.15.162:3300/0 conn(0x7f7d8815b9c0 0x7f7d88072cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d87fff640 1 -- 172.21.15.133:0/1334360426 >> v1:172.21.15.162:6789/0 conn(0x7f7d880ccf90 legacy=0x7f7d8806fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d87fff640 1 -- 172.21.15.133:0/1334360426 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d880cda10 con 0x7f7d880704a0 2024-04-25T10:16:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d87fff640 1 --2- 172.21.15.133:0/1334360426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d880704a0 0x7f7d880cf950 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f7d880b3720 tx=0x7f7d8001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794f1f268dd1794e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d86ffd640 1 -- 172.21.15.133:0/1334360426 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d800a56d0 con 0x7f7d880704a0 2024-04-25T10:16:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d86ffd640 1 -- 172.21.15.133:0/1334360426 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d800a5870 con 0x7f7d880704a0 2024-04-25T10:16:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.837+0000 7f7d86ffd640 1 -- 172.21.15.133:0/1334360426 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d800a5c40 con 0x7f7d880704a0 2024-04-25T10:16:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 -- 172.21.15.133:0/1334360426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d880704a0 msgr2=0x7f7d880cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 --2- 172.21.15.133:0/1334360426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d880704a0 0x7f7d880cf950 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f7d880b3720 tx=0x7f7d8001c900 comp rx=0 tx=0).stop 2024-04-25T10:16:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 -- 172.21.15.133:0/1334360426 shutdown_connections 2024-04-25T10:16:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 --2- 172.21.15.133:0/1334360426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d880704a0 0x7f7d880cf950 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 --2- 172.21.15.133:0/1334360426 >> v2:172.21.15.162:3300/0 conn(0x7f7d8815b9c0 0x7f7d88072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 -- 172.21.15.133:0/1334360426 >> 172.21.15.133:0/1334360426 conn(0x7f7d880b2370 msgr2=0x7f7d880b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 -- 172.21.15.133:0/1334360426 shutdown_connections 2024-04-25T10:16:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 -- 172.21.15.133:0/1334360426 wait complete. 2024-04-25T10:16:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 Processor -- start 2024-04-25T10:16:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 -- start start 2024-04-25T10:16:12.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d880704a0 0x7f7d88153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d880ccf90 0x7f7d8814dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8815b9c0 0x7f7d881366a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880d0a70 con 0x7f7d8815b9c0 2024-04-25T10:16:12.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880d08f0 con 0x7f7d880ccf90 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880d0bf0 con 0x7f7d880704a0 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d87fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8815b9c0 0x7f7d881366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d87fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8815b9c0 0x7f7d881366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55752/0 (socket says 172.21.15.133:55752) 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d87fff640 1 -- 172.21.15.133:0/72063727 learned_addr learned my addr 172.21.15.133:0/72063727 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d877fe640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d880704a0 0x7f7d88153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d7ffff640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d880ccf90 0x7f7d8814dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d87fff640 1 -- 172.21.15.133:0/72063727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d880704a0 msgr2=0x7f7d88153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d87fff640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d880704a0 0x7f7d88153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d87fff640 1 -- 172.21.15.133:0/72063727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d880ccf90 msgr2=0x7f7d8814dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d87fff640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d880ccf90 0x7f7d8814dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d87fff640 1 -- 172.21.15.133:0/72063727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d881526d0 con 0x7f7d8815b9c0 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d7ffff640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d880ccf90 0x7f7d8814dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:12.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d87fff640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8815b9c0 0x7f7d881366a0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f7d8001a530 tx=0x7f7d800196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d84ff9640 1 -- 172.21.15.133:0/72063727 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d800bc020 con 0x7f7d8815b9c0 2024-04-25T10:16:12.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d84ff9640 1 -- 172.21.15.133:0/72063727 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d800b8670 con 0x7f7d8815b9c0 2024-04-25T10:16:12.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 -- 172.21.15.133:0/72063727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d881529b0 con 0x7f7d8815b9c0 2024-04-25T10:16:12.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d84ff9640 1 -- 172.21.15.133:0/72063727 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d800b8a60 con 0x7f7d8815b9c0 2024-04-25T10:16:12.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.841+0000 7f7d8dad4640 1 -- 172.21.15.133:0/72063727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d88152f70 con 0x7f7d8815b9c0 2024-04-25T10:16:12.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.845+0000 7f7d8dad4640 1 -- 172.21.15.133:0/72063727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d88067c70 con 0x7f7d8815b9c0 2024-04-25T10:16:12.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.845+0000 7f7d84ff9640 1 -- 172.21.15.133:0/72063727 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d800bf020 con 0x7f7d8815b9c0 2024-04-25T10:16:12.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.845+0000 7f7d84ff9640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d68066f60 0x7f7d68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:12.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.845+0000 7f7d84ff9640 1 -- 172.21.15.133:0/72063727 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7d80120050 con 0x7f7d8815b9c0 2024-04-25T10:16:12.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.845+0000 7f7d877fe640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d68066f60 0x7f7d68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:12.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.845+0000 7f7d877fe640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d68066f60 0x7f7d68069420 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800b800 tx=0x7f7d78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:12.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:12.849+0000 7f7d84ff9640 1 -- 172.21.15.133:0/72063727 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d800c6030 con 0x7f7d8815b9c0 2024-04-25T10:16:13.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.005+0000 7f7d8dad4640 1 -- 172.21.15.133:0/72063727 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d880af530 con 0x7f7d68066f60 2024-04-25T10:16:13.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.013+0000 7f7d84ff9640 1 -- 172.21.15.133:0/72063727 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252413 (secure 0 0 0) 0x7f7d880af530 con 0x7f7d68066f60 2024-04-25T10:16:13.018 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:13.019 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:16:13.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 -- 172.21.15.133:0/72063727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d68066f60 msgr2=0x7f7d68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d68066f60 0x7f7d68069420 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800b800 tx=0x7f7d78008040 comp rx=0 tx=0).stop 2024-04-25T10:16:13.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 -- 172.21.15.133:0/72063727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8815b9c0 msgr2=0x7f7d881366a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8815b9c0 0x7f7d881366a0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f7d8001a530 tx=0x7f7d800196a0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 -- 172.21.15.133:0/72063727 shutdown_connections 2024-04-25T10:16:13.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d68066f60 0x7f7d68069420 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8815b9c0 0x7f7d881366a0 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d880ccf90 0x7f7d8814dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 --2- 172.21.15.133:0/72063727 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d880704a0 0x7f7d88153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 -- 172.21.15.133:0/72063727 >> 172.21.15.133:0/72063727 conn(0x7f7d880b2370 msgr2=0x7f7d8815bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 -- 172.21.15.133:0/72063727 shutdown_connections 2024-04-25T10:16:13.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.017+0000 7f7d8dad4640 1 -- 172.21.15.133:0/72063727 wait complete. 2024-04-25T10:16:13.037 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":596,"stamp":"2024-04-25T10:16:11.979833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":500324,"kb_used_data":17636,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754474396,"statfs":{"total":773094113280,"available":772581781504,"internally_reserved":0,"allocated":18059264,"data_stored":9077869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.360482"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.688199+0000","last_change":"2024-04-25T10:16:09.687114+0000","last_active":"2024-04-25T10:16:09.688199+0000","last_peered":"2024-04-25T10:16:09.688199+0000","last_clean":"2024-04-25T10:16:09.688199+0000","last_became_active":"2024-04-25T10:16:09.681399+0000","last_became_peered":"2024-04-25T10:16:09.681399+0000","last_unstale":"2024-04-25T10:16:09.688199+0000","last_undegraded":"2024-04-25T10:16:09.688199+0000","last_fullsized":"2024-04-25T10:16:09.688199+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:31.820766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687435+0000","last_change":"2024-04-25T10:16:09.667761+0000","last_active":"2024-04-25T10:16:09.687435+0000","last_peered":"2024-04-25T10:16:09.687435+0000","last_clean":"2024-04-25T10:16:09.687435+0000","last_became_active":"2024-04-25T10:16:09.667475+0000","last_became_peered":"2024-04-25T10:16:09.667475+0000","last_unstale":"2024-04-25T10:16:09.687435+0000","last_undegraded":"2024-04-25T10:16:09.687435+0000","last_fullsized":"2024-04-25T10:16:09.687435+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.210290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685279+0000","last_change":"2024-04-25T10:16:09.684653+0000","last_active":"2024-04-25T10:16:09.685279+0000","last_peered":"2024-04-25T10:16:09.685279+0000","last_clean":"2024-04-25T10:16:09.685279+0000","last_became_active":"2024-04-25T10:16:09.684288+0000","last_became_peered":"2024-04-25T10:16:09.684288+0000","last_unstale":"2024-04-25T10:16:09.685279+0000","last_undegraded":"2024-04-25T10:16:09.685279+0000","last_fullsized":"2024-04-25T10:16:09.685279+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:07.168646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.682410+0000","last_change":"2024-04-25T10:16:09.671709+0000","last_active":"2024-04-25T10:16:09.682410+0000","last_peered":"2024-04-25T10:16:09.682410+0000","last_clean":"2024-04-25T10:16:09.682410+0000","last_became_active":"2024-04-25T10:16:09.671403+0000","last_became_peered":"2024-04-25T10:16:09.671403+0000","last_unstale":"2024-04-25T10:16:09.682410+0000","last_undegraded":"2024-04-25T10:16:09.682410+0000","last_fullsized":"2024-04-25T10:16:09.682410+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:21.825271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.662101+0000","last_change":"2024-04-25T10:16:09.660728+0000","last_active":"2024-04-25T10:16:09.662101+0000","last_peered":"2024-04-25T10:16:09.662101+0000","last_clean":"2024-04-25T10:16:09.662101+0000","last_became_active":"2024-04-25T10:16:09.660389+0000","last_became_peered":"2024-04-25T10:16:09.660389+0000","last_unstale":"2024-04-25T10:16:09.662101+0000","last_undegraded":"2024-04-25T10:16:09.662101+0000","last_fullsized":"2024-04-25T10:16:09.662101+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:41.488428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683183+0000","last_change":"2024-04-25T10:16:09.682229+0000","last_active":"2024-04-25T10:16:09.683183+0000","last_peered":"2024-04-25T10:16:09.683183+0000","last_clean":"2024-04-25T10:16:09.683183+0000","last_became_active":"2024-04-25T10:16:09.681906+0000","last_became_peered":"2024-04-25T10:16:09.681906+0000","last_unstale":"2024-04-25T10:16:09.683183+0000","last_undegraded":"2024-04-25T10:16:09.683183+0000","last_fullsized":"2024-04-25T10:16:09.683183+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:21.825271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.674217+0000","last_change":"2024-04-25T10:16:09.673038+0000","last_active":"2024-04-25T10:16:09.674217+0000","last_peered":"2024-04-25T10:16:09.674217+0000","last_clean":"2024-04-25T10:16:09.674217+0000","last_became_active":"2024-04-25T10:16:09.672234+0000","last_became_peered":"2024-04-25T10:16:09.672234+0000","last_unstale":"2024-04-25T10:16:09.674217+0000","last_undegraded":"2024-04-25T10:16:09.674217+0000","last_fullsized":"2024-04-25T10:16:09.674217+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:15.317455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.671262+0000","last_change":"2024-04-25T10:16:09.664558+0000","last_active":"2024-04-25T10:16:09.671262+0000","last_peered":"2024-04-25T10:16:09.671262+0000","last_clean":"2024-04-25T10:16:09.671262+0000","last_became_active":"2024-04-25T10:16:09.663971+0000","last_became_peered":"2024-04-25T10:16:09.663971+0000","last_unstale":"2024-04-25T10:16:09.671262+0000","last_undegraded":"2024-04-25T10:16:09.671262+0000","last_fullsized":"2024-04-25T10:16:09.671262+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:21.421172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.689367+0000","last_change":"2024-04-25T10:16:09.688239+0000","last_active":"2024-04-25T10:16:09.689367+0000","last_peered":"2024-04-25T10:16:09.689367+0000","last_clean":"2024-04-25T10:16:09.689367+0000","last_became_active":"2024-04-25T10:16:09.687942+0000","last_became_peered":"2024-04-25T10:16:09.687942+0000","last_unstale":"2024-04-25T10:16:09.689367+0000","last_undegraded":"2024-04-25T10:16:09.689367+0000","last_fullsized":"2024-04-25T10:16:09.689367+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:01.194919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.670988+0000","last_change":"2024-04-25T10:16:09.670365+0000","last_active":"2024-04-25T10:16:09.670988+0000","last_peered":"2024-04-25T10:16:09.670988+0000","last_clean":"2024-04-25T10:16:09.670988+0000","last_became_active":"2024-04-25T10:16:09.669994+0000","last_became_peered":"2024-04-25T10:16:09.669994+0000","last_unstale":"2024-04-25T10:16:09.670988+0000","last_undegraded":"2024-04-25T10:16:09.670988+0000","last_fullsized":"2024-04-25T10:16:09.670988+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:29.226767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.663550+0000","last_change":"2024-04-25T10:16:09.659496+0000","last_active":"2024-04-25T10:16:09.663550+0000","last_peered":"2024-04-25T10:16:09.663550+0000","last_clean":"2024-04-25T10:16:09.663550+0000","last_became_active":"2024-04-25T10:16:09.659172+0000","last_became_peered":"2024-04-25T10:16:09.659172+0000","last_unstale":"2024-04-25T10:16:09.663550+0000","last_undegraded":"2024-04-25T10:16:09.663550+0000","last_fullsized":"2024-04-25T10:16:09.663550+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:47.498485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683873+0000","last_change":"2024-04-25T10:16:09.679406+0000","last_active":"2024-04-25T10:16:09.683873+0000","last_peered":"2024-04-25T10:16:09.683873+0000","last_clean":"2024-04-25T10:16:09.683873+0000","last_became_active":"2024-04-25T10:16:09.679138+0000","last_became_peered":"2024-04-25T10:16:09.679138+0000","last_unstale":"2024-04-25T10:16:09.683873+0000","last_undegraded":"2024-04-25T10:16:09.683873+0000","last_fullsized":"2024-04-25T10:16:09.683873+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:47.587222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.669595+0000","last_change":"2024-04-25T10:16:09.668009+0000","last_active":"2024-04-25T10:16:09.669595+0000","last_peered":"2024-04-25T10:16:09.669595+0000","last_clean":"2024-04-25T10:16:09.669595+0000","last_became_active":"2024-04-25T10:16:09.665994+0000","last_became_peered":"2024-04-25T10:16:09.665994+0000","last_unstale":"2024-04-25T10:16:09.669595+0000","last_undegraded":"2024-04-25T10:16:09.669595+0000","last_fullsized":"2024-04-25T10:16:09.669595+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:24.073522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.668467+0000","last_change":"2024-04-25T10:16:09.668036+0000","last_active":"2024-04-25T10:16:09.668467+0000","last_peered":"2024-04-25T10:16:09.668467+0000","last_clean":"2024-04-25T10:16:09.668467+0000","last_became_active":"2024-04-25T10:16:09.667800+0000","last_became_peered":"2024-04-25T10:16:09.667800+0000","last_unstale":"2024-04-25T10:16:09.668467+0000","last_undegraded":"2024-04-25T10:16:09.668467+0000","last_fullsized":"2024-04-25T10:16:09.668467+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:41.237468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687694+0000","last_change":"2024-04-25T10:16:09.668837+0000","last_active":"2024-04-25T10:16:09.687694+0000","last_peered":"2024-04-25T10:16:09.687694+0000","last_clean":"2024-04-25T10:16:09.687694+0000","last_became_active":"2024-04-25T10:16:09.668350+0000","last_became_peered":"2024-04-25T10:16:09.668350+0000","last_unstale":"2024-04-25T10:16:09.687694+0000","last_undegraded":"2024-04-25T10:16:09.687694+0000","last_fullsized":"2024-04-25T10:16:09.687694+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:01.211474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.663567+0000","last_change":"2024-04-25T10:16:09.661024+0000","last_active":"2024-04-25T10:16:09.663567+0000","last_peered":"2024-04-25T10:16:09.663567+0000","last_clean":"2024-04-25T10:16:09.663567+0000","last_became_active":"2024-04-25T10:16:09.660729+0000","last_became_peered":"2024-04-25T10:16:09.660729+0000","last_unstale":"2024-04-25T10:16:09.663567+0000","last_undegraded":"2024-04-25T10:16:09.663567+0000","last_fullsized":"2024-04-25T10:16:09.663567+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:35.154068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664227+0000","last_change":"2024-04-25T10:16:09.661977+0000","last_active":"2024-04-25T10:16:09.664227+0000","last_peered":"2024-04-25T10:16:09.664227+0000","last_clean":"2024-04-25T10:16:09.664227+0000","last_became_active":"2024-04-25T10:16:09.661717+0000","last_became_peered":"2024-04-25T10:16:09.661717+0000","last_unstale":"2024-04-25T10:16:09.664227+0000","last_undegraded":"2024-04-25T10:16:09.664227+0000","last_fullsized":"2024-04-25T10:16:09.664227+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:03.526115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.684987+0000","last_change":"2024-04-25T10:16:09.678827+0000","last_active":"2024-04-25T10:16:09.684987+0000","last_peered":"2024-04-25T10:16:09.684987+0000","last_clean":"2024-04-25T10:16:09.684987+0000","last_became_active":"2024-04-25T10:16:09.674760+0000","last_became_peered":"2024-04-25T10:16:09.674760+0000","last_unstale":"2024-04-25T10:16:09.684987+0000","last_undegraded":"2024-04-25T10:16:09.684987+0000","last_fullsized":"2024-04-25T10:16:09.684987+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:24.605384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685074+0000","last_change":"2024-04-25T10:16:09.678709+0000","last_active":"2024-04-25T10:16:09.685074+0000","last_peered":"2024-04-25T10:16:09.685074+0000","last_clean":"2024-04-25T10:16:09.685074+0000","last_became_active":"2024-04-25T10:16:09.675827+0000","last_became_peered":"2024-04-25T10:16:09.675827+0000","last_unstale":"2024-04-25T10:16:09.685074+0000","last_undegraded":"2024-04-25T10:16:09.685074+0000","last_fullsized":"2024-04-25T10:16:09.685074+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:29.879558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.670669+0000","last_change":"2024-04-25T10:16:09.669962+0000","last_active":"2024-04-25T10:16:09.670669+0000","last_peered":"2024-04-25T10:16:09.670669+0000","last_clean":"2024-04-25T10:16:09.670669+0000","last_became_active":"2024-04-25T10:16:09.669669+0000","last_became_peered":"2024-04-25T10:16:09.669669+0000","last_unstale":"2024-04-25T10:16:09.670669+0000","last_undegraded":"2024-04-25T10:16:09.670669+0000","last_fullsized":"2024-04-25T10:16:09.670669+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:44.205361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687698+0000","last_change":"2024-04-25T10:16:09.668989+0000","last_active":"2024-04-25T10:16:09.687698+0000","last_peered":"2024-04-25T10:16:09.687698+0000","last_clean":"2024-04-25T10:16:09.687698+0000","last_became_active":"2024-04-25T10:16:09.668771+0000","last_became_peered":"2024-04-25T10:16:09.668771+0000","last_unstale":"2024-04-25T10:16:09.687698+0000","last_undegraded":"2024-04-25T10:16:09.687698+0000","last_fullsized":"2024-04-25T10:16:09.687698+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:56.779556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687845+0000","last_change":"2024-04-25T10:16:09.680455+0000","last_active":"2024-04-25T10:16:09.687845+0000","last_peered":"2024-04-25T10:16:09.687845+0000","last_clean":"2024-04-25T10:16:09.687845+0000","last_became_active":"2024-04-25T10:16:09.679782+0000","last_became_peered":"2024-04-25T10:16:09.679782+0000","last_unstale":"2024-04-25T10:16:09.687845+0000","last_undegraded":"2024-04-25T10:16:09.687845+0000","last_fullsized":"2024-04-25T10:16:09.687845+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:54.967800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.678084+0000","last_change":"2024-04-25T10:16:09.671269+0000","last_active":"2024-04-25T10:16:09.678084+0000","last_peered":"2024-04-25T10:16:09.678084+0000","last_clean":"2024-04-25T10:16:09.678084+0000","last_became_active":"2024-04-25T10:16:09.670540+0000","last_became_peered":"2024-04-25T10:16:09.670540+0000","last_unstale":"2024-04-25T10:16:09.678084+0000","last_undegraded":"2024-04-25T10:16:09.678084+0000","last_fullsized":"2024-04-25T10:16:09.678084+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:44.205361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.661682+0000","last_change":"2024-04-25T10:16:09.657110+0000","last_active":"2024-04-25T10:16:09.661682+0000","last_peered":"2024-04-25T10:16:09.661682+0000","last_clean":"2024-04-25T10:16:09.661682+0000","last_became_active":"2024-04-25T10:16:09.656671+0000","last_became_peered":"2024-04-25T10:16:09.656671+0000","last_unstale":"2024-04-25T10:16:09.661682+0000","last_undegraded":"2024-04-25T10:16:09.661682+0000","last_fullsized":"2024-04-25T10:16:09.661682+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:19.724589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.674127+0000","last_change":"2024-04-25T10:16:09.673169+0000","last_active":"2024-04-25T10:16:09.674127+0000","last_peered":"2024-04-25T10:16:09.674127+0000","last_clean":"2024-04-25T10:16:09.674127+0000","last_became_active":"2024-04-25T10:16:09.672080+0000","last_became_peered":"2024-04-25T10:16:09.672080+0000","last_unstale":"2024-04-25T10:16:09.674127+0000","last_undegraded":"2024-04-25T10:16:09.674127+0000","last_fullsized":"2024-04-25T10:16:09.674127+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:13.261659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664271+0000","last_change":"2024-04-25T10:16:09.662065+0000","last_active":"2024-04-25T10:16:09.664271+0000","last_peered":"2024-04-25T10:16:09.664271+0000","last_clean":"2024-04-25T10:16:09.664271+0000","last_became_active":"2024-04-25T10:16:09.661589+0000","last_became_peered":"2024-04-25T10:16:09.661589+0000","last_unstale":"2024-04-25T10:16:09.664271+0000","last_undegraded":"2024-04-25T10:16:09.664271+0000","last_fullsized":"2024-04-25T10:16:09.664271+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:19.724589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.680258+0000","last_change":"2024-04-25T10:16:09.677095+0000","last_active":"2024-04-25T10:16:09.680258+0000","last_peered":"2024-04-25T10:16:09.680258+0000","last_clean":"2024-04-25T10:16:09.680258+0000","last_became_active":"2024-04-25T10:16:09.676578+0000","last_became_peered":"2024-04-25T10:16:09.676578+0000","last_unstale":"2024-04-25T10:16:09.680258+0000","last_undegraded":"2024-04-25T10:16:09.680258+0000","last_fullsized":"2024-04-25T10:16:09.680258+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:50.230673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.662475+0000","last_change":"2024-04-25T10:16:09.660900+0000","last_active":"2024-04-25T10:16:09.662475+0000","last_peered":"2024-04-25T10:16:09.662475+0000","last_clean":"2024-04-25T10:16:09.662475+0000","last_became_active":"2024-04-25T10:16:09.660234+0000","last_became_peered":"2024-04-25T10:16:09.660234+0000","last_unstale":"2024-04-25T10:16:09.662475+0000","last_undegraded":"2024-04-25T10:16:09.662475+0000","last_fullsized":"2024-04-25T10:16:09.662475+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:18.584380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.684776+0000","last_change":"2024-04-25T10:16:09.682493+0000","last_active":"2024-04-25T10:16:09.684776+0000","last_peered":"2024-04-25T10:16:09.684776+0000","last_clean":"2024-04-25T10:16:09.684776+0000","last_became_active":"2024-04-25T10:16:09.682209+0000","last_became_peered":"2024-04-25T10:16:09.682209+0000","last_unstale":"2024-04-25T10:16:09.684776+0000","last_undegraded":"2024-04-25T10:16:09.684776+0000","last_fullsized":"2024-04-25T10:16:09.684776+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:08.143697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664447+0000","last_change":"2024-04-25T10:16:09.663926+0000","last_active":"2024-04-25T10:16:09.664447+0000","last_peered":"2024-04-25T10:16:09.664447+0000","last_clean":"2024-04-25T10:16:09.664447+0000","last_became_active":"2024-04-25T10:16:09.663595+0000","last_became_peered":"2024-04-25T10:16:09.663595+0000","last_unstale":"2024-04-25T10:16:09.664447+0000","last_undegraded":"2024-04-25T10:16:09.664447+0000","last_fullsized":"2024-04-25T10:16:09.664447+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.292196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.667808+0000","last_change":"2024-04-25T10:16:09.666375+0000","last_active":"2024-04-25T10:16:09.667808+0000","last_peered":"2024-04-25T10:16:09.667808+0000","last_clean":"2024-04-25T10:16:09.667808+0000","last_became_active":"2024-04-25T10:16:09.665958+0000","last_became_peered":"2024-04-25T10:16:09.665958+0000","last_unstale":"2024-04-25T10:16:09.667808+0000","last_undegraded":"2024-04-25T10:16:09.667808+0000","last_fullsized":"2024-04-25T10:16:09.667808+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:17.438843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.668164+0000","last_change":"2024-04-25T10:16:09.666873+0000","last_active":"2024-04-25T10:16:09.668164+0000","last_peered":"2024-04-25T10:16:09.668164+0000","last_clean":"2024-04-25T10:16:09.668164+0000","last_became_active":"2024-04-25T10:16:09.666255+0000","last_became_peered":"2024-04-25T10:16:09.666255+0000","last_unstale":"2024-04-25T10:16:09.668164+0000","last_undegraded":"2024-04-25T10:16:09.668164+0000","last_fullsized":"2024-04-25T10:16:09.668164+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:21.421172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.680316+0000","last_change":"2024-04-25T10:16:09.677648+0000","last_active":"2024-04-25T10:16:09.680316+0000","last_peered":"2024-04-25T10:16:09.680316+0000","last_clean":"2024-04-25T10:16:09.680316+0000","last_became_active":"2024-04-25T10:16:09.677260+0000","last_became_peered":"2024-04-25T10:16:09.677260+0000","last_unstale":"2024-04-25T10:16:09.680316+0000","last_undegraded":"2024-04-25T10:16:09.680316+0000","last_fullsized":"2024-04-25T10:16:09.680316+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:21.825271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"168'4","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.684029+0000","last_change":"2024-04-25T10:16:08.659302+0000","last_active":"2024-04-25T10:16:09.684029+0000","last_peered":"2024-04-25T10:16:09.684029+0000","last_clean":"2024-04-25T10:16:09.684029+0000","last_became_active":"2024-04-25T10:16:08.658876+0000","last_became_peered":"2024-04-25T10:16:08.658876+0000","last_unstale":"2024-04-25T10:16:09.684029+0000","last_undegraded":"2024-04-25T10:16:09.684029+0000","last_fullsized":"2024-04-25T10:16:09.684029+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:20.418213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.655302+0000","last_change":"2024-04-25T10:16:09.650330+0000","last_active":"2024-04-25T10:16:09.655302+0000","last_peered":"2024-04-25T10:16:09.655302+0000","last_clean":"2024-04-25T10:16:09.655302+0000","last_became_active":"2024-04-25T10:16:09.649113+0000","last_became_peered":"2024-04-25T10:16:09.649113+0000","last_unstale":"2024-04-25T10:16:09.655302+0000","last_undegraded":"2024-04-25T10:16:09.655302+0000","last_fullsized":"2024-04-25T10:16:09.655302+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:24.034526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.680542+0000","last_change":"2024-04-25T10:16:08.647667+0000","last_active":"2024-04-25T10:16:09.680542+0000","last_peered":"2024-04-25T10:16:09.680542+0000","last_clean":"2024-04-25T10:16:09.680542+0000","last_became_active":"2024-04-25T10:16:08.647314+0000","last_became_peered":"2024-04-25T10:16:08.647314+0000","last_unstale":"2024-04-25T10:16:09.680542+0000","last_undegraded":"2024-04-25T10:16:09.680542+0000","last_fullsized":"2024-04-25T10:16:09.680542+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:14.935716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664480+0000","last_change":"2024-04-25T10:16:09.663845+0000","last_active":"2024-04-25T10:16:09.664480+0000","last_peered":"2024-04-25T10:16:09.664480+0000","last_clean":"2024-04-25T10:16:09.664480+0000","last_became_active":"2024-04-25T10:16:09.663450+0000","last_became_peered":"2024-04-25T10:16:09.663450+0000","last_unstale":"2024-04-25T10:16:09.664480+0000","last_undegraded":"2024-04-25T10:16:09.664480+0000","last_fullsized":"2024-04-25T10:16:09.664480+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:54.880074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685332+0000","last_change":"2024-04-25T10:16:08.660011+0000","last_active":"2024-04-25T10:16:09.685332+0000","last_peered":"2024-04-25T10:16:09.685332+0000","last_clean":"2024-04-25T10:16:09.685332+0000","last_became_active":"2024-04-25T10:16:08.659221+0000","last_became_peered":"2024-04-25T10:16:08.659221+0000","last_unstale":"2024-04-25T10:16:09.685332+0000","last_undegraded":"2024-04-25T10:16:09.685332+0000","last_fullsized":"2024-04-25T10:16:09.685332+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:40.572850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.660403+0000","last_change":"2024-04-25T10:16:09.657996+0000","last_active":"2024-04-25T10:16:09.660403+0000","last_peered":"2024-04-25T10:16:09.660403+0000","last_clean":"2024-04-25T10:16:09.660403+0000","last_became_active":"2024-04-25T10:16:09.657718+0000","last_became_peered":"2024-04-25T10:16:09.657718+0000","last_unstale":"2024-04-25T10:16:09.660403+0000","last_undegraded":"2024-04-25T10:16:09.660403+0000","last_fullsized":"2024-04-25T10:16:09.660403+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:51.747050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685461+0000","last_change":"2024-04-25T10:16:08.661916+0000","last_active":"2024-04-25T10:16:09.685461+0000","last_peered":"2024-04-25T10:16:09.685461+0000","last_clean":"2024-04-25T10:16:09.685461+0000","last_became_active":"2024-04-25T10:16:08.658155+0000","last_became_peered":"2024-04-25T10:16:08.658155+0000","last_unstale":"2024-04-25T10:16:09.685461+0000","last_undegraded":"2024-04-25T10:16:09.685461+0000","last_fullsized":"2024-04-25T10:16:09.685461+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:28.978672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.671642+0000","last_change":"2024-04-25T10:16:09.670313+0000","last_active":"2024-04-25T10:16:09.671642+0000","last_peered":"2024-04-25T10:16:09.671642+0000","last_clean":"2024-04-25T10:16:09.671642+0000","last_became_active":"2024-04-25T10:16:09.670046+0000","last_became_peered":"2024-04-25T10:16:09.670046+0000","last_unstale":"2024-04-25T10:16:09.671642+0000","last_undegraded":"2024-04-25T10:16:09.671642+0000","last_fullsized":"2024-04-25T10:16:09.671642+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.700394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.681337+0000","last_change":"2024-04-25T10:16:08.646440+0000","last_active":"2024-04-25T10:16:09.681337+0000","last_peered":"2024-04-25T10:16:09.681337+0000","last_clean":"2024-04-25T10:16:09.681337+0000","last_became_active":"2024-04-25T10:16:08.646012+0000","last_became_peered":"2024-04-25T10:16:08.646012+0000","last_unstale":"2024-04-25T10:16:09.681337+0000","last_undegraded":"2024-04-25T10:16:09.681337+0000","last_fullsized":"2024-04-25T10:16:09.681337+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:31.545225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687731+0000","last_change":"2024-04-25T10:16:09.669241+0000","last_active":"2024-04-25T10:16:09.687731+0000","last_peered":"2024-04-25T10:16:09.687731+0000","last_clean":"2024-04-25T10:16:09.687731+0000","last_became_active":"2024-04-25T10:16:09.668714+0000","last_became_peered":"2024-04-25T10:16:09.668714+0000","last_unstale":"2024-04-25T10:16:09.687731+0000","last_undegraded":"2024-04-25T10:16:09.687731+0000","last_fullsized":"2024-04-25T10:16:09.687731+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:05.969499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683593+0000","last_change":"2024-04-25T10:16:08.649906+0000","last_active":"2024-04-25T10:16:09.683593+0000","last_peered":"2024-04-25T10:16:09.683593+0000","last_clean":"2024-04-25T10:16:09.683593+0000","last_became_active":"2024-04-25T10:16:08.649477+0000","last_became_peered":"2024-04-25T10:16:08.649477+0000","last_unstale":"2024-04-25T10:16:09.683593+0000","last_undegraded":"2024-04-25T10:16:09.683593+0000","last_fullsized":"2024-04-25T10:16:09.683593+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:34.151109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687765+0000","last_change":"2024-04-25T10:16:09.679979+0000","last_active":"2024-04-25T10:16:09.687765+0000","last_peered":"2024-04-25T10:16:09.687765+0000","last_clean":"2024-04-25T10:16:09.687765+0000","last_became_active":"2024-04-25T10:16:09.669194+0000","last_became_peered":"2024-04-25T10:16:09.669194+0000","last_unstale":"2024-04-25T10:16:09.687765+0000","last_undegraded":"2024-04-25T10:16:09.687765+0000","last_fullsized":"2024-04-25T10:16:09.687765+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:31.274242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.661599+0000","last_change":"2024-04-25T10:16:08.664827+0000","last_active":"2024-04-25T10:16:09.661599+0000","last_peered":"2024-04-25T10:16:09.661599+0000","last_clean":"2024-04-25T10:16:09.661599+0000","last_became_active":"2024-04-25T10:16:08.664221+0000","last_became_peered":"2024-04-25T10:16:08.664221+0000","last_unstale":"2024-04-25T10:16:09.661599+0000","last_undegraded":"2024-04-25T10:16:09.661599+0000","last_fullsized":"2024-04-25T10:16:09.661599+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:53.436437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.678076+0000","last_change":"2024-04-25T10:16:09.671723+0000","last_active":"2024-04-25T10:16:09.678076+0000","last_peered":"2024-04-25T10:16:09.678076+0000","last_clean":"2024-04-25T10:16:09.678076+0000","last_became_active":"2024-04-25T10:16:09.671394+0000","last_became_peered":"2024-04-25T10:16:09.671394+0000","last_unstale":"2024-04-25T10:16:09.678076+0000","last_undegraded":"2024-04-25T10:16:09.678076+0000","last_fullsized":"2024-04-25T10:16:09.678076+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:29.226767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.658963+0000","last_change":"2024-04-25T10:16:08.645210+0000","last_active":"2024-04-25T10:16:09.658963+0000","last_peered":"2024-04-25T10:16:09.658963+0000","last_clean":"2024-04-25T10:16:09.658963+0000","last_became_active":"2024-04-25T10:16:08.644570+0000","last_became_peered":"2024-04-25T10:16:08.644570+0000","last_unstale":"2024-04-25T10:16:09.658963+0000","last_undegraded":"2024-04-25T10:16:09.658963+0000","last_fullsized":"2024-04-25T10:16:09.658963+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:06.442796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.674084+0000","last_change":"2024-04-25T10:16:09.672960+0000","last_active":"2024-04-25T10:16:09.674084+0000","last_peered":"2024-04-25T10:16:09.674084+0000","last_clean":"2024-04-25T10:16:09.674084+0000","last_became_active":"2024-04-25T10:16:09.672734+0000","last_became_peered":"2024-04-25T10:16:09.672734+0000","last_unstale":"2024-04-25T10:16:09.674084+0000","last_undegraded":"2024-04-25T10:16:09.674084+0000","last_fullsized":"2024-04-25T10:16:09.674084+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:05.018533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.657712+0000","last_change":"2024-04-25T10:16:08.642261+0000","last_active":"2024-04-25T10:16:09.657712+0000","last_peered":"2024-04-25T10:16:09.657712+0000","last_clean":"2024-04-25T10:16:09.657712+0000","last_became_active":"2024-04-25T10:16:08.641042+0000","last_became_peered":"2024-04-25T10:16:08.641042+0000","last_unstale":"2024-04-25T10:16:09.657712+0000","last_undegraded":"2024-04-25T10:16:09.657712+0000","last_fullsized":"2024-04-25T10:16:09.657712+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:53.877115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.669805+0000","last_change":"2024-04-25T10:16:09.668618+0000","last_active":"2024-04-25T10:16:09.669805+0000","last_peered":"2024-04-25T10:16:09.669805+0000","last_clean":"2024-04-25T10:16:09.669805+0000","last_became_active":"2024-04-25T10:16:09.668318+0000","last_became_peered":"2024-04-25T10:16:09.668318+0000","last_unstale":"2024-04-25T10:16:09.669805+0000","last_undegraded":"2024-04-25T10:16:09.669805+0000","last_fullsized":"2024-04-25T10:16:09.669805+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:32.090033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"168'3","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683744+0000","last_change":"2024-04-25T10:16:08.650684+0000","last_active":"2024-04-25T10:16:09.683744+0000","last_peered":"2024-04-25T10:16:09.683744+0000","last_clean":"2024-04-25T10:16:09.683744+0000","last_became_active":"2024-04-25T10:16:08.650302+0000","last_became_peered":"2024-04-25T10:16:08.650302+0000","last_unstale":"2024-04-25T10:16:09.683744+0000","last_undegraded":"2024-04-25T10:16:09.683744+0000","last_fullsized":"2024-04-25T10:16:09.683744+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:28.693060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.688160+0000","last_change":"2024-04-25T10:16:09.686754+0000","last_active":"2024-04-25T10:16:09.688160+0000","last_peered":"2024-04-25T10:16:09.688160+0000","last_clean":"2024-04-25T10:16:09.688160+0000","last_became_active":"2024-04-25T10:16:09.680912+0000","last_became_peered":"2024-04-25T10:16:09.680912+0000","last_unstale":"2024-04-25T10:16:09.688160+0000","last_undegraded":"2024-04-25T10:16:09.688160+0000","last_fullsized":"2024-04-25T10:16:09.688160+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:52.642922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"168'6","reported_seq":18,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.688029+0000","last_change":"2024-04-25T10:16:08.663000+0000","last_active":"2024-04-25T10:16:09.688029+0000","last_peered":"2024-04-25T10:16:09.688029+0000","last_clean":"2024-04-25T10:16:09.688029+0000","last_became_active":"2024-04-25T10:16:08.658822+0000","last_became_peered":"2024-04-25T10:16:08.658822+0000","last_unstale":"2024-04-25T10:16:09.688029+0000","last_undegraded":"2024-04-25T10:16:09.688029+0000","last_fullsized":"2024-04-25T10:16:09.688029+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:44.462957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.666714+0000","last_change":"2024-04-25T10:16:09.664532+0000","last_active":"2024-04-25T10:16:09.666714+0000","last_peered":"2024-04-25T10:16:09.666714+0000","last_clean":"2024-04-25T10:16:09.666714+0000","last_became_active":"2024-04-25T10:16:09.664150+0000","last_became_peered":"2024-04-25T10:16:09.664150+0000","last_unstale":"2024-04-25T10:16:09.666714+0000","last_undegraded":"2024-04-25T10:16:09.666714+0000","last_fullsized":"2024-04-25T10:16:09.666714+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:00.702346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685995+0000","last_change":"2024-04-25T10:16:08.657294+0000","last_active":"2024-04-25T10:16:09.685995+0000","last_peered":"2024-04-25T10:16:09.685995+0000","last_clean":"2024-04-25T10:16:09.685995+0000","last_became_active":"2024-04-25T10:16:08.655030+0000","last_became_peered":"2024-04-25T10:16:08.655030+0000","last_unstale":"2024-04-25T10:16:09.685995+0000","last_undegraded":"2024-04-25T10:16:09.685995+0000","last_fullsized":"2024-04-25T10:16:09.685995+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:02.771600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.661408+0000","last_change":"2024-04-25T10:16:09.660672+0000","last_active":"2024-04-25T10:16:09.661408+0000","last_peered":"2024-04-25T10:16:09.661408+0000","last_clean":"2024-04-25T10:16:09.661408+0000","last_became_active":"2024-04-25T10:16:09.660097+0000","last_became_peered":"2024-04-25T10:16:09.660097+0000","last_unstale":"2024-04-25T10:16:09.661408+0000","last_undegraded":"2024-04-25T10:16:09.661408+0000","last_fullsized":"2024-04-25T10:16:09.661408+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:56.268533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.655661+0000","last_change":"2024-04-25T10:16:08.634116+0000","last_active":"2024-04-25T10:16:09.655661+0000","last_peered":"2024-04-25T10:16:09.655661+0000","last_clean":"2024-04-25T10:16:09.655661+0000","last_became_active":"2024-04-25T10:16:08.633654+0000","last_became_peered":"2024-04-25T10:16:08.633654+0000","last_unstale":"2024-04-25T10:16:09.655661+0000","last_undegraded":"2024-04-25T10:16:09.655661+0000","last_fullsized":"2024-04-25T10:16:09.655661+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:29.647658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685170+0000","last_change":"2024-04-25T10:16:09.679744+0000","last_active":"2024-04-25T10:16:09.685170+0000","last_peered":"2024-04-25T10:16:09.685170+0000","last_clean":"2024-04-25T10:16:09.685170+0000","last_became_active":"2024-04-25T10:16:09.676445+0000","last_became_peered":"2024-04-25T10:16:09.676445+0000","last_unstale":"2024-04-25T10:16:09.685170+0000","last_undegraded":"2024-04-25T10:16:09.685170+0000","last_fullsized":"2024-04-25T10:16:09.685170+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:05.808729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.659392+0000","last_change":"2024-04-25T10:16:08.660720+0000","last_active":"2024-04-25T10:16:09.659392+0000","last_peered":"2024-04-25T10:16:09.659392+0000","last_clean":"2024-04-25T10:16:09.659392+0000","last_became_active":"2024-04-25T10:16:08.660299+0000","last_became_peered":"2024-04-25T10:16:08.660299+0000","last_unstale":"2024-04-25T10:16:09.659392+0000","last_undegraded":"2024-04-25T10:16:09.659392+0000","last_fullsized":"2024-04-25T10:16:09.659392+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:04.015574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.689055+0000","last_change":"2024-04-25T10:16:09.688609+0000","last_active":"2024-04-25T10:16:09.689055+0000","last_peered":"2024-04-25T10:16:09.689055+0000","last_clean":"2024-04-25T10:16:09.689055+0000","last_became_active":"2024-04-25T10:16:09.688354+0000","last_became_peered":"2024-04-25T10:16:09.688354+0000","last_unstale":"2024-04-25T10:16:09.689055+0000","last_undegraded":"2024-04-25T10:16:09.689055+0000","last_fullsized":"2024-04-25T10:16:09.689055+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:44.590356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"168'8","reported_seq":20,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685489+0000","last_change":"2024-04-25T10:16:08.641352+0000","last_active":"2024-04-25T10:16:09.685489+0000","last_peered":"2024-04-25T10:16:09.685489+0000","last_clean":"2024-04-25T10:16:09.685489+0000","last_became_active":"2024-04-25T10:16:08.640634+0000","last_became_peered":"2024-04-25T10:16:08.640634+0000","last_unstale":"2024-04-25T10:16:09.685489+0000","last_undegraded":"2024-04-25T10:16:09.685489+0000","last_fullsized":"2024-04-25T10:16:09.685489+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:23.431546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.671369+0000","last_change":"2024-04-25T10:16:09.666851+0000","last_active":"2024-04-25T10:16:09.671369+0000","last_peered":"2024-04-25T10:16:09.671369+0000","last_clean":"2024-04-25T10:16:09.671369+0000","last_became_active":"2024-04-25T10:16:09.665403+0000","last_became_peered":"2024-04-25T10:16:09.665403+0000","last_unstale":"2024-04-25T10:16:09.671369+0000","last_undegraded":"2024-04-25T10:16:09.671369+0000","last_fullsized":"2024-04-25T10:16:09.671369+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:58.561617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683791+0000","last_change":"2024-04-25T10:16:08.643698+0000","last_active":"2024-04-25T10:16:09.683791+0000","last_peered":"2024-04-25T10:16:09.683791+0000","last_clean":"2024-04-25T10:16:09.683791+0000","last_became_active":"2024-04-25T10:16:08.642974+0000","last_became_peered":"2024-04-25T10:16:08.642974+0000","last_unstale":"2024-04-25T10:16:09.683791+0000","last_undegraded":"2024-04-25T10:16:09.683791+0000","last_fullsized":"2024-04-25T10:16:09.683791+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:02.927081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.667516+0000","last_change":"2024-04-25T10:16:09.666603+0000","last_active":"2024-04-25T10:16:09.667516+0000","last_peered":"2024-04-25T10:16:09.667516+0000","last_clean":"2024-04-25T10:16:09.667516+0000","last_became_active":"2024-04-25T10:16:09.666280+0000","last_became_peered":"2024-04-25T10:16:09.666280+0000","last_unstale":"2024-04-25T10:16:09.667516+0000","last_undegraded":"2024-04-25T10:16:09.667516+0000","last_fullsized":"2024-04-25T10:16:09.667516+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:29.696019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685538+0000","last_change":"2024-04-25T10:16:08.660345+0000","last_active":"2024-04-25T10:16:09.685538+0000","last_peered":"2024-04-25T10:16:09.685538+0000","last_clean":"2024-04-25T10:16:09.685538+0000","last_became_active":"2024-04-25T10:16:08.660066+0000","last_became_peered":"2024-04-25T10:16:08.660066+0000","last_unstale":"2024-04-25T10:16:09.685538+0000","last_undegraded":"2024-04-25T10:16:09.685538+0000","last_fullsized":"2024-04-25T10:16:09.685538+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:17.581421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664249+0000","last_change":"2024-04-25T10:16:09.661838+0000","last_active":"2024-04-25T10:16:09.664249+0000","last_peered":"2024-04-25T10:16:09.664249+0000","last_clean":"2024-04-25T10:16:09.664249+0000","last_became_active":"2024-04-25T10:16:09.661469+0000","last_became_peered":"2024-04-25T10:16:09.661469+0000","last_unstale":"2024-04-25T10:16:09.664249+0000","last_undegraded":"2024-04-25T10:16:09.664249+0000","last_fullsized":"2024-04-25T10:16:09.664249+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:30.650616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"168'1","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.682058+0000","last_change":"2024-04-25T10:16:08.647360+0000","last_active":"2024-04-25T10:16:09.682058+0000","last_peered":"2024-04-25T10:16:09.682058+0000","last_clean":"2024-04-25T10:16:09.682058+0000","last_became_active":"2024-04-25T10:16:08.646610+0000","last_became_peered":"2024-04-25T10:16:08.646610+0000","last_unstale":"2024-04-25T10:16:09.682058+0000","last_undegraded":"2024-04-25T10:16:09.682058+0000","last_fullsized":"2024-04-25T10:16:09.682058+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:07.439112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.688115+0000","last_change":"2024-04-25T10:16:09.680473+0000","last_active":"2024-04-25T10:16:09.688115+0000","last_peered":"2024-04-25T10:16:09.688115+0000","last_clean":"2024-04-25T10:16:09.688115+0000","last_became_active":"2024-04-25T10:16:09.680151+0000","last_became_peered":"2024-04-25T10:16:09.680151+0000","last_unstale":"2024-04-25T10:16:09.688115+0000","last_undegraded":"2024-04-25T10:16:09.688115+0000","last_fullsized":"2024-04-25T10:16:09.688115+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:15.368473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.653279+0000","last_change":"2024-04-25T10:16:08.649787+0000","last_active":"2024-04-25T10:16:09.653279+0000","last_peered":"2024-04-25T10:16:09.653279+0000","last_clean":"2024-04-25T10:16:09.653279+0000","last_became_active":"2024-04-25T10:16:08.649013+0000","last_became_peered":"2024-04-25T10:16:08.649013+0000","last_unstale":"2024-04-25T10:16:09.653279+0000","last_undegraded":"2024-04-25T10:16:09.653279+0000","last_fullsized":"2024-04-25T10:16:09.653279+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:47.939850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.680723+0000","last_change":"2024-04-25T10:16:09.662397+0000","last_active":"2024-04-25T10:16:09.680723+0000","last_peered":"2024-04-25T10:16:09.680723+0000","last_clean":"2024-04-25T10:16:09.680723+0000","last_became_active":"2024-04-25T10:16:09.662071+0000","last_became_peered":"2024-04-25T10:16:09.662071+0000","last_unstale":"2024-04-25T10:16:09.680723+0000","last_undegraded":"2024-04-25T10:16:09.680723+0000","last_fullsized":"2024-04-25T10:16:09.680723+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:49.292196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.685685+0000","last_change":"2024-04-25T10:16:08.659010+0000","last_active":"2024-04-25T10:16:09.685685+0000","last_peered":"2024-04-25T10:16:09.685685+0000","last_clean":"2024-04-25T10:16:09.685685+0000","last_became_active":"2024-04-25T10:16:08.657905+0000","last_became_peered":"2024-04-25T10:16:08.657905+0000","last_unstale":"2024-04-25T10:16:09.685685+0000","last_undegraded":"2024-04-25T10:16:09.685685+0000","last_fullsized":"2024-04-25T10:16:09.685685+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:02.369700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.682185+0000","last_change":"2024-04-25T10:16:09.672257+0000","last_active":"2024-04-25T10:16:09.682185+0000","last_peered":"2024-04-25T10:16:09.682185+0000","last_clean":"2024-04-25T10:16:09.682185+0000","last_became_active":"2024-04-25T10:16:09.671973+0000","last_became_peered":"2024-04-25T10:16:09.671973+0000","last_unstale":"2024-04-25T10:16:09.682185+0000","last_undegraded":"2024-04-25T10:16:09.682185+0000","last_fullsized":"2024-04-25T10:16:09.682185+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:53.185119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.650116+0000","last_change":"2024-04-25T10:16:08.658908+0000","last_active":"2024-04-25T10:16:09.650116+0000","last_peered":"2024-04-25T10:16:09.650116+0000","last_clean":"2024-04-25T10:16:09.650116+0000","last_became_active":"2024-04-25T10:16:08.658614+0000","last_became_peered":"2024-04-25T10:16:08.658614+0000","last_unstale":"2024-04-25T10:16:09.650116+0000","last_undegraded":"2024-04-25T10:16:09.650116+0000","last_fullsized":"2024-04-25T10:16:09.650116+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:02.523156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.659904+0000","last_change":"2024-04-25T10:16:09.654012+0000","last_active":"2024-04-25T10:16:09.659904+0000","last_peered":"2024-04-25T10:16:09.659904+0000","last_clean":"2024-04-25T10:16:09.659904+0000","last_became_active":"2024-04-25T10:16:09.653563+0000","last_became_peered":"2024-04-25T10:16:09.653563+0000","last_unstale":"2024-04-25T10:16:09.659904+0000","last_undegraded":"2024-04-25T10:16:09.659904+0000","last_fullsized":"2024-04-25T10:16:09.659904+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:29.226767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.646966+0000","last_change":"2024-04-25T10:16:08.653087+0000","last_active":"2024-04-25T10:16:09.646966+0000","last_peered":"2024-04-25T10:16:09.646966+0000","last_clean":"2024-04-25T10:16:09.646966+0000","last_became_active":"2024-04-25T10:16:08.652267+0000","last_became_peered":"2024-04-25T10:16:08.652267+0000","last_unstale":"2024-04-25T10:16:09.646966+0000","last_undegraded":"2024-04-25T10:16:09.646966+0000","last_fullsized":"2024-04-25T10:16:09.646966+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:49.884625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.678681+0000","last_change":"2024-04-25T10:16:09.674875+0000","last_active":"2024-04-25T10:16:09.678681+0000","last_peered":"2024-04-25T10:16:09.678681+0000","last_clean":"2024-04-25T10:16:09.678681+0000","last_became_active":"2024-04-25T10:16:09.674500+0000","last_became_peered":"2024-04-25T10:16:09.674500+0000","last_unstale":"2024-04-25T10:16:09.678681+0000","last_undegraded":"2024-04-25T10:16:09.678681+0000","last_fullsized":"2024-04-25T10:16:09.678681+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:50.230673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.649613+0000","last_change":"2024-04-25T10:16:08.651364+0000","last_active":"2024-04-25T10:16:09.649613+0000","last_peered":"2024-04-25T10:16:09.649613+0000","last_clean":"2024-04-25T10:16:09.649613+0000","last_became_active":"2024-04-25T10:16:08.650588+0000","last_became_peered":"2024-04-25T10:16:08.650588+0000","last_unstale":"2024-04-25T10:16:09.649613+0000","last_undegraded":"2024-04-25T10:16:09.649613+0000","last_fullsized":"2024-04-25T10:16:09.649613+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:02.718326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.671114+0000","last_change":"2024-04-25T10:16:09.667103+0000","last_active":"2024-04-25T10:16:09.671114+0000","last_peered":"2024-04-25T10:16:09.671114+0000","last_clean":"2024-04-25T10:16:09.671114+0000","last_became_active":"2024-04-25T10:16:09.666267+0000","last_became_peered":"2024-04-25T10:16:09.666267+0000","last_unstale":"2024-04-25T10:16:09.671114+0000","last_undegraded":"2024-04-25T10:16:09.671114+0000","last_fullsized":"2024-04-25T10:16:09.671114+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:38.179681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"168'2","reported_seq":14,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.683886+0000","last_change":"2024-04-25T10:16:08.651567+0000","last_active":"2024-04-25T10:16:09.683886+0000","last_peered":"2024-04-25T10:16:09.683886+0000","last_clean":"2024-04-25T10:16:09.683886+0000","last_became_active":"2024-04-25T10:16:08.650547+0000","last_became_peered":"2024-04-25T10:16:08.650547+0000","last_unstale":"2024-04-25T10:16:09.683886+0000","last_undegraded":"2024-04-25T10:16:09.683886+0000","last_fullsized":"2024-04-25T10:16:09.683886+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:43.202403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687469+0000","last_change":"2024-04-25T10:16:09.667922+0000","last_active":"2024-04-25T10:16:09.687469+0000","last_peered":"2024-04-25T10:16:09.687469+0000","last_clean":"2024-04-25T10:16:09.687469+0000","last_became_active":"2024-04-25T10:16:09.662755+0000","last_became_peered":"2024-04-25T10:16:09.662755+0000","last_unstale":"2024-04-25T10:16:09.687469+0000","last_undegraded":"2024-04-25T10:16:09.687469+0000","last_fullsized":"2024-04-25T10:16:09.687469+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:14.638281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.646615+0000","last_change":"2024-04-25T10:16:08.655603+0000","last_active":"2024-04-25T10:16:09.646615+0000","last_peered":"2024-04-25T10:16:09.646615+0000","last_clean":"2024-04-25T10:16:09.646615+0000","last_became_active":"2024-04-25T10:16:08.655227+0000","last_became_peered":"2024-04-25T10:16:08.655227+0000","last_unstale":"2024-04-25T10:16:09.646615+0000","last_undegraded":"2024-04-25T10:16:09.646615+0000","last_fullsized":"2024-04-25T10:16:09.646615+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:55.635776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.664542+0000","last_change":"2024-04-25T10:16:09.662444+0000","last_active":"2024-04-25T10:16:09.664542+0000","last_peered":"2024-04-25T10:16:09.664542+0000","last_clean":"2024-04-25T10:16:09.664542+0000","last_became_active":"2024-04-25T10:16:09.662108+0000","last_became_peered":"2024-04-25T10:16:09.662108+0000","last_unstale":"2024-04-25T10:16:09.664542+0000","last_undegraded":"2024-04-25T10:16:09.664542+0000","last_fullsized":"2024-04-25T10:16:09.664542+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:29.696019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.659201+0000","last_change":"2024-04-25T10:16:08.651246+0000","last_active":"2024-04-25T10:16:09.659201+0000","last_peered":"2024-04-25T10:16:09.659201+0000","last_clean":"2024-04-25T10:16:09.659201+0000","last_became_active":"2024-04-25T10:16:08.650658+0000","last_became_peered":"2024-04-25T10:16:08.650658+0000","last_unstale":"2024-04-25T10:16:09.659201+0000","last_undegraded":"2024-04-25T10:16:09.659201+0000","last_fullsized":"2024-04-25T10:16:09.659201+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:07.522631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.667744+0000","last_change":"2024-04-25T10:16:09.665694+0000","last_active":"2024-04-25T10:16:09.667744+0000","last_peered":"2024-04-25T10:16:09.667744+0000","last_clean":"2024-04-25T10:16:09.667744+0000","last_became_active":"2024-04-25T10:16:09.665125+0000","last_became_peered":"2024-04-25T10:16:09.665125+0000","last_unstale":"2024-04-25T10:16:09.667744+0000","last_undegraded":"2024-04-25T10:16:09.667744+0000","last_fullsized":"2024-04-25T10:16:09.667744+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:44.911985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.659413+0000","last_change":"2024-04-25T10:16:08.641461+0000","last_active":"2024-04-25T10:16:09.659413+0000","last_peered":"2024-04-25T10:16:09.659413+0000","last_clean":"2024-04-25T10:16:09.659413+0000","last_became_active":"2024-04-25T10:16:08.641065+0000","last_became_peered":"2024-04-25T10:16:08.641065+0000","last_unstale":"2024-04-25T10:16:09.659413+0000","last_undegraded":"2024-04-25T10:16:09.659413+0000","last_fullsized":"2024-04-25T10:16:09.659413+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:22.264331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.668835+0000","last_change":"2024-04-25T10:16:09.667553+0000","last_active":"2024-04-25T10:16:09.668835+0000","last_peered":"2024-04-25T10:16:09.668835+0000","last_clean":"2024-04-25T10:16:09.668835+0000","last_became_active":"2024-04-25T10:16:09.667299+0000","last_became_peered":"2024-04-25T10:16:09.667299+0000","last_unstale":"2024-04-25T10:16:09.668835+0000","last_undegraded":"2024-04-25T10:16:09.668835+0000","last_fullsized":"2024-04-25T10:16:09.668835+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:48.108484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.651309+0000","last_change":"2024-04-25T10:16:08.660279+0000","last_active":"2024-04-25T10:16:09.651309+0000","last_peered":"2024-04-25T10:16:09.651309+0000","last_clean":"2024-04-25T10:16:09.651309+0000","last_became_active":"2024-04-25T10:16:08.659536+0000","last_became_peered":"2024-04-25T10:16:08.659536+0000","last_unstale":"2024-04-25T10:16:09.651309+0000","last_undegraded":"2024-04-25T10:16:09.651309+0000","last_fullsized":"2024-04-25T10:16:09.651309+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:34.151109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.669894+0000","last_change":"2024-04-25T10:16:09.668702+0000","last_active":"2024-04-25T10:16:09.669894+0000","last_peered":"2024-04-25T10:16:09.669894+0000","last_clean":"2024-04-25T10:16:09.669894+0000","last_became_active":"2024-04-25T10:16:09.668381+0000","last_became_peered":"2024-04-25T10:16:09.668381+0000","last_unstale":"2024-04-25T10:16:09.669894+0000","last_undegraded":"2024-04-25T10:16:09.669894+0000","last_fullsized":"2024-04-25T10:16:09.669894+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:33.485775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.652359+0000","last_change":"2024-04-25T10:16:08.650806+0000","last_active":"2024-04-25T10:16:09.652359+0000","last_peered":"2024-04-25T10:16:09.652359+0000","last_clean":"2024-04-25T10:16:09.652359+0000","last_became_active":"2024-04-25T10:16:08.650004+0000","last_became_peered":"2024-04-25T10:16:08.650004+0000","last_unstale":"2024-04-25T10:16:09.652359+0000","last_undegraded":"2024-04-25T10:16:09.652359+0000","last_fullsized":"2024-04-25T10:16:09.652359+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:02.523156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.687395+0000","last_change":"2024-04-25T10:16:09.663171+0000","last_active":"2024-04-25T10:16:09.687395+0000","last_peered":"2024-04-25T10:16:09.687395+0000","last_clean":"2024-04-25T10:16:09.687395+0000","last_became_active":"2024-04-25T10:16:09.662888+0000","last_became_peered":"2024-04-25T10:16:09.662888+0000","last_unstale":"2024-04-25T10:16:09.687395+0000","last_undegraded":"2024-04-25T10:16:09.687395+0000","last_fullsized":"2024-04-25T10:16:09.687395+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:08.143697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.647265+0000","last_change":"2024-04-25T10:16:08.652691+0000","last_active":"2024-04-25T10:16:09.647265+0000","last_peered":"2024-04-25T10:16:09.647265+0000","last_clean":"2024-04-25T10:16:09.647265+0000","last_became_active":"2024-04-25T10:16:08.651694+0000","last_became_peered":"2024-04-25T10:16:08.651694+0000","last_unstale":"2024-04-25T10:16:09.647265+0000","last_undegraded":"2024-04-25T10:16:09.647265+0000","last_fullsized":"2024-04-25T10:16:09.647265+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:47.276424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.680221+0000","last_change":"2024-04-25T10:16:09.672755+0000","last_active":"2024-04-25T10:16:09.680221+0000","last_peered":"2024-04-25T10:16:09.680221+0000","last_clean":"2024-04-25T10:16:09.680221+0000","last_became_active":"2024-04-25T10:16:09.672278+0000","last_became_peered":"2024-04-25T10:16:09.672278+0000","last_unstale":"2024-04-25T10:16:09.680221+0000","last_undegraded":"2024-04-25T10:16:09.680221+0000","last_fullsized":"2024-04-25T10:16:09.680221+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:05.018533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.649998+0000","last_change":"2024-04-25T10:16:08.643862+0000","last_active":"2024-04-25T10:16:09.649998+0000","last_peered":"2024-04-25T10:16:09.649998+0000","last_clean":"2024-04-25T10:16:09.649998+0000","last_became_active":"2024-04-25T10:16:08.642583+0000","last_became_peered":"2024-04-25T10:16:08.642583+0000","last_unstale":"2024-04-25T10:16:09.649998+0000","last_undegraded":"2024-04-25T10:16:09.649998+0000","last_fullsized":"2024-04-25T10:16:09.649998+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:22.320789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.667033+0000","last_change":"2024-04-25T10:16:09.666613+0000","last_active":"2024-04-25T10:16:09.667033+0000","last_peered":"2024-04-25T10:16:09.667033+0000","last_clean":"2024-04-25T10:16:09.667033+0000","last_became_active":"2024-04-25T10:16:09.663230+0000","last_became_peered":"2024-04-25T10:16:09.663230+0000","last_unstale":"2024-04-25T10:16:09.667033+0000","last_undegraded":"2024-04-25T10:16:09.667033+0000","last_fullsized":"2024-04-25T10:16:09.667033+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:08.612950+0000","last_clean_scrub_stamp":"2024-04-25T10:16:08.612950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:05.598402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.659756+0000","last_change":"2024-04-25T10:16:08.649365+0000","last_active":"2024-04-25T10:16:09.659756+0000","last_peered":"2024-04-25T10:16:09.659756+0000","last_clean":"2024-04-25T10:16:09.659756+0000","last_became_active":"2024-04-25T10:16:08.648780+0000","last_became_peered":"2024-04-25T10:16:08.648780+0000","last_unstale":"2024-04-25T10:16:09.659756+0000","last_undegraded":"2024-04-25T10:16:09.659756+0000","last_fullsized":"2024-04-25T10:16:09.659756+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:07.609991+0000","last_clean_scrub_stamp":"2024-04-25T10:16:07.609991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:49.227714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":449,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-04-25T10:16:09.650184+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:16:09.650184+0000","last_peered":"2024-04-25T10:16:09.650184+0000","last_clean":"2024-04-25T10:16:09.650184+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:16:09.650184+0000","last_undegraded":"2024-04-25T10:16:09.650184+0000","last_fullsized":"2024-04-25T10:16:09.650184+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575026,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74072,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297768,"statfs":{"total":96636764160,"available":96560914432,"internally_reserved":0,"allocated":2699264,"data_stored":1576408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51288,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320552,"statfs":{"total":96636764160,"available":96584245248,"internally_reserved":0,"allocated":2105344,"data_stored":986040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575026,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47124,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324716,"statfs":{"total":96636764160,"available":96588509184,"internally_reserved":0,"allocated":2105344,"data_stored":986040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575026,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77684,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294156,"statfs":{"total":96636764160,"available":96557215744,"internally_reserved":0,"allocated":2134016,"data_stored":994853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575026,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77652,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294188,"statfs":{"total":96636764160,"available":96557248512,"internally_reserved":0,"allocated":2105344,"data_stored":986040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575026,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51292,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320548,"statfs":{"total":96636764160,"available":96584241152,"internally_reserved":0,"allocated":2105344,"data_stored":986040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575026,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47132,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324708,"statfs":{"total":96636764160,"available":96588500992,"internally_reserved":0,"allocated":2105344,"data_stored":986040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575027,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74080,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297760,"statfs":{"total":96636764160,"available":96560906240,"internally_reserved":0,"allocated":2699264,"data_stored":1576408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:13.041 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:16:13.041 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:16:13.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc4640640 1 Processor -- start 2024-04-25T10:16:13.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc4640640 1 -- start start 2024-04-25T10:16:13.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc4640640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cbc145170 0x7f4cbc06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc4640640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4cbc06e4d0 0x7f4cbc0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc4640640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cbc0ce2e0 con 0x7f4cbc0cdd60 2024-04-25T10:16:13.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc4640640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cbc06db00 con 0x7f4cbc06e4d0 2024-04-25T10:16:13.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc4640640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc145550 con 0x7f4cbc145170 2024-04-25T10:16:13.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc1bdb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4cbc06e4d0 0x7f4cbc0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc1bdb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4cbc06e4d0 0x7f4cbc0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55762/0 (socket says 172.21.15.133:55762) 2024-04-25T10:16:13.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc1bdb640 1 -- 172.21.15.133:0/2032791534 learned_addr learned my addr 172.21.15.133:0/2032791534 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:13.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc23dc640 1 --2- 172.21.15.133:0/2032791534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cbc145170 0x7f4cbc06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc1bdb640 1 -- 172.21.15.133:0/2032791534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cbc145170 msgr2=0x7f4cbc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:13.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc1bdb640 1 --2- 172.21.15.133:0/2032791534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cbc145170 0x7f4cbc06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc1bdb640 1 -- 172.21.15.133:0/2032791534 >> v1:172.21.15.162:6789/0 conn(0x7f4cbc0cdd60 legacy=0x7f4cbc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:13.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.205+0000 7f4cc1bdb640 1 -- 172.21.15.133:0/2032791534 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cbc06d4b0 con 0x7f4cbc06e4d0 2024-04-25T10:16:13.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc1bdb640 1 --2- 172.21.15.133:0/2032791534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc06e4d0 0x7f4cbc0cd820 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009870 tx=0x7f4cb001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adec99a4590b360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc13da640 1 -- 172.21.15.133:0/2032791534 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00b2070 con 0x7f4cbc06e4d0 2024-04-25T10:16:13.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc13da640 1 -- 172.21.15.133:0/2032791534 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb00a9070 con 0x7f4cbc06e4d0 2024-04-25T10:16:13.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc13da640 1 -- 172.21.15.133:0/2032791534 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00ad880 con 0x7f4cbc06e4d0 2024-04-25T10:16:13.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc4640640 1 -- 172.21.15.133:0/2032791534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc06e4d0 msgr2=0x7f4cbc0cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc4640640 1 --2- 172.21.15.133:0/2032791534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc06e4d0 0x7f4cbc0cd820 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009870 tx=0x7f4cb001c560 comp rx=0 tx=0).stop 2024-04-25T10:16:13.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc4640640 1 -- 172.21.15.133:0/2032791534 shutdown_connections 2024-04-25T10:16:13.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc4640640 1 --2- 172.21.15.133:0/2032791534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc06e4d0 0x7f4cbc0cd820 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc4640640 1 --2- 172.21.15.133:0/2032791534 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cbc145170 0x7f4cbc06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc4640640 1 -- 172.21.15.133:0/2032791534 >> 172.21.15.133:0/2032791534 conn(0x7f4cbc066f70 msgr2=0x7f4cbc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc4640640 1 -- 172.21.15.133:0/2032791534 shutdown_connections 2024-04-25T10:16:13.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc4640640 1 -- 172.21.15.133:0/2032791534 wait complete. 2024-04-25T10:16:13.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc4640640 1 Processor -- start 2024-04-25T10:16:13.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.209+0000 7f4cc4640640 1 -- start start 2024-04-25T10:16:13.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc4640640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cbc06e4d0 0x7f4cbc138ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc4640640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cbc0cdd60 0x7f4cbc160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc4640640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc145170 0x7f4cbc1f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc4640640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc168490 con 0x7f4cbc145170 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc4640640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc15d7f0 con 0x7f4cbc06e4d0 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc4640640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc168610 con 0x7f4cbc0cdd60 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc2bdd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc145170 0x7f4cbc1f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc2bdd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc145170 0x7f4cbc1f7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55778/0 (socket says 172.21.15.133:55778) 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc2bdd640 1 -- 172.21.15.133:0/3156315442 learned_addr learned my addr 172.21.15.133:0/3156315442 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc23dc640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cbc06e4d0 0x7f4cbc138ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc1bdb640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cbc0cdd60 0x7f4cbc160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc2bdd640 1 -- 172.21.15.133:0/3156315442 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cbc0cdd60 msgr2=0x7f4cbc160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc2bdd640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cbc0cdd60 0x7f4cbc160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc2bdd640 1 -- 172.21.15.133:0/3156315442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cbc06e4d0 msgr2=0x7f4cbc138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc2bdd640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cbc06e4d0 0x7f4cbc138ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc2bdd640 1 -- 172.21.15.133:0/3156315442 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cbc15fee0 con 0x7f4cbc145170 2024-04-25T10:16:13.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc23dc640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cbc06e4d0 0x7f4cbc138ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:13.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc2bdd640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc145170 0x7f4cbc1f7d00 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400c730 tx=0x7f4cb400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4caeffd640 1 -- 172.21.15.133:0/3156315442 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4003b80 con 0x7f4cbc145170 2024-04-25T10:16:13.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4caeffd640 1 -- 172.21.15.133:0/3156315442 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb4003d20 con 0x7f4cbc145170 2024-04-25T10:16:13.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc4640640 1 -- 172.21.15.133:0/3156315442 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cbc13c460 con 0x7f4cbc145170 2024-04-25T10:16:13.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4caeffd640 1 -- 172.21.15.133:0/3156315442 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400aa30 con 0x7f4cbc145170 2024-04-25T10:16:13.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc4640640 1 -- 172.21.15.133:0/3156315442 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cbc13ca20 con 0x7f4cbc145170 2024-04-25T10:16:13.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc4640640 1 -- 172.21.15.133:0/3156315442 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cbc068e80 con 0x7f4cbc145170 2024-04-25T10:16:13.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4caeffd640 1 -- 172.21.15.133:0/3156315442 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4cb400abd0 con 0x7f4cbc145170 2024-04-25T10:16:13.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4caeffd640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca0066f90 0x7f4ca0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc23dc640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca0066f90 0x7f4ca0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4caeffd640 1 -- 172.21.15.133:0/3156315442 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4cb4085020 con 0x7f4cbc145170 2024-04-25T10:16:13.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.213+0000 7f4cc23dc640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca0066f90 0x7f4ca0069450 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc15ed40 tx=0x7f4cb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.217+0000 7f4caeffd640 1 -- 172.21.15.133:0/3156315442 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4cb4055ea0 con 0x7f4cbc145170 2024-04-25T10:16:13.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.429+0000 7f4cc4640640 1 -- 172.21.15.133:0/3156315442 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4cbc160640 con 0x7f4cbc145170 2024-04-25T10:16:13.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.433+0000 7f4caeffd640 1 -- 172.21.15.133:0/3156315442 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4cb40557a0 con 0x7f4cbc145170 2024-04-25T10:16:13.434 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:16:13.434 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:16:13.434 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:16:13.434 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:16:13.434 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:16:13.435 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:16:13.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.433+0000 7f4cc4640640 1 -- 172.21.15.133:0/3156315442 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca0066f90 msgr2=0x7f4ca0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.433+0000 7f4cc4640640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca0066f90 0x7f4ca0069450 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc15ed40 tx=0x7f4cb8008040 comp rx=0 tx=0).stop 2024-04-25T10:16:13.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.433+0000 7f4cc4640640 1 -- 172.21.15.133:0/3156315442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc145170 msgr2=0x7f4cbc1f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.433+0000 7f4cc4640640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc145170 0x7f4cbc1f7d00 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400c730 tx=0x7f4cb400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.433+0000 7f4cc4640640 1 -- 172.21.15.133:0/3156315442 shutdown_connections 2024-04-25T10:16:13.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.433+0000 7f4cc4640640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ca0066f90 0x7f4ca0069450 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.437+0000 7f4cc4640640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cbc145170 0x7f4cbc1f7d00 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.437+0000 7f4cc4640640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4cbc0cdd60 0x7f4cbc160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.437+0000 7f4cc4640640 1 --2- 172.21.15.133:0/3156315442 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4cbc06e4d0 0x7f4cbc138ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.437+0000 7f4cc4640640 1 -- 172.21.15.133:0/3156315442 >> 172.21.15.133:0/3156315442 conn(0x7f4cbc066f70 msgr2=0x7f4cbc0b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.437+0000 7f4cc4640640 1 -- 172.21.15.133:0/3156315442 shutdown_connections 2024-04-25T10:16:13.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.437+0000 7f4cc4640640 1 -- 172.21.15.133:0/3156315442 wait complete. 2024-04-25T10:16:13.451 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:13.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.561+0000 7f96bd074640 1 Processor -- start 2024-04-25T10:16:13.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.561+0000 7f96bd074640 1 -- start start 2024-04-25T10:16:13.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96bd074640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f96b8158110 0x7f96b80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96bd074640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b80cbe60 con 0x7f96b806eb70 2024-04-25T10:16:13.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96bd074640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b80b19b0 con 0x7f96b81584f0 2024-04-25T10:16:13.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96bd074640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b815b360 con 0x7f96b8158110 2024-04-25T10:16:13.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b759e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f96b806eb70 0x7f96b80cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40968/0 (socket says 172.21.15.133:40968) 2024-04-25T10:16:13.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b759e640 1 -- 172.21.15.133:0/4293647743 learned_addr learned my addr 172.21.15.133:0/4293647743 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:13.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b6d9d640 1 --2- 172.21.15.133:0/4293647743 >> v2:172.21.15.162:3300/0 conn(0x7f96b8158110 0x7f96b80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683333107 0 0) 0x7f96b80cbe60 con 0x7f96b806eb70 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96a4003530 con 0x7f96b806eb70 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1857014844 0 0) 0x7f96a4003530 con 0x7f96b806eb70 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 >> v2:172.21.15.162:3300/0 conn(0x7f96b8158110 msgr2=0x7f96b80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 --2- 172.21.15.133:0/4293647743 >> v2:172.21.15.162:3300/0 conn(0x7f96b8158110 0x7f96b80b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 >> v1:172.21.15.162:6789/0 conn(0x7f96b81584f0 legacy=0x7f96b806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b815b640 con 0x7f96b806eb70 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f96b0001680 con 0x7f96b806eb70 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 >> v1:172.21.15.133:6789/0 conn(0x7f96b806eb70 legacy=0x7f96b80cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 --2- 172.21.15.133:0/4293647743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 0x7f96a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 --2- 172.21.15.133:0/4293647743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96a4004810 0x7f96a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 --2- 172.21.15.133:0/4293647743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96a40051d0 0x7f96a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b815b360 con 0x7f96a40039c0 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a400c070 con 0x7f96a40051d0 2024-04-25T10:16:13.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a400c270 con 0x7f96a4004810 2024-04-25T10:16:13.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b6d9d640 1 --2- 172.21.15.133:0/4293647743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 0x7f96a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b6d9d640 1 -- 172.21.15.133:0/4293647743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96a4004810 msgr2=0x7f96a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:13.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b6d9d640 1 --2- 172.21.15.133:0/4293647743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96a4004810 0x7f96a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b6d9d640 1 -- 172.21.15.133:0/4293647743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96a40051d0 msgr2=0x7f96a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:13.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b759e640 1 --2- 172.21.15.133:0/4293647743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96a40051d0 0x7f96a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b6d9d640 1 --2- 172.21.15.133:0/4293647743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96a40051d0 0x7f96a400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b6d9d640 1 -- 172.21.15.133:0/4293647743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a4003530 con 0x7f96a40039c0 2024-04-25T10:16:13.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b759e640 1 -- 172.21.15.133:0/4293647743 reap_dead start 2024-04-25T10:16:13.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b6d9d640 1 --2- 172.21.15.133:0/4293647743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 0x7f96a4003de0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009fc0 tx=0x7f96a801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9852b5905ad3d50f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a80b2070 con 0x7f96a40039c0 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a80a5b20 con 0x7f96a40039c0 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.565+0000 7f96b5d9b640 1 -- 172.21.15.133:0/4293647743 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8009c10 con 0x7f96a40039c0 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- 172.21.15.133:0/4293647743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 msgr2=0x7f96a4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 --2- 172.21.15.133:0/4293647743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 0x7f96a4003de0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009fc0 tx=0x7f96a801cc50 comp rx=0 tx=0).stop 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- 172.21.15.133:0/4293647743 shutdown_connections 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 --2- 172.21.15.133:0/4293647743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 0x7f96a4003de0 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- 172.21.15.133:0/4293647743 >> 172.21.15.133:0/4293647743 conn(0x7f96b8067000 msgr2=0x7f96b806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- 172.21.15.133:0/4293647743 shutdown_connections 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- 172.21.15.133:0/4293647743 wait complete. 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 Processor -- start 2024-04-25T10:16:13.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- start start 2024-04-25T10:16:13.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 0x7f96b814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96b814e7f0 0x7f96b81fa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b8151d90 0x7f96b8152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b806e460 con 0x7f96a40039c0 2024-04-25T10:16:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b815b120 con 0x7f96b8151d90 2024-04-25T10:16:13.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b806e5e0 con 0x7f96b814e7f0 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96b6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 0x7f96b814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96b6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 0x7f96b814deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55806/0 (socket says 172.21.15.133:55806) 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96b6d9d640 1 -- 172.21.15.133:0/4144286276 learned_addr learned my addr 172.21.15.133:0/4144286276 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96b659c640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96b814e7f0 0x7f96b81fa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96b759e640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b8151d90 0x7f96b8152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96b6d9d640 1 -- 172.21.15.133:0/4144286276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96b814e7f0 msgr2=0x7f96b81fa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96b6d9d640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96b814e7f0 0x7f96b81fa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96b6d9d640 1 -- 172.21.15.133:0/4144286276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b8151d90 msgr2=0x7f96b8152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96b6d9d640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b8151d90 0x7f96b8152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96b6d9d640 1 -- 172.21.15.133:0/4144286276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b81fad50 con 0x7f96a40039c0 2024-04-25T10:16:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96b6d9d640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 0x7f96b814deb0 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f96a80b0040 tx=0x7f96a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96af7fe640 1 -- 172.21.15.133:0/4144286276 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a80ba070 con 0x7f96a40039c0 2024-04-25T10:16:13.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96af7fe640 1 -- 172.21.15.133:0/4144286276 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a80adac0 con 0x7f96a40039c0 2024-04-25T10:16:13.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- 172.21.15.133:0/4144286276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b81fb030 con 0x7f96a40039c0 2024-04-25T10:16:13.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96af7fe640 1 -- 172.21.15.133:0/4144286276 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a80b2040 con 0x7f96a40039c0 2024-04-25T10:16:13.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- 172.21.15.133:0/4144286276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b8153f10 con 0x7f96a40039c0 2024-04-25T10:16:13.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.569+0000 7f96bd074640 1 -- 172.21.15.133:0/4144286276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b806bc20 con 0x7f96a40039c0 2024-04-25T10:16:13.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.573+0000 7f96af7fe640 1 -- 172.21.15.133:0/4144286276 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96a80b6410 con 0x7f96a40039c0 2024-04-25T10:16:13.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.573+0000 7f96af7fe640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9698066f90 0x7f9698069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:13.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.573+0000 7f96af7fe640 1 -- 172.21.15.133:0/4144286276 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f96a811f7e0 con 0x7f96a40039c0 2024-04-25T10:16:13.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.573+0000 7f96b659c640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9698066f90 0x7f9698069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:13.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.573+0000 7f96b659c640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9698066f90 0x7f9698069450 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f96b81eb350 tx=0x7f96a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:13.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.577+0000 7f96af7fe640 1 -- 172.21.15.133:0/4144286276 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f96a8016990 con 0x7f96a40039c0 2024-04-25T10:16:13.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.757+0000 7f96bd074640 1 -- 172.21.15.133:0/4144286276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96b80cd020 con 0x7f96a40039c0 2024-04-25T10:16:13.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.757+0000 7f96af7fe640 1 -- 172.21.15.133:0/4144286276 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4962 (secure 0 0 0) 0x7f96a80f00a0 con 0x7f96a40039c0 2024-04-25T10:16:13.758 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:13.758 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1498957356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1498957356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1498957356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/883424012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":883424012},{"type":"v1","addr":"172.21.15.133:6835","nonce":883424012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2040533186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2040533186},{"type":"v1","addr":"172.21.15.133:6837","nonce":2040533186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:08.983208+0000","modified":"2024-04-25T10:16:11.017848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11037":{"gid":11037,"name":"d","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/2658062772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2658062772},{"type":"v1","addr":"172.21.15.162:6833","nonce":2658062772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:16:13.759 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 105 2024-04-25T10:16:13.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 -- 172.21.15.133:0/4144286276 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9698066f90 msgr2=0x7f9698069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9698066f90 0x7f9698069450 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f96b81eb350 tx=0x7f96a0009040 comp rx=0 tx=0).stop 2024-04-25T10:16:13.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 -- 172.21.15.133:0/4144286276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 msgr2=0x7f96b814deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:13.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 0x7f96b814deb0 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f96a80b0040 tx=0x7f96a8009750 comp rx=0 tx=0).stop 2024-04-25T10:16:13.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 -- 172.21.15.133:0/4144286276 shutdown_connections 2024-04-25T10:16:13.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b8151d90 0x7f96b8152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96b814e7f0 0x7f96b81fa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9698066f90 0x7f9698069450 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 --2- 172.21.15.133:0/4144286276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a40039c0 0x7f96b814deb0 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:13.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 -- 172.21.15.133:0/4144286276 >> 172.21.15.133:0/4144286276 conn(0x7f96b8067000 msgr2=0x7f96b80711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:13.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 -- 172.21.15.133:0/4144286276 shutdown_connections 2024-04-25T10:16:13.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:13.761+0000 7f96bd074640 1 -- 172.21.15.133:0/4144286276 wait complete. 2024-04-25T10:16:14.179 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:08.983208+0000', 'modified': '2024-04-25T10:16:11.017848+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11037}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11037': {'gid': 11037, 'name': 'd', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6833/2658062772', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6832', 'nonce': 2658062772}, {'type': 'v1', 'addr': '172.21.15.162:6833', 'nonce': 2658062772}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:14.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:16:14.180 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:16:14.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.309+0000 7fd74319f640 1 Processor -- start 2024-04-25T10:16:14.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.309+0000 7fd74319f640 1 -- start start 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.309+0000 7fd74319f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd73c145170 0x7fd73c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.309+0000 7fd74319f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd73c06e4d0 0x7fd73c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.309+0000 7fd74319f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd73c0ce2e0 con 0x7fd73c0cdd60 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.309+0000 7fd74319f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd73c06db00 con 0x7fd73c145170 2024-04-25T10:16:14.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.309+0000 7fd74319f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c145550 con 0x7fd73c06e4d0 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.309+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd73c06e4d0 0x7fd73c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.309+0000 7fd740f3b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd73c145170 0x7fd73c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.309+0000 7fd74173c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd73c0cdd60 0x7fd73c15ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:51754/0 (socket says 172.21.15.133:51754) 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.309+0000 7fd74173c640 1 -- 172.21.15.133:0/775021693 learned_addr learned my addr 172.21.15.133:0/775021693 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd73bfff640 1 -- 172.21.15.133:0/775021693 >> v2:172.21.15.162:3300/0 conn(0x7fd73c145170 msgr2=0x7fd73c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd73bfff640 1 --2- 172.21.15.133:0/775021693 >> v2:172.21.15.162:3300/0 conn(0x7fd73c145170 0x7fd73c06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd73bfff640 1 -- 172.21.15.133:0/775021693 >> v1:172.21.15.162:6789/0 conn(0x7fd73c0cdd60 legacy=0x7fd73c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd73bfff640 1 -- 172.21.15.133:0/775021693 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c06d4b0 con 0x7fd73c06e4d0 2024-04-25T10:16:14.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd740f3b640 1 --2- 172.21.15.133:0/775021693 >> v2:172.21.15.162:3300/0 conn(0x7fd73c145170 0x7fd73c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd73bfff640 1 --2- 172.21.15.133:0/775021693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd73c06e4d0 0x7fd73c0cd820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd7300097c0 tx=0x7fd73001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a238eb76050dca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd73b7fe640 1 -- 172.21.15.133:0/775021693 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7300b0070 con 0x7fd73c06e4d0 2024-04-25T10:16:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd73b7fe640 1 -- 172.21.15.133:0/775021693 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7300ab600 con 0x7fd73c06e4d0 2024-04-25T10:16:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd73b7fe640 1 -- 172.21.15.133:0/775021693 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7300ab9d0 con 0x7fd73c06e4d0 2024-04-25T10:16:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd74319f640 1 -- 172.21.15.133:0/775021693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd73c06e4d0 msgr2=0x7fd73c0cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd74319f640 1 --2- 172.21.15.133:0/775021693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd73c06e4d0 0x7fd73c0cd820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd7300097c0 tx=0x7fd73001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd74319f640 1 -- 172.21.15.133:0/775021693 shutdown_connections 2024-04-25T10:16:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd74319f640 1 --2- 172.21.15.133:0/775021693 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd73c06e4d0 0x7fd73c0cd820 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd74319f640 1 --2- 172.21.15.133:0/775021693 >> v2:172.21.15.162:3300/0 conn(0x7fd73c145170 0x7fd73c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd74319f640 1 -- 172.21.15.133:0/775021693 >> 172.21.15.133:0/775021693 conn(0x7fd73c066f70 msgr2=0x7fd73c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd74319f640 1 -- 172.21.15.133:0/775021693 shutdown_connections 2024-04-25T10:16:14.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.313+0000 7fd74319f640 1 -- 172.21.15.133:0/775021693 wait complete. 2024-04-25T10:16:14.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74319f640 1 Processor -- start 2024-04-25T10:16:14.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74319f640 1 -- start start 2024-04-25T10:16:14.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74319f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd73c06e4d0 0x7fd73c143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74319f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd73c0cdd60 0x7fd73c141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74319f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd73c145170 0x7fd73c129880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74319f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c168490 con 0x7fd73c145170 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74319f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c15d7f0 con 0x7fd73c06e4d0 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74319f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c168610 con 0x7fd73c0cdd60 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74173c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd73c145170 0x7fd73c129880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74173c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd73c145170 0x7fd73c129880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55814/0 (socket says 172.21.15.133:55814) 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74173c640 1 -- 172.21.15.133:0/1103319537 learned_addr learned my addr 172.21.15.133:0/1103319537 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd740f3b640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd73c06e4d0 0x7fd73c143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd73bfff640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd73c0cdd60 0x7fd73c141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74173c640 1 -- 172.21.15.133:0/1103319537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd73c0cdd60 msgr2=0x7fd73c141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74173c640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd73c0cdd60 0x7fd73c141bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74173c640 1 -- 172.21.15.133:0/1103319537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd73c06e4d0 msgr2=0x7fd73c143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74173c640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd73c06e4d0 0x7fd73c143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74173c640 1 -- 172.21.15.133:0/1103319537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c1420f0 con 0x7fd73c145170 2024-04-25T10:16:14.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74173c640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd73c145170 0x7fd73c129880 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7fd73400ca80 tx=0x7fd73400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd7397fa640 1 -- 172.21.15.133:0/1103319537 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734017070 con 0x7fd73c145170 2024-04-25T10:16:14.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd7397fa640 1 -- 172.21.15.133:0/1103319537 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73400ac30 con 0x7fd73c145170 2024-04-25T10:16:14.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74319f640 1 -- 172.21.15.133:0/1103319537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c160b90 con 0x7fd73c145170 2024-04-25T10:16:14.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd7397fa640 1 -- 172.21.15.133:0/1103319537 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734012600 con 0x7fd73c145170 2024-04-25T10:16:14.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.317+0000 7fd74319f640 1 -- 172.21.15.133:0/1103319537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c143010 con 0x7fd73c145170 2024-04-25T10:16:14.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd7397fa640 1 -- 172.21.15.133:0/1103319537 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd73401a070 con 0x7fd73c145170 2024-04-25T10:16:14.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd7397fa640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd728066f90 0x7fd728069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd7397fa640 1 -- 172.21.15.133:0/1103319537 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd734084da0 con 0x7fd73c145170 2024-04-25T10:16:14.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd740f3b640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd728066f90 0x7fd728069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd740f3b640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd728066f90 0x7fd728069450 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009580 tx=0x7fd72c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 Processor -- start 2024-04-25T10:16:14.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 -- start start 2024-04-25T10:16:14.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c321d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c322240 0x56434c2d79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c2d7f30 0x56434c2d8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56434c1bf5f0 con 0x56434c321920 2024-04-25T10:16:14.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56434c126da0 con 0x56434c2d7f30 2024-04-25T10:16:14.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56434c0af150 con 0x56434c322240 2024-04-25T10:16:14.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd740f3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c321d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd740f3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c321d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55828/0 (socket says 172.21.15.133:55828) 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd740f3b640 1 -- 172.21.15.133:0/302622033 learned_addr learned my addr 172.21.15.133:0/302622033 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd740f3b640 1 -- 172.21.15.133:0/302622033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c322240 msgr2=0x56434c2d79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd740f3b640 1 --2- 172.21.15.133:0/302622033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c322240 0x56434c2d79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd740f3b640 1 -- 172.21.15.133:0/302622033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c2d7f30 msgr2=0x56434c2d8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd740f3b640 1 --2- 172.21.15.133:0/302622033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c2d7f30 0x56434c2d8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd740f3b640 1 -- 172.21.15.133:0/302622033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56434c0c8ca0 con 0x56434c321920 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd740f3b640 1 --2- 172.21.15.133:0/302622033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c321d00 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009370 tx=0x7fd72c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9045dceff4430ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd7227fc640 1 -- 172.21.15.133:0/302622033 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c011dc0 con 0x56434c321920 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd7227fc640 1 -- 172.21.15.133:0/302622033 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c026070 con 0x56434c321920 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd7227fc640 1 -- 172.21.15.133:0/302622033 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c021640 con 0x56434c321920 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 -- 172.21.15.133:0/302622033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 msgr2=0x56434c321d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 --2- 172.21.15.133:0/302622033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c321d00 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009370 tx=0x7fd72c014000 comp rx=0 tx=0).stop 2024-04-25T10:16:14.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 -- 172.21.15.133:0/302622033 shutdown_connections 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 --2- 172.21.15.133:0/302622033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c2d7f30 0x56434c2d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 --2- 172.21.15.133:0/302622033 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c322240 0x56434c2d79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 --2- 172.21.15.133:0/302622033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c321d00 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 -- 172.21.15.133:0/302622033 >> 172.21.15.133:0/302622033 conn(0x56434c2de2a0 msgr2=0x56434c2d62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 -- 172.21.15.133:0/302622033 shutdown_connections 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.321+0000 7fd74630c000 1 -- 172.21.15.133:0/302622033 wait complete. 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 -- start start 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 Processor -- start 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c2d7f30 0x56434c328d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c321920 0x56434c3292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c322240 0x56434c2dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56434c0e1990 con 0x56434c2d7f30 2024-04-25T10:16:14.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56434c122e40 con 0x56434c321920 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56434c0b2900 con 0x56434c322240 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 10 client.0 _ll_register_callbacks cb 0x56434c1da1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd740f3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c2d7f30 0x56434c328d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd740f3b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c2d7f30 0x56434c328d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55842/0 (socket says 172.21.15.133:55842) 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd740f3b640 1 -- 172.21.15.133:0/2656588317 learned_addr learned my addr 172.21.15.133:0/2656588317 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd740f3b640 1 -- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c322240 msgr2=0x56434c2dbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd73bfff640 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c321920 0x56434c3292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74173c640 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c322240 0x56434c2dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd740f3b640 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c322240 0x56434c2dbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd740f3b640 1 -- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c321920 msgr2=0x56434c3292a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd740f3b640 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c321920 0x56434c3292a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd740f3b640 1 -- 172.21.15.133:0/2656588317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56434c0df630 con 0x56434c2d7f30 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd740f3b640 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c2d7f30 0x56434c328d60 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fd72c008010 tx=0x7fd72c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed861ccc4989df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd71b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd71b7fe640 1 -- 172.21.15.133:0/2656588317 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c011e80 con 0x56434c2d7f30 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 -- 172.21.15.133:0/2656588317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd72c026070 con 0x56434c2d7f30 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd71b7fe640 1 -- 172.21.15.133:0/2656588317 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd72c021b20 con 0x56434c2d7f30 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd71b7fe640 1 -- 172.21.15.133:0/2656588317 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd72c03d050 con 0x56434c2d7f30 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 10 client.11178 fetch_fsmap learned FSMap version 105 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 -- 172.21.15.133:0/2656588317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x56434c0af3a0 con 0x56434c2d7f30 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd71b7fe640 1 -- 172.21.15.133:0/2656588317 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c026070 con 0x56434c2d7f30 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd71b7fe640 1 -- 172.21.15.133:0/2656588317 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd72c040020 con 0x56434c2d7f30 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 10 client.11178 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 10 client.11178 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 20 client.11178 populate_metadata read hostname 'smithi133' 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56434c2dd160 0x56434c332dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 4 client.11178 mds_command: new command op to 11037 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:14.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd74630c000 1 -- 172.21.15.133:0/2656588317 --> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56434c0c2a50 con 0x56434c2dd160 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd73bfff640 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56434c2dd160 0x56434c332dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd71b7fe640 10 client.11178 ms_handle_connect on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.325+0000 7fd73bfff640 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56434c2dd160 0x56434c332dc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=433b8e0c68e87f07 server_cookie=9092543d2b69f078 in_seq=0 out_seq=0 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd71b7fe640 1 -- 172.21.15.133:0/2656588317 <== mds.0 v2:172.21.15.162:6832/2658062772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56434c0c2a50 con 0x56434c2dd160 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd71b7fe640 10 client.11178 handle_command_reply: tid=0 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 1 client.11178 shutdown 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 1 -- 172.21.15.133:0/2656588317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c2d7f30 msgr2=0x56434c328d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c2d7f30 0x56434c328d60 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fd72c008010 tx=0x7fd72c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 1 -- 172.21.15.133:0/2656588317 shutdown_connections 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56434c2dd160 0x56434c332dc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c322240 0x56434c2dbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c321920 0x56434c3292a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd71b7fe640 0 client.11178 ms_handle_reset on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:14.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 1 --2- 172.21.15.133:0/2656588317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c2d7f30 0x56434c328d60 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 1 -- 172.21.15.133:0/2656588317 >> 172.21.15.133:0/2656588317 conn(0x56434c2de2a0 msgr2=0x56434c2d69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 1 -- 172.21.15.133:0/2656588317 shutdown_connections 2024-04-25T10:16:14.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 1 -- 172.21.15.133:0/2656588317 wait complete. 2024-04-25T10:16:14.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.329+0000 7fd74630c000 20 client.11178 trim_cache size 0 max 16384 2024-04-25T10:16:14.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.341+0000 7fd74630c000 1 Processor -- start 2024-04-25T10:16:14.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.341+0000 7fd74630c000 1 -- start start 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c2d7f30 0x56434c2db330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c2db870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c322240 0x56434c361ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c138e10 con 0x56434c321920 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c138210 con 0x56434c322240 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c138810 con 0x56434c2d7f30 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c2db870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c2db870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55850/0 (socket says 172.21.15.133:55850) 2024-04-25T10:16:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd73bfff640 1 -- 172.21.15.133:0/1444642611 learned_addr learned my addr 172.21.15.133:0/1444642611 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74173c640 1 --2- 172.21.15.133:0/1444642611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c322240 0x56434c361ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd73bfff640 1 -- 172.21.15.133:0/1444642611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c2d7f30 msgr2=0x56434c2db330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd740f3b640 1 --2- 172.21.15.133:0/1444642611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c2d7f30 0x56434c2db330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd73bfff640 1 --2- 172.21.15.133:0/1444642611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c2d7f30 0x56434c2db330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd73bfff640 1 -- 172.21.15.133:0/1444642611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c322240 msgr2=0x56434c361ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd73bfff640 1 --2- 172.21.15.133:0/1444642611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c322240 0x56434c361ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd73bfff640 1 -- 172.21.15.133:0/1444642611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56434c322070 con 0x56434c321920 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd73bfff640 1 --2- 172.21.15.133:0/1444642611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c2db870 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fd7300097c0 tx=0x7fd730004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288b87b71ec88cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd721ffb640 1 -- 172.21.15.133:0/1444642611 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7300c5070 con 0x56434c321920 2024-04-25T10:16:14.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd721ffb640 1 -- 172.21.15.133:0/1444642611 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd730003620 con 0x56434c321920 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd721ffb640 1 -- 172.21.15.133:0/1444642611 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7300b0040 con 0x56434c321920 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- 172.21.15.133:0/1444642611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 msgr2=0x56434c2db870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 --2- 172.21.15.133:0/1444642611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c2db870 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fd7300097c0 tx=0x7fd730004740 comp rx=0 tx=0).stop 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- 172.21.15.133:0/1444642611 shutdown_connections 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 --2- 172.21.15.133:0/1444642611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c322240 0x56434c361ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 --2- 172.21.15.133:0/1444642611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c321920 0x56434c2db870 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 --2- 172.21.15.133:0/1444642611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c2d7f30 0x56434c2db330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- 172.21.15.133:0/1444642611 >> 172.21.15.133:0/1444642611 conn(0x56434c2de2a0 msgr2=0x56434c30e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- 172.21.15.133:0/1444642611 shutdown_connections 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- 172.21.15.133:0/1444642611 wait complete. 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- start start 2024-04-25T10:16:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 Processor -- start 2024-04-25T10:16:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c2d7f30 0x56434c3659e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c321920 0x56434c32eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c322240 0x56434c32f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56434c12b3d0 con 0x56434c322240 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56434c1e2d50 con 0x56434c321920 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56434c0c1980 con 0x56434c2d7f30 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74630c000 10 client.0 _ll_register_callbacks cb 0x7fd72c0097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74173c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c322240 0x56434c32f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74173c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c322240 0x56434c32f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55866/0 (socket says 172.21.15.133:55866) 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74173c640 1 -- 172.21.15.133:0/3398823774 learned_addr learned my addr 172.21.15.133:0/3398823774 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd740f3b640 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c2d7f30 0x56434c3659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd73bfff640 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c321920 0x56434c32eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74173c640 1 -- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c2d7f30 msgr2=0x56434c3659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74173c640 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c2d7f30 0x56434c3659e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74173c640 1 -- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c321920 msgr2=0x56434c32eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.345+0000 7fd74173c640 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c321920 0x56434c32eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74173c640 1 -- 172.21.15.133:0/3398823774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56434c1a8600 con 0x56434c322240 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd73bfff640 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c321920 0x56434c32eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74173c640 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c322240 0x56434c32f000 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fd7340519b0 tx=0x7fd734055610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58432c71e36507f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd71b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd71b7fe640 1 -- 172.21.15.133:0/3398823774 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73405dbb0 con 0x56434c322240 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 -- 172.21.15.133:0/3398823774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd730003620 con 0x56434c322240 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd71b7fe640 1 -- 172.21.15.133:0/3398823774 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7340591b0 con 0x56434c322240 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd71b7fe640 1 -- 172.21.15.133:0/3398823774 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7340593b0 con 0x56434c322240 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 10 client.11184 fetch_fsmap learned FSMap version 105 2024-04-25T10:16:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 -- 172.21.15.133:0/3398823774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x56434c0bb9b0 con 0x56434c322240 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd71b7fe640 1 -- 172.21.15.133:0/3398823774 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd730003620 con 0x56434c322240 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd71b7fe640 1 -- 172.21.15.133:0/3398823774 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd73405d720 con 0x56434c322240 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 10 client.11184 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 10 client.11184 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 20 client.11184 populate_metadata read hostname 'smithi133' 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56434c366310 0x56434c3666f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 4 client.11184 mds_command: new command op to 11037 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 -- 172.21.15.133:0/3398823774 --> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56434c1342e0 con 0x56434c366310 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd740f3b640 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56434c366310 0x56434c3666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd71b7fe640 10 client.11184 ms_handle_connect on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd740f3b640 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56434c366310 0x56434c3666f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f1636c9da77a31e server_cookie=33fd744a39c675aa in_seq=0 out_seq=0 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd71b7fe640 1 -- 172.21.15.133:0/3398823774 <== mds.0 v2:172.21.15.162:6832/2658062772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x56434c1342e0 con 0x56434c366310 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd71b7fe640 10 client.11184 handle_command_reply: tid=0 2024-04-25T10:16:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 client.11184 shutdown 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 -- 172.21.15.133:0/3398823774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c322240 msgr2=0x56434c32f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c322240 0x56434c32f000 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fd7340519b0 tx=0x7fd734055610 comp rx=0 tx=0).stop 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 -- 172.21.15.133:0/3398823774 shutdown_connections 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56434c366310 0x56434c3666f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56434c322240 0x56434c32f000 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56434c321920 0x56434c32eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd71b7fe640 0 client.11184 ms_handle_reset on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 --2- 172.21.15.133:0/3398823774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56434c2d7f30 0x56434c3659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 -- 172.21.15.133:0/3398823774 >> 172.21.15.133:0/3398823774 conn(0x56434c2de2a0 msgr2=0x56434c2da320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 -- 172.21.15.133:0/3398823774 shutdown_connections 2024-04-25T10:16:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 1 -- 172.21.15.133:0/3398823774 wait complete. 2024-04-25T10:16:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74630c000 20 client.11184 trim_cache size 0 max 16384 2024-04-25T10:16:14.358 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:16:14.358 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:16:14.358 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:16:14.358 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 11037, 2024-04-25T10:16:14.358 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:16:14.358 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:16:14.358 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:16:14.359 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 4.7093082280000003, 2024-04-25T10:16:14.359 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 105, 2024-04-25T10:16:14.359 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 169, 2024-04-25T10:16:14.359 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 169, 2024-04-25T10:16:14.359 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 10.088205122 2024-04-25T10:16:14.359 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:16:14.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 -- 172.21.15.133:0/1103319537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd728066f90 msgr2=0x7fd728069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd728066f90 0x7fd728069450 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009580 tx=0x7fd72c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:14.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 -- 172.21.15.133:0/1103319537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd73c145170 msgr2=0x7fd73c129880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd73c145170 0x7fd73c129880 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7fd73400ca80 tx=0x7fd73400cf40 comp rx=0 tx=0).stop 2024-04-25T10:16:14.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 -- 172.21.15.133:0/1103319537 shutdown_connections 2024-04-25T10:16:14.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd728066f90 0x7fd728069450 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd73c145170 0x7fd73c129880 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd73c0cdd60 0x7fd73c141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 --2- 172.21.15.133:0/1103319537 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd73c06e4d0 0x7fd73c143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 -- 172.21.15.133:0/1103319537 >> 172.21.15.133:0/1103319537 conn(0x7fd73c066f70 msgr2=0x7fd73c0b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 -- 172.21.15.133:0/1103319537 shutdown_connections 2024-04-25T10:16:14.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.349+0000 7fd74319f640 1 -- 172.21.15.133:0/1103319537 wait complete. 2024-04-25T10:16:14.367 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:16:14.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.469+0000 7f4713c6d640 1 Processor -- start 2024-04-25T10:16:14.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.469+0000 7f4713c6d640 1 -- start start 2024-04-25T10:16:14.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4713c6d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f470c15b9c0 0x7f470c072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4713c6d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f470c0ccf90 0x7f470c06fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4713c6d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470c070700 con 0x7f470c0702d0 2024-04-25T10:16:14.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4713c6d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c15bda0 con 0x7f470c15b9c0 2024-04-25T10:16:14.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4713c6d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c0cd450 con 0x7f470c0ccf90 2024-04-25T10:16:14.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4711a09640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f470c15b9c0 0x7f470c072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f471220a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f470c0702d0 0x7f470c15f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40974/0 (socket says 172.21.15.133:40974) 2024-04-25T10:16:14.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f471220a640 1 -- 172.21.15.133:0/3440174941 learned_addr learned my addr 172.21.15.133:0/3440174941 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:14.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4711208640 1 --2- 172.21.15.133:0/3440174941 >> v2:172.21.15.162:3300/0 conn(0x7f470c0ccf90 0x7f470c06fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4711a09640 1 -- 172.21.15.133:0/3440174941 >> v2:172.21.15.162:3300/0 conn(0x7f470c0ccf90 msgr2=0x7f470c06fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4711a09640 1 --2- 172.21.15.133:0/3440174941 >> v2:172.21.15.162:3300/0 conn(0x7f470c0ccf90 0x7f470c06fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4711a09640 1 -- 172.21.15.133:0/3440174941 >> v1:172.21.15.133:6789/0 conn(0x7f470c0702d0 legacy=0x7f470c15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4710a07640 1 -- 172.21.15.133:0/3440174941 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972631839 0 0) 0x7f47040014a0 con 0x7f470c0702d0 2024-04-25T10:16:14.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4711a09640 1 -- 172.21.15.133:0/3440174941 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c0cd730 con 0x7f470c15b9c0 2024-04-25T10:16:14.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4711a09640 1 --2- 172.21.15.133:0/3440174941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f470c15b9c0 0x7f470c072cb0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f470c06bad0 tx=0x7f470801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8805d42e503d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.473+0000 7f4710a07640 1 -- 172.21.15.133:0/3440174941 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47080b0070 con 0x7f470c15b9c0 2024-04-25T10:16:14.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4710a07640 1 -- 172.21.15.133:0/3440174941 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47080ab600 con 0x7f470c15b9c0 2024-04-25T10:16:14.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4710a07640 1 -- 172.21.15.133:0/3440174941 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47080ab9d0 con 0x7f470c15b9c0 2024-04-25T10:16:14.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- 172.21.15.133:0/3440174941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f470c15b9c0 msgr2=0x7f470c072cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 --2- 172.21.15.133:0/3440174941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f470c15b9c0 0x7f470c072cb0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f470c06bad0 tx=0x7f470801c3f0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- 172.21.15.133:0/3440174941 shutdown_connections 2024-04-25T10:16:14.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 --2- 172.21.15.133:0/3440174941 >> v2:172.21.15.162:3300/0 conn(0x7f470c0ccf90 0x7f470c06fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 --2- 172.21.15.133:0/3440174941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f470c15b9c0 0x7f470c072cb0 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- 172.21.15.133:0/3440174941 >> 172.21.15.133:0/3440174941 conn(0x7f470c0b2370 msgr2=0x7f470c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- 172.21.15.133:0/3440174941 shutdown_connections 2024-04-25T10:16:14.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- 172.21.15.133:0/3440174941 wait complete. 2024-04-25T10:16:14.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 Processor -- start 2024-04-25T10:16:14.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- start start 2024-04-25T10:16:14.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c0702d0 0x7f470c13d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f470c0ccf90 0x7f470c15dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f470c15b9c0 0x7f470c14bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c1689c0 con 0x7f470c0ccf90 2024-04-25T10:16:14.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c168840 con 0x7f470c0702d0 2024-04-25T10:16:14.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c168b40 con 0x7f470c15b9c0 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4711208640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f470c0ccf90 0x7f470c15dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4711208640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f470c0ccf90 0x7f470c15dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55882/0 (socket says 172.21.15.133:55882) 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4711208640 1 -- 172.21.15.133:0/3509676677 learned_addr learned my addr 172.21.15.133:0/3509676677 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f471220a640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f470c15b9c0 0x7f470c14bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4711a09640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c0702d0 0x7f470c13d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4711208640 1 -- 172.21.15.133:0/3509676677 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f470c15b9c0 msgr2=0x7f470c14bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4711208640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f470c15b9c0 0x7f470c14bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4711208640 1 -- 172.21.15.133:0/3509676677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c0702d0 msgr2=0x7f470c13d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4711208640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c0702d0 0x7f470c13d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4711208640 1 -- 172.21.15.133:0/3509676677 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c15cd40 con 0x7f470c0ccf90 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4711208640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f470c0ccf90 0x7f470c15dde0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f470000d830 tx=0x7f470000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f46fe7fc640 1 -- 172.21.15.133:0/3509676677 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470000b6c0 con 0x7f470c0ccf90 2024-04-25T10:16:14.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f46fe7fc640 1 -- 172.21.15.133:0/3509676677 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4700019070 con 0x7f470c0ccf90 2024-04-25T10:16:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f46fe7fc640 1 -- 172.21.15.133:0/3509676677 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470000bae0 con 0x7f470c0ccf90 2024-04-25T10:16:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- 172.21.15.133:0/3509676677 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c15d020 con 0x7f470c0ccf90 2024-04-25T10:16:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- 172.21.15.133:0/3509676677 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c15d5e0 con 0x7f470c0ccf90 2024-04-25T10:16:14.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.477+0000 7f4713c6d640 1 -- 172.21.15.133:0/3509676677 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c067c70 con 0x7f470c0ccf90 2024-04-25T10:16:14.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.481+0000 7f46fe7fc640 1 -- 172.21.15.133:0/3509676677 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f470001e460 con 0x7f470c0ccf90 2024-04-25T10:16:14.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.481+0000 7f46fe7fc640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f46f0066f60 0x7f46f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.481+0000 7f46fe7fc640 1 -- 172.21.15.133:0/3509676677 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4700085910 con 0x7f470c0ccf90 2024-04-25T10:16:14.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.481+0000 7f4711a09640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f46f0066f60 0x7f46f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.481+0000 7f4711a09640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f46f0066f60 0x7f46f0069420 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f47080ad000 tx=0x7f47080a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.485+0000 7f46fe7fc640 1 -- 172.21.15.133:0/3509676677 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4700056790 con 0x7f470c0ccf90 2024-04-25T10:16:14.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.665+0000 7f4713c6d640 1 -- 172.21.15.133:0/3509676677 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f470c070ba0 con 0x7f470c0ccf90 2024-04-25T10:16:14.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.665+0000 7f46fe7fc640 1 -- 172.21.15.133:0/3509676677 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4700056090 con 0x7f470c0ccf90 2024-04-25T10:16:14.666 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:16:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 -- 172.21.15.133:0/3509676677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f46f0066f60 msgr2=0x7f46f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f46f0066f60 0x7f46f0069420 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f47080ad000 tx=0x7f47080a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 -- 172.21.15.133:0/3509676677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f470c0ccf90 msgr2=0x7f470c15dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f470c0ccf90 0x7f470c15dde0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f470000d830 tx=0x7f470000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 -- 172.21.15.133:0/3509676677 shutdown_connections 2024-04-25T10:16:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f470c15b9c0 0x7f470c14bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f46f0066f60 0x7f46f0069420 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f470c0ccf90 0x7f470c15dde0 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 --2- 172.21.15.133:0/3509676677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f470c0702d0 0x7f470c13d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 -- 172.21.15.133:0/3509676677 >> 172.21.15.133:0/3509676677 conn(0x7f470c0b2370 msgr2=0x7f470c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 -- 172.21.15.133:0/3509676677 shutdown_connections 2024-04-25T10:16:14.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:14.669+0000 7f4713c6d640 1 -- 172.21.15.133:0/3509676677 wait complete. 2024-04-25T10:16:14.684 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:16:14.684 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:16:14.684 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:16:14.684 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:16:14.684 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:14.684 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:16:14.684 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:16:14.684 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:16:14.684 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:14.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.685 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:16:14.689 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:16:14.690 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41657sec preferred_lft 41657sec 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:16:14.691 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:14.692 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:16:14.692 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.692 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:16:14.692 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.692 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:14.692 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:14.692 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:16:14.692 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.693 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:14.693 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:14.693 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:16:14.693 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:14.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.693 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:16:14.742 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:16:14.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.743 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:16:14.793 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:16:14.793 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:16:14.794 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:16:14.794 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:16:14.794 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:16:14.794 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:16:14.794 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:16:14.794 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:14.794 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:14.794 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:16:14.794 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:16:14.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.795 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:16:14.795 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:16:14.795 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:16:14.795 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:16:14.795 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:16:14.795 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:16:14.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.868 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:16:14.918 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:16:14.918 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:16:14.918 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:14.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:14.918 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:16:14.973 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:16:14.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:14.974 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:15.021 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:15.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:15.022 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:15.069 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:16:15.130 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:15.179 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:16:15.179 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:16:15.179 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:16:15.179 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:16:15.180 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:15.181 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:15.182 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:15.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:15.183 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:16:15.229 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:16:15.229 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:16:15.274 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:16:15.289 INFO:tasks.ceph.mds.d.smithi162.stderr:2024-04-25T10:16:15.288+0000 7f49a5cba640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:15.290 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:15.339 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:16:15.340 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:15.341 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:15.342 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:15.342 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:15.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:15.342 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:16:15.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:16:15.353+0000 7f1f986f2580 -1 init, newargv = 0x55f557655740 newargc=15 2024-04-25T10:16:15.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[89319]: starting ceph client 2024-04-25T10:16:15.360 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[89319]: starting fuse 2024-04-25T10:16:16.459 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:16:16.475 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:16:16.523 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:16:16.523 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:16:16.523 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:16:16.523 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:16:16.523 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:16:16.523 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:16:16.524 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:16.525 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:16.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:16.527 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:16:16.573 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:16:16.573 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:16:16.573 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:16:16.574 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:16:16.574 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:16:16.574 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:16:16.574 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:16:16.574 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:16:16.574 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:16:16.574 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:16:16.574 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:16:16.574 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:16:16.575 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:16:16.575 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:16:16.575 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:16:16.575 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:16:16.575 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:16:16.575 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:16:16.575 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:16:16.575 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:16:16.575 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:16:16.575 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:16:16.576 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:16:16.576 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:16:16.576 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:16:16.576 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:16:16.576 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:16:16.576 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:16:16.576 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:16:16.702 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.89319.asok 2024-04-25T10:16:16.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:16.714 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.89319.asok 2024-04-25T10:16:16.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:16.714 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.89319.asok status 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "89319", 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:16:16.871 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 11199, 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 11199 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2189407753 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:16:16.872 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2189407753 2024-04-25T10:16:16.873 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:16.873 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.11199 192.168.0.1:0/2189407753", 2024-04-25T10:16:16.873 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/2189407753", 2024-04-25T10:16:16.873 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:16:16.873 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 105, 2024-04-25T10:16:16.873 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 169, 2024-04-25T10:16:16.873 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:16:16.873 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:16:16.873 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:16:16.873 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:16:16.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:16.885 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:16:16.885 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:16.891 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:16:16.891 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:16.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:16.891 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:16.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:16.946 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-04-25T10:16:17.008 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:16:17.008 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:17.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:17.058 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:16:17.184 INFO:teuthology.orchestra.run.smithi133.stderr:6+0 records in 2024-04-25T10:16:17.184 INFO:teuthology.orchestra.run.smithi133.stderr:6+0 records out 2024-04-25T10:16:17.184 INFO:teuthology.orchestra.run.smithi133.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0719963 s, 87.4 MB/s 2024-04-25T10:16:17.186 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:17.189 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:17.215 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-04-25T10:16:17.215 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.216 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-04-25T10:16:17.217 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-04-25T10:16:17.218 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.133,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-04-25T10:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:17.220 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 20 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 20 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 10 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-04-25T10:16:17.221 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-25T10:16:17.222 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:17.223 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:17.224 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2024-04-25T10:16:17.225 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2024-04-25T10:16:17.226 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:16:17.226 INFO:teuthology.orchestra.run.smithi133.stdout: mds cache max size = 1000 2024-04-25T10:16:17.226 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2024-04-25T10:16:17.226 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = True 2024-04-25T10:16:17.226 INFO:teuthology.orchestra.run.smithi133.stdout: mds log max segments = 1 2024-04-25T10:16:17.226 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2024-04-25T10:16:17.226 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2024-04-25T10:16:17.226 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-04-25T10:16:17.226 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-04-25T10:16:17.226 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-04-25T10:16:17.227 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-04-25T10:16:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2024-04-25T10:16:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2024-04-25T10:16:17.227 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:17.228 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.229 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.133,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2024-04-25T10:16:17.230 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug trim objects = true 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2024-04-25T10:16:17.231 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2024-04-25T10:16:17.232 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2024-04-25T10:16:17.233 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:17.234 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:17.235 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = "" 2024-04-25T10:16:17.236 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds balancer = 20 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: mds cache max size = 1000 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2024-04-25T10:16:17.237 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = True 2024-04-25T10:16:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: mds log max segments = 1 2024-04-25T10:16:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2024-04-25T10:16:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = False 2024-04-25T10:16:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-04-25T10:16:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-04-25T10:16:17.238 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-04-25T10:16:17.240 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:17.286 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-04-25T10:16:17.311 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:17.312 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.313 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.133,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-04-25T10:16:17.314 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-04-25T10:16:17.315 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 20 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 20 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 10 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:17.316 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:16:17.317 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.318 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:17.319 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-04-25T10:16:17.320 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2024-04-25T10:16:17.321 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2024-04-25T10:16:17.321 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:16:17.321 INFO:teuthology.orchestra.run.smithi133.stdout: mds cache max size = 1000 2024-04-25T10:16:17.321 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2024-04-25T10:16:17.321 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2024-04-25T10:16:17.321 INFO:teuthology.orchestra.run.smithi133.stdout: mds log max segments = 1 2024-04-25T10:16:17.321 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2024-04-25T10:16:17.321 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2024-04-25T10:16:17.321 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-04-25T10:16:17.321 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-04-25T10:16:17.321 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-04-25T10:16:17.322 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-04-25T10:16:17.322 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2024-04-25T10:16:17.322 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:17.322 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2024-04-25T10:16:17.322 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.322 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2024-04-25T10:16:17.322 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.322 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:17.322 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.322 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:17.322 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2024-04-25T10:16:17.323 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.324 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.133,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:17.325 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug trim objects = true 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.326 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:17.327 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2024-04-25T10:16:17.328 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.329 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:17.330 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = "" 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2024-04-25T10:16:17.331 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds balancer = 20 2024-04-25T10:16:17.332 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:16:17.332 INFO:teuthology.orchestra.run.smithi162.stdout: mds cache max size = 1000 2024-04-25T10:16:17.332 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2024-04-25T10:16:17.332 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = False 2024-04-25T10:16:17.332 INFO:teuthology.orchestra.run.smithi162.stdout: mds log max segments = 1 2024-04-25T10:16:17.332 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2024-04-25T10:16:17.332 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = False 2024-04-25T10:16:17.332 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-04-25T10:16:17.332 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-04-25T10:16:17.332 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-04-25T10:16:17.334 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-04-25T10:16:17.334 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:16:17.334 DEBUG:teuthology.orchestra.run.smithi133:> import stat 2024-04-25T10:16:17.334 DEBUG:teuthology.orchestra.run.smithi133:> import json 2024-04-25T10:16:17.334 DEBUG:teuthology.orchestra.run.smithi133:> import sys 2024-04-25T10:16:17.334 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:16:17.334 DEBUG:teuthology.orchestra.run.smithi133:> try: 2024-04-25T10:16:17.334 DEBUG:teuthology.orchestra.run.smithi133:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:16:17.334 DEBUG:teuthology.orchestra.run.smithi133:> except OSError as e: 2024-04-25T10:16:17.334 DEBUG:teuthology.orchestra.run.smithi133:> sys.exit(e.errno) 2024-04-25T10:16:17.334 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:16:17.335 DEBUG:teuthology.orchestra.run.smithi133:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:16:17.335 DEBUG:teuthology.orchestra.run.smithi133:> print(json.dumps( 2024-04-25T10:16:17.335 DEBUG:teuthology.orchestra.run.smithi133:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:16:17.335 DEBUG:teuthology.orchestra.run.smithi133:> indent=2)) 2024-04-25T10:16:17.335 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:16:17.446 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:16:17.446 INFO:teuthology.orchestra.run.smithi133.stdout: "st_mode": 33204, 2024-04-25T10:16:17.446 INFO:teuthology.orchestra.run.smithi133.stdout: "st_ino": 1099511627777, 2024-04-25T10:16:17.446 INFO:teuthology.orchestra.run.smithi133.stdout: "st_dev": 45, 2024-04-25T10:16:17.446 INFO:teuthology.orchestra.run.smithi133.stdout: "st_nlink": 1, 2024-04-25T10:16:17.447 INFO:teuthology.orchestra.run.smithi133.stdout: "st_uid": 1000, 2024-04-25T10:16:17.447 INFO:teuthology.orchestra.run.smithi133.stdout: "st_gid": 1271, 2024-04-25T10:16:17.447 INFO:teuthology.orchestra.run.smithi133.stdout: "st_size": 6291456, 2024-04-25T10:16:17.447 INFO:teuthology.orchestra.run.smithi133.stdout: "st_atime": 1714040177.1103745, 2024-04-25T10:16:17.447 INFO:teuthology.orchestra.run.smithi133.stdout: "st_mtime": 1714040177.1591718, 2024-04-25T10:16:17.447 INFO:teuthology.orchestra.run.smithi133.stdout: "st_ctime": 1714040177.1591718 2024-04-25T10:16:17.447 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:16:17.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:17.456 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:17.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d7483640 1 Processor -- start 2024-04-25T10:16:17.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d7483640 1 -- start start 2024-04-25T10:16:17.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d7483640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2d015e220 0x7fb2d0164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:17.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d7483640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d00ce880 con 0x7fb2d015a1a0 2024-04-25T10:16:17.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d7483640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d006e360 con 0x7fb2d0159dc0 2024-04-25T10:16:17.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d7483640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d00c49e0 con 0x7fb2d015e220 2024-04-25T10:16:17.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d5a20640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb2d015e220 0x7fb2d0164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:17.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d4a1e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb2d015a1a0 0x7fb2d015db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33194/0 (socket says 172.21.15.133:33194) 2024-04-25T10:16:17.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d4a1e640 1 -- 172.21.15.133:0/3803258138 learned_addr learned my addr 172.21.15.133:0/3803258138 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:17.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d5a20640 1 --2- 172.21.15.133:0/3803258138 >> v2:172.21.15.133:3300/0 conn(0x7fb2d015e220 0x7fb2d0164ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60226/0 (socket says 172.21.15.133:60226) 2024-04-25T10:16:17.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d5a20640 1 -- 172.21.15.133:0/3803258138 >> v1:172.21.15.162:6789/0 conn(0x7fb2d0159dc0 legacy=0x7fb2d00cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:17.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d5a20640 1 -- 172.21.15.133:0/3803258138 >> v1:172.21.15.133:6789/0 conn(0x7fb2d015a1a0 legacy=0x7fb2d015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:17.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d5a20640 1 -- 172.21.15.133:0/3803258138 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d00ceb20 con 0x7fb2d015e220 2024-04-25T10:16:17.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.625+0000 7fb2d5a20640 1 --2- 172.21.15.133:0/3803258138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d015e220 0x7fb2d0164ea0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80097c0 tx=0x7fb2c801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b066187d3f69e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:17.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2cffff640 1 -- 172.21.15.133:0/3803258138 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80b0070 con 0x7fb2d015e220 2024-04-25T10:16:17.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2cffff640 1 -- 172.21.15.133:0/3803258138 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c80ab600 con 0x7fb2d015e220 2024-04-25T10:16:17.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2cffff640 1 -- 172.21.15.133:0/3803258138 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80ab9d0 con 0x7fb2d015e220 2024-04-25T10:16:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 -- 172.21.15.133:0/3803258138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d015e220 msgr2=0x7fb2d0164ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 --2- 172.21.15.133:0/3803258138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d015e220 0x7fb2d0164ea0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80097c0 tx=0x7fb2c801c460 comp rx=0 tx=0).stop 2024-04-25T10:16:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 -- 172.21.15.133:0/3803258138 shutdown_connections 2024-04-25T10:16:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 --2- 172.21.15.133:0/3803258138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d015e220 0x7fb2d0164ea0 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 -- 172.21.15.133:0/3803258138 >> 172.21.15.133:0/3803258138 conn(0x7fb2d0067000 msgr2=0x7fb2d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 -- 172.21.15.133:0/3803258138 shutdown_connections 2024-04-25T10:16:17.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 -- 172.21.15.133:0/3803258138 wait complete. 2024-04-25T10:16:17.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 Processor -- start 2024-04-25T10:16:17.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 -- start start 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2d0159dc0 0x7fb2d00c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2d015e220 0x7fb2d0152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d0152f90 0x7fb2d00c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0168680 con 0x7fb2d0152f90 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0168500 con 0x7fb2d0159dc0 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d7483640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0168800 con 0x7fb2d015e220 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d5a20640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d0152f90 0x7fb2d00c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d5a20640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d0152f90 0x7fb2d00c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60242/0 (socket says 172.21.15.133:60242) 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d5a20640 1 -- 172.21.15.133:0/3193675497 learned_addr learned my addr 172.21.15.133:0/3193675497 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d5a20640 1 -- 172.21.15.133:0/3193675497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2d015e220 msgr2=0x7fb2d0152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d5a20640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2d015e220 0x7fb2d0152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d521f640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2d0159dc0 0x7fb2d00c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:17.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d5a20640 1 -- 172.21.15.133:0/3193675497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2d0159dc0 msgr2=0x7fb2d00c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:17.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d5a20640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2d0159dc0 0x7fb2d00c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:17.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d5a20640 1 -- 172.21.15.133:0/3193675497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0155dd0 con 0x7fb2d0152f90 2024-04-25T10:16:17.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2d5a20640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d0152f90 0x7fb2d00c7510 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8019070 tx=0x7fb2c801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:17.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2cdffb640 1 -- 172.21.15.133:0/3193675497 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80b0050 con 0x7fb2d0152f90 2024-04-25T10:16:17.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2cdffb640 1 -- 172.21.15.133:0/3193675497 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c80bc070 con 0x7fb2d0152f90 2024-04-25T10:16:17.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.629+0000 7fb2cdffb640 1 -- 172.21.15.133:0/3193675497 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80b5830 con 0x7fb2d0152f90 2024-04-25T10:16:17.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.633+0000 7fb2d7483640 1 -- 172.21.15.133:0/3193675497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d01560b0 con 0x7fb2d0152f90 2024-04-25T10:16:17.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.633+0000 7fb2d7483640 1 -- 172.21.15.133:0/3193675497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d0156670 con 0x7fb2d0152f90 2024-04-25T10:16:17.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.633+0000 7fb2d7483640 1 -- 172.21.15.133:0/3193675497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d0068ef0 con 0x7fb2d0152f90 2024-04-25T10:16:17.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.633+0000 7fb2cdffb640 1 -- 172.21.15.133:0/3193675497 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2c80b59d0 con 0x7fb2d0152f90 2024-04-25T10:16:17.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.633+0000 7fb2cdffb640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb2bc066f60 0x7fb2bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:17.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.633+0000 7fb2cdffb640 1 -- 172.21.15.133:0/3193675497 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb2c811fa60 con 0x7fb2d0152f90 2024-04-25T10:16:17.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.633+0000 7fb2d521f640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb2bc066f60 0x7fb2bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:17.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.633+0000 7fb2d521f640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb2bc066f60 0x7fb2bc069420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0004450 tx=0x7fb2c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:17.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.637+0000 7fb2cdffb640 1 -- 172.21.15.133:0/3193675497 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2c8016050 con 0x7fb2d0152f90 2024-04-25T10:16:17.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.833+0000 7fb2d7483640 1 -- 172.21.15.133:0/3193675497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2d00cc970 con 0x7fb2d0152f90 2024-04-25T10:16:17.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.833+0000 7fb2cdffb640 1 -- 172.21.15.133:0/3193675497 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4962 (secure 0 0 0) 0x7fb2c80f0a00 con 0x7fb2d0152f90 2024-04-25T10:16:17.834 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:17.835 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1498957356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1498957356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1498957356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/883424012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":883424012},{"type":"v1","addr":"172.21.15.133:6835","nonce":883424012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2040533186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2040533186},{"type":"v1","addr":"172.21.15.133:6837","nonce":2040533186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:08.983208+0000","modified":"2024-04-25T10:16:11.017848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11037":{"gid":11037,"name":"d","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/2658062772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2658062772},{"type":"v1","addr":"172.21.15.162:6833","nonce":2658062772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:16:17.835 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 105 2024-04-25T10:16:17.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 -- 172.21.15.133:0/3193675497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb2bc066f60 msgr2=0x7fb2bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:17.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb2bc066f60 0x7fb2bc069420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0004450 tx=0x7fb2c0009040 comp rx=0 tx=0).stop 2024-04-25T10:16:17.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 -- 172.21.15.133:0/3193675497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d0152f90 msgr2=0x7fb2d00c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:17.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d0152f90 0x7fb2d00c7510 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8019070 tx=0x7fb2c801aca0 comp rx=0 tx=0).stop 2024-04-25T10:16:17.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 -- 172.21.15.133:0/3193675497 shutdown_connections 2024-04-25T10:16:17.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb2bc066f60 0x7fb2bc069420 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:17.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d0152f90 0x7fb2d00c7510 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:17.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb2d015e220 0x7fb2d0152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:17.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 --2- 172.21.15.133:0/3193675497 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb2d0159dc0 0x7fb2d00c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:17.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 -- 172.21.15.133:0/3193675497 >> 172.21.15.133:0/3193675497 conn(0x7fb2d0067000 msgr2=0x7fb2d015bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:17.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 -- 172.21.15.133:0/3193675497 shutdown_connections 2024-04-25T10:16:17.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:17.837+0000 7fb2d7483640 1 -- 172.21.15.133:0/3193675497 wait complete. 2024-04-25T10:16:17.854 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T10:16:18.011 INFO:teuthology.orchestra.run.smithi162.stdout:{"message":"","return_code":0} 2024-04-25T10:16:18.012 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:16:18.012 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent '' 2024-04-25T10:16:18.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 Processor -- start 2024-04-25T10:16:18.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 -- start start 2024-04-25T10:16:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561057ee9160 0x561057f90a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561057ee9570 0x561057ee6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 0x561057ee7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561057e5e9f0 con 0x561057f8f520 2024-04-25T10:16:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561057e62950 con 0x561057ee9160 2024-04-25T10:16:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561057eeb920 con 0x561057ee9570 2024-04-25T10:16:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde577c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 0x561057ee7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde577c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 0x561057ee7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60248/0 (socket says 172.21.15.133:60248) 2024-04-25T10:16:18.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde577c640 1 -- 172.21.15.133:0/1552726236 learned_addr learned my addr 172.21.15.133:0/1552726236 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde4f7b640 1 --2- 172.21.15.133:0/1552726236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561057ee9160 0x561057f90a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde477a640 1 --2- 172.21.15.133:0/1552726236 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561057ee9570 0x561057ee6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde577c640 1 -- 172.21.15.133:0/1552726236 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561057ee9570 msgr2=0x561057ee6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde577c640 1 --2- 172.21.15.133:0/1552726236 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561057ee9570 0x561057ee6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde577c640 1 -- 172.21.15.133:0/1552726236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561057ee9160 msgr2=0x561057f90a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde577c640 1 --2- 172.21.15.133:0/1552726236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561057ee9160 0x561057f90a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde577c640 1 -- 172.21.15.133:0/1552726236 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561057f91380 con 0x561057f8f520 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde577c640 1 --2- 172.21.15.133:0/1552726236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 0x561057ee7320 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7efddc001690 tx=0x7efddc001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aa5f37ddfc97cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde3778640 1 -- 172.21.15.133:0/1552726236 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc004530 con 0x561057f8f520 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde3778640 1 -- 172.21.15.133:0/1552726236 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc0046d0 con 0x561057f8f520 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde3778640 1 -- 172.21.15.133:0/1552726236 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc004ac0 con 0x561057f8f520 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 -- 172.21.15.133:0/1552726236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 msgr2=0x561057ee7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 --2- 172.21.15.133:0/1552726236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 0x561057ee7320 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7efddc001690 tx=0x7efddc001b50 comp rx=0 tx=0).stop 2024-04-25T10:16:18.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 -- 172.21.15.133:0/1552726236 shutdown_connections 2024-04-25T10:16:18.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 --2- 172.21.15.133:0/1552726236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 0x561057ee7320 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 --2- 172.21.15.133:0/1552726236 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561057ee9570 0x561057ee6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 --2- 172.21.15.133:0/1552726236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561057ee9160 0x561057f90a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 -- 172.21.15.133:0/1552726236 >> 172.21.15.133:0/1552726236 conn(0x561057ebe020 msgr2=0x561057f8f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 -- 172.21.15.133:0/1552726236 shutdown_connections 2024-04-25T10:16:18.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 -- 172.21.15.133:0/1552726236 wait complete. 2024-04-25T10:16:18.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 Processor -- start 2024-04-25T10:16:18.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.057+0000 7efde745d080 1 -- start start 2024-04-25T10:16:18.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde745d080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561057ee9160 0x561057f8e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde745d080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561057ee9570 0x561058023400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde745d080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 0x561058023940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde745d080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561057f918c0 con 0x561057f8f520 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde745d080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561057f91620 con 0x561057ee9160 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde745d080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561057f91b60 con 0x561057ee9570 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde577c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 0x561058023940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde577c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 0x561058023940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60264/0 (socket says 172.21.15.133:60264) 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde577c640 1 -- 172.21.15.133:0/2170229822 learned_addr learned my addr 172.21.15.133:0/2170229822 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde4f7b640 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561057ee9160 0x561057f8e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde477a640 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561057ee9570 0x561058023400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde577c640 1 -- 172.21.15.133:0/2170229822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561057ee9570 msgr2=0x561058023400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde577c640 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561057ee9570 0x561058023400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde577c640 1 -- 172.21.15.133:0/2170229822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561057ee9160 msgr2=0x561057f8e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde577c640 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561057ee9160 0x561057f8e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde577c640 1 -- 172.21.15.133:0/2170229822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56105802aea0 con 0x561057f8f520 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde577c640 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 0x561058023940 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7efddc002050 tx=0x7efddc001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde2776640 1 -- 172.21.15.133:0/2170229822 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc018040 con 0x561057f8f520 2024-04-25T10:16:18.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde2776640 1 -- 172.21.15.133:0/2170229822 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc022070 con 0x561057f8f520 2024-04-25T10:16:18.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde745d080 1 -- 172.21.15.133:0/2170229822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56105802b140 con 0x561057f8f520 2024-04-25T10:16:18.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde2776640 1 -- 172.21.15.133:0/2170229822 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc013660 con 0x561057f8f520 2024-04-25T10:16:18.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde745d080 1 -- 172.21.15.133:0/2170229822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56105802b680 con 0x561057f8f520 2024-04-25T10:16:18.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde2776640 1 -- 172.21.15.133:0/2170229822 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efddc010070 con 0x561057f8f520 2024-04-25T10:16:18.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde2776640 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efdc4065940 0x7efdc4067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde2776640 1 -- 172.21.15.133:0/2170229822 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efddc085190 con 0x561057f8f520 2024-04-25T10:16:18.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde4f7b640 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efdc4065940 0x7efdc4067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde745d080 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x561058028460 0x561058028840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde745d080 1 -- 172.21.15.133:0/2170229822 --> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] -- osd_op(unknown.0.0:1 26.7 26:e2e07930:::10000000001.00000000:head [setxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169) v9 -- 0x561058028d80 con 0x561058028460 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde477a640 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x561058028460 0x561058028840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.061+0000 7efde4f7b640 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efdc4065940 0x7efdc4067e00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7efdd4000f70 tx=0x7efdd40017b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde477a640 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x561058028460 0x561058028840 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde477a640 1 -- 172.21.15.133:0/2170229822 <== osd.2 v2:172.21.15.133:6817/148430844 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr] v169'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7efdcc0b2040 con 0x561058028460 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde745d080 1 -- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x561058028460 msgr2=0x561058028840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde745d080 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x561058028460 0x561058028840 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde745d080 1 -- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efdc4065940 msgr2=0x7efdc4067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde745d080 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efdc4065940 0x7efdc4067e00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7efdd4000f70 tx=0x7efdd40017b0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde745d080 1 -- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 msgr2=0x561058023940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde745d080 1 --2- 172.21.15.133:0/2170229822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561057f8f520 0x561058023940 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7efddc002050 tx=0x7efddc001260 comp rx=0 tx=0).stop 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde577c640 1 -- 172.21.15.133:0/2170229822 reap_dead start 2024-04-25T10:16:18.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde745d080 1 -- 172.21.15.133:0/2170229822 shutdown_connections 2024-04-25T10:16:18.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde745d080 1 -- 172.21.15.133:0/2170229822 >> 172.21.15.133:0/2170229822 conn(0x561057ebe020 msgr2=0x561057f70310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde745d080 1 -- 172.21.15.133:0/2170229822 shutdown_connections 2024-04-25T10:16:18.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.065+0000 7efde745d080 1 -- 172.21.15.133:0/2170229822 wait complete. 2024-04-25T10:16:18.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:18.070 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub start / recursive,repair 2024-04-25T10:16:18.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.189+0000 7f82f85aa640 1 Processor -- start 2024-04-25T10:16:18.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.189+0000 7f82f85aa640 1 -- start start 2024-04-25T10:16:18.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.189+0000 7f82f85aa640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f82f015b8f0 0x7f82f00b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.189+0000 7f82f85aa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82f006ac90 0x7f82f006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.189+0000 7f82f85aa640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f015c060 con 0x7f82f00d0540 2024-04-25T10:16:18.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.189+0000 7f82f85aa640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f015cc30 con 0x7f82f015b8f0 2024-04-25T10:16:18.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.189+0000 7f82f85aa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00b0df0 con 0x7f82f006ac90 2024-04-25T10:16:18.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f6346640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f82f015b8f0 0x7f82f00b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f6346640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f82f015b8f0 0x7f82f00b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60268/0 (socket says 172.21.15.133:60268) 2024-04-25T10:16:18.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f6b47640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f82f00d0540 0x7f82f00d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33202/0 (socket says 172.21.15.133:33202) 2024-04-25T10:16:18.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f6346640 1 -- 172.21.15.133:0/2220481319 learned_addr learned my addr 172.21.15.133:0/2220481319 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f5b45640 1 --2- 172.21.15.133:0/2220481319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82f006ac90 0x7f82f006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f6346640 1 -- 172.21.15.133:0/2220481319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82f006ac90 msgr2=0x7f82f006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f6346640 1 --2- 172.21.15.133:0/2220481319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82f006ac90 0x7f82f006b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f6346640 1 -- 172.21.15.133:0/2220481319 >> v1:172.21.15.133:6789/0 conn(0x7f82f00d0540 legacy=0x7f82f00d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f6346640 1 -- 172.21.15.133:0/2220481319 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f015c300 con 0x7f82f015b8f0 2024-04-25T10:16:18.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f5344640 1 -- 172.21.15.133:0/2220481319 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316631573 0 0) 0x7f82f015c060 con 0x7f82f00d0540 2024-04-25T10:16:18.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f6346640 1 --2- 172.21.15.133:0/2220481319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82f015b8f0 0x7f82f00b17a0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009870 tx=0x7f82ec01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4effa301264ff8eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f5344640 1 -- 172.21.15.133:0/2220481319 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0b0070 con 0x7f82f015b8f0 2024-04-25T10:16:18.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f5344640 1 -- 172.21.15.133:0/2220481319 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec0ab600 con 0x7f82f015b8f0 2024-04-25T10:16:18.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f5344640 1 -- 172.21.15.133:0/2220481319 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0ab9d0 con 0x7f82f015b8f0 2024-04-25T10:16:18.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f85aa640 1 -- 172.21.15.133:0/2220481319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82f015b8f0 msgr2=0x7f82f00b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f85aa640 1 --2- 172.21.15.133:0/2220481319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82f015b8f0 0x7f82f00b17a0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009870 tx=0x7f82ec01c4d0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f85aa640 1 -- 172.21.15.133:0/2220481319 shutdown_connections 2024-04-25T10:16:18.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f85aa640 1 --2- 172.21.15.133:0/2220481319 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82f006ac90 0x7f82f006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f85aa640 1 --2- 172.21.15.133:0/2220481319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82f015b8f0 0x7f82f00b17a0 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f85aa640 1 -- 172.21.15.133:0/2220481319 >> 172.21.15.133:0/2220481319 conn(0x7f82f00726f0 msgr2=0x7f82f00999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f85aa640 1 -- 172.21.15.133:0/2220481319 shutdown_connections 2024-04-25T10:16:18.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.193+0000 7f82f85aa640 1 -- 172.21.15.133:0/2220481319 wait complete. 2024-04-25T10:16:18.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f85aa640 1 Processor -- start 2024-04-25T10:16:18.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f85aa640 1 -- start start 2024-04-25T10:16:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f85aa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82f006ac90 0x7f82f00b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f85aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82f00d0540 0x7f82f009a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f85aa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82f015b8f0 0x7f82f00c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f85aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00b1570 con 0x7f82f00d0540 2024-04-25T10:16:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f85aa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00b13f0 con 0x7f82f006ac90 2024-04-25T10:16:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f85aa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00cc1e0 con 0x7f82f015b8f0 2024-04-25T10:16:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f5b45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82f00d0540 0x7f82f009a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f5b45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82f00d0540 0x7f82f009a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60274/0 (socket says 172.21.15.133:60274) 2024-04-25T10:16:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f5b45640 1 -- 172.21.15.133:0/342284879 learned_addr learned my addr 172.21.15.133:0/342284879 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f6346640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82f006ac90 0x7f82f00b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f6b47640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82f015b8f0 0x7f82f00c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f5b45640 1 -- 172.21.15.133:0/342284879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82f015b8f0 msgr2=0x7f82f00c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f5b45640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82f015b8f0 0x7f82f00c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f5b45640 1 -- 172.21.15.133:0/342284879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82f006ac90 msgr2=0x7f82f00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f5b45640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82f006ac90 0x7f82f00b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f5b45640 1 -- 172.21.15.133:0/342284879 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f00b61c0 con 0x7f82f00d0540 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f5b45640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82f00d0540 0x7f82f009a3b0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f82e000d620 tx=0x7f82e000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82e6ffd640 1 -- 172.21.15.133:0/342284879 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e000b620 con 0x7f82f00d0540 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82e6ffd640 1 -- 172.21.15.133:0/342284879 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e0019070 con 0x7f82f00d0540 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f85aa640 1 -- 172.21.15.133:0/342284879 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f00c9af0 con 0x7f82f00d0540 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82f85aa640 1 -- 172.21.15.133:0/342284879 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f00ca0b0 con 0x7f82f00d0540 2024-04-25T10:16:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.197+0000 7f82e6ffd640 1 -- 172.21.15.133:0/342284879 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e000ba20 con 0x7f82f00d0540 2024-04-25T10:16:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82e6ffd640 1 -- 172.21.15.133:0/342284879 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82e000bbc0 con 0x7f82f00d0540 2024-04-25T10:16:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82e6ffd640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82d4066f90 0x7f82d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82e6ffd640 1 -- 172.21.15.133:0/342284879 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f82e0085520 con 0x7f82f00d0540 2024-04-25T10:16:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6346640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82d4066f90 0x7f82d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6346640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82d4066f90 0x7f82d4069450 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0ad000 tx=0x7f82ec0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 Processor -- start 2024-04-25T10:16:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 -- start start 2024-04-25T10:16:18.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a75b0 0x5623ff35cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff35d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ff243f40 con 0x5623ff3a6c90 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ff1ab070 con 0x5623ff3a75b0 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623ff1331a0 con 0x5623ff35d290 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6346640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6346640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3a7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60290/0 (socket says 172.21.15.133:60290) 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6346640 1 -- 172.21.15.133:0/3936449520 learned_addr learned my addr 172.21.15.133:0/3936449520 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6b47640 1 --2- 172.21.15.133:0/3936449520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff35d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f5b45640 1 --2- 172.21.15.133:0/3936449520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a75b0 0x5623ff35cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6346640 1 -- 172.21.15.133:0/3936449520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 msgr2=0x5623ff35d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6346640 1 --2- 172.21.15.133:0/3936449520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff35d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6346640 1 -- 172.21.15.133:0/3936449520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a75b0 msgr2=0x5623ff35cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6346640 1 --2- 172.21.15.133:0/3936449520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a75b0 0x5623ff35cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6346640 1 -- 172.21.15.133:0/3936449520 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623ff131510 con 0x5623ff3a6c90 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82f6346640 1 --2- 172.21.15.133:0/3936449520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3a7070 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f82ec097da0 tx=0x7f82ec004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe60f02722f0098 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82dbfff640 1 -- 172.21.15.133:0/3936449520 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0c7070 con 0x5623ff3a6c90 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82dbfff640 1 -- 172.21.15.133:0/3936449520 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec0abe10 con 0x5623ff3a6c90 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82dbfff640 1 -- 172.21.15.133:0/3936449520 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0b0040 con 0x5623ff3a6c90 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 -- 172.21.15.133:0/3936449520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 msgr2=0x5623ff3a7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 --2- 172.21.15.133:0/3936449520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3a7070 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f82ec097da0 tx=0x7f82ec004740 comp rx=0 tx=0).stop 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 -- 172.21.15.133:0/3936449520 shutdown_connections 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 --2- 172.21.15.133:0/3936449520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff35d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 --2- 172.21.15.133:0/3936449520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a75b0 0x5623ff35cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 --2- 172.21.15.133:0/3936449520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3a7070 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 -- 172.21.15.133:0/3936449520 >> 172.21.15.133:0/3936449520 conn(0x5623ff359240 msgr2=0x5623ff35b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 -- 172.21.15.133:0/3936449520 shutdown_connections 2024-04-25T10:16:18.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.201+0000 7f82fb717000 1 -- 172.21.15.133:0/3936449520 wait complete. 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 -- start start 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 Processor -- start 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff367e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a6c90 0x5623ff368370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff3688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ff1656c0 con 0x5623ff3a75b0 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ff1a7110 con 0x5623ff3a6c90 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623ff261f20 con 0x5623ff35d290 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 10 client.0 _ll_register_callbacks cb 0x5623ff348570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6b47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff3688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6b47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff3688b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60306/0 (socket says 172.21.15.133:60306) 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6b47640 1 -- 172.21.15.133:0/3810492281 learned_addr learned my addr 172.21.15.133:0/3810492281 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f5b45640 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a6c90 0x5623ff368370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6346640 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff367e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6b47640 1 -- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 msgr2=0x5623ff367e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6b47640 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff367e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6b47640 1 -- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a6c90 msgr2=0x5623ff368370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6b47640 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a6c90 0x5623ff368370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6b47640 1 -- 172.21.15.133:0/3810492281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623ff1333b0 con 0x5623ff3a75b0 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6b47640 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff3688b0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f82e8007ba0 tx=0x7f82e8001900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b4dfd836d8492d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82d97fa640 1 -- 172.21.15.133:0/3810492281 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8013570 con 0x5623ff3a75b0 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82d97fa640 1 -- 172.21.15.133:0/3810492281 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f82e8013710 con 0x5623ff3a75b0 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 -- 172.21.15.133:0/3810492281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82ec0abe10 con 0x5623ff3a75b0 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82d97fa640 1 -- 172.21.15.133:0/3810492281 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82e8021050 con 0x5623ff3a75b0 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 10 client.11232 fetch_fsmap learned FSMap version 105 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 -- 172.21.15.133:0/3810492281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x5623ff22ce00 con 0x5623ff3a75b0 2024-04-25T10:16:18.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82d97fa640 1 -- 172.21.15.133:0/3810492281 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec0abe10 con 0x5623ff3a75b0 2024-04-25T10:16:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82d97fa640 1 -- 172.21.15.133:0/3810492281 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f82e8024020 con 0x5623ff3a75b0 2024-04-25T10:16:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 10 client.11232 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:16:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 10 client.11232 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:16:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 20 client.11232 populate_metadata read hostname 'smithi133' 2024-04-25T10:16:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x5623ff369150 0x5623ff3b8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 4 client.11232 mds_command: new command op to 11037 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82fb717000 1 -- 172.21.15.133:0/3810492281 --> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623ff146d60 con 0x5623ff369150 2024-04-25T10:16:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6346640 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x5623ff369150 0x5623ff3b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82f6346640 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x5623ff369150 0x5623ff3b8410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf580ec37f05b42f server_cookie=3e7a2986129097c4 in_seq=0 out_seq=0 2024-04-25T10:16:18.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.205+0000 7f82d97fa640 10 client.11232 ms_handle_connect on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82d97fa640 1 -- 172.21.15.133:0/3810492281 <== mds.0 v2:172.21.15.162:6832/2658062772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623ff146d60 con 0x5623ff369150 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82d97fa640 10 client.11232 handle_command_reply: tid=0 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 1 client.11232 shutdown 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 1 -- 172.21.15.133:0/3810492281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 msgr2=0x5623ff3688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff3688b0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f82e8007ba0 tx=0x7f82e8001900 comp rx=0 tx=0).stop 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 1 -- 172.21.15.133:0/3810492281 shutdown_connections 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x5623ff369150 0x5623ff3b8410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff3688b0 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a6c90 0x5623ff368370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82d97fa640 0 client.11232 ms_handle_reset on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 1 --2- 172.21.15.133:0/3810492281 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff367e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 1 -- 172.21.15.133:0/3810492281 >> 172.21.15.133:0/3810492281 conn(0x5623ff359240 msgr2=0x5623ff35bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 1 -- 172.21.15.133:0/3810492281 shutdown_connections 2024-04-25T10:16:18.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 1 -- 172.21.15.133:0/3810492281 wait complete. 2024-04-25T10:16:18.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.209+0000 7f82fb717000 20 client.11232 trim_cache size 0 max 16384 2024-04-25T10:16:18.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 Processor -- start 2024-04-25T10:16:18.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 -- start start 2024-04-25T10:16:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff3ba420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a6c90 0x5623ff3ba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff40c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d000cc20 con 0x5623ff3a75b0 2024-04-25T10:16:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d000bd40 con 0x5623ff3a6c90 2024-04-25T10:16:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d000b100 con 0x5623ff35d290 2024-04-25T10:16:18.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82f6b47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff40c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82f6b47640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff40c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60310/0 (socket says 172.21.15.133:60310) 2024-04-25T10:16:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82f6b47640 1 -- 172.21.15.133:0/3043961091 learned_addr learned my addr 172.21.15.133:0/3043961091 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82f6b47640 1 -- 172.21.15.133:0/3043961091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 msgr2=0x5623ff3ba420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82f6b47640 1 --2- 172.21.15.133:0/3043961091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff3ba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82f6b47640 1 -- 172.21.15.133:0/3043961091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a6c90 msgr2=0x5623ff3ba960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82f6b47640 1 --2- 172.21.15.133:0/3043961091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a6c90 0x5623ff3ba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82f6b47640 1 -- 172.21.15.133:0/3043961091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623ff202c50 con 0x5623ff3a75b0 2024-04-25T10:16:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82f6b47640 1 --2- 172.21.15.133:0/3043961091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff40c6e0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f82e8005f00 tx=0x7f82e800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04a81b5f5ec1092 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82db7fe640 1 -- 172.21.15.133:0/3043961091 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8001e00 con 0x5623ff3a75b0 2024-04-25T10:16:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82db7fe640 1 -- 172.21.15.133:0/3043961091 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8019070 con 0x5623ff3a75b0 2024-04-25T10:16:18.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82db7fe640 1 -- 172.21.15.133:0/3043961091 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8013570 con 0x5623ff3a75b0 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 -- 172.21.15.133:0/3043961091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 msgr2=0x5623ff40c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 --2- 172.21.15.133:0/3043961091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff40c6e0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f82e8005f00 tx=0x7f82e800e000 comp rx=0 tx=0).stop 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 -- 172.21.15.133:0/3043961091 shutdown_connections 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 --2- 172.21.15.133:0/3043961091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a75b0 0x5623ff40c6e0 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 --2- 172.21.15.133:0/3043961091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a6c90 0x5623ff3ba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 --2- 172.21.15.133:0/3043961091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff3ba420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 -- 172.21.15.133:0/3043961091 >> 172.21.15.133:0/3043961091 conn(0x5623ff359240 msgr2=0x5623ff361d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 -- 172.21.15.133:0/3043961091 shutdown_connections 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 -- 172.21.15.133:0/3043961091 wait complete. 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 -- start start 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.225+0000 7f82fb717000 1 Processor -- start 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff3ba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a75b0 0x5623ff3c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623ff3590c0 con 0x5623ff3a6c90 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d000b700 con 0x5623ff3a75b0 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623ff2676a0 con 0x5623ff35d290 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 10 client.0 _ll_register_callbacks cb 0x7f82ec0ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f5b45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f5b45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60314/0 (socket says 172.21.15.133:60314) 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f5b45640 1 -- 172.21.15.133:0/3918115498 learned_addr learned my addr 172.21.15.133:0/3918115498 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f5b45640 1 -- 172.21.15.133:0/3918115498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 msgr2=0x5623ff3ba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f6b47640 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a75b0 0x5623ff3c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f5b45640 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff3ba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f5b45640 1 -- 172.21.15.133:0/3918115498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a75b0 msgr2=0x5623ff3c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f5b45640 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a75b0 0x5623ff3c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f5b45640 1 -- 172.21.15.133:0/3918115498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623ff173120 con 0x5623ff3a6c90 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f5b45640 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3c72f0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f82e00559a0 tx=0x7f82e00556c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deeac8cf14a3afa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82d97fa640 1 -- 172.21.15.133:0/3918115498 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0055ca0 con 0x5623ff3a6c90 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82d97fa640 1 -- 172.21.15.133:0/3918115498 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f82e005df20 con 0x5623ff3a6c90 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 1 -- 172.21.15.133:0/3918115498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82e8019070 con 0x5623ff3a6c90 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82d97fa640 1 -- 172.21.15.133:0/3918115498 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82e0059df0 con 0x5623ff3a6c90 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 10 client.11238 fetch_fsmap learned FSMap version 105 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 1 -- 172.21.15.133:0/3918115498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x5623ff152170 con 0x5623ff3a6c90 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82d97fa640 1 -- 172.21.15.133:0/3918115498 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8019070 con 0x5623ff3a6c90 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82d97fa640 1 -- 172.21.15.133:0/3918115498 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f82e0080070 con 0x5623ff3a6c90 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 10 client.11238 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 10 client.11238 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 20 client.11238 populate_metadata read hostname 'smithi133' 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x5623ff40af60 0x5623ff40b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 4 client.11238 mds_command: new command op to 11037 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:16:18.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82fb717000 1 -- 172.21.15.133:0/3918115498 --> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x5623ff1b85b0 con 0x5623ff40af60 2024-04-25T10:16:18.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f6346640 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x5623ff40af60 0x5623ff40b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82d97fa640 10 client.11238 ms_handle_connect on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:18.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.229+0000 7f82f6346640 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x5623ff40af60 0x5623ff40b370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899250eaa3b5bbf1 server_cookie=dff19dae5f8b5624 in_seq=0 out_seq=0 2024-04-25T10:16:18.234 INFO:tasks.ceph.mds.d.smithi162.stderr:2024-04-25T10:16:18.232+0000 7f49a1cb2640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:16:17.151178+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={11199=0-12582912@1} caps={11199=pAsxLsXsxFsxcrwb/pAsxXsxFxcwb@4},l=11199 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5637c34c1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(26)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:16:18.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82d97fa640 1 -- 172.21.15.133:0/3918115498 <== mds.0 v2:172.21.15.162:6832/2658062772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5623ff1b85b0 con 0x5623ff40af60 2024-04-25T10:16:18.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82d97fa640 10 client.11238 handle_command_reply: tid=0 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 1 client.11238 shutdown 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 1 -- 172.21.15.133:0/3918115498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 msgr2=0x5623ff3c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3c72f0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f82e00559a0 tx=0x7f82e00556c0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 1 -- 172.21.15.133:0/3918115498 shutdown_connections 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x5623ff40af60 0x5623ff40b370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5623ff3a75b0 0x5623ff3c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ff3a6c90 0x5623ff3c72f0 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82d97fa640 0 client.11238 ms_handle_reset on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 1 --2- 172.21.15.133:0/3918115498 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5623ff35d290 0x5623ff3ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 1 -- 172.21.15.133:0/3918115498 >> 172.21.15.133:0/3918115498 conn(0x5623ff359240 msgr2=0x5623ff35c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 1 -- 172.21.15.133:0/3918115498 shutdown_connections 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 1 -- 172.21.15.133:0/3918115498 wait complete. 2024-04-25T10:16:18.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82fb717000 20 client.11238 trim_cache size 0 max 16384 2024-04-25T10:16:18.237 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:16:18.237 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:16:18.237 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "1daadb6e-5aeb-43cc-ada1-41edbec0d05b", 2024-04-25T10:16:18.237 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:16:18.237 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:16:18.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 -- 172.21.15.133:0/342284879 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82d4066f90 msgr2=0x7f82d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82d4066f90 0x7f82d4069450 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0ad000 tx=0x7f82ec0a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:18.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 -- 172.21.15.133:0/342284879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82f00d0540 msgr2=0x7f82f009a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82f00d0540 0x7f82f009a3b0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f82e000d620 tx=0x7f82e000dae0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 -- 172.21.15.133:0/342284879 shutdown_connections 2024-04-25T10:16:18.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f82f015b8f0 0x7f82f00c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82d4066f90 0x7f82d4069450 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82f00d0540 0x7f82f009a3b0 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 --2- 172.21.15.133:0/342284879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82f006ac90 0x7f82f00b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 -- 172.21.15.133:0/342284879 >> 172.21.15.133:0/342284879 conn(0x7f82f00726f0 msgr2=0x7f82f00999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 -- 172.21.15.133:0/342284879 shutdown_connections 2024-04-25T10:16:18.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.233+0000 7f82f85aa640 1 -- 172.21.15.133:0/342284879 wait complete. 2024-04-25T10:16:18.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:18.250 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:16:18.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.365+0000 7fa8a7d5b640 1 Processor -- start 2024-04-25T10:16:18.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.365+0000 7fa8a7d5b640 1 -- start start 2024-04-25T10:16:18.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a7d5b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a00d2030 0x7fa8a00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a7d5b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a00d24c0 con 0x7fa8a00ce600 2024-04-25T10:16:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a7d5b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a015ebd0 con 0x7fa8a015d9a0 2024-04-25T10:16:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a7d5b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a015eeb0 con 0x7fa8a00d2030 2024-04-25T10:16:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a52f6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa8a00ce600 0x7fa8a00d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33208/0 (socket says 172.21.15.133:33208) 2024-04-25T10:16:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a52f6640 1 -- 172.21.15.133:0/670915293 learned_addr learned my addr 172.21.15.133:0/670915293 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a62f8640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a00d2030 0x7fa8a00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947760097 0 0) 0x7fa8a00d24c0 con 0x7fa8a00ce600 2024-04-25T10:16:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa88c003530 con 0x7fa8a00ce600 2024-04-25T10:16:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 900414023 0 0) 0x7fa88c003530 con 0x7fa8a00ce600 2024-04-25T10:16:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a00d2030 msgr2=0x7fa8a00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a00d2030 0x7fa8a00cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 >> v1:172.21.15.162:6789/0 conn(0x7fa8a015d9a0 legacy=0x7fa8a00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a00d0330 con 0x7fa8a00ce600 2024-04-25T10:16:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fa890001680 con 0x7fa8a00ce600 2024-04-25T10:16:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 >> v1:172.21.15.133:6789/0 conn(0x7fa8a00ce600 legacy=0x7fa8a00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a00d2030 0x7fa88c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa88c003f70 0x7fa88c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa88c004d50 0x7fa88c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a015eeb0 con 0x7fa88c003f70 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c00bb60 con 0x7fa88c004d50 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c00bd60 con 0x7fa8a00d2030 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa88c003f70 0x7fa88c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a62f8640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa88c004d50 0x7fa88c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a00d2030 0x7fa88c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a52f6640 1 -- 172.21.15.133:0/670915293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a00d2030 msgr2=0x7fa88c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a00d2030 0x7fa88c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a52f6640 1 -- 172.21.15.133:0/670915293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa88c004d50 msgr2=0x7fa88c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa88c004d50 0x7fa88c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.369+0000 7fa8a52f6640 1 -- 172.21.15.133:0/670915293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c003530 con 0x7fa88c003f70 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa88c003f70 0x7fa88c004810 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c240 tx=0x7fa89001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2b2eebb38227fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890007d20 con 0x7fa88c003f70 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8900a9070 con 0x7fa88c003f70 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a4af5640 1 -- 172.21.15.133:0/670915293 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8900a5dd0 con 0x7fa88c003f70 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/670915293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa88c003f70 msgr2=0x7fa88c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 --2- 172.21.15.133:0/670915293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa88c003f70 0x7fa88c004810 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c240 tx=0x7fa89001cc50 comp rx=0 tx=0).stop 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a62f8640 1 -- 172.21.15.133:0/670915293 reap_dead start 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/670915293 shutdown_connections 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/670915293 >> 172.21.15.133:0/670915293 conn(0x7fa8a006bf80 msgr2=0x7fa8a006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/670915293 shutdown_connections 2024-04-25T10:16:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/670915293 wait complete. 2024-04-25T10:16:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 Processor -- start 2024-04-25T10:16:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 -- start start 2024-04-25T10:16:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa88c003f70 0x7fa8a014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a015c490 0x7fa8a014ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a015c920 0x7fa8a00c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00b9820 con 0x7fa8a015c490 2024-04-25T10:16:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00b96a0 con 0x7fa88c003f70 2024-04-25T10:16:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a7d5b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a015e590 con 0x7fa8a015c920 2024-04-25T10:16:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a52f6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a015c490 0x7fa8a014ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a52f6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a015c490 0x7fa8a014ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60326/0 (socket says 172.21.15.133:60326) 2024-04-25T10:16:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a52f6640 1 -- 172.21.15.133:0/3482422793 learned_addr learned my addr 172.21.15.133:0/3482422793 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa88c003f70 0x7fa8a014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a62f8640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a015c920 0x7fa8a00c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a52f6640 1 -- 172.21.15.133:0/3482422793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a015c920 msgr2=0x7fa8a00c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a015c920 0x7fa8a00c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a52f6640 1 -- 172.21.15.133:0/3482422793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa88c003f70 msgr2=0x7fa8a014a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa88c003f70 0x7fa8a014a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a52f6640 1 -- 172.21.15.133:0/3482422793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a01489b0 con 0x7fa8a015c490 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa88c003f70 0x7fa8a014a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.373+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a015c490 0x7fa8a014ade0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fa89001d130 tx=0x7fa88c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8967fc640 1 -- 172.21.15.133:0/3482422793 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890001440 con 0x7fa8a015c490 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8967fc640 1 -- 172.21.15.133:0/3482422793 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8900015e0 con 0x7fa8a015c490 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/3482422793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a0148c90 con 0x7fa8a015c490 2024-04-25T10:16:18.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8967fc640 1 -- 172.21.15.133:0/3482422793 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8900047f0 con 0x7fa8a015c490 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/3482422793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a0149250 con 0x7fa8a015c490 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8967fc640 1 -- 172.21.15.133:0/3482422793 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa890004990 con 0x7fa8a015c490 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8967fc640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa884066f90 0x7fa884069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8967fc640 1 -- 172.21.15.133:0/3482422793 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa89011ed00 con 0x7fa8a015c490 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa884066f90 0x7fa884069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa884066f90 0x7fa884069450 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa89c000c10 tx=0x7fa89c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8aaec8000 1 Processor -- start 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.377+0000 7fa8aaec8000 1 -- start start 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2049d0 0x56019e204db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2052f0 0x56019e1bab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e1bb0a0 0x56019e1bb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019e007ff0 con 0x56019e2049d0 2024-04-25T10:16:18.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019df90150 con 0x56019e1bb0a0 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019e004090 con 0x56019e2052f0 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2049d0 0x56019e204db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2049d0 0x56019e204db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60332/0 (socket says 172.21.15.133:60332) 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 -- 172.21.15.133:0/1062393856 learned_addr learned my addr 172.21.15.133:0/1062393856 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a62f8640 1 --2- 172.21.15.133:0/1062393856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e1bb0a0 0x56019e1bb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/1062393856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2052f0 0x56019e1bab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 -- 172.21.15.133:0/1062393856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2052f0 msgr2=0x56019e1bab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/1062393856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2052f0 0x56019e1bab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 -- 172.21.15.133:0/1062393856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e1bb0a0 msgr2=0x56019e1bb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/1062393856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e1bb0a0 0x56019e1bb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 -- 172.21.15.133:0/1062393856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56019e19e0d0 con 0x56019e2049d0 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/1062393856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2049d0 0x56019e204db0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7fa89c001650 tx=0x7fa89c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ac5685141acf2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8837fe640 1 -- 172.21.15.133:0/1062393856 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c027070 con 0x56019e2049d0 2024-04-25T10:16:18.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8837fe640 1 -- 172.21.15.133:0/1062393856 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c0147c0 con 0x56019e2049d0 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8837fe640 1 -- 172.21.15.133:0/1062393856 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c014b70 con 0x56019e2049d0 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- 172.21.15.133:0/1062393856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2049d0 msgr2=0x56019e204db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/1062393856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2049d0 0x56019e204db0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7fa89c001650 tx=0x7fa89c013000 comp rx=0 tx=0).stop 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- 172.21.15.133:0/1062393856 shutdown_connections 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/1062393856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e1bb0a0 0x56019e1bb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/1062393856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2052f0 0x56019e1bab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/1062393856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2049d0 0x56019e204db0 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- 172.21.15.133:0/1062393856 >> 172.21.15.133:0/1062393856 conn(0x56019e1b7020 msgr2=0x56019e1b9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- 172.21.15.133:0/1062393856 shutdown_connections 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- 172.21.15.133:0/1062393856 wait complete. 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- start start 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 Processor -- start 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e1e97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2049d0 0x56019e1e9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2052f0 0x56019e1ea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019dfcc250 con 0x56019e1bb0a0 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019dfc2690 con 0x56019e2052f0 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019df90a40 con 0x56019e2049d0 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8aaec8000 10 client.0 _ll_register_callbacks cb 0x56019e1a5e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e1e97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e1e97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60342/0 (socket says 172.21.15.133:60342) 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 -- 172.21.15.133:0/2373610276 learned_addr learned my addr 172.21.15.133:0/2373610276 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a62f8640 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2052f0 0x56019e1ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2049d0 0x56019e1e9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 -- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2049d0 msgr2=0x56019e1e9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2049d0 0x56019e1e9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 -- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2052f0 msgr2=0x56019e1ea270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2052f0 0x56019e1ea270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 -- 172.21.15.133:0/2373610276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56019e0b51c0 con 0x56019e1bb0a0 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e1e97f0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fa89c025040 tx=0x7fa89c0011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfde8cc0785e96db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa880ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.381+0000 7fa880ff9640 1 -- 172.21.15.133:0/2373610276 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c027030 con 0x56019e1bb0a0 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2373610276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa89c0147c0 con 0x56019e1bb0a0 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa880ff9640 1 -- 172.21.15.133:0/2373610276 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa89c0216f0 con 0x56019e1bb0a0 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa880ff9640 1 -- 172.21.15.133:0/2373610276 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa89c03b050 con 0x56019e1bb0a0 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 10 client.11250 fetch_fsmap learned FSMap version 105 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2373610276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x56019e0b1170 con 0x56019e1bb0a0 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa880ff9640 1 -- 172.21.15.133:0/2373610276 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c0147c0 con 0x56019e1bb0a0 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa880ff9640 1 -- 172.21.15.133:0/2373610276 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa89c03e020 con 0x56019e1bb0a0 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 10 client.11250 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 10 client.11250 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:16:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 20 client.11250 populate_metadata read hostname 'smithi133' 2024-04-25T10:16:18.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56019e214c80 0x56019e217070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 4 client.11250 mds_command: new command op to 11037 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:18.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2373610276 --> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56019dfa3c60 con 0x56019e214c80 2024-04-25T10:16:18.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56019e214c80 0x56019e217070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa880ff9640 10 client.11250 ms_handle_connect on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:18.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56019e214c80 0x56019e217070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30903857d6899db server_cookie=85183cf86036dbc0 in_seq=0 out_seq=0 2024-04-25T10:16:18.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa880ff9640 1 -- 172.21.15.133:0/2373610276 <== mds.0 v2:172.21.15.162:6832/2658062772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56019dfa3c60 con 0x56019e214c80 2024-04-25T10:16:18.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa880ff9640 10 client.11250 handle_command_reply: tid=0 2024-04-25T10:16:18.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 client.11250 shutdown 2024-04-25T10:16:18.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2373610276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 msgr2=0x56019e1e97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e1e97f0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fa89c025040 tx=0x7fa89c0011a0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2373610276 shutdown_connections 2024-04-25T10:16:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56019e214c80 0x56019e217070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2052f0 0x56019e1ea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2049d0 0x56019e1e9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/2373610276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e1e97f0 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa880ff9640 0 client.11250 ms_handle_reset on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2373610276 >> 172.21.15.133:0/2373610276 conn(0x56019e1b7020 msgr2=0x56019e1b9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2373610276 shutdown_connections 2024-04-25T10:16:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2373610276 wait complete. 2024-04-25T10:16:18.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.385+0000 7fa8aaec8000 20 client.11250 trim_cache size 0 max 16384 2024-04-25T10:16:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8aaec8000 1 Processor -- start 2024-04-25T10:16:18.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8aaec8000 1 -- start start 2024-04-25T10:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e269b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2049d0 0x56019e26a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2052f0 0x56019e26a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0203c50 con 0x56019e1bb0a0 2024-04-25T10:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0202e50 con 0x56019e2052f0 2024-04-25T10:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0202250 con 0x56019e2049d0 2024-04-25T10:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8a5af7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e269b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8a5af7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e269b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60350/0 (socket says 172.21.15.133:60350) 2024-04-25T10:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8a5af7640 1 -- 172.21.15.133:0/2256496799 learned_addr learned my addr 172.21.15.133:0/2256496799 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8a5af7640 1 -- 172.21.15.133:0/2256496799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2049d0 msgr2=0x56019e26a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/2256496799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2049d0 0x56019e26a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8a5af7640 1 -- 172.21.15.133:0/2256496799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2052f0 msgr2=0x56019e26a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/2256496799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2052f0 0x56019e26a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8a5af7640 1 -- 172.21.15.133:0/2256496799 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56019e04ede0 con 0x56019e1bb0a0 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/2256496799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e269b10 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fa89c014920 tx=0x7fa89c02d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248b986b7f59ba5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa882ffd640 1 -- 172.21.15.133:0/2256496799 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c027070 con 0x56019e1bb0a0 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa882ffd640 1 -- 172.21.15.133:0/2256496799 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c044070 con 0x56019e1bb0a0 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.401+0000 7fa882ffd640 1 -- 172.21.15.133:0/2256496799 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c021bc0 con 0x56019e1bb0a0 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2256496799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 msgr2=0x56019e269b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/2256496799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e269b10 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fa89c014920 tx=0x7fa89c02d000 comp rx=0 tx=0).stop 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2256496799 shutdown_connections 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/2256496799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2052f0 0x56019e26a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/2256496799 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e2049d0 0x56019e26a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/2256496799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e1bb0a0 0x56019e269b10 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2256496799 >> 172.21.15.133:0/2256496799 conn(0x56019e1b7020 msgr2=0x56019e2030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2256496799 shutdown_connections 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- 172.21.15.133:0/2256496799 wait complete. 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- start start 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 Processor -- start 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e1bb0a0 0x56019e217770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2049d0 0x56019e217cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2052f0 0x56019e2181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019dfa2e00 con 0x56019e2052f0 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0202850 con 0x56019e2049d0 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019e007300 con 0x56019e1bb0a0 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 10 client.0 _ll_register_callbacks cb 0x56019e186040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a62f8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2052f0 0x56019e2181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a62f8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2052f0 0x56019e2181f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60356/0 (socket says 172.21.15.133:60356) 2024-04-25T10:16:18.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a62f8640 1 -- 172.21.15.133:0/610750606 learned_addr learned my addr 172.21.15.133:0/610750606 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e1bb0a0 0x56019e217770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a52f6640 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2049d0 0x56019e217cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a62f8640 1 -- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e1bb0a0 msgr2=0x56019e217770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a62f8640 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e1bb0a0 0x56019e217770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a62f8640 1 -- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2049d0 msgr2=0x56019e217cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a62f8640 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2049d0 0x56019e217cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a62f8640 1 -- 172.21.15.133:0/610750606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56019dfa98e0 con 0x56019e2052f0 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a62f8640 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2052f0 0x56019e2181f0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00b6420 tx=0x7fa898004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56557c8eb3a9612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa880ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa880ff9640 1 -- 172.21.15.133:0/610750606 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89801b020 con 0x56019e2052f0 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa880ff9640 1 -- 172.21.15.133:0/610750606 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa898007a70 con 0x56019e2052f0 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- 172.21.15.133:0/610750606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa89c044070 con 0x56019e2052f0 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa880ff9640 1 -- 172.21.15.133:0/610750606 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa898025050 con 0x56019e2052f0 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 10 client.11256 fetch_fsmap learned FSMap version 105 2024-04-25T10:16:18.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- 172.21.15.133:0/610750606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x56019e0618f0 con 0x56019e2052f0 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa880ff9640 1 -- 172.21.15.133:0/610750606 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c044070 con 0x56019e2052f0 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa880ff9640 1 -- 172.21.15.133:0/610750606 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa898028020 con 0x56019e2052f0 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 10 client.11256 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 10 client.11256 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 20 client.11256 populate_metadata read hostname 'smithi133' 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56019e273760 0x56019e275b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 4 client.11256 mds_command: new command op to 11037 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8aaec8000 1 -- 172.21.15.133:0/610750606 --> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56019e015530 con 0x56019e273760 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.405+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56019e273760 0x56019e275b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa880ff9640 10 client.11256 ms_handle_connect on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a5af7640 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56019e273760 0x56019e275b50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b2b6be8f2c7c6a6 server_cookie=66de554b01ebba71 in_seq=0 out_seq=0 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa880ff9640 1 -- 172.21.15.133:0/610750606 <== mds.0 v2:172.21.15.162:6832/2658062772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x56019e015530 con 0x56019e273760 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa880ff9640 10 client.11256 handle_command_reply: tid=0 2024-04-25T10:16:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 1 client.11256 shutdown 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 1 -- 172.21.15.133:0/610750606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2052f0 msgr2=0x56019e2181f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2052f0 0x56019e2181f0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00b6420 tx=0x7fa898004700 comp rx=0 tx=0).stop 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 1 -- 172.21.15.133:0/610750606 shutdown_connections 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x56019e273760 0x56019e275b50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56019e2052f0 0x56019e2181f0 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56019e2049d0 0x56019e217cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 1 --2- 172.21.15.133:0/610750606 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56019e1bb0a0 0x56019e217770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 1 -- 172.21.15.133:0/610750606 >> 172.21.15.133:0/610750606 conn(0x56019e1b7020 msgr2=0x56019e203960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa880ff9640 0 client.11256 ms_handle_reset on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 1 -- 172.21.15.133:0/610750606 shutdown_connections 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 1 -- 172.21.15.133:0/610750606 wait complete. 2024-04-25T10:16:18.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8aaec8000 20 client.11256 trim_cache size 0 max 16384 2024-04-25T10:16:18.416 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:16:18.416 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:16:18.416 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-04-25T10:16:18.416 INFO:teuthology.orchestra.run.smithi133.stdout: "1daadb6e-5aeb-43cc-ada1-41edbec0d05b": { 2024-04-25T10:16:18.416 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-04-25T10:16:18.416 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "1daadb6e-5aeb-43cc-ada1-41edbec0d05b", 2024-04-25T10:16:18.416 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,repair" 2024-04-25T10:16:18.416 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:18.416 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:18.416 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:16:18.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/3482422793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa884066f90 msgr2=0x7fa884069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa884066f90 0x7fa884069450 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa89c000c10 tx=0x7fa89c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:18.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/3482422793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a015c490 msgr2=0x7fa8a014ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:18.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a015c490 0x7fa8a014ade0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fa89001d130 tx=0x7fa88c005840 comp rx=0 tx=0).stop 2024-04-25T10:16:18.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/3482422793 shutdown_connections 2024-04-25T10:16:18.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa8a015c920 0x7fa8a00c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa884066f90 0x7fa884069450 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8a015c490 0x7fa8a014ade0 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 --2- 172.21.15.133:0/3482422793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa88c003f70 0x7fa8a014a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:18.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/3482422793 >> 172.21.15.133:0/3482422793 conn(0x7fa8a006bf80 msgr2=0x7fa8a00d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:18.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/3482422793 shutdown_connections 2024-04-25T10:16:18.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:18.409+0000 7fa8a7d5b640 1 -- 172.21.15.133:0/3482422793 wait complete. 2024-04-25T10:16:18.426 INFO:tasks.cephfs.filesystem:scrub status for tag:1daadb6e-5aeb-43cc-ada1-41edbec0d05b - {'path': '/', 'tag': '1daadb6e-5aeb-43cc-ada1-41edbec0d05b', 'options': 'recursive,repair'} 2024-04-25T10:16:23.949 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:23.952 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:48.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:48.427 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:16:48.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.560+0000 7fc3151f7640 1 Processor -- start 2024-04-25T10:16:48.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.560+0000 7fc3151f7640 1 -- start start 2024-04-25T10:16:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.560+0000 7fc3151f7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc3100cd820 0x7fc3100cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.560+0000 7fc3151f7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31015ced0 0x7fc31015d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.560+0000 7fc3151f7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3100ce2e0 con 0x7fc310145170 2024-04-25T10:16:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.560+0000 7fc3151f7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31006db00 con 0x7fc3100cd820 2024-04-25T10:16:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.560+0000 7fc3151f7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc310145550 con 0x7fc31015ced0 2024-04-25T10:16:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30ed9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc310145170 0x7fc31006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53550/0 (socket says 172.21.15.133:53550) 2024-04-25T10:16:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30ed9d640 1 -- 172.21.15.133:0/3597787659 learned_addr learned my addr 172.21.15.133:0/3597787659 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30659c640 1 --2- 172.21.15.133:0/3597787659 >> v2:172.21.15.162:3300/0 conn(0x7fc3100cd820 0x7fc3100cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30f59e640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31015ced0 0x7fc31015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553111928 0 0) 0x7fc3100ce2e0 con 0x7fc310145170 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2fc003530 con 0x7fc310145170 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4249283746 0 0) 0x7fc2fc003530 con 0x7fc310145170 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31015ced0 msgr2=0x7fc31015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31015ced0 0x7fc31015d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 >> v2:172.21.15.162:3300/0 conn(0x7fc3100cd820 msgr2=0x7fc3100cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 --2- 172.21.15.133:0/3597787659 >> v2:172.21.15.162:3300/0 conn(0x7fc3100cd820 0x7fc3100cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31006d4b0 con 0x7fc310145170 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fc300001680 con 0x7fc310145170 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 >> v1:172.21.15.133:6789/0 conn(0x7fc310145170 legacy=0x7fc31006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2fc0039c0 0x7fc2fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2fc004810 0x7fc2fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31015ced0 0x7fc2fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31006db00 con 0x7fc2fc0039c0 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc310145550 con 0x7fc2fc004810 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc00b9d0 con 0x7fc31015ced0 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2fc0039c0 0x7fc2fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30659c640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2fc004810 0x7fc2fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30f59e640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31015ced0 0x7fc2fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30ed9d640 1 -- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31015ced0 msgr2=0x7fc2fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31015ced0 0x7fc2fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30ed9d640 1 -- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2fc004810 msgr2=0x7fc2fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2fc004810 0x7fc2fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30ed9d640 1 -- 172.21.15.133:0/3597787659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc003530 con 0x7fc2fc0039c0 2024-04-25T10:16:48.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30f59e640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31015ced0 0x7fc2fc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.564+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2fc0039c0 0x7fc2fc003de0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fc30000c240 tx=0x7fc30001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3523573671d8d30f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300007d60 con 0x7fc2fc0039c0 2024-04-25T10:16:48.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3000a9020 con 0x7fc2fc0039c0 2024-04-25T10:16:48.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30e59c640 1 -- 172.21.15.133:0/3597787659 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300005ae0 con 0x7fc2fc0039c0 2024-04-25T10:16:48.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 -- 172.21.15.133:0/3597787659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2fc0039c0 msgr2=0x7fc2fc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 --2- 172.21.15.133:0/3597787659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2fc0039c0 0x7fc2fc003de0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fc30000c240 tx=0x7fc30001ccc0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30f59e640 1 -- 172.21.15.133:0/3597787659 reap_dead start 2024-04-25T10:16:48.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 -- 172.21.15.133:0/3597787659 shutdown_connections 2024-04-25T10:16:48.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 -- 172.21.15.133:0/3597787659 >> 172.21.15.133:0/3597787659 conn(0x7fc310066f70 msgr2=0x7fc31006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 -- 172.21.15.133:0/3597787659 shutdown_connections 2024-04-25T10:16:48.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 -- 172.21.15.133:0/3597787659 wait complete. 2024-04-25T10:16:48.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 Processor -- start 2024-04-25T10:16:48.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 -- start start 2024-04-25T10:16:48.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2fc0039c0 0x7fc31015fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31013d310 0x7fc310139a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3101633e0 0x7fc310163860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc310166480 con 0x7fc3101633e0 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31015cb70 con 0x7fc2fc0039c0 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc310166600 con 0x7fc31013d310 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3101633e0 0x7fc310163860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3101633e0 0x7fc310163860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41418/0 (socket says 172.21.15.133:41418) 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30f59e640 1 -- 172.21.15.133:0/4153081992 learned_addr learned my addr 172.21.15.133:0/4153081992 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2fc0039c0 0x7fc31015fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30659c640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31013d310 0x7fc310139a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30f59e640 1 -- 172.21.15.133:0/4153081992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31013d310 msgr2=0x7fc310139a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30f59e640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31013d310 0x7fc310139a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30f59e640 1 -- 172.21.15.133:0/4153081992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2fc0039c0 msgr2=0x7fc31015fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30f59e640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2fc0039c0 0x7fc31015fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30f59e640 1 -- 172.21.15.133:0/4153081992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31013a1b0 con 0x7fc3101633e0 2024-04-25T10:16:48.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30659c640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31013d310 0x7fc310139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc30f59e640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3101633e0 0x7fc310163860 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7fc3100af440 tx=0x7fc30800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc307fff640 1 -- 172.21.15.133:0/4153081992 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30801c070 con 0x7fc3101633e0 2024-04-25T10:16:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc307fff640 1 -- 172.21.15.133:0/4153081992 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc308017400 con 0x7fc3101633e0 2024-04-25T10:16:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 -- 172.21.15.133:0/4153081992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3101437a0 con 0x7fc3101633e0 2024-04-25T10:16:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc307fff640 1 -- 172.21.15.133:0/4153081992 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3080177d0 con 0x7fc3101633e0 2024-04-25T10:16:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.568+0000 7fc3151f7640 1 -- 172.21.15.133:0/4153081992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc310143d60 con 0x7fc3101633e0 2024-04-25T10:16:48.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc307fff640 1 -- 172.21.15.133:0/4153081992 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc30801f070 con 0x7fc3101633e0 2024-04-25T10:16:48.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc307fff640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f0066f60 0x7fc2f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc307fff640 1 -- 172.21.15.133:0/4153081992 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fc3080852d0 con 0x7fc3101633e0 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f0066f60 0x7fc2f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f0066f60 0x7fc2f0069420 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc3000aa000 tx=0x7fc300001ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc318364000 1 Processor -- start 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc318364000 1 -- start start 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc318364000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9af09db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc318364000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9af0a2f0 0x562c9aebfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc318364000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9aec0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc318364000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9ad0cff0 con 0x562c9aec00a0 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc318364000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9ac95150 con 0x562c9af0a2f0 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc318364000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c9ad09090 con 0x562c9af099d0 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc30f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9aec0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc30f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9aec0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41420/0 (socket says 172.21.15.133:41420) 2024-04-25T10:16:48.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.572+0000 7fc30f59e640 1 -- 172.21.15.133:0/4227944889 learned_addr learned my addr 172.21.15.133:0/4227944889 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/4227944889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9af09db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30659c640 1 --2- 172.21.15.133:0/4227944889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9af0a2f0 0x562c9aebfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 -- 172.21.15.133:0/4227944889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 msgr2=0x562c9af09db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 --2- 172.21.15.133:0/4227944889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9af09db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 -- 172.21.15.133:0/4227944889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9af0a2f0 msgr2=0x562c9aebfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 --2- 172.21.15.133:0/4227944889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9af0a2f0 0x562c9aebfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 -- 172.21.15.133:0/4227944889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c9aea30d0 con 0x562c9aec00a0 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/4227944889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9af09db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 --2- 172.21.15.133:0/4227944889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9aec0480 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fc308010010 tx=0x7fc308055700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2044ed3899d5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc304d99640 1 -- 172.21.15.133:0/4227944889 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308053b30 con 0x562c9aec00a0 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc304d99640 1 -- 172.21.15.133:0/4227944889 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc308053d90 con 0x562c9aec00a0 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc304d99640 1 -- 172.21.15.133:0/4227944889 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308055cf0 con 0x562c9aec00a0 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 -- 172.21.15.133:0/4227944889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 msgr2=0x562c9aec0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 --2- 172.21.15.133:0/4227944889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9aec0480 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fc308010010 tx=0x7fc308055700 comp rx=0 tx=0).stop 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 -- 172.21.15.133:0/4227944889 shutdown_connections 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 --2- 172.21.15.133:0/4227944889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9aec0480 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 --2- 172.21.15.133:0/4227944889 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9af0a2f0 0x562c9aebfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 --2- 172.21.15.133:0/4227944889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9af09db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 -- 172.21.15.133:0/4227944889 >> 172.21.15.133:0/4227944889 conn(0x562c9aebc020 msgr2=0x562c9aebe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 -- 172.21.15.133:0/4227944889 shutdown_connections 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 -- 172.21.15.133:0/4227944889 wait complete. 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 -- start start 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 Processor -- start 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9aec00a0 0x562c9aeee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9aeeed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aeef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9acd1250 con 0x562c9af0a2f0 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9acc7690 con 0x562c9aec00a0 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c9ac95a40 con 0x562c9af099d0 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 10 client.0 _ll_register_callbacks cb 0x562c9aeaae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aeef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aeef270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41434/0 (socket says 172.21.15.133:41434) 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 -- 172.21.15.133:0/2117427126 learned_addr learned my addr 172.21.15.133:0/2117427126 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9aec00a0 0x562c9aeee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30659c640 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9aeeed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 -- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 msgr2=0x562c9aeeed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9aeeed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 -- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9aec00a0 msgr2=0x562c9aeee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9aec00a0 0x562c9aeee7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 -- 172.21.15.133:0/2117427126 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c9adba1c0 con 0x562c9af0a2f0 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30659c640 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9aeeed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9aec00a0 0x562c9aeee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc30f59e640 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aeef270 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fc308055a20 tx=0x7fc308085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701532023eb935ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc2edffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc2edffb640 1 -- 172.21.15.133:0/2117427126 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30805dd10 con 0x562c9af0a2f0 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 -- 172.21.15.133:0/2117427126 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc308053d90 con 0x562c9af0a2f0 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc2edffb640 1 -- 172.21.15.133:0/2117427126 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fc30805deb0 con 0x562c9af0a2f0 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc2edffb640 1 -- 172.21.15.133:0/2117427126 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc308054ee0 con 0x562c9af0a2f0 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 10 client.11268 fetch_fsmap learned FSMap version 105 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc318364000 1 -- 172.21.15.133:0/2117427126 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x562c9adb6170 con 0x562c9af0a2f0 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc2edffb640 1 -- 172.21.15.133:0/2117427126 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc308053d90 con 0x562c9af0a2f0 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.576+0000 7fc2edffb640 1 -- 172.21.15.133:0/2117427126 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc308055080 con 0x562c9af0a2f0 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 10 client.11268 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 10 client.11268 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 20 client.11268 populate_metadata read hostname 'smithi133' 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x562c9af19c80 0x562c9af1c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 4 client.11268 mds_command: new command op to 11037 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:48.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 -- 172.21.15.133:0/2117427126 --> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c9aca8c60 con 0x562c9af19c80 2024-04-25T10:16:48.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x562c9af19c80 0x562c9af1c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x562c9af19c80 0x562c9af1c070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da6179b701794464 server_cookie=ebe73d6c0549c955 in_seq=0 out_seq=0 2024-04-25T10:16:48.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc2edffb640 10 client.11268 ms_handle_connect on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc2edffb640 1 -- 172.21.15.133:0/2117427126 <== mds.0 v2:172.21.15.162:6832/2658062772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c9aca8c60 con 0x562c9af19c80 2024-04-25T10:16:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc2edffb640 10 client.11268 handle_command_reply: tid=0 2024-04-25T10:16:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 client.11268 shutdown 2024-04-25T10:16:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 -- 172.21.15.133:0/2117427126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 msgr2=0x562c9aeef270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aeef270 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7fc308055a20 tx=0x7fc308085040 comp rx=0 tx=0).stop 2024-04-25T10:16:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 -- 172.21.15.133:0/2117427126 shutdown_connections 2024-04-25T10:16:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x562c9af19c80 0x562c9af1c070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aeef270 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9aeeed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 --2- 172.21.15.133:0/2117427126 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9aec00a0 0x562c9aeee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc2edffb640 0 client.11268 ms_handle_reset on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:48.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 -- 172.21.15.133:0/2117427126 >> 172.21.15.133:0/2117427126 conn(0x562c9aebc020 msgr2=0x562c9aebeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 -- 172.21.15.133:0/2117427126 shutdown_connections 2024-04-25T10:16:48.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 1 -- 172.21.15.133:0/2117427126 wait complete. 2024-04-25T10:16:48.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.580+0000 7fc318364000 20 client.11268 trim_cache size 0 max 16384 2024-04-25T10:16:48.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc318364000 1 Processor -- start 2024-04-25T10:16:48.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc318364000 1 -- start start 2024-04-25T10:16:48.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc318364000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9aec00a0 0x562c9af16e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc318364000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9af17380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc318364000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aebe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc318364000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc310201e90 con 0x562c9af0a2f0 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc318364000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc310201030 con 0x562c9aec00a0 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc318364000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc310200430 con 0x562c9af099d0 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc30f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aebe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc30f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aebe410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41444/0 (socket says 172.21.15.133:41444) 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc30f59e640 1 -- 172.21.15.133:0/498433723 learned_addr learned my addr 172.21.15.133:0/498433723 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/498433723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9aec00a0 0x562c9af16e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc30f59e640 1 -- 172.21.15.133:0/498433723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 msgr2=0x562c9af17380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc30f59e640 1 --2- 172.21.15.133:0/498433723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9af17380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc30f59e640 1 -- 172.21.15.133:0/498433723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9aec00a0 msgr2=0x562c9af16e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc30f59e640 1 --2- 172.21.15.133:0/498433723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9aec00a0 0x562c9af16e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc30f59e640 1 -- 172.21.15.133:0/498433723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c9ad53de0 con 0x562c9af0a2f0 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc30f59e640 1 --2- 172.21.15.133:0/498433723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aebe410 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7fc308051ef0 tx=0x7fc308023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164d70c71318c70d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc2effff640 1 -- 172.21.15.133:0/498433723 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3080830d0 con 0x562c9af0a2f0 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc2effff640 1 -- 172.21.15.133:0/498433723 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc308053d90 con 0x562c9af0a2f0 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.596+0000 7fc2effff640 1 -- 172.21.15.133:0/498433723 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30805dd10 con 0x562c9af0a2f0 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- 172.21.15.133:0/498433723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 msgr2=0x562c9aebe410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 --2- 172.21.15.133:0/498433723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aebe410 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7fc308051ef0 tx=0x7fc308023000 comp rx=0 tx=0).stop 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- 172.21.15.133:0/498433723 shutdown_connections 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 --2- 172.21.15.133:0/498433723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9af0a2f0 0x562c9aebe410 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 --2- 172.21.15.133:0/498433723 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af099d0 0x562c9af17380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 --2- 172.21.15.133:0/498433723 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9aec00a0 0x562c9af16e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- 172.21.15.133:0/498433723 >> 172.21.15.133:0/498433723 conn(0x562c9aebc020 msgr2=0x562c9af16730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- 172.21.15.133:0/498433723 shutdown_connections 2024-04-25T10:16:48.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- 172.21.15.133:0/498433723 wait complete. 2024-04-25T10:16:48.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- start start 2024-04-25T10:16:48.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 Processor -- start 2024-04-25T10:16:48.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9af69720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9af099d0 0x562c9af69c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af0a2f0 0x562c9af1b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9aca7e00 con 0x562c9aec00a0 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc310200a30 con 0x562c9af099d0 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c9ad0c300 con 0x562c9af0a2f0 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 10 client.0 _ll_register_callbacks cb 0x562c9ae8b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc30ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9af69720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc30ed9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9af69720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41446/0 (socket says 172.21.15.133:41446) 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc30ed9d640 1 -- 172.21.15.133:0/2800816304 learned_addr learned my addr 172.21.15.133:0/2800816304 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc30ed9d640 1 -- 172.21.15.133:0/2800816304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af0a2f0 msgr2=0x562c9af1b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/2800816304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af0a2f0 0x562c9af1b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc30ed9d640 1 -- 172.21.15.133:0/2800816304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9af099d0 msgr2=0x562c9af69c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/2800816304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9af099d0 0x562c9af69c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc30ed9d640 1 -- 172.21.15.133:0/2800816304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c9acae8e0 con 0x562c9aec00a0 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc30ed9d640 1 --2- 172.21.15.133:0/2800816304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9af69720 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fc30001caf0 tx=0x7fc300019a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380a3754ed453a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc2edffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:16:48.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc2edffb640 1 -- 172.21.15.133:0/2800816304 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300001780 con 0x562c9aec00a0 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc2edffb640 1 -- 172.21.15.133:0/2800816304 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fc300004c40 con 0x562c9aec00a0 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- 172.21.15.133:0/2800816304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc308053d90 con 0x562c9aec00a0 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc2edffb640 1 -- 172.21.15.133:0/2800816304 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3000d4050 con 0x562c9aec00a0 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 10 client.11274 fetch_fsmap learned FSMap version 105 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- 172.21.15.133:0/2800816304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x562c9ad668f0 con 0x562c9aec00a0 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc2edffb640 1 -- 172.21.15.133:0/2800816304 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc308053d90 con 0x562c9aec00a0 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc2edffb640 1 -- 172.21.15.133:0/2800816304 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc3000d7020 con 0x562c9aec00a0 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 10 client.11274 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 10 client.11274 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 20 client.11274 populate_metadata read hostname 'smithi133' 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 --2- 172.21.15.133:0/2800816304 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x562c9af1bd20 0x562c9af1c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 4 client.11274 mds_command: new command op to 11037 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc318364000 1 -- 172.21.15.133:0/2800816304 --> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562c9ad1a530 con 0x562c9af1bd20 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc30659c640 1 --2- 172.21.15.133:0/2800816304 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x562c9af1bd20 0x562c9af1c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc30659c640 1 --2- 172.21.15.133:0/2800816304 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x562c9af1bd20 0x562c9af1c130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64eab36b45a94db0 server_cookie=2adc87433cf91ee in_seq=0 out_seq=0 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.600+0000 7fc2edffb640 10 client.11274 ms_handle_connect on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc2edffb640 1 -- 172.21.15.133:0/2800816304 <== mds.0 v2:172.21.15.162:6832/2658062772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562c9ad1a530 con 0x562c9af1bd20 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc2edffb640 10 client.11274 handle_command_reply: tid=0 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 1 client.11274 shutdown 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 1 -- 172.21.15.133:0/2800816304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 msgr2=0x562c9af69720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 1 --2- 172.21.15.133:0/2800816304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9af69720 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fc30001caf0 tx=0x7fc300019a60 comp rx=0 tx=0).stop 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 1 -- 172.21.15.133:0/2800816304 shutdown_connections 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 1 --2- 172.21.15.133:0/2800816304 >> [v2:172.21.15.162:6832/2658062772,v1:172.21.15.162:6833/2658062772] conn(0x562c9af1bd20 0x562c9af1c130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 1 --2- 172.21.15.133:0/2800816304 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x562c9af0a2f0 0x562c9af1b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 1 --2- 172.21.15.133:0/2800816304 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562c9af099d0 0x562c9af69c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 1 --2- 172.21.15.133:0/2800816304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c9aec00a0 0x562c9af69720 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc2edffb640 0 client.11274 ms_handle_reset on v2:172.21.15.162:6832/2658062772 2024-04-25T10:16:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 1 -- 172.21.15.133:0/2800816304 >> 172.21.15.133:0/2800816304 conn(0x562c9aebc020 msgr2=0x562c9af16fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 1 -- 172.21.15.133:0/2800816304 shutdown_connections 2024-04-25T10:16:48.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 1 -- 172.21.15.133:0/2800816304 wait complete. 2024-04-25T10:16:48.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc318364000 20 client.11274 trim_cache size 0 max 16384 2024-04-25T10:16:48.610 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:16:48.610 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-04-25T10:16:48.610 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:16:48.610 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:16:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 -- 172.21.15.133:0/4153081992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f0066f60 msgr2=0x7fc2f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f0066f60 0x7fc2f0069420 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc3000aa000 tx=0x7fc300001ac0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 -- 172.21.15.133:0/4153081992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3101633e0 msgr2=0x7fc310163860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3101633e0 0x7fc310163860 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7fc3100af440 tx=0x7fc30800f4a0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 -- 172.21.15.133:0/4153081992 shutdown_connections 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc2f0066f60 0x7fc2f0069420 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3101633e0 0x7fc310163860 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc31013d310 0x7fc310139a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 --2- 172.21.15.133:0/4153081992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc2fc0039c0 0x7fc31015fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 -- 172.21.15.133:0/4153081992 >> 172.21.15.133:0/4153081992 conn(0x7fc310066f70 msgr2=0x7fc31006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 -- 172.21.15.133:0/4153081992 shutdown_connections 2024-04-25T10:16:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.604+0000 7fc3151f7640 1 -- 172.21.15.133:0/4153081992 wait complete. 2024-04-25T10:16:48.622 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:16:48.622 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-04-25T10:16:48.622 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:16:48.622 DEBUG:teuthology.orchestra.run.smithi133:> import stat 2024-04-25T10:16:48.622 DEBUG:teuthology.orchestra.run.smithi133:> import json 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> import sys 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> try: 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> except OSError as e: 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> sys.exit(e.errno) 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> print(json.dumps( 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> indent=2)) 2024-04-25T10:16:48.623 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:16:48.684 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:16:48.684 INFO:teuthology.orchestra.run.smithi133.stdout: "st_mode": 33204, 2024-04-25T10:16:48.685 INFO:teuthology.orchestra.run.smithi133.stdout: "st_ino": 1099511627777, 2024-04-25T10:16:48.685 INFO:teuthology.orchestra.run.smithi133.stdout: "st_dev": 45, 2024-04-25T10:16:48.685 INFO:teuthology.orchestra.run.smithi133.stdout: "st_nlink": 1, 2024-04-25T10:16:48.685 INFO:teuthology.orchestra.run.smithi133.stdout: "st_uid": 1000, 2024-04-25T10:16:48.685 INFO:teuthology.orchestra.run.smithi133.stdout: "st_gid": 1271, 2024-04-25T10:16:48.685 INFO:teuthology.orchestra.run.smithi133.stdout: "st_size": 6291456, 2024-04-25T10:16:48.685 INFO:teuthology.orchestra.run.smithi133.stdout: "st_atime": 1714040177.1103745, 2024-04-25T10:16:48.685 INFO:teuthology.orchestra.run.smithi133.stdout: "st_mtime": 1714040177.1591718, 2024-04-25T10:16:48.685 INFO:teuthology.orchestra.run.smithi133.stdout: "st_ctime": 1714040177.1591718 2024-04-25T10:16:48.685 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:16:48.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:48.693 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:48.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b544a2640 1 Processor -- start 2024-04-25T10:16:48.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b544a2640 1 -- start start 2024-04-25T10:16:48.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b544a2640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6b4c06dca0 0x7f6b4c0b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b544a2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4c0cfdc0 con 0x7f6b4c06bda0 2024-04-25T10:16:48.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b544a2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4c0d00a0 con 0x7f6b4c0cc250 2024-04-25T10:16:48.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b544a2640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4c0d0660 con 0x7f6b4c06dca0 2024-04-25T10:16:48.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b5223e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6b4c06dca0 0x7f6b4c0b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b51a3d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6b4c06bda0 0x7f6b4c0b1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53566/0 (socket says 172.21.15.133:53566) 2024-04-25T10:16:48.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b51a3d640 1 -- 172.21.15.133:0/687627447 learned_addr learned my addr 172.21.15.133:0/687627447 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:48.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b5223e640 1 -- 172.21.15.133:0/687627447 >> v1:172.21.15.162:6789/0 conn(0x7f6b4c0cc250 legacy=0x7f6b4c0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b5223e640 1 -- 172.21.15.133:0/687627447 >> v1:172.21.15.133:6789/0 conn(0x7f6b4c06bda0 legacy=0x7f6b4c0b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b5223e640 1 -- 172.21.15.133:0/687627447 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c0d0380 con 0x7f6b4c06dca0 2024-04-25T10:16:48.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.852+0000 7f6b5123c640 1 -- 172.21.15.133:0/687627447 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942107564 0 0) 0x7f6b4c0cfdc0 con 0x7f6b4c06bda0 2024-04-25T10:16:48.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b5223e640 1 --2- 172.21.15.133:0/687627447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c06dca0 0x7f6b4c0b1620 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f6b480097c0 tx=0x7f6b4801cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5bff4246393ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b5123c640 1 -- 172.21.15.133:0/687627447 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480b0070 con 0x7f6b4c06dca0 2024-04-25T10:16:48.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b5123c640 1 -- 172.21.15.133:0/687627447 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b480ab600 con 0x7f6b4c06dca0 2024-04-25T10:16:48.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b5123c640 1 -- 172.21.15.133:0/687627447 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480ab9d0 con 0x7f6b4c06dca0 2024-04-25T10:16:48.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b544a2640 1 -- 172.21.15.133:0/687627447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c06dca0 msgr2=0x7f6b4c0b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b544a2640 1 --2- 172.21.15.133:0/687627447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c06dca0 0x7f6b4c0b1620 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f6b480097c0 tx=0x7f6b4801cda0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b544a2640 1 -- 172.21.15.133:0/687627447 shutdown_connections 2024-04-25T10:16:48.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b544a2640 1 --2- 172.21.15.133:0/687627447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c06dca0 0x7f6b4c0b1620 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b544a2640 1 -- 172.21.15.133:0/687627447 >> 172.21.15.133:0/687627447 conn(0x7f6b4c0b31a0 msgr2=0x7f6b4c0b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b544a2640 1 -- 172.21.15.133:0/687627447 shutdown_connections 2024-04-25T10:16:48.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b544a2640 1 -- 172.21.15.133:0/687627447 wait complete. 2024-04-25T10:16:48.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b544a2640 1 Processor -- start 2024-04-25T10:16:48.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.856+0000 7f6b544a2640 1 -- start start 2024-04-25T10:16:48.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b544a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c06bda0 0x7f6b4c14f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b544a2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b4c06dca0 0x7f6b4c14f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b544a2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b4c0cc250 0x7f6b4c0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b544a2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c16b930 con 0x7f6b4c06bda0 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b544a2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c16b7b0 con 0x7f6b4c0cc250 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b544a2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c16bab0 con 0x7f6b4c06dca0 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b5223e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c06bda0 0x7f6b4c14f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b5223e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c06bda0 0x7f6b4c14f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41460/0 (socket says 172.21.15.133:41460) 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b5223e640 1 -- 172.21.15.133:0/3903510985 learned_addr learned my addr 172.21.15.133:0/3903510985 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b52a3f640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b4c0cc250 0x7f6b4c0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b5223e640 1 -- 172.21.15.133:0/3903510985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b4c06dca0 msgr2=0x7f6b4c14f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b51a3d640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b4c06dca0 0x7f6b4c14f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b5223e640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b4c06dca0 0x7f6b4c14f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b5223e640 1 -- 172.21.15.133:0/3903510985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b4c0cc250 msgr2=0x7f6b4c0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b5223e640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b4c0cc250 0x7f6b4c0c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b5223e640 1 -- 172.21.15.133:0/3903510985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c15c500 con 0x7f6b4c06bda0 2024-04-25T10:16:48.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b52a3f640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b4c0cc250 0x7f6b4c0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b5223e640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c06bda0 0x7f6b4c14f0d0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f6b48018f50 tx=0x7f6b4801ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b42ffd640 1 -- 172.21.15.133:0/3903510985 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480b0050 con 0x7f6b4c06bda0 2024-04-25T10:16:48.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b42ffd640 1 -- 172.21.15.133:0/3903510985 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b480bc070 con 0x7f6b4c06bda0 2024-04-25T10:16:48.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b42ffd640 1 -- 172.21.15.133:0/3903510985 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480b5a70 con 0x7f6b4c06bda0 2024-04-25T10:16:48.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b544a2640 1 -- 172.21.15.133:0/3903510985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c15c220 con 0x7f6b4c06bda0 2024-04-25T10:16:48.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b544a2640 1 -- 172.21.15.133:0/3903510985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c0b6d70 con 0x7f6b4c06bda0 2024-04-25T10:16:48.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b544a2640 1 -- 172.21.15.133:0/3903510985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c067c70 con 0x7f6b4c06bda0 2024-04-25T10:16:48.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b42ffd640 1 -- 172.21.15.133:0/3903510985 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b480ab4c0 con 0x7f6b4c06bda0 2024-04-25T10:16:48.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b42ffd640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b30066f90 0x7f6b30069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b42ffd640 1 -- 172.21.15.133:0/3903510985 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f6b4811f890 con 0x7f6b4c06bda0 2024-04-25T10:16:48.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.860+0000 7f6b51a3d640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b30066f90 0x7f6b30069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.864+0000 7f6b51a3d640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b30066f90 0x7f6b30069450 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c15d010 tx=0x7f6b3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:48.868+0000 7f6b42ffd640 1 -- 172.21.15.133:0/3903510985 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b48016050 con 0x7f6b4c06bda0 2024-04-25T10:16:49.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.060+0000 7f6b544a2640 1 -- 172.21.15.133:0/3903510985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b4c15ca80 con 0x7f6b4c06bda0 2024-04-25T10:16:49.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.060+0000 7f6b42ffd640 1 -- 172.21.15.133:0/3903510985 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4962 (secure 0 0 0) 0x7f6b480f0750 con 0x7f6b4c06bda0 2024-04-25T10:16:49.064 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:49.064 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1498957356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1498957356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1498957356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/883424012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":883424012},{"type":"v1","addr":"172.21.15.133:6835","nonce":883424012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2040533186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2040533186},{"type":"v1","addr":"172.21.15.133:6837","nonce":2040533186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:08.983208+0000","modified":"2024-04-25T10:16:11.017848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11037":{"gid":11037,"name":"d","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/2658062772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2658062772},{"type":"v1","addr":"172.21.15.162:6833","nonce":2658062772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:16:49.064 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 105 2024-04-25T10:16:49.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 -- 172.21.15.133:0/3903510985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b30066f90 msgr2=0x7f6b30069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b30066f90 0x7f6b30069450 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c15d010 tx=0x7f6b3c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:49.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 -- 172.21.15.133:0/3903510985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c06bda0 msgr2=0x7f6b4c14f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c06bda0 0x7f6b4c14f0d0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f6b48018f50 tx=0x7f6b4801ab80 comp rx=0 tx=0).stop 2024-04-25T10:16:49.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 -- 172.21.15.133:0/3903510985 shutdown_connections 2024-04-25T10:16:49.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b4c0cc250 0x7f6b4c0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6b4c06dca0 0x7f6b4c14f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6b30066f90 0x7f6b30069450 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 --2- 172.21.15.133:0/3903510985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c06bda0 0x7f6b4c14f0d0 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 -- 172.21.15.133:0/3903510985 >> 172.21.15.133:0/3903510985 conn(0x7f6b4c0b31a0 msgr2=0x7f6b4c0ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 -- 172.21.15.133:0/3903510985 shutdown_connections 2024-04-25T10:16:49.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.064+0000 7f6b544a2640 1 -- 172.21.15.133:0/3903510985 wait complete. 2024-04-25T10:16:49.083 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T10:16:49.239 INFO:teuthology.orchestra.run.smithi162.stdout:{"message":"","return_code":0} 2024-04-25T10:16:49.239 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:16:49.240 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:16:49.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 Processor -- start 2024-04-25T10:16:49.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 -- start start 2024-04-25T10:16:49.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6907c3330 0x55a6907a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a69084e330 0x55a6907bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a69084e710 0x55a690851400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a69071f9f0 con 0x55a6907c3330 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a690723950 con 0x55a69084e710 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a69084ff90 con 0x55a69084e330 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab50191640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6907c3330 0x55a6907a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab50191640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6907c3330 0x55a6907a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41464/0 (socket says 172.21.15.133:41464) 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab50191640 1 -- 172.21.15.133:0/1092639409 learned_addr learned my addr 172.21.15.133:0/1092639409 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab50992640 1 --2- 172.21.15.133:0/1092639409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a69084e710 0x55a690851400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab4f990640 1 --2- 172.21.15.133:0/1092639409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a69084e330 0x55a6907bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab50191640 1 -- 172.21.15.133:0/1092639409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a69084e330 msgr2=0x55a6907bfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab50191640 1 --2- 172.21.15.133:0/1092639409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a69084e330 0x55a6907bfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab50191640 1 -- 172.21.15.133:0/1092639409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a69084e710 msgr2=0x55a690851400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab50191640 1 --2- 172.21.15.133:0/1092639409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a69084e710 0x55a690851400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab50191640 1 -- 172.21.15.133:0/1092639409 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6907a4a00 con 0x55a6907c3330 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab50992640 1 --2- 172.21.15.133:0/1092639409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a69084e710 0x55a690851400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab50191640 1 --2- 172.21.15.133:0/1092639409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6907c3330 0x55a6907a54d0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7fab4000a530 tx=0x7fab40003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95735279b68caa1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab4e98e640 1 -- 172.21.15.133:0/1092639409 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab400af040 con 0x55a6907c3330 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab4e98e640 1 -- 172.21.15.133:0/1092639409 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab400b9070 con 0x55a6907c3330 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab4e98e640 1 -- 172.21.15.133:0/1092639409 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4000a310 con 0x55a6907c3330 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 -- 172.21.15.133:0/1092639409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6907c3330 msgr2=0x55a6907a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 --2- 172.21.15.133:0/1092639409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6907c3330 0x55a6907a54d0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7fab4000a530 tx=0x7fab40003030 comp rx=0 tx=0).stop 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 -- 172.21.15.133:0/1092639409 shutdown_connections 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 --2- 172.21.15.133:0/1092639409 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a69084e710 0x55a690851400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 --2- 172.21.15.133:0/1092639409 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a69084e330 0x55a6907bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 --2- 172.21.15.133:0/1092639409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6907c3330 0x55a6907a54d0 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 -- 172.21.15.133:0/1092639409 >> 172.21.15.133:0/1092639409 conn(0x55a690778a00 msgr2=0x55a690778de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 -- 172.21.15.133:0/1092639409 shutdown_connections 2024-04-25T10:16:49.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 -- 172.21.15.133:0/1092639409 wait complete. 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 Processor -- start 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.280+0000 7fab52673080 1 -- start start 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab52673080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a6907c3330 0x55a6907bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab52673080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a69084e330 0x55a690847700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab52673080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a69084e710 0x55a69084cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab52673080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6907a4ca0 con 0x55a69084e330 2024-04-25T10:16:49.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab52673080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6907a5a10 con 0x55a6907c3330 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab52673080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6907a4f40 con 0x55a69084e710 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4f990640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a69084e330 0x55a690847700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4f990640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a69084e330 0x55a690847700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41474/0 (socket says 172.21.15.133:41474) 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4f990640 1 -- 172.21.15.133:0/2414592970 learned_addr learned my addr 172.21.15.133:0/2414592970 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab50191640 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a6907c3330 0x55a6907bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab50992640 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a69084e710 0x55a69084cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4f990640 1 -- 172.21.15.133:0/2414592970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a69084e710 msgr2=0x55a69084cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4f990640 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a69084e710 0x55a69084cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4f990640 1 -- 172.21.15.133:0/2414592970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a6907c3330 msgr2=0x55a6907bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4f990640 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a6907c3330 0x55a6907bfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4f990640 1 -- 172.21.15.133:0/2414592970 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a69077b320 con 0x55a69084e330 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4f990640 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a69084e330 0x55a690847700 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fab4400d430 tx=0x7fab4400d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4d98c640 1 -- 172.21.15.133:0/2414592970 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44006240 con 0x55a69084e330 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4d98c640 1 -- 172.21.15.133:0/2414592970 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab44005430 con 0x55a69084e330 2024-04-25T10:16:49.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab52673080 1 -- 172.21.15.133:0/2414592970 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a69077b5c0 con 0x55a69084e330 2024-04-25T10:16:49.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4d98c640 1 -- 172.21.15.133:0/2414592970 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44005620 con 0x55a69084e330 2024-04-25T10:16:49.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab52673080 1 -- 172.21.15.133:0/2414592970 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6908e9300 con 0x55a69084e330 2024-04-25T10:16:49.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4d98c640 1 -- 172.21.15.133:0/2414592970 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab44019020 con 0x55a69084e330 2024-04-25T10:16:49.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4d98c640 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab3c065970 0x7fab3c067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab4d98c640 1 -- 172.21.15.133:0/2414592970 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fab44083370 con 0x55a69084e330 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.284+0000 7fab50191640 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab3c065970 0x7fab3c067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x55a6908e97a0 0x55a6908e9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 -- 172.21.15.133:0/2414592970 --> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] -- osd_op(unknown.0.0:1 26.7 26:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171) v9 -- 0x55a6908ea0c0 con 0x55a6908e97a0 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab50992640 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x55a6908e97a0 0x55a6908e9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab50191640 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab3c065970 0x7fab3c067e30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fab400b8000 tx=0x7fab400a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab50992640 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x55a6908e97a0 0x55a6908e9b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab50992640 1 -- 172.21.15.133:0/2414592970 <== osd.2 v2:172.21.15.133:6817/148430844 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fab480106b0 con 0x55a6908e97a0 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 -- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x55a6908e97a0 msgr2=0x55a6908e9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x55a6908e97a0 0x55a6908e9b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 -- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab3c065970 msgr2=0x7fab3c067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fab3c065970 0x7fab3c067e30 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fab400b8000 tx=0x7fab400a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:49.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 -- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a69084e330 msgr2=0x55a690847700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 --2- 172.21.15.133:0/2414592970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a69084e330 0x55a690847700 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fab4400d430 tx=0x7fab4400d8f0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab50992640 1 -- 172.21.15.133:0/2414592970 reap_dead start 2024-04-25T10:16:49.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 -- 172.21.15.133:0/2414592970 shutdown_connections 2024-04-25T10:16:49.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 -- 172.21.15.133:0/2414592970 >> 172.21.15.133:0/2414592970 conn(0x55a690778a00 msgr2=0x55a6907c29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 -- 172.21.15.133:0/2414592970 shutdown_connections 2024-04-25T10:16:49.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.288+0000 7fab52673080 1 -- 172.21.15.133:0/2414592970 wait complete. 2024-04-25T10:16:49.293 INFO:teuthology.orchestra.run.smithi133.stdout:Ysixmegssubdir  2024-04-25T10:16:49.293 DEBUG:teuthology.orchestra.run.smithi133:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:16:49.380 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:16:49.380 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 1099511627777, 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: "ancestors": [ 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: "dirino": 1099511627776, 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: "dname": "sixmegs", 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 8 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: "dirino": 1, 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: "dname": "subdir", 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: "version": 11 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: "pool": 26, 2024-04-25T10:16:49.381 INFO:teuthology.orchestra.run.smithi133.stdout: "old_pools": [] 2024-04-25T10:16:49.382 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:16:49.382 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:49.383 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:16:49.384 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:49.462 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:49.509 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:16:49.509 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:16:49.509 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:49.515 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.680+0000 7efd71e19640 1 Processor -- start 2024-04-25T10:16:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.680+0000 7efd71e19640 1 -- start start 2024-04-25T10:16:49.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd71e19640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd6c158110 0x7efd6c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd71e19640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6c0cbe60 con 0x7efd6c1584f0 2024-04-25T10:16:49.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd71e19640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6c0b19b0 con 0x7efd6c06eb70 2024-04-25T10:16:49.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd71e19640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd6c15b360 con 0x7efd6c158110 2024-04-25T10:16:49.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd6b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd6c158110 0x7efd6c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd63fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7efd6c1584f0 0x7efd6c06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53592/0 (socket says 172.21.15.133:53592) 2024-04-25T10:16:49.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd63fff640 1 -- 172.21.15.133:0/3161163909 learned_addr learned my addr 172.21.15.133:0/3161163909 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:49.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd6b7fe640 1 -- 172.21.15.133:0/3161163909 >> v1:172.21.15.162:6789/0 conn(0x7efd6c06eb70 legacy=0x7efd6c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:49.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd6b7fe640 1 -- 172.21.15.133:0/3161163909 >> v1:172.21.15.133:6789/0 conn(0x7efd6c1584f0 legacy=0x7efd6c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd6b7fe640 1 -- 172.21.15.133:0/3161163909 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c15b640 con 0x7efd6c158110 2024-04-25T10:16:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd6b7fe640 1 --2- 172.21.15.133:0/3161163909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c158110 0x7efd6c0b1f00 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0097c0 tx=0x7efd5c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7542d385178ff3f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd6affd640 1 -- 172.21.15.133:0/3161163909 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0b0070 con 0x7efd6c158110 2024-04-25T10:16:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd6affd640 1 -- 172.21.15.133:0/3161163909 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c0ab600 con 0x7efd6c158110 2024-04-25T10:16:49.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd6affd640 1 -- 172.21.15.133:0/3161163909 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0ab9d0 con 0x7efd6c158110 2024-04-25T10:16:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd71e19640 1 -- 172.21.15.133:0/3161163909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c158110 msgr2=0x7efd6c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd71e19640 1 --2- 172.21.15.133:0/3161163909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c158110 0x7efd6c0b1f00 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0097c0 tx=0x7efd5c01c430 comp rx=0 tx=0).stop 2024-04-25T10:16:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd71e19640 1 -- 172.21.15.133:0/3161163909 shutdown_connections 2024-04-25T10:16:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd71e19640 1 --2- 172.21.15.133:0/3161163909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c158110 0x7efd6c0b1f00 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd71e19640 1 -- 172.21.15.133:0/3161163909 >> 172.21.15.133:0/3161163909 conn(0x7efd6c067000 msgr2=0x7efd6c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd71e19640 1 -- 172.21.15.133:0/3161163909 shutdown_connections 2024-04-25T10:16:49.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.684+0000 7efd71e19640 1 -- 172.21.15.133:0/3161163909 wait complete. 2024-04-25T10:16:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd71e19640 1 Processor -- start 2024-04-25T10:16:49.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd71e19640 1 -- start start 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd71e19640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c06eb70 0x7efd6c143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd71e19640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c158110 0x7efd6c15de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd71e19640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd6c1584f0 0x7efd6c14e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd71e19640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c168600 con 0x7efd6c06eb70 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd71e19640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c168480 con 0x7efd6c158110 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd71e19640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c168780 con 0x7efd6c1584f0 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd6b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c06eb70 0x7efd6c143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd6b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c06eb70 0x7efd6c143af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41482/0 (socket says 172.21.15.133:41482) 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd6b7fe640 1 -- 172.21.15.133:0/1989457035 learned_addr learned my addr 172.21.15.133:0/1989457035 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd63fff640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c158110 0x7efd6c15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd6bfff640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd6c1584f0 0x7efd6c14e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd6b7fe640 1 -- 172.21.15.133:0/1989457035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd6c1584f0 msgr2=0x7efd6c14e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd6b7fe640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd6c1584f0 0x7efd6c14e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd6b7fe640 1 -- 172.21.15.133:0/1989457035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c158110 msgr2=0x7efd6c15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd6b7fe640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c158110 0x7efd6c15de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd6b7fe640 1 -- 172.21.15.133:0/1989457035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c15e860 con 0x7efd6c06eb70 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd6b7fe640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c06eb70 0x7efd6c143af0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7efd5c019040 tx=0x7efd5c0191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd68ff9640 1 -- 172.21.15.133:0/1989457035 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0b9020 con 0x7efd6c06eb70 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd68ff9640 1 -- 172.21.15.133:0/1989457035 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c0b5480 con 0x7efd6c06eb70 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd71e19640 1 -- 172.21.15.133:0/1989457035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c15eb40 con 0x7efd6c06eb70 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd68ff9640 1 -- 172.21.15.133:0/1989457035 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0b0040 con 0x7efd6c06eb70 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.688+0000 7efd71e19640 1 -- 172.21.15.133:0/1989457035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c15f100 con 0x7efd6c06eb70 2024-04-25T10:16:49.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.692+0000 7efd71e19640 1 -- 172.21.15.133:0/1989457035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6c06bc20 con 0x7efd6c06eb70 2024-04-25T10:16:49.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.692+0000 7efd68ff9640 1 -- 172.21.15.133:0/1989457035 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd5c0bc020 con 0x7efd6c06eb70 2024-04-25T10:16:49.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.692+0000 7efd68ff9640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efd4c066f90 0x7efd4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.692+0000 7efd68ff9640 1 -- 172.21.15.133:0/1989457035 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7efd5c11f290 con 0x7efd6c06eb70 2024-04-25T10:16:49.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.692+0000 7efd63fff640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efd4c066f90 0x7efd4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.692+0000 7efd63fff640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efd4c066f90 0x7efd4c069450 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7efd6c150860 tx=0x7efd54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.696+0000 7efd68ff9640 1 -- 172.21.15.133:0/1989457035 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd5c0c3030 con 0x7efd6c06eb70 2024-04-25T10:16:49.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.856+0000 7efd71e19640 1 -- 172.21.15.133:0/1989457035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd6c068ef0 con 0x7efd6c06eb70 2024-04-25T10:16:49.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.856+0000 7efd68ff9640 1 -- 172.21.15.133:0/1989457035 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+17052 (secure 0 0 0) 0x7efd5c016050 con 0x7efd6c06eb70 2024-04-25T10:16:49.860 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:49.860 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":171,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:16:35.087106+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:07.256836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:07.975826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2","mappings":[{"from":1,"to":0}]},{"pgid":"26.8","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 -- 172.21.15.133:0/1989457035 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efd4c066f90 msgr2=0x7efd4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efd4c066f90 0x7efd4c069450 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7efd6c150860 tx=0x7efd54009040 comp rx=0 tx=0).stop 2024-04-25T10:16:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 -- 172.21.15.133:0/1989457035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c06eb70 msgr2=0x7efd6c143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c06eb70 0x7efd6c143af0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7efd5c019040 tx=0x7efd5c0191d0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 -- 172.21.15.133:0/1989457035 shutdown_connections 2024-04-25T10:16:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd6c1584f0 0x7efd6c14e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd6c158110 0x7efd6c15de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efd4c066f90 0x7efd4c069450 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 --2- 172.21.15.133:0/1989457035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c06eb70 0x7efd6c143af0 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 -- 172.21.15.133:0/1989457035 >> 172.21.15.133:0/1989457035 conn(0x7efd6c067000 msgr2=0x7efd6c0cd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 -- 172.21.15.133:0/1989457035 shutdown_connections 2024-04-25T10:16:49.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.860+0000 7efd71e19640 1 -- 172.21.15.133:0/1989457035 wait complete. 2024-04-25T10:16:49.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:07.256836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:07.975826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:49.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:49.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f2385c89640 1 Processor -- start 2024-04-25T10:16:49.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f2385c89640 1 -- start start 2024-04-25T10:16:49.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f2385c89640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f238006e8b0 0x7f238015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f2385c89640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f23800aebc0 0x7f23800af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f2385c89640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23801471f0 con 0x7f23800b1e50 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f2385c89640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2380147490 con 0x7f23800aebc0 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f2385c89640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23800b19c0 con 0x7f238006e8b0 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f237ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f23800aebc0 0x7f23800af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f237ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f23800aebc0 0x7f23800af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41488/0 (socket says 172.21.15.133:41488) 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f237ffff640 1 -- 172.21.15.133:0/995200261 learned_addr learned my addr 172.21.15.133:0/995200261 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f237f7fe640 1 --2- 172.21.15.133:0/995200261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f238006e8b0 0x7f238015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f237ffff640 1 -- 172.21.15.133:0/995200261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f238006e8b0 msgr2=0x7f238015eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f237ffff640 1 --2- 172.21.15.133:0/995200261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f238006e8b0 0x7f238015eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f237ffff640 1 -- 172.21.15.133:0/995200261 >> v1:172.21.15.162:6789/0 conn(0x7f23800b1e50 legacy=0x7f2380146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:49.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:49.996+0000 7f237ffff640 1 -- 172.21.15.133:0/995200261 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23800b1090 con 0x7f23800aebc0 2024-04-25T10:16:50.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f237ffff640 1 --2- 172.21.15.133:0/995200261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23800aebc0 0x7f23800af050 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f2370009e80 tx=0x7f237001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91ddc1d2a47320b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f237e7fc640 1 -- 172.21.15.133:0/995200261 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23700a55c0 con 0x7f23800aebc0 2024-04-25T10:16:50.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f237e7fc640 1 -- 172.21.15.133:0/995200261 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23700a5760 con 0x7f23800aebc0 2024-04-25T10:16:50.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f237e7fc640 1 -- 172.21.15.133:0/995200261 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23700a5b30 con 0x7f23800aebc0 2024-04-25T10:16:50.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 -- 172.21.15.133:0/995200261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23800aebc0 msgr2=0x7f23800af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 --2- 172.21.15.133:0/995200261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23800aebc0 0x7f23800af050 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f2370009e80 tx=0x7f237001c940 comp rx=0 tx=0).stop 2024-04-25T10:16:50.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 -- 172.21.15.133:0/995200261 shutdown_connections 2024-04-25T10:16:50.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 --2- 172.21.15.133:0/995200261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23800aebc0 0x7f23800af050 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 --2- 172.21.15.133:0/995200261 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f238006e8b0 0x7f238015eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 -- 172.21.15.133:0/995200261 >> 172.21.15.133:0/995200261 conn(0x7f2380066f70 msgr2=0x7f238006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 -- 172.21.15.133:0/995200261 shutdown_connections 2024-04-25T10:16:50.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 -- 172.21.15.133:0/995200261 wait complete. 2024-04-25T10:16:50.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 Processor -- start 2024-04-25T10:16:50.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 -- start start 2024-04-25T10:16:50.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f238006e8b0 0x7f238015b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23800aebc0 0x7f23801367b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23800b1e50 0x7f2380144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238006dc30 con 0x7f23800b1e50 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238006dab0 con 0x7f238006e8b0 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f2385c89640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238006ddb0 con 0x7f23800aebc0 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f237ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23800b1e50 0x7f2380144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f237ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23800b1e50 0x7f2380144e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41500/0 (socket says 172.21.15.133:41500) 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.000+0000 7f237ffff640 1 -- 172.21.15.133:0/2803395899 learned_addr learned my addr 172.21.15.133:0/2803395899 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f237f7fe640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f238006e8b0 0x7f238015b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f237effd640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23800aebc0 0x7f23801367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f237ffff640 1 -- 172.21.15.133:0/2803395899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23800aebc0 msgr2=0x7f23801367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f237ffff640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23800aebc0 0x7f23801367b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f237ffff640 1 -- 172.21.15.133:0/2803395899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f238006e8b0 msgr2=0x7f238015b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f237ffff640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f238006e8b0 0x7f238015b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f237ffff640 1 -- 172.21.15.133:0/2803395899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23801388d0 con 0x7f23800b1e50 2024-04-25T10:16:50.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f237ffff640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23800b1e50 0x7f2380144e00 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f23700af040 tx=0x7f23700196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f2367fff640 1 -- 172.21.15.133:0/2803395899 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23700b8670 con 0x7f23800b1e50 2024-04-25T10:16:50.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f2367fff640 1 -- 172.21.15.133:0/2803395899 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23700be070 con 0x7f23800b1e50 2024-04-25T10:16:50.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f2385c89640 1 -- 172.21.15.133:0/2803395899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2380138bb0 con 0x7f23800b1e50 2024-04-25T10:16:50.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f2367fff640 1 -- 172.21.15.133:0/2803395899 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23700b8a90 con 0x7f23800b1e50 2024-04-25T10:16:50.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f2385c89640 1 -- 172.21.15.133:0/2803395899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2380139170 con 0x7f23800b1e50 2024-04-25T10:16:50.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f2385c89640 1 -- 172.21.15.133:0/2803395899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2380068e80 con 0x7f23800b1e50 2024-04-25T10:16:50.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f2367fff640 1 -- 172.21.15.133:0/2803395899 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23700b8c30 con 0x7f23800b1e50 2024-04-25T10:16:50.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f2367fff640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f236c066f60 0x7f236c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f2367fff640 1 -- 172.21.15.133:0/2803395899 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f2370120090 con 0x7f23800b1e50 2024-04-25T10:16:50.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.004+0000 7f237f7fe640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f236c066f60 0x7f236c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.008+0000 7f237f7fe640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f236c066f60 0x7f236c069420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2378007a60 tx=0x7f2378008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.012+0000 7f2367fff640 1 -- 172.21.15.133:0/2803395899 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23700165c0 con 0x7f23800b1e50 2024-04-25T10:16:50.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 -- 172.21.15.133:0/2803395899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f238006b960 con 0x7f23800b1e50 2024-04-25T10:16:50.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2367fff640 1 -- 172.21.15.133:0/2803395899 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v171) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23700f07e0 con 0x7f23800b1e50 2024-04-25T10:16:50.174 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:16:50.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 -- 172.21.15.133:0/2803395899 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f236c066f60 msgr2=0x7f236c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f236c066f60 0x7f236c069420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2378007a60 tx=0x7f2378008040 comp rx=0 tx=0).stop 2024-04-25T10:16:50.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 -- 172.21.15.133:0/2803395899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23800b1e50 msgr2=0x7f2380144e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23800b1e50 0x7f2380144e00 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f23700af040 tx=0x7f23700196e0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 -- 172.21.15.133:0/2803395899 shutdown_connections 2024-04-25T10:16:50.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f236c066f60 0x7f236c069420 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23800b1e50 0x7f2380144e00 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f23800aebc0 0x7f23801367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 --2- 172.21.15.133:0/2803395899 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f238006e8b0 0x7f238015b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 -- 172.21.15.133:0/2803395899 >> 172.21.15.133:0/2803395899 conn(0x7f2380066f70 msgr2=0x7f23801400a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 -- 172.21.15.133:0/2803395899 shutdown_connections 2024-04-25T10:16:50.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.172+0000 7f2385c89640 1 -- 172.21.15.133:0/2803395899 wait complete. 2024-04-25T10:16:50.193 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb91093c640 1 Processor -- start 2024-04-25T10:16:50.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb91093c640 1 -- start start 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb91093c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb90c0cc490 0x7fb90c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb91093c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb90c0d0060 0x7fb90c06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb91093c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90c06b710 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb91093c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb90c1623d0 con 0x7fb90c0cc490 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb91093c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb90c1617c0 con 0x7fb90c0d0060 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb909d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb90c0cc490 0x7fb90c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb90a59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb90c0cc0b0 0x7fb90c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53602/0 (socket says 172.21.15.133:53602) 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb90a59c640 1 -- 172.21.15.133:0/1008162422 learned_addr learned my addr 172.21.15.133:0/1008162422 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb90ad9d640 1 --2- 172.21.15.133:0/1008162422 >> v2:172.21.15.162:3300/0 conn(0x7fb90c0d0060 0x7fb90c06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb909d9b640 1 -- 172.21.15.133:0/1008162422 >> v2:172.21.15.162:3300/0 conn(0x7fb90c0d0060 msgr2=0x7fb90c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb909d9b640 1 --2- 172.21.15.133:0/1008162422 >> v2:172.21.15.162:3300/0 conn(0x7fb90c0d0060 0x7fb90c06b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb909d9b640 1 -- 172.21.15.133:0/1008162422 >> v1:172.21.15.133:6789/0 conn(0x7fb90c0cc0b0 legacy=0x7fb90c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb909d9b640 1 -- 172.21.15.133:0/1008162422 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c161aa0 con 0x7fb90c0cc490 2024-04-25T10:16:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.312+0000 7fb90959a640 1 -- 172.21.15.133:0/1008162422 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570198704 0 0) 0x7fb90c06b710 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb909d9b640 1 --2- 172.21.15.133:0/1008162422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb90c0cc490 0x7fb90c06ac90 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fb90c072dd0 tx=0x7fb90001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13f04d85bb5a79e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb90959a640 1 -- 172.21.15.133:0/1008162422 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9000b2070 con 0x7fb90c0cc490 2024-04-25T10:16:50.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb90959a640 1 -- 172.21.15.133:0/1008162422 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9000a9070 con 0x7fb90c0cc490 2024-04-25T10:16:50.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb90959a640 1 -- 172.21.15.133:0/1008162422 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9000ad880 con 0x7fb90c0cc490 2024-04-25T10:16:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 -- 172.21.15.133:0/1008162422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb90c0cc490 msgr2=0x7fb90c06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 --2- 172.21.15.133:0/1008162422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb90c0cc490 0x7fb90c06ac90 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fb90c072dd0 tx=0x7fb90001c450 comp rx=0 tx=0).stop 2024-04-25T10:16:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 -- 172.21.15.133:0/1008162422 shutdown_connections 2024-04-25T10:16:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 --2- 172.21.15.133:0/1008162422 >> v2:172.21.15.162:3300/0 conn(0x7fb90c0d0060 0x7fb90c06b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 --2- 172.21.15.133:0/1008162422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb90c0cc490 0x7fb90c06ac90 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 -- 172.21.15.133:0/1008162422 >> 172.21.15.133:0/1008162422 conn(0x7fb90c0999d0 msgr2=0x7fb90c0d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 -- 172.21.15.133:0/1008162422 shutdown_connections 2024-04-25T10:16:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 -- 172.21.15.133:0/1008162422 wait complete. 2024-04-25T10:16:50.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 Processor -- start 2024-04-25T10:16:50.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 -- start start 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb90c0cc0b0 0x7fb90c158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb90c0cc490 0x7fb90c0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb90c0d0060 0x7fb90c0cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c15de60 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c15dce0 con 0x7fb90c0d0060 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.316+0000 7fb91093c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90c15dfe0 con 0x7fb90c0cc490 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb90a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb90c0cc0b0 0x7fb90c158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb90a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb90c0cc0b0 0x7fb90c158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41524/0 (socket says 172.21.15.133:41524) 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb90a59c640 1 -- 172.21.15.133:0/4176060565 learned_addr learned my addr 172.21.15.133:0/4176060565 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb909d9b640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb90c0cc490 0x7fb90c0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb90a59c640 1 -- 172.21.15.133:0/4176060565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb90c0cc490 msgr2=0x7fb90c0b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb90a59c640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb90c0cc490 0x7fb90c0b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb90ad9d640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb90c0d0060 0x7fb90c0cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb90a59c640 1 -- 172.21.15.133:0/4176060565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb90c0d0060 msgr2=0x7fb90c0cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb90a59c640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb90c0d0060 0x7fb90c0cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb90a59c640 1 -- 172.21.15.133:0/4176060565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c161100 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb90a59c640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb90c0cc0b0 0x7fb90c158e10 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00c410 tx=0x7fb8fc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb8faffd640 1 -- 172.21.15.133:0/4176060565 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0074e0 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb8faffd640 1 -- 172.21.15.133:0/4176060565 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8fc0047f0 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb8faffd640 1 -- 172.21.15.133:0/4176060565 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc007680 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb91093c640 1 -- 172.21.15.133:0/4176060565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90c0b4300 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb91093c640 1 -- 172.21.15.133:0/4176060565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90c156a20 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb91093c640 1 -- 172.21.15.133:0/4176060565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90c067c70 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb8faffd640 1 -- 172.21.15.133:0/4176060565 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8fc011070 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb8faffd640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8ec066f60 0x7fb8ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb8faffd640 1 -- 172.21.15.133:0/4176060565 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fb8fc084e10 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb909d9b640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8ec066f60 0x7fb8ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.320+0000 7fb909d9b640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8ec066f60 0x7fb8ec069420 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb9000af000 tx=0x7fb9000a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.324+0000 7fb8faffd640 1 -- 172.21.15.133:0/4176060565 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb90c161100 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.480+0000 7fb91093c640 1 -- 172.21.15.133:0/4176060565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb90c0b6540 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.480+0000 7fb8faffd640 1 -- 172.21.15.133:0/4176060565 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v171) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8fc055c40 con 0x7fb90c0cc0b0 2024-04-25T10:16:50.482 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:16:50.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.480+0000 7fb91093c640 1 -- 172.21.15.133:0/4176060565 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8ec066f60 msgr2=0x7fb8ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.480+0000 7fb91093c640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8ec066f60 0x7fb8ec069420 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb9000af000 tx=0x7fb9000a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:50.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.484+0000 7fb91093c640 1 -- 172.21.15.133:0/4176060565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb90c0cc0b0 msgr2=0x7fb90c158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.484+0000 7fb91093c640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb90c0cc0b0 0x7fb90c158e10 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00c410 tx=0x7fb8fc00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.484+0000 7fb91093c640 1 -- 172.21.15.133:0/4176060565 shutdown_connections 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.484+0000 7fb91093c640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb90c0d0060 0x7fb90c0cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.484+0000 7fb91093c640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb90c0cc490 0x7fb90c0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.484+0000 7fb91093c640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb8ec066f60 0x7fb8ec069420 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.484+0000 7fb91093c640 1 --2- 172.21.15.133:0/4176060565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb90c0cc0b0 0x7fb90c158e10 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.484+0000 7fb91093c640 1 -- 172.21.15.133:0/4176060565 >> 172.21.15.133:0/4176060565 conn(0x7fb90c0999d0 msgr2=0x7fb90c0b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.484+0000 7fb91093c640 1 -- 172.21.15.133:0/4176060565 shutdown_connections 2024-04-25T10:16:50.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.484+0000 7fb91093c640 1 -- 172.21.15.133:0/4176060565 wait complete. 2024-04-25T10:16:50.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba3241b640 1 Processor -- start 2024-04-25T10:16:50.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba3241b640 1 -- start start 2024-04-25T10:16:50.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba3241b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fba2c15b8b0 0x7fba2c0ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba3241b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba2c0d03a0 0x7fba2c0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba3241b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2c0cc6a0 con 0x7fba2c15bc90 2024-04-25T10:16:50.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba3241b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2c0cc980 con 0x7fba2c0d03a0 2024-04-25T10:16:50.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba3241b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2c0ccc60 con 0x7fba2c15b8b0 2024-04-25T10:16:50.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba309b8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba2c0d03a0 0x7fba2c0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba2b7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fba2c15bc90 0x7fba2c0cfc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53610/0 (socket says 172.21.15.133:53610) 2024-04-25T10:16:50.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba2b7fe640 1 -- 172.21.15.133:0/2599777841 learned_addr learned my addr 172.21.15.133:0/2599777841 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:50.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba2bfff640 1 --2- 172.21.15.133:0/2599777841 >> v2:172.21.15.162:3300/0 conn(0x7fba2c15b8b0 0x7fba2c0ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba309b8640 1 -- 172.21.15.133:0/2599777841 >> v2:172.21.15.162:3300/0 conn(0x7fba2c15b8b0 msgr2=0x7fba2c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:50.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba309b8640 1 --2- 172.21.15.133:0/2599777841 >> v2:172.21.15.162:3300/0 conn(0x7fba2c15b8b0 0x7fba2c0ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba309b8640 1 -- 172.21.15.133:0/2599777841 >> v1:172.21.15.133:6789/0 conn(0x7fba2c15bc90 legacy=0x7fba2c0cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba309b8640 1 -- 172.21.15.133:0/2599777841 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c0ccf40 con 0x7fba2c0d03a0 2024-04-25T10:16:50.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.624+0000 7fba2bfff640 1 --2- 172.21.15.133:0/2599777841 >> v2:172.21.15.162:3300/0 conn(0x7fba2c15b8b0 0x7fba2c0ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:50.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba309b8640 1 --2- 172.21.15.133:0/2599777841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba2c0d03a0 0x7fba2c0b0aa0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7fba2c072c90 tx=0x7fba2401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64d0e9780c07611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2affd640 1 -- 172.21.15.133:0/2599777841 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240b2070 con 0x7fba2c0d03a0 2024-04-25T10:16:50.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2affd640 1 -- 172.21.15.133:0/2599777841 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba240a5870 con 0x7fba2c0d03a0 2024-04-25T10:16:50.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2affd640 1 -- 172.21.15.133:0/2599777841 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240a5c40 con 0x7fba2c0d03a0 2024-04-25T10:16:50.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 -- 172.21.15.133:0/2599777841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba2c0d03a0 msgr2=0x7fba2c0b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 --2- 172.21.15.133:0/2599777841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba2c0d03a0 0x7fba2c0b0aa0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7fba2c072c90 tx=0x7fba2401c8a0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 -- 172.21.15.133:0/2599777841 shutdown_connections 2024-04-25T10:16:50.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 --2- 172.21.15.133:0/2599777841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba2c0d03a0 0x7fba2c0b0aa0 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 --2- 172.21.15.133:0/2599777841 >> v2:172.21.15.162:3300/0 conn(0x7fba2c15b8b0 0x7fba2c0ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 -- 172.21.15.133:0/2599777841 >> 172.21.15.133:0/2599777841 conn(0x7fba2c09a990 msgr2=0x7fba2c0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 -- 172.21.15.133:0/2599777841 shutdown_connections 2024-04-25T10:16:50.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 -- 172.21.15.133:0/2599777841 wait complete. 2024-04-25T10:16:50.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 Processor -- start 2024-04-25T10:16:50.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 -- start start 2024-04-25T10:16:50.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba2c0d03a0 0x7fba2c14fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba2c15b8b0 0x7fba2c0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba2c15bc90 0x7fba2c154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0b17f0 con 0x7fba2c0d03a0 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0cc440 con 0x7fba2c15bc90 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba3241b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0b1970 con 0x7fba2c15b8b0 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba2c0d03a0 0x7fba2c14fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba2c0d03a0 0x7fba2c14fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41540/0 (socket says 172.21.15.133:41540) 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2bfff640 1 -- 172.21.15.133:0/251360811 learned_addr learned my addr 172.21.15.133:0/251360811 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2b7fe640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba2c15b8b0 0x7fba2c0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2bfff640 1 -- 172.21.15.133:0/251360811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba2c15b8b0 msgr2=0x7fba2c0c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba309b8640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba2c15bc90 0x7fba2c154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2bfff640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba2c15b8b0 0x7fba2c0c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2bfff640 1 -- 172.21.15.133:0/251360811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba2c15bc90 msgr2=0x7fba2c154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2bfff640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba2c15bc90 0x7fba2c154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2bfff640 1 -- 172.21.15.133:0/251360811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c155a10 con 0x7fba2c0d03a0 2024-04-25T10:16:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.628+0000 7fba2bfff640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba2c0d03a0 0x7fba2c14fe40 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c7f0 tx=0x7fba1c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.632+0000 7fba28ff9640 1 -- 172.21.15.133:0/251360811 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c007a30 con 0x7fba2c0d03a0 2024-04-25T10:16:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.632+0000 7fba28ff9640 1 -- 172.21.15.133:0/251360811 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c007bd0 con 0x7fba2c0d03a0 2024-04-25T10:16:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.632+0000 7fba3241b640 1 -- 172.21.15.133:0/251360811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c06f410 con 0x7fba2c0d03a0 2024-04-25T10:16:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.632+0000 7fba28ff9640 1 -- 172.21.15.133:0/251360811 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0113f0 con 0x7fba2c0d03a0 2024-04-25T10:16:50.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.632+0000 7fba3241b640 1 -- 172.21.15.133:0/251360811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c06f9d0 con 0x7fba2c0d03a0 2024-04-25T10:16:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.632+0000 7fba3241b640 1 -- 172.21.15.133:0/251360811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c06f200 con 0x7fba2c0d03a0 2024-04-25T10:16:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.632+0000 7fba28ff9640 1 -- 172.21.15.133:0/251360811 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba1c011590 con 0x7fba2c0d03a0 2024-04-25T10:16:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.632+0000 7fba28ff9640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba18066f60 0x7fba18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.632+0000 7fba28ff9640 1 -- 172.21.15.133:0/251360811 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fba1c0854e0 con 0x7fba2c0d03a0 2024-04-25T10:16:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.632+0000 7fba2b7fe640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba18066f60 0x7fba18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.632+0000 7fba2b7fe640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba18066f60 0x7fba18069420 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0d0b00 tx=0x7fba20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.636+0000 7fba28ff9640 1 -- 172.21.15.133:0/251360811 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba1c015020 con 0x7fba2c0d03a0 2024-04-25T10:16:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.784+0000 7fba3241b640 1 -- 172.21.15.133:0/251360811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba2c067c70 con 0x7fba2c0d03a0 2024-04-25T10:16:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.784+0000 7fba28ff9640 1 -- 172.21.15.133:0/251360811 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v171) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba1c056280 con 0x7fba2c0d03a0 2024-04-25T10:16:50.787 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:16:50.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 -- 172.21.15.133:0/251360811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba18066f60 msgr2=0x7fba18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba18066f60 0x7fba18069420 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0d0b00 tx=0x7fba20009040 comp rx=0 tx=0).stop 2024-04-25T10:16:50.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 -- 172.21.15.133:0/251360811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba2c0d03a0 msgr2=0x7fba2c14fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba2c0d03a0 0x7fba2c14fe40 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c7f0 tx=0x7fba1c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 -- 172.21.15.133:0/251360811 shutdown_connections 2024-04-25T10:16:50.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba2c15bc90 0x7fba2c154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba2c15b8b0 0x7fba2c0c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba18066f60 0x7fba18069420 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 --2- 172.21.15.133:0/251360811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba2c0d03a0 0x7fba2c14fe40 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 -- 172.21.15.133:0/251360811 >> 172.21.15.133:0/251360811 conn(0x7fba2c09a990 msgr2=0x7fba2c15a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 -- 172.21.15.133:0/251360811 shutdown_connections 2024-04-25T10:16:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.788+0000 7fba3241b640 1 -- 172.21.15.133:0/251360811 wait complete. 2024-04-25T10:16:50.805 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.916+0000 7f9d6dea6640 1 Processor -- start 2024-04-25T10:16:50.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.916+0000 7f9d6dea6640 1 -- start start 2024-04-25T10:16:50.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.916+0000 7f9d6dea6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9d68142920 0x7f9d68143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.916+0000 7f9d6dea6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9d68140a40 0x7f9d680ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.916+0000 7f9d6dea6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d680cf000 con 0x7f9d6815d9f0 2024-04-25T10:16:50.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.916+0000 7f9d6dea6640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d68140ea0 con 0x7f9d68142920 2024-04-25T10:16:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.916+0000 7f9d6dea6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d680cf280 con 0x7f9d68140a40 2024-04-25T10:16:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.916+0000 7f9d677fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9d68142920 0x7f9d68143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.916+0000 7f9d677fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9d68142920 0x7f9d68143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41548/0 (socket says 172.21.15.133:41548) 2024-04-25T10:16:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.916+0000 7f9d677fe640 1 -- 172.21.15.133:0/726515043 learned_addr learned my addr 172.21.15.133:0/726515043 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.916+0000 7f9d5effd640 1 --2- 172.21.15.133:0/726515043 >> v2:172.21.15.162:3300/0 conn(0x7f9d68140a40 0x7f9d680ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d677fe640 1 -- 172.21.15.133:0/726515043 >> v2:172.21.15.162:3300/0 conn(0x7f9d68140a40 msgr2=0x7f9d680ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d677fe640 1 --2- 172.21.15.133:0/726515043 >> v2:172.21.15.162:3300/0 conn(0x7f9d68140a40 0x7f9d680ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d677fe640 1 -- 172.21.15.133:0/726515043 >> v1:172.21.15.162:6789/0 conn(0x7f9d6815d9f0 legacy=0x7f9d6806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d677fe640 1 -- 172.21.15.133:0/726515043 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6806d440 con 0x7f9d68142920 2024-04-25T10:16:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d677fe640 1 --2- 172.21.15.133:0/726515043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d68142920 0x7f9d68143560 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f9d6806f440 tx=0x7f9d5801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb99174b712f9ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d66ffd640 1 -- 172.21.15.133:0/726515043 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d580b2070 con 0x7f9d68142920 2024-04-25T10:16:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d66ffd640 1 -- 172.21.15.133:0/726515043 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d580ad680 con 0x7f9d68142920 2024-04-25T10:16:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d66ffd640 1 -- 172.21.15.133:0/726515043 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d580ada50 con 0x7f9d68142920 2024-04-25T10:16:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d6dea6640 1 -- 172.21.15.133:0/726515043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d68142920 msgr2=0x7f9d68143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d6dea6640 1 --2- 172.21.15.133:0/726515043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d68142920 0x7f9d68143560 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f9d6806f440 tx=0x7f9d5801c490 comp rx=0 tx=0).stop 2024-04-25T10:16:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d6dea6640 1 -- 172.21.15.133:0/726515043 shutdown_connections 2024-04-25T10:16:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d6dea6640 1 --2- 172.21.15.133:0/726515043 >> v2:172.21.15.162:3300/0 conn(0x7f9d68140a40 0x7f9d680ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d6dea6640 1 --2- 172.21.15.133:0/726515043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d68142920 0x7f9d68143560 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d6dea6640 1 -- 172.21.15.133:0/726515043 >> 172.21.15.133:0/726515043 conn(0x7f9d68066f70 msgr2=0x7f9d6806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d6dea6640 1 -- 172.21.15.133:0/726515043 shutdown_connections 2024-04-25T10:16:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.920+0000 7f9d6dea6640 1 -- 172.21.15.133:0/726515043 wait complete. 2024-04-25T10:16:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d6dea6640 1 Processor -- start 2024-04-25T10:16:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d6dea6640 1 -- start start 2024-04-25T10:16:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d6dea6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d68140a40 0x7f9d68158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d6dea6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d68142920 0x7f9d680ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d6dea6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d6815d9f0 0x7f9d6813ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d6dea6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6806ebc0 con 0x7f9d6815d9f0 2024-04-25T10:16:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d6dea6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6806ea40 con 0x7f9d68140a40 2024-04-25T10:16:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d6dea6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d68142d00 con 0x7f9d68142920 2024-04-25T10:16:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d67fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d6815d9f0 0x7f9d6813ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d67fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d6815d9f0 0x7f9d6813ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41554/0 (socket says 172.21.15.133:41554) 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d67fff640 1 -- 172.21.15.133:0/3461575034 learned_addr learned my addr 172.21.15.133:0/3461575034 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d677fe640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d68140a40 0x7f9d68158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d67fff640 1 -- 172.21.15.133:0/3461575034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d68142920 msgr2=0x7f9d680ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d5effd640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d68142920 0x7f9d680ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d67fff640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d68142920 0x7f9d680ad4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d67fff640 1 -- 172.21.15.133:0/3461575034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d68140a40 msgr2=0x7f9d68158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d67fff640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d68140a40 0x7f9d68158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d67fff640 1 -- 172.21.15.133:0/3461575034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d680ada30 con 0x7f9d6815d9f0 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d677fe640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d68140a40 0x7f9d68158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d67fff640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d6815d9f0 0x7f9d6813ff90 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000c730 tx=0x7f9d6000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d64ff9640 1 -- 172.21.15.133:0/3461575034 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60016020 con 0x7f9d6815d9f0 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d64ff9640 1 -- 172.21.15.133:0/3461575034 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d60006d60 con 0x7f9d6815d9f0 2024-04-25T10:16:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d6dea6640 1 -- 172.21.15.133:0/3461575034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d680adcd0 con 0x7f9d6815d9f0 2024-04-25T10:16:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d64ff9640 1 -- 172.21.15.133:0/3461575034 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60003bc0 con 0x7f9d6815d9f0 2024-04-25T10:16:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d6dea6640 1 -- 172.21.15.133:0/3461575034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d68158370 con 0x7f9d6815d9f0 2024-04-25T10:16:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d6dea6640 1 -- 172.21.15.133:0/3461575034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d680ae1f0 con 0x7f9d6815d9f0 2024-04-25T10:16:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d64ff9640 1 -- 172.21.15.133:0/3461575034 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d600050d0 con 0x7f9d6815d9f0 2024-04-25T10:16:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d64ff9640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d48066f60 0x7f9d48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d64ff9640 1 -- 172.21.15.133:0/3461575034 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f9d60084da0 con 0x7f9d6815d9f0 2024-04-25T10:16:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.924+0000 7f9d677fe640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d48066f60 0x7f9d48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.928+0000 7f9d677fe640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d48066f60 0x7f9d48069420 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9d580af000 tx=0x7f9d58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:50.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:50.928+0000 7f9d64ff9640 1 -- 172.21.15.133:0/3461575034 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d680ae1f0 con 0x7f9d6815d9f0 2024-04-25T10:16:51.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.088+0000 7f9d6dea6640 1 -- 172.21.15.133:0/3461575034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9d68068e80 con 0x7f9d6815d9f0 2024-04-25T10:16:51.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.088+0000 7f9d64ff9640 1 -- 172.21.15.133:0/3461575034 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v171) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9d60020020 con 0x7f9d6815d9f0 2024-04-25T10:16:51.091 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:16:51.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 -- 172.21.15.133:0/3461575034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d48066f60 msgr2=0x7f9d48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d48066f60 0x7f9d48069420 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9d580af000 tx=0x7f9d58008040 comp rx=0 tx=0).stop 2024-04-25T10:16:51.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 -- 172.21.15.133:0/3461575034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d6815d9f0 msgr2=0x7f9d6813ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d6815d9f0 0x7f9d6813ff90 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000c730 tx=0x7f9d6000cb30 comp rx=0 tx=0).stop 2024-04-25T10:16:51.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 -- 172.21.15.133:0/3461575034 shutdown_connections 2024-04-25T10:16:51.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9d48066f60 0x7f9d48069420 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d6815d9f0 0x7f9d6813ff90 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9d68142920 0x7f9d680ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 --2- 172.21.15.133:0/3461575034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9d68140a40 0x7f9d68158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 -- 172.21.15.133:0/3461575034 >> 172.21.15.133:0/3461575034 conn(0x7f9d68066f70 msgr2=0x7f9d680ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:51.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 -- 172.21.15.133:0/3461575034 shutdown_connections 2024-04-25T10:16:51.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.092+0000 7f9d6dea6640 1 -- 172.21.15.133:0/3461575034 wait complete. 2024-04-25T10:16:51.108 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:16:51.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:51.109 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:51.119 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[89319]: fuse finished with error 0 and tester_r 0 2024-04-25T10:16:51.121 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:16:51.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:51.122 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:16:51.190 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:16:51.190 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:51.197 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:51.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:16:51.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:51.197 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:16:51.243 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:16:51.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:16:51.244 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:16:51.244 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:51.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:51.249 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:16:51.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:51.249 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:16:51.296 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:16:51.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:16:51.297 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:16:51.297 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:51.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:51.303 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:16:51.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:51.303 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:16:51.351 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:16:51.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:16:51.352 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:51.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff70807d640 1 Processor -- start 2024-04-25T10:16:51.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff70807d640 1 -- start start 2024-04-25T10:16:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff70807d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff7000cee80 0x7ff700168850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff70807d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7000ca9b0 con 0x7ff70015d9b0 2024-04-25T10:16:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff70807d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7000cac90 con 0x7ff70015dd90 2024-04-25T10:16:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff70807d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7000cbe30 con 0x7ff7000cee80 2024-04-25T10:16:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff70661a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff7000cee80 0x7ff700168850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff705e19640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff70015d9b0 0x7ff7000cb5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53616/0 (socket says 172.21.15.133:53616) 2024-04-25T10:16:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff70661a640 1 --2- 172.21.15.133:0/2354005295 >> v2:172.21.15.133:3300/0 conn(0x7ff7000cee80 0x7ff700168850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41558/0 (socket says 172.21.15.133:41558) 2024-04-25T10:16:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff705e19640 1 -- 172.21.15.133:0/2354005295 learned_addr learned my addr 172.21.15.133:0/2354005295 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:51.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff70661a640 1 -- 172.21.15.133:0/2354005295 >> v1:172.21.15.162:6789/0 conn(0x7ff70015dd90 legacy=0x7ff7000ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff704e17640 1 -- 172.21.15.133:0/2354005295 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540535052 0 0) 0x7ff7000ca9b0 con 0x7ff70015d9b0 2024-04-25T10:16:51.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff70661a640 1 -- 172.21.15.133:0/2354005295 >> v1:172.21.15.133:6789/0 conn(0x7ff70015d9b0 legacy=0x7ff7000cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.476+0000 7ff70661a640 1 -- 172.21.15.133:0/2354005295 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000caf70 con 0x7ff7000cee80 2024-04-25T10:16:51.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70661a640 1 --2- 172.21.15.133:0/2354005295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7000cee80 0x7ff700168850 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009d90 tx=0x7ff6f801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f05853f8aca0f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:51.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff704e17640 1 -- 172.21.15.133:0/2354005295 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80b2070 con 0x7ff7000cee80 2024-04-25T10:16:51.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff704e17640 1 -- 172.21.15.133:0/2354005295 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f80a59c0 con 0x7ff7000cee80 2024-04-25T10:16:51.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff704e17640 1 -- 172.21.15.133:0/2354005295 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80a5d90 con 0x7ff7000cee80 2024-04-25T10:16:51.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 -- 172.21.15.133:0/2354005295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7000cee80 msgr2=0x7ff700168850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 --2- 172.21.15.133:0/2354005295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7000cee80 0x7ff700168850 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009d90 tx=0x7ff6f801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 -- 172.21.15.133:0/2354005295 shutdown_connections 2024-04-25T10:16:51.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 --2- 172.21.15.133:0/2354005295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7000cee80 0x7ff700168850 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 -- 172.21.15.133:0/2354005295 >> 172.21.15.133:0/2354005295 conn(0x7ff700067000 msgr2=0x7ff70006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:51.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 -- 172.21.15.133:0/2354005295 shutdown_connections 2024-04-25T10:16:51.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 -- 172.21.15.133:0/2354005295 wait complete. 2024-04-25T10:16:51.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 Processor -- start 2024-04-25T10:16:51.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 -- start start 2024-04-25T10:16:51.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7000cee80 0x7ff70014edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:51.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70015d9b0 0x7ff70015e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:51.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70015dd90 0x7ff700137e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:51.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70016bf60 con 0x7ff70015dd90 2024-04-25T10:16:51.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70016bde0 con 0x7ff70015d9b0 2024-04-25T10:16:51.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70016c0e0 con 0x7ff7000cee80 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70661a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70015dd90 0x7ff700137e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70661a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70015dd90 0x7ff700137e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41568/0 (socket says 172.21.15.133:41568) 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70661a640 1 -- 172.21.15.133:0/2451133071 learned_addr learned my addr 172.21.15.133:0/2451133071 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70661a640 1 -- 172.21.15.133:0/2451133071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7000cee80 msgr2=0x7ff70014edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff705e19640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7000cee80 0x7ff70014edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff705618640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70015d9b0 0x7ff70015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70661a640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7000cee80 0x7ff70014edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70661a640 1 -- 172.21.15.133:0/2451133071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70015d9b0 msgr2=0x7ff70015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70661a640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70015d9b0 0x7ff70015e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70661a640 1 -- 172.21.15.133:0/2451133071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000cd3e0 con 0x7ff70015dd90 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff705e19640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7000cee80 0x7ff70014edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff705618640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70015d9b0 0x7ff70015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70661a640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70015dd90 0x7ff700137e10 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80b0040 tx=0x7ff6f8019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff6f67fc640 1 -- 172.21.15.133:0/2451133071 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80bd020 con 0x7ff70015dd90 2024-04-25T10:16:51.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff6f67fc640 1 -- 172.21.15.133:0/2451133071 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f80ad740 con 0x7ff70015dd90 2024-04-25T10:16:51.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 -- 172.21.15.133:0/2451133071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70015e6b0 con 0x7ff70015dd90 2024-04-25T10:16:51.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff6f67fc640 1 -- 172.21.15.133:0/2451133071 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80b2040 con 0x7ff70015dd90 2024-04-25T10:16:51.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.480+0000 7ff70807d640 1 -- 172.21.15.133:0/2451133071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70014dd20 con 0x7ff70015dd90 2024-04-25T10:16:51.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.484+0000 7ff70807d640 1 -- 172.21.15.133:0/2451133071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff700068ef0 con 0x7ff70015dd90 2024-04-25T10:16:51.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.484+0000 7ff6f67fc640 1 -- 172.21.15.133:0/2451133071 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6f80c0020 con 0x7ff70015dd90 2024-04-25T10:16:51.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.484+0000 7ff6f67fc640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6e4066f60 0x7ff6e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:51.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.484+0000 7ff6f67fc640 1 -- 172.21.15.133:0/2451133071 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7ff6f81202e0 con 0x7ff70015dd90 2024-04-25T10:16:51.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.484+0000 7ff705e19640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6e4066f60 0x7ff6e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:51.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.484+0000 7ff705e19640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6e4066f60 0x7ff6e4069420 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc000f70 tx=0x7ff6fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:51.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.488+0000 7ff6f67fc640 1 -- 172.21.15.133:0/2451133071 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff6f80c7070 con 0x7ff70015dd90 2024-04-25T10:16:51.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.680+0000 7ff70807d640 1 -- 172.21.15.133:0/2451133071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7000b1750 con 0x7ff70015dd90 2024-04-25T10:16:51.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff6f67fc640 1 -- 172.21.15.133:0/2451133071 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4962 (secure 0 0 0) 0x7ff6f8016560 con 0x7ff70015dd90 2024-04-25T10:16:51.685 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:51.685 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1498957356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1498957356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1498957356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/883424012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":883424012},{"type":"v1","addr":"172.21.15.133:6835","nonce":883424012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2040533186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2040533186},{"type":"v1","addr":"172.21.15.133:6837","nonce":2040533186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:08.983208+0000","modified":"2024-04-25T10:16:11.017848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11037":{"gid":11037,"name":"d","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/2658062772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2658062772},{"type":"v1","addr":"172.21.15.162:6833","nonce":2658062772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:16:51.685 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 105 2024-04-25T10:16:51.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 -- 172.21.15.133:0/2451133071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6e4066f60 msgr2=0x7ff6e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6e4066f60 0x7ff6e4069420 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc000f70 tx=0x7ff6fc009040 comp rx=0 tx=0).stop 2024-04-25T10:16:51.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 -- 172.21.15.133:0/2451133071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70015dd90 msgr2=0x7ff700137e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70015dd90 0x7ff700137e10 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80b0040 tx=0x7ff6f8019680 comp rx=0 tx=0).stop 2024-04-25T10:16:51.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 -- 172.21.15.133:0/2451133071 shutdown_connections 2024-04-25T10:16:51.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff6e4066f60 0x7ff6e4069420 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff70015dd90 0x7ff700137e10 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff70015d9b0 0x7ff70015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 --2- 172.21.15.133:0/2451133071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff7000cee80 0x7ff70014edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 -- 172.21.15.133:0/2451133071 >> 172.21.15.133:0/2451133071 conn(0x7ff700067000 msgr2=0x7ff700155940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:51.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 -- 172.21.15.133:0/2451133071 shutdown_connections 2024-04-25T10:16:51.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.684+0000 7ff70807d640 1 -- 172.21.15.133:0/2451133071 wait complete. 2024-04-25T10:16:51.705 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:51.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f517a2cf640 1 Processor -- start 2024-04-25T10:16:51.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f517a2cf640 1 -- start start 2024-04-25T10:16:51.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f517a2cf640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51740d0770 0x7f51740d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:51.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f517a2cf640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f51740b87e0 0x7f51740b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:51.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f517a2cf640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51740ceb20 con 0x7f517415db00 2024-04-25T10:16:51.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f517a2cf640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51740d1110 con 0x7f51740d0770 2024-04-25T10:16:51.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f517a2cf640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51740b30a0 con 0x7f51740b87e0 2024-04-25T10:16:51.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f5173fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f517415db00 0x7f51740b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53630/0 (socket says 172.21.15.133:53630) 2024-04-25T10:16:51.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f5173fff640 1 -- 172.21.15.133:0/4067850301 learned_addr learned my addr 172.21.15.133:0/4067850301 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:51.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f516bfff640 1 --2- 172.21.15.133:0/4067850301 >> v2:172.21.15.133:3300/0 conn(0x7f51740d0770 0x7f51740d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:51.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f517886c640 1 --2- 172.21.15.133:0/4067850301 >> v2:172.21.15.162:3300/0 conn(0x7f51740b87e0 0x7f51740b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:51.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f516bfff640 1 -- 172.21.15.133:0/4067850301 >> v2:172.21.15.162:3300/0 conn(0x7f51740b87e0 msgr2=0x7f51740b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:51.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f516bfff640 1 --2- 172.21.15.133:0/4067850301 >> v2:172.21.15.162:3300/0 conn(0x7f51740b87e0 0x7f51740b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f516bfff640 1 -- 172.21.15.133:0/4067850301 >> v1:172.21.15.133:6789/0 conn(0x7f517415db00 legacy=0x7f51740b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f516bfff640 1 -- 172.21.15.133:0/4067850301 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51740b7740 con 0x7f51740d0770 2024-04-25T10:16:51.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.820+0000 7f51737fe640 1 -- 172.21.15.133:0/4067850301 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670880424 0 0) 0x7f51740ceb20 con 0x7f517415db00 2024-04-25T10:16:51.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f516bfff640 1 --2- 172.21.15.133:0/4067850301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51740d0770 0x7f51740d0bd0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f51740b5a00 tx=0x7f515c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92d509df75d96ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:51.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f51737fe640 1 -- 172.21.15.133:0/4067850301 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0ab650 con 0x7f51740d0770 2024-04-25T10:16:51.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f51737fe640 1 -- 172.21.15.133:0/4067850301 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c0b0070 con 0x7f51740d0770 2024-04-25T10:16:51.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f51737fe640 1 -- 172.21.15.133:0/4067850301 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0aba10 con 0x7f51740d0770 2024-04-25T10:16:51.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 -- 172.21.15.133:0/4067850301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51740d0770 msgr2=0x7f51740d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 --2- 172.21.15.133:0/4067850301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51740d0770 0x7f51740d0bd0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f51740b5a00 tx=0x7f515c01c450 comp rx=0 tx=0).stop 2024-04-25T10:16:51.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 -- 172.21.15.133:0/4067850301 shutdown_connections 2024-04-25T10:16:51.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 --2- 172.21.15.133:0/4067850301 >> v2:172.21.15.162:3300/0 conn(0x7f51740b87e0 0x7f51740b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 --2- 172.21.15.133:0/4067850301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51740d0770 0x7f51740d0bd0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 -- 172.21.15.133:0/4067850301 >> 172.21.15.133:0/4067850301 conn(0x7f51740727c0 msgr2=0x7f5174072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:51.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 -- 172.21.15.133:0/4067850301 shutdown_connections 2024-04-25T10:16:51.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 -- 172.21.15.133:0/4067850301 wait complete. 2024-04-25T10:16:51.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 Processor -- start 2024-04-25T10:16:51.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 -- start start 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51740b87e0 0x7f5174155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51740d0770 0x7f517415bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f517415db00 0x7f517415c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51740b7430 con 0x7f51740b87e0 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51740b82c0 con 0x7f51740d0770 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51740d18a0 con 0x7f517415db00 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f5173fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51740b87e0 0x7f5174155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f5173fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51740b87e0 0x7f5174155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41586/0 (socket says 172.21.15.133:41586) 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f5173fff640 1 -- 172.21.15.133:0/1453468856 learned_addr learned my addr 172.21.15.133:0/1453468856 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517886c640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f517415db00 0x7f517415c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f5173fff640 1 -- 172.21.15.133:0/1453468856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f517415db00 msgr2=0x7f517415c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f516bfff640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51740d0770 0x7f517415bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f5173fff640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f517415db00 0x7f517415c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f5173fff640 1 -- 172.21.15.133:0/1453468856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51740d0770 msgr2=0x7f517415bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f5173fff640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51740d0770 0x7f517415bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f5173fff640 1 -- 172.21.15.133:0/1453468856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517415ab30 con 0x7f51740b87e0 2024-04-25T10:16:51.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517886c640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f517415db00 0x7f517415c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:51.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f5173fff640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51740b87e0 0x7f5174155240 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f51640011f0 tx=0x7f516400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:51.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f51717fa640 1 -- 172.21.15.133:0/1453468856 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164018070 con 0x7f51740b87e0 2024-04-25T10:16:51.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f51717fa640 1 -- 172.21.15.133:0/1453468856 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51640074e0 con 0x7f51740b87e0 2024-04-25T10:16:51.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 -- 172.21.15.133:0/1453468856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517415ae10 con 0x7f51740b87e0 2024-04-25T10:16:51.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f51717fa640 1 -- 172.21.15.133:0/1453468856 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164004a10 con 0x7f51740b87e0 2024-04-25T10:16:51.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.824+0000 7f517a2cf640 1 -- 172.21.15.133:0/1453468856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517415b3d0 con 0x7f51740b87e0 2024-04-25T10:16:51.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.828+0000 7f517a2cf640 1 -- 172.21.15.133:0/1453468856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5174068ef0 con 0x7f51740b87e0 2024-04-25T10:16:51.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.828+0000 7f51717fa640 1 -- 172.21.15.133:0/1453468856 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f516401b070 con 0x7f51740b87e0 2024-04-25T10:16:51.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.828+0000 7f51717fa640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5154066f60 0x7f5154069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:51.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.828+0000 7f51717fa640 1 -- 172.21.15.133:0/1453468856 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f5164084650 con 0x7f51740b87e0 2024-04-25T10:16:51.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.828+0000 7f516bfff640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5154066f60 0x7f5154069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:51.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.828+0000 7f516bfff640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5154066f60 0x7f5154069420 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f515c0af000 tx=0x7f515c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:51.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.832+0000 7f51717fa640 1 -- 172.21.15.133:0/1453468856 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51640553f0 con 0x7f51740b87e0 2024-04-25T10:16:51.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.980+0000 7f517a2cf640 1 -- 172.21.15.133:0/1453468856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51740b2750 con 0x7f51740b87e0 2024-04-25T10:16:51.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f51717fa640 1 -- 172.21.15.133:0/1453468856 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+17052 (secure 0 0 0) 0x7f5164054cf0 con 0x7f51740b87e0 2024-04-25T10:16:51.985 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:51.986 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":171,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:16:35.087106+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:07.256836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:07.975826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2","mappings":[{"from":1,"to":0}]},{"pgid":"26.8","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:51.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 -- 172.21.15.133:0/1453468856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5154066f60 msgr2=0x7f5154069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5154066f60 0x7f5154069420 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f515c0af000 tx=0x7f515c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:16:51.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 -- 172.21.15.133:0/1453468856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51740b87e0 msgr2=0x7f5174155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:51.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51740b87e0 0x7f5174155240 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f51640011f0 tx=0x7f516400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 -- 172.21.15.133:0/1453468856 shutdown_connections 2024-04-25T10:16:51.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f517415db00 0x7f517415c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f51740d0770 0x7f517415bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5154066f60 0x7f5154069420 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 --2- 172.21.15.133:0/1453468856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51740b87e0 0x7f5174155240 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:51.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 -- 172.21.15.133:0/1453468856 >> 172.21.15.133:0/1453468856 conn(0x7f51740727c0 msgr2=0x7f51740ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:51.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 -- 172.21.15.133:0/1453468856 shutdown_connections 2024-04-25T10:16:51.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:51.984+0000 7f517a2cf640 1 -- 172.21.15.133:0/1453468856 wait complete. 2024-04-25T10:16:52.003 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:07.256836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:07.975826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:52.004 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:52.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.120+0000 7fa7ee0e2640 1 Processor -- start 2024-04-25T10:16:52.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.120+0000 7fa7ee0e2640 1 -- start start 2024-04-25T10:16:52.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7ee0e2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 0x7fa7e806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7ee0e2640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa7e806e4d0 0x7fa7e80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7ee0e2640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e80ce2e0 con 0x7fa7e80cdd60 2024-04-25T10:16:52.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7ee0e2640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e806db00 con 0x7fa7e806e4d0 2024-04-25T10:16:52.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7ee0e2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e8145550 con 0x7fa7e8145170 2024-04-25T10:16:52.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7e77fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 0x7fa7e806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7e6ffd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa7e806e4d0 0x7fa7e80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7e77fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 0x7fa7e806df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:35844/0 (socket says 172.21.15.133:35844) 2024-04-25T10:16:52.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7e77fe640 1 -- 172.21.15.133:0/1570866183 learned_addr learned my addr 172.21.15.133:0/1570866183 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:52.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7e77fe640 1 -- 172.21.15.133:0/1570866183 >> v2:172.21.15.162:3300/0 conn(0x7fa7e806e4d0 msgr2=0x7fa7e80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:52.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7e77fe640 1 --2- 172.21.15.133:0/1570866183 >> v2:172.21.15.162:3300/0 conn(0x7fa7e806e4d0 0x7fa7e80cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7e77fe640 1 -- 172.21.15.133:0/1570866183 >> v1:172.21.15.162:6789/0 conn(0x7fa7e80cdd60 legacy=0x7fa7e815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.124+0000 7fa7e77fe640 1 -- 172.21.15.133:0/1570866183 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e806d4b0 con 0x7fa7e8145170 2024-04-25T10:16:52.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e77fe640 1 --2- 172.21.15.133:0/1570866183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 0x7fa7e806df90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009870 tx=0x7fa7d001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2444f44744ef9a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e67fc640 1 -- 172.21.15.133:0/1570866183 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00b2070 con 0x7fa7e8145170 2024-04-25T10:16:52.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e67fc640 1 -- 172.21.15.133:0/1570866183 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d00a9070 con 0x7fa7e8145170 2024-04-25T10:16:52.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e67fc640 1 -- 172.21.15.133:0/1570866183 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00ad880 con 0x7fa7e8145170 2024-04-25T10:16:52.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/1570866183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 msgr2=0x7fa7e806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 --2- 172.21.15.133:0/1570866183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 0x7fa7e806df90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009870 tx=0x7fa7d001c560 comp rx=0 tx=0).stop 2024-04-25T10:16:52.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/1570866183 shutdown_connections 2024-04-25T10:16:52.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 --2- 172.21.15.133:0/1570866183 >> v2:172.21.15.162:3300/0 conn(0x7fa7e806e4d0 0x7fa7e80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 --2- 172.21.15.133:0/1570866183 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 0x7fa7e806df90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/1570866183 >> 172.21.15.133:0/1570866183 conn(0x7fa7e8066f70 msgr2=0x7fa7e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:52.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/1570866183 shutdown_connections 2024-04-25T10:16:52.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/1570866183 wait complete. 2024-04-25T10:16:52.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 Processor -- start 2024-04-25T10:16:52.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 -- start start 2024-04-25T10:16:52.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e806e4d0 0x7fa7e8139500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e80cdd60 0x7fa7e813e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 0x7fa7e813e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e8168490 con 0x7fa7e806e4d0 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e815d7f0 con 0x7fa7e80cdd60 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7ee0e2640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e8168610 con 0x7fa7e8145170 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e806e4d0 0x7fa7e8139500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e806e4d0 0x7fa7e8139500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41598/0 (socket says 172.21.15.133:41598) 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e77fe640 1 -- 172.21.15.133:0/2285311432 learned_addr learned my addr 172.21.15.133:0/2285311432 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e77fe640 1 -- 172.21.15.133:0/2285311432 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 msgr2=0x7fa7e813e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e7fff640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 0x7fa7e813e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e6ffd640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e80cdd60 0x7fa7e813e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e77fe640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 0x7fa7e813e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e77fe640 1 -- 172.21.15.133:0/2285311432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e80cdd60 msgr2=0x7fa7e813e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e77fe640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e80cdd60 0x7fa7e813e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e77fe640 1 -- 172.21.15.133:0/2285311432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e8142530 con 0x7fa7e806e4d0 2024-04-25T10:16:52.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7e77fe640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e806e4d0 0x7fa7e8139500 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00b0040 tx=0x7fa7d001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.128+0000 7fa7dffff640 1 -- 172.21.15.133:0/2285311432 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00b2040 con 0x7fa7e806e4d0 2024-04-25T10:16:52.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.132+0000 7fa7dffff640 1 -- 172.21.15.133:0/2285311432 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d00be070 con 0x7fa7e806e4d0 2024-04-25T10:16:52.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.132+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/2285311432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e8142810 con 0x7fa7e806e4d0 2024-04-25T10:16:52.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.132+0000 7fa7dffff640 1 -- 172.21.15.133:0/2285311432 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00b8420 con 0x7fa7e806e4d0 2024-04-25T10:16:52.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.132+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/2285311432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e81432f0 con 0x7fa7e806e4d0 2024-04-25T10:16:52.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.132+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/2285311432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e8068e80 con 0x7fa7e806e4d0 2024-04-25T10:16:52.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.132+0000 7fa7dffff640 1 -- 172.21.15.133:0/2285311432 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7d00ab070 con 0x7fa7e806e4d0 2024-04-25T10:16:52.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.132+0000 7fa7dffff640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7c8066f60 0x7fa7c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.132+0000 7fa7dffff640 1 -- 172.21.15.133:0/2285311432 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fa7d011f1f0 con 0x7fa7e806e4d0 2024-04-25T10:16:52.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.132+0000 7fa7e6ffd640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7c8066f60 0x7fa7c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.132+0000 7fa7e6ffd640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7c8066f60 0x7fa7c8069420 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80b2cd0 tx=0x7fa7d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.136+0000 7fa7dffff640 1 -- 172.21.15.133:0/2285311432 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7d0016150 con 0x7fa7e806e4d0 2024-04-25T10:16:52.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/2285311432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7e8141e10 con 0x7fa7e806e4d0 2024-04-25T10:16:52.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7dffff640 1 -- 172.21.15.133:0/2285311432 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v171) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7d00f0020 con 0x7fa7e806e4d0 2024-04-25T10:16:52.305 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:16:52.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/2285311432 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7c8066f60 msgr2=0x7fa7c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7ee0e2640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7c8066f60 0x7fa7c8069420 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80b2cd0 tx=0x7fa7d8008040 comp rx=0 tx=0).stop 2024-04-25T10:16:52.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/2285311432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e806e4d0 msgr2=0x7fa7e8139500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7ee0e2640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e806e4d0 0x7fa7e8139500 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00b0040 tx=0x7fa7d001ada0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/2285311432 shutdown_connections 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7ee0e2640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa7e8145170 0x7fa7e813e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7ee0e2640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa7e80cdd60 0x7fa7e813e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7ee0e2640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa7c8066f60 0x7fa7c8069420 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7ee0e2640 1 --2- 172.21.15.133:0/2285311432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e806e4d0 0x7fa7e8139500 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.304+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/2285311432 >> 172.21.15.133:0/2285311432 conn(0x7fa7e8066f70 msgr2=0x7fa7e80b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.308+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/2285311432 shutdown_connections 2024-04-25T10:16:52.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.308+0000 7fa7ee0e2640 1 -- 172.21.15.133:0/2285311432 wait complete. 2024-04-25T10:16:52.325 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:52.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c52f16640 1 Processor -- start 2024-04-25T10:16:52.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c52f16640 1 -- start start 2024-04-25T10:16:52.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c52f16640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4c15eb40 0x7f8c4c0b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c52f16640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c4c0aebc0 0x7f8c4c0af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c52f16640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4c1471f0 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c52f16640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4c147490 con 0x7f8c4c0aebc0 2024-04-25T10:16:52.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c52f16640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0b19c0 con 0x7f8c4c15eb40 2024-04-25T10:16:52.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c514b3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c4c0aebc0 0x7f8c4c0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c514b3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c4c0aebc0 0x7f8c4c0af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41600/0 (socket says 172.21.15.133:41600) 2024-04-25T10:16:52.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c514b3640 1 -- 172.21.15.133:0/949236010 learned_addr learned my addr 172.21.15.133:0/949236010 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:52.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c4bfff640 1 --2- 172.21.15.133:0/949236010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4c15eb40 0x7f8c4c0b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c514b3640 1 -- 172.21.15.133:0/949236010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4c15eb40 msgr2=0x7f8c4c0b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:52.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c514b3640 1 --2- 172.21.15.133:0/949236010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4c15eb40 0x7f8c4c0b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c514b3640 1 -- 172.21.15.133:0/949236010 >> v1:172.21.15.162:6789/0 conn(0x7f8c4c06e8b0 legacy=0x7f8c4c146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:52.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c514b3640 1 -- 172.21.15.133:0/949236010 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c0b1090 con 0x7f8c4c0aebc0 2024-04-25T10:16:52.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.444+0000 7f8c4bfff640 1 --2- 172.21.15.133:0/949236010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4c15eb40 0x7f8c4c0b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:52.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c514b3640 1 --2- 172.21.15.133:0/949236010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4c0aebc0 0x7f8c4c0af050 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009e80 tx=0x7f8c4401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e77b022adbc4b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c4b7fe640 1 -- 172.21.15.133:0/949236010 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c440a55c0 con 0x7f8c4c0aebc0 2024-04-25T10:16:52.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c4b7fe640 1 -- 172.21.15.133:0/949236010 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c440a5760 con 0x7f8c4c0aebc0 2024-04-25T10:16:52.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c4b7fe640 1 -- 172.21.15.133:0/949236010 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c440a5b30 con 0x7f8c4c0aebc0 2024-04-25T10:16:52.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 -- 172.21.15.133:0/949236010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4c0aebc0 msgr2=0x7f8c4c0af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 --2- 172.21.15.133:0/949236010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4c0aebc0 0x7f8c4c0af050 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009e80 tx=0x7f8c4401c940 comp rx=0 tx=0).stop 2024-04-25T10:16:52.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 -- 172.21.15.133:0/949236010 shutdown_connections 2024-04-25T10:16:52.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 --2- 172.21.15.133:0/949236010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4c0aebc0 0x7f8c4c0af050 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 --2- 172.21.15.133:0/949236010 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4c15eb40 0x7f8c4c0b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 -- 172.21.15.133:0/949236010 >> 172.21.15.133:0/949236010 conn(0x7f8c4c066f70 msgr2=0x7f8c4c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:52.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 -- 172.21.15.133:0/949236010 shutdown_connections 2024-04-25T10:16:52.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 -- 172.21.15.133:0/949236010 wait complete. 2024-04-25T10:16:52.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 Processor -- start 2024-04-25T10:16:52.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 -- start start 2024-04-25T10:16:52.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4c06e8b0 0x7f8c4c144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4c0aebc0 0x7f8c4c0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c4c15eb40 0x7f8c4c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c06dcc0 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c15ef20 con 0x7f8c4c15eb40 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c52f16640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c06de40 con 0x7f8c4c0aebc0 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c50cb2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4c06e8b0 0x7f8c4c144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c50cb2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4c06e8b0 0x7f8c4c144bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41608/0 (socket says 172.21.15.133:41608) 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c50cb2640 1 -- 172.21.15.133:0/3477043528 learned_addr learned my addr 172.21.15.133:0/3477043528 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c514b3640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c4c15eb40 0x7f8c4c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c4bfff640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4c0aebc0 0x7f8c4c0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c50cb2640 1 -- 172.21.15.133:0/3477043528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4c0aebc0 msgr2=0x7f8c4c0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c50cb2640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4c0aebc0 0x7f8c4c0b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c50cb2640 1 -- 172.21.15.133:0/3477043528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c4c15eb40 msgr2=0x7f8c4c0b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c50cb2640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c4c15eb40 0x7f8c4c0b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.448+0000 7f8c50cb2640 1 -- 172.21.15.133:0/3477043528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c136cc0 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c50cb2640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4c06e8b0 0x7f8c4c144bc0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00c480 tx=0x7f8c3c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c497fa640 1 -- 172.21.15.133:0/3477043528 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c012830 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c497fa640 1 -- 172.21.15.133:0/3477043528 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c017070 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c52f16640 1 -- 172.21.15.133:0/3477043528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c136fa0 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c497fa640 1 -- 172.21.15.133:0/3477043528 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c012c30 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c52f16640 1 -- 172.21.15.133:0/3477043528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c1443c0 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c52f16640 1 -- 172.21.15.133:0/3477043528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4c068e80 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c497fa640 1 -- 172.21.15.133:0/3477043528 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c3c01c460 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c497fa640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c38066f60 0x7f8c38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c4bfff640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c38066f60 0x7f8c38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c497fa640 1 -- 172.21.15.133:0/3477043528 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f8c3c085a20 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.452+0000 7f8c4bfff640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c38066f60 0x7f8c38069420 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8c40006fd0 tx=0x7f8c40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.456+0000 7f8c497fa640 1 -- 172.21.15.133:0/3477043528 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c3c0569c0 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 -- 172.21.15.133:0/3477043528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c4c13f100 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c497fa640 1 -- 172.21.15.133:0/3477043528 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v171) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c3c0562c0 con 0x7f8c4c06e8b0 2024-04-25T10:16:52.614 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:16:52.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 -- 172.21.15.133:0/3477043528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c38066f60 msgr2=0x7f8c38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c38066f60 0x7f8c38069420 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8c40006fd0 tx=0x7f8c40008040 comp rx=0 tx=0).stop 2024-04-25T10:16:52.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 -- 172.21.15.133:0/3477043528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4c06e8b0 msgr2=0x7f8c4c144bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4c06e8b0 0x7f8c4c144bc0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00c480 tx=0x7f8c3c00c940 comp rx=0 tx=0).stop 2024-04-25T10:16:52.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 -- 172.21.15.133:0/3477043528 shutdown_connections 2024-04-25T10:16:52.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8c4c15eb40 0x7f8c4c0b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f8c4c0aebc0 0x7f8c4c0b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8c38066f60 0x7f8c38069420 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 --2- 172.21.15.133:0/3477043528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c4c06e8b0 0x7f8c4c144bc0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 -- 172.21.15.133:0/3477043528 >> 172.21.15.133:0/3477043528 conn(0x7f8c4c066f70 msgr2=0x7f8c4c15f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:52.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 -- 172.21.15.133:0/3477043528 shutdown_connections 2024-04-25T10:16:52.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.612+0000 7f8c52f16640 1 -- 172.21.15.133:0/3477043528 wait complete. 2024-04-25T10:16:52.632 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:52.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f365e1eb640 1 Processor -- start 2024-04-25T10:16:52.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f365e1eb640 1 -- start start 2024-04-25T10:16:52.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f365e1eb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3658143ea0 0x7f36580cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f365e1eb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f365806eac0 0x7f36580b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f365e1eb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365815ac50 con 0x7f36580b2150 2024-04-25T10:16:52.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f365e1eb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36580cd180 con 0x7f365806eac0 2024-04-25T10:16:52.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f365e1eb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365815aed0 con 0x7f3658143ea0 2024-04-25T10:16:52.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f3657fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f365806eac0 0x7f36580b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f3657fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f365806eac0 0x7f36580b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41622/0 (socket says 172.21.15.133:41622) 2024-04-25T10:16:52.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f3657fff640 1 -- 172.21.15.133:0/1839441454 learned_addr learned my addr 172.21.15.133:0/1839441454 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:52.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f36577fe640 1 --2- 172.21.15.133:0/1839441454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3658143ea0 0x7f36580cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f3657fff640 1 -- 172.21.15.133:0/1839441454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3658143ea0 msgr2=0x7f36580cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:52.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f3657fff640 1 --2- 172.21.15.133:0/1839441454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3658143ea0 0x7f36580cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f3657fff640 1 -- 172.21.15.133:0/1839441454 >> v1:172.21.15.162:6789/0 conn(0x7f36580b2150 legacy=0x7f365806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:52.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f3657fff640 1 -- 172.21.15.133:0/1839441454 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365815b1b0 con 0x7f365806eac0 2024-04-25T10:16:52.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.744+0000 7f36577fe640 1 --2- 172.21.15.133:0/1839441454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3658143ea0 0x7f36580cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:52.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f3657fff640 1 --2- 172.21.15.133:0/1839441454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365806eac0 0x7f36580b06d0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f3650009e50 tx=0x7f365001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639f624b5b60791c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f3656ffd640 1 -- 172.21.15.133:0/1839441454 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36500a55a0 con 0x7f365806eac0 2024-04-25T10:16:52.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f3656ffd640 1 -- 172.21.15.133:0/1839441454 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36500a5740 con 0x7f365806eac0 2024-04-25T10:16:52.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f3656ffd640 1 -- 172.21.15.133:0/1839441454 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36500a5b10 con 0x7f365806eac0 2024-04-25T10:16:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f365e1eb640 1 -- 172.21.15.133:0/1839441454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365806eac0 msgr2=0x7f36580b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f365e1eb640 1 --2- 172.21.15.133:0/1839441454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365806eac0 0x7f36580b06d0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f3650009e50 tx=0x7f365001c810 comp rx=0 tx=0).stop 2024-04-25T10:16:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f365e1eb640 1 -- 172.21.15.133:0/1839441454 shutdown_connections 2024-04-25T10:16:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f365e1eb640 1 --2- 172.21.15.133:0/1839441454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365806eac0 0x7f36580b06d0 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f365e1eb640 1 --2- 172.21.15.133:0/1839441454 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3658143ea0 0x7f36580cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f365e1eb640 1 -- 172.21.15.133:0/1839441454 >> 172.21.15.133:0/1839441454 conn(0x7f3658066f70 msgr2=0x7f365806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f365e1eb640 1 -- 172.21.15.133:0/1839441454 shutdown_connections 2024-04-25T10:16:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f365e1eb640 1 -- 172.21.15.133:0/1839441454 wait complete. 2024-04-25T10:16:52.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f365e1eb640 1 Processor -- start 2024-04-25T10:16:52.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.748+0000 7f365e1eb640 1 -- start start 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f365e1eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365806eac0 0x7f3658163f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f365e1eb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36580b2150 0x7f3658141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f365e1eb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3658143ea0 0x7f3658162490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f365e1eb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3658167550 con 0x7f365806eac0 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f365e1eb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36581673d0 con 0x7f3658143ea0 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f365e1eb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36581676d0 con 0x7f36580b2150 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f36577fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365806eac0 0x7f3658163f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f36577fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365806eac0 0x7f3658163f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41636/0 (socket says 172.21.15.133:41636) 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f36577fe640 1 -- 172.21.15.133:0/3505553008 learned_addr learned my addr 172.21.15.133:0/3505553008 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f3657fff640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3658143ea0 0x7f3658162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f36577fe640 1 -- 172.21.15.133:0/3505553008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36580b2150 msgr2=0x7f3658141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f36577fe640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36580b2150 0x7f3658141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f36577fe640 1 -- 172.21.15.133:0/3505553008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3658143ea0 msgr2=0x7f3658162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f36577fe640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3658143ea0 0x7f3658162490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f36577fe640 1 -- 172.21.15.133:0/3505553008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658162ee0 con 0x7f365806eac0 2024-04-25T10:16:52.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f36577fe640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365806eac0 0x7f3658163f80 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f364800ed70 tx=0x7f3648007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f3654ff9640 1 -- 172.21.15.133:0/3505553008 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364800ce10 con 0x7f365806eac0 2024-04-25T10:16:52.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f3654ff9640 1 -- 172.21.15.133:0/3505553008 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3648010440 con 0x7f365806eac0 2024-04-25T10:16:52.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f3654ff9640 1 -- 172.21.15.133:0/3505553008 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648010830 con 0x7f365806eac0 2024-04-25T10:16:52.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f365e1eb640 1 -- 172.21.15.133:0/3505553008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36581631c0 con 0x7f365806eac0 2024-04-25T10:16:52.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f365e1eb640 1 -- 172.21.15.133:0/3505553008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3658163780 con 0x7f365806eac0 2024-04-25T10:16:52.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f365e1eb640 1 -- 172.21.15.133:0/3505553008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3658068e80 con 0x7f365806eac0 2024-04-25T10:16:52.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f3654ff9640 1 -- 172.21.15.133:0/3505553008 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f364800b030 con 0x7f365806eac0 2024-04-25T10:16:52.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f3654ff9640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3638066f60 0x7f3638069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:52.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f364ffff640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3638066f60 0x7f3638069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:52.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f3654ff9640 1 -- 172.21.15.133:0/3505553008 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f3648087960 con 0x7f365806eac0 2024-04-25T10:16:52.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.752+0000 7f364ffff640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3638066f60 0x7f3638069420 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f365815e680 tx=0x7f36400079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:52.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.756+0000 7f3654ff9640 1 -- 172.21.15.133:0/3505553008 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3648058820 con 0x7f365806eac0 2024-04-25T10:16:52.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.904+0000 7f365e1eb640 1 -- 172.21.15.133:0/3505553008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36580cebd0 con 0x7f365806eac0 2024-04-25T10:16:52.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.904+0000 7f3654ff9640 1 -- 172.21.15.133:0/3505553008 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v171) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3648058120 con 0x7f365806eac0 2024-04-25T10:16:52.907 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:16:52.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 -- 172.21.15.133:0/3505553008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3638066f60 msgr2=0x7f3638069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3638066f60 0x7f3638069420 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f365815e680 tx=0x7f36400079c0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 -- 172.21.15.133:0/3505553008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365806eac0 msgr2=0x7f3658163f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:52.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365806eac0 0x7f3658163f80 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f364800ed70 tx=0x7f3648007410 comp rx=0 tx=0).stop 2024-04-25T10:16:52.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 -- 172.21.15.133:0/3505553008 shutdown_connections 2024-04-25T10:16:52.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3658143ea0 0x7f3658162490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f36580b2150 0x7f3658141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3638066f60 0x7f3638069420 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 --2- 172.21.15.133:0/3505553008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f365806eac0 0x7f3658163f80 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 -- 172.21.15.133:0/3505553008 >> 172.21.15.133:0/3505553008 conn(0x7f3658066f70 msgr2=0x7f365815d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 -- 172.21.15.133:0/3505553008 shutdown_connections 2024-04-25T10:16:52.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:52.908+0000 7f365e1eb640 1 -- 172.21.15.133:0/3505553008 wait complete. 2024-04-25T10:16:52.925 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.028+0000 7fa781467640 1 Processor -- start 2024-04-25T10:16:53.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.028+0000 7fa781467640 1 -- start start 2024-04-25T10:16:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.028+0000 7fa781467640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fa77c15d9b0 0x7fa77c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.028+0000 7fa781467640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa77c0cee80 0x7fa77c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.028+0000 7fa781467640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0ca9b0 con 0x7fa77c15dd90 2024-04-25T10:16:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.028+0000 7fa781467640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0cbe30 con 0x7fa77c0cee80 2024-04-25T10:16:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.028+0000 7fa781467640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0cac90 con 0x7fa77c15d9b0 2024-04-25T10:16:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa77b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa77c0cee80 0x7fa77c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa77b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa77c0cee80 0x7fa77c06d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41650/0 (socket says 172.21.15.133:41650) 2024-04-25T10:16:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa77a7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa77c15dd90 0x7fa77c0ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53636/0 (socket says 172.21.15.133:53636) 2024-04-25T10:16:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa77b7fe640 1 -- 172.21.15.133:0/3494829047 learned_addr learned my addr 172.21.15.133:0/3494829047 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa77b7fe640 1 -- 172.21.15.133:0/3494829047 >> v2:172.21.15.162:3300/0 conn(0x7fa77c15d9b0 msgr2=0x7fa77c0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa77b7fe640 1 --2- 172.21.15.133:0/3494829047 >> v2:172.21.15.162:3300/0 conn(0x7fa77c15d9b0 0x7fa77c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa77b7fe640 1 -- 172.21.15.133:0/3494829047 >> v1:172.21.15.133:6789/0 conn(0x7fa77c15dd90 legacy=0x7fa77c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa77b7fe640 1 -- 172.21.15.133:0/3494829047 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c0caf70 con 0x7fa77c0cee80 2024-04-25T10:16:53.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa779ffb640 1 -- 172.21.15.133:0/3494829047 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835578083 0 0) 0x7fa77c0ca9b0 con 0x7fa77c15dd90 2024-04-25T10:16:53.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa77b7fe640 1 --2- 172.21.15.133:0/3494829047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa77c0cee80 0x7fa77c06d6f0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0b4c70 tx=0x7fa77401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb74a622c0e5652 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa779ffb640 1 -- 172.21.15.133:0/3494829047 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7740a57e0 con 0x7fa77c0cee80 2024-04-25T10:16:53.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa779ffb640 1 -- 172.21.15.133:0/3494829047 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7740a5980 con 0x7fa77c0cee80 2024-04-25T10:16:53.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa779ffb640 1 -- 172.21.15.133:0/3494829047 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7740a5d50 con 0x7fa77c0cee80 2024-04-25T10:16:53.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa781467640 1 -- 172.21.15.133:0/3494829047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa77c0cee80 msgr2=0x7fa77c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa781467640 1 --2- 172.21.15.133:0/3494829047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa77c0cee80 0x7fa77c06d6f0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0b4c70 tx=0x7fa77401c900 comp rx=0 tx=0).stop 2024-04-25T10:16:53.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa781467640 1 -- 172.21.15.133:0/3494829047 shutdown_connections 2024-04-25T10:16:53.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa781467640 1 --2- 172.21.15.133:0/3494829047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa77c0cee80 0x7fa77c06d6f0 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa781467640 1 --2- 172.21.15.133:0/3494829047 >> v2:172.21.15.162:3300/0 conn(0x7fa77c15d9b0 0x7fa77c0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.032+0000 7fa781467640 1 -- 172.21.15.133:0/3494829047 >> 172.21.15.133:0/3494829047 conn(0x7fa77c067000 msgr2=0x7fa77c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 -- 172.21.15.133:0/3494829047 shutdown_connections 2024-04-25T10:16:53.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 -- 172.21.15.133:0/3494829047 wait complete. 2024-04-25T10:16:53.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 Processor -- start 2024-04-25T10:16:53.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 -- start start 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa77c0cee80 0x7fa77c158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa77c15d9b0 0x7fa77c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa77c15dd90 0x7fa77c152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c06e7d0 con 0x7fa77c0cee80 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c06e650 con 0x7fa77c15d9b0 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c06e950 con 0x7fa77c15dd90 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa77c0cee80 0x7fa77c158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa77c0cee80 0x7fa77c158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41662/0 (socket says 172.21.15.133:41662) 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77affd640 1 -- 172.21.15.133:0/2212237471 learned_addr learned my addr 172.21.15.133:0/2212237471 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77b7fe640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa77c15dd90 0x7fa77c152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77a7fc640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa77c15d9b0 0x7fa77c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77affd640 1 -- 172.21.15.133:0/2212237471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa77c15dd90 msgr2=0x7fa77c152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77affd640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa77c15dd90 0x7fa77c152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77affd640 1 -- 172.21.15.133:0/2212237471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa77c15d9b0 msgr2=0x7fa77c151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77affd640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa77c15d9b0 0x7fa77c151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77affd640 1 -- 172.21.15.133:0/2212237471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c1f87a0 con 0x7fa77c0cee80 2024-04-25T10:16:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77a7fc640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa77c15d9b0 0x7fa77c151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa77affd640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa77c0cee80 0x7fa77c158f30 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00b550 tx=0x7fa76c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa7637fe640 1 -- 172.21.15.133:0/2212237471 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c014070 con 0x7fa77c0cee80 2024-04-25T10:16:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa7637fe640 1 -- 172.21.15.133:0/2212237471 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c010890 con 0x7fa77c0cee80 2024-04-25T10:16:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 -- 172.21.15.133:0/2212237471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c1f8a80 con 0x7fa77c0cee80 2024-04-25T10:16:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa7637fe640 1 -- 172.21.15.133:0/2212237471 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c010c80 con 0x7fa77c0cee80 2024-04-25T10:16:53.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.036+0000 7fa781467640 1 -- 172.21.15.133:0/2212237471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77c1f9040 con 0x7fa77c0cee80 2024-04-25T10:16:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.040+0000 7fa781467640 1 -- 172.21.15.133:0/2212237471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77c068ef0 con 0x7fa77c0cee80 2024-04-25T10:16:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.040+0000 7fa7637fe640 1 -- 172.21.15.133:0/2212237471 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa76c017070 con 0x7fa77c0cee80 2024-04-25T10:16:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.040+0000 7fa7637fe640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa768066f60 0x7fa768069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.040+0000 7fa7637fe640 1 -- 172.21.15.133:0/2212237471 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fa76c084fe0 con 0x7fa77c0cee80 2024-04-25T10:16:53.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.040+0000 7fa77a7fc640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa768066f60 0x7fa768069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.040+0000 7fa77a7fc640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa768066f60 0x7fa768069420 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa77c15d3d0 tx=0x7fa770009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.044+0000 7fa7637fe640 1 -- 172.21.15.133:0/2212237471 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa76c055e10 con 0x7fa77c0cee80 2024-04-25T10:16:53.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.232+0000 7fa781467640 1 -- 172.21.15.133:0/2212237471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa77c0ccf50 con 0x7fa77c0cee80 2024-04-25T10:16:53.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.232+0000 7fa7637fe640 1 -- 172.21.15.133:0/2212237471 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4962 (secure 0 0 0) 0x7fa76c010e20 con 0x7fa77c0cee80 2024-04-25T10:16:53.236 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:53.236 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1498957356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1498957356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1498957356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/883424012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":883424012},{"type":"v1","addr":"172.21.15.133:6835","nonce":883424012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2040533186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2040533186},{"type":"v1","addr":"172.21.15.133:6837","nonce":2040533186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:08.983208+0000","modified":"2024-04-25T10:16:11.017848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11037":{"gid":11037,"name":"d","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/2658062772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2658062772},{"type":"v1","addr":"172.21.15.162:6833","nonce":2658062772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:16:53.236 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 105 2024-04-25T10:16:53.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 -- 172.21.15.133:0/2212237471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa768066f60 msgr2=0x7fa768069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa768066f60 0x7fa768069420 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa77c15d3d0 tx=0x7fa770009040 comp rx=0 tx=0).stop 2024-04-25T10:16:53.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 -- 172.21.15.133:0/2212237471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa77c0cee80 msgr2=0x7fa77c158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa77c0cee80 0x7fa77c158f30 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00b550 tx=0x7fa76c00ba10 comp rx=0 tx=0).stop 2024-04-25T10:16:53.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 -- 172.21.15.133:0/2212237471 shutdown_connections 2024-04-25T10:16:53.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa77c15dd90 0x7fa77c152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa77c15d9b0 0x7fa77c151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa768066f60 0x7fa768069420 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 --2- 172.21.15.133:0/2212237471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa77c0cee80 0x7fa77c158f30 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 -- 172.21.15.133:0/2212237471 >> 172.21.15.133:0/2212237471 conn(0x7fa77c067000 msgr2=0x7fa77c0b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 -- 172.21.15.133:0/2212237471 shutdown_connections 2024-04-25T10:16:53.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.236+0000 7fa781467640 1 -- 172.21.15.133:0/2212237471 wait complete. 2024-04-25T10:16:53.254 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:53.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a40e0640 1 Processor -- start 2024-04-25T10:16:53.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a40e0640 1 -- start start 2024-04-25T10:16:53.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a40e0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f589c15c9a0 0x7f589c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a40e0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f589c0b1f70 0x7f589c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a40e0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589c0c61d0 con 0x7f589c0ce330 2024-04-25T10:16:53.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a40e0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589c0c64b0 con 0x7f589c0b1f70 2024-04-25T10:16:53.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a40e0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589c06d4f0 con 0x7f589c15c9a0 2024-04-25T10:16:53.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a167b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f589c0b1f70 0x7f589c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a167b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f589c0b1f70 0x7f589c06dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41664/0 (socket says 172.21.15.133:41664) 2024-04-25T10:16:53.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a167b640 1 -- 172.21.15.133:0/3757655290 learned_addr learned my addr 172.21.15.133:0/3757655290 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:53.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a1e7c640 1 --2- 172.21.15.133:0/3757655290 >> v2:172.21.15.162:3300/0 conn(0x7f589c15c9a0 0x7f589c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a167b640 1 -- 172.21.15.133:0/3757655290 >> v2:172.21.15.162:3300/0 conn(0x7f589c15c9a0 msgr2=0x7f589c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a167b640 1 --2- 172.21.15.133:0/3757655290 >> v2:172.21.15.162:3300/0 conn(0x7f589c15c9a0 0x7f589c0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a167b640 1 -- 172.21.15.133:0/3757655290 >> v1:172.21.15.162:6789/0 conn(0x7f589c0ce330 legacy=0x7f589c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a167b640 1 -- 172.21.15.133:0/3757655290 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c06d7d0 con 0x7f589c0b1f70 2024-04-25T10:16:53.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.360+0000 7f58a167b640 1 --2- 172.21.15.133:0/3757655290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f589c0b1f70 0x7f589c06dcf0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f589c06f5f0 tx=0x7f588c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41391c491427634 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a0e7a640 1 -- 172.21.15.133:0/3757655290 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c0aa600 con 0x7f589c0b1f70 2024-04-25T10:16:53.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a0e7a640 1 -- 172.21.15.133:0/3757655290 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c0af070 con 0x7f589c0b1f70 2024-04-25T10:16:53.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a0e7a640 1 -- 172.21.15.133:0/3757655290 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c0aa9c0 con 0x7f589c0b1f70 2024-04-25T10:16:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 -- 172.21.15.133:0/3757655290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f589c0b1f70 msgr2=0x7f589c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 --2- 172.21.15.133:0/3757655290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f589c0b1f70 0x7f589c06dcf0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f589c06f5f0 tx=0x7f588c01c480 comp rx=0 tx=0).stop 2024-04-25T10:16:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 -- 172.21.15.133:0/3757655290 shutdown_connections 2024-04-25T10:16:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 --2- 172.21.15.133:0/3757655290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f589c0b1f70 0x7f589c06dcf0 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 --2- 172.21.15.133:0/3757655290 >> v2:172.21.15.162:3300/0 conn(0x7f589c15c9a0 0x7f589c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 -- 172.21.15.133:0/3757655290 >> 172.21.15.133:0/3757655290 conn(0x7f589c067000 msgr2=0x7f589c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 -- 172.21.15.133:0/3757655290 shutdown_connections 2024-04-25T10:16:53.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 -- 172.21.15.133:0/3757655290 wait complete. 2024-04-25T10:16:53.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 Processor -- start 2024-04-25T10:16:53.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 -- start start 2024-04-25T10:16:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f589c0b1f70 0x7f589c146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c0ce330 0x7f589c14a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f589c15c9a0 0x7f589c146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c15cd80 con 0x7f589c0b1f70 2024-04-25T10:16:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c06e230 con 0x7f589c15c9a0 2024-04-25T10:16:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589c15cf00 con 0x7f589c0ce330 2024-04-25T10:16:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a1e7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f589c0b1f70 0x7f589c146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a1e7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f589c0b1f70 0x7f589c146ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41678/0 (socket says 172.21.15.133:41678) 2024-04-25T10:16:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a1e7c640 1 -- 172.21.15.133:0/3451686407 learned_addr learned my addr 172.21.15.133:0/3451686407 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a267d640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f589c15c9a0 0x7f589c146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a1e7c640 1 -- 172.21.15.133:0/3451686407 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c0ce330 msgr2=0x7f589c14a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a1e7c640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c0ce330 0x7f589c14a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a1e7c640 1 -- 172.21.15.133:0/3451686407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f589c15c9a0 msgr2=0x7f589c146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a1e7c640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f589c15c9a0 0x7f589c146700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a1e7c640 1 -- 172.21.15.133:0/3451686407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c145940 con 0x7f589c0b1f70 2024-04-25T10:16:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a1e7c640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f589c0b1f70 0x7f589c146ce0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f589800c7f0 tx=0x7f589800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58927fc640 1 -- 172.21.15.133:0/3451686407 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898015070 con 0x7f589c0b1f70 2024-04-25T10:16:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58927fc640 1 -- 172.21.15.133:0/3451686407 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5898004240 con 0x7f589c0b1f70 2024-04-25T10:16:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 -- 172.21.15.133:0/3451686407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589c145c20 con 0x7f589c0b1f70 2024-04-25T10:16:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58927fc640 1 -- 172.21.15.133:0/3451686407 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898003620 con 0x7f589c0b1f70 2024-04-25T10:16:53.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.364+0000 7f58a40e0640 1 -- 172.21.15.133:0/3451686407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589c1461e0 con 0x7f589c0b1f70 2024-04-25T10:16:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.368+0000 7f58a40e0640 1 -- 172.21.15.133:0/3451686407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f589c06bc20 con 0x7f589c0b1f70 2024-04-25T10:16:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.368+0000 7f58927fc640 1 -- 172.21.15.133:0/3451686407 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58980039d0 con 0x7f589c0b1f70 2024-04-25T10:16:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.368+0000 7f58927fc640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5880066f60 0x7f5880069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.368+0000 7f58927fc640 1 -- 172.21.15.133:0/3451686407 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f58980852d0 con 0x7f589c0b1f70 2024-04-25T10:16:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.368+0000 7f58a167b640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5880066f60 0x7f5880069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.368+0000 7f58a167b640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5880066f60 0x7f5880069420 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f588c0ae000 tx=0x7f588c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.372+0000 7f58927fc640 1 -- 172.21.15.133:0/3451686407 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5898056100 con 0x7f589c0b1f70 2024-04-25T10:16:53.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.520+0000 7f58a40e0640 1 -- 172.21.15.133:0/3451686407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f589c068ed0 con 0x7f589c0b1f70 2024-04-25T10:16:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58927fc640 1 -- 172.21.15.133:0/3451686407 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+17052 (secure 0 0 0) 0x7f5898055a00 con 0x7f589c0b1f70 2024-04-25T10:16:53.525 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:53.526 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":171,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:16:35.087106+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:07.256836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:07.975826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2","mappings":[{"from":1,"to":0}]},{"pgid":"26.8","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 -- 172.21.15.133:0/3451686407 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5880066f60 msgr2=0x7f5880069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5880066f60 0x7f5880069420 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f588c0ae000 tx=0x7f588c004740 comp rx=0 tx=0).stop 2024-04-25T10:16:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 -- 172.21.15.133:0/3451686407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f589c0b1f70 msgr2=0x7f589c146ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f589c0b1f70 0x7f589c146ce0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f589800c7f0 tx=0x7f589800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 -- 172.21.15.133:0/3451686407 shutdown_connections 2024-04-25T10:16:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f589c15c9a0 0x7f589c146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f589c0ce330 0x7f589c14a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5880066f60 0x7f5880069420 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 --2- 172.21.15.133:0/3451686407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f589c0b1f70 0x7f589c146ce0 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 -- 172.21.15.133:0/3451686407 >> 172.21.15.133:0/3451686407 conn(0x7f589c067000 msgr2=0x7f589c15e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 -- 172.21.15.133:0/3451686407 shutdown_connections 2024-04-25T10:16:53.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.524+0000 7f58a40e0640 1 -- 172.21.15.133:0/3451686407 wait complete. 2024-04-25T10:16:53.543 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:16:53.544 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:53.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.648+0000 7f5769e76640 1 Processor -- start 2024-04-25T10:16:53.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.648+0000 7f5769e76640 1 -- start start 2024-04-25T10:16:53.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.648+0000 7f5769e76640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5764158110 0x7f57640b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.648+0000 7f5769e76640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f576415ad10 0x7f576415ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.648+0000 7f5769e76640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57640cbe60 con 0x7f57641584f0 2024-04-25T10:16:53.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.648+0000 7f5769e76640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57640b19b0 con 0x7f576415ad10 2024-04-25T10:16:53.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.648+0000 7f5769e76640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576415b360 con 0x7f5764158110 2024-04-25T10:16:53.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5763fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f576415ad10 0x7f576415ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5763fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f576415ad10 0x7f576415ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41684/0 (socket says 172.21.15.133:41684) 2024-04-25T10:16:53.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5763fff640 1 -- 172.21.15.133:0/1618075897 learned_addr learned my addr 172.21.15.133:0/1618075897 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:53.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f57637fe640 1 --2- 172.21.15.133:0/1618075897 >> v2:172.21.15.162:3300/0 conn(0x7f5764158110 0x7f57640b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5763fff640 1 -- 172.21.15.133:0/1618075897 >> v2:172.21.15.162:3300/0 conn(0x7f5764158110 msgr2=0x7f57640b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5763fff640 1 --2- 172.21.15.133:0/1618075897 >> v2:172.21.15.162:3300/0 conn(0x7f5764158110 0x7f57640b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5763fff640 1 -- 172.21.15.133:0/1618075897 >> v1:172.21.15.162:6789/0 conn(0x7f57641584f0 legacy=0x7f576415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5763fff640 1 -- 172.21.15.133:0/1618075897 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576415b640 con 0x7f576415ad10 2024-04-25T10:16:53.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5763fff640 1 --2- 172.21.15.133:0/1618075897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576415ad10 0x7f576415ea10 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f57640b4d40 tx=0x7f575c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e22a955f2ae629c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5762ffd640 1 -- 172.21.15.133:0/1618075897 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0a5670 con 0x7f576415ad10 2024-04-25T10:16:53.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5762ffd640 1 -- 172.21.15.133:0/1618075897 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c0a5810 con 0x7f576415ad10 2024-04-25T10:16:53.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5762ffd640 1 -- 172.21.15.133:0/1618075897 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0a5be0 con 0x7f576415ad10 2024-04-25T10:16:53.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5769e76640 1 -- 172.21.15.133:0/1618075897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576415ad10 msgr2=0x7f576415ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5769e76640 1 --2- 172.21.15.133:0/1618075897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576415ad10 0x7f576415ea10 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f57640b4d40 tx=0x7f575c01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5769e76640 1 -- 172.21.15.133:0/1618075897 shutdown_connections 2024-04-25T10:16:53.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5769e76640 1 --2- 172.21.15.133:0/1618075897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576415ad10 0x7f576415ea10 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5769e76640 1 --2- 172.21.15.133:0/1618075897 >> v2:172.21.15.162:3300/0 conn(0x7f5764158110 0x7f57640b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5769e76640 1 -- 172.21.15.133:0/1618075897 >> 172.21.15.133:0/1618075897 conn(0x7f5764067000 msgr2=0x7f576406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5769e76640 1 -- 172.21.15.133:0/1618075897 shutdown_connections 2024-04-25T10:16:53.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.652+0000 7f5769e76640 1 -- 172.21.15.133:0/1618075897 wait complete. 2024-04-25T10:16:53.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5769e76640 1 Processor -- start 2024-04-25T10:16:53.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5769e76640 1 -- start start 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5769e76640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5764158110 0x7f57641571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5769e76640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57641584f0 0x7f5764165690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5769e76640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576415ad10 0x7f5764165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5769e76640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576415f0d0 con 0x7f576415ad10 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5769e76640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576415ef50 con 0x7f5764158110 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5769e76640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576406db60 con 0x7f57641584f0 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5763fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576415ad10 0x7f5764165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5763fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576415ad10 0x7f5764165bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41686/0 (socket says 172.21.15.133:41686) 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5763fff640 1 -- 172.21.15.133:0/797369801 learned_addr learned my addr 172.21.15.133:0/797369801 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f57637fe640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5764158110 0x7f57641571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f575affd640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57641584f0 0x7f5764165690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5763fff640 1 -- 172.21.15.133:0/797369801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57641584f0 msgr2=0x7f5764165690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5763fff640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57641584f0 0x7f5764165690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5763fff640 1 -- 172.21.15.133:0/797369801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5764158110 msgr2=0x7f57641571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5763fff640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5764158110 0x7f57641571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5763fff640 1 -- 172.21.15.133:0/797369801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576415d890 con 0x7f576415ad10 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f57637fe640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5764158110 0x7f57641571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5763fff640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576415ad10 0x7f5764165bd0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f575c01a5e0 tx=0x7f575c019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5760ff9640 1 -- 172.21.15.133:0/797369801 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c009510 con 0x7f576415ad10 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5760ff9640 1 -- 172.21.15.133:0/797369801 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c0be070 con 0x7f576415ad10 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5760ff9640 1 -- 172.21.15.133:0/797369801 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0b88b0 con 0x7f576415ad10 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5769e76640 1 -- 172.21.15.133:0/797369801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576415cd10 con 0x7f576415ad10 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5769e76640 1 -- 172.21.15.133:0/797369801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576415d2d0 con 0x7f576415ad10 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5769e76640 1 -- 172.21.15.133:0/797369801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5764068ef0 con 0x7f576415ad10 2024-04-25T10:16:53.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5760ff9640 1 -- 172.21.15.133:0/797369801 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f575c0b8a50 con 0x7f576415ad10 2024-04-25T10:16:53.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5760ff9640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5744066f60 0x7f5744069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f5760ff9640 1 -- 172.21.15.133:0/797369801 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f575c120460 con 0x7f576415ad10 2024-04-25T10:16:53.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.656+0000 7f57637fe640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5744066f60 0x7f5744069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.660+0000 7f57637fe640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5744066f60 0x7f5744069420 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5754009840 tx=0x7f5754008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.660+0000 7f5760ff9640 1 -- 172.21.15.133:0/797369801 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f575c0165a0 con 0x7f576415ad10 2024-04-25T10:16:53.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.840+0000 7f5769e76640 1 -- 172.21.15.133:0/797369801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5764167230 con 0x7f576415ad10 2024-04-25T10:16:53.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.840+0000 7f5760ff9640 1 -- 172.21.15.133:0/797369801 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4962 (secure 0 0 0) 0x7f575c0f13b0 con 0x7f576415ad10 2024-04-25T10:16:53.843 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:53.843 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1498957356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1498957356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1498957356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/883424012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":883424012},{"type":"v1","addr":"172.21.15.133:6835","nonce":883424012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2040533186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2040533186},{"type":"v1","addr":"172.21.15.133:6837","nonce":2040533186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:08.983208+0000","modified":"2024-04-25T10:16:11.017848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11037":{"gid":11037,"name":"d","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/2658062772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2658062772},{"type":"v1","addr":"172.21.15.162:6833","nonce":2658062772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:16:53.843 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 105 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 -- 172.21.15.133:0/797369801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5744066f60 msgr2=0x7f5744069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5744066f60 0x7f5744069420 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5754009840 tx=0x7f5754008040 comp rx=0 tx=0).stop 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 -- 172.21.15.133:0/797369801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576415ad10 msgr2=0x7f5764165bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576415ad10 0x7f5764165bd0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f575c01a5e0 tx=0x7f575c019750 comp rx=0 tx=0).stop 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 -- 172.21.15.133:0/797369801 shutdown_connections 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5744066f60 0x7f5744069420 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576415ad10 0x7f5764165bd0 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f57641584f0 0x7f5764165690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 --2- 172.21.15.133:0/797369801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5764158110 0x7f57641571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 -- 172.21.15.133:0/797369801 >> 172.21.15.133:0/797369801 conn(0x7f5764067000 msgr2=0x7f576415a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 -- 172.21.15.133:0/797369801 shutdown_connections 2024-04-25T10:16:53.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.844+0000 7f5769e76640 1 -- 172.21.15.133:0/797369801 wait complete. 2024-04-25T10:16:53.861 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:53.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.980+0000 7f1a8b382640 1 Processor -- start 2024-04-25T10:16:53.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.980+0000 7f1a8b382640 1 -- start start 2024-04-25T10:16:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.980+0000 7f1a8b382640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a840ced30 0x7f1a84168950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.980+0000 7f1a8b382640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a84161e70 con 0x7f1a8415dcf0 2024-04-25T10:16:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.980+0000 7f1a8b382640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a84162150 con 0x7f1a8415d910 2024-04-25T10:16:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.980+0000 7f1a8b382640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84162430 con 0x7f1a840ced30 2024-04-25T10:16:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.980+0000 7f1a8891d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1a8415dcf0 0x7f1a840ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53644/0 (socket says 172.21.15.133:53644) 2024-04-25T10:16:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.980+0000 7f1a8891d640 1 -- 172.21.15.133:0/1489924629 learned_addr learned my addr 172.21.15.133:0/1489924629 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a8991f640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a840ced30 0x7f1a84168950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757674623 0 0) 0x7f1a84161e70 con 0x7f1a8415dcf0 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a70003530 con 0x7f1a8415dcf0 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1525063649 0 0) 0x7f1a70003530 con 0x7f1a8415dcf0 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a840ced30 msgr2=0x7f1a84168950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a840ced30 0x7f1a84168950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 >> v1:172.21.15.162:6789/0 conn(0x7f1a8415d910 legacy=0x7f1a84161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a840d1d20 con 0x7f1a8415dcf0 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1a74001680 con 0x7f1a8415dcf0 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 >> v1:172.21.15.133:6789/0 conn(0x7f1a8415dcf0 legacy=0x7f1a840ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a840ced30 0x7f1a700039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a70003f30 0x7f1a70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70004d50 0x7f1a700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84162430 con 0x7f1a70004d50 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7000ba80 con 0x7f1a70003f30 2024-04-25T10:16:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7000bc80 con 0x7f1a840ced30 2024-04-25T10:16:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a8991f640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70004d50 0x7f1a700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a8911e640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a840ced30 0x7f1a700039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a8891d640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a70003f30 0x7f1a70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a8991f640 1 -- 172.21.15.133:0/1489924629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a840ced30 msgr2=0x7f1a700039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a8991f640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a840ced30 0x7f1a700039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a8991f640 1 -- 172.21.15.133:0/1489924629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a70003f30 msgr2=0x7f1a70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a8991f640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a70003f30 0x7f1a70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a8991f640 1 -- 172.21.15.133:0/1489924629 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70003530 con 0x7f1a70004d50 2024-04-25T10:16:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a8991f640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70004d50 0x7f1a700051d0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009fc0 tx=0x7f1a7c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57727942db25b6aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0b2070 con 0x7f1a70004d50 2024-04-25T10:16:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7c0a5c90 con 0x7f1a70004d50 2024-04-25T10:16:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.984+0000 7f1a7bfff640 1 -- 172.21.15.133:0/1489924629 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c009c10 con 0x7f1a70004d50 2024-04-25T10:16:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 -- 172.21.15.133:0/1489924629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70004d50 msgr2=0x7f1a700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 --2- 172.21.15.133:0/1489924629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70004d50 0x7f1a700051d0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009fc0 tx=0x7f1a7c01cc80 comp rx=0 tx=0).stop 2024-04-25T10:16:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8991f640 1 -- 172.21.15.133:0/1489924629 reap_dead start 2024-04-25T10:16:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 -- 172.21.15.133:0/1489924629 shutdown_connections 2024-04-25T10:16:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 -- 172.21.15.133:0/1489924629 >> 172.21.15.133:0/1489924629 conn(0x7f1a840d03d0 msgr2=0x7f1a840d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 -- 172.21.15.133:0/1489924629 shutdown_connections 2024-04-25T10:16:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 -- 172.21.15.133:0/1489924629 wait complete. 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 Processor -- start 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 -- start start 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a70004d50 0x7f1a841f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a841f6a80 0x7f1a840aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a8415af10 0x7f1a8415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a840d3530 con 0x7f1a8415af10 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a840b6af0 con 0x7f1a841f6a80 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a840d36b0 con 0x7f1a70004d50 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8991f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a8415af10 0x7f1a8415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8991f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a8415af10 0x7f1a8415b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41706/0 (socket says 172.21.15.133:41706) 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8991f640 1 -- 172.21.15.133:0/747491216 learned_addr learned my addr 172.21.15.133:0/747491216 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8911e640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a70004d50 0x7f1a841f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8891d640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a841f6a80 0x7f1a840aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8991f640 1 -- 172.21.15.133:0/747491216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a70004d50 msgr2=0x7f1a841f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8991f640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a70004d50 0x7f1a841f2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8991f640 1 -- 172.21.15.133:0/747491216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a841f6a80 msgr2=0x7f1a840aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8991f640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a841f6a80 0x7f1a840aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8991f640 1 -- 172.21.15.133:0/747491216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a840ab890 con 0x7f1a8415af10 2024-04-25T10:16:53.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8991f640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a8415af10 0x7f1a8415b3a0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c002240 tx=0x7f1a7c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a79ffb640 1 -- 172.21.15.133:0/747491216 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0bd070 con 0x7f1a8415af10 2024-04-25T10:16:53.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a79ffb640 1 -- 172.21.15.133:0/747491216 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7c0a5a90 con 0x7f1a8415af10 2024-04-25T10:16:53.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 -- 172.21.15.133:0/747491216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a840aac10 con 0x7f1a8415af10 2024-04-25T10:16:53.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a79ffb640 1 -- 172.21.15.133:0/747491216 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0b2040 con 0x7f1a8415af10 2024-04-25T10:16:53.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.988+0000 7f1a8b382640 1 -- 172.21.15.133:0/747491216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a840ab1d0 con 0x7f1a8415af10 2024-04-25T10:16:53.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.992+0000 7f1a8b382640 1 -- 172.21.15.133:0/747491216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a84068ef0 con 0x7f1a8415af10 2024-04-25T10:16:53.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.992+0000 7f1a79ffb640 1 -- 172.21.15.133:0/747491216 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a7c0c0070 con 0x7f1a8415af10 2024-04-25T10:16:53.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.992+0000 7f1a79ffb640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 0x7f1a68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.992+0000 7f1a79ffb640 1 -- 172.21.15.133:0/747491216 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f1a7c11ef50 con 0x7f1a8415af10 2024-04-25T10:16:53.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.992+0000 7f1a8911e640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 0x7f1a68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.992+0000 7f1a8911e640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 0x7f1a68069420 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1a80007fd0 tx=0x7f1a80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:53.996+0000 7f1a79ffb640 1 -- 172.21.15.133:0/747491216 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a7c016990 con 0x7f1a8415af10 2024-04-25T10:16:54.001 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:54.003 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.188+0000 7f1a8b382640 1 -- 172.21.15.133:0/747491216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a840d15f0 con 0x7f1a8415af10 2024-04-25T10:16:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a79ffb640 1 -- 172.21.15.133:0/747491216 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4962 (secure 0 0 0) 0x7f1a7c0a5c60 con 0x7f1a8415af10 2024-04-25T10:16:54.193 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:54.193 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1498957356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1498957356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1498957356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/883424012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":883424012},{"type":"v1","addr":"172.21.15.133:6835","nonce":883424012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2040533186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2040533186},{"type":"v1","addr":"172.21.15.133:6837","nonce":2040533186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:08.983208+0000","modified":"2024-04-25T10:16:11.017848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11037":{"gid":11037,"name":"d","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6833/2658062772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":2658062772},{"type":"v1","addr":"172.21.15.162:6833","nonce":2658062772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:16:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 105 2024-04-25T10:16:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 -- 172.21.15.133:0/747491216 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 msgr2=0x7f1a68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 0x7f1a68069420 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1a80007fd0 tx=0x7f1a80009040 comp rx=0 tx=0).stop 2024-04-25T10:16:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 -- 172.21.15.133:0/747491216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a8415af10 msgr2=0x7f1a8415b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a8415af10 0x7f1a8415b3a0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c002240 tx=0x7f1a7c009750 comp rx=0 tx=0).stop 2024-04-25T10:16:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 -- 172.21.15.133:0/747491216 shutdown_connections 2024-04-25T10:16:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a68066f60 0x7f1a68069420 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a8415af10 0x7f1a8415b3a0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a841f6a80 0x7f1a840aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 --2- 172.21.15.133:0/747491216 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a70004d50 0x7f1a841f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 -- 172.21.15.133:0/747491216 >> 172.21.15.133:0/747491216 conn(0x7f1a840d03d0 msgr2=0x7f1a841594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 -- 172.21.15.133:0/747491216 shutdown_connections 2024-04-25T10:16:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.192+0000 7f1a8b382640 1 -- 172.21.15.133:0/747491216 wait complete. 2024-04-25T10:16:54.211 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:54.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.320+0000 7f8318cb4640 1 Processor -- start 2024-04-25T10:16:54.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.320+0000 7f8318cb4640 1 -- start start 2024-04-25T10:16:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8318cb4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f831415c9a0 0x7f83140c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8318cb4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f83140b1f70 0x7f831406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8318cb4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83140c61d0 con 0x7f83140ce330 2024-04-25T10:16:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8318cb4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83140c64b0 con 0x7f83140b1f70 2024-04-25T10:16:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8318cb4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831406d4f0 con 0x7f831415c9a0 2024-04-25T10:16:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8312d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f83140ce330 0x7f831415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53650/0 (socket says 172.21.15.133:53650) 2024-04-25T10:16:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8312d9d640 1 -- 172.21.15.133:0/2546523681 learned_addr learned my addr 172.21.15.133:0/2546523681 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288359719 0 0) 0x7f83140c61d0 con 0x7f83140ce330 2024-04-25T10:16:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82fc003530 con 0x7f83140ce330 2024-04-25T10:16:54.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8311d9b640 1 --2- 172.21.15.133:0/2546523681 >> v2:172.21.15.162:3300/0 conn(0x7f83140b1f70 0x7f831406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831259c640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f831415c9a0 0x7f83140c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1398126256 0 0) 0x7f82fc003530 con 0x7f83140ce330 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f831415c9a0 msgr2=0x7f83140c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f831415c9a0 0x7f83140c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 >> v2:172.21.15.162:3300/0 conn(0x7f83140b1f70 msgr2=0x7f831406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 --2- 172.21.15.133:0/2546523681 >> v2:172.21.15.162:3300/0 conn(0x7f83140b1f70 0x7f831406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831406d7d0 con 0x7f83140ce330 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f8304001a00 con 0x7f83140ce330 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831259c640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f831415c9a0 0x7f83140c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 >> v1:172.21.15.133:6789/0 conn(0x7f83140ce330 legacy=0x7f831415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82fc0039c0 0x7f82fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82fc004810 0x7f82fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f831415c9a0 0x7f82fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83140c64b0 con 0x7f82fc004810 2024-04-25T10:16:54.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831406d4f0 con 0x7f82fc0039c0 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc00ba50 con 0x7f831415c9a0 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8311d9b640 1 --2- 172.21.15.133:0/2546523681 >> v2:172.21.15.162:3300/0 conn(0x7f83140b1f70 0x7f831406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8311d9b640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82fc004810 0x7f82fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f831259c640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82fc0039c0 0x7f82fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8312d9d640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f831415c9a0 0x7f82fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8311d9b640 1 -- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f831415c9a0 msgr2=0x7f82fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8311d9b640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f831415c9a0 0x7f82fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8311d9b640 1 -- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82fc0039c0 msgr2=0x7f82fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8311d9b640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82fc0039c0 0x7f82fc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8311d9b640 1 -- 172.21.15.133:0/2546523681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc003530 con 0x7f82fc004810 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.324+0000 7f8311d9b640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82fc004810 0x7f82fc004c90 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f83080098b0 tx=0x7f830801c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdfd1dd3718fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830800f040 con 0x7f82fc004810 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8308004020 con 0x7f82fc004810 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f831159a640 1 -- 172.21.15.133:0/2546523681 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83080043f0 con 0x7f82fc004810 2024-04-25T10:16:54.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 -- 172.21.15.133:0/2546523681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82fc004810 msgr2=0x7f82fc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 --2- 172.21.15.133:0/2546523681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82fc004810 0x7f82fc004c90 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f83080098b0 tx=0x7f830801c630 comp rx=0 tx=0).stop 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8312d9d640 1 -- 172.21.15.133:0/2546523681 reap_dead start 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 -- 172.21.15.133:0/2546523681 shutdown_connections 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 -- 172.21.15.133:0/2546523681 >> 172.21.15.133:0/2546523681 conn(0x7f8314067000 msgr2=0x7f831406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 -- 172.21.15.133:0/2546523681 shutdown_connections 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 -- 172.21.15.133:0/2546523681 wait complete. 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 Processor -- start 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 -- start start 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82fc004810 0x7f83140a8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140ca200 0x7f83141f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83141f1ab0 0x7f83140a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831415cf00 con 0x7f83141f1ab0 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831415cd80 con 0x7f82fc004810 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831415d850 con 0x7f83140ca200 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8312d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83141f1ab0 0x7f83140a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8312d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83141f1ab0 0x7f83140a9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41730/0 (socket says 172.21.15.133:41730) 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8312d9d640 1 -- 172.21.15.133:0/938162001 learned_addr learned my addr 172.21.15.133:0/938162001 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8311d9b640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140ca200 0x7f83141f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f831259c640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82fc004810 0x7f83140a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8312d9d640 1 -- 172.21.15.133:0/938162001 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140ca200 msgr2=0x7f83141f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8312d9d640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140ca200 0x7f83141f1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8312d9d640 1 -- 172.21.15.133:0/938162001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82fc004810 msgr2=0x7f83140a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8312d9d640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82fc004810 0x7f83140a8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8312d9d640 1 -- 172.21.15.133:0/938162001 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83140a8970 con 0x7f83141f1ab0 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8312d9d640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83141f1ab0 0x7f83140a9c90 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f8314070fb0 tx=0x7f8304011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8302ffd640 1 -- 172.21.15.133:0/938162001 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304010040 con 0x7f83141f1ab0 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8302ffd640 1 -- 172.21.15.133:0/938162001 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830401a070 con 0x7f83141f1ab0 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 -- 172.21.15.133:0/938162001 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83140a8c10 con 0x7f83141f1ab0 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8302ffd640 1 -- 172.21.15.133:0/938162001 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830400fad0 con 0x7f83141f1ab0 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.328+0000 7f8318cb4640 1 -- 172.21.15.133:0/938162001 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83140a91d0 con 0x7f83141f1ab0 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.332+0000 7f8318cb4640 1 -- 172.21.15.133:0/938162001 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8314068ef0 con 0x7f83141f1ab0 2024-04-25T10:16:54.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.332+0000 7f8302ffd640 1 -- 172.21.15.133:0/938162001 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f830400fc70 con 0x7f83141f1ab0 2024-04-25T10:16:54.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.332+0000 7f8302ffd640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82f4066f60 0x7f82f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.332+0000 7f8302ffd640 1 -- 172.21.15.133:0/938162001 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f8304087f60 con 0x7f83141f1ab0 2024-04-25T10:16:54.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.332+0000 7f831259c640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82f4066f60 0x7f82f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.332+0000 7f831259c640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82f4066f60 0x7f82f4069420 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f830c00c6c0 tx=0x7f830c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.336+0000 7f8302ffd640 1 -- 172.21.15.133:0/938162001 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8304058e20 con 0x7f83141f1ab0 2024-04-25T10:16:54.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.480+0000 7f8318cb4640 1 -- 172.21.15.133:0/938162001 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f831415dfa0 con 0x7f83141f1ab0 2024-04-25T10:16:54.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.484+0000 7f8302ffd640 1 -- 172.21.15.133:0/938162001 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+17052 (secure 0 0 0) 0x7f8304058720 con 0x7f83141f1ab0 2024-04-25T10:16:54.485 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:54.485 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":171,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:16:35.087106+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:07.256836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:07.975826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.2","mappings":[{"from":1,"to":0}]},{"pgid":"26.8","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 -- 172.21.15.133:0/938162001 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82f4066f60 msgr2=0x7f82f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82f4066f60 0x7f82f4069420 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f830c00c6c0 tx=0x7f830c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 -- 172.21.15.133:0/938162001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83141f1ab0 msgr2=0x7f83140a9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83141f1ab0 0x7f83140a9c90 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f8314070fb0 tx=0x7f8304011c20 comp rx=0 tx=0).stop 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 -- 172.21.15.133:0/938162001 shutdown_connections 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f82f4066f60 0x7f82f4069420 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83141f1ab0 0x7f83140a9c90 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f83140ca200 0x7f83141f1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 --2- 172.21.15.133:0/938162001 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f82fc004810 0x7f83140a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 -- 172.21.15.133:0/938162001 >> 172.21.15.133:0/938162001 conn(0x7f8314067000 msgr2=0x7f831406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 -- 172.21.15.133:0/938162001 shutdown_connections 2024-04-25T10:16:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.488+0000 7f8318cb4640 1 -- 172.21.15.133:0/938162001 wait complete. 2024-04-25T10:16:54.505 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:16:54.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f53278ea640 1 Processor -- start 2024-04-25T10:16:54.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f53278ea640 1 -- start start 2024-04-25T10:16:54.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f53278ea640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f53200ce600 0x7f53200cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f53278ea640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532015ebd0 con 0x7f53200d1920 2024-04-25T10:16:54.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f53278ea640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532015eeb0 con 0x7f532015d9a0 2024-04-25T10:16:54.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f53278ea640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53200d0330 con 0x7f53200ce600 2024-04-25T10:16:54.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5325e87640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f53200d1920 0x7f53200d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53654/0 (socket says 172.21.15.133:53654) 2024-04-25T10:16:54.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5325e87640 1 -- 172.21.15.133:0/2740689508 learned_addr learned my addr 172.21.15.133:0/2740689508 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:54.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5324e85640 1 --2- 172.21.15.133:0/2740689508 >> v2:172.21.15.162:3300/0 conn(0x7f53200ce600 0x7f53200cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151044710 0 0) 0x7f532015ebd0 con 0x7f53200d1920 2024-04-25T10:16:54.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f530c003530 con 0x7f53200d1920 2024-04-25T10:16:54.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3980480474 0 0) 0x7f530c003530 con 0x7f53200d1920 2024-04-25T10:16:54.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 >> v2:172.21.15.162:3300/0 conn(0x7f53200ce600 msgr2=0x7f53200cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 --2- 172.21.15.133:0/2740689508 >> v2:172.21.15.162:3300/0 conn(0x7f53200ce600 0x7f53200cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 >> v1:172.21.15.162:6789/0 conn(0x7f532015d9a0 legacy=0x7f53200d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53200d0610 con 0x7f53200d1920 2024-04-25T10:16:54.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5318001a00 con 0x7f53200d1920 2024-04-25T10:16:54.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 >> v1:172.21.15.133:6789/0 conn(0x7f53200d1920 legacy=0x7f53200d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f530c0039c0 0x7f530c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f530c004810 0x7f530c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0051d0 0x7f530c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53200d0330 con 0x7f530c0039c0 2024-04-25T10:16:54.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c00c170 con 0x7f530c004810 2024-04-25T10:16:54.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c00c370 con 0x7f530c0051d0 2024-04-25T10:16:54.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5325686640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f530c0039c0 0x7f530c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.620+0000 7f5324e85640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f530c004810 0x7f530c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5325e87640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0051d0 0x7f530c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5325686640 1 -- 172.21.15.133:0/2740689508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0051d0 msgr2=0x7f530c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5325686640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0051d0 0x7f530c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5325686640 1 -- 172.21.15.133:0/2740689508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f530c004810 msgr2=0x7f530c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5325686640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f530c004810 0x7f530c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5325686640 1 -- 172.21.15.133:0/2740689508 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c003530 con 0x7f530c0039c0 2024-04-25T10:16:54.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5325e87640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0051d0 0x7f530c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:54.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5325e87640 1 -- 172.21.15.133:0/2740689508 reap_dead start 2024-04-25T10:16:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5325686640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f530c0039c0 0x7f530c003de0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f531c001780 tx=0x7f531c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756fcb9ea784b805 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0013a0 con 0x7f530c0039c0 2024-04-25T10:16:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c0052f0 con 0x7f530c0039c0 2024-04-25T10:16:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f5317fff640 1 -- 172.21.15.133:0/2740689508 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c003840 con 0x7f530c0039c0 2024-04-25T10:16:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f53278ea640 1 -- 172.21.15.133:0/2740689508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f530c0039c0 msgr2=0x7f530c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f53278ea640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f530c0039c0 0x7f530c003de0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f531c001780 tx=0x7f531c01c400 comp rx=0 tx=0).stop 2024-04-25T10:16:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f53278ea640 1 -- 172.21.15.133:0/2740689508 shutdown_connections 2024-04-25T10:16:54.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f53278ea640 1 --2- 172.21.15.133:0/2740689508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f530c0039c0 0x7f530c003de0 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f53278ea640 1 -- 172.21.15.133:0/2740689508 >> 172.21.15.133:0/2740689508 conn(0x7f532006bf80 msgr2=0x7f532006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f53278ea640 1 -- 172.21.15.133:0/2740689508 shutdown_connections 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.624+0000 7f53278ea640 1 -- 172.21.15.133:0/2740689508 wait complete. 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f53278ea640 1 Processor -- start 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f53278ea640 1 -- start start 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f53278ea640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0039c0 0x7f53200b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f53278ea640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f532015c1c0 0x7f532015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f53278ea640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f532015c640 0x7f532015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f53278ea640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53200d0f40 con 0x7f532015c1c0 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f53278ea640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53200d2540 con 0x7f532015c640 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f53278ea640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53200d10c0 con 0x7f530c0039c0 2024-04-25T10:16:54.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5324e85640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f532015c1c0 0x7f532015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5324e85640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f532015c1c0 0x7f532015a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41746/0 (socket says 172.21.15.133:41746) 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5324e85640 1 -- 172.21.15.133:0/3043475141 learned_addr learned my addr 172.21.15.133:0/3043475141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5325e87640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f532015c640 0x7f532015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5325686640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0039c0 0x7f53200b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5324e85640 1 -- 172.21.15.133:0/3043475141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0039c0 msgr2=0x7f53200b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5324e85640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0039c0 0x7f53200b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5324e85640 1 -- 172.21.15.133:0/3043475141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f532015c640 msgr2=0x7f532015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5324e85640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f532015c640 0x7f532015ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5324e85640 1 -- 172.21.15.133:0/3043475141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53201e9dc0 con 0x7f532015c1c0 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5325686640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0039c0 0x7f53200b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5324e85640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f532015c1c0 0x7f532015a5d0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f5310011660 tx=0x7f5310011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5315ffb640 1 -- 172.21.15.133:0/3043475141 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531000f9c0 con 0x7f532015c1c0 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5315ffb640 1 -- 172.21.15.133:0/3043475141 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531000fb60 con 0x7f532015c1c0 2024-04-25T10:16:54.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f53278ea640 1 -- 172.21.15.133:0/3043475141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532015b140 con 0x7f532015c1c0 2024-04-25T10:16:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5315ffb640 1 -- 172.21.15.133:0/3043475141 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310015460 con 0x7f532015c1c0 2024-04-25T10:16:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f53278ea640 1 -- 172.21.15.133:0/3043475141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532015b700 con 0x7f532015c1c0 2024-04-25T10:16:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f53278ea640 1 -- 172.21.15.133:0/3043475141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5320068e80 con 0x7f532015c1c0 2024-04-25T10:16:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5315ffb640 1 -- 172.21.15.133:0/3043475141 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5310015600 con 0x7f532015c1c0 2024-04-25T10:16:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5315ffb640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5304066f60 0x7f5304069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5315ffb640 1 -- 172.21.15.133:0/3043475141 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f5310085a90 con 0x7f532015c1c0 2024-04-25T10:16:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.628+0000 7f5325686640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5304066f60 0x7f5304069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.632+0000 7f5325686640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5304066f60 0x7f5304069420 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f531c004000 tx=0x7f531c0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.632+0000 7f5315ffb640 1 -- 172.21.15.133:0/3043475141 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5310019030 con 0x7f532015c1c0 2024-04-25T10:16:54.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.828+0000 7f53278ea640 1 -- 172.21.15.133:0/3043475141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f53200b72d0 con 0x7f532015c1c0 2024-04-25T10:16:54.829 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:16:54.828+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:16:54.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.828+0000 7f5315ffb640 1 -- 172.21.15.133:0/3043475141 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5310056a30 con 0x7f532015c1c0 2024-04-25T10:16:54.832 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:16:54.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 -- 172.21.15.133:0/3043475141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5304066f60 msgr2=0x7f5304069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5304066f60 0x7f5304069420 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f531c004000 tx=0x7f531c0011a0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 -- 172.21.15.133:0/3043475141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f532015c1c0 msgr2=0x7f532015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f532015c1c0 0x7f532015a5d0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f5310011660 tx=0x7f5310011b20 comp rx=0 tx=0).stop 2024-04-25T10:16:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 -- 172.21.15.133:0/3043475141 shutdown_connections 2024-04-25T10:16:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f532015c640 0x7f532015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5304066f60 0x7f5304069420 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f532015c1c0 0x7f532015a5d0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 --2- 172.21.15.133:0/3043475141 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f530c0039c0 0x7f53200b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 -- 172.21.15.133:0/3043475141 >> 172.21.15.133:0/3043475141 conn(0x7f532006bf80 msgr2=0x7f53200cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 -- 172.21.15.133:0/3043475141 shutdown_connections 2024-04-25T10:16:54.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.832+0000 7f53278ea640 1 -- 172.21.15.133:0/3043475141 wait complete. 2024-04-25T10:16:54.848 INFO:tasks.ceph.mds.d.smithi162.stderr: -5109> 2024-04-25T10:16:15.288+0000 7f49a5cba640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:54.854 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:16:54.855 INFO:tasks.ceph.mds.d.smithi162.stderr: -3298> 2024-04-25T10:16:18.232+0000 7f49a1cb2640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:16:17.151178+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={11199=0-12582912@1} caps={11199=pAsxLsXsxFsxcrwb/pAsxXsxFxcwb@4},l=11199 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5637c34c1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(26)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:16:54.893 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:16:54.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.972+0000 7f5637f1a640 1 Processor -- start 2024-04-25T10:16:54.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.972+0000 7f5637f1a640 1 -- start start 2024-04-25T10:16:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.972+0000 7f5637f1a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f563015d5e0 0x7f56300cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.972+0000 7f5637f1a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56300ce2e0 con 0x7f5630145170 2024-04-25T10:16:54.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.972+0000 7f5637f1a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563006d4b0 con 0x7f56300cd820 2024-04-25T10:16:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.972+0000 7f5637f1a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563006db00 con 0x7f563015d5e0 2024-04-25T10:16:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.972+0000 7f5635cb6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5630145170 0x7f563006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53656/0 (socket says 172.21.15.133:53656) 2024-04-25T10:16:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.972+0000 7f5635cb6640 1 -- 172.21.15.133:0/830577183 learned_addr learned my addr 172.21.15.133:0/830577183 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f56364b7640 1 --2- 172.21.15.133:0/830577183 >> v2:172.21.15.162:3300/0 conn(0x7f563015d5e0 0x7f56300cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5634cb4640 1 -- 172.21.15.133:0/830577183 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709244787 0 0) 0x7f56300ce2e0 con 0x7f5630145170 2024-04-25T10:16:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5634cb4640 1 -- 172.21.15.133:0/830577183 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f561c003530 con 0x7f5630145170 2024-04-25T10:16:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f56364b7640 1 -- 172.21.15.133:0/830577183 >> v1:172.21.15.162:6789/0 conn(0x7f56300cd820 legacy=0x7f563015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5634cb4640 1 -- 172.21.15.133:0/830577183 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3099273472 0 0) 0x7f561c003530 con 0x7f5630145170 2024-04-25T10:16:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f56364b7640 1 -- 172.21.15.133:0/830577183 >> v1:172.21.15.133:6789/0 conn(0x7f5630145170 legacy=0x7f563006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f56364b7640 1 -- 172.21.15.133:0/830577183 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563006d790 con 0x7f563015d5e0 2024-04-25T10:16:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f56364b7640 1 --2- 172.21.15.133:0/830577183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f563015d5e0 0x7f56300cdca0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f56280097c0 tx=0x7f562801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2eeeb6fbb8dc73b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5634cb4640 1 -- 172.21.15.133:0/830577183 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56280b0070 con 0x7f563015d5e0 2024-04-25T10:16:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5634cb4640 1 -- 172.21.15.133:0/830577183 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56280ab400 con 0x7f563015d5e0 2024-04-25T10:16:54.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5634cb4640 1 -- 172.21.15.133:0/830577183 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c001a00 con 0x7f563015d5e0 2024-04-25T10:16:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5637f1a640 1 -- 172.21.15.133:0/830577183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f563015d5e0 msgr2=0x7f56300cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5637f1a640 1 --2- 172.21.15.133:0/830577183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f563015d5e0 0x7f56300cdca0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f56280097c0 tx=0x7f562801c3c0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5637f1a640 1 -- 172.21.15.133:0/830577183 shutdown_connections 2024-04-25T10:16:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5637f1a640 1 --2- 172.21.15.133:0/830577183 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f563015d5e0 0x7f56300cdca0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5637f1a640 1 -- 172.21.15.133:0/830577183 >> 172.21.15.133:0/830577183 conn(0x7f5630066f70 msgr2=0x7f563006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5637f1a640 1 -- 172.21.15.133:0/830577183 shutdown_connections 2024-04-25T10:16:54.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.976+0000 7f5637f1a640 1 -- 172.21.15.133:0/830577183 wait complete. 2024-04-25T10:16:54.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5637f1a640 1 Processor -- start 2024-04-25T10:16:54.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5637f1a640 1 -- start start 2024-04-25T10:16:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5637f1a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56300cd820 0x7f563013b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5637f1a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5630145170 0x7f56301f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5637f1a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563013c080 0x7f56301411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5637f1a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56301696b0 con 0x7f563013c080 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5637f1a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5630145550 con 0x7f56300cd820 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5637f1a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5630169830 con 0x7f5630145170 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56364b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563013c080 0x7f56301411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56364b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563013c080 0x7f56301411c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41754/0 (socket says 172.21.15.133:41754) 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56364b7640 1 -- 172.21.15.133:0/933996782 learned_addr learned my addr 172.21.15.133:0/933996782 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56354b5640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5630145170 0x7f56301f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5635cb6640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56300cd820 0x7f563013b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56364b7640 1 -- 172.21.15.133:0/933996782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5630145170 msgr2=0x7f56301f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56364b7640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5630145170 0x7f56301f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56364b7640 1 -- 172.21.15.133:0/933996782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56300cd820 msgr2=0x7f563013b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56364b7640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56300cd820 0x7f563013b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56364b7640 1 -- 172.21.15.133:0/933996782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56301f4d60 con 0x7f563013c080 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5635cb6640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56300cd820 0x7f563013b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56364b7640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563013c080 0x7f56301411c0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f56280097c0 tx=0x7f562801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56267fc640 1 -- 172.21.15.133:0/933996782 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56280b0050 con 0x7f563013c080 2024-04-25T10:16:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56267fc640 1 -- 172.21.15.133:0/933996782 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56280b1040 con 0x7f563013c080 2024-04-25T10:16:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5637f1a640 1 -- 172.21.15.133:0/933996782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5630141ac0 con 0x7f563013c080 2024-04-25T10:16:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56267fc640 1 -- 172.21.15.133:0/933996782 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5628004220 con 0x7f563013c080 2024-04-25T10:16:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5637f1a640 1 -- 172.21.15.133:0/933996782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5630142080 con 0x7f563013c080 2024-04-25T10:16:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f5637f1a640 1 -- 172.21.15.133:0/933996782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5630068e80 con 0x7f563013c080 2024-04-25T10:16:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.980+0000 7f56267fc640 1 -- 172.21.15.133:0/933996782 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56280be020 con 0x7f563013c080 2024-04-25T10:16:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.984+0000 7f56267fc640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5614066f60 0x7f5614069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.984+0000 7f56267fc640 1 -- 172.21.15.133:0/933996782 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f562811f060 con 0x7f563013c080 2024-04-25T10:16:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.984+0000 7f5635cb6640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5614066f60 0x7f5614069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.984+0000 7f5635cb6640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5614066f60 0x7f5614069420 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f562c005c10 tx=0x7f562c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:54.988+0000 7f56267fc640 1 -- 172.21.15.133:0/933996782 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5628015fb0 con 0x7f563013c080 2024-04-25T10:16:55.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.168+0000 7f5637f1a640 1 -- 172.21.15.133:0/933996782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f563006b110 con 0x7f563013c080 2024-04-25T10:16:55.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.832+0000 7f56267fc640 1 -- 172.21.15.133:0/933996782 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v107) v1 ==== 104+0+0 (secure 0 0 0) 0x7f56280effc0 con 0x7f563013c080 2024-04-25T10:16:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 -- 172.21.15.133:0/933996782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5614066f60 msgr2=0x7f5614069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5614066f60 0x7f5614069420 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f562c005c10 tx=0x7f562c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 -- 172.21.15.133:0/933996782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563013c080 msgr2=0x7f56301411c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563013c080 0x7f56301411c0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f56280097c0 tx=0x7f562801ac00 comp rx=0 tx=0).stop 2024-04-25T10:16:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 -- 172.21.15.133:0/933996782 shutdown_connections 2024-04-25T10:16:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5614066f60 0x7f5614069420 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f563013c080 0x7f56301411c0 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5630145170 0x7f56301f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 --2- 172.21.15.133:0/933996782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56300cd820 0x7f563013b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 -- 172.21.15.133:0/933996782 >> 172.21.15.133:0/933996782 conn(0x7f5630066f70 msgr2=0x7f563015e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 -- 172.21.15.133:0/933996782 shutdown_connections 2024-04-25T10:16:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.836+0000 7f5637f1a640 1 -- 172.21.15.133:0/933996782 wait complete. 2024-04-25T10:16:55.859 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:16:55.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c6b6d0640 1 Processor -- start 2024-04-25T10:16:55.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c6b6d0640 1 -- start start 2024-04-25T10:16:55.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c6b6d0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c64071cb0 0x7f5c6415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c6b6d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c6415c250 0x7f5c640cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c6b6d0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6415c9b0 con 0x7f5c640b4940 2024-04-25T10:16:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c6b6d0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c64072a90 con 0x7f5c64071cb0 2024-04-25T10:16:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c6b6d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640a85d0 con 0x7f5c6415c250 2024-04-25T10:16:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c68c6b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c64071cb0 0x7f5c6415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c68c6b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c64071cb0 0x7f5c6415bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41766/0 (socket says 172.21.15.133:41766) 2024-04-25T10:16:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c68c6b640 1 -- 172.21.15.133:0/1217618824 learned_addr learned my addr 172.21.15.133:0/1217618824 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c69c6d640 1 --2- 172.21.15.133:0/1217618824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c6415c250 0x7f5c640cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c68c6b640 1 -- 172.21.15.133:0/1217618824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c6415c250 msgr2=0x7f5c640cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c68c6b640 1 --2- 172.21.15.133:0/1217618824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c6415c250 0x7f5c640cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c68c6b640 1 -- 172.21.15.133:0/1217618824 >> v1:172.21.15.133:6789/0 conn(0x7f5c640b4940 legacy=0x7f5c640b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.984+0000 7f5c68c6b640 1 -- 172.21.15.133:0/1217618824 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6415cc90 con 0x7f5c64071cb0 2024-04-25T10:16:55.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c68c6b640 1 --2- 172.21.15.133:0/1217618824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c64071cb0 0x7f5c6415bd10 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009870 tx=0x7f5c5801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47b2001acfc553 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c63fff640 1 -- 172.21.15.133:0/1217618824 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c580a95f0 con 0x7f5c64071cb0 2024-04-25T10:16:55.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c63fff640 1 -- 172.21.15.133:0/1217618824 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c580a9790 con 0x7f5c64071cb0 2024-04-25T10:16:55.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c63fff640 1 -- 172.21.15.133:0/1217618824 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c580a9b60 con 0x7f5c64071cb0 2024-04-25T10:16:55.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/1217618824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c64071cb0 msgr2=0x7f5c6415bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c6b6d0640 1 --2- 172.21.15.133:0/1217618824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c64071cb0 0x7f5c6415bd10 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009870 tx=0x7f5c5801c530 comp rx=0 tx=0).stop 2024-04-25T10:16:55.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/1217618824 shutdown_connections 2024-04-25T10:16:55.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c6b6d0640 1 --2- 172.21.15.133:0/1217618824 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c6415c250 0x7f5c640cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c6b6d0640 1 --2- 172.21.15.133:0/1217618824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c64071cb0 0x7f5c6415bd10 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/1217618824 >> 172.21.15.133:0/1217618824 conn(0x7f5c6406d370 msgr2=0x7f5c6406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/1217618824 shutdown_connections 2024-04-25T10:16:55.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/1217618824 wait complete. 2024-04-25T10:16:55.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c6b6d0640 1 Processor -- start 2024-04-25T10:16:55.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.988+0000 7f5c6b6d0640 1 -- start start 2024-04-25T10:16:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6b6d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c64071cb0 0x7f5c6413c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6b6d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c640b4940 0x7f5c6413c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6b6d0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c6415c250 0x7f5c64159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6b6d0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640cc960 con 0x7f5c6415c250 2024-04-25T10:16:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6b6d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6415c660 con 0x7f5c64071cb0 2024-04-25T10:16:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6b6d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640ccae0 con 0x7f5c640b4940 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c69c6d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c6415c250 0x7f5c64159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c69c6d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c6415c250 0x7f5c64159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41768/0 (socket says 172.21.15.133:41768) 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c69c6d640 1 -- 172.21.15.133:0/2664561097 learned_addr learned my addr 172.21.15.133:0/2664561097 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c68c6b640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c640b4940 0x7f5c6413c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6946c640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c64071cb0 0x7f5c6413c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c69c6d640 1 -- 172.21.15.133:0/2664561097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c640b4940 msgr2=0x7f5c6413c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c69c6d640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c640b4940 0x7f5c6413c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c69c6d640 1 -- 172.21.15.133:0/2664561097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c64071cb0 msgr2=0x7f5c6413c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c69c6d640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c64071cb0 0x7f5c6413c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c69c6d640 1 -- 172.21.15.133:0/2664561097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640ad6e0 con 0x7f5c6415c250 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c69c6d640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c6415c250 0x7f5c64159890 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00d590 tx=0x7f5c5c00da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c61ffb640 1 -- 172.21.15.133:0/2664561097 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0125f0 con 0x7f5c6415c250 2024-04-25T10:16:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c61ffb640 1 -- 172.21.15.133:0/2664561097 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c019070 con 0x7f5c6415c250 2024-04-25T10:16:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/2664561097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c640ad9c0 con 0x7f5c6415c250 2024-04-25T10:16:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c61ffb640 1 -- 172.21.15.133:0/2664561097 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0129f0 con 0x7f5c6415c250 2024-04-25T10:16:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/2664561097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c640adf80 con 0x7f5c6415c250 2024-04-25T10:16:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/2664561097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c64069000 con 0x7f5c6415c250 2024-04-25T10:16:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c61ffb640 1 -- 172.21.15.133:0/2664561097 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c5c012b90 con 0x7f5c6415c250 2024-04-25T10:16:55.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c61ffb640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c50066f60 0x7f5c50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c61ffb640 1 -- 172.21.15.133:0/2664561097 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f5c5c0856b0 con 0x7f5c6415c250 2024-04-25T10:16:55.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6946c640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c50066f60 0x7f5c50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.992+0000 7f5c6946c640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c50066f60 0x7f5c50069420 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5c641592c0 tx=0x7f5c54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:55.996+0000 7f5c61ffb640 1 -- 172.21.15.133:0/2664561097 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c5c056480 con 0x7f5c6415c250 2024-04-25T10:16:56.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.156+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/2664561097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5c6415e6f0 con 0x7f5c6415c250 2024-04-25T10:16:56.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.844+0000 7f5c61ffb640 1 -- 172.21.15.133:0/2664561097 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v173) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5c5c055d80 con 0x7f5c6415c250 2024-04-25T10:16:56.845 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:16:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.848+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/2664561097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c50066f60 msgr2=0x7f5c50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.848+0000 7f5c6b6d0640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c50066f60 0x7f5c50069420 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5c641592c0 tx=0x7f5c54009040 comp rx=0 tx=0).stop 2024-04-25T10:16:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.848+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/2664561097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c6415c250 msgr2=0x7f5c64159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.848+0000 7f5c6b6d0640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c6415c250 0x7f5c64159890 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00d590 tx=0x7f5c5c00da50 comp rx=0 tx=0).stop 2024-04-25T10:16:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.848+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/2664561097 shutdown_connections 2024-04-25T10:16:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.848+0000 7f5c6b6d0640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c50066f60 0x7f5c50069420 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.848+0000 7f5c6b6d0640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c6415c250 0x7f5c64159890 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.848+0000 7f5c6b6d0640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c640b4940 0x7f5c6413c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.848+0000 7f5c6b6d0640 1 --2- 172.21.15.133:0/2664561097 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c64071cb0 0x7f5c6413c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:56.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.848+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/2664561097 >> 172.21.15.133:0/2664561097 conn(0x7f5c6406d370 msgr2=0x7f5c640735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:56.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.852+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/2664561097 shutdown_connections 2024-04-25T10:16:56.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:56.852+0000 7f5c6b6d0640 1 -- 172.21.15.133:0/2664561097 wait complete. 2024-04-25T10:16:56.883 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb54f1b3640 1 Processor -- start 2024-04-25T10:16:57.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb54f1b3640 1 -- start start 2024-04-25T10:16:57.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb54f1b3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb54815b910 0x7fb5480ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb54f1b3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54815bf40 con 0x7fb5480cebd0 2024-04-25T10:16:57.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb54f1b3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54815c200 con 0x7fb54815b530 2024-04-25T10:16:57.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb54f1b3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5481589c0 con 0x7fb54815b910 2024-04-25T10:16:57.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb54d750640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb5480cebd0 0x7fb54815cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38110/0 (socket says 172.21.15.133:38110) 2024-04-25T10:16:57.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb54d750640 1 -- 172.21.15.133:0/1960168257 learned_addr learned my addr 172.21.15.133:0/1960168257 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:57.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb547fff640 1 --2- 172.21.15.133:0/1960168257 >> v2:172.21.15.162:3300/0 conn(0x7fb54815b910 0x7fb5480ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413121223 0 0) 0x7fb54815bf40 con 0x7fb5480cebd0 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb530003530 con 0x7fb5480cebd0 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4061326641 0 0) 0x7fb530003530 con 0x7fb5480cebd0 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 >> v2:172.21.15.162:3300/0 conn(0x7fb54815b910 msgr2=0x7fb5480ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 --2- 172.21.15.133:0/1960168257 >> v2:172.21.15.162:3300/0 conn(0x7fb54815b910 0x7fb5480ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 >> v1:172.21.15.162:6789/0 conn(0x7fb54815b530 legacy=0x7fb5480cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54815c4e0 con 0x7fb5480cebd0 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb540001a00 con 0x7fb5480cebd0 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 >> v1:172.21.15.133:6789/0 conn(0x7fb5480cebd0 legacy=0x7fb54815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5300039c0 0x7fb530003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5300043c0 0x7fb530004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb530005130 0x7fb53000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53000c040 con 0x7fb5300039c0 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53000c1f0 con 0x7fb5300043c0 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53000e600 con 0x7fb530005130 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.008+0000 7fb54cf4f640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5300039c0 0x7fb530003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb547fff640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5300043c0 0x7fb530004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54cf4f640 1 -- 172.21.15.133:0/1960168257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb530005130 msgr2=0x7fb53000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54d750640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb530005130 0x7fb53000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54cf4f640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb530005130 0x7fb53000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54cf4f640 1 -- 172.21.15.133:0/1960168257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5300043c0 msgr2=0x7fb530004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54cf4f640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5300043c0 0x7fb530004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54cf4f640 1 -- 172.21.15.133:0/1960168257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb530003530 con 0x7fb5300039c0 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb547fff640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5300043c0 0x7fb530004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:57.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54d750640 1 -- 172.21.15.133:0/1960168257 reap_dead start 2024-04-25T10:16:57.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54cf4f640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5300039c0 0x7fb530003da0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fb538001860 tx=0x7fb53801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5081cfa432c84a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:57.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5380b0070 con 0x7fb5300039c0 2024-04-25T10:16:57.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5380ab600 con 0x7fb5300039c0 2024-04-25T10:16:57.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb5477fe640 1 -- 172.21.15.133:0/1960168257 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538001460 con 0x7fb5300039c0 2024-04-25T10:16:57.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54f1b3640 1 -- 172.21.15.133:0/1960168257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5300039c0 msgr2=0x7fb530003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:57.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54f1b3640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5300039c0 0x7fb530003da0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fb538001860 tx=0x7fb53801c500 comp rx=0 tx=0).stop 2024-04-25T10:16:57.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54f1b3640 1 -- 172.21.15.133:0/1960168257 shutdown_connections 2024-04-25T10:16:57.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54f1b3640 1 --2- 172.21.15.133:0/1960168257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5300039c0 0x7fb530003da0 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:57.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54f1b3640 1 -- 172.21.15.133:0/1960168257 >> 172.21.15.133:0/1960168257 conn(0x7fb54806d370 msgr2=0x7fb54806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:57.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54f1b3640 1 -- 172.21.15.133:0/1960168257 shutdown_connections 2024-04-25T10:16:57.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.012+0000 7fb54f1b3640 1 -- 172.21.15.133:0/1960168257 wait complete. 2024-04-25T10:16:57.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54f1b3640 1 Processor -- start 2024-04-25T10:16:57.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54f1b3640 1 -- start start 2024-04-25T10:16:57.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54f1b3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5300039c0 0x7fb548145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54f1b3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5481599f0 0x7fb548159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54f1b3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5480b42b0 0x7fb5480b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54f1b3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815e700 con 0x7fb5481599f0 2024-04-25T10:16:57.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54f1b3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815e580 con 0x7fb5300039c0 2024-04-25T10:16:57.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54f1b3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815e880 con 0x7fb5480b42b0 2024-04-25T10:16:57.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb547fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5481599f0 0x7fb548159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb547fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5481599f0 0x7fb548159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43930/0 (socket says 172.21.15.133:43930) 2024-04-25T10:16:57.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb547fff640 1 -- 172.21.15.133:0/209861642 learned_addr learned my addr 172.21.15.133:0/209861642 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54cf4f640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5300039c0 0x7fb548145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54d750640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5480b42b0 0x7fb5480b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb547fff640 1 -- 172.21.15.133:0/209861642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5480b42b0 msgr2=0x7fb5480b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb547fff640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5480b42b0 0x7fb5480b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb547fff640 1 -- 172.21.15.133:0/209861642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5300039c0 msgr2=0x7fb548145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb547fff640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5300039c0 0x7fb548145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb547fff640 1 -- 172.21.15.133:0/209861642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb548151ad0 con 0x7fb5481599f0 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54cf4f640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5300039c0 0x7fb548145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb547fff640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5481599f0 0x7fb548159e70 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7fb53c011510 tx=0x7fb53c0119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb5457fa640 1 -- 172.21.15.133:0/209861642 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c004220 con 0x7fb5481599f0 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb5457fa640 1 -- 172.21.15.133:0/209861642 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c0043c0 con 0x7fb5481599f0 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54f1b3640 1 -- 172.21.15.133:0/209861642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb548151d70 con 0x7fb5481599f0 2024-04-25T10:16:57.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb5457fa640 1 -- 172.21.15.133:0/209861642 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c015890 con 0x7fb5481599f0 2024-04-25T10:16:57.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54f1b3640 1 -- 172.21.15.133:0/209861642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5480b5080 con 0x7fb5481599f0 2024-04-25T10:16:57.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb54f1b3640 1 -- 172.21.15.133:0/209861642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548069000 con 0x7fb5481599f0 2024-04-25T10:16:57.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb5457fa640 1 -- 172.21.15.133:0/209861642 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb53c015a30 con 0x7fb5481599f0 2024-04-25T10:16:57.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.016+0000 7fb5457fa640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb534066f60 0x7fb534069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:57.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.020+0000 7fb5457fa640 1 -- 172.21.15.133:0/209861642 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5836+0+0 (secure 0 0 0) 0x7fb53c0854f0 con 0x7fb5481599f0 2024-04-25T10:16:57.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.020+0000 7fb54cf4f640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb534066f60 0x7fb534069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:57.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.020+0000 7fb54cf4f640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb534066f60 0x7fb534069420 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb5380ad000 tx=0x7fb5380a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:57.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.024+0000 7fb5457fa640 1 -- 172.21.15.133:0/209861642 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb53c019030 con 0x7fb5481599f0 2024-04-25T10:16:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:57.184+0000 7fb54f1b3640 1 -- 172.21.15.133:0/209861642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb548072690 con 0x7fb5481599f0 2024-04-25T10:16:58.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.188+0000 7fb5457fa640 1 -- 172.21.15.133:0/209861642 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v174) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb53c056510 con 0x7fb5481599f0 2024-04-25T10:16:58.189 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:16:58.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 -- 172.21.15.133:0/209861642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb534066f60 msgr2=0x7fb534069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb534066f60 0x7fb534069420 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb5380ad000 tx=0x7fb5380a6000 comp rx=0 tx=0).stop 2024-04-25T10:16:58.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 -- 172.21.15.133:0/209861642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5481599f0 msgr2=0x7fb548159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5481599f0 0x7fb548159e70 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7fb53c011510 tx=0x7fb53c0119d0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 -- 172.21.15.133:0/209861642 shutdown_connections 2024-04-25T10:16:58.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5480b42b0 0x7fb5480b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb534066f60 0x7fb534069420 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5481599f0 0x7fb548159e70 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 --2- 172.21.15.133:0/209861642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5300039c0 0x7fb548145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 -- 172.21.15.133:0/209861642 >> 172.21.15.133:0/209861642 conn(0x7fb54806d370 msgr2=0x7fb5480af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:58.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 -- 172.21.15.133:0/209861642 shutdown_connections 2024-04-25T10:16:58.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.192+0000 7fb54f1b3640 1 -- 172.21.15.133:0/209861642 wait complete. 2024-04-25T10:16:58.217 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:16:58.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.340+0000 7f9c647ba640 1 Processor -- start 2024-04-25T10:16:58.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.340+0000 7f9c647ba640 1 -- start start 2024-04-25T10:16:58.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.340+0000 7f9c647ba640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9c5c15d9a0 0x7f9c5c0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.340+0000 7f9c647ba640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c5c0ce600 0x7f9c5c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.340+0000 7f9c647ba640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c15ebd0 con 0x7f9c5c0d1920 2024-04-25T10:16:58.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.340+0000 7f9c647ba640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c15eeb0 con 0x7f9c5c0ce600 2024-04-25T10:16:58.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.340+0000 7f9c647ba640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c0d0330 con 0x7f9c5c15d9a0 2024-04-25T10:16:58.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.340+0000 7f9c61d55640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c5c0ce600 0x7f9c5c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.340+0000 7f9c62d57640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9c5c0d1920 0x7f9c5c0d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38122/0 (socket says 172.21.15.133:38122) 2024-04-25T10:16:58.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.340+0000 7f9c62d57640 1 -- 172.21.15.133:0/2553364156 learned_addr learned my addr 172.21.15.133:0/2553364156 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:58.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c62556640 1 --2- 172.21.15.133:0/2553364156 >> v2:172.21.15.162:3300/0 conn(0x7f9c5c15d9a0 0x7f9c5c0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c61d55640 1 -- 172.21.15.133:0/2553364156 >> v2:172.21.15.162:3300/0 conn(0x7f9c5c15d9a0 msgr2=0x7f9c5c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:58.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c61d55640 1 --2- 172.21.15.133:0/2553364156 >> v2:172.21.15.162:3300/0 conn(0x7f9c5c15d9a0 0x7f9c5c0d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c61d55640 1 -- 172.21.15.133:0/2553364156 >> v1:172.21.15.133:6789/0 conn(0x7f9c5c0d1920 legacy=0x7f9c5c0d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c61d55640 1 -- 172.21.15.133:0/2553364156 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c0d0610 con 0x7f9c5c0ce600 2024-04-25T10:16:58.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c61554640 1 -- 172.21.15.133:0/2553364156 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972037842 0 0) 0x7f9c5c15ebd0 con 0x7f9c5c0d1920 2024-04-25T10:16:58.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c61d55640 1 --2- 172.21.15.133:0/2553364156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0ce600 0x7f9c5c0cea90 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0d4750 tx=0x7f9c5001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c51ad82db688c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:58.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c61554640 1 -- 172.21.15.133:0/2553364156 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c500b0070 con 0x7f9c5c0ce600 2024-04-25T10:16:58.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c61554640 1 -- 172.21.15.133:0/2553364156 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c500ab600 con 0x7f9c5c0ce600 2024-04-25T10:16:58.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c61554640 1 -- 172.21.15.133:0/2553364156 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c500ab9d0 con 0x7f9c5c0ce600 2024-04-25T10:16:58.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c647ba640 1 -- 172.21.15.133:0/2553364156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0ce600 msgr2=0x7f9c5c0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c647ba640 1 --2- 172.21.15.133:0/2553364156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0ce600 0x7f9c5c0cea90 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0d4750 tx=0x7f9c5001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c647ba640 1 -- 172.21.15.133:0/2553364156 shutdown_connections 2024-04-25T10:16:58.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c647ba640 1 --2- 172.21.15.133:0/2553364156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0ce600 0x7f9c5c0cea90 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c647ba640 1 --2- 172.21.15.133:0/2553364156 >> v2:172.21.15.162:3300/0 conn(0x7f9c5c15d9a0 0x7f9c5c0d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c647ba640 1 -- 172.21.15.133:0/2553364156 >> 172.21.15.133:0/2553364156 conn(0x7f9c5c06bf80 msgr2=0x7f9c5c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:58.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c647ba640 1 -- 172.21.15.133:0/2553364156 shutdown_connections 2024-04-25T10:16:58.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.344+0000 7f9c647ba640 1 -- 172.21.15.133:0/2553364156 wait complete. 2024-04-25T10:16:58.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c647ba640 1 Processor -- start 2024-04-25T10:16:58.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c647ba640 1 -- start start 2024-04-25T10:16:58.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c647ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0ce600 0x7f9c5c156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c647ba640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c5c0d1920 0x7f9c5c1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c647ba640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c5c15d9a0 0x7f9c5c0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c647ba640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0b9820 con 0x7f9c5c0ce600 2024-04-25T10:16:58.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c647ba640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0b96a0 con 0x7f9c5c15d9a0 2024-04-25T10:16:58.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c647ba640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c15e590 con 0x7f9c5c0d1920 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c62556640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0ce600 0x7f9c5c156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c62556640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0ce600 0x7f9c5c156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43950/0 (socket says 172.21.15.133:43950) 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c62556640 1 -- 172.21.15.133:0/931665624 learned_addr learned my addr 172.21.15.133:0/931665624 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c61d55640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c5c0d1920 0x7f9c5c1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c62556640 1 -- 172.21.15.133:0/931665624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c5c0d1920 msgr2=0x7f9c5c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c62556640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c5c0d1920 0x7f9c5c1563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c62556640 1 -- 172.21.15.133:0/931665624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c5c15d9a0 msgr2=0x7f9c5c0c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c62556640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c5c15d9a0 0x7f9c5c0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c62556640 1 -- 172.21.15.133:0/931665624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c156a90 con 0x7f9c5c0ce600 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c62556640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0ce600 0x7f9c5c156ee0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400ccd0 tx=0x7f9c5400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c5affd640 1 -- 172.21.15.133:0/931665624 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c540039a0 con 0x7f9c5c0ce600 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c5affd640 1 -- 172.21.15.133:0/931665624 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c54003b40 con 0x7f9c5c0ce600 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c647ba640 1 -- 172.21.15.133:0/931665624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c15f290 con 0x7f9c5c0ce600 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c5affd640 1 -- 172.21.15.133:0/931665624 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54004020 con 0x7f9c5c0ce600 2024-04-25T10:16:58.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c647ba640 1 -- 172.21.15.133:0/931665624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c15f850 con 0x7f9c5c0ce600 2024-04-25T10:16:58.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.348+0000 7f9c647ba640 1 -- 172.21.15.133:0/931665624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c068e80 con 0x7f9c5c0ce600 2024-04-25T10:16:58.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.352+0000 7f9c5affd640 1 -- 172.21.15.133:0/931665624 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c540041c0 con 0x7f9c5c0ce600 2024-04-25T10:16:58.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.352+0000 7f9c5affd640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9c48066f90 0x7f9c48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.352+0000 7f9c5affd640 1 -- 172.21.15.133:0/931665624 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9c54085390 con 0x7f9c5c0ce600 2024-04-25T10:16:58.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.352+0000 7f9c61d55640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9c48066f90 0x7f9c48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.352+0000 7f9c61d55640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9c48066f90 0x7f9c48069450 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9c500ad000 tx=0x7f9c50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:58.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.356+0000 7f9c5affd640 1 -- 172.21.15.133:0/931665624 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c54014030 con 0x7f9c5c0ce600 2024-04-25T10:16:58.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.516+0000 7f9c647ba640 1 -- 172.21.15.133:0/931665624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9c5c0b8940 con 0x7f9c5c0ce600 2024-04-25T10:16:58.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c5affd640 1 -- 172.21.15.133:0/931665624 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9c54056500 con 0x7f9c5c0ce600 2024-04-25T10:16:58.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 -- 172.21.15.133:0/931665624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9c48066f90 msgr2=0x7f9c48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9c48066f90 0x7f9c48069450 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f9c500ad000 tx=0x7f9c50008040 comp rx=0 tx=0).stop 2024-04-25T10:16:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 -- 172.21.15.133:0/931665624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0ce600 msgr2=0x7f9c5c156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0ce600 0x7f9c5c156ee0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400ccd0 tx=0x7f9c5400a5c0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 -- 172.21.15.133:0/931665624 shutdown_connections 2024-04-25T10:16:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9c5c15d9a0 0x7f9c5c0c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9c5c0d1920 0x7f9c5c1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9c48066f90 0x7f9c48069450 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 --2- 172.21.15.133:0/931665624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c5c0ce600 0x7f9c5c156ee0 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 -- 172.21.15.133:0/931665624 >> 172.21.15.133:0/931665624 conn(0x7f9c5c06bf80 msgr2=0x7f9c5c0b7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 -- 172.21.15.133:0/931665624 shutdown_connections 2024-04-25T10:16:58.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.524+0000 7f9c647ba640 1 -- 172.21.15.133:0/931665624 wait complete. 2024-04-25T10:16:58.546 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:16:58.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.664+0000 7fbda1694640 1 Processor -- start 2024-04-25T10:16:58.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbda1694640 1 -- start start 2024-04-25T10:16:58.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbda1694640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd9c15c9a0 0x7fbd9c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbda1694640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbd9c0ce330 0x7fbd9c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbda1694640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c0c61d0 con 0x7fbd9c0b1f70 2024-04-25T10:16:58.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbda1694640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c0cece0 con 0x7fbd9c0ce330 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbda1694640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0c64b0 con 0x7fbd9c15c9a0 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd927fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fbd9c0b1f70 0x7fbd9c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38124/0 (socket says 172.21.15.133:38124) 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd927fc640 1 -- 172.21.15.133:0/388720907 learned_addr learned my addr 172.21.15.133:0/388720907 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9b7fe640 1 --2- 172.21.15.133:0/388720907 >> v2:172.21.15.162:3300/0 conn(0x7fbd9c0ce330 0x7fbd9c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9affd640 1 --2- 172.21.15.133:0/388720907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd9c15c9a0 0x7fbd9c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556748396 0 0) 0x7fbd9c0c61d0 con 0x7fbd9c0b1f70 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd88003530 con 0x7fbd9c0b1f70 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1171279248 0 0) 0x7fbd88003530 con 0x7fbd9c0b1f70 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd9c15c9a0 msgr2=0x7fbd9c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 --2- 172.21.15.133:0/388720907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd9c15c9a0 0x7fbd9c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 >> v2:172.21.15.162:3300/0 conn(0x7fbd9c0ce330 msgr2=0x7fbd9c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 --2- 172.21.15.133:0/388720907 >> v2:172.21.15.162:3300/0 conn(0x7fbd9c0ce330 0x7fbd9c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c06d4f0 con 0x7fbd9c0b1f70 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fbd8c001680 con 0x7fbd9c0b1f70 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 >> v1:172.21.15.133:6789/0 conn(0x7fbd9c0b1f70 legacy=0x7fbd9c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 --2- 172.21.15.133:0/388720907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd9c15c9a0 0x7fbd880039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 --2- 172.21.15.133:0/388720907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd88003f40 0x7fbd88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 --2- 172.21.15.133:0/388720907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd88004d50 0x7fbd880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0cece0 con 0x7fbd88003f40 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0c64b0 con 0x7fbd88004d50 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8800b9d0 con 0x7fbd9c15c9a0 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd927fc640 1 --2- 172.21.15.133:0/388720907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd88003f40 0x7fbd88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd9b7fe640 1 --2- 172.21.15.133:0/388720907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd88004d50 0x7fbd880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd927fc640 1 -- 172.21.15.133:0/388720907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd9c15c9a0 msgr2=0x7fbd880039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd927fc640 1 --2- 172.21.15.133:0/388720907 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd9c15c9a0 0x7fbd880039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd927fc640 1 -- 172.21.15.133:0/388720907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd88004d50 msgr2=0x7fbd880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd927fc640 1 --2- 172.21.15.133:0/388720907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd88004d50 0x7fbd880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.668+0000 7fbd927fc640 1 -- 172.21.15.133:0/388720907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd88003530 con 0x7fbd88003f40 2024-04-25T10:16:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd927fc640 1 --2- 172.21.15.133:0/388720907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd88003f40 0x7fbd88004810 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00c210 tx=0x7fbd8c01f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a65327b2a3448a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c007d60 con 0x7fbd88003f40 2024-04-25T10:16:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c005ae0 con 0x7fbd88003f40 2024-04-25T10:16:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd9a7fc640 1 -- 172.21.15.133:0/388720907 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c007070 con 0x7fbd88003f40 2024-04-25T10:16:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 -- 172.21.15.133:0/388720907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd88003f40 msgr2=0x7fbd88004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 --2- 172.21.15.133:0/388720907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd88003f40 0x7fbd88004810 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00c210 tx=0x7fbd8c01f390 comp rx=0 tx=0).stop 2024-04-25T10:16:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd9b7fe640 1 -- 172.21.15.133:0/388720907 reap_dead start 2024-04-25T10:16:58.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 -- 172.21.15.133:0/388720907 shutdown_connections 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 -- 172.21.15.133:0/388720907 >> 172.21.15.133:0/388720907 conn(0x7fbd9c067000 msgr2=0x7fbd9c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 -- 172.21.15.133:0/388720907 shutdown_connections 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 -- 172.21.15.133:0/388720907 wait complete. 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 Processor -- start 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 -- start start 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd88003f40 0x7fbd9c150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd9c1f15f0 0x7fbd9c138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd9c1523d0 0x7fbd9c15ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c15cf00 con 0x7fbd9c1f15f0 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c15cd80 con 0x7fbd9c1523d0 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbda1694640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0c5980 con 0x7fbd88003f40 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd927fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd9c1f15f0 0x7fbd9c138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd927fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd9c1f15f0 0x7fbd9c138d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43968/0 (socket says 172.21.15.133:43968) 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd927fc640 1 -- 172.21.15.133:0/3036361056 learned_addr learned my addr 172.21.15.133:0/3036361056 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd9b7fe640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd9c1523d0 0x7fbd9c15ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd9affd640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd88003f40 0x7fbd9c150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd927fc640 1 -- 172.21.15.133:0/3036361056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd88003f40 msgr2=0x7fbd9c150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd927fc640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd88003f40 0x7fbd9c150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd927fc640 1 -- 172.21.15.133:0/3036361056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd9c1523d0 msgr2=0x7fbd9c15ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd927fc640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd9c1523d0 0x7fbd9c15ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd927fc640 1 -- 172.21.15.133:0/3036361056 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c153760 con 0x7fbd9c1f15f0 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.672+0000 7fbd9b7fe640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd9c1523d0 0x7fbd9c15ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbd927fc640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd9c1f15f0 0x7fbd9c138d60 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c01f1c0 tx=0x7fbd88005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbd93fff640 1 -- 172.21.15.133:0/3036361056 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c005c90 con 0x7fbd9c1f15f0 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbd93fff640 1 -- 172.21.15.133:0/3036361056 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c005e30 con 0x7fbd9c1f15f0 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbda1694640 1 -- 172.21.15.133:0/3036361056 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c153a40 con 0x7fbd9c1f15f0 2024-04-25T10:16:58.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbd93fff640 1 -- 172.21.15.133:0/3036361056 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0049f0 con 0x7fbd9c1f15f0 2024-04-25T10:16:58.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbda1694640 1 -- 172.21.15.133:0/3036361056 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c154000 con 0x7fbd9c1f15f0 2024-04-25T10:16:58.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbda1694640 1 -- 172.21.15.133:0/3036361056 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c068ef0 con 0x7fbd9c1f15f0 2024-04-25T10:16:58.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbd93fff640 1 -- 172.21.15.133:0/3036361056 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd8c0ba030 con 0x7fbd9c1f15f0 2024-04-25T10:16:58.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbd93fff640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbd7c066f90 0x7fbd7c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:58.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbd93fff640 1 -- 172.21.15.133:0/3036361056 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbd8c121c20 con 0x7fbd9c1f15f0 2024-04-25T10:16:58.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbd9affd640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbd7c066f90 0x7fbd7c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:58.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.676+0000 7fbd9affd640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbd7c066f90 0x7fbd7c069450 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c1500f0 tx=0x7fbd84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:58.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.680+0000 7fbd93fff640 1 -- 172.21.15.133:0/3036361056 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd8c0c1030 con 0x7fbd9c1f15f0 2024-04-25T10:16:58.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:58.896+0000 7fbda1694640 1 -- 172.21.15.133:0/3036361056 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7fbd9c15d5b0 con 0x7fbd9c1f15f0 2024-04-25T10:16:59.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.188+0000 7fbd93fff640 1 -- 172.21.15.133:0/3036361056 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v677) v1 ==== 133+0+0 (secure 0 0 0) 0x7fbd8c019100 con 0x7fbd9c1f15f0 2024-04-25T10:16:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 -- 172.21.15.133:0/3036361056 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbd7c066f90 msgr2=0x7fbd7c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbd7c066f90 0x7fbd7c069450 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c1500f0 tx=0x7fbd84008040 comp rx=0 tx=0).stop 2024-04-25T10:16:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 -- 172.21.15.133:0/3036361056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd9c1f15f0 msgr2=0x7fbd9c138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd9c1f15f0 0x7fbd9c138d60 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c01f1c0 tx=0x7fbd88005840 comp rx=0 tx=0).stop 2024-04-25T10:16:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 -- 172.21.15.133:0/3036361056 shutdown_connections 2024-04-25T10:16:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbd9c1523d0 0x7fbd9c15ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbd7c066f90 0x7fbd7c069450 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd9c1f15f0 0x7fbd9c138d60 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 --2- 172.21.15.133:0/3036361056 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbd88003f40 0x7fbd9c150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 -- 172.21.15.133:0/3036361056 >> 172.21.15.133:0/3036361056 conn(0x7fbd9c067000 msgr2=0x7fbd9c0b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 -- 172.21.15.133:0/3036361056 shutdown_connections 2024-04-25T10:16:59.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.192+0000 7fbda1694640 1 -- 172.21.15.133:0/3036361056 wait complete. 2024-04-25T10:16:59.214 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:16:59.214 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:16:59.214 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18194fc640 1 Processor -- start 2024-04-25T10:16:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18194fc640 1 -- start start 2024-04-25T10:16:59.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18194fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f181406dca0 0x7f18140b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18194fc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f181406bda0 0x7f18140b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18194fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f18140b20a0 0x7f18140999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18194fc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18140d0660 con 0x7f18140b20a0 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18194fc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18140cfdc0 con 0x7f181406bda0 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18194fc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18140d00a0 con 0x7f181406dca0 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18137fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f18140b20a0 0x7f18140999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18137fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f18140b20a0 0x7f18140999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43984/0 (socket says 172.21.15.133:43984) 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18137fe640 1 -- 172.21.15.133:0/773881988 learned_addr learned my addr 172.21.15.133:0/773881988 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f1812ffd640 1 --2- 172.21.15.133:0/773881988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f181406dca0 0x7f18140b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18127fc640 1 --2- 172.21.15.133:0/773881988 >> v2:172.21.15.162:3300/0 conn(0x7f181406bda0 0x7f18140b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18137fe640 1 -- 172.21.15.133:0/773881988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f181406dca0 msgr2=0x7f18140b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18137fe640 1 --2- 172.21.15.133:0/773881988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f181406dca0 0x7f18140b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18137fe640 1 -- 172.21.15.133:0/773881988 >> v2:172.21.15.162:3300/0 conn(0x7f181406bda0 msgr2=0x7f18140b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18137fe640 1 --2- 172.21.15.133:0/773881988 >> v2:172.21.15.162:3300/0 conn(0x7f181406bda0 0x7f18140b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.336+0000 7f18137fe640 1 -- 172.21.15.133:0/773881988 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18140d0380 con 0x7f18140b20a0 2024-04-25T10:16:59.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18137fe640 1 --2- 172.21.15.133:0/773881988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18140b20a0 0x7f18140999d0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f1814073130 tx=0x7f180c01c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1c91ecd2f1e9d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f1811ffb640 1 -- 172.21.15.133:0/773881988 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c0a6040 con 0x7f18140b20a0 2024-04-25T10:16:59.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f1811ffb640 1 -- 172.21.15.133:0/773881988 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c0af070 con 0x7f18140b20a0 2024-04-25T10:16:59.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f1811ffb640 1 -- 172.21.15.133:0/773881988 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c0aa660 con 0x7f18140b20a0 2024-04-25T10:16:59.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 -- 172.21.15.133:0/773881988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18140b20a0 msgr2=0x7f18140999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 --2- 172.21.15.133:0/773881988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18140b20a0 0x7f18140999d0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f1814073130 tx=0x7f180c01c610 comp rx=0 tx=0).stop 2024-04-25T10:16:59.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 -- 172.21.15.133:0/773881988 shutdown_connections 2024-04-25T10:16:59.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 --2- 172.21.15.133:0/773881988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18140b20a0 0x7f18140999d0 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 --2- 172.21.15.133:0/773881988 >> v2:172.21.15.162:3300/0 conn(0x7f181406bda0 0x7f18140b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 --2- 172.21.15.133:0/773881988 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f181406dca0 0x7f18140b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 -- 172.21.15.133:0/773881988 >> 172.21.15.133:0/773881988 conn(0x7f18140b31a0 msgr2=0x7f18140b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 -- 172.21.15.133:0/773881988 shutdown_connections 2024-04-25T10:16:59.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 -- 172.21.15.133:0/773881988 wait complete. 2024-04-25T10:16:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 Processor -- start 2024-04-25T10:16:59.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 -- start start 2024-04-25T10:16:59.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181406bda0 0x7f18140cf470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f181406dca0 0x7f1814156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18140b20a0 0x7f18140ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18140cc5e0 con 0x7f181406bda0 2024-04-25T10:16:59.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18140cc460 con 0x7f18140b20a0 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18194fc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18140cc760 con 0x7f181406dca0 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f1812ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181406bda0 0x7f18140cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f1812ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181406bda0 0x7f18140cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43996/0 (socket says 172.21.15.133:43996) 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f1812ffd640 1 -- 172.21.15.133:0/655408423 learned_addr learned my addr 172.21.15.133:0/655408423 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18127fc640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f181406dca0 0x7f1814156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.340+0000 7f18137fe640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18140b20a0 0x7f18140ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f1812ffd640 1 -- 172.21.15.133:0/655408423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f181406dca0 msgr2=0x7f1814156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f1812ffd640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f181406dca0 0x7f1814156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f1812ffd640 1 -- 172.21.15.133:0/655408423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18140b20a0 msgr2=0x7f18140ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f1812ffd640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18140b20a0 0x7f18140ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f1812ffd640 1 -- 172.21.15.133:0/655408423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1814158a80 con 0x7f181406bda0 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f18137fe640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18140b20a0 0x7f18140ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:59.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f1812ffd640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181406bda0 0x7f18140cf470 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f180800d830 tx=0x7f180800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f17fb7fe640 1 -- 172.21.15.133:0/655408423 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180800b6c0 con 0x7f181406bda0 2024-04-25T10:16:59.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f17fb7fe640 1 -- 172.21.15.133:0/655408423 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1808019070 con 0x7f181406bda0 2024-04-25T10:16:59.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f18194fc640 1 -- 172.21.15.133:0/655408423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18141576d0 con 0x7f181406bda0 2024-04-25T10:16:59.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f17fb7fe640 1 -- 172.21.15.133:0/655408423 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180800bae0 con 0x7f181406bda0 2024-04-25T10:16:59.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f18194fc640 1 -- 172.21.15.133:0/655408423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1814157c90 con 0x7f181406bda0 2024-04-25T10:16:59.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f18194fc640 1 -- 172.21.15.133:0/655408423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1814067c70 con 0x7f181406bda0 2024-04-25T10:16:59.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f17fb7fe640 1 -- 172.21.15.133:0/655408423 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f180801e460 con 0x7f181406bda0 2024-04-25T10:16:59.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f17fb7fe640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1800066f60 0x7f1800069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f18127fc640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1800066f60 0x7f1800069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f17fb7fe640 1 -- 172.21.15.133:0/655408423 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1808085540 con 0x7f181406bda0 2024-04-25T10:16:59.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.344+0000 7f18127fc640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1800066f60 0x7f1800069420 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f181409f6d0 tx=0x7f1804008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.348+0000 7f17fb7fe640 1 -- 172.21.15.133:0/655408423 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1808056750 con 0x7f181406bda0 2024-04-25T10:16:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 -- 172.21.15.133:0/655408423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1814070020 con 0x7f181406bda0 2024-04-25T10:16:59.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f17fb7fe640 1 -- 172.21.15.133:0/655408423 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+13329 (secure 0 0 0) 0x7f1808056050 con 0x7f181406bda0 2024-04-25T10:16:59.510 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:16:59.510 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":174,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:16:58.180142+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6833/2658062772":"2024-04-26T10:16:54.831635+0000","172.21.15.162:6832/2658062772":"2024-04-26T10:16:54.831635+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:59.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 -- 172.21.15.133:0/655408423 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1800066f60 msgr2=0x7f1800069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1800066f60 0x7f1800069420 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f181409f6d0 tx=0x7f1804008040 comp rx=0 tx=0).stop 2024-04-25T10:16:59.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 -- 172.21.15.133:0/655408423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181406bda0 msgr2=0x7f18140cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181406bda0 0x7f18140cf470 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f180800d830 tx=0x7f180800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 -- 172.21.15.133:0/655408423 shutdown_connections 2024-04-25T10:16:59.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18140b20a0 0x7f18140ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f181406dca0 0x7f1814156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1800066f60 0x7f1800069420 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 --2- 172.21.15.133:0/655408423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181406bda0 0x7f18140cf470 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 -- 172.21.15.133:0/655408423 >> 172.21.15.133:0/655408423 conn(0x7f18140b31a0 msgr2=0x7f18140af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 -- 172.21.15.133:0/655408423 shutdown_connections 2024-04-25T10:16:59.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.508+0000 7f18194fc640 1 -- 172.21.15.133:0/655408423 wait complete. 2024-04-25T10:16:59.529 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:59.530 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.640+0000 7f93a2f57640 1 Processor -- start 2024-04-25T10:16:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.640+0000 7f93a2f57640 1 -- start start 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f939c0cd820 0x7f939c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f939c15ced0 0x7f939c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939c0ce2e0 con 0x7f939c145170 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939c06db00 con 0x7f939c15ced0 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c145550 con 0x7f939c0cd820 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a14f4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f939c15ced0 0x7f939c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a14f4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f939c15ced0 0x7f939c15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43998/0 (socket says 172.21.15.133:43998) 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a14f4640 1 -- 172.21.15.133:0/3133259523 learned_addr learned my addr 172.21.15.133:0/3133259523 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f939bfff640 1 --2- 172.21.15.133:0/3133259523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f939c0cd820 0x7f939c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a14f4640 1 -- 172.21.15.133:0/3133259523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f939c0cd820 msgr2=0x7f939c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a14f4640 1 --2- 172.21.15.133:0/3133259523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f939c0cd820 0x7f939c0cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a14f4640 1 -- 172.21.15.133:0/3133259523 >> v1:172.21.15.162:6789/0 conn(0x7f939c145170 legacy=0x7f939c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a14f4640 1 -- 172.21.15.133:0/3133259523 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c06d4b0 con 0x7f939c15ced0 2024-04-25T10:16:59.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a14f4640 1 --2- 172.21.15.133:0/3133259523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939c15ced0 0x7f939c15d360 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f9394009e50 tx=0x7f939401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2302271438d1f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f939b7fe640 1 -- 172.21.15.133:0/3133259523 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940a5670 con 0x7f939c15ced0 2024-04-25T10:16:59.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f939b7fe640 1 -- 172.21.15.133:0/3133259523 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93940a5810 con 0x7f939c15ced0 2024-04-25T10:16:59.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f939b7fe640 1 -- 172.21.15.133:0/3133259523 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940a5be0 con 0x7f939c15ced0 2024-04-25T10:16:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 -- 172.21.15.133:0/3133259523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939c15ced0 msgr2=0x7f939c15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 --2- 172.21.15.133:0/3133259523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939c15ced0 0x7f939c15d360 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f9394009e50 tx=0x7f939401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 -- 172.21.15.133:0/3133259523 shutdown_connections 2024-04-25T10:16:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 --2- 172.21.15.133:0/3133259523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939c15ced0 0x7f939c15d360 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 --2- 172.21.15.133:0/3133259523 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f939c0cd820 0x7f939c0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 -- 172.21.15.133:0/3133259523 >> 172.21.15.133:0/3133259523 conn(0x7f939c066f70 msgr2=0x7f939c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 -- 172.21.15.133:0/3133259523 shutdown_connections 2024-04-25T10:16:59.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.644+0000 7f93a2f57640 1 -- 172.21.15.133:0/3133259523 wait complete. 2024-04-25T10:16:59.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a2f57640 1 Processor -- start 2024-04-25T10:16:59.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a2f57640 1 -- start start 2024-04-25T10:16:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a2f57640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939c0cd820 0x7f939c137ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a2f57640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f939c145170 0x7f939c13ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a2f57640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f939c15ced0 0x7f939c135640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a2f57640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c168610 con 0x7f939c0cd820 2024-04-25T10:16:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a2f57640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c168490 con 0x7f939c15ced0 2024-04-25T10:16:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a2f57640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c168790 con 0x7f939c145170 2024-04-25T10:16:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a0cf3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939c0cd820 0x7f939c137ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a0cf3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939c0cd820 0x7f939c137ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44002/0 (socket says 172.21.15.133:44002) 2024-04-25T10:16:59.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a0cf3640 1 -- 172.21.15.133:0/211627242 learned_addr learned my addr 172.21.15.133:0/211627242 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f939bfff640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f939c145170 0x7f939c13ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a14f4640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f939c15ced0 0x7f939c135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a0cf3640 1 -- 172.21.15.133:0/211627242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f939c145170 msgr2=0x7f939c13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a0cf3640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f939c145170 0x7f939c13ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a0cf3640 1 -- 172.21.15.133:0/211627242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f939c15ced0 msgr2=0x7f939c135640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a0cf3640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f939c15ced0 0x7f939c135640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a0cf3640 1 -- 172.21.15.133:0/211627242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c13b0a0 con 0x7f939c0cd820 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a0cf3640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939c0cd820 0x7f939c137ea0 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f938c00c480 tx=0x7f938c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93997fa640 1 -- 172.21.15.133:0/211627242 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c012830 con 0x7f939c0cd820 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93997fa640 1 -- 172.21.15.133:0/211627242 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c017070 con 0x7f939c0cd820 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a2f57640 1 -- 172.21.15.133:0/211627242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c13db60 con 0x7f939c0cd820 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93997fa640 1 -- 172.21.15.133:0/211627242 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c012c30 con 0x7f939c0cd820 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a2f57640 1 -- 172.21.15.133:0/211627242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c13e120 con 0x7f939c0cd820 2024-04-25T10:16:59.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93a2f57640 1 -- 172.21.15.133:0/211627242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939c068e80 con 0x7f939c0cd820 2024-04-25T10:16:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.648+0000 7f93997fa640 1 -- 172.21.15.133:0/211627242 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f938c01c460 con 0x7f939c0cd820 2024-04-25T10:16:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.652+0000 7f93997fa640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9388066f60 0x7f9388069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.652+0000 7f93997fa640 1 -- 172.21.15.133:0/211627242 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f938c084c70 con 0x7f939c0cd820 2024-04-25T10:16:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.652+0000 7f939bfff640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9388066f60 0x7f9388069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.652+0000 7f939bfff640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9388066f60 0x7f9388069420 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f939c15fde0 tx=0x7f9390008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.656+0000 7f93997fa640 1 -- 172.21.15.133:0/211627242 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f938c055e80 con 0x7f939c0cd820 2024-04-25T10:16:59.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.808+0000 7f93a2f57640 1 -- 172.21.15.133:0/211627242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f939c13b360 con 0x7f939c0cd820 2024-04-25T10:16:59.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93997fa640 1 -- 172.21.15.133:0/211627242 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v174) v1 ==== 93+0+10 (secure 0 0 0) 0x7f938c055780 con 0x7f939c0cd820 2024-04-25T10:16:59.812 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:16:59.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 -- 172.21.15.133:0/211627242 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9388066f60 msgr2=0x7f9388069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9388066f60 0x7f9388069420 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f939c15fde0 tx=0x7f9390008040 comp rx=0 tx=0).stop 2024-04-25T10:16:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 -- 172.21.15.133:0/211627242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939c0cd820 msgr2=0x7f939c137ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939c0cd820 0x7f939c137ea0 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f938c00c480 tx=0x7f938c00c940 comp rx=0 tx=0).stop 2024-04-25T10:16:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 -- 172.21.15.133:0/211627242 shutdown_connections 2024-04-25T10:16:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f939c15ced0 0x7f939c135640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f939c145170 0x7f939c13ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9388066f60 0x7f9388069420 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 --2- 172.21.15.133:0/211627242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f939c0cd820 0x7f939c137ea0 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 -- 172.21.15.133:0/211627242 >> 172.21.15.133:0/211627242 conn(0x7f939c066f70 msgr2=0x7f939c144180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 -- 172.21.15.133:0/211627242 shutdown_connections 2024-04-25T10:16:59.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.812+0000 7f93a2f57640 1 -- 172.21.15.133:0/211627242 wait complete. 2024-04-25T10:16:59.830 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:16:59.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5715227640 1 Processor -- start 2024-04-25T10:16:59.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5715227640 1 -- start start 2024-04-25T10:16:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5715227640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571015db00 0x7f57100b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5715227640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57100d0770 0x7f57100d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5715227640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f57101614c0 0x7f5710161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5715227640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100d1110 con 0x7f57100d0770 2024-04-25T10:16:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5715227640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100ceb20 con 0x7f57101614c0 2024-04-25T10:16:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5715227640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57100b7740 con 0x7f571015db00 2024-04-25T10:16:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5707fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57100d0770 0x7f57100d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5707fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57100d0770 0x7f57100d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44006/0 (socket says 172.21.15.133:44006) 2024-04-25T10:16:59.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5707fff640 1 -- 172.21.15.133:0/1184486629 learned_addr learned my addr 172.21.15.133:0/1184486629 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f570ed9d640 1 --2- 172.21.15.133:0/1184486629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571015db00 0x7f57100b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f570f59e640 1 --2- 172.21.15.133:0/1184486629 >> v2:172.21.15.162:3300/0 conn(0x7f57101614c0 0x7f5710161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5707fff640 1 -- 172.21.15.133:0/1184486629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571015db00 msgr2=0x7f57100b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5707fff640 1 --2- 172.21.15.133:0/1184486629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571015db00 0x7f57100b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5707fff640 1 -- 172.21.15.133:0/1184486629 >> v2:172.21.15.162:3300/0 conn(0x7f57101614c0 msgr2=0x7f5710161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5707fff640 1 --2- 172.21.15.133:0/1184486629 >> v2:172.21.15.162:3300/0 conn(0x7f57101614c0 0x7f5710161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.948+0000 7f5707fff640 1 -- 172.21.15.133:0/1184486629 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57100b7a20 con 0x7f57100d0770 2024-04-25T10:16:59.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5707fff640 1 --2- 172.21.15.133:0/1184486629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57100d0770 0x7f57100d0bd0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f571006cf70 tx=0x7f56f801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f125dd10951521c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570e59c640 1 -- 172.21.15.133:0/1184486629 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80aa610 con 0x7f57100d0770 2024-04-25T10:16:59.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570e59c640 1 -- 172.21.15.133:0/1184486629 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f80af070 con 0x7f57100d0770 2024-04-25T10:16:59.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570e59c640 1 -- 172.21.15.133:0/1184486629 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80aa9d0 con 0x7f57100d0770 2024-04-25T10:16:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 -- 172.21.15.133:0/1184486629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57100d0770 msgr2=0x7f57100d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 --2- 172.21.15.133:0/1184486629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57100d0770 0x7f57100d0bd0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f571006cf70 tx=0x7f56f801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 -- 172.21.15.133:0/1184486629 shutdown_connections 2024-04-25T10:16:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 --2- 172.21.15.133:0/1184486629 >> v2:172.21.15.162:3300/0 conn(0x7f57101614c0 0x7f5710161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 --2- 172.21.15.133:0/1184486629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57100d0770 0x7f57100d0bd0 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 --2- 172.21.15.133:0/1184486629 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571015db00 0x7f57100b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 -- 172.21.15.133:0/1184486629 >> 172.21.15.133:0/1184486629 conn(0x7f57100727c0 msgr2=0x7f5710072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 -- 172.21.15.133:0/1184486629 shutdown_connections 2024-04-25T10:16:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 -- 172.21.15.133:0/1184486629 wait complete. 2024-04-25T10:16:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 Processor -- start 2024-04-25T10:16:59.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 -- start start 2024-04-25T10:16:59.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57100d0770 0x7f57101f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571015db00 0x7f571014bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57101614c0 0x7f57101581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100b7430 con 0x7f57101614c0 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100b82c0 con 0x7f57100d0770 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57100d18a0 con 0x7f571015db00 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57101614c0 0x7f57101581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57101614c0 0x7f57101581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44020/0 (socket says 172.21.15.133:44020) 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570f59e640 1 -- 172.21.15.133:0/3516015573 learned_addr learned my addr 172.21.15.133:0/3516015573 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5707fff640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571015db00 0x7f571014bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570ed9d640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57100d0770 0x7f57101f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570f59e640 1 -- 172.21.15.133:0/3516015573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571015db00 msgr2=0x7f571014bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570f59e640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571015db00 0x7f571014bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570f59e640 1 -- 172.21.15.133:0/3516015573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57100d0770 msgr2=0x7f57101f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:59.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570f59e640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57100d0770 0x7f57101f9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570f59e640 1 -- 172.21.15.133:0/3516015573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57101fa4f0 con 0x7f57101614c0 2024-04-25T10:16:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f570f59e640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57101614c0 0x7f57101581c0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f5708007e40 tx=0x7f570800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f57077fe640 1 -- 172.21.15.133:0/3516015573 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708014620 con 0x7f57101614c0 2024-04-25T10:16:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f57077fe640 1 -- 172.21.15.133:0/3516015573 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5708019070 con 0x7f57101614c0 2024-04-25T10:16:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 -- 172.21.15.133:0/3516015573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57101fa7d0 con 0x7f57101614c0 2024-04-25T10:16:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f57077fe640 1 -- 172.21.15.133:0/3516015573 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708014a20 con 0x7f57101614c0 2024-04-25T10:16:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.952+0000 7f5715227640 1 -- 172.21.15.133:0/3516015573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5710152ef0 con 0x7f57101614c0 2024-04-25T10:16:59.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.956+0000 7f5715227640 1 -- 172.21.15.133:0/3516015573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57100c91a0 con 0x7f57101614c0 2024-04-25T10:16:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.956+0000 7f57077fe640 1 -- 172.21.15.133:0/3516015573 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5708014bc0 con 0x7f57101614c0 2024-04-25T10:16:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.956+0000 7f57077fe640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56f0066f60 0x7f56f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.956+0000 7f57077fe640 1 -- 172.21.15.133:0/3516015573 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f57080857b0 con 0x7f57101614c0 2024-04-25T10:16:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.956+0000 7f570ed9d640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56f0066f60 0x7f56f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:59.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.956+0000 7f570ed9d640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56f0066f60 0x7f56f0069420 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5710157e40 tx=0x7f5700008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:59.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:16:59.960+0000 7f57077fe640 1 -- 172.21.15.133:0/3516015573 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5708056920 con 0x7f57101614c0 2024-04-25T10:17:00.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.160+0000 7f5715227640 1 -- 172.21.15.133:0/3516015573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f5710068ef0 con 0x7f57101614c0 2024-04-25T10:17:00.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.196+0000 7f57077fe640 1 -- 172.21.15.133:0/3516015573 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v678) v1 ==== 150+0+0 (secure 0 0 0) 0x7f5708056220 con 0x7f57101614c0 2024-04-25T10:17:00.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 -- 172.21.15.133:0/3516015573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56f0066f60 msgr2=0x7f56f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56f0066f60 0x7f56f0069420 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5710157e40 tx=0x7f5700008040 comp rx=0 tx=0).stop 2024-04-25T10:17:00.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 -- 172.21.15.133:0/3516015573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57101614c0 msgr2=0x7f57101581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57101614c0 0x7f57101581c0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f5708007e40 tx=0x7f570800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 -- 172.21.15.133:0/3516015573 shutdown_connections 2024-04-25T10:17:00.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f56f0066f60 0x7f56f0069420 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57101614c0 0x7f57101581c0 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f571015db00 0x7f571014bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 --2- 172.21.15.133:0/3516015573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f57100d0770 0x7f57101f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 -- 172.21.15.133:0/3516015573 >> 172.21.15.133:0/3516015573 conn(0x7f57100727c0 msgr2=0x7f5710072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:00.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 -- 172.21.15.133:0/3516015573 shutdown_connections 2024-04-25T10:17:00.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.200+0000 7f5715227640 1 -- 172.21.15.133:0/3516015573 wait complete. 2024-04-25T10:17:00.217 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:17:00.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0c005a640 1 Processor -- start 2024-04-25T10:17:00.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0c005a640 1 -- start start 2024-04-25T10:17:00.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0c005a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b815d9a0 0x7ff0b80d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0c005a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff0b80ce600 0x7ff0b80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0c005a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b80d24c0 con 0x7ff0b80d1920 2024-04-25T10:17:00.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0c005a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b80cefa0 con 0x7ff0b80ce600 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0c005a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b815ebd0 con 0x7ff0b815d9a0 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0be5f7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff0b80d1920 0x7ff0b80d1db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38132/0 (socket says 172.21.15.133:38132) 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0be5f7640 1 -- 172.21.15.133:0/1412249452 learned_addr learned my addr 172.21.15.133:0/1412249452 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016534655 0 0) 0x7ff0b80d24c0 con 0x7ff0b80d1920 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0a4003530 con 0x7ff0b80d1920 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bd5f5640 1 --2- 172.21.15.133:0/1412249452 >> v2:172.21.15.162:3300/0 conn(0x7ff0b80ce600 0x7ff0b80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bddf6640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b815d9a0 0x7ff0b80d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3435548123 0 0) 0x7ff0a4003530 con 0x7ff0b80d1920 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b815d9a0 msgr2=0x7ff0b80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b815d9a0 0x7ff0b80d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 >> v2:172.21.15.162:3300/0 conn(0x7ff0b80ce600 msgr2=0x7ff0b80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 --2- 172.21.15.133:0/1412249452 >> v2:172.21.15.162:3300/0 conn(0x7ff0b80ce600 0x7ff0b80cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b815eeb0 con 0x7ff0b80d1920 2024-04-25T10:17:00.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff0b4001680 con 0x7ff0b80d1920 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bd5f5640 1 --2- 172.21.15.133:0/1412249452 >> v2:172.21.15.162:3300/0 conn(0x7ff0b80ce600 0x7ff0b80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 >> v1:172.21.15.133:6789/0 conn(0x7ff0b80d1920 legacy=0x7ff0b80d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a40039c0 0x7ff0a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 0x7ff0a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b815d9a0 0x7ff0a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b815ebd0 con 0x7ff0a4004810 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a400bab0 con 0x7ff0a40039c0 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a400bcb0 con 0x7ff0b815d9a0 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bd5f5640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 0x7ff0a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bddf6640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a40039c0 0x7ff0a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0be5f7640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b815d9a0 0x7ff0a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bd5f5640 1 -- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b815d9a0 msgr2=0x7ff0a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:00.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bd5f5640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b815d9a0 0x7ff0a40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bd5f5640 1 -- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a40039c0 msgr2=0x7ff0a4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:00.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bd5f5640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a40039c0 0x7ff0a4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bd5f5640 1 -- 172.21.15.133:0/1412249452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a4003530 con 0x7ff0a4004810 2024-04-25T10:17:00.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.332+0000 7ff0bddf6640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0a40039c0 0x7ff0a4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:00.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0bd5f5640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 0x7ff0a4004c90 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0098a0 tx=0x7ff0ac01c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a2ff1d5daa4156 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac00f040 con 0x7ff0a4004810 2024-04-25T10:17:00.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac0a8610 con 0x7ff0a4004810 2024-04-25T10:17:00.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0bcdf4640 1 -- 172.21.15.133:0/1412249452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac0a89c0 con 0x7ff0a4004810 2024-04-25T10:17:00.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0c005a640 1 -- 172.21.15.133:0/1412249452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 msgr2=0x7ff0a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0c005a640 1 --2- 172.21.15.133:0/1412249452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 0x7ff0a4004c90 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0098a0 tx=0x7ff0ac01c5c0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0be5f7640 1 -- 172.21.15.133:0/1412249452 reap_dead start 2024-04-25T10:17:00.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0c005a640 1 -- 172.21.15.133:0/1412249452 shutdown_connections 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0c005a640 1 -- 172.21.15.133:0/1412249452 >> 172.21.15.133:0/1412249452 conn(0x7ff0b806bf80 msgr2=0x7ff0b806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0c005a640 1 -- 172.21.15.133:0/1412249452 shutdown_connections 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0c005a640 1 -- 172.21.15.133:0/1412249452 wait complete. 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0c005a640 1 Processor -- start 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.336+0000 7ff0c005a640 1 -- start start 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0c005a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 0x7ff0b815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0c005a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b80b6ec0 0x7ff0b80b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0c005a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0b80cc360 0x7ff0b80cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0c005a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80b9820 con 0x7ff0a4004810 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0c005a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80b96a0 con 0x7ff0b80cc360 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0c005a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80d0f40 con 0x7ff0b80b6ec0 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bddf6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 0x7ff0b815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bddf6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 0x7ff0b815ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44032/0 (socket says 172.21.15.133:44032) 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bddf6640 1 -- 172.21.15.133:0/4099507323 learned_addr learned my addr 172.21.15.133:0/4099507323 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:00.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bd5f5640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b80b6ec0 0x7ff0b80b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bddf6640 1 -- 172.21.15.133:0/4099507323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b80b6ec0 msgr2=0x7ff0b80b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bddf6640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b80b6ec0 0x7ff0b80b7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bddf6640 1 -- 172.21.15.133:0/4099507323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0b80cc360 msgr2=0x7ff0b80cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bddf6640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0b80cc360 0x7ff0b80cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bddf6640 1 -- 172.21.15.133:0/4099507323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b80ccd00 con 0x7ff0a4004810 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bddf6640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 0x7ff0b815ad30 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009fd0 tx=0x7ff0b000f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0aa7fc640 1 -- 172.21.15.133:0/4099507323 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0018020 con 0x7ff0a4004810 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0aa7fc640 1 -- 172.21.15.133:0/4099507323 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b0014610 con 0x7ff0a4004810 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0c005a640 1 -- 172.21.15.133:0/4099507323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b81f6640 con 0x7ff0a4004810 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0aa7fc640 1 -- 172.21.15.133:0/4099507323 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0014a00 con 0x7ff0a4004810 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0c005a640 1 -- 172.21.15.133:0/4099507323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b81f6c00 con 0x7ff0a4004810 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0c005a640 1 -- 172.21.15.133:0/4099507323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b8068e80 con 0x7ff0a4004810 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0aa7fc640 1 -- 172.21.15.133:0/4099507323 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0b001b020 con 0x7ff0a4004810 2024-04-25T10:17:00.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0aa7fc640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff09c066f90 0x7ff09c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0aa7fc640 1 -- 172.21.15.133:0/4099507323 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff0b0085a60 con 0x7ff0a4004810 2024-04-25T10:17:00.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bd5f5640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff09c066f90 0x7ff09c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.340+0000 7ff0bd5f5640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff09c066f90 0x7ff09c069450 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0af000 tx=0x7ff0ac0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.344+0000 7ff0aa7fc640 1 -- 172.21.15.133:0/4099507323 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0b0022020 con 0x7ff0a4004810 2024-04-25T10:17:00.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.492+0000 7ff0c005a640 1 -- 172.21.15.133:0/4099507323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff0b81f6350 con 0x7ff0a4004810 2024-04-25T10:17:00.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.496+0000 7ff0aa7fc640 1 -- 172.21.15.133:0/4099507323 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff0b0056b50 con 0x7ff0a4004810 2024-04-25T10:17:00.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 -- 172.21.15.133:0/4099507323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff09c066f90 msgr2=0x7ff09c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff09c066f90 0x7ff09c069450 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0af000 tx=0x7ff0ac0046d0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 -- 172.21.15.133:0/4099507323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 msgr2=0x7ff0b815ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 0x7ff0b815ad30 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009fd0 tx=0x7ff0b000f5e0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 -- 172.21.15.133:0/4099507323 shutdown_connections 2024-04-25T10:17:00.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0b80cc360 0x7ff0b80cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0b80b6ec0 0x7ff0b80b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff09c066f90 0x7ff09c069450 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 --2- 172.21.15.133:0/4099507323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0a4004810 0x7ff0b815ad30 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 -- 172.21.15.133:0/4099507323 >> 172.21.15.133:0/4099507323 conn(0x7ff0b806bf80 msgr2=0x7ff0b81603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:00.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 -- 172.21.15.133:0/4099507323 shutdown_connections 2024-04-25T10:17:00.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.500+0000 7ff0c005a640 1 -- 172.21.15.133:0/4099507323 wait complete. 2024-04-25T10:17:00.520 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:17:00.520 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:00.554 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:00.612 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:17:00.612 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:17:00.612 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:00.618 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:00.618 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:17:00.618 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:00.624 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:00.624 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:17:00.624 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:00.665 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:00.665 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:17:00.665 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:00.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:00.671 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:17:00.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:00.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafd5437640 1 Processor -- start 2024-04-25T10:17:00.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafd5437640 1 -- start start 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafd5437640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd015d9b0 0x7fafd00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafd5437640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fafd015dd90 0x7fafd00ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafd5437640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fafd00b19f0 0x7fafd00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafd5437640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd00cbe30 con 0x7fafd015dd90 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafd5437640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd00ca9b0 con 0x7fafd00b19f0 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafd5437640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00cac90 con 0x7fafd015d9b0 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafce7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fafd015dd90 0x7fafd00ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafce7fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fafd015dd90 0x7fafd00ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44040/0 (socket says 172.21.15.133:44040) 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafce7fc640 1 -- 172.21.15.133:0/2524696489 learned_addr learned my addr 172.21.15.133:0/2524696489 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafceffd640 1 --2- 172.21.15.133:0/2524696489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd015d9b0 0x7fafd00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafcf7fe640 1 --2- 172.21.15.133:0/2524696489 >> v2:172.21.15.162:3300/0 conn(0x7fafd00b19f0 0x7fafd00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafce7fc640 1 -- 172.21.15.133:0/2524696489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd015d9b0 msgr2=0x7fafd00cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafce7fc640 1 --2- 172.21.15.133:0/2524696489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd015d9b0 0x7fafd00cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafce7fc640 1 -- 172.21.15.133:0/2524696489 >> v2:172.21.15.162:3300/0 conn(0x7fafd00b19f0 msgr2=0x7fafd00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafce7fc640 1 --2- 172.21.15.133:0/2524696489 >> v2:172.21.15.162:3300/0 conn(0x7fafd00b19f0 0x7fafd00cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.828+0000 7fafce7fc640 1 -- 172.21.15.133:0/2524696489 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd00caf70 con 0x7fafd015dd90 2024-04-25T10:17:00.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafce7fc640 1 --2- 172.21.15.133:0/2524696489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd015dd90 0x7fafd00ce770 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fafd006b4e0 tx=0x7fafc001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39e701abf2da25d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafcdffb640 1 -- 172.21.15.133:0/2524696489 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00ab610 con 0x7fafd015dd90 2024-04-25T10:17:00.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafcdffb640 1 -- 172.21.15.133:0/2524696489 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc00b0070 con 0x7fafd015dd90 2024-04-25T10:17:00.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafcdffb640 1 -- 172.21.15.133:0/2524696489 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00ab9d0 con 0x7fafd015dd90 2024-04-25T10:17:00.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 -- 172.21.15.133:0/2524696489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd015dd90 msgr2=0x7fafd00ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 --2- 172.21.15.133:0/2524696489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd015dd90 0x7fafd00ce770 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fafd006b4e0 tx=0x7fafc001c430 comp rx=0 tx=0).stop 2024-04-25T10:17:00.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 -- 172.21.15.133:0/2524696489 shutdown_connections 2024-04-25T10:17:00.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 --2- 172.21.15.133:0/2524696489 >> v2:172.21.15.162:3300/0 conn(0x7fafd00b19f0 0x7fafd00cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 --2- 172.21.15.133:0/2524696489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd015dd90 0x7fafd00ce770 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 --2- 172.21.15.133:0/2524696489 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd015d9b0 0x7fafd00cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 -- 172.21.15.133:0/2524696489 >> 172.21.15.133:0/2524696489 conn(0x7fafd0067000 msgr2=0x7fafd006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 -- 172.21.15.133:0/2524696489 shutdown_connections 2024-04-25T10:17:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 -- 172.21.15.133:0/2524696489 wait complete. 2024-04-25T10:17:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 Processor -- start 2024-04-25T10:17:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 -- start start 2024-04-25T10:17:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd00b19f0 0x7fafd013fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd015d9b0 0x7fafd0150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafd015dd90 0x7fafd0151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd0162290 con 0x7fafd015d9b0 2024-04-25T10:17:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd0162110 con 0x7fafd015dd90 2024-04-25T10:17:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafd5437640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd0162410 con 0x7fafd00b19f0 2024-04-25T10:17:00.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafce7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd015d9b0 0x7fafd0150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafce7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd015d9b0 0x7fafd0150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44050/0 (socket says 172.21.15.133:44050) 2024-04-25T10:17:00.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafce7fc640 1 -- 172.21.15.133:0/3086019154 learned_addr learned my addr 172.21.15.133:0/3086019154 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:00.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafceffd640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd00b19f0 0x7fafd013fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafcf7fe640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafd015dd90 0x7fafd0151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafce7fc640 1 -- 172.21.15.133:0/3086019154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd00b19f0 msgr2=0x7fafd013fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafce7fc640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd00b19f0 0x7fafd013fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafce7fc640 1 -- 172.21.15.133:0/3086019154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafd015dd90 msgr2=0x7fafd0151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:00.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafce7fc640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafd015dd90 0x7fafd0151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:00.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafce7fc640 1 -- 172.21.15.133:0/3086019154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd015e4e0 con 0x7fafd015d9b0 2024-04-25T10:17:00.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafcf7fe640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafd015dd90 0x7fafd0151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:00.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.832+0000 7fafceffd640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd00b19f0 0x7fafd013fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:00.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafce7fc640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd015d9b0 0x7fafd0150fd0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7fafc00ad040 tx=0x7fafc001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafc77fe640 1 -- 172.21.15.133:0/3086019154 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00b4800 con 0x7fafd015d9b0 2024-04-25T10:17:00.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafc77fe640 1 -- 172.21.15.133:0/3086019154 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc00b49a0 con 0x7fafd015d9b0 2024-04-25T10:17:00.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafc77fe640 1 -- 172.21.15.133:0/3086019154 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00b4d90 con 0x7fafd015d9b0 2024-04-25T10:17:00.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafd5437640 1 -- 172.21.15.133:0/3086019154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd015e7c0 con 0x7fafd015d9b0 2024-04-25T10:17:00.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafd5437640 1 -- 172.21.15.133:0/3086019154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd015ed80 con 0x7fafd015d9b0 2024-04-25T10:17:00.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafd5437640 1 -- 172.21.15.133:0/3086019154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd0068ef0 con 0x7fafd015d9b0 2024-04-25T10:17:00.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafc77fe640 1 -- 172.21.15.133:0/3086019154 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafc00c8420 con 0x7fafd015d9b0 2024-04-25T10:17:00.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafc77fe640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafb0066f60 0x7fafb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:00.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafc77fe640 1 -- 172.21.15.133:0/3086019154 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fafc011f930 con 0x7fafd015d9b0 2024-04-25T10:17:00.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafceffd640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafb0066f60 0x7fafb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:00.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.836+0000 7fafceffd640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafb0066f60 0x7fafb0069420 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fafd0156b90 tx=0x7fafb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:00.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:00.840+0000 7fafc77fe640 1 -- 172.21.15.133:0/3086019154 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fafc00c4060 con 0x7fafd015d9b0 2024-04-25T10:17:01.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.016+0000 7fafd5437640 1 -- 172.21.15.133:0/3086019154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafd00c9d70 con 0x7fafd015d9b0 2024-04-25T10:17:01.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.016+0000 7fafc77fe640 1 -- 172.21.15.133:0/3086019154 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+3715 (secure 0 0 0) 0x7fafc00a8050 con 0x7fafd015d9b0 2024-04-25T10:17:01.020 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:01.020 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":107,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6899,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1498957356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1498957356},{"type":"v1","addr":"172.21.15.162:6835","nonce":1498957356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7867,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/3188103229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":3188103229},{"type":"v1","addr":"172.21.15.162:6833","nonce":3188103229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":11022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/883424012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":883424012},{"type":"v1","addr":"172.21.15.133:6835","nonce":883424012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11031,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2040533186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2040533186},{"type":"v1","addr":"172.21.15.133:6837","nonce":2040533186}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[]} 2024-04-25T10:17:01.020 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 107 2024-04-25T10:17:01.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 -- 172.21.15.133:0/3086019154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafb0066f60 msgr2=0x7fafb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafb0066f60 0x7fafb0069420 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fafd0156b90 tx=0x7fafb8008040 comp rx=0 tx=0).stop 2024-04-25T10:17:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 -- 172.21.15.133:0/3086019154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd015d9b0 msgr2=0x7fafd0150fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd015d9b0 0x7fafd0150fd0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7fafc00ad040 tx=0x7fafc001acd0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 -- 172.21.15.133:0/3086019154 shutdown_connections 2024-04-25T10:17:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fafd015dd90 0x7fafd0151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fafb0066f60 0x7fafb0069420 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafd015d9b0 0x7fafd0150fd0 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 --2- 172.21.15.133:0/3086019154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fafd00b19f0 0x7fafd013fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 -- 172.21.15.133:0/3086019154 >> 172.21.15.133:0/3086019154 conn(0x7fafd0067000 msgr2=0x7fafd00b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 -- 172.21.15.133:0/3086019154 shutdown_connections 2024-04-25T10:17:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.020+0000 7fafd5437640 1 -- 172.21.15.133:0/3086019154 wait complete. 2024-04-25T10:17:01.038 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:17:01.039 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:17:01.039 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:17:01.039 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:01.039 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:17:01.039 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:17:01.039 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:17:01.039 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:01.040 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:17:01.040 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:17:01.040 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:17:01.040 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:01.040 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:17:01.040 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:17:01.040 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:17:01.041 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:01.048 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:17:01.049 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:17:01.051 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:17:01.051 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:17:01.053 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:17:01.053 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:17:01.055 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:17:01.055 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:17:01.093 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:17:01.094 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:01.094 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:17:01.095 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:01.099 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:17:01.100 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:01.101 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:17:01.101 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:01.101 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:17:01.154 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:17:01.157 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:17:01.169 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:17:01.170 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:17:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.208+0000 7f5d5a0c5640 1 Processor -- start 2024-04-25T10:17:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.208+0000 7f5d5a0c5640 1 -- start start 2024-04-25T10:17:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.208+0000 7f5d5a0c5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5d5415df00 0x7f5d5406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.208+0000 7f5d5a0c5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d540cfaf0 con 0x7f5d5409a710 2024-04-25T10:17:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.208+0000 7f5d5a0c5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d540cfdd0 con 0x7f5d5415db20 2024-04-25T10:17:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.208+0000 7f5d5a0c5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540d00b0 con 0x7f5d5415df00 2024-04-25T10:17:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d53fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5d5409a710 0x7f5d540999d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38152/0 (socket says 172.21.15.133:38152) 2024-04-25T10:17:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d53fff640 1 -- 172.21.15.133:0/2161495265 learned_addr learned my addr 172.21.15.133:0/2161495265 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:01.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d52ffd640 1 --2- 172.21.15.133:0/2161495265 >> v2:172.21.15.162:3300/0 conn(0x7f5d5415df00 0x7f5d5406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d52ffd640 1 -- 172.21.15.133:0/2161495265 >> v1:172.21.15.162:6789/0 conn(0x7f5d5415db20 legacy=0x7f5d540d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d52ffd640 1 -- 172.21.15.133:0/2161495265 >> v1:172.21.15.133:6789/0 conn(0x7f5d5409a710 legacy=0x7f5d540999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d52ffd640 1 -- 172.21.15.133:0/2161495265 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540d0390 con 0x7f5d5415df00 2024-04-25T10:17:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d52ffd640 1 --2- 172.21.15.133:0/2161495265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5415df00 0x7f5d5406c9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0097c0 tx=0x7f5d3c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbbe2a2a148ebd8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d527fc640 1 -- 172.21.15.133:0/2161495265 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0b0070 con 0x7f5d5415df00 2024-04-25T10:17:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d527fc640 1 -- 172.21.15.133:0/2161495265 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c0ab600 con 0x7f5d5415df00 2024-04-25T10:17:01.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d527fc640 1 -- 172.21.15.133:0/2161495265 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0ab9d0 con 0x7f5d5415df00 2024-04-25T10:17:01.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/2161495265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5415df00 msgr2=0x7f5d5406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.212+0000 7f5d5a0c5640 1 --2- 172.21.15.133:0/2161495265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5415df00 0x7f5d5406c9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0097c0 tx=0x7f5d3c01c410 comp rx=0 tx=0).stop 2024-04-25T10:17:01.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/2161495265 shutdown_connections 2024-04-25T10:17:01.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 --2- 172.21.15.133:0/2161495265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5415df00 0x7f5d5406c9c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/2161495265 >> 172.21.15.133:0/2161495265 conn(0x7f5d540b3170 msgr2=0x7f5d540b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:01.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/2161495265 shutdown_connections 2024-04-25T10:17:01.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/2161495265 wait complete. 2024-04-25T10:17:01.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 Processor -- start 2024-04-25T10:17:01.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 -- start start 2024-04-25T10:17:01.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5409a710 0x7f5d54159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5415db20 0x7f5d540c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5415df00 0x7f5d540c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540cc4e0 con 0x7f5d5409a710 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5409a330 con 0x7f5d5415df00 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540cc660 con 0x7f5d5415db20 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d537fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5409a710 0x7f5d54159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d537fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5409a710 0x7f5d54159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44066/0 (socket says 172.21.15.133:44066) 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d537fe640 1 -- 172.21.15.133:0/3483547225 learned_addr learned my addr 172.21.15.133:0/3483547225 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d52ffd640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5415db20 0x7f5d540c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d53fff640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5415df00 0x7f5d540c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d537fe640 1 -- 172.21.15.133:0/3483547225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5415db20 msgr2=0x7f5d540c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d537fe640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5415db20 0x7f5d540c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d537fe640 1 -- 172.21.15.133:0/3483547225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5415df00 msgr2=0x7f5d540c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d537fe640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5415df00 0x7f5d540c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:01.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d537fe640 1 -- 172.21.15.133:0/3483547225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540c7790 con 0x7f5d5409a710 2024-04-25T10:17:01.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d537fe640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5409a710 0x7f5d54159f70 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400c3a0 tx=0x7f5d4400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d4bfff640 1 -- 172.21.15.133:0/3483547225 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d440052d0 con 0x7f5d5409a710 2024-04-25T10:17:01.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d4bfff640 1 -- 172.21.15.133:0/3483547225 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d44005470 con 0x7f5d5409a710 2024-04-25T10:17:01.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/3483547225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d540c7a30 con 0x7f5d5409a710 2024-04-25T10:17:01.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d4bfff640 1 -- 172.21.15.133:0/3483547225 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44007260 con 0x7f5d5409a710 2024-04-25T10:17:01.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/3483547225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d540c7ff0 con 0x7f5d5409a710 2024-04-25T10:17:01.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/3483547225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d540c8230 con 0x7f5d5409a710 2024-04-25T10:17:01.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.220+0000 7f5d4bfff640 1 -- 172.21.15.133:0/3483547225 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d44007400 con 0x7f5d5409a710 2024-04-25T10:17:01.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.220+0000 7f5d4bfff640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d34066f60 0x7f5d34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:01.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.220+0000 7f5d4bfff640 1 -- 172.21.15.133:0/3483547225 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5d44084210 con 0x7f5d5409a710 2024-04-25T10:17:01.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.220+0000 7f5d52ffd640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d34066f60 0x7f5d34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:01.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.220+0000 7f5d52ffd640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d34066f60 0x7f5d34069420 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0ad000 tx=0x7f5d3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:01.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.224+0000 7f5d4bfff640 1 -- 172.21.15.133:0/3483547225 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d540c8230 con 0x7f5d5409a710 2024-04-25T10:17:01.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:01.392+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/3483547225 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5d54073dd0 con 0x7f5d5409a710 2024-04-25T10:17:02.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.212+0000 7f5d4bfff640 1 -- 172.21.15.133:0/3483547225 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v179) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5d44054dd0 con 0x7f5d5409a710 2024-04-25T10:17:02.214 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:17:02.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/3483547225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d34066f60 msgr2=0x7f5d34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d34066f60 0x7f5d34069420 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0ad000 tx=0x7f5d3c008040 comp rx=0 tx=0).stop 2024-04-25T10:17:02.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/3483547225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5409a710 msgr2=0x7f5d54159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5409a710 0x7f5d54159f70 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400c3a0 tx=0x7f5d4400c860 comp rx=0 tx=0).stop 2024-04-25T10:17:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/3483547225 shutdown_connections 2024-04-25T10:17:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5d5415df00 0x7f5d540c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5d5415db20 0x7f5d540c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5d34066f60 0x7f5d34069420 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 --2- 172.21.15.133:0/3483547225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d5409a710 0x7f5d54159f70 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/3483547225 >> 172.21.15.133:0/3483547225 conn(0x7f5d540b3170 msgr2=0x7f5d540b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/3483547225 shutdown_connections 2024-04-25T10:17:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.216+0000 7f5d5a0c5640 1 -- 172.21.15.133:0/3483547225 wait complete. 2024-04-25T10:17:02.237 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:17:02.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f904b4c9640 1 Processor -- start 2024-04-25T10:17:02.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f904b4c9640 1 -- start start 2024-04-25T10:17:02.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f904b4c9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f904415c9a0 0x7f90440c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f904b4c9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f90440b1f70 0x7f904406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f904b4c9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90440c61d0 con 0x7f90440ce330 2024-04-25T10:17:02.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f904b4c9640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90440c64b0 con 0x7f904415c9a0 2024-04-25T10:17:02.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f904b4c9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904406d4f0 con 0x7f90440b1f70 2024-04-25T10:17:02.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f9049265640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f904415c9a0 0x7f90440c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f9049265640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f904415c9a0 0x7f90440c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44068/0 (socket says 172.21.15.133:44068) 2024-04-25T10:17:02.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f9049265640 1 -- 172.21.15.133:0/2962872550 learned_addr learned my addr 172.21.15.133:0/2962872550 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:02.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f9049265640 1 -- 172.21.15.133:0/2962872550 >> v2:172.21.15.162:3300/0 conn(0x7f90440b1f70 msgr2=0x7f904406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:02.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f9048a64640 1 --2- 172.21.15.133:0/2962872550 >> v2:172.21.15.162:3300/0 conn(0x7f90440b1f70 0x7f904406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f9049265640 1 --2- 172.21.15.133:0/2962872550 >> v2:172.21.15.162:3300/0 conn(0x7f90440b1f70 0x7f904406dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f9049265640 1 -- 172.21.15.133:0/2962872550 >> v1:172.21.15.162:6789/0 conn(0x7f90440ce330 legacy=0x7f904415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:02.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f9049265640 1 -- 172.21.15.133:0/2962872550 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904406d7d0 con 0x7f904415c9a0 2024-04-25T10:17:02.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f9049265640 1 --2- 172.21.15.133:0/2962872550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f904415c9a0 0x7f90440c5980 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f9044072670 tx=0x7f904001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a7105c8bac62e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.356+0000 7f903bfff640 1 -- 172.21.15.133:0/2962872550 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400b2070 con 0x7f904415c9a0 2024-04-25T10:17:02.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f903bfff640 1 -- 172.21.15.133:0/2962872550 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90400ad640 con 0x7f904415c9a0 2024-04-25T10:17:02.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f903bfff640 1 -- 172.21.15.133:0/2962872550 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400ada10 con 0x7f904415c9a0 2024-04-25T10:17:02.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 -- 172.21.15.133:0/2962872550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f904415c9a0 msgr2=0x7f90440c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 --2- 172.21.15.133:0/2962872550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f904415c9a0 0x7f90440c5980 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f9044072670 tx=0x7f904001c480 comp rx=0 tx=0).stop 2024-04-25T10:17:02.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 -- 172.21.15.133:0/2962872550 shutdown_connections 2024-04-25T10:17:02.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 --2- 172.21.15.133:0/2962872550 >> v2:172.21.15.162:3300/0 conn(0x7f90440b1f70 0x7f904406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 --2- 172.21.15.133:0/2962872550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f904415c9a0 0x7f90440c5980 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 -- 172.21.15.133:0/2962872550 >> 172.21.15.133:0/2962872550 conn(0x7f9044067000 msgr2=0x7f904406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:02.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 -- 172.21.15.133:0/2962872550 shutdown_connections 2024-04-25T10:17:02.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 -- 172.21.15.133:0/2962872550 wait complete. 2024-04-25T10:17:02.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 Processor -- start 2024-04-25T10:17:02.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 -- start start 2024-04-25T10:17:02.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90440b1f70 0x7f904413bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90440ce330 0x7f90440cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f904415c9a0 0x7f90440cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904415cd80 con 0x7f90440ce330 2024-04-25T10:17:02.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904406e230 con 0x7f904415c9a0 2024-04-25T10:17:02.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904415cf00 con 0x7f90440b1f70 2024-04-25T10:17:02.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9048a64640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90440ce330 0x7f90440cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9048a64640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90440ce330 0x7f90440cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44078/0 (socket says 172.21.15.133:44078) 2024-04-25T10:17:02.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9048a64640 1 -- 172.21.15.133:0/1022916128 learned_addr learned my addr 172.21.15.133:0/1022916128 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9049265640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90440b1f70 0x7f904413bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9048a64640 1 -- 172.21.15.133:0/1022916128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90440b1f70 msgr2=0x7f904413bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9048a64640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90440b1f70 0x7f904413bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9049a66640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f904415c9a0 0x7f90440cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9048a64640 1 -- 172.21.15.133:0/1022916128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f904415c9a0 msgr2=0x7f90440cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9048a64640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f904415c9a0 0x7f90440cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9048a64640 1 -- 172.21.15.133:0/1022916128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440cd2d0 con 0x7f90440ce330 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9049265640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90440b1f70 0x7f904413bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9048a64640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90440ce330 0x7f90440cc810 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f903400c7f0 tx=0x7f903400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9039ffb640 1 -- 172.21.15.133:0/1022916128 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034015070 con 0x7f90440ce330 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9039ffb640 1 -- 172.21.15.133:0/1022916128 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9034007820 con 0x7f90440ce330 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 -- 172.21.15.133:0/1022916128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90440cd5b0 con 0x7f90440ce330 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f9039ffb640 1 -- 172.21.15.133:0/1022916128 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034007c10 con 0x7f90440ce330 2024-04-25T10:17:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.360+0000 7f904b4c9640 1 -- 172.21.15.133:0/1022916128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90440cde50 con 0x7f90440ce330 2024-04-25T10:17:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.364+0000 7f904b4c9640 1 -- 172.21.15.133:0/1022916128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904406bc20 con 0x7f90440ce330 2024-04-25T10:17:02.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.364+0000 7f9039ffb640 1 -- 172.21.15.133:0/1022916128 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9034018070 con 0x7f90440ce330 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.364+0000 7f9039ffb640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9028066f90 0x7f9028069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.364+0000 7f9039ffb640 1 -- 172.21.15.133:0/1022916128 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9034084f60 con 0x7f90440ce330 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.364+0000 7f9049265640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9028066f90 0x7f9028069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.364+0000 7f9049265640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9028066f90 0x7f9028069450 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f90400af000 tx=0x7f9040008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.368+0000 7f9039ffb640 1 -- 172.21.15.133:0/1022916128 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9034056090 con 0x7f90440ce330 2024-04-25T10:17:02.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.580+0000 7f904b4c9640 1 -- 172.21.15.133:0/1022916128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f90440cd780 con 0x7f90440ce330 2024-04-25T10:17:02.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.580+0000 7f9039ffb640 1 -- 172.21.15.133:0/1022916128 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9034055990 con 0x7f90440ce330 2024-04-25T10:17:02.584 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:02.584 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:17:02.584 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:17:02.584 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.584 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:17:02.585 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.587 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:17:02.588 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:17:02.590 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:02.592 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.593 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:17:02.594 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:17:02.594 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.594 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:02.594 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:17:02.594 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:17:02.594 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:17:02.594 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.594 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.594 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.594 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:17:02.594 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:17:02.595 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:02.597 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:02.597 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:17:02.597 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:17:02.597 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:02.597 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:17:02.597 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:17:02.597 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:02.597 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.597 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:02.597 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:17:02.597 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:17:02.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 -- 172.21.15.133:0/1022916128 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9028066f90 msgr2=0x7f9028069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9028066f90 0x7f9028069450 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f90400af000 tx=0x7f9040008040 comp rx=0 tx=0).stop 2024-04-25T10:17:02.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 -- 172.21.15.133:0/1022916128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90440ce330 msgr2=0x7f90440cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90440ce330 0x7f90440cc810 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f903400c7f0 tx=0x7f903400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 -- 172.21.15.133:0/1022916128 shutdown_connections 2024-04-25T10:17:02.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f904415c9a0 0x7f90440cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9028066f90 0x7f9028069450 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90440ce330 0x7f90440cc810 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 --2- 172.21.15.133:0/1022916128 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f90440b1f70 0x7f904413bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 -- 172.21.15.133:0/1022916128 >> 172.21.15.133:0/1022916128 conn(0x7f9044067000 msgr2=0x7f90441568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:02.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 -- 172.21.15.133:0/1022916128 shutdown_connections 2024-04-25T10:17:02.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.584+0000 7f904b4c9640 1 -- 172.21.15.133:0/1022916128 wait complete. 2024-04-25T10:17:02.602 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:02.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f2885456640 1 Processor -- start 2024-04-25T10:17:02.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f2885456640 1 -- start start 2024-04-25T10:17:02.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f2885456640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2880158110 0x7f28800b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f2885456640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f288006eb70 0x7f288015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f2885456640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28800cbe60 con 0x7f28801584f0 2024-04-25T10:17:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f2885456640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28800b19b0 con 0x7f2880158110 2024-04-25T10:17:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f2885456640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288015b360 con 0x7f288006eb70 2024-04-25T10:17:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f287effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2880158110 0x7f28800b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f287effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2880158110 0x7f28800b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44090/0 (socket says 172.21.15.133:44090) 2024-04-25T10:17:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f287effd640 1 -- 172.21.15.133:0/412440757 learned_addr learned my addr 172.21.15.133:0/412440757 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f287f7fe640 1 --2- 172.21.15.133:0/412440757 >> v2:172.21.15.162:3300/0 conn(0x7f288006eb70 0x7f288015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f287effd640 1 -- 172.21.15.133:0/412440757 >> v2:172.21.15.162:3300/0 conn(0x7f288006eb70 msgr2=0x7f288015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f287effd640 1 --2- 172.21.15.133:0/412440757 >> v2:172.21.15.162:3300/0 conn(0x7f288006eb70 0x7f288015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f287effd640 1 -- 172.21.15.133:0/412440757 >> v1:172.21.15.162:6789/0 conn(0x7f28801584f0 legacy=0x7f288006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.724+0000 7f287effd640 1 -- 172.21.15.133:0/412440757 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288015b640 con 0x7f2880158110 2024-04-25T10:17:02.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f287effd640 1 --2- 172.21.15.133:0/412440757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2880158110 0x7f28800b1f00 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f28800b2470 tx=0x7f287001ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbe529d334bdfc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f287dffb640 1 -- 172.21.15.133:0/412440757 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700b4070 con 0x7f2880158110 2024-04-25T10:17:02.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f287dffb640 1 -- 172.21.15.133:0/412440757 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28700a7bb0 con 0x7f2880158110 2024-04-25T10:17:02.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f287dffb640 1 -- 172.21.15.133:0/412440757 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700af440 con 0x7f2880158110 2024-04-25T10:17:02.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 -- 172.21.15.133:0/412440757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2880158110 msgr2=0x7f28800b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 --2- 172.21.15.133:0/412440757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2880158110 0x7f28800b1f00 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f28800b2470 tx=0x7f287001ebc0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 -- 172.21.15.133:0/412440757 shutdown_connections 2024-04-25T10:17:02.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 --2- 172.21.15.133:0/412440757 >> v2:172.21.15.162:3300/0 conn(0x7f288006eb70 0x7f288015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 --2- 172.21.15.133:0/412440757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2880158110 0x7f28800b1f00 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 -- 172.21.15.133:0/412440757 >> 172.21.15.133:0/412440757 conn(0x7f2880067000 msgr2=0x7f288006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:02.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 -- 172.21.15.133:0/412440757 shutdown_connections 2024-04-25T10:17:02.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 -- 172.21.15.133:0/412440757 wait complete. 2024-04-25T10:17:02.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 Processor -- start 2024-04-25T10:17:02.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 -- start start 2024-04-25T10:17:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f288006eb70 0x7f288014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2880158110 0x7f2880163800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28801584f0 0x7f28800b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288006dce0 con 0x7f288006eb70 2024-04-25T10:17:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288006db60 con 0x7f28801584f0 2024-04-25T10:17:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f2885456640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288006de60 con 0x7f2880158110 2024-04-25T10:17:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f287effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f288006eb70 0x7f288014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f287effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f288006eb70 0x7f288014ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44100/0 (socket says 172.21.15.133:44100) 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.728+0000 7f287effd640 1 -- 172.21.15.133:0/1075618501 learned_addr learned my addr 172.21.15.133:0/1075618501 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f287e7fc640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2880158110 0x7f2880163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f287effd640 1 -- 172.21.15.133:0/1075618501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2880158110 msgr2=0x7f2880163800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f287effd640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2880158110 0x7f2880163800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f287effd640 1 -- 172.21.15.133:0/1075618501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28801584f0 msgr2=0x7f28800b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f287f7fe640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28801584f0 0x7f28800b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f287effd640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28801584f0 0x7f28800b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f287effd640 1 -- 172.21.15.133:0/1075618501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2880164280 con 0x7f288006eb70 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f287effd640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f288006eb70 0x7f288014ec10 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f287001c7f0 tx=0x7f287001b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f28677fe640 1 -- 172.21.15.133:0/1075618501 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700b4030 con 0x7f288006eb70 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f28677fe640 1 -- 172.21.15.133:0/1075618501 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28700b6070 con 0x7f288006eb70 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f2885456640 1 -- 172.21.15.133:0/1075618501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2880164560 con 0x7f288006eb70 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f28677fe640 1 -- 172.21.15.133:0/1075618501 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700ad550 con 0x7f288006eb70 2024-04-25T10:17:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f2885456640 1 -- 172.21.15.133:0/1075618501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2880164b20 con 0x7f288006eb70 2024-04-25T10:17:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f2885456640 1 -- 172.21.15.133:0/1075618501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288006bc20 con 0x7f288006eb70 2024-04-25T10:17:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f28677fe640 1 -- 172.21.15.133:0/1075618501 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28700c1060 con 0x7f288006eb70 2024-04-25T10:17:02.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f28677fe640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f286c066f90 0x7f286c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f28677fe640 1 -- 172.21.15.133:0/1075618501 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f287011f2b0 con 0x7f288006eb70 2024-04-25T10:17:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f287e7fc640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f286c066f90 0x7f286c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.732+0000 7f287e7fc640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f286c066f90 0x7f286c069450 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2874007fd0 tx=0x7f2874009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.736+0000 7f28677fe640 1 -- 172.21.15.133:0/1075618501 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28700187e0 con 0x7f288006eb70 2024-04-25T10:17:02.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.896+0000 7f2885456640 1 -- 172.21.15.133:0/1075618501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f288015a130 con 0x7f288006eb70 2024-04-25T10:17:02.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.896+0000 7f28677fe640 1 -- 172.21.15.133:0/1075618501 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+13198 (secure 0 0 0) 0x7f28700efe30 con 0x7f288006eb70 2024-04-25T10:17:02.900 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:02.900 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":179,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:02.213803+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:02.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 -- 172.21.15.133:0/1075618501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f286c066f90 msgr2=0x7f286c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f286c066f90 0x7f286c069450 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2874007fd0 tx=0x7f2874009040 comp rx=0 tx=0).stop 2024-04-25T10:17:02.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 -- 172.21.15.133:0/1075618501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f288006eb70 msgr2=0x7f288014ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:02.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f288006eb70 0x7f288014ec10 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f287001c7f0 tx=0x7f287001b960 comp rx=0 tx=0).stop 2024-04-25T10:17:02.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 -- 172.21.15.133:0/1075618501 shutdown_connections 2024-04-25T10:17:02.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28801584f0 0x7f28800b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2880158110 0x7f2880163800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f286c066f90 0x7f286c069450 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 --2- 172.21.15.133:0/1075618501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f288006eb70 0x7f288014ec10 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:02.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 -- 172.21.15.133:0/1075618501 >> 172.21.15.133:0/1075618501 conn(0x7f2880067000 msgr2=0x7f288006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:02.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 -- 172.21.15.133:0/1075618501 shutdown_connections 2024-04-25T10:17:02.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:02.900+0000 7f2885456640 1 -- 172.21.15.133:0/1075618501 wait complete. 2024-04-25T10:17:02.919 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:02.919 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:03.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb9a875640 1 Processor -- start 2024-04-25T10:17:03.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb9a875640 1 -- start start 2024-04-25T10:17:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb9a875640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb94143ea0 0x7ffb940cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb9a875640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffb940b2150 0x7ffb9406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb9a875640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb9415b9b0 0x7ffb9406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb9a875640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9415ac50 con 0x7ffb9415b9b0 2024-04-25T10:17:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb9a875640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb940cd180 con 0x7ffb940b2150 2024-04-25T10:17:03.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb9a875640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9415aed0 con 0x7ffb94143ea0 2024-04-25T10:17:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb98e12640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb9415b9b0 0x7ffb9406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb98e12640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb9415b9b0 0x7ffb9406e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44108/0 (socket says 172.21.15.133:44108) 2024-04-25T10:17:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb98e12640 1 -- 172.21.15.133:0/2230970262 learned_addr learned my addr 172.21.15.133:0/2230970262 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb8bfff640 1 --2- 172.21.15.133:0/2230970262 >> v2:172.21.15.162:3300/0 conn(0x7ffb940b2150 0x7ffb9406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb98e12640 1 -- 172.21.15.133:0/2230970262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb94143ea0 msgr2=0x7ffb940cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb98e12640 1 --2- 172.21.15.133:0/2230970262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb94143ea0 0x7ffb940cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb98e12640 1 -- 172.21.15.133:0/2230970262 >> v2:172.21.15.162:3300/0 conn(0x7ffb940b2150 msgr2=0x7ffb9406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb98e12640 1 --2- 172.21.15.133:0/2230970262 >> v2:172.21.15.162:3300/0 conn(0x7ffb940b2150 0x7ffb9406e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb98e12640 1 -- 172.21.15.133:0/2230970262 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9415b1b0 con 0x7ffb9415b9b0 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.040+0000 7ffb98e12640 1 --2- 172.21.15.133:0/2230970262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb9415b9b0 0x7ffb9406e8f0 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7ffb9406eff0 tx=0x7ffb8c01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fbfba7029f97a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb937fe640 1 -- 172.21.15.133:0/2230970262 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c00b2d0 con 0x7ffb9415b9b0 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb937fe640 1 -- 172.21.15.133:0/2230970262 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c00b470 con 0x7ffb9415b9b0 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb937fe640 1 -- 172.21.15.133:0/2230970262 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0a7bb0 con 0x7ffb9415b9b0 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 -- 172.21.15.133:0/2230970262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb9415b9b0 msgr2=0x7ffb9406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 --2- 172.21.15.133:0/2230970262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb9415b9b0 0x7ffb9406e8f0 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7ffb9406eff0 tx=0x7ffb8c01ebf0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 -- 172.21.15.133:0/2230970262 shutdown_connections 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 --2- 172.21.15.133:0/2230970262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb9415b9b0 0x7ffb9406e8f0 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 --2- 172.21.15.133:0/2230970262 >> v2:172.21.15.162:3300/0 conn(0x7ffb940b2150 0x7ffb9406e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 --2- 172.21.15.133:0/2230970262 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb94143ea0 0x7ffb940cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 -- 172.21.15.133:0/2230970262 >> 172.21.15.133:0/2230970262 conn(0x7ffb94066f70 msgr2=0x7ffb9406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:03.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 -- 172.21.15.133:0/2230970262 shutdown_connections 2024-04-25T10:17:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 -- 172.21.15.133:0/2230970262 wait complete. 2024-04-25T10:17:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 Processor -- start 2024-04-25T10:17:03.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 -- start start 2024-04-25T10:17:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb940b2150 0x7ffb941404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb94143ea0 0x7ffb94163330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb9415b9b0 0x7ffb94160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9415ede0 con 0x7ffb94143ea0 2024-04-25T10:17:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9415ec60 con 0x7ffb9415b9b0 2024-04-25T10:17:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9415ef60 con 0x7ffb940b2150 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb8bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb94143ea0 0x7ffb94163330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb8bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb94143ea0 0x7ffb94163330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44110/0 (socket says 172.21.15.133:44110) 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb8bfff640 1 -- 172.21.15.133:0/2345944996 learned_addr learned my addr 172.21.15.133:0/2345944996 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb93fff640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb940b2150 0x7ffb941404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb98e12640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb9415b9b0 0x7ffb94160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb8bfff640 1 -- 172.21.15.133:0/2345944996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb940b2150 msgr2=0x7ffb941404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb8bfff640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb940b2150 0x7ffb941404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb8bfff640 1 -- 172.21.15.133:0/2345944996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb9415b9b0 msgr2=0x7ffb94160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb8bfff640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb9415b9b0 0x7ffb94160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb8bfff640 1 -- 172.21.15.133:0/2345944996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb94163f50 con 0x7ffb94143ea0 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb8bfff640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb94143ea0 0x7ffb94163330 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00c820 tx=0x7ffb7c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb917fa640 1 -- 172.21.15.133:0/2345944996 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c015070 con 0x7ffb94143ea0 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb917fa640 1 -- 172.21.15.133:0/2345944996 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c004240 con 0x7ffb94143ea0 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 -- 172.21.15.133:0/2345944996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb94140c30 con 0x7ffb94143ea0 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb917fa640 1 -- 172.21.15.133:0/2345944996 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00ce10 con 0x7ffb94143ea0 2024-04-25T10:17:03.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.044+0000 7ffb9a875640 1 -- 172.21.15.133:0/2345944996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9415f840 con 0x7ffb94143ea0 2024-04-25T10:17:03.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.048+0000 7ffb9a875640 1 -- 172.21.15.133:0/2345944996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb94068e80 con 0x7ffb94143ea0 2024-04-25T10:17:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.048+0000 7ffb917fa640 1 -- 172.21.15.133:0/2345944996 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb7c007830 con 0x7ffb94143ea0 2024-04-25T10:17:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.048+0000 7ffb917fa640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb74066f60 0x7ffb74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.048+0000 7ffb917fa640 1 -- 172.21.15.133:0/2345944996 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ffb7c084c50 con 0x7ffb94143ea0 2024-04-25T10:17:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.048+0000 7ffb93fff640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb74066f60 0x7ffb74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.048+0000 7ffb93fff640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb74066f60 0x7ffb74069420 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ffb84006fd0 tx=0x7ffb84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.052+0000 7ffb917fa640 1 -- 172.21.15.133:0/2345944996 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb7c08a020 con 0x7ffb94143ea0 2024-04-25T10:17:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 -- 172.21.15.133:0/2345944996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb940b4980 con 0x7ffb94143ea0 2024-04-25T10:17:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb917fa640 1 -- 172.21.15.133:0/2345944996 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v179) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb7c055eb0 con 0x7ffb94143ea0 2024-04-25T10:17:03.210 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:17:03.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 -- 172.21.15.133:0/2345944996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb74066f60 msgr2=0x7ffb74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb74066f60 0x7ffb74069420 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ffb84006fd0 tx=0x7ffb84008040 comp rx=0 tx=0).stop 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 -- 172.21.15.133:0/2345944996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb94143ea0 msgr2=0x7ffb94163330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb94143ea0 0x7ffb94163330 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00c820 tx=0x7ffb7c00cce0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 -- 172.21.15.133:0/2345944996 shutdown_connections 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb9415b9b0 0x7ffb94160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb74066f60 0x7ffb74069420 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb94143ea0 0x7ffb94163330 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 --2- 172.21.15.133:0/2345944996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb940b2150 0x7ffb941404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 -- 172.21.15.133:0/2345944996 >> 172.21.15.133:0/2345944996 conn(0x7ffb94066f70 msgr2=0x7ffb9415bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 -- 172.21.15.133:0/2345944996 shutdown_connections 2024-04-25T10:17:03.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.208+0000 7ffb9a875640 1 -- 172.21.15.133:0/2345944996 wait complete. 2024-04-25T10:17:03.228 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:17:03.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05adeac640 1 Processor -- start 2024-04-25T10:17:03.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05adeac640 1 -- start start 2024-04-25T10:17:03.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05adeac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815c9a0 0x7f05a80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05adeac640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f05a80b1f70 0x7f05a806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05adeac640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f05a80ce330 0x7f05a80ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05adeac640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80cece0 con 0x7f05a80ce330 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05adeac640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80c61d0 con 0x7f05a80b1f70 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05adeac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80c64b0 con 0x7f05a815c9a0 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05a7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f05a80ce330 0x7f05a80ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05a7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f05a80ce330 0x7f05a80ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44126/0 (socket says 172.21.15.133:44126) 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05a7fff640 1 -- 172.21.15.133:0/1825628657 learned_addr learned my addr 172.21.15.133:0/1825628657 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f059ffff640 1 --2- 172.21.15.133:0/1825628657 >> v2:172.21.15.162:3300/0 conn(0x7f05a80b1f70 0x7f05a806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05a77fe640 1 --2- 172.21.15.133:0/1825628657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815c9a0 0x7f05a80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05a7fff640 1 -- 172.21.15.133:0/1825628657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815c9a0 msgr2=0x7f05a80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05a7fff640 1 --2- 172.21.15.133:0/1825628657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815c9a0 0x7f05a80c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05a7fff640 1 -- 172.21.15.133:0/1825628657 >> v2:172.21.15.162:3300/0 conn(0x7f05a80b1f70 msgr2=0x7f05a806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05a7fff640 1 --2- 172.21.15.133:0/1825628657 >> v2:172.21.15.162:3300/0 conn(0x7f05a80b1f70 0x7f05a806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.340+0000 7f05a7fff640 1 -- 172.21.15.133:0/1825628657 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a806d4f0 con 0x7f05a80ce330 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a7fff640 1 --2- 172.21.15.133:0/1825628657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a80ce330 0x7f05a80ce7a0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f05a806f500 tx=0x7f05a001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d937dcee605a79a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a6ffd640 1 -- 172.21.15.133:0/1825628657 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a00a5750 con 0x7f05a80ce330 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a6ffd640 1 -- 172.21.15.133:0/1825628657 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a00a58f0 con 0x7f05a80ce330 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a6ffd640 1 -- 172.21.15.133:0/1825628657 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a00a5cc0 con 0x7f05a80ce330 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 -- 172.21.15.133:0/1825628657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a80ce330 msgr2=0x7f05a80ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 --2- 172.21.15.133:0/1825628657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a80ce330 0x7f05a80ce7a0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f05a806f500 tx=0x7f05a001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 -- 172.21.15.133:0/1825628657 shutdown_connections 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 --2- 172.21.15.133:0/1825628657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a80ce330 0x7f05a80ce7a0 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 --2- 172.21.15.133:0/1825628657 >> v2:172.21.15.162:3300/0 conn(0x7f05a80b1f70 0x7f05a806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 --2- 172.21.15.133:0/1825628657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815c9a0 0x7f05a80c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 -- 172.21.15.133:0/1825628657 >> 172.21.15.133:0/1825628657 conn(0x7f05a8067000 msgr2=0x7f05a806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 -- 172.21.15.133:0/1825628657 shutdown_connections 2024-04-25T10:17:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 -- 172.21.15.133:0/1825628657 wait complete. 2024-04-25T10:17:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 Processor -- start 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 -- start start 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a80b1f70 0x7f05a814d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05a80ce330 0x7f05a8155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815c9a0 0x7f05a8158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a815cf00 con 0x7f05a80b1f70 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a815cd80 con 0x7f05a80ce330 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05adeac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a815d950 con 0x7f05a815c9a0 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a80b1f70 0x7f05a814d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a80b1f70 0x7f05a814d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44132/0 (socket says 172.21.15.133:44132) 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a77fe640 1 -- 172.21.15.133:0/626493024 learned_addr learned my addr 172.21.15.133:0/626493024 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a7fff640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815c9a0 0x7f05a8158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f059ffff640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05a80ce330 0x7f05a8155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a77fe640 1 -- 172.21.15.133:0/626493024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815c9a0 msgr2=0x7f05a8158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a77fe640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815c9a0 0x7f05a8158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a77fe640 1 -- 172.21.15.133:0/626493024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05a80ce330 msgr2=0x7f05a8155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a77fe640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05a80ce330 0x7f05a8155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.344+0000 7f05a77fe640 1 -- 172.21.15.133:0/626493024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a81f79c0 con 0x7f05a80b1f70 2024-04-25T10:17:03.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f05a77fe640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a80b1f70 0x7f05a814d040 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f059800d5f0 tx=0x7f059800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f05a4ff9640 1 -- 172.21.15.133:0/626493024 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980125f0 con 0x7f05a80b1f70 2024-04-25T10:17:03.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f05a4ff9640 1 -- 172.21.15.133:0/626493024 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0598019070 con 0x7f05a80b1f70 2024-04-25T10:17:03.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f05adeac640 1 -- 172.21.15.133:0/626493024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a81f7ca0 con 0x7f05a80b1f70 2024-04-25T10:17:03.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f05a4ff9640 1 -- 172.21.15.133:0/626493024 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980129d0 con 0x7f05a80b1f70 2024-04-25T10:17:03.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f05adeac640 1 -- 172.21.15.133:0/626493024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a81f8260 con 0x7f05a80b1f70 2024-04-25T10:17:03.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f05adeac640 1 -- 172.21.15.133:0/626493024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a8068ef0 con 0x7f05a80b1f70 2024-04-25T10:17:03.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f05a4ff9640 1 -- 172.21.15.133:0/626493024 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0598012b70 con 0x7f05a80b1f70 2024-04-25T10:17:03.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f05a4ff9640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0588066f90 0x7f0588069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f059ffff640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0588066f90 0x7f0588069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f05a4ff9640 1 -- 172.21.15.133:0/626493024 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0598085310 con 0x7f05a80b1f70 2024-04-25T10:17:03.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.348+0000 7f059ffff640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0588066f90 0x7f0588069450 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f05a8166580 tx=0x7f0590008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.352+0000 7f05a4ff9640 1 -- 172.21.15.133:0/626493024 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0598056450 con 0x7f05a80b1f70 2024-04-25T10:17:03.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.500+0000 7f05adeac640 1 -- 172.21.15.133:0/626493024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f05a8150f60 con 0x7f05a80b1f70 2024-04-25T10:17:03.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.500+0000 7f05a4ff9640 1 -- 172.21.15.133:0/626493024 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v179) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0598083e20 con 0x7f05a80b1f70 2024-04-25T10:17:03.502 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:03.502 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:17:03.502 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:03.502 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:17:03.502 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:17:03.502 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:03.502 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:17:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.500+0000 7f05adeac640 1 -- 172.21.15.133:0/626493024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0588066f90 msgr2=0x7f0588069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.500+0000 7f05adeac640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0588066f90 0x7f0588069450 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f05a8166580 tx=0x7f0590008040 comp rx=0 tx=0).stop 2024-04-25T10:17:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.500+0000 7f05adeac640 1 -- 172.21.15.133:0/626493024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a80b1f70 msgr2=0x7f05a814d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.500+0000 7f05adeac640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a80b1f70 0x7f05a814d040 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f059800d5f0 tx=0x7f059800dab0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.504+0000 7f05adeac640 1 -- 172.21.15.133:0/626493024 shutdown_connections 2024-04-25T10:17:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.504+0000 7f05adeac640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f05a815c9a0 0x7f05a8158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.504+0000 7f05adeac640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05a80ce330 0x7f05a8155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.504+0000 7f05adeac640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0588066f90 0x7f0588069450 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.504+0000 7f05adeac640 1 --2- 172.21.15.133:0/626493024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05a80b1f70 0x7f05a814d040 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.504+0000 7f05adeac640 1 -- 172.21.15.133:0/626493024 >> 172.21.15.133:0/626493024 conn(0x7f05a8067000 msgr2=0x7f05a815afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.504+0000 7f05adeac640 1 -- 172.21.15.133:0/626493024 shutdown_connections 2024-04-25T10:17:03.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.504+0000 7f05adeac640 1 -- 172.21.15.133:0/626493024 wait complete. 2024-04-25T10:17:03.520 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:17:03.520 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:17:03.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.628+0000 7efc6d203640 1 Processor -- start 2024-04-25T10:17:03.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.628+0000 7efc6d203640 1 -- start start 2024-04-25T10:17:03.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.628+0000 7efc6d203640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc6815aac0 0x7efc680ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.628+0000 7efc6d203640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efc680b0220 0x7efc6815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.628+0000 7efc6d203640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc6815b7a0 con 0x7efc680d01f0 2024-04-25T10:17:03.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.628+0000 7efc6d203640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc6815ba80 con 0x7efc680b0220 2024-04-25T10:17:03.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.628+0000 7efc6d203640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6815bd60 con 0x7efc6815aac0 2024-04-25T10:17:03.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.628+0000 7efc6659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efc680b0220 0x7efc6815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.628+0000 7efc6659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efc680b0220 0x7efc6815aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44146/0 (socket says 172.21.15.133:44146) 2024-04-25T10:17:03.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.628+0000 7efc6659c640 1 -- 172.21.15.133:0/4270461357 learned_addr learned my addr 172.21.15.133:0/4270461357 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:03.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.628+0000 7efc66d9d640 1 --2- 172.21.15.133:0/4270461357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc6815aac0 0x7efc680ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6659c640 1 -- 172.21.15.133:0/4270461357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc6815aac0 msgr2=0x7efc680ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6659c640 1 --2- 172.21.15.133:0/4270461357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc6815aac0 0x7efc680ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6659c640 1 -- 172.21.15.133:0/4270461357 >> v1:172.21.15.162:6789/0 conn(0x7efc680d01f0 legacy=0x7efc680d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:03.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6659c640 1 -- 172.21.15.133:0/4270461357 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc680cb1f0 con 0x7efc680b0220 2024-04-25T10:17:03.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc66d9d640 1 --2- 172.21.15.133:0/4270461357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc6815aac0 0x7efc680ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:03.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6659c640 1 --2- 172.21.15.133:0/4270461357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc680b0220 0x7efc6815aea0 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7efc50009870 tx=0x7efc5001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8b42c4e16c7df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc65d9b640 1 -- 172.21.15.133:0/4270461357 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500b0070 con 0x7efc680b0220 2024-04-25T10:17:03.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc65d9b640 1 -- 172.21.15.133:0/4270461357 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc500ab600 con 0x7efc680b0220 2024-04-25T10:17:03.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc65d9b640 1 -- 172.21.15.133:0/4270461357 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500ab9d0 con 0x7efc680b0220 2024-04-25T10:17:03.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6d203640 1 -- 172.21.15.133:0/4270461357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc680b0220 msgr2=0x7efc6815aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6d203640 1 --2- 172.21.15.133:0/4270461357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc680b0220 0x7efc6815aea0 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7efc50009870 tx=0x7efc5001c4d0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6d203640 1 -- 172.21.15.133:0/4270461357 shutdown_connections 2024-04-25T10:17:03.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6d203640 1 --2- 172.21.15.133:0/4270461357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc680b0220 0x7efc6815aea0 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6d203640 1 --2- 172.21.15.133:0/4270461357 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc6815aac0 0x7efc680ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6d203640 1 -- 172.21.15.133:0/4270461357 >> 172.21.15.133:0/4270461357 conn(0x7efc680726f0 msgr2=0x7efc680999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:03.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6d203640 1 -- 172.21.15.133:0/4270461357 shutdown_connections 2024-04-25T10:17:03.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.632+0000 7efc6d203640 1 -- 172.21.15.133:0/4270461357 wait complete. 2024-04-25T10:17:03.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6d203640 1 Processor -- start 2024-04-25T10:17:03.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6d203640 1 -- start start 2024-04-25T10:17:03.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6d203640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc680b0220 0x7efc681512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6d203640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc680d01f0 0x7efc68157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6d203640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6815aac0 0x7efc68156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6d203640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc680cc2b0 con 0x7efc6815aac0 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6d203640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc680cc130 con 0x7efc680d01f0 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6d203640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc680cc430 con 0x7efc680b0220 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6815aac0 0x7efc68156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6815aac0 0x7efc68156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44156/0 (socket says 172.21.15.133:44156) 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6759e640 1 -- 172.21.15.133:0/1095560160 learned_addr learned my addr 172.21.15.133:0/1095560160 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc66d9d640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc680b0220 0x7efc681512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6759e640 1 -- 172.21.15.133:0/1095560160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc680b0220 msgr2=0x7efc681512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6759e640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc680b0220 0x7efc681512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6759e640 1 -- 172.21.15.133:0/1095560160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc680d01f0 msgr2=0x7efc68157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6759e640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc680d01f0 0x7efc68157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6759e640 1 -- 172.21.15.133:0/1095560160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68154000 con 0x7efc6815aac0 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6759e640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6815aac0 0x7efc68156510 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7efc6000c310 tx=0x7efc6000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc5f7fe640 1 -- 172.21.15.133:0/1095560160 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc60017070 con 0x7efc6815aac0 2024-04-25T10:17:03.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc5f7fe640 1 -- 172.21.15.133:0/1095560160 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc600052d0 con 0x7efc6815aac0 2024-04-25T10:17:03.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6d203640 1 -- 172.21.15.133:0/1095560160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc68153d20 con 0x7efc6815aac0 2024-04-25T10:17:03.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc5f7fe640 1 -- 172.21.15.133:0/1095560160 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc60003800 con 0x7efc6815aac0 2024-04-25T10:17:03.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6d203640 1 -- 172.21.15.133:0/1095560160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc68155d10 con 0x7efc6815aac0 2024-04-25T10:17:03.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc6d203640 1 -- 172.21.15.133:0/1095560160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc68069000 con 0x7efc6815aac0 2024-04-25T10:17:03.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc5f7fe640 1 -- 172.21.15.133:0/1095560160 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc6001a070 con 0x7efc6815aac0 2024-04-25T10:17:03.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.636+0000 7efc5f7fe640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc48066f60 0x7efc48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:03.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.640+0000 7efc66d9d640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc48066f60 0x7efc48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:03.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.640+0000 7efc5f7fe640 1 -- 172.21.15.133:0/1095560160 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efc60083f70 con 0x7efc6815aac0 2024-04-25T10:17:03.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.640+0000 7efc66d9d640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc48066f60 0x7efc48069420 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7efc58008f80 tx=0x7efc580084d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:03.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.644+0000 7efc5f7fe640 1 -- 172.21.15.133:0/1095560160 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efc60005480 con 0x7efc6815aac0 2024-04-25T10:17:03.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:03.800+0000 7efc6d203640 1 -- 172.21.15.133:0/1095560160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efc68158ad0 con 0x7efc6815aac0 2024-04-25T10:17:04.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc5f7fe640 1 -- 172.21.15.133:0/1095560160 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v180) v1 ==== 138+0+0 (secure 0 0 0) 0x7efc60054b80 con 0x7efc6815aac0 2024-04-25T10:17:04.229 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 -- 172.21.15.133:0/1095560160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc48066f60 msgr2=0x7efc48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc48066f60 0x7efc48069420 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7efc58008f80 tx=0x7efc580084d0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 -- 172.21.15.133:0/1095560160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6815aac0 msgr2=0x7efc68156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6815aac0 0x7efc68156510 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7efc6000c310 tx=0x7efc6000c7d0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 -- 172.21.15.133:0/1095560160 shutdown_connections 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efc48066f60 0x7efc48069420 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc6815aac0 0x7efc68156510 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc680d01f0 0x7efc68157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 --2- 172.21.15.133:0/1095560160 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efc680b0220 0x7efc681512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 -- 172.21.15.133:0/1095560160 >> 172.21.15.133:0/1095560160 conn(0x7efc680726f0 msgr2=0x7efc680cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 -- 172.21.15.133:0/1095560160 shutdown_connections 2024-04-25T10:17:04.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.228+0000 7efc6d203640 1 -- 172.21.15.133:0/1095560160 wait complete. 2024-04-25T10:17:04.258 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:17:04.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c97de0640 1 Processor -- start 2024-04-25T10:17:04.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c97de0640 1 -- start start 2024-04-25T10:17:04.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c97de0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c90145170 0x7f5c9006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c97de0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c900ce2e0 con 0x7f5c9015ced0 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c97de0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9006d4b0 con 0x7f5c9006e4d0 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c97de0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9006db00 con 0x7f5c90145170 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c9637d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5c9015ced0 0x7f5c900b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38164/0 (socket says 172.21.15.133:38164) 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c9637d640 1 -- 172.21.15.133:0/2881629483 learned_addr learned my addr 172.21.15.133:0/2881629483 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c95b7c640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c90145170 0x7f5c9006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037106026 0 0) 0x7f5c900ce2e0 con 0x7f5c9015ced0 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c7c003530 con 0x7f5c9015ced0 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 701886740 0 0) 0x7f5c7c003530 con 0x7f5c9015ced0 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c90145170 msgr2=0x7f5c9006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c90145170 0x7f5c9006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 >> v1:172.21.15.162:6789/0 conn(0x7f5c9006e4d0 legacy=0x7f5c900cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9006d790 con 0x7f5c9015ced0 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5c88001680 con 0x7f5c9015ced0 2024-04-25T10:17:04.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 >> v1:172.21.15.133:6789/0 conn(0x7f5c9015ced0 legacy=0x7f5c900b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c90145170 0x7f5c7c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c7c003f70 0x7f5c7c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c7c004d50 0x7f5c7c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9006db00 con 0x7f5c7c003f70 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c00bab0 con 0x7f5c7c004d50 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c00bcb0 con 0x7f5c90145170 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.384+0000 7f5c9537b640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c7c003f70 0x7f5c7c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c95b7c640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c90145170 0x7f5c7c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c9637d640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c7c004d50 0x7f5c7c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c9537b640 1 -- 172.21.15.133:0/2881629483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c90145170 msgr2=0x7f5c7c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c9537b640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c90145170 0x7f5c7c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c9537b640 1 -- 172.21.15.133:0/2881629483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c7c004d50 msgr2=0x7f5c7c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:04.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c9537b640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c7c004d50 0x7f5c7c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c9537b640 1 -- 172.21.15.133:0/2881629483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c003530 con 0x7f5c7c003f70 2024-04-25T10:17:04.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c9537b640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c7c003f70 0x7f5c7c004810 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f5c80001750 tx=0x7f5c8001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cdfc90c3cb94fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:04.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800013a0 con 0x7f5c7c003f70 2024-04-25T10:17:04.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c800052f0 con 0x7f5c7c003f70 2024-04-25T10:17:04.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c94b7a640 1 -- 172.21.15.133:0/2881629483 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c80003840 con 0x7f5c7c003f70 2024-04-25T10:17:04.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c97de0640 1 -- 172.21.15.133:0/2881629483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c7c003f70 msgr2=0x7f5c7c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c97de0640 1 --2- 172.21.15.133:0/2881629483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c7c003f70 0x7f5c7c004810 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f5c80001750 tx=0x7f5c8001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c9637d640 1 -- 172.21.15.133:0/2881629483 reap_dead start 2024-04-25T10:17:04.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c97de0640 1 -- 172.21.15.133:0/2881629483 shutdown_connections 2024-04-25T10:17:04.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c97de0640 1 -- 172.21.15.133:0/2881629483 >> 172.21.15.133:0/2881629483 conn(0x7f5c90066f70 msgr2=0x7f5c9006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:04.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c97de0640 1 -- 172.21.15.133:0/2881629483 shutdown_connections 2024-04-25T10:17:04.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.388+0000 7f5c97de0640 1 -- 172.21.15.133:0/2881629483 wait complete. 2024-04-25T10:17:04.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c97de0640 1 Processor -- start 2024-04-25T10:17:04.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c97de0640 1 -- start start 2024-04-25T10:17:04.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c97de0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c7c003f70 0x7f5c90138df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c97de0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9013c6c0 0x7f5c900b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c97de0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c900ad630 0x7f5c900b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c97de0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c900cdf30 con 0x7f5c9013c6c0 2024-04-25T10:17:04.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c97de0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c90145550 con 0x7f5c7c003f70 2024-04-25T10:17:04.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c97de0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c90152f60 con 0x7f5c900ad630 2024-04-25T10:17:04.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c9537b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9013c6c0 0x7f5c900b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c9537b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9013c6c0 0x7f5c900b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44178/0 (socket says 172.21.15.133:44178) 2024-04-25T10:17:04.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c9537b640 1 -- 172.21.15.133:0/2991162986 learned_addr learned my addr 172.21.15.133:0/2991162986 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:04.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c9637d640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c900ad630 0x7f5c900b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c95b7c640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c7c003f70 0x7f5c90138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c9537b640 1 -- 172.21.15.133:0/2991162986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c900ad630 msgr2=0x7f5c900b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c9537b640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c900ad630 0x7f5c900b0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c9537b640 1 -- 172.21.15.133:0/2991162986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c7c003f70 msgr2=0x7f5c90138df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c9537b640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c7c003f70 0x7f5c90138df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c9537b640 1 -- 172.21.15.133:0/2991162986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900adbf0 con 0x7f5c9013c6c0 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c9537b640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9013c6c0 0x7f5c900b04c0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f5c80005040 tx=0x7f5c7c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c867fc640 1 -- 172.21.15.133:0/2991162986 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c80000ee0 con 0x7f5c9013c6c0 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c867fc640 1 -- 172.21.15.133:0/2991162986 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c800bb070 con 0x7f5c9013c6c0 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c97de0640 1 -- 172.21.15.133:0/2991162986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c900aded0 con 0x7f5c9013c6c0 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c867fc640 1 -- 172.21.15.133:0/2991162986 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800069e0 con 0x7f5c9013c6c0 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c97de0640 1 -- 172.21.15.133:0/2991162986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c900ae490 con 0x7f5c9013c6c0 2024-04-25T10:17:04.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c73fff640 1 -- 172.21.15.133:0/2991162986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c0052a0 con 0x7f5c9013c6c0 2024-04-25T10:17:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.392+0000 7f5c867fc640 1 -- 172.21.15.133:0/2991162986 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c800b4040 con 0x7f5c9013c6c0 2024-04-25T10:17:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.396+0000 7f5c867fc640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c74067130 0x7f5c740695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.396+0000 7f5c867fc640 1 -- 172.21.15.133:0/2991162986 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f5c8011f550 con 0x7f5c9013c6c0 2024-04-25T10:17:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.396+0000 7f5c95b7c640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c74067130 0x7f5c740695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:04.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.396+0000 7f5c95b7c640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c74067130 0x7f5c740695f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5c901f6fb0 tx=0x7f5c8c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:04.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.400+0000 7f5c867fc640 1 -- 172.21.15.133:0/2991162986 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c800160f0 con 0x7f5c9013c6c0 2024-04-25T10:17:04.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:04.560+0000 7f5c73fff640 1 -- 172.21.15.133:0/2991162986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5c6c005590 con 0x7f5c9013c6c0 2024-04-25T10:17:05.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.232+0000 7f5c867fc640 1 -- 172.21.15.133:0/2991162986 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v181) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5c800aa3a0 con 0x7f5c9013c6c0 2024-04-25T10:17:05.233 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:17:05.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.232+0000 7f5c73fff640 1 -- 172.21.15.133:0/2991162986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c74067130 msgr2=0x7f5c740695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:05.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.232+0000 7f5c73fff640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c74067130 0x7f5c740695f0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5c901f6fb0 tx=0x7f5c8c00b040 comp rx=0 tx=0).stop 2024-04-25T10:17:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.232+0000 7f5c73fff640 1 -- 172.21.15.133:0/2991162986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9013c6c0 msgr2=0x7f5c900b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.232+0000 7f5c73fff640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9013c6c0 0x7f5c900b04c0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f5c80005040 tx=0x7f5c7c005840 comp rx=0 tx=0).stop 2024-04-25T10:17:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.236+0000 7f5c73fff640 1 -- 172.21.15.133:0/2991162986 shutdown_connections 2024-04-25T10:17:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.236+0000 7f5c73fff640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c900ad630 0x7f5c900b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.236+0000 7f5c73fff640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c74067130 0x7f5c740695f0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.236+0000 7f5c73fff640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9013c6c0 0x7f5c900b04c0 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.236+0000 7f5c73fff640 1 --2- 172.21.15.133:0/2991162986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c7c003f70 0x7f5c90138df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.236+0000 7f5c73fff640 1 -- 172.21.15.133:0/2991162986 >> 172.21.15.133:0/2991162986 conn(0x7f5c90066f70 msgr2=0x7f5c900cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.236+0000 7f5c73fff640 1 -- 172.21.15.133:0/2991162986 shutdown_connections 2024-04-25T10:17:05.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.236+0000 7f5c73fff640 1 -- 172.21.15.133:0/2991162986 wait complete. 2024-04-25T10:17:05.269 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:17:05.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e9d49640 1 Processor -- start 2024-04-25T10:17:05.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e9d49640 1 -- start start 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e9d49640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7e415d9a0 0x7fd7e40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e9d49640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7e40d2030 0x7fd7e40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e9d49640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e40d24c0 con 0x7fd7e40ce600 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e9d49640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e415ebd0 con 0x7fd7e415d9a0 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e9d49640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e415eeb0 con 0x7fd7e40d2030 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e37fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7e415d9a0 0x7fd7e40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e37fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7e415d9a0 0x7fd7e40d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44190/0 (socket says 172.21.15.133:44190) 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e37fe640 1 -- 172.21.15.133:0/3211785440 learned_addr learned my addr 172.21.15.133:0/3211785440 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e3fff640 1 --2- 172.21.15.133:0/3211785440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7e40d2030 0x7fd7e40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e37fe640 1 -- 172.21.15.133:0/3211785440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7e40d2030 msgr2=0x7fd7e40cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e37fe640 1 --2- 172.21.15.133:0/3211785440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7e40d2030 0x7fd7e40cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e37fe640 1 -- 172.21.15.133:0/3211785440 >> v1:172.21.15.162:6789/0 conn(0x7fd7e40ce600 legacy=0x7fd7e40d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:05.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.396+0000 7fd7e37fe640 1 -- 172.21.15.133:0/3211785440 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e40d0330 con 0x7fd7e415d9a0 2024-04-25T10:17:05.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e37fe640 1 --2- 172.21.15.133:0/3211785440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e415d9a0 0x7fd7e40d0b30 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009870 tx=0x7fd7d401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e030cfcbca4d48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:05.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e27fc640 1 -- 172.21.15.133:0/3211785440 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d40b0070 con 0x7fd7e415d9a0 2024-04-25T10:17:05.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e27fc640 1 -- 172.21.15.133:0/3211785440 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d40ab600 con 0x7fd7e415d9a0 2024-04-25T10:17:05.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e27fc640 1 -- 172.21.15.133:0/3211785440 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d40ab9d0 con 0x7fd7e415d9a0 2024-04-25T10:17:05.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 -- 172.21.15.133:0/3211785440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e415d9a0 msgr2=0x7fd7e40d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:05.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 --2- 172.21.15.133:0/3211785440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e415d9a0 0x7fd7e40d0b30 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009870 tx=0x7fd7d401c490 comp rx=0 tx=0).stop 2024-04-25T10:17:05.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 -- 172.21.15.133:0/3211785440 shutdown_connections 2024-04-25T10:17:05.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 --2- 172.21.15.133:0/3211785440 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7e40d2030 0x7fd7e40cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 --2- 172.21.15.133:0/3211785440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e415d9a0 0x7fd7e40d0b30 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 -- 172.21.15.133:0/3211785440 >> 172.21.15.133:0/3211785440 conn(0x7fd7e406bf80 msgr2=0x7fd7e406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:05.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 -- 172.21.15.133:0/3211785440 shutdown_connections 2024-04-25T10:17:05.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 -- 172.21.15.133:0/3211785440 wait complete. 2024-04-25T10:17:05.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 Processor -- start 2024-04-25T10:17:05.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 -- start start 2024-04-25T10:17:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e40ce600 0x7fd7e40c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7e40d2030 0x7fd7e41f9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e415d9a0 0x7fd7e41fa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40b9820 con 0x7fd7e40ce600 2024-04-25T10:17:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40b96a0 con 0x7fd7e415d9a0 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e9d49640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e415e590 con 0x7fd7e40d2030 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e40ce600 0x7fd7e40c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e40ce600 0x7fd7e40c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44206/0 (socket says 172.21.15.133:44206) 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.400+0000 7fd7e37fe640 1 -- 172.21.15.133:0/1409744285 learned_addr learned my addr 172.21.15.133:0/1409744285 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e3fff640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e415d9a0 0x7fd7e41fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e2ffd640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7e40d2030 0x7fd7e41f9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e37fe640 1 -- 172.21.15.133:0/1409744285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7e40d2030 msgr2=0x7fd7e41f9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e37fe640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7e40d2030 0x7fd7e41f9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e37fe640 1 -- 172.21.15.133:0/1409744285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e415d9a0 msgr2=0x7fd7e41fa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e37fe640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e415d9a0 0x7fd7e41fa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e37fe640 1 -- 172.21.15.133:0/1409744285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e4158a00 con 0x7fd7e40ce600 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e3fff640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e415d9a0 0x7fd7e41fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e37fe640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e40ce600 0x7fd7e40c92c0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40ae040 tx=0x7fd7d401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:05.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7cbfff640 1 -- 172.21.15.133:0/1409744285 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d40ba020 con 0x7fd7e40ce600 2024-04-25T10:17:05.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7cbfff640 1 -- 172.21.15.133:0/1409744285 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d40b6640 con 0x7fd7e40ce600 2024-04-25T10:17:05.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e9d49640 1 -- 172.21.15.133:0/1409744285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e4158160 con 0x7fd7e40ce600 2024-04-25T10:17:05.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7cbfff640 1 -- 172.21.15.133:0/1409744285 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d40b0040 con 0x7fd7e40ce600 2024-04-25T10:17:05.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e9d49640 1 -- 172.21.15.133:0/1409744285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e4158720 con 0x7fd7e40ce600 2024-04-25T10:17:05.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e9d49640 1 -- 172.21.15.133:0/1409744285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e4068e80 con 0x7fd7e40ce600 2024-04-25T10:17:05.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7cbfff640 1 -- 172.21.15.133:0/1409744285 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7d40bd020 con 0x7fd7e40ce600 2024-04-25T10:17:05.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7cbfff640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7d0066f60 0x7fd7d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:05.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7cbfff640 1 -- 172.21.15.133:0/1409744285 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd7d411f290 con 0x7fd7e40ce600 2024-04-25T10:17:05.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e2ffd640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7d0066f60 0x7fd7d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:05.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.404+0000 7fd7e2ffd640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7d0066f60 0x7fd7d0069420 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd7e41f9aa0 tx=0x7fd7d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:05.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.408+0000 7fd7cbfff640 1 -- 172.21.15.133:0/1409744285 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7d40c4030 con 0x7fd7e40ce600 2024-04-25T10:17:05.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:05.600+0000 7fd7e9d49640 1 -- 172.21.15.133:0/1409744285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd7e40b6ac0 con 0x7fd7e40ce600 2024-04-25T10:17:06.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.284+0000 7fd7cbfff640 1 -- 172.21.15.133:0/1409744285 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v109) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd7d4016110 con 0x7fd7e40ce600 2024-04-25T10:17:06.296 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 27 and data pool 28 2024-04-25T10:17:06.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 -- 172.21.15.133:0/1409744285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7d0066f60 msgr2=0x7fd7d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7d0066f60 0x7fd7d0069420 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd7e41f9aa0 tx=0x7fd7d8009040 comp rx=0 tx=0).stop 2024-04-25T10:17:06.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 -- 172.21.15.133:0/1409744285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e40ce600 msgr2=0x7fd7e40c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e40ce600 0x7fd7e40c92c0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40ae040 tx=0x7fd7d401acd0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 -- 172.21.15.133:0/1409744285 shutdown_connections 2024-04-25T10:17:06.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7e415d9a0 0x7fd7e41fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd7e40d2030 0x7fd7e41f9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7d0066f60 0x7fd7d0069420 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 --2- 172.21.15.133:0/1409744285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e40ce600 0x7fd7e40c92c0 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 -- 172.21.15.133:0/1409744285 >> 172.21.15.133:0/1409744285 conn(0x7fd7e406bf80 msgr2=0x7fd7e40d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:06.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 -- 172.21.15.133:0/1409744285 shutdown_connections 2024-04-25T10:17:06.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.304+0000 7fd7e9d49640 1 -- 172.21.15.133:0/1409744285 wait complete. 2024-04-25T10:17:06.337 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:06.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.448+0000 7f9dc4a96640 1 Processor -- start 2024-04-25T10:17:06.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.448+0000 7f9dc4a96640 1 -- start start 2024-04-25T10:17:06.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dc4a96640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9dc006c3b0 0x7f9dc006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dc4a96640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc015c6f0 0x7f9dc015d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dc4a96640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc015cb60 con 0x7f9dc01591d0 2024-04-25T10:17:06.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dc4a96640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc006d9e0 con 0x7f9dc006c3b0 2024-04-25T10:17:06.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dc4a96640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc006e150 con 0x7f9dc015c6f0 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbe59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9dc01591d0 0x7f9dc006bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55512/0 (socket says 172.21.15.133:55512) 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbe59c640 1 -- 172.21.15.133:0/3983856305 learned_addr learned my addr 172.21.15.133:0/3983856305 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbed9d640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc015c6f0 0x7f9dc015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9db7fff640 1 --2- 172.21.15.133:0/3983856305 >> v2:172.21.15.162:3300/0 conn(0x7f9dc006c3b0 0x7f9dc006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394837460 0 0) 0x7f9dc015cb60 con 0x7f9dc01591d0 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dac003530 con 0x7f9dc01591d0 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2246743356 0 0) 0x7f9dac003530 con 0x7f9dc01591d0 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc015c6f0 msgr2=0x7f9dc015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc015c6f0 0x7f9dc015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 >> v2:172.21.15.162:3300/0 conn(0x7f9dc006c3b0 msgr2=0x7f9dc006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 --2- 172.21.15.133:0/3983856305 >> v2:172.21.15.162:3300/0 conn(0x7f9dc006c3b0 0x7f9dc006fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc006e410 con 0x7f9dc01591d0 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f9da8001680 con 0x7f9dc01591d0 2024-04-25T10:17:06.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 >> v1:172.21.15.133:6789/0 conn(0x7f9dc01591d0 legacy=0x7f9dc006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc015c6f0 0x7f9dac0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 0x7f9dac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dac004d50 0x7f9dac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc006e150 con 0x7f9dac003f90 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac00b9d0 con 0x7f9dac004d50 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac00bb50 con 0x7f9dc015c6f0 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9db7fff640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 0x7f9dac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbe59c640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc015c6f0 0x7f9dac0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9dbed9d640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dac004d50 0x7f9dac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9db7fff640 1 -- 172.21.15.133:0/3983856305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc015c6f0 msgr2=0x7f9dac0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9db7fff640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc015c6f0 0x7f9dac0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9db7fff640 1 -- 172.21.15.133:0/3983856305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dac004d50 msgr2=0x7f9dac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9db7fff640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dac004d50 0x7f9dac0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9db7fff640 1 -- 172.21.15.133:0/3983856305 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac003530 con 0x7f9dac003f90 2024-04-25T10:17:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.452+0000 7f9db7fff640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 0x7f9dac004810 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009b10 tx=0x7f9db001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7707ff83805ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00b2070 con 0x7f9dac003f90 2024-04-25T10:17:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db00a56a0 con 0x7f9dac003f90 2024-04-25T10:17:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbdd9b640 1 -- 172.21.15.133:0/3983856305 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00096b0 con 0x7f9dac003f90 2024-04-25T10:17:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 -- 172.21.15.133:0/3983856305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 msgr2=0x7f9dac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 --2- 172.21.15.133:0/3983856305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 0x7f9dac004810 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009b10 tx=0x7f9db001cab0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbed9d640 1 -- 172.21.15.133:0/3983856305 reap_dead start 2024-04-25T10:17:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 -- 172.21.15.133:0/3983856305 shutdown_connections 2024-04-25T10:17:06.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 -- 172.21.15.133:0/3983856305 >> 172.21.15.133:0/3983856305 conn(0x7f9dc0071aa0 msgr2=0x7f9dc0071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 -- 172.21.15.133:0/3983856305 shutdown_connections 2024-04-25T10:17:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 -- 172.21.15.133:0/3983856305 wait complete. 2024-04-25T10:17:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 Processor -- start 2024-04-25T10:17:06.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 -- start start 2024-04-25T10:17:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 0x7f9dc01407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc0156480 0x7f9dc015d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dc0156900 0x7f9dc015ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc01595b0 con 0x7f9dac003f90 2024-04-25T10:17:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc015d040 con 0x7f9dc0156900 2024-04-25T10:17:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc015d560 con 0x7f9dc0156480 2024-04-25T10:17:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbe59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 0x7f9dc01407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbe59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 0x7f9dc01407c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54602/0 (socket says 172.21.15.133:54602) 2024-04-25T10:17:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbe59c640 1 -- 172.21.15.133:0/1827456392 learned_addr learned my addr 172.21.15.133:0/1827456392 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9db7fff640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc0156480 0x7f9dc015d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbed9d640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dc0156900 0x7f9dc015ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbe59c640 1 -- 172.21.15.133:0/1827456392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc0156480 msgr2=0x7f9dc015d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbe59c640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc0156480 0x7f9dc015d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbe59c640 1 -- 172.21.15.133:0/1827456392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dc0156900 msgr2=0x7f9dc015ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbe59c640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dc0156900 0x7f9dc015ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbe59c640 1 -- 172.21.15.133:0/1827456392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc0154da0 con 0x7f9dac003f90 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dbe59c640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 0x7f9dc01407c0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f9da8011980 tx=0x7f9da8011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9db6ffd640 1 -- 172.21.15.133:0/1827456392 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8010040 con 0x7f9dac003f90 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9db6ffd640 1 -- 172.21.15.133:0/1827456392 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da801a070 con 0x7f9dac003f90 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 -- 172.21.15.133:0/1827456392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc0155080 con 0x7f9dac003f90 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9db6ffd640 1 -- 172.21.15.133:0/1827456392 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da800fe20 con 0x7f9dac003f90 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.456+0000 7f9dc4a96640 1 -- 172.21.15.133:0/1827456392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc0155640 con 0x7f9dac003f90 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.460+0000 7f9dc4a96640 1 -- 172.21.15.133:0/1827456392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc0067c70 con 0x7f9dac003f90 2024-04-25T10:17:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.460+0000 7f9db6ffd640 1 -- 172.21.15.133:0/1827456392 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9da801f460 con 0x7f9dac003f90 2024-04-25T10:17:06.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.460+0000 7f9db6ffd640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9da0066f90 0x7f9da0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.460+0000 7f9db7fff640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9da0066f90 0x7f9da0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.460+0000 7f9db6ffd640 1 -- 172.21.15.133:0/1827456392 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9da80884b0 con 0x7f9dac003f90 2024-04-25T10:17:06.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.460+0000 7f9db7fff640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9da0066f90 0x7f9da0069450 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9db00af000 tx=0x7f9db00046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.464+0000 7f9db6ffd640 1 -- 172.21.15.133:0/1827456392 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9da8059330 con 0x7f9dac003f90 2024-04-25T10:17:06.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.624+0000 7f9dc4a96640 1 -- 172.21.15.133:0/1827456392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dc0072ca0 con 0x7f9dac003f90 2024-04-25T10:17:06.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.624+0000 7f9db6ffd640 1 -- 172.21.15.133:0/1827456392 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16989 (secure 0 0 0) 0x7f9da8085270 con 0x7f9dac003f90 2024-04-25T10:17:06.628 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:06.628 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":182,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:05.606652+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:03.841068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:04.604661+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 -- 172.21.15.133:0/1827456392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9da0066f90 msgr2=0x7f9da0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9da0066f90 0x7f9da0069450 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9db00af000 tx=0x7f9db00046d0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 -- 172.21.15.133:0/1827456392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 msgr2=0x7f9dc01407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 0x7f9dc01407c0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f9da8011980 tx=0x7f9da8011e40 comp rx=0 tx=0).stop 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 -- 172.21.15.133:0/1827456392 shutdown_connections 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9dc0156900 0x7f9dc015ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9dc0156480 0x7f9dc015d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9da0066f90 0x7f9da0069450 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 --2- 172.21.15.133:0/1827456392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9dac003f90 0x7f9dc01407c0 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 -- 172.21.15.133:0/1827456392 >> 172.21.15.133:0/1827456392 conn(0x7f9dc0071aa0 msgr2=0x7f9dc006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 -- 172.21.15.133:0/1827456392 shutdown_connections 2024-04-25T10:17:06.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.628+0000 7f9dc4a96640 1 -- 172.21.15.133:0/1827456392 wait complete. 2024-04-25T10:17:06.647 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:06.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea8103640 1 Processor -- start 2024-04-25T10:17:06.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea8103640 1 -- start start 2024-04-25T10:17:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea8103640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcea0159dc0 0x7fcea00cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea8103640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcea015e220 0x7fcea006d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea8103640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea00ce880 con 0x7fcea015a1a0 2024-04-25T10:17:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea8103640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea00c49e0 con 0x7fcea0159dc0 2024-04-25T10:17:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea8103640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea00ceb20 con 0x7fcea015e220 2024-04-25T10:17:06.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea5e9f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcea0159dc0 0x7fcea00cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea569e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcea015a1a0 0x7fcea015db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55528/0 (socket says 172.21.15.133:55528) 2024-04-25T10:17:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea569e640 1 -- 172.21.15.133:0/378328874 learned_addr learned my addr 172.21.15.133:0/378328874 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea66a0640 1 --2- 172.21.15.133:0/378328874 >> v2:172.21.15.162:3300/0 conn(0x7fcea015e220 0x7fcea006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea5e9f640 1 -- 172.21.15.133:0/378328874 >> v2:172.21.15.162:3300/0 conn(0x7fcea015e220 msgr2=0x7fcea006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea5e9f640 1 --2- 172.21.15.133:0/378328874 >> v2:172.21.15.162:3300/0 conn(0x7fcea015e220 0x7fcea006d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea5e9f640 1 -- 172.21.15.133:0/378328874 >> v1:172.21.15.133:6789/0 conn(0x7fcea015a1a0 legacy=0x7fcea015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea5e9f640 1 -- 172.21.15.133:0/378328874 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea00cbfa0 con 0x7fcea0159dc0 2024-04-25T10:17:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.764+0000 7fcea4e9d640 1 -- 172.21.15.133:0/378328874 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221190595 0 0) 0x7fcea00ce880 con 0x7fcea015a1a0 2024-04-25T10:17:06.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea66a0640 1 --2- 172.21.15.133:0/378328874 >> v2:172.21.15.162:3300/0 conn(0x7fcea015e220 0x7fcea006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:06.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea5e9f640 1 --2- 172.21.15.133:0/378328874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea0159dc0 0x7fcea00cba60 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fcea00cee00 tx=0x7fce9c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d12408a7e4f5012 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:06.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea4e9d640 1 -- 172.21.15.133:0/378328874 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0b0070 con 0x7fcea0159dc0 2024-04-25T10:17:06.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea4e9d640 1 -- 172.21.15.133:0/378328874 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c0ab600 con 0x7fcea0159dc0 2024-04-25T10:17:06.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea4e9d640 1 -- 172.21.15.133:0/378328874 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0ab9d0 con 0x7fcea0159dc0 2024-04-25T10:17:06.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea8103640 1 -- 172.21.15.133:0/378328874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea0159dc0 msgr2=0x7fcea00cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea8103640 1 --2- 172.21.15.133:0/378328874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea0159dc0 0x7fcea00cba60 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fcea00cee00 tx=0x7fce9c01c480 comp rx=0 tx=0).stop 2024-04-25T10:17:06.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea8103640 1 -- 172.21.15.133:0/378328874 shutdown_connections 2024-04-25T10:17:06.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea8103640 1 --2- 172.21.15.133:0/378328874 >> v2:172.21.15.162:3300/0 conn(0x7fcea015e220 0x7fcea006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea8103640 1 --2- 172.21.15.133:0/378328874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea0159dc0 0x7fcea00cba60 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea8103640 1 -- 172.21.15.133:0/378328874 >> 172.21.15.133:0/378328874 conn(0x7fcea0067000 msgr2=0x7fcea006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:06.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea8103640 1 -- 172.21.15.133:0/378328874 shutdown_connections 2024-04-25T10:17:06.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.768+0000 7fcea8103640 1 -- 172.21.15.133:0/378328874 wait complete. 2024-04-25T10:17:06.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea8103640 1 Processor -- start 2024-04-25T10:17:06.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea8103640 1 -- start start 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea8103640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea0159dc0 0x7fcea0153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea8103640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcea015e220 0x7fcea01544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea8103640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcea0158f90 0x7fcea014de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea8103640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea006da10 con 0x7fcea0159dc0 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea8103640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea006d890 con 0x7fcea0158f90 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea8103640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea006db90 con 0x7fcea015e220 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea5e9f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea0159dc0 0x7fcea0153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea5e9f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea0159dc0 0x7fcea0153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54612/0 (socket says 172.21.15.133:54612) 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea5e9f640 1 -- 172.21.15.133:0/450418626 learned_addr learned my addr 172.21.15.133:0/450418626 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea5e9f640 1 -- 172.21.15.133:0/450418626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcea015e220 msgr2=0x7fcea01544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea5e9f640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcea015e220 0x7fcea01544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea5e9f640 1 -- 172.21.15.133:0/450418626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcea0158f90 msgr2=0x7fcea014de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea5e9f640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcea0158f90 0x7fcea014de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea5e9f640 1 -- 172.21.15.133:0/450418626 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea0148b40 con 0x7fcea0159dc0 2024-04-25T10:17:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea5e9f640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea0159dc0 0x7fcea0153f90 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0ae040 tx=0x7fce9c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fce967fc640 1 -- 172.21.15.133:0/450418626 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0b0040 con 0x7fcea0159dc0 2024-04-25T10:17:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fce967fc640 1 -- 172.21.15.133:0/450418626 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c0ba070 con 0x7fcea0159dc0 2024-04-25T10:17:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea8103640 1 -- 172.21.15.133:0/450418626 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea014ec70 con 0x7fcea0159dc0 2024-04-25T10:17:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fce967fc640 1 -- 172.21.15.133:0/450418626 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0b4810 con 0x7fcea0159dc0 2024-04-25T10:17:06.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea8103640 1 -- 172.21.15.133:0/450418626 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea01464e0 con 0x7fcea0159dc0 2024-04-25T10:17:06.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fcea8103640 1 -- 172.21.15.133:0/450418626 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea0068ed0 con 0x7fcea0159dc0 2024-04-25T10:17:06.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.772+0000 7fce967fc640 1 -- 172.21.15.133:0/450418626 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce9c0ab4a0 con 0x7fcea0159dc0 2024-04-25T10:17:06.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.776+0000 7fce967fc640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce84066f60 0x7fce84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:06.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.776+0000 7fce967fc640 1 -- 172.21.15.133:0/450418626 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce9c11f150 con 0x7fcea0159dc0 2024-04-25T10:17:06.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.776+0000 7fcea569e640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce84066f60 0x7fce84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:06.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.776+0000 7fcea569e640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce84066f60 0x7fce84069420 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fce900011f0 tx=0x7fce90007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:06.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.780+0000 7fce967fc640 1 -- 172.21.15.133:0/450418626 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce9c016070 con 0x7fcea0159dc0 2024-04-25T10:17:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.944+0000 7fcea8103640 1 -- 172.21.15.133:0/450418626 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcea00cd760 con 0x7fcea0159dc0 2024-04-25T10:17:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.944+0000 7fce967fc640 1 -- 172.21.15.133:0/450418626 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16989 (secure 0 0 0) 0x7fce9c0eff40 con 0x7fcea0159dc0 2024-04-25T10:17:06.947 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:06.947 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":182,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:05.606652+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:03.841068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:04.604661+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:06.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 -- 172.21.15.133:0/450418626 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce84066f60 msgr2=0x7fce84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce84066f60 0x7fce84069420 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fce900011f0 tx=0x7fce90007930 comp rx=0 tx=0).stop 2024-04-25T10:17:06.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 -- 172.21.15.133:0/450418626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea0159dc0 msgr2=0x7fcea0153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:06.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea0159dc0 0x7fcea0153f90 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0ae040 tx=0x7fce9c019220 comp rx=0 tx=0).stop 2024-04-25T10:17:06.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 -- 172.21.15.133:0/450418626 shutdown_connections 2024-04-25T10:17:06.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcea0158f90 0x7fcea014de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcea015e220 0x7fcea01544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fce84066f60 0x7fce84069420 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 --2- 172.21.15.133:0/450418626 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcea0159dc0 0x7fcea0153f90 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:06.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 -- 172.21.15.133:0/450418626 >> 172.21.15.133:0/450418626 conn(0x7fcea0067000 msgr2=0x7fcea00cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:06.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 -- 172.21.15.133:0/450418626 shutdown_connections 2024-04-25T10:17:06.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:06.948+0000 7fcea8103640 1 -- 172.21.15.133:0/450418626 wait complete. 2024-04-25T10:17:06.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:17:07.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.088+0000 7f467903e640 1 Processor -- start 2024-04-25T10:17:07.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.088+0000 7f467903e640 1 -- start start 2024-04-25T10:17:07.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467903e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f46740ce600 0x7f46740cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467903e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f46740d1920 0x7f46740d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467903e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467415ebd0 con 0x7f467415d9a0 2024-04-25T10:17:07.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467903e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467415eeb0 con 0x7f46740ce600 2024-04-25T10:17:07.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467903e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46740d0330 con 0x7f46740d1920 2024-04-25T10:17:07.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f46740ce600 0x7f46740cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f46740ce600 0x7f46740cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54614/0 (socket says 172.21.15.133:54614) 2024-04-25T10:17:07.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f4672d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f467415d9a0 0x7f46740d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55534/0 (socket says 172.21.15.133:55534) 2024-04-25T10:17:07.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467259c640 1 -- 172.21.15.133:0/2498331680 learned_addr learned my addr 172.21.15.133:0/2498331680 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:07.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467259c640 1 -- 172.21.15.133:0/2498331680 >> v2:172.21.15.162:3300/0 conn(0x7f46740d1920 msgr2=0x7f46740d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:07.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467259c640 1 --2- 172.21.15.133:0/2498331680 >> v2:172.21.15.162:3300/0 conn(0x7f46740d1920 0x7f46740d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467259c640 1 -- 172.21.15.133:0/2498331680 >> v1:172.21.15.133:6789/0 conn(0x7f467415d9a0 legacy=0x7f46740d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f4671d9b640 1 -- 172.21.15.133:0/2498331680 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656712809 0 0) 0x7f467415ebd0 con 0x7f467415d9a0 2024-04-25T10:17:07.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467259c640 1 -- 172.21.15.133:0/2498331680 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46740d0610 con 0x7f46740ce600 2024-04-25T10:17:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f467259c640 1 --2- 172.21.15.133:0/2498331680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46740ce600 0x7f46740cea90 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f467408ccc0 tx=0x7f465c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c537588fd45462c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.092+0000 7f4671d9b640 1 -- 172.21.15.133:0/2498331680 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0b0070 con 0x7f46740ce600 2024-04-25T10:17:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f4671d9b640 1 -- 172.21.15.133:0/2498331680 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c0ab600 con 0x7f46740ce600 2024-04-25T10:17:07.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f4671d9b640 1 -- 172.21.15.133:0/2498331680 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0ab9d0 con 0x7f46740ce600 2024-04-25T10:17:07.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 -- 172.21.15.133:0/2498331680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46740ce600 msgr2=0x7f46740cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 --2- 172.21.15.133:0/2498331680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46740ce600 0x7f46740cea90 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f467408ccc0 tx=0x7f465c01c490 comp rx=0 tx=0).stop 2024-04-25T10:17:07.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 -- 172.21.15.133:0/2498331680 shutdown_connections 2024-04-25T10:17:07.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 --2- 172.21.15.133:0/2498331680 >> v2:172.21.15.162:3300/0 conn(0x7f46740d1920 0x7f46740d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 --2- 172.21.15.133:0/2498331680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46740ce600 0x7f46740cea90 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 -- 172.21.15.133:0/2498331680 >> 172.21.15.133:0/2498331680 conn(0x7f467406bf80 msgr2=0x7f467406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:07.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 -- 172.21.15.133:0/2498331680 shutdown_connections 2024-04-25T10:17:07.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 -- 172.21.15.133:0/2498331680 wait complete. 2024-04-25T10:17:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 Processor -- start 2024-04-25T10:17:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 -- start start 2024-04-25T10:17:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46740ce600 0x7f4674157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46740d1920 0x7f46740cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f467415d9a0 0x7f4674157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467415e960 con 0x7f46740d1920 2024-04-25T10:17:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467415e7e0 con 0x7f46740ce600 2024-04-25T10:17:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467416a7e0 con 0x7f467415d9a0 2024-04-25T10:17:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46740d1920 0x7f46740cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46740d1920 0x7f46740cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54622/0 (socket says 172.21.15.133:54622) 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467259c640 1 -- 172.21.15.133:0/206803018 learned_addr learned my addr 172.21.15.133:0/206803018 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467259c640 1 -- 172.21.15.133:0/206803018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f467415d9a0 msgr2=0x7f4674157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467259c640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f467415d9a0 0x7f4674157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467259c640 1 -- 172.21.15.133:0/206803018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46740ce600 msgr2=0x7f4674157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467259c640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46740ce600 0x7f4674157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467259c640 1 -- 172.21.15.133:0/206803018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46740cd7c0 con 0x7f46740d1920 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467259c640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46740d1920 0x7f46740cd280 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f465c0ae040 tx=0x7f465c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f466b7fe640 1 -- 172.21.15.133:0/206803018 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0b0040 con 0x7f46740d1920 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f466b7fe640 1 -- 172.21.15.133:0/206803018 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c0b9070 con 0x7f46740d1920 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 -- 172.21.15.133:0/206803018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46740c8460 con 0x7f46740d1920 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f466b7fe640 1 -- 172.21.15.133:0/206803018 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0b48a0 con 0x7f46740d1920 2024-04-25T10:17:07.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.096+0000 7f467903e640 1 -- 172.21.15.133:0/206803018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46740c8a20 con 0x7f46740d1920 2024-04-25T10:17:07.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.100+0000 7f467903e640 1 -- 172.21.15.133:0/206803018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4674068e80 con 0x7f46740d1920 2024-04-25T10:17:07.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.100+0000 7f466b7fe640 1 -- 172.21.15.133:0/206803018 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f465c0ab4c0 con 0x7f46740d1920 2024-04-25T10:17:07.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.100+0000 7f466b7fe640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4654066f60 0x7f4654069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:07.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.100+0000 7f466b7fe640 1 -- 172.21.15.133:0/206803018 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f465c11efb0 con 0x7f46740d1920 2024-04-25T10:17:07.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.100+0000 7f4672d9d640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4654066f60 0x7f4654069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:07.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.100+0000 7f4672d9d640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4654066f60 0x7f4654069420 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f46740b4d30 tx=0x7f4664007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:07.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.104+0000 7f466b7fe640 1 -- 172.21.15.133:0/206803018 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f465c016110 con 0x7f46740d1920 2024-04-25T10:17:07.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:07.300+0000 7f467903e640 1 -- 172.21.15.133:0/206803018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f46740cdf00 con 0x7f46740d1920 2024-04-25T10:17:08.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.276+0000 7f466b7fe640 1 -- 172.21.15.133:0/206803018 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v111) v1 ==== 118+0+0 (secure 0 0 0) 0x7f465c0efe30 con 0x7f46740d1920 2024-04-25T10:17:08.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 -- 172.21.15.133:0/206803018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4654066f60 msgr2=0x7f4654069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4654066f60 0x7f4654069420 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f46740b4d30 tx=0x7f4664007cf0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 -- 172.21.15.133:0/206803018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46740d1920 msgr2=0x7f46740cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46740d1920 0x7f46740cd280 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f465c0ae040 tx=0x7f465c019230 comp rx=0 tx=0).stop 2024-04-25T10:17:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 -- 172.21.15.133:0/206803018 shutdown_connections 2024-04-25T10:17:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f467415d9a0 0x7f4674157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4654066f60 0x7f4654069420 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46740d1920 0x7f46740cd280 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 --2- 172.21.15.133:0/206803018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46740ce600 0x7f4674157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 -- 172.21.15.133:0/206803018 >> 172.21.15.133:0/206803018 conn(0x7f467406bf80 msgr2=0x7f4674167f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 -- 172.21.15.133:0/206803018 shutdown_connections 2024-04-25T10:17:08.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.280+0000 7f467903e640 1 -- 172.21.15.133:0/206803018 wait complete. 2024-04-25T10:17:08.304 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:17:08.304 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:17:08.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.428+0000 7fd2f13ab640 1 Processor -- start 2024-04-25T10:17:08.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2f13ab640 1 -- start start 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2f13ab640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2ec143ea0 0x7fd2ec0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2f13ab640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2ec06eac0 0x7fd2ec0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2f13ab640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ec15ac50 con 0x7fd2ec0b2150 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2f13ab640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ec0cd180 con 0x7fd2ec06eac0 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2f13ab640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec15aed0 con 0x7fd2ec143ea0 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2eb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2ec06eac0 0x7fd2ec0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2eb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2ec06eac0 0x7fd2ec0b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54626/0 (socket says 172.21.15.133:54626) 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2eb7fe640 1 -- 172.21.15.133:0/1705381091 learned_addr learned my addr 172.21.15.133:0/1705381091 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2eaffd640 1 --2- 172.21.15.133:0/1705381091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2ec143ea0 0x7fd2ec0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2eb7fe640 1 -- 172.21.15.133:0/1705381091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2ec143ea0 msgr2=0x7fd2ec0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:08.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2eb7fe640 1 --2- 172.21.15.133:0/1705381091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2ec143ea0 0x7fd2ec0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2eb7fe640 1 -- 172.21.15.133:0/1705381091 >> v1:172.21.15.162:6789/0 conn(0x7fd2ec0b2150 legacy=0x7fd2ec06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:08.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2eb7fe640 1 -- 172.21.15.133:0/1705381091 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec15b1b0 con 0x7fd2ec06eac0 2024-04-25T10:17:08.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2eaffd640 1 --2- 172.21.15.133:0/1705381091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2ec143ea0 0x7fd2ec0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:08.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.432+0000 7fd2eb7fe640 1 --2- 172.21.15.133:0/1705381091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ec06eac0 0x7fd2ec0b06d0 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009e50 tx=0x7fd2e401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c304f148e6e930fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:08.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2ea7fc640 1 -- 172.21.15.133:0/1705381091 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40a55a0 con 0x7fd2ec06eac0 2024-04-25T10:17:08.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2ea7fc640 1 -- 172.21.15.133:0/1705381091 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e40a5740 con 0x7fd2ec06eac0 2024-04-25T10:17:08.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2ea7fc640 1 -- 172.21.15.133:0/1705381091 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40a5b10 con 0x7fd2ec06eac0 2024-04-25T10:17:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1705381091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ec06eac0 msgr2=0x7fd2ec0b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 --2- 172.21.15.133:0/1705381091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ec06eac0 0x7fd2ec0b06d0 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009e50 tx=0x7fd2e401c810 comp rx=0 tx=0).stop 2024-04-25T10:17:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1705381091 shutdown_connections 2024-04-25T10:17:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 --2- 172.21.15.133:0/1705381091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ec06eac0 0x7fd2ec0b06d0 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 --2- 172.21.15.133:0/1705381091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2ec143ea0 0x7fd2ec0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1705381091 >> 172.21.15.133:0/1705381091 conn(0x7fd2ec066f70 msgr2=0x7fd2ec06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:08.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1705381091 shutdown_connections 2024-04-25T10:17:08.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1705381091 wait complete. 2024-04-25T10:17:08.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 Processor -- start 2024-04-25T10:17:08.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 -- start start 2024-04-25T10:17:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2ec06eac0 0x7fd2ec13cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2ec0b2150 0x7fd2ec0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:08.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ec143ea0 0x7fd2ec0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec167550 con 0x7fd2ec143ea0 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1673d0 con 0x7fd2ec0b2150 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec1676d0 con 0x7fd2ec06eac0 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2eb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ec143ea0 0x7fd2ec0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2eb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ec143ea0 0x7fd2ec0cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54630/0 (socket says 172.21.15.133:54630) 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2eb7fe640 1 -- 172.21.15.133:0/1948848058 learned_addr learned my addr 172.21.15.133:0/1948848058 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2eaffd640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2ec06eac0 0x7fd2ec13cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2e27fc640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2ec0b2150 0x7fd2ec0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2eb7fe640 1 -- 172.21.15.133:0/1948848058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2ec06eac0 msgr2=0x7fd2ec13cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2eb7fe640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2ec06eac0 0x7fd2ec13cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2eb7fe640 1 -- 172.21.15.133:0/1948848058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2ec0b2150 msgr2=0x7fd2ec0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2eb7fe640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2ec0b2150 0x7fd2ec0cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2eb7fe640 1 -- 172.21.15.133:0/1948848058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec1f70a0 con 0x7fd2ec143ea0 2024-04-25T10:17:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2eb7fe640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ec143ea0 0x7fd2ec0cc480 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40af040 tx=0x7fd2e40195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2e3fff640 1 -- 172.21.15.133:0/1948848058 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40bc020 con 0x7fd2ec143ea0 2024-04-25T10:17:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2e3fff640 1 -- 172.21.15.133:0/1948848058 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e40b8670 con 0x7fd2ec143ea0 2024-04-25T10:17:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1948848058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec13fb10 con 0x7fd2ec143ea0 2024-04-25T10:17:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2e3fff640 1 -- 172.21.15.133:0/1948848058 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4009510 con 0x7fd2ec143ea0 2024-04-25T10:17:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.436+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1948848058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec1f7600 con 0x7fd2ec143ea0 2024-04-25T10:17:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.440+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1948848058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ec068e80 con 0x7fd2ec143ea0 2024-04-25T10:17:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.440+0000 7fd2e3fff640 1 -- 172.21.15.133:0/1948848058 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2e40bf020 con 0x7fd2ec143ea0 2024-04-25T10:17:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.440+0000 7fd2e3fff640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2cc066f60 0x7fd2cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.440+0000 7fd2e3fff640 1 -- 172.21.15.133:0/1948848058 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2e41201f0 con 0x7fd2ec143ea0 2024-04-25T10:17:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.440+0000 7fd2eaffd640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2cc066f60 0x7fd2cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.440+0000 7fd2eaffd640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2cc066f60 0x7fd2cc069420 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00a9a0 tx=0x7fd2dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:08.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.444+0000 7fd2e3fff640 1 -- 172.21.15.133:0/1948848058 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2e40c6030 con 0x7fd2ec143ea0 2024-04-25T10:17:08.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:08.644+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1948848058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd2ec1435d0 con 0x7fd2ec143ea0 2024-04-25T10:17:09.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.292+0000 7fd2e3fff640 1 -- 172.21.15.133:0/1948848058 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v112) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd2ec1f70a0 con 0x7fd2ec143ea0 2024-04-25T10:17:09.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1948848058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2cc066f60 msgr2=0x7fd2cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2cc066f60 0x7fd2cc069420 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00a9a0 tx=0x7fd2dc008040 comp rx=0 tx=0).stop 2024-04-25T10:17:09.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1948848058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ec143ea0 msgr2=0x7fd2ec0cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ec143ea0 0x7fd2ec0cc480 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40af040 tx=0x7fd2e40195b0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1948848058 shutdown_connections 2024-04-25T10:17:09.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd2cc066f60 0x7fd2cc069420 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ec143ea0 0x7fd2ec0cc480 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd2ec0b2150 0x7fd2ec0cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 --2- 172.21.15.133:0/1948848058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2ec06eac0 0x7fd2ec13cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1948848058 >> 172.21.15.133:0/1948848058 conn(0x7fd2ec066f70 msgr2=0x7fd2ec1644b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1948848058 shutdown_connections 2024-04-25T10:17:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.296+0000 7fd2f13ab640 1 -- 172.21.15.133:0/1948848058 wait complete. 2024-04-25T10:17:09.318 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:17:09.319 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:09.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca9b70640 1 Processor -- start 2024-04-25T10:17:09.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca9b70640 1 -- start start 2024-04-25T10:17:09.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca9b70640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdca40d0770 0x7fdca40d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca9b70640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca40ceb20 con 0x7fdca40b87e0 2024-04-25T10:17:09.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca9b70640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca40b30a0 con 0x7fdca415db00 2024-04-25T10:17:09.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca9b70640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca40d1110 con 0x7fdca40d0770 2024-04-25T10:17:09.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca3fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdca40b87e0 0x7fdca40b8c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55542/0 (socket says 172.21.15.133:55542) 2024-04-25T10:17:09.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca3fff640 1 -- 172.21.15.133:0/1545068069 learned_addr learned my addr 172.21.15.133:0/1545068069 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:09.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca2ffd640 1 --2- 172.21.15.133:0/1545068069 >> v2:172.21.15.162:3300/0 conn(0x7fdca40d0770 0x7fdca40d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca27fc640 1 -- 172.21.15.133:0/1545068069 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978117462 0 0) 0x7fdca40ceb20 con 0x7fdca40b87e0 2024-04-25T10:17:09.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca27fc640 1 -- 172.21.15.133:0/1545068069 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc8c003530 con 0x7fdca40b87e0 2024-04-25T10:17:09.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca2ffd640 1 -- 172.21.15.133:0/1545068069 >> v1:172.21.15.162:6789/0 conn(0x7fdca415db00 legacy=0x7fdca40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca2ffd640 1 -- 172.21.15.133:0/1545068069 >> v1:172.21.15.133:6789/0 conn(0x7fdca40b87e0 legacy=0x7fdca40b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.440+0000 7fdca2ffd640 1 -- 172.21.15.133:0/1545068069 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca40b7740 con 0x7fdca40d0770 2024-04-25T10:17:09.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca2ffd640 1 --2- 172.21.15.133:0/1545068069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca40d0770 0x7fdca40d0bd0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdc980097c0 tx=0x7fdc9801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20b6444f39a4d0b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca27fc640 1 -- 172.21.15.133:0/1545068069 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980a7070 con 0x7fdca40d0770 2024-04-25T10:17:09.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca27fc640 1 -- 172.21.15.133:0/1545068069 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc980b0070 con 0x7fdca40d0770 2024-04-25T10:17:09.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca27fc640 1 -- 172.21.15.133:0/1545068069 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980ab870 con 0x7fdca40d0770 2024-04-25T10:17:09.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca9b70640 1 -- 172.21.15.133:0/1545068069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca40d0770 msgr2=0x7fdca40d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca9b70640 1 --2- 172.21.15.133:0/1545068069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca40d0770 0x7fdca40d0bd0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdc980097c0 tx=0x7fdc9801c470 comp rx=0 tx=0).stop 2024-04-25T10:17:09.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca9b70640 1 -- 172.21.15.133:0/1545068069 shutdown_connections 2024-04-25T10:17:09.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca9b70640 1 --2- 172.21.15.133:0/1545068069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca40d0770 0x7fdca40d0bd0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca9b70640 1 -- 172.21.15.133:0/1545068069 >> 172.21.15.133:0/1545068069 conn(0x7fdca40727c0 msgr2=0x7fdca4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:09.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca9b70640 1 -- 172.21.15.133:0/1545068069 shutdown_connections 2024-04-25T10:17:09.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca9b70640 1 -- 172.21.15.133:0/1545068069 wait complete. 2024-04-25T10:17:09.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca9b70640 1 Processor -- start 2024-04-25T10:17:09.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.444+0000 7fdca9b70640 1 -- start start 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca9b70640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca40b87e0 0x7fdca415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca9b70640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdca40d0770 0x7fdca40cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca9b70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca415db00 0x7fdca40cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca9b70640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40b82c0 con 0x7fdca415db00 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca9b70640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40b7fe0 con 0x7fdca40b87e0 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca9b70640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40b7430 con 0x7fdca40d0770 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca415db00 0x7fdca40cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca415db00 0x7fdca40cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54640/0 (socket says 172.21.15.133:54640) 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca3fff640 1 -- 172.21.15.133:0/573665131 learned_addr learned my addr 172.21.15.133:0/573665131 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca2ffd640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdca40d0770 0x7fdca40cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca37fe640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca40b87e0 0x7fdca415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca3fff640 1 -- 172.21.15.133:0/573665131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdca40d0770 msgr2=0x7fdca40cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca3fff640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdca40d0770 0x7fdca40cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca3fff640 1 -- 172.21.15.133:0/573665131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca40b87e0 msgr2=0x7fdca415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca3fff640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca40b87e0 0x7fdca415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca3fff640 1 -- 172.21.15.133:0/573665131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca40cdcc0 con 0x7fdca415db00 2024-04-25T10:17:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca37fe640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca40b87e0 0x7fdca415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca3fff640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca415db00 0x7fdca40cd640 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c480 tx=0x7fdc9c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdc8bfff640 1 -- 172.21.15.133:0/573665131 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c017070 con 0x7fdca415db00 2024-04-25T10:17:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdc8bfff640 1 -- 172.21.15.133:0/573665131 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c005cf0 con 0x7fdca415db00 2024-04-25T10:17:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca9b70640 1 -- 172.21.15.133:0/573665131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca40cdfa0 con 0x7fdca415db00 2024-04-25T10:17:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdc8bfff640 1 -- 172.21.15.133:0/573665131 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0072b0 con 0x7fdca415db00 2024-04-25T10:17:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca9b70640 1 -- 172.21.15.133:0/573665131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca40ce560 con 0x7fdca415db00 2024-04-25T10:17:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca9b70640 1 -- 172.21.15.133:0/573665131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca4068ef0 con 0x7fdca415db00 2024-04-25T10:17:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdc8bfff640 1 -- 172.21.15.133:0/573665131 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc9c01a070 con 0x7fdca415db00 2024-04-25T10:17:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdc8bfff640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc90066f60 0x7fdc90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdc8bfff640 1 -- 172.21.15.133:0/573665131 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdc9c084470 con 0x7fdca415db00 2024-04-25T10:17:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca37fe640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc90066f60 0x7fdc90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.448+0000 7fdca37fe640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc90066f60 0x7fdc90069420 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdc94007fd0 tx=0x7fdc94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.452+0000 7fdc8bfff640 1 -- 172.21.15.133:0/573665131 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc9c055380 con 0x7fdca415db00 2024-04-25T10:17:09.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.644+0000 7fdca9b70640 1 -- 172.21.15.133:0/573665131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdca40b2750 con 0x7fdca415db00 2024-04-25T10:17:09.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.644+0000 7fdc8bfff640 1 -- 172.21.15.133:0/573665131 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4962 (secure 0 0 0) 0x7fdc9c054c80 con 0x7fdca415db00 2024-04-25T10:17:09.646 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:09.647 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":112,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1718839127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1718839127},{"type":"v1","addr":"172.21.15.162:6833","nonce":1718839127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/368827103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":368827103},{"type":"v1","addr":"172.21.15.133:6835","nonce":368827103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1274732295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1274732295},{"type":"v1","addr":"172.21.15.133:6837","nonce":1274732295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:05.607319+0000","modified":"2024-04-25T10:17:08.648334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11502":{"gid":11502,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1744425810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1744425810},{"type":"v1","addr":"172.21.15.162:6835","nonce":1744425810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:17:09.647 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 112 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 -- 172.21.15.133:0/573665131 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc90066f60 msgr2=0x7fdc90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc90066f60 0x7fdc90069420 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdc94007fd0 tx=0x7fdc94009040 comp rx=0 tx=0).stop 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 -- 172.21.15.133:0/573665131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca415db00 msgr2=0x7fdca40cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca415db00 0x7fdca40cd640 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c480 tx=0x7fdc9c00c940 comp rx=0 tx=0).stop 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 -- 172.21.15.133:0/573665131 shutdown_connections 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc90066f60 0x7fdc90069420 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdca415db00 0x7fdca40cd640 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdca40d0770 0x7fdca40cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 --2- 172.21.15.133:0/573665131 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdca40b87e0 0x7fdca415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 -- 172.21.15.133:0/573665131 >> 172.21.15.133:0/573665131 conn(0x7fdca40727c0 msgr2=0x7fdca40d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 -- 172.21.15.133:0/573665131 shutdown_connections 2024-04-25T10:17:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.648+0000 7fdca9b70640 1 -- 172.21.15.133:0/573665131 wait complete. 2024-04-25T10:17:09.665 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:09.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee371e1640 1 Processor -- start 2024-04-25T10:17:09.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee371e1640 1 -- start start 2024-04-25T10:17:09.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee371e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee300d0770 0x7fee300d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee371e1640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee300ceb20 con 0x7fee3015db00 2024-04-25T10:17:09.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee371e1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee300b30a0 con 0x7fee300b87e0 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee371e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee300d1110 con 0x7fee300d0770 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee34f7d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fee3015db00 0x7fee300b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55550/0 (socket says 172.21.15.133:55550) 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee34f7d640 1 -- 172.21.15.133:0/1294859170 learned_addr learned my addr 172.21.15.133:0/1294859170 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2ffff640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee300d0770 0x7fee300d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470123416 0 0) 0x7fee300ceb20 con 0x7fee3015db00 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee10003530 con 0x7fee3015db00 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2546399027 0 0) 0x7fee10003530 con 0x7fee3015db00 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee300d0770 msgr2=0x7fee300d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee300d0770 0x7fee300d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 >> v1:172.21.15.162:6789/0 conn(0x7fee300b87e0 legacy=0x7fee300b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee300b7740 con 0x7fee3015db00 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fee20001680 con 0x7fee3015db00 2024-04-25T10:17:09.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 >> v1:172.21.15.133:6789/0 conn(0x7fee3015db00 legacy=0x7fee300b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee300d0770 0x7fee100039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee10003f70 0x7fee10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee10004d50 0x7fee100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1000ba80 con 0x7fee10004d50 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1000bc30 con 0x7fee10003f70 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1000be30 con 0x7fee300d0770 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee3577e640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee10004d50 0x7fee100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee2ffff640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee10003f70 0x7fee10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee34f7d640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee300d0770 0x7fee100039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee3577e640 1 -- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee300d0770 msgr2=0x7fee100039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee3577e640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee300d0770 0x7fee100039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee3577e640 1 -- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee10003f70 msgr2=0x7fee10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:09.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee3577e640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee10003f70 0x7fee10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee3577e640 1 -- 172.21.15.133:0/1294859170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee10003530 con 0x7fee10004d50 2024-04-25T10:17:09.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.784+0000 7fee34f7d640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee300d0770 0x7fee100039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:09.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee3577e640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee10004d50 0x7fee100051d0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7fee28001780 tx=0x7fee2801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf322f03f2937477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee280013a0 con 0x7fee10004d50 2024-04-25T10:17:09.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee280052f0 con 0x7fee10004d50 2024-04-25T10:17:09.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee2f7fe640 1 -- 172.21.15.133:0/1294859170 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28003840 con 0x7fee10004d50 2024-04-25T10:17:09.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 -- 172.21.15.133:0/1294859170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee10004d50 msgr2=0x7fee100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 --2- 172.21.15.133:0/1294859170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee10004d50 0x7fee100051d0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7fee28001780 tx=0x7fee2801c240 comp rx=0 tx=0).stop 2024-04-25T10:17:09.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee3577e640 1 -- 172.21.15.133:0/1294859170 reap_dead start 2024-04-25T10:17:09.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 -- 172.21.15.133:0/1294859170 shutdown_connections 2024-04-25T10:17:09.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 -- 172.21.15.133:0/1294859170 >> 172.21.15.133:0/1294859170 conn(0x7fee300727c0 msgr2=0x7fee30072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:09.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 -- 172.21.15.133:0/1294859170 shutdown_connections 2024-04-25T10:17:09.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 -- 172.21.15.133:0/1294859170 wait complete. 2024-04-25T10:17:09.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 Processor -- start 2024-04-25T10:17:09.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 -- start start 2024-04-25T10:17:09.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee10004d50 0x7fee301f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee300cc350 0x7fee300ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee300cd220 0x7fee300ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee300b7fe0 con 0x7fee300cd220 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee300b7d00 con 0x7fee300cc350 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee371e1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee300b82c0 con 0x7fee10004d50 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee3577e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee300cd220 0x7fee300ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee3577e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee300cd220 0x7fee300ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54662/0 (socket says 172.21.15.133:54662) 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee3577e640 1 -- 172.21.15.133:0/3365142736 learned_addr learned my addr 172.21.15.133:0/3365142736 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee34f7d640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee10004d50 0x7fee301f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.788+0000 7fee2ffff640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee300cc350 0x7fee300ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee3577e640 1 -- 172.21.15.133:0/3365142736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee10004d50 msgr2=0x7fee301f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee3577e640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee10004d50 0x7fee301f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee3577e640 1 -- 172.21.15.133:0/3365142736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee300cc350 msgr2=0x7fee300ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee3577e640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee300cc350 0x7fee300ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee3577e640 1 -- 172.21.15.133:0/3365142736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee301f43d0 con 0x7fee300cd220 2024-04-25T10:17:09.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee3577e640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee300cd220 0x7fee300ca8f0 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fee28001780 tx=0x7fee10005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee2d7fa640 1 -- 172.21.15.133:0/3365142736 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee280ba020 con 0x7fee300cd220 2024-04-25T10:17:09.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee2d7fa640 1 -- 172.21.15.133:0/3365142736 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee280054a0 con 0x7fee300cd220 2024-04-25T10:17:09.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee371e1640 1 -- 172.21.15.133:0/3365142736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee300c7350 con 0x7fee300cd220 2024-04-25T10:17:09.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee2d7fa640 1 -- 172.21.15.133:0/3365142736 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee280013a0 con 0x7fee300cd220 2024-04-25T10:17:09.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee371e1640 1 -- 172.21.15.133:0/3365142736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee300c78d0 con 0x7fee300cd220 2024-04-25T10:17:09.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee371e1640 1 -- 172.21.15.133:0/3365142736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee30068ef0 con 0x7fee300cd220 2024-04-25T10:17:09.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee2d7fa640 1 -- 172.21.15.133:0/3365142736 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee280bd070 con 0x7fee300cd220 2024-04-25T10:17:09.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee2d7fa640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 0x7fee14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:09.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee2d7fa640 1 -- 172.21.15.133:0/3365142736 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fee2811ea10 con 0x7fee300cd220 2024-04-25T10:17:09.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee34f7d640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 0x7fee14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:09.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.792+0000 7fee34f7d640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 0x7fee14069420 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fee20001210 tx=0x7fee20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:09.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.796+0000 7fee2d7fa640 1 -- 172.21.15.133:0/3365142736 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee301f43d0 con 0x7fee300cd220 2024-04-25T10:17:09.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.956+0000 7fee371e1640 1 -- 172.21.15.133:0/3365142736 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee300b2750 con 0x7fee300cd220 2024-04-25T10:17:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.956+0000 7fee2d7fa640 1 -- 172.21.15.133:0/3365142736 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16976 (secure 0 0 0) 0x7fee300b30a0 con 0x7fee300cd220 2024-04-25T10:17:09.959 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:09.959 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":183,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:07.261559+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:03.841068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:04.604661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 -- 172.21.15.133:0/3365142736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 msgr2=0x7fee14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 0x7fee14069420 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fee20001210 tx=0x7fee20009040 comp rx=0 tx=0).stop 2024-04-25T10:17:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 -- 172.21.15.133:0/3365142736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee300cd220 msgr2=0x7fee300ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee300cd220 0x7fee300ca8f0 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fee28001780 tx=0x7fee10005840 comp rx=0 tx=0).stop 2024-04-25T10:17:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 -- 172.21.15.133:0/3365142736 shutdown_connections 2024-04-25T10:17:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fee14066f60 0x7fee14069420 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee300cd220 0x7fee300ca8f0 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fee300cc350 0x7fee300ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 --2- 172.21.15.133:0/3365142736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fee10004d50 0x7fee301f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 -- 172.21.15.133:0/3365142736 >> 172.21.15.133:0/3365142736 conn(0x7fee300727c0 msgr2=0x7fee3015b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 -- 172.21.15.133:0/3365142736 shutdown_connections 2024-04-25T10:17:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:09.960+0000 7fee371e1640 1 -- 172.21.15.133:0/3365142736 wait complete. 2024-04-25T10:17:09.979 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:17:09.979 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:10.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a60bd1640 1 Processor -- start 2024-04-25T10:17:10.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a60bd1640 1 -- start start 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a60bd1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1a5c152750 0x7f1a5c0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a60bd1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a5c152b30 0x7f1a5c0ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a60bd1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c06d350 con 0x7f1a5c0b17b0 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a60bd1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5c15bf90 con 0x7f1a5c152750 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a60bd1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c06d5d0 con 0x7f1a5c152b30 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a5a59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1a5c152750 0x7f1a5c0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a5a59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1a5c152750 0x7f1a5c0ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54676/0 (socket says 172.21.15.133:54676) 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a5a59c640 1 -- 172.21.15.133:0/1255991339 learned_addr learned my addr 172.21.15.133:0/1255991339 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a59d9b640 1 --2- 172.21.15.133:0/1255991339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a5c152b30 0x7f1a5c0ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a5a59c640 1 -- 172.21.15.133:0/1255991339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a5c152b30 msgr2=0x7f1a5c0ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a5a59c640 1 --2- 172.21.15.133:0/1255991339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a5c152b30 0x7f1a5c0ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a5a59c640 1 -- 172.21.15.133:0/1255991339 >> v1:172.21.15.162:6789/0 conn(0x7f1a5c0b17b0 legacy=0x7f1a5c0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:10.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a5a59c640 1 -- 172.21.15.133:0/1255991339 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c15c230 con 0x7f1a5c152750 2024-04-25T10:17:10.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a5a59c640 1 --2- 172.21.15.133:0/1255991339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a5c152750 0x7f1a5c0ce330 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009870 tx=0x7f1a4c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6321188b972228c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.096+0000 7f1a5959a640 1 -- 172.21.15.133:0/1255991339 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0b0070 con 0x7f1a5c152750 2024-04-25T10:17:10.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5959a640 1 -- 172.21.15.133:0/1255991339 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c0ab600 con 0x7f1a5c152750 2024-04-25T10:17:10.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5959a640 1 -- 172.21.15.133:0/1255991339 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0ab9d0 con 0x7f1a5c152750 2024-04-25T10:17:10.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 -- 172.21.15.133:0/1255991339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a5c152750 msgr2=0x7f1a5c0ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 --2- 172.21.15.133:0/1255991339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a5c152750 0x7f1a5c0ce330 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009870 tx=0x7f1a4c01c340 comp rx=0 tx=0).stop 2024-04-25T10:17:10.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 -- 172.21.15.133:0/1255991339 shutdown_connections 2024-04-25T10:17:10.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 --2- 172.21.15.133:0/1255991339 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a5c152b30 0x7f1a5c0ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 --2- 172.21.15.133:0/1255991339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a5c152750 0x7f1a5c0ce330 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 -- 172.21.15.133:0/1255991339 >> 172.21.15.133:0/1255991339 conn(0x7f1a5c067000 msgr2=0x7f1a5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:10.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 -- 172.21.15.133:0/1255991339 shutdown_connections 2024-04-25T10:17:10.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 -- 172.21.15.133:0/1255991339 wait complete. 2024-04-25T10:17:10.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 Processor -- start 2024-04-25T10:17:10.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 -- start start 2024-04-25T10:17:10.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a5c0b17b0 0x7f1a5c1486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a5c152750 0x7f1a5c148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a5c152b30 0x7f1a5c14d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c1600a0 con 0x7f1a5c152b30 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c15ff20 con 0x7f1a5c152750 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a60bd1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c160220 con 0x7f1a5c0b17b0 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a5c152b30 0x7f1a5c14d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a5c152b30 0x7f1a5c14d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54678/0 (socket says 172.21.15.133:54678) 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5ad9d640 1 -- 172.21.15.133:0/3915320071 learned_addr learned my addr 172.21.15.133:0/3915320071 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5a59c640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a5c0b17b0 0x7f1a5c1486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a59d9b640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a5c152750 0x7f1a5c148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5ad9d640 1 -- 172.21.15.133:0/3915320071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a5c0b17b0 msgr2=0x7f1a5c1486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5ad9d640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a5c0b17b0 0x7f1a5c1486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5ad9d640 1 -- 172.21.15.133:0/3915320071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a5c152750 msgr2=0x7f1a5c148bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5ad9d640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a5c152750 0x7f1a5c148bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5ad9d640 1 -- 172.21.15.133:0/3915320071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0c9c30 con 0x7f1a5c152b30 2024-04-25T10:17:10.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a59d9b640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a5c152750 0x7f1a5c148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:10.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.100+0000 7f1a5ad9d640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a5c152b30 0x7f1a5c14d4a0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400b970 tx=0x7f1a5400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.104+0000 7f1a4affd640 1 -- 172.21.15.133:0/3915320071 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54019070 con 0x7f1a5c152b30 2024-04-25T10:17:10.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.104+0000 7f1a4affd640 1 -- 172.21.15.133:0/3915320071 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5400cb40 con 0x7f1a5c152b30 2024-04-25T10:17:10.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.104+0000 7f1a60bd1640 1 -- 172.21.15.133:0/3915320071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c0c9f10 con 0x7f1a5c152b30 2024-04-25T10:17:10.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.104+0000 7f1a4affd640 1 -- 172.21.15.133:0/3915320071 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54014440 con 0x7f1a5c152b30 2024-04-25T10:17:10.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.104+0000 7f1a60bd1640 1 -- 172.21.15.133:0/3915320071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c0ca4d0 con 0x7f1a5c152b30 2024-04-25T10:17:10.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.104+0000 7f1a60bd1640 1 -- 172.21.15.133:0/3915320071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5c068ef0 con 0x7f1a5c152b30 2024-04-25T10:17:10.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.104+0000 7f1a4affd640 1 -- 172.21.15.133:0/3915320071 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a5401c070 con 0x7f1a5c152b30 2024-04-25T10:17:10.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.104+0000 7f1a4affd640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a3c066f60 0x7f1a3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.104+0000 7f1a4affd640 1 -- 172.21.15.133:0/3915320071 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a54084e70 con 0x7f1a5c152b30 2024-04-25T10:17:10.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.104+0000 7f1a5a59c640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a3c066f60 0x7f1a3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.104+0000 7f1a5a59c640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a3c066f60 0x7f1a3c069420 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0ad000 tx=0x7f1a4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.108+0000 7f1a4affd640 1 -- 172.21.15.133:0/3915320071 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a54055d80 con 0x7f1a5c152b30 2024-04-25T10:17:10.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.264+0000 7f1a60bd1640 1 -- 172.21.15.133:0/3915320071 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a5c06e900 con 0x7f1a3c066f60 2024-04-25T10:17:10.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.276+0000 7f1a4affd640 1 -- 172.21.15.133:0/3915320071 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252493 (secure 0 0 0) 0x7f1a5c06e900 con 0x7f1a3c066f60 2024-04-25T10:17:10.280 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:10.282 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:17:10.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 -- 172.21.15.133:0/3915320071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a3c066f60 msgr2=0x7f1a3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a3c066f60 0x7f1a3c069420 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0ad000 tx=0x7f1a4c008040 comp rx=0 tx=0).stop 2024-04-25T10:17:10.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 -- 172.21.15.133:0/3915320071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a5c152b30 msgr2=0x7f1a5c14d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a5c152b30 0x7f1a5c14d4a0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400b970 tx=0x7f1a5400be30 comp rx=0 tx=0).stop 2024-04-25T10:17:10.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 -- 172.21.15.133:0/3915320071 shutdown_connections 2024-04-25T10:17:10.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1a3c066f60 0x7f1a3c069420 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a5c152b30 0x7f1a5c14d4a0 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a5c152750 0x7f1a5c148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 --2- 172.21.15.133:0/3915320071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1a5c0b17b0 0x7f1a5c1486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 -- 172.21.15.133:0/3915320071 >> 172.21.15.133:0/3915320071 conn(0x7f1a5c067000 msgr2=0x7f1a5c0c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:10.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 -- 172.21.15.133:0/3915320071 shutdown_connections 2024-04-25T10:17:10.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.280+0000 7f1a60bd1640 1 -- 172.21.15.133:0/3915320071 wait complete. 2024-04-25T10:17:10.298 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":639,"stamp":"2024-04-25T10:17:09.997823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":524156,"kb_used_data":19272,"kb_used_omap":12,"kb_used_meta":504755,"kb_avail":754450564,"statfs":{"total":773094113280,"available":772557377536,"internally_reserved":0,"allocated":19734528,"data_stored":9875268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":516869681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.764981"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079359+0000","last_change":"2024-04-25T10:17:06.306658+0000","last_active":"2024-04-25T10:17:08.079359+0000","last_peered":"2024-04-25T10:17:08.079359+0000","last_clean":"2024-04-25T10:17:08.079359+0000","last_became_active":"2024-04-25T10:17:06.305948+0000","last_became_peered":"2024-04-25T10:17:06.305948+0000","last_unstale":"2024-04-25T10:17:08.079359+0000","last_undegraded":"2024-04-25T10:17:08.079359+0000","last_fullsized":"2024-04-25T10:17:08.079359+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:03.718103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077095+0000","last_change":"2024-04-25T10:17:06.299852+0000","last_active":"2024-04-25T10:17:08.077095+0000","last_peered":"2024-04-25T10:17:08.077095+0000","last_clean":"2024-04-25T10:17:08.077095+0000","last_became_active":"2024-04-25T10:17:06.293561+0000","last_became_peered":"2024-04-25T10:17:06.293561+0000","last_unstale":"2024-04-25T10:17:08.077095+0000","last_undegraded":"2024-04-25T10:17:08.077095+0000","last_fullsized":"2024-04-25T10:17:08.077095+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:05.505613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299782+0000","last_change":"2024-04-25T10:17:06.302106+0000","last_active":"2024-04-25T10:17:07.299782+0000","last_peered":"2024-04-25T10:17:07.299782+0000","last_clean":"2024-04-25T10:17:07.299782+0000","last_became_active":"2024-04-25T10:17:06.301792+0000","last_became_peered":"2024-04-25T10:17:06.301792+0000","last_unstale":"2024-04-25T10:17:07.299782+0000","last_undegraded":"2024-04-25T10:17:07.299782+0000","last_fullsized":"2024-04-25T10:17:07.299782+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:34.223680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078111+0000","last_change":"2024-04-25T10:17:06.294925+0000","last_active":"2024-04-25T10:17:08.078111+0000","last_peered":"2024-04-25T10:17:08.078111+0000","last_clean":"2024-04-25T10:17:08.078111+0000","last_became_active":"2024-04-25T10:17:06.294551+0000","last_became_peered":"2024-04-25T10:17:06.294551+0000","last_unstale":"2024-04-25T10:17:08.078111+0000","last_undegraded":"2024-04-25T10:17:08.078111+0000","last_fullsized":"2024-04-25T10:17:08.078111+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:09.737034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282332+0000","last_change":"2024-04-25T10:17:06.296451+0000","last_active":"2024-04-25T10:17:07.282332+0000","last_peered":"2024-04-25T10:17:07.282332+0000","last_clean":"2024-04-25T10:17:07.282332+0000","last_became_active":"2024-04-25T10:17:06.292410+0000","last_became_peered":"2024-04-25T10:17:06.292410+0000","last_unstale":"2024-04-25T10:17:07.282332+0000","last_undegraded":"2024-04-25T10:17:07.282332+0000","last_fullsized":"2024-04-25T10:17:07.282332+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:44.116534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077329+0000","last_change":"2024-04-25T10:17:06.293993+0000","last_active":"2024-04-25T10:17:08.077329+0000","last_peered":"2024-04-25T10:17:08.077329+0000","last_clean":"2024-04-25T10:17:08.077329+0000","last_became_active":"2024-04-25T10:17:06.293546+0000","last_became_peered":"2024-04-25T10:17:06.293546+0000","last_unstale":"2024-04-25T10:17:08.077329+0000","last_undegraded":"2024-04-25T10:17:08.077329+0000","last_fullsized":"2024-04-25T10:17:08.077329+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:06.488061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.281594+0000","last_change":"2024-04-25T10:17:06.294503+0000","last_active":"2024-04-25T10:17:07.281594+0000","last_peered":"2024-04-25T10:17:07.281594+0000","last_clean":"2024-04-25T10:17:07.281594+0000","last_became_active":"2024-04-25T10:17:06.291408+0000","last_became_peered":"2024-04-25T10:17:06.291408+0000","last_unstale":"2024-04-25T10:17:07.281594+0000","last_undegraded":"2024-04-25T10:17:07.281594+0000","last_fullsized":"2024-04-25T10:17:07.281594+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:38.193858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.284681+0000","last_change":"2024-04-25T10:17:06.303945+0000","last_active":"2024-04-25T10:17:07.284681+0000","last_peered":"2024-04-25T10:17:07.284681+0000","last_clean":"2024-04-25T10:17:07.284681+0000","last_became_active":"2024-04-25T10:17:06.303488+0000","last_became_peered":"2024-04-25T10:17:06.303488+0000","last_unstale":"2024-04-25T10:17:07.284681+0000","last_undegraded":"2024-04-25T10:17:07.284681+0000","last_fullsized":"2024-04-25T10:17:07.284681+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:11.256331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637957+0000","last_change":"2024-04-25T10:17:06.306233+0000","last_active":"2024-04-25T10:17:07.637957+0000","last_peered":"2024-04-25T10:17:07.637957+0000","last_clean":"2024-04-25T10:17:07.637957+0000","last_became_active":"2024-04-25T10:17:06.305499+0000","last_became_peered":"2024-04-25T10:17:06.305499+0000","last_unstale":"2024-04-25T10:17:07.637957+0000","last_undegraded":"2024-04-25T10:17:07.637957+0000","last_fullsized":"2024-04-25T10:17:07.637957+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:49.319459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287499+0000","last_change":"2024-04-25T10:17:06.308358+0000","last_active":"2024-04-25T10:17:07.287499+0000","last_peered":"2024-04-25T10:17:07.287499+0000","last_clean":"2024-04-25T10:17:07.287499+0000","last_became_active":"2024-04-25T10:17:06.307999+0000","last_became_peered":"2024-04-25T10:17:06.307999+0000","last_unstale":"2024-04-25T10:17:07.287499+0000","last_undegraded":"2024-04-25T10:17:07.287499+0000","last_fullsized":"2024-04-25T10:17:07.287499+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:23.198544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.300000+0000","last_change":"2024-04-25T10:17:06.308623+0000","last_active":"2024-04-25T10:17:07.300000+0000","last_peered":"2024-04-25T10:17:07.300000+0000","last_clean":"2024-04-25T10:17:07.300000+0000","last_became_active":"2024-04-25T10:17:06.308353+0000","last_became_peered":"2024-04-25T10:17:06.308353+0000","last_unstale":"2024-04-25T10:17:07.300000+0000","last_undegraded":"2024-04-25T10:17:07.300000+0000","last_fullsized":"2024-04-25T10:17:07.300000+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:15.841404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282246+0000","last_change":"2024-04-25T10:17:06.281476+0000","last_active":"2024-04-25T10:17:07.282246+0000","last_peered":"2024-04-25T10:17:07.282246+0000","last_clean":"2024-04-25T10:17:07.282246+0000","last_became_active":"2024-04-25T10:17:06.280613+0000","last_became_peered":"2024-04-25T10:17:06.280613+0000","last_unstale":"2024-04-25T10:17:07.282246+0000","last_undegraded":"2024-04-25T10:17:07.282246+0000","last_fullsized":"2024-04-25T10:17:07.282246+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:00.548089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078081+0000","last_change":"2024-04-25T10:17:06.309849+0000","last_active":"2024-04-25T10:17:08.078081+0000","last_peered":"2024-04-25T10:17:08.078081+0000","last_clean":"2024-04-25T10:17:08.078081+0000","last_became_active":"2024-04-25T10:17:06.309566+0000","last_became_peered":"2024-04-25T10:17:06.309566+0000","last_unstale":"2024-04-25T10:17:08.078081+0000","last_undegraded":"2024-04-25T10:17:08.078081+0000","last_fullsized":"2024-04-25T10:17:08.078081+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:04.975165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286660+0000","last_change":"2024-04-25T10:17:06.316432+0000","last_active":"2024-04-25T10:17:07.286660+0000","last_peered":"2024-04-25T10:17:07.286660+0000","last_clean":"2024-04-25T10:17:07.286660+0000","last_became_active":"2024-04-25T10:17:06.316148+0000","last_became_peered":"2024-04-25T10:17:06.316148+0000","last_unstale":"2024-04-25T10:17:07.286660+0000","last_undegraded":"2024-04-25T10:17:07.286660+0000","last_fullsized":"2024-04-25T10:17:07.286660+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:49.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285678+0000","last_change":"2024-04-25T10:17:06.308978+0000","last_active":"2024-04-25T10:17:07.285678+0000","last_peered":"2024-04-25T10:17:07.285678+0000","last_clean":"2024-04-25T10:17:07.285678+0000","last_became_active":"2024-04-25T10:17:06.308696+0000","last_became_peered":"2024-04-25T10:17:06.308696+0000","last_unstale":"2024-04-25T10:17:07.285678+0000","last_undegraded":"2024-04-25T10:17:07.285678+0000","last_fullsized":"2024-04-25T10:17:07.285678+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.443320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299399+0000","last_change":"2024-04-25T10:17:06.303492+0000","last_active":"2024-04-25T10:17:07.299399+0000","last_peered":"2024-04-25T10:17:07.299399+0000","last_clean":"2024-04-25T10:17:07.299399+0000","last_became_active":"2024-04-25T10:17:06.303212+0000","last_became_peered":"2024-04-25T10:17:06.303212+0000","last_unstale":"2024-04-25T10:17:07.299399+0000","last_undegraded":"2024-04-25T10:17:07.299399+0000","last_fullsized":"2024-04-25T10:17:07.299399+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:44.715314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.638080+0000","last_change":"2024-04-25T10:17:06.298837+0000","last_active":"2024-04-25T10:17:07.638080+0000","last_peered":"2024-04-25T10:17:07.638080+0000","last_clean":"2024-04-25T10:17:07.638080+0000","last_became_active":"2024-04-25T10:17:06.298056+0000","last_became_peered":"2024-04-25T10:17:06.298056+0000","last_unstale":"2024-04-25T10:17:07.638080+0000","last_undegraded":"2024-04-25T10:17:07.638080+0000","last_fullsized":"2024-04-25T10:17:07.638080+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:37.978479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637675+0000","last_change":"2024-04-25T10:17:06.295992+0000","last_active":"2024-04-25T10:17:07.637675+0000","last_peered":"2024-04-25T10:17:07.637675+0000","last_clean":"2024-04-25T10:17:07.637675+0000","last_became_active":"2024-04-25T10:17:06.295172+0000","last_became_peered":"2024-04-25T10:17:06.295172+0000","last_unstale":"2024-04-25T10:17:07.637675+0000","last_undegraded":"2024-04-25T10:17:07.637675+0000","last_fullsized":"2024-04-25T10:17:07.637675+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:45.792654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287442+0000","last_change":"2024-04-25T10:17:06.303425+0000","last_active":"2024-04-25T10:17:07.287442+0000","last_peered":"2024-04-25T10:17:07.287442+0000","last_clean":"2024-04-25T10:17:07.287442+0000","last_became_active":"2024-04-25T10:17:06.303023+0000","last_became_peered":"2024-04-25T10:17:06.303023+0000","last_unstale":"2024-04-25T10:17:07.287442+0000","last_undegraded":"2024-04-25T10:17:07.287442+0000","last_fullsized":"2024-04-25T10:17:07.287442+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.812969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286529+0000","last_change":"2024-04-25T10:17:06.298783+0000","last_active":"2024-04-25T10:17:07.286529+0000","last_peered":"2024-04-25T10:17:07.286529+0000","last_clean":"2024-04-25T10:17:07.286529+0000","last_became_active":"2024-04-25T10:17:06.298491+0000","last_became_peered":"2024-04-25T10:17:06.298491+0000","last_unstale":"2024-04-25T10:17:07.286529+0000","last_undegraded":"2024-04-25T10:17:07.286529+0000","last_fullsized":"2024-04-25T10:17:07.286529+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:11.986523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285772+0000","last_change":"2024-04-25T10:17:06.309141+0000","last_active":"2024-04-25T10:17:07.285772+0000","last_peered":"2024-04-25T10:17:07.285772+0000","last_clean":"2024-04-25T10:17:07.285772+0000","last_became_active":"2024-04-25T10:17:06.308818+0000","last_became_peered":"2024-04-25T10:17:06.308818+0000","last_unstale":"2024-04-25T10:17:07.285772+0000","last_undegraded":"2024-04-25T10:17:07.285772+0000","last_fullsized":"2024-04-25T10:17:07.285772+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:53.397605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285886+0000","last_change":"2024-04-25T10:17:06.312702+0000","last_active":"2024-04-25T10:17:07.285886+0000","last_peered":"2024-04-25T10:17:07.285886+0000","last_clean":"2024-04-25T10:17:07.285886+0000","last_became_active":"2024-04-25T10:17:06.312379+0000","last_became_peered":"2024-04-25T10:17:06.312379+0000","last_unstale":"2024-04-25T10:17:07.285886+0000","last_undegraded":"2024-04-25T10:17:07.285886+0000","last_fullsized":"2024-04-25T10:17:07.285886+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:02.587548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076441+0000","last_change":"2024-04-25T10:17:06.312878+0000","last_active":"2024-04-25T10:17:08.076441+0000","last_peered":"2024-04-25T10:17:08.076441+0000","last_clean":"2024-04-25T10:17:08.076441+0000","last_became_active":"2024-04-25T10:17:06.312559+0000","last_became_peered":"2024-04-25T10:17:06.312559+0000","last_unstale":"2024-04-25T10:17:08.076441+0000","last_undegraded":"2024-04-25T10:17:08.076441+0000","last_fullsized":"2024-04-25T10:17:08.076441+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:01.695656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.281057+0000","last_change":"2024-04-25T10:17:06.300742+0000","last_active":"2024-04-25T10:17:07.281057+0000","last_peered":"2024-04-25T10:17:07.281057+0000","last_clean":"2024-04-25T10:17:07.281057+0000","last_became_active":"2024-04-25T10:17:06.300393+0000","last_became_peered":"2024-04-25T10:17:06.300393+0000","last_unstale":"2024-04-25T10:17:07.281057+0000","last_undegraded":"2024-04-25T10:17:07.281057+0000","last_fullsized":"2024-04-25T10:17:07.281057+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:45.560858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.638164+0000","last_change":"2024-04-25T10:17:06.295132+0000","last_active":"2024-04-25T10:17:07.638164+0000","last_peered":"2024-04-25T10:17:07.638164+0000","last_clean":"2024-04-25T10:17:07.638164+0000","last_became_active":"2024-04-25T10:17:06.294396+0000","last_became_peered":"2024-04-25T10:17:06.294396+0000","last_unstale":"2024-04-25T10:17:07.638164+0000","last_undegraded":"2024-04-25T10:17:07.638164+0000","last_fullsized":"2024-04-25T10:17:07.638164+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:02.629892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079143+0000","last_change":"2024-04-25T10:17:06.306354+0000","last_active":"2024-04-25T10:17:08.079143+0000","last_peered":"2024-04-25T10:17:08.079143+0000","last_clean":"2024-04-25T10:17:08.079143+0000","last_became_active":"2024-04-25T10:17:06.306023+0000","last_became_peered":"2024-04-25T10:17:06.306023+0000","last_unstale":"2024-04-25T10:17:08.079143+0000","last_undegraded":"2024-04-25T10:17:08.079143+0000","last_fullsized":"2024-04-25T10:17:08.079143+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:56.696520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.300180+0000","last_change":"2024-04-25T10:17:06.300866+0000","last_active":"2024-04-25T10:17:07.300180+0000","last_peered":"2024-04-25T10:17:07.300180+0000","last_clean":"2024-04-25T10:17:07.300180+0000","last_became_active":"2024-04-25T10:17:06.300579+0000","last_became_peered":"2024-04-25T10:17:06.300579+0000","last_unstale":"2024-04-25T10:17:07.300180+0000","last_undegraded":"2024-04-25T10:17:07.300180+0000","last_fullsized":"2024-04-25T10:17:07.300180+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:18.541432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282909+0000","last_change":"2024-04-25T10:17:06.291974+0000","last_active":"2024-04-25T10:17:07.282909+0000","last_peered":"2024-04-25T10:17:07.282909+0000","last_clean":"2024-04-25T10:17:07.282909+0000","last_became_active":"2024-04-25T10:17:06.290630+0000","last_became_peered":"2024-04-25T10:17:06.290630+0000","last_unstale":"2024-04-25T10:17:07.282909+0000","last_undegraded":"2024-04-25T10:17:07.282909+0000","last_fullsized":"2024-04-25T10:17:07.282909+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:51.498124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637298+0000","last_change":"2024-04-25T10:17:06.280992+0000","last_active":"2024-04-25T10:17:07.637298+0000","last_peered":"2024-04-25T10:17:07.637298+0000","last_clean":"2024-04-25T10:17:07.637298+0000","last_became_active":"2024-04-25T10:17:06.278612+0000","last_became_peered":"2024-04-25T10:17:06.278612+0000","last_unstale":"2024-04-25T10:17:07.637298+0000","last_undegraded":"2024-04-25T10:17:07.637298+0000","last_fullsized":"2024-04-25T10:17:07.637298+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:49.221348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285350+0000","last_change":"2024-04-25T10:17:06.295566+0000","last_active":"2024-04-25T10:17:07.285350+0000","last_peered":"2024-04-25T10:17:07.285350+0000","last_clean":"2024-04-25T10:17:07.285350+0000","last_became_active":"2024-04-25T10:17:06.294921+0000","last_became_peered":"2024-04-25T10:17:06.294921+0000","last_unstale":"2024-04-25T10:17:07.285350+0000","last_undegraded":"2024-04-25T10:17:07.285350+0000","last_fullsized":"2024-04-25T10:17:07.285350+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:46.848723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077285+0000","last_change":"2024-04-25T10:17:06.307165+0000","last_active":"2024-04-25T10:17:08.077285+0000","last_peered":"2024-04-25T10:17:08.077285+0000","last_clean":"2024-04-25T10:17:08.077285+0000","last_became_active":"2024-04-25T10:17:06.306910+0000","last_became_peered":"2024-04-25T10:17:06.306910+0000","last_unstale":"2024-04-25T10:17:08.077285+0000","last_undegraded":"2024-04-25T10:17:08.077285+0000","last_fullsized":"2024-04-25T10:17:08.077285+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:05.505613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077146+0000","last_change":"2024-04-25T10:17:06.286675+0000","last_active":"2024-04-25T10:17:08.077146+0000","last_peered":"2024-04-25T10:17:08.077146+0000","last_clean":"2024-04-25T10:17:08.077146+0000","last_became_active":"2024-04-25T10:17:06.285954+0000","last_became_peered":"2024-04-25T10:17:06.285954+0000","last_unstale":"2024-04-25T10:17:08.077146+0000","last_undegraded":"2024-04-25T10:17:08.077146+0000","last_fullsized":"2024-04-25T10:17:08.077146+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:42.905954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077181+0000","last_change":"2024-04-25T10:17:06.302689+0000","last_active":"2024-04-25T10:17:08.077181+0000","last_peered":"2024-04-25T10:17:08.077181+0000","last_clean":"2024-04-25T10:17:08.077181+0000","last_became_active":"2024-04-25T10:17:06.302063+0000","last_became_peered":"2024-04-25T10:17:06.302063+0000","last_unstale":"2024-04-25T10:17:08.077181+0000","last_undegraded":"2024-04-25T10:17:08.077181+0000","last_fullsized":"2024-04-25T10:17:08.077181+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:06.488061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"182'4","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286596+0000","last_change":"2024-04-25T10:17:05.274637+0000","last_active":"2024-04-25T10:17:07.286596+0000","last_peered":"2024-04-25T10:17:07.286596+0000","last_clean":"2024-04-25T10:17:07.286596+0000","last_became_active":"2024-04-25T10:17:05.273900+0000","last_became_peered":"2024-04-25T10:17:05.273900+0000","last_unstale":"2024-04-25T10:17:07.286596+0000","last_undegraded":"2024-04-25T10:17:07.286596+0000","last_fullsized":"2024-04-25T10:17:07.286596+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:30.759665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078367+0000","last_change":"2024-04-25T10:17:06.305742+0000","last_active":"2024-04-25T10:17:08.078367+0000","last_peered":"2024-04-25T10:17:08.078367+0000","last_clean":"2024-04-25T10:17:08.078367+0000","last_became_active":"2024-04-25T10:17:06.305363+0000","last_became_peered":"2024-04-25T10:17:06.305363+0000","last_unstale":"2024-04-25T10:17:08.078367+0000","last_undegraded":"2024-04-25T10:17:08.078367+0000","last_fullsized":"2024-04-25T10:17:08.078367+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:59.006495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079156+0000","last_change":"2024-04-25T10:17:05.280219+0000","last_active":"2024-04-25T10:17:08.079156+0000","last_peered":"2024-04-25T10:17:08.079156+0000","last_clean":"2024-04-25T10:17:08.079156+0000","last_became_active":"2024-04-25T10:17:05.269919+0000","last_became_peered":"2024-04-25T10:17:05.269919+0000","last_unstale":"2024-04-25T10:17:08.079156+0000","last_undegraded":"2024-04-25T10:17:08.079156+0000","last_fullsized":"2024-04-25T10:17:08.079156+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:50.572814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286244+0000","last_change":"2024-04-25T10:17:06.312559+0000","last_active":"2024-04-25T10:17:07.286244+0000","last_peered":"2024-04-25T10:17:07.286244+0000","last_clean":"2024-04-25T10:17:07.286244+0000","last_became_active":"2024-04-25T10:17:06.312115+0000","last_became_peered":"2024-04-25T10:17:06.312115+0000","last_unstale":"2024-04-25T10:17:07.286244+0000","last_undegraded":"2024-04-25T10:17:07.286244+0000","last_fullsized":"2024-04-25T10:17:07.286244+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:27.892291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076748+0000","last_change":"2024-04-25T10:17:05.252648+0000","last_active":"2024-04-25T10:17:08.076748+0000","last_peered":"2024-04-25T10:17:08.076748+0000","last_clean":"2024-04-25T10:17:08.076748+0000","last_became_active":"2024-04-25T10:17:05.252325+0000","last_became_peered":"2024-04-25T10:17:05.252325+0000","last_unstale":"2024-04-25T10:17:08.076748+0000","last_undegraded":"2024-04-25T10:17:08.076748+0000","last_fullsized":"2024-04-25T10:17:08.076748+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:41.072995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282411+0000","last_change":"2024-04-25T10:17:06.295782+0000","last_active":"2024-04-25T10:17:07.282411+0000","last_peered":"2024-04-25T10:17:07.282411+0000","last_clean":"2024-04-25T10:17:07.282411+0000","last_became_active":"2024-04-25T10:17:06.294875+0000","last_became_peered":"2024-04-25T10:17:06.294875+0000","last_unstale":"2024-04-25T10:17:07.282411+0000","last_undegraded":"2024-04-25T10:17:07.282411+0000","last_fullsized":"2024-04-25T10:17:07.282411+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:27.268666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637396+0000","last_change":"2024-04-25T10:17:05.273926+0000","last_active":"2024-04-25T10:17:07.637396+0000","last_peered":"2024-04-25T10:17:07.637396+0000","last_clean":"2024-04-25T10:17:07.637396+0000","last_became_active":"2024-04-25T10:17:05.273604+0000","last_became_peered":"2024-04-25T10:17:05.273604+0000","last_unstale":"2024-04-25T10:17:07.637396+0000","last_undegraded":"2024-04-25T10:17:07.637396+0000","last_fullsized":"2024-04-25T10:17:07.637396+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:13.306138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286801+0000","last_change":"2024-04-25T10:17:06.304864+0000","last_active":"2024-04-25T10:17:07.286801+0000","last_peered":"2024-04-25T10:17:07.286801+0000","last_clean":"2024-04-25T10:17:07.286801+0000","last_became_active":"2024-04-25T10:17:06.304534+0000","last_became_peered":"2024-04-25T10:17:06.304534+0000","last_unstale":"2024-04-25T10:17:07.286801+0000","last_undegraded":"2024-04-25T10:17:07.286801+0000","last_fullsized":"2024-04-25T10:17:07.286801+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:49.803168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078219+0000","last_change":"2024-04-25T10:17:05.279953+0000","last_active":"2024-04-25T10:17:08.078219+0000","last_peered":"2024-04-25T10:17:08.078219+0000","last_clean":"2024-04-25T10:17:08.078219+0000","last_became_active":"2024-04-25T10:17:05.277343+0000","last_became_peered":"2024-04-25T10:17:05.277343+0000","last_unstale":"2024-04-25T10:17:08.078219+0000","last_undegraded":"2024-04-25T10:17:08.078219+0000","last_fullsized":"2024-04-25T10:17:08.078219+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:53.402593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287147+0000","last_change":"2024-04-25T10:17:06.305581+0000","last_active":"2024-04-25T10:17:07.287147+0000","last_peered":"2024-04-25T10:17:07.287147+0000","last_clean":"2024-04-25T10:17:07.287147+0000","last_became_active":"2024-04-25T10:17:06.305289+0000","last_became_peered":"2024-04-25T10:17:06.305289+0000","last_unstale":"2024-04-25T10:17:07.287147+0000","last_undegraded":"2024-04-25T10:17:07.287147+0000","last_fullsized":"2024-04-25T10:17:07.287147+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:44.205272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077433+0000","last_change":"2024-04-25T10:17:05.283780+0000","last_active":"2024-04-25T10:17:08.077433+0000","last_peered":"2024-04-25T10:17:08.077433+0000","last_clean":"2024-04-25T10:17:08.077433+0000","last_became_active":"2024-04-25T10:17:05.282949+0000","last_became_peered":"2024-04-25T10:17:05.282949+0000","last_unstale":"2024-04-25T10:17:08.077433+0000","last_undegraded":"2024-04-25T10:17:08.077433+0000","last_fullsized":"2024-04-25T10:17:08.077433+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:53.803204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.638453+0000","last_change":"2024-04-25T10:17:06.280219+0000","last_active":"2024-04-25T10:17:07.638453+0000","last_peered":"2024-04-25T10:17:07.638453+0000","last_clean":"2024-04-25T10:17:07.638453+0000","last_became_active":"2024-04-25T10:17:06.279521+0000","last_became_peered":"2024-04-25T10:17:06.279521+0000","last_unstale":"2024-04-25T10:17:07.638453+0000","last_undegraded":"2024-04-25T10:17:07.638453+0000","last_fullsized":"2024-04-25T10:17:07.638453+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:24.805711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299304+0000","last_change":"2024-04-25T10:17:05.276531+0000","last_active":"2024-04-25T10:17:07.299304+0000","last_peered":"2024-04-25T10:17:07.299304+0000","last_clean":"2024-04-25T10:17:07.299304+0000","last_became_active":"2024-04-25T10:17:05.276173+0000","last_became_peered":"2024-04-25T10:17:05.276173+0000","last_unstale":"2024-04-25T10:17:07.299304+0000","last_undegraded":"2024-04-25T10:17:07.299304+0000","last_fullsized":"2024-04-25T10:17:07.299304+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:29.917533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285316+0000","last_change":"2024-04-25T10:17:06.277049+0000","last_active":"2024-04-25T10:17:07.285316+0000","last_peered":"2024-04-25T10:17:07.285316+0000","last_clean":"2024-04-25T10:17:07.285316+0000","last_became_active":"2024-04-25T10:17:06.276270+0000","last_became_peered":"2024-04-25T10:17:06.276270+0000","last_unstale":"2024-04-25T10:17:07.285316+0000","last_undegraded":"2024-04-25T10:17:07.285316+0000","last_fullsized":"2024-04-25T10:17:07.285316+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.636583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282842+0000","last_change":"2024-04-25T10:17:05.261659+0000","last_active":"2024-04-25T10:17:07.282842+0000","last_peered":"2024-04-25T10:17:07.282842+0000","last_clean":"2024-04-25T10:17:07.282842+0000","last_became_active":"2024-04-25T10:17:05.261064+0000","last_became_peered":"2024-04-25T10:17:05.261064+0000","last_unstale":"2024-04-25T10:17:07.282842+0000","last_undegraded":"2024-04-25T10:17:07.282842+0000","last_fullsized":"2024-04-25T10:17:07.282842+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:27.695631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299835+0000","last_change":"2024-04-25T10:17:06.308008+0000","last_active":"2024-04-25T10:17:07.299835+0000","last_peered":"2024-04-25T10:17:07.299835+0000","last_clean":"2024-04-25T10:17:07.299835+0000","last_became_active":"2024-04-25T10:17:06.307274+0000","last_became_peered":"2024-04-25T10:17:06.307274+0000","last_unstale":"2024-04-25T10:17:07.299835+0000","last_undegraded":"2024-04-25T10:17:07.299835+0000","last_fullsized":"2024-04-25T10:17:07.299835+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:05.329111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078583+0000","last_change":"2024-04-25T10:17:05.292064+0000","last_active":"2024-04-25T10:17:08.078583+0000","last_peered":"2024-04-25T10:17:08.078583+0000","last_clean":"2024-04-25T10:17:08.078583+0000","last_became_active":"2024-04-25T10:17:05.291243+0000","last_became_peered":"2024-04-25T10:17:05.291243+0000","last_unstale":"2024-04-25T10:17:08.078583+0000","last_undegraded":"2024-04-25T10:17:08.078583+0000","last_fullsized":"2024-04-25T10:17:08.078583+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.532980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637202+0000","last_change":"2024-04-25T10:17:06.311718+0000","last_active":"2024-04-25T10:17:07.637202+0000","last_peered":"2024-04-25T10:17:07.637202+0000","last_clean":"2024-04-25T10:17:07.637202+0000","last_became_active":"2024-04-25T10:17:06.311471+0000","last_became_peered":"2024-04-25T10:17:06.311471+0000","last_unstale":"2024-04-25T10:17:07.637202+0000","last_undegraded":"2024-04-25T10:17:07.637202+0000","last_fullsized":"2024-04-25T10:17:07.637202+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:58.375434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"182'3","reported_seq":17,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637600+0000","last_change":"2024-04-25T10:17:05.253995+0000","last_active":"2024-04-25T10:17:07.637600+0000","last_peered":"2024-04-25T10:17:07.637600+0000","last_clean":"2024-04-25T10:17:07.637600+0000","last_became_active":"2024-04-25T10:17:05.253098+0000","last_became_peered":"2024-04-25T10:17:05.253098+0000","last_unstale":"2024-04-25T10:17:07.637600+0000","last_undegraded":"2024-04-25T10:17:07.637600+0000","last_fullsized":"2024-04-25T10:17:07.637600+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:12.401265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282759+0000","last_change":"2024-04-25T10:17:06.302036+0000","last_active":"2024-04-25T10:17:07.282759+0000","last_peered":"2024-04-25T10:17:07.282759+0000","last_clean":"2024-04-25T10:17:07.282759+0000","last_became_active":"2024-04-25T10:17:06.301271+0000","last_became_peered":"2024-04-25T10:17:06.301271+0000","last_unstale":"2024-04-25T10:17:07.282759+0000","last_undegraded":"2024-04-25T10:17:07.282759+0000","last_fullsized":"2024-04-25T10:17:07.282759+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:16.342639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"182'6","reported_seq":20,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078241+0000","last_change":"2024-04-25T10:17:05.277327+0000","last_active":"2024-04-25T10:17:08.078241+0000","last_peered":"2024-04-25T10:17:08.078241+0000","last_clean":"2024-04-25T10:17:08.078241+0000","last_became_active":"2024-04-25T10:17:05.276483+0000","last_became_peered":"2024-04-25T10:17:05.276483+0000","last_unstale":"2024-04-25T10:17:08.078241+0000","last_undegraded":"2024-04-25T10:17:08.078241+0000","last_fullsized":"2024-04-25T10:17:08.078241+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:46.938160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076940+0000","last_change":"2024-04-25T10:17:06.308192+0000","last_active":"2024-04-25T10:17:08.076940+0000","last_peered":"2024-04-25T10:17:08.076940+0000","last_clean":"2024-04-25T10:17:08.076940+0000","last_became_active":"2024-04-25T10:17:06.307915+0000","last_became_peered":"2024-04-25T10:17:06.307915+0000","last_unstale":"2024-04-25T10:17:08.076940+0000","last_undegraded":"2024-04-25T10:17:08.076940+0000","last_fullsized":"2024-04-25T10:17:08.076940+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:40.876455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076689+0000","last_change":"2024-04-25T10:17:05.265780+0000","last_active":"2024-04-25T10:17:08.076689+0000","last_peered":"2024-04-25T10:17:08.076689+0000","last_clean":"2024-04-25T10:17:08.076689+0000","last_became_active":"2024-04-25T10:17:05.265463+0000","last_became_peered":"2024-04-25T10:17:05.265463+0000","last_unstale":"2024-04-25T10:17:08.076689+0000","last_undegraded":"2024-04-25T10:17:08.076689+0000","last_fullsized":"2024-04-25T10:17:08.076689+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:13.306138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.300218+0000","last_change":"2024-04-25T10:17:06.292009+0000","last_active":"2024-04-25T10:17:07.300218+0000","last_peered":"2024-04-25T10:17:07.300218+0000","last_clean":"2024-04-25T10:17:07.300218+0000","last_became_active":"2024-04-25T10:17:06.291005+0000","last_became_peered":"2024-04-25T10:17:06.291005+0000","last_unstale":"2024-04-25T10:17:07.300218+0000","last_undegraded":"2024-04-25T10:17:07.300218+0000","last_fullsized":"2024-04-25T10:17:07.300218+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:17.023613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299363+0000","last_change":"2024-04-25T10:17:05.259439+0000","last_active":"2024-04-25T10:17:07.299363+0000","last_peered":"2024-04-25T10:17:07.299363+0000","last_clean":"2024-04-25T10:17:07.299363+0000","last_became_active":"2024-04-25T10:17:05.259119+0000","last_became_peered":"2024-04-25T10:17:05.259119+0000","last_unstale":"2024-04-25T10:17:07.299363+0000","last_undegraded":"2024-04-25T10:17:07.299363+0000","last_fullsized":"2024-04-25T10:17:07.299363+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:57.994043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637922+0000","last_change":"2024-04-25T10:17:06.301936+0000","last_active":"2024-04-25T10:17:07.637922+0000","last_peered":"2024-04-25T10:17:07.637922+0000","last_clean":"2024-04-25T10:17:07.637922+0000","last_became_active":"2024-04-25T10:17:06.301279+0000","last_became_peered":"2024-04-25T10:17:06.301279+0000","last_unstale":"2024-04-25T10:17:07.637922+0000","last_undegraded":"2024-04-25T10:17:07.637922+0000","last_fullsized":"2024-04-25T10:17:07.637922+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:27.368095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.636758+0000","last_change":"2024-04-25T10:17:05.269183+0000","last_active":"2024-04-25T10:17:07.636758+0000","last_peered":"2024-04-25T10:17:07.636758+0000","last_clean":"2024-04-25T10:17:07.636758+0000","last_became_active":"2024-04-25T10:17:05.268612+0000","last_became_peered":"2024-04-25T10:17:05.268612+0000","last_unstale":"2024-04-25T10:17:07.636758+0000","last_undegraded":"2024-04-25T10:17:07.636758+0000","last_fullsized":"2024-04-25T10:17:07.636758+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.532980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077948+0000","last_change":"2024-04-25T10:17:06.313653+0000","last_active":"2024-04-25T10:17:08.077948+0000","last_peered":"2024-04-25T10:17:08.077948+0000","last_clean":"2024-04-25T10:17:08.077948+0000","last_became_active":"2024-04-25T10:17:06.313405+0000","last_became_peered":"2024-04-25T10:17:06.313405+0000","last_unstale":"2024-04-25T10:17:08.077948+0000","last_undegraded":"2024-04-25T10:17:08.077948+0000","last_fullsized":"2024-04-25T10:17:08.077948+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:56.971134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"182'8","reported_seq":22,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.280970+0000","last_change":"2024-04-25T10:17:05.274541+0000","last_active":"2024-04-25T10:17:07.280970+0000","last_peered":"2024-04-25T10:17:07.280970+0000","last_clean":"2024-04-25T10:17:07.280970+0000","last_became_active":"2024-04-25T10:17:05.274012+0000","last_became_peered":"2024-04-25T10:17:05.274012+0000","last_unstale":"2024-04-25T10:17:07.280970+0000","last_undegraded":"2024-04-25T10:17:07.280970+0000","last_fullsized":"2024-04-25T10:17:07.280970+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:59.380156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287431+0000","last_change":"2024-04-25T10:17:06.315211+0000","last_active":"2024-04-25T10:17:07.287431+0000","last_peered":"2024-04-25T10:17:07.287431+0000","last_clean":"2024-04-25T10:17:07.287431+0000","last_became_active":"2024-04-25T10:17:06.312743+0000","last_became_peered":"2024-04-25T10:17:06.312743+0000","last_unstale":"2024-04-25T10:17:07.287431+0000","last_undegraded":"2024-04-25T10:17:07.287431+0000","last_fullsized":"2024-04-25T10:17:07.287431+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:44.422561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079301+0000","last_change":"2024-04-25T10:17:05.287915+0000","last_active":"2024-04-25T10:17:08.079301+0000","last_peered":"2024-04-25T10:17:08.079301+0000","last_clean":"2024-04-25T10:17:08.079301+0000","last_became_active":"2024-04-25T10:17:05.287226+0000","last_became_peered":"2024-04-25T10:17:05.287226+0000","last_unstale":"2024-04-25T10:17:08.079301+0000","last_undegraded":"2024-04-25T10:17:08.079301+0000","last_fullsized":"2024-04-25T10:17:08.079301+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:03.313674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078742+0000","last_change":"2024-04-25T10:17:06.317023+0000","last_active":"2024-04-25T10:17:08.078742+0000","last_peered":"2024-04-25T10:17:08.078742+0000","last_clean":"2024-04-25T10:17:08.078742+0000","last_became_active":"2024-04-25T10:17:06.316453+0000","last_became_peered":"2024-04-25T10:17:06.316453+0000","last_unstale":"2024-04-25T10:17:08.078742+0000","last_undegraded":"2024-04-25T10:17:08.078742+0000","last_fullsized":"2024-04-25T10:17:08.078742+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:30.842625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079283+0000","last_change":"2024-04-25T10:17:05.285851+0000","last_active":"2024-04-25T10:17:08.079283+0000","last_peered":"2024-04-25T10:17:08.079283+0000","last_clean":"2024-04-25T10:17:08.079283+0000","last_became_active":"2024-04-25T10:17:05.284722+0000","last_became_peered":"2024-04-25T10:17:05.284722+0000","last_unstale":"2024-04-25T10:17:08.079283+0000","last_undegraded":"2024-04-25T10:17:08.079283+0000","last_fullsized":"2024-04-25T10:17:08.079283+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:58.353363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078687+0000","last_change":"2024-04-25T10:17:06.311043+0000","last_active":"2024-04-25T10:17:08.078687+0000","last_peered":"2024-04-25T10:17:08.078687+0000","last_clean":"2024-04-25T10:17:08.078687+0000","last_became_active":"2024-04-25T10:17:06.310681+0000","last_became_peered":"2024-04-25T10:17:06.310681+0000","last_unstale":"2024-04-25T10:17:08.078687+0000","last_undegraded":"2024-04-25T10:17:08.078687+0000","last_fullsized":"2024-04-25T10:17:08.078687+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:09.040936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076754+0000","last_change":"2024-04-25T10:17:05.276187+0000","last_active":"2024-04-25T10:17:08.076754+0000","last_peered":"2024-04-25T10:17:08.076754+0000","last_clean":"2024-04-25T10:17:08.076754+0000","last_became_active":"2024-04-25T10:17:05.275862+0000","last_became_peered":"2024-04-25T10:17:05.275862+0000","last_unstale":"2024-04-25T10:17:08.076754+0000","last_undegraded":"2024-04-25T10:17:08.076754+0000","last_fullsized":"2024-04-25T10:17:08.076754+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:17.844725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287189+0000","last_change":"2024-04-25T10:17:06.303966+0000","last_active":"2024-04-25T10:17:07.287189+0000","last_peered":"2024-04-25T10:17:07.287189+0000","last_clean":"2024-04-25T10:17:07.287189+0000","last_became_active":"2024-04-25T10:17:06.303463+0000","last_became_peered":"2024-04-25T10:17:06.303463+0000","last_unstale":"2024-04-25T10:17:07.287189+0000","last_undegraded":"2024-04-25T10:17:07.287189+0000","last_fullsized":"2024-04-25T10:17:07.287189+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:28.438816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282380+0000","last_change":"2024-04-25T10:17:05.288488+0000","last_active":"2024-04-25T10:17:07.282380+0000","last_peered":"2024-04-25T10:17:07.282380+0000","last_clean":"2024-04-25T10:17:07.282380+0000","last_became_active":"2024-04-25T10:17:05.288145+0000","last_became_peered":"2024-04-25T10:17:05.288145+0000","last_unstale":"2024-04-25T10:17:07.282380+0000","last_undegraded":"2024-04-25T10:17:07.282380+0000","last_fullsized":"2024-04-25T10:17:07.282380+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:58.978256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077035+0000","last_change":"2024-04-25T10:17:06.299136+0000","last_active":"2024-04-25T10:17:08.077035+0000","last_peered":"2024-04-25T10:17:08.077035+0000","last_clean":"2024-04-25T10:17:08.077035+0000","last_became_active":"2024-04-25T10:17:06.293833+0000","last_became_peered":"2024-04-25T10:17:06.293833+0000","last_unstale":"2024-04-25T10:17:08.077035+0000","last_undegraded":"2024-04-25T10:17:08.077035+0000","last_fullsized":"2024-04-25T10:17:08.077035+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:42.905954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282539+0000","last_change":"2024-04-25T10:17:05.261872+0000","last_active":"2024-04-25T10:17:07.282539+0000","last_peered":"2024-04-25T10:17:07.282539+0000","last_clean":"2024-04-25T10:17:07.282539+0000","last_became_active":"2024-04-25T10:17:05.261364+0000","last_became_peered":"2024-04-25T10:17:05.261364+0000","last_unstale":"2024-04-25T10:17:07.282539+0000","last_undegraded":"2024-04-25T10:17:07.282539+0000","last_fullsized":"2024-04-25T10:17:07.282539+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:29.091373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077792+0000","last_change":"2024-04-25T10:17:06.302371+0000","last_active":"2024-04-25T10:17:08.077792+0000","last_peered":"2024-04-25T10:17:08.077792+0000","last_clean":"2024-04-25T10:17:08.077792+0000","last_became_active":"2024-04-25T10:17:06.302053+0000","last_became_peered":"2024-04-25T10:17:06.302053+0000","last_unstale":"2024-04-25T10:17:08.077792+0000","last_undegraded":"2024-04-25T10:17:08.077792+0000","last_fullsized":"2024-04-25T10:17:08.077792+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:51.335789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076423+0000","last_change":"2024-04-25T10:17:05.262142+0000","last_active":"2024-04-25T10:17:08.076423+0000","last_peered":"2024-04-25T10:17:08.076423+0000","last_clean":"2024-04-25T10:17:08.076423+0000","last_became_active":"2024-04-25T10:17:05.261685+0000","last_became_peered":"2024-04-25T10:17:05.261685+0000","last_unstale":"2024-04-25T10:17:08.076423+0000","last_undegraded":"2024-04-25T10:17:08.076423+0000","last_fullsized":"2024-04-25T10:17:08.076423+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.532980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.638007+0000","last_change":"2024-04-25T10:17:06.281624+0000","last_active":"2024-04-25T10:17:07.638007+0000","last_peered":"2024-04-25T10:17:07.638007+0000","last_clean":"2024-04-25T10:17:07.638007+0000","last_became_active":"2024-04-25T10:17:06.280903+0000","last_became_peered":"2024-04-25T10:17:06.280903+0000","last_unstale":"2024-04-25T10:17:07.638007+0000","last_undegraded":"2024-04-25T10:17:07.638007+0000","last_fullsized":"2024-04-25T10:17:07.638007+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:13.013098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285672+0000","last_change":"2024-04-25T10:17:05.280949+0000","last_active":"2024-04-25T10:17:07.285672+0000","last_peered":"2024-04-25T10:17:07.285672+0000","last_clean":"2024-04-25T10:17:07.285672+0000","last_became_active":"2024-04-25T10:17:05.277678+0000","last_became_peered":"2024-04-25T10:17:05.277678+0000","last_unstale":"2024-04-25T10:17:07.285672+0000","last_undegraded":"2024-04-25T10:17:07.285672+0000","last_fullsized":"2024-04-25T10:17:07.285672+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:37.094026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287855+0000","last_change":"2024-04-25T10:17:06.312734+0000","last_active":"2024-04-25T10:17:07.287855+0000","last_peered":"2024-04-25T10:17:07.287855+0000","last_clean":"2024-04-25T10:17:07.287855+0000","last_became_active":"2024-04-25T10:17:06.312278+0000","last_became_peered":"2024-04-25T10:17:06.312278+0000","last_unstale":"2024-04-25T10:17:07.287855+0000","last_undegraded":"2024-04-25T10:17:07.287855+0000","last_fullsized":"2024-04-25T10:17:07.287855+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:13.273521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078763+0000","last_change":"2024-04-25T10:17:05.284291+0000","last_active":"2024-04-25T10:17:08.078763+0000","last_peered":"2024-04-25T10:17:08.078763+0000","last_clean":"2024-04-25T10:17:08.078763+0000","last_became_active":"2024-04-25T10:17:05.283622+0000","last_became_peered":"2024-04-25T10:17:05.283622+0000","last_unstale":"2024-04-25T10:17:08.078763+0000","last_undegraded":"2024-04-25T10:17:08.078763+0000","last_fullsized":"2024-04-25T10:17:08.078763+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:43.192820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285134+0000","last_change":"2024-04-25T10:17:06.308081+0000","last_active":"2024-04-25T10:17:07.285134+0000","last_peered":"2024-04-25T10:17:07.285134+0000","last_clean":"2024-04-25T10:17:07.285134+0000","last_became_active":"2024-04-25T10:17:06.307803+0000","last_became_peered":"2024-04-25T10:17:06.307803+0000","last_unstale":"2024-04-25T10:17:07.285134+0000","last_undegraded":"2024-04-25T10:17:07.285134+0000","last_fullsized":"2024-04-25T10:17:07.285134+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:57.829524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.638244+0000","last_change":"2024-04-25T10:17:05.255829+0000","last_active":"2024-04-25T10:17:07.638244+0000","last_peered":"2024-04-25T10:17:07.638244+0000","last_clean":"2024-04-25T10:17:07.638244+0000","last_became_active":"2024-04-25T10:17:05.255551+0000","last_became_peered":"2024-04-25T10:17:05.255551+0000","last_unstale":"2024-04-25T10:17:07.638244+0000","last_undegraded":"2024-04-25T10:17:07.638244+0000","last_fullsized":"2024-04-25T10:17:07.638244+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:53.175315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299504+0000","last_change":"2024-04-25T10:17:06.309951+0000","last_active":"2024-04-25T10:17:07.299504+0000","last_peered":"2024-04-25T10:17:07.299504+0000","last_clean":"2024-04-25T10:17:07.299504+0000","last_became_active":"2024-04-25T10:17:06.309400+0000","last_became_peered":"2024-04-25T10:17:06.309400+0000","last_unstale":"2024-04-25T10:17:07.299504+0000","last_undegraded":"2024-04-25T10:17:07.299504+0000","last_fullsized":"2024-04-25T10:17:07.299504+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:36.045880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"183'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287687+0000","last_change":"2024-04-25T10:17:05.266933+0000","last_active":"2024-04-25T10:17:07.287687+0000","last_peered":"2024-04-25T10:17:07.287687+0000","last_clean":"2024-04-25T10:17:07.287687+0000","last_became_active":"2024-04-25T10:17:05.266096+0000","last_became_peered":"2024-04-25T10:17:05.266096+0000","last_unstale":"2024-04-25T10:17:07.287687+0000","last_undegraded":"2024-04-25T10:17:07.287687+0000","last_fullsized":"2024-04-25T10:17:07.287687+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:20.210981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637171+0000","last_change":"2024-04-25T10:17:06.305649+0000","last_active":"2024-04-25T10:17:07.637171+0000","last_peered":"2024-04-25T10:17:07.637171+0000","last_clean":"2024-04-25T10:17:07.637171+0000","last_became_active":"2024-04-25T10:17:06.305256+0000","last_became_peered":"2024-04-25T10:17:06.305256+0000","last_unstale":"2024-04-25T10:17:07.637171+0000","last_undegraded":"2024-04-25T10:17:07.637171+0000","last_fullsized":"2024-04-25T10:17:07.637171+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:14.977772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286519+0000","last_change":"2024-04-25T10:17:05.270685+0000","last_active":"2024-04-25T10:17:07.286519+0000","last_peered":"2024-04-25T10:17:07.286519+0000","last_clean":"2024-04-25T10:17:07.286519+0000","last_became_active":"2024-04-25T10:17:05.269724+0000","last_became_peered":"2024-04-25T10:17:05.269724+0000","last_unstale":"2024-04-25T10:17:07.286519+0000","last_undegraded":"2024-04-25T10:17:07.286519+0000","last_fullsized":"2024-04-25T10:17:07.286519+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.189977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077807+0000","last_change":"2024-04-25T10:17:06.312347+0000","last_active":"2024-04-25T10:17:08.077807+0000","last_peered":"2024-04-25T10:17:08.077807+0000","last_clean":"2024-04-25T10:17:08.077807+0000","last_became_active":"2024-04-25T10:17:06.311697+0000","last_became_peered":"2024-04-25T10:17:06.311697+0000","last_unstale":"2024-04-25T10:17:08.077807+0000","last_undegraded":"2024-04-25T10:17:08.077807+0000","last_fullsized":"2024-04-25T10:17:08.077807+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:30.929985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078334+0000","last_change":"2024-04-25T10:17:05.288602+0000","last_active":"2024-04-25T10:17:08.078334+0000","last_peered":"2024-04-25T10:17:08.078334+0000","last_clean":"2024-04-25T10:17:08.078334+0000","last_became_active":"2024-04-25T10:17:05.287869+0000","last_became_peered":"2024-04-25T10:17:05.287869+0000","last_unstale":"2024-04-25T10:17:08.078334+0000","last_undegraded":"2024-04-25T10:17:08.078334+0000","last_fullsized":"2024-04-25T10:17:08.078334+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:42.530292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286401+0000","last_change":"2024-04-25T10:17:06.307406+0000","last_active":"2024-04-25T10:17:07.286401+0000","last_peered":"2024-04-25T10:17:07.286401+0000","last_clean":"2024-04-25T10:17:07.286401+0000","last_became_active":"2024-04-25T10:17:06.303507+0000","last_became_peered":"2024-04-25T10:17:06.303507+0000","last_unstale":"2024-04-25T10:17:07.286401+0000","last_undegraded":"2024-04-25T10:17:07.286401+0000","last_fullsized":"2024-04-25T10:17:07.286401+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:51.585849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285915+0000","last_change":"2024-04-25T10:17:05.258680+0000","last_active":"2024-04-25T10:17:07.285915+0000","last_peered":"2024-04-25T10:17:07.285915+0000","last_clean":"2024-04-25T10:17:07.285915+0000","last_became_active":"2024-04-25T10:17:05.257882+0000","last_became_peered":"2024-04-25T10:17:05.257882+0000","last_unstale":"2024-04-25T10:17:07.285915+0000","last_undegraded":"2024-04-25T10:17:07.285915+0000","last_fullsized":"2024-04-25T10:17:07.285915+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.305991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078841+0000","last_change":"2024-04-25T10:17:06.310242+0000","last_active":"2024-04-25T10:17:08.078841+0000","last_peered":"2024-04-25T10:17:08.078841+0000","last_clean":"2024-04-25T10:17:08.078841+0000","last_became_active":"2024-04-25T10:17:06.309971+0000","last_became_peered":"2024-04-25T10:17:06.309971+0000","last_unstale":"2024-04-25T10:17:08.078841+0000","last_undegraded":"2024-04-25T10:17:08.078841+0000","last_fullsized":"2024-04-25T10:17:08.078841+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:58.375434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637387+0000","last_change":"2024-04-25T10:17:05.273922+0000","last_active":"2024-04-25T10:17:07.637387+0000","last_peered":"2024-04-25T10:17:07.637387+0000","last_clean":"2024-04-25T10:17:07.637387+0000","last_became_active":"2024-04-25T10:17:05.273593+0000","last_became_peered":"2024-04-25T10:17:05.273593+0000","last_unstale":"2024-04-25T10:17:07.637387+0000","last_undegraded":"2024-04-25T10:17:07.637387+0000","last_fullsized":"2024-04-25T10:17:07.637387+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:17.844725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078448+0000","last_change":"2024-04-25T10:17:06.310744+0000","last_active":"2024-04-25T10:17:08.078448+0000","last_peered":"2024-04-25T10:17:08.078448+0000","last_clean":"2024-04-25T10:17:08.078448+0000","last_became_active":"2024-04-25T10:17:06.310060+0000","last_became_peered":"2024-04-25T10:17:06.310060+0000","last_unstale":"2024-04-25T10:17:08.078448+0000","last_undegraded":"2024-04-25T10:17:08.078448+0000","last_fullsized":"2024-04-25T10:17:08.078448+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:14.977772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079010+0000","last_change":"2024-04-25T10:17:05.286524+0000","last_active":"2024-04-25T10:17:08.079010+0000","last_peered":"2024-04-25T10:17:08.079010+0000","last_clean":"2024-04-25T10:17:08.079010+0000","last_became_active":"2024-04-25T10:17:05.285810+0000","last_became_peered":"2024-04-25T10:17:05.285810+0000","last_unstale":"2024-04-25T10:17:08.079010+0000","last_undegraded":"2024-04-25T10:17:08.079010+0000","last_fullsized":"2024-04-25T10:17:08.079010+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:40.092160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299942+0000","last_change":"2024-04-25T10:17:06.300864+0000","last_active":"2024-04-25T10:17:07.299942+0000","last_peered":"2024-04-25T10:17:07.299942+0000","last_clean":"2024-04-25T10:17:07.299942+0000","last_became_active":"2024-04-25T10:17:06.300535+0000","last_became_peered":"2024-04-25T10:17:06.300535+0000","last_unstale":"2024-04-25T10:17:07.299942+0000","last_undegraded":"2024-04-25T10:17:07.299942+0000","last_fullsized":"2024-04-25T10:17:07.299942+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:32.451444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.281119+0000","last_change":"2024-04-25T10:17:05.274379+0000","last_active":"2024-04-25T10:17:07.281119+0000","last_peered":"2024-04-25T10:17:07.281119+0000","last_clean":"2024-04-25T10:17:07.281119+0000","last_became_active":"2024-04-25T10:17:05.274028+0000","last_became_peered":"2024-04-25T10:17:05.274028+0000","last_unstale":"2024-04-25T10:17:07.281119+0000","last_undegraded":"2024-04-25T10:17:07.281119+0000","last_fullsized":"2024-04-25T10:17:07.281119+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:50.044993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078928+0000","last_change":"2024-04-25T10:17:06.313892+0000","last_active":"2024-04-25T10:17:08.078928+0000","last_peered":"2024-04-25T10:17:08.078928+0000","last_clean":"2024-04-25T10:17:08.078928+0000","last_became_active":"2024-04-25T10:17:06.313558+0000","last_became_peered":"2024-04-25T10:17:06.313558+0000","last_unstale":"2024-04-25T10:17:08.078928+0000","last_undegraded":"2024-04-25T10:17:08.078928+0000","last_fullsized":"2024-04-25T10:17:08.078928+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:13.660835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637374+0000","last_change":"2024-04-25T10:17:05.253379+0000","last_active":"2024-04-25T10:17:07.637374+0000","last_peered":"2024-04-25T10:17:07.637374+0000","last_clean":"2024-04-25T10:17:07.637374+0000","last_became_active":"2024-04-25T10:17:05.252888+0000","last_became_peered":"2024-04-25T10:17:05.252888+0000","last_unstale":"2024-04-25T10:17:07.637374+0000","last_undegraded":"2024-04-25T10:17:07.637374+0000","last_fullsized":"2024-04-25T10:17:07.637374+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:41.072995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":477,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077063+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:17:08.077063+0000","last_peered":"2024-04-25T10:17:08.077063+0000","last_clean":"2024-04-25T10:17:08.077063+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:17:08.077063+0000","last_undegraded":"2024-04-25T10:17:08.077063+0000","last_fullsized":"2024-04-25T10:17:08.077063+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74292,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297548,"statfs":{"total":96636764160,"available":96560689152,"internally_reserved":0,"allocated":2920448,"data_stored":1678264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575039,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51504,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320336,"statfs":{"total":96636764160,"available":96584024064,"internally_reserved":0,"allocated":2326528,"data_stored":1088408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47336,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324504,"statfs":{"total":96636764160,"available":96588292096,"internally_reserved":0,"allocated":2318336,"data_stored":1086270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95908,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275932,"statfs":{"total":96636764160,"available":96538554368,"internally_reserved":0,"allocated":2314240,"data_stored":1086796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82024,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289816,"statfs":{"total":96636764160,"available":96552771584,"internally_reserved":0,"allocated":2322432,"data_stored":1086340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575038,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51492,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320348,"statfs":{"total":96636764160,"available":96584036352,"internally_reserved":0,"allocated":2314240,"data_stored":1086304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47324,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324516,"statfs":{"total":96636764160,"available":96588304384,"internally_reserved":0,"allocated":2310144,"data_stored":1086214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575039,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74276,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297564,"statfs":{"total":96636764160,"available":96560705536,"internally_reserved":0,"allocated":2908160,"data_stored":1676672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:10.302 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:10.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.444+0000 7fb636d0f640 1 Processor -- start 2024-04-25T10:17:10.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.444+0000 7fb636d0f640 1 -- start start 2024-04-25T10:17:10.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb636d0f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb63015c9a0 0x7fb6300c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb636d0f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb6300b1f70 0x7fb63006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb636d0f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6300c61d0 con 0x7fb6300ce330 2024-04-25T10:17:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb636d0f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6300c64b0 con 0x7fb63015c9a0 2024-04-25T10:17:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb636d0f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63006d4f0 con 0x7fb6300b1f70 2024-04-25T10:17:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb634aab640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb63015c9a0 0x7fb6300c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb6352ac640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb6300ce330 0x7fb63015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55554/0 (socket says 172.21.15.133:55554) 2024-04-25T10:17:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb6352ac640 1 -- 172.21.15.133:0/3238978547 learned_addr learned my addr 172.21.15.133:0/3238978547 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb634aab640 1 -- 172.21.15.133:0/3238978547 >> v2:172.21.15.162:3300/0 conn(0x7fb6300b1f70 msgr2=0x7fb63006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb634aab640 1 --2- 172.21.15.133:0/3238978547 >> v2:172.21.15.162:3300/0 conn(0x7fb6300b1f70 0x7fb63006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb634aab640 1 -- 172.21.15.133:0/3238978547 >> v1:172.21.15.133:6789/0 conn(0x7fb6300ce330 legacy=0x7fb63015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb634aab640 1 -- 172.21.15.133:0/3238978547 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63006d7d0 con 0x7fb63015c9a0 2024-04-25T10:17:10.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb62f7fe640 1 -- 172.21.15.133:0/3238978547 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434719773 0 0) 0x7fb6300c61d0 con 0x7fb6300ce330 2024-04-25T10:17:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb634aab640 1 --2- 172.21.15.133:0/3238978547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63015c9a0 0x7fb6300c5980 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fb630072670 tx=0x7fb62001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d58622015e7b86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb62f7fe640 1 -- 172.21.15.133:0/3238978547 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200ab640 con 0x7fb63015c9a0 2024-04-25T10:17:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb62f7fe640 1 -- 172.21.15.133:0/3238978547 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6200b0070 con 0x7fb63015c9a0 2024-04-25T10:17:10.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.448+0000 7fb62f7fe640 1 -- 172.21.15.133:0/3238978547 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200aba00 con 0x7fb63015c9a0 2024-04-25T10:17:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- 172.21.15.133:0/3238978547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63015c9a0 msgr2=0x7fb6300c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 --2- 172.21.15.133:0/3238978547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63015c9a0 0x7fb6300c5980 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fb630072670 tx=0x7fb62001c420 comp rx=0 tx=0).stop 2024-04-25T10:17:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- 172.21.15.133:0/3238978547 shutdown_connections 2024-04-25T10:17:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 --2- 172.21.15.133:0/3238978547 >> v2:172.21.15.162:3300/0 conn(0x7fb6300b1f70 0x7fb63006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 --2- 172.21.15.133:0/3238978547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63015c9a0 0x7fb6300c5980 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- 172.21.15.133:0/3238978547 >> 172.21.15.133:0/3238978547 conn(0x7fb630067000 msgr2=0x7fb63006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- 172.21.15.133:0/3238978547 shutdown_connections 2024-04-25T10:17:10.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- 172.21.15.133:0/3238978547 wait complete. 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 Processor -- start 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- start start 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6300b1f70 0x7fb63014c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6300ce330 0x7fb630152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63015c9a0 0x7fb6301473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6300ceb50 con 0x7fb6300b1f70 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63015cd80 con 0x7fb6300ce330 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb630147fb0 con 0x7fb63015c9a0 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb634aab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6300b1f70 0x7fb63014c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb634aab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6300b1f70 0x7fb63014c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54698/0 (socket says 172.21.15.133:54698) 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb634aab640 1 -- 172.21.15.133:0/2072106263 learned_addr learned my addr 172.21.15.133:0/2072106263 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:10.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb6352ac640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63015c9a0 0x7fb6301473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb62ffff640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6300ce330 0x7fb630152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb634aab640 1 -- 172.21.15.133:0/2072106263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63015c9a0 msgr2=0x7fb6301473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb634aab640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63015c9a0 0x7fb6301473f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb634aab640 1 -- 172.21.15.133:0/2072106263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6300ce330 msgr2=0x7fb630152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb634aab640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6300ce330 0x7fb630152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb634aab640 1 -- 172.21.15.133:0/2072106263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6300c8280 con 0x7fb6300b1f70 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb6352ac640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63015c9a0 0x7fb6301473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb634aab640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6300b1f70 0x7fb63014c820 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7fb6200ad040 tx=0x7fb62001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb62d7fa640 1 -- 172.21.15.133:0/2072106263 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620004030 con 0x7fb6300b1f70 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb62d7fa640 1 -- 172.21.15.133:0/2072106263 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6200041d0 con 0x7fb6300b1f70 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb62d7fa640 1 -- 172.21.15.133:0/2072106263 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620003620 con 0x7fb6300b1f70 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- 172.21.15.133:0/2072106263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6300c8560 con 0x7fb6300b1f70 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- 172.21.15.133:0/2072106263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6300c8b20 con 0x7fb6300b1f70 2024-04-25T10:17:10.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.452+0000 7fb636d0f640 1 -- 172.21.15.133:0/2072106263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb630068ef0 con 0x7fb6300b1f70 2024-04-25T10:17:10.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.456+0000 7fb62d7fa640 1 -- 172.21.15.133:0/2072106263 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6200037c0 con 0x7fb6300b1f70 2024-04-25T10:17:10.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.456+0000 7fb62d7fa640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb61c066f60 0x7fb61c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.456+0000 7fb62ffff640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb61c066f60 0x7fb61c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.456+0000 7fb62d7fa640 1 -- 172.21.15.133:0/2072106263 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb62011fd10 con 0x7fb6300b1f70 2024-04-25T10:17:10.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.456+0000 7fb62ffff640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb61c066f60 0x7fb61c069420 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb624006fd0 tx=0x7fb624008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.460+0000 7fb62d7fa640 1 -- 172.21.15.133:0/2072106263 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb620016070 con 0x7fb6300b1f70 2024-04-25T10:17:10.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.616+0000 7fb636d0f640 1 -- 172.21.15.133:0/2072106263 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb63015e010 con 0x7fb61c066f60 2024-04-25T10:17:10.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.624+0000 7fb62d7fa640 1 -- 172.21.15.133:0/2072106263 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252493 (secure 0 0 0) 0x7fb63015e010 con 0x7fb61c066f60 2024-04-25T10:17:10.628 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:10.629 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:17:10.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 -- 172.21.15.133:0/2072106263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb61c066f60 msgr2=0x7fb61c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb61c066f60 0x7fb61c069420 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb624006fd0 tx=0x7fb624008040 comp rx=0 tx=0).stop 2024-04-25T10:17:10.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 -- 172.21.15.133:0/2072106263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6300b1f70 msgr2=0x7fb63014c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6300b1f70 0x7fb63014c820 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7fb6200ad040 tx=0x7fb62001acc0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 -- 172.21.15.133:0/2072106263 shutdown_connections 2024-04-25T10:17:10.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb63015c9a0 0x7fb6301473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6300ce330 0x7fb630152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb61c066f60 0x7fb61c069420 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 --2- 172.21.15.133:0/2072106263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6300b1f70 0x7fb63014c820 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 -- 172.21.15.133:0/2072106263 >> 172.21.15.133:0/2072106263 conn(0x7fb630067000 msgr2=0x7fb6301565c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:10.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 -- 172.21.15.133:0/2072106263 shutdown_connections 2024-04-25T10:17:10.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.628+0000 7fb636d0f640 1 -- 172.21.15.133:0/2072106263 wait complete. 2024-04-25T10:17:10.646 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":639,"stamp":"2024-04-25T10:17:09.997823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":524156,"kb_used_data":19272,"kb_used_omap":12,"kb_used_meta":504755,"kb_avail":754450564,"statfs":{"total":773094113280,"available":772557377536,"internally_reserved":0,"allocated":19734528,"data_stored":9875268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":516869681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.764981"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079359+0000","last_change":"2024-04-25T10:17:06.306658+0000","last_active":"2024-04-25T10:17:08.079359+0000","last_peered":"2024-04-25T10:17:08.079359+0000","last_clean":"2024-04-25T10:17:08.079359+0000","last_became_active":"2024-04-25T10:17:06.305948+0000","last_became_peered":"2024-04-25T10:17:06.305948+0000","last_unstale":"2024-04-25T10:17:08.079359+0000","last_undegraded":"2024-04-25T10:17:08.079359+0000","last_fullsized":"2024-04-25T10:17:08.079359+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:03.718103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077095+0000","last_change":"2024-04-25T10:17:06.299852+0000","last_active":"2024-04-25T10:17:08.077095+0000","last_peered":"2024-04-25T10:17:08.077095+0000","last_clean":"2024-04-25T10:17:08.077095+0000","last_became_active":"2024-04-25T10:17:06.293561+0000","last_became_peered":"2024-04-25T10:17:06.293561+0000","last_unstale":"2024-04-25T10:17:08.077095+0000","last_undegraded":"2024-04-25T10:17:08.077095+0000","last_fullsized":"2024-04-25T10:17:08.077095+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:05.505613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299782+0000","last_change":"2024-04-25T10:17:06.302106+0000","last_active":"2024-04-25T10:17:07.299782+0000","last_peered":"2024-04-25T10:17:07.299782+0000","last_clean":"2024-04-25T10:17:07.299782+0000","last_became_active":"2024-04-25T10:17:06.301792+0000","last_became_peered":"2024-04-25T10:17:06.301792+0000","last_unstale":"2024-04-25T10:17:07.299782+0000","last_undegraded":"2024-04-25T10:17:07.299782+0000","last_fullsized":"2024-04-25T10:17:07.299782+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:34.223680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078111+0000","last_change":"2024-04-25T10:17:06.294925+0000","last_active":"2024-04-25T10:17:08.078111+0000","last_peered":"2024-04-25T10:17:08.078111+0000","last_clean":"2024-04-25T10:17:08.078111+0000","last_became_active":"2024-04-25T10:17:06.294551+0000","last_became_peered":"2024-04-25T10:17:06.294551+0000","last_unstale":"2024-04-25T10:17:08.078111+0000","last_undegraded":"2024-04-25T10:17:08.078111+0000","last_fullsized":"2024-04-25T10:17:08.078111+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:09.737034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282332+0000","last_change":"2024-04-25T10:17:06.296451+0000","last_active":"2024-04-25T10:17:07.282332+0000","last_peered":"2024-04-25T10:17:07.282332+0000","last_clean":"2024-04-25T10:17:07.282332+0000","last_became_active":"2024-04-25T10:17:06.292410+0000","last_became_peered":"2024-04-25T10:17:06.292410+0000","last_unstale":"2024-04-25T10:17:07.282332+0000","last_undegraded":"2024-04-25T10:17:07.282332+0000","last_fullsized":"2024-04-25T10:17:07.282332+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:44.116534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077329+0000","last_change":"2024-04-25T10:17:06.293993+0000","last_active":"2024-04-25T10:17:08.077329+0000","last_peered":"2024-04-25T10:17:08.077329+0000","last_clean":"2024-04-25T10:17:08.077329+0000","last_became_active":"2024-04-25T10:17:06.293546+0000","last_became_peered":"2024-04-25T10:17:06.293546+0000","last_unstale":"2024-04-25T10:17:08.077329+0000","last_undegraded":"2024-04-25T10:17:08.077329+0000","last_fullsized":"2024-04-25T10:17:08.077329+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:06.488061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.281594+0000","last_change":"2024-04-25T10:17:06.294503+0000","last_active":"2024-04-25T10:17:07.281594+0000","last_peered":"2024-04-25T10:17:07.281594+0000","last_clean":"2024-04-25T10:17:07.281594+0000","last_became_active":"2024-04-25T10:17:06.291408+0000","last_became_peered":"2024-04-25T10:17:06.291408+0000","last_unstale":"2024-04-25T10:17:07.281594+0000","last_undegraded":"2024-04-25T10:17:07.281594+0000","last_fullsized":"2024-04-25T10:17:07.281594+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:38.193858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.284681+0000","last_change":"2024-04-25T10:17:06.303945+0000","last_active":"2024-04-25T10:17:07.284681+0000","last_peered":"2024-04-25T10:17:07.284681+0000","last_clean":"2024-04-25T10:17:07.284681+0000","last_became_active":"2024-04-25T10:17:06.303488+0000","last_became_peered":"2024-04-25T10:17:06.303488+0000","last_unstale":"2024-04-25T10:17:07.284681+0000","last_undegraded":"2024-04-25T10:17:07.284681+0000","last_fullsized":"2024-04-25T10:17:07.284681+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:11.256331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637957+0000","last_change":"2024-04-25T10:17:06.306233+0000","last_active":"2024-04-25T10:17:07.637957+0000","last_peered":"2024-04-25T10:17:07.637957+0000","last_clean":"2024-04-25T10:17:07.637957+0000","last_became_active":"2024-04-25T10:17:06.305499+0000","last_became_peered":"2024-04-25T10:17:06.305499+0000","last_unstale":"2024-04-25T10:17:07.637957+0000","last_undegraded":"2024-04-25T10:17:07.637957+0000","last_fullsized":"2024-04-25T10:17:07.637957+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:49.319459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287499+0000","last_change":"2024-04-25T10:17:06.308358+0000","last_active":"2024-04-25T10:17:07.287499+0000","last_peered":"2024-04-25T10:17:07.287499+0000","last_clean":"2024-04-25T10:17:07.287499+0000","last_became_active":"2024-04-25T10:17:06.307999+0000","last_became_peered":"2024-04-25T10:17:06.307999+0000","last_unstale":"2024-04-25T10:17:07.287499+0000","last_undegraded":"2024-04-25T10:17:07.287499+0000","last_fullsized":"2024-04-25T10:17:07.287499+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:23.198544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.300000+0000","last_change":"2024-04-25T10:17:06.308623+0000","last_active":"2024-04-25T10:17:07.300000+0000","last_peered":"2024-04-25T10:17:07.300000+0000","last_clean":"2024-04-25T10:17:07.300000+0000","last_became_active":"2024-04-25T10:17:06.308353+0000","last_became_peered":"2024-04-25T10:17:06.308353+0000","last_unstale":"2024-04-25T10:17:07.300000+0000","last_undegraded":"2024-04-25T10:17:07.300000+0000","last_fullsized":"2024-04-25T10:17:07.300000+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:15.841404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282246+0000","last_change":"2024-04-25T10:17:06.281476+0000","last_active":"2024-04-25T10:17:07.282246+0000","last_peered":"2024-04-25T10:17:07.282246+0000","last_clean":"2024-04-25T10:17:07.282246+0000","last_became_active":"2024-04-25T10:17:06.280613+0000","last_became_peered":"2024-04-25T10:17:06.280613+0000","last_unstale":"2024-04-25T10:17:07.282246+0000","last_undegraded":"2024-04-25T10:17:07.282246+0000","last_fullsized":"2024-04-25T10:17:07.282246+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:00.548089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078081+0000","last_change":"2024-04-25T10:17:06.309849+0000","last_active":"2024-04-25T10:17:08.078081+0000","last_peered":"2024-04-25T10:17:08.078081+0000","last_clean":"2024-04-25T10:17:08.078081+0000","last_became_active":"2024-04-25T10:17:06.309566+0000","last_became_peered":"2024-04-25T10:17:06.309566+0000","last_unstale":"2024-04-25T10:17:08.078081+0000","last_undegraded":"2024-04-25T10:17:08.078081+0000","last_fullsized":"2024-04-25T10:17:08.078081+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:04.975165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286660+0000","last_change":"2024-04-25T10:17:06.316432+0000","last_active":"2024-04-25T10:17:07.286660+0000","last_peered":"2024-04-25T10:17:07.286660+0000","last_clean":"2024-04-25T10:17:07.286660+0000","last_became_active":"2024-04-25T10:17:06.316148+0000","last_became_peered":"2024-04-25T10:17:06.316148+0000","last_unstale":"2024-04-25T10:17:07.286660+0000","last_undegraded":"2024-04-25T10:17:07.286660+0000","last_fullsized":"2024-04-25T10:17:07.286660+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:49.260972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285678+0000","last_change":"2024-04-25T10:17:06.308978+0000","last_active":"2024-04-25T10:17:07.285678+0000","last_peered":"2024-04-25T10:17:07.285678+0000","last_clean":"2024-04-25T10:17:07.285678+0000","last_became_active":"2024-04-25T10:17:06.308696+0000","last_became_peered":"2024-04-25T10:17:06.308696+0000","last_unstale":"2024-04-25T10:17:07.285678+0000","last_undegraded":"2024-04-25T10:17:07.285678+0000","last_fullsized":"2024-04-25T10:17:07.285678+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.443320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299399+0000","last_change":"2024-04-25T10:17:06.303492+0000","last_active":"2024-04-25T10:17:07.299399+0000","last_peered":"2024-04-25T10:17:07.299399+0000","last_clean":"2024-04-25T10:17:07.299399+0000","last_became_active":"2024-04-25T10:17:06.303212+0000","last_became_peered":"2024-04-25T10:17:06.303212+0000","last_unstale":"2024-04-25T10:17:07.299399+0000","last_undegraded":"2024-04-25T10:17:07.299399+0000","last_fullsized":"2024-04-25T10:17:07.299399+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:44.715314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.638080+0000","last_change":"2024-04-25T10:17:06.298837+0000","last_active":"2024-04-25T10:17:07.638080+0000","last_peered":"2024-04-25T10:17:07.638080+0000","last_clean":"2024-04-25T10:17:07.638080+0000","last_became_active":"2024-04-25T10:17:06.298056+0000","last_became_peered":"2024-04-25T10:17:06.298056+0000","last_unstale":"2024-04-25T10:17:07.638080+0000","last_undegraded":"2024-04-25T10:17:07.638080+0000","last_fullsized":"2024-04-25T10:17:07.638080+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:37.978479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637675+0000","last_change":"2024-04-25T10:17:06.295992+0000","last_active":"2024-04-25T10:17:07.637675+0000","last_peered":"2024-04-25T10:17:07.637675+0000","last_clean":"2024-04-25T10:17:07.637675+0000","last_became_active":"2024-04-25T10:17:06.295172+0000","last_became_peered":"2024-04-25T10:17:06.295172+0000","last_unstale":"2024-04-25T10:17:07.637675+0000","last_undegraded":"2024-04-25T10:17:07.637675+0000","last_fullsized":"2024-04-25T10:17:07.637675+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:45.792654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287442+0000","last_change":"2024-04-25T10:17:06.303425+0000","last_active":"2024-04-25T10:17:07.287442+0000","last_peered":"2024-04-25T10:17:07.287442+0000","last_clean":"2024-04-25T10:17:07.287442+0000","last_became_active":"2024-04-25T10:17:06.303023+0000","last_became_peered":"2024-04-25T10:17:06.303023+0000","last_unstale":"2024-04-25T10:17:07.287442+0000","last_undegraded":"2024-04-25T10:17:07.287442+0000","last_fullsized":"2024-04-25T10:17:07.287442+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.812969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286529+0000","last_change":"2024-04-25T10:17:06.298783+0000","last_active":"2024-04-25T10:17:07.286529+0000","last_peered":"2024-04-25T10:17:07.286529+0000","last_clean":"2024-04-25T10:17:07.286529+0000","last_became_active":"2024-04-25T10:17:06.298491+0000","last_became_peered":"2024-04-25T10:17:06.298491+0000","last_unstale":"2024-04-25T10:17:07.286529+0000","last_undegraded":"2024-04-25T10:17:07.286529+0000","last_fullsized":"2024-04-25T10:17:07.286529+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:11.986523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285772+0000","last_change":"2024-04-25T10:17:06.309141+0000","last_active":"2024-04-25T10:17:07.285772+0000","last_peered":"2024-04-25T10:17:07.285772+0000","last_clean":"2024-04-25T10:17:07.285772+0000","last_became_active":"2024-04-25T10:17:06.308818+0000","last_became_peered":"2024-04-25T10:17:06.308818+0000","last_unstale":"2024-04-25T10:17:07.285772+0000","last_undegraded":"2024-04-25T10:17:07.285772+0000","last_fullsized":"2024-04-25T10:17:07.285772+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:53.397605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285886+0000","last_change":"2024-04-25T10:17:06.312702+0000","last_active":"2024-04-25T10:17:07.285886+0000","last_peered":"2024-04-25T10:17:07.285886+0000","last_clean":"2024-04-25T10:17:07.285886+0000","last_became_active":"2024-04-25T10:17:06.312379+0000","last_became_peered":"2024-04-25T10:17:06.312379+0000","last_unstale":"2024-04-25T10:17:07.285886+0000","last_undegraded":"2024-04-25T10:17:07.285886+0000","last_fullsized":"2024-04-25T10:17:07.285886+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:02.587548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076441+0000","last_change":"2024-04-25T10:17:06.312878+0000","last_active":"2024-04-25T10:17:08.076441+0000","last_peered":"2024-04-25T10:17:08.076441+0000","last_clean":"2024-04-25T10:17:08.076441+0000","last_became_active":"2024-04-25T10:17:06.312559+0000","last_became_peered":"2024-04-25T10:17:06.312559+0000","last_unstale":"2024-04-25T10:17:08.076441+0000","last_undegraded":"2024-04-25T10:17:08.076441+0000","last_fullsized":"2024-04-25T10:17:08.076441+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:01.695656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.281057+0000","last_change":"2024-04-25T10:17:06.300742+0000","last_active":"2024-04-25T10:17:07.281057+0000","last_peered":"2024-04-25T10:17:07.281057+0000","last_clean":"2024-04-25T10:17:07.281057+0000","last_became_active":"2024-04-25T10:17:06.300393+0000","last_became_peered":"2024-04-25T10:17:06.300393+0000","last_unstale":"2024-04-25T10:17:07.281057+0000","last_undegraded":"2024-04-25T10:17:07.281057+0000","last_fullsized":"2024-04-25T10:17:07.281057+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:45.560858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.638164+0000","last_change":"2024-04-25T10:17:06.295132+0000","last_active":"2024-04-25T10:17:07.638164+0000","last_peered":"2024-04-25T10:17:07.638164+0000","last_clean":"2024-04-25T10:17:07.638164+0000","last_became_active":"2024-04-25T10:17:06.294396+0000","last_became_peered":"2024-04-25T10:17:06.294396+0000","last_unstale":"2024-04-25T10:17:07.638164+0000","last_undegraded":"2024-04-25T10:17:07.638164+0000","last_fullsized":"2024-04-25T10:17:07.638164+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:02.629892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079143+0000","last_change":"2024-04-25T10:17:06.306354+0000","last_active":"2024-04-25T10:17:08.079143+0000","last_peered":"2024-04-25T10:17:08.079143+0000","last_clean":"2024-04-25T10:17:08.079143+0000","last_became_active":"2024-04-25T10:17:06.306023+0000","last_became_peered":"2024-04-25T10:17:06.306023+0000","last_unstale":"2024-04-25T10:17:08.079143+0000","last_undegraded":"2024-04-25T10:17:08.079143+0000","last_fullsized":"2024-04-25T10:17:08.079143+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:56.696520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.300180+0000","last_change":"2024-04-25T10:17:06.300866+0000","last_active":"2024-04-25T10:17:07.300180+0000","last_peered":"2024-04-25T10:17:07.300180+0000","last_clean":"2024-04-25T10:17:07.300180+0000","last_became_active":"2024-04-25T10:17:06.300579+0000","last_became_peered":"2024-04-25T10:17:06.300579+0000","last_unstale":"2024-04-25T10:17:07.300180+0000","last_undegraded":"2024-04-25T10:17:07.300180+0000","last_fullsized":"2024-04-25T10:17:07.300180+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:18.541432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282909+0000","last_change":"2024-04-25T10:17:06.291974+0000","last_active":"2024-04-25T10:17:07.282909+0000","last_peered":"2024-04-25T10:17:07.282909+0000","last_clean":"2024-04-25T10:17:07.282909+0000","last_became_active":"2024-04-25T10:17:06.290630+0000","last_became_peered":"2024-04-25T10:17:06.290630+0000","last_unstale":"2024-04-25T10:17:07.282909+0000","last_undegraded":"2024-04-25T10:17:07.282909+0000","last_fullsized":"2024-04-25T10:17:07.282909+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:51.498124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637298+0000","last_change":"2024-04-25T10:17:06.280992+0000","last_active":"2024-04-25T10:17:07.637298+0000","last_peered":"2024-04-25T10:17:07.637298+0000","last_clean":"2024-04-25T10:17:07.637298+0000","last_became_active":"2024-04-25T10:17:06.278612+0000","last_became_peered":"2024-04-25T10:17:06.278612+0000","last_unstale":"2024-04-25T10:17:07.637298+0000","last_undegraded":"2024-04-25T10:17:07.637298+0000","last_fullsized":"2024-04-25T10:17:07.637298+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:49.221348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285350+0000","last_change":"2024-04-25T10:17:06.295566+0000","last_active":"2024-04-25T10:17:07.285350+0000","last_peered":"2024-04-25T10:17:07.285350+0000","last_clean":"2024-04-25T10:17:07.285350+0000","last_became_active":"2024-04-25T10:17:06.294921+0000","last_became_peered":"2024-04-25T10:17:06.294921+0000","last_unstale":"2024-04-25T10:17:07.285350+0000","last_undegraded":"2024-04-25T10:17:07.285350+0000","last_fullsized":"2024-04-25T10:17:07.285350+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:46.848723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077285+0000","last_change":"2024-04-25T10:17:06.307165+0000","last_active":"2024-04-25T10:17:08.077285+0000","last_peered":"2024-04-25T10:17:08.077285+0000","last_clean":"2024-04-25T10:17:08.077285+0000","last_became_active":"2024-04-25T10:17:06.306910+0000","last_became_peered":"2024-04-25T10:17:06.306910+0000","last_unstale":"2024-04-25T10:17:08.077285+0000","last_undegraded":"2024-04-25T10:17:08.077285+0000","last_fullsized":"2024-04-25T10:17:08.077285+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:05.505613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077146+0000","last_change":"2024-04-25T10:17:06.286675+0000","last_active":"2024-04-25T10:17:08.077146+0000","last_peered":"2024-04-25T10:17:08.077146+0000","last_clean":"2024-04-25T10:17:08.077146+0000","last_became_active":"2024-04-25T10:17:06.285954+0000","last_became_peered":"2024-04-25T10:17:06.285954+0000","last_unstale":"2024-04-25T10:17:08.077146+0000","last_undegraded":"2024-04-25T10:17:08.077146+0000","last_fullsized":"2024-04-25T10:17:08.077146+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:42.905954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077181+0000","last_change":"2024-04-25T10:17:06.302689+0000","last_active":"2024-04-25T10:17:08.077181+0000","last_peered":"2024-04-25T10:17:08.077181+0000","last_clean":"2024-04-25T10:17:08.077181+0000","last_became_active":"2024-04-25T10:17:06.302063+0000","last_became_peered":"2024-04-25T10:17:06.302063+0000","last_unstale":"2024-04-25T10:17:08.077181+0000","last_undegraded":"2024-04-25T10:17:08.077181+0000","last_fullsized":"2024-04-25T10:17:08.077181+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:06.488061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"182'4","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286596+0000","last_change":"2024-04-25T10:17:05.274637+0000","last_active":"2024-04-25T10:17:07.286596+0000","last_peered":"2024-04-25T10:17:07.286596+0000","last_clean":"2024-04-25T10:17:07.286596+0000","last_became_active":"2024-04-25T10:17:05.273900+0000","last_became_peered":"2024-04-25T10:17:05.273900+0000","last_unstale":"2024-04-25T10:17:07.286596+0000","last_undegraded":"2024-04-25T10:17:07.286596+0000","last_fullsized":"2024-04-25T10:17:07.286596+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:30.759665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078367+0000","last_change":"2024-04-25T10:17:06.305742+0000","last_active":"2024-04-25T10:17:08.078367+0000","last_peered":"2024-04-25T10:17:08.078367+0000","last_clean":"2024-04-25T10:17:08.078367+0000","last_became_active":"2024-04-25T10:17:06.305363+0000","last_became_peered":"2024-04-25T10:17:06.305363+0000","last_unstale":"2024-04-25T10:17:08.078367+0000","last_undegraded":"2024-04-25T10:17:08.078367+0000","last_fullsized":"2024-04-25T10:17:08.078367+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:59.006495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079156+0000","last_change":"2024-04-25T10:17:05.280219+0000","last_active":"2024-04-25T10:17:08.079156+0000","last_peered":"2024-04-25T10:17:08.079156+0000","last_clean":"2024-04-25T10:17:08.079156+0000","last_became_active":"2024-04-25T10:17:05.269919+0000","last_became_peered":"2024-04-25T10:17:05.269919+0000","last_unstale":"2024-04-25T10:17:08.079156+0000","last_undegraded":"2024-04-25T10:17:08.079156+0000","last_fullsized":"2024-04-25T10:17:08.079156+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:50.572814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286244+0000","last_change":"2024-04-25T10:17:06.312559+0000","last_active":"2024-04-25T10:17:07.286244+0000","last_peered":"2024-04-25T10:17:07.286244+0000","last_clean":"2024-04-25T10:17:07.286244+0000","last_became_active":"2024-04-25T10:17:06.312115+0000","last_became_peered":"2024-04-25T10:17:06.312115+0000","last_unstale":"2024-04-25T10:17:07.286244+0000","last_undegraded":"2024-04-25T10:17:07.286244+0000","last_fullsized":"2024-04-25T10:17:07.286244+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:27.892291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076748+0000","last_change":"2024-04-25T10:17:05.252648+0000","last_active":"2024-04-25T10:17:08.076748+0000","last_peered":"2024-04-25T10:17:08.076748+0000","last_clean":"2024-04-25T10:17:08.076748+0000","last_became_active":"2024-04-25T10:17:05.252325+0000","last_became_peered":"2024-04-25T10:17:05.252325+0000","last_unstale":"2024-04-25T10:17:08.076748+0000","last_undegraded":"2024-04-25T10:17:08.076748+0000","last_fullsized":"2024-04-25T10:17:08.076748+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:41.072995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282411+0000","last_change":"2024-04-25T10:17:06.295782+0000","last_active":"2024-04-25T10:17:07.282411+0000","last_peered":"2024-04-25T10:17:07.282411+0000","last_clean":"2024-04-25T10:17:07.282411+0000","last_became_active":"2024-04-25T10:17:06.294875+0000","last_became_peered":"2024-04-25T10:17:06.294875+0000","last_unstale":"2024-04-25T10:17:07.282411+0000","last_undegraded":"2024-04-25T10:17:07.282411+0000","last_fullsized":"2024-04-25T10:17:07.282411+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:27.268666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637396+0000","last_change":"2024-04-25T10:17:05.273926+0000","last_active":"2024-04-25T10:17:07.637396+0000","last_peered":"2024-04-25T10:17:07.637396+0000","last_clean":"2024-04-25T10:17:07.637396+0000","last_became_active":"2024-04-25T10:17:05.273604+0000","last_became_peered":"2024-04-25T10:17:05.273604+0000","last_unstale":"2024-04-25T10:17:07.637396+0000","last_undegraded":"2024-04-25T10:17:07.637396+0000","last_fullsized":"2024-04-25T10:17:07.637396+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:13.306138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286801+0000","last_change":"2024-04-25T10:17:06.304864+0000","last_active":"2024-04-25T10:17:07.286801+0000","last_peered":"2024-04-25T10:17:07.286801+0000","last_clean":"2024-04-25T10:17:07.286801+0000","last_became_active":"2024-04-25T10:17:06.304534+0000","last_became_peered":"2024-04-25T10:17:06.304534+0000","last_unstale":"2024-04-25T10:17:07.286801+0000","last_undegraded":"2024-04-25T10:17:07.286801+0000","last_fullsized":"2024-04-25T10:17:07.286801+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:49.803168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078219+0000","last_change":"2024-04-25T10:17:05.279953+0000","last_active":"2024-04-25T10:17:08.078219+0000","last_peered":"2024-04-25T10:17:08.078219+0000","last_clean":"2024-04-25T10:17:08.078219+0000","last_became_active":"2024-04-25T10:17:05.277343+0000","last_became_peered":"2024-04-25T10:17:05.277343+0000","last_unstale":"2024-04-25T10:17:08.078219+0000","last_undegraded":"2024-04-25T10:17:08.078219+0000","last_fullsized":"2024-04-25T10:17:08.078219+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:53.402593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287147+0000","last_change":"2024-04-25T10:17:06.305581+0000","last_active":"2024-04-25T10:17:07.287147+0000","last_peered":"2024-04-25T10:17:07.287147+0000","last_clean":"2024-04-25T10:17:07.287147+0000","last_became_active":"2024-04-25T10:17:06.305289+0000","last_became_peered":"2024-04-25T10:17:06.305289+0000","last_unstale":"2024-04-25T10:17:07.287147+0000","last_undegraded":"2024-04-25T10:17:07.287147+0000","last_fullsized":"2024-04-25T10:17:07.287147+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:44.205272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077433+0000","last_change":"2024-04-25T10:17:05.283780+0000","last_active":"2024-04-25T10:17:08.077433+0000","last_peered":"2024-04-25T10:17:08.077433+0000","last_clean":"2024-04-25T10:17:08.077433+0000","last_became_active":"2024-04-25T10:17:05.282949+0000","last_became_peered":"2024-04-25T10:17:05.282949+0000","last_unstale":"2024-04-25T10:17:08.077433+0000","last_undegraded":"2024-04-25T10:17:08.077433+0000","last_fullsized":"2024-04-25T10:17:08.077433+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:53.803204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.638453+0000","last_change":"2024-04-25T10:17:06.280219+0000","last_active":"2024-04-25T10:17:07.638453+0000","last_peered":"2024-04-25T10:17:07.638453+0000","last_clean":"2024-04-25T10:17:07.638453+0000","last_became_active":"2024-04-25T10:17:06.279521+0000","last_became_peered":"2024-04-25T10:17:06.279521+0000","last_unstale":"2024-04-25T10:17:07.638453+0000","last_undegraded":"2024-04-25T10:17:07.638453+0000","last_fullsized":"2024-04-25T10:17:07.638453+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:24.805711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299304+0000","last_change":"2024-04-25T10:17:05.276531+0000","last_active":"2024-04-25T10:17:07.299304+0000","last_peered":"2024-04-25T10:17:07.299304+0000","last_clean":"2024-04-25T10:17:07.299304+0000","last_became_active":"2024-04-25T10:17:05.276173+0000","last_became_peered":"2024-04-25T10:17:05.276173+0000","last_unstale":"2024-04-25T10:17:07.299304+0000","last_undegraded":"2024-04-25T10:17:07.299304+0000","last_fullsized":"2024-04-25T10:17:07.299304+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:29.917533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285316+0000","last_change":"2024-04-25T10:17:06.277049+0000","last_active":"2024-04-25T10:17:07.285316+0000","last_peered":"2024-04-25T10:17:07.285316+0000","last_clean":"2024-04-25T10:17:07.285316+0000","last_became_active":"2024-04-25T10:17:06.276270+0000","last_became_peered":"2024-04-25T10:17:06.276270+0000","last_unstale":"2024-04-25T10:17:07.285316+0000","last_undegraded":"2024-04-25T10:17:07.285316+0000","last_fullsized":"2024-04-25T10:17:07.285316+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:01.636583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282842+0000","last_change":"2024-04-25T10:17:05.261659+0000","last_active":"2024-04-25T10:17:07.282842+0000","last_peered":"2024-04-25T10:17:07.282842+0000","last_clean":"2024-04-25T10:17:07.282842+0000","last_became_active":"2024-04-25T10:17:05.261064+0000","last_became_peered":"2024-04-25T10:17:05.261064+0000","last_unstale":"2024-04-25T10:17:07.282842+0000","last_undegraded":"2024-04-25T10:17:07.282842+0000","last_fullsized":"2024-04-25T10:17:07.282842+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:27.695631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299835+0000","last_change":"2024-04-25T10:17:06.308008+0000","last_active":"2024-04-25T10:17:07.299835+0000","last_peered":"2024-04-25T10:17:07.299835+0000","last_clean":"2024-04-25T10:17:07.299835+0000","last_became_active":"2024-04-25T10:17:06.307274+0000","last_became_peered":"2024-04-25T10:17:06.307274+0000","last_unstale":"2024-04-25T10:17:07.299835+0000","last_undegraded":"2024-04-25T10:17:07.299835+0000","last_fullsized":"2024-04-25T10:17:07.299835+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:05.329111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078583+0000","last_change":"2024-04-25T10:17:05.292064+0000","last_active":"2024-04-25T10:17:08.078583+0000","last_peered":"2024-04-25T10:17:08.078583+0000","last_clean":"2024-04-25T10:17:08.078583+0000","last_became_active":"2024-04-25T10:17:05.291243+0000","last_became_peered":"2024-04-25T10:17:05.291243+0000","last_unstale":"2024-04-25T10:17:08.078583+0000","last_undegraded":"2024-04-25T10:17:08.078583+0000","last_fullsized":"2024-04-25T10:17:08.078583+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.532980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637202+0000","last_change":"2024-04-25T10:17:06.311718+0000","last_active":"2024-04-25T10:17:07.637202+0000","last_peered":"2024-04-25T10:17:07.637202+0000","last_clean":"2024-04-25T10:17:07.637202+0000","last_became_active":"2024-04-25T10:17:06.311471+0000","last_became_peered":"2024-04-25T10:17:06.311471+0000","last_unstale":"2024-04-25T10:17:07.637202+0000","last_undegraded":"2024-04-25T10:17:07.637202+0000","last_fullsized":"2024-04-25T10:17:07.637202+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:58.375434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"182'3","reported_seq":17,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637600+0000","last_change":"2024-04-25T10:17:05.253995+0000","last_active":"2024-04-25T10:17:07.637600+0000","last_peered":"2024-04-25T10:17:07.637600+0000","last_clean":"2024-04-25T10:17:07.637600+0000","last_became_active":"2024-04-25T10:17:05.253098+0000","last_became_peered":"2024-04-25T10:17:05.253098+0000","last_unstale":"2024-04-25T10:17:07.637600+0000","last_undegraded":"2024-04-25T10:17:07.637600+0000","last_fullsized":"2024-04-25T10:17:07.637600+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:12.401265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282759+0000","last_change":"2024-04-25T10:17:06.302036+0000","last_active":"2024-04-25T10:17:07.282759+0000","last_peered":"2024-04-25T10:17:07.282759+0000","last_clean":"2024-04-25T10:17:07.282759+0000","last_became_active":"2024-04-25T10:17:06.301271+0000","last_became_peered":"2024-04-25T10:17:06.301271+0000","last_unstale":"2024-04-25T10:17:07.282759+0000","last_undegraded":"2024-04-25T10:17:07.282759+0000","last_fullsized":"2024-04-25T10:17:07.282759+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:16.342639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"182'6","reported_seq":20,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078241+0000","last_change":"2024-04-25T10:17:05.277327+0000","last_active":"2024-04-25T10:17:08.078241+0000","last_peered":"2024-04-25T10:17:08.078241+0000","last_clean":"2024-04-25T10:17:08.078241+0000","last_became_active":"2024-04-25T10:17:05.276483+0000","last_became_peered":"2024-04-25T10:17:05.276483+0000","last_unstale":"2024-04-25T10:17:08.078241+0000","last_undegraded":"2024-04-25T10:17:08.078241+0000","last_fullsized":"2024-04-25T10:17:08.078241+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:46.938160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076940+0000","last_change":"2024-04-25T10:17:06.308192+0000","last_active":"2024-04-25T10:17:08.076940+0000","last_peered":"2024-04-25T10:17:08.076940+0000","last_clean":"2024-04-25T10:17:08.076940+0000","last_became_active":"2024-04-25T10:17:06.307915+0000","last_became_peered":"2024-04-25T10:17:06.307915+0000","last_unstale":"2024-04-25T10:17:08.076940+0000","last_undegraded":"2024-04-25T10:17:08.076940+0000","last_fullsized":"2024-04-25T10:17:08.076940+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:40.876455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076689+0000","last_change":"2024-04-25T10:17:05.265780+0000","last_active":"2024-04-25T10:17:08.076689+0000","last_peered":"2024-04-25T10:17:08.076689+0000","last_clean":"2024-04-25T10:17:08.076689+0000","last_became_active":"2024-04-25T10:17:05.265463+0000","last_became_peered":"2024-04-25T10:17:05.265463+0000","last_unstale":"2024-04-25T10:17:08.076689+0000","last_undegraded":"2024-04-25T10:17:08.076689+0000","last_fullsized":"2024-04-25T10:17:08.076689+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:13.306138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.300218+0000","last_change":"2024-04-25T10:17:06.292009+0000","last_active":"2024-04-25T10:17:07.300218+0000","last_peered":"2024-04-25T10:17:07.300218+0000","last_clean":"2024-04-25T10:17:07.300218+0000","last_became_active":"2024-04-25T10:17:06.291005+0000","last_became_peered":"2024-04-25T10:17:06.291005+0000","last_unstale":"2024-04-25T10:17:07.300218+0000","last_undegraded":"2024-04-25T10:17:07.300218+0000","last_fullsized":"2024-04-25T10:17:07.300218+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:17.023613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299363+0000","last_change":"2024-04-25T10:17:05.259439+0000","last_active":"2024-04-25T10:17:07.299363+0000","last_peered":"2024-04-25T10:17:07.299363+0000","last_clean":"2024-04-25T10:17:07.299363+0000","last_became_active":"2024-04-25T10:17:05.259119+0000","last_became_peered":"2024-04-25T10:17:05.259119+0000","last_unstale":"2024-04-25T10:17:07.299363+0000","last_undegraded":"2024-04-25T10:17:07.299363+0000","last_fullsized":"2024-04-25T10:17:07.299363+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:57.994043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637922+0000","last_change":"2024-04-25T10:17:06.301936+0000","last_active":"2024-04-25T10:17:07.637922+0000","last_peered":"2024-04-25T10:17:07.637922+0000","last_clean":"2024-04-25T10:17:07.637922+0000","last_became_active":"2024-04-25T10:17:06.301279+0000","last_became_peered":"2024-04-25T10:17:06.301279+0000","last_unstale":"2024-04-25T10:17:07.637922+0000","last_undegraded":"2024-04-25T10:17:07.637922+0000","last_fullsized":"2024-04-25T10:17:07.637922+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:27.368095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.636758+0000","last_change":"2024-04-25T10:17:05.269183+0000","last_active":"2024-04-25T10:17:07.636758+0000","last_peered":"2024-04-25T10:17:07.636758+0000","last_clean":"2024-04-25T10:17:07.636758+0000","last_became_active":"2024-04-25T10:17:05.268612+0000","last_became_peered":"2024-04-25T10:17:05.268612+0000","last_unstale":"2024-04-25T10:17:07.636758+0000","last_undegraded":"2024-04-25T10:17:07.636758+0000","last_fullsized":"2024-04-25T10:17:07.636758+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.532980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077948+0000","last_change":"2024-04-25T10:17:06.313653+0000","last_active":"2024-04-25T10:17:08.077948+0000","last_peered":"2024-04-25T10:17:08.077948+0000","last_clean":"2024-04-25T10:17:08.077948+0000","last_became_active":"2024-04-25T10:17:06.313405+0000","last_became_peered":"2024-04-25T10:17:06.313405+0000","last_unstale":"2024-04-25T10:17:08.077948+0000","last_undegraded":"2024-04-25T10:17:08.077948+0000","last_fullsized":"2024-04-25T10:17:08.077948+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:56.971134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"182'8","reported_seq":22,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.280970+0000","last_change":"2024-04-25T10:17:05.274541+0000","last_active":"2024-04-25T10:17:07.280970+0000","last_peered":"2024-04-25T10:17:07.280970+0000","last_clean":"2024-04-25T10:17:07.280970+0000","last_became_active":"2024-04-25T10:17:05.274012+0000","last_became_peered":"2024-04-25T10:17:05.274012+0000","last_unstale":"2024-04-25T10:17:07.280970+0000","last_undegraded":"2024-04-25T10:17:07.280970+0000","last_fullsized":"2024-04-25T10:17:07.280970+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:59.380156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287431+0000","last_change":"2024-04-25T10:17:06.315211+0000","last_active":"2024-04-25T10:17:07.287431+0000","last_peered":"2024-04-25T10:17:07.287431+0000","last_clean":"2024-04-25T10:17:07.287431+0000","last_became_active":"2024-04-25T10:17:06.312743+0000","last_became_peered":"2024-04-25T10:17:06.312743+0000","last_unstale":"2024-04-25T10:17:07.287431+0000","last_undegraded":"2024-04-25T10:17:07.287431+0000","last_fullsized":"2024-04-25T10:17:07.287431+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:44.422561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079301+0000","last_change":"2024-04-25T10:17:05.287915+0000","last_active":"2024-04-25T10:17:08.079301+0000","last_peered":"2024-04-25T10:17:08.079301+0000","last_clean":"2024-04-25T10:17:08.079301+0000","last_became_active":"2024-04-25T10:17:05.287226+0000","last_became_peered":"2024-04-25T10:17:05.287226+0000","last_unstale":"2024-04-25T10:17:08.079301+0000","last_undegraded":"2024-04-25T10:17:08.079301+0000","last_fullsized":"2024-04-25T10:17:08.079301+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:03.313674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078742+0000","last_change":"2024-04-25T10:17:06.317023+0000","last_active":"2024-04-25T10:17:08.078742+0000","last_peered":"2024-04-25T10:17:08.078742+0000","last_clean":"2024-04-25T10:17:08.078742+0000","last_became_active":"2024-04-25T10:17:06.316453+0000","last_became_peered":"2024-04-25T10:17:06.316453+0000","last_unstale":"2024-04-25T10:17:08.078742+0000","last_undegraded":"2024-04-25T10:17:08.078742+0000","last_fullsized":"2024-04-25T10:17:08.078742+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:30.842625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079283+0000","last_change":"2024-04-25T10:17:05.285851+0000","last_active":"2024-04-25T10:17:08.079283+0000","last_peered":"2024-04-25T10:17:08.079283+0000","last_clean":"2024-04-25T10:17:08.079283+0000","last_became_active":"2024-04-25T10:17:05.284722+0000","last_became_peered":"2024-04-25T10:17:05.284722+0000","last_unstale":"2024-04-25T10:17:08.079283+0000","last_undegraded":"2024-04-25T10:17:08.079283+0000","last_fullsized":"2024-04-25T10:17:08.079283+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:58.353363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078687+0000","last_change":"2024-04-25T10:17:06.311043+0000","last_active":"2024-04-25T10:17:08.078687+0000","last_peered":"2024-04-25T10:17:08.078687+0000","last_clean":"2024-04-25T10:17:08.078687+0000","last_became_active":"2024-04-25T10:17:06.310681+0000","last_became_peered":"2024-04-25T10:17:06.310681+0000","last_unstale":"2024-04-25T10:17:08.078687+0000","last_undegraded":"2024-04-25T10:17:08.078687+0000","last_fullsized":"2024-04-25T10:17:08.078687+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:09.040936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076754+0000","last_change":"2024-04-25T10:17:05.276187+0000","last_active":"2024-04-25T10:17:08.076754+0000","last_peered":"2024-04-25T10:17:08.076754+0000","last_clean":"2024-04-25T10:17:08.076754+0000","last_became_active":"2024-04-25T10:17:05.275862+0000","last_became_peered":"2024-04-25T10:17:05.275862+0000","last_unstale":"2024-04-25T10:17:08.076754+0000","last_undegraded":"2024-04-25T10:17:08.076754+0000","last_fullsized":"2024-04-25T10:17:08.076754+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:17.844725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287189+0000","last_change":"2024-04-25T10:17:06.303966+0000","last_active":"2024-04-25T10:17:07.287189+0000","last_peered":"2024-04-25T10:17:07.287189+0000","last_clean":"2024-04-25T10:17:07.287189+0000","last_became_active":"2024-04-25T10:17:06.303463+0000","last_became_peered":"2024-04-25T10:17:06.303463+0000","last_unstale":"2024-04-25T10:17:07.287189+0000","last_undegraded":"2024-04-25T10:17:07.287189+0000","last_fullsized":"2024-04-25T10:17:07.287189+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:28.438816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282380+0000","last_change":"2024-04-25T10:17:05.288488+0000","last_active":"2024-04-25T10:17:07.282380+0000","last_peered":"2024-04-25T10:17:07.282380+0000","last_clean":"2024-04-25T10:17:07.282380+0000","last_became_active":"2024-04-25T10:17:05.288145+0000","last_became_peered":"2024-04-25T10:17:05.288145+0000","last_unstale":"2024-04-25T10:17:07.282380+0000","last_undegraded":"2024-04-25T10:17:07.282380+0000","last_fullsized":"2024-04-25T10:17:07.282380+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:58.978256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077035+0000","last_change":"2024-04-25T10:17:06.299136+0000","last_active":"2024-04-25T10:17:08.077035+0000","last_peered":"2024-04-25T10:17:08.077035+0000","last_clean":"2024-04-25T10:17:08.077035+0000","last_became_active":"2024-04-25T10:17:06.293833+0000","last_became_peered":"2024-04-25T10:17:06.293833+0000","last_unstale":"2024-04-25T10:17:08.077035+0000","last_undegraded":"2024-04-25T10:17:08.077035+0000","last_fullsized":"2024-04-25T10:17:08.077035+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:42.905954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.282539+0000","last_change":"2024-04-25T10:17:05.261872+0000","last_active":"2024-04-25T10:17:07.282539+0000","last_peered":"2024-04-25T10:17:07.282539+0000","last_clean":"2024-04-25T10:17:07.282539+0000","last_became_active":"2024-04-25T10:17:05.261364+0000","last_became_peered":"2024-04-25T10:17:05.261364+0000","last_unstale":"2024-04-25T10:17:07.282539+0000","last_undegraded":"2024-04-25T10:17:07.282539+0000","last_fullsized":"2024-04-25T10:17:07.282539+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:29.091373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077792+0000","last_change":"2024-04-25T10:17:06.302371+0000","last_active":"2024-04-25T10:17:08.077792+0000","last_peered":"2024-04-25T10:17:08.077792+0000","last_clean":"2024-04-25T10:17:08.077792+0000","last_became_active":"2024-04-25T10:17:06.302053+0000","last_became_peered":"2024-04-25T10:17:06.302053+0000","last_unstale":"2024-04-25T10:17:08.077792+0000","last_undegraded":"2024-04-25T10:17:08.077792+0000","last_fullsized":"2024-04-25T10:17:08.077792+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:51.335789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.076423+0000","last_change":"2024-04-25T10:17:05.262142+0000","last_active":"2024-04-25T10:17:08.076423+0000","last_peered":"2024-04-25T10:17:08.076423+0000","last_clean":"2024-04-25T10:17:08.076423+0000","last_became_active":"2024-04-25T10:17:05.261685+0000","last_became_peered":"2024-04-25T10:17:05.261685+0000","last_unstale":"2024-04-25T10:17:08.076423+0000","last_undegraded":"2024-04-25T10:17:08.076423+0000","last_fullsized":"2024-04-25T10:17:08.076423+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.532980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.638007+0000","last_change":"2024-04-25T10:17:06.281624+0000","last_active":"2024-04-25T10:17:07.638007+0000","last_peered":"2024-04-25T10:17:07.638007+0000","last_clean":"2024-04-25T10:17:07.638007+0000","last_became_active":"2024-04-25T10:17:06.280903+0000","last_became_peered":"2024-04-25T10:17:06.280903+0000","last_unstale":"2024-04-25T10:17:07.638007+0000","last_undegraded":"2024-04-25T10:17:07.638007+0000","last_fullsized":"2024-04-25T10:17:07.638007+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:13.013098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285672+0000","last_change":"2024-04-25T10:17:05.280949+0000","last_active":"2024-04-25T10:17:07.285672+0000","last_peered":"2024-04-25T10:17:07.285672+0000","last_clean":"2024-04-25T10:17:07.285672+0000","last_became_active":"2024-04-25T10:17:05.277678+0000","last_became_peered":"2024-04-25T10:17:05.277678+0000","last_unstale":"2024-04-25T10:17:07.285672+0000","last_undegraded":"2024-04-25T10:17:07.285672+0000","last_fullsized":"2024-04-25T10:17:07.285672+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:37.094026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287855+0000","last_change":"2024-04-25T10:17:06.312734+0000","last_active":"2024-04-25T10:17:07.287855+0000","last_peered":"2024-04-25T10:17:07.287855+0000","last_clean":"2024-04-25T10:17:07.287855+0000","last_became_active":"2024-04-25T10:17:06.312278+0000","last_became_peered":"2024-04-25T10:17:06.312278+0000","last_unstale":"2024-04-25T10:17:07.287855+0000","last_undegraded":"2024-04-25T10:17:07.287855+0000","last_fullsized":"2024-04-25T10:17:07.287855+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:13.273521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078763+0000","last_change":"2024-04-25T10:17:05.284291+0000","last_active":"2024-04-25T10:17:08.078763+0000","last_peered":"2024-04-25T10:17:08.078763+0000","last_clean":"2024-04-25T10:17:08.078763+0000","last_became_active":"2024-04-25T10:17:05.283622+0000","last_became_peered":"2024-04-25T10:17:05.283622+0000","last_unstale":"2024-04-25T10:17:08.078763+0000","last_undegraded":"2024-04-25T10:17:08.078763+0000","last_fullsized":"2024-04-25T10:17:08.078763+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:43.192820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285134+0000","last_change":"2024-04-25T10:17:06.308081+0000","last_active":"2024-04-25T10:17:07.285134+0000","last_peered":"2024-04-25T10:17:07.285134+0000","last_clean":"2024-04-25T10:17:07.285134+0000","last_became_active":"2024-04-25T10:17:06.307803+0000","last_became_peered":"2024-04-25T10:17:06.307803+0000","last_unstale":"2024-04-25T10:17:07.285134+0000","last_undegraded":"2024-04-25T10:17:07.285134+0000","last_fullsized":"2024-04-25T10:17:07.285134+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:57.829524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.638244+0000","last_change":"2024-04-25T10:17:05.255829+0000","last_active":"2024-04-25T10:17:07.638244+0000","last_peered":"2024-04-25T10:17:07.638244+0000","last_clean":"2024-04-25T10:17:07.638244+0000","last_became_active":"2024-04-25T10:17:05.255551+0000","last_became_peered":"2024-04-25T10:17:05.255551+0000","last_unstale":"2024-04-25T10:17:07.638244+0000","last_undegraded":"2024-04-25T10:17:07.638244+0000","last_fullsized":"2024-04-25T10:17:07.638244+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:53.175315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299504+0000","last_change":"2024-04-25T10:17:06.309951+0000","last_active":"2024-04-25T10:17:07.299504+0000","last_peered":"2024-04-25T10:17:07.299504+0000","last_clean":"2024-04-25T10:17:07.299504+0000","last_became_active":"2024-04-25T10:17:06.309400+0000","last_became_peered":"2024-04-25T10:17:06.309400+0000","last_unstale":"2024-04-25T10:17:07.299504+0000","last_undegraded":"2024-04-25T10:17:07.299504+0000","last_fullsized":"2024-04-25T10:17:07.299504+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:36.045880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"183'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.287687+0000","last_change":"2024-04-25T10:17:05.266933+0000","last_active":"2024-04-25T10:17:07.287687+0000","last_peered":"2024-04-25T10:17:07.287687+0000","last_clean":"2024-04-25T10:17:07.287687+0000","last_became_active":"2024-04-25T10:17:05.266096+0000","last_became_peered":"2024-04-25T10:17:05.266096+0000","last_unstale":"2024-04-25T10:17:07.287687+0000","last_undegraded":"2024-04-25T10:17:07.287687+0000","last_fullsized":"2024-04-25T10:17:07.287687+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:20.210981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637171+0000","last_change":"2024-04-25T10:17:06.305649+0000","last_active":"2024-04-25T10:17:07.637171+0000","last_peered":"2024-04-25T10:17:07.637171+0000","last_clean":"2024-04-25T10:17:07.637171+0000","last_became_active":"2024-04-25T10:17:06.305256+0000","last_became_peered":"2024-04-25T10:17:06.305256+0000","last_unstale":"2024-04-25T10:17:07.637171+0000","last_undegraded":"2024-04-25T10:17:07.637171+0000","last_fullsized":"2024-04-25T10:17:07.637171+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:14.977772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286519+0000","last_change":"2024-04-25T10:17:05.270685+0000","last_active":"2024-04-25T10:17:07.286519+0000","last_peered":"2024-04-25T10:17:07.286519+0000","last_clean":"2024-04-25T10:17:07.286519+0000","last_became_active":"2024-04-25T10:17:05.269724+0000","last_became_peered":"2024-04-25T10:17:05.269724+0000","last_unstale":"2024-04-25T10:17:07.286519+0000","last_undegraded":"2024-04-25T10:17:07.286519+0000","last_fullsized":"2024-04-25T10:17:07.286519+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.189977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077807+0000","last_change":"2024-04-25T10:17:06.312347+0000","last_active":"2024-04-25T10:17:08.077807+0000","last_peered":"2024-04-25T10:17:08.077807+0000","last_clean":"2024-04-25T10:17:08.077807+0000","last_became_active":"2024-04-25T10:17:06.311697+0000","last_became_peered":"2024-04-25T10:17:06.311697+0000","last_unstale":"2024-04-25T10:17:08.077807+0000","last_undegraded":"2024-04-25T10:17:08.077807+0000","last_fullsized":"2024-04-25T10:17:08.077807+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:30.929985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078334+0000","last_change":"2024-04-25T10:17:05.288602+0000","last_active":"2024-04-25T10:17:08.078334+0000","last_peered":"2024-04-25T10:17:08.078334+0000","last_clean":"2024-04-25T10:17:08.078334+0000","last_became_active":"2024-04-25T10:17:05.287869+0000","last_became_peered":"2024-04-25T10:17:05.287869+0000","last_unstale":"2024-04-25T10:17:08.078334+0000","last_undegraded":"2024-04-25T10:17:08.078334+0000","last_fullsized":"2024-04-25T10:17:08.078334+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:42.530292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.286401+0000","last_change":"2024-04-25T10:17:06.307406+0000","last_active":"2024-04-25T10:17:07.286401+0000","last_peered":"2024-04-25T10:17:07.286401+0000","last_clean":"2024-04-25T10:17:07.286401+0000","last_became_active":"2024-04-25T10:17:06.303507+0000","last_became_peered":"2024-04-25T10:17:06.303507+0000","last_unstale":"2024-04-25T10:17:07.286401+0000","last_undegraded":"2024-04-25T10:17:07.286401+0000","last_fullsized":"2024-04-25T10:17:07.286401+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:51.585849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.285915+0000","last_change":"2024-04-25T10:17:05.258680+0000","last_active":"2024-04-25T10:17:07.285915+0000","last_peered":"2024-04-25T10:17:07.285915+0000","last_clean":"2024-04-25T10:17:07.285915+0000","last_became_active":"2024-04-25T10:17:05.257882+0000","last_became_peered":"2024-04-25T10:17:05.257882+0000","last_unstale":"2024-04-25T10:17:07.285915+0000","last_undegraded":"2024-04-25T10:17:07.285915+0000","last_fullsized":"2024-04-25T10:17:07.285915+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:06.305991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078841+0000","last_change":"2024-04-25T10:17:06.310242+0000","last_active":"2024-04-25T10:17:08.078841+0000","last_peered":"2024-04-25T10:17:08.078841+0000","last_clean":"2024-04-25T10:17:08.078841+0000","last_became_active":"2024-04-25T10:17:06.309971+0000","last_became_peered":"2024-04-25T10:17:06.309971+0000","last_unstale":"2024-04-25T10:17:08.078841+0000","last_undegraded":"2024-04-25T10:17:08.078841+0000","last_fullsized":"2024-04-25T10:17:08.078841+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:58.375434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637387+0000","last_change":"2024-04-25T10:17:05.273922+0000","last_active":"2024-04-25T10:17:07.637387+0000","last_peered":"2024-04-25T10:17:07.637387+0000","last_clean":"2024-04-25T10:17:07.637387+0000","last_became_active":"2024-04-25T10:17:05.273593+0000","last_became_peered":"2024-04-25T10:17:05.273593+0000","last_unstale":"2024-04-25T10:17:07.637387+0000","last_undegraded":"2024-04-25T10:17:07.637387+0000","last_fullsized":"2024-04-25T10:17:07.637387+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:17.844725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078448+0000","last_change":"2024-04-25T10:17:06.310744+0000","last_active":"2024-04-25T10:17:08.078448+0000","last_peered":"2024-04-25T10:17:08.078448+0000","last_clean":"2024-04-25T10:17:08.078448+0000","last_became_active":"2024-04-25T10:17:06.310060+0000","last_became_peered":"2024-04-25T10:17:06.310060+0000","last_unstale":"2024-04-25T10:17:08.078448+0000","last_undegraded":"2024-04-25T10:17:08.078448+0000","last_fullsized":"2024-04-25T10:17:08.078448+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:14.977772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.079010+0000","last_change":"2024-04-25T10:17:05.286524+0000","last_active":"2024-04-25T10:17:08.079010+0000","last_peered":"2024-04-25T10:17:08.079010+0000","last_clean":"2024-04-25T10:17:08.079010+0000","last_became_active":"2024-04-25T10:17:05.285810+0000","last_became_peered":"2024-04-25T10:17:05.285810+0000","last_unstale":"2024-04-25T10:17:08.079010+0000","last_undegraded":"2024-04-25T10:17:08.079010+0000","last_fullsized":"2024-04-25T10:17:08.079010+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:40.092160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.299942+0000","last_change":"2024-04-25T10:17:06.300864+0000","last_active":"2024-04-25T10:17:07.299942+0000","last_peered":"2024-04-25T10:17:07.299942+0000","last_clean":"2024-04-25T10:17:07.299942+0000","last_became_active":"2024-04-25T10:17:06.300535+0000","last_became_peered":"2024-04-25T10:17:06.300535+0000","last_unstale":"2024-04-25T10:17:07.299942+0000","last_undegraded":"2024-04-25T10:17:07.299942+0000","last_fullsized":"2024-04-25T10:17:07.299942+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:32.451444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.281119+0000","last_change":"2024-04-25T10:17:05.274379+0000","last_active":"2024-04-25T10:17:07.281119+0000","last_peered":"2024-04-25T10:17:07.281119+0000","last_clean":"2024-04-25T10:17:07.281119+0000","last_became_active":"2024-04-25T10:17:05.274028+0000","last_became_peered":"2024-04-25T10:17:05.274028+0000","last_unstale":"2024-04-25T10:17:07.281119+0000","last_undegraded":"2024-04-25T10:17:07.281119+0000","last_fullsized":"2024-04-25T10:17:07.281119+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:50.044993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.078928+0000","last_change":"2024-04-25T10:17:06.313892+0000","last_active":"2024-04-25T10:17:08.078928+0000","last_peered":"2024-04-25T10:17:08.078928+0000","last_clean":"2024-04-25T10:17:08.078928+0000","last_became_active":"2024-04-25T10:17:06.313558+0000","last_became_peered":"2024-04-25T10:17:06.313558+0000","last_unstale":"2024-04-25T10:17:08.078928+0000","last_undegraded":"2024-04-25T10:17:08.078928+0000","last_fullsized":"2024-04-25T10:17:08.078928+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:05.230999+0000","last_clean_scrub_stamp":"2024-04-25T10:17:05.230999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:13.660835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:07.637374+0000","last_change":"2024-04-25T10:17:05.253379+0000","last_active":"2024-04-25T10:17:07.637374+0000","last_peered":"2024-04-25T10:17:07.637374+0000","last_clean":"2024-04-25T10:17:07.637374+0000","last_became_active":"2024-04-25T10:17:05.252888+0000","last_became_peered":"2024-04-25T10:17:05.252888+0000","last_unstale":"2024-04-25T10:17:07.637374+0000","last_undegraded":"2024-04-25T10:17:07.637374+0000","last_fullsized":"2024-04-25T10:17:07.637374+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:04.218547+0000","last_clean_scrub_stamp":"2024-04-25T10:17:04.218547+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:41.072995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":477,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:17:08.077063+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:17:08.077063+0000","last_peered":"2024-04-25T10:17:08.077063+0000","last_clean":"2024-04-25T10:17:08.077063+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:17:08.077063+0000","last_undegraded":"2024-04-25T10:17:08.077063+0000","last_fullsized":"2024-04-25T10:17:08.077063+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74292,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297548,"statfs":{"total":96636764160,"available":96560689152,"internally_reserved":0,"allocated":2920448,"data_stored":1678264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575039,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51504,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320336,"statfs":{"total":96636764160,"available":96584024064,"internally_reserved":0,"allocated":2326528,"data_stored":1088408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47336,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324504,"statfs":{"total":96636764160,"available":96588292096,"internally_reserved":0,"allocated":2318336,"data_stored":1086270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95908,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275932,"statfs":{"total":96636764160,"available":96538554368,"internally_reserved":0,"allocated":2314240,"data_stored":1086796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82024,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289816,"statfs":{"total":96636764160,"available":96552771584,"internally_reserved":0,"allocated":2322432,"data_stored":1086340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575038,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51492,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320348,"statfs":{"total":96636764160,"available":96584036352,"internally_reserved":0,"allocated":2314240,"data_stored":1086304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47324,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324516,"statfs":{"total":96636764160,"available":96588304384,"internally_reserved":0,"allocated":2310144,"data_stored":1086214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575039,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74276,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297564,"statfs":{"total":96636764160,"available":96560705536,"internally_reserved":0,"allocated":2908160,"data_stored":1676672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:10.649 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:17:10.649 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:17:10.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421edde640 1 Processor -- start 2024-04-25T10:17:10.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421edde640 1 -- start start 2024-04-25T10:17:10.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421edde640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4218143ea0 0x7f42180cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421edde640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f421806eac0 0x7f42180b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421edde640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421815ac50 con 0x7f42180b2150 2024-04-25T10:17:10.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421edde640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42180cd180 con 0x7f4218143ea0 2024-04-25T10:17:10.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421edde640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421815aed0 con 0x7f421806eac0 2024-04-25T10:17:10.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421cb7a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4218143ea0 0x7f42180cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f4217fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f42180b2150 0x7f421806e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55570/0 (socket says 172.21.15.133:55570) 2024-04-25T10:17:10.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f4217fff640 1 -- 172.21.15.133:0/802701972 learned_addr learned my addr 172.21.15.133:0/802701972 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:10.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421d37b640 1 --2- 172.21.15.133:0/802701972 >> v2:172.21.15.162:3300/0 conn(0x7f421806eac0 0x7f42180b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f42177fe640 1 -- 172.21.15.133:0/802701972 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024880515 0 0) 0x7f420c001120 con 0x7f42180b2150 2024-04-25T10:17:10.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f42177fe640 1 -- 172.21.15.133:0/802701972 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4200003530 con 0x7f42180b2150 2024-04-25T10:17:10.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421cb7a640 1 -- 172.21.15.133:0/802701972 >> v2:172.21.15.162:3300/0 conn(0x7f421806eac0 msgr2=0x7f42180b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:10.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421cb7a640 1 --2- 172.21.15.133:0/802701972 >> v2:172.21.15.162:3300/0 conn(0x7f421806eac0 0x7f42180b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421cb7a640 1 -- 172.21.15.133:0/802701972 >> v1:172.21.15.133:6789/0 conn(0x7f42180b2150 legacy=0x7f421806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421cb7a640 1 -- 172.21.15.133:0/802701972 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421815b1b0 con 0x7f4218143ea0 2024-04-25T10:17:10.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.808+0000 7f421d37b640 1 --2- 172.21.15.133:0/802701972 >> v2:172.21.15.162:3300/0 conn(0x7f421806eac0 0x7f42180b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f421cb7a640 1 --2- 172.21.15.133:0/802701972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4218143ea0 0x7f42180cc810 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f42180cd820 tx=0x7f420801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c71dd10fd0aff56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f42177fe640 1 -- 172.21.15.133:0/802701972 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42080b0070 con 0x7f4218143ea0 2024-04-25T10:17:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f42177fe640 1 -- 172.21.15.133:0/802701972 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42080ab600 con 0x7f4218143ea0 2024-04-25T10:17:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f42177fe640 1 -- 172.21.15.133:0/802701972 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42080ab9d0 con 0x7f4218143ea0 2024-04-25T10:17:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f421edde640 1 -- 172.21.15.133:0/802701972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4218143ea0 msgr2=0x7f42180cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f421edde640 1 --2- 172.21.15.133:0/802701972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4218143ea0 0x7f42180cc810 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f42180cd820 tx=0x7f420801c3f0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f421edde640 1 -- 172.21.15.133:0/802701972 shutdown_connections 2024-04-25T10:17:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f421edde640 1 --2- 172.21.15.133:0/802701972 >> v2:172.21.15.162:3300/0 conn(0x7f421806eac0 0x7f42180b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f421edde640 1 --2- 172.21.15.133:0/802701972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4218143ea0 0x7f42180cc810 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f421edde640 1 -- 172.21.15.133:0/802701972 >> 172.21.15.133:0/802701972 conn(0x7f4218066f70 msgr2=0x7f421806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f421edde640 1 -- 172.21.15.133:0/802701972 shutdown_connections 2024-04-25T10:17:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f421edde640 1 -- 172.21.15.133:0/802701972 wait complete. 2024-04-25T10:17:10.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.812+0000 7f421edde640 1 Processor -- start 2024-04-25T10:17:10.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421edde640 1 -- start start 2024-04-25T10:17:10.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421edde640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f421806eac0 0x7f421815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421edde640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180b2150 0x7f4218150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421edde640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4218143ea0 0x7f42181f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421edde640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4218167550 con 0x7f4218143ea0 2024-04-25T10:17:10.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421edde640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42181673d0 con 0x7f42180b2150 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421edde640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42181676d0 con 0x7f421806eac0 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421d37b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4218143ea0 0x7f42181f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421d37b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4218143ea0 0x7f42181f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54716/0 (socket says 172.21.15.133:54716) 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421d37b640 1 -- 172.21.15.133:0/814327177 learned_addr learned my addr 172.21.15.133:0/814327177 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421cb7a640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f421806eac0 0x7f421815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f4217fff640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180b2150 0x7f4218150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421d37b640 1 -- 172.21.15.133:0/814327177 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f421806eac0 msgr2=0x7f421815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421d37b640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f421806eac0 0x7f421815cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421d37b640 1 -- 172.21.15.133:0/814327177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180b2150 msgr2=0x7f4218150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421d37b640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180b2150 0x7f4218150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421d37b640 1 -- 172.21.15.133:0/814327177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42181580d0 con 0x7f4218143ea0 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f4217fff640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180b2150 0x7f4218150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421d37b640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4218143ea0 0x7f42181f74f0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f421000cd50 tx=0x7f421000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f42157fa640 1 -- 172.21.15.133:0/814327177 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421000f680 con 0x7f4218143ea0 2024-04-25T10:17:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f42157fa640 1 -- 172.21.15.133:0/814327177 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421000f840 con 0x7f4218143ea0 2024-04-25T10:17:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f42157fa640 1 -- 172.21.15.133:0/814327177 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421000fc30 con 0x7f4218143ea0 2024-04-25T10:17:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421edde640 1 -- 172.21.15.133:0/814327177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42181583b0 con 0x7f4218143ea0 2024-04-25T10:17:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421edde640 1 -- 172.21.15.133:0/814327177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421815cae0 con 0x7f4218143ea0 2024-04-25T10:17:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421edde640 1 -- 172.21.15.133:0/814327177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4218068e80 con 0x7f4218143ea0 2024-04-25T10:17:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f42157fa640 1 -- 172.21.15.133:0/814327177 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f421000e3e0 con 0x7f4218143ea0 2024-04-25T10:17:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f42157fa640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4204066f90 0x7f4204069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f42157fa640 1 -- 172.21.15.133:0/814327177 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4210085e80 con 0x7f4218143ea0 2024-04-25T10:17:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.816+0000 7f421cb7a640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4204066f90 0x7f4204069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.820+0000 7f421cb7a640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4204066f90 0x7f4204069450 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f42080ad000 tx=0x7f42080a4f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:10.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:10.820+0000 7f42157fa640 1 -- 172.21.15.133:0/814327177 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4210014030 con 0x7f4218143ea0 2024-04-25T10:17:11.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.028+0000 7f421edde640 1 -- 172.21.15.133:0/814327177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f421813efe0 con 0x7f4218143ea0 2024-04-25T10:17:11.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.032+0000 7f42157fa640 1 -- 172.21.15.133:0/814327177 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4210056d90 con 0x7f4218143ea0 2024-04-25T10:17:11.034 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:17:11.034 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:17:11.034 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:17:11.034 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:17:11.035 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:17:11.035 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:17:11.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 -- 172.21.15.133:0/814327177 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4204066f90 msgr2=0x7f4204069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4204066f90 0x7f4204069450 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f42080ad000 tx=0x7f42080a4f90 comp rx=0 tx=0).stop 2024-04-25T10:17:11.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 -- 172.21.15.133:0/814327177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4218143ea0 msgr2=0x7f42181f74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4218143ea0 0x7f42181f74f0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f421000cd50 tx=0x7f421000a650 comp rx=0 tx=0).stop 2024-04-25T10:17:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 -- 172.21.15.133:0/814327177 shutdown_connections 2024-04-25T10:17:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4204066f90 0x7f4204069450 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4218143ea0 0x7f42181f74f0 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f42180b2150 0x7f4218150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 --2- 172.21.15.133:0/814327177 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f421806eac0 0x7f421815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 -- 172.21.15.133:0/814327177 >> 172.21.15.133:0/814327177 conn(0x7f4218066f70 msgr2=0x7f4218142d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 -- 172.21.15.133:0/814327177 shutdown_connections 2024-04-25T10:17:11.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.036+0000 7f421edde640 1 -- 172.21.15.133:0/814327177 wait complete. 2024-04-25T10:17:11.053 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:11.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.168+0000 7f197d3ec640 1 Processor -- start 2024-04-25T10:17:11.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.168+0000 7f197d3ec640 1 -- start start 2024-04-25T10:17:11.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f197d3ec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19780b1f70 0x7f197806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f197d3ec640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f19780ce330 0x7f19780ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f197d3ec640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19780c61d0 con 0x7f197815c9a0 2024-04-25T10:17:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f197d3ec640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19780cece0 con 0x7f19780ce330 2024-04-25T10:17:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f197d3ec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19780c64b0 con 0x7f19780b1f70 2024-04-25T10:17:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1976ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f197815c9a0 0x7f19780c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55576/0 (socket says 172.21.15.133:55576) 2024-04-25T10:17:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1976ffd640 1 -- 172.21.15.133:0/3081716953 learned_addr learned my addr 172.21.15.133:0/3081716953 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f19767fc640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19780b1f70 0x7f197806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f19777fe640 1 --2- 172.21.15.133:0/3081716953 >> v2:172.21.15.162:3300/0 conn(0x7f19780ce330 0x7f19780ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876193013 0 0) 0x7f19780c61d0 con 0x7f197815c9a0 2024-04-25T10:17:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1960003530 con 0x7f197815c9a0 2024-04-25T10:17:11.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1226496113 0 0) 0x7f1960003530 con 0x7f197815c9a0 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19780b1f70 msgr2=0x7f197806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19780b1f70 0x7f197806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 >> v2:172.21.15.162:3300/0 conn(0x7f19780ce330 msgr2=0x7f19780ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 --2- 172.21.15.133:0/3081716953 >> v2:172.21.15.162:3300/0 conn(0x7f19780ce330 0x7f19780ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197806d4f0 con 0x7f197815c9a0 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1968001680 con 0x7f197815c9a0 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 >> v1:172.21.15.133:6789/0 conn(0x7f197815c9a0 legacy=0x7f19780c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19600039c0 0x7f1960003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1960004810 0x7f1960004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19780b1f70 0x7f19600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780cece0 con 0x7f19600039c0 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780c64b0 con 0x7f1960004810 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196000b9d0 con 0x7f19780b1f70 2024-04-25T10:17:11.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1976ffd640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19600039c0 0x7f1960003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f19767fc640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1960004810 0x7f1960004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f19777fe640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19780b1f70 0x7f19600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1976ffd640 1 -- 172.21.15.133:0/3081716953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19780b1f70 msgr2=0x7f19600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1976ffd640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19780b1f70 0x7f19600051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1976ffd640 1 -- 172.21.15.133:0/3081716953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1960004810 msgr2=0x7f1960004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1976ffd640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1960004810 0x7f1960004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1976ffd640 1 -- 172.21.15.133:0/3081716953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960003530 con 0x7f19600039c0 2024-04-25T10:17:11.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.172+0000 7f1976ffd640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19600039c0 0x7f1960003de0 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f196800c210 tx=0x7f196801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9564fd7fde505e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968007d60 con 0x7f19600039c0 2024-04-25T10:17:11.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19680a9020 con 0x7f19600039c0 2024-04-25T10:17:11.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f1975ffb640 1 -- 172.21.15.133:0/3081716953 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968005b20 con 0x7f19600039c0 2024-04-25T10:17:11.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 -- 172.21.15.133:0/3081716953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19600039c0 msgr2=0x7f1960003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 --2- 172.21.15.133:0/3081716953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19600039c0 0x7f1960003de0 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f196800c210 tx=0x7f196801cc70 comp rx=0 tx=0).stop 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19777fe640 1 -- 172.21.15.133:0/3081716953 reap_dead start 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 -- 172.21.15.133:0/3081716953 shutdown_connections 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 -- 172.21.15.133:0/3081716953 >> 172.21.15.133:0/3081716953 conn(0x7f1978067000 msgr2=0x7f197806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 -- 172.21.15.133:0/3081716953 shutdown_connections 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 -- 172.21.15.133:0/3081716953 wait complete. 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 Processor -- start 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 -- start start 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19600039c0 0x7f19781619d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f197815bbe0 0x7f19780ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19780cac20 0x7f19780c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197815d510 con 0x7f19780cac20 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197815d390 con 0x7f197815bbe0 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197815d690 con 0x7f19600039c0 2024-04-25T10:17:11.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19777fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19780cac20 0x7f19780c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19777fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19780cac20 0x7f19780c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54748/0 (socket says 172.21.15.133:54748) 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19777fe640 1 -- 172.21.15.133:0/1972775344 learned_addr learned my addr 172.21.15.133:0/1972775344 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f1976ffd640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19600039c0 0x7f19781619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19767fc640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f197815bbe0 0x7f19780ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19777fe640 1 -- 172.21.15.133:0/1972775344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19600039c0 msgr2=0x7f19781619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19777fe640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19600039c0 0x7f19781619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19777fe640 1 -- 172.21.15.133:0/1972775344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f197815bbe0 msgr2=0x7f19780ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19777fe640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f197815bbe0 0x7f19780ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19777fe640 1 -- 172.21.15.133:0/1972775344 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1978160930 con 0x7f19780cac20 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19767fc640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f197815bbe0 0x7f19780ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f19777fe640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19780cac20 0x7f19780c7c50 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f1970009fd0 tx=0x7f197000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f195f7fe640 1 -- 172.21.15.133:0/1972775344 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970016620 con 0x7f19780cac20 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f195f7fe640 1 -- 172.21.15.133:0/1972775344 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197001b070 con 0x7f19780cac20 2024-04-25T10:17:11.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 -- 172.21.15.133:0/1972775344 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1978160c10 con 0x7f19780cac20 2024-04-25T10:17:11.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f195f7fe640 1 -- 172.21.15.133:0/1972775344 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970016a20 con 0x7f19780cac20 2024-04-25T10:17:11.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.176+0000 7f197d3ec640 1 -- 172.21.15.133:0/1972775344 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19781611d0 con 0x7f19780cac20 2024-04-25T10:17:11.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.180+0000 7f197d3ec640 1 -- 172.21.15.133:0/1972775344 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1978161f10 con 0x7f19780cac20 2024-04-25T10:17:11.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.180+0000 7f195f7fe640 1 -- 172.21.15.133:0/1972775344 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1970016bc0 con 0x7f19780cac20 2024-04-25T10:17:11.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.180+0000 7f195f7fe640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1964066f60 0x7f1964069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.180+0000 7f1976ffd640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1964066f60 0x7f1964069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.180+0000 7f195f7fe640 1 -- 172.21.15.133:0/1972775344 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1970085db0 con 0x7f19780cac20 2024-04-25T10:17:11.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.180+0000 7f1976ffd640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1964066f60 0x7f1964069420 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f19680aa000 tx=0x7f1968000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.184+0000 7f195f7fe640 1 -- 172.21.15.133:0/1972775344 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1970056c40 con 0x7f19780cac20 2024-04-25T10:17:11.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.360+0000 7f197d3ec640 1 -- 172.21.15.133:0/1972775344 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1978068ef0 con 0x7f19780cac20 2024-04-25T10:17:11.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f195f7fe640 1 -- 172.21.15.133:0/1972775344 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4962 (secure 0 0 0) 0x7f1970056540 con 0x7f19780cac20 2024-04-25T10:17:11.365 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:11.365 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":112,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1718839127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1718839127},{"type":"v1","addr":"172.21.15.162:6833","nonce":1718839127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/368827103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":368827103},{"type":"v1","addr":"172.21.15.133:6835","nonce":368827103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1274732295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1274732295},{"type":"v1","addr":"172.21.15.133:6837","nonce":1274732295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:05.607319+0000","modified":"2024-04-25T10:17:08.648334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11502":{"gid":11502,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1744425810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1744425810},{"type":"v1","addr":"172.21.15.162:6835","nonce":1744425810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:17:11.366 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 112 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 -- 172.21.15.133:0/1972775344 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1964066f60 msgr2=0x7f1964069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1964066f60 0x7f1964069420 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f19680aa000 tx=0x7f1968000ff0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 -- 172.21.15.133:0/1972775344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19780cac20 msgr2=0x7f19780c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19780cac20 0x7f19780c7c50 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f1970009fd0 tx=0x7f197000f520 comp rx=0 tx=0).stop 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 -- 172.21.15.133:0/1972775344 shutdown_connections 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1964066f60 0x7f1964069420 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19780cac20 0x7f19780c7c50 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f197815bbe0 0x7f19780ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 --2- 172.21.15.133:0/1972775344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f19600039c0 0x7f19781619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 -- 172.21.15.133:0/1972775344 >> 172.21.15.133:0/1972775344 conn(0x7f1978067000 msgr2=0x7f19780b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 -- 172.21.15.133:0/1972775344 shutdown_connections 2024-04-25T10:17:11.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.364+0000 7f197d3ec640 1 -- 172.21.15.133:0/1972775344 wait complete. 2024-04-25T10:17:11.515 INFO:tasks.ceph.mds.b.smithi162.stderr:2024-04-25T10:17:11.515+0000 7f99dcb54640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:17:11.786 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:05.607319+0000', 'modified': '2024-04-25T10:17:08.648334+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11502': {'gid': 11502, 'name': 'b', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.162:6835/1744425810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6834', 'nonce': 1744425810}, {'type': 'v1', 'addr': '172.21.15.162:6835', 'nonce': 1744425810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:17:11.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:17:11.787 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:17:11.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac516de640 1 Processor -- start 2024-04-25T10:17:11.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac516de640 1 -- start start 2024-04-25T10:17:11.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac516de640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fac4c06e8b0 0x7fac4c15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac516de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c0b1e50 0x7fac4c146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac516de640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c0b1090 con 0x7fac4c0aebc0 2024-04-25T10:17:11.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac516de640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c147490 con 0x7fac4c06e8b0 2024-04-25T10:17:11.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac516de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0b19c0 con 0x7fac4c0b1e50 2024-04-25T10:17:11.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac4a7fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c0b1e50 0x7fac4c146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac4affd640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fac4c06e8b0 0x7fac4c15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac4a7fc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c0b1e50 0x7fac4c146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:36786/0 (socket says 172.21.15.133:36786) 2024-04-25T10:17:11.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac4a7fc640 1 -- 172.21.15.133:0/2347593943 learned_addr learned my addr 172.21.15.133:0/2347593943 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:11.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac4a7fc640 1 -- 172.21.15.133:0/2347593943 >> v2:172.21.15.162:3300/0 conn(0x7fac4c06e8b0 msgr2=0x7fac4c15eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac4a7fc640 1 --2- 172.21.15.133:0/2347593943 >> v2:172.21.15.162:3300/0 conn(0x7fac4c06e8b0 0x7fac4c15eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac4a7fc640 1 -- 172.21.15.133:0/2347593943 >> v1:172.21.15.162:6789/0 conn(0x7fac4c0aebc0 legacy=0x7fac4c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.920+0000 7fac4a7fc640 1 -- 172.21.15.133:0/2347593943 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0b1370 con 0x7fac4c0b1e50 2024-04-25T10:17:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac4a7fc640 1 --2- 172.21.15.133:0/2347593943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c0b1e50 0x7fac4c146ae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac44009e80 tx=0x7fac4401c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1eccd313b76ba92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac49ffb640 1 -- 172.21.15.133:0/2347593943 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440a56c0 con 0x7fac4c0b1e50 2024-04-25T10:17:11.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac49ffb640 1 -- 172.21.15.133:0/2347593943 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac440a5860 con 0x7fac4c0b1e50 2024-04-25T10:17:11.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac49ffb640 1 -- 172.21.15.133:0/2347593943 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440a5c30 con 0x7fac4c0b1e50 2024-04-25T10:17:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac516de640 1 -- 172.21.15.133:0/2347593943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c0b1e50 msgr2=0x7fac4c146ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac516de640 1 --2- 172.21.15.133:0/2347593943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c0b1e50 0x7fac4c146ae0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fac44009e80 tx=0x7fac4401c980 comp rx=0 tx=0).stop 2024-04-25T10:17:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac516de640 1 -- 172.21.15.133:0/2347593943 shutdown_connections 2024-04-25T10:17:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac516de640 1 --2- 172.21.15.133:0/2347593943 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c0b1e50 0x7fac4c146ae0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac516de640 1 --2- 172.21.15.133:0/2347593943 >> v2:172.21.15.162:3300/0 conn(0x7fac4c06e8b0 0x7fac4c15eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac516de640 1 -- 172.21.15.133:0/2347593943 >> 172.21.15.133:0/2347593943 conn(0x7fac4c066f70 msgr2=0x7fac4c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac516de640 1 -- 172.21.15.133:0/2347593943 shutdown_connections 2024-04-25T10:17:11.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac516de640 1 -- 172.21.15.133:0/2347593943 wait complete. 2024-04-25T10:17:11.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac516de640 1 Processor -- start 2024-04-25T10:17:11.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.924+0000 7fac516de640 1 -- start start 2024-04-25T10:17:11.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac516de640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c06e8b0 0x7fac4c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac516de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac4c0aebc0 0x7fac4c140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac516de640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c0b1e50 0x7fac4c144b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac516de640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0af3e0 con 0x7fac4c0aebc0 2024-04-25T10:17:11.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac516de640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c147020 con 0x7fac4c0b1e50 2024-04-25T10:17:11.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac516de640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c13ce00 con 0x7fac4c06e8b0 2024-04-25T10:17:11.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac4c0aebc0 0x7fac4c140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac4c0aebc0 0x7fac4c140850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54758/0 (socket says 172.21.15.133:54758) 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4a7fc640 1 -- 172.21.15.133:0/2946272243 learned_addr learned my addr 172.21.15.133:0/2946272243 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4affd640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c06e8b0 0x7fac4c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c0b1e50 0x7fac4c144b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4a7fc640 1 -- 172.21.15.133:0/2946272243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c06e8b0 msgr2=0x7fac4c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4a7fc640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c06e8b0 0x7fac4c15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4a7fc640 1 -- 172.21.15.133:0/2946272243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c0b1e50 msgr2=0x7fac4c144b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4a7fc640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c0b1e50 0x7fac4c144b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4a7fc640 1 -- 172.21.15.133:0/2946272243 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c143dd0 con 0x7fac4c0aebc0 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c0b1e50 0x7fac4c144b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4a7fc640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac4c0aebc0 0x7fac4c140850 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fac4401a5b0 tx=0x7fac44019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac437fe640 1 -- 172.21.15.133:0/2946272243 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440bc020 con 0x7fac4c0aebc0 2024-04-25T10:17:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac437fe640 1 -- 172.21.15.133:0/2946272243 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac440b8670 con 0x7fac4c0aebc0 2024-04-25T10:17:11.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac437fe640 1 -- 172.21.15.133:0/2946272243 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440b8a20 con 0x7fac4c0aebc0 2024-04-25T10:17:11.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac516de640 1 -- 172.21.15.133:0/2946272243 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c1440b0 con 0x7fac4c0aebc0 2024-04-25T10:17:11.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac516de640 1 -- 172.21.15.133:0/2946272243 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c1450b0 con 0x7fac4c0aebc0 2024-04-25T10:17:11.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac437fe640 1 -- 172.21.15.133:0/2946272243 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac440bf020 con 0x7fac4c0aebc0 2024-04-25T10:17:11.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac437fe640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac2c066f60 0x7fac2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac437fe640 1 -- 172.21.15.133:0/2946272243 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac44120080 con 0x7fac4c0aebc0 2024-04-25T10:17:11.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.928+0000 7fac4affd640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac2c066f60 0x7fac2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4affd640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac2c066f60 0x7fac2c069420 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fac4c13f660 tx=0x7fac34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 Processor -- start 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 -- start start 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d3043dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d3043e240 0x559d303f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d303f3f30 0x559d303f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d302db5f0 con 0x559d303f3f30 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d30242da0 con 0x559d3043e240 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d301cb150 con 0x559d3043d920 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d303f3f30 0x559d303f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d303f3f30 0x559d303f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54766/0 (socket says 172.21.15.133:54766) 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4b7fe640 1 -- 172.21.15.133:0/281396029 learned_addr learned my addr 172.21.15.133:0/281396029 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4a7fc640 1 --2- 172.21.15.133:0/281396029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d3043e240 0x559d303f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4affd640 1 --2- 172.21.15.133:0/281396029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d3043dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4b7fe640 1 -- 172.21.15.133:0/281396029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 msgr2=0x559d3043dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/281396029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d3043dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4b7fe640 1 -- 172.21.15.133:0/281396029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d3043e240 msgr2=0x559d303f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/281396029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d3043e240 0x559d303f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4b7fe640 1 -- 172.21.15.133:0/281396029 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d301e4ca0 con 0x559d303f3f30 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/281396029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d303f3f30 0x559d303f4310 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0016f0 tx=0x7fac3c001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dcc86fc463a4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac40ff9640 1 -- 172.21.15.133:0/281396029 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c018070 con 0x559d303f3f30 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac40ff9640 1 -- 172.21.15.133:0/281396029 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c0067e0 con 0x559d303f3f30 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac40ff9640 1 -- 172.21.15.133:0/281396029 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c006bd0 con 0x559d303f3f30 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 -- 172.21.15.133:0/281396029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d303f3f30 msgr2=0x559d303f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 --2- 172.21.15.133:0/281396029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d303f3f30 0x559d303f4310 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0016f0 tx=0x7fac3c001bb0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 -- 172.21.15.133:0/281396029 shutdown_connections 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 --2- 172.21.15.133:0/281396029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d303f3f30 0x559d303f4310 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 --2- 172.21.15.133:0/281396029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d3043e240 0x559d303f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 --2- 172.21.15.133:0/281396029 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d3043dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 -- 172.21.15.133:0/281396029 >> 172.21.15.133:0/281396029 conn(0x559d303fa2a0 msgr2=0x559d303f22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 -- 172.21.15.133:0/281396029 shutdown_connections 2024-04-25T10:17:11.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 -- 172.21.15.133:0/281396029 wait complete. 2024-04-25T10:17:11.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 -- start start 2024-04-25T10:17:11.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.932+0000 7fac5484b000 1 Processor -- start 2024-04-25T10:17:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d303f3f30 0x559d30444dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043d920 0x559d30445310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d3043e240 0x559d303f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d301fd990 con 0x559d3043d920 2024-04-25T10:17:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3023ee40 con 0x559d3043e240 2024-04-25T10:17:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d301ce900 con 0x559d303f3f30 2024-04-25T10:17:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 10 client.0 _ll_register_callbacks cb 0x559d302f61c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043d920 0x559d30445310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043d920 0x559d30445310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54782/0 (socket says 172.21.15.133:54782) 2024-04-25T10:17:11.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4a7fc640 1 -- 172.21.15.133:0/861876653 learned_addr learned my addr 172.21.15.133:0/861876653 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4affd640 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d303f3f30 0x559d30444dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d3043e240 0x559d303f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4a7fc640 1 -- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d303f3f30 msgr2=0x559d30444dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4a7fc640 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d303f3f30 0x559d30444dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4a7fc640 1 -- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d3043e240 msgr2=0x559d303f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4a7fc640 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d3043e240 0x559d303f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4a7fc640 1 -- 172.21.15.133:0/861876653 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d301fb630 con 0x559d3043d920 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4a7fc640 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043d920 0x559d30445310 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fac4401b030 tx=0x7fac440a5860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4174ca7faf631118 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac29ffb640 1 -- 172.21.15.133:0/861876653 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440f0e10 con 0x559d3043d920 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac29ffb640 1 -- 172.21.15.133:0/861876653 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac440f8c30 con 0x559d3043d920 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 1 -- 172.21.15.133:0/861876653 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac3c0067e0 con 0x559d3043d920 2024-04-25T10:17:11.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac29ffb640 1 -- 172.21.15.133:0/861876653 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac440ace30 con 0x559d3043d920 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 10 client.11628 fetch_fsmap learned FSMap version 112 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 1 -- 172.21.15.133:0/861876653 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x559d301cb3a0 con 0x559d3043d920 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac29ffb640 1 -- 172.21.15.133:0/861876653 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c0067e0 con 0x559d3043d920 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac29ffb640 1 -- 172.21.15.133:0/861876653 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fac440f1ce0 con 0x559d3043d920 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 10 client.11628 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 10 client.11628 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 20 client.11628 populate_metadata read hostname 'smithi133' 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x559d3043d540 0x559d303f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 4 client.11628 mds_command: new command op to 11502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 1 -- 172.21.15.133:0/861876653 --> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d301dea50 con 0x559d3043d540 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4affd640 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x559d3043d540 0x559d303f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac4affd640 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x559d3043d540 0x559d303f84b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dee8246b39d6479 server_cookie=acfc92a5c00fd2f8 in_seq=0 out_seq=0 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac29ffb640 10 client.11628 ms_handle_connect on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:11.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac29ffb640 1 -- 172.21.15.133:0/861876653 <== mds.0 v2:172.21.15.162:6834/1744425810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d301dea50 con 0x559d3043d540 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac29ffb640 10 client.11628 handle_command_reply: tid=0 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.936+0000 7fac5484b000 1 client.11628 shutdown 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac5484b000 1 -- 172.21.15.133:0/861876653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043d920 msgr2=0x559d30445310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac5484b000 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043d920 0x559d30445310 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fac4401b030 tx=0x7fac440a5860 comp rx=0 tx=0).stop 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac5484b000 1 -- 172.21.15.133:0/861876653 shutdown_connections 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac5484b000 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x559d3043d540 0x559d303f84b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac5484b000 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d3043e240 0x559d303f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac29ffb640 0 client.11628 ms_handle_reset on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac5484b000 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043d920 0x559d30445310 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac5484b000 1 --2- 172.21.15.133:0/861876653 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d303f3f30 0x559d30444dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac5484b000 1 -- 172.21.15.133:0/861876653 >> 172.21.15.133:0/861876653 conn(0x559d303fa2a0 msgr2=0x559d303f29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac5484b000 1 -- 172.21.15.133:0/861876653 shutdown_connections 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac5484b000 1 -- 172.21.15.133:0/861876653 wait complete. 2024-04-25T10:17:11.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.940+0000 7fac5484b000 20 client.11628 trim_cache size 0 max 16384 2024-04-25T10:17:11.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.952+0000 7fac5484b000 1 Processor -- start 2024-04-25T10:17:11.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.952+0000 7fac5484b000 1 -- start start 2024-04-25T10:17:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d303f3f30 0x559d3042a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d303f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d303f7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c2029d0 con 0x559d3043e240 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c13c380 con 0x559d303f3f30 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c13c980 con 0x559d3043d920 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d303f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d303f7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54784/0 (socket says 172.21.15.133:54784) 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 -- 172.21.15.133:0/681968271 learned_addr learned my addr 172.21.15.133:0/681968271 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 -- 172.21.15.133:0/681968271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 msgr2=0x559d303f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/681968271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d303f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 -- 172.21.15.133:0/681968271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d303f3f30 msgr2=0x559d3042a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/681968271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d303f3f30 0x559d3042a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 -- 172.21.15.133:0/681968271 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d302c4600 con 0x559d3043e240 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/681968271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d303f7800 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x559d301e6200 tx=0x7fac3c00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3980f3339daba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac2bfff640 1 -- 172.21.15.133:0/681968271 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c018040 con 0x559d3043e240 2024-04-25T10:17:11.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac2bfff640 1 -- 172.21.15.133:0/681968271 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c019040 con 0x559d3043e240 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac2bfff640 1 -- 172.21.15.133:0/681968271 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c0067e0 con 0x559d3043e240 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- 172.21.15.133:0/681968271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 msgr2=0x559d303f7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 --2- 172.21.15.133:0/681968271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d303f7800 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x559d301e6200 tx=0x7fac3c00b000 comp rx=0 tx=0).stop 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- 172.21.15.133:0/681968271 shutdown_connections 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 --2- 172.21.15.133:0/681968271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d303f7800 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 --2- 172.21.15.133:0/681968271 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d303f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 --2- 172.21.15.133:0/681968271 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d303f3f30 0x559d3042a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- 172.21.15.133:0/681968271 >> 172.21.15.133:0/681968271 conn(0x559d303fa2a0 msgr2=0x559d303f6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- 172.21.15.133:0/681968271 shutdown_connections 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- 172.21.15.133:0/681968271 wait complete. 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- start start 2024-04-25T10:17:11.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 Processor -- start 2024-04-25T10:17:11.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d303f3f30 0x559d3044a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d30461d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d30462240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d302473d0 con 0x559d3043e240 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d302fed50 con 0x559d303f3f30 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d301dd980 con 0x559d3043d920 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac5484b000 10 client.0 _ll_register_callbacks cb 0x7fac3c007e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d30462240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d30462240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54796/0 (socket says 172.21.15.133:54796) 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 -- 172.21.15.133:0/3869045929 learned_addr learned my addr 172.21.15.133:0/3869045929 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4affd640 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d303f3f30 0x559d3044a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4a7fc640 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d30461d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 -- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 msgr2=0x559d30461d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d30461d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 -- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d303f3f30 msgr2=0x559d3044a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d303f3f30 0x559d3044a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 -- 172.21.15.133:0/3869045929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d301d79b0 con 0x559d3043e240 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4a7fc640 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d30461d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4affd640 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d303f3f30 0x559d3044a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac4b7fe640 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d30462240 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0016f0 tx=0x7fac3c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265da08b8038757a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.956+0000 7fac29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac29ffb640 1 -- 172.21.15.133:0/3869045929 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c006b30 con 0x559d3043e240 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac29ffb640 1 -- 172.21.15.133:0/3869045929 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac3c006cd0 con 0x559d3043e240 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 -- 172.21.15.133:0/3869045929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac3c019040 con 0x559d3043e240 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac29ffb640 1 -- 172.21.15.133:0/3869045929 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac3c034050 con 0x559d3043e240 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 10 client.11634 fetch_fsmap learned FSMap version 112 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 -- 172.21.15.133:0/3869045929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x559d301ea0b0 con 0x559d3043e240 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac29ffb640 1 -- 172.21.15.133:0/3869045929 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c019040 con 0x559d3043e240 2024-04-25T10:17:11.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac29ffb640 1 -- 172.21.15.133:0/3869045929 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fac3c037020 con 0x559d3043e240 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 10 client.11634 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 10 client.11634 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 20 client.11634 populate_metadata read hostname 'smithi133' 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x559d30462780 0x559d304a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 4 client.11634 mds_command: new command op to 11502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 -- 172.21.15.133:0/3869045929 --> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559d302502e0 con 0x559d30462780 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac4affd640 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x559d30462780 0x559d304a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac4affd640 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x559d30462780 0x559d304a6bd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8448939d60c2990 server_cookie=be825c5a46f3d244 in_seq=0 out_seq=0 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac29ffb640 10 client.11634 ms_handle_connect on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac29ffb640 1 -- 172.21.15.133:0/3869045929 <== mds.0 v2:172.21.15.162:6834/1744425810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x559d302502e0 con 0x559d30462780 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac29ffb640 10 client.11634 handle_command_reply: tid=0 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 client.11634 shutdown 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 -- 172.21.15.133:0/3869045929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 msgr2=0x559d30462240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d30462240 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0016f0 tx=0x7fac3c000bd0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 -- 172.21.15.133:0/3869045929 shutdown_connections 2024-04-25T10:17:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x559d30462780 0x559d304a6bd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d3043e240 0x559d30462240 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559d3043d920 0x559d30461d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 --2- 172.21.15.133:0/3869045929 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559d303f3f30 0x559d3044a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac29ffb640 0 client.11634 ms_handle_reset on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 -- 172.21.15.133:0/3869045929 >> 172.21.15.133:0/3869045929 conn(0x559d303fa2a0 msgr2=0x559d303f72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 -- 172.21.15.133:0/3869045929 shutdown_connections 2024-04-25T10:17:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 1 -- 172.21.15.133:0/3869045929 wait complete. 2024-04-25T10:17:11.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac5484b000 20 client.11634 trim_cache size 0 max 16384 2024-04-25T10:17:11.968 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:17:11.968 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:17:11.969 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:17:11.969 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 11502, 2024-04-25T10:17:11.969 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:17:11.969 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:17:11.969 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:17:11.969 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 5.6741644579999999, 2024-04-25T10:17:11.969 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 112, 2024-04-25T10:17:11.969 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 183, 2024-04-25T10:17:11.969 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 183, 2024-04-25T10:17:11.969 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 10.790007072 2024-04-25T10:17:11.969 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:17:11.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 -- 172.21.15.133:0/2946272243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac2c066f60 msgr2=0x7fac2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac2c066f60 0x7fac2c069420 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fac4c13f660 tx=0x7fac34008040 comp rx=0 tx=0).stop 2024-04-25T10:17:11.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 -- 172.21.15.133:0/2946272243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac4c0aebc0 msgr2=0x7fac4c140850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:11.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac4c0aebc0 0x7fac4c140850 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fac4401a5b0 tx=0x7fac44019720 comp rx=0 tx=0).stop 2024-04-25T10:17:11.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 -- 172.21.15.133:0/2946272243 shutdown_connections 2024-04-25T10:17:11.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac4c0b1e50 0x7fac4c144b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fac2c066f60 0x7fac2c069420 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac4c0aebc0 0x7fac4c140850 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 --2- 172.21.15.133:0/2946272243 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fac4c06e8b0 0x7fac4c15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:11.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 -- 172.21.15.133:0/2946272243 >> 172.21.15.133:0/2946272243 conn(0x7fac4c066f70 msgr2=0x7fac4c1400a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:11.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 -- 172.21.15.133:0/2946272243 shutdown_connections 2024-04-25T10:17:11.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:11.960+0000 7fac516de640 1 -- 172.21.15.133:0/2946272243 wait complete. 2024-04-25T10:17:11.979 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:17:12.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.084+0000 7f7d2d250640 1 Processor -- start 2024-04-25T10:17:12.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.084+0000 7f7d2d250640 1 -- start start 2024-04-25T10:17:12.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2d250640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7d2815b9b0 0x7f7d280cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2d250640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d280d0380 0x7f7d280c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2d250640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2815bd90 con 0x7f7d280cf7c0 2024-04-25T10:17:12.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2d250640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d280c5610 con 0x7f7d2815b9b0 2024-04-25T10:17:12.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2d250640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2809ad10 con 0x7f7d280d0380 2024-04-25T10:17:12.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d1ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7d280cf7c0 0x7f7d280cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55588/0 (socket says 172.21.15.133:55588) 2024-04-25T10:17:12.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d1ffff640 1 -- 172.21.15.133:0/4032238176 learned_addr learned my addr 172.21.15.133:0/4032238176 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:12.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2759e640 1 --2- 172.21.15.133:0/4032238176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d280d0380 0x7f7d280c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425295469 0 0) 0x7f7d2815bd90 con 0x7f7d280cf7c0 2024-04-25T10:17:12.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d26d9d640 1 --2- 172.21.15.133:0/4032238176 >> v2:172.21.15.162:3300/0 conn(0x7f7d2815b9b0 0x7f7d280cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d14003530 con 0x7f7d280cf7c0 2024-04-25T10:17:12.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 704920902 0 0) 0x7f7d14003530 con 0x7f7d280cf7c0 2024-04-25T10:17:12.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d280d0380 msgr2=0x7f7d280c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 --2- 172.21.15.133:0/4032238176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d280d0380 0x7f7d280c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 >> v2:172.21.15.162:3300/0 conn(0x7f7d2815b9b0 msgr2=0x7f7d280cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 --2- 172.21.15.133:0/4032238176 >> v2:172.21.15.162:3300/0 conn(0x7f7d2815b9b0 0x7f7d280cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2809a750 con 0x7f7d280cf7c0 2024-04-25T10:17:12.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7d180016a0 con 0x7f7d280cf7c0 2024-04-25T10:17:12.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 >> v1:172.21.15.133:6789/0 conn(0x7f7d280cf7c0 legacy=0x7f7d280cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 --2- 172.21.15.133:0/4032238176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d280d0380 0x7f7d140039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 --2- 172.21.15.133:0/4032238176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d14003f40 0x7f7d14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 --2- 172.21.15.133:0/4032238176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d14004d50 0x7f7d140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280c5610 con 0x7f7d14004d50 2024-04-25T10:17:12.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2809ad10 con 0x7f7d14003f40 2024-04-25T10:17:12.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1400ba20 con 0x7f7d280d0380 2024-04-25T10:17:12.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d26d9d640 1 --2- 172.21.15.133:0/4032238176 >> v2:172.21.15.162:3300/0 conn(0x7f7d2815b9b0 0x7f7d280cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:12.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2759e640 1 --2- 172.21.15.133:0/4032238176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d14004d50 0x7f7d140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2759e640 1 -- 172.21.15.133:0/4032238176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d280d0380 msgr2=0x7f7d140039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2759e640 1 --2- 172.21.15.133:0/4032238176 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d280d0380 0x7f7d140039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d1ffff640 1 --2- 172.21.15.133:0/4032238176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d14003f40 0x7f7d14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2759e640 1 -- 172.21.15.133:0/4032238176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d14003f40 msgr2=0x7f7d14004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:12.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2759e640 1 --2- 172.21.15.133:0/4032238176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d14003f40 0x7f7d14004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2759e640 1 -- 172.21.15.133:0/4032238176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d14003530 con 0x7f7d14004d50 2024-04-25T10:17:12.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.088+0000 7f7d2759e640 1 --2- 172.21.15.133:0/4032238176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d14004d50 0x7f7d140051d0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009fc0 tx=0x7f7d2001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e34982fa519cde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d200b2070 con 0x7f7d14004d50 2024-04-25T10:17:12.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d200a5cd0 con 0x7f7d14004d50 2024-04-25T10:17:12.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2659c640 1 -- 172.21.15.133:0/4032238176 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20009c10 con 0x7f7d14004d50 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 -- 172.21.15.133:0/4032238176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d14004d50 msgr2=0x7f7d140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 --2- 172.21.15.133:0/4032238176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d14004d50 0x7f7d140051d0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009fc0 tx=0x7f7d2001ccc0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2759e640 1 -- 172.21.15.133:0/4032238176 reap_dead start 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 -- 172.21.15.133:0/4032238176 shutdown_connections 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 -- 172.21.15.133:0/4032238176 >> 172.21.15.133:0/4032238176 conn(0x7f7d280b31a0 msgr2=0x7f7d280b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 -- 172.21.15.133:0/4032238176 shutdown_connections 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 -- 172.21.15.133:0/4032238176 wait complete. 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 Processor -- start 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 -- start start 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d14004d50 0x7f7d2806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d280ce790 0x7f7d280c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d28157d30 0x7f7d280c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2809a3e0 con 0x7f7d280ce790 2024-04-25T10:17:12.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280c52a0 con 0x7f7d14004d50 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280c5aa0 con 0x7f7d28157d30 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d280ce790 0x7f7d280c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d280ce790 0x7f7d280c7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54806/0 (socket says 172.21.15.133:54806) 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1ffff640 1 -- 172.21.15.133:0/1834715204 learned_addr learned my addr 172.21.15.133:0/1834715204 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1ffff640 1 -- 172.21.15.133:0/1834715204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d28157d30 msgr2=0x7f7d280c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1ffff640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d28157d30 0x7f7d280c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d26d9d640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d14004d50 0x7f7d2806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1ffff640 1 -- 172.21.15.133:0/1834715204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d14004d50 msgr2=0x7f7d2806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1ffff640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d14004d50 0x7f7d2806d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1ffff640 1 -- 172.21.15.133:0/1834715204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d280cef00 con 0x7f7d280ce790 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1ffff640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d280ce790 0x7f7d280c7470 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800d890 tx=0x7f7d1800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1f7fe640 1 -- 172.21.15.133:0/1834715204 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18019070 con 0x7f7d280ce790 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1f7fe640 1 -- 172.21.15.133:0/1834715204 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d180074e0 con 0x7f7d280ce790 2024-04-25T10:17:12.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 -- 172.21.15.133:0/1834715204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d280cec20 con 0x7f7d280ce790 2024-04-25T10:17:12.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d1f7fe640 1 -- 172.21.15.133:0/1834715204 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18004a10 con 0x7f7d280ce790 2024-04-25T10:17:12.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.092+0000 7f7d2d250640 1 -- 172.21.15.133:0/1834715204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2806d130 con 0x7f7d280ce790 2024-04-25T10:17:12.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.096+0000 7f7d2d250640 1 -- 172.21.15.133:0/1834715204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d28067c70 con 0x7f7d280ce790 2024-04-25T10:17:12.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.096+0000 7f7d1f7fe640 1 -- 172.21.15.133:0/1834715204 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d1801c070 con 0x7f7d280ce790 2024-04-25T10:17:12.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.096+0000 7f7d1f7fe640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d08066f60 0x7f7d08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:12.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.096+0000 7f7d1f7fe640 1 -- 172.21.15.133:0/1834715204 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7d18084c90 con 0x7f7d280ce790 2024-04-25T10:17:12.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.096+0000 7f7d26d9d640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d08066f60 0x7f7d08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:12.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.096+0000 7f7d26d9d640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d08066f60 0x7f7d08069420 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7d28099b20 tx=0x7f7d10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:12.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.100+0000 7f7d1f7fe640 1 -- 172.21.15.133:0/1834715204 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d18055c60 con 0x7f7d280ce790 2024-04-25T10:17:12.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 -- 172.21.15.133:0/1834715204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7d280c7b80 con 0x7f7d280ce790 2024-04-25T10:17:12.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d1f7fe640 1 -- 172.21.15.133:0/1834715204 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7d18055560 con 0x7f7d280ce790 2024-04-25T10:17:12.290 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:17:12.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 -- 172.21.15.133:0/1834715204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d08066f60 msgr2=0x7f7d08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d08066f60 0x7f7d08069420 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7d28099b20 tx=0x7f7d10008040 comp rx=0 tx=0).stop 2024-04-25T10:17:12.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 -- 172.21.15.133:0/1834715204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d280ce790 msgr2=0x7f7d280c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:12.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d280ce790 0x7f7d280c7470 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800d890 tx=0x7f7d1800dd50 comp rx=0 tx=0).stop 2024-04-25T10:17:12.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 -- 172.21.15.133:0/1834715204 shutdown_connections 2024-04-25T10:17:12.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7d28157d30 0x7f7d280c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7d08066f60 0x7f7d08069420 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d280ce790 0x7f7d280c7470 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 --2- 172.21.15.133:0/1834715204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7d14004d50 0x7f7d2806d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:12.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 -- 172.21.15.133:0/1834715204 >> 172.21.15.133:0/1834715204 conn(0x7f7d280b31a0 msgr2=0x7f7d280b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:12.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 -- 172.21.15.133:0/1834715204 shutdown_connections 2024-04-25T10:17:12.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:12.288+0000 7f7d2d250640 1 -- 172.21.15.133:0/1834715204 wait complete. 2024-04-25T10:17:12.307 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:17:12.307 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:17:12.308 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:17:12.308 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:17:12.308 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:12.308 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:17:12.308 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:17:12.308 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:17:12.308 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:12.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:12.308 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:12.314 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41599sec preferred_lft 41599sec 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:17:12.315 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:12.316 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:17:12.316 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:12.316 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:17:12.316 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:12.316 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:17:12.316 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:12.316 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:17:12.316 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:12.316 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:17:12.316 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:12.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:12.316 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:17:12.366 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:17:12.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:12.368 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:17:12.426 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:17:12.426 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:17:12.426 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:17:12.426 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:17:12.426 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:17:12.426 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:17:12.426 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:17:12.426 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:17:12.427 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:17:12.427 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:17:12.427 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:17:12.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:12.427 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:17:12.427 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:17:12.427 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:17:12.427 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:17:12.427 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:17:12.427 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:17:12.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:12.498 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:17:12.545 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:17:12.545 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:17:12.546 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:12.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:12.546 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:17:12.600 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:17:12.601 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:17:12.601 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:12.649 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:12.649 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:17:12.649 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:12.697 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:17:12.754 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:17:12.802 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:17:12.802 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:17:12.802 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:17:12.802 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:17:12.802 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:17:12.802 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:17:12.802 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:17:12.802 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:17:12.802 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:17:12.803 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:12.804 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:12.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:12.805 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:17:12.853 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:17:12.853 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:17:12.898 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:17:12.913 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:17:12.961 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:17:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:17:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:17:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:17:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:17:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:17:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:17:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:17:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:17:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:17:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:17:12.962 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:17:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:17:12.964 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:17:12.964 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:17:12.964 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:17:12.964 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:17:12.964 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:12.964 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:12.964 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:12.964 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:12.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:12.964 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:17:12.976 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:17:12.972+0000 7fcf5455e580 -1 init, newargv = 0x5622848ffec0 newargc=15 2024-04-25T10:17:12.976 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[90976]: starting ceph client 2024-04-25T10:17:12.983 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[90976]: starting fuse 2024-04-25T10:17:14.082 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:17:14.098 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:17:14.146 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:17:14.146 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:17:14.146 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:17:14.146 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:17:14.146 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:17:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:17:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:17:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:17:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:17:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:17:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:17:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:17:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:17:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:17:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:17:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:17:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:14.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:14.150 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:17:14.197 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:17:14.198 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:17:14.198 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:17:14.198 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:17:14.198 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:17:14.198 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:17:14.198 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:17:14.198 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:14.198 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:17:14.198 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:17:14.198 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:17:14.198 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:17:14.199 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:17:14.316 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.90976.asok 2024-04-25T10:17:14.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:14.327 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.90976.asok 2024-04-25T10:17:14.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:14.328 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.90976.asok status 2024-04-25T10:17:14.481 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "90976", 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:17:14.482 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 11643, 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 11643 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 106219908 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:17:14.483 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:17:14.484 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 106219908 2024-04-25T10:17:14.484 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:14.484 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.11643 192.168.0.1:0/106219908", 2024-04-25T10:17:14.484 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/106219908", 2024-04-25T10:17:14.484 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:17:14.484 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 112, 2024-04-25T10:17:14.484 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 183, 2024-04-25T10:17:14.484 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:17:14.484 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:17:14.484 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:17:14.484 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:17:14.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:14.495 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:17:14.496 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:14.502 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:17:14.502 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:14.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:14.503 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:14.557 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:17:14.558 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:14.606 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-04-25T10:17:14.606 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:17:14.606 DEBUG:teuthology.orchestra.run.smithi133:> import uuid 2024-04-25T10:17:14.606 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:14.606 DEBUG:teuthology.orchestra.run.smithi133:> n = 1000 2024-04-25T10:17:14.606 DEBUG:teuthology.orchestra.run.smithi133:> create_hard_links = False 2024-04-25T10:17:14.606 DEBUG:teuthology.orchestra.run.smithi133:> if 0 > 0: 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> create_hard_links = True 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> path = "/home/ubuntu/cephtest/mnt.0/scrub-new-files/file" 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> dpath = os.path.dirname(path) 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> fnameprefix = os.path.basename(path) 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> try: 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> for i in range(n): 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> f.write(f"{i}") 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> f.flush() 2024-04-25T10:17:14.607 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(f.fileno()) 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> os.unlink(fpath) 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(dirfd) 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> if create_hard_links: 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> for j in range(0): 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(dirfd) 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> finally: 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> os.close(dirfd) 2024-04-25T10:17:14.608 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:17:18.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:18.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:18.477 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-25T10:17:18.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.648+0000 7fe3ecc2c640 1 Processor -- start 2024-04-25T10:17:18.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.648+0000 7fe3ecc2c640 1 -- start start 2024-04-25T10:17:18.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3ecc2c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe3e80b2150 0x7fe3e806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3ecc2c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e815ac50 con 0x7fe3e815b9b0 2024-04-25T10:17:18.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3ecc2c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e80cd180 con 0x7fe3e8143ea0 2024-04-25T10:17:18.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3ecc2c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e80ccf20 con 0x7fe3e80b2150 2024-04-25T10:17:18.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3e5d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe3e80b2150 0x7fe3e806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3e6d9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe3e815b9b0 0x7fe3e80b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46534/0 (socket says 172.21.15.133:46534) 2024-04-25T10:17:18.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/345902308 learned_addr learned my addr 172.21.15.133:0/345902308 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:18.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3e5d9b640 1 -- 172.21.15.133:0/345902308 >> v1:172.21.15.162:6789/0 conn(0x7fe3e8143ea0 legacy=0x7fe3e80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3e5d9b640 1 -- 172.21.15.133:0/345902308 >> v1:172.21.15.133:6789/0 conn(0x7fe3e815b9b0 legacy=0x7fe3e80b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3e5d9b640 1 -- 172.21.15.133:0/345902308 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e815aed0 con 0x7fe3e80b2150 2024-04-25T10:17:18.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3e559a640 1 -- 172.21.15.133:0/345902308 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142940113 0 0) 0x7fe3e00014a0 con 0x7fe3e815b9b0 2024-04-25T10:17:18.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3e5d9b640 1 --2- 172.21.15.133:0/345902308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e80b2150 0x7fe3e806e3b0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0097c0 tx=0x7fe3dc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5c98dd74fedd74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3e559a640 1 -- 172.21.15.133:0/345902308 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc0b0070 con 0x7fe3e80b2150 2024-04-25T10:17:18.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3e559a640 1 -- 172.21.15.133:0/345902308 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc0ab600 con 0x7fe3e80b2150 2024-04-25T10:17:18.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3e559a640 1 -- 172.21.15.133:0/345902308 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc0ab9d0 con 0x7fe3e80b2150 2024-04-25T10:17:18.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/345902308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e80b2150 msgr2=0x7fe3e806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.652+0000 7fe3ecc2c640 1 --2- 172.21.15.133:0/345902308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e80b2150 0x7fe3e806e3b0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0097c0 tx=0x7fe3dc01c3d0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/345902308 shutdown_connections 2024-04-25T10:17:18.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 --2- 172.21.15.133:0/345902308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e80b2150 0x7fe3e806e3b0 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/345902308 >> 172.21.15.133:0/345902308 conn(0x7fe3e8066f70 msgr2=0x7fe3e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/345902308 shutdown_connections 2024-04-25T10:17:18.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/345902308 wait complete. 2024-04-25T10:17:18.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 Processor -- start 2024-04-25T10:17:18.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 -- start start 2024-04-25T10:17:18.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3e80b2150 0x7fe3e80b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e8143ea0 0x7fe3e81399c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e815b9b0 0x7fe3e8164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e81685f0 con 0x7fe3e815b9b0 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e806ed00 con 0x7fe3e8143ea0 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8168770 con 0x7fe3e80b2150 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e815b9b0 0x7fe3e8164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e815b9b0 0x7fe3e8164bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56540/0 (socket says 172.21.15.133:56540) 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/824240933 learned_addr learned my addr 172.21.15.133:0/824240933 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e659c640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3e80b2150 0x7fe3e80b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e5d9b640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e8143ea0 0x7fe3e81399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/824240933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3e80b2150 msgr2=0x7fe3e80b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e6d9d640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3e80b2150 0x7fe3e80b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/824240933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e8143ea0 msgr2=0x7fe3e81399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e6d9d640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e8143ea0 0x7fe3e81399c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/824240933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e80b2860 con 0x7fe3e815b9b0 2024-04-25T10:17:18.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e5d9b640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e8143ea0 0x7fe3e81399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:18.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3e6d9d640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e815b9b0 0x7fe3e8164bd0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000c750 tx=0x7fe3e000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3d6ffd640 1 -- 172.21.15.133:0/824240933 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0018070 con 0x7fe3e815b9b0 2024-04-25T10:17:18.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3d6ffd640 1 -- 172.21.15.133:0/824240933 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e000a640 con 0x7fe3e815b9b0 2024-04-25T10:17:18.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3d6ffd640 1 -- 172.21.15.133:0/824240933 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000ce60 con 0x7fe3e815b9b0 2024-04-25T10:17:18.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/824240933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e80b2b00 con 0x7fe3e815b9b0 2024-04-25T10:17:18.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.656+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/824240933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e80b30c0 con 0x7fe3e815b9b0 2024-04-25T10:17:18.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3d6ffd640 1 -- 172.21.15.133:0/824240933 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3e001b070 con 0x7fe3e815b9b0 2024-04-25T10:17:18.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3d6ffd640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3c8066f60 0x7fe3c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e659c640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3c8066f60 0x7fe3c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3d6ffd640 1 -- 172.21.15.133:0/824240933 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe3e0084880 con 0x7fe3e815b9b0 2024-04-25T10:17:18.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e659c640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3c8066f60 0x7fe3c8069420 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8004e30 tx=0x7fe3d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3efd99000 1 Processor -- start 2024-04-25T10:17:18.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3efd99000 1 -- start start 2024-04-25T10:17:18.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659342adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342b2f0 0x5565933e0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x5565933e1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3efd99000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659322dff0 con 0x55659342a9d0 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3efd99000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565931b6150 con 0x5565933e10a0 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3efd99000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55659322a090 con 0x55659342b2f0 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659342adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659342adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56552/0 (socket says 172.21.15.133:56552) 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e659c640 1 -- 172.21.15.133:0/4061782792 learned_addr learned my addr 172.21.15.133:0/4061782792 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e6d9d640 1 --2- 172.21.15.133:0/4061782792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x5565933e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e5d9b640 1 --2- 172.21.15.133:0/4061782792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342b2f0 0x5565933e0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e659c640 1 -- 172.21.15.133:0/4061782792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342b2f0 msgr2=0x5565933e0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e659c640 1 --2- 172.21.15.133:0/4061782792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342b2f0 0x5565933e0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e659c640 1 -- 172.21.15.133:0/4061782792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 msgr2=0x5565933e1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e659c640 1 --2- 172.21.15.133:0/4061782792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x5565933e1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e659c640 1 -- 172.21.15.133:0/4061782792 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565933c40d0 con 0x55659342a9d0 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3e659c640 1 --2- 172.21.15.133:0/4061782792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659342adb0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80055d0 tx=0x7fe3d8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be48e1a04c8bfc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3c7fff640 1 -- 172.21.15.133:0/4061782792 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8025070 con 0x55659342a9d0 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3c7fff640 1 -- 172.21.15.133:0/4061782792 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8015880 con 0x55659342a9d0 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.660+0000 7fe3c7fff640 1 -- 172.21.15.133:0/4061782792 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8015c30 con 0x55659342a9d0 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- 172.21.15.133:0/4061782792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 msgr2=0x55659342adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 --2- 172.21.15.133:0/4061782792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659342adb0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80055d0 tx=0x7fe3d8011000 comp rx=0 tx=0).stop 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- 172.21.15.133:0/4061782792 shutdown_connections 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 --2- 172.21.15.133:0/4061782792 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x5565933e1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 --2- 172.21.15.133:0/4061782792 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342b2f0 0x5565933e0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 --2- 172.21.15.133:0/4061782792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659342adb0 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- 172.21.15.133:0/4061782792 >> 172.21.15.133:0/4061782792 conn(0x5565933dd020 msgr2=0x5565933df440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- 172.21.15.133:0/4061782792 shutdown_connections 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- 172.21.15.133:0/4061782792 wait complete. 2024-04-25T10:17:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- start start 2024-04-25T10:17:18.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 Processor -- start 2024-04-25T10:17:18.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659340f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342a9d0 0x55659340fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593410270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565931f2250 con 0x55659342b2f0 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565931e8690 con 0x5565933e10a0 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565931b6a40 con 0x55659342a9d0 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 10 client.0 _ll_register_callbacks cb 0x5565933cbe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593410270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593410270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56568/0 (socket says 172.21.15.133:56568) 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/409222450 learned_addr learned my addr 172.21.15.133:0/409222450 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e5d9b640 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342a9d0 0x55659340fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e659c640 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659340f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342a9d0 msgr2=0x55659340fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e6d9d640 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342a9d0 0x55659340fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 msgr2=0x55659340f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e6d9d640 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659340f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/409222450 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565932db1c0 con 0x55659342b2f0 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e6d9d640 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593410270 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0051450 tx=0x7fe3e0054ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efc437c0b7622e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3c57fa640 1 -- 172.21.15.133:0/409222450 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0054640 con 0x55659342b2f0 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3c57fa640 1 -- 172.21.15.133:0/409222450 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe3e005b0e0 con 0x55659342b2f0 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- 172.21.15.133:0/409222450 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3d8015880 con 0x55659342b2f0 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3c57fa640 1 -- 172.21.15.133:0/409222450 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e005d700 con 0x55659342b2f0 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 10 client.11658 fetch_fsmap learned FSMap version 112 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- 172.21.15.133:0/409222450 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x5565932d7170 con 0x55659342b2f0 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3c57fa640 1 -- 172.21.15.133:0/409222450 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8015880 con 0x55659342b2f0 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3c57fa640 1 -- 172.21.15.133:0/409222450 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe3e0050d40 con 0x55659342b2f0 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 10 client.11658 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T10:17:18.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 10 client.11658 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.b 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 20 client.11658 populate_metadata read hostname 'smithi133' 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55659343ac80 0x55659343d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 4 client.11658 mds_command: new command op to 11502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3efd99000 1 -- 172.21.15.133:0/409222450 --> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565931c9c60 con 0x55659343ac80 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.664+0000 7fe3e659c640 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55659343ac80 0x55659343d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3c57fa640 10 client.11658 ms_handle_connect on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3e659c640 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55659343ac80 0x55659343d070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b33776873574b server_cookie=2ae1ac4f60692cc4 in_seq=0 out_seq=0 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3c57fa640 1 -- 172.21.15.133:0/409222450 <== mds.0 v2:172.21.15.162:6834/1744425810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565931c9c60 con 0x55659343ac80 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3c57fa640 10 client.11658 handle_command_reply: tid=0 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 1 client.11658 shutdown 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 1 -- 172.21.15.133:0/409222450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 msgr2=0x556593410270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593410270 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0051450 tx=0x7fe3e0054ff0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 1 -- 172.21.15.133:0/409222450 shutdown_connections 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55659343ac80 0x55659343d070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593410270 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342a9d0 0x55659340fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3c57fa640 0 client.11658 ms_handle_reset on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:18.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 1 --2- 172.21.15.133:0/409222450 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659340f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 1 -- 172.21.15.133:0/409222450 >> 172.21.15.133:0/409222450 conn(0x5565933dd020 msgr2=0x5565933dfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 1 -- 172.21.15.133:0/409222450 shutdown_connections 2024-04-25T10:17:18.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 1 -- 172.21.15.133:0/409222450 wait complete. 2024-04-25T10:17:18.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.668+0000 7fe3efd99000 20 client.11658 trim_cache size 0 max 16384 2024-04-25T10:17:18.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 Processor -- start 2024-04-25T10:17:18.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 -- start start 2024-04-25T10:17:18.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659348fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342a9d0 0x556593490050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593490590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8203a30 con 0x55659342b2f0 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8202bf0 con 0x5565933e10a0 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8201ff0 con 0x55659342a9d0 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593490590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593490590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56578/0 (socket says 172.21.15.133:56578) 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/3062495418 learned_addr learned my addr 172.21.15.133:0/3062495418 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e5d9b640 1 --2- 172.21.15.133:0/3062495418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342a9d0 0x556593490050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e659c640 1 --2- 172.21.15.133:0/3062495418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659348fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/3062495418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342a9d0 msgr2=0x556593490050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e6d9d640 1 --2- 172.21.15.133:0/3062495418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342a9d0 0x556593490050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/3062495418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 msgr2=0x55659348fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e6d9d640 1 --2- 172.21.15.133:0/3062495418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659348fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e6d9d640 1 -- 172.21.15.133:0/3062495418 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556593274de0 con 0x55659342b2f0 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e659c640 1 --2- 172.21.15.133:0/3062495418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659348fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3e6d9d640 1 --2- 172.21.15.133:0/3062495418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593490590 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0054f80 tx=0x7fe3e0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2589356411bf17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3c77fe640 1 -- 172.21.15.133:0/3062495418 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0051290 con 0x55659342b2f0 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3c77fe640 1 -- 172.21.15.133:0/3062495418 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e0054440 con 0x55659342b2f0 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3c77fe640 1 -- 172.21.15.133:0/3062495418 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0053eb0 con 0x55659342b2f0 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 -- 172.21.15.133:0/3062495418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 msgr2=0x556593490590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 --2- 172.21.15.133:0/3062495418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593490590 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0054f80 tx=0x7fe3e0014000 comp rx=0 tx=0).stop 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 -- 172.21.15.133:0/3062495418 shutdown_connections 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 --2- 172.21.15.133:0/3062495418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342b2f0 0x556593490590 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 --2- 172.21.15.133:0/3062495418 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342a9d0 0x556593490050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 --2- 172.21.15.133:0/3062495418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659348fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 -- 172.21.15.133:0/3062495418 >> 172.21.15.133:0/3062495418 conn(0x5565933dd020 msgr2=0x5565934290d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 -- 172.21.15.133:0/3062495418 shutdown_connections 2024-04-25T10:17:18.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.684+0000 7fe3efd99000 1 -- 172.21.15.133:0/3062495418 wait complete. 2024-04-25T10:17:18.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 -- start start 2024-04-25T10:17:18.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 Processor -- start 2024-04-25T10:17:18.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659343d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659343dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342b2f0 0x55659343e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565931c8e00 con 0x55659342a9d0 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e82025f0 con 0x5565933e10a0 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55659322d300 con 0x55659342b2f0 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 10 client.0 _ll_register_callbacks cb 0x5565933ac040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659343dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e5d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659343dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56588/0 (socket says 172.21.15.133:56588) 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e5d9b640 1 -- 172.21.15.133:0/988333083 learned_addr learned my addr 172.21.15.133:0/988333083 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e6d9d640 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342b2f0 0x55659343e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e659c640 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659343d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e5d9b640 1 -- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342b2f0 msgr2=0x55659343e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e5d9b640 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342b2f0 0x55659343e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e5d9b640 1 -- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 msgr2=0x55659343d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e5d9b640 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659343d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e5d9b640 1 -- 172.21.15.133:0/988333083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565931cf8e0 con 0x55659342a9d0 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e5d9b640 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659343dcb0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc01a000 tx=0x7fe3dc01ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7b19e825d7d386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3c57fa640 1 -- 172.21.15.133:0/988333083 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc0b0040 con 0x55659342a9d0 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3c57fa640 1 -- 172.21.15.133:0/988333083 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe3dc0b5b10 con 0x55659342a9d0 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 -- 172.21.15.133:0/988333083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3e0054440 con 0x55659342a9d0 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3c57fa640 1 -- 172.21.15.133:0/988333083 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3dc0c8050 con 0x55659342a9d0 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 10 client.11664 fetch_fsmap learned FSMap version 112 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 -- 172.21.15.133:0/988333083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x5565932878f0 con 0x55659342a9d0 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3c57fa640 1 -- 172.21.15.133:0/988333083 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e0054440 con 0x55659342a9d0 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3c57fa640 1 -- 172.21.15.133:0/988333083 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe3dc0cb020 con 0x55659342a9d0 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 10 client.11664 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 10 client.11664 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.b 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 20 client.11664 populate_metadata read hostname 'smithi133' 2024-04-25T10:17:18.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x556593499760 0x55659349bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 4 client.11664 mds_command: new command op to 11502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 -- 172.21.15.133:0/988333083 --> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55659323b530 con 0x556593499760 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e659c640 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x556593499760 0x55659349bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3c57fa640 10 client.11664 ms_handle_connect on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3e659c640 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x556593499760 0x55659349bb50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c4c132a3bcdbf78 server_cookie=77037001a95128e7 in_seq=0 out_seq=0 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3c57fa640 1 -- 172.21.15.133:0/988333083 <== mds.0 v2:172.21.15.162:6834/1744425810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55659323b530 con 0x556593499760 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3c57fa640 10 client.11664 handle_command_reply: tid=0 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.688+0000 7fe3efd99000 1 client.11664 shutdown 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3efd99000 1 -- 172.21.15.133:0/988333083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 msgr2=0x55659343dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3efd99000 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659343dcb0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc01a000 tx=0x7fe3dc01ac10 comp rx=0 tx=0).stop 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3efd99000 1 -- 172.21.15.133:0/988333083 shutdown_connections 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3efd99000 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x556593499760 0x55659349bb50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3efd99000 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55659342b2f0 0x55659343e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3efd99000 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55659342a9d0 0x55659343dcb0 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3c57fa640 0 client.11664 ms_handle_reset on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:18.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3efd99000 1 --2- 172.21.15.133:0/988333083 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5565933e10a0 0x55659343d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3efd99000 1 -- 172.21.15.133:0/988333083 >> 172.21.15.133:0/988333083 conn(0x5565933dd020 msgr2=0x556593429960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3efd99000 1 -- 172.21.15.133:0/988333083 shutdown_connections 2024-04-25T10:17:18.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3efd99000 1 -- 172.21.15.133:0/988333083 wait complete. 2024-04-25T10:17:18.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3efd99000 20 client.11664 trim_cache size 0 max 16384 2024-04-25T10:17:18.698 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:17:18.698 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-04-25T10:17:18.698 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:17:18.698 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:17:18.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/824240933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3c8066f60 msgr2=0x7fe3c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3c8066f60 0x7fe3c8069420 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8004e30 tx=0x7fe3d8009040 comp rx=0 tx=0).stop 2024-04-25T10:17:18.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/824240933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e815b9b0 msgr2=0x7fe3e8164bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e815b9b0 0x7fe3e8164bd0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000c750 tx=0x7fe3e000cc10 comp rx=0 tx=0).stop 2024-04-25T10:17:18.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/824240933 shutdown_connections 2024-04-25T10:17:18.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe3c8066f60 0x7fe3c8069420 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e815b9b0 0x7fe3e8164bd0 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe3e8143ea0 0x7fe3e81399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 --2- 172.21.15.133:0/824240933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe3e80b2150 0x7fe3e80b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/824240933 >> 172.21.15.133:0/824240933 conn(0x7fe3e8066f70 msgr2=0x7fe3e815d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/824240933 shutdown_connections 2024-04-25T10:17:18.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.692+0000 7fe3ecc2c640 1 -- 172.21.15.133:0/824240933 wait complete. 2024-04-25T10:17:18.709 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:17:18.710 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:17:18.710 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 damage ls 2024-04-25T10:17:18.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.824+0000 7fdb38ddb640 1 Processor -- start 2024-04-25T10:17:18.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.824+0000 7fdb38ddb640 1 -- start start 2024-04-25T10:17:18.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.824+0000 7fdb38ddb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb3415bc90 0x7fdb340cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.824+0000 7fdb38ddb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdb340d01d0 0x7fdb340b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.824+0000 7fdb38ddb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb340cc6a0 con 0x7fdb3415b8b0 2024-04-25T10:17:18.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.824+0000 7fdb38ddb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb340d0640 con 0x7fdb340d01d0 2024-04-25T10:17:18.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.824+0000 7fdb38ddb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb340cc980 con 0x7fdb3415bc90 2024-04-25T10:17:18.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.824+0000 7fdb32d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fdb340d01d0 0x7fdb340b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.824+0000 7fdb3259c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fdb3415b8b0 0x7fdb340ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:57804/0 (socket says 172.21.15.133:57804) 2024-04-25T10:17:18.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.824+0000 7fdb2bfff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb3415bc90 0x7fdb340cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.824+0000 7fdb3259c640 1 -- 172.21.15.133:0/1005210111 learned_addr learned my addr 172.21.15.133:0/1005210111 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:18.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb32d9d640 1 -- 172.21.15.133:0/1005210111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb3415bc90 msgr2=0x7fdb340cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/1005210111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb3415bc90 0x7fdb340cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb32d9d640 1 -- 172.21.15.133:0/1005210111 >> v1:172.21.15.162:6789/0 conn(0x7fdb3415b8b0 legacy=0x7fdb340ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb32d9d640 1 -- 172.21.15.133:0/1005210111 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb340ccc60 con 0x7fdb340d01d0 2024-04-25T10:17:18.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/1005210111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb340d01d0 0x7fdb340b0aa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009d90 tx=0x7fdb2c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f5a27535252acc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb31d9b640 1 -- 172.21.15.133:0/1005210111 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0b2070 con 0x7fdb340d01d0 2024-04-25T10:17:18.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb31d9b640 1 -- 172.21.15.133:0/1005210111 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c0a5850 con 0x7fdb340d01d0 2024-04-25T10:17:18.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb31d9b640 1 -- 172.21.15.133:0/1005210111 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0a5c20 con 0x7fdb340d01d0 2024-04-25T10:17:18.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb38ddb640 1 -- 172.21.15.133:0/1005210111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb340d01d0 msgr2=0x7fdb340b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb38ddb640 1 --2- 172.21.15.133:0/1005210111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb340d01d0 0x7fdb340b0aa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009d90 tx=0x7fdb2c01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb38ddb640 1 -- 172.21.15.133:0/1005210111 shutdown_connections 2024-04-25T10:17:18.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb38ddb640 1 --2- 172.21.15.133:0/1005210111 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb3415bc90 0x7fdb340cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb38ddb640 1 --2- 172.21.15.133:0/1005210111 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb340d01d0 0x7fdb340b0aa0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb38ddb640 1 -- 172.21.15.133:0/1005210111 >> 172.21.15.133:0/1005210111 conn(0x7fdb3409a990 msgr2=0x7fdb340d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb38ddb640 1 -- 172.21.15.133:0/1005210111 shutdown_connections 2024-04-25T10:17:18.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.828+0000 7fdb38ddb640 1 -- 172.21.15.133:0/1005210111 wait complete. 2024-04-25T10:17:18.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb38ddb640 1 Processor -- start 2024-04-25T10:17:18.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb38ddb640 1 -- start start 2024-04-25T10:17:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb38ddb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb340d01d0 0x7fdb34155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb38ddb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb3415b8b0 0x7fdb34154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb38ddb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb3415bc90 0x7fdb34070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb38ddb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb340b17f0 con 0x7fdb340d01d0 2024-04-25T10:17:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb38ddb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb340cc440 con 0x7fdb3415b8b0 2024-04-25T10:17:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb38ddb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb340b1970 con 0x7fdb3415bc90 2024-04-25T10:17:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb3259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb340d01d0 0x7fdb34155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb3259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb340d01d0 0x7fdb34155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56600/0 (socket says 172.21.15.133:56600) 2024-04-25T10:17:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb3259c640 1 -- 172.21.15.133:0/611280095 learned_addr learned my addr 172.21.15.133:0/611280095 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb3415b8b0 0x7fdb34154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb3415bc90 0x7fdb34070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb3259c640 1 -- 172.21.15.133:0/611280095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb3415bc90 msgr2=0x7fdb34070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb3259c640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb3415bc90 0x7fdb34070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb3259c640 1 -- 172.21.15.133:0/611280095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb3415b8b0 msgr2=0x7fdb34154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb3259c640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb3415b8b0 0x7fdb34154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb3259c640 1 -- 172.21.15.133:0/611280095 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb340b4420 con 0x7fdb340d01d0 2024-04-25T10:17:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb3259c640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb340d01d0 0x7fdb34155ac0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400c420 tx=0x7fdb2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb2affd640 1 -- 172.21.15.133:0/611280095 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24017070 con 0x7fdb340d01d0 2024-04-25T10:17:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb2affd640 1 -- 172.21.15.133:0/611280095 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb24005cf0 con 0x7fdb340d01d0 2024-04-25T10:17:18.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb2affd640 1 -- 172.21.15.133:0/611280095 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb240072f0 con 0x7fdb340d01d0 2024-04-25T10:17:18.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb38ddb640 1 -- 172.21.15.133:0/611280095 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb341561a0 con 0x7fdb340d01d0 2024-04-25T10:17:18.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.832+0000 7fdb38ddb640 1 -- 172.21.15.133:0/611280095 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb34156760 con 0x7fdb340d01d0 2024-04-25T10:17:18.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb2affd640 1 -- 172.21.15.133:0/611280095 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb2401a070 con 0x7fdb340d01d0 2024-04-25T10:17:18.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb2affd640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb14066f60 0x7fdb14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb2affd640 1 -- 172.21.15.133:0/611280095 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdb24084580 con 0x7fdb340d01d0 2024-04-25T10:17:18.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb14066f60 0x7fdb14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb14066f60 0x7fdb14069420 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdb340c5030 tx=0x7fdb1c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb3bf48000 1 Processor -- start 2024-04-25T10:17:18.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb3bf48000 1 -- start start 2024-04-25T10:17:18.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662180 0x55d29b662560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662aa0 0x55d29b6180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b6189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29b465f20 con 0x55d29b618600 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29b3ee000 con 0x55d29b662180 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d29b461fc0 con 0x55d29b662aa0 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb32d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b6189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb32d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b6189e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56602/0 (socket says 172.21.15.133:56602) 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb32d9d640 1 -- 172.21.15.133:0/1588682546 learned_addr learned my addr 172.21.15.133:0/1588682546 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/1588682546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662aa0 0x55d29b6180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb32d9d640 1 -- 172.21.15.133:0/1588682546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662aa0 msgr2=0x55d29b6180c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/1588682546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662aa0 0x55d29b6180c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb32d9d640 1 -- 172.21.15.133:0/1588682546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662180 msgr2=0x55d29b662560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb3259c640 1 --2- 172.21.15.133:0/1588682546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662180 0x55d29b662560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/1588682546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662180 0x55d29b662560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb32d9d640 1 -- 172.21.15.133:0/1588682546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d29b441ba0 con 0x55d29b618600 2024-04-25T10:17:18.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/1588682546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b6189e0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009d90 tx=0x7fdb2c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ecded902822f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb13fff640 1 -- 172.21.15.133:0/1588682546 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0be070 con 0x55d29b618600 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb13fff640 1 -- 172.21.15.133:0/1588682546 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c0b3040 con 0x55d29b618600 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.836+0000 7fdb13fff640 1 -- 172.21.15.133:0/1588682546 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0b2040 con 0x55d29b618600 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- 172.21.15.133:0/1588682546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 msgr2=0x55d29b6189e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/1588682546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b6189e0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009d90 tx=0x7fdb2c004ac0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- 172.21.15.133:0/1588682546 shutdown_connections 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/1588682546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b6189e0 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/1588682546 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662aa0 0x55d29b6180c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/1588682546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662180 0x55d29b662560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- 172.21.15.133:0/1588682546 >> 172.21.15.133:0/1588682546 conn(0x55d29b61e760 msgr2=0x55d29b6169a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- 172.21.15.133:0/1588682546 shutdown_connections 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- 172.21.15.133:0/1588682546 wait complete. 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- start start 2024-04-25T10:17:18.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 Processor -- start 2024-04-25T10:17:18.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b61d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662180 0x55d29b61dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662aa0 0x55d29b650450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29b5239c0 con 0x55d29b618600 2024-04-25T10:17:18.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29b4205d0 con 0x55d29b662180 2024-04-25T10:17:18.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d29b4304b0 con 0x55d29b662aa0 2024-04-25T10:17:18.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 10 client.0 _ll_register_callbacks cb 0x55d29b603a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:18.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b61d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b61d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56610/0 (socket says 172.21.15.133:56610) 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3259c640 1 -- 172.21.15.133:0/3178066366 learned_addr learned my addr 172.21.15.133:0/3178066366 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662aa0 0x55d29b650450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662180 0x55d29b61dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3259c640 1 -- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662aa0 msgr2=0x55d29b650450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3259c640 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662aa0 0x55d29b650450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3259c640 1 -- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662180 msgr2=0x55d29b61dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3259c640 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662180 0x55d29b61dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3259c640 1 -- 172.21.15.133:0/3178066366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d29b4ac5a0 con 0x55d29b618600 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662180 0x55d29b61dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3259c640 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b61d6e0 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7fdb2405ce40 tx=0x7fdb24054730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e000d6b72e3ad10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb117fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb117fa640 1 -- 172.21.15.133:0/3178066366 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24007690 con 0x55d29b618600 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb117fa640 1 -- 172.21.15.133:0/3178066366 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdb24084070 con 0x55d29b618600 2024-04-25T10:17:18.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- 172.21.15.133:0/3178066366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb2c0b3040 con 0x55d29b618600 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb117fa640 1 -- 172.21.15.133:0/3178066366 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb2405d370 con 0x55d29b618600 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 10 client.11673 fetch_fsmap learned FSMap version 112 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- 172.21.15.133:0/3178066366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55d29b43d2b0 con 0x55d29b618600 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb117fa640 1 -- 172.21.15.133:0/3178066366 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c0b3040 con 0x55d29b618600 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb117fa640 1 -- 172.21.15.133:0/3178066366 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb24082ce0 con 0x55d29b618600 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 10 client.11673 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 10 client.11673 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.b 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 20 client.11673 populate_metadata read hostname 'smithi133' 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55d29b64fe60 0x55d29b650e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 4 client.11673 mds_command: new command op to 11502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb3bf48000 1 -- 172.21.15.133:0/3178066366 --> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d29b401c70 con 0x55d29b64fe60 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.840+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55d29b64fe60 0x55d29b650e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb117fa640 10 client.11673 ms_handle_connect on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55d29b64fe60 0x55d29b650e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83750ffee742a55 server_cookie=4e51d57283e2a188 in_seq=0 out_seq=0 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb117fa640 1 -- 172.21.15.133:0/3178066366 <== mds.0 v2:172.21.15.162:6834/1744425810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d29b401c70 con 0x55d29b64fe60 2024-04-25T10:17:18.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb117fa640 10 client.11673 handle_command_reply: tid=0 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 1 client.11673 shutdown 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 1 -- 172.21.15.133:0/3178066366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 msgr2=0x55d29b61d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b61d6e0 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7fdb2405ce40 tx=0x7fdb24054730 comp rx=0 tx=0).stop 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 1 -- 172.21.15.133:0/3178066366 shutdown_connections 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55d29b64fe60 0x55d29b650e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662aa0 0x55d29b650450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662180 0x55d29b61dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb117fa640 0 client.11673 ms_handle_reset on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/3178066366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b618600 0x55d29b61d6e0 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 1 -- 172.21.15.133:0/3178066366 >> 172.21.15.133:0/3178066366 conn(0x55d29b61e760 msgr2=0x55d29b6170b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 1 -- 172.21.15.133:0/3178066366 shutdown_connections 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 1 -- 172.21.15.133:0/3178066366 wait complete. 2024-04-25T10:17:18.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.844+0000 7fdb3bf48000 20 client.11673 trim_cache size 0 max 16384 2024-04-25T10:17:18.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb3bf48000 1 Processor -- start 2024-04-25T10:17:18.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb3bf48000 1 -- start start 2024-04-25T10:17:18.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b618600 0x55d29b6615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662180 0x55d29b661b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662aa0 0x55d29b617120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb34200df0 con 0x55d29b662aa0 2024-04-25T10:17:18.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341ffff0 con 0x55d29b618600 2024-04-25T10:17:18.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341ff3f0 con 0x55d29b662180 2024-04-25T10:17:18.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb32d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662aa0 0x55d29b617120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb32d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662aa0 0x55d29b617120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56616/0 (socket says 172.21.15.133:56616) 2024-04-25T10:17:18.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb32d9d640 1 -- 172.21.15.133:0/893797953 learned_addr learned my addr 172.21.15.133:0/893797953 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:18.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/893797953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662180 0x55d29b661b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb32d9d640 1 -- 172.21.15.133:0/893797953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662180 msgr2=0x55d29b661b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/893797953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662180 0x55d29b661b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb32d9d640 1 -- 172.21.15.133:0/893797953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b618600 msgr2=0x55d29b6615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/893797953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b618600 0x55d29b6615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb32d9d640 1 -- 172.21.15.133:0/893797953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d29b3ee250 con 0x55d29b662aa0 2024-04-25T10:17:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/893797953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662180 0x55d29b661b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/893797953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662aa0 0x55d29b617120 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009d90 tx=0x7fdb2c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab86649ada5722f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb137fe640 1 -- 172.21.15.133:0/893797953 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0b2040 con 0x55d29b662aa0 2024-04-25T10:17:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb137fe640 1 -- 172.21.15.133:0/893797953 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c0a5b10 con 0x55d29b662aa0 2024-04-25T10:17:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.856+0000 7fdb137fe640 1 -- 172.21.15.133:0/893797953 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0be040 con 0x55d29b662aa0 2024-04-25T10:17:18.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- 172.21.15.133:0/893797953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662aa0 msgr2=0x55d29b617120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/893797953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662aa0 0x55d29b617120 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009d90 tx=0x7fdb2c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:17:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- 172.21.15.133:0/893797953 shutdown_connections 2024-04-25T10:17:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/893797953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662aa0 0x55d29b617120 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/893797953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b662180 0x55d29b661b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/893797953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b618600 0x55d29b6615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- 172.21.15.133:0/893797953 >> 172.21.15.133:0/893797953 conn(0x55d29b61e760 msgr2=0x55d29b660ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- 172.21.15.133:0/893797953 shutdown_connections 2024-04-25T10:17:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- 172.21.15.133:0/893797953 wait complete. 2024-04-25T10:17:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- start start 2024-04-25T10:17:18.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 Processor -- start 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b618600 0x55d29b676b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662180 0x55d29b677090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662aa0 0x55d29b6775d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d29b4112f0 con 0x55d29b662180 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341ff9f0 con 0x55d29b662aa0 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d29b462f90 con 0x55d29b618600 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 10 client.0 _ll_register_callbacks cb 0x55d29b5e4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb2bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662180 0x55d29b677090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb2bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662180 0x55d29b677090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56622/0 (socket says 172.21.15.133:56622) 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb2bfff640 1 -- 172.21.15.133:0/59825558 learned_addr learned my addr 172.21.15.133:0/59825558 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3259c640 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b618600 0x55d29b676b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662aa0 0x55d29b6775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb2bfff640 1 -- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b618600 msgr2=0x55d29b676b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b618600 0x55d29b676b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb2bfff640 1 -- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662aa0 msgr2=0x55d29b6775d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662aa0 0x55d29b6775d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb2bfff640 1 -- 172.21.15.133:0/59825558 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d29b3fab90 con 0x55d29b662180 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb32d9d640 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662aa0 0x55d29b6775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb2bfff640 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662180 0x55d29b677090 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c011690 tx=0x7fdb1c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92f847ed88d1b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb117fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb117fa640 1 -- 172.21.15.133:0/59825558 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c01e040 con 0x55d29b662180 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb117fa640 1 -- 172.21.15.133:0/59825558 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdb1c00f040 con 0x55d29b662180 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- 172.21.15.133:0/59825558 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb2c0a5b10 con 0x55d29b662180 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb117fa640 1 -- 172.21.15.133:0/59825558 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb1c037050 con 0x55d29b662180 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 10 client.11679 fetch_fsmap learned FSMap version 112 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- 172.21.15.133:0/59825558 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55d29b40d010 con 0x55d29b662180 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb117fa640 1 -- 172.21.15.133:0/59825558 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c0a5b10 con 0x55d29b662180 2024-04-25T10:17:18.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb117fa640 1 -- 172.21.15.133:0/59825558 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb1c03a020 con 0x55d29b662180 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 10 client.11679 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 10 client.11679 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.b 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 20 client.11679 populate_metadata read hostname 'smithi133' 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55d29b6d05f0 0x55d29b6d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 4 client.11679 mds_command: new command op to 11502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3bf48000 1 -- 172.21.15.133:0/59825558 --> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d29b6507e0 con 0x55d29b6d05f0 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.860+0000 7fdb3259c640 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55d29b6d05f0 0x55d29b6d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb117fa640 10 client.11679 ms_handle_connect on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3259c640 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55d29b6d05f0 0x55d29b6d29e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe8228962c6aa37 server_cookie=203d7899467bcb98 in_seq=0 out_seq=0 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb117fa640 1 -- 172.21.15.133:0/59825558 <== mds.0 v2:172.21.15.162:6834/1744425810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d29b6507e0 con 0x55d29b6d05f0 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb117fa640 10 client.11679 handle_command_reply: tid=0 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 1 client.11679 shutdown 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 1 -- 172.21.15.133:0/59825558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662180 msgr2=0x55d29b677090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662180 0x55d29b677090 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c011690 tx=0x7fdb1c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 1 -- 172.21.15.133:0/59825558 shutdown_connections 2024-04-25T10:17:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:6834/1744425810,v1:172.21.15.162:6835/1744425810] conn(0x55d29b6d05f0 0x55d29b6d29e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d29b662aa0 0x55d29b6775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb117fa640 0 client.11679 ms_handle_reset on v2:172.21.15.162:6834/1744425810 2024-04-25T10:17:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29b662180 0x55d29b677090 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 1 --2- 172.21.15.133:0/59825558 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55d29b618600 0x55d29b676b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 1 -- 172.21.15.133:0/59825558 >> 172.21.15.133:0/59825558 conn(0x55d29b61e760 msgr2=0x55d29b661770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 1 -- 172.21.15.133:0/59825558 shutdown_connections 2024-04-25T10:17:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 1 -- 172.21.15.133:0/59825558 wait complete. 2024-04-25T10:17:18.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb3bf48000 20 client.11679 trim_cache size 0 max 16384 2024-04-25T10:17:18.876 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2024-04-25T10:17:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 -- 172.21.15.133:0/611280095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb14066f60 msgr2=0x7fdb14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb14066f60 0x7fdb14069420 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdb340c5030 tx=0x7fdb1c009190 comp rx=0 tx=0).stop 2024-04-25T10:17:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 -- 172.21.15.133:0/611280095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb340d01d0 msgr2=0x7fdb34155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb340d01d0 0x7fdb34155ac0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400c420 tx=0x7fdb2400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 -- 172.21.15.133:0/611280095 shutdown_connections 2024-04-25T10:17:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdb3415bc90 0x7fdb34070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdb3415b8b0 0x7fdb34154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdb14066f60 0x7fdb14069420 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 --2- 172.21.15.133:0/611280095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb340d01d0 0x7fdb34155ac0 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 -- 172.21.15.133:0/611280095 >> 172.21.15.133:0/611280095 conn(0x7fdb3409a990 msgr2=0x7fdb340b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 -- 172.21.15.133:0/611280095 shutdown_connections 2024-04-25T10:17:18.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:18.864+0000 7fdb38ddb640 1 -- 172.21.15.133:0/611280095 wait complete. 2024-04-25T10:17:18.881 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:17:18.882 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:18.930 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:18.972 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:17:18.972 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:17:18.972 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:18.977 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:19.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf85cc640 1 Processor -- start 2024-04-25T10:17:19.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf85cc640 1 -- start start 2024-04-25T10:17:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf85cc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9cf006eb70 0x7f9cf00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf85cc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf01584f0 0x7f9cf015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf85cc640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf00cbe60 con 0x7f9cf0158110 2024-04-25T10:17:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf85cc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf00b19b0 con 0x7f9cf006eb70 2024-04-25T10:17:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf85cc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf015b360 con 0x7f9cf01584f0 2024-04-25T10:17:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf6b69640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf01584f0 0x7f9cf015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf5b67640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f9cf006eb70 0x7f9cf00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf6368640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7f9cf0158110 0x7f9cf006e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:57814/0 (socket says 172.21.15.133:57814) 2024-04-25T10:17:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf6368640 1 -- 172.21.15.133:0/3805608335 learned_addr learned my addr 172.21.15.133:0/3805608335 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf6b69640 1 -- 172.21.15.133:0/3805608335 >> v2:172.21.15.162:3300/0 conn(0x7f9cf006eb70 msgr2=0x7f9cf00b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf6b69640 1 --2- 172.21.15.133:0/3805608335 >> v2:172.21.15.162:3300/0 conn(0x7f9cf006eb70 0x7f9cf00b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf6b69640 1 -- 172.21.15.133:0/3805608335 >> v1:172.21.15.162:6789/0 conn(0x7f9cf0158110 legacy=0x7f9cf006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.144+0000 7f9cf6b69640 1 -- 172.21.15.133:0/3805608335 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf015b640 con 0x7f9cf01584f0 2024-04-25T10:17:19.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf6b69640 1 --2- 172.21.15.133:0/3805608335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf01584f0 0x7f9cf015ad10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009d90 tx=0x7f9ce801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5d3372a246d532 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf5366640 1 -- 172.21.15.133:0/3805608335 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80b2070 con 0x7f9cf01584f0 2024-04-25T10:17:19.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf5366640 1 -- 172.21.15.133:0/3805608335 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce80a5a20 con 0x7f9cf01584f0 2024-04-25T10:17:19.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf5366640 1 -- 172.21.15.133:0/3805608335 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80a5df0 con 0x7f9cf01584f0 2024-04-25T10:17:19.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf85cc640 1 -- 172.21.15.133:0/3805608335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf01584f0 msgr2=0x7f9cf015ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf85cc640 1 --2- 172.21.15.133:0/3805608335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf01584f0 0x7f9cf015ad10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009d90 tx=0x7f9ce801c940 comp rx=0 tx=0).stop 2024-04-25T10:17:19.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf85cc640 1 -- 172.21.15.133:0/3805608335 shutdown_connections 2024-04-25T10:17:19.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf85cc640 1 --2- 172.21.15.133:0/3805608335 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf01584f0 0x7f9cf015ad10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf85cc640 1 --2- 172.21.15.133:0/3805608335 >> v2:172.21.15.162:3300/0 conn(0x7f9cf006eb70 0x7f9cf00b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf85cc640 1 -- 172.21.15.133:0/3805608335 >> 172.21.15.133:0/3805608335 conn(0x7f9cf0067000 msgr2=0x7f9cf006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:19.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf85cc640 1 -- 172.21.15.133:0/3805608335 shutdown_connections 2024-04-25T10:17:19.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf85cc640 1 -- 172.21.15.133:0/3805608335 wait complete. 2024-04-25T10:17:19.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf85cc640 1 Processor -- start 2024-04-25T10:17:19.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.148+0000 7f9cf85cc640 1 -- start start 2024-04-25T10:17:19.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf85cc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf006eb70 0x7f9cf00cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf85cc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cf0158110 0x7f9cf00c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf85cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf01584f0 0x7f9cf0142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf85cc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00ccca0 con 0x7f9cf01584f0 2024-04-25T10:17:19.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf85cc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00ccb20 con 0x7f9cf0158110 2024-04-25T10:17:19.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf85cc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00cce20 con 0x7f9cf006eb70 2024-04-25T10:17:19.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6b69640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf01584f0 0x7f9cf0142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6b69640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf01584f0 0x7f9cf0142d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56630/0 (socket says 172.21.15.133:56630) 2024-04-25T10:17:19.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6b69640 1 -- 172.21.15.133:0/2251669000 learned_addr learned my addr 172.21.15.133:0/2251669000 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6368640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf006eb70 0x7f9cf00cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf5b67640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cf0158110 0x7f9cf00c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6b69640 1 -- 172.21.15.133:0/2251669000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf006eb70 msgr2=0x7f9cf00cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6b69640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf006eb70 0x7f9cf00cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6b69640 1 -- 172.21.15.133:0/2251669000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cf0158110 msgr2=0x7f9cf00c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6b69640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cf0158110 0x7f9cf00c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6b69640 1 -- 172.21.15.133:0/2251669000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf00c49e0 con 0x7f9cf01584f0 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6368640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf006eb70 0x7f9cf00cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6b69640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf01584f0 0x7f9cf0142d70 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80b0040 tx=0x7f9ce80196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9ce6ffd640 1 -- 172.21.15.133:0/2251669000 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80b2050 con 0x7f9cf01584f0 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9ce6ffd640 1 -- 172.21.15.133:0/2251669000 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce80be070 con 0x7f9cf01584f0 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf85cc640 1 -- 172.21.15.133:0/2251669000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf00c7490 con 0x7f9cf01584f0 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9ce6ffd640 1 -- 172.21.15.133:0/2251669000 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80adad0 con 0x7f9cf01584f0 2024-04-25T10:17:19.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf85cc640 1 -- 172.21.15.133:0/2251669000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf00c7710 con 0x7f9cf01584f0 2024-04-25T10:17:19.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf85cc640 1 -- 172.21.15.133:0/2251669000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf0068ef0 con 0x7f9cf01584f0 2024-04-25T10:17:19.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9ce6ffd640 1 -- 172.21.15.133:0/2251669000 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ce80adc70 con 0x7f9cf01584f0 2024-04-25T10:17:19.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9ce6ffd640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9cd4066f90 0x7f9cd4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9ce6ffd640 1 -- 172.21.15.133:0/2251669000 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ce8120870 con 0x7f9cf01584f0 2024-04-25T10:17:19.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6368640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9cd4066f90 0x7f9cd4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.152+0000 7f9cf6368640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9cd4066f90 0x7f9cd4069450 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000f70 tx=0x7f9cec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.156+0000 7f9ce6ffd640 1 -- 172.21.15.133:0/2251669000 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ce80c7020 con 0x7f9cf01584f0 2024-04-25T10:17:19.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.316+0000 7f9cf85cc640 1 -- 172.21.15.133:0/2251669000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cf0155c30 con 0x7f9cf01584f0 2024-04-25T10:17:19.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.320+0000 7f9ce6ffd640 1 -- 172.21.15.133:0/2251669000 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f9cf00c7490 con 0x7f9cf01584f0 2024-04-25T10:17:19.322 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:19.323 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":183,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:07.261559+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:03.841068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:04.604661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 -- 172.21.15.133:0/2251669000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9cd4066f90 msgr2=0x7f9cd4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9cd4066f90 0x7f9cd4069450 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000f70 tx=0x7f9cec009040 comp rx=0 tx=0).stop 2024-04-25T10:17:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 -- 172.21.15.133:0/2251669000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf01584f0 msgr2=0x7f9cf0142d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf01584f0 0x7f9cf0142d70 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80b0040 tx=0x7f9ce80196e0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 -- 172.21.15.133:0/2251669000 shutdown_connections 2024-04-25T10:17:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9cd4066f90 0x7f9cd4069450 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf01584f0 0x7f9cf0142d70 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cf0158110 0x7f9cf00c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 --2- 172.21.15.133:0/2251669000 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9cf006eb70 0x7f9cf00cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 -- 172.21.15.133:0/2251669000 >> 172.21.15.133:0/2251669000 conn(0x7f9cf0067000 msgr2=0x7f9cf00ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:19.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 -- 172.21.15.133:0/2251669000 shutdown_connections 2024-04-25T10:17:19.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.324+0000 7f9cf85cc640 1 -- 172.21.15.133:0/2251669000 wait complete. 2024-04-25T10:17:19.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:03.841068+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:04.604661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:19.342 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:19.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f706b431640 1 Processor -- start 2024-04-25T10:17:19.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f706b431640 1 -- start start 2024-04-25T10:17:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f706b431640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f706415d5e0 0x7f70640cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f706b431640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70640ce2e0 con 0x7f70640cd820 2024-04-25T10:17:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f706b431640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706406d4b0 con 0x7f7064145170 2024-04-25T10:17:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f706b431640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706406db00 con 0x7f706415d5e0 2024-04-25T10:17:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f70689cc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f70640cd820 0x7f706415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46540/0 (socket says 172.21.15.133:46540) 2024-04-25T10:17:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f70699ce640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f706415d5e0 0x7f70640cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f70689cc640 1 -- 172.21.15.133:0/3489380495 learned_addr learned my addr 172.21.15.133:0/3489380495 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:19.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f705bfff640 1 -- 172.21.15.133:0/3489380495 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18679704 0 0) 0x7f70640ce2e0 con 0x7f70640cd820 2024-04-25T10:17:19.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f705bfff640 1 -- 172.21.15.133:0/3489380495 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7050003530 con 0x7f70640cd820 2024-04-25T10:17:19.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f70699ce640 1 -- 172.21.15.133:0/3489380495 >> v1:172.21.15.162:6789/0 conn(0x7f7064145170 legacy=0x7f706406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:19.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f70699ce640 1 -- 172.21.15.133:0/3489380495 >> v1:172.21.15.133:6789/0 conn(0x7f70640cd820 legacy=0x7f706415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.468+0000 7f70699ce640 1 -- 172.21.15.133:0/3489380495 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706406d790 con 0x7f706415d5e0 2024-04-25T10:17:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70699ce640 1 --2- 172.21.15.133:0/3489380495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f706415d5e0 0x7f70640cdca0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f705c009d90 tx=0x7f705c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17383e747e8dac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f705bfff640 1 -- 172.21.15.133:0/3489380495 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c0b2070 con 0x7f706415d5e0 2024-04-25T10:17:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f705bfff640 1 -- 172.21.15.133:0/3489380495 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c0a57f0 con 0x7f706415d5e0 2024-04-25T10:17:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f705bfff640 1 -- 172.21.15.133:0/3489380495 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c0a5bc0 con 0x7f706415d5e0 2024-04-25T10:17:19.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 -- 172.21.15.133:0/3489380495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f706415d5e0 msgr2=0x7f70640cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 --2- 172.21.15.133:0/3489380495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f706415d5e0 0x7f70640cdca0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f705c009d90 tx=0x7f705c01c930 comp rx=0 tx=0).stop 2024-04-25T10:17:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 -- 172.21.15.133:0/3489380495 shutdown_connections 2024-04-25T10:17:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 --2- 172.21.15.133:0/3489380495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f706415d5e0 0x7f70640cdca0 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 -- 172.21.15.133:0/3489380495 >> 172.21.15.133:0/3489380495 conn(0x7f7064066f70 msgr2=0x7f706406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 -- 172.21.15.133:0/3489380495 shutdown_connections 2024-04-25T10:17:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 -- 172.21.15.133:0/3489380495 wait complete. 2024-04-25T10:17:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 Processor -- start 2024-04-25T10:17:19.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 -- start start 2024-04-25T10:17:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70640cd820 0x7f706413d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7064145170 0x7f70640b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70640b11f0 0x7f706415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70641696b0 con 0x7f70640b11f0 2024-04-25T10:17:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7064145550 con 0x7f7064145170 2024-04-25T10:17:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f706b431640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7064169830 con 0x7f70640cd820 2024-04-25T10:17:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70699ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70640b11f0 0x7f706415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70699ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70640b11f0 0x7f706415a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56660/0 (socket says 172.21.15.133:56660) 2024-04-25T10:17:19.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70699ce640 1 -- 172.21.15.133:0/4024128774 learned_addr learned my addr 172.21.15.133:0/4024128774 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70691cd640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70640cd820 0x7f706413d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70699ce640 1 -- 172.21.15.133:0/4024128774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70640cd820 msgr2=0x7f706413d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70689cc640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7064145170 0x7f70640b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70699ce640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70640cd820 0x7f706413d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70699ce640 1 -- 172.21.15.133:0/4024128774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7064145170 msgr2=0x7f70640b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70699ce640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7064145170 0x7f70640b0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70699ce640 1 -- 172.21.15.133:0/4024128774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7064142ee0 con 0x7f70640b11f0 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70691cd640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70640cd820 0x7f706413d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.472+0000 7f70699ce640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70640b11f0 0x7f706415a7c0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f705c0b0040 tx=0x7f705c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.476+0000 7f7059ffb640 1 -- 172.21.15.133:0/4024128774 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c0b2040 con 0x7f70640b11f0 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.476+0000 7f7059ffb640 1 -- 172.21.15.133:0/4024128774 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c0bf070 con 0x7f70640b11f0 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.476+0000 7f706b431640 1 -- 172.21.15.133:0/4024128774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7064142360 con 0x7f70640b11f0 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.476+0000 7f7059ffb640 1 -- 172.21.15.133:0/4024128774 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c0b86b0 con 0x7f70640b11f0 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.476+0000 7f706b431640 1 -- 172.21.15.133:0/4024128774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7064142920 con 0x7f70640b11f0 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.476+0000 7f706b431640 1 -- 172.21.15.133:0/4024128774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7064068e80 con 0x7f70640b11f0 2024-04-25T10:17:19.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.476+0000 7f7059ffb640 1 -- 172.21.15.133:0/4024128774 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f705c0b8850 con 0x7f70640b11f0 2024-04-25T10:17:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.476+0000 7f7059ffb640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7048066f60 0x7f7048069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.476+0000 7f70691cd640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7048066f60 0x7f7048069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.476+0000 7f7059ffb640 1 -- 172.21.15.133:0/4024128774 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f705c120340 con 0x7f70640b11f0 2024-04-25T10:17:19.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.476+0000 7f70691cd640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7048066f60 0x7f7048069420 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7060003e40 tx=0x7f7060009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.480+0000 7f7059ffb640 1 -- 172.21.15.133:0/4024128774 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f705c016520 con 0x7f70640b11f0 2024-04-25T10:17:19.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.640+0000 7f706b431640 1 -- 172.21.15.133:0/4024128774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70640b4860 con 0x7f70640b11f0 2024-04-25T10:17:19.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.640+0000 7f7059ffb640 1 -- 172.21.15.133:0/4024128774 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) v1 ==== 93+0+10 (secure 0 0 0) 0x7f705c0f12f0 con 0x7f70640b11f0 2024-04-25T10:17:19.642 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:17:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 -- 172.21.15.133:0/4024128774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7048066f60 msgr2=0x7f7048069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7048066f60 0x7f7048069420 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7060003e40 tx=0x7f7060009040 comp rx=0 tx=0).stop 2024-04-25T10:17:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 -- 172.21.15.133:0/4024128774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70640b11f0 msgr2=0x7f706415a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70640b11f0 0x7f706415a7c0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f705c0b0040 tx=0x7f705c0196d0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 -- 172.21.15.133:0/4024128774 shutdown_connections 2024-04-25T10:17:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7048066f60 0x7f7048069420 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70640b11f0 0x7f706415a7c0 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7064145170 0x7f70640b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 --2- 172.21.15.133:0/4024128774 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70640cd820 0x7f706413d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 -- 172.21.15.133:0/4024128774 >> 172.21.15.133:0/4024128774 conn(0x7f7064066f70 msgr2=0x7f706415f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:19.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 -- 172.21.15.133:0/4024128774 shutdown_connections 2024-04-25T10:17:19.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.644+0000 7f706b431640 1 -- 172.21.15.133:0/4024128774 wait complete. 2024-04-25T10:17:19.662 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:17:19.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1da1a10640 1 Processor -- start 2024-04-25T10:17:19.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1da1a10640 1 -- start start 2024-04-25T10:17:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1da1a10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9c0cd820 0x7f1d9c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1da1a10640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1d9c15ced0 0x7f1d9c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1da1a10640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9c0ce2e0 con 0x7f1d9c145170 2024-04-25T10:17:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1da1a10640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9c06db00 con 0x7f1d9c15ced0 2024-04-25T10:17:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1da1a10640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c145550 con 0x7f1d9c0cd820 2024-04-25T10:17:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1d9affd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1d9c145170 0x7f1d9c06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46554/0 (socket says 172.21.15.133:46554) 2024-04-25T10:17:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1d9affd640 1 -- 172.21.15.133:0/2965778933 learned_addr learned my addr 172.21.15.133:0/2965778933 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1d9b7fe640 1 --2- 172.21.15.133:0/2965778933 >> v2:172.21.15.133:3300/0 conn(0x7f1d9c15ced0 0x7f1d9c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1d9a7fc640 1 --2- 172.21.15.133:0/2965778933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9c0cd820 0x7f1d9c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1d99ffb640 1 -- 172.21.15.133:0/2965778933 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508488626 0 0) 0x7f1d9c0ce2e0 con 0x7f1d9c145170 2024-04-25T10:17:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1d99ffb640 1 -- 172.21.15.133:0/2965778933 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d84003530 con 0x7f1d9c145170 2024-04-25T10:17:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1d9b7fe640 1 -- 172.21.15.133:0/2965778933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9c0cd820 msgr2=0x7f1d9c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1d9b7fe640 1 --2- 172.21.15.133:0/2965778933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9c0cd820 0x7f1d9c0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1d9b7fe640 1 -- 172.21.15.133:0/2965778933 >> v1:172.21.15.133:6789/0 conn(0x7f1d9c145170 legacy=0x7f1d9c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1d9b7fe640 1 -- 172.21.15.133:0/2965778933 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c06d4b0 con 0x7f1d9c15ced0 2024-04-25T10:17:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.780+0000 7f1d9a7fc640 1 --2- 172.21.15.133:0/2965778933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9c0cd820 0x7f1d9c0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1d9b7fe640 1 --2- 172.21.15.133:0/2965778933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c15ced0 0x7f1d9c15d360 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009d90 tx=0x7f1d9401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce9f3da536b1509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1d99ffb640 1 -- 172.21.15.133:0/2965778933 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d940b2070 con 0x7f1d9c15ced0 2024-04-25T10:17:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1d99ffb640 1 -- 172.21.15.133:0/2965778933 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d940a5830 con 0x7f1d9c15ced0 2024-04-25T10:17:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1d99ffb640 1 -- 172.21.15.133:0/2965778933 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d940a5c00 con 0x7f1d9c15ced0 2024-04-25T10:17:19.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 -- 172.21.15.133:0/2965778933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c15ced0 msgr2=0x7f1d9c15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 --2- 172.21.15.133:0/2965778933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c15ced0 0x7f1d9c15d360 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009d90 tx=0x7f1d9401c970 comp rx=0 tx=0).stop 2024-04-25T10:17:19.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 -- 172.21.15.133:0/2965778933 shutdown_connections 2024-04-25T10:17:19.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 --2- 172.21.15.133:0/2965778933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c15ced0 0x7f1d9c15d360 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 --2- 172.21.15.133:0/2965778933 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9c0cd820 0x7f1d9c0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 -- 172.21.15.133:0/2965778933 >> 172.21.15.133:0/2965778933 conn(0x7f1d9c066f70 msgr2=0x7f1d9c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:19.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 -- 172.21.15.133:0/2965778933 shutdown_connections 2024-04-25T10:17:19.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 -- 172.21.15.133:0/2965778933 wait complete. 2024-04-25T10:17:19.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 Processor -- start 2024-04-25T10:17:19.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 -- start start 2024-04-25T10:17:19.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d9c0cd820 0x7f1d9c13b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c145170 0x7f1d9c13b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9c15ced0 0x7f1d9c0b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c168610 con 0x7f1d9c145170 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c168490 con 0x7f1d9c0cd820 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1da1a10640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c168790 con 0x7f1d9c15ced0 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1d9a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c145170 0x7f1d9c13b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1d9a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c145170 0x7f1d9c13b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56686/0 (socket says 172.21.15.133:56686) 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.784+0000 7f1d9a7fc640 1 -- 172.21.15.133:0/1921196930 learned_addr learned my addr 172.21.15.133:0/1921196930 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d9b7fe640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9c15ced0 0x7f1d9c0b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d9affd640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d9c0cd820 0x7f1d9c13b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d9a7fc640 1 -- 172.21.15.133:0/1921196930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9c15ced0 msgr2=0x7f1d9c0b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d9a7fc640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9c15ced0 0x7f1d9c0b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d9a7fc640 1 -- 172.21.15.133:0/1921196930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d9c0cd820 msgr2=0x7f1d9c13b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d9a7fc640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d9c0cd820 0x7f1d9c13b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d9a7fc640 1 -- 172.21.15.133:0/1921196930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c13c1a0 con 0x7f1d9c145170 2024-04-25T10:17:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d9affd640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d9c0cd820 0x7f1d9c13b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d9a7fc640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c145170 0x7f1d9c13b0d0 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c7c0 tx=0x7f1d9000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d83fff640 1 -- 172.21.15.133:0/1921196930 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90007a30 con 0x7f1d9c145170 2024-04-25T10:17:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d83fff640 1 -- 172.21.15.133:0/1921196930 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d90007bd0 con 0x7f1d9c145170 2024-04-25T10:17:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d83fff640 1 -- 172.21.15.133:0/1921196930 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d900113f0 con 0x7f1d9c145170 2024-04-25T10:17:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1da1a10640 1 -- 172.21.15.133:0/1921196930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c13c480 con 0x7f1d9c145170 2024-04-25T10:17:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1da1a10640 1 -- 172.21.15.133:0/1921196930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c13ca40 con 0x7f1d9c145170 2024-04-25T10:17:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1da1a10640 1 -- 172.21.15.133:0/1921196930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c068e80 con 0x7f1d9c145170 2024-04-25T10:17:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d83fff640 1 -- 172.21.15.133:0/1921196930 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d90011590 con 0x7f1d9c145170 2024-04-25T10:17:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d83fff640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d88066f60 0x7f1d88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d83fff640 1 -- 172.21.15.133:0/1921196930 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1d90085350 con 0x7f1d9c145170 2024-04-25T10:17:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d9affd640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d88066f60 0x7f1d88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.788+0000 7f1d9affd640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d88066f60 0x7f1d88069420 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c005890 tx=0x7f1d8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.792+0000 7f1d83fff640 1 -- 172.21.15.133:0/1921196930 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d90015020 con 0x7f1d9c145170 2024-04-25T10:17:19.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.948+0000 7f1da1a10640 1 -- 172.21.15.133:0/1921196930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d9c0738b0 con 0x7f1d9c145170 2024-04-25T10:17:19.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1d83fff640 1 -- 172.21.15.133:0/1921196930 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v183) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d90056260 con 0x7f1d9c145170 2024-04-25T10:17:19.953 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:17:19.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 -- 172.21.15.133:0/1921196930 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d88066f60 msgr2=0x7f1d88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d88066f60 0x7f1d88069420 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c005890 tx=0x7f1d8c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:19.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 -- 172.21.15.133:0/1921196930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c145170 msgr2=0x7f1d9c13b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c145170 0x7f1d9c13b0d0 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c7c0 tx=0x7f1d9000cc80 comp rx=0 tx=0).stop 2024-04-25T10:17:19.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 -- 172.21.15.133:0/1921196930 shutdown_connections 2024-04-25T10:17:19.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1d9c15ced0 0x7f1d9c0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1d88066f60 0x7f1d88069420 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d9c145170 0x7f1d9c13b0d0 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 --2- 172.21.15.133:0/1921196930 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1d9c0cd820 0x7f1d9c13b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 -- 172.21.15.133:0/1921196930 >> 172.21.15.133:0/1921196930 conn(0x7f1d9c066f70 msgr2=0x7f1d9c144300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:19.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 -- 172.21.15.133:0/1921196930 shutdown_connections 2024-04-25T10:17:19.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:19.952+0000 7f1da1a10640 1 -- 172.21.15.133:0/1921196930 wait complete. 2024-04-25T10:17:19.970 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:17:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96e69c5640 1 Processor -- start 2024-04-25T10:17:20.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96e69c5640 1 -- start start 2024-04-25T10:17:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96e69c5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96e0145170 0x7f96e006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96e69c5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f96e006e4d0 0x7f96e00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96e69c5640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e00ce2e0 con 0x7f96e00cdd60 2024-04-25T10:17:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96e69c5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e006db00 con 0x7f96e0145170 2024-04-25T10:17:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96e69c5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e0145550 con 0x7f96e006e4d0 2024-04-25T10:17:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96dffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96e0145170 0x7f96e006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96dffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96e0145170 0x7f96e006df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56688/0 (socket says 172.21.15.133:56688) 2024-04-25T10:17:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96dffff640 1 -- 172.21.15.133:0/3488326637 learned_addr learned my addr 172.21.15.133:0/3488326637 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96df7fe640 1 --2- 172.21.15.133:0/3488326637 >> v2:172.21.15.162:3300/0 conn(0x7f96e006e4d0 0x7f96e00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96dffff640 1 -- 172.21.15.133:0/3488326637 >> v2:172.21.15.162:3300/0 conn(0x7f96e006e4d0 msgr2=0x7f96e00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96dffff640 1 --2- 172.21.15.133:0/3488326637 >> v2:172.21.15.162:3300/0 conn(0x7f96e006e4d0 0x7f96e00cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96dffff640 1 -- 172.21.15.133:0/3488326637 >> v1:172.21.15.162:6789/0 conn(0x7f96e00cdd60 legacy=0x7f96e015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.092+0000 7f96dffff640 1 -- 172.21.15.133:0/3488326637 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e006d4b0 con 0x7f96e0145170 2024-04-25T10:17:20.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96dffff640 1 --2- 172.21.15.133:0/3488326637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e0145170 0x7f96e006df90 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f96e00b4dd0 tx=0x7f96d001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd008523864b0988 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96deffd640 1 -- 172.21.15.133:0/3488326637 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00b0070 con 0x7f96e0145170 2024-04-25T10:17:20.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96deffd640 1 -- 172.21.15.133:0/3488326637 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d00ab600 con 0x7f96e0145170 2024-04-25T10:17:20.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96deffd640 1 -- 172.21.15.133:0/3488326637 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00ab9d0 con 0x7f96e0145170 2024-04-25T10:17:20.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 -- 172.21.15.133:0/3488326637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e0145170 msgr2=0x7f96e006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 --2- 172.21.15.133:0/3488326637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e0145170 0x7f96e006df90 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f96e00b4dd0 tx=0x7f96d001c490 comp rx=0 tx=0).stop 2024-04-25T10:17:20.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 -- 172.21.15.133:0/3488326637 shutdown_connections 2024-04-25T10:17:20.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 --2- 172.21.15.133:0/3488326637 >> v2:172.21.15.162:3300/0 conn(0x7f96e006e4d0 0x7f96e00cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 --2- 172.21.15.133:0/3488326637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e0145170 0x7f96e006df90 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 -- 172.21.15.133:0/3488326637 >> 172.21.15.133:0/3488326637 conn(0x7f96e0066f70 msgr2=0x7f96e006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:20.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 -- 172.21.15.133:0/3488326637 shutdown_connections 2024-04-25T10:17:20.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 -- 172.21.15.133:0/3488326637 wait complete. 2024-04-25T10:17:20.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 Processor -- start 2024-04-25T10:17:20.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 -- start start 2024-04-25T10:17:20.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e006e4d0 0x7f96e01520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e00cdd60 0x7f96e013c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96e0145170 0x7f96e0143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e0168490 con 0x7f96e00cdd60 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e015d7f0 con 0x7f96e006e4d0 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e69c5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e0168610 con 0x7f96e0145170 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96df7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e00cdd60 0x7f96e013c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96df7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e00cdd60 0x7f96e013c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56704/0 (socket says 172.21.15.133:56704) 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96df7fe640 1 -- 172.21.15.133:0/3016436814 learned_addr learned my addr 172.21.15.133:0/3016436814 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96e4f62640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96e0145170 0x7f96e0143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96df7fe640 1 -- 172.21.15.133:0/3016436814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96e0145170 msgr2=0x7f96e0143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96df7fe640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96e0145170 0x7f96e0143c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96dffff640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e006e4d0 0x7f96e01520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96df7fe640 1 -- 172.21.15.133:0/3016436814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e006e4d0 msgr2=0x7f96e01520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96df7fe640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e006e4d0 0x7f96e01520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96df7fe640 1 -- 172.21.15.133:0/3016436814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e0139610 con 0x7f96e00cdd60 2024-04-25T10:17:20.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.096+0000 7f96df7fe640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e00cdd60 0x7f96e013c280 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f96d400d800 tx=0x7f96d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.100+0000 7f96dcff9640 1 -- 172.21.15.133:0/3016436814 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d400b6c0 con 0x7f96e00cdd60 2024-04-25T10:17:20.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.100+0000 7f96dcff9640 1 -- 172.21.15.133:0/3016436814 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d4019070 con 0x7f96e00cdd60 2024-04-25T10:17:20.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.100+0000 7f96e69c5640 1 -- 172.21.15.133:0/3016436814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e013ccd0 con 0x7f96e00cdd60 2024-04-25T10:17:20.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.100+0000 7f96dcff9640 1 -- 172.21.15.133:0/3016436814 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d400ba80 con 0x7f96e00cdd60 2024-04-25T10:17:20.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.100+0000 7f96e69c5640 1 -- 172.21.15.133:0/3016436814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e013d290 con 0x7f96e00cdd60 2024-04-25T10:17:20.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.100+0000 7f96e69c5640 1 -- 172.21.15.133:0/3016436814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e0068e80 con 0x7f96e00cdd60 2024-04-25T10:17:20.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.100+0000 7f96dcff9640 1 -- 172.21.15.133:0/3016436814 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96d400bc20 con 0x7f96e00cdd60 2024-04-25T10:17:20.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.100+0000 7f96dcff9640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96cc066f60 0x7f96cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.100+0000 7f96dffff640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96cc066f60 0x7f96cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.100+0000 7f96dcff9640 1 -- 172.21.15.133:0/3016436814 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f96d40855b0 con 0x7f96e00cdd60 2024-04-25T10:17:20.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.100+0000 7f96dffff640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96cc066f60 0x7f96cc069420 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f96d00ad000 tx=0x7f96d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.104+0000 7f96dcff9640 1 -- 172.21.15.133:0/3016436814 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f96d4056430 con 0x7f96e00cdd60 2024-04-25T10:17:20.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.260+0000 7f96e69c5640 1 -- 172.21.15.133:0/3016436814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96e015f350 con 0x7f96e00cdd60 2024-04-25T10:17:20.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.260+0000 7f96dcff9640 1 -- 172.21.15.133:0/3016436814 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v183) v1 ==== 100+0+11 (secure 0 0 0) 0x7f96d4055d30 con 0x7f96e00cdd60 2024-04-25T10:17:20.264 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:17:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 -- 172.21.15.133:0/3016436814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96cc066f60 msgr2=0x7f96cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96cc066f60 0x7f96cc069420 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f96d00ad000 tx=0x7f96d0008040 comp rx=0 tx=0).stop 2024-04-25T10:17:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 -- 172.21.15.133:0/3016436814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e00cdd60 msgr2=0x7f96e013c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e00cdd60 0x7f96e013c280 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f96d400d800 tx=0x7f96d400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 -- 172.21.15.133:0/3016436814 shutdown_connections 2024-04-25T10:17:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f96e0145170 0x7f96e0143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f96cc066f60 0x7f96cc069420 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e00cdd60 0x7f96e013c280 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 --2- 172.21.15.133:0/3016436814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96e006e4d0 0x7f96e01520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 -- 172.21.15.133:0/3016436814 >> 172.21.15.133:0/3016436814 conn(0x7f96e0066f70 msgr2=0x7f96e00b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 -- 172.21.15.133:0/3016436814 shutdown_connections 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.264+0000 7f96e69c5640 1 -- 172.21.15.133:0/3016436814 wait complete. 2024-04-25T10:17:20.284 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:17:20.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0e6755640 1 Processor -- start 2024-04-25T10:17:20.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0e6755640 1 -- start start 2024-04-25T10:17:20.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0e6755640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc0e015b810 0x7fc0e00cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0e6755640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e00cc7f0 0x7fc0e015f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0e6755640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e015ff40 con 0x7fc0e015f9c0 2024-04-25T10:17:20.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0e6755640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e00b1320 con 0x7fc0e015b810 2024-04-25T10:17:20.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0e6755640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e00b18e0 con 0x7fc0e00cc7f0 2024-04-25T10:17:20.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0dffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc0e015b810 0x7fc0e00cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0dffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc0e015b810 0x7fc0e00cc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56706/0 (socket says 172.21.15.133:56706) 2024-04-25T10:17:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0dffff640 1 -- 172.21.15.133:0/3956198872 learned_addr learned my addr 172.21.15.133:0/3956198872 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0d7fff640 1 --2- 172.21.15.133:0/3956198872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e00cc7f0 0x7fc0e015f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0dffff640 1 -- 172.21.15.133:0/3956198872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e00cc7f0 msgr2=0x7fc0e015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0dffff640 1 --2- 172.21.15.133:0/3956198872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e00cc7f0 0x7fc0e015f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0dffff640 1 -- 172.21.15.133:0/3956198872 >> v1:172.21.15.162:6789/0 conn(0x7fc0e015f9c0 legacy=0x7fc0e00cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.404+0000 7fc0dffff640 1 -- 172.21.15.133:0/3956198872 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e0160220 con 0x7fc0e015b810 2024-04-25T10:17:20.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0dffff640 1 --2- 172.21.15.133:0/3956198872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e015b810 0x7fc0e00cc2b0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009870 tx=0x7fc0d001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfab56f7950add7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0df7fe640 1 -- 172.21.15.133:0/3956198872 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d00b5070 con 0x7fc0e015b810 2024-04-25T10:17:20.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0df7fe640 1 -- 172.21.15.133:0/3956198872 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d00a8a50 con 0x7fc0e015b810 2024-04-25T10:17:20.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0df7fe640 1 -- 172.21.15.133:0/3956198872 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d00a8e20 con 0x7fc0e015b810 2024-04-25T10:17:20.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0e6755640 1 -- 172.21.15.133:0/3956198872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e015b810 msgr2=0x7fc0e00cc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0e6755640 1 --2- 172.21.15.133:0/3956198872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e015b810 0x7fc0e00cc2b0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009870 tx=0x7fc0d001ed50 comp rx=0 tx=0).stop 2024-04-25T10:17:20.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0e6755640 1 -- 172.21.15.133:0/3956198872 shutdown_connections 2024-04-25T10:17:20.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0e6755640 1 --2- 172.21.15.133:0/3956198872 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e00cc7f0 0x7fc0e015f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0e6755640 1 --2- 172.21.15.133:0/3956198872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e015b810 0x7fc0e00cc2b0 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0e6755640 1 -- 172.21.15.133:0/3956198872 >> 172.21.15.133:0/3956198872 conn(0x7fc0e008af60 msgr2=0x7fc0e008b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:20.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0e6755640 1 -- 172.21.15.133:0/3956198872 shutdown_connections 2024-04-25T10:17:20.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0e6755640 1 -- 172.21.15.133:0/3956198872 wait complete. 2024-04-25T10:17:20.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0e6755640 1 Processor -- start 2024-04-25T10:17:20.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.408+0000 7fc0e6755640 1 -- start start 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0e6755640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e00cc7f0 0x7fc0e00af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0e6755640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e015b810 0x7fc0e00c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0e6755640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e015f9c0 0x7fc0e00c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0e6755640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e015bbf0 con 0x7fc0e015b810 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0e6755640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e00cf490 con 0x7fc0e00cc7f0 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0e6755640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e015bd70 con 0x7fc0e015f9c0 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e015b810 0x7fc0e00c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e015b810 0x7fc0e00c51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56718/0 (socket says 172.21.15.133:56718) 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0d7fff640 1 -- 172.21.15.133:0/2792468435 learned_addr learned my addr 172.21.15.133:0/2792468435 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0dffff640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e00cc7f0 0x7fc0e00af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0e4cf2640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e015f9c0 0x7fc0e00c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0d7fff640 1 -- 172.21.15.133:0/2792468435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e015f9c0 msgr2=0x7fc0e00c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0d7fff640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e015f9c0 0x7fc0e00c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0d7fff640 1 -- 172.21.15.133:0/2792468435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e00cc7f0 msgr2=0x7fc0e00af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0d7fff640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e00cc7f0 0x7fc0e00af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0d7fff640 1 -- 172.21.15.133:0/2792468435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e00c7bd0 con 0x7fc0e015b810 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0d7fff640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e015b810 0x7fc0e00c51d0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800e960 tx=0x7fc0c800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0dd7fa640 1 -- 172.21.15.133:0/2792468435 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8017070 con 0x7fc0e015b810 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0dd7fa640 1 -- 172.21.15.133:0/2792468435 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c800ca00 con 0x7fc0e015b810 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0e6755640 1 -- 172.21.15.133:0/2792468435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e00c7eb0 con 0x7fc0e015b810 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0dd7fa640 1 -- 172.21.15.133:0/2792468435 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c800cdb0 con 0x7fc0e015b810 2024-04-25T10:17:20.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0e6755640 1 -- 172.21.15.133:0/2792468435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e015c480 con 0x7fc0e015b810 2024-04-25T10:17:20.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0e6755640 1 -- 172.21.15.133:0/2792468435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e0067c70 con 0x7fc0e015b810 2024-04-25T10:17:20.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.412+0000 7fc0dd7fa640 1 -- 172.21.15.133:0/2792468435 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0c801a070 con 0x7fc0e015b810 2024-04-25T10:17:20.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.416+0000 7fc0dd7fa640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0c0066f60 0x7fc0c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.416+0000 7fc0dd7fa640 1 -- 172.21.15.133:0/2792468435 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc0c8085540 con 0x7fc0e015b810 2024-04-25T10:17:20.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.416+0000 7fc0dffff640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0c0066f60 0x7fc0c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.416+0000 7fc0dffff640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0c0066f60 0x7fc0c0069420 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00b2000 tx=0x7fc0d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.420+0000 7fc0dd7fa640 1 -- 172.21.15.133:0/2792468435 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0c8056450 con 0x7fc0e015b810 2024-04-25T10:17:20.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.588+0000 7fc0e6755640 1 -- 172.21.15.133:0/2792468435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc0e015ae00 con 0x7fc0e015b810 2024-04-25T10:17:20.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.588+0000 7fc0dd7fa640 1 -- 172.21.15.133:0/2792468435 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v183) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc0c8055d50 con 0x7fc0e015b810 2024-04-25T10:17:20.592 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:17:20.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 -- 172.21.15.133:0/2792468435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0c0066f60 msgr2=0x7fc0c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0c0066f60 0x7fc0c0069420 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00b2000 tx=0x7fc0d0008040 comp rx=0 tx=0).stop 2024-04-25T10:17:20.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 -- 172.21.15.133:0/2792468435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e015b810 msgr2=0x7fc0e00c51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e015b810 0x7fc0e00c51d0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800e960 tx=0x7fc0c800ee20 comp rx=0 tx=0).stop 2024-04-25T10:17:20.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 -- 172.21.15.133:0/2792468435 shutdown_connections 2024-04-25T10:17:20.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0e015f9c0 0x7fc0e00c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc0c0066f60 0x7fc0c0069420 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e015b810 0x7fc0e00c51d0 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 --2- 172.21.15.133:0/2792468435 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0e00cc7f0 0x7fc0e00af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 -- 172.21.15.133:0/2792468435 >> 172.21.15.133:0/2792468435 conn(0x7fc0e008af60 msgr2=0x7fc0e00ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 -- 172.21.15.133:0/2792468435 shutdown_connections 2024-04-25T10:17:20.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:20.592+0000 7fc0e6755640 1 -- 172.21.15.133:0/2792468435 wait complete. 2024-04-25T10:17:20.611 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:17:20.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:20.611 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:20.621 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[90976]: fuse finished with error 0 and tester_r 0 2024-04-25T10:17:20.625 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:17:20.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:20.625 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:17:20.793 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:17:20.793 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:20.799 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:20.799 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:17:20.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:20.799 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:17:20.847 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:17:20.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:20.848 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:17:20.848 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:20.854 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:20.854 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:17:20.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:20.855 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:17:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:17:20.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:20.902 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:17:20.902 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:20.908 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:20.908 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:17:20.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:20.908 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:17:20.955 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:17:20.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:20.956 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f964745d640 1 Processor -- start 2024-04-25T10:17:21.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f964745d640 1 -- start start 2024-04-25T10:17:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f964745d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015d9b0 0x7f96400cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f964745d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f96400cee80 0x7f964006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f964745d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96400ca9b0 con 0x7f964015dd90 2024-04-25T10:17:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f964745d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96400cbe30 con 0x7f96400cee80 2024-04-25T10:17:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f964745d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96400cac90 con 0x7f964015d9b0 2024-04-25T10:17:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96449f8640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f964015dd90 0x7f96400ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46560/0 (socket says 172.21.15.133:46560) 2024-04-25T10:17:21.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96449f8640 1 -- 172.21.15.133:0/2988900855 learned_addr learned my addr 172.21.15.133:0/2988900855 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96451f9640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015d9b0 0x7f96400cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96459fa640 1 --2- 172.21.15.133:0/2988900855 >> v2:172.21.15.162:3300/0 conn(0x7f96400cee80 0x7f964006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862349028 0 0) 0x7f96400ca9b0 con 0x7f964015dd90 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9628003530 con 0x7f964015dd90 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3515689970 0 0) 0x7f9628003530 con 0x7f964015dd90 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015d9b0 msgr2=0x7f96400cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015d9b0 0x7f96400cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 >> v2:172.21.15.162:3300/0 conn(0x7f96400cee80 msgr2=0x7f964006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 --2- 172.21.15.133:0/2988900855 >> v2:172.21.15.162:3300/0 conn(0x7f96400cee80 0x7f964006d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400caf70 con 0x7f964015dd90 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f9634001680 con 0x7f964015dd90 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 >> v1:172.21.15.133:6789/0 conn(0x7f964015dd90 legacy=0x7f96400ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96280039c0 0x7f9628003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015d9b0 0x7f9628004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9628004d50 0x7f96280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400cbe30 con 0x7f96280039c0 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400cac90 con 0x7f9628004d50 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962800b9d0 con 0x7f964015d9b0 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96451f9640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96280039c0 0x7f9628003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96449f8640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015d9b0 0x7f9628004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96459fa640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9628004d50 0x7f96280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96451f9640 1 -- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015d9b0 msgr2=0x7f9628004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96451f9640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015d9b0 0x7f9628004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96451f9640 1 -- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9628004d50 msgr2=0x7f96280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96451f9640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9628004d50 0x7f96280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96451f9640 1 -- 172.21.15.133:0/2988900855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9628003530 con 0x7f96280039c0 2024-04-25T10:17:21.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.084+0000 7f96459fa640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9628004d50 0x7f96280051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:21.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f96451f9640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96280039c0 0x7f9628003e00 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f9630009fa0 tx=0x7f963001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116567e36b79c5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96300b2070 con 0x7f96280039c0 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96300a5d10 con 0x7f96280039c0 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f963ffff640 1 -- 172.21.15.133:0/2988900855 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630009bf0 con 0x7f96280039c0 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f964745d640 1 -- 172.21.15.133:0/2988900855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96280039c0 msgr2=0x7f9628003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f964745d640 1 --2- 172.21.15.133:0/2988900855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96280039c0 0x7f9628003e00 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f9630009fa0 tx=0x7f963001cc40 comp rx=0 tx=0).stop 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f96459fa640 1 -- 172.21.15.133:0/2988900855 reap_dead start 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f964745d640 1 -- 172.21.15.133:0/2988900855 shutdown_connections 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f964745d640 1 -- 172.21.15.133:0/2988900855 >> 172.21.15.133:0/2988900855 conn(0x7f9640067000 msgr2=0x7f964006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f964745d640 1 -- 172.21.15.133:0/2988900855 shutdown_connections 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f964745d640 1 -- 172.21.15.133:0/2988900855 wait complete. 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f964745d640 1 Processor -- start 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.088+0000 7f964745d640 1 -- start start 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f964745d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96280039c0 0x7f964014d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f964745d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015b0e0 0x7f964015b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f964745d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f964015a2e0 0x7f964015a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f964745d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964006e7d0 con 0x7f964015a2e0 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f964745d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964006e650 con 0x7f96280039c0 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f964745d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964006e950 con 0x7f964015b0e0 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96459fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f964015a2e0 0x7f964015a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96459fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f964015a2e0 0x7f964015a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56744/0 (socket says 172.21.15.133:56744) 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96459fa640 1 -- 172.21.15.133:0/2662078166 learned_addr learned my addr 172.21.15.133:0/2662078166 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96451f9640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96280039c0 0x7f964014d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96449f8640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015b0e0 0x7f964015b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96459fa640 1 -- 172.21.15.133:0/2662078166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015b0e0 msgr2=0x7f964015b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96459fa640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015b0e0 0x7f964015b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96459fa640 1 -- 172.21.15.133:0/2662078166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96280039c0 msgr2=0x7f964014d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96459fa640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96280039c0 0x7f964014d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96459fa640 1 -- 172.21.15.133:0/2662078166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964014da60 con 0x7f964015a2e0 2024-04-25T10:17:21.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96451f9640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96280039c0 0x7f964014d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96459fa640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f964015a2e0 0x7f964015a790 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f964006bc70 tx=0x7f9638010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f963dffb640 1 -- 172.21.15.133:0/2662078166 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963800f040 con 0x7f964015a2e0 2024-04-25T10:17:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f963dffb640 1 -- 172.21.15.133:0/2662078166 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963800e8d0 con 0x7f964015a2e0 2024-04-25T10:17:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f964745d640 1 -- 172.21.15.133:0/2662078166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964015baa0 con 0x7f964015a2e0 2024-04-25T10:17:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f963dffb640 1 -- 172.21.15.133:0/2662078166 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963800ec80 con 0x7f964015a2e0 2024-04-25T10:17:21.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f964745d640 1 -- 172.21.15.133:0/2662078166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96401511b0 con 0x7f964015a2e0 2024-04-25T10:17:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f964745d640 1 -- 172.21.15.133:0/2662078166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9640068ef0 con 0x7f964015a2e0 2024-04-25T10:17:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f963dffb640 1 -- 172.21.15.133:0/2662078166 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96380143f0 con 0x7f964015a2e0 2024-04-25T10:17:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f963dffb640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f962c066f90 0x7f962c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96451f9640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f962c066f90 0x7f962c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f963dffb640 1 -- 172.21.15.133:0/2662078166 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9638085390 con 0x7f964015a2e0 2024-04-25T10:17:21.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.092+0000 7f96451f9640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f962c066f90 0x7f962c069450 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f96300af000 tx=0x7f9630004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.096+0000 7f963dffb640 1 -- 172.21.15.133:0/2662078166 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f964015baa0 con 0x7f964015a2e0 2024-04-25T10:17:21.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.295+0000 7f964745d640 1 -- 172.21.15.133:0/2662078166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96400cc700 con 0x7f964015a2e0 2024-04-25T10:17:21.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.295+0000 7f963dffb640 1 -- 172.21.15.133:0/2662078166 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4962 (secure 0 0 0) 0x7f9638018030 con 0x7f964015a2e0 2024-04-25T10:17:21.298 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:21.298 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":112,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1718839127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1718839127},{"type":"v1","addr":"172.21.15.162:6833","nonce":1718839127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/368827103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":368827103},{"type":"v1","addr":"172.21.15.133:6835","nonce":368827103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1274732295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1274732295},{"type":"v1","addr":"172.21.15.133:6837","nonce":1274732295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:05.607319+0000","modified":"2024-04-25T10:17:08.648334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11502":{"gid":11502,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1744425810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1744425810},{"type":"v1","addr":"172.21.15.162:6835","nonce":1744425810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:17:21.299 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 112 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 -- 172.21.15.133:0/2662078166 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f962c066f90 msgr2=0x7f962c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f962c066f90 0x7f962c069450 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f96300af000 tx=0x7f9630004ac0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 -- 172.21.15.133:0/2662078166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f964015a2e0 msgr2=0x7f964015a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f964015a2e0 0x7f964015a790 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f964006bc70 tx=0x7f9638010b70 comp rx=0 tx=0).stop 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 -- 172.21.15.133:0/2662078166 shutdown_connections 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f962c066f90 0x7f962c069450 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f964015a2e0 0x7f964015a790 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f964015b0e0 0x7f964015b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 --2- 172.21.15.133:0/2662078166 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96280039c0 0x7f964014d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 -- 172.21.15.133:0/2662078166 >> 172.21.15.133:0/2662078166 conn(0x7f9640067000 msgr2=0x7f96400ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 -- 172.21.15.133:0/2662078166 shutdown_connections 2024-04-25T10:17:21.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.299+0000 7f964745d640 1 -- 172.21.15.133:0/2662078166 wait complete. 2024-04-25T10:17:21.318 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:21.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.435+0000 7f3f6ab68640 1 Processor -- start 2024-04-25T10:17:21.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.435+0000 7f3f6ab68640 1 -- start start 2024-04-25T10:17:21.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f6ab68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6415e220 0x7f3f64164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f6ab68640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f640ce880 con 0x7f3f64159dc0 2024-04-25T10:17:21.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f6ab68640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6406e360 con 0x7f3f6415a1a0 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f6ab68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640c49e0 con 0x7f3f6415e220 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f68904640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3f64159dc0 0x7f3f640cba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46572/0 (socket says 172.21.15.133:46572) 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f68904640 1 -- 172.21.15.133:0/3559233459 learned_addr learned my addr 172.21.15.133:0/3559233459 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f69105640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6415e220 0x7f3f64164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520359791 0 0) 0x7f3f640ce880 con 0x7f3f64159dc0 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f4c003450 con 0x7f3f64159dc0 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312933485 0 0) 0x7f3f6406e360 con 0x7f3f6415a1a0 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 --> v1:172.21.15.162:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f640ce880 con 0x7f3f6415a1a0 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3873823744 0 0) 0x7f3f4c003450 con 0x7f3f64159dc0 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6415e220 msgr2=0x7f3f64164ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6415e220 0x7f3f64164ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 >> v1:172.21.15.162:6789/0 conn(0x7f3f6415a1a0 legacy=0x7f3f6415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640ceb20 con 0x7f3f64159dc0 2024-04-25T10:17:21.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f3f540016a0 con 0x7f3f64159dc0 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 >> v1:172.21.15.133:6789/0 conn(0x7f3f64159dc0 legacy=0x7f3f640cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f4c003c70 0x7f3f4c008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6415e220 0x7f3f4c00adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4c00b2f0 0x7f3f4c00d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640c49e0 con 0x7f3f4c00b2f0 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c00dcd0 con 0x7f3f4c003c70 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c00ded0 con 0x7f3f6415e220 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f69105640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4c00b2f0 0x7f3f4c00d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f63fff640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6415e220 0x7f3f4c00adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f68904640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f4c003c70 0x7f3f4c008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f69105640 1 -- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6415e220 msgr2=0x7f3f4c00adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f69105640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f6415e220 0x7f3f4c00adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f69105640 1 -- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f4c003c70 msgr2=0x7f3f4c008860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f69105640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f4c003c70 0x7f3f4c008860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f69105640 1 -- 172.21.15.133:0/3559233459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c003450 con 0x7f3f4c00b2f0 2024-04-25T10:17:21.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.439+0000 7f3f68904640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f4c003c70 0x7f3f4c008860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f69105640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4c00b2f0 0x7f3f4c00d6e0 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009f70 tx=0x7f3f5c01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8d0bc66dce6701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0b2070 con 0x7f3f4c00b2f0 2024-04-25T10:17:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c0a5cf0 con 0x7f3f4c00b2f0 2024-04-25T10:17:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f637fe640 1 -- 172.21.15.133:0/3559233459 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c009bf0 con 0x7f3f4c00b2f0 2024-04-25T10:17:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 -- 172.21.15.133:0/3559233459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4c00b2f0 msgr2=0x7f3f4c00d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 --2- 172.21.15.133:0/3559233459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f4c00b2f0 0x7f3f4c00d6e0 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009f70 tx=0x7f3f5c01cc20 comp rx=0 tx=0).stop 2024-04-25T10:17:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f69105640 1 -- 172.21.15.133:0/3559233459 reap_dead start 2024-04-25T10:17:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 -- 172.21.15.133:0/3559233459 shutdown_connections 2024-04-25T10:17:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 -- 172.21.15.133:0/3559233459 >> 172.21.15.133:0/3559233459 conn(0x7f3f64067000 msgr2=0x7f3f6406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 -- 172.21.15.133:0/3559233459 shutdown_connections 2024-04-25T10:17:21.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 -- 172.21.15.133:0/3559233459 wait complete. 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 Processor -- start 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 -- start start 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f64148910 0x7f3f641f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f64148cf0 0x7f3f64150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641506d0 0x7f3f640ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640c47c0 con 0x7f3f641506d0 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640c4640 con 0x7f3f64148cf0 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f6ab68640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f640c4cc0 con 0x7f3f64148910 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f69105640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641506d0 0x7f3f640ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f69105640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641506d0 0x7f3f640ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56756/0 (socket says 172.21.15.133:56756) 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f69105640 1 -- 172.21.15.133:0/2716310635 learned_addr learned my addr 172.21.15.133:0/2716310635 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.443+0000 7f3f68904640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f64148910 0x7f3f641f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f63fff640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f64148cf0 0x7f3f64150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f69105640 1 -- 172.21.15.133:0/2716310635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f64148910 msgr2=0x7f3f641f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f69105640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f64148910 0x7f3f641f7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f69105640 1 -- 172.21.15.133:0/2716310635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f64148cf0 msgr2=0x7f3f64150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f69105640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f64148cf0 0x7f3f64150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f69105640 1 -- 172.21.15.133:0/2716310635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f64150d30 con 0x7f3f641506d0 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f63fff640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f64148cf0 0x7f3f64150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f69105640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641506d0 0x7f3f640ca650 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0b0040 tx=0x7f3f4c003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f617fa640 1 -- 172.21.15.133:0/2716310635 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0be070 con 0x7f3f641506d0 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f617fa640 1 -- 172.21.15.133:0/2716310635 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c0b3040 con 0x7f3f641506d0 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f617fa640 1 -- 172.21.15.133:0/2716310635 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0b2050 con 0x7f3f641506d0 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f6ab68640 1 -- 172.21.15.133:0/2716310635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f64151010 con 0x7f3f641506d0 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f6ab68640 1 -- 172.21.15.133:0/2716310635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f64156cd0 con 0x7f3f641506d0 2024-04-25T10:17:21.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f6ab68640 1 -- 172.21.15.133:0/2716310635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f64068ef0 con 0x7f3f641506d0 2024-04-25T10:17:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f617fa640 1 -- 172.21.15.133:0/2716310635 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f5c0c1070 con 0x7f3f641506d0 2024-04-25T10:17:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f617fa640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f50066f60 0x7f3f50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f68904640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f50066f60 0x7f3f50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f617fa640 1 -- 172.21.15.133:0/2716310635 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3f5c11f6b0 con 0x7f3f641506d0 2024-04-25T10:17:21.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.447+0000 7f3f68904640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f50066f60 0x7f3f50069420 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3f54000c10 tx=0x7f3f54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.451+0000 7f3f617fa640 1 -- 172.21.15.133:0/2716310635 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f5c016930 con 0x7f3f641506d0 2024-04-25T10:17:21.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.611+0000 7f3f6ab68640 1 -- 172.21.15.133:0/2716310635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f640cdaf0 con 0x7f3f641506d0 2024-04-25T10:17:21.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f617fa640 1 -- 172.21.15.133:0/2716310635 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f3f5c0f0090 con 0x7f3f641506d0 2024-04-25T10:17:21.617 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:21.617 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":183,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:07.261559+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:03.841068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:04.604661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 -- 172.21.15.133:0/2716310635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f50066f60 msgr2=0x7f3f50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f50066f60 0x7f3f50069420 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3f54000c10 tx=0x7f3f54009040 comp rx=0 tx=0).stop 2024-04-25T10:17:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 -- 172.21.15.133:0/2716310635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641506d0 msgr2=0x7f3f640ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641506d0 0x7f3f640ca650 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0b0040 tx=0x7f3f4c003740 comp rx=0 tx=0).stop 2024-04-25T10:17:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 -- 172.21.15.133:0/2716310635 shutdown_connections 2024-04-25T10:17:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3f50066f60 0x7f3f50069420 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641506d0 0x7f3f640ca650 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f64148cf0 0x7f3f64150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 --2- 172.21.15.133:0/2716310635 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3f64148910 0x7f3f641f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 -- 172.21.15.133:0/2716310635 >> 172.21.15.133:0/2716310635 conn(0x7f3f64067000 msgr2=0x7f3f64159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:21.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 -- 172.21.15.133:0/2716310635 shutdown_connections 2024-04-25T10:17:21.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.615+0000 7f3f6ab68640 1 -- 172.21.15.133:0/2716310635 wait complete. 2024-04-25T10:17:21.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:03.841068+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:04.604661+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:21.637 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:21.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb348fb5640 1 Processor -- start 2024-04-25T10:17:21.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb348fb5640 1 -- start start 2024-04-25T10:17:21.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb348fb5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb34415b9d0 0x7fb3440b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb348fb5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb34406ac90 0x7fb34406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb348fb5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb34406b650 0x7fb34415f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb348fb5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34415bdb0 con 0x7fb34406ac90 2024-04-25T10:17:21.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb348fb5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3440b1e70 con 0x7fb34406b650 2024-04-25T10:17:21.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb348fb5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440b0f80 con 0x7fb34415b9d0 2024-04-25T10:17:21.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb341d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb34406ac90 0x7fb34406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb341d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb34406ac90 0x7fb34406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56762/0 (socket says 172.21.15.133:56762) 2024-04-25T10:17:21.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb341d9b640 1 -- 172.21.15.133:0/2675348758 learned_addr learned my addr 172.21.15.133:0/2675348758 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:21.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb34259c640 1 --2- 172.21.15.133:0/2675348758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb34415b9d0 0x7fb3440b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb342d9d640 1 --2- 172.21.15.133:0/2675348758 >> v2:172.21.15.162:3300/0 conn(0x7fb34406b650 0x7fb34415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb341d9b640 1 -- 172.21.15.133:0/2675348758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb34415b9d0 msgr2=0x7fb3440b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb341d9b640 1 --2- 172.21.15.133:0/2675348758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb34415b9d0 0x7fb3440b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb341d9b640 1 -- 172.21.15.133:0/2675348758 >> v2:172.21.15.162:3300/0 conn(0x7fb34406b650 msgr2=0x7fb34415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb341d9b640 1 --2- 172.21.15.133:0/2675348758 >> v2:172.21.15.162:3300/0 conn(0x7fb34406b650 0x7fb34415f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb341d9b640 1 -- 172.21.15.133:0/2675348758 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440b1b90 con 0x7fb34406ac90 2024-04-25T10:17:21.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.759+0000 7fb342d9d640 1 --2- 172.21.15.133:0/2675348758 >> v2:172.21.15.162:3300/0 conn(0x7fb34406b650 0x7fb34415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:21.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb341d9b640 1 --2- 172.21.15.133:0/2675348758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb34406ac90 0x7fb34406b110 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7fb3440c71f0 tx=0x7fb33401c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b01720c2c05d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb34159a640 1 -- 172.21.15.133:0/2675348758 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340b0070 con 0x7fb34406ac90 2024-04-25T10:17:21.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb34159a640 1 -- 172.21.15.133:0/2675348758 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3340ab600 con 0x7fb34406ac90 2024-04-25T10:17:21.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb34159a640 1 -- 172.21.15.133:0/2675348758 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340ab9d0 con 0x7fb34406ac90 2024-04-25T10:17:21.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 -- 172.21.15.133:0/2675348758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb34406ac90 msgr2=0x7fb34406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 --2- 172.21.15.133:0/2675348758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb34406ac90 0x7fb34406b110 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7fb3440c71f0 tx=0x7fb33401c1d0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 -- 172.21.15.133:0/2675348758 shutdown_connections 2024-04-25T10:17:21.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 --2- 172.21.15.133:0/2675348758 >> v2:172.21.15.162:3300/0 conn(0x7fb34406b650 0x7fb34415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 --2- 172.21.15.133:0/2675348758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb34406ac90 0x7fb34406b110 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 --2- 172.21.15.133:0/2675348758 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb34415b9d0 0x7fb3440b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 -- 172.21.15.133:0/2675348758 >> 172.21.15.133:0/2675348758 conn(0x7fb3440999d0 msgr2=0x7fb3440d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:21.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 -- 172.21.15.133:0/2675348758 shutdown_connections 2024-04-25T10:17:21.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 -- 172.21.15.133:0/2675348758 wait complete. 2024-04-25T10:17:21.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 Processor -- start 2024-04-25T10:17:21.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 -- start start 2024-04-25T10:17:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb34406ac90 0x7fb3440ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb34406b650 0x7fb3440ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb34415b9d0 0x7fb3440b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440cc430 con 0x7fb34406ac90 2024-04-25T10:17:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440cc2b0 con 0x7fb34406b650 2024-04-25T10:17:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb348fb5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440cc5b0 con 0x7fb34415b9d0 2024-04-25T10:17:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb34259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb34406ac90 0x7fb3440ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb34259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb34406ac90 0x7fb3440ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56778/0 (socket says 172.21.15.133:56778) 2024-04-25T10:17:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.763+0000 7fb34259c640 1 -- 172.21.15.133:0/12575691 learned_addr learned my addr 172.21.15.133:0/12575691 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:21.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb341d9b640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb34406b650 0x7fb3440ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb342d9d640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb34415b9d0 0x7fb3440b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb34259c640 1 -- 172.21.15.133:0/12575691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb34415b9d0 msgr2=0x7fb3440b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb34259c640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb34415b9d0 0x7fb3440b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb34259c640 1 -- 172.21.15.133:0/12575691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb34406b650 msgr2=0x7fb3440ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb34259c640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb34406b650 0x7fb3440ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb34259c640 1 -- 172.21.15.133:0/12575691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440cfa60 con 0x7fb34406ac90 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb341d9b640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb34406b650 0x7fb3440ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb34259c640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb34406ac90 0x7fb3440ce170 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7fb33800e870 tx=0x7fb33800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb332ffd640 1 -- 172.21.15.133:0/12575691 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338017070 con 0x7fb34406ac90 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb332ffd640 1 -- 172.21.15.133:0/12575691 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33800c940 con 0x7fb34406ac90 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb348fb5640 1 -- 172.21.15.133:0/12575691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3440cfce0 con 0x7fb34406ac90 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb332ffd640 1 -- 172.21.15.133:0/12575691 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800ccf0 con 0x7fb34406ac90 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb348fb5640 1 -- 172.21.15.133:0/12575691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3440cffa0 con 0x7fb34406ac90 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb348fb5640 1 -- 172.21.15.133:0/12575691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb344067c70 con 0x7fb34406ac90 2024-04-25T10:17:21.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb332ffd640 1 -- 172.21.15.133:0/12575691 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb33801a070 con 0x7fb34406ac90 2024-04-25T10:17:21.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb332ffd640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb324066f60 0x7fb324069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb332ffd640 1 -- 172.21.15.133:0/12575691 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb338085010 con 0x7fb34406ac90 2024-04-25T10:17:21.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb341d9b640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb324066f60 0x7fb324069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.767+0000 7fb341d9b640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb324066f60 0x7fb324069420 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb3340ad000 tx=0x7fb334008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.771+0000 7fb332ffd640 1 -- 172.21.15.133:0/12575691 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3440cfa60 con 0x7fb34406ac90 2024-04-25T10:17:21.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.931+0000 7fb348fb5640 1 -- 172.21.15.133:0/12575691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3440b6540 con 0x7fb34406ac90 2024-04-25T10:17:21.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.931+0000 7fb332ffd640 1 -- 172.21.15.133:0/12575691 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb338055f20 con 0x7fb34406ac90 2024-04-25T10:17:21.936 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:17:21.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 -- 172.21.15.133:0/12575691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb324066f60 msgr2=0x7fb324069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb324066f60 0x7fb324069420 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb3340ad000 tx=0x7fb334008040 comp rx=0 tx=0).stop 2024-04-25T10:17:21.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 -- 172.21.15.133:0/12575691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb34406ac90 msgr2=0x7fb3440ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb34406ac90 0x7fb3440ce170 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7fb33800e870 tx=0x7fb33800ec70 comp rx=0 tx=0).stop 2024-04-25T10:17:21.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 -- 172.21.15.133:0/12575691 shutdown_connections 2024-04-25T10:17:21.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb34415b9d0 0x7fb3440b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb34406b650 0x7fb3440ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb324066f60 0x7fb324069420 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 --2- 172.21.15.133:0/12575691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb34406ac90 0x7fb3440ce170 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 -- 172.21.15.133:0/12575691 >> 172.21.15.133:0/12575691 conn(0x7fb3440999d0 msgr2=0x7fb3440cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:21.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 -- 172.21.15.133:0/12575691 shutdown_connections 2024-04-25T10:17:21.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:21.935+0000 7fb348fb5640 1 -- 172.21.15.133:0/12575691 wait complete. 2024-04-25T10:17:21.955 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:17:22.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61df4ee640 1 Processor -- start 2024-04-25T10:17:22.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61df4ee640 1 -- start start 2024-04-25T10:17:22.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61df4ee640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f61d815d6d0 0x7f61d80b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61df4ee640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d80b72d0 con 0x7f61d80b9590 2024-04-25T10:17:22.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61df4ee640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d80b75b0 con 0x7f61d80b6dc0 2024-04-25T10:17:22.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61df4ee640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d80b7890 con 0x7f61d815d6d0 2024-04-25T10:17:22.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61dca89640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f61d80b9590 0x7f61d80d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46574/0 (socket says 172.21.15.133:46574) 2024-04-25T10:17:22.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61dca89640 1 -- 172.21.15.133:0/3701819674 learned_addr learned my addr 172.21.15.133:0/3701819674 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:22.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61dd28a640 1 --2- 172.21.15.133:0/3701819674 >> v2:172.21.15.162:3300/0 conn(0x7f61d815d6d0 0x7f61d80b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915823484 0 0) 0x7f61d80b72d0 con 0x7f61d80b9590 2024-04-25T10:17:22.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c4003530 con 0x7f61d80b9590 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1522410925 0 0) 0x7f61c4003530 con 0x7f61d80b9590 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 >> v2:172.21.15.162:3300/0 conn(0x7f61d815d6d0 msgr2=0x7f61d80b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 --2- 172.21.15.133:0/3701819674 >> v2:172.21.15.162:3300/0 conn(0x7f61d815d6d0 0x7f61d80b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 >> v1:172.21.15.162:6789/0 conn(0x7f61d80b6dc0 legacy=0x7f61d80b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d80b7b70 con 0x7f61d80b9590 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f61c8001680 con 0x7f61d80b9590 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 >> v1:172.21.15.133:6789/0 conn(0x7f61d80b9590 legacy=0x7f61d80d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 --2- 172.21.15.133:0/3701819674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c40039c0 0x7f61c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 --2- 172.21.15.133:0/3701819674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c4004810 0x7f61c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 --2- 172.21.15.133:0/3701819674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 0x7f61c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80b7890 con 0x7f61c40051d0 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c400c080 con 0x7f61c4004810 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c400c280 con 0x7f61c40039c0 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61dda8b640 1 --2- 172.21.15.133:0/3701819674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 0x7f61c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61dd28a640 1 --2- 172.21.15.133:0/3701819674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c40039c0 0x7f61c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61dda8b640 1 -- 172.21.15.133:0/3701819674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c40039c0 msgr2=0x7f61c4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:22.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61dda8b640 1 --2- 172.21.15.133:0/3701819674 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61c40039c0 0x7f61c4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61dda8b640 1 -- 172.21.15.133:0/3701819674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c4004810 msgr2=0x7f61c4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:22.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61dca89640 1 --2- 172.21.15.133:0/3701819674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c4004810 0x7f61c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61dda8b640 1 --2- 172.21.15.133:0/3701819674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61c4004810 0x7f61c4004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.083+0000 7f61dda8b640 1 -- 172.21.15.133:0/3701819674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4003530 con 0x7f61c40051d0 2024-04-25T10:17:22.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61dda8b640 1 -- 172.21.15.133:0/3701819674 reap_dead start 2024-04-25T10:17:22.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61dda8b640 1 --2- 172.21.15.133:0/3701819674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 0x7f61c400b9d0 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f61d0001780 tx=0x7f61d001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91cc3fe529bf084a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d00013a0 con 0x7f61c40051d0 2024-04-25T10:17:22.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d00a9070 con 0x7f61c40051d0 2024-04-25T10:17:22.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61cffff640 1 -- 172.21.15.133:0/3701819674 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0003600 con 0x7f61c40051d0 2024-04-25T10:17:22.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 -- 172.21.15.133:0/3701819674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 msgr2=0x7f61c400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 --2- 172.21.15.133:0/3701819674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 0x7f61c400b9d0 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f61d0001780 tx=0x7f61d001c420 comp rx=0 tx=0).stop 2024-04-25T10:17:22.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 -- 172.21.15.133:0/3701819674 shutdown_connections 2024-04-25T10:17:22.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 --2- 172.21.15.133:0/3701819674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 0x7f61c400b9d0 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 -- 172.21.15.133:0/3701819674 >> 172.21.15.133:0/3701819674 conn(0x7f61d80727c0 msgr2=0x7f61d8072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:22.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 -- 172.21.15.133:0/3701819674 shutdown_connections 2024-04-25T10:17:22.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 -- 172.21.15.133:0/3701819674 wait complete. 2024-04-25T10:17:22.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 Processor -- start 2024-04-25T10:17:22.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 -- start start 2024-04-25T10:17:22.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 0x7f61d81f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61d814dbd0 0x7f61d80cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61d80d2690 0x7f61d80d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d815dc30 con 0x7f61c40051d0 2024-04-25T10:17:22.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d815dab0 con 0x7f61d814dbd0 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61df4ee640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d815ddb0 con 0x7f61d80d2690 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61dd28a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 0x7f61d81f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61dd28a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 0x7f61d81f7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56794/0 (socket says 172.21.15.133:56794) 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61dd28a640 1 -- 172.21.15.133:0/411126323 learned_addr learned my addr 172.21.15.133:0/411126323 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.087+0000 7f61dca89640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61d814dbd0 0x7f61d80cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61dda8b640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61d80d2690 0x7f61d80d2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61dd28a640 1 -- 172.21.15.133:0/411126323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61d80d2690 msgr2=0x7f61d80d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61dd28a640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61d80d2690 0x7f61d80d2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61dd28a640 1 -- 172.21.15.133:0/411126323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61d814dbd0 msgr2=0x7f61d80cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61dd28a640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61d814dbd0 0x7f61d80cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61dd28a640 1 -- 172.21.15.133:0/411126323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d80c8070 con 0x7f61c40051d0 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61dd28a640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 0x7f61d81f7570 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009fb0 tx=0x7f61d40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61cdffb640 1 -- 172.21.15.133:0/411126323 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d401a020 con 0x7f61c40051d0 2024-04-25T10:17:22.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61cdffb640 1 -- 172.21.15.133:0/411126323 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d4016470 con 0x7f61c40051d0 2024-04-25T10:17:22.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61df4ee640 1 -- 172.21.15.133:0/411126323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d80c6fd0 con 0x7f61c40051d0 2024-04-25T10:17:22.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61cdffb640 1 -- 172.21.15.133:0/411126323 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4016860 con 0x7f61c40051d0 2024-04-25T10:17:22.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61df4ee640 1 -- 172.21.15.133:0/411126323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d80c7590 con 0x7f61c40051d0 2024-04-25T10:17:22.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61df4ee640 1 -- 172.21.15.133:0/411126323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d8068e80 con 0x7f61c40051d0 2024-04-25T10:17:22.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61cdffb640 1 -- 172.21.15.133:0/411126323 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61d401d020 con 0x7f61c40051d0 2024-04-25T10:17:22.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61cdffb640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61bc066f60 0x7f61bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61cdffb640 1 -- 172.21.15.133:0/411126323 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f61d4085f70 con 0x7f61c40051d0 2024-04-25T10:17:22.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61dca89640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61bc066f60 0x7f61bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.091+0000 7f61dca89640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61bc066f60 0x7f61bc069420 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f61c8001ec0 tx=0x7f61c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.095+0000 7f61cdffb640 1 -- 172.21.15.133:0/411126323 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61d4024030 con 0x7f61c40051d0 2024-04-25T10:17:22.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 -- 172.21.15.133:0/411126323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61d80ce7e0 con 0x7f61c40051d0 2024-04-25T10:17:22.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61cdffb640 1 -- 172.21.15.133:0/411126323 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v183) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61d4056e80 con 0x7f61c40051d0 2024-04-25T10:17:22.258 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:17:22.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 -- 172.21.15.133:0/411126323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61bc066f60 msgr2=0x7f61bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61bc066f60 0x7f61bc069420 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f61c8001ec0 tx=0x7f61c8009040 comp rx=0 tx=0).stop 2024-04-25T10:17:22.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 -- 172.21.15.133:0/411126323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 msgr2=0x7f61d81f7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 0x7f61d81f7570 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009fb0 tx=0x7f61d40115a0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 -- 172.21.15.133:0/411126323 shutdown_connections 2024-04-25T10:17:22.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61d80d2690 0x7f61d80d2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61d814dbd0 0x7f61d80cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61bc066f60 0x7f61bc069420 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 --2- 172.21.15.133:0/411126323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61c40051d0 0x7f61d81f7570 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 -- 172.21.15.133:0/411126323 >> 172.21.15.133:0/411126323 conn(0x7f61d80727c0 msgr2=0x7f61d814e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:22.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 -- 172.21.15.133:0/411126323 shutdown_connections 2024-04-25T10:17:22.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.255+0000 7f61df4ee640 1 -- 172.21.15.133:0/411126323 wait complete. 2024-04-25T10:17:22.278 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:17:22.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b560a5640 1 Processor -- start 2024-04-25T10:17:22.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b560a5640 1 -- start start 2024-04-25T10:17:22.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b560a5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2b50145170 0x7f2b5006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b560a5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2b5006e4d0 0x7f2b500cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b560a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b500cdd60 0x7f2b5015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b560a5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5006db00 con 0x7f2b5006e4d0 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b560a5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b50145550 con 0x7f2b50145170 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b560a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500ce2e0 con 0x7f2b500cdd60 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b47fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2b5006e4d0 0x7f2b500cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b47fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2b5006e4d0 0x7f2b500cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56810/0 (socket says 172.21.15.133:56810) 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b47fff640 1 -- 172.21.15.133:0/3622887714 learned_addr learned my addr 172.21.15.133:0/3622887714 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b4ffff640 1 --2- 172.21.15.133:0/3622887714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b500cdd60 0x7f2b5015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b4f7fe640 1 --2- 172.21.15.133:0/3622887714 >> v2:172.21.15.162:3300/0 conn(0x7f2b50145170 0x7f2b5006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b47fff640 1 -- 172.21.15.133:0/3622887714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b500cdd60 msgr2=0x7f2b5015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b47fff640 1 --2- 172.21.15.133:0/3622887714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b500cdd60 0x7f2b5015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b47fff640 1 -- 172.21.15.133:0/3622887714 >> v2:172.21.15.162:3300/0 conn(0x7f2b50145170 msgr2=0x7f2b5006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b47fff640 1 --2- 172.21.15.133:0/3622887714 >> v2:172.21.15.162:3300/0 conn(0x7f2b50145170 0x7f2b5006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b47fff640 1 -- 172.21.15.133:0/3622887714 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5006d4b0 con 0x7f2b5006e4d0 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b4ffff640 1 --2- 172.21.15.133:0/3622887714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b500cdd60 0x7f2b5015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:22.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b4f7fe640 1 --2- 172.21.15.133:0/3622887714 >> v2:172.21.15.162:3300/0 conn(0x7f2b50145170 0x7f2b5006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:22.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.403+0000 7f2b47fff640 1 --2- 172.21.15.133:0/3622887714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b5006e4d0 0x7f2b500cd820 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006b790 tx=0x7f2b3801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597d805f16f91ada server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b4effd640 1 -- 172.21.15.133:0/3622887714 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b380b0070 con 0x7f2b5006e4d0 2024-04-25T10:17:22.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b4effd640 1 -- 172.21.15.133:0/3622887714 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b380ab600 con 0x7f2b5006e4d0 2024-04-25T10:17:22.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b4effd640 1 -- 172.21.15.133:0/3622887714 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b380ab9d0 con 0x7f2b5006e4d0 2024-04-25T10:17:22.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 -- 172.21.15.133:0/3622887714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b5006e4d0 msgr2=0x7f2b500cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 --2- 172.21.15.133:0/3622887714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b5006e4d0 0x7f2b500cd820 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006b790 tx=0x7f2b3801c520 comp rx=0 tx=0).stop 2024-04-25T10:17:22.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 -- 172.21.15.133:0/3622887714 shutdown_connections 2024-04-25T10:17:22.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 --2- 172.21.15.133:0/3622887714 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b500cdd60 0x7f2b5015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 --2- 172.21.15.133:0/3622887714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b5006e4d0 0x7f2b500cd820 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 --2- 172.21.15.133:0/3622887714 >> v2:172.21.15.162:3300/0 conn(0x7f2b50145170 0x7f2b5006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 -- 172.21.15.133:0/3622887714 >> 172.21.15.133:0/3622887714 conn(0x7f2b50066f70 msgr2=0x7f2b5006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:22.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 -- 172.21.15.133:0/3622887714 shutdown_connections 2024-04-25T10:17:22.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 -- 172.21.15.133:0/3622887714 wait complete. 2024-04-25T10:17:22.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 Processor -- start 2024-04-25T10:17:22.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 -- start start 2024-04-25T10:17:22.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b5006e4d0 0x7f2b50152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b500cdd60 0x7f2b5013d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b50145170 0x7f2b5006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5015f690 con 0x7f2b500cdd60 2024-04-25T10:17:22.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5015f510 con 0x7f2b50145170 2024-04-25T10:17:22.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b560a5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5015f810 con 0x7f2b5006e4d0 2024-04-25T10:17:22.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b500cdd60 0x7f2b5013d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b500cdd60 0x7f2b5013d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56812/0 (socket says 172.21.15.133:56812) 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b47fff640 1 -- 172.21.15.133:0/791465448 learned_addr learned my addr 172.21.15.133:0/791465448 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b4f7fe640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b5006e4d0 0x7f2b50152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b4ffff640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b50145170 0x7f2b5006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b47fff640 1 -- 172.21.15.133:0/791465448 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b5006e4d0 msgr2=0x7f2b50152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b47fff640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b5006e4d0 0x7f2b50152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b47fff640 1 -- 172.21.15.133:0/791465448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b50145170 msgr2=0x7f2b5006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b47fff640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b50145170 0x7f2b5006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b47fff640 1 -- 172.21.15.133:0/791465448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b50152740 con 0x7f2b500cdd60 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b47fff640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b500cdd60 0x7f2b5013d670 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f2b3801cdc0 tx=0x7f2b380192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.407+0000 7f2b4cff9640 1 -- 172.21.15.133:0/791465448 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b380ba070 con 0x7f2b500cdd60 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.411+0000 7f2b4cff9640 1 -- 172.21.15.133:0/791465448 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b380b4610 con 0x7f2b500cdd60 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.411+0000 7f2b560a5640 1 -- 172.21.15.133:0/791465448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5013fd10 con 0x7f2b500cdd60 2024-04-25T10:17:22.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.411+0000 7f2b4cff9640 1 -- 172.21.15.133:0/791465448 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b380b0040 con 0x7f2b500cdd60 2024-04-25T10:17:22.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.411+0000 7f2b560a5640 1 -- 172.21.15.133:0/791465448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b501402d0 con 0x7f2b500cdd60 2024-04-25T10:17:22.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.411+0000 7f2b560a5640 1 -- 172.21.15.133:0/791465448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b50068e80 con 0x7f2b500cdd60 2024-04-25T10:17:22.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.411+0000 7f2b4cff9640 1 -- 172.21.15.133:0/791465448 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b380bd070 con 0x7f2b500cdd60 2024-04-25T10:17:22.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.411+0000 7f2b4cff9640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b30066f60 0x7f2b30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.411+0000 7f2b4cff9640 1 -- 172.21.15.133:0/791465448 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2b3811e390 con 0x7f2b500cdd60 2024-04-25T10:17:22.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.411+0000 7f2b4f7fe640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b30066f60 0x7f2b30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.411+0000 7f2b4f7fe640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b30066f60 0x7f2b30069420 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2b40006fd0 tx=0x7f2b40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.415+0000 7f2b4cff9640 1 -- 172.21.15.133:0/791465448 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b38016110 con 0x7f2b500cdd60 2024-04-25T10:17:22.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.571+0000 7f2b560a5640 1 -- 172.21.15.133:0/791465448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2b50141c10 con 0x7f2b500cdd60 2024-04-25T10:17:22.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b4cff9640 1 -- 172.21.15.133:0/791465448 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v183) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2b380ef2a0 con 0x7f2b500cdd60 2024-04-25T10:17:22.576 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 -- 172.21.15.133:0/791465448 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b30066f60 msgr2=0x7f2b30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b30066f60 0x7f2b30069420 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2b40006fd0 tx=0x7f2b40008040 comp rx=0 tx=0).stop 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 -- 172.21.15.133:0/791465448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b500cdd60 msgr2=0x7f2b5013d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b500cdd60 0x7f2b5013d670 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f2b3801cdc0 tx=0x7f2b380192c0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 -- 172.21.15.133:0/791465448 shutdown_connections 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b50145170 0x7f2b5006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2b30066f60 0x7f2b30069420 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b500cdd60 0x7f2b5013d670 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 --2- 172.21.15.133:0/791465448 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2b5006e4d0 0x7f2b50152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 -- 172.21.15.133:0/791465448 >> 172.21.15.133:0/791465448 conn(0x7f2b50066f70 msgr2=0x7f2b5013eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 -- 172.21.15.133:0/791465448 shutdown_connections 2024-04-25T10:17:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.575+0000 7f2b560a5640 1 -- 172.21.15.133:0/791465448 wait complete. 2024-04-25T10:17:22.596 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:22.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76abb6b640 1 Processor -- start 2024-04-25T10:17:22.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76abb6b640 1 -- start start 2024-04-25T10:17:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76abb6b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f76a40d0770 0x7f76a40d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76abb6b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40ceb20 con 0x7f76a415db00 2024-04-25T10:17:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76abb6b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40b30a0 con 0x7f76a40b87e0 2024-04-25T10:17:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76abb6b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a40d1110 con 0x7f76a40d0770 2024-04-25T10:17:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76a9106640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f76a40d0770 0x7f76a40d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76a9106640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f76a40d0770 0x7f76a40d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56818/0 (socket says 172.21.15.133:56818) 2024-04-25T10:17:22.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76a9907640 1 --1- 172.21.15.133:0/494366877 >> v1:172.21.15.133:6789/0 conn(0x7f76a415db00 0x7f76a40b2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46588/0 (socket says 172.21.15.133:46588) 2024-04-25T10:17:22.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76a9106640 1 -- 172.21.15.133:0/494366877 learned_addr learned my addr 172.21.15.133:0/494366877 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:22.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76a9106640 1 -- 172.21.15.133:0/494366877 >> v1:172.21.15.162:6789/0 conn(0x7f76a40b87e0 legacy=0x7f76a40b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:22.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76a9106640 1 -- 172.21.15.133:0/494366877 >> v1:172.21.15.133:6789/0 conn(0x7f76a415db00 legacy=0x7f76a40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76a8905640 1 -- 172.21.15.133:0/494366877 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674939417 0 0) 0x7f76a40ceb20 con 0x7f76a415db00 2024-04-25T10:17:22.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.707+0000 7f76a9106640 1 -- 172.21.15.133:0/494366877 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a40b7740 con 0x7f76a40d0770 2024-04-25T10:17:22.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76a9106640 1 --2- 172.21.15.133:0/494366877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a40d0770 0x7f76a40d0bd0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f7694009870 tx=0x7f769401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa4a9c9367dfd63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76a8905640 1 -- 172.21.15.133:0/494366877 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76940a95f0 con 0x7f76a40d0770 2024-04-25T10:17:22.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76a8905640 1 -- 172.21.15.133:0/494366877 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76940a9790 con 0x7f76a40d0770 2024-04-25T10:17:22.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76a8905640 1 -- 172.21.15.133:0/494366877 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76940a9b60 con 0x7f76a40d0770 2024-04-25T10:17:22.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 -- 172.21.15.133:0/494366877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a40d0770 msgr2=0x7f76a40d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 --2- 172.21.15.133:0/494366877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a40d0770 0x7f76a40d0bd0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f7694009870 tx=0x7f769401c530 comp rx=0 tx=0).stop 2024-04-25T10:17:22.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 -- 172.21.15.133:0/494366877 shutdown_connections 2024-04-25T10:17:22.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 --2- 172.21.15.133:0/494366877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a40d0770 0x7f76a40d0bd0 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 -- 172.21.15.133:0/494366877 >> 172.21.15.133:0/494366877 conn(0x7f76a40727c0 msgr2=0x7f76a4072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:22.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 -- 172.21.15.133:0/494366877 shutdown_connections 2024-04-25T10:17:22.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 -- 172.21.15.133:0/494366877 wait complete. 2024-04-25T10:17:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 Processor -- start 2024-04-25T10:17:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 -- start start 2024-04-25T10:17:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76a40b87e0 0x7f76a415df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a40d0770 0x7f76a4154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a415db00 0x7f76a41605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40b82c0 con 0x7f76a415db00 2024-04-25T10:17:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40b7fe0 con 0x7f76a40b87e0 2024-04-25T10:17:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40b7430 con 0x7f76a40d0770 2024-04-25T10:17:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76aa108640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a415db00 0x7f76a41605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76aa108640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a415db00 0x7f76a41605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56822/0 (socket says 172.21.15.133:56822) 2024-04-25T10:17:22.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76aa108640 1 -- 172.21.15.133:0/3279578849 learned_addr learned my addr 172.21.15.133:0/3279578849 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76a9106640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a40d0770 0x7f76a4154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76aa108640 1 -- 172.21.15.133:0/3279578849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a40d0770 msgr2=0x7f76a4154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76aa108640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a40d0770 0x7f76a4154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76aa108640 1 -- 172.21.15.133:0/3279578849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76a40b87e0 msgr2=0x7f76a415df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76aa108640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76a40b87e0 0x7f76a415df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76aa108640 1 -- 172.21.15.133:0/3279578849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a41f9940 con 0x7f76a415db00 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76a9106640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a40d0770 0x7f76a4154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76aa108640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a415db00 0x7f76a41605d0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c770 tx=0x7f769c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f769a7fc640 1 -- 172.21.15.133:0/3279578849 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c003b80 con 0x7f76a415db00 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f769a7fc640 1 -- 172.21.15.133:0/3279578849 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c003d20 con 0x7f76a415db00 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 -- 172.21.15.133:0/3279578849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a41f9c20 con 0x7f76a415db00 2024-04-25T10:17:22.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f769a7fc640 1 -- 172.21.15.133:0/3279578849 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00aa90 con 0x7f76a415db00 2024-04-25T10:17:22.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.711+0000 7f76abb6b640 1 -- 172.21.15.133:0/3279578849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a41fa1e0 con 0x7f76a415db00 2024-04-25T10:17:22.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.715+0000 7f76abb6b640 1 -- 172.21.15.133:0/3279578849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a4068ef0 con 0x7f76a415db00 2024-04-25T10:17:22.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.715+0000 7f769a7fc640 1 -- 172.21.15.133:0/3279578849 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f769c00ac30 con 0x7f76a415db00 2024-04-25T10:17:22.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.715+0000 7f769a7fc640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f60 0x7f7688069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.715+0000 7f76a9907640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f60 0x7f7688069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.715+0000 7f769a7fc640 1 -- 172.21.15.133:0/3279578849 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f769c085140 con 0x7f76a415db00 2024-04-25T10:17:22.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.715+0000 7f76a9907640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f60 0x7f7688069420 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000c10 tx=0x7f76a00079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.719+0000 7f769a7fc640 1 -- 172.21.15.133:0/3279578849 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76a40b30a0 con 0x7f76a415db00 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.903+0000 7f76abb6b640 1 -- 172.21.15.133:0/3279578849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76a40b2750 con 0x7f76a415db00 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.903+0000 7f769a7fc640 1 -- 172.21.15.133:0/3279578849 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4962 (secure 0 0 0) 0x7f769c055fc0 con 0x7f76a415db00 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:22.908 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":112,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1718839127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1718839127},{"type":"v1","addr":"172.21.15.162:6833","nonce":1718839127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/368827103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":368827103},{"type":"v1","addr":"172.21.15.133:6835","nonce":368827103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1274732295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1274732295},{"type":"v1","addr":"172.21.15.133:6837","nonce":1274732295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:05.607319+0000","modified":"2024-04-25T10:17:08.648334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11502":{"gid":11502,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1744425810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1744425810},{"type":"v1","addr":"172.21.15.162:6835","nonce":1744425810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:17:22.908 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 112 2024-04-25T10:17:22.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 -- 172.21.15.133:0/3279578849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f60 msgr2=0x7f7688069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f60 0x7f7688069420 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000c10 tx=0x7f76a00079d0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 -- 172.21.15.133:0/3279578849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a415db00 msgr2=0x7f76a41605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a415db00 0x7f76a41605d0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c770 tx=0x7f769c00cc30 comp rx=0 tx=0).stop 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 -- 172.21.15.133:0/3279578849 shutdown_connections 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7688066f60 0x7f7688069420 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a415db00 0x7f76a41605d0 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f76a40d0770 0x7f76a4154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 --2- 172.21.15.133:0/3279578849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f76a40b87e0 0x7f76a415df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 -- 172.21.15.133:0/3279578849 >> 172.21.15.133:0/3279578849 conn(0x7f76a40727c0 msgr2=0x7f76a41580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 -- 172.21.15.133:0/3279578849 shutdown_connections 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:22.907+0000 7f76abb6b640 1 -- 172.21.15.133:0/3279578849 wait complete. 2024-04-25T10:17:22.925 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.027+0000 7eff4ba76640 1 Processor -- start 2024-04-25T10:17:23.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.027+0000 7eff4ba76640 1 -- start start 2024-04-25T10:17:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff4ba76640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff4415c9a0 0x7eff440c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff4ba76640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff440b1f70 0x7eff4406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff4ba76640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff440c61d0 con 0x7eff440ce330 2024-04-25T10:17:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff4ba76640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff440c64b0 con 0x7eff4415c9a0 2024-04-25T10:17:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff4ba76640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4406d4f0 con 0x7eff440b1f70 2024-04-25T10:17:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff49812640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff4415c9a0 0x7eff440c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff49812640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff4415c9a0 0x7eff440c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56838/0 (socket says 172.21.15.133:56838) 2024-04-25T10:17:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff49812640 1 -- 172.21.15.133:0/3427679134 learned_addr learned my addr 172.21.15.133:0/3427679134 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff49011640 1 --2- 172.21.15.133:0/3427679134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff440b1f70 0x7eff4406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff49812640 1 -- 172.21.15.133:0/3427679134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff440b1f70 msgr2=0x7eff4406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff49812640 1 --2- 172.21.15.133:0/3427679134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff440b1f70 0x7eff4406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff49812640 1 -- 172.21.15.133:0/3427679134 >> v1:172.21.15.162:6789/0 conn(0x7eff440ce330 legacy=0x7eff4415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff49812640 1 -- 172.21.15.133:0/3427679134 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4406d7d0 con 0x7eff4415c9a0 2024-04-25T10:17:23.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff49812640 1 --2- 172.21.15.133:0/3427679134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff4415c9a0 0x7eff440c5980 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7eff40009870 tx=0x7eff4001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313aff0deb9299d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff48810640 1 -- 172.21.15.133:0/3427679134 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff400ab610 con 0x7eff4415c9a0 2024-04-25T10:17:23.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff48810640 1 -- 172.21.15.133:0/3427679134 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff400b0070 con 0x7eff4415c9a0 2024-04-25T10:17:23.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.031+0000 7eff48810640 1 -- 172.21.15.133:0/3427679134 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff400ab9d0 con 0x7eff4415c9a0 2024-04-25T10:17:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 -- 172.21.15.133:0/3427679134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff4415c9a0 msgr2=0x7eff440c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 --2- 172.21.15.133:0/3427679134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff4415c9a0 0x7eff440c5980 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7eff40009870 tx=0x7eff4001c500 comp rx=0 tx=0).stop 2024-04-25T10:17:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 -- 172.21.15.133:0/3427679134 shutdown_connections 2024-04-25T10:17:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 --2- 172.21.15.133:0/3427679134 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff440b1f70 0x7eff4406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 --2- 172.21.15.133:0/3427679134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff4415c9a0 0x7eff440c5980 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 -- 172.21.15.133:0/3427679134 >> 172.21.15.133:0/3427679134 conn(0x7eff44067000 msgr2=0x7eff4406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 -- 172.21.15.133:0/3427679134 shutdown_connections 2024-04-25T10:17:23.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 -- 172.21.15.133:0/3427679134 wait complete. 2024-04-25T10:17:23.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 Processor -- start 2024-04-25T10:17:23.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 -- start start 2024-04-25T10:17:23.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff440b1f70 0x7eff4415e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff440ce330 0x7eff44155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff4415c9a0 0x7eff44154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff440ceb50 con 0x7eff440b1f70 2024-04-25T10:17:23.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4415cd80 con 0x7eff440ce330 2024-04-25T10:17:23.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff44154090 con 0x7eff4415c9a0 2024-04-25T10:17:23.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff49812640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff440b1f70 0x7eff4415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff49812640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff440b1f70 0x7eff4415e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56852/0 (socket says 172.21.15.133:56852) 2024-04-25T10:17:23.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff49812640 1 -- 172.21.15.133:0/2395741196 learned_addr learned my addr 172.21.15.133:0/2395741196 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:23.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff49011640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff440ce330 0x7eff44155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4a013640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff4415c9a0 0x7eff44154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff49812640 1 -- 172.21.15.133:0/2395741196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff4415c9a0 msgr2=0x7eff44154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff49812640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff4415c9a0 0x7eff44154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff49812640 1 -- 172.21.15.133:0/2395741196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff440ce330 msgr2=0x7eff44155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff49812640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff440ce330 0x7eff44155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff49812640 1 -- 172.21.15.133:0/2395741196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44150100 con 0x7eff440b1f70 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff49011640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff440ce330 0x7eff44155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff49812640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff440b1f70 0x7eff4415e390 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7eff400ad040 tx=0x7eff4001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff3a7fc640 1 -- 172.21.15.133:0/2395741196 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff400be5f0 con 0x7eff440b1f70 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff3a7fc640 1 -- 172.21.15.133:0/2395741196 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff400be7b0 con 0x7eff440b1f70 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 -- 172.21.15.133:0/2395741196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4415e970 con 0x7eff440b1f70 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff3a7fc640 1 -- 172.21.15.133:0/2395741196 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff400beba0 con 0x7eff440b1f70 2024-04-25T10:17:23.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.035+0000 7eff4ba76640 1 -- 172.21.15.133:0/2395741196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff441545c0 con 0x7eff440b1f70 2024-04-25T10:17:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.039+0000 7eff4ba76640 1 -- 172.21.15.133:0/2395741196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4406bc20 con 0x7eff440b1f70 2024-04-25T10:17:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.039+0000 7eff3a7fc640 1 -- 172.21.15.133:0/2395741196 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff400b43f0 con 0x7eff440b1f70 2024-04-25T10:17:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.039+0000 7eff3a7fc640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff28066f60 0x7eff28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.039+0000 7eff3a7fc640 1 -- 172.21.15.133:0/2395741196 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7eff4011fd80 con 0x7eff440b1f70 2024-04-25T10:17:23.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.039+0000 7eff49011640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff28066f60 0x7eff28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.039+0000 7eff49011640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff28066f60 0x7eff28069420 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7eff440cc000 tx=0x7eff34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.043+0000 7eff3a7fc640 1 -- 172.21.15.133:0/2395741196 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff400c4060 con 0x7eff440b1f70 2024-04-25T10:17:23.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.191+0000 7eff4ba76640 1 -- 172.21.15.133:0/2395741196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff44155300 con 0x7eff440b1f70 2024-04-25T10:17:23.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff3a7fc640 1 -- 172.21.15.133:0/2395741196 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16976 (secure 0 0 0) 0x7eff400a8050 con 0x7eff440b1f70 2024-04-25T10:17:23.197 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:23.197 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":183,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:07.261559+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:03.841068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:04.604661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 -- 172.21.15.133:0/2395741196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff28066f60 msgr2=0x7eff28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff28066f60 0x7eff28069420 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7eff440cc000 tx=0x7eff34008040 comp rx=0 tx=0).stop 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 -- 172.21.15.133:0/2395741196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff440b1f70 msgr2=0x7eff4415e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff440b1f70 0x7eff4415e390 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7eff400ad040 tx=0x7eff4001ada0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 -- 172.21.15.133:0/2395741196 shutdown_connections 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff4415c9a0 0x7eff44154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff440ce330 0x7eff44155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff28066f60 0x7eff28069420 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 --2- 172.21.15.133:0/2395741196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff440b1f70 0x7eff4415e390 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 -- 172.21.15.133:0/2395741196 >> 172.21.15.133:0/2395741196 conn(0x7eff44067000 msgr2=0x7eff441537c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 -- 172.21.15.133:0/2395741196 shutdown_connections 2024-04-25T10:17:23.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.195+0000 7eff4ba76640 1 -- 172.21.15.133:0/2395741196 wait complete. 2024-04-25T10:17:23.215 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:17:23.216 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:23.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dc21d0640 1 Processor -- start 2024-04-25T10:17:23.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dc21d0640 1 -- start start 2024-04-25T10:17:23.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dc21d0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f2dbc0ce330 0x7f2dbc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dc21d0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc0c61d0 con 0x7f2dbc15c9a0 2024-04-25T10:17:23.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dc21d0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc0c64b0 con 0x7f2dbc0b1f70 2024-04-25T10:17:23.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dc21d0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dbc0cecc0 con 0x7f2dbc0ce330 2024-04-25T10:17:23.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dbb7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2dbc15c9a0 0x7f2dbc0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46590/0 (socket says 172.21.15.133:46590) 2024-04-25T10:17:23.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dbb7fe640 1 -- 172.21.15.133:0/2132615345 learned_addr learned my addr 172.21.15.133:0/2132615345 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:23.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/2132615345 >> v2:172.21.15.162:3300/0 conn(0x7f2dbc0ce330 0x7f2dbc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616516596 0 0) 0x7f2dbc0c61d0 con 0x7f2dbc15c9a0 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2da4003530 con 0x7f2dbc15c9a0 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1923265599 0 0) 0x7f2da4003530 con 0x7f2dbc15c9a0 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 >> v2:172.21.15.162:3300/0 conn(0x7f2dbc0ce330 msgr2=0x7f2dbc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 --2- 172.21.15.133:0/2132615345 >> v2:172.21.15.162:3300/0 conn(0x7f2dbc0ce330 0x7f2dbc0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 >> v1:172.21.15.162:6789/0 conn(0x7f2dbc0b1f70 legacy=0x7f2dbc06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc06d4f0 con 0x7f2dbc15c9a0 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2dac001680 con 0x7f2dbc15c9a0 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 >> v1:172.21.15.133:6789/0 conn(0x7f2dbc15c9a0 legacy=0x7f2dbc0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 --2- 172.21.15.133:0/2132615345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da40039c0 0x7f2da4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 --2- 172.21.15.133:0/2132615345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2da4004810 0x7f2da4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 --2- 172.21.15.133:0/2132615345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da40051d0 0x7f2da400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0cecc0 con 0x7f2da40051d0 2024-04-25T10:17:23.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da400c070 con 0x7f2da40039c0 2024-04-25T10:17:23.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da400c270 con 0x7f2da4004810 2024-04-25T10:17:23.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/2132615345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da40051d0 0x7f2da400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/2132615345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da40039c0 0x7f2da4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dbaffd640 1 --2- 172.21.15.133:0/2132615345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2da4004810 0x7f2da4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dbbfff640 1 -- 172.21.15.133:0/2132615345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2da4004810 msgr2=0x7f2da4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:23.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/2132615345 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2da4004810 0x7f2da4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dbbfff640 1 -- 172.21.15.133:0/2132615345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da40039c0 msgr2=0x7f2da4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:23.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/2132615345 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da40039c0 0x7f2da4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.331+0000 7f2dbbfff640 1 -- 172.21.15.133:0/2132615345 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da4003530 con 0x7f2da40051d0 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbbfff640 1 -- 172.21.15.133:0/2132615345 reap_dead start 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/2132615345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da40051d0 0x7f2da400b9d0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009fc0 tx=0x7f2db401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0b6909480eb43d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4009c10 con 0x7f2da40051d0 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db40a5d10 con 0x7f2da40051d0 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dba7fc640 1 -- 172.21.15.133:0/2132615345 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4003be0 con 0x7f2da40051d0 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 -- 172.21.15.133:0/2132615345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da40051d0 msgr2=0x7f2da400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 --2- 172.21.15.133:0/2132615345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da40051d0 0x7f2da400b9d0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009fc0 tx=0x7f2db401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 -- 172.21.15.133:0/2132615345 shutdown_connections 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 --2- 172.21.15.133:0/2132615345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2da40051d0 0x7f2da400b9d0 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 -- 172.21.15.133:0/2132615345 >> 172.21.15.133:0/2132615345 conn(0x7f2dbc067000 msgr2=0x7f2dbc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 -- 172.21.15.133:0/2132615345 shutdown_connections 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 -- 172.21.15.133:0/2132615345 wait complete. 2024-04-25T10:17:23.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 Processor -- start 2024-04-25T10:17:23.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 -- start start 2024-04-25T10:17:23.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da40051d0 0x7f2dbc1f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dbc139f10 0x7f2dbc1ee960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc0b2350 0x7f2dbc0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc15cf00 con 0x7f2dbc0b2350 2024-04-25T10:17:23.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc15cd80 con 0x7f2da40051d0 2024-04-25T10:17:23.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dc21d0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0ce780 con 0x7f2dbc139f10 2024-04-25T10:17:23.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc0b2350 0x7f2dbc0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc0b2350 0x7f2dbc0b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56868/0 (socket says 172.21.15.133:56868) 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbbfff640 1 -- 172.21.15.133:0/1261774826 learned_addr learned my addr 172.21.15.133:0/1261774826 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbaffd640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dbc139f10 0x7f2dbc1ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da40051d0 0x7f2dbc1f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbbfff640 1 -- 172.21.15.133:0/1261774826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dbc139f10 msgr2=0x7f2dbc1ee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dbc139f10 0x7f2dbc1ee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbbfff640 1 -- 172.21.15.133:0/1261774826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da40051d0 msgr2=0x7f2dbc1f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da40051d0 0x7f2dbc1f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbbfff640 1 -- 172.21.15.133:0/1261774826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc1f2570 con 0x7f2dbc0b2350 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.335+0000 7f2dbaffd640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dbc139f10 0x7f2dbc1ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2dbbfff640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc0b2350 0x7f2dbc0b27d0 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f2db401d1a0 tx=0x7f2db4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2da3fff640 1 -- 172.21.15.133:0/1261774826 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40aed80 con 0x7f2dbc0b2350 2024-04-25T10:17:23.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2da3fff640 1 -- 172.21.15.133:0/1261774826 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db40bd070 con 0x7f2dbc0b2350 2024-04-25T10:17:23.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2dc21d0640 1 -- 172.21.15.133:0/1261774826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc139190 con 0x7f2dbc0b2350 2024-04-25T10:17:23.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2da3fff640 1 -- 172.21.15.133:0/1261774826 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4003ba0 con 0x7f2dbc0b2350 2024-04-25T10:17:23.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2dc21d0640 1 -- 172.21.15.133:0/1261774826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc137cd0 con 0x7f2dbc0b2350 2024-04-25T10:17:23.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2dc21d0640 1 -- 172.21.15.133:0/1261774826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc068ef0 con 0x7f2dbc0b2350 2024-04-25T10:17:23.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2da3fff640 1 -- 172.21.15.133:0/1261774826 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2db40a5d10 con 0x7f2dbc0b2350 2024-04-25T10:17:23.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2da3fff640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2da8066f90 0x7f2da8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2da3fff640 1 -- 172.21.15.133:0/1261774826 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2db4120ae0 con 0x7f2dbc0b2350 2024-04-25T10:17:23.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2da8066f90 0x7f2da8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.339+0000 7f2dbb7fe640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2da8066f90 0x7f2da8069450 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2dac001ec0 tx=0x7f2dac000f20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.343+0000 7f2da3fff640 1 -- 172.21.15.133:0/1261774826 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2db40c7020 con 0x7f2dbc0b2350 2024-04-25T10:17:23.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.539+0000 7f2dc21d0640 1 -- 172.21.15.133:0/1261774826 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dbc15e090 con 0x7f2dbc0b2350 2024-04-25T10:17:23.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.539+0000 7f2da3fff640 1 -- 172.21.15.133:0/1261774826 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4962 (secure 0 0 0) 0x7f2dbc1f2570 con 0x7f2dbc0b2350 2024-04-25T10:17:23.544 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:23.544 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":112,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1718839127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1718839127},{"type":"v1","addr":"172.21.15.162:6833","nonce":1718839127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/368827103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":368827103},{"type":"v1","addr":"172.21.15.133:6835","nonce":368827103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1274732295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1274732295},{"type":"v1","addr":"172.21.15.133:6837","nonce":1274732295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:05.607319+0000","modified":"2024-04-25T10:17:08.648334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11502":{"gid":11502,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1744425810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1744425810},{"type":"v1","addr":"172.21.15.162:6835","nonce":1744425810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:17:23.544 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 112 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 -- 172.21.15.133:0/1261774826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2da8066f90 msgr2=0x7f2da8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2da8066f90 0x7f2da8069450 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2dac001ec0 tx=0x7f2dac000f20 comp rx=0 tx=0).stop 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 -- 172.21.15.133:0/1261774826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc0b2350 msgr2=0x7f2dbc0b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc0b2350 0x7f2dbc0b27d0 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f2db401d1a0 tx=0x7f2db4009750 comp rx=0 tx=0).stop 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 -- 172.21.15.133:0/1261774826 shutdown_connections 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2da8066f90 0x7f2da8069450 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dbc0b2350 0x7f2dbc0b27d0 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2dbc139f10 0x7f2dbc1ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 --2- 172.21.15.133:0/1261774826 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2da40051d0 0x7f2dbc1f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 -- 172.21.15.133:0/1261774826 >> 172.21.15.133:0/1261774826 conn(0x7f2dbc067000 msgr2=0x7f2dbc15b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 -- 172.21.15.133:0/1261774826 shutdown_connections 2024-04-25T10:17:23.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.543+0000 7f2dc21d0640 1 -- 172.21.15.133:0/1261774826 wait complete. 2024-04-25T10:17:23.563 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:23.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f6b5a640 1 Processor -- start 2024-04-25T10:17:23.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f6b5a640 1 -- start start 2024-04-25T10:17:23.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f6b5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f015c9a0 0x7f60f00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f6b5a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f60f00ce330 0x7f60f00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f6b5a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f00c61d0 con 0x7f60f00b1f70 2024-04-25T10:17:23.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f6b5a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f00cece0 con 0x7f60f00ce330 2024-04-25T10:17:23.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f6b5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00c64b0 con 0x7f60f015c9a0 2024-04-25T10:17:23.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f50f7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f60f00ce330 0x7f60f00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60effff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f60f00b1f70 0x7f60f006dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46596/0 (socket says 172.21.15.133:46596) 2024-04-25T10:17:23.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60effff640 1 -- 172.21.15.133:0/3107990860 learned_addr learned my addr 172.21.15.133:0/3107990860 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:23.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f48f6640 1 --2- 172.21.15.133:0/3107990860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f015c9a0 0x7f60f00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f50f7640 1 -- 172.21.15.133:0/3107990860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f015c9a0 msgr2=0x7f60f00c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:23.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f50f7640 1 --2- 172.21.15.133:0/3107990860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f015c9a0 0x7f60f00c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f50f7640 1 -- 172.21.15.133:0/3107990860 >> v1:172.21.15.133:6789/0 conn(0x7f60f00b1f70 legacy=0x7f60f006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60ef7fe640 1 -- 172.21.15.133:0/3107990860 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172880612 0 0) 0x7f60f00c61d0 con 0x7f60f00b1f70 2024-04-25T10:17:23.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.679+0000 7f60f50f7640 1 -- 172.21.15.133:0/3107990860 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f006d4f0 con 0x7f60f00ce330 2024-04-25T10:17:23.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f50f7640 1 --2- 172.21.15.133:0/3107990860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60f00ce330 0x7f60f00ce7a0 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009d90 tx=0x7f60e801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a64e095e037b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60ef7fe640 1 -- 172.21.15.133:0/3107990860 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e80b2070 con 0x7f60f00ce330 2024-04-25T10:17:23.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60ef7fe640 1 -- 172.21.15.133:0/3107990860 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e80a58e0 con 0x7f60f00ce330 2024-04-25T10:17:23.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60ef7fe640 1 -- 172.21.15.133:0/3107990860 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e80a5cb0 con 0x7f60f00ce330 2024-04-25T10:17:23.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 -- 172.21.15.133:0/3107990860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60f00ce330 msgr2=0x7f60f00ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 --2- 172.21.15.133:0/3107990860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60f00ce330 0x7f60f00ce7a0 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009d90 tx=0x7f60e801c970 comp rx=0 tx=0).stop 2024-04-25T10:17:23.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 -- 172.21.15.133:0/3107990860 shutdown_connections 2024-04-25T10:17:23.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 --2- 172.21.15.133:0/3107990860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60f00ce330 0x7f60f00ce7a0 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 --2- 172.21.15.133:0/3107990860 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f015c9a0 0x7f60f00c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 -- 172.21.15.133:0/3107990860 >> 172.21.15.133:0/3107990860 conn(0x7f60f0067000 msgr2=0x7f60f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:23.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 -- 172.21.15.133:0/3107990860 shutdown_connections 2024-04-25T10:17:23.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 -- 172.21.15.133:0/3107990860 wait complete. 2024-04-25T10:17:23.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 Processor -- start 2024-04-25T10:17:23.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 -- start start 2024-04-25T10:17:23.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00b1f70 0x7f60f0147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60f00ce330 0x7f60f0155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f015c9a0 0x7f60f00b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f015dad0 con 0x7f60f00ce330 2024-04-25T10:17:23.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f015d950 con 0x7f60f00b1f70 2024-04-25T10:17:23.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f6b5a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f015dc50 con 0x7f60f015c9a0 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60effff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60f00ce330 0x7f60f0155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60effff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60f00ce330 0x7f60f0155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56878/0 (socket says 172.21.15.133:56878) 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60effff640 1 -- 172.21.15.133:0/2420422642 learned_addr learned my addr 172.21.15.133:0/2420422642 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f48f6640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00b1f70 0x7f60f0147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f50f7640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f015c9a0 0x7f60f00b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60effff640 1 -- 172.21.15.133:0/2420422642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f015c9a0 msgr2=0x7f60f00b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60effff640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f015c9a0 0x7f60f00b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60effff640 1 -- 172.21.15.133:0/2420422642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00b1f70 msgr2=0x7f60f0147200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60effff640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00b1f70 0x7f60f0147200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60effff640 1 -- 172.21.15.133:0/2420422642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f0156d10 con 0x7f60f00ce330 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f50f7640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f015c9a0 0x7f60f00b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60f48f6640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00b1f70 0x7f60f0147200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60effff640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60f00ce330 0x7f60f0155b40 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00c410 tx=0x7f60dc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.683+0000 7f60ed7fa640 1 -- 172.21.15.133:0/2420422642 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0074e0 con 0x7f60f00ce330 2024-04-25T10:17:23.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.687+0000 7f60ed7fa640 1 -- 172.21.15.133:0/2420422642 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc0047f0 con 0x7f60f00ce330 2024-04-25T10:17:23.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.687+0000 7f60f6b5a640 1 -- 172.21.15.133:0/2420422642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f0155060 con 0x7f60f00ce330 2024-04-25T10:17:23.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.687+0000 7f60ed7fa640 1 -- 172.21.15.133:0/2420422642 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc007680 con 0x7f60f00ce330 2024-04-25T10:17:23.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.687+0000 7f60f6b5a640 1 -- 172.21.15.133:0/2420422642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f0155620 con 0x7f60f00ce330 2024-04-25T10:17:23.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.687+0000 7f60f6b5a640 1 -- 172.21.15.133:0/2420422642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f006bc20 con 0x7f60f00ce330 2024-04-25T10:17:23.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.687+0000 7f60ed7fa640 1 -- 172.21.15.133:0/2420422642 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60dc011070 con 0x7f60f00ce330 2024-04-25T10:17:23.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.687+0000 7f60ed7fa640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60d4066f60 0x7f60d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.687+0000 7f60f48f6640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60d4066f60 0x7f60d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.687+0000 7f60ed7fa640 1 -- 172.21.15.133:0/2420422642 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f60dc084e40 con 0x7f60f00ce330 2024-04-25T10:17:23.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.687+0000 7f60f48f6640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60d4066f60 0x7f60d4069420 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009840 tx=0x7f60e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.691+0000 7f60ed7fa640 1 -- 172.21.15.133:0/2420422642 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60dc055cc0 con 0x7f60f00ce330 2024-04-25T10:17:23.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.883+0000 7f60f6b5a640 1 -- 172.21.15.133:0/2420422642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60f0068ef0 con 0x7f60f00ce330 2024-04-25T10:17:23.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60ed7fa640 1 -- 172.21.15.133:0/2420422642 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4962 (secure 0 0 0) 0x7f60dc0555c0 con 0x7f60f00ce330 2024-04-25T10:17:23.889 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:23.889 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":112,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1718839127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1718839127},{"type":"v1","addr":"172.21.15.162:6833","nonce":1718839127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/368827103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":368827103},{"type":"v1","addr":"172.21.15.133:6835","nonce":368827103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1274732295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1274732295},{"type":"v1","addr":"172.21.15.133:6837","nonce":1274732295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:05.607319+0000","modified":"2024-04-25T10:17:08.648334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11502":{"gid":11502,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.162:6835/1744425810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1744425810},{"type":"v1","addr":"172.21.15.162:6835","nonce":1744425810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:17:23.889 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 112 2024-04-25T10:17:23.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 -- 172.21.15.133:0/2420422642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60d4066f60 msgr2=0x7f60d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60d4066f60 0x7f60d4069420 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009840 tx=0x7f60e4008040 comp rx=0 tx=0).stop 2024-04-25T10:17:23.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 -- 172.21.15.133:0/2420422642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60f00ce330 msgr2=0x7f60f0155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60f00ce330 0x7f60f0155b40 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00c410 tx=0x7f60dc00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 -- 172.21.15.133:0/2420422642 shutdown_connections 2024-04-25T10:17:23.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60f015c9a0 0x7f60f00b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60d4066f60 0x7f60d4069420 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60f00ce330 0x7f60f0155b40 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 --2- 172.21.15.133:0/2420422642 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60f00b1f70 0x7f60f0147200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 -- 172.21.15.133:0/2420422642 >> 172.21.15.133:0/2420422642 conn(0x7f60f0067000 msgr2=0x7f60f015e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:23.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 -- 172.21.15.133:0/2420422642 shutdown_connections 2024-04-25T10:17:23.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:23.887+0000 7f60f6b5a640 1 -- 172.21.15.133:0/2420422642 wait complete. 2024-04-25T10:17:23.908 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:24.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.023+0000 7f9ba28f0640 1 Processor -- start 2024-04-25T10:17:24.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.023+0000 7f9ba28f0640 1 -- start start 2024-04-25T10:17:24.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba28f0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b9c15c9a0 0x7f9b9c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba28f0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9b9c0ce330 0x7f9b9c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba28f0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9c0c61d0 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba28f0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9c0cece0 con 0x7f9b9c0ce330 2024-04-25T10:17:24.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba28f0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0c64b0 con 0x7f9b9c15c9a0 2024-04-25T10:17:24.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba0e8d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9b9c0ce330 0x7f9b9c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9b937fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9b9c0b1f70 0x7f9b9c06dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46602/0 (socket says 172.21.15.133:46602) 2024-04-25T10:17:24.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9b937fe640 1 -- 172.21.15.133:0/2764983145 learned_addr learned my addr 172.21.15.133:0/2764983145 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:24.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9b9bfff640 1 --2- 172.21.15.133:0/2764983145 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b9c15c9a0 0x7f9b9c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba0e8d640 1 -- 172.21.15.133:0/2764983145 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b9c15c9a0 msgr2=0x7f9b9c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:24.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba0e8d640 1 --2- 172.21.15.133:0/2764983145 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b9c15c9a0 0x7f9b9c0c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba0e8d640 1 -- 172.21.15.133:0/2764983145 >> v1:172.21.15.133:6789/0 conn(0x7f9b9c0b1f70 legacy=0x7f9b9c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba0e8d640 1 -- 172.21.15.133:0/2764983145 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c06d4f0 con 0x7f9b9c0ce330 2024-04-25T10:17:24.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba0e8d640 1 --2- 172.21.15.133:0/2764983145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b9c0ce330 0x7f9b9c0ce7a0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009d90 tx=0x7f9b9401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65c30b64396f509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9b9b7fe640 1 -- 172.21.15.133:0/2764983145 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b940b2070 con 0x7f9b9c0ce330 2024-04-25T10:17:24.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9b9b7fe640 1 -- 172.21.15.133:0/2764983145 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b940a5830 con 0x7f9b9c0ce330 2024-04-25T10:17:24.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9b9b7fe640 1 -- 172.21.15.133:0/2764983145 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b940a5c00 con 0x7f9b9c0ce330 2024-04-25T10:17:24.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba28f0640 1 -- 172.21.15.133:0/2764983145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b9c0ce330 msgr2=0x7f9b9c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba28f0640 1 --2- 172.21.15.133:0/2764983145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b9c0ce330 0x7f9b9c0ce7a0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009d90 tx=0x7f9b9401c970 comp rx=0 tx=0).stop 2024-04-25T10:17:24.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba28f0640 1 -- 172.21.15.133:0/2764983145 shutdown_connections 2024-04-25T10:17:24.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba28f0640 1 --2- 172.21.15.133:0/2764983145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b9c0ce330 0x7f9b9c0ce7a0 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba28f0640 1 --2- 172.21.15.133:0/2764983145 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b9c15c9a0 0x7f9b9c0c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.027+0000 7f9ba28f0640 1 -- 172.21.15.133:0/2764983145 >> 172.21.15.133:0/2764983145 conn(0x7f9b9c067000 msgr2=0x7f9b9c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:24.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 -- 172.21.15.133:0/2764983145 shutdown_connections 2024-04-25T10:17:24.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 -- 172.21.15.133:0/2764983145 wait complete. 2024-04-25T10:17:24.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 Processor -- start 2024-04-25T10:17:24.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 -- start start 2024-04-25T10:17:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b9c0b1f70 0x7f9b9c145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b9c0ce330 0x7f9b9c145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b9c15c9a0 0x7f9b9c0b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c15dad0 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c15d950 con 0x7f9b9c0ce330 2024-04-25T10:17:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c15dc50 con 0x7f9b9c15c9a0 2024-04-25T10:17:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b9bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b9c0b1f70 0x7f9b9c145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b9bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b9c0b1f70 0x7f9b9c145630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56904/0 (socket says 172.21.15.133:56904) 2024-04-25T10:17:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b9bfff640 1 -- 172.21.15.133:0/1257368982 learned_addr learned my addr 172.21.15.133:0/1257368982 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba0e8d640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b9c15c9a0 0x7f9b9c0b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b937fe640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b9c0ce330 0x7f9b9c145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b9bfff640 1 -- 172.21.15.133:0/1257368982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b9c15c9a0 msgr2=0x7f9b9c0b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b9bfff640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b9c15c9a0 0x7f9b9c0b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b9bfff640 1 -- 172.21.15.133:0/1257368982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b9c0ce330 msgr2=0x7f9b9c145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b9bfff640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b9c0ce330 0x7f9b9c145d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b9bfff640 1 -- 172.21.15.133:0/1257368982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c145b70 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba0e8d640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b9c15c9a0 0x7f9b9c0b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b9bfff640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b9c0b1f70 0x7f9b9c145630 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00d830 tx=0x7f9b8c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b997fa640 1 -- 172.21.15.133:0/1257368982 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00b6c0 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b997fa640 1 -- 172.21.15.133:0/1257368982 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c019070 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9b997fa640 1 -- 172.21.15.133:0/1257368982 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00ba60 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 -- 172.21.15.133:0/1257368982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c146640 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 -- 172.21.15.133:0/1257368982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c146c00 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.031+0000 7f9ba28f0640 1 -- 172.21.15.133:0/1257368982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c06bc20 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.035+0000 7f9b997fa640 1 -- 172.21.15.133:0/1257368982 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b8c01e460 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.035+0000 7f9b997fa640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b7c066f60 0x7f9b7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.035+0000 7f9b997fa640 1 -- 172.21.15.133:0/1257368982 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b8c085810 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.035+0000 7f9b937fe640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b7c066f60 0x7f9b7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.035+0000 7f9b937fe640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b7c066f60 0x7f9b7c069420 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0cc0c0 tx=0x7f9b84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.039+0000 7f9b997fa640 1 -- 172.21.15.133:0/1257368982 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b8c056720 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.187+0000 7f9ba28f0640 1 -- 172.21.15.133:0/1257368982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b9c068ef0 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.187+0000 7f9b997fa640 1 -- 172.21.15.133:0/1257368982 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16976 (secure 0 0 0) 0x7f9b8c056020 con 0x7f9b9c0b1f70 2024-04-25T10:17:24.192 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:24.192 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":183,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:07.261559+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:03.841068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:04.604661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:24.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 -- 172.21.15.133:0/1257368982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b7c066f60 msgr2=0x7f9b7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b7c066f60 0x7f9b7c069420 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0cc0c0 tx=0x7f9b84009040 comp rx=0 tx=0).stop 2024-04-25T10:17:24.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 -- 172.21.15.133:0/1257368982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b9c0b1f70 msgr2=0x7f9b9c145630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b9c0b1f70 0x7f9b9c145630 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00d830 tx=0x7f9b8c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 -- 172.21.15.133:0/1257368982 shutdown_connections 2024-04-25T10:17:24.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9b9c15c9a0 0x7f9b9c0b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b9c0ce330 0x7f9b9c145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9b7c066f60 0x7f9b7c069420 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 --2- 172.21.15.133:0/1257368982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b9c0b1f70 0x7f9b9c145630 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 -- 172.21.15.133:0/1257368982 >> 172.21.15.133:0/1257368982 conn(0x7f9b9c067000 msgr2=0x7f9b9c15b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:24.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 -- 172.21.15.133:0/1257368982 shutdown_connections 2024-04-25T10:17:24.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.191+0000 7f9ba28f0640 1 -- 172.21.15.133:0/1257368982 wait complete. 2024-04-25T10:17:24.209 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:17:24.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:24.258 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:24.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde6152e640 1 Processor -- start 2024-04-25T10:17:24.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde6152e640 1 -- start start 2024-04-25T10:17:24.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde6152e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fde5c15d5e0 0x7fde5c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde6152e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c0ce2e0 con 0x7fde5c145170 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde6152e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c06d4b0 con 0x7fde5c0cd820 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde6152e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5c06db00 con 0x7fde5c15d5e0 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde5affd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fde5c145170 0x7fde5c06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46614/0 (socket says 172.21.15.133:46614) 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde5affd640 1 -- 172.21.15.133:0/747453603 learned_addr learned my addr 172.21.15.133:0/747453603 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde5b7fe640 1 --2- 172.21.15.133:0/747453603 >> v2:172.21.15.133:3300/0 conn(0x7fde5c15d5e0 0x7fde5c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde59ffb640 1 -- 172.21.15.133:0/747453603 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946922441 0 0) 0x7fde5c0ce2e0 con 0x7fde5c145170 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde59ffb640 1 -- 172.21.15.133:0/747453603 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde44003530 con 0x7fde5c145170 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde5b7fe640 1 -- 172.21.15.133:0/747453603 >> v1:172.21.15.162:6789/0 conn(0x7fde5c0cd820 legacy=0x7fde5c15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde5b7fe640 1 -- 172.21.15.133:0/747453603 >> v1:172.21.15.133:6789/0 conn(0x7fde5c145170 legacy=0x7fde5c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.315+0000 7fde5b7fe640 1 -- 172.21.15.133:0/747453603 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c06d790 con 0x7fde5c15d5e0 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde5b7fe640 1 --2- 172.21.15.133:0/747453603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde5c15d5e0 0x7fde5c0cdca0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7fde54009d90 tx=0x7fde5401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412ef628df8b2aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde59ffb640 1 -- 172.21.15.133:0/747453603 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540b2070 con 0x7fde5c15d5e0 2024-04-25T10:17:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde59ffb640 1 -- 172.21.15.133:0/747453603 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde540a57f0 con 0x7fde5c15d5e0 2024-04-25T10:17:24.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde59ffb640 1 -- 172.21.15.133:0/747453603 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540a5bc0 con 0x7fde5c15d5e0 2024-04-25T10:17:24.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 -- 172.21.15.133:0/747453603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde5c15d5e0 msgr2=0x7fde5c0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 --2- 172.21.15.133:0/747453603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde5c15d5e0 0x7fde5c0cdca0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7fde54009d90 tx=0x7fde5401c930 comp rx=0 tx=0).stop 2024-04-25T10:17:24.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 -- 172.21.15.133:0/747453603 shutdown_connections 2024-04-25T10:17:24.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 --2- 172.21.15.133:0/747453603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde5c15d5e0 0x7fde5c0cdca0 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 -- 172.21.15.133:0/747453603 >> 172.21.15.133:0/747453603 conn(0x7fde5c066f70 msgr2=0x7fde5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:24.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 -- 172.21.15.133:0/747453603 shutdown_connections 2024-04-25T10:17:24.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 -- 172.21.15.133:0/747453603 wait complete. 2024-04-25T10:17:24.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 Processor -- start 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 -- start start 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde5c0cd820 0x7fde5c13bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde5c145170 0x7fde5c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde5c15adb0 0x7fde5c159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c1696b0 con 0x7fde5c15adb0 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c145550 con 0x7fde5c145170 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.319+0000 7fde6152e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c169830 con 0x7fde5c0cd820 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde5c15adb0 0x7fde5c159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde5c15adb0 0x7fde5c159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56924/0 (socket says 172.21.15.133:56924) 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5b7fe640 1 -- 172.21.15.133:0/448813154 learned_addr learned my addr 172.21.15.133:0/448813154 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5affd640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde5c0cd820 0x7fde5c13bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5a7fc640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde5c145170 0x7fde5c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5b7fe640 1 -- 172.21.15.133:0/448813154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde5c0cd820 msgr2=0x7fde5c13bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5b7fe640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde5c0cd820 0x7fde5c13bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5b7fe640 1 -- 172.21.15.133:0/448813154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde5c145170 msgr2=0x7fde5c15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5b7fe640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde5c145170 0x7fde5c15a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5b7fe640 1 -- 172.21.15.133:0/448813154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c159f50 con 0x7fde5c15adb0 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5b7fe640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde5c15adb0 0x7fde5c159200 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7fde54009d90 tx=0x7fde540196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde437fe640 1 -- 172.21.15.133:0/448813154 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540b2050 con 0x7fde5c15adb0 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde437fe640 1 -- 172.21.15.133:0/448813154 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde540be070 con 0x7fde5c15adb0 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde6152e640 1 -- 172.21.15.133:0/448813154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5c15a230 con 0x7fde5c15adb0 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde437fe640 1 -- 172.21.15.133:0/448813154 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540ad890 con 0x7fde5c15adb0 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde6152e640 1 -- 172.21.15.133:0/448813154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c142e50 con 0x7fde5c15adb0 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde6152e640 1 -- 172.21.15.133:0/448813154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde5c068e80 con 0x7fde5c15adb0 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde437fe640 1 -- 172.21.15.133:0/448813154 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde540ada30 con 0x7fde5c15adb0 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde437fe640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde48066f90 0x7fde48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde437fe640 1 -- 172.21.15.133:0/448813154 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fde5411fb60 con 0x7fde5c15adb0 2024-04-25T10:17:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5affd640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde48066f90 0x7fde48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.323+0000 7fde5affd640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde48066f90 0x7fde48069450 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fde50005890 tx=0x7fde50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.327+0000 7fde437fe640 1 -- 172.21.15.133:0/448813154 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde540c7020 con 0x7fde5c15adb0 2024-04-25T10:17:24.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.523+0000 7fde6152e640 1 -- 172.21.15.133:0/448813154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fde5c15bad0 con 0x7fde5c15adb0 2024-04-25T10:17:24.525 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:17:24.523+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:17:24.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde437fe640 1 -- 172.21.15.133:0/448813154 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) v1 ==== 161+0+0 (secure 0 0 0) 0x7fde54016520 con 0x7fde5c15adb0 2024-04-25T10:17:24.529 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:17:24.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 -- 172.21.15.133:0/448813154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde48066f90 msgr2=0x7fde48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde48066f90 0x7fde48069450 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fde50005890 tx=0x7fde50009040 comp rx=0 tx=0).stop 2024-04-25T10:17:24.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 -- 172.21.15.133:0/448813154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde5c15adb0 msgr2=0x7fde5c159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde5c15adb0 0x7fde5c159200 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7fde54009d90 tx=0x7fde540196d0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 -- 172.21.15.133:0/448813154 shutdown_connections 2024-04-25T10:17:24.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fde48066f90 0x7fde48069450 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde5c15adb0 0x7fde5c159200 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde5c145170 0x7fde5c15a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 --2- 172.21.15.133:0/448813154 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fde5c0cd820 0x7fde5c13bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 -- 172.21.15.133:0/448813154 >> 172.21.15.133:0/448813154 conn(0x7fde5c066f70 msgr2=0x7fde5c15bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:24.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 -- 172.21.15.133:0/448813154 shutdown_connections 2024-04-25T10:17:24.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.527+0000 7fde6152e640 1 -- 172.21.15.133:0/448813154 wait complete. 2024-04-25T10:17:24.550 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:17:24.614 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:17:24.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c93d4640 1 Processor -- start 2024-04-25T10:17:24.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c93d4640 1 -- start start 2024-04-25T10:17:24.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c93d4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff1c415e100 0x7ff1c4140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c93d4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c40cf280 con 0x7ff1c4143560 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c93d4640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c406d440 con 0x7ff1c4142920 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c93d4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c406d720 con 0x7ff1c415e100 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c27fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff1c4143560 0x7ff1c415d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46618/0 (socket says 172.21.15.133:46618) 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c27fc640 1 -- 172.21.15.133:0/2107059912 learned_addr learned my addr 172.21.15.133:0/2107059912 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016887762 0 0) 0x7ff1c40cf280 con 0x7ff1c4143560 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c37fe640 1 --2- 172.21.15.133:0/2107059912 >> v2:172.21.15.162:3300/0 conn(0x7ff1c415e100 0x7ff1c4140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1ac003450 con 0x7ff1c4143560 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 832573508 0 0) 0x7ff1ac003450 con 0x7ff1c4143560 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 >> v2:172.21.15.162:3300/0 conn(0x7ff1c415e100 msgr2=0x7ff1c4140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 --2- 172.21.15.133:0/2107059912 >> v2:172.21.15.162:3300/0 conn(0x7ff1c415e100 0x7ff1c4140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 >> v1:172.21.15.162:6789/0 conn(0x7ff1c4142920 legacy=0x7ff1c40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c406da00 con 0x7ff1c4143560 2024-04-25T10:17:24.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff1b8001680 con 0x7ff1c4143560 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 >> v1:172.21.15.133:6789/0 conn(0x7ff1c4143560 legacy=0x7ff1c415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 --2- 172.21.15.133:0/2107059912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ac003ab0 0x7ff1ac003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 --2- 172.21.15.133:0/2107059912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ac008860 0x7ff1ac00acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 --2- 172.21.15.133:0/2107059912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ac00b570 0x7ff1ac00da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c406d720 con 0x7ff1ac003ab0 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac00e310 con 0x7ff1ac008860 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac00e510 con 0x7ff1ac00b570 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c2ffd640 1 --2- 172.21.15.133:0/2107059912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ac003ab0 0x7ff1ac003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c37fe640 1 --2- 172.21.15.133:0/2107059912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ac00b570 0x7ff1ac00da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c2ffd640 1 -- 172.21.15.133:0/2107059912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ac00b570 msgr2=0x7ff1ac00da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c2ffd640 1 --2- 172.21.15.133:0/2107059912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1ac00b570 0x7ff1ac00da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c2ffd640 1 -- 172.21.15.133:0/2107059912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ac008860 msgr2=0x7ff1ac00acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:24.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c2ffd640 1 --2- 172.21.15.133:0/2107059912 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ac008860 0x7ff1ac00acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c2ffd640 1 -- 172.21.15.133:0/2107059912 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac003450 con 0x7ff1ac003ab0 2024-04-25T10:17:24.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.707+0000 7ff1c37fe640 1 -- 172.21.15.133:0/2107059912 reap_dead start 2024-04-25T10:17:24.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c2ffd640 1 --2- 172.21.15.133:0/2107059912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ac003ab0 0x7ff1ac003ed0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4001780 tx=0x7ff1b401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc5341530aadc83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40013a0 con 0x7ff1ac003ab0 2024-04-25T10:17:24.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b40052f0 con 0x7ff1ac003ab0 2024-04-25T10:17:24.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c1ffb640 1 -- 172.21.15.133:0/2107059912 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4003840 con 0x7ff1ac003ab0 2024-04-25T10:17:24.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c93d4640 1 -- 172.21.15.133:0/2107059912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ac003ab0 msgr2=0x7ff1ac003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c93d4640 1 --2- 172.21.15.133:0/2107059912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ac003ab0 0x7ff1ac003ed0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4001780 tx=0x7ff1b401c400 comp rx=0 tx=0).stop 2024-04-25T10:17:24.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c93d4640 1 -- 172.21.15.133:0/2107059912 shutdown_connections 2024-04-25T10:17:24.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c93d4640 1 --2- 172.21.15.133:0/2107059912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ac003ab0 0x7ff1ac003ed0 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c93d4640 1 -- 172.21.15.133:0/2107059912 >> 172.21.15.133:0/2107059912 conn(0x7ff1c4066f70 msgr2=0x7ff1c406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:24.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c93d4640 1 -- 172.21.15.133:0/2107059912 shutdown_connections 2024-04-25T10:17:24.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c93d4640 1 -- 172.21.15.133:0/2107059912 wait complete. 2024-04-25T10:17:24.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c93d4640 1 Processor -- start 2024-04-25T10:17:24.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.711+0000 7ff1c93d4640 1 -- start start 2024-04-25T10:17:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c93d4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ac003ab0 0x7ff1c41f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c93d4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c4157200 0x7ff1c41e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c93d4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4157680 0x7ff1c41e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c93d4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40ceac0 con 0x7ff1c4157200 2024-04-25T10:17:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c93d4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c4143940 con 0x7ff1ac003ab0 2024-04-25T10:17:24.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c93d4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40cec40 con 0x7ff1c4157680 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c4157200 0x7ff1c41e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c4157200 0x7ff1c41e7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56950/0 (socket says 172.21.15.133:56950) 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c27fc640 1 -- 172.21.15.133:0/241643125 learned_addr learned my addr 172.21.15.133:0/241643125 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c37fe640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4157680 0x7ff1c41e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c2ffd640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ac003ab0 0x7ff1c41f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c27fc640 1 -- 172.21.15.133:0/241643125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4157680 msgr2=0x7ff1c41e82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c27fc640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4157680 0x7ff1c41e82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c27fc640 1 -- 172.21.15.133:0/241643125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ac003ab0 msgr2=0x7ff1c41f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c27fc640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ac003ab0 0x7ff1c41f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c27fc640 1 -- 172.21.15.133:0/241643125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c41f6c70 con 0x7ff1c4157200 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c37fe640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4157680 0x7ff1c41e82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c2ffd640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ac003ab0 0x7ff1c41f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c27fc640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c4157200 0x7ff1c41e7d80 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c700 tx=0x7ff1b800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1ab7fe640 1 -- 172.21.15.133:0/241643125 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8018070 con 0x7ff1c4157200 2024-04-25T10:17:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1ab7fe640 1 -- 172.21.15.133:0/241643125 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b80072d0 con 0x7ff1c4157200 2024-04-25T10:17:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c93d4640 1 -- 172.21.15.133:0/241643125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c41596b0 con 0x7ff1c4157200 2024-04-25T10:17:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1ab7fe640 1 -- 172.21.15.133:0/241643125 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80047f0 con 0x7ff1c4157200 2024-04-25T10:17:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c93d4640 1 -- 172.21.15.133:0/241643125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c4159c70 con 0x7ff1c4157200 2024-04-25T10:17:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c93d4640 1 -- 172.21.15.133:0/241643125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c4068e80 con 0x7ff1c4157200 2024-04-25T10:17:24.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1ab7fe640 1 -- 172.21.15.133:0/241643125 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1b801b070 con 0x7ff1c4157200 2024-04-25T10:17:24.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1ab7fe640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1b0066f90 0x7ff1b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1ab7fe640 1 -- 172.21.15.133:0/241643125 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff1b8084770 con 0x7ff1c4157200 2024-04-25T10:17:24.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.715+0000 7ff1c2ffd640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1b0066f90 0x7ff1b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.719+0000 7ff1c2ffd640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1b0066f90 0x7ff1b0069450 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4001750 tx=0x7ff1b40011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.719+0000 7ff1ab7fe640 1 -- 172.21.15.133:0/241643125 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1b8055710 con 0x7ff1c4157200 2024-04-25T10:17:24.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:24.915+0000 7ff1c93d4640 1 -- 172.21.15.133:0/241643125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff1c40ad4e0 con 0x7ff1c4157200 2024-04-25T10:17:25.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.527+0000 7ff1ab7fe640 1 -- 172.21.15.133:0/241643125 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff1b8055010 con 0x7ff1c4157200 2024-04-25T10:17:25.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 -- 172.21.15.133:0/241643125 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1b0066f90 msgr2=0x7ff1b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1b0066f90 0x7ff1b0069450 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4001750 tx=0x7ff1b40011a0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 -- 172.21.15.133:0/241643125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c4157200 msgr2=0x7ff1c41e7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c4157200 0x7ff1c41e7d80 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c700 tx=0x7ff1b800cbc0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 -- 172.21.15.133:0/241643125 shutdown_connections 2024-04-25T10:17:25.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff1c4157680 0x7ff1c41e82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff1b0066f90 0x7ff1b0069450 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1c4157200 0x7ff1c41e7d80 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 --2- 172.21.15.133:0/241643125 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff1ac003ab0 0x7ff1c41f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 -- 172.21.15.133:0/241643125 >> 172.21.15.133:0/241643125 conn(0x7ff1c4066f70 msgr2=0x7ff1c40cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:25.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 -- 172.21.15.133:0/241643125 shutdown_connections 2024-04-25T10:17:25.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.531+0000 7ff1c93d4640 1 -- 172.21.15.133:0/241643125 wait complete. 2024-04-25T10:17:25.555 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:17:25.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.679+0000 7f4f44f23640 1 Processor -- start 2024-04-25T10:17:25.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.679+0000 7f4f44f23640 1 -- start start 2024-04-25T10:17:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.679+0000 7f4f44f23640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f4f4015aac0 0x7f4f400ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.679+0000 7f4f44f23640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4015b7a0 con 0x7f4f400d01f0 2024-04-25T10:17:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.679+0000 7f4f44f23640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4015ba80 con 0x7f4f400b0220 2024-04-25T10:17:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.679+0000 7f4f44f23640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4015bd60 con 0x7f4f4015aac0 2024-04-25T10:17:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3ed9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4f400d01f0 0x7f4f400d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46630/0 (socket says 172.21.15.133:46630) 2024-04-25T10:17:25.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3ed9d640 1 -- 172.21.15.133:0/4042357632 learned_addr learned my addr 172.21.15.133:0/4042357632 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579405592 0 0) 0x7f4f4015b7a0 con 0x7f4f400d01f0 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3e59c640 1 --2- 172.21.15.133:0/4042357632 >> v2:172.21.15.162:3300/0 conn(0x7f4f4015aac0 0x7f4f400ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f28003530 con 0x7f4f400d01f0 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 874606309 0 0) 0x7f4f28003530 con 0x7f4f400d01f0 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 >> v2:172.21.15.162:3300/0 conn(0x7f4f4015aac0 msgr2=0x7f4f400ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 --2- 172.21.15.133:0/4042357632 >> v2:172.21.15.162:3300/0 conn(0x7f4f4015aac0 0x7f4f400ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 >> v1:172.21.15.162:6789/0 conn(0x7f4f400b0220 legacy=0x7f4f4015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f400cb1f0 con 0x7f4f400d01f0 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4f38001a00 con 0x7f4f400d01f0 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 >> v1:172.21.15.133:6789/0 conn(0x7f4f400d01f0 legacy=0x7f4f400d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f280039c0 0x7f4f28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f28004810 0x7f4f28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 0x7f4f2800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4015bd60 con 0x7f4f280051d0 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2800c170 con 0x7f4f280039c0 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2800e580 con 0x7f4f28004810 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3ed9d640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 0x7f4f2800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3e59c640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f280039c0 0x7f4f28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3dd9b640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f28004810 0x7f4f28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3ed9d640 1 -- 172.21.15.133:0/4042357632 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f28004810 msgr2=0x7f4f28004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3ed9d640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f28004810 0x7f4f28004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3ed9d640 1 -- 172.21.15.133:0/4042357632 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f280039c0 msgr2=0x7f4f28003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3ed9d640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f280039c0 0x7f4f28003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3ed9d640 1 -- 172.21.15.133:0/4042357632 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f28003530 con 0x7f4f280051d0 2024-04-25T10:17:25.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3dd9b640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f28004810 0x7f4f28004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:25.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3ed9d640 1 -- 172.21.15.133:0/4042357632 reap_dead start 2024-04-25T10:17:25.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3ed9d640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 0x7f4f2800b9d0 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800a850 tx=0x7f4f3801fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d288364891597f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:25.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800a4a0 con 0x7f4f280051d0 2024-04-25T10:17:25.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f380ad070 con 0x7f4f280051d0 2024-04-25T10:17:25.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.683+0000 7f4f3d59a640 1 -- 172.21.15.133:0/4042357632 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f380a9980 con 0x7f4f280051d0 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- 172.21.15.133:0/4042357632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 msgr2=0x7f4f2800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 0x7f4f2800b9d0 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800a850 tx=0x7f4f3801fb00 comp rx=0 tx=0).stop 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- 172.21.15.133:0/4042357632 shutdown_connections 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 --2- 172.21.15.133:0/4042357632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 0x7f4f2800b9d0 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- 172.21.15.133:0/4042357632 >> 172.21.15.133:0/4042357632 conn(0x7f4f400726f0 msgr2=0x7f4f400999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- 172.21.15.133:0/4042357632 shutdown_connections 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- 172.21.15.133:0/4042357632 wait complete. 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 Processor -- start 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- start start 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 0x7f4f40158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f400b5ca0 0x7f4f400cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f400b1250 0x7f4f400b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400ce720 con 0x7f4f280051d0 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400ce5a0 con 0x7f4f400b1250 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400ce8a0 con 0x7f4f400b5ca0 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 0x7f4f40158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 0x7f4f40158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56968/0 (socket says 172.21.15.133:56968) 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3e59c640 1 -- 172.21.15.133:0/2955876066 learned_addr learned my addr 172.21.15.133:0/2955876066 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3dd9b640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f400b5ca0 0x7f4f400cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3ed9d640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f400b1250 0x7f4f400b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3e59c640 1 -- 172.21.15.133:0/2955876066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f400b5ca0 msgr2=0x7f4f400cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3e59c640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f400b5ca0 0x7f4f400cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3e59c640 1 -- 172.21.15.133:0/2955876066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f400b1250 msgr2=0x7f4f400b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3e59c640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f400b1250 0x7f4f400b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3e59c640 1 -- 172.21.15.133:0/2955876066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f400c4260 con 0x7f4f280051d0 2024-04-25T10:17:25.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3ed9d640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f400b1250 0x7f4f400b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:25.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f3e59c640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 0x7f4f40158a50 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f4f30011590 tx=0x7f4f30011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:25.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f2effd640 1 -- 172.21.15.133:0/2955876066 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30015930 con 0x7f4f280051d0 2024-04-25T10:17:25.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f2effd640 1 -- 172.21.15.133:0/2955876066 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f30015ad0 con 0x7f4f280051d0 2024-04-25T10:17:25.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- 172.21.15.133:0/2955876066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f400c4540 con 0x7f4f280051d0 2024-04-25T10:17:25.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f2effd640 1 -- 172.21.15.133:0/2955876066 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000f460 con 0x7f4f280051d0 2024-04-25T10:17:25.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- 172.21.15.133:0/2955876066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f40158f90 con 0x7f4f280051d0 2024-04-25T10:17:25.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.687+0000 7f4f44f23640 1 -- 172.21.15.133:0/2955876066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f400b1c10 con 0x7f4f280051d0 2024-04-25T10:17:25.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.691+0000 7f4f2effd640 1 -- 172.21.15.133:0/2955876066 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f3000f600 con 0x7f4f280051d0 2024-04-25T10:17:25.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.691+0000 7f4f2effd640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f20066f60 0x7f4f20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.691+0000 7f4f2effd640 1 -- 172.21.15.133:0/2955876066 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4f30085b00 con 0x7f4f280051d0 2024-04-25T10:17:25.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.691+0000 7f4f3dd9b640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f20066f60 0x7f4f20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.691+0000 7f4f3dd9b640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f20066f60 0x7f4f20069420 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4f4015a190 tx=0x7f4f34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:25.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.695+0000 7f4f2effd640 1 -- 172.21.15.133:0/2955876066 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f30019030 con 0x7f4f280051d0 2024-04-25T10:17:25.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:25.855+0000 7f4f44f23640 1 -- 172.21.15.133:0/2955876066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4f40069000 con 0x7f4f280051d0 2024-04-25T10:17:26.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.539+0000 7f4f2effd640 1 -- 172.21.15.133:0/2955876066 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v185) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4f300562e0 con 0x7f4f280051d0 2024-04-25T10:17:26.542 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:17:26.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 -- 172.21.15.133:0/2955876066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f20066f60 msgr2=0x7f4f20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f20066f60 0x7f4f20069420 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4f4015a190 tx=0x7f4f34009040 comp rx=0 tx=0).stop 2024-04-25T10:17:26.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 -- 172.21.15.133:0/2955876066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 msgr2=0x7f4f40158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 0x7f4f40158a50 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f4f30011590 tx=0x7f4f30011a50 comp rx=0 tx=0).stop 2024-04-25T10:17:26.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 -- 172.21.15.133:0/2955876066 shutdown_connections 2024-04-25T10:17:26.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4f400b1250 0x7f4f400b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4f400b5ca0 0x7f4f400cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4f20066f60 0x7f4f20069420 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 --2- 172.21.15.133:0/2955876066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f280051d0 0x7f4f40158a50 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 -- 172.21.15.133:0/2955876066 >> 172.21.15.133:0/2955876066 conn(0x7f4f400726f0 msgr2=0x7f4f4015b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:26.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 -- 172.21.15.133:0/2955876066 shutdown_connections 2024-04-25T10:17:26.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.543+0000 7f4f44f23640 1 -- 172.21.15.133:0/2955876066 wait complete. 2024-04-25T10:17:26.570 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:17:26.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.695+0000 7fdc851c3640 1 Processor -- start 2024-04-25T10:17:26.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.695+0000 7fdc851c3640 1 -- start start 2024-04-25T10:17:26.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.695+0000 7fdc851c3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc800719c0 0x7fdc80140270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.695+0000 7fdc851c3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc8015da20 0x7fdc8015ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.695+0000 7fdc851c3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc800cc070 con 0x7fdc80157970 2024-04-25T10:17:26.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.695+0000 7fdc851c3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc800cc910 con 0x7fdc800719c0 2024-04-25T10:17:26.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.695+0000 7fdc851c3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc800cc350 con 0x7fdc8015da20 2024-04-25T10:17:26.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.695+0000 7fdc77fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc800719c0 0x7fdc80140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc77fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc800719c0 0x7fdc80140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55284/0 (socket says 172.21.15.133:55284) 2024-04-25T10:17:26.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc77fff640 1 -- 172.21.15.133:0/899742566 learned_addr learned my addr 172.21.15.133:0/899742566 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:26.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc7f59e640 1 --2- 172.21.15.133:0/899742566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc8015da20 0x7fdc8015ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc77fff640 1 -- 172.21.15.133:0/899742566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc8015da20 msgr2=0x7fdc8015ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:26.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc77fff640 1 --2- 172.21.15.133:0/899742566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc8015da20 0x7fdc8015ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc77fff640 1 -- 172.21.15.133:0/899742566 >> v1:172.21.15.162:6789/0 conn(0x7fdc80157970 legacy=0x7fdc800cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:26.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc77fff640 1 -- 172.21.15.133:0/899742566 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc800cc630 con 0x7fdc800719c0 2024-04-25T10:17:26.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc7f59e640 1 --2- 172.21.15.133:0/899742566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc8015da20 0x7fdc8015ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:26.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc77fff640 1 --2- 172.21.15.133:0/899742566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc800719c0 0x7fdc80140270 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009870 tx=0x7fdc7001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a4a01ad385dbc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:26.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc7e59c640 1 -- 172.21.15.133:0/899742566 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc700b0070 con 0x7fdc800719c0 2024-04-25T10:17:26.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc7e59c640 1 -- 172.21.15.133:0/899742566 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc700ab600 con 0x7fdc800719c0 2024-04-25T10:17:26.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc7e59c640 1 -- 172.21.15.133:0/899742566 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc700ab9d0 con 0x7fdc800719c0 2024-04-25T10:17:26.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc851c3640 1 -- 172.21.15.133:0/899742566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc800719c0 msgr2=0x7fdc80140270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc851c3640 1 --2- 172.21.15.133:0/899742566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc800719c0 0x7fdc80140270 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009870 tx=0x7fdc7001c520 comp rx=0 tx=0).stop 2024-04-25T10:17:26.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc851c3640 1 -- 172.21.15.133:0/899742566 shutdown_connections 2024-04-25T10:17:26.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc851c3640 1 --2- 172.21.15.133:0/899742566 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc8015da20 0x7fdc8015ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc851c3640 1 --2- 172.21.15.133:0/899742566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc800719c0 0x7fdc80140270 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc851c3640 1 -- 172.21.15.133:0/899742566 >> 172.21.15.133:0/899742566 conn(0x7fdc8006d370 msgr2=0x7fdc8006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:26.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc851c3640 1 -- 172.21.15.133:0/899742566 shutdown_connections 2024-04-25T10:17:26.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.699+0000 7fdc851c3640 1 -- 172.21.15.133:0/899742566 wait complete. 2024-04-25T10:17:26.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc851c3640 1 Processor -- start 2024-04-25T10:17:26.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc851c3640 1 -- start start 2024-04-25T10:17:26.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc851c3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc800719c0 0x7fdc801381c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc851c3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc80157970 0x7fdc800b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc851c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc8015da20 0x7fdc80163790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc851c3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8015b010 con 0x7fdc8015da20 2024-04-25T10:17:26.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc851c3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8015ae90 con 0x7fdc80157970 2024-04-25T10:17:26.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc851c3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8015b190 con 0x7fdc800719c0 2024-04-25T10:17:26.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc7f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc8015da20 0x7fdc80163790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc7f59e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc8015da20 0x7fdc80163790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55294/0 (socket says 172.21.15.133:55294) 2024-04-25T10:17:26.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc7f59e640 1 -- 172.21.15.133:0/157524180 learned_addr learned my addr 172.21.15.133:0/157524180 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:26.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc77fff640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc80157970 0x7fdc800b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc7ed9d640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc800719c0 0x7fdc801381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc7f59e640 1 -- 172.21.15.133:0/157524180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc800719c0 msgr2=0x7fdc801381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc7f59e640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc800719c0 0x7fdc801381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc7f59e640 1 -- 172.21.15.133:0/157524180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc80157970 msgr2=0x7fdc800b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc7f59e640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc80157970 0x7fdc800b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc7f59e640 1 -- 172.21.15.133:0/157524180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc80138700 con 0x7fdc8015da20 2024-04-25T10:17:26.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc7ed9d640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc800719c0 0x7fdc801381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:26.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc7f59e640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc8015da20 0x7fdc80163790 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800cd80 tx=0x7fdc7800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:26.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc777fe640 1 -- 172.21.15.133:0/157524180 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800f680 con 0x7fdc8015da20 2024-04-25T10:17:26.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc777fe640 1 -- 172.21.15.133:0/157524180 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7800f840 con 0x7fdc8015da20 2024-04-25T10:17:26.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc851c3640 1 -- 172.21.15.133:0/157524180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc800af760 con 0x7fdc8015da20 2024-04-25T10:17:26.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc777fe640 1 -- 172.21.15.133:0/157524180 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800fc30 con 0x7fdc8015da20 2024-04-25T10:17:26.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc851c3640 1 -- 172.21.15.133:0/157524180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc800afd20 con 0x7fdc8015da20 2024-04-25T10:17:26.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc851c3640 1 -- 172.21.15.133:0/157524180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc801394c0 con 0x7fdc8015da20 2024-04-25T10:17:26.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.703+0000 7fdc777fe640 1 -- 172.21.15.133:0/157524180 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc7800e3e0 con 0x7fdc8015da20 2024-04-25T10:17:26.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.707+0000 7fdc777fe640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc60066f60 0x7fdc60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.707+0000 7fdc777fe640 1 -- 172.21.15.133:0/157524180 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fdc78085e60 con 0x7fdc8015da20 2024-04-25T10:17:26.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.707+0000 7fdc7ed9d640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc60066f60 0x7fdc60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.707+0000 7fdc7ed9d640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc60066f60 0x7fdc60069420 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800a850 tx=0x7fdc68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:26.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.711+0000 7fdc777fe640 1 -- 172.21.15.133:0/157524180 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc78014030 con 0x7fdc8015da20 2024-04-25T10:17:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:26.867+0000 7fdc851c3640 1 -- 172.21.15.133:0/157524180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdc80069000 con 0x7fdc8015da20 2024-04-25T10:17:27.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.543+0000 7fdc777fe640 1 -- 172.21.15.133:0/157524180 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v186) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdc78056dc0 con 0x7fdc8015da20 2024-04-25T10:17:27.545 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:17:27.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 -- 172.21.15.133:0/157524180 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc60066f60 msgr2=0x7fdc60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc60066f60 0x7fdc60069420 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800a850 tx=0x7fdc68008040 comp rx=0 tx=0).stop 2024-04-25T10:17:27.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 -- 172.21.15.133:0/157524180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc8015da20 msgr2=0x7fdc80163790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc8015da20 0x7fdc80163790 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800cd80 tx=0x7fdc7800a650 comp rx=0 tx=0).stop 2024-04-25T10:17:27.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 -- 172.21.15.133:0/157524180 shutdown_connections 2024-04-25T10:17:27.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc60066f60 0x7fdc60069420 secure :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800a850 tx=0x7fdc68008040 comp rx=0 tx=0).stop 2024-04-25T10:17:27.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc8015da20 0x7fdc80163790 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc80157970 0x7fdc800b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 --2- 172.21.15.133:0/157524180 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc800719c0 0x7fdc801381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 -- 172.21.15.133:0/157524180 >> 172.21.15.133:0/157524180 conn(0x7fdc8006d370 msgr2=0x7fdc80162f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:27.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 -- 172.21.15.133:0/157524180 shutdown_connections 2024-04-25T10:17:27.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.547+0000 7fdc851c3640 1 -- 172.21.15.133:0/157524180 wait complete. 2024-04-25T10:17:27.579 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:17:27.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff823635640 1 Processor -- start 2024-04-25T10:17:27.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff823635640 1 -- start start 2024-04-25T10:17:27.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff823635640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff81c0b0a80 0x7ff81c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff823635640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c0b14d0 0x7ff81c0b1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff823635640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c0b1e80 con 0x7ff81c0cc090 2024-04-25T10:17:27.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff823635640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c06b4a0 con 0x7ff81c0b0a80 2024-04-25T10:17:27.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff823635640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0affa0 con 0x7ff81c0b14d0 2024-04-25T10:17:27.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff8213d1640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff81c0cc090 0x7ff81c06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55778/0 (socket says 172.21.15.133:55778) 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff8213d1640 1 -- 172.21.15.133:0/4015021242 learned_addr learned my addr 172.21.15.133:0/4015021242 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff821bd2640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c0b14d0 0x7ff81c0b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff820bd0640 1 --2- 172.21.15.133:0/4015021242 >> v2:172.21.15.162:3300/0 conn(0x7ff81c0b0a80 0x7ff81c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572340658 0 0) 0x7ff8100014a0 con 0x7ff81c0cc090 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff808003530 con 0x7ff81c0cc090 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1266411910 0 0) 0x7ff808003530 con 0x7ff81c0cc090 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c0b14d0 msgr2=0x7ff81c0b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c0b14d0 0x7ff81c0b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 >> v2:172.21.15.162:3300/0 conn(0x7ff81c0b0a80 msgr2=0x7ff81c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 --2- 172.21.15.133:0/4015021242 >> v2:172.21.15.162:3300/0 conn(0x7ff81c0b0a80 0x7ff81c0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0b0280 con 0x7ff81c0cc090 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff820bd0640 1 --2- 172.21.15.133:0/4015021242 >> v2:172.21.15.162:3300/0 conn(0x7ff81c0b0a80 0x7ff81c0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:27.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7ff810001bd0 con 0x7ff81c0cc090 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 >> v1:172.21.15.133:6789/0 conn(0x7ff81c0cc090 legacy=0x7ff81c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8080039c0 0x7ff808003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c0b14d0 0x7ff808004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff808004d50 0x7ff8080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c06b4a0 con 0x7ff808004d50 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0affa0 con 0x7ff8080039c0 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80800b9d0 con 0x7ff81c0b14d0 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff821bd2640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff808004d50 0x7ff8080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff820bd0640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c0b14d0 0x7ff808004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff8213d1640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8080039c0 0x7ff808003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff821bd2640 1 -- 172.21.15.133:0/4015021242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c0b14d0 msgr2=0x7ff808004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff821bd2640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c0b14d0 0x7ff808004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff821bd2640 1 -- 172.21.15.133:0/4015021242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8080039c0 msgr2=0x7ff808003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:27.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff821bd2640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff8080039c0 0x7ff808003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.707+0000 7ff821bd2640 1 -- 172.21.15.133:0/4015021242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808003530 con 0x7ff808004d50 2024-04-25T10:17:27.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff821bd2640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff808004d50 0x7ff8080051d0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7ff814009b10 tx=0x7ff81401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95302af2c61b295 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:27.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140b2070 con 0x7ff808004d50 2024-04-25T10:17:27.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8140a5660 con 0x7ff808004d50 2024-04-25T10:17:27.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff80ffff640 1 -- 172.21.15.133:0/4015021242 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140096b0 con 0x7ff808004d50 2024-04-25T10:17:27.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff823635640 1 -- 172.21.15.133:0/4015021242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff808004d50 msgr2=0x7ff8080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff823635640 1 --2- 172.21.15.133:0/4015021242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff808004d50 0x7ff8080051d0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7ff814009b10 tx=0x7ff81401ca70 comp rx=0 tx=0).stop 2024-04-25T10:17:27.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff821bd2640 1 -- 172.21.15.133:0/4015021242 reap_dead start 2024-04-25T10:17:27.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff823635640 1 -- 172.21.15.133:0/4015021242 shutdown_connections 2024-04-25T10:17:27.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff823635640 1 -- 172.21.15.133:0/4015021242 >> 172.21.15.133:0/4015021242 conn(0x7ff81c0999d0 msgr2=0x7ff81c0d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:27.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff823635640 1 -- 172.21.15.133:0/4015021242 shutdown_connections 2024-04-25T10:17:27.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff823635640 1 -- 172.21.15.133:0/4015021242 wait complete. 2024-04-25T10:17:27.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff823635640 1 Processor -- start 2024-04-25T10:17:27.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.711+0000 7ff823635640 1 -- start start 2024-04-25T10:17:27.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff823635640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff81c0c7b00 0x7ff81c1f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff823635640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff81c0c5120 0x7ff81c1f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff823635640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c1f76f0 0x7ff81c0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff823635640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c06b780 con 0x7ff81c0c5120 2024-04-25T10:17:27.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff823635640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0cc470 con 0x7ff81c0c7b00 2024-04-25T10:17:27.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff823635640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0b1940 con 0x7ff81c1f76f0 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff820bd0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff81c0c5120 0x7ff81c1f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff820bd0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff81c0c5120 0x7ff81c1f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55324/0 (socket says 172.21.15.133:55324) 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff820bd0640 1 -- 172.21.15.133:0/200976794 learned_addr learned my addr 172.21.15.133:0/200976794 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff821bd2640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c1f76f0 0x7ff81c0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff8213d1640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff81c0c7b00 0x7ff81c1f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff820bd0640 1 -- 172.21.15.133:0/200976794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c1f76f0 msgr2=0x7ff81c0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff820bd0640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c1f76f0 0x7ff81c0c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff820bd0640 1 -- 172.21.15.133:0/200976794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff81c0c7b00 msgr2=0x7ff81c1f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff820bd0640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff81c0c7b00 0x7ff81c1f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff820bd0640 1 -- 172.21.15.133:0/200976794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0c5950 con 0x7ff81c0c5120 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff8213d1640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff81c0c7b00 0x7ff81c1f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff820bd0640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff81c0c5120 0x7ff81c1f71b0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7ff818010810 tx=0x7ff818010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff80dffb640 1 -- 172.21.15.133:0/200976794 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81801a070 con 0x7ff81c0c5120 2024-04-25T10:17:27.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff80dffb640 1 -- 172.21.15.133:0/200976794 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81800eb00 con 0x7ff81c0c5120 2024-04-25T10:17:27.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff80dffb640 1 -- 172.21.15.133:0/200976794 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800f040 con 0x7ff81c0c5120 2024-04-25T10:17:27.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff823635640 1 -- 172.21.15.133:0/200976794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c0c5c30 con 0x7ff81c0c5120 2024-04-25T10:17:27.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff823635640 1 -- 172.21.15.133:0/200976794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c0c61f0 con 0x7ff81c0c5120 2024-04-25T10:17:27.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff823635640 1 -- 172.21.15.133:0/200976794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c067c70 con 0x7ff81c0c5120 2024-04-25T10:17:27.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff80dffb640 1 -- 172.21.15.133:0/200976794 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff81801d070 con 0x7ff81c0c5120 2024-04-25T10:17:27.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff80dffb640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff800066f90 0x7ff800069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff80dffb640 1 -- 172.21.15.133:0/200976794 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff818085410 con 0x7ff81c0c5120 2024-04-25T10:17:27.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.715+0000 7ff8213d1640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff800066f90 0x7ff800069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.719+0000 7ff8213d1640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff800066f90 0x7ff800069450 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff810001680 tx=0x7ff810001370 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:27.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.719+0000 7ff80dffb640 1 -- 172.21.15.133:0/200976794 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff818056590 con 0x7ff81c0c5120 2024-04-25T10:17:27.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.879+0000 7ff823635640 1 -- 172.21.15.133:0/200976794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff81c1f7ad0 con 0x7ff81c0c5120 2024-04-25T10:17:27.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.883+0000 7ff80dffb640 1 -- 172.21.15.133:0/200976794 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff818055e90 con 0x7ff81c0c5120 2024-04-25T10:17:27.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 -- 172.21.15.133:0/200976794 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff800066f90 msgr2=0x7ff800069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff800066f90 0x7ff800069450 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff810001680 tx=0x7ff810001370 comp rx=0 tx=0).stop 2024-04-25T10:17:27.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 -- 172.21.15.133:0/200976794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff81c0c5120 msgr2=0x7ff81c1f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff81c0c5120 0x7ff81c1f71b0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7ff818010810 tx=0x7ff818010cd0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 -- 172.21.15.133:0/200976794 shutdown_connections 2024-04-25T10:17:27.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff81c1f76f0 0x7ff81c0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff800066f90 0x7ff800069450 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff81c0c5120 0x7ff81c1f71b0 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 --2- 172.21.15.133:0/200976794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff81c0c7b00 0x7ff81c1f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 -- 172.21.15.133:0/200976794 >> 172.21.15.133:0/200976794 conn(0x7ff81c0999d0 msgr2=0x7ff81c06e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:27.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 -- 172.21.15.133:0/200976794 shutdown_connections 2024-04-25T10:17:27.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:27.887+0000 7ff823635640 1 -- 172.21.15.133:0/200976794 wait complete. 2024-04-25T10:17:27.907 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:17:28.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c33bca640 1 Processor -- start 2024-04-25T10:17:28.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c33bca640 1 -- start start 2024-04-25T10:17:28.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c33bca640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c15db00 0x7f5c2c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c33bca640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5c2c0d0bd0 0x7f5c2c161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c33bca640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2c0ceb20 con 0x7f5c2c0d0770 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c33bca640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2c0b7740 con 0x7f5c2c0d0bd0 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c33bca640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0b7a20 con 0x7f5c2c15db00 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c31165640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5c2c0d0770 0x7f5c2c1614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55788/0 (socket says 172.21.15.133:55788) 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c31165640 1 -- 172.21.15.133:0/1968837993 learned_addr learned my addr 172.21.15.133:0/1968837993 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c31966640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c15db00 0x7f5c2c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c32167640 1 --2- 172.21.15.133:0/1968837993 >> v2:172.21.15.162:3300/0 conn(0x7f5c2c0d0bd0 0x7f5c2c161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304488418 0 0) 0x7f5c2c0ceb20 con 0x7f5c2c0d0770 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c18003530 con 0x7f5c2c0d0770 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3511356222 0 0) 0x7f5c18003530 con 0x7f5c2c0d0770 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c15db00 msgr2=0x7f5c2c0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c15db00 0x7f5c2c0b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 >> v2:172.21.15.162:3300/0 conn(0x7f5c2c0d0bd0 msgr2=0x7f5c2c161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 --2- 172.21.15.133:0/1968837993 >> v2:172.21.15.162:3300/0 conn(0x7f5c2c0d0bd0 0x7f5c2c161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c0b97f0 con 0x7f5c2c0d0770 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5c1c001680 con 0x7f5c2c0d0770 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c32167640 1 --2- 172.21.15.133:0/1968837993 >> v2:172.21.15.162:3300/0 conn(0x7f5c2c0d0bd0 0x7f5c2c161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 >> v1:172.21.15.133:6789/0 conn(0x7f5c2c0d0770 legacy=0x7f5c2c1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 0x7f5c18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c18004810 0x7f5c18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c15db00 0x7f5c180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0b7740 con 0x7f5c180039c0 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0b7a20 con 0x7f5c18004810 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1800b9d0 con 0x7f5c2c15db00 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c31966640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 0x7f5c18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c32167640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c15db00 0x7f5c180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c31165640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c18004810 0x7f5c18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c31966640 1 -- 172.21.15.133:0/1968837993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c15db00 msgr2=0x7f5c180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c31966640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c15db00 0x7f5c180051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c31966640 1 -- 172.21.15.133:0/1968837993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c18004810 msgr2=0x7f5c18004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c31966640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c18004810 0x7f5c18004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.023+0000 7f5c31966640 1 -- 172.21.15.133:0/1968837993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c18003530 con 0x7f5c180039c0 2024-04-25T10:17:28.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c31966640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 0x7f5c18003de0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009ac0 tx=0x7f5c2801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a36a669aaf4c598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c280b2070 con 0x7f5c180039c0 2024-04-25T10:17:28.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c280a5510 con 0x7f5c180039c0 2024-04-25T10:17:28.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c30964640 1 -- 172.21.15.133:0/1968837993 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28009690 con 0x7f5c180039c0 2024-04-25T10:17:28.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c33bca640 1 -- 172.21.15.133:0/1968837993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 msgr2=0x7f5c18003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c33bca640 1 --2- 172.21.15.133:0/1968837993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 0x7f5c18003de0 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009ac0 tx=0x7f5c2801ca20 comp rx=0 tx=0).stop 2024-04-25T10:17:28.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c32167640 1 -- 172.21.15.133:0/1968837993 reap_dead start 2024-04-25T10:17:28.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c33bca640 1 -- 172.21.15.133:0/1968837993 shutdown_connections 2024-04-25T10:17:28.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c33bca640 1 -- 172.21.15.133:0/1968837993 >> 172.21.15.133:0/1968837993 conn(0x7f5c2c0727c0 msgr2=0x7f5c2c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:28.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c33bca640 1 -- 172.21.15.133:0/1968837993 shutdown_connections 2024-04-25T10:17:28.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c33bca640 1 -- 172.21.15.133:0/1968837993 wait complete. 2024-04-25T10:17:28.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c33bca640 1 Processor -- start 2024-04-25T10:17:28.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.027+0000 7f5c33bca640 1 -- start start 2024-04-25T10:17:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c33bca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 0x7f5c2c1fa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c33bca640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c158e00 0x7f5c2c15fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c33bca640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c2c160430 0x7f5c2c0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c33bca640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0d0fe0 con 0x7f5c180039c0 2024-04-25T10:17:28.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c33bca640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0b7430 con 0x7f5c2c160430 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c33bca640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c0d18a0 con 0x7f5c2c158e00 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31966640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 0x7f5c2c1fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31966640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 0x7f5c2c1fa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55338/0 (socket says 172.21.15.133:55338) 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31966640 1 -- 172.21.15.133:0/1110153795 learned_addr learned my addr 172.21.15.133:0/1110153795 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31165640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c158e00 0x7f5c2c15fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c32167640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c2c160430 0x7f5c2c0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31966640 1 -- 172.21.15.133:0/1110153795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c158e00 msgr2=0x7f5c2c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31966640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c158e00 0x7f5c2c15fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31966640 1 -- 172.21.15.133:0/1110153795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c2c160430 msgr2=0x7f5c2c0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31966640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c2c160430 0x7f5c2c0ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31966640 1 -- 172.21.15.133:0/1110153795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c0ab170 con 0x7f5c180039c0 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31966640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 0x7f5c2c1fa360 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f5c280b0040 tx=0x7f5c18005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c227fc640 1 -- 172.21.15.133:0/1110153795 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c280b2040 con 0x7f5c180039c0 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c227fc640 1 -- 172.21.15.133:0/1110153795 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c280a5d40 con 0x7f5c180039c0 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c33bca640 1 -- 172.21.15.133:0/1110153795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2c0aa560 con 0x7f5c180039c0 2024-04-25T10:17:28.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c227fc640 1 -- 172.21.15.133:0/1110153795 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c280ad640 con 0x7f5c180039c0 2024-04-25T10:17:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c33bca640 1 -- 172.21.15.133:0/1110153795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2c0aab20 con 0x7f5c180039c0 2024-04-25T10:17:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c33bca640 1 -- 172.21.15.133:0/1110153795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2c068ef0 con 0x7f5c180039c0 2024-04-25T10:17:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c227fc640 1 -- 172.21.15.133:0/1110153795 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c280b5070 con 0x7f5c180039c0 2024-04-25T10:17:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c227fc640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c10066f90 0x7f5c10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c227fc640 1 -- 172.21.15.133:0/1110153795 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5c2811f390 con 0x7f5c180039c0 2024-04-25T10:17:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31165640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c10066f90 0x7f5c10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.031+0000 7f5c31165640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c10066f90 0x7f5c10069450 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c000c10 tx=0x7f5c1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.035+0000 7f5c227fc640 1 -- 172.21.15.133:0/1110153795 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c280bc020 con 0x7f5c180039c0 2024-04-25T10:17:28.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.271+0000 7f5c33bca640 1 -- 172.21.15.133:0/1110153795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f5c2c160810 con 0x7f5c180039c0 2024-04-25T10:17:28.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.543+0000 7f5c227fc640 1 -- 172.21.15.133:0/1110153795 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v702) v1 ==== 147+0+0 (secure 0 0 0) 0x7f5c28016760 con 0x7f5c180039c0 2024-04-25T10:17:28.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 -- 172.21.15.133:0/1110153795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c10066f90 msgr2=0x7f5c10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c10066f90 0x7f5c10069450 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c000c10 tx=0x7f5c1c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 -- 172.21.15.133:0/1110153795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 msgr2=0x7f5c2c1fa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 0x7f5c2c1fa360 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f5c280b0040 tx=0x7f5c18005840 comp rx=0 tx=0).stop 2024-04-25T10:17:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 -- 172.21.15.133:0/1110153795 shutdown_connections 2024-04-25T10:17:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c2c160430 0x7f5c2c0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c2c158e00 0x7f5c2c15fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c10066f90 0x7f5c10069450 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 --2- 172.21.15.133:0/1110153795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c180039c0 0x7f5c2c1fa360 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 -- 172.21.15.133:0/1110153795 >> 172.21.15.133:0/1110153795 conn(0x7f5c2c0727c0 msgr2=0x7f5c2c0b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 -- 172.21.15.133:0/1110153795 shutdown_connections 2024-04-25T10:17:28.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.547+0000 7f5c33bca640 1 -- 172.21.15.133:0/1110153795 wait complete. 2024-04-25T10:17:28.572 INFO:tasks.cephfs_test_runner:test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:17:28.572 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:17:28.573 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:28.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.691+0000 7f75c4e1f640 1 Processor -- start 2024-04-25T10:17:28.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75c4e1f640 1 -- start start 2024-04-25T10:17:28.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75c4e1f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75c01591d0 0x7f75c006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75c4e1f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f75c015c6f0 0x7f75c006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75c4e1f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c006c1e0 0x7f75c015f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75c4e1f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c006d9e0 con 0x7f75c01591d0 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75c4e1f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c006c630 con 0x7f75c015c6f0 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75c4e1f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c015cb60 con 0x7f75c006c1e0 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75be59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75c01591d0 0x7f75c006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75be59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f75c01591d0 0x7f75c006fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55342/0 (socket says 172.21.15.133:55342) 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75be59c640 1 -- 172.21.15.133:0/2390095452 learned_addr learned my addr 172.21.15.133:0/2390095452 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75bdd9b640 1 --2- 172.21.15.133:0/2390095452 >> v2:172.21.15.162:3300/0 conn(0x7f75c015c6f0 0x7f75c006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75bed9d640 1 --2- 172.21.15.133:0/2390095452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c006c1e0 0x7f75c015f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75be59c640 1 -- 172.21.15.133:0/2390095452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c006c1e0 msgr2=0x7f75c015f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75be59c640 1 --2- 172.21.15.133:0/2390095452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c006c1e0 0x7f75c015f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75be59c640 1 -- 172.21.15.133:0/2390095452 >> v2:172.21.15.162:3300/0 conn(0x7f75c015c6f0 msgr2=0x7f75c006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75be59c640 1 --2- 172.21.15.133:0/2390095452 >> v2:172.21.15.162:3300/0 conn(0x7f75c015c6f0 0x7f75c006bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75be59c640 1 -- 172.21.15.133:0/2390095452 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c006e150 con 0x7f75c01591d0 2024-04-25T10:17:28.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75bed9d640 1 --2- 172.21.15.133:0/2390095452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c006c1e0 0x7f75c015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:28.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75be59c640 1 --2- 172.21.15.133:0/2390095452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c01591d0 0x7f75c006fdc0 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f75c015f1a0 tx=0x7f75b801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ec16331d9c306e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.695+0000 7f75bd59a640 1 -- 172.21.15.133:0/2390095452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b80b0070 con 0x7f75c01591d0 2024-04-25T10:17:28.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bd59a640 1 -- 172.21.15.133:0/2390095452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b80ab600 con 0x7f75c01591d0 2024-04-25T10:17:28.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bd59a640 1 -- 172.21.15.133:0/2390095452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b80ab9d0 con 0x7f75c01591d0 2024-04-25T10:17:28.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 -- 172.21.15.133:0/2390095452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c01591d0 msgr2=0x7f75c006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 --2- 172.21.15.133:0/2390095452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c01591d0 0x7f75c006fdc0 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f75c015f1a0 tx=0x7f75b801c260 comp rx=0 tx=0).stop 2024-04-25T10:17:28.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 -- 172.21.15.133:0/2390095452 shutdown_connections 2024-04-25T10:17:28.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 --2- 172.21.15.133:0/2390095452 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c006c1e0 0x7f75c015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 --2- 172.21.15.133:0/2390095452 >> v2:172.21.15.162:3300/0 conn(0x7f75c015c6f0 0x7f75c006bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 --2- 172.21.15.133:0/2390095452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c01591d0 0x7f75c006fdc0 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 -- 172.21.15.133:0/2390095452 >> 172.21.15.133:0/2390095452 conn(0x7f75c0071aa0 msgr2=0x7f75c0071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:28.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 -- 172.21.15.133:0/2390095452 shutdown_connections 2024-04-25T10:17:28.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 -- 172.21.15.133:0/2390095452 wait complete. 2024-04-25T10:17:28.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 Processor -- start 2024-04-25T10:17:28.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 -- start start 2024-04-25T10:17:28.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75c006c1e0 0x7f75c014f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c01591d0 0x7f75c00b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c015c6f0 0x7f75c014dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c015d560 con 0x7f75c01591d0 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c01595b0 con 0x7f75c006c1e0 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c015d6e0 con 0x7f75c015c6f0 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bdd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c01591d0 0x7f75c00b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bdd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c01591d0 0x7f75c00b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55350/0 (socket says 172.21.15.133:55350) 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bdd9b640 1 -- 172.21.15.133:0/3235976686 learned_addr learned my addr 172.21.15.133:0/3235976686 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75be59c640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75c006c1e0 0x7f75c014f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bed9d640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c015c6f0 0x7f75c014dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bdd9b640 1 -- 172.21.15.133:0/3235976686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c015c6f0 msgr2=0x7f75c014dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bdd9b640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c015c6f0 0x7f75c014dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bdd9b640 1 -- 172.21.15.133:0/3235976686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75c006c1e0 msgr2=0x7f75c014f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bdd9b640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75c006c1e0 0x7f75c014f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bdd9b640 1 -- 172.21.15.133:0/3235976686 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c00b1010 con 0x7f75c01591d0 2024-04-25T10:17:28.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75be59c640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75c006c1e0 0x7f75c014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:28.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bed9d640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c015c6f0 0x7f75c014dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:28.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75bdd9b640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c01591d0 0x7f75c00b3fc0 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f75a800d5f0 tx=0x7f75a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75b6ffd640 1 -- 172.21.15.133:0/3235976686 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a80125f0 con 0x7f75c01591d0 2024-04-25T10:17:28.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75b6ffd640 1 -- 172.21.15.133:0/3235976686 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a8019070 con 0x7f75c01591d0 2024-04-25T10:17:28.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 -- 172.21.15.133:0/3235976686 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c00b12d0 con 0x7f75c01591d0 2024-04-25T10:17:28.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75b6ffd640 1 -- 172.21.15.133:0/3235976686 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a80129b0 con 0x7f75c01591d0 2024-04-25T10:17:28.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.699+0000 7f75c4e1f640 1 -- 172.21.15.133:0/3235976686 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c0156f80 con 0x7f75c01591d0 2024-04-25T10:17:28.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.703+0000 7f75c4e1f640 1 -- 172.21.15.133:0/3235976686 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c0067c70 con 0x7f75c01591d0 2024-04-25T10:17:28.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.703+0000 7f75b6ffd640 1 -- 172.21.15.133:0/3235976686 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75a8012b50 con 0x7f75c01591d0 2024-04-25T10:17:28.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.703+0000 7f75b6ffd640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f75a0066f90 0x7f75a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.703+0000 7f75b6ffd640 1 -- 172.21.15.133:0/3235976686 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f75a8085d30 con 0x7f75c01591d0 2024-04-25T10:17:28.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.703+0000 7f75be59c640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f75a0066f90 0x7f75a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.703+0000 7f75be59c640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f75a0066f90 0x7f75a0069450 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f75b80ad000 tx=0x7f75b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.707+0000 7f75b6ffd640 1 -- 172.21.15.133:0/3235976686 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75a8013390 con 0x7f75c01591d0 2024-04-25T10:17:28.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.867+0000 7f75c4e1f640 1 -- 172.21.15.133:0/3235976686 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75c0072ca0 con 0x7f75c01591d0 2024-04-25T10:17:28.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.867+0000 7f75b6ffd640 1 -- 172.21.15.133:0/3235976686 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+13329 (secure 0 0 0) 0x7f75a8056f20 con 0x7f75c01591d0 2024-04-25T10:17:28.871 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:28.871 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":186,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:27.544217+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.162:6835/1744425810":"2024-04-26T10:17:24.527736+0000","172.21.15.162:6834/1744425810":"2024-04-26T10:17:24.527736+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 -- 172.21.15.133:0/3235976686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f75a0066f90 msgr2=0x7f75a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f75a0066f90 0x7f75a0069450 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f75b80ad000 tx=0x7f75b8008040 comp rx=0 tx=0).stop 2024-04-25T10:17:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 -- 172.21.15.133:0/3235976686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c01591d0 msgr2=0x7f75c00b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c01591d0 0x7f75c00b3fc0 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f75a800d5f0 tx=0x7f75a800dab0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 -- 172.21.15.133:0/3235976686 shutdown_connections 2024-04-25T10:17:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f75c015c6f0 0x7f75c014dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f75a0066f90 0x7f75a0069450 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c01591d0 0x7f75c00b3fc0 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 --2- 172.21.15.133:0/3235976686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f75c006c1e0 0x7f75c014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 -- 172.21.15.133:0/3235976686 >> 172.21.15.133:0/3235976686 conn(0x7f75c0071aa0 msgr2=0x7f75c0155930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:28.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 -- 172.21.15.133:0/3235976686 shutdown_connections 2024-04-25T10:17:28.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:28.871+0000 7f75c4e1f640 1 -- 172.21.15.133:0/3235976686 wait complete. 2024-04-25T10:17:28.891 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:28.891 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:29.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.007+0000 7f1467ddd640 1 Processor -- start 2024-04-25T10:17:29.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.007+0000 7f1467ddd640 1 -- start start 2024-04-25T10:17:29.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1467ddd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1460142920 0x7f1460143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1467ddd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1460140ea0 con 0x7f1460140a40 2024-04-25T10:17:29.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1467ddd640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14600cf280 con 0x7f146015d9f0 2024-04-25T10:17:29.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1467ddd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146006d440 con 0x7f1460142920 2024-04-25T10:17:29.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1465378640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1460140a40 0x7f14600ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55790/0 (socket says 172.21.15.133:55790) 2024-04-25T10:17:29.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1465378640 1 -- 172.21.15.133:0/1723693920 learned_addr learned my addr 172.21.15.133:0/1723693920 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1465b79640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1460142920 0x7f1460143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741480004 0 0) 0x7f1460140ea0 con 0x7f1460140a40 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f144c003450 con 0x7f1460140a40 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1352670768 0 0) 0x7f144c003450 con 0x7f1460140a40 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1460142920 msgr2=0x7f1460143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1460142920 0x7f1460143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 >> v1:172.21.15.162:6789/0 conn(0x7f146015d9f0 legacy=0x7f146006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146006d720 con 0x7f1460140a40 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1450001680 con 0x7f1460140a40 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 >> v1:172.21.15.133:6789/0 conn(0x7f1460140a40 legacy=0x7f14600ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f144c003ae0 0x7f144c003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1460142920 0x7f144c00a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f144c00aff0 0x7f144c00d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146006d440 con 0x7f144c003ae0 2024-04-25T10:17:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c00dd00 con 0x7f144c00aff0 2024-04-25T10:17:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c00df00 con 0x7f1460142920 2024-04-25T10:17:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1465b79640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f144c003ae0 0x7f144c003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1465378640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1460142920 0x7f144c00a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f146637a640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f144c00aff0 0x7f144c00d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1465b79640 1 -- 172.21.15.133:0/1723693920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1460142920 msgr2=0x7f144c00a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1465b79640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1460142920 0x7f144c00a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1465b79640 1 -- 172.21.15.133:0/1723693920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f144c00aff0 msgr2=0x7f144c00d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1465b79640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f144c00aff0 0x7f144c00d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.011+0000 7f1465b79640 1 -- 172.21.15.133:0/1723693920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c003450 con 0x7f144c003ae0 2024-04-25T10:17:29.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1465b79640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f144c003ae0 0x7f144c003f00 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f145c009ae0 tx=0x7f145c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c88b6ef15228330 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c0096b0 con 0x7f144c003ae0 2024-04-25T10:17:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c0a6450 con 0x7f144c003ae0 2024-04-25T10:17:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1464b77640 1 -- 172.21.15.133:0/1723693920 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c0a6840 con 0x7f144c003ae0 2024-04-25T10:17:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 -- 172.21.15.133:0/1723693920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f144c003ae0 msgr2=0x7f144c003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 --2- 172.21.15.133:0/1723693920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f144c003ae0 0x7f144c003f00 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f145c009ae0 tx=0x7f145c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:17:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f146637a640 1 -- 172.21.15.133:0/1723693920 reap_dead start 2024-04-25T10:17:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 -- 172.21.15.133:0/1723693920 shutdown_connections 2024-04-25T10:17:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 -- 172.21.15.133:0/1723693920 >> 172.21.15.133:0/1723693920 conn(0x7f1460066f70 msgr2=0x7f146006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 -- 172.21.15.133:0/1723693920 shutdown_connections 2024-04-25T10:17:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 -- 172.21.15.133:0/1723693920 wait complete. 2024-04-25T10:17:29.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 Processor -- start 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 -- start start 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f144c003ae0 0x7f146013e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14600adfd0 0x7f14601415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1460141b30 0x7f14600aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146006e7b0 con 0x7f1460141b30 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146006e630 con 0x7f14600adfd0 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1467ddd640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146006e930 con 0x7f144c003ae0 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f146637a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1460141b30 0x7f14600aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f146637a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1460141b30 0x7f14600aad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55362/0 (socket says 172.21.15.133:55362) 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f146637a640 1 -- 172.21.15.133:0/496426521 learned_addr learned my addr 172.21.15.133:0/496426521 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1465b79640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f144c003ae0 0x7f146013e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.015+0000 7f1465378640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14600adfd0 0x7f14601415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f146637a640 1 -- 172.21.15.133:0/496426521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f144c003ae0 msgr2=0x7f146013e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f146637a640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f144c003ae0 0x7f146013e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f146637a640 1 -- 172.21.15.133:0/496426521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14600adfd0 msgr2=0x7f14601415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f146637a640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14600adfd0 0x7f14601415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f146637a640 1 -- 172.21.15.133:0/496426521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14600abb00 con 0x7f1460141b30 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f146637a640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1460141b30 0x7f14600aad90 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f146006f600 tx=0x7f145800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f14567fc640 1 -- 172.21.15.133:0/496426521 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458003980 con 0x7f1460141b30 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f14567fc640 1 -- 172.21.15.133:0/496426521 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1458018070 con 0x7f1460141b30 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f1467ddd640 1 -- 172.21.15.133:0/496426521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14600ad210 con 0x7f1460141b30 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f14567fc640 1 -- 172.21.15.133:0/496426521 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458003d60 con 0x7f1460141b30 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f1467ddd640 1 -- 172.21.15.133:0/496426521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14600ad7d0 con 0x7f1460141b30 2024-04-25T10:17:29.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f1467ddd640 1 -- 172.21.15.133:0/496426521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1460068e80 con 0x7f1460141b30 2024-04-25T10:17:29.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f14567fc640 1 -- 172.21.15.133:0/496426521 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14580050d0 con 0x7f1460141b30 2024-04-25T10:17:29.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f14567fc640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1444066f60 0x7f1444069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f14567fc640 1 -- 172.21.15.133:0/496426521 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1458084980 con 0x7f1460141b30 2024-04-25T10:17:29.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f1465b79640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1444066f60 0x7f1444069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.019+0000 7f1465b79640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1444066f60 0x7f1444069420 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f145c0ad000 tx=0x7f145c0b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.023+0000 7f14567fc640 1 -- 172.21.15.133:0/496426521 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1458055540 con 0x7f1460141b30 2024-04-25T10:17:29.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.183+0000 7f1467ddd640 1 -- 172.21.15.133:0/496426521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14600afa70 con 0x7f1460141b30 2024-04-25T10:17:29.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f14567fc640 1 -- 172.21.15.133:0/496426521 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14580595e0 con 0x7f1460141b30 2024-04-25T10:17:29.189 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:17:29.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 -- 172.21.15.133:0/496426521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1444066f60 msgr2=0x7f1444069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1444066f60 0x7f1444069420 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f145c0ad000 tx=0x7f145c0b3040 comp rx=0 tx=0).stop 2024-04-25T10:17:29.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 -- 172.21.15.133:0/496426521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1460141b30 msgr2=0x7f14600aad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1460141b30 0x7f14600aad90 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f146006f600 tx=0x7f145800ec10 comp rx=0 tx=0).stop 2024-04-25T10:17:29.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 -- 172.21.15.133:0/496426521 shutdown_connections 2024-04-25T10:17:29.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1444066f60 0x7f1444069420 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1460141b30 0x7f14600aad90 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f14600adfd0 0x7f14601415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 --2- 172.21.15.133:0/496426521 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f144c003ae0 0x7f146013e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 -- 172.21.15.133:0/496426521 >> 172.21.15.133:0/496426521 conn(0x7f1460066f70 msgr2=0x7f14601401e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 -- 172.21.15.133:0/496426521 shutdown_connections 2024-04-25T10:17:29.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.187+0000 7f1467ddd640 1 -- 172.21.15.133:0/496426521 wait complete. 2024-04-25T10:17:29.208 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:17:29.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664c2e7640 1 Processor -- start 2024-04-25T10:17:29.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664c2e7640 1 -- start start 2024-04-25T10:17:29.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664c2e7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f664415a5f0 0x7f66440c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664c2e7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f664406eb70 0x7f66440cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664c2e7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66440cc5a0 con 0x7f664415a9d0 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664c2e7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66440cc880 con 0x7f664406eb70 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664c2e7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66440ccb60 con 0x7f664415a5f0 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664a884640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f664406eb70 0x7f66440cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f6649882640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f664415a9d0 0x7f664406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55800/0 (socket says 172.21.15.133:55800) 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f6649882640 1 -- 172.21.15.133:0/2946094554 learned_addr learned my addr 172.21.15.133:0/2946094554 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664a083640 1 --2- 172.21.15.133:0/2946094554 >> v2:172.21.15.162:3300/0 conn(0x7f664415a5f0 0x7f66440c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664a884640 1 -- 172.21.15.133:0/2946094554 >> v2:172.21.15.162:3300/0 conn(0x7f664415a5f0 msgr2=0x7f66440c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664a884640 1 --2- 172.21.15.133:0/2946094554 >> v2:172.21.15.162:3300/0 conn(0x7f664415a5f0 0x7f66440c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664a884640 1 -- 172.21.15.133:0/2946094554 >> v1:172.21.15.133:6789/0 conn(0x7f664415a9d0 legacy=0x7f664406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f6649081640 1 -- 172.21.15.133:0/2946094554 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171399028 0 0) 0x7f66440cc5a0 con 0x7f664415a9d0 2024-04-25T10:17:29.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.327+0000 7f664a884640 1 -- 172.21.15.133:0/2946094554 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66440cce40 con 0x7f664406eb70 2024-04-25T10:17:29.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a884640 1 --2- 172.21.15.133:0/2946094554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f664406eb70 0x7f66440cbe50 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f664406f350 tx=0x7f663c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31294722558fa02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f6649081640 1 -- 172.21.15.133:0/2946094554 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c0b2070 con 0x7f664406eb70 2024-04-25T10:17:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f6649081640 1 -- 172.21.15.133:0/2946094554 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c0a57f0 con 0x7f664406eb70 2024-04-25T10:17:29.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f6649081640 1 -- 172.21.15.133:0/2946094554 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c0a5bc0 con 0x7f664406eb70 2024-04-25T10:17:29.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 -- 172.21.15.133:0/2946094554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f664406eb70 msgr2=0x7f66440cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 --2- 172.21.15.133:0/2946094554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f664406eb70 0x7f66440cbe50 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f664406f350 tx=0x7f663c01c930 comp rx=0 tx=0).stop 2024-04-25T10:17:29.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 -- 172.21.15.133:0/2946094554 shutdown_connections 2024-04-25T10:17:29.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 --2- 172.21.15.133:0/2946094554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f664406eb70 0x7f66440cbe50 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 --2- 172.21.15.133:0/2946094554 >> v2:172.21.15.162:3300/0 conn(0x7f664415a5f0 0x7f66440c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 -- 172.21.15.133:0/2946094554 >> 172.21.15.133:0/2946094554 conn(0x7f6644067000 msgr2=0x7f664406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 -- 172.21.15.133:0/2946094554 shutdown_connections 2024-04-25T10:17:29.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 -- 172.21.15.133:0/2946094554 wait complete. 2024-04-25T10:17:29.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 Processor -- start 2024-04-25T10:17:29.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 -- start start 2024-04-25T10:17:29.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f664406eb70 0x7f664413ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f664415a5f0 0x7f66441512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664415a9d0 0x7f664415e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440ce470 con 0x7f664406eb70 2024-04-25T10:17:29.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66440ce2f0 con 0x7f664415a9d0 2024-04-25T10:17:29.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664c2e7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66440ce5f0 con 0x7f664415a5f0 2024-04-25T10:17:29.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a083640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f664406eb70 0x7f664413ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a083640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f664406eb70 0x7f664413ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55390/0 (socket says 172.21.15.133:55390) 2024-04-25T10:17:29.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a083640 1 -- 172.21.15.133:0/1293896624 learned_addr learned my addr 172.21.15.133:0/1293896624 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a884640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664415a9d0 0x7f664415e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f6649882640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f664415a5f0 0x7f66441512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a083640 1 -- 172.21.15.133:0/1293896624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f664415a5f0 msgr2=0x7f66441512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a083640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f664415a5f0 0x7f66441512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a083640 1 -- 172.21.15.133:0/1293896624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664415a9d0 msgr2=0x7f664415e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a083640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664415a9d0 0x7f664415e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a083640 1 -- 172.21.15.133:0/1293896624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644140440 con 0x7f664406eb70 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f6649882640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f664415a5f0 0x7f66441512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a884640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664415a9d0 0x7f664415e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.331+0000 7f664a083640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f664406eb70 0x7f664413ff00 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f663400d830 tx=0x7f663400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.335+0000 7f663affd640 1 -- 172.21.15.133:0/1293896624 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663400b6c0 con 0x7f664406eb70 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.335+0000 7f663affd640 1 -- 172.21.15.133:0/1293896624 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6634019070 con 0x7f664406eb70 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.335+0000 7f664c2e7640 1 -- 172.21.15.133:0/1293896624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6644152600 con 0x7f664406eb70 2024-04-25T10:17:29.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.335+0000 7f663affd640 1 -- 172.21.15.133:0/1293896624 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663400ba60 con 0x7f664406eb70 2024-04-25T10:17:29.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.335+0000 7f664c2e7640 1 -- 172.21.15.133:0/1293896624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6644152040 con 0x7f664406eb70 2024-04-25T10:17:29.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.335+0000 7f664c2e7640 1 -- 172.21.15.133:0/1293896624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6644068ef0 con 0x7f664406eb70 2024-04-25T10:17:29.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.335+0000 7f663affd640 1 -- 172.21.15.133:0/1293896624 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f663401e460 con 0x7f664406eb70 2024-04-25T10:17:29.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.335+0000 7f663affd640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6628066f60 0x7f6628069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.335+0000 7f663affd640 1 -- 172.21.15.133:0/1293896624 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6634085570 con 0x7f664406eb70 2024-04-25T10:17:29.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.335+0000 7f6649882640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6628066f60 0x7f6628069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.335+0000 7f6649882640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6628066f60 0x7f6628069420 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6644073b90 tx=0x7f6640009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.339+0000 7f663affd640 1 -- 172.21.15.133:0/1293896624 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f66340566d0 con 0x7f664406eb70 2024-04-25T10:17:29.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:29.555+0000 7f664c2e7640 1 -- 172.21.15.133:0/1293896624 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7f66440738e0 con 0x7f664406eb70 2024-04-25T10:17:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.555+0000 7f663affd640 1 -- 172.21.15.133:0/1293896624 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v704) v1 ==== 136+0+0 (secure 0 0 0) 0x7f6634055fd0 con 0x7f664406eb70 2024-04-25T10:17:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.559+0000 7f664c2e7640 1 -- 172.21.15.133:0/1293896624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6628066f60 msgr2=0x7f6628069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.559+0000 7f664c2e7640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6628066f60 0x7f6628069420 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6644073b90 tx=0x7f6640009040 comp rx=0 tx=0).stop 2024-04-25T10:17:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.559+0000 7f664c2e7640 1 -- 172.21.15.133:0/1293896624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f664406eb70 msgr2=0x7f664413ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.559+0000 7f664c2e7640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f664406eb70 0x7f664413ff00 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f663400d830 tx=0x7f663400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.559+0000 7f664c2e7640 1 -- 172.21.15.133:0/1293896624 shutdown_connections 2024-04-25T10:17:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.559+0000 7f664c2e7640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f664415a9d0 0x7f664415e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.559+0000 7f664c2e7640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f664415a5f0 0x7f66441512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.559+0000 7f664c2e7640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6628066f60 0x7f6628069420 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.559+0000 7f664c2e7640 1 --2- 172.21.15.133:0/1293896624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f664406eb70 0x7f664413ff00 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.559+0000 7f664c2e7640 1 -- 172.21.15.133:0/1293896624 >> 172.21.15.133:0/1293896624 conn(0x7f6644067000 msgr2=0x7f664406e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.559+0000 7f664c2e7640 1 -- 172.21.15.133:0/1293896624 shutdown_connections 2024-04-25T10:17:30.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.563+0000 7f664c2e7640 1 -- 172.21.15.133:0/1293896624 wait complete. 2024-04-25T10:17:30.588 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:17:30.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f694c77a640 1 Processor -- start 2024-04-25T10:17:30.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f694c77a640 1 -- start start 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f694c77a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f694406b3a0 0x7f69440ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f694c77a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69440d0300 con 0x7f69440d1060 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f694c77a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69440cfaa0 con 0x7f69440b5ef0 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f694c77a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69440cfd40 con 0x7f694406b3a0 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f694ad17640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f69440d1060 0x7f69440cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55806/0 (socket says 172.21.15.133:55806) 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f694ad17640 1 -- 172.21.15.133:0/1658744319 learned_addr learned my addr 172.21.15.133:0/1658744319 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f6949d15640 1 --2- 172.21.15.133:0/1658744319 >> v2:172.21.15.133:3300/0 conn(0x7f694406b3a0 0x7f69440ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f6949514640 1 -- 172.21.15.133:0/1658744319 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620375648 0 0) 0x7f69440d0300 con 0x7f69440d1060 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f6949514640 1 -- 172.21.15.133:0/1658744319 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6930003530 con 0x7f69440d1060 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f6949d15640 1 -- 172.21.15.133:0/1658744319 >> v1:172.21.15.162:6789/0 conn(0x7f69440b5ef0 legacy=0x7f694406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f6949d15640 1 -- 172.21.15.133:0/1658744319 >> v1:172.21.15.133:6789/0 conn(0x7f69440d1060 legacy=0x7f69440cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.715+0000 7f6949d15640 1 -- 172.21.15.133:0/1658744319 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440d0020 con 0x7f694406b3a0 2024-04-25T10:17:30.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f6949d15640 1 --2- 172.21.15.133:0/1658744319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f694406b3a0 0x7f69440ce140 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f69340097c0 tx=0x7f693401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb0c7296b0731f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f6949514640 1 -- 172.21.15.133:0/1658744319 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69340a7070 con 0x7f694406b3a0 2024-04-25T10:17:30.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f6949514640 1 -- 172.21.15.133:0/1658744319 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69340b0070 con 0x7f694406b3a0 2024-04-25T10:17:30.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f6949514640 1 -- 172.21.15.133:0/1658744319 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69340ab890 con 0x7f694406b3a0 2024-04-25T10:17:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 -- 172.21.15.133:0/1658744319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f694406b3a0 msgr2=0x7f69440ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 --2- 172.21.15.133:0/1658744319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f694406b3a0 0x7f69440ce140 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f69340097c0 tx=0x7f693401c470 comp rx=0 tx=0).stop 2024-04-25T10:17:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 -- 172.21.15.133:0/1658744319 shutdown_connections 2024-04-25T10:17:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 --2- 172.21.15.133:0/1658744319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f694406b3a0 0x7f69440ce140 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 -- 172.21.15.133:0/1658744319 >> 172.21.15.133:0/1658744319 conn(0x7f69440d0570 msgr2=0x7f69440d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:30.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 -- 172.21.15.133:0/1658744319 shutdown_connections 2024-04-25T10:17:30.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 -- 172.21.15.133:0/1658744319 wait complete. 2024-04-25T10:17:30.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 Processor -- start 2024-04-25T10:17:30.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 -- start start 2024-04-25T10:17:30.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f694406b3a0 0x7f694414ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69440b5ef0 0x7f69440c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69440d1060 0x7f6944154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440b6450 con 0x7f69440b5ef0 2024-04-25T10:17:30.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440b62d0 con 0x7f69440d1060 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f694c77a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694416b8b0 con 0x7f694406b3a0 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f6949d15640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69440b5ef0 0x7f69440c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f6949d15640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69440b5ef0 0x7f69440c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55406/0 (socket says 172.21.15.133:55406) 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.719+0000 7f6949d15640 1 -- 172.21.15.133:0/2712478321 learned_addr learned my addr 172.21.15.133:0/2712478321 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f6949d15640 1 -- 172.21.15.133:0/2712478321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f694406b3a0 msgr2=0x7f694414ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f694a516640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f694406b3a0 0x7f694414ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f694ad17640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69440d1060 0x7f6944154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f6949d15640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f694406b3a0 0x7f694414ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f6949d15640 1 -- 172.21.15.133:0/2712478321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69440d1060 msgr2=0x7f6944154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f6949d15640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69440d1060 0x7f6944154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f6949d15640 1 -- 172.21.15.133:0/2712478321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6944155530 con 0x7f69440b5ef0 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f694a516640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f694406b3a0 0x7f694414ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f694ad17640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69440d1060 0x7f6944154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:30.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f6949d15640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69440b5ef0 0x7f69440c8330 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f69340ad040 tx=0x7f693401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f693affd640 1 -- 172.21.15.133:0/2712478321 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69340bf610 con 0x7f69440b5ef0 2024-04-25T10:17:30.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f693affd640 1 -- 172.21.15.133:0/2712478321 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69340bf7d0 con 0x7f69440b5ef0 2024-04-25T10:17:30.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f694c77a640 1 -- 172.21.15.133:0/2712478321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6944155810 con 0x7f69440b5ef0 2024-04-25T10:17:30.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f693affd640 1 -- 172.21.15.133:0/2712478321 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69340bfbc0 con 0x7f69440b5ef0 2024-04-25T10:17:30.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f694c77a640 1 -- 172.21.15.133:0/2712478321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694414f8f0 con 0x7f69440b5ef0 2024-04-25T10:17:30.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f694c77a640 1 -- 172.21.15.133:0/2712478321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694415f6c0 con 0x7f69440b5ef0 2024-04-25T10:17:30.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f693affd640 1 -- 172.21.15.133:0/2712478321 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69340bd020 con 0x7f69440b5ef0 2024-04-25T10:17:30.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f693affd640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6928066f60 0x7f6928069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f693affd640 1 -- 172.21.15.133:0/2712478321 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f693411f5c0 con 0x7f69440b5ef0 2024-04-25T10:17:30.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f694a516640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6928066f60 0x7f6928069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.723+0000 7f694a516640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6928066f60 0x7f6928069420 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f69440b2ea0 tx=0x7f6940009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.727+0000 7f693affd640 1 -- 172.21.15.133:0/2712478321 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6934016090 con 0x7f69440b5ef0 2024-04-25T10:17:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.887+0000 7f694c77a640 1 -- 172.21.15.133:0/2712478321 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f69440ce2c0 con 0x7f69440b5ef0 2024-04-25T10:17:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.891+0000 7f693affd640 1 -- 172.21.15.133:0/2712478321 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f69340f07d0 con 0x7f69440b5ef0 2024-04-25T10:17:30.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 -- 172.21.15.133:0/2712478321 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6928066f60 msgr2=0x7f6928069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6928066f60 0x7f6928069420 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f69440b2ea0 tx=0x7f6940009040 comp rx=0 tx=0).stop 2024-04-25T10:17:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 -- 172.21.15.133:0/2712478321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69440b5ef0 msgr2=0x7f69440c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69440b5ef0 0x7f69440c8330 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f69340ad040 tx=0x7f693401acb0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 -- 172.21.15.133:0/2712478321 shutdown_connections 2024-04-25T10:17:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f69440d1060 0x7f6944154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6928066f60 0x7f6928069420 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69440b5ef0 0x7f69440c8330 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 --2- 172.21.15.133:0/2712478321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f694406b3a0 0x7f694414ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 -- 172.21.15.133:0/2712478321 >> 172.21.15.133:0/2712478321 conn(0x7f69440d0570 msgr2=0x7f694406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 -- 172.21.15.133:0/2712478321 shutdown_connections 2024-04-25T10:17:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:30.895+0000 7f694c77a640 1 -- 172.21.15.133:0/2712478321 wait complete. 2024-04-25T10:17:30.916 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:17:30.916 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:30.962 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:31.008 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:17:31.008 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:17:31.009 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:31.014 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:31.014 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:17:31.014 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:31.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:31.020 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:17:31.020 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:31.061 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:31.061 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:17:31.061 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:31.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:31.067 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:17:31.067 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.227+0000 7f64c86ea640 1 Processor -- start 2024-04-25T10:17:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.227+0000 7f64c86ea640 1 -- start start 2024-04-25T10:17:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c86ea640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c006eb70 0x7f64c00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c86ea640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c00cbe60 con 0x7f64c01584f0 2024-04-25T10:17:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c86ea640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c00b19b0 con 0x7f64c0158110 2024-04-25T10:17:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c86ea640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c015b360 con 0x7f64c006eb70 2024-04-25T10:17:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c6c87640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f64c01584f0 0x7f64c00cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55818/0 (socket says 172.21.15.133:55818) 2024-04-25T10:17:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c6c87640 1 -- 172.21.15.133:0/1904660092 learned_addr learned my addr 172.21.15.133:0/1904660092 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5c85640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c006eb70 0x7f64c00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746068561 0 0) 0x7f64c00cbe60 con 0x7f64c01584f0 2024-04-25T10:17:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64ac003530 con 0x7f64c01584f0 2024-04-25T10:17:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2820156544 0 0) 0x7f64ac003530 con 0x7f64c01584f0 2024-04-25T10:17:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c006eb70 msgr2=0x7f64c00b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c006eb70 0x7f64c00b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 >> v1:172.21.15.162:6789/0 conn(0x7f64c0158110 legacy=0x7f64c006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c015b640 con 0x7f64c01584f0 2024-04-25T10:17:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f64b8001a00 con 0x7f64c01584f0 2024-04-25T10:17:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 >> v1:172.21.15.133:6789/0 conn(0x7f64c01584f0 legacy=0x7f64c00cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64ac0039f0 0x7f64ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64ac004810 0x7f64ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c006eb70 0x7f64ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c015b360 con 0x7f64ac0039f0 2024-04-25T10:17:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac00bab0 con 0x7f64ac004810 2024-04-25T10:17:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac00bcb0 con 0x7f64c006eb70 2024-04-25T10:17:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c6486640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64ac0039f0 0x7f64ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c6c87640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c006eb70 0x7f64ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5c85640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64ac004810 0x7f64ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c6486640 1 -- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c006eb70 msgr2=0x7f64ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c6486640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c006eb70 0x7f64ac0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c6486640 1 -- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64ac004810 msgr2=0x7f64ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c6486640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64ac004810 0x7f64ac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c6486640 1 -- 172.21.15.133:0/1904660092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac003530 con 0x7f64ac0039f0 2024-04-25T10:17:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.231+0000 7f64c5c85640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64ac004810 0x7f64ac004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:31.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c6486640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64ac0039f0 0x7f64ac003e10 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f64b4001750 tx=0x7f64b401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe424c556c71753 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b40013a0 con 0x7f64ac0039f0 2024-04-25T10:17:31.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b40a4e50 con 0x7f64ac0039f0 2024-04-25T10:17:31.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c5484640 1 -- 172.21.15.133:0/1904660092 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4003600 con 0x7f64ac0039f0 2024-04-25T10:17:31.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 -- 172.21.15.133:0/1904660092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64ac0039f0 msgr2=0x7f64ac003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 --2- 172.21.15.133:0/1904660092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64ac0039f0 0x7f64ac003e10 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f64b4001750 tx=0x7f64b401c390 comp rx=0 tx=0).stop 2024-04-25T10:17:31.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c6c87640 1 -- 172.21.15.133:0/1904660092 reap_dead start 2024-04-25T10:17:31.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 -- 172.21.15.133:0/1904660092 shutdown_connections 2024-04-25T10:17:31.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 -- 172.21.15.133:0/1904660092 >> 172.21.15.133:0/1904660092 conn(0x7f64c0067000 msgr2=0x7f64c006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:31.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 -- 172.21.15.133:0/1904660092 shutdown_connections 2024-04-25T10:17:31.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 -- 172.21.15.133:0/1904660092 wait complete. 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 Processor -- start 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 -- start start 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64ac0039f0 0x7f64c01f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c01f8f30 0x7f64c0160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c01649f0 0x7f64c0156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00cd670 con 0x7f64c01649f0 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c015b120 con 0x7f64ac0039f0 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c86ea640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00cd7f0 con 0x7f64c01f8f30 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c6c87640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c01649f0 0x7f64c0156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c6c87640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c01649f0 0x7f64c0156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55424/0 (socket says 172.21.15.133:55424) 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c6c87640 1 -- 172.21.15.133:0/2096955473 learned_addr learned my addr 172.21.15.133:0/2096955473 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c6486640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64ac0039f0 0x7f64c01f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.235+0000 7f64c5c85640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c01f8f30 0x7f64c0160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c6c87640 1 -- 172.21.15.133:0/2096955473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c01f8f30 msgr2=0x7f64c0160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c6c87640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c01f8f30 0x7f64c0160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c6c87640 1 -- 172.21.15.133:0/2096955473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64ac0039f0 msgr2=0x7f64c01f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c6c87640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64ac0039f0 0x7f64c01f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c6c87640 1 -- 172.21.15.133:0/2096955473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c01f4c90 con 0x7f64c01649f0 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c5c85640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c01f8f30 0x7f64c0160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c6486640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64ac0039f0 0x7f64c01f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c6c87640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c01649f0 0x7f64c0156110 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f64c00b4da0 tx=0x7f64b800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64b2ffd640 1 -- 172.21.15.133:0/2096955473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8004d70 con 0x7f64c01649f0 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64b2ffd640 1 -- 172.21.15.133:0/2096955473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b800bb80 con 0x7f64c01649f0 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c86ea640 1 -- 172.21.15.133:0/2096955473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c01656d0 con 0x7f64c01649f0 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64b2ffd640 1 -- 172.21.15.133:0/2096955473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8007110 con 0x7f64c01649f0 2024-04-25T10:17:31.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c86ea640 1 -- 172.21.15.133:0/2096955473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c01f4f50 con 0x7f64c01649f0 2024-04-25T10:17:31.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c86ea640 1 -- 172.21.15.133:0/2096955473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c006bc20 con 0x7f64c01649f0 2024-04-25T10:17:31.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64b2ffd640 1 -- 172.21.15.133:0/2096955473 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64b80072b0 con 0x7f64c01649f0 2024-04-25T10:17:31.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64b2ffd640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64a4066f90 0x7f64a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64b2ffd640 1 -- 172.21.15.133:0/2096955473 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f64b8084e20 con 0x7f64c01649f0 2024-04-25T10:17:31.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c6486640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64a4066f90 0x7f64a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.239+0000 7f64c6486640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64a4066f90 0x7f64a4069450 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0057f0 tx=0x7f64b40011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.243+0000 7f64b2ffd640 1 -- 172.21.15.133:0/2096955473 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64b8055f90 con 0x7f64c01649f0 2024-04-25T10:17:31.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.439+0000 7f64c86ea640 1 -- 172.21.15.133:0/2096955473 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64c015ee20 con 0x7f64c01649f0 2024-04-25T10:17:31.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.439+0000 7f64b2ffd640 1 -- 172.21.15.133:0/2096955473 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+3715 (secure 0 0 0) 0x7f64b8055890 con 0x7f64c01649f0 2024-04-25T10:17:31.443 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:31.443 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":114,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/1718839127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":1718839127},{"type":"v1","addr":"172.21.15.162:6833","nonce":1718839127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":8086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/2164142470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":2164142470},{"type":"v1","addr":"172.21.15.162:6835","nonce":2164142470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114},{"gid":11487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/368827103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":368827103},{"type":"v1","addr":"172.21.15.133:6835","nonce":368827103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":11490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1274732295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1274732295},{"type":"v1","addr":"172.21.15.133:6837","nonce":1274732295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[]} 2024-04-25T10:17:31.443 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 114 2024-04-25T10:17:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 -- 172.21.15.133:0/2096955473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64a4066f90 msgr2=0x7f64a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64a4066f90 0x7f64a4069450 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0057f0 tx=0x7f64b40011a0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 -- 172.21.15.133:0/2096955473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c01649f0 msgr2=0x7f64c0156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c01649f0 0x7f64c0156110 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f64c00b4da0 tx=0x7f64b800b4a0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 -- 172.21.15.133:0/2096955473 shutdown_connections 2024-04-25T10:17:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f64a4066f90 0x7f64a4069450 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c01649f0 0x7f64c0156110 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f64c01f8f30 0x7f64c0160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 --2- 172.21.15.133:0/2096955473 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f64ac0039f0 0x7f64c01f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 -- 172.21.15.133:0/2096955473 >> 172.21.15.133:0/2096955473 conn(0x7f64c0067000 msgr2=0x7f64c0159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 -- 172.21.15.133:0/2096955473 shutdown_connections 2024-04-25T10:17:31.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.443+0000 7f64c86ea640 1 -- 172.21.15.133:0/2096955473 wait complete. 2024-04-25T10:17:31.463 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:17:31.463 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:17:31.463 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:17:31.463 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:31.464 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:17:31.464 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:17:31.464 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:17:31.464 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:31.464 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:17:31.464 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:17:31.464 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:17:31.465 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:31.465 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:17:31.465 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:17:31.465 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:17:31.465 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:31.478 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:17:31.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:17:31.479 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:17:31.479 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:17:31.482 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:17:31.482 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:17:31.483 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:17:31.483 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:17:31.522 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:17:31.522 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:31.523 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:17:31.523 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:31.528 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:17:31.528 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:31.529 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:17:31.529 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:31.529 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:17:31.585 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-04-25T10:17:31.592 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:17:31.598 INFO:tasks.ceph.mds.d.smithi162.stdout:starting mds.d at 2024-04-25T10:17:31.602 INFO:tasks.ceph.mds.b.smithi162.stdout:starting mds.b at 2024-04-25T10:17:31.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22ca7cc640 1 Processor -- start 2024-04-25T10:17:31.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22ca7cc640 1 -- start start 2024-04-25T10:17:31.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22ca7cc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22c40cdf50 0x7f22c40d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22ca7cc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22c40d1c30 0x7f22c40cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22ca7cc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c40d20c0 con 0x7f22c415dab0 2024-04-25T10:17:31.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22ca7cc640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c40b1150 con 0x7f22c40d1c30 2024-04-25T10:17:31.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22ca7cc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c415ed70 con 0x7f22c40cdf50 2024-04-25T10:17:31.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22c8d69640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22c40d1c30 0x7f22c40cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22c3fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f22c415dab0 0x7f22c406d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55826/0 (socket says 172.21.15.133:55826) 2024-04-25T10:17:31.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22c3fff640 1 -- 172.21.15.133:0/1275884989 learned_addr learned my addr 172.21.15.133:0/1275884989 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:31.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22c37fe640 1 --2- 172.21.15.133:0/1275884989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22c40cdf50 0x7f22c40d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22c8d69640 1 -- 172.21.15.133:0/1275884989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22c40cdf50 msgr2=0x7f22c40d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:31.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22c8d69640 1 --2- 172.21.15.133:0/1275884989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22c40cdf50 0x7f22c40d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22c8d69640 1 -- 172.21.15.133:0/1275884989 >> v1:172.21.15.133:6789/0 conn(0x7f22c415dab0 legacy=0x7f22c406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22c2ffd640 1 -- 172.21.15.133:0/1275884989 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003779875 0 0) 0x7f22c40d20c0 con 0x7f22c415dab0 2024-04-25T10:17:31.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.643+0000 7f22c8d69640 1 -- 172.21.15.133:0/1275884989 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c415de90 con 0x7f22c40d1c30 2024-04-25T10:17:31.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22c8d69640 1 --2- 172.21.15.133:0/1275884989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22c40d1c30 0x7f22c40cd290 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009d90 tx=0x7f22bc01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57d74b328ff22b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22c2ffd640 1 -- 172.21.15.133:0/1275884989 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0b4070 con 0x7f22c40d1c30 2024-04-25T10:17:31.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22c2ffd640 1 -- 172.21.15.133:0/1275884989 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc0af600 con 0x7f22c40d1c30 2024-04-25T10:17:31.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22c2ffd640 1 -- 172.21.15.133:0/1275884989 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0af9d0 con 0x7f22c40d1c30 2024-04-25T10:17:31.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1275884989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22c40d1c30 msgr2=0x7f22c40cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22ca7cc640 1 --2- 172.21.15.133:0/1275884989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22c40d1c30 0x7f22c40cd290 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009d90 tx=0x7f22bc01f150 comp rx=0 tx=0).stop 2024-04-25T10:17:31.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1275884989 shutdown_connections 2024-04-25T10:17:31.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22ca7cc640 1 --2- 172.21.15.133:0/1275884989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22c40d1c30 0x7f22c40cd290 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22ca7cc640 1 --2- 172.21.15.133:0/1275884989 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22c40cdf50 0x7f22c40d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1275884989 >> 172.21.15.133:0/1275884989 conn(0x7f22c40b3370 msgr2=0x7f22c40b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:31.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1275884989 shutdown_connections 2024-04-25T10:17:31.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1275884989 wait complete. 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22ca7cc640 1 Processor -- start 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.647+0000 7f22ca7cc640 1 -- start start 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22ca7cc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22c40cdf50 0x7f22c4157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22ca7cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22c40d1c30 0x7f22c40cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22ca7cc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22c415dab0 0x7f22c414fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22ca7cc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c415e440 con 0x7f22c40d1c30 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22ca7cc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c40ce390 con 0x7f22c415dab0 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22ca7cc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c415e720 con 0x7f22c40cdf50 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22c40d1c30 0x7f22c40cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22c40d1c30 0x7f22c40cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55450/0 (socket says 172.21.15.133:55450) 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c37fe640 1 -- 172.21.15.133:0/1473046611 learned_addr learned my addr 172.21.15.133:0/1473046611 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c3fff640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22c40cdf50 0x7f22c4157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c8d69640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22c415dab0 0x7f22c414fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c37fe640 1 -- 172.21.15.133:0/1473046611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22c40cdf50 msgr2=0x7f22c4157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c37fe640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22c40cdf50 0x7f22c4157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c37fe640 1 -- 172.21.15.133:0/1473046611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22c415dab0 msgr2=0x7f22c414fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c37fe640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22c415dab0 0x7f22c414fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c37fe640 1 -- 172.21.15.133:0/1473046611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c40cb7b0 con 0x7f22c40d1c30 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c37fe640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22c40d1c30 0x7f22c40cb270 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f22b800c820 tx=0x7f22b800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c0ff9640 1 -- 172.21.15.133:0/1473046611 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8015070 con 0x7f22c40d1c30 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c0ff9640 1 -- 172.21.15.133:0/1473046611 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b8004240 con 0x7f22c40d1c30 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c0ff9640 1 -- 172.21.15.133:0/1473046611 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b800ce10 con 0x7f22c40d1c30 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1473046611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c4157fb0 con 0x7f22c40d1c30 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1473046611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c4158570 con 0x7f22c40d1c30 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1473046611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c415f230 con 0x7f22c40d1c30 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c0ff9640 1 -- 172.21.15.133:0/1473046611 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22b8007830 con 0x7f22c40d1c30 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c0ff9640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22b0066f60 0x7f22b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c0ff9640 1 -- 172.21.15.133:0/1473046611 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f22b8084bf0 con 0x7f22c40d1c30 2024-04-25T10:17:31.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.651+0000 7f22c3fff640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22b0066f60 0x7f22b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.655+0000 7f22c3fff640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22b0066f60 0x7f22b0069420 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f22b4000f70 tx=0x7f22b4007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.655+0000 7f22c0ff9640 1 -- 172.21.15.133:0/1473046611 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22b80557b0 con 0x7f22c40d1c30 2024-04-25T10:17:31.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:31.823+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1473046611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f22c415e9f0 con 0x7f22c40d1c30 2024-04-25T10:17:33.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:32.999+0000 7f22c0ff9640 1 -- 172.21.15.133:0/1473046611 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v191) v1 ==== 97+0+0 (secure 0 0 0) 0x7f22b8059850 con 0x7f22c40d1c30 2024-04-25T10:17:33.002 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-04-25T10:17:33.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:32.999+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1473046611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22b0066f60 msgr2=0x7f22b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:32.999+0000 7f22ca7cc640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22b0066f60 0x7f22b0069420 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f22b4000f70 tx=0x7f22b4007df0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.003+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1473046611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22c40d1c30 msgr2=0x7f22c40cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.003+0000 7f22ca7cc640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22c40d1c30 0x7f22c40cb270 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f22b800c820 tx=0x7f22b800cce0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.003+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1473046611 shutdown_connections 2024-04-25T10:17:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.003+0000 7f22ca7cc640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22c415dab0 0x7f22c414fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.003+0000 7f22ca7cc640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f22b0066f60 0x7f22b0069420 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.003+0000 7f22ca7cc640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22c40d1c30 0x7f22c40cb270 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.003+0000 7f22ca7cc640 1 --2- 172.21.15.133:0/1473046611 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22c40cdf50 0x7f22c4157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.003+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1473046611 >> 172.21.15.133:0/1473046611 conn(0x7f22c40b3370 msgr2=0x7f22c406f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.003+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1473046611 shutdown_connections 2024-04-25T10:17:33.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.003+0000 7f22ca7cc640 1 -- 172.21.15.133:0/1473046611 wait complete. 2024-04-25T10:17:33.026 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:17:33.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bbaf34640 1 Processor -- start 2024-04-25T10:17:33.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bbaf34640 1 -- start start 2024-04-25T10:17:33.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bbaf34640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1bb415c9a0 0x7f1bb40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bbaf34640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bb40b1f70 0x7f1bb406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bbaf34640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb40c61d0 con 0x7f1bb40ce330 2024-04-25T10:17:33.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bbaf34640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb40c64b0 con 0x7f1bb415c9a0 2024-04-25T10:17:33.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bbaf34640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb406d4f0 con 0x7f1bb40b1f70 2024-04-25T10:17:33.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bb8cd0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1bb415c9a0 0x7f1bb40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bb8cd0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1bb415c9a0 0x7f1bb40c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55464/0 (socket says 172.21.15.133:55464) 2024-04-25T10:17:33.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bb8cd0640 1 -- 172.21.15.133:0/3866400970 learned_addr learned my addr 172.21.15.133:0/3866400970 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:33.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bb3fff640 1 --2- 172.21.15.133:0/3866400970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bb40b1f70 0x7f1bb406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bb8cd0640 1 -- 172.21.15.133:0/3866400970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bb40b1f70 msgr2=0x7f1bb406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:33.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bb8cd0640 1 --2- 172.21.15.133:0/3866400970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bb40b1f70 0x7f1bb406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bb8cd0640 1 -- 172.21.15.133:0/3866400970 >> v1:172.21.15.162:6789/0 conn(0x7f1bb40ce330 legacy=0x7f1bb415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.147+0000 7f1bb8cd0640 1 -- 172.21.15.133:0/3866400970 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb406d7d0 con 0x7f1bb415c9a0 2024-04-25T10:17:33.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bb8cd0640 1 --2- 172.21.15.133:0/3866400970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb415c9a0 0x7f1bb40c5980 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009870 tx=0x7f1ba401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0cefb44e670827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bb37fe640 1 -- 172.21.15.133:0/3866400970 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40ab610 con 0x7f1bb415c9a0 2024-04-25T10:17:33.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bb37fe640 1 -- 172.21.15.133:0/3866400970 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba40b0070 con 0x7f1bb415c9a0 2024-04-25T10:17:33.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bb37fe640 1 -- 172.21.15.133:0/3866400970 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40ab9d0 con 0x7f1bb415c9a0 2024-04-25T10:17:33.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 -- 172.21.15.133:0/3866400970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb415c9a0 msgr2=0x7f1bb40c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 --2- 172.21.15.133:0/3866400970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb415c9a0 0x7f1bb40c5980 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009870 tx=0x7f1ba401c500 comp rx=0 tx=0).stop 2024-04-25T10:17:33.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 -- 172.21.15.133:0/3866400970 shutdown_connections 2024-04-25T10:17:33.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 --2- 172.21.15.133:0/3866400970 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bb40b1f70 0x7f1bb406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 --2- 172.21.15.133:0/3866400970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb415c9a0 0x7f1bb40c5980 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 -- 172.21.15.133:0/3866400970 >> 172.21.15.133:0/3866400970 conn(0x7f1bb4067000 msgr2=0x7f1bb406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:33.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 -- 172.21.15.133:0/3866400970 shutdown_connections 2024-04-25T10:17:33.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 -- 172.21.15.133:0/3866400970 wait complete. 2024-04-25T10:17:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 Processor -- start 2024-04-25T10:17:33.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 -- start start 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bb40b1f70 0x7f1bb4151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb40ce330 0x7f1bb41567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bb415c9a0 0x7f1bb40b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40ceb50 con 0x7f1bb40ce330 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb415cd80 con 0x7f1bb40b1f70 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.151+0000 7f1bbaf34640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb4159100 con 0x7f1bb415c9a0 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb40ce330 0x7f1bb41567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb40ce330 0x7f1bb41567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55476/0 (socket says 172.21.15.133:55476) 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb3fff640 1 -- 172.21.15.133:0/822060490 learned_addr learned my addr 172.21.15.133:0/822060490 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb8cd0640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bb40b1f70 0x7f1bb4151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb94d1640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bb415c9a0 0x7f1bb40b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb3fff640 1 -- 172.21.15.133:0/822060490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bb415c9a0 msgr2=0x7f1bb40b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb3fff640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bb415c9a0 0x7f1bb40b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb3fff640 1 -- 172.21.15.133:0/822060490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bb40b1f70 msgr2=0x7f1bb4151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb3fff640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bb40b1f70 0x7f1bb4151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb3fff640 1 -- 172.21.15.133:0/822060490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40c9780 con 0x7f1bb40ce330 2024-04-25T10:17:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb8cd0640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bb40b1f70 0x7f1bb4151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb3fff640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb40ce330 0x7f1bb41567c0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800d5f0 tx=0x7f1ba800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb17fa640 1 -- 172.21.15.133:0/822060490 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80125f0 con 0x7f1bb40ce330 2024-04-25T10:17:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb17fa640 1 -- 172.21.15.133:0/822060490 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba8019070 con 0x7f1bb40ce330 2024-04-25T10:17:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bbaf34640 1 -- 172.21.15.133:0/822060490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb40c9a60 con 0x7f1bb40ce330 2024-04-25T10:17:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb17fa640 1 -- 172.21.15.133:0/822060490 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80129b0 con 0x7f1bb40ce330 2024-04-25T10:17:33.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bbaf34640 1 -- 172.21.15.133:0/822060490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb40ca020 con 0x7f1bb40ce330 2024-04-25T10:17:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bbaf34640 1 -- 172.21.15.133:0/822060490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb406bc20 con 0x7f1bb40ce330 2024-04-25T10:17:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb17fa640 1 -- 172.21.15.133:0/822060490 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ba8012b50 con 0x7f1bb40ce330 2024-04-25T10:17:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb17fa640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ba0066f60 0x7f1ba0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb8cd0640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ba0066f60 0x7f1ba0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb17fa640 1 -- 172.21.15.133:0/822060490 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1ba8085310 con 0x7f1bb40ce330 2024-04-25T10:17:33.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.155+0000 7f1bb8cd0640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ba0066f60 0x7f1ba0069420 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40af000 tx=0x7f1ba40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.159+0000 7f1bb17fa640 1 -- 172.21.15.133:0/822060490 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ba8056450 con 0x7f1bb40ce330 2024-04-25T10:17:33.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.375+0000 7f1bbaf34640 1 -- 172.21.15.133:0/822060490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1bb415bfb0 con 0x7f1bb40ce330 2024-04-25T10:17:33.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.375+0000 7f1bb17fa640 1 -- 172.21.15.133:0/822060490 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v27) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1ba8083e20 con 0x7f1bb40ce330 2024-04-25T10:17:33.380 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:33.381 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:17:33.381 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-04-25T10:17:33.381 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.381 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-04-25T10:17:33.381 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm0RT2GBAAGSOwdMBCA3NfQ0IWCJpPyg==", 2024-04-25T10:17:33.381 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.381 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:33.381 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:17:33.381 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:17:33.381 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.381 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.382 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.382 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.382 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-04-25T10:17:33.382 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmq0x/JxAAS9Hn5XIVn1sWCRGpnf9Cug==", 2024-04-25T10:17:33.382 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmDK4zIRAAL7tMZQDwKTHtPyHdvKVfMw==", 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.383 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmFtc9LRAAAMTMWVsA8wiiwwAJHhOX2g==", 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-04-25T10:17:33.384 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDYKSpmSSj4ABAAlWHlA9+9txDXx9fs+0dGmA==", 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDbKSpmC6hDBhAAQZG7hMl+NQsxnvtuST4MKQ==", 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.385 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDeKSpmLpS2DRAA0UC1jDIFgej240BROoAfRA==", 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDhKSpmkWyuFBAAc96Q6TPob9tduWjeqYDNRA==", 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.386 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDnKSpm+HWAJxAAY2BWNhmQfiUvoUiBsGeNzg==", 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.387 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.388 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-04-25T10:17:33.388 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDqKSpmpyGWKxAAXkUDIPsGD1FkTFCrfL/7Qg==", 2024-04-25T10:17:33.388 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.388 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.388 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.388 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.388 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.388 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.388 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-04-25T10:17:33.389 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDtKSpmec8UMhAANWKk0QuPrYp+nDIb/6b5qw==", 2024-04-25T10:17:33.389 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.389 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.389 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.389 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.389 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.389 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.389 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.389 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-04-25T10:17:33.389 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDwKSpmKiX9NxAA5H/N4pVAG/tt69Sb8Stq2A==", 2024-04-25T10:17:33.389 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.390 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.390 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.390 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.390 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.390 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.390 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.390 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-04-25T10:17:33.390 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug==", 2024-04-25T10:17:33.390 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmJSXDOhAAkkC7sCooTAo2d4MwJC2erg==", 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:17:33.391 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmznL4NxAAjWpKiTDlC7xig+1BvdP3xw==", 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDVKSpm1AZVBBAAKWhbDCoc9ozE4gTVuwI3ew==", 2024-04-25T10:17:33.392 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpm2opyAhAAXejV4stcq6xezBOdShNG5w==", 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.393 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmbBbwFRAA5Eyp+oYH7cJpSLmgHS9gTA==", 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-04-25T10:17:33.394 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDUKSpmcwgfExAAXDMu4WJ3QrrTEpSHQElupg==", 2024-04-25T10:17:33.395 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-04-25T10:17:33.395 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-04-25T10:17:33.395 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-04-25T10:17:33.395 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-04-25T10:17:33.395 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.395 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:33.395 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T10:17:33.395 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:17:33.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 -- 172.21.15.133:0/822060490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ba0066f60 msgr2=0x7f1ba0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ba0066f60 0x7f1ba0069420 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40af000 tx=0x7f1ba40a7000 comp rx=0 tx=0).stop 2024-04-25T10:17:33.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 -- 172.21.15.133:0/822060490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb40ce330 msgr2=0x7f1bb41567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb40ce330 0x7f1bb41567c0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800d5f0 tx=0x7f1ba800dab0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 -- 172.21.15.133:0/822060490 shutdown_connections 2024-04-25T10:17:33.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bb415c9a0 0x7f1bb40b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ba0066f60 0x7f1ba0069420 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bb40ce330 0x7f1bb41567c0 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 --2- 172.21.15.133:0/822060490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bb40b1f70 0x7f1bb4151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 -- 172.21.15.133:0/822060490 >> 172.21.15.133:0/822060490 conn(0x7f1bb4067000 msgr2=0x7f1bb40b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:33.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 -- 172.21.15.133:0/822060490 shutdown_connections 2024-04-25T10:17:33.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.379+0000 7f1bbaf34640 1 -- 172.21.15.133:0/822060490 wait complete. 2024-04-25T10:17:33.399 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:33.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.503+0000 7f0f6b9dc640 1 Processor -- start 2024-04-25T10:17:33.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.503+0000 7f0f6b9dc640 1 -- start start 2024-04-25T10:17:33.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f6b9dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f640b1f70 0x7f0f6406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f6b9dc640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0f640ce330 0x7f0f640ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f6b9dc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f640c61d0 con 0x7f0f6415c9a0 2024-04-25T10:17:33.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f6b9dc640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f640cece0 con 0x7f0f640ce330 2024-04-25T10:17:33.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f6b9dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f640c64b0 con 0x7f0f640b1f70 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f69778640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0f6415c9a0 0x7f0f640c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55830/0 (socket says 172.21.15.133:55830) 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f69778640 1 -- 172.21.15.133:0/516279364 learned_addr learned my addr 172.21.15.133:0/516279364 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f69f79640 1 --2- 172.21.15.133:0/516279364 >> v2:172.21.15.162:3300/0 conn(0x7f0f640ce330 0x7f0f640ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f68f77640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f640b1f70 0x7f0f6406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583234311 0 0) 0x7f0f640c61d0 con 0x7f0f6415c9a0 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f50003530 con 0x7f0f6415c9a0 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1392318073 0 0) 0x7f0f50003530 con 0x7f0f6415c9a0 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f640b1f70 msgr2=0x7f0f6406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f640b1f70 0x7f0f6406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 >> v2:172.21.15.162:3300/0 conn(0x7f0f640ce330 msgr2=0x7f0f640ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 --2- 172.21.15.133:0/516279364 >> v2:172.21.15.162:3300/0 conn(0x7f0f640ce330 0x7f0f640ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6406d4f0 con 0x7f0f6415c9a0 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0f60001680 con 0x7f0f6415c9a0 2024-04-25T10:17:33.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 >> v1:172.21.15.133:6789/0 conn(0x7f0f6415c9a0 legacy=0x7f0f640c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f500039c0 0x7f0f50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f640b1f70 0x7f0f50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f50004d50 0x7f0f500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f640cece0 con 0x7f0f500039c0 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f640c64b0 con 0x7f0f50004d50 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5000b9d0 con 0x7f0f640b1f70 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f69778640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f500039c0 0x7f0f50003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f68f77640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f640b1f70 0x7f0f50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f69f79640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f50004d50 0x7f0f500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f69778640 1 -- 172.21.15.133:0/516279364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f640b1f70 msgr2=0x7f0f50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f69778640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f640b1f70 0x7f0f50004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f69778640 1 -- 172.21.15.133:0/516279364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f50004d50 msgr2=0x7f0f500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f69778640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f50004d50 0x7f0f500051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f69778640 1 -- 172.21.15.133:0/516279364 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50003530 con 0x7f0f500039c0 2024-04-25T10:17:33.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f69778640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f500039c0 0x7f0f50003de0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000c210 tx=0x7f0f6001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83732207e9b3ff0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60007d60 con 0x7f0f500039c0 2024-04-25T10:17:33.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f600a9020 con 0x7f0f500039c0 2024-04-25T10:17:33.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.507+0000 7f0f5bfff640 1 -- 172.21.15.133:0/516279364 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60005b20 con 0x7f0f500039c0 2024-04-25T10:17:33.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/516279364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f500039c0 msgr2=0x7f0f50003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 --2- 172.21.15.133:0/516279364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f500039c0 0x7f0f50003de0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000c210 tx=0x7f0f6001cc70 comp rx=0 tx=0).stop 2024-04-25T10:17:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f69f79640 1 -- 172.21.15.133:0/516279364 reap_dead start 2024-04-25T10:17:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/516279364 shutdown_connections 2024-04-25T10:17:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/516279364 >> 172.21.15.133:0/516279364 conn(0x7f0f64067000 msgr2=0x7f0f6406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/516279364 shutdown_connections 2024-04-25T10:17:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/516279364 wait complete. 2024-04-25T10:17:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 Processor -- start 2024-04-25T10:17:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 -- start start 2024-04-25T10:17:33.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f500039c0 0x7f0f64161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6414cda0 0x7f0f640cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f640ccef0 0x7f0f640c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6415d510 con 0x7f0f640ccef0 2024-04-25T10:17:33.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6415d390 con 0x7f0f6414cda0 2024-04-25T10:17:33.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6415d690 con 0x7f0f500039c0 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f69f79640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f640ccef0 0x7f0f640c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f69f79640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f640ccef0 0x7f0f640c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55492/0 (socket says 172.21.15.133:55492) 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f69f79640 1 -- 172.21.15.133:0/2909835299 learned_addr learned my addr 172.21.15.133:0/2909835299 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f69778640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f500039c0 0x7f0f64161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f69f79640 1 -- 172.21.15.133:0/2909835299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f500039c0 msgr2=0x7f0f64161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f69f79640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f500039c0 0x7f0f64161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f69f79640 1 -- 172.21.15.133:0/2909835299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6414cda0 msgr2=0x7f0f640cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f69f79640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6414cda0 0x7f0f640cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f69f79640 1 -- 172.21.15.133:0/2909835299 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f64161eb0 con 0x7f0f640ccef0 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f69f79640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f640ccef0 0x7f0f640c7c50 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f0f6415f4d0 tx=0x7f0f5c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f59ffb640 1 -- 172.21.15.133:0/2909835299 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c016620 con 0x7f0f640ccef0 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f59ffb640 1 -- 172.21.15.133:0/2909835299 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5c01b070 con 0x7f0f640ccef0 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/2909835299 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f641608d0 con 0x7f0f640ccef0 2024-04-25T10:17:33.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f59ffb640 1 -- 172.21.15.133:0/2909835299 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c016a20 con 0x7f0f640ccef0 2024-04-25T10:17:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.511+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/2909835299 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f64160e90 con 0x7f0f640ccef0 2024-04-25T10:17:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.515+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/2909835299 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f64068ef0 con 0x7f0f640ccef0 2024-04-25T10:17:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.515+0000 7f0f59ffb640 1 -- 172.21.15.133:0/2909835299 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f5c016bc0 con 0x7f0f640ccef0 2024-04-25T10:17:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.515+0000 7f0f59ffb640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f48066f60 0x7f0f48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.515+0000 7f0f59ffb640 1 -- 172.21.15.133:0/2909835299 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0f5c085ca0 con 0x7f0f640ccef0 2024-04-25T10:17:33.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.515+0000 7f0f69778640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f48066f60 0x7f0f48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.515+0000 7f0f69778640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f48066f60 0x7f0f48069420 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0f600aa000 tx=0x7f0f60000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.519+0000 7f0f59ffb640 1 -- 172.21.15.133:0/2909835299 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f5c025020 con 0x7f0f640ccef0 2024-04-25T10:17:33.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.679+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/2909835299 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f6415de50 con 0x7f0f640ccef0 2024-04-25T10:17:33.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.679+0000 7f0f59ffb640 1 -- 172.21.15.133:0/2909835299 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+13198 (secure 0 0 0) 0x7f0f64161eb0 con 0x7f0f640ccef0 2024-04-25T10:17:33.684 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:33.685 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":191,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:32.994604+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:33.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/2909835299 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f48066f60 msgr2=0x7f0f48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f48066f60 0x7f0f48069420 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0f600aa000 tx=0x7f0f60000ff0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/2909835299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f640ccef0 msgr2=0x7f0f640c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f640ccef0 0x7f0f640c7c50 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f0f6415f4d0 tx=0x7f0f5c00f520 comp rx=0 tx=0).stop 2024-04-25T10:17:33.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/2909835299 shutdown_connections 2024-04-25T10:17:33.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f48066f60 0x7f0f48069420 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f640ccef0 0x7f0f640c7c50 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f6414cda0 0x7f0f640cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 --2- 172.21.15.133:0/2909835299 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f500039c0 0x7f0f64161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/2909835299 >> 172.21.15.133:0/2909835299 conn(0x7f0f64067000 msgr2=0x7f0f640b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:33.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/2909835299 shutdown_connections 2024-04-25T10:17:33.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.683+0000 7f0f6b9dc640 1 -- 172.21.15.133:0/2909835299 wait complete. 2024-04-25T10:17:33.702 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:33.703 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:33.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bc2613640 1 Processor -- start 2024-04-25T10:17:33.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bc2613640 1 -- start start 2024-04-25T10:17:33.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bc2613640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3bbc0b5ef0 0x7f3bbc0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bc2613640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f3bbc06b3a0 0x7f3bbc0cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bc2613640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bbc0d0300 con 0x7f3bbc0d1060 2024-04-25T10:17:33.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bc2613640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bbc0cfaa0 con 0x7f3bbc0b5ef0 2024-04-25T10:17:33.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bc2613640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bbc0cfd40 con 0x7f3bbc06b3a0 2024-04-25T10:17:33.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bbbfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3bbc0b5ef0 0x7f3bbc0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bbb7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3bbc0d1060 0x7f3bbc06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55838/0 (socket says 172.21.15.133:55838) 2024-04-25T10:17:33.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bbb7fe640 1 -- 172.21.15.133:0/2212765163 learned_addr learned my addr 172.21.15.133:0/2212765163 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:33.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bc0bb0640 1 --2- 172.21.15.133:0/2212765163 >> v2:172.21.15.162:3300/0 conn(0x7f3bbc06b3a0 0x7f3bbc0cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bbbfff640 1 -- 172.21.15.133:0/2212765163 >> v2:172.21.15.162:3300/0 conn(0x7f3bbc06b3a0 msgr2=0x7f3bbc0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:33.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bbbfff640 1 --2- 172.21.15.133:0/2212765163 >> v2:172.21.15.162:3300/0 conn(0x7f3bbc06b3a0 0x7f3bbc0cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bbbfff640 1 -- 172.21.15.133:0/2212765163 >> v1:172.21.15.133:6789/0 conn(0x7f3bbc0d1060 legacy=0x7f3bbc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bbaffd640 1 -- 172.21.15.133:0/2212765163 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314517505 0 0) 0x7f3bbc0d0300 con 0x7f3bbc0d1060 2024-04-25T10:17:33.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bbbfff640 1 -- 172.21.15.133:0/2212765163 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc0d0020 con 0x7f3bbc0b5ef0 2024-04-25T10:17:33.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.827+0000 7f3bbbfff640 1 --2- 172.21.15.133:0/2212765163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bbc0b5ef0 0x7f3bbc0ce140 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc071be0 tx=0x7f3bac01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80058cddd8b25f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bbaffd640 1 -- 172.21.15.133:0/2212765163 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0ab650 con 0x7f3bbc0b5ef0 2024-04-25T10:17:33.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bbaffd640 1 -- 172.21.15.133:0/2212765163 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bac0b0070 con 0x7f3bbc0b5ef0 2024-04-25T10:17:33.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bbaffd640 1 -- 172.21.15.133:0/2212765163 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0aba10 con 0x7f3bbc0b5ef0 2024-04-25T10:17:33.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 -- 172.21.15.133:0/2212765163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bbc0b5ef0 msgr2=0x7f3bbc0ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 --2- 172.21.15.133:0/2212765163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bbc0b5ef0 0x7f3bbc0ce140 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc071be0 tx=0x7f3bac01c450 comp rx=0 tx=0).stop 2024-04-25T10:17:33.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 -- 172.21.15.133:0/2212765163 shutdown_connections 2024-04-25T10:17:33.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 --2- 172.21.15.133:0/2212765163 >> v2:172.21.15.162:3300/0 conn(0x7f3bbc06b3a0 0x7f3bbc0cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 --2- 172.21.15.133:0/2212765163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bbc0b5ef0 0x7f3bbc0ce140 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 -- 172.21.15.133:0/2212765163 >> 172.21.15.133:0/2212765163 conn(0x7f3bbc0d0570 msgr2=0x7f3bbc0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:33.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 -- 172.21.15.133:0/2212765163 shutdown_connections 2024-04-25T10:17:33.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 -- 172.21.15.133:0/2212765163 wait complete. 2024-04-25T10:17:33.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 Processor -- start 2024-04-25T10:17:33.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 -- start start 2024-04-25T10:17:33.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bbc06b3a0 0x7f3bbc1603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bbc0b5ef0 0x7f3bbc155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bbc0d1060 0x7f3bbc0b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0b6450 con 0x7f3bbc0d1060 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0b62d0 con 0x7f3bbc06b3a0 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0cc220 con 0x7f3bbc0b5ef0 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc0bb0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bbc0d1060 0x7f3bbc0b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc0bb0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bbc0d1060 0x7f3bbc0b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55514/0 (socket says 172.21.15.133:55514) 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc0bb0640 1 -- 172.21.15.133:0/1280931877 learned_addr learned my addr 172.21.15.133:0/1280931877 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bbbfff640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bbc06b3a0 0x7f3bbc1603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bbb7fe640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bbc0b5ef0 0x7f3bbc155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc0bb0640 1 -- 172.21.15.133:0/1280931877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bbc0b5ef0 msgr2=0x7f3bbc155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc0bb0640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bbc0b5ef0 0x7f3bbc155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc0bb0640 1 -- 172.21.15.133:0/1280931877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bbc06b3a0 msgr2=0x7f3bbc1603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc0bb0640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bbc06b3a0 0x7f3bbc1603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc0bb0640 1 -- 172.21.15.133:0/1280931877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc0cdd90 con 0x7f3bbc0d1060 2024-04-25T10:17:33.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bbb7fe640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bbc0b5ef0 0x7f3bbc155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:33.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc0bb0640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bbc0d1060 0x7f3bbc0b2250 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4007de0 tx=0x7f3bb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bb8ff9640 1 -- 172.21.15.133:0/1280931877 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4014620 con 0x7f3bbc0d1060 2024-04-25T10:17:33.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bb8ff9640 1 -- 172.21.15.133:0/1280931877 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb4019070 con 0x7f3bbc0d1060 2024-04-25T10:17:33.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 -- 172.21.15.133:0/1280931877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc0cd7d0 con 0x7f3bbc0d1060 2024-04-25T10:17:33.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bb8ff9640 1 -- 172.21.15.133:0/1280931877 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4014a20 con 0x7f3bbc0d1060 2024-04-25T10:17:33.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.831+0000 7f3bc2613640 1 -- 172.21.15.133:0/1280931877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc152ff0 con 0x7f3bbc0d1060 2024-04-25T10:17:33.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.835+0000 7f3bc2613640 1 -- 172.21.15.133:0/1280931877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc1531f0 con 0x7f3bbc0d1060 2024-04-25T10:17:33.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.835+0000 7f3bb8ff9640 1 -- 172.21.15.133:0/1280931877 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3bb4014bc0 con 0x7f3bbc0d1060 2024-04-25T10:17:33.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.835+0000 7f3bb8ff9640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ba8066f60 0x7f3ba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.835+0000 7f3bb8ff9640 1 -- 172.21.15.133:0/1280931877 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3bb4085710 con 0x7f3bbc0d1060 2024-04-25T10:17:33.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.835+0000 7f3bbbfff640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ba8066f60 0x7f3ba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.835+0000 7f3bbbfff640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ba8066f60 0x7f3ba8069420 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0af000 tx=0x7f3bac0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.839+0000 7f3bb8ff9640 1 -- 172.21.15.133:0/1280931877 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3bb4056970 con 0x7f3bbc0d1060 2024-04-25T10:17:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.999+0000 7f3bc2613640 1 -- 172.21.15.133:0/1280931877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3bbc067c70 con 0x7f3bbc0d1060 2024-04-25T10:17:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:33.999+0000 7f3bb8ff9640 1 -- 172.21.15.133:0/1280931877 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v191) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3bb4056270 con 0x7f3bbc0d1060 2024-04-25T10:17:34.003 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:17:34.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 -- 172.21.15.133:0/1280931877 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ba8066f60 msgr2=0x7f3ba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ba8066f60 0x7f3ba8069420 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0af000 tx=0x7f3bac0a7000 comp rx=0 tx=0).stop 2024-04-25T10:17:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 -- 172.21.15.133:0/1280931877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bbc0d1060 msgr2=0x7f3bbc0b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bbc0d1060 0x7f3bbc0b2250 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4007de0 tx=0x7f3bb400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 -- 172.21.15.133:0/1280931877 shutdown_connections 2024-04-25T10:17:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3ba8066f60 0x7f3ba8069420 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bbc0d1060 0x7f3bbc0b2250 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f3bbc0b5ef0 0x7f3bbc155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 --2- 172.21.15.133:0/1280931877 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3bbc06b3a0 0x7f3bbc1603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 -- 172.21.15.133:0/1280931877 >> 172.21.15.133:0/1280931877 conn(0x7f3bbc0d0570 msgr2=0x7f3bbc0d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 -- 172.21.15.133:0/1280931877 shutdown_connections 2024-04-25T10:17:34.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.003+0000 7f3bc2613640 1 -- 172.21.15.133:0/1280931877 wait complete. 2024-04-25T10:17:34.022 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:17:34.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.131+0000 7f0c830f7640 1 Processor -- start 2024-04-25T10:17:34.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.131+0000 7f0c830f7640 1 -- start start 2024-04-25T10:17:34.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c830f7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0c7c06eb70 0x7f0c7c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c830f7640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0c7c1584f0 0x7f0c7c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c830f7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7c0cbe60 con 0x7f0c7c158110 2024-04-25T10:17:34.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c830f7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c7c0b19b0 con 0x7f0c7c06eb70 2024-04-25T10:17:34.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c830f7640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c7c15b360 con 0x7f0c7c1584f0 2024-04-25T10:17:34.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c80e93640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0c7c158110 0x7f0c7c06e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55854/0 (socket says 172.21.15.133:55854) 2024-04-25T10:17:34.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c80e93640 1 -- 172.21.15.133:0/2637277421 learned_addr learned my addr 172.21.15.133:0/2637277421 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:34.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c7bfff640 1 --2- 172.21.15.133:0/2637277421 >> v2:172.21.15.133:3300/0 conn(0x7f0c7c06eb70 0x7f0c7c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c81694640 1 --2- 172.21.15.133:0/2637277421 >> v2:172.21.15.162:3300/0 conn(0x7f0c7c1584f0 0x7f0c7c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c7bfff640 1 -- 172.21.15.133:0/2637277421 >> v2:172.21.15.162:3300/0 conn(0x7f0c7c1584f0 msgr2=0x7f0c7c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:34.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c7bfff640 1 --2- 172.21.15.133:0/2637277421 >> v2:172.21.15.162:3300/0 conn(0x7f0c7c1584f0 0x7f0c7c15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c7bfff640 1 -- 172.21.15.133:0/2637277421 >> v1:172.21.15.133:6789/0 conn(0x7f0c7c158110 legacy=0x7f0c7c06e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c7bfff640 1 -- 172.21.15.133:0/2637277421 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c15b640 con 0x7f0c7c06eb70 2024-04-25T10:17:34.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c7bfff640 1 --2- 172.21.15.133:0/2637277421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c06eb70 0x7f0c7c0b1f00 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0b2470 tx=0x7f0c7001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7402de6e80a896af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c7b7fe640 1 -- 172.21.15.133:0/2637277421 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700a7070 con 0x7f0c7c06eb70 2024-04-25T10:17:34.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c7b7fe640 1 -- 172.21.15.133:0/2637277421 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c700b0070 con 0x7f0c7c06eb70 2024-04-25T10:17:34.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.135+0000 7f0c7b7fe640 1 -- 172.21.15.133:0/2637277421 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700ab890 con 0x7f0c7c06eb70 2024-04-25T10:17:34.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- 172.21.15.133:0/2637277421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c06eb70 msgr2=0x7f0c7c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 --2- 172.21.15.133:0/2637277421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c06eb70 0x7f0c7c0b1f00 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0b2470 tx=0x7f0c7001c450 comp rx=0 tx=0).stop 2024-04-25T10:17:34.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- 172.21.15.133:0/2637277421 shutdown_connections 2024-04-25T10:17:34.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 --2- 172.21.15.133:0/2637277421 >> v2:172.21.15.162:3300/0 conn(0x7f0c7c1584f0 0x7f0c7c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 --2- 172.21.15.133:0/2637277421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c06eb70 0x7f0c7c0b1f00 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- 172.21.15.133:0/2637277421 >> 172.21.15.133:0/2637277421 conn(0x7f0c7c067000 msgr2=0x7f0c7c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- 172.21.15.133:0/2637277421 shutdown_connections 2024-04-25T10:17:34.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- 172.21.15.133:0/2637277421 wait complete. 2024-04-25T10:17:34.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 Processor -- start 2024-04-25T10:17:34.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- start start 2024-04-25T10:17:34.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c7c06eb70 0x7f0c7c140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c158110 0x7f0c7c14e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c7c1584f0 0x7f0c7c154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c0ccca0 con 0x7f0c7c158110 2024-04-25T10:17:34.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c0ccb20 con 0x7f0c7c1584f0 2024-04-25T10:17:34.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c0cce20 con 0x7f0c7c06eb70 2024-04-25T10:17:34.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c158110 0x7f0c7c14e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c158110 0x7f0c7c14e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55522/0 (socket says 172.21.15.133:55522) 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c7bfff640 1 -- 172.21.15.133:0/168172575 learned_addr learned my addr 172.21.15.133:0/168172575 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c81694640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c7c1584f0 0x7f0c7c154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c80e93640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c7c06eb70 0x7f0c7c140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c7bfff640 1 -- 172.21.15.133:0/168172575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c7c06eb70 msgr2=0x7f0c7c140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c7bfff640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c7c06eb70 0x7f0c7c140090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c7bfff640 1 -- 172.21.15.133:0/168172575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c7c1584f0 msgr2=0x7f0c7c154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c7bfff640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c7c1584f0 0x7f0c7c154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c7bfff640 1 -- 172.21.15.133:0/168172575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c15bfe0 con 0x7f0c7c158110 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c7bfff640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c158110 0x7f0c7c14e480 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f0c700ad040 tx=0x7f0c7001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c797fa640 1 -- 172.21.15.133:0/168172575 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700bf610 con 0x7f0c7c158110 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c797fa640 1 -- 172.21.15.133:0/168172575 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c700bf7d0 con 0x7f0c7c158110 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- 172.21.15.133:0/168172575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7c15c2c0 con 0x7f0c7c158110 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c797fa640 1 -- 172.21.15.133:0/168172575 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700bfbc0 con 0x7f0c7c158110 2024-04-25T10:17:34.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- 172.21.15.133:0/168172575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c15c880 con 0x7f0c7c158110 2024-04-25T10:17:34.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.139+0000 7f0c830f7640 1 -- 172.21.15.133:0/168172575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7c06bc20 con 0x7f0c7c158110 2024-04-25T10:17:34.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.143+0000 7f0c797fa640 1 -- 172.21.15.133:0/168172575 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c700bd020 con 0x7f0c7c158110 2024-04-25T10:17:34.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.143+0000 7f0c797fa640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0c68066f60 0x7f0c68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.143+0000 7f0c797fa640 1 -- 172.21.15.133:0/168172575 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0c7011fc80 con 0x7f0c7c158110 2024-04-25T10:17:34.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.143+0000 7f0c80e93640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0c68066f60 0x7f0c68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.143+0000 7f0c80e93640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0c68066f60 0x7f0c68069420 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000f70 tx=0x7f0c6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.147+0000 7f0c797fa640 1 -- 172.21.15.133:0/168172575 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c70016070 con 0x7f0c7c158110 2024-04-25T10:17:34.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.307+0000 7f0c830f7640 1 -- 172.21.15.133:0/168172575 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0c7c068ef0 con 0x7f0c7c158110 2024-04-25T10:17:34.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.307+0000 7f0c797fa640 1 -- 172.21.15.133:0/168172575 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v191) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0c700f1060 con 0x7f0c7c158110 2024-04-25T10:17:34.311 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-04-25T10:17:34.312 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-04-25T10:17:34.312 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-04-25T10:17:34.312 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-04-25T10:17:34.312 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-04-25T10:17:34.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 -- 172.21.15.133:0/168172575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0c68066f60 msgr2=0x7f0c68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0c68066f60 0x7f0c68069420 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000f70 tx=0x7f0c6c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:34.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 -- 172.21.15.133:0/168172575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c158110 msgr2=0x7f0c7c14e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c158110 0x7f0c7c14e480 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f0c700ad040 tx=0x7f0c7001ac90 comp rx=0 tx=0).stop 2024-04-25T10:17:34.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 -- 172.21.15.133:0/168172575 shutdown_connections 2024-04-25T10:17:34.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c7c1584f0 0x7f0c7c154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0c68066f60 0x7f0c68069420 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c7c158110 0x7f0c7c14e480 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 --2- 172.21.15.133:0/168172575 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0c7c06eb70 0x7f0c7c140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 -- 172.21.15.133:0/168172575 >> 172.21.15.133:0/168172575 conn(0x7f0c7c067000 msgr2=0x7f0c7c0cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 -- 172.21.15.133:0/168172575 shutdown_connections 2024-04-25T10:17:34.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.311+0000 7f0c830f7640 1 -- 172.21.15.133:0/168172575 wait complete. 2024-04-25T10:17:34.329 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:17:34.330 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:17:34.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f16a558e640 1 Processor -- start 2024-04-25T10:17:34.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f16a558e640 1 -- start start 2024-04-25T10:17:34.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f16a558e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f16a00b4940 0x7f16a00b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f16a558e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f16a015c420 0x7f16a00cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f16a558e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a00b5260 con 0x7f16a0071cb0 2024-04-25T10:17:34.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f16a558e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a0072a90 con 0x7f16a00b4940 2024-04-25T10:17:34.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f16a558e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a00a85d0 con 0x7f16a015c420 2024-04-25T10:17:34.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f169effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f16a00b4940 0x7f16a00b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f169e7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f16a0071cb0 0x7f16a015bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55856/0 (socket says 172.21.15.133:55856) 2024-04-25T10:17:34.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f169e7fc640 1 -- 172.21.15.133:0/1719531684 learned_addr learned my addr 172.21.15.133:0/1719531684 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:34.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f169f7fe640 1 --2- 172.21.15.133:0/1719531684 >> v2:172.21.15.162:3300/0 conn(0x7f16a015c420 0x7f16a00cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f169effd640 1 -- 172.21.15.133:0/1719531684 >> v2:172.21.15.162:3300/0 conn(0x7f16a015c420 msgr2=0x7f16a00cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:34.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f169effd640 1 --2- 172.21.15.133:0/1719531684 >> v2:172.21.15.162:3300/0 conn(0x7f16a015c420 0x7f16a00cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f169effd640 1 -- 172.21.15.133:0/1719531684 >> v1:172.21.15.133:6789/0 conn(0x7f16a0071cb0 legacy=0x7f16a015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.451+0000 7f169effd640 1 -- 172.21.15.133:0/1719531684 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a015c9b0 con 0x7f16a00b4940 2024-04-25T10:17:34.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f169effd640 1 --2- 172.21.15.133:0/1719531684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a00b4940 0x7f16a00b4d20 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f16a00cbfc0 tx=0x7f169001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2826c0196fb655a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f169dffb640 1 -- 172.21.15.133:0/1719531684 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900ab650 con 0x7f16a00b4940 2024-04-25T10:17:34.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f169dffb640 1 -- 172.21.15.133:0/1719531684 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16900b0070 con 0x7f16a00b4940 2024-04-25T10:17:34.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f169dffb640 1 -- 172.21.15.133:0/1719531684 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900aba10 con 0x7f16a00b4940 2024-04-25T10:17:34.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f16a558e640 1 -- 172.21.15.133:0/1719531684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a00b4940 msgr2=0x7f16a00b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f16a558e640 1 --2- 172.21.15.133:0/1719531684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a00b4940 0x7f16a00b4d20 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f16a00cbfc0 tx=0x7f169001c450 comp rx=0 tx=0).stop 2024-04-25T10:17:34.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f16a558e640 1 -- 172.21.15.133:0/1719531684 shutdown_connections 2024-04-25T10:17:34.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f16a558e640 1 --2- 172.21.15.133:0/1719531684 >> v2:172.21.15.162:3300/0 conn(0x7f16a015c420 0x7f16a00cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f16a558e640 1 --2- 172.21.15.133:0/1719531684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a00b4940 0x7f16a00b4d20 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f16a558e640 1 -- 172.21.15.133:0/1719531684 >> 172.21.15.133:0/1719531684 conn(0x7f16a006d370 msgr2=0x7f16a006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f16a558e640 1 -- 172.21.15.133:0/1719531684 shutdown_connections 2024-04-25T10:17:34.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.455+0000 7f16a558e640 1 -- 172.21.15.133:0/1719531684 wait complete. 2024-04-25T10:17:34.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16a558e640 1 Processor -- start 2024-04-25T10:17:34.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16a558e640 1 -- start start 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16a558e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16a0071cb0 0x7f16a0138050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16a558e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a00b4940 0x7f16a01429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16a558e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16a015c420 0x7f16a015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16a558e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01452d0 con 0x7f16a00b4940 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16a558e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0071790 con 0x7f16a0071cb0 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16a558e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0145450 con 0x7f16a015c420 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a00b4940 0x7f16a01429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a00b4940 0x7f16a01429d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55538/0 (socket says 172.21.15.133:55538) 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169e7fc640 1 -- 172.21.15.133:0/1817999744 learned_addr learned my addr 172.21.15.133:0/1817999744 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169effd640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16a0071cb0 0x7f16a0138050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169f7fe640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16a015c420 0x7f16a015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169e7fc640 1 -- 172.21.15.133:0/1817999744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16a015c420 msgr2=0x7f16a015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169e7fc640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16a015c420 0x7f16a015a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169e7fc640 1 -- 172.21.15.133:0/1817999744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16a0071cb0 msgr2=0x7f16a0138050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169e7fc640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16a0071cb0 0x7f16a0138050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169e7fc640 1 -- 172.21.15.133:0/1817999744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a0142f10 con 0x7f16a00b4940 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169f7fe640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16a015c420 0x7f16a015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169e7fc640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a00b4940 0x7f16a01429d0 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f169400c450 tx=0x7f169400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16877fe640 1 -- 172.21.15.133:0/1817999744 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694017070 con 0x7f16a00b4940 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16877fe640 1 -- 172.21.15.133:0/1817999744 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1694005cf0 con 0x7f16a00b4940 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16a558e640 1 -- 172.21.15.133:0/1817999744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a013e9a0 con 0x7f16a00b4940 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16877fe640 1 -- 172.21.15.133:0/1817999744 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694007290 con 0x7f16a00b4940 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16a558e640 1 -- 172.21.15.133:0/1817999744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a0141ef0 con 0x7f16a00b4940 2024-04-25T10:17:34.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16a558e640 1 -- 172.21.15.133:0/1817999744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a0069000 con 0x7f16a00b4940 2024-04-25T10:17:34.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16877fe640 1 -- 172.21.15.133:0/1817999744 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f169401a070 con 0x7f16a00b4940 2024-04-25T10:17:34.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16877fe640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f168c066f90 0x7f168c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f169effd640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f168c066f90 0x7f168c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.459+0000 7f16877fe640 1 -- 172.21.15.133:0/1817999744 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1694084170 con 0x7f16a00b4940 2024-04-25T10:17:34.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.463+0000 7f169effd640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f168c066f90 0x7f168c069450 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f16900af000 tx=0x7f16900a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.463+0000 7f16877fe640 1 -- 172.21.15.133:0/1817999744 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1694055340 con 0x7f16a00b4940 2024-04-25T10:17:34.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:34.619+0000 7f16a558e640 1 -- 172.21.15.133:0/1817999744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f16a00b1080 con 0x7f16a00b4940 2024-04-25T10:17:35.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.591+0000 7f16877fe640 1 -- 172.21.15.133:0/1817999744 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v192) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1694054c40 con 0x7f16a00b4940 2024-04-25T10:17:35.596 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-04-25T10:17:35.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.595+0000 7f16a558e640 1 -- 172.21.15.133:0/1817999744 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f168c066f90 msgr2=0x7f168c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.595+0000 7f16a558e640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f168c066f90 0x7f168c069450 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f16900af000 tx=0x7f16900a7000 comp rx=0 tx=0).stop 2024-04-25T10:17:35.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.595+0000 7f16a558e640 1 -- 172.21.15.133:0/1817999744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a00b4940 msgr2=0x7f16a01429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.595+0000 7f16a558e640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a00b4940 0x7f16a01429d0 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f169400c450 tx=0x7f169400c910 comp rx=0 tx=0).stop 2024-04-25T10:17:35.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.595+0000 7f16a558e640 1 -- 172.21.15.133:0/1817999744 shutdown_connections 2024-04-25T10:17:35.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.595+0000 7f16a558e640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f16a015c420 0x7f16a015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.595+0000 7f16a558e640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f168c066f90 0x7f168c069450 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.595+0000 7f16a558e640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a00b4940 0x7f16a01429d0 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.595+0000 7f16a558e640 1 --2- 172.21.15.133:0/1817999744 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f16a0071cb0 0x7f16a0138050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.595+0000 7f16a558e640 1 -- 172.21.15.133:0/1817999744 >> 172.21.15.133:0/1817999744 conn(0x7f16a006d370 msgr2=0x7f16a006d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.599+0000 7f16a558e640 1 -- 172.21.15.133:0/1817999744 shutdown_connections 2024-04-25T10:17:35.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.599+0000 7f16a558e640 1 -- 172.21.15.133:0/1817999744 wait complete. 2024-04-25T10:17:35.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:17:35.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80d012640 1 Processor -- start 2024-04-25T10:17:35.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80d012640 1 -- start start 2024-04-25T10:17:35.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80d012640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd8080ce990 0x7fd8080d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80d012640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8080ced70 0x7fd8080ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80d012640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080cb2e0 con 0x7fd8080ce1e0 2024-04-25T10:17:35.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80d012640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080cb5e0 con 0x7fd8080ced70 2024-04-25T10:17:35.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80d012640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080cd920 con 0x7fd8080ce990 2024-04-25T10:17:35.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80659c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8080ced70 0x7fd8080ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80759e640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd8080ce1e0 0x7fd80815da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55866/0 (socket says 172.21.15.133:55866) 2024-04-25T10:17:35.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80759e640 1 -- 172.21.15.133:0/2169695026 learned_addr learned my addr 172.21.15.133:0/2169695026 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:35.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd806d9d640 1 --2- 172.21.15.133:0/2169695026 >> v2:172.21.15.162:3300/0 conn(0x7fd8080ce990 0x7fd8080d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80659c640 1 -- 172.21.15.133:0/2169695026 >> v2:172.21.15.162:3300/0 conn(0x7fd8080ce990 msgr2=0x7fd8080d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:35.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80659c640 1 --2- 172.21.15.133:0/2169695026 >> v2:172.21.15.162:3300/0 conn(0x7fd8080ce990 0x7fd8080d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80659c640 1 -- 172.21.15.133:0/2169695026 >> v1:172.21.15.133:6789/0 conn(0x7fd8080ce1e0 legacy=0x7fd80815da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd805d9b640 1 -- 172.21.15.133:0/2169695026 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862095268 0 0) 0x7fd8080cb2e0 con 0x7fd8080ce1e0 2024-04-25T10:17:35.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.755+0000 7fd80659c640 1 -- 172.21.15.133:0/2169695026 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8080cdbc0 con 0x7fd8080ced70 2024-04-25T10:17:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80659c640 1 --2- 172.21.15.133:0/2169695026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080ced70 0x7fd8080ad330 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7fd808099f00 tx=0x7fd7fc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc7836f29efd4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd805d9b640 1 -- 172.21.15.133:0/2169695026 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0aa600 con 0x7fd8080ced70 2024-04-25T10:17:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd805d9b640 1 -- 172.21.15.133:0/2169695026 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc0af070 con 0x7fd8080ced70 2024-04-25T10:17:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd805d9b640 1 -- 172.21.15.133:0/2169695026 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0aa9c0 con 0x7fd8080ced70 2024-04-25T10:17:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 -- 172.21.15.133:0/2169695026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080ced70 msgr2=0x7fd8080ad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 --2- 172.21.15.133:0/2169695026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080ced70 0x7fd8080ad330 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7fd808099f00 tx=0x7fd7fc01c480 comp rx=0 tx=0).stop 2024-04-25T10:17:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 -- 172.21.15.133:0/2169695026 shutdown_connections 2024-04-25T10:17:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 --2- 172.21.15.133:0/2169695026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080ced70 0x7fd8080ad330 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 --2- 172.21.15.133:0/2169695026 >> v2:172.21.15.162:3300/0 conn(0x7fd8080ce990 0x7fd8080d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 -- 172.21.15.133:0/2169695026 >> 172.21.15.133:0/2169695026 conn(0x7fd8080999d0 msgr2=0x7fd8080d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 -- 172.21.15.133:0/2169695026 shutdown_connections 2024-04-25T10:17:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 -- 172.21.15.133:0/2169695026 wait complete. 2024-04-25T10:17:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 Processor -- start 2024-04-25T10:17:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 -- start start 2024-04-25T10:17:35.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8080ce1e0 0x7fd8081479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080ce990 0x7fd80815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080ced70 0x7fd80815b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080cc3e0 con 0x7fd8080ce990 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080cc260 con 0x7fd8080ced70 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.759+0000 7fd80d012640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd808167960 con 0x7fd8080ce1e0 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080ce990 0x7fd80815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080ce990 0x7fd80815ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55568/0 (socket says 172.21.15.133:55568) 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80659c640 1 -- 172.21.15.133:0/1828821047 learned_addr learned my addr 172.21.15.133:0/1828821047 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80759e640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080ced70 0x7fd80815b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd806d9d640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8080ce1e0 0x7fd8081479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80659c640 1 -- 172.21.15.133:0/1828821047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8080ce1e0 msgr2=0x7fd8081479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80659c640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8080ce1e0 0x7fd8081479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80659c640 1 -- 172.21.15.133:0/1828821047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080ced70 msgr2=0x7fd80815b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80659c640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080ced70 0x7fd80815b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80659c640 1 -- 172.21.15.133:0/1828821047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80806bbd0 con 0x7fd8080ce990 2024-04-25T10:17:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd806d9d640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8080ce1e0 0x7fd8081479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:35.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80759e640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080ced70 0x7fd80815b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:35.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80659c640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080ce990 0x7fd80815ad20 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc019090 tx=0x7fd7fc019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd7ef7fe640 1 -- 172.21.15.133:0/1828821047 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0aad80 con 0x7fd8080ce990 2024-04-25T10:17:35.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd7ef7fe640 1 -- 172.21.15.133:0/1828821047 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc0be450 con 0x7fd8080ce990 2024-04-25T10:17:35.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80d012640 1 -- 172.21.15.133:0/1828821047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8080ca420 con 0x7fd8080ce990 2024-04-25T10:17:35.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd7ef7fe640 1 -- 172.21.15.133:0/1828821047 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0be840 con 0x7fd8080ce990 2024-04-25T10:17:35.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80d012640 1 -- 172.21.15.133:0/1828821047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8080b4030 con 0x7fd8080ce990 2024-04-25T10:17:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd80d012640 1 -- 172.21.15.133:0/1828821047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd808067c70 con 0x7fd8080ce990 2024-04-25T10:17:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd7ef7fe640 1 -- 172.21.15.133:0/1828821047 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7fc0a9070 con 0x7fd8080ce990 2024-04-25T10:17:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd7ef7fe640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 0x7fd7f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd7ef7fe640 1 -- 172.21.15.133:0/1828821047 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fd7fc11f5f0 con 0x7fd8080ce990 2024-04-25T10:17:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd806d9d640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 0x7fd7f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.763+0000 7fd806d9d640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 0x7fd7f4069420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd800007330 tx=0x7fd80000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.767+0000 7fd7ef7fe640 1 -- 172.21.15.133:0/1828821047 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7fc016070 con 0x7fd8080ce990 2024-04-25T10:17:35.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:35.931+0000 7fd80d012640 1 -- 172.21.15.133:0/1828821047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd8080d0850 con 0x7fd8080ce990 2024-04-25T10:17:36.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.595+0000 7fd7ef7fe640 1 -- 172.21.15.133:0/1828821047 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v193) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd7fc0f07c0 con 0x7fd8080ce990 2024-04-25T10:17:36.602 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-04-25T10:17:36.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 -- 172.21.15.133:0/1828821047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 msgr2=0x7fd7f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 0x7fd7f4069420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd800007330 tx=0x7fd80000b040 comp rx=0 tx=0).stop 2024-04-25T10:17:36.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 -- 172.21.15.133:0/1828821047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080ce990 msgr2=0x7fd80815ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080ce990 0x7fd80815ad20 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc019090 tx=0x7fd7fc019220 comp rx=0 tx=0).stop 2024-04-25T10:17:36.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 -- 172.21.15.133:0/1828821047 shutdown_connections 2024-04-25T10:17:36.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd8080ced70 0x7fd80815b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd7f4066f60 0x7fd7f4069420 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8080ce990 0x7fd80815ad20 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 --2- 172.21.15.133:0/1828821047 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd8080ce1e0 0x7fd8081479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 -- 172.21.15.133:0/1828821047 >> 172.21.15.133:0/1828821047 conn(0x7fd8080999d0 msgr2=0x7fd8080aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 -- 172.21.15.133:0/1828821047 shutdown_connections 2024-04-25T10:17:36.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.599+0000 7fd80d012640 1 -- 172.21.15.133:0/1828821047 wait complete. 2024-04-25T10:17:36.634 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:17:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.751+0000 7f4391691640 1 Processor -- start 2024-04-25T10:17:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.751+0000 7f4391691640 1 -- start start 2024-04-25T10:17:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f4391691640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f438c15f020 0x7f438c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f4391691640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f438c06f8f0 0x7f438c06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f4391691640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438c0cefa0 con 0x7f438c0d21c0 2024-04-25T10:17:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f4391691640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438c0cd690 con 0x7f438c06f8f0 2024-04-25T10:17:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f4391691640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0ccdf0 con 0x7f438c15f020 2024-04-25T10:17:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f438b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f438c06f8f0 0x7f438c06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f438a7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f438c0d21c0 0x7f438c0cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49974/0 (socket says 172.21.15.133:49974) 2024-04-25T10:17:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f438a7fc640 1 -- 172.21.15.133:0/3922148491 learned_addr learned my addr 172.21.15.133:0/3922148491 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f438affd640 1 --2- 172.21.15.133:0/3922148491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f438c15f020 0x7f438c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f438b7fe640 1 -- 172.21.15.133:0/3922148491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f438c15f020 msgr2=0x7f438c0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f438b7fe640 1 --2- 172.21.15.133:0/3922148491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f438c15f020 0x7f438c0cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f438b7fe640 1 -- 172.21.15.133:0/3922148491 >> v1:172.21.15.133:6789/0 conn(0x7f438c0d21c0 legacy=0x7f438c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f438b7fe640 1 -- 172.21.15.133:0/3922148491 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c0cd0d0 con 0x7f438c06f8f0 2024-04-25T10:17:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f4389ffb640 1 -- 172.21.15.133:0/3922148491 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002757356 0 0) 0x7f4380001120 con 0x7f438c0d21c0 2024-04-25T10:17:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.755+0000 7f438b7fe640 1 --2- 172.21.15.133:0/3922148491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f438c06f8f0 0x7f438c06fd80 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f4384009d90 tx=0x7f438401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60673a69b0bc8a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4389ffb640 1 -- 172.21.15.133:0/3922148491 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43840b2070 con 0x7f438c06f8f0 2024-04-25T10:17:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4389ffb640 1 -- 172.21.15.133:0/3922148491 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43840a5940 con 0x7f438c06f8f0 2024-04-25T10:17:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4389ffb640 1 -- 172.21.15.133:0/3922148491 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43840a5d10 con 0x7f438c06f8f0 2024-04-25T10:17:36.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 -- 172.21.15.133:0/3922148491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f438c06f8f0 msgr2=0x7f438c06fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 --2- 172.21.15.133:0/3922148491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f438c06f8f0 0x7f438c06fd80 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f4384009d90 tx=0x7f438401c970 comp rx=0 tx=0).stop 2024-04-25T10:17:36.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 -- 172.21.15.133:0/3922148491 shutdown_connections 2024-04-25T10:17:36.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 --2- 172.21.15.133:0/3922148491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f438c06f8f0 0x7f438c06fd80 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 --2- 172.21.15.133:0/3922148491 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f438c15f020 0x7f438c0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 -- 172.21.15.133:0/3922148491 >> 172.21.15.133:0/3922148491 conn(0x7f438c0d16a0 msgr2=0x7f438c0d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 -- 172.21.15.133:0/3922148491 shutdown_connections 2024-04-25T10:17:36.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 -- 172.21.15.133:0/3922148491 wait complete. 2024-04-25T10:17:36.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 Processor -- start 2024-04-25T10:17:36.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 -- start start 2024-04-25T10:17:36.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f438c06f8f0 0x7f438c0c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f438c0d21c0 0x7f438c0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f438c15f020 0x7f438c15b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c168560 con 0x7f438c0d21c0 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0d00d0 con 0x7f438c15f020 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c1686e0 con 0x7f438c06f8f0 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f438c0d21c0 0x7f438c0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438a7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f438c0d21c0 0x7f438c0ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46672/0 (socket says 172.21.15.133:46672) 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438a7fc640 1 -- 172.21.15.133:0/1514376365 learned_addr learned my addr 172.21.15.133:0/1514376365 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438affd640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f438c06f8f0 0x7f438c0c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438a7fc640 1 -- 172.21.15.133:0/1514376365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f438c06f8f0 msgr2=0x7f438c0c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438a7fc640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f438c06f8f0 0x7f438c0c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438a7fc640 1 -- 172.21.15.133:0/1514376365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f438c15f020 msgr2=0x7f438c15b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438b7fe640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f438c15f020 0x7f438c15b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438a7fc640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f438c15f020 0x7f438c15b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438a7fc640 1 -- 172.21.15.133:0/1514376365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c0c89c0 con 0x7f438c0d21c0 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438affd640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f438c06f8f0 0x7f438c0c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f438a7fc640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f438c0d21c0 0x7f438c0ca090 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f438000c4b0 tx=0x7f438000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f43737fe640 1 -- 172.21.15.133:0/1514376365 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380018070 con 0x7f438c0d21c0 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f43737fe640 1 -- 172.21.15.133:0/1514376365 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4380007500 con 0x7f438c0d21c0 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 -- 172.21.15.133:0/1514376365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438c0c8ca0 con 0x7f438c0d21c0 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f43737fe640 1 -- 172.21.15.133:0/1514376365 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380004a30 con 0x7f438c0d21c0 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.759+0000 7f4391691640 1 -- 172.21.15.133:0/1514376365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c0ca5d0 con 0x7f438c0d21c0 2024-04-25T10:17:36.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.763+0000 7f4391691640 1 -- 172.21.15.133:0/1514376365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438c0c8750 con 0x7f438c0d21c0 2024-04-25T10:17:36.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.763+0000 7f43737fe640 1 -- 172.21.15.133:0/1514376365 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f438001b070 con 0x7f438c0d21c0 2024-04-25T10:17:36.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.763+0000 7f43737fe640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f90 0x7f4378069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.763+0000 7f43737fe640 1 -- 172.21.15.133:0/1514376365 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f43800844c0 con 0x7f438c0d21c0 2024-04-25T10:17:36.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.763+0000 7f438affd640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f90 0x7f4378069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.763+0000 7f438affd640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f90 0x7f4378069450 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f437c009840 tx=0x7f437c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.767+0000 7f43737fe640 1 -- 172.21.15.133:0/1514376365 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f438c0cefa0 con 0x7f438c0d21c0 2024-04-25T10:17:36.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:36.963+0000 7f4391691640 1 -- 172.21.15.133:0/1514376365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f438c067610 con 0x7f438c0d21c0 2024-04-25T10:17:37.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.655+0000 7f43737fe640 1 -- 172.21.15.133:0/1514376365 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 29 and data pool 30 v116) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4380055310 con 0x7f438c0d21c0 2024-04-25T10:17:37.665 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 29 and data pool 30 2024-04-25T10:17:37.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 -- 172.21.15.133:0/1514376365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f90 msgr2=0x7f4378069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f90 0x7f4378069450 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f437c009840 tx=0x7f437c008040 comp rx=0 tx=0).stop 2024-04-25T10:17:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 -- 172.21.15.133:0/1514376365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f438c0d21c0 msgr2=0x7f438c0ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f438c0d21c0 0x7f438c0ca090 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f438000c4b0 tx=0x7f438000c970 comp rx=0 tx=0).stop 2024-04-25T10:17:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 -- 172.21.15.133:0/1514376365 shutdown_connections 2024-04-25T10:17:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f438c15f020 0x7f438c15b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4378066f90 0x7f4378069450 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f438c0d21c0 0x7f438c0ca090 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 --2- 172.21.15.133:0/1514376365 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f438c06f8f0 0x7f438c0c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 -- 172.21.15.133:0/1514376365 >> 172.21.15.133:0/1514376365 conn(0x7f438c0d16a0 msgr2=0x7f438c15d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 -- 172.21.15.133:0/1514376365 shutdown_connections 2024-04-25T10:17:37.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.671+0000 7f4391691640 1 -- 172.21.15.133:0/1514376365 wait complete. 2024-04-25T10:17:37.698 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e4ea14640 1 Processor -- start 2024-04-25T10:17:37.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e4ea14640 1 -- start start 2024-04-25T10:17:37.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e4ea14640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4e4815d9b0 0x7f4e480cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e4ea14640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e480cee80 0x7f4e4806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e4ea14640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e480ca9b0 con 0x7f4e4815dd90 2024-04-25T10:17:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e4ea14640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e480cbe30 con 0x7f4e4815d9b0 2024-04-25T10:17:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e4ea14640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e480cac90 con 0x7f4e480cee80 2024-04-25T10:17:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e47fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4e4815d9b0 0x7f4e480cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e47fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4e4815d9b0 0x7f4e480cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46688/0 (socket says 172.21.15.133:46688) 2024-04-25T10:17:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e47fff640 1 -- 172.21.15.133:0/595426543 learned_addr learned my addr 172.21.15.133:0/595426543 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e4cfb1640 1 --2- 172.21.15.133:0/595426543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e480cee80 0x7f4e4806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e47fff640 1 -- 172.21.15.133:0/595426543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e480cee80 msgr2=0x7f4e4806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:37.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e47fff640 1 --2- 172.21.15.133:0/595426543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e480cee80 0x7f4e4806d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e47fff640 1 -- 172.21.15.133:0/595426543 >> v1:172.21.15.162:6789/0 conn(0x7f4e4815dd90 legacy=0x7f4e480ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:37.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.811+0000 7f4e47fff640 1 -- 172.21.15.133:0/595426543 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480caf70 con 0x7f4e4815d9b0 2024-04-25T10:17:37.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e47fff640 1 --2- 172.21.15.133:0/595426543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4815d9b0 0x7f4e480cb5e0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f4e480cbfb0 tx=0x7f4e3801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352bd9bdcf19e4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e477fe640 1 -- 172.21.15.133:0/595426543 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380ab5f0 con 0x7f4e4815d9b0 2024-04-25T10:17:37.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e477fe640 1 -- 172.21.15.133:0/595426543 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e380b0070 con 0x7f4e4815d9b0 2024-04-25T10:17:37.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e477fe640 1 -- 172.21.15.133:0/595426543 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380ab9b0 con 0x7f4e4815d9b0 2024-04-25T10:17:37.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e4ea14640 1 -- 172.21.15.133:0/595426543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4815d9b0 msgr2=0x7f4e480cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e4ea14640 1 --2- 172.21.15.133:0/595426543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4815d9b0 0x7f4e480cb5e0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f4e480cbfb0 tx=0x7f4e3801c410 comp rx=0 tx=0).stop 2024-04-25T10:17:37.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e4ea14640 1 -- 172.21.15.133:0/595426543 shutdown_connections 2024-04-25T10:17:37.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e4ea14640 1 --2- 172.21.15.133:0/595426543 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e480cee80 0x7f4e4806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e4ea14640 1 --2- 172.21.15.133:0/595426543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4815d9b0 0x7f4e480cb5e0 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e4ea14640 1 -- 172.21.15.133:0/595426543 >> 172.21.15.133:0/595426543 conn(0x7f4e48067000 msgr2=0x7f4e4806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e4ea14640 1 -- 172.21.15.133:0/595426543 shutdown_connections 2024-04-25T10:17:37.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.815+0000 7f4e4ea14640 1 -- 172.21.15.133:0/595426543 wait complete. 2024-04-25T10:17:37.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e4ea14640 1 Processor -- start 2024-04-25T10:17:37.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e4ea14640 1 -- start start 2024-04-25T10:17:37.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e4ea14640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e480cee80 0x7f4e48166e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e4ea14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4815d9b0 0x7f4e48164e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e4ea14640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e4815dd90 0x7f4e48165340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e4ea14640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4806e810 con 0x7f4e4815d9b0 2024-04-25T10:17:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e4ea14640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4806e690 con 0x7f4e480cee80 2024-04-25T10:17:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e4ea14640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4806e990 con 0x7f4e4815dd90 2024-04-25T10:17:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e3ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4815d9b0 0x7f4e48164e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e3ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4815d9b0 0x7f4e48164e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46702/0 (socket says 172.21.15.133:46702) 2024-04-25T10:17:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e3ffff640 1 -- 172.21.15.133:0/2242231966 learned_addr learned my addr 172.21.15.133:0/2242231966 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:37.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e47fff640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e480cee80 0x7f4e48166e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e4cfb1640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e4815dd90 0x7f4e48165340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e3ffff640 1 -- 172.21.15.133:0/2242231966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e4815dd90 msgr2=0x7f4e48165340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e3ffff640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e4815dd90 0x7f4e48165340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e3ffff640 1 -- 172.21.15.133:0/2242231966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e480cee80 msgr2=0x7f4e48166e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e3ffff640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e480cee80 0x7f4e48166e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e3ffff640 1 -- 172.21.15.133:0/2242231966 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e481676d0 con 0x7f4e4815d9b0 2024-04-25T10:17:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e3ffff640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4815d9b0 0x7f4e48164e00 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000b410 tx=0x7f4e3000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e457fa640 1 -- 172.21.15.133:0/2242231966 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30015030 con 0x7f4e4815d9b0 2024-04-25T10:17:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e457fa640 1 -- 172.21.15.133:0/2242231966 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e30004020 con 0x7f4e4815d9b0 2024-04-25T10:17:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e457fa640 1 -- 172.21.15.133:0/2242231966 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30004450 con 0x7f4e4815d9b0 2024-04-25T10:17:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e4ea14640 1 -- 172.21.15.133:0/2242231966 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e481679b0 con 0x7f4e4815d9b0 2024-04-25T10:17:37.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.819+0000 7f4e4ea14640 1 -- 172.21.15.133:0/2242231966 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e48167f70 con 0x7f4e4815d9b0 2024-04-25T10:17:37.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.823+0000 7f4e4ea14640 1 -- 172.21.15.133:0/2242231966 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e48068ef0 con 0x7f4e4815d9b0 2024-04-25T10:17:37.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.823+0000 7f4e457fa640 1 -- 172.21.15.133:0/2242231966 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e30018070 con 0x7f4e4815d9b0 2024-04-25T10:17:37.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.823+0000 7f4e457fa640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e28066f60 0x7f4e28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.823+0000 7f4e457fa640 1 -- 172.21.15.133:0/2242231966 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4e30084c10 con 0x7f4e4815d9b0 2024-04-25T10:17:37.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.823+0000 7f4e47fff640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e28066f60 0x7f4e28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.823+0000 7f4e47fff640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e28066f60 0x7f4e28069420 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4e380af000 tx=0x7f4e380a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.827+0000 7f4e457fa640 1 -- 172.21.15.133:0/2242231966 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e30055b20 con 0x7f4e4815d9b0 2024-04-25T10:17:37.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.987+0000 7f4e4ea14640 1 -- 172.21.15.133:0/2242231966 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e48153610 con 0x7f4e4815d9b0 2024-04-25T10:17:37.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e457fa640 1 -- 172.21.15.133:0/2242231966 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+16921 (secure 0 0 0) 0x7f4e30055420 con 0x7f4e4815d9b0 2024-04-25T10:17:37.993 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:37.993 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":194,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:36.968809+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:34.663983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:35.974809+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:37.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 -- 172.21.15.133:0/2242231966 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e28066f60 msgr2=0x7f4e28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e28066f60 0x7f4e28069420 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4e380af000 tx=0x7f4e380a7000 comp rx=0 tx=0).stop 2024-04-25T10:17:37.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 -- 172.21.15.133:0/2242231966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4815d9b0 msgr2=0x7f4e48164e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4815d9b0 0x7f4e48164e00 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000b410 tx=0x7f4e3000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 -- 172.21.15.133:0/2242231966 shutdown_connections 2024-04-25T10:17:37.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4e4815dd90 0x7f4e48165340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4e28066f60 0x7f4e28069420 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4815d9b0 0x7f4e48164e00 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 --2- 172.21.15.133:0/2242231966 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e480cee80 0x7f4e48166e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 -- 172.21.15.133:0/2242231966 >> 172.21.15.133:0/2242231966 conn(0x7f4e48067000 msgr2=0x7f4e480b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 -- 172.21.15.133:0/2242231966 shutdown_connections 2024-04-25T10:17:37.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:37.991+0000 7f4e4ea14640 1 -- 172.21.15.133:0/2242231966 wait complete. 2024-04-25T10:17:38.013 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:38.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54f3e4640 1 Processor -- start 2024-04-25T10:17:38.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54f3e4640 1 -- start start 2024-04-25T10:17:38.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54f3e4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb54806fcc0 0x7fb5480cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54f3e4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb548066df0 0x7fb54806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54f3e4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5480cba80 con 0x7fb54815bb00 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54f3e4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480b5d00 con 0x7fb54806fcc0 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54f3e4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480cb7a0 con 0x7fb548066df0 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54d180640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb54815bb00 0x7fb54806f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:49984/0 (socket says 172.21.15.133:49984) 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54d180640 1 -- 172.21.15.133:0/2445000974 learned_addr learned my addr 172.21.15.133:0/2445000974 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951116138 0 0) 0x7fb5480cba80 con 0x7fb54815bb00 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54c97f640 1 --2- 172.21.15.133:0/2445000974 >> v2:172.21.15.162:3300/0 conn(0x7fb54806fcc0 0x7fb5480cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54d981640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb548066df0 0x7fb54806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb534003530 con 0x7fb54815bb00 2024-04-25T10:17:38.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2325818505 0 0) 0x7fb534003530 con 0x7fb54815bb00 2024-04-25T10:17:38.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb548066df0 msgr2=0x7fb54806dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb548066df0 0x7fb54806dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 >> v2:172.21.15.162:3300/0 conn(0x7fb54806fcc0 msgr2=0x7fb5480cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 --2- 172.21.15.133:0/2445000974 >> v2:172.21.15.162:3300/0 conn(0x7fb54806fcc0 0x7fb5480cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480b5990 con 0x7fb54815bb00 2024-04-25T10:17:38.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb544001680 con 0x7fb54815bb00 2024-04-25T10:17:38.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54c97f640 1 --2- 172.21.15.133:0/2445000974 >> v2:172.21.15.162:3300/0 conn(0x7fb54806fcc0 0x7fb5480cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:38.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 >> v1:172.21.15.133:6789/0 conn(0x7fb54815bb00 legacy=0x7fb54806f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5340039c0 0x7fb534003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb548066df0 0x7fb534004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb534004d50 0x7fb5340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480b5d00 con 0x7fb534004d50 2024-04-25T10:17:38.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480cb7a0 con 0x7fb5340039c0 2024-04-25T10:17:38.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53400bab0 con 0x7fb548066df0 2024-04-25T10:17:38.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54d981640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb534004d50 0x7fb5340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54d180640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5340039c0 0x7fb534003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54c97f640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb548066df0 0x7fb534004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54d981640 1 -- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb548066df0 msgr2=0x7fb534004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54d981640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb548066df0 0x7fb534004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54d981640 1 -- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5340039c0 msgr2=0x7fb534003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54d981640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5340039c0 0x7fb534003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54d981640 1 -- 172.21.15.133:0/2445000974 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb534003530 con 0x7fb534004d50 2024-04-25T10:17:38.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.131+0000 7fb54d180640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5340039c0 0x7fb534003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54d981640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb534004d50 0x7fb5340051d0 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7fb540009fc0 tx=0x7fb54001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f947477389f373 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5400b2070 con 0x7fb534004d50 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5400a5b80 con 0x7fb534004d50 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb53ffff640 1 -- 172.21.15.133:0/2445000974 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540009c10 con 0x7fb534004d50 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 -- 172.21.15.133:0/2445000974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb534004d50 msgr2=0x7fb5340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 --2- 172.21.15.133:0/2445000974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb534004d50 0x7fb5340051d0 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7fb540009fc0 tx=0x7fb54001cc80 comp rx=0 tx=0).stop 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54d981640 1 -- 172.21.15.133:0/2445000974 reap_dead start 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 -- 172.21.15.133:0/2445000974 shutdown_connections 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 -- 172.21.15.133:0/2445000974 >> 172.21.15.133:0/2445000974 conn(0x7fb5480cc8e0 msgr2=0x7fb5480cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 -- 172.21.15.133:0/2445000974 shutdown_connections 2024-04-25T10:17:38.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 -- 172.21.15.133:0/2445000974 wait complete. 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 Processor -- start 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 -- start start 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54809ffe0 0x7fb5480c3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5480a03c0 0x7fb5481f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480c7330 0x7fb5481f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815c060 con 0x7fb5480c7330 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815bee0 con 0x7fb54809ffe0 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.135+0000 7fb54f3e4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54806dbc0 con 0x7fb5480a03c0 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d981640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480c7330 0x7fb5481f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d981640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480c7330 0x7fb5481f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46722/0 (socket says 172.21.15.133:46722) 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d981640 1 -- 172.21.15.133:0/3561971698 learned_addr learned my addr 172.21.15.133:0/3561971698 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d180640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54809ffe0 0x7fb5480c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54c97f640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5480a03c0 0x7fb5481f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d981640 1 -- 172.21.15.133:0/3561971698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5480a03c0 msgr2=0x7fb5481f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d981640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5480a03c0 0x7fb5481f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d981640 1 -- 172.21.15.133:0/3561971698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54809ffe0 msgr2=0x7fb5480c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d981640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54809ffe0 0x7fb5480c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d981640 1 -- 172.21.15.133:0/3561971698 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480c4f90 con 0x7fb5480c7330 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d981640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480c7330 0x7fb5481f4270 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7fb5400b0040 tx=0x7fb540009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb53dffb640 1 -- 172.21.15.133:0/3561971698 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5400bd070 con 0x7fb5480c7330 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb53dffb640 1 -- 172.21.15.133:0/3561971698 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5400a5980 con 0x7fb5480c7330 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54f3e4640 1 -- 172.21.15.133:0/3561971698 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5480c41e0 con 0x7fb5480c7330 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb53dffb640 1 -- 172.21.15.133:0/3561971698 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5400b2040 con 0x7fb5480c7330 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54f3e4640 1 -- 172.21.15.133:0/3561971698 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5480c47a0 con 0x7fb5480c7330 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54f3e4640 1 -- 172.21.15.133:0/3561971698 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548067c70 con 0x7fb5480c7330 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb53dffb640 1 -- 172.21.15.133:0/3561971698 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5400c0070 con 0x7fb5480c7330 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb53dffb640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb52c066f90 0x7fb52c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb53dffb640 1 -- 172.21.15.133:0/3561971698 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb54011efb0 con 0x7fb5480c7330 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d180640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb52c066f90 0x7fb52c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.139+0000 7fb54d180640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb52c066f90 0x7fb52c069450 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb544001060 tx=0x7fb544001440 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.143+0000 7fb53dffb640 1 -- 172.21.15.133:0/3561971698 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb540016990 con 0x7fb5480c7330 2024-04-25T10:17:38.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb54f3e4640 1 -- 172.21.15.133:0/3561971698 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5480b3120 con 0x7fb5480c7330 2024-04-25T10:17:38.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb53dffb640 1 -- 172.21.15.133:0/3561971698 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+16921 (secure 0 0 0) 0x7fb5400ef750 con 0x7fb5480c7330 2024-04-25T10:17:38.310 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:38.310 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":194,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:36.968809+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:34.663983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:35.974809+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb54f3e4640 1 -- 172.21.15.133:0/3561971698 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb52c066f90 msgr2=0x7fb52c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb54f3e4640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb52c066f90 0x7fb52c069450 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb544001060 tx=0x7fb544001440 comp rx=0 tx=0).stop 2024-04-25T10:17:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb54f3e4640 1 -- 172.21.15.133:0/3561971698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480c7330 msgr2=0x7fb5481f4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb54f3e4640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480c7330 0x7fb5481f4270 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7fb5400b0040 tx=0x7fb540009750 comp rx=0 tx=0).stop 2024-04-25T10:17:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb54f3e4640 1 -- 172.21.15.133:0/3561971698 shutdown_connections 2024-04-25T10:17:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb54f3e4640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb52c066f90 0x7fb52c069450 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb54f3e4640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480c7330 0x7fb5481f4270 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb54f3e4640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb5480a03c0 0x7fb5481f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb54f3e4640 1 --2- 172.21.15.133:0/3561971698 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb54809ffe0 0x7fb5480c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.307+0000 7fb54f3e4640 1 -- 172.21.15.133:0/3561971698 >> 172.21.15.133:0/3561971698 conn(0x7fb5480cc8e0 msgr2=0x7fb54815eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.311+0000 7fb54f3e4640 1 -- 172.21.15.133:0/3561971698 shutdown_connections 2024-04-25T10:17:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.311+0000 7fb54f3e4640 1 -- 172.21.15.133:0/3561971698 wait complete. 2024-04-25T10:17:38.330 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:17:38.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f843b85e640 1 Processor -- start 2024-04-25T10:17:38.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f843b85e640 1 -- start start 2024-04-25T10:17:38.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f843b85e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f84340ce600 0x7f84340cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f843b85e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84340d1920 0x7f84340d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f843b85e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843415ebd0 con 0x7f843415d9a0 2024-04-25T10:17:38.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f843b85e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843415eeb0 con 0x7f84340ce600 2024-04-25T10:17:38.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f843b85e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84340d0330 con 0x7f84340d1920 2024-04-25T10:17:38.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f84395fa640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f843415d9a0 0x7f84340d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50000/0 (socket says 172.21.15.133:50000) 2024-04-25T10:17:38.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f84395fa640 1 -- 172.21.15.133:0/3883601084 learned_addr learned my addr 172.21.15.133:0/3883601084 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:38.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f8438df9640 1 --2- 172.21.15.133:0/3883601084 >> v2:172.21.15.133:3300/0 conn(0x7f84340ce600 0x7f84340cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f8439dfb640 1 --2- 172.21.15.133:0/3883601084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84340d1920 0x7f84340d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f842bfff640 1 -- 172.21.15.133:0/3883601084 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3670275939 0 0) 0x7f843415ebd0 con 0x7f843415d9a0 2024-04-25T10:17:38.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f842bfff640 1 -- 172.21.15.133:0/3883601084 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8420003530 con 0x7f843415d9a0 2024-04-25T10:17:38.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f8438df9640 1 -- 172.21.15.133:0/3883601084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84340d1920 msgr2=0x7f84340d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f8438df9640 1 --2- 172.21.15.133:0/3883601084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84340d1920 0x7f84340d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f8438df9640 1 -- 172.21.15.133:0/3883601084 >> v1:172.21.15.133:6789/0 conn(0x7f843415d9a0 legacy=0x7f84340d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.439+0000 7f8438df9640 1 -- 172.21.15.133:0/3883601084 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84340d0610 con 0x7f84340ce600 2024-04-25T10:17:38.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f8438df9640 1 --2- 172.21.15.133:0/3883601084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84340ce600 0x7f84340cea90 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f84240097c0 tx=0x7f842401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8b1e1e46a0bea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f842bfff640 1 -- 172.21.15.133:0/3883601084 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84240b0070 con 0x7f84340ce600 2024-04-25T10:17:38.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f842bfff640 1 -- 172.21.15.133:0/3883601084 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84240ab600 con 0x7f84340ce600 2024-04-25T10:17:38.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f842bfff640 1 -- 172.21.15.133:0/3883601084 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84240ab9b0 con 0x7f84340ce600 2024-04-25T10:17:38.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 -- 172.21.15.133:0/3883601084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84340ce600 msgr2=0x7f84340cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 --2- 172.21.15.133:0/3883601084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84340ce600 0x7f84340cea90 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f84240097c0 tx=0x7f842401c410 comp rx=0 tx=0).stop 2024-04-25T10:17:38.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 -- 172.21.15.133:0/3883601084 shutdown_connections 2024-04-25T10:17:38.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 --2- 172.21.15.133:0/3883601084 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f84340d1920 0x7f84340d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 --2- 172.21.15.133:0/3883601084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84340ce600 0x7f84340cea90 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 -- 172.21.15.133:0/3883601084 >> 172.21.15.133:0/3883601084 conn(0x7f843406bf80 msgr2=0x7f843406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 -- 172.21.15.133:0/3883601084 shutdown_connections 2024-04-25T10:17:38.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 -- 172.21.15.133:0/3883601084 wait complete. 2024-04-25T10:17:38.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 Processor -- start 2024-04-25T10:17:38.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 -- start start 2024-04-25T10:17:38.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84340ce600 0x7f843415a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84340d1920 0x7f84341555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f843415d9a0 0x7f84340b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843415e710 con 0x7f84340ce600 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843415e590 con 0x7f84340d1920 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843415e890 con 0x7f843415d9a0 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f84395fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84340ce600 0x7f843415a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f84395fa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84340ce600 0x7f843415a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46736/0 (socket says 172.21.15.133:46736) 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f84395fa640 1 -- 172.21.15.133:0/1662851972 learned_addr learned my addr 172.21.15.133:0/1662851972 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f8438df9640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84340d1920 0x7f84341555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f8439dfb640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f843415d9a0 0x7f84340b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f84395fa640 1 -- 172.21.15.133:0/1662851972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f843415d9a0 msgr2=0x7f84340b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f84395fa640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f843415d9a0 0x7f84340b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f84395fa640 1 -- 172.21.15.133:0/1662851972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84340d1920 msgr2=0x7f84341555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f84395fa640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84340d1920 0x7f84341555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f84395fa640 1 -- 172.21.15.133:0/1662851972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843415a860 con 0x7f84340ce600 2024-04-25T10:17:38.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f84395fa640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84340ce600 0x7f843415a280 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f843000c450 tx=0x7f843000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f8429ffb640 1 -- 172.21.15.133:0/1662851972 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430017070 con 0x7f84340ce600 2024-04-25T10:17:38.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f8429ffb640 1 -- 172.21.15.133:0/1662851972 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8430005cf0 con 0x7f84340ce600 2024-04-25T10:17:38.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.443+0000 7f843b85e640 1 -- 172.21.15.133:0/1662851972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843415ab40 con 0x7f84340ce600 2024-04-25T10:17:38.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.447+0000 7f8429ffb640 1 -- 172.21.15.133:0/1662851972 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300072f0 con 0x7f84340ce600 2024-04-25T10:17:38.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.447+0000 7f843b85e640 1 -- 172.21.15.133:0/1662851972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8434155af0 con 0x7f84340ce600 2024-04-25T10:17:38.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.447+0000 7f843b85e640 1 -- 172.21.15.133:0/1662851972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8434068e80 con 0x7f84340ce600 2024-04-25T10:17:38.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.447+0000 7f8429ffb640 1 -- 172.21.15.133:0/1662851972 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f843001a070 con 0x7f84340ce600 2024-04-25T10:17:38.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.447+0000 7f8429ffb640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f60 0x7f8418069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.447+0000 7f8429ffb640 1 -- 172.21.15.133:0/1662851972 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8430084a00 con 0x7f84340ce600 2024-04-25T10:17:38.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.447+0000 7f8438df9640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f60 0x7f8418069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.447+0000 7f8438df9640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f60 0x7f8418069420 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f84240ad000 tx=0x7f84240a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.451+0000 7f8429ffb640 1 -- 172.21.15.133:0/1662851972 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8430021020 con 0x7f84340ce600 2024-04-25T10:17:38.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.631+0000 7f843b85e640 1 -- 172.21.15.133:0/1662851972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f843406c7c0 con 0x7f84340ce600 2024-04-25T10:17:38.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f8429ffb640 1 -- 172.21.15.133:0/1662851972 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v117) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8430055960 con 0x7f84340ce600 2024-04-25T10:17:38.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 -- 172.21.15.133:0/1662851972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f60 msgr2=0x7f8418069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f60 0x7f8418069420 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f84240ad000 tx=0x7f84240a6000 comp rx=0 tx=0).stop 2024-04-25T10:17:38.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 -- 172.21.15.133:0/1662851972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84340ce600 msgr2=0x7f843415a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84340ce600 0x7f843415a280 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f843000c450 tx=0x7f843000c910 comp rx=0 tx=0).stop 2024-04-25T10:17:38.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 -- 172.21.15.133:0/1662851972 shutdown_connections 2024-04-25T10:17:38.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f843415d9a0 0x7f84340b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f84340d1920 0x7f84341555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8418066f60 0x7f8418069420 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 --2- 172.21.15.133:0/1662851972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84340ce600 0x7f843415a280 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 -- 172.21.15.133:0/1662851972 >> 172.21.15.133:0/1662851972 conn(0x7f843406bf80 msgr2=0x7f8434167fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 -- 172.21.15.133:0/1662851972 shutdown_connections 2024-04-25T10:17:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.659+0000 7f843b85e640 1 -- 172.21.15.133:0/1662851972 wait complete. 2024-04-25T10:17:38.680 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:17:38.680 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:17:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a13d21640 1 Processor -- start 2024-04-25T10:17:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a13d21640 1 -- start start 2024-04-25T10:17:38.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a13d21640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7a0c0d10c0 0x7f7a0c06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a13d21640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7a0c0cabb0 0x7f7a0c0cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a13d21640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0c0cc950 con 0x7f7a0c06ac90 2024-04-25T10:17:38.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a13d21640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c06b5b0 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a13d21640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c0b1ac0 con 0x7f7a0c0d10c0 2024-04-25T10:17:38.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a122be640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7a0c0cabb0 0x7f7a0c0cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a11abd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7a0c06ac90 0x7f7a0c0b1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50002/0 (socket says 172.21.15.133:50002) 2024-04-25T10:17:38.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a11abd640 1 -- 172.21.15.133:0/1666910080 learned_addr learned my addr 172.21.15.133:0/1666910080 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:38.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a122be640 1 -- 172.21.15.133:0/1666910080 >> v2:172.21.15.162:3300/0 conn(0x7f7a0c0d10c0 msgr2=0x7f7a0c06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a112bc640 1 --2- 172.21.15.133:0/1666910080 >> v2:172.21.15.162:3300/0 conn(0x7f7a0c0d10c0 0x7f7a0c06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a122be640 1 --2- 172.21.15.133:0/1666910080 >> v2:172.21.15.162:3300/0 conn(0x7f7a0c0d10c0 0x7f7a0c06b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a122be640 1 -- 172.21.15.133:0/1666910080 >> v1:172.21.15.133:6789/0 conn(0x7f7a0c06ac90 legacy=0x7f7a0c0b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a122be640 1 -- 172.21.15.133:0/1666910080 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0ccbc0 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.799+0000 7f7a10abb640 1 -- 172.21.15.133:0/1666910080 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407243773 0 0) 0x7f7a0c0cc950 con 0x7f7a0c06ac90 2024-04-25T10:17:38.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a122be640 1 --2- 172.21.15.133:0/1666910080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a0c0cabb0 0x7f7a0c0cd3c0 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c071b20 tx=0x7f7a0401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515f6e57db0e4781 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a10abb640 1 -- 172.21.15.133:0/1666910080 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a040b2070 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a10abb640 1 -- 172.21.15.133:0/1666910080 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a040a5870 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a10abb640 1 -- 172.21.15.133:0/1666910080 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a040a5c40 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 -- 172.21.15.133:0/1666910080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a0c0cabb0 msgr2=0x7f7a0c0cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 --2- 172.21.15.133:0/1666910080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a0c0cabb0 0x7f7a0c0cd3c0 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c071b20 tx=0x7f7a0401c8a0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 -- 172.21.15.133:0/1666910080 shutdown_connections 2024-04-25T10:17:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 --2- 172.21.15.133:0/1666910080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a0c0cabb0 0x7f7a0c0cd3c0 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 --2- 172.21.15.133:0/1666910080 >> v2:172.21.15.162:3300/0 conn(0x7f7a0c0d10c0 0x7f7a0c06b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 -- 172.21.15.133:0/1666910080 >> 172.21.15.133:0/1666910080 conn(0x7f7a0c0999d0 msgr2=0x7f7a0c0d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 -- 172.21.15.133:0/1666910080 shutdown_connections 2024-04-25T10:17:38.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 -- 172.21.15.133:0/1666910080 wait complete. 2024-04-25T10:17:38.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 Processor -- start 2024-04-25T10:17:38.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 -- start start 2024-04-25T10:17:38.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a0c06ac90 0x7f7a0c0c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a0c0cabb0 0x7f7a0c0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a0c0d10c0 0x7f7a0c0ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c15df60 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c15dde0 con 0x7f7a0c06ac90 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a13d21640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c16ae90 con 0x7f7a0c0d10c0 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a112bc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a0c0cabb0 0x7f7a0c0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a112bc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a0c0cabb0 0x7f7a0c0b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46756/0 (socket says 172.21.15.133:46756) 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a112bc640 1 -- 172.21.15.133:0/1577259699 learned_addr learned my addr 172.21.15.133:0/1577259699 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a11abd640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a0c06ac90 0x7f7a0c0c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a122be640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a0c0d10c0 0x7f7a0c0ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a112bc640 1 -- 172.21.15.133:0/1577259699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a0c0d10c0 msgr2=0x7f7a0c0ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a112bc640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a0c0d10c0 0x7f7a0c0ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a112bc640 1 -- 172.21.15.133:0/1577259699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a0c06ac90 msgr2=0x7f7a0c0c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a112bc640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a0c06ac90 0x7f7a0c0c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a112bc640 1 -- 172.21.15.133:0/1577259699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0b57a0 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.803+0000 7f7a11abd640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a0c06ac90 0x7f7a0c0c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a112bc640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a0c0cabb0 0x7f7a0c0b5130 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00c7f0 tx=0x7f79fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a027fc640 1 -- 172.21.15.133:0/1577259699 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc015070 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a027fc640 1 -- 172.21.15.133:0/1577259699 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc007820 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a13d21640 1 -- 172.21.15.133:0/1577259699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c0b5a80 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a027fc640 1 -- 172.21.15.133:0/1577259699 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc007c10 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a13d21640 1 -- 172.21.15.133:0/1577259699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c0b6040 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a13d21640 1 -- 172.21.15.133:0/1577259699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c067c70 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a027fc640 1 -- 172.21.15.133:0/1577259699 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79fc018070 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a027fc640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f0066f60 0x7f79f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a027fc640 1 -- 172.21.15.133:0/1577259699 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79fc0851c0 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a11abd640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f0066f60 0x7f79f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.807+0000 7f7a11abd640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f0066f60 0x7f79f0069420 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7a08007fd0 tx=0x7f7a08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.811+0000 7f7a027fc640 1 -- 172.21.15.133:0/1577259699 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79fc0560d0 con 0x7f7a0c0cabb0 2024-04-25T10:17:38.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:38.995+0000 7f7a13d21640 1 -- 172.21.15.133:0/1577259699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7a0c0cd560 con 0x7f7a0c0cabb0 2024-04-25T10:17:39.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.659+0000 7f7a027fc640 1 -- 172.21.15.133:0/1577259699 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v118) v1 ==== 122+0+0 (secure 0 0 0) 0x7f79fc0559d0 con 0x7f7a0c0cabb0 2024-04-25T10:17:39.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 -- 172.21.15.133:0/1577259699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f0066f60 msgr2=0x7f79f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f0066f60 0x7f79f0069420 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7a08007fd0 tx=0x7f7a08009040 comp rx=0 tx=0).stop 2024-04-25T10:17:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 -- 172.21.15.133:0/1577259699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a0c0cabb0 msgr2=0x7f7a0c0b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a0c0cabb0 0x7f7a0c0b5130 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00c7f0 tx=0x7f79fc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 -- 172.21.15.133:0/1577259699 shutdown_connections 2024-04-25T10:17:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7a0c0d10c0 0x7f7a0c0ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f79f0066f60 0x7f79f0069420 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a0c0cabb0 0x7f7a0c0b5130 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 --2- 172.21.15.133:0/1577259699 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a0c06ac90 0x7f7a0c0c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 -- 172.21.15.133:0/1577259699 >> 172.21.15.133:0/1577259699 conn(0x7f7a0c0999d0 msgr2=0x7f7a0c0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 -- 172.21.15.133:0/1577259699 shutdown_connections 2024-04-25T10:17:39.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.663+0000 7f7a13d21640 1 -- 172.21.15.133:0/1577259699 wait complete. 2024-04-25T10:17:39.688 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:17:39.688 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:39.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f1104a85640 1 Processor -- start 2024-04-25T10:17:39.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f1104a85640 1 -- start start 2024-04-25T10:17:39.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f1104a85640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f110015c9a0 0x7f11000c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f1104a85640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11000ce330 0x7f11000ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f1104a85640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11000c61d0 con 0x7f11000b1f70 2024-04-25T10:17:39.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f1104a85640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11000cecc0 con 0x7f110015c9a0 2024-04-25T10:17:39.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f1104a85640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11000c64b0 con 0x7f11000ce330 2024-04-25T10:17:39.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f10fe59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f110015c9a0 0x7f11000c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f10fe59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f110015c9a0 0x7f11000c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46762/0 (socket says 172.21.15.133:46762) 2024-04-25T10:17:39.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f10fe59c640 1 -- 172.21.15.133:0/3314634889 learned_addr learned my addr 172.21.15.133:0/3314634889 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:39.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f10fe59c640 1 -- 172.21.15.133:0/3314634889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11000ce330 msgr2=0x7f11000ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:39.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f10fe59c640 1 --2- 172.21.15.133:0/3314634889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11000ce330 0x7f11000ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f10fe59c640 1 -- 172.21.15.133:0/3314634889 >> v1:172.21.15.162:6789/0 conn(0x7f11000b1f70 legacy=0x7f110006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:39.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.811+0000 7f10fe59c640 1 -- 172.21.15.133:0/3314634889 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110006d4f0 con 0x7f110015c9a0 2024-04-25T10:17:39.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f10fe59c640 1 --2- 172.21.15.133:0/3314634889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f110015c9a0 0x7f11000c5980 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009870 tx=0x7f10f001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f7d478295e869a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f10fd59a640 1 -- 172.21.15.133:0/3314634889 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00b2070 con 0x7f110015c9a0 2024-04-25T10:17:39.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f10fd59a640 1 -- 172.21.15.133:0/3314634889 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f00ad640 con 0x7f110015c9a0 2024-04-25T10:17:39.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f10fd59a640 1 -- 172.21.15.133:0/3314634889 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00ada10 con 0x7f110015c9a0 2024-04-25T10:17:39.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 -- 172.21.15.133:0/3314634889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f110015c9a0 msgr2=0x7f11000c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 --2- 172.21.15.133:0/3314634889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f110015c9a0 0x7f11000c5980 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009870 tx=0x7f10f001c520 comp rx=0 tx=0).stop 2024-04-25T10:17:39.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 -- 172.21.15.133:0/3314634889 shutdown_connections 2024-04-25T10:17:39.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 --2- 172.21.15.133:0/3314634889 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11000ce330 0x7f11000ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 --2- 172.21.15.133:0/3314634889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f110015c9a0 0x7f11000c5980 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 -- 172.21.15.133:0/3314634889 >> 172.21.15.133:0/3314634889 conn(0x7f1100067000 msgr2=0x7f110006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:39.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 -- 172.21.15.133:0/3314634889 shutdown_connections 2024-04-25T10:17:39.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 -- 172.21.15.133:0/3314634889 wait complete. 2024-04-25T10:17:39.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 Processor -- start 2024-04-25T10:17:39.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 -- start start 2024-04-25T10:17:39.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11000b1f70 0x7f1100159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000ce330 0x7f1100147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110015c9a0 0x7f1100157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110015dad0 con 0x7f11000ce330 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110015d950 con 0x7f110015c9a0 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.815+0000 7f1104a85640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110015dc50 con 0x7f11000b1f70 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fdd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000ce330 0x7f1100147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fdd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000ce330 0x7f1100147450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46772/0 (socket says 172.21.15.133:46772) 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fdd9b640 1 -- 172.21.15.133:0/2971646749 learned_addr learned my addr 172.21.15.133:0/2971646749 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fed9d640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110015c9a0 0x7f1100157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fe59c640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11000b1f70 0x7f1100159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fdd9b640 1 -- 172.21.15.133:0/2971646749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11000b1f70 msgr2=0x7f1100159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fdd9b640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11000b1f70 0x7f1100159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fdd9b640 1 -- 172.21.15.133:0/2971646749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110015c9a0 msgr2=0x7f1100157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fdd9b640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110015c9a0 0x7f1100157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fdd9b640 1 -- 172.21.15.133:0/2971646749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110014a4f0 con 0x7f11000ce330 2024-04-25T10:17:39.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fdd9b640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000ce330 0x7f1100147450 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f10f400b410 tx=0x7f10f400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10eeffd640 1 -- 172.21.15.133:0/2971646749 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4015030 con 0x7f11000ce330 2024-04-25T10:17:39.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10eeffd640 1 -- 172.21.15.133:0/2971646749 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4004020 con 0x7f11000ce330 2024-04-25T10:17:39.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f1104a85640 1 -- 172.21.15.133:0/2971646749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1100149b30 con 0x7f11000ce330 2024-04-25T10:17:39.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10eeffd640 1 -- 172.21.15.133:0/2971646749 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4004450 con 0x7f11000ce330 2024-04-25T10:17:39.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f1104a85640 1 -- 172.21.15.133:0/2971646749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1100149750 con 0x7f11000ce330 2024-04-25T10:17:39.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f1104a85640 1 -- 172.21.15.133:0/2971646749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1100158480 con 0x7f11000ce330 2024-04-25T10:17:39.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10eeffd640 1 -- 172.21.15.133:0/2971646749 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10f4018070 con 0x7f11000ce330 2024-04-25T10:17:39.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10eeffd640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 0x7f10e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10eeffd640 1 -- 172.21.15.133:0/2971646749 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10f4084cc0 con 0x7f11000ce330 2024-04-25T10:17:39.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fe59c640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 0x7f10e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.819+0000 7f10fe59c640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 0x7f10e0069420 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f10f00af000 tx=0x7f10f00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:39.823+0000 7f10eeffd640 1 -- 172.21.15.133:0/2971646749 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10f4055bd0 con 0x7f11000ce330 2024-04-25T10:17:40.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.007+0000 7f1104a85640 1 -- 172.21.15.133:0/2971646749 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1100068ef0 con 0x7f11000ce330 2024-04-25T10:17:40.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f10eeffd640 1 -- 172.21.15.133:0/2971646749 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4961 (secure 0 0 0) 0x7f10f40554d0 con 0x7f11000ce330 2024-04-25T10:17:40.013 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:40.013 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:36.969368+0000","modified":"2024-04-25T10:17:38.999991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11874":{"gid":11874,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/1916780253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1916780253},{"type":"v1","addr":"172.21.15.133:6837","nonce":1916780253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:17:40.013 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 118 2024-04-25T10:17:40.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 -- 172.21.15.133:0/2971646749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 msgr2=0x7f10e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 0x7f10e0069420 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f10f00af000 tx=0x7f10f00a7070 comp rx=0 tx=0).stop 2024-04-25T10:17:40.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 -- 172.21.15.133:0/2971646749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000ce330 msgr2=0x7f1100147450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000ce330 0x7f1100147450 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f10f400b410 tx=0x7f10f400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 -- 172.21.15.133:0/2971646749 shutdown_connections 2024-04-25T10:17:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f110015c9a0 0x7f1100157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f10e0066f60 0x7f10e0069420 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000ce330 0x7f1100147450 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 --2- 172.21.15.133:0/2971646749 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f11000b1f70 0x7f1100159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 -- 172.21.15.133:0/2971646749 >> 172.21.15.133:0/2971646749 conn(0x7f1100067000 msgr2=0x7f11000c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 -- 172.21.15.133:0/2971646749 shutdown_connections 2024-04-25T10:17:40.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.011+0000 7f1104a85640 1 -- 172.21.15.133:0/2971646749 wait complete. 2024-04-25T10:17:40.030 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:40.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd27b7a0640 1 Processor -- start 2024-04-25T10:17:40.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd27b7a0640 1 -- start start 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd27b7a0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2740afc00 0x7fd2740d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd27b7a0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd27406ac90 0x7fd27406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd27b7a0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2740b3e40 con 0x7fd27415dbf0 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd27b7a0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2740b3880 con 0x7fd2740afc00 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd27b7a0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2740b3b60 con 0x7fd27406ac90 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd278d3b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2740afc00 0x7fd2740d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd278d3b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd2740afc00 0x7fd2740d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46778/0 (socket says 172.21.15.133:46778) 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd278d3b640 1 -- 172.21.15.133:0/285115071 learned_addr learned my addr 172.21.15.133:0/285115071 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd279d3d640 1 --2- 172.21.15.133:0/285115071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd27406ac90 0x7fd27406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd278d3b640 1 -- 172.21.15.133:0/285115071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd27406ac90 msgr2=0x7fd27406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd278d3b640 1 --2- 172.21.15.133:0/285115071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd27406ac90 0x7fd27406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd278d3b640 1 -- 172.21.15.133:0/285115071 >> v1:172.21.15.162:6789/0 conn(0x7fd27415dbf0 legacy=0x7fd27406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.147+0000 7fd278d3b640 1 -- 172.21.15.133:0/285115071 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27406e540 con 0x7fd2740afc00 2024-04-25T10:17:40.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd278d3b640 1 --2- 172.21.15.133:0/285115071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2740afc00 0x7fd2740d0f80 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7fd264009870 tx=0x7fd26401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68ae3287666bf5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd26bfff640 1 -- 172.21.15.133:0/285115071 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2640b0070 con 0x7fd2740afc00 2024-04-25T10:17:40.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd26bfff640 1 -- 172.21.15.133:0/285115071 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2640ab600 con 0x7fd2740afc00 2024-04-25T10:17:40.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd26bfff640 1 -- 172.21.15.133:0/285115071 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2640ab9d0 con 0x7fd2740afc00 2024-04-25T10:17:40.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 -- 172.21.15.133:0/285115071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2740afc00 msgr2=0x7fd2740d0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 --2- 172.21.15.133:0/285115071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2740afc00 0x7fd2740d0f80 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7fd264009870 tx=0x7fd26401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 -- 172.21.15.133:0/285115071 shutdown_connections 2024-04-25T10:17:40.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 --2- 172.21.15.133:0/285115071 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd27406ac90 0x7fd27406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 --2- 172.21.15.133:0/285115071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2740afc00 0x7fd2740d0f80 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 -- 172.21.15.133:0/285115071 >> 172.21.15.133:0/285115071 conn(0x7fd274072430 msgr2=0x7fd274072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 -- 172.21.15.133:0/285115071 shutdown_connections 2024-04-25T10:17:40.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 -- 172.21.15.133:0/285115071 wait complete. 2024-04-25T10:17:40.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 Processor -- start 2024-04-25T10:17:40.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 -- start start 2024-04-25T10:17:40.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd27406ac90 0x7fd274157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2740afc00 0x7fd27415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd27415dbf0 0x7fd27415fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd274162280 con 0x7fd27415dbf0 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd274162100 con 0x7fd27406ac90 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27b7a0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27416aa40 con 0x7fd2740afc00 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd279d3d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd27415dbf0 0x7fd27415fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd279d3d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd27415dbf0 0x7fd27415fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46790/0 (socket says 172.21.15.133:46790) 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd279d3d640 1 -- 172.21.15.133:0/3284708437 learned_addr learned my addr 172.21.15.133:0/3284708437 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd278d3b640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2740afc00 0x7fd27415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd27953c640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd27406ac90 0x7fd274157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd279d3d640 1 -- 172.21.15.133:0/3284708437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2740afc00 msgr2=0x7fd27415e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd279d3d640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2740afc00 0x7fd27415e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd279d3d640 1 -- 172.21.15.133:0/3284708437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd27406ac90 msgr2=0x7fd274157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd279d3d640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd27406ac90 0x7fd274157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.151+0000 7fd279d3d640 1 -- 172.21.15.133:0/3284708437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274160670 con 0x7fd27415dbf0 2024-04-25T10:17:40.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd27953c640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd27406ac90 0x7fd274157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:40.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd279d3d640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd27415dbf0 0x7fd27415fe00 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7fd26c007e60 tx=0x7fd26c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd269ffb640 1 -- 172.21.15.133:0/3284708437 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c014420 con 0x7fd27415dbf0 2024-04-25T10:17:40.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd269ffb640 1 -- 172.21.15.133:0/3284708437 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c019070 con 0x7fd27415dbf0 2024-04-25T10:17:40.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd269ffb640 1 -- 172.21.15.133:0/3284708437 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c014820 con 0x7fd27415dbf0 2024-04-25T10:17:40.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd27b7a0640 1 -- 172.21.15.133:0/3284708437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd274160950 con 0x7fd27415dbf0 2024-04-25T10:17:40.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd27b7a0640 1 -- 172.21.15.133:0/3284708437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd274160f10 con 0x7fd27415dbf0 2024-04-25T10:17:40.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd27b7a0640 1 -- 172.21.15.133:0/3284708437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd274067c70 con 0x7fd27415dbf0 2024-04-25T10:17:40.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd269ffb640 1 -- 172.21.15.133:0/3284708437 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd26c0149c0 con 0x7fd27415dbf0 2024-04-25T10:17:40.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd269ffb640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd258066f60 0x7fd258069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd269ffb640 1 -- 172.21.15.133:0/3284708437 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd26c085a60 con 0x7fd27415dbf0 2024-04-25T10:17:40.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd27953c640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd258066f60 0x7fd258069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.155+0000 7fd27953c640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd258066f60 0x7fd258069420 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd2700016f0 tx=0x7fd270009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.159+0000 7fd269ffb640 1 -- 172.21.15.133:0/3284708437 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd26c0568e0 con 0x7fd27415dbf0 2024-04-25T10:17:40.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.307+0000 7fd27b7a0640 1 -- 172.21.15.133:0/3284708437 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2740d0d60 con 0x7fd27415dbf0 2024-04-25T10:17:40.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd269ffb640 1 -- 172.21.15.133:0/3284708437 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+16908 (secure 0 0 0) 0x7fd26c0561e0 con 0x7fd27415dbf0 2024-04-25T10:17:40.313 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:40.314 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":195,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:38.655088+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:34.663983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:35.974809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:40.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 -- 172.21.15.133:0/3284708437 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd258066f60 msgr2=0x7fd258069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd258066f60 0x7fd258069420 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd2700016f0 tx=0x7fd270009040 comp rx=0 tx=0).stop 2024-04-25T10:17:40.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 -- 172.21.15.133:0/3284708437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd27415dbf0 msgr2=0x7fd27415fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd27415dbf0 0x7fd27415fe00 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7fd26c007e60 tx=0x7fd26c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 -- 172.21.15.133:0/3284708437 shutdown_connections 2024-04-25T10:17:40.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd258066f60 0x7fd258069420 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd27415dbf0 0x7fd27415fe00 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd2740afc00 0x7fd27415e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 --2- 172.21.15.133:0/3284708437 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd27406ac90 0x7fd274157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 -- 172.21.15.133:0/3284708437 >> 172.21.15.133:0/3284708437 conn(0x7fd274072430 msgr2=0x7fd27406ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 -- 172.21.15.133:0/3284708437 shutdown_connections 2024-04-25T10:17:40.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.311+0000 7fd27b7a0640 1 -- 172.21.15.133:0/3284708437 wait complete. 2024-04-25T10:17:40.331 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:17:40.332 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07edbb640 1 Processor -- start 2024-04-25T10:17:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07edbb640 1 -- start start 2024-04-25T10:17:40.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07edbb640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc07815b9b0 0x7fc0780cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07edbb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc0780d0380 0x7fc0780c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07edbb640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07815bd90 con 0x7fc0780cf7c0 2024-04-25T10:17:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07edbb640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0780c5610 con 0x7fc0780d0380 2024-04-25T10:17:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07edbb640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07809ad10 con 0x7fc07815b9b0 2024-04-25T10:17:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07d358640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc0780d0380 0x7fc0780c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07d358640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc0780d0380 0x7fc0780c5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46804/0 (socket says 172.21.15.133:46804) 2024-04-25T10:17:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07d358640 1 -- 172.21.15.133:0/414354808 learned_addr learned my addr 172.21.15.133:0/414354808 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07cb57640 1 --2- 172.21.15.133:0/414354808 >> v2:172.21.15.162:3300/0 conn(0x7fc07815b9b0 0x7fc0780cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07d358640 1 -- 172.21.15.133:0/414354808 >> v2:172.21.15.162:3300/0 conn(0x7fc07815b9b0 msgr2=0x7fc0780cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07d358640 1 --2- 172.21.15.133:0/414354808 >> v2:172.21.15.162:3300/0 conn(0x7fc07815b9b0 0x7fc0780cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07d358640 1 -- 172.21.15.133:0/414354808 >> v1:172.21.15.162:6789/0 conn(0x7fc0780cf7c0 legacy=0x7fc0780cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.451+0000 7fc07d358640 1 -- 172.21.15.133:0/414354808 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07809a750 con 0x7fc0780d0380 2024-04-25T10:17:40.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07d358640 1 --2- 172.21.15.133:0/414354808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0780d0380 0x7fc0780c5aa0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7fc07806f160 tx=0x7fc07001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d3a5d4f50b491a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc06f7fe640 1 -- 172.21.15.133:0/414354808 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0700a5880 con 0x7fc0780d0380 2024-04-25T10:17:40.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc06f7fe640 1 -- 172.21.15.133:0/414354808 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0700a5a20 con 0x7fc0780d0380 2024-04-25T10:17:40.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc06f7fe640 1 -- 172.21.15.133:0/414354808 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0700a5df0 con 0x7fc0780d0380 2024-04-25T10:17:40.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 -- 172.21.15.133:0/414354808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0780d0380 msgr2=0x7fc0780c5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 --2- 172.21.15.133:0/414354808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0780d0380 0x7fc0780c5aa0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7fc07806f160 tx=0x7fc07001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 -- 172.21.15.133:0/414354808 shutdown_connections 2024-04-25T10:17:40.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 --2- 172.21.15.133:0/414354808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0780d0380 0x7fc0780c5aa0 unknown :-1 s=CLOSED pgs=2866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 --2- 172.21.15.133:0/414354808 >> v2:172.21.15.162:3300/0 conn(0x7fc07815b9b0 0x7fc0780cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 -- 172.21.15.133:0/414354808 >> 172.21.15.133:0/414354808 conn(0x7fc0780b31a0 msgr2=0x7fc0780b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 -- 172.21.15.133:0/414354808 shutdown_connections 2024-04-25T10:17:40.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 -- 172.21.15.133:0/414354808 wait complete. 2024-04-25T10:17:40.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 Processor -- start 2024-04-25T10:17:40.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 -- start start 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0780cf7c0 0x7fc0780cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0780d0380 0x7fc078153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc07815b9b0 0x7fc078152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780c6970 con 0x7fc07815b9b0 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780c67f0 con 0x7fc0780cf7c0 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07edbb640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780c6af0 con 0x7fc0780d0380 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07d358640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc07815b9b0 0x7fc078152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07d358640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc07815b9b0 0x7fc078152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46820/0 (socket says 172.21.15.133:46820) 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.455+0000 7fc07d358640 1 -- 172.21.15.133:0/2794037983 learned_addr learned my addr 172.21.15.133:0/2794037983 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07cb57640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0780cf7c0 0x7fc0780cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07d358640 1 -- 172.21.15.133:0/2794037983 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0780d0380 msgr2=0x7fc078153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07d358640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0780d0380 0x7fc078153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07d358640 1 -- 172.21.15.133:0/2794037983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0780cf7c0 msgr2=0x7fc0780cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07d358640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0780cf7c0 0x7fc0780cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07d358640 1 -- 172.21.15.133:0/2794037983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078153c80 con 0x7fc07815b9b0 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07cb57640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0780cf7c0 0x7fc0780cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07d358640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc07815b9b0 0x7fc078152c50 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7fc0700af040 tx=0x7fc070019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc06d7fa640 1 -- 172.21.15.133:0/2794037983 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0700b8670 con 0x7fc07815b9b0 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc06d7fa640 1 -- 172.21.15.133:0/2794037983 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0700be070 con 0x7fc07815b9b0 2024-04-25T10:17:40.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07edbb640 1 -- 172.21.15.133:0/2794037983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc078153f60 con 0x7fc07815b9b0 2024-04-25T10:17:40.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc06d7fa640 1 -- 172.21.15.133:0/2794037983 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0700b8a50 con 0x7fc07815b9b0 2024-04-25T10:17:40.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07edbb640 1 -- 172.21.15.133:0/2794037983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc078154520 con 0x7fc07815b9b0 2024-04-25T10:17:40.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07edbb640 1 -- 172.21.15.133:0/2794037983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc078067c70 con 0x7fc07815b9b0 2024-04-25T10:17:40.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc06d7fa640 1 -- 172.21.15.133:0/2794037983 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0700b8bf0 con 0x7fc07815b9b0 2024-04-25T10:17:40.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc06d7fa640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc064066f60 0x7fc064069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07cb57640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc064066f60 0x7fc064069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc06d7fa640 1 -- 172.21.15.133:0/2794037983 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc0701203b0 con 0x7fc07815b9b0 2024-04-25T10:17:40.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.459+0000 7fc07cb57640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc064066f60 0x7fc064069420 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc074009a40 tx=0x7fc074008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.463+0000 7fc06d7fa640 1 -- 172.21.15.133:0/2794037983 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0700165e0 con 0x7fc07815b9b0 2024-04-25T10:17:40.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.619+0000 7fc07edbb640 1 -- 172.21.15.133:0/2794037983 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc078158430 con 0x7fc064066f60 2024-04-25T10:17:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.635+0000 7fc06d7fa640 1 -- 172.21.15.133:0/2794037983 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252488 (secure 0 0 0) 0x7fc078158430 con 0x7fc064066f60 2024-04-25T10:17:40.641 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 -- 172.21.15.133:0/2794037983 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc064066f60 msgr2=0x7fc064069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc064066f60 0x7fc064069420 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc074009a40 tx=0x7fc074008040 comp rx=0 tx=0).stop 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 -- 172.21.15.133:0/2794037983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc07815b9b0 msgr2=0x7fc078152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc07815b9b0 0x7fc078152c50 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7fc0700af040 tx=0x7fc070019760 comp rx=0 tx=0).stop 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 -- 172.21.15.133:0/2794037983 shutdown_connections 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc064066f60 0x7fc064069420 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc07815b9b0 0x7fc078152c50 unknown :-1 s=CLOSED pgs=2867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc0780d0380 0x7fc078153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 --2- 172.21.15.133:0/2794037983 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc0780cf7c0 0x7fc0780cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 -- 172.21.15.133:0/2794037983 >> 172.21.15.133:0/2794037983 conn(0x7fc0780b31a0 msgr2=0x7fc07806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 -- 172.21.15.133:0/2794037983 shutdown_connections 2024-04-25T10:17:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.639+0000 7fc07edbb640 1 -- 172.21.15.133:0/2794037983 wait complete. 2024-04-25T10:17:40.659 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":666,"stamp":"2024-04-25T10:17:40.006515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":535036,"kb_used_data":20476,"kb_used_omap":12,"kb_used_meta":514419,"kb_avail":754439684,"statfs":{"total":773094113280,"available":772546236416,"internally_reserved":0,"allocated":20967424,"data_stored":10454328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":526765617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.397882"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745794+0000","last_change":"2024-04-25T10:17:37.664859+0000","last_active":"2024-04-25T10:17:38.745794+0000","last_peered":"2024-04-25T10:17:38.745794+0000","last_clean":"2024-04-25T10:17:38.745794+0000","last_became_active":"2024-04-25T10:17:37.664142+0000","last_became_peered":"2024-04-25T10:17:37.664142+0000","last_unstale":"2024-04-25T10:17:38.745794+0000","last_undegraded":"2024-04-25T10:17:38.745794+0000","last_fullsized":"2024-04-25T10:17:38.745794+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.967136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667707+0000","last_change":"2024-04-25T10:17:37.665180+0000","last_active":"2024-04-25T10:17:38.667707+0000","last_peered":"2024-04-25T10:17:38.667707+0000","last_clean":"2024-04-25T10:17:38.667707+0000","last_became_active":"2024-04-25T10:17:37.659367+0000","last_became_peered":"2024-04-25T10:17:37.659367+0000","last_unstale":"2024-04-25T10:17:38.667707+0000","last_undegraded":"2024-04-25T10:17:38.667707+0000","last_fullsized":"2024-04-25T10:17:38.667707+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:16.888327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917861+0000","last_change":"2024-04-25T10:17:37.673532+0000","last_active":"2024-04-25T10:17:38.917861+0000","last_peered":"2024-04-25T10:17:38.917861+0000","last_clean":"2024-04-25T10:17:38.917861+0000","last_became_active":"2024-04-25T10:17:37.671351+0000","last_became_peered":"2024-04-25T10:17:37.671351+0000","last_unstale":"2024-04-25T10:17:38.917861+0000","last_undegraded":"2024-04-25T10:17:38.917861+0000","last_fullsized":"2024-04-25T10:17:38.917861+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:34.420652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671627+0000","last_change":"2024-04-25T10:17:37.690110+0000","last_active":"2024-04-25T10:17:38.671627+0000","last_peered":"2024-04-25T10:17:38.671627+0000","last_clean":"2024-04-25T10:17:38.671627+0000","last_became_active":"2024-04-25T10:17:37.689395+0000","last_became_peered":"2024-04-25T10:17:37.689395+0000","last_unstale":"2024-04-25T10:17:38.671627+0000","last_undegraded":"2024-04-25T10:17:38.671627+0000","last_fullsized":"2024-04-25T10:17:38.671627+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:54.604229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745401+0000","last_change":"2024-04-25T10:17:37.663192+0000","last_active":"2024-04-25T10:17:38.745401+0000","last_peered":"2024-04-25T10:17:38.745401+0000","last_clean":"2024-04-25T10:17:38.745401+0000","last_became_active":"2024-04-25T10:17:37.662420+0000","last_became_peered":"2024-04-25T10:17:37.662420+0000","last_unstale":"2024-04-25T10:17:38.745401+0000","last_undegraded":"2024-04-25T10:17:38.745401+0000","last_fullsized":"2024-04-25T10:17:38.745401+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:26.638574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918292+0000","last_change":"2024-04-25T10:17:37.693412+0000","last_active":"2024-04-25T10:17:38.918292+0000","last_peered":"2024-04-25T10:17:38.918292+0000","last_clean":"2024-04-25T10:17:38.918292+0000","last_became_active":"2024-04-25T10:17:37.692250+0000","last_became_peered":"2024-04-25T10:17:37.692250+0000","last_unstale":"2024-04-25T10:17:38.918292+0000","last_undegraded":"2024-04-25T10:17:38.918292+0000","last_fullsized":"2024-04-25T10:17:38.918292+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:36.341736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917173+0000","last_change":"2024-04-25T10:17:37.685257+0000","last_active":"2024-04-25T10:17:38.917173+0000","last_peered":"2024-04-25T10:17:38.917173+0000","last_clean":"2024-04-25T10:17:38.917173+0000","last_became_active":"2024-04-25T10:17:37.684949+0000","last_became_peered":"2024-04-25T10:17:37.684949+0000","last_unstale":"2024-04-25T10:17:38.917173+0000","last_undegraded":"2024-04-25T10:17:38.917173+0000","last_fullsized":"2024-04-25T10:17:38.917173+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:16.888327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917656+0000","last_change":"2024-04-25T10:17:37.692154+0000","last_active":"2024-04-25T10:17:38.917656+0000","last_peered":"2024-04-25T10:17:38.917656+0000","last_clean":"2024-04-25T10:17:38.917656+0000","last_became_active":"2024-04-25T10:17:37.691633+0000","last_became_peered":"2024-04-25T10:17:37.691633+0000","last_unstale":"2024-04-25T10:17:38.917656+0000","last_undegraded":"2024-04-25T10:17:38.917656+0000","last_fullsized":"2024-04-25T10:17:38.917656+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:14.272525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918932+0000","last_change":"2024-04-25T10:17:37.662311+0000","last_active":"2024-04-25T10:17:38.918932+0000","last_peered":"2024-04-25T10:17:38.918932+0000","last_clean":"2024-04-25T10:17:38.918932+0000","last_became_active":"2024-04-25T10:17:37.660865+0000","last_became_peered":"2024-04-25T10:17:37.660865+0000","last_unstale":"2024-04-25T10:17:38.918932+0000","last_undegraded":"2024-04-25T10:17:38.918932+0000","last_fullsized":"2024-04-25T10:17:38.918932+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:57.680640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917600+0000","last_change":"2024-04-25T10:17:37.682006+0000","last_active":"2024-04-25T10:17:38.917600+0000","last_peered":"2024-04-25T10:17:38.917600+0000","last_clean":"2024-04-25T10:17:38.917600+0000","last_became_active":"2024-04-25T10:17:37.681625+0000","last_became_peered":"2024-04-25T10:17:37.681625+0000","last_unstale":"2024-04-25T10:17:38.917600+0000","last_undegraded":"2024-04-25T10:17:38.917600+0000","last_fullsized":"2024-04-25T10:17:38.917600+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:26.638574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671627+0000","last_change":"2024-04-25T10:17:37.683091+0000","last_active":"2024-04-25T10:17:38.671627+0000","last_peered":"2024-04-25T10:17:38.671627+0000","last_clean":"2024-04-25T10:17:38.671627+0000","last_became_active":"2024-04-25T10:17:37.682413+0000","last_became_peered":"2024-04-25T10:17:37.682413+0000","last_unstale":"2024-04-25T10:17:38.671627+0000","last_undegraded":"2024-04-25T10:17:38.671627+0000","last_fullsized":"2024-04-25T10:17:38.671627+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:29.997840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671200+0000","last_change":"2024-04-25T10:17:37.662747+0000","last_active":"2024-04-25T10:17:38.671200+0000","last_peered":"2024-04-25T10:17:38.671200+0000","last_clean":"2024-04-25T10:17:38.671200+0000","last_became_active":"2024-04-25T10:17:37.661005+0000","last_became_peered":"2024-04-25T10:17:37.661005+0000","last_unstale":"2024-04-25T10:17:38.671200+0000","last_undegraded":"2024-04-25T10:17:38.671200+0000","last_fullsized":"2024-04-25T10:17:38.671200+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:03.818015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917970+0000","last_change":"2024-04-25T10:17:37.693335+0000","last_active":"2024-04-25T10:17:38.917970+0000","last_peered":"2024-04-25T10:17:38.917970+0000","last_clean":"2024-04-25T10:17:38.917970+0000","last_became_active":"2024-04-25T10:17:37.693046+0000","last_became_peered":"2024-04-25T10:17:37.693046+0000","last_unstale":"2024-04-25T10:17:38.917970+0000","last_undegraded":"2024-04-25T10:17:38.917970+0000","last_fullsized":"2024-04-25T10:17:38.917970+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.854632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670346+0000","last_change":"2024-04-25T10:17:37.669119+0000","last_active":"2024-04-25T10:17:38.670346+0000","last_peered":"2024-04-25T10:17:38.670346+0000","last_clean":"2024-04-25T10:17:38.670346+0000","last_became_active":"2024-04-25T10:17:37.668207+0000","last_became_peered":"2024-04-25T10:17:37.668207+0000","last_unstale":"2024-04-25T10:17:38.670346+0000","last_undegraded":"2024-04-25T10:17:38.670346+0000","last_fullsized":"2024-04-25T10:17:38.670346+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:16.081885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670376+0000","last_change":"2024-04-25T10:17:37.667584+0000","last_active":"2024-04-25T10:17:38.670376+0000","last_peered":"2024-04-25T10:17:38.670376+0000","last_clean":"2024-04-25T10:17:38.670376+0000","last_became_active":"2024-04-25T10:17:37.667134+0000","last_became_peered":"2024-04-25T10:17:37.667134+0000","last_unstale":"2024-04-25T10:17:38.670376+0000","last_undegraded":"2024-04-25T10:17:38.670376+0000","last_fullsized":"2024-04-25T10:17:38.670376+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:20.587919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917700+0000","last_change":"2024-04-25T10:17:37.663028+0000","last_active":"2024-04-25T10:17:38.917700+0000","last_peered":"2024-04-25T10:17:38.917700+0000","last_clean":"2024-04-25T10:17:38.917700+0000","last_became_active":"2024-04-25T10:17:37.662316+0000","last_became_peered":"2024-04-25T10:17:37.662316+0000","last_unstale":"2024-04-25T10:17:38.917700+0000","last_undegraded":"2024-04-25T10:17:38.917700+0000","last_fullsized":"2024-04-25T10:17:38.917700+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:36.872185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670579+0000","last_change":"2024-04-25T10:17:37.670245+0000","last_active":"2024-04-25T10:17:38.670579+0000","last_peered":"2024-04-25T10:17:38.670579+0000","last_clean":"2024-04-25T10:17:38.670579+0000","last_became_active":"2024-04-25T10:17:37.669503+0000","last_became_peered":"2024-04-25T10:17:37.669503+0000","last_unstale":"2024-04-25T10:17:38.670579+0000","last_undegraded":"2024-04-25T10:17:38.670579+0000","last_fullsized":"2024-04-25T10:17:38.670579+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:07.412452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917833+0000","last_change":"2024-04-25T10:17:37.689729+0000","last_active":"2024-04-25T10:17:38.917833+0000","last_peered":"2024-04-25T10:17:38.917833+0000","last_clean":"2024-04-25T10:17:38.917833+0000","last_became_active":"2024-04-25T10:17:37.689380+0000","last_became_peered":"2024-04-25T10:17:37.689380+0000","last_unstale":"2024-04-25T10:17:38.917833+0000","last_undegraded":"2024-04-25T10:17:38.917833+0000","last_fullsized":"2024-04-25T10:17:38.917833+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.967136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745354+0000","last_change":"2024-04-25T10:17:37.672527+0000","last_active":"2024-04-25T10:17:38.745354+0000","last_peered":"2024-04-25T10:17:38.745354+0000","last_clean":"2024-04-25T10:17:38.745354+0000","last_became_active":"2024-04-25T10:17:37.670628+0000","last_became_peered":"2024-04-25T10:17:37.670628+0000","last_unstale":"2024-04-25T10:17:38.745354+0000","last_undegraded":"2024-04-25T10:17:38.745354+0000","last_fullsized":"2024-04-25T10:17:38.745354+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:00.878676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918682+0000","last_change":"2024-04-25T10:17:37.662891+0000","last_active":"2024-04-25T10:17:38.918682+0000","last_peered":"2024-04-25T10:17:38.918682+0000","last_clean":"2024-04-25T10:17:38.918682+0000","last_became_active":"2024-04-25T10:17:37.661454+0000","last_became_peered":"2024-04-25T10:17:37.661454+0000","last_unstale":"2024-04-25T10:17:38.918682+0000","last_undegraded":"2024-04-25T10:17:38.918682+0000","last_fullsized":"2024-04-25T10:17:38.918682+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:41.103605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917385+0000","last_change":"2024-04-25T10:17:37.674695+0000","last_active":"2024-04-25T10:17:38.917385+0000","last_peered":"2024-04-25T10:17:38.917385+0000","last_clean":"2024-04-25T10:17:38.917385+0000","last_became_active":"2024-04-25T10:17:37.674361+0000","last_became_peered":"2024-04-25T10:17:37.674361+0000","last_unstale":"2024-04-25T10:17:38.917385+0000","last_undegraded":"2024-04-25T10:17:38.917385+0000","last_fullsized":"2024-04-25T10:17:38.917385+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:20.188344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670446+0000","last_change":"2024-04-25T10:17:37.658044+0000","last_active":"2024-04-25T10:17:38.670446+0000","last_peered":"2024-04-25T10:17:38.670446+0000","last_clean":"2024-04-25T10:17:38.670446+0000","last_became_active":"2024-04-25T10:17:37.656211+0000","last_became_peered":"2024-04-25T10:17:37.656211+0000","last_unstale":"2024-04-25T10:17:38.670446+0000","last_undegraded":"2024-04-25T10:17:38.670446+0000","last_fullsized":"2024-04-25T10:17:38.670446+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:58.734666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918564+0000","last_change":"2024-04-25T10:17:37.662031+0000","last_active":"2024-04-25T10:17:38.918564+0000","last_peered":"2024-04-25T10:17:38.918564+0000","last_clean":"2024-04-25T10:17:38.918564+0000","last_became_active":"2024-04-25T10:17:37.661460+0000","last_became_peered":"2024-04-25T10:17:37.661460+0000","last_unstale":"2024-04-25T10:17:38.918564+0000","last_undegraded":"2024-04-25T10:17:38.918564+0000","last_fullsized":"2024-04-25T10:17:38.918564+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:03.138689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.744869+0000","last_change":"2024-04-25T10:17:37.671990+0000","last_active":"2024-04-25T10:17:38.744869+0000","last_peered":"2024-04-25T10:17:38.744869+0000","last_clean":"2024-04-25T10:17:38.744869+0000","last_became_active":"2024-04-25T10:17:37.671038+0000","last_became_peered":"2024-04-25T10:17:37.671038+0000","last_unstale":"2024-04-25T10:17:38.744869+0000","last_undegraded":"2024-04-25T10:17:38.744869+0000","last_fullsized":"2024-04-25T10:17:38.744869+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:16.888327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745815+0000","last_change":"2024-04-25T10:17:37.669069+0000","last_active":"2024-04-25T10:17:38.745815+0000","last_peered":"2024-04-25T10:17:38.745815+0000","last_clean":"2024-04-25T10:17:38.745815+0000","last_became_active":"2024-04-25T10:17:37.668324+0000","last_became_peered":"2024-04-25T10:17:37.668324+0000","last_unstale":"2024-04-25T10:17:38.745815+0000","last_undegraded":"2024-04-25T10:17:38.745815+0000","last_fullsized":"2024-04-25T10:17:38.745815+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:34.420652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918790+0000","last_change":"2024-04-25T10:17:37.673423+0000","last_active":"2024-04-25T10:17:38.918790+0000","last_peered":"2024-04-25T10:17:38.918790+0000","last_clean":"2024-04-25T10:17:38.918790+0000","last_became_active":"2024-04-25T10:17:37.671530+0000","last_became_peered":"2024-04-25T10:17:37.671530+0000","last_unstale":"2024-04-25T10:17:38.918790+0000","last_undegraded":"2024-04-25T10:17:38.918790+0000","last_fullsized":"2024-04-25T10:17:38.918790+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:28.948966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671947+0000","last_change":"2024-04-25T10:17:37.682771+0000","last_active":"2024-04-25T10:17:38.671947+0000","last_peered":"2024-04-25T10:17:38.671947+0000","last_clean":"2024-04-25T10:17:38.671947+0000","last_became_active":"2024-04-25T10:17:37.682029+0000","last_became_peered":"2024-04-25T10:17:37.682029+0000","last_unstale":"2024-04-25T10:17:38.671947+0000","last_undegraded":"2024-04-25T10:17:38.671947+0000","last_fullsized":"2024-04-25T10:17:38.671947+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:16.081885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671171+0000","last_change":"2024-04-25T10:17:37.656015+0000","last_active":"2024-04-25T10:17:38.671171+0000","last_peered":"2024-04-25T10:17:38.671171+0000","last_clean":"2024-04-25T10:17:38.671171+0000","last_became_active":"2024-04-25T10:17:37.654387+0000","last_became_peered":"2024-04-25T10:17:37.654387+0000","last_unstale":"2024-04-25T10:17:38.671171+0000","last_undegraded":"2024-04-25T10:17:38.671171+0000","last_fullsized":"2024-04-25T10:17:38.671171+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:36.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667655+0000","last_change":"2024-04-25T10:17:37.692954+0000","last_active":"2024-04-25T10:17:38.667655+0000","last_peered":"2024-04-25T10:17:38.667655+0000","last_clean":"2024-04-25T10:17:38.667655+0000","last_became_active":"2024-04-25T10:17:37.692584+0000","last_became_peered":"2024-04-25T10:17:37.692584+0000","last_unstale":"2024-04-25T10:17:38.667655+0000","last_undegraded":"2024-04-25T10:17:38.667655+0000","last_fullsized":"2024-04-25T10:17:38.667655+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:27.043929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918005+0000","last_change":"2024-04-25T10:17:37.662080+0000","last_active":"2024-04-25T10:17:38.918005+0000","last_peered":"2024-04-25T10:17:38.918005+0000","last_clean":"2024-04-25T10:17:38.918005+0000","last_became_active":"2024-04-25T10:17:37.661305+0000","last_became_peered":"2024-04-25T10:17:37.661305+0000","last_unstale":"2024-04-25T10:17:38.918005+0000","last_undegraded":"2024-04-25T10:17:38.918005+0000","last_fullsized":"2024-04-25T10:17:38.918005+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:22.702360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918994+0000","last_change":"2024-04-25T10:17:37.672148+0000","last_active":"2024-04-25T10:17:38.918994+0000","last_peered":"2024-04-25T10:17:38.918994+0000","last_clean":"2024-04-25T10:17:38.918994+0000","last_became_active":"2024-04-25T10:17:37.671839+0000","last_became_peered":"2024-04-25T10:17:37.671839+0000","last_unstale":"2024-04-25T10:17:38.918994+0000","last_undegraded":"2024-04-25T10:17:38.918994+0000","last_fullsized":"2024-04-25T10:17:38.918994+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:12.189982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918115+0000","last_change":"2024-04-25T10:17:37.653383+0000","last_active":"2024-04-25T10:17:38.918115+0000","last_peered":"2024-04-25T10:17:38.918115+0000","last_clean":"2024-04-25T10:17:38.918115+0000","last_became_active":"2024-04-25T10:17:37.650614+0000","last_became_peered":"2024-04-25T10:17:37.650614+0000","last_unstale":"2024-04-25T10:17:38.918115+0000","last_undegraded":"2024-04-25T10:17:38.918115+0000","last_fullsized":"2024-04-25T10:17:38.918115+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:28.337705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918283+0000","last_change":"2024-04-25T10:17:37.673070+0000","last_active":"2024-04-25T10:17:38.918283+0000","last_peered":"2024-04-25T10:17:38.918283+0000","last_clean":"2024-04-25T10:17:38.918283+0000","last_became_active":"2024-04-25T10:17:37.670871+0000","last_became_peered":"2024-04-25T10:17:37.670871+0000","last_unstale":"2024-04-25T10:17:38.918283+0000","last_undegraded":"2024-04-25T10:17:38.918283+0000","last_fullsized":"2024-04-25T10:17:38.918283+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:46.076920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"194'4","reported_seq":18,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.919114+0000","last_change":"2024-04-25T10:17:36.624285+0000","last_active":"2024-04-25T10:17:38.919114+0000","last_peered":"2024-04-25T10:17:38.919114+0000","last_clean":"2024-04-25T10:17:38.919114+0000","last_became_active":"2024-04-25T10:17:36.623990+0000","last_became_peered":"2024-04-25T10:17:36.623990+0000","last_unstale":"2024-04-25T10:17:38.919114+0000","last_undegraded":"2024-04-25T10:17:38.919114+0000","last_fullsized":"2024-04-25T10:17:38.919114+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:45.716255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917952+0000","last_change":"2024-04-25T10:17:37.670717+0000","last_active":"2024-04-25T10:17:38.917952+0000","last_peered":"2024-04-25T10:17:38.917952+0000","last_clean":"2024-04-25T10:17:38.917952+0000","last_became_active":"2024-04-25T10:17:37.667877+0000","last_became_peered":"2024-04-25T10:17:37.667877+0000","last_unstale":"2024-04-25T10:17:38.917952+0000","last_undegraded":"2024-04-25T10:17:38.917952+0000","last_fullsized":"2024-04-25T10:17:38.917952+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:15.072919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671736+0000","last_change":"2024-04-25T10:17:36.653523+0000","last_active":"2024-04-25T10:17:38.671736+0000","last_peered":"2024-04-25T10:17:38.671736+0000","last_clean":"2024-04-25T10:17:38.671736+0000","last_became_active":"2024-04-25T10:17:36.652827+0000","last_became_peered":"2024-04-25T10:17:36.652827+0000","last_unstale":"2024-04-25T10:17:38.671736+0000","last_undegraded":"2024-04-25T10:17:38.671736+0000","last_fullsized":"2024-04-25T10:17:38.671736+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:47.688629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671774+0000","last_change":"2024-04-25T10:17:37.669799+0000","last_active":"2024-04-25T10:17:38.671774+0000","last_peered":"2024-04-25T10:17:38.671774+0000","last_clean":"2024-04-25T10:17:38.671774+0000","last_became_active":"2024-04-25T10:17:37.668425+0000","last_became_peered":"2024-04-25T10:17:37.668425+0000","last_unstale":"2024-04-25T10:17:38.671774+0000","last_undegraded":"2024-04-25T10:17:38.671774+0000","last_fullsized":"2024-04-25T10:17:38.671774+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:47.207975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918532+0000","last_change":"2024-04-25T10:17:36.641983+0000","last_active":"2024-04-25T10:17:38.918532+0000","last_peered":"2024-04-25T10:17:38.918532+0000","last_clean":"2024-04-25T10:17:38.918532+0000","last_became_active":"2024-04-25T10:17:36.641080+0000","last_became_peered":"2024-04-25T10:17:36.641080+0000","last_unstale":"2024-04-25T10:17:38.918532+0000","last_undegraded":"2024-04-25T10:17:38.918532+0000","last_fullsized":"2024-04-25T10:17:38.918532+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:10.900721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.916993+0000","last_change":"2024-04-25T10:17:37.679299+0000","last_active":"2024-04-25T10:17:38.916993+0000","last_peered":"2024-04-25T10:17:38.916993+0000","last_clean":"2024-04-25T10:17:38.916993+0000","last_became_active":"2024-04-25T10:17:37.679042+0000","last_became_peered":"2024-04-25T10:17:37.679042+0000","last_unstale":"2024-04-25T10:17:38.916993+0000","last_undegraded":"2024-04-25T10:17:38.916993+0000","last_fullsized":"2024-04-25T10:17:38.916993+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:36.330126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670469+0000","last_change":"2024-04-25T10:17:36.659961+0000","last_active":"2024-04-25T10:17:38.670469+0000","last_peered":"2024-04-25T10:17:38.670469+0000","last_clean":"2024-04-25T10:17:38.670469+0000","last_became_active":"2024-04-25T10:17:36.659723+0000","last_became_peered":"2024-04-25T10:17:36.659723+0000","last_unstale":"2024-04-25T10:17:38.670469+0000","last_undegraded":"2024-04-25T10:17:38.670469+0000","last_fullsized":"2024-04-25T10:17:38.670469+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:35.860903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.669921+0000","last_change":"2024-04-25T10:17:37.654467+0000","last_active":"2024-04-25T10:17:38.669921+0000","last_peered":"2024-04-25T10:17:38.669921+0000","last_clean":"2024-04-25T10:17:38.669921+0000","last_became_active":"2024-04-25T10:17:37.653714+0000","last_became_peered":"2024-04-25T10:17:37.653714+0000","last_unstale":"2024-04-25T10:17:38.669921+0000","last_undegraded":"2024-04-25T10:17:38.669921+0000","last_fullsized":"2024-04-25T10:17:38.669921+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:54.998816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745172+0000","last_change":"2024-04-25T10:17:36.661718+0000","last_active":"2024-04-25T10:17:38.745172+0000","last_peered":"2024-04-25T10:17:38.745172+0000","last_clean":"2024-04-25T10:17:38.745172+0000","last_became_active":"2024-04-25T10:17:36.661093+0000","last_became_peered":"2024-04-25T10:17:36.661093+0000","last_unstale":"2024-04-25T10:17:38.745172+0000","last_undegraded":"2024-04-25T10:17:38.745172+0000","last_fullsized":"2024-04-25T10:17:38.745172+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:39.439515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670400+0000","last_change":"2024-04-25T10:17:37.658750+0000","last_active":"2024-04-25T10:17:38.670400+0000","last_peered":"2024-04-25T10:17:38.670400+0000","last_clean":"2024-04-25T10:17:38.670400+0000","last_became_active":"2024-04-25T10:17:37.657273+0000","last_became_peered":"2024-04-25T10:17:37.657273+0000","last_unstale":"2024-04-25T10:17:38.670400+0000","last_undegraded":"2024-04-25T10:17:38.670400+0000","last_fullsized":"2024-04-25T10:17:38.670400+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:17.159225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.669478+0000","last_change":"2024-04-25T10:17:36.656913+0000","last_active":"2024-04-25T10:17:38.669478+0000","last_peered":"2024-04-25T10:17:38.669478+0000","last_clean":"2024-04-25T10:17:38.669478+0000","last_became_active":"2024-04-25T10:17:36.656237+0000","last_became_peered":"2024-04-25T10:17:36.656237+0000","last_unstale":"2024-04-25T10:17:38.669478+0000","last_undegraded":"2024-04-25T10:17:38.669478+0000","last_fullsized":"2024-04-25T10:17:38.669478+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:29.361785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918988+0000","last_change":"2024-04-25T10:17:37.680702+0000","last_active":"2024-04-25T10:17:38.918988+0000","last_peered":"2024-04-25T10:17:38.918988+0000","last_clean":"2024-04-25T10:17:38.918988+0000","last_became_active":"2024-04-25T10:17:37.680420+0000","last_became_peered":"2024-04-25T10:17:37.680420+0000","last_unstale":"2024-04-25T10:17:38.918988+0000","last_undegraded":"2024-04-25T10:17:38.918988+0000","last_fullsized":"2024-04-25T10:17:38.918988+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:49.405793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917260+0000","last_change":"2024-04-25T10:17:36.627592+0000","last_active":"2024-04-25T10:17:38.917260+0000","last_peered":"2024-04-25T10:17:38.917260+0000","last_clean":"2024-04-25T10:17:38.917260+0000","last_became_active":"2024-04-25T10:17:36.627144+0000","last_became_peered":"2024-04-25T10:17:36.627144+0000","last_unstale":"2024-04-25T10:17:38.917260+0000","last_undegraded":"2024-04-25T10:17:38.917260+0000","last_fullsized":"2024-04-25T10:17:38.917260+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:55.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670650+0000","last_change":"2024-04-25T10:17:37.682060+0000","last_active":"2024-04-25T10:17:38.670650+0000","last_peered":"2024-04-25T10:17:38.670650+0000","last_clean":"2024-04-25T10:17:38.670650+0000","last_became_active":"2024-04-25T10:17:37.681777+0000","last_became_peered":"2024-04-25T10:17:37.681777+0000","last_unstale":"2024-04-25T10:17:38.670650+0000","last_undegraded":"2024-04-25T10:17:38.670650+0000","last_fullsized":"2024-04-25T10:17:38.670650+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:44.379669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.746087+0000","last_change":"2024-04-25T10:17:36.650543+0000","last_active":"2024-04-25T10:17:38.746087+0000","last_peered":"2024-04-25T10:17:38.746087+0000","last_clean":"2024-04-25T10:17:38.746087+0000","last_became_active":"2024-04-25T10:17:36.649524+0000","last_became_peered":"2024-04-25T10:17:36.649524+0000","last_unstale":"2024-04-25T10:17:38.746087+0000","last_undegraded":"2024-04-25T10:17:38.746087+0000","last_fullsized":"2024-04-25T10:17:38.746087+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.385986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670781+0000","last_change":"2024-04-25T10:17:37.656908+0000","last_active":"2024-04-25T10:17:38.670781+0000","last_peered":"2024-04-25T10:17:38.670781+0000","last_clean":"2024-04-25T10:17:38.670781+0000","last_became_active":"2024-04-25T10:17:37.656144+0000","last_became_peered":"2024-04-25T10:17:37.656144+0000","last_unstale":"2024-04-25T10:17:38.670781+0000","last_undegraded":"2024-04-25T10:17:38.670781+0000","last_fullsized":"2024-04-25T10:17:38.670781+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:09.345050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.673040+0000","last_change":"2024-04-25T10:17:36.656914+0000","last_active":"2024-04-25T10:17:38.673040+0000","last_peered":"2024-04-25T10:17:38.673040+0000","last_clean":"2024-04-25T10:17:38.673040+0000","last_became_active":"2024-04-25T10:17:36.656348+0000","last_became_peered":"2024-04-25T10:17:36.656348+0000","last_unstale":"2024-04-25T10:17:38.673040+0000","last_undegraded":"2024-04-25T10:17:38.673040+0000","last_fullsized":"2024-04-25T10:17:38.673040+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:01.197915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918451+0000","last_change":"2024-04-25T10:17:37.673148+0000","last_active":"2024-04-25T10:17:38.918451+0000","last_peered":"2024-04-25T10:17:38.918451+0000","last_clean":"2024-04-25T10:17:38.918451+0000","last_became_active":"2024-04-25T10:17:37.671219+0000","last_became_peered":"2024-04-25T10:17:37.671219+0000","last_unstale":"2024-04-25T10:17:38.918451+0000","last_undegraded":"2024-04-25T10:17:38.918451+0000","last_fullsized":"2024-04-25T10:17:38.918451+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:53.995344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"194'3","reported_seq":17,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745574+0000","last_change":"2024-04-25T10:17:36.645043+0000","last_active":"2024-04-25T10:17:38.745574+0000","last_peered":"2024-04-25T10:17:38.745574+0000","last_clean":"2024-04-25T10:17:38.745574+0000","last_became_active":"2024-04-25T10:17:36.644445+0000","last_became_peered":"2024-04-25T10:17:36.644445+0000","last_unstale":"2024-04-25T10:17:38.745574+0000","last_undegraded":"2024-04-25T10:17:38.745574+0000","last_fullsized":"2024-04-25T10:17:38.745574+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:46.839439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.672948+0000","last_change":"2024-04-25T10:17:37.682345+0000","last_active":"2024-04-25T10:17:38.672948+0000","last_peered":"2024-04-25T10:17:38.672948+0000","last_clean":"2024-04-25T10:17:38.672948+0000","last_became_active":"2024-04-25T10:17:37.682073+0000","last_became_peered":"2024-04-25T10:17:37.682073+0000","last_unstale":"2024-04-25T10:17:38.672948+0000","last_undegraded":"2024-04-25T10:17:38.672948+0000","last_fullsized":"2024-04-25T10:17:38.672948+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:07.412452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"194'6","reported_seq":20,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918845+0000","last_change":"2024-04-25T10:17:36.641772+0000","last_active":"2024-04-25T10:17:38.918845+0000","last_peered":"2024-04-25T10:17:38.918845+0000","last_clean":"2024-04-25T10:17:38.918845+0000","last_became_active":"2024-04-25T10:17:36.641102+0000","last_became_peered":"2024-04-25T10:17:36.641102+0000","last_unstale":"2024-04-25T10:17:38.918845+0000","last_undegraded":"2024-04-25T10:17:38.918845+0000","last_fullsized":"2024-04-25T10:17:38.918845+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:44.673880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667278+0000","last_change":"2024-04-25T10:17:37.689343+0000","last_active":"2024-04-25T10:17:38.667278+0000","last_peered":"2024-04-25T10:17:38.667278+0000","last_clean":"2024-04-25T10:17:38.667278+0000","last_became_active":"2024-04-25T10:17:37.689081+0000","last_became_peered":"2024-04-25T10:17:37.689081+0000","last_unstale":"2024-04-25T10:17:38.667278+0000","last_undegraded":"2024-04-25T10:17:38.667278+0000","last_fullsized":"2024-04-25T10:17:38.667278+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:12.382293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.672951+0000","last_change":"2024-04-25T10:17:36.659027+0000","last_active":"2024-04-25T10:17:38.672951+0000","last_peered":"2024-04-25T10:17:38.672951+0000","last_clean":"2024-04-25T10:17:38.672951+0000","last_became_active":"2024-04-25T10:17:36.654181+0000","last_became_peered":"2024-04-25T10:17:36.654181+0000","last_unstale":"2024-04-25T10:17:38.672951+0000","last_undegraded":"2024-04-25T10:17:38.672951+0000","last_fullsized":"2024-04-25T10:17:38.672951+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:34.681416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670337+0000","last_change":"2024-04-25T10:17:37.652524+0000","last_active":"2024-04-25T10:17:38.670337+0000","last_peered":"2024-04-25T10:17:38.670337+0000","last_clean":"2024-04-25T10:17:38.670337+0000","last_became_active":"2024-04-25T10:17:37.651692+0000","last_became_peered":"2024-04-25T10:17:37.651692+0000","last_unstale":"2024-04-25T10:17:38.670337+0000","last_undegraded":"2024-04-25T10:17:38.670337+0000","last_fullsized":"2024-04-25T10:17:38.670337+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:29.997840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667511+0000","last_change":"2024-04-25T10:17:36.623640+0000","last_active":"2024-04-25T10:17:38.667511+0000","last_peered":"2024-04-25T10:17:38.667511+0000","last_clean":"2024-04-25T10:17:38.667511+0000","last_became_active":"2024-04-25T10:17:36.622668+0000","last_became_peered":"2024-04-25T10:17:36.622668+0000","last_unstale":"2024-04-25T10:17:38.667511+0000","last_undegraded":"2024-04-25T10:17:38.667511+0000","last_fullsized":"2024-04-25T10:17:38.667511+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:46.839439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.916962+0000","last_change":"2024-04-25T10:17:37.667495+0000","last_active":"2024-04-25T10:17:38.916962+0000","last_peered":"2024-04-25T10:17:38.916962+0000","last_clean":"2024-04-25T10:17:38.916962+0000","last_became_active":"2024-04-25T10:17:37.667165+0000","last_became_peered":"2024-04-25T10:17:37.667165+0000","last_unstale":"2024-04-25T10:17:38.916962+0000","last_undegraded":"2024-04-25T10:17:38.916962+0000","last_fullsized":"2024-04-25T10:17:38.916962+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:12.382293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917573+0000","last_change":"2024-04-25T10:17:36.623729+0000","last_active":"2024-04-25T10:17:38.917573+0000","last_peered":"2024-04-25T10:17:38.917573+0000","last_clean":"2024-04-25T10:17:38.917573+0000","last_became_active":"2024-04-25T10:17:36.623001+0000","last_became_peered":"2024-04-25T10:17:36.623001+0000","last_unstale":"2024-04-25T10:17:38.917573+0000","last_undegraded":"2024-04-25T10:17:38.917573+0000","last_fullsized":"2024-04-25T10:17:38.917573+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.305901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670446+0000","last_change":"2024-04-25T10:17:37.657805+0000","last_active":"2024-04-25T10:17:38.670446+0000","last_peered":"2024-04-25T10:17:38.670446+0000","last_clean":"2024-04-25T10:17:38.670446+0000","last_became_active":"2024-04-25T10:17:37.656981+0000","last_became_peered":"2024-04-25T10:17:37.656981+0000","last_unstale":"2024-04-25T10:17:38.670446+0000","last_undegraded":"2024-04-25T10:17:38.670446+0000","last_fullsized":"2024-04-25T10:17:38.670446+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:20.686031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"194'8","reported_seq":22,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670497+0000","last_change":"2024-04-25T10:17:36.646281+0000","last_active":"2024-04-25T10:17:38.670497+0000","last_peered":"2024-04-25T10:17:38.670497+0000","last_clean":"2024-04-25T10:17:38.670497+0000","last_became_active":"2024-04-25T10:17:36.645591+0000","last_became_peered":"2024-04-25T10:17:36.645591+0000","last_unstale":"2024-04-25T10:17:38.670497+0000","last_undegraded":"2024-04-25T10:17:38.670497+0000","last_fullsized":"2024-04-25T10:17:38.670497+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:14.560561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667745+0000","last_change":"2024-04-25T10:17:37.660546+0000","last_active":"2024-04-25T10:17:38.667745+0000","last_peered":"2024-04-25T10:17:38.667745+0000","last_clean":"2024-04-25T10:17:38.667745+0000","last_became_active":"2024-04-25T10:17:37.659700+0000","last_became_peered":"2024-04-25T10:17:37.659700+0000","last_unstale":"2024-04-25T10:17:38.667745+0000","last_undegraded":"2024-04-25T10:17:38.667745+0000","last_fullsized":"2024-04-25T10:17:38.667745+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:46.076920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918807+0000","last_change":"2024-04-25T10:17:36.630837+0000","last_active":"2024-04-25T10:17:38.918807+0000","last_peered":"2024-04-25T10:17:38.918807+0000","last_clean":"2024-04-25T10:17:38.918807+0000","last_became_active":"2024-04-25T10:17:36.630509+0000","last_became_peered":"2024-04-25T10:17:36.630509+0000","last_unstale":"2024-04-25T10:17:38.918807+0000","last_undegraded":"2024-04-25T10:17:38.918807+0000","last_fullsized":"2024-04-25T10:17:38.918807+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:57.918425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745324+0000","last_change":"2024-04-25T10:17:37.690908+0000","last_active":"2024-04-25T10:17:38.745324+0000","last_peered":"2024-04-25T10:17:38.745324+0000","last_clean":"2024-04-25T10:17:38.745324+0000","last_became_active":"2024-04-25T10:17:37.690613+0000","last_became_peered":"2024-04-25T10:17:37.690613+0000","last_unstale":"2024-04-25T10:17:38.745324+0000","last_undegraded":"2024-04-25T10:17:38.745324+0000","last_fullsized":"2024-04-25T10:17:38.745324+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:20.188344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918324+0000","last_change":"2024-04-25T10:17:36.621384+0000","last_active":"2024-04-25T10:17:38.918324+0000","last_peered":"2024-04-25T10:17:38.918324+0000","last_clean":"2024-04-25T10:17:38.918324+0000","last_became_active":"2024-04-25T10:17:36.621009+0000","last_became_peered":"2024-04-25T10:17:36.621009+0000","last_unstale":"2024-04-25T10:17:38.918324+0000","last_undegraded":"2024-04-25T10:17:38.918324+0000","last_fullsized":"2024-04-25T10:17:38.918324+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:38.254914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918694+0000","last_change":"2024-04-25T10:17:37.673461+0000","last_active":"2024-04-25T10:17:38.918694+0000","last_peered":"2024-04-25T10:17:38.918694+0000","last_clean":"2024-04-25T10:17:38.918694+0000","last_became_active":"2024-04-25T10:17:37.673186+0000","last_became_peered":"2024-04-25T10:17:37.673186+0000","last_unstale":"2024-04-25T10:17:38.918694+0000","last_undegraded":"2024-04-25T10:17:38.918694+0000","last_fullsized":"2024-04-25T10:17:38.918694+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:37.405035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670021+0000","last_change":"2024-04-25T10:17:36.639127+0000","last_active":"2024-04-25T10:17:38.670021+0000","last_peered":"2024-04-25T10:17:38.670021+0000","last_clean":"2024-04-25T10:17:38.670021+0000","last_became_active":"2024-04-25T10:17:36.638346+0000","last_became_peered":"2024-04-25T10:17:36.638346+0000","last_unstale":"2024-04-25T10:17:38.670021+0000","last_undegraded":"2024-04-25T10:17:38.670021+0000","last_fullsized":"2024-04-25T10:17:38.670021+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:55.489855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671119+0000","last_change":"2024-04-25T10:17:37.691777+0000","last_active":"2024-04-25T10:17:38.671119+0000","last_peered":"2024-04-25T10:17:38.671119+0000","last_clean":"2024-04-25T10:17:38.671119+0000","last_became_active":"2024-04-25T10:17:37.691377+0000","last_became_peered":"2024-04-25T10:17:37.691377+0000","last_unstale":"2024-04-25T10:17:38.671119+0000","last_undegraded":"2024-04-25T10:17:38.671119+0000","last_fullsized":"2024-04-25T10:17:38.671119+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:05.590252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671346+0000","last_change":"2024-04-25T10:17:36.639269+0000","last_active":"2024-04-25T10:17:38.671346+0000","last_peered":"2024-04-25T10:17:38.671346+0000","last_clean":"2024-04-25T10:17:38.671346+0000","last_became_active":"2024-04-25T10:17:36.638503+0000","last_became_peered":"2024-04-25T10:17:36.638503+0000","last_unstale":"2024-04-25T10:17:38.671346+0000","last_undegraded":"2024-04-25T10:17:38.671346+0000","last_fullsized":"2024-04-25T10:17:38.671346+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:06.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670202+0000","last_change":"2024-04-25T10:17:37.669336+0000","last_active":"2024-04-25T10:17:38.670202+0000","last_peered":"2024-04-25T10:17:38.670202+0000","last_clean":"2024-04-25T10:17:38.670202+0000","last_became_active":"2024-04-25T10:17:37.668537+0000","last_became_peered":"2024-04-25T10:17:37.668537+0000","last_unstale":"2024-04-25T10:17:38.670202+0000","last_undegraded":"2024-04-25T10:17:38.670202+0000","last_fullsized":"2024-04-25T10:17:38.670202+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:54.604229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.673129+0000","last_change":"2024-04-25T10:17:36.655379+0000","last_active":"2024-04-25T10:17:38.673129+0000","last_peered":"2024-04-25T10:17:38.673129+0000","last_clean":"2024-04-25T10:17:38.673129+0000","last_became_active":"2024-04-25T10:17:36.646567+0000","last_became_peered":"2024-04-25T10:17:36.646567+0000","last_unstale":"2024-04-25T10:17:38.673129+0000","last_undegraded":"2024-04-25T10:17:38.673129+0000","last_fullsized":"2024-04-25T10:17:38.673129+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:29.721104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745016+0000","last_change":"2024-04-25T10:17:37.664398+0000","last_active":"2024-04-25T10:17:38.745016+0000","last_peered":"2024-04-25T10:17:38.745016+0000","last_clean":"2024-04-25T10:17:38.745016+0000","last_became_active":"2024-04-25T10:17:37.663672+0000","last_became_peered":"2024-04-25T10:17:37.663672+0000","last_unstale":"2024-04-25T10:17:38.745016+0000","last_undegraded":"2024-04-25T10:17:38.745016+0000","last_fullsized":"2024-04-25T10:17:38.745016+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:46.076920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670511+0000","last_change":"2024-04-25T10:17:36.650440+0000","last_active":"2024-04-25T10:17:38.670511+0000","last_peered":"2024-04-25T10:17:38.670511+0000","last_clean":"2024-04-25T10:17:38.670511+0000","last_became_active":"2024-04-25T10:17:36.650123+0000","last_became_peered":"2024-04-25T10:17:36.650123+0000","last_unstale":"2024-04-25T10:17:38.670511+0000","last_undegraded":"2024-04-25T10:17:38.670511+0000","last_fullsized":"2024-04-25T10:17:38.670511+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:36.843350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667359+0000","last_change":"2024-04-25T10:17:37.654496+0000","last_active":"2024-04-25T10:17:38.667359+0000","last_peered":"2024-04-25T10:17:38.667359+0000","last_clean":"2024-04-25T10:17:38.667359+0000","last_became_active":"2024-04-25T10:17:37.653801+0000","last_became_peered":"2024-04-25T10:17:37.653801+0000","last_unstale":"2024-04-25T10:17:38.667359+0000","last_undegraded":"2024-04-25T10:17:38.667359+0000","last_fullsized":"2024-04-25T10:17:38.667359+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:36.330126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670379+0000","last_change":"2024-04-25T10:17:36.647950+0000","last_active":"2024-04-25T10:17:38.670379+0000","last_peered":"2024-04-25T10:17:38.670379+0000","last_clean":"2024-04-25T10:17:38.670379+0000","last_became_active":"2024-04-25T10:17:36.647739+0000","last_became_peered":"2024-04-25T10:17:36.647739+0000","last_unstale":"2024-04-25T10:17:38.670379+0000","last_undegraded":"2024-04-25T10:17:38.670379+0000","last_fullsized":"2024-04-25T10:17:38.670379+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:32.050946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917401+0000","last_change":"2024-04-25T10:17:37.681361+0000","last_active":"2024-04-25T10:17:38.917401+0000","last_peered":"2024-04-25T10:17:38.917401+0000","last_clean":"2024-04-25T10:17:38.917401+0000","last_became_active":"2024-04-25T10:17:37.681034+0000","last_became_peered":"2024-04-25T10:17:37.681034+0000","last_unstale":"2024-04-25T10:17:38.917401+0000","last_undegraded":"2024-04-25T10:17:38.917401+0000","last_fullsized":"2024-04-25T10:17:38.917401+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:00.878676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917687+0000","last_change":"2024-04-25T10:17:36.621218+0000","last_active":"2024-04-25T10:17:38.917687+0000","last_peered":"2024-04-25T10:17:38.917687+0000","last_clean":"2024-04-25T10:17:38.917687+0000","last_became_active":"2024-04-25T10:17:36.620883+0000","last_became_peered":"2024-04-25T10:17:36.620883+0000","last_unstale":"2024-04-25T10:17:38.917687+0000","last_undegraded":"2024-04-25T10:17:38.917687+0000","last_fullsized":"2024-04-25T10:17:38.917687+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:18.426640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667909+0000","last_change":"2024-04-25T10:17:37.664826+0000","last_active":"2024-04-25T10:17:38.667909+0000","last_peered":"2024-04-25T10:17:38.667909+0000","last_clean":"2024-04-25T10:17:38.667909+0000","last_became_active":"2024-04-25T10:17:37.664487+0000","last_became_peered":"2024-04-25T10:17:37.664487+0000","last_unstale":"2024-04-25T10:17:38.667909+0000","last_undegraded":"2024-04-25T10:17:38.667909+0000","last_fullsized":"2024-04-25T10:17:38.667909+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:20.188344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671410+0000","last_change":"2024-04-25T10:17:36.660226+0000","last_active":"2024-04-25T10:17:38.671410+0000","last_peered":"2024-04-25T10:17:38.671410+0000","last_clean":"2024-04-25T10:17:38.671410+0000","last_became_active":"2024-04-25T10:17:36.659670+0000","last_became_peered":"2024-04-25T10:17:36.659670+0000","last_unstale":"2024-04-25T10:17:38.671410+0000","last_undegraded":"2024-04-25T10:17:38.671410+0000","last_fullsized":"2024-04-25T10:17:38.671410+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:21.940556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745571+0000","last_change":"2024-04-25T10:17:37.685235+0000","last_active":"2024-04-25T10:17:38.745571+0000","last_peered":"2024-04-25T10:17:38.745571+0000","last_clean":"2024-04-25T10:17:38.745571+0000","last_became_active":"2024-04-25T10:17:37.684577+0000","last_became_peered":"2024-04-25T10:17:37.684577+0000","last_unstale":"2024-04-25T10:17:38.745571+0000","last_undegraded":"2024-04-25T10:17:38.745571+0000","last_fullsized":"2024-04-25T10:17:38.745571+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:42.325439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"195'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.672749+0000","last_change":"2024-04-25T10:17:36.643280+0000","last_active":"2024-04-25T10:17:38.672749+0000","last_peered":"2024-04-25T10:17:38.672749+0000","last_clean":"2024-04-25T10:17:38.672749+0000","last_became_active":"2024-04-25T10:17:36.642896+0000","last_became_peered":"2024-04-25T10:17:36.642896+0000","last_unstale":"2024-04-25T10:17:38.672749+0000","last_undegraded":"2024-04-25T10:17:38.672749+0000","last_fullsized":"2024-04-25T10:17:38.672749+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:13.261243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918813+0000","last_change":"2024-04-25T10:17:37.652386+0000","last_active":"2024-04-25T10:17:38.918813+0000","last_peered":"2024-04-25T10:17:38.918813+0000","last_clean":"2024-04-25T10:17:38.918813+0000","last_became_active":"2024-04-25T10:17:37.650491+0000","last_became_peered":"2024-04-25T10:17:37.650491+0000","last_unstale":"2024-04-25T10:17:38.918813+0000","last_undegraded":"2024-04-25T10:17:38.918813+0000","last_fullsized":"2024-04-25T10:17:38.918813+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:46.344344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670778+0000","last_change":"2024-04-25T10:17:36.624911+0000","last_active":"2024-04-25T10:17:38.670778+0000","last_peered":"2024-04-25T10:17:38.670778+0000","last_clean":"2024-04-25T10:17:38.670778+0000","last_became_active":"2024-04-25T10:17:36.624574+0000","last_became_peered":"2024-04-25T10:17:36.624574+0000","last_unstale":"2024-04-25T10:17:38.670778+0000","last_undegraded":"2024-04-25T10:17:38.670778+0000","last_fullsized":"2024-04-25T10:17:38.670778+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:11.231745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918669+0000","last_change":"2024-04-25T10:17:37.672523+0000","last_active":"2024-04-25T10:17:38.918669+0000","last_peered":"2024-04-25T10:17:38.918669+0000","last_clean":"2024-04-25T10:17:38.918669+0000","last_became_active":"2024-04-25T10:17:37.671909+0000","last_became_peered":"2024-04-25T10:17:37.671909+0000","last_unstale":"2024-04-25T10:17:38.918669+0000","last_undegraded":"2024-04-25T10:17:38.918669+0000","last_fullsized":"2024-04-25T10:17:38.918669+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:31.510736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671460+0000","last_change":"2024-04-25T10:17:36.647369+0000","last_active":"2024-04-25T10:17:38.671460+0000","last_peered":"2024-04-25T10:17:38.671460+0000","last_clean":"2024-04-25T10:17:38.671460+0000","last_became_active":"2024-04-25T10:17:36.646681+0000","last_became_peered":"2024-04-25T10:17:36.646681+0000","last_unstale":"2024-04-25T10:17:38.671460+0000","last_undegraded":"2024-04-25T10:17:38.671460+0000","last_fullsized":"2024-04-25T10:17:38.671460+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:21.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917000+0000","last_change":"2024-04-25T10:17:37.672342+0000","last_active":"2024-04-25T10:17:38.917000+0000","last_peered":"2024-04-25T10:17:38.917000+0000","last_clean":"2024-04-25T10:17:38.917000+0000","last_became_active":"2024-04-25T10:17:37.670575+0000","last_became_peered":"2024-04-25T10:17:37.670575+0000","last_unstale":"2024-04-25T10:17:38.917000+0000","last_undegraded":"2024-04-25T10:17:38.917000+0000","last_fullsized":"2024-04-25T10:17:38.917000+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:27.043929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.673275+0000","last_change":"2024-04-25T10:17:36.658998+0000","last_active":"2024-04-25T10:17:38.673275+0000","last_peered":"2024-04-25T10:17:38.673275+0000","last_clean":"2024-04-25T10:17:38.673275+0000","last_became_active":"2024-04-25T10:17:36.652636+0000","last_became_peered":"2024-04-25T10:17:36.652636+0000","last_unstale":"2024-04-25T10:17:38.673275+0000","last_undegraded":"2024-04-25T10:17:38.673275+0000","last_fullsized":"2024-04-25T10:17:38.673275+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.898034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917732+0000","last_change":"2024-04-25T10:17:37.692786+0000","last_active":"2024-04-25T10:17:38.917732+0000","last_peered":"2024-04-25T10:17:38.917732+0000","last_clean":"2024-04-25T10:17:38.917732+0000","last_became_active":"2024-04-25T10:17:37.692490+0000","last_became_peered":"2024-04-25T10:17:37.692490+0000","last_unstale":"2024-04-25T10:17:38.917732+0000","last_undegraded":"2024-04-25T10:17:38.917732+0000","last_fullsized":"2024-04-25T10:17:38.917732+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:42.325439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.672259+0000","last_change":"2024-04-25T10:17:36.640756+0000","last_active":"2024-04-25T10:17:38.672259+0000","last_peered":"2024-04-25T10:17:38.672259+0000","last_clean":"2024-04-25T10:17:38.672259+0000","last_became_active":"2024-04-25T10:17:36.635329+0000","last_became_peered":"2024-04-25T10:17:36.635329+0000","last_unstale":"2024-04-25T10:17:38.672259+0000","last_undegraded":"2024-04-25T10:17:38.672259+0000","last_fullsized":"2024-04-25T10:17:38.672259+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:39.396226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918280+0000","last_change":"2024-04-25T10:17:37.690468+0000","last_active":"2024-04-25T10:17:38.918280+0000","last_peered":"2024-04-25T10:17:38.918280+0000","last_clean":"2024-04-25T10:17:38.918280+0000","last_became_active":"2024-04-25T10:17:37.690161+0000","last_became_peered":"2024-04-25T10:17:37.690161+0000","last_unstale":"2024-04-25T10:17:38.918280+0000","last_undegraded":"2024-04-25T10:17:38.918280+0000","last_fullsized":"2024-04-25T10:17:38.918280+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:46.569000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.744984+0000","last_change":"2024-04-25T10:17:36.651302+0000","last_active":"2024-04-25T10:17:38.744984+0000","last_peered":"2024-04-25T10:17:38.744984+0000","last_clean":"2024-04-25T10:17:38.744984+0000","last_became_active":"2024-04-25T10:17:36.649361+0000","last_became_peered":"2024-04-25T10:17:36.649361+0000","last_unstale":"2024-04-25T10:17:38.744984+0000","last_undegraded":"2024-04-25T10:17:38.744984+0000","last_fullsized":"2024-04-25T10:17:38.744984+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:46.859015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.672726+0000","last_change":"2024-04-25T10:17:37.694511+0000","last_active":"2024-04-25T10:17:38.672726+0000","last_peered":"2024-04-25T10:17:38.672726+0000","last_clean":"2024-04-25T10:17:38.672726+0000","last_became_active":"2024-04-25T10:17:37.693817+0000","last_became_peered":"2024-04-25T10:17:37.693817+0000","last_unstale":"2024-04-25T10:17:38.672726+0000","last_undegraded":"2024-04-25T10:17:38.672726+0000","last_fullsized":"2024-04-25T10:17:38.672726+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:49.908003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670747+0000","last_change":"2024-04-25T10:17:36.646184+0000","last_active":"2024-04-25T10:17:38.670747+0000","last_peered":"2024-04-25T10:17:38.670747+0000","last_clean":"2024-04-25T10:17:38.670747+0000","last_became_active":"2024-04-25T10:17:36.645470+0000","last_became_peered":"2024-04-25T10:17:36.645470+0000","last_unstale":"2024-04-25T10:17:38.670747+0000","last_undegraded":"2024-04-25T10:17:38.670747+0000","last_fullsized":"2024-04-25T10:17:38.670747+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.459902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918115+0000","last_change":"2024-04-25T10:17:37.681037+0000","last_active":"2024-04-25T10:17:38.918115+0000","last_peered":"2024-04-25T10:17:38.918115+0000","last_clean":"2024-04-25T10:17:38.918115+0000","last_became_active":"2024-04-25T10:17:37.680748+0000","last_became_peered":"2024-04-25T10:17:37.680748+0000","last_unstale":"2024-04-25T10:17:38.918115+0000","last_undegraded":"2024-04-25T10:17:38.918115+0000","last_fullsized":"2024-04-25T10:17:38.918115+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:42.366029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745288+0000","last_change":"2024-04-25T10:17:36.634207+0000","last_active":"2024-04-25T10:17:38.745288+0000","last_peered":"2024-04-25T10:17:38.745288+0000","last_clean":"2024-04-25T10:17:38.745288+0000","last_became_active":"2024-04-25T10:17:36.633606+0000","last_became_peered":"2024-04-25T10:17:36.633606+0000","last_unstale":"2024-04-25T10:17:38.745288+0000","last_undegraded":"2024-04-25T10:17:38.745288+0000","last_fullsized":"2024-04-25T10:17:38.745288+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.691809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":498,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.744764+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:17:38.744764+0000","last_peered":"2024-04-25T10:17:38.744764+0000","last_clean":"2024-04-25T10:17:38.744764+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:17:38.744764+0000","last_undegraded":"2024-04-25T10:17:38.744764+0000","last_fullsized":"2024-04-25T10:17:38.744764+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74432,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297408,"statfs":{"total":96636764160,"available":96560545792,"internally_reserved":0,"allocated":3067904,"data_stored":1750473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73812,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298028,"statfs":{"total":96636764160,"available":96561180672,"internally_reserved":0,"allocated":2490368,"data_stored":1161171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73071046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47492,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324348,"statfs":{"total":96636764160,"available":96588132352,"internally_reserved":0,"allocated":2482176,"data_stored":1161115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96056,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275784,"statfs":{"total":96636764160,"available":96538402816,"internally_reserved":0,"allocated":2469888,"data_stored":1159523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65524,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306316,"statfs":{"total":96636764160,"available":96569667584,"internally_reserved":0,"allocated":2457600,"data_stored":1152763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51652,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320188,"statfs":{"total":96636764160,"available":96583872512,"internally_reserved":0,"allocated":2473984,"data_stored":1160049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51640,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320200,"statfs":{"total":96636764160,"available":96583884800,"internally_reserved":0,"allocated":2465792,"data_stored":1159433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575045,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74428,"kb_used_data":2988,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297412,"statfs":{"total":96636764160,"available":96560549888,"internally_reserved":0,"allocated":3059712,"data_stored":1749801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:40.663 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:40.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.815+0000 7fc78be6a640 1 Processor -- start 2024-04-25T10:17:40.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.815+0000 7fc78be6a640 1 -- start start 2024-04-25T10:17:40.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc78be6a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc78415db00 0x7fc7840b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc78be6a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7840d0bd0 0x7fc784161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc78be6a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7840ceb20 con 0x7fc7840d0770 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc78be6a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7840b7740 con 0x7fc78415db00 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc78be6a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840b7a20 con 0x7fc7840d0bd0 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc789c06640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc78415db00 0x7fc7840b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc789405640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc7840d0770 0x7fc7841614c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50014/0 (socket says 172.21.15.133:50014) 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc789405640 1 -- 172.21.15.133:0/1741283177 learned_addr learned my addr 172.21.15.133:0/1741283177 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc78a407640 1 --2- 172.21.15.133:0/1741283177 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7840d0bd0 0x7fc784161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc789c06640 1 -- 172.21.15.133:0/1741283177 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7840d0bd0 msgr2=0x7fc784161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc789c06640 1 --2- 172.21.15.133:0/1741283177 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7840d0bd0 0x7fc784161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc789c06640 1 -- 172.21.15.133:0/1741283177 >> v1:172.21.15.133:6789/0 conn(0x7fc7840d0770 legacy=0x7fc7841614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc789c06640 1 -- 172.21.15.133:0/1741283177 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7840b97f0 con 0x7fc78415db00 2024-04-25T10:17:40.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc788c04640 1 -- 172.21.15.133:0/1741283177 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473488303 0 0) 0x7fc7840ceb20 con 0x7fc7840d0770 2024-04-25T10:17:40.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc78a407640 1 --2- 172.21.15.133:0/1741283177 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7840d0bd0 0x7fc784161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc789c06640 1 --2- 172.21.15.133:0/1741283177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc78415db00 0x7fc7840b2990 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7fc7800097c0 tx=0x7fc78001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a421c6f4baa57da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc788c04640 1 -- 172.21.15.133:0/1741283177 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7800a7070 con 0x7fc78415db00 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc788c04640 1 -- 172.21.15.133:0/1741283177 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7800b0070 con 0x7fc78415db00 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.819+0000 7fc788c04640 1 -- 172.21.15.133:0/1741283177 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7800ab890 con 0x7fc78415db00 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 -- 172.21.15.133:0/1741283177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc78415db00 msgr2=0x7fc7840b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 --2- 172.21.15.133:0/1741283177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc78415db00 0x7fc7840b2990 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7fc7800097c0 tx=0x7fc78001c470 comp rx=0 tx=0).stop 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 -- 172.21.15.133:0/1741283177 shutdown_connections 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 --2- 172.21.15.133:0/1741283177 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7840d0bd0 0x7fc784161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 --2- 172.21.15.133:0/1741283177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc78415db00 0x7fc7840b2990 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 -- 172.21.15.133:0/1741283177 >> 172.21.15.133:0/1741283177 conn(0x7fc7840727c0 msgr2=0x7fc784072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 -- 172.21.15.133:0/1741283177 shutdown_connections 2024-04-25T10:17:40.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 -- 172.21.15.133:0/1741283177 wait complete. 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 Processor -- start 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 -- start start 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7840d0770 0x7fc78414f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7840d0bd0 0x7fc78415bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc78415db00 0x7fc784154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840d0fe0 con 0x7fc78415db00 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840b7430 con 0x7fc7840d0770 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840d18a0 con 0x7fc7840d0bd0 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78a407640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc78415db00 0x7fc784154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78a407640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc78415db00 0x7fc784154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46836/0 (socket says 172.21.15.133:46836) 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78a407640 1 -- 172.21.15.133:0/3079111129 learned_addr learned my addr 172.21.15.133:0/3079111129 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc789405640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7840d0bd0 0x7fc78415bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc789c06640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7840d0770 0x7fc78414f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78a407640 1 -- 172.21.15.133:0/3079111129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7840d0bd0 msgr2=0x7fc78415bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78a407640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7840d0bd0 0x7fc78415bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78a407640 1 -- 172.21.15.133:0/3079111129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7840d0770 msgr2=0x7fc78414f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78a407640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7840d0770 0x7fc78414f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78a407640 1 -- 172.21.15.133:0/3079111129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78415c5b0 con 0x7fc78415db00 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78a407640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc78415db00 0x7fc784154a90 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00cd50 tx=0x7fc77c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc77a7fc640 1 -- 172.21.15.133:0/3079111129 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00f680 con 0x7fc78415db00 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc77a7fc640 1 -- 172.21.15.133:0/3079111129 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c00f840 con 0x7fc78415db00 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 -- 172.21.15.133:0/3079111129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78415a540 con 0x7fc78415db00 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc77a7fc640 1 -- 172.21.15.133:0/3079111129 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00fc30 con 0x7fc78415db00 2024-04-25T10:17:40.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.823+0000 7fc78be6a640 1 -- 172.21.15.133:0/3079111129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78415ab00 con 0x7fc78415db00 2024-04-25T10:17:40.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.827+0000 7fc78be6a640 1 -- 172.21.15.133:0/3079111129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc784068ef0 con 0x7fc78415db00 2024-04-25T10:17:40.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.827+0000 7fc77a7fc640 1 -- 172.21.15.133:0/3079111129 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc77c0213f0 con 0x7fc78415db00 2024-04-25T10:17:40.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.827+0000 7fc77a7fc640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc768066f60 0x7fc768069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.827+0000 7fc77a7fc640 1 -- 172.21.15.133:0/3079111129 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc77c085ef0 con 0x7fc78415db00 2024-04-25T10:17:40.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.827+0000 7fc789c06640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc768066f60 0x7fc768069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.827+0000 7fc789c06640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc768066f60 0x7fc768069420 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc7800af000 tx=0x7fc7800a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.831+0000 7fc77a7fc640 1 -- 172.21.15.133:0/3079111129 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc77c014030 con 0x7fc78415db00 2024-04-25T10:17:40.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.975+0000 7fc78be6a640 1 -- 172.21.15.133:0/3079111129 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7840b2750 con 0x7fc768066f60 2024-04-25T10:17:40.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.991+0000 7fc77a7fc640 1 -- 172.21.15.133:0/3079111129 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252488 (secure 0 0 0) 0x7fc7840b2750 con 0x7fc768066f60 2024-04-25T10:17:40.996 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:40.997 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 -- 172.21.15.133:0/3079111129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc768066f60 msgr2=0x7fc768069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc768066f60 0x7fc768069420 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc7800af000 tx=0x7fc7800a7000 comp rx=0 tx=0).stop 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 -- 172.21.15.133:0/3079111129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc78415db00 msgr2=0x7fc784154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc78415db00 0x7fc784154a90 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00cd50 tx=0x7fc77c00a650 comp rx=0 tx=0).stop 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 -- 172.21.15.133:0/3079111129 shutdown_connections 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc768066f60 0x7fc768069420 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc78415db00 0x7fc784154a90 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc7840d0bd0 0x7fc78415bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 --2- 172.21.15.133:0/3079111129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc7840d0770 0x7fc78414f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 -- 172.21.15.133:0/3079111129 >> 172.21.15.133:0/3079111129 conn(0x7fc7840727c0 msgr2=0x7fc7841567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 -- 172.21.15.133:0/3079111129 shutdown_connections 2024-04-25T10:17:40.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:40.995+0000 7fc78be6a640 1 -- 172.21.15.133:0/3079111129 wait complete. 2024-04-25T10:17:41.013 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":666,"stamp":"2024-04-25T10:17:40.006515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":535036,"kb_used_data":20476,"kb_used_omap":12,"kb_used_meta":514419,"kb_avail":754439684,"statfs":{"total":773094113280,"available":772546236416,"internally_reserved":0,"allocated":20967424,"data_stored":10454328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":526765617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.397882"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745794+0000","last_change":"2024-04-25T10:17:37.664859+0000","last_active":"2024-04-25T10:17:38.745794+0000","last_peered":"2024-04-25T10:17:38.745794+0000","last_clean":"2024-04-25T10:17:38.745794+0000","last_became_active":"2024-04-25T10:17:37.664142+0000","last_became_peered":"2024-04-25T10:17:37.664142+0000","last_unstale":"2024-04-25T10:17:38.745794+0000","last_undegraded":"2024-04-25T10:17:38.745794+0000","last_fullsized":"2024-04-25T10:17:38.745794+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.967136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667707+0000","last_change":"2024-04-25T10:17:37.665180+0000","last_active":"2024-04-25T10:17:38.667707+0000","last_peered":"2024-04-25T10:17:38.667707+0000","last_clean":"2024-04-25T10:17:38.667707+0000","last_became_active":"2024-04-25T10:17:37.659367+0000","last_became_peered":"2024-04-25T10:17:37.659367+0000","last_unstale":"2024-04-25T10:17:38.667707+0000","last_undegraded":"2024-04-25T10:17:38.667707+0000","last_fullsized":"2024-04-25T10:17:38.667707+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:16.888327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917861+0000","last_change":"2024-04-25T10:17:37.673532+0000","last_active":"2024-04-25T10:17:38.917861+0000","last_peered":"2024-04-25T10:17:38.917861+0000","last_clean":"2024-04-25T10:17:38.917861+0000","last_became_active":"2024-04-25T10:17:37.671351+0000","last_became_peered":"2024-04-25T10:17:37.671351+0000","last_unstale":"2024-04-25T10:17:38.917861+0000","last_undegraded":"2024-04-25T10:17:38.917861+0000","last_fullsized":"2024-04-25T10:17:38.917861+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:34.420652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671627+0000","last_change":"2024-04-25T10:17:37.690110+0000","last_active":"2024-04-25T10:17:38.671627+0000","last_peered":"2024-04-25T10:17:38.671627+0000","last_clean":"2024-04-25T10:17:38.671627+0000","last_became_active":"2024-04-25T10:17:37.689395+0000","last_became_peered":"2024-04-25T10:17:37.689395+0000","last_unstale":"2024-04-25T10:17:38.671627+0000","last_undegraded":"2024-04-25T10:17:38.671627+0000","last_fullsized":"2024-04-25T10:17:38.671627+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:54.604229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745401+0000","last_change":"2024-04-25T10:17:37.663192+0000","last_active":"2024-04-25T10:17:38.745401+0000","last_peered":"2024-04-25T10:17:38.745401+0000","last_clean":"2024-04-25T10:17:38.745401+0000","last_became_active":"2024-04-25T10:17:37.662420+0000","last_became_peered":"2024-04-25T10:17:37.662420+0000","last_unstale":"2024-04-25T10:17:38.745401+0000","last_undegraded":"2024-04-25T10:17:38.745401+0000","last_fullsized":"2024-04-25T10:17:38.745401+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:26.638574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918292+0000","last_change":"2024-04-25T10:17:37.693412+0000","last_active":"2024-04-25T10:17:38.918292+0000","last_peered":"2024-04-25T10:17:38.918292+0000","last_clean":"2024-04-25T10:17:38.918292+0000","last_became_active":"2024-04-25T10:17:37.692250+0000","last_became_peered":"2024-04-25T10:17:37.692250+0000","last_unstale":"2024-04-25T10:17:38.918292+0000","last_undegraded":"2024-04-25T10:17:38.918292+0000","last_fullsized":"2024-04-25T10:17:38.918292+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:36.341736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917173+0000","last_change":"2024-04-25T10:17:37.685257+0000","last_active":"2024-04-25T10:17:38.917173+0000","last_peered":"2024-04-25T10:17:38.917173+0000","last_clean":"2024-04-25T10:17:38.917173+0000","last_became_active":"2024-04-25T10:17:37.684949+0000","last_became_peered":"2024-04-25T10:17:37.684949+0000","last_unstale":"2024-04-25T10:17:38.917173+0000","last_undegraded":"2024-04-25T10:17:38.917173+0000","last_fullsized":"2024-04-25T10:17:38.917173+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:16.888327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917656+0000","last_change":"2024-04-25T10:17:37.692154+0000","last_active":"2024-04-25T10:17:38.917656+0000","last_peered":"2024-04-25T10:17:38.917656+0000","last_clean":"2024-04-25T10:17:38.917656+0000","last_became_active":"2024-04-25T10:17:37.691633+0000","last_became_peered":"2024-04-25T10:17:37.691633+0000","last_unstale":"2024-04-25T10:17:38.917656+0000","last_undegraded":"2024-04-25T10:17:38.917656+0000","last_fullsized":"2024-04-25T10:17:38.917656+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:14.272525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918932+0000","last_change":"2024-04-25T10:17:37.662311+0000","last_active":"2024-04-25T10:17:38.918932+0000","last_peered":"2024-04-25T10:17:38.918932+0000","last_clean":"2024-04-25T10:17:38.918932+0000","last_became_active":"2024-04-25T10:17:37.660865+0000","last_became_peered":"2024-04-25T10:17:37.660865+0000","last_unstale":"2024-04-25T10:17:38.918932+0000","last_undegraded":"2024-04-25T10:17:38.918932+0000","last_fullsized":"2024-04-25T10:17:38.918932+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:57.680640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917600+0000","last_change":"2024-04-25T10:17:37.682006+0000","last_active":"2024-04-25T10:17:38.917600+0000","last_peered":"2024-04-25T10:17:38.917600+0000","last_clean":"2024-04-25T10:17:38.917600+0000","last_became_active":"2024-04-25T10:17:37.681625+0000","last_became_peered":"2024-04-25T10:17:37.681625+0000","last_unstale":"2024-04-25T10:17:38.917600+0000","last_undegraded":"2024-04-25T10:17:38.917600+0000","last_fullsized":"2024-04-25T10:17:38.917600+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:26.638574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671627+0000","last_change":"2024-04-25T10:17:37.683091+0000","last_active":"2024-04-25T10:17:38.671627+0000","last_peered":"2024-04-25T10:17:38.671627+0000","last_clean":"2024-04-25T10:17:38.671627+0000","last_became_active":"2024-04-25T10:17:37.682413+0000","last_became_peered":"2024-04-25T10:17:37.682413+0000","last_unstale":"2024-04-25T10:17:38.671627+0000","last_undegraded":"2024-04-25T10:17:38.671627+0000","last_fullsized":"2024-04-25T10:17:38.671627+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:29.997840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671200+0000","last_change":"2024-04-25T10:17:37.662747+0000","last_active":"2024-04-25T10:17:38.671200+0000","last_peered":"2024-04-25T10:17:38.671200+0000","last_clean":"2024-04-25T10:17:38.671200+0000","last_became_active":"2024-04-25T10:17:37.661005+0000","last_became_peered":"2024-04-25T10:17:37.661005+0000","last_unstale":"2024-04-25T10:17:38.671200+0000","last_undegraded":"2024-04-25T10:17:38.671200+0000","last_fullsized":"2024-04-25T10:17:38.671200+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:03.818015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917970+0000","last_change":"2024-04-25T10:17:37.693335+0000","last_active":"2024-04-25T10:17:38.917970+0000","last_peered":"2024-04-25T10:17:38.917970+0000","last_clean":"2024-04-25T10:17:38.917970+0000","last_became_active":"2024-04-25T10:17:37.693046+0000","last_became_peered":"2024-04-25T10:17:37.693046+0000","last_unstale":"2024-04-25T10:17:38.917970+0000","last_undegraded":"2024-04-25T10:17:38.917970+0000","last_fullsized":"2024-04-25T10:17:38.917970+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:37.854632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670346+0000","last_change":"2024-04-25T10:17:37.669119+0000","last_active":"2024-04-25T10:17:38.670346+0000","last_peered":"2024-04-25T10:17:38.670346+0000","last_clean":"2024-04-25T10:17:38.670346+0000","last_became_active":"2024-04-25T10:17:37.668207+0000","last_became_peered":"2024-04-25T10:17:37.668207+0000","last_unstale":"2024-04-25T10:17:38.670346+0000","last_undegraded":"2024-04-25T10:17:38.670346+0000","last_fullsized":"2024-04-25T10:17:38.670346+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:16.081885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670376+0000","last_change":"2024-04-25T10:17:37.667584+0000","last_active":"2024-04-25T10:17:38.670376+0000","last_peered":"2024-04-25T10:17:38.670376+0000","last_clean":"2024-04-25T10:17:38.670376+0000","last_became_active":"2024-04-25T10:17:37.667134+0000","last_became_peered":"2024-04-25T10:17:37.667134+0000","last_unstale":"2024-04-25T10:17:38.670376+0000","last_undegraded":"2024-04-25T10:17:38.670376+0000","last_fullsized":"2024-04-25T10:17:38.670376+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:20.587919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917700+0000","last_change":"2024-04-25T10:17:37.663028+0000","last_active":"2024-04-25T10:17:38.917700+0000","last_peered":"2024-04-25T10:17:38.917700+0000","last_clean":"2024-04-25T10:17:38.917700+0000","last_became_active":"2024-04-25T10:17:37.662316+0000","last_became_peered":"2024-04-25T10:17:37.662316+0000","last_unstale":"2024-04-25T10:17:38.917700+0000","last_undegraded":"2024-04-25T10:17:38.917700+0000","last_fullsized":"2024-04-25T10:17:38.917700+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:36.872185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670579+0000","last_change":"2024-04-25T10:17:37.670245+0000","last_active":"2024-04-25T10:17:38.670579+0000","last_peered":"2024-04-25T10:17:38.670579+0000","last_clean":"2024-04-25T10:17:38.670579+0000","last_became_active":"2024-04-25T10:17:37.669503+0000","last_became_peered":"2024-04-25T10:17:37.669503+0000","last_unstale":"2024-04-25T10:17:38.670579+0000","last_undegraded":"2024-04-25T10:17:38.670579+0000","last_fullsized":"2024-04-25T10:17:38.670579+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:07.412452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917833+0000","last_change":"2024-04-25T10:17:37.689729+0000","last_active":"2024-04-25T10:17:38.917833+0000","last_peered":"2024-04-25T10:17:38.917833+0000","last_clean":"2024-04-25T10:17:38.917833+0000","last_became_active":"2024-04-25T10:17:37.689380+0000","last_became_peered":"2024-04-25T10:17:37.689380+0000","last_unstale":"2024-04-25T10:17:38.917833+0000","last_undegraded":"2024-04-25T10:17:38.917833+0000","last_fullsized":"2024-04-25T10:17:38.917833+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:44.967136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745354+0000","last_change":"2024-04-25T10:17:37.672527+0000","last_active":"2024-04-25T10:17:38.745354+0000","last_peered":"2024-04-25T10:17:38.745354+0000","last_clean":"2024-04-25T10:17:38.745354+0000","last_became_active":"2024-04-25T10:17:37.670628+0000","last_became_peered":"2024-04-25T10:17:37.670628+0000","last_unstale":"2024-04-25T10:17:38.745354+0000","last_undegraded":"2024-04-25T10:17:38.745354+0000","last_fullsized":"2024-04-25T10:17:38.745354+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:00.878676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918682+0000","last_change":"2024-04-25T10:17:37.662891+0000","last_active":"2024-04-25T10:17:38.918682+0000","last_peered":"2024-04-25T10:17:38.918682+0000","last_clean":"2024-04-25T10:17:38.918682+0000","last_became_active":"2024-04-25T10:17:37.661454+0000","last_became_peered":"2024-04-25T10:17:37.661454+0000","last_unstale":"2024-04-25T10:17:38.918682+0000","last_undegraded":"2024-04-25T10:17:38.918682+0000","last_fullsized":"2024-04-25T10:17:38.918682+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:41.103605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917385+0000","last_change":"2024-04-25T10:17:37.674695+0000","last_active":"2024-04-25T10:17:38.917385+0000","last_peered":"2024-04-25T10:17:38.917385+0000","last_clean":"2024-04-25T10:17:38.917385+0000","last_became_active":"2024-04-25T10:17:37.674361+0000","last_became_peered":"2024-04-25T10:17:37.674361+0000","last_unstale":"2024-04-25T10:17:38.917385+0000","last_undegraded":"2024-04-25T10:17:38.917385+0000","last_fullsized":"2024-04-25T10:17:38.917385+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:20.188344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670446+0000","last_change":"2024-04-25T10:17:37.658044+0000","last_active":"2024-04-25T10:17:38.670446+0000","last_peered":"2024-04-25T10:17:38.670446+0000","last_clean":"2024-04-25T10:17:38.670446+0000","last_became_active":"2024-04-25T10:17:37.656211+0000","last_became_peered":"2024-04-25T10:17:37.656211+0000","last_unstale":"2024-04-25T10:17:38.670446+0000","last_undegraded":"2024-04-25T10:17:38.670446+0000","last_fullsized":"2024-04-25T10:17:38.670446+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:58.734666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918564+0000","last_change":"2024-04-25T10:17:37.662031+0000","last_active":"2024-04-25T10:17:38.918564+0000","last_peered":"2024-04-25T10:17:38.918564+0000","last_clean":"2024-04-25T10:17:38.918564+0000","last_became_active":"2024-04-25T10:17:37.661460+0000","last_became_peered":"2024-04-25T10:17:37.661460+0000","last_unstale":"2024-04-25T10:17:38.918564+0000","last_undegraded":"2024-04-25T10:17:38.918564+0000","last_fullsized":"2024-04-25T10:17:38.918564+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:03.138689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.744869+0000","last_change":"2024-04-25T10:17:37.671990+0000","last_active":"2024-04-25T10:17:38.744869+0000","last_peered":"2024-04-25T10:17:38.744869+0000","last_clean":"2024-04-25T10:17:38.744869+0000","last_became_active":"2024-04-25T10:17:37.671038+0000","last_became_peered":"2024-04-25T10:17:37.671038+0000","last_unstale":"2024-04-25T10:17:38.744869+0000","last_undegraded":"2024-04-25T10:17:38.744869+0000","last_fullsized":"2024-04-25T10:17:38.744869+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:16.888327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745815+0000","last_change":"2024-04-25T10:17:37.669069+0000","last_active":"2024-04-25T10:17:38.745815+0000","last_peered":"2024-04-25T10:17:38.745815+0000","last_clean":"2024-04-25T10:17:38.745815+0000","last_became_active":"2024-04-25T10:17:37.668324+0000","last_became_peered":"2024-04-25T10:17:37.668324+0000","last_unstale":"2024-04-25T10:17:38.745815+0000","last_undegraded":"2024-04-25T10:17:38.745815+0000","last_fullsized":"2024-04-25T10:17:38.745815+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:34.420652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918790+0000","last_change":"2024-04-25T10:17:37.673423+0000","last_active":"2024-04-25T10:17:38.918790+0000","last_peered":"2024-04-25T10:17:38.918790+0000","last_clean":"2024-04-25T10:17:38.918790+0000","last_became_active":"2024-04-25T10:17:37.671530+0000","last_became_peered":"2024-04-25T10:17:37.671530+0000","last_unstale":"2024-04-25T10:17:38.918790+0000","last_undegraded":"2024-04-25T10:17:38.918790+0000","last_fullsized":"2024-04-25T10:17:38.918790+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:28.948966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671947+0000","last_change":"2024-04-25T10:17:37.682771+0000","last_active":"2024-04-25T10:17:38.671947+0000","last_peered":"2024-04-25T10:17:38.671947+0000","last_clean":"2024-04-25T10:17:38.671947+0000","last_became_active":"2024-04-25T10:17:37.682029+0000","last_became_peered":"2024-04-25T10:17:37.682029+0000","last_unstale":"2024-04-25T10:17:38.671947+0000","last_undegraded":"2024-04-25T10:17:38.671947+0000","last_fullsized":"2024-04-25T10:17:38.671947+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:16.081885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671171+0000","last_change":"2024-04-25T10:17:37.656015+0000","last_active":"2024-04-25T10:17:38.671171+0000","last_peered":"2024-04-25T10:17:38.671171+0000","last_clean":"2024-04-25T10:17:38.671171+0000","last_became_active":"2024-04-25T10:17:37.654387+0000","last_became_peered":"2024-04-25T10:17:37.654387+0000","last_unstale":"2024-04-25T10:17:38.671171+0000","last_undegraded":"2024-04-25T10:17:38.671171+0000","last_fullsized":"2024-04-25T10:17:38.671171+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:36.695682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667655+0000","last_change":"2024-04-25T10:17:37.692954+0000","last_active":"2024-04-25T10:17:38.667655+0000","last_peered":"2024-04-25T10:17:38.667655+0000","last_clean":"2024-04-25T10:17:38.667655+0000","last_became_active":"2024-04-25T10:17:37.692584+0000","last_became_peered":"2024-04-25T10:17:37.692584+0000","last_unstale":"2024-04-25T10:17:38.667655+0000","last_undegraded":"2024-04-25T10:17:38.667655+0000","last_fullsized":"2024-04-25T10:17:38.667655+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:27.043929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918005+0000","last_change":"2024-04-25T10:17:37.662080+0000","last_active":"2024-04-25T10:17:38.918005+0000","last_peered":"2024-04-25T10:17:38.918005+0000","last_clean":"2024-04-25T10:17:38.918005+0000","last_became_active":"2024-04-25T10:17:37.661305+0000","last_became_peered":"2024-04-25T10:17:37.661305+0000","last_unstale":"2024-04-25T10:17:38.918005+0000","last_undegraded":"2024-04-25T10:17:38.918005+0000","last_fullsized":"2024-04-25T10:17:38.918005+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:22.702360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918994+0000","last_change":"2024-04-25T10:17:37.672148+0000","last_active":"2024-04-25T10:17:38.918994+0000","last_peered":"2024-04-25T10:17:38.918994+0000","last_clean":"2024-04-25T10:17:38.918994+0000","last_became_active":"2024-04-25T10:17:37.671839+0000","last_became_peered":"2024-04-25T10:17:37.671839+0000","last_unstale":"2024-04-25T10:17:38.918994+0000","last_undegraded":"2024-04-25T10:17:38.918994+0000","last_fullsized":"2024-04-25T10:17:38.918994+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:12.189982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918115+0000","last_change":"2024-04-25T10:17:37.653383+0000","last_active":"2024-04-25T10:17:38.918115+0000","last_peered":"2024-04-25T10:17:38.918115+0000","last_clean":"2024-04-25T10:17:38.918115+0000","last_became_active":"2024-04-25T10:17:37.650614+0000","last_became_peered":"2024-04-25T10:17:37.650614+0000","last_unstale":"2024-04-25T10:17:38.918115+0000","last_undegraded":"2024-04-25T10:17:38.918115+0000","last_fullsized":"2024-04-25T10:17:38.918115+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:28.337705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918283+0000","last_change":"2024-04-25T10:17:37.673070+0000","last_active":"2024-04-25T10:17:38.918283+0000","last_peered":"2024-04-25T10:17:38.918283+0000","last_clean":"2024-04-25T10:17:38.918283+0000","last_became_active":"2024-04-25T10:17:37.670871+0000","last_became_peered":"2024-04-25T10:17:37.670871+0000","last_unstale":"2024-04-25T10:17:38.918283+0000","last_undegraded":"2024-04-25T10:17:38.918283+0000","last_fullsized":"2024-04-25T10:17:38.918283+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:46.076920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"194'4","reported_seq":18,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.919114+0000","last_change":"2024-04-25T10:17:36.624285+0000","last_active":"2024-04-25T10:17:38.919114+0000","last_peered":"2024-04-25T10:17:38.919114+0000","last_clean":"2024-04-25T10:17:38.919114+0000","last_became_active":"2024-04-25T10:17:36.623990+0000","last_became_peered":"2024-04-25T10:17:36.623990+0000","last_unstale":"2024-04-25T10:17:38.919114+0000","last_undegraded":"2024-04-25T10:17:38.919114+0000","last_fullsized":"2024-04-25T10:17:38.919114+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:45.716255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917952+0000","last_change":"2024-04-25T10:17:37.670717+0000","last_active":"2024-04-25T10:17:38.917952+0000","last_peered":"2024-04-25T10:17:38.917952+0000","last_clean":"2024-04-25T10:17:38.917952+0000","last_became_active":"2024-04-25T10:17:37.667877+0000","last_became_peered":"2024-04-25T10:17:37.667877+0000","last_unstale":"2024-04-25T10:17:38.917952+0000","last_undegraded":"2024-04-25T10:17:38.917952+0000","last_fullsized":"2024-04-25T10:17:38.917952+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:15.072919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671736+0000","last_change":"2024-04-25T10:17:36.653523+0000","last_active":"2024-04-25T10:17:38.671736+0000","last_peered":"2024-04-25T10:17:38.671736+0000","last_clean":"2024-04-25T10:17:38.671736+0000","last_became_active":"2024-04-25T10:17:36.652827+0000","last_became_peered":"2024-04-25T10:17:36.652827+0000","last_unstale":"2024-04-25T10:17:38.671736+0000","last_undegraded":"2024-04-25T10:17:38.671736+0000","last_fullsized":"2024-04-25T10:17:38.671736+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:47.688629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671774+0000","last_change":"2024-04-25T10:17:37.669799+0000","last_active":"2024-04-25T10:17:38.671774+0000","last_peered":"2024-04-25T10:17:38.671774+0000","last_clean":"2024-04-25T10:17:38.671774+0000","last_became_active":"2024-04-25T10:17:37.668425+0000","last_became_peered":"2024-04-25T10:17:37.668425+0000","last_unstale":"2024-04-25T10:17:38.671774+0000","last_undegraded":"2024-04-25T10:17:38.671774+0000","last_fullsized":"2024-04-25T10:17:38.671774+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:47.207975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918532+0000","last_change":"2024-04-25T10:17:36.641983+0000","last_active":"2024-04-25T10:17:38.918532+0000","last_peered":"2024-04-25T10:17:38.918532+0000","last_clean":"2024-04-25T10:17:38.918532+0000","last_became_active":"2024-04-25T10:17:36.641080+0000","last_became_peered":"2024-04-25T10:17:36.641080+0000","last_unstale":"2024-04-25T10:17:38.918532+0000","last_undegraded":"2024-04-25T10:17:38.918532+0000","last_fullsized":"2024-04-25T10:17:38.918532+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:10.900721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.916993+0000","last_change":"2024-04-25T10:17:37.679299+0000","last_active":"2024-04-25T10:17:38.916993+0000","last_peered":"2024-04-25T10:17:38.916993+0000","last_clean":"2024-04-25T10:17:38.916993+0000","last_became_active":"2024-04-25T10:17:37.679042+0000","last_became_peered":"2024-04-25T10:17:37.679042+0000","last_unstale":"2024-04-25T10:17:38.916993+0000","last_undegraded":"2024-04-25T10:17:38.916993+0000","last_fullsized":"2024-04-25T10:17:38.916993+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:36.330126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670469+0000","last_change":"2024-04-25T10:17:36.659961+0000","last_active":"2024-04-25T10:17:38.670469+0000","last_peered":"2024-04-25T10:17:38.670469+0000","last_clean":"2024-04-25T10:17:38.670469+0000","last_became_active":"2024-04-25T10:17:36.659723+0000","last_became_peered":"2024-04-25T10:17:36.659723+0000","last_unstale":"2024-04-25T10:17:38.670469+0000","last_undegraded":"2024-04-25T10:17:38.670469+0000","last_fullsized":"2024-04-25T10:17:38.670469+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:35.860903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.669921+0000","last_change":"2024-04-25T10:17:37.654467+0000","last_active":"2024-04-25T10:17:38.669921+0000","last_peered":"2024-04-25T10:17:38.669921+0000","last_clean":"2024-04-25T10:17:38.669921+0000","last_became_active":"2024-04-25T10:17:37.653714+0000","last_became_peered":"2024-04-25T10:17:37.653714+0000","last_unstale":"2024-04-25T10:17:38.669921+0000","last_undegraded":"2024-04-25T10:17:38.669921+0000","last_fullsized":"2024-04-25T10:17:38.669921+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:54.998816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745172+0000","last_change":"2024-04-25T10:17:36.661718+0000","last_active":"2024-04-25T10:17:38.745172+0000","last_peered":"2024-04-25T10:17:38.745172+0000","last_clean":"2024-04-25T10:17:38.745172+0000","last_became_active":"2024-04-25T10:17:36.661093+0000","last_became_peered":"2024-04-25T10:17:36.661093+0000","last_unstale":"2024-04-25T10:17:38.745172+0000","last_undegraded":"2024-04-25T10:17:38.745172+0000","last_fullsized":"2024-04-25T10:17:38.745172+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:39.439515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670400+0000","last_change":"2024-04-25T10:17:37.658750+0000","last_active":"2024-04-25T10:17:38.670400+0000","last_peered":"2024-04-25T10:17:38.670400+0000","last_clean":"2024-04-25T10:17:38.670400+0000","last_became_active":"2024-04-25T10:17:37.657273+0000","last_became_peered":"2024-04-25T10:17:37.657273+0000","last_unstale":"2024-04-25T10:17:38.670400+0000","last_undegraded":"2024-04-25T10:17:38.670400+0000","last_fullsized":"2024-04-25T10:17:38.670400+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:17.159225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.669478+0000","last_change":"2024-04-25T10:17:36.656913+0000","last_active":"2024-04-25T10:17:38.669478+0000","last_peered":"2024-04-25T10:17:38.669478+0000","last_clean":"2024-04-25T10:17:38.669478+0000","last_became_active":"2024-04-25T10:17:36.656237+0000","last_became_peered":"2024-04-25T10:17:36.656237+0000","last_unstale":"2024-04-25T10:17:38.669478+0000","last_undegraded":"2024-04-25T10:17:38.669478+0000","last_fullsized":"2024-04-25T10:17:38.669478+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:29.361785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918988+0000","last_change":"2024-04-25T10:17:37.680702+0000","last_active":"2024-04-25T10:17:38.918988+0000","last_peered":"2024-04-25T10:17:38.918988+0000","last_clean":"2024-04-25T10:17:38.918988+0000","last_became_active":"2024-04-25T10:17:37.680420+0000","last_became_peered":"2024-04-25T10:17:37.680420+0000","last_unstale":"2024-04-25T10:17:38.918988+0000","last_undegraded":"2024-04-25T10:17:38.918988+0000","last_fullsized":"2024-04-25T10:17:38.918988+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:49.405793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917260+0000","last_change":"2024-04-25T10:17:36.627592+0000","last_active":"2024-04-25T10:17:38.917260+0000","last_peered":"2024-04-25T10:17:38.917260+0000","last_clean":"2024-04-25T10:17:38.917260+0000","last_became_active":"2024-04-25T10:17:36.627144+0000","last_became_peered":"2024-04-25T10:17:36.627144+0000","last_unstale":"2024-04-25T10:17:38.917260+0000","last_undegraded":"2024-04-25T10:17:38.917260+0000","last_fullsized":"2024-04-25T10:17:38.917260+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:55.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670650+0000","last_change":"2024-04-25T10:17:37.682060+0000","last_active":"2024-04-25T10:17:38.670650+0000","last_peered":"2024-04-25T10:17:38.670650+0000","last_clean":"2024-04-25T10:17:38.670650+0000","last_became_active":"2024-04-25T10:17:37.681777+0000","last_became_peered":"2024-04-25T10:17:37.681777+0000","last_unstale":"2024-04-25T10:17:38.670650+0000","last_undegraded":"2024-04-25T10:17:38.670650+0000","last_fullsized":"2024-04-25T10:17:38.670650+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:44.379669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.746087+0000","last_change":"2024-04-25T10:17:36.650543+0000","last_active":"2024-04-25T10:17:38.746087+0000","last_peered":"2024-04-25T10:17:38.746087+0000","last_clean":"2024-04-25T10:17:38.746087+0000","last_became_active":"2024-04-25T10:17:36.649524+0000","last_became_peered":"2024-04-25T10:17:36.649524+0000","last_unstale":"2024-04-25T10:17:38.746087+0000","last_undegraded":"2024-04-25T10:17:38.746087+0000","last_fullsized":"2024-04-25T10:17:38.746087+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.385986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670781+0000","last_change":"2024-04-25T10:17:37.656908+0000","last_active":"2024-04-25T10:17:38.670781+0000","last_peered":"2024-04-25T10:17:38.670781+0000","last_clean":"2024-04-25T10:17:38.670781+0000","last_became_active":"2024-04-25T10:17:37.656144+0000","last_became_peered":"2024-04-25T10:17:37.656144+0000","last_unstale":"2024-04-25T10:17:38.670781+0000","last_undegraded":"2024-04-25T10:17:38.670781+0000","last_fullsized":"2024-04-25T10:17:38.670781+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:09.345050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.673040+0000","last_change":"2024-04-25T10:17:36.656914+0000","last_active":"2024-04-25T10:17:38.673040+0000","last_peered":"2024-04-25T10:17:38.673040+0000","last_clean":"2024-04-25T10:17:38.673040+0000","last_became_active":"2024-04-25T10:17:36.656348+0000","last_became_peered":"2024-04-25T10:17:36.656348+0000","last_unstale":"2024-04-25T10:17:38.673040+0000","last_undegraded":"2024-04-25T10:17:38.673040+0000","last_fullsized":"2024-04-25T10:17:38.673040+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:01.197915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918451+0000","last_change":"2024-04-25T10:17:37.673148+0000","last_active":"2024-04-25T10:17:38.918451+0000","last_peered":"2024-04-25T10:17:38.918451+0000","last_clean":"2024-04-25T10:17:38.918451+0000","last_became_active":"2024-04-25T10:17:37.671219+0000","last_became_peered":"2024-04-25T10:17:37.671219+0000","last_unstale":"2024-04-25T10:17:38.918451+0000","last_undegraded":"2024-04-25T10:17:38.918451+0000","last_fullsized":"2024-04-25T10:17:38.918451+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:53.995344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"194'3","reported_seq":17,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745574+0000","last_change":"2024-04-25T10:17:36.645043+0000","last_active":"2024-04-25T10:17:38.745574+0000","last_peered":"2024-04-25T10:17:38.745574+0000","last_clean":"2024-04-25T10:17:38.745574+0000","last_became_active":"2024-04-25T10:17:36.644445+0000","last_became_peered":"2024-04-25T10:17:36.644445+0000","last_unstale":"2024-04-25T10:17:38.745574+0000","last_undegraded":"2024-04-25T10:17:38.745574+0000","last_fullsized":"2024-04-25T10:17:38.745574+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:46.839439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.672948+0000","last_change":"2024-04-25T10:17:37.682345+0000","last_active":"2024-04-25T10:17:38.672948+0000","last_peered":"2024-04-25T10:17:38.672948+0000","last_clean":"2024-04-25T10:17:38.672948+0000","last_became_active":"2024-04-25T10:17:37.682073+0000","last_became_peered":"2024-04-25T10:17:37.682073+0000","last_unstale":"2024-04-25T10:17:38.672948+0000","last_undegraded":"2024-04-25T10:17:38.672948+0000","last_fullsized":"2024-04-25T10:17:38.672948+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:07.412452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"194'6","reported_seq":20,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918845+0000","last_change":"2024-04-25T10:17:36.641772+0000","last_active":"2024-04-25T10:17:38.918845+0000","last_peered":"2024-04-25T10:17:38.918845+0000","last_clean":"2024-04-25T10:17:38.918845+0000","last_became_active":"2024-04-25T10:17:36.641102+0000","last_became_peered":"2024-04-25T10:17:36.641102+0000","last_unstale":"2024-04-25T10:17:38.918845+0000","last_undegraded":"2024-04-25T10:17:38.918845+0000","last_fullsized":"2024-04-25T10:17:38.918845+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:44.673880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667278+0000","last_change":"2024-04-25T10:17:37.689343+0000","last_active":"2024-04-25T10:17:38.667278+0000","last_peered":"2024-04-25T10:17:38.667278+0000","last_clean":"2024-04-25T10:17:38.667278+0000","last_became_active":"2024-04-25T10:17:37.689081+0000","last_became_peered":"2024-04-25T10:17:37.689081+0000","last_unstale":"2024-04-25T10:17:38.667278+0000","last_undegraded":"2024-04-25T10:17:38.667278+0000","last_fullsized":"2024-04-25T10:17:38.667278+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:12.382293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.672951+0000","last_change":"2024-04-25T10:17:36.659027+0000","last_active":"2024-04-25T10:17:38.672951+0000","last_peered":"2024-04-25T10:17:38.672951+0000","last_clean":"2024-04-25T10:17:38.672951+0000","last_became_active":"2024-04-25T10:17:36.654181+0000","last_became_peered":"2024-04-25T10:17:36.654181+0000","last_unstale":"2024-04-25T10:17:38.672951+0000","last_undegraded":"2024-04-25T10:17:38.672951+0000","last_fullsized":"2024-04-25T10:17:38.672951+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:34.681416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670337+0000","last_change":"2024-04-25T10:17:37.652524+0000","last_active":"2024-04-25T10:17:38.670337+0000","last_peered":"2024-04-25T10:17:38.670337+0000","last_clean":"2024-04-25T10:17:38.670337+0000","last_became_active":"2024-04-25T10:17:37.651692+0000","last_became_peered":"2024-04-25T10:17:37.651692+0000","last_unstale":"2024-04-25T10:17:38.670337+0000","last_undegraded":"2024-04-25T10:17:38.670337+0000","last_fullsized":"2024-04-25T10:17:38.670337+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:29.997840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667511+0000","last_change":"2024-04-25T10:17:36.623640+0000","last_active":"2024-04-25T10:17:38.667511+0000","last_peered":"2024-04-25T10:17:38.667511+0000","last_clean":"2024-04-25T10:17:38.667511+0000","last_became_active":"2024-04-25T10:17:36.622668+0000","last_became_peered":"2024-04-25T10:17:36.622668+0000","last_unstale":"2024-04-25T10:17:38.667511+0000","last_undegraded":"2024-04-25T10:17:38.667511+0000","last_fullsized":"2024-04-25T10:17:38.667511+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:46.839439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.916962+0000","last_change":"2024-04-25T10:17:37.667495+0000","last_active":"2024-04-25T10:17:38.916962+0000","last_peered":"2024-04-25T10:17:38.916962+0000","last_clean":"2024-04-25T10:17:38.916962+0000","last_became_active":"2024-04-25T10:17:37.667165+0000","last_became_peered":"2024-04-25T10:17:37.667165+0000","last_unstale":"2024-04-25T10:17:38.916962+0000","last_undegraded":"2024-04-25T10:17:38.916962+0000","last_fullsized":"2024-04-25T10:17:38.916962+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:12.382293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917573+0000","last_change":"2024-04-25T10:17:36.623729+0000","last_active":"2024-04-25T10:17:38.917573+0000","last_peered":"2024-04-25T10:17:38.917573+0000","last_clean":"2024-04-25T10:17:38.917573+0000","last_became_active":"2024-04-25T10:17:36.623001+0000","last_became_peered":"2024-04-25T10:17:36.623001+0000","last_unstale":"2024-04-25T10:17:38.917573+0000","last_undegraded":"2024-04-25T10:17:38.917573+0000","last_fullsized":"2024-04-25T10:17:38.917573+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:18.305901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670446+0000","last_change":"2024-04-25T10:17:37.657805+0000","last_active":"2024-04-25T10:17:38.670446+0000","last_peered":"2024-04-25T10:17:38.670446+0000","last_clean":"2024-04-25T10:17:38.670446+0000","last_became_active":"2024-04-25T10:17:37.656981+0000","last_became_peered":"2024-04-25T10:17:37.656981+0000","last_unstale":"2024-04-25T10:17:38.670446+0000","last_undegraded":"2024-04-25T10:17:38.670446+0000","last_fullsized":"2024-04-25T10:17:38.670446+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:20.686031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"194'8","reported_seq":22,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670497+0000","last_change":"2024-04-25T10:17:36.646281+0000","last_active":"2024-04-25T10:17:38.670497+0000","last_peered":"2024-04-25T10:17:38.670497+0000","last_clean":"2024-04-25T10:17:38.670497+0000","last_became_active":"2024-04-25T10:17:36.645591+0000","last_became_peered":"2024-04-25T10:17:36.645591+0000","last_unstale":"2024-04-25T10:17:38.670497+0000","last_undegraded":"2024-04-25T10:17:38.670497+0000","last_fullsized":"2024-04-25T10:17:38.670497+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:14.560561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667745+0000","last_change":"2024-04-25T10:17:37.660546+0000","last_active":"2024-04-25T10:17:38.667745+0000","last_peered":"2024-04-25T10:17:38.667745+0000","last_clean":"2024-04-25T10:17:38.667745+0000","last_became_active":"2024-04-25T10:17:37.659700+0000","last_became_peered":"2024-04-25T10:17:37.659700+0000","last_unstale":"2024-04-25T10:17:38.667745+0000","last_undegraded":"2024-04-25T10:17:38.667745+0000","last_fullsized":"2024-04-25T10:17:38.667745+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:46.076920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918807+0000","last_change":"2024-04-25T10:17:36.630837+0000","last_active":"2024-04-25T10:17:38.918807+0000","last_peered":"2024-04-25T10:17:38.918807+0000","last_clean":"2024-04-25T10:17:38.918807+0000","last_became_active":"2024-04-25T10:17:36.630509+0000","last_became_peered":"2024-04-25T10:17:36.630509+0000","last_unstale":"2024-04-25T10:17:38.918807+0000","last_undegraded":"2024-04-25T10:17:38.918807+0000","last_fullsized":"2024-04-25T10:17:38.918807+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:57.918425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745324+0000","last_change":"2024-04-25T10:17:37.690908+0000","last_active":"2024-04-25T10:17:38.745324+0000","last_peered":"2024-04-25T10:17:38.745324+0000","last_clean":"2024-04-25T10:17:38.745324+0000","last_became_active":"2024-04-25T10:17:37.690613+0000","last_became_peered":"2024-04-25T10:17:37.690613+0000","last_unstale":"2024-04-25T10:17:38.745324+0000","last_undegraded":"2024-04-25T10:17:38.745324+0000","last_fullsized":"2024-04-25T10:17:38.745324+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:20.188344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918324+0000","last_change":"2024-04-25T10:17:36.621384+0000","last_active":"2024-04-25T10:17:38.918324+0000","last_peered":"2024-04-25T10:17:38.918324+0000","last_clean":"2024-04-25T10:17:38.918324+0000","last_became_active":"2024-04-25T10:17:36.621009+0000","last_became_peered":"2024-04-25T10:17:36.621009+0000","last_unstale":"2024-04-25T10:17:38.918324+0000","last_undegraded":"2024-04-25T10:17:38.918324+0000","last_fullsized":"2024-04-25T10:17:38.918324+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:38.254914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918694+0000","last_change":"2024-04-25T10:17:37.673461+0000","last_active":"2024-04-25T10:17:38.918694+0000","last_peered":"2024-04-25T10:17:38.918694+0000","last_clean":"2024-04-25T10:17:38.918694+0000","last_became_active":"2024-04-25T10:17:37.673186+0000","last_became_peered":"2024-04-25T10:17:37.673186+0000","last_unstale":"2024-04-25T10:17:38.918694+0000","last_undegraded":"2024-04-25T10:17:38.918694+0000","last_fullsized":"2024-04-25T10:17:38.918694+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:37.405035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670021+0000","last_change":"2024-04-25T10:17:36.639127+0000","last_active":"2024-04-25T10:17:38.670021+0000","last_peered":"2024-04-25T10:17:38.670021+0000","last_clean":"2024-04-25T10:17:38.670021+0000","last_became_active":"2024-04-25T10:17:36.638346+0000","last_became_peered":"2024-04-25T10:17:36.638346+0000","last_unstale":"2024-04-25T10:17:38.670021+0000","last_undegraded":"2024-04-25T10:17:38.670021+0000","last_fullsized":"2024-04-25T10:17:38.670021+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:55.489855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671119+0000","last_change":"2024-04-25T10:17:37.691777+0000","last_active":"2024-04-25T10:17:38.671119+0000","last_peered":"2024-04-25T10:17:38.671119+0000","last_clean":"2024-04-25T10:17:38.671119+0000","last_became_active":"2024-04-25T10:17:37.691377+0000","last_became_peered":"2024-04-25T10:17:37.691377+0000","last_unstale":"2024-04-25T10:17:38.671119+0000","last_undegraded":"2024-04-25T10:17:38.671119+0000","last_fullsized":"2024-04-25T10:17:38.671119+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:05.590252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671346+0000","last_change":"2024-04-25T10:17:36.639269+0000","last_active":"2024-04-25T10:17:38.671346+0000","last_peered":"2024-04-25T10:17:38.671346+0000","last_clean":"2024-04-25T10:17:38.671346+0000","last_became_active":"2024-04-25T10:17:36.638503+0000","last_became_peered":"2024-04-25T10:17:36.638503+0000","last_unstale":"2024-04-25T10:17:38.671346+0000","last_undegraded":"2024-04-25T10:17:38.671346+0000","last_fullsized":"2024-04-25T10:17:38.671346+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:06.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670202+0000","last_change":"2024-04-25T10:17:37.669336+0000","last_active":"2024-04-25T10:17:38.670202+0000","last_peered":"2024-04-25T10:17:38.670202+0000","last_clean":"2024-04-25T10:17:38.670202+0000","last_became_active":"2024-04-25T10:17:37.668537+0000","last_became_peered":"2024-04-25T10:17:37.668537+0000","last_unstale":"2024-04-25T10:17:38.670202+0000","last_undegraded":"2024-04-25T10:17:38.670202+0000","last_fullsized":"2024-04-25T10:17:38.670202+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:54.604229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.673129+0000","last_change":"2024-04-25T10:17:36.655379+0000","last_active":"2024-04-25T10:17:38.673129+0000","last_peered":"2024-04-25T10:17:38.673129+0000","last_clean":"2024-04-25T10:17:38.673129+0000","last_became_active":"2024-04-25T10:17:36.646567+0000","last_became_peered":"2024-04-25T10:17:36.646567+0000","last_unstale":"2024-04-25T10:17:38.673129+0000","last_undegraded":"2024-04-25T10:17:38.673129+0000","last_fullsized":"2024-04-25T10:17:38.673129+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:29.721104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745016+0000","last_change":"2024-04-25T10:17:37.664398+0000","last_active":"2024-04-25T10:17:38.745016+0000","last_peered":"2024-04-25T10:17:38.745016+0000","last_clean":"2024-04-25T10:17:38.745016+0000","last_became_active":"2024-04-25T10:17:37.663672+0000","last_became_peered":"2024-04-25T10:17:37.663672+0000","last_unstale":"2024-04-25T10:17:38.745016+0000","last_undegraded":"2024-04-25T10:17:38.745016+0000","last_fullsized":"2024-04-25T10:17:38.745016+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:46.076920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670511+0000","last_change":"2024-04-25T10:17:36.650440+0000","last_active":"2024-04-25T10:17:38.670511+0000","last_peered":"2024-04-25T10:17:38.670511+0000","last_clean":"2024-04-25T10:17:38.670511+0000","last_became_active":"2024-04-25T10:17:36.650123+0000","last_became_peered":"2024-04-25T10:17:36.650123+0000","last_unstale":"2024-04-25T10:17:38.670511+0000","last_undegraded":"2024-04-25T10:17:38.670511+0000","last_fullsized":"2024-04-25T10:17:38.670511+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:36.843350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667359+0000","last_change":"2024-04-25T10:17:37.654496+0000","last_active":"2024-04-25T10:17:38.667359+0000","last_peered":"2024-04-25T10:17:38.667359+0000","last_clean":"2024-04-25T10:17:38.667359+0000","last_became_active":"2024-04-25T10:17:37.653801+0000","last_became_peered":"2024-04-25T10:17:37.653801+0000","last_unstale":"2024-04-25T10:17:38.667359+0000","last_undegraded":"2024-04-25T10:17:38.667359+0000","last_fullsized":"2024-04-25T10:17:38.667359+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:36.330126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670379+0000","last_change":"2024-04-25T10:17:36.647950+0000","last_active":"2024-04-25T10:17:38.670379+0000","last_peered":"2024-04-25T10:17:38.670379+0000","last_clean":"2024-04-25T10:17:38.670379+0000","last_became_active":"2024-04-25T10:17:36.647739+0000","last_became_peered":"2024-04-25T10:17:36.647739+0000","last_unstale":"2024-04-25T10:17:38.670379+0000","last_undegraded":"2024-04-25T10:17:38.670379+0000","last_fullsized":"2024-04-25T10:17:38.670379+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:32.050946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917401+0000","last_change":"2024-04-25T10:17:37.681361+0000","last_active":"2024-04-25T10:17:38.917401+0000","last_peered":"2024-04-25T10:17:38.917401+0000","last_clean":"2024-04-25T10:17:38.917401+0000","last_became_active":"2024-04-25T10:17:37.681034+0000","last_became_peered":"2024-04-25T10:17:37.681034+0000","last_unstale":"2024-04-25T10:17:38.917401+0000","last_undegraded":"2024-04-25T10:17:38.917401+0000","last_fullsized":"2024-04-25T10:17:38.917401+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:00.878676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917687+0000","last_change":"2024-04-25T10:17:36.621218+0000","last_active":"2024-04-25T10:17:38.917687+0000","last_peered":"2024-04-25T10:17:38.917687+0000","last_clean":"2024-04-25T10:17:38.917687+0000","last_became_active":"2024-04-25T10:17:36.620883+0000","last_became_peered":"2024-04-25T10:17:36.620883+0000","last_unstale":"2024-04-25T10:17:38.917687+0000","last_undegraded":"2024-04-25T10:17:38.917687+0000","last_fullsized":"2024-04-25T10:17:38.917687+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:18.426640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.667909+0000","last_change":"2024-04-25T10:17:37.664826+0000","last_active":"2024-04-25T10:17:38.667909+0000","last_peered":"2024-04-25T10:17:38.667909+0000","last_clean":"2024-04-25T10:17:38.667909+0000","last_became_active":"2024-04-25T10:17:37.664487+0000","last_became_peered":"2024-04-25T10:17:37.664487+0000","last_unstale":"2024-04-25T10:17:38.667909+0000","last_undegraded":"2024-04-25T10:17:38.667909+0000","last_fullsized":"2024-04-25T10:17:38.667909+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:20.188344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671410+0000","last_change":"2024-04-25T10:17:36.660226+0000","last_active":"2024-04-25T10:17:38.671410+0000","last_peered":"2024-04-25T10:17:38.671410+0000","last_clean":"2024-04-25T10:17:38.671410+0000","last_became_active":"2024-04-25T10:17:36.659670+0000","last_became_peered":"2024-04-25T10:17:36.659670+0000","last_unstale":"2024-04-25T10:17:38.671410+0000","last_undegraded":"2024-04-25T10:17:38.671410+0000","last_fullsized":"2024-04-25T10:17:38.671410+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:21.940556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745571+0000","last_change":"2024-04-25T10:17:37.685235+0000","last_active":"2024-04-25T10:17:38.745571+0000","last_peered":"2024-04-25T10:17:38.745571+0000","last_clean":"2024-04-25T10:17:38.745571+0000","last_became_active":"2024-04-25T10:17:37.684577+0000","last_became_peered":"2024-04-25T10:17:37.684577+0000","last_unstale":"2024-04-25T10:17:38.745571+0000","last_undegraded":"2024-04-25T10:17:38.745571+0000","last_fullsized":"2024-04-25T10:17:38.745571+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:42.325439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"195'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.672749+0000","last_change":"2024-04-25T10:17:36.643280+0000","last_active":"2024-04-25T10:17:38.672749+0000","last_peered":"2024-04-25T10:17:38.672749+0000","last_clean":"2024-04-25T10:17:38.672749+0000","last_became_active":"2024-04-25T10:17:36.642896+0000","last_became_peered":"2024-04-25T10:17:36.642896+0000","last_unstale":"2024-04-25T10:17:38.672749+0000","last_undegraded":"2024-04-25T10:17:38.672749+0000","last_fullsized":"2024-04-25T10:17:38.672749+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:13.261243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918813+0000","last_change":"2024-04-25T10:17:37.652386+0000","last_active":"2024-04-25T10:17:38.918813+0000","last_peered":"2024-04-25T10:17:38.918813+0000","last_clean":"2024-04-25T10:17:38.918813+0000","last_became_active":"2024-04-25T10:17:37.650491+0000","last_became_peered":"2024-04-25T10:17:37.650491+0000","last_unstale":"2024-04-25T10:17:38.918813+0000","last_undegraded":"2024-04-25T10:17:38.918813+0000","last_fullsized":"2024-04-25T10:17:38.918813+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:46.344344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670778+0000","last_change":"2024-04-25T10:17:36.624911+0000","last_active":"2024-04-25T10:17:38.670778+0000","last_peered":"2024-04-25T10:17:38.670778+0000","last_clean":"2024-04-25T10:17:38.670778+0000","last_became_active":"2024-04-25T10:17:36.624574+0000","last_became_peered":"2024-04-25T10:17:36.624574+0000","last_unstale":"2024-04-25T10:17:38.670778+0000","last_undegraded":"2024-04-25T10:17:38.670778+0000","last_fullsized":"2024-04-25T10:17:38.670778+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:11.231745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918669+0000","last_change":"2024-04-25T10:17:37.672523+0000","last_active":"2024-04-25T10:17:38.918669+0000","last_peered":"2024-04-25T10:17:38.918669+0000","last_clean":"2024-04-25T10:17:38.918669+0000","last_became_active":"2024-04-25T10:17:37.671909+0000","last_became_peered":"2024-04-25T10:17:37.671909+0000","last_unstale":"2024-04-25T10:17:38.918669+0000","last_undegraded":"2024-04-25T10:17:38.918669+0000","last_fullsized":"2024-04-25T10:17:38.918669+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:31.510736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.671460+0000","last_change":"2024-04-25T10:17:36.647369+0000","last_active":"2024-04-25T10:17:38.671460+0000","last_peered":"2024-04-25T10:17:38.671460+0000","last_clean":"2024-04-25T10:17:38.671460+0000","last_became_active":"2024-04-25T10:17:36.646681+0000","last_became_peered":"2024-04-25T10:17:36.646681+0000","last_unstale":"2024-04-25T10:17:38.671460+0000","last_undegraded":"2024-04-25T10:17:38.671460+0000","last_fullsized":"2024-04-25T10:17:38.671460+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:21.316065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917000+0000","last_change":"2024-04-25T10:17:37.672342+0000","last_active":"2024-04-25T10:17:38.917000+0000","last_peered":"2024-04-25T10:17:38.917000+0000","last_clean":"2024-04-25T10:17:38.917000+0000","last_became_active":"2024-04-25T10:17:37.670575+0000","last_became_peered":"2024-04-25T10:17:37.670575+0000","last_unstale":"2024-04-25T10:17:38.917000+0000","last_undegraded":"2024-04-25T10:17:38.917000+0000","last_fullsized":"2024-04-25T10:17:38.917000+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:27.043929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.673275+0000","last_change":"2024-04-25T10:17:36.658998+0000","last_active":"2024-04-25T10:17:38.673275+0000","last_peered":"2024-04-25T10:17:38.673275+0000","last_clean":"2024-04-25T10:17:38.673275+0000","last_became_active":"2024-04-25T10:17:36.652636+0000","last_became_peered":"2024-04-25T10:17:36.652636+0000","last_unstale":"2024-04-25T10:17:38.673275+0000","last_undegraded":"2024-04-25T10:17:38.673275+0000","last_fullsized":"2024-04-25T10:17:38.673275+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:13.898034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.917732+0000","last_change":"2024-04-25T10:17:37.692786+0000","last_active":"2024-04-25T10:17:38.917732+0000","last_peered":"2024-04-25T10:17:38.917732+0000","last_clean":"2024-04-25T10:17:38.917732+0000","last_became_active":"2024-04-25T10:17:37.692490+0000","last_became_peered":"2024-04-25T10:17:37.692490+0000","last_unstale":"2024-04-25T10:17:38.917732+0000","last_undegraded":"2024-04-25T10:17:38.917732+0000","last_fullsized":"2024-04-25T10:17:38.917732+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:42.325439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.672259+0000","last_change":"2024-04-25T10:17:36.640756+0000","last_active":"2024-04-25T10:17:38.672259+0000","last_peered":"2024-04-25T10:17:38.672259+0000","last_clean":"2024-04-25T10:17:38.672259+0000","last_became_active":"2024-04-25T10:17:36.635329+0000","last_became_peered":"2024-04-25T10:17:36.635329+0000","last_unstale":"2024-04-25T10:17:38.672259+0000","last_undegraded":"2024-04-25T10:17:38.672259+0000","last_fullsized":"2024-04-25T10:17:38.672259+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:39.396226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918280+0000","last_change":"2024-04-25T10:17:37.690468+0000","last_active":"2024-04-25T10:17:38.918280+0000","last_peered":"2024-04-25T10:17:38.918280+0000","last_clean":"2024-04-25T10:17:38.918280+0000","last_became_active":"2024-04-25T10:17:37.690161+0000","last_became_peered":"2024-04-25T10:17:37.690161+0000","last_unstale":"2024-04-25T10:17:38.918280+0000","last_undegraded":"2024-04-25T10:17:38.918280+0000","last_fullsized":"2024-04-25T10:17:38.918280+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:46.569000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.744984+0000","last_change":"2024-04-25T10:17:36.651302+0000","last_active":"2024-04-25T10:17:38.744984+0000","last_peered":"2024-04-25T10:17:38.744984+0000","last_clean":"2024-04-25T10:17:38.744984+0000","last_became_active":"2024-04-25T10:17:36.649361+0000","last_became_peered":"2024-04-25T10:17:36.649361+0000","last_unstale":"2024-04-25T10:17:38.744984+0000","last_undegraded":"2024-04-25T10:17:38.744984+0000","last_fullsized":"2024-04-25T10:17:38.744984+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:46.859015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.672726+0000","last_change":"2024-04-25T10:17:37.694511+0000","last_active":"2024-04-25T10:17:38.672726+0000","last_peered":"2024-04-25T10:17:38.672726+0000","last_clean":"2024-04-25T10:17:38.672726+0000","last_became_active":"2024-04-25T10:17:37.693817+0000","last_became_peered":"2024-04-25T10:17:37.693817+0000","last_unstale":"2024-04-25T10:17:38.672726+0000","last_undegraded":"2024-04-25T10:17:38.672726+0000","last_fullsized":"2024-04-25T10:17:38.672726+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:49.908003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.670747+0000","last_change":"2024-04-25T10:17:36.646184+0000","last_active":"2024-04-25T10:17:38.670747+0000","last_peered":"2024-04-25T10:17:38.670747+0000","last_clean":"2024-04-25T10:17:38.670747+0000","last_became_active":"2024-04-25T10:17:36.645470+0000","last_became_peered":"2024-04-25T10:17:36.645470+0000","last_unstale":"2024-04-25T10:17:38.670747+0000","last_undegraded":"2024-04-25T10:17:38.670747+0000","last_fullsized":"2024-04-25T10:17:38.670747+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.459902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.918115+0000","last_change":"2024-04-25T10:17:37.681037+0000","last_active":"2024-04-25T10:17:38.918115+0000","last_peered":"2024-04-25T10:17:38.918115+0000","last_clean":"2024-04-25T10:17:38.918115+0000","last_became_active":"2024-04-25T10:17:37.680748+0000","last_became_peered":"2024-04-25T10:17:37.680748+0000","last_unstale":"2024-04-25T10:17:38.918115+0000","last_undegraded":"2024-04-25T10:17:38.918115+0000","last_fullsized":"2024-04-25T10:17:38.918115+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.597571+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.597571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:42.366029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.745288+0000","last_change":"2024-04-25T10:17:36.634207+0000","last_active":"2024-04-25T10:17:38.745288+0000","last_peered":"2024-04-25T10:17:38.745288+0000","last_clean":"2024-04-25T10:17:38.745288+0000","last_became_active":"2024-04-25T10:17:36.633606+0000","last_became_peered":"2024-04-25T10:17:36.633606+0000","last_unstale":"2024-04-25T10:17:38.745288+0000","last_undegraded":"2024-04-25T10:17:38.745288+0000","last_fullsized":"2024-04-25T10:17:38.745288+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:35.586289+0000","last_clean_scrub_stamp":"2024-04-25T10:17:35.586289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.691809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":498,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:17:38.744764+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:17:38.744764+0000","last_peered":"2024-04-25T10:17:38.744764+0000","last_clean":"2024-04-25T10:17:38.744764+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:17:38.744764+0000","last_undegraded":"2024-04-25T10:17:38.744764+0000","last_fullsized":"2024-04-25T10:17:38.744764+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74432,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297408,"statfs":{"total":96636764160,"available":96560545792,"internally_reserved":0,"allocated":3067904,"data_stored":1750473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73812,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298028,"statfs":{"total":96636764160,"available":96561180672,"internally_reserved":0,"allocated":2490368,"data_stored":1161171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73071046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47492,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324348,"statfs":{"total":96636764160,"available":96588132352,"internally_reserved":0,"allocated":2482176,"data_stored":1161115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96056,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275784,"statfs":{"total":96636764160,"available":96538402816,"internally_reserved":0,"allocated":2469888,"data_stored":1159523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65524,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306316,"statfs":{"total":96636764160,"available":96569667584,"internally_reserved":0,"allocated":2457600,"data_stored":1152763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51652,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320188,"statfs":{"total":96636764160,"available":96583872512,"internally_reserved":0,"allocated":2473984,"data_stored":1160049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51640,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320200,"statfs":{"total":96636764160,"available":96583884800,"internally_reserved":0,"allocated":2465792,"data_stored":1159433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575045,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74428,"kb_used_data":2988,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297412,"statfs":{"total":96636764160,"available":96560549888,"internally_reserved":0,"allocated":3059712,"data_stored":1749801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:41.017 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:17:41.017 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:17:41.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c3af22640 1 Processor -- start 2024-04-25T10:17:41.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c3af22640 1 -- start start 2024-04-25T10:17:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c3af22640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f5c340cd820 0x7f5c340cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c3af22640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c3415ced0 0x7f5c3415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c3af22640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c340ce2e0 con 0x7f5c34145170 2024-04-25T10:17:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c3af22640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c3406db00 con 0x7f5c3415ced0 2024-04-25T10:17:41.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c3af22640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c34145550 con 0x7f5c340cd820 2024-04-25T10:17:41.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c394bf640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c3415ced0 0x7f5c3415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c394bf640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c3415ced0 0x7f5c3415d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46838/0 (socket says 172.21.15.133:46838) 2024-04-25T10:17:41.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c394bf640 1 -- 172.21.15.133:0/3051630895 learned_addr learned my addr 172.21.15.133:0/3051630895 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:41.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c33fff640 1 --2- 172.21.15.133:0/3051630895 >> v2:172.21.15.162:3300/0 conn(0x7f5c340cd820 0x7f5c340cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c394bf640 1 -- 172.21.15.133:0/3051630895 >> v2:172.21.15.162:3300/0 conn(0x7f5c340cd820 msgr2=0x7f5c340cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:41.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c394bf640 1 --2- 172.21.15.133:0/3051630895 >> v2:172.21.15.162:3300/0 conn(0x7f5c340cd820 0x7f5c340cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c394bf640 1 -- 172.21.15.133:0/3051630895 >> v1:172.21.15.162:6789/0 conn(0x7f5c34145170 legacy=0x7f5c3406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:41.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.179+0000 7f5c394bf640 1 -- 172.21.15.133:0/3051630895 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3406d4b0 con 0x7f5c3415ced0 2024-04-25T10:17:41.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c394bf640 1 --2- 172.21.15.133:0/3051630895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3415ced0 0x7f5c3415d320 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f5c340b4dd0 tx=0x7f5c2c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b29fa3bd531740c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c337fe640 1 -- 172.21.15.133:0/3051630895 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c0a5670 con 0x7f5c3415ced0 2024-04-25T10:17:41.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c337fe640 1 -- 172.21.15.133:0/3051630895 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c0a5810 con 0x7f5c3415ced0 2024-04-25T10:17:41.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c337fe640 1 -- 172.21.15.133:0/3051630895 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c0a5be0 con 0x7f5c3415ced0 2024-04-25T10:17:41.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c3af22640 1 -- 172.21.15.133:0/3051630895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3415ced0 msgr2=0x7f5c3415d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c3af22640 1 --2- 172.21.15.133:0/3051630895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3415ced0 0x7f5c3415d320 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f5c340b4dd0 tx=0x7f5c2c01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c3af22640 1 -- 172.21.15.133:0/3051630895 shutdown_connections 2024-04-25T10:17:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c3af22640 1 --2- 172.21.15.133:0/3051630895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3415ced0 0x7f5c3415d320 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c3af22640 1 --2- 172.21.15.133:0/3051630895 >> v2:172.21.15.162:3300/0 conn(0x7f5c340cd820 0x7f5c340cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c3af22640 1 -- 172.21.15.133:0/3051630895 >> 172.21.15.133:0/3051630895 conn(0x7f5c34066f70 msgr2=0x7f5c3406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c3af22640 1 -- 172.21.15.133:0/3051630895 shutdown_connections 2024-04-25T10:17:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c3af22640 1 -- 172.21.15.133:0/3051630895 wait complete. 2024-04-25T10:17:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c3af22640 1 Processor -- start 2024-04-25T10:17:41.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.183+0000 7f5c3af22640 1 -- start start 2024-04-25T10:17:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c3af22640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c340cd820 0x7f5c34140370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c3af22640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c34145170 0x7f5c341408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c3af22640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c3415ced0 0x7f5c340b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c3af22640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34168610 con 0x7f5c34145170 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c3af22640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34168490 con 0x7f5c3415ced0 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c3af22640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34168790 con 0x7f5c340cd820 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c33fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c34145170 0x7f5c341408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c33fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c34145170 0x7f5c341408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46844/0 (socket says 172.21.15.133:46844) 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c33fff640 1 -- 172.21.15.133:0/2845703747 learned_addr learned my addr 172.21.15.133:0/2845703747 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c394bf640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c3415ced0 0x7f5c340b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c38cbe640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c340cd820 0x7f5c34140370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c33fff640 1 -- 172.21.15.133:0/2845703747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c340cd820 msgr2=0x7f5c34140370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c33fff640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c340cd820 0x7f5c34140370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c33fff640 1 -- 172.21.15.133:0/2845703747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c3415ced0 msgr2=0x7f5c340b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c33fff640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c3415ced0 0x7f5c340b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c33fff640 1 -- 172.21.15.133:0/2845703747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c340af4b0 con 0x7f5c34145170 2024-04-25T10:17:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c33fff640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c34145170 0x7f5c341408b0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400d620 tx=0x7f5c2400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c317fa640 1 -- 172.21.15.133:0/2845703747 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2400b620 con 0x7f5c34145170 2024-04-25T10:17:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c3af22640 1 -- 172.21.15.133:0/2845703747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c340af790 con 0x7f5c34145170 2024-04-25T10:17:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c317fa640 1 -- 172.21.15.133:0/2845703747 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c24019070 con 0x7f5c34145170 2024-04-25T10:17:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c317fa640 1 -- 172.21.15.133:0/2845703747 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2400b9e0 con 0x7f5c34145170 2024-04-25T10:17:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c3af22640 1 -- 172.21.15.133:0/2845703747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c340afd50 con 0x7f5c34145170 2024-04-25T10:17:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c3af22640 1 -- 172.21.15.133:0/2845703747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c34068e80 con 0x7f5c34145170 2024-04-25T10:17:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c317fa640 1 -- 172.21.15.133:0/2845703747 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c2400bb80 con 0x7f5c34145170 2024-04-25T10:17:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c317fa640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c14066f60 0x7f5c14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c317fa640 1 -- 172.21.15.133:0/2845703747 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5c24086040 con 0x7f5c34145170 2024-04-25T10:17:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.187+0000 7f5c38cbe640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c14066f60 0x7f5c14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.191+0000 7f5c38cbe640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c14066f60 0x7f5c14069420 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00b810 tx=0x7f5c1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.191+0000 7f5c317fa640 1 -- 172.21.15.133:0/2845703747 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c24056f50 con 0x7f5c34145170 2024-04-25T10:17:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.411+0000 7f5c3af22640 1 -- 172.21.15.133:0/2845703747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5c34152630 con 0x7f5c34145170 2024-04-25T10:17:41.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c317fa640 1 -- 172.21.15.133:0/2845703747 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v28) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5c24056850 con 0x7f5c34145170 2024-04-25T10:17:41.417 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-04-25T10:17:41.417 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDUKSpmm57LMhAAG6KT8/9VvU4nwDpMAuBeug== 2024-04-25T10:17:41.417 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-04-25T10:17:41.417 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-04-25T10:17:41.417 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:17:41.418 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 -- 172.21.15.133:0/2845703747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c14066f60 msgr2=0x7f5c14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c14066f60 0x7f5c14069420 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00b810 tx=0x7f5c1c008040 comp rx=0 tx=0).stop 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 -- 172.21.15.133:0/2845703747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c34145170 msgr2=0x7f5c341408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c34145170 0x7f5c341408b0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400d620 tx=0x7f5c2400dae0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 -- 172.21.15.133:0/2845703747 shutdown_connections 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c3415ced0 0x7f5c340b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5c14066f60 0x7f5c14069420 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c34145170 0x7f5c341408b0 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 --2- 172.21.15.133:0/2845703747 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5c340cd820 0x7f5c34140370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 -- 172.21.15.133:0/2845703747 >> 172.21.15.133:0/2845703747 conn(0x7f5c34066f70 msgr2=0x7f5c3413da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 -- 172.21.15.133:0/2845703747 shutdown_connections 2024-04-25T10:17:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.415+0000 7f5c3af22640 1 -- 172.21.15.133:0/2845703747 wait complete. 2024-04-25T10:17:41.436 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.555+0000 7f2623ac8640 1 Processor -- start 2024-04-25T10:17:41.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.555+0000 7f2623ac8640 1 -- start start 2024-04-25T10:17:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2623ac8640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f261c0cd5e0 0x7f261c16ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2623ac8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261c06e640 con 0x7f261c161cb0 2024-04-25T10:17:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2623ac8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261c06e920 con 0x7f261c0cc860 2024-04-25T10:17:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2623ac8640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261c06ec00 con 0x7f261c0cd5e0 2024-04-25T10:17:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2621864640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f261c161cb0 0x7f261c06dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50022/0 (socket says 172.21.15.133:50022) 2024-04-25T10:17:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2621864640 1 -- 172.21.15.133:0/623662717 learned_addr learned my addr 172.21.15.133:0/623662717 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118186799 0 0) 0x7f261c06e640 con 0x7f261c161cb0 2024-04-25T10:17:41.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2622065640 1 --2- 172.21.15.133:0/623662717 >> v2:172.21.15.162:3300/0 conn(0x7f261c0cd5e0 0x7f261c16ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2608003530 con 0x7f261c161cb0 2024-04-25T10:17:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1992891098 0 0) 0x7f2608003530 con 0x7f261c161cb0 2024-04-25T10:17:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 >> v2:172.21.15.162:3300/0 conn(0x7f261c0cd5e0 msgr2=0x7f261c16ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 --2- 172.21.15.133:0/623662717 >> v2:172.21.15.162:3300/0 conn(0x7f261c0cd5e0 0x7f261c16ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 >> v1:172.21.15.162:6789/0 conn(0x7f261c0cc860 legacy=0x7f261c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c1625b0 con 0x7f261c161cb0 2024-04-25T10:17:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f2618001680 con 0x7f261c161cb0 2024-04-25T10:17:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 >> v1:172.21.15.133:6789/0 conn(0x7f261c161cb0 legacy=0x7f261c06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 0x7f2608003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2608004810 0x7f2608004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2608005220 0x7f260800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c06ec00 con 0x7f26080039c0 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260800c150 con 0x7f2608004810 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260800c350 con 0x7f2608005220 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2621864640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 0x7f2608003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2621063640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2608004810 0x7f2608004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2622065640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2608005220 0x7f260800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2621864640 1 -- 172.21.15.133:0/623662717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2608005220 msgr2=0x7f260800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2621864640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f2608005220 0x7f260800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2621864640 1 -- 172.21.15.133:0/623662717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2608004810 msgr2=0x7f2608004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2621864640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2608004810 0x7f2608004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2622065640 1 -- 172.21.15.133:0/623662717 reap_dead start 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2621864640 1 -- 172.21.15.133:0/623662717 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2608003530 con 0x7f26080039c0 2024-04-25T10:17:41.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.559+0000 7f2621063640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2608004810 0x7f2608004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:41.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2621864640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 0x7f2608003e00 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f261800c1c0 tx=0x7f261801cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4735782971f34823 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618007d00 con 0x7f26080039c0 2024-04-25T10:17:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26180a9020 con 0x7f26080039c0 2024-04-25T10:17:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2620862640 1 -- 172.21.15.133:0/623662717 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618005ae0 con 0x7f26080039c0 2024-04-25T10:17:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 -- 172.21.15.133:0/623662717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 msgr2=0x7f2608003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 0x7f2608003e00 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f261800c1c0 tx=0x7f261801cbb0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 -- 172.21.15.133:0/623662717 shutdown_connections 2024-04-25T10:17:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 --2- 172.21.15.133:0/623662717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 0x7f2608003e00 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 -- 172.21.15.133:0/623662717 >> 172.21.15.133:0/623662717 conn(0x7f261c067000 msgr2=0x7f261c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 -- 172.21.15.133:0/623662717 shutdown_connections 2024-04-25T10:17:41.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 -- 172.21.15.133:0/623662717 wait complete. 2024-04-25T10:17:41.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 Processor -- start 2024-04-25T10:17:41.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 -- start start 2024-04-25T10:17:41.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 0x7f261c15eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f261c14bb10 0x7f261c14bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f261c1fb8c0 0x7f261c1fbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c0b0fd0 con 0x7f26080039c0 2024-04-25T10:17:41.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c0b0e50 con 0x7f261c1fb8c0 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.563+0000 7f2623ac8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c0b1150 con 0x7f261c14bb10 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621864640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 0x7f261c15eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621864640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 0x7f261c15eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46868/0 (socket says 172.21.15.133:46868) 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621864640 1 -- 172.21.15.133:0/601918777 learned_addr learned my addr 172.21.15.133:0/601918777 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621063640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f261c14bb10 0x7f261c14bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2622065640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f261c1fb8c0 0x7f261c1fbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621864640 1 -- 172.21.15.133:0/601918777 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f261c14bb10 msgr2=0x7f261c14bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621864640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f261c14bb10 0x7f261c14bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621864640 1 -- 172.21.15.133:0/601918777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f261c1fb8c0 msgr2=0x7f261c1fbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621864640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f261c1fb8c0 0x7f261c1fbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621864640 1 -- 172.21.15.133:0/601918777 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c1fc5f0 con 0x7f26080039c0 2024-04-25T10:17:41.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621864640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 0x7f261c15eea0 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f26180ab040 tx=0x7f2608005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f26127fc640 1 -- 172.21.15.133:0/601918777 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618007d00 con 0x7f26080039c0 2024-04-25T10:17:41.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f26127fc640 1 -- 172.21.15.133:0/601918777 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26180b2070 con 0x7f26080039c0 2024-04-25T10:17:41.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2623ac8640 1 -- 172.21.15.133:0/601918777 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261c14c4d0 con 0x7f26080039c0 2024-04-25T10:17:41.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f26127fc640 1 -- 172.21.15.133:0/601918777 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618001c40 con 0x7f26080039c0 2024-04-25T10:17:41.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2623ac8640 1 -- 172.21.15.133:0/601918777 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261c15a960 con 0x7f26080039c0 2024-04-25T10:17:41.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2623ac8640 1 -- 172.21.15.133:0/601918777 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261c068ef0 con 0x7f26080039c0 2024-04-25T10:17:41.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f26127fc640 1 -- 172.21.15.133:0/601918777 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2618004bf0 con 0x7f26080039c0 2024-04-25T10:17:41.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f26127fc640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2600066f60 0x7f2600069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f26127fc640 1 -- 172.21.15.133:0/601918777 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f261811e450 con 0x7f26080039c0 2024-04-25T10:17:41.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621063640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2600066f60 0x7f2600069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.567+0000 7f2621063640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2600066f60 0x7f2600069420 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f260c000c10 tx=0x7f260c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.571+0000 7f26127fc640 1 -- 172.21.15.133:0/601918777 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26180168f0 con 0x7f26080039c0 2024-04-25T10:17:41.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.763+0000 7f2623ac8640 1 -- 172.21.15.133:0/601918777 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f261c0cb7d0 con 0x7f26080039c0 2024-04-25T10:17:41.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.763+0000 7f26127fc640 1 -- 172.21.15.133:0/601918777 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4961 (secure 0 0 0) 0x7f26180ef360 con 0x7f26080039c0 2024-04-25T10:17:41.767 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:41.767 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:36.969368+0000","modified":"2024-04-25T10:17:38.999991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11874":{"gid":11874,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/1916780253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1916780253},{"type":"v1","addr":"172.21.15.133:6837","nonce":1916780253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:17:41.767 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 118 2024-04-25T10:17:41.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 -- 172.21.15.133:0/601918777 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2600066f60 msgr2=0x7f2600069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2600066f60 0x7f2600069420 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f260c000c10 tx=0x7f260c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:41.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 -- 172.21.15.133:0/601918777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 msgr2=0x7f261c15eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 0x7f261c15eea0 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f26180ab040 tx=0x7f2608005840 comp rx=0 tx=0).stop 2024-04-25T10:17:41.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 -- 172.21.15.133:0/601918777 shutdown_connections 2024-04-25T10:17:41.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f261c1fb8c0 0x7f261c1fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f261c14bb10 0x7f261c14bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2600066f60 0x7f2600069420 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 --2- 172.21.15.133:0/601918777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26080039c0 0x7f261c15eea0 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 -- 172.21.15.133:0/601918777 >> 172.21.15.133:0/601918777 conn(0x7f261c067000 msgr2=0x7f261c15ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:41.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 -- 172.21.15.133:0/601918777 shutdown_connections 2024-04-25T10:17:41.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:41.767+0000 7f2623ac8640 1 -- 172.21.15.133:0/601918777 wait complete. 2024-04-25T10:17:42.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 118, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:36.969368+0000', 'modified': '2024-04-25T10:17:38.999991+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11874}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11874': {'gid': 11874, 'name': 'c', 'rank': 0, 'incarnation': 116, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/1916780253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 1916780253}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 1916780253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:17:42.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:17:42.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:17:42.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.319+0000 7f63fa977640 1 Processor -- start 2024-04-25T10:17:42.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.319+0000 7f63fa977640 1 -- start start 2024-04-25T10:17:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63fa977640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f40cd820 0x7f63f40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63fa977640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f40ce2e0 con 0x7f63f4145170 2024-04-25T10:17:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63fa977640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f406d4b0 con 0x7f63f415ced0 2024-04-25T10:17:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63fa977640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f406db00 con 0x7f63f40cd820 2024-04-25T10:17:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f3fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f63f4145170 0x7f63f406df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50028/0 (socket says 172.21.15.133:50028) 2024-04-25T10:17:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f3fff640 1 -- 172.21.15.133:0/4240010344 learned_addr learned my addr 172.21.15.133:0/4240010344 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f37fe640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f40cd820 0x7f63f40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922033686 0 0) 0x7f63f40ce2e0 con 0x7f63f4145170 2024-04-25T10:17:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63dc003530 con 0x7f63f4145170 2024-04-25T10:17:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 797035162 0 0) 0x7f63dc003530 con 0x7f63f4145170 2024-04-25T10:17:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f40cd820 msgr2=0x7f63f40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f40cd820 0x7f63f40cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 >> v1:172.21.15.162:6789/0 conn(0x7f63f415ced0 legacy=0x7f63f40b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f406d790 con 0x7f63f4145170 2024-04-25T10:17:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f63ec001a00 con 0x7f63f4145170 2024-04-25T10:17:42.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 >> v1:172.21.15.133:6789/0 conn(0x7f63f4145170 legacy=0x7f63f406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f40cd820 0x7f63dc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc003f70 0x7f63dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63dc004d50 0x7f63dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f406db00 con 0x7f63dc003f70 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc00bab0 con 0x7f63dc004d50 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc00bcb0 con 0x7f63f40cd820 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f37fe640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc003f70 0x7f63dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f3fff640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f40cd820 0x7f63dc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f8f14640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63dc004d50 0x7f63dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f37fe640 1 -- 172.21.15.133:0/4240010344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f40cd820 msgr2=0x7f63dc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f37fe640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f40cd820 0x7f63dc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f37fe640 1 -- 172.21.15.133:0/4240010344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63dc004d50 msgr2=0x7f63dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f37fe640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63dc004d50 0x7f63dc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.323+0000 7f63f37fe640 1 -- 172.21.15.133:0/4240010344 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc003530 con 0x7f63dc003f70 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f37fe640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc003f70 0x7f63dc004810 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009ae0 tx=0x7f63e801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b555d214d5524044 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e80b2070 con 0x7f63dc003f70 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e80a5550 con 0x7f63dc003f70 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f2ffd640 1 -- 172.21.15.133:0/4240010344 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e80096b0 con 0x7f63dc003f70 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 -- 172.21.15.133:0/4240010344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc003f70 msgr2=0x7f63dc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 --2- 172.21.15.133:0/4240010344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc003f70 0x7f63dc004810 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009ae0 tx=0x7f63e801ca70 comp rx=0 tx=0).stop 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f8f14640 1 -- 172.21.15.133:0/4240010344 reap_dead start 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 -- 172.21.15.133:0/4240010344 shutdown_connections 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 -- 172.21.15.133:0/4240010344 >> 172.21.15.133:0/4240010344 conn(0x7f63f4066f70 msgr2=0x7f63f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 -- 172.21.15.133:0/4240010344 shutdown_connections 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 -- 172.21.15.133:0/4240010344 wait complete. 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 Processor -- start 2024-04-25T10:17:42.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 -- start start 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63f40cd1b0 0x7f63f41e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f40af4b0 0x7f63f41f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f4142e30 0x7f63f41432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f415cb70 con 0x7f63f40af4b0 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f4145550 con 0x7f63f40cd1b0 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f40b1dd0 con 0x7f63f4142e30 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f40af4b0 0x7f63f41f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f40af4b0 0x7f63f41f4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46878/0 (socket says 172.21.15.133:46878) 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f37fe640 1 -- 172.21.15.133:0/45375906 learned_addr learned my addr 172.21.15.133:0/45375906 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f37fe640 1 -- 172.21.15.133:0/45375906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f4142e30 msgr2=0x7f63f41432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f37fe640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f4142e30 0x7f63f41432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f3fff640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63f40cd1b0 0x7f63f41e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f37fe640 1 -- 172.21.15.133:0/45375906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63f40cd1b0 msgr2=0x7f63f41e6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f37fe640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63f40cd1b0 0x7f63f41e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f37fe640 1 -- 172.21.15.133:0/45375906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f40cd590 con 0x7f63f40af4b0 2024-04-25T10:17:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f37fe640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f40af4b0 0x7f63f41f4650 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f63e801c8a0 tx=0x7f63e8019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f0ff9640 1 -- 172.21.15.133:0/45375906 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e80b2040 con 0x7f63f40af4b0 2024-04-25T10:17:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63f0ff9640 1 -- 172.21.15.133:0/45375906 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e800f000 con 0x7f63f40af4b0 2024-04-25T10:17:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.327+0000 7f63fa977640 1 -- 172.21.15.133:0/45375906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f4140700 con 0x7f63f40af4b0 2024-04-25T10:17:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63f0ff9640 1 -- 172.21.15.133:0/45375906 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8004260 con 0x7f63f40af4b0 2024-04-25T10:17:42.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63fa977640 1 -- 172.21.15.133:0/45375906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f40afc20 con 0x7f63f40af4b0 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63f0ff9640 1 -- 172.21.15.133:0/45375906 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63e80bf030 con 0x7f63f40af4b0 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63f0ff9640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 0x7f63e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63f0ff9640 1 -- 172.21.15.133:0/45375906 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63e811eec0 con 0x7f63f40af4b0 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63f3fff640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 0x7f63e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63f3fff640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 0x7f63e0069420 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0013e0 tx=0x7f63ec001370 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63fdae4000 1 Processor -- start 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63fdae4000 1 -- start start 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d8920 0x55a8c58d8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d9240 0x55a8c588e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c588ef30 0x55a8c588f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63fdae4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c57765f0 con 0x55a8c58d8920 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63fdae4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c56ddda0 con 0x55a8c58d9240 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63fdae4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5666150 con 0x55a8c588ef30 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.331+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d8920 0x55a8c58d8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d8920 0x55a8c58d8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46892/0 (socket says 172.21.15.133:46892) 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f3fff640 1 -- 172.21.15.133:0/1546939996 learned_addr learned my addr 172.21.15.133:0/1546939996 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f8f14640 1 --2- 172.21.15.133:0/1546939996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c588ef30 0x55a8c588f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f37fe640 1 --2- 172.21.15.133:0/1546939996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d9240 0x55a8c588e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f3fff640 1 -- 172.21.15.133:0/1546939996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c588ef30 msgr2=0x55a8c588f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f3fff640 1 --2- 172.21.15.133:0/1546939996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c588ef30 0x55a8c588f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f3fff640 1 -- 172.21.15.133:0/1546939996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d9240 msgr2=0x55a8c588e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f3fff640 1 --2- 172.21.15.133:0/1546939996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d9240 0x55a8c588e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f3fff640 1 -- 172.21.15.133:0/1546939996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8c567fca0 con 0x55a8c58d8920 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f37fe640 1 --2- 172.21.15.133:0/1546939996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d9240 0x55a8c588e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f3fff640 1 --2- 172.21.15.133:0/1546939996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d8920 0x55a8c58d8d00 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f63ec012cd0 tx=0x7f63ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd270a6847b2970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63d9ffb640 1 -- 172.21.15.133:0/1546939996 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec029070 con 0x55a8c58d8920 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63d9ffb640 1 -- 172.21.15.133:0/1546939996 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ec012d00 con 0x55a8c58d8920 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63d9ffb640 1 -- 172.21.15.133:0/1546939996 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec0155e0 con 0x55a8c58d8920 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 -- 172.21.15.133:0/1546939996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d8920 msgr2=0x55a8c58d8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 --2- 172.21.15.133:0/1546939996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d8920 0x55a8c58d8d00 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f63ec012cd0 tx=0x7f63ec015000 comp rx=0 tx=0).stop 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 -- 172.21.15.133:0/1546939996 shutdown_connections 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 --2- 172.21.15.133:0/1546939996 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c588ef30 0x55a8c588f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 --2- 172.21.15.133:0/1546939996 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d9240 0x55a8c588e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 --2- 172.21.15.133:0/1546939996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d8920 0x55a8c58d8d00 unknown :-1 s=CLOSED pgs=2879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 -- 172.21.15.133:0/1546939996 >> 172.21.15.133:0/1546939996 conn(0x55a8c58952a0 msgr2=0x55a8c588d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 -- 172.21.15.133:0/1546939996 shutdown_connections 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 -- 172.21.15.133:0/1546939996 wait complete. 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 -- start start 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 Processor -- start 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c588ef30 0x55a8c58dfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d8920 0x55a8c58e01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d9240 0x55a8c5892aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5698990 con 0x55a8c58d9240 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c56d9e40 con 0x55a8c588ef30 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5669900 con 0x55a8c58d8920 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 10 client.0 _ll_register_callbacks cb 0x55a8c57911c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f8f14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d9240 0x55a8c5892aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f8f14640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d9240 0x55a8c5892aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46908/0 (socket says 172.21.15.133:46908) 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f8f14640 1 -- 172.21.15.133:0/3254487638 learned_addr learned my addr 172.21.15.133:0/3254487638 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f37fe640 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d8920 0x55a8c58e01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f3fff640 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c588ef30 0x55a8c58dfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f8f14640 1 -- 172.21.15.133:0/3254487638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d8920 msgr2=0x55a8c58e01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f8f14640 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d8920 0x55a8c58e01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f8f14640 1 -- 172.21.15.133:0/3254487638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c588ef30 msgr2=0x55a8c58dfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f8f14640 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c588ef30 0x55a8c58dfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f8f14640 1 -- 172.21.15.133:0/3254487638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8c5696630 con 0x55a8c58d9240 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f3fff640 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c588ef30 0x55a8c58dfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63f8f14640 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d9240 0x55a8c5892aa0 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f63f406f700 tx=0x7f63e4003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52534f1a6d90c834 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63d2ffd640 1 -- 172.21.15.133:0/3254487638 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4001770 con 0x55a8c58d9240 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63d2ffd640 1 -- 172.21.15.133:0/3254487638 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63e4001910 con 0x55a8c58d9240 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.335+0000 7f63fdae4000 1 -- 172.21.15.133:0/3254487638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63ec012d00 con 0x55a8c58d9240 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63d2ffd640 1 -- 172.21.15.133:0/3254487638 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63e4021050 con 0x55a8c58d9240 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 10 client.12033 fetch_fsmap learned FSMap version 118 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 -- 172.21.15.133:0/3254487638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) v3 -- 0x55a8c56663a0 con 0x55a8c58d9240 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63d2ffd640 1 -- 172.21.15.133:0/3254487638 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ec012d00 con 0x55a8c58d9240 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63d2ffd640 1 -- 172.21.15.133:0/3254487638 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 118) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f63e4024020 con 0x55a8c58d9240 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 10 client.12033 fetch_fsmap finished waiting for FSMap version 118 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 10 client.12033 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 20 client.12033 populate_metadata read hostname 'smithi133' 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.133:6836/1916780253,v1:172.21.15.133:6837/1916780253] conn(0x55a8c58934b0 0x55a8c5893890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 4 client.12033 mds_command: new command op to 11874 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 -- 172.21.15.133:0/3254487638 --> [v2:172.21.15.133:6836/1916780253,v1:172.21.15.133:6837/1916780253] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8c5679a50 con 0x55a8c58934b0 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63f3fff640 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.133:6836/1916780253,v1:172.21.15.133:6837/1916780253] conn(0x55a8c58934b0 0x55a8c5893890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63f3fff640 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.133:6836/1916780253,v1:172.21.15.133:6837/1916780253] conn(0x55a8c58934b0 0x55a8c5893890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50a93b85ea622012 server_cookie=b4a12b32afb4df9 in_seq=0 out_seq=0 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63d2ffd640 10 client.12033 ms_handle_connect on v2:172.21.15.133:6836/1916780253 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63d2ffd640 1 -- 172.21.15.133:0/3254487638 <== mds.0 v2:172.21.15.133:6836/1916780253 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a8c5679a50 con 0x55a8c58934b0 2024-04-25T10:17:42.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63d2ffd640 10 client.12033 handle_command_reply: tid=0 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 client.12033 shutdown 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 -- 172.21.15.133:0/3254487638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d9240 msgr2=0x55a8c5892aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d9240 0x55a8c5892aa0 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f63f406f700 tx=0x7f63e4003c00 comp rx=0 tx=0).stop 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 -- 172.21.15.133:0/3254487638 shutdown_connections 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c58d9240 0x55a8c5892aa0 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d8920 0x55a8c58e01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.133:6836/1916780253,v1:172.21.15.133:6837/1916780253] conn(0x55a8c58934b0 0x55a8c5893890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 --2- 172.21.15.133:0/3254487638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c588ef30 0x55a8c58dfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 -- 172.21.15.133:0/3254487638 >> 172.21.15.133:0/3254487638 conn(0x55a8c58952a0 msgr2=0x55a8c588d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63d2ffd640 0 client.12033 ms_handle_reset on v2:172.21.15.133:6836/1916780253 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 -- 172.21.15.133:0/3254487638 shutdown_connections 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 1 -- 172.21.15.133:0/3254487638 wait complete. 2024-04-25T10:17:42.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.339+0000 7f63fdae4000 20 client.12033 trim_cache size 0 max 16384 2024-04-25T10:17:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 Processor -- start 2024-04-25T10:17:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 -- start start 2024-04-25T10:17:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d8920 0x55a8c5892800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d9240 0x55a8c58e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f4200110 con 0x55a8c588ef30 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f41ff5e0 con 0x55a8c58d8920 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f41ffb10 con 0x55a8c58d9240 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58922c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46920/0 (socket says 172.21.15.133:46920) 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63f3fff640 1 -- 172.21.15.133:0/3773366565 learned_addr learned my addr 172.21.15.133:0/3773366565 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63f3fff640 1 -- 172.21.15.133:0/3773366565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d9240 msgr2=0x55a8c58e28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63f3fff640 1 --2- 172.21.15.133:0/3773366565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d9240 0x55a8c58e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63f3fff640 1 -- 172.21.15.133:0/3773366565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d8920 msgr2=0x55a8c5892800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63f37fe640 1 --2- 172.21.15.133:0/3773366565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d8920 0x55a8c5892800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63f3fff640 1 --2- 172.21.15.133:0/3773366565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d8920 0x55a8c5892800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63f3fff640 1 -- 172.21.15.133:0/3773366565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8c58d9070 con 0x55a8c588ef30 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63f3fff640 1 --2- 172.21.15.133:0/3773366565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58922c0 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f63ec027040 tx=0x7f63ec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c1a90d377f9015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63d97fa640 1 -- 172.21.15.133:0/3773366565 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec029040 con 0x55a8c588ef30 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63d97fa640 1 -- 172.21.15.133:0/3773366565 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ec012c80 con 0x55a8c588ef30 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63d97fa640 1 -- 172.21.15.133:0/3773366565 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec0333a0 con 0x55a8c588ef30 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 -- 172.21.15.133:0/3773366565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 msgr2=0x55a8c58922c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 --2- 172.21.15.133:0/3773366565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58922c0 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f63ec027040 tx=0x7f63ec016000 comp rx=0 tx=0).stop 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 -- 172.21.15.133:0/3773366565 shutdown_connections 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 --2- 172.21.15.133:0/3773366565 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d9240 0x55a8c58e28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 --2- 172.21.15.133:0/3773366565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d8920 0x55a8c5892800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 --2- 172.21.15.133:0/3773366565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58922c0 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 -- 172.21.15.133:0/3773366565 >> 172.21.15.133:0/3773366565 conn(0x55a8c58952a0 msgr2=0x55a8c5891bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 -- 172.21.15.133:0/3773366565 shutdown_connections 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.355+0000 7f63fdae4000 1 -- 172.21.15.133:0/3773366565 wait complete. 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- start start 2024-04-25T10:17:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 Processor -- start 2024-04-25T10:17:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d8920 0x55a8c5934f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d9240 0x55a8c59354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c56e23d0 con 0x55a8c588ef30 2024-04-25T10:17:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5799d50 con 0x55a8c58d8920 2024-04-25T10:17:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8c5678980 con 0x55a8c58d9240 2024-04-25T10:17:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 10 client.0 _ll_register_callbacks cb 0x7f63ec0014e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58e58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46926/0 (socket says 172.21.15.133:46926) 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f3fff640 1 -- 172.21.15.133:0/1755682857 learned_addr learned my addr 172.21.15.133:0/1755682857 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f8f14640 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d9240 0x55a8c59354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f37fe640 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d8920 0x55a8c5934f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f3fff640 1 -- 172.21.15.133:0/1755682857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d9240 msgr2=0x55a8c59354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f3fff640 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d9240 0x55a8c59354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f3fff640 1 -- 172.21.15.133:0/1755682857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d8920 msgr2=0x55a8c5934f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f3fff640 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d8920 0x55a8c5934f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f3fff640 1 -- 172.21.15.133:0/1755682857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8c575f600 con 0x55a8c588ef30 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f3fff640 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58e58d0 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00b040 tx=0x7f63ec001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ea30cce4ad116a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63d2ffd640 1 -- 172.21.15.133:0/1755682857 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec029040 con 0x55a8c588ef30 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63d2ffd640 1 -- 172.21.15.133:0/1755682857 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63ec033160 con 0x55a8c588ef30 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- 172.21.15.133:0/1755682857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63ec012c80 con 0x55a8c588ef30 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63d2ffd640 1 -- 172.21.15.133:0/1755682857 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63ec049050 con 0x55a8c588ef30 2024-04-25T10:17:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 10 client.12039 fetch_fsmap learned FSMap version 118 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- 172.21.15.133:0/1755682857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) v3 -- 0x55a8c56729b0 con 0x55a8c588ef30 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63d2ffd640 1 -- 172.21.15.133:0/1755682857 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ec012c80 con 0x55a8c588ef30 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63d2ffd640 1 -- 172.21.15.133:0/1755682857 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 118) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f63ec04c020 con 0x55a8c588ef30 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 10 client.12039 fetch_fsmap finished waiting for FSMap version 118 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 10 client.12039 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 20 client.12039 populate_metadata read hostname 'smithi133' 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.133:6836/1916780253,v1:172.21.15.133:6837/1916780253] conn(0x55a8c58e5e10 0x55a8c59359e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 4 client.12039 mds_command: new command op to 11874 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- 172.21.15.133:0/1755682857 --> [v2:172.21.15.133:6836/1916780253,v1:172.21.15.133:6837/1916780253] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a8c56eb2e0 con 0x55a8c58e5e10 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f37fe640 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.133:6836/1916780253,v1:172.21.15.133:6837/1916780253] conn(0x55a8c58e5e10 0x55a8c59359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63d2ffd640 10 client.12039 ms_handle_connect on v2:172.21.15.133:6836/1916780253 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63f37fe640 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.133:6836/1916780253,v1:172.21.15.133:6837/1916780253] conn(0x55a8c58e5e10 0x55a8c59359e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d03fe96e66f06b8b server_cookie=d32b5534372322e in_seq=0 out_seq=0 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63d2ffd640 1 -- 172.21.15.133:0/1755682857 <== mds.0 v2:172.21.15.133:6836/1916780253 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x55a8c56eb2e0 con 0x55a8c58e5e10 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63d2ffd640 10 client.12039 handle_command_reply: tid=0 2024-04-25T10:17:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 client.12039 shutdown 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- 172.21.15.133:0/1755682857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 msgr2=0x55a8c58e58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58e58d0 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00b040 tx=0x7f63ec001dd0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- 172.21.15.133:0/1755682857 shutdown_connections 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a8c58d9240 0x55a8c59354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.133:6836/1916780253,v1:172.21.15.133:6837/1916780253] conn(0x55a8c58e5e10 0x55a8c59359e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a8c58d8920 0x55a8c5934f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 --2- 172.21.15.133:0/1755682857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8c588ef30 0x55a8c58e58d0 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63d2ffd640 0 client.12039 ms_handle_reset on v2:172.21.15.133:6836/1916780253 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- 172.21.15.133:0/1755682857 >> 172.21.15.133:0/1755682857 conn(0x55a8c58952a0 msgr2=0x55a8c58c4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- 172.21.15.133:0/1755682857 shutdown_connections 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 1 -- 172.21.15.133:0/1755682857 wait complete. 2024-04-25T10:17:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.359+0000 7f63fdae4000 20 client.12039 trim_cache size 0 max 16384 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 11874, 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 4.7045975269999998, 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 118, 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 195, 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 195, 2024-04-25T10:17:42.370 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 10.769907258 2024-04-25T10:17:42.371 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:17:42.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 -- 172.21.15.133:0/45375906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 msgr2=0x7f63e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 0x7f63e0069420 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0013e0 tx=0x7f63ec001370 comp rx=0 tx=0).stop 2024-04-25T10:17:42.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 -- 172.21.15.133:0/45375906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f40af4b0 msgr2=0x7f63f41f4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f40af4b0 0x7f63f41f4650 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f63e801c8a0 tx=0x7f63e8019810 comp rx=0 tx=0).stop 2024-04-25T10:17:42.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 -- 172.21.15.133:0/45375906 shutdown_connections 2024-04-25T10:17:42.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f63f4142e30 0x7f63f41432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f63e0066f60 0x7f63e0069420 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63f40af4b0 0x7f63f41f4650 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 --2- 172.21.15.133:0/45375906 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63f40cd1b0 0x7f63f41e6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 -- 172.21.15.133:0/45375906 >> 172.21.15.133:0/45375906 conn(0x7f63f4066f70 msgr2=0x7f63f41409e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 -- 172.21.15.133:0/45375906 shutdown_connections 2024-04-25T10:17:42.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.363+0000 7f63fa977640 1 -- 172.21.15.133:0/45375906 wait complete. 2024-04-25T10:17:42.381 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:17:42.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.487+0000 7f59f8fd3640 1 Processor -- start 2024-04-25T10:17:42.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f8fd3640 1 -- start start 2024-04-25T10:17:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f8fd3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f59f40b1f70 0x7f59f406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f8fd3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59f40ce330 0x7f59f40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f8fd3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f40c61d0 con 0x7f59f415c9a0 2024-04-25T10:17:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f8fd3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f40cecc0 con 0x7f59f40b1f70 2024-04-25T10:17:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f8fd3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f40c64b0 con 0x7f59f40ce330 2024-04-25T10:17:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f259c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f59f415c9a0 0x7f59f40c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50044/0 (socket says 172.21.15.133:50044) 2024-04-25T10:17:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f259c640 1 -- 172.21.15.133:0/2186517956 learned_addr learned my addr 172.21.15.133:0/2186517956 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f2d9d640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59f40ce330 0x7f59f40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f1d9b640 1 --2- 172.21.15.133:0/2186517956 >> v2:172.21.15.162:3300/0 conn(0x7f59f40b1f70 0x7f59f406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472468125 0 0) 0x7f59f40c61d0 con 0x7f59f415c9a0 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59e0003530 con 0x7f59f415c9a0 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2101188681 0 0) 0x7f59e0003530 con 0x7f59f415c9a0 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59f40ce330 msgr2=0x7f59f40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59f40ce330 0x7f59f40ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 >> v2:172.21.15.162:3300/0 conn(0x7f59f40b1f70 msgr2=0x7f59f406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 --2- 172.21.15.133:0/2186517956 >> v2:172.21.15.162:3300/0 conn(0x7f59f40b1f70 0x7f59f406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f406d4f0 con 0x7f59f415c9a0 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f59ec001a00 con 0x7f59f415c9a0 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 >> v1:172.21.15.133:6789/0 conn(0x7f59f415c9a0 legacy=0x7f59f40c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59e00039c0 0x7f59e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59e0004810 0x7f59e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59f40ce330 0x7f59e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f40cecc0 con 0x7f59e00039c0 2024-04-25T10:17:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f40c64b0 con 0x7f59e0004810 2024-04-25T10:17:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e000b9d0 con 0x7f59f40ce330 2024-04-25T10:17:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f259c640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59e00039c0 0x7f59e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f1d9b640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59e0004810 0x7f59e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f2d9d640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59f40ce330 0x7f59e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f259c640 1 -- 172.21.15.133:0/2186517956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59f40ce330 msgr2=0x7f59e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f259c640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59f40ce330 0x7f59e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f259c640 1 -- 172.21.15.133:0/2186517956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59e0004810 msgr2=0x7f59e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f259c640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59e0004810 0x7f59e0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.491+0000 7f59f259c640 1 -- 172.21.15.133:0/2186517956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e0003530 con 0x7f59e00039c0 2024-04-25T10:17:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f259c640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59e00039c0 0x7f59e0003de0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00a870 tx=0x7f59ec01d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd7b84d17ae6500 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec00a4c0 con 0x7f59e00039c0 2024-04-25T10:17:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec0ad630 con 0x7f59e00039c0 2024-04-25T10:17:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f159a640 1 -- 172.21.15.133:0/2186517956 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec0ada20 con 0x7f59e00039c0 2024-04-25T10:17:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2186517956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59e00039c0 msgr2=0x7f59e0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 --2- 172.21.15.133:0/2186517956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59e00039c0 0x7f59e0003de0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00a870 tx=0x7f59ec01d370 comp rx=0 tx=0).stop 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f2d9d640 1 -- 172.21.15.133:0/2186517956 reap_dead start 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2186517956 shutdown_connections 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2186517956 >> 172.21.15.133:0/2186517956 conn(0x7f59f4067000 msgr2=0x7f59f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2186517956 shutdown_connections 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2186517956 wait complete. 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 Processor -- start 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 -- start start 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59e00039c0 0x7f59f40ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59f415b8c0 0x7f59f413de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59f415a370 0x7f59f415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f415dad0 con 0x7f59f415a370 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f415d950 con 0x7f59f415b8c0 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f8fd3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f415dc50 con 0x7f59e00039c0 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.495+0000 7f59f2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59f415a370 0x7f59f415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59f415a370 0x7f59f415a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46952/0 (socket says 172.21.15.133:46952) 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f2d9d640 1 -- 172.21.15.133:0/2779972673 learned_addr learned my addr 172.21.15.133:0/2779972673 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f1d9b640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59f415b8c0 0x7f59f413de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f259c640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59e00039c0 0x7f59f40ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f2d9d640 1 -- 172.21.15.133:0/2779972673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59e00039c0 msgr2=0x7f59f40ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f2d9d640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59e00039c0 0x7f59f40ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f2d9d640 1 -- 172.21.15.133:0/2779972673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59f415b8c0 msgr2=0x7f59f413de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f2d9d640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59f415b8c0 0x7f59f413de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f2d9d640 1 -- 172.21.15.133:0/2779972673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f41595f0 con 0x7f59f415a370 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f1d9b640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59f415b8c0 0x7f59f413de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f2d9d640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59f415a370 0x7f59f415a7f0 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f59f4071160 tx=0x7f59e4011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59eaffd640 1 -- 172.21.15.133:0/2779972673 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e4003620 con 0x7f59f415a370 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59eaffd640 1 -- 172.21.15.133:0/2779972673 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e401b070 con 0x7f59f415a370 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2779972673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f415c0d0 con 0x7f59f415a370 2024-04-25T10:17:42.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59eaffd640 1 -- 172.21.15.133:0/2779972673 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e4003a40 con 0x7f59f415a370 2024-04-25T10:17:42.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2779972673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f4159b70 con 0x7f59f415a370 2024-04-25T10:17:42.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2779972673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f4068ef0 con 0x7f59f415a370 2024-04-25T10:17:42.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59eaffd640 1 -- 172.21.15.133:0/2779972673 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59e4003be0 con 0x7f59f415a370 2024-04-25T10:17:42.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59eaffd640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59d4066f60 0x7f59d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59eaffd640 1 -- 172.21.15.133:0/2779972673 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f59e4085610 con 0x7f59f415a370 2024-04-25T10:17:42.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f259c640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59d4066f60 0x7f59d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.499+0000 7f59f259c640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59d4066f60 0x7f59d4069420 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0ad000 tx=0x7f59ec001370 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.503+0000 7f59eaffd640 1 -- 172.21.15.133:0/2779972673 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59e4056400 con 0x7f59f415a370 2024-04-25T10:17:42.582 INFO:tasks.ceph.mds.c.smithi133.stderr:2024-04-25T10:17:42.579+0000 7f72158f3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:17:42.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2779972673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f59f4155560 con 0x7f59f415a370 2024-04-25T10:17:42.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59eaffd640 1 -- 172.21.15.133:0/2779972673 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v118) v1 ==== 53+0+73 (secure 0 0 0) 0x7f59e4055d00 con 0x7f59f415a370 2024-04-25T10:17:42.690 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:17:42.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2779972673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59d4066f60 msgr2=0x7f59d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59d4066f60 0x7f59d4069420 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0ad000 tx=0x7f59ec001370 comp rx=0 tx=0).stop 2024-04-25T10:17:42.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2779972673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59f415a370 msgr2=0x7f59f415a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59f415a370 0x7f59f415a7f0 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f59f4071160 tx=0x7f59e4011920 comp rx=0 tx=0).stop 2024-04-25T10:17:42.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2779972673 shutdown_connections 2024-04-25T10:17:42.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f59d4066f60 0x7f59d4069420 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59f415a370 0x7f59f415a7f0 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59f415b8c0 0x7f59f413de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 --2- 172.21.15.133:0/2779972673 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f59e00039c0 0x7f59f40ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2779972673 >> 172.21.15.133:0/2779972673 conn(0x7f59f4067000 msgr2=0x7f59f414b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2779972673 shutdown_connections 2024-04-25T10:17:42.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:42.687+0000 7f59f8fd3640 1 -- 172.21.15.133:0/2779972673 wait complete. 2024-04-25T10:17:42.707 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:17:42.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:17:42.708 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-04-25T10:17:42.708 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:17:42.708 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:42.708 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:17:42.708 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:17:42.708 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:17:42.708 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:42.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:42.708 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-04-25T10:17:42.714 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:17:42.714 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:17:42.714 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:17:42.714 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:42.714 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-04-25T10:17:42.714 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:42.714 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:17:42.714 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41569sec preferred_lft 41569sec 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:17:42.715 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 02:42:b3:63:05:9f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:17:42.716 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:42.717 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:17:42.717 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:42.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:42.717 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:17:42.765 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-04-25T10:17:42.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:42.766 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-04-25T10:17:42.816 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-04-25T10:17:42.816 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:17:42.816 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:17:42.816 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:17:42.816 INFO:teuthology.orchestra.run.smithi133.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:17:42.816 INFO:teuthology.orchestra.run.smithi133.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:17:42.816 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:17:42.816 INFO:teuthology.orchestra.run.smithi133.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:17:42.816 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:17:42.817 INFO:teuthology.orchestra.run.smithi133.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:17:42.817 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:17:42.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:42.817 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-04-25T10:17:42.817 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-04-25T10:17:42.817 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:17:42.817 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:17:42.817 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:17:42.817 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-04-25T10:17:42.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:42.892 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:17:42.936 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:17:42.936 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:17:42.937 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:42.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:42.937 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-04-25T10:17:42.990 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:17:42.991 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:17:42.991 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:43.037 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:43.038 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:17:43.038 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:43.085 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:17:43.142 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:17:43.190 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:17:43.190 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:17:43.190 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:17:43.190 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:17:43.190 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:17:43.191 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:43.192 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:43.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:43.193 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:17:43.241 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:17:43.241 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:17:43.290 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:17:43.306 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:17:43.358 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:17:43.358 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:17:43.358 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:17:43.358 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:43.358 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:43.358 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:43.358 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:43.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:43.359 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:17:43.368 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:2024-04-25T10:17:43.367+0000 7f10b112d580 -1 init, newargv = 0x559cf2237dc0 newargc=15 2024-04-25T10:17:43.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[92467]: starting ceph client 2024-04-25T10:17:43.375 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[92467]: starting fuse 2024-04-25T10:17:44.475 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-04-25T10:17:44.491 DEBUG:teuthology.orchestra.run.smithi133:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:17:44.540 INFO:teuthology.orchestra.run.smithi133.stdout:/sys 2024-04-25T10:17:44.540 INFO:teuthology.orchestra.run.smithi133.stdout:/proc 2024-04-25T10:17:44.540 INFO:teuthology.orchestra.run.smithi133.stdout:/dev 2024-04-25T10:17:44.540 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/pts 2024-04-25T10:17:44.540 INFO:teuthology.orchestra.run.smithi133.stdout:/run 2024-04-25T10:17:44.540 INFO:teuthology.orchestra.run.smithi133.stdout:/ 2024-04-25T10:17:44.540 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/security 2024-04-25T10:17:44.541 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/shm 2024-04-25T10:17:44.541 INFO:teuthology.orchestra.run.smithi133.stdout:/run/lock 2024-04-25T10:17:44.541 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/cgroup 2024-04-25T10:17:44.541 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/pstore 2024-04-25T10:17:44.541 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/bpf 2024-04-25T10:17:44.541 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:17:44.541 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/hugepages 2024-04-25T10:17:44.541 INFO:teuthology.orchestra.run.smithi133.stdout:/dev/mqueue 2024-04-25T10:17:44.542 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/debug 2024-04-25T10:17:44.542 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/tracing 2024-04-25T10:17:44.542 INFO:teuthology.orchestra.run.smithi133.stdout:/proc/fs/nfsd 2024-04-25T10:17:44.542 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/fs/fuse/connections 2024-04-25T10:17:44.542 INFO:teuthology.orchestra.run.smithi133.stdout:/sys/kernel/config 2024-04-25T10:17:44.542 INFO:teuthology.orchestra.run.smithi133.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:17:44.542 INFO:teuthology.orchestra.run.smithi133.stdout:/run/rpc_pipefs 2024-04-25T10:17:44.543 INFO:teuthology.orchestra.run.smithi133.stdout:/run/user/1000 2024-04-25T10:17:44.543 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph 2024-04-25T10:17:44.543 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:17:44.543 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:17:44.543 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:17:44.543 INFO:teuthology.orchestra.run.smithi133.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:17:44.543 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns 2024-04-25T10:17:44.543 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:44.544 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:44.544 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:44.544 INFO:teuthology.orchestra.run.smithi133.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:44.544 INFO:teuthology.orchestra.run.smithi133.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:44.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:44.545 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-04-25T10:17:44.597 INFO:teuthology.orchestra.run.smithi133.stdout:45 2024-04-25T10:17:44.597 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:17:44.598 DEBUG:teuthology.orchestra.run.smithi133:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:17:44.598 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-04-25T10:17:44.598 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-04-25T10:17:44.598 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-04-25T10:17:44.598 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-04-25T10:17:44.598 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:44.598 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-04-25T10:17:44.598 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:17:44.598 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-04-25T10:17:44.598 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:17:44.598 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.0")) 2024-04-25T10:17:44.599 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-04-25T10:17:44.722 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.0.92467.asok 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:44.734 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.92467.asok 2024-04-25T10:17:44.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:44.735 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.92467.asok status 2024-04-25T10:17:44.890 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:17:44.890 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-04-25T10:17:44.890 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:17:44.890 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:17:44.890 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "0", 2024-04-25T10:17:44.890 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-04-25T10:17:44.890 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "92467", 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 12051, 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 12051 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:17:44.891 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 4145013848 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 4145013848 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.12051 192.168.0.1:0/4145013848", 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/4145013848", 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 118, 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 195, 2024-04-25T10:17:44.892 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:17:44.893 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-04-25T10:17:44.893 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-04-25T10:17:44.893 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:17:44.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:44.904 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-04-25T10:17:44.904 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:44.910 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-04-25T10:17:44.911 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:44.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:44.911 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:44.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:44.966 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent) 2024-04-25T10:17:45.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:45.023 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/child) 2024-04-25T10:17:45.075 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:17:45.075 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:45.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:45.122 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/parentfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:17:45.251 INFO:teuthology.orchestra.run.smithi133.stderr:6+0 records in 2024-04-25T10:17:45.251 INFO:teuthology.orchestra.run.smithi133.stderr:6+0 records out 2024-04-25T10:17:45.251 INFO:teuthology.orchestra.run.smithi133.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0783799 s, 80.3 MB/s 2024-04-25T10:17:45.252 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:17:45.252 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:45.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:45.259 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/child/childfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:17:45.385 INFO:teuthology.orchestra.run.smithi133.stderr:6+0 records in 2024-04-25T10:17:45.386 INFO:teuthology.orchestra.run.smithi133.stderr:6+0 records out 2024-04-25T10:17:45.386 INFO:teuthology.orchestra.run.smithi133.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0754379 s, 83.4 MB/s 2024-04-25T10:17:45.388 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:17:45.390 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.417 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-04-25T10:17:45.418 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-04-25T10:17:45.419 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.133,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.420 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.421 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 20 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 20 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 10 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-04-25T10:17:45.422 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-04-25T10:17:45.423 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:17:45.424 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2024-04-25T10:17:45.425 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:17:45.426 INFO:teuthology.orchestra.run.smithi133.stdout: mds cache max size = 1000 2024-04-25T10:17:45.427 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2024-04-25T10:17:45.427 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2024-04-25T10:17:45.427 INFO:teuthology.orchestra.run.smithi133.stdout: mds log max segments = 1 2024-04-25T10:17:45.427 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2024-04-25T10:17:45.427 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2024-04-25T10:17:45.427 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-04-25T10:17:45.427 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-04-25T10:17:45.427 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-04-25T10:17:45.427 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.428 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:17:45.429 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.133,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.430 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug trim objects = true 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.431 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2024-04-25T10:17:45.432 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-04-25T10:17:45.433 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2024-04-25T10:17:45.434 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2024-04-25T10:17:45.435 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = "" 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2024-04-25T10:17:45.436 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds balancer = 20 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout: mds cache max size = 1000 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = False 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout: mds log max segments = 1 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = False 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-04-25T10:17:45.437 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-04-25T10:17:45.438 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-04-25T10:17:45.440 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:17:45.482 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.133,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-04-25T10:17:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 20 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 20 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 10 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-04-25T10:17:45.520 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout: mds cache max size = 1000 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout: mds log max segments = 1 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-04-25T10:17:45.521 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-04-25T10:17:45.522 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-04-25T10:17:45.523 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-04-25T10:17:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2024-04-25T10:17:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:17:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2024-04-25T10:17:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2024-04-25T10:17:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:17:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.133,172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790]" 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 599cdb88-206d-4299-bbcd-9f7da90fb181 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug trim objects = true 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2024-04-25T10:17:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 20 2024-04-25T10:17:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 20 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 10 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-04-25T10:17:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2024-04-25T10:17:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: mon op complaint time = 120 2024-04-25T10:17:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: client mount timeout = 600 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: debug client = 20 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = "" 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2024-04-25T10:17:45.532 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.a] 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.c] 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.b] 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout:[mds.d] 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout:[mds] 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds = 20 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: debug mds balancer = 20 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: mds cache max size = 1000 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug frag = True 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: mds debug scatterstat = False 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: mds log max segments = 1 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: mds op complaint time = 180 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: mds verify scatter = False 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: osd op complaint time = 180 2024-04-25T10:17:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: rados mon op timeout = 900 2024-04-25T10:17:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: rados osd op timeout = 900 2024-04-25T10:17:45.534 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:17:45.534 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:45.570 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:45.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.743+0000 7f4bb9bd3640 1 Processor -- start 2024-04-25T10:17:45.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.743+0000 7f4bb9bd3640 1 -- start start 2024-04-25T10:17:45.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb9bd3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4bb406dca0 0x7f4bb40b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb9bd3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bb40cc250 0x7f4bb40cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb9bd3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb40cfdc0 con 0x7f4bb406bda0 2024-04-25T10:17:45.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb9bd3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb40d0660 con 0x7f4bb406dca0 2024-04-25T10:17:45.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb9bd3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb40d00a0 con 0x7f4bb40cc250 2024-04-25T10:17:45.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb37fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4bb406dca0 0x7f4bb40b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:45.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb37fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4bb406dca0 0x7f4bb40b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46960/0 (socket says 172.21.15.133:46960) 2024-04-25T10:17:45.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb37fe640 1 -- 172.21.15.133:0/2048922099 learned_addr learned my addr 172.21.15.133:0/2048922099 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:45.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb3fff640 1 --2- 172.21.15.133:0/2048922099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bb40cc250 0x7f4bb40cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:45.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb37fe640 1 -- 172.21.15.133:0/2048922099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bb40cc250 msgr2=0x7f4bb40cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:45.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb37fe640 1 --2- 172.21.15.133:0/2048922099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bb40cc250 0x7f4bb40cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb37fe640 1 -- 172.21.15.133:0/2048922099 >> v1:172.21.15.162:6789/0 conn(0x7f4bb406bda0 legacy=0x7f4bb40b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb37fe640 1 -- 172.21.15.133:0/2048922099 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb40d0380 con 0x7f4bb406dca0 2024-04-25T10:17:45.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb37fe640 1 --2- 172.21.15.133:0/2048922099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bb406dca0 0x7f4bb40b1620 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009870 tx=0x7f4ba801cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceddb682407eac42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:45.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb27fc640 1 -- 172.21.15.133:0/2048922099 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80b0070 con 0x7f4bb406dca0 2024-04-25T10:17:45.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb27fc640 1 -- 172.21.15.133:0/2048922099 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba8004230 con 0x7f4bb406dca0 2024-04-25T10:17:45.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb27fc640 1 -- 172.21.15.133:0/2048922099 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8003620 con 0x7f4bb406dca0 2024-04-25T10:17:45.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2048922099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bb406dca0 msgr2=0x7f4bb40b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb9bd3640 1 --2- 172.21.15.133:0/2048922099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bb406dca0 0x7f4bb40b1620 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009870 tx=0x7f4ba801cea0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2048922099 shutdown_connections 2024-04-25T10:17:45.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb9bd3640 1 --2- 172.21.15.133:0/2048922099 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bb40cc250 0x7f4bb40cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.747+0000 7f4bb9bd3640 1 --2- 172.21.15.133:0/2048922099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bb406dca0 0x7f4bb40b1620 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2048922099 >> 172.21.15.133:0/2048922099 conn(0x7f4bb40b31a0 msgr2=0x7f4bb40b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:45.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2048922099 shutdown_connections 2024-04-25T10:17:45.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2048922099 wait complete. 2024-04-25T10:17:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 Processor -- start 2024-04-25T10:17:45.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 -- start start 2024-04-25T10:17:45.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bb406bda0 0x7f4bb4159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bb406dca0 0x7f4bb414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb40cc250 0x7f4bb415ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb416a920 con 0x7f4bb406bda0 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb416a7a0 con 0x7f4bb40cc250 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb416aaa0 con 0x7f4bb406dca0 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bb406bda0 0x7f4bb4159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bb406bda0 0x7f4bb4159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46970/0 (socket says 172.21.15.133:46970) 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb37fe640 1 -- 172.21.15.133:0/2553238363 learned_addr learned my addr 172.21.15.133:0/2553238363 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb3fff640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb40cc250 0x7f4bb415ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb2ffd640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bb406dca0 0x7f4bb414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb37fe640 1 -- 172.21.15.133:0/2553238363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bb406dca0 msgr2=0x7f4bb414eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb37fe640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bb406dca0 0x7f4bb414eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb37fe640 1 -- 172.21.15.133:0/2553238363 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb40cc250 msgr2=0x7f4bb415ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb37fe640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb40cc250 0x7f4bb415ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb37fe640 1 -- 172.21.15.133:0/2553238363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb415fb90 con 0x7f4bb406bda0 2024-04-25T10:17:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb2ffd640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bb406dca0 0x7f4bb414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb3fff640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb40cc250 0x7f4bb415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb37fe640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bb406bda0 0x7f4bb4159ce0 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f4ba801aaf0 tx=0x7f4ba80191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4b9bfff640 1 -- 172.21.15.133:0/2553238363 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80b9020 con 0x7f4bb406bda0 2024-04-25T10:17:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4b9bfff640 1 -- 172.21.15.133:0/2553238363 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba80b1040 con 0x7f4bb406bda0 2024-04-25T10:17:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2553238363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb4158ff0 con 0x7f4bb406bda0 2024-04-25T10:17:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4b9bfff640 1 -- 172.21.15.133:0/2553238363 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80b0050 con 0x7f4bb406bda0 2024-04-25T10:17:45.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2553238363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb41595b0 con 0x7f4bb406bda0 2024-04-25T10:17:45.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.751+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2553238363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb4067c70 con 0x7f4bb406bda0 2024-04-25T10:17:45.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.755+0000 7f4b9bfff640 1 -- 172.21.15.133:0/2553238363 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ba80bc020 con 0x7f4bb406bda0 2024-04-25T10:17:45.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.755+0000 7f4b9bfff640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ba0066f60 0x7f4ba0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.755+0000 7f4b9bfff640 1 -- 172.21.15.133:0/2553238363 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ba811ee50 con 0x7f4bb406bda0 2024-04-25T10:17:45.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.755+0000 7f4bb2ffd640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ba0066f60 0x7f4ba0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:45.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.755+0000 7f4bb2ffd640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ba0066f60 0x7f4ba0069420 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4157f80 tx=0x7f4ba4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:45.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.759+0000 7f4b9bfff640 1 -- 172.21.15.133:0/2553238363 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ba80c3030 con 0x7f4bb406bda0 2024-04-25T10:17:45.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.915+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2553238363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bb40b0690 con 0x7f4bb406bda0 2024-04-25T10:17:45.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.919+0000 7f4b9bfff640 1 -- 172.21.15.133:0/2553238363 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+16908 (secure 0 0 0) 0x7f4ba8016150 con 0x7f4bb406bda0 2024-04-25T10:17:45.923 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:45.923 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":195,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:38.655088+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:34.663983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:35.974809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:45.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2553238363 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ba0066f60 msgr2=0x7f4ba0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ba0066f60 0x7f4ba0069420 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4157f80 tx=0x7f4ba4009040 comp rx=0 tx=0).stop 2024-04-25T10:17:45.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2553238363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bb406bda0 msgr2=0x7f4bb4159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bb406bda0 0x7f4bb4159ce0 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f4ba801aaf0 tx=0x7f4ba80191e0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2553238363 shutdown_connections 2024-04-25T10:17:45.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4bb40cc250 0x7f4bb415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4bb406dca0 0x7f4bb414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4ba0066f60 0x7f4ba0069420 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 --2- 172.21.15.133:0/2553238363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bb406bda0 0x7f4bb4159ce0 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2553238363 >> 172.21.15.133:0/2553238363 conn(0x7f4bb40b31a0 msgr2=0x7f4bb40b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:45.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2553238363 shutdown_connections 2024-04-25T10:17:45.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:45.923+0000 7f4bb9bd3640 1 -- 172.21.15.133:0/2553238363 wait complete. 2024-04-25T10:17:45.942 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:34.663983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '194', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:35.974809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:45.942 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:46.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a7914640 1 Processor -- start 2024-04-25T10:17:46.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a7914640 1 -- start start 2024-04-25T10:17:46.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a7914640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0a0145170 0x7fd0a006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a7914640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a006e4d0 0x7fd0a00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a7914640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a00ce2e0 con 0x7fd0a00cdd60 2024-04-25T10:17:46.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a7914640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a006db00 con 0x7fd0a0145170 2024-04-25T10:17:46.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a7914640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a0145550 con 0x7fd0a006e4d0 2024-04-25T10:17:46.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a56b0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0a0145170 0x7fd0a006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a56b0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0a0145170 0x7fd0a006df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46984/0 (socket says 172.21.15.133:46984) 2024-04-25T10:17:46.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a56b0640 1 -- 172.21.15.133:0/2313747399 learned_addr learned my addr 172.21.15.133:0/2313747399 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:46.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a4eaf640 1 --2- 172.21.15.133:0/2313747399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a006e4d0 0x7fd0a00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a56b0640 1 -- 172.21.15.133:0/2313747399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a006e4d0 msgr2=0x7fd0a00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:46.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a56b0640 1 --2- 172.21.15.133:0/2313747399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a006e4d0 0x7fd0a00cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a56b0640 1 -- 172.21.15.133:0/2313747399 >> v1:172.21.15.162:6789/0 conn(0x7fd0a00cdd60 legacy=0x7fd0a015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.063+0000 7fd0a56b0640 1 -- 172.21.15.133:0/2313747399 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a006d4b0 con 0x7fd0a0145170 2024-04-25T10:17:46.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd0a56b0640 1 --2- 172.21.15.133:0/2313747399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a0145170 0x7fd0a006df90 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009870 tx=0x7fd09c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0b920291107fc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd097fff640 1 -- 172.21.15.133:0/2313747399 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0b2070 con 0x7fd0a0145170 2024-04-25T10:17:46.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd097fff640 1 -- 172.21.15.133:0/2313747399 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c0a9070 con 0x7fd0a0145170 2024-04-25T10:17:46.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd097fff640 1 -- 172.21.15.133:0/2313747399 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0ad880 con 0x7fd0a0145170 2024-04-25T10:17:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd0a7914640 1 -- 172.21.15.133:0/2313747399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a0145170 msgr2=0x7fd0a006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd0a7914640 1 --2- 172.21.15.133:0/2313747399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a0145170 0x7fd0a006df90 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009870 tx=0x7fd09c01c560 comp rx=0 tx=0).stop 2024-04-25T10:17:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd0a7914640 1 -- 172.21.15.133:0/2313747399 shutdown_connections 2024-04-25T10:17:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd0a7914640 1 --2- 172.21.15.133:0/2313747399 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a006e4d0 0x7fd0a00cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd0a7914640 1 --2- 172.21.15.133:0/2313747399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a0145170 0x7fd0a006df90 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd0a7914640 1 -- 172.21.15.133:0/2313747399 >> 172.21.15.133:0/2313747399 conn(0x7fd0a0066f70 msgr2=0x7fd0a006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd0a7914640 1 -- 172.21.15.133:0/2313747399 shutdown_connections 2024-04-25T10:17:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.067+0000 7fd0a7914640 1 -- 172.21.15.133:0/2313747399 wait complete. 2024-04-25T10:17:46.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a7914640 1 Processor -- start 2024-04-25T10:17:46.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a7914640 1 -- start start 2024-04-25T10:17:46.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a7914640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0a006e4d0 0x7fd0a013b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a7914640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a00cdd60 0x7fd0a0143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a7914640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0145170 0x7fd0a0143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a7914640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a0168490 con 0x7fd0a00cdd60 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a7914640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a015d7f0 con 0x7fd0a006e4d0 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a7914640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a0168610 con 0x7fd0a0145170 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a4eaf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a00cdd60 0x7fd0a0143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a4eaf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a00cdd60 0x7fd0a0143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46990/0 (socket says 172.21.15.133:46990) 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a4eaf640 1 -- 172.21.15.133:0/3505765200 learned_addr learned my addr 172.21.15.133:0/3505765200 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a56b0640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0a006e4d0 0x7fd0a013b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a5eb1640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0145170 0x7fd0a0143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a4eaf640 1 -- 172.21.15.133:0/3505765200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0145170 msgr2=0x7fd0a0143a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a4eaf640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0145170 0x7fd0a0143a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a4eaf640 1 -- 172.21.15.133:0/3505765200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0a006e4d0 msgr2=0x7fd0a013b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a4eaf640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0a006e4d0 0x7fd0a013b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a4eaf640 1 -- 172.21.15.133:0/3505765200 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a0139f20 con 0x7fd0a00cdd60 2024-04-25T10:17:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a5eb1640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0145170 0x7fd0a0143a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a4eaf640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a00cdd60 0x7fd0a0143510 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7fd09000e870 tx=0x7fd09000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd095ffb640 1 -- 172.21.15.133:0/3505765200 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090017070 con 0x7fd0a00cdd60 2024-04-25T10:17:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd095ffb640 1 -- 172.21.15.133:0/3505765200 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09000c940 con 0x7fd0a00cdd60 2024-04-25T10:17:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd095ffb640 1 -- 172.21.15.133:0/3505765200 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09000cd30 con 0x7fd0a00cdd60 2024-04-25T10:17:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a7914640 1 -- 172.21.15.133:0/3505765200 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a013a1c0 con 0x7fd0a00cdd60 2024-04-25T10:17:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a7914640 1 -- 172.21.15.133:0/3505765200 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a013b5a0 con 0x7fd0a00cdd60 2024-04-25T10:17:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.071+0000 7fd0a7914640 1 -- 172.21.15.133:0/3505765200 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a0068e80 con 0x7fd0a00cdd60 2024-04-25T10:17:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.075+0000 7fd095ffb640 1 -- 172.21.15.133:0/3505765200 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd09001a070 con 0x7fd0a00cdd60 2024-04-25T10:17:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.075+0000 7fd095ffb640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd084066f60 0x7fd084069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.075+0000 7fd0a56b0640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd084066f60 0x7fd084069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.075+0000 7fd095ffb640 1 -- 172.21.15.133:0/3505765200 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd090085380 con 0x7fd0a00cdd60 2024-04-25T10:17:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.075+0000 7fd0a56b0640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd084066f60 0x7fd084069420 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0af000 tx=0x7fd09c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.079+0000 7fd095ffb640 1 -- 172.21.15.133:0/3505765200 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd090056290 con 0x7fd0a00cdd60 2024-04-25T10:17:46.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 -- 172.21.15.133:0/3505765200 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0a0141e10 con 0x7fd0a00cdd60 2024-04-25T10:17:46.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd095ffb640 1 -- 172.21.15.133:0/3505765200 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v195) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd090055b90 con 0x7fd0a00cdd60 2024-04-25T10:17:46.230 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:17:46.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 -- 172.21.15.133:0/3505765200 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd084066f60 msgr2=0x7fd084069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd084066f60 0x7fd084069420 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0af000 tx=0x7fd09c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:17:46.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 -- 172.21.15.133:0/3505765200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a00cdd60 msgr2=0x7fd0a0143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a00cdd60 0x7fd0a0143510 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7fd09000e870 tx=0x7fd09000ec70 comp rx=0 tx=0).stop 2024-04-25T10:17:46.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 -- 172.21.15.133:0/3505765200 shutdown_connections 2024-04-25T10:17:46.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0a0145170 0x7fd0a0143a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd084066f60 0x7fd084069420 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a00cdd60 0x7fd0a0143510 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 --2- 172.21.15.133:0/3505765200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd0a006e4d0 0x7fd0a013b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 -- 172.21.15.133:0/3505765200 >> 172.21.15.133:0/3505765200 conn(0x7fd0a0066f70 msgr2=0x7fd0a00b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:46.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 -- 172.21.15.133:0/3505765200 shutdown_connections 2024-04-25T10:17:46.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.227+0000 7fd0a7914640 1 -- 172.21.15.133:0/3505765200 wait complete. 2024-04-25T10:17:46.247 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:17:46.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.363+0000 7fb7fb5f3640 1 Processor -- start 2024-04-25T10:17:46.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.363+0000 7fb7fb5f3640 1 -- start start 2024-04-25T10:17:46.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7fb5f3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb7f415d9a0 0x7fb7f40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7fb5f3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7f40ce600 0x7fb7f40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7fb5f3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f415ebd0 con 0x7fb7f40d1920 2024-04-25T10:17:46.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7fb5f3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f415eeb0 con 0x7fb7f40ce600 2024-04-25T10:17:46.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7fb5f3640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f40d0330 con 0x7fb7f415d9a0 2024-04-25T10:17:46.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7f8b8e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7f40ce600 0x7fb7f40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7f8b8e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7f40ce600 0x7fb7f40cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45528/0 (socket says 172.21.15.133:45528) 2024-04-25T10:17:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7f8b8e640 1 -- 172.21.15.133:0/399827577 learned_addr learned my addr 172.21.15.133:0/399827577 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7f938f640 1 --2- 172.21.15.133:0/399827577 >> v2:172.21.15.162:3300/0 conn(0x7fb7f415d9a0 0x7fb7f40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7f8b8e640 1 -- 172.21.15.133:0/399827577 >> v2:172.21.15.162:3300/0 conn(0x7fb7f415d9a0 msgr2=0x7fb7f40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7f8b8e640 1 --2- 172.21.15.133:0/399827577 >> v2:172.21.15.162:3300/0 conn(0x7fb7f415d9a0 0x7fb7f40d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7f8b8e640 1 -- 172.21.15.133:0/399827577 >> v1:172.21.15.162:6789/0 conn(0x7fb7f40d1920 legacy=0x7fb7f40d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7f8b8e640 1 -- 172.21.15.133:0/399827577 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f40d0610 con 0x7fb7f40ce600 2024-04-25T10:17:46.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7f938f640 1 --2- 172.21.15.133:0/399827577 >> v2:172.21.15.162:3300/0 conn(0x7fb7f415d9a0 0x7fb7f40d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7f8b8e640 1 --2- 172.21.15.133:0/399827577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f40ce600 0x7fb7f40cea90 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40d4750 tx=0x7fb7e401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec94e9f196c5f91e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7ebfff640 1 -- 172.21.15.133:0/399827577 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40b0070 con 0x7fb7f40ce600 2024-04-25T10:17:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7ebfff640 1 -- 172.21.15.133:0/399827577 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e40ab600 con 0x7fb7f40ce600 2024-04-25T10:17:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.367+0000 7fb7ebfff640 1 -- 172.21.15.133:0/399827577 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40ab9d0 con 0x7fb7f40ce600 2024-04-25T10:17:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/399827577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f40ce600 msgr2=0x7fb7f40cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 --2- 172.21.15.133:0/399827577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f40ce600 0x7fb7f40cea90 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40d4750 tx=0x7fb7e401c450 comp rx=0 tx=0).stop 2024-04-25T10:17:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/399827577 shutdown_connections 2024-04-25T10:17:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 --2- 172.21.15.133:0/399827577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f40ce600 0x7fb7f40cea90 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 --2- 172.21.15.133:0/399827577 >> v2:172.21.15.162:3300/0 conn(0x7fb7f415d9a0 0x7fb7f40d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/399827577 >> 172.21.15.133:0/399827577 conn(0x7fb7f406bf80 msgr2=0x7fb7f406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/399827577 shutdown_connections 2024-04-25T10:17:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/399827577 wait complete. 2024-04-25T10:17:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 Processor -- start 2024-04-25T10:17:46.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 -- start start 2024-04-25T10:17:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7f40ce600 0x7fb7f4143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f40d1920 0x7fb7f40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f415d9a0 0x7fb7f4156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f40b9820 con 0x7fb7f415d9a0 2024-04-25T10:17:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f40b96a0 con 0x7fb7f40ce600 2024-04-25T10:17:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f415e590 con 0x7fb7f40d1920 2024-04-25T10:17:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7f9b90640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f415d9a0 0x7fb7f4156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7f9b90640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f415d9a0 0x7fb7f4156030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45532/0 (socket says 172.21.15.133:45532) 2024-04-25T10:17:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7f9b90640 1 -- 172.21.15.133:0/2535641426 learned_addr learned my addr 172.21.15.133:0/2535641426 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7f938f640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7f40ce600 0x7fb7f4143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7f9b90640 1 -- 172.21.15.133:0/2535641426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f40d1920 msgr2=0x7fb7f40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7f9b90640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f40d1920 0x7fb7f40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7f9b90640 1 -- 172.21.15.133:0/2535641426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7f40ce600 msgr2=0x7fb7f4143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7f9b90640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7f40ce600 0x7fb7f4143e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7f9b90640 1 -- 172.21.15.133:0/2535641426 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f41443a0 con 0x7fb7f415d9a0 2024-04-25T10:17:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7f9b90640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f415d9a0 0x7fb7f4156030 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c780 tx=0x7fb7ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7e9ffb640 1 -- 172.21.15.133:0/2535641426 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec004b70 con 0x7fb7f415d9a0 2024-04-25T10:17:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7e9ffb640 1 -- 172.21.15.133:0/2535641426 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec004d10 con 0x7fb7f415d9a0 2024-04-25T10:17:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7e9ffb640 1 -- 172.21.15.133:0/2535641426 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00aa90 con 0x7fb7f415d9a0 2024-04-25T10:17:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/2535641426 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f4155570 con 0x7fb7f415d9a0 2024-04-25T10:17:46.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.371+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/2535641426 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f40c9ba0 con 0x7fb7f415d9a0 2024-04-25T10:17:46.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.375+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/2535641426 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f4068e80 con 0x7fb7f415d9a0 2024-04-25T10:17:46.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.375+0000 7fb7e9ffb640 1 -- 172.21.15.133:0/2535641426 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7ec00ac30 con 0x7fb7f415d9a0 2024-04-25T10:17:46.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.375+0000 7fb7e9ffb640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7d8066f60 0x7fb7d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.375+0000 7fb7f938f640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7d8066f60 0x7fb7d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.375+0000 7fb7e9ffb640 1 -- 172.21.15.133:0/2535641426 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb7ec0853a0 con 0x7fb7f415d9a0 2024-04-25T10:17:46.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.375+0000 7fb7f938f640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7d8066f60 0x7fb7d8069420 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0006fd0 tx=0x7fb7f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.379+0000 7fb7e9ffb640 1 -- 172.21.15.133:0/2535641426 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7ec020070 con 0x7fb7f415d9a0 2024-04-25T10:17:46.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.535+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/2535641426 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7f40c9e70 con 0x7fb7f415d9a0 2024-04-25T10:17:46.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7e9ffb640 1 -- 172.21.15.133:0/2535641426 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v195) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7f41443a0 con 0x7fb7f415d9a0 2024-04-25T10:17:46.541 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:17:46.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/2535641426 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7d8066f60 msgr2=0x7fb7d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7d8066f60 0x7fb7d8069420 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0006fd0 tx=0x7fb7f0008040 comp rx=0 tx=0).stop 2024-04-25T10:17:46.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/2535641426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f415d9a0 msgr2=0x7fb7f4156030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f415d9a0 0x7fb7f4156030 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c780 tx=0x7fb7ec00cc40 comp rx=0 tx=0).stop 2024-04-25T10:17:46.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/2535641426 shutdown_connections 2024-04-25T10:17:46.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7d8066f60 0x7fb7d8069420 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f415d9a0 0x7fb7f4156030 unknown :-1 s=CLOSED pgs=2893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f40d1920 0x7fb7f40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 --2- 172.21.15.133:0/2535641426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7f40ce600 0x7fb7f4143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/2535641426 >> 172.21.15.133:0/2535641426 conn(0x7fb7f406bf80 msgr2=0x7fb7f4167fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:46.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/2535641426 shutdown_connections 2024-04-25T10:17:46.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.539+0000 7fb7fb5f3640 1 -- 172.21.15.133:0/2535641426 wait complete. 2024-04-25T10:17:46.560 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:17:46.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f552064c640 1 Processor -- start 2024-04-25T10:17:46.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f552064c640 1 -- start start 2024-04-25T10:17:46.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f552064c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f551815b8b0 0x7f55180ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f552064c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f55180d03a0 0x7f55180b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f552064c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55180cc6a0 con 0x7f551815bc90 2024-04-25T10:17:46.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f552064c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55180cc980 con 0x7f55180d03a0 2024-04-25T10:17:46.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f552064c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55180ccc60 con 0x7f551815b8b0 2024-04-25T10:17:46.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551dbe7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f551815bc90 0x7f55180cfc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48388/0 (socket says 172.21.15.133:48388) 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551dbe7640 1 -- 172.21.15.133:0/336587005 learned_addr learned my addr 172.21.15.133:0/336587005 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551e3e8640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f551815b8b0 0x7f55180ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551ebe9640 1 --2- 172.21.15.133:0/336587005 >> v2:172.21.15.162:3300/0 conn(0x7f55180d03a0 0x7f55180b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761088598 0 0) 0x7f55180cc6a0 con 0x7f551815bc90 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5504003530 con 0x7f551815bc90 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3494610547 0 0) 0x7f5504003530 con 0x7f551815bc90 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f551815b8b0 msgr2=0x7f55180ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f551815b8b0 0x7f55180ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 >> v2:172.21.15.162:3300/0 conn(0x7f55180d03a0 msgr2=0x7f55180b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 --2- 172.21.15.133:0/336587005 >> v2:172.21.15.162:3300/0 conn(0x7f55180d03a0 0x7f55180b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55180ccf40 con 0x7f551815bc90 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f5508001680 con 0x7f551815bc90 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 >> v1:172.21.15.133:6789/0 conn(0x7f551815bc90 legacy=0x7f55180cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55040039c0 0x7f5504003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5504004810 0x7f5504004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f551815b8b0 0x7f55040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55180cc980 con 0x7f55040039c0 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55180ccc60 con 0x7f5504004810 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550400b9d0 con 0x7f551815b8b0 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551e3e8640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55040039c0 0x7f5504003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551dbe7640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5504004810 0x7f5504004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551ebe9640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f551815b8b0 0x7f55040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551e3e8640 1 -- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f551815b8b0 msgr2=0x7f55040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551e3e8640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f551815b8b0 0x7f55040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551e3e8640 1 -- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5504004810 msgr2=0x7f5504004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551e3e8640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5504004810 0x7f5504004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551e3e8640 1 -- 172.21.15.133:0/336587005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5504003530 con 0x7f55040039c0 2024-04-25T10:17:46.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551dbe7640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5504004810 0x7f5504004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:46.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.679+0000 7f551ebe9640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f551815b8b0 0x7f55040051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f551e3e8640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55040039c0 0x7f5504003de0 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f5514009af0 tx=0x7f551401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d965f328030c949 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514009690 con 0x7f55040039c0 2024-04-25T10:17:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55140a6440 con 0x7f55040039c0 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f551d3e6640 1 -- 172.21.15.133:0/336587005 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55140a6810 con 0x7f55040039c0 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 -- 172.21.15.133:0/336587005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55040039c0 msgr2=0x7f5504003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 --2- 172.21.15.133:0/336587005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55040039c0 0x7f5504003de0 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f5514009af0 tx=0x7f551401ca30 comp rx=0 tx=0).stop 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f551ebe9640 1 -- 172.21.15.133:0/336587005 reap_dead start 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 -- 172.21.15.133:0/336587005 shutdown_connections 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 -- 172.21.15.133:0/336587005 >> 172.21.15.133:0/336587005 conn(0x7f551809a990 msgr2=0x7f55180d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 -- 172.21.15.133:0/336587005 shutdown_connections 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 -- 172.21.15.133:0/336587005 wait complete. 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 Processor -- start 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 -- start start 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55040039c0 0x7f551815a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55180b6bc0 0x7f55180b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55180ca230 0x7f55180ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55180b17f0 con 0x7f55180ca230 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55180ced30 con 0x7f55180b6bc0 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.683+0000 7f552064c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55180b1970 con 0x7f55040039c0 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551ebe9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55180ca230 0x7f55180ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551ebe9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55180ca230 0x7f55180ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45550/0 (socket says 172.21.15.133:45550) 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551ebe9640 1 -- 172.21.15.133:0/3921421809 learned_addr learned my addr 172.21.15.133:0/3921421809 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551e3e8640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55040039c0 0x7f551815a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551dbe7640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55180b6bc0 0x7f55180b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551ebe9640 1 -- 172.21.15.133:0/3921421809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55040039c0 msgr2=0x7f551815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551ebe9640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55040039c0 0x7f551815a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551ebe9640 1 -- 172.21.15.133:0/3921421809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55180b6bc0 msgr2=0x7f55180b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551ebe9640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55180b6bc0 0x7f55180b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551ebe9640 1 -- 172.21.15.133:0/3921421809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55180b5820 con 0x7f55180ca230 2024-04-25T10:17:46.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551ebe9640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55180ca230 0x7f55180ca6b0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f55180cd960 tx=0x7f551000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f550effd640 1 -- 172.21.15.133:0/3921421809 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551001c070 con 0x7f55180ca230 2024-04-25T10:17:46.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f550effd640 1 -- 172.21.15.133:0/3921421809 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5510017400 con 0x7f55180ca230 2024-04-25T10:17:46.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f552064c640 1 -- 172.21.15.133:0/3921421809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55180b5b00 con 0x7f55180ca230 2024-04-25T10:17:46.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f550effd640 1 -- 172.21.15.133:0/3921421809 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55100177d0 con 0x7f55180ca230 2024-04-25T10:17:46.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f552064c640 1 -- 172.21.15.133:0/3921421809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551815ac10 con 0x7f55180ca230 2024-04-25T10:17:46.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f552064c640 1 -- 172.21.15.133:0/3921421809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551815c650 con 0x7f55180ca230 2024-04-25T10:17:46.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f550effd640 1 -- 172.21.15.133:0/3921421809 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f551001f070 con 0x7f55180ca230 2024-04-25T10:17:46.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f550effd640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54fc066f60 0x7f54fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f550effd640 1 -- 172.21.15.133:0/3921421809 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55100851e0 con 0x7f55180ca230 2024-04-25T10:17:46.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551e3e8640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54fc066f60 0x7f54fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.687+0000 7f551e3e8640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54fc066f60 0x7f54fc069420 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f55140ad000 tx=0x7f55140b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.691+0000 7f550effd640 1 -- 172.21.15.133:0/3921421809 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5510012070 con 0x7f55180ca230 2024-04-25T10:17:46.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.851+0000 7f552064c640 1 -- 172.21.15.133:0/3921421809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5518067c70 con 0x7f55180ca230 2024-04-25T10:17:46.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.851+0000 7f550effd640 1 -- 172.21.15.133:0/3921421809 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v195) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5510056060 con 0x7f55180ca230 2024-04-25T10:17:46.855 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:17:46.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 -- 172.21.15.133:0/3921421809 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54fc066f60 msgr2=0x7f54fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54fc066f60 0x7f54fc069420 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f55140ad000 tx=0x7f55140b2040 comp rx=0 tx=0).stop 2024-04-25T10:17:46.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 -- 172.21.15.133:0/3921421809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55180ca230 msgr2=0x7f55180ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55180ca230 0x7f55180ca6b0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f55180cd960 tx=0x7f551000f4a0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 -- 172.21.15.133:0/3921421809 shutdown_connections 2024-04-25T10:17:46.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f54fc066f60 0x7f54fc069420 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55180ca230 0x7f55180ca6b0 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55180b6bc0 0x7f55180b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 --2- 172.21.15.133:0/3921421809 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55040039c0 0x7f551815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 -- 172.21.15.133:0/3921421809 >> 172.21.15.133:0/3921421809 conn(0x7f551809a990 msgr2=0x7f55180735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:46.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 -- 172.21.15.133:0/3921421809 shutdown_connections 2024-04-25T10:17:46.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.855+0000 7f552064c640 1 -- 172.21.15.133:0/3921421809 wait complete. 2024-04-25T10:17:46.874 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:17:46.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca753b640 1 Processor -- start 2024-04-25T10:17:46.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca753b640 1 -- start start 2024-04-25T10:17:46.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca753b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6ca015d5e0 0x7f6ca00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca753b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca00ce2e0 con 0x7f6ca00cd820 2024-04-25T10:17:46.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca753b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca006d4b0 con 0x7f6ca0145170 2024-04-25T10:17:46.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca753b640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca006db00 con 0x7f6ca015d5e0 2024-04-25T10:17:46.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca4ad6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6ca00cd820 0x7f6ca015ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48390/0 (socket says 172.21.15.133:48390) 2024-04-25T10:17:46.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca4ad6640 1 -- 172.21.15.133:0/2389859082 learned_addr learned my addr 172.21.15.133:0/2389859082 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:46.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca5ad8640 1 --2- 172.21.15.133:0/2389859082 >> v2:172.21.15.133:3300/0 conn(0x7f6ca015d5e0 0x7f6ca00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6c9ffff640 1 -- 172.21.15.133:0/2389859082 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909533324 0 0) 0x7f6ca00ce2e0 con 0x7f6ca00cd820 2024-04-25T10:17:46.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6c9ffff640 1 -- 172.21.15.133:0/2389859082 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c88003530 con 0x7f6ca00cd820 2024-04-25T10:17:46.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca5ad8640 1 -- 172.21.15.133:0/2389859082 >> v1:172.21.15.162:6789/0 conn(0x7f6ca0145170 legacy=0x7f6ca006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca5ad8640 1 -- 172.21.15.133:0/2389859082 >> v1:172.21.15.133:6789/0 conn(0x7f6ca00cd820 legacy=0x7f6ca015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.987+0000 7f6ca5ad8640 1 -- 172.21.15.133:0/2389859082 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca006d790 con 0x7f6ca015d5e0 2024-04-25T10:17:46.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6ca5ad8640 1 --2- 172.21.15.133:0/2389859082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca015d5e0 0x7f6ca00cdca0 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f6c980097c0 tx=0x7f6c9801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec72922ff9f0968c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6c9ffff640 1 -- 172.21.15.133:0/2389859082 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980b0070 con 0x7f6ca015d5e0 2024-04-25T10:17:46.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6c9ffff640 1 -- 172.21.15.133:0/2389859082 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c980ab600 con 0x7f6ca015d5e0 2024-04-25T10:17:46.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6c9ffff640 1 -- 172.21.15.133:0/2389859082 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980ab9d0 con 0x7f6ca015d5e0 2024-04-25T10:17:46.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6ca753b640 1 -- 172.21.15.133:0/2389859082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca015d5e0 msgr2=0x7f6ca00cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6ca753b640 1 --2- 172.21.15.133:0/2389859082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca015d5e0 0x7f6ca00cdca0 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f6c980097c0 tx=0x7f6c9801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6ca753b640 1 -- 172.21.15.133:0/2389859082 shutdown_connections 2024-04-25T10:17:46.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6ca753b640 1 --2- 172.21.15.133:0/2389859082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca015d5e0 0x7f6ca00cdca0 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6ca753b640 1 -- 172.21.15.133:0/2389859082 >> 172.21.15.133:0/2389859082 conn(0x7f6ca0066f70 msgr2=0x7f6ca006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:46.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6ca753b640 1 -- 172.21.15.133:0/2389859082 shutdown_connections 2024-04-25T10:17:46.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6ca753b640 1 -- 172.21.15.133:0/2389859082 wait complete. 2024-04-25T10:17:46.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.991+0000 7f6ca753b640 1 Processor -- start 2024-04-25T10:17:46.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca753b640 1 -- start start 2024-04-25T10:17:46.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca753b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ca00cd820 0x7f6ca0155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca753b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ca0145170 0x7f6ca0154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca753b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca0153b20 0x7f6ca013b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca753b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01696b0 con 0x7f6ca0153b20 2024-04-25T10:17:46.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca753b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0145550 con 0x7f6ca00cd820 2024-04-25T10:17:46.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca753b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0169830 con 0x7f6ca0145170 2024-04-25T10:17:46.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca5ad8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca0153b20 0x7f6ca013b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca5ad8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca0153b20 0x7f6ca013b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45574/0 (socket says 172.21.15.133:45574) 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca5ad8640 1 -- 172.21.15.133:0/3506658358 learned_addr learned my addr 172.21.15.133:0/3506658358 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca5ad8640 1 -- 172.21.15.133:0/3506658358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ca0145170 msgr2=0x7f6ca0154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca5ad8640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ca0145170 0x7f6ca0154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca5ad8640 1 -- 172.21.15.133:0/3506658358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ca00cd820 msgr2=0x7f6ca0155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca5ad8640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ca00cd820 0x7f6ca0155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca5ad8640 1 -- 172.21.15.133:0/3506658358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca013bff0 con 0x7f6ca0153b20 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca5ad8640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca0153b20 0x7f6ca013b4e0 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7f6c980ae040 tx=0x7f6c98019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6c9dffb640 1 -- 172.21.15.133:0/3506658358 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980b8020 con 0x7f6ca0153b20 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6c9dffb640 1 -- 172.21.15.133:0/3506658358 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c980b4410 con 0x7f6ca0153b20 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca753b640 1 -- 172.21.15.133:0/3506658358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca0154cd0 con 0x7f6ca0153b20 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6c9dffb640 1 -- 172.21.15.133:0/3506658358 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980b0040 con 0x7f6ca0153b20 2024-04-25T10:17:46.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca753b640 1 -- 172.21.15.133:0/3506658358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca0153320 con 0x7f6ca0153b20 2024-04-25T10:17:47.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca753b640 1 -- 172.21.15.133:0/3506658358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca0068e80 con 0x7f6ca0153b20 2024-04-25T10:17:47.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6c9dffb640 1 -- 172.21.15.133:0/3506658358 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c980bb020 con 0x7f6ca0153b20 2024-04-25T10:17:47.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6c9dffb640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c8c066f60 0x7f6c8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6c9dffb640 1 -- 172.21.15.133:0/3506658358 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c9811ec50 con 0x7f6ca0153b20 2024-04-25T10:17:47.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca52d7640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c8c066f60 0x7f6c8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.995+0000 7f6ca52d7640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c8c066f60 0x7f6c8c069420 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6c90000f70 tx=0x7f6c900079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:46.999+0000 7f6c9dffb640 1 -- 172.21.15.133:0/3506658358 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c98124020 con 0x7f6ca0153b20 2024-04-25T10:17:47.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.155+0000 7f6ca753b640 1 -- 172.21.15.133:0/3506658358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6ca0166750 con 0x7f6ca0153b20 2024-04-25T10:17:47.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6c9dffb640 1 -- 172.21.15.133:0/3506658358 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v195) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6c980c2030 con 0x7f6ca0153b20 2024-04-25T10:17:47.161 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:17:47.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 -- 172.21.15.133:0/3506658358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c8c066f60 msgr2=0x7f6c8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c8c066f60 0x7f6c8c069420 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6c90000f70 tx=0x7f6c900079c0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 -- 172.21.15.133:0/3506658358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca0153b20 msgr2=0x7f6ca013b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca0153b20 0x7f6ca013b4e0 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7f6c980ae040 tx=0x7f6c98019240 comp rx=0 tx=0).stop 2024-04-25T10:17:47.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 -- 172.21.15.133:0/3506658358 shutdown_connections 2024-04-25T10:17:47.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6c8c066f60 0x7f6c8c069420 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca0153b20 0x7f6ca013b4e0 unknown :-1 s=CLOSED pgs=2899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6ca0145170 0x7f6ca0154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 --2- 172.21.15.133:0/3506658358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6ca00cd820 0x7f6ca0155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 -- 172.21.15.133:0/3506658358 >> 172.21.15.133:0/3506658358 conn(0x7f6ca0066f70 msgr2=0x7f6ca015dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:47.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 -- 172.21.15.133:0/3506658358 shutdown_connections 2024-04-25T10:17:47.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.159+0000 7f6ca753b640 1 -- 172.21.15.133:0/3506658358 wait complete. 2024-04-25T10:17:47.178 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:17:47.178 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:47.183 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:47.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.343+0000 7fcf54a1d640 1 Processor -- start 2024-04-25T10:17:47.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.343+0000 7fcf54a1d640 1 -- start start 2024-04-25T10:17:47.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf54a1d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcf50159280 0x7fcf50072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf54a1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf5006b3a0 0x7fcf500ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf54a1d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5015cdc0 con 0x7fcf500730e0 2024-04-25T10:17:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf54a1d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5006fb80 con 0x7fcf50159280 2024-04-25T10:17:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf54a1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf500709e0 con 0x7fcf5006b3a0 2024-04-25T10:17:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf4e59c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcf50159280 0x7fcf50072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf4ed9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf5006b3a0 0x7fcf500ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf4e59c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcf50159280 0x7fcf50072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:55940/0 (socket says 172.21.15.133:55940) 2024-04-25T10:17:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf4e59c640 1 -- 172.21.15.133:0/2742576859 learned_addr learned my addr 172.21.15.133:0/2742576859 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf45d9b640 1 --1- 172.21.15.133:0/2742576859 >> v1:172.21.15.162:6789/0 conn(0x7fcf500730e0 0x7fcf5006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:33324/0 (socket says 172.21.15.133:33324) 2024-04-25T10:17:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf4ed9d640 1 -- 172.21.15.133:0/2742576859 >> v2:172.21.15.162:3300/0 conn(0x7fcf50159280 msgr2=0x7fcf50072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf4ed9d640 1 --2- 172.21.15.133:0/2742576859 >> v2:172.21.15.162:3300/0 conn(0x7fcf50159280 0x7fcf50072ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf4ed9d640 1 -- 172.21.15.133:0/2742576859 >> v1:172.21.15.162:6789/0 conn(0x7fcf500730e0 legacy=0x7fcf5006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf4dd9b640 1 -- 172.21.15.133:0/2742576859 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794850768 0 0) 0x7fcf5015cdc0 con 0x7fcf500730e0 2024-04-25T10:17:47.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf4ed9d640 1 -- 172.21.15.133:0/2742576859 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5006fdd0 con 0x7fcf5006b3a0 2024-04-25T10:17:47.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf4e59c640 1 --2- 172.21.15.133:0/2742576859 >> v2:172.21.15.162:3300/0 conn(0x7fcf50159280 0x7fcf50072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:47.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.347+0000 7fcf4ed9d640 1 --2- 172.21.15.133:0/2742576859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf5006b3a0 0x7fcf500ccee0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009d90 tx=0x7fcf4801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbe29ed906ce67c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf4dd9b640 1 -- 172.21.15.133:0/2742576859 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf480b2070 con 0x7fcf5006b3a0 2024-04-25T10:17:47.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf4dd9b640 1 -- 172.21.15.133:0/2742576859 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf480a5a00 con 0x7fcf5006b3a0 2024-04-25T10:17:47.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf4dd9b640 1 -- 172.21.15.133:0/2742576859 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf480a5dd0 con 0x7fcf5006b3a0 2024-04-25T10:17:47.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 -- 172.21.15.133:0/2742576859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf5006b3a0 msgr2=0x7fcf500ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 --2- 172.21.15.133:0/2742576859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf5006b3a0 0x7fcf500ccee0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009d90 tx=0x7fcf4801c940 comp rx=0 tx=0).stop 2024-04-25T10:17:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 -- 172.21.15.133:0/2742576859 shutdown_connections 2024-04-25T10:17:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 --2- 172.21.15.133:0/2742576859 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf5006b3a0 0x7fcf500ccee0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 --2- 172.21.15.133:0/2742576859 >> v2:172.21.15.162:3300/0 conn(0x7fcf50159280 0x7fcf50072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 -- 172.21.15.133:0/2742576859 >> 172.21.15.133:0/2742576859 conn(0x7fcf500b2370 msgr2=0x7fcf500b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 -- 172.21.15.133:0/2742576859 shutdown_connections 2024-04-25T10:17:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 -- 172.21.15.133:0/2742576859 wait complete. 2024-04-25T10:17:47.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 Processor -- start 2024-04-25T10:17:47.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 -- start start 2024-04-25T10:17:47.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5006b3a0 0x7fcf5014f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf50159280 0x7fcf50141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf500a84c0 0x7fcf5015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf501597e0 con 0x7fcf50159280 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf50159660 con 0x7fcf5006b3a0 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf54a1d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf500cdc30 con 0x7fcf500a84c0 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf45d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf50159280 0x7fcf50141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf45d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf50159280 0x7fcf50141020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45576/0 (socket says 172.21.15.133:45576) 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf45d9b640 1 -- 172.21.15.133:0/3507450666 learned_addr learned my addr 172.21.15.133:0/3507450666 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf4ed9d640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf500a84c0 0x7fcf5015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf4e59c640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5006b3a0 0x7fcf5014f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf45d9b640 1 -- 172.21.15.133:0/3507450666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf500a84c0 msgr2=0x7fcf5015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf45d9b640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf500a84c0 0x7fcf5015a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf45d9b640 1 -- 172.21.15.133:0/3507450666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5006b3a0 msgr2=0x7fcf5014f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf45d9b640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5006b3a0 0x7fcf5014f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf45d9b640 1 -- 172.21.15.133:0/3507450666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5014f840 con 0x7fcf50159280 2024-04-25T10:17:47.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.351+0000 7fcf45d9b640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf50159280 0x7fcf50141020 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7fcf380011f0 tx=0x7fcf3800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.355+0000 7fcf477fe640 1 -- 172.21.15.133:0/3507450666 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38018070 con 0x7fcf50159280 2024-04-25T10:17:47.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.355+0000 7fcf477fe640 1 -- 172.21.15.133:0/3507450666 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf380074e0 con 0x7fcf50159280 2024-04-25T10:17:47.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.355+0000 7fcf477fe640 1 -- 172.21.15.133:0/3507450666 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38004a10 con 0x7fcf50159280 2024-04-25T10:17:47.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.355+0000 7fcf54a1d640 1 -- 172.21.15.133:0/3507450666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5014fb20 con 0x7fcf50159280 2024-04-25T10:17:47.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.355+0000 7fcf54a1d640 1 -- 172.21.15.133:0/3507450666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf501500e0 con 0x7fcf50159280 2024-04-25T10:17:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.355+0000 7fcf54a1d640 1 -- 172.21.15.133:0/3507450666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf50067c70 con 0x7fcf50159280 2024-04-25T10:17:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.355+0000 7fcf477fe640 1 -- 172.21.15.133:0/3507450666 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf38004bb0 con 0x7fcf50159280 2024-04-25T10:17:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.355+0000 7fcf477fe640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf30066f90 0x7fcf30069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.355+0000 7fcf477fe640 1 -- 172.21.15.133:0/3507450666 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcf380843d0 con 0x7fcf50159280 2024-04-25T10:17:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.355+0000 7fcf4e59c640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf30066f90 0x7fcf30069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.355+0000 7fcf4e59c640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf30066f90 0x7fcf30069450 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009960 tx=0x7fcf40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.359+0000 7fcf477fe640 1 -- 172.21.15.133:0/3507450666 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcf38055250 con 0x7fcf50159280 2024-04-25T10:17:47.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.519+0000 7fcf54a1d640 1 -- 172.21.15.133:0/3507450666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf500ce6c0 con 0x7fcf50159280 2024-04-25T10:17:47.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.523+0000 7fcf477fe640 1 -- 172.21.15.133:0/3507450666 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+16908 (secure 0 0 0) 0x7fcf38054b50 con 0x7fcf50159280 2024-04-25T10:17:47.526 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:47.526 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":195,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:17:38.655088+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:34.663983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:35.974809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:47.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.523+0000 7fcf54a1d640 1 -- 172.21.15.133:0/3507450666 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf30066f90 msgr2=0x7fcf30069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.523+0000 7fcf54a1d640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf30066f90 0x7fcf30069450 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009960 tx=0x7fcf40008040 comp rx=0 tx=0).stop 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.523+0000 7fcf54a1d640 1 -- 172.21.15.133:0/3507450666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf50159280 msgr2=0x7fcf50141020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.523+0000 7fcf54a1d640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf50159280 0x7fcf50141020 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7fcf380011f0 tx=0x7fcf3800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.523+0000 7fcf54a1d640 1 -- 172.21.15.133:0/3507450666 shutdown_connections 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.523+0000 7fcf54a1d640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcf500a84c0 0x7fcf5015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.523+0000 7fcf54a1d640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcf30066f90 0x7fcf30069450 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.523+0000 7fcf54a1d640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf50159280 0x7fcf50141020 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.523+0000 7fcf54a1d640 1 --2- 172.21.15.133:0/3507450666 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf5006b3a0 0x7fcf5014f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.527+0000 7fcf54a1d640 1 -- 172.21.15.133:0/3507450666 >> 172.21.15.133:0/3507450666 conn(0x7fcf500b2370 msgr2=0x7fcf50155070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.527+0000 7fcf54a1d640 1 -- 172.21.15.133:0/3507450666 shutdown_connections 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.527+0000 7fcf54a1d640 1 -- 172.21.15.133:0/3507450666 wait complete. 2024-04-25T10:17:47.545 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:34.663983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '194', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:35.974809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:47.546 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:47.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.659+0000 7fcc2eaaa640 1 Processor -- start 2024-04-25T10:17:47.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.659+0000 7fcc2eaaa640 1 -- start start 2024-04-25T10:17:47.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc2eaaa640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcc280b2150 0x7fcc2806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc2eaaa640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2815ac50 con 0x7fcc2815b9b0 2024-04-25T10:17:47.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc2eaaa640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc280cd180 con 0x7fcc28143ea0 2024-04-25T10:17:47.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc2eaaa640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc280ccf20 con 0x7fcc280b2150 2024-04-25T10:17:47.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc2d047640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcc2815b9b0 0x7fcc280b06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48398/0 (socket says 172.21.15.133:48398) 2024-04-25T10:17:47.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc2d047640 1 -- 172.21.15.133:0/2356281958 learned_addr learned my addr 172.21.15.133:0/2356281958 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:47.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc27fff640 1 --2- 172.21.15.133:0/2356281958 >> v2:172.21.15.162:3300/0 conn(0x7fcc280b2150 0x7fcc2806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942694694 0 0) 0x7fcc200014a0 con 0x7fcc2815b9b0 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc18003530 con 0x7fcc2815b9b0 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3006096067 0 0) 0x7fcc18003530 con 0x7fcc2815b9b0 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 >> v2:172.21.15.162:3300/0 conn(0x7fcc280b2150 msgr2=0x7fcc2806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 --2- 172.21.15.133:0/2356281958 >> v2:172.21.15.162:3300/0 conn(0x7fcc280b2150 0x7fcc2806e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 >> v1:172.21.15.162:6789/0 conn(0x7fcc28143ea0 legacy=0x7fcc280cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2815aed0 con 0x7fcc2815b9b0 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fcc20001bd0 con 0x7fcc2815b9b0 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 >> v1:172.21.15.133:6789/0 conn(0x7fcc2815b9b0 legacy=0x7fcc280b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 --2- 172.21.15.133:0/2356281958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc180039c0 0x7fcc18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 --2- 172.21.15.133:0/2356281958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc18004810 0x7fcc18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 --2- 172.21.15.133:0/2356281958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc180051d0 0x7fcc1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280ccf20 con 0x7fcc18004810 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1800c170 con 0x7fcc180039c0 2024-04-25T10:17:47.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1800c370 con 0x7fcc180051d0 2024-04-25T10:17:47.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc27fff640 1 --2- 172.21.15.133:0/2356281958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc18004810 0x7fcc18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc2c846640 1 --2- 172.21.15.133:0/2356281958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc180039c0 0x7fcc18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc27fff640 1 -- 172.21.15.133:0/2356281958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc180051d0 msgr2=0x7fcc1800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:47.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc27fff640 1 --2- 172.21.15.133:0/2356281958 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc180051d0 0x7fcc1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc27fff640 1 -- 172.21.15.133:0/2356281958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc180039c0 msgr2=0x7fcc18003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:47.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc27fff640 1 --2- 172.21.15.133:0/2356281958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc180039c0 0x7fcc18003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc27fff640 1 -- 172.21.15.133:0/2356281958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc18003530 con 0x7fcc18004810 2024-04-25T10:17:47.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc2d047640 1 -- 172.21.15.133:0/2356281958 reap_dead start 2024-04-25T10:17:47.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.663+0000 7fcc2c846640 1 --2- 172.21.15.133:0/2356281958 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc180039c0 0x7fcc18003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:47.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc27fff640 1 --2- 172.21.15.133:0/2356281958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc18004810 0x7fcc18004c90 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7fcc2806f700 tx=0x7fcc1401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4ea28409e692a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14009690 con 0x7fcc18004810 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc140a64c0 con 0x7fcc18004810 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc277fe640 1 -- 172.21.15.133:0/2356281958 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc140a6890 con 0x7fcc18004810 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/2356281958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc18004810 msgr2=0x7fcc18004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 --2- 172.21.15.133:0/2356281958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc18004810 0x7fcc18004c90 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7fcc2806f700 tx=0x7fcc1401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/2356281958 shutdown_connections 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 --2- 172.21.15.133:0/2356281958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc18004810 0x7fcc18004c90 unknown :-1 s=CLOSED pgs=2902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/2356281958 >> 172.21.15.133:0/2356281958 conn(0x7fcc28066f70 msgr2=0x7fcc2806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/2356281958 shutdown_connections 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/2356281958 wait complete. 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 Processor -- start 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 -- start start 2024-04-25T10:17:47.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc18004810 0x7fcc28136c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc28156700 0x7fcc281537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc28156b80 0x7fcc28153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2806ed40 con 0x7fcc28156700 2024-04-25T10:17:47.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2806ebc0 con 0x7fcc18004810 2024-04-25T10:17:47.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2eaaa640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2815e240 con 0x7fcc28156b80 2024-04-25T10:17:47.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc28156700 0x7fcc281537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc28156700 0x7fcc281537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45592/0 (socket says 172.21.15.133:45592) 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc27fff640 1 -- 172.21.15.133:0/3844248679 learned_addr learned my addr 172.21.15.133:0/3844248679 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2c846640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc18004810 0x7fcc28136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2d047640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc28156b80 0x7fcc28153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc27fff640 1 -- 172.21.15.133:0/3844248679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc28156b80 msgr2=0x7fcc28153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc27fff640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc28156b80 0x7fcc28153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc27fff640 1 -- 172.21.15.133:0/3844248679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc18004810 msgr2=0x7fcc28136c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc27fff640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc18004810 0x7fcc28136c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc27fff640 1 -- 172.21.15.133:0/3844248679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc28137150 con 0x7fcc28156700 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc2d047640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc28156b80 0x7fcc28153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc27fff640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc28156700 0x7fcc281537b0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7fcc140b0040 tx=0x7fcc14019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc257fa640 1 -- 172.21.15.133:0/3844248679 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc140a6670 con 0x7fcc28156700 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.667+0000 7fcc257fa640 1 -- 172.21.15.133:0/3844248679 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc140be070 con 0x7fcc28156700 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.671+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/3844248679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc28155660 con 0x7fcc28156700 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.671+0000 7fcc257fa640 1 -- 172.21.15.133:0/3844248679 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc140a6a50 con 0x7fcc28156700 2024-04-25T10:17:47.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.671+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/3844248679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc28155c20 con 0x7fcc28156700 2024-04-25T10:17:47.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.671+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/3844248679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc28068e80 con 0x7fcc28156700 2024-04-25T10:17:47.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.671+0000 7fcc257fa640 1 -- 172.21.15.133:0/3844248679 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc140a6bf0 con 0x7fcc28156700 2024-04-25T10:17:47.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.671+0000 7fcc257fa640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc0c066f60 0x7fcc0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.671+0000 7fcc257fa640 1 -- 172.21.15.133:0/3844248679 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc1411f890 con 0x7fcc28156700 2024-04-25T10:17:47.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.671+0000 7fcc2c846640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc0c066f60 0x7fcc0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.671+0000 7fcc2c846640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc0c066f60 0x7fcc0c069420 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000c10 tx=0x7fcc1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.675+0000 7fcc257fa640 1 -- 172.21.15.133:0/3844248679 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc28137150 con 0x7fcc28156700 2024-04-25T10:17:47.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.819+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/3844248679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc28159ea0 con 0x7fcc28156700 2024-04-25T10:17:47.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc257fa640 1 -- 172.21.15.133:0/3844248679 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v195) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc14016700 con 0x7fcc28156700 2024-04-25T10:17:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:17:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/3844248679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc0c066f60 msgr2=0x7fcc0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc0c066f60 0x7fcc0c069420 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000c10 tx=0x7fcc1c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/3844248679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc28156700 msgr2=0x7fcc281537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc28156700 0x7fcc281537b0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7fcc140b0040 tx=0x7fcc14019790 comp rx=0 tx=0).stop 2024-04-25T10:17:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/3844248679 shutdown_connections 2024-04-25T10:17:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc28156b80 0x7fcc28153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc0c066f60 0x7fcc0c069420 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc28156700 0x7fcc281537b0 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 --2- 172.21.15.133:0/3844248679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc18004810 0x7fcc28136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/3844248679 >> 172.21.15.133:0/3844248679 conn(0x7fcc28066f70 msgr2=0x7fcc280caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/3844248679 shutdown_connections 2024-04-25T10:17:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.823+0000 7fcc2eaaa640 1 -- 172.21.15.133:0/3844248679 wait complete. 2024-04-25T10:17:47.842 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:17:47.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f6635f04640 1 Processor -- start 2024-04-25T10:17:47.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f6635f04640 1 -- start start 2024-04-25T10:17:47.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f6635f04640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f66300b2150 0x7f663006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f6635f04640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663015ac50 con 0x7f6630143ea0 2024-04-25T10:17:47.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f6635f04640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66300cd180 con 0x7f663015b9b0 2024-04-25T10:17:47.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f6635f04640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66300ccf20 con 0x7f66300b2150 2024-04-25T10:17:47.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f662effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f66300b2150 0x7f663006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f662effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f66300b2150 0x7f663006e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45608/0 (socket says 172.21.15.133:45608) 2024-04-25T10:17:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f662effd640 1 -- 172.21.15.133:0/96717690 learned_addr learned my addr 172.21.15.133:0/96717690 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f662f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6630143ea0 0x7f66300cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48412/0 (socket says 172.21.15.133:48412) 2024-04-25T10:17:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f662effd640 1 -- 172.21.15.133:0/96717690 >> v1:172.21.15.162:6789/0 conn(0x7f663015b9b0 legacy=0x7f66300b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f662effd640 1 -- 172.21.15.133:0/96717690 >> v1:172.21.15.133:6789/0 conn(0x7f6630143ea0 legacy=0x7f66300cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f662e7fc640 1 -- 172.21.15.133:0/96717690 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521692209 0 0) 0x7f6620001120 con 0x7f6630143ea0 2024-04-25T10:17:47.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.955+0000 7f662effd640 1 -- 172.21.15.133:0/96717690 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663015aed0 con 0x7f66300b2150 2024-04-25T10:17:47.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f662effd640 1 --2- 172.21.15.133:0/96717690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66300b2150 0x7f663006e3b0 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f662400bb10 tx=0x7f662401eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24886dfe5ff04f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f662e7fc640 1 -- 172.21.15.133:0/96717690 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66240b4070 con 0x7f66300b2150 2024-04-25T10:17:47.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f662e7fc640 1 -- 172.21.15.133:0/96717690 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66240a7b20 con 0x7f66300b2150 2024-04-25T10:17:47.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f662e7fc640 1 -- 172.21.15.133:0/96717690 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66240af440 con 0x7f66300b2150 2024-04-25T10:17:47.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f6635f04640 1 -- 172.21.15.133:0/96717690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66300b2150 msgr2=0x7f663006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f6635f04640 1 --2- 172.21.15.133:0/96717690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66300b2150 0x7f663006e3b0 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f662400bb10 tx=0x7f662401eb60 comp rx=0 tx=0).stop 2024-04-25T10:17:47.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f6635f04640 1 -- 172.21.15.133:0/96717690 shutdown_connections 2024-04-25T10:17:47.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f6635f04640 1 --2- 172.21.15.133:0/96717690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66300b2150 0x7f663006e3b0 unknown :-1 s=CLOSED pgs=2905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f6635f04640 1 -- 172.21.15.133:0/96717690 >> 172.21.15.133:0/96717690 conn(0x7f6630066f70 msgr2=0x7f663006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:47.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f6635f04640 1 -- 172.21.15.133:0/96717690 shutdown_connections 2024-04-25T10:17:47.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f6635f04640 1 -- 172.21.15.133:0/96717690 wait complete. 2024-04-25T10:17:47.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.959+0000 7f6635f04640 1 Processor -- start 2024-04-25T10:17:47.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6635f04640 1 -- start start 2024-04-25T10:17:47.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6635f04640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66300b2150 0x7f6630139a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6635f04640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6630143ea0 0x7f6630165570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6635f04640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f663015b9b0 0x7f66300b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6635f04640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663006d630 con 0x7f663015b9b0 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6635f04640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663006d4b0 con 0x7f6630143ea0 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6635f04640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663006d7b0 con 0x7f66300b2150 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f662ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f663015b9b0 0x7f66300b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f662ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f663015b9b0 0x7f66300b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45612/0 (socket says 172.21.15.133:45612) 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f662ffff640 1 -- 172.21.15.133:0/3776041018 learned_addr learned my addr 172.21.15.133:0/3776041018 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f662effd640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6630143ea0 0x7f6630165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f662f7fe640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66300b2150 0x7f6630139a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f662ffff640 1 -- 172.21.15.133:0/3776041018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66300b2150 msgr2=0x7f6630139a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f662ffff640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66300b2150 0x7f6630139a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f662ffff640 1 -- 172.21.15.133:0/3776041018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6630143ea0 msgr2=0x7f6630165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f662ffff640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6630143ea0 0x7f6630165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f662ffff640 1 -- 172.21.15.133:0/3776041018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6630165ab0 con 0x7f663015b9b0 2024-04-25T10:17:47.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f662ffff640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f663015b9b0 0x7f66300b3460 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f662800ba10 tx=0x7f662800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6617fff640 1 -- 172.21.15.133:0/3776041018 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628019070 con 0x7f663015b9b0 2024-04-25T10:17:47.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6617fff640 1 -- 172.21.15.133:0/3776041018 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662800cb40 con 0x7f663015b9b0 2024-04-25T10:17:47.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6617fff640 1 -- 172.21.15.133:0/3776041018 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628014440 con 0x7f663015b9b0 2024-04-25T10:17:47.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6635f04640 1 -- 172.21.15.133:0/3776041018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663013e0f0 con 0x7f663015b9b0 2024-04-25T10:17:47.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6635f04640 1 -- 172.21.15.133:0/3776041018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663013e6b0 con 0x7f663015b9b0 2024-04-25T10:17:47.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6635f04640 1 -- 172.21.15.133:0/3776041018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6630068e80 con 0x7f663015b9b0 2024-04-25T10:17:47.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6617fff640 1 -- 172.21.15.133:0/3776041018 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f662801c070 con 0x7f663015b9b0 2024-04-25T10:17:47.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.963+0000 7f6617fff640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f661c066f60 0x7f661c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.967+0000 7f6617fff640 1 -- 172.21.15.133:0/3776041018 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6628084e50 con 0x7f663015b9b0 2024-04-25T10:17:47.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.967+0000 7f662f7fe640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f661c066f60 0x7f661c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.967+0000 7f662f7fe640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f661c066f60 0x7f661c069420 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6620007fd0 tx=0x7f6620007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:47.967+0000 7f6617fff640 1 -- 172.21.15.133:0/3776041018 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6630165ab0 con 0x7f663015b9b0 2024-04-25T10:17:48.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.115+0000 7f6635f04640 1 -- 172.21.15.133:0/3776041018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66300cbd00 con 0x7f663015b9b0 2024-04-25T10:17:48.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.115+0000 7f6617fff640 1 -- 172.21.15.133:0/3776041018 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v195) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6628055e40 con 0x7f663015b9b0 2024-04-25T10:17:48.119 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:17:48.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 -- 172.21.15.133:0/3776041018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f661c066f60 msgr2=0x7f661c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f661c066f60 0x7f661c069420 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6620007fd0 tx=0x7f6620007980 comp rx=0 tx=0).stop 2024-04-25T10:17:48.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 -- 172.21.15.133:0/3776041018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f663015b9b0 msgr2=0x7f66300b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f663015b9b0 0x7f66300b3460 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f662800ba10 tx=0x7f662800bed0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 -- 172.21.15.133:0/3776041018 shutdown_connections 2024-04-25T10:17:48.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f661c066f60 0x7f661c069420 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f663015b9b0 0x7f66300b3460 unknown :-1 s=CLOSED pgs=2906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6630143ea0 0x7f6630165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 --2- 172.21.15.133:0/3776041018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f66300b2150 0x7f6630139a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 -- 172.21.15.133:0/3776041018 >> 172.21.15.133:0/3776041018 conn(0x7f6630066f70 msgr2=0x7f6630140410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 -- 172.21.15.133:0/3776041018 shutdown_connections 2024-04-25T10:17:48.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.119+0000 7f6635f04640 1 -- 172.21.15.133:0/3776041018 wait complete. 2024-04-25T10:17:48.136 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:17:48.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4460d9d640 1 Processor -- start 2024-04-25T10:17:48.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4460d9d640 1 -- start start 2024-04-25T10:17:48.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4460d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f445c0b2150 0x7f445c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4460d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f445c15b9b0 0x7f445c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4460d9d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445c15ac50 con 0x7f445c143ea0 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4460d9d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445c0ccf20 con 0x7f445c0b2150 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4460d9d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445c0cd180 con 0x7f445c15b9b0 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4459d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f445c0b2150 0x7f445c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4459d9b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f445c0b2150 0x7f445c06e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45614/0 (socket says 172.21.15.133:45614) 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4459d9b640 1 -- 172.21.15.133:0/3048882883 learned_addr learned my addr 172.21.15.133:0/3048882883 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4459d9b640 1 -- 172.21.15.133:0/3048882883 >> v2:172.21.15.162:3300/0 conn(0x7f445c15b9b0 msgr2=0x7f445c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4459d9b640 1 --2- 172.21.15.133:0/3048882883 >> v2:172.21.15.162:3300/0 conn(0x7f445c15b9b0 0x7f445c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4459d9b640 1 -- 172.21.15.133:0/3048882883 >> v1:172.21.15.133:6789/0 conn(0x7f445c143ea0 legacy=0x7f445c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f4459d9b640 1 -- 172.21.15.133:0/3048882883 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c15aed0 con 0x7f445c0b2150 2024-04-25T10:17:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.255+0000 7f445959a640 1 -- 172.21.15.133:0/3048882883 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912473451 0 0) 0x7f444c001120 con 0x7f445c143ea0 2024-04-25T10:17:48.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4459d9b640 1 --2- 172.21.15.133:0/3048882883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0b2150 0x7f445c06e3b0 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f445c0cd7f0 tx=0x7f445001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cfae9f0c647cf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445959a640 1 -- 172.21.15.133:0/3048882883 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44500b4070 con 0x7f445c0b2150 2024-04-25T10:17:48.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445959a640 1 -- 172.21.15.133:0/3048882883 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44500a7aa0 con 0x7f445c0b2150 2024-04-25T10:17:48.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445959a640 1 -- 172.21.15.133:0/3048882883 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44500af440 con 0x7f445c0b2150 2024-04-25T10:17:48.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 -- 172.21.15.133:0/3048882883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0b2150 msgr2=0x7f445c06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 --2- 172.21.15.133:0/3048882883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0b2150 0x7f445c06e3b0 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f445c0cd7f0 tx=0x7f445001eb60 comp rx=0 tx=0).stop 2024-04-25T10:17:48.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 -- 172.21.15.133:0/3048882883 shutdown_connections 2024-04-25T10:17:48.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 --2- 172.21.15.133:0/3048882883 >> v2:172.21.15.162:3300/0 conn(0x7f445c15b9b0 0x7f445c06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 --2- 172.21.15.133:0/3048882883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0b2150 0x7f445c06e3b0 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 -- 172.21.15.133:0/3048882883 >> 172.21.15.133:0/3048882883 conn(0x7f445c066f70 msgr2=0x7f445c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 -- 172.21.15.133:0/3048882883 shutdown_connections 2024-04-25T10:17:48.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 -- 172.21.15.133:0/3048882883 wait complete. 2024-04-25T10:17:48.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 Processor -- start 2024-04-25T10:17:48.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 -- start start 2024-04-25T10:17:48.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f445c0b2150 0x7f445c13fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f445c143ea0 0x7f445c0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c15b9b0 0x7f445c0c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0b1270 con 0x7f445c15b9b0 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0b10f0 con 0x7f445c0b2150 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0b13f0 con 0x7f445c143ea0 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c15b9b0 0x7f445c0c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c15b9b0 0x7f445c0c4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45624/0 (socket says 172.21.15.133:45624) 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445ad9d640 1 -- 172.21.15.133:0/1739584091 learned_addr learned my addr 172.21.15.133:0/1739584091 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4459d9b640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f445c143ea0 0x7f445c0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445a59c640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f445c0b2150 0x7f445c13fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445ad9d640 1 -- 172.21.15.133:0/1739584091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f445c143ea0 msgr2=0x7f445c0b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445ad9d640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f445c143ea0 0x7f445c0b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445ad9d640 1 -- 172.21.15.133:0/1739584091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f445c0b2150 msgr2=0x7f445c13fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445ad9d640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f445c0b2150 0x7f445c13fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445ad9d640 1 -- 172.21.15.133:0/1739584091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c12e470 con 0x7f445c15b9b0 2024-04-25T10:17:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4459d9b640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f445c143ea0 0x7f445c0b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445a59c640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f445c0b2150 0x7f445c13fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f445ad9d640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c15b9b0 0x7f445c0c4930 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f445400b9e0 tx=0x7f445400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f444affd640 1 -- 172.21.15.133:0/1739584091 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454019070 con 0x7f445c15b9b0 2024-04-25T10:17:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f444affd640 1 -- 172.21.15.133:0/1739584091 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445400cb40 con 0x7f445c15b9b0 2024-04-25T10:17:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 -- 172.21.15.133:0/1739584091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c12e750 con 0x7f445c15b9b0 2024-04-25T10:17:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f444affd640 1 -- 172.21.15.133:0/1739584091 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454014440 con 0x7f445c15b9b0 2024-04-25T10:17:48.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.259+0000 7f4460d9d640 1 -- 172.21.15.133:0/1739584091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c12ed10 con 0x7f445c15b9b0 2024-04-25T10:17:48.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.263+0000 7f4460d9d640 1 -- 172.21.15.133:0/1739584091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445c068e80 con 0x7f445c15b9b0 2024-04-25T10:17:48.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.263+0000 7f444affd640 1 -- 172.21.15.133:0/1739584091 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f445401c070 con 0x7f445c15b9b0 2024-04-25T10:17:48.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.263+0000 7f444affd640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f443c066f60 0x7f443c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.263+0000 7f444affd640 1 -- 172.21.15.133:0/1739584091 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4454084e70 con 0x7f445c15b9b0 2024-04-25T10:17:48.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.263+0000 7f445a59c640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f443c066f60 0x7f443c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.263+0000 7f445a59c640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f443c066f60 0x7f443c069420 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f444c000f70 tx=0x7f444c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.267+0000 7f444affd640 1 -- 172.21.15.133:0/1739584091 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4454055cf0 con 0x7f445c15b9b0 2024-04-25T10:17:48.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.423+0000 7f4460d9d640 1 -- 172.21.15.133:0/1739584091 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f445c0cca20 con 0x7f445c15b9b0 2024-04-25T10:17:48.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.423+0000 7f444affd640 1 -- 172.21.15.133:0/1739584091 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v195) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44540555f0 con 0x7f445c15b9b0 2024-04-25T10:17:48.427 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 -- 172.21.15.133:0/1739584091 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f443c066f60 msgr2=0x7f443c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f443c066f60 0x7f443c069420 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f444c000f70 tx=0x7f444c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 -- 172.21.15.133:0/1739584091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c15b9b0 msgr2=0x7f445c0c4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c15b9b0 0x7f445c0c4930 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f445400b9e0 tx=0x7f445400bea0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 -- 172.21.15.133:0/1739584091 shutdown_connections 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f443c066f60 0x7f443c069420 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c15b9b0 0x7f445c0c4930 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f445c143ea0 0x7f445c0b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 --2- 172.21.15.133:0/1739584091 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f445c0b2150 0x7f445c13fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 -- 172.21.15.133:0/1739584091 >> 172.21.15.133:0/1739584091 conn(0x7f445c066f70 msgr2=0x7f445c15e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 -- 172.21.15.133:0/1739584091 shutdown_connections 2024-04-25T10:17:48.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.427+0000 7f4460d9d640 1 -- 172.21.15.133:0/1739584091 wait complete. 2024-04-25T10:17:48.446 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:17:48.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.551+0000 7f985479d640 1 Processor -- start 2024-04-25T10:17:48.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.551+0000 7f985479d640 1 -- start start 2024-04-25T10:17:48.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f984c0ce3b0 0x7f984c0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f984c0ce790 0x7f984c0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f984c0b1900 con 0x7f984c06c7b0 2024-04-25T10:17:48.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984c0b2510 con 0x7f984c0ce3b0 2024-04-25T10:17:48.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984c0b1be0 con 0x7f984c0ce790 2024-04-25T10:17:48.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9852539640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f984c0ce3b0 0x7f984c0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9852539640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f984c0ce3b0 0x7f984c0b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45630/0 (socket says 172.21.15.133:45630) 2024-04-25T10:17:48.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9852539640 1 -- 172.21.15.133:0/4198916836 learned_addr learned my addr 172.21.15.133:0/4198916836 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:48.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9851d38640 1 --2- 172.21.15.133:0/4198916836 >> v2:172.21.15.162:3300/0 conn(0x7f984c0ce790 0x7f984c0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9852539640 1 -- 172.21.15.133:0/4198916836 >> v2:172.21.15.162:3300/0 conn(0x7f984c0ce790 msgr2=0x7f984c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9852539640 1 --2- 172.21.15.133:0/4198916836 >> v2:172.21.15.162:3300/0 conn(0x7f984c0ce790 0x7f984c0cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9852539640 1 -- 172.21.15.133:0/4198916836 >> v1:172.21.15.162:6789/0 conn(0x7f984c06c7b0 legacy=0x7f984c15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9852539640 1 -- 172.21.15.133:0/4198916836 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c0b1ec0 con 0x7f984c0ce3b0 2024-04-25T10:17:48.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9851d38640 1 --2- 172.21.15.133:0/4198916836 >> v2:172.21.15.162:3300/0 conn(0x7f984c0ce790 0x7f984c0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:48.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9852539640 1 --2- 172.21.15.133:0/4198916836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f984c0ce3b0 0x7f984c0b1320 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f984c0b10d0 tx=0x7f984801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873e3d6af16dd833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9851537640 1 -- 172.21.15.133:0/4198916836 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98480b0070 con 0x7f984c0ce3b0 2024-04-25T10:17:48.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9851537640 1 -- 172.21.15.133:0/4198916836 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98480ab600 con 0x7f984c0ce3b0 2024-04-25T10:17:48.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f9851537640 1 -- 172.21.15.133:0/4198916836 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98480ab9d0 con 0x7f984c0ce3b0 2024-04-25T10:17:48.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 -- 172.21.15.133:0/4198916836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f984c0ce3b0 msgr2=0x7f984c0b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 --2- 172.21.15.133:0/4198916836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f984c0ce3b0 0x7f984c0b1320 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f984c0b10d0 tx=0x7f984801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 -- 172.21.15.133:0/4198916836 shutdown_connections 2024-04-25T10:17:48.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 --2- 172.21.15.133:0/4198916836 >> v2:172.21.15.162:3300/0 conn(0x7f984c0ce790 0x7f984c0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 --2- 172.21.15.133:0/4198916836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f984c0ce3b0 0x7f984c0b1320 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 -- 172.21.15.133:0/4198916836 >> 172.21.15.133:0/4198916836 conn(0x7f984c08af60 msgr2=0x7f984c08b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 -- 172.21.15.133:0/4198916836 shutdown_connections 2024-04-25T10:17:48.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.555+0000 7f985479d640 1 -- 172.21.15.133:0/4198916836 wait complete. 2024-04-25T10:17:48.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f985479d640 1 Processor -- start 2024-04-25T10:17:48.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f985479d640 1 -- start start 2024-04-25T10:17:48.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f985479d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f984c06c7b0 0x7f984c150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f985479d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f984c0ce3b0 0x7f984c14ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f985479d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f984c0ce790 0x7f984c15ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f985479d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c0cc760 con 0x7f984c0ce790 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f985479d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c0cc5e0 con 0x7f984c0ce3b0 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f985479d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c0cc8e0 con 0x7f984c06c7b0 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9852d3a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f984c0ce790 0x7f984c15ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9852d3a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f984c0ce790 0x7f984c15ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45642/0 (socket says 172.21.15.133:45642) 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9852d3a640 1 -- 172.21.15.133:0/1007413637 learned_addr learned my addr 172.21.15.133:0/1007413637 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9851d38640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f984c0ce3b0 0x7f984c14ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9852539640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f984c06c7b0 0x7f984c150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9852d3a640 1 -- 172.21.15.133:0/1007413637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f984c06c7b0 msgr2=0x7f984c150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9852d3a640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f984c06c7b0 0x7f984c150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9852d3a640 1 -- 172.21.15.133:0/1007413637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f984c0ce3b0 msgr2=0x7f984c14ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9852d3a640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f984c0ce3b0 0x7f984c14ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9852d3a640 1 -- 172.21.15.133:0/1007413637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c15ab10 con 0x7f984c0ce790 2024-04-25T10:17:48.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9851d38640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f984c0ce3b0 0x7f984c14ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:48.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9852d3a640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f984c0ce790 0x7f984c15ca60 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f984400c780 tx=0x7f984400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9842ffd640 1 -- 172.21.15.133:0/1007413637 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844004b70 con 0x7f984c0ce790 2024-04-25T10:17:48.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9842ffd640 1 -- 172.21.15.133:0/1007413637 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9844004d10 con 0x7f984c0ce790 2024-04-25T10:17:48.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f9842ffd640 1 -- 172.21.15.133:0/1007413637 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984400aa90 con 0x7f984c0ce790 2024-04-25T10:17:48.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f985479d640 1 -- 172.21.15.133:0/1007413637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c14ffe0 con 0x7f984c0ce790 2024-04-25T10:17:48.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f985479d640 1 -- 172.21.15.133:0/1007413637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c1510a0 con 0x7f984c0ce790 2024-04-25T10:17:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.559+0000 7f985479d640 1 -- 172.21.15.133:0/1007413637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984c067c70 con 0x7f984c0ce790 2024-04-25T10:17:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.563+0000 7f9842ffd640 1 -- 172.21.15.133:0/1007413637 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f984400ac30 con 0x7f984c0ce790 2024-04-25T10:17:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.563+0000 7f9842ffd640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9830066f60 0x7f9830069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.563+0000 7f9842ffd640 1 -- 172.21.15.133:0/1007413637 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9844085480 con 0x7f984c0ce790 2024-04-25T10:17:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.563+0000 7f9852539640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9830066f60 0x7f9830069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.563+0000 7f9852539640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9830066f60 0x7f9830069420 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f98480ad000 tx=0x7f9848008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.567+0000 7f9842ffd640 1 -- 172.21.15.133:0/1007413637 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9844020070 con 0x7f984c0ce790 2024-04-25T10:17:48.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 -- 172.21.15.133:0/1007413637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f984c06c590 con 0x7f984c0ce790 2024-04-25T10:17:48.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f9842ffd640 1 -- 172.21.15.133:0/1007413637 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v195) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9844056300 con 0x7f984c0ce790 2024-04-25T10:17:48.725 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-04-25T10:17:48.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 -- 172.21.15.133:0/1007413637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9830066f60 msgr2=0x7f9830069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9830066f60 0x7f9830069420 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f98480ad000 tx=0x7f9848008040 comp rx=0 tx=0).stop 2024-04-25T10:17:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 -- 172.21.15.133:0/1007413637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f984c0ce790 msgr2=0x7f984c15ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f984c0ce790 0x7f984c15ca60 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f984400c780 tx=0x7f984400cc40 comp rx=0 tx=0).stop 2024-04-25T10:17:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 -- 172.21.15.133:0/1007413637 shutdown_connections 2024-04-25T10:17:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9830066f60 0x7f9830069420 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f984c0ce790 0x7f984c15ca60 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f984c0ce3b0 0x7f984c14ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 --2- 172.21.15.133:0/1007413637 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f984c06c7b0 0x7f984c150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 -- 172.21.15.133:0/1007413637 >> 172.21.15.133:0/1007413637 conn(0x7f984c08af60 msgr2=0x7f984c0d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 -- 172.21.15.133:0/1007413637 shutdown_connections 2024-04-25T10:17:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:48.723+0000 7f985479d640 1 -- 172.21.15.133:0/1007413637 wait complete. 2024-04-25T10:17:48.743 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-04-25T10:17:48.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:48.743 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:48.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[92467]: fuse finished with error 0 and tester_r 0 2024-04-25T10:17:48.754 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:54.300 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:54.303 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:54.856 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:17:54.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:54.856 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:17:54.864 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:55.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.031+0000 7fc5ac274640 1 Processor -- start 2024-04-25T10:17:55.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.031+0000 7fc5ac274640 1 -- start start 2024-04-25T10:17:55.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5ac274640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fc5a40b2a10 0x7fc5a40704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5ac274640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a40b3d30 con 0x7fc5a4072e40 2024-04-25T10:17:55.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5ac274640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a415de20 con 0x7fc5a415da40 2024-04-25T10:17:55.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5ac274640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a4070300 con 0x7fc5a40b2a10 2024-04-25T10:17:55.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5aa811640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc5a4072e40 0x7fc5a4161550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48440/0 (socket says 172.21.15.133:48440) 2024-04-25T10:17:55.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5aa811640 1 -- 172.21.15.133:0/2075233007 learned_addr learned my addr 172.21.15.133:0/2075233007 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:55.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a980f640 1 --2- 172.21.15.133:0/2075233007 >> v2:172.21.15.162:3300/0 conn(0x7fc5a40b2a10 0x7fc5a40704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043794407 0 0) 0x7fc5a40b3d30 con 0x7fc5a4072e40 2024-04-25T10:17:55.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc590003530 con 0x7fc5a4072e40 2024-04-25T10:17:55.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2592939396 0 0) 0x7fc590003530 con 0x7fc5a4072e40 2024-04-25T10:17:55.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 >> v2:172.21.15.162:3300/0 conn(0x7fc5a40b2a10 msgr2=0x7fc5a40704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:55.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 --2- 172.21.15.133:0/2075233007 >> v2:172.21.15.162:3300/0 conn(0x7fc5a40b2a10 0x7fc5a40704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 >> v1:172.21.15.162:6789/0 conn(0x7fc5a415da40 legacy=0x7fc5a40b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a406fd40 con 0x7fc5a4072e40 2024-04-25T10:17:55.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fc59c001a00 con 0x7fc5a4072e40 2024-04-25T10:17:55.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 >> v1:172.21.15.133:6789/0 conn(0x7fc5a4072e40 legacy=0x7fc5a4161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 --2- 172.21.15.133:0/2075233007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5900039c0 0x7fc590003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 --2- 172.21.15.133:0/2075233007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5900043c0 0x7fc590004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 --2- 172.21.15.133:0/2075233007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc590005130 0x7fc59000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59000c040 con 0x7fc5900039c0 2024-04-25T10:17:55.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5aa010640 1 --2- 172.21.15.133:0/2075233007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5900039c0 0x7fc590003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59000c1f0 con 0x7fc5900043c0 2024-04-25T10:17:55.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59000c3f0 con 0x7fc590005130 2024-04-25T10:17:55.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5aa010640 1 -- 172.21.15.133:0/2075233007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc590005130 msgr2=0x7fc59000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:55.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5aa010640 1 --2- 172.21.15.133:0/2075233007 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc590005130 0x7fc59000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5a980f640 1 --2- 172.21.15.133:0/2075233007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5900043c0 0x7fc590004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5aa010640 1 -- 172.21.15.133:0/2075233007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5900043c0 msgr2=0x7fc590004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:55.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5aa010640 1 --2- 172.21.15.133:0/2075233007 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5900043c0 0x7fc590004890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5aa811640 1 -- 172.21.15.133:0/2075233007 reap_dead start 2024-04-25T10:17:55.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.035+0000 7fc5aa010640 1 -- 172.21.15.133:0/2075233007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc590003530 con 0x7fc5900039c0 2024-04-25T10:17:55.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa010640 1 --2- 172.21.15.133:0/2075233007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5900039c0 0x7fc590003da0 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001780 tx=0x7fc5a001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1434b449cf7e51c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00013a0 con 0x7fc5900039c0 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a00052f0 con 0x7fc5900039c0 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5a900e640 1 -- 172.21.15.133:0/2075233007 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0003840 con 0x7fc5900039c0 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 -- 172.21.15.133:0/2075233007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5900039c0 msgr2=0x7fc590003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 --2- 172.21.15.133:0/2075233007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5900039c0 0x7fc590003da0 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001780 tx=0x7fc5a001c420 comp rx=0 tx=0).stop 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 -- 172.21.15.133:0/2075233007 shutdown_connections 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 --2- 172.21.15.133:0/2075233007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5900039c0 0x7fc590003da0 unknown :-1 s=CLOSED pgs=2913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 -- 172.21.15.133:0/2075233007 >> 172.21.15.133:0/2075233007 conn(0x7fc5a406e440 msgr2=0x7fc5a406e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 -- 172.21.15.133:0/2075233007 shutdown_connections 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 -- 172.21.15.133:0/2075233007 wait complete. 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 Processor -- start 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 -- start start 2024-04-25T10:17:55.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5900039c0 0x7fc5a4148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5a40cb450 0x7fc5a4157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a40cb8d0 0x7fc5a4158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40b2710 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40cd410 con 0x7fc5a40cb450 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5ac274640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40b2890 con 0x7fc5900039c0 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa811640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a40cb8d0 0x7fc5a4158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa811640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a40cb8d0 0x7fc5a4158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45660/0 (socket says 172.21.15.133:45660) 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa811640 1 -- 172.21.15.133:0/2803600325 learned_addr learned my addr 172.21.15.133:0/2803600325 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa811640 1 -- 172.21.15.133:0/2803600325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5900039c0 msgr2=0x7fc5a4148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa010640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5900039c0 0x7fc5a4148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5a980f640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5a40cb450 0x7fc5a4157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa811640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5900039c0 0x7fc5a4148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa811640 1 -- 172.21.15.133:0/2803600325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5a40cb450 msgr2=0x7fc5a4157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa811640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5a40cb450 0x7fc5a4157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa811640 1 -- 172.21.15.133:0/2803600325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40cc380 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa010640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5900039c0 0x7fc5a4148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:55.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc5aa811640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a40cb8d0 0x7fc5a4158380 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4071fa0 tx=0x7fc59c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.039+0000 7fc59affd640 1 -- 172.21.15.133:0/2803600325 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00aae0 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.043+0000 7fc59affd640 1 -- 172.21.15.133:0/2803600325 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c00aca0 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.043+0000 7fc5ac274640 1 -- 172.21.15.133:0/2803600325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a40cbdc0 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.043+0000 7fc59affd640 1 -- 172.21.15.133:0/2803600325 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c014660 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.043+0000 7fc5ac274640 1 -- 172.21.15.133:0/2803600325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a415eda0 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.043+0000 7fc5ac274640 1 -- 172.21.15.133:0/2803600325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a4067c70 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.043+0000 7fc59affd640 1 -- 172.21.15.133:0/2803600325 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc59c011070 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.043+0000 7fc59affd640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc588066f60 0x7fc588069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.043+0000 7fc59affd640 1 -- 172.21.15.133:0/2803600325 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc59c084d00 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.043+0000 7fc5aa010640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc588066f60 0x7fc588069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.043+0000 7fc5aa010640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc588066f60 0x7fc588069420 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001750 tx=0x7fc5a00011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.047+0000 7fc59affd640 1 -- 172.21.15.133:0/2803600325 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc59c01a060 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.239+0000 7fc5ac274640 1 -- 172.21.15.133:0/2803600325 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5a4072710 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.239+0000 7fc59affd640 1 -- 172.21.15.133:0/2803600325 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4961 (secure 0 0 0) 0x7fc59c055c10 con 0x7fc5a40cb8d0 2024-04-25T10:17:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:55.244 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:36.969368+0000","modified":"2024-04-25T10:17:38.999991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11874},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11874":{"gid":11874,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/1916780253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":1916780253},{"type":"v1","addr":"172.21.15.133:6837","nonce":1916780253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:17:55.244 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 118 2024-04-25T10:17:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 -- 172.21.15.133:0/2803600325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc588066f60 msgr2=0x7fc588069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc588066f60 0x7fc588069420 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001750 tx=0x7fc5a00011a0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 -- 172.21.15.133:0/2803600325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a40cb8d0 msgr2=0x7fc5a4158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a40cb8d0 0x7fc5a4158380 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4071fa0 tx=0x7fc59c00cf00 comp rx=0 tx=0).stop 2024-04-25T10:17:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 -- 172.21.15.133:0/2803600325 shutdown_connections 2024-04-25T10:17:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fc588066f60 0x7fc588069420 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a40cb8d0 0x7fc5a4158380 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc5a40cb450 0x7fc5a4157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 --2- 172.21.15.133:0/2803600325 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fc5900039c0 0x7fc5a4148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 -- 172.21.15.133:0/2803600325 >> 172.21.15.133:0/2803600325 conn(0x7fc5a406e440 msgr2=0x7fc5a415c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 -- 172.21.15.133:0/2803600325 shutdown_connections 2024-04-25T10:17:55.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.243+0000 7fc5ac274640 1 -- 172.21.15.133:0/2803600325 wait complete. 2024-04-25T10:17:55.263 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:17:55.413 INFO:teuthology.orchestra.run.smithi133.stdout:{"message":"","return_code":0} 2024-04-25T10:17:55.413 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:17:55.413 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:17:55.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cba638640 1 Processor -- start 2024-04-25T10:17:55.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cba638640 1 -- start start 2024-04-25T10:17:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cba638640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6cb4161ec0 0x7f6cb40b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cba638640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb4166460 con 0x7f6cb406e530 2024-04-25T10:17:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cba638640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb4166700 con 0x7f6cb40c34f0 2024-04-25T10:17:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cba638640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb40b2c80 con 0x7f6cb4161ec0 2024-04-25T10:17:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cb3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6cb4161ec0 0x7f6cb40b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cb37fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6cb406e530 0x7f6cb406f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:48452/0 (socket says 172.21.15.133:48452) 2024-04-25T10:17:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cb37fe640 1 -- 172.21.15.133:0/2651587126 learned_addr learned my addr 172.21.15.133:0/2651587126 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cb3fff640 1 -- 172.21.15.133:0/2651587126 >> v1:172.21.15.162:6789/0 conn(0x7f6cb40c34f0 legacy=0x7f6cb40c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cb3fff640 1 -- 172.21.15.133:0/2651587126 >> v1:172.21.15.133:6789/0 conn(0x7f6cb406e530 legacy=0x7f6cb406f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cb3fff640 1 -- 172.21.15.133:0/2651587126 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb406ead0 con 0x7f6cb4161ec0 2024-04-25T10:17:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.535+0000 7f6cb2ffd640 1 -- 172.21.15.133:0/2651587126 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466968179 0 0) 0x7f6cb4166460 con 0x7f6cb406e530 2024-04-25T10:17:55.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb3fff640 1 --2- 172.21.15.133:0/2651587126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cb4161ec0 0x7f6cb40b2740 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40097c0 tx=0x7f6ca401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35d388a6ce4f2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb2ffd640 1 -- 172.21.15.133:0/2651587126 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40b0070 con 0x7f6cb4161ec0 2024-04-25T10:17:55.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb2ffd640 1 -- 172.21.15.133:0/2651587126 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca40ab600 con 0x7f6cb4161ec0 2024-04-25T10:17:55.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb2ffd640 1 -- 172.21.15.133:0/2651587126 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40ab9d0 con 0x7f6cb4161ec0 2024-04-25T10:17:55.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 -- 172.21.15.133:0/2651587126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cb4161ec0 msgr2=0x7f6cb40b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 --2- 172.21.15.133:0/2651587126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cb4161ec0 0x7f6cb40b2740 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40097c0 tx=0x7f6ca401c3d0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 -- 172.21.15.133:0/2651587126 shutdown_connections 2024-04-25T10:17:55.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 --2- 172.21.15.133:0/2651587126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cb4161ec0 0x7f6cb40b2740 unknown :-1 s=CLOSED pgs=2916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 -- 172.21.15.133:0/2651587126 >> 172.21.15.133:0/2651587126 conn(0x7f6cb40999d0 msgr2=0x7f6cb40b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 -- 172.21.15.133:0/2651587126 shutdown_connections 2024-04-25T10:17:55.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 -- 172.21.15.133:0/2651587126 wait complete. 2024-04-25T10:17:55.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 Processor -- start 2024-04-25T10:17:55.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 -- start start 2024-04-25T10:17:55.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cb406e530 0x7f6cb40c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cb40c34f0 0x7f6cb4155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cb4161ec0 0x7f6cb4156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40cf600 con 0x7f6cb406e530 2024-04-25T10:17:55.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40cf480 con 0x7f6cb4161ec0 2024-04-25T10:17:55.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cba638640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40cf780 con 0x7f6cb40c34f0 2024-04-25T10:17:55.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cb406e530 0x7f6cb40c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cb406e530 0x7f6cb40c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45680/0 (socket says 172.21.15.133:45680) 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb3fff640 1 -- 172.21.15.133:0/3488779582 learned_addr learned my addr 172.21.15.133:0/3488779582 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb8bd5640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cb4161ec0 0x7f6cb4156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb37fe640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cb40c34f0 0x7f6cb4155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb3fff640 1 -- 172.21.15.133:0/3488779582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cb40c34f0 msgr2=0x7f6cb4155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb3fff640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cb40c34f0 0x7f6cb4155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb3fff640 1 -- 172.21.15.133:0/3488779582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cb4161ec0 msgr2=0x7f6cb4156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb3fff640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cb4161ec0 0x7f6cb4156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.539+0000 7f6cb3fff640 1 -- 172.21.15.133:0/3488779582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40b42c0 con 0x7f6cb406e530 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cb3fff640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cb406e530 0x7f6cb40c9a20 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40ae040 tx=0x7f6ca401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cb0ff9640 1 -- 172.21.15.133:0/3488779582 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40b0050 con 0x7f6cb406e530 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cb0ff9640 1 -- 172.21.15.133:0/3488779582 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca40bc070 con 0x7f6cb406e530 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cba638640 1 -- 172.21.15.133:0/3488779582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb41582a0 con 0x7f6cb406e530 2024-04-25T10:17:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cb0ff9640 1 -- 172.21.15.133:0/3488779582 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40b5870 con 0x7f6cb406e530 2024-04-25T10:17:55.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cba638640 1 -- 172.21.15.133:0/3488779582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb40b4840 con 0x7f6cb406e530 2024-04-25T10:17:55.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cba638640 1 -- 172.21.15.133:0/3488779582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb4067c70 con 0x7f6cb406e530 2024-04-25T10:17:55.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cb0ff9640 1 -- 172.21.15.133:0/3488779582 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ca40b5a10 con 0x7f6cb406e530 2024-04-25T10:17:55.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cb0ff9640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ca0066f60 0x7f6ca0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cb0ff9640 1 -- 172.21.15.133:0/3488779582 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ca411fa10 con 0x7f6cb406e530 2024-04-25T10:17:55.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cb37fe640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ca0066f60 0x7f6ca0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.543+0000 7f6cb37fe640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ca0066f60 0x7f6ca0069420 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6cb415a650 tx=0x7f6ca8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.547+0000 7f6cb0ff9640 1 -- 172.21.15.133:0/3488779582 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ca4016050 con 0x7f6cb406e530 2024-04-25T10:17:55.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.739+0000 7f6cba638640 1 -- 172.21.15.133:0/3488779582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6cb40b64a0 con 0x7f6cb406e530 2024-04-25T10:17:55.834 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:17:55.743+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.743+0000 7f6cb0ff9640 1 -- 172.21.15.133:0/3488779582 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v119) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6ca40f0890 con 0x7f6cb406e530 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 -- 172.21.15.133:0/3488779582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ca0066f60 msgr2=0x7f6ca0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ca0066f60 0x7f6ca0069420 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6cb415a650 tx=0x7f6ca8009040 comp rx=0 tx=0).stop 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 -- 172.21.15.133:0/3488779582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cb406e530 msgr2=0x7f6cb40c9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cb406e530 0x7f6cb40c9a20 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40ae040 tx=0x7f6ca401ac10 comp rx=0 tx=0).stop 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 -- 172.21.15.133:0/3488779582 shutdown_connections 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6cb4161ec0 0x7f6cb4156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6cb40c34f0 0x7f6cb4155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6ca0066f60 0x7f6ca0069420 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 --2- 172.21.15.133:0/3488779582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cb406e530 0x7f6cb40c9a20 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 -- 172.21.15.133:0/3488779582 >> 172.21.15.133:0/3488779582 conn(0x7f6cb40999d0 msgr2=0x7f6cb40ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 -- 172.21.15.133:0/3488779582 shutdown_connections 2024-04-25T10:17:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.747+0000 7f6cba638640 1 -- 172.21.15.133:0/3488779582 wait complete. 2024-04-25T10:17:55.837 INFO:tasks.ceph.mds.c.smithi133.stderr: -4482> 2024-04-25T10:17:42.579+0000 7f72158f3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:17:55.838 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapval 10000000000.00000000 parentfile_head - 2024-04-25T10:17:55.838 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:17:55.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.867+0000 7fdc738e0080 1 Processor -- start 2024-04-25T10:17:55.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.867+0000 7fdc738e0080 1 -- start start 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba317220 0x558dba3bb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558dba330610 0x558dba3bee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558dba3309f0 0x558dba3c1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dba2e94a0 con 0x558dba317220 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dba28d9f0 con 0x558dba330610 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dba2e9620 con 0x558dba3309f0 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc713fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba317220 0x558dba3bb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc713fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba317220 0x558dba3bb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45712/0 (socket says 172.21.15.133:45712) 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc713fe640 1 -- 172.21.15.133:0/2784534520 learned_addr learned my addr 172.21.15.133:0/2784534520 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc71bff640 1 --2- 172.21.15.133:0/2784534520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558dba3309f0 0x558dba3c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc70bfd640 1 --2- 172.21.15.133:0/2784534520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558dba330610 0x558dba3bee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc713fe640 1 -- 172.21.15.133:0/2784534520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558dba3309f0 msgr2=0x558dba3c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc713fe640 1 --2- 172.21.15.133:0/2784534520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558dba3309f0 0x558dba3c1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc713fe640 1 -- 172.21.15.133:0/2784534520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558dba330610 msgr2=0x558dba3bee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc713fe640 1 --2- 172.21.15.133:0/2784534520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558dba330610 0x558dba3bee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc713fe640 1 -- 172.21.15.133:0/2784534520 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dba3bfeb0 con 0x558dba317220 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc71bff640 1 --2- 172.21.15.133:0/2784534520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558dba3309f0 0x558dba3c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc70bfd640 1 --2- 172.21.15.133:0/2784534520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558dba330610 0x558dba3bee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc713fe640 1 --2- 172.21.15.133:0/2784534520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba317220 0x558dba3bb0e0 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7fdc640ad040 tx=0x7fdc64003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ca8abaa78ae6fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc6b7fe640 1 -- 172.21.15.133:0/2784534520 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc640af040 con 0x558dba317220 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc6b7fe640 1 -- 172.21.15.133:0/2784534520 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc640b9070 con 0x558dba317220 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc6b7fe640 1 -- 172.21.15.133:0/2784534520 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc640aae50 con 0x558dba317220 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- 172.21.15.133:0/2784534520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba317220 msgr2=0x558dba3bb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 --2- 172.21.15.133:0/2784534520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba317220 0x558dba3bb0e0 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7fdc640ad040 tx=0x7fdc64003030 comp rx=0 tx=0).stop 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- 172.21.15.133:0/2784534520 shutdown_connections 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 --2- 172.21.15.133:0/2784534520 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558dba3309f0 0x558dba3c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 --2- 172.21.15.133:0/2784534520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558dba330610 0x558dba3bee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 --2- 172.21.15.133:0/2784534520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba317220 0x558dba3bb0e0 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- 172.21.15.133:0/2784534520 >> 172.21.15.133:0/2784534520 conn(0x558dba2e6180 msgr2=0x558dba3bba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- 172.21.15.133:0/2784534520 shutdown_connections 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- 172.21.15.133:0/2784534520 wait complete. 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 Processor -- start 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- start start 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558dba317220 0x558dba3b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba330610 0x558dba3144d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558dba3309f0 0x558dba314a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dba3bb620 con 0x558dba330610 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dba3bd450 con 0x558dba317220 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc738e0080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dba3b4ae0 con 0x558dba3309f0 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc70bfd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba330610 0x558dba3144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc70bfd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba330610 0x558dba3144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45714/0 (socket says 172.21.15.133:45714) 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc70bfd640 1 -- 172.21.15.133:0/269231969 learned_addr learned my addr 172.21.15.133:0/269231969 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc713fe640 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558dba317220 0x558dba3b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc71bff640 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558dba3309f0 0x558dba314a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc70bfd640 1 -- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558dba3309f0 msgr2=0x558dba314a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc70bfd640 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x558dba3309f0 0x558dba314a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc70bfd640 1 -- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558dba317220 msgr2=0x558dba3b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc70bfd640 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558dba317220 0x558dba3b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc70bfd640 1 -- 172.21.15.133:0/269231969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dba3b4d80 con 0x558dba330610 2024-04-25T10:17:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.871+0000 7fdc713fe640 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558dba317220 0x558dba3b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc70bfd640 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba330610 0x558dba3144d0 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009400 tx=0x7fdc5c0098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc6a7fc640 1 -- 172.21.15.133:0/269231969 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c00f780 con 0x558dba330610 2024-04-25T10:17:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc6a7fc640 1 -- 172.21.15.133:0/269231969 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5c00f920 con 0x558dba330610 2024-04-25T10:17:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc6a7fc640 1 -- 172.21.15.133:0/269231969 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c00fcf0 con 0x558dba330610 2024-04-25T10:17:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc738e0080 1 -- 172.21.15.133:0/269231969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558dba3b5020 con 0x558dba330610 2024-04-25T10:17:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc738e0080 1 -- 172.21.15.133:0/269231969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dba3b5560 con 0x558dba330610 2024-04-25T10:17:55.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc6a7fc640 1 -- 172.21.15.133:0/269231969 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc5c01a420 con 0x558dba330610 2024-04-25T10:17:55.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc6a7fc640 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc54065970 0x7fdc54067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc6a7fc640 1 -- 172.21.15.133:0/269231969 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdc5c083f50 con 0x558dba330610 2024-04-25T10:17:55.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc713fe640 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc54065970 0x7fdc54067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc738e0080 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x558dba3b4610 0x558dba3b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc738e0080 1 -- 172.21.15.133:0/269231969 --> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] -- osd_op(unknown.0.0:1 29.10 29:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e196) v9 -- 0x558dba3b78f0 con 0x558dba3b4610 2024-04-25T10:17:55.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc713fe640 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc54065970 0x7fdc54067e30 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdc64019a50 tx=0x7fdc640ac040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc71bff640 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x558dba3b4610 0x558dba3b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.875+0000 7fdc71bff640 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x558dba3b4610 0x558dba3b73b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc71bff640 1 -- 172.21.15.133:0/269231969 <== osd.6 v2:172.21.15.162:6824/2819373209 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-vals-by-keys out=527b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+527 (crc 0 0 0) 0x7fdc6c007700 con 0x558dba3b4610 2024-04-25T10:17:55.884 INFO:teuthology.orchestra.run.smithi133.stderr:Writing to - 2024-04-25T10:17:55.885 INFO:teuthology.orchestra.run.smithi133.stdout:i����-*fT$ ����@@`���������-*fT$ �-*fE�P 2024-04-25T10:17:55.885 INFO:teuthology.orchestra.run.smithi133.stdout:(8`�-*fT$ 8`�-*fT$ ���������-*fE�P 2024-04-25T10:17:55.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc738e0080 1 -- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x558dba3b4610 msgr2=0x558dba3b73b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc738e0080 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x558dba3b4610 0x558dba3b73b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc738e0080 1 -- 172.21.15.133:0/269231969 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc54065970 msgr2=0x7fdc54067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc738e0080 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdc54065970 0x7fdc54067e30 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fdc64019a50 tx=0x7fdc640ac040 comp rx=0 tx=0).stop 2024-04-25T10:17:55.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc738e0080 1 -- 172.21.15.133:0/269231969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba330610 msgr2=0x558dba3144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc738e0080 1 --2- 172.21.15.133:0/269231969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dba330610 0x558dba3144d0 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009400 tx=0x7fdc5c0098c0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc71bff640 1 -- 172.21.15.133:0/269231969 reap_dead start 2024-04-25T10:17:55.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc738e0080 1 -- 172.21.15.133:0/269231969 shutdown_connections 2024-04-25T10:17:55.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc738e0080 1 -- 172.21.15.133:0/269231969 >> 172.21.15.133:0/269231969 conn(0x558dba2e6180 msgr2=0x558dba3bdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc738e0080 1 -- 172.21.15.133:0/269231969 shutdown_connections 2024-04-25T10:17:55.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.879+0000 7fdc738e0080 1 -- 172.21.15.133:0/269231969 wait complete. 2024-04-25T10:17:55.887 INFO:teuthology.orchestra.run.smithi133.stdout:0������������ 2024-04-25T10:17:55.887 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 shadow_head 2024-04-25T10:17:55.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.955+0000 7fedd011f080 1 Processor -- start 2024-04-25T10:17:55.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.955+0000 7fedd011f080 1 -- start start 2024-04-25T10:17:55.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.955+0000 7fedd011f080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedbc003cc0 0x561e541e3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.955+0000 7fedd011f080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 0x561e541df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561e541dd400 0x561e541df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e540b39f0 con 0x561e541e4b80 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e540b7950 con 0x7fedbc003cc0 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e54156040 con 0x561e541dd400 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 0x561e541df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 0x561e541df2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45734/0 (socket says 172.21.15.133:45734) 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 -- 172.21.15.133:0/2897238684 learned_addr learned my addr 172.21.15.133:0/2897238684 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcdc3d640 1 --2- 172.21.15.133:0/2897238684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedbc003cc0 0x561e541e3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedce43e640 1 --2- 172.21.15.133:0/2897238684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561e541dd400 0x561e541df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 -- 172.21.15.133:0/2897238684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561e541dd400 msgr2=0x561e541df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 --2- 172.21.15.133:0/2897238684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561e541dd400 0x561e541df7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 -- 172.21.15.133:0/2897238684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedbc003cc0 msgr2=0x561e541e3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 --2- 172.21.15.133:0/2897238684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedbc003cc0 0x561e541e3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 -- 172.21.15.133:0/2897238684 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e541de630 con 0x561e541e4b80 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcdc3d640 1 --2- 172.21.15.133:0/2897238684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedbc003cc0 0x561e541e3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 --2- 172.21.15.133:0/2897238684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 0x561e541df2b0 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c260 tx=0x7fedb800c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86898dbb474bfe99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedc7fff640 1 -- 172.21.15.133:0/2897238684 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8010440 con 0x561e541e4b80 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedc7fff640 1 -- 172.21.15.133:0/2897238684 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb80105e0 con 0x561e541e4b80 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedc7fff640 1 -- 172.21.15.133:0/2897238684 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb80109b0 con 0x561e541e4b80 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- 172.21.15.133:0/2897238684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 msgr2=0x561e541df2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 --2- 172.21.15.133:0/2897238684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 0x561e541df2b0 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c260 tx=0x7fedb800c720 comp rx=0 tx=0).stop 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- 172.21.15.133:0/2897238684 shutdown_connections 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 --2- 172.21.15.133:0/2897238684 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561e541dd400 0x561e541df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 --2- 172.21.15.133:0/2897238684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 0x561e541df2b0 unknown :-1 s=CLOSED pgs=2924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 --2- 172.21.15.133:0/2897238684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedbc003cc0 0x561e541e3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- 172.21.15.133:0/2897238684 >> 172.21.15.133:0/2897238684 conn(0x561e54138f00 msgr2=0x561e5414e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- 172.21.15.133:0/2897238684 shutdown_connections 2024-04-25T10:17:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- 172.21.15.133:0/2897238684 wait complete. 2024-04-25T10:17:55.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 Processor -- start 2024-04-25T10:17:55.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- start start 2024-04-25T10:17:55.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561e541dd400 0x561e541ed770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 0x561e541dda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedbc003cc0 0x561e541ddf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e541deb70 con 0x561e541e4b80 2024-04-25T10:17:55.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e541de8d0 con 0x7fedbc003cc0 2024-04-25T10:17:55.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e541dee10 con 0x561e541dd400 2024-04-25T10:17:55.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 0x561e541dda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 0x561e541dda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45740/0 (socket says 172.21.15.133:45740) 2024-04-25T10:17:55.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 -- 172.21.15.133:0/3221433103 learned_addr learned my addr 172.21.15.133:0/3221433103 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcdc3d640 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561e541dd400 0x561e541ed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedce43e640 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedbc003cc0 0x561e541ddf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 -- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561e541dd400 msgr2=0x561e541ed770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x561e541dd400 0x561e541ed770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 -- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedbc003cc0 msgr2=0x561e541ddf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedbc003cc0 0x561e541ddf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 -- 172.21.15.133:0/3221433103 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e54151f80 con 0x561e541e4b80 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedce43e640 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fedbc003cc0 0x561e541ddf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedcd43c640 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 0x561e541dda50 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7fedb800f690 tx=0x7fedb80011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedc6ffd640 1 -- 172.21.15.133:0/3221433103 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8021070 con 0x561e541e4b80 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedc6ffd640 1 -- 172.21.15.133:0/3221433103 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb800f870 con 0x561e541e4b80 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- 172.21.15.133:0/3221433103 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561e54152220 con 0x561e541e4b80 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedc6ffd640 1 -- 172.21.15.133:0/3221433103 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8010440 con 0x561e541e4b80 2024-04-25T10:17:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.959+0000 7fedd011f080 1 -- 172.21.15.133:0/3221433103 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e54152760 con 0x561e541e4b80 2024-04-25T10:17:55.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.963+0000 7fedc6ffd640 1 -- 172.21.15.133:0/3221433103 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fedb8024070 con 0x561e541e4b80 2024-04-25T10:17:55.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.963+0000 7fedc6ffd640 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fedb0065970 0x7fedb0067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.963+0000 7fedc6ffd640 1 -- 172.21.15.133:0/3221433103 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fedb8085d20 con 0x561e541e4b80 2024-04-25T10:17:55.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.963+0000 7fedcdc3d640 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fedb0065970 0x7fedb0067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.963+0000 7fedd011f080 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x561e541cbd20 0x561e5414f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.963+0000 7fedd011f080 1 -- 172.21.15.133:0/3221433103 --> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] -- osd_op(unknown.0.0:1 29.10 29:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=523b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e196) v9 -- 0x561e5414f6b0 con 0x561e541cbd20 2024-04-25T10:17:55.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.963+0000 7fedcdc3d640 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fedb0065970 0x7fedb0067e30 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005fd0 tx=0x7fedc800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.963+0000 7fedce43e640 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x561e541cbd20 0x561e5414f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.963+0000 7fedce43e640 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x561e541cbd20 0x561e5414f170 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedce43e640 1 -- 172.21.15.133:0/3221433103 <== osd.6 v2:172.21.15.162:6824/2819373209 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v196'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fedc0001260 con 0x561e541cbd20 2024-04-25T10:17:55.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedd011f080 1 -- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x561e541cbd20 msgr2=0x561e5414f170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedd011f080 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x561e541cbd20 0x561e5414f170 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedd011f080 1 -- 172.21.15.133:0/3221433103 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fedb0065970 msgr2=0x7fedb0067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedd011f080 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fedb0065970 0x7fedb0067e30 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005fd0 tx=0x7fedc800b040 comp rx=0 tx=0).stop 2024-04-25T10:17:55.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedd011f080 1 -- 172.21.15.133:0/3221433103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 msgr2=0x561e541dda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedd011f080 1 --2- 172.21.15.133:0/3221433103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e541e4b80 0x561e541dda50 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7fedb800f690 tx=0x7fedb80011a0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedce43e640 1 -- 172.21.15.133:0/3221433103 reap_dead start 2024-04-25T10:17:55.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedd011f080 1 -- 172.21.15.133:0/3221433103 shutdown_connections 2024-04-25T10:17:55.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedd011f080 1 -- 172.21.15.133:0/3221433103 >> 172.21.15.133:0/3221433103 conn(0x561e54138f00 msgr2=0x561e541550e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedd011f080 1 -- 172.21.15.133:0/3221433103 shutdown_connections 2024-04-25T10:17:55.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:55.967+0000 7fedd011f080 1 -- 172.21.15.133:0/3221433103 wait complete. 2024-04-25T10:17:55.973 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_hack_allow_loading_invalid_metadata True 2024-04-25T10:17:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.135+0000 7ff3c2636640 1 Processor -- start 2024-04-25T10:17:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.135+0000 7ff3c2636640 1 -- start start 2024-04-25T10:17:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3c2636640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc145170 0x7ff3bc06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3c2636640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff3bc06e4d0 0x7ff3bc0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3c2636640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff3bc0cdd60 0x7ff3bc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3c2636640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc06db00 con 0x7ff3bc06e4d0 2024-04-25T10:17:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3c2636640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc145550 con 0x7ff3bc0cdd60 2024-04-25T10:17:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3c2636640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0ce2e0 con 0x7ff3bc145170 2024-04-25T10:17:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3bb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff3bc06e4d0 0x7ff3bc0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3bb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff3bc06e4d0 0x7ff3bc0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45752/0 (socket says 172.21.15.133:45752) 2024-04-25T10:17:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3bb7fe640 1 -- 172.21.15.133:0/3394183736 learned_addr learned my addr 172.21.15.133:0/3394183736 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3bbfff640 1 --2- 172.21.15.133:0/3394183736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc145170 0x7ff3bc06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3c0bd3640 1 --2- 172.21.15.133:0/3394183736 >> v2:172.21.15.162:3300/0 conn(0x7ff3bc0cdd60 0x7ff3bc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3bb7fe640 1 -- 172.21.15.133:0/3394183736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc145170 msgr2=0x7ff3bc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3bb7fe640 1 --2- 172.21.15.133:0/3394183736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc145170 0x7ff3bc06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3bb7fe640 1 -- 172.21.15.133:0/3394183736 >> v2:172.21.15.162:3300/0 conn(0x7ff3bc0cdd60 msgr2=0x7ff3bc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3bb7fe640 1 --2- 172.21.15.133:0/3394183736 >> v2:172.21.15.162:3300/0 conn(0x7ff3bc0cdd60 0x7ff3bc15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3bb7fe640 1 -- 172.21.15.133:0/3394183736 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc06d4b0 con 0x7ff3bc06e4d0 2024-04-25T10:17:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3bbfff640 1 --2- 172.21.15.133:0/3394183736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc145170 0x7ff3bc06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3c0bd3640 1 --2- 172.21.15.133:0/3394183736 >> v2:172.21.15.162:3300/0 conn(0x7ff3bc0cdd60 0x7ff3bc15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:56.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3bb7fe640 1 --2- 172.21.15.133:0/3394183736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3bc06e4d0 0x7ff3bc0cd820 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0a8550 tx=0x7ff3b001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f02b56059b3dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3baffd640 1 -- 172.21.15.133:0/3394183736 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00b0070 con 0x7ff3bc06e4d0 2024-04-25T10:17:56.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3baffd640 1 -- 172.21.15.133:0/3394183736 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b00ab600 con 0x7ff3bc06e4d0 2024-04-25T10:17:56.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.139+0000 7ff3baffd640 1 -- 172.21.15.133:0/3394183736 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00ab9d0 con 0x7ff3bc06e4d0 2024-04-25T10:17:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 -- 172.21.15.133:0/3394183736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3bc06e4d0 msgr2=0x7ff3bc0cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 --2- 172.21.15.133:0/3394183736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3bc06e4d0 0x7ff3bc0cd820 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0a8550 tx=0x7ff3b001c520 comp rx=0 tx=0).stop 2024-04-25T10:17:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 -- 172.21.15.133:0/3394183736 shutdown_connections 2024-04-25T10:17:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 --2- 172.21.15.133:0/3394183736 >> v2:172.21.15.162:3300/0 conn(0x7ff3bc0cdd60 0x7ff3bc15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 --2- 172.21.15.133:0/3394183736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3bc06e4d0 0x7ff3bc0cd820 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 --2- 172.21.15.133:0/3394183736 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc145170 0x7ff3bc06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 -- 172.21.15.133:0/3394183736 >> 172.21.15.133:0/3394183736 conn(0x7ff3bc066f70 msgr2=0x7ff3bc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 -- 172.21.15.133:0/3394183736 shutdown_connections 2024-04-25T10:17:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 -- 172.21.15.133:0/3394183736 wait complete. 2024-04-25T10:17:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 Processor -- start 2024-04-25T10:17:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 -- start start 2024-04-25T10:17:56.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc06e4d0 0x7ff3bc165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3bc0cdd60 0x7ff3bc1fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3bc145170 0x7ff3bc144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc15f690 con 0x7ff3bc145170 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc15f510 con 0x7ff3bc0cdd60 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc15f810 con 0x7ff3bc06e4d0 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c0bd3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3bc145170 0x7ff3bc144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c0bd3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3bc145170 0x7ff3bc144030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45766/0 (socket says 172.21.15.133:45766) 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c0bd3640 1 -- 172.21.15.133:0/2448527244 learned_addr learned my addr 172.21.15.133:0/2448527244 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3bbfff640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc06e4d0 0x7ff3bc165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3bb7fe640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3bc0cdd60 0x7ff3bc1fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c0bd3640 1 -- 172.21.15.133:0/2448527244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc06e4d0 msgr2=0x7ff3bc165820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c0bd3640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc06e4d0 0x7ff3bc165820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c0bd3640 1 -- 172.21.15.133:0/2448527244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3bc0cdd60 msgr2=0x7ff3bc1fa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c0bd3640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3bc0cdd60 0x7ff3bc1fa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c0bd3640 1 -- 172.21.15.133:0/2448527244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc0af550 con 0x7ff3bc145170 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c0bd3640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3bc145170 0x7ff3bc144030 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400cef0 tx=0x7ff3b400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3b8ff9640 1 -- 172.21.15.133:0/2448527244 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b401a070 con 0x7ff3bc145170 2024-04-25T10:17:56.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3b8ff9640 1 -- 172.21.15.133:0/2448527244 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b400d840 con 0x7ff3bc145170 2024-04-25T10:17:56.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3b8ff9640 1 -- 172.21.15.133:0/2448527244 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400dc30 con 0x7ff3bc145170 2024-04-25T10:17:56.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 -- 172.21.15.133:0/2448527244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3bc1fac70 con 0x7ff3bc145170 2024-04-25T10:17:56.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.143+0000 7ff3c2636640 1 -- 172.21.15.133:0/2448527244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3bc1fa900 con 0x7ff3bc145170 2024-04-25T10:17:56.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.147+0000 7ff3c2636640 1 -- 172.21.15.133:0/2448527244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3bc068e80 con 0x7ff3bc145170 2024-04-25T10:17:56.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.147+0000 7ff3b8ff9640 1 -- 172.21.15.133:0/2448527244 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3b401d070 con 0x7ff3bc145170 2024-04-25T10:17:56.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.147+0000 7ff3b8ff9640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3a8066f60 0x7ff3a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.147+0000 7ff3bbfff640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3a8066f60 0x7ff3a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.147+0000 7ff3b8ff9640 1 -- 172.21.15.133:0/2448527244 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff3b4085a40 con 0x7ff3bc145170 2024-04-25T10:17:56.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.147+0000 7ff3bbfff640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3a8066f60 0x7ff3a8069420 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac006fd0 tx=0x7ff3ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.151+0000 7ff3b8ff9640 1 -- 172.21.15.133:0/2448527244 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3bc1fac70 con 0x7ff3bc145170 2024-04-25T10:17:56.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.307+0000 7ff3c2636640 1 -- 172.21.15.133:0/2448527244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}] v 0) v1 -- 0x7ff3bc141d70 con 0x7ff3bc145170 2024-04-25T10:17:56.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.311+0000 7ff3b8ff9640 1 -- 172.21.15.133:0/2448527244 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}]=0 v35)=0 v35) v1 ==== 140+0+0 (secure 0 0 0) 0x7ff3b4085e20 con 0x7ff3bc145170 2024-04-25T10:17:56.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 -- 172.21.15.133:0/2448527244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3a8066f60 msgr2=0x7ff3a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3a8066f60 0x7ff3a8069420 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac006fd0 tx=0x7ff3ac008040 comp rx=0 tx=0).stop 2024-04-25T10:17:56.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 -- 172.21.15.133:0/2448527244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3bc145170 msgr2=0x7ff3bc144030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3bc145170 0x7ff3bc144030 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400cef0 tx=0x7ff3b400a710 comp rx=0 tx=0).stop 2024-04-25T10:17:56.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 -- 172.21.15.133:0/2448527244 shutdown_connections 2024-04-25T10:17:56.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff3a8066f60 0x7ff3a8069420 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3bc145170 0x7ff3bc144030 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3bc0cdd60 0x7ff3bc1fa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 --2- 172.21.15.133:0/2448527244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff3bc06e4d0 0x7ff3bc165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 -- 172.21.15.133:0/2448527244 >> 172.21.15.133:0/2448527244 conn(0x7ff3bc066f70 msgr2=0x7ff3bc15d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 -- 172.21.15.133:0/2448527244 shutdown_connections 2024-04-25T10:17:56.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.315+0000 7ff3c2636640 1 -- 172.21.15.133:0/2448527244 wait complete. 2024-04-25T10:17:56.335 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:17:56.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.455+0000 7f94c549a640 1 Processor -- start 2024-04-25T10:17:56.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94c549a640 1 -- start start 2024-04-25T10:17:56.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94c549a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f94c0145170 0x7f94c006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94c549a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c00ce2e0 con 0x7f94c006e4d0 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94c549a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c006d4b0 con 0x7f94c015ced0 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94c549a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c006db00 con 0x7f94c0145170 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94be7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f94c006e4d0 0x7f94c00cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37700/0 (socket says 172.21.15.133:37700) 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94be7fc640 1 -- 172.21.15.133:0/3241655584 learned_addr learned my addr 172.21.15.133:0/3241655584 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94beffd640 1 --2- 172.21.15.133:0/3241655584 >> v2:172.21.15.162:3300/0 conn(0x7f94c0145170 0x7f94c006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3370846406 0 0) 0x7f94c00ce2e0 con 0x7f94c006e4d0 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a8003530 con 0x7f94c006e4d0 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2268979935 0 0) 0x7f94a8003530 con 0x7f94c006e4d0 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 >> v2:172.21.15.162:3300/0 conn(0x7f94c0145170 msgr2=0x7f94c006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 --2- 172.21.15.133:0/3241655584 >> v2:172.21.15.162:3300/0 conn(0x7f94c0145170 0x7f94c006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 >> v1:172.21.15.162:6789/0 conn(0x7f94c015ced0 legacy=0x7f94c00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c006d790 con 0x7f94c006e4d0 2024-04-25T10:17:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f94b4001680 con 0x7f94c006e4d0 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 >> v1:172.21.15.133:6789/0 conn(0x7f94c006e4d0 legacy=0x7f94c00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 0x7f94a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94a8004810 0x7f94a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94a80051d0 0x7f94a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c006db00 con 0x7f94a80039c0 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a800c070 con 0x7f94a80051d0 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a800c270 con 0x7f94a8004810 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94beffd640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 0x7f94a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94be7fc640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94a8004810 0x7f94a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bf7fe640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94a80051d0 0x7f94a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94beffd640 1 -- 172.21.15.133:0/3241655584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94a8004810 msgr2=0x7f94a8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94beffd640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94a8004810 0x7f94a8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94beffd640 1 -- 172.21.15.133:0/3241655584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94a80051d0 msgr2=0x7f94a800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94beffd640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94a80051d0 0x7f94a800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94beffd640 1 -- 172.21.15.133:0/3241655584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a8003530 con 0x7f94a80039c0 2024-04-25T10:17:56.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bf7fe640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94a80051d0 0x7f94a800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:56.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.459+0000 7f94bf7fe640 1 -- 172.21.15.133:0/3241655584 reap_dead start 2024-04-25T10:17:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94beffd640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 0x7f94a8003de0 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009ae0 tx=0x7f94b001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868dd94f7b36d50b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00096b0 con 0x7f94a80039c0 2024-04-25T10:17:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b00a6420 con 0x7f94a80039c0 2024-04-25T10:17:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94bdffb640 1 -- 172.21.15.133:0/3241655584 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00a6810 con 0x7f94a80039c0 2024-04-25T10:17:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94c549a640 1 -- 172.21.15.133:0/3241655584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 msgr2=0x7f94a8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94c549a640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 0x7f94a8003de0 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009ae0 tx=0x7f94b001ca70 comp rx=0 tx=0).stop 2024-04-25T10:17:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94c549a640 1 -- 172.21.15.133:0/3241655584 shutdown_connections 2024-04-25T10:17:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94c549a640 1 --2- 172.21.15.133:0/3241655584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 0x7f94a8003de0 unknown :-1 s=CLOSED pgs=2929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94c549a640 1 -- 172.21.15.133:0/3241655584 >> 172.21.15.133:0/3241655584 conn(0x7f94c0066f70 msgr2=0x7f94c006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94c549a640 1 -- 172.21.15.133:0/3241655584 shutdown_connections 2024-04-25T10:17:56.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94c549a640 1 -- 172.21.15.133:0/3241655584 wait complete. 2024-04-25T10:17:56.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94c549a640 1 Processor -- start 2024-04-25T10:17:56.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.463+0000 7f94c549a640 1 -- start start 2024-04-25T10:17:56.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94c549a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 0x7f94c0139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94c549a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94c013df70 0x7f94c015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94c549a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94c00b30d0 0x7f94c00b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94c549a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00b16c0 con 0x7f94a80039c0 2024-04-25T10:17:56.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94c549a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0145550 con 0x7f94c013df70 2024-04-25T10:17:56.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94c549a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00b1840 con 0x7f94c00b30d0 2024-04-25T10:17:56.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94beffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 0x7f94c0139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94beffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 0x7f94c0139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41978/0 (socket says 172.21.15.133:41978) 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94beffd640 1 -- 172.21.15.133:0/3708344845 learned_addr learned my addr 172.21.15.133:0/3708344845 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94be7fc640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94c013df70 0x7f94c015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94bf7fe640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94c00b30d0 0x7f94c00b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94beffd640 1 -- 172.21.15.133:0/3708344845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94c00b30d0 msgr2=0x7f94c00b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94beffd640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94c00b30d0 0x7f94c00b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94beffd640 1 -- 172.21.15.133:0/3708344845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94c013df70 msgr2=0x7f94c015bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94beffd640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94c013df70 0x7f94c015bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94beffd640 1 -- 172.21.15.133:0/3708344845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c013e3f0 con 0x7f94a80039c0 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94bf7fe640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94c00b30d0 0x7f94c00b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94beffd640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 0x7f94c0139ae0 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f94b001c8a0 tx=0x7f94b0019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94a77fe640 1 -- 172.21.15.133:0/3708344845 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00a6e50 con 0x7f94a80039c0 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94a77fe640 1 -- 172.21.15.133:0/3708344845 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b00b9440 con 0x7f94a80039c0 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94c549a640 1 -- 172.21.15.133:0/3708344845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c015c260 con 0x7f94a80039c0 2024-04-25T10:17:56.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94a77fe640 1 -- 172.21.15.133:0/3708344845 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00b9810 con 0x7f94a80039c0 2024-04-25T10:17:56.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94c549a640 1 -- 172.21.15.133:0/3708344845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c01f8820 con 0x7f94a80039c0 2024-04-25T10:17:56.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94c549a640 1 -- 172.21.15.133:0/3708344845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c0068e80 con 0x7f94a80039c0 2024-04-25T10:17:56.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94a77fe640 1 -- 172.21.15.133:0/3708344845 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94b00a7400 con 0x7f94a80039c0 2024-04-25T10:17:56.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94a77fe640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94ac067130 0x7f94ac0695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94be7fc640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94ac067130 0x7f94ac0695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94a77fe640 1 -- 172.21.15.133:0/3708344845 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f94b011f490 con 0x7f94a80039c0 2024-04-25T10:17:56.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.467+0000 7f94be7fc640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94ac067130 0x7f94ac0695f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f94c0140a50 tx=0x7f94b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.471+0000 7f94a77fe640 1 -- 172.21.15.133:0/3708344845 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94b00bf060 con 0x7f94a80039c0 2024-04-25T10:17:56.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.659+0000 7f94c549a640 1 -- 172.21.15.133:0/3708344845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f94c00b4510 con 0x7f94a80039c0 2024-04-25T10:17:56.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.747+0000 7f94a77fe640 1 -- 172.21.15.133:0/3708344845 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v120) v1 ==== 162+0+0 (secure 0 0 0) 0x7f94b00ac020 con 0x7f94a80039c0 2024-04-25T10:17:56.750 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 -- 172.21.15.133:0/3708344845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94ac067130 msgr2=0x7f94ac0695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94ac067130 0x7f94ac0695f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f94c0140a50 tx=0x7f94b4009040 comp rx=0 tx=0).stop 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 -- 172.21.15.133:0/3708344845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 msgr2=0x7f94c0139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 0x7f94c0139ae0 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f94b001c8a0 tx=0x7f94b0019810 comp rx=0 tx=0).stop 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 -- 172.21.15.133:0/3708344845 shutdown_connections 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f94c00b30d0 0x7f94c00b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f94c013df70 0x7f94c015bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f94ac067130 0x7f94ac0695f0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 --2- 172.21.15.133:0/3708344845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a80039c0 0x7f94c0139ae0 unknown :-1 s=CLOSED pgs=2930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 -- 172.21.15.133:0/3708344845 >> 172.21.15.133:0/3708344845 conn(0x7f94c0066f70 msgr2=0x7f94c00737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 -- 172.21.15.133:0/3708344845 shutdown_connections 2024-04-25T10:17:56.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.751+0000 7f94c549a640 1 -- 172.21.15.133:0/3708344845 wait complete. 2024-04-25T10:17:56.775 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:56.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.875+0000 7f855e082640 1 Processor -- start 2024-04-25T10:17:56.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.875+0000 7f855e082640 1 -- start start 2024-04-25T10:17:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f855e082640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8558161050 0x7f85580999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f855e082640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f855806cdb0 0x7f85580cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f855e082640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85580b06d0 con 0x7f85580b3ea0 2024-04-25T10:17:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f855e082640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85580d0480 con 0x7f8558161050 2024-04-25T10:17:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f855e082640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85580d0760 con 0x7f855806cdb0 2024-04-25T10:17:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f85577fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8558161050 0x7f85580999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f8556ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f85580b3ea0 0x7f8558099f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37716/0 (socket says 172.21.15.133:37716) 2024-04-25T10:17:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f8556ffd640 1 -- 172.21.15.133:0/2100769486 learned_addr learned my addr 172.21.15.133:0/2100769486 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:17:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f85577fe640 1 -- 172.21.15.133:0/2100769486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f855806cdb0 msgr2=0x7f85580cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f85577fe640 1 --2- 172.21.15.133:0/2100769486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f855806cdb0 0x7f85580cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f85577fe640 1 -- 172.21.15.133:0/2100769486 >> v1:172.21.15.133:6789/0 conn(0x7f85580b3ea0 legacy=0x7f8558099f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f85577fe640 1 -- 172.21.15.133:0/2100769486 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855809a620 con 0x7f8558161050 2024-04-25T10:17:56.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f85577fe640 1 --2- 172.21.15.133:0/2100769486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8558161050 0x7f85580999d0 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7f85480097c0 tx=0x7f854801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642b0cdd7c08bf85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f85567fc640 1 -- 172.21.15.133:0/2100769486 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480b0070 con 0x7f8558161050 2024-04-25T10:17:56.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f85567fc640 1 -- 172.21.15.133:0/2100769486 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85480ab600 con 0x7f8558161050 2024-04-25T10:17:56.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.879+0000 7f85567fc640 1 -- 172.21.15.133:0/2100769486 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480ab9d0 con 0x7f8558161050 2024-04-25T10:17:56.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 -- 172.21.15.133:0/2100769486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8558161050 msgr2=0x7f85580999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 --2- 172.21.15.133:0/2100769486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8558161050 0x7f85580999d0 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7f85480097c0 tx=0x7f854801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 -- 172.21.15.133:0/2100769486 shutdown_connections 2024-04-25T10:17:56.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 --2- 172.21.15.133:0/2100769486 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f855806cdb0 0x7f85580cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 --2- 172.21.15.133:0/2100769486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8558161050 0x7f85580999d0 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 -- 172.21.15.133:0/2100769486 >> 172.21.15.133:0/2100769486 conn(0x7f85580b31a0 msgr2=0x7f85580b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 -- 172.21.15.133:0/2100769486 shutdown_connections 2024-04-25T10:17:56.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 -- 172.21.15.133:0/2100769486 wait complete. 2024-04-25T10:17:56.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 Processor -- start 2024-04-25T10:17:56.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 -- start start 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855806cdb0 0x7f85580c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85580b3ea0 0x7f85580b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8558161050 0x7f8558156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85580cce90 con 0x7f855806cdb0 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558161640 con 0x7f8558161050 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85580cd010 con 0x7f85580b3ea0 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f85577fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855806cdb0 0x7f85580c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f85577fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855806cdb0 0x7f85580c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42002/0 (socket says 172.21.15.133:42002) 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f85577fe640 1 -- 172.21.15.133:0/80073424 learned_addr learned my addr 172.21.15.133:0/80073424 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f85577fe640 1 -- 172.21.15.133:0/80073424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85580b3ea0 msgr2=0x7f85580b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f85577fe640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85580b3ea0 0x7f85580b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f85577fe640 1 -- 172.21.15.133:0/80073424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8558161050 msgr2=0x7f8558156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f85577fe640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8558161050 0x7f8558156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f85577fe640 1 -- 172.21.15.133:0/80073424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85580c5720 con 0x7f855806cdb0 2024-04-25T10:17:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f85577fe640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855806cdb0 0x7f85580c51e0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f85480ae040 tx=0x7f854801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f854ffff640 1 -- 172.21.15.133:0/80073424 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480b0050 con 0x7f855806cdb0 2024-04-25T10:17:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f854ffff640 1 -- 172.21.15.133:0/80073424 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85480bc070 con 0x7f855806cdb0 2024-04-25T10:17:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 -- 172.21.15.133:0/80073424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8558070a20 con 0x7f855806cdb0 2024-04-25T10:17:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f854ffff640 1 -- 172.21.15.133:0/80073424 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480b5a30 con 0x7f855806cdb0 2024-04-25T10:17:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.883+0000 7f855e082640 1 -- 172.21.15.133:0/80073424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85580b6140 con 0x7f855806cdb0 2024-04-25T10:17:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.887+0000 7f855e082640 1 -- 172.21.15.133:0/80073424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8558068ef0 con 0x7f855806cdb0 2024-04-25T10:17:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.887+0000 7f854ffff640 1 -- 172.21.15.133:0/80073424 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85480b5bd0 con 0x7f855806cdb0 2024-04-25T10:17:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.887+0000 7f854ffff640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8538066f60 0x7f8538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.887+0000 7f8556ffd640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8538066f60 0x7f8538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.887+0000 7f854ffff640 1 -- 172.21.15.133:0/80073424 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f854811fd90 con 0x7f855806cdb0 2024-04-25T10:17:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.887+0000 7f8556ffd640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8538066f60 0x7f8538069420 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f85580ce340 tx=0x7f8540007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:56.891+0000 7f854ffff640 1 -- 172.21.15.133:0/80073424 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8548016090 con 0x7f855806cdb0 2024-04-25T10:17:57.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.083+0000 7f855e082640 1 -- 172.21.15.133:0/80073424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85580cbeb0 con 0x7f855806cdb0 2024-04-25T10:17:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.083+0000 7f854ffff640 1 -- 172.21.15.133:0/80073424 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f85580cbeb0 con 0x7f855806cdb0 2024-04-25T10:17:57.088 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:57.088 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":121,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:36.969368+0000","modified":"2024-04-25T10:17:56.755453+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":196,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12135":{"gid":12135,"name":"c","rank":0,"incarnation":121,"state":"up:replay","state_seq":1,"addr":"172.21.15.133:6837/2510709242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2510709242},{"type":"v1","addr":"172.21.15.133:6837","nonce":2510709242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:17:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 121 2024-04-25T10:17:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 -- 172.21.15.133:0/80073424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8538066f60 msgr2=0x7f8538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8538066f60 0x7f8538069420 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f85580ce340 tx=0x7f8540007a90 comp rx=0 tx=0).stop 2024-04-25T10:17:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 -- 172.21.15.133:0/80073424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855806cdb0 msgr2=0x7f85580c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855806cdb0 0x7f85580c51e0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f85480ae040 tx=0x7f854801ace0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 -- 172.21.15.133:0/80073424 shutdown_connections 2024-04-25T10:17:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8558161050 0x7f8558156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f85580b3ea0 0x7f85580b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f8538066f60 0x7f8538069420 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 --2- 172.21.15.133:0/80073424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855806cdb0 0x7f85580c51e0 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 -- 172.21.15.133:0/80073424 >> 172.21.15.133:0/80073424 conn(0x7f85580b31a0 msgr2=0x7f8558157330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 -- 172.21.15.133:0/80073424 shutdown_connections 2024-04-25T10:17:57.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:57.087+0000 7f855e082640 1 -- 172.21.15.133:0/80073424 wait complete. 2024-04-25T10:17:57.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 121, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:36.969368+0000', 'modified': '2024-04-25T10:17:56.755453+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 196, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12135': {'gid': 12135, 'name': 'c', 'rank': 0, 'incarnation': 121, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.133:6837/2510709242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 2510709242}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 2510709242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:17:57.509 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_12135:up:replay 2024-04-25T10:17:58.510 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:58.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.635+0000 7f61f6f9e640 1 Processor -- start 2024-04-25T10:17:58.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.635+0000 7f61f6f9e640 1 -- start start 2024-04-25T10:17:58.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f6f9e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f61f015c9a0 0x7f61f00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f6f9e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61f00ce330 0x7f61f00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f6f9e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f00c61d0 con 0x7f61f00b1f70 2024-04-25T10:17:58.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f6f9e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f00cecc0 con 0x7f61f00ce330 2024-04-25T10:17:58.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f6f9e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f00c64b0 con 0x7f61f015c9a0 2024-04-25T10:17:58.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f553b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61f00ce330 0x7f61f00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f553b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61f00ce330 0x7f61f00ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42006/0 (socket says 172.21.15.133:42006) 2024-04-25T10:17:58.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f553b640 1 -- 172.21.15.133:0/1683648537 learned_addr learned my addr 172.21.15.133:0/1683648537 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:58.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f4d3a640 1 --2- 172.21.15.133:0/1683648537 >> v2:172.21.15.162:3300/0 conn(0x7f61f015c9a0 0x7f61f00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f553b640 1 -- 172.21.15.133:0/1683648537 >> v2:172.21.15.162:3300/0 conn(0x7f61f015c9a0 msgr2=0x7f61f00c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:58.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f553b640 1 --2- 172.21.15.133:0/1683648537 >> v2:172.21.15.162:3300/0 conn(0x7f61f015c9a0 0x7f61f00c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f553b640 1 -- 172.21.15.133:0/1683648537 >> v1:172.21.15.162:6789/0 conn(0x7f61f00b1f70 legacy=0x7f61f006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:58.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f553b640 1 -- 172.21.15.133:0/1683648537 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f006d4f0 con 0x7f61f00ce330 2024-04-25T10:17:58.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f553b640 1 --2- 172.21.15.133:0/1683648537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00ce330 0x7f61f00ce780 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f61f006f5f0 tx=0x7f61e801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce1ad91150f68e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61ef7fe640 1 -- 172.21.15.133:0/1683648537 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80a5720 con 0x7f61f00ce330 2024-04-25T10:17:58.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61ef7fe640 1 -- 172.21.15.133:0/1683648537 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e80a58c0 con 0x7f61f00ce330 2024-04-25T10:17:58.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61ef7fe640 1 -- 172.21.15.133:0/1683648537 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80a5c90 con 0x7f61f00ce330 2024-04-25T10:17:58.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1683648537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00ce330 msgr2=0x7f61f00ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.639+0000 7f61f6f9e640 1 --2- 172.21.15.133:0/1683648537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00ce330 0x7f61f00ce780 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f61f006f5f0 tx=0x7f61e801c9b0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1683648537 shutdown_connections 2024-04-25T10:17:58.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 --2- 172.21.15.133:0/1683648537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00ce330 0x7f61f00ce780 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 --2- 172.21.15.133:0/1683648537 >> v2:172.21.15.162:3300/0 conn(0x7f61f015c9a0 0x7f61f00c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1683648537 >> 172.21.15.133:0/1683648537 conn(0x7f61f0067000 msgr2=0x7f61f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:58.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1683648537 shutdown_connections 2024-04-25T10:17:58.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1683648537 wait complete. 2024-04-25T10:17:58.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 Processor -- start 2024-04-25T10:17:58.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 -- start start 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61f00b1f70 0x7f61f00cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00ce330 0x7f61f0153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61f015c9a0 0x7f61f00cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f015dad0 con 0x7f61f00ce330 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f015d950 con 0x7f61f015c9a0 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f015dc50 con 0x7f61f00b1f70 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61effff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00ce330 0x7f61f0153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61effff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00ce330 0x7f61f0153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42014/0 (socket says 172.21.15.133:42014) 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61effff640 1 -- 172.21.15.133:0/1566480644 learned_addr learned my addr 172.21.15.133:0/1566480644 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f553b640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61f015c9a0 0x7f61f00cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f4d3a640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61f00b1f70 0x7f61f00cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61effff640 1 -- 172.21.15.133:0/1566480644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61f00b1f70 msgr2=0x7f61f00cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61effff640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61f00b1f70 0x7f61f00cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61effff640 1 -- 172.21.15.133:0/1566480644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61f015c9a0 msgr2=0x7f61f00cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61effff640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61f015c9a0 0x7f61f00cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61effff640 1 -- 172.21.15.133:0/1566480644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00ce120 con 0x7f61f00ce330 2024-04-25T10:17:58.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f4d3a640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61f00b1f70 0x7f61f00cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:58.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61effff640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00ce330 0x7f61f0153950 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7f61e000b410 tx=0x7f61e000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61ed7fa640 1 -- 172.21.15.133:0/1566480644 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0015020 con 0x7f61f00ce330 2024-04-25T10:17:58.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61ed7fa640 1 -- 172.21.15.133:0/1566480644 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e0011830 con 0x7f61f00ce330 2024-04-25T10:17:58.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1566480644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f0150480 con 0x7f61f00ce330 2024-04-25T10:17:58.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61ed7fa640 1 -- 172.21.15.133:0/1566480644 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0011c20 con 0x7f61f00ce330 2024-04-25T10:17:58.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.643+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1566480644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f014e720 con 0x7f61f00ce330 2024-04-25T10:17:58.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.647+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1566480644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f0068ef0 con 0x7f61f00ce330 2024-04-25T10:17:58.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.647+0000 7f61ed7fa640 1 -- 172.21.15.133:0/1566480644 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61e0018020 con 0x7f61f00ce330 2024-04-25T10:17:58.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.647+0000 7f61ed7fa640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61dc066f60 0x7f61dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.647+0000 7f61ed7fa640 1 -- 172.21.15.133:0/1566480644 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f61e0085140 con 0x7f61f00ce330 2024-04-25T10:17:58.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.647+0000 7f61f4d3a640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61dc066f60 0x7f61dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.647+0000 7f61f4d3a640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61dc066f60 0x7f61dc069420 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f61e4006fd0 tx=0x7f61e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.651+0000 7f61ed7fa640 1 -- 172.21.15.133:0/1566480644 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61e001f020 con 0x7f61f00ce330 2024-04-25T10:17:58.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.847+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1566480644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61f006c2d0 con 0x7f61f00ce330 2024-04-25T10:17:58.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.847+0000 7f61ed7fa640 1 -- 172.21.15.133:0/1566480644 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f61e00172c0 con 0x7f61f00ce330 2024-04-25T10:17:58.852 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:17:58.852 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":123,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:36.969368+0000","modified":"2024-04-25T10:17:57.769230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":196,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12135":{"gid":12135,"name":"c","rank":0,"incarnation":121,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.133:6837/2510709242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2510709242},{"type":"v1","addr":"172.21.15.133:6837","nonce":2510709242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:17:58.852 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 123 2024-04-25T10:17:58.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1566480644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61dc066f60 msgr2=0x7f61dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61dc066f60 0x7f61dc069420 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f61e4006fd0 tx=0x7f61e4008040 comp rx=0 tx=0).stop 2024-04-25T10:17:58.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1566480644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00ce330 msgr2=0x7f61f0153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00ce330 0x7f61f0153950 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7f61e000b410 tx=0x7f61e000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1566480644 shutdown_connections 2024-04-25T10:17:58.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f61f015c9a0 0x7f61f00cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f61dc066f60 0x7f61dc069420 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00ce330 0x7f61f0153950 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 --2- 172.21.15.133:0/1566480644 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f61f00b1f70 0x7f61f00cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1566480644 >> 172.21.15.133:0/1566480644 conn(0x7f61f0067000 msgr2=0x7f61f01513e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:58.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1566480644 shutdown_connections 2024-04-25T10:17:58.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:17:58.851+0000 7f61f6f9e640 1 -- 172.21.15.133:0/1566480644 wait complete. 2024-04-25T10:17:59.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 123, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:36.969368+0000', 'modified': '2024-04-25T10:17:57.769230+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 196, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12135': {'gid': 12135, 'name': 'c', 'rank': 0, 'incarnation': 121, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.133:6837/2510709242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 2510709242}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 2510709242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:17:59.273 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_12135:up:rejoin 2024-04-25T10:18:00.274 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:00.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.402+0000 7fd07f8f9640 1 Processor -- start 2024-04-25T10:18:00.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.402+0000 7fd07f8f9640 1 -- start start 2024-04-25T10:18:00.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07f8f9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07815c9a0 0x7fd0780c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07f8f9640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd0780b1f70 0x7fd07806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07f8f9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0780c61d0 con 0x7fd0780ce330 2024-04-25T10:18:00.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07f8f9640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0780c64b0 con 0x7fd0780b1f70 2024-04-25T10:18:00.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07f8f9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07806d4f0 con 0x7fd07815c9a0 2024-04-25T10:18:00.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07ce94640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd0780b1f70 0x7fd07806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07d695640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07815c9a0 0x7fd0780c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07ce94640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fd0780b1f70 0x7fd07806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:35638/0 (socket says 172.21.15.133:35638) 2024-04-25T10:18:00.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07de96640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fd0780ce330 0x7fd07815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:52136/0 (socket says 172.21.15.133:52136) 2024-04-25T10:18:00.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07d695640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07815c9a0 0x7fd0780c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:36760/0 (socket says 172.21.15.133:36760) 2024-04-25T10:18:00.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07ce94640 1 -- 172.21.15.133:0/2006718739 learned_addr learned my addr 172.21.15.133:0/2006718739 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:00.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07d695640 1 -- 172.21.15.133:0/2006718739 >> v2:172.21.15.162:3300/0 conn(0x7fd0780b1f70 msgr2=0x7fd07806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:00.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07d695640 1 --2- 172.21.15.133:0/2006718739 >> v2:172.21.15.162:3300/0 conn(0x7fd0780b1f70 0x7fd07806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07d695640 1 -- 172.21.15.133:0/2006718739 >> v1:172.21.15.162:6789/0 conn(0x7fd0780ce330 legacy=0x7fd07815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07d695640 1 -- 172.21.15.133:0/2006718739 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07806d7d0 con 0x7fd07815c9a0 2024-04-25T10:18:00.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.406+0000 7fd07ce94640 1 --2- 172.21.15.133:0/2006718739 >> v2:172.21.15.162:3300/0 conn(0x7fd0780b1f70 0x7fd07806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:00.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd07d695640 1 --2- 172.21.15.133:0/2006718739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07815c9a0 0x7fd0780c5980 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd0740097c0 tx=0x7fd07401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ae7eac954311320 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:00.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd06bfff640 1 -- 172.21.15.133:0/2006718739 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0740b0070 con 0x7fd07815c9a0 2024-04-25T10:18:00.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd06bfff640 1 -- 172.21.15.133:0/2006718739 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd074004230 con 0x7fd07815c9a0 2024-04-25T10:18:00.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd06bfff640 1 -- 172.21.15.133:0/2006718739 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074003620 con 0x7fd07815c9a0 2024-04-25T10:18:00.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2006718739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07815c9a0 msgr2=0x7fd0780c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd07f8f9640 1 --2- 172.21.15.133:0/2006718739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07815c9a0 0x7fd0780c5980 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd0740097c0 tx=0x7fd07401c440 comp rx=0 tx=0).stop 2024-04-25T10:18:00.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2006718739 shutdown_connections 2024-04-25T10:18:00.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd07f8f9640 1 --2- 172.21.15.133:0/2006718739 >> v2:172.21.15.162:3300/0 conn(0x7fd0780b1f70 0x7fd07806dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd07f8f9640 1 --2- 172.21.15.133:0/2006718739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd07815c9a0 0x7fd0780c5980 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2006718739 >> 172.21.15.133:0/2006718739 conn(0x7fd078067000 msgr2=0x7fd07806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:00.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2006718739 shutdown_connections 2024-04-25T10:18:00.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2006718739 wait complete. 2024-04-25T10:18:00.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd07f8f9640 1 Processor -- start 2024-04-25T10:18:00.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.410+0000 7fd07f8f9640 1 -- start start 2024-04-25T10:18:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07f8f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0780b1f70 0x7fd078148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07f8f9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0780ce330 0x7fd0781fb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07f8f9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd078157f20 0x7fd078156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07f8f9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780ceb50 con 0x7fd0780b1f70 2024-04-25T10:18:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07f8f9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07815cd80 con 0x7fd078157f20 2024-04-25T10:18:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07f8f9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd078157d30 con 0x7fd0780ce330 2024-04-25T10:18:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07d695640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0780b1f70 0x7fd078148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07d695640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0780b1f70 0x7fd078148b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42028/0 (socket says 172.21.15.133:42028) 2024-04-25T10:18:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07d695640 1 -- 172.21.15.133:0/2153552358 learned_addr learned my addr 172.21.15.133:0/2153552358 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:00.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07ce94640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0780ce330 0x7fd0781fb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07de96640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd078157f20 0x7fd078156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07d695640 1 -- 172.21.15.133:0/2153552358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0780ce330 msgr2=0x7fd0781fb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07d695640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0780ce330 0x7fd0781fb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07d695640 1 -- 172.21.15.133:0/2153552358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd078157f20 msgr2=0x7fd078156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07d695640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd078157f20 0x7fd078156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07d695640 1 -- 172.21.15.133:0/2153552358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078155c00 con 0x7fd0780b1f70 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07d695640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0780b1f70 0x7fd078148b40 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7fd0740190b0 tx=0x7fd074019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd069ffb640 1 -- 172.21.15.133:0/2153552358 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0740b9070 con 0x7fd0780b1f70 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd069ffb640 1 -- 172.21.15.133:0/2153552358 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd074003df0 con 0x7fd0780b1f70 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2153552358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd078155ec0 con 0x7fd0780b1f70 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd069ffb640 1 -- 172.21.15.133:0/2153552358 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0740b0040 con 0x7fd0780b1f70 2024-04-25T10:18:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2153552358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0781583b0 con 0x7fd0780b1f70 2024-04-25T10:18:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2153552358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07806bc20 con 0x7fd0780b1f70 2024-04-25T10:18:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd069ffb640 1 -- 172.21.15.133:0/2153552358 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0740bc070 con 0x7fd0780b1f70 2024-04-25T10:18:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd069ffb640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd05c066f60 0x7fd05c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd069ffb640 1 -- 172.21.15.133:0/2153552358 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd07411f4f0 con 0x7fd0780b1f70 2024-04-25T10:18:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07ce94640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd05c066f60 0x7fd05c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.414+0000 7fd07ce94640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd05c066f60 0x7fd05c069420 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd078156320 tx=0x7fd06c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:00.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.418+0000 7fd069ffb640 1 -- 172.21.15.133:0/2153552358 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd074016090 con 0x7fd0780b1f70 2024-04-25T10:18:00.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.610+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2153552358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd078068ef0 con 0x7fd0780b1f70 2024-04-25T10:18:00.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.610+0000 7fd069ffb640 1 -- 172.21.15.133:0/2153552358 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4963 (secure 0 0 0) 0x7fd0740f04d0 con 0x7fd0780b1f70 2024-04-25T10:18:00.616 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:18:00.616 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":124,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:36.969368+0000","modified":"2024-04-25T10:17:58.776693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":196,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12135":{"gid":12135,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/2510709242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2510709242},{"type":"v1","addr":"172.21.15.133:6837","nonce":2510709242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:18:00.616 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 124 2024-04-25T10:18:00.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2153552358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd05c066f60 msgr2=0x7fd05c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd05c066f60 0x7fd05c069420 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd078156320 tx=0x7fd06c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:00.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2153552358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0780b1f70 msgr2=0x7fd078148b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0780b1f70 0x7fd078148b40 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7fd0740190b0 tx=0x7fd074019240 comp rx=0 tx=0).stop 2024-04-25T10:18:00.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2153552358 shutdown_connections 2024-04-25T10:18:00.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd078157f20 0x7fd078156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fd0780ce330 0x7fd0781fb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fd05c066f60 0x7fd05c069420 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 --2- 172.21.15.133:0/2153552358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0780b1f70 0x7fd078148b40 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2153552358 >> 172.21.15.133:0/2153552358 conn(0x7fd078067000 msgr2=0x7fd078149750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:00.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2153552358 shutdown_connections 2024-04-25T10:18:00.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:00.614+0000 7fd07f8f9640 1 -- 172.21.15.133:0/2153552358 wait complete. 2024-04-25T10:18:01.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:36.969368+0000', 'modified': '2024-04-25T10:17:58.776693+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 196, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12135': {'gid': 12135, 'name': 'c', 'rank': 0, 'incarnation': 121, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.133:6837/2510709242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 2510709242}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 2510709242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:18:01.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:18:01.037 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:18:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.162+0000 7fcc38855640 1 Processor -- start 2024-04-25T10:18:01.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.162+0000 7fcc38855640 1 -- start start 2024-04-25T10:18:01.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.162+0000 7fcc38855640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcc340ce600 0x7fcc340cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.162+0000 7fcc38855640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340d1920 0x7fcc340d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.162+0000 7fcc38855640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3415ebd0 con 0x7fcc3415d9a0 2024-04-25T10:18:01.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.162+0000 7fcc38855640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3415eeb0 con 0x7fcc340ce600 2024-04-25T10:18:01.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.162+0000 7fcc38855640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340d0330 con 0x7fcc340d1920 2024-04-25T10:18:01.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340d1920 0x7fcc340d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc29d9b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcc340ce600 0x7fcc340cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340d1920 0x7fcc340d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:36772/0 (socket says 172.21.15.133:36772) 2024-04-25T10:18:01.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2070159231 learned_addr learned my addr 172.21.15.133:0/2070159231 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2070159231 >> v2:172.21.15.162:3300/0 conn(0x7fcc340ce600 msgr2=0x7fcc340cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/2070159231 >> v2:172.21.15.162:3300/0 conn(0x7fcc340ce600 0x7fcc340cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2070159231 >> v1:172.21.15.162:6789/0 conn(0x7fcc3415d9a0 legacy=0x7fcc340d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2070159231 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc340d0610 con 0x7fcc340d1920 2024-04-25T10:18:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/2070159231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340d1920 0x7fcc340d1db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009e50 tx=0x7fcc2c01c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30258e3922a6100 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc31d9b640 1 -- 172.21.15.133:0/2070159231 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0a5680 con 0x7fcc340d1920 2024-04-25T10:18:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc31d9b640 1 -- 172.21.15.133:0/2070159231 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2c0a5820 con 0x7fcc340d1920 2024-04-25T10:18:01.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.166+0000 7fcc31d9b640 1 -- 172.21.15.133:0/2070159231 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0a5bf0 con 0x7fcc340d1920 2024-04-25T10:18:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 -- 172.21.15.133:0/2070159231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340d1920 msgr2=0x7fcc340d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 --2- 172.21.15.133:0/2070159231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340d1920 0x7fcc340d1db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009e50 tx=0x7fcc2c01c9a0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 -- 172.21.15.133:0/2070159231 shutdown_connections 2024-04-25T10:18:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 --2- 172.21.15.133:0/2070159231 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340d1920 0x7fcc340d1db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 --2- 172.21.15.133:0/2070159231 >> v2:172.21.15.162:3300/0 conn(0x7fcc340ce600 0x7fcc340cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 -- 172.21.15.133:0/2070159231 >> 172.21.15.133:0/2070159231 conn(0x7fcc3406bf80 msgr2=0x7fcc3406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 -- 172.21.15.133:0/2070159231 shutdown_connections 2024-04-25T10:18:01.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 -- 172.21.15.133:0/2070159231 wait complete. 2024-04-25T10:18:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 Processor -- start 2024-04-25T10:18:01.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 -- start start 2024-04-25T10:18:01.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340ce600 0x7fcc340b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc340d1920 0x7fcc34158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc3415d9a0 0x7fcc340b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3415e710 con 0x7fcc340d1920 2024-04-25T10:18:01.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3415e590 con 0x7fcc3415d9a0 2024-04-25T10:18:01.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3415e890 con 0x7fcc340ce600 2024-04-25T10:18:01.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc29d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc340d1920 0x7fcc34158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc29d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc340d1920 0x7fcc34158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42038/0 (socket says 172.21.15.133:42038) 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc29d9b640 1 -- 172.21.15.133:0/3007974743 learned_addr learned my addr 172.21.15.133:0/3007974743 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc3415d9a0 0x7fcc340b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc3259c640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340ce600 0x7fcc340b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc29d9b640 1 -- 172.21.15.133:0/3007974743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340ce600 msgr2=0x7fcc340b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc29d9b640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340ce600 0x7fcc340b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc29d9b640 1 -- 172.21.15.133:0/3007974743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc3415d9a0 msgr2=0x7fcc340b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc29d9b640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc3415d9a0 0x7fcc340b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc29d9b640 1 -- 172.21.15.133:0/3007974743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc340b36f0 con 0x7fcc340d1920 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc29d9b640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc340d1920 0x7fcc34158ca0 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e870 tx=0x7fcc1c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc2b7fe640 1 -- 172.21.15.133:0/3007974743 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c017070 con 0x7fcc340d1920 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc2b7fe640 1 -- 172.21.15.133:0/3007974743 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c00c940 con 0x7fcc340d1920 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 -- 172.21.15.133:0/3007974743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc340b39d0 con 0x7fcc340d1920 2024-04-25T10:18:01.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc2b7fe640 1 -- 172.21.15.133:0/3007974743 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c00ccf0 con 0x7fcc340d1920 2024-04-25T10:18:01.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.170+0000 7fcc38855640 1 -- 172.21.15.133:0/3007974743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc34153490 con 0x7fcc340d1920 2024-04-25T10:18:01.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc2b7fe640 1 -- 172.21.15.133:0/3007974743 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc1c01a070 con 0x7fcc340d1920 2024-04-25T10:18:01.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc2b7fe640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc14066f60 0x7fcc14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc2b7fe640 1 -- 172.21.15.133:0/3007974743 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcc1c085560 con 0x7fcc340d1920 2024-04-25T10:18:01.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3259c640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc14066f60 0x7fcc14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3259c640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc14066f60 0x7fcc14069420 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcc3415aaa0 tx=0x7fcc24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3b9c2000 1 Processor -- start 2024-04-25T10:18:01.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3b9c2000 1 -- start start 2024-04-25T10:18:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21cdaed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cdaf240 0x55a21cd649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cd64f30 0x55a21cd65310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a21cc4c5f0 con 0x55a21cdae920 2024-04-25T10:18:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a21cbb3da0 con 0x55a21cd64f30 2024-04-25T10:18:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a21cb3c150 con 0x55a21cdaf240 2024-04-25T10:18:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21cdaed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21cdaed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42046/0 (socket says 172.21.15.133:42046) 2024-04-25T10:18:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3259c640 1 -- 172.21.15.133:0/1537653360 learned_addr learned my addr 172.21.15.133:0/1537653360 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/1537653360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cd64f30 0x55a21cd65310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc29d9b640 1 --2- 172.21.15.133:0/1537653360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cdaf240 0x55a21cd649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3259c640 1 -- 172.21.15.133:0/1537653360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cdaf240 msgr2=0x55a21cd649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3259c640 1 --2- 172.21.15.133:0/1537653360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cdaf240 0x55a21cd649f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3259c640 1 -- 172.21.15.133:0/1537653360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cd64f30 msgr2=0x55a21cd65310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3259c640 1 --2- 172.21.15.133:0/1537653360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cd64f30 0x55a21cd65310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3259c640 1 -- 172.21.15.133:0/1537653360 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a21cb55ca0 con 0x55a21cdae920 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/1537653360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cd64f30 0x55a21cd65310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc3259c640 1 --2- 172.21.15.133:0/1537653360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21cdaed00 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7fcc240052a0 tx=0x7fcc24011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645312f202b6ff1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc13fff640 1 -- 172.21.15.133:0/1537653360 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400e740 con 0x55a21cdae920 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc13fff640 1 -- 172.21.15.133:0/1537653360 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2400e8e0 con 0x55a21cdae920 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.174+0000 7fcc13fff640 1 -- 172.21.15.133:0/1537653360 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400ecb0 con 0x55a21cdae920 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1537653360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 msgr2=0x55a21cdaed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/1537653360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21cdaed00 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7fcc240052a0 tx=0x7fcc24011000 comp rx=0 tx=0).stop 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1537653360 shutdown_connections 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/1537653360 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cd64f30 0x55a21cd65310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/1537653360 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cdaf240 0x55a21cd649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/1537653360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21cdaed00 unknown :-1 s=CLOSED pgs=2938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1537653360 >> 172.21.15.133:0/1537653360 conn(0x55a21cd6b2a0 msgr2=0x55a21cd632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1537653360 shutdown_connections 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1537653360 wait complete. 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- start start 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 Processor -- start 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21cdb5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdae920 0x55a21cdb62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21cd68aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a21cb6e990 con 0x55a21cdaf240 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a21cbafe40 con 0x55a21cdae920 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a21cb3f900 con 0x55a21cd64f30 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 10 client.0 _ll_register_callbacks cb 0x55a21cc671c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21cd68aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21cd68aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42048/0 (socket says 172.21.15.133:42048) 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2516791728 learned_addr learned my addr 172.21.15.133:0/2516791728 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3259c640 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21cdb5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc29d9b640 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdae920 0x55a21cdb62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2516791728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 msgr2=0x55a21cdb5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21cdb5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2516791728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdae920 msgr2=0x55a21cdb62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdae920 0x55a21cdb62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2516791728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a21cb6c630 con 0x55a21cdaf240 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3259c640 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21cdb5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21cd68aa0 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c01b050 tx=0x7fcc2c01b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8846d1d9a250b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc117fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc117fa640 1 -- 172.21.15.133:0/2516791728 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0a59c0 con 0x55a21cdaf240 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc117fa640 1 -- 172.21.15.133:0/2516791728 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcc2c0a5b60 con 0x55a21cdaf240 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2516791728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc2400e8e0 con 0x55a21cdaf240 2024-04-25T10:18:01.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc117fa640 1 -- 172.21.15.133:0/2516791728 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc2c0c8050 con 0x55a21cdaf240 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 10 client.12192 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2516791728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x55a21cb3c3a0 con 0x55a21cdaf240 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc117fa640 1 -- 172.21.15.133:0/2516791728 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2400e8e0 con 0x55a21cdaf240 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc117fa640 1 -- 172.21.15.133:0/2516791728 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcc2c0cb020 con 0x55a21cdaf240 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 10 client.12192 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 10 client.12192 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 20 client.12192 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55a21cd694b0 0x55a21cd69890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 4 client.12192 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2516791728 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a21cb4fa50 con 0x55a21cd694b0 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3259c640 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55a21cd694b0 0x55a21cd69890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc3259c640 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55a21cd694b0 0x55a21cd69890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3bd1bb019d8cc9d server_cookie=69d8604a51d36834 in_seq=0 out_seq=0 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.178+0000 7fcc117fa640 10 client.12192 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc117fa640 1 -- 172.21.15.133:0/2516791728 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a21cb4fa50 con 0x55a21cd694b0 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc117fa640 10 client.12192 handle_command_reply: tid=0 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 1 client.12192 shutdown 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2516791728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 msgr2=0x55a21cd68aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21cd68aa0 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c01b050 tx=0x7fcc2c01b1e0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2516791728 shutdown_connections 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55a21cd694b0 0x55a21cd69890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21cd68aa0 unknown :-1 s=CLOSED pgs=2939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdae920 0x55a21cdb62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc117fa640 0 client.12192 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/2516791728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21cdb5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2516791728 >> 172.21.15.133:0/2516791728 conn(0x55a21cd6b2a0 msgr2=0x55a21cd639e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2516791728 shutdown_connections 2024-04-25T10:18:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2516791728 wait complete. 2024-04-25T10:18:01.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.182+0000 7fcc3b9c2000 20 client.12192 trim_cache size 0 max 16384 2024-04-25T10:18:01.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.194+0000 7fcc3b9c2000 1 Processor -- start 2024-04-25T10:18:01.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.194+0000 7fcc3b9c2000 1 -- start start 2024-04-25T10:18:01.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.194+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21ce0a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.194+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdae920 0x55a21ce0a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.194+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21ce0ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.194+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34202580 con 0x55a21cdaf240 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.194+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34157f00 con 0x55a21cdae920 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.194+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc342020a0 con 0x55a21cd64f30 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21ce0ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21ce0ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42060/0 (socket says 172.21.15.133:42060) 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2575164900 learned_addr learned my addr 172.21.15.133:0/2575164900 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3259c640 1 --2- 172.21.15.133:0/2575164900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21ce0a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc29d9b640 1 --2- 172.21.15.133:0/2575164900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdae920 0x55a21ce0a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2575164900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 msgr2=0x55a21ce0a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/2575164900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21ce0a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2575164900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdae920 msgr2=0x55a21ce0a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/2575164900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdae920 0x55a21ce0a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc32d9d640 1 -- 172.21.15.133:0/2575164900 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a21cdaf070 con 0x55a21cdaf240 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc29d9b640 1 --2- 172.21.15.133:0/2575164900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdae920 0x55a21ce0a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/2575164900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21ce0ab30 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c01a5d0 tx=0x7fcc2c0b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a682733ae2bb03e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc137fe640 1 -- 172.21.15.133:0/2575164900 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0c0440 con 0x55a21cdaf240 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc137fe640 1 -- 172.21.15.133:0/2575164900 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2c0d0070 con 0x55a21cdaf240 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc137fe640 1 -- 172.21.15.133:0/2575164900 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c003bc0 con 0x55a21cdaf240 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2575164900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 msgr2=0x55a21ce0ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/2575164900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21ce0ab30 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c01a5d0 tx=0x7fcc2c0b7000 comp rx=0 tx=0).stop 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2575164900 shutdown_connections 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/2575164900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdaf240 0x55a21ce0ab30 unknown :-1 s=CLOSED pgs=2940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/2575164900 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdae920 0x55a21ce0a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/2575164900 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21ce0a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2575164900 >> 172.21.15.133:0/2575164900 conn(0x55a21cd6b2a0 msgr2=0x55a21cd68220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2575164900 shutdown_connections 2024-04-25T10:18:01.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/2575164900 wait complete. 2024-04-25T10:18:01.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 -- start start 2024-04-25T10:18:01.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 Processor -- start 2024-04-25T10:18:01.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21cdbb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21ce0f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdaf240 0x55a21ce0f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a21cbb83d0 con 0x55a21cdae920 2024-04-25T10:18:01.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a21cc6fd50 con 0x55a21cdaf240 2024-04-25T10:18:01.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a21cb4e980 con 0x55a21cd64f30 2024-04-25T10:18:01.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 10 client.0 _ll_register_callbacks cb 0x7fcc24001330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc29d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21ce0f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc29d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21ce0f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42064/0 (socket says 172.21.15.133:42064) 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc29d9b640 1 -- 172.21.15.133:0/1365285690 learned_addr learned my addr 172.21.15.133:0/1365285690 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3259c640 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21cdbb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdaf240 0x55a21ce0f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc29d9b640 1 -- 172.21.15.133:0/1365285690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 msgr2=0x55a21cdbb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc29d9b640 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21cdbb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc29d9b640 1 -- 172.21.15.133:0/1365285690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdaf240 msgr2=0x55a21ce0f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc29d9b640 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdaf240 0x55a21ce0f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc29d9b640 1 -- 172.21.15.133:0/1365285690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a21cc35600 con 0x55a21cdae920 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc32d9d640 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdaf240 0x55a21ce0f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc29d9b640 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21ce0f120 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c055870 tx=0x7fcc1c0554d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fb4dcc31773f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc117fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc117fa640 1 -- 172.21.15.133:0/1365285690 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c021040 con 0x55a21cdae920 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc117fa640 1 -- 172.21.15.133:0/1365285690 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcc1c05dff0 con 0x55a21cdae920 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1365285690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc2c0d0070 con 0x55a21cdae920 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc117fa640 1 -- 172.21.15.133:0/1365285690 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc1c059cc0 con 0x55a21cdae920 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 10 client.12198 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.198+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1365285690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x55a21cb489b0 con 0x55a21cdae920 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc117fa640 1 -- 172.21.15.133:0/1365285690 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2c0d0070 con 0x55a21cdae920 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc117fa640 1 -- 172.21.15.133:0/1365285690 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcc1c0850c0 con 0x55a21cdae920 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 10 client.12198 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 10 client.12198 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 20 client.12198 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55a21ce0fba0 0x55a21ce0baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 4 client.12198 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1365285690 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a21cbc12e0 con 0x55a21ce0fba0 2024-04-25T10:18:01.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3259c640 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55a21ce0fba0 0x55a21ce0baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc117fa640 10 client.12198 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3259c640 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55a21ce0fba0 0x55a21ce0baa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4bd8f839f3e7caf server_cookie=2fc6b158fc308308 in_seq=0 out_seq=0 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc117fa640 1 -- 172.21.15.133:0/1365285690 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55a21cbc12e0 con 0x55a21ce0fba0 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc117fa640 10 client.12198 handle_command_reply: tid=0 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 client.12198 shutdown 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1365285690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 msgr2=0x55a21ce0f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21ce0f120 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c055870 tx=0x7fcc1c0554d0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1365285690 shutdown_connections 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55a21ce0fba0 0x55a21ce0baa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a21cdaf240 0x55a21ce0f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a21cdae920 0x55a21ce0f120 unknown :-1 s=CLOSED pgs=2941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 --2- 172.21.15.133:0/1365285690 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55a21cd64f30 0x55a21cdbb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc117fa640 0 client.12198 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1365285690 >> 172.21.15.133:0/1365285690 conn(0x55a21cd6b2a0 msgr2=0x55a21cdb6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1365285690 shutdown_connections 2024-04-25T10:18:01.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 1 -- 172.21.15.133:0/1365285690 wait complete. 2024-04-25T10:18:01.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc3b9c2000 20 client.12198 trim_cache size 0 max 16384 2024-04-25T10:18:01.213 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:18:01.213 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:18:01.213 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:18:01.213 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 12135, 2024-04-25T10:18:01.213 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:18:01.213 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:18:01.214 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:18:01.214 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 4.4452279309999998, 2024-04-25T10:18:01.214 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 124, 2024-04-25T10:18:01.214 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 196, 2024-04-25T10:18:01.214 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 196, 2024-04-25T10:18:01.214 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 5.4081307379999997 2024-04-25T10:18:01.214 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:18:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 -- 172.21.15.133:0/3007974743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc14066f60 msgr2=0x7fcc14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc14066f60 0x7fcc14069420 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcc3415aaa0 tx=0x7fcc24009040 comp rx=0 tx=0).stop 2024-04-25T10:18:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 -- 172.21.15.133:0/3007974743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc340d1920 msgr2=0x7fcc34158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc340d1920 0x7fcc34158ca0 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00e870 tx=0x7fcc1c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:18:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 -- 172.21.15.133:0/3007974743 shutdown_connections 2024-04-25T10:18:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc3415d9a0 0x7fcc340b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc14066f60 0x7fcc14069420 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc340d1920 0x7fcc34158ca0 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 --2- 172.21.15.133:0/3007974743 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc340ce600 0x7fcc340b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 -- 172.21.15.133:0/3007974743 >> 172.21.15.133:0/3007974743 conn(0x7fcc3406bf80 msgr2=0x7fcc340b7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 -- 172.21.15.133:0/3007974743 shutdown_connections 2024-04-25T10:18:01.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.202+0000 7fcc38855640 1 -- 172.21.15.133:0/3007974743 wait complete. 2024-04-25T10:18:01.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:01.222 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:18:01.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55bdfdf640 1 Processor -- start 2024-04-25T10:18:01.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55bdfdf640 1 -- start start 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55bdfdf640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f55b81407f0 0x7f55b80b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55bdfdf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55b80b2d40 0x7f55b815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55bdfdf640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b815edd0 con 0x7f55b8140bd0 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55bdfdf640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b80b47d0 con 0x7f55b81407f0 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55bdfdf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b80b4ab0 con 0x7f55b80b2d40 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55b6ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f55b8140bd0 0x7f55b80b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37718/0 (socket says 172.21.15.133:37718) 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55b6ffd640 1 -- 172.21.15.133:0/1687893829 learned_addr learned my addr 172.21.15.133:0/1687893829 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55b77fe640 1 --2- 172.21.15.133:0/1687893829 >> v2:172.21.15.133:3300/0 conn(0x7f55b81407f0 0x7f55b80b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55b7fff640 1 --2- 172.21.15.133:0/1687893829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55b80b2d40 0x7f55b815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.334+0000 7f55b67fc640 1 -- 172.21.15.133:0/1687893829 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073972821 0 0) 0x7f55b815edd0 con 0x7f55b8140bd0 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55b67fc640 1 -- 172.21.15.133:0/1687893829 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a0003530 con 0x7f55b8140bd0 2024-04-25T10:18:01.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55b77fe640 1 -- 172.21.15.133:0/1687893829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55b80b2d40 msgr2=0x7f55b815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55b77fe640 1 --2- 172.21.15.133:0/1687893829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55b80b2d40 0x7f55b815e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55b77fe640 1 -- 172.21.15.133:0/1687893829 >> v1:172.21.15.133:6789/0 conn(0x7f55b8140bd0 legacy=0x7f55b80b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55b77fe640 1 -- 172.21.15.133:0/1687893829 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b80b4d90 con 0x7f55b81407f0 2024-04-25T10:18:01.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55b7fff640 1 --2- 172.21.15.133:0/1687893829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55b80b2d40 0x7f55b815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:01.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55b77fe640 1 --2- 172.21.15.133:0/1687893829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55b81407f0 0x7f55b80b52b0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f55a80097c0 tx=0x7f55a801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcffa45cb30ecf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55b67fc640 1 -- 172.21.15.133:0/1687893829 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a80b1070 con 0x7f55b81407f0 2024-04-25T10:18:01.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55b67fc640 1 -- 172.21.15.133:0/1687893829 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a80ac410 con 0x7f55b81407f0 2024-04-25T10:18:01.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55b67fc640 1 -- 172.21.15.133:0/1687893829 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a80ac7c0 con 0x7f55b81407f0 2024-04-25T10:18:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55bdfdf640 1 -- 172.21.15.133:0/1687893829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55b81407f0 msgr2=0x7f55b80b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55bdfdf640 1 --2- 172.21.15.133:0/1687893829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55b81407f0 0x7f55b80b52b0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f55a80097c0 tx=0x7f55a801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55bdfdf640 1 -- 172.21.15.133:0/1687893829 shutdown_connections 2024-04-25T10:18:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55bdfdf640 1 --2- 172.21.15.133:0/1687893829 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55b80b2d40 0x7f55b815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55bdfdf640 1 --2- 172.21.15.133:0/1687893829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55b81407f0 0x7f55b80b52b0 unknown :-1 s=CLOSED pgs=2943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55bdfdf640 1 -- 172.21.15.133:0/1687893829 >> 172.21.15.133:0/1687893829 conn(0x7f55b806d370 msgr2=0x7f55b806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55bdfdf640 1 -- 172.21.15.133:0/1687893829 shutdown_connections 2024-04-25T10:18:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.338+0000 7f55bdfdf640 1 -- 172.21.15.133:0/1687893829 wait complete. 2024-04-25T10:18:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55bdfdf640 1 Processor -- start 2024-04-25T10:18:01.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55bdfdf640 1 -- start start 2024-04-25T10:18:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55bdfdf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55b80b2d40 0x7f55b813b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55bdfdf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55b81407f0 0x7f55b8151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55bdfdf640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55b8140bd0 0x7f55b8139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55bdfdf640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8071cd0 con 0x7f55b81407f0 2024-04-25T10:18:01.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55bdfdf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8071b50 con 0x7f55b80b2d40 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55bdfdf640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8071e50 con 0x7f55b8140bd0 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55b6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55b81407f0 0x7f55b8151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55b6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55b81407f0 0x7f55b8151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42080/0 (socket says 172.21.15.133:42080) 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55b6ffd640 1 -- 172.21.15.133:0/2011118657 learned_addr learned my addr 172.21.15.133:0/2011118657 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55b6ffd640 1 -- 172.21.15.133:0/2011118657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55b8140bd0 msgr2=0x7f55b8139650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55b6ffd640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55b8140bd0 0x7f55b8139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55b77fe640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55b80b2d40 0x7f55b813b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55b6ffd640 1 -- 172.21.15.133:0/2011118657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55b80b2d40 msgr2=0x7f55b813b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55b6ffd640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55b80b2d40 0x7f55b813b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55b6ffd640 1 -- 172.21.15.133:0/2011118657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b813b6f0 con 0x7f55b81407f0 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55b6ffd640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55b81407f0 0x7f55b8151e90 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00c430 tx=0x7f55ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f559ffff640 1 -- 172.21.15.133:0/2011118657 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac017070 con 0x7f55b81407f0 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f559ffff640 1 -- 172.21.15.133:0/2011118657 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac005cf0 con 0x7f55b81407f0 2024-04-25T10:18:01.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55bdfdf640 1 -- 172.21.15.133:0/2011118657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b8152c40 con 0x7f55b81407f0 2024-04-25T10:18:01.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f559ffff640 1 -- 172.21.15.133:0/2011118657 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0072d0 con 0x7f55b81407f0 2024-04-25T10:18:01.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.342+0000 7f55bdfdf640 1 -- 172.21.15.133:0/2011118657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b8138e50 con 0x7f55b81407f0 2024-04-25T10:18:01.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f559ffff640 1 -- 172.21.15.133:0/2011118657 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55ac01a070 con 0x7f55b81407f0 2024-04-25T10:18:01.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f559ffff640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a4066f60 0x7f55a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f559ffff640 1 -- 172.21.15.133:0/2011118657 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f55ac0845f0 con 0x7f55b81407f0 2024-04-25T10:18:01.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b77fe640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a4066f60 0x7f55a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b77fe640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a4066f60 0x7f55a4069420 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f55a80ae000 tx=0x7f55a80a6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 Processor -- start 2024-04-25T10:18:01.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 -- start start 2024-04-25T10:18:01.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd6c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c45b0 0x564cbd679d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 0x564cbd67a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cbd560f40 con 0x564cbd6c3c90 2024-04-25T10:18:01.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cbd4c8070 con 0x564cbd6c45b0 2024-04-25T10:18:01.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cbd4501a0 con 0x564cbd67a290 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd6c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd6c4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42086/0 (socket says 172.21.15.133:42086) 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b77fe640 1 -- 172.21.15.133:0/864050009 learned_addr learned my addr 172.21.15.133:0/864050009 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b7fff640 1 --2- 172.21.15.133:0/864050009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 0x564cbd67a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b6ffd640 1 --2- 172.21.15.133:0/864050009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c45b0 0x564cbd679d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b77fe640 1 -- 172.21.15.133:0/864050009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 msgr2=0x564cbd67a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b77fe640 1 --2- 172.21.15.133:0/864050009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 0x564cbd67a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b77fe640 1 -- 172.21.15.133:0/864050009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c45b0 msgr2=0x564cbd679d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b77fe640 1 --2- 172.21.15.133:0/864050009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c45b0 0x564cbd679d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b77fe640 1 -- 172.21.15.133:0/864050009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cbd44e510 con 0x564cbd6c3c90 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55b77fe640 1 --2- 172.21.15.133:0/864050009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd6c4070 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f55a8000c10 tx=0x7f55a80a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a02880bd06c8c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f559d7fa640 1 -- 172.21.15.133:0/864050009 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a80c7070 con 0x564cbd6c3c90 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f559d7fa640 1 -- 172.21.15.133:0/864050009 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a80acd00 con 0x564cbd6c3c90 2024-04-25T10:18:01.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f559d7fa640 1 -- 172.21.15.133:0/864050009 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a80b1040 con 0x564cbd6c3c90 2024-04-25T10:18:01.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 -- 172.21.15.133:0/864050009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 msgr2=0x564cbd6c4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 --2- 172.21.15.133:0/864050009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd6c4070 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f55a8000c10 tx=0x7f55a80a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:01.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 -- 172.21.15.133:0/864050009 shutdown_connections 2024-04-25T10:18:01.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 --2- 172.21.15.133:0/864050009 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 0x564cbd67a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 --2- 172.21.15.133:0/864050009 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c45b0 0x564cbd679d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 --2- 172.21.15.133:0/864050009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd6c4070 unknown :-1 s=CLOSED pgs=2945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 -- 172.21.15.133:0/864050009 >> 172.21.15.133:0/864050009 conn(0x564cbd676240 msgr2=0x564cbd678630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 -- 172.21.15.133:0/864050009 shutdown_connections 2024-04-25T10:18:01.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.346+0000 7f55c114c000 1 -- 172.21.15.133:0/864050009 wait complete. 2024-04-25T10:18:01.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- start start 2024-04-25T10:18:01.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 Processor -- start 2024-04-25T10:18:01.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 0x564cbd67f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c3c90 0x564cbd6b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c45b0 0x564cbd6b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cbd4826c0 con 0x564cbd6c45b0 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cbd4c4110 con 0x564cbd6c3c90 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cbd6760c0 con 0x564cbd67a290 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 10 client.0 _ll_register_callbacks cb 0x564cbd665570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c45b0 0x564cbd6b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c45b0 0x564cbd6b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42088/0 (socket says 172.21.15.133:42088) 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b7fff640 1 -- 172.21.15.133:0/4004180519 learned_addr learned my addr 172.21.15.133:0/4004180519 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b6ffd640 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c3c90 0x564cbd6b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b77fe640 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 0x564cbd67f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b7fff640 1 -- 172.21.15.133:0/4004180519 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 msgr2=0x564cbd67f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b7fff640 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 0x564cbd67f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b7fff640 1 -- 172.21.15.133:0/4004180519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c3c90 msgr2=0x564cbd6b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b7fff640 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c3c90 0x564cbd6b0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b7fff640 1 -- 172.21.15.133:0/4004180519 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cbd4503b0 con 0x564cbd6c45b0 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b7fff640 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c45b0 0x564cbd6b1450 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7f55b0007940 tx=0x7f55b0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c1da3728b5fe79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55967fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55967fc640 1 -- 172.21.15.133:0/4004180519 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b000ebf0 con 0x564cbd6c45b0 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55967fc640 1 -- 172.21.15.133:0/4004180519 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f55b000ed90 con 0x564cbd6c45b0 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- 172.21.15.133:0/4004180519 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55a80acd00 con 0x564cbd6c45b0 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55967fc640 1 -- 172.21.15.133:0/4004180519 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55b0022050 con 0x564cbd6c45b0 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 10 client.12213 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- 172.21.15.133:0/4004180519 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x564cbd549e00 con 0x564cbd6c45b0 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55967fc640 1 -- 172.21.15.133:0/4004180519 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a80acd00 con 0x564cbd6c45b0 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55967fc640 1 -- 172.21.15.133:0/4004180519 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f55b0025020 con 0x564cbd6c45b0 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 10 client.12213 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 10 client.12213 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 20 client.12213 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x564cbd6ca4e0 0x564cbd6ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 4 client.12213 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- 172.21.15.133:0/4004180519 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cbd463d60 con 0x564cbd6ca4e0 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b77fe640 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x564cbd6ca4e0 0x564cbd6ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55b77fe640 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x564cbd6ca4e0 0x564cbd6ca8c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0e5495fd6d6cced server_cookie=aa0460dde2d855c2 in_seq=0 out_seq=0 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55967fc640 10 client.12213 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55967fc640 1 -- 172.21.15.133:0/4004180519 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564cbd463d60 con 0x564cbd6ca4e0 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55967fc640 10 client.12213 handle_command_reply: tid=0 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 client.12213 shutdown 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- 172.21.15.133:0/4004180519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c45b0 msgr2=0x564cbd6b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c45b0 0x564cbd6b1450 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7f55b0007940 tx=0x7f55b0007e00 comp rx=0 tx=0).stop 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- 172.21.15.133:0/4004180519 shutdown_connections 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x564cbd6ca4e0 0x564cbd6ca8c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c45b0 0x564cbd6b1450 unknown :-1 s=CLOSED pgs=2946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c3c90 0x564cbd6b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55967fc640 0 client.12213 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 --2- 172.21.15.133:0/4004180519 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 0x564cbd67f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- 172.21.15.133:0/4004180519 >> 172.21.15.133:0/4004180519 conn(0x564cbd676240 msgr2=0x564cbd678ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- 172.21.15.133:0/4004180519 shutdown_connections 2024-04-25T10:18:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 1 -- 172.21.15.133:0/4004180519 wait complete. 2024-04-25T10:18:01.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.350+0000 7f55c114c000 20 client.12213 trim_cache size 0 max 16384 2024-04-25T10:18:01.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.366+0000 7f55c114c000 1 Processor -- start 2024-04-25T10:18:01.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.366+0000 7f55c114c000 1 -- start start 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.366+0000 7f55c114c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 0x564cbd728cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.366+0000 7f55c114c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd729230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.366+0000 7f55c114c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c45b0 0x564cbd729770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.366+0000 7f55c114c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8203530 con 0x564cbd6c3c90 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.366+0000 7f55c114c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b815fa50 con 0x564cbd6c45b0 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.366+0000 7f55c114c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b815ee50 con 0x564cbd67a290 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.366+0000 7f55b6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd729230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.366+0000 7f55b6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd729230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42104/0 (socket says 172.21.15.133:42104) 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.366+0000 7f55b6ffd640 1 -- 172.21.15.133:0/2285368488 learned_addr learned my addr 172.21.15.133:0/2285368488 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b6ffd640 1 -- 172.21.15.133:0/2285368488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 msgr2=0x564cbd728cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b6ffd640 1 --2- 172.21.15.133:0/2285368488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 0x564cbd728cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b6ffd640 1 -- 172.21.15.133:0/2285368488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c45b0 msgr2=0x564cbd729770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b6ffd640 1 --2- 172.21.15.133:0/2285368488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c45b0 0x564cbd729770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b6ffd640 1 -- 172.21.15.133:0/2285368488 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cbd51fc50 con 0x564cbd6c3c90 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b6ffd640 1 --2- 172.21.15.133:0/2285368488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd729230 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f55ac05cfc0 tx=0x7f55ac054ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8cb6f0ac4db8d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f559cff9640 1 -- 172.21.15.133:0/2285368488 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0530e0 con 0x564cbd6c3c90 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f559cff9640 1 -- 172.21.15.133:0/2285368488 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac059e50 con 0x564cbd6c3c90 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f559cff9640 1 -- 172.21.15.133:0/2285368488 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac051450 con 0x564cbd6c3c90 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- 172.21.15.133:0/2285368488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 msgr2=0x564cbd729230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 --2- 172.21.15.133:0/2285368488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd729230 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f55ac05cfc0 tx=0x7f55ac054ea0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- 172.21.15.133:0/2285368488 shutdown_connections 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 --2- 172.21.15.133:0/2285368488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c45b0 0x564cbd729770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 --2- 172.21.15.133:0/2285368488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd6c3c90 0x564cbd729230 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 --2- 172.21.15.133:0/2285368488 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd67a290 0x564cbd728cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- 172.21.15.133:0/2285368488 >> 172.21.15.133:0/2285368488 conn(0x564cbd676240 msgr2=0x564cbd678630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- 172.21.15.133:0/2285368488 shutdown_connections 2024-04-25T10:18:01.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- 172.21.15.133:0/2285368488 wait complete. 2024-04-25T10:18:01.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- start start 2024-04-25T10:18:01.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 Processor -- start 2024-04-25T10:18:01.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd67a290 0x564cbd6c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c3c90 0x564cbd6e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd6c45b0 0x564cbd6e9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cbd683200 con 0x564cbd67a290 2024-04-25T10:18:01.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b815f450 con 0x564cbd6c3c90 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x564cbd4c7380 con 0x564cbd6c45b0 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 10 client.0 _ll_register_callbacks cb 0x7f55a80af070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd67a290 0x564cbd6c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd67a290 0x564cbd6c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42110/0 (socket says 172.21.15.133:42110) 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b77fe640 1 -- 172.21.15.133:0/1058524396 learned_addr learned my addr 172.21.15.133:0/1058524396 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b6ffd640 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c3c90 0x564cbd6e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b7fff640 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd6c45b0 0x564cbd6e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b77fe640 1 -- 172.21.15.133:0/1058524396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd6c45b0 msgr2=0x564cbd6e9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b77fe640 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd6c45b0 0x564cbd6e9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b77fe640 1 -- 172.21.15.133:0/1058524396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c3c90 msgr2=0x564cbd6e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b77fe640 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c3c90 0x564cbd6e92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b77fe640 1 -- 172.21.15.133:0/1058524396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cbd490120 con 0x564cbd67a290 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b77fe640 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd67a290 0x564cbd6c90b0 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009580 tx=0x7f55a8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2118d849c7dcc6fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55967fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55967fc640 1 -- 172.21.15.133:0/1058524396 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a80d2060 con 0x564cbd67a290 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55967fc640 1 -- 172.21.15.133:0/1058524396 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f55a80043a0 con 0x564cbd67a290 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- 172.21.15.133:0/1058524396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55ac059e50 con 0x564cbd67a290 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55967fc640 1 -- 172.21.15.133:0/1058524396 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55a80d8050 con 0x564cbd67a290 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 10 client.12219 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- 172.21.15.133:0/1058524396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x564cbd46f170 con 0x564cbd67a290 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55967fc640 1 -- 172.21.15.133:0/1058524396 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac059e50 con 0x564cbd67a290 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55967fc640 1 -- 172.21.15.133:0/1058524396 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f55a80db020 con 0x564cbd67a290 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 10 client.12219 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 10 client.12219 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 20 client.12219 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x564cbd72c910 0x564cbd72cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 4 client.12219 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:18:01.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55c114c000 1 -- 172.21.15.133:0/1058524396 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x564cbd4d55b0 con 0x564cbd72c910 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.370+0000 7f55b6ffd640 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x564cbd72c910 0x564cbd72cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55b6ffd640 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x564cbd72c910 0x564cbd72cd20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e3c88b4f78b779 server_cookie=a5511da3cb9ff286 in_seq=0 out_seq=0 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55967fc640 10 client.12219 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55967fc640 1 -- 172.21.15.133:0/1058524396 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564cbd4d55b0 con 0x564cbd72c910 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55967fc640 10 client.12219 handle_command_reply: tid=0 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 1 client.12219 shutdown 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 1 -- 172.21.15.133:0/1058524396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd67a290 msgr2=0x564cbd6c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd67a290 0x564cbd6c90b0 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009580 tx=0x7f55a8019240 comp rx=0 tx=0).stop 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 1 -- 172.21.15.133:0/1058524396 shutdown_connections 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x564cbd72c910 0x564cbd72cd20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x564cbd6c45b0 0x564cbd6e9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x564cbd6c3c90 0x564cbd6e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55967fc640 0 client.12219 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 1 --2- 172.21.15.133:0/1058524396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cbd67a290 0x564cbd6c90b0 unknown :-1 s=CLOSED pgs=2948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 1 -- 172.21.15.133:0/1058524396 >> 172.21.15.133:0/1058524396 conn(0x564cbd676240 msgr2=0x564cbd678ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 1 -- 172.21.15.133:0/1058524396 shutdown_connections 2024-04-25T10:18:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 1 -- 172.21.15.133:0/1058524396 wait complete. 2024-04-25T10:18:01.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55c114c000 20 client.12219 trim_cache size 0 max 16384 2024-04-25T10:18:01.383 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:18:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:18:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "b51d7dc2-4157-4ba7-9602-225b1f33e599", 2024-04-25T10:18:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:18:01.383 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:18:01.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 -- 172.21.15.133:0/2011118657 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a4066f60 msgr2=0x7f55a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a4066f60 0x7f55a4069420 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f55a80ae000 tx=0x7f55a80a6040 comp rx=0 tx=0).stop 2024-04-25T10:18:01.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 -- 172.21.15.133:0/2011118657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55b81407f0 msgr2=0x7f55b8151e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55b81407f0 0x7f55b8151e90 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00c430 tx=0x7f55ac00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 -- 172.21.15.133:0/2011118657 shutdown_connections 2024-04-25T10:18:01.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f55b8140bd0 0x7f55b8139650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f55a4066f60 0x7f55a4069420 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55b81407f0 0x7f55b8151e90 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 --2- 172.21.15.133:0/2011118657 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55b80b2d40 0x7f55b813b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 -- 172.21.15.133:0/2011118657 >> 172.21.15.133:0/2011118657 conn(0x7f55b806d370 msgr2=0x7f55b80b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 -- 172.21.15.133:0/2011118657 shutdown_connections 2024-04-25T10:18:01.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.374+0000 7f55bdfdf640 1 -- 172.21.15.133:0/2011118657 wait complete. 2024-04-25T10:18:01.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:01.395 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:18:01.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.502+0000 7f1bd7b4a640 1 Processor -- start 2024-04-25T10:18:01.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.502+0000 7f1bd7b4a640 1 -- start start 2024-04-25T10:18:01.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.502+0000 7f1bd7b4a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1bd0145170 0x7f1bd006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.502+0000 7f1bd7b4a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd006e4d0 0x7f1bd00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.502+0000 7f1bd7b4a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd00ce2e0 con 0x7f1bd00cdd60 2024-04-25T10:18:01.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.502+0000 7f1bd7b4a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd006db00 con 0x7f1bd0145170 2024-04-25T10:18:01.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.506+0000 7f1bd7b4a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd0145550 con 0x7f1bd006e4d0 2024-04-25T10:18:01.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.506+0000 7f1bd58e6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1bd0145170 0x7f1bd006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.506+0000 7f1bd50e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd006e4d0 0x7f1bd00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.506+0000 7f1bd50e5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd006e4d0 0x7f1bd00cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:36890/0 (socket says 172.21.15.133:36890) 2024-04-25T10:18:01.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.506+0000 7f1bd50e5640 1 -- 172.21.15.133:0/859244528 learned_addr learned my addr 172.21.15.133:0/859244528 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.506+0000 7f1bd50e5640 1 -- 172.21.15.133:0/859244528 >> v2:172.21.15.162:3300/0 conn(0x7f1bd0145170 msgr2=0x7f1bd006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.506+0000 7f1bd50e5640 1 --2- 172.21.15.133:0/859244528 >> v2:172.21.15.162:3300/0 conn(0x7f1bd0145170 0x7f1bd006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.506+0000 7f1bd50e5640 1 -- 172.21.15.133:0/859244528 >> v1:172.21.15.162:6789/0 conn(0x7f1bd00cdd60 legacy=0x7f1bd015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.506+0000 7f1bd50e5640 1 -- 172.21.15.133:0/859244528 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd006d4b0 con 0x7f1bd006e4d0 2024-04-25T10:18:01.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.506+0000 7f1bd48e4640 1 -- 172.21.15.133:0/859244528 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437406254 0 0) 0x7f1bd00ce2e0 con 0x7f1bd00cdd60 2024-04-25T10:18:01.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.506+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/859244528 >> v2:172.21.15.162:3300/0 conn(0x7f1bd0145170 0x7f1bd006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:01.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd50e5640 1 --2- 172.21.15.133:0/859244528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd006e4d0 0x7f1bd00cd820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009870 tx=0x7f1bc001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba10b74656c585b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd48e4640 1 -- 172.21.15.133:0/859244528 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00b2070 con 0x7f1bd006e4d0 2024-04-25T10:18:01.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd48e4640 1 -- 172.21.15.133:0/859244528 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc00a9070 con 0x7f1bd006e4d0 2024-04-25T10:18:01.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd48e4640 1 -- 172.21.15.133:0/859244528 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00ad880 con 0x7f1bd006e4d0 2024-04-25T10:18:01.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/859244528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd006e4d0 msgr2=0x7f1bd00cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 --2- 172.21.15.133:0/859244528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd006e4d0 0x7f1bd00cd820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009870 tx=0x7f1bc001c560 comp rx=0 tx=0).stop 2024-04-25T10:18:01.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/859244528 shutdown_connections 2024-04-25T10:18:01.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 --2- 172.21.15.133:0/859244528 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd006e4d0 0x7f1bd00cd820 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 --2- 172.21.15.133:0/859244528 >> v2:172.21.15.162:3300/0 conn(0x7f1bd0145170 0x7f1bd006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/859244528 >> 172.21.15.133:0/859244528 conn(0x7f1bd0066f70 msgr2=0x7f1bd006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/859244528 shutdown_connections 2024-04-25T10:18:01.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/859244528 wait complete. 2024-04-25T10:18:01.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 Processor -- start 2024-04-25T10:18:01.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 -- start start 2024-04-25T10:18:01.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd00cdd60 0x7f1bd01612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd0145170 0x7f1bd01398b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd01648d0 0x7f1bd0164d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd0168490 con 0x7f1bd01648d0 2024-04-25T10:18:01.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd015d7f0 con 0x7f1bd00cdd60 2024-04-25T10:18:01.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd7b4a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd0168610 con 0x7f1bd0145170 2024-04-25T10:18:01.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd60e7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd01648d0 0x7f1bd0164d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd60e7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd01648d0 0x7f1bd0164d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42112/0 (socket says 172.21.15.133:42112) 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.510+0000 7f1bd60e7640 1 -- 172.21.15.133:0/1112631034 learned_addr learned my addr 172.21.15.133:0/1112631034 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd00cdd60 0x7f1bd01612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd50e5640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd0145170 0x7f1bd01398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd60e7640 1 -- 172.21.15.133:0/1112631034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd0145170 msgr2=0x7f1bd01398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd60e7640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd0145170 0x7f1bd01398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd60e7640 1 -- 172.21.15.133:0/1112631034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd00cdd60 msgr2=0x7f1bd01612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd60e7640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd00cdd60 0x7f1bd01612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd60e7640 1 -- 172.21.15.133:0/1112631034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd0067530 con 0x7f1bd01648d0 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd50e5640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd0145170 0x7f1bd01398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd00cdd60 0x7f1bd01612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd60e7640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd01648d0 0x7f1bd0164d50 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800c780 tx=0x7f1bc800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bc67fc640 1 -- 172.21.15.133:0/1112631034 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8003db0 con 0x7f1bd01648d0 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/1112631034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd013a4d0 con 0x7f1bd01648d0 2024-04-25T10:18:01.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bc67fc640 1 -- 172.21.15.133:0/1112631034 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc8005070 con 0x7f1bd01648d0 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bc67fc640 1 -- 172.21.15.133:0/1112631034 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8005460 con 0x7f1bd01648d0 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/1112631034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd013aa90 con 0x7f1bd01648d0 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bc67fc640 1 -- 172.21.15.133:0/1112631034 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1bc8005600 con 0x7f1bd01648d0 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bc67fc640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1bb4066f60 0x7f1bb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bc67fc640 1 -- 172.21.15.133:0/1112631034 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1bc8085170 con 0x7f1bd01648d0 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1bb4066f60 0x7f1bb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1bb4066f60 0x7f1bb4069420 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1bd01f8020 tx=0x7f1bcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bdacb7000 1 Processor -- start 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.514+0000 7f1bdacb7000 1 -- start start 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211139d0 0x556121113db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 0x5561210c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x5561210ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556120f16ff0 con 0x5561210ca0a0 2024-04-25T10:18:01.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556120e9f150 con 0x5561211139d0 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556120f13090 con 0x5561211142f0 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd60e7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x5561210ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd60e7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x5561210ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42120/0 (socket says 172.21.15.133:42120) 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd60e7640 1 -- 172.21.15.133:0/2854243797 learned_addr learned my addr 172.21.15.133:0/2854243797 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd60e7640 1 -- 172.21.15.133:0/2854243797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 msgr2=0x5561210c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd60e7640 1 --2- 172.21.15.133:0/2854243797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 0x5561210c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd60e7640 1 -- 172.21.15.133:0/2854243797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211139d0 msgr2=0x556121113db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd60e7640 1 --2- 172.21.15.133:0/2854243797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211139d0 0x556121113db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd60e7640 1 -- 172.21.15.133:0/2854243797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561210ad0d0 con 0x5561210ca0a0 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd60e7640 1 --2- 172.21.15.133:0/2854243797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x5561210ca480 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f1bc805da90 tx=0x7f1bc800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1d0bcb9b72a4ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bb37fe640 1 -- 172.21.15.133:0/2854243797 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc806d020 con 0x5561210ca0a0 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bb37fe640 1 -- 172.21.15.133:0/2854243797 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc8059a00 con 0x5561210ca0a0 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bb37fe640 1 -- 172.21.15.133:0/2854243797 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc805dfc0 con 0x5561210ca0a0 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2854243797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 msgr2=0x5561210ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2854243797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x5561210ca480 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f1bc805da90 tx=0x7f1bc800cf90 comp rx=0 tx=0).stop 2024-04-25T10:18:01.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2854243797 shutdown_connections 2024-04-25T10:18:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2854243797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x5561210ca480 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2854243797 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 0x5561210c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2854243797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211139d0 0x556121113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2854243797 >> 172.21.15.133:0/2854243797 conn(0x5561210c6020 msgr2=0x5561210c8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2854243797 shutdown_connections 2024-04-25T10:18:01.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2854243797 wait complete. 2024-04-25T10:18:01.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- start start 2024-04-25T10:18:01.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 Processor -- start 2024-04-25T10:18:01.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561210ca0a0 0x5561210f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561210f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 0x5561210f9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556120edb250 con 0x5561211139d0 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556120ed1690 con 0x5561210ca0a0 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556120e9fa40 con 0x5561211142f0 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 10 client.0 _ll_register_callbacks cb 0x5561210b4e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd50e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561210f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd50e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561210f8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42124/0 (socket says 172.21.15.133:42124) 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd50e5640 1 -- 172.21.15.133:0/2705246972 learned_addr learned my addr 172.21.15.133:0/2705246972 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561210ca0a0 0x5561210f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd60e7640 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 0x5561210f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd50e5640 1 -- 172.21.15.133:0/2705246972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 msgr2=0x5561210f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd50e5640 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 0x5561210f9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd50e5640 1 -- 172.21.15.133:0/2705246972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561210ca0a0 msgr2=0x5561210f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd50e5640 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561210ca0a0 0x5561210f87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd50e5640 1 -- 172.21.15.133:0/2705246972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556120fc41c0 con 0x5561211139d0 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bd50e5640 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561210f8d30 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00b0040 tx=0x7f1bc001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b5d067ee4042 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/2705246972 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00b2040 con 0x5561211139d0 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/2705246972 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1bc00adba0 con 0x5561211139d0 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.518+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2705246972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bc8059a00 con 0x5561211139d0 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/2705246972 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bc00c6050 con 0x5561211139d0 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 10 client.12228 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2705246972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x556120fc0170 con 0x5561211139d0 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/2705246972 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc8059a00 con 0x5561211139d0 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/2705246972 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1bc00c9020 con 0x5561211139d0 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 10 client.12228 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 10 client.12228 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 20 client.12228 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:01.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556121123c80 0x556121126070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 4 client.12228 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2705246972 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556120eb2c60 con 0x556121123c80 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556121123c80 0x556121126070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556121123c80 0x556121126070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e0c90fe9dd6d460 server_cookie=a1553c6ac090a8f6 in_seq=0 out_seq=0 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bb0ff9640 10 client.12228 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/2705246972 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556120eb2c60 con 0x556121123c80 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bb0ff9640 10 client.12228 handle_command_reply: tid=0 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 client.12228 shutdown 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2705246972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 msgr2=0x5561210f8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561210f8d30 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00b0040 tx=0x7f1bc001ada0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2705246972 shutdown_connections 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556121123c80 0x556121126070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 0x5561210f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561210f8d30 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2705246972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561210ca0a0 0x5561210f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bb0ff9640 0 client.12228 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2705246972 >> 172.21.15.133:0/2705246972 conn(0x5561210c6020 msgr2=0x5561210c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2705246972 shutdown_connections 2024-04-25T10:18:01.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2705246972 wait complete. 2024-04-25T10:18:01.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.522+0000 7f1bdacb7000 20 client.12228 trim_cache size 0 max 16384 2024-04-25T10:18:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 Processor -- start 2024-04-25T10:18:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 -- start start 2024-04-25T10:18:01.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x55612116fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211139d0 0x556121170420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 0x556121170960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd02017a0 con 0x5561210ca0a0 2024-04-25T10:18:01.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd0200960 con 0x5561211139d0 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd01629f0 con 0x5561211142f0 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bd58e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x55612116fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bd58e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x55612116fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42140/0 (socket says 172.21.15.133:42140) 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bd58e6640 1 -- 172.21.15.133:0/2972090285 learned_addr learned my addr 172.21.15.133:0/2972090285 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bd60e7640 1 --2- 172.21.15.133:0/2972090285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 0x556121170960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bd50e5640 1 --2- 172.21.15.133:0/2972090285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211139d0 0x556121170420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bd58e6640 1 -- 172.21.15.133:0/2972090285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 msgr2=0x556121170960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/2972090285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 0x556121170960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bd58e6640 1 -- 172.21.15.133:0/2972090285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211139d0 msgr2=0x556121170420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/2972090285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211139d0 0x556121170420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bd58e6640 1 -- 172.21.15.133:0/2972090285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556120f5dde0 con 0x5561210ca0a0 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/2972090285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x55612116fee0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0090e0 tx=0x7f1bcc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edfdbf87a47273f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bb2ffd640 1 -- 172.21.15.133:0/2972090285 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc029030 con 0x5561210ca0a0 2024-04-25T10:18:01.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bb2ffd640 1 -- 172.21.15.133:0/2972090285 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc0169e0 con 0x5561210ca0a0 2024-04-25T10:18:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bb2ffd640 1 -- 172.21.15.133:0/2972090285 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc00f040 con 0x5561210ca0a0 2024-04-25T10:18:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2972090285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 msgr2=0x55612116fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2972090285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x55612116fee0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0090e0 tx=0x7f1bcc013000 comp rx=0 tx=0).stop 2024-04-25T10:18:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2972090285 shutdown_connections 2024-04-25T10:18:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2972090285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561211142f0 0x556121170960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2972090285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211139d0 0x556121170420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/2972090285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561210ca0a0 0x55612116fee0 unknown :-1 s=CLOSED pgs=2952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2972090285 >> 172.21.15.133:0/2972090285 conn(0x5561210c6020 msgr2=0x5561211120d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2972090285 shutdown_connections 2024-04-25T10:18:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 -- 172.21.15.133:0/2972090285 wait complete. 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 -- start start 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.538+0000 7f1bdacb7000 1 Processor -- start 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561210ca0a0 0x556121126380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561211268c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211142f0 0x556121126e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556120eb1e00 con 0x5561211139d0 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd0200410 con 0x5561211142f0 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556120f16300 con 0x5561210ca0a0 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 10 client.0 _ll_register_callbacks cb 0x556121095040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd50e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561211268c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd50e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561211268c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42148/0 (socket says 172.21.15.133:42148) 2024-04-25T10:18:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd50e5640 1 -- 172.21.15.133:0/747169121 learned_addr learned my addr 172.21.15.133:0/747169121 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd60e7640 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211142f0 0x556121126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561210ca0a0 0x556121126380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd50e5640 1 -- 172.21.15.133:0/747169121 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561210ca0a0 msgr2=0x556121126380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd50e5640 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561210ca0a0 0x556121126380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd50e5640 1 -- 172.21.15.133:0/747169121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211142f0 msgr2=0x556121126e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd50e5640 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211142f0 0x556121126e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd50e5640 1 -- 172.21.15.133:0/747169121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556120eb88e0 con 0x5561211139d0 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd50e5640 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561211268c0 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0019170 tx=0x7f1bc0009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fd2b84be740b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/747169121 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00ce030 con 0x5561211139d0 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/747169121 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1bc00a4df0 con 0x5561211139d0 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 -- 172.21.15.133:0/747169121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bcc0169e0 con 0x5561211139d0 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/747169121 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bc00d9050 con 0x5561211139d0 2024-04-25T10:18:01.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 10 client.12234 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 -- 172.21.15.133:0/747169121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x556120f708f0 con 0x5561211139d0 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/747169121 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc0169e0 con 0x5561211139d0 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/747169121 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1bc00dc020 con 0x5561211139d0 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 10 client.12234 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 10 client.12234 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 20 client.12234 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x5561211277c0 0x55612117bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 4 client.12234 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 -- 172.21.15.133:0/747169121 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556120f24530 con 0x5561211277c0 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x5561211277c0 0x55612117bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bb0ff9640 10 client.12234 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd58e6640 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x5561211277c0 0x55612117bf60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b4f4e785a16b2c6 server_cookie=e2bf66af7ce337f4 in_seq=0 out_seq=0 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bb0ff9640 1 -- 172.21.15.133:0/747169121 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x556120f24530 con 0x5561211277c0 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bb0ff9640 10 client.12234 handle_command_reply: tid=0 2024-04-25T10:18:01.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 client.12234 shutdown 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 -- 172.21.15.133:0/747169121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 msgr2=0x5561211268c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561211268c0 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0019170 tx=0x7f1bc0009240 comp rx=0 tx=0).stop 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 -- 172.21.15.133:0/747169121 shutdown_connections 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x5561211277c0 0x55612117bf60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5561211142f0 0x556121126e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561211139d0 0x5561211268c0 unknown :-1 s=CLOSED pgs=2953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 --2- 172.21.15.133:0/747169121 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5561210ca0a0 0x556121126380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bb0ff9640 0 client.12234 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 -- 172.21.15.133:0/747169121 >> 172.21.15.133:0/747169121 conn(0x5561210c6020 msgr2=0x556121112960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 -- 172.21.15.133:0/747169121 shutdown_connections 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 1 -- 172.21.15.133:0/747169121 wait complete. 2024-04-25T10:18:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bdacb7000 20 client.12234 trim_cache size 0 max 16384 2024-04-25T10:18:01.553 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:18:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:18:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-04-25T10:18:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "b51d7dc2-4157-4ba7-9602-225b1f33e599": { 2024-04-25T10:18:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-04-25T10:18:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "b51d7dc2-4157-4ba7-9602-225b1f33e599", 2024-04-25T10:18:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,repair" 2024-04-25T10:18:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:18:01.553 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:18:01.553 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:18:01.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/1112631034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1bb4066f60 msgr2=0x7f1bb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1bb4066f60 0x7f1bb4069420 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1bd01f8020 tx=0x7f1bcc008040 comp rx=0 tx=0).stop 2024-04-25T10:18:01.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/1112631034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd01648d0 msgr2=0x7f1bd0164d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd01648d0 0x7f1bd0164d50 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800c780 tx=0x7f1bc800cc40 comp rx=0 tx=0).stop 2024-04-25T10:18:01.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/1112631034 shutdown_connections 2024-04-25T10:18:01.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1bb4066f60 0x7f1bb4069420 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd01648d0 0x7f1bd0164d50 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1bd0145170 0x7f1bd01398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 --2- 172.21.15.133:0/1112631034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1bd00cdd60 0x7f1bd01612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/1112631034 >> 172.21.15.133:0/1112631034 conn(0x7f1bd0066f70 msgr2=0x7f1bd015f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/1112631034 shutdown_connections 2024-04-25T10:18:01.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:01.542+0000 7f1bd7b4a640 1 -- 172.21.15.133:0/1112631034 wait complete. 2024-04-25T10:18:01.566 INFO:tasks.cephfs.filesystem:scrub status for tag:b51d7dc2-4157-4ba7-9602-225b1f33e599 - {'path': '/', 'tag': 'b51d7dc2-4157-4ba7-9602-225b1f33e599', 'options': 'recursive,repair'} 2024-04-25T10:18:02.760 INFO:tasks.ceph.mds.c.smithi133.stderr:2024-04-25T10:18:02.758+0000 7fc92de52640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:18:24.363 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:24.366 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:31.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:31.568 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:18:31.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.698+0000 7fb62f355640 1 Processor -- start 2024-04-25T10:18:31.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.698+0000 7fb62f355640 1 -- start start 2024-04-25T10:18:31.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62f355640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb62815d9a0 0x7fb6280d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62f355640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6280d24c0 con 0x7fb6280d2030 2024-04-25T10:18:31.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62f355640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62815ebd0 con 0x7fb6280ce600 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62f355640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62815eeb0 con 0x7fb62815d9a0 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62d8f2640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb6280d2030 0x7fb62815dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50174/0 (socket says 172.21.15.133:50174) 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62d8f2640 1 -- 172.21.15.133:0/1311870533 learned_addr learned my addr 172.21.15.133:0/1311870533 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb62815d9a0 0x7fb6280d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253710033 0 0) 0x7fb6280d24c0 con 0x7fb6280d2030 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb610003530 con 0x7fb6280d2030 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1277750249 0 0) 0x7fb610003530 con 0x7fb6280d2030 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb62815d9a0 msgr2=0x7fb6280d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb62815d9a0 0x7fb6280d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 >> v1:172.21.15.162:6789/0 conn(0x7fb6280ce600 legacy=0x7fb6280d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6280d0330 con 0x7fb6280d2030 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb618001680 con 0x7fb6280d2030 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 >> v1:172.21.15.133:6789/0 conn(0x7fb6280d2030 legacy=0x7fb62815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6100039f0 0x7fb610003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 0x7fb610004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb62815d9a0 0x7fb610005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62815eeb0 con 0x7fb610004810 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61000bb60 con 0x7fb6100039f0 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61000bd60 con 0x7fb62815d9a0 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 0x7fb610004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62d8f2640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb62815d9a0 0x7fb610005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6100039f0 0x7fb610003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62c8f0640 1 -- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb62815d9a0 msgr2=0x7fb610005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb62815d9a0 0x7fb610005280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62c8f0640 1 -- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6100039f0 msgr2=0x7fb610003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6100039f0 0x7fb610003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62c8f0640 1 -- 172.21.15.133:0/1311870533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb610003530 con 0x7fb610004810 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.702+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6100039f0 0x7fb610003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:31.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 0x7fb610004c90 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7fb61c001780 tx=0x7fb61c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0861a8a465b9fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0013a0 con 0x7fb610004810 2024-04-25T10:18:31.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c0052f0 con 0x7fb610004810 2024-04-25T10:18:31.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb627fff640 1 -- 172.21.15.133:0/1311870533 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c003840 con 0x7fb610004810 2024-04-25T10:18:31.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 -- 172.21.15.133:0/1311870533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 msgr2=0x7fb610004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 --2- 172.21.15.133:0/1311870533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 0x7fb610004c90 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7fb61c001780 tx=0x7fb61c01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62d8f2640 1 -- 172.21.15.133:0/1311870533 reap_dead start 2024-04-25T10:18:31.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 -- 172.21.15.133:0/1311870533 shutdown_connections 2024-04-25T10:18:31.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 -- 172.21.15.133:0/1311870533 >> 172.21.15.133:0/1311870533 conn(0x7fb62806bf80 msgr2=0x7fb62806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 -- 172.21.15.133:0/1311870533 shutdown_connections 2024-04-25T10:18:31.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 -- 172.21.15.133:0/1311870533 wait complete. 2024-04-25T10:18:31.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 Processor -- start 2024-04-25T10:18:31.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 -- start start 2024-04-25T10:18:31.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 0x7fb6280b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6280caa10 0x7fb628155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6281fa210 0x7fb6281fa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280b9820 con 0x7fb610004810 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280b96a0 con 0x7fb6280caa10 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280cedf0 con 0x7fb6281fa210 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62d0f1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 0x7fb6280b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62d0f1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 0x7fb6280b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58176/0 (socket says 172.21.15.133:58176) 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62d0f1640 1 -- 172.21.15.133:0/2072723463 learned_addr learned my addr 172.21.15.133:0/2072723463 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6280caa10 0x7fb628155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62d0f1640 1 -- 172.21.15.133:0/2072723463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6281fa210 msgr2=0x7fb6281fa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62d8f2640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6281fa210 0x7fb6281fa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6281fa210 0x7fb6281fa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62d0f1640 1 -- 172.21.15.133:0/2072723463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6280caa10 msgr2=0x7fb628155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6280caa10 0x7fb628155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62d0f1640 1 -- 172.21.15.133:0/2072723463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb628157130 con 0x7fb610004810 2024-04-25T10:18:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6280caa10 0x7fb628155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 0x7fb6280b5580 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7fb62000f350 tx=0x7fb62000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb625ffb640 1 -- 172.21.15.133:0/2072723463 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620003ba0 con 0x7fb610004810 2024-04-25T10:18:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb625ffb640 1 -- 172.21.15.133:0/2072723463 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb620003d40 con 0x7fb610004810 2024-04-25T10:18:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 -- 172.21.15.133:0/2072723463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb628157410 con 0x7fb610004810 2024-04-25T10:18:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb625ffb640 1 -- 172.21.15.133:0/2072723463 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620004240 con 0x7fb610004810 2024-04-25T10:18:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.706+0000 7fb62f355640 1 -- 172.21.15.133:0/2072723463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62815a9b0 con 0x7fb610004810 2024-04-25T10:18:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb625ffb640 1 -- 172.21.15.133:0/2072723463 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6200043e0 con 0x7fb610004810 2024-04-25T10:18:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb625ffb640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb614066f90 0x7fb614069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb625ffb640 1 -- 172.21.15.133:0/2072723463 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb6200857f0 con 0x7fb610004810 2024-04-25T10:18:31.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb614066f90 0x7fb614069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb614066f90 0x7fb614069450 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0ac000 tx=0x7fb61c0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb6324c2000 1 Processor -- start 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb6324c2000 1 -- start start 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6129d0 0x55b70a612db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a5c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb6324c2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a415ff0 con 0x55b70a6132f0 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb6324c2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a39e150 con 0x55b70a5c90a0 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb6324c2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b70a412090 con 0x55b70a6129d0 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb62c8f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb62c8f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58178/0 (socket says 172.21.15.133:58178) 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb62c8f0640 1 -- 172.21.15.133:0/855025259 learned_addr learned my addr 172.21.15.133:0/855025259 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/855025259 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6129d0 0x55b70a612db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.710+0000 7fb62d8f2640 1 --2- 172.21.15.133:0/855025259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a5c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62c8f0640 1 -- 172.21.15.133:0/855025259 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6129d0 msgr2=0x55b70a612db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/855025259 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6129d0 0x55b70a612db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62c8f0640 1 -- 172.21.15.133:0/855025259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 msgr2=0x55b70a5c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/855025259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a5c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62c8f0640 1 -- 172.21.15.133:0/855025259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b70a5ac0d0 con 0x55b70a6132f0 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/855025259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5c8b60 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7fb61c007bb0 tx=0x7fb61c001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e42adfd3e61478 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb60effd640 1 -- 172.21.15.133:0/855025259 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0ba040 con 0x55b70a6132f0 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb60effd640 1 -- 172.21.15.133:0/855025259 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c007400 con 0x55b70a6132f0 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb60effd640 1 -- 172.21.15.133:0/855025259 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0067e0 con 0x55b70a6132f0 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 -- 172.21.15.133:0/855025259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 msgr2=0x55b70a5c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 --2- 172.21.15.133:0/855025259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5c8b60 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7fb61c007bb0 tx=0x7fb61c001130 comp rx=0 tx=0).stop 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 -- 172.21.15.133:0/855025259 shutdown_connections 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 --2- 172.21.15.133:0/855025259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a5c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 --2- 172.21.15.133:0/855025259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5c8b60 unknown :-1 s=CLOSED pgs=2957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 --2- 172.21.15.133:0/855025259 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6129d0 0x55b70a612db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 -- 172.21.15.133:0/855025259 >> 172.21.15.133:0/855025259 conn(0x55b70a5c5020 msgr2=0x55b70a5c7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 -- 172.21.15.133:0/855025259 shutdown_connections 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 -- 172.21.15.133:0/855025259 wait complete. 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 -- start start 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 Processor -- start 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a5f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6129d0 0x55b70a5f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5f8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a3da250 con 0x55b70a6132f0 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a3d0690 con 0x55b70a5c90a0 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b70a39ea40 con 0x55b70a6129d0 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 10 client.0 _ll_register_callbacks cb 0x55b70a5b3e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:31.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62d8f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62d8f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5f8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58184/0 (socket says 172.21.15.133:58184) 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62d8f2640 1 -- 172.21.15.133:0/1574346525 learned_addr learned my addr 172.21.15.133:0/1574346525 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a5f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6129d0 0x55b70a5f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62d8f2640 1 -- 172.21.15.133:0/1574346525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6129d0 msgr2=0x55b70a5f7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62d8f2640 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6129d0 0x55b70a5f7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62d8f2640 1 -- 172.21.15.133:0/1574346525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 msgr2=0x55b70a5f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62d8f2640 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a5f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62d8f2640 1 -- 172.21.15.133:0/1574346525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b70a4c31c0 con 0x55b70a6132f0 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb62d8f2640 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5f8270 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 crypto rx=0x7fb6280d4750 tx=0x7fb618010610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16bf14dc5b211da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb607fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb607fff640 1 -- 172.21.15.133:0/1574346525 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618014660 con 0x55b70a6132f0 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 -- 172.21.15.133:0/1574346525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb61c007400 con 0x55b70a6132f0 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb607fff640 1 -- 172.21.15.133:0/1574346525 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb618014800 con 0x55b70a6132f0 2024-04-25T10:18:31.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb607fff640 1 -- 172.21.15.133:0/1574346525 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb618021050 con 0x55b70a6132f0 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 10 client.12246 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 1 -- 172.21.15.133:0/1574346525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x55b70a4bf170 con 0x55b70a6132f0 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb607fff640 1 -- 172.21.15.133:0/1574346525 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c007400 con 0x55b70a6132f0 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb607fff640 1 -- 172.21.15.133:0/1574346525 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb618024020 con 0x55b70a6132f0 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 10 client.12246 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.714+0000 7fb6324c2000 10 client.12246 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 20 client.12246 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55b70a622c80 0x55b70a625070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 4 client.12246 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 -- 172.21.15.133:0/1574346525 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b70a3b1c60 con 0x55b70a622c80 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55b70a622c80 0x55b70a625070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55b70a622c80 0x55b70a625070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76eb08bd256b4181 server_cookie=83f7984858812fb8 in_seq=0 out_seq=0 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb607fff640 10 client.12246 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:31.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb607fff640 1 -- 172.21.15.133:0/1574346525 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b70a3b1c60 con 0x55b70a622c80 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb607fff640 10 client.12246 handle_command_reply: tid=0 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 client.12246 shutdown 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 -- 172.21.15.133:0/1574346525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 msgr2=0x55b70a5f8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5f8270 secure :-1 s=READY pgs=2958 cs=0 l=1 rev1=1 crypto rx=0x7fb6280d4750 tx=0x7fb618010610 comp rx=0 tx=0).stop 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 -- 172.21.15.133:0/1574346525 shutdown_connections 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55b70a622c80 0x55b70a625070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6132f0 0x55b70a5f8270 unknown :-1 s=CLOSED pgs=2958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6129d0 0x55b70a5f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb607fff640 0 client.12246 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 --2- 172.21.15.133:0/1574346525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a5f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 -- 172.21.15.133:0/1574346525 >> 172.21.15.133:0/1574346525 conn(0x55b70a5c5020 msgr2=0x55b70a5c7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 -- 172.21.15.133:0/1574346525 shutdown_connections 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 1 -- 172.21.15.133:0/1574346525 wait complete. 2024-04-25T10:18:31.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.718+0000 7fb6324c2000 20 client.12246 trim_cache size 0 max 16384 2024-04-25T10:18:31.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 Processor -- start 2024-04-25T10:18:31.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 -- start start 2024-04-25T10:18:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a611e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a5c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6132f0 0x55b70a5c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628204020 con 0x55b70a6129d0 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6282031e0 con 0x55b70a5c90a0 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6282026e0 con 0x55b70a6132f0 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb62c8f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a5c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb62c8f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a5c8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58194/0 (socket says 172.21.15.133:58194) 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb62c8f0640 1 -- 172.21.15.133:0/3999022671 learned_addr learned my addr 172.21.15.133:0/3999022671 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb62c8f0640 1 -- 172.21.15.133:0/3999022671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6132f0 msgr2=0x55b70a5c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/3999022671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6132f0 0x55b70a5c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb62c8f0640 1 -- 172.21.15.133:0/3999022671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 msgr2=0x55b70a611e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/3999022671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a611e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb62c8f0640 1 -- 172.21.15.133:0/3999022671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b70a45cde0 con 0x55b70a6129d0 2024-04-25T10:18:31.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/3999022671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a5c8420 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7fb61c01c8d0 tx=0x7fb61c0b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0457d07e33dfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb60e7fc640 1 -- 172.21.15.133:0/3999022671 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c007500 con 0x55b70a6129d0 2024-04-25T10:18:31.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb60e7fc640 1 -- 172.21.15.133:0/3999022671 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c0076a0 con 0x55b70a6129d0 2024-04-25T10:18:31.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb60e7fc640 1 -- 172.21.15.133:0/3999022671 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0069e0 con 0x55b70a6129d0 2024-04-25T10:18:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 -- 172.21.15.133:0/3999022671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 msgr2=0x55b70a5c8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 --2- 172.21.15.133:0/3999022671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a5c8420 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7fb61c01c8d0 tx=0x7fb61c0b8040 comp rx=0 tx=0).stop 2024-04-25T10:18:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 -- 172.21.15.133:0/3999022671 shutdown_connections 2024-04-25T10:18:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 --2- 172.21.15.133:0/3999022671 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6132f0 0x55b70a5c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 --2- 172.21.15.133:0/3999022671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a5c8420 unknown :-1 s=CLOSED pgs=2959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 --2- 172.21.15.133:0/3999022671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a611e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 -- 172.21.15.133:0/3999022671 >> 172.21.15.133:0/3999022671 conn(0x55b70a5c5020 msgr2=0x55b70a6110d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 -- 172.21.15.133:0/3999022671 shutdown_connections 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 -- 172.21.15.133:0/3999022671 wait complete. 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 -- start start 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.734+0000 7fb6324c2000 1 Processor -- start 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a673520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a673a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6132f0 0x55b70a625380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b70a3b0e00 con 0x55b70a6129d0 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628202be0 con 0x55b70a5c90a0 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b70a415300 con 0x55b70a6132f0 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 10 client.0 _ll_register_callbacks cb 0x55b70a594040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:31.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62c8f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a673a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62c8f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a673a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58210/0 (socket says 172.21.15.133:58210) 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62c8f0640 1 -- 172.21.15.133:0/3535263043 learned_addr learned my addr 172.21.15.133:0/3535263043 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62d8f2640 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6132f0 0x55b70a625380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62c8f0640 1 -- 172.21.15.133:0/3535263043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6132f0 msgr2=0x55b70a625380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6132f0 0x55b70a625380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62c8f0640 1 -- 172.21.15.133:0/3535263043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 msgr2=0x55b70a673520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a673520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a673520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62c8f0640 1 -- 172.21.15.133:0/3535263043 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b70a3b78e0 con 0x55b70a6129d0 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62c8f0640 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a673a60 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005040 tx=0x7fb61c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c3e2a24f9a1d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb607fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb607fff640 1 -- 172.21.15.133:0/3535263043 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0ba070 con 0x55b70a6129d0 2024-04-25T10:18:31.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb607fff640 1 -- 172.21.15.133:0/3535263043 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb61c006920 con 0x55b70a6129d0 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 -- 172.21.15.133:0/3535263043 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb61c0076a0 con 0x55b70a6129d0 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb607fff640 1 -- 172.21.15.133:0/3535263043 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb61c0df050 con 0x55b70a6129d0 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 10 client.12252 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 -- 172.21.15.133:0/3535263043 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x55b70a46f8f0 con 0x55b70a6129d0 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb607fff640 1 -- 172.21.15.133:0/3535263043 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c0076a0 con 0x55b70a6129d0 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb607fff640 1 -- 172.21.15.133:0/3535263043 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb61c0e2020 con 0x55b70a6129d0 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 10 client.12252 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 10 client.12252 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 20 client.12252 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55b70a625ce0 0x55b70a6260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 4 client.12252 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 -- 172.21.15.133:0/3535263043 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b70a423530 con 0x55b70a625ce0 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55b70a625ce0 0x55b70a6260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb607fff640 10 client.12252 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62d0f1640 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55b70a625ce0 0x55b70a6260f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523655ea11acee91 server_cookie=c8e2626269cc3685 in_seq=0 out_seq=0 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb607fff640 1 -- 172.21.15.133:0/3535263043 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b70a423530 con 0x55b70a625ce0 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb607fff640 10 client.12252 handle_command_reply: tid=0 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 client.12252 shutdown 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 -- 172.21.15.133:0/3535263043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 msgr2=0x55b70a673a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a673a60 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005040 tx=0x7fb61c019190 comp rx=0 tx=0).stop 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 -- 172.21.15.133:0/3535263043 shutdown_connections 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x55b70a625ce0 0x55b70a6260f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x55b70a6132f0 0x55b70a625380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b70a6129d0 0x55b70a673a60 unknown :-1 s=CLOSED pgs=2960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 --2- 172.21.15.133:0/3535263043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b70a5c90a0 0x55b70a673520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb607fff640 0 client.12252 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 -- 172.21.15.133:0/3535263043 >> 172.21.15.133:0/3535263043 conn(0x55b70a5c5020 msgr2=0x55b70a611960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 -- 172.21.15.133:0/3535263043 shutdown_connections 2024-04-25T10:18:31.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 1 -- 172.21.15.133:0/3535263043 wait complete. 2024-04-25T10:18:31.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb6324c2000 20 client.12252 trim_cache size 0 max 16384 2024-04-25T10:18:31.745 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:18:31.745 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-04-25T10:18:31.745 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:18:31.745 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:18:31.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 -- 172.21.15.133:0/2072723463 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb614066f90 msgr2=0x7fb614069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb614066f90 0x7fb614069450 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0ac000 tx=0x7fb61c0011a0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 -- 172.21.15.133:0/2072723463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 msgr2=0x7fb6280b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 0x7fb6280b5580 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7fb62000f350 tx=0x7fb62000f810 comp rx=0 tx=0).stop 2024-04-25T10:18:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 -- 172.21.15.133:0/2072723463 shutdown_connections 2024-04-25T10:18:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6281fa210 0x7fb6281fa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6280caa10 0x7fb628155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb614066f90 0x7fb614069450 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 --2- 172.21.15.133:0/2072723463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610004810 0x7fb6280b5580 unknown :-1 s=CLOSED pgs=2956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 -- 172.21.15.133:0/2072723463 >> 172.21.15.133:0/2072723463 conn(0x7fb62806bf80 msgr2=0x7fb6280d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 -- 172.21.15.133:0/2072723463 shutdown_connections 2024-04-25T10:18:31.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.738+0000 7fb62f355640 1 -- 172.21.15.133:0/2072723463 wait complete. 2024-04-25T10:18:31.756 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:18:31.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:31.756 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:18:31.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f013429b640 1 Processor -- start 2024-04-25T10:18:31.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f013429b640 1 -- start start 2024-04-25T10:18:31.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f013429b640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f012c0d0900 0x7f012c0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f013429b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f012c15b7a0 con 0x7f012c15aac0 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f013429b640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f012c15ba80 con 0x7f012c0ce5a0 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f013429b640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012c15bd60 con 0x7f012c0d0900 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132037640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f012c15aac0 0x7f012c15aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50184/0 (socket says 172.21.15.133:50184) 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132037640 1 -- 172.21.15.133:0/1611240457 learned_addr learned my addr 172.21.15.133:0/1611240457 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132838640 1 --2- 172.21.15.133:0/1611240457 >> v2:172.21.15.162:3300/0 conn(0x7f012c0d0900 0x7f012c0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532554027 0 0) 0x7f012c15b7a0 con 0x7f012c15aac0 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0118003530 con 0x7f012c15aac0 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1106372052 0 0) 0x7f0118003530 con 0x7f012c15aac0 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 >> v2:172.21.15.162:3300/0 conn(0x7f012c0d0900 msgr2=0x7f012c0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 --2- 172.21.15.133:0/1611240457 >> v2:172.21.15.162:3300/0 conn(0x7f012c0d0900 0x7f012c0b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 >> v1:172.21.15.162:6789/0 conn(0x7f012c0ce5a0 legacy=0x7f012c0d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c0cb1f0 con 0x7f012c15aac0 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0128001680 con 0x7f012c15aac0 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 >> v1:172.21.15.133:6789/0 conn(0x7f012c15aac0 legacy=0x7f012c15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 --2- 172.21.15.133:0/1611240457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 0x7f0118003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 --2- 172.21.15.133:0/1611240457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0118004810 0x7f0118004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 --2- 172.21.15.133:0/1611240457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01180051d0 0x7f011800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c15bd60 con 0x7f01180039c0 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011800c070 con 0x7f01180051d0 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011800c270 con 0x7f0118004810 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132037640 1 --2- 172.21.15.133:0/1611240457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 0x7f0118003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132037640 1 -- 172.21.15.133:0/1611240457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0118004810 msgr2=0x7f0118004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132037640 1 --2- 172.21.15.133:0/1611240457 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0118004810 0x7f0118004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132037640 1 -- 172.21.15.133:0/1611240457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01180051d0 msgr2=0x7f011800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132838640 1 --2- 172.21.15.133:0/1611240457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01180051d0 0x7f011800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132037640 1 --2- 172.21.15.133:0/1611240457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01180051d0 0x7f011800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132037640 1 -- 172.21.15.133:0/1611240457 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0118003530 con 0x7f01180039c0 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132838640 1 --2- 172.21.15.133:0/1611240457 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f01180051d0 0x7f011800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:31.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.874+0000 7f0132838640 1 -- 172.21.15.133:0/1611240457 reap_dead start 2024-04-25T10:18:31.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f0132037640 1 --2- 172.21.15.133:0/1611240457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 0x7f0118003de0 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f012800c210 tx=0x7f012801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e8c29f78e34034 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128007d40 con 0x7f01180039c0 2024-04-25T10:18:31.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01280a9070 con 0x7f01180039c0 2024-04-25T10:18:31.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f0131035640 1 -- 172.21.15.133:0/1611240457 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128005d80 con 0x7f01180039c0 2024-04-25T10:18:31.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f013429b640 1 -- 172.21.15.133:0/1611240457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 msgr2=0x7f0118003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f013429b640 1 --2- 172.21.15.133:0/1611240457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 0x7f0118003de0 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f012800c210 tx=0x7f012801cc60 comp rx=0 tx=0).stop 2024-04-25T10:18:31.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f013429b640 1 -- 172.21.15.133:0/1611240457 shutdown_connections 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f013429b640 1 --2- 172.21.15.133:0/1611240457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 0x7f0118003de0 unknown :-1 s=CLOSED pgs=2962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f013429b640 1 -- 172.21.15.133:0/1611240457 >> 172.21.15.133:0/1611240457 conn(0x7f012c0726f0 msgr2=0x7f012c0999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f013429b640 1 -- 172.21.15.133:0/1611240457 shutdown_connections 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f013429b640 1 -- 172.21.15.133:0/1611240457 wait complete. 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f013429b640 1 Processor -- start 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.878+0000 7f013429b640 1 -- start start 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f013429b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 0x7f012c0c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f013429b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f012c145500 0x7f012c0c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f013429b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f012c14fb50 0x7f012c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f013429b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0cbaa0 con 0x7f01180039c0 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f013429b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0cb920 con 0x7f012c145500 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f013429b640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0cbc20 con 0x7f012c14fb50 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0132037640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 0x7f012c0c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0132037640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 0x7f012c0c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58230/0 (socket says 172.21.15.133:58230) 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0132037640 1 -- 172.21.15.133:0/716183728 learned_addr learned my addr 172.21.15.133:0/716183728 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0132838640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f012c14fb50 0x7f012c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0131836640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f012c145500 0x7f012c0c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0132037640 1 -- 172.21.15.133:0/716183728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f012c14fb50 msgr2=0x7f012c14ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0132037640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f012c14fb50 0x7f012c14ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0132037640 1 -- 172.21.15.133:0/716183728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f012c145500 msgr2=0x7f012c0c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0132037640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f012c145500 0x7f012c0c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0132037640 1 -- 172.21.15.133:0/716183728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c06bba0 con 0x7f01180039c0 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0132838640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f012c14fb50 0x7f012c14ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0132037640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 0x7f012c0c9fd0 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7f012801c6d0 tx=0x7f0118005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0122ffd640 1 -- 172.21.15.133:0/716183728 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128007390 con 0x7f01180039c0 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0122ffd640 1 -- 172.21.15.133:0/716183728 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0128007530 con 0x7f01180039c0 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f013429b640 1 -- 172.21.15.133:0/716183728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012c14eab0 con 0x7f01180039c0 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0122ffd640 1 -- 172.21.15.133:0/716183728 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01280047f0 con 0x7f01180039c0 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f013429b640 1 -- 172.21.15.133:0/716183728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012c14f070 con 0x7f01180039c0 2024-04-25T10:18:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0122ffd640 1 -- 172.21.15.133:0/716183728 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0128004990 con 0x7f01180039c0 2024-04-25T10:18:31.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0122ffd640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0110066f90 0x7f0110069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0131836640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0110066f90 0x7f0110069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0122ffd640 1 -- 172.21.15.133:0/716183728 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f012811e760 con 0x7f01180039c0 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0131836640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0110066f90 0x7f0110069450 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f012c150940 tx=0x7f011c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.882+0000 7f0137408000 1 Processor -- start 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- start start 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d201c90 0x56543d202070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d2025b0 0x56543d1b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d1b8290 0x56543d1b8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543d09ef40 con 0x56543d1b8290 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543d006070 con 0x56543d201c90 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56543cf8e1a0 con 0x56543d2025b0 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d1b8290 0x56543d1b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d1b8290 0x56543d1b8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58234/0 (socket says 172.21.15.133:58234) 2024-04-25T10:18:31.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 -- 172.21.15.133:0/1918219992 learned_addr learned my addr 172.21.15.133:0/1918219992 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0131836640 1 --2- 172.21.15.133:0/1918219992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d2025b0 0x56543d1b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132037640 1 --2- 172.21.15.133:0/1918219992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d201c90 0x56543d202070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 -- 172.21.15.133:0/1918219992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d2025b0 msgr2=0x56543d1b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 --2- 172.21.15.133:0/1918219992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d2025b0 0x56543d1b7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 -- 172.21.15.133:0/1918219992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d201c90 msgr2=0x56543d202070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 --2- 172.21.15.133:0/1918219992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d201c90 0x56543d202070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 -- 172.21.15.133:0/1918219992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56543cf8c510 con 0x56543d1b8290 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0131836640 1 --2- 172.21.15.133:0/1918219992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d2025b0 0x56543d1b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 --2- 172.21.15.133:0/1918219992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d1b8290 0x56543d1b8670 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7f012c0b1e70 tx=0x7f012400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a20b4db2a02b188 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f010ffff640 1 -- 172.21.15.133:0/1918219992 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01240039a0 con 0x56543d1b8290 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f010ffff640 1 -- 172.21.15.133:0/1918219992 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0124003b40 con 0x56543d1b8290 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f010ffff640 1 -- 172.21.15.133:0/1918219992 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124015650 con 0x56543d1b8290 2024-04-25T10:18:31.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- 172.21.15.133:0/1918219992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d1b8290 msgr2=0x56543d1b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 --2- 172.21.15.133:0/1918219992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d1b8290 0x56543d1b8670 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7f012c0b1e70 tx=0x7f012400b040 comp rx=0 tx=0).stop 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- 172.21.15.133:0/1918219992 shutdown_connections 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 --2- 172.21.15.133:0/1918219992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d1b8290 0x56543d1b8670 unknown :-1 s=CLOSED pgs=2964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 --2- 172.21.15.133:0/1918219992 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d2025b0 0x56543d1b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 --2- 172.21.15.133:0/1918219992 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d201c90 0x56543d202070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- 172.21.15.133:0/1918219992 >> 172.21.15.133:0/1918219992 conn(0x56543d1b4240 msgr2=0x56543d1b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- 172.21.15.133:0/1918219992 shutdown_connections 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- 172.21.15.133:0/1918219992 wait complete. 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- start start 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 Processor -- start 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d1b8290 0x56543d1c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d201c90 0x56543d1c3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d1c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543cfc06c0 con 0x56543d2025b0 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543d002110 con 0x56543d1b8290 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56543d005380 con 0x56543d201c90 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0137408000 10 client.0 _ll_register_callbacks cb 0x56543d1a3570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d1c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d1c38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58244/0 (socket says 172.21.15.133:58244) 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132838640 1 -- 172.21.15.133:0/1256070524 learned_addr learned my addr 172.21.15.133:0/1256070524 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0132037640 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d1b8290 0x56543d1c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.886+0000 7f0131836640 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d201c90 0x56543d1c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0132838640 1 -- 172.21.15.133:0/1256070524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d201c90 msgr2=0x56543d1c3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0132838640 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d201c90 0x56543d1c3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0132838640 1 -- 172.21.15.133:0/1256070524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d1b8290 msgr2=0x56543d1c2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0132838640 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d1b8290 0x56543d1c2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0132838640 1 -- 172.21.15.133:0/1256070524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56543cf8e3b0 con 0x56543d2025b0 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0132838640 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d1c38b0 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7f012c0cabb0 tx=0x7f0124004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350854039766cd35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f010d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f010d7fa640 1 -- 172.21.15.133:0/1256070524 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124023070 con 0x56543d2025b0 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f010d7fa640 1 -- 172.21.15.133:0/1256070524 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0124016c50 con 0x56543d2025b0 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 -- 172.21.15.133:0/1256070524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0124003b40 con 0x56543d2025b0 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f010d7fa640 1 -- 172.21.15.133:0/1256070524 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0124030050 con 0x56543d2025b0 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 10 client.12264 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 -- 172.21.15.133:0/1256070524 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x56543d087e00 con 0x56543d2025b0 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f010d7fa640 1 -- 172.21.15.133:0/1256070524 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0124003b40 con 0x56543d2025b0 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f010d7fa640 1 -- 172.21.15.133:0/1256070524 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0124033020 con 0x56543d2025b0 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 10 client.12264 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 10 client.12264 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:18:31.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 20 client.12264 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56543d1c4150 0x56543d213410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 4 client.12264 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 -- 172.21.15.133:0/1256070524 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56543cfa1d60 con 0x56543d1c4150 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0132037640 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56543d1c4150 0x56543d213410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f010d7fa640 10 client.12264 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0132037640 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56543d1c4150 0x56543d213410 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2da8e01c1d7fce3a server_cookie=341145f563cb922c in_seq=0 out_seq=0 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f010d7fa640 1 -- 172.21.15.133:0/1256070524 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56543cfa1d60 con 0x56543d1c4150 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f010d7fa640 10 client.12264 handle_command_reply: tid=0 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 client.12264 shutdown 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 -- 172.21.15.133:0/1256070524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 msgr2=0x56543d1c38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d1c38b0 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7f012c0cabb0 tx=0x7f0124004a50 comp rx=0 tx=0).stop 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 -- 172.21.15.133:0/1256070524 shutdown_connections 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56543d1c4150 0x56543d213410 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d1c38b0 unknown :-1 s=CLOSED pgs=2965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d201c90 0x56543d1c3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 --2- 172.21.15.133:0/1256070524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d1b8290 0x56543d1c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f010d7fa640 0 client.12264 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:31.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 -- 172.21.15.133:0/1256070524 >> 172.21.15.133:0/1256070524 conn(0x56543d1b4240 msgr2=0x56543d1b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 -- 172.21.15.133:0/1256070524 shutdown_connections 2024-04-25T10:18:31.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 1 -- 172.21.15.133:0/1256070524 wait complete. 2024-04-25T10:18:31.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.890+0000 7f0137408000 20 client.12264 trim_cache size 0 max 16384 2024-04-25T10:18:31.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 Processor -- start 2024-04-25T10:18:31.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 -- start start 2024-04-25T10:18:31.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d1b8290 0x56543d1b7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d201c90 0x56543d1b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d213410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c204380 con 0x56543d2025b0 2024-04-25T10:18:31.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c2034e0 con 0x56543d1b8290 2024-04-25T10:18:31.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0adfc0 con 0x56543d201c90 2024-04-25T10:18:31.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0132838640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d213410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0132838640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d213410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58252/0 (socket says 172.21.15.133:58252) 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0132838640 1 -- 172.21.15.133:0/448131745 learned_addr learned my addr 172.21.15.133:0/448131745 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0132037640 1 --2- 172.21.15.133:0/448131745 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d1b8290 0x56543d1b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0131836640 1 --2- 172.21.15.133:0/448131745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d201c90 0x56543d1b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0132838640 1 -- 172.21.15.133:0/448131745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d201c90 msgr2=0x56543d1b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0132838640 1 --2- 172.21.15.133:0/448131745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d201c90 0x56543d1b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0132838640 1 -- 172.21.15.133:0/448131745 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d1b8290 msgr2=0x56543d1b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0132838640 1 --2- 172.21.15.133:0/448131745 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d1b8290 0x56543d1b7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0132838640 1 -- 172.21.15.133:0/448131745 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56543d05dc50 con 0x56543d2025b0 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0132037640 1 --2- 172.21.15.133:0/448131745 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d1b8290 0x56543d1b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0132838640 1 --2- 172.21.15.133:0/448131745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d213410 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7f0124009c10 tx=0x7f012400b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c521ed9f58a0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f010f7fe640 1 -- 172.21.15.133:0/448131745 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012402d040 con 0x56543d2025b0 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f010f7fe640 1 -- 172.21.15.133:0/448131745 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0124009410 con 0x56543d2025b0 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f010f7fe640 1 -- 172.21.15.133:0/448131745 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124023040 con 0x56543d2025b0 2024-04-25T10:18:31.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 -- 172.21.15.133:0/448131745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 msgr2=0x56543d213410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 --2- 172.21.15.133:0/448131745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d213410 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7f0124009c10 tx=0x7f012400b070 comp rx=0 tx=0).stop 2024-04-25T10:18:31.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 -- 172.21.15.133:0/448131745 shutdown_connections 2024-04-25T10:18:31.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 --2- 172.21.15.133:0/448131745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d2025b0 0x56543d213410 unknown :-1 s=CLOSED pgs=2966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 --2- 172.21.15.133:0/448131745 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d201c90 0x56543d1b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 --2- 172.21.15.133:0/448131745 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d1b8290 0x56543d1b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 -- 172.21.15.133:0/448131745 >> 172.21.15.133:0/448131745 conn(0x56543d1b4240 msgr2=0x56543d1bcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 -- 172.21.15.133:0/448131745 shutdown_connections 2024-04-25T10:18:31.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 -- 172.21.15.133:0/448131745 wait complete. 2024-04-25T10:18:31.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 -- start start 2024-04-25T10:18:31.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.906+0000 7f0137408000 1 Processor -- start 2024-04-25T10:18:31.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d1b8290 0x56543d215580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d201c90 0x56543d21dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d2025b0 0x56543d21e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56543d1bdb70 con 0x56543d201c90 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c202f10 con 0x56543d2025b0 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56543d1b40c0 con 0x56543d1b8290 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 10 client.0 _ll_register_callbacks cb 0x7f0118004710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0131836640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d201c90 0x56543d21dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0131836640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d201c90 0x56543d21dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58266/0 (socket says 172.21.15.133:58266) 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0131836640 1 -- 172.21.15.133:0/1741485004 learned_addr learned my addr 172.21.15.133:0/1741485004 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0132037640 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d1b8290 0x56543d215580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0132838640 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d2025b0 0x56543d21e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0131836640 1 -- 172.21.15.133:0/1741485004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d1b8290 msgr2=0x56543d215580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0131836640 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d1b8290 0x56543d215580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0131836640 1 -- 172.21.15.133:0/1741485004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d2025b0 msgr2=0x56543d21e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0131836640 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d2025b0 0x56543d21e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0131836640 1 -- 172.21.15.133:0/1741485004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56543cfce120 con 0x56543d201c90 2024-04-25T10:18:31.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0131836640 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d201c90 0x56543d21dee0 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7f011c007fd0 tx=0x7f011c007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e730050ae679d575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f010d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f010d7fa640 1 -- 172.21.15.133:0/1741485004 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c028060 con 0x56543d201c90 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f010d7fa640 1 -- 172.21.15.133:0/1741485004 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f011c023c80 con 0x56543d201c90 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 -- 172.21.15.133:0/1741485004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0124009410 con 0x56543d201c90 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f010d7fa640 1 -- 172.21.15.133:0/1741485004 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f011c01b070 con 0x56543d201c90 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 10 client.12270 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 -- 172.21.15.133:0/1741485004 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x56543cfad170 con 0x56543d201c90 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f010d7fa640 1 -- 172.21.15.133:0/1741485004 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0124009410 con 0x56543d201c90 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f010d7fa640 1 -- 172.21.15.133:0/1741485004 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f011c021030 con 0x56543d201c90 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 10 client.12270 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 10 client.12270 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 20 client.12270 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:31.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56543d21e960 0x56543d261f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 4 client.12270 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 -- 172.21.15.133:0/1741485004 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x56543d0135b0 con 0x56543d21e960 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0132037640 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56543d21e960 0x56543d261f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f010d7fa640 10 client.12270 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0132037640 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56543d21e960 0x56543d261f70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f4a1bc7f7defe6 server_cookie=5833176a23c1a55b in_seq=0 out_seq=0 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f010d7fa640 1 -- 172.21.15.133:0/1741485004 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56543d0135b0 con 0x56543d21e960 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f010d7fa640 10 client.12270 handle_command_reply: tid=0 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 client.12270 shutdown 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 -- 172.21.15.133:0/1741485004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d201c90 msgr2=0x56543d21dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d201c90 0x56543d21dee0 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7f011c007fd0 tx=0x7f011c007850 comp rx=0 tx=0).stop 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 -- 172.21.15.133:0/1741485004 shutdown_connections 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56543d21e960 0x56543d261f70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56543d2025b0 0x56543d21e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56543d201c90 0x56543d21dee0 unknown :-1 s=CLOSED pgs=2967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 --2- 172.21.15.133:0/1741485004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56543d1b8290 0x56543d215580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 -- 172.21.15.133:0/1741485004 >> 172.21.15.133:0/1741485004 conn(0x56543d1b4240 msgr2=0x56543d1b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f010d7fa640 0 client.12270 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:31.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 -- 172.21.15.133:0/1741485004 shutdown_connections 2024-04-25T10:18:31.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 1 -- 172.21.15.133:0/1741485004 wait complete. 2024-04-25T10:18:31.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f0137408000 20 client.12270 trim_cache size 0 max 16384 2024-04-25T10:18:31.918 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:18:31.918 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2024-04-25T10:18:31.918 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "e6da3468-4b87-4959-9e23-51ee30a9ae64", 2024-04-25T10:18:31.918 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2024-04-25T10:18:31.918 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:18:31.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 -- 172.21.15.133:0/716183728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0110066f90 msgr2=0x7f0110069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0110066f90 0x7f0110069450 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f012c150940 tx=0x7f011c009040 comp rx=0 tx=0).stop 2024-04-25T10:18:31.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 -- 172.21.15.133:0/716183728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 msgr2=0x7f012c0c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 0x7f012c0c9fd0 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7f012801c6d0 tx=0x7f0118005840 comp rx=0 tx=0).stop 2024-04-25T10:18:31.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 -- 172.21.15.133:0/716183728 shutdown_connections 2024-04-25T10:18:31.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f012c14fb50 0x7f012c14ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f012c145500 0x7f012c0c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0110066f90 0x7f0110069450 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 --2- 172.21.15.133:0/716183728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01180039c0 0x7f012c0c9fd0 unknown :-1 s=CLOSED pgs=2963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 -- 172.21.15.133:0/716183728 >> 172.21.15.133:0/716183728 conn(0x7f012c0726f0 msgr2=0x7f012c0b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 -- 172.21.15.133:0/716183728 shutdown_connections 2024-04-25T10:18:31.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:31.910+0000 7f013429b640 1 -- 172.21.15.133:0/716183728 wait complete. 2024-04-25T10:18:31.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:31.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:18:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.050+0000 7fcb6603a640 1 Processor -- start 2024-04-25T10:18:32.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.050+0000 7fcb6603a640 1 -- start start 2024-04-25T10:18:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.050+0000 7fcb6603a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcb6015eb40 0x7fcb600b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.050+0000 7fcb6603a640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcb600aebc0 0x7fcb600af010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.050+0000 7fcb6603a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb601471f0 con 0x7fcb6006e8b0 2024-04-25T10:18:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.050+0000 7fcb6603a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb600af550 con 0x7fcb6015eb40 2024-04-25T10:18:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.050+0000 7fcb6603a640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb60147490 con 0x7fcb600aebc0 2024-04-25T10:18:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.050+0000 7fcb5effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcb6015eb40 0x7fcb600b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.054+0000 7fcb5effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcb6015eb40 0x7fcb600b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58270/0 (socket says 172.21.15.133:58270) 2024-04-25T10:18:32.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.054+0000 7fcb5effd640 1 -- 172.21.15.133:0/826933855 learned_addr learned my addr 172.21.15.133:0/826933855 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.054+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/826933855 >> v2:172.21.15.162:3300/0 conn(0x7fcb600aebc0 0x7fcb600af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.054+0000 7fcb5effd640 1 -- 172.21.15.133:0/826933855 >> v2:172.21.15.162:3300/0 conn(0x7fcb600aebc0 msgr2=0x7fcb600af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.054+0000 7fcb5effd640 1 --2- 172.21.15.133:0/826933855 >> v2:172.21.15.162:3300/0 conn(0x7fcb600aebc0 0x7fcb600af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.054+0000 7fcb5effd640 1 -- 172.21.15.133:0/826933855 >> v1:172.21.15.162:6789/0 conn(0x7fcb6006e8b0 legacy=0x7fcb60146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.054+0000 7fcb5effd640 1 -- 172.21.15.133:0/826933855 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb600b1090 con 0x7fcb6015eb40 2024-04-25T10:18:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.054+0000 7fcb5effd640 1 --2- 172.21.15.133:0/826933855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb6015eb40 0x7fcb600b1e50 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7fcb600b27f0 tx=0x7fcb5801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1f961ddef63301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5e7fc640 1 -- 172.21.15.133:0/826933855 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb580b0070 con 0x7fcb6015eb40 2024-04-25T10:18:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5e7fc640 1 -- 172.21.15.133:0/826933855 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb580ab600 con 0x7fcb6015eb40 2024-04-25T10:18:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5e7fc640 1 -- 172.21.15.133:0/826933855 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb580ab9d0 con 0x7fcb6015eb40 2024-04-25T10:18:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 -- 172.21.15.133:0/826933855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb6015eb40 msgr2=0x7fcb600b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 --2- 172.21.15.133:0/826933855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb6015eb40 0x7fcb600b1e50 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7fcb600b27f0 tx=0x7fcb5801c450 comp rx=0 tx=0).stop 2024-04-25T10:18:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 -- 172.21.15.133:0/826933855 shutdown_connections 2024-04-25T10:18:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 --2- 172.21.15.133:0/826933855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb6015eb40 0x7fcb600b1e50 unknown :-1 s=CLOSED pgs=2968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 --2- 172.21.15.133:0/826933855 >> v2:172.21.15.162:3300/0 conn(0x7fcb600aebc0 0x7fcb600af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 -- 172.21.15.133:0/826933855 >> 172.21.15.133:0/826933855 conn(0x7fcb60066f70 msgr2=0x7fcb6006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 -- 172.21.15.133:0/826933855 shutdown_connections 2024-04-25T10:18:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 -- 172.21.15.133:0/826933855 wait complete. 2024-04-25T10:18:32.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 Processor -- start 2024-04-25T10:18:32.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 -- start start 2024-04-25T10:18:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb6006e8b0 0x7fcb600cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb600aebc0 0x7fcb60143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb6015eb40 0x7fcb60144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6006de40 con 0x7fcb6015eb40 2024-04-25T10:18:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6006dcc0 con 0x7fcb6006e8b0 2024-04-25T10:18:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600b2390 con 0x7fcb600aebc0 2024-04-25T10:18:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb6015eb40 0x7fcb60144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb6015eb40 0x7fcb60144380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58286/0 (socket says 172.21.15.133:58286) 2024-04-25T10:18:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5ffff640 1 -- 172.21.15.133:0/350532218 learned_addr learned my addr 172.21.15.133:0/350532218 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5effd640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb600aebc0 0x7fcb60143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb6006e8b0 0x7fcb600cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5ffff640 1 -- 172.21.15.133:0/350532218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb600aebc0 msgr2=0x7fcb60143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb600aebc0 0x7fcb60143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5ffff640 1 -- 172.21.15.133:0/350532218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb6006e8b0 msgr2=0x7fcb600cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb6006e8b0 0x7fcb600cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5ffff640 1 -- 172.21.15.133:0/350532218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb60146560 con 0x7fcb6015eb40 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb6006e8b0 0x7fcb600cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb6015eb40 0x7fcb60144380 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800d800 tx=0x7fcb4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb57fff640 1 -- 172.21.15.133:0/350532218 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4800b6c0 con 0x7fcb6015eb40 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb57fff640 1 -- 172.21.15.133:0/350532218 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb48019070 con 0x7fcb6015eb40 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 -- 172.21.15.133:0/350532218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb60144f10 con 0x7fcb6015eb40 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb57fff640 1 -- 172.21.15.133:0/350532218 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4800baa0 con 0x7fcb6015eb40 2024-04-25T10:18:32.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.058+0000 7fcb6603a640 1 -- 172.21.15.133:0/350532218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb601454d0 con 0x7fcb6015eb40 2024-04-25T10:18:32.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb57fff640 1 -- 172.21.15.133:0/350532218 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb4800bc40 con 0x7fcb6015eb40 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb57fff640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb40066f60 0x7fcb40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb57fff640 1 -- 172.21.15.133:0/350532218 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcb480856f0 con 0x7fcb6015eb40 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb40066f60 0x7fcb40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb40066f60 0x7fcb40069420 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcb50007fd0 tx=0x7fcb50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb691a7000 1 Processor -- start 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb691a7000 1 -- start start 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x56526466fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646702f0 0x565264625b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 0x565264626480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb691a7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x565264472ff0 con 0x56526466f9d0 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb691a7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652643fb150 con 0x5652646260a0 2024-04-25T10:18:32.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb691a7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x56526446f090 con 0x5652646702f0 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x56526466fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x56526466fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58302/0 (socket says 172.21.15.133:58302) 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5f7fe640 1 -- 172.21.15.133:0/3843695495 learned_addr learned my addr 172.21.15.133:0/3843695495 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5effd640 1 --2- 172.21.15.133:0/3843695495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646702f0 0x565264625b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5f7fe640 1 -- 172.21.15.133:0/3843695495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646702f0 msgr2=0x565264625b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/3843695495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646702f0 0x565264625b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/3843695495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 0x565264626480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5f7fe640 1 -- 172.21.15.133:0/3843695495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 msgr2=0x565264626480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/3843695495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 0x565264626480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5f7fe640 1 -- 172.21.15.133:0/3843695495 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652646090d0 con 0x56526466f9d0 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.062+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/3843695495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x56526466fdb0 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000dd00 tx=0x7fcb50004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ffc0d7699c684b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb557fa640 1 -- 172.21.15.133:0/3843695495 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50024070 con 0x56526466f9d0 2024-04-25T10:18:32.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb557fa640 1 -- 172.21.15.133:0/3843695495 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5001f600 con 0x56526466f9d0 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb557fa640 1 -- 172.21.15.133:0/3843695495 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5001f9b0 con 0x56526466f9d0 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- 172.21.15.133:0/3843695495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 msgr2=0x56526466fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 --2- 172.21.15.133:0/3843695495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x56526466fdb0 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000dd00 tx=0x7fcb50004940 comp rx=0 tx=0).stop 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- 172.21.15.133:0/3843695495 shutdown_connections 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 --2- 172.21.15.133:0/3843695495 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 0x565264626480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 --2- 172.21.15.133:0/3843695495 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646702f0 0x565264625b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 --2- 172.21.15.133:0/3843695495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x56526466fdb0 unknown :-1 s=CLOSED pgs=2970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- 172.21.15.133:0/3843695495 >> 172.21.15.133:0/3843695495 conn(0x565264622020 msgr2=0x565264624440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- 172.21.15.133:0/3843695495 shutdown_connections 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- 172.21.15.133:0/3843695495 wait complete. 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- start start 2024-04-25T10:18:32.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 Processor -- start 2024-04-25T10:18:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646260a0 0x5652646547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56526466f9d0 0x565264654d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264655270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x565264437250 con 0x5652646702f0 2024-04-25T10:18:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526442d690 con 0x56526466f9d0 2024-04-25T10:18:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652643fba40 con 0x5652646260a0 2024-04-25T10:18:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 10 client.0 _ll_register_callbacks cb 0x565264610e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264655270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264655270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58310/0 (socket says 172.21.15.133:58310) 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb5ffff640 1 -- 172.21.15.133:0/3548194379 learned_addr learned my addr 172.21.15.133:0/3548194379 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb5ffff640 1 -- 172.21.15.133:0/3548194379 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646260a0 msgr2=0x5652646547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/3548194379 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646260a0 0x5652646547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb5ffff640 1 -- 172.21.15.133:0/3548194379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56526466f9d0 msgr2=0x565264654d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/3548194379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56526466f9d0 0x565264654d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb5ffff640 1 -- 172.21.15.133:0/3548194379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652645201c0 con 0x5652646702f0 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/3548194379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264655270 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7fcb4805df60 tx=0x7fcb480565c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a422748b14df28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/3548194379 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4806d020 con 0x5652646702f0 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/3548194379 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcb48054080 con 0x5652646702f0 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- 172.21.15.133:0/3548194379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb5001f600 con 0x5652646702f0 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/3548194379 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb48072050 con 0x5652646702f0 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 10 client.12282 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:32.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- 172.21.15.133:0/3548194379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x56526451c170 con 0x5652646702f0 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/3548194379 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5001f600 con 0x5652646702f0 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/3548194379 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb48013070 con 0x5652646702f0 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 10 client.12282 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 10 client.12282 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 20 client.12282 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 --2- 172.21.15.133:0/3548194379 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56526467fc80 0x565264682070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 4 client.12282 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb691a7000 1 -- 172.21.15.133:0/3548194379 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56526440ec60 con 0x56526467fc80 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/3548194379 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56526467fc80 0x565264682070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb3e7fc640 10 client.12282 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.066+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/3548194379 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56526467fc80 0x565264682070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac9a7cc4f2a3edf server_cookie=84022afb44a1f58 in_seq=0 out_seq=0 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/3548194379 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56526440ec60 con 0x56526467fc80 2024-04-25T10:18:32.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb3e7fc640 10 client.12282 handle_command_reply: tid=0 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 1 client.12282 shutdown 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 1 -- 172.21.15.133:0/3548194379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 msgr2=0x565264655270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 1 --2- 172.21.15.133:0/3548194379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264655270 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7fcb4805df60 tx=0x7fcb480565c0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 1 -- 172.21.15.133:0/3548194379 shutdown_connections 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 1 --2- 172.21.15.133:0/3548194379 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x56526467fc80 0x565264682070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 1 --2- 172.21.15.133:0/3548194379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264655270 unknown :-1 s=CLOSED pgs=2971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 1 --2- 172.21.15.133:0/3548194379 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56526466f9d0 0x565264654d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 1 --2- 172.21.15.133:0/3548194379 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646260a0 0x5652646547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb3e7fc640 0 client.12282 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 1 -- 172.21.15.133:0/3548194379 >> 172.21.15.133:0/3548194379 conn(0x565264622020 msgr2=0x565264624ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 1 -- 172.21.15.133:0/3548194379 shutdown_connections 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 1 -- 172.21.15.133:0/3548194379 wait complete. 2024-04-25T10:18:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.070+0000 7fcb691a7000 20 client.12282 trim_cache size 0 max 16384 2024-04-25T10:18:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.082+0000 7fcb691a7000 1 Processor -- start 2024-04-25T10:18:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.082+0000 7fcb691a7000 1 -- start start 2024-04-25T10:18:32.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 0x565264624410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56526466f9d0 0x565264624950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264624e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60203110 con 0x5652646702f0 2024-04-25T10:18:32.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60202310 con 0x5652646260a0 2024-04-25T10:18:32.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601390d0 con 0x56526466f9d0 2024-04-25T10:18:32.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264624e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264624e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58312/0 (socket says 172.21.15.133:58312) 2024-04-25T10:18:32.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5ffff640 1 -- 172.21.15.133:0/1942643589 learned_addr learned my addr 172.21.15.133:0/1942643589 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:32.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5ffff640 1 -- 172.21.15.133:0/1942643589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56526466f9d0 msgr2=0x565264624950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/1942643589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56526466f9d0 0x565264624950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5ffff640 1 -- 172.21.15.133:0/1942643589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 msgr2=0x565264624410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/1942643589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 0x565264624410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5ffff640 1 -- 172.21.15.133:0/1942643589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652644b9de0 con 0x5652646702f0 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/1942643589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264624e90 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7fcb48052410 tx=0x7fcb48020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12a5c67638a6f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb54ff9640 1 -- 172.21.15.133:0/1942643589 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4806d040 con 0x5652646702f0 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb54ff9640 1 -- 172.21.15.133:0/1942643589 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4805c150 con 0x5652646702f0 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb54ff9640 1 -- 172.21.15.133:0/1942643589 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480556e0 con 0x5652646702f0 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- 172.21.15.133:0/1942643589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 msgr2=0x565264624e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 --2- 172.21.15.133:0/1942643589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264624e90 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7fcb48052410 tx=0x7fcb48020000 comp rx=0 tx=0).stop 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- 172.21.15.133:0/1942643589 shutdown_connections 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 --2- 172.21.15.133:0/1942643589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652646702f0 0x565264624e90 unknown :-1 s=CLOSED pgs=2972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 --2- 172.21.15.133:0/1942643589 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x56526466f9d0 0x565264624950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 --2- 172.21.15.133:0/1942643589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 0x565264624410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- 172.21.15.133:0/1942643589 >> 172.21.15.133:0/1942643589 conn(0x565264622020 msgr2=0x56526467cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:32.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- 172.21.15.133:0/1942643589 shutdown_connections 2024-04-25T10:18:32.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- 172.21.15.133:0/1942643589 wait complete. 2024-04-25T10:18:32.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- start start 2024-04-25T10:18:32.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 Processor -- start 2024-04-25T10:18:32.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 0x5652646d8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x5652646d8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646702f0 0x565264681fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526440de00 con 0x56526466f9d0 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60201d60 con 0x5652646260a0 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x565264472300 con 0x5652646702f0 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 10 client.0 _ll_register_callbacks cb 0x5652645f1040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x5652646d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x5652646d8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58320/0 (socket says 172.21.15.133:58320) 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5effd640 1 -- 172.21.15.133:0/902818200 learned_addr learned my addr 172.21.15.133:0/902818200 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5ffff640 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646702f0 0x565264681fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 0x5652646d8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5effd640 1 -- 172.21.15.133:0/902818200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646702f0 msgr2=0x565264681fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5effd640 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646702f0 0x565264681fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5effd640 1 -- 172.21.15.133:0/902818200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 msgr2=0x5652646d8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5effd640 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 0x5652646d8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5effd640 1 -- 172.21.15.133:0/902818200 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652644148e0 con 0x56526466f9d0 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb5effd640 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x5652646d8680 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7fcb580ae040 tx=0x7fcb580ab8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fbe74e9630d21a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/902818200 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb580bc020 con 0x56526466f9d0 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/902818200 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcb580abc20 con 0x56526466f9d0 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.086+0000 7fcb691a7000 1 -- 172.21.15.133:0/902818200 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb4805c150 con 0x56526466f9d0 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/902818200 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb580c6050 con 0x56526466f9d0 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 10 client.12288 fetch_fsmap learned FSMap version 124 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 -- 172.21.15.133:0/902818200 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x5652644cc8f0 con 0x56526466f9d0 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/902818200 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4805c150 con 0x56526466f9d0 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/902818200 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb580c9020 con 0x56526466f9d0 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 10 client.12288 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 10 client.12288 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 20 client.12288 populate_metadata read hostname 'smithi133' 2024-04-25T10:18:32.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x565264682c70 0x565264683080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 4 client.12288 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 -- 172.21.15.133:0/902818200 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565264480530 con 0x565264682c70 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x565264682c70 0x565264683080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb3e7fc640 10 client.12288 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb5f7fe640 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x565264682c70 0x565264683080 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe18e71831aa708 server_cookie=27b798086ddef7ff in_seq=0 out_seq=0 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb3e7fc640 1 -- 172.21.15.133:0/902818200 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x565264480530 con 0x565264682c70 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb3e7fc640 10 client.12288 handle_command_reply: tid=0 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 client.12288 shutdown 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 -- 172.21.15.133:0/902818200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 msgr2=0x5652646d8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x5652646d8680 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7fcb580ae040 tx=0x7fcb580ab8b0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 -- 172.21.15.133:0/902818200 shutdown_connections 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x565264682c70 0x565264683080 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x5652646702f0 0x565264681fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56526466f9d0 0x5652646d8680 unknown :-1 s=CLOSED pgs=2973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb3e7fc640 0 client.12288 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:18:32.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 --2- 172.21.15.133:0/902818200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5652646260a0 0x5652646d8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 -- 172.21.15.133:0/902818200 >> 172.21.15.133:0/902818200 conn(0x565264622020 msgr2=0x56526466e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:32.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 -- 172.21.15.133:0/902818200 shutdown_connections 2024-04-25T10:18:32.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 1 -- 172.21.15.133:0/902818200 wait complete. 2024-04-25T10:18:32.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb691a7000 20 client.12288 trim_cache size 0 max 16384 2024-04-25T10:18:32.096 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:18:32.096 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:18:32.096 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2024-04-25T10:18:32.097 INFO:teuthology.orchestra.run.smithi133.stdout: "e6da3468-4b87-4959-9e23-51ee30a9ae64": { 2024-04-25T10:18:32.097 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2024-04-25T10:18:32.097 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "e6da3468-4b87-4959-9e23-51ee30a9ae64", 2024-04-25T10:18:32.097 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,repair" 2024-04-25T10:18:32.097 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:18:32.097 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T10:18:32.097 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:18:32.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 -- 172.21.15.133:0/350532218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb40066f60 msgr2=0x7fcb40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb40066f60 0x7fcb40069420 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcb50007fd0 tx=0x7fcb50009040 comp rx=0 tx=0).stop 2024-04-25T10:18:32.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 -- 172.21.15.133:0/350532218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb6015eb40 msgr2=0x7fcb60144380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb6015eb40 0x7fcb60144380 secure :-1 s=READY pgs=2969 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800d800 tx=0x7fcb4800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 -- 172.21.15.133:0/350532218 shutdown_connections 2024-04-25T10:18:32.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcb40066f60 0x7fcb40069420 secure :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcb50007fd0 tx=0x7fcb50009040 comp rx=0 tx=0).stop 2024-04-25T10:18:32.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb6015eb40 0x7fcb60144380 unknown :-1 s=CLOSED pgs=2969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcb600aebc0 0x7fcb60143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 --2- 172.21.15.133:0/350532218 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcb6006e8b0 0x7fcb600cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 -- 172.21.15.133:0/350532218 >> 172.21.15.133:0/350532218 conn(0x7fcb60066f70 msgr2=0x7fcb6015fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:32.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 -- 172.21.15.133:0/350532218 shutdown_connections 2024-04-25T10:18:32.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:18:32.090+0000 7fcb6603a640 1 -- 172.21.15.133:0/350532218 wait complete. 2024-04-25T10:18:32.106 INFO:tasks.cephfs.filesystem:scrub status for tag:e6da3468-4b87-4959-9e23-51ee30a9ae64 - {'path': '/', 'tag': 'e6da3468-4b87-4959-9e23-51ee30a9ae64', 'options': 'recursive,repair'} 2024-04-25T10:18:54.466 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:54.469 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:02.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:02.107 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:19:02.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913bf80640 1 Processor -- start 2024-04-25T10:19:02.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913bf80640 1 -- start start 2024-04-25T10:19:02.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913bf80640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f913415d9a0 0x7f91340d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913bf80640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913415ebd0 con 0x7f91340ce600 2024-04-25T10:19:02.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913bf80640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913415eeb0 con 0x7f91340d2030 2024-04-25T10:19:02.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913bf80640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91340d0330 con 0x7f913415d9a0 2024-04-25T10:19:02.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913951b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f91340ce600 0x7f91340d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34314/0 (socket says 172.21.15.133:34314) 2024-04-25T10:19:02.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913951b640 1 -- 172.21.15.133:0/1805791762 learned_addr learned my addr 172.21.15.133:0/1805791762 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:02.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948516709 0 0) 0x7f913415ebd0 con 0x7f91340ce600 2024-04-25T10:19:02.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9139d1c640 1 --2- 172.21.15.133:0/1805791762 >> v2:172.21.15.162:3300/0 conn(0x7f913415d9a0 0x7f91340d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9120003530 con 0x7f91340ce600 2024-04-25T10:19:02.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 109858572 0 0) 0x7f9120003530 con 0x7f91340ce600 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 >> v2:172.21.15.162:3300/0 conn(0x7f913415d9a0 msgr2=0x7f91340d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 --2- 172.21.15.133:0/1805791762 >> v2:172.21.15.162:3300/0 conn(0x7f913415d9a0 0x7f91340d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 >> v1:172.21.15.162:6789/0 conn(0x7f91340d2030 legacy=0x7f913415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340d0610 con 0x7f91340ce600 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f9130001680 con 0x7f91340ce600 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 >> v1:172.21.15.133:6789/0 conn(0x7f91340ce600 legacy=0x7f91340d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 --2- 172.21.15.133:0/1805791762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91200039c0 0x7f9120003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 --2- 172.21.15.133:0/1805791762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9120004810 0x7f9120004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 --2- 172.21.15.133:0/1805791762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91200051d0 0x7f912000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91340d0330 con 0x7f9120004810 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912000c170 con 0x7f91200039c0 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912000c370 con 0x7f91200051d0 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913951b640 1 --2- 172.21.15.133:0/1805791762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9120004810 0x7f9120004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f9139d1c640 1 --2- 172.21.15.133:0/1805791762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91200039c0 0x7f9120003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913a51d640 1 --2- 172.21.15.133:0/1805791762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91200051d0 0x7f912000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913951b640 1 -- 172.21.15.133:0/1805791762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91200051d0 msgr2=0x7f912000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913951b640 1 --2- 172.21.15.133:0/1805791762 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f91200051d0 0x7f912000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913951b640 1 -- 172.21.15.133:0/1805791762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91200039c0 msgr2=0x7f9120003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913951b640 1 --2- 172.21.15.133:0/1805791762 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91200039c0 0x7f9120003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913951b640 1 -- 172.21.15.133:0/1805791762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9120003530 con 0x7f9120004810 2024-04-25T10:19:02.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.245+0000 7f913a51d640 1 -- 172.21.15.133:0/1805791762 reap_dead start 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913951b640 1 --2- 172.21.15.133:0/1805791762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9120004810 0x7f9120004c90 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f913000c1c0 tx=0x7f913001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff416a51c25489ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130007d00 con 0x7f9120004810 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91300a9020 con 0x7f9120004810 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f9138d1a640 1 -- 172.21.15.133:0/1805791762 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300a5b50 con 0x7f9120004810 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 -- 172.21.15.133:0/1805791762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9120004810 msgr2=0x7f9120004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 --2- 172.21.15.133:0/1805791762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9120004810 0x7f9120004c90 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f913000c1c0 tx=0x7f913001cc30 comp rx=0 tx=0).stop 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 -- 172.21.15.133:0/1805791762 shutdown_connections 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 --2- 172.21.15.133:0/1805791762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9120004810 0x7f9120004c90 unknown :-1 s=CLOSED pgs=2975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 -- 172.21.15.133:0/1805791762 >> 172.21.15.133:0/1805791762 conn(0x7f913406bf80 msgr2=0x7f913406c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 -- 172.21.15.133:0/1805791762 shutdown_connections 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 -- 172.21.15.133:0/1805791762 wait complete. 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 Processor -- start 2024-04-25T10:19:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 -- start start 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9120004810 0x7f913415c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f913414d220 0x7f91340cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340cc430 0x7f91341f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91340d1eb0 con 0x7f913414d220 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91340d1d30 con 0x7f91340cc430 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913bf80640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91340cee60 con 0x7f9120004810 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913951b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f913414d220 0x7f91340cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913951b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f913414d220 0x7f91340cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60232/0 (socket says 172.21.15.133:60232) 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.249+0000 7f913951b640 1 -- 172.21.15.133:0/3408257424 learned_addr learned my addr 172.21.15.133:0/3408257424 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f9139d1c640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9120004810 0x7f913415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f913a51d640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340cc430 0x7f91341f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f913951b640 1 -- 172.21.15.133:0/3408257424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9120004810 msgr2=0x7f913415c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f913951b640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9120004810 0x7f913415c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f913951b640 1 -- 172.21.15.133:0/3408257424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340cc430 msgr2=0x7f91341f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f913951b640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340cc430 0x7f91341f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f913951b640 1 -- 172.21.15.133:0/3408257424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913414d9e0 con 0x7f913414d220 2024-04-25T10:19:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f913951b640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f913414d220 0x7f91340cbef0 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7f91300ab040 tx=0x7f9120005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f912a7fc640 1 -- 172.21.15.133:0/3408257424 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300a5b30 con 0x7f913414d220 2024-04-25T10:19:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f912a7fc640 1 -- 172.21.15.133:0/3408257424 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91300b2070 con 0x7f913414d220 2024-04-25T10:19:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f913bf80640 1 -- 172.21.15.133:0/3408257424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913414dcc0 con 0x7f913414d220 2024-04-25T10:19:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f912a7fc640 1 -- 172.21.15.133:0/3408257424 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300047f0 con 0x7f913414d220 2024-04-25T10:19:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f913bf80640 1 -- 172.21.15.133:0/3408257424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913414e280 con 0x7f913414d220 2024-04-25T10:19:02.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f912a7fc640 1 -- 172.21.15.133:0/3408257424 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9130004990 con 0x7f913414d220 2024-04-25T10:19:02.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f912a7fc640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9118066f60 0x7f9118069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f912a7fc640 1 -- 172.21.15.133:0/3408257424 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f913011ea10 con 0x7f913414d220 2024-04-25T10:19:02.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f9139d1c640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9118066f60 0x7f9118069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f913f0ed000 1 Processor -- start 2024-04-25T10:19:02.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f9139d1c640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9118066f60 0x7f9118069420 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f91340ccff0 tx=0x7f9124008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.253+0000 7f913f0ed000 1 -- start start 2024-04-25T10:19:02.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c739d0 0x556778c73db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c742f0 0x556778c29b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c2a0a0 0x556778c2a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556778a76ff0 con 0x556778c742f0 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567789ff150 con 0x556778c739d0 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556778a73090 con 0x556778c2a0a0 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913951b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c742f0 0x556778c29b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913951b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c742f0 0x556778c29b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60246/0 (socket says 172.21.15.133:60246) 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913951b640 1 -- 172.21.15.133:0/1319478204 learned_addr learned my addr 172.21.15.133:0/1319478204 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f9139d1c640 1 --2- 172.21.15.133:0/1319478204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c739d0 0x556778c73db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913a51d640 1 --2- 172.21.15.133:0/1319478204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c2a0a0 0x556778c2a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913951b640 1 -- 172.21.15.133:0/1319478204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c2a0a0 msgr2=0x556778c2a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913951b640 1 --2- 172.21.15.133:0/1319478204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c2a0a0 0x556778c2a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913951b640 1 -- 172.21.15.133:0/1319478204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c739d0 msgr2=0x556778c73db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913951b640 1 --2- 172.21.15.133:0/1319478204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c739d0 0x556778c73db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913951b640 1 -- 172.21.15.133:0/1319478204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556778c0d0d0 con 0x556778c742f0 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f9139d1c640 1 --2- 172.21.15.133:0/1319478204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c739d0 0x556778c73db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:02.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913951b640 1 --2- 172.21.15.133:0/1319478204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c742f0 0x556778c29b60 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7f91300eb650 tx=0x7f91300eb510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14184d493e934a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f91177fe640 1 -- 172.21.15.133:0/1319478204 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300be030 con 0x556778c742f0 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f91177fe640 1 -- 172.21.15.133:0/1319478204 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91300a9040 con 0x556778c742f0 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f91177fe640 1 -- 172.21.15.133:0/1319478204 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300eb860 con 0x556778c742f0 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- 172.21.15.133:0/1319478204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c742f0 msgr2=0x556778c29b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 --2- 172.21.15.133:0/1319478204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c742f0 0x556778c29b60 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7f91300eb650 tx=0x7f91300eb510 comp rx=0 tx=0).stop 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- 172.21.15.133:0/1319478204 shutdown_connections 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 --2- 172.21.15.133:0/1319478204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c2a0a0 0x556778c2a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 --2- 172.21.15.133:0/1319478204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c742f0 0x556778c29b60 unknown :-1 s=CLOSED pgs=2977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 --2- 172.21.15.133:0/1319478204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c739d0 0x556778c73db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- 172.21.15.133:0/1319478204 >> 172.21.15.133:0/1319478204 conn(0x556778c26020 msgr2=0x556778c28440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- 172.21.15.133:0/1319478204 shutdown_connections 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- 172.21.15.133:0/1319478204 wait complete. 2024-04-25T10:19:02.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- start start 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 Processor -- start 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c2a0a0 0x556778c587f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c739d0 0x556778c58d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c742f0 0x556778c59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556778a3b250 con 0x556778c2a0a0 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x556778a31690 con 0x556778c742f0 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567789ffa40 con 0x556778c739d0 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f913f0ed000 10 client.0 _ll_register_callbacks cb 0x556778c14e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f9139d1c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c2a0a0 0x556778c587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f9139d1c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c2a0a0 0x556778c587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60254/0 (socket says 172.21.15.133:60254) 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.257+0000 7f9139d1c640 1 -- 172.21.15.133:0/2088103459 learned_addr learned my addr 172.21.15.133:0/2088103459 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913951b640 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c739d0 0x556778c58d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913a51d640 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c742f0 0x556778c59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9139d1c640 1 -- 172.21.15.133:0/2088103459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c739d0 msgr2=0x556778c58d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9139d1c640 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c739d0 0x556778c58d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9139d1c640 1 -- 172.21.15.133:0/2088103459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c742f0 msgr2=0x556778c59270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9139d1c640 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c742f0 0x556778c59270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9139d1c640 1 -- 172.21.15.133:0/2088103459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556778b241c0 con 0x556778c2a0a0 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913951b640 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c739d0 0x556778c58d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913a51d640 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c742f0 0x556778c59270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9139d1c640 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c2a0a0 0x556778c587f0 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7f9124016490 tx=0x7f9124004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92607ef6b0df3871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9114ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:19:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9114ff9640 1 -- 172.21.15.133:0/2088103459 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124028070 con 0x556778c2a0a0 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 -- 172.21.15.133:0/2088103459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91300a9040 con 0x556778c2a0a0 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9114ff9640 1 -- 172.21.15.133:0/2088103459 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f9124024830 con 0x556778c2a0a0 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9114ff9640 1 -- 172.21.15.133:0/2088103459 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9124033050 con 0x556778c2a0a0 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 10 client.12300 fetch_fsmap learned FSMap version 124 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 -- 172.21.15.133:0/2088103459 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x556778b20170 con 0x556778c2a0a0 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9114ff9640 1 -- 172.21.15.133:0/2088103459 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91300a9040 con 0x556778c2a0a0 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9114ff9640 1 -- 172.21.15.133:0/2088103459 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9124036020 con 0x556778c2a0a0 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 10 client.12300 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 10 client.12300 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 20 client.12300 populate_metadata read hostname 'smithi133' 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556778c83c80 0x556778c86070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 4 client.12300 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 -- 172.21.15.133:0/2088103459 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556778a12c60 con 0x556778c83c80 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913951b640 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556778c83c80 0x556778c86070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9114ff9640 10 client.12300 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913951b640 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556778c83c80 0x556778c86070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9c4b8b95d9fca3 server_cookie=e193722b280ebaf3 in_seq=0 out_seq=0 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9114ff9640 1 -- 172.21.15.133:0/2088103459 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556778a12c60 con 0x556778c83c80 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9114ff9640 10 client.12300 handle_command_reply: tid=0 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 client.12300 shutdown 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 -- 172.21.15.133:0/2088103459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c2a0a0 msgr2=0x556778c587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c2a0a0 0x556778c587f0 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7f9124016490 tx=0x7f9124004740 comp rx=0 tx=0).stop 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 -- 172.21.15.133:0/2088103459 shutdown_connections 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556778c83c80 0x556778c86070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c742f0 0x556778c59270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c739d0 0x556778c58d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f9114ff9640 0 client.12300 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2088103459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c2a0a0 0x556778c587f0 unknown :-1 s=CLOSED pgs=2978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 -- 172.21.15.133:0/2088103459 >> 172.21.15.133:0/2088103459 conn(0x556778c26020 msgr2=0x556778c28ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 -- 172.21.15.133:0/2088103459 shutdown_connections 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 1 -- 172.21.15.133:0/2088103459 wait complete. 2024-04-25T10:19:02.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.261+0000 7f913f0ed000 20 client.12300 trim_cache size 0 max 16384 2024-04-25T10:19:02.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913f0ed000 1 Processor -- start 2024-04-25T10:19:02.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913f0ed000 1 -- start start 2024-04-25T10:19:02.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c2a0a0 0x556778c28410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778c28950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c742f0 0x556778c28e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913f0ed000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91342042d0 con 0x556778c739d0 2024-04-25T10:19:02.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913f0ed000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91342034b0 con 0x556778c2a0a0 2024-04-25T10:19:02.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913f0ed000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913414bce0 con 0x556778c742f0 2024-04-25T10:19:02.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913951b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778c28950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913951b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778c28950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60256/0 (socket says 172.21.15.133:60256) 2024-04-25T10:19:02.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913951b640 1 -- 172.21.15.133:0/3915900863 learned_addr learned my addr 172.21.15.133:0/3915900863 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:02.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913a51d640 1 --2- 172.21.15.133:0/3915900863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c742f0 0x556778c28e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913951b640 1 -- 172.21.15.133:0/3915900863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c742f0 msgr2=0x556778c28e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913951b640 1 --2- 172.21.15.133:0/3915900863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c742f0 0x556778c28e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913951b640 1 -- 172.21.15.133:0/3915900863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c2a0a0 msgr2=0x556778c28410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f9139d1c640 1 --2- 172.21.15.133:0/3915900863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c2a0a0 0x556778c28410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913951b640 1 --2- 172.21.15.133:0/3915900863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c2a0a0 0x556778c28410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913951b640 1 -- 172.21.15.133:0/3915900863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556778abdde0 con 0x556778c739d0 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f913951b640 1 --2- 172.21.15.133:0/3915900863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778c28950 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7f91300efaa0 tx=0x7f91300bb000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7151d653a43679 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f9116ffd640 1 -- 172.21.15.133:0/3915900863 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300be040 con 0x556778c739d0 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f9116ffd640 1 -- 172.21.15.133:0/3915900863 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91300a9050 con 0x556778c739d0 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.277+0000 7f9116ffd640 1 -- 172.21.15.133:0/3915900863 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300eb860 con 0x556778c739d0 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- 172.21.15.133:0/3915900863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 msgr2=0x556778c28950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- 172.21.15.133:0/3915900863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778c28950 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7f91300efaa0 tx=0x7f91300bb000 comp rx=0 tx=0).stop 2024-04-25T10:19:02.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- 172.21.15.133:0/3915900863 shutdown_connections 2024-04-25T10:19:02.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- 172.21.15.133:0/3915900863 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c742f0 0x556778c28e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- 172.21.15.133:0/3915900863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778c28950 unknown :-1 s=CLOSED pgs=2979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- 172.21.15.133:0/3915900863 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c2a0a0 0x556778c28410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- 172.21.15.133:0/3915900863 >> 172.21.15.133:0/3915900863 conn(0x556778c26020 msgr2=0x556778c80fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- 172.21.15.133:0/3915900863 shutdown_connections 2024-04-25T10:19:02.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- 172.21.15.133:0/3915900863 wait complete. 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- start start 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 Processor -- start 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c2a0a0 0x556778cd3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778cd3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c742f0 0x556778c85fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556778a11e00 con 0x556778c739d0 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9134203050 con 0x556778c2a0a0 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x556778a76300 con 0x556778c742f0 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 10 client.0 _ll_register_callbacks cb 0x556778bf5040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913951b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778cd3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913951b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778cd3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60262/0 (socket says 172.21.15.133:60262) 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913951b640 1 -- 172.21.15.133:0/2108155529 learned_addr learned my addr 172.21.15.133:0/2108155529 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913a51d640 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c742f0 0x556778c85fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9139d1c640 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c2a0a0 0x556778cd3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913951b640 1 -- 172.21.15.133:0/2108155529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c742f0 msgr2=0x556778c85fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913951b640 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c742f0 0x556778c85fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913951b640 1 -- 172.21.15.133:0/2108155529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c2a0a0 msgr2=0x556778cd3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913951b640 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c2a0a0 0x556778cd3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913951b640 1 -- 172.21.15.133:0/2108155529 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556778a188e0 con 0x556778c739d0 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913a51d640 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c742f0 0x556778c85fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9139d1c640 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c2a0a0 0x556778cd3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913951b640 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778cd3e70 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f913001d110 tx=0x7f913001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13385892e39d28dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9114ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9114ff9640 1 -- 172.21.15.133:0/2108155529 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300ff050 con 0x556778c739d0 2024-04-25T10:19:02.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9114ff9640 1 -- 172.21.15.133:0/2108155529 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f91300bad30 con 0x556778c739d0 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- 172.21.15.133:0/2108155529 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91300a9050 con 0x556778c739d0 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9114ff9640 1 -- 172.21.15.133:0/2108155529 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f913010e050 con 0x556778c739d0 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 10 client.12306 fetch_fsmap learned FSMap version 124 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- 172.21.15.133:0/2108155529 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x556778ad08f0 con 0x556778c739d0 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9114ff9640 1 -- 172.21.15.133:0/2108155529 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91300a9050 con 0x556778c739d0 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9114ff9640 1 -- 172.21.15.133:0/2108155529 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9130111020 con 0x556778c739d0 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 10 client.12306 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 10 client.12306 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 20 client.12306 populate_metadata read hostname 'smithi133' 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556778c86950 0x556778c86d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 4 client.12306 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- 172.21.15.133:0/2108155529 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556778a84530 con 0x556778c86950 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9139d1c640 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556778c86950 0x556778c86d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9114ff9640 10 client.12306 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9139d1c640 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556778c86950 0x556778c86d60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6c04c4fdaa8ec0 server_cookie=81d22827914e22a0 in_seq=0 out_seq=0 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9114ff9640 1 -- 172.21.15.133:0/2108155529 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556778a84530 con 0x556778c86950 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9114ff9640 10 client.12306 handle_command_reply: tid=0 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 client.12306 shutdown 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- 172.21.15.133:0/2108155529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 msgr2=0x556778cd3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778cd3e70 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f913001d110 tx=0x7f913001cc30 comp rx=0 tx=0).stop 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- 172.21.15.133:0/2108155529 shutdown_connections 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x556778c86950 0x556778c86d60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x556778c742f0 0x556778c85fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556778c739d0 0x556778cd3e70 unknown :-1 s=CLOSED pgs=2980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 --2- 172.21.15.133:0/2108155529 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556778c2a0a0 0x556778cd3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f9114ff9640 0 client.12306 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.281+0000 7f913f0ed000 1 -- 172.21.15.133:0/2108155529 >> 172.21.15.133:0/2108155529 conn(0x556778c26020 msgr2=0x556778c720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913f0ed000 1 -- 172.21.15.133:0/2108155529 shutdown_connections 2024-04-25T10:19:02.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913f0ed000 1 -- 172.21.15.133:0/2108155529 wait complete. 2024-04-25T10:19:02.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913f0ed000 20 client.12306 trim_cache size 0 max 16384 2024-04-25T10:19:02.291 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:19:02.291 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2024-04-25T10:19:02.291 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2024-04-25T10:19:02.291 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:19:02.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 -- 172.21.15.133:0/3408257424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9118066f60 msgr2=0x7f9118069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9118066f60 0x7f9118069420 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f91340ccff0 tx=0x7f9124008040 comp rx=0 tx=0).stop 2024-04-25T10:19:02.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 -- 172.21.15.133:0/3408257424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f913414d220 msgr2=0x7f91340cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f913414d220 0x7f91340cbef0 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7f91300ab040 tx=0x7f9120005840 comp rx=0 tx=0).stop 2024-04-25T10:19:02.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 -- 172.21.15.133:0/3408257424 shutdown_connections 2024-04-25T10:19:02.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91340cc430 0x7f91341f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f9118066f60 0x7f9118069420 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f913414d220 0x7f91340cbef0 unknown :-1 s=CLOSED pgs=2976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 --2- 172.21.15.133:0/3408257424 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f9120004810 0x7f913415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 -- 172.21.15.133:0/3408257424 >> 172.21.15.133:0/3408257424 conn(0x7f913406bf80 msgr2=0x7f91340b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 -- 172.21.15.133:0/3408257424 shutdown_connections 2024-04-25T10:19:02.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.285+0000 7f913bf80640 1 -- 172.21.15.133:0/3408257424 wait complete. 2024-04-25T10:19:02.302 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:19:02.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:02.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.837+0000 7fcc7d34f640 1 Processor -- start 2024-04-25T10:19:02.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.837+0000 7fcc7d34f640 1 -- start start 2024-04-25T10:19:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fcc7815c9a0 0x7fcc780c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcc780b1f70 0x7fcc7806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc780c61d0 con 0x7fcc780ce330 2024-04-25T10:19:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc780c64b0 con 0x7fcc780b1f70 2024-04-25T10:19:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7806d4f0 con 0x7fcc7815c9a0 2024-04-25T10:19:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc767fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcc780b1f70 0x7fcc7806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc767fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcc780b1f70 0x7fcc7806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60270/0 (socket says 172.21.15.133:60270) 2024-04-25T10:19:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc767fc640 1 -- 172.21.15.133:0/469234960 learned_addr learned my addr 172.21.15.133:0/469234960 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc76ffd640 1 --2- 172.21.15.133:0/469234960 >> v2:172.21.15.162:3300/0 conn(0x7fcc7815c9a0 0x7fcc780c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc767fc640 1 -- 172.21.15.133:0/469234960 >> v2:172.21.15.162:3300/0 conn(0x7fcc7815c9a0 msgr2=0x7fcc780c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc767fc640 1 --2- 172.21.15.133:0/469234960 >> v2:172.21.15.162:3300/0 conn(0x7fcc7815c9a0 0x7fcc780c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc767fc640 1 -- 172.21.15.133:0/469234960 >> v1:172.21.15.162:6789/0 conn(0x7fcc780ce330 legacy=0x7fcc7815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc767fc640 1 -- 172.21.15.133:0/469234960 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7806d7d0 con 0x7fcc780b1f70 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc767fc640 1 --2- 172.21.15.133:0/469234960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc780b1f70 0x7fcc7806dcf0 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7fcc7806f5f0 tx=0x7fcc6c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22374d13b06de888 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc75ffb640 1 -- 172.21.15.133:0/469234960 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0aa600 con 0x7fcc780b1f70 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc75ffb640 1 -- 172.21.15.133:0/469234960 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c0af070 con 0x7fcc780b1f70 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc75ffb640 1 -- 172.21.15.133:0/469234960 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0aa9c0 con 0x7fcc780b1f70 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 -- 172.21.15.133:0/469234960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc780b1f70 msgr2=0x7fcc7806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 --2- 172.21.15.133:0/469234960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc780b1f70 0x7fcc7806dcf0 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7fcc7806f5f0 tx=0x7fcc6c01c480 comp rx=0 tx=0).stop 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 -- 172.21.15.133:0/469234960 shutdown_connections 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 --2- 172.21.15.133:0/469234960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc780b1f70 0x7fcc7806dcf0 unknown :-1 s=CLOSED pgs=2981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 --2- 172.21.15.133:0/469234960 >> v2:172.21.15.162:3300/0 conn(0x7fcc7815c9a0 0x7fcc780c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 -- 172.21.15.133:0/469234960 >> 172.21.15.133:0/469234960 conn(0x7fcc78067000 msgr2=0x7fcc7806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 -- 172.21.15.133:0/469234960 shutdown_connections 2024-04-25T10:19:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.841+0000 7fcc7d34f640 1 -- 172.21.15.133:0/469234960 wait complete. 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc7d34f640 1 Processor -- start 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc7d34f640 1 -- start start 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc7d34f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc780b1f70 0x7fcc78159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc7d34f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc780ce330 0x7fcc78156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc7d34f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc7815c9a0 0x7fcc78146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc7d34f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7815cd80 con 0x7fcc780b1f70 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc7d34f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7806e230 con 0x7fcc780ce330 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc7d34f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7815cf00 con 0x7fcc7815c9a0 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc76ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc780b1f70 0x7fcc78159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc76ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc780b1f70 0x7fcc78159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60282/0 (socket says 172.21.15.133:60282) 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc76ffd640 1 -- 172.21.15.133:0/3489151735 learned_addr learned my addr 172.21.15.133:0/3489151735 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc767fc640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc780ce330 0x7fcc78156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc777fe640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc7815c9a0 0x7fcc78146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc76ffd640 1 -- 172.21.15.133:0/3489151735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc7815c9a0 msgr2=0x7fcc78146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc76ffd640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc7815c9a0 0x7fcc78146ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc76ffd640 1 -- 172.21.15.133:0/3489151735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc780ce330 msgr2=0x7fcc78156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc76ffd640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc780ce330 0x7fcc78156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc76ffd640 1 -- 172.21.15.133:0/3489151735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc78156fe0 con 0x7fcc780b1f70 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc76ffd640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc780b1f70 0x7fcc78159680 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800c820 tx=0x7fcc6800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc5f7fe640 1 -- 172.21.15.133:0/3489151735 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68015070 con 0x7fcc780b1f70 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc5f7fe640 1 -- 172.21.15.133:0/3489151735 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc68004240 con 0x7fcc780b1f70 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc7d34f640 1 -- 172.21.15.133:0/3489151735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7814f520 con 0x7fcc780b1f70 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc5f7fe640 1 -- 172.21.15.133:0/3489151735 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800ce10 con 0x7fcc780b1f70 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc7d34f640 1 -- 172.21.15.133:0/3489151735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7814f7c0 con 0x7fcc780b1f70 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.845+0000 7fcc7d34f640 1 -- 172.21.15.133:0/3489151735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7806bc20 con 0x7fcc780b1f70 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.849+0000 7fcc5f7fe640 1 -- 172.21.15.133:0/3489151735 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc68007830 con 0x7fcc780b1f70 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.849+0000 7fcc5f7fe640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc64066f90 0x7fcc64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.849+0000 7fcc5f7fe640 1 -- 172.21.15.133:0/3489151735 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fcc68085060 con 0x7fcc780b1f70 2024-04-25T10:19:02.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.849+0000 7fcc767fc640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc64066f90 0x7fcc64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.849+0000 7fcc767fc640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc64066f90 0x7fcc64069450 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0ae000 tx=0x7fcc6c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:02.853+0000 7fcc5f7fe640 1 -- 172.21.15.133:0/3489151735 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc78156fe0 con 0x7fcc780b1f70 2024-04-25T10:19:03.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.041+0000 7fcc7d34f640 1 -- 172.21.15.133:0/3489151735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc780c5c80 con 0x7fcc780b1f70 2024-04-25T10:19:03.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.041+0000 7fcc5f7fe640 1 -- 172.21.15.133:0/3489151735 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4963 (secure 0 0 0) 0x7fcc68055eb0 con 0x7fcc780b1f70 2024-04-25T10:19:03.044 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:03.044 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":124,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:36.969368+0000","modified":"2024-04-25T10:17:58.776693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":196,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12135":{"gid":12135,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/2510709242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2510709242},{"type":"v1","addr":"172.21.15.133:6837","nonce":2510709242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:19:03.044 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 124 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 -- 172.21.15.133:0/3489151735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc64066f90 msgr2=0x7fcc64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc64066f90 0x7fcc64069450 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0ae000 tx=0x7fcc6c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 -- 172.21.15.133:0/3489151735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc780b1f70 msgr2=0x7fcc78159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc780b1f70 0x7fcc78159680 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800c820 tx=0x7fcc6800cce0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 -- 172.21.15.133:0/3489151735 shutdown_connections 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fcc7815c9a0 0x7fcc78146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcc780ce330 0x7fcc78156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcc64066f90 0x7fcc64069450 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 --2- 172.21.15.133:0/3489151735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc780b1f70 0x7fcc78159680 unknown :-1 s=CLOSED pgs=2982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 -- 172.21.15.133:0/3489151735 >> 172.21.15.133:0/3489151735 conn(0x7fcc78067000 msgr2=0x7fcc7815e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 -- 172.21.15.133:0/3489151735 shutdown_connections 2024-04-25T10:19:03.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.045+0000 7fcc7d34f640 1 -- 172.21.15.133:0/3489151735 wait complete. 2024-04-25T10:19:03.061 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:36.969368+0000', 'modified': '2024-04-25T10:17:58.776693+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 196, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12135': {'gid': 12135, 'name': 'c', 'rank': 0, 'incarnation': 121, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.133:6837/2510709242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 2510709242}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 2510709242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:19:03.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:19:03.062 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:19:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.181+0000 7f5087f3c640 1 Processor -- start 2024-04-25T10:19:03.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.181+0000 7f5087f3c640 1 -- start start 2024-04-25T10:19:03.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.181+0000 7f5087f3c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f508015e100 0x7f5080140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.181+0000 7f5087f3c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50800cf280 con 0x7f5080143560 2024-04-25T10:19:03.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.181+0000 7f5087f3c640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508006d440 con 0x7f5080142920 2024-04-25T10:19:03.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.181+0000 7f5087f3c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508006d720 con 0x7f508015e100 2024-04-25T10:19:03.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.181+0000 7f50854d7640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5080143560 0x7f508015d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34330/0 (socket says 172.21.15.133:34330) 2024-04-25T10:19:03.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.181+0000 7f50854d7640 1 -- 172.21.15.133:0/561259957 learned_addr learned my addr 172.21.15.133:0/561259957 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.181+0000 7f50864d9640 1 --2- 172.21.15.133:0/561259957 >> v2:172.21.15.133:3300/0 conn(0x7f508015e100 0x7f5080140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f50864d9640 1 -- 172.21.15.133:0/561259957 >> v1:172.21.15.162:6789/0 conn(0x7f5080142920 legacy=0x7f50800ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f50864d9640 1 -- 172.21.15.133:0/561259957 >> v1:172.21.15.133:6789/0 conn(0x7f5080143560 legacy=0x7f508015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f50864d9640 1 -- 172.21.15.133:0/561259957 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508006da00 con 0x7f508015e100 2024-04-25T10:19:03.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f5084cd6640 1 -- 172.21.15.133:0/561259957 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614711363 0 0) 0x7f50800cf280 con 0x7f5080143560 2024-04-25T10:19:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f50864d9640 1 --2- 172.21.15.133:0/561259957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f508015e100 0x7f5080140a40 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f5078009d90 tx=0x7f507801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7107f46bbe99d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f5084cd6640 1 -- 172.21.15.133:0/561259957 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780b2070 con 0x7f508015e100 2024-04-25T10:19:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f5084cd6640 1 -- 172.21.15.133:0/561259957 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50780a57f0 con 0x7f508015e100 2024-04-25T10:19:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f5084cd6640 1 -- 172.21.15.133:0/561259957 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780a5bc0 con 0x7f508015e100 2024-04-25T10:19:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f5087f3c640 1 -- 172.21.15.133:0/561259957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f508015e100 msgr2=0x7f5080140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f5087f3c640 1 --2- 172.21.15.133:0/561259957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f508015e100 0x7f5080140a40 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f5078009d90 tx=0x7f507801c8a0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f5087f3c640 1 -- 172.21.15.133:0/561259957 shutdown_connections 2024-04-25T10:19:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f5087f3c640 1 --2- 172.21.15.133:0/561259957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f508015e100 0x7f5080140a40 unknown :-1 s=CLOSED pgs=2984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f5087f3c640 1 -- 172.21.15.133:0/561259957 >> 172.21.15.133:0/561259957 conn(0x7f5080066f70 msgr2=0x7f508006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f5087f3c640 1 -- 172.21.15.133:0/561259957 shutdown_connections 2024-04-25T10:19:03.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.185+0000 7f5087f3c640 1 -- 172.21.15.133:0/561259957 wait complete. 2024-04-25T10:19:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5087f3c640 1 Processor -- start 2024-04-25T10:19:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5087f3c640 1 -- start start 2024-04-25T10:19:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5087f3c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5080142920 0x7f508015cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5087f3c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5080143560 0x7f508015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5087f3c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f508015e100 0x7f5080131880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5087f3c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508006e9c0 con 0x7f508015e100 2024-04-25T10:19:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5087f3c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5080143980 con 0x7f5080143560 2024-04-25T10:19:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5087f3c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508006eb40 con 0x7f5080142920 2024-04-25T10:19:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50864d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f508015e100 0x7f5080131880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50864d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f508015e100 0x7f5080131880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60294/0 (socket says 172.21.15.133:60294) 2024-04-25T10:19:03.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50864d9640 1 -- 172.21.15.133:0/67843798 learned_addr learned my addr 172.21.15.133:0/67843798 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50854d7640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5080143560 0x7f508015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5085cd8640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5080142920 0x7f508015cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50864d9640 1 -- 172.21.15.133:0/67843798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5080142920 msgr2=0x7f508015cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50864d9640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5080142920 0x7f508015cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50864d9640 1 -- 172.21.15.133:0/67843798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5080143560 msgr2=0x7f508015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50864d9640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5080143560 0x7f508015aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50864d9640 1 -- 172.21.15.133:0/67843798 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50800ce4a0 con 0x7f508015e100 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5085cd8640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5080142920 0x7f508015cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50854d7640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5080143560 0x7f508015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50864d9640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f508015e100 0x7f5080131880 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 crypto rx=0x7f50780b0040 tx=0x7f5078019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50767fc640 1 -- 172.21.15.133:0/67843798 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780bd030 con 0x7f508015e100 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50767fc640 1 -- 172.21.15.133:0/67843798 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50780a55f0 con 0x7f508015e100 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50767fc640 1 -- 172.21.15.133:0/67843798 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780b2040 con 0x7f508015e100 2024-04-25T10:19:03.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5087f3c640 1 -- 172.21.15.133:0/67843798 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50800b03e0 con 0x7f508015e100 2024-04-25T10:19:03.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f5087f3c640 1 -- 172.21.15.133:0/67843798 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50800b09a0 con 0x7f508015e100 2024-04-25T10:19:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.189+0000 7f50767fc640 1 -- 172.21.15.133:0/67843798 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50780c0070 con 0x7f508015e100 2024-04-25T10:19:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f50767fc640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5064066f60 0x7f5064069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f5085cd8640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5064066f60 0x7f5064069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f50767fc640 1 -- 172.21.15.133:0/67843798 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f507811fbc0 con 0x7f508015e100 2024-04-25T10:19:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f5085cd8640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5064066f60 0x7f5064069420 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f50800afe90 tx=0x7f507c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 Processor -- start 2024-04-25T10:19:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 -- start start 2024-04-25T10:19:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74dd5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74dd6240 0x559f74d8b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74d8bf30 0x559f74d8c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f74c735f0 con 0x559f74dd5920 2024-04-25T10:19:03.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f74bdada0 con 0x559f74dd6240 2024-04-25T10:19:03.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f74b63150 con 0x559f74d8bf30 2024-04-25T10:19:03.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f5085cd8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74dd5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f5085cd8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74dd5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60310/0 (socket says 172.21.15.133:60310) 2024-04-25T10:19:03.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f5085cd8640 1 -- 172.21.15.133:0/3991345398 learned_addr learned my addr 172.21.15.133:0/3991345398 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:03.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f50864d9640 1 --2- 172.21.15.133:0/3991345398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74d8bf30 0x559f74d8c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f50854d7640 1 --2- 172.21.15.133:0/3991345398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74dd6240 0x559f74d8b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f5085cd8640 1 -- 172.21.15.133:0/3991345398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74d8bf30 msgr2=0x559f74d8c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f5085cd8640 1 --2- 172.21.15.133:0/3991345398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74d8bf30 0x559f74d8c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f5085cd8640 1 -- 172.21.15.133:0/3991345398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74dd6240 msgr2=0x559f74d8b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f5085cd8640 1 --2- 172.21.15.133:0/3991345398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74dd6240 0x559f74d8b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f5085cd8640 1 -- 172.21.15.133:0/3991345398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f74b7cca0 con 0x559f74dd5920 2024-04-25T10:19:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f5085cd8640 1 --2- 172.21.15.133:0/3991345398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74dd5d00 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7f507c003ca0 tx=0x7f507c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72dc7d8307f80eee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f50637fe640 1 -- 172.21.15.133:0/3991345398 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c024070 con 0x559f74dd5920 2024-04-25T10:19:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f50637fe640 1 -- 172.21.15.133:0/3991345398 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c014950 con 0x559f74dd5920 2024-04-25T10:19:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f50637fe640 1 -- 172.21.15.133:0/3991345398 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c014d20 con 0x559f74dd5920 2024-04-25T10:19:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 -- 172.21.15.133:0/3991345398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 msgr2=0x559f74dd5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 --2- 172.21.15.133:0/3991345398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74dd5d00 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7f507c003ca0 tx=0x7f507c010000 comp rx=0 tx=0).stop 2024-04-25T10:19:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 -- 172.21.15.133:0/3991345398 shutdown_connections 2024-04-25T10:19:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 --2- 172.21.15.133:0/3991345398 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74d8bf30 0x559f74d8c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 --2- 172.21.15.133:0/3991345398 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74dd6240 0x559f74d8b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 --2- 172.21.15.133:0/3991345398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74dd5d00 unknown :-1 s=CLOSED pgs=2986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 -- 172.21.15.133:0/3991345398 >> 172.21.15.133:0/3991345398 conn(0x559f74d922a0 msgr2=0x559f74d8a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 -- 172.21.15.133:0/3991345398 shutdown_connections 2024-04-25T10:19:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.193+0000 7f508b0a9000 1 -- 172.21.15.133:0/3991345398 wait complete. 2024-04-25T10:19:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- start start 2024-04-25T10:19:03.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 Processor -- start 2024-04-25T10:19:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74d8bf30 0x559f74ddccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74dd5920 0x559f74ddd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd6240 0x559f74d8faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f74b95990 con 0x559f74dd6240 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f74bd6e40 con 0x559f74dd5920 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f74b66900 con 0x559f74d8bf30 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 10 client.0 _ll_register_callbacks cb 0x559f74c8e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f50864d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd6240 0x559f74d8faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f50864d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd6240 0x559f74d8faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60320/0 (socket says 172.21.15.133:60320) 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f50864d9640 1 -- 172.21.15.133:0/255907257 learned_addr learned my addr 172.21.15.133:0/255907257 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5085cd8640 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74d8bf30 0x559f74ddccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f50854d7640 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74dd5920 0x559f74ddd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f50864d9640 1 -- 172.21.15.133:0/255907257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74d8bf30 msgr2=0x559f74ddccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f50864d9640 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74d8bf30 0x559f74ddccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f50864d9640 1 -- 172.21.15.133:0/255907257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74dd5920 msgr2=0x559f74ddd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f50864d9640 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74dd5920 0x559f74ddd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f50864d9640 1 -- 172.21.15.133:0/255907257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f74b93630 con 0x559f74dd6240 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f50864d9640 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd6240 0x559f74d8faa0 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7f50780ec640 tx=0x7f50780f08e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baca59098a106609 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5060ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5060ff9640 1 -- 172.21.15.133:0/255907257 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780f4330 con 0x559f74dd6240 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- 172.21.15.133:0/255907257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f507c014950 con 0x559f74dd6240 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5060ff9640 1 -- 172.21.15.133:0/255907257 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f50780f40d0 con 0x559f74dd6240 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5060ff9640 1 -- 172.21.15.133:0/255907257 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50780f88f0 con 0x559f74dd6240 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 10 client.12327 fetch_fsmap learned FSMap version 124 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- 172.21.15.133:0/255907257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x559f74b633a0 con 0x559f74dd6240 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5060ff9640 1 -- 172.21.15.133:0/255907257 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c014950 con 0x559f74dd6240 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5060ff9640 1 -- 172.21.15.133:0/255907257 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f50780f8460 con 0x559f74dd6240 2024-04-25T10:19:03.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 10 client.12327 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:19:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 10 client.12327 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:19:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 20 client.12327 populate_metadata read hostname 'smithi133' 2024-04-25T10:19:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x559f74d91160 0x559f74de6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 4 client.12327 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- 172.21.15.133:0/255907257 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f74b76a50 con 0x559f74d91160 2024-04-25T10:19:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5085cd8640 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x559f74d91160 0x559f74de6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5060ff9640 10 client.12327 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:19:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5085cd8640 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x559f74d91160 0x559f74de6d50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e274c89db86e5073 server_cookie=f6f5d85030c5883a in_seq=0 out_seq=0 2024-04-25T10:19:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5060ff9640 1 -- 172.21.15.133:0/255907257 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559f74b76a50 con 0x559f74d91160 2024-04-25T10:19:03.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5060ff9640 10 client.12327 handle_command_reply: tid=0 2024-04-25T10:19:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 client.12327 shutdown 2024-04-25T10:19:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- 172.21.15.133:0/255907257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd6240 msgr2=0x559f74d8faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd6240 0x559f74d8faa0 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7f50780ec640 tx=0x7f50780f08e0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- 172.21.15.133:0/255907257 shutdown_connections 2024-04-25T10:19:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x559f74d91160 0x559f74de6d50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd6240 0x559f74d8faa0 unknown :-1 s=CLOSED pgs=2987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74dd5920 0x559f74ddd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 --2- 172.21.15.133:0/255907257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74d8bf30 0x559f74ddccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f5060ff9640 0 client.12327 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:19:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- 172.21.15.133:0/255907257 >> 172.21.15.133:0/255907257 conn(0x559f74d922a0 msgr2=0x559f74d8a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- 172.21.15.133:0/255907257 shutdown_connections 2024-04-25T10:19:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 1 -- 172.21.15.133:0/255907257 wait complete. 2024-04-25T10:19:03.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.197+0000 7f508b0a9000 20 client.12327 trim_cache size 0 max 16384 2024-04-25T10:19:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.213+0000 7f508b0a9000 1 Processor -- start 2024-04-25T10:19:03.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.213+0000 7f508b0a9000 1 -- start start 2024-04-25T10:19:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74d8bf30 0x559f74de2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74de3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74dd6240 0x559f74df07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5080200800 con 0x559f74dd5920 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50801f8fd0 con 0x559f74d8bf30 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5080138fa0 con 0x559f74dd6240 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74de3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74de3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60336/0 (socket says 172.21.15.133:60336) 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 -- 172.21.15.133:0/3911596270 learned_addr learned my addr 172.21.15.133:0/3911596270 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 -- 172.21.15.133:0/3911596270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74dd6240 msgr2=0x559f74df07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50864d9640 1 --2- 172.21.15.133:0/3911596270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74dd6240 0x559f74df07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 --2- 172.21.15.133:0/3911596270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74dd6240 0x559f74df07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f5085cd8640 1 --2- 172.21.15.133:0/3911596270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74d8bf30 0x559f74de2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 -- 172.21.15.133:0/3911596270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74d8bf30 msgr2=0x559f74de2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 --2- 172.21.15.133:0/3911596270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74d8bf30 0x559f74de2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 -- 172.21.15.133:0/3911596270 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f74dd6070 con 0x559f74dd5920 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 --2- 172.21.15.133:0/3911596270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74de3220 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f50700074f0 tx=0x7f5070005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9132132298d525 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f5062ffd640 1 -- 172.21.15.133:0/3911596270 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507001a070 con 0x559f74dd5920 2024-04-25T10:19:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f5062ffd640 1 -- 172.21.15.133:0/3911596270 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5070001aa0 con 0x559f74dd5920 2024-04-25T10:19:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f5062ffd640 1 -- 172.21.15.133:0/3911596270 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50700133c0 con 0x559f74dd5920 2024-04-25T10:19:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- 172.21.15.133:0/3911596270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 msgr2=0x559f74de3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 --2- 172.21.15.133:0/3911596270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74de3220 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7f50700074f0 tx=0x7f5070005bd0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- 172.21.15.133:0/3911596270 shutdown_connections 2024-04-25T10:19:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 --2- 172.21.15.133:0/3911596270 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74dd6240 0x559f74df07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 --2- 172.21.15.133:0/3911596270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74de3220 unknown :-1 s=CLOSED pgs=2988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 --2- 172.21.15.133:0/3911596270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74d8bf30 0x559f74de2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- 172.21.15.133:0/3911596270 >> 172.21.15.133:0/3911596270 conn(0x559f74d922a0 msgr2=0x559f74d8f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- 172.21.15.133:0/3911596270 shutdown_connections 2024-04-25T10:19:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- 172.21.15.133:0/3911596270 wait complete. 2024-04-25T10:19:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- start start 2024-04-25T10:19:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 Processor -- start 2024-04-25T10:19:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74d8bf30 0x559f74e1d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74e1dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74dd6240 0x559f74e1e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f74bdf3d0 con 0x559f74dd5920 2024-04-25T10:19:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f74c96d50 con 0x559f74d8bf30 2024-04-25T10:19:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f74b75980 con 0x559f74dd6240 2024-04-25T10:19:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 10 client.0 _ll_register_callbacks cb 0x7f507c005050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:03.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74e1dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74e1dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60352/0 (socket says 172.21.15.133:60352) 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 -- 172.21.15.133:0/1673332075 learned_addr learned my addr 172.21.15.133:0/1673332075 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f5085cd8640 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74d8bf30 0x559f74e1d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50864d9640 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74dd6240 0x559f74e1e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 -- 172.21.15.133:0/1673332075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74dd6240 msgr2=0x559f74e1e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74dd6240 0x559f74e1e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 -- 172.21.15.133:0/1673332075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74d8bf30 msgr2=0x559f74e1d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74d8bf30 0x559f74e1d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 -- 172.21.15.133:0/1673332075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f74c5c600 con 0x559f74dd5920 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f5085cd8640 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74d8bf30 0x559f74e1d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f50854d7640 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74e1dd00 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7f5070007120 tx=0x7f5070000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293d8da06b44bf7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f5060ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f5060ff9640 1 -- 172.21.15.133:0/1673332075 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070025020 con 0x559f74dd5920 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f5060ff9640 1 -- 172.21.15.133:0/1673332075 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5070015c00 con 0x559f74dd5920 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- 172.21.15.133:0/1673332075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5070001aa0 con 0x559f74dd5920 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f5060ff9640 1 -- 172.21.15.133:0/1673332075 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) v2 ==== 24+0+0 (secure 0 0 0) 0x7f507002f050 con 0x559f74dd5920 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 10 client.12333 fetch_fsmap learned FSMap version 124 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.217+0000 7f508b0a9000 1 -- 172.21.15.133:0/1673332075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) v3 -- 0x559f74b6f9b0 con 0x559f74dd5920 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5060ff9640 1 -- 172.21.15.133:0/1673332075 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5070001aa0 con 0x559f74dd5920 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5060ff9640 1 -- 172.21.15.133:0/1673332075 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 124) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5070032020 con 0x559f74dd5920 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 10 client.12333 fetch_fsmap finished waiting for FSMap version 124 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 10 client.12333 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 20 client.12333 populate_metadata read hostname 'smithi133' 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x559f74e1e780 0x559f74e1eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 4 client.12333 mds_command: new command op to 12135 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 -- 172.21.15.133:0/1673332075 --> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559f74be82e0 con 0x559f74e1e780 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5085cd8640 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x559f74e1e780 0x559f74e1eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5085cd8640 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x559f74e1e780 0x559f74e1eb60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e36d216315a50b09 server_cookie=23d2ec1f49ccc601 in_seq=0 out_seq=0 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5060ff9640 10 client.12333 ms_handle_connect on v2:172.21.15.133:6836/2510709242 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5060ff9640 1 -- 172.21.15.133:0/1673332075 <== mds.0 v2:172.21.15.133:6836/2510709242 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x559f74be82e0 con 0x559f74e1e780 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5060ff9640 10 client.12333 handle_command_reply: tid=0 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 client.12333 shutdown 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 -- 172.21.15.133:0/1673332075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 msgr2=0x559f74e1dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74e1dd00 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7f5070007120 tx=0x7f5070000ee0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 -- 172.21.15.133:0/1673332075 shutdown_connections 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.133:6836/2510709242,v1:172.21.15.133:6837/2510709242] conn(0x559f74e1e780 0x559f74e1eb60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x559f74dd6240 0x559f74e1e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5060ff9640 0 client.12333 ms_handle_reset on v2:172.21.15.133:6836/2510709242 2024-04-25T10:19:03.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f74dd5920 0x559f74e1dd00 unknown :-1 s=CLOSED pgs=2989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 --2- 172.21.15.133:0/1673332075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559f74d8bf30 0x559f74e1d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 -- 172.21.15.133:0/1673332075 >> 172.21.15.133:0/1673332075 conn(0x559f74d922a0 msgr2=0x559f74de1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 -- 172.21.15.133:0/1673332075 shutdown_connections 2024-04-25T10:19:03.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 1 -- 172.21.15.133:0/1673332075 wait complete. 2024-04-25T10:19:03.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f508b0a9000 20 client.12333 trim_cache size 0 max 16384 2024-04-25T10:19:03.231 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:19:03.232 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "599cdb88-206d-4299-bbcd-9f7da90fb181", 2024-04-25T10:19:03.232 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-04-25T10:19:03.232 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 12135, 2024-04-25T10:19:03.232 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-04-25T10:19:03.232 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-04-25T10:19:03.232 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-04-25T10:19:03.233 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 66.461100486999996, 2024-04-25T10:19:03.233 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 124, 2024-04-25T10:19:03.233 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 198, 2024-04-25T10:19:03.233 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 196, 2024-04-25T10:19:03.233 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 67.424004343999997 2024-04-25T10:19:03.233 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:19:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 -- 172.21.15.133:0/67843798 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5064066f60 msgr2=0x7f5064069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5064066f60 0x7f5064069420 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f50800afe90 tx=0x7f507c009040 comp rx=0 tx=0).stop 2024-04-25T10:19:03.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 -- 172.21.15.133:0/67843798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f508015e100 msgr2=0x7f5080131880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f508015e100 0x7f5080131880 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 crypto rx=0x7f50780b0040 tx=0x7f5078019640 comp rx=0 tx=0).stop 2024-04-25T10:19:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 -- 172.21.15.133:0/67843798 shutdown_connections 2024-04-25T10:19:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f5064066f60 0x7f5064069420 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f508015e100 0x7f5080131880 unknown :-1 s=CLOSED pgs=2985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5080143560 0x7f508015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 --2- 172.21.15.133:0/67843798 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f5080142920 0x7f508015cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 -- 172.21.15.133:0/67843798 >> 172.21.15.133:0/67843798 conn(0x7f5080066f70 msgr2=0x7f508013b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 -- 172.21.15.133:0/67843798 shutdown_connections 2024-04-25T10:19:03.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.221+0000 7f5087f3c640 1 -- 172.21.15.133:0/67843798 wait complete. 2024-04-25T10:19:03.239 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:19:03.239 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:19:03.290 DEBUG:teuthology.orchestra.run.smithi162:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:19:03.324 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:19:03.325 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:19:03.325 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:03.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:19:03.331 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:19:03.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:03.332 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:19:03.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:19:03.377 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:19:03.377 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:19:03.378 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:03.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:03.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:19:03.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:03.384 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:19:03.430 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:19:03.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:19:03.431 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:19:03.431 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:03.436 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:19:03.436 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:19:03.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:03.436 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:19:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:19:03.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:19:03.481 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:19:03.481 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:03.487 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:03.487 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:19:03.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:03.487 DEBUG:teuthology.orchestra.run.smithi162:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:19:03.534 INFO:teuthology.orchestra.run.smithi162.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:19:03.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:19:03.536 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:03.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.657+0000 7ffb5fc9e640 1 Processor -- start 2024-04-25T10:19:03.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.657+0000 7ffb5fc9e640 1 -- start start 2024-04-25T10:19:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5fc9e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb580ce330 0x7ffb580ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5fc9e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb580c61d0 con 0x7ffb580b1f70 2024-04-25T10:19:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5fc9e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb580c64b0 con 0x7ffb5815c9a0 2024-04-25T10:19:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5fc9e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb580cecc0 con 0x7ffb580ce330 2024-04-25T10:19:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5e23b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb580ce330 0x7ffb580ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5e23b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb580ce330 0x7ffb580ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60362/0 (socket says 172.21.15.133:60362) 2024-04-25T10:19:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5e23b640 1 -- 172.21.15.133:0/846676883 learned_addr learned my addr 172.21.15.133:0/846676883 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5d239640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ffb580b1f70 0x7ffb5806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34346/0 (socket says 172.21.15.133:34346) 2024-04-25T10:19:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5e23b640 1 -- 172.21.15.133:0/846676883 >> v1:172.21.15.162:6789/0 conn(0x7ffb5815c9a0 legacy=0x7ffb580c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5e23b640 1 -- 172.21.15.133:0/846676883 >> v1:172.21.15.133:6789/0 conn(0x7ffb580b1f70 legacy=0x7ffb5806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5e23b640 1 -- 172.21.15.133:0/846676883 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5806d4f0 con 0x7ffb580ce330 2024-04-25T10:19:03.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5ca38640 1 -- 172.21.15.133:0/846676883 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87152243 0 0) 0x7ffb580c61d0 con 0x7ffb580b1f70 2024-04-25T10:19:03.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5e23b640 1 --2- 172.21.15.133:0/846676883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb580ce330 0x7ffb580ce780 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009e50 tx=0x7ffb5401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb47d2c923cb3bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5ca38640 1 -- 172.21.15.133:0/846676883 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb540a57d0 con 0x7ffb580ce330 2024-04-25T10:19:03.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5ca38640 1 -- 172.21.15.133:0/846676883 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb540a5970 con 0x7ffb580ce330 2024-04-25T10:19:03.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.661+0000 7ffb5ca38640 1 -- 172.21.15.133:0/846676883 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb540a5d40 con 0x7ffb580ce330 2024-04-25T10:19:03.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/846676883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb580ce330 msgr2=0x7ffb580ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 --2- 172.21.15.133:0/846676883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb580ce330 0x7ffb580ce780 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009e50 tx=0x7ffb5401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/846676883 shutdown_connections 2024-04-25T10:19:03.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 --2- 172.21.15.133:0/846676883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb580ce330 0x7ffb580ce780 unknown :-1 s=CLOSED pgs=2991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/846676883 >> 172.21.15.133:0/846676883 conn(0x7ffb58067000 msgr2=0x7ffb5806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/846676883 shutdown_connections 2024-04-25T10:19:03.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/846676883 wait complete. 2024-04-25T10:19:03.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 Processor -- start 2024-04-25T10:19:03.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 -- start start 2024-04-25T10:19:03.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb580b1f70 0x7ffb58152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb580ce330 0x7ffb58150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5815c9a0 0x7ffb58154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb581695d0 con 0x7ffb580ce330 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5815dc20 con 0x7ffb5815c9a0 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58169750 con 0x7ffb580b1f70 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5d239640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb580ce330 0x7ffb58150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5d239640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb580ce330 0x7ffb58150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60374/0 (socket says 172.21.15.133:60374) 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5d239640 1 -- 172.21.15.133:0/3093814244 learned_addr learned my addr 172.21.15.133:0/3093814244 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5e23b640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5815c9a0 0x7ffb58154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5d239640 1 -- 172.21.15.133:0/3093814244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb580b1f70 msgr2=0x7ffb58152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5da3a640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb580b1f70 0x7ffb58152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5d239640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb580b1f70 0x7ffb58152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5d239640 1 -- 172.21.15.133:0/3093814244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5815c9a0 msgr2=0x7ffb58154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5d239640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5815c9a0 0x7ffb58154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5d239640 1 -- 172.21.15.133:0/3093814244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58156270 con 0x7ffb580ce330 2024-04-25T10:19:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5e23b640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5815c9a0 0x7ffb58154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5d239640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb580ce330 0x7ffb58150370 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c4b0 tx=0x7ffb4800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb4e7fc640 1 -- 172.21.15.133:0/3093814244 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48018070 con 0x7ffb580ce330 2024-04-25T10:19:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb4e7fc640 1 -- 172.21.15.133:0/3093814244 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb48007500 con 0x7ffb580ce330 2024-04-25T10:19:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/3093814244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb58156550 con 0x7ffb580ce330 2024-04-25T10:19:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb4e7fc640 1 -- 172.21.15.133:0/3093814244 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48004a30 con 0x7ffb580ce330 2024-04-25T10:19:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.665+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/3093814244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb58154250 con 0x7ffb580ce330 2024-04-25T10:19:03.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.669+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/3093814244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb58068ef0 con 0x7ffb580ce330 2024-04-25T10:19:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.669+0000 7ffb4e7fc640 1 -- 172.21.15.133:0/3093814244 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb4801b070 con 0x7ffb580ce330 2024-04-25T10:19:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.669+0000 7ffb4e7fc640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb3c066f60 0x7ffb3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.669+0000 7ffb5da3a640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb3c066f60 0x7ffb3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.669+0000 7ffb4e7fc640 1 -- 172.21.15.133:0/3093814244 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ffb48084530 con 0x7ffb580ce330 2024-04-25T10:19:03.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.669+0000 7ffb5da3a640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb3c066f60 0x7ffb3c069420 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ffb50003d60 tx=0x7ffb50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.673+0000 7ffb4e7fc640 1 -- 172.21.15.133:0/3093814244 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb48055380 con 0x7ffb580ce330 2024-04-25T10:19:03.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.865+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/3093814244 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb5806b480 con 0x7ffb580ce330 2024-04-25T10:19:03.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.865+0000 7ffb4e7fc640 1 -- 172.21.15.133:0/3093814244 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4963 (secure 0 0 0) 0x7ffb48054c80 con 0x7ffb580ce330 2024-04-25T10:19:03.868 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:03.868 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":124,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:36.969368+0000","modified":"2024-04-25T10:17:58.776693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":196,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12135":{"gid":12135,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/2510709242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2510709242},{"type":"v1","addr":"172.21.15.133:6837","nonce":2510709242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:19:03.868 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 124 2024-04-25T10:19:03.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/3093814244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb3c066f60 msgr2=0x7ffb3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb3c066f60 0x7ffb3c069420 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ffb50003d60 tx=0x7ffb50009040 comp rx=0 tx=0).stop 2024-04-25T10:19:03.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/3093814244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb580ce330 msgr2=0x7ffb58150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb580ce330 0x7ffb58150370 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c4b0 tx=0x7ffb4800c970 comp rx=0 tx=0).stop 2024-04-25T10:19:03.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/3093814244 shutdown_connections 2024-04-25T10:19:03.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb5815c9a0 0x7ffb58154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffb3c066f60 0x7ffb3c069420 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb580ce330 0x7ffb58150370 unknown :-1 s=CLOSED pgs=2992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 --2- 172.21.15.133:0/3093814244 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffb580b1f70 0x7ffb58152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/3093814244 >> 172.21.15.133:0/3093814244 conn(0x7ffb58067000 msgr2=0x7ffb580b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/3093814244 shutdown_connections 2024-04-25T10:19:03.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.869+0000 7ffb5fc9e640 1 -- 172.21.15.133:0/3093814244 wait complete. 2024-04-25T10:19:03.887 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:03.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.993+0000 7fa14a5d1640 1 Processor -- start 2024-04-25T10:19:03.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.993+0000 7fa14a5d1640 1 -- start start 2024-04-25T10:19:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa14a5d1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa14415a1a0 0x7fa14415db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa14a5d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa14415e050 0x7fa14406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa14a5d1640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1440ce880 con 0x7fa144159dc0 2024-04-25T10:19:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa14a5d1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14415e480 con 0x7fa14415a1a0 2024-04-25T10:19:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa14a5d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1440c49e0 con 0x7fa14415e050 2024-04-25T10:19:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa1437fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa14415a1a0 0x7fa14415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa1437fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa14415a1a0 0x7fa14415db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60386/0 (socket says 172.21.15.133:60386) 2024-04-25T10:19:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa1437fe640 1 -- 172.21.15.133:0/3585340004 learned_addr learned my addr 172.21.15.133:0/3585340004 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa148b6e640 1 --2- 172.21.15.133:0/3585340004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa14415e050 0x7fa14406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa1437fe640 1 -- 172.21.15.133:0/3585340004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa14415e050 msgr2=0x7fa14406d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa1437fe640 1 --2- 172.21.15.133:0/3585340004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa14415e050 0x7fa14406d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa1437fe640 1 -- 172.21.15.133:0/3585340004 >> v1:172.21.15.162:6789/0 conn(0x7fa144159dc0 legacy=0x7fa1440cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa1437fe640 1 -- 172.21.15.133:0/3585340004 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1440ceb20 con 0x7fa14415a1a0 2024-04-25T10:19:04.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa1437fe640 1 --2- 172.21.15.133:0/3585340004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14415a1a0 0x7fa14415db10 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7fa14406f6e0 tx=0x7fa13801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41acca07675544fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa142ffd640 1 -- 172.21.15.133:0/3585340004 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1380b0070 con 0x7fa14415a1a0 2024-04-25T10:19:04.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa142ffd640 1 -- 172.21.15.133:0/3585340004 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1380ab600 con 0x7fa14415a1a0 2024-04-25T10:19:04.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:03.997+0000 7fa142ffd640 1 -- 172.21.15.133:0/3585340004 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1380ab9d0 con 0x7fa14415a1a0 2024-04-25T10:19:04.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 -- 172.21.15.133:0/3585340004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14415a1a0 msgr2=0x7fa14415db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 --2- 172.21.15.133:0/3585340004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14415a1a0 0x7fa14415db10 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7fa14406f6e0 tx=0x7fa13801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 -- 172.21.15.133:0/3585340004 shutdown_connections 2024-04-25T10:19:04.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 --2- 172.21.15.133:0/3585340004 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa14415e050 0x7fa14406d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 --2- 172.21.15.133:0/3585340004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14415a1a0 0x7fa14415db10 unknown :-1 s=CLOSED pgs=2993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 -- 172.21.15.133:0/3585340004 >> 172.21.15.133:0/3585340004 conn(0x7fa144067000 msgr2=0x7fa14406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:04.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 -- 172.21.15.133:0/3585340004 shutdown_connections 2024-04-25T10:19:04.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 -- 172.21.15.133:0/3585340004 wait complete. 2024-04-25T10:19:04.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 Processor -- start 2024-04-25T10:19:04.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 -- start start 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa144159dc0 0x7fa144143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa14415e050 0x7fa14415aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14414f510 0x7fa144151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14406e220 con 0x7fa14414f510 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14406e0a0 con 0x7fa144159dc0 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14406e3a0 con 0x7fa14415e050 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa148b6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14414f510 0x7fa144151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa148b6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14414f510 0x7fa144151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60392/0 (socket says 172.21.15.133:60392) 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa148b6e640 1 -- 172.21.15.133:0/741114857 learned_addr learned my addr 172.21.15.133:0/741114857 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa143fff640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa144159dc0 0x7fa144143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa1437fe640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa14415e050 0x7fa14415aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa148b6e640 1 -- 172.21.15.133:0/741114857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa14415e050 msgr2=0x7fa14415aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa148b6e640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa14415e050 0x7fa14415aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa148b6e640 1 -- 172.21.15.133:0/741114857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa144159dc0 msgr2=0x7fa144143ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa148b6e640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa144159dc0 0x7fa144143ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa148b6e640 1 -- 172.21.15.133:0/741114857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1441524b0 con 0x7fa14414f510 2024-04-25T10:19:04.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa148b6e640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14414f510 0x7fa144151b40 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7fa13c007e60 tx=0x7fa13c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa140ff9640 1 -- 172.21.15.133:0/741114857 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c014420 con 0x7fa14414f510 2024-04-25T10:19:04.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa140ff9640 1 -- 172.21.15.133:0/741114857 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c019070 con 0x7fa14414f510 2024-04-25T10:19:04.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa140ff9640 1 -- 172.21.15.133:0/741114857 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c014820 con 0x7fa14414f510 2024-04-25T10:19:04.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 -- 172.21.15.133:0/741114857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa144152790 con 0x7fa14414f510 2024-04-25T10:19:04.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.001+0000 7fa14a5d1640 1 -- 172.21.15.133:0/741114857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14415a9a0 con 0x7fa14414f510 2024-04-25T10:19:04.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.005+0000 7fa14a5d1640 1 -- 172.21.15.133:0/741114857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa144068ef0 con 0x7fa14414f510 2024-04-25T10:19:04.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.005+0000 7fa140ff9640 1 -- 172.21.15.133:0/741114857 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa13c0149c0 con 0x7fa14414f510 2024-04-25T10:19:04.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.005+0000 7fa140ff9640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa130066f60 0x7fa130069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.005+0000 7fa140ff9640 1 -- 172.21.15.133:0/741114857 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa13c085900 con 0x7fa14414f510 2024-04-25T10:19:04.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.005+0000 7fa143fff640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa130066f60 0x7fa130069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.005+0000 7fa143fff640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa130066f60 0x7fa130069420 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa144164e70 tx=0x7fa134007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.009+0000 7fa140ff9640 1 -- 172.21.15.133:0/741114857 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa13c056100 con 0x7fa14414f510 2024-04-25T10:19:04.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.153+0000 7fa14a5d1640 1 -- 172.21.15.133:0/741114857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1440cc970 con 0x7fa14414f510 2024-04-25T10:19:04.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa140ff9640 1 -- 172.21.15.133:0/741114857 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17234 (secure 0 0 0) 0x7fa13c05a1a0 con 0x7fa14414f510 2024-04-25T10:19:04.162 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:04.163 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":198,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:18:36.089833+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:34.663983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:35.974809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":6,"to":0}]},{"pgid":"29.c","mappings":[{"from":6,"to":0}]},{"pgid":"29.d","mappings":[{"from":6,"to":0}]},{"pgid":"29.14","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/1916780253":"2024-04-26T10:17:55.746978+0000","172.21.15.133:6836/1916780253":"2024-04-26T10:17:55.746978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:04.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 -- 172.21.15.133:0/741114857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa130066f60 msgr2=0x7fa130069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa130066f60 0x7fa130069420 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa144164e70 tx=0x7fa134007e40 comp rx=0 tx=0).stop 2024-04-25T10:19:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 -- 172.21.15.133:0/741114857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14414f510 msgr2=0x7fa144151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14414f510 0x7fa144151b40 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7fa13c007e60 tx=0x7fa13c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 -- 172.21.15.133:0/741114857 shutdown_connections 2024-04-25T10:19:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fa130066f60 0x7fa130069420 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa14414f510 0x7fa144151b40 unknown :-1 s=CLOSED pgs=2994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fa14415e050 0x7fa14415aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 --2- 172.21.15.133:0/741114857 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa144159dc0 0x7fa144143ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 -- 172.21.15.133:0/741114857 >> 172.21.15.133:0/741114857 conn(0x7fa144067000 msgr2=0x7fa144073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 -- 172.21.15.133:0/741114857 shutdown_connections 2024-04-25T10:19:04.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.161+0000 7fa14a5d1640 1 -- 172.21.15.133:0/741114857 wait complete. 2024-04-25T10:19:04.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:39.778509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:34.663983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '194', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:35.974809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:19:04.181 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:19:04.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ef5c06640 1 Processor -- start 2024-04-25T10:19:04.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ef5c06640 1 -- start start 2024-04-25T10:19:04.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ef5c06640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef0142920 0x7f1ef0143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ef5c06640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef0140ea0 con 0x7f1ef0140a40 2024-04-25T10:19:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ef5c06640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef00cf280 con 0x7f1ef015d9f0 2024-04-25T10:19:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ef5c06640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef006d440 con 0x7f1ef0142920 2024-04-25T10:19:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ee6ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1ef0140a40 0x7f1ef00ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34356/0 (socket says 172.21.15.133:34356) 2024-04-25T10:19:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ee6ffd640 1 -- 172.21.15.133:0/3365067354 learned_addr learned my addr 172.21.15.133:0/3365067354 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474926536 0 0) 0x7f1ef0140ea0 con 0x7f1ef0140a40 2024-04-25T10:19:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eef7fe640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef0142920 0x7f1ef0143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1edc003450 con 0x7f1ef0140a40 2024-04-25T10:19:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2845422270 0 0) 0x7f1edc003450 con 0x7f1ef0140a40 2024-04-25T10:19:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef0142920 msgr2=0x7f1ef0143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef0142920 0x7f1ef0143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 >> v1:172.21.15.162:6789/0 conn(0x7f1ef015d9f0 legacy=0x7f1ef006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef006d720 con 0x7f1ef0140a40 2024-04-25T10:19:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1ed8001680 con 0x7f1ef0140a40 2024-04-25T10:19:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 >> v1:172.21.15.133:6789/0 conn(0x7f1ef0140a40 legacy=0x7f1ef00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef0142920 0x7f1edc003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 0x7f1edc00a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1edc00b000 0x7f1edc00d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef006d440 con 0x7f1edc004060 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc00dd70 con 0x7f1edc00b000 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc00df70 con 0x7f1ef0142920 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ee6ffd640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 0x7f1edc00a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eef7fe640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef0142920 0x7f1edc003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ee6ffd640 1 -- 172.21.15.133:0/3365067354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef0142920 msgr2=0x7f1edc003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ee6ffd640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef0142920 0x7f1edc003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ee6ffd640 1 -- 172.21.15.133:0/3365067354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1edc00b000 msgr2=0x7f1edc00d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1eeffff640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1edc00b000 0x7f1edc00d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ee6ffd640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1edc00b000 0x7f1edc00d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.297+0000 7f1ee6ffd640 1 -- 172.21.15.133:0/3365067354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc003450 con 0x7f1edc004060 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ee6ffd640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 0x7f1edc00a870 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800c210 tx=0x7f1ed801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7b39730806d2f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8007d40 con 0x7f1edc004060 2024-04-25T10:19:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed80a9070 con 0x7f1edc004060 2024-04-25T10:19:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eeeffd640 1 -- 172.21.15.133:0/3365067354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8005d80 con 0x7f1edc004060 2024-04-25T10:19:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3365067354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 msgr2=0x7f1edc00a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 --2- 172.21.15.133:0/3365067354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 0x7f1edc00a870 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800c210 tx=0x7f1ed801cca0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eeffff640 1 -- 172.21.15.133:0/3365067354 reap_dead start 2024-04-25T10:19:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3365067354 shutdown_connections 2024-04-25T10:19:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3365067354 >> 172.21.15.133:0/3365067354 conn(0x7f1ef0066f70 msgr2=0x7f1ef006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3365067354 shutdown_connections 2024-04-25T10:19:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3365067354 wait complete. 2024-04-25T10:19:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 Processor -- start 2024-04-25T10:19:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 -- start start 2024-04-25T10:19:04.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 0x7f1ef00caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef01f6010 0x7f1ef012fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef0130200 0x7f1ef01f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef006e7b0 con 0x7f1edc004060 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef006e630 con 0x7f1ef0130200 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ef5c06640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef006e930 con 0x7f1ef01f6010 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 0x7f1ef00caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 0x7f1ef00caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60402/0 (socket says 172.21.15.133:60402) 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eef7fe640 1 -- 172.21.15.133:0/3684968754 learned_addr learned my addr 172.21.15.133:0/3684968754 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eeffff640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef0130200 0x7f1ef01f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1ee6ffd640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef01f6010 0x7f1ef012fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eef7fe640 1 -- 172.21.15.133:0/3684968754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef01f6010 msgr2=0x7f1ef012fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eef7fe640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef01f6010 0x7f1ef012fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eef7fe640 1 -- 172.21.15.133:0/3684968754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef0130200 msgr2=0x7f1ef01f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eef7fe640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef0130200 0x7f1ef01f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.301+0000 7f1eef7fe640 1 -- 172.21.15.133:0/3684968754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00a9520 con 0x7f1edc004060 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1eef7fe640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 0x7f1ef00caf80 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00115e0 tx=0x7f1ee0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1eecff9640 1 -- 172.21.15.133:0/3684968754 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000f8f0 con 0x7f1edc004060 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1eecff9640 1 -- 172.21.15.133:0/3684968754 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee000fa90 con 0x7f1edc004060 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3684968754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef00a9800 con 0x7f1edc004060 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1eecff9640 1 -- 172.21.15.133:0/3684968754 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000fe40 con 0x7f1edc004060 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3684968754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef00a9dc0 con 0x7f1edc004060 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3684968754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef0068e80 con 0x7f1edc004060 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1eecff9640 1 -- 172.21.15.133:0/3684968754 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ee0015460 con 0x7f1edc004060 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1eecff9640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ed0066f60 0x7f1ed0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1eecff9640 1 -- 172.21.15.133:0/3684968754 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1ee0085c40 con 0x7f1edc004060 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1ee6ffd640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ed0066f60 0x7f1ed0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.305+0000 7f1ee6ffd640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ed0066f60 0x7f1ed0069420 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80ab000 tx=0x7f1ed80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.309+0000 7f1eecff9640 1 -- 172.21.15.133:0/3684968754 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ee0019030 con 0x7f1edc004060 2024-04-25T10:19:04.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3684968754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ef00afa70 con 0x7f1edc004060 2024-04-25T10:19:04.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1eecff9640 1 -- 172.21.15.133:0/3684968754 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v198) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ee0056a00 con 0x7f1edc004060 2024-04-25T10:19:04.467 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-04-25T10:19:04.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3684968754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ed0066f60 msgr2=0x7f1ed0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ed0066f60 0x7f1ed0069420 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80ab000 tx=0x7f1ed80a7070 comp rx=0 tx=0).stop 2024-04-25T10:19:04.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3684968754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 msgr2=0x7f1ef00caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 0x7f1ef00caf80 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00115e0 tx=0x7f1ee0011aa0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3684968754 shutdown_connections 2024-04-25T10:19:04.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ef0130200 0x7f1ef01f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1ef01f6010 0x7f1ef012fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ed0066f60 0x7f1ed0069420 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 --2- 172.21.15.133:0/3684968754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1edc004060 0x7f1ef00caf80 unknown :-1 s=CLOSED pgs=2997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3684968754 >> 172.21.15.133:0/3684968754 conn(0x7f1ef0066f70 msgr2=0x7f1ef00ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:04.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3684968754 shutdown_connections 2024-04-25T10:19:04.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.465+0000 7f1ef5c06640 1 -- 172.21.15.133:0/3684968754 wait complete. 2024-04-25T10:19:04.484 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:19:04.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.593+0000 7ffa65ea4640 1 Processor -- start 2024-04-25T10:19:04.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.593+0000 7ffa65ea4640 1 -- start start 2024-04-25T10:19:04.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa65ea4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffa600cd820 0x7ffa600cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa65ea4640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ffa6015ced0 0x7ffa6015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa65ea4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa600ce2e0 con 0x7ffa60145170 2024-04-25T10:19:04.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa65ea4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6006db00 con 0x7ffa600cd820 2024-04-25T10:19:04.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa65ea4640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa60145550 con 0x7ffa6015ced0 2024-04-25T10:19:04.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffa600cd820 0x7ffa600cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ffa60145170 0x7ffa6006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34372/0 (socket says 172.21.15.133:34372) 2024-04-25T10:19:04.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5f7fe640 1 -- 172.21.15.133:0/1398581655 learned_addr learned my addr 172.21.15.133:0/1398581655 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:04.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5ffff640 1 --2- 172.21.15.133:0/1398581655 >> v2:172.21.15.162:3300/0 conn(0x7ffa6015ced0 0x7ffa6015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5effd640 1 -- 172.21.15.133:0/1398581655 >> v2:172.21.15.162:3300/0 conn(0x7ffa6015ced0 msgr2=0x7ffa6015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5effd640 1 --2- 172.21.15.133:0/1398581655 >> v2:172.21.15.162:3300/0 conn(0x7ffa6015ced0 0x7ffa6015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5effd640 1 -- 172.21.15.133:0/1398581655 >> v1:172.21.15.133:6789/0 conn(0x7ffa60145170 legacy=0x7ffa6006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5effd640 1 -- 172.21.15.133:0/1398581655 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6006d4b0 con 0x7ffa600cd820 2024-04-25T10:19:04.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5effd640 1 --2- 172.21.15.133:0/1398581655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa600cd820 0x7ffa600cdca0 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7ffa600b27f0 tx=0x7ffa5401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7ff3ad56b0f6c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5e7fc640 1 -- 172.21.15.133:0/1398581655 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa540b0070 con 0x7ffa600cd820 2024-04-25T10:19:04.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5e7fc640 1 -- 172.21.15.133:0/1398581655 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa540ab600 con 0x7ffa600cd820 2024-04-25T10:19:04.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.597+0000 7ffa5e7fc640 1 -- 172.21.15.133:0/1398581655 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa540ab9d0 con 0x7ffa600cd820 2024-04-25T10:19:04.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1398581655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa600cd820 msgr2=0x7ffa600cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 --2- 172.21.15.133:0/1398581655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa600cd820 0x7ffa600cdca0 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7ffa600b27f0 tx=0x7ffa5401c480 comp rx=0 tx=0).stop 2024-04-25T10:19:04.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1398581655 shutdown_connections 2024-04-25T10:19:04.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 --2- 172.21.15.133:0/1398581655 >> v2:172.21.15.162:3300/0 conn(0x7ffa6015ced0 0x7ffa6015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 --2- 172.21.15.133:0/1398581655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa600cd820 0x7ffa600cdca0 unknown :-1 s=CLOSED pgs=2999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1398581655 >> 172.21.15.133:0/1398581655 conn(0x7ffa60066f70 msgr2=0x7ffa6006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:04.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1398581655 shutdown_connections 2024-04-25T10:19:04.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1398581655 wait complete. 2024-04-25T10:19:04.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 Processor -- start 2024-04-25T10:19:04.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 -- start start 2024-04-25T10:19:04.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa600cd820 0x7ffa601522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa60145170 0x7ffa60165490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa6015ced0 0x7ffa601429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa60168610 con 0x7ffa600cd820 2024-04-25T10:19:04.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa60168490 con 0x7ffa60145170 2024-04-25T10:19:04.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa60168790 con 0x7ffa6015ced0 2024-04-25T10:19:04.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa600cd820 0x7ffa601522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa600cd820 0x7ffa601522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60422/0 (socket says 172.21.15.133:60422) 2024-04-25T10:19:04.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5f7fe640 1 -- 172.21.15.133:0/1092437541 learned_addr learned my addr 172.21.15.133:0/1092437541 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5effd640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa60145170 0x7ffa60165490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5f7fe640 1 -- 172.21.15.133:0/1092437541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa6015ced0 msgr2=0x7ffa601429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5ffff640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa6015ced0 0x7ffa601429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5f7fe640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa6015ced0 0x7ffa601429b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5f7fe640 1 -- 172.21.15.133:0/1092437541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa60145170 msgr2=0x7ffa60165490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5f7fe640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa60145170 0x7ffa60165490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5f7fe640 1 -- 172.21.15.133:0/1092437541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6015e2f0 con 0x7ffa600cd820 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5effd640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa60145170 0x7ffa60165490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa5f7fe640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa600cd820 0x7ffa601522d0 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000c450 tx=0x7ffa5000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa47fff640 1 -- 172.21.15.133:0/1092437541 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50017070 con 0x7ffa600cd820 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa47fff640 1 -- 172.21.15.133:0/1092437541 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa50005cf0 con 0x7ffa600cd820 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1092437541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6013e770 con 0x7ffa600cd820 2024-04-25T10:19:04.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa47fff640 1 -- 172.21.15.133:0/1092437541 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa500072f0 con 0x7ffa600cd820 2024-04-25T10:19:04.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.601+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1092437541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa60165c70 con 0x7ffa600cd820 2024-04-25T10:19:04.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.605+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1092437541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa60068e80 con 0x7ffa600cd820 2024-04-25T10:19:04.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.605+0000 7ffa47fff640 1 -- 172.21.15.133:0/1092437541 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa5001a070 con 0x7ffa600cd820 2024-04-25T10:19:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.605+0000 7ffa47fff640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffa4c066f60 0x7ffa4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.605+0000 7ffa47fff640 1 -- 172.21.15.133:0/1092437541 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ffa50084650 con 0x7ffa600cd820 2024-04-25T10:19:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.605+0000 7ffa5effd640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffa4c066f60 0x7ffa4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.605+0000 7ffa5effd640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffa4c066f60 0x7ffa4c069420 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ffa540ad000 tx=0x7ffa54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.609+0000 7ffa47fff640 1 -- 172.21.15.133:0/1092437541 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa50055410 con 0x7ffa600cd820 2024-04-25T10:19:04.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.765+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1092437541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa600b3180 con 0x7ffa600cd820 2024-04-25T10:19:04.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.765+0000 7ffa47fff640 1 -- 172.21.15.133:0/1092437541 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v198) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa50054d10 con 0x7ffa600cd820 2024-04-25T10:19:04.768 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-04-25T10:19:04.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1092437541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffa4c066f60 msgr2=0x7ffa4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffa4c066f60 0x7ffa4c069420 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ffa540ad000 tx=0x7ffa54008040 comp rx=0 tx=0).stop 2024-04-25T10:19:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1092437541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa600cd820 msgr2=0x7ffa601522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa600cd820 0x7ffa601522d0 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000c450 tx=0x7ffa5000c910 comp rx=0 tx=0).stop 2024-04-25T10:19:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1092437541 shutdown_connections 2024-04-25T10:19:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffa6015ced0 0x7ffa601429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa60145170 0x7ffa60165490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffa4c066f60 0x7ffa4c069420 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 --2- 172.21.15.133:0/1092437541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa600cd820 0x7ffa601522d0 unknown :-1 s=CLOSED pgs=3000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1092437541 >> 172.21.15.133:0/1092437541 conn(0x7ffa60066f70 msgr2=0x7ffa6015f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1092437541 shutdown_connections 2024-04-25T10:19:04.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.769+0000 7ffa65ea4640 1 -- 172.21.15.133:0/1092437541 wait complete. 2024-04-25T10:19:04.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:19:04.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.901+0000 7fdc1a434640 1 Processor -- start 2024-04-25T10:19:04.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.901+0000 7fdc1a434640 1 -- start start 2024-04-25T10:19:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.901+0000 7fdc1a434640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc1406eac0 0x7fdc141653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.901+0000 7fdc1a434640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1415ac50 con 0x7fdc140b2150 2024-04-25T10:19:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.901+0000 7fdc1a434640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc140cd180 con 0x7fdc14143ea0 2024-04-25T10:19:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.901+0000 7fdc1a434640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc140ccf20 con 0x7fdc1406eac0 2024-04-25T10:19:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.901+0000 7fdc189d1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc1406eac0 0x7fdc141653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc0bfff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdc140b2150 0x7fdc1406e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34388/0 (socket says 172.21.15.133:34388) 2024-04-25T10:19:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc0bfff640 1 -- 172.21.15.133:0/816604298 learned_addr learned my addr 172.21.15.133:0/816604298 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc189d1640 1 -- 172.21.15.133:0/816604298 >> v1:172.21.15.162:6789/0 conn(0x7fdc14143ea0 legacy=0x7fdc140cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc189d1640 1 -- 172.21.15.133:0/816604298 >> v1:172.21.15.133:6789/0 conn(0x7fdc140b2150 legacy=0x7fdc1406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc189d1640 1 -- 172.21.15.133:0/816604298 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1415aed0 con 0x7fdc1406eac0 2024-04-25T10:19:04.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc137fe640 1 -- 172.21.15.133:0/816604298 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323573605 0 0) 0x7fdbfc001120 con 0x7fdc140b2150 2024-04-25T10:19:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc189d1640 1 --2- 172.21.15.133:0/816604298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1406eac0 0x7fdc141653c0 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009d90 tx=0x7fdc0c01eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f41194f98c6054f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc137fe640 1 -- 172.21.15.133:0/816604298 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0b4070 con 0x7fdc1406eac0 2024-04-25T10:19:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc137fe640 1 -- 172.21.15.133:0/816604298 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c0a7a00 con 0x7fdc1406eac0 2024-04-25T10:19:04.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc137fe640 1 -- 172.21.15.133:0/816604298 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0a7dd0 con 0x7fdc1406eac0 2024-04-25T10:19:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc1a434640 1 -- 172.21.15.133:0/816604298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1406eac0 msgr2=0x7fdc141653c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc1a434640 1 --2- 172.21.15.133:0/816604298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1406eac0 0x7fdc141653c0 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009d90 tx=0x7fdc0c01eb40 comp rx=0 tx=0).stop 2024-04-25T10:19:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc1a434640 1 -- 172.21.15.133:0/816604298 shutdown_connections 2024-04-25T10:19:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc1a434640 1 --2- 172.21.15.133:0/816604298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc1406eac0 0x7fdc141653c0 unknown :-1 s=CLOSED pgs=3002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc1a434640 1 -- 172.21.15.133:0/816604298 >> 172.21.15.133:0/816604298 conn(0x7fdc14066f70 msgr2=0x7fdc1406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc1a434640 1 -- 172.21.15.133:0/816604298 shutdown_connections 2024-04-25T10:19:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.905+0000 7fdc1a434640 1 -- 172.21.15.133:0/816604298 wait complete. 2024-04-25T10:19:04.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc1a434640 1 Processor -- start 2024-04-25T10:19:04.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc1a434640 1 -- start start 2024-04-25T10:19:04.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc1a434640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc1406eac0 0x7fdc14151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc1a434640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc140b2150 0x7fdc141571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc1a434640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc14143ea0 0x7fdc14157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc1a434640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1406de80 con 0x7fdc140b2150 2024-04-25T10:19:04.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc1a434640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1406dd00 con 0x7fdc1406eac0 2024-04-25T10:19:04.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc1a434640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1406e000 con 0x7fdc14143ea0 2024-04-25T10:19:04.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc140b2150 0x7fdc141571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc140b2150 0x7fdc141571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60440/0 (socket says 172.21.15.133:60440) 2024-04-25T10:19:04.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc0bfff640 1 -- 172.21.15.133:0/4022816142 learned_addr learned my addr 172.21.15.133:0/4022816142 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:04.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc13fff640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc1406eac0 0x7fdc14151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc189d1640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc14143ea0 0x7fdc14157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc0bfff640 1 -- 172.21.15.133:0/4022816142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc14143ea0 msgr2=0x7fdc14157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc0bfff640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc14143ea0 0x7fdc14157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc0bfff640 1 -- 172.21.15.133:0/4022816142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc1406eac0 msgr2=0x7fdc14151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc0bfff640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc1406eac0 0x7fdc14151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc0bfff640 1 -- 172.21.15.133:0/4022816142 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc141541d0 con 0x7fdc140b2150 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc13fff640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc1406eac0 0x7fdc14151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc0bfff640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc140b2150 0x7fdc141571d0 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00c7b0 tx=0x7fdbfc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc117fa640 1 -- 172.21.15.133:0/4022816142 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc018070 con 0x7fdc140b2150 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc117fa640 1 -- 172.21.15.133:0/4022816142 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbfc0072f0 con 0x7fdc140b2150 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc1a434640 1 -- 172.21.15.133:0/4022816142 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc14156d80 con 0x7fdc140b2150 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc117fa640 1 -- 172.21.15.133:0/4022816142 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00ce60 con 0x7fdc140b2150 2024-04-25T10:19:04.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc1a434640 1 -- 172.21.15.133:0/4022816142 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc14152c40 con 0x7fdc140b2150 2024-04-25T10:19:04.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc1a434640 1 -- 172.21.15.133:0/4022816142 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc14068e80 con 0x7fdc140b2150 2024-04-25T10:19:04.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc117fa640 1 -- 172.21.15.133:0/4022816142 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdbfc01b070 con 0x7fdc140b2150 2024-04-25T10:19:04.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc117fa640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdbf4066f60 0x7fdbf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc13fff640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdbf4066f60 0x7fdbf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.909+0000 7fdc117fa640 1 -- 172.21.15.133:0/4022816142 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fdbfc0844e0 con 0x7fdc140b2150 2024-04-25T10:19:04.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.913+0000 7fdc13fff640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdbf4066f60 0x7fdbf4069420 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400a850 tx=0x7fdc04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:04.913+0000 7fdc117fa640 1 -- 172.21.15.133:0/4022816142 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc1415ac50 con 0x7fdc140b2150 2024-04-25T10:19:05.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.069+0000 7fdc1a434640 1 -- 172.21.15.133:0/4022816142 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc140cdf20 con 0x7fdc140b2150 2024-04-25T10:19:05.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.069+0000 7fdc117fa640 1 -- 172.21.15.133:0/4022816142 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v198) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdbfc0552a0 con 0x7fdc140b2150 2024-04-25T10:19:05.072 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-04-25T10:19:05.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 -- 172.21.15.133:0/4022816142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdbf4066f60 msgr2=0x7fdbf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdbf4066f60 0x7fdbf4069420 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400a850 tx=0x7fdc04008040 comp rx=0 tx=0).stop 2024-04-25T10:19:05.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 -- 172.21.15.133:0/4022816142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc140b2150 msgr2=0x7fdc141571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc140b2150 0x7fdc141571d0 secure :-1 s=READY pgs=3003 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00c7b0 tx=0x7fdbfc00cc70 comp rx=0 tx=0).stop 2024-04-25T10:19:05.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 -- 172.21.15.133:0/4022816142 shutdown_connections 2024-04-25T10:19:05.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fdc14143ea0 0x7fdc14157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fdbf4066f60 0x7fdbf4069420 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc140b2150 0x7fdc141571d0 unknown :-1 s=CLOSED pgs=3003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 --2- 172.21.15.133:0/4022816142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdc1406eac0 0x7fdc14151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 -- 172.21.15.133:0/4022816142 >> 172.21.15.133:0/4022816142 conn(0x7fdc14066f70 msgr2=0x7fdc1413b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 -- 172.21.15.133:0/4022816142 shutdown_connections 2024-04-25T10:19:05.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.073+0000 7fdc1a434640 1 -- 172.21.15.133:0/4022816142 wait complete. 2024-04-25T10:19:05.091 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:05.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.205+0000 7f0e52fd3640 1 Processor -- start 2024-04-25T10:19:05.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.205+0000 7f0e52fd3640 1 -- start start 2024-04-25T10:19:05.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e52fd3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e4c0ce330 0x7f0e4c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e52fd3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4c0c61d0 con 0x7f0e4c15c9a0 2024-04-25T10:19:05.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e52fd3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4c0c64b0 con 0x7f0e4c0b1f70 2024-04-25T10:19:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e52fd3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4c0cecc0 con 0x7f0e4c0ce330 2024-04-25T10:19:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e51570640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e4c0ce330 0x7f0e4c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e50d6f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0e4c15c9a0 0x7f0e4c0c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34392/0 (socket says 172.21.15.133:34392) 2024-04-25T10:19:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e50d6f640 1 -- 172.21.15.133:0/1172876979 learned_addr learned my addr 172.21.15.133:0/1172876979 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e51570640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e4c0ce330 0x7f0e4c0ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60444/0 (socket says 172.21.15.133:60444) 2024-04-25T10:19:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e51570640 1 -- 172.21.15.133:0/1172876979 >> v1:172.21.15.162:6789/0 conn(0x7f0e4c0b1f70 legacy=0x7f0e4c06dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e51570640 1 -- 172.21.15.133:0/1172876979 >> v1:172.21.15.133:6789/0 conn(0x7f0e4c15c9a0 legacy=0x7f0e4c0c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e51570640 1 -- 172.21.15.133:0/1172876979 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c06d4f0 con 0x7f0e4c0ce330 2024-04-25T10:19:05.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e4b7fe640 1 -- 172.21.15.133:0/1172876979 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406922593 0 0) 0x7f0e4c0c61d0 con 0x7f0e4c15c9a0 2024-04-25T10:19:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e51570640 1 --2- 172.21.15.133:0/1172876979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4c0ce330 0x7f0e4c0ce780 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009d90 tx=0x7f0e4401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a429e20562c69a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e4b7fe640 1 -- 172.21.15.133:0/1172876979 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e440b2070 con 0x7f0e4c0ce330 2024-04-25T10:19:05.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e4b7fe640 1 -- 172.21.15.133:0/1172876979 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e440a5900 con 0x7f0e4c0ce330 2024-04-25T10:19:05.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.209+0000 7f0e4b7fe640 1 -- 172.21.15.133:0/1172876979 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e440a5cd0 con 0x7f0e4c0ce330 2024-04-25T10:19:05.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 -- 172.21.15.133:0/1172876979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4c0ce330 msgr2=0x7f0e4c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 --2- 172.21.15.133:0/1172876979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4c0ce330 0x7f0e4c0ce780 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009d90 tx=0x7f0e4401c930 comp rx=0 tx=0).stop 2024-04-25T10:19:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 -- 172.21.15.133:0/1172876979 shutdown_connections 2024-04-25T10:19:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 --2- 172.21.15.133:0/1172876979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4c0ce330 0x7f0e4c0ce780 unknown :-1 s=CLOSED pgs=3005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 -- 172.21.15.133:0/1172876979 >> 172.21.15.133:0/1172876979 conn(0x7f0e4c067000 msgr2=0x7f0e4c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 -- 172.21.15.133:0/1172876979 shutdown_connections 2024-04-25T10:19:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 -- 172.21.15.133:0/1172876979 wait complete. 2024-04-25T10:19:05.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 Processor -- start 2024-04-25T10:19:05.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 -- start start 2024-04-25T10:19:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e4c0b1f70 0x7f0e4c0cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4c0ce330 0x7f0e4c0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e4c15c9a0 0x7f0e4c148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c1695d0 con 0x7f0e4c0ce330 2024-04-25T10:19:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c15dc20 con 0x7f0e4c0b1f70 2024-04-25T10:19:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e52fd3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c169750 con 0x7f0e4c15c9a0 2024-04-25T10:19:05.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e4bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4c0ce330 0x7f0e4c0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e4bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4c0ce330 0x7f0e4c0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60450/0 (socket says 172.21.15.133:60450) 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e4bfff640 1 -- 172.21.15.133:0/121072978 learned_addr learned my addr 172.21.15.133:0/121072978 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e50d6f640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e4c0b1f70 0x7f0e4c0cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e4bfff640 1 -- 172.21.15.133:0/121072978 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e4c15c9a0 msgr2=0x7f0e4c148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e51570640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e4c15c9a0 0x7f0e4c148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e4bfff640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e4c15c9a0 0x7f0e4c148910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e4bfff640 1 -- 172.21.15.133:0/121072978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e4c0b1f70 msgr2=0x7f0e4c0cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e4bfff640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e4c0b1f70 0x7f0e4c0cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e4bfff640 1 -- 172.21.15.133:0/121072978 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c0ca530 con 0x7f0e4c0ce330 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e50d6f640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e4c0b1f70 0x7f0e4c0cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.213+0000 7f0e4bfff640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4c0ce330 0x7f0e4c0c9c90 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000b410 tx=0x7f0e4000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.217+0000 7f0e497fa640 1 -- 172.21.15.133:0/121072978 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40015020 con 0x7f0e4c0ce330 2024-04-25T10:19:05.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.217+0000 7f0e497fa640 1 -- 172.21.15.133:0/121072978 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e40011830 con 0x7f0e4c0ce330 2024-04-25T10:19:05.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.217+0000 7f0e52fd3640 1 -- 172.21.15.133:0/121072978 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4c0ca810 con 0x7f0e4c0ce330 2024-04-25T10:19:05.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.217+0000 7f0e497fa640 1 -- 172.21.15.133:0/121072978 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40011c20 con 0x7f0e4c0ce330 2024-04-25T10:19:05.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.217+0000 7f0e52fd3640 1 -- 172.21.15.133:0/121072978 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c0ca1d0 con 0x7f0e4c0ce330 2024-04-25T10:19:05.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.217+0000 7f0e52fd3640 1 -- 172.21.15.133:0/121072978 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4c068ef0 con 0x7f0e4c0ce330 2024-04-25T10:19:05.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.217+0000 7f0e497fa640 1 -- 172.21.15.133:0/121072978 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e40018020 con 0x7f0e4c0ce330 2024-04-25T10:19:05.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.217+0000 7f0e497fa640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e38066f60 0x7f0e38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.217+0000 7f0e497fa640 1 -- 172.21.15.133:0/121072978 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f0e400851b0 con 0x7f0e4c0ce330 2024-04-25T10:19:05.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.217+0000 7f0e50d6f640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e38066f60 0x7f0e38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.217+0000 7f0e50d6f640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e38066f60 0x7f0e38069420 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c000f70 tx=0x7f0e3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.221+0000 7f0e497fa640 1 -- 172.21.15.133:0/121072978 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e4001f020 con 0x7f0e4c0ce330 2024-04-25T10:19:05.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.397+0000 7f0e52fd3640 1 -- 172.21.15.133:0/121072978 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e4c0cbb50 con 0x7f0e4c0ce330 2024-04-25T10:19:05.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e497fa640 1 -- 172.21.15.133:0/121072978 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f0e400172c0 con 0x7f0e4c0ce330 2024-04-25T10:19:05.402 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:05.402 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":124,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:36.969368+0000","modified":"2024-04-25T10:17:58.776693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":196,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12135":{"gid":12135,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/2510709242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2510709242},{"type":"v1","addr":"172.21.15.133:6837","nonce":2510709242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:19:05.402 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 124 2024-04-25T10:19:05.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 -- 172.21.15.133:0/121072978 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e38066f60 msgr2=0x7f0e38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e38066f60 0x7f0e38069420 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c000f70 tx=0x7f0e3c009040 comp rx=0 tx=0).stop 2024-04-25T10:19:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 -- 172.21.15.133:0/121072978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4c0ce330 msgr2=0x7f0e4c0c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4c0ce330 0x7f0e4c0c9c90 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000b410 tx=0x7f0e4000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 -- 172.21.15.133:0/121072978 shutdown_connections 2024-04-25T10:19:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0e4c15c9a0 0x7f0e4c148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0e38066f60 0x7f0e38069420 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4c0ce330 0x7f0e4c0c9c90 unknown :-1 s=CLOSED pgs=3006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 --2- 172.21.15.133:0/121072978 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0e4c0b1f70 0x7f0e4c0cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 -- 172.21.15.133:0/121072978 >> 172.21.15.133:0/121072978 conn(0x7f0e4c067000 msgr2=0x7f0e4c06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 -- 172.21.15.133:0/121072978 shutdown_connections 2024-04-25T10:19:05.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.401+0000 7f0e52fd3640 1 -- 172.21.15.133:0/121072978 wait complete. 2024-04-25T10:19:05.419 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:05.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f1b91e640 1 Processor -- start 2024-04-25T10:19:05.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f1b91e640 1 -- start start 2024-04-25T10:19:05.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f1b91e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f1f14158110 0x7f1f140b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f1b91e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f140cbe60 con 0x7f1f141584f0 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f1b91e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f140b19b0 con 0x7f1f1406eb70 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f1b91e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1415b360 con 0x7f1f14158110 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f18eb9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1f141584f0 0x7f1f1406e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34400/0 (socket says 172.21.15.133:34400) 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f18eb9640 1 -- 172.21.15.133:0/1748387170 learned_addr learned my addr 172.21.15.133:0/1748387170 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f196ba640 1 --2- 172.21.15.133:0/1748387170 >> v2:172.21.15.162:3300/0 conn(0x7f1f14158110 0x7f1f140b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807143896 0 0) 0x7f1f140cbe60 con 0x7f1f141584f0 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f00003530 con 0x7f1f141584f0 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1597365404 0 0) 0x7f1f00003530 con 0x7f1f141584f0 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 >> v2:172.21.15.162:3300/0 conn(0x7f1f14158110 msgr2=0x7f1f140b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 --2- 172.21.15.133:0/1748387170 >> v2:172.21.15.162:3300/0 conn(0x7f1f14158110 0x7f1f140b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 >> v1:172.21.15.162:6789/0 conn(0x7f1f1406eb70 legacy=0x7f1f140cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1415b640 con 0x7f1f141584f0 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f1f04001680 con 0x7f1f141584f0 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 >> v1:172.21.15.133:6789/0 conn(0x7f1f141584f0 legacy=0x7f1f1406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 --2- 172.21.15.133:0/1748387170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f000039c0 0x7f1f00003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 --2- 172.21.15.133:0/1748387170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 0x7f1f00004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 --2- 172.21.15.133:0/1748387170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f000051d0 0x7f1f0000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1415b360 con 0x7f1f00004810 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0000c070 con 0x7f1f000051d0 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0000c270 con 0x7f1f000039c0 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f18eb9640 1 --2- 172.21.15.133:0/1748387170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 0x7f1f00004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f196ba640 1 --2- 172.21.15.133:0/1748387170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f000039c0 0x7f1f00003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f19ebb640 1 --2- 172.21.15.133:0/1748387170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f000051d0 0x7f1f0000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f18eb9640 1 -- 172.21.15.133:0/1748387170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f000039c0 msgr2=0x7f1f00003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f18eb9640 1 --2- 172.21.15.133:0/1748387170 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f000039c0 0x7f1f00003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f18eb9640 1 -- 172.21.15.133:0/1748387170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f000051d0 msgr2=0x7f1f0000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f18eb9640 1 --2- 172.21.15.133:0/1748387170 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f000051d0 0x7f1f0000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f19ebb640 1 -- 172.21.15.133:0/1748387170 reap_dead start 2024-04-25T10:19:05.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.525+0000 7f1f18eb9640 1 -- 172.21.15.133:0/1748387170 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f00003530 con 0x7f1f00004810 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f18eb9640 1 --2- 172.21.15.133:0/1748387170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 0x7f1f00004c90 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7f1f140cf5f0 tx=0x7f1f0401f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6a746726586342 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04007d20 con 0x7f1f00004810 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f04005ae0 con 0x7f1f00004810 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f0bfff640 1 -- 172.21.15.133:0/1748387170 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04007070 con 0x7f1f00004810 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 -- 172.21.15.133:0/1748387170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 msgr2=0x7f1f00004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 --2- 172.21.15.133:0/1748387170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 0x7f1f00004c90 secure :-1 s=READY pgs=3008 cs=0 l=1 rev1=1 crypto rx=0x7f1f140cf5f0 tx=0x7f1f0401f390 comp rx=0 tx=0).stop 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 -- 172.21.15.133:0/1748387170 shutdown_connections 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 --2- 172.21.15.133:0/1748387170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 0x7f1f00004c90 unknown :-1 s=CLOSED pgs=3008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 -- 172.21.15.133:0/1748387170 >> 172.21.15.133:0/1748387170 conn(0x7f1f14067000 msgr2=0x7f1f1406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 -- 172.21.15.133:0/1748387170 shutdown_connections 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 -- 172.21.15.133:0/1748387170 wait complete. 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 Processor -- start 2024-04-25T10:19:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 -- start start 2024-04-25T10:19:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 0x7f1f140c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140c9330 0x7f1f140c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f140c9cf0 0x7f1f14158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140cd670 con 0x7f1f00004810 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1415bc00 con 0x7f1f140c9cf0 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f1b91e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140cd7f0 con 0x7f1f140c9330 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f196ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 0x7f1f140c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f196ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 0x7f1f140c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60462/0 (socket says 172.21.15.133:60462) 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.529+0000 7f1f196ba640 1 -- 172.21.15.133:0/186064739 learned_addr learned my addr 172.21.15.133:0/186064739 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f19ebb640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f140c9cf0 0x7f1f14158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f18eb9640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140c9330 0x7f1f140c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f196ba640 1 -- 172.21.15.133:0/186064739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140c9330 msgr2=0x7f1f140c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f196ba640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140c9330 0x7f1f140c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f196ba640 1 -- 172.21.15.133:0/186064739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f140c9cf0 msgr2=0x7f1f14158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f196ba640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f140c9cf0 0x7f1f14158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f196ba640 1 -- 172.21.15.133:0/186064739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f140ca290 con 0x7f1f00004810 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f196ba640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 0x7f1f140c75c0 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7f1f10010900 tx=0x7f1f10010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f09ffb640 1 -- 172.21.15.133:0/186064739 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000f040 con 0x7f1f00004810 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f09ffb640 1 -- 172.21.15.133:0/186064739 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1000ec20 con 0x7f1f00004810 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f1b91e640 1 -- 172.21.15.133:0/186064739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f140ca570 con 0x7f1f00004810 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f09ffb640 1 -- 172.21.15.133:0/186064739 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1001c400 con 0x7f1f00004810 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f1b91e640 1 -- 172.21.15.133:0/186064739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f14152e10 con 0x7f1f00004810 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f1b91e640 1 -- 172.21.15.133:0/186064739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1406bc20 con 0x7f1f00004810 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f09ffb640 1 -- 172.21.15.133:0/186064739 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f1001c5a0 con 0x7f1f00004810 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f09ffb640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ef8066f90 0x7f1ef8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f09ffb640 1 -- 172.21.15.133:0/186064739 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1f10085990 con 0x7f1f00004810 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f18eb9640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ef8066f90 0x7f1ef8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.533+0000 7f1f18eb9640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ef8066f90 0x7f1ef8069450 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1f040af000 tx=0x7f1f040aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.537+0000 7f1f09ffb640 1 -- 172.21.15.133:0/186064739 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f10018030 con 0x7f1f00004810 2024-04-25T10:19:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.697+0000 7f1f1b91e640 1 -- 172.21.15.133:0/186064739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f14068ef0 con 0x7f1f00004810 2024-04-25T10:19:05.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f09ffb640 1 -- 172.21.15.133:0/186064739 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17234 (secure 0 0 0) 0x7f1f10056750 con 0x7f1f00004810 2024-04-25T10:19:05.702 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:05.702 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":198,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:18:36.089833+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:34.663983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:35.974809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":6,"to":0}]},{"pgid":"29.c","mappings":[{"from":6,"to":0}]},{"pgid":"29.d","mappings":[{"from":6,"to":0}]},{"pgid":"29.14","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/1916780253":"2024-04-26T10:17:55.746978+0000","172.21.15.133:6836/1916780253":"2024-04-26T10:17:55.746978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 -- 172.21.15.133:0/186064739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ef8066f90 msgr2=0x7f1ef8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ef8066f90 0x7f1ef8069450 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1f040af000 tx=0x7f1f040aa000 comp rx=0 tx=0).stop 2024-04-25T10:19:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 -- 172.21.15.133:0/186064739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 msgr2=0x7f1f140c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 0x7f1f140c75c0 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7f1f10010900 tx=0x7f1f10010dc0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 -- 172.21.15.133:0/186064739 shutdown_connections 2024-04-25T10:19:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1f140c9cf0 0x7f1f14158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1f140c9330 0x7f1f140c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1ef8066f90 0x7f1ef8069450 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 --2- 172.21.15.133:0/186064739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00004810 0x7f1f140c75c0 unknown :-1 s=CLOSED pgs=3009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 -- 172.21.15.133:0/186064739 >> 172.21.15.133:0/186064739 conn(0x7f1f14067000 msgr2=0x7f1f140cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 -- 172.21.15.133:0/186064739 shutdown_connections 2024-04-25T10:19:05.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.701+0000 7f1f1b91e640 1 -- 172.21.15.133:0/186064739 wait complete. 2024-04-25T10:19:05.720 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:19:05.720 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:05.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.833+0000 7ff0ce1ff640 1 Processor -- start 2024-04-25T10:19:05.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.833+0000 7ff0ce1ff640 1 -- start start 2024-04-25T10:19:05.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0ce1ff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0c8158110 0x7ff0c80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0ce1ff640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7ff0c81584f0 0x7ff0c815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0ce1ff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c806e460 0x7ff0c806e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0ce1ff640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c806ee10 con 0x7ff0c8158110 2024-04-25T10:19:05.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0ce1ff640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c80cbe60 con 0x7ff0c81584f0 2024-04-25T10:19:05.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0ce1ff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80b19b0 con 0x7ff0c806e460 2024-04-25T10:19:05.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0c8158110 0x7ff0c80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c77fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0c8158110 0x7ff0c80b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60476/0 (socket says 172.21.15.133:60476) 2024-04-25T10:19:05.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c77fe640 1 -- 172.21.15.133:0/2455830444 learned_addr learned my addr 172.21.15.133:0/2455830444 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:05.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c6ffd640 1 --2- 172.21.15.133:0/2455830444 >> v2:172.21.15.162:3300/0 conn(0x7ff0c81584f0 0x7ff0c815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c7fff640 1 --2- 172.21.15.133:0/2455830444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c806e460 0x7ff0c806e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c77fe640 1 -- 172.21.15.133:0/2455830444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c806e460 msgr2=0x7ff0c806e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c77fe640 1 --2- 172.21.15.133:0/2455830444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c806e460 0x7ff0c806e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c77fe640 1 -- 172.21.15.133:0/2455830444 >> v2:172.21.15.162:3300/0 conn(0x7ff0c81584f0 msgr2=0x7ff0c815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c77fe640 1 --2- 172.21.15.133:0/2455830444 >> v2:172.21.15.162:3300/0 conn(0x7ff0c81584f0 0x7ff0c815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c77fe640 1 -- 172.21.15.133:0/2455830444 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c815b360 con 0x7ff0c8158110 2024-04-25T10:19:05.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c77fe640 1 --2- 172.21.15.133:0/2455830444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c8158110 0x7ff0c80b1f00 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80b4da0 tx=0x7ff0b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd0e578f9d10840 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c67fc640 1 -- 172.21.15.133:0/2455830444 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b80aa610 con 0x7ff0c8158110 2024-04-25T10:19:05.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c67fc640 1 -- 172.21.15.133:0/2455830444 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b80af070 con 0x7ff0c8158110 2024-04-25T10:19:05.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0c67fc640 1 -- 172.21.15.133:0/2455830444 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b80aa9d0 con 0x7ff0c8158110 2024-04-25T10:19:05.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2455830444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c8158110 msgr2=0x7ff0c80b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.837+0000 7ff0ce1ff640 1 --2- 172.21.15.133:0/2455830444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c8158110 0x7ff0c80b1f00 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80b4da0 tx=0x7ff0b801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2455830444 shutdown_connections 2024-04-25T10:19:05.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 --2- 172.21.15.133:0/2455830444 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c806e460 0x7ff0c806e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 --2- 172.21.15.133:0/2455830444 >> v2:172.21.15.162:3300/0 conn(0x7ff0c81584f0 0x7ff0c815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 --2- 172.21.15.133:0/2455830444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c8158110 0x7ff0c80b1f00 unknown :-1 s=CLOSED pgs=3010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2455830444 >> 172.21.15.133:0/2455830444 conn(0x7ff0c8067000 msgr2=0x7ff0c806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2455830444 shutdown_connections 2024-04-25T10:19:05.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2455830444 wait complete. 2024-04-25T10:19:05.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 Processor -- start 2024-04-25T10:19:05.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 -- start start 2024-04-25T10:19:05.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0c806e460 0x7ff0c8139020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c8158110 0x7ff0c8166b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c81584f0 0x7ff0c8163840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80cd7f0 con 0x7ff0c8158110 2024-04-25T10:19:05.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80cd670 con 0x7ff0c806e460 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c815eb10 con 0x7ff0c81584f0 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c8158110 0x7ff0c8166b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c8158110 0x7ff0c8166b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60490/0 (socket says 172.21.15.133:60490) 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c6ffd640 1 -- 172.21.15.133:0/2600758161 learned_addr learned my addr 172.21.15.133:0/2600758161 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c77fe640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0c806e460 0x7ff0c8139020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c7fff640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c81584f0 0x7ff0c8163840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c6ffd640 1 -- 172.21.15.133:0/2600758161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c81584f0 msgr2=0x7ff0c8163840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c6ffd640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c81584f0 0x7ff0c8163840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c6ffd640 1 -- 172.21.15.133:0/2600758161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0c806e460 msgr2=0x7ff0c8139020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c6ffd640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0c806e460 0x7ff0c8139020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c6ffd640 1 -- 172.21.15.133:0/2600758161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c81599c0 con 0x7ff0c8158110 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c7fff640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c81584f0 0x7ff0c8163840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0c6ffd640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c8158110 0x7ff0c8166b80 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00d5f0 tx=0x7ff0bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0affff640 1 -- 172.21.15.133:0/2600758161 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc004240 con 0x7ff0c8158110 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0affff640 1 -- 172.21.15.133:0/2600758161 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0043e0 con 0x7ff0c8158110 2024-04-25T10:19:05.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2600758161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c8159ca0 con 0x7ff0c8158110 2024-04-25T10:19:05.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0affff640 1 -- 172.21.15.133:0/2600758161 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc003840 con 0x7ff0c8158110 2024-04-25T10:19:05.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2600758161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c815a260 con 0x7ff0c8158110 2024-04-25T10:19:05.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.841+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2600758161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c8068ef0 con 0x7ff0c8158110 2024-04-25T10:19:05.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.845+0000 7ff0affff640 1 -- 172.21.15.133:0/2600758161 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0bc0039e0 con 0x7ff0c8158110 2024-04-25T10:19:05.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.845+0000 7ff0affff640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0b4066f60 0x7ff0b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.845+0000 7ff0affff640 1 -- 172.21.15.133:0/2600758161 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ff0bc0854d0 con 0x7ff0c8158110 2024-04-25T10:19:05.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.845+0000 7ff0c77fe640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0b4066f60 0x7ff0b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.845+0000 7ff0c77fe640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0b4066f60 0x7ff0b4069420 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80ae000 tx=0x7ff0b80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:05.849+0000 7ff0affff640 1 -- 172.21.15.133:0/2600758161 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0bc01c360 con 0x7ff0c8158110 2024-04-25T10:19:06.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.021+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2600758161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0c80cb400 con 0x7ff0c8158110 2024-04-25T10:19:06.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0affff640 1 -- 172.21.15.133:0/2600758161 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4963 (secure 0 0 0) 0x7ff0bc056320 con 0x7ff0c8158110 2024-04-25T10:19:06.025 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:06.025 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":124,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:36.969368+0000","modified":"2024-04-25T10:17:58.776693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":196,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12135":{"gid":12135,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/2510709242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2510709242},{"type":"v1","addr":"172.21.15.133:6837","nonce":2510709242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:19:06.026 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 124 2024-04-25T10:19:06.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2600758161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0b4066f60 msgr2=0x7ff0b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0b4066f60 0x7ff0b4069420 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80ae000 tx=0x7ff0b80a7000 comp rx=0 tx=0).stop 2024-04-25T10:19:06.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2600758161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c8158110 msgr2=0x7ff0c8166b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c8158110 0x7ff0c8166b80 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00d5f0 tx=0x7ff0bc00dab0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2600758161 shutdown_connections 2024-04-25T10:19:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ff0c81584f0 0x7ff0c8163840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ff0b4066f60 0x7ff0b4069420 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0c8158110 0x7ff0c8166b80 unknown :-1 s=CLOSED pgs=3011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 --2- 172.21.15.133:0/2600758161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0c806e460 0x7ff0c8139020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2600758161 >> 172.21.15.133:0/2600758161 conn(0x7ff0c8067000 msgr2=0x7ff0c80cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2600758161 shutdown_connections 2024-04-25T10:19:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.025+0000 7ff0ce1ff640 1 -- 172.21.15.133:0/2600758161 wait complete. 2024-04-25T10:19:06.043 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:06.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.161+0000 7f32fccf0640 1 Processor -- start 2024-04-25T10:19:06.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.161+0000 7f32fccf0640 1 -- start start 2024-04-25T10:19:06.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.161+0000 7f32fccf0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32f8158110 0x7f32f80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.161+0000 7f32fccf0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f32f806eb70 0x7f32f815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.161+0000 7f32fccf0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f80cbe60 con 0x7f32f81584f0 2024-04-25T10:19:06.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.161+0000 7f32fccf0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f80b19b0 con 0x7f32f806eb70 2024-04-25T10:19:06.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.161+0000 7f32fccf0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815b360 con 0x7f32f8158110 2024-04-25T10:19:06.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.161+0000 7f32f5d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f32f81584f0 0x7f32f806e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34408/0 (socket says 172.21.15.133:34408) 2024-04-25T10:19:06.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.161+0000 7f32f5d9b640 1 -- 172.21.15.133:0/860224807 learned_addr learned my addr 172.21.15.133:0/860224807 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:06.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f6d9d640 1 --2- 172.21.15.133:0/860224807 >> v2:172.21.15.162:3300/0 conn(0x7f32f806eb70 0x7f32f815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f659c640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32f8158110 0x7f32f80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676925488 0 0) 0x7f32f80cbe60 con 0x7f32f81584f0 2024-04-25T10:19:06.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e0003530 con 0x7f32f81584f0 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3690732773 0 0) 0x7f32e0003530 con 0x7f32f81584f0 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32f8158110 msgr2=0x7f32f80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32f8158110 0x7f32f80b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 >> v2:172.21.15.162:3300/0 conn(0x7f32f806eb70 msgr2=0x7f32f815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 --2- 172.21.15.133:0/860224807 >> v2:172.21.15.162:3300/0 conn(0x7f32f806eb70 0x7f32f815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f815b640 con 0x7f32f81584f0 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f32ec001680 con 0x7f32f81584f0 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 >> v1:172.21.15.133:6789/0 conn(0x7f32f81584f0 legacy=0x7f32f806e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32e00039c0 0x7f32e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32e0004810 0x7f32e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32f8158110 0x7f32e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80b19b0 con 0x7f32e0004810 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815b360 con 0x7f32e00039c0 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e000b9d0 con 0x7f32f8158110 2024-04-25T10:19:06.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f5d9b640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32e0004810 0x7f32e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f659c640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32e00039c0 0x7f32e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f5d9b640 1 -- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32f8158110 msgr2=0x7f32e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f6d9d640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32f8158110 0x7f32e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f5d9b640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32f8158110 0x7f32e00051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f5d9b640 1 -- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32e00039c0 msgr2=0x7f32e0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f5d9b640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32e00039c0 0x7f32e0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f5d9b640 1 -- 172.21.15.133:0/860224807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0003530 con 0x7f32e0004810 2024-04-25T10:19:06.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f659c640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32e00039c0 0x7f32e0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f5d9b640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32e0004810 0x7f32e0004c90 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c210 tx=0x7f32ec01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7317ef1084771e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec007d60 con 0x7f32e0004810 2024-04-25T10:19:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec005ae0 con 0x7f32e0004810 2024-04-25T10:19:06.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.165+0000 7f32f559a640 1 -- 172.21.15.133:0/860224807 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec007070 con 0x7f32e0004810 2024-04-25T10:19:06.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 -- 172.21.15.133:0/860224807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32e0004810 msgr2=0x7f32e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 --2- 172.21.15.133:0/860224807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32e0004810 0x7f32e0004c90 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c210 tx=0x7f32ec01f3f0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f6d9d640 1 -- 172.21.15.133:0/860224807 reap_dead start 2024-04-25T10:19:06.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 -- 172.21.15.133:0/860224807 shutdown_connections 2024-04-25T10:19:06.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 -- 172.21.15.133:0/860224807 >> 172.21.15.133:0/860224807 conn(0x7f32f8067000 msgr2=0x7f32f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 -- 172.21.15.133:0/860224807 shutdown_connections 2024-04-25T10:19:06.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 -- 172.21.15.133:0/860224807 wait complete. 2024-04-25T10:19:06.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 Processor -- start 2024-04-25T10:19:06.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 -- start start 2024-04-25T10:19:06.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32e0004810 0x7f32f815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32f80c6b80 0x7f32f814a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32f80cb2b0 0x7f32f8157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80cd670 con 0x7f32f80cb2b0 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815bc00 con 0x7f32f80c6b80 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80cd7f0 con 0x7f32e0004810 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32f80cb2b0 0x7f32f8157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32f80cb2b0 0x7f32f8157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46538/0 (socket says 172.21.15.133:46538) 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f6d9d640 1 -- 172.21.15.133:0/2023326088 learned_addr learned my addr 172.21.15.133:0/2023326088 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f5d9b640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32f80c6b80 0x7f32f814a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f659c640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32e0004810 0x7f32f815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f6d9d640 1 -- 172.21.15.133:0/2023326088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32e0004810 msgr2=0x7f32f815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f6d9d640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32e0004810 0x7f32f815c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f6d9d640 1 -- 172.21.15.133:0/2023326088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32f80c6b80 msgr2=0x7f32f814a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f6d9d640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32f80c6b80 0x7f32f814a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f6d9d640 1 -- 172.21.15.133:0/2023326088 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f80c7010 con 0x7f32f80cb2b0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32f6d9d640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32f80cb2b0 0x7f32f8157180 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7f32f80cf5d0 tx=0x7f32f000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32e6ffd640 1 -- 172.21.15.133:0/2023326088 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f001c070 con 0x7f32f80cb2b0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32e6ffd640 1 -- 172.21.15.133:0/2023326088 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f0017400 con 0x7f32f80cb2b0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 -- 172.21.15.133:0/2023326088 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f814fae0 con 0x7f32f80cb2b0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32e6ffd640 1 -- 172.21.15.133:0/2023326088 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f00177f0 con 0x7f32f80cb2b0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.169+0000 7f32fccf0640 1 -- 172.21.15.133:0/2023326088 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f81576c0 con 0x7f32f80cb2b0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.173+0000 7f32fccf0640 1 -- 172.21.15.133:0/2023326088 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f806bc20 con 0x7f32f80cb2b0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.173+0000 7f32e6ffd640 1 -- 172.21.15.133:0/2023326088 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32f001f070 con 0x7f32f80cb2b0 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.173+0000 7f32e6ffd640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32d8066f90 0x7f32d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.173+0000 7f32f659c640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32d8066f90 0x7f32d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.173+0000 7f32e6ffd640 1 -- 172.21.15.133:0/2023326088 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f32f0085620 con 0x7f32f80cb2b0 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.173+0000 7f32f659c640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32d8066f90 0x7f32d8069450 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f32f81f4d70 tx=0x7f32e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.177+0000 7f32e6ffd640 1 -- 172.21.15.133:0/2023326088 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32f0012070 con 0x7f32f80cb2b0 2024-04-25T10:19:06.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.369+0000 7f32fccf0640 1 -- 172.21.15.133:0/2023326088 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32f8068ef0 con 0x7f32f80cb2b0 2024-04-25T10:19:06.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.369+0000 7f32e6ffd640 1 -- 172.21.15.133:0/2023326088 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f32f0056470 con 0x7f32f80cb2b0 2024-04-25T10:19:06.373 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:06.373 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":124,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:36.969368+0000","modified":"2024-04-25T10:17:58.776693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":196,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12135":{"gid":12135,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/2510709242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2510709242},{"type":"v1","addr":"172.21.15.133:6837","nonce":2510709242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:19:06.373 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 124 2024-04-25T10:19:06.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 -- 172.21.15.133:0/2023326088 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32d8066f90 msgr2=0x7f32d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32d8066f90 0x7f32d8069450 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f32f81f4d70 tx=0x7f32e8008040 comp rx=0 tx=0).stop 2024-04-25T10:19:06.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 -- 172.21.15.133:0/2023326088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32f80cb2b0 msgr2=0x7f32f8157180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32f80cb2b0 0x7f32f8157180 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7f32f80cf5d0 tx=0x7f32f000f4a0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 -- 172.21.15.133:0/2023326088 shutdown_connections 2024-04-25T10:19:06.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f32d8066f90 0x7f32d8069450 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32f80cb2b0 0x7f32f8157180 unknown :-1 s=CLOSED pgs=3014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32f80c6b80 0x7f32f814a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 --2- 172.21.15.133:0/2023326088 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32e0004810 0x7f32f815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 -- 172.21.15.133:0/2023326088 >> 172.21.15.133:0/2023326088 conn(0x7f32f8067000 msgr2=0x7f32f80711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 -- 172.21.15.133:0/2023326088 shutdown_connections 2024-04-25T10:19:06.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.373+0000 7f32fccf0640 1 -- 172.21.15.133:0/2023326088 wait complete. 2024-04-25T10:19:06.391 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:06.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98dabc3640 1 Processor -- start 2024-04-25T10:19:06.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98dabc3640 1 -- start start 2024-04-25T10:19:06.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98dabc3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f98d4152b30 0x7f98d40b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98dabc3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d406d350 con 0x7f98d4152750 2024-04-25T10:19:06.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98dabc3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d415bf90 con 0x7f98d40cea40 2024-04-25T10:19:06.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98dabc3640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d406d5d0 con 0x7f98d4152b30 2024-04-25T10:19:06.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98d895f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f98d4152750 0x7f98d40ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40918/0 (socket says 172.21.15.133:40918) 2024-04-25T10:19:06.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98d895f640 1 -- 172.21.15.133:0/2820227014 learned_addr learned my addr 172.21.15.133:0/2820227014 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:06.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98d3fff640 1 --2- 172.21.15.133:0/2820227014 >> v2:172.21.15.133:3300/0 conn(0x7f98d4152b30 0x7f98d40b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98d3fff640 1 -- 172.21.15.133:0/2820227014 >> v1:172.21.15.162:6789/0 conn(0x7f98d40cea40 legacy=0x7f98d40b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98d3fff640 1 -- 172.21.15.133:0/2820227014 >> v1:172.21.15.133:6789/0 conn(0x7f98d4152750 legacy=0x7f98d40ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.505+0000 7f98d3fff640 1 -- 172.21.15.133:0/2820227014 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d415c230 con 0x7f98d4152b30 2024-04-25T10:19:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d3fff640 1 --2- 172.21.15.133:0/2820227014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98d4152b30 0x7f98d40b17b0 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 crypto rx=0x7f98c00097c0 tx=0x7f98c001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956ab8f94d83896e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d37fe640 1 -- 172.21.15.133:0/2820227014 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00b0070 con 0x7f98d4152b30 2024-04-25T10:19:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d37fe640 1 -- 172.21.15.133:0/2820227014 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c0004230 con 0x7f98d4152b30 2024-04-25T10:19:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d37fe640 1 -- 172.21.15.133:0/2820227014 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0003620 con 0x7f98d4152b30 2024-04-25T10:19:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 -- 172.21.15.133:0/2820227014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98d4152b30 msgr2=0x7f98d40b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 --2- 172.21.15.133:0/2820227014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98d4152b30 0x7f98d40b17b0 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 crypto rx=0x7f98c00097c0 tx=0x7f98c001c460 comp rx=0 tx=0).stop 2024-04-25T10:19:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 -- 172.21.15.133:0/2820227014 shutdown_connections 2024-04-25T10:19:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 --2- 172.21.15.133:0/2820227014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98d4152b30 0x7f98d40b17b0 unknown :-1 s=CLOSED pgs=3016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 -- 172.21.15.133:0/2820227014 >> 172.21.15.133:0/2820227014 conn(0x7f98d4067000 msgr2=0x7f98d406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 -- 172.21.15.133:0/2820227014 shutdown_connections 2024-04-25T10:19:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 -- 172.21.15.133:0/2820227014 wait complete. 2024-04-25T10:19:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 Processor -- start 2024-04-25T10:19:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 -- start start 2024-04-25T10:19:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98d40cea40 0x7f98d41385b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98d4152750 0x7f98d41564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98d4152b30 0x7f98d41569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d4169180 con 0x7f98d40cea40 2024-04-25T10:19:06.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d4169000 con 0x7f98d4152b30 2024-04-25T10:19:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d4169300 con 0x7f98d4152750 2024-04-25T10:19:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d895f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98d40cea40 0x7f98d41385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d895f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98d40cea40 0x7f98d41385b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46570/0 (socket says 172.21.15.133:46570) 2024-04-25T10:19:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d895f640 1 -- 172.21.15.133:0/665270076 learned_addr learned my addr 172.21.15.133:0/665270076 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d895f640 1 -- 172.21.15.133:0/665270076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98d4152750 msgr2=0x7f98d41564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d895f640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98d4152750 0x7f98d41564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d895f640 1 -- 172.21.15.133:0/665270076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98d4152b30 msgr2=0x7f98d41569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d895f640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98d4152b30 0x7f98d41569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d895f640 1 -- 172.21.15.133:0/665270076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d4159200 con 0x7f98d40cea40 2024-04-25T10:19:06.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d895f640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98d40cea40 0x7f98d41385b0 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c420 tx=0x7f98c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d17fa640 1 -- 172.21.15.133:0/665270076 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8017070 con 0x7f98d40cea40 2024-04-25T10:19:06.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d17fa640 1 -- 172.21.15.133:0/665270076 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c8005cf0 con 0x7f98d40cea40 2024-04-25T10:19:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 -- 172.21.15.133:0/665270076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d41594e0 con 0x7f98d40cea40 2024-04-25T10:19:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98d17fa640 1 -- 172.21.15.133:0/665270076 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c80072f0 con 0x7f98d40cea40 2024-04-25T10:19:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.509+0000 7f98dabc3640 1 -- 172.21.15.133:0/665270076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d4150db0 con 0x7f98d40cea40 2024-04-25T10:19:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.513+0000 7f98dabc3640 1 -- 172.21.15.133:0/665270076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d4068ef0 con 0x7f98d40cea40 2024-04-25T10:19:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.513+0000 7f98d17fa640 1 -- 172.21.15.133:0/665270076 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98c801a070 con 0x7f98d40cea40 2024-04-25T10:19:06.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.513+0000 7f98d17fa640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98b8066f60 0x7f98b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.513+0000 7f98d17fa640 1 -- 172.21.15.133:0/665270076 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f98c80845b0 con 0x7f98d40cea40 2024-04-25T10:19:06.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.513+0000 7f98d3fff640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98b8066f60 0x7f98b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.513+0000 7f98d3fff640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98b8066f60 0x7f98b8069420 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f98c00ad000 tx=0x7f98c0004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.517+0000 7f98d17fa640 1 -- 172.21.15.133:0/665270076 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98d406d350 con 0x7f98d40cea40 2024-04-25T10:19:06.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.665+0000 7f98dabc3640 1 -- 172.21.15.133:0/665270076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98d415f0e0 con 0x7f98d40cea40 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98d17fa640 1 -- 172.21.15.133:0/665270076 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17234 (secure 0 0 0) 0x7f98c8055400 con 0x7f98d40cea40 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":198,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:18:36.089833+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:34.663983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:35.974809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":6,"to":0}]},{"pgid":"29.c","mappings":[{"from":6,"to":0}]},{"pgid":"29.d","mappings":[{"from":6,"to":0}]},{"pgid":"29.14","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/1916780253":"2024-04-26T10:17:55.746978+0000","172.21.15.133:6836/1916780253":"2024-04-26T10:17:55.746978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 -- 172.21.15.133:0/665270076 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98b8066f60 msgr2=0x7f98b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98b8066f60 0x7f98b8069420 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f98c00ad000 tx=0x7f98c0004740 comp rx=0 tx=0).stop 2024-04-25T10:19:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 -- 172.21.15.133:0/665270076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98d40cea40 msgr2=0x7f98d41385b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98d40cea40 0x7f98d41385b0 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c420 tx=0x7f98c800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 -- 172.21.15.133:0/665270076 shutdown_connections 2024-04-25T10:19:06.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98d4152b30 0x7f98d41569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f98d4152750 0x7f98d41564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f98b8066f60 0x7f98b8069420 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 --2- 172.21.15.133:0/665270076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98d40cea40 0x7f98d41385b0 unknown :-1 s=CLOSED pgs=3017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 -- 172.21.15.133:0/665270076 >> 172.21.15.133:0/665270076 conn(0x7f98d4067000 msgr2=0x7f98d40c5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 -- 172.21.15.133:0/665270076 shutdown_connections 2024-04-25T10:19:06.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.669+0000 7f98dabc3640 1 -- 172.21.15.133:0/665270076 wait complete. 2024-04-25T10:19:06.688 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:19:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.801+0000 7f17666f8640 1 Processor -- start 2024-04-25T10:19:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.801+0000 7f17666f8640 1 -- start start 2024-04-25T10:19:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f176015b850 0x7f17600cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f17600d0100 0x7f17600b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17600cc530 con 0x7f17600b2cf0 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17600b3160 con 0x7f17600d0100 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17600cd140 con 0x7f176015b850 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f175f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f17600d0100 0x7f17600b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f1764c95640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f17600b2cf0 0x7f176008af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40928/0 (socket says 172.21.15.133:40928) 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f1764c95640 1 -- 172.21.15.133:0/1023493321 learned_addr learned my addr 172.21.15.133:0/1023493321 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f175ffff640 1 --2- 172.21.15.133:0/1023493321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f176015b850 0x7f17600cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f175f7fe640 1 -- 172.21.15.133:0/1023493321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f176015b850 msgr2=0x7f17600cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f175f7fe640 1 --2- 172.21.15.133:0/1023493321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f176015b850 0x7f17600cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f175f7fe640 1 -- 172.21.15.133:0/1023493321 >> v1:172.21.15.133:6789/0 conn(0x7f17600b2cf0 legacy=0x7f176008af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f175f7fe640 1 -- 172.21.15.133:0/1023493321 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17600cc810 con 0x7f17600d0100 2024-04-25T10:19:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f175effd640 1 -- 172.21.15.133:0/1023493321 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150518325 0 0) 0x7f17600cc530 con 0x7f17600b2cf0 2024-04-25T10:19:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f175f7fe640 1 --2- 172.21.15.133:0/1023493321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17600d0100 0x7f17600b27b0 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 crypto rx=0x7f17540097c0 tx=0x7f175401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569adde9c8544313 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f175effd640 1 -- 172.21.15.133:0/1023493321 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17540b2070 con 0x7f17600d0100 2024-04-25T10:19:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f175effd640 1 -- 172.21.15.133:0/1023493321 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17540ad640 con 0x7f17600d0100 2024-04-25T10:19:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f175effd640 1 -- 172.21.15.133:0/1023493321 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17540ada10 con 0x7f17600d0100 2024-04-25T10:19:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 -- 172.21.15.133:0/1023493321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17600d0100 msgr2=0x7f17600b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 --2- 172.21.15.133:0/1023493321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17600d0100 0x7f17600b27b0 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 crypto rx=0x7f17540097c0 tx=0x7f175401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 -- 172.21.15.133:0/1023493321 shutdown_connections 2024-04-25T10:19:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 --2- 172.21.15.133:0/1023493321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17600d0100 0x7f17600b27b0 unknown :-1 s=CLOSED pgs=3019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 --2- 172.21.15.133:0/1023493321 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f176015b850 0x7f17600cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 -- 172.21.15.133:0/1023493321 >> 172.21.15.133:0/1023493321 conn(0x7f176009ab90 msgr2=0x7f17600d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 -- 172.21.15.133:0/1023493321 shutdown_connections 2024-04-25T10:19:06.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.805+0000 7f17666f8640 1 -- 172.21.15.133:0/1023493321 wait complete. 2024-04-25T10:19:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f17666f8640 1 Processor -- start 2024-04-25T10:19:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f17666f8640 1 -- start start 2024-04-25T10:19:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f17666f8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17600b2cf0 0x7f1760165460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f17666f8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600d0100 0x7f1760163640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f17666f8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f176015b850 0x7f1760154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f17666f8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176015bdb0 con 0x7f17600b2cf0 2024-04-25T10:19:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f17666f8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176015bc30 con 0x7f17600d0100 2024-04-25T10:19:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f17666f8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17601684b0 con 0x7f176015b850 2024-04-25T10:19:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17600b2cf0 0x7f1760165460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17600b2cf0 0x7f1760165460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46578/0 (socket says 172.21.15.133:46578) 2024-04-25T10:19:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175ffff640 1 -- 172.21.15.133:0/2745637112 learned_addr learned my addr 172.21.15.133:0/2745637112 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f1764c95640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f176015b850 0x7f1760154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175ffff640 1 -- 172.21.15.133:0/2745637112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f176015b850 msgr2=0x7f1760154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175ffff640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f176015b850 0x7f1760154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175ffff640 1 -- 172.21.15.133:0/2745637112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600d0100 msgr2=0x7f1760163640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175ffff640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600d0100 0x7f1760163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175ffff640 1 -- 172.21.15.133:0/2745637112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17600afa10 con 0x7f17600b2cf0 2024-04-25T10:19:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175ffff640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17600b2cf0 0x7f1760165460 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7f175000d830 tx=0x7f175000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175cff9640 1 -- 172.21.15.133:0/2745637112 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17500059a0 con 0x7f17600b2cf0 2024-04-25T10:19:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175cff9640 1 -- 172.21.15.133:0/2745637112 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1750005b40 con 0x7f17600b2cf0 2024-04-25T10:19:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f17666f8640 1 -- 172.21.15.133:0/2745637112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176006c230 con 0x7f17600b2cf0 2024-04-25T10:19:06.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f175cff9640 1 -- 172.21.15.133:0/2745637112 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750005d30 con 0x7f17600b2cf0 2024-04-25T10:19:06.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f17666f8640 1 -- 172.21.15.133:0/2745637112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176006c7f0 con 0x7f17600b2cf0 2024-04-25T10:19:06.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.809+0000 7f17666f8640 1 -- 172.21.15.133:0/2745637112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1760068e80 con 0x7f17600b2cf0 2024-04-25T10:19:06.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.813+0000 7f175cff9640 1 -- 172.21.15.133:0/2745637112 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f175001b020 con 0x7f17600b2cf0 2024-04-25T10:19:06.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.813+0000 7f175cff9640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f174c066f60 0x7f174c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.813+0000 7f175cff9640 1 -- 172.21.15.133:0/2745637112 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1750085060 con 0x7f17600b2cf0 2024-04-25T10:19:06.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.813+0000 7f175f7fe640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f174c066f60 0x7f174c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.813+0000 7f175f7fe640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f174c066f60 0x7f174c069420 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f17540af000 tx=0x7f1754004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:06.817+0000 7f175cff9640 1 -- 172.21.15.133:0/2745637112 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1750055eb0 con 0x7f17600b2cf0 2024-04-25T10:19:07.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.009+0000 7f17666f8640 1 -- 172.21.15.133:0/2745637112 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1760070f80 con 0x7f17600b2cf0 2024-04-25T10:19:07.011 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-04-25T10:19:07.009+0000 7fded207d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:19:07.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.013+0000 7f175cff9640 1 -- 172.21.15.133:0/2745637112 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v125) v1 ==== 161+0+0 (secure 0 0 0) 0x7f17500557b0 con 0x7f17600b2cf0 2024-04-25T10:19:07.014 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:19:07.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 -- 172.21.15.133:0/2745637112 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f174c066f60 msgr2=0x7f174c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f174c066f60 0x7f174c069420 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f17540af000 tx=0x7f1754004740 comp rx=0 tx=0).stop 2024-04-25T10:19:07.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 -- 172.21.15.133:0/2745637112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17600b2cf0 msgr2=0x7f1760165460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17600b2cf0 0x7f1760165460 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7f175000d830 tx=0x7f175000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 -- 172.21.15.133:0/2745637112 shutdown_connections 2024-04-25T10:19:07.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f176015b850 0x7f1760154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17600d0100 0x7f1760163640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f174c066f60 0x7f174c069420 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 --2- 172.21.15.133:0/2745637112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17600b2cf0 0x7f1760165460 unknown :-1 s=CLOSED pgs=3020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 -- 172.21.15.133:0/2745637112 >> 172.21.15.133:0/2745637112 conn(0x7f176009ab90 msgr2=0x7f1760159920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 -- 172.21.15.133:0/2745637112 shutdown_connections 2024-04-25T10:19:07.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.017+0000 7f17666f8640 1 -- 172.21.15.133:0/2745637112 wait complete. 2024-04-25T10:19:07.028 INFO:tasks.ceph.mds.c.smithi133.stderr: -2540> 2024-04-25T10:18:02.758+0000 7fc92de52640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:19:07.035 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:19:07.062 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-04-25T10:19:07.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9fbae9640 1 Processor -- start 2024-04-25T10:19:07.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9fbae9640 1 -- start start 2024-04-25T10:19:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9fbae9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4145170 0x7fe9f406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9fbae9640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f40ce2e0 con 0x7fe9f415ced0 2024-04-25T10:19:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9fbae9640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f406d4b0 con 0x7fe9f406e4d0 2024-04-25T10:19:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9fbae9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f406db00 con 0x7fe9f4145170 2024-04-25T10:19:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9fa086640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe9f415ced0 0x7fe9f40b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40940/0 (socket says 172.21.15.133:40940) 2024-04-25T10:19:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9fa086640 1 -- 172.21.15.133:0/531379598 learned_addr learned my addr 172.21.15.133:0/531379598 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f9885640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4145170 0x7fe9f406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709497833 0 0) 0x7fe9f40ce2e0 con 0x7fe9f415ced0 2024-04-25T10:19:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e0003530 con 0x7fe9f415ced0 2024-04-25T10:19:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 244516145 0 0) 0x7fe9e0003530 con 0x7fe9f415ced0 2024-04-25T10:19:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4145170 msgr2=0x7fe9f406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4145170 0x7fe9f406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 >> v1:172.21.15.162:6789/0 conn(0x7fe9f406e4d0 legacy=0x7fe9f40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f406d790 con 0x7fe9f415ced0 2024-04-25T10:19:07.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe9ec001a00 con 0x7fe9f415ced0 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 >> v1:172.21.15.133:6789/0 conn(0x7fe9f415ced0 legacy=0x7fe9f40b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9e00039f0 0x7fe9e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9e0004810 0x7fe9e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4145170 0x7fe9e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f406db00 con 0x7fe9e00039f0 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e000ba80 con 0x7fe9e0004810 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.161+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e000bc80 con 0x7fe9f4145170 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f9885640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9e00039f0 0x7fe9e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9fa086640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4145170 0x7fe9e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f9084640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9e0004810 0x7fe9e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f9885640 1 -- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4145170 msgr2=0x7fe9e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f9885640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4145170 0x7fe9e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f9885640 1 -- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9e0004810 msgr2=0x7fe9e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f9885640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9e0004810 0x7fe9e0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f9885640 1 -- 172.21.15.133:0/531379598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e0003530 con 0x7fe9e00039f0 2024-04-25T10:19:07.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f9084640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9e0004810 0x7fe9e0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f9885640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9e00039f0 0x7fe9e0003e10 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009ab0 tx=0x7fe9f001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4720fb40f01c3359 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00096b0 con 0x7fe9e00039f0 2024-04-25T10:19:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f00a6410 con 0x7fe9e00039f0 2024-04-25T10:19:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9f8883640 1 -- 172.21.15.133:0/531379598 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00a6800 con 0x7fe9e00039f0 2024-04-25T10:19:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9fbae9640 1 -- 172.21.15.133:0/531379598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9e00039f0 msgr2=0x7fe9e0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9fbae9640 1 --2- 172.21.15.133:0/531379598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9e00039f0 0x7fe9e0003e10 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009ab0 tx=0x7fe9f001ca30 comp rx=0 tx=0).stop 2024-04-25T10:19:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9fa086640 1 -- 172.21.15.133:0/531379598 reap_dead start 2024-04-25T10:19:07.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9fbae9640 1 -- 172.21.15.133:0/531379598 shutdown_connections 2024-04-25T10:19:07.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9fbae9640 1 -- 172.21.15.133:0/531379598 >> 172.21.15.133:0/531379598 conn(0x7fe9f4066f70 msgr2=0x7fe9f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9fbae9640 1 -- 172.21.15.133:0/531379598 shutdown_connections 2024-04-25T10:19:07.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.165+0000 7fe9fbae9640 1 -- 172.21.15.133:0/531379598 wait complete. 2024-04-25T10:19:07.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fbae9640 1 Processor -- start 2024-04-25T10:19:07.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fbae9640 1 -- start start 2024-04-25T10:19:07.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fbae9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9e00039f0 0x7fe9f413b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fbae9640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4151f40 0x7fe9f413cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fbae9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f41523c0 0x7fe9f415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fbae9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f4145550 con 0x7fe9f41523c0 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fbae9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40b1fe0 con 0x7fe9e00039f0 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fbae9640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f406df70 con 0x7fe9f4151f40 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fa086640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f41523c0 0x7fe9f415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fa086640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f41523c0 0x7fe9f415e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46598/0 (socket says 172.21.15.133:46598) 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fa086640 1 -- 172.21.15.133:0/3910622065 learned_addr learned my addr 172.21.15.133:0/3910622065 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9f9885640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9e00039f0 0x7fe9f413b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9f9084640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4151f40 0x7fe9f413cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fa086640 1 -- 172.21.15.133:0/3910622065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4151f40 msgr2=0x7fe9f413cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fa086640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4151f40 0x7fe9f413cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fa086640 1 -- 172.21.15.133:0/3910622065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9e00039f0 msgr2=0x7fe9f413b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fa086640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9e00039f0 0x7fe9f413b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fa086640 1 -- 172.21.15.133:0/3910622065 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f413da40 con 0x7fe9f41523c0 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9f9084640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4151f40 0x7fe9f413cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:07.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9f9885640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9e00039f0 0x7fe9f413b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:07.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fa086640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f41523c0 0x7fe9f415e4c0 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7fe9f406f700 tx=0x7fe9ec007ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9ea7fc640 1 -- 172.21.15.133:0/3910622065 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00eaa0 con 0x7fe9f41523c0 2024-04-25T10:19:07.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9ea7fc640 1 -- 172.21.15.133:0/3910622065 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec00ec40 con 0x7fe9f41523c0 2024-04-25T10:19:07.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9ea7fc640 1 -- 172.21.15.133:0/3910622065 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00ee30 con 0x7fe9f41523c0 2024-04-25T10:19:07.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fbae9640 1 -- 172.21.15.133:0/3910622065 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f413a4a0 con 0x7fe9f41523c0 2024-04-25T10:19:07.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fbae9640 1 -- 172.21.15.133:0/3910622065 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f413aa60 con 0x7fe9f41523c0 2024-04-25T10:19:07.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9fbae9640 1 -- 172.21.15.133:0/3910622065 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f4068e80 con 0x7fe9f41523c0 2024-04-25T10:19:07.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9ea7fc640 1 -- 172.21.15.133:0/3910622065 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9ec01a020 con 0x7fe9f41523c0 2024-04-25T10:19:07.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9ea7fc640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9d8066f60 0x7fe9d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9ea7fc640 1 -- 172.21.15.133:0/3910622065 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe9ec084d20 con 0x7fe9f41523c0 2024-04-25T10:19:07.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.169+0000 7fe9f9885640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9d8066f60 0x7fe9d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.173+0000 7fe9f9885640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9d8066f60 0x7fe9d8069420 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00ad000 tx=0x7fe9f00046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.173+0000 7fe9ea7fc640 1 -- 172.21.15.133:0/3910622065 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9ec021020 con 0x7fe9f41523c0 2024-04-25T10:19:07.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:07.357+0000 7fe9fbae9640 1 -- 172.21.15.133:0/3910622065 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe9f40735d0 con 0x7fe9f41523c0 2024-04-25T10:19:08.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.013+0000 7fe9ea7fc640 1 -- 172.21.15.133:0/3910622065 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v126) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe9ec055b40 con 0x7fe9f41523c0 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 -- 172.21.15.133:0/3910622065 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9d8066f60 msgr2=0x7fe9d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9d8066f60 0x7fe9d8069420 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00ad000 tx=0x7fe9f00046d0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 -- 172.21.15.133:0/3910622065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f41523c0 msgr2=0x7fe9f415e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f41523c0 0x7fe9f415e4c0 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7fe9f406f700 tx=0x7fe9ec007ba0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 -- 172.21.15.133:0/3910622065 shutdown_connections 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9d8066f60 0x7fe9d8069420 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f41523c0 0x7fe9f415e4c0 unknown :-1 s=CLOSED pgs=3024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9f4151f40 0x7fe9f413cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 --2- 172.21.15.133:0/3910622065 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9e00039f0 0x7fe9f413b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 -- 172.21.15.133:0/3910622065 >> 172.21.15.133:0/3910622065 conn(0x7fe9f4066f70 msgr2=0x7fe9f415d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 -- 172.21.15.133:0/3910622065 shutdown_connections 2024-04-25T10:19:08.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.017+0000 7fe9fbae9640 1 -- 172.21.15.133:0/3910622065 wait complete. 2024-04-25T10:19:08.043 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:19:08.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60d6160640 1 Processor -- start 2024-04-25T10:19:08.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60d6160640 1 -- start start 2024-04-25T10:19:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60d6160640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f60d015b530 0x7f60d015b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60d6160640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ceda0 0x7f60d00ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60d6160640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d015bf40 con 0x7f60d00cc890 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60d6160640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d01589c0 con 0x7f60d015b530 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60d6160640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d015c200 con 0x7f60d00ceda0 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ceffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f60d00cc890 0x7f60d00ce690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40950/0 (socket says 172.21.15.133:40950) 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ceffd640 1 -- 172.21.15.133:0/2525429351 learned_addr learned my addr 172.21.15.133:0/2525429351 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031334923 0 0) 0x7f60d015bf40 con 0x7f60d00cc890 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60cf7fe640 1 --2- 172.21.15.133:0/2525429351 >> v2:172.21.15.162:3300/0 conn(0x7f60d015b530 0x7f60d015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60b8003530 con 0x7f60d00cc890 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60cffff640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ceda0 0x7f60d00ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 57497913 0 0) 0x7f60b8003530 con 0x7f60d00cc890 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ceda0 msgr2=0x7f60d00ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ceda0 0x7f60d00ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 >> v2:172.21.15.162:3300/0 conn(0x7f60d015b530 msgr2=0x7f60d015b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 --2- 172.21.15.133:0/2525429351 >> v2:172.21.15.162:3300/0 conn(0x7f60d015b530 0x7f60d015b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d015c4e0 con 0x7f60d00cc890 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f60c4001680 con 0x7f60d00cc890 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 >> v1:172.21.15.133:6789/0 conn(0x7f60d00cc890 legacy=0x7f60d00ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60b80039c0 0x7f60b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60b8004810 0x7f60b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ceda0 0x7f60b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d015c200 con 0x7f60b80039c0 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b800b9d0 con 0x7f60b8004810 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.209+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b800bb50 con 0x7f60d00ceda0 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60cf7fe640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60b80039c0 0x7f60b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60cffff640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ceda0 0x7f60b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60ceffd640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60b8004810 0x7f60b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60cf7fe640 1 -- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ceda0 msgr2=0x7f60b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:08.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60cf7fe640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ceda0 0x7f60b80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60cf7fe640 1 -- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60b8004810 msgr2=0x7f60b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:08.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60cf7fe640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60b8004810 0x7f60b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60cf7fe640 1 -- 172.21.15.133:0/2525429351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b8003530 con 0x7f60b80039c0 2024-04-25T10:19:08.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60ceffd640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60b8004810 0x7f60b8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:08.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60cf7fe640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60b80039c0 0x7f60b8003de0 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009b10 tx=0x7f60c001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7add221fac640386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:08.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c00096b0 con 0x7f60b80039c0 2024-04-25T10:19:08.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c00af070 con 0x7f60b80039c0 2024-04-25T10:19:08.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60ce7fc640 1 -- 172.21.15.133:0/2525429351 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c00aa690 con 0x7f60b80039c0 2024-04-25T10:19:08.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60d6160640 1 -- 172.21.15.133:0/2525429351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60b80039c0 msgr2=0x7f60b8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:08.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60d6160640 1 --2- 172.21.15.133:0/2525429351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60b80039c0 0x7f60b8003de0 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009b10 tx=0x7f60c001cab0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60cffff640 1 -- 172.21.15.133:0/2525429351 reap_dead start 2024-04-25T10:19:08.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60d6160640 1 -- 172.21.15.133:0/2525429351 shutdown_connections 2024-04-25T10:19:08.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60d6160640 1 -- 172.21.15.133:0/2525429351 >> 172.21.15.133:0/2525429351 conn(0x7f60d006d370 msgr2=0x7f60d006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:08.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60d6160640 1 -- 172.21.15.133:0/2525429351 shutdown_connections 2024-04-25T10:19:08.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.213+0000 7f60d6160640 1 -- 172.21.15.133:0/2525429351 wait complete. 2024-04-25T10:19:08.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60d6160640 1 Processor -- start 2024-04-25T10:19:08.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60d6160640 1 -- start start 2024-04-25T10:19:08.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60d6160640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60b80039c0 0x7f60d00b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60d6160640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ad360 0x7f60d0148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60d6160640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d01529b0 0x7f60d0152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60d6160640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d00725a0 con 0x7f60d01529b0 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60d6160640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d0072420 con 0x7f60b80039c0 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60d6160640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d0072720 con 0x7f60d00ad360 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d01529b0 0x7f60d0152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60cffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d01529b0 0x7f60d0152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46630/0 (socket says 172.21.15.133:46630) 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60cffff640 1 -- 172.21.15.133:0/855355796 learned_addr learned my addr 172.21.15.133:0/855355796 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60cf7fe640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60b80039c0 0x7f60d00b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60ceffd640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ad360 0x7f60d0148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60cffff640 1 -- 172.21.15.133:0/855355796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ad360 msgr2=0x7f60d0148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60cffff640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ad360 0x7f60d0148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60cffff640 1 -- 172.21.15.133:0/855355796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60b80039c0 msgr2=0x7f60d00b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60cffff640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60b80039c0 0x7f60d00b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60cffff640 1 -- 172.21.15.133:0/855355796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d01494a0 con 0x7f60d01529b0 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60cffff640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d01529b0 0x7f60d0152e40 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 crypto rx=0x7f60d0070ef0 tx=0x7f60c80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60b7fff640 1 -- 172.21.15.133:0/855355796 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c8011e10 con 0x7f60d01529b0 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60b7fff640 1 -- 172.21.15.133:0/855355796 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c801b070 con 0x7f60d01529b0 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60d6160640 1 -- 172.21.15.133:0/855355796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d00adad0 con 0x7f60d01529b0 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60b7fff640 1 -- 172.21.15.133:0/855355796 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c80039a0 con 0x7f60d01529b0 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60d6160640 1 -- 172.21.15.133:0/855355796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d00ae090 con 0x7f60d01529b0 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60d6160640 1 -- 172.21.15.133:0/855355796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d0069000 con 0x7f60d01529b0 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60b7fff640 1 -- 172.21.15.133:0/855355796 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60c8003b40 con 0x7f60d01529b0 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.217+0000 7f60b7fff640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60bc066f60 0x7f60bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.221+0000 7f60b7fff640 1 -- 172.21.15.133:0/855355796 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f60c8085a70 con 0x7f60d01529b0 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.221+0000 7f60cf7fe640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60bc066f60 0x7f60bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.221+0000 7f60cf7fe640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60bc066f60 0x7f60bc069420 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f60c00ae000 tx=0x7f60c00046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:08.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.225+0000 7f60b7fff640 1 -- 172.21.15.133:0/855355796 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60c8025020 con 0x7f60d01529b0 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:08.385+0000 7f60d6160640 1 -- 172.21.15.133:0/855355796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f60d014f4b0 con 0x7f60d01529b0 2024-04-25T10:19:09.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.185+0000 7f60b7fff640 1 -- 172.21.15.133:0/855355796 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v200) v1 ==== 181+0+0 (secure 0 0 0) 0x7f60c8056860 con 0x7f60d01529b0 2024-04-25T10:19:09.186 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:19:09.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 -- 172.21.15.133:0/855355796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60bc066f60 msgr2=0x7f60bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60bc066f60 0x7f60bc069420 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f60c00ae000 tx=0x7f60c00046d0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 -- 172.21.15.133:0/855355796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d01529b0 msgr2=0x7f60d0152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d01529b0 0x7f60d0152e40 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 crypto rx=0x7f60d0070ef0 tx=0x7f60c80115a0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 -- 172.21.15.133:0/855355796 shutdown_connections 2024-04-25T10:19:09.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f60bc066f60 0x7f60bc069420 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60d01529b0 0x7f60d0152e40 unknown :-1 s=CLOSED pgs=3027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f60d00ad360 0x7f60d0148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 --2- 172.21.15.133:0/855355796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f60b80039c0 0x7f60d00b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 -- 172.21.15.133:0/855355796 >> 172.21.15.133:0/855355796 conn(0x7f60d006d370 msgr2=0x7f60d00ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 -- 172.21.15.133:0/855355796 shutdown_connections 2024-04-25T10:19:09.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.189+0000 7f60d6160640 1 -- 172.21.15.133:0/855355796 wait complete. 2024-04-25T10:19:09.218 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:19:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.341+0000 7fb4cc95f640 1 Processor -- start 2024-04-25T10:19:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.341+0000 7fb4cc95f640 1 -- start start 2024-04-25T10:19:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.341+0000 7fb4cc95f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c80b4940 0x7fb4c80b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.341+0000 7fb4cc95f640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4c815c420 0x7fb4c80cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.341+0000 7fb4cc95f640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c80b5260 con 0x7fb4c8071cb0 2024-04-25T10:19:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.341+0000 7fb4cc95f640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c8072a90 con 0x7fb4c815c420 2024-04-25T10:19:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.341+0000 7fb4cc95f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c80a85d0 con 0x7fb4c80b4940 2024-04-25T10:19:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.345+0000 7fb4c659c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c80b4940 0x7fb4c80b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.345+0000 7fb4c6d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb4c815c420 0x7fb4c80cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.345+0000 7fb4c659c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c80b4940 0x7fb4c80b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:52228/0 (socket says 172.21.15.133:52228) 2024-04-25T10:19:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.345+0000 7fb4c659c640 1 -- 172.21.15.133:0/3791691367 learned_addr learned my addr 172.21.15.133:0/3791691367 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.345+0000 7fb4bdd9b640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fb4c8071cb0 0x7fb4c815bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:34196/0 (socket says 172.21.15.133:34196) 2024-04-25T10:19:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.345+0000 7fb4c6d9d640 1 -- 172.21.15.133:0/3791691367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c80b4940 msgr2=0x7fb4c80b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.345+0000 7fb4c6d9d640 1 --2- 172.21.15.133:0/3791691367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c80b4940 0x7fb4c80b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.345+0000 7fb4c6d9d640 1 -- 172.21.15.133:0/3791691367 >> v1:172.21.15.162:6789/0 conn(0x7fb4c8071cb0 legacy=0x7fb4c815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.345+0000 7fb4c6d9d640 1 -- 172.21.15.133:0/3791691367 --> v2:172.21.15.162:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c815c9b0 con 0x7fb4c815c420 2024-04-25T10:19:09.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.345+0000 7fb4c659c640 1 --2- 172.21.15.133:0/3791691367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c80b4940 0x7fb4c80b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:09.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.345+0000 7fb4c6d9d640 1 --2- 172.21.15.133:0/3791691367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4c815c420 0x7fb4c80cc010 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009d90 tx=0x7fb4c001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=306ec6402f78c37a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4c5d9b640 1 -- 172.21.15.133:0/3791691367 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00b2070 con 0x7fb4c815c420 2024-04-25T10:19:09.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4c5d9b640 1 -- 172.21.15.133:0/3791691367 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c00a5a00 con 0x7fb4c815c420 2024-04-25T10:19:09.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4c5d9b640 1 -- 172.21.15.133:0/3791691367 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c00a5dd0 con 0x7fb4c815c420 2024-04-25T10:19:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3791691367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4c815c420 msgr2=0x7fb4c80cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 --2- 172.21.15.133:0/3791691367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4c815c420 0x7fb4c80cc010 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009d90 tx=0x7fb4c001c940 comp rx=0 tx=0).stop 2024-04-25T10:19:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3791691367 shutdown_connections 2024-04-25T10:19:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 --2- 172.21.15.133:0/3791691367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4c815c420 0x7fb4c80cc010 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 --2- 172.21.15.133:0/3791691367 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c80b4940 0x7fb4c80b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3791691367 >> 172.21.15.133:0/3791691367 conn(0x7fb4c806d370 msgr2=0x7fb4c806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3791691367 shutdown_connections 2024-04-25T10:19:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3791691367 wait complete. 2024-04-25T10:19:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 Processor -- start 2024-04-25T10:19:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 -- start start 2024-04-25T10:19:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c8071cb0 0x7fb4c813d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4c80b4940 0x7fb4c815dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4c815c420 0x7fb4c8158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c80cc960 con 0x7fb4c80b4940 2024-04-25T10:19:09.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c8071790 con 0x7fb4c815c420 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4cc95f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c80ccae0 con 0x7fb4c8071cb0 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4bdd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4c80b4940 0x7fb4c815dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4bdd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4c80b4940 0x7fb4c815dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46642/0 (socket says 172.21.15.133:46642) 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4bdd9b640 1 -- 172.21.15.133:0/3985132315 learned_addr learned my addr 172.21.15.133:0/3985132315 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4c659c640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c8071cb0 0x7fb4c813d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.349+0000 7fb4c6d9d640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4c815c420 0x7fb4c8158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bdd9b640 1 -- 172.21.15.133:0/3985132315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c8071cb0 msgr2=0x7fb4c813d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bdd9b640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c8071cb0 0x7fb4c813d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bdd9b640 1 -- 172.21.15.133:0/3985132315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4c815c420 msgr2=0x7fb4c8158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bdd9b640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4c815c420 0x7fb4c8158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bdd9b640 1 -- 172.21.15.133:0/3985132315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c813cb50 con 0x7fb4c80b4940 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bdd9b640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4c80b4940 0x7fb4c815dfe0 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000c420 tx=0x7fb4b000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bf7fe640 1 -- 172.21.15.133:0/3985132315 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00074e0 con 0x7fb4c80b4940 2024-04-25T10:19:09.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bf7fe640 1 -- 172.21.15.133:0/3985132315 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b0007680 con 0x7fb4c80b4940 2024-04-25T10:19:09.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bf7fe640 1 -- 172.21.15.133:0/3985132315 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0004a30 con 0x7fb4c80b4940 2024-04-25T10:19:09.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3985132315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c8140a70 con 0x7fb4c80b4940 2024-04-25T10:19:09.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3985132315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c8141030 con 0x7fb4c80b4940 2024-04-25T10:19:09.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3985132315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c8069000 con 0x7fb4c80b4940 2024-04-25T10:19:09.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bf7fe640 1 -- 172.21.15.133:0/3985132315 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4b001a020 con 0x7fb4c80b4940 2024-04-25T10:19:09.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bf7fe640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb4a8066f60 0x7fb4a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4bf7fe640 1 -- 172.21.15.133:0/3985132315 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5835+0+0 (secure 0 0 0) 0x7fb4b0084f70 con 0x7fb4c80b4940 2024-04-25T10:19:09.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4c659c640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb4a8066f60 0x7fb4a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.353+0000 7fb4c659c640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb4a8066f60 0x7fb4a8069420 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009af0 tx=0x7fb4b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.357+0000 7fb4bf7fe640 1 -- 172.21.15.133:0/3985132315 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4b0021070 con 0x7fb4c80b4940 2024-04-25T10:19:09.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:09.517+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3985132315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb4c81416f0 con 0x7fb4c80b4940 2024-04-25T10:19:10.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.213+0000 7fb4bf7fe640 1 -- 172.21.15.133:0/3985132315 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v201) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb4b0056070 con 0x7fb4c80b4940 2024-04-25T10:19:10.215 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-04-25T10:19:10.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.217+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3985132315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb4a8066f60 msgr2=0x7fb4a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.217+0000 7fb4cc95f640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb4a8066f60 0x7fb4a8069420 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009af0 tx=0x7fb4b8008040 comp rx=0 tx=0).stop 2024-04-25T10:19:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.217+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3985132315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4c80b4940 msgr2=0x7fb4c815dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.217+0000 7fb4cc95f640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4c80b4940 0x7fb4c815dfe0 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000c420 tx=0x7fb4b000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.217+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3985132315 shutdown_connections 2024-04-25T10:19:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.217+0000 7fb4cc95f640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4c815c420 0x7fb4c8158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.217+0000 7fb4cc95f640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb4a8066f60 0x7fb4a8069420 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.217+0000 7fb4cc95f640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4c80b4940 0x7fb4c815dfe0 unknown :-1 s=CLOSED pgs=3028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.217+0000 7fb4cc95f640 1 --2- 172.21.15.133:0/3985132315 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb4c8071cb0 0x7fb4c813d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.217+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3985132315 >> 172.21.15.133:0/3985132315 conn(0x7fb4c806d370 msgr2=0x7fb4c80b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:10.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.217+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3985132315 shutdown_connections 2024-04-25T10:19:10.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.221+0000 7fb4cc95f640 1 -- 172.21.15.133:0/3985132315 wait complete. 2024-04-25T10:19:10.247 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_hack_allow_loading_invalid_metadata 2024-04-25T10:19:10.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.369+0000 7f495a19f640 1 Processor -- start 2024-04-25T10:19:10.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.369+0000 7f495a19f640 1 -- start start 2024-04-25T10:19:10.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.369+0000 7f495a19f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4954145170 0x7f495406df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.369+0000 7f495a19f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495406e4d0 0x7f49540cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.369+0000 7f495a19f640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49540ce2e0 con 0x7f49540cdd60 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.369+0000 7f495a19f640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495406db00 con 0x7f4954145170 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.369+0000 7f495a19f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4954145550 con 0x7f495406e4d0 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f49537fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4954145170 0x7f495406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f4953fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f49540cdd60 0x7f495415ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40956/0 (socket says 172.21.15.133:40956) 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f4953fff640 1 -- 172.21.15.133:0/2193989783 learned_addr learned my addr 172.21.15.133:0/2193989783 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f4952ffd640 1 --2- 172.21.15.133:0/2193989783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495406e4d0 0x7f49540cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f49527fc640 1 -- 172.21.15.133:0/2193989783 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924252263 0 0) 0x7f49540ce2e0 con 0x7f49540cdd60 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f49527fc640 1 -- 172.21.15.133:0/2193989783 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f493c003530 con 0x7f49540cdd60 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f49537fe640 1 -- 172.21.15.133:0/2193989783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495406e4d0 msgr2=0x7f49540cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f49537fe640 1 --2- 172.21.15.133:0/2193989783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495406e4d0 0x7f49540cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f49537fe640 1 -- 172.21.15.133:0/2193989783 >> v1:172.21.15.133:6789/0 conn(0x7f49540cdd60 legacy=0x7f495415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f49537fe640 1 -- 172.21.15.133:0/2193989783 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495406d4b0 con 0x7f4954145170 2024-04-25T10:19:10.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f49537fe640 1 --2- 172.21.15.133:0/2193989783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4954145170 0x7f495406df90 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f49440097c0 tx=0x7f494401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c496f54493284d71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:10.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f49527fc640 1 -- 172.21.15.133:0/2193989783 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49440b0070 con 0x7f4954145170 2024-04-25T10:19:10.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f49527fc640 1 -- 172.21.15.133:0/2193989783 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49440ab600 con 0x7f4954145170 2024-04-25T10:19:10.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f49527fc640 1 -- 172.21.15.133:0/2193989783 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49440ab9d0 con 0x7f4954145170 2024-04-25T10:19:10.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f495a19f640 1 -- 172.21.15.133:0/2193989783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4954145170 msgr2=0x7f495406df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f495a19f640 1 --2- 172.21.15.133:0/2193989783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4954145170 0x7f495406df90 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f49440097c0 tx=0x7f494401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f495a19f640 1 -- 172.21.15.133:0/2193989783 shutdown_connections 2024-04-25T10:19:10.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f495a19f640 1 --2- 172.21.15.133:0/2193989783 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495406e4d0 0x7f49540cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f495a19f640 1 --2- 172.21.15.133:0/2193989783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4954145170 0x7f495406df90 unknown :-1 s=CLOSED pgs=3030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f495a19f640 1 -- 172.21.15.133:0/2193989783 >> 172.21.15.133:0/2193989783 conn(0x7f4954066f70 msgr2=0x7f495406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:10.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f495a19f640 1 -- 172.21.15.133:0/2193989783 shutdown_connections 2024-04-25T10:19:10.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.373+0000 7f495a19f640 1 -- 172.21.15.133:0/2193989783 wait complete. 2024-04-25T10:19:10.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f495a19f640 1 Processor -- start 2024-04-25T10:19:10.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f495a19f640 1 -- start start 2024-04-25T10:19:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f495a19f640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495406e4d0 0x7f49540b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f495a19f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49540cdd60 0x7f4954135b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f495a19f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4954145170 0x7f495413a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f495a19f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4954168490 con 0x7f49540cdd60 2024-04-25T10:19:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f495a19f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495415d7f0 con 0x7f4954145170 2024-04-25T10:19:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f495a19f640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4954168610 con 0x7f495406e4d0 2024-04-25T10:19:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49540cdd60 0x7f4954135b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f4952ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49540cdd60 0x7f4954135b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46670/0 (socket says 172.21.15.133:46670) 2024-04-25T10:19:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f4952ffd640 1 -- 172.21.15.133:0/2835264287 learned_addr learned my addr 172.21.15.133:0/2835264287 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f49537fe640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495406e4d0 0x7f49540b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f4953fff640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4954145170 0x7f495413a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f4952ffd640 1 -- 172.21.15.133:0/2835264287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495406e4d0 msgr2=0x7f49540b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f4952ffd640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495406e4d0 0x7f49540b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f4952ffd640 1 -- 172.21.15.133:0/2835264287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4954145170 msgr2=0x7f495413a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f4952ffd640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4954145170 0x7f495413a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f4952ffd640 1 -- 172.21.15.133:0/2835264287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495413a7e0 con 0x7f49540cdd60 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f49537fe640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495406e4d0 0x7f49540b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f4953fff640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4954145170 0x7f495413a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f4952ffd640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49540cdd60 0x7f4954135b10 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f494800d7d0 tx=0x7f494800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f493bfff640 1 -- 172.21.15.133:0/2835264287 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800b660 con 0x7f49540cdd60 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f493bfff640 1 -- 172.21.15.133:0/2835264287 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4948019070 con 0x7f49540cdd60 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f495a19f640 1 -- 172.21.15.133:0/2835264287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495413ac30 con 0x7f49540cdd60 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f493bfff640 1 -- 172.21.15.133:0/2835264287 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800ba20 con 0x7f49540cdd60 2024-04-25T10:19:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f495a19f640 1 -- 172.21.15.133:0/2835264287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495413b1f0 con 0x7f49540cdd60 2024-04-25T10:19:10.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.377+0000 7f495a19f640 1 -- 172.21.15.133:0/2835264287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4954068e80 con 0x7f49540cdd60 2024-04-25T10:19:10.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.381+0000 7f493bfff640 1 -- 172.21.15.133:0/2835264287 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f494800bbc0 con 0x7f49540cdd60 2024-04-25T10:19:10.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.381+0000 7f493bfff640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4940066f60 0x7f4940069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.381+0000 7f493bfff640 1 -- 172.21.15.133:0/2835264287 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4948085d30 con 0x7f49540cdd60 2024-04-25T10:19:10.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.381+0000 7f49537fe640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4940066f60 0x7f4940069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.381+0000 7f49537fe640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4940066f60 0x7f4940069420 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f49440ad000 tx=0x7f4944008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:10.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.385+0000 7f493bfff640 1 -- 172.21.15.133:0/2835264287 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4948056f40 con 0x7f49540cdd60 2024-04-25T10:19:10.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.541+0000 7f495a19f640 1 -- 172.21.15.133:0/2835264287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"} v 0) v1 -- 0x7f495415e3e0 con 0x7f49540cdd60 2024-04-25T10:19:10.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.545+0000 7f493bfff640 1 -- 172.21.15.133:0/2835264287 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"}]=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4948056840 con 0x7f49540cdd60 2024-04-25T10:19:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 -- 172.21.15.133:0/2835264287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4940066f60 msgr2=0x7f4940069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4940066f60 0x7f4940069420 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f49440ad000 tx=0x7f4944008040 comp rx=0 tx=0).stop 2024-04-25T10:19:10.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 -- 172.21.15.133:0/2835264287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49540cdd60 msgr2=0x7f4954135b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49540cdd60 0x7f4954135b10 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f494800d7d0 tx=0x7f494800dc90 comp rx=0 tx=0).stop 2024-04-25T10:19:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 -- 172.21.15.133:0/2835264287 shutdown_connections 2024-04-25T10:19:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4954145170 0x7f495413a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4940066f60 0x7f4940069420 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49540cdd60 0x7f4954135b10 unknown :-1 s=CLOSED pgs=3031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 --2- 172.21.15.133:0/2835264287 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f495406e4d0 0x7f49540b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 -- 172.21.15.133:0/2835264287 >> 172.21.15.133:0/2835264287 conn(0x7f4954066f70 msgr2=0x7f49540b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 -- 172.21.15.133:0/2835264287 shutdown_connections 2024-04-25T10:19:10.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.549+0000 7f495a19f640 1 -- 172.21.15.133:0/2835264287 wait complete. 2024-04-25T10:19:10.568 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:19:10.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.685+0000 7fe9c2057640 1 Processor -- start 2024-04-25T10:19:10.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.685+0000 7fe9c2057640 1 -- start start 2024-04-25T10:19:10.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9c2057640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc0ce600 0x7fe9bc0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9c2057640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc0d24c0 con 0x7fe9bc15d9a0 2024-04-25T10:19:10.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9c2057640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc15ebd0 con 0x7fe9bc0d1920 2024-04-25T10:19:10.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9c2057640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0cefa0 con 0x7fe9bc0ce600 2024-04-25T10:19:10.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9bb7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe9bc15d9a0 0x7fe9bc0d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40972/0 (socket says 172.21.15.133:40972) 2024-04-25T10:19:10.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9bb7fe640 1 -- 172.21.15.133:0/2881368018 learned_addr learned my addr 172.21.15.133:0/2881368018 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:10.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9baffd640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc0ce600 0x7fe9bc0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309678820 0 0) 0x7fe9bc0d24c0 con 0x7fe9bc15d9a0 2024-04-25T10:19:10.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a4003530 con 0x7fe9bc15d9a0 2024-04-25T10:19:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3530328666 0 0) 0x7fe9a4003530 con 0x7fe9bc15d9a0 2024-04-25T10:19:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc0ce600 msgr2=0x7fe9bc0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:10.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc0ce600 0x7fe9bc0cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 >> v1:172.21.15.162:6789/0 conn(0x7fe9bc0d1920 legacy=0x7fe9bc0d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc15eeb0 con 0x7fe9bc15d9a0 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fe9b0001680 con 0x7fe9bc15d9a0 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 >> v1:172.21.15.133:6789/0 conn(0x7fe9bc15d9a0 legacy=0x7fe9bc0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc0ce600 0x7fe9a40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9a4003fc0 0x7fe9a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a4004d50 0x7fe9a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a400bb90 con 0x7fe9a4003fc0 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a400bd40 con 0x7fe9a4004d50 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a400bf40 con 0x7fe9bc0ce600 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9baffd640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9a4003fc0 0x7fe9a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9bb7fe640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc0ce600 0x7fe9a40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9bbfff640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a4004d50 0x7fe9a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9baffd640 1 -- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc0ce600 msgr2=0x7fe9a40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:10.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9baffd640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc0ce600 0x7fe9a40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9baffd640 1 -- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a4004d50 msgr2=0x7fe9a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:10.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9baffd640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a4004d50 0x7fe9a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9baffd640 1 -- 172.21.15.133:0/2881368018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a4003530 con 0x7fe9a4003fc0 2024-04-25T10:19:10.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.689+0000 7fe9bbfff640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a4004d50 0x7fe9a40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:10.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9baffd640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9a4003fc0 0x7fe9a4004810 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009ae0 tx=0x7fe9ac01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8886e5927e7aab6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:10.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0b2070 con 0x7fe9a4003fc0 2024-04-25T10:19:10.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac0a5520 con 0x7fe9a4003fc0 2024-04-25T10:19:10.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9ba7fc640 1 -- 172.21.15.133:0/2881368018 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0096b0 con 0x7fe9a4003fc0 2024-04-25T10:19:10.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 -- 172.21.15.133:0/2881368018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9a4003fc0 msgr2=0x7fe9a4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 --2- 172.21.15.133:0/2881368018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9a4003fc0 0x7fe9a4004810 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009ae0 tx=0x7fe9ac01ca40 comp rx=0 tx=0).stop 2024-04-25T10:19:10.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9bbfff640 1 -- 172.21.15.133:0/2881368018 reap_dead start 2024-04-25T10:19:10.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 -- 172.21.15.133:0/2881368018 shutdown_connections 2024-04-25T10:19:10.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 -- 172.21.15.133:0/2881368018 >> 172.21.15.133:0/2881368018 conn(0x7fe9bc06bf80 msgr2=0x7fe9bc06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:10.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 -- 172.21.15.133:0/2881368018 shutdown_connections 2024-04-25T10:19:10.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 -- 172.21.15.133:0/2881368018 wait complete. 2024-04-25T10:19:10.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 Processor -- start 2024-04-25T10:19:10.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 -- start start 2024-04-25T10:19:10.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a4003fc0 0x7fe9bc1f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc1596a0 0x7fe9bc0b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9bc0b8740 0x7fe9bc15f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0b9820 con 0x7fe9bc0b8740 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0b96a0 con 0x7fe9a4003fc0 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9c2057640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc15e590 con 0x7fe9bc1596a0 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9bc0b8740 0x7fe9bc15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9bc0b8740 0x7fe9bc15f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46692/0 (socket says 172.21.15.133:46692) 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9bbfff640 1 -- 172.21.15.133:0/674561221 learned_addr learned my addr 172.21.15.133:0/674561221 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9bb7fe640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a4003fc0 0x7fe9bc1f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9baffd640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc1596a0 0x7fe9bc0b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9bbfff640 1 -- 172.21.15.133:0/674561221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc1596a0 msgr2=0x7fe9bc0b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9bbfff640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc1596a0 0x7fe9bc0b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9bbfff640 1 -- 172.21.15.133:0/674561221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a4003fc0 msgr2=0x7fe9bc1f6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9bbfff640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a4003fc0 0x7fe9bc1f6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9bbfff640 1 -- 172.21.15.133:0/674561221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc0cc870 con 0x7fe9bc0b8740 2024-04-25T10:19:10.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9baffd640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc1596a0 0x7fe9bc0b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:10.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.693+0000 7fe9bb7fe640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a4003fc0 0x7fe9bc1f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:10.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9bbfff640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9bc0b8740 0x7fe9bc15f190 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc08bdd0 tx=0x7fe9b400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:10.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9a3fff640 1 -- 172.21.15.133:0/674561221 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4006d60 con 0x7fe9bc0b8740 2024-04-25T10:19:10.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9a3fff640 1 -- 172.21.15.133:0/674561221 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b4003980 con 0x7fe9bc0b8740 2024-04-25T10:19:10.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9c2057640 1 -- 172.21.15.133:0/674561221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc0ccb50 con 0x7fe9bc0b8740 2024-04-25T10:19:10.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9a3fff640 1 -- 172.21.15.133:0/674561221 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4003d70 con 0x7fe9bc0b8740 2024-04-25T10:19:10.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9c2057640 1 -- 172.21.15.133:0/674561221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc0caa10 con 0x7fe9bc0b8740 2024-04-25T10:19:10.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9c2057640 1 -- 172.21.15.133:0/674561221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9bc0cac10 con 0x7fe9bc0b8740 2024-04-25T10:19:10.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9a3fff640 1 -- 172.21.15.133:0/674561221 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9b401b020 con 0x7fe9bc0b8740 2024-04-25T10:19:10.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9a3fff640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9a8066f60 0x7fe9a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9bb7fe640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9a8066f60 0x7fe9a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9a3fff640 1 -- 172.21.15.133:0/674561221 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe9b4085380 con 0x7fe9bc0b8740 2024-04-25T10:19:10.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.697+0000 7fe9bb7fe640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9a8066f60 0x7fe9a8069420 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0001060 tx=0x7fe9b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:10.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.701+0000 7fe9a3fff640 1 -- 172.21.15.133:0/674561221 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9bc0cac10 con 0x7fe9bc0b8740 2024-04-25T10:19:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.853+0000 7fe9c2057640 1 -- 172.21.15.133:0/674561221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe9bc068e80 con 0x7fe9bc0b8740 2024-04-25T10:19:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9a3fff640 1 -- 172.21.15.133:0/674561221 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe9b4022070 con 0x7fe9bc0b8740 2024-04-25T10:19:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 -- 172.21.15.133:0/674561221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9a8066f60 msgr2=0x7fe9a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9a8066f60 0x7fe9a8069420 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0001060 tx=0x7fe9b0009040 comp rx=0 tx=0).stop 2024-04-25T10:19:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 -- 172.21.15.133:0/674561221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9bc0b8740 msgr2=0x7fe9bc15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9bc0b8740 0x7fe9bc15f190 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc08bdd0 tx=0x7fe9b400dc00 comp rx=0 tx=0).stop 2024-04-25T10:19:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 -- 172.21.15.133:0/674561221 shutdown_connections 2024-04-25T10:19:10.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fe9a8066f60 0x7fe9a8069420 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9bc0b8740 0x7fe9bc15f190 unknown :-1 s=CLOSED pgs=3034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fe9bc1596a0 0x7fe9bc0b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 --2- 172.21.15.133:0/674561221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe9a4003fc0 0x7fe9bc1f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 -- 172.21.15.133:0/674561221 >> 172.21.15.133:0/674561221 conn(0x7fe9bc06bf80 msgr2=0x7fe9bc0cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:10.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 -- 172.21.15.133:0/674561221 shutdown_connections 2024-04-25T10:19:10.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.861+0000 7fe9c2057640 1 -- 172.21.15.133:0/674561221 wait complete. 2024-04-25T10:19:10.880 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:19:10.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b9141640 1 Processor -- start 2024-04-25T10:19:10.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b9141640 1 -- start start 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b9141640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f46b415c9a0 0x7f46b40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b9141640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f46b40b1f70 0x7f46b406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b9141640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b40c61d0 con 0x7f46b40ce330 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b9141640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b40c64b0 con 0x7f46b415c9a0 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b9141640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b406d4f0 con 0x7f46b40b1f70 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b2d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f46b415c9a0 0x7f46b40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b2d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f46b415c9a0 0x7f46b40c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46708/0 (socket says 172.21.15.133:46708) 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b2d9d640 1 -- 172.21.15.133:0/3876222147 learned_addr learned my addr 172.21.15.133:0/3876222147 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46abfff640 1 --2- 172.21.15.133:0/3876222147 >> v2:172.21.15.162:3300/0 conn(0x7f46b40b1f70 0x7f46b406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b2d9d640 1 -- 172.21.15.133:0/3876222147 >> v2:172.21.15.162:3300/0 conn(0x7f46b40b1f70 msgr2=0x7f46b406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b2d9d640 1 --2- 172.21.15.133:0/3876222147 >> v2:172.21.15.162:3300/0 conn(0x7f46b40b1f70 0x7f46b406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b2d9d640 1 -- 172.21.15.133:0/3876222147 >> v1:172.21.15.162:6789/0 conn(0x7f46b40ce330 legacy=0x7f46b415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:10.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46b2d9d640 1 -- 172.21.15.133:0/3876222147 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b406d7d0 con 0x7f46b415c9a0 2024-04-25T10:19:10.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.985+0000 7f46abfff640 1 --2- 172.21.15.133:0/3876222147 >> v2:172.21.15.162:3300/0 conn(0x7f46b40b1f70 0x7f46b406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:10.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b2d9d640 1 --2- 172.21.15.133:0/3876222147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b415c9a0 0x7f46b40c5980 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7f46b4072670 tx=0x7f46a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f27f3cc335bb52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:10.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b259c640 1 -- 172.21.15.133:0/3876222147 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a40b2070 con 0x7f46b415c9a0 2024-04-25T10:19:10.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b259c640 1 -- 172.21.15.133:0/3876222147 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a40ad640 con 0x7f46b415c9a0 2024-04-25T10:19:10.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b259c640 1 -- 172.21.15.133:0/3876222147 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a40ada10 con 0x7f46b415c9a0 2024-04-25T10:19:10.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 -- 172.21.15.133:0/3876222147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b415c9a0 msgr2=0x7f46b40c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 --2- 172.21.15.133:0/3876222147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b415c9a0 0x7f46b40c5980 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7f46b4072670 tx=0x7f46a401c480 comp rx=0 tx=0).stop 2024-04-25T10:19:10.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 -- 172.21.15.133:0/3876222147 shutdown_connections 2024-04-25T10:19:10.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 --2- 172.21.15.133:0/3876222147 >> v2:172.21.15.162:3300/0 conn(0x7f46b40b1f70 0x7f46b406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 --2- 172.21.15.133:0/3876222147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b415c9a0 0x7f46b40c5980 unknown :-1 s=CLOSED pgs=3035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 -- 172.21.15.133:0/3876222147 >> 172.21.15.133:0/3876222147 conn(0x7f46b4067000 msgr2=0x7f46b406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:10.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 -- 172.21.15.133:0/3876222147 shutdown_connections 2024-04-25T10:19:10.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 -- 172.21.15.133:0/3876222147 wait complete. 2024-04-25T10:19:10.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 Processor -- start 2024-04-25T10:19:10.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 -- start start 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46b40b1f70 0x7f46b414cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46b40ce330 0x7f46b415e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b415c9a0 0x7f46b4151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b415cd80 con 0x7f46b415c9a0 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b406e230 con 0x7f46b40ce330 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.989+0000 7f46b9141640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b415cf00 con 0x7f46b40b1f70 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b415c9a0 0x7f46b4151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b359e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b415c9a0 0x7f46b4151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46712/0 (socket says 172.21.15.133:46712) 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b359e640 1 -- 172.21.15.133:0/3661425132 learned_addr learned my addr 172.21.15.133:0/3661425132 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46abfff640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46b40ce330 0x7f46b415e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b2d9d640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46b40b1f70 0x7f46b414cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b359e640 1 -- 172.21.15.133:0/3661425132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46b40b1f70 msgr2=0x7f46b414cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b359e640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46b40b1f70 0x7f46b414cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b359e640 1 -- 172.21.15.133:0/3661425132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46b40ce330 msgr2=0x7f46b415e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:10.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b359e640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46b40ce330 0x7f46b415e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:10.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b359e640 1 -- 172.21.15.133:0/3661425132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b415eb70 con 0x7f46b415c9a0 2024-04-25T10:19:10.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b359e640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b415c9a0 0x7f46b4151a40 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b990 tx=0x7f46ac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:10.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46ab7fe640 1 -- 172.21.15.133:0/3661425132 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac019070 con 0x7f46b415c9a0 2024-04-25T10:19:10.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46ab7fe640 1 -- 172.21.15.133:0/3661425132 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac00cba0 con 0x7f46b415c9a0 2024-04-25T10:19:10.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46ab7fe640 1 -- 172.21.15.133:0/3661425132 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac014440 con 0x7f46b415c9a0 2024-04-25T10:19:10.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b9141640 1 -- 172.21.15.133:0/3661425132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b4149820 con 0x7f46b415c9a0 2024-04-25T10:19:10.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b9141640 1 -- 172.21.15.133:0/3661425132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b414e260 con 0x7f46b415c9a0 2024-04-25T10:19:10.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b9141640 1 -- 172.21.15.133:0/3661425132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b406bc20 con 0x7f46b415c9a0 2024-04-25T10:19:10.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46ab7fe640 1 -- 172.21.15.133:0/3661425132 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46ac01c070 con 0x7f46b415c9a0 2024-04-25T10:19:10.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46ab7fe640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4694066f90 0x7f4694069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:10.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46ab7fe640 1 -- 172.21.15.133:0/3661425132 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f46ac084c30 con 0x7f46b415c9a0 2024-04-25T10:19:10.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b2d9d640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4694066f90 0x7f4694069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:10.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.993+0000 7f46b2d9d640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4694066f90 0x7f4694069450 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f46a40af000 tx=0x7f46a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:10.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:10.997+0000 7f46ab7fe640 1 -- 172.21.15.133:0/3661425132 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46ac055d60 con 0x7f46b415c9a0 2024-04-25T10:19:11.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.197+0000 7f46b9141640 1 -- 172.21.15.133:0/3661425132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7f46b4153050 con 0x7f46b415c9a0 2024-04-25T10:19:11.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.213+0000 7f46ab7fe640 1 -- 172.21.15.133:0/3661425132 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v773) v1 ==== 133+0+0 (secure 0 0 0) 0x7f46ac055660 con 0x7f46b415c9a0 2024-04-25T10:19:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 -- 172.21.15.133:0/3661425132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4694066f90 msgr2=0x7f4694069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4694066f90 0x7f4694069450 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f46a40af000 tx=0x7f46a4008040 comp rx=0 tx=0).stop 2024-04-25T10:19:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 -- 172.21.15.133:0/3661425132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b415c9a0 msgr2=0x7f46b4151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b415c9a0 0x7f46b4151a40 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b990 tx=0x7f46ac00be50 comp rx=0 tx=0).stop 2024-04-25T10:19:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 -- 172.21.15.133:0/3661425132 shutdown_connections 2024-04-25T10:19:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4694066f90 0x7f4694069450 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:11.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46b415c9a0 0x7f46b4151a40 unknown :-1 s=CLOSED pgs=3036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f46b40ce330 0x7f46b415e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 --2- 172.21.15.133:0/3661425132 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f46b40b1f70 0x7f46b414cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 -- 172.21.15.133:0/3661425132 >> 172.21.15.133:0/3661425132 conn(0x7f46b4067000 msgr2=0x7f46b40b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 -- 172.21.15.133:0/3661425132 shutdown_connections 2024-04-25T10:19:11.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:11.217+0000 7f46b9141640 1 -- 172.21.15.133:0/3661425132 wait complete. 2024-04-25T10:19:11.235 INFO:tasks.cephfs_test_runner:test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:19:11.235 INFO:tasks.cephfs_test_runner: 2024-04-25T10:19:11.235 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:19:11.235 INFO:tasks.cephfs_test_runner:Ran 13 tests in 985.655s 2024-04-25T10:19:11.235 INFO:tasks.cephfs_test_runner: 2024-04-25T10:19:11.235 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:19:11.236 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:19:11.245 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T10:19:11.252 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T10:19:11.253 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:19:11.253 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:11.258 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:19:11.259 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T10:19:11.259 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:11.305 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15957 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:19:11.305 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:19:11.305 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:11.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:11.312 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-04-25T10:19:11.312 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:11.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336512k,nr_inodes=4084128,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19920 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:11.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:11.359 DEBUG:teuthology.orchestra.run.smithi162:> ip netns list 2024-04-25T10:19:11.407 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:19:11.407 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:19:11.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:11.407 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:19:11.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:11.464 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:19:11.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:11.532 DEBUG:teuthology.orchestra.run.smithi162:> sudo ip link delete ceph-brx 2024-04-25T10:19:11.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:11.744 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-04-25T10:19:11.751 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:19:11.751 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:19:11.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:11.751 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:19:11.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:11.807 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:19:11.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:11.898 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-04-25T10:19:12.018 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:19:12.027 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:19:12.027 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:12.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f630ad6c640 1 Processor -- start 2024-04-25T10:19:12.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f630ad6c640 1 -- start start 2024-04-25T10:19:12.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f630ad6c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415db00 0x7f63040b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f630ad6c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f63040d0770 0x7f63040d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f630ad6c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63040ceb20 con 0x7f63041614c0 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f630ad6c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63040d1110 con 0x7f63040d0770 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f630ad6c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63040b7740 con 0x7f630415db00 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6309309640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f63041614c0 0x7f6304161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40988/0 (socket says 172.21.15.133:40988) 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6309309640 1 -- 172.21.15.133:0/3292966075 learned_addr learned my addr 172.21.15.133:0/3292966075 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6308b08640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415db00 0x7f63040b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051706815 0 0) 0x7f63040ceb20 con 0x7f63041614c0 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6303fff640 1 --2- 172.21.15.133:0/3292966075 >> v2:172.21.15.162:3300/0 conn(0x7f63040d0770 0x7f63040d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62f4003530 con 0x7f63041614c0 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2486950488 0 0) 0x7f62f4003530 con 0x7f63041614c0 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415db00 msgr2=0x7f63040b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415db00 0x7f63040b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 >> v2:172.21.15.162:3300/0 conn(0x7f63040d0770 msgr2=0x7f63040d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 --2- 172.21.15.133:0/3292966075 >> v2:172.21.15.162:3300/0 conn(0x7f63040d0770 0x7f63040d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63040b97f0 con 0x7f63041614c0 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f62fc001a00 con 0x7f63041614c0 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6308b08640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415db00 0x7f63040b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 >> v1:172.21.15.133:6789/0 conn(0x7f63041614c0 legacy=0x7f6304161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415db00 0x7f62f40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62f4003f00 0x7f62f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f4004d50 0x7f62f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63040b7740 con 0x7f62f4004d50 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40059a0 con 0x7f62f4003f00 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4005ba0 con 0x7f630415db00 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6309309640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f4004d50 0x7f62f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6308b08640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415db00 0x7f62f40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6303fff640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62f4003f00 0x7f62f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6309309640 1 -- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415db00 msgr2=0x7f62f40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6309309640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415db00 0x7f62f40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6309309640 1 -- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62f4003f00 msgr2=0x7f62f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6309309640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62f4003f00 0x7f62f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6309309640 1 -- 172.21.15.133:0/3292966075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f4003530 con 0x7f62f4004d50 2024-04-25T10:19:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.153+0000 7f6303fff640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62f4003f00 0x7f62f4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f6309309640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f4004d50 0x7f62f40051d0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00a8a0 tx=0x7f62fc01d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3ff9dcf15d2032 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc00a4c0 con 0x7f62f4004d50 2024-04-25T10:19:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc0a9020 con 0x7f62f4004d50 2024-04-25T10:19:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f63037fe640 1 -- 172.21.15.133:0/3292966075 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0072d0 con 0x7f62f4004d50 2024-04-25T10:19:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 -- 172.21.15.133:0/3292966075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f4004d50 msgr2=0x7f62f40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 --2- 172.21.15.133:0/3292966075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f4004d50 0x7f62f40051d0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00a8a0 tx=0x7f62fc01d3c0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f6309309640 1 -- 172.21.15.133:0/3292966075 reap_dead start 2024-04-25T10:19:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 -- 172.21.15.133:0/3292966075 shutdown_connections 2024-04-25T10:19:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 -- 172.21.15.133:0/3292966075 >> 172.21.15.133:0/3292966075 conn(0x7f63040727c0 msgr2=0x7f6304072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 -- 172.21.15.133:0/3292966075 shutdown_connections 2024-04-25T10:19:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 -- 172.21.15.133:0/3292966075 wait complete. 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 Processor -- start 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 -- start start 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62f4004d50 0x7f630415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415f850 0x7f63040c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630415fcb0 0x7f63040c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63040b7fe0 con 0x7f630415fcb0 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63040b7d00 con 0x7f62f4004d50 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f630ad6c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63040b82c0 con 0x7f630415f850 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f6309309640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630415fcb0 0x7f63040c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f6309309640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630415fcb0 0x7f63040c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46728/0 (socket says 172.21.15.133:46728) 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f6309309640 1 -- 172.21.15.133:0/70537058 learned_addr learned my addr 172.21.15.133:0/70537058 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f6308b08640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62f4004d50 0x7f630415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.157+0000 7f6303fff640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415f850 0x7f63040c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f6309309640 1 -- 172.21.15.133:0/70537058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415f850 msgr2=0x7f63040c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f6309309640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415f850 0x7f63040c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f6309309640 1 -- 172.21.15.133:0/70537058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62f4004d50 msgr2=0x7f630415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f6309309640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62f4004d50 0x7f630415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f6309309640 1 -- 172.21.15.133:0/70537058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63040cb4a0 con 0x7f630415fcb0 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f6309309640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630415fcb0 0x7f63040c9a60 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7f62fc007040 tx=0x7f62f4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f63017fa640 1 -- 172.21.15.133:0/70537058 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc002020 con 0x7f630415fcb0 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f63017fa640 1 -- 172.21.15.133:0/70537058 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc0074e0 con 0x7f630415fcb0 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f630ad6c640 1 -- 172.21.15.133:0/70537058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63040cc500 con 0x7f630415fcb0 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f63017fa640 1 -- 172.21.15.133:0/70537058 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0078b0 con 0x7f630415fcb0 2024-04-25T10:19:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f630ad6c640 1 -- 172.21.15.133:0/70537058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63040cba20 con 0x7f630415fcb0 2024-04-25T10:19:12.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f630ad6c640 1 -- 172.21.15.133:0/70537058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6304068ef0 con 0x7f630415fcb0 2024-04-25T10:19:12.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f63017fa640 1 -- 172.21.15.133:0/70537058 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62fc0af440 con 0x7f630415fcb0 2024-04-25T10:19:12.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f63017fa640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f60 0x7f62e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f63017fa640 1 -- 172.21.15.133:0/70537058 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f62fc11f830 con 0x7f630415fcb0 2024-04-25T10:19:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f6308b08640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f60 0x7f62e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.161+0000 7f6308b08640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f60 0x7f62e8069420 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f62f800e880 tx=0x7f62f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.165+0000 7f63017fa640 1 -- 172.21.15.133:0/70537058 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62fc0bf060 con 0x7f630415fcb0 2024-04-25T10:19:12.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.325+0000 7f630ad6c640 1 -- 172.21.15.133:0/70537058 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6304072d90 con 0x7f62e8066f60 2024-04-25T10:19:12.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.325+0000 7f63017fa640 1 -- 172.21.15.133:0/70537058 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13524 (secure 0 0 0) 0x7f6304072d90 con 0x7f62e8066f60 2024-04-25T10:19:12.327 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:12.328 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:19:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 -- 172.21.15.133:0/70537058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f60 msgr2=0x7f62e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f60 0x7f62e8069420 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f62f800e880 tx=0x7f62f8008040 comp rx=0 tx=0).stop 2024-04-25T10:19:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 -- 172.21.15.133:0/70537058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630415fcb0 msgr2=0x7f63040c9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630415fcb0 0x7f63040c9a60 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7f62fc007040 tx=0x7f62f4005840 comp rx=0 tx=0).stop 2024-04-25T10:19:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 -- 172.21.15.133:0/70537058 shutdown_connections 2024-04-25T10:19:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f62e8066f60 0x7f62e8069420 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f630415fcb0 0x7f63040c9a60 unknown :-1 s=CLOSED pgs=3039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f630415f850 0x7f63040c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 --2- 172.21.15.133:0/70537058 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f62f4004d50 0x7f630415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 -- 172.21.15.133:0/70537058 >> 172.21.15.133:0/70537058 conn(0x7f63040727c0 msgr2=0x7f63040b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:12.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 -- 172.21.15.133:0/70537058 shutdown_connections 2024-04-25T10:19:12.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.329+0000 7f630ad6c640 1 -- 172.21.15.133:0/70537058 wait complete. 2024-04-25T10:19:12.346 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":718,"stamp":"2024-04-25T10:19:12.038189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":586416,"kb_used_data":45508,"kb_used_omap":12,"kb_used_meta":540787,"kb_avail":754388304,"statfs":{"total":773094113280,"available":772493623296,"internally_reserved":0,"allocated":46600192,"data_stored":35894583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":553766449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12618969,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-101,"num_read_kb":-105,"num_write":-98,"num_write_kb":-12356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-5,"num_bytes_recovered":-35313,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001869"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":509,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-04-25T10:19:09.208823+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:19:09.208823+0000","last_peered":"2024-04-25T10:19:09.208823+0000","last_clean":"2024-04-25T10:19:09.208823+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:19:09.208823+0000","last_undegraded":"2024-04-25T10:19:09.208823+0000","last_fullsized":"2024-04-25T10:19:09.208823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575062,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78576,"kb_used_data":7140,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94293264,"statfs":{"total":96636764160,"available":96556302336,"internally_reserved":0,"allocated":7311360,"data_stored":5971063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575063,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73848,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297992,"statfs":{"total":96636764160,"available":96561143808,"internally_reserved":0,"allocated":2531328,"data_stored":1192268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73071046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575062,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51668,"kb_used_data":6600,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94320172,"statfs":{"total":96636764160,"available":96583856128,"internally_reserved":0,"allocated":6758400,"data_stored":5415336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575062,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96104,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275736,"statfs":{"total":96636764160,"available":96538353664,"internally_reserved":0,"allocated":2519040,"data_stored":1185809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93876,"kb_used_data":12712,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94277964,"statfs":{"total":96636764160,"available":96540635136,"internally_reserved":0,"allocated":13017088,"data_stored":11678028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83098055},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575062,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57912,"kb_used_data":4520,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313928,"statfs":{"total":96636764160,"available":96577462272,"internally_reserved":0,"allocated":4628480,"data_stored":3289420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53732,"kb_used_data":4504,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318108,"statfs":{"total":96636764160,"available":96581742592,"internally_reserved":0,"allocated":4612096,"data_stored":3282871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575063,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80700,"kb_used_data":5100,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291140,"statfs":{"total":96636764160,"available":96554127360,"internally_reserved":0,"allocated":5222400,"data_stored":3879788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:12.347 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff6a7ac640 1 Processor -- start 2024-04-25T10:19:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff6a7ac640 1 -- start start 2024-04-25T10:19:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff6a7ac640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7eff6415c9a0 0x7eff640c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff6a7ac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff640b1f70 0x7eff6406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff6a7ac640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff640ce330 0x7eff640ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff6a7ac640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff640cece0 con 0x7eff640ce330 2024-04-25T10:19:12.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff6a7ac640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff640c61d0 con 0x7eff6415c9a0 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff6a7ac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff640c64b0 con 0x7eff640b1f70 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff68d49640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff640ce330 0x7eff640ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff68d49640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff640ce330 0x7eff640ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46730/0 (socket says 172.21.15.133:46730) 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff68d49640 1 -- 172.21.15.133:0/1324086657 learned_addr learned my addr 172.21.15.133:0/1324086657 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff63fff640 1 --2- 172.21.15.133:0/1324086657 >> v2:172.21.15.162:3300/0 conn(0x7eff6415c9a0 0x7eff640c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff637fe640 1 --2- 172.21.15.133:0/1324086657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff640b1f70 0x7eff6406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff68d49640 1 -- 172.21.15.133:0/1324086657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff640b1f70 msgr2=0x7eff6406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff68d49640 1 --2- 172.21.15.133:0/1324086657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff640b1f70 0x7eff6406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff68d49640 1 -- 172.21.15.133:0/1324086657 >> v2:172.21.15.162:3300/0 conn(0x7eff6415c9a0 msgr2=0x7eff640c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff68d49640 1 --2- 172.21.15.133:0/1324086657 >> v2:172.21.15.162:3300/0 conn(0x7eff6415c9a0 0x7eff640c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff68d49640 1 -- 172.21.15.133:0/1324086657 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6406d4f0 con 0x7eff640ce330 2024-04-25T10:19:12.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.461+0000 7eff637fe640 1 --2- 172.21.15.133:0/1324086657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff640b1f70 0x7eff6406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:12.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff68d49640 1 --2- 172.21.15.133:0/1324086657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff640ce330 0x7eff640ce7a0 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7eff6406f5f0 tx=0x7eff5c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97bcc8b77774e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:12.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff62ffd640 1 -- 172.21.15.133:0/1324086657 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0a5750 con 0x7eff640ce330 2024-04-25T10:19:12.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff62ffd640 1 -- 172.21.15.133:0/1324086657 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c0a58f0 con 0x7eff640ce330 2024-04-25T10:19:12.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff62ffd640 1 -- 172.21.15.133:0/1324086657 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0a5cc0 con 0x7eff640ce330 2024-04-25T10:19:12.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1324086657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff640ce330 msgr2=0x7eff640ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 --2- 172.21.15.133:0/1324086657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff640ce330 0x7eff640ce7a0 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7eff6406f5f0 tx=0x7eff5c01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1324086657 shutdown_connections 2024-04-25T10:19:12.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 --2- 172.21.15.133:0/1324086657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff640ce330 0x7eff640ce7a0 unknown :-1 s=CLOSED pgs=3040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 --2- 172.21.15.133:0/1324086657 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff640b1f70 0x7eff6406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 --2- 172.21.15.133:0/1324086657 >> v2:172.21.15.162:3300/0 conn(0x7eff6415c9a0 0x7eff640c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1324086657 >> 172.21.15.133:0/1324086657 conn(0x7eff64067000 msgr2=0x7eff6406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1324086657 shutdown_connections 2024-04-25T10:19:12.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1324086657 wait complete. 2024-04-25T10:19:12.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 Processor -- start 2024-04-25T10:19:12.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 -- start start 2024-04-25T10:19:12.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff640b1f70 0x7eff641467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff640ce330 0x7eff64158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff6415c9a0 0x7eff6415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6415cf00 con 0x7eff640b1f70 2024-04-25T10:19:12.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6415cd80 con 0x7eff640ce330 2024-04-25T10:19:12.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff6a7ac640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6415d950 con 0x7eff6415c9a0 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff63fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff640b1f70 0x7eff641467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff63fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff640b1f70 0x7eff641467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46732/0 (socket says 172.21.15.133:46732) 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff63fff640 1 -- 172.21.15.133:0/1166078285 learned_addr learned my addr 172.21.15.133:0/1166078285 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff637fe640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff640ce330 0x7eff64158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff68d49640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff6415c9a0 0x7eff6415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff63fff640 1 -- 172.21.15.133:0/1166078285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff6415c9a0 msgr2=0x7eff6415a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff63fff640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff6415c9a0 0x7eff6415a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff63fff640 1 -- 172.21.15.133:0/1166078285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff640ce330 msgr2=0x7eff64158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff63fff640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff640ce330 0x7eff64158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff63fff640 1 -- 172.21.15.133:0/1166078285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff64159350 con 0x7eff640b1f70 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff63fff640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff640b1f70 0x7eff641467e0 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7eff5400d620 tx=0x7eff5400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.465+0000 7eff60ff9640 1 -- 172.21.15.133:0/1166078285 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5400b620 con 0x7eff640b1f70 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.469+0000 7eff60ff9640 1 -- 172.21.15.133:0/1166078285 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff54019070 con 0x7eff640b1f70 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.469+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1166078285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff64145a20 con 0x7eff640b1f70 2024-04-25T10:19:12.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.469+0000 7eff60ff9640 1 -- 172.21.15.133:0/1166078285 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5400b9e0 con 0x7eff640b1f70 2024-04-25T10:19:12.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.469+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1166078285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff64145fe0 con 0x7eff640b1f70 2024-04-25T10:19:12.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.469+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1166078285 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6406bc20 con 0x7eff640b1f70 2024-04-25T10:19:12.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.469+0000 7eff60ff9640 1 -- 172.21.15.133:0/1166078285 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff5400bb80 con 0x7eff640b1f70 2024-04-25T10:19:12.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.469+0000 7eff60ff9640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff50066f90 0x7eff50069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.469+0000 7eff637fe640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff50066f90 0x7eff50069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.469+0000 7eff60ff9640 1 -- 172.21.15.133:0/1166078285 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7eff54085370 con 0x7eff640b1f70 2024-04-25T10:19:12.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.469+0000 7eff637fe640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff50066f90 0x7eff50069450 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7eff64154770 tx=0x7eff58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:12.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.473+0000 7eff60ff9640 1 -- 172.21.15.133:0/1166078285 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff64159350 con 0x7eff640b1f70 2024-04-25T10:19:12.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.629+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1166078285 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff64068ef0 con 0x7eff50066f90 2024-04-25T10:19:12.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.629+0000 7eff60ff9640 1 -- 172.21.15.133:0/1166078285 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13524 (secure 0 0 0) 0x7eff64068ef0 con 0x7eff50066f90 2024-04-25T10:19:12.633 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:12.633 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:19:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1166078285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff50066f90 msgr2=0x7eff50069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff50066f90 0x7eff50069450 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7eff64154770 tx=0x7eff58008040 comp rx=0 tx=0).stop 2024-04-25T10:19:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1166078285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff640b1f70 msgr2=0x7eff641467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff640b1f70 0x7eff641467e0 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7eff5400d620 tx=0x7eff5400dae0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1166078285 shutdown_connections 2024-04-25T10:19:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7eff6415c9a0 0x7eff6415a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7eff640ce330 0x7eff64158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7eff50066f90 0x7eff50069450 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 --2- 172.21.15.133:0/1166078285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff640b1f70 0x7eff641467e0 unknown :-1 s=CLOSED pgs=3041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1166078285 >> 172.21.15.133:0/1166078285 conn(0x7eff64067000 msgr2=0x7eff6415df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:12.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1166078285 shutdown_connections 2024-04-25T10:19:12.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.633+0000 7eff6a7ac640 1 -- 172.21.15.133:0/1166078285 wait complete. 2024-04-25T10:19:12.651 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":718,"stamp":"2024-04-25T10:19:12.038189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":586416,"kb_used_data":45508,"kb_used_omap":12,"kb_used_meta":540787,"kb_avail":754388304,"statfs":{"total":773094113280,"available":772493623296,"internally_reserved":0,"allocated":46600192,"data_stored":35894583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":553766449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12618969,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-101,"num_read_kb":-105,"num_write":-98,"num_write_kb":-12356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-5,"num_bytes_recovered":-35313,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001869"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":509,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-04-25T10:19:09.208823+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:19:09.208823+0000","last_peered":"2024-04-25T10:19:09.208823+0000","last_clean":"2024-04-25T10:19:09.208823+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:19:09.208823+0000","last_undegraded":"2024-04-25T10:19:09.208823+0000","last_fullsized":"2024-04-25T10:19:09.208823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575062,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78576,"kb_used_data":7140,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94293264,"statfs":{"total":96636764160,"available":96556302336,"internally_reserved":0,"allocated":7311360,"data_stored":5971063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575063,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73848,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297992,"statfs":{"total":96636764160,"available":96561143808,"internally_reserved":0,"allocated":2531328,"data_stored":1192268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73071046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575062,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51668,"kb_used_data":6600,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94320172,"statfs":{"total":96636764160,"available":96583856128,"internally_reserved":0,"allocated":6758400,"data_stored":5415336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575062,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96104,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275736,"statfs":{"total":96636764160,"available":96538353664,"internally_reserved":0,"allocated":2519040,"data_stored":1185809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93876,"kb_used_data":12712,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94277964,"statfs":{"total":96636764160,"available":96540635136,"internally_reserved":0,"allocated":13017088,"data_stored":11678028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83098055},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575062,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57912,"kb_used_data":4520,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313928,"statfs":{"total":96636764160,"available":96577462272,"internally_reserved":0,"allocated":4628480,"data_stored":3289420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53732,"kb_used_data":4504,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318108,"statfs":{"total":96636764160,"available":96581742592,"internally_reserved":0,"allocated":4612096,"data_stored":3282871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575063,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80700,"kb_used_data":5100,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291140,"statfs":{"total":96636764160,"available":96554127360,"internally_reserved":0,"allocated":5222400,"data_stored":3879788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:12.652 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:19:12.652 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:12.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.769+0000 7fce18f90640 1 Processor -- start 2024-04-25T10:19:12.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.769+0000 7fce18f90640 1 -- start start 2024-04-25T10:19:12.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce18f90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce1415dd90 0x7fce140ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce18f90640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fce140b19f0 0x7fce140cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce18f90640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce140ca9b0 con 0x7fce1415d9b0 2024-04-25T10:19:12.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce18f90640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce140cbe30 con 0x7fce140b19f0 2024-04-25T10:19:12.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce18f90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce140cac90 con 0x7fce1415dd90 2024-04-25T10:19:12.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce12d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fce140b19f0 0x7fce140cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce09d9b640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce1415dd90 0x7fce140ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce12d9d640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fce140b19f0 0x7fce140cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.133:34100/0 (socket says 172.21.15.133:34100) 2024-04-25T10:19:12.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce1259c640 1 --1- >> v1:172.21.15.162:6789/0 conn(0x7fce1415d9b0 0x7fce140cb5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.162:6789/0 says I am v1:172.21.15.133:34240/0 (socket says 172.21.15.133:34240) 2024-04-25T10:19:12.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce12d9d640 1 -- 172.21.15.133:0/2264015275 learned_addr learned my addr 172.21.15.133:0/2264015275 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:12.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce09d9b640 1 -- 172.21.15.133:0/2264015275 >> v2:172.21.15.162:3300/0 conn(0x7fce140b19f0 msgr2=0x7fce140cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:12.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce09d9b640 1 --2- 172.21.15.133:0/2264015275 >> v2:172.21.15.162:3300/0 conn(0x7fce140b19f0 0x7fce140cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce09d9b640 1 -- 172.21.15.133:0/2264015275 >> v1:172.21.15.162:6789/0 conn(0x7fce1415d9b0 legacy=0x7fce140cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce11d9b640 1 -- 172.21.15.133:0/2264015275 <== mon.1 v1:172.21.15.162:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113935563 0 0) 0x7fce140ca9b0 con 0x7fce1415d9b0 2024-04-25T10:19:12.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.773+0000 7fce09d9b640 1 -- 172.21.15.133:0/2264015275 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140caf70 con 0x7fce1415dd90 2024-04-25T10:19:12.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce09d9b640 1 --2- 172.21.15.133:0/2264015275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce1415dd90 0x7fce140ce770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0097c0 tx=0x7fcdfc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=172d572ae6b68cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:12.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce11d9b640 1 -- 172.21.15.133:0/2264015275 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0ab430 con 0x7fce1415dd90 2024-04-25T10:19:12.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce11d9b640 1 -- 172.21.15.133:0/2264015275 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc0b0070 con 0x7fce1415dd90 2024-04-25T10:19:12.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce11d9b640 1 -- 172.21.15.133:0/2264015275 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0ab7f0 con 0x7fce1415dd90 2024-04-25T10:19:12.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 -- 172.21.15.133:0/2264015275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce1415dd90 msgr2=0x7fce140ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 --2- 172.21.15.133:0/2264015275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce1415dd90 0x7fce140ce770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0097c0 tx=0x7fcdfc01c410 comp rx=0 tx=0).stop 2024-04-25T10:19:12.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 -- 172.21.15.133:0/2264015275 shutdown_connections 2024-04-25T10:19:12.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 --2- 172.21.15.133:0/2264015275 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce1415dd90 0x7fce140ce770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 --2- 172.21.15.133:0/2264015275 >> v2:172.21.15.162:3300/0 conn(0x7fce140b19f0 0x7fce140cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 -- 172.21.15.133:0/2264015275 >> 172.21.15.133:0/2264015275 conn(0x7fce14067000 msgr2=0x7fce1406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:12.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 -- 172.21.15.133:0/2264015275 shutdown_connections 2024-04-25T10:19:12.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 -- 172.21.15.133:0/2264015275 wait complete. 2024-04-25T10:19:12.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 Processor -- start 2024-04-25T10:19:12.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 -- start start 2024-04-25T10:19:12.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce140b19f0 0x7fce14151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce1415d9b0 0x7fce1414bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1415dd90 0x7fce1414c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1406e290 con 0x7fce1415d9b0 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1406e110 con 0x7fce1415dd90 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce18f90640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1406e410 con 0x7fce140b19f0 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce09d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce1415d9b0 0x7fce1414bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce09d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce1415d9b0 0x7fce1414bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46742/0 (socket says 172.21.15.133:46742) 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.777+0000 7fce09d9b640 1 -- 172.21.15.133:0/414529233 learned_addr learned my addr 172.21.15.133:0/414529233 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce12d9d640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1415dd90 0x7fce1414c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce1259c640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce140b19f0 0x7fce14151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce09d9b640 1 -- 172.21.15.133:0/414529233 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce140b19f0 msgr2=0x7fce14151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce09d9b640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce140b19f0 0x7fce14151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce09d9b640 1 -- 172.21.15.133:0/414529233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1415dd90 msgr2=0x7fce1414c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce09d9b640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1415dd90 0x7fce1414c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce09d9b640 1 -- 172.21.15.133:0/414529233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce14150030 con 0x7fce1415d9b0 2024-04-25T10:19:12.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce1259c640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce140b19f0 0x7fce14151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:12.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce09d9b640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce1415d9b0 0x7fce1414bb70 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0097c0 tx=0x7fcdfc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:12.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce0b7fe640 1 -- 172.21.15.133:0/414529233 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc004020 con 0x7fce1415d9b0 2024-04-25T10:19:12.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce18f90640 1 -- 172.21.15.133:0/414529233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce14150310 con 0x7fce1415d9b0 2024-04-25T10:19:12.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce0b7fe640 1 -- 172.21.15.133:0/414529233 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc0041c0 con 0x7fce1415d9b0 2024-04-25T10:19:12.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce0b7fe640 1 -- 172.21.15.133:0/414529233 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc003620 con 0x7fce1415d9b0 2024-04-25T10:19:12.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce18f90640 1 -- 172.21.15.133:0/414529233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce141508d0 con 0x7fce1415d9b0 2024-04-25T10:19:12.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce18f90640 1 -- 172.21.15.133:0/414529233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce14068ef0 con 0x7fce1415d9b0 2024-04-25T10:19:12.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce0b7fe640 1 -- 172.21.15.133:0/414529233 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcdfc0bd070 con 0x7fce1415d9b0 2024-04-25T10:19:12.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce0b7fe640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdf4066f60 0x7fcdf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:12.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce1259c640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdf4066f60 0x7fcdf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:12.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce0b7fe640 1 -- 172.21.15.133:0/414529233 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fcdfc11f530 con 0x7fce1415d9b0 2024-04-25T10:19:12.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.781+0000 7fce1259c640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdf4066f60 0x7fcdf4069420 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fce04000f70 tx=0x7fce04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:12.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.785+0000 7fce0b7fe640 1 -- 172.21.15.133:0/414529233 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcdfc016090 con 0x7fce1415d9b0 2024-04-25T10:19:12.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.941+0000 7fce18f90640 1 -- 172.21.15.133:0/414529233 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce140b3ac0 con 0x7fcdf4066f60 2024-04-25T10:19:12.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.941+0000 7fce0b7fe640 1 -- 172.21.15.133:0/414529233 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13524 (secure 0 0 0) 0x7fce140b3ac0 con 0x7fcdf4066f60 2024-04-25T10:19:12.944 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:12.944 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:19:12.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 -- 172.21.15.133:0/414529233 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdf4066f60 msgr2=0x7fcdf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdf4066f60 0x7fcdf4069420 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fce04000f70 tx=0x7fce04009040 comp rx=0 tx=0).stop 2024-04-25T10:19:12.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 -- 172.21.15.133:0/414529233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce1415d9b0 msgr2=0x7fce1414bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce1415d9b0 0x7fce1414bb70 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0097c0 tx=0x7fcdfc01ac50 comp rx=0 tx=0).stop 2024-04-25T10:19:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 -- 172.21.15.133:0/414529233 shutdown_connections 2024-04-25T10:19:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fce1415dd90 0x7fce1414c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fcdf4066f60 0x7fcdf4069420 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce1415d9b0 0x7fce1414bb70 unknown :-1 s=CLOSED pgs=3042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 --2- 172.21.15.133:0/414529233 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fce140b19f0 0x7fce14151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 -- 172.21.15.133:0/414529233 >> 172.21.15.133:0/414529233 conn(0x7fce14067000 msgr2=0x7fce140b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 -- 172.21.15.133:0/414529233 shutdown_connections 2024-04-25T10:19:12.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:12.945+0000 7fce18f90640 1 -- 172.21.15.133:0/414529233 wait complete. 2024-04-25T10:19:12.963 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":718,"stamp":"2024-04-25T10:19:12.038189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":586416,"kb_used_data":45508,"kb_used_omap":12,"kb_used_meta":540787,"kb_avail":754388304,"statfs":{"total":773094113280,"available":772493623296,"internally_reserved":0,"allocated":46600192,"data_stored":35894583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":553766449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12618969,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-101,"num_read_kb":-105,"num_write":-98,"num_write_kb":-12356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-5,"num_bytes_recovered":-35313,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001869"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":509,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-04-25T10:19:09.208823+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:19:09.208823+0000","last_peered":"2024-04-25T10:19:09.208823+0000","last_clean":"2024-04-25T10:19:09.208823+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:19:09.208823+0000","last_undegraded":"2024-04-25T10:19:09.208823+0000","last_fullsized":"2024-04-25T10:19:09.208823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575062,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78576,"kb_used_data":7140,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94293264,"statfs":{"total":96636764160,"available":96556302336,"internally_reserved":0,"allocated":7311360,"data_stored":5971063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575063,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73848,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297992,"statfs":{"total":96636764160,"available":96561143808,"internally_reserved":0,"allocated":2531328,"data_stored":1192268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73071046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575062,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51668,"kb_used_data":6600,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94320172,"statfs":{"total":96636764160,"available":96583856128,"internally_reserved":0,"allocated":6758400,"data_stored":5415336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575062,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96104,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275736,"statfs":{"total":96636764160,"available":96538353664,"internally_reserved":0,"allocated":2519040,"data_stored":1185809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93876,"kb_used_data":12712,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94277964,"statfs":{"total":96636764160,"available":96540635136,"internally_reserved":0,"allocated":13017088,"data_stored":11678028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83098055},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575062,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57912,"kb_used_data":4520,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313928,"statfs":{"total":96636764160,"available":96577462272,"internally_reserved":0,"allocated":4628480,"data_stored":3289420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575062,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53732,"kb_used_data":4504,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318108,"statfs":{"total":96636764160,"available":96581742592,"internally_reserved":0,"allocated":4612096,"data_stored":3282871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575063,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80700,"kb_used_data":5100,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94291140,"statfs":{"total":96636764160,"available":96554127360,"internally_reserved":0,"allocated":5222400,"data_stored":3879788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:12.964 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6da9b0640 1 Processor -- start 2024-04-25T10:19:13.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6da9b0640 1 -- start start 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6da9b0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6d415c9a0 0x7fb6d40c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6da9b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d40b1f70 0x7fb6d406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6da9b0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d40c61d0 con 0x7fb6d40ce330 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6da9b0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d40c64b0 con 0x7fb6d415c9a0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6da9b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d406d4f0 con 0x7fb6d40b1f70 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6d3fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6d415c9a0 0x7fb6d40c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6d8f4d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb6d40ce330 0x7fb6d415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40996/0 (socket says 172.21.15.133:40996) 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6d8f4d640 1 -- 172.21.15.133:0/4156400811 learned_addr learned my addr 172.21.15.133:0/4156400811 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6cbfff640 1 --2- 172.21.15.133:0/4156400811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d40b1f70 0x7fb6d406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6d3fff640 1 -- 172.21.15.133:0/4156400811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d40b1f70 msgr2=0x7fb6d406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6d3fff640 1 --2- 172.21.15.133:0/4156400811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d40b1f70 0x7fb6d406dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6d3fff640 1 -- 172.21.15.133:0/4156400811 >> v1:172.21.15.133:6789/0 conn(0x7fb6d40ce330 legacy=0x7fb6d415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6d3fff640 1 -- 172.21.15.133:0/4156400811 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d406d7d0 con 0x7fb6d415c9a0 2024-04-25T10:19:13.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6d37fe640 1 -- 172.21.15.133:0/4156400811 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64051238 0 0) 0x7fb6d40c61d0 con 0x7fb6d40ce330 2024-04-25T10:19:13.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.085+0000 7fb6cbfff640 1 --2- 172.21.15.133:0/4156400811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d40b1f70 0x7fb6d406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d3fff640 1 --2- 172.21.15.133:0/4156400811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6d415c9a0 0x7fb6d40c5980 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0097c0 tx=0x7fb6bc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a649c2c8d0019f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d37fe640 1 -- 172.21.15.133:0/4156400811 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0b0070 con 0x7fb6d415c9a0 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d37fe640 1 -- 172.21.15.133:0/4156400811 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6bc004230 con 0x7fb6d415c9a0 2024-04-25T10:19:13.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d37fe640 1 -- 172.21.15.133:0/4156400811 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc003620 con 0x7fb6d415c9a0 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 -- 172.21.15.133:0/4156400811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6d415c9a0 msgr2=0x7fb6d40c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 --2- 172.21.15.133:0/4156400811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6d415c9a0 0x7fb6d40c5980 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0097c0 tx=0x7fb6bc01c440 comp rx=0 tx=0).stop 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 -- 172.21.15.133:0/4156400811 shutdown_connections 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 --2- 172.21.15.133:0/4156400811 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d40b1f70 0x7fb6d406dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 --2- 172.21.15.133:0/4156400811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6d415c9a0 0x7fb6d40c5980 unknown :-1 s=CLOSED pgs=3044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 -- 172.21.15.133:0/4156400811 >> 172.21.15.133:0/4156400811 conn(0x7fb6d4067000 msgr2=0x7fb6d406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 -- 172.21.15.133:0/4156400811 shutdown_connections 2024-04-25T10:19:13.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 -- 172.21.15.133:0/4156400811 wait complete. 2024-04-25T10:19:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 Processor -- start 2024-04-25T10:19:13.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 -- start start 2024-04-25T10:19:13.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d40b1f70 0x7fb6d415de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d40ce330 0x7fb6d415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6d415c9a0 0x7fb6d41430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d415cd80 con 0x7fb6d415c9a0 2024-04-25T10:19:13.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40ceb70 con 0x7fb6d40b1f70 2024-04-25T10:19:13.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d415cf00 con 0x7fb6d40ce330 2024-04-25T10:19:13.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d8f4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6d415c9a0 0x7fb6d41430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d8f4d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6d415c9a0 0x7fb6d41430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46766/0 (socket says 172.21.15.133:46766) 2024-04-25T10:19:13.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d8f4d640 1 -- 172.21.15.133:0/3555575494 learned_addr learned my addr 172.21.15.133:0/3555575494 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:13.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d8f4d640 1 -- 172.21.15.133:0/3555575494 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d40ce330 msgr2=0x7fb6d415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d8f4d640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d40ce330 0x7fb6d415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d8f4d640 1 -- 172.21.15.133:0/3555575494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d40b1f70 msgr2=0x7fb6d415de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d8f4d640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d40b1f70 0x7fb6d415de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d8f4d640 1 -- 172.21.15.133:0/3555575494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d414a720 con 0x7fb6d415c9a0 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d8f4d640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6d415c9a0 0x7fb6d41430a0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc001570 tx=0x7fb6cc00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d17fa640 1 -- 172.21.15.133:0/3555575494 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc018070 con 0x7fb6d415c9a0 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d17fa640 1 -- 172.21.15.133:0/3555575494 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc00a640 con 0x7fb6d415c9a0 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6da9b0640 1 -- 172.21.15.133:0/3555575494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d4144050 con 0x7fb6d415c9a0 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.089+0000 7fb6d17fa640 1 -- 172.21.15.133:0/3555575494 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00aa30 con 0x7fb6d415c9a0 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.093+0000 7fb6da9b0640 1 -- 172.21.15.133:0/3555575494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d4144610 con 0x7fb6d415c9a0 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.093+0000 7fb6da9b0640 1 -- 172.21.15.133:0/3555575494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d4068ef0 con 0x7fb6d415c9a0 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.093+0000 7fb6d17fa640 1 -- 172.21.15.133:0/3555575494 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6cc01b070 con 0x7fb6d415c9a0 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.093+0000 7fb6d17fa640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6b4066f60 0x7fb6b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.093+0000 7fb6d17fa640 1 -- 172.21.15.133:0/3555575494 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb6cc084570 con 0x7fb6d415c9a0 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.093+0000 7fb6d3fff640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6b4066f60 0x7fb6b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:13.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.093+0000 7fb6d3fff640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6b4066f60 0x7fb6b4069420 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0ad000 tx=0x7fb6bc004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:13.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.097+0000 7fb6d17fa640 1 -- 172.21.15.133:0/3555575494 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6cc055730 con 0x7fb6d415c9a0 2024-04-25T10:19:13.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.257+0000 7fb6da9b0640 1 -- 172.21.15.133:0/3555575494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6d414a8f0 con 0x7fb6d415c9a0 2024-04-25T10:19:13.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.257+0000 7fb6d17fa640 1 -- 172.21.15.133:0/3555575494 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+13461 (secure 0 0 0) 0x7fb6cc055030 con 0x7fb6d415c9a0 2024-04-25T10:19:13.261 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:13.261 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":201,"fsid":"599cdb88-206d-4299-bbcd-9f7da90fb181","created":"2024-04-25T10:01:27.080113+0000","modified":"2024-04-25T10:19:10.208216+0000","last_up_change":"2024-04-25T10:01:38.774614+0000","last_in_change":"2024-04-25T10:01:29.753511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:39.778509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b6489a91-3b21-46bb-9548-89484d0e1f11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6818","nonce":136834454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6822","nonce":136834454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6829","nonce":136834454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":136834454},{"type":"v1","addr":"172.21.15.133:6825","nonce":136834454}]},"public_addr":"172.21.15.133:6818/136834454","cluster_addr":"172.21.15.133:6822/136834454","heartbeat_back_addr":"172.21.15.133:6829/136834454","heartbeat_front_addr":"172.21.15.133:6825/136834454","state":["exists","up"]},{"osd":1,"uuid":"afabe68c-82ae-4784-9c51-6998ffa2016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6801","nonce":2082178647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6803","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6805","nonce":2082178647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6811","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6813","nonce":2082178647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6807","nonce":2082178647},{"type":"v1","addr":"172.21.15.133:6809","nonce":2082178647}]},"public_addr":"172.21.15.133:6801/2082178647","cluster_addr":"172.21.15.133:6805/2082178647","heartbeat_back_addr":"172.21.15.133:6813/2082178647","heartbeat_front_addr":"172.21.15.133:6809/2082178647","state":["exists","up"]},{"osd":2,"uuid":"3ebbc6d1-c28b-4e7c-a9b6-de419e68550b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6817","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6819","nonce":148430844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6823","nonce":148430844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6831","nonce":148430844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":148430844},{"type":"v1","addr":"172.21.15.133:6828","nonce":148430844}]},"public_addr":"172.21.15.133:6819/148430844","cluster_addr":"172.21.15.133:6823/148430844","heartbeat_back_addr":"172.21.15.133:6831/148430844","heartbeat_front_addr":"172.21.15.133:6828/148430844","state":["exists","up"]},{"osd":3,"uuid":"2c2000e2-21f4-40f6-b8f6-a7a181a2dd49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6804","nonce":447485461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6808","nonce":447485461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6815","nonce":447485461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":447485461},{"type":"v1","addr":"172.21.15.133:6812","nonce":447485461}]},"public_addr":"172.21.15.133:6804/447485461","cluster_addr":"172.21.15.133:6808/447485461","heartbeat_back_addr":"172.21.15.133:6815/447485461","heartbeat_front_addr":"172.21.15.133:6812/447485461","state":["exists","up"]},{"osd":4,"uuid":"322330bb-a145-4561-9bf4-24b807d2e51a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6801","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6804","nonce":3188774481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6807","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6811","nonce":3188774481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6823","nonce":3188774481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":3188774481},{"type":"v1","addr":"172.21.15.162:6817","nonce":3188774481}]},"public_addr":"172.21.15.162:6804/3188774481","cluster_addr":"172.21.15.162:6811/3188774481","heartbeat_back_addr":"172.21.15.162:6823/3188774481","heartbeat_front_addr":"172.21.15.162:6817/3188774481","state":["exists","up"]},{"osd":5,"uuid":"58d88fea-42ba-4586-853d-fcfcb2883e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6803","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6806","nonce":4006238640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6810","nonce":4006238640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6821","nonce":4006238640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6813","nonce":4006238640},{"type":"v1","addr":"172.21.15.162:6816","nonce":4006238640}]},"public_addr":"172.21.15.162:6806/4006238640","cluster_addr":"172.21.15.162:6810/4006238640","heartbeat_back_addr":"172.21.15.162:6821/4006238640","heartbeat_front_addr":"172.21.15.162:6816/4006238640","state":["exists","up"]},{"osd":6,"uuid":"ada17685-6a97-424f-822e-89180562d474","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6825","nonce":2819373209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6827","nonce":2819373209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6831","nonce":2819373209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":2819373209},{"type":"v1","addr":"172.21.15.162:6829","nonce":2819373209}]},"public_addr":"172.21.15.162:6825/2819373209","cluster_addr":"172.21.15.162:6827/2819373209","heartbeat_back_addr":"172.21.15.162:6831/2819373209","heartbeat_front_addr":"172.21.15.162:6829/2819373209","state":["exists","up"]},{"osd":7,"uuid":"11a90ffe-04b2-4c01-aa42-faaf9fd347d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6802","nonce":1166832594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6805","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6809","nonce":1166832594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6819","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6822","nonce":1166832594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":1166832594},{"type":"v1","addr":"172.21.15.162:6815","nonce":1166832594}]},"public_addr":"172.21.15.162:6802/1166832594","cluster_addr":"172.21.15.162:6809/1166832594","heartbeat_back_addr":"172.21.15.162:6822/1166832594","heartbeat_front_addr":"172.21.15.162:6815/1166832594","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.295287+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.204797+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.336532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.258944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.283763+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.360324+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.354026+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:35.350774+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/2510709242":"2024-04-26T10:19:07.013056+0000","172.21.15.133:6836/2510709242":"2024-04-26T10:19:07.013056+0000","172.21.15.133:6837/1916780253":"2024-04-26T10:17:55.746978+0000","172.21.15.133:6836/1916780253":"2024-04-26T10:17:55.746978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:13.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 -- 172.21.15.133:0/3555575494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6b4066f60 msgr2=0x7fb6b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6b4066f60 0x7fb6b4069420 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0ad000 tx=0x7fb6bc004740 comp rx=0 tx=0).stop 2024-04-25T10:19:13.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 -- 172.21.15.133:0/3555575494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6d415c9a0 msgr2=0x7fb6d41430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:13.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6d415c9a0 0x7fb6d41430a0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc001570 tx=0x7fb6cc00cca0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 -- 172.21.15.133:0/3555575494 shutdown_connections 2024-04-25T10:19:13.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb6b4066f60 0x7fb6b4069420 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6d415c9a0 0x7fb6d41430a0 unknown :-1 s=CLOSED pgs=3045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb6d40ce330 0x7fb6d415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 --2- 172.21.15.133:0/3555575494 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d40b1f70 0x7fb6d415de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:13.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 -- 172.21.15.133:0/3555575494 >> 172.21.15.133:0/3555575494 conn(0x7fb6d4067000 msgr2=0x7fb6d40b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:13.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 -- 172.21.15.133:0/3555575494 shutdown_connections 2024-04-25T10:19:13.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:13.261+0000 7fb6da9b0640 1 -- 172.21.15.133:0/3555575494 wait complete. 2024-04-25T10:19:14.281 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:19:14.281 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:19:14.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba90060640 1 Processor -- start 2024-04-25T10:19:14.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba90060640 1 -- start start 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba90060640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba88071cb0 0x7fba8815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba90060640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba8815c250 0x7fba880cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba90060640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8815c9b0 con 0x7fba880b4940 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba90060640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba88072a90 con 0x7fba88071cb0 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba90060640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba880a85d0 con 0x7fba8815c250 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba8d5fb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba88071cb0 0x7fba8815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba8d5fb640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba88071cb0 0x7fba8815bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46770/0 (socket says 172.21.15.133:46770) 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba8d5fb640 1 -- 172.21.15.133:0/1872621841 learned_addr learned my addr 172.21.15.133:0/1872621841 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba8e5fd640 1 --2- 172.21.15.133:0/1872621841 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba8815c250 0x7fba880cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba8d5fb640 1 -- 172.21.15.133:0/1872621841 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba8815c250 msgr2=0x7fba880cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba8d5fb640 1 --2- 172.21.15.133:0/1872621841 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba8815c250 0x7fba880cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba8cdfa640 1 -- 172.21.15.133:0/1872621841 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397172963 0 0) 0x7fba8815c9b0 con 0x7fba880b4940 2024-04-25T10:19:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba8d5fb640 1 -- 172.21.15.133:0/1872621841 >> v1:172.21.15.133:6789/0 conn(0x7fba880b4940 legacy=0x7fba880b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba8d5fb640 1 -- 172.21.15.133:0/1872621841 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8815cc90 con 0x7fba88071cb0 2024-04-25T10:19:14.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.421+0000 7fba8e5fd640 1 --2- 172.21.15.133:0/1872621841 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba8815c250 0x7fba880cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba8d5fb640 1 --2- 172.21.15.133:0/1872621841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88071cb0 0x7fba8815bd10 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7fba78009870 tx=0x7fba7801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0dcb903728360f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba8cdfa640 1 -- 172.21.15.133:0/1872621841 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780a95f0 con 0x7fba88071cb0 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba8cdfa640 1 -- 172.21.15.133:0/1872621841 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba780a9790 con 0x7fba88071cb0 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba8cdfa640 1 -- 172.21.15.133:0/1872621841 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780a9b60 con 0x7fba88071cb0 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba90060640 1 -- 172.21.15.133:0/1872621841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88071cb0 msgr2=0x7fba8815bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba90060640 1 --2- 172.21.15.133:0/1872621841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88071cb0 0x7fba8815bd10 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7fba78009870 tx=0x7fba7801c530 comp rx=0 tx=0).stop 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba90060640 1 -- 172.21.15.133:0/1872621841 shutdown_connections 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba90060640 1 --2- 172.21.15.133:0/1872621841 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba8815c250 0x7fba880cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba90060640 1 --2- 172.21.15.133:0/1872621841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba88071cb0 0x7fba8815bd10 unknown :-1 s=CLOSED pgs=3047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba90060640 1 -- 172.21.15.133:0/1872621841 >> 172.21.15.133:0/1872621841 conn(0x7fba8806d370 msgr2=0x7fba8806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba90060640 1 -- 172.21.15.133:0/1872621841 shutdown_connections 2024-04-25T10:19:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba90060640 1 -- 172.21.15.133:0/1872621841 wait complete. 2024-04-25T10:19:14.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba90060640 1 Processor -- start 2024-04-25T10:19:14.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.425+0000 7fba90060640 1 -- start start 2024-04-25T10:19:14.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba90060640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba88071cb0 0x7fba8813d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba90060640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba880b4940 0x7fba880b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba90060640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba8815c250 0x7fba880afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba90060640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba880cc960 con 0x7fba880b4940 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba90060640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8815c660 con 0x7fba8815c250 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba90060640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba880ccae0 con 0x7fba88071cb0 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8d5fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba880b4940 0x7fba880b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8d5fb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba880b4940 0x7fba880b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46774/0 (socket says 172.21.15.133:46774) 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8d5fb640 1 -- 172.21.15.133:0/2163280963 learned_addr learned my addr 172.21.15.133:0/2163280963 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8ddfc640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba88071cb0 0x7fba8813d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8d5fb640 1 -- 172.21.15.133:0/2163280963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba88071cb0 msgr2=0x7fba8813d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8d5fb640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fba88071cb0 0x7fba8813d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8d5fb640 1 -- 172.21.15.133:0/2163280963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba8815c250 msgr2=0x7fba880afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8d5fb640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba8815c250 0x7fba880afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8d5fb640 1 -- 172.21.15.133:0/2163280963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba88073350 con 0x7fba880b4940 2024-04-25T10:19:14.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8d5fb640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba880b4940 0x7fba880b2730 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7fba780ab040 tx=0x7fba7801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba7e7fc640 1 -- 172.21.15.133:0/2163280963 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780b1800 con 0x7fba880b4940 2024-04-25T10:19:14.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba7e7fc640 1 -- 172.21.15.133:0/2163280963 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba780b19a0 con 0x7fba880b4940 2024-04-25T10:19:14.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba90060640 1 -- 172.21.15.133:0/2163280963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba880b31b0 con 0x7fba880b4940 2024-04-25T10:19:14.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba7e7fc640 1 -- 172.21.15.133:0/2163280963 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780b1d90 con 0x7fba880b4940 2024-04-25T10:19:14.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba90060640 1 -- 172.21.15.133:0/2163280963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba880b3770 con 0x7fba880b4940 2024-04-25T10:19:14.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba90060640 1 -- 172.21.15.133:0/2163280963 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fba88069000 con 0x7fba880b4940 2024-04-25T10:19:14.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba7e7fc640 1 -- 172.21.15.133:0/2163280963 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba780a9650 con 0x7fba880b4940 2024-04-25T10:19:14.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba7e7fc640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba6c066f60 0x7fba6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba7e7fc640 1 -- 172.21.15.133:0/2163280963 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fba78120040 con 0x7fba880b4940 2024-04-25T10:19:14.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8ddfc640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba6c066f60 0x7fba6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba7e7fc640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x7fba6c06d4a0 0x7fba6c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba7e7fc640 1 -- 172.21.15.133:0/2163280963 --> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba780b2e50 con 0x7fba6c06d4a0 2024-04-25T10:19:14.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba7e7fc640 1 -- 172.21.15.133:0/2163280963 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba781204d0 con 0x7fba880b4940 2024-04-25T10:19:14.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8e5fd640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x7fba6c06d4a0 0x7fba6c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.429+0000 7fba8ddfc640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba6c066f60 0x7fba6c069420 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fba880b4910 tx=0x7fba84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.433+0000 7fba8e5fd640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x7fba6c06d4a0 0x7fba6c06f8e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.433+0000 7fba7e7fc640 1 -- 172.21.15.133:0/2163280963 <== osd.0 v2:172.21.15.133:6816/136834454 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fba780b2e50 con 0x7fba6c06d4a0 2024-04-25T10:19:14.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba90060640 1 -- 172.21.15.133:0/2163280963 --> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fba880724c0 con 0x7fba6c06d4a0 2024-04-25T10:19:14.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba7e7fc640 1 -- 172.21.15.133:0/2163280963 <== osd.0 v2:172.21.15.133:6816/136834454 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fba880724c0 con 0x7fba6c06d4a0 2024-04-25T10:19:14.447 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:19:14.447 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:19:14.447 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:19:14.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba90060640 1 -- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x7fba6c06d4a0 msgr2=0x7fba6c06f8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba90060640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:6816/136834454,v1:172.21.15.133:6818/136834454] conn(0x7fba6c06d4a0 0x7fba6c06f8e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba90060640 1 -- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba6c066f60 msgr2=0x7fba6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba90060640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fba6c066f60 0x7fba6c069420 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fba880b4910 tx=0x7fba84009040 comp rx=0 tx=0).stop 2024-04-25T10:19:14.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba90060640 1 -- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba880b4940 msgr2=0x7fba880b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba90060640 1 --2- 172.21.15.133:0/2163280963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba880b4940 0x7fba880b2730 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7fba780ab040 tx=0x7fba7801ad70 comp rx=0 tx=0).stop 2024-04-25T10:19:14.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba8e5fd640 1 -- 172.21.15.133:0/2163280963 reap_dead start 2024-04-25T10:19:14.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba90060640 1 -- 172.21.15.133:0/2163280963 shutdown_connections 2024-04-25T10:19:14.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba90060640 1 -- 172.21.15.133:0/2163280963 >> 172.21.15.133:0/2163280963 conn(0x7fba8806d370 msgr2=0x7fba880735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:14.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba90060640 1 -- 172.21.15.133:0/2163280963 shutdown_connections 2024-04-25T10:19:14.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.445+0000 7fba90060640 1 -- 172.21.15.133:0/2163280963 wait complete. 2024-04-25T10:19:14.464 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:19:14.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18d63e6640 1 Processor -- start 2024-04-25T10:19:14.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18d63e6640 1 -- start start 2024-04-25T10:19:14.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18d63e6640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f18d006e970 0x7f18d0162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18d63e6640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d00cd6f0 con 0x7f18d00ce8e0 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18d63e6640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d0163340 con 0x7f18d00ce450 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18d63e6640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d00cf270 con 0x7f18d006e970 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18d4983640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f18d00ce8e0 0x7f18d006d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41018/0 (socket says 172.21.15.133:41018) 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18d4983640 1 -- 172.21.15.133:0/3894877636 learned_addr learned my addr 172.21.15.133:0/3894877636 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201039335 0 0) 0x7f18c80014a0 con 0x7f18d00ce8e0 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18cffff640 1 --2- 172.21.15.133:0/3894877636 >> v2:172.21.15.162:3300/0 conn(0x7f18d006e970 0x7f18d0162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b8003530 con 0x7f18d00ce8e0 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2302075974 0 0) 0x7f18b8003530 con 0x7f18d00ce8e0 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 >> v2:172.21.15.162:3300/0 conn(0x7f18d006e970 msgr2=0x7f18d0162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 --2- 172.21.15.133:0/3894877636 >> v2:172.21.15.162:3300/0 conn(0x7f18d006e970 0x7f18d0162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 >> v1:172.21.15.162:6789/0 conn(0x7f18d00ce450 legacy=0x7f18d00b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00cd970 con 0x7f18d00ce8e0 2024-04-25T10:19:14.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f18c8001bd0 con 0x7f18d00ce8e0 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 >> v1:172.21.15.133:6789/0 conn(0x7f18d00ce8e0 legacy=0x7f18d006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18b80039c0 0x7f18b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18b8004810 0x7f18b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18b80051d0 0x7f18b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00cf270 con 0x7f18b80051d0 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b800c170 con 0x7f18b8004810 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b800c370 con 0x7f18b80039c0 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18d4983640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18b80051d0 0x7f18b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.573+0000 7f18cffff640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18b80039c0 0x7f18b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d4983640 1 -- 172.21.15.133:0/3894877636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18b80039c0 msgr2=0x7f18b8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d4983640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18b80039c0 0x7f18b8003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18cf7fe640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18b8004810 0x7f18b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d4983640 1 -- 172.21.15.133:0/3894877636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18b8004810 msgr2=0x7f18b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:14.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d4983640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18b8004810 0x7f18b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d4983640 1 -- 172.21.15.133:0/3894877636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b8003530 con 0x7f18b80051d0 2024-04-25T10:19:14.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18cf7fe640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18b8004810 0x7f18b8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d4983640 1 -- 172.21.15.133:0/3894877636 reap_dead start 2024-04-25T10:19:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d4983640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18b80051d0 0x7f18b800b9d0 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 crypto rx=0x7f18c800aa30 tx=0x7f18c801fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358f2cf93f6766dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c800a650 con 0x7f18b80051d0 2024-04-25T10:19:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c80aa510 con 0x7f18b80051d0 2024-04-25T10:19:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18ceffd640 1 -- 172.21.15.133:0/3894877636 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c80aa920 con 0x7f18b80051d0 2024-04-25T10:19:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d63e6640 1 -- 172.21.15.133:0/3894877636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18b80051d0 msgr2=0x7f18b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d63e6640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18b80051d0 0x7f18b800b9d0 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 crypto rx=0x7f18c800aa30 tx=0x7f18c801fae0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d63e6640 1 -- 172.21.15.133:0/3894877636 shutdown_connections 2024-04-25T10:19:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d63e6640 1 --2- 172.21.15.133:0/3894877636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18b80051d0 0x7f18b800b9d0 unknown :-1 s=CLOSED pgs=3050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d63e6640 1 -- 172.21.15.133:0/3894877636 >> 172.21.15.133:0/3894877636 conn(0x7f18d0066f70 msgr2=0x7f18d006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:14.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d63e6640 1 -- 172.21.15.133:0/3894877636 shutdown_connections 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.577+0000 7f18d63e6640 1 -- 172.21.15.133:0/3894877636 wait complete. 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d63e6640 1 Processor -- start 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d63e6640 1 -- start start 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d63e6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18b80051d0 0x7f18d0157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d63e6640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18d015d520 0x7f18d00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d63e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18d015c690 0x7f18d015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d63e6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00b21c0 con 0x7f18d015c690 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d63e6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00cdf30 con 0x7f18b80051d0 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d63e6640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00b2340 con 0x7f18d015d520 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d4983640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18d015c690 0x7f18d015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d4983640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18d015c690 0x7f18d015cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46806/0 (socket says 172.21.15.133:46806) 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d4983640 1 -- 172.21.15.133:0/708070031 learned_addr learned my addr 172.21.15.133:0/708070031 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18cf7fe640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18d015d520 0x7f18d00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18cffff640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18b80051d0 0x7f18d0157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d4983640 1 -- 172.21.15.133:0/708070031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18d015d520 msgr2=0x7f18d00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d4983640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18d015d520 0x7f18d00b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d4983640 1 -- 172.21.15.133:0/708070031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18b80051d0 msgr2=0x7f18d0157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d4983640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18b80051d0 0x7f18d0157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d4983640 1 -- 172.21.15.133:0/708070031 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00b44a0 con 0x7f18d015c690 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18cffff640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18b80051d0 0x7f18d0157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d4983640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18d015c690 0x7f18d015cb20 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f18c801f910 tx=0x7f18b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18ccff9640 1 -- 172.21.15.133:0/708070031 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c800a650 con 0x7f18d015c690 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18ccff9640 1 -- 172.21.15.133:0/708070031 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c80014a0 con 0x7f18d015c690 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d63e6640 1 -- 172.21.15.133:0/708070031 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d00b3400 con 0x7f18d015c690 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18ccff9640 1 -- 172.21.15.133:0/708070031 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8004d70 con 0x7f18d015c690 2024-04-25T10:19:14.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d63e6640 1 -- 172.21.15.133:0/708070031 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d00b39c0 con 0x7f18d015c690 2024-04-25T10:19:14.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18d63e6640 1 -- 172.21.15.133:0/708070031 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d0068e80 con 0x7f18d015c690 2024-04-25T10:19:14.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18ccff9640 1 -- 172.21.15.133:0/708070031 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18c80bd060 con 0x7f18d015c690 2024-04-25T10:19:14.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.581+0000 7f18ccff9640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18bc066f60 0x7f18bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.585+0000 7f18ccff9640 1 -- 172.21.15.133:0/708070031 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f18c8121bd0 con 0x7f18d015c690 2024-04-25T10:19:14.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.585+0000 7f18cffff640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18bc066f60 0x7f18bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.585+0000 7f18cffff640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18bc066f60 0x7f18bc069420 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f18d0165680 tx=0x7f18c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.585+0000 7f18ccff9640 1 -- 172.21.15.133:0/708070031 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18d00cd6f0 con 0x7f18d015c690 2024-04-25T10:19:14.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.733+0000 7f18d63e6640 1 -- 172.21.15.133:0/708070031 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f18d006b960 con 0x7f18bc066f60 2024-04-25T10:19:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.733+0000 7f18ccff9640 1 -- 172.21.15.133:0/708070031 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f18d006b960 con 0x7f18bc066f60 2024-04-25T10:19:14.735 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:19:14.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 -- 172.21.15.133:0/708070031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18bc066f60 msgr2=0x7f18bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18bc066f60 0x7f18bc069420 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f18d0165680 tx=0x7f18c4008040 comp rx=0 tx=0).stop 2024-04-25T10:19:14.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 -- 172.21.15.133:0/708070031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18d015c690 msgr2=0x7f18d015cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18d015c690 0x7f18d015cb20 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f18c801f910 tx=0x7f18b8005840 comp rx=0 tx=0).stop 2024-04-25T10:19:14.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 -- 172.21.15.133:0/708070031 shutdown_connections 2024-04-25T10:19:14.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18bc066f60 0x7f18bc069420 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18d015c690 0x7f18d015cb20 unknown :-1 s=CLOSED pgs=3051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18d015d520 0x7f18d00b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 --2- 172.21.15.133:0/708070031 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18b80051d0 0x7f18d0157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 -- 172.21.15.133:0/708070031 >> 172.21.15.133:0/708070031 conn(0x7f18d0066f70 msgr2=0x7f18d0160830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:14.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 -- 172.21.15.133:0/708070031 shutdown_connections 2024-04-25T10:19:14.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.737+0000 7f18d63e6640 1 -- 172.21.15.133:0/708070031 wait complete. 2024-04-25T10:19:14.753 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:19:14.753 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:19:14.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd99788640 1 Processor -- start 2024-04-25T10:19:14.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd99788640 1 -- start start 2024-04-25T10:19:14.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd99788640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7efd940b4940 0x7efd940b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd99788640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd94071cb0 0x7efd9415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd99788640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd9415c250 0x7efd940cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd99788640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd94072a90 con 0x7efd9415c250 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd99788640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd940a85d0 con 0x7efd940b4940 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd99788640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd940b5260 con 0x7efd94071cb0 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd937fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd9415c250 0x7efd940cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd937fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd9415c250 0x7efd940cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46808/0 (socket says 172.21.15.133:46808) 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd937fe640 1 -- 172.21.15.133:0/3090524075 learned_addr learned my addr 172.21.15.133:0/3090524075 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd92ffd640 1 --2- 172.21.15.133:0/3090524075 >> v2:172.21.15.162:3300/0 conn(0x7efd940b4940 0x7efd940b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd8a7fc640 1 --2- 172.21.15.133:0/3090524075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd94071cb0 0x7efd9415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd937fe640 1 -- 172.21.15.133:0/3090524075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd94071cb0 msgr2=0x7efd9415bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd937fe640 1 --2- 172.21.15.133:0/3090524075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd94071cb0 0x7efd9415bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd937fe640 1 -- 172.21.15.133:0/3090524075 >> v2:172.21.15.162:3300/0 conn(0x7efd940b4940 msgr2=0x7efd940b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd937fe640 1 --2- 172.21.15.133:0/3090524075 >> v2:172.21.15.162:3300/0 conn(0x7efd940b4940 0x7efd940b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd937fe640 1 -- 172.21.15.133:0/3090524075 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9415c9b0 con 0x7efd9415c250 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd92ffd640 1 --2- 172.21.15.133:0/3090524075 >> v2:172.21.15.162:3300/0 conn(0x7efd940b4940 0x7efd940b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:14.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.869+0000 7efd8a7fc640 1 --2- 172.21.15.133:0/3090524075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd94071cb0 0x7efd9415bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:14.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd937fe640 1 --2- 172.21.15.133:0/3090524075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd9415c250 0x7efd940cc010 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7efd94070a40 tx=0x7efd8c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea294290cd1782b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd927fc640 1 -- 172.21.15.133:0/3090524075 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0a5840 con 0x7efd9415c250 2024-04-25T10:19:14.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd927fc640 1 -- 172.21.15.133:0/3090524075 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c0a59e0 con 0x7efd9415c250 2024-04-25T10:19:14.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd927fc640 1 -- 172.21.15.133:0/3090524075 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0a5db0 con 0x7efd9415c250 2024-04-25T10:19:14.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd99788640 1 -- 172.21.15.133:0/3090524075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd9415c250 msgr2=0x7efd940cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd99788640 1 --2- 172.21.15.133:0/3090524075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd9415c250 0x7efd940cc010 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7efd94070a40 tx=0x7efd8c01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd99788640 1 -- 172.21.15.133:0/3090524075 shutdown_connections 2024-04-25T10:19:14.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd99788640 1 --2- 172.21.15.133:0/3090524075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd9415c250 0x7efd940cc010 unknown :-1 s=CLOSED pgs=3052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd99788640 1 --2- 172.21.15.133:0/3090524075 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd94071cb0 0x7efd9415bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd99788640 1 --2- 172.21.15.133:0/3090524075 >> v2:172.21.15.162:3300/0 conn(0x7efd940b4940 0x7efd940b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd99788640 1 -- 172.21.15.133:0/3090524075 >> 172.21.15.133:0/3090524075 conn(0x7efd9406d370 msgr2=0x7efd9406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:14.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd99788640 1 -- 172.21.15.133:0/3090524075 shutdown_connections 2024-04-25T10:19:14.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd99788640 1 -- 172.21.15.133:0/3090524075 wait complete. 2024-04-25T10:19:14.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd99788640 1 Processor -- start 2024-04-25T10:19:14.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.873+0000 7efd99788640 1 -- start start 2024-04-25T10:19:14.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd99788640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd94071cb0 0x7efd9415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd99788640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd940b4940 0x7efd940ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd99788640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd9415c250 0x7efd94143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd99788640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9415c680 con 0x7efd94071cb0 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd99788640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd94071790 con 0x7efd940b4940 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd99788640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd940cc960 con 0x7efd9415c250 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd937fe640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd9415c250 0x7efd94143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd92ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd94071cb0 0x7efd9415a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd92ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd94071cb0 0x7efd9415a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46814/0 (socket says 172.21.15.133:46814) 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8a7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd940b4940 0x7efd940ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd92ffd640 1 -- 172.21.15.133:0/2951970051 learned_addr learned my addr 172.21.15.133:0/2951970051 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd92ffd640 1 -- 172.21.15.133:0/2951970051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd9415c250 msgr2=0x7efd94143a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd92ffd640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7efd9415c250 0x7efd94143a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd92ffd640 1 -- 172.21.15.133:0/2951970051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd940b4940 msgr2=0x7efd940ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd92ffd640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efd940b4940 0x7efd940ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd92ffd640 1 -- 172.21.15.133:0/2951970051 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd940ad070 con 0x7efd94071cb0 2024-04-25T10:19:14.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd92ffd640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd94071cb0 0x7efd9415a1d0 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7efd8400c7f0 tx=0x7efd8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8bfff640 1 -- 172.21.15.133:0/2951970051 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84007a30 con 0x7efd94071cb0 2024-04-25T10:19:14.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8bfff640 1 -- 172.21.15.133:0/2951970051 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd84007bd0 con 0x7efd94071cb0 2024-04-25T10:19:14.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd99788640 1 -- 172.21.15.133:0/2951970051 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd940ad350 con 0x7efd94071cb0 2024-04-25T10:19:14.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8bfff640 1 -- 172.21.15.133:0/2951970051 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd840113f0 con 0x7efd94071cb0 2024-04-25T10:19:14.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd99788640 1 -- 172.21.15.133:0/2951970051 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd940adbf0 con 0x7efd94071cb0 2024-04-25T10:19:14.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd99788640 1 -- 172.21.15.133:0/2951970051 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd94069000 con 0x7efd94071cb0 2024-04-25T10:19:14.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8bfff640 1 -- 172.21.15.133:0/2951970051 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd84011590 con 0x7efd94071cb0 2024-04-25T10:19:14.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8bfff640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efd74066f60 0x7efd74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8a7fc640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efd74066f60 0x7efd74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8bfff640 1 -- 172.21.15.133:0/2951970051 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7efd84085600 con 0x7efd94071cb0 2024-04-25T10:19:14.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8bfff640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] conn(0x7efd7406d4a0 0x7efd7406f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:14.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8bfff640 1 -- 172.21.15.133:0/2951970051 --> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd84018ec0 con 0x7efd7406d4a0 2024-04-25T10:19:14.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8bfff640 1 -- 172.21.15.133:0/2951970051 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd84085a90 con 0x7efd94071cb0 2024-04-25T10:19:14.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd937fe640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] conn(0x7efd7406d4a0 0x7efd7406f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:14.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.877+0000 7efd8a7fc640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efd74066f60 0x7efd74069420 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7efd94164f10 tx=0x7efd7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.881+0000 7efd937fe640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] conn(0x7efd7406d4a0 0x7efd7406f8e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:14.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.881+0000 7efd8bfff640 1 -- 172.21.15.133:0/2951970051 <== osd.1 v2:172.21.15.133:6800/2082178647 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efd84018ec0 con 0x7efd7406d4a0 2024-04-25T10:19:14.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd99788640 1 -- 172.21.15.133:0/2951970051 --> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efd940cbff0 con 0x7efd7406d4a0 2024-04-25T10:19:14.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd8bfff640 1 -- 172.21.15.133:0/2951970051 <== osd.1 v2:172.21.15.133:6800/2082178647 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7efd940cbff0 con 0x7efd7406d4a0 2024-04-25T10:19:14.895 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:19:14.895 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:19:14.895 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:19:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd99788640 1 -- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] conn(0x7efd7406d4a0 msgr2=0x7efd7406f8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd99788640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:6800/2082178647,v1:172.21.15.133:6801/2082178647] conn(0x7efd7406d4a0 0x7efd7406f8e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd99788640 1 -- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efd74066f60 msgr2=0x7efd74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd99788640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7efd74066f60 0x7efd74069420 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7efd94164f10 tx=0x7efd7c008040 comp rx=0 tx=0).stop 2024-04-25T10:19:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd99788640 1 -- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd94071cb0 msgr2=0x7efd9415a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd99788640 1 --2- 172.21.15.133:0/2951970051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd94071cb0 0x7efd9415a1d0 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7efd8400c7f0 tx=0x7efd8400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:19:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd937fe640 1 -- 172.21.15.133:0/2951970051 reap_dead start 2024-04-25T10:19:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd99788640 1 -- 172.21.15.133:0/2951970051 shutdown_connections 2024-04-25T10:19:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd99788640 1 -- 172.21.15.133:0/2951970051 >> 172.21.15.133:0/2951970051 conn(0x7efd9406d370 msgr2=0x7efd9413f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:14.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd99788640 1 -- 172.21.15.133:0/2951970051 shutdown_connections 2024-04-25T10:19:14.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:14.893+0000 7efd99788640 1 -- 172.21.15.133:0/2951970051 wait complete. 2024-04-25T10:19:14.910 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:19:15.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.021+0000 7f48a7e07640 1 Processor -- start 2024-04-25T10:19:15.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.021+0000 7f48a7e07640 1 -- start start 2024-04-25T10:19:15.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.021+0000 7f48a7e07640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f48a0145170 0x7f48a006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.021+0000 7f48a7e07640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 0x7f48a00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.021+0000 7f48a7e07640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a00ce2e0 con 0x7f48a00cdd60 2024-04-25T10:19:15.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.021+0000 7f48a7e07640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a006db00 con 0x7f48a0145170 2024-04-25T10:19:15.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.021+0000 7f48a7e07640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0145550 con 0x7f48a006e4d0 2024-04-25T10:19:15.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a53a2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 0x7f48a00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a53a2640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 0x7f48a00cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3301/0 says I am v2:172.21.15.133:52484/0 (socket says 172.21.15.133:52484) 2024-04-25T10:19:15.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a53a2640 1 -- 172.21.15.133:0/117700912 learned_addr learned my addr 172.21.15.133:0/117700912 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:15.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a5ba3640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f48a0145170 0x7f48a006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a53a2640 1 -- 172.21.15.133:0/117700912 >> v2:172.21.15.162:3300/0 conn(0x7f48a0145170 msgr2=0x7f48a006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:15.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a53a2640 1 --2- 172.21.15.133:0/117700912 >> v2:172.21.15.162:3300/0 conn(0x7f48a0145170 0x7f48a006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a53a2640 1 -- 172.21.15.133:0/117700912 >> v1:172.21.15.162:6789/0 conn(0x7f48a00cdd60 legacy=0x7f48a015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a53a2640 1 -- 172.21.15.133:0/117700912 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a006d4b0 con 0x7f48a006e4d0 2024-04-25T10:19:15.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a5ba3640 1 --2- 172.21.15.133:0/117700912 >> v2:172.21.15.162:3300/0 conn(0x7f48a0145170 0x7f48a006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:15.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a53a2640 1 --2- 172.21.15.133:0/117700912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 0x7f48a00cd820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4890009870 tx=0x7f489001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5362b6e99e4e082 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a4ba1640 1 -- 172.21.15.133:0/117700912 <== mon.2 v2:172.21.15.162:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900b2070 con 0x7f48a006e4d0 2024-04-25T10:19:15.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a4ba1640 1 -- 172.21.15.133:0/117700912 <== mon.2 v2:172.21.15.162:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48900a9070 con 0x7f48a006e4d0 2024-04-25T10:19:15.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a4ba1640 1 -- 172.21.15.133:0/117700912 <== mon.2 v2:172.21.15.162:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900ad880 con 0x7f48a006e4d0 2024-04-25T10:19:15.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a7e07640 1 -- 172.21.15.133:0/117700912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 msgr2=0x7f48a00cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a7e07640 1 --2- 172.21.15.133:0/117700912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 0x7f48a00cd820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4890009870 tx=0x7f489001c560 comp rx=0 tx=0).stop 2024-04-25T10:19:15.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a7e07640 1 -- 172.21.15.133:0/117700912 shutdown_connections 2024-04-25T10:19:15.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a7e07640 1 --2- 172.21.15.133:0/117700912 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 0x7f48a00cd820 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a7e07640 1 --2- 172.21.15.133:0/117700912 >> v2:172.21.15.162:3300/0 conn(0x7f48a0145170 0x7f48a006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a7e07640 1 -- 172.21.15.133:0/117700912 >> 172.21.15.133:0/117700912 conn(0x7f48a0066f70 msgr2=0x7f48a006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.025+0000 7f48a7e07640 1 -- 172.21.15.133:0/117700912 shutdown_connections 2024-04-25T10:19:15.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a7e07640 1 -- 172.21.15.133:0/117700912 wait complete. 2024-04-25T10:19:15.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a7e07640 1 Processor -- start 2024-04-25T10:19:15.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a7e07640 1 -- start start 2024-04-25T10:19:15.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a7e07640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 0x7f48a0165710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a7e07640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a00cdd60 0x7f48a0139d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a7e07640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48a0145170 0x7f48a0162f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a7e07640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0168490 con 0x7f48a00cdd60 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a7e07640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a015d7f0 con 0x7f48a0145170 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a7e07640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0168610 con 0x7f48a006e4d0 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a53a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a00cdd60 0x7f48a0139d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a53a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a00cdd60 0x7f48a0139d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46828/0 (socket says 172.21.15.133:46828) 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a53a2640 1 -- 172.21.15.133:0/857834814 learned_addr learned my addr 172.21.15.133:0/857834814 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a63a4640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48a0145170 0x7f48a0162f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a5ba3640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 0x7f48a0165710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a53a2640 1 -- 172.21.15.133:0/857834814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 msgr2=0x7f48a0165710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a53a2640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 0x7f48a0165710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a53a2640 1 -- 172.21.15.133:0/857834814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48a0145170 msgr2=0x7f48a0162f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a53a2640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48a0145170 0x7f48a0162f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a53a2640 1 -- 172.21.15.133:0/857834814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a01405b0 con 0x7f48a00cdd60 2024-04-25T10:19:15.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a63a4640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48a0145170 0x7f48a0162f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:15.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a53a2640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a00cdd60 0x7f48a0139d90 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7f48900b0040 tx=0x7f489001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48967fc640 1 -- 172.21.15.133:0/857834814 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900b2040 con 0x7f48a00cdd60 2024-04-25T10:19:15.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48967fc640 1 -- 172.21.15.133:0/857834814 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48900adb80 con 0x7f48a00cdd60 2024-04-25T10:19:15.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a7e07640 1 -- 172.21.15.133:0/857834814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a0140890 con 0x7f48a00cdd60 2024-04-25T10:19:15.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48967fc640 1 -- 172.21.15.133:0/857834814 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900b8420 con 0x7f48a00cdd60 2024-04-25T10:19:15.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.029+0000 7f48a7e07640 1 -- 172.21.15.133:0/857834814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a013a2d0 con 0x7f48a00cdd60 2024-04-25T10:19:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.033+0000 7f48a7e07640 1 -- 172.21.15.133:0/857834814 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a0068e80 con 0x7f48a00cdd60 2024-04-25T10:19:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.033+0000 7f48967fc640 1 -- 172.21.15.133:0/857834814 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48900c0030 con 0x7f48a00cdd60 2024-04-25T10:19:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.033+0000 7f48967fc640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4884066f90 0x7f4884069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.033+0000 7f48a5ba3640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4884066f90 0x7f4884069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.033+0000 7f48967fc640 1 -- 172.21.15.133:0/857834814 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f489011e710 con 0x7f48a00cdd60 2024-04-25T10:19:15.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.033+0000 7f48a5ba3640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4884066f90 0x7f4884069450 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f489c009580 tx=0x7f489c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.037+0000 7f48967fc640 1 -- 172.21.15.133:0/857834814 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48900c7060 con 0x7f48a00cdd60 2024-04-25T10:19:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 -- 172.21.15.133:0/857834814 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f48a015c3d0 con 0x7f4884066f90 2024-04-25T10:19:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48967fc640 1 -- 172.21.15.133:0/857834814 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f48a015c3d0 con 0x7f4884066f90 2024-04-25T10:19:15.186 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:19:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 -- 172.21.15.133:0/857834814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4884066f90 msgr2=0x7f4884069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4884066f90 0x7f4884069450 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f489c009580 tx=0x7f489c008040 comp rx=0 tx=0).stop 2024-04-25T10:19:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 -- 172.21.15.133:0/857834814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a00cdd60 msgr2=0x7f48a0139d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a00cdd60 0x7f48a0139d90 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7f48900b0040 tx=0x7f489001ada0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 -- 172.21.15.133:0/857834814 shutdown_connections 2024-04-25T10:19:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f48a0145170 0x7f48a0162f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4884066f90 0x7f4884069450 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a00cdd60 0x7f48a0139d90 unknown :-1 s=CLOSED pgs=3054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 --2- 172.21.15.133:0/857834814 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f48a006e4d0 0x7f48a0165710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 -- 172.21.15.133:0/857834814 >> 172.21.15.133:0/857834814 conn(0x7f48a0066f70 msgr2=0x7f48a0141140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 -- 172.21.15.133:0/857834814 shutdown_connections 2024-04-25T10:19:15.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.185+0000 7f48a7e07640 1 -- 172.21.15.133:0/857834814 wait complete. 2024-04-25T10:19:15.205 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:19:15.205 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:19:15.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.321+0000 7f4d507f3640 1 Processor -- start 2024-04-25T10:19:15.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.321+0000 7f4d507f3640 1 -- start start 2024-04-25T10:19:15.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.321+0000 7f4d507f3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4806b3a0 0x7f4d480cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.321+0000 7f4d507f3640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d480d0460 con 0x7f4d480cc720 2024-04-25T10:19:15.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.321+0000 7f4d507f3640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d480d0740 con 0x7f4d4815dd30 2024-04-25T10:19:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.321+0000 7f4d507f3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480cd040 con 0x7f4d4806b3a0 2024-04-25T10:19:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.321+0000 7f4d4ed90640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4d480cc720 0x7f4d480cb140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41028/0 (socket says 172.21.15.133:41028) 2024-04-25T10:19:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.321+0000 7f4d4ed90640 1 -- 172.21.15.133:0/535568069 learned_addr learned my addr 172.21.15.133:0/535568069 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752995710 0 0) 0x7f4d480d0460 con 0x7f4d480cc720 2024-04-25T10:19:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d34003530 con 0x7f4d480cc720 2024-04-25T10:19:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4dd8e640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4806b3a0 0x7f4d480cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132279844 0 0) 0x7f4d34003530 con 0x7f4d480cc720 2024-04-25T10:19:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4806b3a0 msgr2=0x7f4d480cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4806b3a0 0x7f4d480cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 >> v1:172.21.15.162:6789/0 conn(0x7f4d4815dd30 legacy=0x7f4d4806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d480d0a20 con 0x7f4d480cc720 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4d40001a00 con 0x7f4d480cc720 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4dd8e640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4806b3a0 0x7f4d480cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 >> v1:172.21.15.133:6789/0 conn(0x7f4d480cc720 legacy=0x7f4d480cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d340039f0 0x7f4d34003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4806b3a0 0x7f4d34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d34004d50 0x7f4d340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480cd040 con 0x7f4d34004d50 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3400bb60 con 0x7f4d340039f0 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3400bd30 con 0x7f4d4806b3a0 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4ed90640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d34004d50 0x7f4d340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4dd8e640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4806b3a0 0x7f4d34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4e58f640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d340039f0 0x7f4d34003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4ed90640 1 -- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4806b3a0 msgr2=0x7f4d34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4ed90640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d4806b3a0 0x7f4d34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4ed90640 1 -- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d340039f0 msgr2=0x7f4d34003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:15.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4ed90640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d340039f0 0x7f4d34003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4ed90640 1 -- 172.21.15.133:0/535568069 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d34003530 con 0x7f4d34004d50 2024-04-25T10:19:15.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4e58f640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d340039f0 0x7f4d34003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:15.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4ed90640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d34004d50 0x7f4d340051d0 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000a780 tx=0x7f4d4001d260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba841633018d5040 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000a3d0 con 0x7f4d34004d50 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d400ad630 con 0x7f4d34004d50 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4d58d640 1 -- 172.21.15.133:0/535568069 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d400ada60 con 0x7f4d34004d50 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d507f3640 1 -- 172.21.15.133:0/535568069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d34004d50 msgr2=0x7f4d340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d507f3640 1 --2- 172.21.15.133:0/535568069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d34004d50 0x7f4d340051d0 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000a780 tx=0x7f4d4001d260 comp rx=0 tx=0).stop 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d4ed90640 1 -- 172.21.15.133:0/535568069 reap_dead start 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d507f3640 1 -- 172.21.15.133:0/535568069 shutdown_connections 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d507f3640 1 -- 172.21.15.133:0/535568069 >> 172.21.15.133:0/535568069 conn(0x7f4d480726f0 msgr2=0x7f4d480999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d507f3640 1 -- 172.21.15.133:0/535568069 shutdown_connections 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.325+0000 7f4d507f3640 1 -- 172.21.15.133:0/535568069 wait complete. 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d507f3640 1 Processor -- start 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d507f3640 1 -- start start 2024-04-25T10:19:15.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d507f3640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d48153b70 0x7f4d480af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d507f3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d480adde0 0x7f4d480ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d507f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d480b30f0 0x7f4d480b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d507f3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480b4410 con 0x7f4d480b30f0 2024-04-25T10:19:15.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d507f3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480cc310 con 0x7f4d480adde0 2024-04-25T10:19:15.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d507f3640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480d1220 con 0x7f4d48153b70 2024-04-25T10:19:15.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d4ed90640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d480b30f0 0x7f4d480b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d4ed90640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d480b30f0 0x7f4d480b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46852/0 (socket says 172.21.15.133:46852) 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d4ed90640 1 -- 172.21.15.133:0/3389827564 learned_addr learned my addr 172.21.15.133:0/3389827564 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d4ed90640 1 -- 172.21.15.133:0/3389827564 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d48153b70 msgr2=0x7f4d480af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d4ed90640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4d48153b70 0x7f4d480af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d4ed90640 1 -- 172.21.15.133:0/3389827564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d480adde0 msgr2=0x7f4d480ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d4dd8e640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d480adde0 0x7f4d480ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d4ed90640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d480adde0 0x7f4d480ae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d4ed90640 1 -- 172.21.15.133:0/3389827564 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d480ae950 con 0x7f4d480b30f0 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d4ed90640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d480b30f0 0x7f4d480b3570 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000a780 tx=0x7f4d34005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d3affd640 1 -- 172.21.15.133:0/3389827564 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000a3d0 con 0x7f4d480b30f0 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d3affd640 1 -- 172.21.15.133:0/3389827564 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d400ad430 con 0x7f4d480b30f0 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d507f3640 1 -- 172.21.15.133:0/3389827564 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d480aec30 con 0x7f4d480b30f0 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d3affd640 1 -- 172.21.15.133:0/3389827564 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d400011d0 con 0x7f4d480b30f0 2024-04-25T10:19:15.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d507f3640 1 -- 172.21.15.133:0/3389827564 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d480af1f0 con 0x7f4d480b30f0 2024-04-25T10:19:15.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d507f3640 1 -- 172.21.15.133:0/3389827564 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4d48069000 con 0x7f4d480b30f0 2024-04-25T10:19:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.329+0000 7f4d3affd640 1 -- 172.21.15.133:0/3389827564 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d400bd030 con 0x7f4d480b30f0 2024-04-25T10:19:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.333+0000 7f4d3affd640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d2c066f60 0x7f4d2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.333+0000 7f4d3affd640 1 -- 172.21.15.133:0/3389827564 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4d4011ecd0 con 0x7f4d480b30f0 2024-04-25T10:19:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.333+0000 7f4d4e58f640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d2c066f60 0x7f4d2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.333+0000 7f4d3affd640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x7f4d2c06d4a0 0x7f4d2c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.333+0000 7f4d3affd640 1 -- 172.21.15.133:0/3389827564 --> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d2c06ffb0 con 0x7f4d2c06d4a0 2024-04-25T10:19:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.333+0000 7f4d3affd640 1 -- 172.21.15.133:0/3389827564 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d401213d0 con 0x7f4d480b30f0 2024-04-25T10:19:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.333+0000 7f4d4dd8e640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x7f4d2c06d4a0 0x7f4d2c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.333+0000 7f4d4e58f640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d2c066f60 0x7f4d2c069420 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4d44001700 tx=0x7f4d44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.333+0000 7f4d4dd8e640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x7f4d2c06d4a0 0x7f4d2c06f8e0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.333+0000 7f4d3affd640 1 -- 172.21.15.133:0/3389827564 <== osd.2 v2:172.21.15.133:6817/148430844 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4d2c06ffb0 con 0x7f4d2c06d4a0 2024-04-25T10:19:15.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d507f3640 1 -- 172.21.15.133:0/3389827564 --> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4d4815ce10 con 0x7f4d2c06d4a0 2024-04-25T10:19:15.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d3affd640 1 -- 172.21.15.133:0/3389827564 <== osd.2 v2:172.21.15.133:6817/148430844 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4d4815ce10 con 0x7f4d2c06d4a0 2024-04-25T10:19:15.347 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:19:15.347 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:19:15.347 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:19:15.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d507f3640 1 -- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x7f4d2c06d4a0 msgr2=0x7f4d2c06f8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d507f3640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:6817/148430844,v1:172.21.15.133:6819/148430844] conn(0x7f4d2c06d4a0 0x7f4d2c06f8e0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d507f3640 1 -- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d2c066f60 msgr2=0x7f4d2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d507f3640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f4d2c066f60 0x7f4d2c069420 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4d44001700 tx=0x7f4d44009040 comp rx=0 tx=0).stop 2024-04-25T10:19:15.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d507f3640 1 -- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d480b30f0 msgr2=0x7f4d480b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d507f3640 1 --2- 172.21.15.133:0/3389827564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d480b30f0 0x7f4d480b3570 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000a780 tx=0x7f4d34005840 comp rx=0 tx=0).stop 2024-04-25T10:19:15.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d4ed90640 1 -- 172.21.15.133:0/3389827564 reap_dead start 2024-04-25T10:19:15.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d507f3640 1 -- 172.21.15.133:0/3389827564 shutdown_connections 2024-04-25T10:19:15.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d507f3640 1 -- 172.21.15.133:0/3389827564 >> 172.21.15.133:0/3389827564 conn(0x7f4d480726f0 msgr2=0x7f4d480b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d507f3640 1 -- 172.21.15.133:0/3389827564 shutdown_connections 2024-04-25T10:19:15.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.345+0000 7f4d507f3640 1 -- 172.21.15.133:0/3389827564 wait complete. 2024-04-25T10:19:15.364 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:19:15.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffae27f0640 1 Processor -- start 2024-04-25T10:19:15.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffae27f0640 1 -- start start 2024-04-25T10:19:15.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffae27f0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffadc15bd30 0x7ffadc0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffae27f0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffadc15ccd0 con 0x7ffadc0cf930 2024-04-25T10:19:15.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffae27f0640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffadc15c430 con 0x7ffadc0cc7e0 2024-04-25T10:19:15.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffae27f0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffadc0cdfe0 con 0x7ffadc15bd30 2024-04-25T10:19:15.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffae0d8d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ffadc0cf930 0x7ffadc0d17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41030/0 (socket says 172.21.15.133:41030) 2024-04-25T10:19:15.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffadb7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffadc15bd30 0x7ffadc0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffae0d8d640 1 -- 172.21.15.133:0/444176604 learned_addr learned my addr 172.21.15.133:0/444176604 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffadb7fe640 1 -- 172.21.15.133:0/444176604 >> v1:172.21.15.162:6789/0 conn(0x7ffadc0cc7e0 legacy=0x7ffadc0b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffadb7fe640 1 -- 172.21.15.133:0/444176604 >> v1:172.21.15.133:6789/0 conn(0x7ffadc0cf930 legacy=0x7ffadc0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffadaffd640 1 -- 172.21.15.133:0/444176604 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393370092 0 0) 0x7ffadc15ccd0 con 0x7ffadc0cf930 2024-04-25T10:19:15.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.465+0000 7ffadb7fe640 1 -- 172.21.15.133:0/444176604 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc15c710 con 0x7ffadc15bd30 2024-04-25T10:19:15.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffadb7fe640 1 --2- 172.21.15.133:0/444176604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffadc15bd30 0x7ffadc0cf3f0 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7ffad00097f0 tx=0x7ffad001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba506ad581ecee84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffadaffd640 1 -- 172.21.15.133:0/444176604 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad00b0070 con 0x7ffadc15bd30 2024-04-25T10:19:15.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffadaffd640 1 -- 172.21.15.133:0/444176604 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad00ab600 con 0x7ffadc15bd30 2024-04-25T10:19:15.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffadaffd640 1 -- 172.21.15.133:0/444176604 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad00ab9d0 con 0x7ffadc15bd30 2024-04-25T10:19:15.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffae27f0640 1 -- 172.21.15.133:0/444176604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffadc15bd30 msgr2=0x7ffadc0cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffae27f0640 1 --2- 172.21.15.133:0/444176604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffadc15bd30 0x7ffadc0cf3f0 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7ffad00097f0 tx=0x7ffad001c460 comp rx=0 tx=0).stop 2024-04-25T10:19:15.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffae27f0640 1 -- 172.21.15.133:0/444176604 shutdown_connections 2024-04-25T10:19:15.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffae27f0640 1 --2- 172.21.15.133:0/444176604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffadc15bd30 0x7ffadc0cf3f0 unknown :-1 s=CLOSED pgs=3059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffae27f0640 1 -- 172.21.15.133:0/444176604 >> 172.21.15.133:0/444176604 conn(0x7ffadc0b3370 msgr2=0x7ffadc0b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffae27f0640 1 -- 172.21.15.133:0/444176604 shutdown_connections 2024-04-25T10:19:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffae27f0640 1 -- 172.21.15.133:0/444176604 wait complete. 2024-04-25T10:19:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffae27f0640 1 Processor -- start 2024-04-25T10:19:15.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.469+0000 7ffae27f0640 1 -- start start 2024-04-25T10:19:15.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffae27f0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffadc0cc7e0 0x7ffadc149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffae27f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffadc0cf930 0x7ffadc147050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffae27f0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffadc15bd30 0x7ffadc147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffae27f0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0d2040 con 0x7ffadc0cf930 2024-04-25T10:19:15.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffae27f0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0d1ec0 con 0x7ffadc0cc7e0 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffae27f0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc155410 con 0x7ffadc15bd30 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffadc0cf930 0x7ffadc147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffadc0cf930 0x7ffadc147050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46876/0 (socket says 172.21.15.133:46876) 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadb7fe640 1 -- 172.21.15.133:0/3099428257 learned_addr learned my addr 172.21.15.133:0/3099428257 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadb7fe640 1 -- 172.21.15.133:0/3099428257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffadc15bd30 msgr2=0x7ffadc147590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadbfff640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffadc0cc7e0 0x7ffadc149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadb7fe640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffadc15bd30 0x7ffadc147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadb7fe640 1 -- 172.21.15.133:0/3099428257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffadc0cc7e0 msgr2=0x7ffadc149ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadb7fe640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffadc0cc7e0 0x7ffadc149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadb7fe640 1 -- 172.21.15.133:0/3099428257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc0c7250 con 0x7ffadc0cf930 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadb7fe640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffadc0cf930 0x7ffadc147050 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7ffad0019070 tx=0x7ffad001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffad8ff9640 1 -- 172.21.15.133:0/3099428257 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad00b0050 con 0x7ffadc0cf930 2024-04-25T10:19:15.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffad8ff9640 1 -- 172.21.15.133:0/3099428257 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad00bc070 con 0x7ffadc0cf930 2024-04-25T10:19:15.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffae27f0640 1 -- 172.21.15.133:0/3099428257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffadc0c61b0 con 0x7ffadc0cf930 2024-04-25T10:19:15.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffad8ff9640 1 -- 172.21.15.133:0/3099428257 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad00b5a30 con 0x7ffadc0cf930 2024-04-25T10:19:15.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffae27f0640 1 -- 172.21.15.133:0/3099428257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffadc0c6770 con 0x7ffadc0cf930 2024-04-25T10:19:15.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffae27f0640 1 -- 172.21.15.133:0/3099428257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffadc068e80 con 0x7ffadc0cf930 2024-04-25T10:19:15.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffad8ff9640 1 -- 172.21.15.133:0/3099428257 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffad00ab4c0 con 0x7ffadc0cf930 2024-04-25T10:19:15.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffad8ff9640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffac8066f60 0x7ffac8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffad8ff9640 1 -- 172.21.15.133:0/3099428257 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ffad011f8b0 con 0x7ffadc0cf930 2024-04-25T10:19:15.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadbfff640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffac8066f60 0x7ffac8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.473+0000 7ffadbfff640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffac8066f60 0x7ffac8069420 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ffacc004450 tx=0x7ffacc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.477+0000 7ffad8ff9640 1 -- 172.21.15.133:0/3099428257 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffad0016050 con 0x7ffadc0cf930 2024-04-25T10:19:15.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.625+0000 7ffae27f0640 1 -- 172.21.15.133:0/3099428257 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7ffadc14d450 con 0x7ffac8066f60 2024-04-25T10:19:15.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.625+0000 7ffad8ff9640 1 -- 172.21.15.133:0/3099428257 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ffadc14d450 con 0x7ffac8066f60 2024-04-25T10:19:15.628 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 -- 172.21.15.133:0/3099428257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffac8066f60 msgr2=0x7ffac8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffac8066f60 0x7ffac8069420 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ffacc004450 tx=0x7ffacc009040 comp rx=0 tx=0).stop 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 -- 172.21.15.133:0/3099428257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffadc0cf930 msgr2=0x7ffadc147050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffadc0cf930 0x7ffadc147050 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7ffad0019070 tx=0x7ffad001aca0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 -- 172.21.15.133:0/3099428257 shutdown_connections 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7ffadc15bd30 0x7ffadc147590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7ffac8066f60 0x7ffac8069420 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffadc0cf930 0x7ffadc147050 unknown :-1 s=CLOSED pgs=3060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 --2- 172.21.15.133:0/3099428257 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffadc0cc7e0 0x7ffadc149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 -- 172.21.15.133:0/3099428257 >> 172.21.15.133:0/3099428257 conn(0x7ffadc0b3370 msgr2=0x7ffadc0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 -- 172.21.15.133:0/3099428257 shutdown_connections 2024-04-25T10:19:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.629+0000 7ffae27f0640 1 -- 172.21.15.133:0/3099428257 wait complete. 2024-04-25T10:19:15.645 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:19:15.646 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:19:15.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7f992640 1 Processor -- start 2024-04-25T10:19:15.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7f992640 1 -- start start 2024-04-25T10:19:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7f992640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b7815d8c0 0x7f7b78161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7f992640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f7b780cef10 0x7f7b780d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7f992640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b780d1cf0 con 0x7f7b78161c60 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7f992640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b781626f0 con 0x7f7b780cef10 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7f992640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78162130 con 0x7f7b7815d8c0 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7cf2d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7b78161c60 0x7f7b780ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41046/0 (socket says 172.21.15.133:41046) 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7cf2d640 1 -- 172.21.15.133:0/2445722368 learned_addr learned my addr 172.21.15.133:0/2445722368 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386255438 0 0) 0x7f7b780d1cf0 con 0x7f7b78161c60 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7d72e640 1 --2- 172.21.15.133:0/2445722368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b7815d8c0 0x7f7b78161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7df2f640 1 --2- 172.21.15.133:0/2445722368 >> v2:172.21.15.162:3300/0 conn(0x7f7b780cef10 0x7f7b780d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b64003530 con 0x7f7b78161c60 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1813677028 0 0) 0x7f7b64003530 con 0x7f7b78161c60 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b7815d8c0 msgr2=0x7f7b78161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 --2- 172.21.15.133:0/2445722368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b7815d8c0 0x7f7b78161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 >> v2:172.21.15.162:3300/0 conn(0x7f7b780cef10 msgr2=0x7f7b780d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 --2- 172.21.15.133:0/2445722368 >> v2:172.21.15.162:3300/0 conn(0x7f7b780cef10 0x7f7b780d24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b78162410 con 0x7f7b78161c60 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f7b68001680 con 0x7f7b78161c60 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 >> v1:172.21.15.133:6789/0 conn(0x7f7b78161c60 legacy=0x7f7b780ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 --2- 172.21.15.133:0/2445722368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b7815d8c0 0x7f7b640039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 --2- 172.21.15.133:0/2445722368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b64003f40 0x7f7b64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 --2- 172.21.15.133:0/2445722368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b64004d50 0x7f7b640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b781626f0 con 0x7f7b64004d50 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78162130 con 0x7f7b64003f40 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6400ba20 con 0x7f7b7815d8c0 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7df2f640 1 --2- 172.21.15.133:0/2445722368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b64004d50 0x7f7b640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7df2f640 1 -- 172.21.15.133:0/2445722368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b7815d8c0 msgr2=0x7f7b640039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7df2f640 1 --2- 172.21.15.133:0/2445722368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b7815d8c0 0x7f7b640039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7df2f640 1 -- 172.21.15.133:0/2445722368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b64003f40 msgr2=0x7f7b64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7df2f640 1 --2- 172.21.15.133:0/2445722368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b64003f40 0x7f7b64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.769+0000 7f7b7df2f640 1 -- 172.21.15.133:0/2445722368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64003530 con 0x7f7b64004d50 2024-04-25T10:19:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b7df2f640 1 --2- 172.21.15.133:0/2445722368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b64004d50 0x7f7b640051d0 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009fc0 tx=0x7f7b7001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35993020e213953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b700b2070 con 0x7f7b64004d50 2024-04-25T10:19:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b700a5c90 con 0x7f7b64004d50 2024-04-25T10:19:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b6ffff640 1 -- 172.21.15.133:0/2445722368 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70009c10 con 0x7f7b64004d50 2024-04-25T10:19:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b7f992640 1 -- 172.21.15.133:0/2445722368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b64004d50 msgr2=0x7f7b640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b7f992640 1 --2- 172.21.15.133:0/2445722368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b64004d50 0x7f7b640051d0 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009fc0 tx=0x7f7b7001cc80 comp rx=0 tx=0).stop 2024-04-25T10:19:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b7df2f640 1 -- 172.21.15.133:0/2445722368 reap_dead start 2024-04-25T10:19:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b7f992640 1 -- 172.21.15.133:0/2445722368 shutdown_connections 2024-04-25T10:19:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b7f992640 1 -- 172.21.15.133:0/2445722368 >> 172.21.15.133:0/2445722368 conn(0x7f7b780723c0 msgr2=0x7f7b780727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b7f992640 1 -- 172.21.15.133:0/2445722368 shutdown_connections 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b7f992640 1 -- 172.21.15.133:0/2445722368 wait complete. 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b7f992640 1 Processor -- start 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.773+0000 7f7b7f992640 1 -- start start 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7f992640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b64004d50 0x7f7b780c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7f992640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b781520b0 0x7f7b78152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7f992640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b7815f880 0x7f7b7815fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7f992640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780b9990 con 0x7f7b7815f880 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7f992640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780b9810 con 0x7f7b64004d50 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7f992640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7815dca0 con 0x7f7b781520b0 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7df2f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b7815f880 0x7f7b7815fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7df2f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b7815f880 0x7f7b7815fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46906/0 (socket says 172.21.15.133:46906) 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7df2f640 1 -- 172.21.15.133:0/1604466725 learned_addr learned my addr 172.21.15.133:0/1604466725 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7cf2d640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b781520b0 0x7f7b78152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7d72e640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b64004d50 0x7f7b780c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7df2f640 1 -- 172.21.15.133:0/1604466725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b781520b0 msgr2=0x7f7b78152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7df2f640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f7b781520b0 0x7f7b78152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7df2f640 1 -- 172.21.15.133:0/1604466725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b64004d50 msgr2=0x7f7b780c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7df2f640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7b64004d50 0x7f7b780c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7df2f640 1 -- 172.21.15.133:0/1604466725 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b781605a0 con 0x7f7b7815f880 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7df2f640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b7815f880 0x7f7b7815fd00 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7f7b70002240 tx=0x7f7b70009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b6dffb640 1 -- 172.21.15.133:0/1604466725 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b700bd070 con 0x7f7b7815f880 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b6dffb640 1 -- 172.21.15.133:0/1604466725 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b700a5a90 con 0x7f7b7815f880 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7f992640 1 -- 172.21.15.133:0/1604466725 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7814f6a0 con 0x7f7b7815f880 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b6dffb640 1 -- 172.21.15.133:0/1604466725 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b700b2040 con 0x7f7b7815f880 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7f992640 1 -- 172.21.15.133:0/1604466725 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b78160880 con 0x7f7b7815f880 2024-04-25T10:19:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7f992640 1 -- 172.21.15.133:0/1604466725 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b78069000 con 0x7f7b7815f880 2024-04-25T10:19:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b6dffb640 1 -- 172.21.15.133:0/1604466725 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b700c0070 con 0x7f7b7815f880 2024-04-25T10:19:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b6dffb640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7b5c066f60 0x7f7b5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b6dffb640 1 -- 172.21.15.133:0/1604466725 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7b7011f120 con 0x7f7b7815f880 2024-04-25T10:19:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7d72e640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7b5c066f60 0x7f7b5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b6dffb640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] conn(0x7f7b5c06d4a0 0x7f7b5c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b6dffb640 1 -- 172.21.15.133:0/1604466725 --> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b5c06ffb0 con 0x7f7b5c06d4a0 2024-04-25T10:19:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b6dffb640 1 -- 172.21.15.133:0/1604466725 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b7011f5b0 con 0x7f7b7815f880 2024-04-25T10:19:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7cf2d640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] conn(0x7f7b5c06d4a0 0x7f7b5c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7d72e640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7b5c066f60 0x7f7b5c069420 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f7b740047c0 tx=0x7f7b74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b7cf2d640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] conn(0x7f7b5c06d4a0 0x7f7b5c06f8e0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.777+0000 7f7b6dffb640 1 -- 172.21.15.133:0/1604466725 <== osd.3 v2:172.21.15.133:6802/447485461 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7b5c06ffb0 con 0x7f7b5c06d4a0 2024-04-25T10:19:15.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7f992640 1 -- 172.21.15.133:0/1604466725 --> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7b780c6fa0 con 0x7f7b5c06d4a0 2024-04-25T10:19:15.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b6dffb640 1 -- 172.21.15.133:0/1604466725 <== osd.3 v2:172.21.15.133:6802/447485461 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7b780c6fa0 con 0x7f7b5c06d4a0 2024-04-25T10:19:15.793 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:19:15.793 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:19:15.793 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:19:15.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7f992640 1 -- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] conn(0x7f7b5c06d4a0 msgr2=0x7f7b5c06f8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7f992640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:6802/447485461,v1:172.21.15.133:6804/447485461] conn(0x7f7b5c06d4a0 0x7f7b5c06f8e0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7f992640 1 -- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7b5c066f60 msgr2=0x7f7b5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7f992640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f7b5c066f60 0x7f7b5c069420 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f7b740047c0 tx=0x7f7b74008040 comp rx=0 tx=0).stop 2024-04-25T10:19:15.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7f992640 1 -- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b7815f880 msgr2=0x7f7b7815fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7f992640 1 --2- 172.21.15.133:0/1604466725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b7815f880 0x7f7b7815fd00 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7f7b70002240 tx=0x7f7b70009750 comp rx=0 tx=0).stop 2024-04-25T10:19:15.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7df2f640 1 -- 172.21.15.133:0/1604466725 reap_dead start 2024-04-25T10:19:15.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7f992640 1 -- 172.21.15.133:0/1604466725 shutdown_connections 2024-04-25T10:19:15.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7f992640 1 -- 172.21.15.133:0/1604466725 >> 172.21.15.133:0/1604466725 conn(0x7f7b780723c0 msgr2=0x7f7b780735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7f992640 1 -- 172.21.15.133:0/1604466725 shutdown_connections 2024-04-25T10:19:15.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.793+0000 7f7b7f992640 1 -- 172.21.15.133:0/1604466725 wait complete. 2024-04-25T10:19:15.808 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:19:15.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.909+0000 7f17d3d8d640 1 Processor -- start 2024-04-25T10:19:15.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.909+0000 7f17d3d8d640 1 -- start start 2024-04-25T10:19:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.909+0000 7f17d3d8d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17cc143ea0 0x7f17cc0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.909+0000 7f17d3d8d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f17cc06eac0 0x7f17cc0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.909+0000 7f17d3d8d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17cc15ac50 con 0x7f17cc0b2150 2024-04-25T10:19:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.909+0000 7f17d3d8d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc0cd180 con 0x7f17cc06eac0 2024-04-25T10:19:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.909+0000 7f17d3d8d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc15aed0 con 0x7f17cc143ea0 2024-04-25T10:19:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.909+0000 7f17d232a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f17cc06eac0 0x7f17cc0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.909+0000 7f17d1328640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f17cc0b2150 0x7f17cc06e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41052/0 (socket says 172.21.15.133:41052) 2024-04-25T10:19:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.909+0000 7f17d1328640 1 -- 172.21.15.133:0/1478659646 learned_addr learned my addr 172.21.15.133:0/1478659646 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d1b29640 1 --2- 172.21.15.133:0/1478659646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17cc143ea0 0x7f17cc0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d232a640 1 -- 172.21.15.133:0/1478659646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17cc143ea0 msgr2=0x7f17cc0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:15.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d232a640 1 --2- 172.21.15.133:0/1478659646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17cc143ea0 0x7f17cc0cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d232a640 1 -- 172.21.15.133:0/1478659646 >> v1:172.21.15.133:6789/0 conn(0x7f17cc0b2150 legacy=0x7f17cc06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d232a640 1 -- 172.21.15.133:0/1478659646 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc15b1b0 con 0x7f17cc06eac0 2024-04-25T10:19:15.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d0b27640 1 -- 172.21.15.133:0/1478659646 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332636466 0 0) 0x7f17bc001120 con 0x7f17cc0b2150 2024-04-25T10:19:15.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d232a640 1 --2- 172.21.15.133:0/1478659646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17cc06eac0 0x7f17cc0b06d0 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009d90 tx=0x7f17c401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cd4c4cef64d5eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d0b27640 1 -- 172.21.15.133:0/1478659646 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40b2070 con 0x7f17cc06eac0 2024-04-25T10:19:15.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d0b27640 1 -- 172.21.15.133:0/1478659646 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c40a5940 con 0x7f17cc06eac0 2024-04-25T10:19:15.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d0b27640 1 -- 172.21.15.133:0/1478659646 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40a5d10 con 0x7f17cc06eac0 2024-04-25T10:19:15.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1478659646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17cc06eac0 msgr2=0x7f17cc0b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d3d8d640 1 --2- 172.21.15.133:0/1478659646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17cc06eac0 0x7f17cc0b06d0 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009d90 tx=0x7f17c401c970 comp rx=0 tx=0).stop 2024-04-25T10:19:15.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1478659646 shutdown_connections 2024-04-25T10:19:15.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d3d8d640 1 --2- 172.21.15.133:0/1478659646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17cc06eac0 0x7f17cc0b06d0 unknown :-1 s=CLOSED pgs=3065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d3d8d640 1 --2- 172.21.15.133:0/1478659646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17cc143ea0 0x7f17cc0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1478659646 >> 172.21.15.133:0/1478659646 conn(0x7f17cc066f70 msgr2=0x7f17cc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:15.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1478659646 shutdown_connections 2024-04-25T10:19:15.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.913+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1478659646 wait complete. 2024-04-25T10:19:15.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d3d8d640 1 Processor -- start 2024-04-25T10:19:15.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d3d8d640 1 -- start start 2024-04-25T10:19:15.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d3d8d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17cc06eac0 0x7f17cc142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d3d8d640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17cc0b2150 0x7f17cc1647e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d3d8d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17cc143ea0 0x7f17cc1623d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d3d8d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc167550 con 0x7f17cc143ea0 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d3d8d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc1673d0 con 0x7f17cc06eac0 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d3d8d640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc1676d0 con 0x7f17cc0b2150 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d232a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17cc143ea0 0x7f17cc1623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d232a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17cc143ea0 0x7f17cc1623d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46918/0 (socket says 172.21.15.133:46918) 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d232a640 1 -- 172.21.15.133:0/1501073551 learned_addr learned my addr 172.21.15.133:0/1501073551 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d232a640 1 -- 172.21.15.133:0/1501073551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17cc0b2150 msgr2=0x7f17cc1647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d1b29640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17cc06eac0 0x7f17cc142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d232a640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17cc0b2150 0x7f17cc1647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d232a640 1 -- 172.21.15.133:0/1501073551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17cc06eac0 msgr2=0x7f17cc142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d232a640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17cc06eac0 0x7f17cc142090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d232a640 1 -- 172.21.15.133:0/1501073551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc0b3460 con 0x7f17cc143ea0 2024-04-25T10:19:15.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d232a640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17cc143ea0 0x7f17cc1623d0 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7f17c401b020 tx=0x7f17c4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17c27fc640 1 -- 172.21.15.133:0/1501073551 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40b2050 con 0x7f17cc143ea0 2024-04-25T10:19:15.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17c27fc640 1 -- 172.21.15.133:0/1501073551 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c40be070 con 0x7f17cc143ea0 2024-04-25T10:19:15.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1501073551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc0b3740 con 0x7f17cc143ea0 2024-04-25T10:19:15.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17c27fc640 1 -- 172.21.15.133:0/1501073551 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40adad0 con 0x7f17cc143ea0 2024-04-25T10:19:15.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1501073551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc1425d0 con 0x7f17cc143ea0 2024-04-25T10:19:15.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1501073551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc068e80 con 0x7f17cc143ea0 2024-04-25T10:19:15.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17c27fc640 1 -- 172.21.15.133:0/1501073551 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17c40adc70 con 0x7f17cc143ea0 2024-04-25T10:19:15.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17c27fc640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f17b0066f90 0x7f17b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:15.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17c27fc640 1 -- 172.21.15.133:0/1501073551 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f17c411ff70 con 0x7f17cc143ea0 2024-04-25T10:19:15.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.917+0000 7f17d1b29640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f17b0066f90 0x7f17b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:15.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.921+0000 7f17d1b29640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f17b0066f90 0x7f17b0069450 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f17c800b800 tx=0x7f17c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:15.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:15.921+0000 7f17c27fc640 1 -- 172.21.15.133:0/1501073551 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17c4016560 con 0x7f17cc143ea0 2024-04-25T10:19:16.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1501073551 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f17cc06c2d0 con 0x7f17b0066f90 2024-04-25T10:19:16.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17c27fc640 1 -- 172.21.15.133:0/1501073551 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f17cc06c2d0 con 0x7f17b0066f90 2024-04-25T10:19:16.070 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:19:16.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1501073551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f17b0066f90 msgr2=0x7f17b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f17b0066f90 0x7f17b0069450 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f17c800b800 tx=0x7f17c8008040 comp rx=0 tx=0).stop 2024-04-25T10:19:16.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1501073551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17cc143ea0 msgr2=0x7f17cc1623d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17cc143ea0 0x7f17cc1623d0 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7f17c401b020 tx=0x7f17c4019710 comp rx=0 tx=0).stop 2024-04-25T10:19:16.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1501073551 shutdown_connections 2024-04-25T10:19:16.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f17b0066f90 0x7f17b0069450 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17cc143ea0 0x7f17cc1623d0 unknown :-1 s=CLOSED pgs=3066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f17cc0b2150 0x7f17cc1647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 --2- 172.21.15.133:0/1501073551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17cc06eac0 0x7f17cc142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1501073551 >> 172.21.15.133:0/1501073551 conn(0x7f17cc066f70 msgr2=0x7f17cc15c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:16.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1501073551 shutdown_connections 2024-04-25T10:19:16.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.069+0000 7f17d3d8d640 1 -- 172.21.15.133:0/1501073551 wait complete. 2024-04-25T10:19:16.087 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:19:16.088 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:19:16.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f326c10a640 1 Processor -- start 2024-04-25T10:19:16.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f326c10a640 1 -- start start 2024-04-25T10:19:16.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f326c10a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f32640cee30 0x7f32640ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f326c10a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32640b4c20 0x7f32640b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f326c10a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32640b55f0 con 0x7f326414b8c0 2024-04-25T10:19:16.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f326c10a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32640ce9d0 con 0x7f32640cee30 2024-04-25T10:19:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f326c10a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3264071260 con 0x7f32640b4c20 2024-04-25T10:19:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f32696a5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f32640cee30 0x7f32640ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f32696a5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f32640cee30 0x7f32640ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50714/0 (socket says 172.21.15.133:50714) 2024-04-25T10:19:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f32696a5640 1 -- 172.21.15.133:0/1752831647 learned_addr learned my addr 172.21.15.133:0/1752831647 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f326a6a7640 1 --2- 172.21.15.133:0/1752831647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32640b4c20 0x7f32640b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f32696a5640 1 -- 172.21.15.133:0/1752831647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32640b4c20 msgr2=0x7f32640b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f32696a5640 1 --2- 172.21.15.133:0/1752831647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32640b4c20 0x7f32640b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f32696a5640 1 -- 172.21.15.133:0/1752831647 >> v1:172.21.15.162:6789/0 conn(0x7f326414b8c0 legacy=0x7f32640b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f32696a5640 1 -- 172.21.15.133:0/1752831647 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3264071540 con 0x7f32640cee30 2024-04-25T10:19:16.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.201+0000 7f326a6a7640 1 --2- 172.21.15.133:0/1752831647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32640b4c20 0x7f32640b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:16.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f32696a5640 1 --2- 172.21.15.133:0/1752831647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32640cee30 0x7f32640ce060 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7f3254009870 tx=0x7f325401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae16b197ef95bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f3268ea4640 1 -- 172.21.15.133:0/1752831647 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540b0070 con 0x7f32640cee30 2024-04-25T10:19:16.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f3268ea4640 1 -- 172.21.15.133:0/1752831647 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32540ab600 con 0x7f32640cee30 2024-04-25T10:19:16.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f3268ea4640 1 -- 172.21.15.133:0/1752831647 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540ab9d0 con 0x7f32640cee30 2024-04-25T10:19:16.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 -- 172.21.15.133:0/1752831647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32640cee30 msgr2=0x7f32640ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 --2- 172.21.15.133:0/1752831647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32640cee30 0x7f32640ce060 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7f3254009870 tx=0x7f325401c490 comp rx=0 tx=0).stop 2024-04-25T10:19:16.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 -- 172.21.15.133:0/1752831647 shutdown_connections 2024-04-25T10:19:16.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 --2- 172.21.15.133:0/1752831647 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32640b4c20 0x7f32640b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 --2- 172.21.15.133:0/1752831647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32640cee30 0x7f32640ce060 unknown :-1 s=CLOSED pgs=3067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 -- 172.21.15.133:0/1752831647 >> 172.21.15.133:0/1752831647 conn(0x7f326406d370 msgr2=0x7f326406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:16.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 -- 172.21.15.133:0/1752831647 shutdown_connections 2024-04-25T10:19:16.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 -- 172.21.15.133:0/1752831647 wait complete. 2024-04-25T10:19:16.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 Processor -- start 2024-04-25T10:19:16.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 -- start start 2024-04-25T10:19:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32640b4c20 0x7f3264145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32640cee30 0x7f326414e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f326414b8c0 0x7f326414e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32640729b0 con 0x7f326414b8c0 2024-04-25T10:19:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3264072830 con 0x7f32640cee30 2024-04-25T10:19:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326c10a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3264072b30 con 0x7f32640b4c20 2024-04-25T10:19:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326a6a7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f326414b8c0 0x7f326414e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326a6a7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f326414b8c0 0x7f326414e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50716/0 (socket says 172.21.15.133:50716) 2024-04-25T10:19:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326a6a7640 1 -- 172.21.15.133:0/2305167366 learned_addr learned my addr 172.21.15.133:0/2305167366 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f32696a5640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32640cee30 0x7f326414e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f3269ea6640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32640b4c20 0x7f3264145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326a6a7640 1 -- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32640b4c20 msgr2=0x7f3264145700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326a6a7640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32640b4c20 0x7f3264145700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326a6a7640 1 -- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32640cee30 msgr2=0x7f326414e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326a6a7640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f32640cee30 0x7f326414e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f326a6a7640 1 -- 172.21.15.133:0/2305167366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326413c1c0 con 0x7f326414b8c0 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.205+0000 7f3269ea6640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f32640b4c20 0x7f3264145700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f326a6a7640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f326414b8c0 0x7f326414e6b0 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 crypto rx=0x7f326000cd50 tx=0x7f326000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f325a7fc640 1 -- 172.21.15.133:0/2305167366 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326000f680 con 0x7f326414b8c0 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f325a7fc640 1 -- 172.21.15.133:0/2305167366 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326000f840 con 0x7f326414b8c0 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f326c10a640 1 -- 172.21.15.133:0/2305167366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3264145c40 con 0x7f326414b8c0 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f325a7fc640 1 -- 172.21.15.133:0/2305167366 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326000fc30 con 0x7f326414b8c0 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f326c10a640 1 -- 172.21.15.133:0/2305167366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326413b6e0 con 0x7f326414b8c0 2024-04-25T10:19:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f3247fff640 1 -- 172.21.15.133:0/2305167366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3240001040 con 0x7f326414b8c0 2024-04-25T10:19:16.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f325a7fc640 1 -- 172.21.15.133:0/2305167366 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32600213f0 con 0x7f326414b8c0 2024-04-25T10:19:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f325a7fc640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3248066f60 0x7f3248069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f325a7fc640 1 -- 172.21.15.133:0/2305167366 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f32600860d0 con 0x7f326414b8c0 2024-04-25T10:19:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f3269ea6640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3248066f60 0x7f3248069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f325a7fc640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x7f324806d4a0 0x7f324806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f325a7fc640 1 -- 172.21.15.133:0/2305167366 --> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32600020c0 con 0x7f324806d4a0 2024-04-25T10:19:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f325a7fc640 1 -- 172.21.15.133:0/2305167366 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3260086560 con 0x7f326414b8c0 2024-04-25T10:19:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f3269ea6640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3248066f60 0x7f3248069420 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f326414f250 tx=0x7f325c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f32696a5640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x7f324806d4a0 0x7f324806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.209+0000 7f32696a5640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x7f324806d4a0 0x7f324806f8e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.213+0000 7f325a7fc640 1 -- 172.21.15.133:0/2305167366 <== osd.4 v2:172.21.15.162:6801/3188774481 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f32600020c0 con 0x7f324806d4a0 2024-04-25T10:19:16.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f3247fff640 1 -- 172.21.15.133:0/2305167366 --> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3240002e40 con 0x7f324806d4a0 2024-04-25T10:19:16.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f325a7fc640 1 -- 172.21.15.133:0/2305167366 <== osd.4 v2:172.21.15.162:6801/3188774481 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3240002e40 con 0x7f324806d4a0 2024-04-25T10:19:16.227 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:19:16.227 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:19:16.227 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:19:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f3247fff640 1 -- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x7f324806d4a0 msgr2=0x7f324806f8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f3247fff640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.162:6801/3188774481,v1:172.21.15.162:6804/3188774481] conn(0x7f324806d4a0 0x7f324806f8e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f3247fff640 1 -- 172.21.15.133:0/2305167366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3248066f60 msgr2=0x7f3248069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f3247fff640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f3248066f60 0x7f3248069420 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f326414f250 tx=0x7f325c009040 comp rx=0 tx=0).stop 2024-04-25T10:19:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f3247fff640 1 -- 172.21.15.133:0/2305167366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f326414b8c0 msgr2=0x7f326414e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f3247fff640 1 --2- 172.21.15.133:0/2305167366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f326414b8c0 0x7f326414e6b0 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 crypto rx=0x7f326000cd50 tx=0x7f326000a650 comp rx=0 tx=0).stop 2024-04-25T10:19:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f326a6a7640 1 -- 172.21.15.133:0/2305167366 reap_dead start 2024-04-25T10:19:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f3247fff640 1 -- 172.21.15.133:0/2305167366 shutdown_connections 2024-04-25T10:19:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f3247fff640 1 -- 172.21.15.133:0/2305167366 >> 172.21.15.133:0/2305167366 conn(0x7f326406d370 msgr2=0x7f32640ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f3247fff640 1 -- 172.21.15.133:0/2305167366 shutdown_connections 2024-04-25T10:19:16.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.225+0000 7f3247fff640 1 -- 172.21.15.133:0/2305167366 wait complete. 2024-04-25T10:19:16.241 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:19:16.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c60d19640 1 Processor -- start 2024-04-25T10:19:16.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c60d19640 1 -- start start 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c60d19640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1c5c143ea0 0x7f1c5c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c60d19640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c5c06eac0 0x7f1c5c0b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c60d19640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c15ac50 con 0x7f1c5c0b2150 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c60d19640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c0cd180 con 0x7f1c5c143ea0 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c60d19640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c15aed0 con 0x7f1c5c06eac0 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c5a59c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1c5c143ea0 0x7f1c5c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c59d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1c5c0b2150 0x7f1c5c06e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44640/0 (socket says 172.21.15.133:44640) 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c59d9b640 1 -- 172.21.15.133:0/1804576979 learned_addr learned my addr 172.21.15.133:0/1804576979 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c5a59c640 1 -- 172.21.15.133:0/1804576979 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c5c06eac0 msgr2=0x7f1c5c0b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c5a59c640 1 --2- 172.21.15.133:0/1804576979 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c5c06eac0 0x7f1c5c0b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c5a59c640 1 -- 172.21.15.133:0/1804576979 >> v1:172.21.15.133:6789/0 conn(0x7f1c5c0b2150 legacy=0x7f1c5c06e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c5959a640 1 -- 172.21.15.133:0/1804576979 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124098502 0 0) 0x7f1c50001120 con 0x7f1c5c0b2150 2024-04-25T10:19:16.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.353+0000 7f1c5a59c640 1 -- 172.21.15.133:0/1804576979 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c15b1b0 con 0x7f1c5c143ea0 2024-04-25T10:19:16.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c5a59c640 1 --2- 172.21.15.133:0/1804576979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c5c143ea0 0x7f1c5c0cc810 secure :-1 s=READY pgs=3070 cs=0 l=1 rev1=1 crypto rx=0x7f1c540097c0 tx=0x7f1c5401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0b88b0b84b8757 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c5959a640 1 -- 172.21.15.133:0/1804576979 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540b1070 con 0x7f1c5c143ea0 2024-04-25T10:19:16.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c5959a640 1 -- 172.21.15.133:0/1804576979 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c540ac410 con 0x7f1c5c143ea0 2024-04-25T10:19:16.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c5959a640 1 -- 172.21.15.133:0/1804576979 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540ac7e0 con 0x7f1c5c143ea0 2024-04-25T10:19:16.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 -- 172.21.15.133:0/1804576979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c5c143ea0 msgr2=0x7f1c5c0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 --2- 172.21.15.133:0/1804576979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c5c143ea0 0x7f1c5c0cc810 secure :-1 s=READY pgs=3070 cs=0 l=1 rev1=1 crypto rx=0x7f1c540097c0 tx=0x7f1c5401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 -- 172.21.15.133:0/1804576979 shutdown_connections 2024-04-25T10:19:16.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 --2- 172.21.15.133:0/1804576979 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c5c06eac0 0x7f1c5c0b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 --2- 172.21.15.133:0/1804576979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c5c143ea0 0x7f1c5c0cc810 unknown :-1 s=CLOSED pgs=3070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 -- 172.21.15.133:0/1804576979 >> 172.21.15.133:0/1804576979 conn(0x7f1c5c066f70 msgr2=0x7f1c5c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:16.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 -- 172.21.15.133:0/1804576979 shutdown_connections 2024-04-25T10:19:16.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 -- 172.21.15.133:0/1804576979 wait complete. 2024-04-25T10:19:16.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 Processor -- start 2024-04-25T10:19:16.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 -- start start 2024-04-25T10:19:16.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c5c06eac0 0x7f1c5c154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c5c0b2150 0x7f1c5c163780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c5c143ea0 0x7f1c5c163cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c06d420 con 0x7f1c5c0b2150 2024-04-25T10:19:16.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c06d2a0 con 0x7f1c5c143ea0 2024-04-25T10:19:16.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c06d5a0 con 0x7f1c5c06eac0 2024-04-25T10:19:16.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c59d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c5c0b2150 0x7f1c5c163780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c59d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c5c0b2150 0x7f1c5c163780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50732/0 (socket says 172.21.15.133:50732) 2024-04-25T10:19:16.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c59d9b640 1 -- 172.21.15.133:0/2810959204 learned_addr learned my addr 172.21.15.133:0/2810959204 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:16.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c5a59c640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c5c06eac0 0x7f1c5c154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c59d9b640 1 -- 172.21.15.133:0/2810959204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c5c06eac0 msgr2=0x7f1c5c154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c59d9b640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c5c06eac0 0x7f1c5c154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c59d9b640 1 -- 172.21.15.133:0/2810959204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c5c143ea0 msgr2=0x7f1c5c163cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c59d9b640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c5c143ea0 0x7f1c5c163cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c59d9b640 1 -- 172.21.15.133:0/2810959204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c152060 con 0x7f1c5c0b2150 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c59d9b640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c5c0b2150 0x7f1c5c163780 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c780 tx=0x7f1c5000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c4affd640 1 -- 172.21.15.133:0/2810959204 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50018070 con 0x7f1c5c0b2150 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c4affd640 1 -- 172.21.15.133:0/2810959204 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c500070d0 con 0x7f1c5c0b2150 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 -- 172.21.15.133:0/2810959204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c152340 con 0x7f1c5c0b2150 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c4affd640 1 -- 172.21.15.133:0/2810959204 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c500074c0 con 0x7f1c5c0b2150 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.357+0000 7f1c60d19640 1 -- 172.21.15.133:0/2810959204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c153fe0 con 0x7f1c5c0b2150 2024-04-25T10:19:16.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.361+0000 7f1c60d19640 1 -- 172.21.15.133:0/2810959204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c1541e0 con 0x7f1c5c0b2150 2024-04-25T10:19:16.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.361+0000 7f1c4affd640 1 -- 172.21.15.133:0/2810959204 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c5001b070 con 0x7f1c5c0b2150 2024-04-25T10:19:16.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.361+0000 7f1c4affd640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c3c066f60 0x7f1c3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.361+0000 7f1c4affd640 1 -- 172.21.15.133:0/2810959204 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f1c50084220 con 0x7f1c5c0b2150 2024-04-25T10:19:16.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.361+0000 7f1c5a59c640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c3c066f60 0x7f1c3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.361+0000 7f1c5a59c640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c3c066f60 0x7f1c3c069420 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1c540ae000 tx=0x7f1c54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.365+0000 7f1c4affd640 1 -- 172.21.15.133:0/2810959204 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c5c1541e0 con 0x7f1c5c0b2150 2024-04-25T10:19:16.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.521+0000 7f1c60d19640 1 -- 172.21.15.133:0/2810959204 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c5c068e80 con 0x7f1c3c066f60 2024-04-25T10:19:16.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c4affd640 1 -- 172.21.15.133:0/2810959204 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1c5c068e80 con 0x7f1c3c066f60 2024-04-25T10:19:16.525 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:19:16.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 -- 172.21.15.133:0/2810959204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c3c066f60 msgr2=0x7f1c3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c3c066f60 0x7f1c3c069420 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1c540ae000 tx=0x7f1c54008040 comp rx=0 tx=0).stop 2024-04-25T10:19:16.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 -- 172.21.15.133:0/2810959204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c5c0b2150 msgr2=0x7f1c5c163780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c5c0b2150 0x7f1c5c163780 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c780 tx=0x7f1c5000cc40 comp rx=0 tx=0).stop 2024-04-25T10:19:16.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 -- 172.21.15.133:0/2810959204 shutdown_connections 2024-04-25T10:19:16.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1c5c143ea0 0x7f1c5c163cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1c3c066f60 0x7f1c3c069420 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c5c0b2150 0x7f1c5c163780 unknown :-1 s=CLOSED pgs=3071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 --2- 172.21.15.133:0/2810959204 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f1c5c06eac0 0x7f1c5c154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 -- 172.21.15.133:0/2810959204 >> 172.21.15.133:0/2810959204 conn(0x7f1c5c066f70 msgr2=0x7f1c5c15d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:16.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 -- 172.21.15.133:0/2810959204 shutdown_connections 2024-04-25T10:19:16.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.525+0000 7f1c60d19640 1 -- 172.21.15.133:0/2810959204 wait complete. 2024-04-25T10:19:16.543 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:19:16.543 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:19:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.661+0000 7f2189f29640 1 Processor -- start 2024-04-25T10:19:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.661+0000 7f2189f29640 1 -- start start 2024-04-25T10:19:16.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.661+0000 7f2189f29640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f218415b640 0x7f218415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.661+0000 7f2189f29640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f21840d03e0 0x7f21840b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.661+0000 7f2189f29640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21840cc300 con 0x7f21840cf2c0 2024-04-25T10:19:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.661+0000 7f2189f29640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218415f460 con 0x7f218415b640 2024-04-25T10:19:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.661+0000 7f2189f29640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21841602c0 con 0x7f21840d03e0 2024-04-25T10:19:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.661+0000 7f21837fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f218415b640 0x7f218415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.661+0000 7f21837fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f218415b640 0x7f218415ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50736/0 (socket says 172.21.15.133:50736) 2024-04-25T10:19:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.661+0000 7f21837fe640 1 -- 172.21.15.133:0/294799568 learned_addr learned my addr 172.21.15.133:0/294799568 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.665+0000 7f2183fff640 1 --2- 172.21.15.133:0/294799568 >> v2:172.21.15.162:3300/0 conn(0x7f21840d03e0 0x7f21840b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.665+0000 7f21837fe640 1 -- 172.21.15.133:0/294799568 >> v2:172.21.15.162:3300/0 conn(0x7f21840d03e0 msgr2=0x7f21840b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.665+0000 7f21837fe640 1 --2- 172.21.15.133:0/294799568 >> v2:172.21.15.162:3300/0 conn(0x7f21840d03e0 0x7f21840b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.665+0000 7f21837fe640 1 -- 172.21.15.133:0/294799568 >> v1:172.21.15.162:6789/0 conn(0x7f21840cf2c0 legacy=0x7f21840cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:16.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.665+0000 7f21837fe640 1 -- 172.21.15.133:0/294799568 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218415f6b0 con 0x7f218415b640 2024-04-25T10:19:16.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.665+0000 7f21837fe640 1 --2- 172.21.15.133:0/294799568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f218415b640 0x7f218415ba20 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 crypto rx=0x7f21840d08c0 tx=0x7f217401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e672c60dd889e6eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.665+0000 7f2182ffd640 1 -- 172.21.15.133:0/294799568 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740b0070 con 0x7f218415b640 2024-04-25T10:19:16.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.665+0000 7f2182ffd640 1 -- 172.21.15.133:0/294799568 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21740ab600 con 0x7f218415b640 2024-04-25T10:19:16.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.665+0000 7f2182ffd640 1 -- 172.21.15.133:0/294799568 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740ab9d0 con 0x7f218415b640 2024-04-25T10:19:16.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.665+0000 7f2189f29640 1 -- 172.21.15.133:0/294799568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f218415b640 msgr2=0x7f218415ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.665+0000 7f2189f29640 1 --2- 172.21.15.133:0/294799568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f218415b640 0x7f218415ba20 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 crypto rx=0x7f21840d08c0 tx=0x7f217401c300 comp rx=0 tx=0).stop 2024-04-25T10:19:16.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 -- 172.21.15.133:0/294799568 shutdown_connections 2024-04-25T10:19:16.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 --2- 172.21.15.133:0/294799568 >> v2:172.21.15.162:3300/0 conn(0x7f21840d03e0 0x7f21840b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 --2- 172.21.15.133:0/294799568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f218415b640 0x7f218415ba20 unknown :-1 s=CLOSED pgs=3072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 -- 172.21.15.133:0/294799568 >> 172.21.15.133:0/294799568 conn(0x7f218409aa00 msgr2=0x7f218408af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:16.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 -- 172.21.15.133:0/294799568 shutdown_connections 2024-04-25T10:19:16.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 -- 172.21.15.133:0/294799568 wait complete. 2024-04-25T10:19:16.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 Processor -- start 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 -- start start 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21840cf2c0 0x7f2184151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21840d03e0 0x7f21840c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f218415b640 0x7f21840c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21840b26b0 con 0x7f218415b640 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21840b2530 con 0x7f21840d03e0 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21840b2830 con 0x7f21840cf2c0 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2183fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f218415b640 0x7f21840c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2183fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f218415b640 0x7f21840c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50746/0 (socket says 172.21.15.133:50746) 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2183fff640 1 -- 172.21.15.133:0/2111238628 learned_addr learned my addr 172.21.15.133:0/2111238628 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f21837fe640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21840cf2c0 0x7f2184151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f217bfff640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21840d03e0 0x7f21840c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2183fff640 1 -- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21840cf2c0 msgr2=0x7f2184151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2183fff640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21840cf2c0 0x7f2184151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2183fff640 1 -- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21840d03e0 msgr2=0x7f21840c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2183fff640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21840d03e0 0x7f21840c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2183fff640 1 -- 172.21.15.133:0/2111238628 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21841524e0 con 0x7f218415b640 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f21837fe640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f21840cf2c0 0x7f2184151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f217bfff640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f21840d03e0 0x7f21840c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2183fff640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f218415b640 0x7f21840c6c10 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 crypto rx=0x7f217c00cd20 tx=0x7f217c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2180ff9640 1 -- 172.21.15.133:0/2111238628 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c003ba0 con 0x7f218415b640 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2180ff9640 1 -- 172.21.15.133:0/2111238628 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c003d40 con 0x7f218415b640 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 -- 172.21.15.133:0/2111238628 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21840c9540 con 0x7f218415b640 2024-04-25T10:19:16.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2180ff9640 1 -- 172.21.15.133:0/2111238628 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c004240 con 0x7f218415b640 2024-04-25T10:19:16.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 -- 172.21.15.133:0/2111238628 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21840c9b00 con 0x7f218415b640 2024-04-25T10:19:16.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.669+0000 7f2189f29640 1 -- 172.21.15.133:0/2111238628 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2184069000 con 0x7f218415b640 2024-04-25T10:19:16.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.673+0000 7f2180ff9640 1 -- 172.21.15.133:0/2111238628 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f217c0043e0 con 0x7f218415b640 2024-04-25T10:19:16.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.673+0000 7f2180ff9640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2164066f90 0x7f2164069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.673+0000 7f2180ff9640 1 -- 172.21.15.133:0/2111238628 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f217c085eb0 con 0x7f218415b640 2024-04-25T10:19:16.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.673+0000 7f21837fe640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2164066f90 0x7f2164069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.673+0000 7f2180ff9640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] conn(0x7f216406d4d0 0x7f216406f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.673+0000 7f2180ff9640 1 -- 172.21.15.133:0/2111238628 --> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f217c022ec0 con 0x7f216406d4d0 2024-04-25T10:19:16.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.673+0000 7f2180ff9640 1 -- 172.21.15.133:0/2111238628 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f217c086340 con 0x7f218415b640 2024-04-25T10:19:16.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.673+0000 7f21837fe640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2164066f90 0x7f2164069450 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f21740ad000 tx=0x7f2174008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.673+0000 7f217bfff640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] conn(0x7f216406d4d0 0x7f216406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.673+0000 7f217bfff640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] conn(0x7f216406d4d0 0x7f216406f910 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.673+0000 7f2180ff9640 1 -- 172.21.15.133:0/2111238628 <== osd.5 v2:172.21.15.162:6803/4006238640 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f217c022ec0 con 0x7f216406d4d0 2024-04-25T10:19:16.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2189f29640 1 -- 172.21.15.133:0/2111238628 --> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f21840b11b0 con 0x7f216406d4d0 2024-04-25T10:19:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2180ff9640 1 -- 172.21.15.133:0/2111238628 <== osd.5 v2:172.21.15.162:6803/4006238640 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f21840b11b0 con 0x7f216406d4d0 2024-04-25T10:19:16.690 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:19:16.690 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:19:16.690 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:19:16.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2189f29640 1 -- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] conn(0x7f216406d4d0 msgr2=0x7f216406f910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2189f29640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.162:6803/4006238640,v1:172.21.15.162:6806/4006238640] conn(0x7f216406d4d0 0x7f216406f910 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2189f29640 1 -- 172.21.15.133:0/2111238628 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2164066f90 msgr2=0x7f2164069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2189f29640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2164066f90 0x7f2164069450 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f21740ad000 tx=0x7f2174008040 comp rx=0 tx=0).stop 2024-04-25T10:19:16.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2189f29640 1 -- 172.21.15.133:0/2111238628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f218415b640 msgr2=0x7f21840c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2189f29640 1 --2- 172.21.15.133:0/2111238628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f218415b640 0x7f21840c6c10 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 crypto rx=0x7f217c00cd20 tx=0x7f217c00a660 comp rx=0 tx=0).stop 2024-04-25T10:19:16.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2183fff640 1 -- 172.21.15.133:0/2111238628 reap_dead start 2024-04-25T10:19:16.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2189f29640 1 -- 172.21.15.133:0/2111238628 shutdown_connections 2024-04-25T10:19:16.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2189f29640 1 -- 172.21.15.133:0/2111238628 >> 172.21.15.133:0/2111238628 conn(0x7f218409aa00 msgr2=0x7f21840ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:16.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2189f29640 1 -- 172.21.15.133:0/2111238628 shutdown_connections 2024-04-25T10:19:16.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.689+0000 7f2189f29640 1 -- 172.21.15.133:0/2111238628 wait complete. 2024-04-25T10:19:16.705 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:19:16.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f244830a640 1 Processor -- start 2024-04-25T10:19:16.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f244830a640 1 -- start start 2024-04-25T10:19:16.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f244830a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f24400b1420 0x7f24400b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f244830a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24400d0380 con 0x7f244015dc60 2024-04-25T10:19:16.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f244830a640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24400cacf0 con 0x7f24400b0a00 2024-04-25T10:19:16.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f244830a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24400cfe60 con 0x7f24400b1420 2024-04-25T10:19:16.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f24468a7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f24400b1420 0x7f24400b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f24460a6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f244015dc60 0x7f24400cc690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44654/0 (socket says 172.21.15.133:44654) 2024-04-25T10:19:16.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f24460a6640 1 -- 172.21.15.133:0/3913535600 learned_addr learned my addr 172.21.15.133:0/3913535600 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:16.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f24468a7640 1 -- 172.21.15.133:0/3913535600 >> v1:172.21.15.162:6789/0 conn(0x7f24400b0a00 legacy=0x7f24400cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:16.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f24468a7640 1 -- 172.21.15.133:0/3913535600 >> v1:172.21.15.133:6789/0 conn(0x7f244015dc60 legacy=0x7f24400cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f24468a7640 1 -- 172.21.15.133:0/3913535600 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24400d00a0 con 0x7f24400b1420 2024-04-25T10:19:16.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f24450a4640 1 -- 172.21.15.133:0/3913535600 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300397308 0 0) 0x7f24400d0380 con 0x7f244015dc60 2024-04-25T10:19:16.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f24468a7640 1 --2- 172.21.15.133:0/3913535600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24400b1420 0x7f24400b18d0 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 crypto rx=0x7f24340097c0 tx=0x7f243401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a23b5db3463fb8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f24450a4640 1 -- 172.21.15.133:0/3913535600 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340b0070 con 0x7f24400b1420 2024-04-25T10:19:16.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f24450a4640 1 -- 172.21.15.133:0/3913535600 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24340ab600 con 0x7f24400b1420 2024-04-25T10:19:16.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.809+0000 7f24450a4640 1 -- 172.21.15.133:0/3913535600 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340ab9d0 con 0x7f24400b1420 2024-04-25T10:19:16.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 -- 172.21.15.133:0/3913535600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24400b1420 msgr2=0x7f24400b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 --2- 172.21.15.133:0/3913535600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24400b1420 0x7f24400b18d0 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 crypto rx=0x7f24340097c0 tx=0x7f243401c340 comp rx=0 tx=0).stop 2024-04-25T10:19:16.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 -- 172.21.15.133:0/3913535600 shutdown_connections 2024-04-25T10:19:16.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 --2- 172.21.15.133:0/3913535600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24400b1420 0x7f24400b18d0 unknown :-1 s=CLOSED pgs=3075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 -- 172.21.15.133:0/3913535600 >> 172.21.15.133:0/3913535600 conn(0x7f24400999d0 msgr2=0x7f24400d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:16.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 -- 172.21.15.133:0/3913535600 shutdown_connections 2024-04-25T10:19:16.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 -- 172.21.15.133:0/3913535600 wait complete. 2024-04-25T10:19:16.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 Processor -- start 2024-04-25T10:19:16.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 -- start start 2024-04-25T10:19:16.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24400b0a00 0x7f244015c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24400b1420 0x7f24400b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f244015dc60 0x7f24400c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24400cb300 con 0x7f244015dc60 2024-04-25T10:19:16.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24400cc1b0 con 0x7f24400b0a00 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244016b670 con 0x7f24400b1420 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24468a7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f244015dc60 0x7f24400c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24468a7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f244015dc60 0x7f24400c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50772/0 (socket says 172.21.15.133:50772) 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24468a7640 1 -- 172.21.15.133:0/1343579263 learned_addr learned my addr 172.21.15.133:0/1343579263 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24460a6640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24400b0a00 0x7f244015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24458a5640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24400b1420 0x7f24400b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24468a7640 1 -- 172.21.15.133:0/1343579263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24400b1420 msgr2=0x7f24400b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24468a7640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24400b1420 0x7f24400b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24468a7640 1 -- 172.21.15.133:0/1343579263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24400b0a00 msgr2=0x7f244015c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24468a7640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24400b0a00 0x7f244015c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24468a7640 1 -- 172.21.15.133:0/1343579263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244015c8d0 con 0x7f244015dc60 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24460a6640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24400b0a00 0x7f244015c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:16.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f24468a7640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f244015dc60 0x7f24400c77b0 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 crypto rx=0x7f24340ae040 tx=0x7f243401ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f2432ffd640 1 -- 172.21.15.133:0/1343579263 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340bc070 con 0x7f244015dc60 2024-04-25T10:19:16.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f2432ffd640 1 -- 172.21.15.133:0/1343579263 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24340b1040 con 0x7f244015dc60 2024-04-25T10:19:16.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f2432ffd640 1 -- 172.21.15.133:0/1343579263 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340b0050 con 0x7f244015dc60 2024-04-25T10:19:16.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 -- 172.21.15.133:0/1343579263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2440153c20 con 0x7f244015dc60 2024-04-25T10:19:16.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.813+0000 7f244830a640 1 -- 172.21.15.133:0/1343579263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24401541e0 con 0x7f244015dc60 2024-04-25T10:19:16.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.817+0000 7f244830a640 1 -- 172.21.15.133:0/1343579263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2440067c70 con 0x7f244015dc60 2024-04-25T10:19:16.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.817+0000 7f2432ffd640 1 -- 172.21.15.133:0/1343579263 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2434003620 con 0x7f244015dc60 2024-04-25T10:19:16.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.817+0000 7f2432ffd640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2424066f60 0x7f2424069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:16.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.817+0000 7f24460a6640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2424066f60 0x7f2424069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:16.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.817+0000 7f2432ffd640 1 -- 172.21.15.133:0/1343579263 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f243411eb10 con 0x7f244015dc60 2024-04-25T10:19:16.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.817+0000 7f24460a6640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2424066f60 0x7f2424069420 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2438007fd0 tx=0x7f2438009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:16.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.821+0000 7f2432ffd640 1 -- 172.21.15.133:0/1343579263 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2434016050 con 0x7f244015dc60 2024-04-25T10:19:16.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.965+0000 7f244830a640 1 -- 172.21.15.133:0/1343579263 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f2440099f20 con 0x7f2424066f60 2024-04-25T10:19:16.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.965+0000 7f2432ffd640 1 -- 172.21.15.133:0/1343579263 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2440099f20 con 0x7f2424066f60 2024-04-25T10:19:16.968 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:19:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 -- 172.21.15.133:0/1343579263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2424066f60 msgr2=0x7f2424069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2424066f60 0x7f2424069420 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2438007fd0 tx=0x7f2438009040 comp rx=0 tx=0).stop 2024-04-25T10:19:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 -- 172.21.15.133:0/1343579263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f244015dc60 msgr2=0x7f24400c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f244015dc60 0x7f24400c77b0 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 crypto rx=0x7f24340ae040 tx=0x7f243401ab80 comp rx=0 tx=0).stop 2024-04-25T10:19:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 -- 172.21.15.133:0/1343579263 shutdown_connections 2024-04-25T10:19:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f2424066f60 0x7f2424069420 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f244015dc60 0x7f24400c77b0 unknown :-1 s=CLOSED pgs=3076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f24400b1420 0x7f24400b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 --2- 172.21.15.133:0/1343579263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24400b0a00 0x7f244015c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 -- 172.21.15.133:0/1343579263 >> 172.21.15.133:0/1343579263 conn(0x7f24400999d0 msgr2=0x7f24400b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:16.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 -- 172.21.15.133:0/1343579263 shutdown_connections 2024-04-25T10:19:16.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:16.969+0000 7f244830a640 1 -- 172.21.15.133:0/1343579263 wait complete. 2024-04-25T10:19:16.986 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:19:16.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:19:17.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f6c0c640 1 Processor -- start 2024-04-25T10:19:17.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f6c0c640 1 -- start start 2024-04-25T10:19:17.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f6c0c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f01407f0 0x7fb7f00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f6c0c640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fb7f0140bd0 0x7fb7f00b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f6c0c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f015edd0 con 0x7fb7f00b2b70 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f6c0c640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f00b47d0 con 0x7fb7f0140bd0 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f6c0c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00b4ab0 con 0x7fb7f01407f0 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f51a9640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb7f00b2b70 0x7fb7f015e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44662/0 (socket says 172.21.15.133:44662) 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f51a9640 1 -- 172.21.15.133:0/445631953 learned_addr learned my addr 172.21.15.133:0/445631953 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7effff640 1 --2- 172.21.15.133:0/445631953 >> v2:172.21.15.162:3300/0 conn(0x7fb7f0140bd0 0x7fb7f00b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f49a8640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f01407f0 0x7fb7f00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517002246 0 0) 0x7fb7f015edd0 con 0x7fb7f00b2b70 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7dc003530 con 0x7fb7f00b2b70 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 481521589 0 0) 0x7fb7dc003530 con 0x7fb7f00b2b70 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f01407f0 msgr2=0x7fb7f00b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f01407f0 0x7fb7f00b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 >> v2:172.21.15.162:3300/0 conn(0x7fb7f0140bd0 msgr2=0x7fb7f00b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:17.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 --2- 172.21.15.133:0/445631953 >> v2:172.21.15.162:3300/0 conn(0x7fb7f0140bd0 0x7fb7f00b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f00b4d90 con 0x7fb7f00b2b70 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7fb7e0001680 con 0x7fb7f00b2b70 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 >> v1:172.21.15.133:6789/0 conn(0x7fb7f00b2b70 legacy=0x7fb7f015e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0039c0 0x7fb7dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f01407f0 0x7fb7dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc004d50 0x7fb7dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00b47d0 con 0x7fb7dc0039c0 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00b4ab0 con 0x7fb7dc004d50 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc00b9d0 con 0x7fb7f01407f0 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f49a8640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0039c0 0x7fb7dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f49a8640 1 -- 172.21.15.133:0/445631953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f01407f0 msgr2=0x7fb7dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f49a8640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f01407f0 0x7fb7dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f49a8640 1 -- 172.21.15.133:0/445631953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc004d50 msgr2=0x7fb7dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:17.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f51a9640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc004d50 0x7fb7dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f49a8640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc004d50 0x7fb7dc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f49a8640 1 -- 172.21.15.133:0/445631953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc003530 con 0x7fb7dc0039c0 2024-04-25T10:19:17.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.097+0000 7fb7f51a9640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc004d50 0x7fb7dc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:17.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7f49a8640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0039c0 0x7fb7dc003de0 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009fa0 tx=0x7fb7e801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d942625bc8004d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e80b2070 con 0x7fb7dc0039c0 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e80a5b30 con 0x7fb7dc0039c0 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7ef7fe640 1 -- 172.21.15.133:0/445631953 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8009bf0 con 0x7fb7dc0039c0 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/445631953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0039c0 msgr2=0x7fb7dc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7f6c0c640 1 --2- 172.21.15.133:0/445631953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7dc0039c0 0x7fb7dc003de0 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009fa0 tx=0x7fb7e801cc80 comp rx=0 tx=0).stop 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7f51a9640 1 -- 172.21.15.133:0/445631953 reap_dead start 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/445631953 shutdown_connections 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/445631953 >> 172.21.15.133:0/445631953 conn(0x7fb7f006d370 msgr2=0x7fb7f006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/445631953 shutdown_connections 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/445631953 wait complete. 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7f6c0c640 1 Processor -- start 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7f6c0c640 1 -- start start 2024-04-25T10:19:17.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.101+0000 7fb7f6c0c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc0039c0 0x7fb7f0138e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f6c0c640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f0158780 0x7fb7f013ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f6c0c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f013c770 0x7fb7f00ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f6c0c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0071cd0 con 0x7fb7f013c770 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f6c0c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0071b50 con 0x7fb7dc0039c0 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f6c0c640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0071e50 con 0x7fb7f0158780 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f51a9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f013c770 0x7fb7f00ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f51a9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f013c770 0x7fb7f00ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50790/0 (socket says 172.21.15.133:50790) 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f51a9640 1 -- 172.21.15.133:0/3969607190 learned_addr learned my addr 172.21.15.133:0/3969607190 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f49a8640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc0039c0 0x7fb7f0138e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7effff640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f0158780 0x7fb7f013ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f51a9640 1 -- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f0158780 msgr2=0x7fb7f013ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f51a9640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f0158780 0x7fb7f013ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f51a9640 1 -- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc0039c0 msgr2=0x7fb7f0138e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f51a9640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7dc0039c0 0x7fb7f0138e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f51a9640 1 -- 172.21.15.133:0/3969607190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f01393a0 con 0x7fb7f013c770 2024-04-25T10:19:17.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7effff640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fb7f0158780 0x7fb7f013ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:17.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f51a9640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f013c770 0x7fb7f00ae0e0 secure :-1 s=READY pgs=3079 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00cf550 tx=0x7fb7e000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7ed7fa640 1 -- 172.21.15.133:0/3969607190 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0007230 con 0x7fb7f013c770 2024-04-25T10:19:17.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7ed7fa640 1 -- 172.21.15.133:0/3969607190 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e00073d0 con 0x7fb7f013c770 2024-04-25T10:19:17.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/3969607190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f013b6d0 con 0x7fb7f013c770 2024-04-25T10:19:17.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7ed7fa640 1 -- 172.21.15.133:0/3969607190 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e00047f0 con 0x7fb7f013c770 2024-04-25T10:19:17.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/3969607190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f013bc90 con 0x7fb7f013c770 2024-04-25T10:19:17.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/3969607190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7f0069000 con 0x7fb7f013c770 2024-04-25T10:19:17.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7ed7fa640 1 -- 172.21.15.133:0/3969607190 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7e001b020 con 0x7fb7f013c770 2024-04-25T10:19:17.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7ed7fa640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7d0066f60 0x7fb7d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f49a8640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7d0066f60 0x7fb7d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7ed7fa640 1 -- 172.21.15.133:0/3969607190 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb7e0085280 con 0x7fb7f013c770 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7ed7fa640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x7fb7d006d4a0 0x7fb7d006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7ed7fa640 1 -- 172.21.15.133:0/3969607190 --> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7e0009e50 con 0x7fb7d006d4a0 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7ed7fa640 1 -- 172.21.15.133:0/3969607190 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7e0085710 con 0x7fb7f013c770 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7f49a8640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7d0066f60 0x7fb7d0069420 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80af000 tx=0x7fb7e8004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7effff640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x7fb7d006d4a0 0x7fb7d006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.105+0000 7fb7effff640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x7fb7d006d4a0 0x7fb7d006f8e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.109+0000 7fb7ed7fa640 1 -- 172.21.15.133:0/3969607190 <== osd.6 v2:172.21.15.162:6824/2819373209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb7e0009e50 con 0x7fb7d006d4a0 2024-04-25T10:19:17.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/3969607190 --> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb7f00732c0 con 0x7fb7d006d4a0 2024-04-25T10:19:17.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7ed7fa640 1 -- 172.21.15.133:0/3969607190 <== osd.6 v2:172.21.15.162:6824/2819373209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb7f00732c0 con 0x7fb7d006d4a0 2024-04-25T10:19:17.124 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:19:17.124 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:19:17.124 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:19:17.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x7fb7d006d4a0 msgr2=0x7fb7d006f8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f6c0c640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.162:6824/2819373209,v1:172.21.15.162:6825/2819373209] conn(0x7fb7d006d4a0 0x7fb7d006f8e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/3969607190 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7d0066f60 msgr2=0x7fb7d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f6c0c640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fb7d0066f60 0x7fb7d0069420 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80af000 tx=0x7fb7e8004ac0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/3969607190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f013c770 msgr2=0x7fb7f00ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f6c0c640 1 --2- 172.21.15.133:0/3969607190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f013c770 0x7fb7f00ae0e0 secure :-1 s=READY pgs=3079 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00cf550 tx=0x7fb7e000dc00 comp rx=0 tx=0).stop 2024-04-25T10:19:17.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f51a9640 1 -- 172.21.15.133:0/3969607190 reap_dead start 2024-04-25T10:19:17.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/3969607190 shutdown_connections 2024-04-25T10:19:17.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/3969607190 >> 172.21.15.133:0/3969607190 conn(0x7fb7f006d370 msgr2=0x7fb7f013f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:17.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/3969607190 shutdown_connections 2024-04-25T10:19:17.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.121+0000 7fb7f6c0c640 1 -- 172.21.15.133:0/3969607190 wait complete. 2024-04-25T10:19:17.139 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:19:17.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e777b8640 1 Processor -- start 2024-04-25T10:19:17.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e777b8640 1 -- start start 2024-04-25T10:19:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e777b8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6e700b1be0 0x7f6e7015f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e777b8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7006b3a0 con 0x7f6e7015b8e0 2024-04-25T10:19:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e777b8640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7006b650 con 0x7f6e7015c820 2024-04-25T10:19:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e777b8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7015c390 con 0x7f6e700b1be0 2024-04-25T10:19:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e75d55640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6e700b1be0 0x7f6e7015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e75554640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6e7015b8e0 0x7f6e7006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44668/0 (socket says 172.21.15.133:44668) 2024-04-25T10:19:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e75554640 1 -- 172.21.15.133:0/1300314661 learned_addr learned my addr 172.21.15.133:0/1300314661 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e67fff640 1 -- 172.21.15.133:0/1300314661 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112161579 0 0) 0x7f6e7006b3a0 con 0x7f6e7015b8e0 2024-04-25T10:19:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e67fff640 1 -- 172.21.15.133:0/1300314661 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e5c003530 con 0x7f6e7015b8e0 2024-04-25T10:19:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e75d55640 1 -- 172.21.15.133:0/1300314661 >> v1:172.21.15.162:6789/0 conn(0x7f6e7015c820 legacy=0x7f6e700b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e75d55640 1 -- 172.21.15.133:0/1300314661 >> v1:172.21.15.133:6789/0 conn(0x7f6e7015b8e0 legacy=0x7f6e7006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.249+0000 7f6e75d55640 1 -- 172.21.15.133:0/1300314661 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7006b930 con 0x7f6e700b1be0 2024-04-25T10:19:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e75d55640 1 --2- 172.21.15.133:0/1300314661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700b1be0 0x7f6e7015f350 secure :-1 s=READY pgs=3081 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009d90 tx=0x7f6e6801d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21486e102f42a610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e67fff640 1 -- 172.21.15.133:0/1300314661 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680b2070 con 0x7f6e700b1be0 2024-04-25T10:19:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e67fff640 1 -- 172.21.15.133:0/1300314661 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e680a5900 con 0x7f6e700b1be0 2024-04-25T10:19:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e67fff640 1 -- 172.21.15.133:0/1300314661 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680a5cd0 con 0x7f6e700b1be0 2024-04-25T10:19:17.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e777b8640 1 -- 172.21.15.133:0/1300314661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700b1be0 msgr2=0x7f6e7015f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e777b8640 1 --2- 172.21.15.133:0/1300314661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700b1be0 0x7f6e7015f350 secure :-1 s=READY pgs=3081 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009d90 tx=0x7f6e6801d390 comp rx=0 tx=0).stop 2024-04-25T10:19:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e777b8640 1 -- 172.21.15.133:0/1300314661 shutdown_connections 2024-04-25T10:19:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e777b8640 1 --2- 172.21.15.133:0/1300314661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700b1be0 0x7f6e7015f350 unknown :-1 s=CLOSED pgs=3081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e777b8640 1 -- 172.21.15.133:0/1300314661 >> 172.21.15.133:0/1300314661 conn(0x7f6e700999d0 msgr2=0x7f6e700d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e777b8640 1 -- 172.21.15.133:0/1300314661 shutdown_connections 2024-04-25T10:19:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e777b8640 1 -- 172.21.15.133:0/1300314661 wait complete. 2024-04-25T10:19:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.253+0000 7f6e777b8640 1 Processor -- start 2024-04-25T10:19:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e777b8640 1 -- start start 2024-04-25T10:19:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e777b8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700b1be0 0x7f6e7013ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e777b8640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7015b8e0 0x7f6e701532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e777b8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e7015c820 0x7f6e7006f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e777b8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700d1270 con 0x7f6e700b1be0 2024-04-25T10:19:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e777b8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e700d10f0 con 0x7f6e7015c820 2024-04-25T10:19:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e777b8640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70169700 con 0x7f6e7015b8e0 2024-04-25T10:19:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e75554640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700b1be0 0x7f6e7013ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e75554640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700b1be0 0x7f6e7013ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50804/0 (socket says 172.21.15.133:50804) 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e75554640 1 -- 172.21.15.133:0/3916531688 learned_addr learned my addr 172.21.15.133:0/3916531688 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e74d53640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7015b8e0 0x7f6e701532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e75d55640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e7015c820 0x7f6e7006f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e75554640 1 -- 172.21.15.133:0/3916531688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7015b8e0 msgr2=0x7f6e701532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e75554640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7015b8e0 0x7f6e701532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e75554640 1 -- 172.21.15.133:0/3916531688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e7015c820 msgr2=0x7f6e7006f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e75554640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e7015c820 0x7f6e7006f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e75554640 1 -- 172.21.15.133:0/3916531688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e70153cd0 con 0x7f6e700b1be0 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e74d53640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7015b8e0 0x7f6e701532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e75554640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700b1be0 0x7f6e7013ff70 secure :-1 s=READY pgs=3082 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00c450 tx=0x7f6e6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e65ffb640 1 -- 172.21.15.133:0/3916531688 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c017070 con 0x7f6e700b1be0 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e65ffb640 1 -- 172.21.15.133:0/3916531688 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c005cf0 con 0x7f6e700b1be0 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e65ffb640 1 -- 172.21.15.133:0/3916531688 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c007290 con 0x7f6e700b1be0 2024-04-25T10:19:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e777b8640 1 -- 172.21.15.133:0/3916531688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7015abd0 con 0x7f6e700b1be0 2024-04-25T10:19:17.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e777b8640 1 -- 172.21.15.133:0/3916531688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7015a8f0 con 0x7f6e700b1be0 2024-04-25T10:19:17.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e777b8640 1 -- 172.21.15.133:0/3916531688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e70067c70 con 0x7f6e700b1be0 2024-04-25T10:19:17.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.257+0000 7f6e65ffb640 1 -- 172.21.15.133:0/3916531688 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e6c01a070 con 0x7f6e700b1be0 2024-04-25T10:19:17.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.261+0000 7f6e65ffb640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e54066f60 0x7f6e54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.261+0000 7f6e65ffb640 1 -- 172.21.15.133:0/3916531688 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f6e6c084320 con 0x7f6e700b1be0 2024-04-25T10:19:17.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.261+0000 7f6e74d53640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e54066f60 0x7f6e54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.261+0000 7f6e74d53640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e54066f60 0x7f6e54069420 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f6e700c3b90 tx=0x7f6e60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.265+0000 7f6e65ffb640 1 -- 172.21.15.133:0/3916531688 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e6c055450 con 0x7f6e700b1be0 2024-04-25T10:19:17.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.425+0000 7f6e777b8640 1 -- 172.21.15.133:0/3916531688 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e70099e80 con 0x7f6e54066f60 2024-04-25T10:19:17.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e65ffb640 1 -- 172.21.15.133:0/3916531688 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6e70099e80 con 0x7f6e54066f60 2024-04-25T10:19:17.429 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 -- 172.21.15.133:0/3916531688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e54066f60 msgr2=0x7f6e54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e54066f60 0x7f6e54069420 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f6e700c3b90 tx=0x7f6e60008040 comp rx=0 tx=0).stop 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 -- 172.21.15.133:0/3916531688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700b1be0 msgr2=0x7f6e7013ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700b1be0 0x7f6e7013ff70 secure :-1 s=READY pgs=3082 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00c450 tx=0x7f6e6c00c910 comp rx=0 tx=0).stop 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 -- 172.21.15.133:0/3916531688 shutdown_connections 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e7015c820 0x7f6e7006f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f6e7015b8e0 0x7f6e701532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f6e54066f60 0x7f6e54069420 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 --2- 172.21.15.133:0/3916531688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700b1be0 0x7f6e7013ff70 unknown :-1 s=CLOSED pgs=3082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 -- 172.21.15.133:0/3916531688 >> 172.21.15.133:0/3916531688 conn(0x7f6e700999d0 msgr2=0x7f6e700afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 -- 172.21.15.133:0/3916531688 shutdown_connections 2024-04-25T10:19:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.429+0000 7f6e777b8640 1 -- 172.21.15.133:0/3916531688 wait complete. 2024-04-25T10:19:17.447 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:19:17.447 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:19:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f157a99e640 1 Processor -- start 2024-04-25T10:19:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f157a99e640 1 -- start start 2024-04-25T10:19:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f157a99e640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f157415b910 0x7f15740ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f157a99e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f15740cebd0 0x7f157415cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f157a99e640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157415bf40 con 0x7f157415b530 2024-04-25T10:19:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f157a99e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15741589c0 con 0x7f15740cebd0 2024-04-25T10:19:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f157a99e640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157415c200 con 0x7f157415b910 2024-04-25T10:19:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f1578f3b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f15740cebd0 0x7f157415cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f1578f3b640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f15740cebd0 0x7f157415cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50810/0 (socket says 172.21.15.133:50810) 2024-04-25T10:19:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f1578f3b640 1 -- 172.21.15.133:0/1048444226 learned_addr learned my addr 172.21.15.133:0/1048444226 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f15737fe640 1 --2- 172.21.15.133:0/1048444226 >> v2:172.21.15.162:3300/0 conn(0x7f157415b910 0x7f15740ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f1578f3b640 1 -- 172.21.15.133:0/1048444226 >> v2:172.21.15.162:3300/0 conn(0x7f157415b910 msgr2=0x7f15740ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f1578f3b640 1 --2- 172.21.15.133:0/1048444226 >> v2:172.21.15.162:3300/0 conn(0x7f157415b910 0x7f15740ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f1578f3b640 1 -- 172.21.15.133:0/1048444226 >> v1:172.21.15.162:6789/0 conn(0x7f157415b530 legacy=0x7f15740cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:17.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.569+0000 7f1578f3b640 1 -- 172.21.15.133:0/1048444226 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157415c4e0 con 0x7f15740cebd0 2024-04-25T10:19:17.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f1578f3b640 1 --2- 172.21.15.133:0/1048444226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15740cebd0 0x7f157415cfc0 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f157406e370 tx=0x7f156c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3187647299e0c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f1572ffd640 1 -- 172.21.15.133:0/1048444226 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c0a55e0 con 0x7f15740cebd0 2024-04-25T10:19:17.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f1572ffd640 1 -- 172.21.15.133:0/1048444226 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c0a5780 con 0x7f15740cebd0 2024-04-25T10:19:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f1572ffd640 1 -- 172.21.15.133:0/1048444226 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c0a5b50 con 0x7f15740cebd0 2024-04-25T10:19:17.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 -- 172.21.15.133:0/1048444226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15740cebd0 msgr2=0x7f157415cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 --2- 172.21.15.133:0/1048444226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15740cebd0 0x7f157415cfc0 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f157406e370 tx=0x7f156c01c920 comp rx=0 tx=0).stop 2024-04-25T10:19:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 -- 172.21.15.133:0/1048444226 shutdown_connections 2024-04-25T10:19:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 --2- 172.21.15.133:0/1048444226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15740cebd0 0x7f157415cfc0 unknown :-1 s=CLOSED pgs=3083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 --2- 172.21.15.133:0/1048444226 >> v2:172.21.15.162:3300/0 conn(0x7f157415b910 0x7f15740ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 -- 172.21.15.133:0/1048444226 >> 172.21.15.133:0/1048444226 conn(0x7f157406d370 msgr2=0x7f157406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 -- 172.21.15.133:0/1048444226 shutdown_connections 2024-04-25T10:19:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 -- 172.21.15.133:0/1048444226 wait complete. 2024-04-25T10:19:17.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 Processor -- start 2024-04-25T10:19:17.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 -- start start 2024-04-25T10:19:17.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15740cebd0 0x7f15741437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f157415b530 0x7f15741647c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f157415b910 0x7f15740b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157415de90 con 0x7f15740cebd0 2024-04-25T10:19:17.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157415dd10 con 0x7f157415b910 2024-04-25T10:19:17.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f157a99e640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157415e010 con 0x7f157415b530 2024-04-25T10:19:17.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f1573fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15740cebd0 0x7f15741437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f1573fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15740cebd0 0x7f15741437d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50826/0 (socket says 172.21.15.133:50826) 2024-04-25T10:19:17.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f1573fff640 1 -- 172.21.15.133:0/1458230709 learned_addr learned my addr 172.21.15.133:0/1458230709 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:17.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f15737fe640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f157415b530 0x7f15741647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f1573fff640 1 -- 172.21.15.133:0/1458230709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f157415b530 msgr2=0x7f15741647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f1578f3b640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f157415b910 0x7f15740b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.573+0000 7f1573fff640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f157415b530 0x7f15741647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1573fff640 1 -- 172.21.15.133:0/1458230709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f157415b910 msgr2=0x7f15740b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1573fff640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f157415b910 0x7f15740b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1573fff640 1 -- 172.21.15.133:0/1458230709 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1574143d10 con 0x7f15740cebd0 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1573fff640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15740cebd0 0x7f15741437d0 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 crypto rx=0x7f156400b440 tx=0x7f156400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1570ff9640 1 -- 172.21.15.133:0/1458230709 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564015020 con 0x7f15740cebd0 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1570ff9640 1 -- 172.21.15.133:0/1458230709 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1564011890 con 0x7f15740cebd0 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f157a99e640 1 -- 172.21.15.133:0/1458230709 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15740b3a60 con 0x7f15740cebd0 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1570ff9640 1 -- 172.21.15.133:0/1458230709 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564011c80 con 0x7f15740cebd0 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f157a99e640 1 -- 172.21.15.133:0/1458230709 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15740b4020 con 0x7f15740cebd0 2024-04-25T10:19:17.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f157a99e640 1 -- 172.21.15.133:0/1458230709 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1574069000 con 0x7f15740cebd0 2024-04-25T10:19:17.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1570ff9640 1 -- 172.21.15.133:0/1458230709 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1564018020 con 0x7f15740cebd0 2024-04-25T10:19:17.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1570ff9640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1560066f90 0x7f1560069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1570ff9640 1 -- 172.21.15.133:0/1458230709 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f15640852b0 con 0x7f15740cebd0 2024-04-25T10:19:17.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f15737fe640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1560066f90 0x7f1560069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1570ff9640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] conn(0x7f156006d4d0 0x7f156006f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1570ff9640 1 -- 172.21.15.133:0/1458230709 --> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f156006ffe0 con 0x7f156006d4d0 2024-04-25T10:19:17.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1570ff9640 1 -- 172.21.15.133:0/1458230709 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1564085740 con 0x7f15740cebd0 2024-04-25T10:19:17.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f15737fe640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1560066f90 0x7f1560069450 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1574146a10 tx=0x7f1568008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1578f3b640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] conn(0x7f156006d4d0 0x7f156006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.577+0000 7f1578f3b640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] conn(0x7f156006d4d0 0x7f156006f910 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.581+0000 7f1570ff9640 1 -- 172.21.15.133:0/1458230709 <== osd.7 v2:172.21.15.162:6800/1166832594 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f156006ffe0 con 0x7f156006d4d0 2024-04-25T10:19:17.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f157a99e640 1 -- 172.21.15.133:0/1458230709 --> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f15740ae730 con 0x7f156006d4d0 2024-04-25T10:19:17.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f1570ff9640 1 -- 172.21.15.133:0/1458230709 <== osd.7 v2:172.21.15.162:6800/1166832594 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f15740ae730 con 0x7f156006d4d0 2024-04-25T10:19:17.595 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T10:19:17.595 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:19:17.595 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T10:19:17.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f157a99e640 1 -- 172.21.15.133:0/1458230709 >> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] conn(0x7f156006d4d0 msgr2=0x7f156006f910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f157a99e640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.162:6800/1166832594,v1:172.21.15.162:6802/1166832594] conn(0x7f156006d4d0 0x7f156006f910 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f157a99e640 1 -- 172.21.15.133:0/1458230709 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1560066f90 msgr2=0x7f1560069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f157a99e640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f1560066f90 0x7f1560069450 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1574146a10 tx=0x7f1568008040 comp rx=0 tx=0).stop 2024-04-25T10:19:17.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f157a99e640 1 -- 172.21.15.133:0/1458230709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15740cebd0 msgr2=0x7f15741437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f157a99e640 1 --2- 172.21.15.133:0/1458230709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15740cebd0 0x7f15741437d0 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 crypto rx=0x7f156400b440 tx=0x7f156400b900 comp rx=0 tx=0).stop 2024-04-25T10:19:17.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f1578f3b640 1 -- 172.21.15.133:0/1458230709 reap_dead start 2024-04-25T10:19:17.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f157a99e640 1 -- 172.21.15.133:0/1458230709 shutdown_connections 2024-04-25T10:19:17.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f157a99e640 1 -- 172.21.15.133:0/1458230709 >> 172.21.15.133:0/1458230709 conn(0x7f157406d370 msgr2=0x7f1574157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:17.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f157a99e640 1 -- 172.21.15.133:0/1458230709 shutdown_connections 2024-04-25T10:19:17.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.593+0000 7f157a99e640 1 -- 172.21.15.133:0/1458230709 wait complete. 2024-04-25T10:19:17.610 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:19:17.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f4768323640 1 Processor -- start 2024-04-25T10:19:17.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f4768323640 1 -- start start 2024-04-25T10:19:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f4768323640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f47600d1190 0x7f47600cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f4768323640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47600cde70 con 0x7f47600c3540 2024-04-25T10:19:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f4768323640 1 -- --> v1:172.21.15.162:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47600cd5d0 con 0x7f47600b1290 2024-04-25T10:19:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f4768323640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47600cee60 con 0x7f47600d1190 2024-04-25T10:19:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47668c0640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f47600c3540 0x7f47600cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44682/0 (socket says 172.21.15.133:44682) 2024-04-25T10:19:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47668c0640 1 -- 172.21.15.133:0/2990584646 learned_addr learned my addr 172.21.15.133:0/2990584646 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47658be640 1 --2- 172.21.15.133:0/2990584646 >> v2:172.21.15.162:3300/0 conn(0x7f47600d1190 0x7f47600cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034610770 0 0) 0x7f47600cde70 con 0x7f47600c3540 2024-04-25T10:19:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4748003530 con 0x7f47600c3540 2024-04-25T10:19:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3077752023 0 0) 0x7f4748003530 con 0x7f47600c3540 2024-04-25T10:19:17.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 >> v2:172.21.15.162:3300/0 conn(0x7f47600d1190 msgr2=0x7f47600cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 --2- 172.21.15.133:0/2990584646 >> v2:172.21.15.162:3300/0 conn(0x7f47600d1190 0x7f47600cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 >> v1:172.21.15.162:6789/0 conn(0x7f47600b1290 legacy=0x7f47600ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47600cd8b0 con 0x7f47600c3540 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f4750001680 con 0x7f47600c3540 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 >> v1:172.21.15.133:6789/0 conn(0x7f47600c3540 legacy=0x7f47600cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47480039c0 0x7f4748003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4748004810 0x7f4748004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47480051d0 0x7f474800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47600cee60 con 0x7f47480039c0 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474800c070 con 0x7f47480051d0 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474800c270 con 0x7f4748004810 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47660bf640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47480039c0 0x7f4748003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47658be640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4748004810 0x7f4748004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47668c0640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47480051d0 0x7f474800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47660bf640 1 -- 172.21.15.133:0/2990584646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4748004810 msgr2=0x7f4748004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47660bf640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4748004810 0x7f4748004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47660bf640 1 -- 172.21.15.133:0/2990584646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47480051d0 msgr2=0x7f474800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47660bf640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f47480051d0 0x7f474800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47668c0640 1 -- 172.21.15.133:0/2990584646 reap_dead start 2024-04-25T10:19:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47660bf640 1 -- 172.21.15.133:0/2990584646 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4748003530 con 0x7f47480039c0 2024-04-25T10:19:17.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.729+0000 7f47658be640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f4748004810 0x7f4748004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:17.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f47660bf640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47480039c0 0x7f4748003de0 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7f4758001e70 tx=0x7f475801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63488e13d3aeba38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758001a90 con 0x7f47480039c0 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4758003b80 con 0x7f47480039c0 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f47650bd640 1 -- 172.21.15.133:0/2990584646 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758005080 con 0x7f47480039c0 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f4768323640 1 -- 172.21.15.133:0/2990584646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47480039c0 msgr2=0x7f4748003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f4768323640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47480039c0 0x7f4748003de0 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7f4758001e70 tx=0x7f475801c730 comp rx=0 tx=0).stop 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f4768323640 1 -- 172.21.15.133:0/2990584646 shutdown_connections 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f4768323640 1 --2- 172.21.15.133:0/2990584646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47480039c0 0x7f4748003de0 unknown :-1 s=CLOSED pgs=3086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f4768323640 1 -- 172.21.15.133:0/2990584646 >> 172.21.15.133:0/2990584646 conn(0x7f47600999d0 msgr2=0x7f47600d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f4768323640 1 -- 172.21.15.133:0/2990584646 shutdown_connections 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f4768323640 1 -- 172.21.15.133:0/2990584646 wait complete. 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f4768323640 1 Processor -- start 2024-04-25T10:19:17.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.733+0000 7f4768323640 1 -- start start 2024-04-25T10:19:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f4768323640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47480039c0 0x7f47600b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f4768323640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47600c6b60 0x7f4760160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f4768323640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4760153a80 0x7f47600b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f4768323640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47600cfae0 con 0x7f47600c6b60 2024-04-25T10:19:17.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f4768323640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47600cafd0 con 0x7f4760153a80 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f4768323640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47600cfc60 con 0x7f47480039c0 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47658be640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47600c6b60 0x7f4760160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47658be640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47600c6b60 0x7f4760160e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50850/0 (socket says 172.21.15.133:50850) 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47658be640 1 -- 172.21.15.133:0/1938456686 learned_addr learned my addr 172.21.15.133:0/1938456686 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47660bf640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47480039c0 0x7f47600b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47668c0640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4760153a80 0x7f47600b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47658be640 1 -- 172.21.15.133:0/1938456686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47480039c0 msgr2=0x7f47600b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47658be640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47480039c0 0x7f47600b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47658be640 1 -- 172.21.15.133:0/1938456686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4760153a80 msgr2=0x7f47600b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47658be640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4760153a80 0x7f47600b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47658be640 1 -- 172.21.15.133:0/1938456686 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47601532c0 con 0x7f47600c6b60 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47658be640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47600c6b60 0x7f4760160e70 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7f47540107c0 tx=0x7f4754010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f475effd640 1 -- 172.21.15.133:0/1938456686 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475400f040 con 0x7f47600c6b60 2024-04-25T10:19:17.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f475effd640 1 -- 172.21.15.133:0/1938456686 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4754004220 con 0x7f47600c6b60 2024-04-25T10:19:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f4768323640 1 -- 172.21.15.133:0/1938456686 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47600b36d0 con 0x7f47600c6b60 2024-04-25T10:19:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f475effd640 1 -- 172.21.15.133:0/1938456686 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754010e70 con 0x7f47600c6b60 2024-04-25T10:19:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f4768323640 1 -- 172.21.15.133:0/1938456686 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47600b3c80 con 0x7f47600c6b60 2024-04-25T10:19:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f4768323640 1 -- 172.21.15.133:0/1938456686 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4760067c70 con 0x7f47600c6b60 2024-04-25T10:19:17.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f475effd640 1 -- 172.21.15.133:0/1938456686 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f475400e960 con 0x7f47600c6b60 2024-04-25T10:19:17.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f475effd640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f474c066f90 0x7f474c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:17.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f47660bf640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f474c066f90 0x7f474c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:17.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.737+0000 7f475effd640 1 -- 172.21.15.133:0/1938456686 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4754085020 con 0x7f47600c6b60 2024-04-25T10:19:17.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.741+0000 7f47660bf640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f474c066f90 0x7f474c069450 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4758005000 tx=0x7f4758009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:17.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.741+0000 7f475effd640 1 -- 172.21.15.133:0/1938456686 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4754018030 con 0x7f47600c6b60 2024-04-25T10:19:17.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.888+0000 7f4768323640 1 -- 172.21.15.133:0/1938456686 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f47600cc480 con 0x7f474c066f90 2024-04-25T10:19:17.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.888+0000 7f475effd640 1 -- 172.21.15.133:0/1938456686 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f47600cc480 con 0x7f474c066f90 2024-04-25T10:19:17.893 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 -- 172.21.15.133:0/1938456686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f474c066f90 msgr2=0x7f474c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f474c066f90 0x7f474c069450 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4758005000 tx=0x7f4758009040 comp rx=0 tx=0).stop 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 -- 172.21.15.133:0/1938456686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47600c6b60 msgr2=0x7f4760160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47600c6b60 0x7f4760160e70 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7f47540107c0 tx=0x7f4754010c80 comp rx=0 tx=0).stop 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 -- 172.21.15.133:0/1938456686 shutdown_connections 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4760153a80 0x7f47600b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f474c066f90 0x7f474c069450 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47600c6b60 0x7f4760160e70 unknown :-1 s=CLOSED pgs=3087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 --2- 172.21.15.133:0/1938456686 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f47480039c0 0x7f47600b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 -- 172.21.15.133:0/1938456686 >> 172.21.15.133:0/1938456686 conn(0x7f47600999d0 msgr2=0x7f476006e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 -- 172.21.15.133:0/1938456686 shutdown_connections 2024-04-25T10:19:17.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:17.892+0000 7f4768323640 1 -- 172.21.15.133:0/1938456686 wait complete. 2024-04-25T10:19:17.910 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:18.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.024+0000 7f22a71b5640 1 Processor -- start 2024-04-25T10:19:18.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.024+0000 7f22a71b5640 1 -- start start 2024-04-25T10:19:18.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f22a015db00 0x7f22a00b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22a00d0770 0x7f22a00d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22a01614c0 0x7f22a0161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a00d1110 con 0x7f22a00d0770 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a00ceb20 con 0x7f22a015db00 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00b7740 con 0x7f22a01614c0 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22a00d0770 0x7f22a00d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229ffff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f22a00d0770 0x7f22a00d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50858/0 (socket says 172.21.15.133:50858) 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229ffff640 1 -- 172.21.15.133:0/1092636825 learned_addr learned my addr 172.21.15.133:0/1092636825 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a5752640 1 --2- 172.21.15.133:0/1092636825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22a01614c0 0x7f22a0161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a4f51640 1 --2- 172.21.15.133:0/1092636825 >> v2:172.21.15.162:3300/0 conn(0x7f22a015db00 0x7f22a00b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229ffff640 1 -- 172.21.15.133:0/1092636825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22a01614c0 msgr2=0x7f22a0161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229ffff640 1 --2- 172.21.15.133:0/1092636825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22a01614c0 0x7f22a0161950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229ffff640 1 -- 172.21.15.133:0/1092636825 >> v2:172.21.15.162:3300/0 conn(0x7f22a015db00 msgr2=0x7f22a00b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:18.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229ffff640 1 --2- 172.21.15.133:0/1092636825 >> v2:172.21.15.162:3300/0 conn(0x7f22a015db00 0x7f22a00b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229ffff640 1 -- 172.21.15.133:0/1092636825 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a00b7a20 con 0x7f22a00d0770 2024-04-25T10:19:18.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229ffff640 1 --2- 172.21.15.133:0/1092636825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d0770 0x7f22a00d0bd0 secure :-1 s=READY pgs=3088 cs=0 l=1 rev1=1 crypto rx=0x7f22a00b53d0 tx=0x7f229001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fde014ff0da0c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229f7fe640 1 -- 172.21.15.133:0/1092636825 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22900aa610 con 0x7f22a00d0770 2024-04-25T10:19:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229f7fe640 1 -- 172.21.15.133:0/1092636825 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22900af070 con 0x7f22a00d0770 2024-04-25T10:19:18.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f229f7fe640 1 -- 172.21.15.133:0/1092636825 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22900aa9d0 con 0x7f22a00d0770 2024-04-25T10:19:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 -- 172.21.15.133:0/1092636825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d0770 msgr2=0x7f22a00d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 --2- 172.21.15.133:0/1092636825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d0770 0x7f22a00d0bd0 secure :-1 s=READY pgs=3088 cs=0 l=1 rev1=1 crypto rx=0x7f22a00b53d0 tx=0x7f229001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 -- 172.21.15.133:0/1092636825 shutdown_connections 2024-04-25T10:19:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 --2- 172.21.15.133:0/1092636825 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22a01614c0 0x7f22a0161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 --2- 172.21.15.133:0/1092636825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d0770 0x7f22a00d0bd0 unknown :-1 s=CLOSED pgs=3088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 --2- 172.21.15.133:0/1092636825 >> v2:172.21.15.162:3300/0 conn(0x7f22a015db00 0x7f22a00b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 -- 172.21.15.133:0/1092636825 >> 172.21.15.133:0/1092636825 conn(0x7f22a00727c0 msgr2=0x7f22a0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 -- 172.21.15.133:0/1092636825 shutdown_connections 2024-04-25T10:19:18.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.028+0000 7f22a71b5640 1 -- 172.21.15.133:0/1092636825 wait complete. 2024-04-25T10:19:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a71b5640 1 Processor -- start 2024-04-25T10:19:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a71b5640 1 -- start start 2024-04-25T10:19:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a71b5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d0770 0x7f22a014e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a71b5640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22a015db00 0x7f22a01f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a71b5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22a01614c0 0x7f22a00b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a71b5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00b7430 con 0x7f22a00d0770 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a71b5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00b82c0 con 0x7f22a01614c0 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a71b5640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00d18a0 con 0x7f22a015db00 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a4f51640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d0770 0x7f22a014e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a4f51640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d0770 0x7f22a014e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50864/0 (socket says 172.21.15.133:50864) 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a4f51640 1 -- 172.21.15.133:0/1326733925 learned_addr learned my addr 172.21.15.133:0/1326733925 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a4f51640 1 -- 172.21.15.133:0/1326733925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22a015db00 msgr2=0x7f22a01f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a5752640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22a01614c0 0x7f22a00b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a4f51640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22a015db00 0x7f22a01f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a4f51640 1 -- 172.21.15.133:0/1326733925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22a01614c0 msgr2=0x7f22a00b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a4f51640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22a01614c0 0x7f22a00b42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a4f51640 1 -- 172.21.15.133:0/1326733925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a0157f10 con 0x7f22a00d0770 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a4f51640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d0770 0x7f22a014e8a0 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7f229400c5f0 tx=0x7f229400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:18.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f229d7fa640 1 -- 172.21.15.133:0/1326733925 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294015070 con 0x7f22a00d0770 2024-04-25T10:19:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f229d7fa640 1 -- 172.21.15.133:0/1326733925 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2294011800 con 0x7f22a00d0770 2024-04-25T10:19:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a71b5640 1 -- 172.21.15.133:0/1326733925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a01541e0 con 0x7f22a00d0770 2024-04-25T10:19:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f229d7fa640 1 -- 172.21.15.133:0/1326733925 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294011bd0 con 0x7f22a00d0770 2024-04-25T10:19:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a71b5640 1 -- 172.21.15.133:0/1326733925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a01547a0 con 0x7f22a00d0770 2024-04-25T10:19:18.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.032+0000 7f22a71b5640 1 -- 172.21.15.133:0/1326733925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a0068ef0 con 0x7f22a00d0770 2024-04-25T10:19:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.036+0000 7f229d7fa640 1 -- 172.21.15.133:0/1326733925 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2294018070 con 0x7f22a00d0770 2024-04-25T10:19:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.036+0000 7f229d7fa640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f228c066f60 0x7f228c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.036+0000 7f229ffff640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f228c066f60 0x7f228c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.036+0000 7f229d7fa640 1 -- 172.21.15.133:0/1326733925 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2294084c70 con 0x7f22a00d0770 2024-04-25T10:19:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.036+0000 7f229ffff640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f228c066f60 0x7f228c069420 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f22900ae000 tx=0x7f2290004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:18.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.040+0000 7f229d7fa640 1 -- 172.21.15.133:0/1326733925 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2294055e30 con 0x7f22a00d0770 2024-04-25T10:19:18.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.196+0000 7f22a71b5640 1 -- 172.21.15.133:0/1326733925 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22a014ede0 con 0x7f228c066f60 2024-04-25T10:19:18.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.196+0000 7f229d7fa640 1 -- 172.21.15.133:0/1326733925 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13497 (secure 0 0 0) 0x7f22a014ede0 con 0x7f228c066f60 2024-04-25T10:19:18.199 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:18.200 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:19:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 -- 172.21.15.133:0/1326733925 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f228c066f60 msgr2=0x7f228c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f228c066f60 0x7f228c069420 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f22900ae000 tx=0x7f2290004740 comp rx=0 tx=0).stop 2024-04-25T10:19:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 -- 172.21.15.133:0/1326733925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d0770 msgr2=0x7f22a014e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d0770 0x7f22a014e8a0 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7f229400c5f0 tx=0x7f229400cab0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 -- 172.21.15.133:0/1326733925 shutdown_connections 2024-04-25T10:19:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22a01614c0 0x7f22a00b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f22a015db00 0x7f22a01f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f228c066f60 0x7f228c069420 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 --2- 172.21.15.133:0/1326733925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a00d0770 0x7f22a014e8a0 unknown :-1 s=CLOSED pgs=3089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 -- 172.21.15.133:0/1326733925 >> 172.21.15.133:0/1326733925 conn(0x7f22a00727c0 msgr2=0x7f22a00b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 -- 172.21.15.133:0/1326733925 shutdown_connections 2024-04-25T10:19:18.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:18.200+0000 7f22a71b5640 1 -- 172.21.15.133:0/1326733925 wait complete. 2024-04-25T10:19:18.218 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":721,"stamp":"2024-04-25T10:19:18.039573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":566136,"kb_used_data":21064,"kb_used_omap":12,"kb_used_meta":544947,"kb_avail":754408584,"statfs":{"total":773094113280,"available":772514390016,"internally_reserved":0,"allocated":21569536,"data_stored":10767224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":558026289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12582912,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-8,"num_write_kb":-12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.850571"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":511,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-04-25T10:19:10.766052+0000","last_change":"2024-04-25T10:01:41.215016+0000","last_active":"2024-04-25T10:19:10.766052+0000","last_peered":"2024-04-25T10:19:10.766052+0000","last_clean":"2024-04-25T10:19:10.766052+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:19:10.766052+0000","last_undegraded":"2024-04-25T10:19:10.766052+0000","last_fullsized":"2024-04-25T10:19:10.766052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:39.845174+0000","last_clean_scrub_stamp":"2024-04-25T10:01:39.845174+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:35.765029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575063,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74508,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297332,"statfs":{"total":96636764160,"available":96560467968,"internally_reserved":0,"allocated":3141632,"data_stored":1788679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73860,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297980,"statfs":{"total":96636764160,"available":96561131520,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73071046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575063,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51720,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320120,"statfs":{"total":96636764160,"available":96583802880,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96136,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275704,"statfs":{"total":96636764160,"available":96538320896,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83656,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288184,"statfs":{"total":96636764160,"available":96551100416,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83098055},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55876,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315964,"statfs":{"total":96636764160,"available":96579547136,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51716,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320124,"statfs":{"total":96636764160,"available":96583806976,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575064,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78664,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293176,"statfs":{"total":96636764160,"available":96556212224,"internally_reserved":0,"allocated":3141632,"data_stored":1788679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:18.221 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T10:01:39.845174+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=1, tm_sec=39, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=19, tm_sec=13, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:19:18.221 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T10:19:24.595 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:24.598 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:38.222 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f1900097640 1 Processor -- start 2024-04-25T10:19:38.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f1900097640 1 -- start start 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f1900097640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18f80b1f70 0x7f18f806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f1900097640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f18f80ce330 0x7f18f80ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f1900097640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f80c61d0 con 0x7f18f815c9a0 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f1900097640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f80cece0 con 0x7f18f80ce330 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f1900097640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80c64b0 con 0x7f18f80b1f70 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fde33640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f18f815c9a0 0x7f18f80c5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36172/0 (socket says 172.21.15.133:36172) 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fde33640 1 -- 172.21.15.133:0/2196606165 learned_addr learned my addr 172.21.15.133:0/2196606165 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fe634640 1 --2- 172.21.15.133:0/2196606165 >> v2:172.21.15.162:3300/0 conn(0x7f18f80ce330 0x7f18f80ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fd632640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18f80b1f70 0x7f18f806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282485564 0 0) 0x7f18f80c61d0 con 0x7f18f815c9a0 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e4003530 con 0x7f18f815c9a0 2024-04-25T10:19:38.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2787196069 0 0) 0x7f18e4003530 con 0x7f18f815c9a0 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18f80b1f70 msgr2=0x7f18f806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18f80b1f70 0x7f18f806dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 >> v2:172.21.15.162:3300/0 conn(0x7f18f80ce330 msgr2=0x7f18f80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 --2- 172.21.15.133:0/2196606165 >> v2:172.21.15.162:3300/0 conn(0x7f18f80ce330 0x7f18f80ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f806d4f0 con 0x7f18f815c9a0 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f18f4001680 con 0x7f18f815c9a0 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 >> v1:172.21.15.133:6789/0 conn(0x7f18f815c9a0 legacy=0x7f18f80c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18f80b1f70 0x7f18e40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18e4003f40 0x7f18e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18e4004d50 0x7f18e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80cece0 con 0x7f18e4004d50 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f80c64b0 con 0x7f18e4003f40 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e400b9d0 con 0x7f18f80b1f70 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fe634640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18e4004d50 0x7f18e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fd632640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18e4003f40 0x7f18e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fde33640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18f80b1f70 0x7f18e40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fe634640 1 -- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18f80b1f70 msgr2=0x7f18e40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fe634640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18f80b1f70 0x7f18e40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fe634640 1 -- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18e4003f40 msgr2=0x7f18e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fe634640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18e4003f40 0x7f18e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fe634640 1 -- 172.21.15.133:0/2196606165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e4003530 con 0x7f18e4004d50 2024-04-25T10:19:38.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.352+0000 7f18fd632640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18e4003f40 0x7f18e4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:38.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f18fe634640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18e4004d50 0x7f18e40051d0 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009fc0 tx=0x7f18f001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a3d7ceeccd11cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f00b2070 con 0x7f18e4004d50 2024-04-25T10:19:38.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f00a5cd0 con 0x7f18e4004d50 2024-04-25T10:19:38.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f18fce31640 1 -- 172.21.15.133:0/2196606165 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0009c10 con 0x7f18e4004d50 2024-04-25T10:19:38.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 -- 172.21.15.133:0/2196606165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18e4004d50 msgr2=0x7f18e40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 --2- 172.21.15.133:0/2196606165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18e4004d50 0x7f18e40051d0 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009fc0 tx=0x7f18f001cc60 comp rx=0 tx=0).stop 2024-04-25T10:19:38.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f18fe634640 1 -- 172.21.15.133:0/2196606165 reap_dead start 2024-04-25T10:19:38.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 -- 172.21.15.133:0/2196606165 shutdown_connections 2024-04-25T10:19:38.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 -- 172.21.15.133:0/2196606165 >> 172.21.15.133:0/2196606165 conn(0x7f18f8067000 msgr2=0x7f18f806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:38.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 -- 172.21.15.133:0/2196606165 shutdown_connections 2024-04-25T10:19:38.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 -- 172.21.15.133:0/2196606165 wait complete. 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 Processor -- start 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 -- start start 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18e4004d50 0x7f18f80b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80b0cb0 0x7f18f80b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f80c9e40 0x7f18f80ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f815cf00 con 0x7f18f80b0cb0 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f815cd80 con 0x7f18f80c9e40 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f1900097640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f806dcf0 con 0x7f18e4004d50 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.356+0000 7f18fd632640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80b0cb0 0x7f18f80b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fd632640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80b0cb0 0x7f18f80b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35884/0 (socket says 172.21.15.133:35884) 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fd632640 1 -- 172.21.15.133:0/3712489796 learned_addr learned my addr 172.21.15.133:0/3712489796 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:38.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fde33640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18e4004d50 0x7f18f80b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fe634640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f80c9e40 0x7f18f80ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fd632640 1 -- 172.21.15.133:0/3712489796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18e4004d50 msgr2=0x7f18f80b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fd632640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18e4004d50 0x7f18f80b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fd632640 1 -- 172.21.15.133:0/3712489796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f80c9e40 msgr2=0x7f18f80ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fd632640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f80c9e40 0x7f18f80ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fd632640 1 -- 172.21.15.133:0/3712489796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f81f37c0 con 0x7f18f80b0cb0 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fd632640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80b0cb0 0x7f18f80b1130 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7f18e80115f0 tx=0x7f18e8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18ee7fc640 1 -- 172.21.15.133:0/3712489796 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8004220 con 0x7f18f80b0cb0 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18ee7fc640 1 -- 172.21.15.133:0/3712489796 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e80043c0 con 0x7f18f80b0cb0 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f1900097640 1 -- 172.21.15.133:0/3712489796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f81f3aa0 con 0x7f18f80b0cb0 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18ee7fc640 1 -- 172.21.15.133:0/3712489796 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e800f8f0 con 0x7f18f80b0cb0 2024-04-25T10:19:38.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f1900097640 1 -- 172.21.15.133:0/3712489796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f81f4060 con 0x7f18f80b0cb0 2024-04-25T10:19:38.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f1900097640 1 -- 172.21.15.133:0/3712489796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f8068ef0 con 0x7f18f80b0cb0 2024-04-25T10:19:38.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18ee7fc640 1 -- 172.21.15.133:0/3712489796 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18e800fa90 con 0x7f18f80b0cb0 2024-04-25T10:19:38.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18ee7fc640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18dc066f60 0x7f18dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18ee7fc640 1 -- 172.21.15.133:0/3712489796 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f18e8085650 con 0x7f18f80b0cb0 2024-04-25T10:19:38.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fde33640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18dc066f60 0x7f18dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.360+0000 7f18fde33640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18dc066f60 0x7f18dc069420 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f18f4001ec0 tx=0x7f18f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.364+0000 7f18ee7fc640 1 -- 172.21.15.133:0/3712489796 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18e8019030 con 0x7f18f80b0cb0 2024-04-25T10:19:38.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.516+0000 7f1900097640 1 -- 172.21.15.133:0/3712489796 --> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18f815e2e0 con 0x7f18dc066f60 2024-04-25T10:19:38.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f18ee7fc640 1 -- 172.21.15.133:0/3712489796 <== mgr.4101 v2:172.21.15.133:6832/1012310613 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13387 (secure 0 0 0) 0x7f18f815e2e0 con 0x7f18dc066f60 2024-04-25T10:19:38.522 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:38.522 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-04-25T10:19:38.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 -- 172.21.15.133:0/3712489796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18dc066f60 msgr2=0x7f18dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18dc066f60 0x7f18dc069420 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f18f4001ec0 tx=0x7f18f4009040 comp rx=0 tx=0).stop 2024-04-25T10:19:38.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 -- 172.21.15.133:0/3712489796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80b0cb0 msgr2=0x7f18f80b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80b0cb0 0x7f18f80b1130 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7f18e80115f0 tx=0x7f18e8011ab0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 -- 172.21.15.133:0/3712489796 shutdown_connections 2024-04-25T10:19:38.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18f80c9e40 0x7f18f80ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f18dc066f60 0x7f18dc069420 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18f80b0cb0 0x7f18f80b1130 unknown :-1 s=CLOSED pgs=3092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 --2- 172.21.15.133:0/3712489796 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f18e4004d50 0x7f18f80b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 -- 172.21.15.133:0/3712489796 >> 172.21.15.133:0/3712489796 conn(0x7f18f8067000 msgr2=0x7f18f815b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:38.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 -- 172.21.15.133:0/3712489796 shutdown_connections 2024-04-25T10:19:38.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.520+0000 7f1900097640 1 -- 172.21.15.133:0/3712489796 wait complete. 2024-04-25T10:19:38.539 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":731,"stamp":"2024-04-25T10:19:38.045557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":566124,"kb_used_data":21064,"kb_used_omap":12,"kb_used_meta":544947,"kb_avail":754408596,"statfs":{"total":773094113280,"available":772514402304,"internally_reserved":0,"allocated":21569536,"data_stored":10767224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":558026289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003758"},"pg_stats":[{"pgid":"1.0","version":"201'119","reported_seq":522,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-04-25T10:19:18.350130+0000","last_change":"2024-04-25T10:19:18.350201+0000","last_active":"2024-04-25T10:19:18.350130+0000","last_peered":"2024-04-25T10:19:18.350130+0000","last_clean":"2024-04-25T10:19:18.350130+0000","last_became_active":"2024-04-25T10:01:41.209816+0000","last_became_peered":"2024-04-25T10:01:41.209816+0000","last_unstale":"2024-04-25T10:19:18.350130+0000","last_undegraded":"2024-04-25T10:19:18.350130+0000","last_fullsized":"2024-04-25T10:19:18.350130+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'119","last_scrub_stamp":"2024-04-25T10:19:18.349933+0000","last_deep_scrub":"201'119","last_deep_scrub_stamp":"2024-04-25T10:19:18.349933+0000","last_clean_scrub_stamp":"2024-04-25T10:19:18.349933+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:41.475841+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575067,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74500,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297340,"statfs":{"total":96636764160,"available":96560476160,"internally_reserved":0,"allocated":3141632,"data_stored":1788679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73860,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94297980,"statfs":{"total":96636764160,"available":96561131520,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73071046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51720,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320120,"statfs":{"total":96636764160,"available":96583802880,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":96136,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94275704,"statfs":{"total":96636764160,"available":96538320896,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":83656,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288184,"statfs":{"total":96636764160,"available":96551100416,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83098055},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55876,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315964,"statfs":{"total":96636764160,"available":96579547136,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51716,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320124,"statfs":{"total":96636764160,"available":96583806976,"internally_reserved":0,"allocated":2547712,"data_stored":1198311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575068,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78660,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293180,"statfs":{"total":96636764160,"available":96556216320,"internally_reserved":0,"allocated":3141632,"data_stored":1788679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:38.540 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:19:38.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.660+0000 7f0f57563640 1 Processor -- start 2024-04-25T10:19:38.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.660+0000 7f0f57563640 1 -- start start 2024-04-25T10:19:38.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f57563640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015ba80 0x7f0f500cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f57563640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7f0f5015cb80 0x7f0f500afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f57563640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f500ccd80 con 0x7f0f500b02a0 2024-04-25T10:19:38.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f57563640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f500cc870 con 0x7f0f5015cb80 2024-04-25T10:19:38.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f57563640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5015be60 con 0x7f0f5015ba80 2024-04-25T10:19:38.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f55b00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0f500b02a0 0x7f0f500ccf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:36178/0 (socket says 172.21.15.133:36178) 2024-04-25T10:19:38.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f55b00640 1 -- 172.21.15.133:0/2014419368 learned_addr learned my addr 172.21.15.133:0/2014419368 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-04-25T10:19:38.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f54afe640 1 --2- 172.21.15.133:0/2014419368 >> v2:172.21.15.162:3300/0 conn(0x7f0f5015cb80 0x7f0f500afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f552ff640 1 --2- 172.21.15.133:0/2014419368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015ba80 0x7f0f500cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188606483 0 0) 0x7f0f500ccd80 con 0x7f0f500b02a0 2024-04-25T10:19:38.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f3c003530 con 0x7f0f500b02a0 2024-04-25T10:19:38.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3837434190 0 0) 0x7f0f3c003530 con 0x7f0f500b02a0 2024-04-25T10:19:38.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015ba80 msgr2=0x7f0f500cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 --2- 172.21.15.133:0/2014419368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015ba80 0x7f0f500cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 >> v2:172.21.15.162:3300/0 conn(0x7f0f5015cb80 msgr2=0x7f0f500afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 --2- 172.21.15.133:0/2014419368 >> v2:172.21.15.162:3300/0 conn(0x7f0f5015cb80 0x7f0f500afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f500cadc0 con 0x7f0f500b02a0 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 833204609 0 0) 0x7f0f48001a00 con 0x7f0f500b02a0 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 >> v1:172.21.15.133:6789/0 conn(0x7f0f500b02a0 legacy=0x7f0f500ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 --2- 172.21.15.133:0/2014419368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 0x7f0f3c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 --2- 172.21.15.133:0/2014419368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f3c004810 0x7f0f3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 --2- 172.21.15.133:0/2014419368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015ba80 0x7f0f3c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5015be60 con 0x7f0f3c0039c0 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c00b9d0 con 0x7f0f3c004810 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c00bb50 con 0x7f0f5015ba80 2024-04-25T10:19:38.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f552ff640 1 --2- 172.21.15.133:0/2014419368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 0x7f0f3c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f552ff640 1 -- 172.21.15.133:0/2014419368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015ba80 msgr2=0x7f0f3c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f552ff640 1 --2- 172.21.15.133:0/2014419368 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015ba80 0x7f0f3c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f552ff640 1 -- 172.21.15.133:0/2014419368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f3c004810 msgr2=0x7f0f3c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f552ff640 1 --2- 172.21.15.133:0/2014419368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f3c004810 0x7f0f3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.664+0000 7f0f552ff640 1 -- 172.21.15.133:0/2014419368 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c003530 con 0x7f0f3c0039c0 2024-04-25T10:19:38.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f552ff640 1 --2- 172.21.15.133:0/2014419368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 0x7f0f3c003de0 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009ae0 tx=0x7f0f4c01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74c8e9b691e38c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0096b0 con 0x7f0f3c0039c0 2024-04-25T10:19:38.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c0a6490 con 0x7f0f3c0039c0 2024-04-25T10:19:38.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f47fff640 1 -- 172.21.15.133:0/2014419368 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0a6880 con 0x7f0f3c0039c0 2024-04-25T10:19:38.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 -- 172.21.15.133:0/2014419368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 msgr2=0x7f0f3c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 --2- 172.21.15.133:0/2014419368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 0x7f0f3c003de0 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009ae0 tx=0x7f0f4c01ca80 comp rx=0 tx=0).stop 2024-04-25T10:19:38.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f55b00640 1 -- 172.21.15.133:0/2014419368 reap_dead start 2024-04-25T10:19:38.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 -- 172.21.15.133:0/2014419368 shutdown_connections 2024-04-25T10:19:38.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 -- 172.21.15.133:0/2014419368 >> 172.21.15.133:0/2014419368 conn(0x7f0f500d0570 msgr2=0x7f0f500d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:38.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 -- 172.21.15.133:0/2014419368 shutdown_connections 2024-04-25T10:19:38.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 -- 172.21.15.133:0/2014419368 wait complete. 2024-04-25T10:19:38.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 Processor -- start 2024-04-25T10:19:38.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 -- start start 2024-04-25T10:19:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 0x7f0f501f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015cff0 0x7f0f5015fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f50163860 0x7f0f50164870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5015c660 con 0x7f0f3c0039c0 2024-04-25T10:19:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5015c380 con 0x7f0f50163860 2024-04-25T10:19:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f57563640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50166530 con 0x7f0f5015cff0 2024-04-25T10:19:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f552ff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 0x7f0f501f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f552ff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 0x7f0f501f8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35912/0 (socket says 172.21.15.133:35912) 2024-04-25T10:19:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f552ff640 1 -- 172.21.15.133:0/3014987742 learned_addr learned my addr 172.21.15.133:0/3014987742 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f54afe640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015cff0 0x7f0f5015fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f55b00640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f50163860 0x7f0f50164870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f552ff640 1 -- 172.21.15.133:0/3014987742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015cff0 msgr2=0x7f0f5015fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f552ff640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015cff0 0x7f0f5015fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f552ff640 1 -- 172.21.15.133:0/3014987742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f50163860 msgr2=0x7f0f50164870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f552ff640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f50163860 0x7f0f50164870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f552ff640 1 -- 172.21.15.133:0/3014987742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5015e830 con 0x7f0f3c0039c0 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.668+0000 7f0f55b00640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f50163860 0x7f0f50164870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f552ff640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 0x7f0f501f8b60 secure :-1 s=READY pgs=3095 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c01cf60 tx=0x7f0f3c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f45ffb640 1 -- 172.21.15.133:0/3014987742 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0a6bc0 con 0x7f0f3c0039c0 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f45ffb640 1 -- 172.21.15.133:0/3014987742 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c0a6d60 con 0x7f0f3c0039c0 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f57563640 1 -- 172.21.15.133:0/3014987742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f501f90a0 con 0x7f0f3c0039c0 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f45ffb640 1 -- 172.21.15.133:0/3014987742 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0a7690 con 0x7f0f3c0039c0 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f57563640 1 -- 172.21.15.133:0/3014987742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5015ed90 con 0x7f0f3c0039c0 2024-04-25T10:19:38.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f57563640 1 -- 172.21.15.133:0/3014987742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f50067c70 con 0x7f0f3c0039c0 2024-04-25T10:19:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f45ffb640 1 -- 172.21.15.133:0/3014987742 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f4c0a7830 con 0x7f0f3c0039c0 2024-04-25T10:19:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f45ffb640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f34066f60 0x7f0f34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f45ffb640 1 -- 172.21.15.133:0/3014987742 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f0f4c11f320 con 0x7f0f3c0039c0 2024-04-25T10:19:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f54afe640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f34066f60 0x7f0f34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.672+0000 7f0f54afe640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f34066f60 0x7f0f34069420 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0f50166290 tx=0x7f0f40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.676+0000 7f0f45ffb640 1 -- 172.21.15.133:0/3014987742 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f501f90a0 con 0x7f0f3c0039c0 2024-04-25T10:19:38.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.836+0000 7f0f57563640 1 -- 172.21.15.133:0/3014987742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f0f500b6430 con 0x7f0f3c0039c0 2024-04-25T10:19:38.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.840+0000 7f0f45ffb640 1 -- 172.21.15.133:0/3014987742 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v38)=0 v38) v1 ==== 123+0+0 (secure 0 0 0) 0x7f0f5015e830 con 0x7f0f3c0039c0 2024-04-25T10:19:38.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.840+0000 7f0f45ffb640 1 -- 172.21.15.133:0/3014987742 <== mon.0 v2:172.21.15.133:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0f4c0f4e50 con 0x7f0f3c0039c0 2024-04-25T10:19:38.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 -- 172.21.15.133:0/3014987742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f34066f60 msgr2=0x7f0f34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f34066f60 0x7f0f34069420 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0f50166290 tx=0x7f0f40008040 comp rx=0 tx=0).stop 2024-04-25T10:19:38.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 -- 172.21.15.133:0/3014987742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 msgr2=0x7f0f501f8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 0x7f0f501f8b60 secure :-1 s=READY pgs=3095 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c01cf60 tx=0x7f0f3c005840 comp rx=0 tx=0).stop 2024-04-25T10:19:38.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 -- 172.21.15.133:0/3014987742 shutdown_connections 2024-04-25T10:19:38.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f50163860 0x7f0f50164870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f0f5015cff0 0x7f0f5015fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7f0f34066f60 0x7f0f34069420 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 --2- 172.21.15.133:0/3014987742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f3c0039c0 0x7f0f501f8b60 unknown :-1 s=CLOSED pgs=3095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 -- 172.21.15.133:0/3014987742 >> 172.21.15.133:0/3014987742 conn(0x7f0f500d0570 msgr2=0x7f0f500ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:38.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 -- 172.21.15.133:0/3014987742 shutdown_connections 2024-04-25T10:19:38.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.844+0000 7f0f57563640 1 -- 172.21.15.133:0/3014987742 wait complete. 2024-04-25T10:19:38.863 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:19:38.864 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:38.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.980+0000 7fbb99af0640 1 Processor -- start 2024-04-25T10:19:38.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.980+0000 7fbb99af0640 1 -- start start 2024-04-25T10:19:38.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb99af0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb94158110 0x7fbb940b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb99af0640 1 --2- >> v2:172.21.15.162:3300/0 conn(0x7fbb941584f0 0x7fbb9415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb99af0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbb9406e460 0x7fbb9406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb99af0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb9406edf0 con 0x7fbb9406e460 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb99af0640 1 -- --> v2:172.21.15.162:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb940cbe60 con 0x7fbb941584f0 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb99af0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940b19b0 con 0x7fbb94158110 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb93fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbb9406e460 0x7fbb9406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb93fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbb9406e460 0x7fbb9406e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35924/0 (socket says 172.21.15.133:35924) 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb93fff640 1 -- 172.21.15.133:0/1699752081 learned_addr learned my addr 172.21.15.133:0/1699752081 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb937fe640 1 --2- 172.21.15.133:0/1699752081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb94158110 0x7fbb940b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb93fff640 1 -- 172.21.15.133:0/1699752081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb94158110 msgr2=0x7fbb940b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb93fff640 1 --2- 172.21.15.133:0/1699752081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb94158110 0x7fbb940b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb92ffd640 1 --2- 172.21.15.133:0/1699752081 >> v2:172.21.15.162:3300/0 conn(0x7fbb941584f0 0x7fbb9415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb93fff640 1 -- 172.21.15.133:0/1699752081 >> v2:172.21.15.162:3300/0 conn(0x7fbb941584f0 msgr2=0x7fbb9415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb93fff640 1 --2- 172.21.15.133:0/1699752081 >> v2:172.21.15.162:3300/0 conn(0x7fbb941584f0 0x7fbb9415ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb93fff640 1 -- 172.21.15.133:0/1699752081 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9415b360 con 0x7fbb9406e460 2024-04-25T10:19:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb937fe640 1 --2- 172.21.15.133:0/1699752081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb94158110 0x7fbb940b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:38.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb92ffd640 1 --2- 172.21.15.133:0/1699752081 >> v2:172.21.15.162:3300/0 conn(0x7fbb941584f0 0x7fbb9415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:38.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb93fff640 1 --2- 172.21.15.133:0/1699752081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb9406e460 0x7fbb9406e8b0 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7fbb9406f050 tx=0x7fbb8c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a868610b97d7a9c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb927fc640 1 -- 172.21.15.133:0/1699752081 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0aa610 con 0x7fbb9406e460 2024-04-25T10:19:38.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb927fc640 1 -- 172.21.15.133:0/1699752081 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbb8c0af070 con 0x7fbb9406e460 2024-04-25T10:19:38.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.984+0000 7fbb927fc640 1 -- 172.21.15.133:0/1699752081 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0aa9f0 con 0x7fbb9406e460 2024-04-25T10:19:38.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 -- 172.21.15.133:0/1699752081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb9406e460 msgr2=0x7fbb9406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 --2- 172.21.15.133:0/1699752081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb9406e460 0x7fbb9406e8b0 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7fbb9406f050 tx=0x7fbb8c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 -- 172.21.15.133:0/1699752081 shutdown_connections 2024-04-25T10:19:38.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 --2- 172.21.15.133:0/1699752081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb9406e460 0x7fbb9406e8b0 unknown :-1 s=CLOSED pgs=3096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 --2- 172.21.15.133:0/1699752081 >> v2:172.21.15.162:3300/0 conn(0x7fbb941584f0 0x7fbb9415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 --2- 172.21.15.133:0/1699752081 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb94158110 0x7fbb940b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 -- 172.21.15.133:0/1699752081 >> 172.21.15.133:0/1699752081 conn(0x7fbb94067000 msgr2=0x7fbb9406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:38.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 -- 172.21.15.133:0/1699752081 shutdown_connections 2024-04-25T10:19:38.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 -- 172.21.15.133:0/1699752081 wait complete. 2024-04-25T10:19:38.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 Processor -- start 2024-04-25T10:19:38.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 -- start start 2024-04-25T10:19:38.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 --2- >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb9406e460 0x7fbb9414dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb94158110 0x7fbb94157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb941584f0 0x7fbb9415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940cd7f0 con 0x7fbb941584f0 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940cd670 con 0x7fbb94158110 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 -- --> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9415eb10 con 0x7fbb9406e460 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb941584f0 0x7fbb9415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb941584f0 0x7fbb9415ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35940/0 (socket says 172.21.15.133:35940) 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb93fff640 1 -- 172.21.15.133:0/4279535551 learned_addr learned my addr 172.21.15.133:0/4279535551 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb937fe640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb9406e460 0x7fbb9414dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb92ffd640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb94158110 0x7fbb94157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb93fff640 1 -- 172.21.15.133:0/4279535551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb9406e460 msgr2=0x7fbb9414dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb93fff640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb9406e460 0x7fbb9414dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb93fff640 1 -- 172.21.15.133:0/4279535551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb94158110 msgr2=0x7fbb94157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:38.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb93fff640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb94158110 0x7fbb94157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:38.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb93fff640 1 -- 172.21.15.133:0/4279535551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94156260 con 0x7fbb941584f0 2024-04-25T10:19:38.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb93fff640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb941584f0 0x7fbb9415ce60 secure :-1 s=READY pgs=3097 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0ac040 tx=0x7fbb8c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb7bfff640 1 -- 172.21.15.133:0/4279535551 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0b3800 con 0x7fbb941584f0 2024-04-25T10:19:38.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb7bfff640 1 -- 172.21.15.133:0/4279535551 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbb8c0aae50 con 0x7fbb941584f0 2024-04-25T10:19:38.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 -- 172.21.15.133:0/4279535551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb94156540 con 0x7fbb941584f0 2024-04-25T10:19:38.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb7bfff640 1 -- 172.21.15.133:0/4279535551 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0b3c00 con 0x7fbb941584f0 2024-04-25T10:19:38.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.988+0000 7fbb99af0640 1 -- 172.21.15.133:0/4279535551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb94156b00 con 0x7fbb941584f0 2024-04-25T10:19:38.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.992+0000 7fbb99af0640 1 -- 172.21.15.133:0/4279535551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb94068ef0 con 0x7fbb941584f0 2024-04-25T10:19:38.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.992+0000 7fbb7bfff640 1 -- 172.21.15.133:0/4279535551 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb8c0c7420 con 0x7fbb941584f0 2024-04-25T10:19:38.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.992+0000 7fbb7bfff640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbb80066f60 0x7fbb80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:38.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.992+0000 7fbb7bfff640 1 -- 172.21.15.133:0/4279535551 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fbb8c11f910 con 0x7fbb941584f0 2024-04-25T10:19:38.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.992+0000 7fbb937fe640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbb80066f60 0x7fbb80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:38.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.992+0000 7fbb937fe640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbb80066f60 0x7fbb80069420 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fbb940b1950 tx=0x7fbb840077c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:38.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:38.996+0000 7fbb7bfff640 1 -- 172.21.15.133:0/4279535551 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb8c0c3060 con 0x7fbb941584f0 2024-04-25T10:19:39.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.172+0000 7fbb99af0640 1 -- 172.21.15.133:0/4279535551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb941542d0 con 0x7fbb941584f0 2024-04-25T10:19:39.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.172+0000 7fbb7bfff640 1 -- 172.21.15.133:0/4279535551 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+3712 (secure 0 0 0) 0x7fbb8c0a7070 con 0x7fbb941584f0 2024-04-25T10:19:39.176 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T10:19:39.176 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":126,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/220798982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":220798982},{"type":"v1","addr":"172.21.15.162:6833","nonce":220798982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":7247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/3260148614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":3260148614},{"type":"v1","addr":"172.21.15.162:6835","nonce":3260148614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11868,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2269510561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2269510561},{"type":"v1","addr":"172.21.15.133:6835","nonce":2269510561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":12420,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/163918758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":163918758},{"type":"v1","addr":"172.21.15.133:6837","nonce":163918758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126}],"filesystems":[]} 2024-04-25T10:19:39.177 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 126 2024-04-25T10:19:39.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 -- 172.21.15.133:0/4279535551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbb80066f60 msgr2=0x7fbb80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:39.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbb80066f60 0x7fbb80069420 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fbb940b1950 tx=0x7fbb840077c0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 -- 172.21.15.133:0/4279535551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb941584f0 msgr2=0x7fbb9415ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb941584f0 0x7fbb9415ce60 secure :-1 s=READY pgs=3097 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0ac040 tx=0x7fbb8c01ad30 comp rx=0 tx=0).stop 2024-04-25T10:19:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 -- 172.21.15.133:0/4279535551 shutdown_connections 2024-04-25T10:19:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.133:6832/1012310613,v1:172.21.15.133:6833/1012310613] conn(0x7fbb80066f60 0x7fbb80069420 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb941584f0 0x7fbb9415ce60 unknown :-1 s=CLOSED pgs=3097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb94158110 0x7fbb94157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 --2- 172.21.15.133:0/4279535551 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7fbb9406e460 0x7fbb9414dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 -- 172.21.15.133:0/4279535551 >> 172.21.15.133:0/4279535551 conn(0x7fbb94067000 msgr2=0x7fbb9406db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 -- 172.21.15.133:0/4279535551 shutdown_connections 2024-04-25T10:19:39.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T10:19:39.176+0000 7fbb99af0640 1 -- 172.21.15.133:0/4279535551 wait complete. 2024-04-25T10:19:39.194 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:19:39.194 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:19:39.194 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:19:39.194 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:39.210 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:19:39.210 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:19:39.210 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:39.264 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:19:39.265 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:19:39.265 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:39.284 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:19:39.284 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:19:39.284 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:39.342 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:19:39.342 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:19:39.342 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:19:39.342 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:39.433 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:19:39.434 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:19:39.434 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:39.509 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:19:39.509 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:19:39.509 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:39.588 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:19:39.588 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:19:39.588 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:39.685 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:19:39.686 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:19:39.686 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:42.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-25T10:19:45.787 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:19:45.788 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:19:45.788 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:48.620 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-25T10:19:48.621 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-25T10:19:51.890 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:19:51.890 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:19:51.890 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:51.961 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:19:51.961 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:19:51.961 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:52.060 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:19:52.061 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:19:52.061 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:19:52.061 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:52.091 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:19:52.091 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:19:52.091 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:52.118 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:19:52.119 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:19:52.119 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:19:52.119 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:52.147 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:19:52.147 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:19:52.147 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:52.177 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:19:52.177 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:19:52.177 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:52.231 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:19:52.232 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:19:52.232 DEBUG:teuthology.orchestra.run.smithi133:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T10:19:52.249 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:19:52.249 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:19:52.518 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:19:52.519 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:19:52.760 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:19:52.761 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:19:52.968 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:19:52.969 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:19:53.137 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:19:53.137 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:19:53.346 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:19:53.346 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:19:53.533 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:19:53.533 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:19:53.622 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-25T10:19:53.691 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:19:53.691 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:19:53.877 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:19:53.878 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035/data/mon.a.tgz 2024-04-25T10:19:53.878 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-04-25T10:19:53.884 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.bJTffiBLtj 2024-04-25T10:19:53.885 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.bJTffiBLtj 2024-04-25T10:19:54.041 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.bJTffiBLtj 2024-04-25T10:19:54.105 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.bJTffiBLtj is 482KB 2024-04-25T10:19:54.167 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.bJTffiBLtj 2024-04-25T10:19:54.172 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035/data/mon.b.tgz 2024-04-25T10:19:54.172 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-04-25T10:19:54.178 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.arPLPaBCal 2024-04-25T10:19:54.178 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.arPLPaBCal 2024-04-25T10:19:54.329 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.arPLPaBCal 2024-04-25T10:19:54.394 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.arPLPaBCal is 491KB 2024-04-25T10:19:54.456 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.arPLPaBCal 2024-04-25T10:19:54.461 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035/data/mon.c.tgz 2024-04-25T10:19:54.462 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-04-25T10:19:54.510 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.EZxZlPWxdb 2024-04-25T10:19:54.510 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.EZxZlPWxdb 2024-04-25T10:19:54.667 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.EZxZlPWxdb 2024-04-25T10:19:54.690 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:54.692 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:54.811 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.EZxZlPWxdb is 491KB 2024-04-25T10:19:54.872 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.EZxZlPWxdb 2024-04-25T10:19:54.877 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:19:54.877 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:19:54.880 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:19:54.940 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:19:54.982 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:19:54.982 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:19:54.991 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035/remote/smithi133/crash 2024-04-25T10:19:54.992 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:19:55.006 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035/remote/smithi162/crash 2024-04-25T10:19:55.007 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:19:55.038 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:19:55.038 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:55.054 DEBUG:teuthology.orchestra.run.smithi162:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:55.119 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:55.149 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:55.173 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:19:55.174 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:19:55.198 DEBUG:teuthology.orchestra.run.smithi162:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:19:55.223 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84013.log 2024-04-25T10:19:55.224 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73322.log 2024-04-25T10:19:55.224 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91774.log 2024-04-25T10:19:55.225 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84013.log.gz 2024-04-25T10:19:55.225 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83734.log 2024-04-25T10:19:55.226 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79593.log 2024-04-25T10:19:55.226 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.73322.log.gz 2024-04-25T10:19:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83734.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91774.log.gz 2024-04-25T10:19:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76455.log 2024-04-25T10:19:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79593.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83734.log.gz 2024-04-25T10:19:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56065.log 2024-04-25T10:19:55.227 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79593.log.gz 2024-04-25T10:19:55.227 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84330.log 2024-04-25T10:19:55.229 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51636.log 2024-04-25T10:19:55.229 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.76455.log.gz 2024-04-25T10:19:55.229 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56553.log 2024-04-25T10:19:55.229 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56065.log.gz 2024-04-25T10:19:55.229 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35299.log 2024-04-25T10:19:55.229 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.84330.log.gz 2024-04-25T10:19:55.229 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88559.log 2024-04-25T10:19:55.230 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51636.log.gz 2024-04-25T10:19:55.230 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56553.log.gz 2024-04-25T10:19:55.230 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86176.log 2024-04-25T10:19:55.230 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92655.log 2024-04-25T10:19:55.230 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35299.log.gz 2024-04-25T10:19:55.230 INFO:teuthology.orchestra.run.smithi133.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88559.log.gz 2024-04-25T10:19:55.230 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83935.log 2024-04-25T10:19:55.233 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86176.log: /var/log/ceph/ceph-client.admin.92655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71033.log 2024-04-25T10:19:55.233 INFO:teuthology.orchestra.run.smithi133.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92655.log.gz 2024-04-25T10:19:55.233 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53037.log 2024-04-25T10:19:55.233 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.86176.log.gz 2024-04-25T10:19:55.233 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33576.log 2024-04-25T10:19:55.233 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83935.log.gz 2024-04-25T10:19:55.233 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71033.log.gz 2024-04-25T10:19:55.233 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79975.log 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53037.log.gz 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86850.log 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76713.log 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33576.log.gz 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83630.log 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79975.log.gz 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76713.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86850.log.gz 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56611.log 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76713.log.gz 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37099.log 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83630.log.gz 2024-04-25T10:19:55.234 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81695.log 2024-04-25T10:19:55.237 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76687.log 2024-04-25T10:19:55.237 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56611.log.gz 2024-04-25T10:19:55.237 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81695.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37099.log.gz 2024-04-25T10:19:55.237 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55302.log 2024-04-25T10:19:55.237 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81695.log.gz 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57118.log 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.55302.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51842.log 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76687.log.gz 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84419.log 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55302.log.gz 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57118.log.gz 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93856.log 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94476.log 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.51842.log.gz 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78255.log 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84419.log.gz 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87245.log 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93856.log.gz 2024-04-25T10:19:55.238 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94476.log.gz 2024-04-25T10:19:55.239 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.78255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81430.log 2024-04-25T10:19:55.239 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87245.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27728.log 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27752.log 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27728.log: /var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27848.log 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27752.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27728.log.gz 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi162.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27752.log.gz 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33406.log 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27776.log 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27848.log.gz 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.1.33406.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27800.log 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.1.33406.log.gz 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27776.log.gz 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27879.log 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27800.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27800.log.gz 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27879.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27824.log 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27879.log.gz 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27824.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27824.log.gz 2024-04-25T10:19:55.249 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33522.log 2024-04-25T10:19:55.250 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:19:55.250 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.3.33522.log: 91.2% -- replaced with /var/log/ceph/ceph-client.3.33522.log.gz 2024-04-25T10:19:55.251 INFO:teuthology.orchestra.run.smithi162.stderr: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:19:55.252 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27704.log 2024-04-25T10:19:55.252 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:19:55.252 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27704.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27704.log.gz 2024-04-25T10:19:55.252 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:19:55.257 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mds.b.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87245.log.gz 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80764.log 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mgr.y.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.81430.log.gz --verbose 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr: -- /var/log/ceph/ceph-client.admin.51686.log 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53781.log 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.80764.log.gz 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90239.log 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80999.log 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53781.log.gz 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89399.log 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51686.log.gz 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80999.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90239.log.gz 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88429.log 2024-04-25T10:19:55.262 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.80999.log.gz 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.89399.log: /var/log/ceph/ceph-client.admin.73906.log 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34631.log 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89399.log.gz 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73906.log: gzip -5 89.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.88429.log.gz /var/log/ceph/ceph-client.admin.52959.log 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34631.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73906.log.gz 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37681.log 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34631.log.gz 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92891.log 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52959.log.gz 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92258.log 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92891.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37681.log.gz 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70817.log 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92258.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.92891.log.gz 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83072.log 2024-04-25T10:19:55.263 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70817.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92258.log.gz 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56610.log 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70817.log.gz/var/log/ceph/ceph-client.admin.83072.log: 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93551.log 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83072.log.gz 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91382.log 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56610.log.gz 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83708.log 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91382.log: gzip 88.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.93551.log.gz -- 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr: /var/log/ceph/ceph-client.admin.87375.log 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83708.log: gzip -5 89.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.91382.log.gz /var/log/ceph/ceph-client.admin.90213.log 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.83708.log.gz 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79492.log 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87375.log.gz 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.90213.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.71753.log 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38519.log 2024-04-25T10:19:55.264 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79492.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.90213.log.gz 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54225.log 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71753.log.gz 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.79492.log.gz 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90663.log 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38519.log.gz 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81274.log 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74368.log 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81274.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90663.log.gz 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54225.log.gz 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77966.log 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80712.log 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74368.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81274.log.gz 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83604.log 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74368.log.gz 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77966.log.gz 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58140.log 2024-04-25T10:19:55.265 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3%/var/log/ceph/ceph-client.admin.83604.log: -- replaced with /var/log/ceph/ceph-client.admin.80712.log.gz 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90356.log 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.83604.log.gz 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92734.log 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90356.log: gzip -5 --verbose 89.3% -- /var/log/ceph/ceph-client.admin.93284.log -- replaced with /var/log/ceph/ceph-client.admin.58140.log.gz 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92734.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90356.log.gz 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54111.log 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92734.log.gz 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54564.log 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94502.log 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38467.log 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54564.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.93284.log.gz 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54111.log.gz 2024-04-25T10:19:55.266 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88455.log 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38467.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94502.log.gz 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38922.log 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.54564.log.gz 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38467.log.gz 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91983.log 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38922.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88455.log.gz 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80485.log 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91983.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38922.log.gz 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53246.log 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80485.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91983.log.gz 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39538.log 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80485.log.gz 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34397.log 2024-04-25T10:19:55.267 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.39538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55347.log 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53246.log.gz 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34397.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34397.log.gz 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80537.log 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55347.log: /var/log/ceph/ceph-osd.0.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55347.log.gz 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88195.log 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37942.log 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80537.log: /var/log/ceph/ceph-client.admin.88195.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80537.log.gz 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53885.log 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88195.log.gz 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81025.log 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37942.log.gz 2024-04-25T10:19:55.268 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58296.log 2024-04-25T10:19:55.270 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:19:55.270 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mon.b.log: 94.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:19:55.284 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36728.log 2024-04-25T10:19:55.284 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53885.log.gz 2024-04-25T10:19:55.284 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94242.log 2024-04-25T10:19:55.284 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81025.log.gz 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39355.log 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36728.log: /var/log/ceph/ceph-client.admin.94242.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.58296.log.gz 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94242.log.gz 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36728.log.gz 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91034.log 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54275.log 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39355.log.gz 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34904.log 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74107.log 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87921.log 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.91034.log.gz 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54275.log.gz 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34904.log.gz 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74107.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74107.log.gz 2024-04-25T10:19:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74814.log 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33360.log 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77429.log 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74814.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.87921.log.gz/var/log/ceph/ceph-client.admin.33360.log: 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71163.log 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33360.log.gz 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88003.log 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77429.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.74814.log.gz 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71163.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77429.log.gz 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71163.log.gz 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92065.log 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74424.log 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92065.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88003.log.gz 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70765.log 2024-04-25T10:19:55.286 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92065.log.gz 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80407.log 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74424.log.gz 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81051.log 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80407.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70765.log.gz 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77992.log 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80407.log.gz 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56311.log 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81051.log.gz 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33771.log 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9%/var/log/ceph/ceph-client.admin.56311.log: -- replaced with /var/log/ceph/ceph-client.admin.77992.log.gz 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35543.log 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33771.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89683.log 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56311.log.gz 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35543.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33771.log.gz 2024-04-25T10:19:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91460.log 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35543.log.gz 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89422.log 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89683.log: /var/log/ceph/ceph-client.admin.91460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87505.log 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89683.log.gz 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52501.log 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91460.log.gz 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89422.log: /var/log/ceph/ceph-client.admin.87505.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.89422.log.gz 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.87505.log.gz 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39592.log 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94528.log 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52501.log.gz 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34475.log 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74580.log 2024-04-25T10:19:55.288 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94528.log: /var/log/ceph/ceph-client.admin.34475.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39592.log.gz 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90637.log 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94528.log.gz 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34475.log.gz 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74580.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74580.log.gz 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58244.log 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53456.log 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90637.log.gz 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-client.admin.58244.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55198.log 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71529.log 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58244.log.gz 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94268.log 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr: 91.4% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55198.log.gz 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71529.log: -- replaced with /var/log/ceph/ceph-client.admin.53456.log.gz 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52423.log 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71529.log.gz 2024-04-25T10:19:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93132.log 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94268.log: /var/log/ceph/ceph-client.admin.52423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74198.log 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94268.log.gz 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93132.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.52423.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.76661.log 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74198.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.93132.log.gz 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74198.log.gz 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37073.log 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71085.log 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76661.log.gz 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34913.log 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36337.log 2024-04-25T10:19:55.290 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37073.log.gz 2024-04-25T10:19:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34913.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84445.log 2024-04-25T10:19:55.322 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71085.log.gz 2024-04-25T10:19:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36337.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34913.log.gz 2024-04-25T10:19:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78200.log 2024-04-25T10:19:55.322 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36623.log 2024-04-25T10:19:55.322 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36337.log.gz/var/log/ceph/ceph-client.admin.84445.log: 2024-04-25T10:19:55.322 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.78200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84198.log 2024-04-25T10:19:55.322 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.84445.log.gz 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91566.log 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78200.log.gz 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80217.log 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36623.log.gz 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84198.log: /var/log/ceph/ceph-client.admin.91566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84065.log 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33492.log 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91566.log.gz 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.84198.log.gz 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84065.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80243.log 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80217.log.gz 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.84065.log.gz 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52527.log 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33492.log.gz 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94450.log 2024-04-25T10:19:55.323 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80243.log: /var/log/ceph/ceph-client.admin.52527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74710.log 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39329.log 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52527.log.gz 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80243.log.gz 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94450.log: /var/log/ceph/ceph-client.admin.74710.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94450.log.gz 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74710.log.gz 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79671.log 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37445.log 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39329.log: /var/log/ceph/ceph-client.admin.79671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90921.log 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37445.log: gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.54764.log 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79671.log.gz 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39329.log.gz 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90921.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37445.log.gz 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83019.log 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90921.log.gz 2024-04-25T10:19:55.324 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39277.log 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54764.log: /var/log/ceph/ceph-client.admin.83019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88169.log 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39277.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83019.log.gz 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54085.log 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54764.log.gz 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88169.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39277.log.gz 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36962.log 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88169.log.gz 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34579.log 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54085.log: /var/log/ceph/ceph-client.admin.36962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35297.log 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55016.log 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54085.log.gz 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36962.log.gz 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35297.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34579.log.gz 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86358.log 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35297.log.gz 2024-04-25T10:19:55.325 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71007.log 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55016.log: /var/log/ceph/ceph-client.admin.86358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93182.log 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71007.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89977.log 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86358.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55016.log.gz 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93182.log: 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71007.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57589.log 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36571.log 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89977.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.93182.log.gz 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93464.log 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36571.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57589.log.gz 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92336.log 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89977.log.gz 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58192.log 2024-04-25T10:19:55.326 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71690.log 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0%/var/log/ceph/ceph-client.admin.92336.log: -- replaced with /var/log/ceph/ceph-client.admin.93464.log.gz 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92786.log 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39169.log 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92336.log.gz 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58192.log.gz 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92786.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71690.log.gz 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57464.log 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.92786.log.gz 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87636.log 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39169.log: /var/log/ceph/ceph-client.admin.57464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87271.log 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.87636.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57464.log.gz 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39249.log 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39169.log.gz 2024-04-25T10:19:55.327 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87690.log 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87271.log.gz 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55764.log 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39249.log: /var/log/ceph/ceph-client.admin.87690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33611.log 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.87636.log.gz 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55764.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39249.log.gz 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58036.log 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55764.log.gz 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.87690.log.gz 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77299.log 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58036.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33611.log.gz 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37523.log 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8%/var/log/ceph/ceph-client.admin.77299.log: -- replaced with /var/log/ceph/ceph-client.admin.58036.log.gz 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33466.log 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37523.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38207.log 2024-04-25T10:19:55.328 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77299.log.gz 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5%/var/log/ceph/ceph-client.admin.33466.log: -- replaced with /var/log/ceph/ceph-client.admin.37523.log.gz 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80001.log 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.54614.log 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33466.log.gz 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38207.log: /var/log/ceph/ceph-client.admin.80001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38623.log 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38207.log.gz 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88611.log 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80001.log.gz 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54614.log: /var/log/ceph/ceph-client.admin.38623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37258.log 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88611.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38623.log.gz 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83327.log 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33386.log 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88611.log.gz 2024-04-25T10:19:55.329 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54614.log.gz 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83327.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37258.log.gz 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92206.log 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83327.log.gz 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54539.log 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33386.log: /var/log/ceph/ceph-client.admin.92206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39483.log 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77768.log 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33386.log.gz 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92206.log.gz 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81142.log 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.54539.log.gz 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39483.log.gz 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77768.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77768.log.gz 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89058.log 2024-04-25T10:19:55.330 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92232.log 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81142.log: /var/log/ceph/ceph-client.admin.89058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70505.log 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83812.log 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89058.log.gz 89.8% -- replaced with /var/log/ceph/ceph-client.admin.81142.log.gz 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70505.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.92232.log.gz 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38694.log 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93934.log 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83812.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70505.log.gz 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.38694.log: -- /var/log/ceph/ceph-client.admin.87163.log 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93934.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83812.log.gz 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38694.log.gz 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94086.log 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39078.log 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93934.log.gz 2024-04-25T10:19:55.331 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6%/var/log/ceph/ceph-client.admin.94086.log: -- replaced with /var/log/ceph/ceph-client.admin.87163.log.gz 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34423.log 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94086.log.gz 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84250.log 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39078.log: /var/log/ceph/ceph-client.admin.34423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80191.log 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81669.log 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34423.log.gz 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39078.log.gz 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80191.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35121.log 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84250.log.gz 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.80191.log.gz 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53194.log 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81669.log: /var/log/ceph/ceph-client.admin.35121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71989.log 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1%gzip/var/log/ceph/ceph-client.admin.53194.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.93082.log 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35121.log.gz 2024-04-25T10:19:55.332 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81669.log.gz 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71989.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53194.log.gz 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57928.log 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94320.log 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93082.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.71989.log.gz 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78096.log 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8%/var/log/ceph/ceph-client.admin.94320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56337.log 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57928.log.gz 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.78096.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94320.log.gz 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33414.log 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78096.log.gz 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.93082.log.gz 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86444.log 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56337.log.gz 2024-04-25T10:19:55.333 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76555.log 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33414.log: /var/log/ceph/ceph-client.admin.86444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86772.log 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93577.log 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86444.log.gz 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33414.log.gz/var/log/ceph/ceph-client.admin.86772.log: 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76555.log.gz 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86772.log.gz 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80329.log 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74684.log 2024-04-25T10:19:55.334 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93577.log: /var/log/ceph/ceph-client.admin.80329.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:19:55.360 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37968.log 2024-04-25T10:19:55.360 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80329.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93577.log.gz 2024-04-25T10:19:55.360 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.360 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.2.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74684.log.gz 2024-04-25T10:19:55.360 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57693.log 2024-04-25T10:19:55.360 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35321.log 2024-04-25T10:19:55.360 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37968.log: /var/log/ceph/ceph-client.admin.57693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74788.log 2024-04-25T10:19:55.360 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35321.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37968.log.gz 2024-04-25T10:19:55.360 INFO:teuthology.orchestra.run.smithi133.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57693.log.gz 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92126.log 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35321.log.gz 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86497.log 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80355.log 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92126.log: 88.7%/var/log/ceph/ceph-client.admin.86497.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74788.log.gz 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92126.log.gz 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86497.log.gz 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56416.log 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55425.log 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9%/var/log/ceph/ceph-client.admin.56416.log: -- replaced with /var/log/ceph/ceph-client.admin.80355.log.gzgzip 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.88029.log 2024-04-25T10:19:55.361 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56416.log.gz 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55425.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55425.log.gz 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51952.log 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94346.log 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92603.log 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51952.log: /var/log/ceph/ceph-client.admin.94346.log: 88.5% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88029.log.gz 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51952.log.gz 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94346.log.gz 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83046.log 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93438.log 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92603.log: /var/log/ceph/ceph-client.admin.83046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83760.log 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93438.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92603.log.gz 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83046.log.gz 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90425.log 2024-04-25T10:19:55.362 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93438.log.gz 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83760.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83760.log.gz 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56951.log 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79849.log 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90425.log: /var/log/ceph/ceph-client.0.56951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81300.log 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79849.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90425.log.gz 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87401.log 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81300.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.79849.log.gz 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81300.log.gz 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52168.log 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.56951.log.gz 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87297.log 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87401.log: /var/log/ceph/ceph-client.admin.52168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54664.log 2024-04-25T10:19:55.363 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87297.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87401.log.gz 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52168.log.gz 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92284.log 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87297.log.gz 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93028.log 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54664.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.92284.log: -- /var/log/ceph/ceph-client.admin.37601.log 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93028.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92284.log.gz 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93028.log.gz 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.54664.log.gz 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84577.log 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33823.log 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37601.log: /var/log/ceph/ceph-client.admin.84577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34919.log 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33823.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37601.log.gz 2024-04-25T10:19:55.364 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.84577.log.gz 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 88.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.33823.log.gz /var/log/ceph/ceph-client.admin.77195.log 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57230.log 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34919.log: /var/log/ceph/ceph-client.admin.77195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56468.log 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57230.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34919.log.gz 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77195.log.gz 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57230.log.gz 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74606.log 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79442.log 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81196.log 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74606.log: /var/log/ceph/ceph-client.admin.79442.log: 88.6% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56468.log.gz 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74606.log.gz 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34314.log 2024-04-25T10:19:55.365 INFO:teuthology.orchestra.run.smithi133.stderr: 90.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76299.log 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79442.log.gz 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81196.log: /var/log/ceph/ceph-client.admin.34314.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81196.log.gz 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34314.log.gz 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88055.log 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90793.log 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76299.log: /var/log/ceph/ceph-client.admin.88055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92152.log 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90793.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76299.log.gz 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88055.log.gz 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77169.log 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90793.log.gz 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77850.log 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77169.log: /var/log/ceph/ceph-client.admin.92152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92180.log 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77850.log: 89.0% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92152.log.gz 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77169.log.gz 2024-04-25T10:19:55.366 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.77850.log.gz -5 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94190.log 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73448.log 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92180.log: /var/log/ceph/ceph-client.admin.94190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36279.log 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73448.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94190.log.gz 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92180.log.gz 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.73448.log.gz 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36884.log 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71464.log 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36279.log: /var/log/ceph/ceph-client.admin.36884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87766.log 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71464.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36279.log.gz 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36884.log.gz 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90871.log 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71464.log.gz 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77715.log 2024-04-25T10:19:55.367 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87766.log: /var/log/ceph/ceph-client.admin.90871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53616.log 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77715.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.87766.log.gz 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% 91.3% -- replaced with /var/log/ceph/ceph-client.admin.90871.log.gz 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77715.log.gz 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35099.log 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57641.log 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53616.log: gzip/var/log/ceph/ceph-client.admin.35099.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.86028.log 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57641.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35099.log.gz 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57641.log.gz 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53616.log.gz 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76403.log 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38103.log 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91244.log 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.86028.log.gz 88.8%/var/log/ceph/ceph-client.admin.38103.log: -- replaced with /var/log/ceph/ceph-client.admin.76403.log.gz 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38103.log.gz 2024-04-25T10:19:55.368 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56091.log 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91244.log: gzip/var/log/ceph/ceph-mon.a.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.87189.log 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91244.log.gz 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56091.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56091.log.gz 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71937.log 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52116.log 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87189.log.gz 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71937.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71937.log.gz 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92938.log 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71432.log 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52116.log: /var/log/ceph/ceph-client.admin.92938.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52116.log.gz 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.92938.log.gz 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74658.log 2024-04-25T10:19:55.369 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71432.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71432.log.gz 2024-04-25T10:19:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38337.log 2024-04-25T10:19:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35907.log 2024-04-25T10:19:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74658.log: /var/log/ceph/ceph-client.admin.38337.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74658.log.gz 2024-04-25T10:19:55.375 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38337.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57008.log 2024-04-25T10:19:55.375 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35907.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.35907.log.gz 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83909.log 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56222.log 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57008.log: /var/log/ceph/ceph-client.admin.83909.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83909.log.gz 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57008.log.gz 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35699.log 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87531.log 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.56222.log: /var/log/ceph/ceph-client.admin.35699.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35699.log.gz 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.0.56222.log.gz 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80634.log 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83961.log 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87531.log: /var/log/ceph/ceph-client.admin.80634.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80634.log.gz 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.87531.log.gz 2024-04-25T10:19:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90819.log 2024-04-25T10:19:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56820.log 2024-04-25T10:19:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83961.log: /var/log/ceph/ceph-client.admin.90819.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83961.log.gz 2024-04-25T10:19:55.377 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90819.log.gz 2024-04-25T10:19:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86902.log 2024-04-25T10:19:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89319.log 2024-04-25T10:19:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56820.log: /var/log/ceph/ceph-client.admin.86902.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56820.log.gz 2024-04-25T10:19:55.377 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86902.log.gz 2024-04-25T10:19:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70791.log 2024-04-25T10:19:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35855.log 2024-04-25T10:19:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.89319.log: /var/log/ceph/ceph-client.admin.70791.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70791.log.gz 2024-04-25T10:19:55.379 INFO:teuthology.orchestra.run.smithi133.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.89319.log.gz 2024-04-25T10:19:55.379 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52246.log 2024-04-25T10:19:55.380 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53141.log 2024-04-25T10:19:55.380 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35855.log: /var/log/ceph/ceph-client.admin.52246.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35855.log.gz 2024-04-25T10:19:55.380 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52246.log.gz 2024-04-25T10:19:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82893.log 2024-04-25T10:19:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76791.log 2024-04-25T10:19:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53141.log: /var/log/ceph/ceph-client.admin.82893.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53141.log.gz 2024-04-25T10:19:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90767.log 2024-04-25T10:19:55.382 INFO:teuthology.orchestra.run.smithi133.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.82893.log.gz 2024-04-25T10:19:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76791.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76791.log.gz 2024-04-25T10:19:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94424.log 2024-04-25T10:19:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84358.log 2024-04-25T10:19:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90767.log: /var/log/ceph/ceph-client.admin.94424.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90767.log.gz 2024-04-25T10:19:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83098.log 2024-04-25T10:19:55.385 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94424.log.gz 2024-04-25T10:19:55.385 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84358.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84358.log.gz 2024-04-25T10:19:55.385 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53089.log 2024-04-25T10:19:55.386 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55224.log 2024-04-25T10:19:55.386 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83098.log: /var/log/ceph/ceph-client.admin.53089.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.83098.log.gz 2024-04-25T10:19:55.387 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53089.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53860.log 2024-04-25T10:19:55.387 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.387 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55224.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55224.log.gz 2024-04-25T10:19:55.387 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33667.log 2024-04-25T10:19:55.388 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34903.log 2024-04-25T10:19:55.389 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53860.log: /var/log/ceph/ceph-client.admin.33667.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53860.log.gz 2024-04-25T10:19:55.389 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33667.log.gz 2024-04-25T10:19:55.389 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52449.log 2024-04-25T10:19:55.390 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80790.log 2024-04-25T10:19:55.390 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34903.log: /var/log/ceph/ceph-client.admin.52449.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34903.log.gz 2024-04-25T10:19:55.390 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52449.log.gz 2024-04-25T10:19:55.390 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34553.log 2024-04-25T10:19:55.391 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76739.log 2024-04-25T10:19:55.392 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80790.log: /var/log/ceph/ceph-client.admin.34553.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80790.log.gz 2024-04-25T10:19:55.392 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34553.log.gz 2024-04-25T10:19:55.392 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37232.log 2024-04-25T10:19:55.392 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71608.log 2024-04-25T10:19:55.393 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76739.log: /var/log/ceph/ceph-client.admin.37232.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37232.log.gz 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76739.log.gz 2024-04-25T10:19:55.393 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:19:55.393 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57752.log 2024-04-25T10:19:55.394 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90976.log 2024-04-25T10:19:55.394 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71608.log: /var/log/ceph/ceph-client.admin.57752.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71608.log.gz 2024-04-25T10:19:55.395 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57752.log.gz 2024-04-25T10:19:55.395 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83987.log 2024-04-25T10:19:55.401 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.90976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55987.log 2024-04-25T10:19:55.402 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83987.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83987.log.gz 2024-04-25T10:19:55.402 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80053.log 2024-04-25T10:19:55.403 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55987.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55987.log.gz 2024-04-25T10:19:55.403 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70709.log 2024-04-25T10:19:55.404 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80053.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.80053.log.gz 2024-04-25T10:19:55.405 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34703.log 2024-04-25T10:19:55.406 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70709.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.70709.log.gz 2024-04-25T10:19:55.407 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35122.log 2024-04-25T10:19:55.408 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34703.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34703.log.gz 2024-04-25T10:19:55.408 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87349.log 2024-04-25T10:19:55.409 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35122.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35122.log.gz 2024-04-25T10:19:55.409 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51898.log 2024-04-25T10:19:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87349.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.87349.log.gz 2024-04-25T10:19:55.410 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73854.log 2024-04-25T10:19:55.411 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51898.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51898.log.gz 2024-04-25T10:19:55.412 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86418.log 2024-04-25T10:19:55.413 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73854.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73854.log.gz 2024-04-25T10:19:55.413 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58401.log 2024-04-25T10:19:55.414 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86418.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86418.log.gz 2024-04-25T10:19:55.415 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33263.log 2024-04-25T10:19:55.429 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.58401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57849.log 2024-04-25T10:19:55.430 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33263.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33263.log.gz 2024-04-25T10:19:55.437 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84224.log 2024-04-25T10:19:55.438 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57849.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57849.log.gz 2024-04-25T10:19:55.445 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92629.log 2024-04-25T10:19:55.446 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84224.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84224.log.gz 2024-04-25T10:19:55.453 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37393.log 2024-04-25T10:19:55.453 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92629.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.92629.log.gz 2024-04-25T10:19:55.458 INFO:teuthology.orchestra.run.smithi133.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:19:55.458 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90611.log 2024-04-25T10:19:55.459 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37393.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37393.log.gz 2024-04-25T10:19:55.460 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77559.log 2024-04-25T10:19:55.461 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90611.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.90611.log.gz 2024-04-25T10:19:55.461 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37890.log 2024-04-25T10:19:55.462 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77559.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77559.log.gz 2024-04-25T10:19:55.462 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55373.log 2024-04-25T10:19:55.463 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37890.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37890.log.gz 2024-04-25T10:19:55.464 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84039.log 2024-04-25T10:19:55.465 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55373.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55373.log.gz 2024-04-25T10:19:55.465 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90845.log 2024-04-25T10:19:55.466 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84039.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84039.log.gz 2024-04-25T10:19:55.466 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86928.log 2024-04-25T10:19:55.467 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90845.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90845.log.gz 2024-04-25T10:19:55.468 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86280.log 2024-04-25T10:19:55.469 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86928.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86928.log.gz 2024-04-25T10:19:55.469 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83355.log 2024-04-25T10:19:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86280.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86280.log.gz 2024-04-25T10:19:55.470 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73828.log 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83355.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83355.log.gz 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94034.log 2024-04-25T10:19:55.473 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73828.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73828.log.gz 2024-04-25T10:19:55.473 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39000.log 2024-04-25T10:19:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94034.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94034.log.gz 2024-04-25T10:19:55.474 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94112.log 2024-04-25T10:19:55.475 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39000.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39000.log.gz 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77507.log 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94112.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94112.log.gz 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83206.log 2024-04-25T10:19:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77507.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77507.log.gz 2024-04-25T10:19:55.478 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76273.log 2024-04-25T10:19:55.479 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83206.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83206.log.gz 2024-04-25T10:19:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86575.log 2024-04-25T10:19:55.480 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76273.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76273.log.gz 2024-04-25T10:19:55.481 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80849.log 2024-04-25T10:19:55.482 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86575.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86575.log.gz 2024-04-25T10:19:55.482 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88585.log 2024-04-25T10:19:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80849.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80849.log.gz 2024-04-25T10:19:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77822.log 2024-04-25T10:19:55.484 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88585.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88585.log.gz 2024-04-25T10:19:55.485 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37125.log 2024-04-25T10:19:55.486 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77822.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77822.log.gz 2024-04-25T10:19:55.486 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89214.log 2024-04-25T10:19:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37125.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37125.log.gz 2024-04-25T10:19:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55529.log 2024-04-25T10:19:55.489 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89214.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89214.log.gz 2024-04-25T10:19:55.489 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91592.log 2024-04-25T10:19:55.490 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55529.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55529.log.gz 2024-04-25T10:19:55.490 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77273.log 2024-04-25T10:19:55.491 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91592.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91592.log.gz 2024-04-25T10:19:55.492 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91512.log 2024-04-25T10:19:55.493 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77273.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77273.log.gz 2024-04-25T10:19:55.493 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54375.log 2024-04-25T10:19:55.494 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91512.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91512.log.gz 2024-04-25T10:19:55.494 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57386.log 2024-04-25T10:19:55.496 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54375.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54375.log.gz 2024-04-25T10:19:55.496 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37471.log 2024-04-25T10:19:55.497 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57386.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57386.log.gz 2024-04-25T10:19:55.497 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57535.log 2024-04-25T10:19:55.498 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37471.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37471.log.gz 2024-04-25T10:19:55.499 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37336.log 2024-04-25T10:19:55.500 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57535.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57535.log.gz 2024-04-25T10:19:55.500 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93234.log 2024-04-25T10:19:55.501 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37336.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37336.log.gz 2024-04-25T10:19:55.501 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53935.log 2024-04-25T10:19:55.502 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93234.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.93234.log.gz 2024-04-25T10:19:55.503 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52618.log 2024-04-25T10:19:55.504 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53935.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.53935.log.gz 2024-04-25T10:19:55.505 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91839.log 2024-04-25T10:19:55.505 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52618.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52618.log.gz 2024-04-25T10:19:55.506 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88507.log 2024-04-25T10:19:55.507 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91839.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91839.log.gz 2024-04-25T10:19:55.507 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81107.log 2024-04-25T10:19:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88507.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88507.log.gz 2024-04-25T10:19:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53506.log 2024-04-25T10:19:55.509 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81107.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81107.log.gz 2024-04-25T10:19:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77481.log 2024-04-25T10:19:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53506.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53506.log.gz 2024-04-25T10:19:55.511 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83518.log 2024-04-25T10:19:55.512 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77481.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77481.log.gz 2024-04-25T10:19:55.512 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86653.log 2024-04-25T10:19:55.513 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83518.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83518.log.gz 2024-04-25T10:19:55.513 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90451.log 2024-04-25T10:19:55.514 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86653.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86653.log.gz 2024-04-25T10:19:55.515 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56616.log 2024-04-25T10:19:55.516 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90451.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90451.log.gz 2024-04-25T10:19:55.516 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77065.log 2024-04-25T10:19:55.517 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56616.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56616.log.gz 2024-04-25T10:19:55.517 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89006.log 2024-04-25T10:19:55.519 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77065.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77065.log.gz 2024-04-25T10:19:55.519 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71215.log 2024-04-25T10:19:55.520 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89006.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89006.log.gz 2024-04-25T10:19:55.520 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88377.log 2024-04-25T10:19:55.521 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71215.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71215.log.gz 2024-04-25T10:19:55.521 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81168.log 2024-04-25T10:19:55.522 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88377.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88377.log.gz 2024-04-25T10:19:55.522 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87479.log 2024-04-25T10:19:55.523 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81168.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81168.log.gz 2024-04-25T10:19:55.524 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90003.log 2024-04-25T10:19:55.525 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87479.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87479.log.gz 2024-04-25T10:19:55.525 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33797.log 2024-04-25T10:19:55.526 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90003.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90003.log.gz 2024-04-25T10:19:55.526 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77117.log 2024-04-25T10:19:55.527 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33797.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33797.log.gz 2024-04-25T10:19:55.527 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78070.log 2024-04-25T10:19:55.528 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77117.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77117.log.gz 2024-04-25T10:19:55.529 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83552.log 2024-04-25T10:19:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.78070.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78070.log.gz 2024-04-25T10:19:55.530 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89550.log 2024-04-25T10:19:55.531 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83552.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83552.log.gz 2024-04-25T10:19:55.531 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89657.log 2024-04-25T10:19:55.533 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89550.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.89550.log.gz 2024-04-25T10:19:55.533 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35647.log 2024-04-25T10:19:55.534 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89657.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89657.log.gz 2024-04-25T10:19:55.534 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34711.log 2024-04-25T10:19:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35647.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35647.log.gz 2024-04-25T10:19:55.535 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80973.log 2024-04-25T10:19:55.536 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34711.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34711.log.gz 2024-04-25T10:19:55.537 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83260.log 2024-04-25T10:19:55.538 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80973.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.80973.log.gz 2024-04-25T10:19:55.538 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73272.log 2024-04-25T10:19:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83260.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83260.log.gz 2024-04-25T10:19:55.539 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71371.log 2024-04-25T10:19:55.541 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73272.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.73272.log.gz 2024-04-25T10:19:55.541 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86549.log 2024-04-25T10:19:55.542 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71371.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71371.log.gz 2024-04-25T10:19:55.542 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93811.log 2024-04-25T10:19:55.543 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86549.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86549.log.gz 2024-04-25T10:19:55.544 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91330.log 2024-04-25T10:19:55.545 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93811.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93811.log.gz 2024-04-25T10:19:55.545 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74502.log 2024-04-25T10:19:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91330.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.91330.log.gz 2024-04-25T10:19:55.546 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56389.log 2024-04-25T10:19:55.547 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74502.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74502.log.gz 2024-04-25T10:19:55.547 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39221.log 2024-04-25T10:19:55.548 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56389.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56389.log.gz 2024-04-25T10:19:55.549 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77143.log 2024-04-25T10:19:55.550 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39221.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39221.log.gz 2024-04-25T10:19:55.550 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35109.log 2024-04-25T10:19:55.551 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77143.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77143.log.gz 2024-04-25T10:19:55.551 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35725.log 2024-04-25T10:19:55.552 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35109.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35109.log.gz 2024-04-25T10:19:55.552 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55855.log 2024-04-25T10:19:55.553 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35725.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35725.log.gz 2024-04-25T10:19:55.554 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84629.log 2024-04-25T10:19:55.555 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55855.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55855.log.gz 2024-04-25T10:19:55.555 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73534.log 2024-04-25T10:19:55.556 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84629.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.84629.log.gz 2024-04-25T10:19:55.556 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76325.log 2024-04-25T10:19:55.557 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73534.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73534.log.gz 2024-04-25T10:19:55.557 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38571.log 2024-04-25T10:19:55.558 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76325.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76325.log.gz 2024-04-25T10:19:55.559 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52142.log 2024-04-25T10:19:55.559 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38571.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38571.log.gz 2024-04-25T10:19:55.560 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52220.log 2024-04-25T10:19:55.561 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52142.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52142.log.gz 2024-04-25T10:19:55.561 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51736.log 2024-04-25T10:19:55.562 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52220.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52220.log.gz 2024-04-25T10:19:55.562 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36936.log 2024-04-25T10:19:55.564 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51736.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51736.log.gz 2024-04-25T10:19:55.564 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:19:55.565 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36936.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36936.log.gz 2024-04-25T10:19:55.565 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38233.log 2024-04-25T10:19:55.581 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36138.log 2024-04-25T10:19:55.582 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38233.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38233.log.gz 2024-04-25T10:19:55.597 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51926.log 2024-04-25T10:19:55.598 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36138.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36138.log.gz 2024-04-25T10:19:55.605 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34907.log 2024-04-25T10:19:55.605 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51926.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51926.log.gz 2024-04-25T10:19:55.613 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38843.log 2024-04-25T10:19:55.613 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34907.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34907.log.gz 2024-04-25T10:19:55.621 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37419.log 2024-04-25T10:19:55.621 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38843.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38843.log.gz 2024-04-25T10:19:55.637 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56039.log 2024-04-25T10:19:55.637 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37419.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37419.log.gz 2024-04-25T10:19:55.643 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71634.log 2024-04-25T10:19:55.644 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56039.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56039.log.gz 2024-04-25T10:19:55.653 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81404.log 2024-04-25T10:19:55.654 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71634.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71634.log.gz 2024-04-25T10:19:55.654 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83124.log 2024-04-25T10:19:55.655 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81404.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81404.log.gz 2024-04-25T10:19:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87085.log 2024-04-25T10:19:55.673 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34696.log 2024-04-25T10:19:55.673 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87085.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87085.log.gz 2024-04-25T10:19:55.677 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83124.log.gz 2024-04-25T10:19:55.681 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35123.log 2024-04-25T10:19:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34696.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34696.log.gz 2024-04-25T10:19:55.693 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84151.log 2024-04-25T10:19:55.694 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35123.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35123.log.gz 2024-04-25T10:19:55.705 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52369.log 2024-04-25T10:19:55.706 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84151.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.84151.log.gz 2024-04-25T10:19:55.717 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39303.log 2024-04-25T10:19:55.718 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52369.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52369.log.gz 2024-04-25T10:19:55.729 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91304.log 2024-04-25T10:19:55.730 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39303.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39303.log.gz 2024-04-25T10:19:55.741 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86798.log 2024-04-25T10:19:55.742 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91304.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91304.log.gz 2024-04-25T10:19:55.753 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94580.log 2024-04-25T10:19:55.754 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86798.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86798.log.gz 2024-04-25T10:19:55.759 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.4.log: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:19:55.765 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91084.log 2024-04-25T10:19:55.766 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94580.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94580.log.gz 2024-04-25T10:19:55.777 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55451.log 2024-04-25T10:19:55.778 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91084.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.91084.log.gz 2024-04-25T10:19:55.789 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72039.log 2024-04-25T10:19:55.790 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55451.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55451.log.gz 2024-04-25T10:19:55.801 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92362.log 2024-04-25T10:19:55.802 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.72039.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72039.log.gz 2024-04-25T10:19:55.813 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84603.log 2024-04-25T10:19:55.814 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92362.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.92362.log.gz 2024-04-25T10:19:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82993.log 2024-04-25T10:19:55.826 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84603.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.84603.log.gz 2024-04-25T10:19:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57901.log 2024-04-25T10:19:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.82993.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.82993.log.gz 2024-04-25T10:19:55.849 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38869.log 2024-04-25T10:19:55.850 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57901.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57901.log.gz 2024-04-25T10:19:55.853 INFO:teuthology.orchestra.run.smithi162.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:19:55.861 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57282.log 2024-04-25T10:19:55.862 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38869.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38869.log.gz 2024-04-25T10:19:55.873 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86332.log 2024-04-25T10:19:55.874 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57282.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57282.log.gz 2024-04-25T10:19:55.885 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77689.log 2024-04-25T10:19:55.886 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86332.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86332.log.gz 2024-04-25T10:19:55.897 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54902.log 2024-04-25T10:19:55.898 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77689.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77689.log.gz 2024-04-25T10:19:55.909 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92100.log 2024-04-25T10:19:55.910 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54902.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54902.log.gz 2024-04-25T10:19:55.925 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71833.log 2024-04-25T10:19:55.926 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92100.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.92100.log.gz 2024-04-25T10:19:55.927 INFO:teuthology.orchestra.run.smithi133.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.90976.log.gz 2024-04-25T10:19:55.929 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74554.log 2024-04-25T10:19:55.930 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71833.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71833.log.gz 2024-04-25T10:19:55.930 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53296.log 2024-04-25T10:19:55.931 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74554.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74554.log.gz 2024-04-25T10:19:55.931 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88533.log 2024-04-25T10:19:55.932 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53296.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.53296.log.gz 2024-04-25T10:19:55.933 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76197.log 2024-04-25T10:19:55.934 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88533.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88533.log.gz 2024-04-25T10:19:55.934 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36363.log 2024-04-25T10:19:55.935 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76197.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.76197.log.gz 2024-04-25T10:19:55.935 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91192.log 2024-04-25T10:19:55.936 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36363.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36363.log.gz 2024-04-25T10:19:55.937 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34709.log 2024-04-25T10:19:55.938 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91192.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91192.log.gz 2024-04-25T10:19:55.938 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81248.log 2024-04-25T10:19:55.939 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34709.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34709.log.gz 2024-04-25T10:19:55.939 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38415.log 2024-04-25T10:19:55.940 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81248.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81248.log.gz 2024-04-25T10:19:55.940 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71885.log 2024-04-25T10:19:55.941 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38415.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38415.log.gz 2024-04-25T10:19:55.942 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74166.log 2024-04-25T10:19:55.943 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71885.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71885.log.gz 2024-04-25T10:19:55.943 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74762.log 2024-04-25T10:19:55.944 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74166.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74166.log.gz 2024-04-25T10:19:55.944 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56580.log 2024-04-25T10:19:55.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74762.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74762.log.gz 2024-04-25T10:19:55.946 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39381.log 2024-04-25T10:19:55.946 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56580.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.56580.log.gz 2024-04-25T10:19:55.947 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80105.log 2024-04-25T10:19:55.948 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39381.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39381.log.gz 2024-04-25T10:19:55.948 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37367.log 2024-04-25T10:19:55.949 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80105.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80105.log.gz 2024-04-25T10:19:55.949 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55276.log 2024-04-25T10:19:55.950 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37367.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37367.log.gz 2024-04-25T10:19:55.950 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74133.log 2024-04-25T10:19:55.951 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55276.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55276.log.gz 2024-04-25T10:19:55.952 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78044.log 2024-04-25T10:19:55.953 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74133.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74133.log.gz 2024-04-25T10:19:55.953 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38493.log 2024-04-25T10:19:55.954 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.78044.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78044.log.gz 2024-04-25T10:19:55.954 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52090.log 2024-04-25T10:19:55.955 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38493.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38493.log.gz 2024-04-25T10:19:55.955 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57360.log 2024-04-25T10:19:55.956 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52090.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52090.log.gz 2024-04-25T10:19:55.957 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88928.log 2024-04-25T10:19:55.958 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57360.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57360.log.gz 2024-04-25T10:19:55.958 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88980.log 2024-04-25T10:19:55.959 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88928.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88928.log.gz 2024-04-25T10:19:55.959 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38597.log 2024-04-25T10:19:55.960 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88980.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88980.log.gz 2024-04-25T10:19:55.960 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90585.log 2024-04-25T10:19:55.961 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38597.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38597.log.gz 2024-04-25T10:19:55.962 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34906.log 2024-04-25T10:19:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90585.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90585.log.gz 2024-04-25T10:19:55.963 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53536.log 2024-04-25T10:19:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34906.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34906.log.gz 2024-04-25T10:19:55.964 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55146.log 2024-04-25T10:19:55.965 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53536.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53536.log.gz 2024-04-25T10:19:55.966 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52844.log 2024-04-25T10:19:55.967 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55146.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55146.log.gz 2024-04-25T10:19:55.967 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71137.log 2024-04-25T10:19:55.968 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52844.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52844.log.gz 2024-04-25T10:19:55.968 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83786.log 2024-04-25T10:19:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71137.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.71137.log.gz 2024-04-25T10:19:55.969 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57066.log 2024-04-25T10:19:55.970 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83786.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83786.log.gz 2024-04-25T10:19:55.971 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55562.log 2024-04-25T10:19:55.971 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57066.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57066.log.gz 2024-04-25T10:19:55.972 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87816.log 2024-04-25T10:19:55.973 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55562.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55562.log.gz 2024-04-25T10:19:55.973 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35569.log 2024-04-25T10:19:55.974 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87816.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.87816.log.gz 2024-04-25T10:19:55.975 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91408.log 2024-04-25T10:19:55.975 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35569.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35569.log.gz 2024-04-25T10:19:55.976 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55503.log 2024-04-25T10:19:55.977 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91408.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91408.log.gz 2024-04-25T10:19:55.977 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92708.log 2024-04-25T10:19:55.978 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55503.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55503.log.gz 2024-04-25T10:19:55.978 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35107.log 2024-04-25T10:19:55.979 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92708.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92708.log.gz 2024-04-25T10:19:55.980 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84302.log 2024-04-25T10:19:55.980 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35107.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35107.log.gz 2024-04-25T10:19:55.985 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74263.log 2024-04-25T10:19:55.986 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84302.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84302.log.gz 2024-04-25T10:19:55.986 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94398.log 2024-04-25T10:19:55.987 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74263.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74263.log.gz 2024-04-25T10:19:55.987 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89264.log 2024-04-25T10:19:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94398.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94398.log.gz 2024-04-25T10:19:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89735.log 2024-04-25T10:19:55.989 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89264.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89264.log.gz 2024-04-25T10:19:55.990 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71727.log 2024-04-25T10:19:55.990 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89735.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89735.log.gz 2024-04-25T10:19:55.991 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89447.log 2024-04-25T10:19:55.992 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71727.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.71727.log.gz 2024-04-25T10:19:55.992 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77455.log 2024-04-25T10:19:55.993 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89447.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89447.log.gz 2024-04-25T10:19:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33440.log 2024-04-25T10:19:55.994 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77455.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77455.log.gz 2024-04-25T10:19:55.995 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52985.log 2024-04-25T10:19:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33440.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33440.log.gz 2024-04-25T10:19:55.996 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57308.log 2024-04-25T10:19:55.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52985.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52985.log.gz 2024-04-25T10:19:55.997 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38155.log 2024-04-25T10:19:55.998 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57308.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57308.log.gz 2024-04-25T10:19:55.999 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94060.log 2024-04-25T10:19:55.999 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38155.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.38155.log.gz 2024-04-25T10:19:56.000 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91748.log 2024-04-25T10:19:56.001 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94060.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94060.log.gz 2024-04-25T10:19:56.001 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80165.log 2024-04-25T10:19:56.002 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91748.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91748.log.gz 2024-04-25T10:19:56.002 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38668.log 2024-04-25T10:19:56.003 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80165.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80165.log.gz 2024-04-25T10:19:56.003 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:19:56.004 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38668.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38668.log.gz 2024-04-25T10:19:56.005 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77741.log 2024-04-25T10:19:56.013 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36780.log 2024-04-25T10:19:56.014 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77741.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77741.log.gz 2024-04-25T10:19:56.029 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93629.log 2024-04-25T10:19:56.030 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36780.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36780.log.gz 2024-04-25T10:19:56.030 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38974.log 2024-04-25T10:19:56.031 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93629.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93629.log.gz 2024-04-25T10:19:56.049 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37497.log 2024-04-25T10:19:56.050 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38974.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38974.log.gz 2024-04-25T10:19:56.057 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83286.log 2024-04-25T10:19:56.057 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37497.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37497.log.gz 2024-04-25T10:19:56.065 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84473.log 2024-04-25T10:19:56.065 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83286.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83286.log.gz 2024-04-25T10:19:56.072 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86225.log 2024-04-25T10:19:56.081 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84473.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86627.log 2024-04-25T10:19:56.082 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86225.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.86225.log.gz 2024-04-25T10:19:56.089 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84473.log.gz 2024-04-25T10:19:56.089 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55250.log 2024-04-25T10:19:56.089 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86627.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86627.log.gz 2024-04-25T10:19:56.097 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39052.log 2024-04-25T10:19:56.097 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55250.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55250.log.gz 2024-04-25T10:19:56.105 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83150.log 2024-04-25T10:19:56.105 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39052.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39052.log.gz 2024-04-25T10:19:56.113 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73698.log 2024-04-25T10:19:56.113 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83150.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83150.log.gz 2024-04-25T10:19:56.133 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79747.log 2024-04-25T10:19:56.133 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73698.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73698.log.gz 2024-04-25T10:19:56.153 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36305.log 2024-04-25T10:19:56.154 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79747.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.79747.log.gz 2024-04-25T10:19:56.173 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37864.log 2024-04-25T10:19:56.173 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36305.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36305.log.gz 2024-04-25T10:19:56.185 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70981.log 2024-04-25T10:19:56.185 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37864.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37864.log.gz 2024-04-25T10:19:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77351.log 2024-04-25T10:19:56.197 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70981.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70981.log.gz 2024-04-25T10:19:56.217 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74450.log 2024-04-25T10:19:56.217 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77351.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77351.log.gz 2024-04-25T10:19:56.225 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71241.log 2024-04-25T10:19:56.225 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74450.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74450.log.gz 2024-04-25T10:19:56.233 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83656.log 2024-04-25T10:19:56.233 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71241.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71241.log.gz 2024-04-25T10:19:56.241 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33849.log 2024-04-25T10:19:56.241 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83656.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83656.log.gz 2024-04-25T10:19:56.249 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37653.log 2024-04-25T10:19:56.250 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33849.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.33849.log.gz 2024-04-25T10:19:56.257 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79899.log 2024-04-25T10:19:56.257 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37653.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37653.log.gz 2024-04-25T10:19:56.265 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78122.log 2024-04-25T10:19:56.266 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79899.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.79899.log.gz 2024-04-25T10:19:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57563.log 2024-04-25T10:19:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.78122.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78122.log.gz 2024-04-25T10:19:56.281 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36702.log 2024-04-25T10:19:56.281 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57563.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57563.log.gz 2024-04-25T10:19:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36806.log 2024-04-25T10:19:56.301 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36702.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36702.log.gz 2024-04-25T10:19:56.309 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57334.log 2024-04-25T10:19:56.309 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36806.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36806.log.gz 2024-04-25T10:19:56.319 INFO:teuthology.orchestra.run.smithi133.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.78255.log.gz 2024-04-25T10:19:56.320 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76927.log 2024-04-25T10:19:56.320 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57334.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57334.log.gz 2024-04-25T10:19:56.321 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86523.log 2024-04-25T10:19:56.322 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76927.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76927.log.gz 2024-04-25T10:19:56.322 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35298.log 2024-04-25T10:19:56.322 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86523.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86523.log.gz 2024-04-25T10:19:56.323 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73977.log 2024-04-25T10:19:56.323 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35298.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35298.log.gz 2024-04-25T10:19:56.324 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55961.log 2024-04-25T10:19:56.325 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73977.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73977.log.gz 2024-04-25T10:19:56.325 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93056.log 2024-04-25T10:19:56.326 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55961.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55961.log.gz 2024-04-25T10:19:56.326 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37785.log 2024-04-25T10:19:56.327 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93056.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93056.log.gz 2024-04-25T10:19:56.327 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83682.log 2024-04-25T10:19:56.328 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37785.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37785.log.gz 2024-04-25T10:19:56.328 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80660.log 2024-04-25T10:19:56.329 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83682.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83682.log.gz 2024-04-25T10:19:56.330 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80511.log 2024-04-25T10:19:56.330 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80660.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80660.log.gz 2024-04-25T10:19:56.331 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91722.log 2024-04-25T10:19:56.331 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80511.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80511.log.gz 2024-04-25T10:19:56.332 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70929.log 2024-04-25T10:19:56.332 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91722.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91722.log.gz 2024-04-25T10:19:56.333 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35111.log 2024-04-25T10:19:56.333 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70929.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70929.log.gz 2024-04-25T10:19:56.334 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36650.log 2024-04-25T10:19:56.335 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35111.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35111.log.gz 2024-04-25T10:19:56.335 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53115.log 2024-04-25T10:19:56.336 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36650.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36650.log.gz 2024-04-25T10:19:56.336 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55172.log 2024-04-25T10:19:56.337 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53115.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53115.log.gz 2024-04-25T10:19:56.337 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77403.log 2024-04-25T10:19:56.338 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55172.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55172.log.gz 2024-04-25T10:19:56.338 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86876.log 2024-04-25T10:19:56.339 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77403.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77403.log.gz 2024-04-25T10:19:56.340 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38896.log 2024-04-25T10:19:56.340 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86876.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86876.log.gz 2024-04-25T10:19:56.341 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94556.log 2024-04-25T10:19:56.342 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38896.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38896.log.gz 2024-04-25T10:19:56.342 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55042.log 2024-04-25T10:19:56.343 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94556.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94556.log.gz 2024-04-25T10:19:56.343 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76817.log 2024-04-25T10:19:56.344 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55042.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55042.log.gz 2024-04-25T10:19:56.344 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57256.log 2024-04-25T10:19:56.345 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76817.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76817.log.gz 2024-04-25T10:19:56.345 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94216.log 2024-04-25T10:19:56.346 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57256.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57256.log.gz 2024-04-25T10:19:56.347 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38817.log 2024-04-25T10:19:56.347 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94216.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94216.log.gz 2024-04-25T10:19:56.348 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93785.log 2024-04-25T10:19:56.348 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38817.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38817.log.gz 2024-04-25T10:19:56.349 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92577.log 2024-04-25T10:19:56.350 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93785.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93785.log.gz 2024-04-25T10:19:56.350 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90503.log 2024-04-25T10:19:56.351 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92577.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.92577.log.gz 2024-04-25T10:19:56.351 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76581.log 2024-04-25T10:19:56.352 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90503.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.90503.log.gz 2024-04-25T10:19:56.352 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35325.log 2024-04-25T10:19:56.353 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76581.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76581.log.gz 2024-04-25T10:19:56.353 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89084.log 2024-04-25T10:19:56.354 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35325.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35325.log.gz 2024-04-25T10:19:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53351.log 2024-04-25T10:19:56.355 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89084.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89084.log.gz 2024-04-25T10:19:56.356 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52038.log 2024-04-25T10:19:56.377 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.53351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54930.log 2024-04-25T10:19:56.377 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52038.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52038.log.gz 2024-04-25T10:19:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34262.log 2024-04-25T10:19:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54930.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54930.log.gz 2024-04-25T10:19:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89162.log 2024-04-25T10:19:56.398 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34262.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34262.log.gz 2024-04-25T10:19:56.405 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36229.log 2024-04-25T10:19:56.405 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89162.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89162.log.gz 2024-04-25T10:19:56.413 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27599.log 2024-04-25T10:19:56.414 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.2.36229.log: 91.5% -- replaced with /var/log/ceph/ceph-client.2.36229.log.gz 2024-04-25T10:19:56.421 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36088.log 2024-04-25T10:19:56.421 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.27599.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.27599.log.gz 2024-04-25T10:19:56.437 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87740.log 2024-04-25T10:19:56.438 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.36088.log: 90.8% -- replaced with /var/log/ceph/ceph-client.0.36088.log.gz 2024-04-25T10:19:56.449 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33296.log 2024-04-25T10:19:56.449 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87740.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87740.log.gz 2024-04-25T10:19:56.457 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35621.log 2024-04-25T10:19:56.457 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33296.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33296.log.gz 2024-04-25T10:19:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70869.log 2024-04-25T10:19:56.465 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35621.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35621.log.gz 2024-04-25T10:19:56.473 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55712.log 2024-04-25T10:19:56.473 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70869.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70869.log.gz 2024-04-25T10:19:56.481 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80269.log 2024-04-25T10:19:56.481 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55712.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55712.log.gz 2024-04-25T10:19:56.493 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34236.log 2024-04-25T10:19:56.493 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80269.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.80269.log.gz 2024-04-25T10:19:56.501 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77885.log 2024-04-25T10:19:56.501 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34236.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34236.log.gz 2024-04-25T10:19:56.509 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87111.log 2024-04-25T10:19:56.509 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77885.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77885.log.gz 2024-04-25T10:19:56.517 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36441.log 2024-04-25T10:19:56.517 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87111.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87111.log.gz 2024-04-25T10:19:56.525 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87453.log 2024-04-25T10:19:56.525 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36441.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36441.log.gz 2024-04-25T10:19:56.533 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78150.log 2024-04-25T10:19:56.533 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87453.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87453.log.gz 2024-04-25T10:19:56.541 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89188.log 2024-04-25T10:19:56.542 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.78150.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78150.log.gz 2024-04-25T10:19:56.549 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91957.log 2024-04-25T10:19:56.549 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89188.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89188.log.gz 2024-04-25T10:19:56.557 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88273.log 2024-04-25T10:19:56.557 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91957.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91957.log.gz 2024-04-25T10:19:56.565 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74528.log 2024-04-25T10:19:56.565 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88273.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88273.log.gz 2024-04-25T10:19:56.581 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56167.log 2024-04-25T10:19:56.581 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74528.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74528.log.gz 2024-04-25T10:19:56.589 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93002.log 2024-04-25T10:19:56.589 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56167.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56167.log.gz 2024-04-25T10:19:56.597 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57438.log 2024-04-25T10:19:56.597 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93002.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93002.log.gz 2024-04-25T10:19:56.605 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52272.log 2024-04-25T10:19:56.605 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57438.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57438.log.gz 2024-04-25T10:19:56.613 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57144.log 2024-04-25T10:19:56.613 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52272.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52272.log.gz 2024-04-25T10:19:56.621 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52064.log 2024-04-25T10:19:56.621 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57144.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57144.log.gz 2024-04-25T10:19:56.629 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34705.log 2024-04-25T10:19:56.629 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52064.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52064.log.gz 2024-04-25T10:19:56.645 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76505.log 2024-04-25T10:19:56.645 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34705.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34705.log.gz 2024-04-25T10:19:56.653 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92820.log 2024-04-25T10:19:56.654 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76505.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.76505.log.gz 2024-04-25T10:19:56.661 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90557.log 2024-04-25T10:19:56.661 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92820.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92820.log.gz 2024-04-25T10:19:56.677 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88670.log 2024-04-25T10:19:56.677 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90557.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90557.log.gz 2024-04-25T10:19:56.685 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73508.log 2024-04-25T10:19:56.685 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88670.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88670.log.gz 2024-04-25T10:19:56.693 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36415.log 2024-04-25T10:19:56.694 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73508.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73508.log.gz 2024-04-25T10:19:56.694 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71807.log 2024-04-25T10:19:56.695 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36415.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36415.log.gz 2024-04-25T10:19:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34079.log 2024-04-25T10:19:56.713 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88247.log 2024-04-25T10:19:56.713 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34079.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34079.log.gz 2024-04-25T10:19:56.717 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71807.log.gz 2024-04-25T10:19:56.721 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70843.log 2024-04-25T10:19:56.722 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88247.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88247.log.gz 2024-04-25T10:19:56.733 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73422.log 2024-04-25T10:19:56.734 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70843.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70843.log.gz 2024-04-25T10:19:56.745 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80686.log 2024-04-25T10:19:56.746 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73422.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73422.log.gz 2024-04-25T10:19:56.757 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37047.log 2024-04-25T10:19:56.758 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80686.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.80686.log.gz 2024-04-25T10:19:56.773 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37177.log 2024-04-25T10:19:56.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37047.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37047.log.gz 2024-04-25T10:19:56.785 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:19:56.786 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37177.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37177.log.gz 2024-04-25T10:19:56.801 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87971.log 2024-04-25T10:19:56.805 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.log: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:19:56.817 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93960.log 2024-04-25T10:19:56.818 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87971.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87971.log.gz 2024-04-25T10:19:56.833 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36545.log 2024-04-25T10:19:56.834 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93960.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93960.log.gz 2024-04-25T10:19:56.849 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80563.log 2024-04-25T10:19:56.850 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36545.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36545.log.gz 2024-04-25T10:19:56.865 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53985.log 2024-04-25T10:19:56.866 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80563.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80563.log.gz 2024-04-25T10:19:56.873 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90689.log 2024-04-25T10:19:56.874 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53985.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53985.log.gz 2024-04-25T10:19:56.875 INFO:teuthology.orchestra.run.smithi133.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:19:56.877 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87323.log 2024-04-25T10:19:56.878 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90689.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90689.log.gz 2024-04-25T10:19:56.878 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52298.log 2024-04-25T10:19:56.879 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87323.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87323.log.gz 2024-04-25T10:19:56.879 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51792.log 2024-04-25T10:19:56.880 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52298.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52298.log.gz 2024-04-25T10:19:56.881 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84499.log 2024-04-25T10:19:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51792.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51792.log.gz 2024-04-25T10:19:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76869.log 2024-04-25T10:19:56.883 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84499.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.84499.log.gz 2024-04-25T10:19:56.884 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83492.log 2024-04-25T10:19:56.885 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76869.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76869.log.gz 2024-04-25T10:19:56.885 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86824.log 2024-04-25T10:19:56.886 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83492.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83492.log.gz 2024-04-25T10:19:56.886 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70955.log 2024-04-25T10:19:56.887 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86824.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86824.log.gz 2024-04-25T10:19:56.887 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88900.log 2024-04-25T10:19:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70955.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70955.log.gz 2024-04-25T10:19:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38545.log 2024-04-25T10:19:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88900.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88900.log.gz 2024-04-25T10:19:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82843.log 2024-04-25T10:19:56.891 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38545.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38545.log.gz 2024-04-25T10:19:56.891 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86127.log 2024-04-25T10:19:56.893 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.82843.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.82843.log.gz 2024-04-25T10:19:56.893 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35465.log 2024-04-25T10:19:56.894 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86127.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.86127.log.gz 2024-04-25T10:19:56.895 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71293.log 2024-04-25T10:19:56.896 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35465.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35465.log.gz 2024-04-25T10:19:56.896 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71267.log 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71293.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71293.log.gz 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52762.log 2024-04-25T10:19:56.898 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71267.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71267.log.gz 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39026.log 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52762.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52762.log.gz 2024-04-25T10:19:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55907.log 2024-04-25T10:19:56.901 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39026.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39026.log.gz 2024-04-25T10:19:56.901 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34131.log 2024-04-25T10:19:56.902 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55907.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55907.log.gz 2024-04-25T10:19:56.902 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73620.log 2024-04-25T10:19:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34131.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34131.log.gz 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80589.log 2024-04-25T10:19:56.905 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73620.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73620.log.gz 2024-04-25T10:19:56.905 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92009.log 2024-04-25T10:19:56.906 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80589.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.80589.log.gz 2024-04-25T10:19:56.906 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83234.log 2024-04-25T10:19:56.907 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92009.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92009.log.gz 2024-04-25T10:19:56.907 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85963.log 2024-04-25T10:19:56.908 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83234.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83234.log.gz 2024-04-25T10:19:56.909 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89873.log 2024-04-25T10:19:56.910 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.85963.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85963.log.gz 2024-04-25T10:19:56.910 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70659.log 2024-04-25T10:19:56.911 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89873.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89873.log.gz 2024-04-25T10:19:56.911 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34340.log 2024-04-25T10:19:56.913 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70659.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.70659.log.gz 2024-04-25T10:19:56.913 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92467.log 2024-04-25T10:19:56.914 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34340.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34340.log.gz 2024-04-25T10:19:56.914 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33719.log 2024-04-25T10:19:56.923 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.92467.log: 94.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84705.log 2024-04-25T10:19:56.924 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33719.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33719.log.gz 2024-04-25T10:19:56.924 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.0.92467.log.gz 2024-04-25T10:19:56.925 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56520.log 2024-04-25T10:19:56.926 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84705.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.84705.log.gz 2024-04-25T10:19:56.926 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80816.log 2024-04-25T10:19:56.927 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56520.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56520.log.gz 2024-04-25T10:19:56.927 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94606.log 2024-04-25T10:19:56.928 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80816.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80816.log.gz 2024-04-25T10:19:56.929 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74029.log 2024-04-25T10:19:56.929 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94606.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.94606.log.gz 2024-04-25T10:19:56.930 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57954.log 2024-04-25T10:19:56.931 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74029.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74029.log.gz 2024-04-25T10:19:56.931 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90715.log 2024-04-25T10:19:56.932 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57954.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57954.log.gz 2024-04-25T10:19:56.932 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37811.log 2024-04-25T10:19:56.933 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90715.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90715.log.gz 2024-04-25T10:19:56.934 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34657.log 2024-04-25T10:19:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37811.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37811.log.gz 2024-04-25T10:19:56.935 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39134.log 2024-04-25T10:19:56.936 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34657.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34657.log.gz 2024-04-25T10:19:56.936 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53696.log 2024-04-25T10:19:56.937 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39134.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39134.log.gz 2024-04-25T10:19:56.937 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52475.log 2024-04-25T10:19:56.939 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53696.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53696.log.gz 2024-04-25T10:19:56.939 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53063.log 2024-04-25T10:19:56.940 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52475.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52475.log.gz 2024-04-25T10:19:56.940 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79797.log 2024-04-25T10:19:56.941 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53063.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53063.log.gz 2024-04-25T10:19:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58010.log 2024-04-25T10:19:56.943 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79797.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79797.log.gz 2024-04-25T10:19:56.943 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84655.log 2024-04-25T10:19:56.944 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58010.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58010.log.gz 2024-04-25T10:19:56.944 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86987.log 2024-04-25T10:19:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84655.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.84655.log.gz 2024-04-25T10:19:56.946 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88768.log 2024-04-25T10:19:56.947 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86987.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86987.log.gz 2024-04-25T10:19:56.947 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78018.log 2024-04-25T10:19:56.948 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88768.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88768.log.gz 2024-04-25T10:19:56.949 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54982.log 2024-04-25T10:19:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.78018.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78018.log.gz 2024-04-25T10:19:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57982.log 2024-04-25T10:19:56.951 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54982.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54982.log.gz 2024-04-25T10:19:56.951 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84525.log 2024-04-25T10:19:56.952 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57982.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57982.log.gz 2024-04-25T10:19:56.952 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54161.log 2024-04-25T10:19:56.953 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84525.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84525.log.gz 2024-04-25T10:19:56.954 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83883.log 2024-04-25T10:19:56.954 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54161.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54161.log.gz 2024-04-25T10:19:56.955 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93388.log 2024-04-25T10:19:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83883.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83883.log.gz 2024-04-25T10:19:56.956 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56442.log 2024-04-25T10:19:56.957 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93388.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.93388.log.gz 2024-04-25T10:19:56.958 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70903.log 2024-04-25T10:19:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56442.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56442.log.gz 2024-04-25T10:19:56.959 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58455.log 2024-04-25T10:19:56.960 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70903.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70903.log.gz 2024-04-25T10:19:56.961 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80079.log 2024-04-25T10:19:56.961 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58455.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58455.log.gz 2024-04-25T10:19:56.961 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93603.log 2024-04-25T10:19:56.962 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80079.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80079.log.gz 2024-04-25T10:19:56.963 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33899.log 2024-04-25T10:19:56.964 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93603.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93603.log.gz 2024-04-25T10:19:56.964 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77911.log 2024-04-25T10:19:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33899.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33899.log.gz 2024-04-25T10:19:56.965 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76765.log 2024-04-25T10:19:56.966 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77911.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77911.log.gz 2024-04-25T10:19:56.967 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81613.log 2024-04-25T10:19:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76765.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.76765.log.gz 2024-04-25T10:19:56.968 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38025.log 2024-04-25T10:19:56.981 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.81613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84098.log 2024-04-25T10:19:56.982 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38025.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38025.log.gz 2024-04-25T10:19:57.001 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53168.log 2024-04-25T10:19:57.002 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84098.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.84098.log.gz 2024-04-25T10:19:57.009 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81352.log 2024-04-25T10:19:57.009 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53168.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53168.log.gz 2024-04-25T10:19:57.017 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90187.log 2024-04-25T10:19:57.017 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81352.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81352.log.gz 2024-04-25T10:19:57.034 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91618.log 2024-04-25T10:19:57.034 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90187.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90187.log.gz 2024-04-25T10:19:57.041 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34915.log 2024-04-25T10:19:57.041 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91618.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91618.log.gz 2024-04-25T10:19:57.049 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77325.log 2024-04-25T10:19:57.049 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34915.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34915.log.gz 2024-04-25T10:19:57.061 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55881.log 2024-04-25T10:19:57.061 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77325.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77325.log.gz 2024-04-25T10:19:57.069 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37151.log 2024-04-25T10:19:57.070 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55881.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55881.log.gz 2024-04-25T10:19:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88702.log 2024-04-25T10:19:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37151.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37151.log.gz 2024-04-25T10:19:57.089 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90031.log 2024-04-25T10:19:57.090 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88702.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88702.log.gz 2024-04-25T10:19:57.101 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52788.log 2024-04-25T10:19:57.102 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90031.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.90031.log.gz 2024-04-25T10:19:57.113 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34704.log 2024-04-25T10:19:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52788.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52788.log.gz 2024-04-25T10:19:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76351.log 2024-04-25T10:19:57.126 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34704.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34704.log.gz 2024-04-25T10:19:57.137 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38077.log 2024-04-25T10:19:57.138 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76351.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76351.log.gz 2024-04-25T10:19:57.149 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52931.log 2024-04-25T10:19:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38077.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38077.log.gz 2024-04-25T10:19:57.161 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77377.log 2024-04-25T10:19:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52931.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52931.log.gz 2024-04-25T10:19:57.181 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71581.log 2024-04-25T10:19:57.182 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77377.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77377.log.gz 2024-04-25T10:19:57.193 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53586.log 2024-04-25T10:19:57.194 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71581.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71581.log.gz 2024-04-25T10:19:57.205 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86954.log 2024-04-25T10:19:57.206 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53586.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53586.log.gz 2024-04-25T10:19:57.221 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58062.log 2024-04-25T10:19:57.222 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86954.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86954.log.gz 2024-04-25T10:19:57.237 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93759.log 2024-04-25T10:19:57.238 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58062.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58062.log.gz 2024-04-25T10:19:57.253 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81222.log 2024-04-25T10:19:57.254 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93759.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93759.log.gz 2024-04-25T10:19:57.277 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58346.log 2024-04-25T10:19:57.278 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81222.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81222.log.gz 2024-04-25T10:19:57.293 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92412.log 2024-04-25T10:19:57.294 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58346.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58346.log.gz 2024-04-25T10:19:57.305 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55792.log 2024-04-25T10:19:57.306 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92412.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.92412.log.gz 2024-04-25T10:19:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92551.log 2024-04-25T10:19:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55792.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55792.log.gz 2024-04-25T10:19:57.337 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54035.log 2024-04-25T10:19:57.338 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92551.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92551.log.gz 2024-04-25T10:19:57.349 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92850.log 2024-04-25T10:19:57.350 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54035.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54035.log.gz 2024-04-25T10:19:57.373 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87427.log 2024-04-25T10:19:57.374 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92850.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92850.log.gz 2024-04-25T10:19:57.385 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90057.log 2024-04-25T10:19:57.386 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87427.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87427.log.gz 2024-04-25T10:19:57.397 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86601.log 2024-04-25T10:19:57.398 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90057.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90057.log.gz 2024-04-25T10:19:57.409 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86078.log 2024-04-25T10:19:57.410 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86601.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86601.log.gz 2024-04-25T10:19:57.425 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54460.log 2024-04-25T10:19:57.426 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86078.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.86078.log.gz 2024-04-25T10:19:57.437 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87137.log 2024-04-25T10:19:57.438 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54460.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54460.log.gz 2024-04-25T10:19:57.449 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71111.log 2024-04-25T10:19:57.450 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87137.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87137.log.gz 2024-04-25T10:19:57.461 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39433.log 2024-04-25T10:19:57.462 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71111.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71111.log.gz 2024-04-25T10:19:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52736.log 2024-04-25T10:19:57.478 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39433.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39433.log.gz 2024-04-25T10:19:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73482.log 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52736.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52736.log.gz 2024-04-25T10:19:57.509 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93707.log 2024-04-25T10:19:57.510 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73482.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.73482.log.gz 2024-04-25T10:19:57.521 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77794.log 2024-04-25T10:19:57.522 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93707.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.93707.log.gz 2024-04-25T10:19:57.533 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36493.log 2024-04-25T10:19:57.534 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77794.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77794.log.gz 2024-04-25T10:19:57.549 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37014.log 2024-04-25T10:19:57.550 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36493.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36493.log.gz 2024-04-25T10:19:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83381.log 2024-04-25T10:19:57.570 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37014.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37014.log.gz 2024-04-25T10:19:57.577 INFO:teuthology.orchestra.run.smithi162.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:19:57.585 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54840.log 2024-04-25T10:19:57.586 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83381.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83381.log.gz 2024-04-25T10:19:57.605 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86002.log 2024-04-25T10:19:57.606 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54840.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54840.log.gz 2024-04-25T10:19:57.621 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89847.log 2024-04-25T10:19:57.622 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86002.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.86002.log.gz 2024-04-25T10:19:57.637 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87019.log 2024-04-25T10:19:57.638 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89847.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89847.log.gz 2024-04-25T10:19:57.653 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38051.log 2024-04-25T10:19:57.654 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87019.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87019.log.gz 2024-04-25T10:19:57.665 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22791.log 2024-04-25T10:19:57.666 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38051.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38051.log.gz 2024-04-25T10:19:57.677 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53220.log 2024-04-25T10:19:57.677 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.tmp-client.admin.22791.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22791.log.gz 2024-04-25T10:19:57.689 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71779.log 2024-04-25T10:19:57.690 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53220.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53220.log.gz 2024-04-25T10:19:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52684.log 2024-04-25T10:19:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71779.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71779.log.gz 2024-04-25T10:19:57.717 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34022.log 2024-04-25T10:19:57.718 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52684.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52684.log.gz 2024-04-25T10:19:57.733 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91278.log 2024-04-25T10:19:57.735 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34022.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.34022.log.gz 2024-04-25T10:19:57.749 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74736.log 2024-04-25T10:19:57.750 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91278.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.91278.log.gz 2024-04-25T10:19:57.769 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38389.log 2024-04-25T10:19:57.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74736.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74736.log.gz 2024-04-25T10:19:57.785 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80738.log 2024-04-25T10:19:57.786 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38389.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38389.log.gz 2024-04-25T10:19:57.801 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81079.log 2024-04-25T10:19:57.802 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80738.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80738.log.gz 2024-04-25T10:19:57.813 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90109.log 2024-04-25T10:19:57.814 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81079.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81079.log.gz 2024-04-25T10:19:57.829 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35283.log 2024-04-25T10:19:57.830 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90109.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90109.log.gz 2024-04-25T10:19:57.845 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38948.log 2024-04-25T10:19:57.846 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35283.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35283.log.gz 2024-04-25T10:19:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35881.log 2024-04-25T10:19:57.862 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38948.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38948.log.gz 2024-04-25T10:19:57.872 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93733.log 2024-04-25T10:19:57.873 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35881.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35881.log.gz 2024-04-25T10:19:57.885 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53813.log 2024-04-25T10:19:57.886 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93733.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93733.log.gz 2024-04-25T10:19:57.901 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:19:57.902 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53813.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53813.log.gz 2024-04-25T10:19:57.914 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83176.log 2024-04-25T10:19:57.929 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.audit.log: 89.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91140.log 2024-04-25T10:19:57.929 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83176.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83176.log.gz 2024-04-25T10:19:57.933 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:19:57.937 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:19:57.938 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91140.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91140.log.gz 2024-04-25T10:19:57.949 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36754.log 2024-04-25T10:19:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93334.log 2024-04-25T10:19:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36754.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36754.log.gz 2024-04-25T10:19:57.969 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77031.log 2024-04-25T10:19:57.970 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93334.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.93334.log.gz 2024-04-25T10:19:57.977 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91670.log 2024-04-25T10:19:57.977 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77031.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77031.log.gz 2024-04-25T10:19:57.985 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84276.log 2024-04-25T10:19:57.985 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91670.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91670.log.gz 2024-04-25T10:19:57.997 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35491.log 2024-04-25T10:19:57.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84276.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84276.log.gz 2024-04-25T10:19:58.005 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54876.log 2024-04-25T10:19:58.005 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35491.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35491.log.gz 2024-04-25T10:19:58.013 INFO:teuthology.orchestra.run.smithi133.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.0.81613.log.gz 2024-04-25T10:19:58.013 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77624.log 2024-04-25T10:19:58.013 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57170.log 2024-04-25T10:19:58.014 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54876.log.gz 2024-04-25T10:19:58.015 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77624.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77624.log.gz 2024-04-25T10:19:58.015 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77005.log 2024-04-25T10:19:58.015 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57170.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57170.log.gz 2024-04-25T10:19:58.016 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:19:58.017 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77005.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77005.log.gz 2024-04-25T10:19:58.017 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52397.log 2024-04-25T10:19:58.033 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74919.log 2024-04-25T10:19:58.034 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52397.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52397.log.gz 2024-04-25T10:19:58.049 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89625.log 2024-04-25T10:19:58.081 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.74919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81508.log 2024-04-25T10:19:58.082 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89625.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89625.log.gz 2024-04-25T10:19:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92037.log 2024-04-25T10:19:58.090 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81508.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.81508.log.gz 2024-04-25T10:19:58.101 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35673.log 2024-04-25T10:19:58.101 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92037.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92037.log.gz 2024-04-25T10:19:58.121 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56117.log 2024-04-25T10:19:58.121 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35673.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35673.log.gz 2024-04-25T10:19:58.129 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76895.log 2024-04-25T10:19:58.130 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56117.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56117.log.gz 2024-04-25T10:19:58.141 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88872.log 2024-04-25T10:19:58.141 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76895.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76895.log.gz 2024-04-25T10:19:58.149 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86471.log 2024-04-25T10:19:58.149 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88872.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88872.log.gz 2024-04-25T10:19:58.161 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73750.log 2024-04-25T10:19:58.161 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86471.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86471.log.gz 2024-04-25T10:19:58.169 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56013.log 2024-04-25T10:19:58.169 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73750.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73750.log.gz 2024-04-25T10:19:58.181 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33745.log 2024-04-25T10:19:58.181 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56013.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56013.log.gz 2024-04-25T10:19:58.189 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56494.log 2024-04-25T10:19:58.189 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33745.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33745.log.gz 2024-04-25T10:19:58.201 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33693.log 2024-04-25T10:19:58.201 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56494.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56494.log.gz 2024-04-25T10:19:58.209 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92310.log 2024-04-25T10:19:58.209 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33693.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33693.log.gz 2024-04-25T10:19:58.221 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55660.log 2024-04-25T10:19:58.221 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92310.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92310.log.gz 2024-04-25T10:19:58.241 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89110.log 2024-04-25T10:19:58.241 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55660.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55660.log.gz 2024-04-25T10:19:58.253 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77221.log 2024-04-25T10:19:58.253 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89110.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.89110.log.gz 2024-04-25T10:19:58.261 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55594.log 2024-04-25T10:19:58.261 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77221.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.77221.log.gz 2024-04-25T10:19:58.273 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54956.log 2024-04-25T10:19:58.273 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55594.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55594.log.gz 2024-04-25T10:19:58.281 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58114.log 2024-04-25T10:19:58.281 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54956.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54956.log.gz 2024-04-25T10:19:58.293 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84551.log 2024-04-25T10:19:58.293 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58114.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58114.log.gz 2024-04-25T10:19:58.313 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71963.log 2024-04-25T10:19:58.314 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84551.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84551.log.gz 2024-04-25T10:19:58.321 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73372.log 2024-04-25T10:19:58.321 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71963.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71963.log.gz 2024-04-25T10:19:58.329 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89821.log 2024-04-25T10:19:58.330 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73372.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.73372.log.gz 2024-04-25T10:19:58.337 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54790.log 2024-04-25T10:19:58.337 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89821.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89821.log.gz 2024-04-25T10:19:58.345 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37759.log 2024-04-25T10:19:58.346 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54790.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54790.log.gz 2024-04-25T10:19:58.353 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34366.log 2024-04-25T10:19:58.353 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37759.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37759.log.gz 2024-04-25T10:19:58.361 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36467.log 2024-04-25T10:19:58.361 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34366.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34366.log.gz 2024-04-25T10:19:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91166.log 2024-04-25T10:19:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36467.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36467.log.gz 2024-04-25T10:19:58.377 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74396.log 2024-04-25T10:19:58.378 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91166.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91166.log.gz 2024-04-25T10:19:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87581.log 2024-04-25T10:19:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74396.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74396.log.gz 2024-04-25T10:19:58.401 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87871.log 2024-04-25T10:19:58.402 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87581.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.87581.log.gz 2024-04-25T10:19:58.417 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57615.log 2024-04-25T10:19:58.418 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87871.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.87871.log.gz 2024-04-25T10:19:58.433 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55120.log 2024-04-25T10:19:58.434 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57615.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57615.log.gz 2024-04-25T10:19:58.457 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71397.log 2024-04-25T10:19:58.458 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55120.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55120.log.gz 2024-04-25T10:19:58.473 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93908.log 2024-04-25T10:19:58.474 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71397.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71397.log.gz 2024-04-25T10:19:58.489 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51978.log 2024-04-25T10:19:58.490 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93908.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93908.log.gz 2024-04-25T10:19:58.505 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74003.log 2024-04-25T10:19:58.506 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51978.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51978.log.gz 2024-04-25T10:19:58.517 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70555.log 2024-04-25T10:19:58.518 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74003.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74003.log.gz 2024-04-25T10:19:58.533 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79542.log 2024-04-25T10:19:58.534 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70555.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70555.log.gz 2024-04-25T10:19:58.541 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52710.log 2024-04-25T10:19:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79542.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.79542.log.gz 2024-04-25T10:19:58.549 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71189.log 2024-04-25T10:19:58.549 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52710.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52710.log.gz 2024-04-25T10:19:58.557 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88403.log 2024-04-25T10:19:58.557 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71189.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71189.log.gz 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55399.log 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88403.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88403.log.gz 2024-04-25T10:19:58.577 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52905.log 2024-04-25T10:19:58.577 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55399.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55399.log.gz 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89925.log 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52905.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52905.log.gz 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37999.log 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89925.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89925.log.gz 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37204.log 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37999.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37999.log.gz 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34157.log 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37204.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37204.log.gz 2024-04-25T10:19:58.645 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71319.log 2024-04-25T10:19:58.645 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34157.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34157.log.gz 2024-04-25T10:19:58.657 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34914.log 2024-04-25T10:19:58.657 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71319.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71319.log.gz 2024-04-25T10:19:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73880.log 2024-04-25T10:19:58.665 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34914.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34914.log.gz 2024-04-25T10:19:58.673 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80947.log 2024-04-25T10:19:58.674 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73880.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73880.log.gz 2024-04-25T10:19:58.681 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88081.log 2024-04-25T10:19:58.681 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80947.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80947.log.gz 2024-04-25T10:19:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38181.log 2024-04-25T10:19:58.689 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88081.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88081.log.gz 2024-04-25T10:19:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56766.log 2024-04-25T10:19:58.697 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38181.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38181.log.gz 2024-04-25T10:19:58.705 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53748.log 2024-04-25T10:19:58.705 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56766.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56766.log.gz 2024-04-25T10:19:58.713 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79619.log 2024-04-25T10:19:58.714 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53748.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53748.log.gz 2024-04-25T10:19:58.721 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90324.log 2024-04-25T10:19:58.721 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79619.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79619.log.gz 2024-04-25T10:19:58.729 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34708.log 2024-04-25T10:19:58.729 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90324.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.90324.log.gz 2024-04-25T10:19:58.737 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89032.log 2024-04-25T10:19:58.738 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34708.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34708.log.gz 2024-04-25T10:19:58.745 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91356.log 2024-04-25T10:19:58.745 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89032.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.89032.log.gz 2024-04-25T10:19:58.753 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89709.log 2024-04-25T10:19:58.753 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91356.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91356.log.gz 2024-04-25T10:19:58.761 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90161.log 2024-04-25T10:19:58.761 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89709.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89709.log.gz 2024-04-25T10:19:58.769 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36676.log 2024-04-25T10:19:58.769 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90161.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90161.log.gz 2024-04-25T10:19:58.777 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94294.log 2024-04-25T10:19:58.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36676.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36676.log.gz 2024-04-25T10:19:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58088.log 2024-04-25T10:19:58.793 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94294.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94294.log.gz 2024-04-25T10:19:58.801 INFO:teuthology.orchestra.run.smithi133.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.53351.log.gz 2024-04-25T10:19:58.809 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34605.log 2024-04-25T10:19:58.810 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58088.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58088.log.gz 2024-04-25T10:19:58.829 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39106.log 2024-04-25T10:19:58.829 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34605.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34605.log.gz 2024-04-25T10:19:58.845 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55477.log 2024-04-25T10:19:58.846 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39106.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39106.log.gz 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94138.log 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55477.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55477.log.gz 2024-04-25T10:19:58.873 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52194.log 2024-04-25T10:19:58.874 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94138.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94138.log.gz 2024-04-25T10:19:58.893 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83414.log 2024-04-25T10:19:58.894 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52194.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52194.log.gz 2024-04-25T10:19:58.909 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76097.log 2024-04-25T10:19:58.909 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83414.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83414.log.gz 2024-04-25T10:19:58.921 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52586.log 2024-04-25T10:19:58.922 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76097.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.76097.log.gz 2024-04-25T10:19:58.937 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35829.log 2024-04-25T10:19:58.938 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52586.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52586.log.gz 2024-04-25T10:19:58.953 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84760.log 2024-04-25T10:19:58.954 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35829.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35829.log.gz 2024-04-25T10:19:58.969 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74864.log 2024-04-25T10:19:58.985 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.84760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77592.log 2024-04-25T10:19:58.986 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74864.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74864.log.gz 2024-04-25T10:19:58.993 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35777.log 2024-04-25T10:19:58.994 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77592.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77592.log.gz 2024-04-25T10:19:59.005 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81482.log 2024-04-25T10:19:59.006 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35777.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35777.log.gz 2024-04-25T10:19:59.017 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54195.log 2024-04-25T10:19:59.018 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81482.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81482.log.gz 2024-04-25T10:19:59.033 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76843.log 2024-04-25T10:19:59.034 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54195.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54195.log.gz 2024-04-25T10:19:59.049 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76377.log 2024-04-25T10:19:59.050 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76843.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76843.log.gz 2024-04-25T10:19:59.065 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70605.log 2024-04-25T10:19:59.066 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76377.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76377.log.gz 2024-04-25T10:19:59.081 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89951.log 2024-04-25T10:19:59.082 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.70605.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.70605.log.gz 2024-04-25T10:19:59.101 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74632.log 2024-04-25T10:19:59.102 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89951.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89951.log.gz 2024-04-25T10:19:59.113 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80881.log 2024-04-25T10:19:59.114 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74632.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74632.log.gz 2024-04-25T10:19:59.129 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79697.log 2024-04-25T10:19:59.130 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80881.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80881.log.gz 2024-04-25T10:19:59.145 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76953.log 2024-04-25T10:19:59.146 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79697.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.79697.log.gz 2024-04-25T10:19:59.157 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77091.log 2024-04-25T10:19:59.158 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76953.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76953.log.gz 2024-04-25T10:19:59.173 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77247.log 2024-04-25T10:19:59.174 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77091.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77091.log.gz 2024-04-25T10:19:59.185 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81378.log 2024-04-25T10:19:59.186 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77247.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77247.log.gz 2024-04-25T10:19:59.205 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58166.log 2024-04-25T10:19:59.206 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81378.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81378.log.gz 2024-04-25T10:19:59.221 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34288.log 2024-04-25T10:19:59.222 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58166.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58166.log.gz 2024-04-25T10:19:59.237 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38285.log 2024-04-25T10:19:59.238 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34288.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34288.log.gz 2024-04-25T10:19:59.254 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89795.log 2024-04-25T10:19:59.255 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38285.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38285.log.gz 2024-04-25T10:19:59.269 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92913.log 2024-04-25T10:19:59.270 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89795.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89795.log.gz 2024-04-25T10:19:59.285 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76979.log 2024-04-25T10:19:59.286 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92913.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.92913.log.gz 2024-04-25T10:19:59.301 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83578.log 2024-04-25T10:19:59.302 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76979.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76979.log.gz 2024-04-25T10:19:59.313 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91486.log 2024-04-25T10:19:59.313 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83578.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83578.log.gz 2024-04-25T10:19:59.321 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74055.log 2024-04-25T10:19:59.321 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91486.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91486.log.gz 2024-04-25T10:19:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76607.log 2024-04-25T10:19:59.337 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74055.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74055.log.gz 2024-04-25T10:19:59.345 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38363.log 2024-04-25T10:19:59.345 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76607.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76607.log.gz 2024-04-25T10:19:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52004.log 2024-04-25T10:19:59.353 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38363.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38363.log.gz 2024-04-25T10:19:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38441.log 2024-04-25T10:19:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52004.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52004.log.gz 2024-04-25T10:19:59.369 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76635.log 2024-04-25T10:19:59.369 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38441.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38441.log.gz 2024-04-25T10:19:59.377 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87217.log 2024-04-25T10:19:59.377 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76635.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76635.log.gz 2024-04-25T10:19:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88351.log 2024-04-25T10:19:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.87217.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87217.log.gz 2024-04-25T10:19:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88299.log 2024-04-25T10:19:59.393 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88351.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88351.log.gz 2024-04-25T10:19:59.401 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35595.log 2024-04-25T10:19:59.401 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88299.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88299.log.gz 2024-04-25T10:19:59.409 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80381.log 2024-04-25T10:19:59.409 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35595.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35595.log.gz 2024-04-25T10:19:59.417 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86708.log 2024-04-25T10:19:59.417 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80381.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80381.log.gz 2024-04-25T10:19:59.423 INFO:teuthology.orchestra.run.smithi133.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.74919.log.gz 2024-04-25T10:19:59.424 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88820.log 2024-04-25T10:19:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86708.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86708.log.gz 2024-04-25T10:19:59.425 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57204.log 2024-04-25T10:19:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88820.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88820.log.gz 2024-04-25T10:19:59.426 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84393.log 2024-04-25T10:19:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57204.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57204.log.gz 2024-04-25T10:19:59.427 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79645.log 2024-04-25T10:19:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.84393.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84393.log.gz 2024-04-25T10:19:59.428 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39195.log 2024-04-25T10:19:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79645.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.79645.log.gz 2024-04-25T10:19:59.429 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37707.log 2024-04-25T10:19:59.430 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39195.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39195.log.gz 2024-04-25T10:19:59.431 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90135.log 2024-04-25T10:19:59.431 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37707.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37707.log.gz 2024-04-25T10:19:59.432 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80303.log 2024-04-25T10:19:59.432 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90135.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90135.log.gz 2024-04-25T10:19:59.433 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56617.log 2024-04-25T10:19:59.433 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80303.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80303.log.gz 2024-04-25T10:19:59.434 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58270.log 2024-04-25T10:19:59.435 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56617.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56617.log.gz 2024-04-25T10:19:59.435 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90291.log 2024-04-25T10:19:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58270.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58270.log.gz 2024-04-25T10:19:59.436 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76429.log 2024-04-25T10:19:59.437 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90291.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90291.log.gz 2024-04-25T10:19:59.437 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33639.log 2024-04-25T10:19:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76429.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76429.log.gz 2024-04-25T10:19:59.438 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88143.log 2024-04-25T10:19:59.439 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33639.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33639.log.gz 2024-04-25T10:19:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56285.log 2024-04-25T10:19:59.440 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88143.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88143.log.gz 2024-04-25T10:19:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91434.log 2024-04-25T10:19:59.441 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56285.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56285.log.gz 2024-04-25T10:19:59.442 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33546.log 2024-04-25T10:19:59.442 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91434.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91434.log.gz 2024-04-25T10:19:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54427.log 2024-04-25T10:19:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33546.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33546.log.gz 2024-04-25T10:19:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80459.log 2024-04-25T10:19:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54427.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54427.log.gz 2024-04-25T10:19:59.445 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36519.log 2024-04-25T10:19:59.445 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80459.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80459.log.gz 2024-04-25T10:19:59.446 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80433.log 2024-04-25T10:19:59.447 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36519.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.36519.log.gz 2024-04-25T10:19:59.447 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37838.log 2024-04-25T10:19:59.448 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80433.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80433.log.gz 2024-04-25T10:19:59.448 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72094.log 2024-04-25T10:19:59.449 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37838.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37838.log.gz 2024-04-25T10:19:59.449 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80027.log 2024-04-25T10:19:59.469 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.72094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27715.log 2024-04-25T10:19:59.470 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80027.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80027.log.gz 2024-04-25T10:19:59.485 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36832.log 2024-04-25T10:19:59.486 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.27715.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27715.log.gz 2024-04-25T10:19:59.486 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33518.log 2024-04-25T10:19:59.487 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36832.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36832.log.gz 2024-04-25T10:19:59.501 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77937.log 2024-04-25T10:19:59.502 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33518.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33518.log.gz 2024-04-25T10:19:59.521 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37310.log 2024-04-25T10:19:59.521 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77937.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77937.log.gz 2024-04-25T10:19:59.529 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88109.log 2024-04-25T10:19:59.529 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37310.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37310.log.gz 2024-04-25T10:19:59.545 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56363.log 2024-04-25T10:19:59.545 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88109.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88109.log.gz 2024-04-25T10:19:59.553 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74476.log 2024-04-25T10:19:59.553 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56363.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56363.log.gz 2024-04-25T10:19:59.561 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34105.log 2024-04-25T10:19:59.561 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74476.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.74476.log.gz 2024-04-25T10:19:59.569 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52879.log 2024-04-25T10:19:59.569 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34105.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34105.log.gz 2024-04-25T10:19:59.589 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88954.log 2024-04-25T10:19:59.589 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52879.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52879.log.gz 2024-04-25T10:19:59.609 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73560.log 2024-04-25T10:19:59.609 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88954.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88954.log.gz 2024-04-25T10:19:59.617 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34449.log 2024-04-25T10:19:59.617 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73560.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73560.log.gz 2024-04-25T10:19:59.625 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37575.log 2024-04-25T10:19:59.625 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34449.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34449.log.gz 2024-04-25T10:19:59.637 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91807.log 2024-04-25T10:19:59.637 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37575.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37575.log.gz 2024-04-25T10:19:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38129.log 2024-04-25T10:19:59.644 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91807.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91807.log.gz 2024-04-25T10:19:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91905.log 2024-04-25T10:19:59.653 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38129.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38129.log.gz 2024-04-25T10:19:59.661 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90083.log 2024-04-25T10:19:59.662 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91905.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91905.log.gz 2024-04-25T10:19:59.669 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37627.log 2024-04-25T10:19:59.670 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90083.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90083.log.gz 2024-04-25T10:19:59.689 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91538.log 2024-04-25T10:19:59.690 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37627.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37627.log.gz 2024-04-25T10:19:59.705 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93655.log 2024-04-25T10:19:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91538.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91538.log.gz 2024-04-25T10:19:59.721 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33195.log 2024-04-25T10:19:59.722 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93655.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93655.log.gz 2024-04-25T10:19:59.741 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90477.log 2024-04-25T10:19:59.742 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.33195.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33195.log.gz 2024-04-25T10:19:59.757 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36988.log 2024-04-25T10:19:59.758 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90477.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90477.log.gz 2024-04-25T10:19:59.773 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52324.log 2024-04-25T10:19:59.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36988.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36988.log.gz 2024-04-25T10:19:59.789 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36389.log 2024-04-25T10:19:59.790 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52324.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52324.log.gz 2024-04-25T10:19:59.801 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37916.log 2024-04-25T10:19:59.801 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36389.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36389.log.gz 2024-04-25T10:19:59.809 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36858.log 2024-04-25T10:19:59.809 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37916.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37916.log.gz 2024-04-25T10:19:59.817 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55820.log 2024-04-25T10:19:59.817 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36858.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36858.log.gz 2024-04-25T10:19:59.825 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76247.log 2024-04-25T10:19:59.825 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55820.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55820.log.gz 2024-04-25T10:19:59.833 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35968.log 2024-04-25T10:19:59.833 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76247.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76247.log.gz 2024-04-25T10:19:59.841 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81326.log 2024-04-25T10:19:59.841 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35968.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35968.log.gz 2024-04-25T10:19:59.857 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53426.log 2024-04-25T10:19:59.857 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81326.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81326.log.gz 2024-04-25T10:19:59.865 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53666.log 2024-04-25T10:19:59.865 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53426.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53426.log.gz 2024-04-25T10:19:59.873 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71555.log 2024-04-25T10:19:59.873 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53666.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53666.log.gz 2024-04-25T10:19:59.881 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94372.log 2024-04-25T10:19:59.881 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71555.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71555.log.gz 2024-04-25T10:19:59.889 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52553.log 2024-04-25T10:19:59.889 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94372.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94372.log.gz 2024-04-25T10:19:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85937.log 2024-04-25T10:19:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52553.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52553.log.gz 2024-04-25T10:19:59.913 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91696.log 2024-04-25T10:19:59.913 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.85937.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.85937.log.gz 2024-04-25T10:19:59.921 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35751.log 2024-04-25T10:19:59.921 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91696.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.91696.log.gz 2024-04-25T10:19:59.929 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73646.log 2024-04-25T10:19:59.929 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35751.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35751.log.gz 2024-04-25T10:19:59.937 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38752.log 2024-04-25T10:19:59.937 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73646.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.73646.log.gz 2024-04-25T10:19:59.945 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90741.log 2024-04-25T10:19:59.945 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38752.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38752.log.gz 2024-04-25T10:19:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55738.log 2024-04-25T10:19:59.953 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90741.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90741.log.gz 2024-04-25T10:19:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36597.log 2024-04-25T10:19:59.969 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55738.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55738.log.gz 2024-04-25T10:19:59.977 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92964.log 2024-04-25T10:19:59.977 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36597.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36597.log.gz 2024-04-25T10:19:59.985 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56896.log 2024-04-25T10:19:59.985 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92964.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92964.log.gz 2024-04-25T10:19:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35517.log 2024-04-25T10:19:59.993 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56896.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56896.log.gz 2024-04-25T10:20:00.001 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86384.log 2024-04-25T10:20:00.001 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35517.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35517.log.gz 2024-04-25T10:20:00.009 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79823.log 2024-04-25T10:20:00.009 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86384.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86384.log.gz 2024-04-25T10:20:00.025 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56846.log 2024-04-25T10:20:00.026 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79823.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79823.log.gz 2024-04-25T10:20:00.045 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55094.log 2024-04-25T10:20:00.046 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56846.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.56846.log.gz 2024-04-25T10:20:00.053 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54492.log 2024-04-25T10:20:00.053 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55094.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55094.log.gz 2024-04-25T10:20:00.061 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27819.log 2024-04-25T10:20:00.062 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54492.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.54492.log.gz 2024-04-25T10:20:00.069 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34527.log 2024-04-25T10:20:00.069 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.27819.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27819.log.gz 2024-04-25T10:20:00.077 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54325.log 2024-04-25T10:20:00.077 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34527.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34527.log.gz 2024-04-25T10:20:00.085 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37284.log 2024-04-25T10:20:00.086 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54325.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54325.log.gz 2024-04-25T10:20:00.101 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38720.log 2024-04-25T10:20:00.103 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37284.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37284.log.gz 2024-04-25T10:20:00.121 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81558.log 2024-04-25T10:20:00.122 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38720.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38720.log.gz 2024-04-25T10:20:00.141 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58218.log 2024-04-25T10:20:00.142 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81558.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81558.log.gz 2024-04-25T10:20:00.165 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56792.log 2024-04-25T10:20:00.166 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.58218.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58218.log.gz 2024-04-25T10:20:00.181 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94164.log 2024-04-25T10:20:00.182 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.56792.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56792.log.gz 2024-04-25T10:20:00.197 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71059.log 2024-04-25T10:20:00.198 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94164.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94164.log.gz 2024-04-25T10:20:00.213 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74342.log 2024-04-25T10:20:00.213 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71059.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71059.log.gz 2024-04-25T10:20:00.229 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71345.log 2024-04-25T10:20:00.229 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74342.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74342.log.gz 2024-04-25T10:20:00.241 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91931.log 2024-04-25T10:20:00.242 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71345.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71345.log.gz 2024-04-25T10:20:00.257 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39407.log 2024-04-25T10:20:00.258 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91931.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91931.log.gz 2024-04-25T10:20:00.273 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37733.log 2024-04-25T10:20:00.274 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.39407.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39407.log.gz 2024-04-25T10:20:00.289 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83466.log 2024-04-25T10:20:00.290 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37733.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37733.log.gz 2024-04-25T10:20:00.305 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38311.log 2024-04-25T10:20:00.306 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83466.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83466.log.gz 2024-04-25T10:20:00.316 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73586.log 2024-04-25T10:20:00.317 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38311.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38311.log.gz 2024-04-25T10:20:00.329 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88325.log 2024-04-25T10:20:00.330 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73586.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73586.log.gz 2024-04-25T10:20:00.341 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89602.log 2024-04-25T10:20:00.342 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88325.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88325.log.gz 2024-04-25T10:20:00.361 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57490.log 2024-04-25T10:20:00.361 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89602.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89602.log.gz 2024-04-25T10:20:00.389 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86679.log 2024-04-25T10:20:00.390 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57490.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57490.log.gz 2024-04-25T10:20:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89761.log 2024-04-25T10:20:00.405 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86679.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86679.log.gz 2024-04-25T10:20:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34183.log 2024-04-25T10:20:00.422 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89761.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89761.log.gz 2024-04-25T10:20:00.437 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88221.log 2024-04-25T10:20:00.437 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34183.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34183.log.gz 2024-04-25T10:20:00.449 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34209.log 2024-04-25T10:20:00.449 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88221.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88221.log.gz 2024-04-25T10:20:00.469 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94006.log 2024-04-25T10:20:00.470 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34209.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34209.log.gz 2024-04-25T10:20:00.477 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74289.log 2024-04-25T10:20:00.477 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.94006.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94006.log.gz 2024-04-25T10:20:00.481 INFO:teuthology.orchestra.run.smithi133.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:20:00.497 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77533.log 2024-04-25T10:20:00.498 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74289.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74289.log.gz 2024-04-25T10:20:00.499 INFO:teuthology.orchestra.run.smithi133.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.84760.log.gz 2024-04-25T10:20:00.499 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86306.log 2024-04-25T10:20:00.500 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.77533.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77533.log.gz 2024-04-25T10:20:00.500 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35803.log 2024-04-25T10:20:00.501 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.86306.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86306.log.gz 2024-04-25T10:20:00.501 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76147.log 2024-04-25T10:20:00.502 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35803.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35803.log.gz 2024-04-25T10:20:00.502 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82943.log 2024-04-25T10:20:00.503 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.76147.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.76147.log.gz 2024-04-25T10:20:00.503 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34710.log 2024-04-25T10:20:00.504 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.82943.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.82943.log.gz 2024-04-25T10:20:00.505 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38259.log 2024-04-25T10:20:00.505 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34710.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34710.log.gz 2024-04-25T10:20:00.506 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55935.log 2024-04-25T10:20:00.506 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.38259.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38259.log.gz 2024-04-25T10:20:00.507 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35287.log 2024-04-25T10:20:00.507 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55935.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55935.log.gz 2024-04-25T10:20:00.508 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71911.log 2024-04-25T10:20:00.508 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35287.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35287.log.gz 2024-04-25T10:20:00.509 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52816.log 2024-04-25T10:20:00.509 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71911.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71911.log.gz 2024-04-25T10:20:00.509 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88794.log 2024-04-25T10:20:00.510 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52816.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52816.log.gz 2024-04-25T10:20:00.511 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90265.log 2024-04-25T10:20:00.511 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88794.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.88794.log.gz 2024-04-25T10:20:00.511 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79949.log 2024-04-25T10:20:00.512 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90265.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90265.log.gz 2024-04-25T10:20:00.513 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73951.log 2024-04-25T10:20:00.513 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.79949.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79949.log.gz 2024-04-25T10:20:00.513 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74081.log 2024-04-25T10:20:00.514 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73951.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73951.log.gz 2024-04-25T10:20:00.514 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57092.log 2024-04-25T10:20:00.515 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74081.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74081.log.gz 2024-04-25T10:20:00.515 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57412.log 2024-04-25T10:20:00.516 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57092.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57092.log.gz 2024-04-25T10:20:00.516 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57667.log 2024-04-25T10:20:00.517 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57412.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57412.log.gz 2024-04-25T10:20:00.517 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93525.log 2024-04-25T10:20:00.518 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57667.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57667.log.gz 2024-04-25T10:20:00.518 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57875.log 2024-04-25T10:20:00.519 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93525.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93525.log.gz 2024-04-25T10:20:00.519 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92760.log 2024-04-25T10:20:00.520 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57875.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57875.log.gz 2024-04-25T10:20:00.520 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73802.log 2024-04-25T10:20:00.521 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92760.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92760.log.gz 2024-04-25T10:20:00.521 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92682.log 2024-04-25T10:20:00.522 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73802.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73802.log.gz 2024-04-25T10:20:00.522 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57784.log 2024-04-25T10:20:00.523 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.92682.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92682.log.gz 2024-04-25T10:20:00.523 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73776.log 2024-04-25T10:20:00.524 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57784.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57784.log.gz 2024-04-25T10:20:00.525 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83440.log 2024-04-25T10:20:00.525 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73776.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.73776.log.gz 2024-04-25T10:20:00.525 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73724.log 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83440.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83440.log.gz 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53011.log 2024-04-25T10:20:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73724.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73724.log.gz 2024-04-25T10:20:00.527 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88846.log 2024-04-25T10:20:00.528 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53011.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53011.log.gz 2024-04-25T10:20:00.529 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54714.log 2024-04-25T10:20:00.529 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88846.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88846.log.gz 2024-04-25T10:20:00.529 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88481.log 2024-04-25T10:20:00.530 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54714.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54714.log.gz 2024-04-25T10:20:00.531 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93681.log 2024-04-25T10:20:00.531 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88481.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88481.log.gz 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89899.log 2024-04-25T10:20:00.533 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93681.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93681.log.gz 2024-04-25T10:20:00.533 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81456.log 2024-04-25T10:20:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89899.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89899.log.gz 2024-04-25T10:20:00.534 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55686.log 2024-04-25T10:20:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.81456.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81456.log.gz 2024-04-25T10:20:00.535 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89136.log 2024-04-25T10:20:00.536 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55686.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55686.log.gz 2024-04-25T10:20:00.536 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91644.log 2024-04-25T10:20:00.537 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89136.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89136.log.gz 2024-04-25T10:20:00.537 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83838.log 2024-04-25T10:20:00.538 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91644.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.91644.log.gz 2024-04-25T10:20:00.539 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90529.log 2024-04-25T10:20:00.539 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.83838.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83838.log.gz 2024-04-25T10:20:00.540 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88637.log 2024-04-25T10:20:00.540 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.90529.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90529.log.gz 2024-04-25T10:20:00.541 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37549.log 2024-04-25T10:20:00.542 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.88637.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.88637.log.gz 2024-04-25T10:20:00.542 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34501.log 2024-04-25T10:20:00.543 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37549.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37549.log.gz 2024-04-25T10:20:00.543 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89497.log 2024-04-25T10:20:00.543 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.34501.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34501.log.gz 2024-04-25T10:20:00.544 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36910.log 2024-04-25T10:20:00.545 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.89497.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.89497.log.gz 2024-04-25T10:20:00.545 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57719.log 2024-04-25T10:20:00.546 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36910.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36910.log.gz 2024-04-25T10:20:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93882.log 2024-04-25T10:20:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.57719.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57719.log.gz 2024-04-25T10:20:00.548 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71662.log 2024-04-25T10:20:00.548 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.93882.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93882.log.gz 2024-04-25T10:20:00.549 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71859.log 2024-04-25T10:20:00.549 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71662.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71662.log.gz 2024-04-25T10:20:00.550 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55068.log 2024-04-25T10:20:00.550 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.71859.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71859.log.gz 2024-04-25T10:20:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74315.log 2024-04-25T10:20:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55068.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55068.log.gz 2024-04-25T10:20:00.552 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91218.log 2024-04-25T10:20:00.552 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.74315.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74315.log.gz 2024-04-25T10:20:00.553 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80131.log 2024-04-25T10:20:00.553 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.91218.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91218.log.gz 2024-04-25T10:20:00.553 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73672.log 2024-04-25T10:20:00.554 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.80131.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80131.log.gz 2024-04-25T10:20:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.73672.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73672.log.gz 2024-04-25T10:20:00.793 INFO:teuthology.orchestra.run.smithi162.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:20:00.824 INFO:teuthology.orchestra.run.smithi133.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.72094.log.gz 2024-04-25T10:20:00.947 INFO:teuthology.orchestra.run.smithi162.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:20:02.081 INFO:teuthology.orchestra.run.smithi162.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:20:02.275 INFO:teuthology.orchestra.run.smithi162.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:20:02.789 INFO:teuthology.orchestra.run.smithi133.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.0.58401.log.gz 2024-04-25T10:20:03.229 INFO:teuthology.orchestra.run.smithi133.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:20:03.332 INFO:teuthology.orchestra.run.smithi133.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.0.39538.log.gz 2024-04-25T10:20:03.373 INFO:teuthology.orchestra.run.smithi133.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:20:03.485 INFO:teuthology.orchestra.run.smithi133.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:20:04.670 INFO:teuthology.orchestra.run.smithi133.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:20:15.560 INFO:teuthology.orchestra.run.smithi162.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:20:15.562 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-04-25T10:20:15.562 INFO:teuthology.orchestra.run.smithi162.stderr:real 0m20.341s 2024-04-25T10:20:15.562 INFO:teuthology.orchestra.run.smithi162.stderr:user 0m47.465s 2024-04-25T10:20:15.563 INFO:teuthology.orchestra.run.smithi162.stderr:sys 0m1.450s 2024-04-25T10:20:24.654 INFO:teuthology.orchestra.run.smithi133.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:20:24.656 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T10:20:24.656 INFO:teuthology.orchestra.run.smithi133.stderr:real 0m29.455s 2024-04-25T10:20:24.656 INFO:teuthology.orchestra.run.smithi133.stderr:user 1m21.463s 2024-04-25T10:20:24.656 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m2.736s 2024-04-25T10:20:24.656 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:20:24.656 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035/remote/smithi133/log 2024-04-25T10:20:24.657 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:20:33.898 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035/remote/smithi162/log 2024-04-25T10:20:33.899 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:20:39.495 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:20:39.505 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:20:39.505 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:20:39.508 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:20:39.585 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:20:39.594 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:20:39.594 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:20:39.596 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:20:39.607 INFO:teuthology.orchestra.run.smithi133.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:20:39.607 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================== 2024-04-25T10:20:39.608 INFO:teuthology.orchestra.run.smithi133.stdout:+hv01.front.sepi 67.205.162.81 3 u 32 64 377 0.080 -5.399 0.775 2024-04-25T10:20:39.608 INFO:teuthology.orchestra.run.smithi133.stdout:+hv02.front.sepi 63.231.80.2 3 u 40 64 377 0.080 -3.325 0.642 2024-04-25T10:20:39.608 INFO:teuthology.orchestra.run.smithi133.stdout:*hv03.front.sepi 74.6.168.72 3 u 41 64 377 0.068 -3.741 0.758 2024-04-25T10:20:39.608 INFO:teuthology.orchestra.run.smithi133.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T10:20:39.609 INFO:teuthology.orchestra.run.smithi162.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:20:39.609 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================== 2024-04-25T10:20:39.609 INFO:teuthology.orchestra.run.smithi162.stdout:*hv01.front.sepi 67.205.162.81 3 u 33 64 377 0.096 -5.913 0.846 2024-04-25T10:20:39.609 INFO:teuthology.orchestra.run.smithi162.stdout:+hv02.front.sepi 63.231.80.2 3 u 24 64 377 0.088 -5.008 0.856 2024-04-25T10:20:39.610 INFO:teuthology.orchestra.run.smithi162.stdout:+hv03.front.sepi 74.6.168.72 3 u 42 64 377 0.095 -5.166 0.507 2024-04-25T10:20:39.610 INFO:teuthology.orchestra.run.smithi162.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T10:20:39.611 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:20:39.619 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:20:39.619 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:20:39.628 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:20:39.636 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:20:39.648 INFO:teuthology.task.internal:Duration was 1781.212032 seconds 2024-04-25T10:20:39.648 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:20:39.656 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:20:39.656 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:20:39.659 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:20:39.694 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:20:39.694 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T10:20:39.694 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:20:39.746 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2024-04-25T10:20:39.747 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:20:39.757 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:20:39.757 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:20:39.790 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:20:39.827 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:20:39.828 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:20:39.866 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:20:39.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:20:39.963 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:20:39.963 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:20:39.975 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:20:39.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:20:39.997 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:20:40.018 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:20:40.026 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-04-25T10:20:40.040 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2024-04-25T10:20:40.053 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:20:40.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:20:40.084 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:20:40.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:20:40.097 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:20:40.108 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:20:40.108 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035/remote/smithi133 2024-04-25T10:20:40.109 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:20:40.151 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673035/remote/smithi162 2024-04-25T10:20:40.152 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:20:40.174 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:20:40.174 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:20:40.190 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:20:40.226 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:20:40.238 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:20:40.238 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:20:40.248 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:20:40.249 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:20:40.251 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:20:40.254 INFO:teuthology.orchestra.run.smithi133.stdout: 658638 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:20 /home/ubuntu/cephtest 2024-04-25T10:20:40.274 INFO:teuthology.orchestra.run.smithi162.stdout: 658639 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:20 /home/ubuntu/cephtest 2024-04-25T10:20:40.276 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:20:40.287 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:20:40.309 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} duration: 1781.2120323181152 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:20:40.309 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:20:40.393 INFO:teuthology.run:pass